@jaypie/mcp 0.8.165 → 0.8.167
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/assets.js +3 -3
- package/package.json +1 -1
- package/release-notes/llm/1.3.47.md +20 -0
- package/release-notes/llm/1.3.48.md +21 -0
- package/skills/llm.md +129 -104
package/dist/assets.js
CHANGED
|
@@ -55,11 +55,11 @@ function getMcpAssetPaths() {
|
|
|
55
55
|
*/
|
|
56
56
|
function getMcpBuildInfo() {
|
|
57
57
|
return {
|
|
58
|
-
commit: "
|
|
58
|
+
commit: "8df47237"
|
|
59
59
|
,
|
|
60
|
-
version: "0.8.
|
|
60
|
+
version: "0.8.167"
|
|
61
61
|
,
|
|
62
|
-
versionString: "@jaypie/mcp@0.8.
|
|
62
|
+
versionString: "@jaypie/mcp@0.8.167#8df47237"
|
|
63
63
|
,
|
|
64
64
|
};
|
|
65
65
|
}
|
package/package.json
CHANGED
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
---
|
|
2
|
+
version: 1.3.47
|
|
3
|
+
date: 2026-09-22
|
|
4
|
+
summary: MODEL.OPUS names claude-opus-5-5; MODEL.SOL and MODEL.LUNA name the GPT-6 tier
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## Changes
|
|
8
|
+
|
|
9
|
+
- `MODEL.OPUS` names `claude-opus-5-5`, superseding `claude-opus-5`. The tier alias moves rather than gaining a new key, so every `MODEL.OPUS` consumer follows, including the Anthropic hot test, the document suite, and the live capability matrix
|
|
10
|
+
- `COST["claude-opus-5-5"]` prices the model at $4.00 input, $20.00 output, $5.00 five-minute cache write, $8.00 one-hour cache write, and $0.20 cache read per million tokens. Anthropic reads cache on Opus 5.5 at 0.05x input, half the standard 0.1x multiplier
|
|
11
|
+
- `MODEL.SOL` names `gpt-6-sol` and `MODEL.LUNA` names `gpt-6-luna`, superseding `gpt-5.6-sol` and `gpt-5.6-luna`. `PROVIDER.OPENAI.DEFAULT` is `MODEL.SOL` and follows. `MODEL.TERRA` stays on `gpt-5.6-terra`: OpenAI has released no GPT-6 model in that tier
|
|
12
|
+
- `MODEL.OPENROUTER.LUNA` names `openai/gpt-6-luna`, keeping the gateway route on the same tier's current model. Verified served on the OpenRouter models endpoint
|
|
13
|
+
- `COST["gpt-6-luna"]` at $0.10 input, $0.50 output, $0.01 cache read, and `COST["gpt-6-sol"]` at $2.00 input, $10.00 output, $0.20 cache read per million tokens. Both launched at standard rates with no promotional window footnoted, unlike GPT-5.6 Sol. Their long-context tiers are excluded as the long-prompt surcharge
|
|
14
|
+
- `COST["claude-opus-5"]`, `COST["gpt-5.6-sol"]`, and `COST["gpt-5.6-luna"]` remain so historic usage records stay replayable
|
|
15
|
+
|
|
16
|
+
## Testing
|
|
17
|
+
|
|
18
|
+
- `npm run test -w packages/llm`: full unit suite passes
|
|
19
|
+
- Live, 2026-09-22 (`APP_MODELS=claude-opus-5-5 npm run test:llm:matrix`): all seven capability cells passed — plain, tools, structured, both, pdf, image, and temperature
|
|
20
|
+
- Live, 2026-09-22 (`APP_MODELS=gpt-6-sol,gpt-6-luna,openai/gpt-6-luna npm run test:llm:matrix`): all seven capability cells passed on all three ids
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
---
|
|
2
|
+
version: 1.3.48
|
|
3
|
+
date: 2026-09-23
|
|
4
|
+
summary: MODEL.FIREWORKS.GLM names glm-5p3; adds MODEL.FIREWORKS.GLM_FLASH, MODEL.FIREWORKS.DEEPSEEK_FLASH, and MODEL.OPENROUTER.MIMO
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## Changes
|
|
8
|
+
|
|
9
|
+
- `MODEL.FIREWORKS.GLM` names `accounts/fireworks/models/glm-5p3`, superseding `glm-5p2`. `PROVIDER.FIREWORKS.DEFAULT` is `MODEL.FIREWORKS.GLM` and follows
|
|
10
|
+
- `MODEL.FIREWORKS.GLM_FLASH` names `accounts/fireworks/models/glm-5p3-flash`
|
|
11
|
+
- `MODEL.FIREWORKS.DEEPSEEK_FLASH` names `accounts/fireworks/models/deepseek-v4p1-flash`, alongside `MODEL.FIREWORKS.DEEPSEEK` (`deepseek-v4-pro-0813`)
|
|
12
|
+
- `MODEL.OPENROUTER.MIMO` names `xiaomi/mimo-v2.6-flash`. OpenRouter routes stay unpriced
|
|
13
|
+
- `COST` prices `glm-5p3` at $1.40 input, $4.40 output, $0.26 cache read; `glm-5p3-flash` at $0.15 input, $0.50 output, $0.03 cache read; `deepseek-v4p1-flash` at $0.30 input, $1.20 output, $0.006 cache read per million tokens (Fireworks standard serverless rates)
|
|
14
|
+
- `COST["accounts/fireworks/models/glm-5p2"]` remains so historic usage records stay replayable
|
|
15
|
+
- Matrix expectations: `GLM_FLASH` and `DEEPSEEK_FLASH` advertise image input and skip only `pdf`; `GLM` (`glm-5p3`) advertises no image input and skips both
|
|
16
|
+
|
|
17
|
+
## Testing
|
|
18
|
+
|
|
19
|
+
- `npm run test -w packages/llm`: full unit suite passes
|
|
20
|
+
- Live, 2026-09-23 (`APP_MODELS=accounts/fireworks/models/deepseek-v4p1-flash,accounts/fireworks/models/glm-5p3,accounts/fireworks/models/glm-5p3-flash npm run test:llm:matrix`): every exercised cell passed on all three ids
|
|
21
|
+
- Live, 2026-09-23 (`APP_MODELS=xiaomi/mimo-v2.6-flash npm run test:llm:matrix`): all seven capability cells passed — plain, tools, structured, both, pdf, image, and temperature
|
package/skills/llm.md
CHANGED
|
@@ -13,7 +13,9 @@ Unified interface for calling LLM providers with multi-turn conversations, tool
|
|
|
13
13
|
import Llm from "@jaypie/llm";
|
|
14
14
|
|
|
15
15
|
// Auto-detect provider from model name
|
|
16
|
-
const response = await Llm.operate("What is 2+2?", {
|
|
16
|
+
const response = await Llm.operate("What is 2+2?", {
|
|
17
|
+
model: "claude-sonnet-5",
|
|
18
|
+
});
|
|
17
19
|
console.log(response.content); // "4"
|
|
18
20
|
```
|
|
19
21
|
|
|
@@ -30,19 +32,19 @@ Each `PROVIDER.*.DEFAULT` is itself a `MODEL.*` reference, so this table names
|
|
|
30
32
|
the alias rather than the id it currently resolves to. Read the id off
|
|
31
33
|
`constants.ts`, which is the only place any id lives.
|
|
32
34
|
|
|
33
|
-
| Provider
|
|
34
|
-
|
|
35
|
-
| OpenAI
|
|
36
|
-
| Anthropic
|
|
37
|
-
| Google
|
|
38
|
-
| Fireworks
|
|
39
|
-
| Mistral
|
|
40
|
-
| OpenRouter | "openrouter"
|
|
41
|
-
| xAI
|
|
42
|
-
| Meta
|
|
43
|
-
| Bedrock
|
|
44
|
-
| TypeSafe
|
|
45
|
-
| LlamaCloud | "llamacloud", "llamaindex", "llamaparse"
|
|
35
|
+
| Provider | Match Keywords | Default Model (`PROVIDER.*.DEFAULT`) |
|
|
36
|
+
| ---------- | ---------------------------------------------------------------------------------------------------------------- | ------------------------------------ |
|
|
37
|
+
| OpenAI | "openai", "gpt", "astra", "sol", "terra", "luna", /^o\d/ | `MODEL.SOL` |
|
|
38
|
+
| Anthropic | "anthropic", "claude", "fable", "haiku", "mythos", "opus", "sonnet" | `MODEL.SONNET` |
|
|
39
|
+
| Google | "google", "gemini" | `MODEL.GEMINI_FLASH` |
|
|
40
|
+
| Fireworks | "fireworks" (also matched inside ids like `accounts/fireworks/models/...`) | `MODEL.FIREWORKS.GLM` |
|
|
41
|
+
| Mistral | "mistral", "ministral", "codestral", "devstral", "magistral", "pixtral", "voxtral" | `MODEL.MISTRAL.LARGE` |
|
|
42
|
+
| OpenRouter | "openrouter" | `MODEL.OPENROUTER.SONNET` |
|
|
43
|
+
| xAI | "xai", "grok" | `MODEL.GROK` |
|
|
44
|
+
| Meta | "muse" (plus the exact name `meta` and a `meta:` prefix) | `MODEL.MUSE_SPARK` |
|
|
45
|
+
| Bedrock | "amazon.nova", "anthropic.claude", "meta.llama", "deepseek.", "google.gemma", "moonshotai.", "openai.gpt-oss", … | `MODEL.NOVA_PRO` |
|
|
46
|
+
| TypeSafe | "jev", "typesafe" | `MODEL.JEV` |
|
|
47
|
+
| LlamaCloud | "llamacloud", "llamaindex", "llamaparse" | `MODEL.LLAMAPARSE.AGENTIC` |
|
|
46
48
|
|
|
47
49
|
The provider name for Gemini models is `"google"` — `"gemini"` is accepted as a deprecated alias.
|
|
48
50
|
|
|
@@ -57,7 +59,7 @@ LlamaCloud serves LlamaParse, which extracts documents and generates no text. Th
|
|
|
57
59
|
### Model Constants
|
|
58
60
|
|
|
59
61
|
- **`PROVIDER.<name>.DEFAULT`** — the single default model per provider (above), used when no `model` is given.
|
|
60
|
-
- **`LLM.MODEL.*`** — the named model catalog (e.g. `MODEL.SONNET`, `MODEL.ASTRA`, `MODEL.SOL`, `MODEL.GEMINI_FLASH`, `MODEL.GROK`, `MODEL.MUSE_SPARK`, `MODEL.MUSE_SPARK_CONTRIBUTOR`, `MODEL.NOVA_PRO`, `MODEL.NOVA_LITE`), plus three nested subtrees: `MODEL.FIREWORKS.*` for Fireworks serverless models (`DEEPSEEK`, `GLM`, `GPT_OSS`, `INKLING`, `KIMI`, `MINIMAX`, `NEMOTRON`, `QWEN`), `MODEL.MISTRAL.*` (`LARGE`, `OCR`, `SMALL`), `MODEL.LLAMAPARSE.*` for LlamaParse tiers (`FAST`, `COST_EFFECTIVE`, `AGENTIC`, `AGENTIC_PLUS`), and `MODEL.OPENROUTER.*` for provider-prefixed routes (`GLM`, `LUNA`, `SONNET`). Pick specific models from here. `MODEL.MISTRAL.OCR` and `MODEL.LLAMAPARSE.*` are document-extraction engines reached through `Llm.ocr`, not chat completions: they are priced per page in `LLM.PAGE_COST` (USD per 1,000 pages, keyed by literal id) and carry no `COST` entry. Amazon's Nova models are first-class ids served over Bedrock; Bedrock's third-party routes are not catalogued — pass the literal id (e.g. `us.anthropic.claude-sonnet-4-6`) and `determineModelProvider` resolves it to `bedrock`.
|
|
62
|
+
- **`LLM.MODEL.*`** — the named model catalog (e.g. `MODEL.SONNET`, `MODEL.ASTRA`, `MODEL.SOL`, `MODEL.GEMINI_FLASH`, `MODEL.GROK`, `MODEL.MUSE_SPARK`, `MODEL.MUSE_SPARK_CONTRIBUTOR`, `MODEL.NOVA_PRO`, `MODEL.NOVA_LITE`), plus three nested subtrees: `MODEL.FIREWORKS.*` for Fireworks serverless models (`DEEPSEEK`, `DEEPSEEK_FLASH`, `GLM`, `GLM_FLASH`, `GPT_OSS`, `INKLING`, `KIMI`, `MINIMAX`, `NEMOTRON`, `QWEN`), `MODEL.MISTRAL.*` (`LARGE`, `OCR`, `SMALL`), `MODEL.LLAMAPARSE.*` for LlamaParse tiers (`FAST`, `COST_EFFECTIVE`, `AGENTIC`, `AGENTIC_PLUS`), and `MODEL.OPENROUTER.*` for provider-prefixed routes (`GLM`, `LUNA`, `MIMO`, `SONNET`). Pick specific models from here. `MODEL.MISTRAL.OCR` and `MODEL.LLAMAPARSE.*` are document-extraction engines reached through `Llm.ocr`, not chat completions: they are priced per page in `LLM.PAGE_COST` (USD per 1,000 pages, keyed by literal id) and carry no `COST` entry. Amazon's Nova models are first-class ids served over Bedrock; Bedrock's third-party routes are not catalogued — pass the literal id (e.g. `us.anthropic.claude-sonnet-4-6`) and `determineModelProvider` resolves it to `bedrock`.
|
|
61
63
|
- `MODEL.MUSE_SPARK` and `MODEL.MUSE_SPARK_CONTRIBUTOR` are the same Meta model at two tiers. The contributor tier is roughly a tenth of the price because prompts and completions may train Meta models; it is limited to 100 RPM and does not accept `reasoning.effort: "max"`. It is an explicit opt-in: `PROVIDER.META.DEFAULT` is `MODEL.MUSE_SPARK`.
|
|
62
64
|
- The catalog is the **single source of truth for CI coverage**: `packages/llm/test/models.ts` derives the live capability matrix from `MODEL.*` plus each `PROVIDER.*.DEFAULT`, and the workflow shards it by provider. Adding a model to `MODEL.*` puts it under test; no id list exists anywhere else.
|
|
63
65
|
- **Deprecated:** the size-tier map `PROVIDER.<name>.MODEL.{DEFAULT,LARGE,SMALL,TINY}`, the `DEFAULT.MODEL` bundle, and `ALL` are `@deprecated` and retired in 2.0 — use `PROVIDER.*.DEFAULT` for defaults and `MODEL.*` for named models.
|
|
@@ -70,24 +72,26 @@ LlamaCloud serves LlamaParse, which extracts documents and generates no text. Th
|
|
|
70
72
|
import { LLM, type LlmModelCost } from "@jaypie/llm";
|
|
71
73
|
|
|
72
74
|
const rate: LlmModelCost | undefined = LLM.COST[response.model];
|
|
73
|
-
const dollars = rate
|
|
75
|
+
const dollars = rate
|
|
76
|
+
? (usage.input * rate.input + usage.output * rate.output) / 1e6
|
|
77
|
+
: 0;
|
|
74
78
|
```
|
|
75
79
|
|
|
76
|
-
| Field
|
|
77
|
-
|
|
78
|
-
| `input`
|
|
79
|
-
| `output`
|
|
80
|
-
| `cachedInputRead`
|
|
80
|
+
| Field | Meaning |
|
|
81
|
+
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
82
|
+
| `input` | Uncached input tokens |
|
|
83
|
+
| `output` | Output tokens |
|
|
84
|
+
| `cachedInputRead` | Cache-read (hit) tokens. Omitted when the provider does not price reads separately |
|
|
81
85
|
| `cachedInputWrite` | Cache-write tokens. Scalar when TTL-invariant (`0` where the provider publishes a free write, as Bedrock does for Nova), or keyed `{ "5m", "1h" }` matching `LlmCache`. Omitted means writes bill at `input` (a TTL-keyed premium is Anthropic-only) |
|
|
82
|
-
| `reasoning`
|
|
86
|
+
| `reasoning` | Reasoning tokens when billed apart from `output`. Unset everywhere today |
|
|
83
87
|
|
|
84
88
|
Rates are the standard short-context text tier. Introductory, batch, flex, priority, fast-mode, and data-residency pricing are excluded, as are long-prompt surcharges (Gemini 3.1 Pro above 200K, Grok at 200K, GPT-5.5 above 272K). Amazon's Nova models are priced at the standard US on-demand rate (`us.` geo profile for Nova 2 Lite; the cheaper `global.` profile is not modeled). **Gateway routes are deliberately unpriced**: `MODEL.OPENROUTER.*`, and any Bedrock id reselling a third-party model, cost per route and per region, so `COST` returns `undefined` for them — price those against the backend model or the gateway's own published rate. Unlisted ids return `undefined` — always handle a miss.
|
|
85
89
|
|
|
86
90
|
```typescript
|
|
87
91
|
// Provider auto-detected from the model string. Literal ids here on purpose:
|
|
88
92
|
// the match is on a keyword inside the id, which an alias would hide.
|
|
89
|
-
await Llm.operate(input, { model: "gpt-5.6-sol" });
|
|
90
|
-
await Llm.operate(input, { model: "claude-opus-5" });
|
|
93
|
+
await Llm.operate(input, { model: "gpt-5.6-sol" }); // "gpt" -> OpenAI
|
|
94
|
+
await Llm.operate(input, { model: "claude-opus-5" }); // "claude" -> Anthropic
|
|
91
95
|
await Llm.operate(input, { model: "gemini-3.8-flash" }); // "gemini" -> Google
|
|
92
96
|
```
|
|
93
97
|
|
|
@@ -106,11 +110,11 @@ const response = await Llm.operate(input, {
|
|
|
106
110
|
});
|
|
107
111
|
|
|
108
112
|
// Response structure
|
|
109
|
-
response.content;
|
|
110
|
-
response.history;
|
|
111
|
-
response.usage;
|
|
112
|
-
response.reasoning;
|
|
113
|
-
response.status;
|
|
113
|
+
response.content; // string | object (structured output)
|
|
114
|
+
response.history; // LlmHistory - for follow-up calls
|
|
115
|
+
response.usage; // Token usage per turn
|
|
116
|
+
response.reasoning; // Extended thinking (if available)
|
|
117
|
+
response.status; // "completed" | "incomplete" | "in_progress"
|
|
114
118
|
```
|
|
115
119
|
|
|
116
120
|
### send() - Simple Completions
|
|
@@ -129,7 +133,9 @@ const response = await Llm.send("Explain REST APIs", {
|
|
|
129
133
|
For real-time output:
|
|
130
134
|
|
|
131
135
|
```typescript
|
|
132
|
-
for await (const chunk of Llm.stream("Tell me a story", {
|
|
136
|
+
for await (const chunk of Llm.stream("Tell me a story", {
|
|
137
|
+
model: LLM.MODEL.SOL,
|
|
138
|
+
})) {
|
|
133
139
|
switch (chunk.type) {
|
|
134
140
|
case "text":
|
|
135
141
|
process.stdout.write(chunk.content);
|
|
@@ -184,7 +190,7 @@ import Llm, { tools, JaypieToolkit } from "@jaypie/llm";
|
|
|
184
190
|
// Use individual tools
|
|
185
191
|
const response = await Llm.operate("Roll 2d6", {
|
|
186
192
|
model: LLM.MODEL.SOL,
|
|
187
|
-
tools,
|
|
193
|
+
tools, // Includes: random, roll, time, weather
|
|
188
194
|
});
|
|
189
195
|
|
|
190
196
|
// Or use the pre-configured toolkit
|
|
@@ -217,6 +223,7 @@ const response = await Llm.operate("What's the weather?", {
|
|
|
217
223
|
```
|
|
218
224
|
|
|
219
225
|
When enabled:
|
|
226
|
+
|
|
220
227
|
- Each tool receives an `__Explanation` parameter requiring the model to state why it's calling the tool
|
|
221
228
|
- The explanation is stripped before the tool executes (tools receive clean arguments)
|
|
222
229
|
- Useful for debugging and understanding LLM decision-making
|
|
@@ -239,6 +246,7 @@ const toolkit = new Toolkit([
|
|
|
239
246
|
```
|
|
240
247
|
|
|
241
248
|
The resolved message surfaces in three places during `operate()`/`stream()`:
|
|
249
|
+
|
|
242
250
|
- The Toolkit's `log` option: `new Toolkit(tools, { log: (message, { name, args }) => ... })`
|
|
243
251
|
- The `beforeEachTool`, `afterEachTool`, and `onToolError` hooks as `message`
|
|
244
252
|
- The `tool_call` progress event as `tool.message` (`operate()` only)
|
|
@@ -251,12 +259,12 @@ Tools may declare `readOnly: true` (mirroring MCP's `readOnlyHint`) to state the
|
|
|
251
259
|
|
|
252
260
|
```typescript
|
|
253
261
|
const toolkit = new Toolkit([
|
|
254
|
-
{ name: "search_docs", readOnly: true
|
|
255
|
-
{ name: "post_message"
|
|
262
|
+
{ name: "search_docs", readOnly: true /* ... */ },
|
|
263
|
+
{ name: "post_message" /* ... */ },
|
|
256
264
|
]);
|
|
257
265
|
|
|
258
266
|
const verification = toolkit.filter({ readOnly: true }); // search_docs only
|
|
259
|
-
const effectful = toolkit.filter({ readOnly: false });
|
|
267
|
+
const effectful = toolkit.filter({ readOnly: false }); // post_message only
|
|
260
268
|
const custom = toolkit.filter((tool) => tool.name.startsWith("search_"));
|
|
261
269
|
```
|
|
262
270
|
|
|
@@ -316,14 +324,17 @@ const done = await Llm.operate(undefined, {
|
|
|
316
324
|
### Natural Schema
|
|
317
325
|
|
|
318
326
|
```typescript
|
|
319
|
-
const result = await Llm.operate(
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
327
|
+
const result = await Llm.operate(
|
|
328
|
+
"Extract contact info from: John Doe, john@example.com, 555-1234",
|
|
329
|
+
{
|
|
330
|
+
model: LLM.MODEL.SOL,
|
|
331
|
+
format: {
|
|
332
|
+
name: String,
|
|
333
|
+
email: String,
|
|
334
|
+
phone: String,
|
|
335
|
+
},
|
|
325
336
|
},
|
|
326
|
-
|
|
337
|
+
);
|
|
327
338
|
// result.content = { name: "John Doe", email: "john@example.com", phone: "555-1234" }
|
|
328
339
|
```
|
|
329
340
|
|
|
@@ -365,10 +376,13 @@ const PersonSchema = z.object({
|
|
|
365
376
|
hobbies: z.array(z.string()),
|
|
366
377
|
});
|
|
367
378
|
|
|
368
|
-
const result = await Llm.operate(
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
379
|
+
const result = await Llm.operate(
|
|
380
|
+
"Parse: Alice is 30 and likes hiking and reading",
|
|
381
|
+
{
|
|
382
|
+
model: LLM.MODEL.SOL,
|
|
383
|
+
format: PersonSchema,
|
|
384
|
+
},
|
|
385
|
+
);
|
|
372
386
|
```
|
|
373
387
|
|
|
374
388
|
### Bare JSON Schema
|
|
@@ -392,7 +406,10 @@ const result = await Llm.operate("Analyze this chargeback", {
|
|
|
392
406
|
### Converting between Natural Schema and JSON Schema
|
|
393
407
|
|
|
394
408
|
```typescript
|
|
395
|
-
import {
|
|
409
|
+
import {
|
|
410
|
+
jsonSchemaToNaturalSchema,
|
|
411
|
+
naturalSchemaToJsonSchema,
|
|
412
|
+
} from "@jaypie/llm";
|
|
396
413
|
|
|
397
414
|
naturalSchemaToJsonSchema({ name: String, age: Number });
|
|
398
415
|
// { type: "object", properties: { name: { type: "string" }, age: { type: "number" } }, required: ["name", "age"] }
|
|
@@ -442,17 +459,17 @@ const { answers, emulated } = await Llm.question(ticket, {
|
|
|
442
459
|
});
|
|
443
460
|
|
|
444
461
|
answers.department; // { type: "choice", choice: "billing", confidence: 0.98, probabilities: {...} }
|
|
445
|
-
answers.is_urgent;
|
|
462
|
+
answers.is_urgent; // { type: "noul", noul: 0.94 }
|
|
446
463
|
answers.frustration; // { type: "score", score: 1.7, confidence: 0.56, legend: {...}, probabilities: {...} }
|
|
447
464
|
```
|
|
448
465
|
|
|
449
466
|
### Question Types
|
|
450
467
|
|
|
451
|
-
| Type
|
|
452
|
-
|
|
453
|
-
| `noul`
|
|
454
|
-
| `choice` | `Record<option, description \| null>`, 1 to 255 options | `{ type, choice, confidence, probabilities }` — `choice` is the argmax, `probabilities` keyed by option name
|
|
455
|
-
| `score`
|
|
468
|
+
| Type | Criteria | Answer |
|
|
469
|
+
| -------- | ------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
470
|
+
| `noul` | optional `{ true, false }` descriptions | `{ type, noul }` — probability 0 to 1, no confidence: the number is the degree of belief |
|
|
471
|
+
| `choice` | `Record<option, description \| null>`, 1 to 255 options | `{ type, choice, confidence, probabilities }` — `choice` is the argmax, `probabilities` keyed by option name |
|
|
472
|
+
| `score` | ordered `string[]`, 2 to 10 levels lowest first | `{ type, score, confidence, legend, probabilities }` — `score` is the probability-weighted level index, `legend` and `probabilities` keyed by level index as a string |
|
|
456
473
|
|
|
457
474
|
`instructions` and every criteria value accept a string, an object, or an
|
|
458
475
|
array. `state` accepts the same three and is text only: no files, no images,
|
|
@@ -569,13 +586,16 @@ const second = await Llm.operate("What's my name?", {
|
|
|
569
586
|
## Input with Files and Images
|
|
570
587
|
|
|
571
588
|
```typescript
|
|
572
|
-
const response = await Llm.operate(
|
|
573
|
-
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
|
|
589
|
+
const response = await Llm.operate(
|
|
590
|
+
[
|
|
591
|
+
"Analyze these documents",
|
|
592
|
+
{ file: "report.pdf", bucket: "my-bucket" },
|
|
593
|
+
{ image: "chart.png" },
|
|
594
|
+
],
|
|
595
|
+
{
|
|
596
|
+
model: LLM.MODEL.SONNET,
|
|
597
|
+
},
|
|
598
|
+
);
|
|
579
599
|
```
|
|
580
600
|
|
|
581
601
|
## Placeholders and Data
|
|
@@ -639,16 +659,16 @@ const response = await Llm.operate(input, {
|
|
|
639
659
|
|
|
640
660
|
Fields carried by each event (`turn` is 1-indexed):
|
|
641
661
|
|
|
642
|
-
| Event
|
|
643
|
-
|
|
644
|
-
| `start`
|
|
645
|
-
| `model_request`
|
|
646
|
-
| `model_response` | `turn`, `content` (text, if any), `toolCalls` (`[{ name, arguments }]`, if any), `usage` (this turn)
|
|
647
|
-
| `tool_call`
|
|
648
|
-
| `tool_result`
|
|
649
|
-
| `tool_error`
|
|
650
|
-
| `retry`
|
|
651
|
-
| `done`
|
|
662
|
+
| Event | Fields |
|
|
663
|
+
| ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
664
|
+
| `start` | `model`, `provider`, `maxTurns` |
|
|
665
|
+
| `model_request` | `turn`, `model` |
|
|
666
|
+
| `model_response` | `turn`, `content` (text, if any), `toolCalls` (`[{ name, arguments }]`, if any), `usage` (this turn) |
|
|
667
|
+
| `tool_call` | `turn`, `tool: { name, arguments, message }` — fires before the tool runs; `arguments` is the JSON string; `message` is the resolved `LlmTool.message`, when the tool defines one |
|
|
668
|
+
| `tool_result` | `turn`, `tool: { name }` — the result value is deliberately omitted (it can be arbitrarily large); use the `afterEachTool` hook to receive it |
|
|
669
|
+
| `tool_error` | `turn`, `tool: { name }`, `error` (message string) |
|
|
670
|
+
| `retry` | `turn`, `error` (message string) |
|
|
671
|
+
| `done` | `turn` (total turns used), `content` (final text or structured output), `usage` (cumulative) |
|
|
652
672
|
|
|
653
673
|
Errors thrown by the callback are logged and never interrupt the loop. Hooks remain the right choice when you need the full provider request/response payloads. `stream()` communicates progress through its chunks; `onProgress` applies to `operate()`.
|
|
654
674
|
|
|
@@ -693,7 +713,7 @@ Set `LLM_EXCHANGE_ENABLED` (truthy, except `false`/`0`) and every `operate()` an
|
|
|
693
713
|
|
|
694
714
|
### Registering the Store
|
|
695
715
|
|
|
696
|
-
The lazy resolution loads `@jaypie/dynamodb` from `node_modules`. When the host's copy is bundler-managed instead — a Next.js server bundle, for example — that is a
|
|
716
|
+
The lazy resolution loads `@jaypie/dynamodb` from `node_modules`. When the host's copy is bundler-managed instead — a Next.js server bundle, for example — that is a _second_ instance: the host initialized its copy, the loaded one is uninitialized, and every exchange is dropped with `[storeExchange] DynamoDB client is not initialized`. Register the instance at bootstrap to name which copy to use:
|
|
697
717
|
|
|
698
718
|
```typescript
|
|
699
719
|
import * as dynamodb from "@jaypie/dynamodb";
|
|
@@ -761,8 +781,8 @@ const response = await Llm.operate(input, {
|
|
|
761
781
|
### Fallback Response Metadata
|
|
762
782
|
|
|
763
783
|
```typescript
|
|
764
|
-
response.provider;
|
|
765
|
-
response.fallbackUsed;
|
|
784
|
+
response.provider; // Which provider handled the request
|
|
785
|
+
response.fallbackUsed; // true if a fallback was used
|
|
766
786
|
response.fallbackAttempts; // Number of providers tried (1 = primary only)
|
|
767
787
|
```
|
|
768
788
|
|
|
@@ -785,7 +805,7 @@ The first constructor argument may be a provider name **or** a model name. When
|
|
|
785
805
|
```typescript
|
|
786
806
|
import Llm, { LLM } from "@jaypie/llm";
|
|
787
807
|
|
|
788
|
-
const llm = new Llm("claude-sonnet-4-6");
|
|
808
|
+
const llm = new Llm("claude-sonnet-4-6"); // -> anthropic, retained verbatim
|
|
789
809
|
const flash = new Llm(LLM.MODEL.GEMINI_FLASH); // -> google, whatever the alias names
|
|
790
810
|
```
|
|
791
811
|
|
|
@@ -820,7 +840,12 @@ Inside a Jaypie handler (`expressHandler`, `lambdaHandler`, and stream variants)
|
|
|
820
840
|
"tools": { "get_weather": 2, "roll": 1 },
|
|
821
841
|
"turns": 5,
|
|
822
842
|
"usage": {
|
|
823
|
-
"anthropic:claude-sonnet-4-6": {
|
|
843
|
+
"anthropic:claude-sonnet-4-6": {
|
|
844
|
+
"input": 1840,
|
|
845
|
+
"output": 912,
|
|
846
|
+
"reasoning": 0,
|
|
847
|
+
"total": 2752
|
|
848
|
+
}
|
|
824
849
|
}
|
|
825
850
|
}
|
|
826
851
|
}
|
|
@@ -847,7 +872,7 @@ DD_LLMOBS_ML_APP=my-app # ML app name (dd-trace standard)
|
|
|
847
872
|
Behavior:
|
|
848
873
|
|
|
849
874
|
- **Opt-in and lazy** — `dd-trace` is resolved at runtime via a computed module specifier, so it is **bundler-safe** (esbuild will not bundle it) and **not** a dependency. Absence is a silent no-op; instrumentation failures never break the LLM call.
|
|
850
|
-
- **Parenting is AsyncLocalStorage-based** — spans attach to whatever LLMObs span is active when created. Wrapping a call in a consumer span (e.g. `llmobs.trace({ kind: "workflow" }, () => Llm.operate(...))`) nests ours under it. The Datadog Lambda layer provides APM spans automatically, but not an enclosing
|
|
875
|
+
- **Parenting is AsyncLocalStorage-based** — spans attach to whatever LLMObs span is active when created. Wrapping a call in a consumer span (e.g. `llmobs.trace({ kind: "workflow" }, () => Llm.operate(...))`) nests ours under it. The Datadog Lambda layer provides APM spans automatically, but not an enclosing _LLMObs_ span around an arbitrary handler.
|
|
851
876
|
- **`operate()`** spans form a full tree (model + tool nest under the enclosing span).
|
|
852
877
|
- **`stream()`** spans attach to any active enclosing span, but within a single stream the `llm` and `tool` spans are **siblings** — the streamed model span is held open across `yield` boundaries, so it is not the active span when tools run.
|
|
853
878
|
|
|
@@ -914,11 +939,11 @@ budget runs out. Those errors carry `error.reason` (`LlmResponseErrorReason`)
|
|
|
914
939
|
because status alone cannot identify them: an exhausted turn budget is a 429,
|
|
915
940
|
exactly like a provider rate limit.
|
|
916
941
|
|
|
917
|
-
| `reason`
|
|
918
|
-
|
|
919
|
-
| `max_turns`
|
|
920
|
-
| `tool_errors` | 502
|
|
921
|
-
| `incomplete`
|
|
942
|
+
| `reason` | Status | Meaning |
|
|
943
|
+
| ------------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
944
|
+
| `max_turns` | 429 | The model asked for another tool call after `turns` ran out. Nothing failed; the run did not converge. |
|
|
945
|
+
| `tool_errors` | 502 | Tool execution failed six times in a row and the loop stopped. |
|
|
946
|
+
| `incomplete` | 502 | The provider cut the model off before it finished (an output token ceiling, a content filter). `content` holds the partial text; `error.detail` names the provider's reason. |
|
|
922
947
|
|
|
923
948
|
```typescript
|
|
924
949
|
import { LlmResponseErrorReason } from "@jaypie/llm";
|
|
@@ -971,27 +996,27 @@ describe("LLM Integration", () => {
|
|
|
971
996
|
|
|
972
997
|
```typescript
|
|
973
998
|
interface LlmOperateOptions {
|
|
974
|
-
cache?: boolean | 0 | "5m" | "1h";
|
|
975
|
-
data?: Record<string, any>;
|
|
976
|
-
effort?: LlmEffort;
|
|
999
|
+
cache?: boolean | 0 | "5m" | "1h"; // Prompt caching (default on: 1h Anthropic, 5m elsewhere; false/0 disables)
|
|
1000
|
+
data?: Record<string, any>; // Placeholder substitution data
|
|
1001
|
+
effort?: LlmEffort; // Provider-neutral reasoning effort (lowest|low|medium|high|highest)
|
|
977
1002
|
fallback?: LlmFallbackConfig[] | false; // Fallback provider chain
|
|
978
1003
|
format?: NaturalSchema | JsonObject | ZodType; // Structured output schema (natural syntax preferred)
|
|
979
|
-
history?: LlmHistory;
|
|
980
|
-
hooks?: LlmHooks;
|
|
981
|
-
instructions?: string;
|
|
982
|
-
model?: string;
|
|
983
|
-
providerOptions?: JsonObject;
|
|
984
|
-
system?: string;
|
|
985
|
-
temperature?: number;
|
|
986
|
-
tools?: LlmTool[] | Toolkit;
|
|
987
|
-
turns?: boolean | number;
|
|
988
|
-
user?: string;
|
|
1004
|
+
history?: LlmHistory; // Previous conversation
|
|
1005
|
+
hooks?: LlmHooks; // Lifecycle callbacks
|
|
1006
|
+
instructions?: string; // Additional instructions
|
|
1007
|
+
model?: string; // Model override
|
|
1008
|
+
providerOptions?: JsonObject; // Provider-specific request fields (passthrough)
|
|
1009
|
+
system?: string; // System prompt
|
|
1010
|
+
temperature?: number; // Sampling temperature (0-2)
|
|
1011
|
+
tools?: LlmTool[] | Toolkit; // Available tools
|
|
1012
|
+
turns?: boolean | number; // Max conversation turns
|
|
1013
|
+
user?: string; // User ID for logging
|
|
989
1014
|
}
|
|
990
1015
|
|
|
991
1016
|
interface LlmFallbackConfig {
|
|
992
|
-
provider: string;
|
|
993
|
-
model?: string;
|
|
994
|
-
apiKey?: string;
|
|
1017
|
+
provider: string; // Provider name (e.g., "openai", "anthropic", "google")
|
|
1018
|
+
model?: string; // Model to use (optional, uses provider default)
|
|
1019
|
+
apiKey?: string; // API key (optional, uses environment variable)
|
|
995
1020
|
}
|
|
996
1021
|
```
|
|
997
1022
|
|
|
@@ -1010,7 +1035,7 @@ import Llm, { LLM } from "@jaypie/llm";
|
|
|
1010
1035
|
|
|
1011
1036
|
await Llm.operate("Solve this step by step", {
|
|
1012
1037
|
model: LLM.MODEL.SOL,
|
|
1013
|
-
effort: "high",
|
|
1038
|
+
effort: "high", // or LLM.EFFORT.HIGH
|
|
1014
1039
|
});
|
|
1015
1040
|
```
|
|
1016
1041
|
|
|
@@ -1018,12 +1043,12 @@ Per-provider translation (`medium`/`high` stay aligned across providers; ends
|
|
|
1018
1043
|
collapse where a provider has fewer rungs):
|
|
1019
1044
|
|
|
1020
1045
|
| `effort` | OpenAI `reasoning.effort` | Anthropic `output_config.effort` | Gemini 3 `thinkingLevel` | Gemini 2.5 `thinkingBudget` | Grok `reasoning_effort` | Meta `reasoning.effort` | OpenRouter `reasoning.effort` | Fireworks `reasoning_effort` | Mistral `reasoning_effort` |
|
|
1021
|
-
|
|
1022
|
-
| `lowest` | minimal
|
|
1023
|
-
| `low` | low
|
|
1024
|
-
| `medium` | medium
|
|
1025
|
-
| `high` | high
|
|
1026
|
-
| `highest` | xhigh
|
|
1046
|
+
| --------- | ------------------------- | -------------------------------- | ------------------------ | --------------------------- | ----------------------- | ----------------------- | ----------------------------- | ---------------------------- | -------------------------- |
|
|
1047
|
+
| `lowest` | minimal | low | MINIMAL | 512 | low | minimal | minimal | low | none |
|
|
1048
|
+
| `low` | low | low | LOW | 4096 | low | low | low | low | high |
|
|
1049
|
+
| `medium` | medium | medium | MEDIUM | 8192 | medium | medium | medium | medium | high |
|
|
1050
|
+
| `high` | high | high | HIGH | 16384 | high | high | high | high | high |
|
|
1051
|
+
| `highest` | xhigh | max | HIGH | 24576 | high | max | xhigh | high | high |
|
|
1027
1052
|
|
|
1028
1053
|
When a neutral level has no distinct native rung and collapses onto a neighbor
|
|
1029
1054
|
it is "papered over." Papering that follows from the provider's native scale
|
|
@@ -1080,7 +1105,7 @@ tool-calling loop is billed at the provider's cache-read rate (~0.1x input)
|
|
|
1080
1105
|
after the first write. Control with the scalar `cache` option:
|
|
1081
1106
|
|
|
1082
1107
|
```typescript
|
|
1083
|
-
await Llm.operate(input, { system: SYSTEM });
|
|
1108
|
+
await Llm.operate(input, { system: SYSTEM }); // cached (1h Anthropic, else 5m)
|
|
1084
1109
|
await Llm.operate(input, { system: SYSTEM, cache: "5m" }); // cached @ 5m
|
|
1085
1110
|
await Llm.operate(input, { system: SYSTEM, cache: false }); // opt out
|
|
1086
1111
|
```
|