@owlmeans/llm 0.1.18-rc.11 → 0.1.18-rc.13

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (69) hide show
  1. package/README.md +2 -2
  2. package/agent-meta/manifest.json +2 -2
  3. package/agent-meta/skills/llm/SKILL.md +198 -190
  4. package/agent-meta/skills/llm-prompt-caching/SKILL.md +82 -4
  5. package/build/errors.d.ts.map +1 -1
  6. package/build/errors.js.map +1 -1
  7. package/build/execution/service.d.ts.map +1 -1
  8. package/build/execution/service.js +10 -1
  9. package/build/execution/service.js.map +1 -1
  10. package/build/execution/types.d.ts +10 -0
  11. package/build/execution/types.d.ts.map +1 -1
  12. package/build/execution/utils.d.ts.map +1 -1
  13. package/build/execution/utils.js +15 -9
  14. package/build/execution/utils.js.map +1 -1
  15. package/build/helpers/cache.d.ts.map +1 -1
  16. package/build/helpers/json.d.ts.map +1 -1
  17. package/build/helpers/messages.d.ts.map +1 -1
  18. package/build/helpers/retry.d.ts +14 -0
  19. package/build/helpers/retry.d.ts.map +1 -1
  20. package/build/helpers/retry.js +14 -0
  21. package/build/helpers/retry.js.map +1 -1
  22. package/build/helpers/spectate.d.ts.map +1 -1
  23. package/build/model.d.ts +1 -1
  24. package/build/model.d.ts.map +1 -1
  25. package/build/model.js +30 -11
  26. package/build/model.js.map +1 -1
  27. package/build/plugins/anthropic.d.ts +27 -0
  28. package/build/plugins/anthropic.d.ts.map +1 -1
  29. package/build/plugins/anthropic.js +37 -1
  30. package/build/plugins/anthropic.js.map +1 -1
  31. package/build/plugins/index.d.ts.map +1 -1
  32. package/build/plugins/openai.d.ts.map +1 -1
  33. package/build/plugins/types.d.ts +7 -0
  34. package/build/plugins/types.d.ts.map +1 -1
  35. package/build/plugins/utils.d.ts.map +1 -1
  36. package/build/prompt/render.d.ts.map +1 -1
  37. package/build/prompt/service.d.ts.map +1 -1
  38. package/build/prompt/service.js +11 -0
  39. package/build/prompt/service.js.map +1 -1
  40. package/build/prompt/types.d.ts +20 -0
  41. package/build/prompt/types.d.ts.map +1 -1
  42. package/build/service.d.ts.map +1 -1
  43. package/build/service.js.map +1 -1
  44. package/build/types.d.ts +6 -0
  45. package/build/types.d.ts.map +1 -1
  46. package/build/utils/config.d.ts.map +1 -1
  47. package/build/utils/null-report.d.ts.map +1 -1
  48. package/build/utils/null-report.js +9 -1
  49. package/build/utils/null-report.js.map +1 -1
  50. package/build/utils/prompt.d.ts.map +1 -1
  51. package/build/utils/schema.d.ts.map +1 -1
  52. package/build/utils/stream.d.ts.map +1 -1
  53. package/build/utils/stream.js.map +1 -1
  54. package/package.json +6 -6
  55. package/src/execution/service.ts +12 -1
  56. package/src/execution/types.ts +10 -0
  57. package/src/execution/utils.ts +16 -9
  58. package/src/helpers/retry.ts +16 -0
  59. package/src/model.ts +33 -10
  60. package/src/plugins/anthropic.ts +43 -1
  61. package/src/plugins/types.ts +8 -0
  62. package/src/prompt/service.ts +12 -0
  63. package/src/prompt/types.ts +20 -0
  64. package/src/types.ts +6 -0
  65. package/src/utils/null-report.ts +9 -1
  66. package/tests/context.ts +3 -0
  67. package/tests/execution.spec.ts +60 -1
  68. package/tests/plugins.spec.ts +70 -0
  69. package/tests/prompt.spec.ts +43 -0
package/README.md CHANGED
@@ -20,7 +20,7 @@ abstraction that resolves models from an inheritable policy.
20
20
  ## Installation
21
21
 
22
22
  ```bash
23
- bun add @owlmeans/llm @owlmeans/llm-common
23
+ bun add @owlmeans/llm@^0.1.18-rc.12 @owlmeans/llm-common@^0.1.18-rc.11
24
24
  bun add @langchain/core @langchain/openai @langchain/anthropic # peer dependencies
25
25
  ```
26
26
 
@@ -184,7 +184,7 @@ This package ships embedded agent skills under `agent-meta/`. After installing y
184
184
  your project's skill store (`.agents/skills/`):
185
185
 
186
186
  ```sh
187
- npx @owlmeans/agent-skills
187
+ npx @owlmeans/agent-skills@^0.1.18-rc.12
188
188
  ```
189
189
 
190
190
  The embedded files are version-matched to this package release. Do not edit them
@@ -1,8 +1,8 @@
1
1
  {
2
2
  "schemaVersion": 2,
3
3
  "package": "@owlmeans/llm",
4
- "version": "0.1.18-rc.11",
5
- "generatedAt": "2026-09-01T16:28:56.988Z",
4
+ "version": "0.1.18-rc.13",
5
+ "generatedAt": "2026-09-04T22:43:25.441Z",
6
6
  "canonicalRepo": "https://github.com/owlmeans/common",
7
7
  "entries": [
8
8
  {
@@ -8,47 +8,37 @@ user-invocable: false
8
8
  # @owlmeans/llm
9
9
 
10
10
  **Layer:** Core
11
- **Install:** `"@owlmeans/llm": "^0.1.18-rc.11"` in `dependencies` (plus the `@langchain/*` peers)
11
+ **Install:** `"@owlmeans/llm": "^0.1.18-rc.13"` in `dependencies` (plus the `@langchain/*` peers)
12
12
 
13
- The inference runtime. Everything provider-specific is a **plugin**; the model itself only
14
- owns the provider-independent parts (streaming discipline, retries, validation,
15
- observability). Serializable contracts live in `@owlmeans/llm-common`.
13
+ The inference runtime. Everything provider-specific is a **plugin**; the model itself only owns the
14
+ provider-independent parts (streaming discipline, retries, validation, observability). Serializable
15
+ contracts live in `@owlmeans/llm-common`. `src/helpers/` is exported — functions a **consumer** may
16
+ use alongside a model; `src/utils/` is internal and **never exported** (a spec that needs one imports
17
+ it from `../src/utils/…`). Decide the side before placing a function, and never export a `utils/`
18
+ symbol "because a test needs it".
16
19
 
17
20
  ## Key Exports
18
21
 
19
22
  | Export | Description |
20
23
  |--------|-------------|
21
- | `makeLlmModel(options, spectator)` | The four-method model: `ask` / `talk` / `invoke` / `request`. |
22
- | `makeLlmService(options, alias?)` · `appendLlmService(ctx, options, alias?)` | Model factory/registry — resolves a `ModelConfig` by alias, memoized per alias+override. |
23
- | `llmServiceApi(options, self)` | The factory half WITHOUT `createService`, to compose into your own service (role accessors, domain helpers). |
24
- | `makeExecutionService(alias?, options?)` · `appendExecutionService(ctx, alias?, options?)` | Frozen 3-level executions + policy resolution + snapshot/restore/checkpoint + advice. |
24
+ | `makeLlmModel({ model, purpose, prompt?, prompts?, files?, utility?, retries?, … }, spectator)` | The four-method model: `ask` / `talk` / `invoke(input, schema, opts)` / `request`. `model` is an already-resolved `BaseChatModel`. |
25
+ | `makeLlmService(options, alias?)` · `appendLlmService(ctx, options, alias?)` · `llmServiceApi(options, self)` | Model factory/registry — `makeLlmService({ models: () => configs }).getModel(alias, override?)` resolves a `ModelConfig` by alias, memoized per alias+override. The `…Api` half omits `createService`, to compose into your own service (role accessors, domain helpers). |
26
+ | `makeExecutionService(alias?, options?)` · `appendExecutionService(ctx, alias?, options?)` · `executionServiceApi(options, self)` | Frozen 3-level executions + policy resolution + snapshot/restore/checkpoint + advice, and the same composable half. |
25
27
  | `makePromptService(options?, alias?)` · `appendPromptService(ctx, options?, alias?)` · `promptServiceApi(options, self)` | Skill registry + the composition plugin chain. Also at `@owlmeans/llm/prompt`. |
26
28
  | `rolePlugin`, `skillsPlugin`, `contextPlugin`, `BUILT_IN_PROMPT_PLUGINS` | The built-in composition plugins. |
27
- | `renderSkill`, `sortSkills`, `joinChunks`, `compareAlias`, `prefixHash` | Deterministic rendering primitives — reuse them, never re-implement. |
28
- | `readCacheUsage`, `hasCacheActivity` | Normalized prompt-cache accounting from a completion. |
29
- | `executionServiceApi(options, self)` | The execution half without `createService`, for the same composition pattern. |
29
+ | `PromptContext.claim(key)` · `PromptComposeParams.utility` | Per-composition ownership of a key; a cheap model for one plugin-side call. |
30
+ | `renderSkill`, `sortSkills`, `joinChunks`, `compareAlias`, `prefixHash` · `readCacheUsage`, `hasCacheActivity` | Deterministic rendering primitives — reuse them, never re-implement — and normalized prompt-cache accounting from a completion. |
30
31
  | `plugins`, `registerLlmPlugin`, `resolvePlugin`, `pluginOf`, `pluginFor` | The provider-plugin registry. Also at `@owlmeans/llm/plugins`. |
31
32
  | `anthropicPlugin`, `openAiPlugin`, `compatiblePlugin`, `openAiFamily` | Built-in providers; `openAiFamily` is the shared OpenAI-client behaviour to spread into a new plugin. |
32
- | `NO_SAMPLING_PREFIXES`, `rejectsSampling(model)` | Which Claude families reject `temperature`/`top_p`/`top_k` (4.7+ and the 5 family). Consumers pin presets against it. |
33
- | `RESPONSES_API_PREFIXES`, `usesResponsesApi(model)` | Which OpenAI families go through the Responses API and therefore reject `temperature`/`top_p` (`gpt-5*`, `codex-*`). The OpenAI counterpart of the pair above; consumers pin presets against it. |
34
- | `withRetry`, `registerFatalError`, `spectate`, `normalizeInput`, `parseJsonContent`, `coerceToSchema` | Helpers usable alongside a model. Also at `@owlmeans/llm/helpers`. |
33
+ | `NO_SAMPLING_PREFIXES` / `rejectsSampling(model)` · `RESPONSES_API_PREFIXES` / `usesResponsesApi(model)` | Which families reject which sampling parameters — see the table below. Consumers pin presets against them. |
34
+ | `withRetry`, `registerFatalError`, `isFatalError`, `spectate`, `normalizeInput`, `parseJsonContent`, `coerceToSchema` | Helpers usable alongside a model. Also at `@owlmeans/llm/helpers`. |
35
35
  | `LlmError`, `LlmModelError`, `LlmMissconfiguredError`, `LlmPluginError`, `LlmRetryExceededError` | `ResilientError` family. `LlmModelError` is the RETRYABLE one. |
36
36
  | `mergePrompt`, `mergePolicy`, `resolveRole`, `effortPatch` | Execution merge helpers; `mergePrompt` unions skills and takes the deepest role. |
37
37
  | `DEFAULT_MODEL_RETRIES`, `MODEL_STREAM_TIMEOUT_MS` (3 min idle), `FALLBACK_AFTER_ATTEMPTS`, `DEFAULT_EFFORT`, `EFFORT_TABLE`, `MAX_CACHE_BREAKPOINTS`, `MAX_SYSTEM_BREAKPOINTS`, `MIN_CACHEABLE_TOKENS`, `LLM_SERVICE`, `EXECUTION_SERVICE`, `PROMPT_SERVICE` | Tuning + aliases. |
38
38
 
39
- ## helpers/ vs utils/ — the rule this package follows
40
-
41
- - `src/helpers/` — functions a **consumer** may use alongside a model. Exported.
42
- - `src/utils/` — used only inside the library. **Never exported**; a spec that needs one
43
- imports it from `../src/utils/…`.
44
-
45
- Adding a function? Decide which side it belongs to first, then place it. Do not export a
46
- `utils/` symbol "because a test needs it".
47
-
48
39
  ## Provider differences are plugins, never `if`s
49
40
 
50
- `LlmPlugin` is the single seam. If you find yourself writing `instanceof ChatAnthropic` or
51
- `config.provider === …` in `model.ts` or `service.ts`, it belongs on the plugin instead:
41
+ `LlmPlugin` is the single seam: an `instanceof ChatAnthropic` or `config.provider === …` in `model.ts` or `service.ts` belongs on the plugin.
52
42
 
53
43
  | Plugin member | Replaces |
54
44
  |---|---|
@@ -61,176 +51,155 @@ Adding a function? Decide which side it belongs to first, then place it. Do not
61
51
  | `patchCache` | the message-prefix cache marker |
62
52
  | `cacheKey` (via `ModelConfig.cacheKey`) | provider cache-routing hints such as OpenAI's `prompt_cache_key` |
63
53
  | `isFatal` | "this error can never be retried" |
54
+ | `suppressesThinking` | whether the plugin turns reasoning off on the wire FOR THIS CONFIG, which is what drops the `/no_think` prompt directive from the prepared messages |
64
55
 
65
56
  ### A provider that removes a parameter is a plugin concern too
66
57
 
67
- `build` is not the only place a parameter reaches the wire, and `refine` is not only a retry
68
- hook — **every** call is made on the instance `refine` returns, attempt 0 included. A knob
69
- suppressed in `build` and re-applied in `refine` therefore ships on every single request, not
70
- on a retry: an unsupported `temperature` restored there 400s the model's whole family on the
71
- first call, and `isFatal` correctly refuses to retry it, so the failure is immediate and total.
72
-
73
- Both built-in plugins gate their rejected parameters in **both** hooks, through a predicate the
74
- package root exports:
58
+ `build` is not the only place a parameter reaches the wire, and `refine` is not only a retry hook —
59
+ **every** call is made on the instance `refine` returns, attempt 0 included. A knob suppressed in
60
+ `build` and re-applied in `refine` therefore ships on every single request: an unsupported
61
+ `temperature` restored there 400s the model's whole family on the first call, and `isFatal`
62
+ correctly refuses to retry it. Both built-in plugins gate their rejected parameters in **both**
63
+ hooks, through a predicate the package root exports:
75
64
 
76
65
  | Family | Rejects | Predicate |
77
66
  |---|---|---|
78
67
  | Claude 4.7+ and the 5 family | `temperature`, `top_p`, `top_k` | `NO_SAMPLING_PREFIXES` / `rejectsSampling(model)` |
79
68
  | OpenAI Responses API (`gpt-5*`, `codex-*`) | `temperature`, `top_p` | `RESPONSES_API_PREFIXES` / `usesResponsesApi(model)` |
80
69
 
81
- Models below those lines keep the deterministic `temperature: 0` default. `refine` reads the
82
- id off the ACTIVE base (`model.model ?? lc_kwargs.model`), so a same-family `fallback` rung is
83
- judged on its own id rather than the primary's, and the OpenAI check also accepts a
84
- `useResponsesApi` already on `lc_kwargs`.
85
-
86
- Export the predicate rather than keeping it private: consumers pin presets against it
87
- (`viable-agent`'s `tests/presets.test.ts` asserts no preset entry declares a parameter its
88
- model rejects), and a second hand-written copy of the list drifts exactly when a new family is
89
- added — the moment it has to be right.
70
+ Models below those lines keep the deterministic `temperature: 0` default. Each `refine` re-derives
71
+ the family from the ACTIVE base instance it is handed — Anthropic from `modelName ?? model`, OpenAI
72
+ from `model ?? lc_kwargs.model` plus a `useResponsesApi` already on `lc_kwargs` — so a same-family
73
+ `fallback` rung is judged on its own id, not the primary's. Keep the predicate exported: consumers
74
+ pin presets against it (viable-agent's `tests/presets.test.ts` asserts no preset entry declares a
75
+ parameter its model rejects), and a second hand-written copy drifts when a family is added.
90
76
 
91
- **Registration order is load-bearing.** Instance-based lookup (`pluginFor`) returns the
92
- FIRST plugin whose `owns` matches. `compatible` is registered before `openai` because both
93
- build a `ChatOpenAI`, and assuming the tool-calling hack for an unlabelled model is safe
94
- everywhere while assuming native JSON-schema support is not.
77
+ **Registration order is load-bearing.** Instance lookup (`pluginFor`) returns the FIRST plugin whose
78
+ `owns` matches, and `compatible` is registered before `openai` because both build a `ChatOpenAI`:
79
+ assuming the tool-calling hack for an unlabelled model is safe everywhere, assuming native
80
+ JSON-schema support is not.
95
81
 
96
82
  ## System prompts: a role and skills, never a hand-built message
97
83
 
98
84
  `makeLlmModel` takes `prompt` (a `PromptInput`) and a `prompts` resolver. The prompt service
99
- composes them into an ordered, cacheable system message; a caller's own leading
100
- `SystemMessage` is folded into the volatile `Context` block, so an unmigrated call site
101
- still works. **Do not build a persona as a `SystemMessage` in a helper** — declare it as
102
- `PromptPolicy.role` plus registered skills, or the knowledge duplicates and the cache
103
- prefix stops being stable.
104
-
105
- Skills accumulate down the execution chain (project → task → helper) and the deepest
106
- declared `role` wins — see `mergePrompt`. Full rules, breakpoint budget and the provider
107
- facts behind them: [[llm-prompt-caching]].
85
+ composes them into an ordered, cacheable system message; a caller's own leading `SystemMessage` is
86
+ folded into the volatile `Context` block, so an unmigrated call site still works. **Do not build a
87
+ persona as a `SystemMessage` in a helper** — declare it as `PromptPolicy.role` plus registered
88
+ skills, or the knowledge duplicates and the cache prefix stops being stable. Skills accumulate down
89
+ the execution chain (project → task → helper) and the deepest declared `role` wins (`mergePrompt`).
90
+ Block order, the breakpoint budget, the provider facts behind them, and the plugin seams
91
+ `claim(key)` / `utility`: [[llm-prompt-caching]].
108
92
 
109
93
  ## Execution: policy in, model out
110
94
 
111
- ```
112
- ProjectExecution ← root: policy + purpose + models resolver
113
- └─ TaskExecution ← + resumable state (phase/cursor/completed/data)
114
- └─ HelperExecution ← + a RESOLVED model + temperatureFactory, bound to a role
115
- ```
116
-
117
- `prompt` (role + skills) travels on `ExecutionState`, so it survives snapshot/restore;
118
- `prompts` and `files` are collaborators and never enter a snapshot.
119
-
120
- Every method returns a NEW `Object.freeze`d object. Resolution precedence in
121
- `model(exec, role, override)`: **roleOverride → modelOverride → effort tier →
122
- `LlmService.getModel`**. `escalate(exec, { effort })` raises the tier once and cascades to
123
- everything derived from it.
124
-
125
- Extending it for a domain: declare your own `Execution`/input types, list your collaborator
126
- fields in `ExecutionServiceOptions.collaboratorKeys` so they stay out of snapshots, and
127
- instantiate the service generic with your own `ExecutionShape` — **do not narrow the
128
- inherited method signatures**, which would be a contravariance error.
129
-
130
- `snapshot` excludes `state` itself; without that, every `derive`/`escalate`/`withPurpose`
131
- on a task would nest another copy of the previous state.
132
-
133
- `forHelper` also accepts `output` — an initial `maxTokens` for a helper whose one answer is
134
- genuinely large (a whole source file rather than a decision). It is a model selector, not a
135
- field the helper carries: it becomes a call override, is clamped to `maxOutput`, doubles from
136
- there under retry, and survives `temperatureFactory`.
95
+ `ProjectExecution` (policy + purpose + models resolver) → `TaskExecution` (+ resumable state:
96
+ phase/cursor/completed/data) → `HelperExecution` (+ a RESOLVED model + `temperatureFactory`, a role).
97
+
98
+ `prompt` (role + skills) travels on `ExecutionState`, so it survives snapshot/restore; `prompts` and
99
+ `files` are collaborators and never enter a snapshot. Every method returns a NEW `Object.freeze`d
100
+ object. Resolution precedence in `model(exec, role, override)`: **roleOverride → modelOverride →
101
+ effort tier → `LlmService.getModel`**; `escalate(exec, { effort })` raises the tier once and cascades
102
+ to everything derived from it. `snapshot` excludes `state` itself — without that, every
103
+ `derive`/`escalate`/`withPurpose` on a task would nest another copy of the previous state.
104
+
105
+ Extending it for a domain: declare your own `Execution`/input types, list collaborator fields in
106
+ `ExecutionServiceOptions.collaboratorKeys` (kept out of snapshots), instantiate the service generic
107
+ with your own `ExecutionShape`, and **never narrow an inherited method signature** (contravariance).
108
+
109
+ `forHelper` also accepts `output` — an initial `maxTokens` for a helper whose one answer is genuinely
110
+ large (a whole source file, not a decision). It is a model selector, not a field the helper carries:
111
+ it becomes a call override, clamped to `maxOutput`, doubling under retry, surviving `temperatureFactory`.
112
+
113
+ ### The cheap tier: `utility(exec, override?)`
114
+
115
+ Work that is not the work — a relevance pick, a classification, a one-line judgement a plugin needs
116
+ before the real call can be shaped — runs on `ExecutionService.utility`, never on the helper's own
117
+ model. It resolves `policy.utilityRole ?? UTILITY_ROLE` (`@owlmeans/llm-common`, value `'utility'`)
118
+ at `ExecutionEffort.Economy`, through the SAME ladder as `model()`: `roleOverrides` remap it and
119
+ `modelOverrides` pin it as for any other role. The economy floor is local — the execution it was
120
+ asked on keeps its own tier — and `utilityRole` travels on `ModelPolicy`, so it survives
121
+ `forTask`/`escalate` and a snapshot/restore round trip. `utility` returns a `BaseChatModel`, never
122
+ `undefined`: an alias with no registered config reaches `createModel` through `model()` and throws
123
+ `LlmMissconfiguredError`, like any other unregistered role. The `undefined` a prompt plugin has to
124
+ handle comes from the other end — `PromptComposeParams.utility` (and `AgentOptions.utility`) is an
125
+ OPTIONAL resolver, unset wherever no cheap tier is wired, so a plugin that cannot get one degrades
126
+ rather than fails.
137
127
 
138
128
  ### The plugin seam has two hooks, dispatched independently
139
129
 
140
- `ExecutionPlugin` carries `onCheckpoint`/`onRestore` (persist and resume an
141
- `ExecutionState`) and `advise` (answer a performer's question about the project it is
142
- working in — `ExecutionService.advise(exec, request)`, first usable answer wins, a throwing
143
- plugin is skipped, `null` when nobody answers). Advice is advisory by contract: a caller
144
- appends whatever comes back and proceeds unchanged on `null`.
145
-
146
- `checkpoint` dispatches on plugins that declare **`onCheckpoint`**, not on the plugin count —
147
- otherwise registering an advise-only plugin would silently start composing snapshots nobody
148
- consumes.
130
+ `ExecutionPlugin` carries `onCheckpoint`/`onRestore` (persist and resume an `ExecutionState`) and
131
+ `advise` (answer a performer's question about the project it works in —
132
+ `ExecutionService.advise(exec, request)`, first usable answer wins, a throwing plugin is skipped,
133
+ `null` when nobody answers).
134
+ Advice is advisory by contract: a caller appends whatever comes back and proceeds unchanged on
135
+ `null`. `checkpoint` dispatches on plugins declaring **`onCheckpoint`**, not on the plugin count, or
136
+ an advise-only plugin would silently start composing snapshots nobody consumes.
149
137
 
150
138
  ## Classify a provider error by walking `cause`, never by `instanceof` or a surface read
151
139
 
152
- Two independent layers hide the wire shape of a provider failure, and each one alone is
153
- enough to make a fatal error look retryable — which costs all eight attempts with the real
154
- message buried under the repeats.
140
+ Two independent layers hide the wire shape of a provider failure, and each alone makes a fatal error
141
+ look retryable — costing the whole retry budget with the real message buried under the repeats.
155
142
 
156
- 1. **Nested SDK copies.** `@langchain/anthropic` and `@langchain/openai` bundle their OWN
157
- copies of the provider SDKs, so an error they throw is an instance of a DIFFERENT class
158
- than the one this package imports — `e instanceof BadRequestError` silently returns
159
- `false`.
143
+ 1. **Nested SDK copies.** `@langchain/anthropic` and `@langchain/openai` bundle their OWN copies of
144
+ the provider SDKs, so `e instanceof BadRequestError` compares against a DIFFERENT class than the
145
+ one this package imports and silently returns `false`.
160
146
  2. **Langchain's own error wrappers.** A failure is re-wrapped in a typed langchain error
161
- (`ContextOverflowError` for an input past the context window, and its siblings) that
162
- carries the original **only under `cause`** and has no `status` of its own — so
163
- `e.status === 400` misses it too.
164
-
165
- Use `isBadRequest` from `plugins/utils.ts`: it walks the `cause` chain looking for
166
- `status === 400`, bounded in depth so a self-referential chain terminates. Both built-in
167
- `isFatal` implementations go through it.
168
-
169
- A context overflow is the case that makes this urgent rather than merely untidy: `refine`
170
- escalates the **output** budget on each retry, so an over-limit **input** can never improve —
171
- every attempt re-sends the identical oversized request. Consumers hold their locks for the
172
- whole loop, so a single unfixable call becomes minutes of thrash on the caller's side.
147
+ (`ContextOverflowError` for an input past the context window, and its siblings) carrying the
148
+ original **only under `cause`**, no `status` of its own — so `e.status === 400` misses it.
173
149
 
174
- The same trap applies to any cross-copy `instanceof`; it is the runtime face of the
175
- peer-dependency identity rule below.
150
+ Use `isBadRequest` from `plugins/utils.ts`: it walks the `cause` chain for `status === 400`, bounded
151
+ in depth so a self-referential chain terminates. Both built-in `isFatal` implementations go through
152
+ it. A context overflow makes this urgent: `refine` escalates the **output** budget on each retry, so
153
+ an over-limit **input** can never improve — every attempt re-sends the identical oversized request,
154
+ and consumers hold their locks for the whole loop. The same trap applies to any cross-copy
155
+ `instanceof`; it is the runtime face of the peer-dependency rule below.
176
156
 
177
157
  ## Peer-dependency rule (langchain identity)
178
158
 
179
- `@langchain/core`, `@langchain/openai` and `@langchain/anthropic` are **peer** dependencies:
180
- model instances cross the package boundary, and two installed copies of a class with
181
- protected members are nominally distinct types. In a linked-workspace checkout the consumer
182
- must pin them to a single copy — see the `bun-linked-workspaces` skill.
183
-
184
- ## Usage
185
-
186
- ```typescript
187
- import { makeLlmModel, makeLlmService } from '@owlmeans/llm'
188
- import { ModelProvider } from '@owlmeans/llm-common'
189
-
190
- const llm = makeLlmService({ models: () => configs })
191
- const model = makeLlmModel(
192
- { model: llm.getModel('analyst'), purpose: { type: 'analysis' } }, spectator
193
- )
194
- const spec = await model.invoke('Describe the app', SpecSchema, { action: 'spec' })
195
- ```
159
+ `@langchain/core`, `@langchain/openai` and `@langchain/anthropic` are **peer** dependencies: model
160
+ instances cross the package boundary, and two installed copies of a class with protected members are
161
+ nominally distinct types. A consumer must end up with exactly ONE copy of each — declare them at one
162
+ range and check no nested `node_modules` holds a second, or every model instance crossing a boundary
163
+ becomes a foreign type and `instanceof` starts lying.
196
164
 
197
165
  ## Hangs are bounded by an IDLE deadline, not a total one
198
166
 
199
- A stalled provider is aborted after `MODEL_STREAM_TIMEOUT_MS` (3 min) of SILENCE and
200
- surfaces as a retryable `LlmModelError`, so the escalator moves on. The timer re-arms on
201
- every token, so a long-but-productive generation is never cut off — which is why the value
202
- can be low. Set it for a whole deployment with `LlmServiceOptions.streamTimeout` where the
203
- application composes its context; a preset that names its own `ModelConfig.streamTimeout`
204
- keeps it.
205
-
206
- Note what this does NOT bound: a call that keeps streaming forever, and retries. A fatal
207
- error misclassified as retryable multiplies its own latency by `DEFAULT_MODEL_RETRIES` —
208
- see the `instanceof` trap above.
167
+ A stalled provider is aborted after `MODEL_STREAM_TIMEOUT_MS` (3 min) of SILENCE and surfaces as a
168
+ retryable `LlmModelError`, so the escalator moves on. The timer re-arms on every token, so a
169
+ long-but-productive generation is never cut off — which is why the value can be low. Set it for a
170
+ deployment with `LlmServiceOptions.streamTimeout` where the application composes its context; a
171
+ preset naming its own `ModelConfig.streamTimeout` keeps it. It does NOT bound a call that keeps
172
+ streaming forever, nor retries — a fatal error misclassified as retryable multiplies its own latency
173
+ by `DEFAULT_MODEL_RETRIES`.
209
174
 
210
175
  ## An outer validation loop must pass its attempt in
211
176
 
212
- A caller that validates the OUTPUT — a diff that has to apply, a file that must not come
213
- back truncated — runs its own retry loop around whole `ask` calls. Every one of those calls
214
- starts a FRESH inner loop at attempt 0, so the escalator's two rungs never move: same model,
215
- same output budget, same deterministic answer, N times. Pass `escalation: <outer attempt>`
216
- in `LlmCallOptions` and the per-call escalator starts that far up its ladder instead —
217
- `maxTokens` doubling and the `FALLBACK_AFTER_ATTEMPTS` switch to `ModelConfig.fallback` both
218
- advance. It is clamped to `retries - 1`, moves the STARTING rung only, and never changes how
219
- many attempts the call itself makes.
220
-
221
- Two things this depends on, and both are preset data rather than code: the role must
222
- actually declare a `fallback`, and that fallback must be in the same plugin `family` (a
223
- cross-family one is skipped with a warning, because switching provider mid-call flips the
224
- structured-output shape).
225
-
226
- `LlmCallOptions.fatal` is the matching lever in the other direction — a per-call resolver
227
- consulted before the global ones and the plugin's `isFatal`, for an error the caller knows
228
- no retry can fix.
177
+ A caller that validates the OUTPUT — a diff that has to apply, a file that must not come back
178
+ truncated — runs its own retry loop around whole `ask` calls. Every one of those calls starts a FRESH
179
+ inner loop at attempt 0, so the escalator's two rungs never move: same model, same output budget,
180
+ same deterministic answer, N times. Pass `escalation: <outer attempt>` in `LlmCallOptions` and the
181
+ per-call escalator starts that far up its ladder instead — `maxTokens` doubling and the
182
+ `FALLBACK_AFTER_ATTEMPTS` switch to `ModelConfig.fallback` both advance. It is clamped to
183
+ `retries - 1`, moves the STARTING rung only, and never changes how many attempts the call makes. Two
184
+ things it depends on, both preset data rather than code: the role must declare a `fallback`, and that
185
+ fallback must be in the same plugin `family` (a cross-family one is skipped with a warning, because
186
+ switching provider mid-call flips the structured-output shape). `LlmCallOptions.fatal` is the lever
187
+ in the other direction — a per-call resolver consulted before the global ones and the plugin's
188
+ `isFatal`, for an error the caller knows no retry can fix.
189
+
190
+ ### A loop ABOVE the model asks the same question with `isFatalError`
191
+
192
+ A retry loop is not the only place that decides to carry on: a fix ladder rescues a failed repair and
193
+ climbs to a stronger model, an agent runner catches a round that threw and reports "gave up". Both
194
+ are right for a model that answered badly and wrong for a budget that ran out, and a blanket `catch`
195
+ cannot tell them apart — an exhausted balance becomes more expensive calls instead of a halt.
196
+ `isFatalError(e, fatal?)` runs the same resolvers, in the same order, that `withRetry` uses, and
197
+ returns the error to abort WITH (a resolver may unwrap a carrier and hand back the real cause) or
198
+ `null` when nothing considers it terminal. Ask it rather than re-deriving the rule.
229
199
 
230
200
  ## Output caps: what the deployment wants vs what the provider allows
231
201
 
232
- Four fields, and conflating them is what turns an escalation into a fatal 400 hours into a
233
- run:
202
+ Four fields, and conflating them turns an escalation into a fatal 400 hours into a run:
234
203
 
235
204
  | Field | Means |
236
205
  |---|---|
@@ -239,55 +208,94 @@ run:
239
208
  | `maxOutput` | what the PROVIDER accepts in one request — a fact about the model |
240
209
  | `contextWindow` | total window (input + output); informational, never sent |
241
210
 
242
- `resolveOutputCap` (`utils/config.ts`) is the one place that reconciles them: the declared
243
- cap chooses the ceiling and the capability trims it, and `DEFAULT_MAX_OUTPUT_CAP` applies
244
- only when neither is stated. `createModel` additionally clamps `maxTokens` to `maxOutput`
245
- and warns about a cap above it. For an aggregated model `maxOutput` is the limit of the
246
- `inferenceProvider` actually pinned, which is often far below what the model can do
247
- elsewhere. `combinedWindow: true` marks a model whose window is shared between input and
248
- output (MiniMax M2.x, gpt-oss) — nothing enforces it at runtime; it keeps presets honest
249
- about leaving room for the prompt.
250
-
251
- **A `fallback` that changes `model` must restate `contextWindow`/`maxOutput`** (and reset
252
- `combinedWindow`): the fallback config is `{...primaryConfig, ...fallback}`, so every field
253
- the patch does not name is inherited from a different model.
211
+ `resolveOutputCap` (`utils/config.ts`) reconciles them: the declared cap chooses the ceiling and the
212
+ capability trims it, and `DEFAULT_MAX_OUTPUT_CAP` applies only when neither is stated. `createModel`
213
+ also clamps `maxTokens` to `maxOutput` and warns about a cap above it. For an aggregated model
214
+ `maxOutput` is the limit of the `inferenceProvider` actually pinned, often far below what the model
215
+ can do elsewhere. `combinedWindow: true` marks a model whose window is shared between input and
216
+ output (MiniMax M2.x, gpt-oss) — nothing enforces it at runtime; it keeps presets honest about
217
+ leaving room for the prompt. **A `fallback` that changes `model` must restate
218
+ `contextWindow`/`maxOutput`** (and reset `combinedWindow`): the fallback config is
219
+ `{...primaryConfig, ...fallback}`, so every field the patch does not name is inherited from a
220
+ different model.
221
+
222
+ ### Reasoning is off unless a preset asks for it — and it is billed against the same budget
223
+
224
+ The models `NO_SAMPLING_PREFIXES` names think ADAPTIVELY unless the request says otherwise: an absent
225
+ `thinking` parameter means adaptive, and `@langchain/anthropic` forwards the parameter only when the
226
+ caller sets it (its own field default of `disabled` is never sent). Left on, the reasoning costs
227
+ twice — it is spent from `max_tokens`, the same allowance as the answer (a budget sized for the
228
+ answer alone goes entirely on reasoning and the completion arrives well-formed, `stop_reason:
229
+ "max_tokens"`, with no text block), and its SUMMARISED stream arrives in bursts minutes apart, which
230
+ the idle deadline reads as a dead connection and retries from scratch
231
+ (`llm:model:stream-stalled:no token for 180000ms (idle deadline)`).
232
+
233
+ **`ModelConfig.disableThinking` is the switch, and where it lands depends on the plugin.**
234
+ `makeLlmModel` appends the literal `/no_think` to every request's prepared messages whenever the flag
235
+ is set AND the plugin's `suppressesThinking(config)` does not answer `true` — the soft switch for
236
+ models with no request-level control (Qwen3). The Anthropic plugin answers `true` only for
237
+ `rejectsSampling(model)`, and for those puts `thinking: { type: 'disabled' }` on the request in
238
+ `build`, which `refine` carries through `lc_kwargs` on every attempt. Below that line
239
+ (`claude-haiku-4-5`, `claude-sonnet-4-6`) and under any plugin declaring no hook the flag injects
240
+ prompt text instead, so set it where the wire honours it. **A preset must set the flag on every
241
+ adaptive Anthropic role**; it reaches the `fallback` rung only by inheritance from the entry
242
+ (viable-agent's `presets.test.ts` pins this). Turning reasoning ON is a per-role decision.
243
+
244
+ `ADAPTIVE_MIN_MAX_TOKENS` (32k) is the output floor the Anthropic plugin's `build` applies to every
245
+ `rejectsSampling(model)` config — `disableThinking` is not consulted, so a role with reasoning turned
246
+ off is floored just the same. It is a floor, not an override (a preset asking for more keeps it) and
247
+ it is clamped through `resolveOutputCap`, so it can never exceed what the provider accepts and turn a
248
+ retryable empty answer into a fatal 400. Raising or removing it re-opens empty completions.
249
+
250
+ Two diagnostics the above depends on:
251
+
252
+ - **An empty completion is a null result, not a filter rejection.** `ask` tests emptiness BEFORE the
253
+ caller's filter — every shipped filter returns null only for empty input, so a filter run first
254
+ blames the caller for a provider problem and skips `reportNull`, losing the `finishReason` and
255
+ `outputTokens` that name the cause.
256
+ - **Anthropic's stop reason is not `finish_reason`.** langchain puts it in
257
+ `additional_kwargs.stop_reason` and `response_metadata.finish_reason` does not exist there, so
258
+ `null-report.ts` reads both; `thinkingOnly` marks a completion that was all reasoning.
259
+
260
+ OpenAI reasoning models get it handled by shrinking the reasoning cap on retry (`plugins/openai.ts`);
261
+ escalating `maxTokens` alone fixes neither family — the retry draws from an unchanged distribution.
254
262
 
255
263
  ## Config precedence: a preset is a base, not a final word
256
264
 
257
- `createModel` layers four sources, lowest first:
258
-
259
- presetOf(base.preset) < base < presetOf(override.preset) < override
265
+ `createModel` layers four sources, lowest first: `presetOf(base.preset)` < `base` < `presetOf(override.preset)` < `override`.
260
266
 
261
- A `preset` is a BASE that its referent refines, so it sits UNDER the config naming it.
262
- Assigning it last — as this did until the layering was fixed — meant a role declaring
263
- `preset:` silently discarded both its own fields and the caller's override, which is how
264
- effort-tier token caps and `temperatureFactory`'s temperature vanished for preset-based
265
- roles. An override naming a preset (how the execution layer delivers a `modelOverrides`
266
- string pin) outranks the alias but still yields to explicit override fields. Resolution is
267
- ONE level deep: a preset meant to carry a model must name one.
267
+ A `preset` is a BASE that its referent refines, so it sits UNDER the config naming it. Assign it last
268
+ and a role declaring `preset:` silently discards both its own fields and the caller's override — that
269
+ is how effort-tier token caps and `temperatureFactory`'s temperature disappear for preset-based
270
+ roles. An override naming a preset (how the execution layer delivers a `modelOverrides` string pin)
271
+ outranks the alias but yields to explicit override fields; resolution is ONE level deep, so a preset
272
+ meant to carry a model must name one.
268
273
 
269
274
  ## Resilience already handled — do not reimplement
270
275
 
271
- Idle stream deadline · duplicate-final-chunk dedup · output-budget escalation · reasoning-cap
272
- shrink · same-family fallback model · caller-seeded ladder position (`escalation`) · schema
273
- coercion · JSON salvage from prose · `NullCapture`
274
- diagnostics · fatal-error short-circuit · blank-content sanitization (whitespace-only text
275
- blocks are dropped before every call — a blank block, e.g. an empty file read pasted into a
276
- prompt, is otherwise a fatal Anthropic 400; blank tool results are stubbed to keep their
277
- `tool_use` pairing). Details: package `README.md`.
276
+ Idle stream deadline · duplicate-final-chunk dedup · output-budget escalation · reasoning-cap shrink ·
277
+ adaptive-thinking budget floor · same-family fallback model · caller-seeded ladder position
278
+ (`escalation`) · schema coercion · JSON salvage from prose · `NullCapture` diagnostics · fatal-error
279
+ short-circuit · blank-content sanitization (whitespace-only text blocks are dropped before every call
280
+ — a blank block, e.g. an empty file read pasted into a prompt, is otherwise a fatal Anthropic 400;
281
+ blank tool results are stubbed to keep their `tool_use` pairing). Details: package `README.md`.
278
282
 
279
283
  ## Tests
280
284
 
281
- `bun test ./tests` in the package. Offline specs always run; `tests/model.spec.ts` is gated
282
- on `OPENROUTER_SECRET` / `ANTHROPIC_SECRET` in the repo-root `.env` and self-skips otherwise.
285
+ `bun test ./tests` in the package; offline specs always run. In `tests/model.spec.ts` the anthropic live
286
+ suite is gated on `ANTHROPIC_SECRET` and self-skips with a printed reason without it, and the OpenRouter
287
+ suite is disabled unconditionally: an aggregator on a separate account serving models no deployment runs,
288
+ whose `402 requires more credits` reads as a failure of the code under test. `plugins.spec.ts` covers the
289
+ `Compatible` provider offline.
283
290
 
284
291
  ## Depends On
285
292
 
286
293
  - `@owlmeans/llm-common` · `@owlmeans/context` · `@owlmeans/error` · `@owlmeans/basic-ids` · `ajv`
294
+ - `@anthropic-ai/sdk` — runtime, for the `BadRequestError` the fatal-error rules are written around
287
295
  - peer `@langchain/core`, `@langchain/openai`, `@langchain/anthropic`
288
296
 
289
297
  ## Related
290
298
 
291
- - [[llm-common]] — the serializable contracts
292
- - [[llm-prompt-caching]] — prompt composition, block order and the cache invariants
299
+ - [[llm-common]] — the serializable contracts · [[llm-prompt-caching]] — prompt composition, block
300
+ order and the cache invariants
293
301
  - [[context]] — service registration · [[error]] — the `ResilientError` family