@ultimat3/ai 27.9.0 → 27.10.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CLAUDE.md CHANGED
@@ -9,18 +9,13 @@ does), `jobs` (`agentJob()`), `money`, `policy`, `schema`, `time`.
9
9
 
10
10
  `jobs` is declared for `agentJob()`: the action→job bridge has to live at tier 4+.
11
11
 
12
- `mcp` is the same tier, so the LLM-tool projection is restated structurally in `tools.ts`
13
- rather than imported. Same contract, two wire formats — and the same *decision*: `toLlmTools` and
14
- `runLlmToolCall` ask `isMcpExposed` from `@ultimat3/core` (tier 0, reachable by both), never a
15
- local `=== true`. An in-app agent and an external one must be offered exactly the same tools.
16
-
17
- **And the same NAME**: `toLlmTool` passes `action.name` through untouched (the export name
18
- `@ultimat3/mcp` serves); `llm()` and `agent()` tool names are verbatim too. Never derive one.
19
-
20
- **And the same TOOL**: `asProjectableAction` and mcp's `primitiveFromAction` are two copies the
21
- tier table forces; `scripts/ai-mcp-tool-parity.test.ts` holds them equal — name, description,
22
- exposure, the whole input schema (an idempotent action's reserved `idempotencyKey` argument,
23
- `tool-idempotency.ts`), the policy verdict and the replay. A field one adds, the other adds.
12
+ `mcp` is the same tier, so the LLM-tool projection is restated structurally in `tools.ts`, never
13
+ imported: same contract, same *decision* (`isMcpExposed` from `@ultimat3/core`, never a local
14
+ `=== true`), same NAME (`action.name` verbatim, never derived) and the same TOOL —
15
+ `asProjectableAction` and mcp's `primitiveFromAction` are two copies the tier table forces, held
16
+ equal by `scripts/ai-mcp-tool-parity.test.ts` (name, description, exposure, the whole input schema
17
+ incl. `tool-idempotency.ts`'s reserved `idempotencyKey`, the policy verdict, the replay). A field
18
+ one adds, the other adds.
24
19
 
25
20
  ## Owns
26
21
 
@@ -29,7 +24,7 @@ exposure, the whole input schema (an idempotent action's reserved `idempotencyKe
29
24
  | `models.ts` | the model REGISTRY: `registerModel`, limits, prices, the reasoning controls each one accepts — and NO row of its own |
30
25
  | `model-resolve.ts` | `resolveModel` / `modelForCall` — declaration → prompt → gateway `defaultModel`, none is `X_AI_MODEL_UNRESOLVED` naming all three; `describedModel` for a published fact |
31
26
  | `model-fixture.ts` | the rows this package's own suites register (`useFixtureModels()`). Not shipped |
32
- | `provider.ts` | `Provider` interface, the request half, the money arithmetic, `anthropicProvider()` |
27
+ | `provider.ts` | `Provider` interface, the request half, `anthropicProvider()` |
33
28
  | `echo-provider.ts` | `echoProvider()`, the deterministic double — split from `provider.ts` at its line ceiling |
34
29
  | `content-blocks.ts` | the `image` / `document` block shapes, the screen every request runs them through, their token estimate |
35
30
  | `content-errors.ts` | the `X_AI_CONTENT_UNSUPPORTED` class; its code and title belong in `errors.ts` |
@@ -57,17 +52,21 @@ exposure, the whole input schema (an idempotent action's reserved `idempotencyKe
57
52
  | `rag.ts` | chunker, retriever, reranker, budgeted context assembler |
58
53
  | `tools.ts` | action → LLM tool definition; the `AgentTool` union and `asProjectableAction`; `runLlmToolCall` |
59
54
  | `tool-idempotency.ts` | an idempotent action's reserved `idempotencyKey` tool argument — MCP's, restated (same tier), held equal by the parity test |
60
- | `failure-disclosure.ts` | `discloseFailure` — the ONE rule for what of a throw a remote reader sees (tool result, hive member); the withheld half is logged as code, name and stack frames — never its message or cause |
55
+ | `tool-arguments.ts` / `answer-errors.ts` | one argument-string reader for both wires and what an unreadable one means (`assertToolArguments`); `X_LLM_TRUNCATED` + `X_LLM_TOOL_ARGUMENTS_INVALID` |
56
+ | `billed.ts` / `run-meter.ts` | `meta.billed` + `billedOf`; the meter every `llm()`/`agent()` provider call goes through |
57
+ | `failure-disclosure.ts` | `discloseFailure` — the ONE rule for what of a throw a remote reader sees (tool result, hive member); the withheld half is logged, never its message or cause |
61
58
  | `llm.ts` | `llm()` — the model call, declared as an `action`; and what a streamed answer must satisfy |
62
59
  | `respond.ts` | the `respond` tool for one `output` (a non-object one wrapped in `{ value }`), its reader, and `parseJsonish` |
63
60
  | `llm-stream.ts` | `.stream()`'s plumbing: the sink, the ambient mark, the one-turn drive |
64
61
  | `agent.ts` | `agent()` — the tool loop, declared as an `action` |
65
- | `agent-def.ts` / `agent-system.ts` / `agent-usage.ts` | the declaration's types; the system one run sends; its running usage |
62
+ | `agent-def.ts` / `agent-declared.ts` / `agent-system.ts` / `agent-usage.ts` | the declaration's types; its defaults, bound screen, limits and fact row; the system one run sends; its running usage |
63
+ | `llm-def.ts` / `cost.ts` | `llm()`'s declaration types and `LlmCall`; the money arithmetic (`costOf`, `exactCostOf`) |
66
64
  | `agent-guards.ts` / `agent-errors.ts` | the repeated-turn detector and the run deadline; their two classes |
67
65
  | `prompt-cache.ts` | PROVIDER prompt caching: `SystemParts`, `PromptCacheHint`, `PROMPT_CACHE_RULES` (sources in its header) |
68
66
  | `prompt-cache-blocks.ts` | the Anthropic wire's marks (chat-completions': `openai-messages.ts`) |
67
+ | `agent-continue.ts` / `agent-turn.ts` | `.run()` / `.continue()`, the transcript handle and its screen; one turn's report and refusals |
69
68
  | `agent-transcript.ts` | what one turn leaves in the transcript: the assistant replay, the tool results, the correction |
70
- | `agent-facts.ts` | `describeAgents()` — the agent registry, and the row a manifest WOULD publish: nothing reads it yet, because `manifest` is tier 4 like this package and the consumer has to be `cli` at tier 5 |
69
+ | `agent-facts.ts` | `describeAgents()` — the agent registry, and the row a manifest WOULD publish (its consumer has to be `cli`, tier 5) |
71
70
  | `agent-job.ts` | `agentJob()` — an agent as a real `JobHandle`, composed from `job()` |
72
71
  | `hive.ts` | `hive()` — one action fanned out over many inputs, declared as an `action` |
73
72
  | `hive-result.ts` | `HiveMember` / `HiveResult`, and the SCHEMA built from the member's own `output` |
@@ -81,65 +80,54 @@ exposure, the whole input schema (an idempotent action's reserved `idempotencyKe
81
80
 
82
81
  ## Invariants — bounds and budget
83
82
 
83
+ Why each is shaped this way: [`docs/history/ai.md`](../../docs/history/ai.md#bounds-and-budget).
84
+
84
85
  - **Every numeric option is screened; `??` is not the screen** (`finiteOption`/`finiteCount`;
85
- `bun run finite-bounds` is the ratchet, blind to a required option, a default parameter and a
86
- cross-field product). **`maxTokens` is the worst**: it IS the budget's pre-flight estimate, and a
87
- `NaN` switches every ceiling off — the gateway, `registerModel` (`maxOutput`) and `llm()`/`agent()`
88
- (at DECLARATION) each screen theirs.
89
- - **A bound is screened under the key the DECLARATION uses** (`budget.tokensPerRun` in `limitsOf`
90
- before the ledger's `request`; `retrieve()`'s own `k`; `remoteEmbedder()`'s `maxResponseBytes`).
91
- - A per-call budget `derive`s from the ambient ledger and only TIGHTENS. **A derived ledger reports
92
- back up the chain** (every debit on every ancestor; `reserve` checks each `request` scope); the
93
- STORE is written once, by the ledger the call was made on.
94
- - **`reserve` debits, then checks — never a read-then-write across an `await`.** The store's scopes
95
- go through `BudgetStore.take(key, tokens, limit)`: REQUIRED, atomic (never `spent` + `add`). A
96
- refusal at a later scope gives back what the earlier ones took.
97
- - **The store is touched only for a scope whose ceiling is DECLARED** (`takeScope`, `meteredKeys`).
98
- `memoryBudgetStore()` has no window and no eviction (a ceiling bypass); a zero counter is deleted.
86
+ `bun run finite-bounds`). **`maxTokens` is the worst**: it IS the pre-flight estimate, and a `NaN`
87
+ switches every ceiling off — screened by the gateway, `registerModel` and `llm()`/`agent()`.
88
+ - **A bound is screened under the key the DECLARATION uses** (`budget.tokensPerRun`, not `request`).
89
+ - A per-call budget `derive`s from the ambient ledger and only TIGHTENS; a derived ledger reports
90
+ back up the chain; the STORE is written once, by the ledger the call was made on.
91
+ - **`reserve` debits, then checks — never a read-then-write across an `await`**:
92
+ `BudgetStore.take(key, tokens, limit)` is REQUIRED and atomic; a later refusal gives back.
93
+ - **The store is touched only for a scope whose ceiling is DECLARED**; `memoryBudgetStore()` has no
94
+ window and no eviction; a zero counter is deleted.
99
95
  - **`llm()`, `agent()`, `hive()` root as `currentBudget() ?? gateway.callLedger(budgetKeysFor(ctx.actor))`**
100
- — the gateway's ceilings, keyed `actor:<kind>:<id>` / `org:<orgId>` (none for an empty org).
101
- `callLedger` is required on `Gateway`; `hive()` asks `installedGateway()`.
102
- - **`BudgetStore` is where `actor` and `org` live; the default is per PROCESS**
103
- (`providerGateway({ budgetStore })`). `add` takes a negative `tokens` for a release — a store that
104
- clamps at zero leaks the ceiling.
105
- - Cost is `Money` (integer minor units), rounded **up**. A budget throws `X_AI_BUDGET_EXCEEDED`
106
- **before** the provider call. Never truncate. List price over-reserves (no introductory prices).
107
- - **`Gateway.stream` routes BEFORE it reserves**, and `settled = true` comes after
108
- `await ledger.record(...)`.
96
+ (`actor:<kind>:<id>` / `org:<orgId>`). `callLedger` is required on `Gateway`.
97
+ - **`BudgetStore` is where `actor` and `org` live; the default is per PROCESS**; `add` takes a
98
+ negative `tokens` for a release — a store that clamps at zero leaks the ceiling.
99
+ - A budget throws `X_AI_BUDGET_EXCEEDED` **before** the provider call. Never truncate. List price
100
+ over-reserves.
101
+ - **Cost is exact in bigint (`cost.ts`, `Money` at `costScale`), rounded UP once**: per call by
102
+ `costOf`, per ledger at `report()`, per run from the summed usage — never a sum of round-ups. A
103
+ price may state a `scale` (≤ `PRICE_SCALE_DIGITS` finer); it is always read through `minorAt`.
104
+ - **`Gateway.stream` routes BEFORE it reserves**; `settled = true` comes after `ledger.record`.
109
105
 
110
106
  ## Invariants — `llm()`
111
107
 
108
+ Why each is shaped this way: [`docs/history/ai.md`](../../docs/history/ai.md#llm-invariants).
109
+
112
110
  - **`llm()` returns an `action`**, never a ninth primitive; it adds only the model half: render the
113
- prompt, project `output` into the one `respond` tool, reserve the budget, consult the semantic cache.
114
- - **A non-object `output` is wrapped in `{ value }` and unwrapped** (`respond.ts`) — a tool's input is
115
- an object; never refused, because `.stream()` serves `output: t.string`. `parseJsonish` is two
116
- `indexOf` scans, never a backtracking pattern.
111
+ prompt, project `output` into the one `respond` tool, reserve the budget, consult the cache.
112
+ - **A non-object `output` is wrapped in `{ value }` and unwrapped** (`respond.ts`), never refused;
113
+ `parseJsonish` is two `indexOf` scans, never a backtracking pattern.
117
114
  - `src/index.ts` re-exports `t` from `@ultimat3/schema` **verbatim** (`index.test.ts`).
118
115
  - One repair turn, then `X_LLM_OUTPUT_INVALID`. A refusal (a 200 with no answer) is `X_LLM_REFUSED`
119
- before the parse, carrying `stopDetails.category`; a truncated invalid answer is `X_LLM_TRUNCATED`.
120
- The gateway never caches a refusal.
121
- - **Semantic scopes are separate cache INSTANCES**; the instance key carries the prompt hash AND
122
- `ctx.locale` (in the unconditional half). **The default scope is the calling ACTOR**
123
- (`JSON.stringify([actor.kind, actor.id, actor.orgId ?? null])`); `scope` receives
124
- `{ input, ctx }`; widening is `scope: () => 'global'`. The instance map is bounded
125
- (core's `MAX_CACHED_FORMATTERS`).
126
- - `cache.invalidates` is not on `llm()`: a `SemanticCache` is not a `CacheTier`. Version bump + `ttl`
127
- is the invalidation.
128
- - The gateway is ambient (`configureAi`); absent at call time is `X_AI_GATEWAY_MISSING`, never a
129
- default provider.
116
+ before the parse; a truncated turn is `X_LLM_TRUNCATED` before the schema AND before any tool
117
+ runs (a cut `respond` reads `{}`). The gateway never caches a refusal.
118
+ - **Semantic scopes are separate cache INSTANCES** (key: prompt hash AND `ctx.locale`); **the
119
+ default scope is the calling ACTOR**; widening is `scope: () => 'global'`; the map is bounded.
120
+ `cache.invalidates` is not on `llm()`: version bump + `ttl` is the invalidation.
121
+ - The gateway is ambient (`configureAi`); absent at call time is `X_AI_GATEWAY_MISSING`.
130
122
  - **`llm()` forwards `ctx.signal`** onto `GenerateRequest`; `.stream()` inherits it.
131
123
  - **`.stream()` is the SAME action**, marked with an ambient sink: UNVALIDATED text, then one `done`
132
- with the validated value; no repair turn (`X_LLM_STREAM_INVALID`), no `respond` tool; budget
133
- reserved before the provider, reconciled at `done`. LAZY.
124
+ with the validated value; no repair turn (`X_LLM_STREAM_INVALID`), no `respond` tool. LAZY.
134
125
  - **`configureAi({ redact })` is the one seam between `vars()` and the provider**; a `Secret` in
135
- `vars()` is `X_AI_PROMPT_SECRET` regardless. The `llm.redacted` span attribute records it.
126
+ `vars()` is `X_AI_PROMPT_SECRET` regardless.
136
127
  - **Fallback is across PROVIDERS serving one model, never across models**; `GenerateResult.provider`
137
- is stamped by the gateway and put on the span as `llm.provider`.
138
- - Server-side `fallbacks` (beta) are never sent — the stable `2023-06-01` surface only.
139
- - **A slot inside a tag pair is DATA** (`prompt-fence.ts`, #689): `render` breaks, in the ASSEMBLED
140
- prompt, every fence closer a fenced value wrote any character of (never per value: two slots can
141
- each hold half) — never deletes, never touches a template-only closer. The
142
- fences are read once at `definePrompt`; no hash moves.
128
+ is stamped by the gateway. Server-side `fallbacks` (beta) are never sent.
129
+ - **A slot inside a tag pair is DATA** (`prompt-fence.ts`, #689): `render` breaks every fence closer
130
+ a fenced value wrote, in the ASSEMBLED prompt — never deletes, never touches a template's own.
143
131
  - `definePrompt` refuses a re-registered version whose hash moved; the hash is over core's
144
132
  `canonicalJson`; an absent schema hashes as `''`.
145
133
 
@@ -147,7 +135,8 @@ exposure, the whole input schema (an idempotent action's reserved `idempotencyKe
147
135
 
148
136
  - **The response cache key is core's `fingerprint` over the request, each tool WHOLE** (every `llm()`
149
137
  tool is `respond`); a throwing `cache.set` is logged (`ai.cache.write_failed`), never a failure.
150
- - **An unknown stop/finish reason FAILS CLOSED as `max_tokens` on both formats**, streamed or not.
138
+ - **An unknown stop reason FAILS CLOSED as `max_tokens` on the Anthropic wire; on chat-completions
139
+ it is UNKNOWN** (`readFinishReason`): a whole turn is a stop, else `X_AI_PROVIDER_UNEXPECTED`.
151
140
  - **One failure-body reader** (`error-body.ts`'s `detailOf`, bounded at `READ_LIMIT`) and one
152
141
  `withoutKey` scrub — the embedder included.
153
142
  - **`attempt` collects TRANSPORT failures only**: an `UltimateError` other than
@@ -156,33 +145,37 @@ exposure, the whole input schema (an idempotent action's reserved `idempotencyKe
156
145
  - **The gateway's reads of a provider's throw are total**: `isRetryable` fails closed; the line goes
157
146
  through `renderThrowable`.
158
147
  - **`X_AI_PROVIDER_UNAVAILABLE` is the one code classified `retryable`** (`AI_ERROR_RETRY`);
159
- `AiProviderUnavailableError({ unserved: true })` carries a per-instance `terminal`. Every other
160
- code but the two agent stops stays UNREGISTERED on purpose.
148
+ `AiProviderUnavailableError({ unserved: true })` carries a per-instance `terminal`. Six
149
+ deterministic failures are `terminal` (`errors-retry.ts`); the rest stay UNREGISTERED on purpose.
161
150
  - **The retry SCHEDULE is core's (`backoffDelay`, `isRetryableStatus`); the LOOP is this package's** —
162
151
  core's `retry()` retries unclassified throws, so it would retry a 400 (`gateway-backoff.test.ts`).
163
152
  - Every non-2xx and every in-band `error` frame is `AiTransportError` with a real `status`.
164
153
  - **Both wire formats answer the same question the same way** (`provider-parity.test.ts`): an in-band
165
154
  `error` in a 200 throws (`throwInBandError`); a `refusal` detail forces `stopReason: 'refusal'`; a
166
- tool call's `input` is parsed (`asToolInput`); the credential is scrubbed from `detail`.
155
+ tool call's `input` is parsed (`parseToolArguments`); the credential is scrubbed from `detail`.
156
+ - **Arguments that are not JSON are never thrown by a wire reader**: recorded as `malformed`, and
157
+ the PROVIDER decides once usage is known — dropped on a truncated turn, else
158
+ `X_LLM_TOOL_ARGUMENTS_INVALID` with `meta.billed`, which the gateway RECORDS (`settleFailure`).
159
+ - **Every throw of `llm()`/`agent()` after a provider call STARTED carries `meta.billed`**
160
+ (`meter.fail`): answered calls summed, `unsettled` counted and never priced.
167
161
  - A stream ending without `message_stop` throws. **`readSse` caps the unterminated buffer**
168
162
  (`MAX_FRAME_CHARS`); `provider` is required so a transport error names its endpoint.
169
163
  - A tool call is emitted whole. Thinking chunks are never appended to `text`.
170
- - Anthropic body: no `temperature`/`top_p`/`top_k`, no `budget_tokens`, `effort` inside
171
- `output_config`. Model IDs are exact aliases, never date-suffixed.
172
- - **Apps bring their own models and providers (M12, 25.0.0): no vendor choice, no vendor data.** No
173
- default model, no catalogue row, no id list ships; the registry starts EMPTY and `resetModels()`
174
- empties it. A model resolves declaration → prompt → `Gateway.defaultModel` through `resolveModel`
175
- / `modelForCall` only; none is `X_AI_MODEL_UNRESOLVED` naming all three (with no gateway and nothing
176
- declared, `X_AI_GATEWAY_MISSING` — the third place does not exist yet). `vendor-neutral.test.ts`
177
- proves it in a fresh process; no fix line names a vendor id.
164
+ - Anthropic body: no `top_p`/`top_k`, no `budget_tokens`, `effort` inside `output_config`. Model
165
+ IDs are exact aliases, never date-suffixed.
166
+ - **`temperature`/`seed` (`sampling.ts`) are sent ONLY to a row with `sampling: { … : true }`** —
167
+ else left out (never refused; `llm.sampling.ignored`); no `seed` on the Anthropic wire; in
168
+ `promptCacheKey` only when set.
169
+ - **Apps bring their own models and providers (M12, 25.0.0): no vendor choice, no vendor data.** The
170
+ registry starts EMPTY; a model resolves declaration → prompt → `Gateway.defaultModel`
171
+ (`resolveModel` / `modelForCall`), none is `X_AI_MODEL_UNRESOLVED` (no gateway at all:
172
+ `X_AI_GATEWAY_MISSING`). `vendor-neutral.test.ts` proves it; no fix line names a vendor id.
178
173
  - **`ModelId` is `string`; the catalogue is an open registry** — `modelSpec(id)` refuses an
179
- unregistered id (`X_AI_MODEL_UNKNOWN`, fix `registerModel(…)`) at `costOf`, the budget estimate and
180
- the body builders, before a socket or a reservation. **Re-registering REPLACES the spec and keeps
181
- its rung** (negotiated rates). **Registration order is the ladder within a `family`**, read only by
182
- `moreCapableThan`, over the APP's rows; `X_LLM_REFUSED`'s fix names a rung ABOVE or nothing.
183
- - **Every provider serves the app's list**: `anthropicProvider({ models })` / `openAiProvider({ models })`
184
- require a non-empty one (`X_AI_REQUEST_INVALID`); `echoProvider().models` reads the registry. **No
185
- provider has a model of its own**: a direct call naming none is `X_AI_MODEL_UNRESOLVED` from all three.
174
+ unregistered id (`X_AI_MODEL_UNKNOWN`) before a socket or a reservation. Re-registering REPLACES
175
+ the spec and keeps its rung; registration order is the ladder within a `family`
176
+ (`moreCapableThan`); `X_LLM_REFUSED`'s fix names a rung ABOVE or nothing.
177
+ - **Every provider serves the app's list** (`{ models }` required, non-empty); no provider has a
178
+ model of its own.
186
179
  - **The reasoning controls are PER MODEL (`models.ts`)**: an unasked control is omitted; an asked
187
180
  control the model lacks is `X_AI_REQUEST_INVALID`, never dropped. Adding a model is a row the app
188
181
  registers. `disableThinkingUpTo: 'never'` refuses `thinking: 'disabled'` at every effort;
@@ -197,12 +190,13 @@ exposure, the whole input schema (an idempotent action's reserved `idempotencyKe
197
190
  - **`openAiProvider()` is a FORMAT, not a vendor** — `baseUrl`, `auth`, `headers` are the
198
191
  differences; never a second class per vendor:
199
192
  - structured output is the `respond` tool, never `response_format`; `tool_choice` names the tool
200
- only when the request offers exactly one;
193
+ only when the request offers exactly one and the row is not `forcedToolChoice: false`;
201
194
  - `strict: true` only when `satisfiesStrictMode` says the schema qualifies;
202
195
  - `max_completion_tokens`; `role: 'system'`; `stream_options: { include_usage: true }` and an
203
196
  ESTIMATE when usage never arrives; subtract `cached_tokens` from `prompt_tokens`;
204
- - tool-call deltas merge by `tool_calls[].index`; `isComplete()` accepts `[DONE]` or a finish reason,
205
- except `[DONE]` with pending tool fragments, which is refused;
197
+ - tool-call deltas merge by `tool_calls[].index`, a differing `id` starting a new call; calls are
198
+ read at `end()` (`[DONE]` or socket close), never at the finish reason; `isComplete()` accepts
199
+ `[DONE]` or a finish reason, except `[DONE]` with pending tool fragments, which is refused;
206
200
  - no code of its own (a media refusal is the shared `X_AI_CONTENT_UNSUPPORTED`);
207
201
  `AiTransportError` takes the provider's `envVar`; the key is revealed late,
208
202
  never stored, scrubbed from `detail`.
@@ -223,7 +217,9 @@ exposure, the whole input schema (an idempotent action's reserved `idempotencyKe
223
217
  - `tools` takes a real `action()` (`AgentTool = AnyAction | ProjectableAction`,
224
218
  `asProjectableAction`); projection happens on the FIRST RUN, memoised. An agent can be another
225
219
  agent's tool.
226
- - A tool not `mcp: { expose: true }` is `X_AGENT_TOOL_UNEXPOSED` at DECLARATION.
220
+ - A tool is an exposed action or an `inProcessTool()` (`isOfferedTool`); anything else is
221
+ `X_AGENT_TOOL_UNEXPOSED` at DECLARATION. An in-process tool is never an action, never projected
222
+ to MCP, has no policy (the agent's decides) and parses its own input.
227
223
  - `ctx.signal` is read at the top of every turn, before every tool batch, and forwarded to
228
224
  `fetch`; `X_ABORTED` is core's.
229
225
  - One turn's tools run concurrently, results paired by index; no tool STARTS after an abort.
@@ -237,7 +233,14 @@ exposure, the whole input schema (an idempotent action's reserved `idempotencyKe
237
233
  - **A 5xx code's cause reaches the model only when core's `hasPublicCause` allows it** (status from
238
234
  `@ultimat3/http`'s `statusFor`) — else `CODE: HIDDEN_TOOL_CAUSE`, plus a branded `callerFix`. The
239
235
  same verdict a problem document and `@ultimat3/mcp` give, from `failure-disclosure.ts` — the
240
- rule a hive member's `reason` follows too. Never a second copy.
236
+ rule a hive member's `reason` follows too. Never a second copy. One widening, for an
237
+ `inProcess` tool only (`discloseOwnFailure`): a branded error under a code with NO declared
238
+ status is shown as `{ error: { code, cause, fix } }`; `discloseErrors: false` opts out.
239
+ - **`.run()`/`.continue()` are the SAME action**, marked by an ambient session only the declaring
240
+ agent may claim, once (a nested agent starts fresh). The handle is SIGNED (`sealTranscript`: the
241
+ cursor secret, own label) and bound to agent, model, tool names and `orgId`; screened before
242
+ anything is loaded. `maxTurns`/`tokensPerRun` are cumulative — the remainder is the ceiling,
243
+ NEVER a ledger debit. A failure seals an updated handle onto the error (`transcriptOf`).
241
244
  - **`hive()` is a fan-out action** (`PRIMITIVE_FACTORIES`): `HiveResult` is a SCHEMA from the
242
245
  member's `output`; three arms (`ok`/`failed`/`skipped`) and three counters; `members` in SPLIT
243
246
  order with `index`; the hive never names an actor; no hive budget code; a member's throw goes
@@ -270,27 +273,20 @@ exposure, the whole input schema (an idempotent action's reserved `idempotencyKe
270
273
 
271
274
  ## Invariants — evals, retrieval, fix lines
272
275
 
273
- - **No fix line may name `x ai`** (planned). An eval is selected with `x test eval --filter <name>`;
274
- `eval-errors.test.ts` asserts every `x test <word>` is one of the six types.
275
- - Every eval result carries the prompt hash. An eval gates on the DROP from its recorded baseline,
276
- mean AND per case, both sides rounded as the baseline is written (`X_EVAL_BASELINE_MISSING` /
277
- `X_EVAL_BASELINE_INVALID`). `baseline` is `import.meta.resolve('./…')`. Every prompt is named by an
278
- eval (`X_EVAL_MISSING`). `ULTIMATE_EVAL_RECORD=1` records; a deliberately-worse test calls `run`,
279
- never `assert`; `x verify` with the flag is `X_EVAL_RECORDING` and runs no eval.
280
- - Retrieval is hybrid by default; no vector-only path. **`chunk()` hard-wraps**, the overlap
281
- carry stops at `buffer.length - 1` and yields to a next unit it does not fit beside (no chunk over
282
- `size`); `metadata.source` is written AFTER the caller's spread. `VectorStore.prune` is required.
283
- - **A store with no tenant bound, read inside a request whose actor has an `orgId`, is
284
- `X_VECTOR_UNSCOPED`** (`assertTenantRead`, both stores, every read). `scope: UNSCOPED`
285
- (`crossTenant: true`) is the named opt-in; an omitted scope is `UNBOUND`.
286
- - `postgresVectorStore()` is the only production vector path (pgvector + FTS in the app's Postgres);
287
- `memoryVectorStore()` enforces the same envelope. **`pg-vector.live.test.ts` refuses to skip** on a
288
- Postgres without the extension (CI uses `pgvector/pgvector:pg17`).
289
- - The distance ordering lives in an ascending raw subquery (the only shape hnsw answers), pinned by a
290
- plan assertion. hnsw scopes after the scan — assert rows, never the node; `analyze` after a backfill.
291
- - Tenant and policy filters are in SQL on every statement (`conditionsSql`), on BOTH fusion halves.
292
- `(tenant, id)` is the primary key. `scoped()` only TIGHTENS (`X_VECTOR_SCOPE_WIDENED`). Metadata is
293
- bound `::text::jsonb`.
276
+ Full text of each rule: [`docs/history/ai.md`](../../docs/history/ai.md#evals-retrieval-fix-lines).
277
+
278
+ - **No fix line may name `x ai`** (planned); an eval is selected with `x test eval --filter <name>`.
279
+ - Every eval result carries the prompt hash; an eval gates on the DROP from its recorded baseline,
280
+ mean AND per case (`X_EVAL_BASELINE_MISSING` / `_INVALID`); every prompt is named by an eval
281
+ (`X_EVAL_MISSING`); `ULTIMATE_EVAL_RECORD=1` records, and `x verify` with it is `X_EVAL_RECORDING`.
282
+ - Retrieval is hybrid by default, no vector-only path. `chunk()` hard-wraps (no chunk over `size`);
283
+ `metadata.source` is written AFTER the caller's spread; `VectorStore.prune` is required.
284
+ - **An unbound store read inside a request whose actor has an `orgId` is `X_VECTOR_UNSCOPED`**
285
+ (both stores, every read); `scope: UNSCOPED` is the named opt-in, an omitted scope is `UNBOUND`.
286
+ - `postgresVectorStore()` is the only production vector path; `pg-vector.live.test.ts` refuses to
287
+ skip without the extension. Distance ordering is an ascending raw subquery (the shape hnsw
288
+ answers), pinned by a plan assertion. Tenant and policy filters are in SQL on every statement and
289
+ BOTH fusion halves; `(tenant, id)` is the key; `scoped()` only TIGHTENS (`X_VECTOR_SCOPE_WIDENED`).
294
290
 
295
291
  ## Commands
296
292
 
package/README.md CHANGED
@@ -205,7 +205,7 @@ registerModel({
205
205
  id: 'llama-internal-70b',
206
206
  contextWindow: 128_000,
207
207
  maxOutput: 8_192,
208
- inputPerMillion: { minor: 20, currency: 'USD' }, // YOUR price, integer minor units
208
+ inputPerMillion: { minor: 20, currency: 'USD' }, // YOUR price, integer minor units — or finer: { minor: 75, currency: 'USD', scale: 3 } is $0.075
209
209
  outputPerMillion: { minor: 40, currency: 'USD' },
210
210
  cacheMinimumTokens: 0,
211
211
  reasoning: { effort: false, adaptive: false, disableThinkingUpTo: undefined },
@@ -214,6 +214,8 @@ registerModel({
214
214
  configureAi({ gateway: providerGateway({ providers: [new InternalGatewayProvider()] }) });
215
215
  ```
216
216
 
217
+ Two optional per-model rows: `sampling: { temperature: true, seed: true }` (the model takes the `temperature` / `seed` an `llm()` or `agent()` declares — absent, they are not sent) and `forcedToolChoice: false` (the model rejects a forced `tool_choice`, so `llm()`'s one `respond` tool is offered under the endpoint's default).
218
+
217
219
  **The framework registers no model.** `registerModel` is the one way into the catalogue and only the
218
220
  app calls it — at boot, by convention in its own `models.ts`, each row with the page and the date its
219
221
  numbers were read from, so a stale price is a diff in the app's repo rather than a framework release.
@@ -433,7 +435,7 @@ sub-agent in its own `tools` and the sub-agent runs under the same actor, throug
433
435
  | an aborted `ctx` unwinds the run — at the top of every turn, before every tool batch, and on the socket | the transcript IS the request, so a loop that keeps going after the caller disconnects re-sends it once per remaining turn, runs every remaining side effect and discards the answer. `ctx.signal` rides on `GenerateRequest` too, so a call already in flight is cut rather than paid for |
434
436
  | the tools of **one turn** run concurrently, results paired by `tool_use` id | a turn asking for five tools cost 5x wall clock and nothing said so. Order is positional, never by completion; the batch is bounded by what one turn asked for, and each tool is an action with its own `policy` and `rateLimit`, so a second ceiling here would be a throttle competing with those |
435
437
  | `onTurn` reports each completed turn as it happens (and an `agent.turn` span event, always) | a 90-second run emitted nothing until it returned. Observation only — it cannot steer the loop, see the transcript or reach the actor — and a throw from it fails the run rather than being swallowed |
436
- | a tool that is not `mcp: { expose: true }` is `X_AGENT_TOOL_UNEXPOSED` **at declaration** | a silently dropped tool reads as offered and is not; `isMcpExposed` is the one predicate, so an in-app agent and an external MCP client see the same catalogue |
438
+ | a tool is an `mcp: { expose: true }` action or an `inProcessTool()`; anything else is `X_AGENT_TOOL_UNEXPOSED` **at declaration** | a silently dropped tool reads as offered and is not; `isMcpExposed` is the one predicate for ACTIONS, so an in-app agent and an external MCP client see the same catalogue of them — and a tool no MCP client may see is declared as one that is this process's alone |
437
439
  | running out of turns is `X_AGENT_MAX_TURNS`, never a partial answer | a half-finished transcript returned as a result is working notes presented as a decision |
438
440
  | a cycle of 1 to 3 tool turns — same calls, arguments, results and failures — gone round `maxRepeatedToolCalls` (default 3) times is `X_AGENT_TOOL_LOOP` | such a turn can only be followed by another like it; a changed result is progress; a repair turn neither counts nor resets |
439
441
  | `deadlineMs` (no default) is the run's wall-clock ceiling: `X_AGENT_DEADLINE` | the in-flight model call is aborted on the socket and every tool's `ctx.signal` fires; a tool that ignores it still finishes late. A caller that leaves first is still `X_ABORTED` |
@@ -443,6 +445,14 @@ sub-agent in its own `tools` and the sub-agent runs under the same actor, throug
443
445
  | a tool result is truncated, and says so | the transcript IS the request, so an untruncated result is re-billed once per remaining turn |
444
446
  | **no semantic cache** | similar prompts do not have similar answers once the answer depends on what `lookupOrder` returned this second |
445
447
 
448
+ ### What a call cost
449
+
450
+ `cost` is the currency's minor units, rounded up once per call; `costExact` beside it is the same figure exactly (`Money` at a finer `scale`, from `exactCostOf`). `agent()` reports both per turn and for the run in `onTurn`; `llm()` per provider call in `onCall`; a ledger sums exact costs and rounds once when read. `repairOf` on either event is what a repair turn was told to fix.
451
+
452
+ ### Continuing a run
453
+
454
+ `agent()` returns an `AgentAction`: the action, plus `.run(input, opts)` → `{ output, transcript, turns, usage }` and `.continue(input, { transcript, user }, opts)`, which appends one user turn to a stored transcript and resumes the loop. The transcript is re-sent byte for byte (the provider's prompt cache reads it back; no tool re-runs); `maxTurns` and `budget.tokensPerRun` are cumulative; the handle is signed, bound to the agent, model, tools and tenant, and screened (`X_AGENT_TRANSCRIPT_INVALID`); a failed call leaves an updated one on the error (`transcriptOf(error)`), which is the one a job stores. Full contract: [Agents → Continuing a run](https://github.com/developerz-ai/ultimate/wiki/Agents#continuing-a-run-run--continue).
455
+
446
456
  ### Provider prompt caching
447
457
 
448
458
  Not the gateway's response cache and not `llm()`'s semantic one: this asks the **provider** to keep
@@ -504,7 +514,7 @@ ever sent.
504
514
 
505
515
  | Limit | Detail |
506
516
  |---|---|
507
- | `run.cost` vs the ledger | `AgentTurn.run.cost` is summed usage priced once; `gateway.spent()` adds each turn's cost, rounded up per turn (input and output halves), so it can be higher by under 2 minor units per turn. Budgets are enforced on the ledger; report `run.cost` |
517
+ | `run.cost` and the ledger | one contract: cost is computed exactly and rounded up once. `AgentTurn.run.cost` is the run's summed usage priced once; `gateway.spent()` / `BudgetLedger.report().cost` is the exact cost of every recorded call summed, rounded once when read (`costExact` beside it). They agree for a run on built-in providers; a `Provider` that states no `costExact` is summed from its rounded `cost`. Budgets are enforced against the ledger |
508
518
  | the Anthropic row matches any host | an `anthropicProvider({ baseUrl })` behind a proxy still sends marks |
509
519
  | more than about 9 tool calls in one turn, on OpenRouter | only the tail is marked there and the provider looks back 20 blocks, so that turn's transcript is rewritten, not read |
510
520
  | a mark never lands on an empty or whitespace-only text block, and never makes a fifth | it moves back a block or is dropped; a raw caller's own marks are counted first |
@@ -925,6 +935,8 @@ An in-app agent and an external MCP agent both end at the same `invoke` — `run
925
935
  carries it, and an action facade has no `.run` of its own. So they authorize identically. The
926
936
  actor comes from the request context, never from the model.
927
937
 
938
+ `inProcessTool({ name, description, input, execute })` is a tool that is this process's alone: no `mcp`, no policy of its own (the agent's decides), in no catalogue but the agent's. Its own typed refusal reads back as `{"error":{"code","cause","fix"}}`. [Agents → In-process tools](https://github.com/developerz-ai/ultimate/wiki/Agents#in-process-tools).
939
+
928
940
  A failed tool reads back as `CODE: cause (fix: …)` — except a 5xx code whose cause core's
929
941
  `hasPublicCause` does not declare public, which reads `CODE: ` plus a fixed sentence (and a
930
942
  `callerFix`, when the `UltimateError` declared one): the rule a production problem document follows,
@@ -953,7 +965,7 @@ fail-closed `terminal` default.
953
965
  | `X_LLM_OUTPUT_INVALID` | the model failed its `output` schema on the answer and on the repair turn |
954
966
  | `X_LLM_STREAM_INVALID` | a streamed answer failed its schema, and a stream cannot take a repair turn |
955
967
  | `X_AGENT_MAX_TURNS` | an `agent()` used every turn without answering |
956
- | `X_AGENT_TOOL_UNEXPOSED` | an `agent()` lists an action no MCP surface exposes |
968
+ | `X_AGENT_TOOL_UNEXPOSED` | an `agent()` lists a tool that is neither in-process nor an MCP-exposed action |
957
969
  | `X_AGENT_TOOL_LOOP` | an `agent()` made the same tool calls and read the same results `maxRepeatedToolCalls` turns in a row |
958
970
  | `X_AGENT_DEADLINE` | an `agent()` outlived its declared `deadlineMs` |
959
971
  | `X_EVAL_THRESHOLD` | an eval scored below its bar |
@@ -962,6 +974,8 @@ fail-closed `terminal` default.
962
974
  | `X_VECTOR_UNSCOPED` | a store with no tenant bound was read inside a request acting for an org; `scope: UNSCOPED` is the named opt-out |
963
975
  | `X_NOT_IMPLEMENTED` | a remote driver with no key or transport; the fix names the env var |
964
976
 
977
+ A failure thrown after a provider call was started carries what the run was billed on `meta.billed` (`billedOf(error)` → `{ usage, cost, calls, unsettled }`); `unsettled` counts the calls whose bill cannot be known. [Agents → Unattended runs](https://github.com/developerz-ai/ultimate/wiki/Agents#unattended-runs).
978
+
965
979
  ### Error classes
966
980
 
967
981
  Every error class `src/index.ts` exports, for `instanceof` inside one process. Across a wire or
@@ -972,6 +986,7 @@ a job boundary the class is gone and the `code` is what survives — match on th
972
986
  | `AgentDeadlineError` | `X_AGENT_DEADLINE` | `src/agent-errors.ts` |
973
987
  | `AgentMaxTurnsError` | `X_AGENT_MAX_TURNS` | `src/errors.ts` |
974
988
  | `AgentToolLoopError` | `X_AGENT_TOOL_LOOP` | `src/agent-errors.ts` |
989
+ | `AgentTranscriptInvalidError` | `X_AGENT_TRANSCRIPT_INVALID` | `src/agent-errors.ts` |
975
990
  | `AgentToolUnexposedError` | `X_AGENT_TOOL_UNEXPOSED` | `src/errors.ts` |
976
991
  | `AiBudgetExceededError` | `X_AI_BUDGET_EXCEEDED` | `src/errors.ts` |
977
992
  | `AiContentUnsupportedError` | `X_AI_CONTENT_UNSUPPORTED` | `src/content-errors.ts` |
@@ -994,7 +1009,8 @@ a job boundary the class is gone and the `code` is what survives — match on th
994
1009
  | `LlmOutputInvalidError` | `X_LLM_OUTPUT_INVALID` | `src/errors.ts` |
995
1010
  | `LlmRefusedError` | `X_LLM_REFUSED` | `src/errors.ts` |
996
1011
  | `LlmStreamInvalidError` | `X_LLM_STREAM_INVALID` | `src/errors.ts` |
997
- | `LlmTruncatedError` | `X_LLM_TRUNCATED` | `src/errors.ts` |
1012
+ | `LlmToolArgumentsError` | `X_LLM_TOOL_ARGUMENTS_INVALID` | `src/answer-errors.ts` |
1013
+ | `LlmTruncatedError` | `X_LLM_TRUNCATED` | `src/answer-errors.ts` |
998
1014
  | `VectorDimMismatchError` | `X_VECTOR_DIM_MISMATCH` | `src/errors.ts` |
999
1015
  | `VectorScopeWidenedError` | `X_VECTOR_SCOPE_WIDENED` | `src/errors.ts` |
1000
1016
  | `VectorUnscopedError` | `X_VECTOR_UNSCOPED` | `src/vector-scope.ts` |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@ultimat3/ai",
3
- "version": "27.9.0",
3
+ "version": "27.10.1",
4
4
  "description": "LLM gateway, versioned prompts, evals as tests, embeddings, hybrid vector search, RAG",
5
5
  "license": "MIT",
6
6
  "type": "module",
@@ -32,15 +32,15 @@
32
32
  "test": "bun test"
33
33
  },
34
34
  "dependencies": {
35
- "@ultimat3/action": "27.9.0",
36
- "@ultimat3/cache": "27.9.0",
37
- "@ultimat3/core": "27.9.0",
38
- "@ultimat3/db": "27.9.0",
39
- "@ultimat3/http": "27.9.0",
40
- "@ultimat3/jobs": "27.9.0",
41
- "@ultimat3/money": "27.9.0",
42
- "@ultimat3/policy": "27.9.0",
43
- "@ultimat3/schema": "27.9.0",
44
- "@ultimat3/time": "27.9.0"
35
+ "@ultimat3/action": "27.10.1",
36
+ "@ultimat3/cache": "27.10.1",
37
+ "@ultimat3/core": "27.10.1",
38
+ "@ultimat3/db": "27.10.1",
39
+ "@ultimat3/http": "27.10.1",
40
+ "@ultimat3/jobs": "27.10.1",
41
+ "@ultimat3/money": "27.10.1",
42
+ "@ultimat3/policy": "27.10.1",
43
+ "@ultimat3/schema": "27.10.1",
44
+ "@ultimat3/time": "27.10.1"
45
45
  }
46
46
  }