@ultimat3/ai 24.0.0 → 25.1.0

package/CLAUDE.md

@@ -15,14 +15,24 @@ rather than imported. Same contract, two wire formats — and the same *decision
 local `=== true`. An in-app agent and an external one must be offered exactly the same tools.
 
 **And the same NAME**: `toLlmTool` passes `action.name` through untouched (the export name
-`@ultimat3/mcp` serves); `llm()` and `agent()` `.tool()` names are verbatim too. Never derive one.
+`@ultimat3/mcp` serves); `llm()` and `agent()` tool names are verbatim too. Never derive one.
+
+**And the same TOOL**: `asProjectableAction` and mcp's `primitiveFromAction` are two copies the
+tier table forces; `scripts/ai-mcp-tool-parity.test.ts` holds them equal — name, description,
+exposure, the whole input schema (an idempotent action's reserved `idempotencyKey` argument,
+`tool-idempotency.ts`), the policy verdict and the replay. A field one adds, the other adds.
 
 ## Owns
 
 | File | Job |
 |---|---|
-| `models.ts` | the model REGISTRY: `registerModel`, limits, prices, the reasoning controls each one accepts |
-| `provider.ts` | `Provider` interface, the request half, the money arithmetic, Anthropic + Echo |
+| `models.ts` | the model REGISTRY: `registerModel`, limits, prices, the reasoning controls each one accepts — and NO row of its own |
+| `model-resolve.ts` | `resolveModel` / `modelForCall` — declaration → prompt → gateway `defaultModel`, none is `X_AI_MODEL_UNRESOLVED` naming all three; `describedModel` for a published fact |
+| `model-fixture.ts` | the rows this package's own suites register (`useFixtureModels()`), as an app's `models.ts` would. Not shipped (`!src/**/*-fixture.ts`) |
+| `provider.ts` | `Provider` interface, the request half, the money arithmetic, `anthropicProvider()` |
+| `echo-provider.ts` | `echoProvider()`, the deterministic double — split from `provider.ts` at its line ceiling |
+| `content-blocks.ts` | the `image` / `document` block shapes, the screen every request runs them through, their token estimate |
+| `content-errors.ts` | the `X_AI_CONTENT_UNSUPPORTED` class; its code and title belong in `errors.ts` |
 | `wire.ts` | the response half: `usage` / `stop_reason` shapes, and the SSE `MessageStream` |
 | `error-body.ts` | what a failure body SAYS (`detailOf`) and what must never survive into it (`withoutKey`) — one copy, both transports |
 | `sse.ts` | Server-Sent Events framing — protocol only, knows nothing about Anthropic |
@@ -30,12 +40,12 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
 | `openai-messages.ts` | the format mapping's request half: `AiMessage` blocks → OpenAI messages, `LlmTool` → functions, `tool_choice` |
 | `openai-body.ts` | one chat-completions body, and the per-model reasoning mapping |
 | `openai-wire.ts` | its response half: one completion, and `ChatCompletionStream` (fragmented tool calls, trailing usage, `[DONE]`) |
-| `openai-models.ts` | the OpenAI-format price rows, registered through the same `registerModel` |
 | `gateway.ts` | routing, retries, cache, budget wiring |
 | `budget.ts` | token ledgers per request/actor/org, ALS carrier |
-| `prompt.ts` | `definePrompt`, content hashing, version registry |
-| `embeddings.ts` | `Embedder`, `HashEmbedder`, cosine helpers |
-| `remote-embedder.ts` | `RemoteEmbedder` — the production `/v1/embeddings` client |
+| `prompt.ts` | `definePrompt`, `promptHash`, version registry |
+| `prompt-fence.ts` | a template's data fences: which tag pairs enclose each slot (`promptFences`), and the closer-breaking `render` applies inside one |
+| `embeddings.ts` | `Embedder`, `hashEmbedder()`, cosine helpers |
+| `remote-embedder.ts` | `remoteEmbedder()` — the production `/v1/embeddings` client |
 | `evals.ts` | `defineEval`, the run, the baseline gate, prompt coverage |
 | `eval-baseline.ts` | the recorded scores: path, read/write, what counts as a regression |
 | `scorers.ts` | what a `Scorer` is, the built-in ones, and `llmJudge` |
@@ -43,9 +53,11 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
 | `pg-vector.live.test.ts` | the same store against a real pgvector — DDL, fusion, scope, plan |
 | `vector-scope.ts` | the tenant + policy envelope, and the tighten-only derive rule |
 | `pg-vector-sql.ts` | every pgvector statement: DDL, upsert, cosine, FTS, RRF fusion |
-| `pg-vector.ts` | `PgVectorStore` — the production store |
+| `pg-vector.ts` | `postgresVectorStore()` — the production store |
 | `rag.ts` | chunker, retriever, reranker, budgeted context assembler |
 | `tools.ts` | action → LLM tool definition; the `AgentTool` union and `asProjectableAction`; `runLlmToolCall` |
+| `tool-idempotency.ts` | an idempotent action's reserved `idempotencyKey` tool argument — MCP's, restated (same tier), held equal by the parity test |
+| `failure-disclosure.ts` | `discloseFailure` — the ONE rule for what of a throw a remote reader sees (tool result, hive member); the withheld half is logged as code, name and stack frames — never its message or cause |
 | `llm.ts` | `llm()` — the model call, declared as an `action`; and what a streamed answer must satisfy |
 | `respond.ts` | the `respond` tool for one `output` (a non-object one wrapped in `{ value }`), its reader, and `parseJsonish` |
 | `llm-stream.ts` | `.stream()`'s plumbing: the sink, the ambient mark, the one-turn drive |
@@ -61,9 +73,9 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
 | `llm-cache.ts` | the semantic cache half of `llm()`: what a declaration may partition on, and the store it reaches |
 | `llm-fixture.ts` | the harness `llm.test.ts` and `llm-cache.test.ts` share. Not shipped (`!src/**/*-fixture.ts`) |
 | `bounds-fixture.ts` | the one `refusal`/`asyncRefusal` every numeric-bound suite here asserts through. Not shipped, same rule |
-| `agent-bounds.test.ts` | the agent loop's ceilings, split out because `agent.test.ts` is at the 500-line ceiling |
+| `agent-bounds.test.ts` | the agent loop's ceilings (split from `agent.test.ts`, at its line ceiling) |
 | `runtime.ts` | the ambient gateway / embedder / semantic caches an `llm()` reaches |
-| `fix-line.ts` / `fix-line.evals.ts` / `fix-line.v1.baseline.json` / `fix-line.eval.test.ts` | the package's own dogfood eval — the first framework-level `*.eval.test.ts`, proving the `defineEval`/baseline convention actually fails a build |
+| `fix-line-fixture.ts` / `fix-line.evals-fixture.ts` / `fix-line.v1.baseline.json` / `fix-line.eval.test.ts` | the package's dogfood eval — proof the baseline convention fails a build |
 
 ## Invariants — bounds and budget
 
@@ -74,7 +86,7 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
   off. `Gateway.generate`/`stream` screen it at the one seam, `registerModel` screens `maxOutput`, and
   `llm()`/`agent()` screen theirs at DECLARATION.
 - **A bound is screened under the key the DECLARATION uses** (`budget.tokensPerRun` in `limitsOf`
-  before the ledger's `request`; `retrieve()`'s own `k`; `RemoteEmbedder`'s `maxResponseBytes`).
+  before the ledger's `request`; `retrieve()`'s own `k`; `remoteEmbedder()`'s `maxResponseBytes`).
 - A per-call budget `derive`s from the ambient ledger and only TIGHTENS. **A derived ledger reports
   back up the chain** (every debit on every ancestor; `reserve` checks each `request` scope); the
   STORE is written once, by the ledger the call was made on.
@@ -84,13 +96,14 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
   at a later scope gives back what the earlier ones took.
 - **The store is touched only for a scope whose ceiling is DECLARED** (`takeScope`, `meteredKeys`):
   every call carries keys, so a write per key grew the default store by one entry per caller.
-  `MemoryBudgetStore` has no window and no eviction (eviction is a ceiling bypass); a counter back
+  `memoryBudgetStore()` has no window and no eviction (eviction is a ceiling bypass); a counter back
   at zero is deleted.
 - **`llm()`, `agent()`, `hive()` root as `currentBudget() ?? gateway.callLedger(budgetKeysFor(ctx.actor))`**
-  — the gateway's ceilings, keyed `actor:<kind>:<id>` / `org:<orgId>`. `callLedger` is required on
+  — the gateway's ceilings, keyed `actor:<kind>:<id>` / `org:<orgId>` (none for an `undefined`/`null`/`''`
+  org — the `orgless` test `query`'s cache uses). `callLedger` is required on
   `Gateway`; `hive()` asks `installedGateway()` so a hive of plain actions needs none.
 - **`BudgetStore` is where `actor` and `org` live; the default is per PROCESS**
-  (`createGateway({ budgetStore })`). `add` takes a negative `tokens` for a release — a store that
+  (`providerGateway({ budgetStore })`). `add` takes a negative `tokens` for a release — a store that
   clamps at zero leaks the ceiling.
 - Cost is `Money` (integer minor units), rounded **up**. A budget throws `X_AI_BUDGET_EXCEEDED`
   **before** the provider call. Never truncate. List price over-reserves (no introductory prices).
@@ -112,7 +125,7 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
   `ctx.locale` (in the unconditional half). **The default scope is the calling ACTOR**
   (`JSON.stringify([actor.kind, actor.id, actor.orgId ?? null])`); `scope` receives
   `{ input, ctx }`; widening is `scope: () => 'global'`. The instance map is bounded
-  (`MAX_SEMANTIC_CACHE_SCOPES`).
+  (core's `MAX_CACHED_FORMATTERS`).
 - `cache.invalidates` is not on `llm()`: a `SemanticCache` is not a `CacheTier`. Version bump + `ttl`
   is the invalidation.
 - The gateway is ambient (`configureAi`); absent at call time is `X_AI_GATEWAY_MISSING`, never a
@@ -127,6 +140,10 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
 - **Fallback is across PROVIDERS serving one model, never across models**; `GenerateResult.provider`
   is stamped by the gateway and put on the span as `llm.provider`.
 - Server-side `fallbacks` (beta) are never sent — the stable `2023-06-01` surface only.
+- **A slot inside a tag pair is DATA** (`prompt-fence.ts`, #689): `render` breaks, in the ASSEMBLED
+  prompt, every fence closer a fenced value wrote any character of (never per value: two slots can
+  each hold half) — never deletes, never touches a template-only closer. The
+  fences are read once at `definePrompt`; no hash moves.
 - `definePrompt` refuses a re-registered version whose hash moved; the hash is over core's
   `canonicalJson` (`prompt.test.ts` pins one hash literally); an absent schema hashes as `''`.
 
@@ -147,7 +164,7 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
   stays UNREGISTERED on purpose.
 - **The retry SCHEDULE is core's (`backoffDelay`, `isRetryableStatus`); the LOOP is this package's** —
   core's `retry()` retries unclassified throws, and every value here is an app `Provider`'s plain
-  object, so it would retry a 400. `gateway-backoff.test.ts`. `RetryPolicy`/`DEFAULT_RETRY` keep the
+  object, so it would retry a 400. `gateway-backoff.test.ts`. `RetryPolicy`/`DEFAULT_GATEWAY_RETRY` keep the
   field names an app writes.
 - Every non-2xx and every in-band `error` frame is `AiTransportError` with a real `status`.
 - **Both wire formats answer the same question the same way** (`provider-parity.test.ts`): an in-band
@@ -160,15 +177,33 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
 - A tool call is emitted whole. Thinking chunks are never appended to `text`.
 - Anthropic body: no `temperature`/`top_p`/`top_k`, no `budget_tokens`, `effort` inside
   `output_config`. Model IDs are exact aliases, never date-suffixed.
+- **Apps bring their own models and providers (M12, 25.0.0): no vendor choice, no vendor data.** No
+  default model, no catalogue row, no id list ships; the registry starts EMPTY and `resetModels()`
+  empties it. A model resolves declaration → prompt → `Gateway.defaultModel` through `resolveModel`
+  / `modelForCall` only; none is `X_AI_MODEL_UNRESOLVED` naming all three (with no gateway and nothing
+  declared, `X_AI_GATEWAY_MISSING` — the third place does not exist yet). `vendor-neutral.test.ts`
+  proves it in a fresh process; no fix line names a vendor id.
 - **`ModelId` is `string`; the catalogue is an open registry** — `modelSpec(id)` refuses an
-  unregistered id (`X_AI_MODEL_UNKNOWN`). The built-ins register through the same `registerModel`.
-  **Re-registering REPLACES the spec and keeps its rung** (negotiated rates). **Registration order is
-  the ladder within a `family`**, read only by `moreCapableThan`; `X_LLM_REFUSED`'s fix names a rung
-  ABOVE or nothing.
-- `AnthropicProvider.models` is its own list; `EchoProvider.models` reads the registry.
+  unregistered id (`X_AI_MODEL_UNKNOWN`, fix `registerModel(…)`) at `costOf`, the budget estimate and
+  the body builders, before a socket or a reservation. **Re-registering REPLACES the spec and keeps
+  its rung** (negotiated rates). **Registration order is the ladder within a `family`**, read only by
+  `moreCapableThan`, over the APP's rows; `X_LLM_REFUSED`'s fix names a rung ABOVE or nothing.
+- **Every provider serves the app's list**: `anthropicProvider({ models })` and
+  `openAiProvider({ models })` require a non-empty one (`X_AI_REQUEST_INVALID`, a missing config
+  too); `echoProvider().models` reads the registry. **No provider has a model of its own**: a direct
+  call naming none is `X_AI_MODEL_UNRESOLVED` from all three — the list is what it serves, never a
+  fourth place (`vendor-neutral.test.ts`, one parity case).
 - **The reasoning controls are PER MODEL (`models.ts`)**: an unasked control is omitted; an asked
-  control the model lacks is `X_AI_REQUEST_INVALID`, never dropped. Adding a model is a row in `MODELS`.
+  control the model lacks is `X_AI_REQUEST_INVALID`, never dropped. Adding a model is a row the app
+  registers. `disableThinkingUpTo: 'never'` refuses `thinking: 'disabled'` at every effort;
+  `disabledThinking: 'between_tools'` is the wire spelling of the same declaration
+  (`model-rules.test.ts`, over the fixture rows).
+- **Media blocks are screened once, by `assertMediaContent`, at the top of BOTH body builders** —
+  malformed is `X_AI_REQUEST_INVALID`, untakeable (role, the row's `input`, the wire) is
+  `X_AI_CONTENT_UNSUPPORTED`; never dropped, never approximated (`provider-parity-content.test.ts`).
 - `generate()` above `STREAM_ONLY_MAX_TOKENS` runs the streaming transport.
+- **Every provider and embedder is built by its factory** (`anthropicProvider()`, `echoProvider()`,
+  `hashEmbedder()`, `remoteEmbedder()`); the classes are type exports only (`bun run factory-names`).
 - **`openAiProvider()` is a FORMAT, not a vendor** — `baseUrl`, `auth`, `headers` are the
   differences; never a second class per vendor:
   - structured output is the `respond` tool, never `response_format`; `tool_choice` names the tool
@@ -178,17 +213,16 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
     ESTIMATE when usage never arrives; subtract `cached_tokens` from `prompt_tokens`;
   - tool-call deltas merge by `tool_calls[].index`; `isComplete()` accepts `[DONE]` or a finish reason,
     except `[DONE]` with pending tool fragments, which is refused;
-  - only `gpt-5.6-sol` / `-terra` / `-luna` are priced; `registerOpenAiModels()` re-registers after
-    `resetModels()` (every `openai-*.test.ts` calls it in `beforeEach`);
-  - no new `X_*` code; `AiTransportError` takes the provider's `envVar`; the key is revealed late,
+  - no code of its own (a media refusal is the shared `X_AI_CONTENT_UNSUPPORTED`);
+    `AiTransportError` takes the provider's `envVar`; the key is revealed late,
     never stored, scrubbed from `detail`.
 - **`embedBatched` enforces the `Embedder` arity per batch** (`X_AI_EMBEDDER_INVALID` with both
-  counts). One `RemoteEmbedder` for every vendor; vectors L2-normalised; a wrong width is
+  counts). One `remoteEmbedder()` for every vendor; vectors L2-normalised; a wrong width is
   `X_VECTOR_DIM_MISMATCH`.
 - **A caught value is read with `renderThrowable`** (`remote-embedder.ts`, `wire.ts`,
   `openai-wire.ts`) — `scripts/error-render.ts` cannot see `catch` bindings.
 - **A caller's string is never an object KEY**: the wire tables are `Map`s; `prompt.render` and
-  `EchoProvider`'s `replies` use `Object.hasOwn`.
+  `echoProvider()`'s `replies` use `Object.hasOwn`.
 
 ## Invariants — `agent()`, `hive()`, `agentJob()`
 
@@ -211,19 +245,23 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
     never as a failure); the throw is read with `stringField`.
   - **A 5xx code's cause reaches the model only when core's `hasPublicCause` allows it** (status from
     `@ultimat3/http`'s `statusFor`) — else `CODE: HIDDEN_TOOL_CAUSE`, plus a branded `callerFix`. The
-    same verdict a problem document and `@ultimat3/mcp` give.
+    same verdict a problem document and `@ultimat3/mcp` give, from `failure-disclosure.ts` — the
+    rule a hive member's `reason` follows too. Never a second copy.
 - **`hive()` is a fan-out action** (`PRIMITIVE_FACTORIES`): `HiveResult` is a SCHEMA from the
   member's `output`; three arms (`ok`/`failed`/`skipped`) and three counters; `members` in SPLIT
   order with `index`; the hive never names an actor; no hive budget code (the reservation holds
-  it); a member's throw is recorded via `isThrownError`/`stringField`; `skipped` reasons
+  it); a member's throw is recorded through `discloseFailure` — a 5xx cause or an uncoded throw's
+  message becomes `HIDDEN_MEMBER_CAUSE`, logged as facts only; `skipped` reasons
   `SKIPPED_ABORTED` / `SKIPPED_NO_INPUT`; `onMemberError` is required; `concurrency` 4, `minMembers`
   2 (a below-floor split still runs serially); an empty split is `X_HIVE_EMPTY`; an aborted `ctx`
   unwinds with `X_ABORTED`.
 - **`describeAgents()` / `registeredModels()` are offered, not published** — their only legal consumer
   is `@ultimat3/cli`. The facts are a THUNK; an unnamed agent has no row.
-- **`agentJob()` composes `job()`** (never an imitation handle). `name`, `tenant`, `retry` are
-  REQUIRED; both reads of `target.job()` are LAZY. **Every tool an `agentJob()`'d agent may call must
-  be idempotent** — stated in `AgentJobOptions.idempotencyKey`'s doc and the README, not enforceable.
+- **`agentJob()` composes `job()`** (never an imitation). `name`, `tenant`, `retry` are
+  REQUIRED; both reads of `target.job()` are LAZY. No result is kept: wrap an ACTION that runs the
+  agent and writes, target bound from its input — never a model-aimed write tool; every write
+  idempotent (unenforceable). **`actor`** re-resolves who it acts FOR each attempt (a served worker:
+  ANONYMOUS) via `impersonate`; off-tenant is TERMINAL `X_JOB_TENANT_MISMATCH`.
 
 ## Invariants — evals, retrieval, fix lines
 
@@ -241,8 +279,8 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
 - **A store with no tenant bound, read inside a request whose actor has an `orgId`, is
   `X_VECTOR_UNSCOPED`** (`assertTenantRead`, both stores, every read). `scope: UNSCOPED`
   (`crossTenant: true`) is the named opt-in; an omitted scope is `UNBOUND`.
-- `PgVectorStore` is the only production vector path (pgvector + FTS in the app's Postgres);
-  `MemoryVectorStore` enforces the same envelope. **`pg-vector.live.test.ts` refuses to skip** on a
+- `postgresVectorStore()` is the only production vector path (pgvector + FTS in the app's Postgres);
+  `memoryVectorStore()` enforces the same envelope. **`pg-vector.live.test.ts` refuses to skip** on a
   Postgres without the extension (CI uses `pgvector/pgvector:pg17`).
 - The distance ordering lives in an ascending raw subquery (the only shape hnsw answers), pinned by a
   plan assertion. hnsw scopes after the scan — assert rows, never the node; `analyze` after a backfill.