@ultimat3/ai 23.0.0 → 25.0.0

package/CLAUDE.md

@@ -4,7 +4,8 @@ Tier 4. May import tier 0–3: `core schema i18n money time cache seo entity pol
 query jobs realtime`. **Never** `mcp manifest render pwa ui admin testing cli`.
 
 Declared today: `action` (the primitive `llm()` returns), `cache` (semantic cache), `core`,
-`db` (pgvector), `jobs` (`agentJob()`), `money`, `policy`, `schema`, `time`.
+`db` (pgvector), `http` (`statusFor`, so a tool result redacts a 5xx cause as a problem document
+does), `jobs` (`agentJob()`), `money`, `policy`, `schema`, `time`.
 
 `jobs` is declared for `agentJob()`: the action→job bridge has to live at tier 4+.
 
@@ -14,14 +15,24 @@ rather than imported. Same contract, two wire formats — and the same *decision
 local `=== true`. An in-app agent and an external one must be offered exactly the same tools.
 
 **And the same NAME**: `toLlmTool` passes `action.name` through untouched (the export name
-`@ultimat3/mcp` serves); `llm()` and `agent()` `.tool()` names are verbatim too. Never derive one.
+`@ultimat3/mcp` serves); `llm()` and `agent()` tool names are verbatim too. Never derive one.
+
+**And the same TOOL**: `asProjectableAction` and mcp's `primitiveFromAction` are two copies the
+tier table forces; `scripts/ai-mcp-tool-parity.test.ts` holds them equal — name, description,
+exposure, the whole input schema (an idempotent action's reserved `idempotencyKey` argument,
+`tool-idempotency.ts`), the policy verdict and the replay. A field one adds, the other adds.
 
 ## Owns
 
 | File | Job |
 |---|---|
-| `models.ts` | the model REGISTRY: `registerModel`, limits, prices, the reasoning controls each one accepts |
-| `provider.ts` | `Provider` interface, the request half, the money arithmetic, Anthropic + Echo |
+| `models.ts` | the model REGISTRY: `registerModel`, limits, prices, the reasoning controls each one accepts — and NO row of its own |
+| `model-resolve.ts` | `resolveModel` / `modelForCall` — declaration → prompt → gateway `defaultModel`, none is `X_AI_MODEL_UNRESOLVED` naming all three; `describedModel` for a published fact |
+| `model-fixture.ts` | the rows this package's own suites register (`useFixtureModels()`), as an app's `models.ts` would. Not shipped (`!src/**/*-fixture.ts`) |
+| `provider.ts` | `Provider` interface, the request half, the money arithmetic, `anthropicProvider()` |
+| `echo-provider.ts` | `echoProvider()`, the deterministic double — split from `provider.ts` at its line ceiling |
+| `content-blocks.ts` | the `image` / `document` block shapes, the screen every request runs them through, their token estimate |
+| `content-errors.ts` | the `X_AI_CONTENT_UNSUPPORTED` class; its code and title belong in `errors.ts` |
 | `wire.ts` | the response half: `usage` / `stop_reason` shapes, and the SSE `MessageStream` |
 | `error-body.ts` | what a failure body SAYS (`detailOf`) and what must never survive into it (`withoutKey`) — one copy, both transports |
 | `sse.ts` | Server-Sent Events framing — protocol only, knows nothing about Anthropic |
@@ -29,12 +40,12 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
 | `openai-messages.ts` | the format mapping's request half: `AiMessage` blocks → OpenAI messages, `LlmTool` → functions, `tool_choice` |
 | `openai-body.ts` | one chat-completions body, and the per-model reasoning mapping |
 | `openai-wire.ts` | its response half: one completion, and `ChatCompletionStream` (fragmented tool calls, trailing usage, `[DONE]`) |
-| `openai-models.ts` | the OpenAI-format price rows, registered through the same `registerModel` |
 | `gateway.ts` | routing, retries, cache, budget wiring |
 | `budget.ts` | token ledgers per request/actor/org, ALS carrier |
-| `prompt.ts` | `definePrompt`, content hashing, version registry |
-| `embeddings.ts` | `Embedder`, `HashEmbedder`, cosine helpers |
-| `remote-embedder.ts` | `RemoteEmbedder` — the production `/v1/embeddings` client |
+| `prompt.ts` | `definePrompt`, `promptHash`, version registry |
+| `prompt-fence.ts` | a template's data fences: which tag pairs enclose each slot (`promptFences`), and the closer-breaking `render` applies inside one |
+| `embeddings.ts` | `Embedder`, `hashEmbedder()`, cosine helpers |
+| `remote-embedder.ts` | `remoteEmbedder()` — the production `/v1/embeddings` client |
 | `evals.ts` | `defineEval`, the run, the baseline gate, prompt coverage |
 | `eval-baseline.ts` | the recorded scores: path, read/write, what counts as a regression |
 | `scorers.ts` | what a `Scorer` is, the built-in ones, and `llmJudge` |
@@ -42,10 +53,13 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
 | `pg-vector.live.test.ts` | the same store against a real pgvector — DDL, fusion, scope, plan |
 | `vector-scope.ts` | the tenant + policy envelope, and the tighten-only derive rule |
 | `pg-vector-sql.ts` | every pgvector statement: DDL, upsert, cosine, FTS, RRF fusion |
-| `pg-vector.ts` | `PgVectorStore` — the production store |
+| `pg-vector.ts` | `postgresVectorStore()` — the production store |
 | `rag.ts` | chunker, retriever, reranker, budgeted context assembler |
 | `tools.ts` | action → LLM tool definition; the `AgentTool` union and `asProjectableAction`; `runLlmToolCall` |
+| `tool-idempotency.ts` | an idempotent action's reserved `idempotencyKey` tool argument — MCP's, restated (same tier), held equal by the parity test |
+| `failure-disclosure.ts` | `discloseFailure` — the ONE rule for what of a throw a remote reader sees (tool result, hive member); the withheld half is logged as code, name and stack frames — never its message or cause |
 | `llm.ts` | `llm()` — the model call, declared as an `action`; and what a streamed answer must satisfy |
+| `respond.ts` | the `respond` tool for one `output` (a non-object one wrapped in `{ value }`), its reader, and `parseJsonish` |
 | `llm-stream.ts` | `.stream()`'s plumbing: the sink, the ambient mark, the one-turn drive |
 | `agent.ts` | `agent()` — the tool loop, declared as an `action` |
 | `agent-transcript.ts` | what one turn leaves in the transcript: the assistant replay, the tool results, the correction |
@@ -72,12 +86,24 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
   off. `Gateway.generate`/`stream` screen it at the one seam, `registerModel` screens `maxOutput`, and
   `llm()`/`agent()` screen theirs at DECLARATION.
 - **A bound is screened under the key the DECLARATION uses** (`budget.tokensPerRun` in `limitsOf`
-  before the ledger's `request`; `retrieve()`'s own `k`; `RemoteEmbedder`'s `maxResponseBytes`).
+  before the ledger's `request`; `retrieve()`'s own `k`; `remoteEmbedder()`'s `maxResponseBytes`).
 - A per-call budget `derive`s from the ambient ledger and only TIGHTENS. **A derived ledger reports
   back up the chain** (every debit on every ancestor; `reserve` checks each `request` scope); the
-  STORE is written once, by the ledger the call was made on; reservations queue on the ROOT's turnstile.
+  STORE is written once, by the ledger the call was made on.
+- **`reserve` debits, then checks — never a read-then-write across an `await`.** The in-memory scopes
+  check and debit synchronously; the store's go through `BudgetStore.take(key, tokens, limit)`, which
+  is REQUIRED and must be atomic (no default over `spent` + `add` — that pair is the race). A refusal
+  at a later scope gives back what the earlier ones took.
+- **The store is touched only for a scope whose ceiling is DECLARED** (`takeScope`, `meteredKeys`):
+  every call carries keys, so a write per key grew the default store by one entry per caller.
+  `memoryBudgetStore()` has no window and no eviction (eviction is a ceiling bypass); a counter back
+  at zero is deleted.
+- **`llm()`, `agent()`, `hive()` root as `currentBudget() ?? gateway.callLedger(budgetKeysFor(ctx.actor))`**
+  — the gateway's ceilings, keyed `actor:<kind>:<id>` / `org:<orgId>` (none for an `undefined`/`null`/`''`
+  org — the `orgless` test `query`'s cache uses). `callLedger` is required on
+  `Gateway`; `hive()` asks `installedGateway()` so a hive of plain actions needs none.
 - **`BudgetStore` is where `actor` and `org` live; the default is per PROCESS**
-  (`createGateway({ budgetStore })`). `add` takes a negative `tokens` for a release — a store that
+  (`providerGateway({ budgetStore })`). `add` takes a negative `tokens` for a release — a store that
   clamps at zero leaks the ceiling.
 - Cost is `Money` (integer minor units), rounded **up**. A budget throws `X_AI_BUDGET_EXCEEDED`
   **before** the provider call. Never truncate. List price over-reserves (no introductory prices).
@@ -88,6 +114,9 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
 
 - **`llm()` returns an `action`**, never a ninth primitive; it adds only the model half: render the
   prompt, project `output` into the one `respond` tool, reserve the budget, consult the semantic cache.
+- **A non-object `output` is wrapped in `{ value }` and unwrapped** (`respond.ts`) — a tool's input is
+  an object; never refused, because `.stream()` serves `output: t.string`. `parseJsonish` is two
+  `indexOf` scans, never a backtracking pattern.
 - `src/index.ts` re-exports `t` from `@ultimat3/schema` **verbatim** (`index.test.ts`).
 - One repair turn, then `X_LLM_OUTPUT_INVALID`. A refusal (a 200 with no answer) is `X_LLM_REFUSED`
   before the parse, carrying `stopDetails.category`; a truncated invalid answer is `X_LLM_TRUNCATED`.
@@ -96,7 +125,7 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
   `ctx.locale` (in the unconditional half). **The default scope is the calling ACTOR**
   (`JSON.stringify([actor.kind, actor.id, actor.orgId ?? null])`); `scope` receives
   `{ input, ctx }`; widening is `scope: () => 'global'`. The instance map is bounded
-  (`MAX_SEMANTIC_CACHE_SCOPES`).
+  (core's `MAX_CACHED_FORMATTERS`).
 - `cache.invalidates` is not on `llm()`: a `SemanticCache` is not a `CacheTier`. Version bump + `ttl`
   is the invalidation.
 - The gateway is ambient (`configureAi`); absent at call time is `X_AI_GATEWAY_MISSING`, never a
@@ -111,11 +140,20 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
 - **Fallback is across PROVIDERS serving one model, never across models**; `GenerateResult.provider`
   is stamped by the gateway and put on the span as `llm.provider`.
 - Server-side `fallbacks` (beta) are never sent — the stable `2023-06-01` surface only.
+- **A slot inside a tag pair is DATA** (`prompt-fence.ts`, #689): `render` breaks, in the ASSEMBLED
+  prompt, every fence closer a fenced value wrote any character of (never per value: two slots can
+  each hold half) — never deletes, never touches a template-only closer. The
+  fences are read once at `definePrompt`; no hash moves.
 - `definePrompt` refuses a re-registered version whose hash moved; the hash is over core's
   `canonicalJson` (`prompt.test.ts` pins one hash literally); an absent schema hashes as `''`.
 
 ## Invariants — providers and the gateway
 
+- **The response cache key is core's `fingerprint` over the request, each tool WHOLE** (every `llm()`
+  tool is `respond`); a throwing `cache.set` is logged (`ai.cache.write_failed`), never a failure.
+- **An unknown stop/finish reason FAILS CLOSED as `max_tokens` on both formats**, streamed or not.
+- **One failure-body reader** (`error-body.ts`'s `detailOf`, bounded at `READ_LIMIT`) and one
+  `withoutKey` scrub — the embedder included.
 - **`attempt` collects TRANSPORT failures only**: an `UltimateError` other than
   `X_AI_PROVIDER_UNAVAILABLE` is rethrown on the spot (the same misconfiguration everywhere). Read
   with `stringField(error, 'code')`.
@@ -126,7 +164,7 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
   stays UNREGISTERED on purpose.
 - **The retry SCHEDULE is core's (`backoffDelay`, `isRetryableStatus`); the LOOP is this package's** —
   core's `retry()` retries unclassified throws, and every value here is an app `Provider`'s plain
-  object, so it would retry a 400. `gateway-backoff.test.ts`. `RetryPolicy`/`DEFAULT_RETRY` keep the
+  object, so it would retry a 400. `gateway-backoff.test.ts`. `RetryPolicy`/`DEFAULT_GATEWAY_RETRY` keep the
   field names an app writes.
 - Every non-2xx and every in-band `error` frame is `AiTransportError` with a real `status`.
 - **Both wire formats answer the same question the same way** (`provider-parity.test.ts`): an in-band
@@ -139,15 +177,33 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
 - A tool call is emitted whole. Thinking chunks are never appended to `text`.
 - Anthropic body: no `temperature`/`top_p`/`top_k`, no `budget_tokens`, `effort` inside
   `output_config`. Model IDs are exact aliases, never date-suffixed.
+- **Apps bring their own models and providers (M12, 25.0.0): no vendor choice, no vendor data.** No
+  default model, no catalogue row, no id list ships; the registry starts EMPTY and `resetModels()`
+  empties it. A model resolves declaration → prompt → `Gateway.defaultModel` through `resolveModel`
+  / `modelForCall` only; none is `X_AI_MODEL_UNRESOLVED` naming all three (with no gateway and nothing
+  declared, `X_AI_GATEWAY_MISSING` — the third place does not exist yet). `vendor-neutral.test.ts`
+  proves it in a fresh process; no fix line names a vendor id.
 - **`ModelId` is `string`; the catalogue is an open registry** — `modelSpec(id)` refuses an
-  unregistered id (`X_AI_MODEL_UNKNOWN`). The built-ins register through the same `registerModel`.
-  **Re-registering REPLACES the spec and keeps its rung** (negotiated rates). **Registration order is
-  the ladder within a `family`**, read only by `moreCapableThan`; `X_LLM_REFUSED`'s fix names a rung
-  ABOVE or nothing.
-- `AnthropicProvider.models` is its own list; `EchoProvider.models` reads the registry.
+  unregistered id (`X_AI_MODEL_UNKNOWN`, fix `registerModel(…)`) at `costOf`, the budget estimate and
+  the body builders, before a socket or a reservation. **Re-registering REPLACES the spec and keeps
+  its rung** (negotiated rates). **Registration order is the ladder within a `family`**, read only by
+  `moreCapableThan`, over the APP's rows; `X_LLM_REFUSED`'s fix names a rung ABOVE or nothing.
+- **Every provider serves the app's list**: `anthropicProvider({ models })` and
+  `openAiProvider({ models })` require a non-empty one (`X_AI_REQUEST_INVALID`, a missing config
+  too); `echoProvider().models` reads the registry. **No provider has a model of its own**: a direct
+  call naming none is `X_AI_MODEL_UNRESOLVED` from all three — the list is what it serves, never a
+  fourth place (`vendor-neutral.test.ts`, one parity case).
 - **The reasoning controls are PER MODEL (`models.ts`)**: an unasked control is omitted; an asked
-  control the model lacks is `X_AI_REQUEST_INVALID`, never dropped. Adding a model is a row in `MODELS`.
+  control the model lacks is `X_AI_REQUEST_INVALID`, never dropped. Adding a model is a row the app
+  registers. `disableThinkingUpTo: 'never'` refuses `thinking: 'disabled'` at every effort;
+  `disabledThinking: 'between_tools'` is the wire spelling of the same declaration
+  (`model-rules.test.ts`, over the fixture rows).
+- **Media blocks are screened once, by `assertMediaContent`, at the top of BOTH body builders** —
+  malformed is `X_AI_REQUEST_INVALID`, untakeable (role, the row's `input`, the wire) is
+  `X_AI_CONTENT_UNSUPPORTED`; never dropped, never approximated (`provider-parity-content.test.ts`).
 - `generate()` above `STREAM_ONLY_MAX_TOKENS` runs the streaming transport.
+- **Every provider and embedder is built by its factory** (`anthropicProvider()`, `echoProvider()`,
+  `hashEmbedder()`, `remoteEmbedder()`); the classes are type exports only (`bun run factory-names`).
 - **`openAiProvider()` is a FORMAT, not a vendor** — `baseUrl`, `auth`, `headers` are the
   differences; never a second class per vendor:
   - structured output is the `respond` tool, never `response_format`; `tool_choice` names the tool
@@ -157,17 +213,16 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
     ESTIMATE when usage never arrives; subtract `cached_tokens` from `prompt_tokens`;
   - tool-call deltas merge by `tool_calls[].index`; `isComplete()` accepts `[DONE]` or a finish reason,
     except `[DONE]` with pending tool fragments, which is refused;
-  - only `gpt-5.6-sol` / `-terra` / `-luna` are priced; `registerOpenAiModels()` re-registers after
-    `resetModels()` (every `openai-*.test.ts` calls it in `beforeEach`);
-  - no new `X_*` code; `AiTransportError` takes the provider's `envVar`; the key is revealed late,
+  - no code of its own (a media refusal is the shared `X_AI_CONTENT_UNSUPPORTED`);
+    `AiTransportError` takes the provider's `envVar`; the key is revealed late,
     never stored, scrubbed from `detail`.
 - **`embedBatched` enforces the `Embedder` arity per batch** (`X_AI_EMBEDDER_INVALID` with both
-  counts). One `RemoteEmbedder` for every vendor; vectors L2-normalised; a wrong width is
+  counts). One `remoteEmbedder()` for every vendor; vectors L2-normalised; a wrong width is
   `X_VECTOR_DIM_MISMATCH`.
 - **A caught value is read with `renderThrowable`** (`remote-embedder.ts`, `wire.ts`,
   `openai-wire.ts`) — `scripts/error-render.ts` cannot see `catch` bindings.
 - **A caller's string is never an object KEY**: the wire tables are `Map`s; `prompt.render` and
-  `EchoProvider`'s `replies` use `Object.hasOwn`.
+  `echoProvider()`'s `replies` use `Object.hasOwn`.
 
 ## Invariants — `agent()`, `hive()`, `agentJob()`
 
@@ -188,10 +243,15 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
     ("superseded").
   - A tool result is rendered totally (`'null'` for `undefined`; a non-JSON result reported as such,
     never as a failure); the throw is read with `stringField`.
+  - **A 5xx code's cause reaches the model only when core's `hasPublicCause` allows it** (status from
+    `@ultimat3/http`'s `statusFor`) — else `CODE: HIDDEN_TOOL_CAUSE`, plus a branded `callerFix`. The
+    same verdict a problem document and `@ultimat3/mcp` give, from `failure-disclosure.ts` — the
+    rule a hive member's `reason` follows too. Never a second copy.
 - **`hive()` is a fan-out action** (`PRIMITIVE_FACTORIES`): `HiveResult` is a SCHEMA from the
   member's `output`; three arms (`ok`/`failed`/`skipped`) and three counters; `members` in SPLIT
-  order with `index`; the hive never names an actor; no hive budget code (the root turnstile holds
-  it); a member's throw is recorded via `isThrownError`/`stringField`; `skipped` reasons
+  order with `index`; the hive never names an actor; no hive budget code (the reservation holds
+  it); a member's throw is recorded through `discloseFailure` — a 5xx cause or an uncoded throw's
+  message becomes `HIDDEN_MEMBER_CAUSE`, logged as facts only; `skipped` reasons
   `SKIPPED_ABORTED` / `SKIPPED_NO_INPUT`; `onMemberError` is required; `concurrency` 4, `minMembers`
   2 (a below-floor split still runs serially); an empty split is `X_HIVE_EMPTY`; an aborted `ctx`
   unwinds with `X_ABORTED`.
@@ -206,15 +266,19 @@ local `=== true`. An in-app agent and an external one must be offered exactly th
 - **No fix line may name `x ai`** (planned). An eval is selected with `x test eval --filter <name>`;
   `eval-errors.test.ts` asserts every `x test <word>` is one of the six types.
 - Every eval result carries the prompt hash. An eval gates on the DROP from its recorded baseline,
-  mean AND per case. Never-recorded is `X_EVAL_BASELINE_MISSING`, corrupt is `X_EVAL_BASELINE_INVALID`.
+  mean AND per case, both sides rounded as the baseline is written. Never-recorded is `X_EVAL_BASELINE_MISSING`, corrupt is `X_EVAL_BASELINE_INVALID`.
   `baseline` is `import.meta.resolve('./…')`. Every prompt is named by an eval (`X_EVAL_MISSING`, by
   prompt ID). `ULTIMATE_EVAL_RECORD=1` records; a deliberately-worse test calls `run`, never
   `assert`; `x verify` with the flag is `X_EVAL_RECORDING` and runs no eval. The gate asks for a
   recorded baseline, not only a declaration.
-- Retrieval is hybrid by default; no vector-only path. **`chunk()` hard-wraps**, and the overlap
-  carry stops at `buffer.length - 1`.
-- `PgVectorStore` is the only production vector path (pgvector + FTS in the app's Postgres);
-  `MemoryVectorStore` enforces the same envelope. **`pg-vector.live.test.ts` refuses to skip** on a
+- Retrieval is hybrid by default; no vector-only path. **`chunk()` hard-wraps**, the overlap
+  carry stops at `buffer.length - 1` and yields to a next unit it does not fit beside (no chunk over
+  `size`); `metadata.source` is written AFTER the caller's spread. `VectorStore.prune` is required.
+- **A store with no tenant bound, read inside a request whose actor has an `orgId`, is
+  `X_VECTOR_UNSCOPED`** (`assertTenantRead`, both stores, every read). `scope: UNSCOPED`
+  (`crossTenant: true`) is the named opt-in; an omitted scope is `UNBOUND`.
+- `postgresVectorStore()` is the only production vector path (pgvector + FTS in the app's Postgres);
+  `memoryVectorStore()` enforces the same envelope. **`pg-vector.live.test.ts` refuses to skip** on a
   Postgres without the extension (CI uses `pgvector/pgvector:pg17`).
 - The distance ordering lives in an ascending raw subquery (the only shape hnsw answers), pinned by a
   plan assertion. hnsw scopes after the scan — assert rows, never the node; `analyze` after a backfill.