@ultimat3/ai 3.0.0 → 4.0.0

package/CLAUDE.md

@@ -61,6 +61,8 @@ until 2026-08, naming a tool no catalog contained (`llm.test.ts`, `agent.test.ts
 | `hive-errors.ts` | the `X_HIVE_*` class; its code and title stay in `errors.ts` |
 | `redaction.ts` | the one gate between `vars()` and the provider: a `Secret` never reaches a prompt |
 | `eval-errors.ts` | the five `X_EVAL_*` classes; their codes and titles stay in `errors.ts` |
+| `llm-cache.ts` | the semantic cache half of `llm()`: what a declaration may partition on, and the store it reaches |
+| `llm-fixture.ts` | the harness `llm.test.ts` and `llm-cache.test.ts` share. Not shipped (`!src/**/*-fixture.ts`) |
 | `runtime.ts` | the ambient gateway / embedder / semantic caches an `llm()` reaches |
 | `fix-line.ts` / `fix-line.evals.ts` / `fix-line.v1.baseline.json` / `fix-line.eval.test.ts` | the package's own dogfood eval — the first framework-level `*.eval.test.ts`, proving the `defineEval`/baseline convention actually fails a build |
 
@@ -78,6 +80,21 @@ until 2026-08, naming a tool no catalog contained (`llm.test.ts`, `agent.test.ts
 - Semantic scopes are separate cache INSTANCES, never a filter over a shared one — cosine
   similarity has no notion of a tenant. The instance key carries the prompt hash too, which is
   what makes a version bump invalidate the cache.
+- **The default scope is the calling ACTOR, and `scope` receives `{ input, ctx }`** (`As of
+  2026-08`). It defaulted to the literal string `'global'` and took the bare `input`, so the rule
+  in the bullet above was contradicted by the very default that shipped: `cache: { semantic: { ttl:
+  '1h' } }` put every tenant in one store, and `lookup` is a nearest neighbour with no tenant
+  predicate — proven by execution, tenant B asking a prompt within 0.92 cosine of tenant A's
+  received A's completion verbatim. And with only `input` to decide from, the one thing a partition
+  may never be chosen by (a value the caller sends) was the only thing it could be chosen by, while
+  `vars()` on the same declaration already took the pair. The default is
+  `JSON.stringify([actor.kind, actor.id, actor.orgId ?? null])` — `@ultimat3/query`'s
+  `readAuthority` rule, verbatim: a declaration that says nothing gets the NARROWEST key, and
+  widening is a written statement about what the answers are (`scope: () => 'global'`). **Breaking**
+  in both halves. `semanticCacheFor`'s instance map is bounded as a consequence
+  (`MAX_SEMANTIC_CACHE_SCOPES`, which IS core's `MAX_CACHED_FORMATTERS` — one bounded FIFO map in
+  the framework, whose name is about its first caller and not its contract): one entry per actor in
+  a process that never restarts is a leak where one entry per process was not.
 - A per-call budget `derive`s from the ambient ledger, so it can only TIGHTEN the actor and org
   ceilings it runs inside. Widening them from a declaration would be a budget that is not one.
   **A derived ledger reports back up the chain**: every debit and every recorded cost lands on it
@@ -135,6 +152,15 @@ until 2026-08, naming a tool no catalog contained (`llm.test.ts`, `agent.test.ts
   *as* differences rather than flattening them.
 - A stream that ends without `message_stop` throws. A truncated answer that returns `end_turn`
   is a confidently wrong answer with no signal, which the budget rule already forbids.
+- **`readSse` caps the unterminated buffer** (`MAX_FRAME_CHARS`, `As of 2026-08`) and refuses with
+  `AiTransportError`. A peer that never sends a frame boundary — an HTML error page, a proxy on the
+  model's port — grew it without limit and no read deadline interrupted it, because every read
+  SUCCEEDED. Same call `@ultimat3/mail`'s `createReplyParser` makes: coded failure > OOM. `provider`
+  is a required argument for that reason — a transport error names the endpoint it is about.
+- **`llm()` forwards `ctx.signal` onto `GenerateRequest`**, `As of 2026-08`, the way `agent()`
+  always did. Without it a model call had no cancellation and no deadline: a caller that hung up
+  left the provider call in flight, billed and unread, and the repair turn bought a second one.
+  `.stream()` inherits it from the same `base`, which is where it matters most.
 - A tool call is emitted whole. `input_json_delta` fragments are not arguments until the block
   closes, so nothing partial reaches a caller.
 - Thinking chunks are never appended to `text`. A consumer concatenating every chunk must not
@@ -217,7 +243,13 @@ until 2026-08, naming a tool no catalog contained (`llm.test.ts`, `agent.test.ts
     per-block stop event in this format.
   - `isComplete()` accepts `[DONE]` **or** a finish reason: plenty of servers in the family close
     the socket straight after the finish chunk, and a finish reason is the model saying why it
-    stopped, which a cut connection cannot produce.
+    stopped, which a cut connection cannot produce. **With one exception, `As of 2026-08`:
+    `[DONE]` while tool-call fragments are still pending is NOT complete.** `onFinish` is the only
+    drain of `pending` and the finish reason is the only close this format has, so the sentinel
+    alone cannot tell "the model finished asking" from "the connection died mid-arguments" —
+    reporting complete discarded a whole tool call and answered an empty, successful `end_turn`.
+    Refused rather than flushed, exactly as the Anthropic half refuses a missing `message_stop`:
+    emitting the fragments would run a tool's side effects from half a JSON object.
   - `role: 'system'`, not `developer` — the newer role is OpenAI's alone and every other server in
     the family knows only `system`.
   - **Only three models are priced** (`gpt-5.6-sol` / `-terra` / `-luna`, list price read
@@ -481,6 +513,17 @@ until 2026-08, naming a tool no catalog contained (`llm.test.ts`, `agent.test.ts
   recorded — one no test asserts, one whose `baseline:` is a cwd-relative string — would otherwise
   satisfy `X_EVAL_MISSING` while gating on nothing.
 - Retrieval is hybrid by default. Do not add a vector-only convenience path.
+- **`chunk()` performs all three splits its header names — paragraph, sentence, HARD WRAP** (`As of
+  2026-08`). The wrap is what bounds a unit, and an oversized unit is one the size check can never
+  flush (it only fires when something is already in the buffer), so it re-seeded every following
+  chunk: a ~1,000-token document indexed as nine chunks totalling ~9,000, each carrying the same
+  sentence. The overlap carry stops at `buffer.length - 1` for the same reason — a flushed unit may
+  never become the whole of the next buffer.
+- **A caller's string is never used as an object KEY.** A `Record` lookup on one answers
+  `constructor`, `toString` and `valueOf` off the prototype chain, and every read here is of a
+  string a provider, a template author or a test fixture chose: the two wire tables are `Map`s,
+  `prompt.render` uses `Object.hasOwn` (a `{{constructor}}` slot rendered JS source into a billed
+  prompt and hashed it into the cache key), and so does `EchoProvider`'s `replies`.
 - `PgVectorStore` is the ONLY production vector path — pgvector and Postgres FTS in the app's own
   Postgres, never a second datastore. `MemoryVectorStore` is the dev twin and enforces the same
   envelope; a leak that only reproduces against real Postgres is a leak nobody finds.

package/README.md

@@ -108,6 +108,8 @@ for await (const chunk of ai.stream({ messages, maxTokens: 64_000 })) {
 | A `tool-call` chunk arrives whole | `input_json_delta` fragments are not arguments until the block closes |
 | `thinking` chunks never join `text` | concatenating every chunk must not ship the reasoning to the user |
 | A stream cut before `message_stop` **throws** | a truncated answer reporting `end_turn` is wrong with no signal |
+| `[DONE]` with a tool call still open **throws** | the OpenAI format has no per-call stop event, so the finish reason is the only close there is; the sentinel alone cannot tell "finished asking" from "cut mid-arguments" |
+| A body with no frame boundary in it **throws** | an SSE peer that never completes a frame is an unbounded allocation no read deadline interrupts |
 | An in-band `error` frame carries a status | `overloaded_error` mid-stream retries like a 529 on the handshake |
 
 ## Embeddings
@@ -249,7 +251,7 @@ export const summarize = llm({
   output: t.object({ summary: t.string, tags: t.array(t.string) }),
   prompt: summarizePrompt,                                       // versioned artifact
   vars:   async ({ input, ctx }) => ({ body: await ctx.posts.body(input.postId) }),
-  cache:  { semantic: { threshold: 0.97, ttl: '7d', scope: ({ orgId }) => orgId } },
+  cache:  { semantic: { threshold: 0.97, ttl: '7d' } },   // scope defaults to the ACTOR
   budget: { tokensIn: 8_000, costPerCall: { minor: 5, currency: 'USD' } },
   policy: can('post:read'),
 });
@@ -265,7 +267,7 @@ summarize.contract();    // the contract tests
 | `output` | projected into the one tool the model may answer through; prose with a fenced JSON block still parses |
 | a schema failure | **one** repair turn naming the issues, then `X_LLM_OUTPUT_INVALID` |
 | `budget` | reserved against the worst case **before** the provider is reached — nothing spent, nothing truncated |
-| `cache.semantic` | one store per scope, keyed by embedding; a prompt version bump reaches a different store, so the bump *is* the invalidation |
+| `cache.semantic` | one store per scope, keyed by embedding; a prompt version bump reaches a different store, so the bump *is* the invalidation. `scope` receives `{ input, ctx }` and **defaults to the calling actor** — the narrowest key, `@ultimat3/query`'s `readAuthority` rule; a shared store is `scope: () => 'global'`, written down |
 | `policy` | the same object every surface evaluates — an MCP call and an HTTP call are denied identically |
 | `vars` | the one declared place a model call loads data, so a reader can see what was sent — and the one place a redactor sees it, and where a `Secret` is refused |
 
@@ -600,7 +602,12 @@ nothing. `scoped()` only ever **tightens** — re-scoping to a different tenant
 `X_VECTOR_SCOPE_WIDENED`, never a silent widening.
 
 `chunk()` is token-aware with overlap and splits at paragraph, then sentence, then hard wrap
-— a fact split across a boundary with no overlap is retrievable by neither chunk.
+— a fact split across a boundary with no overlap is retrievable by neither chunk. All three
+splits are load-bearing: the wrap is what bounds a UNIT (a base64 blob, a minified line, a CJK
+paragraph the sentence alphabet cannot see), and a unit larger than `size` is one the size check
+can never flush, so it rode every chunk after it — `As of 2026-08`, a ~1,000-token document
+indexed as nine chunks of the same sentence. The overlap carries a tail forward and never the
+whole buffer, for the same reason.
 
 ## Tools: the same projection as MCP
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ultimat3/ai",
-  "version": "3.0.0",
+  "version": "4.0.0",
   "description": "LLM gateway, versioned prompts, evals as tests, embeddings, hybrid vector search, RAG",
   "license": "MIT",
   "type": "module",
@@ -19,6 +19,7 @@
   "files": [
     "src",
     "!src/**/*.test.ts",
+    "!src/**/*-fixture.ts",
     "CLAUDE.md",
     "README.md",
     "LICENSE"
@@ -31,14 +32,14 @@
     "test": "bun test"
   },
   "dependencies": {
-    "@ultimat3/action": "3.0.0",
-    "@ultimat3/cache": "3.0.0",
-    "@ultimat3/core": "3.0.0",
-    "@ultimat3/db": "3.0.0",
-    "@ultimat3/jobs": "3.0.0",
-    "@ultimat3/money": "3.0.0",
-    "@ultimat3/policy": "3.0.0",
-    "@ultimat3/schema": "3.0.0",
-    "@ultimat3/time": "3.0.0"
+    "@ultimat3/action": "4.0.0",
+    "@ultimat3/cache": "4.0.0",
+    "@ultimat3/core": "4.0.0",
+    "@ultimat3/db": "4.0.0",
+    "@ultimat3/jobs": "4.0.0",
+    "@ultimat3/money": "4.0.0",
+    "@ultimat3/policy": "4.0.0",
+    "@ultimat3/schema": "4.0.0",
+    "@ultimat3/time": "4.0.0"
   }
 }

package/src/fetch-seam.ts

@@ -0,0 +1,15 @@
+// Single responsibility: the one injectable HTTP call every transport in this package takes.
+//
+// Shared by all three rather than declared three times: both chat providers and the embedder hand
+// a URL and a `RequestInit` to something that answers a `Response`, and three separate spellings
+// of that is three places a test double has to be kept assignable to.
+
+/**
+ * Just the call. `typeof fetch` also carries `preconnect`, which no test double should have to —
+ * and none can supply, so every fake written against `typeof fetch` here needed
+ * `as unknown as typeof fetch` to compile: an option no caller could fill without a double cast.
+ *
+ * The same seam `@ultimat3/cache` (`PurgeFetch`), `@ultimat3/auth` (`OAuthFetch`),
+ * `@ultimat3/mail` (`MailFetch`) and `@ultimat3/scraping` (`ScrapeFetch`) already name.
+ */
+export type AiFetch = (input: string, init: RequestInit) => Promise<Response>;

package/src/gateway.ts

@@ -127,8 +127,14 @@ class GatewayImpl implements Gateway {
     const ledger = currentBudget();
     const reservation = await ledger?.reserve(estimateSpend(resolved));
 
-    // A stream is not retried mid-flight: the consumer has already seen tokens, and
-    // replaying from the top would duplicate them. Only the handshake retries.
+    // The streaming path does not retry AT ALL — not mid-flight, and not on the handshake either.
+    // Mid-flight is the obvious one: the consumer has already been handed tokens, and replaying
+    // from the top would duplicate them. The handshake is not separable from it here, because
+    // `provider.stream()` is one call that yields — there is no point at which the connection is
+    // open and no chunk has been delivered for a retry to hide behind. So `providerFor` picks the
+    // single provider serving this model and that call stands or throws; `attempt`'s backoff and
+    // its fallback across providers belong to `generate` alone. A caller that wants either uses
+    // `generate`, or reconnects itself and knows what it has already shown.
     const provider = this.providerFor(model);
     let settled = false;
     try {

package/src/index.ts

@@ -91,21 +91,16 @@ export {
   promptsWithoutEvals,
   resetEvals,
 } from './evals';
+export type { AiFetch } from './fetch-seam';
 export type { CreateGatewayInput, Gateway, GatewayCache, RetryPolicy } from './gateway';
 export { backoffMs, cacheKeyFor, createGateway, DEFAULT_RETRY, isRetryable } from './gateway';
 export type { HiveDef, HiveSplitArgs } from './hive';
 export { hive } from './hive';
 export { HiveEmptyError } from './hive-errors';
 export type { HiveMember, HiveMemberError, HiveOutput, HiveResult } from './hive-result';
-export type {
-  LlmAction,
-  LlmBudget,
-  LlmCache,
-  LlmDef,
-  LlmSemanticCache,
-  LlmVarsArgs,
-} from './llm';
+export type { LlmAction, LlmBudget, LlmDef, LlmVarsArgs } from './llm';
 export { llm } from './llm';
+export type { LlmCache, LlmScopeArgs, LlmSemanticCache } from './llm-cache';
 export type { LlmStreamChunk } from './llm-stream';
 export type { Effort, ModelId, ModelReasoning, ModelSpec, ThinkingMode } from './models';
 export {
@@ -196,6 +191,7 @@ export {
   aiGateway,
   aiRedactor,
   configureAi,
+  MAX_SEMANTIC_CACHE_SCOPES,
   resetAiRuntime,
   semanticCacheFor,
 } from './runtime';

package/src/llm-cache.ts

@@ -0,0 +1,111 @@
+/**
+ * The semantic cache half of `llm()`: what a declaration may partition on, and the store one
+ * declaration reaches. Split from `llm.ts` so that file stays the model call itself.
+ *
+ * A scope is a separate cache INSTANCE, never a filter over a shared one, and the instance key
+ * carries the prompt VERSION as well — which is what makes "editing a prompt requires a version
+ * bump" invalidate the cache: a bumped version reaches a different store, so an old answer cannot
+ * survive a prompt edit no matter how similar the text.
+ */
+
+import type { Ctx } from '@ultimat3/core';
+import { parseDuration } from '@ultimat3/time';
+import { embedOne, fnv1a } from './embeddings';
+import { aiEmbedder, semanticCacheFor } from './runtime';
+
+/** What a `scope` may decide from. The same pair `vars()` receives, and for the same reason. */
+export interface LlmScopeArgs<TParsed> {
+  readonly input: TParsed;
+  readonly ctx: Ctx;
+}
+
+export interface LlmSemanticCache<TParsed> {
+  /** Cosine floor. Below ~0.9 unrelated prompts collide and the cache answers the wrong one. */
+  readonly threshold?: number;
+  /** Entry lifetime as a duration string — `'7d'`, `'12h'`. `@ultimat3/time` owns the grammar. */
+  readonly ttl?: string;
+  /**
+   * Partition key. Each scope is a separate cache instance, never a filter over a shared one:
+   * cosine similarity has no notion of a tenant, so a shared cache answers one tenant with
+   * another's data — by construction, since `lookup` is a nearest neighbour with no predicate.
+   *
+   * **Omitting it is safe.** The default is the narrowest key the ctx can supply — the actor,
+   * its kind and its org — which is `@ultimat3/query`'s `readAuthority` rule applied here: a
+   * declaration that says nothing gets the narrowest partition, which is always correct, and
+   * WIDENING is a written statement about what the answers are. It defaulted to `'global'`, so a
+   * `cache: { semantic: { ttl: '1h' } }` put every tenant in one store.
+   *
+   * **Breaking: it receives `{ input, ctx }`, not the bare `input`.** Taking `input` alone meant
+   * the partition could only be chosen from a value the caller sends, which is the one thing a
+   * partition may never be chosen by. `vars()` on the same declaration already takes the pair.
+   */
+  readonly scope?: (args: LlmScopeArgs<TParsed>) => string;
+}
+
+export interface LlmCache<TParsed> {
+  readonly semantic: LlmSemanticCache<TParsed>;
+  // `05-caching.md` also declares `invalidates: [tag.post]` here. It is deliberately absent
+  // until `@ultimat3/cache`'s fan-out can reach something that is not a `CacheTier`: storing
+  // tags that the ONE invalidation path never visits would read as wired and silently not be.
+  // Today the invalidation story is the prompt version (a bump reaches a different store) and
+  // `ttl`.
+}
+
+export interface PromptCache {
+  lookup(): Promise<unknown>;
+  remember(value: unknown): Promise<void>;
+}
+
+/**
+ * The partition a declaration that named none gets: the narrowest key this call can prove, which
+ * is the actor itself. Verbatim the shape `@ultimat3/query`'s `actorAuthority` builds, for the
+ * same reason it is JSON rather than a joined string — an actor id is app data and may carry any
+ * separator, and a value that can spell a boundary can spell somebody else's.
+ *
+ * `ctx.actor` is never absent (`createContext` defaults it to `anonymousActor()`), so every
+ * anonymous caller shares one partition — which is what the anonymous actor already means
+ * everywhere else in the framework.
+ */
+function actorScope(ctx: Ctx): string {
+  return JSON.stringify([ctx.actor.kind, ctx.actor.id, ctx.actor.orgId ?? null]);
+}
+
+/**
+ * The semantic cache for one declaration, or `undefined` when none was declared. The instance
+ * is partitioned by prompt VERSION as well as scope, which is what makes "editing a prompt
+ * requires a version bump" invalidate the cache: a bumped version reaches a different store,
+ * so an old answer cannot survive a prompt edit no matter how similar the text.
+ */
+export interface OpenCacheArgs<TParsed> {
+  readonly cache: LlmCache<TParsed> | undefined;
+  /** Identity and content hash of the prompt artifact — the version half of the partition. */
+  readonly prompt: { readonly ref: string; readonly hash: string };
+  readonly input: TParsed;
+  readonly ctx: Ctx;
+  /** The rendered, redacted prompt text: what is embedded and what the entry is keyed on. */
+  readonly rendered: string;
+}
+
+export async function openCache<TParsed>(
+  args: OpenCacheArgs<TParsed>,
+): Promise<PromptCache | undefined> {
+  const semantic = args.cache?.semantic;
+  if (semantic === undefined) return undefined;
+  const scope = semantic.scope?.({ input: args.input, ctx: args.ctx }) ?? actorScope(args.ctx);
+  const store = semanticCacheFor(`${args.prompt.ref}#${args.prompt.hash}::${scope}`);
+  const embedding = Array.from(await embedOne(aiEmbedder(), args.rendered));
+  const ttlMs = semantic.ttl === undefined ? undefined : parseDuration(semantic.ttl);
+  return {
+    async lookup(): Promise<unknown> {
+      return (await store.lookup(embedding, semantic.threshold))?.value;
+    },
+    remember(value: unknown): Promise<void> {
+      return store.remember(
+        `${args.prompt.hash}:${fnv1a(args.rendered).toString(16)}`,
+        embedding,
+        value,
+        ttlMs === undefined ? {} : { ttlMs },
+      );
+    },
+  };
+}

package/src/llm.ts

@@ -28,10 +28,8 @@ import { withSpan } from '@ultimat3/core';
 import type { Money } from '@ultimat3/money';
 import type { InferInput, InferOutput, StandardSchemaV1 } from '@ultimat3/schema';
 import { formatIssues, toMcpInputSchema, validateAsync } from '@ultimat3/schema';
-import { parseDuration } from '@ultimat3/time';
 import type { BudgetLimits } from './budget';
 import { BudgetLedger, currentBudget, withBudget } from './budget';
-import { embedOne, fnv1a } from './embeddings';
 import {
   LlmOutputInvalidError,
   LlmRefusedError,
@@ -39,6 +37,8 @@ import {
   LlmTruncatedError,
 } from './errors';
 import type { Gateway } from './gateway';
+import type { LlmCache } from './llm-cache';
+import { openCache } from './llm-cache';
 import type { LlmSink, LlmStreamChunk } from './llm-stream';
 import { currentLlmSink, llmStream, streamOneTurn, withLlmSink } from './llm-stream';
 import type { ModelId } from './models';
@@ -46,7 +46,7 @@ import { DEFAULT_MODEL, moreCapableThan } from './models';
 import type { Prompt, PromptVars } from './prompt';
 import type { AiMessage, GenerateRequest, GenerateResult } from './provider';
 import { assertNoSecrets } from './redaction';
-import { aiEmbedder, aiGateway, aiRedactor, semanticCacheFor } from './runtime';
+import { aiGateway, aiRedactor } from './runtime';
 import type { LlmTool } from './tools';
 
 /**
@@ -65,27 +65,6 @@ const ATTEMPTS = 2;
  */
 const DEFAULT_MAX_TOKENS = 4_096;
 
-export interface LlmSemanticCache<TParsed> {
-  /** Cosine floor. Below ~0.9 unrelated prompts collide and the cache answers the wrong one. */
-  readonly threshold?: number;
-  /** Entry lifetime as a duration string — `'7d'`, `'12h'`. `@ultimat3/time` owns the grammar. */
-  readonly ttl?: string;
-  /**
-   * Partition key, from the parsed input. Each scope is a separate cache: cosine similarity
-   * has no notion of a tenant, so a shared cache answers one tenant with another's data.
-   */
-  readonly scope?: (input: TParsed) => string;
-}
-
-export interface LlmCache<TParsed> {
-  readonly semantic: LlmSemanticCache<TParsed>;
-  // `05-caching.md` also declares `invalidates: [tag.post]` here. It is deliberately absent
-  // until `@ultimat3/cache`'s fan-out can reach something that is not a `CacheTier`: storing
-  // tags that the ONE invalidation path never visits would read as wired and silently not be.
-  // Today the invalidation story is the prompt version (a bump reaches a different store) and
-  // `ttl`.
-}
-
 /** Per-call ceilings, checked before the provider is reached. Never truncates — refuses. */
 export interface LlmBudget {
   /** Prompt tokens. */
@@ -235,7 +214,13 @@ async function generate<
     // A cached answer is still data of unknown provenance, so it goes through the schema like
     // any other. One that no longer fits — the schema moved under it — is a miss, not a
     // failure: the model can produce a fresh answer, and refusing would be worse than paying.
-    const cache = await openCache(def, args.input, rendered);
+    const cache = await openCache({
+      cache: def.cache,
+      prompt: { ref: prompt.ref, hash: prompt.hash },
+      input: args.input,
+      ctx: args.ctx,
+      rendered,
+    });
     const hit = await accept(def.output, await cache?.lookup());
     span.setAttribute('llm.cache.hit', hit !== undefined);
     if (hit !== undefined) return hit.value;
@@ -250,6 +235,10 @@ async function generate<
       maxTokens: def.maxTokens ?? DEFAULT_MAX_TOKENS,
       ...(prompt.effort === undefined ? {} : { effort: prompt.effort }),
       ...(prompt.thinking === undefined ? {} : { thinking: prompt.thinking }),
+      // The caller's own signal, forwarded to the transport exactly as `agent()` does — and
+      // inherited by `streamedAnswer` from this same `base`. Without it a disconnected caller left
+      // the provider call in flight, billed and unread, and the repair turn bought a SECOND one.
+      signal: args.ctx.signal,
     };
     const request: GenerateRequest = { ...base, tools: [respond] };
 
@@ -447,44 +436,3 @@ function parseJsonish(text: string): unknown {
     return undefined;
   }
 }
-
-interface PromptCache {
-  lookup(): Promise<unknown>;
-  remember(value: unknown): Promise<void>;
-}
-
-/**
- * The semantic cache for one declaration, or `undefined` when none was declared. The instance
- * is partitioned by prompt VERSION as well as scope, which is what makes "editing a prompt
- * requires a version bump" invalidate the cache: a bumped version reaches a different store,
- * so an old answer cannot survive a prompt edit no matter how similar the text.
- */
-async function openCache<
-  TInput extends StandardSchemaV1,
-  TOutput extends StandardSchemaV1,
-  V extends PromptVars,
->(
-  def: LlmDef<TInput, TOutput, V>,
-  input: InferOutput<TInput>,
-  rendered: string,
-): Promise<PromptCache | undefined> {
-  const semantic = def.cache?.semantic;
-  if (semantic === undefined) return undefined;
-  const scope = semantic.scope?.(input) ?? 'global';
-  const store = semanticCacheFor(`${def.prompt.ref}#${def.prompt.hash}::${scope}`);
-  const embedding = Array.from(await embedOne(aiEmbedder(), rendered));
-  const ttlMs = semantic.ttl === undefined ? undefined : parseDuration(semantic.ttl);
-  return {
-    async lookup(): Promise<unknown> {
-      return (await store.lookup(embedding, semantic.threshold))?.value;
-    },
-    remember(value: unknown): Promise<void> {
-      return store.remember(
-        `${def.prompt.hash}:${fnv1a(rendered).toString(16)}`,
-        embedding,
-        value,
-        ttlMs === undefined ? {} : { ttlMs },
-      );
-    },
-  };
-}

package/src/openai-provider.ts

@@ -11,6 +11,7 @@ import type { Secret } from '@ultimat3/core';
 import { isSecret, revealSecret } from '@ultimat3/core';
 import { detailOf, withoutKey } from './error-body';
 import { AiKeyMissingError, AiRequestInvalidError, AiTransportError } from './errors';
+import type { AiFetch } from './fetch-seam';
 import type { ModelId } from './models';
 import { chatCompletionBody } from './openai-body';
 // Imported for its registration side effect: a provider that cannot price what it serves throws
@@ -66,7 +67,7 @@ export interface OpenAiProviderInput {
    */
   readonly name?: string;
   /** Injectable so a test can assert the request body without a network. Defaults to `fetch`. */
-  readonly fetch?: typeof fetch;
+  readonly fetch?: AiFetch;
 }
 
 /**
@@ -138,7 +139,7 @@ class OpenAiProvider implements Provider {
       });
     }
     const completion = new ChatCompletionStream(this.name);
-    for await (const frame of readSse(response.body)) {
+    for await (const frame of readSse(response.body, this.name)) {
       for (const chunk of completion.push(frame)) yield chunk;
     }
     // A connection cut mid-answer must fail, not resolve: partial text reads as a complete answer,
@@ -197,7 +198,7 @@ class OpenAiProvider implements Provider {
     signal: AbortSignal | undefined,
   ): Promise<Response> {
     const apiKey = this.apiKey();
-    const doFetch = this.config.fetch ?? fetch;
+    const doFetch: AiFetch = this.config.fetch ?? fetch;
     const response = await doFetch(this.url(), {
       method: 'POST',
       headers: {

package/src/openai-wire.ts

@@ -24,34 +24,46 @@ export interface ChatAnswer {
   readonly usage: TokenUsage | undefined;
 }
 
-const FINISH_REASONS: Readonly<Record<string, StopReason>> = {
-  stop: 'end_turn',
-  length: 'max_tokens',
-  tool_calls: 'tool_use',
-  // The legacy name for the same event; LiteLLM and older self-hosted servers still send it.
-  function_call: 'tool_use',
-  content_filter: 'refusal',
-};
+// A `Map`, not an object literal: `raw` is the PROVIDER's string on every read, and
+// `FINISH_REASONS['constructor']` on an object answers the `Object` FUNCTION — which
+// `parseFinishReason` returned as a `StopReason`, and which the stream reader below then treated
+// as a finish, so `isComplete()` answered true for a stream that never finished. Same fix core
+// made in `error-retry.ts` for the same shape.
+const FINISH_REASONS: ReadonlyMap<string, StopReason> = new Map(
+  Object.entries({
+    stop: 'end_turn',
+    length: 'max_tokens',
+    tool_calls: 'tool_use',
+    // The legacy name for the same event; LiteLLM and older self-hosted servers still send it.
+    function_call: 'tool_use',
+    content_filter: 'refusal',
+  } as const),
+);
 
 /**
  * In-band error frames carry a type, not a status, and the gateway's retry rule reads a status.
  * Same mapping job as wire.ts's, over this format's own vocabulary.
  */
-const ERROR_STATUS: Readonly<Record<string, number>> = {
-  invalid_request_error: 400,
-  authentication_error: 401,
-  permission_error: 403,
-  not_found_error: 404,
-  rate_limit_exceeded: 429,
-  insufficient_quota: 429,
-  server_error: 500,
-  api_error: 500,
-  overloaded_error: 503,
-};
+// A `Map`, for the reason `FINISH_REASONS` above is one: `type` is the provider's string, and a
+// function where `AiTransportError.status` is declared `number | undefined` makes `isRetryable`
+// answer false for a 429-class frame and interpolates JS source into the operator-facing `cause`.
+const ERROR_STATUS: ReadonlyMap<string, number> = new Map(
+  Object.entries({
+    invalid_request_error: 400,
+    authentication_error: 401,
+    permission_error: 403,
+    not_found_error: 404,
+    rate_limit_exceeded: 429,
+    insufficient_quota: 429,
+    server_error: 500,
+    api_error: 500,
+    overloaded_error: 503,
+  }),
+);
 
 /** A finish reason this format knows, or `undefined` for `null` — which means "still going". */
 export function parseFinishReason(raw: unknown): StopReason | undefined {
-  return typeof raw === 'string' ? FINISH_REASONS[raw] : undefined;
+  return typeof raw === 'string' ? FINISH_REASONS.get(raw) : undefined;
 }
 
 /**
@@ -64,10 +76,10 @@ export function parseFinishReason(raw: unknown): StopReason | undefined {
 export function parseOpenAiUsage(raw: unknown): TokenUsage | undefined {
   const record = asRecord(raw);
   if (record === undefined) return undefined;
-  const prompt = numberOf(record['prompt_tokens']);
-  const completion = numberOf(record['completion_tokens']);
+  const prompt = countOf(record['prompt_tokens']);
+  const completion = countOf(record['completion_tokens']);
   if (prompt === undefined && completion === undefined) return undefined;
-  const cached = numberOf(asRecord(record['prompt_tokens_details'])?.['cached_tokens']) ?? 0;
+  const cached = countOf(asRecord(record['prompt_tokens_details'])?.['cached_tokens']) ?? 0;
   return {
     inputTokens: Math.max((prompt ?? 0) - cached, 0),
     // `completion_tokens` already contains `reasoning_tokens`; adding them is a double count.
@@ -194,7 +206,17 @@ export class ChatCompletionStream {
     // Either sentinel counts. `[DONE]` is the format's own end marker, but plenty of servers in
     // the family close the socket straight after the finish-reason chunk — and a finish reason IS
     // the model saying why it stopped, which is the fact a truncated stream cannot produce.
-    return this.done || this.finished;
+    //
+    // One exception, and it is REFUSAL rather than a flush: `[DONE]` while tool-call fragments are
+    // still open. This format has no per-call stop event, so the finish reason is the only thing
+    // that ever closes a call and `onFinish` is the only drain of `pending` — which means `[DONE]`
+    // alone cannot tell "the model finished asking" from "the connection died mid-arguments".
+    // Reporting complete discarded a whole tool call and answered an empty, successful `end_turn`;
+    // emitting the fragments anyway would run a tool's side effects from arguments that may be
+    // half a JSON object, and would report `end_turn` for a turn that stopped to call one. So the
+    // stream is refused exactly as the Anthropic half refuses a missing `message_stop`
+    // (`provider.ts`), and `openai-provider.ts`'s existing truncation guard is what raises it.
+    return this.finished || (this.done && this.pending.size === 0);
   }
 
   /** What the stream accumulated. `cost` is applied by the provider, which owns prices. */
@@ -320,7 +342,8 @@ function throwInBandError(payload: Record<string, unknown>, provider: string): v
   const message = typeof error['message'] === 'string' ? error['message'] : type;
   throw new AiTransportError({
     provider,
-    status: ERROR_STATUS[type] ?? (code === undefined ? undefined : ERROR_STATUS[code]) ?? 500,
+    status:
+      ERROR_STATUS.get(type) ?? (code === undefined ? undefined : ERROR_STATUS.get(code)) ?? 500,
     detail: message,
   });
 }
@@ -337,3 +360,14 @@ function asRecord(value: unknown): Record<string, unknown> | undefined {
 function numberOf(value: unknown): number | undefined {
   return typeof value === 'number' && Number.isFinite(value) ? value : undefined;
 }
+
+/**
+ * A token count, floored at zero. Usage is the PROVIDER's number and a proxy in front of one can
+ * send anything: a negative count becomes a negative `cost`, and `MemoryBudgetStore.add` takes a
+ * negative debit as a credit deliberately (releasing an unspent reservation IS one) — so an
+ * unclamped `-1` here does not under-report spend, it TOPS THE LEDGER UP. Twin of `wire.ts`'s.
+ */
+function countOf(value: unknown): number | undefined {
+  const count = numberOf(value);
+  return count === undefined ? undefined : Math.max(0, count);
+}

package/src/prompt.ts

@@ -120,7 +120,12 @@ const PLACEHOLDER = /\{\{\s*([a-zA-Z0-9_]+)\s*\}\}/g;
 function render(template: string, vars: PromptVars, ref: string): string {
   const missing: string[] = [];
   const out = template.replace(PLACEHOLDER, (_match, name: string) => {
-    const value = vars[name];
+    // `Object.hasOwn`, never `vars[name] === undefined`: a plain object inherits `constructor`,
+    // `toString` and `valueOf`, so `{{constructor}}` in a template rendered JS SOURCE into the
+    // prompt instead of raising the unfilled-slot error this file promises — and that source was
+    // then hashed into the semantic cache key and paid for at the input rate. The discriminator
+    // `@ultimat3/flags`' `subject.ts` already uses, for the same reason.
+    const value = Object.hasOwn(vars, name) ? vars[name] : undefined;
     if (value === undefined) {
       missing.push(name);
       return '';

package/src/provider.ts

@@ -6,6 +6,7 @@
 import type { Money } from '@ultimat3/money';
 import { detailOf, withoutKey } from './error-body';
 import { AiKeyMissingError, AiTransportError } from './errors';
+import type { AiFetch } from './fetch-seam';
 import type { Effort, ModelId, ThinkingMode } from './models';
 import { ANTHROPIC_MODEL_IDS, DEFAULT_MODEL, modelIds, modelSpec, reasoningBody } from './models';
 import { readSse } from './sse';
@@ -180,7 +181,7 @@ export interface AnthropicProviderInput {
   readonly apiKey?: string;
   readonly baseUrl?: string;
   /** Injectable so a test can assert the request body without a network. Defaults to `fetch`. */
-  readonly fetch?: typeof fetch;
+  readonly fetch?: AiFetch;
 }
 
 const ANTHROPIC_VERSION = '2023-06-01';
@@ -257,7 +258,7 @@ export class AnthropicProvider implements Provider {
       });
     }
     const message = new MessageStream();
-    for await (const frame of readSse(response.body)) {
+    for await (const frame of readSse(response.body, this.name)) {
       for (const chunk of message.push(frame)) yield chunk;
     }
     // A connection cut mid-answer must fail, not resolve: the partial text reads as a complete
@@ -303,7 +304,7 @@ export class AnthropicProvider implements Provider {
     if (apiKey === undefined || apiKey === '') {
       throw new AiKeyMissingError({ provider: this.name, envVar: API_KEY_ENV });
     }
-    const doFetch = this.config.fetch ?? fetch;
+    const doFetch: AiFetch = this.config.fetch ?? fetch;
     const url = `${this.config.baseUrl ?? 'https://api.anthropic.com'}/v1/messages`;
     const response = await doFetch(url, {
       method: 'POST',
@@ -408,7 +409,7 @@ export class EchoProvider implements Provider {
   async generate(request: GenerateRequest): Promise<GenerateResult> {
     const model = request.model ?? DEFAULT_MODEL;
     const prompt = lastUserMessage(request.messages);
-    const text = this.config.replies?.[prompt] ?? this.config.fallback?.(prompt) ?? prompt;
+    const text = this.fixedReply(prompt) ?? this.config.fallback?.(prompt) ?? prompt;
     const usage: TokenUsage = {
       inputTokens: this.config.tokensPerCall ?? estimateTokens(request),
       outputTokens: estimateTextTokens(text),
@@ -426,6 +427,17 @@ export class EchoProvider implements Provider {
     };
   }
 
+  /**
+   * The fixture reply for this prompt. `Object.hasOwn`, never `replies?.[prompt]`: the key is
+   * MESSAGE TEXT, so a prompt of `toString` read a function off the prototype chain and returned
+   * it as the model's answer — a double that answers with JS source is worse than one that cannot.
+   */
+  private fixedReply(prompt: string): string | undefined {
+    const { replies } = this.config;
+    if (replies === undefined || !Object.hasOwn(replies, prompt)) return undefined;
+    return replies[prompt];
+  }
+
   async *stream(request: GenerateRequest): AsyncIterable<StreamChunk> {
     const result = await this.generate(request);
     // One word per chunk: enough to exercise a consumer's assembly logic.

package/src/rag.ts

@@ -31,9 +31,12 @@ export interface ChunkInput {
  * boundary lands at a meaning boundary whenever one is available within the budget.
  */
 export function chunk(input: ChunkInput): readonly Chunk[] {
-  const size = input.size ?? 512;
+  // Floored at one token: `size: 0` makes every comparison below meaningless and the wrap's cut
+  // point zero-width, which is a loop that never advances rather than a chunker that produces
+  // nothing. A budget under one token is not a budget.
+  const size = Math.max(1, Math.floor(input.size ?? 512));
   const overlap = Math.min(input.overlap ?? 64, size - 1);
-  const units = splitUnits(input.text);
+  const units = splitUnits(input.text, size);
   const chunks: Chunk[] = [];
   let buffer: string[] = [];
   let tokens = 0;
@@ -49,10 +52,14 @@ export function chunk(input: ChunkInput): readonly Chunk[] {
         metadata: { source: input.id, ...(input.metadata ?? {}) },
       });
     }
-    // Carry the tail forward as the overlap for the next chunk.
+    // Carry the tail forward as the overlap for the next chunk — at most every unit BUT THE
+    // FIRST. Walking back to index 0 re-seeded the next buffer with everything just flushed, so
+    // a single unit whose own size exceeds `overlap` became the whole of the next chunk, and the
+    // next, and the next: a ~1,000-token document indexed as nine chunks of the same sentence.
+    // The wrap above bounds a unit; this bounds the loop, and both are needed.
     const carried: string[] = [];
     let carriedTokens = 0;
-    for (let i = buffer.length - 1; i >= 0 && carriedTokens < overlap; i -= 1) {
+    for (let i = buffer.length - 1; i >= 1 && carriedTokens < overlap; i -= 1) {
       const unit = buffer[i] ?? '';
       carried.unshift(unit);
       carriedTokens += estimateChunkTokens(unit);
@@ -72,7 +79,7 @@ export function chunk(input: ChunkInput): readonly Chunk[] {
   return chunks;
 }
 
-function splitUnits(text: string): readonly string[] {
+function splitUnits(text: string, size: number): readonly string[] {
   const units: string[] = [];
   for (const paragraph of text.split(/\n{2,}/)) {
     const trimmed = paragraph.trim();
@@ -81,12 +88,65 @@ function splitUnits(text: string): readonly string[] {
     const sentences = trimmed.match(/[^.!?]+[.!?]*\s*/g) ?? [trimmed];
     for (const sentence of sentences) {
       const s = sentence.trim();
-      if (s !== '') units.push(s);
+      if (s !== '') units.push(...hardWrap(s, size));
     }
   }
   return units;
 }
 
+/**
+ * The third split the header promises, and the one that was missing: a unit no larger than the
+ * budget. Neither split above can guarantee it — a base64 blob, a minified line, a CJK paragraph
+ * this splitter's `[.!?]` alphabet cannot see and a legal 400-word sentence all survive both — and
+ * an oversized unit is a chunk the size check can never flush, because the check only fires when
+ * something is ALREADY in the buffer.
+ *
+ * Word boundaries first, so a chunk edge still lands at a meaning boundary when one exists within
+ * the budget; a run with no boundary in it is cut mid-word, because the alternative is no boundary
+ * at all.
+ */
+function hardWrap(unit: string, size: number): readonly string[] {
+  if (estimateChunkTokens(unit) <= size) return [unit];
+  const pieces: string[] = [];
+  let piece = '';
+  const flushPiece = (): void => {
+    if (piece !== '') pieces.push(piece);
+    piece = '';
+  };
+  for (const word of unit.split(/\s+/)) {
+    if (word === '') continue;
+    const candidate = piece === '' ? word : `${piece} ${word}`;
+    if (estimateChunkTokens(candidate) <= size) {
+      piece = candidate;
+      continue;
+    }
+    flushPiece();
+    if (estimateChunkTokens(word) <= size) {
+      piece = word;
+      continue;
+    }
+    pieces.push(...cutToBudget(word, size));
+  }
+  flushPiece();
+  return pieces;
+}
+
+/**
+ * One word longer than the whole budget, cut into pieces that fit. The estimator is linear in
+ * length, so the ratio over what is left gives the cut point directly — `Math.max(1, …)` is what
+ * keeps a pathological ratio from producing a zero-width cut and a loop that never advances.
+ */
+function cutToBudget(run: string, size: number): readonly string[] {
+  const pieces: string[] = [];
+  let rest = run;
+  while (rest !== '') {
+    const fit = Math.max(1, Math.floor((rest.length * size) / estimateChunkTokens(rest)));
+    pieces.push(rest.slice(0, fit));
+    rest = rest.slice(fit);
+  }
+  return pieces;
+}
+
 /** Index a document: chunk, embed, upsert. One call so no step is skipped by accident. */
 export async function indexDocument(input: {
   readonly store: VectorStore;

package/src/remote-embedder.ts

@@ -10,6 +10,7 @@ import { readWithinLimit } from '@ultimat3/core';
 import type { Embedder } from './embeddings';
 import { normalize } from './embeddings';
 import { AiKeyMissingError, AiTransportError, EmbedderDimMismatchError } from './errors';
+import type { AiFetch } from './fetch-seam';
 
 const API_KEY_ENV = 'EMBEDDINGS_API_KEY';
 const DEFAULT_BASE_URL = 'https://api.voyageai.com/v1';
@@ -44,7 +45,7 @@ export interface RemoteEmbedderInput {
   /** Bytes this process will hold of one response. Defaults to 32 MiB. */
   readonly maxResponseBytes?: number;
   /** Injectable so a test can assert the request body without a network. Defaults to `fetch`. */
-  readonly fetch?: typeof fetch;
+  readonly fetch?: AiFetch;
 }
 
 export class RemoteEmbedder implements Embedder {
@@ -77,7 +78,7 @@ export class RemoteEmbedder implements Embedder {
     if (apiKey === undefined || apiKey === '') {
       throw new AiKeyMissingError({ provider: this.name, envVar: API_KEY_ENV });
     }
-    const doFetch = this.config.fetch ?? fetch;
+    const doFetch: AiFetch = this.config.fetch ?? fetch;
     const timeoutMs = this.config.timeoutMs ?? DEFAULT_TIMEOUT_MS;
     const url = `${this.config.baseUrl ?? DEFAULT_BASE_URL}/embeddings`;
     let response: Response;

package/src/runtime.ts

@@ -9,6 +9,7 @@
 
 import type { SemanticCache } from '@ultimat3/cache';
 import { createMemorySemanticCache } from '@ultimat3/cache';
+import { cachedFormatter, MAX_CACHED_FORMATTERS } from '@ultimat3/core';
 import type { Embedder } from './embeddings';
 import { HashEmbedder } from './embeddings';
 import { AiGatewayMissingError } from './errors';
@@ -85,18 +86,31 @@ export function aiRedactor(): Redactor {
   return runtime?.redact ?? noRedaction;
 }
 
+/**
+ * How many scopes hold a live cache instance at once. Core's bound, not a second one — the name
+ * `MAX_CACHED_FORMATTERS` is about `cachedFormatter`'s first caller, never about its contract,
+ * and a second FIFO map written here would be two answers to one question (axiom 1).
+ */
+export const MAX_SEMANTIC_CACHE_SCOPES = MAX_CACHED_FORMATTERS;
+
 /**
  * The cache for one scope. Scopes are separate CACHE INSTANCES, never a filter over a shared
  * one: cosine similarity has no notion of a tenant, so two tenants asking near-identical
  * questions of a shared cache is one tenant reading the other's answer. Partitioning is the
  * only thing that makes that structurally impossible.
+ *
+ * BOUNDED, and that is new with `llm()`'s actor-derived default scope: the default was the single
+ * string `'global'`, so this map held one entry no matter how many callers there were, and the
+ * narrowest-key default makes it one entry per ACTOR in a process that never restarts. Eviction
+ * costs nothing but a rebuild — a `SemanticCache` is a handle, so the durable ones (pgvector) lose
+ * no entries at all and the memory one loses a cache.
  */
 export function semanticCacheFor(scope: string): SemanticCache {
-  const existing = caches.get(scope);
-  if (existing !== undefined) return existing;
-  const created = runtime?.semanticCache(scope) ?? createMemorySemanticCache();
-  caches.set(scope, created);
-  return created;
+  return cachedFormatter(
+    caches,
+    scope,
+    () => runtime?.semanticCache(scope) ?? createMemorySemanticCache(),
+  );
 }
 
 /** Test-only reset. Module-level state otherwise leaks between test files. */

package/src/sse.ts

@@ -5,6 +5,21 @@
 // stream is a boundary search and a field split, and the only interesting property — that a
 // frame may arrive split at any byte offset — is exactly what a library would hide.
 
+import { AiTransportError } from './errors';
+
+/**
+ * The most one unterminated frame may buffer. A peer that sends a body with no frame boundary in
+ * it — an HTML error page, a proxy answering on the model's port, a hung gateway — grows `buffer`
+ * without limit, and no read deadline interrupts it because every individual read SUCCEEDS. Coded
+ * failure > OOM, the same call `@ultimat3/mail`'s `createReplyParser` makes for the same shape.
+ *
+ * Counted in UTF-16 code units rather than bytes: `buffer` holds decoded text, and a decoded unit
+ * is never more than one byte of input, so the cap is conservative in the direction that matters.
+ * A megabyte is orders of magnitude above the largest single delta any provider in this package
+ * sends, so a legitimate stream can never reach it.
+ */
+export const MAX_FRAME_CHARS = 1024 * 1024;
+
 export interface SseFrame {
   /** The `event:` field, or `message` when the frame omits one — the spec's default. */
   readonly event: string;
@@ -57,7 +72,12 @@ function frameOf(block: string): SseFrame | undefined {
  * a stream cut mid-message must fail loudly at the consumer that parses it, and dropping the
  * tail silently would turn a truncated answer into a complete-looking one.
  */
-export async function* readSse(body: ReadableStream<Uint8Array>): AsyncGenerator<SseFrame> {
+export async function* readSse(
+  body: ReadableStream<Uint8Array>,
+  /** Named, never defaulted: the cap below fails as a transport error, and a transport error the
+   * caller reads has to say which endpoint it is about. */
+  provider: string,
+): AsyncGenerator<SseFrame> {
   const reader = body.getReader();
   const decoder = new TextDecoder();
   let buffer = '';
@@ -66,6 +86,7 @@ export async function* readSse(body: ReadableStream<Uint8Array>): AsyncGenerator
       const { done, value } = await reader.read();
       if (done) break;
       buffer += decoder.decode(value, { stream: true });
+      guard(buffer, provider);
       const decoded = decodeSse(buffer);
       buffer = decoded.rest;
       for (const frame of decoded.frames) yield frame;
@@ -79,3 +100,13 @@ export async function* readSse(body: ReadableStream<Uint8Array>): AsyncGenerator
     await reader.cancel().catch(() => undefined);
   }
 }
+
+function guard(buffer: string, provider: string): void {
+  if (buffer.length <= MAX_FRAME_CHARS) return;
+  throw new AiTransportError({
+    provider,
+    detail:
+      `the stream sent more than ${MAX_FRAME_CHARS} characters without completing one SSE ` +
+      'frame — the endpoint is answering with something that is not an event stream',
+  });
+}

package/src/wire.ts

@@ -42,7 +42,11 @@ export function parsePartialUsage(raw: unknown): Partial<TokenUsage> {
   const usage: Partial<Record<keyof TokenUsage, number>> = {};
   for (const [field, wire] of Object.entries(USAGE_FIELDS) as [keyof TokenUsage, string][]) {
     const value = record[wire];
-    if (typeof value === 'number') usage[field] = value;
+    // Floored at zero, and finite: usage is the provider's number, a negative one becomes a
+    // negative `cost`, and `MemoryBudgetStore.add` reads a negative debit as a CREDIT — releasing
+    // an unspent reservation is one — so an unclamped `-1` tops the ledger up instead of
+    // under-reporting it. `NaN` propagates the same way through every later sum.
+    if (typeof value === 'number' && Number.isFinite(value)) usage[field] = Math.max(0, value);
   }
   return usage;
 }
@@ -81,17 +85,23 @@ export function parseStopDetails(raw: unknown): StopDetails | undefined {
  * retry rule in the gateway: an overloaded provider is retryable whether it says so with a
  * 529 on the handshake or with an `overloaded_error` frame ten tokens in.
  */
-const ERROR_STATUS: Readonly<Record<string, number>> = {
-  invalid_request_error: 400,
-  authentication_error: 401,
-  permission_error: 403,
-  not_found_error: 404,
-  request_too_large: 413,
-  rate_limit_error: 429,
-  api_error: 500,
-  timeout_error: 504,
-  overloaded_error: 529,
-};
+// A `Map`, not an object literal: `type` is the PROVIDER's string on the one read below, and
+// `ERROR_STATUS['constructor']` on an object answers the `Object` FUNCTION where
+// `AiTransportError.status` is declared `number | undefined`. Same fix, same reason, as
+// `openai-wire.ts`'s twin and `core`'s `error-retry.ts`.
+const ERROR_STATUS: ReadonlyMap<string, number> = new Map(
+  Object.entries({
+    invalid_request_error: 400,
+    authentication_error: 401,
+    permission_error: 403,
+    not_found_error: 404,
+    request_too_large: 413,
+    rate_limit_error: 429,
+    api_error: 500,
+    timeout_error: 504,
+    overloaded_error: 529,
+  }),
+);
 
 /**
  * A 200 whose body carries an `error` object instead of an answer, refused — how a gateway in
@@ -298,7 +308,7 @@ function inBandFailure(error: Record<string, unknown>): AiTransportError {
   const message = typeof error['message'] === 'string' ? error['message'] : type;
   return new AiTransportError({
     provider: 'anthropic',
-    status: ERROR_STATUS[type],
+    status: ERROR_STATUS.get(type),
     detail: message,
   });
 }