@ultimat3/ai 11.1.0 → 11.3.0

package/CLAUDE.md

@@ -80,6 +80,17 @@ until 2026-08, naming a tool no catalog contained (`llm.test.ts`, `agent.test.ts
 - Semantic scopes are separate cache INSTANCES, never a filter over a shared one — cosine
   similarity has no notion of a tenant. The instance key carries the prompt hash too, which is
   what makes a version bump invalidate the cache.
+- **The instance key carries `ctx.locale` too, `As of 2026-08-24`, and it sits with the prompt hash
+  rather than with the scope.** A prompt taking `locale` as a var — the reference app's `summarize`
+  does, and the model obeys it — differs by ONE token between languages while carrying a whole
+  document, so the two rendered prompts are neighbours: measured with this package's own
+  `HashEmbedder` over that template, **0.9986**, against the declaration's `threshold: 0.97`. The
+  Spanish summary was a hit for an English reader, and the model was never wrong — the cache was.
+  No threshold repairs it, because the same number has to keep an honest repeat above it. It is in
+  the UNCONDITIONAL half of the key because a `scope` answers "who may share this answer" and a
+  locale is part of what the answer IS: an app writing `scope: () => 'global'` is saying its
+  callers may read one another's summaries, never that a Spanish one will do. Same rule
+  `@ultimat3/render` applies to ISR.
 - **The default scope is the calling ACTOR, and `scope` receives `{ input, ctx }`** (`As of
   2026-08`). It defaulted to the literal string `'global'` and took the bare `input`, so the rule
   in the bullet above was contradicted by the very default that shipped: `cache: { semantic: { ttl:
@@ -145,6 +156,29 @@ until 2026-08, naming a tool no catalog contained (`llm.test.ts`, `agent.test.ts
   `renderThrowable` rather than `error.message` / `String(error)` — a renderer that throws replaces
   `X_AI_PROVIDER_UNAVAILABLE` with a bare `TypeError` nothing catches by code, and it bounds a
   provider's 1MB body out of the `cause`.
+- **`X_AI_PROVIDER_UNAVAILABLE` is classified `retryable`, and it is the only one, `As of
+  2026-08-23`.** `AI_ERROR_RETRY`. It rendered `retry: "terminal"` in every problem document while
+  the gateway three lines away was backing off and trying again. `retryable`, not `retry-after`:
+  `statedDelayMs` and `@ultimat3/http`'s `retryAfterOf` read one field, `meta.retryAfterSeconds`, and
+  neither wire format here parses `Retry-After` off a 429 — so there is no time to name. The code's
+  other half, "no configured provider serves this model", is an `app.config.ts` edit and carries a
+  per-instance `retry: 'terminal'` from `AiProviderUnavailableError({ unserved: true })`; registering
+  the code is what makes that override honoured, because core reads an instance `terminal` on an
+  unregistered code as unclassified. Every other code is left UNREGISTERED rather than registered as
+  terminal: `@ultimat3/jobs` dead-letters a registered `terminal` on attempt 1, which is very likely
+  right for `X_LLM_REFUSED` and `X_AI_BUDGET_EXCEEDED` — a re-run is a second identical bill — and is
+  a change to how every app's jobs fail, so it belongs to whoever makes it on purpose.
+- **The retry SCHEDULE is core's, the retry LOOP is this package's, `As of 2026-08-23`.**
+  `backoffMs` is `@ultimat3/core`'s `backoffDelay` with `baseDelayMs`/`maxDelayMs` mapped onto
+  `base`/`max`, and `isRetryable`'s status half is core's `isRetryableStatus`. Two behaviour changes
+  came with it and both are pinned in `gateway-backoff.test.ts`: a delay is **rounded** where it was
+  floored (<=1ms), and **408, 409 and 425 are now retried** where only `429 || >= 500` was. What did
+  NOT move is `retry()`, core's executor, and the reason is the direction it fails: it stops on a
+  `terminal` classification and retries everything else, so an UNCLASSIFIED throw is retried. Every
+  value in this loop is an app `Provider`'s — a plain object with a `status`, never an
+  `UltimateError` — so adopting it would have retried a 400 three times per provider, which is the
+  one thing the loop's own comment says it must not do. `RetryPolicy` and `DEFAULT_RETRY` stay
+  declared here for a second reason: their field names are what an app writes.
 - Every non-2xx and every in-band `error` frame becomes `AiTransportError`, which carries a real
   `status` field — that field IS the gateway's retry rule. A body parsed as a message would read
   as an empty, successful answer, which is the one outcome nothing downstream can detect.

package/README.md

@@ -84,8 +84,8 @@ default store at `replicas: 6` is six ledgers of twenty million, which is a budg
 | The repair turn replays the tool call's arguments, never an empty `text` | an answer through the `respond` tool leaves `text` empty, and an empty text block is a 400 — the repair came back as `X_AI_PROVIDER_UNAVAILABLE` |
 | `reserve()` **debits** the estimate and takes a turn | three concurrent calls otherwise read the same `spent()`, all pass, and all three record against a ceiling only one of them fitted; `record` reconciles and `release` gives it back |
 | A refusal is never cached | a cached one keeps serving a classifier decision after the prompt was fixed |
-| Retries use **full jitter** | synchronised retries from N workers reproduce the rate limit |
-| A 4xx is never retried | the same body gets the same rejection and burns the budget |
+| Retries use **full jitter**, from core's one curve | synchronised retries from N workers reproduce the rate limit. `backoffMs` is `@ultimat3/core`'s `backoffDelay` with the gateway's field names mapped onto it, and the roll is `createGateway({ random })` — injectable, so the schedule is a unit test rather than a range |
+| A 4xx is never retried **except 408, 409 and 425** | the same body gets the same rejection and burns the budget — but a request the server stopped reading (408), a round a concurrent writer won (409) and a handshake that had not finished (425) are transient by construction, and core's `isRetryableStatus` is the one table that says so |
 
 ## Streaming
 
@@ -630,6 +630,14 @@ actor comes from the request context, never from the model.
 
 ## Errors
 
+**`X_AI_PROVIDER_UNAVAILABLE` carries `retry: "retryable"` in `--json`, `As of 2026-08-23`**
+(`AI_ERROR_RETRY`) — it is the one transient code here, and it told every client `terminal` while the
+gateway itself was backing off and trying again. `retryable` and not `retry-after`: that spelling
+means the responder named a time, and no provider in this package parses `Retry-After` off a 429.
+The half of the code that is NOT transient — no configured provider serves the model — carries a
+per-instance `terminal`, because that one is an `app.config.ts` edit. Every other code keeps core's
+fail-closed `terminal` default.
+
 | Code | Meaning |
 |---|---|
 | `X_AI_PROVIDER_UNAVAILABLE` | every provider for the model was unreachable; lists what each said. A TRANSPORT failure only — a coded refusal raised before the socket opens (`X_AI_KEY_MISSING`, `X_AI_REQUEST_INVALID`) reaches the caller as itself, `As of 2026-08-23`, because the same rejection waits on every provider and every attempt and its `fix:` is the whole point of it |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ultimat3/ai",
-  "version": "11.1.0",
+  "version": "11.3.0",
   "description": "LLM gateway, versioned prompts, evals as tests, embeddings, hybrid vector search, RAG",
   "license": "MIT",
   "type": "module",
@@ -32,14 +32,14 @@
     "test": "bun test"
   },
   "dependencies": {
-    "@ultimat3/action": "11.1.0",
-    "@ultimat3/cache": "11.1.0",
-    "@ultimat3/core": "11.1.0",
-    "@ultimat3/db": "11.1.0",
-    "@ultimat3/jobs": "11.1.0",
-    "@ultimat3/money": "11.1.0",
-    "@ultimat3/policy": "11.1.0",
-    "@ultimat3/schema": "11.1.0",
-    "@ultimat3/time": "11.1.0"
+    "@ultimat3/action": "11.3.0",
+    "@ultimat3/cache": "11.3.0",
+    "@ultimat3/core": "11.3.0",
+    "@ultimat3/db": "11.3.0",
+    "@ultimat3/jobs": "11.3.0",
+    "@ultimat3/money": "11.3.0",
+    "@ultimat3/policy": "11.3.0",
+    "@ultimat3/schema": "11.3.0",
+    "@ultimat3/time": "11.3.0"
   }
 }

package/src/errors-retry.ts

@@ -0,0 +1,45 @@
+// The retry classification for this package's codes, apart from ./errors only because one file has
+// one job and that catalogue is at its ceiling. The codes, their titles and the single
+// `registerErrorCodes` call stay in ./errors — one owner, one registration.
+
+import type { ErrorRetry } from '@ultimat3/core';
+import { registerErrorRetry } from '@ultimat3/core';
+import type { AiErrorCode } from './errors';
+
+/**
+ * May a client run this model call again? Unclassified means `terminal` — core fails closed — and
+ * until 2026-08-23 this package classified nothing, so `X_AI_PROVIDER_UNAVAILABLE` told every HTTP
+ * client not to come back from the one failure the gateway's own backoff loop exists for.
+ *
+ * The rule, stated once: **retryable means the same call, made again, has a real chance of a
+ * different answer, and costs nothing to be wrong about beyond one more attempt.** In this package
+ * that is exactly one thing — a provider that could not answer. Everything else is a statement
+ * about the request, the declaration, the catalogue or the money, and a second attempt buys the
+ * same answer at full price.
+ *
+ * **Only the exception is listed.** Everything else keeps the fail-closed default rather than being
+ * registered AS terminal, and the difference is not cosmetic: `@ultimat3/jobs` dead-letters a
+ * registered `terminal` on attempt 1, where an unclassified code keeps the job's own attempt count.
+ * That is very likely the right answer for `X_LLM_REFUSED` and `X_AI_BUDGET_EXCEEDED` — a refusal
+ * re-run is a second identical bill — but it is a change to how every app's jobs fail, and it
+ * belongs to whoever makes it deliberately rather than to a sweep that was fixing the retryable
+ * half.
+ */
+export const AI_ERROR_RETRY = {
+  // The transport code. A 503, a 429, a reset socket, every candidate exhausted — the request was
+  // well formed and nobody could answer it yet. `retryable` and NOT `retry-after`: that spelling
+  // means the responder NAMED a time, `statedDelayMs` reads exactly one field for it
+  // (`meta.retryAfterSeconds`), and neither wire format in this package parses the `Retry-After`
+  // header off a 429 — so the honest answer is "come back", not a delay this package invented.
+  //
+  // The half of this code that is NOT transient — "no configured provider serves this model" — is
+  // an `app.config.ts` edit, and it carries a per-instance `terminal` at its two throw sites
+  // instead of a second code. `UltimateError` supports that override precisely because one code can
+  // be both, and registering the code here is what makes the override honoured: core reads an
+  // instance `terminal` on an UNREGISTERED code as unclassified.
+  X_AI_PROVIDER_UNAVAILABLE: 'retryable',
+} as const satisfies Readonly<Partial<Record<AiErrorCode, ErrorRetry>>>;
+
+// `Partial`, so the table may be a subset — but every key is still checked against the declared set,
+// so a typo or a renamed code is a build error rather than a classification nothing ever throws.
+registerErrorRetry(AI_ERROR_RETRY);

package/src/errors.ts

@@ -74,11 +74,23 @@ registerErrorCodes(
 
 /** Every configured provider refused or errored. Carries what each one said. */
 export class AiProviderUnavailableError extends UltimateError {
-  constructor(input: { model: string; attempts: readonly string[] }) {
+  constructor(input: {
+    model: string;
+    attempts: readonly string[];
+    /**
+     * True when no configured provider LISTS this model — a fact about `app.config.ts`, not about
+     * the moment. It is the same code because a caller cannot act differently on it, and it carries
+     * a per-instance `terminal` because a worker CAN: retrying it burns a whole policy proving the
+     * configuration is still what it was. The call site states the fact; the class decides the
+     * classification, so the retry vocabulary stays in one file.
+     */
+    readonly unserved?: boolean;
+  }) {
     super({
       code: 'X_AI_PROVIDER_UNAVAILABLE',
       cause: `no provider could serve model "${input.model}" (${input.attempts.join(' | ')})`,
       fix: 'check ai.providers in app.config.ts and the provider API key env var',
+      retry: input.unserved === true ? 'terminal' : undefined,
     });
   }
 }

package/src/gateway.ts

@@ -4,7 +4,14 @@
 // budgeted, and cost-accounted in integer minor units. Everything an app does with a model
 // goes through here, so budgets and accounting cannot be bypassed by a stray fetch.
 
-import { isUltimateError, renderThrowable, stringField } from '@ultimat3/core';
+import type { Random } from '@ultimat3/core';
+import {
+  backoffDelay,
+  isRetryableStatus,
+  isUltimateError,
+  renderThrowable,
+  stringField,
+} from '@ultimat3/core';
 import type { Money } from '@ultimat3/money';
 import type { BudgetLimits, BudgetStore } from './budget';
 import { BudgetLedger, currentBudget, estimateSpend, withBudget } from './budget';
@@ -22,6 +29,13 @@ export interface GatewayCache {
   set(key: string, value: string): Promise<void> | void;
 }
 
+/**
+ * A gateway's retry budget. NOT core's `RetryPolicy`, and deliberately still its own declaration:
+ * these three field names are what an app writes in `createGateway({ retry })`, so renaming them
+ * onto `base`/`max`/`jitter` would break every caller for no behaviour. What WAS a duplicate is the
+ * arithmetic, and that is gone — `backoffMs` is core's `backoffDelay` with this shape mapped onto
+ * it, so there is one curve in the framework and one place a jitter bug can live.
+ */
 export interface RetryPolicy {
   /** Total attempts including the first. */
   readonly attempts: number;
@@ -45,6 +59,13 @@ export interface CreateGatewayInput {
   readonly defaultModel?: ModelId;
   /** Overridable so a test can run backoff without waiting. */
   sleep?(ms: number): Promise<void>;
+  /**
+   * The roll behind the backoff's jitter. Injectable for the same reason `sleep` is, and it was the
+   * half that was missing: `Math.random()` read inline made this gateway's retry SCHEDULE provable
+   * only by observing a range, so nothing asserted it and a jitter bug here would have shipped
+   * green. Production never passes one.
+   */
+  readonly random?: Random | undefined;
 }
 
 export interface Gateway {
@@ -64,11 +85,14 @@ class GatewayImpl implements Gateway {
   private readonly config: CreateGatewayInput;
   private readonly retry: RetryPolicy;
   private readonly sleep: (ms: number) => Promise<void>;
+  /** Left `undefined` rather than defaulted, so `backoffDelay` owns the one fallback to `Math.random`. */
+  private readonly random: Random | undefined;
 
   constructor(config: CreateGatewayInput) {
     this.config = config;
     this.retry = config.retry ?? DEFAULT_RETRY;
     this.sleep = config.sleep ?? ((ms) => new Promise((resolve) => setTimeout(resolve, ms)));
+    this.random = config.random;
   }
 
   scope<T>(input: { actorKey?: string; orgKey?: string }, fn: () => Promise<T>): Promise<T> {
@@ -172,6 +196,7 @@ class GatewayImpl implements Gateway {
       throw new AiProviderUnavailableError({
         model,
         attempts: this.config.providers.map((p) => `${p.name}: does not serve ${model}`),
+        unserved: true,
       });
     }
     return provider;
@@ -194,7 +219,11 @@ class GatewayImpl implements Gateway {
     const candidates = this.config.providers.filter((p) => p.models.includes(model));
     const failures: string[] = [];
     if (candidates.length === 0) {
-      throw new AiProviderUnavailableError({ model, attempts: [`no provider serves ${model}`] });
+      throw new AiProviderUnavailableError({
+        model,
+        attempts: [`no provider serves ${model}`],
+        unserved: true,
+      });
     }
 
     for (const provider of candidates) {
@@ -221,7 +250,7 @@ class GatewayImpl implements Gateway {
           // too — a provider's 1MB body is not a cause.
           failures.push(`${provider.name}#${attempt}: ${renderThrowable(error)}`);
           if (!isRetryable(error) || attempt === this.retry.attempts) break;
-          await this.sleep(backoffMs(this.retry, attempt));
+          await this.sleep(backoffMs(this.retry, attempt, this.random));
         }
       }
     }
@@ -229,15 +258,37 @@ class GatewayImpl implements Gateway {
   }
 }
 
-/** Full jitter: a uniform pick from [0, exponential], capped. */
-export function backoffMs(policy: RetryPolicy, attempt: number): number {
-  const ceiling = Math.min(policy.baseDelayMs * 2 ** (attempt - 1), policy.maxDelayMs);
-  return Math.floor(Math.random() * ceiling);
+/**
+ * Full jitter: a uniform pick from [0, exponential], capped BEFORE the roll.
+ *
+ * The arithmetic is core's `backoffDelay` — this is the mapping from the gateway's own field names
+ * onto it, and nothing else. Two things came with the delegation and neither is cosmetic: the
+ * result is ROUNDED where this floored it (a shift of at most 1ms, and the same rounding
+ * `@ultimat3/jobs` and `@ultimat3/realtime` already use), and a policy carrying a `NaN` — which is
+ * what `Number(process.env.…)` answers for an unset variable — waits 0 instead of handing
+ * `setTimeout` a `NaN` it fires on the next tick, i.e. a backoff that is a tight spin.
+ */
+export function backoffMs(policy: RetryPolicy, attempt: number, random?: Random): number {
+  return backoffDelay({
+    attempt,
+    base: policy.baseDelayMs,
+    max: policy.maxDelayMs,
+    curve: 'exponential',
+    jitter: 'full',
+    random,
+  });
 }
 
 /**
  * Retryable = the request was well formed and the provider was momentarily unable. A 400 is
  * never retried: the same body produces the same rejection and only burns the budget.
+ *
+ * The status half is core's `isRetryableStatus`, which is WIDER than the `429 || >= 500` this
+ * gateway shipped: 408, 409 and 425 join it. Each is transient by construction — the server gave up
+ * waiting for a body it never fully read, a concurrent writer won the round, the handshake was not
+ * finished — so not retrying them was a gap rather than a policy, and one narrower table in one
+ * package was how it stayed invisible. The `code` branch has no equivalent in core and stays here:
+ * that table is HTTP status only, and a socket that timed out never produced one.
  */
 export function isRetryable(error: unknown): boolean {
   if (typeof error !== 'object' || error === null) return false;
@@ -247,7 +298,7 @@ export function isRetryable(error: unknown): boolean {
   // to answer with — so it fails closed rather than raising.
   try {
     const e = error as { status?: unknown; code?: unknown };
-    if (typeof e.status === 'number') return e.status === 429 || e.status >= 500;
+    if (typeof e.status === 'number') return isRetryableStatus(e.status);
     return e.code === 'ETIMEDOUT' || e.code === 'ECONNRESET';
   } catch {
     return false;

package/src/index.ts

@@ -58,6 +58,7 @@ export {
   VectorDimMismatchError,
   VectorScopeWidenedError,
 } from './errors';
+export { AI_ERROR_RETRY } from './errors-retry';
 export type { EvalBaseline, Regression } from './eval-baseline';
 export {
   baselinePath,

package/src/llm-cache.ts

@@ -6,6 +6,13 @@
  * carries the prompt VERSION as well — which is what makes "editing a prompt requires a version
  * bump" invalidate the cache: a bumped version reaches a different store, so an old answer cannot
  * survive a prompt edit no matter how similar the text.
+ *
+ * It carries the request LOCALE for the same structural reason, `As of 2026-08-24`. A prompt that
+ * takes `locale` as a var — the reference app's `summarize` does, and the model obeys it — differs
+ * by one token between languages while carrying a whole document, so the two renderings are
+ * neighbours: measured with this package's own `HashEmbedder` over that template, **0.9986**
+ * against a declared threshold of `0.97`. The Spanish answer was therefore a hit for an English
+ * reader. No threshold fixes it, because the same number has to keep an honest repeat above it.
  */
 
 import type { Ctx } from '@ultimat3/core';
@@ -62,6 +69,10 @@ export interface PromptCache {
  * same reason it is JSON rather than a joined string — an actor id is app data and may carry any
  * separator, and a value that can spell a boundary can spell somebody else's.
  *
+ * The locale is deliberately NOT here. This function is the DEFAULT a declaration replaces by
+ * writing its own `scope`, and the locale has to survive that — so it lives in the unconditional
+ * half of the store key, beside the prompt hash.
+ *
  * `ctx.actor` is never absent (`createContext` defaults it to `anonymousActor()`), so every
  * anonymous caller shares one partition — which is what the anonymous actor already means
  * everywhere else in the framework.
@@ -92,7 +103,14 @@ export async function openCache<TParsed>(
   const semantic = args.cache?.semantic;
   if (semantic === undefined) return undefined;
   const scope = semantic.scope?.({ input: args.input, ctx: args.ctx }) ?? actorScope(args.ctx);
-  const store = semanticCacheFor(`${args.prompt.ref}#${args.prompt.hash}::${scope}`);
+  // The locale sits with the prompt HASH and not with the scope, on purpose: a scope answers "who
+  // may share this answer" and the locale is part of what the answer IS — so a written-down
+  // `scope: () => 'global'` is still partitioned by it. An app declaring a shared store is saying
+  // its callers may read one another's summaries, never that a Spanish one will do for an English
+  // reader. `@ultimat3/render`'s ISR keys by locale for exactly this reason.
+  const store = semanticCacheFor(
+    `${args.prompt.ref}#${args.prompt.hash}@${args.ctx.locale}::${scope}`,
+  );
   const embedding = Array.from(await embedOne(aiEmbedder(), args.rendered));
   const ttlMs = semantic.ttl === undefined ? undefined : parseDuration(semantic.ttl);
   return {