@fractaal/pi-ai 1.4.1 → 1.6.0

package/README.md

@@ -64,7 +64,7 @@ Unified LLM API with provider collections, automatic auth resolution, token and
 - **OpenAI** (including the Decisions classifier API)
 - **Ant Ling**
 - **Azure OpenAI (Responses)**
-- **OpenAI Codex (legacy)** (ChatGPT Plus/Pro subscription, requires OAuth, see below)
+- **OpenAI Codex** (ChatGPT Plus/Pro subscription, requires OAuth, see below)
 - **Radius** (API key or OAuth, with a dynamically refreshed gateway catalog)
 - **TypeSafe** (System One classifier API)
 - **DeepSeek**
@@ -103,15 +103,16 @@ Unified LLM API with provider collections, automatic auth resolution, token and
 npm install @earendil-works/pi-ai
 ```
 
-TypeBox exports are re-exported from `@earendil-works/pi-ai`: `Type`, `Static`, and `TSchema`.
+Tool parameters are JSON Schema. The examples build them with `@earendil-works/pi-schema`, which also types the arguments (`npm install @earendil-works/pi-schema`); TypeBox schemas and plain JSON Schema objects work as well.
 
 ## Quick Start
 
 You build a `Models` collection of providers and stream through it. The quickest start registers every built-in provider; apps that care about bundle size register individual providers instead (see [Provider Factories](#provider-factories) and [Bundling and Tree Shaking](#bundling-and-tree-shaking)).
 
 ```typescript
-import { Type, type Context, type Tool } from '@earendil-works/pi-ai';
+import { type Context, type Tool } from '@earendil-works/pi-ai';
 import { builtinModels } from '@earendil-works/pi-ai/providers/all';
+import { obj, opt, str } from '@earendil-works/pi-schema';
 
 // A Models collection with every built-in provider registered
 const models = builtinModels();
@@ -119,12 +120,12 @@ const models = builtinModels();
 // Sync lookup against the collection
 const model = models.getModel('openai', 'gpt-4o-mini')!;
 
-// Define tools with TypeBox schemas for type safety and validation
+// Define tools with JSON Schema; pi-schema's builders also type the arguments
 const tools: Tool[] = [{
   name: 'get_time',
   description: 'Get the current time',
-  parameters: Type.Object({
-    timezone: Type.Optional(Type.String({ description: 'Optional timezone (e.g., America/New_York)' }))
+  parameters: obj({
+    timezone: opt(str({ description: 'Optional timezone (e.g., America/New_York)' }))
   })
 }];
 
@@ -523,6 +524,7 @@ Built-in providers resolve these env vars (Node.js; in browsers pass `apiKey` ex
 | Qwen Token Plan (existing catalog) | `QWEN_TOKEN_PLAN_API_KEY` |
 | Qwen Token Plan (Individual) | `QWEN_TOKEN_PLAN_API_KEY` |
 | Qwen Token Plan (China) | `QWEN_TOKEN_PLAN_CN_API_KEY` |
+| Qwen Token Plan (Individual, China) | `QWEN_TOKEN_PLAN_CN_API_KEY` |
 | Xiaomi MiMo (API billing) | `XIAOMI_API_KEY` |
 | Xiaomi MiMo Token Plan (China) | `XIAOMI_TOKEN_PLAN_CN_API_KEY` |
 | Xiaomi MiMo Token Plan (Amsterdam) | `XIAOMI_TOKEN_PLAN_AMS_API_KEY` |
@@ -532,40 +534,43 @@ Built-in providers resolve these env vars (Node.js; in browsers pass `apiKey` ex
 `qwen-token-plan-individual` and `qwen-token-plan` share the international endpoint and
 `QWEN_TOKEN_PLAN_API_KEY`. The Individual provider exposes only the models documented for Individual
 subscriptions, while the existing provider retains its broader catalog for backward compatibility.
+`qwen-token-plan-individual-cn` applies the same Individual catalog to the China endpoint and
+`QWEN_TOKEN_PLAN_CN_API_KEY`.
 Stored credentials remain provider-scoped, so save the key under the provider ID you register.
 
 Amazon Bedrock resolves ambient AWS credentials (`AWS_PROFILE`, access key pairs, `AWS_BEARER_TOKEN_BEDROCK`, ECS task roles, web identity tokens); its provider-owned login flow supports bearer tokens, AWS profiles, and the existing credential chain. Vertex AI resolves either an explicit key or gcloud Application Default Credentials plus project/location, with a provider-owned login flow for API keys, ADC, and service-account files.
 
 ## Tools
 
-Tools enable LLMs to interact with external systems. This library uses TypeBox schemas for type-safe tool definitions with automatic validation using TypeBox's built-in validator and value conversion utilities. TypeBox schemas can be serialized and deserialized as plain JSON, making them ideal for distributed systems.
+Tools enable LLMs to interact with external systems. A tool's `parameters` is a JSON Schema object: built with `@earendil-works/pi-schema`'s builders (which also type the arguments through its `Static`), with TypeBox, or written by hand. `validateToolCall()` and `validateToolArguments()` validate with pi-schema, coercing loosely typed arguments first (numeric strings, `"true"`, a single value for an array, `null` for an omitted optional property). Schemas are plain JSON, so they serialize with the rest of a context.
 
 ### Defining Tools
 
 ```typescript
-import { Type, type Tool, StringEnum } from '@earendil-works/pi-ai';
+import { type Tool, StringEnum } from '@earendil-works/pi-ai';
+import { arr, obj, str } from '@earendil-works/pi-schema';
 
-// Define tool parameters with TypeBox
+// Define tool parameters with pi-schema's builders
 const weatherTool: Tool = {
   name: 'get_weather',
   description: 'Get current weather for a location',
-  parameters: Type.Object({
-    location: Type.String({ description: 'City name or coordinates' }),
+  parameters: obj({
+    location: str({ description: 'City name or coordinates' }),
     units: StringEnum(['celsius', 'fahrenheit'], { default: 'celsius' })
   })
 };
 
-// Note: For Google API compatibility, use StringEnum helper instead of Type.Enum
-// Type.Enum generates anyOf/const patterns that Google doesn't support
+// Note: StringEnum gives { type: "string", enum: [...] }, which every provider accepts,
+// including Google's, which rejects anyOf/const patterns
 
 const bookMeetingTool: Tool = {
   name: 'book_meeting',
   description: 'Schedule a meeting',
-  parameters: Type.Object({
-    title: Type.String({ minLength: 1 }),
-    startTime: Type.String({ format: 'date-time' }),
-    endTime: Type.String({ format: 'date-time' }),
-    attendees: Type.Array(Type.String({ format: 'email' }), { minItems: 1 })
+  parameters: obj({
+    title: str({ minLength: 1 }),
+    startTime: str({ format: 'date-time' }),
+    endTime: str({ format: 'date-time' }),
+    attendees: arr(str({ format: 'email' }), { minItems: 1 })
   })
 };
 ```
@@ -578,10 +583,7 @@ Tools can opt in to provider-side constrained sampling. For JSON-schema tools, `
 const strictTool: Tool = {
   name: 'edit_file',
   description: 'Edit a file',
-  parameters: Type.Object({
-    path: Type.String(),
-    content: Type.String()
-  }, { additionalProperties: false }),
+  parameters: obj({ path: str(), content: str() }), // obj() is closed: additionalProperties: false
   constrainedSampling: { type: 'json_schema', strict: 'prefer' }
 };
 ```
@@ -592,9 +594,7 @@ Strict JSON-schema constrained sampling is supported for OpenAI, Anthropic, supp
 const patchTool: Tool = {
   name: 'apply_patch',
   description: 'Apply a patch',
-  parameters: Type.Object({
-    input: Type.String()
-  }, { additionalProperties: false }),
+  parameters: obj({ input: str() }),
   constrainedSampling: {
     type: 'grammar',
     variants: {
@@ -698,7 +698,7 @@ for await (const event of s) {
 
 ### Validating Tool Arguments
 
-When implementing your own tool execution loop, use `validateToolCall` to validate arguments before passing them to your tools:
+When implementing your own tool execution loop, use `validateToolCall` to validate arguments before passing them to your tools. It coerces loosely typed arguments with pi-schema (`compile(schema, { coerce: true }).parse()`), returns the coerced arguments, and throws an error listing pi-schema's errors (`path: message`) followed by the arguments it received:
 
 ```typescript
 import { validateToolCall, type Tool } from '@earendil-works/pi-ai';
@@ -894,7 +894,7 @@ console.log(model.output); // ['image'] or ['image', 'text']
 
 ## Classification
 
-Classifier models consume structured JSON state, and optionally images, and answer one or more typed questions. They do not use chat or image-generation APIs. TypeSafe's Jev model, Cloudflare's Clef models, and OpenAI's GPT-6 Luna are available from these built-in providers:
+Classifier models consume structured JSON state, and optionally images, and answer one or more typed questions. They do not use chat or image-generation APIs. TypeSafe's Jev model, Cloudflare's Clef models, OpenAI's GPT-6 Luna, and Microsoft-Decision-1 are available from these built-in providers:
 
 | Provider | Model IDs | Auth |
 | --- | --- | --- |
@@ -904,6 +904,7 @@ Classifier models consume structured JSON state, and optionally images, and answ
 | `vercel-ai-gateway` | `typesafe-ai/jev` | `AI_GATEWAY_API_KEY` |
 | `opencode` | `jev-1.13`, `jev-1.13-free` | `OPENCODE_API_KEY` |
 | `openai` | `gpt-6-luna` | `OPENAI_API_KEY` |
+| `github-copilot` | `microsoft-decision-1` | `COPILOT_GITHUB_TOKEN` or GitHub Copilot OAuth |
 
 ```typescript
 import { builtinModels } from '@earendil-works/pi-ai/providers/all';
@@ -1440,6 +1441,8 @@ Built-in API implementations live under `./api/<api-id>`:
 
 Importing an implementation module loads its SDK. The `./api/<id>.lazy` wrappers (used by the provider factories) defer that load to the first request when the runtime or bundler supports dynamic import chunking. Legacy raw API subpaths from older releases (`./anthropic`, `./google`, `./mistral`, `./openai-completions`, ...) were removed; use `@earendil-works/pi-ai/api/<api-id>`.
 
+**Anthropic message threads:** with `sessionId` set and `cacheRetention` not `"none"`, Anthropic requests continue from the previous reply on the server (`message-threads-2026-08-12` beta) and send only the new messages. When anything earlier in the request changed, or the server can't continue, the full request is sent. Token counts and prompt caching are unchanged. On by default only for the `anthropic` provider on `api.anthropic.com`; set `compat.supportsMessageThreads` to change that, for example for gateways that forward the `thread` field. Set `disableContinuation: true` on requests that repeat an earlier one, such as cache warming. Call `cleanupSessionResources(sessionId)` when a session ends.
+
 ### OpenAI Compatibility Settings
 
 The `openai-completions` API is implemented by many providers with minor differences. By default, the library auto-detects compatibility settings based on `baseUrl` for a small set of known OpenAI-compatible providers (Cerebras, xAI, Chutes, DeepSeek, NVIDIA NIM, Together AI, zAi, OpenCode, Cloudflare Workers AI, etc.). For custom proxies or unknown endpoints, you can override these settings via the `compat` field. For `openai-responses` models, the compat field supports Responses-specific flags.
@@ -1459,6 +1462,7 @@ interface OpenAICompletionsCompat {
   maxTokensField?: 'max_completion_tokens' | 'max_tokens';  // Which field name to use (default: max_completion_tokens)
   requiresToolResultName?: boolean;  // Whether tool results require the `name` field (default: false)
   requiresAssistantAfterToolResult?: boolean; // Whether tool results must be followed by an assistant message (default: false)
+  requiresNonNullAssistantContent?: boolean; // Whether replayed assistant messages with no text must send "" instead of null content (default: false)
   requiresThinkingAsText?: boolean;  // Whether thinking blocks must be converted to text (default: false)
   requiresReasoningContentOnAssistantMessages?: boolean; // Whether all replayed assistant messages must include empty reasoning_content when reasoning is enabled (default: auto-detected for DeepSeek)
   thinkingFormat?: 'openai' | 'openrouter' | 'deepseek' | 'together' | 'baseten' | 'zai' | 'qwen' | 'chat-template' | 'qwen-chat-template' | 'string-thinking' | 'ant-ling'; // Format for reasoning param: 'openai' uses reasoning_effort, 'openrouter' uses reasoning: { effort }, 'deepseek' uses thinking: { type } plus reasoning_effort when supported, 'together' uses reasoning: { enabled } plus reasoning_effort when supported, 'baseten' uses configurable chat_template_args plus reasoning_effort when supported, 'zai' uses thinking: { type }, 'qwen' uses enable_thinking, 'chat-template' uses configurable chat_template_kwargs, 'qwen-chat-template' uses chat_template_kwargs.enable_thinking and preserve_thinking, 'string-thinking' uses top-level thinking, 'ant-ling' uses reasoning: { effort } only for mapped efforts (default: openai)
@@ -1726,8 +1730,8 @@ models.setProvider(openaiProvider());
 
 Rules:
 
-- `@earendil-works/pi-ai/models` exports the model runtime (`createModels`, `createProvider`, model helpers, and their types) without TypeBox, built-in catalogs, or SDK implementations. Other types can still use `import type` from the root.
-- `@earendil-works/pi-ai` is the core entrypoint and does not import built-in catalogs, real provider factories, or SDK implementations, but it eagerly imports TypeBox and schema validation. Unbundled Node scripts do not tree-shake its unused exports; prefer `./models`, `./providers/faux`, and specific `./utils/*` subpaths when those are all you need.
+- `@earendil-works/pi-ai/models` exports the model runtime (`createModels`, `createProvider`, model helpers, and their types) without schema validation, built-in catalogs, or SDK implementations. Other types can still use `import type` from the root.
+- `@earendil-works/pi-ai` is the core entrypoint and does not import built-in catalogs, real provider factories, or SDK implementations, but it eagerly imports schema validation (pi-schema). Unbundled Node scripts do not tree-shake its unused exports; prefer `./models`, `./providers/faux`, and specific `./utils/*` subpaths when those are all you need.
 - `@earendil-works/pi-ai/providers/<provider>` imports that provider's catalog and lazy API wrapper only.
 - `@earendil-works/pi-ai/providers/all` imports every built-in provider factory and all catalogs. Use it only when you want the full built-in set.
 - With code splitting, provider SDKs stay in lazy chunks and load on first request.
@@ -1767,6 +1771,18 @@ setBedrockProviderModule(bedrockProviderModule);
 
 That explicit override bundles the AWS SDK. Without it, Bedrock's opaque runtime import expects the package's Bedrock implementation file to be available at runtime.
 
+Node bundles that split at dynamic imports (esbuild `splitting: true`) can keep Bedrock and the OAuth login flows inside the bundle but lazy: register loaders with literal dynamic imports, which the bundler turns into chunks that load on first use:
+
+```typescript
+import { registerLazyBedrockProvider } from '@earendil-works/pi-ai/lazy-bedrock-provider';
+import { registerLazyOAuthFlows } from '@earendil-works/pi-ai/lazy-oauth';
+
+registerLazyBedrockProvider();
+registerLazyOAuthFlows();
+```
+
+Both modules are Node-only; never import them into a browser bundle.
+
 ### Provider-Scoped Environment Overrides
 
 Pass `env` in stream options to scope provider configuration to a request. Values in `env` are used before process environment variables for provider auth and configuration such as Cloudflare account IDs, Azure OpenAI settings, Vertex project/location, Bedrock settings, `PI_CACHE_RETENTION`, and `HTTP_PROXY`/`HTTPS_PROXY`.
@@ -1792,7 +1808,7 @@ Several providers support OAuth authentication instead of static API keys:
 
 - **Anthropic** (Claude Pro/Max subscription)
 - **OpenAI** (Sign in with ChatGPT: uses the ChatGPT subscription with the OpenAI API)
-- **OpenAI Codex (legacy)** (ChatGPT Plus/Pro subscription, access to GPT-5.x Codex models)
+- **OpenAI Codex** (ChatGPT Plus/Pro subscription, access to GPT-5.x Codex models)
 - **GitHub Copilot** (Copilot subscription)
 - **OpenRouter** (OAuth PKCE that mints a user-controlled API key)
 
@@ -1872,7 +1888,7 @@ Built-in login and refresh flows are private provider implementations. Use provi
 
 Provider notes:
 
-**OpenAI Codex (legacy)**: Superseded by Sign in with ChatGPT on the OpenAI provider. Requires a ChatGPT Plus or Pro subscription. Provides access to GPT-5.x Codex models with extended context windows and reasoning capabilities. The library automatically handles session-based prompt caching when `sessionId` is provided in stream options unless `cacheRetention` is `"none"`. You can set `transport` in stream options to `"sse"`, `"websocket"`, or `"auto"` for Codex Responses transport selection. When using WebSocket with a `sessionId` and cache retention enabled, connections are reused per session and expire after 5 minutes of inactivity. Call `cleanupSessionResources(sessionId)` when finished so the pooled connection does not keep the process alive.
+**OpenAI Codex**: Superseded by Sign in with ChatGPT on the OpenAI provider. Requires a ChatGPT Plus or Pro subscription. Provides access to GPT-5.x Codex models with extended context windows and reasoning capabilities. The library automatically handles session-based prompt caching when `sessionId` is provided in stream options unless `cacheRetention` is `"none"`. You can set `transport` in stream options to `"sse"`, `"websocket"`, or `"auto"` for Codex Responses transport selection. When using WebSocket with a `sessionId` and cache retention enabled, connections are reused per session and expire after 5 minutes of inactivity. Call `cleanupSessionResources(sessionId)` when finished so the pooled connection does not keep the process alive.
 
 **Azure OpenAI (Responses)**: Uses the Responses API only. Set `AZURE_OPENAI_API_KEY` and either `AZURE_OPENAI_BASE_URL` or `AZURE_OPENAI_RESOURCE_NAME`. `AZURE_OPENAI_BASE_URL` supports both `https://<resource>.openai.azure.com` and `https://<resource>.cognitiveservices.azure.com`; root endpoints are normalized to `.../openai/v1` automatically. Use `AZURE_OPENAI_API_VERSION` (defaults to `v1`) to override the API version if needed. Deployment names are treated as model IDs by default, override with `azureDeploymentName` or `AZURE_OPENAI_DEPLOYMENT_NAME_MAP` using comma-separated `model-id=deployment` pairs (for example `gpt-4o-mini=my-deployment,gpt-4o=prod`). Legacy deployment-based URLs are intentionally unsupported.
 
@@ -1928,12 +1944,12 @@ Create a new API implementation file (for example `bedrock-converse-stream.ts`)
 
 Add a lazy wrapper `src/api/<api-id>.lazy.ts` (`<name>Api()` via `lazyApi()`) so providers can reference the implementation without importing its SDK. Add any root-level `export type` re-exports in `src/index.ts` that should remain available from `@earendil-works/pi-ai`.
 
-#### 3. Model Generation (`scripts/generate-models.ts`)
+#### 3. Model Catalog (`catalog/providers/<id>.ts`)
 
-- Add logic to fetch and parse models from the provider's source (e.g., models.dev API)
+- Add a provider definition and register it in `catalog/providers/index.ts`; see `catalog/README.md` for the build order and helpers
 - Map chat/tool-capable provider data to `Model`, image-generation data to `ImageModel`, and models.dev `type: "decision"` entries to `ClassifierModel`; hydration groups the ignored `src/providers/data/<id>.json` values by API while stable `src/providers/<id>.models.ts` wrappers derive exact model/API types directly from those JSON keys
 - Keep model ids unique within each provider and model type; emit separate entries when an upstream model supports multiple operations
-- Handle provider-specific quirks (pricing format, capability flags, model ID transformations)
+- Handle provider-specific quirks (pricing format, capability flags, model ID transformations) in the provider definition
 
 #### 4. Provider Factory (`src/providers/<id>.ts`)
 

package/dist/api/anthropic-messages.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"anthropic-messages.d.ts","sourceRoot":"","sources":["../../src/api/anthropic-messages.ts"],"names":[],"mappings":"AAAA,OAAO,SAAiC,MAAM,mBAAmB,CAAC;AAoBlE,OAAO,KAAK,EASX,mBAAmB,EAEnB,cAAc,EACd,aAAa,EAIb,MAAM,aAAa,CAAC;AA6JrB,MAAM,MAAM,eAAe,GAAG,KAAK,GAAG,QAAQ,GAAG,MAAM,GAAG,OAAO,GAAG,KAAK,CAAC;AAE1E,MAAM,MAAM,wBAAwB,GAAG,YAAY,GAAG,SAAS,CAAC;AA2ChE,MAAM,WAAW,gBAAiB,SAAQ,aAAa;IACtD;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,OAAO,CAAC;IAC1B;;;;OAIG;IACH,oBAAoB,CAAC,EAAE,MAAM,CAAC;IAC9B;;;;;;;;;;;OAWG;IACH,MAAM,CAAC,EAAE,eAAe,CAAC;IACzB;;;;;;;;;;;OAWG;IACH,eAAe,CAAC,EAAE,wBAAwB,CAAC;IAC3C;;;;;OAKG;IACH,mBAAmB,CAAC,EAAE,OAAO,CAAC;IAC9B;;;;OAIG;IACH,UAAU,CAAC,EAAE,MAAM,GAAG,KAAK,GAAG,MAAM,GAAG;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;IACtE;;;;OAIG;IACH,MAAM,CAAC,EAAE,SAAS,CAAC;CACnB;AAoRD,eAAO,MAAM,MAAM,EAAE,cAAc,CAAC,oBAAoB,EAAE,gBAAgB,CAkWzE,CAAC;AA2BF,eAAO,MAAM,YAAY,EAAE,cAAc,CAAC,oBAAoB,EAAE,mBAAmB,CAgDlF,CAAC"}
+{"version":3,"file":"anthropic-messages.d.ts","sourceRoot":"","sources":["../../src/api/anthropic-messages.ts"],"names":[],"mappings":"AAAA,OAAO,SAAiC,MAAM,mBAAmB,CAAC;AAqBlE,OAAO,KAAK,EASX,mBAAmB,EAEnB,cAAc,EACd,aAAa,EAIb,MAAM,aAAa,CAAC;AAiKrB,MAAM,MAAM,eAAe,GAAG,KAAK,GAAG,QAAQ,GAAG,MAAM,GAAG,OAAO,GAAG,KAAK,CAAC;AAE1E,MAAM,MAAM,wBAAwB,GAAG,YAAY,GAAG,SAAS,CAAC;AA6QhE,MAAM,WAAW,gBAAiB,SAAQ,aAAa;IACtD;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,OAAO,CAAC;IAC1B;;;;OAIG;IACH,oBAAoB,CAAC,EAAE,MAAM,CAAC;IAC9B;;;;;;;;;;;OAWG;IACH,MAAM,CAAC,EAAE,eAAe,CAAC;IACzB;;;;;;;;;;;OAWG;IACH,eAAe,CAAC,EAAE,wBAAwB,CAAC;IAC3C;;;;;OAKG;IACH,mBAAmB,CAAC,EAAE,OAAO,CAAC;IAC9B;;;;OAIG;IACH,UAAU,CAAC,EAAE,MAAM,GAAG,KAAK,GAAG,MAAM,GAAG;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;IACtE;;;;OAIG;IACH,MAAM,CAAC,EAAE,SAAS,CAAC;CACnB;AAoRD,eAAO,MAAM,MAAM,EAAE,cAAc,CAAC,oBAAoB,EAAE,gBAAgB,CAgXzE,CAAC;AA2BF,eAAO,MAAM,YAAY,EAAE,cAAc,CAAC,oBAAoB,EAAE,mBAAmB,CAgDlF,CAAC"}

package/dist/api/anthropic-messages.js

@@ -1,8 +1,11 @@
 import Anthropic, {} from "@anthropic-ai/sdk";
 import { ANTHROPIC_FEDERATION_RULE_ID_ENV, ANTHROPIC_IDENTITY_TOKEN_FILE_ENV, ANTHROPIC_ORGANIZATION_ID_ENV, ANTHROPIC_SERVICE_ACCOUNT_ID_ENV, ANTHROPIC_WORKSPACE_ID_ENV, } from "../env-api-keys.js";
 import { calculateCost } from "../models.js";
+import { registerSessionResourceCleanup } from "../session-resources.js";
 import { appendAssistantMessageDiagnostic } from "../utils/diagnostics.js";
+import { withCauseDetail } from "../utils/error-body.js";
 import { AssistantMessageEventStream } from "../utils/event-stream.js";
+import { shortHash } from "../utils/hash.js";
 import { headersToRecord } from "../utils/headers.js";
 import { getIdentityHeaders } from "../utils/identity.js";
 import { parseJsonWithRepair } from "../utils/json-parse.js";
@@ -13,7 +16,7 @@ import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
 import { ToolCallArgumentsStream } from "../utils/streaming-json.js";
 import { getSystemMessageText, renderSystemMessageUpdate } from "../utils/text.js";
 import { getCurrentTools, getInitialSystemMessage, resolveTranscript, } from "../utils/transcript.js";
-import { getJsonSchemaToolParameters, resolveJsonSchemaStrictSampling, } from "./constrained-sampling.js";
+import { getJsonSchemaToolParameters, isStrictToolLimitErrorMessage, prefersJsonSchemaStrictSampling, resolveJsonSchemaStrictSampling, } from "./constrained-sampling.js";
 import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.js";
 import { adjustMaxTokensForThinking, buildBaseOptions, clampMaxTokensToContext } from "./simple-options.js";
 import { splitErrorToolResultImages, transformMessages } from "./transform-messages.js";
@@ -122,6 +125,7 @@ const SERVER_SIDE_FALLBACK_BETA = "server-side-fallback-2026-07-01";
 const MID_CONVERSATION_OUTPUT_CONFIG_BETA = "mid-conversation-output-config-2026-07-01";
 const THINKING_BINDING_CONTROLS_BETA = "thinking-binding-controls-2026-08-01";
 const INLINE_TOOLS_BETA = "inline-tools-2026-09-15";
+const MESSAGE_THREADS_BETA = "message-threads-2026-08-12";
 /**
  * Stable deferred tool declared whenever native tool changes are in use. Anthropic adds
  * hidden prompt scaffolding for mid-conversation tool changes; declaring this placeholder
@@ -151,8 +155,178 @@ function getAnthropicCompat(model) {
         supportsStrictTools: model.compat?.supportsStrictTools ?? false,
         supportsMidConvoSystemMessages: model.compat?.supportsMidConvoSystemMessages ?? false,
         supportsMidConvoToolChanges: model.compat?.supportsMidConvoToolChanges ?? false,
+        // Gateways that drop unknown request fields would silently turn a `continue` into a request
+        // without the earlier conversation, so the default is limited to Anthropic's own endpoint.
+        // Widen or remove this restriction once proxies generally forward the `thread` field.
+        supportsMessageThreads: model.compat?.supportsMessageThreads ??
+            (model.provider === "anthropic" && model.baseUrl.includes("api.anthropic.com")),
     };
 }
+/** Per session: where the next request can continue, or "off" after the endpoint rejected threads. */
+const threadState = new Map();
+const MAX_THREAD_STATE_ENTRIES = 10_000;
+/** Rejections of a `continue` that a new thread fixes: expired, raced, or a fixed setting changed. */
+const NEW_THREAD_ERRORS = new Set([
+    "thread_not_found",
+    "thread_already_continued",
+    "thread_continue_in_flight",
+    "thread_fingerprint_mismatch",
+]);
+/**
+ * Sets a session's thread state, keeping at most `MAX_THREAD_STATE_ENTRIES` sessions.
+ *
+ * State is normally freed by `cleanupSessionResources(sessionId)` when a session ends, as pi's
+ * coding agent does. The cap bounds memory for library users that never call it, for example a
+ * server passing a new `sessionId` per conversation. Dropping the least recently updated session
+ * is cheap: its thread has most likely expired on the server already (after a few minutes idle),
+ * and otherwise its next request just starts a new thread.
+ */
+function setThreadState(sessionId, value) {
+    // Re-insert so Map order stays oldest-first for the cap.
+    threadState.delete(sessionId);
+    threadState.set(sessionId, value);
+    if (threadState.size > MAX_THREAD_STATE_ENTRIES) {
+        const oldest = threadState.keys().next().value;
+        if (oldest !== undefined)
+            threadState.delete(oldest);
+    }
+}
+registerSessionResourceCleanup((sessionId) => {
+    if (sessionId)
+        threadState.delete(sessionId);
+    else
+        threadState.clear();
+});
+/**
+ * Hash of everything a `continue` builds on: the settings that must stay the same within a thread
+ * and the messages the server already holds.
+ *
+ * - Settings, as sent: the endpoint (a thread exists on one endpoint only), and the fixed request
+ *   fields. A `continue` omits `system` and `tools` and the server keeps the thread's own, so only
+ *   this check can notice a change to them, including their cache markers. The server rejects
+ *   changes to the other fields with `thread_fingerprint_mismatch`; checking them here saves that
+ *   rejected request. A fixed field missing here costs only that rejection, which also starts a
+ *   new thread.
+ * - Messages, normalized so messages that mean the same to the API hash the same.
+ */
+function hashThreadPrefix(model, params, messages) {
+    const { system, tools, thinking, output_config, tool_choice, betas, fallbacks } = params;
+    const settings = JSON.stringify([
+        model.provider,
+        model.baseUrl,
+        params.model,
+        system,
+        tools,
+        thinking,
+        output_config,
+        tool_choice,
+        betas,
+        fallbacks,
+    ]);
+    const conversation = JSON.stringify(messages, (key, value) => {
+        // Cache markers don't change what a message means, but they move: pi puts one on the newest
+        // message, so the message that had it in the last request has none in this one. Extensions
+        // may add markers anywhere too.
+        if (key === "cache_control")
+            return undefined;
+        // To attach that marker, the newest user message's string content is sent as one text block;
+        // the next request sends the same message as a string: "hi" vs [{ type: "text", text: "hi" }].
+        if (key === "content" && typeof value === "string")
+            return [{ type: "text", text: value }];
+        return value;
+    });
+    return shortHash(settings + conversation);
+}
+function createThreadRequest(sessionId, params) {
+    return { sessionId, body: { ...params, thread: { type: "create" } } };
+}
+function prepareThreadRequest(model, sessionId, params) {
+    const state = threadState.get(sessionId);
+    if (state === "off")
+        return undefined;
+    if (state !== undefined &&
+        params.messages.length > state.prefixCount &&
+        hashThreadPrefix(model, params, params.messages.slice(0, state.prefixCount)) === state.prefixHash) {
+        // Each reply can be continued once; deleting also keeps a concurrent request from trying.
+        threadState.delete(sessionId);
+        const { system: _system, tools: _tools, ...rest } = params;
+        return {
+            body: {
+                ...rest,
+                messages: params.messages.slice(state.prefixCount),
+                thread: { type: "continue", previous_message_id: state.replyId },
+            },
+            sessionId,
+        };
+    }
+    return createThreadRequest(sessionId, params);
+}
+/** The `error_code` Anthropic puts in `error.details` of thread rejections. */
+function getThreadErrorCode(error) {
+    const body = error.error;
+    const code = body?.error?.details?.error_code;
+    return typeof code === "string" ? code : undefined;
+}
+/**
+ * Sends a thread request. A rejection about the thread (never a started stream) is resent once in
+ * full: as a new thread when the old one can't be continued, or without `thread` when the endpoint
+ * can't use threads, which also turns threads off for the session. Other errors are passed on.
+ * Returns the response and the thread request that produced it, if any.
+ */
+async function sendThreadRequest(thread, params, send, output, signal) {
+    try {
+        return { response: await send(thread.body), thread };
+    }
+    catch (error) {
+        if (signal?.aborted || !(error instanceof Anthropic.APIError))
+            throw error;
+        const errorCode = getThreadErrorCode(error);
+        const startNewThread = errorCode !== undefined && NEW_THREAD_ERRORS.has(errorCode);
+        // The endpoint can't use threads: it can't thread this request, or it doesn't know the
+        // `thread` field and rejects it without a code ("thread: Extra inputs are not permitted"),
+        // e.g. with an `anthropic-beta` header that lacks the threads beta.
+        const threadsUnsupported = errorCode === "thread_unsupported_request" ||
+            (errorCode === undefined && error.status === 400 && error.message.includes("thread"));
+        // Any other error (prompt too long, unknown model, strict tool limits, ...) is not about the
+        // thread, and resending would only fail again.
+        if (!startNewThread && !threadsUnsupported)
+            throw error;
+        // Recorded before the resend, so a failed resend keeps it.
+        appendAssistantMessageDiagnostic(output, {
+            type: "anthropic_thread_fallback",
+            timestamp: Date.now(),
+            error: { message: error.message },
+            details: { reason: errorCode ?? String(error.status) },
+        });
+        if (startNewThread) {
+            const created = createThreadRequest(thread.sessionId, params);
+            return { response: await send(created.body), thread: created };
+        }
+        // With the threads beta, a request without `thread` is a normal request.
+        const response = await send(params);
+        setThreadState(thread.sessionId, "off");
+        return { response, thread: undefined };
+    }
+}
+/**
+ * Records where the session's next request can continue: after `output`, the clean reply to the
+ * thread request `thread`, whose full request was `params`.
+ */
+function recordThreadReply(model, thread, params, output, isOAuth, allowEmptySignature) {
+    // `responseModel` is set only when a fallback model answered; the server does not store those
+    // replies, so they cannot be continued.
+    if (output.responseId === undefined || output.responseModel !== undefined)
+        return;
+    // Convert the reply like the next request will, keeping only the assistant message
+    // (transformMessages adds placeholder results for trailing tool calls).
+    const reply = convertMessages(transformMessages([output], model, normalizeToolCallId), isOAuth, undefined, allowEmptySignature).messages.filter((message) => message.role === "assistant");
+    const prefix = [...params.messages, ...reply];
+    setThreadState(thread.sessionId, {
+        prefixCount: prefix.length,
+        prefixHash: hashThreadPrefix(model, params, prefix),
+        replyId: output.responseId,
+    });
+}
 function mergeHeaders(...headerSources) {
     const merged = {};
     for (const headers of headerSources) {
@@ -377,7 +551,8 @@ async function* iterateAnthropicEvents(response, signal) {
 }
 export const stream = (model, context, options) => {
     const stream = new AssistantMessageEventStream();
-    const normalizedContext = resolveTranscript(context, getAnthropicCompat(model).supportsMidConvoSystemMessages);
+    const compat = getAnthropicCompat(model);
+    const normalizedContext = resolveTranscript(context, compat.supportsMidConvoSystemMessages);
     const currentTools = getCurrentTools(normalizedContext.messages);
     (async () => {
         const output = {
@@ -402,6 +577,12 @@ export const stream = (model, context, options) => {
             let isOAuth;
             let usageModel = model;
             let inputTransformations;
+            const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env);
+            const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
+            // Session whose requests may use threads; undefined when threads don't apply. Sessions where
+            // the endpoint rejected threads ("off") still send the threads beta, which the resend that
+            // turned them off was accepted with; `prepareThreadRequest()` just sends no `thread`.
+            const threadSessionId = compat.supportsMessageThreads ? cacheSessionId : undefined;
             if (options?.client) {
                 client = options.client;
                 isOAuth = false;
@@ -419,8 +600,6 @@ export const stream = (model, context, options) => {
                         hasImages,
                     });
                 }
-                const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env);
-                const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
                 const created = createClient(model, apiKey, options?.headers, options?.fetch, copilotDynamicHeaders, cacheSessionId, federation, options?.identity);
                 client = created.client;
                 isOAuth = created.isOAuthToken;
@@ -430,24 +609,38 @@ export const stream = (model, context, options) => {
                 ...(options?.timeoutMs !== undefined ? { timeout: options.timeoutMs } : {}),
                 maxRetries: 0,
             };
+            const send = (body) => retryProviderRequest(() => client.beta.messages.create(body, requestOptions).asResponse(), {
+                maxRetries: options?.maxRetries,
+                maxRetryDelayMs: options?.maxRetryDelayMs,
+                signal: options?.signal,
+            });
             let disablePreferredStrictTools = false;
             let response;
+            let params;
+            // The thread request that produced the response; undefined when no `thread` was sent.
+            let thread;
             for (;;) {
                 options?.signal?.throwIfAborted();
-                const built = buildParams(model, normalizedContext, isOAuth, options, disablePreferredStrictTools);
+                const built = buildParams(model, normalizedContext, isOAuth, options, {
+                    disablePreferredStrictTools,
+                    messageThreads: threadSessionId !== undefined,
+                });
                 if (built.providerThinkingLevel)
                     output.providerThinkingLevel = built.providerThinkingLevel;
-                let params = built.params;
+                params = built.params;
                 const nextParams = await options?.onPayload?.(params, model);
                 if (nextParams !== undefined) {
                     params = { ...nextParams, stream: true };
                 }
+                const prepared = threadSessionId !== undefined && options?.disableContinuation !== true
+                    ? prepareThreadRequest(model, threadSessionId, params)
+                    : undefined;
                 try {
-                    response = await retryProviderRequest(() => client.beta.messages.create(params, requestOptions).asResponse(), {
-                        maxRetries: options?.maxRetries,
-                        maxRetryDelayMs: options?.maxRetryDelayMs,
-                        signal: options?.signal,
-                    });
+                    const sent = prepared
+                        ? await sendThreadRequest(prepared, params, send, output, options?.signal)
+                        : { response: await send(params), thread: undefined };
+                    response = sent.response;
+                    thread = sent.thread;
                     break;
                 }
                 catch (error) {
@@ -458,11 +651,7 @@ export const stream = (model, context, options) => {
                         !(error instanceof Anthropic.APIError) ||
                         error.status !== 400 ||
                         error.type !== "invalid_request_error" ||
-                        ![
-                            "The compiled grammar is too large",
-                            "Too many strict tools (",
-                            "Schemas contains too many parameters with union types (",
-                        ].some((message) => error.message.includes(message))) {
+                        !isStrictToolLimitErrorMessage(error.message)) {
                         throw error;
                     }
                     disablePreferredStrictTools = true;
@@ -687,6 +876,8 @@ export const stream = (model, context, options) => {
             if (output.stopReason === "aborted" || output.stopReason === "error") {
                 throw new Error(output.errorMessage || "An unknown error occurred");
             }
+            if (thread)
+                recordThreadReply(model, thread, params, output, isOAuth, compat.allowEmptySignature);
             if (inputTransformations && inputTransformations.length > 0) {
                 appendAssistantMessageDiagnostic(output, {
                     type: "anthropic_input_transformations",
@@ -705,7 +896,7 @@ export const stream = (model, context, options) => {
         }
         catch (error) {
             output.stopReason = options?.signal?.aborted ? "aborted" : "error";
-            output.errorMessage = error instanceof Error ? error.message : JSON.stringify(error);
+            output.errorMessage = error instanceof Error ? withCauseDetail(error) : JSON.stringify(error);
             stream.push({ type: "error", reason: output.stopReason, error: output });
             stream.end();
         }
@@ -840,7 +1031,7 @@ function createClient(model, apiKey, optionsHeaders, fetch, dynamicHeaders, sess
     });
     return { client, isOAuthToken: false };
 }
-function getBetaFeatures(model, context, isOAuthToken, nativeToolChanges, options) {
+function getBetaFeatures(model, context, options, { isOAuthToken, nativeToolChanges, messageThreads }) {
     let configuredFeatures;
     for (const headers of [model.headers, options?.headers]) {
         for (const [name, value] of Object.entries(headers ?? {})) {
@@ -876,9 +1067,11 @@ function getBetaFeatures(model, context, isOAuthToken, nativeToolChanges, option
     }
     if (nativeToolChanges)
         features.push(INLINE_TOOLS_BETA);
+    if (messageThreads)
+        features.push(MESSAGE_THREADS_BETA);
     return [...new Set(features)];
 }
-function buildParams(model, context, isOAuthToken, options, disablePreferredStrictTools = false) {
+function buildParams(model, context, isOAuthToken, options, flags) {
     const { cacheControl } = getCacheControl(model, options?.cacheRetention, options?.env);
     const compat = getAnthropicCompat(model);
     const initialSystemMessage = getInitialSystemMessage(context.messages);
@@ -891,11 +1084,8 @@ function buildParams(model, context, isOAuthToken, options, disablePreferredStri
     // active tool to anchor the placeholder. Otherwise the current tool list is sent.
     let hasPreferredStrictTools = false;
     const convertToolDefinitions = (tools, toolCacheControl) => {
-        const definitions = convertTools(tools, isOAuthToken, compat.supportsEagerToolInputStreaming, compat.supportsStrictTools, toolCacheControl, disablePreferredStrictTools);
-        if (definitions.some((definition, index) => {
-            const config = tools[index].constrainedSampling;
-            return definition.strict === true && config && config.type === "json_schema" && config.strict === "prefer";
-        })) {
+        const definitions = convertTools(tools, isOAuthToken, compat.supportsEagerToolInputStreaming, compat.supportsStrictTools, toolCacheControl, flags.disablePreferredStrictTools);
+        if (definitions.some((definition, index) => definition.strict === true && prefersJsonSchemaStrictSampling(tools[index]))) {
             hasPreferredStrictTools = true;
         }
         return definitions;
@@ -907,7 +1097,11 @@ function buildParams(model, context, isOAuthToken, options, disablePreferredStri
     const effortMessages = model.compat?.supportsMidConvoEffort === true && !thinkingDisabled;
     const converted = convertMessages(conversationMessages, isOAuthToken, cacheControl, compat.allowEmptySignature, effortMessages ? model.provider : undefined, nativeToolChanges ? convertToolDefinitions : undefined);
     const activeEffort = options?.effort ?? "high";
-    const betaFeatures = getBetaFeatures(model, context, isOAuthToken, nativeToolChanges, options);
+    const betaFeatures = getBetaFeatures(model, context, options, {
+        isOAuthToken,
+        nativeToolChanges,
+        messageThreads: flags.messageThreads,
+    });
     const params = {
         model: model.id,
         messages: effortMessages ? insertThinkingLevelMessages(converted, activeEffort) : converted.messages,
@@ -1295,16 +1489,19 @@ function convertTools(tools, isOAuthToken, supportsEagerToolInputStreaming, supp
     if (!tools)
         return [];
     return tools.map((tool, index) => {
-        const config = tool.constrainedSampling;
-        const strict = disablePreferredStrictTools && config && config.type === "json_schema" && config.strict === "prefer"
+        const strict = disablePreferredStrictTools && prefersJsonSchemaStrictSampling(tool)
             ? undefined
             : resolveJsonSchemaStrictSampling(tool, supportsStrictTools, isAnthropicStrictUnsupportedKeyword);
         const parameters = getJsonSchemaToolParameters(tool, strict);
         const schema = parameters;
+        // Keep shared definitions so property $refs still resolve. Other root keywords stay out:
+        // Anthropic rejects root anyOf/oneOf/allOf with a 400.
         const legacyInputSchema = {
             type: "object",
             properties: schema.properties ?? {},
             required: schema.required ?? [],
+            ...(schema.$defs === undefined ? {} : { $defs: schema.$defs }),
+            ...(schema.definitions === undefined ? {} : { definitions: schema.definitions }),
         };
         const inputSchema = strict === true
             ? {