@earendil-works/pi-ai 1.5.0 → 1.5.1

package/README.md

@@ -64,7 +64,7 @@ Unified LLM API with provider collections, automatic auth resolution, token and
 - **OpenAI** (including the Decisions classifier API)
 - **Ant Ling**
 - **Azure OpenAI (Responses)**
-- **OpenAI Codex (legacy)** (ChatGPT Plus/Pro subscription, requires OAuth, see below)
+- **OpenAI Codex** (ChatGPT Plus/Pro subscription, requires OAuth, see below)
 - **Radius** (API key or OAuth, with a dynamically refreshed gateway catalog)
 - **TypeSafe** (System One classifier API)
 - **DeepSeek**
@@ -524,6 +524,7 @@ Built-in providers resolve these env vars (Node.js; in browsers pass `apiKey` ex
 | Qwen Token Plan (existing catalog) | `QWEN_TOKEN_PLAN_API_KEY` |
 | Qwen Token Plan (Individual) | `QWEN_TOKEN_PLAN_API_KEY` |
 | Qwen Token Plan (China) | `QWEN_TOKEN_PLAN_CN_API_KEY` |
+| Qwen Token Plan (Individual, China) | `QWEN_TOKEN_PLAN_CN_API_KEY` |
 | Xiaomi MiMo (API billing) | `XIAOMI_API_KEY` |
 | Xiaomi MiMo Token Plan (China) | `XIAOMI_TOKEN_PLAN_CN_API_KEY` |
 | Xiaomi MiMo Token Plan (Amsterdam) | `XIAOMI_TOKEN_PLAN_AMS_API_KEY` |
@@ -533,6 +534,8 @@ Built-in providers resolve these env vars (Node.js; in browsers pass `apiKey` ex
 `qwen-token-plan-individual` and `qwen-token-plan` share the international endpoint and
 `QWEN_TOKEN_PLAN_API_KEY`. The Individual provider exposes only the models documented for Individual
 subscriptions, while the existing provider retains its broader catalog for backward compatibility.
+`qwen-token-plan-individual-cn` applies the same Individual catalog to the China endpoint and
+`QWEN_TOKEN_PLAN_CN_API_KEY`.
 Stored credentials remain provider-scoped, so save the key under the provider ID you register.
 
 Amazon Bedrock resolves ambient AWS credentials (`AWS_PROFILE`, access key pairs, `AWS_BEARER_TOKEN_BEDROCK`, ECS task roles, web identity tokens); its provider-owned login flow supports bearer tokens, AWS profiles, and the existing credential chain. Vertex AI resolves either an explicit key or gcloud Application Default Credentials plus project/location, with a provider-owned login flow for API keys, ADC, and service-account files.
@@ -891,7 +894,7 @@ console.log(model.output); // ['image'] or ['image', 'text']
 
 ## Classification
 
-Classifier models consume structured JSON state, and optionally images, and answer one or more typed questions. They do not use chat or image-generation APIs. TypeSafe's Jev model, Cloudflare's Clef models, and OpenAI's GPT-6 Luna are available from these built-in providers:
+Classifier models consume structured JSON state, and optionally images, and answer one or more typed questions. They do not use chat or image-generation APIs. TypeSafe's Jev model, Cloudflare's Clef models, OpenAI's GPT-6 Luna, and Microsoft-Decision-1 are available from these built-in providers:
 
 | Provider | Model IDs | Auth |
 | --- | --- | --- |
@@ -901,6 +904,7 @@ Classifier models consume structured JSON state, and optionally images, and answ
 | `vercel-ai-gateway` | `typesafe-ai/jev` | `AI_GATEWAY_API_KEY` |
 | `opencode` | `jev-1.13`, `jev-1.13-free` | `OPENCODE_API_KEY` |
 | `openai` | `gpt-6-luna` | `OPENAI_API_KEY` |
+| `github-copilot` | `microsoft-decision-1` | `COPILOT_GITHUB_TOKEN` or GitHub Copilot OAuth |
 
 ```typescript
 import { builtinModels } from '@earendil-works/pi-ai/providers/all';
@@ -1437,6 +1441,8 @@ Built-in API implementations live under `./api/<api-id>`:
 
 Importing an implementation module loads its SDK. The `./api/<id>.lazy` wrappers (used by the provider factories) defer that load to the first request when the runtime or bundler supports dynamic import chunking. Legacy raw API subpaths from older releases (`./anthropic`, `./google`, `./mistral`, `./openai-completions`, ...) were removed; use `@earendil-works/pi-ai/api/<api-id>`.
 
+**Anthropic message threads:** with `sessionId` set and `cacheRetention` not `"none"`, Anthropic requests continue from the previous reply on the server (`message-threads-2026-08-12` beta) and send only the new messages. When anything earlier in the request changed, or the server can't continue, the full request is sent. Token counts and prompt caching are unchanged. On by default only for the `anthropic` provider on `api.anthropic.com`; set `compat.supportsMessageThreads` to change that, for example for gateways that forward the `thread` field. Set `disableContinuation: true` on requests that repeat an earlier one, such as cache warming. Call `cleanupSessionResources(sessionId)` when a session ends.
+
 ### OpenAI Compatibility Settings
 
 The `openai-completions` API is implemented by many providers with minor differences. By default, the library auto-detects compatibility settings based on `baseUrl` for a small set of known OpenAI-compatible providers (Cerebras, xAI, Chutes, DeepSeek, NVIDIA NIM, Together AI, zAi, OpenCode, Cloudflare Workers AI, etc.). For custom proxies or unknown endpoints, you can override these settings via the `compat` field. For `openai-responses` models, the compat field supports Responses-specific flags.
@@ -1456,6 +1462,7 @@ interface OpenAICompletionsCompat {
   maxTokensField?: 'max_completion_tokens' | 'max_tokens';  // Which field name to use (default: max_completion_tokens)
   requiresToolResultName?: boolean;  // Whether tool results require the `name` field (default: false)
   requiresAssistantAfterToolResult?: boolean; // Whether tool results must be followed by an assistant message (default: false)
+  requiresNonNullAssistantContent?: boolean; // Whether replayed assistant messages with no text must send "" instead of null content (default: false)
   requiresThinkingAsText?: boolean;  // Whether thinking blocks must be converted to text (default: false)
   requiresReasoningContentOnAssistantMessages?: boolean; // Whether all replayed assistant messages must include empty reasoning_content when reasoning is enabled (default: auto-detected for DeepSeek)
   thinkingFormat?: 'openai' | 'openrouter' | 'deepseek' | 'together' | 'baseten' | 'zai' | 'qwen' | 'chat-template' | 'qwen-chat-template' | 'string-thinking' | 'ant-ling'; // Format for reasoning param: 'openai' uses reasoning_effort, 'openrouter' uses reasoning: { effort }, 'deepseek' uses thinking: { type } plus reasoning_effort when supported, 'together' uses reasoning: { enabled } plus reasoning_effort when supported, 'baseten' uses configurable chat_template_args plus reasoning_effort when supported, 'zai' uses thinking: { type }, 'qwen' uses enable_thinking, 'chat-template' uses configurable chat_template_kwargs, 'qwen-chat-template' uses chat_template_kwargs.enable_thinking and preserve_thinking, 'string-thinking' uses top-level thinking, 'ant-ling' uses reasoning: { effort } only for mapped efforts (default: openai)
@@ -1764,6 +1771,18 @@ setBedrockProviderModule(bedrockProviderModule);
 
 That explicit override bundles the AWS SDK. Without it, Bedrock's opaque runtime import expects the package's Bedrock implementation file to be available at runtime.
 
+Node bundles that split at dynamic imports (esbuild `splitting: true`) can keep Bedrock and the OAuth login flows inside the bundle but lazy: register loaders with literal dynamic imports, which the bundler turns into chunks that load on first use:
+
+```typescript
+import { registerLazyBedrockProvider } from '@earendil-works/pi-ai/lazy-bedrock-provider';
+import { registerLazyOAuthFlows } from '@earendil-works/pi-ai/lazy-oauth';
+
+registerLazyBedrockProvider();
+registerLazyOAuthFlows();
+```
+
+Both modules are Node-only; never import them into a browser bundle.
+
 ### Provider-Scoped Environment Overrides
 
 Pass `env` in stream options to scope provider configuration to a request. Values in `env` are used before process environment variables for provider auth and configuration such as Cloudflare account IDs, Azure OpenAI settings, Vertex project/location, Bedrock settings, `PI_CACHE_RETENTION`, and `HTTP_PROXY`/`HTTPS_PROXY`.
@@ -1789,7 +1808,7 @@ Several providers support OAuth authentication instead of static API keys:
 
 - **Anthropic** (Claude Pro/Max subscription)
 - **OpenAI** (Sign in with ChatGPT: uses the ChatGPT subscription with the OpenAI API)
-- **OpenAI Codex (legacy)** (ChatGPT Plus/Pro subscription, access to GPT-5.x Codex models)
+- **OpenAI Codex** (ChatGPT Plus/Pro subscription, access to GPT-5.x Codex models)
 - **GitHub Copilot** (Copilot subscription)
 - **OpenRouter** (OAuth PKCE that mints a user-controlled API key)
 
@@ -1869,7 +1888,7 @@ Built-in login and refresh flows are private provider implementations. Use provi
 
 Provider notes:
 
-**OpenAI Codex (legacy)**: Superseded by Sign in with ChatGPT on the OpenAI provider. Requires a ChatGPT Plus or Pro subscription. Provides access to GPT-5.x Codex models with extended context windows and reasoning capabilities. The library automatically handles session-based prompt caching when `sessionId` is provided in stream options unless `cacheRetention` is `"none"`. You can set `transport` in stream options to `"sse"`, `"websocket"`, or `"auto"` for Codex Responses transport selection. When using WebSocket with a `sessionId` and cache retention enabled, connections are reused per session and expire after 5 minutes of inactivity. Call `cleanupSessionResources(sessionId)` when finished so the pooled connection does not keep the process alive.
+**OpenAI Codex**: Superseded by Sign in with ChatGPT on the OpenAI provider. Requires a ChatGPT Plus or Pro subscription. Provides access to GPT-5.x Codex models with extended context windows and reasoning capabilities. The library automatically handles session-based prompt caching when `sessionId` is provided in stream options unless `cacheRetention` is `"none"`. You can set `transport` in stream options to `"sse"`, `"websocket"`, or `"auto"` for Codex Responses transport selection. When using WebSocket with a `sessionId` and cache retention enabled, connections are reused per session and expire after 5 minutes of inactivity. Call `cleanupSessionResources(sessionId)` when finished so the pooled connection does not keep the process alive.
 
 **Azure OpenAI (Responses)**: Uses the Responses API only. Set `AZURE_OPENAI_API_KEY` and either `AZURE_OPENAI_BASE_URL` or `AZURE_OPENAI_RESOURCE_NAME`. `AZURE_OPENAI_BASE_URL` supports both `https://<resource>.openai.azure.com` and `https://<resource>.cognitiveservices.azure.com`; root endpoints are normalized to `.../openai/v1` automatically. Use `AZURE_OPENAI_API_VERSION` (defaults to `v1`) to override the API version if needed. Deployment names are treated as model IDs by default, override with `azureDeploymentName` or `AZURE_OPENAI_DEPLOYMENT_NAME_MAP` using comma-separated `model-id=deployment` pairs (for example `gpt-4o-mini=my-deployment,gpt-4o=prod`). Legacy deployment-based URLs are intentionally unsupported.
 

package/dist/api/anthropic-messages.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"anthropic-messages.d.ts","sourceRoot":"","sources":["../../src/api/anthropic-messages.ts"],"names":[],"mappings":"AAAA,OAAO,SAAiC,MAAM,mBAAmB,CAAC;AAoBlE,OAAO,KAAK,EASX,mBAAmB,EAEnB,cAAc,EACd,aAAa,EAIb,MAAM,aAAa,CAAC;AA+JrB,MAAM,MAAM,eAAe,GAAG,KAAK,GAAG,QAAQ,GAAG,MAAM,GAAG,OAAO,GAAG,KAAK,CAAC;AAE1E,MAAM,MAAM,wBAAwB,GAAG,YAAY,GAAG,SAAS,CAAC;AA2ChE,MAAM,WAAW,gBAAiB,SAAQ,aAAa;IACtD;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,OAAO,CAAC;IAC1B;;;;OAIG;IACH,oBAAoB,CAAC,EAAE,MAAM,CAAC;IAC9B;;;;;;;;;;;OAWG;IACH,MAAM,CAAC,EAAE,eAAe,CAAC;IACzB;;;;;;;;;;;OAWG;IACH,eAAe,CAAC,EAAE,wBAAwB,CAAC;IAC3C;;;;;OAKG;IACH,mBAAmB,CAAC,EAAE,OAAO,CAAC;IAC9B;;;;OAIG;IACH,UAAU,CAAC,EAAE,MAAM,GAAG,KAAK,GAAG,MAAM,GAAG;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;IACtE;;;;OAIG;IACH,MAAM,CAAC,EAAE,SAAS,CAAC;CACnB;AAoRD,eAAO,MAAM,MAAM,EAAE,cAAc,CAAC,oBAAoB,EAAE,gBAAgB,CA8VzE,CAAC;AA2BF,eAAO,MAAM,YAAY,EAAE,cAAc,CAAC,oBAAoB,EAAE,mBAAmB,CAgDlF,CAAC"}
+{"version":3,"file":"anthropic-messages.d.ts","sourceRoot":"","sources":["../../src/api/anthropic-messages.ts"],"names":[],"mappings":"AAAA,OAAO,SAAiC,MAAM,mBAAmB,CAAC;AAqBlE,OAAO,KAAK,EASX,mBAAmB,EAEnB,cAAc,EACd,aAAa,EAIb,MAAM,aAAa,CAAC;AAqKrB,MAAM,MAAM,eAAe,GAAG,KAAK,GAAG,QAAQ,GAAG,MAAM,GAAG,OAAO,GAAG,KAAK,CAAC;AAE1E,MAAM,MAAM,wBAAwB,GAAG,YAAY,GAAG,SAAS,CAAC;AA6QhE,MAAM,WAAW,gBAAiB,SAAQ,aAAa;IACtD;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,OAAO,CAAC;IAC1B;;;;OAIG;IACH,oBAAoB,CAAC,EAAE,MAAM,CAAC;IAC9B;;;;;;;;;;;OAWG;IACH,MAAM,CAAC,EAAE,eAAe,CAAC;IACzB;;;;;;;;;;;OAWG;IACH,eAAe,CAAC,EAAE,wBAAwB,CAAC;IAC3C;;;;;OAKG;IACH,mBAAmB,CAAC,EAAE,OAAO,CAAC;IAC9B;;;;OAIG;IACH,UAAU,CAAC,EAAE,MAAM,GAAG,KAAK,GAAG,MAAM,GAAG;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;IACtE;;;;OAIG;IACH,MAAM,CAAC,EAAE,SAAS,CAAC;CACnB;AAoRD,eAAO,MAAM,MAAM,EAAE,cAAc,CAAC,oBAAoB,EAAE,gBAAgB,CAiXzE,CAAC;AA2BF,eAAO,MAAM,YAAY,EAAE,cAAc,CAAC,oBAAoB,EAAE,mBAAmB,CAgDlF,CAAC"}

package/dist/api/anthropic-messages.js

@@ -1,8 +1,11 @@
 import Anthropic, {} from "@anthropic-ai/sdk";
 import { ANTHROPIC_FEDERATION_RULE_ID_ENV, ANTHROPIC_IDENTITY_TOKEN_FILE_ENV, ANTHROPIC_ORGANIZATION_ID_ENV, ANTHROPIC_SERVICE_ACCOUNT_ID_ENV, ANTHROPIC_WORKSPACE_ID_ENV, } from "../env-api-keys.js";
 import { calculateCost } from "../models.js";
+import { registerSessionResourceCleanup } from "../session-resources.js";
 import { appendAssistantMessageDiagnostic } from "../utils/diagnostics.js";
+import { withCauseDetail } from "../utils/error-body.js";
 import { AssistantMessageEventStream } from "../utils/event-stream.js";
+import { shortHash } from "../utils/hash.js";
 import { headersToRecord } from "../utils/headers.js";
 import { getIdentityHeaders } from "../utils/identity.js";
 import { parseJsonWithRepair } from "../utils/json-parse.js";
@@ -13,9 +16,10 @@ import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
 import { ToolCallArgumentsStream } from "../utils/streaming-json.js";
 import { getSystemMessageText, renderSystemMessageUpdate } from "../utils/text.js";
 import { getCurrentTools, getInitialSystemMessage, resolveTranscript, } from "../utils/transcript.js";
-import { getJsonSchemaToolParameters, isStrictToolLimitErrorMessage, prefersJsonSchemaStrictSampling, resolveJsonSchemaStrictSampling, } from "./constrained-sampling.js";
+import { getJsonSchemaToolParameters, prefersJsonSchemaStrictSampling, resolveJsonSchemaStrictSampling, } from "./constrained-sampling.js";
 import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.js";
 import { adjustMaxTokensForThinking, buildBaseOptions, clampMaxTokensToContext } from "./simple-options.js";
+import { hasExceededStrictToolLimits, isStrictToolLimitErrorMessage, markStrictToolLimitsExceeded, } from "./strict-tool-limits.js";
 import { splitErrorToolResultImages, transformMessages } from "./transform-messages.js";
 /**
  * Resolve cache retention preference.
@@ -122,6 +126,7 @@ const SERVER_SIDE_FALLBACK_BETA = "server-side-fallback-2026-07-01";
 const MID_CONVERSATION_OUTPUT_CONFIG_BETA = "mid-conversation-output-config-2026-07-01";
 const THINKING_BINDING_CONTROLS_BETA = "thinking-binding-controls-2026-08-01";
 const INLINE_TOOLS_BETA = "inline-tools-2026-09-15";
+const MESSAGE_THREADS_BETA = "message-threads-2026-08-12";
 /**
  * Stable deferred tool declared whenever native tool changes are in use. Anthropic adds
  * hidden prompt scaffolding for mid-conversation tool changes; declaring this placeholder
@@ -151,8 +156,178 @@ function getAnthropicCompat(model) {
         supportsStrictTools: model.compat?.supportsStrictTools ?? false,
         supportsMidConvoSystemMessages: model.compat?.supportsMidConvoSystemMessages ?? false,
         supportsMidConvoToolChanges: model.compat?.supportsMidConvoToolChanges ?? false,
+        // Gateways that drop unknown request fields would silently turn a `continue` into a request
+        // without the earlier conversation, so the default is limited to Anthropic's own endpoint.
+        // Widen or remove this restriction once proxies generally forward the `thread` field.
+        supportsMessageThreads: model.compat?.supportsMessageThreads ??
+            (model.provider === "anthropic" && model.baseUrl.includes("api.anthropic.com")),
     };
 }
+/** Per session: where the next request can continue, or "off" after the endpoint rejected threads. */
+const threadState = new Map();
+const MAX_THREAD_STATE_ENTRIES = 10_000;
+/** Rejections of a `continue` that a new thread fixes: expired, raced, or a fixed setting changed. */
+const NEW_THREAD_ERRORS = new Set([
+    "thread_not_found",
+    "thread_already_continued",
+    "thread_continue_in_flight",
+    "thread_fingerprint_mismatch",
+]);
+/**
+ * Sets a session's thread state, keeping at most `MAX_THREAD_STATE_ENTRIES` sessions.
+ *
+ * State is normally freed by `cleanupSessionResources(sessionId)` when a session ends, as pi's
+ * coding agent does. The cap bounds memory for library users that never call it, for example a
+ * server passing a new `sessionId` per conversation. Dropping the least recently updated session
+ * is cheap: its thread has most likely expired on the server already (after a few minutes idle),
+ * and otherwise its next request just starts a new thread.
+ */
+function setThreadState(sessionId, value) {
+    // Re-insert so Map order stays oldest-first for the cap.
+    threadState.delete(sessionId);
+    threadState.set(sessionId, value);
+    if (threadState.size > MAX_THREAD_STATE_ENTRIES) {
+        const oldest = threadState.keys().next().value;
+        if (oldest !== undefined)
+            threadState.delete(oldest);
+    }
+}
+registerSessionResourceCleanup((sessionId) => {
+    if (sessionId)
+        threadState.delete(sessionId);
+    else
+        threadState.clear();
+});
+/**
+ * Hash of everything a `continue` builds on: the settings that must stay the same within a thread
+ * and the messages the server already holds.
+ *
+ * - Settings, as sent: the endpoint (a thread exists on one endpoint only), and the fixed request
+ *   fields. A `continue` omits `system` and `tools` and the server keeps the thread's own, so only
+ *   this check can notice a change to them, including their cache markers. The server rejects
+ *   changes to the other fields with `thread_fingerprint_mismatch`; checking them here saves that
+ *   rejected request. A fixed field missing here costs only that rejection, which also starts a
+ *   new thread.
+ * - Messages, normalized so messages that mean the same to the API hash the same.
+ */
+function hashThreadPrefix(model, params, messages) {
+    const { system, tools, thinking, output_config, tool_choice, betas, fallbacks } = params;
+    const settings = JSON.stringify([
+        model.provider,
+        model.baseUrl,
+        params.model,
+        system,
+        tools,
+        thinking,
+        output_config,
+        tool_choice,
+        betas,
+        fallbacks,
+    ]);
+    const conversation = JSON.stringify(messages, (key, value) => {
+        // Cache markers don't change what a message means, but they move: pi puts one on the newest
+        // message, so the message that had it in the last request has none in this one. Extensions
+        // may add markers anywhere too.
+        if (key === "cache_control")
+            return undefined;
+        // To attach that marker, the newest user message's string content is sent as one text block;
+        // the next request sends the same message as a string: "hi" vs [{ type: "text", text: "hi" }].
+        if (key === "content" && typeof value === "string")
+            return [{ type: "text", text: value }];
+        return value;
+    });
+    return shortHash(settings + conversation);
+}
+function createThreadRequest(sessionId, params) {
+    return { sessionId, body: { ...params, thread: { type: "create" } } };
+}
+function prepareThreadRequest(model, sessionId, params) {
+    const state = threadState.get(sessionId);
+    if (state === "off")
+        return undefined;
+    if (state !== undefined &&
+        params.messages.length > state.prefixCount &&
+        hashThreadPrefix(model, params, params.messages.slice(0, state.prefixCount)) === state.prefixHash) {
+        // Each reply can be continued once; deleting also keeps a concurrent request from trying.
+        threadState.delete(sessionId);
+        const { system: _system, tools: _tools, ...rest } = params;
+        return {
+            body: {
+                ...rest,
+                messages: params.messages.slice(state.prefixCount),
+                thread: { type: "continue", previous_message_id: state.replyId },
+            },
+            sessionId,
+        };
+    }
+    return createThreadRequest(sessionId, params);
+}
+/** The `error_code` Anthropic puts in `error.details` of thread rejections. */
+function getThreadErrorCode(error) {
+    const body = error.error;
+    const code = body?.error?.details?.error_code;
+    return typeof code === "string" ? code : undefined;
+}
+/**
+ * Sends a thread request. A rejection about the thread (never a started stream) is resent once in
+ * full: as a new thread when the old one can't be continued, or without `thread` when the endpoint
+ * can't use threads, which also turns threads off for the session. Other errors are passed on.
+ * Returns the response and the thread request that produced it, if any.
+ */
+async function sendThreadRequest(thread, params, send, output, signal) {
+    try {
+        return { response: await send(thread.body), thread };
+    }
+    catch (error) {
+        if (signal?.aborted || !(error instanceof Anthropic.APIError))
+            throw error;
+        const errorCode = getThreadErrorCode(error);
+        const startNewThread = errorCode !== undefined && NEW_THREAD_ERRORS.has(errorCode);
+        // The endpoint can't use threads: it can't thread this request, or it doesn't know the
+        // `thread` field and rejects it without a code ("thread: Extra inputs are not permitted"),
+        // e.g. with an `anthropic-beta` header that lacks the threads beta.
+        const threadsUnsupported = errorCode === "thread_unsupported_request" ||
+            (errorCode === undefined && error.status === 400 && error.message.includes("thread"));
+        // Any other error (prompt too long, unknown model, strict tool limits, ...) is not about the
+        // thread, and resending would only fail again.
+        if (!startNewThread && !threadsUnsupported)
+            throw error;
+        // Recorded before the resend, so a failed resend keeps it.
+        appendAssistantMessageDiagnostic(output, {
+            type: "anthropic_thread_fallback",
+            timestamp: Date.now(),
+            error: { message: error.message },
+            details: { reason: errorCode ?? String(error.status) },
+        });
+        if (startNewThread) {
+            const created = createThreadRequest(thread.sessionId, params);
+            return { response: await send(created.body), thread: created };
+        }
+        // With the threads beta, a request without `thread` is a normal request.
+        const response = await send(params);
+        setThreadState(thread.sessionId, "off");
+        return { response, thread: undefined };
+    }
+}
+/**
+ * Records where the session's next request can continue: after `output`, the clean reply to the
+ * thread request `thread`, whose full request was `params`.
+ */
+function recordThreadReply(model, thread, params, output, isOAuth, allowEmptySignature) {
+    // `responseModel` is set only when a fallback model answered; the server does not store those
+    // replies, so they cannot be continued.
+    if (output.responseId === undefined || output.responseModel !== undefined)
+        return;
+    // Convert the reply like the next request will, keeping only the assistant message
+    // (transformMessages adds placeholder results for trailing tool calls).
+    const reply = convertMessages(transformMessages([output], model, normalizeToolCallId), isOAuth, undefined, allowEmptySignature).messages.filter((message) => message.role === "assistant");
+    const prefix = [...params.messages, ...reply];
+    setThreadState(thread.sessionId, {
+        prefixCount: prefix.length,
+        prefixHash: hashThreadPrefix(model, params, prefix),
+        replyId: output.responseId,
+    });
+}
 function mergeHeaders(...headerSources) {
     const merged = {};
     for (const headers of headerSources) {
@@ -377,7 +552,8 @@ async function* iterateAnthropicEvents(response, signal) {
 }
 export const stream = (model, context, options) => {
     const stream = new AssistantMessageEventStream();
-    const normalizedContext = resolveTranscript(context, getAnthropicCompat(model).supportsMidConvoSystemMessages);
+    const compat = getAnthropicCompat(model);
+    const normalizedContext = resolveTranscript(context, compat.supportsMidConvoSystemMessages);
     const currentTools = getCurrentTools(normalizedContext.messages);
     (async () => {
         const output = {
@@ -402,6 +578,12 @@ export const stream = (model, context, options) => {
             let isOAuth;
             let usageModel = model;
             let inputTransformations;
+            const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env);
+            const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
+            // Session whose requests may use threads; undefined when threads don't apply. Sessions where
+            // the endpoint rejected threads ("off") still send the threads beta, which the resend that
+            // turned them off was accepted with; `prepareThreadRequest()` just sends no `thread`.
+            const threadSessionId = compat.supportsMessageThreads ? cacheSessionId : undefined;
             if (options?.client) {
                 client = options.client;
                 isOAuth = false;
@@ -419,8 +601,6 @@ export const stream = (model, context, options) => {
                         hasImages,
                     });
                 }
-                const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env);
-                const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
                 const created = createClient(model, apiKey, options?.headers, options?.fetch, copilotDynamicHeaders, cacheSessionId, federation, options?.identity);
                 client = created.client;
                 isOAuth = created.isOAuthToken;
@@ -430,24 +610,38 @@ export const stream = (model, context, options) => {
                 ...(options?.timeoutMs !== undefined ? { timeout: options.timeoutMs } : {}),
                 maxRetries: 0,
             };
-            let disablePreferredStrictTools = false;
+            const send = (body) => retryProviderRequest(() => client.beta.messages.create(body, requestOptions).asResponse(), {
+                maxRetries: options?.maxRetries,
+                maxRetryDelayMs: options?.maxRetryDelayMs,
+                signal: options?.signal,
+            });
+            let disablePreferredStrictTools = hasExceededStrictToolLimits(options?.sessionId);
             let response;
+            let params;
+            // The thread request that produced the response; undefined when no `thread` was sent.
+            let thread;
             for (;;) {
                 options?.signal?.throwIfAborted();
-                const built = buildParams(model, normalizedContext, isOAuth, options, disablePreferredStrictTools);
+                const built = buildParams(model, normalizedContext, isOAuth, options, {
+                    disablePreferredStrictTools,
+                    messageThreads: threadSessionId !== undefined,
+                });
                 if (built.providerThinkingLevel)
                     output.providerThinkingLevel = built.providerThinkingLevel;
-                let params = built.params;
+                params = built.params;
                 const nextParams = await options?.onPayload?.(params, model);
                 if (nextParams !== undefined) {
                     params = { ...nextParams, stream: true };
                 }
+                const prepared = threadSessionId !== undefined && options?.disableContinuation !== true
+                    ? prepareThreadRequest(model, threadSessionId, params)
+                    : undefined;
                 try {
-                    response = await retryProviderRequest(() => client.beta.messages.create(params, requestOptions).asResponse(), {
-                        maxRetries: options?.maxRetries,
-                        maxRetryDelayMs: options?.maxRetryDelayMs,
-                        signal: options?.signal,
-                    });
+                    const sent = prepared
+                        ? await sendThreadRequest(prepared, params, send, output, options?.signal)
+                        : { response: await send(params), thread: undefined };
+                    response = sent.response;
+                    thread = sent.thread;
                     break;
                 }
                 catch (error) {
@@ -462,6 +656,7 @@ export const stream = (model, context, options) => {
                         throw error;
                     }
                     disablePreferredStrictTools = true;
+                    markStrictToolLimitsExceeded(options?.sessionId);
                     appendAssistantMessageDiagnostic(output, {
                         type: "anthropic_strict_tools_fallback",
                         timestamp: Date.now(),
@@ -683,6 +878,8 @@ export const stream = (model, context, options) => {
             if (output.stopReason === "aborted" || output.stopReason === "error") {
                 throw new Error(output.errorMessage || "An unknown error occurred");
             }
+            if (thread)
+                recordThreadReply(model, thread, params, output, isOAuth, compat.allowEmptySignature);
             if (inputTransformations && inputTransformations.length > 0) {
                 appendAssistantMessageDiagnostic(output, {
                     type: "anthropic_input_transformations",
@@ -701,7 +898,7 @@ export const stream = (model, context, options) => {
         }
         catch (error) {
             output.stopReason = options?.signal?.aborted ? "aborted" : "error";
-            output.errorMessage = error instanceof Error ? error.message : JSON.stringify(error);
+            output.errorMessage = error instanceof Error ? withCauseDetail(error) : JSON.stringify(error);
             stream.push({ type: "error", reason: output.stopReason, error: output });
             stream.end();
         }
@@ -836,7 +1033,7 @@ function createClient(model, apiKey, optionsHeaders, fetch, dynamicHeaders, sess
     });
     return { client, isOAuthToken: false };
 }
-function getBetaFeatures(model, context, isOAuthToken, nativeToolChanges, options) {
+function getBetaFeatures(model, context, options, { isOAuthToken, nativeToolChanges, messageThreads }) {
     let configuredFeatures;
     for (const headers of [model.headers, options?.headers]) {
         for (const [name, value] of Object.entries(headers ?? {})) {
@@ -872,9 +1069,11 @@ function getBetaFeatures(model, context, isOAuthToken, nativeToolChanges, option
     }
     if (nativeToolChanges)
         features.push(INLINE_TOOLS_BETA);
+    if (messageThreads)
+        features.push(MESSAGE_THREADS_BETA);
     return [...new Set(features)];
 }
-function buildParams(model, context, isOAuthToken, options, disablePreferredStrictTools = false) {
+function buildParams(model, context, isOAuthToken, options, flags) {
     const { cacheControl } = getCacheControl(model, options?.cacheRetention, options?.env);
     const compat = getAnthropicCompat(model);
     const initialSystemMessage = getInitialSystemMessage(context.messages);
@@ -887,7 +1086,7 @@ function buildParams(model, context, isOAuthToken, options, disablePreferredStri
     // active tool to anchor the placeholder. Otherwise the current tool list is sent.
     let hasPreferredStrictTools = false;
     const convertToolDefinitions = (tools, toolCacheControl) => {
-        const definitions = convertTools(tools, isOAuthToken, compat.supportsEagerToolInputStreaming, compat.supportsStrictTools, toolCacheControl, disablePreferredStrictTools);
+        const definitions = convertTools(tools, isOAuthToken, compat.supportsEagerToolInputStreaming, compat.supportsStrictTools, toolCacheControl, flags.disablePreferredStrictTools);
         if (definitions.some((definition, index) => definition.strict === true && prefersJsonSchemaStrictSampling(tools[index]))) {
             hasPreferredStrictTools = true;
         }
@@ -900,7 +1099,11 @@ function buildParams(model, context, isOAuthToken, options, disablePreferredStri
     const effortMessages = model.compat?.supportsMidConvoEffort === true && !thinkingDisabled;
     const converted = convertMessages(conversationMessages, isOAuthToken, cacheControl, compat.allowEmptySignature, effortMessages ? model.provider : undefined, nativeToolChanges ? convertToolDefinitions : undefined);
     const activeEffort = options?.effort ?? "high";
-    const betaFeatures = getBetaFeatures(model, context, isOAuthToken, nativeToolChanges, options);
+    const betaFeatures = getBetaFeatures(model, context, options, {
+        isOAuthToken,
+        nativeToolChanges,
+        messageThreads: flags.messageThreads,
+    });
     const params = {
         model: model.id,
         messages: effortMessages ? insertThinkingLevelMessages(converted, activeEffort) : converted.messages,
@@ -1293,10 +1496,14 @@ function convertTools(tools, isOAuthToken, supportsEagerToolInputStreaming, supp
             : resolveJsonSchemaStrictSampling(tool, supportsStrictTools, isAnthropicStrictUnsupportedKeyword);
         const parameters = getJsonSchemaToolParameters(tool, strict);
         const schema = parameters;
+        // Keep shared definitions so property $refs still resolve. Other root keywords stay out:
+        // Anthropic rejects root anyOf/oneOf/allOf with a 400.
         const legacyInputSchema = {
             type: "object",
             properties: schema.properties ?? {},
             required: schema.required ?? [],
+            ...(schema.$defs === undefined ? {} : { $defs: schema.$defs }),
+            ...(schema.definitions === undefined ? {} : { definitions: schema.definitions }),
         };
         const inputSchema = strict === true
             ? {