workers-ai-provider 3.2.1 → 3.3.1

package/README.md

@@ -2,6 +2,10 @@
 
 [Workers AI](https://developers.cloudflare.com/workers-ai/) provider for the [AI SDK](https://sdk.vercel.ai/). Run Cloudflare's models for chat, embeddings, image generation, transcription, text-to-speech, reranking, and [AI Search](https://developers.cloudflare.com/ai-search/) — all from a single provider. It can also route **third-party** models (OpenAI, Anthropic, Google, …) through [AI Gateway](https://developers.cloudflare.com/ai-gateway/) — see [Third-party models](#third-party-models-via-ai-gateway).
 
+> 📚 In-depth guides and the AI Gateway **delegate** reference (unified catalog,
+> resumable streaming _(coming soon)_, server-side fallback) live in
+> [`docs/workers-ai-provider`](../../docs/workers-ai-provider/README.md).
+
 ## Quick start
 
 ```bash
@@ -333,7 +337,7 @@ Streaming works the same way — use `streamText` instead of `generateText`.
 
 > **⚠️ Experimental.** Everything in this section (routing third-party models via `createWorkersAI({ providers })`, the provider plugins, the registry, the resume layer, and `createGatewayFetch`/`createGatewayProvider`) is a new and substantial addition — well beyond the package's original job of wrapping Workers AI. Treat the whole surface as experimental: APIs may change, and several behaviors depend on undocumented AI Gateway internals (the `cf-aig-run-id` resume buffer, per-provider run-path wire formats). It does **not** affect the stable Workers AI / AI Search APIs above. Bug reports and feedback are very welcome.
 
-Route **third-party** catalog models — OpenAI, Anthropic, Google, xAI/Grok, Groq, and the OpenAI-compatible long tail — through [AI Gateway](https://developers.cloudflare.com/ai-gateway/) using the same `env.AI` binding, with resumable streaming, BYOK, caching, and fallback.
+Route **third-party** catalog models — OpenAI, Anthropic, Google, xAI/Grok, Groq, and the OpenAI-compatible long tail — through [AI Gateway](https://developers.cloudflare.com/ai-gateway/) using the same `env.AI` binding, with resumable streaming _(coming soon)_, BYOK, caching, and fallback.
 
 Install only the wire-format plugins you actually use. They're **optional** peer dependencies:
 
@@ -360,7 +364,7 @@ const workersai = createWorkersAI({
 	// gateway unless you set one, e.g. gateway: { id: "my-gateway" }.
 });
 
-workersai("@cf/meta/llama-3.1-8b-instruct"); // Workers AI (unchanged)
+workersai("@cf/zai-org/glm-5.2"); // Workers AI (unchanged)
 
 const result = streamText({
 	model: workersai("openai/gpt-5", { resume: true }), // routed through AI Gateway
@@ -369,7 +373,7 @@ const result = streamText({
 // result.response.headers["cf-aig-run-id"] is set — resume from there.
 ```
 
-The settings argument is **typed from the model id**: pass a `"<provider>/<model>"` catalog slug and the second argument autocompletes the per-call gateway options (`resume`, `fallback`, `cacheTtl`, `byok`, `metadata`, …, i.e. `DelegateCallOptions`); pass a `@cf/...` id and it autocompletes the usual `WorkersAIChatSettings`. `providers` is optional and **additive**: leave it unset and `createWorkersAI` behaves exactly as before; passing a catalog slug without it throws a helpful error pointing you here.
+The settings argument is **typed from the model id**: pass a `"<provider>/<model>"` catalog slug and the second argument autocompletes the per-call gateway options (`resume`, `fallback`, `cacheTtl`, `byok`, `metadata`, …, i.e. `DelegateCallOptions`); pass a `@cf/...` id and it autocompletes the usual `WorkersAIChatSettings`. `providers` is optional and **additive**: leave it unset and `createWorkersAI` behaves exactly as before — `@cf/...` ids work as always, and a `"<provider>/<model>"` catalog slug still routes through the unified-billing run path (defaulting to your account's `"default"` gateway, with the built-in OpenAI-compatible parser). Configure `providers` to unlock higher-fidelity parsing plus the per-call delegate options (BYOK, caching, fallback, resume); requesting one of those without `providers` throws a helpful error pointing you here.
 
 `gateway` is optional for catalog routing — when unset, requests use your account's `"default"` AI Gateway. Set `gateway: { id: "…" }` (here or per call) to use a specific gateway.
 
@@ -381,7 +385,7 @@ One plugin per **wire format** serves every provider of that format. The `openai
 
 The registry covers every provider in the [AI Gateway provider directory](https://developers.cloudflare.com/ai-gateway/usage/providers/) — OpenAI, Anthropic, Google AI Studio, Google Vertex AI, xAI/Grok, Groq, DeepSeek, Mistral, Perplexity, Cerebras, OpenRouter, Cohere, Baseten, Parallel, Azure OpenAI, Amazon Bedrock, HuggingFace, Replicate, Fal, Ideogram, Cartesia, Deepgram, ElevenLabs (plus Fireworks) — so `createGatewayFetch` can auto-detect them from the request URL.
 
-**Coverage maturity varies** (the whole feature is experimental). Only the unified-billing run-catalog providers — OpenAI, Anthropic, Google, xAI/Grok, Groq, Alibaba/Qwen, MiniMax — are exercised end-to-end against a live gateway. The remaining registry entries (BYOK gateway-path providers like DeepSeek/Mistral/Perplexity/Cerebras/OpenRouter/Fireworks, and bring-your-own-provider-only ones like Cohere/Baseten/Parallel/Azure OpenAI/Bedrock/HuggingFace/Replicate/Fal/Ideogram/Cartesia/Deepgram/ElevenLabs) are registry-level wiring (gateway id, host pattern, endpoint transform) that is **not yet live-verified** — the routing is in place, but a provider's exact request shape may need adjustment. Please file issues for any that misbehave.
+**Coverage maturity varies** (the whole feature is experimental). Only the unified-billing run-catalog providers — OpenAI, Anthropic, Google, xAI/Grok, Groq, DeepSeek, Alibaba/Qwen, MiniMax — are exercised end-to-end against a live gateway. The remaining registry entries (BYOK gateway-path providers like Mistral/Perplexity/Cerebras/OpenRouter/Fireworks, and bring-your-own-provider-only ones like Cohere/Baseten/Parallel/Azure OpenAI/Bedrock/HuggingFace/Replicate/Fal/Ideogram/Cartesia/Deepgram/ElevenLabs) are registry-level wiring (gateway id, host pattern, endpoint transform) that is **not yet live-verified** — the routing is in place, but a provider's exact request shape may need adjustment. Please file issues for any that misbehave.
 
 > **Run-path wire format is per-provider — not always OpenAI.** On the resumable run path (`env.AI.run`), Cloudflare's unified catalog **normalizes most providers to OpenAI chat-completions** (so `google/…` is parsed with the `openai` plugin on the run path, even though the gateway path uses the native `google` plugin), but **passes Anthropic through natively** — so `anthropic/…` uses the `anthropic` plugin on both paths. In practice: include `openai` for the openai-wire run-path providers (openai, google, xai/grok, groq), and `anthropic` to use `anthropic/…`. The native `google` plugin is only needed if you force google onto the **gateway path**. If a plugin a transport needs is missing, the delegate throws a `GatewayDelegateError` naming it.
 
@@ -389,18 +393,18 @@ The registry covers every provider in the [AI Gateway provider directory](https:
 
 The transport is chosen automatically from the options you pass:
 
-| Transport         | Backed by                    | Resume (`cf-aig-run-id`) | Caching | Server fallback | Billing           |
-| ----------------- | ---------------------------- | ------------------------ | ------- | --------------- | ----------------- |
-| **run** (default) | `env.AI.run(...)`            | ✅                       | ❌      | ❌              | Unified billing   |
-| **gateway**       | `env.AI.gateway(id).run([])` | ❌                       | ✅      | ✅              | BYOK / stored key |
+| Transport         | Backed by                    | Resume _(coming soon)_ (`cf-aig-run-id`) | Caching | Server fallback | Billing           |
+| ----------------- | ---------------------------- | ---------------------------------------- | ------- | --------------- | ----------------- |
+| **run** (default) | `env.AI.run(...)`            | ✅                                       | ❌      | ❌              | Unified billing   |
+| **gateway**       | `env.AI.gateway(id).run([])` | ❌                                       | ✅      | ✅              | BYOK / stored key |
 
-Run-catalog providers (OpenAI, Anthropic, Google, xAI, Groq, plus the unified-catalog chat providers Alibaba/Qwen and MiniMax) default to the resumable **run path**. BYOK-only providers (deepseek, mistral, perplexity, …) always use the **gateway path**. Asking for an impossible combination (e.g. `resume: true` with `fallback.mode: "server"`) throws a `GatewayDelegateError`.
+Run-catalog providers (OpenAI, Anthropic, Google, xAI, Groq, DeepSeek, plus the unified-catalog chat providers Alibaba/Qwen and MiniMax) default to the resumable **run path**. BYOK-only providers (mistral, perplexity, cerebras, …) always use the **gateway path**. Asking for an impossible combination (e.g. `resume: true` with `fallback.mode: "server"`) throws a `GatewayDelegateError`.
 
 > Alibaba and MiniMax are **run-path only** — they're on the unified catalog but not the native gateway directory, so there's no gateway path. Requesting `transport: "gateway"`, caching, or server-side fallback on them throws a clear `GatewayDelegateError` at build time (rather than failing upstream); use the default run path or `fallback.mode: "client"`.
 
 ### BYOK (bring your own key)
 
-On the gateway path, set `byok: true` and supply the upstream key via `extraHeaders`:
+Set `byok: true` to forward your own upstream key via `extraHeaders`. BYOK is a gateway-path feature, so it forces the gateway path even for a run-catalog provider like DeepSeek (resume isn't available on that leg):
 
 ```ts
 streamText({
@@ -447,6 +451,11 @@ workersai("openai/gpt-5", {
 
 ### Resume after disconnect
 
+> **🚧 Coming soon.** Resumable streaming is not generally available yet — the AI
+> Gateway resume backend is still rolling out. The options here are in place so
+> you can adopt them early, but treat resume as experimental until the rollout
+> completes.
+
 The run path wraps the response stream so a transient mid-stream drop reconnects through the gateway resume endpoint transparently. For cross-invocation recovery (e.g. a Durable Object re-attaching after eviction), persist `{ runId, eventOffset }` via `onDispatch` + `onProgress` and re-attach with `createResumableStream`:
 
 ```ts
@@ -482,15 +491,15 @@ The provider id is detected from the request URL (or pass `provider` explicitly)
 
 ### `createWorkersAI(options)`
 
-| Option            | Type                            | Description                                                                                       |
-| ----------------- | ------------------------------- | ------------------------------------------------------------------------------------------------- |
-| `binding`         | `Ai`                            | Workers AI binding (`env.AI`). Use this OR credentials.                                           |
-| `accountId`       | `string`                        | Cloudflare account ID. Required with `apiKey`.                                                    |
-| `apiKey`          | `string`                        | Cloudflare API token. Required with `accountId`.                                                  |
-| `gateway`         | `GatewayOptions`                | Optional [AI Gateway](https://developers.cloudflare.com/ai-gateway/) config.                      |
-| `providers`       | `ProviderPlugin[]`              | _Experimental._ Wire-format plugins that enable routing `"<provider>/<model>"` slugs via gateway. |
-| `resume`          | `boolean`                       | _Experimental._ Default resume behavior for gateway-routed catalog models. Defaults to `true`.    |
-| `onResumeExpired` | `"error"` \| `"accept-partial"` | _Experimental._ Default policy when the gateway resume buffer expires. Defaults to `"error"`.     |
+| Option            | Type                            | Description                                                                                                  |
+| ----------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------ |
+| `binding`         | `Ai`                            | Workers AI binding (`env.AI`). Use this OR credentials.                                                      |
+| `accountId`       | `string`                        | Cloudflare account ID. Required with `apiKey`.                                                               |
+| `apiKey`          | `string`                        | Cloudflare API token. Required with `accountId`.                                                             |
+| `gateway`         | `GatewayOptions`                | Optional [AI Gateway](https://developers.cloudflare.com/ai-gateway/) config.                                 |
+| `providers`       | `ProviderPlugin[]`              | _Experimental._ Wire-format plugins that enable routing `"<provider>/<model>"` slugs via gateway.            |
+| `resume`          | `boolean`                       | _Experimental — coming soon._ Default resume behavior for gateway-routed catalog models. Defaults to `true`. |
+| `onResumeExpired` | `"error"` \| `"accept-partial"` | _Experimental — coming soon._ Default policy when the gateway resume buffer expires. Defaults to `"error"`.  |
 
 Returns a provider with model factories. Each factory accepts an optional second argument for per-model settings:
 

package/dist/anthropic.d.mts

@@ -1,4 +1,4 @@
-import { o as ProviderPlugin } from "./gateway-delegate-BfaUTwDZ.mjs";
+import { a as ProviderPlugin } from "./gateway-delegate-D_zkIp5r.mjs";
 
 //#region src/anthropic.d.ts
 /**

package/dist/{gateway-delegate-BfaUTwDZ.d.mts → gateway-delegate-D_zkIp5r.d.mts}

@@ -1,5 +1,35 @@
 import { LanguageModelV3 } from "@ai-sdk/provider";
 
+//#region ../gateway-core/dist/index.d.mts
+//#region src/errors.d.ts
+/**
+ * Error type shared by the gateway delegate and the resumable-stream engine.
+ *
+ * Lives in `@cloudflare/gateway-core` because the resume engine (here) and the
+ * delegate (in `workers-ai-provider`) both throw it. Note: since each consumer
+ * inlines this source into its own bundle, `instanceof GatewayDelegateError`
+ * only matches within a single package's bundle — match on `.name`/`.kind`
+ * across package boundaries.
+ */
+type GatewayDelegateErrorKind = "config" | "dispatch" | "provider" | "resume-expired";
+declare class GatewayDelegateError extends Error {
+  readonly kind: GatewayDelegateErrorKind;
+  readonly cause?: unknown;
+  constructor(kind: GatewayDelegateErrorKind, message: string, cause?: unknown);
+} //#endregion
+//#region src/gateway-fetch.d.ts
+/**
+ * Shared AI Gateway dispatch primitives: the `cf-aig-*` header builder, the
+ * provider-key / hop-by-hop header strip, body decoding, and universal-endpoint
+ * entry construction.
+ *
+ * These are consumed by `workers-ai-provider` (the gateway-path `createGatewayFetch`
+ * and the delegate's `makeGatewayFetch`/`makeRunFetch`) and by `@cloudflare/tanstack-ai`
+ * (its REST/binding `createGatewayFetch`), so there's a single place that knows how
+ * to translate caching/metadata/log options into gateway headers.
+ */
+/** Metadata values the gateway accepts (`bigint` is serialized to a string). */
+//#endregion
 //#region src/gateway-providers.d.ts
 /**
  * Registry of Cloudflare AI Gateway providers.
@@ -19,10 +49,21 @@ import { LanguageModelV3 } from "@ai-sdk/provider";
  *
  * Slugs mirror the AI Gateway provider directory
  * (developers.cloudflare.com/ai-gateway/usage/providers/); endpoint transforms
- * mirror `ai-gateway-provider`'s provider table. `runCatalog` / `billing` flags
- * follow the documented unified-billing list (OpenAI, Anthropic, Google AI
- * Studio, Google Vertex, xAI/Grok, Groq) and are otherwise conservative — the
- * e2e suite confirms them live, since resume is undocumented upstream.
+ * mirror `ai-gateway-provider`'s provider table.
+ *
+ * `runCatalog` marks providers whose models Cloudflare actually serves on the
+ * unified-billing `env.AI.run` path (resumable, `cf-aig-run-id`). Membership is
+ * NOT "any OpenAI-wire provider" — it's empirically what the run router accepts:
+ * the headline unified providers (OpenAI, Anthropic, Google AI Studio, xAI, Groq),
+ * the DashScope/MiniMax run-only providers, and `deepseek/*` (issue #596). Within
+ * a run-catalog provider, unified-billing eligibility is still decided per-MODEL
+ * (e.g. `deepseek/deepseek-v4-pro` is unified while `deepseek/deepseek-chat`
+ * returns a clear "use BYOK" signal). Everything else — the OpenAI-wire long tail
+ * (mistral, perplexity, cerebras, openrouter, fireworks), the provider-native
+ * `wireFormat`-less providers, and Vertex — is `runCatalog:false` and reached via
+ * the BYOK gateway path. `env.AI.run` distinguishes the two cleanly: `7003
+ * model-not-found` for off-catalog slugs vs `2021 use-BYOK` for a recognized
+ * BYOK-only model. All of this is guarded live by the e2e run-path membership probe.
  */
 /** Response wire format the slug delegate can parse with a built-in `@ai-sdk/*` provider. */
 type WireFormat = "openai" | "anthropic" | "google";
@@ -92,8 +133,7 @@ declare function findProviderBySlug(resolverKey: string): GatewayProviderInfo |
 /** Detect the gateway provider from a wrapped provider's request URL (BYOG). */
 declare function detectProviderByUrl(url: string): GatewayProviderInfo | undefined;
 /** All slug keys with a built-in parser (auto-wireable by the slug delegate). */
-declare function wireableProviders(): GatewayProviderInfo[];
-//#endregion
+declare function wireableProviders(): GatewayProviderInfo[]; //#endregion
 //#region src/resumable-stream.d.ts
 type ResumeExpiredPolicy = "error" | "accept-partial";
 interface ResumableStreamOptions {
@@ -127,8 +167,35 @@ interface ResumableStreamOptions {
    * re-attach (throttle your own writes — this can fire per chunk).
    */
   onProgress?: (eventOffset: number) => void;
+  /**
+   * Abort signal for the consuming request. When it aborts (or the downstream
+   * consumer cancels the wrapped stream), the engine stops **without**
+   * reconnecting — an intentional cancel must never trigger a resume reconnect.
+   * The signal is also forwarded to the resume fetch.
+   */
+  signal?: AbortSignal;
 }
-declare function createResumableStream(options: ResumableStreamOptions): ReadableStream<Uint8Array>;
+declare function createResumableStream(options: ResumableStreamOptions): ReadableStream<Uint8Array>; //#endregion
+//#region src/workers-ai.d.ts
+/**
+ * Shared, framework-agnostic Workers AI helpers.
+ *
+ * These are the pieces that `workers-ai-provider` (native `LanguageModelV3`) and
+ * `@cloudflare/tanstack-ai` (OpenAI-SDK shim) both need: the SSE byte decoder,
+ * message normalization for the binding's stricter schema, response-text
+ * extraction across WAI's response shapes, and the gpt-oss forced-tool-call
+ * salvage.
+ *
+ * IMPORTANT — id/dependency decoupling: nothing here depends on the `ai` package
+ * or mints framework tool-call ids. `parseLeakedToolCalls` returns neutral
+ * `{ toolName, input }` records; each consumer assigns its own ids and adapts to
+ * its own tool-call shape. This keeps `gateway-core` free of an `ai` dependency.
+ */
+/**
+ * TransformStream that decodes a raw byte stream into SSE `data:` payloads.
+ * Each output chunk is the string content after `data: ` (one per SSE event),
+ * with line buffering for partial chunks.
+ */
 //#endregion
 //#region src/errors.d.ts
 /**
@@ -340,6 +407,20 @@ interface DelegateCallOptions {
    * delegate strips provider auth headers so unified billing applies.
    */
   byok?: boolean;
+  /**
+   * Gateway path only: BYOK stored-key alias to authenticate with
+   * (`cf-aig-byok-alias`). Selects a non-`default` key you configured for the
+   * provider on the gateway. Independent of `byok` (which controls whether the
+   * caller's own provider auth header is forwarded vs. stripped).
+   */
+  byokAlias?: string;
+  /**
+   * Per-request Zero Data Retention override (`cf-aig-zdr`), Unified Billing
+   * only: `true` forces ZDR-capable upstreams, `false` disables it for this
+   * request. Overrides the gateway-level ZDR default. Applied on both transports
+   * (run path: passed through gateway options as a header; gateway path: entry header).
+   */
+  zdr?: boolean;
   /** Override the delegate's gateway for this model. */
   gateway?: GatewayOptions | string;
   /**
@@ -374,12 +455,6 @@ interface Selection {
  * requested.
  */
 declare function selectTransport(opts: DelegateCallOptions, resumeExplicitlyTrue: boolean, runCatalog?: boolean, gatewayAvailable?: boolean): Selection;
-type GatewayDelegateErrorKind = "config" | "dispatch" | "provider" | "resume-expired";
-declare class GatewayDelegateError extends Error {
-  readonly kind: GatewayDelegateErrorKind;
-  readonly cause?: unknown;
-  constructor(kind: GatewayDelegateErrorKind, message: string, cause?: unknown);
-}
 //#endregion
-export { WireFormat as C, wireableProviders as E, GatewayProviderInfo as S, findProviderBySlug as T, ResumableStreamOptions as _, ParsedSlug as a, Billing as b, parseSlug as c, createClientFallbackModel as d, FallbackAttempt as f, WorkersAIGatewayError as g, WorkersAIFallbackError as h, GatewayDelegateError as i, selectTransport as l, GatewayErrorContext as m, DispatchInfo as n, ProviderPlugin as o, GatewayErrorCode as p, FallbackOptions as r, Transport as s, DelegateCallOptions as t, FallbackLeg as u, ResumeExpiredPolicy as v, detectProviderByUrl as w, GATEWAY_PROVIDERS as x, createResumableStream as y };
-//# sourceMappingURL=gateway-delegate-BfaUTwDZ.d.mts.map
+export { createResumableStream as C, wireableProviders as E, WireFormat as S, findProviderBySlug as T, GATEWAY_PROVIDERS as _, ProviderPlugin as a, ResumableStreamOptions as b, selectTransport as c, FallbackAttempt as d, GatewayErrorCode as f, Billing as g, WorkersAIGatewayError as h, ParsedSlug as i, FallbackLeg as l, WorkersAIFallbackError as m, DispatchInfo as n, Transport as o, GatewayErrorContext as p, FallbackOptions as r, parseSlug as s, DelegateCallOptions as t, createClientFallbackModel as u, GatewayDelegateError as v, detectProviderByUrl as w, ResumeExpiredPolicy as x, GatewayProviderInfo as y };
+//# sourceMappingURL=gateway-delegate-D_zkIp5r.d.mts.map