workers-ai-provider 3.2.1 → 3.3.0

package/README.md

@@ -2,6 +2,10 @@
 
 [Workers AI](https://developers.cloudflare.com/workers-ai/) provider for the [AI SDK](https://sdk.vercel.ai/). Run Cloudflare's models for chat, embeddings, image generation, transcription, text-to-speech, reranking, and [AI Search](https://developers.cloudflare.com/ai-search/) — all from a single provider. It can also route **third-party** models (OpenAI, Anthropic, Google, …) through [AI Gateway](https://developers.cloudflare.com/ai-gateway/) — see [Third-party models](#third-party-models-via-ai-gateway).
 
+> 📚 In-depth guides and the AI Gateway **delegate** reference (unified catalog,
+> resumable streaming _(coming soon)_, server-side fallback) live in
+> [`docs/workers-ai-provider`](../../docs/workers-ai-provider/README.md).
+
 ## Quick start
 
 ```bash
@@ -333,7 +337,7 @@ Streaming works the same way — use `streamText` instead of `generateText`.
 
 > **⚠️ Experimental.** Everything in this section (routing third-party models via `createWorkersAI({ providers })`, the provider plugins, the registry, the resume layer, and `createGatewayFetch`/`createGatewayProvider`) is a new and substantial addition — well beyond the package's original job of wrapping Workers AI. Treat the whole surface as experimental: APIs may change, and several behaviors depend on undocumented AI Gateway internals (the `cf-aig-run-id` resume buffer, per-provider run-path wire formats). It does **not** affect the stable Workers AI / AI Search APIs above. Bug reports and feedback are very welcome.
 
-Route **third-party** catalog models — OpenAI, Anthropic, Google, xAI/Grok, Groq, and the OpenAI-compatible long tail — through [AI Gateway](https://developers.cloudflare.com/ai-gateway/) using the same `env.AI` binding, with resumable streaming, BYOK, caching, and fallback.
+Route **third-party** catalog models — OpenAI, Anthropic, Google, xAI/Grok, Groq, and the OpenAI-compatible long tail — through [AI Gateway](https://developers.cloudflare.com/ai-gateway/) using the same `env.AI` binding, with resumable streaming _(coming soon)_, BYOK, caching, and fallback.
 
 Install only the wire-format plugins you actually use. They're **optional** peer dependencies:
 
@@ -360,7 +364,7 @@ const workersai = createWorkersAI({
 	// gateway unless you set one, e.g. gateway: { id: "my-gateway" }.
 });
 
-workersai("@cf/meta/llama-3.1-8b-instruct"); // Workers AI (unchanged)
+workersai("@cf/zai-org/glm-5.2"); // Workers AI (unchanged)
 
 const result = streamText({
 	model: workersai("openai/gpt-5", { resume: true }), // routed through AI Gateway
@@ -389,10 +393,10 @@ The registry covers every provider in the [AI Gateway provider directory](https:
 
 The transport is chosen automatically from the options you pass:
 
-| Transport         | Backed by                    | Resume (`cf-aig-run-id`) | Caching | Server fallback | Billing           |
-| ----------------- | ---------------------------- | ------------------------ | ------- | --------------- | ----------------- |
-| **run** (default) | `env.AI.run(...)`            | ✅                       | ❌      | ❌              | Unified billing   |
-| **gateway**       | `env.AI.gateway(id).run([])` | ❌                       | ✅      | ✅              | BYOK / stored key |
+| Transport         | Backed by                    | Resume _(coming soon)_ (`cf-aig-run-id`) | Caching | Server fallback | Billing           |
+| ----------------- | ---------------------------- | ---------------------------------------- | ------- | --------------- | ----------------- |
+| **run** (default) | `env.AI.run(...)`            | ✅                                       | ❌      | ❌              | Unified billing   |
+| **gateway**       | `env.AI.gateway(id).run([])` | ❌                                       | ✅      | ✅              | BYOK / stored key |
 
 Run-catalog providers (OpenAI, Anthropic, Google, xAI, Groq, plus the unified-catalog chat providers Alibaba/Qwen and MiniMax) default to the resumable **run path**. BYOK-only providers (deepseek, mistral, perplexity, …) always use the **gateway path**. Asking for an impossible combination (e.g. `resume: true` with `fallback.mode: "server"`) throws a `GatewayDelegateError`.
 
@@ -447,6 +451,11 @@ workersai("openai/gpt-5", {
 
 ### Resume after disconnect
 
+> **🚧 Coming soon.** Resumable streaming is not generally available yet — the AI
+> Gateway resume backend is still rolling out. The options here are in place so
+> you can adopt them early, but treat resume as experimental until the rollout
+> completes.
+
 The run path wraps the response stream so a transient mid-stream drop reconnects through the gateway resume endpoint transparently. For cross-invocation recovery (e.g. a Durable Object re-attaching after eviction), persist `{ runId, eventOffset }` via `onDispatch` + `onProgress` and re-attach with `createResumableStream`:
 
 ```ts
@@ -482,15 +491,15 @@ The provider id is detected from the request URL (or pass `provider` explicitly)
 
 ### `createWorkersAI(options)`
 
-| Option            | Type                            | Description                                                                                       |
-| ----------------- | ------------------------------- | ------------------------------------------------------------------------------------------------- |
-| `binding`         | `Ai`                            | Workers AI binding (`env.AI`). Use this OR credentials.                                           |
-| `accountId`       | `string`                        | Cloudflare account ID. Required with `apiKey`.                                                    |
-| `apiKey`          | `string`                        | Cloudflare API token. Required with `accountId`.                                                  |
-| `gateway`         | `GatewayOptions`                | Optional [AI Gateway](https://developers.cloudflare.com/ai-gateway/) config.                      |
-| `providers`       | `ProviderPlugin[]`              | _Experimental._ Wire-format plugins that enable routing `"<provider>/<model>"` slugs via gateway. |
-| `resume`          | `boolean`                       | _Experimental._ Default resume behavior for gateway-routed catalog models. Defaults to `true`.    |
-| `onResumeExpired` | `"error"` \| `"accept-partial"` | _Experimental._ Default policy when the gateway resume buffer expires. Defaults to `"error"`.     |
+| Option            | Type                            | Description                                                                                                  |
+| ----------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------ |
+| `binding`         | `Ai`                            | Workers AI binding (`env.AI`). Use this OR credentials.                                                      |
+| `accountId`       | `string`                        | Cloudflare account ID. Required with `apiKey`.                                                               |
+| `apiKey`          | `string`                        | Cloudflare API token. Required with `accountId`.                                                             |
+| `gateway`         | `GatewayOptions`                | Optional [AI Gateway](https://developers.cloudflare.com/ai-gateway/) config.                                 |
+| `providers`       | `ProviderPlugin[]`              | _Experimental._ Wire-format plugins that enable routing `"<provider>/<model>"` slugs via gateway.            |
+| `resume`          | `boolean`                       | _Experimental — coming soon._ Default resume behavior for gateway-routed catalog models. Defaults to `true`. |
+| `onResumeExpired` | `"error"` \| `"accept-partial"` | _Experimental — coming soon._ Default policy when the gateway resume buffer expires. Defaults to `"error"`.  |
 
 Returns a provider with model factories. Each factory accepts an optional second argument for per-model settings:
 

package/dist/anthropic.d.mts

@@ -1,4 +1,4 @@
-import { o as ProviderPlugin } from "./gateway-delegate-BfaUTwDZ.mjs";
+import { a as ProviderPlugin } from "./gateway-delegate-BAALxuBO.mjs";
 
 //#region src/anthropic.d.ts
 /**

package/dist/{gateway-delegate-BfaUTwDZ.d.mts → gateway-delegate-BAALxuBO.d.mts}

@@ -1,5 +1,35 @@
 import { LanguageModelV3 } from "@ai-sdk/provider";
 
+//#region ../gateway-core/dist/index.d.mts
+//#region src/errors.d.ts
+/**
+ * Error type shared by the gateway delegate and the resumable-stream engine.
+ *
+ * Lives in `@cloudflare/gateway-core` because the resume engine (here) and the
+ * delegate (in `workers-ai-provider`) both throw it. Note: since each consumer
+ * inlines this source into its own bundle, `instanceof GatewayDelegateError`
+ * only matches within a single package's bundle — match on `.name`/`.kind`
+ * across package boundaries.
+ */
+type GatewayDelegateErrorKind = "config" | "dispatch" | "provider" | "resume-expired";
+declare class GatewayDelegateError extends Error {
+  readonly kind: GatewayDelegateErrorKind;
+  readonly cause?: unknown;
+  constructor(kind: GatewayDelegateErrorKind, message: string, cause?: unknown);
+} //#endregion
+//#region src/gateway-fetch.d.ts
+/**
+ * Shared AI Gateway dispatch primitives: the `cf-aig-*` header builder, the
+ * provider-key / hop-by-hop header strip, body decoding, and universal-endpoint
+ * entry construction.
+ *
+ * These are consumed by `workers-ai-provider` (the gateway-path `createGatewayFetch`
+ * and the delegate's `makeGatewayFetch`/`makeRunFetch`) and by `@cloudflare/tanstack-ai`
+ * (its REST/binding `createGatewayFetch`), so there's a single place that knows how
+ * to translate caching/metadata/log options into gateway headers.
+ */
+/** Metadata values the gateway accepts (`bigint` is serialized to a string). */
+//#endregion
 //#region src/gateway-providers.d.ts
 /**
  * Registry of Cloudflare AI Gateway providers.
@@ -92,8 +122,7 @@ declare function findProviderBySlug(resolverKey: string): GatewayProviderInfo |
 /** Detect the gateway provider from a wrapped provider's request URL (BYOG). */
 declare function detectProviderByUrl(url: string): GatewayProviderInfo | undefined;
 /** All slug keys with a built-in parser (auto-wireable by the slug delegate). */
-declare function wireableProviders(): GatewayProviderInfo[];
-//#endregion
+declare function wireableProviders(): GatewayProviderInfo[]; //#endregion
 //#region src/resumable-stream.d.ts
 type ResumeExpiredPolicy = "error" | "accept-partial";
 interface ResumableStreamOptions {
@@ -127,8 +156,35 @@ interface ResumableStreamOptions {
    * re-attach (throttle your own writes — this can fire per chunk).
    */
   onProgress?: (eventOffset: number) => void;
+  /**
+   * Abort signal for the consuming request. When it aborts (or the downstream
+   * consumer cancels the wrapped stream), the engine stops **without**
+   * reconnecting — an intentional cancel must never trigger a resume reconnect.
+   * The signal is also forwarded to the resume fetch.
+   */
+  signal?: AbortSignal;
 }
-declare function createResumableStream(options: ResumableStreamOptions): ReadableStream<Uint8Array>;
+declare function createResumableStream(options: ResumableStreamOptions): ReadableStream<Uint8Array>; //#endregion
+//#region src/workers-ai.d.ts
+/**
+ * Shared, framework-agnostic Workers AI helpers.
+ *
+ * These are the pieces that `workers-ai-provider` (native `LanguageModelV3`) and
+ * `@cloudflare/tanstack-ai` (OpenAI-SDK shim) both need: the SSE byte decoder,
+ * message normalization for the binding's stricter schema, response-text
+ * extraction across WAI's response shapes, and the gpt-oss forced-tool-call
+ * salvage.
+ *
+ * IMPORTANT — id/dependency decoupling: nothing here depends on the `ai` package
+ * or mints framework tool-call ids. `parseLeakedToolCalls` returns neutral
+ * `{ toolName, input }` records; each consumer assigns its own ids and adapts to
+ * its own tool-call shape. This keeps `gateway-core` free of an `ai` dependency.
+ */
+/**
+ * TransformStream that decodes a raw byte stream into SSE `data:` payloads.
+ * Each output chunk is the string content after `data: ` (one per SSE event),
+ * with line buffering for partial chunks.
+ */
 //#endregion
 //#region src/errors.d.ts
 /**
@@ -340,6 +396,20 @@ interface DelegateCallOptions {
    * delegate strips provider auth headers so unified billing applies.
    */
   byok?: boolean;
+  /**
+   * Gateway path only: BYOK stored-key alias to authenticate with
+   * (`cf-aig-byok-alias`). Selects a non-`default` key you configured for the
+   * provider on the gateway. Independent of `byok` (which controls whether the
+   * caller's own provider auth header is forwarded vs. stripped).
+   */
+  byokAlias?: string;
+  /**
+   * Per-request Zero Data Retention override (`cf-aig-zdr`), Unified Billing
+   * only: `true` forces ZDR-capable upstreams, `false` disables it for this
+   * request. Overrides the gateway-level ZDR default. Applied on both transports
+   * (run path: passed through gateway options as a header; gateway path: entry header).
+   */
+  zdr?: boolean;
   /** Override the delegate's gateway for this model. */
   gateway?: GatewayOptions | string;
   /**
@@ -374,12 +444,6 @@ interface Selection {
  * requested.
  */
 declare function selectTransport(opts: DelegateCallOptions, resumeExplicitlyTrue: boolean, runCatalog?: boolean, gatewayAvailable?: boolean): Selection;
-type GatewayDelegateErrorKind = "config" | "dispatch" | "provider" | "resume-expired";
-declare class GatewayDelegateError extends Error {
-  readonly kind: GatewayDelegateErrorKind;
-  readonly cause?: unknown;
-  constructor(kind: GatewayDelegateErrorKind, message: string, cause?: unknown);
-}
 //#endregion
-export { WireFormat as C, wireableProviders as E, GatewayProviderInfo as S, findProviderBySlug as T, ResumableStreamOptions as _, ParsedSlug as a, Billing as b, parseSlug as c, createClientFallbackModel as d, FallbackAttempt as f, WorkersAIGatewayError as g, WorkersAIFallbackError as h, GatewayDelegateError as i, selectTransport as l, GatewayErrorContext as m, DispatchInfo as n, ProviderPlugin as o, GatewayErrorCode as p, FallbackOptions as r, Transport as s, DelegateCallOptions as t, FallbackLeg as u, ResumeExpiredPolicy as v, detectProviderByUrl as w, GATEWAY_PROVIDERS as x, createResumableStream as y };
-//# sourceMappingURL=gateway-delegate-BfaUTwDZ.d.mts.map
+export { createResumableStream as C, wireableProviders as E, WireFormat as S, findProviderBySlug as T, GATEWAY_PROVIDERS as _, ProviderPlugin as a, ResumableStreamOptions as b, selectTransport as c, FallbackAttempt as d, GatewayErrorCode as f, Billing as g, WorkersAIGatewayError as h, ParsedSlug as i, FallbackLeg as l, WorkersAIFallbackError as m, DispatchInfo as n, Transport as o, GatewayErrorContext as p, FallbackOptions as r, parseSlug as s, DelegateCallOptions as t, createClientFallbackModel as u, GatewayDelegateError as v, detectProviderByUrl as w, ResumeExpiredPolicy as x, GatewayProviderInfo as y };
+//# sourceMappingURL=gateway-delegate-BAALxuBO.d.mts.map