agents 0.23.0 → 0.25.0

package/docs/lifecycle.md

@@ -138,11 +138,16 @@ sequentially. Request handling is middleware dispatch: it stops at the first
 returned `Response`, and returning `undefined` passes the request on. A phase
 failure propagates, and failed startup can be retried.
 
-A capability installed with `{ fallback: true }` dispatches after every
-non-fallback capability, whenever it was installed. This is for a host's
-catch-all: `Agent` installs its WebSockets capability as a fallback, so a
-subclass that installs request or upgrade middleware from its own
-constructor still runs first, even though `Agent`'s constructor ran earlier.
+A capability declares how it claims traffic with `claims`, which defaults to
+`"selective"`. A `"catch-all"` capability dispatches after every other
+capability, whenever it was installed. Catch-alls are unique per dispatch
+hook: Lifecycle refuses a second catch-all for `onRequest` or for
+`onWebSocketUpgrade`, since it could never be reached, but one of each
+coexists. The `WebSockets` capability is a catch-all for upgrades only, so
+an HTTP catch-all installs beside it, and selective HTTP capabilities that
+target their own routes run before both. A subclass that installs request or
+upgrade middleware from its own constructor still runs first, even though
+`Agent`'s constructor ran earlier.
 
 Capabilities extending `LifecycleCapability` receive one standard service
 surface: storage, readiness, startup state, the job queue, a host
@@ -153,8 +158,8 @@ host implicitly.
 
 Capability hooks run outside host context, but user callbacks run through
 `this.lifecycle.runInHostContext(fn)` inside the host invocation context.
-Scheduler dispatches its registered callbacks through this boundary, and a
-future capability that calls user code should do the same.
+Scheduler and Queue dispatch their registered callbacks through this
+boundary, and a future capability that calls user code should do the same.
 
 ## The job queue
 
@@ -306,8 +311,8 @@ to export `getCurrentAgent()` for the `Agent` class as a compatibility alias.
 
 Lifecycle itself does not model WebSockets. Hosts that want connections
 install the `WebSockets` capability, which owns the subsystem end to end —
-it claims upgrades, accepts hibernating sockets, dispatches handlers inside
-the host invocation boundary, and answers `getConnections()`:
+it claims upgrades, dispatches handlers inside the host invocation boundary,
+and answers `getConnections()`:
 
 ```ts
 import { WebSockets } from "agents/websockets";
@@ -321,7 +326,8 @@ export class MyObject extends DurableObject<Env> {
       onMessage: (connection, message) => {
         connection.send(`echo:${message}`);
       }
-    }
+    },
+    callables: new MyCallables(this)
   });
   readonly lifecycle = Lifecycle.install(this).use(this.webSockets);
 }
@@ -329,18 +335,91 @@ export class MyObject extends DurableObject<Env> {
 
 Without the capability installed, WebSocket upgrades are declined.
 
-The capability can also serve remote methods: pass an `RpcTarget` as
-`callables` and its prototype methods become the complete remote interface,
-served over a Cap'n Web session (`?__agents_rpc=capnweb`). An `Agent` adds
-no new surface for this — its `@callable()`-decorated methods are its
-interface, served on every wire: natively over the legacy JSON RPC protocol
-and, through the decorator-derived target, over the Cap'n Web endpoint.
-
-Connections use Cloudflare's WebSocket Hibernation API. Idle clients remain
-connected while the Durable Object can leave memory; when a message wakes the
-object, its constructor and lifecycle startup run again before `onMessage`.
-State needed after a wake must be stored durably or through
-`connection.setState()`. There is no non-hibernating mode.
+### A plain host works with `useAgent`
+
+The capability speaks the Agent protocol on every connection, so a plain
+Durable Object is reachable from `useAgent` and `AgentClient` exactly like an
+`Agent`:
+
+- On connect it sends the identity frame, then the current state when
+  `state` is set. The identity frame says when a state frame follows it,
+  so the client's `ready` and `identified` wait for that state, and a
+  caller awaiting `ready` never reads the default value.
+  `protocol` controls this: `true` (default) for every connection, a
+  function to decide per connection — `false` marks it no-protocol, so it
+  gets no protocol text frames on connect or by broadcast but still sends
+  and receives ordinary messages and callables — or `false` to have the
+  host drive the sequence itself with `sendConnectFrames()`, or
+  `sendIdentity()` and `sendState()` separately (the client then resolves
+  `ready` on the identity alone).
+  `Agent` passes `false`, because it must decide whether a connection
+  belongs to a facet before any frame is sent.
+- `readonly` decides per connection whether state writes over the wire are
+  refused; `setReadonly()` flips it later. Both flags are stored in the
+  connection's own state under `_cf_` keys, hidden from `connection.state`
+  and preserved across `setState`, so they survive hibernation. `Agent`'s
+  `isConnectionReadonly`, `setConnectionReadonly`, and
+  `isConnectionProtocolEnabled` delegate to the same storage.
+- `callables` is an `RpcTarget` whose prototype methods are the host's
+  complete remote interface, reached through `call()` and `stub`. On the
+  `cf-websocket` wire the capability answers them as JSON `rpc` frames. On
+  the `capnweb` wire they are native Cap'n Web methods on the session root:
+  a returned `RpcTarget` arrives as a live stub the client can keep
+  calling, a `ReadableStream` streams, and chained calls pipeline. Methods
+  run through the host invocation boundary with the calling connection in
+  scope.
+- `state` takes a `State` capability (from `agents/state`) and syncs it over
+  connections, so `useAgent().state` and `setState()` work against a plain
+  host:
+
+  ```ts
+  readonly state = new State({
+    initialState: { count: 0 },
+    // The state owner decides who hears about a change.
+    onChanged: (_state, source) => this.webSockets.broadcastState(source)
+  });
+  readonly webSockets = new WebSockets({ state: this.state });
+  readonly lifecycle = Lifecycle.install(this)
+    .use(this.state)
+    .use(this.webSockets);
+  ```
+
+  The current value is pushed to each new connection after identity. A
+  client's `cf_agent_state` frame goes through the `State` capability, so
+  the host's own `validateStateChange` decides; a readonly connection is
+  refused, and a rejected change is logged server-side and answered with a
+  generic `cf_agent_state_error`. `broadcastState(source)` pushes the
+  current value to every protocol-enabled connection except `source`. This
+  is distinct from `connection.setState()`, which is per-connection and
+  never leaves the host. Without the option, state is never sent or
+  accepted over connections.
+
+- Any other frame goes to `handlers.onMessage`.
+
+An `Agent` adds no new surface for this: its `@callable()`-decorated methods
+are its JSON-wire interface, answered by its own message handler. They are
+not mirrored onto the Cap'n Web root; an Agent that wants native calls on
+`capnweb` passes a `callables` target like any other host.
+
+### Two wires
+
+The client chooses how frames travel:
+
+- **`cf-websocket`** (default) — accepted with Cloudflare's WebSocket Hibernation
+  API. Idle clients remain connected while the Durable Object leaves memory;
+  when a message wakes it, the constructor and lifecycle startup run again
+  before `onMessage`. State needed after a wake belongs in storage or
+  `connection.setState()`.
+- **Cap'n Web** (`?__agents_transport=capnweb`, or
+  `useAgent({ transport: "capnweb" })`) — protocol frames travel through one
+  pipe method on a Cap'n Web session whose root also carries the host's
+  `callables` natively. The connection is an in-memory socket and does not
+  hibernate: the object stays pinned while it is open, and clients reconnect
+  after an eviction.
+
+Handlers are wire-agnostic. Both kinds of connection dispatch the same
+`onConnect`/`onMessage`/`onClose`/`onError`, appear in `getConnections()`,
+and honour `connection.close(code, reason)`.
 
 ## Native RPC
 
@@ -354,7 +433,19 @@ async runTask(): Promise<void> {
 }
 ```
 
-Agent's internal RPC entry points already enforce this boundary.
+An `Agent` enforces this boundary for you. Its internal RPC entry points and
+the public `async` methods your subclass exposes over RPC start the lifecycle
+before they run, so a cold instance does not serve those calls against
+uninitialized state. Synchronous methods are never deferred; if one reads state
+that `onStart` sets up, make it `async` or call `await this.lifecycle.start()`
+in it. Concurrent calls to `start()` share the same startup operation, and
+`lifecycle.isStarted()` returns `true` only after capabilities and `onStart()`
+have finished.
+
+Startup resolves the Durable Object name. An async RPC to an Agent addressed
+with `newUniqueId()` or `idFromString()` fails instead of running before
+initialization. Address the Agent by name with `getAgentByName()`, `getByName()`,
+or `idFromName()`.
 
 ## Object names
 

package/docs/models-pi-ai.md

@@ -0,0 +1,325 @@
+# Models for pi-ai (Experimental)
+
+`agents/models/pi-ai` is the [pi-ai](https://github.com/earendil-works/pi/tree/main/packages/ai)
+twin of [`agents/models/ai-sdk`](./models.md): the same `createAI` factory, the same options,
+the same gateway, returning pi-ai `Model` objects instead of AI SDK ones. Use it with pi-ai's
+`stream`/`complete`, with [pi-durable](https://github.com/earendil-works/pi/tree/main/packages/durable)'s
+`Harness`, or with any framework built on a pi-ai `Models` registry.
+
+## The rule
+
+**Workers AI is the catalog Cloudflare keeps up with. Every other vendor is pi-ai's.**
+
+`agents` ships no vendor ids, wire formats, thinking rules or effort tables for any model
+that is not `@cf/…`. A third-party model comes from pi-ai's own registry — the package that
+tracks that vendor — and `agents` only routes it through AI Gateway. So there are two call
+forms:
+
+```ts
+import { createAI } from "agents/models/pi-ai";
+import { getBuiltinModel } from "@earendil-works/pi-ai/providers/all";
+
+const ai = createAI({ binding: env.AI, id: "prod", cacheTtl: 60 });
+
+ai("@cf/zai-org/glm-4.7-flash"); // Workers AI: our catalog, our compat layer
+ai(getBuiltinModel("anthropic", "claude-opus-4-8")); // their model, our transport
+```
+
+For convenience, pi-ai also ships a generated catalog of the models Cloudflare's gateway
+serves, so a `<provider>/<id>` string resolves without importing anything:
+
+```ts
+ai("anthropic/claude-opus-4.8"); // the same model, as AI Gateway's catalog names it
+ai("openai/gpt-5.4");
+```
+
+Any other string is a `TypeError` that names the way in. There is no prefix guessing here by
+design: a model this package cannot name exactly is yours to construct.
+
+1. **Workers AI ids start with `@cf/`.** Everything else is a vendor's, and comes from a
+   vendor registry pi-ai maintains.
+2. **The gateway is an option, not a product.** Caching, logging, metadata, timeouts and
+   retries are options on `createAI`, on the model, or on the call; the `default` gateway is
+   created on first use.
+3. **No provider keys.** AI Gateway holds the credential — unified billing, or a key you
+   store on the gateway (BYOK) — so nothing vendor-shaped needs a secret in your Worker.
+4. **The binding is the only way in.** Workers AI goes through `env.AI.run`, vendor models
+   through `env.AI.gateway(id).run`; there is no HTTP transport and no API token to hold.
+5. Everything here is experimental and may change in a minor release.
+
+## Quick start
+
+```jsonc
+// wrangler.jsonc
+{ "ai": { "binding": "AI" } }
+```
+
+```sh
+npm install agents @earendil-works/pi-ai
+```
+
+```ts
+import { createAI } from "agents/models/pi-ai";
+
+export default {
+  async fetch(_request: Request, env: { AI: Ai }) {
+    const ai = createAI({ binding: env.AI });
+    const reply = await ai.complete(ai("@cf/zai-org/glm-4.7-flash"), {
+      messages: [{ role: "user", content: "Hello", timestamp: Date.now() }]
+    });
+    // `content` is an array of parts — text, thinking, tool calls.
+    const text = reply.content
+      .filter((part) => part.type === "text")
+      .map((part) => part.text)
+      .join("");
+    return Response.json({ text });
+  }
+};
+```
+
+## Where a model comes from
+
+```ts
+// 1. Workers AI — ours.
+ai("@cf/zai-org/glm-4.7-flash");
+
+// 2. pi-ai's Cloudflare AI Gateway catalog — a string, resolved by pi, not us.
+ai("anthropic/claude-opus-4.8");
+
+// 3. Any pi-ai model object, from any registry you imported.
+import { anthropicProvider } from "@earendil-works/pi-ai/providers/anthropic";
+const claude = anthropicProvider()
+  .getModels()
+  .find((m) => m.id === "claude-opus-4-8")!;
+ai(claude);
+```
+
+Forms 2 and 3 keep the model's own metadata verbatim — `api`, `compat`, `thinkingLevelMap`,
+cost, context window, max tokens. That metadata is why Claude Opus gets
+`thinking: { type: "adaptive" }` and GPT-5 gets `reasoning.effort`: the model's author said
+so, and this package does not second-guess it. Form 3 never mutates the object you
+passed; the options ride on a copy.
+
+## Running a model
+
+The instance carries pi-ai's four entry points, so nothing global is registered:
+
+```ts
+const model = ai("anthropic/claude-opus-4.8");
+const context = {
+  messages: [{ role: "user", content: "Hi", timestamp: Date.now() }]
+};
+
+const message = await ai.complete(model, context); // AssistantMessage
+for await (const event of ai.stream(model, context)) {
+  // AssistantMessageEvent: text_delta, thinking_delta, toolcall_*, done, error
+}
+await ai.completeSimple(model, context, { reasoning: "low" }); // SimpleStreamOptions
+ai.streamSimple(model, context, { reasoning: "high", maxTokens: 2048 });
+```
+
+### With pi-durable
+
+pi-durable's `Harness` takes a `Models` registry. Register the provider on one and name the
+model by id:
+
+```ts
+import { Harness } from "@earendil-works/pi-durable";
+import { createModels } from "@earendil-works/pi-ai/models";
+
+const models = createModels();
+models.setProvider(ai.provider);
+const harness = await Harness.open(storage, { models, registry }, context);
+// a session's model: { provider: "cloudflare", modelId: "anthropic/claude-opus-4.8" }
+```
+
+A session stores its model as `{ provider, modelId }`, not as a model object, so use the
+`@cf/…` and `<provider>/<id>` forms there. Options given to `ai(model, options)` do not reach a
+session; set them on `createAI` instead.
+
+`ai.streamFn` is `streamSimple`, bound, for frameworks that take a stream function, such as
+pi-agent-core's `Agent({ streamFn })`.
+
+### With a pi-ai `Models` registry
+
+```ts
+import { createModels } from "@earendil-works/pi-ai/models";
+
+const models = createModels();
+models.setProvider(ai.provider); // provider id: "cloudflare"
+models.getModel("cloudflare", "anthropic/claude-opus-4.8");
+models.streamSimple(ai("@cf/zai-org/glm-4.7-flash"), context);
+```
+
+The provider's static list is every Workers AI id pi-ai knows plus every `<provider>/<id>` in
+its Cloudflare AI Gateway catalog, so `cloudflare/<id>` specifiers resolve. A model object
+you pass to `ai(model)` streams whether or not it is on that list.
+
+## Options
+
+Gateway options may be given flat or nested, on any of the three layers:
+
+```ts
+const ai = createAI({ binding: env.AI, id: "prod", cacheTtl: 60 });
+
+ai("anthropic/claude-opus-4.8", {
+  gateway: { id: "my-gateway" }, // or a string id
+  cacheTtl: 3600, // the same keys, flat
+  skipCache: false,
+  cacheKey: "greeting-v1",
+  collectLog: true,
+  metadata: { team: "growth" },
+  eventId: "turn-17",
+  requestTimeoutMs: 30_000,
+  retries: { maxAttempts: 3, backoff: "exponential", retryDelayMs: 500 },
+  fallback: ["@cf/zai-org/glm-4.7-flash"], // ids or models, in any mix
+  headers: { "x-request-source": "docs" },
+  sessionAffinity: "chat-42", // Workers AI prefix-cache routing
+  reasoningEffort: "low", // "low" | "medium" | "high" | null
+  chatTemplateKwargs: { enable_thinking: false } // Workers AI only
+});
+```
+
+`reasoningEffort` and `chatTemplateKwargs` are Workers AI settings. On a model object or a
+`<slug>/<id>` model they are not sent; the drop is recorded on the message as a
+`cloudflare-compat` diagnostic, and the pi `reasoning` level (mapped by the model's own
+metadata) is the way to ask a vendor model to think. A string in `fallback` follows the same
+rule as `ai(string)`: a `@cf/` id or a registry-known `<slug>/<id>`, never a raw vendor id.
+
+The `@cf/` form also takes pi-ai metadata overrides — `name`, `contextWindow`, `maxTokens`,
+`reasoning`, `input`, `cost`, `streamIdleTimeoutMs` — because the Workers AI catalog moves
+faster than any generated registry. A model object brings its own; nothing here overrides it.
+
+A pi-ai model is plain data, and these per-model options ride on it under symbol keys, which
+`JSON.stringify` and `structuredClone` drop. Routing survives that — a model stored in `Agent`
+state and read back still reaches its vendor under the vendor's own id — but the options do
+not, so re-apply them with `ai(model, options)` after loading if the model carried any.
+
+Per call, pi-ai's own options map onto the same layers: `headers` are merged last, `metadata`
+(scalars) joins the gateway metadata, `sessionId` sets session affinity, and `reasoning` (a pi
+thinking level) is remapped through the model's own `thinkingLevelMap` — as far as the wire
+can carry it; see [thinking shapes](#thinking-shapes) for the one limit. Precedence is call,
+then model, then `createAI` settings.
+
+## How a request travels
+
+| Model                    | `api`                | Transport                                 |
+| ------------------------ | -------------------- | ----------------------------------------- |
+| `@cf/…`                  | `cloudflare-ai`      | `env.AI.run` + Cloudflare's compat layer  |
+| Anthropic (any registry) | `anthropic-messages` | gateway universal → `v1/messages`         |
+| OpenAI (any registry)    | `openai-responses`   | gateway universal → `v1/responses`        |
+| Groq, DeepSeek, xAI, …   | `openai-completions` | gateway universal → `v1/chat/completions` |
+
+Everything that is not `@cf/` goes through **AI Gateway's universal request**: the vendor's
+own body, built by pi-ai's own converter, posted to the gateway with the provider slug taken
+from the model's `baseUrl` host. The gateway forwards it to the vendor with the credential it
+holds, and streams the vendor's own SSE back untouched — pi-ai's stream processors parse it,
+as they would if you had called the vendor directly, the thinking knobs aside (below). No
+credential header leaves your Worker, and the model's `api`, `provider` and `id` are never
+rewritten, so thinking signatures and encrypted reasoning items replay on the next turn.
+
+The `api` markers in that table are the only ones this package routes, and the table is the
+whole list: `cloudflare-ai`, `anthropic-messages`, `openai-responses`, `openai-completions`.
+Check your model's `api` before you pass it — several pi registries declare something else and
+are not routable today, among them `mistral` (`mistral-conversations`), `google`
+(`google-generative-ai`), `google-vertex`, `amazon-bedrock`, `azure-openai-responses` and
+`openai-codex`.
+
+Workers AI stays on `env.AI.run`. Its departures from strict OpenAI chat completions (native
+events, heartbeats, per-delta usage, per-family request quirks) are absorbed by the same
+compat layer the [AI SDK provider](./models.md) uses, so the parser only ever sees strict
+OpenAI chunks. Anything the layer had to drop is recorded on the message as a
+`cloudflare-compat` diagnostic.
+
+An `api` this package cannot route, or a `baseUrl` whose host AI Gateway does not serve, ends
+the stream with an `error` event naming it.
+
+### Thinking shapes
+
+On the `openai-completions` wire this package builds one thinking shape: OpenAI's
+`reasoning_effort`, with the level remapped through the model's own `thinkingLevelMap`, and
+only when the model reasons and its `compat` does not say `supportsReasoningEffort: false`.
+
+A model whose `compat.thinkingFormat` is anything else — `deepseek`, `zai`, `openrouter`,
+`together`, `ant-ling`, `qwen`, `qwen-chat-template`, `chat-template`, `string-thinking` — has
+its reasoning level **dropped**, and the drop recorded on the message as a `cloudflare-compat`
+diagnostic. Those shapes are the vendor's, and building them here would mean keeping vendor
+wire formats in `agents`, which is the one thing this package does not do: reach such a model
+through pi-ai's own provider if you need its thinking shape. The `anthropic-messages` and
+`openai-responses` wires build the thinking shape their own model metadata declares.
+
+## Unified billing and BYOK
+
+Most vendors work with no key at all: the gateway bills them to your Cloudflare account. A
+vendor that is not on unified billing answers with its own error until you store a key on the
+gateway — Google AI Studio, reached through a routable model, returns `403 unregistered
+callers`. That error is the vendor's, surfaced as-is; the gateway's own failures (no such
+gateway, an unpaid account) become a `CloudflareAIError`, so fallback and retry treat them the
+way they treat Workers AI failures. A model whose `api` is not one of the four routed markers
+never gets that far: the stream ends with an `error` event naming the `api`.
+
+## Observability
+
+Every assistant message carries a `cloudflare` diagnostic with the gateway correlation for its
+response:
+
+```ts
+const message = await ai.complete(model, context);
+const cloudflare = message.diagnostics?.find(
+  (d) => d.type === "cloudflare"
+)?.details;
+// { model, gateway, provider?, specifier?, logId?, eventId?, cacheStatus?,
+//   step?, traceId?, runId? }
+```
+
+A failed dispatch ends the stream with an `error` event whose message carries a
+`cloudflare-error` diagnostic (`status`, `code`, `logId`), and `onResponse` fires with the raw
+status and headers for every request.
+
+## Fallback
+
+```ts
+ai(claude, { fallback: ["@cf/zai-org/glm-4.7-flash"] });
+```
+
+Legs may be Workers AI ids or models, in any mix. They are tried in order: a leg is abandoned
+when its stream ends with an `error` before it produced any output, and once a leg has
+produced output it is committed to. The message that answers carries a `cloudflare-fallback`
+diagnostic listing the abandoned attempts, and `message.model` names the model that actually
+answered.
+
+A leg travels with the chain's gateway options — the chain's `id`, `cacheTtl` and the rest win
+over a leg's own, and `metadata` merges — as they do in the [AI SDK provider](./models.md). A
+leg built as `ai(model, options)` keeps everything else it was built with. A leg named by a
+`@cf/` id has no options of its own, so it also inherits the chain's `headers`, session
+affinity and Workers AI knobs; it never inherits the chain's model metadata (`name`, `cost`,
+`contextWindow`, `maxTokens`, `reasoning`, `input`), which describes the model it was written
+on — the leg's usage and cost are its own registry entry's.
+
+## Images
+
+```ts
+const images = ai.images("@cf/black-forest-labs/flux-1-schnell", { steps: 8 });
+const result = await ai.generateImages(images, {
+  input: [{ type: "text", text: "A red bicycle" }]
+});
+```
+
+Image generation is Workers AI only. The knobs a model takes are the model's:
+`flux-1-schnell` takes a prompt and `steps` and nothing else, so `width`,
+`height`, `seed`, `guidance` and `negativePrompt` are dropped for it and listed
+in a `cloudflare-compat` diagnostic on the result rather than sent and ignored.
+Every other model in the catalog takes the full set.
+
+## Experimental
+
+This subpath needs `@earendil-works/pi-ai@^0.99.2`, an optional peer of `agents`. `agents`
+takes no vendor package as a dependency, optional or otherwise. The surface is experimental
+and may change in a minor release. An unlisted `@cf/` id resolves with default metadata (a
+128k context window, 8192 output tokens, zero cost); pass overrides when you know better.
+
+## See also
+
+- [Models for the AI SDK](./models.md) — the same factory for the Vercel AI SDK
+- [pi-ai](https://github.com/earendil-works/pi/tree/main/packages/ai) and
+  [pi-durable](https://github.com/earendil-works/pi/tree/main/packages/durable)