experimental-a2 0.18.1 → 0.20.0

package/docs/01-quickstart.mdx

@@ -60,7 +60,7 @@ export const ordersServer = createServer({
 
 Development needs no database setup: events go to SQLite at `.a2/dev.db` and
 survive restarts. Tests get an in-memory store. Production needs an explicit
-store; see [Production](/server/production).
+store; see [Stores](/stores/overview).
 
 ## 4. Append the first event
 
@@ -169,11 +169,14 @@ You have a validated write path, a handler, a log you can read, and a view
 folded from it. The rest of these docs apply the same two moves, append an
 event and react to it, to timers, cancellation, live UIs, and production.
 
-<CardGroup cols={2}>
+<CardGroup cols={3}>
   <Card title="Handlers" href="/server/handlers" icon="file">
     Context, chaining, and re-runs.
   </Card>
-  <Card title="Production" href="/server/production" icon="rocket">
-    A store, a scheduler, dead letters.
+  <Card title="Stores" href="/stores/overview" icon="rocket">
+    Where sessions live in production.
+  </Card>
+  <Card title="Schedulers" href="/schedulers/overview" icon="clock">
+    Retries and timers on a queue.
   </Card>
 </CardGroup>

package/docs/actors/01-introduction.mdx

@@ -57,7 +57,7 @@ handler's first lines and throw `NonRetriableError` on bad shapes.
 
 `actor` also accepts `createServer`'s `store`, `scheduler`, and
 `telemetry` options, with the same development default store; see
-[Production](/server/production). A protocol can also declare
+[Stores](/stores/overview) and [Schedulers](/schedulers/overview). A protocol can also declare
 [`presence`](/actors/routes#presence).
 
 ## Call it

package/docs/actors/meta.ts

@@ -1 +1 @@
-export default { order: 6 }
+export default { order: 8 }

package/docs/agents/02-tools.mdx

@@ -81,7 +81,7 @@ for the event payload.
 
 This tool schedules a reminder as a new user message. `ctx.session` belongs to
 the executing agent, so `schedule()` uses that agent's scheduler: add
-`setReminder` to its `tools` and configure a [scheduler](/server/production).
+`setReminder` to its `tools` and configure a [scheduler](/schedulers/overview).
 Naming the timer after `toolCallId` and the message after the triggering
 event's ID makes a retry converge on the same timer and message, even if a
 provider reuses tool-call IDs. The six-day maximum is Vercel Queues'

package/docs/agents/03-conversations.mdx

@@ -393,7 +393,7 @@ retries, and so does one where A2 itself fails while or just after the tool
 runs. Every other tool uses the default, `'rerun'`. See
 [`agent()`](/reference/ai#agentoptions) for the exact cases.
 
-Configure a [production scheduler](/server/production) as for any A2 server. It
+Configure a [production scheduler](/schedulers/overview) as for any A2 server. It
 recovers an incomplete generation after the original serverless invocation
 disappears.
 
@@ -446,9 +446,11 @@ export function Queue({ sessionId }: { sessionId: string }) {
 ```
 
 `sendMessage()` puts a message in the durable inbox, `state.inbox.items`. The
-controller admits one input at a time, freezes its content, and adds it to
-`state.messages`; pending inputs never enter the model prompt. `state.active`
-is the logical turn: its ID, frozen `input`, and `phase`. The turn ID holds
+controller admits one input at a time and adds it to `state.messages`; pending
+inputs never enter the model prompt, and an edit after admission is rejected.
+The log stores a message once, in the command that sent it or the edit that
+last changed it. `state.active` is
+the logical turn: its ID, its `input`, and `phase`. The turn ID holds
 across model steps, tools, and a pause. Stop ends the turn and admits the next
 input when admission is running.
 
@@ -466,15 +468,20 @@ replay cannot recreate the prompt. Removing an input that is no longer pending
 (admitted, removed, or never queued) returns `not-pending`, as does an edit that
 loses to admission; a Stop aimed at a finished turn returns `stale-turn`.
 
-`steer()` puts new input first. If the observed turn is still active, it
-finishes its current model response and that response's tools, including
-authorized deferred provider results, then hands off before another model
-step. Output produced after the click stays, and unanswered approval and input
-requests close, so the unapproved tool never runs; a denied approval never
-blocks the handoff. With no active turn, `steer()` sends like `sendMessage()`.
-
-`queue.sendNow()` makes the same handoff with a queued message, keeping its ID
-and latest accepted content and promoting passive user input to generation.
+`steer()` puts new input right after the turn's earlier steering, or first
+when there is none, so steering hands off in the order it was sent. If the observed turn
+is still active, it finishes its current model response and that response's
+tools, including authorized deferred provider results, then hands off before
+another model step. Everything steering that turn enters together as one new
+turn: the earlier messages as context and the last one generating, so the model
+answers them once. Output produced after the click stays, and unanswered
+approval and input requests close, so the unapproved tool never runs; a denied
+approval never blocks the handoff. With no active turn, `steer()` sends like
+`sendMessage()`.
+
+`queue.sendNow()` makes the same handoff with a queued message, after the
+steering sent before it, keeping its ID and latest accepted content and
+promoting passive user input to generation.
 Non-user context returns `not-user-input`, and a stale observed turn returns
 `stale-turn` without changing the queue. Send now uses the accepted content, so
 save editor text first. `steer()` and `queue.sendNow()` both preserve pause: a
@@ -483,7 +490,8 @@ immediate interruption.
 
 A queued item's optional `afterStepOf` records the observed turn for steering.
 Editing keeps it, removing the item cancels it, and queue order decides which
-input is admitted at the handoff. See [`inputs`](/reference/ai#inputs) for the
+inputs are admitted at the handoff: the run at the front that steers the same
+turn. See [`inputs`](/reference/ai#inputs) for the
 full handoff rules.
 
 ### Input identity
@@ -531,6 +539,10 @@ finish. `pause({ when: 'now' })` also cancels current model and tool work;
 `state.active.phase` is `pausing` while old work settles, then `paused`.
 `resume()` resumes retained work before admitting another input.
 
+A turn paused with Pause now keeps its steering until Resume. Every
+`steer()` or `queue.sendNow()` sent while it's paused targets it, and on Resume
+they enter together as one turn; the paused turn's own work does not resume.
+
 Resume starts a fresh model request from durable state; it does not resume the
 provider's old stream or undo completed tool effects. While paused, accepted
 tool calls are retained with their outcomes recorded, and the turn holds no

package/docs/agents/04-harnesses.mdx

@@ -110,17 +110,63 @@ Both harnesses call models through Vercel AI Gateway. `model` is a Gateway
 model ID, or [`ModelSettings`](/reference/ai#modelsettings) with one, resolved
 for each step like any agent's model; `pi()` takes the Gateway models Pi's own
 catalog lists. `reasoning` becomes FX's effort or Pi's thinking level, and
-`speed: 'fast'` asks FX for the faster tier. Credentials come from
-`AI_GATEWAY_API_KEY`, or from Vercel's OIDC token on Vercel. The harness reads
-the text of each user message; files and seeded assistant messages do not
-reach it.
+`speed: 'fast'` asks AI Gateway for the model's faster tier under either
+harness. Credentials come from `AI_GATEWAY_API_KEY`, or from Vercel's OIDC
+token on Vercel. Seeded assistant messages do not reach the harness.
+
+## Images and files
+
+The harness reads each user message's text and its image files. Neither FX nor
+Pi takes URLs, so A2 fetches each image from its `https:` or `data:` URL and
+hands the harness its bytes. The fetch goes through the
+AI SDK's download, which refuses private addresses, checks each redirect, and
+stops past 32 MiB of one message's files. Each step of a response replays its
+turn, so a process keeps the images it fetched instead of fetching them again.
+
+A file that isn't an image, or that A2 can't fetch, fails the turn with an
+error that names it. So does a steer with files under FX, which steers with
+text only. The harness applies its own rules to the images it gets: FX fails a
+turn whose model doesn't take images, and Pi resizes them. Every step sends the
+conversation so far, files and all, so a file that fails one turn fails the
+turns after it too.
+
+A2 fetches whatever URL a user message names, so check file URLs on the way
+in. An agent's `messageSchema` can require them to point at your storage:
+
+```ts server/reviewer.ts
+import type { UIMessage } from 'ai'
+import { agent } from 'experimental-a2/ai/server'
+import { fx } from 'experimental-a2/ai/fx'
+import { z } from 'zod'
+
+// your storage's public URL:
+const uploads = 'https://uploads.example.com/'
+
+export const reviewer = agent({
+  name: 'reviewer',
+  harness: fx(),
+  model: 'anthropic/claude-sonnet-5.5',
+  instructions: 'Review the screenshots the user sends. Keep replies short.',
+  messageSchema: z
+    .custom<UIMessage>()
+    .refine(
+      (message) =>
+        message.role !== 'user' ||
+        message.parts.every(
+          (part) => part.type !== 'file' || part.url.startsWith(uploads),
+        ),
+      "A user's files must be uploads",
+    ),
+})
+```
 
 ## What the harness owns
 
 The harness owns its loop, its prompt, and its compaction, so `generation`,
 `generate`, `compaction`, and `messageMetadata` exist only in
 `{ type: 'ai-sdk' }`. A harness runs its tools without approval, so a tool that
-sets `needsApproval` is rejected. `maxSteps` limits the model calls in one
+sets `needsApproval` is rejected. It hands the engine only tools A2 runs, so a
+provider-defined tool, such as a provider's web search, is rejected too. `maxSteps` limits the model calls in one
 response under every harness. Steering enters at the next step, as with any
 agent: the harness sees the steered message before its next model call.
 

package/docs/agents/meta.ts

@@ -1 +1 @@
-export default { order: 5 }
+export default { order: 7 }

package/docs/client/meta.ts

@@ -1 +1 @@
-export default { order: 4 }
+export default { order: 6 }

package/docs/contract/03-sessions.mdx

@@ -70,7 +70,7 @@ authorize your own route before calling it.
 
 ### Scheduling a clear
 
-With a [scheduler](/server/production) configured, a handler can schedule a
+With a [scheduler](/schedulers/overview) configured, a handler can schedule a
 clear. This one clears an idle cart a day after its last activity:
 
 ```ts server/carts.ts
@@ -109,7 +109,7 @@ append returned.
 Delivery runs the same conditional clear and finishes the deletion before it
 acknowledges, so provider retries complete an interrupted delete. No event is
 appended. The clear happens at delivery, not at an exact deadline, within
-provider limits: Vercel Queues accepts delays up to six days.
+provider limits: Vercel Queues accepts delays up to seven days.
 
 ## Sessions and your database
 

package/docs/index.mdx

@@ -156,7 +156,7 @@ from the log.
   <Card title="Timers" href="/server/timers" icon="clock">
     Schedule an event for later.
   </Card>
-  <Card title="Production" href="/server/production" icon="rocket">
+  <Card title="Production" href="/stores/overview" icon="rocket">
     A store and a scheduler.
   </Card>
 </CardGroup>
@@ -212,7 +212,7 @@ A durable store backend, such as Postgres or Redis. Add a queue-backed scheduler
 when the contract has handlers or calls `session.schedule`. A history-only
 contract with no timers needs no reaction infrastructure. Development needs
 neither: SQLite appears under `.a2/` and state survives restarts. See
-[Production](/server/production).
+[Stores](/stores/overview) and [Schedulers](/schedulers/overview).
 
 </details>
 

package/docs/reference/01-core.mdx

@@ -17,15 +17,15 @@ description: "Every entry point, and the isomorphic `experimental-a2` core: cont
 | `experimental-a2/ai/pi` | `pi`: the Pi harness for `agent()` | `ai`, `@earendil-works/pi-coding-agent`, `@earendil-works/pi-ai` |
 | `experimental-a2/ai/client` | `createAgentClient`: contract-free agent client | `ai` |
 | `experimental-a2/ai/react` | `useAgent` | `ai`, `react` |
-| [`experimental-a2/store-postgres`](/reference/adapters#experimental-a2store-postgres) | `postgres`: Postgres store backend | `pg` (or inject a client) |
-| [`experimental-a2/store-redis`](/reference/adapters#experimental-a2store-redis) | `redis`: Redis Streams store backend, push-native streaming | `ioredis` (or inject a client) |
-| [`experimental-a2/store-redis-http`](/reference/adapters#experimental-a2store-redis-http) | `redisHttp`: the same Redis store over provider REST APIs (Upstash) | none |
-| `experimental-a2/store-sqlite` | SQLite store backend | none |
-| `experimental-a2/store-memory` | in-memory store backend | none |
-| `experimental-a2/scheduler-vercel` | `vercelQueues`: recovery and timers | `@vercel/queue` |
-| `experimental-a2/scheduler-qstash` | `qstash`: signed recovery and timers | `@upstash/qstash` |
+| [`experimental-a2/store-postgres`](/stores/postgres) | `postgres`: Postgres store backend | `pg` (or inject a client) |
+| [`experimental-a2/store-redis`](/stores/redis) | `redis`: Redis Streams store backend, push-native streaming | `ioredis` (or inject a client) |
+| [`experimental-a2/store-redis-http`](/stores/redis-http) | `redisHttp`: the same Redis store over provider REST APIs (Upstash) | none |
+| [`experimental-a2/store-sqlite`](/stores/sqlite) | SQLite store backend | none |
+| [`experimental-a2/store-memory`](/stores/memory) | in-memory store backend | none |
+| [`experimental-a2/scheduler-vercel`](/schedulers/vercel-queues) | `vercelQueues`: recovery and timers | `@vercel/queue` |
+| [`experimental-a2/scheduler-qstash`](/schedulers/qstash) | `qstash`: signed recovery and timers | `@upstash/qstash` |
 | `experimental-a2/cache-indexeddb` | `indexedDb` browser cache | none |
-| `experimental-a2/otel` | `otel` telemetry adapter | `@opentelemetry/api` |
+| [`experimental-a2/otel`](/reference/telemetry) | `otel` telemetry adapter | `@opentelemetry/api` |
 
 `experimental-a2` imports none of the backends, and a browser-bundle test in CI
 enforces it. Importing a backend's entry point, such as

package/docs/reference/02-server.mdx

@@ -438,7 +438,7 @@ Custom scheduler adapters implement two methods:
 
 Application routes call `scheduler.handler(...servers)` on the configured
 adapter. See
-[Production](/server/production#2-add-a-scheduler-for-handlers-and-timers).
+[Schedulers](/schedulers/overview).
 
 `SchedulerTask` is `SchedulerDrainTask | SchedulerAppendTask |
 SchedulerClearTask`. These task types, plus `ScheduledEvent`, are exported
@@ -522,7 +522,7 @@ session.append.dispatch(
 
 Commits the same atomic batch, then sends pending work to the configured
 scheduler instead of starting an inline drain. See
-[Dispatch in a fresh invocation](/server/production#dispatch-in-a-fresh-invocation).
+[Dispatch in a fresh invocation](/schedulers/overview#dispatch-in-a-fresh-invocation).
 
 - Awaits acceptance of an immediate scheduler message. A batch that leaves no
   pending handler work sends none.
@@ -681,7 +681,7 @@ type DeadLetter = Event & {
 ```
 
 The session's dead-lettered events, oldest first, each with why it failed. See
-[When an event dead-letters](/server/production#3-when-an-event-dead-letters).
+[Resolve a dead letter](/server/handlers#resolve-a-dead-letter).
 
 - `failureCount` counts the attempts that spent its budget: caught failures,
   and attempts that ended before they finished. `attemptCount` counts every

package/docs/reference/04-ai.mdx

@@ -39,8 +39,8 @@ state: serve it with `agent({ contract })` and use it with
   through.
 - An `initialState` key that `AIState` also has throws a `TypeError` and fails
   to typecheck.
-- The reducer name is `AI_STATE_REDUCER` (`a2.ai.state.v26`) without
-  `reducer`, else `${reducer.name}+a2.ai.state.v26`, so checkpoints advance when
+- The reducer name is `AI_STATE_REDUCER` (`a2.ai.state.v27`) without
+  `reducer`, else `${reducer.name}+a2.ai.state.v27`, so checkpoints advance when
   either fold changes. `AgentReducer` types that name (so
   [`createAgentClient()`](#createagentclientoptions) accepts a plain
   `agentContract()`) and carries `X` to `agent()`'s resolvers.
@@ -61,9 +61,9 @@ even when `M` declares its own data parts.
 
 ### `AIMessageOf<Events>`
 
-The `UIMessage` type a contract's `ai.message.created` schema carries, such as
-`AIMessageOf<typeof contract.events>`; `UIMessage` otherwise. `agent()` types
-messages with it.
+The `UIMessage` type a contract's input commands (`ai.control.requested`)
+carry, such as `AIMessageOf<typeof contract.events>`; `UIMessage` otherwise.
+`agent()` types messages with it.
 
 ### `handlerContext(contract?)`
 
@@ -105,13 +105,13 @@ Pure typed builders of `ai.control.requested` commands, for `append()` and
 | `inputs.requestInput(request)` | requests application input from trusted server code |
 | `inputs.retry(options)` | retries a failed response |
 | `inputs.stop({ turnId, lastSeenIndex? })` | ends the observed turn and advances when admission is open |
-| `inputs.steer({ turnId, message, lastSeenIndex? })` | puts the new input first for a handoff after the observed turn's current step |
+| `inputs.steer({ turnId, message, lastSeenIndex? })` | queues the new input right after the observed turn's earlier steering, or first, for a handoff after its current step; the turn's steering enters together, its last generating input starting the turn |
 | `inputs.queue.edit({ message })` | changes a pending input; its ID stays fixed |
 | `inputs.queue.move({ inputId, beforeId })` | moves pending input before an ID; `null` means the end |
 | `inputs.queue.remove({ inputId })` | removes pending input; its submission stays in the log |
-| `inputs.queue.sendNow({ inputId, turnId, lastSeenIndex? })` | promotes an existing queued input for a handoff after the current step; preserves pause |
+| `inputs.queue.sendNow({ inputId, turnId, lastSeenIndex? })` | promotes an existing queued input, after the observed turn's earlier steering, for a handoff after the current step; preserves pause |
 | `inputs.pause({ when: 'now' \| 'after-turn' })` | pauses execution now or inbox admission after the turn |
-| `inputs.resume()` | resumes retained work, then inbox admission |
+| `inputs.resume()` | resumes retained work, then inbox admission; a turn paused now with steering hands it off instead |
 | `inputs.toolResult(result)` | records a deferred provider-executed tool result from trusted server code |
 | `inputs.interrupt(options)` | interrupts a specific generation/request owner |
 
@@ -174,7 +174,9 @@ Pure typed builders of `ai.control.requested` commands, for `append()` and
 - **Interruption.** `ai.message.interrupted` is `{ messageId, requestId,
   reason?, lastSeenIndex? }` or `{ messageId, generationId, reason?,
   lastSeenIndex? }`: copy `activeRequestId` before the generation starts,
-  `activeGeneration.generationId` after. A request-owned interruption survives
+  `activeGeneration.generationId` after. A2 writes the reason
+  `'steered after step'` itself when steering ends a response, so an
+  interruption you send can't use it. A request-owned interruption survives
   that request starting first. An older or mismatched message/owner pair is a
   no-op. An accepted one fences its owner for good: later events stay in raw
   history but cannot alter `AIState`, satisfy pending work, or reactivate the
@@ -249,7 +251,7 @@ extra I/O or raw-history reads.
   win.
 - Steer, and Send now against an active turn, show in `state.messages` after
   the current response, not the inbox, until admission; they stay out of the
-  prompt. Several steering inputs show in admission order, most recent first,
+  prompt. Several steering inputs show in the order they were sent,
   and the display survives receipt and hydration, including while paused.
 - A pending Stop clears the displayed `state.active`, generation, and pending
   approvals and inputs, previews an idle status (paused when the inbox is), and
@@ -274,8 +276,8 @@ events. Encoding is internal; no event or tool payload is truncated.
 | Field | What it holds |
 | --- | --- |
 | `messages` | [`TranscriptMessage<M>`](#transcriptmessagem) values; server-side, without pending inputs |
-| `inbox` | `{ paused, items }`; items are `{ id, message, generate, afterStepOf? }`, where `afterStepOf` is an observed steering turn that hands off after its current step (removal cancels it, admission consumes it) |
-| `active` | `null` or `{ turnId, input, phase }`; admission freezes `input` |
+| `inbox` | `{ paused, items }`; items are `{ id, role, message, generate, afterStepOf? }`, where `afterStepOf` is an observed steering turn that hands off after its current step (removal cancels it, admission consumes it) |
+| `active` | `null` or `{ turnId, input, phase }`, where `input` is the admitted input's `{ id, role, generate }` |
 | `status` | generation status |
 | `starting` | a speculative optimistic turn |
 | `receipt` | the latest user command receipt |
@@ -311,7 +313,10 @@ See [What happens after a send](/agents/introduction#what-happens-after-a-send).
 
 `deriveUIMessages()` folds a raw event history into `TranscriptMessage<M>[]`.
 `reduceAIState()` is the event-at-a-time form the standard reducer uses. Both
-project the same chunk-only protocol synchronously.
+project the same chunk-only protocol synchronously. A message's content is in
+the command that brought it in, so a fold needs each `ai.message.created`'s
+command before it; one without throws. Only A2's control lane writes
+`ai.message.created`.
 
 ## `experimental-a2/ai/server`
 
@@ -357,7 +362,7 @@ stub.
 | `toolLane` | one lane for every local tool execution, one at a time in admission order; model steps keep their own lane |
 | `toolRecovery` | `'rerun'` (default) or `'report'` per tool, the harness's own tools included |
 | `handlers` | application event handlers; naming an event the agent handles throws a `TypeError` |
-| `store`, `scheduler`, `telemetry` | as for any A2 server; `telemetry` covers Core and AI in one trace ([span catalogue](/reference/adapters#the-span-catalogue)) |
+| `store`, `scheduler`, `telemetry` | as for any A2 server; `telemetry` covers Core and AI in one trace ([span catalogue](/reference/telemetry#the-span-catalogue)) |
 
 **Contracts.** A contract not built by `agentContract()` throws a `TypeError`
 and fails to typecheck (serve custom contracts with `createServer()`). Messages
@@ -525,7 +530,11 @@ name the harness's own tools, such as Pi's `bash`. `model` is resolved per step
 as for any agent. `instructions` and `tools` reach the harness; how they
 combine with its own prompt and tools depends on the harness. Tool approval
 policy lives in `harness.generation`, so a harness agent's tools run without
-approval, and a tool that sets `needsApproval` is rejected.
+approval, and a tool that sets `needsApproval` is rejected. So is a
+provider-defined tool (`type: 'provider'`), which the engine can't run. A user
+message reaches the harness as its text and its images, which A2 fetches from
+their `https:` or `data:` URLs; any other file fails the turn. See
+[Images and files](/agents/harnesses#images-and-files).
 
 A harness resumes after a crash at the interrupted step. Finished model calls
 are not made again and finished tool calls do not run again. A2 rebuilds the
@@ -589,8 +598,16 @@ continue a response, A2 restores FX's latest checkpoint and replays the
 response so far from the log: recorded model responses and tool results answer
 the kernel without network calls or tool runs, then FX makes its next model
 call. Replay grows with the response, roughly 0.1 to 0.5 seconds for one of 80
-to 200 tool calls. FX checkpoints hold at most 4 MiB of conversation; past that,
-A2 replays from an earlier checkpoint.
+to 200 tool calls. When libfx can't take a checkpoint, A2 replays from an
+earlier one.
+
+FX's model calls carry the same [AI Gateway](#agentoptions) defaults as the
+AI SDK harness: `caching: 'auto'`, under any Gateway options FX sets itself,
+and the session's stable `x-session-affinity` header in place of FX's own,
+which changes every time A2 rebuilds the kernel. A2 leaves every other byte of
+the request as FX wrote it, so it adds caching only where FX's own options come
+last in the request, as libfx writes them; otherwise the request goes
+unchanged.
 
 Credentials come from `AI_GATEWAY_API_KEY`, or Vercel's OIDC token on Vercel.
 This entry point is server-only.
@@ -617,8 +634,8 @@ forbids, such as a property it does not list. A call to one of the agent's tools
 then goes through the tool's schema, as under `{ type: 'ai-sdk' }`: `execute`
 receives the schema's output, defaults included. Input either check rejects goes
 back to the model as an error, without running the tool. `reasoning` becomes
-Pi's thinking level, and `model` is one of the Gateway models Pi's catalog
-lists. Pi works in the sandbox's working directory; its `bash` runs with the
+Pi's thinking level and `speed: 'fast'` asks AI Gateway for the model's fast
+tier; `model` is one of the Gateway models Pi's catalog lists. Pi works in the sandbox's working directory; its `bash` runs with the
 sandbox's environment, never the server's. Pi compacts its own context; a
 compaction appears in the transcript as a `data-a2-compaction` part with its
 summary.

package/docs/reference/05-telemetry.mdx

@@ -0,0 +1,132 @@
+---
+title: Telemetry
+description: "Trace A2 with OpenTelemetry, and the spans it reports."
+---
+
+```ts server/orders.ts
+import { createServer } from 'experimental-a2/server'
+import { otel } from 'experimental-a2/otel'
+import { orders } from '@/contracts'
+
+export const ordersServer = createServer({
+  contract: orders,
+  telemetry: otel(),
+  // ...store, scheduler, and handlers
+})
+```
+
+With any OpenTelemetry setup registered (on Vercel, `@vercel/otel`), one trace
+shows the request, `a2.append`, the inline `a2.drain`, an `a2.event` span per
+handler, and the appends those handlers make. Failures mark their `a2.event`
+span, and `a2.event.outcome = dead_lettered` is the attribute to alert on.
+Agent servers add `a2.ai.*` spans.
+
+## `experimental-a2/otel`
+
+```ts
+otel(options?: { tracer?: Tracer; tracerName?: string }): A2Telemetry
+```
+
+OpenTelemetry instrumentation for a server (`@opentelemetry/api` is a peer
+dependency). Pass it as `telemetry`. Spans nest through the active context, so
+the happy path lands in one trace. `A2Telemetry` is a one-method interface:
+anything with `span(name, attributes, fn)` works, and `otel()` is the adapter
+A2 ships.
+
+## The span catalogue
+
+This catalogue is a contract: renames and additions are breaking for
+dashboards and are called out in the package's `CHANGELOG.md`. All spans carry
+`a2.contract` and `a2.session_id`. Core spans describe the durable runtime;
+`a2.ai.*` spans describe the built-in AI handlers.
+
+| Layer | Span | Wraps |
+| ----- | ---- | ----- |
+| Core | `a2.append` | validation + the store write (one batch) |
+| Core | `a2.drain` | one drain pass, including concurrent handlers |
+| Core | `a2.event` | one claimed dispatch, including its handler and completion |
+| Core | `a2.claim` | one `claimAvailable` store operation |
+| Core | `a2.commit` | one `completeAttempt` or `failAttempt` store operation |
+| Core | `a2.state` | one `state()` read: snapshot-plus-tail load + reducer fold |
+| Core | `a2.snapshot` | one background batch of checkpoint writes |
+| Core | `a2.redrive` | one `session.redrive()`: the store operation and starting the session's wakeup |
+| AI | `a2.ai.control` | applying an input/work report/session-close decision and preparing its returned events |
+| AI | `a2.ai.generate` | one generation handler, including Core calls and model/SDK work |
+| AI | `a2.ai.context` | constructing model messages, including tool prompt reconstruction when uncached |
+| AI | `a2.ai.compaction` | automatic summarization or the custom compaction callback |
+| AI | `a2.ai.tool` | one durable tool handler, including application execution |
+| AI | `a2.ai.ack` | waiting for a persisted work report's canonical events and control receipt |
+
+Durations include nested work: `a2.event` contains its AI handler and later
+commit, AI handlers call Core, and `a2.state` includes reducer code.
+Acknowledgements overlap control processing, claims can cover several events,
+and snapshots run in the background, so summing spans, or subtracting summed
+children, does not yield exclusive Core or AI overhead. Model and application
+I/O is included; its child spans have the detail.
+
+Attributes arrive in two waves: **at start** (passed to `span()`) and
+**mid-span** (via `handle.setAttribute`, for outcomes and counts known only
+after the work ran). A recorder must merge both; the ones you alert on are all
+mid-span.
+
+| Attribute             | Span        | When  | Values                                                |
+| --------------------- | ----------- | ----- | ----------------------------------------------------- |
+| `a2.append.source`    | `a2.append` | start | `external` \| `handler`                                |
+| `a2.append.mode`      | `a2.append` | start | `inline` \| `dispatch`                                |
+| `a2.append.types`     | `a2.append` | start | comma-joined event types                              |
+| `a2.append.count`     | `a2.append` | start | batch size                                            |
+| `a2.append.armed`     | `a2.append` | mid   | `false` when the scheduler arm failed and this append degraded to append-driven healing |
+| `a2.drain.outcome`    | `a2.drain`  | mid   | `settled` \| `busy` \| `stalled` \| `failed`          |
+| `a2.drain.processed`  | `a2.drain`  | mid   | events processed this pass                            |
+| `a2.claim.outcome` | `a2.claim` | mid | `claimed` \| `busy` \| `settled` |
+| `a2.claim.count` | `a2.claim` | mid | number of claimed events, or zero |
+| `a2.commit.operation` | `a2.commit` | start | `complete` \| `fail` |
+| `a2.commit.index` | `a2.commit` | start | event log position |
+| `a2.commit.attempt` | `a2.commit` | start | durable 1-based attempt ordinal |
+| `a2.commit.outcome` | `a2.commit` | mid | `completed` \| `failed` \| `dead_lettered` \| `superseded` |
+| `a2.event.type`       | `a2.event`  | start | the event's type                                      |
+| `a2.event.index`      | `a2.event`  | start | log position                                          |
+| `a2.event.id`         | `a2.event`  | start | event id                                              |
+| `a2.event.attempt`    | `a2.event`  | start | same durable 1-based ordinal as `ctx.attempt`          |
+| `a2.event.lane`       | `a2.event`  | start | stored lane value; absent for concurrent unlaned work  |
+| `a2.event.handled`    | `a2.event`  | start | normally `true`; `false` when a custom-adapter row has no handler |
+| `a2.event.outcome`    | `a2.event`  | mid   | `processed` \| `failed` \| `dead_lettered` \| `superseded` \| `surrendered` |
+| `a2.event.aborted`    | `a2.event`  | mid   | `true` when `abortOn` fired during the run            |
+| `a2.state.reducer`    | `a2.state`  | start | the reducer's name                                    |
+| `a2.state.snapshot`   | `a2.state`  | mid   | `hit` \| `miss` \| `rejected` (schema guard discarded it) \| `memory` (served from the server's in-memory state) |
+| `a2.state.folded`     | `a2.state`  | mid   | events folded past the snapshot                       |
+| `a2.state.index`      | `a2.state`  | mid   | the frontier the returned state reflects              |
+| `a2.snapshot.reducer` | `a2.snapshot` | start | the reducer's name                                  |
+| `a2.snapshot.index`   | `a2.snapshot` | start | greatest checkpoint index in the write batch       |
+| `a2.snapshot.count`   | `a2.snapshot` | start | number of checkpoints in the write batch           |
+| `a2.redrive.index` | `a2.redrive` | start | log position of the event being redriven |
+| `a2.redrive.action` | `a2.redrive` | start | `retry` \| `skip` |
+| `a2.redrive.redriven` | `a2.redrive` | mid | `true` when the event was dead-lettered and is now resolved |
+| `a2.redrive.armed` | `a2.redrive` | mid | `false` when the scheduler arm failed and the wakeup degraded to append-driven healing |
+| `a2.ai.control.type` | `a2.ai.control` | start | `ai.control.requested` \| `ai.work.reported` \| `ai.session.closed` |
+| `a2.ai.control.id` | `a2.ai.control` | start | triggering event id |
+| `a2.ai.control.attempt` | `a2.ai.control` | start | durable 1-based attempt ordinal |
+| `a2.ai.generate.request_id` | `a2.ai.generate` | start | generation request event id |
+| `a2.ai.generate.attempt` | `a2.ai.generate` | start | durable 1-based attempt ordinal |
+| `a2.ai.context.request_id` | `a2.ai.context` | start | generation request event id |
+| `a2.ai.context.phase` | `a2.ai.context` | start | `generation` \| `refresh` \| `tool` |
+| `a2.ai.compaction.request_id` | `a2.ai.compaction` | start | generation request event id |
+| `a2.ai.compaction.generation_id` | `a2.ai.compaction` | start | generation attempt id |
+| `a2.ai.compaction.policy` | `a2.ai.compaction` | start | `automatic` \| `custom` |
+| `a2.ai.tool.request_id` | `a2.ai.tool` | start | originating generation request event id |
+| `a2.ai.tool.call_id` | `a2.ai.tool` | start | tool call id |
+| `a2.ai.tool.name` | `a2.ai.tool` | start | tool name |
+| `a2.ai.tool.attempt` | `a2.ai.tool` | start | durable 1-based attempt ordinal |
+| `a2.ai.ack.request_id` | `a2.ai.ack` | start | reporting worker event id |
+| `a2.ai.ack.report_id` | `a2.ai.ack` | start | work report event id, matching the control span's id |
+
+- Drain outcomes: `settled` means nothing actionable is left; `busy` means live
+  per-event claims or a failed event's retry hold remain; `stalled` means a
+  caught failure in this drain needs a later retry. `failed` appears only on
+  scheduler deliveries: the drain threw after arming its successor watchdog,
+  the span records the error, and that successor retries the session.
+- A failing handler marks its `a2.event` span with the exception and error
+  status. Alert on `a2.event.outcome = dead_lettered`. A dead-lettered event
+  blocks only later events in its lane, until `session.redrive()` resolves it;
+  other lanes stay eligible.
+- `superseded` means a stale completion or failure did not change the log.

package/docs/reference/06-errors.mdx

@@ -44,7 +44,7 @@ export async function POST(req: Request) {
 | `SESSION_CLEARED` | a state read reaches below the latest clear, a handler reads state past a clear of its event, or an actor call is cleared before it receives its answer | no |
 | `CLAIM_EXPIRED` | never thrown; only the `ctx.signal` abort reason when the attempt's lease lapsed without renewal (assumed loss, decided locally, not proof) | no |
 | `STORE_UNAVAILABLE` | the store backend failed; nothing was written. The original error is the `cause` | **yes** |
-| `STORE_NOT_CONFIGURED` | production boots with no `store` configured (during `next build` page collection, deferred to first use) | no |
+| `STORE_NOT_CONFIGURED` | production boots with no `store` configured (during `next build` page collection, deferred to first use), a store's optional peer dependency is missing, or a Postgres database lacks A2's tables at the version the release needs (or the notification triggers `listenConnectionString` uses). The message names the fix | no |
 
 Retrying an *identical* batch is not `PARTIAL_DUPLICATE_BATCH`: it's an
 idempotent success that returns the previously appended events. A failing

package/docs/reference/meta.ts

@@ -1 +1 @@
-export default { order: 7 }
+export default { order: 9 }