experimental-a2 0.13.0 → 0.14.1

package/docs/guides/06-ai-agents.mdx

@@ -67,11 +67,30 @@ and continuation between steps. Responses have no step limit by default. Set
 `maxSteps` to a positive integer to limit one complete assistant response.
 Individual providers decide which model settings they support. `generation`
 therefore excludes `stopWhen`, tool execution callbacks, tool callers, sandbox
-execution, and the tool approval secret. Everything else, including
-`toolsContext`, passes through to the AI SDK untouched. A local tool reads its
-durable execution context later, from the handler that runs it. Model-step
+execution, and the tool approval secret. A2 preserves all other supplied
+settings, including `toolsContext`, when calling the AI SDK. A local tool
+reads its durable execution context later, from the handler that runs it. Model-step
 timeouts are supported; tool-execution timeouts are not.
 
+For AI Gateway models, the default generator sets
+`generation.providerOptions.gateway.caching` to `'auto'`. A2 merges this
+default with your provider options, with caller-supplied values taking
+precedence. Other Gateway options and provider namespaces are preserved.
+The same default applies to automatic compaction. A custom `generate`
+function receives your original settings and owns its caching policy.
+
+Gateway calls also send an `x-session-affinity` header derived from a SHA-256
+hash of the agent name and session ID. The value stays stable across turns,
+retries, worker restarts, and automatic compaction. No raw session identifier
+is sent in this header. A caller-supplied `generation.headers` entry takes
+precedence, regardless of header casing; an explicit `undefined` suppresses
+A2's default header. Set overrides here, since A2's per-call default can
+override headers configured on the model itself.
+Affinity helps supporting providers reuse their cache; it does not select
+Gateway's provider or guarantee a cache hit.
+See [Gateway automatic caching](https://vercel.com/docs/ai-gateway/models-and-providers/automatic-caching)
+for provider support and cache-write costs.
+
 ### Store assistant metadata
 
 Extend the agent's `UIMessage` type to keep generation status and usage on the
@@ -135,10 +154,10 @@ export const GET = assistantServer.fetch
 export const POST = assistantServer.fetch
 ```
 
-The route never calls the model directly. The browser appends user facts such
-as `ai.message.created`. Built-in server handlers schedule
-`ai.generation.requested`, run the AI SDK, and append progress back to the same
-log.
+The route never calls the model directly. The browser appends
+`ai.control.requested` commands. The controller admits inputs and schedules
+`ai.generation.requested`; workers run the AI SDK and append progress to the
+same log.
 
 ## Bind the reducer to React
 
@@ -309,11 +328,9 @@ export function AgentClient() {
 }
 ```
 
-There is no separate message state to reconcile. `push()` folds the
-`ai.message.created` fact locally before starting the request, so the user
-message appears immediately. If the request fails, A2 removes that optimistic
-event and the component restores the draft. The server owns the corresponding
-generation request.
+`push()` persists the command. Its receipt confirms admission to the inbox.
+`state.inbox.items` contains pending inputs; `state.messages` contains the
+admitted conversation. If persistence fails, the component restores the draft.
 
 The form gives Enter its normal submit behavior. The thinking row is also
 derived from `AIState`: it appears with the Agent label when the server's
@@ -323,7 +340,7 @@ it. Empty stream-start messages never create a blank conversation row.
 Users can send another message while a response is active. The message enters
 the durable log immediately, but its turn waits until the active assistant
 response, including every tool step and approval, reaches a terminal event.
-Queued messages keep log order.
+Inbox order defaults to arrival order and can be edited explicitly.
 
 Pass `{ generate: false }` when a user message should update the conversation
 without starting a model turn:
@@ -343,7 +360,7 @@ export async function recordPassiveMessage(
 }
 ```
 
-The passive user message still appears in `AIState.messages`. A later user
+The passive user message appears in `AIState.messages` when admitted. A later user
 message with the default generation behavior includes passive user messages
 before it in model context. Passive user messages after that trigger remain
 outside its prompt and wait for the next generating message. This ordering
@@ -355,7 +372,10 @@ already queued turn.
 One generating user interaction becomes a durable sequence:
 
 ```text
-browser  ai.message.created             optimistic, then durable
+browser  ai.control.requested           durable input command
+server   ai.message.created             input admitted
+server   ai.control.committed           accepted state changes
+server   ai.control.decided             command receipt
 server   ai.generation.requested         scheduled from durable facts
 server   ai.generation.started
 server   ai.generation.progress          batched UIMessageChunk[]
@@ -414,15 +434,26 @@ continuation that has not been appended yet.
 
 | Input | Events |
 | --- | --- |
-| `inputs.message(message, { generate? })` | records a message fact; `false` keeps context without scheduling |
+| `inputs.message(message, { generate? })` | queues input; `false` queues passive context |
 | `inputs.seed(message)` | records `generate: false`; non-user roles require a trusted append |
-| `inputs.approval(response)` | records an approval decision fact |
+| `inputs.approval(response)` | requests a decision on a pending approval |
 | `inputs.input(response)` | records an application input response fact |
 | `inputs.requestInput(request)` | records a trusted server request for application input |
-| `inputs.retry(options)` | records `ai.retry.requested` for a failed response |
-| `inputs.interrupt(options)` | interrupts an active response |
-
-The builders hide stable event ids, so the same interaction is safe to resend.
+| `inputs.retry(options)` | requests a fresh attempt for a failed response |
+| `inputs.stop({ turnId })` | ends the observed turn and advances |
+| `inputs.steer({ turnId, message })` | prioritizes input after the observed turn's current step finishes |
+| `inputs.queue.edit({ message, expectedRevision })` | edits a pending input |
+| `inputs.queue.move({ inputId, beforeId })` | reorders pending input; `null` means the end |
+| `inputs.queue.remove({ inputId })` | removes pending input from the inbox |
+| `inputs.queue.sendNow({ inputId, turnId })` | atomically prioritizes queued input and interrupts the observed turn |
+| `inputs.pause({ when })` | pauses now or after the active turn |
+| `inputs.resume()` | resumes retained work and inbox admission |
+| `inputs.toolResult(result)` | records a trusted deferred provider-tool result |
+| `inputs.interrupt(options)` | addresses a specific generation or request |
+
+Builders return `ai.control.requested` commands. Message, approval, input, and
+retry builders identify a stable interaction. Other controls use the event ID
+assigned by `push()` or `append()`; transport retries preserve that ID.
 Generation scheduling considers only user-role `ai.message.created` facts.
 Assistant and system message facts remain context regardless of the `generate`
 field, and model output is recorded as generation progress rather than a new
@@ -482,10 +513,9 @@ The AI SDK infers the tool input and output from its schemas and
 scope carrying the current durable handler attempt. Read it with
 `handlerContext(agent)` from `experimental-a2/ai`: it returns the same
 typed context bag an event handler receives, with `event`, `attempt`,
-`session`, and `signal`. Automatic tools see an `ai.tool.called` event.
-Approved tools see the `ai.approval.responded` event that authorized
-them. Narrow `ctx.event.type` if you need fields specific to either
-event. The agent argument carries the types; A2 verifies it against the
+`session`, and `signal`. Tools execute an `ai.tool.execution.requested`
+event after authorization. Its payload contains the admitted `call`, its
+`generation`, and its turn and execution version. The agent argument carries the types; A2 verifies it against the
 server executing the tool and throws when a tool written for one agent
 runs under another, or when `handlerContext()` is called outside a tool
 execution. The scope survives awaited helpers and async iteration, so
@@ -547,7 +577,7 @@ and plan.
 The schedule name uses `toolCallId`, and the message id uses the durable
 triggering event id. A retry therefore converges on the same timer and message,
 even if a provider reuses tool-call ids in a later generation. When the timer
-arrives, `inputs.message()` records an ordinary `ai.message.created` fact. The
+arrives, `inputs.message()` queues a durable input command. The
 queued-turn policy starts a fresh response after the active response completes
 or is interrupted. A failed response must be retried or interrupted first.
 If the scheduler send fails ambiguously or transiently, A2 retries the same
@@ -664,40 +694,15 @@ import { useSession } from '../session'
 
 export function StopButton() {
   const { state, push, index } = useSession()
-  const active = state.activeGeneration
-  const requested =
-    state.activeRequestId && state.activeResponseMessageId
-      ? {
-          messageId: state.activeResponseMessageId,
-          requestId: state.activeRequestId,
-        }
-      : null
-  const waiting =
-    state.pendingApprovals[0] ??
-    state.pendingInputs[0] ??
-    state.tools.find((tool) => tool.status === 'running')
-  const target = active
-    ? {
-        messageId: active.responseMessageId,
-        generationId: active.generationId,
-      }
-    : requested ??
-      (waiting
-        ? {
-            messageId: waiting.messageId,
-            generationId: waiting.generationId,
-          }
-        : null)
-
-  if (!target) return null
+  const active = state.active
+  if (!active) return null
 
   return (
     <button
       onClick={() =>
         void push(
-          ...inputs.interrupt({
-            ...target,
-            reason: 'Stopped by the user',
+          ...inputs.stop({
+            turnId: active.turnId,
             lastSeenIndex: index,
           }),
         )
@@ -709,17 +714,20 @@ export function StopButton() {
 }
 ```
 
-The optimistic event updates the UI immediately and reaches A2's cancellation
-channel. `lastSeenIndex` is the exact confirmed log frontier visible when the
+The accepted command ends the turn and reaches A2's cancellation channel. `lastSeenIndex` is the exact confirmed log frontier visible when the
 user clicked. The reducer rewinds the indexed generation projection to that
-frontier, keeps the partial response the user actually saw, and turns
-incomplete visible tools into `output-error`. An accepted interruption
+frontier and keeps the partial text the user actually saw. Tool parts whose
+arguments are still streaming are drafts: interruption removes them from the
+conversation and subsequent model prompts, even if their current JSON parses.
+Their progress remains in the raw log. Accepted tools still awaiting completion
+become `output-error`. An accepted interruption
 terminally fences that request or generation. Later generation, tool, approval,
 input, and compaction events remain in raw history but cannot reactivate it or
 alter the projection.
 Completed tool results at or before the visible frontier remain completed.
 
-Between `ai.generation.requested` and `ai.generation.started`, copy
+For the lower-level `inputs.interrupt()` builder, between
+`ai.generation.requested` and `ai.generation.started`, copy
 `activeRequestId` as `requestId` and `activeResponseMessageId` as `messageId`.
 Once `activeGeneration` exists, send its `generationId` and
 `responseMessageId` instead. A request-owned interruption remains valid if
@@ -800,10 +808,11 @@ metadata status are available in `state.modelMetadata`.
 positive safe integer. `compaction: { instructions: 'Preserve exact IDs.' }`
 adds guidance to the built-in summary prompt while keeping the derived threshold.
 `compaction: false` disables both compaction and metadata lookup. The `compaction`
-option can also be a synchronous resolver receiving the same
-`{ event, state, history, signal }` context as `model` and `instructions`. It
-returns `false`, automatic options, or a custom policy once per generation.
-Use this to select a session-specific threshold from already-recorded settings.
+option can also be a resolver receiving the same
+`{ event, state, session, signal }` context as `model` and `instructions`. It
+returns `false`, automatic options, or a custom policy once per generation and
+can await `session.state(settingsReducer)` to read application settings.
+Use this to select a threshold from the checkpointed AI state.
 Changes after resolution apply to a later model step. Disabling compaction
 preserves summaries that were already recorded. Custom provider
 models and custom global SDK providers need an explicit threshold. A custom
@@ -959,7 +968,7 @@ export const customGenerationServer = createAgentServer({
 ```
 
 `generate` receives compacted messages, resolved model and instructions, tools,
-generation settings, durable request identity, current state and history, and
+generation settings, durable request identity, current state, and
 the abort signal. It returns exactly one model step as a
 `ReadableStream<UIMessageChunk>`. A tool-aware replacement sends definitions to
 the model without running local `execute` functions. A2 still owns durable
@@ -968,6 +977,33 @@ progress, tool execution, approval, continuation, interruption, and failure.
 function owns its metadata and can pass a callback to `toUIMessageStream()` or
 emit typed metadata chunks itself.
 
+## Checkpoints and recovery reads
+
+Model steps read the AI reducer and coordinator at a fixed log boundary. Model,
+instruction, compaction, and custom-generation callbacks receive derived state.
+They do not receive a raw history array. Their `session.state(reducer)` reads
+application reducers at the captured prompt boundary by default; explicit
+`through` options follow the ordinary state-read contract. Compaction selection
+can be asynchronous when it needs a settings-state read.
+
+Ordinary generation uses checkpointed state. A recovered attempt reads from its
+saved prompt boundary to the captured current boundary, excluding abandoned
+attempts. Tools recovering in a fresh process reconstruct their original prompt
+from that checkpoint and the owning generation's start and compaction events.
+Compacted context uses its checkpoint and subsequent events. These AI history
+reads always have explicit lower and upper bounds. A2 core can rebuild a missing
+or invalid reducer snapshot from the log.
+
+Streaming progress updates the current message incrementally. Snapshots preserve
+open text and reasoning parts and partial tool inputs. Indexed progress remains
+available for exact interruption cutoffs; reordered events rebuild the projection.
+
+While tool arguments arrive, `input-streaming` parts expose a best-effort partial
+JSON value in `input`. A command string can appear and grow before its closing
+quote arrives. This is a display draft, not validated execution input.
+`tool-input-available` supplies the completed input. Interruption still discards
+streaming drafts even when their partial JSON is readable.
+
 ## Delivery semantics
 
 Model steps and tools run inside at-least-once A2 handlers. The returned-event
@@ -993,3 +1029,189 @@ continuation, but it cannot roll back that external effect.
 Configure a [production scheduler](/guides/production) exactly as for any other A2
 server. It recovers an incomplete generation attempt after the original
 serverless invocation disappears.
+
+## Inbox and turn controls
+
+This input protocol requires matching client and server versions. Start a new
+agent session when adopting it. Existing event logs remain readable; finish
+pending work with the runtime that created it before upgrading.
+
+`inputs.message(message)` places a message in the durable inbox. The controller
+admits one input at a time, freezes its revision, and adds it to `state.messages`.
+Pending inputs appear in `state.inbox.items`; they do not enter the model prompt.
+Each item has an `id`, `revision`, `message`, and `generate` flag.
+
+`state.active` identifies the logical turn, its frozen `input`, and its `phase`.
+The turn ID stays the same across model steps, tools, and a pause. Stop ends that
+turn and admits the next queued input when admission is running.
+
+```ts client/turn-controls.ts
+import { inputs, type AIState } from 'experimental-a2/ai'
+import type { UIMessage } from 'ai'
+
+export function editPending(message: UIMessage, expectedRevision: number) {
+  return inputs.queue.edit({ message, expectedRevision })
+}
+
+export function movePending(inputId: string, beforeId: string | null) {
+  return inputs.queue.move({ inputId, beforeId })
+}
+
+export function removePending(inputId: string) {
+  return inputs.queue.remove({ inputId })
+}
+
+export function sendPendingNow(state: AIState, inputId: string) {
+  return inputs.queue.sendNow({ inputId, turnId: state.active?.turnId ?? null })
+}
+
+export function stopCurrent(state: AIState) {
+  return state.active ? inputs.stop({ turnId: state.active.turnId }) : []
+}
+
+export function steerCurrent(state: AIState, message: UIMessage) {
+  return state.active
+    ? inputs.steer({ turnId: state.active.turnId, message })
+    : inputs.message(message)
+}
+
+export const pauseAfterTurn = () => inputs.pause({ when: 'after-turn' })
+export const pauseNow = () => inputs.pause({ when: 'now' })
+export const resume = () => inputs.resume()
+```
+
+Pass these arrays to `push(...events)` from `useSession()`, or to a server
+session's `append(...events)`. `beforeId: null` moves an item to the end.
+Removal only applies to pending input. It preserves the immutable log and keeps
+the input identity reserved, so replay cannot recreate a removed prompt.
+Removing an admitted input returns `already-active`; removing an absent or
+already removed input returns `not-found`.
+
+An edit that loses to admission returns `already-active`. Concurrent edits use
+`expectedRevision`; an outdated revision returns `revision-conflict`. A Stop
+aimed at a completed turn returns `stale-turn`.
+
+Steer validates the new input and places it ahead of the inbox in one decision.
+If the observed turn is still active, it finishes its current model response and
+the tools produced by that response before handing off. Parallel tools all
+settle, and steering runs before the old turn starts another model step.
+Text and tool results produced after the click remain in the conversation.
+`lastSeenIndex` is not used to truncate output for steering or Send now.
+
+When running work settles, unanswered approval or application-input requests
+are closed so the steering message can take over. The unapproved tool never
+executes. Provider-executed deferred results still count as running work when
+execution is authorized; a denied approval cannot block handoff.
+If the targeted turn has already finished, the input starts when idle or waits
+first behind a different active turn. Stop remains an immediate interruption.
+
+Steering preserves the admission gate: steering while paused does not resume
+execution. Duplicate input identities and closed sessions still reject the command.
+
+Send now uses the selected queued message's existing identity and latest
+accepted revision. It validates the selected input and the observed `turnId`,
+then moves the input first for the same handoff after the current step. Use
+`turnId: null` when no turn is active. If that expectation is stale, the command
+returns `stale-turn` without changing the queue.
+
+Send now preserves pause just like steering. A paused queue keeps the selected
+input first until Resume. It accepts user-role inputs, including passive user
+input, which it marks for generation. Non-user context returns `not-user-input`;
+a conflicting response identity returns `duplicate-input`. Unsaved editor text
+is separate from the accepted queue revision; save it before sending.
+
+A queued item's optional `afterStepOf` records the observed turn for steering.
+A matching active turn hands off after its current step.
+Editing preserves this intent; removing the item cancels it. Reordering the
+queue controls which input is admitted at that boundary.
+
+### Optimistic conversation display
+
+With `agent().reducer` or `createReducer()`, the client state is already optimistic.
+An idle, unpaused send appears in `state.messages` immediately, with
+`state.starting: true`. Another send waits in `state.inbox.items` while that
+optimistic turn or a confirmed turn is active. Paused sessions keep new sends in
+the inbox. Render the conversation and queue from `state`; `state.active` and
+execution controls remain server-confirmed.
+Steering appears immediately in `state.messages`, after the current response,
+and stays there while that step finishes. Send now has the same display when
+an active turn is targeted. These inputs are omitted from the client inbox,
+including while paused, but remain pending on the server until admission.
+Accepted steering survives receipt and hydration in this display. Multiple
+steering inputs appear in admission order, with the most recent first.
+No additional request, subscription, or component-owned queue is needed.
+
+```tsx app/agent/[sessionId]/conversation.tsx
+'use client'
+import { useSession } from '../session'
+
+export function Conversation() {
+  const { state } = useSession()
+  const { messages, inbox, starting } = state
+  const text = (message: (typeof messages)[number]) =>
+    message.parts.filter((part) => part.type === 'text').map((part) => part.text).join(' ')
+  return (
+    <section>
+      <div aria-label="Conversation">
+        {messages.map((message) => <p key={message.id}>{text(message)}</p>)}
+        {starting ? <p role="status">Starting…</p> : null}
+      </div>
+      <ol aria-label="Queued messages">
+        {inbox.items.map((item) => <li key={item.id}>{text(item.message)}</li>)}
+      </ol>
+    </section>
+  )
+}
+```
+
+The request echo keeps the optimistic display until `ai.control.decided` arrives. Accepted
+commits replace the confirmed base; rejected receipts remove the pending intent.
+A failed POST rolls back through the ordinary client optimistic overlay.
+Another client's accepted turn can move a speculative ordinary send back into
+the queue. Steering stays visible as a message while waiting behind that turn. `state.starting` describes an optimistic turn awaiting acceptance; it does
+not indicate a running model.
+
+Server `session.state(reducer)`, reducer folds, checkpoints, and model prompts
+contain confirmed messages and inbox entries, with `starting: false`. The client
+applies pending intent and displays accepted steering only when returning its
+state, after folding events.
+A custom reducer that calls `reduceAIState()` or nests `agent.reducer.fold()`
+uses that confirmed fold; it does not inherit the built-in reducer's client display.
+`state.pendingQueueCommands` tracks unresolved queue commands incrementally, so
+the client does not scan raw history. Revisions remain server-confirmed; keep
+repeat editing and Send now disabled while an edit for that item awaits its
+receipt. Moving and removing an item do not require a revision.
+
+### Persistence and application are separate
+
+`push()` confirms that the command is persisted. The `ai.control.decided` event
+confirms its application, using the submitted event's ID as `commandId`. Its
+`outcome` is `applied` or `rejected`; a rejection includes a `reason`.
+`state.receipt` exposes the most recent user command receipt. Applications that
+have several commands in flight correlate receipts through the existing event
+stream. A rejection is a normal result and does not spend a handler retry.
+
+### Pause and resume
+
+Pause after the turn closes inbox admission and lets the active turn finish.
+Pause now also cancels current model and tool execution. `state.active.phase`
+becomes `pausing` while old work settles, then `paused`. Resume reopens admission
+and resumes retained work before admitting another input.
+
+Resume starts a fresh model request from durable state. It does not resume the
+provider's old stream or undo completed tool effects. The controller retains
+accepted tool calls and records their outcomes while paused. A cancelled tool
+without a terminal result can execute again after resume; use the stable SDK
+`toolCallId` for external idempotency. No model or tool claim is held merely to
+keep a turn paused.
+
+The controller runs on a short lane. Each decision commits accepted facts,
+compact state changes, work requests, and its receipt atomically. Workers use
+separate claims, report lifecycle batches, and wait for acceptance at model-start
+and compaction boundaries. Token progress appends directly. Explicit AI history
+reads remain bounded to a relevant generation suffix; core checkpoint rebuilding
+can replay older events when a snapshot is missing.
+
+For a deferred provider-executed tool, trusted server code submits
+`inputs.toolResult(result)`. The controller checks its admitted call and records
+the result before continuing. Browser ingress rejects this server-only input.