experimental-a2 0.12.0 → 0.13.0

package/docs/guides/06-ai-agents.mdx

@@ -762,35 +762,111 @@ interaction that began elsewhere.
 
 ## Compact long conversations
 
-Compaction is optional application policy. A2 records both the decision and
-replacement messages, so later prompts remain explainable from history:
+Compaction is automatic for supported Gateway models. A2 uses the active model
+with the same instructions, tool definitions, and provider settings, and appends
+an internal user message asking for a summary. This preserves the request prefix
+for prompt caching when the provider has a matching cache entry.
 
 ```ts server/compacted.ts
-import type { UIMessage } from 'ai'
 import { createAgentServer } from 'experimental-a2/ai/server'
 import { assistant } from '../assistant'
 
 export const compactedAssistantServer = createAgentServer({
   agent: assistant,
   model: 'openai/gpt-5.6-terra',
-  compaction: {
-    shouldCompact: ({ messages }) => messages.length > 40,
-    compact: async ({ messages }) => {
-      const summary = {
-        id: crypto.randomUUID(),
-        role: 'user',
-        parts: [{ type: 'text', text: 'Summary of the earlier conversation.' }],
-      } satisfies UIMessage
-
-      return [summary, ...messages.slice(-10)]
-    },
-  },
 })
 ```
 
-The callback can call another model, create a deterministic summary, or retain
-selected messages. A2 owns only when the result enters the log and how later
-generations consume it.
+On first use of each Gateway model in a session, A2 includes
+`ai.model.metadata.requested` in the existing generation-start append. A separate
+handler fetches the public Gateway model catalog and records the selected limits
+in `ai.model.metadata.resolved`. It runs outside the AI turn lane. The first model
+request proceeds immediately; metadata that arrives afterward is available to
+later steps. Returning to a previously used model reuses its durable metadata.
+The catalog is also cached in memory for one hour across agent servers, with one
+shared in-flight request and a five-second timeout. A failed lookup follows A2's
+handler retry policy without failing the model generation. A model absent from
+the catalog is recorded as unavailable. A pending entry means no result has been
+recorded, including when the metadata handler exhausts its retry budget. A2 does
+not start a fresh lookup on every turn after exhaustion. Inspect the handler
+failure or use an explicit threshold in that case.
+
+The default threshold is 75% of the context window. A larger configured
+`generation.maxOutputTokens` lowers the threshold further to reserve that output
+allowance. This is an input-context policy, not a new output limit. Limits and
+metadata status are available in `state.modelMetadata`.
+
+`compaction: { thresholdTokens: 80_000 }` overrides discovery with an explicit
+positive safe integer. `compaction: { instructions: 'Preserve exact IDs.' }`
+adds guidance to the built-in summary prompt while keeping the derived threshold.
+`compaction: false` disables both compaction and metadata lookup. The `compaction`
+option can also be a synchronous resolver receiving the same
+`{ event, state, history, signal }` context as `model` and `instructions`. It
+returns `false`, automatic options, or a custom policy once per generation.
+Use this to select a session-specific threshold from already-recorded settings.
+Changes after resolution apply to a later model step. Disabling compaction
+preserves summaries that were already recorded. Custom provider
+models and custom global SDK providers need an explicit threshold. A custom
+`generate` callback keeps compaction disabled unless explicitly configured. Gateway
+fallback model lists also need an explicit threshold appropriate for every model
+they may use.
+
+The input estimate starts from serialized UTF-8 bytes divided by four, including
+model messages, instructions, and tool schemas. When a preceding model call has
+reported token usage for the same model and context, A2 uses that measurement as
+an anchor and estimates subsequent growth. It is not a provider tokenizer and
+does not precisely measure new image or audio tokens. A2 performs no remote token
+counting.
+
+On a cold session without recorded metadata or an explicit threshold, automatic
+compaction waits for discovery. An oversized first prompt is sent normally and
+its provider error becomes a generation failure. A2 does not silently truncate
+it or attempt automatic recovery from a context-limit error. Use an explicit threshold for
+applications that import a large initial conversation. The estimate is a
+compaction trigger, not a definitive overflow check. Provider context-limit
+errors during generation or summarization remain generation failures.
+
+Crossing the threshold adds one summary model request. Application generation
+callbacks, stream transforms, and tool hooks do not run for the summary, and it
+never executes local tools. Provider-executed tools, structured output, and forced
+tool choices disable implicit automatic compaction; explicitly configuring
+automatic compaction for those requests throws. Use a custom policy for them.
+An empty summary or a summary tool call fails the generation instead of replacing
+its context.
+
+A2 records `ai.compaction.requested` before the call and
+`ai.compaction.completed` after a successful summary. The completed event stores
+the summary, its usage, and the exact event frontier it covers. Later model
+requests receive the summary plus everything produced after that frontier,
+including later tool steps in the same assistant message. Queued user messages
+remain queued. Compaction waits until earlier tool calls have terminal results.
+The raw log and `state.messages` remain available in full.
+
+Custom `shouldCompact(context)` and `compact(context)` callbacks remain
+available for deterministic replacement messages or application-specific
+retention. Both receive the typed UI messages and `modelMessages`, the model-ready
+context including any existing summary. Custom generation callbacks also receive
+`modelMessages`; use it when forwarding context to a provider. A2 stores built-in
+summaries separately from application-typed UI messages, so a summary does not
+need your message metadata or schema.
+
+Render compaction as activity alongside the conversation:
+
+```tsx app/agent/compaction-status.tsx
+'use client'
+import { useSession } from './session'
+
+export function CompactionStatus() {
+  const { state } = useSession()
+  return state.compaction?.status === 'running' ? (
+    <p role="status">Compacting…</p>
+  ) : null
+}
+```
+
+A failed or interrupted generation restores the prior compaction state. The
+synthetic summary request is not a user turn or an assistant reply in
+`state.messages`.
 
 ## Extend the assembly
 
@@ -900,6 +976,12 @@ one atomic durable operation. Deterministic ids make scheduling and lifecycle
 appends effectively once, and A2 marks an incomplete model attempt
 `superseded` before starting its replacement.
 
+An expired claim or superseded attempt preserves its abort reason through model,
+resolver, compaction-policy, and tool calls. A2 retries that attempt without
+spending its failure budget. Explicit user interruption remains terminal.
+Streaming tool progress belongs to an execution attempt, so a retry can produce
+different preliminary output while preserving the tool-call id for idempotency.
+
 Provider calls and external tool side effects remain outside that transaction.
 A process can die after an external effect succeeds and before its handler
 completion commits, so recovery may run the call again. Give side-effecting

package/docs/reference/01-api.mdx

@@ -1149,7 +1149,7 @@ superseded owner cannot alter the projection. `terminalRequestIds` and
 and supersession fences across snapshots and recovery. They are optional
 snapshot-compatible fields with the shapes `Record<string, true>` and
 `Record<string, 'completed' | 'failed' | 'interrupted' | 'superseded'>`.
-Extension events are ignored. The default reducer name is `a2.ai.state.v8`.
+Extension events are ignored. The default reducer name is `a2.ai.state.v10`.
 
 ### `deriveUIMessages(history)` and `reduceAIState(state, event)`
 
@@ -1243,9 +1243,46 @@ support wait for their provider result instead of a local executor. An
 authenticated provider callback appends the terminal `ai.tool.result` through
 the trusted server session API; the browser push allowlist rejects it.
 
-`compaction` has `shouldCompact(context)` and `compact(context)` callbacks.
-When selected, both the request and the replacement messages enter the log.
-`progress` controls durable batching with `maxChunks` and `maxDelayMs`.
+`compaction` defaults to automatic for supported Gateway requests. It accepts
+`false` to disable compaction and discovery, `{ thresholdTokens?, instructions? }`
+to override automatic behavior, or the existing custom
+`{ shouldCompact(context), compact(context) }` policy. It also accepts a
+resolver `(context: AgentResolverContext) => CompactionOptions`.
+The resolver runs synchronously once per generation and must return a valid policy
+or `false`. Custom `shouldCompact` and `compact` callbacks can still be asynchronous.
+Static options are validated at construction; resolved options are validated
+before that generation starts. Each session uses its own resolved value. `thresholdTokens` is an
+optional positive safe integer. `instructions` adds to A2's internal summary
+prompt without replacing the agent's instructions.
+
+Without an explicit threshold, the first generation batches a metadata request
+into its existing start write. An independent handler reads the public Gateway
+catalog and records that session/model's limits. It does not delay model dispatch.
+The default threshold is 75% of the context window, lowered for a larger configured
+output allowance. A pending or unavailable lookup leaves the threshold unknown;
+the first request proceeds, and provider context-limit errors fail normally.
+`state.modelMetadata` retains each model's pending, resolved, or unavailable state.
+Pending also covers exhausted handler retries; no new request is added per turn.
+The catalog has a shared one-hour process cache and a five-second fetch timeout.
+
+Direct/custom providers, fallback model lists, and custom generators need explicit
+compaction configuration. Provider-executed tools, structured output, and forced
+tool choices skip implicit compaction and reject explicit automatic policies.
+Custom policies remain available for these configurations.
+
+The input estimate includes messages, tools, and instructions, and uses prior
+reported input usage to anchor later growth where applicable. It is not a provider
+tokenizer. Completed generations record `inputTokenEstimate` beside usage. The
+built-in summary adds one model request and preserves the request prefix for
+caching. It does not invoke application generation callbacks, transforms, or tool
+hooks. Both custom policy callbacks and `generate` receive `modelMessages`, the
+model-ready prompt including summaries, alongside typed UI `messages`.
+
+Built-in summaries remain separate from application-typed messages. Render
+`state.compaction?.status === 'running'` as current activity, and the durable
+`ai.compaction.*` events as historical timeline markers. The conversation in
+`state.messages` remains intact. `progress` controls durable batching with
+`maxChunks` and `maxDelayMs`.
 
 This entry point is server-only and resolves to a throwing browser stub.
 

package/examples/playground/app/agent/[agentId]/agent-client.tsx

@@ -10,6 +10,9 @@ import { inputs } from 'experimental-a2/ai'
 import { ActivityFeed } from '@/app/components/activity-feed'
 import { ConnectionPill } from '@/app/components/connection-pill'
 import { useSession } from '../session'
+import { compactionTimeline } from '../compaction-timeline'
+import { CompactionEvent } from './compaction-event'
+import { CompactionPanel } from './compaction-panel'
 
 type MessagePart = UIMessage['parts'][number]
 type ToolPartView = {
@@ -169,10 +172,12 @@ export function AgentClient({ agentId }: { agentId: string }): ReactNode {
   const [responding, setResponding] = useState(false)
 
   const generating = state.status === 'generating'
+  const compacting = state.compaction?.status === 'running'
   const canSend = state.status === 'idle' || state.status === 'failed'
+  const timeline = compactionTimeline({ state, events })
 
   useEffect(() => {
-    if (canSend) promptRef.current?.focus()
+    if (canSend) promptRef.current?.focus({ preventScroll: true })
   }, [canSend])
 
   const send = async (event: FormEvent): Promise<void> => {
@@ -250,32 +255,43 @@ export function AgentClient({ agentId }: { agentId: string }): ReactNode {
             state.status === 'waiting' ? ' waiting' : ''
           }${state.status === 'failed' ? ' cut' : ''}`}
         >
-          {STATUS_LABEL[state.status]}
+          {compacting ? 'compacting…' : STATUS_LABEL[state.status]}
         </span>
       </header>
 
+      <CompactionPanel agentId={agentId} state={state} events={events} />
+
       <section className="agent-conversation" aria-label="Conversation">
         <div className="agent-transcript" aria-live="polite">
-          {state.messages.length === 0 ? (
+          {timeline.length === 0 ? (
             <p className="agent-empty">
               Ask the agent to investigate an incident, run a Bash command, or
               set a reminder for itself. Its tools and decisions will appear
               here as they enter the log.
             </p>
           ) : (
-            state.messages.map((message) => (
-              <AgentMessage
-                key={message.id}
-                message={message}
-                thinking={
-                  generating &&
-                  message.role === 'assistant' &&
-                  !hasVisibleParts(message)
-                }
-              />
-            ))
+            timeline.map((entry) =>
+              entry.type === 'compaction' ? (
+                <CompactionEvent key={entry.id} entry={entry} />
+              ) : (
+                <AgentMessage
+                  key={entry.id}
+                  message={entry.message}
+                  thinking={
+                    generating &&
+                    !compacting &&
+                    entry.lastSegment &&
+                    entry.message.id ===
+                      state.activeGeneration?.responseMessageId &&
+                    !hasVisibleParts(entry.message)
+                  }
+                />
+              ),
+            )
           )}
-          {generating && state.messages.at(-1)?.role === 'user' ? (
+          {generating &&
+          !compacting &&
+          state.messages.at(-1)?.role === 'user' ? (
             <div className="agent-message assistant">
               <span className="agent-role">Agent</span>
               <div>
@@ -340,7 +356,6 @@ export function AgentClient({ agentId }: { agentId: string }): ReactNode {
             Message
             <textarea
               ref={promptRef}
-              autoFocus
               rows={3}
               value={prompt}
               onChange={(event) => setPrompt(event.target.value)}

package/examples/playground/app/agent/[agentId]/compaction/route.ts

@@ -0,0 +1,14 @@
+import { updateCompactionSettings } from '../../compaction-settings'
+import { agentServer } from '../../server'
+
+export async function POST(
+  request: Request,
+  context: { params: Promise<{ agentId: string }> },
+): Promise<Response> {
+  const { agentId } = await context.params
+  return updateCompactionSettings({
+    request,
+    sessionId: agentId,
+    append: (event) => agentServer.session(agentId).append(event),
+  })
+}

package/examples/playground/app/agent/[agentId]/compaction-event.tsx

@@ -0,0 +1,38 @@
+import type { ReactNode } from 'react'
+import { formatActivityTime } from '@/app/components/activity-feed'
+import type { CompactionEntry } from '../compaction-timeline'
+
+export function CompactionEvent({
+  entry,
+}: {
+  entry: CompactionEntry
+}): ReactNode {
+  const { event, running, outcome } = entry
+  const completed = event.type === 'ai.compaction.completed'
+  const label = completed
+    ? 'Context compacted'
+    : running
+      ? 'Compacting…'
+      : 'Compaction did not complete'
+
+  return (
+    <details className="agent-compaction-event">
+      <summary>
+        <span role={running ? 'status' : undefined}>{label}</span>
+        <time dateTime={event.createdAt.toISOString()}>
+          {formatActivityTime(event.createdAt)}
+        </time>
+      </summary>
+      <p className="hint">
+        Event #{event.index} ·{' '}
+        {event.payload.throughIndex === undefined
+          ? `Through message ${event.payload.throughMessageId}`
+          : `Summarized through event #${event.payload.throughIndex}`}
+        {outcome === undefined ? null : ` · Generation ${outcome}`}
+      </p>
+      {completed && event.payload.summary ? (
+        <pre>{event.payload.summary}</pre>
+      ) : null}
+    </details>
+  )
+}

package/examples/playground/app/agent/[agentId]/compaction-panel.tsx

@@ -0,0 +1,294 @@
+'use client'
+
+import type { FormEvent, ReactNode } from 'react'
+import { useEffect, useId, useRef, useState } from 'react'
+import type { AIState } from 'experimental-a2/ai'
+import { formatActivityTime } from '@/app/components/activity-feed'
+import {
+  agentModel,
+  compactionSettingsSchema,
+  latestCompactionSettings,
+  type AgentEvent,
+  type CompactionSettings,
+} from '../model'
+
+const number = new Intl.NumberFormat('en-US')
+const tokens = (value: number | undefined): string =>
+  value === undefined
+    ? 'Not available'
+    : `${number.format(value)} ${value === 1 ? 'token' : 'tokens'}`
+
+function SettingsForm({
+  agentId,
+  settings,
+  closed,
+}: {
+  agentId: string
+  settings: CompactionSettings
+  closed: boolean
+}): ReactNode {
+  const modeId = useId()
+  const thresholdId = useId()
+  const [mode, setMode] = useState(settings.mode)
+  const [threshold, setThreshold] = useState(
+    settings.mode === 'manual' ? String(settings.thresholdTokens) : '',
+  )
+  const [saving, setSaving] = useState(false)
+  const [error, setError] = useState<string | null>(null)
+  const [notice, setNotice] = useState<string | null>(null)
+  const pending = useRef<AbortController | null>(null)
+
+  useEffect(() => () => pending.current?.abort(), [])
+
+  const save = async (next: CompactionSettings): Promise<void> => {
+    pending.current?.abort()
+    const controller = new AbortController()
+    pending.current = controller
+    setSaving(true)
+    setError(null)
+    setNotice(null)
+    try {
+      const response = await fetch(
+        `/agent/${encodeURIComponent(agentId)}/compaction`,
+        {
+          method: 'POST',
+          headers: { 'content-type': 'application/json' },
+          body: JSON.stringify({ id: crypto.randomUUID(), settings: next }),
+          signal: controller.signal,
+        },
+      )
+      if (!response.ok)
+        throw new Error('Could not save compaction settings. Try again.')
+      setNotice('Saved. The live feed will confirm the latest settings.')
+    } catch (cause) {
+      if (!controller.signal.aborted)
+        setError(cause instanceof Error ? cause.message : String(cause))
+    } finally {
+      if (pending.current === controller) {
+        pending.current = null
+        setSaving(false)
+      }
+    }
+  }
+
+  const submit = (event: FormEvent): void => {
+    event.preventDefault()
+    const parsed = compactionSettingsSchema.safeParse(
+      mode === 'manual'
+        ? { mode, thresholdTokens: Number(threshold) }
+        : { mode },
+    )
+    if (!parsed.success) {
+      setError('Enter a positive whole-number threshold.')
+      return
+    }
+    void save(parsed.data)
+  }
+
+  return (
+    <form className="compaction-form" onSubmit={submit}>
+      <fieldset disabled={saving || closed}>
+        <div className="compaction-fields">
+          <label htmlFor={modeId}>
+            Mode
+            <select
+              id={modeId}
+              value={mode}
+              onChange={(event) =>
+                setMode(event.target.value as CompactionSettings['mode'])
+              }
+            >
+              <option value="automatic">Automatic</option>
+              <option value="manual">Custom threshold</option>
+              <option value="disabled">Disable further compaction</option>
+            </select>
+          </label>
+          <label htmlFor={thresholdId}>
+            Threshold in tokens
+            <input
+              id={thresholdId}
+              type="number"
+              min={1}
+              max={Number.MAX_SAFE_INTEGER}
+              step={1}
+              required={mode === 'manual'}
+              disabled={mode !== 'manual'}
+              value={threshold}
+              onChange={(event) => setThreshold(event.target.value)}
+              placeholder="Automatic"
+            />
+          </label>
+        </div>
+        <div className="compaction-actions">
+          <button type="submit">{saving ? 'Saving…' : 'Apply settings'}</button>
+          <button
+            type="button"
+            className="secondary"
+            onClick={() => void save({ mode: 'manual', thresholdTokens: 1 })}
+          >
+            Set to 1 token
+          </button>
+          <button
+            type="button"
+            className="secondary"
+            onClick={() => void save({ mode: 'automatic' })}
+          >
+            Reset to automatic
+          </button>
+        </div>
+      </fieldset>
+      <p className="hint">
+        Changes apply to the next step that has not started and do not start a
+        new turn. Send a message to test. At 1 token, every eligible step
+        compacts until you reset it.
+      </p>
+      {mode === 'disabled' ? (
+        <p className="hint">
+          Existing summaries remain in use. This stops future compaction.
+        </p>
+      ) : null}
+      {notice ? (
+        <p role="status" className="hint">
+          {notice}
+        </p>
+      ) : null}
+      {error ? (
+        <p role="alert" className="error">
+          {error}
+        </p>
+      ) : null}
+    </form>
+  )
+}
+
+export function CompactionPanel({
+  agentId,
+  state,
+  events,
+}: {
+  agentId: string
+  state: AIState
+  events: readonly AgentEvent[]
+}): ReactNode {
+  const { settings, id } = latestCompactionSettings(events)
+  const metadata = state.modelMetadata[agentModel]
+  const limits = metadata?.status === 'resolved' ? metadata.limits : undefined
+  const automaticThreshold =
+    limits === undefined ? undefined : Math.floor(limits.contextWindow * 0.75)
+  const effectiveThreshold =
+    settings.mode === 'manual' ? settings.thresholdTokens : automaticThreshold
+  const lastStep = events.findLast(
+    (event) => event.type === 'ai.generation.completed',
+  )
+  const completed = events
+    .filter((event) => event.type === 'ai.compaction.completed')
+    .toReversed()
+  const compacting = state.compaction?.status === 'running'
+
+  return (
+    <details className="agent-compaction">
+      <summary>
+        <span className="compaction-chevron" aria-hidden="true">
+          ›
+        </span>
+        <strong>Compaction</strong>
+        <span className={`badge${compacting ? ' live' : ''}`}>
+          {compacting
+            ? 'Compacting…'
+            : settings.mode === 'disabled'
+              ? 'Disabled'
+              : settings.mode === 'manual'
+                ? `Custom · ${tokens(settings.thresholdTokens)}`
+                : 'Automatic'}
+        </span>
+      </summary>
+      <div className="compaction-controls">
+        <p className="hint">
+          Model: <code>{agentModel}</code>. Limits are saved for this session.
+        </p>
+        <dl className="compaction-metrics">
+          <div>
+            <dt>Context window</dt>
+            <dd>{tokens(limits?.contextWindow)}</dd>
+          </div>
+          <div>
+            <dt>Model output ceiling</dt>
+            <dd>{tokens(limits?.maxOutputTokens)}</dd>
+          </div>
+          <div>
+            <dt>Automatic trigger · 75%</dt>
+            <dd>{tokens(automaticThreshold)}</dd>
+          </div>
+          <div>
+            <dt>Threshold for next step</dt>
+            <dd>
+              {settings.mode === 'disabled'
+                ? 'Disabled'
+                : tokens(effectiveThreshold)}
+            </dd>
+          </div>
+          <div>
+            <dt>Last step input · reported</dt>
+            <dd>{tokens(lastStep?.payload.usage?.inputTokens)}</dd>
+          </div>
+          <div>
+            <dt>Last step input · estimated</dt>
+            <dd>{tokens(lastStep?.payload.inputTokenEstimate)}</dd>
+          </div>
+        </dl>
+        {limits === undefined ? (
+          <p className="hint">
+            {metadata?.status === 'unavailable'
+              ? 'Gateway did not report limits for this model. You can set a custom threshold.'
+              : metadata?.status === 'pending'
+                ? 'Model limits have been requested but are not resolved yet. The agent can continue.'
+                : 'Send a message to discover the model limits, or set a custom threshold first.'}
+          </p>
+        ) : null}
+        <SettingsForm
+          key={id}
+          agentId={agentId}
+          settings={settings}
+          closed={state.status === 'closed'}
+        />
+        <details className="compaction-history">
+          <summary>
+            Completed compactions <span>{completed.length}</span>
+          </summary>
+          {completed.length === 0 ? (
+            <p className="hint">
+              No completed compactions yet. Each completion will stay here after
+              reload.
+            </p>
+          ) : (
+            <ol>
+              {completed.map((event) => (
+                <li key={event.id}>
+                  <div className="compaction-marker">
+                    <strong>Context compacted · #{event.index}</strong>
+                    <time dateTime={event.createdAt.toISOString()}>
+                      {formatActivityTime(event.createdAt)}
+                    </time>
+                  </div>
+                  <p className="hint">
+                    {event.payload.throughIndex === undefined
+                      ? `Through message ${event.payload.throughMessageId}`
+                      : `Summarized through event #${event.payload.throughIndex}`}{' '}
+                    · {tokens(event.payload.usage?.inputTokens)} in ·{' '}
+                    {tokens(event.payload.usage?.outputTokens)} out
+                  </p>
+                  {event.payload.summary ? (
+                    <details>
+                      <summary>View summary</summary>
+                      <pre>{event.payload.summary}</pre>
+                    </details>
+                  ) : null}
+                </li>
+              ))}
+            </ol>
+          )}
+        </details>
+      </div>
+    </details>
+  )
+}