@coder/ai-sdk-agent 0.6.1 → 0.6.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -214,7 +214,9 @@ conversation with server‑side history). `agent.chatId` is the current chat id.
214
214
  - `agent.interrupt({ signal? })` — interrupt an in‑flight generation.
215
215
  - `agent.archive({ signal? })` — archive the underlying chat (cleanup; see [Cleanup](#cleanup)).
216
216
  - `agent.listModels()` — list the deployment's model configs, so you don't have to guess the `model` hint.
217
- - Resume a prior chat: `new CoderAgent({ …, chatId: "…" })`.
217
+ - Resume a prior chat: `new CoderAgent({ …, chatId: "…" })` — see
218
+ [Durable workflows](#durable-workflows-persist-resume-recover) for the full
219
+ resumption how‑to.
218
220
 
219
221
  Interrupting is asynchronous on the server: `interrupt()` resolves as soon as the
220
222
  interrupt is acknowledged, and the run keeps winding down for a few seconds
@@ -322,19 +324,19 @@ for (const ev of events) {
322
324
  `timestamp` (`Date.now()` at observation — comparable to server‑side timestamps
323
325
  such as a message's `created_at`, for delivery‑lag measurements):
324
326
 
325
- | event | when | payload (besides `timestamp`) |
326
- | ---------------- | --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
327
- | `http:request` | a REST request is sent | `id` (correlates the pair), `method`, `path` |
328
- | `http:response` | response headers arrive (incl. non‑2xx, `ok:false`) | `id`, `method`, `path`, `status`, `ok`, `durationMs` |
329
- | `http:error` | the fetch itself rejects (network failure, abort) | `id`, `method`, `path`, `message`, `durationMs` |
330
- | `ws:dial` | a stream connection attempt starts | `chatId`, `attempt` (1‑based, increments per redial), `url` |
331
- | `ws:open` | the WebSocket handshake completes | `chatId`, `attempt` |
332
- | `ws:event` | a decoded stream event arrives | `chatId`, `attempt`, `event` (the decoded `ChatStreamEvent`, by reference — don't mutate) |
333
- | `ws:close` | the connection ends (exactly one per dial) | `chatId`, `attempt`, `code`/`reason` when the server/network closed it; absent when the reader closed it (settle, teardown, redial) |
334
- | `ws:error` | a socket error or unparseable frame | `chatId`, `attempt`, `message` |
335
- | `ws:redial` | a dropped connection is about to be redialed | `chatId`, `attempt` (the ended connection), `consecutiveFailures`, `maxConsecutiveFailures`, `backoffMs` |
336
- | `segment:start` | a turn segment (one model round‑trip) starts | `segment` (1‑based per model instance), `chatId` (absent before the first turn creates the chat) |
337
- | `segment:settle` | the segment ends (exactly one per start) | `segment`, `chatId`, and: `status` + `finishReason` on a clean settle, `error` (`{name, message}`, plus `status` if the run still settled terminally) on failure, neither on a teardown |
327
+ | event | when | payload (besides `timestamp`) |
328
+ | ---------------- | --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
329
+ | `http:request` | a REST request is sent | `id` (correlates the pair), `method`, `path` |
330
+ | `http:response` | response headers arrive (incl. non‑2xx, `ok:false`) | `id`, `method`, `path`, `status`, `ok`, `durationMs` |
331
+ | `http:error` | the fetch itself rejects (network failure, abort) | `id`, `method`, `path`, `message`, `durationMs` |
332
+ | `ws:dial` | a stream connection attempt starts | `chatId`, `attempt` (1‑based, increments per redial), `url` |
333
+ | `ws:open` | the WebSocket handshake completes | `chatId`, `attempt` |
334
+ | `ws:event` | a decoded stream event arrives | `chatId`, `attempt`, `event` (the decoded `ChatStreamEvent`, by reference — don't mutate) |
335
+ | `ws:close` | the connection ends (exactly one per dial) | `chatId`, `attempt`, `code`/`reason` when the server/network closed it; absent when the reader closed it (settle, teardown, redial) |
336
+ | `ws:error` | a socket error or unparseable frame | `chatId`, `attempt`, `message` |
337
+ | `ws:redial` | a dropped connection is about to be redialed | `chatId`, `attempt` (the ended connection), `consecutiveFailures`, `maxConsecutiveFailures`, `backoffMs` |
338
+ | `segment:start` | a turn segment (one model round‑trip) starts | `segment` (1‑based per model instance), `chatId` (absent before the first turn creates the chat) |
339
+ | `segment:settle` | the segment ends (exactly one per start) | `segment`, `chatId`, `durationMs`, and: `status` + `finishReason` on a clean settle, `error` (`{name, message}`, plus `status` if the run still settled terminally) on failure, neither on a teardown |
338
340
 
339
341
  Semantics worth knowing:
340
342
 
@@ -668,16 +670,154 @@ credentials — pass them alongside `client` if you construct one yourself.
668
670
  ## Workspaces & quota
669
671
 
670
672
  A `CoderAgent` is one server‑side chat, and — depending on its configuration and
671
- the deployment — a chat may provision a **Coder workspace** to run its tools. A
672
- deployment caps how many workspaces an account may run at once, so **N agents
673
- running concurrently can need N free workspace slots.** Past the cap, a turn can
674
- sit unscheduled and never settle. This is the most important operational fact when
675
- running many agents at once:
676
-
677
- - Keep your own concurrency below the deployment's workspace limit.
678
- - Set `requestTimeoutMs` so an unschedulable turn fails loudly instead of hanging.
679
- - `archive()` / `await using` each agent so finished chats stop holding resources.
680
- - For steps that don't need server‑side tools, prefer the provider — it never touches a workspace.
673
+ the deployment — a chat may be backed by a **Coder workspace** that runs its
674
+ tools. Workspaces are the scarce resource: a deployment budgets how many an
675
+ account may run at once, so **N agents running concurrently can need N
676
+ schedulable workspaces.** Past that bound, a turn can sit unscheduled and never
677
+ settle. This section is the operational guide for running fleets of agents:
678
+ how the binding works, how to size concurrency, what to clean up, and how to
679
+ diagnose a chat that is stuck.
680
+
681
+ ### How a chat binds to a workspace
682
+
683
+ - **One chat, at most one workspace, fixed at creation.** `workspaceId` is sent
684
+ as `workspace_id` when the chat is created; nothing can rebind it afterwards —
685
+ message and update requests carry no workspace field. To move work to another
686
+ workspace, start a new agent/chat.
687
+ - **This SDK never provisions workspaces.** A `workspaceId` you pass must be an
688
+ existing workspace (provision one with
689
+ [`@coder/ai-sdk-sandbox`](../sandbox)'s `ensureCoderWorkspace`, the CLI, or
690
+ the v2 API). A chat created _without_ `workspaceId` can still come back
691
+ workspace‑backed — deployments may assign one server‑side; the SDK reads the
692
+ created chat's `workspace_id` and treats the chat as workspace‑backed from
693
+ then on (which matters for retry ownership, below).
694
+ - **Chat cleanup does not release the workspace.** `archive()` soft‑hides the
695
+ chat only; the workspace keeps running until template autostop or an explicit
696
+ stop — and stopping releases only stop‑scoped quota; persistent resources
697
+ (disks, volumes) keep consuming their cost until the workspace is deleted.
698
+
699
+ ### Sizing a fleet
700
+
701
+ The structural rule: **workspaces that must be running concurrently ≤
702
+ schedulable workspaces.** With one workspace per chat — the common fleet shape,
703
+ and what a deployment that auto‑assigns workspaces produces — that means
704
+ concurrent chats ≤ schedulable workspaces. Chats explicitly bound to a shared
705
+ `workspaceId` count that workspace once, so their concurrency is not
706
+ quota‑bound — at the price of sharing one filesystem and tool environment,
707
+ which is only acceptable within a single tenant / trust boundary.
708
+ What counts as "schedulable" is a deployment property, not an SDK knob —
709
+ whichever of these binds first:
710
+
711
+ - **Workspace quota (premium deployments).** Templates declare per‑resource
712
+ costs; a user's budget is the sum of their groups' quota allowances, enforced
713
+ when a workspace build starts or stops. A start that would exceed the budget
714
+ **fails the build** (error code `INSUFFICIENT_QUOTA`, "insufficient quota"),
715
+ so the turn never gets its workspace — see
716
+ [resource quotas](https://coder.com/docs/admin/users/quotas). Note that a
717
+ _stopped_ workspace typically still consumes its persistent resources' cost,
718
+ so a fleet that only ever stops (never deletes) scratch workspaces converges
719
+ on a full budget.
720
+ - **Infrastructure.** Without quotas there is no per‑user workspace limit by
721
+ default ([workspace lifecycle](https://coder.com/docs/user-guides/workspace-lifecycle)) —
722
+ the bound is provisioner throughput and cluster capacity, and exceeding it
723
+ looks like slow or failing builds rather than a crisp quota error.
724
+
725
+ Practical sizing:
726
+
727
+ - Read headroom before fanning out:
728
+ `GET /api/v2/organizations/{org}/members/{user}/workspace-quota` returns
729
+ `{ "credits_consumed": …, "budget": … }`. Quota is denominated in credits,
730
+ not slots: admit another workspace only while `budget − credits_consumed`
731
+ covers _that workspace's_ cost (the sum of its template's `daily_cost`
732
+ declarations). "Free slots = headroom ÷ cost" only holds for a homogeneous
733
+ fleet on one template; with mixed templates, size against each planned
734
+ workspace's own cost.
735
+ - Keep fan‑out width within that headroom and queue the rest client‑side — an
736
+ unschedulable turn does not queue usefully on the server (see
737
+ [Preventing stuck turns](#preventing-stuck-turns)).
738
+ - Reuse one bound workspace across sequential turns and sessions instead of
739
+ provisioning per request — the workspace is the expensive part, the chat is
740
+ cheap. Reuse only **within one tenant / trust boundary**: workspace‑bound
741
+ agents have file and shell tools, so a reused filesystem carries one
742
+ session's artifacts (and secrets) into the next — provision per tenant, or
743
+ securely reset a workspace before reassigning it.
744
+ - Steps that don't need server‑side tools belong on the
745
+ [provider](../provider) — it never touches a workspace.
746
+
747
+ ### Autostop & cleanup
748
+
749
+ Two lifetimes to manage, separately:
750
+
751
+ - **Chats** — `archive()` / `await using` every agent ([Cleanup](#cleanup)), or
752
+ finished chats keep holding server resources.
753
+ - **Workspaces** — rely on template‑level scheduling rather than manual
754
+ hygiene:
755
+ - **Autostop TTL.** Give fleet templates a default TTL long enough to survive
756
+ a normal session (including idle gaps between turns), short enough that a
757
+ leaked workspace stops burning running‑cost within hours — without
758
+ autostop, a leaked workspace pins its full quota until someone notices.
759
+ Note that stopping only releases the quota of resources that go away on
760
+ stop; persistent resources (disks, volumes) keep consuming their
761
+ `daily_cost`, so a scratch fleet that only ever stops still converges on a
762
+ full budget — pair the TTL with dormancy auto‑deletion or explicit
763
+ deletion. With [`@coder/ai-sdk-sandbox`](../sandbox), `stopAfter: "8h"`
764
+ sets the TTL (`ttl_ms`) at creation.
765
+ - **Activity bump** (default 1 h) extends a running workspace's deadline when
766
+ Coder detects sessions — check
767
+ [what counts as activity](https://coder.com/docs/user-guides/workspace-scheduling)
768
+ before assuming an agent's server‑side tool use keeps its workspace alive.
769
+ - **Dormancy / failure cleanup** reap abandoned and repeatedly‑failing
770
+ workspaces automatically — see
771
+ [template scheduling](https://coder.com/docs/admin/templates/managing-templates/schedule).
772
+
773
+ ### Preventing stuck turns
774
+
775
+ The signature failure mode of an over‑committed fleet: the chat is created, the
776
+ stream opens, and then — nothing. There is **no distinct "quota exceeded" error
777
+ kind on the chat stream**; a chat whose workspace can't be scheduled surfaces
778
+ either a generic turn error or, worse, a chat that sits in a non‑terminal
779
+ status indefinitely. Defend in this order:
780
+
781
+ 1. **Set `requestTimeoutMs` — always, in fleets.** It is unbounded by default.
782
+ On expiry the call rejects with a `CoderChatError` (`kind: "timeout"`,
783
+ `retryable: true`) so your dispatcher gets its slot back, and a server‑side
784
+ interrupt is fired **best‑effort** — fire‑and‑forget, and unreachable when
785
+ the timeout lands before chat creation returned an id — so the run usually
786
+ stops, but is not guaranteed released ([Timeouts](#timeouts--cancellation)).
787
+ Pair it with reconciliation: periodically sweep for non‑terminal chats older
788
+ than your budget and interrupt/archive them.
789
+ 2. **Cap total wall‑clock** with `abortSignal: AbortSignal.timeout(…)` —
790
+ `requestTimeoutMs` bounds each segment, not a whole multi‑tool call.
791
+ 3. **Own the retries.** Workspace‑backed turns are **never auto‑retried**: the
792
+ SDK marks a stream‑loss error `isRetryable` only when the failed turn
793
+ created its chat _and_ had no workspace, no MCP servers, and no fresh
794
+ uploads ([Timeouts](#timeouts--cancellation)). Your dispatcher owns the
795
+ retry decision — and should re‑check quota headroom first, or it re‑queues
796
+ into the same wall.
797
+ 4. **Watch the fleet.** [`watchChats`](#watching-chats) yields status changes
798
+ for every chat **visible to the authenticated user** — with chats spread
799
+ across per‑tenant credentials, run one watcher per identity (or watch with
800
+ a credential that can see them all). Alert on chats sitting in a
801
+ non‑terminal status (e.g. `pending`) longer than your `requestTimeoutMs`.
802
+ Events alone aren't a complete monitor: reconnects resubscribe fresh (no
803
+ cursor or replay), so a chat already stuck before the watcher started — or
804
+ one that transitioned during a gap — never emits an event to start your
805
+ timer from. Seed and periodically reconcile against a chat listing
806
+ (`GET /api/experimental/chats`) instead of trusting the event stream alone.
807
+
808
+ ### Troubleshooting: unschedulable & stuck chats
809
+
810
+ Wire [`onTransportEvent`](#observability) into fleet telemetry — the event
811
+ sequence pinpoints _where_ a turn died:
812
+
813
+ | symptom (transport events) | likely cause | fix |
814
+ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
815
+ | `segment:start` and `ws:open` fired, initial status events, then silence — no `message_part`, no `segment:settle` | if the chat's status is `pending` (or its workspace build is `pending`/`failed`): the workspace can't be scheduled — quota exhausted, no free provisioner, failing template. If the status is `running` with a healthy build, it may just be a slow model/tool step — silence alone doesn't prove scheduling | check quota headroom (`workspace-quota` endpoint) and the workspace's build — a quota failure logs `INSUFFICIENT_QUOTA`; reclaim enough _credits_ for the planned build (stopping helps only for stop‑scoped costs — delete workspaces to reclaim persistent costs, or raise the group allowance) and set `requestTimeoutMs` so this fails loudly next time |
816
+ | `segment:settle` carries `error: { name: "CoderChatError", message: "…requestTimeoutMs budget…" }` | the per‑segment bound expired — wedged server, slow model, or an unschedulable workspace | inspect the workspace via the v2 API/UI: a `pending`/`failed` build means the row above; `running` means the turn was genuinely slow — raise `requestTimeoutMs` for long tool work |
817
+ | repeated `ws:redial` with `consecutiveFailures` climbing toward `maxConsecutiveFailures` (5; backoff 1 s → 2 s → 4 s → 8 s, ≈15 s of redialing without forward progress), then a `CoderStreamError` | the network path to the deployment is failing — not workspace scheduling (the server keeps generating through short gaps) | fix connectivity; mind retry ownership above — on workspace‑backed chats the error is `isRetryable: false`, so the replay decision is yours |
818
+ | `segment:settle` with `status: "error"` and an `error` payload | the turn failed server‑side — a provider/model error, a tool failure, or a scheduling/build failure that terminated the turn instead of leaving it pending | the settle event's `error` carries only `{ name, message }` — branch on `kind` / `retryable` / `statusCode` by catching the thrown `CoderChatError`: at the `generate()` call site, or — for `stream()` — around stream consumption, since mid‑stream failures surface on the stream, not from `await agent.stream()` ([Handling errors](#handling-errors)); check the workspace build state to rule scheduling in or out |
819
+ | turn settled `status: "requires_action"`, follow‑up messages queue forever | the loop ended on an unanswered client tool call | submit the stranded results or interrupt — see rule 4 under [Structured output](#structured-output); if a crash left the pause behind, reconcile effects first ([Make client tools crash-safe](#make-client-tools-crash-safe)) |
820
+ | `archive()` keeps returning 409 and rethrows after ~15 s | the chat never settled server‑side — usually a stuck run still holding its workspace | `interrupt()` with a bounded signal, then re‑archive; if the run stays wedged, stop the workspace itself |
681
821
 
682
822
  ## Configuration
683
823
 
@@ -730,27 +870,710 @@ CoderAgent (implements ai.Agent)
730
870
  revision that appends to an earlier message yields the appended suffix (rewrites that
731
871
  can't be expressed as deltas are safely suppressed).
732
872
 
733
- ## Durable workflows (Vercel Workflow, step functions, …)
734
-
735
- `CoderAgent` talks to Coder over its own REST + WebSocket client, so it can't ride
736
- a `fetch`‑shim durability layer — each turn must run **inside** a durable step. A
737
- few rules keep it well‑behaved across replays:
738
-
739
- - **One turn per step.** Create the agent, run a single `generate()` (not
740
- `stream()`, so the checkpointed value is the finished result), return.
741
- - **Don't persist the instance across steps.** Persist `agent.chatId` (a string)
742
- and resume with `new CoderAgent({ …, chatId })` in the next step. Never persist
743
- or log the token — read it from the environment in each step.
744
- - **Clean up in the step.** `await using` the agent (or `await agent.archive()` in
745
- a `finally`) so a step that returns early doesn't abandon the chat.
746
- - **Bound each step.** Set `requestTimeoutMs` so a wedged turn fails the step (and
747
- lets the workflow retry) instead of hanging the whole run.
748
- - **Mind concurrency vs. workspaces.** Keep fan‑out width under the deployment's
749
- workspace cap — see [Workspaces & quota](#workspaces--quota).
750
- - **Use the provider for pure steps.** Steps that don't need server‑side tools
751
- (plan / extract / synthesize) are cheaper and natively structured through
873
+ ## Durable workflows: persist, resume, recover
874
+
875
+ How to run **one agent session across process boundaries** — queue jobs,
876
+ durable‑workflow steps (Vercel Workflow, step functions, Temporal, …), cron
877
+ ticks — and survive the crashes, stream drops, and timeouts in between. The
878
+ mechanics this how‑to leans on are specified in [Sessions](#sessions),
879
+ [Timeouts & cancellation](#timeouts--cancellation),
880
+ [Handling errors](#handling-errors), and [Observability](#observability).
881
+
882
+ Two facts make the pattern work:
883
+
884
+ - **All chat state lives on the Coder server.** Messages, tool activity, the
885
+ run itself — none of it is in your process. The only durable thing a
886
+ workflow has to carry between steps is **`agent.chatId`: a string.**
887
+ - **`CoderAgent` can't ride a `fetch`‑shim durability layer** (it talks REST +
888
+ WebSocket through its own client), so each turn runs **inside** a durable
889
+ step — and the checkpointed chat id is the thread between steps.
890
+
891
+ Running example: a pipeline whose steps each run as their own job — possibly
892
+ on another machine, hours apart, retried after failures.
893
+
894
+ ### Shape each step: one turn, then checkpoint
895
+
896
+ ```ts
897
+ import { CoderAgent, type CoderTransportEvent } from "@coder/ai-sdk-agent";
898
+
899
+ // Any durable KV your engine gives you: step state, a job row, a DB table.
900
+ declare const checkpoints: {
901
+ get(workflowId: string): Promise<string | undefined>;
902
+ set(workflowId: string, chatId: string): Promise<void>;
903
+ };
904
+
905
+ export async function runTurn(workflowId: string, prompt: string): Promise<string> {
906
+ const agent = new CoderAgent({
907
+ baseUrl: process.env.CODER_URL!,
908
+ token: process.env.CODER_SESSION_TOKEN!, // read per step — never checkpoint or log it
909
+ organizationId: process.env.CODER_ORG_ID!,
910
+ chatId: await checkpoints.get(workflowId), // undefined on the first step → the turn creates the chat
911
+ requestTimeoutMs: 300_000, // always bound workflow steps — see below
912
+ onTransportEvent: observe, // per-step telemetry — see below
913
+ });
914
+
915
+ try {
916
+ const { text } = await agent.generate({
917
+ prompt,
918
+ // Total wall-clock for the step: requestTimeoutMs bounds each segment,
919
+ // not a whole multi-segment (client-tool) turn — see "Bound every step".
920
+ abortSignal: AbortSignal.timeout(600_000),
921
+ });
922
+ return text;
923
+ } finally {
924
+ // The durable resume handle — written even when the turn failed: a first
925
+ // step that fails AFTER creating its chat (timeout, stream loss) must
926
+ // still persist the id, or the retried step orphans a live chat and its
927
+ // partial effects. Exception: after an exhausted stream failure on a
928
+ // chat this very call created, the SDK has already discarded the dead
929
+ // session (agent.chatId is undefined) and nothing is written — see the
930
+ // CoderStreamError notes below for what retrying means then.
931
+ if (agent.chatId) await checkpoints.set(workflowId, agent.chatId);
932
+ }
933
+ }
934
+ ```
935
+
936
+ The shape matters:
937
+
938
+ - **One turn per step, `generate()` not `stream()`.** The checkpointed unit is
939
+ a finished result — and a mid‑`stream()` failure surfaces on the stream,
940
+ outside `generate()`'s error contract, which complicates step retry logic.
941
+ - **Persist the id, not the instance** (and never the token — read it from
942
+ each step's environment).
943
+ - **Checkpoint in `finally`, not only on success**, so failed-but-alive chats
944
+ stay reachable. What remains is the hard-crash window between chat creation
945
+ and the checkpoint write (below).
946
+ - **No `await using` here.** The chat outlives the step; disposal archives it
947
+ and archiving ends resumability (below).
948
+
949
+ ### Resume in the next step — any process, any machine
950
+
951
+ ```ts
952
+ // Step 1 — a queue job on machine A:
953
+ await runTurn("wf-1042", "Investigate the failing nightly build and propose a fix.");
954
+
955
+ // Step 2 — hours later, on machine B: same chat, full server-side history.
956
+ await runTurn("wf-1042", "Apply the fix you proposed and summarize what changed.");
957
+
958
+ // Final step — cleanup belongs to the workflow, not to each step (see below).
959
+ await archiveWorkflowChat("wf-1042");
960
+ ```
961
+
962
+ What resuming does (and doesn't do):
963
+
964
+ - **Construction is free.** `new CoderAgent({ …, chatId })` performs no I/O
965
+ and no validation — a bad or archived id fails the _turn_, not the
966
+ constructor.
967
+ - **The first resumed turn seeds its cursor** from the chat's newest message
968
+ (one single‑message history probe), so it streams only its own events —
969
+ it never re‑absorbs earlier turns' content or usage.
970
+ - **Re‑supply process‑local configuration.** Client `tools` live in your
971
+ process, not on the server, so each step that expects tool calls must pass
972
+ them again. The workspace binding is the opposite: fixed at creation,
973
+ carried by the chat, unchangeable on resume.
974
+ - **Sessions stay single‑flight across the whole system**, not just within one
975
+ process: run steps strictly sequentially per chat. A second job posting to
976
+ the same chat queues behind the live run and burns its own
977
+ `requestTimeoutMs` waiting.
978
+
979
+ ### Let the SDK absorb drops — and handle what it re‑throws
980
+
981
+ Self‑healed, invisible to the step (the run is **not** interrupted):
982
+
983
+ - **Transient stream drops.** The server keeps generating through the gap; the
984
+ SDK redials with backoff and replays what it missed — committed messages
985
+ past the turn's cursor plus the in‑progress message's deltas — deduplicating
986
+ on receipt. A message that committed _while the connection was down_ arrives
987
+ as exactly its missing tail. Drops cost latency, never content, and never
988
+ duplicate content.
989
+ - **Client‑tool pauses ride one connection.** A multi‑segment turn keeps its
990
+ stream across `requires_action` pauses, and a drop while your tool executes
991
+ redials in the background.
992
+ - **A lost tool‑call event.** If a turn pauses for client tools but the pause's
993
+ tool‑call event doesn't arrive within ~2 s, the SDK recovers the pending
994
+ calls from committed history over REST and the turn continues.
995
+
996
+ Surfaced to the step once self‑healing is exhausted:
997
+
998
+ ```ts
999
+ import { CoderApiError, CoderChatError, CoderStreamError } from "@coder/ai-sdk-agent";
1000
+
1001
+ try {
1002
+ await runTurn(workflowId, prompt);
1003
+ } catch (err) {
1004
+ if (err instanceof CoderChatError && err.retryable) {
1005
+ // Timeout or transient turn failure. The run was interrupted (best-effort)
1006
+ // and the chat survives — but a re-run resubmits the prompt as a new user
1007
+ // turn: retry only steps designed to tolerate that (below).
1008
+ } else if (err instanceof CoderStreamError) {
1009
+ // Redial budget exhausted (~15s without forward progress). On a resumed
1010
+ // chat this is never auto-retried — the retry decision is yours (below).
1011
+ } else if (err instanceof CoderApiError) {
1012
+ // Branch on err.status: back off and retry 408/425/429/5xx; fail the workflow
1013
+ // on the rest (expired token, archived/deleted chat, …).
1014
+ }
1015
+ throw err;
1016
+ }
1017
+ ```
1018
+
1019
+ - **`CoderChatError` with `retryable: true`** — the segment's
1020
+ `requestTimeoutMs` expired (`kind: "timeout"`) or the turn failed
1021
+ transiently (`kind: "stream_closed"`, an upstream 5xx). `retryable`
1022
+ classifies the _failure_ as transient — it does not make a re‑run free.
1023
+ Unlike the guarded `CoderStreamError` path below, the session is **not**
1024
+ discarded: the retried step reloads the checkpointed chat id (the `finally`
1025
+ checkpoint above is what makes this hold even when the _first_ step fails
1026
+ after creating its chat) and resubmits the same prompt as a new user turn,
1027
+ with the aborted attempt's partial output in history and any tool effects
1028
+ that ran before the failure not undone. The same idempotency judgment as
1029
+ below applies — retry steps designed to tolerate re‑submission; reconcile
1030
+ first when they aren't. And don't resubmit _immediately_: the expiry's
1031
+ server interrupt is asynchronous and best‑effort, so the timed‑out run may
1032
+ still be committing — run the same interrupt‑and‑wait‑for‑terminal‑status
1033
+ recovery as after a crash (below) before the retry, or its trailing output
1034
+ is absorbed into the retried turn. A timed‑out run can also have _finished_
1035
+ before that interrupt landed — but because the interrupt was already fired,
1036
+ the recovery's [result check](#recover-the-result-before-resubmitting)
1037
+ must treat the attempt as cut short (pin `cutShort` there) rather than
1038
+ trusting its own 409.
1039
+ - **`CoderStreamError`** (an AI SDK `APICallError`) — the stream could not be
1040
+ re‑established. `isRetryable: true` only when the failed turn had just
1041
+ created its chat _and_ had no external effects a replay would repeat (no
1042
+ workspace, no MCP servers, no fresh inline uploads); the SDK then discards
1043
+ the dead session so a retry — `maxRetries` or a re‑run step — starts clean
1044
+ on a fresh chat. The discard happens for _every_ chat the failed turn
1045
+ itself created — **including the effectful, `isRetryable: false` case**:
1046
+ the id is gone, the `finally` checkpoint writes nothing, and the
1047
+ interrupted chat — whose workspace/MCP tools may already have acted —
1048
+ becomes an unacknowledged orphan for the reconciliation sweep (below). A
1049
+ retried step then starts a fresh chat that has _not_ seen the orphan's
1050
+ effects: gate that retry on the same idempotency judgment as any
1051
+ re‑submission. **On a resumed chat it is always `isRetryable: false`:**
1052
+ re‑running the step resubmits the same prompt as a _new user turn_, with the
1053
+ failed attempt's partial output still in history (usually fine — the model
1054
+ sees its own aborted attempt), and any workspace/MCP tool effects that ran
1055
+ before the drop are not undone. That judgment call belongs to your workflow,
1056
+ which is exactly why it isn't automatic.
1057
+ - **`CoderApiError`** — an HTTP request failed; every non‑2xx wraps in this
1058
+ error, so branch on `status` before deciding. Rate limiting and transient
1059
+ server failures (408, 425, 429, 5xx) deserve backoff and a step retry (with
1060
+ the same re‑submission caveat as above). Auth failures and archived/deleted
1061
+ chats (401/403/404, the archived‑chat 400) hit the same wall on every
1062
+ attempt — fail the workflow.
1063
+
1064
+ ### Recover after a crash
1065
+
1066
+ A hard crash (OOM kill, host loss) runs no `finally` blocks. Reason from the
1067
+ checkpoint:
1068
+
1069
+ - **Crashed with an empty checkpoint** — usually nothing exists server‑side
1070
+ and the retried step just creates a fresh chat. But an empty checkpoint is
1071
+ not proof of nothing: a crash or `requestTimeoutMs` expiry while the create
1072
+ request was _in flight_ can leave a chat the server committed but whose id
1073
+ never arrived — unreachable even by the SDK's own interrupt. Treat "no
1074
+ checkpoint" as _up to one unacknowledged chat per attempt may exist_ — not
1075
+ as a clean slate, and automatic step retries can stack several. The
1076
+ reconciliation sweep below must therefore retire _every_ matching orphan
1077
+ (and its workspace/MCP effects, if the deployment auto‑attached any), not
1078
+ stop at the first.
1079
+ - **Crashed after creation, before the checkpoint** — the chat is orphaned:
1080
+ its run keeps generating until it settles on its own — or, if the crash hit
1081
+ one of your client tools, never settles: it stays paused in
1082
+ `requires_action`, holding its workspace, invisible to the retried step
1083
+ (which has no id to interrupt). Either way the retried step starts a fresh
1084
+ chat, so orphans are found only by the reconciliation sweep from
1085
+ [Preventing stuck turns](#preventing-stuck-turns) — that sweep is not
1086
+ optional in this pattern, and it needs a **wider filter** here: the fleet
1087
+ sweep watches for _non‑terminal_ chats stuck past their budget, but an
1088
+ orphaned first turn may settle terminally on its own and then sit as a
1089
+ live, unarchived chat forever. Sweep every chat no checkpoint accounts
1090
+ for — regardless of status — and interrupt/archive it. (To shrink the window, checkpoint eagerly from the
1091
+ first `ws:dial` transport event — it carries the chat id as soon as the chat
1092
+ exists — but then the checkpoint's lifecycle is yours too: the retried step
1093
+ resumes a chat holding a dead half‑turn, and whenever a `CoderStreamError`
1094
+ killed a chat this step created — retryable or not: the SDK discards every
1095
+ such session, but only in memory — you must clear the eagerly checkpointed
1096
+ id yourself before any retry, or "starts fresh after a discard" silently
1097
+ turns into resuming the dead chat.)
1098
+ - **Crashed after the checkpoint** — the chat is resumable, but don't just
1099
+ resubmit. The crashed attempt's run may still be live server‑side, and a
1100
+ resumed turn seeds its message cursor _before_ submitting — whatever the
1101
+ old run commits after that point would be absorbed into the resumed turn's
1102
+ output and usage. A crash _inside one of your client tools_ is worse: the
1103
+ chat is paused in `requires_action`, where new messages wait forever — and
1104
+ if the tool's external effect committed before the crash, that pause is the
1105
+ only pending record of it, so reconcile the tool ledger **before** any
1106
+ interrupt ([Make client tools crash-safe](#make-client-tools-crash-safe)).
1107
+ Both cases have one remedy — stop what the crash left behind and **wait
1108
+ until the chat actually settles** before deciding: a client‑tool pause is
1109
+ reconciled, never interrupted away; anything else is interrupted:
1110
+ `interrupt()` resolves on acknowledgment while the run keeps winding down
1111
+ (and committing) for a few more seconds, so poll the chat's status to a
1112
+ terminal one first. On a chat with no live run the interrupt rejects with a
1113
+ 409 `CoderApiError`; that's the good case — ignore it:
1114
+
1115
+ ```ts
1116
+ // Both shared with "Make client tools crash-safe" (below): the ledger
1117
+ // reconciliation resolves to its `unstarted` — true only when it fell back
1118
+ // to interrupting — and journals each action write-ahead, so a recovery
1119
+ // pass that dies mid-recovery is visible — and classifiable — to the next.
1120
+ declare function reconcileClientTools(): Promise<boolean>;
1121
+ // Entries are per-ATTEMPT: clear/supersede them transactionally with
1122
+ // recording the attempt's result, or a stale verdict misclassifies the
1123
+ // next attempt on this chat — attempt-scoped journal keys close the
1124
+ // remaining double-crash window (see
1125
+ // https://github.com/coder/ai-sdk/issues/92).
1126
+ declare const journal: {
1127
+ get(chatId: string): Promise<"resumed" | "cut-short" | undefined>;
1128
+ set(chatId: string, state: "resumed" | "cut-short"): Promise<void>;
1129
+ };
1130
+
1131
+ // Bounded: a stalled deployment must fail the step, not hang it.
1132
+ const deadline = AbortSignal.timeout(15_000);
1133
+ // Whether an interrupt stopped a live run drives the resubmit decision
1134
+ // below: a 409 means there was nothing to stop.
1135
+ let cutShort = true;
1136
+ let { status } = await agent.client.getChat(agent.chatId!, deadline);
1137
+ const journaled = await journal.get(agent.chatId!);
1138
+ if (status === "requires_action") {
1139
+ // The crash hit a client tool mid-execution: this pause is the only
1140
+ // pending record of its committed effects. Never interrupt it away —
1141
+ // reconcile the ledger instead (below): it submits recorded results
1142
+ // (reviving the run — nothing was cut short) or interrupts itself only
1143
+ // when nothing committed.
1144
+ cutShort = await reconcileClientTools();
1145
+ } else if (journaled) {
1146
+ // A previous recovery pass acted on this chat, then died before the
1147
+ // step's result was recorded. Its journaled state is the verdict:
1148
+ // "resumed" — it revived the run by submitting recorded results;
1149
+ // interrupting now would cut down a run doing the step's work, and the
1150
+ // resubmitted prompt would mint NEW tool-call ids, sidestepping the
1151
+ // idempotency keys. Let it finish; nothing was cut short.
1152
+ // "cut-short" — it went on to interrupt (an unstarted sibling in the
1153
+ // batch); the attempt is truncated, so the check below must resubmit.
1154
+ cutShort = journaled === "cut-short";
1155
+ if (cutShort) {
1156
+ // The journal records write-ahead INTENT, not a landed interrupt: the
1157
+ // pass may have died in between, leaving the revived run working.
1158
+ // Re-drive the interrupt to make the intent true — a 409 means the
1159
+ // chat already settled; the cut-short verdict stands either way.
1160
+ await agent.interrupt({ signal: deadline }).catch((err) => {
1161
+ if (!(err instanceof CoderApiError && err.status === 409)) throw err;
1162
+ });
1163
+ }
1164
+ } else {
1165
+ await agent.interrupt({ signal: deadline }).catch((err) => {
1166
+ // A 409 proves nothing is stoppable NOW — that means "never cut short"
1167
+ // only if the probe saw no interrupt already in flight. A probed
1168
+ // "interrupting" is an earlier interrupt (a timeout's best-effort one,
1169
+ // a sweeper) already truncating the attempt; this 409 just says it
1170
+ // landed. Keep cutShort = true then.
1171
+ if (err instanceof CoderApiError && err.status === 409) cutShort = status === "interrupting";
1172
+ else throw err;
1173
+ });
1174
+ }
1175
+ // The 15 s interrupt bound must not cap what follows: a reconciled pause
1176
+ // revives a full model/tool continuation. Budget the settle poll — and the
1177
+ // history reads after it — on the turn scale, like the step itself.
1178
+ const settle = AbortSignal.timeout(600_000);
1179
+ do {
1180
+ ({ status } = await agent.client.getChat(agent.chatId!, settle));
1181
+ if (status === "waiting" || status === "completed" || status === "error") break;
1182
+ // With no accepted interrupt racing to clear it, requires_action is a
1183
+ // STABLE pause: a run resumed by submitted tool results called a NEW
1184
+ // client tool. Exit and reconcile the ledger again (below) instead of
1185
+ // polling out the deadline.
1186
+ if (status === "requires_action" && !cutShort) break;
1187
+ await new Promise((r) => setTimeout(r, 1_000)); // acknowledged ≠ settled — wait it out
1188
+ } while (true);
1189
+ ```
1190
+
1191
+ Settled is necessary, not sufficient: a terminal status doesn't say whether
1192
+ the crashed attempt already _finished the step's work_ — check history
1193
+ before resubmitting
1194
+ ([Recover the result before resubmitting](#recover-the-result-before-resubmitting)).
1195
+
1196
+ ### Make client tools crash-safe
1197
+
1198
+ Interrupting un‑strands a `requires_action` pause — but if the crashed
1199
+ process died _between_ a non‑idempotent client tool committing its external
1200
+ effect (a payment, a deploy, an email) and that result reaching the server,
1201
+ the pause is the **only pending record of the execution**. Interrupt it away
1202
+ and the resubmitted prompt is free to call the tool again — and duplicate the
1203
+ effect. Close the window with a tool‑invocation ledger in the same durable
1204
+ store as the `chatId` checkpoint, keyed by **chat id + `toolCallId`** — the
1205
+ server assigns the call id and commits it to history, so both sides of the
1206
+ reconciliation survive the crash. The chat id must be part of the key (and
1207
+ of the idempotency key handed to the external system): call ids are only
1208
+ unique _within_ a chat, and a store shared across workflows must never let
1209
+ one chat's `done` entry answer — or deduplicate — another chat's call. In
1210
+ multi‑tenant stores, fold the tenant in too:
1211
+
1212
+ ```ts
1213
+ import { tool } from "ai";
1214
+ import { z } from "zod";
1215
+ import { CoderAgent } from "@coder/ai-sdk-agent";
1216
+
1217
+ type LedgerEntry = { state: "committing" } | { state: "done"; output: string };
1218
+ declare const ledger: {
1219
+ get(key: string): Promise<LedgerEntry | undefined>;
1220
+ set(key: string, entry: LedgerEntry): Promise<void>;
1221
+ };
1222
+ declare const payments: {
1223
+ charge(amountCents: number, opts: { idempotencyKey: string }): Promise<{ receiptId: string }>;
1224
+ };
1225
+
1226
+ const ledgerKey = (chatId: string, toolCallId: string) => `${chatId}/${toolCallId}`;
1227
+ const ChargeArgs = z.object({ amountCents: z.number().int() });
1228
+
1229
+ // A factory taking a getter, not an id: the chat may not exist until the
1230
+ // turn creates it, so the id must be read lazily — it is set before any
1231
+ // tool runs.
1232
+ const chargeCard = (chatId: () => string) =>
1233
+ tool({
1234
+ description: "Charge the customer's card.",
1235
+ inputSchema: ChargeArgs, // shared with recovery's re-drive dispatch below
1236
+ execute: async ({ amountCents }, { toolCallId }) => {
1237
+ const key = ledgerKey(chatId(), toolCallId);
1238
+ // Write-ahead, BEFORE the effect: a crash between the two writes leaves
1239
+ // "committing" (ambiguous, but resolvable below); no entry at all proves
1240
+ // the effect never started.
1241
+ await ledger.set(key, { state: "committing" });
1242
+ // Hand the external system the SAME key as its idempotency key, so
1243
+ // re-driving this exact call can never double-charge.
1244
+ const { receiptId } = await payments.charge(amountCents, { idempotencyKey: key });
1245
+ await ledger.set(key, { state: "done", output: `charged: receipt ${receiptId}` });
1246
+ return `charged: receipt ${receiptId}`;
1247
+ },
1248
+ });
1249
+
1250
+ // Wire-up: a direct self-reference in the constructor call is circular for
1251
+ // the type checker — defer it through a box assigned right after.
1252
+ let self: { chatId?: string } | undefined;
1253
+ const agent = new CoderAgent({
1254
+ /* …base options as in runTurn… */
1255
+ tools: { charge_card: chargeCard(() => self!.chatId!) },
1256
+ });
1257
+ self = agent;
1258
+ ```
1259
+
1260
+ On crash recovery, when the checkpointed chat is paused in `requires_action`,
1261
+ reconcile _before_ any interrupt. The pending calls are in committed history —
1262
+ the same derivation the SDK itself uses to recover a lost `action_required`
1263
+ event — and the ledger holds the verdict on each:
1264
+
1265
+ ```ts
1266
+ // The step's configured ToolSet names: history does not record which calls
1267
+ // were client tools (see [Rehydrating chat history]), so — like the SDK's
1268
+ // own derivation — restrict to the names this step registers; a server/MCP
1269
+ // call must never reach the ledger as "unstarted".
1270
+ const clientTools = new Set(["charge_card"]);
1271
+
1272
+ // Pending = the last assistant message's client (non-provider-executed,
1273
+ // ToolSet-named) tool calls, minus ids answered by a later message's tool
1274
+ // result.
1275
+ const { messages } = await agent.client.getMessages(agent.chatId!, { limit: 200 }, deadline);
1276
+ const ordered = [...messages].sort((a, b) => a.id - b.id);
1277
+ const lastAssistant = ordered.findLast((m) => m.role === "assistant");
1278
+ const handled = new Set(
1279
+ ordered
1280
+ .filter((m) => m.id > (lastAssistant?.id ?? 0))
1281
+ .flatMap((m) => m.content ?? [])
1282
+ .filter((p) => p.type === "tool-result")
1283
+ .map((p) => p.tool_call_id),
1284
+ );
1285
+ const pending = (lastAssistant?.content ?? []).filter(
1286
+ (p) =>
1287
+ p.type === "tool-call" &&
1288
+ !p.provider_executed &&
1289
+ clientTools.has(p.tool_name!) &&
1290
+ !handled.has(p.tool_call_id),
1291
+ );
1292
+
1293
+ const results: { tool_call_id: string; output: unknown }[] = [];
1294
+ let unstarted = false;
1295
+ for (const call of pending) {
1296
+ const key = ledgerKey(agent.chatId!, call.tool_call_id!);
1297
+ const entry = await ledger.get(key);
1298
+ if (entry?.state === "done") {
1299
+ // Effect committed, result stranded: submit the record — the pause is
1300
+ // answered and nothing re-executes.
1301
+ results.push({ tool_call_id: call.tool_call_id!, output: entry.output });
1302
+ } else if (entry) {
1303
+ // "committing" is ambiguous — the crash hit between the write-ahead and
1304
+ // "done". Safe to re-drive: the SAME idempotency key makes a committed
1305
+ // effect replay as a no-op, and the call's args are committed in history.
1306
+ // Dispatch by tool name — every ledgered tool needs its own re-drive
1307
+ // path, its args validated exactly as the live loop would have.
1308
+ if (call.tool_name !== "charge_card")
1309
+ throw new Error(`no re-drive path for tool ${call.tool_name}`);
1310
+ const { amountCents } = ChargeArgs.parse(call.args);
1311
+ const { receiptId } = await payments.charge(amountCents, { idempotencyKey: key });
1312
+ const output = `charged: receipt ${receiptId}`;
1313
+ await ledger.set(key, { state: "done", output });
1314
+ results.push({ tool_call_id: call.tool_call_id!, output });
1315
+ } else {
1316
+ unstarted = true; // the tool never ran — interrupting loses nothing
1317
+ }
1318
+ }
1319
+ if (results.length > 0 || unstarted) {
1320
+ // ONE write-ahead with the batch's FINAL verdict, before any network
1321
+ // call — a mixed batch journals "cut-short" before even its submission,
1322
+ // so no crash point can leave a stale "resumed" masking the pending
1323
+ // interrupt. Every crash then either re-reconciles the stable pause or
1324
+ // re-drives the journaled intent. Clear the entry only once the step's
1325
+ // result is finally recorded.
1326
+ await journal.set(agent.chatId!, unstarted ? "cut-short" : "resumed");
1327
+ }
1328
+ if (results.length > 0) await agent.client.submitToolResults(agent.chatId!, { results }, deadline);
1329
+ if (unstarted) await agent.client.interruptChat(agent.chatId!, deadline);
1330
+ // As `reconcileClientTools()` in the settle-wait snippet above: resolve to
1331
+ // `unstarted` — cutShort is true only when this fell back to interrupting.
1332
+ ```
1333
+
1334
+ - **Only interrupt when the ledger shows no committed effect.** Submitted
1335
+ results are committed to chat history, so even when a mixed batch (one call
1336
+ done, a sibling never started) still ends in an interrupt, the effect's
1337
+ recorded outcome survives for the resubmitted turn to see.
1338
+ - **Submitting answers the pause; it does not revive your tool loop.** Once
1339
+ every pending call is answered, the turn resumes **server‑side** with no
1340
+ process streaming it. That is why the settle‑wait's reconcile branch never
1341
+ interrupts a pause — nothing stranded is left to stop, and an interrupt
1342
+ now would cut down the very run the submission revived — and why the
1343
+ resolved `unstarted` leaves **`cutShort = false`** after a submission:
1344
+ nothing was cut short, and the next section's `!cutShort` check must
1345
+ classify the resumed run's finish as a completed attempt, not resubmit
1346
+ it. Give the poll a turn‑scale deadline rather than the 15 s interrupt
1347
+ bound; it exits on `requires_action` (stable here, with no accepted
1348
+ interrupt clearing it), which means the resumed run called a _new_ tool —
1349
+ reconcile again; a call with no ledger entry is safe to interrupt.
1350
+ - **The finished run's output lands only in history** — recover it like any
1351
+ completed‑but‑unrecorded attempt (next section) instead of resubmitting.
1352
+
1353
+ ### Recover the result before resubmitting
1354
+
1355
+ The settle‑wait resolves overlap with a still‑live run, but not the last
1356
+ crash window: **the server finished the turn, and the process died before the
1357
+ step's return value was recorded.** History now holds a perfectly good
1358
+ result; a retried step that blindly resubmits runs the whole turn again —
1359
+ duplicated output, repeated workspace/MCP/tool effects. This is the
1360
+ at‑least‑once boundary every durable engine has around external effects, and
1361
+ the chat itself is the record that closes it. Make attempts matchable first —
1362
+ the server stores the prompt verbatim, so embed a step‑scoped marker:
1363
+
1364
+ ```ts
1365
+ const marker = `[${workflowId}/step-2]`; // stable across this step's retries
1366
+ const { text } = await agent.generate({ prompt: `${marker} ${prompt}` });
1367
+ ```
1368
+
1369
+ Then, after the settle‑wait, three states are distinguishable — and only one
1370
+ of them resubmits:
1371
+
1372
+ ```ts
1373
+ import { chatMessagesToUIMessages } from "@coder/ai-sdk-agent";
1374
+
1375
+ // A long turn can commit more than a page of messages after the prompt —
1376
+ // page newest-first (the endpoint's default order) until the turn's user
1377
+ // message is on hand; "never submitted" must never be concluded from a
1378
+ // truncated read, or this recovery resubmits exactly the turn it was meant
1379
+ // to deduplicate.
1380
+ const fetched = [];
1381
+ let page = await agent.client.getMessages(agent.chatId!, { limit: 200 }, settle);
1382
+ fetched.push(...page.messages);
1383
+ while (!fetched.some((m) => m.role === "user") && page.has_more) {
1384
+ const oldestId = Math.min(...fetched.map((m) => m.id));
1385
+ page = await agent.client.getMessages(agent.chatId!, { before_id: oldestId, limit: 200 }, settle);
1386
+ fetched.push(...page.messages);
1387
+ }
1388
+ const transcript = chatMessagesToUIMessages(fetched); // chronological
1389
+ const lastUser = transcript.findLast((m) => m.role === "user");
1390
+ const submitted = lastUser?.parts.some((p) => p.type === "text" && p.text.includes(marker));
1391
+
1392
+ if (submitted && !cutShort && (status === "waiting" || status === "completed")) {
1393
+ // The attempt finished; only the recording was lost. Recover what
1394
+ // `generate().text` would have returned — the FINAL step's text. A turn
1395
+ // that crossed tool calls commits narration between them too, so cut the
1396
+ // turn's parts after the last tool part and join the text that follows.
1397
+ const parts = transcript
1398
+ .slice(transcript.indexOf(lastUser!) + 1)
1399
+ .filter((m) => m.role === "assistant")
1400
+ .flatMap((m) => m.parts);
1401
+ return parts
1402
+ .slice(parts.findLastIndex((p) => p.type === "dynamic-tool") + 1)
1403
+ .filter((p) => p.type === "text")
1404
+ .map((p) => p.text)
1405
+ .join("");
1406
+ }
1407
+ // Resubmit only what never finished: the prompt never reached the server
1408
+ // (!submitted), the turn failed (status "error" — normal retry judgment
1409
+ // applies), or the interrupt cut a live run short (cutShort — the model sees
1410
+ // its own aborted attempt in history and continues from it). One exit is
1411
+ // neither: a requires_action settle goes back to ledger reconciliation
1412
+ // (previous section) — never to recovery or resubmission.
1413
+ ```
1414
+
1415
+ - **`cutShort` is the completed/interrupted discriminator — but a 409 only
1416
+ proves nothing is running _now_.** An interrupted attempt leaves the same
1417
+ marker and a truncated answer in history, so history alone can't tell
1418
+ "finished" from "stopped"; what can is whether any interrupt stopped the
1419
+ run. The inference "409 ⇒ the attempt finished" is therefore sound only
1420
+ when recovery's interrupt is the _first_ one that could have: true after a
1421
+ hard crash, false after a `requestTimeoutMs` expiry — the expiry already
1422
+ fired the SDK's own best‑effort interrupt, which may have stopped the run
1423
+ before recovery ever looked. Carry the reason recovery ran: when it follows
1424
+ a known timeout or abort — or a journaled earlier interrupt (write‑ahead,
1425
+ like the tool ledger, if the recovery pass can itself crash between its
1426
+ interrupt and the resubmit) — pin `cutShort` to `true` and let only the
1427
+ hard‑crash path trust the 409.
1428
+ - **The cut lands after the last tool part of _either_ kind.** History does
1429
+ not mark which tool calls were client tools
1430
+ ([Rehydrating chat history](#rehydrating-chat-history)), so when a final
1431
+ step narrates _before_ a server‑side tool call, that preamble is dropped
1432
+ too — the recovered text is the answer following the turn's last tool
1433
+ activity.
1434
+ - **Steps with structured results recover the call's input, not the text.**
1435
+ A [structured output](#structured-output) step's answer is the
1436
+ `structured_output` call's typed input, which rehydrates as a
1437
+ `dynamic-tool` part on the recovered turn
1438
+ ([Rehydrating chat history](#rehydrating-chat-history)) — the text join
1439
+ above would slice it away and return only the ack prose that follows.
1440
+ Recover the filed call instead — this scan replaces the text join
1441
+ _inside_ the recovery branch above, reusing its `parts` — validated
1442
+ client‑side exactly like the live path (rule 2 there — the schema is the
1443
+ real gate):
1444
+
1445
+ ```ts
1446
+ // Scan backward to the last call that VALIDATES, as in the live path — a
1447
+ // schema-invalid re-file must not shadow a valid answer.
1448
+ const filed = parts
1449
+ .filter((p) => p.type === "dynamic-tool")
1450
+ .filter((p) => p.toolName === "structured_output");
1451
+ let answer: z.infer<typeof Answer> | undefined;
1452
+ for (const call of filed.reverse()) {
1453
+ const parsed = Answer.safeParse(call.input);
1454
+ if (parsed.success) {
1455
+ answer = parsed.data;
1456
+ break;
1457
+ }
1458
+ }
1459
+ // recovered ⇔ answer !== undefined
1460
+ ```
1461
+
1462
+ ### Watch turn health from inside the step
1463
+
1464
+ [Transport events](#observability) classify how each segment ended without
1465
+ parsing errors or stream frames — emit them as step metrics/traces:
1466
+
1467
+ ```ts
1468
+ declare const metrics: {
1469
+ ok(segment: number, status: string, ms: number): void;
1470
+ fail(segment: number, error: string, detail: string): void;
1471
+ teardown(segment: number): void;
1472
+ };
1473
+
1474
+ function observe(ev: CoderTransportEvent): void {
1475
+ if (ev.type === "ws:redial") {
1476
+ // Early warning: the stream dropped and is reconnecting. The server keeps
1477
+ // generating — nothing is lost yet; the turn fails if this hits the cap.
1478
+ console.warn(
1479
+ `redial ${ev.consecutiveFailures}/${ev.maxConsecutiveFailures}, next in ${ev.backoffMs}ms`,
1480
+ );
1481
+ } else if (ev.type === "segment:settle") {
1482
+ if (ev.finishReason) metrics.ok(ev.segment, ev.status!, ev.durationMs);
1483
+ else if (ev.error) metrics.fail(ev.segment, ev.error.name, ev.error.message);
1484
+ else metrics.teardown(ev.segment);
1485
+ }
1486
+ }
1487
+ ```
1488
+
1489
+ `segment:settle` fires exactly once per `segment:start` and separates the
1490
+ three endings a workflow cares about:
1491
+
1492
+ - **Clean settle** — `status` + `finishReason` present. `finishReason:
1493
+ "tool-calls"` (status `requires_action`) is a healthy mid‑turn pause for
1494
+ your client tools; terminal statuses end the turn.
1495
+ - **Failure** — `error: { name, message }` present (plus `status` when the
1496
+ server run itself still settled terminally): server‑side turn errors,
1497
+ transport failure after redials were exhausted, `requestTimeoutMs` expiry
1498
+ (`error.name: "CoderChatError"`), and caller aborts — which settle as
1499
+ failures carrying the abort reason's name, never as teardowns:
1500
+ `"TimeoutError"` for an `AbortSignal.timeout(…)` deadline (the running
1501
+ example), `"AbortError"` for an explicit abort.
1502
+ - **Teardown** — neither: the stream consumer itself cancelled the turn
1503
+ (`stream()`'s `ReadableStream.cancel()` with no abort of its own) — rare in
1504
+ `generate()`‑shaped steps.
1505
+
1506
+ The hook is observability‑only — handler exceptions are swallowed and can't
1507
+ disturb the turn, and an unset hook costs nothing. For monitoring _across_
1508
+ steps (chats stuck before a watcher started, fleet‑level sweeps), see
1509
+ [Preventing stuck turns](#preventing-stuck-turns).
1510
+
1511
+ ### Bound every step
1512
+
1513
+ `requestTimeoutMs` arms **one timer per segment** covering everything the
1514
+ segment does: the REST phase (chat creation, message/tool‑result submission,
1515
+ uploads), the stream — and, the part that matters here, **any redial backoff
1516
+ inside the segment**. A segment that spends 12 s reconnecting has 12 s less
1517
+ budget; redials never reset the clock, so a flaky network can't stretch a step
1518
+ past its bound. On expiry the call rejects with `CoderChatError`
1519
+ (`kind: "timeout"`, `retryable: true`) and the server run is interrupted
1520
+ best‑effort.
1521
+
1522
+ Two workflow‑specific notes:
1523
+
1524
+ - A turn that times out (or crashes) before its chat id arrived leaves nothing
1525
+ to _resume_ — the checkpoint stays empty and the retried step starts fresh —
1526
+ but possibly an unacknowledged chat to sweep (see
1527
+ [Recover after a crash](#recover-after-a-crash)).
1528
+ - `requestTimeoutMs` bounds each _segment_; a turn driving client tools runs
1529
+ several. Cap a step's total wall‑clock with
1530
+ `abortSignal: AbortSignal.timeout(…)` —
1531
+ [Timeouts & cancellation](#timeouts--cancellation).
1532
+
1533
+ ### Archive at the end — and only at the end
1534
+
1535
+ **An archived chat cannot be resumed:** the server rejects new messages on it
1536
+ (400, `Cannot send messages to an archived chat`). The per‑request habits from
1537
+ [Cleanup](#cleanup) — `await using`, `archive()` in a `finally` — would
1538
+ destroy the session from inside an intermediate step, so here the _workflow_
1539
+ owns the chat's end of life: archive once, in the final step, and in the
1540
+ workflow's failure/compensation handler so abandoned runs don't accumulate
1541
+ live chats.
1542
+
1543
+ ```ts
1544
+ export async function archiveWorkflowChat(workflowId: string): Promise<void> {
1545
+ const chatId = await checkpoints.get(workflowId);
1546
+ if (!chatId) return; // the workflow died before its first turn created a chat
1547
+ const agent = new CoderAgent({
1548
+ baseUrl: process.env.CODER_URL!,
1549
+ token: process.env.CODER_SESSION_TOKEN!,
1550
+ organizationId: process.env.CODER_ORG_ID!,
1551
+ chatId,
1552
+ });
1553
+ // A crashed step may have left a run live — stop it first, bounded. On an
1554
+ // already-settled chat the interrupt rejects with a 409; ignore it.
1555
+ await agent.interrupt({ signal: AbortSignal.timeout(15_000) }).catch(() => {});
1556
+ await agent.archive(); // retries 409s while the interrupted run winds down (~15s)
1557
+ }
1558
+ ```
1559
+
1560
+ ### Checklist
1561
+
1562
+ - One turn per durable step; `generate()`, not `stream()`.
1563
+ - Persist `agent.chatId` (a string) — never the instance, never the token.
1564
+ - Bound every step: `requestTimeoutMs` per segment, an abort deadline for
1565
+ total wall‑clock.
1566
+ - Let redials self‑heal; own every retry decision — a re‑run step resubmits
1567
+ its prompt as a new user turn.
1568
+ - Before resubmitting after a crash: reconcile the tool ledger before any
1569
+ interrupt, and recover a finished attempt's result from history.
1570
+ - Keep concurrent steps' fan‑out within workspace quota —
1571
+ [Workspaces & quota](#workspaces--quota).
1572
+ - Steps that don't need server‑side tools (plan / extract / synthesize) are
1573
+ cheaper and natively structured through
752
1574
  [`@coder/ai-sdk-provider`](../provider) + `generateObject` — no chat, no
753
1575
  workspace, no cleanup.
1576
+ - Archive in the final step / failure handler — never per step.
754
1577
 
755
1578
  ## Testing
756
1579