@coder/ai-sdk-agent 0.11.7 → 0.11.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +513 -1545
- package/docs/durable-workflows.md +890 -0
- package/docs/observability.md +130 -0
- package/docs/structured-output.md +134 -0
- package/docs/workspaces-and-quota.md +214 -0
- package/package.json +5 -4
package/README.md
CHANGED
|
@@ -4,42 +4,45 @@
|
|
|
4
4
|
[](https://www.npmjs.com/package/@coder/ai-sdk-agent)
|
|
5
5
|
[](./LICENSE)
|
|
6
6
|
|
|
7
|
-
A **Vercel AI SDK
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
(`generate()` / `stream()`)
|
|
7
|
+
A **Vercel AI SDK agent backed by Coder Agents**, Coder's server‑side agent
|
|
8
|
+
runtime. `new CoderAgent(...)` implements the AI SDK's
|
|
9
|
+
[`Agent`](https://ai-sdk.dev/docs/reference/ai-sdk-core/agent) interface
|
|
10
|
+
(`generate()` / `stream()`): script it, stream from it, and attach your own tools
|
|
11
11
|
exactly like the SDK's own `ToolLoopAgent`.
|
|
12
12
|
|
|
13
|
+
Coder Agents runs the complete agent loop **server‑side**: the multi‑step tool
|
|
14
|
+
loop, built‑in tools, MCP, sub‑agents, multi‑provider model routing, and
|
|
15
|
+
automatic context compaction. This package bridges it to the AI SDK's
|
|
16
|
+
client‑side loop without re‑implementing the loop.
|
|
17
|
+
|
|
13
18
|
> Status: targets Coder's stable chat API (`/api/v2/chats`), available since
|
|
14
19
|
> Coder 2.37.0 (September 1, 2026). This package remains pre‑1.0.
|
|
15
20
|
> See [API compatibility](#api-compatibility) for older deployments.
|
|
16
21
|
|
|
17
|
-
|
|
22
|
+
**Contents**
|
|
18
23
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
+
| Get started | Build | Reference | Run in production |
|
|
25
|
+
| ------------------------------------------------------- | ----------------------------------------------------- | --------------------------------------- | -------------------------------------------------------------- |
|
|
26
|
+
| [Install](#install) | [Custom tools](#custom-tools) | [Auth](#auth) | [Timeouts & cancellation](#timeouts--cancellation) |
|
|
27
|
+
| [Quick start](#quick-start) | [Files](#files) | [Configuration](#configuration) | [Handling errors](#handling-errors) |
|
|
28
|
+
| [Examples](#examples) | [Structured output](#structured-output) | [API compatibility](#api-compatibility) | [Cleanup](#cleanup) |
|
|
29
|
+
| [Agent vs. provider](#agent-vs-provider--which-package) | [Sessions](#sessions) | [How it works](#how-it-works) | [Usage & cost](#usage--cost) |
|
|
30
|
+
| | [Rehydrating chat history](#rehydrating-chat-history) | [Testing](#testing) | [Observability](#observability) |
|
|
31
|
+
| | [Watching chats](#watching-chats) | [Limitations](#limitations) | [Workspaces & quota](#workspaces--quota) |
|
|
32
|
+
| | [Workspace previews](#workspace-previews) | | [Durable workflows](#durable-workflows-persist-resume-recover) |
|
|
33
|
+
| | [Sources](#sources) | | |
|
|
24
34
|
|
|
25
35
|
## Agent vs. provider — which package?
|
|
26
36
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
you need **server‑side tools, MCP, or a workspace**.
|
|
34
|
-
- **[`@coder/ai-sdk-provider`](../provider)** — **plain model calls** through
|
|
35
|
-
Coder's AI Gateway. A normal AI SDK provider: `generateText`, `streamText`, and
|
|
36
|
-
**`generateObject` for schema‑constrained structured output**. No chat, no
|
|
37
|
-
workspace, natively cancelable. Reach for it when you just need **a model**
|
|
38
|
-
(plan / extract / summarize / classify) with no server‑side tools.
|
|
37
|
+
| | `@coder/ai-sdk-agent` (this package) | [`@coder/ai-sdk-provider`](https://github.com/coder/ai-sdk/tree/main/packages/provider) |
|
|
38
|
+
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
|
39
|
+
| What runs | Coder's server‑side agent: tool loop, built‑in tools, MCP servers, workspace‑scoped file/shell tools, sub‑agents, and compaction, on the deployment | Plain model calls through Coder's AI Gateway. A normal AI SDK provider: `generateText`, `streamText`, `generateObject` |
|
|
40
|
+
| Server state | Each `CoderAgent` is one server chat ("session"), bound to at most one workspace | No chat, no workspace; natively cancelable |
|
|
41
|
+
| Structured output | Through a tool call ([Structured output](#structured-output)) | `generateObject`, schema‑constrained |
|
|
42
|
+
| Reach for it when | You need **server‑side tools, MCP, or a workspace** | You just need **a model** (plan / extract / summarize / classify) |
|
|
39
43
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
pure text/JSON steps and the Agent only for the steps that touch tools.
|
|
44
|
+
They compose: a multi‑step pipeline often uses the provider for its pure
|
|
45
|
+
text/JSON steps and the Agent only for the steps that touch tools.
|
|
43
46
|
|
|
44
47
|
## Install
|
|
45
48
|
|
|
@@ -47,9 +50,9 @@ pure text/JSON steps and the Agent only for the steps that touch tools.
|
|
|
47
50
|
pnpm add @coder/ai-sdk-agent ai@^7 zod
|
|
48
51
|
```
|
|
49
52
|
|
|
50
|
-
Requires Node ≥ 22 and `ai` v7
|
|
51
|
-
another `ai` major is detected
|
|
52
|
-
can't be resolved
|
|
53
|
+
Requires Node ≥ 22 and `ai` v7. The constructors throw an actionable error when
|
|
54
|
+
another `ai` major is detected, instead of failing cryptically mid‑generation.
|
|
55
|
+
The guard fails open when the installed version can't be resolved.
|
|
53
56
|
|
|
54
57
|
## Quick start
|
|
55
58
|
|
|
@@ -81,14 +84,15 @@ const result = await agent.stream({ prompt: "Write a haiku about Coder." });
|
|
|
81
84
|
for await (const delta of result.textStream) process.stdout.write(delta);
|
|
82
85
|
```
|
|
83
86
|
|
|
84
|
-
`generate()` returns a real AI SDK `GenerateTextResult`; `stream()` returns a
|
|
85
|
-
`StreamTextResult` (
|
|
86
|
-
`.usage`,
|
|
87
|
-
of the AI SDK.
|
|
87
|
+
`generate()` returns a real AI SDK `GenerateTextResult`; `stream()` returns a
|
|
88
|
+
real `StreamTextResult` (`.textStream`, `.fullStream`, `.toUIMessageStream()`,
|
|
89
|
+
`.steps`, `.usage`, …). Because `CoderAgent` _is_ an `Agent`, it composes with
|
|
90
|
+
the rest of the AI SDK.
|
|
88
91
|
|
|
89
92
|
## Examples
|
|
90
93
|
|
|
91
|
-
Runnable scripts
|
|
94
|
+
Runnable scripts in [`examples/`](https://github.com/coder/ai-sdk/tree/main/packages/agent/examples) run against a real deployment via
|
|
95
|
+
`tsx`:
|
|
92
96
|
|
|
93
97
|
```bash
|
|
94
98
|
export CODER_URL=https://dogfood.cdr.dev
|
|
@@ -102,34 +106,41 @@ pnpm example:file # attach a file to a chat (optional: pass a path)
|
|
|
102
106
|
pnpm example:structured # typed structured output via the structured_output tool
|
|
103
107
|
```
|
|
104
108
|
|
|
105
|
-
Each example creates a new chat and archives it when done
|
|
106
|
-
[`examples/README.md`](
|
|
109
|
+
Each example creates a new chat and archives it when done; none touches
|
|
110
|
+
workspaces. Details: [`examples/README.md`](https://github.com/coder/ai-sdk/blob/main/packages/agent/examples/README.md).
|
|
107
111
|
|
|
108
112
|
## Custom tools
|
|
109
113
|
|
|
110
|
-
Tools you pass are registered with Coder Agents as **client‑executed**
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
+
Tools you pass are registered with Coder Agents as **client‑executed**
|
|
115
|
+
("dynamic") tools; your `execute` runs in your process. This is the standard AI
|
|
116
|
+
SDK tool loop:
|
|
117
|
+
|
|
118
|
+
1. The model calls your tool, and the run pauses on the server.
|
|
119
|
+
2. The AI SDK runs your tool's `execute`.
|
|
120
|
+
3. This package submits the result back, and the run resumes.
|
|
121
|
+
|
|
122
|
+
- Give tools an `execute` for scripting use; the loop runs to completion
|
|
123
|
+
automatically.
|
|
124
|
+
- Coder's own server‑side tools (file editing, shell, MCP, …) still run on the
|
|
125
|
+
server. They appear in the transcript as `providerExecuted` tool
|
|
126
|
+
calls/results: you observe them, you don't execute them.
|
|
114
127
|
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
appear in the transcript as `providerExecuted` tool calls/results — you observe them, you
|
|
118
|
-
don't execute them.
|
|
128
|
+
<details>
|
|
129
|
+
<summary>Since v0.2.1: server tools stream as <code>dynamic-tool</code> parts</summary>
|
|
119
130
|
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
parts
|
|
124
|
-
|
|
131
|
+
Server‑executed tools stream with `dynamic: true`: they aren't in your `ToolSet`,
|
|
132
|
+
and the AI SDK only accepts unknown tool names on dynamic calls. In UI message
|
|
133
|
+
streams they therefore surface as `dynamic-tool` parts rather than `tool-{name}`
|
|
134
|
+
parts. Key off `toolName`, not `part.type`, when rendering them.
|
|
135
|
+
|
|
136
|
+
</details>
|
|
125
137
|
|
|
126
138
|
## Files
|
|
127
139
|
|
|
128
|
-
|
|
129
|
-
**read** it or **operate on** it.
|
|
140
|
+
Pick by whether the model should **read** the file or **operate on** it.
|
|
130
141
|
|
|
131
|
-
**Chat attachments**
|
|
132
|
-
`file` part into a message and it's uploaded transparently:
|
|
142
|
+
**Chat attachments** carry content for the model to read (a PDF, image, CSV…).
|
|
143
|
+
Drop a native AI SDK `file` part into a message and it's uploaded transparently:
|
|
133
144
|
|
|
134
145
|
```ts
|
|
135
146
|
import { readFile } from "node:fs/promises";
|
|
@@ -152,10 +163,13 @@ await agent.generate({
|
|
|
152
163
|
});
|
|
153
164
|
```
|
|
154
165
|
|
|
155
|
-
|
|
156
|
-
stream
|
|
166
|
+
To upload once and reuse across turns, use `attach()`. It also accepts a
|
|
167
|
+
`Blob`/`File` or stream; `fs.openAsBlob` avoids reading the whole file into
|
|
168
|
+
memory:
|
|
157
169
|
|
|
158
170
|
```ts
|
|
171
|
+
import { openAsBlob } from "node:fs";
|
|
172
|
+
|
|
159
173
|
const file = await agent.attach({
|
|
160
174
|
content: await openAsBlob("report.pdf"),
|
|
161
175
|
mediaType: "application/pdf",
|
|
@@ -173,17 +187,20 @@ await agent.generate({
|
|
|
173
187
|
});
|
|
174
188
|
```
|
|
175
189
|
|
|
176
|
-
Attachments are capped at **10 MiB** and restricted to
|
|
177
|
-
|
|
178
|
-
Oversized or unsupported files throw a clear error
|
|
190
|
+
Attachments are capped at **10 MiB** and restricted to this media‑type
|
|
191
|
+
allowlist: `application/pdf`, `application/json`, `text/{plain,markdown,csv}`,
|
|
192
|
+
`image/{png,jpeg,gif,webp}`. Oversized or unsupported files throw a clear error
|
|
193
|
+
up front.
|
|
179
194
|
|
|
180
|
-
**Workspace files**
|
|
181
|
-
binary — anything outside the allowlist or over the cap
|
|
182
|
-
and let the agent's tools take over.
|
|
183
|
-
|
|
195
|
+
**Workspace files** are material for the agent to operate on: a zip of assets, a
|
|
196
|
+
dataset, a binary — anything outside the allowlist or over the cap. Write it onto
|
|
197
|
+
the workspace filesystem and let the agent's tools take over.
|
|
198
|
+
|
|
199
|
+
This needs a `workspaceFiles` adapter. The agent core stays dependency‑free, so
|
|
200
|
+
whoever holds a workspace connection supplies a few‑line adapter:
|
|
184
201
|
|
|
185
202
|
```ts
|
|
186
|
-
const agent = new CoderAgent({
|
|
203
|
+
const agent = new CoderAgent({ organizationId, workspaceId: ws.id, workspaceFiles });
|
|
187
204
|
const { path } = await agent.uploadToWorkspace({
|
|
188
205
|
content: await openAsBlob("assets.zip"),
|
|
189
206
|
path: "assets.zip",
|
|
@@ -191,86 +208,56 @@ const { path } = await agent.uploadToWorkspace({
|
|
|
191
208
|
// Then ask the agent to `unzip assets.zip` — uploadToWorkspace writes bytes as-is; it does not unpack.
|
|
192
209
|
```
|
|
193
210
|
|
|
194
|
-
##
|
|
195
|
-
|
|
196
|
-
Pass a Coder **API token** or **session token** as `token`; it is sent as the
|
|
197
|
-
`Coder-Session-Token` header (REST) and used to authenticate the streaming WebSocket. Create
|
|
198
|
-
a token with `coder tokens create`, or reuse your CLI session.
|
|
199
|
-
|
|
200
|
-
When `baseUrl`/`token` are not passed, they default from the `CODER_URL` and
|
|
201
|
-
`CODER_SESSION_TOKEN` environment variables (the same convention as
|
|
202
|
-
`@coder/ai-sdk-sandbox`'s transports); explicit settings win over the
|
|
203
|
-
environment. With the variables set, connection config disappears entirely:
|
|
204
|
-
|
|
205
|
-
```ts
|
|
206
|
-
const agent = new CoderAgent({ organizationId }); // uses CODER_URL + CODER_SESSION_TOKEN
|
|
207
|
-
```
|
|
208
|
-
|
|
209
|
-
You can also pass a pre‑built client:
|
|
210
|
-
|
|
211
|
-
```ts
|
|
212
|
-
import { CoderAgent, CoderChatClient } from "@coder/ai-sdk-agent";
|
|
213
|
-
const client = new CoderChatClient({ baseUrl, token });
|
|
214
|
-
const agent = new CoderAgent({ client, organizationId });
|
|
215
|
-
```
|
|
211
|
+
## Structured output
|
|
216
212
|
|
|
217
|
-
|
|
213
|
+
Coder Agents has no server‑side `response_format`, so `CoderAgent` cannot
|
|
214
|
+
constrain what the model **says** to a JSON schema. A `responseFormat` /
|
|
215
|
+
`experimental_output` request emits a warning and is best‑effort at most.
|
|
218
216
|
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
217
|
+
- **Pure text‑in / JSON‑out, no server‑side tools** → use
|
|
218
|
+
[`@coder/ai-sdk-provider`](https://github.com/coder/ai-sdk/tree/main/packages/provider) with `generateObject` /
|
|
219
|
+
`Output.object` (schema‑constrained; requires AI Gateway on the deployment).
|
|
220
|
+
- **The answer must come out of an agent run** (server‑side tools, MCP, a
|
|
221
|
+
workspace) → have the model submit its answer by _calling a tool_ whose
|
|
222
|
+
`inputSchema` is your Zod schema. The answer arrives as the tool call's typed
|
|
223
|
+
`input`.
|
|
225
224
|
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
`
|
|
229
|
-
only on 404. The first non-404 HTTP response
|
|
230
|
-
(including an error) locks that prefix for the client's lifetime; concurrent
|
|
231
|
-
unresolved calls may probe independently, with the first conclusive response
|
|
232
|
-
winning. Two 404s preserve the original v2 error without locking; non-HTTP
|
|
233
|
-
failures do not lock either. A locked prefix is never renegotiated, even on 404.
|
|
234
|
-
Before opening a per-chat `/stream` or global `/watch` WebSocket, an unresolved
|
|
235
|
-
client preflights with `GET /api/v2/chats` using the same selection rules and
|
|
236
|
-
cancellation signal.
|
|
237
|
-
`ReadableStream` uploads also preflight so their consumed bodies are never
|
|
238
|
-
replayed. Legacy model-config lookup remains independent of chat-prefix
|
|
239
|
-
selection (see [Configuration](#configuration)).
|
|
225
|
+
The tool pattern needs four rules to stay robust (validate client‑side, never
|
|
226
|
+
stop on the call, and more). Guide: [docs/structured-output.md](./docs/structured-output.md).
|
|
227
|
+
Copyable helper: [`examples/06-structured-output.ts`](https://github.com/coder/ai-sdk/blob/main/packages/agent/examples/06-structured-output.ts).
|
|
240
228
|
|
|
241
229
|
## Sessions
|
|
242
230
|
|
|
243
|
-
One `CoderAgent` instance maps to one chat ("session") on the Coder server. The
|
|
244
|
-
created on the first turn and reused
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
231
|
+
One `CoderAgent` instance maps to one chat ("session") on the Coder server. The
|
|
232
|
+
chat is created on the first turn and reused by later `generate()`/`stream()`
|
|
233
|
+
calls, with history kept server‑side. `agent.chatId` is the current chat id.
|
|
234
|
+
|
|
235
|
+
| Call | What it does |
|
|
236
|
+
| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
237
|
+
| `agent.resetSession()` | Start a fresh chat on the next turn. Reuse one instance for sequential turns; you don't need a new agent per turn. |
|
|
238
|
+
| `agent.interrupt({ signal? })` | Interrupt an in‑flight generation. |
|
|
239
|
+
| `agent.archive({ signal? })` | Archive the underlying chat ([Cleanup](#cleanup)). |
|
|
240
|
+
| `agent.listModels()` | List the organization's model configs, so you don't have to guess the `model` hint. |
|
|
241
|
+
| `new CoderAgent({ …, chatId: "…" })` | Resume a prior chat. Optionally pass `lastSeenMessageId` (the persisted resume cursor, read from `agent.lastSeenMessageId`) to skip the resumed turn's pre‑prompt history probe. Full how‑to: [Durable workflows](./docs/durable-workflows.md). |
|
|
242
|
+
|
|
243
|
+
**One generation at a time.** A single instance is **single‑flight**: don't run
|
|
244
|
+
concurrent generations against it. For concurrency, use one instance per session
|
|
245
|
+
(and see [Workspaces & quota](./docs/workspaces-and-quota.md)).
|
|
246
|
+
|
|
247
|
+
**Interrupting is asynchronous on the server.** `interrupt()` resolves as soon
|
|
248
|
+
as the interrupt is acknowledged; the run keeps winding down for a few seconds
|
|
249
|
+
afterwards. The client‑level `client.interruptChat(chatId, { wait: true })`
|
|
250
|
+
sends `?wait=true` to ask the server to hold the response until the run has
|
|
251
|
+
stopped, but current Coder servers ignore the unknown parameter and still return
|
|
252
|
+
immediately. Confirm completion via the event stream (e.g.
|
|
253
|
+
[`watchChats`](#watching-chats)) instead.
|
|
266
254
|
|
|
267
255
|
## Rehydrating chat history
|
|
268
256
|
|
|
269
|
-
|
|
270
|
-
a
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
of the same turn looks like:
|
|
257
|
+
To render an existing chat in a UI (e.g. after a reload), fetch its messages with
|
|
258
|
+
a `CoderChatClient` (`agent.client`, or one you construct — see [Auth](#auth))
|
|
259
|
+
and convert them with `chatMessagesToUIMessages`. The result mirrors a
|
|
260
|
+
live‑streamed transcript of the same turn:
|
|
274
261
|
|
|
275
262
|
```ts
|
|
276
263
|
import { chatMessagesToUIMessages } from "@coder/ai-sdk-agent";
|
|
@@ -280,22 +267,21 @@ const uiMessages = chatMessagesToUIMessages(messages);
|
|
|
280
267
|
// e.g. in React: useChat({ messages: uiMessages })
|
|
281
268
|
```
|
|
282
269
|
|
|
283
|
-
The converter sorts by message id, so the endpoint's
|
|
284
|
-
|
|
285
|
-
receives a chronological transcript.
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
`
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
up without a URL are skipped:
|
|
270
|
+
- **Any page order works.** The converter sorts by message id, so the endpoint's
|
|
271
|
+
newest‑first default (and any pagination order) is safe to pass straight in.
|
|
272
|
+
`useChat` always receives a chronological transcript.
|
|
273
|
+
- **Mapping.** Tool calls become `dynamic-tool` parts with their results folded
|
|
274
|
+
in, and `source` parts become `source-url` parts. Unknown part kinds are skipped
|
|
275
|
+
silently, so history written by newer Coder servers degrades gracefully.
|
|
276
|
+
- **Every tool call rehydrates as `dynamic-tool`.** History doesn't record which
|
|
277
|
+
tool names were client (`ToolSet`) tools; live, client tools stream as
|
|
278
|
+
statically typed `tool-{name}` parts. Render tools by name (ai's
|
|
279
|
+
`isToolOrDynamicToolUIPart` and `getToolOrDynamicToolName`), not by exact
|
|
280
|
+
`part.type`, and the difference disappears.
|
|
281
|
+
- **Files need a `fileUrl` resolver.** Persisted `file` parts carry only a
|
|
282
|
+
`file_id` (no bytes, usually no URL). Download the bytes with
|
|
283
|
+
`client.getChatFile(fileId)` and return a data:/object/proxy URL. Parts left
|
|
284
|
+
without a URL are skipped:
|
|
299
285
|
|
|
300
286
|
```ts
|
|
301
287
|
chatMessagesToUIMessages(messages, {
|
|
@@ -306,8 +292,8 @@ chatMessagesToUIMessages(messages, {
|
|
|
306
292
|
## Watching chats
|
|
307
293
|
|
|
308
294
|
`client.watchChats({ signal })` yields lifecycle events (status/title changes,
|
|
309
|
-
creation, deletion, …) for **every chat visible to the authenticated user
|
|
310
|
-
async iterable
|
|
295
|
+
creation, deletion, …) for **every chat visible to the authenticated user**. It
|
|
296
|
+
is an async iterable backed by the `/api/v2/chats/watch` WebSocket:
|
|
311
297
|
|
|
312
298
|
```ts
|
|
313
299
|
for await (const event of client.watchChats({ signal })) {
|
|
@@ -315,601 +301,110 @@ for await (const event of client.watchChats({ signal })) {
|
|
|
315
301
|
}
|
|
316
302
|
```
|
|
317
303
|
|
|
318
|
-
After [prefix selection](#api-compatibility) (whose preflight
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
Aborting the signal or ending iteration tears down the reader.
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
`watchChatEvents({ baseUrl, token, signal, webSocketFactory })` export defaults
|
|
326
|
-
to the stable prefix without preflighting; use `CoderChatClient` for automatic
|
|
327
|
-
prefix selection.
|
|
328
|
-
|
|
329
|
-
## Observability
|
|
330
|
-
|
|
331
|
-
`onTransportEvent` receives typed transport events — HTTP exchanges, the
|
|
332
|
-
per‑chat stream's WebSocket lifecycle, and turn‑segment boundaries — so timing
|
|
333
|
-
and tracing need no `fetch`/`webSocketFactory` wrapping and no re‑parsing of
|
|
334
|
-
stream frames. Pass it on `CoderAgentSettings` (it reaches both the client and
|
|
335
|
-
the model), on `CoderChatClientOptions`, or on `CoderLanguageModelConfig`:
|
|
336
|
-
|
|
337
|
-
```ts
|
|
338
|
-
import { CoderAgent, type CoderTransportEvent } from "@coder/ai-sdk-agent";
|
|
339
|
-
|
|
340
|
-
const events: CoderTransportEvent[] = [];
|
|
341
|
-
const agent = new CoderAgent({
|
|
342
|
-
baseUrl,
|
|
343
|
-
token,
|
|
344
|
-
organizationId,
|
|
345
|
-
onTransportEvent: (ev) => events.push(ev),
|
|
346
|
-
});
|
|
347
|
-
|
|
348
|
-
await agent.generate({ prompt: "…" });
|
|
349
|
-
|
|
350
|
-
// e.g. attribute where the turn spent its time:
|
|
351
|
-
for (const ev of events) {
|
|
352
|
-
if (ev.type === "http:response")
|
|
353
|
-
console.log(
|
|
354
|
-
`${ev.op}: ${ev.method} ${ev.path} → ${ev.status} in ${ev.durationMs.toFixed(0)}ms`,
|
|
355
|
-
);
|
|
356
|
-
if (ev.type === "ws:event" && ev.event.type === "action_required")
|
|
357
|
-
console.log(`tool calls arrived at +${ev.timestamp - events[0]!.timestamp}ms`);
|
|
358
|
-
if (ev.type === "segment:settle")
|
|
359
|
-
console.log(`segment ${ev.segment}: ${ev.status} in ${ev.durationMs.toFixed(0)}ms`);
|
|
360
|
-
}
|
|
361
|
-
```
|
|
362
|
-
|
|
363
|
-
`CoderTransportEvent` is a discriminated union on `type`. Every event carries
|
|
364
|
-
`timestamp` (`Date.now()` at observation — comparable to server‑side timestamps
|
|
365
|
-
such as a message's `created_at`, for delivery‑lag measurements):
|
|
366
|
-
|
|
367
|
-
| event | when | payload (besides `timestamp`) |
|
|
368
|
-
| ---------------- | --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
369
|
-
| `http:request` | a REST request is sent | `id` (correlates the pair), `op` (the client operation, e.g. `"createChatMessage"`), `method`, `path` |
|
|
370
|
-
| `http:response` | response headers arrive (incl. non‑2xx, `ok:false`) | `id`, `op`, `method`, `path`, `status`, `ok`, `durationMs` |
|
|
371
|
-
| `http:error` | the fetch itself rejects (network failure, abort) | `id`, `op`, `method`, `path`, `message`, `durationMs` |
|
|
372
|
-
| `ws:dial` | a stream connection attempt starts | `chatId`, `reader` (identifies the `streamChatEvents` call), `attempt` (1‑based per reader, increments per redial), `url` |
|
|
373
|
-
| `ws:open` | the WebSocket handshake completes | `chatId`, `reader`, `attempt` |
|
|
374
|
-
| `ws:event` | a decoded stream event arrives | `chatId`, `reader`, `attempt`, `event` (the decoded `ChatStreamEvent`, by reference — don't mutate), `forwarded` (the reader's replay verdict) |
|
|
375
|
-
| `ws:close` | the connection ends (exactly one per dial) | `chatId`, `reader`, `attempt`, `code`/`reason` when the server/network closed it; absent when the reader closed it (settle, teardown, redial) |
|
|
376
|
-
| `ws:error` | a socket error or unparseable frame | `chatId`, `reader`, `attempt`, `message` |
|
|
377
|
-
| `ws:redial` | a dropped connection is about to be redialed | `chatId`, `reader`, `attempt` (the ended connection), `consecutiveFailures`, `maxConsecutiveFailures`, `backoffMs` |
|
|
378
|
-
| `segment:start` | a turn segment (one model round‑trip) starts | `segment` (1‑based per model instance), `chatId` (absent before the first turn creates the chat) |
|
|
379
|
-
| `segment:settle` | the segment ends (exactly one per start) | `segment`, `chatId`, `reader` (the reader that served the segment; absent if none was acquired), `durationMs`, and: `status` + `finishReason` on a clean settle, `error` (`{name, message}`, plus `status` if the run still settled terminally) on failure, neither on a teardown |
|
|
380
|
-
|
|
381
|
-
Semantics worth knowing:
|
|
382
|
-
|
|
383
|
-
- **Isolation** — exceptions thrown by the handler are swallowed; they can never
|
|
384
|
-
alter transport behavior or a turn's outcome.
|
|
385
|
-
- **Zero overhead** — without a handler, no event objects are allocated and no
|
|
386
|
-
extra socket listeners are registered.
|
|
387
|
-
- **No secrets** — events carry no headers and no tokens (auth travels in the
|
|
388
|
-
`Coder-Session-Token` header, which is deliberately excluded); `path`/`url`
|
|
389
|
-
never contain credentials.
|
|
390
|
-
- `http:*` events name their operation: `op` is the public `CoderChatClient`
|
|
391
|
-
method performing the exchange (`"createChat"`, `"createChatMessage"`,
|
|
392
|
-
`"getMessages"`, `"submitToolResults"`, … — the `CoderClientOperation`
|
|
393
|
-
union), so per‑operation classification never has to reverse‑engineer
|
|
394
|
-
`path`. `method`/`path` stay for generic consumers; `archiveChat` stamps its
|
|
395
|
-
own `op` even though it issues the same `PATCH` as `updateChat`. WebSocket
|
|
396
|
-
prefix-selection preflights emit the existing `http:*` event kinds with
|
|
397
|
-
`op: "streamEvents"` or `op: "watchChats"` — no new event kinds.
|
|
398
|
-
- `ws:event` fires at arrival: after a redial, chatd's replay of the
|
|
399
|
-
in‑progress episode is visible here (correlate with `reader`/`attempt`),
|
|
400
|
-
stamped with the reader's own replay verdict — `forwarded: false` exactly on
|
|
401
|
-
the duplicate deltas the reader suppresses from the turn, so subscribers
|
|
402
|
-
never re‑derive the episode filter.
|
|
403
|
-
- `forwarded` does **not** reflect snapshot dedup: repeated or revised
|
|
404
|
-
`message` snapshots are always `forwarded: true`, because reconciling them
|
|
405
|
-
is deliberately the consumer's job past the transport layer —
|
|
406
|
-
`TurnTranslator`'s per‑message ledger decides what a revision re‑emits, and
|
|
407
|
-
that disposition is not stamped on transport frames. Use `ws:event` for
|
|
408
|
-
span pairing and replay accounting; subscribers needing content fidelity
|
|
409
|
-
must consume model output (or `TurnTranslator`), not transport frames.
|
|
410
|
-
- Every `ws:*` event carries `reader` — a monotonic id for the
|
|
411
|
-
`streamChatEvents` call (reader) behind the connection, allocated from one
|
|
412
|
-
process‑wide counter so it stays unique across model and client instances.
|
|
413
|
-
`attempt` restarts at 1 per reader, so identify a connection as
|
|
414
|
-
`(chatId, reader, attempt)`, never `(chatId, attempt)` alone: a client‑tool
|
|
415
|
-
pause the caller abandons is closed fire‑and‑forget when the next turn dials
|
|
416
|
-
its replacement, and the superseded reader's late `ws:close` (or a raced‑in
|
|
417
|
-
frame) can emit after the new reader's `ws:dial` — the reader id is what
|
|
418
|
-
tells them apart. `segment:settle` names the reader that served the segment;
|
|
419
|
-
`segment:start` predates stream acquisition and carries none.
|
|
420
|
-
- A multi‑step turn that drives client tools emits one `segment:start`/
|
|
421
|
-
`segment:settle` pair per round‑trip, all riding **one** `ws:dial`ed
|
|
422
|
-
connection — the stream is retained across `requires_action` pauses. A pause
|
|
423
|
-
settles with `status: "requires_action"`, `finishReason: "tool-calls"`; the
|
|
424
|
-
final settle carries the terminal status (`waiting`/`completed`/`error`).
|
|
425
|
-
- `ws:*` events cover the per‑chat `/stream` reader (turn transport). The
|
|
426
|
-
`watchChats` subscription is not instrumented.
|
|
427
|
-
- With a pre‑built `client` in `CoderAgentSettings`, HTTP/WS events come from
|
|
428
|
-
the hook given to **that client's** options; the agent‑level hook then only
|
|
429
|
-
receives `segment:*` events.
|
|
304
|
+
- **Long‑lived.** After [prefix selection](#api-compatibility) (whose preflight
|
|
305
|
+
errors propagate), dropped connections are redialed automatically with
|
|
306
|
+
exponential backoff: 1s doubling to a 30s cap, reset once an event arrives.
|
|
307
|
+
- **Teardown.** Aborting the signal or ending iteration tears down the reader.
|
|
308
|
+
- **Terminal errors.** A 4xx upgrade rejection throws a terminal
|
|
309
|
+
`CoderApiError`: a bad/expired token, or an older Coder server without the
|
|
310
|
+
endpoint (404).
|
|
430
311
|
|
|
431
|
-
|
|
312
|
+
<details>
|
|
313
|
+
<summary>Custom plumbing: <code>watchChatEvents</code></summary>
|
|
432
314
|
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
315
|
+
For custom plumbing (own client, browser sockets), the standalone
|
|
316
|
+
`watchChatEvents({ baseUrl, token, signal, webSocketFactory })` export defaults
|
|
317
|
+
to the stable prefix without preflighting. Use `CoderChatClient` for automatic
|
|
318
|
+
prefix selection.
|
|
437
319
|
|
|
438
|
-
|
|
439
|
-
server is wedged, or a workspace can't be scheduled), the run is interrupted and
|
|
440
|
-
the call rejects with a retryable `CoderChatError` (`kind: "timeout"`) instead of
|
|
441
|
-
hanging:
|
|
320
|
+
</details>
|
|
442
321
|
|
|
443
|
-
|
|
444
|
-
const agent = new CoderAgent({ /* … */ requestTimeoutMs: 120_000 });
|
|
445
|
-
```
|
|
322
|
+
## Workspace previews
|
|
446
323
|
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
**total** wall‑clock of a multi‑step call, pass a deadline as the signal instead:
|
|
324
|
+
When the agent is bound to a workspace (the `workspaceId` setting), resolve — and
|
|
325
|
+
share — the browser URL where a workspace port is served, e.g. the dev server the
|
|
326
|
+
agent just started:
|
|
451
327
|
|
|
452
328
|
```ts
|
|
453
|
-
await agent.
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
If the event stream drops mid‑turn, the agent redials it automatically with
|
|
457
|
-
exponential backoff, replaying the turn's events from its starting cursor and
|
|
458
|
-
deduplicating them on receipt — the server keeps generating during the gap, so
|
|
459
|
-
a transient drop costs nothing and the run is **not** interrupted. Only when the stream cannot be re‑established (several
|
|
460
|
-
consecutive failed attempts, ~15s) is the server run interrupted and the call
|
|
461
|
-
rejected with a `CoderStreamError` — an AI SDK `APICallError`. When the failed
|
|
462
|
-
turn had just created its chat AND had no external effects a replay would
|
|
463
|
-
repeat — no `workspaceId`, no `mcpServerIds`, and no freshly uploaded inline
|
|
464
|
-
attachments (pre‑uploaded `fileId` references are fine) — the dead session is
|
|
465
|
-
discarded and the error is `isRetryable: true`, so `generate()` calls with
|
|
466
|
-
`maxRetries` set retry the whole turn on a fresh chat automatically. Otherwise
|
|
467
|
-
the error is `isRetryable: false` and retrying is the caller's deliberate
|
|
468
|
-
decision: on a chat with prior state (resumed sessions, later turns,
|
|
469
|
-
tool‑result segments) a re‑invocation would resubmit the same prompt as a new
|
|
470
|
-
user turn, workspace/MCP tools may already have executed side effects, and
|
|
471
|
-
inline attachments would upload again. (For `stream()`, a mid‑stream failure
|
|
472
|
-
surfaces on the stream itself, outside the SDK's retry wrapper — handle it in
|
|
473
|
-
your consumption loop.)
|
|
474
|
-
A non‑transient 4xx upgrade rejection (bad/expired token,
|
|
475
|
-
deleted chat) fails fast with a `CoderApiError` instead of retrying; 408/425/429
|
|
476
|
-
consume the redial budget like any other transient failure.
|
|
477
|
-
|
|
478
|
-
## Cleanup
|
|
479
|
-
|
|
480
|
-
`archive()` soft‑hides the chat (it stays in listings as `archived: true`; there
|
|
481
|
-
is no hard delete yet). A freshly interrupted chat keeps winding down server‑side
|
|
482
|
-
for a few seconds, during which archiving 409s — `archive()` retries those 409s
|
|
483
|
-
(~1s apart, up to ~15s overall; tune with `settleDeadlineMs` /
|
|
484
|
-
`settleRetryDelayMs`) and rethrows the last one if the chat never settles. Any
|
|
485
|
-
other failure, including your own abort, rethrows immediately.
|
|
486
|
-
|
|
487
|
-
`archive()` and `interrupt()` target the session's chat — or, after the session
|
|
488
|
-
was dropped (`resetSession()`, or the automatic discard after a fresh‑chat
|
|
489
|
-
stream failure — see [Handling errors](#handling-errors)), the **last‑known
|
|
490
|
-
chat id** (`agent.lastKnownChatId`), so a stranded chat is still cleaned up
|
|
491
|
-
instead of leaking. Generation never uses the last‑known id: a turn after a
|
|
492
|
-
drop creates a fresh chat as always. Every chat stranded by an _automatic_
|
|
493
|
-
discard is also recorded on a ledger (`agent.strandedChatIds`, oldest first) —
|
|
494
|
-
with `maxRetries` several failed attempts can strand one chat each while only
|
|
495
|
-
the final attempt's error surfaces — and one `archive()` retires them all,
|
|
496
|
-
oldest first, alongside its primary target. Both methods report what they
|
|
497
|
-
acted on instead of silently no‑oping: `archive()` resolves
|
|
498
|
-
`{ archived: true, chatId, archivedChatIds }` (each archived id is cleared as
|
|
499
|
-
a cleanup target) or `{ archived: false }` when no chat exists at all;
|
|
500
|
-
`interrupt()` resolves `{ interrupted: true, chatId }` /
|
|
501
|
-
`{ interrupted: false }` the same way. Deliberate abandonment is different:
|
|
502
|
-
`resetSession()` does **not** add to the ledger (you may want that chat kept),
|
|
503
|
-
so after a manual reset the old chat is targetable only until a new chat
|
|
504
|
-
supersedes `lastKnownChatId`.
|
|
505
|
-
|
|
506
|
-
To make cleanup ride scope exit instead of a `finally` you have to remember, the
|
|
507
|
-
agent is an **async disposable**:
|
|
329
|
+
const { url } = await agent.getPreview({ port: 3000 });
|
|
330
|
+
// → https://3000--main--dev--alice.apps.example.com (private to the workspace owner)
|
|
508
331
|
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
const { text } = await agent.generate({ prompt: "…" });
|
|
512
|
-
// agent.interrupt() + agent.archive() run automatically when the scope exits.
|
|
332
|
+
const shared = await agent.sharePreview({ port: 3000, shareLevel: "authenticated" });
|
|
333
|
+
// shared.url is now reachable by any logged-in user; shared.shareLevel is the level in effect
|
|
513
334
|
```
|
|
514
335
|
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
336
|
+
| Method | What it does |
|
|
337
|
+
| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ |
|
|
338
|
+
| `getPreview({ port, agentName?, protocol?, signal? })` | Composes the subdomain URL. The URL honors the port's current share level: private to the workspace owner unless shared. |
|
|
339
|
+
| `sharePreview({ port, shareLevel?, … })` | Additionally upserts the port's share level (re‑invoking updates it in place) and returns the level in effect. |
|
|
519
340
|
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
341
|
+
- Ports below 1000 are rejected up front. Coder subdomain URLs only encode 4–5
|
|
342
|
+
digit ports, so `80--agent--…` would be parsed as an app named "80" and never
|
|
343
|
+
resolve; serve the preview on a higher port.
|
|
344
|
+
- `agentName` is optional when the workspace has exactly one agent. With
|
|
345
|
+
several, the error lists the candidates.
|
|
346
|
+
- `protocol: "https"` means the app speaks TLS _inside the workspace_. It adds
|
|
347
|
+
the `s` label suffix (`3000s--…`) and does not affect the browser scheme.
|
|
348
|
+
- `shareLevel` values:
|
|
523
349
|
|
|
524
|
-
|
|
350
|
+
| `shareLevel` | Reachable by |
|
|
351
|
+
| --------------------------- | ---------------------------------------------------------------------- |
|
|
352
|
+
| `"authenticated"` (default) | Any logged‑in user |
|
|
353
|
+
| `"organization"` | Members of the workspace's organization; requires a newer Coder server |
|
|
354
|
+
| `"public"` | Anyone — no auth at all; mind what the port serves |
|
|
525
355
|
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
529
|
-
- **`CoderApiError`** — an HTTP request failed. Fields: `status`, `method`, `path`, `detail`.
|
|
530
|
-
- **`CoderChatError`** — a turn ended in an error, timed out, or lost its stream. Fields: `kind`, `retryable`, `statusCode`, `provider`.
|
|
531
|
-
- **`CoderStreamError`** — the event stream dropped and could not be re‑established
|
|
532
|
-
within its redial budget. Extends the AI SDK's `APICallError` (not
|
|
533
|
-
`CoderAgentError`), so `generate()`'s `maxRetries` machinery recognizes it.
|
|
534
|
-
`isRetryable` is `true` only when the failed turn created its chat (the dead
|
|
535
|
-
session is discarded, so a retry starts fresh) and had no external effects to
|
|
536
|
-
repeat (workspace/MCP tooling, fresh attachment uploads); otherwise it is
|
|
537
|
-
`false` — a retry would resubmit the prompt as a new user turn and could
|
|
538
|
-
duplicate those effects. A failure mid‑`stream()` surfaces on the stream,
|
|
539
|
-
outside the retry wrapper. The last transport failure is in `cause`, and
|
|
540
|
-
`chatId` names the chat the failed turn had created or attached to (absent
|
|
541
|
-
when it failed before a chat existed) — after the fresh‑chat discard the
|
|
542
|
-
chat still exists server‑side, and `archive()` keeps targeting it via
|
|
543
|
-
`agent.lastKnownChatId` ([Cleanup](#cleanup)).
|
|
356
|
+
Reverting to owner‑only means deleting the share; `"owner"` is not accepted on
|
|
357
|
+
upsert.
|
|
544
358
|
|
|
545
|
-
|
|
546
|
-
|
|
359
|
+
**Credentials.** The preview helpers call non‑chat endpoints, so they need
|
|
360
|
+
`baseUrl` + `token`. Pass them alongside `client` if you construct one yourself
|
|
361
|
+
(or let them default from `CODER_URL`/`CODER_SESSION_TOKEN`).
|
|
547
362
|
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
} catch (err) {
|
|
551
|
-
if (err instanceof CoderChatError && err.retryable) {
|
|
552
|
-
// transient (timeout, stream_closed, an upstream 5xx) — back off and retry
|
|
553
|
-
} else if (err instanceof CoderApiError && err.status === 429) {
|
|
554
|
-
// rate limited
|
|
555
|
-
} else {
|
|
556
|
-
throw err;
|
|
557
|
-
}
|
|
558
|
-
}
|
|
559
|
-
```
|
|
363
|
+
<details>
|
|
364
|
+
<summary>Compatibility and failure cases (wildcard access URL, Coder < v2.9)</summary>
|
|
560
365
|
|
|
561
|
-
|
|
562
|
-
|
|
563
|
-
|
|
366
|
+
Both helpers are built on the stable v2 workspace APIs (workspace lookup + the
|
|
367
|
+
wildcard apps host; `sharePreview` adds a port‑share upsert), so they work
|
|
368
|
+
against old Coder servers — no experimental endpoints.
|
|
564
369
|
|
|
565
|
-
|
|
370
|
+
They fail clearly instead of returning broken URLs:
|
|
566
371
|
|
|
567
|
-
|
|
568
|
-
|
|
569
|
-
|
|
570
|
-
|
|
571
|
-
turn actually consumed. `inputTokens` is the full prompt size: Coder
|
|
572
|
-
normalizes the wire `input_tokens` to the _uncached_ count (cache reads/writes
|
|
573
|
-
are separate fields), so the SDK adds them back into the total and exposes the
|
|
574
|
-
split via `inputTokenDetails` (`noCacheTokens`, `cacheReadTokens`,
|
|
575
|
-
`cacheWriteTokens`).
|
|
576
|
-
|
|
577
|
-
The snake_case wire usage lives **per step** at `result.steps[i].usage.raw`
|
|
578
|
-
(the AI SDK does not carry `raw` onto the summed `result.usage`) for fields
|
|
579
|
-
the normalized shape has no slot for (`context_limit`, cost, runtime, and any
|
|
580
|
-
newer wire fields, which pass through newest‑value‑wins). `raw` keeps the wire
|
|
581
|
-
convention (`input_tokens` = uncached only) with counters summed over that
|
|
582
|
-
step's server‑side model steps and `context_limit` from the newest one — so
|
|
583
|
-
don't divide `raw`'s summed counters by `context_limit` to estimate context
|
|
584
|
-
fullness; they are turn consumption, not a prompt‑size snapshot. When the
|
|
585
|
-
server reports them, `total_cost_micros` (micro‑USD) and `total_runtime_ms`
|
|
586
|
-
are mirrored the same way under each step's `providerMetadata.coder`
|
|
587
|
-
(`result.providerMetadata` reflects only the final step — sum
|
|
588
|
-
`result.steps[*].providerMetadata.coder` for whole‑turn cost when client tools
|
|
589
|
-
ran). Both are **absence‑tolerant mirrors**: on servers that don't send them
|
|
590
|
-
(cost is otherwise only on the aggregate cost endpoints,
|
|
591
|
-
`/api/v2/chats/cost/*`), nothing is emitted.
|
|
372
|
+
- A deployment without a wildcard access URL (`--wildcard-access-url`) yields an
|
|
373
|
+
explanatory error.
|
|
374
|
+
- A server that predates port sharing (< Coder v2.9) yields a 404
|
|
375
|
+
`CoderApiError` saying so.
|
|
592
376
|
|
|
593
|
-
|
|
594
|
-
|
|
595
|
-
```ts
|
|
596
|
-
const result = await agent.stream({ prompt: "…" });
|
|
597
|
-
return result.toUIMessageStream({
|
|
598
|
-
messageMetadata: ({ part }) =>
|
|
599
|
-
part.type === "finish-step"
|
|
600
|
-
? { usage: part.usage, coder: part.providerMetadata?.coder }
|
|
601
|
-
: undefined,
|
|
602
|
-
});
|
|
603
|
-
```
|
|
377
|
+
</details>
|
|
604
378
|
|
|
605
379
|
## Sources
|
|
606
380
|
|
|
607
|
-
Model configs with web search enabled emit `source` parts.
|
|
608
|
-
`result.sources` and, in UI message streams, `source-url` parts
|
|
609
|
-
`sendSources: true` to `toUIMessageStream
|
|
381
|
+
Model configs with web search enabled emit `source` parts. They flow through to
|
|
382
|
+
`result.sources` and, in UI message streams, to `source-url` parts. Pass
|
|
383
|
+
`sendSources: true` to `toUIMessageStream`; the AI SDK omits them by default.
|
|
610
384
|
Earlier releases dropped them.
|
|
611
385
|
|
|
612
|
-
##
|
|
386
|
+
## Auth
|
|
613
387
|
|
|
614
|
-
Coder
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
by what the step needs:
|
|
388
|
+
Pass a Coder **API token** or **session token** as `token`. It is sent as the
|
|
389
|
+
`Coder-Session-Token` header (REST) and authenticates the streaming WebSocket.
|
|
390
|
+
Create one with `coder tokens create`, or reuse your CLI session.
|
|
618
391
|
|
|
619
|
-
|
|
620
|
-
|
|
621
|
-
|
|
622
|
-
- **The answer must come out of an agent run** (server‑side tools, MCP, a
|
|
623
|
-
workspace) → use the **`structured_output` tool pattern** below. What the
|
|
624
|
-
model _says_ isn't schema‑constrained, but what it passes **into a tool** is
|
|
625
|
-
typed — so have it submit its answer by _calling a tool_ whose `inputSchema`
|
|
626
|
-
is your Zod schema. The answer arrives as the tool call's typed `input`; no
|
|
627
|
-
fishing JSON out of prose.
|
|
392
|
+
`baseUrl`/`token` default from the `CODER_URL` and `CODER_SESSION_TOKEN`
|
|
393
|
+
environment variables, the same convention as `@coder/ai-sdk-sandbox`'s
|
|
394
|
+
transports. Explicit settings win over the environment:
|
|
628
395
|
|
|
629
396
|
```ts
|
|
630
|
-
|
|
631
|
-
import { z } from "zod";
|
|
632
|
-
|
|
633
|
-
const Answer = z.object({ severity: z.enum(["critical", "major", "minor"]), summary: z.string() });
|
|
634
|
-
|
|
635
|
-
const agent = new CoderAgent({
|
|
636
|
-
/* … */
|
|
637
|
-
instructions: "… Submit your final answer by calling the structured_output tool exactly once.",
|
|
638
|
-
tools: {
|
|
639
|
-
structured_output: tool({
|
|
640
|
-
description:
|
|
641
|
-
"Submit your final structured answer as JSON. Call this exactly once, when your work is complete.",
|
|
642
|
-
inputSchema: Answer, // your schema IS the tool's input schema
|
|
643
|
-
// Ack instead of stopping the turn: the model finishes naturally and can
|
|
644
|
-
// wind down anything it still has running (dev servers, watchers, …).
|
|
645
|
-
execute: async () =>
|
|
646
|
-
"Output received. Wind down and end your turn. Do not call structured_output again.",
|
|
647
|
-
}),
|
|
648
|
-
},
|
|
649
|
-
stopWhen: stepCountIs(6), // happy path is 2 steps: file + ack, wind down
|
|
650
|
-
});
|
|
651
|
-
|
|
652
|
-
const result = await agent.generate({ prompt: "…" });
|
|
653
|
-
// toolCalls only holds the LAST step's calls — scan all steps. Take the last call
|
|
654
|
-
// that VALIDATES: a schema-invalid re-file must not shadow a valid answer (rule 2).
|
|
655
|
-
const filed = result.steps
|
|
656
|
-
.flatMap((s) => s.toolCalls)
|
|
657
|
-
.filter((c) => c.toolName === "structured_output");
|
|
658
|
-
let answer: z.infer<typeof Answer> | undefined;
|
|
659
|
-
for (const call of filed.reverse()) {
|
|
660
|
-
const parsed = Answer.safeParse(call.input);
|
|
661
|
-
if (parsed.success) {
|
|
662
|
-
answer = parsed.data; // typed: { severity: "critical" | "major" | "minor"; summary: string }
|
|
663
|
-
break;
|
|
664
|
-
}
|
|
665
|
-
}
|
|
666
|
-
if (answer === undefined)
|
|
667
|
-
throw new Error("no valid structured_output call — nudge once on an idle chat (rule 3)");
|
|
397
|
+
const agent = new CoderAgent({ organizationId }); // uses CODER_URL + CODER_SESSION_TOKEN
|
|
668
398
|
```
|
|
669
399
|
|
|
670
|
-
|
|
671
|
-
|
|
672
|
-
1. **Don't force `toolChoice`, don't stop on the call.** `toolChoice` is
|
|
673
|
-
construction‑time and applies to _every_ segment, so after the ack it would
|
|
674
|
-
force the tool again and again up to the step ceiling (and it blocks any
|
|
675
|
-
other tools the step needs). A `hasToolCall` stop is worse: the server only
|
|
676
|
-
receives a client tool result as a side effect of the _next_ loop segment,
|
|
677
|
-
so ending the loop on the call strands the chat in `requires_action` —
|
|
678
|
-
follow‑up messages queue forever and `archive()` 409s. Instructions plus the
|
|
679
|
-
tool's own description are enough; models file unprompted most of the time.
|
|
680
|
-
2. **Validate client‑side.** The schema is not enforced server‑side —
|
|
681
|
-
`schema.safeParse` on the tool input is the real gate. (Schema‑invalid calls
|
|
682
|
-
that the AI SDK catches in‑loop are automatically answered with a
|
|
683
|
-
`tool-error` result the model retries against.)
|
|
684
|
-
3. **Nudge at most once, and only an idle chat.** If the turn ends in prose
|
|
685
|
-
(`finishReason: "stop"`) without a valid call, send one typed re‑prompt
|
|
686
|
-
("Call the structured_output tool now …"), then fail into your normal error
|
|
687
|
-
handling. Never re‑prompt a chat that isn't idle — the message would queue
|
|
688
|
-
behind whatever the server is still doing.
|
|
689
|
-
4. **Settle a turn that stopped on a tool call.** If the loop stops on a
|
|
690
|
-
tool‑call step — e.g. your `stopWhen` ceiling lands exactly on the
|
|
691
|
-
`structured_output` call (`finishReason: "tool-calls"`) — the tool results
|
|
692
|
-
ran locally but never reached the server. Guard on `agent.chatId` (it is
|
|
693
|
-
`undefined` until the first turn creates the chat), then submit the
|
|
694
|
-
stranded step's (`result.steps.at(-1)`) locally‑executed client outcomes
|
|
695
|
-
directly via
|
|
696
|
-
`agent.client.submitToolResults(chatId, { results: [{ tool_call_id, output, is_error }] }, AbortSignal.timeout(8_000))`
|
|
697
|
-
before touching the chat again, or it strands as in rule 1. Read the
|
|
698
|
-
outcomes off the step's **content parts**: a `tool-result` part is a
|
|
699
|
-
success, a `tool-error` part (the tool's `execute` threw) must be submitted
|
|
700
|
-
with `is_error: true` — mirroring what the resume path would have sent. If
|
|
701
|
-
a pending call has no local outcome (or the submit fails), end the stranded
|
|
702
|
-
turn with `agent.client.interruptChat(chatId, AbortSignal.timeout(8_000))`
|
|
703
|
-
instead. **Bound every one of these recovery requests with an
|
|
704
|
-
`AbortSignal`** — they target a server that may already be stalled, and the
|
|
705
|
-
bare `agent.interrupt()` / `agent.archive()` helpers carry no timeout. A
|
|
706
|
-
settled chat resumes its wind‑down server‑side for a few seconds, so retry
|
|
707
|
-
a 409ing archive (`agent.client.archiveChat(chatId, signal)`, per‑attempt
|
|
708
|
-
bound) under a short deadline instead of giving up.
|
|
709
|
-
|
|
710
|
-
[`examples/06-structured-output.ts`](./examples/06-structured-output.ts) packages
|
|
711
|
-
all four rules into a small copyable helper — `structuredOutput(schema)` returns
|
|
712
|
-
`agentOpts` to spread into the constructor plus a typed `ask(agent, prompt)`
|
|
713
|
-
that runs the settle + one‑nudge ladder and returns a `z.infer<typeof schema>`.
|
|
714
|
-
Compose additional client tools through the helper —
|
|
715
|
-
`structuredOutput(schema, { tools: { myTool } })` merges them into one ToolSet —
|
|
716
|
-
rather than passing `tools:` to the constructor next to the spread, where the
|
|
717
|
-
later key silently clobbers the other map.
|
|
718
|
-
|
|
719
|
-
## Workspace previews
|
|
720
|
-
|
|
721
|
-
When the agent is bound to a workspace (the `workspaceId` setting), you can
|
|
722
|
-
resolve — and share — the browser URL where a port on that workspace is served,
|
|
723
|
-
e.g. the dev server the agent just started:
|
|
400
|
+
You can also pass a pre‑built client:
|
|
724
401
|
|
|
725
402
|
```ts
|
|
726
|
-
|
|
727
|
-
|
|
728
|
-
|
|
729
|
-
const shared = await agent.sharePreview({ port: 3000, shareLevel: "authenticated" });
|
|
730
|
-
// shared.url is now reachable by any logged-in user; shared.shareLevel is the level in effect
|
|
403
|
+
import { CoderAgent, CoderChatClient } from "@coder/ai-sdk-agent";
|
|
404
|
+
const client = new CoderChatClient({ baseUrl, token });
|
|
405
|
+
const agent = new CoderAgent({ client, organizationId });
|
|
731
406
|
```
|
|
732
407
|
|
|
733
|
-
Both are built on the stable v2 workspace APIs (workspace lookup + the wildcard
|
|
734
|
-
apps host; `sharePreview` adds a port‑share upsert), so they work against old
|
|
735
|
-
Coder servers — no experimental endpoints.
|
|
736
|
-
|
|
737
|
-
- `getPreview({ port, agentName?, protocol?, signal? })` composes the subdomain
|
|
738
|
-
URL. The URL honors the port's current share level — private to the workspace
|
|
739
|
-
owner unless shared. `agentName` is optional when the workspace has exactly one
|
|
740
|
-
agent (with several, the error lists the candidates); `protocol: "https"` means
|
|
741
|
-
the app speaks TLS _inside the workspace_ (it adds the `s` label suffix,
|
|
742
|
-
`3000s--…`) and does not affect the browser scheme.
|
|
743
|
-
- `sharePreview({ port, shareLevel?, … })` additionally upserts the port's share
|
|
744
|
-
level (re‑invoking updates it in place) and returns the level in effect.
|
|
745
|
-
`shareLevel` is `"authenticated"` (any logged‑in user; the default),
|
|
746
|
-
`"organization"` (members of the workspace's organization; requires a newer
|
|
747
|
-
Coder server), or `"public"` (no auth at all — mind what the port serves).
|
|
748
|
-
Reverting to owner‑only means deleting the share; `"owner"` is not accepted on
|
|
749
|
-
upsert.
|
|
750
|
-
- Clear failures instead of broken URLs: a deployment without a wildcard access
|
|
751
|
-
URL (`--wildcard-access-url`) yields an explanatory error, and a server that
|
|
752
|
-
predates port sharing (< Coder v2.9) yields a 404 `CoderApiError` saying so.
|
|
753
|
-
Ports below 1000 are rejected up front for the same reason — Coder subdomain
|
|
754
|
-
URLs only encode 4–5 digit ports, so `80--agent--…` would be parsed as an app
|
|
755
|
-
named "80" and never resolve; serve the preview on a higher port.
|
|
756
|
-
|
|
757
|
-
The preview helpers call non‑chat endpoints, so they need `baseUrl` + `token`
|
|
758
|
-
credentials — pass them alongside `client` if you construct one yourself (or
|
|
759
|
-
let them default from `CODER_URL`/`CODER_SESSION_TOKEN`).
|
|
760
|
-
|
|
761
|
-
## Workspaces & quota
|
|
762
|
-
|
|
763
|
-
A `CoderAgent` is one server‑side chat, and — depending on its configuration and
|
|
764
|
-
the deployment — a chat may be backed by a **Coder workspace** that runs its
|
|
765
|
-
tools. Workspaces are the scarce resource: a deployment budgets how many an
|
|
766
|
-
account may run at once, so **N agents running concurrently can need N
|
|
767
|
-
schedulable workspaces.** Past that bound, a turn can sit unscheduled and never
|
|
768
|
-
settle. This section is the operational guide for running fleets of agents:
|
|
769
|
-
how the binding works, how to size concurrency, what to clean up, and how to
|
|
770
|
-
diagnose a chat that is stuck.
|
|
771
|
-
|
|
772
|
-
### How a chat binds to a workspace
|
|
773
|
-
|
|
774
|
-
- **One chat, at most one workspace, fixed at creation.** `workspaceId` is sent
|
|
775
|
-
as `workspace_id` when the chat is created; nothing can rebind it afterwards —
|
|
776
|
-
message and update requests carry no workspace field. To move work to another
|
|
777
|
-
workspace, start a new agent/chat.
|
|
778
|
-
- **This SDK never provisions workspaces.** A `workspaceId` you pass must be an
|
|
779
|
-
existing workspace (provision one with
|
|
780
|
-
[`@coder/ai-sdk-sandbox`](../sandbox)'s `ensureCoderWorkspace`, the CLI, or
|
|
781
|
-
the v2 API). A chat created _without_ `workspaceId` can still come back
|
|
782
|
-
workspace‑backed — deployments may assign one server‑side; the SDK reads the
|
|
783
|
-
created chat's `workspace_id` and treats the chat as workspace‑backed from
|
|
784
|
-
then on (which matters for retry ownership, below).
|
|
785
|
-
- **Chat cleanup does not release the workspace.** `archive()` soft‑hides the
|
|
786
|
-
chat only; the workspace keeps running until template autostop or an explicit
|
|
787
|
-
stop — and stopping releases only stop‑scoped quota; persistent resources
|
|
788
|
-
(disks, volumes) keep consuming their cost until the workspace is deleted.
|
|
789
|
-
|
|
790
|
-
### Sizing a fleet
|
|
791
|
-
|
|
792
|
-
The structural rule: **workspaces that must be running concurrently ≤
|
|
793
|
-
schedulable workspaces.** With one workspace per chat — the common fleet shape,
|
|
794
|
-
and what a deployment that auto‑assigns workspaces produces — that means
|
|
795
|
-
concurrent chats ≤ schedulable workspaces. Chats explicitly bound to a shared
|
|
796
|
-
`workspaceId` count that workspace once, so their concurrency is not
|
|
797
|
-
quota‑bound — at the price of sharing one filesystem and tool environment,
|
|
798
|
-
which is only acceptable within a single tenant / trust boundary.
|
|
799
|
-
What counts as "schedulable" is a deployment property, not an SDK knob —
|
|
800
|
-
whichever of these binds first:
|
|
801
|
-
|
|
802
|
-
- **Workspace quota (premium deployments).** Templates declare per‑resource
|
|
803
|
-
costs; a user's budget is the sum of their groups' quota allowances, enforced
|
|
804
|
-
when a workspace build starts or stops. A start that would exceed the budget
|
|
805
|
-
**fails the build** (error code `INSUFFICIENT_QUOTA`, "insufficient quota"),
|
|
806
|
-
so the turn never gets its workspace — see
|
|
807
|
-
[resource quotas](https://coder.com/docs/admin/users/quotas). Note that a
|
|
808
|
-
_stopped_ workspace typically still consumes its persistent resources' cost,
|
|
809
|
-
so a fleet that only ever stops (never deletes) scratch workspaces converges
|
|
810
|
-
on a full budget.
|
|
811
|
-
- **Infrastructure.** Without quotas there is no per‑user workspace limit by
|
|
812
|
-
default ([workspace lifecycle](https://coder.com/docs/user-guides/workspace-lifecycle)) —
|
|
813
|
-
the bound is provisioner throughput and cluster capacity, and exceeding it
|
|
814
|
-
looks like slow or failing builds rather than a crisp quota error.
|
|
815
|
-
|
|
816
|
-
Practical sizing:
|
|
817
|
-
|
|
818
|
-
- Read headroom before fanning out:
|
|
819
|
-
`GET /api/v2/organizations/{org}/members/{user}/workspace-quota` returns
|
|
820
|
-
`{ "credits_consumed": …, "budget": … }`. Quota is denominated in credits,
|
|
821
|
-
not slots: admit another workspace only while `budget − credits_consumed`
|
|
822
|
-
covers _that workspace's_ cost (the sum of its template's `daily_cost`
|
|
823
|
-
declarations). "Free slots = headroom ÷ cost" only holds for a homogeneous
|
|
824
|
-
fleet on one template; with mixed templates, size against each planned
|
|
825
|
-
workspace's own cost.
|
|
826
|
-
- Keep fan‑out width within that headroom and queue the rest client‑side — an
|
|
827
|
-
unschedulable turn does not queue usefully on the server (see
|
|
828
|
-
[Preventing stuck turns](#preventing-stuck-turns)).
|
|
829
|
-
- Reuse one bound workspace across sequential turns and sessions instead of
|
|
830
|
-
provisioning per request — the workspace is the expensive part, the chat is
|
|
831
|
-
cheap. Reuse only **within one tenant / trust boundary**: workspace‑bound
|
|
832
|
-
agents have file and shell tools, so a reused filesystem carries one
|
|
833
|
-
session's artifacts (and secrets) into the next — provision per tenant, or
|
|
834
|
-
securely reset a workspace before reassigning it.
|
|
835
|
-
- Steps that don't need server‑side tools belong on the
|
|
836
|
-
[provider](../provider) — it never touches a workspace.
|
|
837
|
-
|
|
838
|
-
### Autostop & cleanup
|
|
839
|
-
|
|
840
|
-
Two lifetimes to manage, separately:
|
|
841
|
-
|
|
842
|
-
- **Chats** — `archive()` / `await using` every agent ([Cleanup](#cleanup)), or
|
|
843
|
-
finished chats keep holding server resources.
|
|
844
|
-
- **Workspaces** — rely on template‑level scheduling rather than manual
|
|
845
|
-
hygiene:
|
|
846
|
-
- **Autostop TTL.** Give fleet templates a default TTL long enough to survive
|
|
847
|
-
a normal session (including idle gaps between turns), short enough that a
|
|
848
|
-
leaked workspace stops burning running‑cost within hours — without
|
|
849
|
-
autostop, a leaked workspace pins its full quota until someone notices.
|
|
850
|
-
Note that stopping only releases the quota of resources that go away on
|
|
851
|
-
stop; persistent resources (disks, volumes) keep consuming their
|
|
852
|
-
`daily_cost`, so a scratch fleet that only ever stops still converges on a
|
|
853
|
-
full budget — pair the TTL with dormancy auto‑deletion or explicit
|
|
854
|
-
deletion. With [`@coder/ai-sdk-sandbox`](../sandbox), `stopAfter: "8h"`
|
|
855
|
-
sets the TTL (`ttl_ms`) at creation.
|
|
856
|
-
- **Activity bump** (default 1 h) extends a running workspace's deadline when
|
|
857
|
-
Coder detects sessions — check
|
|
858
|
-
[what counts as activity](https://coder.com/docs/user-guides/workspace-scheduling)
|
|
859
|
-
before assuming an agent's server‑side tool use keeps its workspace alive.
|
|
860
|
-
- **Dormancy / failure cleanup** reap abandoned and repeatedly‑failing
|
|
861
|
-
workspaces automatically — see
|
|
862
|
-
[template scheduling](https://coder.com/docs/admin/templates/managing-templates/schedule).
|
|
863
|
-
|
|
864
|
-
### Preventing stuck turns
|
|
865
|
-
|
|
866
|
-
The signature failure mode of an over‑committed fleet: the chat is created, the
|
|
867
|
-
stream opens, and then — nothing. There is **no distinct "quota exceeded" error
|
|
868
|
-
kind on the chat stream**; a chat whose workspace can't be scheduled surfaces
|
|
869
|
-
either a generic turn error or, worse, a chat that sits in a non‑terminal
|
|
870
|
-
status indefinitely. Defend in this order:
|
|
871
|
-
|
|
872
|
-
1. **Set `requestTimeoutMs` — always, in fleets.** It is unbounded by default.
|
|
873
|
-
On expiry the call rejects with a `CoderChatError` (`kind: "timeout"`,
|
|
874
|
-
`retryable: true`) so your dispatcher gets its slot back, and a server‑side
|
|
875
|
-
interrupt is fired **best‑effort** — fire‑and‑forget, and unreachable when
|
|
876
|
-
the timeout lands before chat creation returned an id — so the run usually
|
|
877
|
-
stops, but is not guaranteed released ([Timeouts](#timeouts--cancellation)).
|
|
878
|
-
Pair it with reconciliation: periodically sweep for non‑terminal chats older
|
|
879
|
-
than your budget and interrupt/archive them.
|
|
880
|
-
2. **Cap total wall‑clock** with `abortSignal: AbortSignal.timeout(…)` —
|
|
881
|
-
`requestTimeoutMs` bounds each segment, not a whole multi‑tool call.
|
|
882
|
-
3. **Own the retries.** Workspace‑backed turns are **never auto‑retried**: the
|
|
883
|
-
SDK marks a stream‑loss error `isRetryable` only when the failed turn
|
|
884
|
-
created its chat _and_ had no workspace, no MCP servers, and no fresh
|
|
885
|
-
uploads ([Timeouts](#timeouts--cancellation)). Your dispatcher owns the
|
|
886
|
-
retry decision — and should re‑check quota headroom first, or it re‑queues
|
|
887
|
-
into the same wall.
|
|
888
|
-
4. **Watch the fleet.** [`watchChats`](#watching-chats) yields status changes
|
|
889
|
-
for every chat **visible to the authenticated user** — with chats spread
|
|
890
|
-
across per‑tenant credentials, run one watcher per identity (or watch with
|
|
891
|
-
a credential that can see them all). Alert on chats sitting in a
|
|
892
|
-
non‑terminal status (e.g. `pending`) longer than your `requestTimeoutMs`.
|
|
893
|
-
Events alone aren't a complete monitor: reconnects resubscribe fresh (no
|
|
894
|
-
cursor or replay), so a chat already stuck before the watcher started — or
|
|
895
|
-
one that transitioned during a gap — never emits an event to start your
|
|
896
|
-
timer from. Seed and periodically reconcile against a chat listing
|
|
897
|
-
(`GET /api/v2/chats`) instead of trusting the event stream alone.
|
|
898
|
-
|
|
899
|
-
### Troubleshooting: unschedulable & stuck chats
|
|
900
|
-
|
|
901
|
-
Wire [`onTransportEvent`](#observability) into fleet telemetry — the event
|
|
902
|
-
sequence pinpoints _where_ a turn died:
|
|
903
|
-
|
|
904
|
-
| symptom (transport events) | likely cause | fix |
|
|
905
|
-
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
906
|
-
| `segment:start` and `ws:open` fired, initial status events, then silence — no `message_part`, no `segment:settle` | if the chat's status is `pending` (or its workspace build is `pending`/`failed`): the workspace can't be scheduled — quota exhausted, no free provisioner, failing template. If the status is `running` with a healthy build, it may just be a slow model/tool step — silence alone doesn't prove scheduling | check quota headroom (`workspace-quota` endpoint) and the workspace's build — a quota failure logs `INSUFFICIENT_QUOTA`; reclaim enough _credits_ for the planned build (stopping helps only for stop‑scoped costs — delete workspaces to reclaim persistent costs, or raise the group allowance) and set `requestTimeoutMs` so this fails loudly next time |
|
|
907
|
-
| `segment:settle` carries `error: { name: "CoderChatError", message: "…requestTimeoutMs budget…" }` | the per‑segment bound expired — wedged server, slow model, or an unschedulable workspace | inspect the workspace via the v2 API/UI: a `pending`/`failed` build means the row above; `running` means the turn was genuinely slow — raise `requestTimeoutMs` for long tool work |
|
|
908
|
-
| repeated `ws:redial` with `consecutiveFailures` climbing toward `maxConsecutiveFailures` (5; backoff 1 s → 2 s → 4 s → 8 s, ≈15 s of redialing without forward progress), then a `CoderStreamError` | the network path to the deployment is failing — not workspace scheduling (the server keeps generating through short gaps) | fix connectivity; mind retry ownership above — on workspace‑backed chats the error is `isRetryable: false`, so the replay decision is yours |
|
|
909
|
-
| `segment:settle` with `status: "error"` and an `error` payload | the turn failed server‑side — a provider/model error, a tool failure, or a scheduling/build failure that terminated the turn instead of leaving it pending | the settle event's `error` carries only `{ name, message }` — branch on `kind` / `retryable` / `statusCode` by catching the thrown `CoderChatError`: at the `generate()` call site, or — for `stream()` — around stream consumption, since mid‑stream failures surface on the stream, not from `await agent.stream()` ([Handling errors](#handling-errors)); check the workspace build state to rule scheduling in or out |
|
|
910
|
-
| turn settled `status: "requires_action"`, follow‑up messages queue forever | the loop ended on an unanswered client tool call | submit the stranded results or interrupt — see rule 4 under [Structured output](#structured-output); if a crash left the pause behind, reconcile effects first ([Make client tools crash-safe](#make-client-tools-crash-safe)) |
|
|
911
|
-
| `archive()` keeps returning 409 and rethrows after ~15 s | the chat never settled server‑side — usually a stuck run still holding its workspace | `interrupt()` with a bounded signal, then re‑archive; if the run stays wedged, stop the workspace itself |
|
|
912
|
-
|
|
913
408
|
## Configuration
|
|
914
409
|
|
|
915
410
|
`CoderAgentSettings`:
|
|
@@ -928,34 +423,44 @@ sequence pinpoints _where_ a turn died:
|
|
|
928
423
|
| `planMode` | enable plan mode (`"plan"`) |
|
|
929
424
|
| `stopWhen` | AI SDK stop condition(s); default `stepCountIs(64)` |
|
|
930
425
|
| `maxRetries` | default `0` — SDK retries can duplicate server‑side turns; override with care |
|
|
931
|
-
| `requestTimeoutMs` | per‑
|
|
426
|
+
| `requestTimeoutMs` | per‑segment time budget (ms), not per call; interrupts the run and rejects (`kind: "timeout"`) instead of hanging |
|
|
932
427
|
| `onTransportEvent` | observability hook for typed transport events (see [Observability](#observability)) |
|
|
933
428
|
| `settleDeadlineMs` | overall deadline for bounded cleanup (`archive()` 409 retries, disposal); default 15 000 |
|
|
934
429
|
| `settleRetryDelayMs` | pause between `archive()` retries while the chat settles; default 1000 |
|
|
935
430
|
| `chatId` | resume an existing chat |
|
|
936
431
|
| `lastSeenMessageId` | resume cursor for `chatId` (persist `agent.lastSeenMessageId`) — skips the resumed turn's pre‑prompt history probe |
|
|
937
432
|
|
|
938
|
-
The `model` hint resolves against the organization's
|
|
939
|
-
|
|
940
|
-
|
|
941
|
-
|
|
942
|
-
|
|
943
|
-
|
|
944
|
-
|
|
945
|
-
|
|
946
|
-
|
|
947
|
-
|
|
948
|
-
|
|
949
|
-
|
|
433
|
+
**Model hint resolution.** The `model` hint resolves against the organization's
|
|
434
|
+
model configs, first match wins:
|
|
435
|
+
|
|
436
|
+
1. a config UUID, used as‑is
|
|
437
|
+
2. an exact `provider:model` match
|
|
438
|
+
3. an exact model id
|
|
439
|
+
4. a display‑name substring (case‑insensitive)
|
|
440
|
+
5. a model‑id substring
|
|
441
|
+
|
|
442
|
+
An unresolvable hint falls back to the server's default model instead of
|
|
443
|
+
failing. Use `agent.listModels()` to see what's available.
|
|
444
|
+
|
|
445
|
+
<details>
|
|
446
|
+
<summary>Where model configs come from, and older deployments</summary>
|
|
447
|
+
|
|
448
|
+
- Configs come from
|
|
449
|
+
`GET /api/v2/organizations/{organizationId}/chats/models`, with the provider
|
|
450
|
+
type joined from the response's provider descriptors.
|
|
451
|
+
- On older deployments where the organization‑scoped route does not exist yet
|
|
452
|
+
(404), resolution falls back once to the legacy deployment‑wide
|
|
453
|
+
`/api/experimental/chats/model-configs` listing, independently of chat‑prefix
|
|
454
|
+
selection.
|
|
455
|
+
- Partial payloads from older/newer servers are tolerated: entries match on the
|
|
456
|
+
fields they carry.
|
|
457
|
+
|
|
458
|
+
</details>
|
|
950
459
|
|
|
951
460
|
### Reasoning effort
|
|
952
461
|
|
|
953
462
|
Set `reasoningEffort` on `CoderAgentSettings` (or `CoderLanguageModelConfig`
|
|
954
|
-
when using the model directly)
|
|
955
|
-
`medium`, `high`, `xhigh`, `max`. The exported `ReasoningEffort` type also accepts
|
|
956
|
-
other strings for forward compatibility. Model configs returned by
|
|
957
|
-
`agent.listModels()` / `client.listModelConfigs(organizationId)` expose the
|
|
958
|
-
optional `reasoning_efforts` list of selectable values for each model.
|
|
463
|
+
when using the model directly):
|
|
959
464
|
|
|
960
465
|
```ts
|
|
961
466
|
import { CoderAgent } from "@coder/ai-sdk-agent";
|
|
@@ -970,865 +475,327 @@ console.log(chat.last_reasoning_effort); // string, null, or absent on the wire
|
|
|
970
475
|
await agent.archive();
|
|
971
476
|
```
|
|
972
477
|
|
|
973
|
-
|
|
974
|
-
|
|
975
|
-
|
|
976
|
-
|
|
977
|
-
|
|
978
|
-
|
|
979
|
-
|
|
980
|
-
|
|
478
|
+
- **Values.** The global scale is `none`, `minimal`, `low`, `medium`, `high`,
|
|
479
|
+
`xhigh`, `max`. The exported `ReasoningEffort` type also accepts other strings
|
|
480
|
+
for forward compatibility.
|
|
481
|
+
- **Per‑model options.** Model configs returned by `agent.listModels()` /
|
|
482
|
+
`client.listModelConfigs(organizationId)` expose the optional
|
|
483
|
+
`reasoning_efforts` list of selectable values for each model.
|
|
484
|
+
- **Independent of the `model` hint.**
|
|
485
|
+
- **No per‑call override.** Configure it on the agent/model instance.
|
|
981
486
|
|
|
982
|
-
|
|
487
|
+
<details>
|
|
488
|
+
<summary>When <code>reasoning_effort</code> is sent, and older servers</summary>
|
|
983
489
|
|
|
984
|
-
|
|
985
|
-
|
|
986
|
-
|
|
987
|
-
|
|
988
|
-
|
|
989
|
-
|
|
990
|
-
|
|
490
|
+
- It is sent as `reasoning_effort` on chat creation and every
|
|
491
|
+
user‑message submission, including submissions to resumed or busy chats that
|
|
492
|
+
queue the message. When unset, the request key is omitted entirely.
|
|
493
|
+
Tool‑result submissions and WebSocket reconnects do not post user messages and
|
|
494
|
+
do not send this field.
|
|
495
|
+
- Older Coder servers that do not recognize `reasoning_effort` ignore the
|
|
496
|
+
unknown JSON field, so setting it is safe but has no effect there.
|
|
991
497
|
|
|
992
|
-
|
|
993
|
-
**pauses** for a client tool (`requires_action`). The SDK loop and the server‑side loop
|
|
994
|
-
mesh at the client‑tool boundary, so there's no double loop.
|
|
995
|
-
- Streaming text is emitted from `message_part` deltas; every `message` snapshot is then
|
|
996
|
-
reconciled against a per‑message emitted‑content ledger, so nothing double‑counts: a
|
|
997
|
-
trailing snapshot after deltas is a no‑op, a fast snapshot‑only turn emits in full, a
|
|
998
|
-
message that commits while the stream is redialing yields exactly its missing tail, and a
|
|
999
|
-
revision that appends to an earlier message yields the appended suffix (rewrites that
|
|
1000
|
-
can't be expressed as deltas are safely suppressed).
|
|
498
|
+
</details>
|
|
1001
499
|
|
|
1002
|
-
##
|
|
500
|
+
## API compatibility
|
|
1003
501
|
|
|
1004
|
-
|
|
1005
|
-
|
|
1006
|
-
|
|
1007
|
-
|
|
1008
|
-
|
|
1009
|
-
[Handling errors](#handling-errors), and [Observability](#observability).
|
|
502
|
+
Coder 2.37.0 promoted chat routes to `/api/v2/chats`
|
|
503
|
+
([server routes](https://github.com/coder/coder/pull/28496),
|
|
504
|
+
[Go SDK](https://github.com/coder/coder/pull/28497)). Coder 2.36.4 has no stable
|
|
505
|
+
chat mount. During the compatibility window (CODAGT-921) the promoted routes also
|
|
506
|
+
remain under `/api/experimental/chats`; legacy removal is tracked in CODAGT-922.
|
|
1010
507
|
|
|
1011
|
-
|
|
508
|
+
Each `CoderChatClient` selects its chat prefix from HTTP responses, not version
|
|
509
|
+
strings: v2 first, then the experimental prefix on a 404. The first non‑404 HTTP
|
|
510
|
+
response locks the prefix for the client's lifetime.
|
|
511
|
+
|
|
512
|
+
<details>
|
|
513
|
+
<summary>Prefix selection rules</summary>
|
|
514
|
+
|
|
515
|
+
- Until a prefix is selected, an actual REST request (including JSON requests
|
|
516
|
+
and `Blob` uploads) tries v2 first and retries once under the experimental
|
|
517
|
+
prefix only on 404.
|
|
518
|
+
- The first non‑404 HTTP response (including an error) locks that prefix for the
|
|
519
|
+
client's lifetime. Concurrent unresolved calls may probe independently; the
|
|
520
|
+
first conclusive response wins.
|
|
521
|
+
- Two 404s preserve the original v2 error without locking. Non‑HTTP failures do
|
|
522
|
+
not lock either.
|
|
523
|
+
- A locked prefix is never renegotiated, even on 404.
|
|
524
|
+
- Before opening a per‑chat `/stream` or global `/watch` WebSocket, an
|
|
525
|
+
unresolved client preflights with `GET /api/v2/chats`, using the same
|
|
526
|
+
selection rules and cancellation signal.
|
|
527
|
+
- `ReadableStream` uploads also preflight, so their consumed bodies are never
|
|
528
|
+
replayed.
|
|
529
|
+
- Legacy model‑config lookup remains independent of chat‑prefix selection (see
|
|
530
|
+
[Configuration](#configuration)).
|
|
531
|
+
|
|
532
|
+
</details>
|
|
1012
533
|
|
|
1013
|
-
|
|
1014
|
-
run itself — none of it is in your process. The only durable thing a
|
|
1015
|
-
workflow has to carry between steps is **`agent.chatId`: a string** — plus,
|
|
1016
|
-
optionally, **`agent.lastSeenMessageId`: the resume cursor**, which saves the
|
|
1017
|
-
resumed turn one serial round‑trip (below).
|
|
1018
|
-
- **`CoderAgent` can't ride a `fetch`‑shim durability layer** (it talks REST +
|
|
1019
|
-
WebSocket through its own client), so each turn runs **inside** a durable
|
|
1020
|
-
step — and the checkpointed chat id is the thread between steps.
|
|
534
|
+
## Timeouts & cancellation
|
|
1021
535
|
|
|
1022
|
-
|
|
1023
|
-
|
|
536
|
+
**Cancel a turn** by passing an `abortSignal` to `generate()`/`stream()`.
|
|
537
|
+
Aborting **interrupts the server‑side run**, not just the local socket, so the
|
|
538
|
+
chat stops generating and releases its resources instead of running on,
|
|
539
|
+
orphaned. Tearing down a `stream()` early (cancelling the stream) interrupts the
|
|
540
|
+
run too.
|
|
1024
541
|
|
|
1025
|
-
|
|
542
|
+
**Bound each segment** with `requestTimeoutMs`. If a segment runs longer (e.g.
|
|
543
|
+
the server is wedged, or a workspace can't be scheduled), the run is interrupted
|
|
544
|
+
and the call rejects with a retryable `CoderChatError` (`kind: "timeout"`)
|
|
545
|
+
instead of hanging:
|
|
1026
546
|
|
|
1027
547
|
```ts
|
|
1028
|
-
|
|
548
|
+
const agent = new CoderAgent({ organizationId, requestTimeoutMs: 120_000 });
|
|
549
|
+
```
|
|
1029
550
|
|
|
1030
|
-
|
|
1031
|
-
|
|
1032
|
-
|
|
1033
|
-
lastSeenMessageId?: number;
|
|
1034
|
-
}
|
|
551
|
+
A segment is one model round‑trip, until it settles or pauses for a client tool.
|
|
552
|
+
A multi‑step `generate()` that drives client tools runs several segments, so
|
|
553
|
+
`requestTimeoutMs` bounds each one, not the whole call.
|
|
1035
554
|
|
|
1036
|
-
|
|
1037
|
-
|
|
1038
|
-
|
|
1039
|
-
|
|
1040
|
-
};
|
|
1041
|
-
|
|
1042
|
-
export async function runTurn(workflowId: string, prompt: string): Promise<string> {
|
|
1043
|
-
const checkpoint = await checkpoints.get(workflowId); // undefined on the first step → the turn creates the chat
|
|
1044
|
-
const agent = new CoderAgent({
|
|
1045
|
-
baseUrl: process.env.CODER_URL!,
|
|
1046
|
-
token: process.env.CODER_SESSION_TOKEN!, // read per step — never checkpoint or log it
|
|
1047
|
-
organizationId: process.env.CODER_ORG_ID!,
|
|
1048
|
-
chatId: checkpoint?.chatId,
|
|
1049
|
-
// The previous step's cursor lets this step skip its pre-prompt history
|
|
1050
|
-
// probe; absent (first resume, older checkpoints), the turn seeds it from
|
|
1051
|
-
// the chat's newest message instead — one extra GET, same result.
|
|
1052
|
-
lastSeenMessageId: checkpoint?.lastSeenMessageId,
|
|
1053
|
-
requestTimeoutMs: 300_000, // always bound workflow steps — see below
|
|
1054
|
-
onTransportEvent: observe, // per-step telemetry — see below
|
|
1055
|
-
});
|
|
1056
|
-
|
|
1057
|
-
try {
|
|
1058
|
-
const { text } = await agent.generate({
|
|
1059
|
-
prompt,
|
|
1060
|
-
// Total wall-clock for the step: requestTimeoutMs bounds each segment,
|
|
1061
|
-
// not a whole multi-segment (client-tool) turn — see "Bound every step".
|
|
1062
|
-
abortSignal: AbortSignal.timeout(600_000),
|
|
1063
|
-
});
|
|
1064
|
-
return text;
|
|
1065
|
-
} finally {
|
|
1066
|
-
// The durable resume handle — written even when the turn failed: a first
|
|
1067
|
-
// step that fails AFTER creating its chat (timeout, stream loss) must
|
|
1068
|
-
// still persist the id, or the retried step orphans a live chat and its
|
|
1069
|
-
// partial effects. Exception: after an exhausted stream failure on a
|
|
1070
|
-
// chat this very call created, the SDK has already discarded the dead
|
|
1071
|
-
// session (agent.chatId is undefined) and nothing is written — the thrown
|
|
1072
|
-
// CoderStreamError's chatId still names the stranded chat, and
|
|
1073
|
-
// agent.archive() retires it via agent.lastKnownChatId — see the
|
|
1074
|
-
// CoderStreamError notes below for what retrying means then.
|
|
1075
|
-
if (agent.chatId) {
|
|
1076
|
-
const next: WorkflowCheckpoint = { chatId: agent.chatId };
|
|
1077
|
-
// The cursor is optional sugar: persist it when a turn established one
|
|
1078
|
-
// (a turn that failed before streaming may not have), skip it otherwise.
|
|
1079
|
-
if (agent.lastSeenMessageId !== undefined) {
|
|
1080
|
-
next.lastSeenMessageId = agent.lastSeenMessageId;
|
|
1081
|
-
}
|
|
1082
|
-
await checkpoints.set(workflowId, next);
|
|
1083
|
-
}
|
|
1084
|
-
}
|
|
1085
|
-
}
|
|
555
|
+
**Cap total wall‑clock** of a multi‑step call with a deadline signal instead:
|
|
556
|
+
|
|
557
|
+
```ts
|
|
558
|
+
await agent.generate({ prompt: "…", abortSignal: AbortSignal.timeout(120_000) });
|
|
1086
559
|
```
|
|
1087
560
|
|
|
1088
|
-
|
|
561
|
+
**Stream drops heal themselves.** If the event stream drops mid‑turn, the agent
|
|
562
|
+
redials it automatically with exponential backoff, replays the turn's events from
|
|
563
|
+
its starting cursor, and deduplicates them on receipt. The server keeps
|
|
564
|
+
generating during the gap, so a transient drop costs nothing and the run is
|
|
565
|
+
**not** interrupted.
|
|
1089
566
|
|
|
1090
|
-
|
|
1091
|
-
|
|
1092
|
-
|
|
1093
|
-
- **Persist the id, not the instance** (and never the token — read it from
|
|
1094
|
-
each step's environment).
|
|
1095
|
-
- **Checkpoint in `finally`, not only on success**, so failed-but-alive chats
|
|
1096
|
-
stay reachable. What remains is the hard-crash window between chat creation
|
|
1097
|
-
and the checkpoint write (below).
|
|
1098
|
-
- **No `await using` here.** The chat outlives the step; disposal archives it
|
|
1099
|
-
and archiving ends resumability (below).
|
|
567
|
+
Only when the stream cannot be re‑established (several consecutive failed
|
|
568
|
+
attempts, ~15s) is the server run interrupted and the call rejected with a
|
|
569
|
+
`CoderStreamError`, an AI SDK `APICallError`:
|
|
1100
570
|
|
|
1101
|
-
|
|
571
|
+
| `isRetryable` | When | Then |
|
|
572
|
+
| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
|
573
|
+
| `true` | The failed turn had just created its chat **and** had no external effects a replay would repeat: no `workspaceId`, no `mcpServerIds`, and no freshly uploaded inline attachments (pre‑uploaded `fileId` references are fine) | The dead session is discarded. `generate()` calls with `maxRetries` set retry the whole turn on a fresh chat automatically. |
|
|
574
|
+
| `false` | Otherwise | Retrying is the caller's deliberate decision. |
|
|
1102
575
|
|
|
1103
|
-
|
|
1104
|
-
|
|
1105
|
-
await runTurn("wf-1042", "Investigate the failing nightly build and propose a fix.");
|
|
576
|
+
<details>
|
|
577
|
+
<summary>Why the other cases aren't retryable</summary>
|
|
1106
578
|
|
|
1107
|
-
|
|
1108
|
-
|
|
579
|
+
On a chat with prior state (resumed sessions, later turns, tool‑result
|
|
580
|
+
segments), a re‑invocation would resubmit the same prompt as a new user turn.
|
|
581
|
+
Workspace/MCP tools may already have executed side effects, and inline
|
|
582
|
+
attachments would upload again.
|
|
1109
583
|
|
|
1110
|
-
|
|
1111
|
-
|
|
1112
|
-
|
|
584
|
+
</details>
|
|
585
|
+
|
|
586
|
+
- For `stream()`, a mid‑stream failure surfaces on the stream itself, outside
|
|
587
|
+
the SDK's retry wrapper. Handle it in your consumption loop.
|
|
588
|
+
- A non‑transient 4xx upgrade rejection (bad/expired token, deleted chat) fails
|
|
589
|
+
fast with a `CoderApiError` instead of retrying. 408/425/429 consume the redial
|
|
590
|
+
budget like any other transient failure.
|
|
1113
591
|
|
|
1114
|
-
|
|
1115
|
-
|
|
1116
|
-
|
|
1117
|
-
|
|
1118
|
-
|
|
1119
|
-
|
|
1120
|
-
|
|
1121
|
-
|
|
1122
|
-
|
|
1123
|
-
|
|
1124
|
-
|
|
1125
|
-
|
|
1126
|
-
|
|
1127
|
-
|
|
1128
|
-
|
|
1129
|
-
|
|
1130
|
-
-
|
|
1131
|
-
|
|
1132
|
-
|
|
1133
|
-
|
|
1134
|
-
|
|
1135
|
-
and if it times out first, it withdraws its queued prompt rather than
|
|
1136
|
-
leaving it to run unattended. The live run's turn is guarded symmetrically:
|
|
1137
|
-
the server starts the queued prompt's run **without ever emitting a
|
|
1138
|
-
terminal settle for the finished one**, so the SDK settles the finishing
|
|
1139
|
-
turn the moment the promoted prompt appears on its stream — nothing of the
|
|
1140
|
-
next turn's output or usage leaks into it. That settle reports
|
|
1141
|
-
`finishReason: "stop"` as a documented approximation: the wire doesn't
|
|
1142
|
-
distinguish a run that completed into a promotion from one interrupted by
|
|
1143
|
-
`busy_behavior: "interrupt"`, so an interrupted turn also settles `"stop"`
|
|
1144
|
-
(with whatever partial output it committed) rather than surfacing an
|
|
1145
|
-
interruption.
|
|
1146
|
-
|
|
1147
|
-
### Let the SDK absorb drops — and handle what it re‑throws
|
|
1148
|
-
|
|
1149
|
-
Self‑healed, invisible to the step (the run is **not** interrupted):
|
|
1150
|
-
|
|
1151
|
-
- **Transient stream drops.** The server keeps generating through the gap; the
|
|
1152
|
-
SDK redials with backoff and replays what it missed — committed messages
|
|
1153
|
-
past the turn's cursor plus the in‑progress message's deltas — deduplicating
|
|
1154
|
-
on receipt. A message that committed _while the connection was down_ arrives
|
|
1155
|
-
as exactly its missing tail. Drops cost latency, never content, and never
|
|
1156
|
-
duplicate content.
|
|
1157
|
-
- **Client‑tool pauses ride one connection.** A multi‑segment turn keeps its
|
|
1158
|
-
stream across `requires_action` pauses, and a drop while your tool executes
|
|
1159
|
-
redials in the background.
|
|
1160
|
-
- **A lost tool‑call event.** If a turn pauses for client tools but the pause's
|
|
1161
|
-
tool‑call event doesn't arrive within ~2 s, the SDK recovers the pending
|
|
1162
|
-
calls from committed history over REST and the turn continues.
|
|
1163
|
-
|
|
1164
|
-
Surfaced to the step once self‑healing is exhausted:
|
|
592
|
+
## Handling errors
|
|
593
|
+
|
|
594
|
+
All errors extend `CoderAgentError`, except `CoderStreamError`.
|
|
595
|
+
|
|
596
|
+
| Error | Extends | Thrown when | Fields |
|
|
597
|
+
| ------------------ | --------------------- | --------------------------------------------------------------------------------- | --------------------------------------------- |
|
|
598
|
+
| `CoderApiError` | `CoderAgentError` | An HTTP request failed | `status`, `method`, `path`, `detail` |
|
|
599
|
+
| `CoderChatError` | `CoderAgentError` | A turn ended in an error, timed out, or lost its stream | `kind`, `retryable`, `statusCode`, `provider` |
|
|
600
|
+
| `CoderStreamError` | AI SDK `APICallError` | The event stream dropped and could not be re‑established within its redial budget | `isRetryable`, `cause`, `chatId` |
|
|
601
|
+
|
|
602
|
+
`CoderStreamError` extends `APICallError` so `generate()`'s `maxRetries`
|
|
603
|
+
machinery recognizes it:
|
|
604
|
+
|
|
605
|
+
- `isRetryable` is `true` only when the failed turn created its chat and had no
|
|
606
|
+
external effects to repeat (workspace/MCP tooling, fresh attachment uploads).
|
|
607
|
+
Full rules: [Timeouts & cancellation](#timeouts--cancellation).
|
|
608
|
+
- `cause` holds the last transport failure.
|
|
609
|
+
- `chatId` names the chat the failed turn had created or attached to (absent
|
|
610
|
+
when it failed before a chat existed). After the fresh‑chat discard the chat
|
|
611
|
+
still exists server‑side, and `archive()` keeps targeting it via
|
|
612
|
+
`agent.lastKnownChatId` ([Cleanup](#cleanup)).
|
|
1165
613
|
|
|
1166
614
|
```ts
|
|
1167
|
-
import { CoderApiError, CoderChatError
|
|
615
|
+
import { CoderApiError, CoderChatError } from "@coder/ai-sdk-agent";
|
|
1168
616
|
|
|
1169
617
|
try {
|
|
1170
|
-
await
|
|
618
|
+
await agent.generate({ prompt: "…" });
|
|
1171
619
|
} catch (err) {
|
|
1172
620
|
if (err instanceof CoderChatError && err.retryable) {
|
|
1173
|
-
//
|
|
1174
|
-
|
|
1175
|
-
//
|
|
1176
|
-
} else
|
|
1177
|
-
|
|
1178
|
-
// chat this is never auto-retried — the retry decision is yours (below).
|
|
1179
|
-
} else if (err instanceof CoderApiError) {
|
|
1180
|
-
// Branch on err.status: back off and retry 408/425/429/5xx; fail the workflow
|
|
1181
|
-
// on the rest (expired token, archived/deleted chat, …).
|
|
621
|
+
// transient (timeout, stream_closed, an upstream 5xx) — back off and retry
|
|
622
|
+
} else if (err instanceof CoderApiError && err.status === 429) {
|
|
623
|
+
// rate limited
|
|
624
|
+
} else {
|
|
625
|
+
throw err;
|
|
1182
626
|
}
|
|
1183
|
-
throw err;
|
|
1184
627
|
}
|
|
1185
628
|
```
|
|
1186
629
|
|
|
1187
|
-
|
|
1188
|
-
|
|
1189
|
-
|
|
1190
|
-
classifies the _failure_ as transient — it does not make a re‑run free.
|
|
1191
|
-
Unlike the guarded `CoderStreamError` path below, the session is **not**
|
|
1192
|
-
discarded: the retried step reloads the checkpointed chat id (the `finally`
|
|
1193
|
-
checkpoint above is what makes this hold even when the _first_ step fails
|
|
1194
|
-
after creating its chat) and resubmits the same prompt as a new user turn,
|
|
1195
|
-
with the aborted attempt's partial output in history and any tool effects
|
|
1196
|
-
that ran before the failure not undone. The same idempotency judgment as
|
|
1197
|
-
below applies — retry steps designed to tolerate re‑submission; reconcile
|
|
1198
|
-
first when they aren't. And don't resubmit _immediately_: the expiry's
|
|
1199
|
-
server interrupt is asynchronous and best‑effort, so the timed‑out run may
|
|
1200
|
-
still be committing — run the same interrupt‑and‑wait‑for‑terminal‑status
|
|
1201
|
-
recovery as after a crash (below) before the retry, or its trailing output
|
|
1202
|
-
is absorbed into the retried turn. A timed‑out run can also have _finished_
|
|
1203
|
-
before that interrupt landed — but because the interrupt was already fired,
|
|
1204
|
-
the recovery's [result check](#recover-the-result-before-resubmitting)
|
|
1205
|
-
must treat the attempt as cut short (pin `cutShort` there) rather than
|
|
1206
|
-
trusting its own 409.
|
|
1207
|
-
- **`CoderStreamError`** (an AI SDK `APICallError`) — the stream could not be
|
|
1208
|
-
re‑established. `isRetryable: true` only when the failed turn had just
|
|
1209
|
-
created its chat _and_ had no external effects a replay would repeat (no
|
|
1210
|
-
workspace, no MCP servers, no fresh inline uploads); the SDK then discards
|
|
1211
|
-
the dead session so a retry — `maxRetries` or a re‑run step — starts clean
|
|
1212
|
-
on a fresh chat. The discard happens for _every_ chat the failed turn
|
|
1213
|
-
itself created — **including the effectful, `isRetryable: false` case**:
|
|
1214
|
-
`agent.chatId` is `undefined` and the `finally` checkpoint writes nothing —
|
|
1215
|
-
but the stranded chat is not nameless: the error's **`chatId`** field names
|
|
1216
|
-
it, and `interrupt()`/`archive()` keep targeting it via
|
|
1217
|
-
`agent.lastKnownChatId` ([Cleanup](#cleanup)), so the step's error path can
|
|
1218
|
-
retire the orphan directly — or checkpoint `err.chatId` for the
|
|
1219
|
-
reconciliation sweep (below) when its workspace/MCP effects need
|
|
1220
|
-
reconciling first. A retried step then starts a fresh chat that has _not_
|
|
1221
|
-
seen the orphan's effects: gate that retry on the same idempotency judgment
|
|
1222
|
-
as any re‑submission. **On a resumed chat it is always `isRetryable: false`:**
|
|
1223
|
-
re‑running the step resubmits the same prompt as a _new user turn_, with the
|
|
1224
|
-
failed attempt's partial output still in history (usually fine — the model
|
|
1225
|
-
sees its own aborted attempt), and any workspace/MCP tool effects that ran
|
|
1226
|
-
before the drop are not undone. That judgment call belongs to your workflow,
|
|
1227
|
-
which is exactly why it isn't automatic.
|
|
1228
|
-
- **`CoderApiError`** — an HTTP request failed; every non‑2xx wraps in this
|
|
1229
|
-
error, so branch on `status` before deciding. Rate limiting and transient
|
|
1230
|
-
server failures (408, 425, 429, 5xx) deserve backoff and a step retry (with
|
|
1231
|
-
the same re‑submission caveat as above). Auth failures and archived/deleted
|
|
1232
|
-
chats (401/403/404, the archived‑chat 400) hit the same wall on every
|
|
1233
|
-
attempt — fail the workflow.
|
|
1234
|
-
|
|
1235
|
-
### Recover after a crash
|
|
1236
|
-
|
|
1237
|
-
A hard crash (OOM kill, host loss) runs no `finally` blocks. Reason from the
|
|
1238
|
-
checkpoint:
|
|
1239
|
-
|
|
1240
|
-
- **Crashed with an empty checkpoint** — usually nothing exists server‑side
|
|
1241
|
-
and the retried step just creates a fresh chat. But an empty checkpoint is
|
|
1242
|
-
not proof of nothing: a crash or `requestTimeoutMs` expiry while the create
|
|
1243
|
-
request was _in flight_ can leave a chat the server committed but whose id
|
|
1244
|
-
never arrived — unreachable even by the SDK's own interrupt. Treat "no
|
|
1245
|
-
checkpoint" as _up to one unacknowledged chat per attempt may exist_ — not
|
|
1246
|
-
as a clean slate, and automatic step retries can stack several. The
|
|
1247
|
-
reconciliation sweep below must therefore retire _every_ matching orphan
|
|
1248
|
-
(and its workspace/MCP effects, if the deployment auto‑attached any), not
|
|
1249
|
-
stop at the first.
|
|
1250
|
-
- **Crashed after creation, before the checkpoint** — the chat is orphaned:
|
|
1251
|
-
its run keeps generating until it settles on its own — or, if the crash hit
|
|
1252
|
-
one of your client tools, never settles: it stays paused in
|
|
1253
|
-
`requires_action`, holding its workspace, invisible to the retried step
|
|
1254
|
-
(which has no id to interrupt). Either way the retried step starts a fresh
|
|
1255
|
-
chat, so orphans are found only by the reconciliation sweep from
|
|
1256
|
-
[Preventing stuck turns](#preventing-stuck-turns) — that sweep is not
|
|
1257
|
-
optional in this pattern, and it needs a **wider filter** here: the fleet
|
|
1258
|
-
sweep watches for _non‑terminal_ chats stuck past their budget, but an
|
|
1259
|
-
orphaned first turn may settle terminally on its own and then sit as a
|
|
1260
|
-
live, unarchived chat forever. Sweep every chat no checkpoint accounts
|
|
1261
|
-
for — regardless of status — and interrupt/archive it. (To shrink the window, checkpoint eagerly from the
|
|
1262
|
-
first `ws:dial` transport event — it carries the chat id as soon as the chat
|
|
1263
|
-
exists — but then the checkpoint's lifecycle is yours too: the retried step
|
|
1264
|
-
resumes a chat holding a dead half‑turn, and whenever a `CoderStreamError`
|
|
1265
|
-
killed a chat this step created — retryable or not: the SDK discards every
|
|
1266
|
-
such session, but only in memory — you must clear the eagerly checkpointed
|
|
1267
|
-
id yourself before any retry, or "starts fresh after a discard" silently
|
|
1268
|
-
turns into resuming the dead chat.)
|
|
1269
|
-
- **Crashed after the checkpoint** — the chat is resumable, but don't just
|
|
1270
|
-
resubmit. The crashed attempt's run may still be live server‑side, and a
|
|
1271
|
-
resubmission then queues behind it: nothing the old run commits is
|
|
1272
|
-
attributed to the resumed turn (its window opens only at its own queued
|
|
1273
|
-
prompt's materialization), but the step burns its `requestTimeoutMs`
|
|
1274
|
-
waiting for the old run to finish first — and a timeout withdraws the
|
|
1275
|
-
queued prompt again. A crash _inside one of your client tools_ is worse: the
|
|
1276
|
-
chat is paused in `requires_action`, where new messages wait forever — and
|
|
1277
|
-
if the tool's external effect committed before the crash, that pause is the
|
|
1278
|
-
only pending record of it, so reconcile the tool ledger **before** any
|
|
1279
|
-
interrupt ([Make client tools crash-safe](#make-client-tools-crash-safe)).
|
|
1280
|
-
Both cases have one remedy — stop what the crash left behind and **wait
|
|
1281
|
-
until the chat actually settles** before deciding: a client‑tool pause is
|
|
1282
|
-
reconciled, never interrupted away; anything else is interrupted:
|
|
1283
|
-
`interrupt()` resolves on acknowledgment while the run keeps winding down
|
|
1284
|
-
(and committing) for a few more seconds, so poll the chat's status to a
|
|
1285
|
-
terminal one first. On a chat with no live run the interrupt rejects with a
|
|
1286
|
-
409 `CoderApiError`; that's the good case — ignore it:
|
|
1287
|
-
|
|
1288
|
-
```ts
|
|
1289
|
-
// Both shared with "Make client tools crash-safe" (below): the ledger
|
|
1290
|
-
// reconciliation resolves to its `unstarted` — true only when it fell back
|
|
1291
|
-
// to interrupting — and journals each action write-ahead, so a recovery
|
|
1292
|
-
// pass that dies mid-recovery is visible — and classifiable — to the next.
|
|
1293
|
-
declare function reconcileClientTools(): Promise<boolean>;
|
|
1294
|
-
// Journal entries are per-ATTEMPT, so the chat id alone cannot scope them:
|
|
1295
|
-
// a verdict from attempt N must not classify attempt N+1 (double crash),
|
|
1296
|
-
// and a later step's recovery on the same chat must not inherit this
|
|
1297
|
-
// one's. Every entry pins the step marker and an attempt count at write
|
|
1298
|
-
// time; a consult applies an entry only while both still match, so each
|
|
1299
|
-
// resubmission retires every earlier verdict by itself — there is no
|
|
1300
|
-
// clear step to forget.
|
|
1301
|
-
type JournalEntry = {
|
|
1302
|
-
// "submitting" is written ahead of every send on a checkpointed chat;
|
|
1303
|
-
// "resumed" / "cut-short" are the ledger reconcile's verdict (both in
|
|
1304
|
-
// the next sections).
|
|
1305
|
-
state: "submitting" | "resumed" | "cut-short";
|
|
1306
|
-
marker: string; // the step-scoped marker ("Recover the result…", below)
|
|
1307
|
-
attempt: number; // markerAttempts() at write time
|
|
1308
|
-
at: number; // write time — bounds the in-flight-send wait below
|
|
1309
|
-
};
|
|
1310
|
-
declare const journal: {
|
|
1311
|
-
get(chatId: string): Promise<JournalEntry | undefined>;
|
|
1312
|
-
set(chatId: string, entry: JournalEntry): Promise<void>;
|
|
1313
|
-
};
|
|
1314
|
-
// This step's attempts committed so far: user messages in the converted
|
|
1315
|
-
// transcript carrying the marker. Steps run sequentially, so they sit
|
|
1316
|
-
// contiguous at the tail of history — page newest-first (as the recovery
|
|
1317
|
-
// scan below already does) and stop at the first user message without it.
|
|
1318
|
-
declare function markerAttempts(): Promise<number>;
|
|
1319
|
-
|
|
1320
|
-
// Bounded: a stalled deployment must fail the step, not hang it.
|
|
1321
|
-
const deadline = AbortSignal.timeout(15_000);
|
|
1322
|
-
// Whether an interrupt stopped a live run drives the resubmit decision
|
|
1323
|
-
// below: a 409 means there was nothing to stop.
|
|
1324
|
-
let cutShort = true;
|
|
1325
|
-
let { status } = await agent.client.getChat(agent.chatId!, deadline);
|
|
1326
|
-
// Recompute what the entry pinned at write time; a mismatch on either
|
|
1327
|
-
// means it classifies an earlier attempt (or another step's) — dead, skip.
|
|
1328
|
-
const entry = await journal.get(agent.chatId!);
|
|
1329
|
-
const attempts = await markerAttempts();
|
|
1330
|
-
const journaled = entry?.marker === marker && entry.attempt === attempts ? entry : undefined;
|
|
1331
|
-
if (status === "requires_action") {
|
|
1332
|
-
// The crash hit a client tool mid-execution: this pause is the only
|
|
1333
|
-
// pending record of its committed effects. Never interrupt it away —
|
|
1334
|
-
// reconcile the ledger instead (below): it submits recorded results
|
|
1335
|
-
// (reviving the run — nothing was cut short) or interrupts itself only
|
|
1336
|
-
// when nothing committed.
|
|
1337
|
-
cutShort = await reconcileClientTools();
|
|
1338
|
-
} else if (journaled && journaled.state !== "submitting") {
|
|
1339
|
-
// A previous recovery pass acted on this attempt, then died before the
|
|
1340
|
-
// step's result was recorded. Its journaled state is the verdict:
|
|
1341
|
-
// "resumed" — it revived the run by submitting recorded results;
|
|
1342
|
-
// interrupting now would cut down a run doing the step's work, and the
|
|
1343
|
-
// resubmitted prompt would mint NEW tool-call ids, sidestepping the
|
|
1344
|
-
// idempotency keys. Let it finish; nothing was cut short.
|
|
1345
|
-
// "cut-short" — it went on to interrupt (an unstarted sibling in the
|
|
1346
|
-
// batch); the attempt is truncated, so the check below must resubmit.
|
|
1347
|
-
cutShort = journaled.state === "cut-short";
|
|
1348
|
-
if (cutShort) {
|
|
1349
|
-
// The journal records write-ahead INTENT, not a landed interrupt: the
|
|
1350
|
-
// pass may have died in between, leaving the revived run working.
|
|
1351
|
-
// Re-drive the interrupt to make the intent true — a 409 means the
|
|
1352
|
-
// chat already settled; the cut-short verdict stands either way.
|
|
1353
|
-
await agent.interrupt({ signal: deadline }).catch((err) => {
|
|
1354
|
-
if (!(err instanceof CoderApiError && err.status === 409)) throw err;
|
|
1355
|
-
});
|
|
1356
|
-
}
|
|
1357
|
-
} else {
|
|
1358
|
-
await agent.interrupt({ signal: deadline }).catch((err) => {
|
|
1359
|
-
// A 409 proves nothing is stoppable NOW — that means "never cut short"
|
|
1360
|
-
// only if the probe saw no interrupt already in flight. A probed
|
|
1361
|
-
// "interrupting" is an earlier interrupt (a timeout's best-effort one,
|
|
1362
|
-
// a sweeper) already truncating the attempt; this 409 just says it
|
|
1363
|
-
// landed. Keep cutShort = true then.
|
|
1364
|
-
if (err instanceof CoderApiError && err.status === 409) cutShort = status === "interrupting";
|
|
1365
|
-
else throw err;
|
|
1366
|
-
});
|
|
1367
|
-
}
|
|
1368
|
-
// One thing can still be in flight here: the crashed attempt's own SEND. A
|
|
1369
|
-
// matching "submitting" entry — its attempt count has not grown — means the
|
|
1370
|
-
// process died with `createChatMessage` possibly en route, and the server
|
|
1371
|
-
// can commit it AFTER every read above (the 409 only said nothing was
|
|
1372
|
-
// running THEN). Sleep out what remains of the entry's commit window —
|
|
1373
|
-
// requestTimeoutMs past the write is the conservative horizon, and a
|
|
1374
|
-
// restart has usually consumed it already — so a late-landing turn starts
|
|
1375
|
-
// before the settle poll below and the next section's negative history
|
|
1376
|
-
// read is past the window. Then recount: a send that never landed OVER AN
|
|
1377
|
-
// EXISTING attempt pins cutShort — the step only ever plans a resubmission
|
|
1378
|
-
// over a cut-short, errored, or unsubmitted attempt, so the write-ahead
|
|
1379
|
-
// that superseded a "cut-short" verdict still carries its consequence, and
|
|
1380
|
-
// the scan below cannot mistake the old attempt's truncated tail for a
|
|
1381
|
-
// finished result.
|
|
1382
|
-
if (journaled?.state === "submitting") {
|
|
1383
|
-
await new Promise((r) => setTimeout(r, Math.max(0, journaled.at + 300_000 - Date.now())));
|
|
1384
|
-
if ((await markerAttempts()) === journaled.attempt && journaled.attempt > 0) cutShort = true;
|
|
1385
|
-
}
|
|
1386
|
-
// The 15 s interrupt bound must not cap what follows: a reconciled pause
|
|
1387
|
-
// revives a full model/tool continuation. Budget the settle poll — and the
|
|
1388
|
-
// history reads after it — on the turn scale, like the step itself.
|
|
1389
|
-
const settle = AbortSignal.timeout(600_000);
|
|
1390
|
-
do {
|
|
1391
|
-
({ status } = await agent.client.getChat(agent.chatId!, settle));
|
|
1392
|
-
if (status === "waiting" || status === "completed" || status === "error") break;
|
|
1393
|
-
// With no accepted interrupt racing to clear it, requires_action is a
|
|
1394
|
-
// STABLE pause: a run resumed by submitted tool results called a NEW
|
|
1395
|
-
// client tool. Exit and reconcile the ledger again (below) instead of
|
|
1396
|
-
// polling out the deadline.
|
|
1397
|
-
if (status === "requires_action" && !cutShort) break;
|
|
1398
|
-
await new Promise((r) => setTimeout(r, 1_000)); // acknowledged ≠ settled — wait it out
|
|
1399
|
-
} while (true);
|
|
1400
|
-
```
|
|
1401
|
-
|
|
1402
|
-
Settled is necessary, not sufficient: a terminal status doesn't say whether
|
|
1403
|
-
the crashed attempt already _finished the step's work_ — check history
|
|
1404
|
-
before resubmitting
|
|
1405
|
-
([Recover the result before resubmitting](#recover-the-result-before-resubmitting)).
|
|
1406
|
-
|
|
1407
|
-
### Make client tools crash-safe
|
|
1408
|
-
|
|
1409
|
-
Interrupting un‑strands a `requires_action` pause — but if the crashed
|
|
1410
|
-
process died _between_ a non‑idempotent client tool committing its external
|
|
1411
|
-
effect (a payment, a deploy, an email) and that result reaching the server,
|
|
1412
|
-
the pause is the **only pending record of the execution**. Interrupt it away
|
|
1413
|
-
and the resubmitted prompt is free to call the tool again — and duplicate the
|
|
1414
|
-
effect. Close the window with a tool‑invocation ledger in the same durable
|
|
1415
|
-
store as the `chatId` checkpoint, keyed by **chat id + `toolCallId`** — the
|
|
1416
|
-
server assigns the call id and commits it to history, so both sides of the
|
|
1417
|
-
reconciliation survive the crash. The chat id must be part of the key (and
|
|
1418
|
-
of the idempotency key handed to the external system): call ids are only
|
|
1419
|
-
unique _within_ a chat, and a store shared across workflows must never let
|
|
1420
|
-
one chat's `done` entry answer — or deduplicate — another chat's call. In
|
|
1421
|
-
multi‑tenant stores, fold the tenant in too:
|
|
630
|
+
**`maxRetries` defaults to `0`.** This agent owns server‑side chat state, so an
|
|
631
|
+
SDK‑level retry could duplicate a turn. Prefer catching `retryable` errors and
|
|
632
|
+
retrying the whole step deliberately.
|
|
1422
633
|
|
|
1423
|
-
|
|
1424
|
-
import { tool } from "ai";
|
|
1425
|
-
import { z } from "zod";
|
|
1426
|
-
import { CoderAgent } from "@coder/ai-sdk-agent";
|
|
634
|
+
## Cleanup
|
|
1427
635
|
|
|
1428
|
-
|
|
1429
|
-
|
|
1430
|
-
get(key: string): Promise<LedgerEntry | undefined>;
|
|
1431
|
-
set(key: string, entry: LedgerEntry): Promise<void>;
|
|
1432
|
-
};
|
|
1433
|
-
declare const payments: {
|
|
1434
|
-
charge(amountCents: number, opts: { idempotencyKey: string }): Promise<{ receiptId: string }>;
|
|
1435
|
-
};
|
|
1436
|
-
|
|
1437
|
-
const ledgerKey = (chatId: string, toolCallId: string) => `${chatId}/${toolCallId}`;
|
|
1438
|
-
const ChargeArgs = z.object({ amountCents: z.number().int() });
|
|
1439
|
-
|
|
1440
|
-
// A factory taking a getter, not an id: the chat may not exist until the
|
|
1441
|
-
// turn creates it, so the id must be read lazily — it is set before any
|
|
1442
|
-
// tool runs.
|
|
1443
|
-
const chargeCard = (chatId: () => string) =>
|
|
1444
|
-
tool({
|
|
1445
|
-
description: "Charge the customer's card.",
|
|
1446
|
-
inputSchema: ChargeArgs, // shared with recovery's re-drive dispatch below
|
|
1447
|
-
execute: async ({ amountCents }, { toolCallId }) => {
|
|
1448
|
-
const key = ledgerKey(chatId(), toolCallId);
|
|
1449
|
-
// Write-ahead, BEFORE the effect: a crash between the two writes leaves
|
|
1450
|
-
// "committing" (ambiguous, but resolvable below); no entry at all proves
|
|
1451
|
-
// the effect never started.
|
|
1452
|
-
await ledger.set(key, { state: "committing" });
|
|
1453
|
-
// Hand the external system the SAME key as its idempotency key, so
|
|
1454
|
-
// re-driving this exact call can never double-charge.
|
|
1455
|
-
const { receiptId } = await payments.charge(amountCents, { idempotencyKey: key });
|
|
1456
|
-
await ledger.set(key, { state: "done", output: `charged: receipt ${receiptId}` });
|
|
1457
|
-
return `charged: receipt ${receiptId}`;
|
|
1458
|
-
},
|
|
1459
|
-
});
|
|
636
|
+
The agent is an **async disposable**, so cleanup can ride scope exit instead of a
|
|
637
|
+
`finally` you have to remember:
|
|
1460
638
|
|
|
1461
|
-
|
|
1462
|
-
|
|
1463
|
-
|
|
1464
|
-
|
|
1465
|
-
/* …base options as in runTurn… */
|
|
1466
|
-
tools: { charge_card: chargeCard(() => self!.chatId!) },
|
|
1467
|
-
});
|
|
1468
|
-
self = agent;
|
|
639
|
+
```ts
|
|
640
|
+
await using agent = new CoderAgent({ organizationId });
|
|
641
|
+
const { text } = await agent.generate({ prompt: "…" });
|
|
642
|
+
// agent.interrupt() + agent.archive() run automatically when the scope exits.
|
|
1469
643
|
```
|
|
1470
644
|
|
|
1471
|
-
|
|
1472
|
-
|
|
1473
|
-
|
|
1474
|
-
|
|
645
|
+
- Disposal interrupts any in‑flight run, then archives.
|
|
646
|
+
- Disposal is **bounded and never throws** (~15s overall, best‑effort). Its
|
|
647
|
+
errors are swallowed so they can't mask the scope's own error. Call
|
|
648
|
+
`archive()` directly when you need guaranteed cleanup.
|
|
649
|
+
- In a request handler that returns before a fire‑and‑forget `archive()`
|
|
650
|
+
settles, the archive can be abandoned. `await using` (or an awaited
|
|
651
|
+
`archive()` in `finally`) avoids accumulating live chats.
|
|
652
|
+
|
|
653
|
+
**What `archive()` does:**
|
|
654
|
+
|
|
655
|
+
- It soft‑hides the chat: the chat stays in listings as `archived: true`. There
|
|
656
|
+
is no hard delete yet.
|
|
657
|
+
- A freshly interrupted chat keeps winding down server‑side for a few seconds,
|
|
658
|
+
and archiving 409s meanwhile. `archive()` retries those 409s (~1s apart, up to
|
|
659
|
+
~15s overall; tune with `settleDeadlineMs` / `settleRetryDelayMs`) and
|
|
660
|
+
rethrows the last one if the chat never settles.
|
|
661
|
+
- Any other failure, including your own abort, rethrows immediately.
|
|
662
|
+
|
|
663
|
+
**Return values.** Both methods report what they acted on instead of silently
|
|
664
|
+
no‑oping:
|
|
665
|
+
|
|
666
|
+
| Method | Acted on a chat | No chat exists at all |
|
|
667
|
+
| ------------- | ----------------------------------------------------------------------------------------------- | ------------------------ |
|
|
668
|
+
| `archive()` | `{ archived: true, chatId, archivedChatIds }` (each archived id is cleared as a cleanup target) | `{ archived: false }` |
|
|
669
|
+
| `interrupt()` | `{ interrupted: true, chatId }` | `{ interrupted: false }` |
|
|
670
|
+
|
|
671
|
+
<details>
|
|
672
|
+
<summary>Which chat <code>archive()</code> and <code>interrupt()</code> target after a session is dropped</summary>
|
|
673
|
+
|
|
674
|
+
- They target the session's chat. After the session was dropped —
|
|
675
|
+
`resetSession()`, or the automatic discard after a fresh‑chat stream failure
|
|
676
|
+
(see [Handling errors](#handling-errors)) — they target the **last‑known chat
|
|
677
|
+
id** (`agent.lastKnownChatId`), so a stranded chat is still cleaned up instead
|
|
678
|
+
of leaking.
|
|
679
|
+
- Generation never uses the last‑known id: a turn after a drop creates a fresh
|
|
680
|
+
chat as always.
|
|
681
|
+
- Every chat stranded by an _automatic_ discard is also recorded on a ledger,
|
|
682
|
+
`agent.strandedChatIds` (oldest first). With `maxRetries`, several failed
|
|
683
|
+
attempts can strand one chat each while only the final attempt's error
|
|
684
|
+
surfaces. One `archive()` retires them all, oldest first, alongside its
|
|
685
|
+
primary target.
|
|
686
|
+
- Deliberate abandonment is different: `resetSession()` does **not** add to the
|
|
687
|
+
ledger (you may want that chat kept). After a manual reset, the old chat is
|
|
688
|
+
targetable only until a new chat supersedes `lastKnownChatId`.
|
|
689
|
+
|
|
690
|
+
</details>
|
|
1475
691
|
|
|
1476
|
-
|
|
1477
|
-
// The step's configured ToolSet names: history does not record which calls
|
|
1478
|
-
// were client tools (see [Rehydrating chat history]), so — like the SDK's
|
|
1479
|
-
// own derivation — restrict to the names this step registers; a server/MCP
|
|
1480
|
-
// call must never reach the ledger as "unstarted". (The recovery scan's
|
|
1481
|
-
// final-text cut reuses this set — next section.)
|
|
1482
|
-
const clientTools = new Set(["charge_card"]);
|
|
1483
|
-
|
|
1484
|
-
// Pending = the last assistant message's client (non-provider-executed,
|
|
1485
|
-
// ToolSet-named) tool calls, minus ids answered by a later message's tool
|
|
1486
|
-
// result.
|
|
1487
|
-
const { messages } = await agent.client.getMessages(agent.chatId!, { limit: 200 }, deadline);
|
|
1488
|
-
const ordered = [...messages].sort((a, b) => a.id - b.id);
|
|
1489
|
-
const lastAssistant = ordered.findLast((m) => m.role === "assistant");
|
|
1490
|
-
const handled = new Set(
|
|
1491
|
-
ordered
|
|
1492
|
-
.filter((m) => m.id > (lastAssistant?.id ?? 0))
|
|
1493
|
-
.flatMap((m) => m.content ?? [])
|
|
1494
|
-
.filter((p) => p.type === "tool-result")
|
|
1495
|
-
.map((p) => p.tool_call_id),
|
|
1496
|
-
);
|
|
1497
|
-
const pending = (lastAssistant?.content ?? []).filter(
|
|
1498
|
-
(p) =>
|
|
1499
|
-
p.type === "tool-call" &&
|
|
1500
|
-
!p.provider_executed &&
|
|
1501
|
-
clientTools.has(p.tool_name!) &&
|
|
1502
|
-
!handled.has(p.tool_call_id),
|
|
1503
|
-
);
|
|
1504
|
-
|
|
1505
|
-
const results: { tool_call_id: string; output: unknown }[] = [];
|
|
1506
|
-
let unstarted = false;
|
|
1507
|
-
for (const call of pending) {
|
|
1508
|
-
const key = ledgerKey(agent.chatId!, call.tool_call_id!);
|
|
1509
|
-
const entry = await ledger.get(key);
|
|
1510
|
-
if (entry?.state === "done") {
|
|
1511
|
-
// Effect committed, result stranded: submit the record — the pause is
|
|
1512
|
-
// answered and nothing re-executes.
|
|
1513
|
-
results.push({ tool_call_id: call.tool_call_id!, output: entry.output });
|
|
1514
|
-
} else if (entry) {
|
|
1515
|
-
// "committing" is ambiguous — the crash hit between the write-ahead and
|
|
1516
|
-
// "done". Safe to re-drive: the SAME idempotency key makes a committed
|
|
1517
|
-
// effect replay as a no-op, and the call's args are committed in history.
|
|
1518
|
-
// Dispatch by tool name — every ledgered tool needs its own re-drive
|
|
1519
|
-
// path, its args validated exactly as the live loop would have.
|
|
1520
|
-
if (call.tool_name !== "charge_card")
|
|
1521
|
-
throw new Error(`no re-drive path for tool ${call.tool_name}`);
|
|
1522
|
-
const { amountCents } = ChargeArgs.parse(call.args);
|
|
1523
|
-
const { receiptId } = await payments.charge(amountCents, { idempotencyKey: key });
|
|
1524
|
-
const output = `charged: receipt ${receiptId}`;
|
|
1525
|
-
await ledger.set(key, { state: "done", output });
|
|
1526
|
-
results.push({ tool_call_id: call.tool_call_id!, output });
|
|
1527
|
-
} else {
|
|
1528
|
-
unstarted = true; // the tool never ran — interrupting loses nothing
|
|
1529
|
-
}
|
|
1530
|
-
}
|
|
1531
|
-
if (results.length > 0 || unstarted) {
|
|
1532
|
-
// ONE write-ahead with the batch's FINAL verdict, before any network
|
|
1533
|
-
// call — a mixed batch journals "cut-short" before even its submission,
|
|
1534
|
-
// so no crash point can leave a stale "resumed" masking the pending
|
|
1535
|
-
// interrupt. Every crash then either re-reconciles the stable pause or
|
|
1536
|
-
// re-drives the journaled intent. Scoped to THIS attempt: a resubmission
|
|
1537
|
-
// grows the marker count and retires the verdict on its own.
|
|
1538
|
-
await journal.set(agent.chatId!, {
|
|
1539
|
-
state: unstarted ? "cut-short" : "resumed",
|
|
1540
|
-
marker,
|
|
1541
|
-
attempt: await markerAttempts(),
|
|
1542
|
-
at: Date.now(),
|
|
1543
|
-
});
|
|
1544
|
-
}
|
|
1545
|
-
if (results.length > 0) await agent.client.submitToolResults(agent.chatId!, { results }, deadline);
|
|
1546
|
-
if (unstarted) await agent.client.interruptChat(agent.chatId!, deadline);
|
|
1547
|
-
// As `reconcileClientTools()` in the settle-wait snippet above: resolve to
|
|
1548
|
-
// `unstarted` — cutShort is true only when this fell back to interrupting.
|
|
1549
|
-
```
|
|
692
|
+
## Usage & cost
|
|
1550
693
|
|
|
1551
|
-
|
|
1552
|
-
|
|
1553
|
-
|
|
1554
|
-
|
|
1555
|
-
|
|
1556
|
-
|
|
1557
|
-
|
|
1558
|
-
|
|
1559
|
-
|
|
1560
|
-
|
|
1561
|
-
|
|
1562
|
-
|
|
1563
|
-
|
|
1564
|
-
|
|
1565
|
-
|
|
1566
|
-
|
|
1567
|
-
|
|
1568
|
-
|
|
1569
|
-
|
|
1570
|
-
|
|
1571
|
-
|
|
1572
|
-
|
|
1573
|
-
|
|
1574
|
-
|
|
1575
|
-
|
|
1576
|
-
|
|
1577
|
-
|
|
1578
|
-
|
|
1579
|
-
|
|
1580
|
-
|
|
1581
|
-
|
|
694
|
+
`result.usage` reflects what the whole turn actually consumed:
|
|
695
|
+
|
|
696
|
+
- A chatd turn runs several model steps server‑side (one per server tool round),
|
|
697
|
+
each reporting its own usage. The SDK **sums every step**, and the AI SDK adds
|
|
698
|
+
up the steps of a turn that paused for client tools.
|
|
699
|
+
- `inputTokens` is the full prompt size. Coder normalizes the wire
|
|
700
|
+
`input_tokens` to the _uncached_ count (cache reads/writes are separate
|
|
701
|
+
fields), so the SDK adds them back into the total and exposes the split via
|
|
702
|
+
`inputTokenDetails` (`noCacheTokens`, `cacheReadTokens`, `cacheWriteTokens`).
|
|
703
|
+
|
|
704
|
+
**Cost and runtime.** When the server reports them, `total_cost_micros`
|
|
705
|
+
(micro‑USD) and `total_runtime_ms` are mirrored under each step's
|
|
706
|
+
`providerMetadata.coder`.
|
|
707
|
+
|
|
708
|
+
- `result.providerMetadata` reflects only the final step. Sum
|
|
709
|
+
`result.steps[*].providerMetadata.coder` for whole‑turn cost when client tools
|
|
710
|
+
ran.
|
|
711
|
+
- Both are absence‑tolerant mirrors: on servers that don't send them, nothing is
|
|
712
|
+
emitted. Cost is otherwise only on the aggregate cost endpoints
|
|
713
|
+
(`/api/v2/chats/cost/*`).
|
|
714
|
+
|
|
715
|
+
<details>
|
|
716
|
+
<summary>Raw wire usage per step (<code>steps[i].usage.raw</code>)</summary>
|
|
717
|
+
|
|
718
|
+
- The snake_case wire usage lives **per step** at `result.steps[i].usage.raw`.
|
|
719
|
+
The AI SDK does not carry `raw` onto the summed `result.usage`.
|
|
720
|
+
- Use it for fields the normalized shape has no slot for: `context_limit`, cost,
|
|
721
|
+
runtime, and any newer wire fields (which pass through newest‑value‑wins).
|
|
722
|
+
- `raw` keeps the wire convention (`input_tokens` = uncached only). Its counters
|
|
723
|
+
are summed over that step's server‑side model steps, with `context_limit` from
|
|
724
|
+
the newest one.
|
|
725
|
+
- So don't divide `raw`'s summed counters by `context_limit` to estimate context
|
|
726
|
+
fullness. They are turn consumption, not a prompt‑size snapshot.
|
|
727
|
+
|
|
728
|
+
</details>
|
|
729
|
+
|
|
730
|
+
Forward usage to a UI via message metadata:
|
|
1582
731
|
|
|
1583
732
|
```ts
|
|
1584
|
-
const
|
|
1585
|
-
|
|
1586
|
-
|
|
1587
|
-
|
|
1588
|
-
|
|
1589
|
-
|
|
1590
|
-
|
|
1591
|
-
// send-crash window belongs to the empty-checkpoint sweep
|
|
1592
|
-
// ("Recover after a crash").
|
|
1593
|
-
if (agent.chatId) {
|
|
1594
|
-
await journal.set(agent.chatId, {
|
|
1595
|
-
state: "submitting",
|
|
1596
|
-
marker,
|
|
1597
|
-
attempt: await markerAttempts(),
|
|
1598
|
-
at: Date.now(),
|
|
1599
|
-
});
|
|
1600
|
-
}
|
|
1601
|
-
const { text } = await agent.generate({ prompt: `${marker} ${prompt}` });
|
|
733
|
+
const result = await agent.stream({ prompt: "…" });
|
|
734
|
+
return result.toUIMessageStream({
|
|
735
|
+
messageMetadata: ({ part }) =>
|
|
736
|
+
part.type === "finish-step"
|
|
737
|
+
? { usage: part.usage, coder: part.providerMetadata?.coder }
|
|
738
|
+
: undefined,
|
|
739
|
+
});
|
|
1602
740
|
```
|
|
1603
741
|
|
|
1604
|
-
|
|
1605
|
-
of them resubmits:
|
|
742
|
+
## Observability
|
|
1606
743
|
|
|
1607
|
-
|
|
1608
|
-
|
|
744
|
+
Pass `onTransportEvent` to receive typed events for HTTP exchanges, the per‑chat
|
|
745
|
+
stream's WebSocket lifecycle, and turn‑segment boundaries. You get timing and
|
|
746
|
+
tracing without wrapping `fetch`/`webSocketFactory` or re‑parsing stream frames.
|
|
1609
747
|
|
|
1610
|
-
|
|
1611
|
-
|
|
1612
|
-
|
|
1613
|
-
// truncated read, or this recovery resubmits exactly the turn it was meant
|
|
1614
|
-
// to deduplicate.
|
|
1615
|
-
const fetched = [];
|
|
1616
|
-
let page = await agent.client.getMessages(agent.chatId!, { limit: 200 }, settle);
|
|
1617
|
-
fetched.push(...page.messages);
|
|
1618
|
-
while (!fetched.some((m) => m.role === "user") && page.has_more) {
|
|
1619
|
-
const oldestId = Math.min(...fetched.map((m) => m.id));
|
|
1620
|
-
page = await agent.client.getMessages(agent.chatId!, { before_id: oldestId, limit: 200 }, settle);
|
|
1621
|
-
fetched.push(...page.messages);
|
|
1622
|
-
}
|
|
1623
|
-
const transcript = chatMessagesToUIMessages(fetched); // chronological
|
|
1624
|
-
const lastUser = transcript.findLast((m) => m.role === "user");
|
|
1625
|
-
const submitted = lastUser?.parts.some((p) => p.type === "text" && p.text.includes(marker));
|
|
1626
|
-
|
|
1627
|
-
if (submitted && !cutShort && (status === "waiting" || status === "completed")) {
|
|
1628
|
-
// The attempt finished; only the recording was lost. Recover what
|
|
1629
|
-
// `generate().text` would have returned — the FINAL step's text. Only a
|
|
1630
|
-
// CLIENT tool ends an AI SDK step, and narration lands on both sides of a
|
|
1631
|
-
// server-side call, so cut after the last part named in `clientTools`
|
|
1632
|
-
// (shared with the reconcile above) and join the text that follows.
|
|
1633
|
-
const parts = transcript
|
|
1634
|
-
.slice(transcript.indexOf(lastUser!) + 1)
|
|
1635
|
-
.filter((m) => m.role === "assistant")
|
|
1636
|
-
.flatMap((m) => m.parts);
|
|
1637
|
-
return parts
|
|
1638
|
-
.slice(parts.findLastIndex((p) => p.type === "dynamic-tool" && clientTools.has(p.toolName)) + 1)
|
|
1639
|
-
.filter((p) => p.type === "text")
|
|
1640
|
-
.map((p) => p.text)
|
|
1641
|
-
.join("");
|
|
1642
|
-
}
|
|
1643
|
-
// Resubmit only what never finished: the prompt never reached the server
|
|
1644
|
-
// (!submitted — sound only behind the write-ahead; see below), the turn
|
|
1645
|
-
// failed (status "error" — normal retry judgment applies), or the interrupt
|
|
1646
|
-
// cut a live run short (cutShort — the model sees its own aborted attempt in
|
|
1647
|
-
// history and continues from it). One exit is neither: a requires_action
|
|
1648
|
-
// settle goes back to ledger reconciliation (previous section) — never to
|
|
1649
|
-
// recovery or resubmission.
|
|
1650
|
-
```
|
|
748
|
+
Handler exceptions are swallowed, there is zero overhead without a handler, and
|
|
749
|
+
events carry no headers or tokens. Example, event reference, and semantics:
|
|
750
|
+
[docs/observability.md](./docs/observability.md).
|
|
1651
751
|
|
|
1652
|
-
|
|
1653
|
-
proves nothing is running _now_.** An interrupted attempt leaves the same
|
|
1654
|
-
marker and a truncated answer in history, so history alone can't tell
|
|
1655
|
-
"finished" from "stopped"; what can is whether any interrupt stopped the
|
|
1656
|
-
run. The inference "409 ⇒ the attempt finished" is therefore sound only
|
|
1657
|
-
when recovery's interrupt is the _first_ one that could have: true after a
|
|
1658
|
-
hard crash, false after a `requestTimeoutMs` expiry — the expiry already
|
|
1659
|
-
fired the SDK's own best‑effort interrupt, which may have stopped the run
|
|
1660
|
-
before recovery ever looked. Carry the reason recovery ran: when it follows
|
|
1661
|
-
a known timeout or abort — or a matching journaled `"cut-short"`, or an
|
|
1662
|
-
unlanded `"submitting"` over an existing attempt (the settle‑wait already
|
|
1663
|
-
pins both) — pin `cutShort` to `true` and let only the hard‑crash path
|
|
1664
|
-
trust the 409.
|
|
1665
|
-
- **`!submitted` races the crashed attempt's own send.** A crash kills the
|
|
1666
|
-
process, not necessarily its in‑flight `createChatMessage` — the server can
|
|
1667
|
-
commit that request _after_ an interrupt 409'd and a history read saw no
|
|
1668
|
-
marker, and a resubmit on that evidence queues the turn twice, duplicating
|
|
1669
|
-
its workspace/MCP/tool effects. The write‑ahead is what makes the negative
|
|
1670
|
-
read honest: no matching `"submitting"` entry proves no send ever started,
|
|
1671
|
-
and a matching one held recovery at the settle‑wait until the entry's
|
|
1672
|
-
commit horizon passed. That narrows the window; only server‑enforced
|
|
1673
|
-
submission idempotency could close it, and `POST /chats/{id}/messages`
|
|
1674
|
-
offers none today. A commit that outlasts even the horizon still lands
|
|
1675
|
-
under the same marker, so the duplicate is at least visible — in history,
|
|
1676
|
-
and to the next consult's attempt count.
|
|
1677
|
-
- **The cut lands at the last _configured client‑tool_ boundary.** History
|
|
1678
|
-
does not mark which calls were client tools
|
|
1679
|
-
([Rehydrating chat history](#rehydrating-chat-history)), so the cut
|
|
1680
|
-
classifies by name against the reconcile's `clientTools` set — narration
|
|
1681
|
-
before a trailing _server‑side_ call survives, matching what
|
|
1682
|
-
`generate().text` aggregates. What name‑matching cannot see: a call
|
|
1683
|
-
recorded under a name the step no longer configures (a tool renamed
|
|
1684
|
-
between deploys) reads as server‑side and leaks earlier‑step narration
|
|
1685
|
-
into the result, and a server tool sharing a configured name still cuts
|
|
1686
|
-
final‑step narration away.
|
|
1687
|
-
- **Steps with structured results recover the call's input, not the text.**
|
|
1688
|
-
A [structured output](#structured-output) step's answer is the
|
|
1689
|
-
`structured_output` call's typed input, which rehydrates as a
|
|
1690
|
-
`dynamic-tool` part on the recovered turn
|
|
1691
|
-
([Rehydrating chat history](#rehydrating-chat-history)) — and
|
|
1692
|
-
`structured_output` is one of the step's configured client tools, so the
|
|
1693
|
-
text join above cuts at it and returns only the ack prose that follows.
|
|
1694
|
-
Recover the filed call instead — this scan replaces the text join
|
|
1695
|
-
_inside_ the recovery branch above, reusing its `parts` — validated
|
|
1696
|
-
client‑side exactly like the live path (rule 2 there — the schema is the
|
|
1697
|
-
real gate):
|
|
1698
|
-
|
|
1699
|
-
```ts
|
|
1700
|
-
// Scan backward to the last call that VALIDATES, as in the live path — a
|
|
1701
|
-
// schema-invalid re-file must not shadow a valid answer.
|
|
1702
|
-
const filed = parts
|
|
1703
|
-
.filter((p) => p.type === "dynamic-tool")
|
|
1704
|
-
.filter((p) => p.toolName === "structured_output");
|
|
1705
|
-
let answer: z.infer<typeof Answer> | undefined;
|
|
1706
|
-
for (const call of filed.reverse()) {
|
|
1707
|
-
const parsed = Answer.safeParse(call.input);
|
|
1708
|
-
if (parsed.success) {
|
|
1709
|
-
answer = parsed.data;
|
|
1710
|
-
break;
|
|
1711
|
-
}
|
|
1712
|
-
}
|
|
1713
|
-
// recovered ⇔ answer !== undefined
|
|
1714
|
-
```
|
|
752
|
+
## Workspaces & quota
|
|
1715
753
|
|
|
1716
|
-
|
|
754
|
+
A chat may be backed by a **Coder workspace** that runs its tools, and
|
|
755
|
+
workspaces are the scarce resource: **N agents running concurrently can need N
|
|
756
|
+
schedulable workspaces**. Past that bound, a turn can sit unscheduled and never
|
|
757
|
+
settle. Read [docs/workspaces-and-quota.md](./docs/workspaces-and-quota.md)
|
|
758
|
+
before running fleets: how chats bind to workspaces, fleet sizing, autostop and
|
|
759
|
+
cleanup, preventing stuck turns, and troubleshooting.
|
|
1717
760
|
|
|
1718
|
-
|
|
1719
|
-
parsing errors or stream frames — emit them as step metrics/traces:
|
|
761
|
+
## Durable workflows: persist, resume, recover
|
|
1720
762
|
|
|
1721
|
-
|
|
1722
|
-
|
|
1723
|
-
|
|
1724
|
-
|
|
1725
|
-
|
|
1726
|
-
|
|
1727
|
-
|
|
1728
|
-
function observe(ev: CoderTransportEvent): void {
|
|
1729
|
-
if (ev.type === "ws:redial") {
|
|
1730
|
-
// Early warning: the stream dropped and is reconnecting. The server keeps
|
|
1731
|
-
// generating — nothing is lost yet; the turn fails if this hits the cap.
|
|
1732
|
-
console.warn(
|
|
1733
|
-
`redial ${ev.consecutiveFailures}/${ev.maxConsecutiveFailures}, next in ${ev.backoffMs}ms`,
|
|
1734
|
-
);
|
|
1735
|
-
} else if (ev.type === "segment:settle") {
|
|
1736
|
-
if (ev.finishReason) metrics.ok(ev.segment, ev.status!, ev.durationMs);
|
|
1737
|
-
else if (ev.error) metrics.fail(ev.segment, ev.error.name, ev.error.message);
|
|
1738
|
-
else metrics.teardown(ev.segment);
|
|
1739
|
-
}
|
|
1740
|
-
}
|
|
1741
|
-
```
|
|
763
|
+
Run **one agent session across process boundaries** — queue jobs,
|
|
764
|
+
durable‑workflow steps (Vercel Workflow, step functions, Temporal, …), cron
|
|
765
|
+
ticks — and survive crashes, stream drops, and timeouts in between. All chat
|
|
766
|
+
state lives on the Coder server, so a workflow carries only `agent.chatId` (plus,
|
|
767
|
+
optionally, the `agent.lastSeenMessageId` resume cursor) between steps, and
|
|
768
|
+
each turn runs inside a durable step. Guide: [docs/durable-workflows.md](./docs/durable-workflows.md).
|
|
1742
769
|
|
|
1743
|
-
|
|
1744
|
-
three endings a workflow cares about:
|
|
1745
|
-
|
|
1746
|
-
- **Clean settle** — `status` + `finishReason` present. `finishReason:
|
|
1747
|
-
"tool-calls"` (status `requires_action`) is a healthy mid‑turn pause for
|
|
1748
|
-
your client tools; terminal statuses end the turn.
|
|
1749
|
-
- **Failure** — `error: { name, message }` present (plus `status` when the
|
|
1750
|
-
server run itself still settled terminally): server‑side turn errors,
|
|
1751
|
-
transport failure after redials were exhausted, `requestTimeoutMs` expiry
|
|
1752
|
-
(`error.name: "CoderChatError"`), and caller aborts — which settle as
|
|
1753
|
-
failures carrying the abort reason's name, never as teardowns:
|
|
1754
|
-
`"TimeoutError"` for an `AbortSignal.timeout(…)` deadline (the running
|
|
1755
|
-
example), `"AbortError"` for an explicit abort.
|
|
1756
|
-
- **Teardown** — neither: the stream consumer itself cancelled the turn
|
|
1757
|
-
(`stream()`'s `ReadableStream.cancel()` with no abort of its own) — rare in
|
|
1758
|
-
`generate()`‑shaped steps.
|
|
1759
|
-
|
|
1760
|
-
The hook is observability‑only — handler exceptions are swallowed and can't
|
|
1761
|
-
disturb the turn, and an unset hook costs nothing. For monitoring _across_
|
|
1762
|
-
steps (chats stuck before a watcher started, fleet‑level sweeps), see
|
|
1763
|
-
[Preventing stuck turns](#preventing-stuck-turns).
|
|
1764
|
-
|
|
1765
|
-
### Bound every step
|
|
1766
|
-
|
|
1767
|
-
`requestTimeoutMs` arms **one timer per segment** covering everything the
|
|
1768
|
-
segment does: the REST phase (chat creation, message/tool‑result submission,
|
|
1769
|
-
uploads), the stream — and, the part that matters here, **any redial backoff
|
|
1770
|
-
inside the segment**. A segment that spends 12 s reconnecting has 12 s less
|
|
1771
|
-
budget; redials never reset the clock, so a flaky network can't stretch a step
|
|
1772
|
-
past its bound. On expiry the call rejects with `CoderChatError`
|
|
1773
|
-
(`kind: "timeout"`, `retryable: true`) and the server run is interrupted
|
|
1774
|
-
best‑effort.
|
|
1775
|
-
|
|
1776
|
-
Two workflow‑specific notes:
|
|
1777
|
-
|
|
1778
|
-
- A turn that times out (or crashes) before its chat id arrived leaves nothing
|
|
1779
|
-
to _resume_ — the checkpoint stays empty and the retried step starts fresh —
|
|
1780
|
-
but possibly an unacknowledged chat to sweep (see
|
|
1781
|
-
[Recover after a crash](#recover-after-a-crash)).
|
|
1782
|
-
- `requestTimeoutMs` bounds each _segment_; a turn driving client tools runs
|
|
1783
|
-
several. Cap a step's total wall‑clock with
|
|
1784
|
-
`abortSignal: AbortSignal.timeout(…)` —
|
|
1785
|
-
[Timeouts & cancellation](#timeouts--cancellation).
|
|
1786
|
-
|
|
1787
|
-
### Archive at the end — and only at the end
|
|
1788
|
-
|
|
1789
|
-
**An archived chat cannot be resumed:** the server rejects new messages on it
|
|
1790
|
-
(400, `Cannot send messages to an archived chat`). The per‑request habits from
|
|
1791
|
-
[Cleanup](#cleanup) — `await using`, `archive()` in a `finally` — would
|
|
1792
|
-
destroy the session from inside an intermediate step, so here the _workflow_
|
|
1793
|
-
owns the chat's end of life: archive once, in the final step, and in the
|
|
1794
|
-
workflow's failure/compensation handler so abandoned runs don't accumulate
|
|
1795
|
-
live chats.
|
|
770
|
+
## How it works
|
|
1796
771
|
|
|
1797
|
-
```ts
|
|
1798
|
-
export async function archiveWorkflowChat(workflowId: string): Promise<void> {
|
|
1799
|
-
const checkpoint = await checkpoints.get(workflowId);
|
|
1800
|
-
if (!checkpoint) return; // the workflow died before its first turn created a chat
|
|
1801
|
-
const agent = new CoderAgent({
|
|
1802
|
-
baseUrl: process.env.CODER_URL!,
|
|
1803
|
-
token: process.env.CODER_SESSION_TOKEN!,
|
|
1804
|
-
organizationId: process.env.CODER_ORG_ID!,
|
|
1805
|
-
chatId: checkpoint.chatId,
|
|
1806
|
-
});
|
|
1807
|
-
// A crashed step may have left a run live — stop it first, bounded. On an
|
|
1808
|
-
// already-settled chat the interrupt rejects with a 409; ignore it.
|
|
1809
|
-
await agent.interrupt({ signal: AbortSignal.timeout(15_000) }).catch(() => {});
|
|
1810
|
-
await agent.archive(); // retries 409s while the interrupted run winds down (~15s)
|
|
1811
|
-
}
|
|
1812
772
|
```
|
|
773
|
+
CoderAgent (implements ai.Agent)
|
|
774
|
+
└─ ToolLoopAgent (ai) ← inherits generate()/stream(), loop control
|
|
775
|
+
└─ CoderLanguageModel ← implements @ai-sdk/provider LanguageModelV4
|
|
776
|
+
└─ CoderChatClient ← REST + WebSocket to /api/v2/chats
|
|
777
|
+
└─ Coder Agents ← runs the agent loop SERVER-side
|
|
778
|
+
```
|
|
779
|
+
|
|
780
|
+
One `doStream` call advances the chat until it **settles** (`waiting`/`completed`)
|
|
781
|
+
or **pauses** for a client tool (`requires_action`). The SDK loop and the
|
|
782
|
+
server‑side loop mesh at the client‑tool boundary, so there's no double loop.
|
|
783
|
+
|
|
784
|
+
<details>
|
|
785
|
+
<summary>How streamed text avoids double‑counting</summary>
|
|
1813
786
|
|
|
1814
|
-
|
|
1815
|
-
|
|
1816
|
-
|
|
1817
|
-
|
|
1818
|
-
-
|
|
1819
|
-
|
|
1820
|
-
-
|
|
1821
|
-
|
|
1822
|
-
-
|
|
1823
|
-
|
|
1824
|
-
|
|
1825
|
-
|
|
1826
|
-
[Workspaces & quota](#workspaces--quota).
|
|
1827
|
-
- Steps that don't need server‑side tools (plan / extract / synthesize) are
|
|
1828
|
-
cheaper and natively structured through
|
|
1829
|
-
[`@coder/ai-sdk-provider`](../provider) + `generateObject` — no chat, no
|
|
1830
|
-
workspace, no cleanup.
|
|
1831
|
-
- Archive in the final step / failure handler — never per step.
|
|
787
|
+
Streaming text is emitted from `message_part` deltas. Every `message` snapshot is
|
|
788
|
+
then reconciled against a per‑message emitted‑content ledger, so nothing
|
|
789
|
+
double‑counts:
|
|
790
|
+
|
|
791
|
+
- a trailing snapshot after deltas is a no‑op;
|
|
792
|
+
- a fast snapshot‑only turn emits in full;
|
|
793
|
+
- a message that commits while the stream is redialing yields exactly its
|
|
794
|
+
missing tail;
|
|
795
|
+
- a revision that appends to an earlier message yields the appended suffix;
|
|
796
|
+
- rewrites that can't be expressed as deltas are safely suppressed.
|
|
797
|
+
|
|
798
|
+
</details>
|
|
1832
799
|
|
|
1833
800
|
## Testing
|
|
1834
801
|
|
|
@@ -1841,7 +808,8 @@ pnpm check # format check + lint + typecheck (CI gate)
|
|
|
1841
808
|
pnpm build
|
|
1842
809
|
```
|
|
1843
810
|
|
|
1844
|
-
End‑to‑end tests run against a live Coder deployment and are opt‑in via env
|
|
811
|
+
End‑to‑end tests run against a live Coder deployment and are opt‑in via env. The
|
|
812
|
+
suite creates **new chats only** (no workspaces) and archives them afterward:
|
|
1845
813
|
|
|
1846
814
|
```bash
|
|
1847
815
|
CODER_URL=https://dogfood.cdr.dev \
|
|
@@ -1849,15 +817,15 @@ CODER_SESSION_TOKEN=$(coder tokens create --name e2e) \
|
|
|
1849
817
|
pnpm test:e2e
|
|
1850
818
|
```
|
|
1851
819
|
|
|
1852
|
-
The e2e suite creates **new chats only** (no workspaces) and archives them afterward.
|
|
1853
|
-
|
|
1854
820
|
## Limitations
|
|
1855
821
|
|
|
1856
|
-
- Older deployments use a temporary experimental
|
|
1857
|
-
guarantee that every chat feature is available (see
|
|
1858
|
-
|
|
1859
|
-
-
|
|
1860
|
-
|
|
822
|
+
- Older deployments use a temporary experimental‑route fallback; it is not a
|
|
823
|
+
guarantee that every chat feature is available (see
|
|
824
|
+
[API compatibility](#api-compatibility)).
|
|
825
|
+
- Designed for Node (WebSocket via `ws`); a browser build can inject a
|
|
826
|
+
`webSocketFactory`.
|
|
827
|
+
- A v7 `@ai-sdk/harness` adapter (the conceptually exact fit) is a future
|
|
828
|
+
direction once that experimental API stabilizes.
|
|
1861
829
|
|
|
1862
830
|
## License
|
|
1863
831
|
|