@coder/ai-sdk-agent 0.6.2 → 0.6.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +353 -9
- package/dist/index.js +55 -21
- package/dist/index.js.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -816,7 +816,7 @@ sequence pinpoints _where_ a turn died:
|
|
|
816
816
|
| `segment:settle` carries `error: { name: "CoderChatError", message: "…requestTimeoutMs budget…" }` | the per‑segment bound expired — wedged server, slow model, or an unschedulable workspace | inspect the workspace via the v2 API/UI: a `pending`/`failed` build means the row above; `running` means the turn was genuinely slow — raise `requestTimeoutMs` for long tool work |
|
|
817
817
|
| repeated `ws:redial` with `consecutiveFailures` climbing toward `maxConsecutiveFailures` (5; backoff 1 s → 2 s → 4 s → 8 s, ≈15 s of redialing without forward progress), then a `CoderStreamError` | the network path to the deployment is failing — not workspace scheduling (the server keeps generating through short gaps) | fix connectivity; mind retry ownership above — on workspace‑backed chats the error is `isRetryable: false`, so the replay decision is yours |
|
|
818
818
|
| `segment:settle` with `status: "error"` and an `error` payload | the turn failed server‑side — a provider/model error, a tool failure, or a scheduling/build failure that terminated the turn instead of leaving it pending | the settle event's `error` carries only `{ name, message }` — branch on `kind` / `retryable` / `statusCode` by catching the thrown `CoderChatError`: at the `generate()` call site, or — for `stream()` — around stream consumption, since mid‑stream failures surface on the stream, not from `await agent.stream()` ([Handling errors](#handling-errors)); check the workspace build state to rule scheduling in or out |
|
|
819
|
-
| turn settled `status: "requires_action"`, follow‑up messages queue forever | the loop ended on an unanswered client tool call | submit the stranded results or interrupt — see rule 4 under [Structured output](#structured-output)
|
|
819
|
+
| turn settled `status: "requires_action"`, follow‑up messages queue forever | the loop ended on an unanswered client tool call | submit the stranded results or interrupt — see rule 4 under [Structured output](#structured-output); if a crash left the pause behind, reconcile effects first ([Make client tools crash-safe](#make-client-tools-crash-safe)) |
|
|
820
820
|
| `archive()` keeps returning 409 and rethrows after ~15 s | the chat never settled server‑side — usually a stuck run still holding its workspace | `interrupt()` with a bounded signal, then re‑archive; if the run stays wedged, stop the workspace itself |
|
|
821
821
|
|
|
822
822
|
## Configuration
|
|
@@ -1031,7 +1031,11 @@ try {
|
|
|
1031
1031
|
server interrupt is asynchronous and best‑effort, so the timed‑out run may
|
|
1032
1032
|
still be committing — run the same interrupt‑and‑wait‑for‑terminal‑status
|
|
1033
1033
|
recovery as after a crash (below) before the retry, or its trailing output
|
|
1034
|
-
is absorbed into the retried turn.
|
|
1034
|
+
is absorbed into the retried turn. A timed‑out run can also have _finished_
|
|
1035
|
+
before that interrupt landed — but because the interrupt was already fired,
|
|
1036
|
+
the recovery's [result check](#recover-the-result-before-resubmitting)
|
|
1037
|
+
must treat the attempt as cut short (pin `cutShort` there) rather than
|
|
1038
|
+
trusting its own 409.
|
|
1035
1039
|
- **`CoderStreamError`** (an AI SDK `APICallError`) — the stream could not be
|
|
1036
1040
|
re‑established. `isRetryable: true` only when the failed turn had just
|
|
1037
1041
|
created its chat _and_ had no external effects a replay would repeat (no
|
|
@@ -1096,27 +1100,365 @@ checkpoint:
|
|
|
1096
1100
|
resumed turn seeds its message cursor _before_ submitting — whatever the
|
|
1097
1101
|
old run commits after that point would be absorbed into the resumed turn's
|
|
1098
1102
|
output and usage. A crash _inside one of your client tools_ is worse: the
|
|
1099
|
-
chat is paused in `requires_action`, where new messages wait forever
|
|
1100
|
-
|
|
1101
|
-
|
|
1102
|
-
|
|
1103
|
+
chat is paused in `requires_action`, where new messages wait forever — and
|
|
1104
|
+
if the tool's external effect committed before the crash, that pause is the
|
|
1105
|
+
only pending record of it, so reconcile the tool ledger **before** any
|
|
1106
|
+
interrupt ([Make client tools crash-safe](#make-client-tools-crash-safe)).
|
|
1107
|
+
Both cases have one remedy — stop what the crash left behind and **wait
|
|
1108
|
+
until the chat actually settles** before deciding: a client‑tool pause is
|
|
1109
|
+
reconciled, never interrupted away; anything else is interrupted:
|
|
1103
1110
|
`interrupt()` resolves on acknowledgment while the run keeps winding down
|
|
1104
1111
|
(and committing) for a few more seconds, so poll the chat's status to a
|
|
1105
1112
|
terminal one first. On a chat with no live run the interrupt rejects with a
|
|
1106
1113
|
409 `CoderApiError`; that's the good case — ignore it:
|
|
1107
1114
|
|
|
1108
1115
|
```ts
|
|
1116
|
+
// Both shared with "Make client tools crash-safe" (below): the ledger
|
|
1117
|
+
// reconciliation resolves to its `unstarted` — true only when it fell back
|
|
1118
|
+
// to interrupting — and journals each action write-ahead, so a recovery
|
|
1119
|
+
// pass that dies mid-recovery is visible — and classifiable — to the next.
|
|
1120
|
+
declare function reconcileClientTools(): Promise<boolean>;
|
|
1121
|
+
// Entries are per-ATTEMPT: clear/supersede them transactionally with
|
|
1122
|
+
// recording the attempt's result, or a stale verdict misclassifies the
|
|
1123
|
+
// next attempt on this chat — attempt-scoped journal keys close the
|
|
1124
|
+
// remaining double-crash window (see
|
|
1125
|
+
// https://github.com/coder/ai-sdk/issues/92).
|
|
1126
|
+
declare const journal: {
|
|
1127
|
+
get(chatId: string): Promise<"resumed" | "cut-short" | undefined>;
|
|
1128
|
+
set(chatId: string, state: "resumed" | "cut-short"): Promise<void>;
|
|
1129
|
+
};
|
|
1130
|
+
|
|
1109
1131
|
// Bounded: a stalled deployment must fail the step, not hang it.
|
|
1110
1132
|
const deadline = AbortSignal.timeout(15_000);
|
|
1111
|
-
|
|
1112
|
-
|
|
1133
|
+
// Whether an interrupt stopped a live run drives the resubmit decision
|
|
1134
|
+
// below: a 409 means there was nothing to stop.
|
|
1135
|
+
let cutShort = true;
|
|
1136
|
+
let { status } = await agent.client.getChat(agent.chatId!, deadline);
|
|
1137
|
+
const journaled = await journal.get(agent.chatId!);
|
|
1138
|
+
if (status === "requires_action") {
|
|
1139
|
+
// The crash hit a client tool mid-execution: this pause is the only
|
|
1140
|
+
// pending record of its committed effects. Never interrupt it away —
|
|
1141
|
+
// reconcile the ledger instead (below): it submits recorded results
|
|
1142
|
+
// (reviving the run — nothing was cut short) or interrupts itself only
|
|
1143
|
+
// when nothing committed.
|
|
1144
|
+
cutShort = await reconcileClientTools();
|
|
1145
|
+
} else if (journaled) {
|
|
1146
|
+
// A previous recovery pass acted on this chat, then died before the
|
|
1147
|
+
// step's result was recorded. Its journaled state is the verdict:
|
|
1148
|
+
// "resumed" — it revived the run by submitting recorded results;
|
|
1149
|
+
// interrupting now would cut down a run doing the step's work, and the
|
|
1150
|
+
// resubmitted prompt would mint NEW tool-call ids, sidestepping the
|
|
1151
|
+
// idempotency keys. Let it finish; nothing was cut short.
|
|
1152
|
+
// "cut-short" — it went on to interrupt (an unstarted sibling in the
|
|
1153
|
+
// batch); the attempt is truncated, so the check below must resubmit.
|
|
1154
|
+
cutShort = journaled === "cut-short";
|
|
1155
|
+
if (cutShort) {
|
|
1156
|
+
// The journal records write-ahead INTENT, not a landed interrupt: the
|
|
1157
|
+
// pass may have died in between, leaving the revived run working.
|
|
1158
|
+
// Re-drive the interrupt to make the intent true — a 409 means the
|
|
1159
|
+
// chat already settled; the cut-short verdict stands either way.
|
|
1160
|
+
await agent.interrupt({ signal: deadline }).catch((err) => {
|
|
1161
|
+
if (!(err instanceof CoderApiError && err.status === 409)) throw err;
|
|
1162
|
+
});
|
|
1163
|
+
}
|
|
1164
|
+
} else {
|
|
1165
|
+
await agent.interrupt({ signal: deadline }).catch((err) => {
|
|
1166
|
+
// A 409 proves nothing is stoppable NOW — that means "never cut short"
|
|
1167
|
+
// only if the probe saw no interrupt already in flight. A probed
|
|
1168
|
+
// "interrupting" is an earlier interrupt (a timeout's best-effort one,
|
|
1169
|
+
// a sweeper) already truncating the attempt; this 409 just says it
|
|
1170
|
+
// landed. Keep cutShort = true then.
|
|
1171
|
+
if (err instanceof CoderApiError && err.status === 409) cutShort = status === "interrupting";
|
|
1172
|
+
else throw err;
|
|
1173
|
+
});
|
|
1174
|
+
}
|
|
1175
|
+
// The 15 s interrupt bound must not cap what follows: a reconciled pause
|
|
1176
|
+
// revives a full model/tool continuation. Budget the settle poll — and the
|
|
1177
|
+
// history reads after it — on the turn scale, like the step itself.
|
|
1178
|
+
const settle = AbortSignal.timeout(600_000);
|
|
1113
1179
|
do {
|
|
1114
|
-
({ status } = await agent.client.getChat(agent.chatId!,
|
|
1180
|
+
({ status } = await agent.client.getChat(agent.chatId!, settle));
|
|
1115
1181
|
if (status === "waiting" || status === "completed" || status === "error") break;
|
|
1182
|
+
// With no accepted interrupt racing to clear it, requires_action is a
|
|
1183
|
+
// STABLE pause: a run resumed by submitted tool results called a NEW
|
|
1184
|
+
// client tool. Exit and reconcile the ledger again (below) instead of
|
|
1185
|
+
// polling out the deadline.
|
|
1186
|
+
if (status === "requires_action" && !cutShort) break;
|
|
1116
1187
|
await new Promise((r) => setTimeout(r, 1_000)); // acknowledged ≠ settled — wait it out
|
|
1117
1188
|
} while (true);
|
|
1118
1189
|
```
|
|
1119
1190
|
|
|
1191
|
+
Settled is necessary, not sufficient: a terminal status doesn't say whether
|
|
1192
|
+
the crashed attempt already _finished the step's work_ — check history
|
|
1193
|
+
before resubmitting
|
|
1194
|
+
([Recover the result before resubmitting](#recover-the-result-before-resubmitting)).
|
|
1195
|
+
|
|
1196
|
+
### Make client tools crash-safe
|
|
1197
|
+
|
|
1198
|
+
Interrupting un‑strands a `requires_action` pause — but if the crashed
|
|
1199
|
+
process died _between_ a non‑idempotent client tool committing its external
|
|
1200
|
+
effect (a payment, a deploy, an email) and that result reaching the server,
|
|
1201
|
+
the pause is the **only pending record of the execution**. Interrupt it away
|
|
1202
|
+
and the resubmitted prompt is free to call the tool again — and duplicate the
|
|
1203
|
+
effect. Close the window with a tool‑invocation ledger in the same durable
|
|
1204
|
+
store as the `chatId` checkpoint, keyed by **chat id + `toolCallId`** — the
|
|
1205
|
+
server assigns the call id and commits it to history, so both sides of the
|
|
1206
|
+
reconciliation survive the crash. The chat id must be part of the key (and
|
|
1207
|
+
of the idempotency key handed to the external system): call ids are only
|
|
1208
|
+
unique _within_ a chat, and a store shared across workflows must never let
|
|
1209
|
+
one chat's `done` entry answer — or deduplicate — another chat's call. In
|
|
1210
|
+
multi‑tenant stores, fold the tenant in too:
|
|
1211
|
+
|
|
1212
|
+
```ts
|
|
1213
|
+
import { tool } from "ai";
|
|
1214
|
+
import { z } from "zod";
|
|
1215
|
+
import { CoderAgent } from "@coder/ai-sdk-agent";
|
|
1216
|
+
|
|
1217
|
+
type LedgerEntry = { state: "committing" } | { state: "done"; output: string };
|
|
1218
|
+
declare const ledger: {
|
|
1219
|
+
get(key: string): Promise<LedgerEntry | undefined>;
|
|
1220
|
+
set(key: string, entry: LedgerEntry): Promise<void>;
|
|
1221
|
+
};
|
|
1222
|
+
declare const payments: {
|
|
1223
|
+
charge(amountCents: number, opts: { idempotencyKey: string }): Promise<{ receiptId: string }>;
|
|
1224
|
+
};
|
|
1225
|
+
|
|
1226
|
+
const ledgerKey = (chatId: string, toolCallId: string) => `${chatId}/${toolCallId}`;
|
|
1227
|
+
const ChargeArgs = z.object({ amountCents: z.number().int() });
|
|
1228
|
+
|
|
1229
|
+
// A factory taking a getter, not an id: the chat may not exist until the
|
|
1230
|
+
// turn creates it, so the id must be read lazily — it is set before any
|
|
1231
|
+
// tool runs.
|
|
1232
|
+
const chargeCard = (chatId: () => string) =>
|
|
1233
|
+
tool({
|
|
1234
|
+
description: "Charge the customer's card.",
|
|
1235
|
+
inputSchema: ChargeArgs, // shared with recovery's re-drive dispatch below
|
|
1236
|
+
execute: async ({ amountCents }, { toolCallId }) => {
|
|
1237
|
+
const key = ledgerKey(chatId(), toolCallId);
|
|
1238
|
+
// Write-ahead, BEFORE the effect: a crash between the two writes leaves
|
|
1239
|
+
// "committing" (ambiguous, but resolvable below); no entry at all proves
|
|
1240
|
+
// the effect never started.
|
|
1241
|
+
await ledger.set(key, { state: "committing" });
|
|
1242
|
+
// Hand the external system the SAME key as its idempotency key, so
|
|
1243
|
+
// re-driving this exact call can never double-charge.
|
|
1244
|
+
const { receiptId } = await payments.charge(amountCents, { idempotencyKey: key });
|
|
1245
|
+
await ledger.set(key, { state: "done", output: `charged: receipt ${receiptId}` });
|
|
1246
|
+
return `charged: receipt ${receiptId}`;
|
|
1247
|
+
},
|
|
1248
|
+
});
|
|
1249
|
+
|
|
1250
|
+
// Wire-up: a direct self-reference in the constructor call is circular for
|
|
1251
|
+
// the type checker — defer it through a box assigned right after.
|
|
1252
|
+
let self: { chatId?: string } | undefined;
|
|
1253
|
+
const agent = new CoderAgent({
|
|
1254
|
+
/* …base options as in runTurn… */
|
|
1255
|
+
tools: { charge_card: chargeCard(() => self!.chatId!) },
|
|
1256
|
+
});
|
|
1257
|
+
self = agent;
|
|
1258
|
+
```
|
|
1259
|
+
|
|
1260
|
+
On crash recovery, when the checkpointed chat is paused in `requires_action`,
|
|
1261
|
+
reconcile _before_ any interrupt. The pending calls are in committed history —
|
|
1262
|
+
the same derivation the SDK itself uses to recover a lost `action_required`
|
|
1263
|
+
event — and the ledger holds the verdict on each:
|
|
1264
|
+
|
|
1265
|
+
```ts
|
|
1266
|
+
// The step's configured ToolSet names: history does not record which calls
|
|
1267
|
+
// were client tools (see [Rehydrating chat history]), so — like the SDK's
|
|
1268
|
+
// own derivation — restrict to the names this step registers; a server/MCP
|
|
1269
|
+
// call must never reach the ledger as "unstarted".
|
|
1270
|
+
const clientTools = new Set(["charge_card"]);
|
|
1271
|
+
|
|
1272
|
+
// Pending = the last assistant message's client (non-provider-executed,
|
|
1273
|
+
// ToolSet-named) tool calls, minus ids answered by a later message's tool
|
|
1274
|
+
// result.
|
|
1275
|
+
const { messages } = await agent.client.getMessages(agent.chatId!, { limit: 200 }, deadline);
|
|
1276
|
+
const ordered = [...messages].sort((a, b) => a.id - b.id);
|
|
1277
|
+
const lastAssistant = ordered.findLast((m) => m.role === "assistant");
|
|
1278
|
+
const handled = new Set(
|
|
1279
|
+
ordered
|
|
1280
|
+
.filter((m) => m.id > (lastAssistant?.id ?? 0))
|
|
1281
|
+
.flatMap((m) => m.content ?? [])
|
|
1282
|
+
.filter((p) => p.type === "tool-result")
|
|
1283
|
+
.map((p) => p.tool_call_id),
|
|
1284
|
+
);
|
|
1285
|
+
const pending = (lastAssistant?.content ?? []).filter(
|
|
1286
|
+
(p) =>
|
|
1287
|
+
p.type === "tool-call" &&
|
|
1288
|
+
!p.provider_executed &&
|
|
1289
|
+
clientTools.has(p.tool_name!) &&
|
|
1290
|
+
!handled.has(p.tool_call_id),
|
|
1291
|
+
);
|
|
1292
|
+
|
|
1293
|
+
const results: { tool_call_id: string; output: unknown }[] = [];
|
|
1294
|
+
let unstarted = false;
|
|
1295
|
+
for (const call of pending) {
|
|
1296
|
+
const key = ledgerKey(agent.chatId!, call.tool_call_id!);
|
|
1297
|
+
const entry = await ledger.get(key);
|
|
1298
|
+
if (entry?.state === "done") {
|
|
1299
|
+
// Effect committed, result stranded: submit the record — the pause is
|
|
1300
|
+
// answered and nothing re-executes.
|
|
1301
|
+
results.push({ tool_call_id: call.tool_call_id!, output: entry.output });
|
|
1302
|
+
} else if (entry) {
|
|
1303
|
+
// "committing" is ambiguous — the crash hit between the write-ahead and
|
|
1304
|
+
// "done". Safe to re-drive: the SAME idempotency key makes a committed
|
|
1305
|
+
// effect replay as a no-op, and the call's args are committed in history.
|
|
1306
|
+
// Dispatch by tool name — every ledgered tool needs its own re-drive
|
|
1307
|
+
// path, its args validated exactly as the live loop would have.
|
|
1308
|
+
if (call.tool_name !== "charge_card")
|
|
1309
|
+
throw new Error(`no re-drive path for tool ${call.tool_name}`);
|
|
1310
|
+
const { amountCents } = ChargeArgs.parse(call.args);
|
|
1311
|
+
const { receiptId } = await payments.charge(amountCents, { idempotencyKey: key });
|
|
1312
|
+
const output = `charged: receipt ${receiptId}`;
|
|
1313
|
+
await ledger.set(key, { state: "done", output });
|
|
1314
|
+
results.push({ tool_call_id: call.tool_call_id!, output });
|
|
1315
|
+
} else {
|
|
1316
|
+
unstarted = true; // the tool never ran — interrupting loses nothing
|
|
1317
|
+
}
|
|
1318
|
+
}
|
|
1319
|
+
if (results.length > 0 || unstarted) {
|
|
1320
|
+
// ONE write-ahead with the batch's FINAL verdict, before any network
|
|
1321
|
+
// call — a mixed batch journals "cut-short" before even its submission,
|
|
1322
|
+
// so no crash point can leave a stale "resumed" masking the pending
|
|
1323
|
+
// interrupt. Every crash then either re-reconciles the stable pause or
|
|
1324
|
+
// re-drives the journaled intent. Clear the entry only once the step's
|
|
1325
|
+
// result is finally recorded.
|
|
1326
|
+
await journal.set(agent.chatId!, unstarted ? "cut-short" : "resumed");
|
|
1327
|
+
}
|
|
1328
|
+
if (results.length > 0) await agent.client.submitToolResults(agent.chatId!, { results }, deadline);
|
|
1329
|
+
if (unstarted) await agent.client.interruptChat(agent.chatId!, deadline);
|
|
1330
|
+
// As `reconcileClientTools()` in the settle-wait snippet above: resolve to
|
|
1331
|
+
// `unstarted` — cutShort is true only when this fell back to interrupting.
|
|
1332
|
+
```
|
|
1333
|
+
|
|
1334
|
+
- **Only interrupt when the ledger shows no committed effect.** Submitted
|
|
1335
|
+
results are committed to chat history, so even when a mixed batch (one call
|
|
1336
|
+
done, a sibling never started) still ends in an interrupt, the effect's
|
|
1337
|
+
recorded outcome survives for the resubmitted turn to see.
|
|
1338
|
+
- **Submitting answers the pause; it does not revive your tool loop.** Once
|
|
1339
|
+
every pending call is answered, the turn resumes **server‑side** with no
|
|
1340
|
+
process streaming it. That is why the settle‑wait's reconcile branch never
|
|
1341
|
+
interrupts a pause — nothing stranded is left to stop, and an interrupt
|
|
1342
|
+
now would cut down the very run the submission revived — and why the
|
|
1343
|
+
resolved `unstarted` leaves **`cutShort = false`** after a submission:
|
|
1344
|
+
nothing was cut short, and the next section's `!cutShort` check must
|
|
1345
|
+
classify the resumed run's finish as a completed attempt, not resubmit
|
|
1346
|
+
it. Give the poll a turn‑scale deadline rather than the 15 s interrupt
|
|
1347
|
+
bound; it exits on `requires_action` (stable here, with no accepted
|
|
1348
|
+
interrupt clearing it), which means the resumed run called a _new_ tool —
|
|
1349
|
+
reconcile again; a call with no ledger entry is safe to interrupt.
|
|
1350
|
+
- **The finished run's output lands only in history** — recover it like any
|
|
1351
|
+
completed‑but‑unrecorded attempt (next section) instead of resubmitting.
|
|
1352
|
+
|
|
1353
|
+
### Recover the result before resubmitting
|
|
1354
|
+
|
|
1355
|
+
The settle‑wait resolves overlap with a still‑live run, but not the last
|
|
1356
|
+
crash window: **the server finished the turn, and the process died before the
|
|
1357
|
+
step's return value was recorded.** History now holds a perfectly good
|
|
1358
|
+
result; a retried step that blindly resubmits runs the whole turn again —
|
|
1359
|
+
duplicated output, repeated workspace/MCP/tool effects. This is the
|
|
1360
|
+
at‑least‑once boundary every durable engine has around external effects, and
|
|
1361
|
+
the chat itself is the record that closes it. Make attempts matchable first —
|
|
1362
|
+
the server stores the prompt verbatim, so embed a step‑scoped marker:
|
|
1363
|
+
|
|
1364
|
+
```ts
|
|
1365
|
+
const marker = `[${workflowId}/step-2]`; // stable across this step's retries
|
|
1366
|
+
const { text } = await agent.generate({ prompt: `${marker} ${prompt}` });
|
|
1367
|
+
```
|
|
1368
|
+
|
|
1369
|
+
Then, after the settle‑wait, three states are distinguishable — and only one
|
|
1370
|
+
of them resubmits:
|
|
1371
|
+
|
|
1372
|
+
```ts
|
|
1373
|
+
import { chatMessagesToUIMessages } from "@coder/ai-sdk-agent";
|
|
1374
|
+
|
|
1375
|
+
// A long turn can commit more than a page of messages after the prompt —
|
|
1376
|
+
// page newest-first (the endpoint's default order) until the turn's user
|
|
1377
|
+
// message is on hand; "never submitted" must never be concluded from a
|
|
1378
|
+
// truncated read, or this recovery resubmits exactly the turn it was meant
|
|
1379
|
+
// to deduplicate.
|
|
1380
|
+
const fetched = [];
|
|
1381
|
+
let page = await agent.client.getMessages(agent.chatId!, { limit: 200 }, settle);
|
|
1382
|
+
fetched.push(...page.messages);
|
|
1383
|
+
while (!fetched.some((m) => m.role === "user") && page.has_more) {
|
|
1384
|
+
const oldestId = Math.min(...fetched.map((m) => m.id));
|
|
1385
|
+
page = await agent.client.getMessages(agent.chatId!, { before_id: oldestId, limit: 200 }, settle);
|
|
1386
|
+
fetched.push(...page.messages);
|
|
1387
|
+
}
|
|
1388
|
+
const transcript = chatMessagesToUIMessages(fetched); // chronological
|
|
1389
|
+
const lastUser = transcript.findLast((m) => m.role === "user");
|
|
1390
|
+
const submitted = lastUser?.parts.some((p) => p.type === "text" && p.text.includes(marker));
|
|
1391
|
+
|
|
1392
|
+
if (submitted && !cutShort && (status === "waiting" || status === "completed")) {
|
|
1393
|
+
// The attempt finished; only the recording was lost. Recover what
|
|
1394
|
+
// `generate().text` would have returned — the FINAL step's text. A turn
|
|
1395
|
+
// that crossed tool calls commits narration between them too, so cut the
|
|
1396
|
+
// turn's parts after the last tool part and join the text that follows.
|
|
1397
|
+
const parts = transcript
|
|
1398
|
+
.slice(transcript.indexOf(lastUser!) + 1)
|
|
1399
|
+
.filter((m) => m.role === "assistant")
|
|
1400
|
+
.flatMap((m) => m.parts);
|
|
1401
|
+
return parts
|
|
1402
|
+
.slice(parts.findLastIndex((p) => p.type === "dynamic-tool") + 1)
|
|
1403
|
+
.filter((p) => p.type === "text")
|
|
1404
|
+
.map((p) => p.text)
|
|
1405
|
+
.join("");
|
|
1406
|
+
}
|
|
1407
|
+
// Resubmit only what never finished: the prompt never reached the server
|
|
1408
|
+
// (!submitted), the turn failed (status "error" — normal retry judgment
|
|
1409
|
+
// applies), or the interrupt cut a live run short (cutShort — the model sees
|
|
1410
|
+
// its own aborted attempt in history and continues from it). One exit is
|
|
1411
|
+
// neither: a requires_action settle goes back to ledger reconciliation
|
|
1412
|
+
// (previous section) — never to recovery or resubmission.
|
|
1413
|
+
```
|
|
1414
|
+
|
|
1415
|
+
- **`cutShort` is the completed/interrupted discriminator — but a 409 only
|
|
1416
|
+
proves nothing is running _now_.** An interrupted attempt leaves the same
|
|
1417
|
+
marker and a truncated answer in history, so history alone can't tell
|
|
1418
|
+
"finished" from "stopped"; what can is whether any interrupt stopped the
|
|
1419
|
+
run. The inference "409 ⇒ the attempt finished" is therefore sound only
|
|
1420
|
+
when recovery's interrupt is the _first_ one that could have: true after a
|
|
1421
|
+
hard crash, false after a `requestTimeoutMs` expiry — the expiry already
|
|
1422
|
+
fired the SDK's own best‑effort interrupt, which may have stopped the run
|
|
1423
|
+
before recovery ever looked. Carry the reason recovery ran: when it follows
|
|
1424
|
+
a known timeout or abort — or a journaled earlier interrupt (write‑ahead,
|
|
1425
|
+
like the tool ledger, if the recovery pass can itself crash between its
|
|
1426
|
+
interrupt and the resubmit) — pin `cutShort` to `true` and let only the
|
|
1427
|
+
hard‑crash path trust the 409.
|
|
1428
|
+
- **The cut lands after the last tool part of _either_ kind.** History does
|
|
1429
|
+
not mark which tool calls were client tools
|
|
1430
|
+
([Rehydrating chat history](#rehydrating-chat-history)), so when a final
|
|
1431
|
+
step narrates _before_ a server‑side tool call, that preamble is dropped
|
|
1432
|
+
too — the recovered text is the answer following the turn's last tool
|
|
1433
|
+
activity.
|
|
1434
|
+
- **Steps with structured results recover the call's input, not the text.**
|
|
1435
|
+
A [structured output](#structured-output) step's answer is the
|
|
1436
|
+
`structured_output` call's typed input, which rehydrates as a
|
|
1437
|
+
`dynamic-tool` part on the recovered turn
|
|
1438
|
+
([Rehydrating chat history](#rehydrating-chat-history)) — the text join
|
|
1439
|
+
above would slice it away and return only the ack prose that follows.
|
|
1440
|
+
Recover the filed call instead — this scan replaces the text join
|
|
1441
|
+
_inside_ the recovery branch above, reusing its `parts` — validated
|
|
1442
|
+
client‑side exactly like the live path (rule 2 there — the schema is the
|
|
1443
|
+
real gate):
|
|
1444
|
+
|
|
1445
|
+
```ts
|
|
1446
|
+
// Scan backward to the last call that VALIDATES, as in the live path — a
|
|
1447
|
+
// schema-invalid re-file must not shadow a valid answer.
|
|
1448
|
+
const filed = parts
|
|
1449
|
+
.filter((p) => p.type === "dynamic-tool")
|
|
1450
|
+
.filter((p) => p.toolName === "structured_output");
|
|
1451
|
+
let answer: z.infer<typeof Answer> | undefined;
|
|
1452
|
+
for (const call of filed.reverse()) {
|
|
1453
|
+
const parsed = Answer.safeParse(call.input);
|
|
1454
|
+
if (parsed.success) {
|
|
1455
|
+
answer = parsed.data;
|
|
1456
|
+
break;
|
|
1457
|
+
}
|
|
1458
|
+
}
|
|
1459
|
+
// recovered ⇔ answer !== undefined
|
|
1460
|
+
```
|
|
1461
|
+
|
|
1120
1462
|
### Watch turn health from inside the step
|
|
1121
1463
|
|
|
1122
1464
|
[Transport events](#observability) classify how each segment ended without
|
|
@@ -1223,6 +1565,8 @@ export async function archiveWorkflowChat(workflowId: string): Promise<void> {
|
|
|
1223
1565
|
total wall‑clock.
|
|
1224
1566
|
- Let redials self‑heal; own every retry decision — a re‑run step resubmits
|
|
1225
1567
|
its prompt as a new user turn.
|
|
1568
|
+
- Before resubmitting after a crash: reconcile the tool ledger before any
|
|
1569
|
+
interrupt, and recover a finished attempt's result from history.
|
|
1226
1570
|
- Keep concurrent steps' fan‑out within workspace quota —
|
|
1227
1571
|
[Workspaces & quota](#workspaces--quota).
|
|
1228
1572
|
- Steps that don't need server‑side tools (plan / extract / synthesize) are
|
package/dist/index.js
CHANGED
|
@@ -1328,10 +1328,12 @@ var TurnTranslator = class {
|
|
|
1328
1328
|
// false only for announce-style empty snapshots, whose in-flight deltas are
|
|
1329
1329
|
// still claimable by a same-id commit.
|
|
1330
1330
|
#emittedByMessageId = /* @__PURE__ */ new Map();
|
|
1331
|
-
// Same-id revision snapshots
|
|
1332
|
-
// open (latest wins). chatd sends each revision
|
|
1333
|
-
// connection, so the suffix cannot wait for a replay — it
|
|
1334
|
-
// soon as the next step's snapshot settles ownership of
|
|
1331
|
+
// Same-id revision snapshots (their full wire content) that arrived while
|
|
1332
|
+
// the NEXT step's deltas were open (latest wins). chatd sends each revision
|
|
1333
|
+
// once on a healthy connection, so the suffix cannot wait for a replay — it
|
|
1334
|
+
// is reconciled as soon as the next step's snapshot settles ownership of
|
|
1335
|
+
// the pending deltas, or flushed by finish() when the segment ends without
|
|
1336
|
+
// one (see #78).
|
|
1335
1337
|
#deferredRevisions = /* @__PURE__ */ new Map();
|
|
1336
1338
|
#error;
|
|
1337
1339
|
#terminalStatus;
|
|
@@ -1406,22 +1408,34 @@ var TurnTranslator = class {
|
|
|
1406
1408
|
* next delta simply opens a fresh block (its pending content is tracked
|
|
1407
1409
|
* independently of blocks).
|
|
1408
1410
|
*/
|
|
1409
|
-
#reconcileRevision(out, rec,
|
|
1410
|
-
const
|
|
1411
|
-
const
|
|
1412
|
-
|
|
1411
|
+
#reconcileRevision(out, rec, content) {
|
|
1412
|
+
const fullText = joinContent(content, "text");
|
|
1413
|
+
const fullReasoning = joinContent(content, "reasoning");
|
|
1414
|
+
const textOk = fullText.length > rec.text.length && fullText.startsWith(rec.text);
|
|
1415
|
+
const reasoningOk = fullReasoning.length > rec.reasoning.length && fullReasoning.startsWith(rec.reasoning);
|
|
1416
|
+
if (!textOk && !reasoningOk) return;
|
|
1413
1417
|
this.#closeText(out);
|
|
1414
1418
|
this.#closeReasoning(out);
|
|
1415
|
-
|
|
1416
|
-
|
|
1417
|
-
|
|
1418
|
-
|
|
1419
|
-
|
|
1420
|
-
|
|
1421
|
-
|
|
1422
|
-
|
|
1423
|
-
|
|
1419
|
+
let seenText = 0;
|
|
1420
|
+
let seenReasoning = 0;
|
|
1421
|
+
for (const p of content) {
|
|
1422
|
+
if (p.type === "text") {
|
|
1423
|
+
const end = seenText + (p.text ?? "").length;
|
|
1424
|
+
const from = Math.max(seenText, rec.text.length);
|
|
1425
|
+
if (textOk && end > from) this.#emitTextDelta(out, fullText.slice(from, end));
|
|
1426
|
+
seenText = end;
|
|
1427
|
+
} else if (p.type === "reasoning") {
|
|
1428
|
+
const end = seenReasoning + (p.text ?? "").length;
|
|
1429
|
+
const from = Math.max(seenReasoning, rec.reasoning.length);
|
|
1430
|
+
if (reasoningOk && end > from)
|
|
1431
|
+
this.#emitReasoningDelta(out, fullReasoning.slice(from, end));
|
|
1432
|
+
seenReasoning = end;
|
|
1433
|
+
}
|
|
1424
1434
|
}
|
|
1435
|
+
if (textOk) rec.text = fullText;
|
|
1436
|
+
if (reasoningOk) rec.reasoning = fullReasoning;
|
|
1437
|
+
this.#closeText(out);
|
|
1438
|
+
this.#closeReasoning(out);
|
|
1425
1439
|
}
|
|
1426
1440
|
// --- tool helpers ---------------------------------------------------------
|
|
1427
1441
|
#isClientTool(name) {
|
|
@@ -1504,6 +1518,7 @@ var TurnTranslator = class {
|
|
|
1504
1518
|
for (const tc of ev.action_required?.tool_calls ?? []) {
|
|
1505
1519
|
if (this.#clientToolCalls.has(tc.tool_call_id)) continue;
|
|
1506
1520
|
if (this.#submittedToolCallIds.has(tc.tool_call_id)) continue;
|
|
1521
|
+
this.#flushDeferredRevisions(out);
|
|
1507
1522
|
this.#closeText(out);
|
|
1508
1523
|
this.#closeReasoning(out);
|
|
1509
1524
|
this.#clientToolCalls.add(tc.tool_call_id);
|
|
@@ -1521,6 +1536,7 @@ var TurnTranslator = class {
|
|
|
1521
1536
|
case "error":
|
|
1522
1537
|
if (ev.error) {
|
|
1523
1538
|
this.#error = ev.error;
|
|
1539
|
+
this.#flushDeferredRevisions(out);
|
|
1524
1540
|
out.push({ type: "error", error: new CoderChatError(ev.error) });
|
|
1525
1541
|
}
|
|
1526
1542
|
break;
|
|
@@ -1586,9 +1602,9 @@ var TurnTranslator = class {
|
|
|
1586
1602
|
const fullReasoning = joinContent(content, "reasoning");
|
|
1587
1603
|
const known = this.#emittedByMessageId.get(message.id);
|
|
1588
1604
|
if (known && message.id !== this.#currentAssistantId) {
|
|
1589
|
-
this.#reconcileRevision(out, known,
|
|
1605
|
+
this.#reconcileRevision(out, known, content);
|
|
1590
1606
|
} else if (known && this.#deltasSinceSnapshot && known.substantive) {
|
|
1591
|
-
this.#deferredRevisions.set(message.id,
|
|
1607
|
+
this.#deferredRevisions.set(message.id, content);
|
|
1592
1608
|
} else {
|
|
1593
1609
|
if (this.#currentAssistantId !== void 0 && this.#currentAssistantId !== message.id && !this.#deltasSinceSnapshot) {
|
|
1594
1610
|
this.#closeText(out);
|
|
@@ -1628,8 +1644,7 @@ var TurnTranslator = class {
|
|
|
1628
1644
|
for (const [id, deferred] of this.#deferredRevisions) {
|
|
1629
1645
|
if (id === message.id) continue;
|
|
1630
1646
|
const deferredRec = this.#emittedByMessageId.get(id);
|
|
1631
|
-
if (deferredRec)
|
|
1632
|
-
this.#reconcileRevision(out, deferredRec, deferred.text, deferred.reasoning);
|
|
1647
|
+
if (deferredRec) this.#reconcileRevision(out, deferredRec, deferred);
|
|
1633
1648
|
this.#deferredRevisions.delete(id);
|
|
1634
1649
|
}
|
|
1635
1650
|
}
|
|
@@ -1649,8 +1664,27 @@ var TurnTranslator = class {
|
|
|
1649
1664
|
}
|
|
1650
1665
|
}
|
|
1651
1666
|
// --- finish ---------------------------------------------------------------
|
|
1667
|
+
/**
|
|
1668
|
+
* Flushes revisions still deferred behind a delta-ownership race that never
|
|
1669
|
+
* settled — the racing step ended terminally without committing a snapshot,
|
|
1670
|
+
* so the drain in #ingestMessage never ran. chatd sent each revision once;
|
|
1671
|
+
* this is the segment's last chance to emit its suffix (#78). Attribution
|
|
1672
|
+
* care: the pending deltas' owner never committed, so they attribute to no
|
|
1673
|
+
* one — reconciliation is against the ledger alone, and only
|
|
1674
|
+
* ledger-extension suffixes may emit. Runs at finish() and, because
|
|
1675
|
+
* doGenerate() throws on a streamed error part and closes the generator
|
|
1676
|
+
* before finish() (issue #72), ahead of the error part in ingest().
|
|
1677
|
+
*/
|
|
1678
|
+
#flushDeferredRevisions(out) {
|
|
1679
|
+
for (const [id, deferred] of this.#deferredRevisions) {
|
|
1680
|
+
const rec = this.#emittedByMessageId.get(id);
|
|
1681
|
+
if (rec) this.#reconcileRevision(out, rec, deferred);
|
|
1682
|
+
}
|
|
1683
|
+
this.#deferredRevisions.clear();
|
|
1684
|
+
}
|
|
1652
1685
|
finish() {
|
|
1653
1686
|
const out = [];
|
|
1687
|
+
this.#flushDeferredRevisions(out);
|
|
1654
1688
|
this.#closeText(out);
|
|
1655
1689
|
this.#closeReasoning(out);
|
|
1656
1690
|
let unified;
|