@ctrl-spc/cs 0.6.0 → 0.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,2516 @@
1
+ /**
2
+ * ═══ AGENT PANEL v3: the poll loop that answers a card. ═══
3
+ *
4
+ * THIS FILE BELONGS TO AGENT PANEL v3. Nothing outside `src/panel3/` may import
5
+ * it, WITH ONE NAMED EXCEPTION: `cli-v2/src/daemon.ts` imports `startPanel` from
6
+ * here and nothing else, which is how `cs start` runs the panel. It is written
7
+ * down in `conventions.md` and checked by `panel3-isolation.contract.test.mjs`;
8
+ * `startPanel`'s own comment at the foot of this file carries the reasoning.
9
+ *
10
+ * ---------------------------------------------------------------------------
11
+ * WHAT THIS IS. `cs3 say` puts a card and a user turn on the record and returns,
12
+ * because ux.md requires the acknowledgement never to wait for a spawn. This is
13
+ * the other half: the loop that notices the turn, leases it, writes a run row,
14
+ * starts a real agent, and puts what that agent said back on the card.
15
+ *
16
+ * ---------------------------------------------------------------------------
17
+ * ═══ THE DAEMON DOES NOT DECIDE WHAT IS TAKEABLE. THE DATABASE DOES. ═══
18
+ *
19
+ * Everything about exclusion lives in `panel3_take_turns()`: the lease, the
20
+ * "no live level 1 on this card" rule, and the skip-locked that keeps two
21
+ * daemons polling the same millisecond from both getting one card. There is no
22
+ * read-then-decide-then-write anywhere in this file, because that shape has a
23
+ * window between the read and the write and the window is exactly what a second
24
+ * daemon finds. This file honours what it was handed and nothing else.
25
+ *
26
+ * ---------------------------------------------------------------------------
27
+ * ═══ THE RUN ROW IS WRITTEN BEFORE THE SPAWN, OR THERE IS NO SPAWN. ═══
28
+ *
29
+ * plan.md's global constraint 8. A process with no row is a process nothing can
30
+ * find after a restart: it cannot be shown, cannot be recovered and cannot be
31
+ * ended, so the card would say working forever with an agent nobody can name.
32
+ *
33
+ * IT IS THE TAKE THAT WRITES IT, in the same statement that leases the turns,
34
+ * which is why this file has no ordering to be careful about: it cannot reach a
35
+ * spawn without a row, because it cannot reach a spawn without a take. The
36
+ * earlier version wrote the row here, two round trips after the lease, and both
37
+ * ends of that gap were holes — a daemon killed inside it stranded the card
38
+ * forever, and a turn arriving inside it could put a second agent on the same
39
+ * card. See 20260821140000.
40
+ *
41
+ * ---------------------------------------------------------------------------
42
+ * ═══ THE DAEMON WRITES THE AGENT TURN FROM WHAT THE PROCESS SAID. ═══
43
+ *
44
+ * The harness is one-shot: a prompt goes in, text comes out, it exits. That
45
+ * final text IS the answer, so this writes it as the card's `role='agent'` turn.
46
+ *
47
+ * ═══ AND IT IS THE SAME PATH AT EVERY LEVEL, WHICH IS ux.md's RULE, NOT A
48
+ * CONVENIENCE. ═══ "Whoever did the work writes the answer. The level 2
49
+ * agent writes it, straight into the card. It does not hand a summary up to
50
+ * level 1 to be re-written." A dispatched agent's text goes onto the card
51
+ * through the same `panel3_answer` its dispatcher's did, so there is nowhere in
52
+ * this file for one agent's words to pass through another's. The coordinator has
53
+ * usually ended long before.
54
+ *
55
+ * ═══ AND A WORKER'S WORDS DO NOT REACH THE CARD AT ALL, WHICH IS THE SAME RULE
56
+ * ONE LEVEL DOWN. ═══ "Each worker writes what it found into the record, and
57
+ * the level 2 agent reads them back." Three workers writing their own turns
58
+ * would be three answers to one request, before the one that covers them all
59
+ * arrives. `panel3_answer` puts a level 3 run's last words in its report instead,
60
+ * from the level on the row — decided there rather than here, because two owners
61
+ * for "does this reach the person" is one owner too many.
62
+ *
63
+ * ---------------------------------------------------------------------------
64
+ * ═══ A QUESTION MOVING, AND EVERYBODY YOU SENT FINISHING, ARE BOTH TAKEABLE
65
+ * WORK, AND THEY COME THROUGH THE SAME POLL. ═══
66
+ *
67
+ * ux.md: "Nothing new is built for this. The daemon already leases takeable work
68
+ * and spawns. What changes is the definition of takeable." So there is a second
69
+ * take beside `panel3_take_turns` and nothing else in this file changes shape:
70
+ * `panel3_take_rearms` hands back runs to start again, for either of the two
71
+ * reasons ux.md's table gives, and they are started, held and settled by the
72
+ * functions that already do those things.
73
+ *
74
+ * THE SECOND REASON IS THE ONE NOTHING ELSE COULD FIRE. An agent that split its
75
+ * work dispatched and ended, because it is forbidden to wait; by the time its
76
+ * workers finish there is no process anywhere holding what they found. Starting
77
+ * it again is what makes "it reads them back and writes the answer" a mechanism
78
+ * rather than a sentence.
79
+ *
80
+ * IT IS THE SAME MECHANISM AS RECOVERY, POINTED AT A DIFFERENT FACT. A run whose
81
+ * process died is started again as itself because the machine failed; a run that
82
+ * asked, or one whose workers are done, is started again as itself because the
83
+ * thing it was waiting on exists now. Same row, same id, same brief, its report
84
+ * carried — and the only difference is one paragraph at the top of the prompt.
85
+ *
86
+ * ---------------------------------------------------------------------------
87
+ * ═══ A DISPATCH IS A SPAWN THE DAEMON DOES ON BEHALF OF AN AGENT. ═══
88
+ *
89
+ * The tools server runs inside this process and `dispatch` is served from it,
90
+ * but the spawning is here: writing the row before the process, recording the
91
+ * pid, waiting for the answer, giving up when the process is gone, and being
92
+ * held open long enough for `--once` to be honest are all things this file
93
+ * already does once, and a second copy of them behind a tool would be a second
94
+ * owner of a child's whole life.
95
+ *
96
+ * WHAT IT DOES NOT DO IS WAIT. The tool call returns as soon as the process
97
+ * exists, and the child's answer settles here, long after the agent that asked
98
+ * for it has ended. That is ux.md's "level 1 never blocks waiting for it", and it
99
+ * is why a dispatched run's promise goes into `inFlight` rather than being
100
+ * awaited by its caller.
101
+ *
102
+ * ---------------------------------------------------------------------------
103
+ * ═══ A CARD MUST NOT SIT LIVE FOREVER. ═══
104
+ *
105
+ * If this daemon is killed mid-run its process dies with it, and the run row is
106
+ * left saying `running` with an end that never comes. `recoverStranded` is what
107
+ * closes that: on every poll it looks at the runs THIS machine owns and asks the
108
+ * operating system whether their processes are still there. A level 1 run's
109
+ * turns are handed back, so they are takeable again; a dispatched run holds no
110
+ * turns and is STARTED AGAIN AS ITSELF, with the brief it was sent and the
111
+ * report it wrote. Either way the user's message is answered late rather than
112
+ * lost.
113
+ *
114
+ * It is careful in three directions, and each is a way the sweep can lie: it
115
+ * never reaps a run this process holds, never reaps a run too young to have
116
+ * recorded a pid (which is another daemon's live work), and never believes a pid
117
+ * that predates this machine's boot (which is somebody else's process wearing a
118
+ * reused number).
119
+ *
120
+ * ═══ AND IT CAN ONLY EVER REACH THIS MACHINE'S RUNS, WHICH IS WHY A PERSON HAS
121
+ * THE OTHER MOVE. ═══
122
+ *
123
+ * A machine that is never coming back sweeps nothing, so its dispatched runs sit
124
+ * `running` forever. `panel3_hand_back` is the person's answer, pressed from the
125
+ * panel on a card nothing is listening for, and `takeHandedBack` is this
126
+ * daemon's half of it: the one read that looks past this machine's own id, so
127
+ * work another machine was holding is carried on from its brief and its report
128
+ * instead of being started over. The sweep is not loosened to do it, and which
129
+ * of the three cares above apply to that read is argued where it is written.
130
+ */
131
+ // The runtime import comes FIRST, deliberately: `tsc` elides a type-only import
132
+ // and takes the leading comment with it, so a file whose first statement is
133
+ // `import type` loses its v3 header in the published `dist/`.
134
+ import { out, returned, signedInClient } from './client.js';
135
+ import { answerPrompt, escalationPrompt, levelOnePrompt, ownerActivationPrompt, readBackPrompt, presentedArtifactAnswerContext, resumePrompt, retryPrompt, } from './prompt.js';
136
+ import { ASK_CONTENT_COLUMNS, attachmentLine, loadAttachments, loadOutputNames, outputOf, withAskContent, } from './show.js';
137
+ import { forgetSecrets, redactSecrets } from './secrets.js';
138
+ import { sayListening, stopListening } from './presence.js';
139
+ import { checkoutForCodebase, hasCheckoutForCodebase } from './checkout.js';
140
+ import { harness, startAgent } from './spawn.js';
141
+ import { establishOwnerSession, listOwnerSessionIds, OWNER_SESSION_GRACE_MS, readOwnerSession, removeOwnerSession, validSessionUuid, writeOwnerSession, } from './session.js';
142
+ import { startToolsServer } from './tools.js';
143
+ import { listPanel3CodexOwnerHomeIds, removePanel3CodexOwnerHome, } from '../codex-home.js';
144
+ import { getMachineIdentity, scratchDir } from '../config.js';
145
+ import { listCodebases } from '../codebases.js';
146
+ import { killTree, processIsAlive } from '../win-shell.js';
147
+ import { hostname, uptime } from 'node:os';
148
+ const USAGE = 'usage: cs3 run [--once]';
149
+ /** How long between takes. Short, because it is the whole delay between a user
150
+ * sending and a card showing an agent on it, and the take is one small indexed
151
+ * read against the user's own rows. */
152
+ const POLL_INTERVAL_MS = 2_000;
153
+ /**
154
+ * How long a run with no pid recorded is given before it counts as gone.
155
+ *
156
+ * ═══ IT EXISTS BECAUSE `mine` IS ONLY THIS PROCESS'S MEMORY. ═══ The take
157
+ * writes the run row and the pid lands a moment later, so between the two the
158
+ * row is indistinguishable from one whose daemon died — to this daemon. To a
159
+ * SECOND daemon on the same machine it is worse than indistinguishable: the run
160
+ * is not in its `mine` set at all, so without this it would hand back the turns
161
+ * of a run that started seconds ago and mark it failed while the agent was
162
+ * genuinely working. That configuration is not hypothetical; it is what the
163
+ * lease is proved with.
164
+ *
165
+ * Thirty seconds is far longer than the one round trip it has to cover and far
166
+ * shorter than any run, so it costs a genuinely dead run one extra poll cycle at
167
+ * most.
168
+ *
169
+ * ═══ AND `panel3_resume` HOLDS THE SAME NUMBER, SO IT IS EXPORTED AND CHECKED.
170
+ * ═══ The claim refuses to hand the same run out twice inside this window, which
171
+ * is this rule from the other side: a run that has just been claimed has no
172
+ * process yet and everything else has to leave it alone long enough for one to
173
+ * exist. Two comments saying they agree is not a check, and the two are in
174
+ * different languages in different files — so the contract test reads the
175
+ * interval out of the function definition and compares it to this. Disagreement
176
+ * either reaps a resume a second after it started, or makes a dead run wait out
177
+ * two windows.
178
+ */
179
+ export const PID_GRACE_MS = 30_000;
180
+ /**
181
+ * How many processes may be started for one run, counting the first.
182
+ *
183
+ * ═══ IT IS NOT A NEW NUMBER, AND THAT IS THE REASON FOR IT. ═══
184
+ * `orchestrator.ts:1107` `MAX_ATTEMPTS = 3` and `orchestrator.ts:4421`
185
+ * `MAX_REPLY_ATTEMPTS = 3` are the same bound, with the reason already written
186
+ * down at `orchestrator.ts:1102-1105`: three attempts is enough to ride out a
187
+ * genuinely transient failure and short enough that a deterministic one stops
188
+ * costing the user minutes of spawn time, and it is not configurable, because a
189
+ * knob here would mostly serve to hide a bug behind a bigger number. A fourth
190
+ * number invented here would be a dialect.
191
+ *
192
+ * ═══ AND THE BOUND ITSELF IS NOT THIS CONSTANT. ═══ `panel3_resume` counts the
193
+ * attempt and refuses past three IN THE SAME STATEMENT, because a bound whose
194
+ * counter can silently miss an attempt is not a bound
195
+ * (`20260806120000:1096-1099`). This is the number the daemon says out loud, and
196
+ * a contract test reads the literal back out of the function definition and
197
+ * compares the two, exactly as `PID_GRACE_MS` is compared against that
198
+ * function's 30 seconds.
199
+ */
200
+ export const MAX_ATTEMPTS = 3;
201
+ /** The level the daemon puts on a card of its own accord. Anything deeper exists
202
+ * because an agent asked for it, and its level comes off the row
203
+ * `panel3_dispatch` wrote rather than from anything here. */
204
+ const LEVEL = 1;
205
+ const artifactAnswer = (id, revision, selected) => {
206
+ if (!id || revision === null || revision === undefined || selected?.length !== 1)
207
+ return null;
208
+ const answer = selected[0]?.trim().toLowerCase();
209
+ return answer === 'approve' || answer === 'request changes'
210
+ ? { id, revision, answer }
211
+ : null;
212
+ };
213
+ export function deliveredArtifactAnswer(events, delivered) {
214
+ if (!delivered.mine || delivered.ask_id === null)
215
+ return null;
216
+ const event = events.find((candidate) => candidate.kind === 'question' && candidate.id === delivered.ask_id);
217
+ const answer = artifactAnswer(event?.relatedArtifactId, event?.relatedArtifactRevision, event?.artifactAnswer ? [event.artifactAnswer] : null);
218
+ return answer === null ? null : { questionId: delivered.ask_id, ...answer };
219
+ }
220
+ async function rearmedArtifactAnswer(client, askId) {
221
+ const asks = await withAskContent(client, await returned(client.from('panel3_asks').select(`id, ${ASK_CONTENT_COLUMNS}`).eq('id', askId), 'read', `question ${askId}`));
222
+ const ask = asks[0];
223
+ return ask === undefined
224
+ ? null
225
+ : artifactAnswer(ask.related_artifact_id, ask.related_artifact_revision, ask.selected_options);
226
+ }
227
+ const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
228
+ /**
229
+ * ═══ NOTHING IN THIS FILE STAMPS A COLUMN FROM THIS MACHINE'S CLOCK ANY MORE,
230
+ * AND THAT IS A RULE RATHER THAN A TIDY-UP. ═══
231
+ *
232
+ * `started_at`, `ended_at` and `resumed_at` are all written by the database's own
233
+ * `now()`, in the statements that end or claim a run, and `panel3_owed_read_back`
234
+ * COMPARES a parent's end with its children's. A second clock writing any of them
235
+ * makes that comparison meaningless in a way nothing reports: skew one way hides
236
+ * the child that genuinely finished last and loses the read-back, with every row
237
+ * individually plausible. `failRun` was the last place that did it and now calls
238
+ * `panel3_end_run`.
239
+ *
240
+ * ═══ THE TWO CLOCKS STILL MEET IN ONE PLACE, AND IT IS DELIBERATE. ═══
241
+ * `recoverStranded` reads `started_at`, which the database stamped, and measures
242
+ * it against `Date.now()` for the pid grace window and against this machine's
243
+ * boot time for the reused-pid check. It has no choice: liveness is a local fact
244
+ * and the boot time is only knowable here. Both directions of skew are real:
245
+ *
246
+ * THE DATABASE AHEAD OF LOCAL BY MORE THAN THE GRACE WINDOW makes every
247
+ * pid-null run look younger than it is, forever, so nothing is ever reaped —
248
+ * which is the permanent strand the recovery exists to prevent.
249
+ *
250
+ * THE DATABASE BEHIND LOCAL BY MORE THAN THIS MACHINE'S UPTIME makes a run
251
+ * that started moments ago look older than the boot, so a live agent is
252
+ * declared gone and its turns are handed to a second one.
253
+ *
254
+ * Considered rather than missed, and left as it is: both machines run NTP — here
255
+ * they are the same machine, since the local stack is a container on it — so the
256
+ * skew is milliseconds against a thirty-second window and an uptime measured in
257
+ * days. A product where the two could drift that far would have to stamp both
258
+ * clocks and compare like with like.
259
+ */
260
+ /* ═══ THE NAME A PERSON TYPED, WHICH IS THE ONLY NAME THEY KNOW. ═══ `cs start`
261
+ is the command in the product's copy everywhere else, and this is the one
262
+ prefix on the one stream a person actually reads. */
263
+ const said = (what) => console.error(`cs start: ${what}`);
264
+ /**
265
+ * The taken rows, grouped by card, each card's turns still in send order.
266
+ *
267
+ * ═══ THE GROUPING IS WHY THE TAKE RETURNS ROWS AND NOT ONE BODY. ═══ ux.md:
268
+ * "A turn takes every unaddressed input on that card, oldest first, and hands
269
+ * them all over. Not the newest. Not one chosen as 'the request'." There is
270
+ * nowhere in this function for the product to choose between them, which is the
271
+ * property that matters: v2 chose, and the message it did not choose was filed
272
+ * as background and never acted on.
273
+ */
274
+ function byCard(taken) {
275
+ const cards = new Map();
276
+ for (const row of taken) {
277
+ const existing = cards.get(row.card_id);
278
+ if (existing)
279
+ existing.push(row);
280
+ else
281
+ cards.set(row.card_id, [row]);
282
+ }
283
+ return cards;
284
+ }
285
+ /**
286
+ * ═══ WHAT THIS CARD HAS ALREADY PRODUCED, NAMED AS IT READS NOW. ═══
287
+ *
288
+ * A card's receipts are level 1's report: every turn is answered by a fresh
289
+ * agent that remembers nothing, and what it most needs from the last one is what
290
+ * that one made. So they are read here and handed over in the prompt, exactly as
291
+ * a dispatched run's report is, rather than being left behind a tool call — see
292
+ * `levelOnePrompt` for the measured failure this closes.
293
+ *
294
+ * THE NAME IS RESOLVED, NOT READ OFF THE RECEIPT, through the one function
295
+ * `cs3 show` uses for it. `label` is written when the thing is made and the
296
+ * coordinator may rename the thing on the next turn; printing the label then
297
+ * hands the agent the name of a different, real object. One owner for that,
298
+ * because two would eventually disagree about which name a card is showing.
299
+ */
300
+ async function receiptsFor(client, cardId) {
301
+ const outputs = await returned(client
302
+ .from('panel3_outputs')
303
+ .select('id, card_id, run_id, kind, ref_id, label, created_at')
304
+ .eq('card_id', cardId)
305
+ .order('created_at'), 'read', `what card ${cardId} has produced`);
306
+ const names = await loadOutputNames(client, outputs);
307
+ return outputs.map((o) => `${o.kind.padEnd(10)} ${outputOf(o, names)} id ${o.ref_id}`);
308
+ }
309
+ /**
310
+ * ═══ WHAT THE PERSON ATTACHED TO THIS CARD, FORMATTED THE ONE WAY THE PRODUCT
311
+ * NAMES AN ATTACHMENT. ═══
312
+ *
313
+ * The product applies the attachment (plan-work-items.md constraint 5): level 1
314
+ * is handed it here rather than asked to go and find it, exactly as `produced`
315
+ * is handed over rather than left behind `list_work_items`. Most cards have
316
+ * none, since Task 2 offers attaching only when a card is started, so this is
317
+ * usually an empty list, and `levelOnePrompt` says nothing about attachments at
318
+ * all when it is.
319
+ */
320
+ // Exported so a test can prove the wiring itself: that this reads the CALLER's
321
+ // own card through `loadAttachments` rather than a copy, the same reason
322
+ // `dispatch`'s equivalent read is proven in panel3-tools.contract.test.mjs
323
+ // instead of only the formatter it feeds.
324
+ export async function attachmentsFor(client, cardId) {
325
+ return (await loadAttachments(client, cardId)).map(attachmentLine);
326
+ }
327
+ /**
328
+ * What level 1 is sent, and also what is written to `panel3_runs.brief`: the
329
+ * column's own question is "what was this agent asked", answered with the exact
330
+ * text it was asked with rather than with a description of it.
331
+ *
332
+ * THE WORDS THEMSELVES ARE IN `prompt.ts`, which is where the rules that govern
333
+ * them are written down. This is only the join: the take hands over rows, the
334
+ * record says what the card has made, and a prompt is about a title, some
335
+ * messages and that list.
336
+ */
337
+ // Exported for the same reason as `attachmentsFor` above: the wiring test
338
+ // calls this with what that function actually returned, so what a coordinator
339
+ // is told is proven to come from the card's own attachment rows rather than
340
+ // only from `levelOnePrompt`'s own formatting tests in panel3-prompt.
341
+ export const briefFor = (title, turns, produced, attachments, codebases = null) => levelOnePrompt(title, turns.map((t) => t.body), produced, attachments, codebases?.map((codebase) => ({
342
+ id: codebase.id,
343
+ name: codebase.name,
344
+ identity: codebase.gitRemoteUrl,
345
+ located: hasCheckoutForCodebase(codebase),
346
+ })) ?? null);
347
+ // ---------------------------------------------------------------------------
348
+ // WRITES. Every one goes through the shared guard in `client.ts`, which returns
349
+ // the rows or throws. `.select()` on each is what makes that possible: a
350
+ // Supabase write with no select returns neither data nor an error, and the guard
351
+ // refuses that shape rather than reading it as success.
352
+ /**
353
+ * ═══ WHAT IS RUNNING A RUN, AND — AT LEVEL 1 — WHAT IT WAS SENT. ═══
354
+ *
355
+ * The run row already exists in both paths: the take wrote it for level 1 and
356
+ * `panel3_dispatch` wrote it for a dispatched agent, which is what makes
357
+ * constraint 8 unconditional rather than an ordering this file has to be careful
358
+ * about. So this is never the row's creation, only the facts that could not be
359
+ * known before the process existed.
360
+ *
361
+ * THE BRIEF IS PASSED ONLY WHERE IT IS STILL A PLACEHOLDER. The take cannot know
362
+ * what an agent will be sent, so level 1's row carries a placeholder until here;
363
+ * a dispatch knows the brief before the row exists and writes it there. Passing
364
+ * it again would be rewriting a brief that ux.md fixes at dispatch.
365
+ *
366
+ * A failure here is NOT fatal. The agent is already running and killing it over
367
+ * a bookkeeping write would cost the user the work. What is lost is precision:
368
+ * the run has no pid, so once this daemon is gone recovery treats it as dead —
369
+ * which errs towards answering again rather than stranding.
370
+ */
371
+ async function recordProcess(client, runId, pid, brief, processToken) {
372
+ let query = client
373
+ .from('panel3_runs')
374
+ .update({ pid, ...(brief === undefined ? {} : { brief }) })
375
+ .eq('id', runId);
376
+ if (processToken !== undefined)
377
+ query = query.eq('process_token', processToken);
378
+ const written = await returned(query.select('id'), 'record what is running', `run ${runId}`);
379
+ if (written.length === 0)
380
+ throw new Error(`could not record what is running for run ${runId}: its activation has ended`);
381
+ }
382
+ /**
383
+ * ═══ A RUN GIVEN UP ON: ENDED, ITS TURNS HANDED BACK, ITS CARD IN HAND. ═══
384
+ *
385
+ * One statement, in `panel3_give_up`, for the reason 20260821170000 gives at
386
+ * length: as two round trips there was an instant in which the turns were
387
+ * takeable and the run still read `running`, and an answer landing in that
388
+ * instant was accepted onto a card another daemon was already free to take.
389
+ *
390
+ * Returns the number of turns handed back, or NULL when the run was no longer
391
+ * running — which is this function losing cleanly to a `panel3_answer` that got
392
+ * there first. It is a fact about the record rather than a failure, so it does
393
+ * not go through `returned()`, exactly as the answer's own null does not.
394
+ */
395
+ async function giveUp(client, runId, reason, processToken) {
396
+ const { data, error } = await client
397
+ .rpc('panel3_give_up', {
398
+ p_run_id: runId,
399
+ p_reason: reason,
400
+ ...(processToken === undefined ? {} : { p_process_token: processToken }),
401
+ });
402
+ if (error)
403
+ throw new Error(`could not give up run ${runId}: ${error.message}`);
404
+ /* THE RUN IS OVER, so whatever it read is dropped. Every ending does this —
405
+ here, `failRun` and `writeAnswer` — because a daemon stays up for days and
406
+ has no business holding Tuesday's secret. */
407
+ forgetSecrets(processToken === undefined ? runId : `${runId}:${processToken}`);
408
+ return data;
409
+ }
410
+ /**
411
+ * ═══ THE ANSWER, THE END OF THE RUN AND THE STATE OF THE CARD ARE ONE
412
+ * STATEMENT. ═══
413
+ *
414
+ * They are one event, and writing them as three round trips left a window where
415
+ * the answer was on the card while the run still read running: recovery then
416
+ * handed the turns back and a second agent answered the same question beside an
417
+ * answer that was already there. `panel3_answer` is where that window used to be.
418
+ * It also decides whether the card is DONE, which it is only when nothing else
419
+ * on it is still live — a coordinator that dispatched and answered must not
420
+ * settle a card whose work is still going. See 20260823090000.
421
+ *
422
+ * ═══ AND IT IS SHARED BY EVERY LEVEL ON PURPOSE. ═══ ux.md's "whoever did the
423
+ * work writes the answer" is only structurally true if there is exactly one way
424
+ * for text to reach a card. A second path for dispatched runs is where a
425
+ * summarising hop would eventually be added.
426
+ *
427
+ * ═══ THIS IS THE ONE v3 WRITE THAT DOES NOT GO THROUGH `returned()`. ═══ The
428
+ * shared guard treats "no error and no data" as a result it does not understand,
429
+ * which is right everywhere else and wrong here: a null from `panel3_answer` is a
430
+ * FACT about the record — the run was no longer running, so nothing was written
431
+ * — and it has its own sentence below. A failure is still a failure and still
432
+ * throws.
433
+ *
434
+ * The run's text is on the card as a turn and is deliberately NOT copied into
435
+ * `report` as well: one fact, one place. `report` carries the reason on the
436
+ * failure paths, where there is no turn and it is the only record.
437
+ *
438
+ * ═══ AND A RUN WITH NOTHING TO SAY PASSES NULL, WHICH IS NOT THE SAME AS
439
+ * SAYING NOTHING. ═══ ux.md's "outbound follows the work": a run started
440
+ * again only to settle a question from somebody it sent did no work for the
441
+ * person, and its words went to that agent through the answer. It is still
442
+ * ended here, and the card is still settled by the same statement, because such
443
+ * a run can be the last thing live on it — but no turn is written, exactly as
444
+ * none is written for a run that stopped to ask.
445
+ */
446
+ async function writeAnswer(client, runId, cardId, text, processToken) {
447
+ /* ═══ THE SECOND OF THE TWO CHOKEPOINTS THE CREDENTIAL RULE RESTS ON. ═══
448
+ This is the ONE path an agent's own words take to a hosted row — a turn on
449
+ the card at levels 1 and 2, and a level 3's `panel3_runs.report`, which
450
+ `panel3_answer` writes instead. Tool arguments are the other, redacted in
451
+ `buildServer`. Identity for a run that read no credential, so every existing
452
+ exact-string assertion is byte-identical. */
453
+ const { data: turnId, error } = await client
454
+ .rpc('panel3_answer', {
455
+ p_run_id: runId,
456
+ p_body: text === null ? null : redactSecrets(processToken === undefined ? runId : `${runId}:${processToken}`, text),
457
+ ...(processToken === undefined ? {} : { p_process_token: processToken }),
458
+ });
459
+ if (error) {
460
+ /* Thrown, not settled as failed. The run stays `running` with a process
461
+ that has now exited, so recovery reaps it: at level 1 the turns are
462
+ answered again, so the user gets an answer late instead of a card marked
463
+ failed for a database hiccup. The caller prints the reason. */
464
+ throw new Error(`could not write the answer to card ${cardId}: ${error.message}`);
465
+ }
466
+ /* ═══ INCLUDING WHEN THE RUN STOPPED TO ASK AND WILL BE STARTED AGAIN. ═══
467
+ That reads like a mistake and is not: the process is gone, and the one that
468
+ takes its place is a fresh process with a fresh context which reads the
469
+ credential again through `get_credential`. What it is handed to start from —
470
+ its own report, and the card — went through this same substitution, so there
471
+ is nothing left for a stale entry to protect. */
472
+ forgetSecrets(processToken === undefined ? runId : `${runId}:${processToken}`);
473
+ if (turnId === null) {
474
+ /* NO TURN WAS WRITTEN, and there are FOUR reasons, which mean different
475
+ things and must not be printed as one line.
476
+
477
+ IT STOPPED TO ASK. Its last words were the question, which is already on
478
+ the card, and its end has just been stamped by the same statement that
479
+ refused the turn. Nothing is wrong and nothing is lost.
480
+
481
+ IT HAD NOTHING TO SAY TO THE PERSON. A run started only to settle a
482
+ question from somebody it sent is asked for exactly this, and it is the
483
+ daemon that asked.
484
+
485
+ IT IS A WORKER, and its words went into its report for whoever sent it.
486
+ ux.md: one request is one answer, written by the level that owns the card.
487
+
488
+ OR IT WAS NO LONGER RUNNING — recovery had given up on it and, at level 1,
489
+ its turns belong to another run now. Writing anyway would put a second
490
+ answer on the card. The process's work is lost, which is the honest cost
491
+ of having been declared dead.
492
+
493
+ ═══ AND THE STATE IS READ FIRST, WHICH IS THE FIX FOR A LINE THAT USED TO
494
+ LIE. ═══ The last of the four was reachable with a null body — a re-armed
495
+ run given up on while its process was still going — and it printed the
496
+ second one's sentence, "answered a question, so it wrote nothing here",
497
+ about a run that had been declared dead and written nothing anywhere. So
498
+ the state gates the two honest sentences rather than sitting under them.
499
+ It decides nothing except which sentence is true: whichever of the four
500
+ happened has already happened, in one statement, before this line runs. */
501
+ const run = await runNow(client, runId);
502
+ if (run.state === 'asked') {
503
+ out(`asked card ${cardId} run ${runId} stopped on a question, so it wrote no answer`);
504
+ }
505
+ else if (run.state !== 'finished') {
506
+ said(`run ${runId} was ${run.state}, so its answer was not written to card ${cardId}`);
507
+ }
508
+ else if (run.level === 1 && run.conversationOwnerId !== null) {
509
+ out(`settled card ${cardId} run ${runId} assigned the conversation, so it wrote nothing here`);
510
+ }
511
+ else if (text === null) {
512
+ out(`settled card ${cardId} run ${runId} answered a question, so it wrote nothing here`);
513
+ }
514
+ else if (run.level === 3) {
515
+ out(`reported run ${runId} ${text.length} characters, to whoever sent it`);
516
+ }
517
+ else {
518
+ /* FINISHED, WITH SOMETHING TO SAY, AND NO TURN. It had already answered
519
+ once, which is the one shape left, and it is said rather than assumed
520
+ away. */
521
+ said(`run ${runId} had already ended, so its answer was not written to card ${cardId}`);
522
+ }
523
+ return ownerSettlementAccepted(run, processToken);
524
+ }
525
+ out(`answered card ${cardId} run ${runId} ${text?.length ?? 0} characters`);
526
+ return true;
527
+ }
528
+ export function ownerSettlementAccepted(run, processToken) {
529
+ return (run.state === 'asked' || run.state === 'finished')
530
+ && (processToken === undefined || run.processToken === processToken);
531
+ }
532
+ /** What the record says a run is now, and at what level. Read only to say the
533
+ * right sentence about something that has already happened; nothing branches on
534
+ * it that could have gone the other way. */
535
+ async function runNow(client, runId) {
536
+ const runs = await returned(client
537
+ .from('panel3_runs')
538
+ .select('state, level, process_token, card:panel3_cards!panel3_runs_card_id_fkey(conversation_run_id)')
539
+ .eq('id', runId), 'read', `the state of run ${runId}`);
540
+ const run = runs[0];
541
+ return run
542
+ ? {
543
+ state: run.state,
544
+ level: run.level,
545
+ conversationOwnerId: run.card?.conversation_run_id ?? null,
546
+ processToken: run.process_token,
547
+ }
548
+ : { state: 'no longer on the record', level: null, conversationOwnerId: null, processToken: null };
549
+ }
550
+ /**
551
+ * A run whose process failed, ended with the reason ON ITS OWN COLUMN, and its
552
+ * turns left exactly where they are.
553
+ *
554
+ * ═══ NEVER INTO `report`, WHICH IS THE RUN'S OWN WORDS. ═══ `report` is what a
555
+ * respawn is handed, and this build starts settled runs again — an answered
556
+ * question does exactly that — so a reason written there would destroy the state
557
+ * the next attempt was going to resume from, and `cs3 show` would print the
558
+ * daemon's sentence under a heading that says the run wrote it. One owner per
559
+ * column: the run writes `report`, whoever ended it writes `failed_because`.
560
+ *
561
+ * ═══ AND IT IS A STATEMENT RATHER THAN AN UPDATE, BECAUSE `ended_at` IS
562
+ * COMPARED AGAINST OTHER ROWS' ENDS. ═══
563
+ *
564
+ * This wrote the stamp from THIS MACHINE'S clock, because a PostgREST update has
565
+ * nowhere to ask for the server's time. Everything else that ends a run —
566
+ * `panel3_answer`, `panel3_give_up` — uses the database's `now()`, and
567
+ * `panel3_owed_read_back` measures a parent's end against its children's. So the
568
+ * two stamps met, and skew between them loses a read-back SILENTLY: a machine
569
+ * clock running ahead writes a parent's end later than the child that genuinely
570
+ * outlived it, the comparison finds nobody who finished after it, and the
571
+ * synthesis never happens with every row individually plausible.
572
+ *
573
+ * The header of this file already argues that v3 must not compare two clocks.
574
+ * This was the one place left that did.
575
+ */
576
+ async function failRun(client, runId, reason) {
577
+ /* ═══ ONLY A RUN THAT HAS NOT ALREADY ENDED, AND THAT GUARD IS IN THE
578
+ STATEMENT. ═══ Unguarded, this wrote over a run that had been started again
579
+ in the meantime — which is not hypothetical now that an answered question
580
+ restarts a settled row — and marked a live agent failed. False is a fact the
581
+ caller reports, not a failure. */
582
+ const { data, error } = await client
583
+ .rpc('panel3_end_run', { p_run_id: runId, p_reason: reason });
584
+ if (error)
585
+ throw new Error(`could not end run ${runId}: ${error.message}`);
586
+ forgetSecrets(runId);
587
+ return data === true;
588
+ }
589
+ /**
590
+ * What the card is doing, written by the daemon.
591
+ *
592
+ * ═══ EXCEPT ON A CARD THE USER STOPPED, WHICH NOTHING HERE MAY OVERWRITE. ═══
593
+ * ux.md: the user's Stop is absolute. Its own effect is the reason this guard is
594
+ * needed rather than a precaution. `panel3_stop_card` ends the runs, this daemon
595
+ * kills their processes, and the level 1 agent then exits badly, which is the
596
+ * one path in this file that writes a card `failed`. Without the predicate the
597
+ * user presses Stop and the card reads Failed a second later, about a failure
598
+ * that never happened.
599
+ *
600
+ * Every other card write already carries the guard in SQL: `panel3_answer` and
601
+ * `panel3_give_up` both refuse a run that is not live, and a stopped run is not.
602
+ */
603
+ async function setCardState(client, cardId, state) {
604
+ await returned(client
605
+ .from('panel3_cards')
606
+ .update({ state })
607
+ .eq('id', cardId)
608
+ .neq('state', 'stopped')
609
+ .select('id'), 'set the state of', `card ${cardId}`);
610
+ }
611
+ // ---------------------------------------------------------------------------
612
+ /**
613
+ * One card, from the turns the take handed over to the answer on its thread.
614
+ *
615
+ * ═══ THERE IS NOTHING TO UNDO HERE, AND THAT IS THE POINT. ═══ The take wrote
616
+ * the run row in the same statement that leased the turns, so this function is
617
+ * never in the position of holding a lease it cannot honour: whatever it fails
618
+ * at, the row exists, `cs3 show` can see it, and recovery can reach it. The
619
+ * earlier version wrote the row two round trips later and had to hand the turns
620
+ * back on failure — which covered a thrown error and not a kill, and a kill in
621
+ * that window stranded the card forever.
622
+ *
623
+ * ═══ AND WHAT HAPPENS WHEN THE PROCESS IS OVER IS `settle`'s, NOT THIS
624
+ * FUNCTION'S. ═══ It used to be written out again here, and the two copies
625
+ * were already the same thing said twice: the agent's words onto the card
626
+ * through `writeAnswer`, or the run ended with its reason. recovery Slice 3 gave
627
+ * that moment a second job, starting a process that died badly all over again,
628
+ * and a second copy of THAT is two features owning one respawn, which is the
629
+ * failure recovery-1/ux.md names in as many words. So there is one owner, and
630
+ * the level is what tells it how a run of this shape ends.
631
+ */
632
+ async function answerCard(client, tools, machineId, cardId, turns) {
633
+ const runId = turns[0].run_id;
634
+ /* BEFORE THE SPAWN, AND ITS FAILURE IS THE SPAWN'S FAILURE. The receipts are
635
+ part of what the agent is sent, so a read that fails must not be papered
636
+ over with an empty list: that reads as a card that has made nothing, which
637
+ is how an agent creates a second epic beside the one it cannot see. The
638
+ attachments read carries the same rule: a failed read here must not read
639
+ as "nothing is attached", which is a different card than the one that was
640
+ actually sent. */
641
+ /* ═══ AND A FAILURE HERE ENDS THE RUN, RATHER THAN LEAVING IT RUNNING WITH NO
642
+ PROCESS. ═══ The take already wrote the run row in the statement that leased
643
+ the turns, so a throw between here and `startAgent` leaves a run reading
644
+ `running` with a null pid and nothing on stderr the person can see.
645
+ `recoverStranded` then reads that as a machine that went away, hands the
646
+ message back, and `panel3_take_turns` leases it to A BRAND NEW RUN whose
647
+ attempts start again at one — so a permanent failure, such as a read this
648
+ build cannot make against the current schema, repeats forever while the card
649
+ says Working and never says why. This is the sixth of `endRun`'s endings and
650
+ the last one that was missing: `resumeRun` and `startRearmed` already end
651
+ their two post-claim failures this way for exactly this reason.
652
+ THE REASON IS SHAREABLE. All three reads are `returned()` calls against the
653
+ database, whose messages name tables and columns and never a local path, so
654
+ constraint 6 is satisfied without a level fork here. */
655
+ let brief;
656
+ try {
657
+ brief = briefFor(turns[0].card_title, turns, await receiptsFor(client, cardId), await attachmentsFor(client, cardId), await codebasesForRun(client, runId));
658
+ }
659
+ catch (error) {
660
+ const why = error instanceof Error ? error.message : String(error);
661
+ await endRun(client, LEVEL, runId, cardId, why);
662
+ throw new Error(`NO AGENT IS RUNNING: ${why}`);
663
+ }
664
+ /* ═══ THE RUN ID IS ON THE URL, AND THAT IS THE WHOLE OF WHAT THE AGENT IS
665
+ TOLD ABOUT ITS OWN STANDING. ═══ The tools server reads the level off the
666
+ run row this id names, so the daemon does not tell the child what it may do
667
+ and the child has nothing to claim. `LEVEL` below decides argv only — which
668
+ of the harness's own tools the process gets — and the two can never disagree
669
+ about the record, because only one of them consults it. */
670
+ /* AN EMPTY DIRECTORY OF THE USER'S OWN, because level 1 has no code tool to
671
+ use a real one with, and the daemon's inherited cwd under a launchd login
672
+ item is the filesystem root. */
673
+ const started = startAgent(brief, LEVEL, tools.urlFor(runId), scratchDir());
674
+ out(`run ${runId} level ${LEVEL} ${started.pid ? `pid ${started.pid}` : 'no process'}`);
675
+ try {
676
+ await recordProcess(client, runId, started.pid, brief);
677
+ }
678
+ catch (error) {
679
+ // Said, not fatal. See `recordProcess` for what this costs and why the
680
+ // agent is not killed over it.
681
+ said(`${error instanceof Error ? error.message : String(error)} (the run is still going)`);
682
+ }
683
+ return settle(client, tools, machineId, LEVEL, runId, cardId, started, false);
684
+ }
685
+ /**
686
+ * WHICH PROJECT A RUN'S CARD IS FILED UNDER, or null when the card has none.
687
+ *
688
+ * ═══ ASKED OF THE RECORD AT EVERY SPAWN, RATHER THAN CARRIED. ═══ The card is
689
+ * the durable object and its project can change between one dispatch and the
690
+ * next; a value threaded through the daemon would be the state of the card when
691
+ * the daemon last looked. It is one indexed read on the path that is about to
692
+ * start a whole agent.
693
+ *
694
+ * ═══ AND A FAILED READ IS NOT "NO PROJECT". ═══ `returned` throws, which the
695
+ * caller turns into a dispatch that did not happen. Returning null here would
696
+ * quietly send the agent to the machine-wide fallback checkout, which is another
697
+ * project's code.
698
+ */
699
+ async function projectOfRun(client, runId) {
700
+ const rows = await returned(client.from('panel3_runs').select('card:panel3_cards!panel3_runs_card_id_fkey(project_id)').eq('id', runId), 'read', 'which project this run belongs to');
701
+ return rows[0]?.card?.project_id ?? null;
702
+ }
703
+ async function codebasesForRun(client, runId) {
704
+ const projectId = await projectOfRun(client, runId);
705
+ return projectId ? listCodebases(client, projectId) : null;
706
+ }
707
+ async function codebaseOfRun(client, runId) {
708
+ const rows = await returned(client
709
+ .from('panel3_runs')
710
+ .select('codebase_id, card:panel3_cards!panel3_runs_card_id_fkey(project_id)')
711
+ .eq('id', runId), 'read', `which codebase run ${runId} belongs to`);
712
+ const row = rows[0];
713
+ if (!row?.codebase_id || !row.card?.project_id) {
714
+ throw new Error('This code work does not name a registered project codebase.');
715
+ }
716
+ const codebase = (await listCodebases(client, row.card.project_id))
717
+ .find((candidate) => candidate.id === row.codebase_id);
718
+ if (!codebase) {
719
+ throw new Error('The selected codebase is no longer registered on this project.');
720
+ }
721
+ return codebase;
722
+ }
723
+ /** One directory rule for every spawn whose level and ownership are known. */
724
+ async function workingDirectory(client, runId, level, isOwner, knownCodebase) {
725
+ if (level === 1)
726
+ return scratchDir();
727
+ if (knownCodebase === null) {
728
+ if (level === 2 && isOwner)
729
+ return scratchDir();
730
+ throw new Error('This code work does not name a registered project codebase.');
731
+ }
732
+ return checkoutForCodebase(knownCodebase ?? await codebaseOfRun(client, runId), hostname());
733
+ }
734
+ /**
735
+ * ═══ ONE AGENT, STARTED UNDER ANOTHER THAT IS STILL RUNNING. ═══
736
+ *
737
+ * Called from the `dispatch` tool, and it returns as soon as the process exists.
738
+ * The promise it hands back settles long afterwards, and the caller puts it
739
+ * where the daemon can wait on it rather than awaiting it itself: an agent that
740
+ * waited for the work it started would hold its own run open for the length of
741
+ * it, which is the thing ux.md forbids in as many words.
742
+ *
743
+ * ═══ THE ORDER IS THE WHOLE CORRECTNESS ARGUMENT. ═══ The working copy is
744
+ * resolved FIRST, before anything is written, so a machine that has none refuses
745
+ * the dispatch rather than leaving a run row no process ever corresponded to.
746
+ * Then the row, then the process, then the pid — constraint 8, in the only order
747
+ * that satisfies it.
748
+ */
749
+ async function startChild(client, tools, machineId, parentRunId, brief, codebase, parentProcessToken) {
750
+ const { data, error } = await client.rpc('panel3_dispatch', {
751
+ p_parent_run_id: parentRunId,
752
+ p_brief: brief,
753
+ p_machine_id: machineId,
754
+ p_codebase_id: codebase?.id ?? null,
755
+ p_codebase_label: codebase?.name ?? null,
756
+ ...(parentProcessToken === undefined ? {} : { p_process_token: parentProcessToken }),
757
+ });
758
+ if (error)
759
+ throw new Error(`could not start an agent under run ${parentRunId}: ${error.message}`);
760
+ const row = data?.[0];
761
+ if (!row) {
762
+ /* NOTHING WAS WRITTEN AND NOTHING IS RUNNING, and the two reasons are said
763
+ together because the caller cannot tell them apart from here and both mean
764
+ the same thing to it. */
765
+ throw new Error(`NO AGENT WAS STARTED and nothing was written: run ${parentRunId} has ended, is already as `
766
+ + 'deep as anything may be sent from, or this conversation already has its owner. Exit now.');
767
+ }
768
+ if (row.run_level !== 2 && row.run_level !== 3) {
769
+ /* UNREACHABLE, AND STILL SETTLED. `panel3_dispatch` writes `parent.level + 1`
770
+ from a parent it has just checked is below 3, so there is no level here
771
+ this cannot spawn. If that ever stops being true, the row exists and
772
+ nothing will ever start for it, and leaving it `running` would make a
773
+ recovery sweep wait out the pid grace window to conclude what is already
774
+ known. */
775
+ const why = `run ${row.run_id} was written at level ${row.run_level}, which cannot be spawned`;
776
+ await giveUp(client, row.run_id, why, row.process_token ?? undefined);
777
+ throw new Error(why);
778
+ }
779
+ const level = row.run_level;
780
+ let cwd;
781
+ let prompt;
782
+ try {
783
+ cwd = await workingDirectory(client, row.run_id, level, level === 2, codebase);
784
+ prompt = level === 2
785
+ ? await initialOwnerPrompt(client, row.run_card_id, parentRunId, brief)
786
+ : brief;
787
+ }
788
+ catch (error) {
789
+ const why = error instanceof Error ? error.message : String(error);
790
+ await giveUp(client, row.run_id, why, row.process_token ?? undefined);
791
+ throw new Error(`NO AGENT IS RUNNING: ${why}`);
792
+ }
793
+ const processToken = row.process_token ?? undefined;
794
+ const isOwner = level === 2;
795
+ const started = startAgent(prompt, level, tools.urlFor(row.run_id, processToken), cwd, isOwner ? { ownerId: row.run_id } : undefined);
796
+ if (started.pid === null) {
797
+ /* THERE IS A ROW AND THERE IS NO PROCESS, which is the one shape the record
798
+ must never be left in quietly. The answer is already settled — nothing ran
799
+ — so the reason is read off it, the run is ended with that reason on it,
800
+ and the tool call fails saying no agent was started. */
801
+ const answer = await started.answered;
802
+ const reason = answer.ok ? 'the process ended before it could be identified' : answer.reason;
803
+ await giveUp(client, row.run_id, reason, processToken);
804
+ throw new Error(`NO AGENT IS RUNNING: ${reason}`);
805
+ }
806
+ out(`dispatch run ${row.run_id} level ${level} under ${parentRunId} pid ${started.pid}`);
807
+ try {
808
+ await recordProcess(client, row.run_id, started.pid, undefined, processToken);
809
+ }
810
+ catch (error) {
811
+ // Said, not fatal, exactly as at level 1: the agent is running and killing
812
+ // it over a bookkeeping write would cost the user the work.
813
+ said(`${error instanceof Error ? error.message : String(error)} (the run is still going)`);
814
+ }
815
+ return {
816
+ runId: row.run_id,
817
+ settled: settle(client, tools, machineId, level, row.run_id, row.run_card_id, started, true, processToken, isOwner && processToken
818
+ ? ownerSessionLifecycle(started, row.run_id, harness(), processToken)
819
+ : undefined),
820
+ };
821
+ }
822
+ /**
823
+ * HOW MANY PROCESSES HAVE BEEN STARTED FOR THIS RUN, counting the first.
824
+ *
825
+ * ═══ IT SUPPLIES THE NUMBER IN THE SENTENCE, NOT THE BOUND. ═══ The bound is
826
+ * `panel3_resume`'s own predicate, counted and tested in the statement that
827
+ * grants the attempt, so a read here that is a moment out of date cannot grant
828
+ * an attempt the record refuses. The worst it can do is decline one the record
829
+ * would have allowed, which is a run that fails a spawn early rather than a
830
+ * fourth process nobody bounded.
831
+ *
832
+ * A row that is not there is not "one attempt". It is a run this daemon can
833
+ * neither retry nor fail, because there is nothing left to write to, and saying
834
+ * so is the caller's business rather than this function's to paper over.
835
+ */
836
+ async function attemptsSoFar(client, runId) {
837
+ const rows = await returned(client.from('panel3_runs').select('attempts').eq('id', runId), 'read', `how many times a process has been started for run ${runId}`);
838
+ const row = rows[0];
839
+ if (!row)
840
+ throw new Error(`run ${runId} is not there, so nothing can be recorded about it`);
841
+ return row.attempts;
842
+ }
843
+ /**
844
+ * ═══ ONE RUN, ENDED WITH ITS REASON, THE WAY A RUN OF ITS LEVEL ENDS. ═══
845
+ *
846
+ * There are two endings and the difference is not a preference, it is what the
847
+ * run is holding.
848
+ *
849
+ * A DISPATCHED RUN HOLDS NO LEASED TURNS. `panel3_give_up` hands nothing back
850
+ * and puts the card in `failed`, because nothing on that card will pick this
851
+ * piece of work up again unless a question outstanding somewhere else on it says
852
+ * work is genuinely coming back.
853
+ *
854
+ * A LEVEL 1 RUN HOLDS THE PERSON'S MESSAGE. Giving up hands it back for the next
855
+ * `panel3_take_turns` to lease to A BRAND NEW RUN, which is recovery's whole
856
+ * point when the MACHINE is gone and is the wrong ending everywhere else: a new
857
+ * run is a new `attempts`, so the bound below would never be spent, a harness
858
+ * that fails every time would be started again forever, and a coordinator that
859
+ * had already dispatched workers would dispatch them a second time. So level 1
860
+ * ends through `panel3_end_run`, which hands no turns back, and the card says
861
+ * failed.
862
+ *
863
+ * ═══ AND THE FORK LIVES HERE AND NOWHERE ELSE. ═══ SIX places end a run this
864
+ * way: the settle, the two failures after the claim in each of `resumeRun` and
865
+ * `startRearmed`, and the pre-spawn failure in `answerCard`. A copy of this rule
866
+ * in each is six places for it to drift, and it drifted the first time it could:
867
+ * those two functions hold the same pair of endings and only one pair was looked
868
+ * at. `answerCard`'s was missing outright, which is how a build that could not
869
+ * read a card's project turned every take into a silent take/recover loop.
870
+ *
871
+ * ALL FIVE CAN SEE LEVEL 1. `resumeRun` serving the retry is what opened its
872
+ * two. `startRearmed`'s were always reachable, because a re-arm has always
873
+ * served level 1, and were always ending one the way a dispatched run ends. The
874
+ * bound is what made that matter: a handed-back message is leased to a NEW run,
875
+ * whose attempts start again at one.
876
+ *
877
+ * ═══ AND THE GIVE-UPS THAT ARE NOT HERE ARE NOT OVERSIGHTS. ═══ `startChild`'s
878
+ * two end a dispatched run by construction, since a dispatch writes its parent's
879
+ * level plus one and there is no parent below level 1. The bad-level branches in
880
+ * `resumeRun` and `startRearmed` hold a level that is by definition none of the
881
+ * three, so there is no fork to make. And `recoverStranded` gives a level 1 run
882
+ * up ON PURPOSE, handing its message back, which is the whole of how another
883
+ * machine takes over a run whose machine is gone.
884
+ *
885
+ * Returns whether THIS call is what ended it. False is a fact about the record,
886
+ * not a failure: the run had already ended, and the caller says so.
887
+ */
888
+ async function endRun(client, level, runId, cardId, why) {
889
+ if (level !== 1)
890
+ return (await giveUp(client, runId, why)) !== null;
891
+ /* ═══ THE RUN FIRST, AND THE CARD ONLY IF THIS RUN WAS STILL THE CARD'S TO
892
+ FAIL. ═══
893
+ The card used to be written first, and the argument for that was a daemon
894
+ killed between the two: it left a card reading `failed` beside a run that
895
+ still read running with a dead process, which the record's own forbidden
896
+ state check reports and recovery corrects. THE PREEMPTION MAKES THAT ORDER A
897
+ LIE THAT NOTHING CORRECTS. A coordinator the person's next message replaced
898
+ is `stopped`, its process is killed by this same daemon, and it arrives here
899
+ having failed: writing the card `failed` then marks the card that a FRESH
900
+ coordinator is working on, about a run nobody is waiting for.
901
+ `panel3_end_run` is what knows the difference, because it refuses a run that
902
+ has already ended, so its answer is what the card write waits for.
903
+ WHAT THE ORDER COSTS is a daemon killed between the two, which leaves the
904
+ run `failed` and the card still reading `working` with nothing on it. The
905
+ product raises that as a forbidden state on the card itself, and the
906
+ person's next message takes the card again. That is a smaller and louder
907
+ wrong than a card marked failed under an agent that is working. */
908
+ const ended = await failRun(client, runId, why);
909
+ if (ended)
910
+ await setCardState(client, cardId, 'failed');
911
+ return ended;
912
+ }
913
+ /**
914
+ * ═══ THE END OF ONE RUN'S PROCESS, WHICHEVER SPAWN STARTED IT. ═══
915
+ *
916
+ * One owner for what happens when an agent's process exits, because there are
917
+ * four ways one comes to exist, being a card taken, a `dispatch`, a resume of a
918
+ * run whose process was found gone, and a re-arm, and they end identically. A
919
+ * second copy would be a second answer to "what does a dead agent mean", and the
920
+ * first thing to drift would be whether the card is told.
921
+ *
922
+ * ═══ `speaksToTheCard` IS THE ONE THING THAT DIFFERS ON THE WAY IN, AND IT IS
923
+ * ux.md's RULE RATHER THAN A SETTING. ═══ "Outbound follows the work." A run
924
+ * started again only to settle a question from somebody it sent did no work for
925
+ * the person: its words went to that agent, through the answer, and putting them
926
+ * on the card as well is a voice in a conversation the person did not ask to
927
+ * follow. It is still ended, and the card is still settled, because it can be
928
+ * the last thing live on it.
929
+ *
930
+ * ═══ AND `level` IS THE ONE THING THAT DIFFERS ON THE WAY OUT, WHICH IS
931
+ * `endRun`'s. ═══ It is above, with the whole argument: a dispatched run
932
+ * holds no leased turns and a level 1 run holds the person's message, so they
933
+ * cannot end the same way. What `answerCard` always did is what level 1 still
934
+ * does, and the only thing that has moved is where it is written.
935
+ */
936
+ function ownerSessionLifecycle(started, ownerId, ownerHarness, processToken, expectedSessionId) {
937
+ const pending = started.session.then((nativeSessionId) => {
938
+ if (nativeSessionId === null)
939
+ return false;
940
+ if (expectedSessionId && nativeSessionId !== expectedSessionId) {
941
+ said(`the resumed harness returned a different conversation for owner ${ownerId}; it will not be reused`);
942
+ return false;
943
+ }
944
+ try {
945
+ writeOwnerSession({
946
+ ownerId,
947
+ harness: ownerHarness,
948
+ nativeSessionId,
949
+ processToken,
950
+ state: 'pending',
951
+ });
952
+ return true;
953
+ }
954
+ catch (error) {
955
+ said(`could not save the local conversation for owner ${ownerId}: ${error instanceof Error ? error.message : String(error)}`);
956
+ return false;
957
+ }
958
+ });
959
+ // Start observing immediately; Codex reports thread.started before its answer.
960
+ void pending;
961
+ return {
962
+ established: async () => {
963
+ if (await pending)
964
+ establishOwnerSession(ownerId, processToken);
965
+ },
966
+ failed: async () => {
967
+ // A failed or rejected generation stays pending and therefore cannot be
968
+ // resumed. Reconciliation is the sole owner of stable-state deletion.
969
+ await pending;
970
+ },
971
+ };
972
+ }
973
+ function settle(client, tools, machineId, level, runId, cardId, started, speaksToTheCard = true, processToken, ownerSession) {
974
+ return started.answered.then(async (answer) => {
975
+ /* ═══ A RUN THAT STOPPED TO ASK DID NOT DIE, WHATEVER THE HARNESS PRINTED
976
+ ON ITS WAY OUT. ═══
977
+
978
+ ux.md: asking is TERMINAL for the agent that asks. It calls the tool, is
979
+ told "you have stopped", and exits — and it owes the person nothing more,
980
+ because its last words were the question and the question is already on
981
+ the card. So a harness that exits having said nothing to the person has
982
+ done exactly what it was told to.
983
+
984
+ `codexAnswer` reads that silence as a death (`spawn.ts`: "codex exited 0
985
+ without saying anything to the person"), which is the right reading of
986
+ every OTHER silent exit and the wrong one of this one. Claude happens to
987
+ emit a closing message after the tool call and Codex does not, so without
988
+ this the same question asked under the two harnesses ends two different
989
+ ways: answerable on one, and on the other retried twice and then FAILED
990
+ with the question still sitting open on a failed card.
991
+
992
+ ═══ SO THE RECORD DECIDES IT, NOT THE STDOUT. ═══ Whether a run asked is a
993
+ column, written by `panel3_ask` in the same statement that stopped the
994
+ run, and it is true or false identically for both harnesses. Read only on
995
+ the failure path, where it is the one thing that can turn a death back
996
+ into an ordinary ending, and then handed to `writeAnswer` — which already
997
+ owns this case and already prints its sentence — rather than to a second
998
+ ending written here beside it. */
999
+ if (!answer.ok) {
1000
+ const run = await runNow(client, runId);
1001
+ if (run.state === 'asked') {
1002
+ const accepted = await writeAnswer(client, runId, cardId, null, processToken);
1003
+ if (accepted)
1004
+ await ownerSession?.established();
1005
+ else
1006
+ await ownerSession?.failed();
1007
+ return;
1008
+ }
1009
+ /* Level 1's only successful result is the owner pointer written by
1010
+ dispatch. Its stdout is deliberately hidden, and Codex may therefore
1011
+ exit cleanly without an agent message after the tool succeeds. The
1012
+ record, not prose the launcher was told not to write, decides success. */
1013
+ if (level === 1 && run.conversationOwnerId !== null) {
1014
+ await writeAnswer(client, runId, cardId, null, processToken);
1015
+ return;
1016
+ }
1017
+ }
1018
+ if (!answer.ok) {
1019
+ await ownerSession?.failed();
1020
+ /* ═══ A PROCESS THAT STARTED AND THEN DIED BADLY IS STARTED AGAIN. ═══
1021
+ recovery-1/ux.md, Slice 3: "the agent's harness fails on something that
1022
+ is nobody's fault: a rate limit, a dropped connection, a provider
1023
+ outage. The product tries again rather than failing the card."
1024
+
1025
+ `started.pid` IS THE WHOLE OF THE DISTINCTION, and it is a fact rather
1026
+ than a reading of a string. What reaches here as `{ ok: false }` is
1027
+ never a verdict about the work. An agent that ran and concluded it
1028
+ could not do the thing exits 0 having said so, and that is an ANSWER,
1029
+ written to the card below. So what is left to separate is a process that
1030
+ ran and died from one that never existed, and `Started.pid` answers it:
1031
+ `could not start claude on this machine: ENOENT` and `claude is not
1032
+ installed` are facts about this machine that will be identical in two
1033
+ seconds and are fixed by a person, while `claude exited 1: API Error:
1034
+ 529 Overloaded` is what a rate limit, a dropped connection and a
1035
+ provider outage all look like from here. plan-slice-3.md section 1.2
1036
+ measured the alternative and rejected it: the provider's own words reach
1037
+ this daemon as a 500 character tail of one of two streams, absent
1038
+ entirely for a rate limit, and different again next release.
1039
+
1040
+ WHAT THIS DELIBERATELY GIVES UP is that a genuinely deterministic
1041
+ harness crash costs two extra spawns before the card fails. That is
1042
+ bounded, and it is the price of not depending on a string.
1043
+
1044
+ THE RESPAWN IS `resumeRun`, WHICH ALREADY EXISTS. ux.md: "two features
1045
+ owning one respawn is how two respawns are born." Nothing new starts a
1046
+ process here; the failure calls the sweep's own respawn, fenced by the
1047
+ pid this daemon watched exit, and AWAITS it, so the daemon's one
1048
+ `inFlight` entry covers the whole chain and `--once` waits for the last
1049
+ attempt rather than exiting mid-retry.
1050
+
1051
+ AND THE OLD RULING HERE IS AMENDED RATHER THAN DELETED. It read: "a
1052
+ process that started and then failed is a failure of the work, not of
1053
+ the machine: resuming it would put the same agent back on the same brief
1054
+ to fail the same way, FOREVER." The half that survives is forever, and
1055
+ the bound is what now provides it. */
1056
+ const attempts = await attemptsSoFar(client, runId);
1057
+ if (started.pid !== null && attempts < MAX_ATTEMPTS) {
1058
+ out(`retry run ${runId} attempt ${attempts} of ${MAX_ATTEMPTS} died: ${answer.reason}`);
1059
+ const again = processToken === undefined
1060
+ ? await resumeRun(client, tools, machineId, runId, started.pid)
1061
+ : await activateOwner(client, tools, machineId, runId, processToken, started.pid);
1062
+ if (again)
1063
+ return again.settled;
1064
+ /* ═══ THE CLAIM MATCHED NOTHING, AND THAT IS NOT ALWAYS SOMEBODY ELSE
1065
+ HOLDING THIS RUN, SO THE FAILURE IS STILL WRITTEN. ═══
1066
+ Six predicates can refuse a retry and two of them mean NOTHING HAS
1067
+ MOVED AT ALL. `pid is null` is the ordinary one, and not exotic:
1068
+ `recordProcess` is caught and carried on from at all four spawn sites
1069
+ precisely because that write is expected to be able to fail, and when
1070
+ it does, the process still ran and still died with a reason nobody
1071
+ else has. `attempts` reaching the bound between the read above and the
1072
+ claim is the other. RETURNING HERE WOULD LEAVE THE RUN `running` WITH
1073
+ A DEAD PROCESS AND THE CARD STILL SAYING WORKING, and the sentence a
1074
+ person eventually read would be the next sweep's guess about a daemon
1075
+ that never recorded a pid rather than the harness's own words.
1076
+
1077
+ SO IT FALLS THROUGH, AND THE ENDING IS THE ONE THIS PATH HAD BEFORE
1078
+ THE RETRY EXISTED. A run that ended or stopped under this daemon is
1079
+ refused by `panel3_end_run` and `panel3_give_up` on their own
1080
+ predicates, and that refusal is said below rather than recorded. A run
1081
+ another machine has taken over is not: neither ending is fenced by the
1082
+ machine or the pid, so this daemon ends it, exactly as it did before
1083
+ this slice. That is a property of the ending rather than of the retry,
1084
+ it is narrower than it was (the claim is fenced, so no second PROCESS
1085
+ starts), and closing it is a change to what a give-up may write. */
1086
+ said(`run ${runId} could not be started again, so it is ending on: ${answer.reason}`);
1087
+ }
1088
+ /* ═══ AND WHEN THE BOUND IS SPENT THE CARD FAILS AND SAYS HOW MANY
1089
+ ATTEMPTS IT TOOK. ═══ ux.md asks for that in as many words. It is said
1090
+ once, here, into the `failed_because` column the panel and the terminal
1091
+ both already read, so the count reaches a person from one place rather
1092
+ than two. */
1093
+ const why = attempts > 1 ? `${answer.reason} (after ${attempts} attempts)` : answer.reason;
1094
+ const ended = processToken === undefined
1095
+ ? await endRun(client, level, runId, cardId, why)
1096
+ : (await giveUp(client, runId, why, processToken)) !== null;
1097
+ if (!ended) {
1098
+ /* IT WAS ALREADY SETTLED, by recovery, which decided this process was
1099
+ gone before it said so itself, or by the person's Stop. Nothing was
1100
+ written here, and that is said rather than reported as a failure this
1101
+ daemon recorded. */
1102
+ said(`run ${runId} had already ended, so nothing was recorded: ${why}`);
1103
+ return;
1104
+ }
1105
+ out(level === 1
1106
+ ? `failed card ${cardId} run ${runId} ${why}`
1107
+ : `failed run ${runId} ${why}`);
1108
+ return;
1109
+ }
1110
+ /* ═══ ITS OWN WORDS, ONTO THE CARD, THROUGH THE SAME STATEMENT EVERY LEVEL
1111
+ USES. ═══ Nothing reads them, nothing shortens them and nothing waits to
1112
+ approve them: ux.md's "whoever did the work writes the answer". */
1113
+ const accepted = await writeAnswer(client, runId, cardId, speaksToTheCard ? answer.text : null, processToken);
1114
+ if (accepted)
1115
+ await ownerSession?.established();
1116
+ else
1117
+ await ownerSession?.failed();
1118
+ });
1119
+ }
1120
+ /**
1121
+ * ═══ ONE RUN, STARTED AGAIN AS ITSELF, WITH WHAT IT WAS SENT AND WHAT IT HAD
1122
+ * GOT AS FAR AS. ═══
1123
+ *
1124
+ * ux.md: "a respawned agent reads three things: its brief, its own report, and
1125
+ * whatever is new", and all three arrive in the prompt because "an agent cannot
1126
+ * fail to read its own input".
1127
+ *
1128
+ * ═══ THE SAME ROW, WHICH IS THE PART THAT MATTERS. ═══ Its children point at
1129
+ * this id. A new row would be a run whose children belong to a dead parent, and
1130
+ * the agent would dispatch them again — the failure ux.md names as the single
1131
+ * most expensive one observed. Nothing is inserted here for the same reason
1132
+ * nothing needs to be: constraint 8 is satisfied by a row that already exists.
1133
+ *
1134
+ * ═══ TWO CALLERS, AND `afterPid` IS THE WHOLE OF WHAT SEPARATES THEM. ═══
1135
+ *
1136
+ * NULL is the sweep, which INFERS that a process is gone from a pid the
1137
+ * operating system no longer knows: `recoverStranded` and `takeHandedBack`. It
1138
+ * behaves exactly as it always has, level 1 refused and the claim window
1139
+ * enforced, and it does not spend an attempt, because a laptop that slept is not
1140
+ * a provider that failed.
1141
+ *
1142
+ * A PID is the retry, made by the daemon that WATCHED that process exit. It may
1143
+ * resume any level, it is fenced by the pid rather than by the window, and the
1144
+ * claim counts the attempt and refuses past three. See 20260909090000 for why
1145
+ * each of those three predicates is about the sweep and not about a failure this
1146
+ * daemon saw with its own eyes.
1147
+ *
1148
+ * recovery-1/ux.md is why there is one function and not two: "two features
1149
+ * owning one respawn is how two respawns are born."
1150
+ *
1151
+ * ═══ AND THE ORDER IS THE SAME ARGUMENT `startChild` MAKES, ON THE SWEEP'S
1152
+ * PATH. ═══ The working copy is resolved BEFORE the claim, so a machine that
1153
+ * has none refuses without having taken a run it cannot start; then the claim,
1154
+ * then the process, then the pid.
1155
+ *
1156
+ * ON THE RETRY'S PATH IT CANNOT BE, AND THAT IS `startRearmed`'s SITUATION
1157
+ * EXACTLY. A level 1 run works in an empty directory of the user's own and
1158
+ * anything deeper gets the working copy, and the level is not known until the
1159
+ * claim returns. So the resolution happens after the claim, and a failure there
1160
+ * ends the run with its reason rather than leaving a claimed run sitting
1161
+ * `running` with no process. The sweep's ordering is untouched, because its
1162
+ * argument still holds there.
1163
+ *
1164
+ * ═══ THE CHILDREN ARE READ AFTER THE CLAIM AND AS LATE AS POSSIBLE. ═══ They
1165
+ * are the volatile half: one of them may finish while this function is running,
1166
+ * and the prompt says out loud that the list is a moment old and is re-readable.
1167
+ * Carrying them in the report instead is the thing ux.md forbids outright.
1168
+ *
1169
+ * Returns null when the claim matched nothing, which is a fact about the record
1170
+ * rather than a failure: nothing was started here. On the sweep's path that
1171
+ * means another daemon got there first. On the retry's it means the same thing
1172
+ * said by the pid having moved, OR that this run has had its three processes and
1173
+ * the record refuses a fourth, and the caller is the one that knows what to say
1174
+ * about the failure it was holding.
1175
+ */
1176
+ async function resumeRun(client, tools, machineId, runId, afterPid) {
1177
+ /* ═══ THE SWEEP RESOLVES ITS WORKING COPY BEFORE THE CLAIM AND THE RETRY
1178
+ CANNOT, AND `afterPid` IS THE WHOLE OF WHAT DECIDES IT. ═══ Said once, here.
1179
+ See the header for both halves of the argument: a machine with no checkout
1180
+ must not take a run it cannot start, and the level a retry needs the answer
1181
+ for is not known until the claim returns. */
1182
+ const sweptCwd = afterPid === null
1183
+ ? checkoutForCodebase(await codebaseOfRun(client, runId), hostname())
1184
+ : null;
1185
+ const { data, error } = await client.rpc('panel3_resume', {
1186
+ p_run_id: runId,
1187
+ p_machine_id: machineId,
1188
+ p_after_pid: afterPid,
1189
+ });
1190
+ if (error)
1191
+ throw new Error(`could not start run ${runId} again: ${error.message}`);
1192
+ const claimed = data?.[0];
1193
+ if (!claimed)
1194
+ return null;
1195
+ const level = claimed.run_level === 1 ? 1 : claimed.run_level === 2 ? 2 : 3;
1196
+ if (claimed.run_level !== level) {
1197
+ /* UNREACHABLE, AND STILL SETTLED, exactly as in `startChild`: the level
1198
+ column is checked at three, so there is no level here this cannot spawn.
1199
+ The claim has already happened, so leaving it would strand the run for a
1200
+ whole grace window before anything looked at it again. */
1201
+ const why = `run ${runId} is at level ${claimed.run_level}, which cannot be spawned`;
1202
+ await giveUp(client, runId, why);
1203
+ throw new Error(why);
1204
+ }
1205
+ /* NON-NULL EXACTLY WHEN THE SWEEP RESOLVED ONE, which is the rule stated at
1206
+ the assignment above. Branching on the VALUE rather than testing `afterPid`
1207
+ a second time is what keeps the resolution and its use from being able to
1208
+ disagree: whatever the sweep resolved is what the sweep runs in. */
1209
+ let cwd;
1210
+ if (sweptCwd !== null) {
1211
+ cwd = sweptCwd;
1212
+ }
1213
+ else {
1214
+ try {
1215
+ cwd = await workingDirectory(client, runId, level, false);
1216
+ }
1217
+ catch (error) {
1218
+ /* ═══ THE REASON IS WRITTEN TO THE DATABASE, SO IT MAY NOT CARRY A PATH.
1219
+ ═══ Constraint 6, and `startRearmed`'s own handling of the same two
1220
+ calls: `workingCopy()` is already careful about this and says so;
1221
+ `scratchDir()` is not, because it is `mkdirSync`, whose EACCES and
1222
+ ENOTDIR messages name the directory they failed on. So the machine's own
1223
+ error is kept for stderr and the record is told only what is true and
1224
+ shareable. */
1225
+ const stderrOnly = error instanceof Error ? error.message : String(error);
1226
+ const why = level === 1
1227
+ ? 'this machine could not make the empty directory this runs in'
1228
+ : stderrOnly;
1229
+ /* ═══ AND IT ENDS THE WAY A RUN OF THIS LEVEL ENDS. ═══ It was
1230
+ `panel3_give_up` outright, which was right while only a dispatched run
1231
+ could reach this function and is a leak now that the retry brings level
1232
+ 1 here: handing the person's message back mints a new run with its
1233
+ attempts at one, which is the bound the retry is under, undone by the
1234
+ one path that could not start. See `endRun`. */
1235
+ await endRun(client, level, runId, claimed.run_card_id, why);
1236
+ throw new Error(`NO AGENT IS RUNNING: ${level === 1 ? stderrOnly : why}`);
1237
+ }
1238
+ }
1239
+ /* WHAT IT SENT OTHERS TO DO, FROM THE RECORD. Level 3 has no `dispatch`, so it
1240
+ has no children to have: null says that, where an empty list would say it
1241
+ chose to send nobody.
1242
+
1243
+ ═══ AND LEVEL 1 HAS THEM TOO, WHICH ONLY THE RETRY CAN REACH. ═══ This read
1244
+ used to be `level === 2`, which was correct only because nothing at level 1
1245
+ ever got here. Left alone it would hand a coordinator a null child list, and
1246
+ a coordinator that came back to no children dispatches its workers a second
1247
+ time: ux.md's single most expensive failure. `startRearmed` reads it the
1248
+ same way, for the same reason. */
1249
+ const children = level === 3 ? null : await childrenOf(client, runId);
1250
+ const started = startAgent(
1251
+ /* ═══ WHY IT DIED IS WHAT DIFFERS, AND IT IS TOLD THE TRUTH ABOUT IT. ═══
1252
+ `resumePrompt` opens by saying the machine went down, which is true of the
1253
+ sweep and false of a retry: the daemon that watched this harness exit is
1254
+ still running. See `retryPrompt`. */
1255
+ afterPid === null
1256
+ ? resumePrompt(claimed.run_brief, claimed.run_report, children)
1257
+ : retryPrompt(claimed.run_brief, claimed.run_report, children), level, tools.urlFor(runId), cwd);
1258
+ if (started.pid === null) {
1259
+ /* THE CLAIM HAPPENED AND NO PROCESS DID, which is the one shape the record
1260
+ must never be left in quietly. Same handling as a dispatch that could not
1261
+ start: the reason is read off the settled answer and the run is ended with
1262
+ it. */
1263
+ const answer = await started.answered;
1264
+ const reason = answer.ok ? 'the process ended before it could be identified' : answer.reason;
1265
+ // The same level fork, for the same reason. See `endRun`.
1266
+ await endRun(client, level, runId, claimed.run_card_id, reason);
1267
+ throw new Error(`NO AGENT IS RUNNING: ${reason}`);
1268
+ }
1269
+ out(`resume run ${runId} level ${level} pid ${started.pid} `
1270
+ + `${claimed.run_report === null ? 'no report to carry' : 'carrying its report'}`);
1271
+ try {
1272
+ await recordProcess(client, runId, started.pid);
1273
+ }
1274
+ catch (error) {
1275
+ // Said, not fatal, as everywhere else: the agent is running and killing it
1276
+ // over a bookkeeping write would cost the user the work a second time.
1277
+ said(`${error instanceof Error ? error.message : String(error)} (the run is still going)`);
1278
+ }
1279
+ return { settled: settle(client, tools, machineId, level, runId, claimed.run_card_id, started) };
1280
+ }
1281
+ /**
1282
+ * ═══ ONE RUN, STARTED AGAIN BECAUSE SOMETHING IT WAS WAITING ON EXISTS NOW. ═══
1283
+ *
1284
+ * The other half of `resumeRun`, and deliberately the same shape: same row, same
1285
+ * id, same brief, the report carried, the children read live. Two facts differ,
1286
+ * and both come off the row `panel3_take_rearms` handed over.
1287
+ *
1288
+ * WHAT IS NEW. The answer to this run's own question, a question from somebody
1289
+ * it sent that it has to answer or pass on, or everybody it sent having
1290
+ * finished. `ask_id` and `mine` say which, and they are the record's word
1291
+ * rather than an inference here.
1292
+ *
1293
+ * WHERE IT RUNS. A level 1 run is started in an empty directory of the user's
1294
+ * own, because level 1 has no code tool to use a real one with; anything deeper
1295
+ * gets the working copy. That is `answerCard` and `startChild`'s rule, and this
1296
+ * is the one path that can be handed either.
1297
+ *
1298
+ * ═══ THE CLAIM HAS ALREADY HAPPENED WHEN THIS IS CALLED, SO A FAILURE ENDS THE
1299
+ * RUN RATHER THAN LEAVING IT. ═══ `panel3_take_rearms` stamps the claim in the
1300
+ * same statement that makes it — the question delivered, and the run's end
1301
+ * cleared — which is what stops two daemons starting two processes for it and
1302
+ * what stops one question, or one set of finished workers, spawning forever. The cost is that this cannot decline afterwards: a run claimed and not
1303
+ * started would sit `running` with no process and its question already spent. So
1304
+ * every failure below ends the run with its reason, which is visible on the card
1305
+ * and in `cs3 show`, rather than quietly hoping the next poll finds it.
1306
+ */
1307
+ async function startRearmed(client, tools, machineId, row) {
1308
+ const level = row.run_level === 1 ? 1 : row.run_level === 2 ? 2 : 3;
1309
+ if (row.run_level !== level) {
1310
+ const why = `run ${row.run_id} is at level ${row.run_level}, which cannot be spawned`;
1311
+ await giveUp(client, row.run_id, why);
1312
+ throw new Error(why);
1313
+ }
1314
+ let cwd;
1315
+ try {
1316
+ cwd = await workingDirectory(client, row.run_id, level, false);
1317
+ }
1318
+ catch (error) {
1319
+ /* ═══ THE REASON IS WRITTEN TO THE DATABASE, SO IT MAY NOT CARRY A PATH.
1320
+ ═══ Constraint 6. `workingCopy()` is already careful about this and says
1321
+ so; `scratchDir()` is not — it is `mkdirSync`, whose EACCES and ENOTDIR
1322
+ messages name the directory they failed on — and `failed_because` is a
1323
+ column `cs3 show` prints. So the machine's own error is kept for stderr
1324
+ and the record is told only what is true and shareable. */
1325
+ const said = error instanceof Error ? error.message : String(error);
1326
+ const why = level === 1
1327
+ ? 'this machine could not make the empty directory this runs in'
1328
+ : said;
1329
+ // The level fork, which this path needs for the same reason `resumeRun`'s
1330
+ // two do: a re-arm serves level 1, and a level 1 run holds the person's
1331
+ // message. See `endRun`.
1332
+ await endRun(client, level, row.run_id, row.run_card_id, why);
1333
+ throw new Error(`NO AGENT IS RUNNING: ${level === 1 ? said : why}`);
1334
+ }
1335
+ /* WHAT IT SENT OTHERS TO DO AND WHAT THEY WROTE, FROM THE RECORD, AS LATE AS
1336
+ POSSIBLE. Level 3 has no `dispatch`, so it has no children to have: null
1337
+ says that, where an empty list would say it chose to send nobody. */
1338
+ const children = level === 3 ? null : await childrenOf(client, row.run_id);
1339
+ const deliveredArtifact = row.ask_id !== null && row.mine
1340
+ ? await rearmedArtifactAnswer(client, row.ask_id)
1341
+ : null;
1342
+ /* ═══ THREE REASONS, AND THE ROW SAYS WHICH. ═══ No question is ux.md's third
1343
+ re-arm: everybody it sent has finished, and it is started to read them back.
1344
+ `children` cannot be null on that path — only a run with children is ever
1345
+ claimed for it — and the prompt takes the list rather than the maybe-list so
1346
+ that is a fact of the signature rather than of a comment. */
1347
+ const prompt = row.ask_id === null
1348
+ ? readBackPrompt(row.run_brief, row.run_report, children ?? [])
1349
+ : row.mine
1350
+ ? answerPrompt(row.run_brief, row.run_report, children, row.question ?? '', row.answer ?? '', deliveredArtifact)
1351
+ : escalationPrompt(row.run_brief, row.run_report, children, row.ask_id, row.question ?? '');
1352
+ const started = startAgent(prompt, level, tools.urlFor(row.run_id), cwd);
1353
+ if (started.pid === null) {
1354
+ const answer = await started.answered;
1355
+ const reason = answer.ok ? 'the process ended before it could be identified' : answer.reason;
1356
+ // The same level fork, for the same reason. See `endRun`.
1357
+ await endRun(client, level, row.run_id, row.run_card_id, reason);
1358
+ throw new Error(`NO AGENT IS RUNNING: ${reason}`);
1359
+ }
1360
+ out(`rearm run ${row.run_id} level ${level} pid ${started.pid} `
1361
+ + `${row.ask_id === null
1362
+ ? 'to read back everybody it sent'
1363
+ : row.mine
1364
+ ? 'with the answer to its own question'
1365
+ : 'with a question it has to settle'}`);
1366
+ try {
1367
+ await recordProcess(client, row.run_id, started.pid);
1368
+ }
1369
+ catch (error) {
1370
+ // Said, not fatal, as everywhere else: the agent is running and killing it
1371
+ // over a bookkeeping write would cost the user the work.
1372
+ said(`${error instanceof Error ? error.message : String(error)} (the run is still going)`);
1373
+ }
1374
+ /* ═══ AND IT SPEAKS TO THE CARD ONLY IF IT IS DOING THE PERSON'S WORK. ═══
1375
+ Exactly the distinction ux.md draws: a run carrying on with its own work has
1376
+ something to say when it finishes, and a run started only to settle somebody
1377
+ else's question does not. A run started to read back everybody it sent is
1378
+ the first kind and the clearest case of it — that reply IS the answer to the
1379
+ request. See `settle`. */
1380
+ return {
1381
+ settled: settle(client, tools, machineId, level, row.run_id, row.run_card_id, started, row.ask_id === null || !!row.mine),
1382
+ };
1383
+ }
1384
+ /**
1385
+ * The runs one run dispatched, AND WHAT EACH OF THEM WROTE, in the words the
1386
+ * prompt prints them in. The same question `list_child_runs` and
1387
+ * `get_child_report` answer, asked here so the answer is IN the prompt rather
1388
+ * than waiting for the agent to think of asking.
1389
+ *
1390
+ * ═══ THE REPORTS ARE HERE BECAUSE OF WHAT THIS IS FOR. ═══ ux.md: "the level 2
1391
+ * agent does not hold its three workers' findings in its head while they run.
1392
+ * Each worker writes what it found into the record, and the level 2 agent reads
1393
+ * them back." The respawn that exists to do exactly that is fired by the record
1394
+ * and given the record; leaving the reports behind a tool call would make the
1395
+ * whole synthesis conditional on an agent choosing to make it, which is the
1396
+ * difference ux.md draws between a guarantee and a hope.
1397
+ *
1398
+ * AND THEY ARE NOT TRUNCATED. A report is rewritten state rather than an append
1399
+ * log, which is what keeps it bounded; cutting one here would hand the agent a
1400
+ * half-sentence and no way to tell it was cut.
1401
+ */
1402
+ async function childrenOf(client, runId) {
1403
+ const children = await returned(client
1404
+ .from('panel3_runs')
1405
+ .select('id, level, state, activity, report, failed_because, ended_at')
1406
+ .eq('parent_run_id', runId)
1407
+ .order('started_at'), 'read', `what run ${runId} sent others to do`);
1408
+ return children.flatMap((c) => [
1409
+ [
1410
+ c.id,
1411
+ `L${c.level}`,
1412
+ c.state === 'running' && !c.ended_at ? 'still going' : c.state,
1413
+ c.activity ?? '',
1414
+ ].join(' ').trimEnd(),
1415
+ /* ONE OWNER PER COLUMN, PRINTED AS TWO FACTS. `report` is what the run
1416
+ wrote and `failed_because` is why the product ended it, and running them
1417
+ together would have an agent read the daemon's sentence as its worker's
1418
+ finding. */
1419
+ ...(c.failed_because === null ? [] : [` it was ended: ${c.failed_because}`]),
1420
+ ...(c.report === null
1421
+ ? [' it wrote nothing down']
1422
+ : c.report.split('\n').map((l) => ` ${l}`)),
1423
+ ]);
1424
+ }
1425
+ /** Private owner context: enough to continue or stop work, without hierarchy or activity text. */
1426
+ async function ownerChildren(client, runId) {
1427
+ const children = await returned(client.from('panel3_runs').select('id, state, report, failed_because')
1428
+ .eq('parent_run_id', runId).order('started_at'), 'read', `what owner ${runId} sent others to do`);
1429
+ return children.map((child) => [
1430
+ `${child.id} ${child.state}`,
1431
+ ...(child.failed_because ? [`ended: ${child.failed_because}`] : []),
1432
+ child.report ?? 'no report',
1433
+ ].join('\n'));
1434
+ }
1435
+ async function ownerConversation(client, cardId, leased, initialLeaseRunId) {
1436
+ const [turns, rawAsks] = await Promise.all([
1437
+ returned(client.from('panel3_turns').select('id, role, body, created_at, lease_id')
1438
+ .eq('card_id', cardId).order('created_at'), 'read', `the visible conversation on card ${cardId}`),
1439
+ returned(client.from('panel3_asks').select(`id, created_at, ${ASK_CONTENT_COLUMNS}`)
1440
+ .eq('card_id', cardId).not('decision_id', 'is', null).order('created_at'), 'read', `the visible questions on card ${cardId}`),
1441
+ ]);
1442
+ const asks = await withAskContent(client, rawAsks);
1443
+ const events = [
1444
+ ...turns.map((turn) => ({
1445
+ at: turn.created_at,
1446
+ kind: turn.role,
1447
+ id: turn.id,
1448
+ body: turn.body,
1449
+ needsReply: turn.role === 'user'
1450
+ && (leased.has(turn.id) || turn.lease_id === initialLeaseRunId),
1451
+ })),
1452
+ ...asks.map((ask) => ({
1453
+ at: ask.created_at,
1454
+ kind: 'question',
1455
+ id: ask.id,
1456
+ body: ask.question ?? '(question unavailable)',
1457
+ answer: ask.answer,
1458
+ relatedArtifactId: ask.related_artifact_id,
1459
+ relatedArtifactRevision: ask.related_artifact_revision,
1460
+ artifactAnswer: artifactAnswer(ask.related_artifact_id, ask.related_artifact_revision, ask.selected_options)?.answer ?? null,
1461
+ })),
1462
+ ];
1463
+ return events.sort((a, b) => a.at.localeCompare(b.at) || a.id.localeCompare(b.id));
1464
+ }
1465
+ /** The first owner gets the same durable context as every later activation. */
1466
+ export async function initialOwnerPrompt(client, cardId, launcherRunId, brief) {
1467
+ const events = await ownerConversation(client, cardId, new Set(), launcherRunId);
1468
+ return ownerActivationPrompt(brief, null, events, [], null);
1469
+ }
1470
+ /** Only facts delivered by this activation enter an already-resumed native
1471
+ * conversation. Older visible turns are already in that conversation. */
1472
+ export function ownerContinuationPrompt(brief, report, events, children, delivered) {
1473
+ const messages = events.filter((event) => event.kind === 'user' && event.needsReply);
1474
+ return [
1475
+ 'CONTINUE THE CTRL+SPC CONVERSATION YOU ALREADY OWN.',
1476
+ 'Use only the new facts below as the new turn. Keep coordinating the whole card and speak as CTRL+SPC.',
1477
+ '',
1478
+ 'NEW USER MESSAGES',
1479
+ ...(messages.length === 0
1480
+ ? ['None.']
1481
+ : messages.flatMap((event) => [`Turn ${event.id}`, event.body])),
1482
+ ...(delivered === null ? [] : delivered.mine ? [
1483
+ '',
1484
+ 'ANSWER TO YOUR QUESTION',
1485
+ `Question ${delivered.id}: ${delivered.question}`,
1486
+ `Answer: ${delivered.answer ?? '(no answer text)'}`,
1487
+ ...presentedArtifactAnswerContext(delivered.artifactAnswer),
1488
+ ] : [
1489
+ '',
1490
+ 'A WORKER NEEDS YOUR DECISION',
1491
+ `Question ${delivered.id}: ${delivered.question}`,
1492
+ 'Answer it with answer_escalation if you can. If the person must decide, ask them directly.',
1493
+ ]),
1494
+ '',
1495
+ 'CURRENT PRIVATE WORKER SNAPSHOT — THIS REPLACES THE PREVIOUS SNAPSHOT',
1496
+ ...(children.length === 0 ? ['You have sent nobody.'] : children),
1497
+ '',
1498
+ 'YOUR CURRENT DURABLE REPORT',
1499
+ report ?? 'Nothing yet.',
1500
+ '',
1501
+ 'YOUR IMMUTABLE RESPONSIBILITY',
1502
+ brief,
1503
+ ].join('\n');
1504
+ }
1505
+ async function ownerCandidate(client, runId) {
1506
+ const candidates = await returned(client.from('panel3_runs')
1507
+ .select('id, card_id, codebase_id, machine_id, harness, state, ended_at, attempts, pid, started_at, resumed_at, process_token, handed_back_at, card:panel3_cards!panel3_runs_card_id_fkey(conversation_run_id, state)')
1508
+ .eq('id', runId), 'read', `conversation owner ${runId}`);
1509
+ const candidate = candidates[0];
1510
+ return candidate?.card?.conversation_run_id === candidate?.id ? candidate : null;
1511
+ }
1512
+ async function ownerDirectory(client, candidate) {
1513
+ return workingDirectory(client, candidate.id, 2, true, candidate.codebase_id === null ? null : undefined);
1514
+ }
1515
+ export function resumableOwnerSessionId(candidate, machineId, machineHarness) {
1516
+ const local = readOwnerSession(candidate.id);
1517
+ return candidate.machine_id === machineId
1518
+ && candidate.harness === machineHarness
1519
+ && local?.state === 'established'
1520
+ && local.harness === machineHarness
1521
+ && local.processToken === candidate.process_token
1522
+ ? local.nativeSessionId
1523
+ : undefined;
1524
+ }
1525
+ async function activateOwner(client, tools, machineId, runId, afterProcessToken = null, afterPid = null) {
1526
+ const candidate = await ownerCandidate(client, runId);
1527
+ if (!candidate)
1528
+ return null;
1529
+ const machineHarness = harness();
1530
+ const resumeSessionId = resumableOwnerSessionId(candidate, machineId, machineHarness);
1531
+ const cwd = await ownerDirectory(client, candidate);
1532
+ const { data, error } = await client.rpc('panel3_take_owner_activation', {
1533
+ p_run_id: runId,
1534
+ p_machine_id: machineId,
1535
+ p_agent: machineHarness,
1536
+ p_after_process_token: afterProcessToken,
1537
+ p_after_pid: afterPid,
1538
+ });
1539
+ if (error)
1540
+ throw new Error(`could not activate conversation owner ${runId}: ${error.message}`);
1541
+ const claimed = data?.[0];
1542
+ if (!claimed)
1543
+ return null;
1544
+ const [events, children] = await Promise.all([
1545
+ ownerConversation(client, claimed.run_card_id, new Set(claimed.turn_ids ?? [])),
1546
+ ownerChildren(client, runId),
1547
+ ]);
1548
+ const currentArtifactAnswer = deliveredArtifactAnswer(events, claimed);
1549
+ const delivered = claimed.ask_id === null ? null : {
1550
+ id: claimed.ask_id,
1551
+ question: claimed.question ?? '(question unavailable)',
1552
+ answer: claimed.answer,
1553
+ mine: claimed.mine === true,
1554
+ artifactAnswer: currentArtifactAnswer,
1555
+ };
1556
+ const prompt = resumeSessionId
1557
+ ? ownerContinuationPrompt(claimed.run_brief, claimed.run_report, events, children, delivered)
1558
+ : ownerActivationPrompt(claimed.run_brief, claimed.run_report, events, children, delivered !== null && !delivered.mine
1559
+ ? { id: delivered.id, question: delivered.question }
1560
+ : null, currentArtifactAnswer,
1561
+ /* ═══ A PROCESS OF ITS OWN ENDED BEFORE IT FINISHED. ═══ `afterPid` is the
1562
+ fact, and it is non-null on all three paths that follow one: a harness
1563
+ that crashed, a machine that went down, and now a person's correction.
1564
+ The sentences it adds say what to do and never why, because those three
1565
+ are not the same event and `prompt.ts` exists to stop an agent being
1566
+ told an untrue reason for its own restart. */
1567
+ afterPid !== null);
1568
+ const started = startAgent(prompt, 2, tools.urlFor(runId, claimed.process_token), cwd, { ownerId: runId, ...(resumeSessionId ? { resumeSessionId } : {}) });
1569
+ if (started.pid === null) {
1570
+ const answer = await started.answered;
1571
+ const reason = answer.ok ? 'the process ended before it could be identified' : answer.reason;
1572
+ await giveUp(client, runId, reason, claimed.process_token);
1573
+ throw new Error(`NO AGENT IS RUNNING: ${reason}`);
1574
+ }
1575
+ try {
1576
+ await recordProcess(client, runId, started.pid, undefined, claimed.process_token);
1577
+ }
1578
+ catch (error) {
1579
+ said(`${error instanceof Error ? error.message : String(error)} (the run is still going)`);
1580
+ }
1581
+ return {
1582
+ settled: settle(client, tools, machineId, 2, runId, claimed.run_card_id, started, true, claimed.process_token, ownerSessionLifecycle(started, runId, machineHarness, claimed.process_token, resumeSessionId)),
1583
+ };
1584
+ }
1585
+ /** The only owner rows the normal poll may try, including an explicit hand-back move. */
1586
+ export function ownerPollClaim(candidate) {
1587
+ if (candidate.state === 'running') {
1588
+ if (candidate.handed_back_at === null || candidate.process_token === null)
1589
+ return null;
1590
+ return { afterProcessToken: candidate.process_token, afterPid: null };
1591
+ }
1592
+ if (candidate.state === 'finished' || candidate.state === 'asked') {
1593
+ return { afterProcessToken: null, afterPid: null };
1594
+ }
1595
+ return null;
1596
+ }
1597
+ /**
1598
+ * ═══ THE PROCESS A PERSON'S CORRECTION CANNOT HAVE REACHED. ═══
1599
+ *
1600
+ * redirect-5/ux.md: stop ends work, and changing its direction mid-flight did
1601
+ * not exist. ux.md's mechanism is "redirect is stop plus respawn", and the
1602
+ * respawn half is already built: a process that dies is started again as itself
1603
+ * by `settle`, and the claim it goes through leases every unaddressed person
1604
+ * turn on its way. **So the only thing missing was the ending**, and this is it.
1605
+ *
1606
+ * ═══ IT IS A MIRROR OF THE RETRY CLAIM, AND MUST BE CHANGED WITH IT. ═══
1607
+ * `20260914090000`'s retry branch requires the token, `running`, `ended_at` null,
1608
+ * the pid, and `attempts < 3`. The kill is the ONE irreversible act in this file:
1609
+ * every other predicate here pre-filters an RPC that can refuse harmlessly, and
1610
+ * a claim refused AFTER a kill falls through `settle` to `giveUp`, which turns a
1611
+ * person's correction into a FAILED CARD. So every condition below is either
1612
+ * that mirror, or a product rule with a named cost.
1613
+ *
1614
+ * Two things are NOT here, because neither is a property of the row: whether
1615
+ * this run owns its card, and whether this daemon holds its process. Both are
1616
+ * the caller's, and `mine` is what makes the respawn possible at all.
1617
+ */
1618
+ export function redirectedProcess(candidate, machineId,
1619
+ /** The newest unaddressed person turn per card. MAX, never first-seen: a turn
1620
+ * left unaddressed while a question was open would otherwise pin its card
1621
+ * below `resumed_at` for good and nothing on it could ever redirect. */
1622
+ waiting, cardId, booted) {
1623
+ // The claim's own five, mirrored.
1624
+ if (candidate.state !== 'running' || candidate.ended_at !== null)
1625
+ return null;
1626
+ if (candidate.attempts >= MAX_ATTEMPTS)
1627
+ return null;
1628
+ // The person moved this card to another machine. Killing here makes `settle`
1629
+ // retry locally and the claim clears `handed_back_at`, silently reversing them.
1630
+ if (candidate.handed_back_at !== null)
1631
+ return null;
1632
+ // `mine` is not enough: a takeover moves the row while this daemon still holds
1633
+ // a promise for it, and the claim then refuses on the machine predicate.
1634
+ if (candidate.machine_id !== machineId)
1635
+ return null;
1636
+ /* ═══ A CARD THAT NEEDS THE PERSON IS NOT A CARD THEY ARE REDIRECTING. ═══
1637
+ The claim refuses to lease turns while a person's question is open, so a
1638
+ kill here would destroy the work and NOT deliver the correction. The card's
1639
+ state is written from a character-identical predicate by the
1640
+ `panel3_prepare_user_turn` trigger on every person message, so it is the
1641
+ same fact, one read earlier. */
1642
+ if (candidate.card_state !== 'working')
1643
+ return null;
1644
+ /* ═══ AND IT HAS TO BE A MESSAGE THIS PROCESS CANNOT HAVE SEEN. ═══ The rule
1645
+ `panel3_take_turns` already applies one level up. Measured against the
1646
+ attempt that is running NOW, which is also what stops a replacement being
1647
+ killed by the very message that caused it. */
1648
+ const said = waiting.get(cardId);
1649
+ if (said === undefined)
1650
+ return null;
1651
+ if (new Date(said).getTime() <= new Date(candidate.resumed_at ?? candidate.started_at).getTime()) {
1652
+ return null;
1653
+ }
1654
+ return runProcessIsAlive(candidate, booted) ? candidate.pid : null;
1655
+ }
1656
+ export function pendingOwnerSessionWithinGrace(candidate, now = Date.now()) {
1657
+ const local = readOwnerSession(candidate.id);
1658
+ return !!local
1659
+ && local.state === 'pending'
1660
+ && local.harness === candidate.harness
1661
+ && local.processToken === candidate.process_token
1662
+ && now - new Date(local.updatedAt).getTime() < OWNER_SESSION_GRACE_MS;
1663
+ }
1664
+ async function takeOwnerActivations(client, tools, machineId, mine, hold) {
1665
+ /* ═══ THE MESSAGES NOBODY HAS TAKEN, READ BEFORE THE RUNS, DELIBERATELY. ═══
1666
+ A claim landing between these two reads stamps a `resumed_at` NEWER than
1667
+ every turn it just leased, so the redirect below refuses and nothing is
1668
+ killed. In the other order the same claim leaves a stale `resumed_at` beside
1669
+ a turn that now reads unaddressed, and the daemon kills the replacement it
1670
+ has just started. One narrow indexed read (`panel3_turns_takeable_idx`),
1671
+ unconditional because that ordering is the whole point of it. */
1672
+ const outstanding = await returned(client.from('panel3_turns')
1673
+ .select('card_id, created_at')
1674
+ .eq('role', 'user')
1675
+ .is('addressed_at', null), 'read', 'the messages nobody has taken yet');
1676
+ const waiting = new Map();
1677
+ for (const turn of outstanding) {
1678
+ // THE NEWEST PER CARD. See `redirectedProcess`: first-wins would pin a card
1679
+ // below its own `resumed_at` for good.
1680
+ const held = waiting.get(turn.card_id);
1681
+ if (held === undefined || turn.created_at > held)
1682
+ waiting.set(turn.card_id, turn.created_at);
1683
+ }
1684
+ const candidates = await returned(client.from('panel3_runs')
1685
+ .select('id, card_id, codebase_id, machine_id, harness, state, ended_at, attempts, pid, started_at, resumed_at, process_token, handed_back_at, card:panel3_cards!panel3_runs_card_id_fkey(conversation_run_id, state)')
1686
+ .in('state', ['finished', 'asked', 'running']), 'read', 'conversation owners that may be ready');
1687
+ for (const candidate of candidates) {
1688
+ if (candidate.card?.conversation_run_id !== candidate.id)
1689
+ continue;
1690
+ /* ═══ A RUN THIS DAEMON IS HOLDING IS THE ONLY ONE A CORRECTION CAN
1691
+ REDIRECT. ═══ Not because of the claim, which is not made here at all, but
1692
+ because of what happens AFTER the kill: the respawn is the dying process's
1693
+ own `settle`, and only the daemon that started it has one. Kill anything
1694
+ else and there is a dead process, a row still reading `running` and a card
1695
+ still saying `working`, with nothing local to retry it. */
1696
+ if (mine.has(candidate.id)) {
1697
+ const pid = redirectedProcess({ ...candidate, card_state: candidate.card?.state ?? null }, machineId, waiting, candidate.card_id, bootedAt());
1698
+ if (pid !== null) {
1699
+ killTree({ pid, kill: (signal) => process.kill(pid, signal) });
1700
+ /* ═══ SIGNALLED, NOT KILLED, AND THE WORD IS THE POINT. ═══ `killTree`
1701
+ swallows a refused signal on both platforms, so saying "killed" would
1702
+ claim a death this daemon never observed. A process that survives is
1703
+ still running, still held, and its turn is still unaddressed and still
1704
+ newer than `resumed_at`, so the next poll signals it again — the same
1705
+ rule `killStopped` states two hundred lines below. */
1706
+ out(`redirect card ${candidate.card_id} run ${candidate.id} signalled pid ${pid}`);
1707
+ }
1708
+ continue;
1709
+ }
1710
+ if (pendingOwnerSessionWithinGrace(candidate))
1711
+ continue;
1712
+ if (candidate.handed_back_at !== null
1713
+ && candidate.machine_id === machineId
1714
+ && (runProcessIsAlive(candidate, bootedAt())
1715
+ || (candidate.pid === null
1716
+ && Date.now() - new Date(candidate.resumed_at ?? candidate.started_at).getTime() < PID_GRACE_MS)))
1717
+ continue;
1718
+ const claim = ownerPollClaim(candidate);
1719
+ if (!claim)
1720
+ continue;
1721
+ try {
1722
+ const activated = await activateOwner(client, tools, machineId, candidate.id, claim.afterProcessToken, claim.afterPid);
1723
+ if (activated)
1724
+ hold(candidate.id, activated.settled);
1725
+ }
1726
+ catch (error) {
1727
+ said(`could not activate owner ${candidate.id}: ${error instanceof Error ? error.message : String(error)}`);
1728
+ }
1729
+ }
1730
+ }
1731
+ // ---------------------------------------------------------------------------
1732
+ /**
1733
+ * ═══ THE TWO ENDS THE PERSON CAUSES, WHICH ARE THE TWO THIS DAEMON KILLS. ═══
1734
+ *
1735
+ * `stopped` is their Stop and `superseded` is the coordinator their next message
1736
+ * replaced. The record keeps them apart on purpose, because a stop is read
1737
+ * across the whole card and being replaced says nothing about anybody else's
1738
+ * work. HERE they are one thing and only one: a process nothing will ever read
1739
+ * from again.
1740
+ */
1741
+ const ENDED_BY_THE_PERSON = ['stopped', 'superseded'];
1742
+ /** When this machine came up. Anything that started before it did has no process
1743
+ * left, whatever a pid lookup says about that number today. */
1744
+ const bootedAt = () => Date.now() - uptime() * 1000;
1745
+ /**
1746
+ * ═══ IS THE PROCESS THIS RUN RECORDED STILL THERE. ═══
1747
+ *
1748
+ * `processIsAlive` is `kill(pid, 0)`, which answers "some process has this
1749
+ * number", not "the process I started has this number". `machine_id` is derived
1750
+ * from the hardware and survives a reboot, pids restart low, so after a restart
1751
+ * a run's recorded pid regularly matches something unrelated and live. A run
1752
+ * that started before this machine booted has no process left by definition,
1753
+ * whatever the number says.
1754
+ *
1755
+ * ONE ANSWER FOR BOTH READERS. Recovery asks it to decide whether to reap a run,
1756
+ * and the stop asks it to decide whether there is anything left to kill. The
1757
+ * second is the one where a wrong yes is worst, because it signals a pid this
1758
+ * machine no longer owns.
1759
+ */
1760
+ function runProcessIsAlive(run, booted) {
1761
+ if (run.pid === null)
1762
+ return false;
1763
+ // The attempt now running began at the resume, if there was one.
1764
+ if (new Date(run.resumed_at ?? run.started_at).getTime() < booted)
1765
+ return false;
1766
+ return processIsAlive(run.pid);
1767
+ }
1768
+ /** Reconcile only CTRL+SPC-owned local session state. Native Claude and Windows
1769
+ * Codex transcripts live in harness-owned locations and are deliberately not
1770
+ * touched here. A current owner row, live PID, in-flight process, or fresh
1771
+ * pid-null claim always defers cleanup. Stable state is removed only after the
1772
+ * owner row disappears, becomes terminal, or no longer owns its card. */
1773
+ export async function reconcileOwnerSessions(client, machineId, _machineHarness, inFlightOwnerIds = new Set()) {
1774
+ const mappingIds = listOwnerSessionIds();
1775
+ const homeIds = listPanel3CodexOwnerHomeIds();
1776
+ const all = [...new Set([...mappingIds, ...homeIds])];
1777
+ const invalid = all.filter((id) => !validSessionUuid(id));
1778
+ for (const id of invalid)
1779
+ removePanel3CodexOwnerHome(id);
1780
+ const ids = all.filter(validSessionUuid);
1781
+ if (ids.length === 0)
1782
+ return;
1783
+ const rows = await returned(client.from('panel3_runs')
1784
+ .select('id, machine_id, harness, state, pid, started_at, resumed_at, ended_at, process_token, card:panel3_cards!panel3_runs_card_id_fkey(conversation_run_id)')
1785
+ .in('id', ids), 'read', 'which local conversation sessions still have an owner');
1786
+ const byId = new Map(rows.map((row) => [row.id, row]));
1787
+ for (const id of ids) {
1788
+ if (inFlightOwnerIds.has(id))
1789
+ continue;
1790
+ const row = byId.get(id);
1791
+ const localProcessInUse = !!row
1792
+ && row.machine_id === machineId
1793
+ && ((row.pid !== null && processIsAlive(row.pid))
1794
+ || (row.state === 'running'
1795
+ && row.ended_at === null
1796
+ && row.pid === null
1797
+ && Date.now() - new Date(row.resumed_at ?? row.started_at).getTime() < PID_GRACE_MS));
1798
+ const rowStillOwnsConversation = !!row
1799
+ && row.card?.conversation_run_id === id
1800
+ && ['running', 'asked', 'finished'].includes(row.state);
1801
+ if (localProcessInUse || rowStillOwnsConversation)
1802
+ continue;
1803
+ removeOwnerSession(id);
1804
+ if (homeIds.includes(id))
1805
+ removePanel3CodexOwnerHome(id);
1806
+ }
1807
+ }
1808
+ /**
1809
+ * ═══ THE USER STOPPED IT, OR THEIR NEXT MESSAGE REPLACED IT, SO THE PROCESS
1810
+ * STOPS. ═══
1811
+ *
1812
+ * ux.md: "The user's Stop is absolute, and nothing else stops work."
1813
+ * `panel3_stop_card` is the whole of the decision, ending the runs and the card
1814
+ * in one statement, and this is the effect.
1815
+ *
1816
+ * ═══ AND `superseded` IS READ OFF THE SAME QUERY, NOT SWEPT SEPARATELY. ═══ A
1817
+ * coordinator the person's next message replaced has a process that will never
1818
+ * be read from again, exactly like a stopped one, and that is the ONE thing the
1819
+ * two states share here. Two sweeps would be two owners of "kill what is not
1820
+ * coming back", and the second would be the one that fell behind. THE RECORD IS THE INTENT AND THE
1821
+ * KILL IS THE EFFECT, in that order, so a machine that is offline when the
1822
+ * button is pressed still honours it on the poll after it comes back.
1823
+ *
1824
+ * ONLY THIS MACHINE'S RUNS, for the same reason recovery only sweeps its own: a
1825
+ * pid is a local fact, and no other machine can act on this one's.
1826
+ *
1827
+ * ═══ A PROCESS THAT IS ALREADY GONE IS NOT AN ERROR. ═══ It is the ordinary
1828
+ * case, being an agent that finished in the second before the button or a
1829
+ * daemon that was restarted since, and there is nothing to report about it.
1830
+ *
1831
+ * ═══ AND THE PID IS CLEARED, WHICH IS WHAT KEEPS THIS SET FROM BECOMING A
1832
+ * MINEFIELD. ═══ A stopped run is stopped forever, so without this its pid
1833
+ * would be re-examined on every poll for the life of the machine, and pids are
1834
+ * reused. Sooner or later that number belongs to something else that is alive
1835
+ * and innocent, and this would signal it. Cleared, the row means what it says:
1836
+ * this machine has no process for this run. It is the same `pid = null` a resume
1837
+ * writes, for the same reason.
1838
+ *
1839
+ * AND ONLY WHEN THAT IS TRUE. A refused signal leaves the pid where it is, so
1840
+ * the next poll tries again: clearing it after an EPERM would say this machine
1841
+ * has no process for a run whose agent is still working.
1842
+ */
1843
+ async function killStopped(client, machineId) {
1844
+ const stopped = await returned(client
1845
+ .from('panel3_runs')
1846
+ .select('id, card_id, pid, started_at, resumed_at')
1847
+ .eq('machine_id', machineId)
1848
+ .in('state', ENDED_BY_THE_PERSON)
1849
+ .not('pid', 'is', null), 'read', 'the runs on this machine that are not coming back');
1850
+ if (stopped.length === 0)
1851
+ return;
1852
+ const booted = bootedAt();
1853
+ for (const run of stopped) {
1854
+ if (runProcessIsAlive(run, booted)) {
1855
+ try {
1856
+ process.kill(run.pid);
1857
+ out(`stopped run ${run.id} card ${run.card_id} killed pid ${run.pid}`);
1858
+ }
1859
+ catch (error) {
1860
+ said(`could not stop pid ${run.pid} for run ${run.id}: ${error instanceof Error ? error.message : String(error)}`);
1861
+ }
1862
+ /* ═══ SIGNALLED IS NOT DEAD, SO THE PID STAYS ON THE ROW. ═══ `kill`
1863
+ returning means the signal was delivered, not that the process acted on
1864
+ it: an agent mid-write, or one ignoring SIGTERM, is still there and
1865
+ still holding the working copy. Clearing the pid now would take the row
1866
+ out of this read for good and it would never be signalled again. The
1867
+ next poll finds it either gone, and clears it below, or still alive, and
1868
+ signals it again. The same line covers a refused signal, where the
1869
+ process is certainly still there. */
1870
+ continue;
1871
+ }
1872
+ try {
1873
+ await returned(client
1874
+ .from('panel3_runs')
1875
+ .update({ pid: null })
1876
+ // STILL NOT COMING BACK. Nothing starts either of these states, so
1877
+ // this cannot lose a race, and saying so in the statement is what
1878
+ // keeps that true if something one day does.
1879
+ .eq('id', run.id)
1880
+ .in('state', ENDED_BY_THE_PERSON)
1881
+ .select('id'), 'clear the process id of', `run ${run.id}`);
1882
+ }
1883
+ catch (error) {
1884
+ // Said, not fatal. The kill has already happened; this is bookkeeping, and
1885
+ // the next poll finds the row again and tries once more.
1886
+ said(`${error instanceof Error ? error.message : String(error)} (it was stopped all the same)`);
1887
+ }
1888
+ }
1889
+ }
1890
+ /**
1891
+ * ═══ A RUN WHOSE PROCESS IS GONE GOES BACK TO TAKEABLE. ═══
1892
+ *
1893
+ * Only this machine's runs, because liveness is a local fact: no other machine
1894
+ * can honestly say whether this one's pid is still there, and guessing from a
1895
+ * timestamp is how a run that is working gets declared dead.
1896
+ *
1897
+ * A run with NO PID RECORDED counts as gone — there is no process to be alive —
1898
+ * but only once it is neither this process's own nor inside `PID_GRACE_MS` of
1899
+ * starting. `mine` covers this daemon; the grace window covers a SECOND daemon
1900
+ * on the same machine, whose `mine` set does not and cannot contain the first
1901
+ * one's runs. Without it, starting a second daemon reaped the first daemon's
1902
+ * live work: turns handed back and a run marked failed while its agent was
1903
+ * genuinely running, producing a duplicate answer.
1904
+ *
1905
+ * ═══ AND A PID THAT PREDATES THIS BOOT IS NOT THIS RUN'S PID, which is
1906
+ * `runProcessIsAlive`'s rule and not this function's. ═══ What it costs here
1907
+ * if it is ever weakened: after a restart a stranded run's pid regularly matches
1908
+ * something unrelated and live, so the card would be skipped on every sweep,
1909
+ * forever, which is the exact failure this function exists to make impossible.
1910
+ *
1911
+ * ═══ ENDING THE RUN AND HANDING ITS TURNS BACK IS ONE STATEMENT. ═══ It used to
1912
+ * be two round trips, turns first so that a failure between them left the run
1913
+ * still `running` for the next sweep to redo. That ordering was safe against a
1914
+ * failure and not against an ANSWER: in the gap the turns were takeable while
1915
+ * the run still read running, so a `panel3_answer` landing there was accepted
1916
+ * onto a card another daemon was already free to take. `panel3_give_up` closes
1917
+ * it, and the loser of that race is now decided by the database rather than by
1918
+ * which write happened to be in flight.
1919
+ *
1920
+ * IT CANNOT UNDO AN ANSWER ANY MORE. A run that has answered is `finished` in
1921
+ * the same statement that wrote the answer (`panel3_answer`), so it is never in
1922
+ * this sweep's result at all. Before that, a daemon killed between the two left
1923
+ * an answered card whose run still read running, and this function blanked
1924
+ * `addressed_at` on turns with their answer sitting beside them.
1925
+ *
1926
+ * ---------------------------------------------------------------------------
1927
+ * ═══ AND A DISPATCHED RUN IS NOT HANDED BACK. IT IS STARTED AGAIN. ═══
1928
+ *
1929
+ * The two kinds of run come back by different routes because their durable
1930
+ * state lives in different places, and this is the fork between them.
1931
+ *
1932
+ * A LEVEL 1 RUN'S WORK IS ITS TURNS. Handing them back is a complete recovery:
1933
+ * the next taker is sent the same thing, built from the same turns.
1934
+ *
1935
+ * A DISPATCHED RUN HOLDS NO TURNS. What it was sent to do and how far it got
1936
+ * are on its own row, so ending it throws both away — and the person's request
1937
+ * dies with a card reading `failed`. `panel3_resume` claims the row instead and
1938
+ * the same agent is started again with its brief, its report and its children.
1939
+ *
1940
+ * ═══ WHICH IS ALSO WHY THE GRACE WINDOW READS `resumed_at`. ═══ A claim nulls
1941
+ * the pid, so a freshly resumed run looks exactly like one whose daemon died
1942
+ * before recording a pid. The window has to be measured from the attempt that is
1943
+ * running now, not from the first one, or every resume would be reaped by the
1944
+ * next sweep a second after it started.
1945
+ */
1946
+ async function recoverStranded(client, tools, machineId, mine, hold) {
1947
+ const live = await returned(client
1948
+ .from('panel3_runs')
1949
+ .select('id, card_id, parent_run_id, state, pid, started_at, resumed_at, process_token, card:panel3_cards!panel3_runs_card_id_fkey(conversation_run_id)')
1950
+ .eq('machine_id', machineId)
1951
+ /* ═══ AND A RUN THAT STOPPED TO ASK IS SWEPT TOO, WHICH IS WHAT MAKES THE
1952
+ RE-ARM'S OWN PREDICATE SAFE. ═══ `panel3_take_rearms` will only start a run
1953
+ again once its last process is accounted for — `ended_at is not null` —
1954
+ so that a settle still in flight cannot land on the attempt that
1955
+ replaced it. The stamp arrives when the process exits; if the daemon
1956
+ dies first, nothing else would ever write it and the branch would wait
1957
+ forever on an answer that could never be delivered. This is where it
1958
+ arrives instead. The two predicates are one rule and may not be changed
1959
+ apart. */
1960
+ .in('state', ['running', 'asked'])
1961
+ .is('ended_at', null), 'read', "this machine's live runs");
1962
+ const booted = bootedAt();
1963
+ for (const run of live) {
1964
+ if (mine.has(run.id))
1965
+ continue;
1966
+ // WHEN THE ATTEMPT NOW RUNNING BEGAN, which is the first one until a resume
1967
+ // says otherwise. See the header.
1968
+ const startedAt = new Date(run.resumed_at ?? run.started_at).getTime();
1969
+ const rebooted = startedAt < booted;
1970
+ // A run too young to have recorded a pid is another daemon's live work. The
1971
+ // reused-pid check that used to sit beside this is `runProcessIsAlive`'s.
1972
+ if (!rebooted && run.pid === null && Date.now() - startedAt < PID_GRACE_MS)
1973
+ continue;
1974
+ if (runProcessIsAlive(run, booted))
1975
+ continue;
1976
+ const isOwner = run.card?.conversation_run_id === run.id && run.process_token !== null;
1977
+ if (isOwner) {
1978
+ try {
1979
+ if (run.state === 'asked') {
1980
+ await writeAnswer(client, run.id, run.card_id, null, run.process_token ?? undefined);
1981
+ }
1982
+ else {
1983
+ const resumed = await activateOwner(client, tools, machineId, run.id, run.process_token, run.pid);
1984
+ if (resumed)
1985
+ hold(run.id, resumed.settled);
1986
+ }
1987
+ }
1988
+ catch (error) {
1989
+ said(`could not recover owner ${run.id}: ${error instanceof Error ? error.message : String(error)}`);
1990
+ }
1991
+ continue;
1992
+ }
1993
+ if (run.state === 'asked') {
1994
+ /* ═══ ITS END IS STAMPED AND ITS TURNS ARE LEFT ALONE. ═══ This run is not
1995
+ handed back and is not started again from here: its continuation is the
1996
+ answer to what it asked, and `panel3_take_rearms` is what brings it back.
1997
+ So it must NOT go through `panel3_give_up`, which would hand back the
1998
+ turns a level 1 run took — putting the same message in front of a second
1999
+ coordinator while the person is still being asked about the first.
2000
+ What is missing is only the end stamp its process never got to write. */
2001
+ const ended = await failRun(client, run.id, 'the process that asked this question was gone before it could finish');
2002
+ out(ended
2003
+ ? `recover run ${run.id} card ${run.card_id} asked, and its process is gone: ended, `
2004
+ + 'and it comes back when its question is answered'
2005
+ : `recover run ${run.id} had already ended, left alone`);
2006
+ continue;
2007
+ }
2008
+ if (run.parent_run_id !== null) {
2009
+ /* STARTED AGAIN AS ITSELF, AND A FAILURE IS SAID RATHER THAN THROWN. What
2010
+ it must not do is take the sweep down mid-loop and leave every other
2011
+ stranded run unlooked-at.
2012
+
2013
+ WHAT IS LEFT BEHIND DEPENDS ON HOW FAR IT GOT, and each outcome is
2014
+ honest. A failure BEFORE the claim — this machine has no working copy —
2015
+ leaves the run exactly as it was, still `running` with a dead process,
2016
+ so the next sweep tries again: right for a checkout that is missing for
2017
+ a minute, and one line per poll for one that is missing for good. A run
2018
+ that was CLAIMED and could not be STARTED is ended inside `resumeRun`,
2019
+ because a claimed run with no process must not sit out a grace window
2020
+ pretending to be alive. A failure between the two — the read of its
2021
+ children — leaves it claimed, which costs it one window before the next
2022
+ sweep claims it again. */
2023
+ try {
2024
+ const resumed = await resumeRun(client, tools, machineId, run.id, null);
2025
+ if (resumed)
2026
+ hold(run.id, resumed.settled);
2027
+ else
2028
+ out(`resume run ${run.id} already claimed or started again, left alone`);
2029
+ }
2030
+ catch (error) {
2031
+ said(`could not start run ${run.id} again: ${error instanceof Error ? error.message : String(error)}`);
2032
+ }
2033
+ continue;
2034
+ }
2035
+ const back = await giveUp(client, run.id, rebooted
2036
+ ? 'this machine has restarted since the run started, so its process is gone'
2037
+ : run.pid === null
2038
+ ? 'the daemon that started this was gone before it recorded a process id'
2039
+ : `the process running this (pid ${run.pid}) is gone and it never ended`);
2040
+ if (back === null) {
2041
+ /* IT ANSWERED WHILE IT WAS BEING GIVEN UP. The two functions race for one
2042
+ run and `panel3_give_up` lost: the card is answered and complete, and
2043
+ nothing here has touched it. Said rather than silent, because a sweep
2044
+ that decided a run was dead and was wrong is worth seeing. */
2045
+ out(`recover run ${run.id} card ${run.card_id} answered as it was being given up, left alone`);
2046
+ continue;
2047
+ }
2048
+ out(`recover run ${run.id} card ${run.card_id} process gone, `
2049
+ + `${back} turn${back === 1 ? '' : 's'} takeable again`);
2050
+ }
2051
+ }
2052
+ /**
2053
+ * ═══ A RUN A PERSON HANDED BACK, TAKEN UP BY WHOEVER IS LISTENING. ═══
2054
+ *
2055
+ * The same act as `recoverStranded`'s dispatched branch, by the other route.
2056
+ * The sweep can only ever reach THIS machine's runs, because liveness is a local
2057
+ * fact and no machine can honestly say whether another one's pid is still there.
2058
+ * So a machine that is never coming back strands its dispatched runs forever:
2059
+ * nothing sweeps them, and `panel3_take_turns` supersedes level 1 only.
2060
+ *
2061
+ * `panel3_hand_back` is the person's answer to that, and it decided everything.
2062
+ * It ran under the card's own row lock, refused any run whose machine had
2063
+ * heartbeated inside the listening window, and stamped `handed_back_at` on the
2064
+ * dispatched ones. THIS IS ONLY THE DISCOVERY, which is the one thing that was
2065
+ * missing: the daemon never asked whether another machine's run was available.
2066
+ *
2067
+ * ═══ AND SO THE READ HAS NO `machine_id` FILTER, WHICH IS THE WHOLE POINT. ═══
2068
+ * RLS scopes it to the person's own account and nothing narrower, because the
2069
+ * machine that stamped the offer is by definition not the machine that should
2070
+ * serve it. `panel3_resume` has never had a machine predicate either, so the
2071
+ * claim was always cross-machine; only the looking was not.
2072
+ *
2073
+ * ---------------------------------------------------------------------------
2074
+ * ═══ WHICH OF THE SWEEP'S SAFETY CHECKS APPLY HERE, AND WHICH CANNOT. ═══
2075
+ *
2076
+ * The sweep refuses three things, and they are not decoration: they are what
2077
+ * stops a machine reaping its own live work. Taken one at a time, because
2078
+ * carrying them across without thinking and dropping them without saying are
2079
+ * the same mistake.
2080
+ *
2081
+ * `mine`, A RUN THIS DAEMON IS HOLDING RIGHT NOW: KEPT. Presence is coarser
2082
+ * than a pid, so a daemon that is alive but has missed the listening window,
2083
+ * meaning a closed lid, a paused VM or a network stall, has its runs offered
2084
+ * while its agents are genuinely working. When it comes back it finds its OWN
2085
+ * work on offer, and without this line it would claim it: `panel3_resume`
2086
+ * writes `pid = null` and a second process starts on the same run id and the
2087
+ * same working copy, while the first one's `settle` is still going to answer
2088
+ * the card.
2089
+ *
2090
+ * ═══ AND IT NARROWS THAT CLASS RATHER THAN CLOSING IT. ═══ A THIRD machine,
2091
+ * listening and healthy, can claim that same live run, because the guard the
2092
+ * database applied is presence and presence is what was wrong. That limit is
2093
+ * the migration header's own and is not this function's to fix; `mine` closes
2094
+ * one half that IS knowable here, which is a run THIS PROCESS is holding, and
2095
+ * the pid check below closes the other, which is a run ANOTHER PROCESS ON THIS
2096
+ * MACHINE is holding.
2097
+ *
2098
+ * `PID_GRACE_MS`, TOO YOUNG TO HAVE RECORDED A PID: KEPT, in its `pid is null`
2099
+ * half only, and it closes the same hole `mine` does one step earlier. A
2100
+ * FRESHLY DISPATCHED run is not in `inFlight` yet: `startChild` writes the row
2101
+ * through `panel3_dispatch` and the poll only calls `hold` once `startChild`
2102
+ * has RESOLVED, so in between the row reads `running`, `parent_run_id` is not
2103
+ * null, `pid` is null, and `mine` does not contain it. The tools server runs
2104
+ * off HTTP rather than off the poll, so a daemon stalled past the listening
2105
+ * window still serves a `dispatch` and still writes that row: the person
2106
+ * presses, the child is stamped, the stall ends, and this read would claim a
2107
+ * run whose own process is in the act of starting. Two agents, one run id, one
2108
+ * working copy, which is what `mine` is here to prevent.
2109
+ *
2110
+ * ═══ AND IT COSTS THE REAL CASE NOTHING. ═══ A genuinely stranded offer is by
2111
+ * construction older than the fifteen seconds the banner needed before a
2112
+ * person could even see it, so the window never fires for one.
2113
+ *
2114
+ * THE SWEEP'S OTHER HALF OF THAT LINE, `rebooted`, STAYS OUT. It is read off
2115
+ * this machine's uptime, and this is the one read that is not about this
2116
+ * machine's runs.
2117
+ *
2118
+ * `runProcessIsAlive`, THE PID PLUS A BOOT-TIME REFUSAL: APPLIED TO THIS
2119
+ * MACHINE'S OWN OFFERED RUNS, AND TO NOBODY ELSE'S. The two halves are
2120
+ * genuinely different questions and were once answered as one.
2121
+ *
2122
+ * ON ANOTHER MACHINE'S RUN IT WOULD BE A DEFECT. That pid belongs to a process
2123
+ * on a box this one cannot see, so `kill(pid, 0)` here asks about an unrelated
2124
+ * local process, and pids are reused: a collision would make this machine skip
2125
+ * a run nobody is working on, forever, which is the exact strand the feature
2126
+ * exists to end. What stands in for it there is the heartbeat guard already
2127
+ * spent inside `panel3_hand_back`, with the honest limit that migration's
2128
+ * header states.
2129
+ *
2130
+ * ON A RUN WHOSE `machine_id` IS THIS MACHINE IT IS EXACTLY AS TRUSTWORTHY AS
2131
+ * IT IS IN `recoverStranded`, and TWO DAEMONS ON ONE MACHINE IS NOT
2132
+ * HYPOTHETICAL: 20260903140000 exists to support one per harness, and
2133
+ * `PID_GRACE_MS`'s own header says so. Without this check, D1 holds a live
2134
+ * dispatched run with its pid recorded, the machine sleeps, both presence rows
2135
+ * go stale, a person hands the card back, the machine wakes and D2 polls
2136
+ * first. The sweep leaves the run alone, correctly, because the process really
2137
+ * is alive. This read would then claim the very same row: `mine` is D2's and
2138
+ * does not hold D1's work, the grace window does not fire because the pid is
2139
+ * there, and `panel3_resume` matches every predicate, because `machine_id`
2140
+ * moves from this machine to this machine. Two agents, one run id, one working
2141
+ * copy, and D1's answer is accepted afterwards by a `panel3_answer` fenced by
2142
+ * neither machine nor pid.
2143
+ *
2144
+ * ---------------------------------------------------------------------------
2145
+ * ═══ AND A FAILURE IS SAID RATHER THAN THROWN, AS IT IS IN THE SWEEP. ═══ The
2146
+ * ordinary one is a machine with no folder for the project: `resumeRun` resolves
2147
+ * the checkout BEFORE the claim, so such a machine never takes a run it cannot
2148
+ * start. It says why on its own stderr and leaves the offer standing for a
2149
+ * machine that has the codebase. That is not silence and it is not a success:
2150
+ * the run is exactly as it was, and the card goes on saying nothing is listening,
2151
+ * which is true until somebody who can serve it is.
2152
+ *
2153
+ * ═══ AND THAT HOLDS ONLY WHERE `CTRL_SPC_V3_WORKING_COPY` IS UNSET, WHICH IS
2154
+ * THIS TASK'S ONE REAL EXPOSURE. ═══
2155
+ *
2156
+ * `checkoutFor` resolves in three steps and the third is a machine-wide folder
2157
+ * named by that variable, which applies EVEN WHEN THE CARD HAS A PROJECT. So a
2158
+ * listening machine with it set resolves a checkout for any project at all,
2159
+ * passes the ordering guarantee above, claims the run and starts an agent in a
2160
+ * folder holding somebody else's code, which is the failure
2161
+ * `panel3-checkout.contract.test.mjs`'s own header names. Before this function
2162
+ * existed it was unreachable across machines, because nothing ever looked at
2163
+ * another machine's run; this read is what makes it reachable, and the
2164
+ * cross-machine walk in plan-slice-2.md section 8 is exactly the setup that sets
2165
+ * the variable, so that walk can produce a false pass.
2166
+ *
2167
+ * ═══ RECORDED, NOT FIXED HERE. ═══ Reordering that resolution is `../codebases-3`'s
2168
+ * assignment, and `checkoutFor` is shared by every spawn path with its own
2169
+ * contract suite: changing it from this slice would widen a fifty-line read into
2170
+ * a change to how every agent in the product finds its code.
2171
+ *
2172
+ * ═══ AND ONE OFFER NOBODY CAN CHECK OUT IS NOW RETRIED BY THE WHOLE FLEET. ═══
2173
+ * Nothing clears `handed_back_at` except a claim, so every listening machine
2174
+ * reads that run every poll and prints one line for it, forever, where
2175
+ * `recoverStranded`'s header accepts the same cost for one machine. Two reads and
2176
+ * a line per machine per poll is cheap and is the honest state of the record: the
2177
+ * work really is offered and really has no taker. Silencing it would mean an
2178
+ * expiry on the offer, which is a rule nothing has asked for and which would
2179
+ * throw the work away on a fleet that was merely all asleep.
2180
+ */
2181
+ export async function takeHandedBack(client, tools, machineId, mine, hold) {
2182
+ /* ITS OWN NARROW ROW, NOT `LiveRun`. `machine_id` is read for one question
2183
+ and one only: whether this row is THIS machine's, which is what decides
2184
+ whether its pid means anything here. See the header. */
2185
+ const offered = await returned(client
2186
+ .from('panel3_runs')
2187
+ .select('id, card_id, machine_id, pid, started_at, resumed_at, process_token, card:panel3_cards!panel3_runs_card_id_fkey(conversation_run_id)')
2188
+ .not('handed_back_at', 'is', null)
2189
+ /* WHAT THE OFFER MEANS, RESTATED IN THE READ. `panel3_hand_back` only ever
2190
+ stamps a dispatched run that is still going, and `panel3_resume` refuses
2191
+ anything else. Saying it here too costs one line and means a row that
2192
+ somehow carried a stale stamp is passed over rather than sent to a
2193
+ function that would refuse it a round trip later. */
2194
+ .eq('state', 'running')
2195
+ .is('ended_at', null)
2196
+ .not('parent_run_id', 'is', null), 'read', 'the runs a person has handed back');
2197
+ // ONCE, OUTSIDE THE LOOP. It is a property of this machine, not of a row.
2198
+ const booted = bootedAt();
2199
+ for (const run of offered) {
2200
+ if (mine.has(run.id)) {
2201
+ /* THIS DAEMON'S OWN LIVE WORK, OFFERED WHILE IT WAS BUSY BEING QUIET. See
2202
+ the header. Said rather than passed over in silence, because a machine
2203
+ whose work was handed back out from under it is the one thing here worth
2204
+ seeing in a terminal. */
2205
+ out(`offered run ${run.id} card ${run.card_id} is this daemon's own live work, left alone`);
2206
+ continue;
2207
+ }
2208
+ /* ═══ AND THE OTHER DAEMON ON THIS MACHINE IS HOLDING IT. ═══ `mine` covers
2209
+ this process and cannot cover its sibling: one machine runs one daemon per
2210
+ harness on purpose (20260903140000), and a machine that slept has both
2211
+ presence rows stale, so `panel3_hand_back` offers work that is still
2212
+ genuinely running. THE PID ANSWERS THAT, AND ONLY FOR THIS MACHINE'S OWN
2213
+ ROWS: for anybody else's it would be a question about an unrelated local
2214
+ process. `recoverStranded` asks it the same way on the same rows and
2215
+ correctly leaves them alone, so without this the sweep declines and the
2216
+ very next read claims. See the header. */
2217
+ if (run.machine_id === machineId && runProcessIsAlive(run, booted)) {
2218
+ out(`offered run ${run.id} card ${run.card_id} is still running on this machine, left alone`);
2219
+ continue;
2220
+ }
2221
+ /* ═══ AND A RUN TOO YOUNG TO HAVE RECORDED A PID IS SOMEBODY'S LIVE WORK
2222
+ TOO, INCLUDING THIS DAEMON'S OWN. ═══ `mine` cannot hold a dispatch
2223
+ between the row `panel3_dispatch` wrote and the `hold` that follows the
2224
+ spawn, and that gap is reachable from here: see the header. This is
2225
+ `recoverStranded`'s own line without its `rebooted` half, which reads this
2226
+ machine's uptime and has no meaning for another machine's run. */
2227
+ if (run.pid === null
2228
+ && Date.now() - new Date(run.resumed_at ?? run.started_at).getTime() < PID_GRACE_MS) {
2229
+ out(`offered run ${run.id} card ${run.card_id} is too young to have a process yet, left alone`);
2230
+ continue;
2231
+ }
2232
+ try {
2233
+ const resumed = run.card?.conversation_run_id === run.id && run.process_token !== null
2234
+ ? await activateOwner(client, tools, machineId, run.id, run.process_token, null)
2235
+ : await resumeRun(client, tools, machineId, run.id, null);
2236
+ if (resumed)
2237
+ hold(run.id, resumed.settled);
2238
+ else
2239
+ out(`resume run ${run.id} already claimed or started again, left alone`);
2240
+ }
2241
+ catch (error) {
2242
+ said(`could not take run ${run.id}, which was handed back: `
2243
+ + `${error instanceof Error ? error.message : String(error)}`);
2244
+ }
2245
+ }
2246
+ }
2247
+ // ---------------------------------------------------------------------------
2248
+ /**
2249
+ * The poll loop: it takes what is waiting and answers it, forever. `--once`
2250
+ * polls a single time and exits when the work it took is done, which is what
2251
+ * makes the loop testable — two of them against one card is how the lease is
2252
+ * proved.
2253
+ *
2254
+ * ═══ ONE LOOP, TWO CALLERS, AND THAT IS THE WHOLE OF recovery-1 SLICE 4. ═══
2255
+ * `startPanel` below is `cs start`'s entry and hands in the session the
2256
+ * installed CLI already holds. `cs3 run` is the acceptance harness's entry and
2257
+ * signs in from the environment against its own isolated account. Neither is a
2258
+ * copy of the other: a second poll loop would be two things leasing the same
2259
+ * turns with two ideas of what recovery may reap.
2260
+ *
2261
+ * @param injected the client to poll and serve tools through, when the caller
2262
+ * already has one. Its presence is also what says THIS PROCESS IS NOT v3's, so
2263
+ * the signals belong to somebody else — see the handler block below.
2264
+ */
2265
+ export async function run(args, injected) {
2266
+ let once = false;
2267
+ for (const arg of args) {
2268
+ if (arg === '--once')
2269
+ once = true;
2270
+ else
2271
+ throw new Error(`unknown option ${arg}. ${USAGE}`);
2272
+ }
2273
+ /* ═══ THE ONE v3 CLIENT THAT OUTLIVES ITS ACCESS TOKEN, SO IT IS THE ONE THAT
2274
+ REFRESHES. ═══ This client polls for as long as the daemon runs AND is the
2275
+ client every tool call is served through, so past the token's TTL both stop
2276
+ working at once. See `client.ts` for why refreshing is safe here:
2277
+ `persistSession` stays false, so the rotated token lives in this process and
2278
+ no file on the machine is written.
2279
+
2280
+ ═══ AND UNDER `cs start` THERE IS NO SIGN-IN AT ALL, WHICH IS THE POINT. ═══
2281
+ The installed daemon's own client is handed in: it refreshes, it retries at
2282
+ the network layer, and it is the ONLY client in that process, so the rotated
2283
+ token has exactly one writer. Signing in again there would be the second
2284
+ refresh loop `client.ts`'s header exists to prevent, this time inside one
2285
+ process rather than across two. */
2286
+ const client = injected ?? await signedInClient(true);
2287
+ const machineId = getMachineIdentity().id;
2288
+ /* THE NAME AND THE HARNESS, RESOLVED ONCE, AT STARTUP, RATHER THAN PER POLL.
2289
+ Both are properties of this machine for the life of this process: the
2290
+ hostname does not change under it, and neither does `CTRL_SPC_V3_AGENT`.
2291
+ Resolving `harness()` here means a machine misconfigured with a name this
2292
+ build cannot spawn (constraint 7: reported, never chosen) fails before it
2293
+ ever says it is listening, rather than on its first poll. */
2294
+ const machineName = hostname();
2295
+ const machineHarness = harness();
2296
+ /* THE RUNS THIS PROCESS IS HOLDING RIGHT NOW, so recovery cannot declare its
2297
+ own live work dead in the moment before a pid is recorded. It covers THIS
2298
+ daemon only, which is why `PID_GRACE_MS` exists for the other ones. Keyed by
2299
+ run id, at every level: a dispatched agent is this daemon's to hold too, and
2300
+ leaving it out would have recovery reap the children of its own runs. */
2301
+ const inFlight = new Map();
2302
+ /** One dispatched run's promise, held where the daemon can wait on it and
2303
+ * recovery can see it. The same bookkeeping the poll does for a level 1 run,
2304
+ * in the one other place a process is started. */
2305
+ const hold = (runId, work) => {
2306
+ inFlight.set(runId, work
2307
+ .catch((error) => {
2308
+ said(`run ${runId}: ${error instanceof Error ? error.message : String(error)}`);
2309
+ })
2310
+ .finally(() => { inFlight.delete(runId); }));
2311
+ };
2312
+ /* ═══ THE TOOLS BEFORE THE FIRST TAKE, OR NO TAKES AT ALL. ═══ A daemon that
2313
+ took a card and then found it could not serve tools would spawn an agent
2314
+ with no hands and write its confusion to the card as an answer. It is one
2315
+ loopback listener on an ephemeral port; failing here is a daemon that never
2316
+ started, which is the honest outcome.
2317
+
2318
+ `tools` is referenced inside the callback it is being given, which is safe
2319
+ for the plain reason that the callback can only run once a request has
2320
+ arrived at a server that by then exists. */
2321
+ const tools = await startToolsServer(client, async (parentRunId, brief, codebase, processToken) => {
2322
+ const child = await startChild(client, tools, machineId, parentRunId, brief, codebase, processToken);
2323
+ hold(child.runId, child.settled);
2324
+ return { runId: child.runId };
2325
+ });
2326
+ out(`daemon machine ${machineId}`);
2327
+ out(`tools ${tools.urlFor('<run-id>')}`);
2328
+ out(once ? 'mode one poll' : `mode polling every ${POLL_INTERVAL_MS / 1000}s, Ctrl-C to stop`);
2329
+ /* ═══ AND THIS MACHINE STOPS SAYING IT IS LISTENING WHEN IT GOES. ═══ ux.md
2330
+ forbids a card reading `working` while nothing is working, and the freshness
2331
+ window is a backstop for the machine that CANNOT say goodbye — a crash, a
2332
+ lost network, a kill -9. A Ctrl-C can, so it does, and the panel tells the
2333
+ truth within a poll rather than within a window.
2334
+
2335
+ `once` never gets here: it returns below, and the same call is made there.
2336
+
2337
+ ═══ AND NOT WHEN A CLIENT WAS HANDED IN, BECAUSE THEN THE PROCESS IS NOT
2338
+ THIS LOOP'S TO EXIT. ═══ Under `cs start` this shares a process with the v2
2339
+ presence heartbeat, which has a shutdown of its own that marks
2340
+ `cliv2_agents` offline and then calls `process.exit`. Two handlers means two
2341
+ exits, and whichever finishes its write first kills the other's mid-flight:
2342
+ a Ctrl-C would leave the machine reading listening in one place and gone in
2343
+ the other, which is Slice 1's card lying in the direction Slice 4 exists to
2344
+ close. `daemon.ts` owns the signal there and does both writes before it
2345
+ goes. */
2346
+ if (!once && !injected) {
2347
+ for (const signal of ['SIGINT', 'SIGTERM']) {
2348
+ process.once(signal, () => {
2349
+ void stopListening(client, machineId, machineHarness).finally(() => {
2350
+ /* The exit code a signal is supposed to produce, and the reason it is
2351
+ said explicitly: `process.once` REPLACES node's default handler, so
2352
+ without this a Ctrl-C would leave the daemon polling forever. */
2353
+ process.exit(signal === 'SIGINT' ? 130 : 143);
2354
+ });
2355
+ });
2356
+ }
2357
+ }
2358
+ for (;;) {
2359
+ /* ═══ ONE POLL FAILING IS NOT THE DAEMON FAILING. ═══ Every read and write
2360
+ here throws on a network or database error, by design (constraint 7), and
2361
+ until Slice 4 that threw straight out of `cs3 run` and exited the process.
2362
+ Inside `cs start` that is no longer an honest outcome twice over: it would
2363
+ take the v2 presence heartbeat down with it, and it would leave a stranded
2364
+ card printing `cs start` at a person who IS running `cs start`, which is
2365
+ the exact lie this slice exists to end. A dropped connection is a poll
2366
+ that did not happen; the next one is two seconds away.
2367
+
2368
+ `--once` still fails loudly, because the acceptance harness reads the exit
2369
+ code and a swallowed failure there would make a broken suite look green. */
2370
+ try {
2371
+ /* ═══ THE FIRST THING EVERY POLL, BECAUSE IT IS WHAT MAKES THE OTHER THINGS
2372
+ LEGIBLE. ═══ A card with an untaken turn reads the same whether a daemon
2373
+ is two seconds away or nobody has one running; this row is the only place
2374
+ the difference exists. It is written before the takes rather than after
2375
+ so that a machine which is up but busy still reads as up. */
2376
+ await sayListening(client, machineId, machineName, machineHarness);
2377
+ /* ═══ THE USER'S STOP IS HONOURED BEFORE ANYTHING ELSE ON THE POLL. ═══ It
2378
+ is the only thing here that a person is waiting on, and the two takes
2379
+ below can spend the rest of the poll starting agents. Nothing else needs
2380
+ to run first: `panel3_stop_card` has already ended the runs, so recovery
2381
+ cannot see them and neither take can start them. */
2382
+ await killStopped(client, machineId);
2383
+ await reconcileOwnerSessions(client, machineId, machineHarness, new Set(inFlight.keys()));
2384
+ await recoverStranded(client, tools, machineId, new Set(inFlight.keys()), hold);
2385
+ /* ═══ AND THEN WHAT SOMEBODY ELSE'S MACHINE WAS HOLDING, IF A PERSON HANDED
2386
+ IT BACK. ═══ AFTER the sweep, deliberately: this machine settles its own
2387
+ runs on local, pid-accurate evidence before it looks at anybody's, and a
2388
+ run of its own that was offered while it was quiet is dealt with there on
2389
+ the better evidence: resumed if its process is really gone, which spends
2390
+ the offer, and skipped if it is not, which is what `mine` then honours
2391
+ here. The set is rebuilt rather than reused because the sweep adds to it.
2392
+ BEFORE both takes, for the reason the sweep is: work that already exists,
2393
+ with a brief and a report behind it, comes before work that has not
2394
+ started. */
2395
+ await takeHandedBack(client, tools, machineId, new Set(inFlight.keys()), hold);
2396
+ /* Every reason a conversational owner may continue is claimed together.
2397
+ The legacy turn and re-arm takes exclude named owners in the database. */
2398
+ await takeOwnerActivations(client, tools, machineId, new Set(inFlight.keys()), hold);
2399
+ /* THE MACHINE ID GOES IN because the take writes the run row, and a run has
2400
+ to say where it is running: the exclusion is cross-machine and recovery is
2401
+ per-machine, so a row with nobody's machine on it could be neither. */
2402
+ const taken = await returned(client.rpc('panel3_take_turns', { p_machine_id: machineId, p_agent: machineHarness }), 'take', 'turns');
2403
+ /* ═══ THE OTHER KIND OF TAKEABLE WORK. ═══ ux.md's re-arm: an answered
2404
+ question makes the branch that asked it takeable again, and a question
2405
+ still walking up makes the run it reached takeable so that level gets its
2406
+ turn. Same poll, same machine id, same holding: only the reason a run is
2407
+ started differs, and the record decides that rather than this file.
2408
+
2409
+ AFTER THE TURNS ARE TAKEN, deliberately, because the take WRITES the run
2410
+ row it leases to. A person who answered and then typed something else has
2411
+ both waiting, and in this order the message's level 1 is already on the
2412
+ record, so the re-arm leaves that card's coordinator alone and comes back
2413
+ to it on a later poll rather than putting two of them on one card. The
2414
+ other order would decide the same question from a snapshot taken before
2415
+ the run existed. */
2416
+ const rearmed = await returned(client.rpc('panel3_take_rearms', { p_machine_id: machineId, p_agent: machineHarness }), 'take', 'runs that can carry on');
2417
+ for (const row of rearmed) {
2418
+ /* NOT AWAITED PAST THE SPAWN, exactly as a taken card is not: the work is a
2419
+ real agent and holding the poll open for it would put every other card
2420
+ behind it. A throw here is caught rather than taking the loop down,
2421
+ because one run that could not be started must not stop the others. */
2422
+ try {
2423
+ hold(row.run_id, (await startRearmed(client, tools, machineId, row)).settled);
2424
+ }
2425
+ catch (error) {
2426
+ said(`could not start run ${row.run_id} again: ${error instanceof Error ? error.message : String(error)}`);
2427
+ }
2428
+ }
2429
+ for (const [cardId, turns] of byCard(taken)) {
2430
+ const runId = turns[0].run_id;
2431
+ out(`took card ${cardId} ${turns.length} turn${turns.length === 1 ? '' : 's'} run ${runId}`);
2432
+ /* NOT AWAITED HERE. A run is a real agent doing real work, and awaiting it
2433
+ in the poll would put every other card behind it — v2's own measured
2434
+ defect (env.ts, I21/I24: "a second request typed seconds after the first
2435
+ sat untouched until the first finished"). Nothing is needed to stop the
2436
+ same card being taken twice, because that is the database's job and it
2437
+ has already been done by the take. */
2438
+ // The catch inside `hold` is the last resort. A run left `running` by a
2439
+ // throw is not lost: its process has exited, so the next sweep reaps it
2440
+ // and the turns are answered again. It is there so the reason is printed
2441
+ // rather than becoming an unhandled rejection.
2442
+ hold(runId, answerCard(client, tools, machineId, cardId, turns));
2443
+ }
2444
+ if (once) {
2445
+ /* ═══ UNTIL NOTHING IS LEFT, NOT ONCE OVER WHAT WAS THERE. ═══ A level 1
2446
+ run dispatches WHILE it is being waited on, so the child appears in
2447
+ `inFlight` after the first wait started. Closing then would kill the
2448
+ tools server under an agent that had just been started, and `--once`
2449
+ would report a card answered while the work on it was still going. */
2450
+ while (inFlight.size > 0)
2451
+ await Promise.all([...inFlight.values()]);
2452
+ // The listener would otherwise keep this process alive after its one poll
2453
+ // was done, which is the whole point of `--once` being testable.
2454
+ await tools.close();
2455
+ // This machine is not listening any more, and it knows that here rather
2456
+ // than fifteen seconds from now. Same reason as the signal handlers above.
2457
+ await stopListening(client, machineId, machineHarness);
2458
+ return;
2459
+ }
2460
+ }
2461
+ catch (error) {
2462
+ if (once)
2463
+ throw error;
2464
+ said(`this poll did not finish: ${error instanceof Error ? error.message : String(error)}`);
2465
+ }
2466
+ await sleep(POLL_INTERVAL_MS);
2467
+ }
2468
+ }
2469
+ // ---------------------------------------------------------------------------
2470
+ /**
2471
+ * ═══ THE ONE THING `cs start` REACHES INTO `panel3/` FOR. ═══
2472
+ *
2473
+ * recovery-1 Slice 4, and Lane's ruling behind it (2026-08-21): *the user
2474
+ * experience must never require them to launch or authenticate multiple CLIs.
2475
+ * They launch the CLI with `cs start`.* Slices 1 to 3 print `cs start` on a
2476
+ * stranded card and, until this function existed, that command started a daemon
2477
+ * which polled no cards at all — the sentence named the right machine and the
2478
+ * wrong command, and no rewording could fix it.
2479
+ *
2480
+ * ═══ IT IS A NAMED EXCEPTION TO THE ISOLATION RULE, NOT A HOLE IN IT. ═══
2481
+ * `conventions.md` forbids anything outside `panel3/` importing anything inside
2482
+ * it, and `panel3-isolation.contract.test.mjs` enforces it. `cli-v2/src/daemon.ts`
2483
+ * is listed there with exactly ONE allowed specifier, `./panel3/run.js`, which is
2484
+ * why this is a single entry point handing back a single closure rather than
2485
+ * three exports the daemon would have to assemble. A second specifier fails the
2486
+ * suite, which is the point: the rule is enumerated, never weakened.
2487
+ *
2488
+ * ═══ THE CLIENT IS THE CALLER'S, AND THAT IS THE WHOLE OF "ONE SIGN-IN". ═══
2489
+ * `startPresence()` hands over the client it built from the session `cs login`
2490
+ * stored. This must never take one of its own: two clients in one process is two
2491
+ * refresh loops on one rotating refresh token, both writing `session.json`
2492
+ * through `getClient`'s `onAuthStateChange`, which is the hazard `client.ts`
2493
+ * describes moved indoors.
2494
+ *
2495
+ * ═══ IT DOES NOT BLOCK, AND IT DOES NOT TAKE THE DAEMON DOWN. ═══ The loop
2496
+ * never returns, so awaiting it here would hang `cs start` before it printed a
2497
+ * line. A throw that escapes the per-poll guard inside it is the panel stopping,
2498
+ * which is said on the one stream a person reads and leaves v2 presence
2499
+ * heartbeating rather than killing the process around it.
2500
+ */
2501
+ export function startPanel(client) {
2502
+ /* Both resolved HERE as well as inside the loop, deliberately: `stop` has to
2503
+ name the same row the loop's heartbeat writes, and it must be able to do
2504
+ that after the loop has thrown. They are the two facts about this machine
2505
+ that cannot change under a running process — the hardware-derived id and
2506
+ `CTRL_SPC_V3_AGENT` — so reading them twice cannot disagree. */
2507
+ const machineId = getMachineIdentity().id;
2508
+ const machineHarness = harness();
2509
+ void run([], client).catch((error) => {
2510
+ said(`the agent panel stopped polling: ${error instanceof Error ? error.message : String(error)}`);
2511
+ });
2512
+ /* `stopListening` is already best-effort and swallows its own failure: a
2513
+ process on its way out must not hang or fail, and the freshness window
2514
+ covers every way a machine can leave without getting here. */
2515
+ return { stop: () => stopListening(client, machineId, machineHarness) };
2516
+ }