omp-conductor 0.3.24 → 0.3.25

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -617,7 +617,7 @@ Then, per admitted issue:
617
617
  | --- | --- | --- | --- |
618
618
  | `pushed-pending` | `agent:in-progress` stays while the daemon rechecks GitHub | removed | none |
619
619
  | `pushed-green` | `agent:in-progress` stays while the PR is open | removed | none |
620
- | `blocked` | swapped to `agent:blocked` | removed | Tier 1 |
620
+ | `blocked` | swapped to `agent:blocked` | dirty tree committed to the branch, then removed | Tier 1 |
621
621
  | `failed` / `killed` | swapped to `agent:failed` | dirty tree committed to the branch, then retained until the PR or issue is terminal | Tier 1 |
622
622
  | unexpected error | swapped to `agent:failed` | same | Tier 1 |
623
623
 
@@ -630,18 +630,38 @@ Then, per admitted issue:
630
630
  leaves a window in which the issue carries no state label at all, which is
631
631
  exactly the shape eligibility reads as fresh work.
632
632
 
633
- **A non-graceful end salvages the tree first.** A turns-cap kill, a
634
- wall-clock kill and a crash are all external and unannounced: they land
635
- mid-edit, and only the run's *branch* is preserved across attempts — the tree
636
- is removed `--force` by the next one. So before the escalation is written,
637
- a dirty tree is committed to the run's own branch as
638
- `wip(#<issue>): attempt <n> killed by <reason> auto-salvaged` (everything,
639
- including files git has never seen) and pushed, and the escalation says where
640
- it went: `WIP committed to <branch> @ <sha>`. A push that is refused leaves
641
- the commit in this host's mirror and says so; a salvage that fails outright
642
- says that, loudly, naming the tree that now holds the only copy. A `blocked`
643
- run is deliberately *not* salvaged it stopped on purpose, with turns still
644
- in hand to commit for itself.
633
+ **Every continuable end salvages the tree first.** A turns-cap kill, a
634
+ wall-clock kill, a crash and a graceful block all leave a tree the next
635
+ attempt removes `--force` — only the run's *branch* is preserved across
636
+ attempts. So before the escalation is written, a dirty tree is committed to
637
+ the run's own branch as `wip(#<issue>): attempt <n> <ending> — auto-salvaged`
638
+ (everything, including files git has never seen) and pushed, and the
639
+ escalation says where it went: `WIP committed to <branch> @ <sha>`. A push
640
+ that is refused leaves the commit in this host's mirror and says so.
641
+
642
+ Blocking was excluded from this until #118, on the argument that a worker
643
+ which stops on purpose has turns left to commit for itself. It cost a
644
+ 34-file refactor: the worker blocked to ask whether a failing test was
645
+ obsolete — which is precisely a worker declining to commit a half-migrated
646
+ tree — and the daemon removed the tree seconds later, leaving the run branch
647
+ and `origin/main` on the same commit. A `pushed-green` or `pushed-pending`
648
+ run is now the only end that does not salvage: its deliverable is already on
649
+ a remote branch, and appending a WIP commit would turn the PR the daemon
650
+ just verified red.
651
+
652
+ **A salvage that fails keeps the tree and stops the issue.** If git refuses
653
+ the commit, the worktree is the only copy in existence, so it is retained
654
+ whatever the run's outcome was, the row records the failure, and the issue
655
+ is held out of dispatch with the `unsalvaged-wip` reason — because claiming
656
+ it is what would finally destroy the tree. `status` shows it under `wip` as
657
+ `UNSALVAGED`, and `omp-conductor unblock <n>` refuses. Recover the tree by
658
+ hand, then `unblock <n> --force` records that you accepted it and releases
659
+ the hold.
660
+
661
+ **A preserved tip is named to the next worker.** The sha is written to the
662
+ run row, shown by `status` and the board, and the continuation brief tells
663
+ the resuming worker the exact commit it is building on and that it is the
664
+ only copy.
645
665
 
646
666
  Later ticks reap retained failure trees in bounded batches after the tracker
647
667
  proves their PR merged/closed or their issue closed, provided no live run or
@@ -1280,11 +1300,14 @@ nowhere. `omp-conductor status` is where they are told apart — its `telegram`
1280
1300
  names the specific fault.
1281
1301
 
1282
1302
  One caveat the file cannot express: omp-telegram binds its own copy of the token
1283
- in `startBot()` at session start, and rebinds it only on `/telegram token` or
1284
- `/telegram on`. Writing a token into `.env` out-of-band therefore restores
1285
- tier-2 paging immediately conductor sends those itselfwhile the bridge's own
1303
+ in `startBot()` at session start, and only when the bridge is switched on. It
1304
+ rebinds only on `/telegram token` or `/telegram on`. So writing a token into
1305
+ `.env` out-of-band or flipping `enabled` to true by hand restores tier-2
1306
+ paging immediately, because conductor sends those itself, while the bridge's own
1286
1307
  tools, `telegram_send` and `telegram_ask`, stay dead until you reload it. After
1287
- adding a token by hand, run `/telegram on` in the orchestrator session.
1308
+ either edit, run `/telegram on` in the orchestrator session. Until you do, ticks
1309
+ carry an explicit note that an amendment cannot be approved on this surface, and
1310
+ the conductor never assumes an answer it did not receive.
1288
1311
 
1289
1312
  Leaving `accessFile` unset passes the gate, because an ordinary developer session
1290
1313
  that happens to have a `.conductor-tick.json` has no bridge to check. It is not an
@@ -1392,7 +1415,7 @@ omp-conductor disarm [--project NAME]
1392
1415
  omp-conductor release-pane [--project NAME]
1393
1416
  omp-conductor tail <issue> [--project NAME]
1394
1417
  omp-conductor extend <issue> --turns N [--project NAME]
1395
- omp-conductor unblock <issue> [--project NAME]
1418
+ omp-conductor unblock <issue> [--force] [--project NAME]
1396
1419
  omp-conductor friction <escalation-digest|report-noise|report-surprise> --detail TEXT [--issue N] [--project NAME]
1397
1420
  omp-conductor daemon [--once] [--port N] [--project NAME]
1398
1421
  omp-conductor pause
@@ -1417,7 +1440,7 @@ omp-conductor help
1417
1440
  | `release-pane [--project NAME]` | Clear the `halt --pane` recovery pin so herdr-conductor may resume the fleet agent again. |
1418
1441
  | `tail <issue>` | Follow the newest run for that issue: the worker's assistant text as `assistant: …` and each tool it calls as `tool: <name>`, printed as they land. Workers are omp sessions inside the daemon rather than terminals, so this is the only way to watch one live — a herdr pane running it becomes an observation window. Starts from the top of the transcript, not the end, so attaching to a run that is already ten turns in shows those ten turns. Exits `1` with `no run recorded for #N` when the issue has never been dispatched, or `no transcript yet (state: …)` when the attempt has not opened one. Otherwise it runs until `Ctrl-C`, or until the run has finished and its transcript has been silent for five seconds, and prints `run ended: <state>`. |
1419
1442
  | `extend <issue> --turns N [--project NAME]` | Monotonically raise that live worker's effective turn ceiling through its owning daemon. The current omp session keeps running; no restart or continuation is created. The daemon persists the new ceiling for `status` and rejects missing, settled, cap-killed, equal, or lower requests instead of implying that an immutable session changed. |
1420
- | `unblock <issue>` | Remove that issue's `blocked` and `failed` labels so an answered escalation can be claimed again. `agent:in-progress` comes off too, but only when the newest recorded run is terminal — that row is the proof no worker still owns the issue, so a live run keeps the label, and so does an issue with no run row at all. Run history remains intact: blocks consume the independent continuation budget, not failed implementation attempts. The output reports both budgets and warns when either will make the next tick escalate instead of dispatch. Exits `2` when the issue number is missing or malformed. |
1443
+ | `unblock <issue> [--force]` | Remove that issue's `blocked` and `failed` labels so an answered escalation can be claimed again. `agent:in-progress` comes off too, but only when the newest recorded run is terminal — that row is the proof no worker still owns the issue, so a live run keeps the label, and so does an issue with no run row at all. Run history remains intact: blocks consume the independent continuation budget, not failed implementation attempts. The output reports both budgets and warns when either will make the next tick escalate instead of dispatch. **Refuses, clearing nothing and exiting `3`, when the newest attempt's work could not be committed and its worktree is the only copy** — re-claiming removes that tree. `--force` records the operator's acceptance on the run row and then clears; the salvage failure stays in history. Exits `2` when the issue number is missing or malformed. |
1421
1444
  | `friction <kind> --detail TEXT [--issue N]` | Record one bounded judgment the daemon cannot infer: an escalation belonged in a digest, or a tick report was noise/surprising. The detail is limited to 160 characters. One event never changes policy; three observations inside seven days make the aggregate eligible for one Learning-loop prompt, followed by a seven-day cooldown. |
1422
1445
  | `daemon` | Run the loop in the **foreground**, ticking every 5 minutes and serving `/healthz`. Admitted workers run in a tracked background pool, so settlement and capacity checks remain periodic while they work; shutdown drains the pool before closing the store. This is what `start` launches and what a systemd unit should call. |
1423
1446
  | `daemon --once` | Run a single tick, wait for workers admitted by that tick, and exit. No HTTP server or pidfile — a drill must not register itself as the daemon, or the next reader believes it and the real daemon's in-flight runs get reconciled as orphans. |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "omp-conductor",
3
- "version": "0.3.24",
3
+ "version": "0.3.25",
4
4
  "type": "module",
5
5
  "license": "MIT",
6
6
  "description": "A 24/7 dispatcher that takes ready GitHub issues to green, mergeable PRs using omp coding sessions, with tiered escalation first to an orchestrator session and then to a human.",
@@ -36,7 +36,7 @@
36
36
  */
37
37
 
38
38
  import { readFileSync } from "node:fs";
39
- import { join } from "node:path";
39
+ import { dirname, join } from "node:path";
40
40
 
41
41
  /** The tool the package floor names for the Learning-loop yes/no amendment
42
42
  * approval (`## Learning loop`, step 2, in `briefs/orchestrator.md`). Held as
@@ -96,6 +96,41 @@ export function hasBotToken(stateDir: string): boolean {
96
96
  return false;
97
97
  }
98
98
 
99
+ /**
100
+ * Whether omp-telegram would have *bound* a token, which is what its own tools
101
+ * depend on and is strictly narrower than the token existing.
102
+ *
103
+ * `startBot()` is the only thing that assigns `token`, and `session_start` calls
104
+ * it only when the bridge is switched on:
105
+ *
106
+ * ```js
107
+ * if (pi.getFlag("telegram") === true || process.env.OMP_TELEGRAM === "1" || access.enabled) await startBot(ctx);
108
+ * ```
109
+ *
110
+ * So a token sitting beside a bridge that was disabled at start was never
111
+ * picked up, and flipping `enabled` to true in the file afterwards does not pick
112
+ * it up either — only `/telegram on` or a restart does. Both halves therefore
113
+ * have to be true *at the same moment*, which is why this is sampled once rather
114
+ * than re-read.
115
+ *
116
+ * `pi.getFlag("telegram")` is omp's own launch flag and is not visible to this
117
+ * extension, so a session started with it but with `enabled: false` on disk
118
+ * reads as unbound here. That is the conservative direction: the cost is a
119
+ * fallback instruction the orchestrator can carry out, against an amendment
120
+ * recorded as approved that nobody answered.
121
+ */
122
+ export function bridgeTokenBound(accessPath: string): boolean {
123
+ if (!hasBotToken(dirname(accessPath))) return false;
124
+ if (process.env["OMP_TELEGRAM"] === "1") return true;
125
+ let access: unknown;
126
+ try {
127
+ access = JSON.parse(readFileSync(accessPath, "utf8"));
128
+ } catch {
129
+ return false;
130
+ }
131
+ return field(access, "enabled") === true;
132
+ }
133
+
99
134
  /** Reads the omp-telegram access file and answers whether a locally injected
100
135
  * turn would resolve an answerable destination.
101
136
  *
package/src/board.ts CHANGED
@@ -418,7 +418,12 @@ function runCardLines(run: RunRecord, snapshot: BoardSnapshot): string[] {
418
418
  `attempt ${run.attempt} · ${run.turns}/${run.maxTurns}t`,
419
419
  `$${run.spendUsd.toFixed(2)} · ${duration}`,
420
420
  ];
421
- if (run.state === "pushed-pending") lines.push("checks pending");
421
+ // Ordered by what needs a human first. An unsalvaged tree outranks even a
422
+ // last error: the error describes a run that is over, the tree is work that
423
+ // is still at risk and an issue that will not dispatch (#118).
424
+ if (run.salvageError !== undefined && run.salvageAckAt === undefined) lines.push("UNSALVAGED WIP");
425
+ else if (run.state === "pushed-pending") lines.push("checks pending");
426
+ else if (run.salvageSha !== undefined) lines.push(`wip @ ${run.salvageSha.slice(0, 7)}`);
422
427
  else if (run.lastError !== undefined) lines.push(run.lastError.replace(/\s+/g, " "));
423
428
  else if (run.prUrl !== undefined) lines.push(run.prUrl.replace(/^https?:\/\//, ""));
424
429
  else lines.push(run.branch);
@@ -125,6 +125,14 @@ which in its output: a run that is genuinely still live, where declining is the
125
125
  correct answer, and no run row at all to prove the worker is gone. In that second
126
126
  case, escalate the stuck issue rather than editing its labels.
127
127
 
128
+ `unblock` refuses outright, clearing nothing, when the newest attempt's work
129
+ could not be committed and its worktree is the only copy of it — re-claiming the
130
+ issue force-removes that tree. Its output names the directory and offers
131
+ `--force`. **`--force` is never yours to pass.** It records that a human
132
+ inspected the tree and accepted the loss, which is a judgement about work you
133
+ cannot see, and the daemon has already paged the operator about it. Report the
134
+ refusal and move on to the next issue.
135
+
128
136
  Never leave an orphan holding a slot "to be safe": a label nobody is working under
129
137
  is not safety, it is a deadlocked fleet that looks busy.
130
138
 
package/src/cli.ts CHANGED
@@ -94,7 +94,7 @@ usage:
94
94
  omp-conductor release-pane [--project NAME]
95
95
  omp-conductor tail <issue> [--project NAME]
96
96
  omp-conductor extend <issue> --turns N [--project NAME]
97
- omp-conductor unblock <issue> [--project NAME]
97
+ omp-conductor unblock <issue> [--force] [--project NAME]
98
98
  omp-conductor daemon [--once] [--port N] [--project NAME]
99
99
  omp-conductor pause
100
100
  omp-conductor resume
@@ -150,6 +150,9 @@ usage:
150
150
  and why the brief's "never hand-edit a state label" rule can stay
151
151
  absolute. Run history is kept; answered blocks consume the separate
152
152
  operational-continuation budget, not failed implementation attempts.
153
+ Refuses when the newest attempt's work could not be committed and
154
+ its worktree is the only copy: re-claiming removes that tree. Use
155
+ --force once you have recovered it or accepted the loss.
153
156
  friction record a bounded observation the daemon cannot classify itself:
154
157
  an escalation that belonged in a digest, or a tick report that was
155
158
  noise/surprising. Repeated observations feed the existing Learning
@@ -636,8 +639,12 @@ try {
636
639
  const project = findProject(cfg, flag(argv, "project"));
637
640
  const store = openStore(dbPath());
638
641
  try {
639
- const outcome = await unblockIssue(project, makeTracker(project), store, issue);
642
+ const outcome = await unblockIssue(project, makeTracker(project), store, issue, {
643
+ force: argv.includes("--force"),
644
+ });
640
645
  process.stdout.write(`${formatUnblock(issue, outcome, project, resolveCaps(project, cfg.defaults))}\n`);
646
+ // A refusal must not read as success to a script or a board keypress.
647
+ if (outcome.refused !== undefined) process.exitCode = 3;
641
648
  } finally {
642
649
  store.close();
643
650
  }
package/src/daemon.ts CHANGED
@@ -387,16 +387,30 @@ async function safeEscalate(d: Pick<Deps, "escalate">, e: Escalation): Promise<b
387
387
  * `checkIntegrity` is — this wording is the whole thing a human acts on, so it
388
388
  * is worth a test holding it, and the sha in it is the only pointer to work
389
389
  * that no longer has any other copy.
390
+ *
391
+ * `retained` is not cosmetic. These lines used to promise a tree "kept for
392
+ * inspection" unconditionally, which was true only because salvage ran only on
393
+ * the paths that keep one. A blocked run's tree is removed the moment its work
394
+ * is safely on the branch, and sending an operator to a path this process just
395
+ * deleted is the same class of mistake as #118 itself.
390
396
  */
391
- export function salvageLines(outcome: SalvageOutcome, worktree: string): string[] {
392
- const kept = `Worktree kept for inspection: ${worktree}`;
397
+ export function salvageLines(
398
+ outcome: SalvageOutcome,
399
+ worktree: string,
400
+ retained: boolean,
401
+ ): string[] {
402
+ const fate = retained
403
+ ? `Worktree kept for inspection: ${worktree}`
404
+ : `Worktree removed: ${worktree}`;
393
405
 
394
- if (outcome.kind === "nothing") return [`${kept} — nothing uncommitted to salvage`];
406
+ if (outcome.kind === "nothing") return [`${fate} — nothing uncommitted to salvage`];
395
407
 
396
408
  if (outcome.kind === "failed") {
397
409
  return [
398
410
  `WIP SALVAGE FAILED: ${outcome.error}`,
399
- `Uncommitted work in ${worktree} is the only copy of it, and the next attempt removes that tree.`,
411
+ `Uncommitted work in ${worktree} is the only copy of it, so the tree was kept.`,
412
+ "This issue is held out of dispatch until the tree is recovered by hand and",
413
+ "`omp-conductor unblock <n> --force` records that you accepted it.",
400
414
  ];
401
415
  }
402
416
 
@@ -415,35 +429,80 @@ export function salvageLines(outcome: SalvageOutcome, worktree: string): string[
415
429
  : `${count}; new: ${outcome.newPaths.slice(0, 12).join(", ")}${
416
430
  outcome.newPaths.length > 12 ? `, … +${outcome.newPaths.length - 12} more` : ""
417
431
  }`;
418
- return [where, manifest, kept];
432
+ return [where, manifest, fate];
433
+ }
434
+
435
+ /** Everything a settled run has to record and say about its worktree. */
436
+ export interface WorktreeSettlement {
437
+ outcome: SalvageOutcome;
438
+ /** Whether the tree still exists now the run is over. */
439
+ retained: boolean;
440
+ /** Escalation lines naming where the work went. */
441
+ lines: string[];
442
+ /** Row fields recording the durable ref, or the failure that blocks a re-claim. */
443
+ patch: Pick<RunRecord, "salvageSha" | "salvageError">;
419
444
  }
420
445
 
421
446
  /**
422
- * Commits and pushes whatever a dead run left uncommitted, logs the outcome and
423
- * returns the escalation lines that say where that work now lives.
447
+ * Decides what becomes of a finished run's worktree: save the work, then keep
448
+ * or remove the tree, then say which.
449
+ *
450
+ * One function because the two halves are one decision and splitting them is
451
+ * how #118 happened — the removal at the end of dispatch had no idea whether
452
+ * anything had been saved, and the salvage at the top of the failure branch had
453
+ * no idea the blocked branch fell through to a `--force` removal.
454
+ *
455
+ * A salvage that *fails* retains the tree whatever the caller asked for. There
456
+ * was real work, git refused to commit it, and the tree is now the only copy in
457
+ * existence: deleting it on schedule would be the data loss this whole path
458
+ * exists to prevent. The issue is held out of dispatch until an operator says
459
+ * otherwise, because the next attempt's `worktree remove --force` would finish
460
+ * the job (see `admitCandidates`).
424
461
  *
425
- * Only ever called on a non-graceful end a cap kill, a crashed session, a
426
- * dispatch error. A `blocked` run stopped on purpose, with turns still in hand
427
- * and a brief that tells it to report rather than push, so nothing is committed
428
- * behind its back. The rest never got the chance: the kill is external and
429
- * lands mid-edit, in the tree the next attempt removes `--force`.
462
+ * Exported so a test can drive the real decision against a real git tree.
430
463
  */
431
- async function salvage(
432
- issue: number,
433
- attempt: number,
434
- reason: string,
435
- worktree: string,
436
- ): Promise<string[]> {
437
- const lines = salvageLines(await salvageWip(worktree, issue, attempt, reason), worktree);
464
+ export async function settleWorktree(
465
+ args: {
466
+ issue: number;
467
+ attempt: number;
468
+ /** Clause for the commit subject: "killed by the turns cap", "blocked …". */
469
+ ending: string;
470
+ worktree: string;
471
+ } & (
472
+ | /** Terminal-failure and orphan trees are evidence, and are kept even when clean. */
473
+ { tree: "keep" }
474
+ | { tree: "remove"; mirrorPath: string }
475
+ ),
476
+ ): Promise<WorktreeSettlement> {
477
+ const { issue, attempt, ending, worktree } = args;
478
+ const outcome = await salvageWip(worktree, issue, attempt, ending);
479
+ const retained = args.tree === "keep" || outcome.kind === "failed";
480
+ if (!retained && args.tree === "remove") await removeWorktree(args.mirrorPath, worktree);
481
+
482
+ const lines = salvageLines(outcome, worktree, retained);
438
483
  log(`#${issue} salvage: ${lines.join(" ")}`);
439
- return lines;
484
+ return {
485
+ outcome,
486
+ retained,
487
+ lines,
488
+ patch:
489
+ outcome.kind === "salvaged"
490
+ ? { salvageSha: outcome.sha }
491
+ : outcome.kind === "failed"
492
+ ? { salvageError: outcome.error }
493
+ : {},
494
+ };
440
495
  }
441
496
 
442
- /** How a run's end is named — in the salvage commit, and to whoever reads it. */
497
+ /**
498
+ * How a run's end is named — in the salvage commit, and to whoever reads it.
499
+ * The whole clause, not a bare reason: a graceful block was not killed by
500
+ * anything, and the commit subject is read during recovery.
501
+ */
443
502
  function endedBy(killedBy: KilledBy | undefined): string {
444
- if (killedBy === "turns") return "the turns cap";
445
- if (killedBy === "wallclock") return "the wall-clock cap";
446
- return "a failed run";
503
+ if (killedBy === "turns") return "killed by the turns cap";
504
+ if (killedBy === "wallclock") return "killed by the wall-clock cap";
505
+ return "killed by a failed run";
447
506
  }
448
507
 
449
508
  /**
@@ -460,7 +519,7 @@ export async function buildBrief(
460
519
  r: Routed,
461
520
  branch: string,
462
521
  worktree: string,
463
- opts: { continuation?: boolean; defaultBranch?: string } = {},
522
+ opts: { continuation?: boolean; defaultBranch?: string; salvagedSha?: string } = {},
464
523
  ): Promise<string> {
465
524
  // Read per dispatch rather than caching: editing the brief then takes effect
466
525
  // on the next issue instead of needing a daemon restart.
@@ -474,6 +533,14 @@ export async function buildBrief(
474
533
  "",
475
534
  `You are **resuming** issue #${r.issue.number}. Branch \`${branch}\` already exists`,
476
535
  "and was reattached with prior commits (and possibly a salvaged WIP tip).",
536
+ ...(opts.salvagedSha === undefined
537
+ ? []
538
+ : [
539
+ "",
540
+ `The previous attempt's uncommitted work was preserved for you as commit`,
541
+ `\`${opts.salvagedSha}\` on this branch. It is the tip you are continuing from,`,
542
+ "and it is the only copy of that work — do not reset past it or force-push over it.",
543
+ ]),
477
544
  "Before writing anything:",
478
545
  "",
479
546
  "```bash",
@@ -599,9 +666,10 @@ async function handleIssue(d: Deps, r: Routed, attempt: number): Promise<void> {
599
666
 
600
667
  let claimed = false;
601
668
  let run: RunRecord | undefined;
602
- // Hoisted out of the try so the catch path can still name the tree: a crash
603
- // mid-dispatch is one of the non-graceful ends whose uncommitted work has to
604
- // be salvaged too, and it is the path least likely to have committed first.
669
+ // Hoisted out of the try so the catch path can still name and save the tree:
670
+ // a crash mid-dispatch is one of the ends whose uncommitted work has to be
671
+ // salvaged too, and it is the path least likely to have committed first.
672
+ const mirrorPath = mirrorPathFor(r.repo, project.mirrorRoot);
605
673
  let worktreePath: string | undefined;
606
674
  let turnLimit: TurnLimitController | undefined;
607
675
 
@@ -612,6 +680,10 @@ async function handleIssue(d: Deps, r: Routed, attempt: number): Promise<void> {
612
680
  // issue out, and a human decides what to do with the orphan.
613
681
  await tracker.addLabel(issue, inProgress);
614
682
  claimed = true;
683
+ // Read before this attempt's own row exists, so `latestRun` still means the
684
+ // attempt whose work this one inherits.
685
+ const priorSalvage = store.latestRun(project.name, issue)?.salvageSha;
686
+
615
687
 
616
688
  run = store.createRun({
617
689
  project: project.name,
@@ -635,7 +707,6 @@ async function handleIssue(d: Deps, r: Routed, attempt: number): Promise<void> {
635
707
  // tolerates a mirror or tree that is not there yet, so this is safe on a
636
708
  // first attempt. addWorktree does its own ensureMirror; calling it here too
637
709
  // would cost a second network fetch per attempt.
638
- const mirrorPath = mirrorPathFor(r.repo, project.mirrorRoot);
639
710
  await removeWorktree(mirrorPath, worktreePathFor(project.workspaceRoot, issue));
640
711
  const provisioned = await addWorktree(
641
712
  r.repo,
@@ -664,6 +735,9 @@ async function handleIssue(d: Deps, r: Routed, attempt: number): Promise<void> {
664
735
  brief: await buildBrief(project, r, branch, worktreePath, {
665
736
  continuation: provisioned.reattached,
666
737
  defaultBranch: r.repo.defaultBranch,
738
+ ...(provisioned.reattached && priorSalvage !== undefined
739
+ ? { salvagedSha: priorSalvage }
740
+ : {}),
667
741
  }),
668
742
  cwd: worktreePath,
669
743
  caps,
@@ -706,6 +780,27 @@ async function handleIssue(d: Deps, r: Routed, attempt: number): Promise<void> {
706
780
  const finalReport =
707
781
  verified.reason === undefined ? result.report : `${verified.reason}\n\n${result.report}`;
708
782
 
783
+ // What becomes of the tree, decided once, before any label or page. A
784
+ // `pushed-*` run is the only end that does not salvage: its deliverable is
785
+ // already on a remote branch, whatever is left loose in the tree is by the
786
+ // worker's own account not part of it, and appending a WIP commit would
787
+ // turn the green PR this daemon just verified red. Every other end is
788
+ // continuable, so its tree is treated as work.
789
+ const settlement =
790
+ state === "pushed-green" || state === "pushed-pending" || state === "merged"
791
+ ? undefined
792
+ : await settleWorktree({
793
+ issue,
794
+ attempt,
795
+ ending:
796
+ state === "blocked" ? "blocked for an operator decision" : endedBy(result.killedBy),
797
+ worktree: worktreePath,
798
+ ...(state === "failed" || state === "killed"
799
+ ? ({ tree: "keep" } as const)
800
+ : ({ tree: "remove", mirrorPath } as const)),
801
+ });
802
+ if (settlement === undefined) await removeWorktree(mirrorPath, worktreePath);
803
+
709
804
  store.updateRun(runId, {
710
805
  state,
711
806
  endedAt: Date.now(),
@@ -715,8 +810,11 @@ async function handleIssue(d: Deps, r: Routed, attempt: number): Promise<void> {
715
810
  headSha: result.headSha,
716
811
  sessionFile: result.sessionFile,
717
812
  ...(verified.reason === undefined ? {} : { lastError: verified.reason }),
813
+ ...settlement?.patch,
718
814
  });
719
815
 
816
+ const salvaged = settlement?.lines ?? [];
817
+
720
818
  if (state === "blocked") {
721
819
  await swapLabel(tracker, issue, inProgress, project.stateLabels.blocked);
722
820
  await safeEscalate(d, {
@@ -725,7 +823,7 @@ async function handleIssue(d: Deps, r: Routed, attempt: number): Promise<void> {
725
823
  issue,
726
824
  runId,
727
825
  summary: `#${issue} is blocked on attempt ${attempt} and needs a decision`,
728
- detail: [`${r.issue.title}`, r.issue.url, "", result.report].join("\n"),
826
+ detail: [`${r.issue.title}`, r.issue.url, "", ...salvaged, "", result.report].join("\n"),
729
827
  });
730
828
  } else if (state === "failed" || state === "killed") {
731
829
  // A turns cap consumes the independent continuation budget, not an
@@ -735,7 +833,6 @@ async function handleIssue(d: Deps, r: Routed, attempt: number): Promise<void> {
735
833
  const continueTurns =
736
834
  result.killedBy === "turns" &&
737
835
  hasContinuationBudget(continuation, caps.maxContinuationsPerIssue);
738
- const salvaged = await salvage(issue, attempt, endedBy(result.killedBy), worktreePath);
739
836
 
740
837
  if (continueTurns) {
741
838
  await tracker.removeLabel(issue, inProgress);
@@ -790,13 +887,6 @@ async function handleIssue(d: Deps, r: Routed, attempt: number): Promise<void> {
790
887
  // checks or merge settle, preventing another worker from duplicating it.
791
888
  log(`#${issue} ${state}${result.prUrl ? ` ${result.prUrl}` : ""}`);
792
889
  }
793
-
794
- // A failed or killed tree is evidence — keep it. Anything else is just
795
- // disk, and the mirror means re-provisioning is cheap. (The kept tree is
796
- // wiped by the next attempt, not left to accumulate forever.)
797
- if (state !== "failed" && state !== "killed") {
798
- await removeWorktree(mirrorPath, worktreePath);
799
- }
800
890
  } catch (err) {
801
891
  // Dispatch setup can fail after the controller opens but before runWorker's
802
892
  // inner settlement guard exists. Latch it before any terminal write or await.
@@ -804,8 +894,25 @@ async function handleIssue(d: Deps, r: Routed, attempt: number): Promise<void> {
804
894
  turnLimit = undefined;
805
895
  const detail = errText(err);
806
896
  log(`#${issue} errored: ${detail}`);
897
+ // A crash lands anywhere, including mid-edit in a tree holding the only
898
+ // copy of real work. Nothing else on this path so much as looks at it.
899
+ const settlement =
900
+ worktreePath === undefined
901
+ ? undefined
902
+ : await settleWorktree({
903
+ issue,
904
+ attempt,
905
+ ending: "killed by a dispatch error",
906
+ worktree: worktreePath,
907
+ tree: "keep",
908
+ });
807
909
  if (run) {
808
- store.updateRun(run.id, { state: "failed", endedAt: Date.now(), lastError: detail });
910
+ store.updateRun(run.id, {
911
+ state: "failed",
912
+ endedAt: Date.now(),
913
+ lastError: detail,
914
+ ...settlement?.patch,
915
+ });
809
916
  }
810
917
  if (claimed) {
811
918
  // Leaving the issue stuck as in-progress would hide it from both the
@@ -816,13 +923,8 @@ async function handleIssue(d: Deps, r: Routed, attempt: number): Promise<void> {
816
923
  log(`#${issue} could not be relabelled: ${errText(relabelErr)}`);
817
924
  }
818
925
  }
819
- // A crash lands anywhere, including mid-edit in a tree holding the only
820
- // copy of real work. Nothing else on this path so much as looks at it.
821
- const salvaged =
822
- worktreePath === undefined
823
- ? []
824
- : await salvage(issue, attempt, "a dispatch error", worktreePath);
825
926
 
927
+ const salvaged = settlement?.lines ?? [];
826
928
  await safeEscalate(d, {
827
929
  tier: 1,
828
930
  project: project.name,
@@ -1278,6 +1380,32 @@ export async function admitCandidates(
1278
1380
  continue;
1279
1381
  }
1280
1382
 
1383
+ // Fail closed on work that exists only in a worktree. `addWorktree` clears
1384
+ // the tree at <workspaceRoot>/<issue> before it provisions, so admitting
1385
+ // this issue is what finally destroys the copy the salvage could not save
1386
+ // (#118). Nothing here can recover it — git already refused once — so the
1387
+ // only safe move is to refuse the claim and keep saying why until an
1388
+ // operator has looked and run `unblock --force`.
1389
+ const newest = store.latestRun(project.name, issue);
1390
+ if (newest?.salvageError !== undefined && newest.salvageAckAt === undefined) {
1391
+ hold(issue, "unsalvaged-wip");
1392
+ await safeEscalate(d, {
1393
+ tier: 1,
1394
+ project: project.name,
1395
+ issue,
1396
+ summary: `#${issue} is holding unsalvaged work and will not be re-claimed`,
1397
+ detail: [
1398
+ r.issue.title,
1399
+ r.issue.url,
1400
+ `Attempt ${newest.attempt} could not commit its uncommitted changes: ${newest.salvageError}`,
1401
+ `The only copy is the worktree ${newest.worktree === "" ? "(path not recorded)" : newest.worktree}.`,
1402
+ "Dispatch is held because claiming this issue removes that tree.",
1403
+ "Recover it by hand, then `omp-conductor unblock <n> --force` to release the hold.",
1404
+ ].join("\n"),
1405
+ });
1406
+ continue;
1407
+ }
1408
+
1281
1409
  // Soft concurrency per epic: at most one in-flight child of a given parent.
1282
1410
  // No parent means today's concurrent admission. Cheap local filters already
1283
1411
  // ran; this sits before the open-PR API call so a held sibling frees the
@@ -1674,6 +1802,9 @@ export interface StatusSnapshot {
1674
1802
  caps: Caps;
1675
1803
  /** Occupied issues: live workers plus green PRs awaiting a human merge. */
1676
1804
  activeRuns: RunRecord[];
1805
+ /** Newest attempts holding a preserved WIP tip, or a tree that is still the
1806
+ * only copy of work the daemon could not save. */
1807
+ salvagedRuns: RunRecord[];
1677
1808
  /** Runs backed by a worker process — the number capacity compares against. */
1678
1809
  liveWorkers: number;
1679
1810
  runsToday: number;
@@ -1694,6 +1825,7 @@ export function statusSnapshotFromStore(p: ProjectConfig, caps: Caps, store: Sto
1694
1825
  paused: isPaused(),
1695
1826
  caps,
1696
1827
  activeRuns: store.activeRuns(p.name),
1828
+ salvagedRuns: store.salvagedRuns(p.name),
1697
1829
  liveWorkers: store.liveRuns(p.name).length,
1698
1830
  runsToday: store.runsStartedSince(p.name, since),
1699
1831
  spendTodayUsd: store.spendSince(p.name, since),
@@ -1737,6 +1869,32 @@ export function formatDispatchSummary(summary?: DispatchSummary): string {
1737
1869
  return lines.join("\n");
1738
1870
  }
1739
1871
 
1872
+ /**
1873
+ * The WIP block: every issue whose newest attempt left work behind, and
1874
+ * whether that work is safe.
1875
+ *
1876
+ * Blocked runs used to be invisible here, which is exactly how #118 stayed
1877
+ * invisible for a full attempt cycle — the operator saw a blocked issue and had
1878
+ * no way to tell "stopped with 34 uncommitted files" from "stopped clean".
1879
+ * A preserved line is informational; an UNSALVAGED line is an alarm, and it
1880
+ * names the directory because that directory is the work.
1881
+ */
1882
+ export function formatSalvagedRuns(runs: readonly RunRecord[]): string[] {
1883
+ if (runs.length === 0) return [];
1884
+ const lines = ["", "wip"];
1885
+ for (const r of runs) {
1886
+ lines.push(
1887
+ r.salvageError !== undefined && r.salvageAckAt === undefined
1888
+ ? ` #${r.issue} UNSALVAGED ${r.worktree === "" ? "(path not recorded)" : r.worktree} — ` +
1889
+ `only copy, dispatch held (${r.salvageError})`
1890
+ : r.salvageError !== undefined
1891
+ ? ` #${r.issue} accepted as lost attempt ${r.attempt} (${r.salvageError})`
1892
+ : ` #${r.issue} preserved ${r.salvageSha ?? "?"} on ${r.branch} (attempt ${r.attempt}, ${r.state})`,
1893
+ );
1894
+ }
1895
+ return lines;
1896
+ }
1897
+
1740
1898
  export function formatStatus(s: StatusSnapshot): string {
1741
1899
  const lines = [
1742
1900
  `project ${s.project}${s.paused ? " (PAUSED)" : ""}`,
@@ -1769,6 +1927,7 @@ export function formatStatus(s: StatusSnapshot): string {
1769
1927
  );
1770
1928
  }
1771
1929
  }
1930
+ lines.push(...formatSalvagedRuns(s.salvagedRuns));
1772
1931
  // Deploy hint: a restart while workers are live orphans them (salvage runs
1773
1932
  // first — #35). Prefer pause + drain to zero live workers when you can wait.
1774
1933
  if (s.liveWorkers > 0) {
@@ -1873,12 +2032,20 @@ export async function reconcileOrphanedRuns(
1873
2032
  const endedAt = Date.now();
1874
2033
  for (const r of stale) {
1875
2034
  // Salvage before the row flips: the worktree path is on the record, and
1876
- // salvageWip is a no-op for a missing/clean tree. Reason string matches
1877
- // the cap-kill wording so triage reads the same either way.
1878
- if (r.worktree !== "") {
1879
- await salvage(r.issue, r.attempt, "a daemon restart", r.worktree);
1880
- }
1881
- store.updateRun(r.id, { state: "orphaned", endedAt });
2035
+ // salvageWip is a no-op for a missing/clean tree. The clause matches the
2036
+ // cap-kill wording so triage reads the same either way, and the tree is
2037
+ // kept because an orphan's remains are the orchestrator's drain-duty call.
2038
+ const settlement =
2039
+ r.worktree === ""
2040
+ ? undefined
2041
+ : await settleWorktree({
2042
+ issue: r.issue,
2043
+ attempt: r.attempt,
2044
+ ending: "killed by a daemon restart",
2045
+ worktree: r.worktree,
2046
+ tree: "keep",
2047
+ });
2048
+ store.updateRun(r.id, { state: "orphaned", endedAt, ...settlement?.patch });
1882
2049
  }
1883
2050
  return stale;
1884
2051
  }
package/src/fleet.ts CHANGED
@@ -30,7 +30,14 @@ import { dirname, join } from "node:path";
30
30
  import { findProject, loadConfig, stateDir } from "./config.ts";
31
31
  import { readApprovalSurface } from "./approval-surface.ts";
32
32
  import { probeCodeGraph, type CodeGraphHealth } from "./graph-health.ts";
33
- import { formatDispatchSummary, isPaused, setPaused, statusSnapshot, type StatusSnapshot } from "./daemon.ts";
33
+ import {
34
+ formatDispatchSummary,
35
+ formatSalvagedRuns,
36
+ isPaused,
37
+ setPaused,
38
+ statusSnapshot,
39
+ type StatusSnapshot,
40
+ } from "./daemon.ts";
34
41
  import {
35
42
  healthCheck,
36
43
  isAlive,
@@ -1029,6 +1036,7 @@ function formatProjectBody(s: StatusSnapshot): string {
1029
1036
  );
1030
1037
  }
1031
1038
  }
1039
+ lines.push(...formatSalvagedRuns(s.salvagedRuns));
1032
1040
  if (s.liveWorkers > 0) {
1033
1041
  lines.push(
1034
1042
  "",
@@ -48,7 +48,12 @@ import { spawnSync } from "node:child_process";
48
48
  import { existsSync, readFileSync, renameSync, rmSync, writeFileSync } from "node:fs";
49
49
  import { dirname, isAbsolute, join, resolve } from "node:path";
50
50
  import { findProject, loadConfig, resolveReleasePolicy } from "./config.ts";
51
- import { hasBotToken, readApprovalSurface, TELEGRAM_APPROVAL_TOOL } from "./approval-surface.ts";
51
+ import {
52
+ bridgeTokenBound,
53
+ hasBotToken,
54
+ readApprovalSurface,
55
+ TELEGRAM_APPROVAL_TOOL,
56
+ } from "./approval-surface.ts";
52
57
  import {
53
58
  briefPathForProject,
54
59
  policyPathForProject,
@@ -1054,6 +1059,15 @@ interface TickSession {
1054
1059
  * background noise.
1055
1060
  */
1056
1061
  approvalToolMissingLogged: boolean;
1062
+ /**
1063
+ * Whether omp-telegram could have bound a bot token when this session started.
1064
+ * False means its `telegram_ask` / `telegram_send` are dead for this session
1065
+ * however good `access.json` looks now, because the bridge resolves its token
1066
+ * once in `startBot()`. Defaults true so a session that never reaches
1067
+ * `session_start` — every unit test that calls `tick()` directly — keeps the
1068
+ * behaviour the access file describes.
1069
+ */
1070
+ bridgeTokenAtStart: boolean;
1057
1071
  /** Consecutive {@link PENDING_REASON} skips — see {@link STALL_MARKER_FILE}. */
1058
1072
  pendingSkips: number;
1059
1073
  }
@@ -1153,11 +1167,28 @@ function tick(pi: TickApi, ctx: TickContext, config: TickConfig, session: TickSe
1153
1167
  // approved here is the one that should read last.
1154
1168
  //
1155
1169
  // No access file configured means no fleet channel to judge, so nothing is
1156
- // claimed: the channel gate above already treats that as "not the fleet". A
1157
- // missing bot token is likewise not this check's business — it fails the
1158
- // channel gate outright, so a tick that reaches here can already send, and the
1159
- // only open question is whether an answer can come back.
1160
- const approval = config.accessFile === undefined ? undefined : readApprovalSurface(config.accessFile);
1170
+ // claimed: the channel gate above already treats that as "not the fleet".
1171
+ //
1172
+ // Two facts have to hold and the access file carries only one. It says whether
1173
+ // a destination would resolve; it cannot say whether the bridge holds a token
1174
+ // to resolve it with, because omp-telegram binds that once in `startBot()`. A
1175
+ // token written out-of-band after this session started opens the channel gate
1176
+ // — conductor pages tier 2 itself, so that much is honest — while leaving
1177
+ // `telegram_ask` and `telegram_send` dead until `/telegram on`. Trusting the
1178
+ // file alone there puts the tick straight back to mandating a call its surface
1179
+ // cannot make, which is #114 exactly.
1180
+ const approval =
1181
+ config.accessFile === undefined
1182
+ ? undefined
1183
+ : session.bridgeTokenAtStart
1184
+ ? readApprovalSurface(config.accessFile)
1185
+ : ({
1186
+ kind: "missing",
1187
+ reason:
1188
+ `${TELEGRAM_APPROVAL_TOOL} unavailable on local ticks: omp-telegram had no bot token when this ` +
1189
+ "session started, so it bound none and its tools stay dead however complete the access file looks " +
1190
+ "now — run `/telegram on` in this session, or restart it, to rebind the bridge",
1191
+ } as const);
1161
1192
  if (approval?.kind === "missing") {
1162
1193
  content = `${content}\n${TICK_APPROVAL_UNAVAILABLE_RULE}`;
1163
1194
  if (!session.approvalToolMissingLogged) {
@@ -1253,6 +1284,7 @@ export default function orchestratorTickExtension(pi: TickApi): void {
1253
1284
  const session: TickSession = {
1254
1285
  scopeFallbackLogged: false,
1255
1286
  approvalToolMissingLogged: false,
1287
+ bridgeTokenAtStart: true,
1256
1288
  pendingSkips: 0,
1257
1289
  };
1258
1290
  let releaseGateArmed = false;
@@ -1352,6 +1384,24 @@ export default function orchestratorTickExtension(pi: TickApi): void {
1352
1384
 
1353
1385
  const config = result.config;
1354
1386
 
1387
+ // Whether omp-telegram could have bound a token when this session started,
1388
+ // which is a different question from whether one exists now and is the only
1389
+ // one the approval preflight can honestly ask. The bridge resolves its token
1390
+ // once, in `startBot()`, and rebinds only on `/telegram token` or
1391
+ // `/telegram on` — so a token written into `.env` out-of-band leaves
1392
+ // `telegram_ask` and `telegram_send` dead for the life of this session even
1393
+ // though the file now looks perfect. Sampling here, next to the same
1394
+ // `session_start` omp-telegram binds on, is as close as another package can
1395
+ // get to that fact.
1396
+ //
1397
+ // Conservative on the other transition: an operator who ran `/telegram on`
1398
+ // after start really does have a working bridge, and this snapshot will keep
1399
+ // saying otherwise until the session restarts. That costs a fallback
1400
+ // instruction the orchestrator can follow, where guessing the other way
1401
+ // costs an amendment recorded as approved that nobody ever answered.
1402
+ session.bridgeTokenAtStart =
1403
+ config.accessFile === undefined ? true : bridgeTokenBound(config.accessFile);
1404
+
1355
1405
  // Activation is a property of the directory, so every omp session started in
1356
1406
  // the fleet's cwd used to become a ticker — and with merge and release
1357
1407
  // delegated in config, a shell opened beside the orchestrator believed it
package/src/store.ts CHANGED
@@ -85,6 +85,9 @@ const UPDATABLE_COLUMNS: Record<string, true> = {
85
85
  sessionFile: true,
86
86
  prUrl: true,
87
87
  headSha: true,
88
+ salvageSha: true,
89
+ salvageError: true,
90
+ salvageAckAt: true,
88
91
  startedAt: true,
89
92
  endedAt: true,
90
93
  lastError: true,
@@ -109,6 +112,9 @@ interface RunRow {
109
112
  sessionFile: string | null;
110
113
  prUrl: string | null;
111
114
  headSha: string | null;
115
+ salvageSha: string | null;
116
+ salvageError: string | null;
117
+ salvageAckAt: number | null;
112
118
  startedAt: number;
113
119
  endedAt: number | null;
114
120
  lastError: string | null;
@@ -146,6 +152,9 @@ CREATE TABLE IF NOT EXISTS runs (
146
152
  sessionFile TEXT,
147
153
  prUrl TEXT,
148
154
  headSha TEXT,
155
+ salvageSha TEXT,
156
+ salvageError TEXT,
157
+ salvageAckAt INTEGER,
149
158
  startedAt INTEGER NOT NULL,
150
159
  endedAt INTEGER,
151
160
  lastError TEXT
@@ -217,6 +226,9 @@ function toRecord(row: RunRow): RunRecord {
217
226
  if (row.sessionFile !== null) record.sessionFile = row.sessionFile;
218
227
  if (row.prUrl !== null) record.prUrl = row.prUrl;
219
228
  if (row.headSha !== null) record.headSha = row.headSha;
229
+ if (row.salvageSha !== null) record.salvageSha = row.salvageSha;
230
+ if (row.salvageError !== null) record.salvageError = row.salvageError;
231
+ if (row.salvageAckAt !== null) record.salvageAckAt = row.salvageAckAt;
220
232
  if (row.endedAt !== null) record.endedAt = row.endedAt;
221
233
  if (row.lastError !== null) record.lastError = row.lastError;
222
234
  return record;
@@ -324,12 +336,27 @@ export function openStore(dbPath: string): Store {
324
336
  if (!columns.some((column) => column.name === "headSha")) {
325
337
  db.exec("ALTER TABLE runs ADD COLUMN headSha TEXT");
326
338
  }
339
+ // v0.3.22 and earlier removed a blocked run's dirty tree without saving it
340
+ // (#118), so no row before this release has anywhere to record where the
341
+ // work went. A NULL `salvageSha` on a historical row therefore means "never
342
+ // asked", which reads the same as "clean tree" — the honest reading for a
343
+ // release that never salvaged one.
344
+ for (const [name, type] of [
345
+ ["salvageSha", "TEXT"],
346
+ ["salvageError", "TEXT"],
347
+ ["salvageAckAt", "INTEGER"],
348
+ ] as const) {
349
+ if (!columns.some((column) => column.name === name)) {
350
+ db.exec(`ALTER TABLE runs ADD COLUMN ${name} ${type}`);
351
+ }
352
+ }
327
353
 
328
354
  const insertRun = db.query<unknown, SqlValue[]>(
329
355
  `INSERT INTO runs (
330
356
  id, project, issue, repo, branch, worktree, state, attempt, turns,
331
- maxTurns, spendUsd, sessionFile, prUrl, headSha, startedAt, endedAt, lastError
332
- ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
357
+ maxTurns, spendUsd, sessionFile, prUrl, headSha, salvageSha, salvageError,
358
+ salvageAckAt, startedAt, endedAt, lastError
359
+ ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
333
360
  );
334
361
  const selectRun = db.query<RunRow, [string]>(`SELECT * FROM runs WHERE id = ?`);
335
362
  const selectActive = db.query<RunRow, SqlValue[]>(
@@ -370,6 +397,21 @@ export function openStore(dbPath: string): Store {
370
397
  `SELECT COUNT(*) AS n FROM runs
371
398
  WHERE project = ? AND issue = ? AND state IN ('killed', 'orphaned', 'blocked')`,
372
399
  );
400
+ // Salvage state that still describes an issue's present, so an operator is
401
+ // shown a preserved WIP tip exactly while it is the thing a re-claim would
402
+ // build on — and an unsalvaged tree for as long as it is the only copy.
403
+ // Newest-attempt-only for the same reason the board is: a later attempt has
404
+ // already consumed or superseded whatever an older one left behind.
405
+ const selectSalvaged = db.query<RunRow, [string]>(
406
+ `SELECT * FROM runs
407
+ WHERE rowid IN (
408
+ SELECT MAX(rowid) FROM runs
409
+ WHERE project = ?
410
+ GROUP BY issue
411
+ )
412
+ AND (salvageSha IS NOT NULL OR salvageError IS NOT NULL)
413
+ ORDER BY issue ASC`,
414
+ );
373
415
  // Newest attempt for one issue. `startedAt` is millisecond-resolution and two
374
416
  // attempts could in principle share one, so rowid breaks the tie by insertion
375
417
  // order — a `tail` that attached to the older of two same-millisecond attempts
@@ -487,6 +529,9 @@ export function openStore(dbPath: string): Store {
487
529
  toSql(record.sessionFile),
488
530
  toSql(record.prUrl),
489
531
  toSql(record.headSha),
532
+ toSql(record.salvageSha),
533
+ toSql(record.salvageError),
534
+ toSql(record.salvageAckAt),
490
535
  record.startedAt,
491
536
  toSql(record.endedAt),
492
537
  toSql(record.lastError),
@@ -532,6 +577,10 @@ export function openStore(dbPath: string): Store {
532
577
  return selectRecentRuns.all(project, mergedSinceEpochMs).map(toRecord);
533
578
  },
534
579
 
580
+ salvagedRuns(project: string): RunRecord[] {
581
+ return selectSalvaged.all(project).map(toRecord);
582
+ },
583
+
535
584
  attemptsFor(project: string, issue: number): number {
536
585
  return countAttempts.get(project, issue)?.n ?? 0;
537
586
  },
package/src/types.ts CHANGED
@@ -373,6 +373,17 @@ export interface RunRecord {
373
373
  prUrl?: string;
374
374
  /** Pull request head the worker observed after its deterministic CI watcher exited. */
375
375
  headSha?: string;
376
+ /** Commit this run's uncommitted work was preserved as before its worktree
377
+ * was removed, on the run's own branch. Absent means the daemon found
378
+ * nothing to save, or never looked — see {@link RunRecord.salvageError}. */
379
+ salvageSha?: string;
380
+ /** Why the salvage failed. Present means the worktree still holds the only
381
+ * copy of real work, so the tree was kept and the issue is held out of
382
+ * dispatch until an operator acknowledges it. */
383
+ salvageError?: string;
384
+ /** When an operator accepted the loss or recovered the tree by hand
385
+ * (`unblock --force`). Clears the hold without erasing what happened. */
386
+ salvageAckAt?: number;
376
387
  startedAt: number;
377
388
  endedAt?: number;
378
389
  /** Last failure text, surfaced verbatim in escalations. */
@@ -388,6 +399,7 @@ export type AdmissionHoldReason =
388
399
  | "sibling-active"
389
400
  | "open-pr-lookup-error"
390
401
  | "open-pr"
402
+ | "unsalvaged-wip"
391
403
  | "daily-spend-cap"
392
404
  | "unroutable:no-repo-label"
393
405
  | "unroutable:multiple-repo-labels"
@@ -470,6 +482,10 @@ export interface Store {
470
482
  /** Newest attempt per issue for the live board. Non-merged work remains
471
483
  * visible; merged rows are bounded by the supplied recent-history cutoff. */
472
484
  recentRuns(project: string, mergedSinceEpochMs: number): RunRecord[];
485
+ /** Newest attempt per issue that preserved work or failed to, so `status`
486
+ * can name every WIP tip a re-claim would build on and every tree that is
487
+ * still the only copy. */
488
+ salvagedRuns(project: string): RunRecord[];
473
489
  /** Total run segments, used only for the monotonically increasing run number. */
474
490
  attemptsFor(project: string, issue: number): number;
475
491
  /** Terminal implementation failures that consume `maxAttemptsPerIssue`. */
package/src/unblock.ts CHANGED
@@ -40,6 +40,11 @@ export interface UnblockOutcome {
40
40
  continuationsUsed: number;
41
41
  /** Newest attempt, when the store has one for this issue at all. */
42
42
  latest?: RunRecord;
43
+ /** Set when nothing was cleared because the newest attempt's work exists
44
+ * only in its worktree. Carries the salvage failure verbatim. */
45
+ refused?: string;
46
+ /** Set when `--force` recorded an operator's acceptance of that loss. */
47
+ forced?: true;
43
48
  }
44
49
 
45
50
  /**
@@ -81,11 +86,35 @@ export async function unblockIssue(
81
86
  tracker: Tracker,
82
87
  store: Store,
83
88
  issue: number,
89
+ opts: { force?: boolean } = {},
84
90
  ): Promise<UnblockOutcome> {
85
91
  // Read before any label is touched: terminality is the whole of the argument
86
92
  // for clearing in-progress, so the row that carries it decides the set.
87
93
  const latest = store.latestRun(project.name, issue);
88
94
  const terminal = latest !== undefined && !LIVE_STATES.includes(latest.state);
95
+ const counts = {
96
+ attemptsUsed: store.attemptsFor(project.name, issue),
97
+ failuresUsed: store.failuresFor(project.name, issue),
98
+ continuationsUsed: store.continuationsFor(project.name, issue),
99
+ };
100
+
101
+ // The one case where this verb refuses. Clearing the labels here re-queues an
102
+ // issue whose next claim starts by force-removing the worktree that holds the
103
+ // only copy of the last attempt's work (#118) — and unlike every other state
104
+ // this function reasons about, that is unrecoverable. `--force` is the way
105
+ // through, and it is deliberately a separate keystroke rather than a prompt
106
+ // the operator can wave past: it records, on the row, that a human accepted
107
+ // the loss or recovered the tree themselves. Narrowed to the row rather than
108
+ // a boolean so the reason printed and the acknowledgement written are
109
+ // provably about the same run.
110
+ const held =
111
+ latest !== undefined && latest.salvageError !== undefined && latest.salvageAckAt === undefined
112
+ ? latest
113
+ : undefined;
114
+ if (held !== undefined && opts.force !== true) {
115
+ return { cleared: [], ...counts, latest: held, refused: held.salvageError };
116
+ }
117
+ if (held !== undefined) store.updateRun(held.id, { salvageAckAt: Date.now() });
89
118
 
90
119
  const cleared: string[] = [];
91
120
  for (const label of new Set([
@@ -99,10 +128,9 @@ export async function unblockIssue(
99
128
 
100
129
  return {
101
130
  cleared,
102
- attemptsUsed: store.attemptsFor(project.name, issue),
103
- failuresUsed: store.failuresFor(project.name, issue),
104
- continuationsUsed: store.continuationsFor(project.name, issue),
131
+ ...counts,
105
132
  ...(latest === undefined ? {} : { latest }),
133
+ ...(held === undefined ? {} : { forced: true as const }),
106
134
  };
107
135
  }
108
136
 
@@ -116,6 +144,11 @@ export async function unblockIssue(
116
144
  * run row at all, where that label may still be sitting there unread. #18 was
117
145
  * filed against this function saying `next tick eligible again` in a case where
118
146
  * it was not, so the wording is a contract rather than prose.
147
+ *
148
+ * A refusal is the loudest thing this verb prints, and it prints instead of
149
+ * everything else: the operator asked to re-queue an issue whose only copy of
150
+ * real work is a directory, and the next line they read has to be the one that
151
+ * stops them typing the same command again with `--force` on the end.
119
152
  */
120
153
  export function formatUnblock(
121
154
  issue: number,
@@ -124,7 +157,24 @@ export function formatUnblock(
124
157
  caps: Caps,
125
158
  ): string {
126
159
  const latest = o.latest;
160
+
161
+ if (o.refused !== undefined) {
162
+ return [
163
+ `#${issue}: REFUSED — nothing was cleared`,
164
+ ` reason attempt ${latest?.attempt ?? "?"} could not commit its work: ${o.refused}`,
165
+ ` only copy ${latest?.worktree === undefined || latest.worktree === "" ? "(worktree path not recorded)" : latest.worktree}`,
166
+ " why re-claiming this issue removes that worktree, and the work is not on any ref",
167
+ ` recover inspect the tree and commit or copy what matters, then re-run with --force`,
168
+ ].join("\n");
169
+ }
170
+
127
171
  const lines = [`#${issue}: cleared ${o.cleared.join(", ")}`];
172
+ if (o.forced === true) {
173
+ lines.push(
174
+ ` forced the unsalvaged worktree was accepted as lost or already recovered by hand — ` +
175
+ "the next claim removes it",
176
+ );
177
+ }
128
178
 
129
179
  if (latest === undefined) {
130
180
  lines.push(" runs none recorded — the terminal labels were cleared anyway; eligibility is read off the tracker");
package/src/worktree.ts CHANGED
@@ -417,15 +417,21 @@ const SALVAGE_COMMIT_CONFIG = [
417
417
  * Subject stays the historical one-liner so status greps keep working; the
418
418
  * body is the manifest (#38). Cap the new-path list so a runaway tree cannot
419
419
  * push a multi-kilobyte commit message into every escalation.
420
+ *
421
+ * `ending` is the whole clause rather than a bare reason because not every
422
+ * salvaged run was killed. A run that blocked for a decision stopped on
423
+ * purpose (#118), and a commit message telling the operator reading it during
424
+ * recovery that the attempt was "killed by a blocked run" is a lie about the
425
+ * one artefact they are using to reconstruct what happened.
420
426
  */
421
427
  export function salvageCommitMessage(
422
428
  issue: number,
423
429
  attempt: number,
424
- reason: string,
430
+ ending: string,
425
431
  files: string[],
426
432
  newPaths: string[],
427
433
  ): { subject: string; body: string } {
428
- const subject = `wip(#${issue}): attempt ${attempt} killed by ${reason} — auto-salvaged`;
434
+ const subject = `wip(#${issue}): attempt ${attempt} ${ending} — auto-salvaged`;
429
435
  const n = files.length;
430
436
  const count = `${n} file${n === 1 ? "" : "s"}`;
431
437
  if (newPaths.length === 0) {
@@ -459,19 +465,27 @@ function parseCachedNameStatus(raw: string): { files: string[]; newPaths: string
459
465
  }
460
466
 
461
467
  /**
462
- * Commits a dead run's uncommitted work to the run's own branch and pushes it,
463
- * so that the tree the next attempt destroys is no longer the only copy.
468
+ * Commits a run's uncommitted work to the run's own branch and pushes it, so
469
+ * that the tree the next attempt destroys is no longer the only copy.
464
470
  *
465
471
  * This closes a deliberate asymmetry. `addWorktree` preserves the run branch
466
472
  * precisely because "that branch can hold the only copy of work attempt 1
467
473
  * committed but never pushed", while `removeWorktree` runs `worktree remove
468
474
  * --force` and `addWorktree` refuses to reuse a tree that "may hold a previous
469
475
  * attempt's uncommitted work" — committed work is kept by design, uncommitted
470
- * work is discarded by design. That trade is fair for a worker that *stops*:
471
- * blocking is a decision it makes with turns left to commit first. It is not
472
- * fair for one killed by the turns cap or the wall clock, or one that crashes:
473
- * that end is external, unannounced, mid-sentence, and it lands hardest on the
474
- * long refactors carrying the most unsaved work. So: non-graceful ends only.
476
+ * work was discarded by design.
477
+ *
478
+ * That trade was first taken only for non-graceful ends, on the argument that
479
+ * a worker which *stops* had turns left to commit for itself. #118 is what
480
+ * that argument cost. veltro#349 blocked on a real decision whether a
481
+ * failing test was obsolete — after editing 34 files across a request/auth
482
+ * boundary, and the daemon force-removed the tree seconds later; the branch
483
+ * and `origin/main` still pointed at the same commit, and the session
484
+ * transcript was the only surviving evidence the work had ever existed. A
485
+ * worker asking permission is precisely a worker declining to commit a
486
+ * half-migrated tree, so "had turns to commit" and "should have committed"
487
+ * were never the same claim. Every end whose tree is about to be removed
488
+ * salvages: killed, crashed, orphaned, and blocked alike.
475
489
  *
476
490
  * Never throws. Every outcome, including its own failure, comes back as a value
477
491
  * for the caller to log and to put in front of a human.
@@ -481,13 +495,13 @@ function parseCachedNameStatus(raw: string): { files: string[]; newPaths: string
481
495
  * commit in this host's mirror, which is strictly better than nothing. It is a
482
496
  * plain fast-forward push — never a force — and if the run already had a PR
483
497
  * open, that PR gains the WIP commit and re-runs its checks. That is the price
484
- * of work outliving its host, and only a run that already failed ever pays it.
498
+ * of work outliving its host.
485
499
  */
486
500
  export async function salvageWip(
487
501
  worktree: string,
488
502
  issue: number,
489
503
  attempt: number,
490
- reason: string,
504
+ ending: string,
491
505
  ): Promise<SalvageOutcome> {
492
506
  try {
493
507
  // A tree that is not there cannot be holding work. Checked before spawning
@@ -532,7 +546,7 @@ export async function salvageWip(
532
546
  return { kind: "nothing" };
533
547
  }
534
548
  const { files, newPaths } = parseCachedNameStatus(cached);
535
- const msg = salvageCommitMessage(issue, attempt, reason, files, newPaths);
549
+ const msg = salvageCommitMessage(issue, attempt, ending, files, newPaths);
536
550
  await git(
537
551
  [
538
552
  ...SALVAGE_COMMIT_CONFIG,