omp-conductor 0.18.2 → 0.19.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (60) hide show
  1. package/README.md +105 -40
  2. package/REFERENCE.md +865 -30
  3. package/package.json +1 -1
  4. package/schema/config.schema.json +26 -0
  5. package/src/admission.ts +212 -26
  6. package/src/ask.ts +288 -1
  7. package/src/briefs/orchestrator.md +6 -5
  8. package/src/cli.ts +5 -1
  9. package/src/command-help.ts +9 -1
  10. package/src/command-manifest.ts +36 -3
  11. package/src/commands/arm.ts +5 -1
  12. package/src/commands/context.ts +2 -0
  13. package/src/commands/message.ts +26 -2
  14. package/src/commands/reconcile-units.ts +104 -0
  15. package/src/commands/release-composition.ts +232 -0
  16. package/src/commands/resume.ts +2 -27
  17. package/src/commands/setup.ts +101 -16
  18. package/src/commands/stats.ts +11 -30
  19. package/src/commands/tail.ts +31 -1
  20. package/src/commands/upgrade.ts +20 -3
  21. package/src/commands/verb.ts +2 -1
  22. package/src/config-schema.ts +19 -0
  23. package/src/config.ts +80 -0
  24. package/src/credential-class.ts +366 -0
  25. package/src/daemon.ts +1218 -288
  26. package/src/dashboard/app.js +504 -2
  27. package/src/dashboard/controls.ts +336 -0
  28. package/src/dashboard/index.html +30 -0
  29. package/src/dashboard/server.ts +271 -30
  30. package/src/dashboard/style.css +116 -0
  31. package/src/dashboard/transcript.ts +173 -0
  32. package/src/doctor.ts +379 -22
  33. package/src/failure-class.ts +59 -0
  34. package/src/fleet.ts +511 -101
  35. package/src/host.ts +6 -130
  36. package/src/omp.ts +29 -0
  37. package/src/orchestrator-tick.ts +343 -88
  38. package/src/pause.ts +233 -0
  39. package/src/settlement.ts +159 -2
  40. package/src/setup-answers.ts +97 -0
  41. package/src/setup-host.ts +325 -1159
  42. package/src/setup-install.ts +204 -27
  43. package/src/setup-wizard.ts +111 -50
  44. package/src/setup.ts +33 -0
  45. package/src/spend-telemetry.ts +117 -0
  46. package/src/stats.ts +35 -0
  47. package/src/status-render.ts +348 -19
  48. package/src/store.ts +1229 -55
  49. package/src/telegram-freshness.ts +269 -0
  50. package/src/to-spec.ts +27 -0
  51. package/src/types.ts +697 -4
  52. package/src/unblock.ts +22 -0
  53. package/src/unit-reconcile.ts +303 -0
  54. package/src/upgrade-verify.ts +8 -1
  55. package/src/upgrade.ts +326 -47
  56. package/src/verbs/actions.ts +124 -10
  57. package/src/verbs/protocol.ts +70 -2
  58. package/src/verbs/server.ts +447 -8
  59. package/src/wake.ts +48 -0
  60. package/src/worker.ts +403 -3
package/REFERENCE.md CHANGED
@@ -180,6 +180,19 @@ than as drift (#511). It still flags genuine drift in `User=`, `ExecStart=`,
180
180
  other `Environment=` lines, `Restart=` and `SuccessExitStatus=`, and the unit
181
181
  itself keeps pinning `SHELL=` so panes do not fall back to dash (#463).
182
182
 
183
+ One finding is a conjunction rather than a fault. `bootstrap-deadlock` (#910)
184
+ fires when no worker on this host can start — because the *installed* conductor
185
+ cannot load its harness, exercised the way a worker does by running
186
+ `harness-loader.ts` under `bun --no-install` — and says the other half out loud:
187
+ a release cannot publish the fix either, because publishing needs a merged
188
+ worker pull request, so the broken install blocks its own replacement. It is
189
+ deliberately its own finding rather than a sentence appended to the stale-setup
190
+ ones, because the two states call for opposite actions: "run `setup host`" is
191
+ correct advice for a stale install and useless here, since no amount of
192
+ re-running setup replaces the installed package. Its fix names the one command
193
+ that breaks the cycle — `upgrade --bootstrap <sha> --source <checkout>` — and an
194
+ ordinary stale host still reads exactly as it did.
195
+
183
196
  ## Onboarding
184
197
 
185
198
  `omp-conductor setup` is the whole of it. One command, in a plain terminal, doing
@@ -271,6 +284,27 @@ names the first missing key instead of prompting. Add `--save-answers FILE`
271
284
  to an interactive run to capture only accepted answers for deterministic
272
285
  replay. Unknown keys are ignored so one file can cover a larger interview.
273
286
 
287
+ **A failed apply keeps its answers (#864).** The interview is the expensive half
288
+ and it usually is not what failed: on 2026-08-21 `setup policy` failed at its
289
+ post-interview gates several times, and every retry re-asked the whole merge,
290
+ release, arming and review interview — the same `runs-settled` value was typed
291
+ five times while the real fault was a runtime gate. So when the apply phase
292
+ throws, the confirmed answers are written to
293
+ `<state dir>/setup-resume-<project>.json` (mode `0600`, beside the config they
294
+ describe) and the failure names the exact command that replays them:
295
+
296
+ ```bash
297
+ omp-conductor setup policy --project conductor --resume ~/.omp/conductor/setup-resume-conductor.json
298
+ ```
299
+
300
+ `--resume` is `--answers` with a binding, and the binding is the point: the file
301
+ records a hash of the config it was answered against and the version that wrote
302
+ it. A changed config refuses, naming both generations — replaying an old plan
303
+ over newer policy would silently overwrite whatever changed it — and so does a
304
+ version mismatch, because the answer keys are a schema an upgrade may have
305
+ renamed. A completed setup deletes its own resume file, so a stale one cannot
306
+ invite a replay of answers that already landed.
307
+
274
308
  To fill in or revise just the brief later — the two `POLICY.md` sections above — run
275
309
  the `brief` area, which re-asks the judgment questions and re-runs the probes:
276
310
 
@@ -446,6 +480,23 @@ Per tick, for the daemon's project:
446
480
  4. **Filter and route.** An issue is eligible only if it carries the queue label
447
481
  and none of the three state labels (`inProgress`, `blocked`, `failed`). Eligible
448
482
  issues are partitioned into routable and unroutable.
483
+
484
+ A lifecycle label read from that **list** is a suspicion, not a verdict
485
+ (#891). `listReady` is a label-filtered search, and GitHub's search index is
486
+ eventually consistent, so an `unblock` that has already landed can still come
487
+ back carrying the label it cleared. Before a candidate is dropped for a
488
+ lifecycle label, the daemon re-reads that one issue's live labels
489
+ (`issueSnapshot`, a direct REST read) and admits normally when the exact read
490
+ is clean. Measured 2026-08-22T09:18Z: `unblock 889` cleared `agent:failed`,
491
+ `gh issue view` returned the clean set immediately, and the pass woken right
492
+ after still held it `stale-lifecycle` with the fleet at 0/5.
493
+
494
+ The re-read is bounded to candidates a lifecycle label would otherwise drop,
495
+ and skips two of those: an issue whose newest run is active (its own row is
496
+ the authority) and an operator-parked one (a park is a decision, not a stale
497
+ label). An unreadable re-read is **fail-closed** — the issue stays out of the
498
+ pass and is held as `issue-state-lookup-error`, never as `stale-lifecycle`,
499
+ because "the tracker could not say" is not evidence of a residual label.
449
500
  5. **Escalate the unroutable** at Tier 1, quoting the repo labels actually seen and
450
501
  the configured repo names. These are never dispatched.
451
502
  6. **Check spend.** If spend since local midnight has reached `dailySpendUsd`, the
@@ -797,6 +848,75 @@ the queue label is refused with `file-lane-unparseable` and an actionable
797
848
  message, because promoting beside overlapping work on a section that plainly
798
849
  tried to declare is the exact gap the interlock exists to close.
799
850
 
851
+ #### What status shows
852
+
853
+ `status` renders the two lifecycle kinds as two blocks, because "a branch is
854
+ holding a file" and "a worker is editing that file" need different answers:
855
+
856
+ - **`mutation leases`** — a live worker, or a dispatched unsettled review
857
+ revision. Each carries the run holding the lease, how long it has held it, the
858
+ read the interlock proves occupancy with (`worktree` for a live checkout,
859
+ `branch` for a worker-free row's mirror branch — the same condition
860
+ `probeRunLane` uses), and the files its declaration occupies.
861
+
862
+ Two further lines hang off a lease when they have something to say. **`pane`**
863
+ names the Herdr pane representing that worker, or `degraded` with the reason
864
+ there is none (#841) — see [the worker-pane
865
+ contract](#a-worker-gets-a-herdr-pane-but-the-pane-is-never-the-worker-840).
866
+ **`tokens`** appears only when a run's reasoning share of its output tokens is
867
+ disproportionate (≥80%, #518): the fact that distinguishes a run deliberating
868
+ expensively in wall clock but cheaply in dollars from one that is idle or
869
+ stuck. Run 42f73f52 read `24/180 turns $0.05` after 43 minutes and looked
870
+ barely started; 96% of its 106.3k output tokens were reasoning, its turns were
871
+ arriving at 200 seconds each, and the 180-turn cap was therefore unreachable —
872
+ the wall clock was the only ceiling that could ever fire. Both numbers come off
873
+ the same `usage` block the spend already comes from, so neither costs a
874
+ provider call, and both are recorded live rather than at settlement: the moment
875
+ this explains is forty minutes before `wall-clock-cap-spinning` (#494) fires.
876
+ An ordinary reasoning share prints nothing, and a run with no token counts
877
+ makes no claim about them — absent is not 0%.
878
+ - **`preserved artifacts`** — worker-free `pushed-green` / `pushed-pending` rows.
879
+ Durable work with nothing editing it, so it is shown with how long it has been
880
+ waiting and on what: a PR to review and merge, or (with no PR) a branch
881
+ `conductor_pr_recover` can still publish. No turn rate and no cap projection —
882
+ those describe a session in progress, and printing them beside work nobody is
883
+ running is how a settled artifact came to read as a busy one.
884
+
885
+ Membership is the union of the run's own live state and the snapshot's recorded
886
+ `leasedRunIds`, never the ids alone: a live worker is a lease by definition, and
887
+ the recorded set exists to add the case a state read cannot see — a dispatched
888
+ revision on a row that still says `pushed-green`. Both blocks are rendered from
889
+ the snapshot alone, so a status read never probes git and never mutates anything.
890
+
891
+ #### What holds a lane, and what merely owns an artifact
892
+
893
+ Occupancy follows the **mutation lease** (#899): the runs something is writing
894
+ through right now — a live worker (`claimed`/`running`), or a review revision the
895
+ daemon has dispatched and that has not settled. A worker-free `pushed-green` /
896
+ `pushed-pending` row still owns its issue and its PR, so its work is never
897
+ re-implemented, but it no longer holds its files: holding them meant one settled
898
+ artifact idled every overlapping candidate until a human merged it.
899
+
900
+ The trade is deliberate. A merge conflict is a later, recoverable event
901
+ (`conductor_pr_update_branch`, or the `merge-conflict` failure class); an idle
902
+ fleet is an immediate, unrecoverable one.
903
+
904
+ Because a released branch's files may legitimately be taken by someone else,
905
+ every path that *resumes* writing one re-proves the lane first (#925):
906
+
907
+ - a **continuation** is held (`file-lane`) when the branch it resumes actually
908
+ carries a file a lease holds — read from the branch, not from the candidate's
909
+ declaration, which may be narrower or absent;
910
+ - **`conductor_pr_recover`** refuses (`recovery-lane-occupied`) rather than
911
+ publishing a frozen branch into review beside a worker still rewriting one of
912
+ its files. Adopting an already-open PR is not gated: it publishes nothing.
913
+
914
+ The two sides fail in opposite directions, on purpose. Admission is unattended
915
+ and continuous, so an unreadable probe admits (fail open) rather than stalling
916
+ the queue on a transient git read. Recovery is one deliberate call whose refusal
917
+ is legible and retryable, so an unreadable lane refuses
918
+ (`recovery-lane-unprovable`) rather than publishing on an unproven one.
919
+
800
920
  ## Routing
801
921
 
802
922
  An issue must carry **exactly one** `repo:<name>` label naming a repo in
@@ -875,7 +995,8 @@ rest. `0` is a real value (a hard stop), not "unset".
875
995
  | --- | --- | --- |
876
996
  | `maxConcurrentWorkers` | `2` (setup may write `1` on &lt;16 GiB hosts) | Parallel omp sessions, each a child process of the daemon and all inside its cgroup. Two, because **CI runner slots, not model tokens, are the usual throughput ceiling** — a third worker would starve its own PR checks on a small self-hosted runner pool. On hosts under ~16 GiB RAM, prefer `1` so the unit stays out of swap ([host sizing](#host-sizing-and-memory)). Raise it only if you actually have the runners *and* the RAM. |
877
997
  | `maxConcurrentWorkersPerRepo` | `1` | Max live workers in the **same repo**. The mirror, branch-protection staleness and shared CI egress are all per-repo collision domains, so extra slots should land on other repos. Raise it only when a repo genuinely needs two workers at once. |
878
- | `dailySpendUsd` | `25` | Rolling-day spend ceiling in USD, or `null` for no spend gate. `0` is a hard stop. Metered from assistant `usage.cost.total`. |
998
+ | `dailySpendUsd` | `25` | Rolling-day spend ceiling in **estimated** USD, or `null` for no spend gate. `0` is a hard stop. Metered from assistant `usage.cost.total` — OMP's own local price for the tokens a session recorded, not the provider's bill. |
999
+ | `maxRunSpendUsd` | `null` (derived: `dailySpendUsd / maxConcurrentWorkers`) | Ceiling on what **one run** may spend, and the amount admission reserves from the day's budget before launching it. Positive, or `null` to derive; never larger than `dailySpendUsd`. See [Per-run spend reservations](#per-run-spend-reservations-851). |
879
1000
  | `planUsage` | `null` (unmetered) | Subscription/plan allowance guard: `{ "windowId": "anthropic:7d", "maxUsedFraction": 0.85 }`, or `null` for no plan gate. Independent of `dailySpendUsd` — see [Plan allowance](#plan-allowance-planusage) below. |
880
1001
  | `workerMaxTurns` | `120` | Base ceiling for each new worker. Catches a session looping without converging; use `omp-conductor extend` to raise one live run or one issue's next attempt without changing this default. |
881
1002
  | `workerMaxTurnsCeiling` | `240` (twice the effective `workerMaxTurns` when omitted) | Upper bound for per-issue turn extensions. Prevents the loopback control from granting an unbounded worker budget. |
@@ -891,6 +1012,149 @@ pages at Tier 2**: a loop that is burning money has to halt itself, because
891
1012
  waiting for someone to notice tomorrow is how a runaway becomes expensive.
892
1013
  Work resumes only after `omp-conductor resume`.
893
1014
 
1015
+ ### Install-surface parity, recorded (#919)
1016
+
1017
+ `omp-conductor` installs onto three surfaces — the Bun-global CLI/daemon tree,
1018
+ the omp plugin, and the herdr recovery plugin — and on a host whose installs are
1019
+ manual they diverge silently. Measured on this host, 2026-08-22: the omp plugin
1020
+ sat on the withdrawn 0.18.1 release beside a 0.18.0 daemon for about a day, with
1021
+ no finding, no status line and no escalation.
1022
+
1023
+ The dispatch pass records the three identities once per pass, and the cheap
1024
+ surfaces read that row:
1025
+
1026
+ - `status` is silent while they agree, prints the mismatch with
1027
+ `omp-conductor upgrade --to <version>` when they disagree, and says
1028
+ **`not observed yet`** before the first pass has looked — never "they agree";
1029
+ - the tick digest names a **proven** mismatch only.
1030
+
1031
+ Neither spawns a process for it, and that is the design rather than an
1032
+ optimisation: probing costs three subprocesses, and doing it at render time took
1033
+ the tick's own suite from 8.4s to 83.4s while spawning three children every
1034
+ fifteen minutes to answer a question that changes only when someone installs
1035
+ something.
1036
+
1037
+ An absent surface, a `local:` herdr link and a pin whose release could not be
1038
+ verified are steady states someone chose, so they produce no status row and no
1039
+ tick line — a warning repeated every fifteen minutes trains an operator to
1040
+ ignore the row that matters. `omp-conductor doctor` keeps the full nuance,
1041
+ including those cases, and is still the on-demand answer.
1042
+
1043
+ ### The one peer plugin that is also a contract (#961)
1044
+
1045
+ The same pass records three more versions, about `omp-telegram`: what is
1046
+ installed, what its running daemon reports, and what is published. That plugin
1047
+ gets this treatment and no other because a *conductor* instruction depends on its
1048
+ behaviour — the floor tells every session answering an inbound message to call
1049
+ `telegram_send` with neither `chat_id` nor `thread_id`, so the reply stays in the
1050
+ topic it arrived in.
1051
+
1052
+ Measured on this host: the installed 0.12.1 predated that ladder. Its send path
1053
+ had two rungs (explicit `chat_id`, then in-process `lastTarget`), so the mandated
1054
+ call refused with `no active telegram chat` three times on 2026-08-21, and each
1055
+ refusal cost an improvised recovery — once by passing route ids by hand, which is
1056
+ precisely the anti-pattern #882 names as its own silent fake. The published
1057
+ 0.12.2 has the four-rung ladder plus per-rung diagnostics. **The fix had been on
1058
+ npm for days, and no surface said so**, so three sessions did transcript
1059
+ archaeology to reach a conclusion `npm view` answers in one call.
1060
+
1061
+ - `status` grows a `tg-plugin` row **only** on a proven divergence: installed
1062
+ behind published, or a daemon serving something other than what is installed.
1063
+ Silent otherwise, and silent when nothing was recorded — a fleet that does not
1064
+ use Telegram must not grow a permanent row about a plugin it does not have.
1065
+ - `doctor` reports the finding at full nuance, always `warn` and never `fail`:
1066
+ the fleet dispatches, merges and reports fine on a stale plugin. One
1067
+ instruction does not, and the remedy is an install the operator owns.
1068
+
1069
+ Behind-published outranks daemon-stale deliberately, because the remedies
1070
+ compose: an install without a restart leaves the daemon on the old code. And an
1071
+ unreachable registry is **`unknown`**, never `current` — a check that could not
1072
+ run must not certify, and a host with no outbound npm access must not grow a
1073
+ permanent warning about it. No minimum version is compiled in: a baked-in
1074
+ baseline goes stale in exactly the way this check exists to catch, and would
1075
+ re-create #904's divergence with one more copy of the number.
1076
+
1077
+ ### Per-run spend reservations (#851)
1078
+
1079
+ A daily cap alone is a *post-spend* stop: it compares spend-to-date at
1080
+ admission and pauses the fleet on the **next** pass, so one expensive run can
1081
+ cross the ceiling by its whole cost before anything objects. Measured on this
1082
+ package's own fleet, 2026-08-21: one `@slow` run spent an estimated $25.16
1083
+ against a $25.00 daily cap, and the fleet paused only after it had settled.
1084
+
1085
+ So the cap is enforced in two places, and both are mechanical:
1086
+
1087
+ - **At admission, as a reservation.** Before a run launches, its whole per-run
1088
+ allowance is reserved out of the day's budget and recorded on the run row. A
1089
+ candidate is held (`spend-reservation`) when spend-to-date plus every live
1090
+ run's reservation plus its own would exceed `dailySpendUsd`. The reservation
1091
+ lives on the row, so a daemon restart re-reads it instead of forgetting it,
1092
+ and a run that settles for less than it reserved releases the remainder
1093
+ simply by ceasing to be live — there is no release step to forget.
1094
+ - **Inside the run, as a ceiling.** A session that reaches its allowance is
1095
+ killed like any other cap kill, through the ordinary salvage path, so its
1096
+ work is preserved and its PR (if any) survives. Response cost is known only
1097
+ when a response completes, so the **declared overshoot is exactly one model
1098
+ response** — never zero, and never a whole run.
1099
+
1100
+ The allowance is `maxRunSpendUsd` when set, and otherwise
1101
+ `dailySpendUsd / maxConcurrentWorkers`: the largest per-run reservation that
1102
+ still lets the fleet run at its configured concurrency. Reserving the whole
1103
+ daily cap per run would bound overshoot just as well and would silently
1104
+ serialise every fleet on defaults, which is a concurrency change disguised as a
1105
+ spend guard; N reservations of cap/N are still the cap, so nothing is given
1106
+ away. Set `maxRunSpendUsd` explicitly to bound one run harder.
1107
+
1108
+ `status` labels the figure **estimated** for the same reason the table above
1109
+ does: on the 2026-08-21 incident the local estimate read $25.16 while roughly
1110
+ $16 of provider credit actually moved, so presenting it as billed spend would
1111
+ assert something conductor cannot see. Reservations are shown beside it, because
1112
+ that — not spend-to-date — is what the next admission subtracts.
1113
+
1114
+ #### And whether that figure means anything (#970)
1115
+
1116
+ Both halves of the cap read one column: `spendUsd`, summed by
1117
+ `store.spendSince`. When the harness stops reporting cost the column reads
1118
+ `0.00`, the sum shrinks, and the cap compares a fraction of reality against its
1119
+ ceiling — #46's *"the daily spend cap is theater"*, recurring. The check's own
1120
+ wording has always said the quiet part: **$0.00 spend is not proof of no spend.**
1121
+
1122
+ Measured on this fleet, counting only runs that did any work:
1123
+
1124
+ | day | worked | reported $0.00 |
1125
+ |---|---|---|
1126
+ | 2026-08-15 → 20 | 15–118 | 0.0–3.8% |
1127
+ | **2026-08-21** | 22 | **31.8%** |
1128
+ | **2026-08-22** | 12 | **75.0%** |
1129
+
1130
+ Six days of baseline, then a regression — with one run taking 213 turns for
1131
+ $0.00. Three things were wrong with the detection, and all three are fixed:
1132
+
1133
+ - **It was all-or-nothing.** The predicate was `window.every(spendUsd === 0)`,
1134
+ so it fired only on total loss. Replayed over all 410 windows in this fleet's
1135
+ recorded history it would have fired 14 times while staying silent through
1136
+ **37** windows that had lost a majority — partial loss, the common case, was
1137
+ the invisible one. On 2026-08-21 every window contained one metered run, so it
1138
+ was silent all day. It is now a share of the window, reported at a majority.
1139
+ - **It counted runs that never worked.** A kill before the first turn records
1140
+ `$0.00` honestly, and 26 of this project's 48 zero-spend rows are exactly
1141
+ those. They are excluded from both sides now, so a burst of administrative
1142
+ kills neither fires a telemetry finding nor dilutes a real one. The sampler
1143
+ over-reads to compensate.
1144
+ - **It was on-demand only.** `spend-telemetry` lived in `doctor` and nowhere
1145
+ else, which is precisely the gap #919 closed for install surfaces — and the
1146
+ stake here is a spend *control*. `status` now carries a `spend telemetry` row
1147
+ directly beneath the figure it qualifies, computed from the same store read,
1148
+ silent while telemetry is healthy.
1149
+
1150
+ The threshold is a majority rather than any single zero, because the healthy
1151
+ baseline is 0–3.8% and not zero: a legitimately cheap run happens, and a row
1152
+ that fires on one of them is a row an operator learns to skip.
1153
+
1154
+ Deliberately absent: any attempt to *estimate* the missing cost from turns or
1155
+ tokens. That would convert a known unknown into a confidently wrong number, on
1156
+ the one control whose job is to stop the fleet spending money.
1157
+
894
1158
  `workerMaxTurns` and `workerWallClockMs` are enforced inside the session driver.
895
1159
  The daemon reads a live run's effective turn ceiling at every turn boundary. Use
896
1160
  `omp-conductor extend <issue> --turns N [--project NAME]` to raise it without
@@ -1019,6 +1283,123 @@ honour the pattern it says so, and the daemon logs that per run:
1019
1283
  Worth reading the log for. A run that quietly used a weaker model than you chose
1020
1284
  otherwise looks like a run that was merely unlucky.
1021
1285
 
1286
+ ### Cap escalation (`workerEscalationModel`)
1287
+
1288
+ A run that reaches a ceiling with no PR, no head and no salvage commit is
1289
+ classified `turn-cap-spinning` or `wall-clock-cap-spinning`, and normally that
1290
+ is a decomposition verdict: the slice was too big, and switching models would
1291
+ hide it. The cap classes are excluded from `modelFallbacks` for exactly that
1292
+ reason.
1293
+
1294
+ `workerEscalationModel` buys one exception per issue, for the case where the
1295
+ slice was fine and the tier was not:
1296
+
1297
+ ```json
1298
+ "workerEscalationModel": "@slow"
1299
+ ```
1300
+
1301
+ It is an opaque selector like `workerModel` — a role, a pattern, whatever omp
1302
+ resolves — and conductor never interprets it. On the **first** artifact-free
1303
+ cap kill in an issue's run chain the settlement sweep records a one-shot marker
1304
+ and hands the queue label back — one store transaction, so a daemon that dies
1305
+ mid-recovery restarts into either "nothing happened" or "the continuation is
1306
+ owed", never a chain that reads as escalated with nothing queued — and the
1307
+ next dispatch launches on that selector; the page it sends says so instead of
1308
+ asking a human to requeue by hand. The run
1309
+ row records the selector it dispatched on (`model`) and, once the session
1310
+ settles, the model the harness actually resolved (`resolvedModel`).
1311
+
1312
+ What it deliberately does not do:
1313
+
1314
+ - **It never fires twice for one issue.** The marker is never removed, so a
1315
+ restart cannot repeat it and a second cap follows the existing escalation
1316
+ path — no climb to a third tier.
1317
+ - **It never fires for a `-progress` cap.** A cap kill with work to continue from
1318
+ keeps decomposing.
1319
+ - **It is not a provider fallback.** `modelFallbacks` answers provider aborts,
1320
+ credit refusals and capacity throttling, and still does: an escalated chain
1321
+ fails over normally on top of its new primary, and a cap never advances or
1322
+ resets the provider streak.
1323
+
1324
+ Omit it and nothing changes: a spinning cap escalates to a human exactly as it
1325
+ did before, and the page says no escalation target is configured.
1326
+
1327
+ ### Required credential class (`requireOauthProviders`)
1328
+
1329
+ On 2026-08-21 an Anthropic OAuth grant's refresh failed with `invalid_grant`. The
1330
+ exact selector `anthropic/claude-opus-5` still resolved — right provider, right
1331
+ model — through the *same provider's* API key, and one run spent an estimated
1332
+ $25.16 on billing nobody chose.
1333
+
1334
+ That is documented harness behaviour, not a bug. `AuthStorage.getApiKey`
1335
+ resolves first-match-wins across runtime override → config override → stored
1336
+ OAuth → login-sourced API key → provider env var → other stored API key →
1337
+ fallback resolver, and **a model selector does not stop that cascade**. Nothing
1338
+ omp exposes can constrain it: `models.yml` `auth: "oauth"` only shapes the
1339
+ request, a provider-id variant like `xai-oauth` is a real provider that also
1340
+ accepts `XAI_API_KEY`, and there is no setting, env var, selector suffix or
1341
+ request option for it. So this is conductor's fence, not a config passthrough:
1342
+
1343
+ ```json
1344
+ "requireOauthProviders": ["anthropic", "openai-codex"]
1345
+ ```
1346
+
1347
+ **Declared per provider, never per model or per run.** A credential belongs to a
1348
+ provider — a model has none of its own — and a run's provider is not knowable in
1349
+ advance: `workerModel` is an opaque selector that may be a role alias, and the
1350
+ provider-fault fallback chain can move a live run onto another provider
1351
+ mid-flight. A per-run inference would be a guess dressed as a fact. So the
1352
+ declaration reads "this provider must bill to its subscription", and while that
1353
+ is untrue every candidate waits.
1354
+
1355
+ It is checked at **three** points, and each closes a window the others cannot:
1356
+
1357
+ - **Admission** holds every candidate with the `credential-class` reason, and the
1358
+ hold's detail names the provider, what it resolves to instead, the disabled
1359
+ cause (`invalid_grant` — the actionable half) and the remediation. Fleet-wide
1360
+ like `plan-usage-cap`, and self-clearing: re-authenticate and the next pass
1361
+ admits. Nothing pages, because a page for something an operator fixes in a
1362
+ minute costs more attention than the guard saves.
1363
+ - **Before provisioning**, so a run that cannot launch never clones a mirror or
1364
+ cuts a worktree.
1365
+ - **Immediately before the spawn**, which is the only one that closes the
1366
+ admission-to-session window — a grant can be disabled in it. A run closed here
1367
+ settles `stopped`, with the reason in its report, and **does not spend a failed
1368
+ attempt**: the issue is not what is wrong, the host's credentials are, and
1369
+ charging an attempt would exhaust an issue while the operator re-authenticates.
1370
+
1371
+ The reader is a runnable probe (`src/credential-class.ts`) spawned per check with
1372
+ `--no-install`, exactly as `harness-loader.ts` asks the installed harness its own
1373
+ question. Out of process for two reasons: it keeps the harness's module graph and
1374
+ credential database out of the long-lived daemon, and every read is taken *now*
1375
+ rather than against a pool this process cached an hour ago. It calls the harness's
1376
+ own exported API — `getCredentialOrigin`, `listDisabledCredentials` — never a
1377
+ query against `agent.db`, and prints only an origin kind, an optional
1378
+ environment-variable *name* and disabled classes with their causes. There is
1379
+ nothing in that surface that could carry credential material.
1380
+
1381
+ **This gate fails closed, and that is deliberate.** A probe that cannot answer —
1382
+ no harness, no `bun`, malformed output, an answer about a different provider —
1383
+ refuses, because "we could not check" costs exactly what "it is wrong" costs.
1384
+ That is the opposite posture from the lane interlock, which is inert without its
1385
+ probe.
1386
+
1387
+ **The residue, which this does not fix and must not pretend to.** It is a state
1388
+ check, not an atomic guarantee. If OAuth becomes invalid *during* a request, the
1389
+ installed resolver still disables it and falls through to a same-provider API
1390
+ key, and nothing conductor can pass prevents that: there is no class constraint,
1391
+ and no after-the-fact answer either — `getApiKey` returns a bearer string with no
1392
+ class, assistant messages carry no credential field, and session `credential_pin`
1393
+ entries are OAuth-only *and* change-only, so neither a stale pin nor a missing one
1394
+ proves anything. A transcript audit of billing class would be a fabrication. The
1395
+ only hard lever is omp's own `disabledProviders`, which removes the provider
1396
+ before credential checks — and also removes the subscription route, which is why
1397
+ it stays an operator action. The real fix is upstream: an `AuthApiKeyOptions` that
1398
+ can require a credential class and refuse rather than descend.
1399
+
1400
+ Omit it and nothing changes — no probe runs, and dispatch is byte-for-byte what
1401
+ it was.
1402
+
1022
1403
  ## Omp settings overlay
1023
1404
 
1024
1405
  `ompSettings` on a project is the fleet-owned channel for saying "this project's
@@ -1333,6 +1714,18 @@ the command prints that held-notice id instead of claiming delivery — a
1333
1714
  not default to the marker. It is also not a report — it
1334
1715
  leaves no `reports` row, and nothing retries it.
1335
1716
 
1717
+ Every held notice also names the live path (#882). A hold reaches nobody now, so
1718
+ it is the wrong answer for somebody who is waiting: on 2026-08-21T20:48Z an
1719
+ interrupted tick fell back to this command and the operator's answer sat in the
1720
+ outbox until the 23:30 digest while they waited in the topic they had written in.
1721
+ The hold was correct policy and "nothing was sent" was already printed; what was
1722
+ missing is that `telegram_send` (or `telegram_ask` for a choice) reaches that
1723
+ person in their own topic right now. The line is conditional, because this
1724
+ command runs in a different process from the session and cannot know whether an
1725
+ inbound message is waiting — guessing would either nag every legitimate
1726
+ digest-only note or quietly bypass the availability policy on a hunch. The caller
1727
+ knows; one sentence makes holding a choice rather than a default.
1728
+
1336
1729
  ### Delivery is at-least-once, and the docs will not pretend otherwise
1337
1730
 
1338
1731
  The Telegram Bot API accepts no client-supplied idempotency key and offers the
@@ -1515,13 +1908,33 @@ omp-conductor watch list
1515
1908
  A watch is a row `kind: watch` that the orchestrator opened for itself — either
1516
1909
  a `--resolves-when` condition (the daemon checks it and wakes the next tick when
1517
1910
  met, exactly as for a question), or a plain carry note the next tick should
1518
- read. It renders under its own "Watches" heading, is never offered to you to
1519
- resolve, and has **no seven-day expiry** that deadline is for a question
1520
- nobody answered; silently expiring a watch could drop a release the day its
1521
- condition finally fires. A superseded watch is closed with the ordinary
1911
+ read. It renders under its own "Watches" heading and is never offered to you to
1912
+ resolve. A superseded watch is closed with the ordinary
1522
1913
  `decision withdraw <id>`. `decision open --resolves-when` still records an
1523
1914
  operator question: a condition governs *when* to ask, not *who* answers.
1524
1915
 
1916
+ **The seven-day deadline starts when the condition fires, not when the watch was
1917
+ opened** (#966). While a watch is waiting — an unfired condition, or no condition
1918
+ at all — it never expires, because that deadline is for a question nobody
1919
+ answered and silently dropping a watch could lose a release the day its condition
1920
+ finally fires. The moment `conditionMetAt` is stamped the row stops waiting: it
1921
+ is an actionable item, which is exactly what the question deadline bounds, so it
1922
+ gets the ordinary window measured from that instant and closes itself if nobody
1923
+ acts.
1924
+
1925
+ Measured on this fleet 2026-08-23, which is why: three watches had carried
1926
+ `[CONDITION MET — act on this now]` for 10–12 hours over work that was entirely
1927
+ closed (#809 and #888 closed, `omp-conductor@0.18.2` published *and* installed),
1928
+ and would have carried it forever. The codebase already names that cost for
1929
+ install surfaces — *a warning repeated every fifteen minutes trains an operator
1930
+ to ignore the row that matters* — and a permanent "act on this now" is that,
1931
+ aimed at the orchestrator's own attention.
1932
+
1933
+ Two details keep it honest. Re-observing an already-met condition is refused, so
1934
+ the deadline cannot be pushed out on every pass. And clearing a met condition —
1935
+ which `pr-checks-green` and `pr-review-ready` do when the head moves (#808) —
1936
+ restores `never`: a PR that went red again is waiting, not ignored.
1937
+
1525
1938
  ## Failure classes and recovery by class (#132)
1526
1939
 
1527
1940
  Every run that did not reach a merged PR used to end at a human. The
@@ -1593,11 +2006,57 @@ Positive evidence only: a tracker that cannot list answers empty, and an empty
1593
2006
  answer removes nothing — the label is the interlock that keeps two workers off
1594
2007
  one issue.
1595
2008
 
2009
+ ### Grooming verdicts are reconciled too (#964)
2010
+
2011
+ Same reasoning, one surface over. A verdict describes an **open** issue —
2012
+ `promotable` means "promote this" — and nothing retired one when its issue
2013
+ closed. `reconcileGrooming` clears only the two per-pass admission holds
2014
+ (`file-lane`, `depends-on`), which are correctly self-clearing; every other row
2015
+ was written once and never removed.
2016
+
2017
+ Measured on this fleet 2026-08-23: **64 of 65 rows named a closed issue, and all
2018
+ 26 `promotable` ones did**, so `status` advertised 26 candidates ready to promote
2019
+ on a repo whose only open issue was blocked on an operator. The tick prompt said
2020
+ "Promote the promotable or groom new issues" over the same rows.
2021
+
2022
+ Each pass now retires every row whose issue is absent from a complete
2023
+ `listOpenIssues` snapshot. Three properties make that safe rather than
2024
+ destructive:
2025
+
2026
+ - **A complete snapshot, never a filtered one.** `listOpenIssues` paginates to
2027
+ the end and is conditional (an unchanged repo costs a 304). Reconciling against
2028
+ `listReady` or `listLabeled` would condemn every verdict the filter did not
2029
+ mention — and a promotable issue carries no label at all, which is also why
2030
+ this cannot live inside the label reconcile above.
2031
+ - **Fail closed.** An unreadable tracker retires nothing and logs why. An
2032
+ unreadable tracker and "no issues are open" look identical, and acting on the
2033
+ second the first time GitHub rate-limits a pass would empty the table.
2034
+ - **Silent when nothing changed.** A line every fifteen minutes saying "0
2035
+ retired" is a line nobody reads.
2036
+
2037
+ Note what was *not* broken: candidate selection already built its pool from a
2038
+ live open-issue snapshot (#848) and used grooming rows only as exclusions, so a
2039
+ closed issue was never offered to a to-spec batch. The rot was in what the
2040
+ surfaces reported, and — latently — in `claimable = routed − knownBlocked`, which
2041
+ is arithmetic over the one subset that *was* being reconciled.
2042
+
1596
2043
  ### Where you see it
1597
2044
 
1598
- - `omp-conductor status` grows a `failure classes (unrecovered)` block, counting
1599
- only rows whose recovery has *not* run. Classes rather than row states,
1600
- because a row state is not an issue state.
2045
+ - `omp-conductor status` grows up to two blocks, and they answer different
2046
+ questions. **`failure classes (awaiting recovery)`** counts rows whose
2047
+ recovery has not run *and* whose action the sweep will actually take; it is
2048
+ omitted when empty, because empty is the healthy answer. **`failure classes
2049
+ (classified, no action by design)`** counts the two recorded-only actions,
2050
+ `none` and `hold` — a `hold` waits on a human by design, and `none` is what
2051
+ the settle sweep writes beside `returned-for-revision` when a reviewer closes
2052
+ pushed work without merging, where the remedy is the queue label the sweep
2053
+ deliberately leaves on rather than anything a sweep does.
2054
+
2055
+ They were one list until it grew a monotonically increasing tally: on this
2056
+ fleet `returned-for-revision` reached **44** while every actionable class sat
2057
+ at zero and invisible beneath it. That is the #109 defect one level up — a row
2058
+ state is not an issue state, and a count nobody can act on is not a backlog.
2059
+ Classes rather than row states, for the same original reason.
1601
2060
  - The board appends `[<class>]` to a card whose newest run carries one.
1602
2061
  - The tick prompt carries one line — `Auto-recovered since last tick: 3
1603
2062
  (merge-conflict #365, admin-kill #82, …) — already handled, do not re-triage
@@ -1927,16 +2386,41 @@ and a decision made out of a model's own wording is one no two runs spell the
1927
2386
  same way. A reason outside its set is refused, and the refusal names every
1928
2387
  accepted value.
1929
2388
 
2389
+ Read from `VERB_SPECS` on 2026-08-23. This table had drifted twice — `label
2390
+ change` was missing `completed`, and `pr review` was absent altogether — which is
2391
+ the ordinary hazard of writing a closed set down in two places. The refusal names
2392
+ the live set, and is the authority when they disagree.
2393
+
1930
2394
  | Verb | Accepted reasons |
1931
2395
  | --- | --- |
1932
2396
  | merge | `preconditions-met`, `behind-base-refreshed`, `operator-instructed`, `release-blocking` |
1933
2397
  | release | `batch-complete`, `epic-closed`, `hotfix`, `operator-instructed` |
1934
- | label change | `promoted-to-queue`, `re-briefed`, `needs-human`, `duplicate`, `superseded`, `out-of-scope` |
2398
+ | label change | `promoted-to-queue`, `re-briefed`, `completed`, `needs-human`, `duplicate`, `superseded`, `out-of-scope` |
2399
+ | pr review | `blocking-findings` |
1935
2400
 
1936
2401
  Free-form rationale still has a home: it rides alongside as a separate
1937
2402
  `rationale` field, is written into the audit trail verbatim, and is never
1938
2403
  parsed or matched by anything.
1939
2404
 
2405
+ **And the refusal says so** (#968), because for a long time it did not. An
2406
+ out-of-enum `reason` was refused with the field, every accepted value and the
2407
+ rejected text quoted back — everything except where to put the sentence. Measured
2408
+ on this fleet 2026-08-23, `reason-not-in-enum` was the **largest single refusal
2409
+ cause in the ledger**: 99 of 241, across all four closed-set verbs, over 7.6
2410
+ days. Every one of the 99 carried real audit content — *"landed directly in PR
2411
+ #940; queue label would re-dispatch finished work"* — and **none supplied
2412
+ `rationale`**, because the moment a caller has a sentence to record is the moment
2413
+ they are told only that `reason` is closed. 7578 characters of accounting were
2414
+ refused and discarded.
2415
+
2416
+ So the refusal now ends: *Pick the enum value that fits and put that sentence in
2417
+ `"rationale"`, which is logged verbatim.* It is keyed on the verb's own spec, not
2418
+ a hardcoded list — advising a field the verb would then refuse as
2419
+ `unknown-argument` turns one wasted round trip into two — and it is offered only
2420
+ for `reason`, never for another enum argument like `action`, where prose was
2421
+ never the intent. A test pins the invariant that makes the advice always
2422
+ available: every verb with a closed `reason` set also declares `rationale`.
2423
+
1940
2424
  ## Orchestrator tick
1941
2425
 
1942
2426
  The escalation path above assumes an orchestrator session that is actually
@@ -2007,6 +2491,41 @@ The refusal of the raw tool is mechanical (the tool-call gate), not a prompt
2007
2491
  reminder: a model cannot wait unbounded on a local tick even by omitting the
2008
2492
  timeout argument.
2009
2493
 
2494
+ #### The spec-out questionnaire (#947)
2495
+
2496
+ Decomposing a big issue needs several judgement calls about **one** issue, and
2497
+ asking them one at a time is what left #291's two — *is a Gitea fleet actually
2498
+ wanted*, *refuse-and-document versus emulate epic links* — as prose in an issue
2499
+ body that nothing tracked. `conductor_questionnaire` asks them together:
2500
+
2501
+ - `spec-issue` names the issue being specced, and every item's decision row
2502
+ carries it. That binding is the point: the answers become that issue's
2503
+ provenance, so the next reader sees why a slice is shaped the way it is
2504
+ instead of re-litigating it. A group without one is refused.
2505
+ - `items` are ordinary bounded asks — the same schema, composed rather than
2506
+ restated, so an item can never accept something `conductor_ask` refuses. At
2507
+ most six, and an item that fails validation (including the question-shape
2508
+ ceiling) refuses the **whole group** rather than delivering a question set the
2509
+ rows behind it do not match.
2510
+ - Every row is written in **one transaction, before the single delivery**, so an
2511
+ operator is never shown a question nothing is waiting on.
2512
+ - **One delivery, N decisions.** The items go out as a single numbered message,
2513
+ answerable in any order, and each resolves independently: answering item 2
2514
+ leaves item 1 open. The group is bounded by the same ask ceiling — a
2515
+ questionnaire never waits longer than one `conductor_ask` may.
2516
+ - At the ceiling each unanswered item takes its **own** declared `on-timeout`:
2517
+ `auto-proceed` resolves with `"<option> (auto-applied on ask timeout)"`,
2518
+ `park` stays open. An item the operator answered keeps that answer — the group
2519
+ never resolves atomically, because that would discard the answers they gave.
2520
+ - This path is deliberately plain text rather than six button posts, since six
2521
+ posts is exactly what it exists not to do. So a prose reply is not itself a row
2522
+ resolution — the same contract a degraded single ask has — and the result names
2523
+ each row id plus `omp-conductor decision resolve <id> --answer "…"`.
2524
+
2525
+ Both surfaces are mounted and unmounted together: a beat that cannot route one
2526
+ cannot route the other, and a half-present ask surface would leave a model
2527
+ choosing between a working tool and a broken one.
2528
+
2010
2529
  #### Upgrading from one shared arm marker
2011
2530
 
2012
2531
  Before per-project markers every project was given the same `<state dir>/armed`,
@@ -2256,14 +2775,14 @@ least of all on a fleet whose session lives somewhere else.
2256
2775
  ## CLI reference
2257
2776
 
2258
2777
  ```bash
2259
- omp-conductor setup [area] [--no-ai] [--answers FILE] [--save-answers FILE] [--project NAME]
2778
+ omp-conductor setup [area] [--no-ai] [--answers FILE] [--save-answers FILE] [--resume FILE] [--project NAME]
2260
2779
  omp-conductor setup host [NAME] [--project NAME]
2261
2780
  omp-conductor setup graph [--no-seed] [--print] [--project NAME]
2262
2781
  omp-conductor start [--port N] [--project NAME]
2263
2782
  omp-conductor --version
2264
2783
  omp-conductor stop
2265
2784
  omp-conductor restart [--now] [--timeout SECONDS] [--port N] [--project NAME]
2266
- omp-conductor upgrade [--to VERSION] [--project NAME]
2785
+ omp-conductor upgrade [--to VERSION] [--bootstrap SHA --source PATH] [--project NAME]
2267
2786
  omp-conductor upgrade-install --to VERSION [--project NAME]
2268
2787
  omp-conductor upgrade-rollback
2269
2788
  omp-conductor status [--project NAME] [--json]
@@ -2305,13 +2824,13 @@ omp-conductor help
2305
2824
  | Command | Scope | Behaviour |
2306
2825
  | --- | --- | --- |
2307
2826
  | `setup [area] [--no-ai] [--answers FILE] [--save-answers FILE] [--project NAME]` | project | The deterministic interview, with styled Clack prompts on an interactive TTY and byte-stable plain output for pipes or `OMP_CONDUCTOR_PLAIN_UI=1`. `--answers` validates a JSON object of stable prompt keys before setup and replaces every prompt; a missing required key exits `1` naming the key and file instead of hanging. `--save-answers` records accepted interactive answers as replayable JSON after a successful run. Bare setup is a full first run, or — when the project already exists — a chooser of which area to amend. Naming an area positionally skips that chooser and amends only that area: `tracker`, `gates`, `caps`, `code-graph`, `authority`, `policy`, `escalation`, `reporting`, `brief`. `host` and `graph` are install subcommands rather than areas and are matched first; anything else exits `2` listing both vocabularies. Every prompt shows its current value as the default, and Enter accepts what you see; `Ctrl-C` at any prompt abandons the run and writes nothing. Setup also **reads your repos to propose answers**: the gates prompt is pre-filled from what CI actually runs, and the brief's `## Project context` and release procedure are drafted from every routing repo and shown for confirmation before anything is written. Each probe is a short session with **no shell, no editor and no verbs** in a throwaway shallow clone, and every answer is a proposal you edit or decline — a probe that cannot clone, cannot reach a model, or answers unusably costs you one warning and the shipped stub. `--no-ai` asks every question with the reading half removed. |
2308
- | `setup host [--project NAME]` | host | Re-render and stage the systemd unit, then **run** the install: `install -m 0644` into `/etc/systemd/system`, `daemon-reload`, `enable`, `restart`. Stages the fleet recovery oneshot (`omp-conductor-recover.service`) and its playbook (`/usr/local/sbin/omp-conductor-recover`) alongside, and installs them **before** the fleet units: both fleet units carry `OnFailure=` to the recovery unit, so a crash-looped daemon or herdr session now collects evidence durably, attempts one bounded recovery, and pages tier-2 instead of dying silently (#485). Every command is shown with its exact argv, one confirm covers the batch, and `sudo` asks for your password once before the first step — or is skipped entirely on a fleet that genuinely runs as root. The first failure stops the rest and prints the un-run remainder verbatim so you can finish by hand. Refuses an *escalated* invocation (`sudo`, or `sudo -i`/`su -` detected by the invoking account disagreeing with the fleet's) before writing anything, naming both accounts, because staging derives the unit's `User=`/`HOME=` from whoever ran it. On a non-Linux host the files are still staged and only the `systemctl` steps are refused. |
2827
+ | `setup host [--project NAME]` | host | Re-render and stage the systemd unit, then **run** the install: `install -m 0644` into `/etc/systemd/system`, `daemon-reload`, `enable`, `restart`. Stages the fleet recovery oneshot (`omp-conductor-recover.service`) and its playbook (`/usr/local/sbin/omp-conductor-recover`) alongside, and installs them **before** the fleet units: both fleet units carry `OnFailure=` to the recovery unit, so a crash-looped daemon or herdr session now collects evidence durably, attempts one bounded recovery, and pages tier-2 instead of dying silently (#485). Every command is shown with its exact argv, one confirm covers the batch, and `sudo` asks for your password once before the first step — or is skipped entirely on a fleet that genuinely runs as root. The first failure stops the rest and prints the un-run remainder verbatim so you can finish by hand. Refuses an *escalated* invocation (`sudo`, or `sudo -i`/`su -` detected by the invoking account disagreeing with the fleet's) before writing anything, naming both accounts, because staging derives the unit's `User=`/`HOME=` from whoever ran it. On a non-Linux host the files are still staged and only the `systemctl` steps are refused. On a herdr host the last step is `systemctl restart herdr-fleet.service`, which kills every pane in that session — so an invocation from *inside* it is refused before the plan is staged or a single step runs, naming the unit and a context the restart cannot reach (a login shell outside herdr, or a detached `systemd-run` unit). The hazard is the reason, not the herdr host: a plan with no such restart step — a non-herdr host, or a no-op re-run — installs normally from a pane, and `upgrade` has refused the same context since #486 (#834). It also decides who owns the fleet Herdr session before it mutates anything (#893): systemd cannot adopt an already-running process, so an unmanaged `herdr --session <name> server` holding the session makes every start of the unit exit 1 — and because the unit is `Type=simple`, `systemctl restart` returns 0 before the child discovers that. An unmanaged session is therefore stopped inside the same confirmed batch (unprivileged, before the unit install) so the unit can take it over, and a fact that cannot be read is refused rather than guessed — the remedy stops somebody's live session. Success is then proven rather than assumed: a bounded readiness gate re-reads until the unit is `active (running)` with a main pid AND the session answers, so an auto-restart loop is a failed setup. That failure leaves dispatch **paused** (unlike an ordinary failed step, which restores it) and names what it saw: a fleet nobody supervises must not admit workers. |
2309
2828
  | `setup graph [--no-seed] [--print] [--project NAME]` | project | The code-graph install end to end, in one preview and one confirm: check the prerequisites read-only and stop before installing anything when `codebase-memory-mcp` is absent or no MCP entry mounts it (printing the entry to add); stage this project's `cbm-reindex-<project>.{sh,service,timer}` **after** the confirm — never before, so a declined run leaves the staged tree byte-identical — refusing a stem that already belongs to another project or to a file it did not generate (#720); `git clone` each missing index-only checkout **as you, never through sudo**; install and enable `cbm-reindex-<project>.timer` as root; then seed one indexing run so the first fetch happens while you watch, and verify with the same probe `status` uses. A repo that does not verify is a failure with the remediation, not a success — staged-but-not-trusted is how you discover months later that no worker read an index. `--no-seed` enables the timer without the seeding run and says plainly the graph is unusable until it first fires; it never skips the prerequisite or clone steps. `--print` changes nothing. Exits `1` when no repo has [`graphProject`](#configuration). |
2310
2829
  | `start` | host | Start `herdr-fleet.service` when that optional unit is installed, clearing a previous pane-recovery pin, then start the dispatch daemon and wait until it answers `GET /healthz`. When `omp-conductor.service` is installed, systemd is the only start path: even `start --project NAME` restores the shared unit and uses the name only to verify that `/healthz` serves the requested project. A detached daemon is allowed only when the unit is proven absent. It never clears pause or arms ticks. Refuses if a daemon is already live, naming its pid; manager refusal or unprovable ownership is an error rather than a detached fallback. |
2311
2830
  | `stop` | fleet | Prefer `systemctl stop omp-conductor.service` when that unit's MainPID is the live daemon — systemd then owns the stop and will not schedule a restart for the exit it just requested. Otherwise `SIGTERM`, then `SIGKILL` after a 10-second grace period. Prints `not running` when there is nothing to stop, and tags the confirmation with `(via systemctl)` when the unit path was used. |
2312
2831
  | `restart [--now] [--timeout SECONDS] [--port N] [--project NAME]` | host | Drains the fleet by default: pause new claims, wait until live workers reach `0 / N` (bounded by `--timeout SECONDS`, default 1800 = 30 min), restart, then restore the prior dispatch state. A daemon serving multiple configured projects makes restart host-wide: `--project` is rejected because draining one queue and restarting the shared process would kill another project's workers. Prefer `systemctl restart` when the unit owns the live pid so the replacement stays supervised; only a host proven not to have the installed unit may fall back to the standalone stop/start path. `--now` skips the drain and restarts immediately, orphaning any live runs (old behaviour). A drain that hits `--timeout` restarts nothing and leaves dispatch paused — `omp-conductor resume` lifts it, or re-run `restart` to keep waiting. The new process **salvages dirty live worktrees before orphaning** those rows — see [Deploying a new package onto a busy fleet](#deploying-a-new-package-onto-a-busy-fleet). |
2313
- | `upgrade [--to VERSION] [--project NAME]` | host | Deterministically update the Bun-global CLI, omp plugin, Herdr recovery plugin, and managed brief as one release. Resolves the npm version and exact `gitHead`, pauses only new claims, drains active workers, installs all surfaces, reloads Herdr and the daemon, waits for pane recovery, verifies identities and fleet health twice, then restores the original dispatch state. Host-wide by default: one daemon serves every configured project, so a bare run drains all of them and refreshes every brief. `--project` is rejected when the live daemon serves several projects — draining one queue and restarting the shared daemon would kill another's workers. A no-op when already current. Failure leaves dispatch paused. Must run outside a Herdr-managed session. The same transaction runs detached, without a human, as the fleet-installs-itself path: the orchestrator calls the `conductor_install` verb under the granted `install` shape, the daemon validates the version against npm and starts a transient systemd unit (`upgrade-install`) outside the pane and the daemon, and the first tick after the restart verifies version, `/healthz`, ticks, pane and `doctor` against the durable upgrade journal before restoring dispatch — rolling back and paging tier-2 on any gap. |
2314
- | `status [--project NAME] [--json]` | fleet | Layered fleet report first: `dispatch` / `ticks` / next scheduled tick / `pane` / `recovery` / `herdr` / `telegram` / `brief` / `decisions` / optional `failure classes` and `code graph` / `daemon`, then the project body. `--json` emits the stable structured snapshot directly, without ANSI or prose, for automation. The text project body includes the latest completed dispatch timestamp, ready/routed/admitted counts, bounded hold groups, and the GitHub API budget (`graphql` / `core` remaining and reset, in the caps block); API failures are marked `DEGRADED` so queue starvation cannot look idle. Active-run lines overlay cooperative worker `paused`/`pausing` from `/healthz` without changing SQLite `running` state or the live worker count. The next tick comes from the live heartbeat process, not a guess from log timestamps. Telegram health uses `getMe` to prove API authentication without sending a message and reports inbound bridge configuration separately. Configured graphs report prerequisites, indexed repos, timer state, and refresh freshness without blocking dispatch. A `reports` block lists everything the outbox has not delivered, with its age, and prints `pending` (nobody has it) differently from `SENDING` (outcome unknown, it may already have arrived) — see [Report delivery](#report-delivery-the-outbox). The daemon block includes `rss` from `/healthz`; live workers add a busy-deploy warning. A `.conductor-stalled` marker adds an `orchestrator STALLED since …` line. |
2832
+ | `upgrade [--to VERSION] [--bootstrap SHA --source PATH] [--project NAME]` | host | Deterministically update the Bun-global CLI, omp plugin, Herdr recovery plugin, and managed brief as one release. Resolves the npm version and exact `gitHead`, pauses only new claims, drains active workers, installs all surfaces, reloads Herdr and the daemon, waits for pane recovery, verifies identities and fleet health twice, then restores the original dispatch state. Host-wide by default: one daemon serves every configured project, so a bare run drains all of them and refreshes every brief. `--project` is rejected when the live daemon serves several projects — draining one queue and restarting the shared daemon would kill another's workers. A no-op when already current. Failure leaves dispatch paused. Must run outside a Herdr-managed session. The same transaction runs detached, without a human, as the fleet-installs-itself path: the orchestrator calls the `conductor_install` verb under the granted `install` shape, the daemon validates the version against npm and starts a transient systemd unit (`upgrade-install`) outside the pane and the daemon, and the first tick after the restart verifies version, `/healthz`, ticks, pane and `doctor` against the durable upgrade journal before restoring dispatch — rolling back and paging tier-2 on any gap. **`--bootstrap SHA --source PATH`** is the second accepted identity kind (#908), for the case a published version cannot express: the installed conductor is what is broken, and publishing the fix needs the workers that are broken. The sha is the whole identity (40 hex, never a range); `--source` is a checkout that must be AT that sha — verified with `git rev-parse HEAD` — and its only job is to be the tree `bun run check` runs against before anything is installed. A wrong sha, an unreadable tree, or a failing check **refuses**, naming the identity and what failed, with nothing installed. Otherwise it is the same transaction: the same journal (with the checks recorded as evidence), the same three surfaces from that one identity, the same attestation, the same rollback. The `--to` and `--bootstrap` identities are mutually exclusive. |
2833
+ | `status [--project NAME] [--json]` | fleet | Layered fleet report first: `dispatch` / `ticks` / next scheduled tick / `pane` / `recovery` / `herdr` / `telegram` / `brief` / `decisions` / optional `failure classes` and `code graph` / `daemon`, then the project body. `--json` emits the stable structured snapshot directly, without ANSI or prose, for automation. The text project body includes the latest completed dispatch timestamp, ready/routed/admitted counts, bounded hold groups, and the GitHub API budget (`graphql` / `core` remaining and reset, in the caps block); API failures are marked `DEGRADED` so queue starvation cannot look idle. Active-run lines overlay cooperative worker `paused`/`pausing` from `/healthz` without changing SQLite `running` state or the live worker count. The next tick comes from the live heartbeat process, not a guess from log timestamps. Telegram health uses `getMe` to prove API authentication without sending a message and reports inbound bridge configuration separately. Configured graphs report prerequisites, indexed repos, timer state, and refresh freshness without blocking dispatch. A `reports` block lists everything the outbox has not delivered, with its age, and prints `pending` (nobody has it) differently from `SENDING` (outcome unknown, it may already have arrived) — see [Report delivery](#report-delivery-the-outbox). The daemon block includes `rss` from `/healthz`; live workers add a busy-deploy warning. A `.conductor-stalled` marker adds an `orchestrator STALLED since …` line. The `dispatch` line judges a **fence's owning process** when it has one (#938): a `setup` transaction records `owner=<pid>` in its sentinel, because its fence lives exactly as long as that process — so a killed setup renders `ABANDONED — its process (pid N) is gone` with the one command that clears it, while a live one renders `held by a live process`. An operator `hold`, and the durable hold a failed apply leaves behind, record no owner at all and are never described that way, whatever their age. Nothing here clears anything: a pid is reusable, a paused fleet is the safe state, and `doctor` carries the same verdict as a `dispatch-fence:<project>` warning. |
2315
2834
  | `ledger [--issue N] [--limit N] [--json]` | project | The action audit: every [mediated-verb](#the-mediated-verbs-126) mutation and every next-attempt turn budget. Verb entries include the arguments, decision, named refusal, and resulting SHA. Turn-budget entries remain after an override is replaced or consumed. Reads (`conductor_pr_status`) are absent so polling cannot bury the signal. `--issue` narrows both histories; `--limit` defaults to 50. `--json` emits one stable object with `project`, optional `issue`, `entries`, `refused`, and `turnOverrides`. Recent verb refusals and pending turn overrides also appear in `status`. |
2316
2835
  | `board [--project NAME]` | fleet | Live keyboard-driven kanban over the same SQLite and `/healthz` truth as `status`, plus the tracker's current labels: Queue, Claimed, Running, Green, Blocked, Failed, Orphaned, the last 24 hours of Merged and Settled, and Parked (an issue the tracker has not confirmed closed — still open, or a label read that failed — so nothing dispatches it until a human labels it). Columns are mutually exclusive and describe current state, not the newest run row, so a requeued issue is queued rather than failed and a closed issue is neither. Refreshes run/spend/turn values every second, and health plus the label read every ten seconds. `Enter` follows the selected transcript in place; `u` invokes the existing unblock workflow on a Blocked, Failed, or Orphaned card; `i` / `p` open the issue / PR; `r` refreshes health; `?` shows all keys. Requires an interactive terminal of at least 50×20. |
2317
2836
  | `hold [--keep-ticks] [--project NAME]` | fleet | Soft stop: pause claiming **and** disarm ticks. Daemon and pane stay up. Prefer this when the intent is "stop the conductor" without killing processes. `--keep-ticks` pauses claiming but leaves the arm marker, so the heartbeat keeps reporting and `resume` alone restores the fleet — no fresh arm challenge. See [Stop the conductor](README.md#stop-the-conductor-hold--stop). |
@@ -2323,15 +2842,16 @@ omp-conductor help
2323
2842
  | `extend <issue> --turns N [--project NAME]` | project | Raise a live worker's effective turn ceiling through its owning daemon without restarting its session. If the latest run is failed, killed, orphaned, or blocked and has no live controller, store a one-shot ceiling for that issue's next claimed attempt instead. A next-attempt value must exceed the project base, every extension must stay at or below `workerMaxTurnsCeiling`, and live extensions remain monotonic. The pending value appears in `status`, is recorded in `ledger`, and is consumed atomically by one claim. |
2324
2843
  | `worker pause <issue>` / `worker resume <issue>` | project | Cooperatively park one live worker without changing its run state or lane. Pause aborts the active turn to harness idle and freezes the remaining wall-clock budget; resume continues the same session with a prompt to re-check its last action before repeating it. This is separate from fleet-level `hold`, which refuses new claims and work-starting mutations while allowing pre-pause completion work and releases. |
2325
2844
  | `worker stop <issue> --reason TEXT [--project NAME]` | project | Terminally end a running or cooperatively paused worker. The reason is required (1–500 characters) and persisted on the run. The command waits for settlement, records the distinct `stopped` state, salvages and publishes dirty work, removes `agent:in-progress` through the durable label outbox, and consumes neither failed-attempt nor continuation budget. If salvage fails, the tree holding the only copy stays in place and the command names it. Repeating stop is idempotent and reports the run's already-terminal state. |
2326
- | `unblock <issue> [--force] [--no-requeue]` | project | Remove that issue's `blocked` and `failed` labels so an answered escalation can be claimed again, and restore the project queue label by default so the dispatcher actually sees it. `agent:in-progress` comes off too, but only when the newest recorded run is terminal — that row is the proof no worker still owns the issue, so a live run keeps the label (and the queue label stays off until that run settles), and so does an issue with no run row at all. Run history remains intact: blocks consume the independent continuation budget, not failed implementation attempts. The output reports both budgets and warns when either will make the next tick escalate instead of dispatch. The label changes go through the [label projection outbox](#how-one-tick-works): they are applied inline before the command returns, but **a tracker that refuses them (403, rate limit) no longer fails the verb** — it exits `0`, the intended label state is durable and the daemon retries it, and the output says `label sync queued (N pending) — the daemon retries` instead of claiming the labels were restored. Safety is preserved, but the issue is only claimable once the queue label itself lands: the queue read asks GitHub for issues carrying that label, so a refused queue-label add keeps the issue out of dispatch until projection succeeds. `--no-requeue` clears the state labels only, leaving the queue label untouched — the case where you are about to close the issue. **Refuses, clearing nothing and exiting `3`, when the newest attempt's work could not be committed and its worktree is the only copy** — re-claiming removes that tree. `--force` records the operator's acceptance on the run row and then clears; the salvage failure stays in history. Exits `2` when the issue number is missing or malformed. |
2327
- | `verb <conductor_*> [--arg k=v ...]` | project | Run one [mediated verb](#the-mediated-verbs-126) as the orchestrator, from the CLI the external-orchestrator half of the verb surface. Every argument goes in as a `--arg k=v` string; an orchestrator can merge (`conductor_pr_merge`), label (`conductor_label`), release (`conductor_release`), update a branch (`conductor_pr_update_branch`) or title/body (`conductor_pr_update`), recover a settled run's missing PR (`conductor_pr_recover`), or read PR state (`conductor_pr_status`). The daemon applies the same checks and writes the same ledger rows a session's call would; a missing `--arg` is refused exactly as a missing tool argument is, worker-only verbs (`conductor_push`, `conductor_pr_create`) are refused with `role-not-allowed`, and a refusal exits `3`. An unknown verb…
2845
+ | `unblock <issue> [--force] [--no-requeue]` | project | Remove that issue's `blocked` and `failed` labels so an answered escalation can be claimed again, and restore the project queue label by default so the dispatcher actually sees it. `agent:in-progress` comes off too, but only when the newest recorded run is terminal — that row is the proof no worker still owns the issue, so a live run keeps the label (and the queue label stays off until that run settles), and so does an issue with no run row at all. Run history remains intact: blocks consume the independent continuation budget, not failed implementation attempts. The output reports both budgets and warns when either will make the next tick escalate instead of dispatch. |
2846
+ | `release-composition declare --campaign ID [--pr URL ...] [--reason TEXT]` · `override --pr URL [--reason TEXT]` · `complete|cancel [--reason TEXT]` · `status` | project | Declare the project's one active release (campaign + exact PR URLs), admit exactly one extra PR by recorded override, retire the guard, or inspect it. While a release is active, `conductor_pr_merge` refuses every other PR with `outside-active-release` the [active release composition](#the-active-release-composition-850) guard and the refusal clears only through this lifecycle, never through a merge reason. Every transition survives restarts and is recorded as an immutable material event. |
2847
+ | `verb <conductor_*> [--arg k=v ...]` | project | Run one [mediated verb](#the-mediated-verbs-126) as the orchestrator, from the CLI — the external-orchestrator half of the verb surface. Every argument goes in as a `--arg k=v` string; an orchestrator can merge (`conductor_pr_merge`), label (`conductor_label`), release (`conductor_release`), update a branch (`conductor_pr_update_branch`) or title/body (`conductor_pr_update`), recover a settled run's missing PR (`conductor_pr_recover`), clear the review evidence standing at one exact head (`conductor_pr_review_clear`), or read PR state (`conductor_pr_status`). The daemon applies the same checks and writes the same ledger rows a session's call would; a missing `--arg` is refused exactly as a missing tool argument is, worker-only verbs (`conductor_push`, `conductor_pr_create`) are refused with `role-not-allowed`, and a refusal exits `3`. An unknown verb…
2328
2848
  | `friction <kind> --detail TEXT [--issue N]` | project | Record one bounded judgment the daemon cannot infer: an escalation belonged in a digest, or a tick report was noise/surprising. The detail is limited to 160 characters. One event never changes policy; three observations inside seven days make the aggregate eligible for one Learning-loop prompt, followed by a seven-day cooldown. |
2329
2849
  | `report --text TEXT [--kind material|digest|tier2|decision-needed|fleet-stopped|confirmed-failure]` | project | Hand a rendered report to the daemon's durable outbox. The command persists the text **before** anything can send and prints a durable handoff id. A material report submitted during quiet hours becomes a held notice until the window opens; otherwise it becomes a report whose delivery the daemon owns, retries with bounded backoff, and records. Delivery is [at-least-once](#report-delivery-the-outbox), so a crash mid-send is retried as a possible repeat and `delivered` never proves exactly one message. `--kind digest` is accepted at most once per local day, decided from the ledger; an unknown `--kind` exits `2`. The remaining kinds declare the report's interrupt category — the escalation handoff: the reporting policy decides between immediate delivery and a durable hold exactly as for a daemon escalation of that category, A repeated identical call exits `2` only while the earlier handoff is still queued undelivered; once it lands, the same text is admitted again (the handoff state decides, not a permanent ledger). Anything still owed appears in `status` with its age. |
2330
2850
  | `decision open --question TEXT [--blocks TEXT] [--resolves-when COND]` | project | Record a question the orchestrator has put to you, and print its id. A question that lives only in a session's context is lost at the next compaction — after which it is either asked twice or dropped silently. `--resolves-when` attaches a machine-checkable condition: `pr-merged:<https url>`, `pr-checks-green:<https url>`, `pr-mergeable:<https url>`, `issue-closed:<n>`, `npm-version:<pkg>@<version>`, or `rate-limit-reset:github`; anything else exits `2` listing the six forms. See [The decision ledger](#the-decision-ledger-136). |
2331
2851
  | `decision resolve <id> --answer TEXT` | project | Record what you decided. Exits `1` naming the id when it is unknown or no longer open, so a second answer cannot overwrite the first. |
2332
2852
  | `decision withdraw <id> [--reason TEXT]` | project | Close a question the session stopped needing, with why. Same guard as `resolve`. |
2333
2853
  | `decision list [--json]` | project | Open questions, oldest first: id, age, what each blocks, whether its condition is met, and the question. `--json` emits `{ project, decisions }`; the empty state is an empty array. Prints `no open decisions` in text mode when there are none. Watches are not listed here — `watch list` shows those. |
2334
- | `watch add --note TEXT [--blocks TEXT] [--resolves-when COND]` | project | Record a condition or carry note the orchestrator set for itself, with no human in the loop (#459). `--resolves-when` attaches a machine-checkable condition the daemon checks for you; a met watch wakes the next tick exactly as a met question does. Renders under its own "Watches" heading, is never counted in `decisions N open`, and has no seven-day expiry. |
2854
+ | `watch add --note TEXT [--blocks TEXT] [--resolves-when COND]` | project | Record a condition or carry note the orchestrator set for itself, with no human in the loop (#459). `--resolves-when` attaches a machine-checkable condition the daemon checks for you; a met watch wakes the next tick exactly as a met question does. Renders under its own "Watches" heading, is never counted in `decisions N open`, and never expires while it is *waiting* — once a condition fires, the ordinary seven-day window runs from that instant (#966). |
2335
2855
  | `watch list [--json]` | project | Open watches, oldest first: id, age, what each blocks, whether its condition is met, and the note. `--json` emits `{ project, watches }`; the empty state is an empty array. Prints `no watches` in text mode when there are none. |
2336
2856
  | `daemon` | host | Run the loop in the **foreground**, ticking every 5 minutes and serving `/healthz`. Admitted workers run in a tracked background pool, so settlement and capacity checks remain periodic while they work; shutdown drains the pool before closing the store. This is what `start` launches and what a systemd unit should call. |
2337
2857
  | `daemon --once` | host | Run a single tick, wait for workers admitted by that tick, and exit. No HTTP server or pidfile — a drill must not register itself as the daemon, or the next reader believes it and the real daemon's in-flight runs get reconciled as orphans. |
@@ -2519,6 +3039,20 @@ Until that uid exists, a root-or-operator daemon still has a mechanical
2519
3039
  worktree gate on structured tools and an integrity tripwire on its own package —
2520
3040
  but `bash` plus host credentials remain a prompt-and-deploy problem.
2521
3041
 
3042
+ **One attempt at this shipped and was withdrawn — read it before rebuilding it.**
3043
+ 0.18.1 created an `omp-worker` account, launched sessions under it with
3044
+ `setpriv`, projected the operator's agent config to it by ACL, and bind-mounted
3045
+ the operator's `node_modules` read-only so the worker could resolve the harness.
3046
+ Every admitted worker then died before its first turn, one resolution layer at a
3047
+ time: config ACLs, a missing `zod`/`yaml`, a native addon looked up through a
3048
+ different per-UID Bun cache, package-export failures out of the bound tree. The
3049
+ lesson is not "fix the mount": exposing package *files* to another uid is not the
3050
+ same as giving it a runnable runtime. The rollback (#892) restored the
3051
+ fleet-account launch, and `setup host`/`upgrade` now retire that host state
3052
+ wherever it still exists. A next attempt needs an immutable self-contained worker
3053
+ runtime (a compiled binary or a package image), a real end-to-end session
3054
+ preflight before the switch, and atomic rollback — not a narrower ACL.
3055
+
2522
3056
  ### The orchestrator is unconfined, deliberately
2523
3057
 
2524
3058
  There is **no mechanical file gate on the orchestrator session**, and that is an
@@ -2559,11 +3093,12 @@ daemon, across a process boundary, not in a prompt the model can rewrite.
2559
3093
  | --- | --- | --- |
2560
3094
  | `conductor_push` | the worker owning the run | The ref is exactly `refs/heads/<that run's branch>`. Fast-forward only; there is no force argument to reject because none is declared. |
2561
3095
  | `conductor_pr_create` | the worker owning the run | The run has no open PR (the same guard admission uses); head is the run branch; base is the repo's configured `defaultBranch`. |
2562
- | `conductor_pr_review` | **orchestrator only** | The PR is one a run of this project opened; the run is in a revisable settled state (`pushed-green`, or `failed`/`killed` after pushing green); the live head still equals the reviewed head and the checks at it are green; no review revision is already in flight; the review-round ceiling has not been reached. |
2563
- | `conductor_pr_recover` | **orchestrator only** | The target issue has a recorded terminal run whose branch still exists at its exact recorded 40-hex head; the run is not live and not merged; the issue is open; no open PR already closes the issue at that head, and a recorded PR is open (returned), closed/merged (refused), definitively missing (replaced) or unreadable (retryable). Creates the one missing PR from that branch to the configured `defaultBranch` and records it on the run. |
3096
+ | `conductor_pr_review` | **orchestrator only** | The PR is one a run of this project opened; the run is in a revisable settled state (`pushed-green`, or `failed`/`killed` after pushing green); the live head still equals the reviewed head and the checks at it are green; no review revision is already in flight. At the review-round ceiling it no longer refuses: it admits one [final adjudication](#the-review-ceiling-adjudication-lifecycle-874) for that exact head under the project's `review.adjudicator` role, idempotently, without consuming a worker round (#932). |
3097
+ | `conductor_pr_review_clear` | **orchestrator only** | The PR is one a run of this project recorded (a `pushed-green` run is the normal case, so revisability is deliberately *not* required); `headSha` equals the live head; something actually blocks the merge at that head; `reason` is non-empty. Green is not required the merge gate re-checks it. Records an append-only clearance plus a material event, and the [exact-head review gate](#the-exact-head-review-gate-888) then stops refusing that head. |
3098
+ | `conductor_pr_recover` | **orchestrator only** | The target issue has a recorded terminal run whose branch still exists at its exact recorded 40-hex head; the run is not live and not merged; the issue is open; no open PR already closes the issue at that head, and a recorded PR is open (returned), closed/merged (refused), definitively missing (replaced) or unreadable (retryable). No live [mutation lease](#what-holds-a-lane-and-what-merely-owns-an-artifact) holds a file the preserved branch carries — an overlap refuses `recovery-lane-occupied`, and a lane that cannot be read refuses `recovery-lane-unprovable` rather than publishing on an unproven one (#925). Creates the one missing PR from that branch to the configured `defaultBranch` and records it on the run. |
2564
3099
  | `conductor_pr_status` | worker or orchestrator | Read-only — nothing to gate. A worker reads only its own run's PR; an orchestrator may name any syntactically valid PR URL, open, merged, or closed, and gets its live state and head (checks are reported when available; a merged or closed PR reports its state instead of an `expected OPEN` refusal). |
2565
3100
  | `conductor_pr_update_branch` | orchestrator, or the worker owning the run | The PR belongs to this project and is open. A worker may only name its own run's PR. |
2566
- | `conductor_pr_merge` | **orchestrator only** | Ordinarily, `authority.merge` equals the caller. A hand-edited `recoveryMerges` entry may instead authorize one exact unrecorded PR/head/reason while held. In both paths, `headSha` equals the live head *at execution time*; checks are green at that same SHA; the project route and migration chain are valid; the project's single merge slot is free. |
3101
+ | `conductor_pr_merge` | **orchestrator only** | Ordinarily, `authority.merge` equals the caller. A hand-edited `recoveryMerges` entry may instead authorize one exact unrecorded PR/head/reason while held. In both paths, while the project has an [active release composition](#the-active-release-composition-850), the PR must be named in it or carry an explicit operator override; then `headSha` equals the live head *at execution time*; checks are green at that same SHA; no [review evidence](#the-exact-head-review-gate-888) stands at that head without a recorded `conductor_pr_review_clear`; the project route and migration chain are valid; the project's single merge slot is free. |
2567
3102
  | `conductor_label` | **orchestrator only** | The label is in the project's own vocabulary. Lifecycle labels stay the daemon's. Adding the queue label echoes the parsed [file lane](#the-file-lane-declaration), or refuses with `file-lane-unparseable` when a clearly delimited write-lane section parsed nothing — it never claims fail-open beside a declaration that was actually attempted. |
2568
3103
  | `conductor_release` | **orchestrator only** | `authority.release` equals the caller; the per-shape grant permits it; the artefact or environment was declared; the release preconditions hold; the `reason` is in the closed enum. `version-bump-pr` creates or re-validates one deterministic version-only PR and, on a later call, merges only its exact green head through the project's single merge slot. |
2569
3104
  | `conductor_install` | **orchestrator only** | Gated like a release act: the `install` shape defaults to `human` and a grant is what moves it. The daemon refuses a version npm does not expose with a full `gitHead`, refuses while another install is still in flight, and otherwise starts a detached transient unit that pauses, drains, installs the CLI/omp plugin/Herdr plugin and reloads — outside this session and the daemon. The unit never declares its own success; the first tick after the restart verifies and reports through the durable outbox. |
@@ -2584,6 +3119,216 @@ live-head or check validation, or make a general class of PRs mergeable.
2584
3119
  Authorizations are re-read from config on every call and every attempted merge
2585
3120
  is written to the ordinary verb ledger, including refusals.
2586
3121
 
3122
+ #### The refusals reach `status`, not only `ledger` (#972)
3123
+
3124
+ `status` closes with a bounded verb-ledger block — newest first, five entries,
3125
+ silent when nothing has been called:
3126
+
3127
+ ```
3128
+ verb ledger 20 recent call(s), 3 refused (omp-conductor ledger for the rest)
3129
+ 2026-08-23 01:13:23 REFUSE conductor_label orchestrator [reason-not-in-enum]
3130
+ ```
3131
+
3132
+ The header counts every row the snapshot carries (`STATUS_LEDGER_SCAN`, 20), not
3133
+ the five printed: a count computed from the visible rows would report "1
3134
+ refused" on a fleet that refused eleven things that morning.
3135
+
3136
+ This is the surface an operator reads, and the refusal *class* is what they come
3137
+ for. It had never appeared there. The block was written for `status` in #133 —
3138
+ its own doc comment and both constants say so — but was wired into a second,
3139
+ unshipped copy of the status body in `daemon.ts`, which was already not the live
3140
+ renderer on the day it landed. So for a fortnight the snapshot performed a
3141
+ 20-row read on every single `status` call and discarded the result, while the
3142
+ dead copy was kept alive by exactly one test.
3143
+
3144
+ What that cost is measurable: #968 — an out-of-enum `reason` accounting for 41%
3145
+ of every refusal on this fleet — was found by querying the sqlite file by hand.
3146
+ A header reading `99 refused` would have surfaced it in the first week.
3147
+
3148
+ ### A review round is charged only when its worker worked (#903)
3149
+
3150
+ The review-round ceiling bounds *polishing*, so it counts rounds a worker
3151
+ actually ran. A revision dispatch that dies before its resumed session takes a
3152
+ turn — a host permission fault, a spawn failure, a daemon shutdown between the
3153
+ claim and the launch — worked no round: the same row is returned to the pending
3154
+ set with its round number, findings and target session unchanged, the run goes
3155
+ back to the exact `pushed-green` state the review verb recorded, and the next
3156
+ dispatch pass resumes it. No label changes and nothing escalates, because
3157
+ nothing has failed that a human can act on.
3158
+
3159
+ The discriminator is "did the resumed session take a turn", never the error
3160
+ text: during the 2026-08-21/22 worker-mount outage every dispatch died at turn
3161
+ zero on an `EACCES` no signature list contained, each death consumed one of the
3162
+ PR's three rounds, and the ceiling then refused any further round — an outage
3163
+ turned into a permanently unmergeable PR. Matching on errno would only wait for
3164
+ the next outage to spell itself differently.
3165
+
3166
+ Retries are counted on the row itself, so they survive a restart, and they are
3167
+ bounded: after three the round settles `failed` exactly as it always did, the
3168
+ issue takes the failed label, and one escalation names it. A round whose worker
3169
+ took even one turn is charged normally, whatever it then did — that is what
3170
+ keeps the ceiling meaningful.
3171
+
3172
+ ### The exact-head review gate (#888)
3173
+
3174
+ Green checks say nothing about the review lifecycle, so `conductor_pr_merge`
3175
+ also refuses — with `merge-blocked-by-review` — when durable review evidence
3176
+ stands at exactly the head being merged: a review round that is queued, one
3177
+ that was dispatched and crashed mid-review, or one that settled `failed`
3178
+ there. The evidence is keyed by project + PR + head, so a finding tied to an
3179
+ older head never poisons a corrected push.
3180
+
3181
+ That leaves the question of how a head that will *not* be corrected ever
3182
+ merges — a PR at the review-round ceiling, or one whose findings a human read
3183
+ and dismissed. Re-reviewing the same head is not the answer and never was:
3184
+ `conductor_pr_review` at an unchanged head appends to the pending round or
3185
+ opens a new one, and both block. The answer is an explicit, recorded
3186
+ disposition:
3187
+
3188
+ ```bash
3189
+ omp-conductor verb conductor_pr_review_clear --project acme \
3190
+ --arg prUrl=https://github.com/acme/api/pull/12 \
3191
+ --arg headSha=<the live head> \
3192
+ --arg reason="findings answered in review thread; ceiling reached"
3193
+ ```
3194
+
3195
+ Three properties make it a disposition rather than a blanket override, and all
3196
+ three are mechanical:
3197
+
3198
+ - **It clears backwards only.** A clearance suppresses evidence whose own last
3199
+ activity is at or before the moment it was taken. A round recorded *after*
3200
+ it blocks the merge again — so a clearance cannot be banked as a standing
3201
+ pass.
3202
+ - **It is head-scoped and non-vacuous.** It names the exact live head, and a
3203
+ clearance at a head where nothing blocks is refused (`review-clearance-vacuous`).
3204
+ - **It bypasses nothing else.** Authority, pause, base-red-freeze, exact-head,
3205
+ green checks, the migration chain, the active release composition and
3206
+ single-flight all still apply to the merge that follows.
3207
+
3208
+ Like the composition override, it is deliberately a separate verb: a merge
3209
+ call's own `reason` is not a clearance. Each clearance is an append-only row
3210
+ with its reason, a verb-ledger entry, and a material event in the digest.
3211
+
3212
+ ### The review-ceiling adjudication lifecycle (#874)
3213
+
3214
+ A PR that reaches `review.maxRounds` has no worker rounds left, and returning it
3215
+ to the operator is what conductor exists to avoid. So the ceiling escalates the
3216
+ **automation**: `review.adjudicator` names an OMP model role (a role name, never
3217
+ a provider/model — OMP owns that selection), and a terminal adjudication decides
3218
+ the head with a stronger model.
3219
+
3220
+ What exists today is the durable lifecycle underneath it. One adjudication is
3221
+ recorded per project + PR + **exact head**, and that uniqueness is a database
3222
+ index rather than a flag a caller remembers to check, so a repeated tick, two
3223
+ dispatchers and a daemon restart all re-read the same row instead of launching a
3224
+ second adjudicator. A live adjudication on the same PR at a *different* head is
3225
+ refused: the PR moved, and two live adjudications on one PR is exactly the
3226
+ duplicate this prevents.
3227
+
3228
+ It is deliberately **not** a round in `review_revisions`. Reusing round N+1
3229
+ would spend a ceiling that is already exhausted, and would resume the same
3230
+ session and model for a nominal fourth opinion — the failure the escalation
3231
+ exists to end. The two tables also answer different questions: a revision says
3232
+ "go and change this", an adjudication says "decide about this".
3233
+
3234
+ The terminal states **are** the verdict — `cleared`, `rejected`,
3235
+ `unavailable-model`, `stale-head`, `failed` — with no second verdict column to
3236
+ disagree with them. Stored beside the state is what it cannot say: the
3237
+ `evidence` that produced it, and the `disposition` applied because of it.
3238
+ `status` renders each one under its run as `adjudication <state> role <asked>
3239
+ → <model actually launched>`, matched on PR + head so a corrected push never
3240
+ appears to carry the older head's decision.
3241
+
3242
+ **Reaching the ceiling now admits one instead of refusing.** `conductor_pr_review`
3243
+ at `maxRounds` used to refuse and tell the orchestrator to leave the PR open and
3244
+ escalate; it now records the adjudication for the head it just verified live and
3245
+ green, carrying the findings that provoked it — the most relevant single input the
3246
+ adjudicator gets. The ceiling on *worker* rounds is unchanged: no fourth revision
3247
+ is dispatched and no round is consumed. A repeated call at the same head is
3248
+ idempotent; a call whose head has moved has already refused (`head-stale`) before
3249
+ the admission, and one made while an adjudication is live at a different head
3250
+ refuses `review-in-flight`.
3251
+
3252
+ **The daemon's own pass launches it**, next to review-revision dispatch and never
3253
+ inside it: a revision resumes an implementation worker to change code, an
3254
+ adjudication opens a fresh read-only session to decide about code, and folding
3255
+ them together is how "escalate the automation" quietly becomes "resume the same
3256
+ worker with a different label". The pass re-reads the live head first (a moved
3257
+ head settles `stale-head` without launching, an unreadable one stays pending for
3258
+ the next tick), claims `pending → running` — that transition is the single-flight
3259
+ guard — assembles the evidence from durable rows plus live tracker reads, and
3260
+ settles the row from the verdict:
3261
+
3262
+ | Result | State |
3263
+ |---|---|
3264
+ | verdict `cleared` | `cleared` — the ordinary orchestrator-owned merge path |
3265
+ | verdict `rejected` | `rejected` — #876's autonomous disposition |
3266
+ | a verdict naming a different head | `stale-head`, fail-closed |
3267
+ | no verdict, zero turns | `unavailable-model` — the role never launched; a configuration fault |
3268
+ | no verdict, turns spent | `failed` |
3269
+
3270
+ An adjudication occupies no worker slot and no issue, charges no failed attempt
3271
+ and no continuation, and its turn ceiling (`ADJUDICATION_MAX_TURNS`) is
3272
+ deliberately small: a verdict is a read and an answer, not an implementation. A
3273
+ daemon restart settles a `running` adjudication `failed` — its session died with
3274
+ the process, an adjudicator has no branch or transcript worth resuming, and
3275
+ re-queueing it would be a second launch for one head.
3276
+
3277
+ **The verdict is then acted on** (#876), in its own pass, because a verdict and
3278
+ what was done about it are two facts:
3279
+
3280
+ - **`cleared`** records a [`conductor_pr_review_clear`](#the-exact-head-review-gate-888)
3281
+ at that exact head, by `adjudication:<role>`. That is the whole mechanism — the
3282
+ merge gate already reads clearances, so there is no adjudication-shaped
3283
+ exception inside `conductor_pr_merge` and one audit trail covers both. The head
3284
+ is re-read first: if it moved, no clearance is recorded and the row says so.
3285
+ - **Every other terminal state** — `rejected`, `failed`, `unavailable-model`,
3286
+ `stale-head` — closes the pull request, having first posted the adjudicator's
3287
+ reasons and every preserved review finding on it. They all mean "this head is
3288
+ not merging and no further round is coming", and the blocking artefact is the
3289
+ same in each case: an open PR occupying the issue. The branch is untouched.
3290
+
3291
+ Nothing here opens a review round, launches a second adjudicator, or asks an
3292
+ operator for anything. No budget is reset either: the closed PR is charged by the
3293
+ ordinary settle sweep exactly as any PR closed without merging, which is what
3294
+ lets this compose with a separate corrective-worker path instead of resetting its
3295
+ counters.
3296
+
3297
+ The recorded `disposition` is the idempotency key — a row that already has one is
3298
+ skipped, so a restart or a retried tick cannot close one PR twice — and it is
3299
+ written only *after* the act succeeds, so a failed close leaves the row
3300
+ undisposed and retryable rather than claiming work it did not do. As a backstop,
3301
+ `conductor_pr_merge` refuses any head whose adjudication settled anything other
3302
+ than `cleared`, in case a close failed or raced.
3303
+
3304
+ ### The active release composition (#850)
3305
+
3306
+ An early patch is only trustworthy while nothing unrelated slips into it, so a
3307
+ project can declare **one active release composition**: a campaign identifier
3308
+ (say `v0.18.1-reliability`) plus the full PR URLs allowed into that release.
3309
+ While one is active, `conductor_pr_merge` refuses every other PR with the
3310
+ `outside-active-release` refusal naming the campaign and the conflicting PR —
3311
+ checked before the merge lock and any tracker call, from SQLite rows that
3312
+ survive daemon restarts, so the guard is mechanical rather than a policy
3313
+ sentence or a status warning.
3314
+
3315
+ The guard moves only through the explicit operator lifecycle:
3316
+
3317
+ ```bash
3318
+ omp-conductor release-composition declare --campaign v0.18.1-reliability \
3319
+ --pr https://github.com/acme/api/pull/12 # open the release
3320
+ omp-conductor release-composition override --pr https://github.com/acme/api/pull/13 \
3321
+ --reason "hotfix belongs in this patch" # admit exactly one extra PR
3322
+ omp-conductor release-composition complete # or cancel — retires the guard
3323
+ ```
3324
+
3325
+ A merge call's free-form `reason` is never an override, and an override bypasses
3326
+ only the composition check: authority, pause, base-red-freeze, review, exact-head,
3327
+ green-checks and single-flight all still apply to the admitted merge. Every
3328
+ transition writes an immutable material event, overrides stay in the audit trail
3329
+ scoped to their campaign, and merging a member PR never retires the composition —
3330
+ only `complete` or `cancel` does.
3331
+
2587
3332
  For a repo with `release.versionFile`, call `conductor_release` with
2588
3333
  `shape=version-bump-pr` and the intended `v<semver>` tag. The requested version
2589
3334
  must be newer than the live semantic version. The first call creates
@@ -2607,6 +3352,31 @@ A `github-release` for the same repo likewise requires that reviewed tag to be
2607
3352
  present on origin and verifies the tag's version file before creating the
2608
3353
  release; it never lets GitHub synthesize the missing tag.
2609
3354
 
3355
+ **Pinning a tag to one commit (#695).** Both tag shapes accept
3356
+ `--arg sha=<commit>`, and it removes the dependence on *when* the tag is cut.
3357
+ Without it the target is whatever the default branch is at cut time, which
3358
+ deadlocks any fleet whose queue is not empty: cutting needs a window with no
3359
+ unsettled run, reaching that window needs the open PR merged, and merging moves
3360
+ the branch past the tag that was just cut — leaving it unpushable (behind main),
3361
+ un-re-cuttable (a tag the tool cut is never force-moved) and with no shape able
3362
+ to clean it up. The 2026-08-18 occurrence burned a version number.
3363
+
3364
+ With a pinned commit:
3365
+
3366
+ - `git-tag` cuts at exactly that commit, and the "behind live main" refusal does
3367
+ not apply — being behind is the point;
3368
+ - `git-push-tags` publishes that tag **unchanged**, instead of re-pointing it to
3369
+ the live branch head as an unpinned push does. A local tag standing somewhere
3370
+ other than the pinned commit is a refusal, never a silent move;
3371
+ - a commit the released repository does not have, or one **not reachable from
3372
+ the released branch**, is refused naming the sha: a tag pointing off-branch is
3373
+ a release of code nobody merged;
3374
+ - omitting `sha` leaves every existing behaviour exactly as it was, and the
3375
+ argument is refused on any shape that cuts no tag rather than being ignored.
3376
+
3377
+ Deliberately not fixed here: the `runs-settled` precondition's scope (see #603,
3378
+ already repo-scoped) and any tag-delete or re-cut shape.
3379
+
2610
3380
  ### The transport
2611
3381
 
2612
3382
  Identity is never an argument. `project`, `run`, `issue` and the caller's role
@@ -2775,16 +3545,81 @@ Known and deliberate in this version:
2775
3545
  re-derived, not forgotten), and one closing notice lands when the orchestrator
2776
3546
  recovers. What remains a separate watchdog is the `.conductor-stalled` wedge
2777
3547
  path, for a session that stays alive but stops draining its queue.
2778
- - **Workers are not terminal panes, so you cannot watch them there.** Each
2779
- worker is an omp session the daemon starts as a child process. The resident
2780
- daemon tracks workers in a background pool so the five-minute loop keeps
2781
- settling PRs and checking capacity; shutdown waits for that pool. Herdr still
2782
- shows exactly one pane (the orchestrator's) regardless of concurrency.
2783
-
2784
- The cap does work. The admission loop (`admitCandidates` in `src/daemon.ts`) computes
2785
- `slots = maxConcurrentWorkers - live workers`, admits at most that many issues
2786
- per tick, and dispatches them together. To see them, read `omp-conductor
2787
- status`, which lists every occupied issue, or follow `daemon.log`.
3548
+ - **A worker gets a Herdr pane, but the pane is never the worker (#840).** Each
3549
+ worker is an omp session the daemon starts as a child process, and that child
3550
+ stays the authoritative one: the typed control socket, the mediated verb
3551
+ socket, the worktree, the transcript, the accounting and the output-schema
3552
+ settlement are all its. What the workspace shows is a *representation* of that
3553
+ exact child — a pane named `worker-<project>-<issue>-a<attempt>-<run>`, created
3554
+ at spawn from the pid the spawn path itself reports, and reported to Herdr
3555
+ through its external-supervisor verbs (`pane report-agent-session` for the run
3556
+ and transcript identity, `pane report-agent` for `working`/`idle`/`blocked`,
3557
+ `pane release-agent` when the child is gone).
3558
+
3559
+ Nothing starts a second OMP process, and nothing reads the pane back: no run
3560
+ state, settlement, turn, spend or model fact is ever derived from terminal
3561
+ output. The pane displays `omp-conductor tail`, the existing read-only
3562
+ follower. Input cannot reach the session either, and that is structural rather
3563
+ than a check — the authoritative child is spawned with `stdin: "ignore"`, so
3564
+ there is no descriptor for a keystroke to travel down, and the pane's own
3565
+ process is a follower whose stdin reaches only itself.
3566
+
3567
+ **The identity is durable, so the workspace survives the process (#842).** The
3568
+ pid, pane id and label are written onto the run row the moment the pane is
3569
+ established — before it is announced, because a pane the store does not know
3570
+ about is a pane a restart cannot reconcile. Three consequences follow, and each
3571
+ is the answer to a way the pre-#842 representation went wrong:
3572
+
3573
+ - **A re-entered launch adopts, never duplicates.** A run that already carries a
3574
+ pane identity is re-reported against that pane; only a run with none creates
3575
+ one. Two panes claiming one worker is worse than none, because both look real.
3576
+ - **A restart releases what it inherits.** The startup orphan sweep hands the
3577
+ pane of every reaped run back to Herdr (`releaseOrphanedWorkerPane`), so a
3578
+ workspace never shows a live worker for a child that died with the previous
3579
+ daemon. The recorded pid is deliberately *not* consulted for liveness: a
3580
+ `session-host` child dies with the daemon that owned its verb socket, so the
3581
+ worker is gone whatever the pid says — and pids are reused, so probing one is
3582
+ exactly how a stranger's process comes to read as a live worker.
3583
+ - **State is projected from transitions, never scraped.** Pause and resume reach
3584
+ the pane through the control registry's own `onPhase` notifier — the phase the
3585
+ control actually reached, fired only on success, so a refused pause never
3586
+ shows as `idle`. A cap kill reports `unknown`. Each report carries the run's
3587
+ own monotonic `seq`, so a late report cannot overwrite a newer state.
3588
+
3589
+ **Herdr unavailability is a named state, and retention is finite (#841).** A
3590
+ worker with no pane keeps working — visibility is never a precondition — but it
3591
+ never does so quietly: the reason is written to its run row and `omp-conductor
3592
+ status` prints `pane degraded — <reason>` on that run's own line, so a fleet
3593
+ running blind cannot be mistaken for a fleet with nothing to show. Every
3594
+ dispatch pass then reconciles the workspace against the live run set
3595
+ (`reconcileWorkerPanes`), which is what makes the surface survive a *Herdr*
3596
+ restart — an event conductor is never told about:
3597
+
3598
+ - A live run whose pane Herdr still has is left alone (`intact`), so repeated
3599
+ passes converge instead of churning panes.
3600
+ - A live run whose pane Herdr lost is re-associated: one new pane, for the pid
3601
+ already on the row. No second OMP process is started, ever. A run with no
3602
+ recorded pid stays `untracked` rather than getting a pane with nothing behind
3603
+ it.
3604
+ - A conductor-owned pane whose run id is not live is released — the retention
3605
+ bound: a settled worker's pane is conductor's only until the next pass. It is
3606
+ matched by the run id Herdr itself reports, never by label or age, so a live
3607
+ worker's pane can never be cleaned up and a stale lookalike can never be
3608
+ mistaken for one. Release, never close: nothing here can terminate a live
3609
+ authoritative worker, because the only mutation is handing lifecycle
3610
+ authority back.
3611
+ - An unreadable or absent Herdr reconciles nothing and says so on every pass.
3612
+ Failing to read the workspace is not evidence that a worker's pane is stale.
3613
+
3614
+ Fleet recovery knows the difference too: `herdr/bin/recover.sh` excludes any
3615
+ pane carrying the `omp-conductor` source from its resume candidates, so a
3616
+ released worker pane — which has no agent and sits in the fleet cwd, and so
3617
+ looks free to every other check — is never where the orchestrator gets started.
3618
+
3619
+ The cap still does the work. The admission loop (`admitCandidates` in
3620
+ `src/daemon.ts`) computes `slots = maxConcurrentWorkers - live workers`, admits
3621
+ at most that many issues per tick, and dispatches them together. `omp-conductor
3622
+ status` remains the authoritative list of occupied issues.
2788
3623
  - **Report delivery is at-least-once, never exactly-once.** The Telegram Bot API
2789
3624
  takes no client-supplied idempotency key, so the window between "Telegram
2790
3625
  accepted it" and "SQLite recorded that" is irreducible. The daemon resolves it