omp-conductor 0.2.1 → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -28,8 +28,9 @@ the line.
28
28
  When a run does get stuck, the first responder is not you. A tier-1 escalation is
29
29
  injected into a long-lived **orchestrator session** that can read the issue and the
30
30
  run's transcript and then either re-brief the worker or decide the problem genuinely
31
- needs a human. It never edits product code, pushes or merges. Only tier 2 pages you
32
- directly.
31
+ needs a human. It never edits product code and never pushes a branch; whether it
32
+ may *merge* is a setup answer (`authority`), and it defaults to no. Only tier 2
33
+ pages you directly.
33
34
 
34
35
  The package ships three deployables, plus one skill:
35
36
 
@@ -37,7 +38,7 @@ The package ships three deployables, plus one skill:
37
38
  | --- | --- | --- |
38
39
  | omp plugin | `/conductor` slash command | Inspect and arm the conductor from inside an omp session: dry-run the queue, read status, pause, resume. |
39
40
  | Standalone daemon | `omp-conductor` binary | The dispatch loop, managed as a background process (`start` / `stop` / `restart`) with a `/healthz` endpoint for a supervisor. |
40
- | Orchestrator heartbeat | omp extension, activated by `.conductor-tick.json` | Prompts a 24/7 orchestrator session on a fixed interval so its standing loop actually runs. Inert in every other session. See [Orchestrator tick](#orchestrator-tick). |
41
+ | Orchestrator heartbeat | omp extension, activated by `.conductor-tick.json` | Prompts a 24/7 orchestrator session on a fixed interval so its standing loop actually runs, and marks the session stalled when its prompts stop being consumed. Inert in every other session. See [Orchestrator tick](#orchestrator-tick). |
41
42
  | Onboarding skill | `skill://conductor-onboarding` | Directs an omp session to interview you, read your repos for real CI gates, and tailor `ORCHESTRATOR.md` — then finish through the wizard. Discovered automatically once the plugin is installed. See [Onboarding](#onboarding). |
42
43
 
43
44
  The first two are thin wrappers over the same `daemon.ts`, so the plugin and the
@@ -66,7 +67,7 @@ second, explicit confirmation. It is a starting point rather than a contract:
66
67
  | Section | Whose |
67
68
  | --- | --- |
68
69
  | Duties (drain, groom, report), escalation tiers, hard boundaries | **Fixed** — they describe how this package already behaves. |
69
- | Releases | **Yours.** Ships defaulting to "humans release; the conductor and its workers never tag, pin, deploy, or publish". Replace it only if you are deliberately delegating releases to that session and then be specific about what, when, on what proof, and what stays permanently forbidden. |
70
+ | Releases | **Yours.** Its opening paragraph is rendered from the `authority` you answered in setup by default "humans release, and you do not merge". Everything under it is the procedure, and a delegated session is told not to cut a release until you have written one: what, when, on what proof, and what stays permanently forbidden. |
70
71
  | Reporting | **Yours**, seeded from the scope you chose in setup. |
71
72
 
72
73
  Reporting is the one half of that the config also knows about, because the wizard
@@ -95,7 +96,7 @@ session that ignores its constraint line still reaches you. Scope is a
95
96
  constraint the model is handed each turn. It is not a gate the model is held to.
96
97
  The enforcement roadmap (a tool-call tripwire, and config-versus-behaviour drift
97
98
  in the daily digest) is
98
- [issue #4](https://github.com/TerrifiedBug/conductor/issues/4).
99
+ [issue #11](https://github.com/TerrifiedBug/conductor/issues/11).
99
100
 
100
101
  Changing the key later does not rewrite an `ORCHESTRATOR.md` you already have:
101
102
  the tick line changes, the brief does not. Edit its Reporting section too, or the
@@ -119,7 +120,7 @@ and create any missing labels):
119
120
  | Key | Meaning |
120
121
  | --- | --- |
121
122
  | `tracker.repo` | The **one** `owner/repo` whose issue list is the queue. This is your planning repo — it does not have to contain any code. |
122
- | `queueLabel` | Open issues in `tracker.repo` carrying this label (default `ready-for-agent`) are the work queue. Nothing else is ever read. |
123
+ | `queueLabel` | Open issues in `tracker.repo` carrying this label are the work queue. Nothing else is ever read. Required: the wizard pre-fills `ready-for-agent`, but a config that omits the key is rejected, not defaulted. |
123
124
  | `routing.repos` + `repo:<name>` labels | Each queued issue must also carry exactly one routing label naming which code repo the work lands in. The conductor cuts the worktree and PR there, from `routing.repos[name].cloneUrl`. An issue with zero or two routing labels is reported as unroutable and skipped — never guessed. |
124
125
 
125
126
  So: one tracker repo supplies the queue, routing labels fan issues out to any
@@ -182,7 +183,9 @@ So the skill does the part a dialog cannot:
182
183
  - **Interviews you** on release policy — humans release (the default), the agent
183
184
  releases to a named boundary, or the agent releases fully — pressing on the one
184
185
  question that makes a delegated release safe: *where does the agent's leg end?*
185
- Plus escalation taste, and which of the two [`reporting.scope`](#your-workflow-vs-the-package)
186
+ The answer becomes the [`authority`](#configuration) pair `/conductor setup`
187
+ records, and the brief is worded from it. Plus escalation taste, and which of
188
+ the two [`reporting.scope`](#your-workflow-vs-the-package)
186
189
  values your answer actually maps to.
187
190
  - **Reads your repos instead of asking about them.** It opens each routing repo's
188
191
  CI workflows, `package.json` scripts and `Makefile`/`justfile`, then *proposes*
@@ -317,12 +320,27 @@ Per tick, for the daemon's project:
317
320
  the configured repo names. These are never dispatched.
318
321
  5. **Check spend.** If spend since local midnight has reached `dailySpendUsd`, the
319
322
  daemon **pauses itself**, pages at Tier 2, and returns.
320
- 6. **Check capacity.** `maxConcurrentWorkers` minus active runs gives the free
321
- slots. If none are free, the tick logs and returns.
323
+ 6. **Check capacity.** `maxConcurrentWorkers` minus *live* workers (runs in
324
+ `claimed` or `running`) gives the free slots. A green PR awaiting a human
325
+ merge occupies its issue but not a slot: its worker is finished, and counting
326
+ it would let two green PRs stop the fleet. If no slot is free, the tick logs
327
+ and returns.
322
328
  7. **Admit issues** up to the free slots, skipping any issue that already has
323
- an active run. An issue that has used `maxAttemptsPerIssue` escalates at Tier 1
329
+ an active run including a green PR, so a second attempt can never land on a
330
+ live PR. An issue that has used `maxAttemptsPerIssue` escalates at Tier 1
324
331
  instead of being admitted.
325
- 8. **Dispatch** the admitted issues concurrently.
332
+ 8. **Ask the tracker whether the work already exists.** For each candidate that
333
+ survived step 7 — so at most one API call per free slot, never one per queued
334
+ issue — the daemon asks whether an **open** PR already closes the issue. If one
335
+ does, the issue is skipped with its PR named in the log. Drafts count: a draft
336
+ PR's branch still holds the only copy of the work. This is the guard the store
337
+ cannot provide, because a store younger than the PRs (a migration, a wiped or
338
+ relocated state directory, a restore onto a new host) has no row to object
339
+ with. If the check itself fails, the candidate is **held**, not admitted, and
340
+ retried next tick: the cost of holding is five minutes, the cost of admitting
341
+ on an unknown is a burned attempt and a duplicate PR. Only that candidate is
342
+ held, so a flaky API cannot stall the rest of the queue.
343
+ 9. **Dispatch** the admitted issues concurrently.
326
344
 
327
345
  Then, per admitted issue:
328
346
 
@@ -330,9 +348,9 @@ Then, per admitted issue:
330
348
  This ordering is the whole crash-safety story: the label, not the local
331
349
  database, is the guard against dispatching the same issue twice. If the process
332
350
  dies at any later point, the next daemon sees the label, eligibility filters the
333
- issue out, and a human decides what to do with the orphan. A store that is lost
334
- can be rebuilt from the tracker; a label that was written too late cannot undo a
335
- duplicate PR.
351
+ issue out, and the orchestrator's drain duty triages the orphan (see below). A
352
+ store that is lost can be rebuilt from the tracker; a label that was written too
353
+ late cannot undo a duplicate PR.
336
354
  2. Create the run row (`claimed`).
337
355
  3. Clear any stale tree for this issue, then add a fresh worktree at
338
356
  `<workspaceRoot>/<issue>` cut from the bare mirror at `<mirrorRoot>/<repo>.git`,
@@ -347,13 +365,46 @@ Then, per admitted issue:
347
365
  | --- | --- | --- | --- |
348
366
  | `pushed-green` | `agent:in-progress` stays until the merge closes the issue | removed | none |
349
367
  | `blocked` | swapped to `agent:blocked` | removed | Tier 1 |
350
- | `failed` / `killed` | swapped to `agent:failed` | **kept** as evidence | Tier 1 |
351
- | unexpected error | swapped to `agent:failed` | kept | Tier 1 |
368
+ | `failed` / `killed` | swapped to `agent:failed` | dirty tree committed to the branch, then **kept** as evidence | Tier 1 |
369
+ | unexpected error | swapped to `agent:failed` | same | Tier 1 |
352
370
 
353
371
  Label swaps add the new label before removing the old one: the reverse order
354
372
  leaves a window in which the issue carries no state label at all, which is
355
373
  exactly the shape eligibility reads as fresh work.
356
374
 
375
+ **A non-graceful end salvages the tree first.** A turns-cap kill, a
376
+ wall-clock kill and a crash are all external and unannounced: they land
377
+ mid-edit, and only the run's *branch* is preserved across attempts — the tree
378
+ is removed `--force` by the next one. So before the escalation is written,
379
+ a dirty tree is committed to the run's own branch as
380
+ `wip(#<issue>): attempt <n> killed by <reason> — auto-salvaged` (everything,
381
+ including files git has never seen) and pushed, and the escalation says where
382
+ it went: `WIP committed to <branch> @ <sha>`. A push that is refused leaves
383
+ the commit in this host's mirror and says so; a salvage that fails outright
384
+ says that, loudly, naming the tree that now holds the only copy. A `blocked`
385
+ run is deliberately *not* salvaged — it stopped on purpose, with turns still
386
+ in hand to commit for itself.
387
+
388
+ ### What a restart does to runs that were in flight
389
+
390
+ A `claimed` or `running` row is a promise that a worker process exists, and a
391
+ daemon that just started knows that promise is broken: its workers died with the
392
+ previous process. At startup — unless another daemon is alive, so a foreground
393
+ `daemon --once` cannot orphan a running daemon's real workers — every such row is
394
+ moved to `orphaned`, with a log line naming the issue, the attempt and the
395
+ worktree. That frees the slots immediately; a fleet must never resume as
396
+ deadlocked as it crashed.
397
+
398
+ Only the rows change. The issue keeps `agent:in-progress` — the label is the
399
+ crash guard against double-dispatch — and deciding what the dead worker's remains
400
+ are worth is the orchestrator's drain-duty judgement, spelled out in its brief:
401
+ an open green PR goes to the merge path, a dirty tree is reported before anything
402
+ destroys it (uncommitted edits have no other copy; unpushed *commits* are safe on
403
+ the run's branch in the mirror, which a retry deliberately reattaches), and a
404
+ clean orphan has its label released so the next tick re-claims it. Orphaned
405
+ attempts still count toward `maxAttemptsPerIssue`, so a crash loop escalates
406
+ instead of redispatching forever.
407
+
357
408
  ### Branch names
358
409
 
359
410
  `<type>/<slug>`, where the type is `fix` when any label's last segment (after `:`
@@ -440,7 +491,7 @@ otherwise looks like a run that was merely unlucky.
440
491
  | Tier | Meaning | Raised by | Delivered to |
441
492
  | --- | --- | --- | --- |
442
493
  | 1 | "Not a human's problem yet" — the run is parked and safe. | Unroutable issue, blocked run, failed or killed run, dispatch error, attempts exhausted. | The orchestrator session, as an injected prompt. Falls back to an issue comment when no orchestrator is running, or when it will not accept the injection. |
443
- | 2 | "The fleet is stopped until you look." | Daily spend cap reached; the project is already paused. | Telegram, when `escalation.telegramChatId` is set and a bot token is readable; otherwise it falls back to the issue comment. |
494
+ | 2 | "The fleet is stopped until you look." | Daily spend cap reached; the installed package changed under the running daemon. Either way the project is already paused. | Telegram, when `escalation.telegramChatId` is set and a bot token is readable; otherwise it falls back to the issue comment. |
444
495
 
445
496
  **The orchestrator** is one persistent, file-backed session per daemon run, resumed
446
497
  across restarts so it remembers what it has already handled. Its `cwd` is the state
@@ -448,9 +499,27 @@ directory, deliberately not a checkout. Delivery resolves when the harness *acce
448
499
  the prompt, not when the model answers it, so a tick never parks behind a model; an
449
500
  injection arriving mid-thought queues as a follow-up instead of interrupting the
450
501
  turn in flight. Its standing orders are explicit: re-brief the worker, file or
451
- comment on issues, or promote to tier 2, and never edit product code, push a branch
452
- or merge a PR. If it fails to start, the daemon logs a warning and runs on, with
453
- tier-1 escalations degraded to issue comments.
502
+ comment on issues, or promote to tier 2, and never edit product code or push a
503
+ branch. Merging is the one line worded from config see [`authority`](#configuration).
504
+ If it fails to start, the daemon logs a warning and runs on, with tier-1
505
+ escalations degraded to issue comments.
506
+
507
+ **Or no orchestrator at all.** Set `escalation.orchestrator` to `"external"` when
508
+ you already run your own supervising session — a visible TUI session in a pane,
509
+ typically. The daemon then starts none of its own and every tier-1 escalation
510
+ posts as an issue comment, which is what that session drains. One brain, and it
511
+ is the one you can watch.
512
+
513
+ **Answering a tier 1 is only half of it.** A blocked or failed run leaves its state
514
+ label on the issue, and eligibility reads any state label as disqualifying, so an
515
+ answered issue that keeps one is never re-claimed and the answer is inert — nothing
516
+ fails, the issue just stops existing as far as dispatch is concerned.
517
+ [`omp-conductor unblock <issue>`](#cli-reference) is the way back: it clears the
518
+ label through the same tracker the dispatcher writes with. The brief tells the
519
+ orchestrator to run that verb rather than edit the label itself, and that is not a
520
+ formality — orphan detection works by comparing `agent:in-progress` labels against
521
+ live runs, and it is only trustworthy while every state label on the tracker was
522
+ written by this package.
454
523
 
455
524
  Tier 2 borrows the bot token that `omp-telegram` already owns, at
456
525
  `~/.omp/agent/telegram/.env` (or `$OMP_TELEGRAM_STATE_DIR/.env`). If you run that
@@ -464,8 +533,8 @@ issue on every poll, so a ledger in the store — keyed by project, issue, tier
464
533
  summary — makes a recurring condition page **once** and suppresses the five-minute
465
534
  repeats. The marker is recorded only on successful delivery, so a page that could
466
535
  not be delivered is retried on the next tick instead of being written off as sent.
467
- The spend-cap summary carries the date, so the same cap pages again tomorrow but
468
- only once per day.
536
+ The spend-cap and integrity-tripwire summaries carry the date, so the same
537
+ condition pages again tomorrow but only once per day.
469
538
 
470
539
  If `fallbackToIssueComment` is off and no Telegram transport is configured,
471
540
  delivery throws instead of dropping silently. The failure is logged and retried,
@@ -484,7 +553,10 @@ Runtime state lives elsewhere, under `$OMP_CONDUCTOR_RUNTIME_DIR` (default
484
553
  atomically, and `daemon.log`, appended across every boot so the previous failure is
485
554
  still there when you go looking. It is kept apart from the config directory because
486
555
  it is meaningless after a reboot, and the pidfile's liveness is probed on every
487
- read — a stale one never blocks a `start`.
556
+ read — a stale one never blocks a `start`. Both `start` and a bare `daemon` write
557
+ the pidfile, so a daemon run in the foreground under systemd is as visible to
558
+ `status` as a backgrounded one; `daemon --once` writes nothing, because that drill
559
+ is exactly what the orphan-reconciliation guard reads the pidfile to protect.
488
560
 
489
561
  The file is validated on every read. A malformed config produces one readable error
490
562
  listing every fault, and the daemon refuses to start rather than running with half
@@ -548,7 +620,12 @@ A complete, valid config for one project with two target repos:
548
620
  "workerModel": "smol",
549
621
  "escalation": {
550
622
  "telegramChatId": "123456789",
551
- "fallbackToIssueComment": true
623
+ "fallbackToIssueComment": true,
624
+ "orchestrator": "embedded"
625
+ },
626
+ "authority": {
627
+ "merge": "human",
628
+ "release": "human"
552
629
  },
553
630
  "reporting": {
554
631
  "scope": "material"
@@ -574,6 +651,8 @@ Field notes:
574
651
  | `gates` | The exact cheap commands CI also runs, each with the `cwd` it runs from (`cwd` defaults to `.`). Running the real gate locally is what makes an unattended push safe — a subset lets an error outside the source dir reach the runners. |
575
652
  | `caps` | Per-project overrides; omit it or pin only the fields you want to change. |
576
653
  | `escalation.fallbackToIssueComment` | Defaults to `true`. Absent means "yes, still tell me". |
654
+ | `escalation.orchestrator` | Optional; `"embedded"` (default) or `"external"`. `external` means an orchestrator session already runs elsewhere: the daemon starts none, and tier-1 escalations post as issue comments for that session to drain. Any other value is an error. |
655
+ | `authority` | Optional; `{ "merge": …, "release": … }`, each `"human"` (default) or `"orchestrator"`. It grants nothing to the daemon — it words the orchestrator's standing orders and the Releases paragraph of the rendered brief, so the config and the prompt cannot disagree about who holds the merge button. Unknown keys and any other value are errors, never folded to the default. |
577
656
  | `reporting.scope` | Optional; `"material"` (default) or `"escalations"`. Every orchestrator tick appends the matching constraint line to its prompt, re-read from this file each tick — see [Your workflow vs. the package](#your-workflow-vs-the-package). It constrains what the session is told to report; it is not an outbound filter. A config written without the key keeps reporting material events. Any other value is an error, never folded to the default. |
578
657
  | `workspaceRoot` / `mirrorRoot` | Optional; default to `worktrees/` and `mirrors/` under the state directory. `~` is expanded. |
579
658
 
@@ -590,13 +669,13 @@ it never gets prompted, so it never runs anything. Installing
590
669
 
591
670
  The heartbeat is **inert unless the session's cwd contains
592
671
  `.conductor-tick.json`**, so it costs an ordinary session nothing. Drop the file
593
- in the orchestrator's working directory (on the fleet host, `/root/fleet`):
672
+ in the orchestrator's working directory:
594
673
 
595
674
  ```json
596
675
  {
597
676
  "intervalSeconds": 900,
598
677
  "armedFile": "state/armed",
599
- "accessFile": "/root/.omp/agent/telegram/access.json",
678
+ "accessFile": "/home/fleet/.omp/agent/telegram/access.json",
600
679
  "message": "Run your standing loop from ORCHESTRATOR.md now."
601
680
  }
602
681
  ```
@@ -606,13 +685,18 @@ in the orchestrator's working directory (on the fleet host, `/root/fleet`):
606
685
  | `intervalSeconds` | yes | — | Whole seconds between ticks, minimum `60`. A tick costs a full turn of a frontier model, so a sub-minute period is refused rather than obeyed. |
607
686
  | `armedFile` | no | none — the gate passes | Path to the arm marker. A tick does nothing while the file is missing. Relative paths resolve against the session cwd, so `state/armed` means `<cwd>/state/armed`. |
608
687
  | `accessFile` | no | none — the gate passes | Path to the Telegram bridge's `access.json`. Every tick re-reads it and requires `enabled: true` with exactly one entry in `allowFrom`. Relative paths resolve against the session cwd. **Configure this on any fleet deploy** — see below. |
609
- | `message` | no | `Tick <ISO timestamp>: run your standing loop from ORCHESTRATOR.md now.` followed by the `reporting.scope` line | Sent verbatim when set — and then it owns the whole contract: no scope line is appended to a prompt you wrote yourself. The default carries the timestamp, which is what makes two consecutive ticks distinguishable in the session log. |
688
+ | `message` | no | `Tick <ISO timestamp>: re-read <workspaceRoot>/ORCHESTRATOR.md from disk, then run your standing loop from it.`, then the `reporting.scope` line, then the delivery rule | Sent verbatim when set — and then it owns the whole contract: neither the scope line nor the delivery rule is appended to a prompt you wrote yourself. Re-read from disk on **every** tick, so rewording it binds the next heartbeat instead of waiting for a session restart; a re-read that fails — caught mid-edit, removed, or invalid — keeps the value read at session start rather than stopping the heartbeat. `intervalSeconds` is *not* re-read: rescheduling a live timer still needs a restart. The default *orders* the session to re-read its brief, naming the path resolved from the conductor config — a 24/7 session otherwise acts on the copy loaded at its start (resume included), and an amendment added between ticks never binds — and carries the timestamp, which makes two consecutive ticks distinguishable in the session log. |
610
689
 
611
690
  A tick sends one message (`customType` `omp-conductor.tick`, attributed to the
612
- user): the standing-loop prompt, plus the one constraint line the project's
691
+ user): the standing-loop prompt, the one constraint line the project's
613
692
  [`reporting.scope`](#your-workflow-vs-the-package) resolves to, re-read from the
614
- conductor config on every tick. It starts a turn if the session is idle; while a
615
- turn is streaming it is queued as a follow-up and consumed when that turn ends.
693
+ conductor config on every tick, and a delivery rule. That last line is there
694
+ because end-of-turn text reaches the operator's Telegram only on a turn that
695
+ *began* as an inbound Telegram message: a tick is injected locally, so anything
696
+ the session merely writes at the end of one is read by nobody, and a reportable
697
+ event has to be delivered by an explicit `telegram_send` call the session
698
+ watched succeed. The tick starts a turn if the session is idle; while a turn is
699
+ streaming it is queued as a follow-up and consumed when that turn ends.
616
700
  It sends **nothing** when:
617
701
 
618
702
  - `/conductor pause` (or `omp-conductor pause`) holds the pause flag, the same
@@ -620,7 +704,9 @@ It sends **nothing** when:
620
704
  - `armedFile` is configured and missing;
621
705
  - `accessFile` is configured and the escalation channel is not verifiably up;
622
706
  - an earlier tick is still queued. Ticks coalesce rather than stack, so a slow
623
- turn cannot leave a backlog of heartbeats behind it.
707
+ turn cannot leave a backlog of heartbeats behind it — and two coalesced ticks
708
+ in a row are the signal that the session is not slow but wedged, which is
709
+ what the [stall marker](#a-wedged-session-and-the-marker-that-notices) is for.
624
710
 
625
711
  ### The escalation channel is a gate, and it fails closed
626
712
 
@@ -642,16 +728,84 @@ Leaving `accessFile` unset passes the gate, because an ordinary developer sessio
642
728
  that happens to have a `.conductor-tick.json` has no bridge to check. It is not an
643
729
  off switch for the check: **a fleet deploy always sets it.**
644
730
 
645
- Every tick — sent or skipped — is logged with its reason (`paused`, `not armed`,
646
- `escalation channel down`, `tick already pending`) to the omp log. Skips are
647
- deliberately silent in the UI: a paused fleet would otherwise raise a notification
648
- every interval, forever. The one exception is a malformed `.conductor-tick.json`,
731
+ Every tick — sent or skipped — is logged with its reason (`not armed`,
732
+ `escalation channel down`, `tick already pending`) to the omp log. `/conductor
733
+ pause` is deliberately **not** one of the gates: pause stops the *dispatcher*
734
+ claiming work, and the tick drives a different session — one whose duties
735
+ (grooming the queue, draining escalations, reporting) are exactly what stays
736
+ useful while dispatch is stopped. Its own off switch is the arm marker. Skips
737
+ are deliberately silent in the UI: a disarmed fleet would otherwise raise a
738
+ notification every interval, forever. The one exception is a malformed
739
+ `.conductor-tick.json`,
649
740
  which notifies once at session start and leaves the heartbeat off; silent failure
650
741
  there is the failure mode the heartbeat exists to prevent. A conductor config that
651
742
  cannot supply a reporting scope logs `tick reporting scope: using material` once
652
743
  per session. The interval does not re-log it, because the file is unlikely to fix
653
744
  itself between two ticks.
654
745
 
746
+ ### A wedged session, and the marker that notices
747
+
748
+ Coalescing is also the only wedge detector this package has. On 2026-08-07 the
749
+ dogfood fleet's orchestrator finished a turn, logged `ui.loop-blocked` right
750
+ after an auto-compaction threshold decision, and never started another. The
751
+ process stayed alive, so herdr's recovery — agent listed AND a non-shell
752
+ foreground process — read healthy. The dispatch daemon is a separate process and
753
+ kept working, so `/healthz` was green all night, while the one brain holding
754
+ merge authority sat on a green PR it never merged. A tick injected two minutes
755
+ into the wedge and an operator's Telegram message five minutes later both went
756
+ unconsumed for 23 minutes, until a manual `SIGTERM`. The heartbeat logged `tick
757
+ skipped: tick already pending` throughout, which is exactly what a merely slow
758
+ turn looks like.
759
+
760
+ So the heartbeat counts them. Two consecutive coalesced ticks — a full hour at
761
+ the reference 1800-second interval, generous by construction — mean the last
762
+ prompt was never consumed, and the extension:
763
+
764
+ - writes `<session cwd>/.conductor-stalled`, one line of `<ISO timestamp>
765
+ <diagnosis>`.
766
+ - logs at **error** level: `orchestrator stalled: 2 ticks queued unconsumed —
767
+ the agent loop is not draining; see .conductor-stalled`.
768
+
769
+ Both escapes deliberately leave the session, because a loop that cannot drain
770
+ its queue cannot report on itself — that is the whole failure.
771
+
772
+ **Two things read it.** A marker nobody consumes is an artifact, not an alert:
773
+
774
+ - **The daemon**, on its own five-minute tick, and *before* its pause check. The
775
+ orchestrator is a different process and can be wedged while the fleet is
776
+ deliberately paused — precisely the state the dogfood fleet was in when this
777
+ happened. One tier-2 page per stall, dated, re-armed when the marker clears,
778
+ and latched only once the page is confirmed delivered, so an escalation
779
+ channel that fails on the one tick that noticed does not buy permanent
780
+ silence. It restarts nothing: a wedge lands mid-turn, and this process cannot
781
+ tell a half-applied edit from an idle loop.
782
+ - **herdr-conductor's recovery**, which treats marker-present as *not live*. Its
783
+ liveness test — agent listed AND a non-shell foreground process — passes
784
+ straight through a wedge, so without this a recovery run certifies a stuck
785
+ session as healthy and returns "nothing to do".
786
+
787
+ The daemon is what covers a wedge beginning *between* herdr's lifecycle events,
788
+ since a session that stays alive and stops working emits none of them.
789
+
790
+ The first tick that actually sends clears the counter and deletes the marker,
791
+ and it deletes one it did not write: recovery normally arrives as a fresh
792
+ process resuming the same transcript, so the session doing the clearing is not
793
+ the session that stalled. Nothing else removes the file. Neither the write nor
794
+ the delete can take the heartbeat down — a filesystem error is logged and the
795
+ tick carries on.
796
+
797
+ `omp-conductor status` reads the same marker from the **state directory** and
798
+ prints one more line under the daemon block:
799
+
800
+ ```text
801
+ orchestrator STALLED since 2026-08-07T06:27:55.123Z — 2 ticks queued unconsumed — the agent loop is not draining
802
+ ```
803
+
804
+ That reading is the reference deploy's convention — the orchestrator session
805
+ runs from `~/.omp/conductor`, which is the state directory — and it is
806
+ one-directional: a line there proves a wedge, and its absence proves nothing,
807
+ least of all on a fleet whose session lives somewhere else.
808
+
655
809
  ## CLI reference
656
810
 
657
811
  ```bash
@@ -659,6 +813,8 @@ omp-conductor start [--port N] [--project NAME]
659
813
  omp-conductor stop
660
814
  omp-conductor restart [--port N] [--project NAME]
661
815
  omp-conductor status [--project NAME]
816
+ omp-conductor tail <issue> [--project NAME]
817
+ omp-conductor unblock <issue> [--project NAME]
662
818
  omp-conductor daemon [--once] [--port N] [--project NAME]
663
819
  omp-conductor pause
664
820
  omp-conductor resume
@@ -671,12 +827,14 @@ omp-conductor help
671
827
  | `start` | Spawn the loop in the background, detached, and wait until it answers `GET /healthz` on `:8787`. Refuses if one is already live, naming its pid. If the process dies or never serves, `start` cleans up after it and quotes the tail of `daemon.log`. |
672
828
  | `stop` | `SIGTERM`, then `SIGKILL` after a 10-second grace period. Prints `not running` when there is nothing to stop. |
673
829
  | `restart` | `stop` then `start`, inheriting the running daemon's port and project unless a flag overrides them — a restart that quietly moved to the default port would leave every existing health check pointing at nothing. |
674
- | `status [--project NAME]` | Pause state, config and state paths, resolved caps, active runs and today's usage, plus a `daemon` block: pid, uptime, port, project, `/healthz` result and log path. Reads while a daemon in another process writes. |
675
- | `daemon` | Run the loop in the **foreground**, ticking every 5 minutes and serving `/healthz`. This is what `start` launches. |
676
- | `daemon --once` | Run a single tick and exit. No HTTP server. |
830
+ | `status [--project NAME]` | Pause state, config and state paths, resolved caps, active runs and today's usage, plus a `daemon` block: pid, uptime, port, project, `/healthz` result and log path. A `.conductor-stalled` marker in the state directory adds an `orchestrator STALLED since …` line — see [the stall marker](#a-wedged-session-and-the-marker-that-notices). Reads while a daemon in another process writes. |
831
+ | `tail <issue>` | Follow the newest run for that issue: the worker's assistant text as `assistant: …` and each tool it calls as `tool: <name>`, printed as they land. Workers are omp sessions inside the daemon rather than terminals, so this is the only way to watch one live — a herdr pane running it becomes an observation window. Starts from the top of the transcript, not the end, so attaching to a run that is already ten turns in shows those ten turns. Exits `1` with `no run recorded for #N` when the issue has never been dispatched, or `no transcript yet (state: …)` when the attempt has not opened one. Otherwise it runs until `Ctrl-C`, or until the run has finished and its transcript has been silent for five seconds, and prints `run ended: <state>`. |
832
+ | `unblock <issue>` | Remove that issue's `blocked` and `failed` state labels through the tracker, so the next tick can claim it again. This is the supported way back for an escalation you answered: eligibility disqualifies any issue carrying a state label, so an answered issue that keeps one is never re-claimed and the answer is inert. Removing a label the issue does not carry is a no-op, so both are always cleared and neither has to be looked up first. `agent:in-progress` is deliberately not touched — it means a worker process exists, which is not something an answer changes. The run history is left exactly as it is: an answered block still spent a worker, so it still counts toward `maxAttemptsPerIssue`, and the output says how many attempts remain — or warns that the next tick will escalate instead of dispatching, when none do. Exits `2` with `unblock needs an issue number` on a missing or malformed positional. |
833
+ | `daemon` | Run the loop in the **foreground**, ticking every 5 minutes and serving `/healthz`. This is what `start` launches, and what a systemd unit should call. Writes the pidfile itself, and refuses with `another daemon is alive (pid N); stop it first` rather than becoming a second dispatcher. |
834
+ | `daemon --once` | Run a single tick and exit. No HTTP server, and no pidfile — a drill must not register itself as the daemon, or the next reader believes it and the real daemon's in-flight runs get reconciled as orphans. |
677
835
  | `--port N` | Accepted by `start`, `restart` and `daemon`. Both `--port 9000` and `--port=9000` work; missing or out of range exits `2` rather than falling back to the default, because probing the wrong endpoint is worse than a hard failure. |
678
836
  | `--project NAME` | Pick the project to service. One daemon process serves exactly one project; with several configured projects the name is required. |
679
- | `pause` | Stop claiming new work. The running daemon notices on its next tick; runs already in flight finish. |
837
+ | `pause` | Stop claiming new work. The running daemon notices on its next tick; runs already in flight finish. The orchestrator heartbeat keeps ticking — its gate is the arm marker, not this flag. |
680
838
  | `resume` | Allow claiming again. |
681
839
  | `brief-upgrade` | Compare a project's `ORCHESTRATOR.md` against the brief this version of the package ships. Reports by default; see [Keeping a brief current](#keeping-a-brief-current). |
682
840
  | `--apply` | Only for `brief-upgrade`. Replaces the half above the `YOURS TO EDIT` banner and keeps everything below it, backing the previous file up first. Ignored when the brief cannot be split or the template is unrendered. |
@@ -717,13 +875,42 @@ dispatcher. The brief is explicit about the boundary:
717
875
  | Add or update tests for behaviour it introduced. | Suppress a warning, delete an assertion, or special-case an input to make a check pass. |
718
876
  | Run the repo's configured cheap gates, each from its listed `cwd`, over the whole tree. | Run docker or image builds, production builds, browser/e2e suites, or the full test suite on the shared host — CI owns the heavy gates. |
719
877
  | Review its whole diff, then commit and **push once**. One corrective push if CI is red. | Force-push, `git add -f`, or add AI/co-author attribution. Red twice means stop and report, not push a third time. |
720
- | Open a PR that links the issue, and watch CI to a verdict with `gh pr checks --watch`. | Run `gh pr merge`. **Merge authority is a human's alone**, so PRs land one at a time with a freshness re-check two workers merging concurrently is how agent PRs clobber each other. |
878
+ | Open a PR that links the issue, and watch CI to a verdict with `gh pr checks --watch`. | Run `gh pr merge`. **A worker never merges** — that one is absolute, whoever else holds the authority — so PRs land one at a time with a freshness re-check; two workers merging concurrently is how agent PRs clobber each other. Who *may* merge is the [`authority`](#configuration) answer, and it is never the worker. |
721
879
  | Escalate: ambiguity, a cross-repo contract, a needed credential, a product or data-migration decision, a blocking existing test, CI red twice, or most of the wall-clock budget burned. | **Cut a release**, push a tag, publish to npm, edit a deployment pin, deploy, or touch infrastructure or secrets — permanently out of scope. Releases are batched and decided outside this loop, so "this needs releasing" is a thing to report, never a task to take on. |
722
880
 
723
881
  The worker ends with a six-line evidence report (issue, pr, state, gates, changed,
724
882
  next). `pushed-green` means it watched the checks go green rather than expecting
725
883
  them to.
726
884
 
885
+ ### The boundary is prompt text — and one tripwire
886
+
887
+ A worker session gets a `cwd` and nothing else: the harness applies no filesystem
888
+ or network restriction, so "touch any path outside its worktree" is a rule the
889
+ session is asked to keep, not one the process cannot break. Target selection *is*
890
+ mechanical — only a repo in `routing.repos` is ever checked out — and so are the
891
+ caps, but between them they bound where work starts and how much of it happens,
892
+ not how far a wandering or prompt-injected session can reach.
893
+
894
+ The one path that is checked mechanically is the conductor itself. At startup the
895
+ daemon sha256s every `.ts` and `.md` file of its own installed `src/` — the
896
+ dispatcher and the briefs both, since rewriting a brief buys more than rewriting
897
+ the loop — and re-walks that tree on every tick (about 0.6 ms). Any difference at
898
+ all, changed or added or removed, is read as the package having been modified
899
+ underneath a running daemon: the tick claims nothing, the fleet is paused, and a
900
+ tier-2 escalation naming the first few differing paths pages you **once**, not
901
+ every five minutes.
902
+
903
+ **A normal deploy never trips it.** The baseline is recorded per daemon process,
904
+ so installing a new build and restarting the unit re-records it from the new
905
+ files; only a change that lands *while* a daemon is holding the package open can
906
+ diverge from it. That also means `omp-conductor resume` on its own will not hold
907
+ — the next tick re-walks, still differs, and pauses again. Put the files back, or
908
+ restart onto the build you meant to be running.
909
+
910
+ This is detection, not prevention. It catches the worker that wandered, and it
911
+ catches the human who edited the live install "just to test something", which in
912
+ practice is the commoner of the two.
913
+
727
914
  ## Limitations
728
915
 
729
916
  Known and deliberate in this version:
@@ -766,12 +953,19 @@ Known and deliberate in this version:
766
953
  (the orchestrator's) no matter how many workers are running, and no amount of
767
954
  `maxConcurrentWorkers` changes that.
768
955
 
769
- The cap does work. The admission loop (`src/daemon.ts:405-445`) computes
770
- `slots = maxConcurrentWorkers - active runs`, admits at most that many issues per
771
- tick, and dispatches them together. To see them, read `omp-conductor status`,
772
- which lists every active run, or follow `daemon.log`.
956
+ The cap does work. The admission loop (`admitCandidates` in `src/daemon.ts`) computes
957
+ `slots = maxConcurrentWorkers - live workers`, admits at most that many issues
958
+ per tick, and dispatches them together. To see them, read `omp-conductor
959
+ status`, which lists every occupied issue, or follow `daemon.log`.
773
960
  - **Merges, releases and deploys are human-only, by design.** The conductor
774
961
  produces green PRs and stops.
962
+ - **Worker confinement is behavioural.** `runWorker` hands the session a `cwd` and
963
+ a brief; nothing stops it reading or writing elsewhere on the host, and on a
964
+ single-user deploy that includes the state directory and the host's `gh`
965
+ credentials. The [integrity tripwire](#the-boundary-is-prompt-text--and-one-tripwire)
966
+ turns one case of this into a paused, paged fleet after the fact; the prevention
967
+ half is a deployment concern — run workers as a least-privileged uid whose write
968
+ access ends at its worktree and mirror.
775
969
 
776
970
  ## License
777
971
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "omp-conductor",
3
- "version": "0.2.1",
3
+ "version": "0.3.1",
4
4
  "type": "module",
5
5
  "license": "MIT",
6
6
  "description": "A 24/7 dispatcher that takes ready GitHub issues to green, mergeable PRs using omp coding sessions, with tiered escalation first to an orchestrator session and then to a human.",
@@ -119,16 +119,25 @@ are the orchestrator and a human.** The orchestrator is the right agent for it i
119
119
  any agent is: it is long-lived, it can see everything that merged since the last
120
120
  release, and batching is exactly the judgement a per-issue session cannot make.
121
121
 
122
- So frame it as **three** options, in this order, and name the default:
122
+ So frame it as **three** options, in this order, and name the default. Each one
123
+ maps to an `authority` answer the wizard asks for directly — the interview
124
+ decides *which answer to give*, and `/conductor setup` is what records it. Never
125
+ write authority into the brief by hand: the config is what words the brief, and
126
+ a hand-edit is a second source that outlives the operator's memory of making it.
123
127
 
124
128
  1. **Humans release.** *(default, and what the package ships)* Work ends at a
125
129
  green PR. Merging is a separate human action; releasing is a separate human
126
130
  action after that. Neither a worker nor the orchestrator ever tags, pins,
127
131
  publishes, or deploys. "This needs releasing" becomes something the
128
- orchestrator *reports*.
132
+ orchestrator *reports*. → `authority: merge=human, release=human`; answer
133
+ **no** to both wizard confirms.
129
134
  2. **The orchestrator releases up to a named boundary.** Delegating part of the
130
135
  release to the supervising session, with the stopping line written down.
131
- 3. **The orchestrator releases fully.**
136
+ usually `merge=orchestrator, release=orchestrator`, with the boundary in
137
+ the procedure rather than in the grant; `merge=orchestrator, release=human`
138
+ when the line falls before the tag.
139
+ 3. **The orchestrator releases fully.** → `merge=orchestrator,
140
+ release=orchestrator`.
132
141
 
133
142
  Then ask the question that makes option 2 real:
134
143
 
@@ -138,8 +147,8 @@ Not "can it release" — *where does its leg stop*. A boundary you cannot state
138
147
  one sentence is not a boundary, and an orchestrator with a vague release mandate
139
148
  is one that eventually publishes something at 03:00.
140
149
 
141
- Give them a worked answer so they can calibrate. The Veltro deployment's answer:
142
- **the agent's leg ends at the merged suite pin.** It may open and land the PR that
150
+ Give them a worked answer so they can calibrate. One real deployment's answer:
151
+ **the agent's leg ends at the merged version pin.** It may open and land the PR that
143
152
  bumps the module image tags, because that artefact is reviewable, reversible by a
144
153
  revert, and its correctness is checkable by a named CI check. Deploying that pin
145
154
  to prod is **operator territory** — it needs a person who can watch it, and who
@@ -344,7 +353,8 @@ about gates: an operator describes the release they *remember*, and a release is
344
353
  the one procedure where being approximately right is worst.
345
354
 
346
355
  Skip this step only for option 1 (humans release). There is nothing to scaffold:
347
- the shipped paragraph is already correct and the brief keeps it verbatim.
356
+ the rendered paragraph already says humans hold both, and there is no procedure
357
+ under it to write.
348
358
 
349
359
  ### Find the release authority
350
360
 
@@ -423,7 +433,8 @@ has a safe fleet. Your job is to raise it to *this* fleet.
423
433
  The file has a hard line in it — an HTML comment banner reading
424
434
  `YOURS TO EDIT`. Respect it in both directions:
425
435
 
426
- **Above the banner — leave it alone.** Duties (drain, groom, report), the
436
+ **Above the banner — leave it alone.** The three duties — drain (unstick what is
437
+ stuck), groom (keep the queue worth draining), report — the
427
438
  escalation-tier table, and the hard boundaries describe how the package already
428
439
  behaves. Rewriting them makes the brief disagree with the code, and the code wins.
429
440
  The evidence rule in particular is not negotiable: *every claim cites evidence — a
@@ -432,24 +443,24 @@ PR URL, an issue number, or a named check actually read.* "Should be fine",
432
443
 
433
444
  Know what is *not* up there, though, because operators expect it to be: the
434
445
  orchestrator's own merge and release authority is **not** a hard boundary. It is
435
- policy, it lives in Releases, and it defaults to none. What is fixed above the
436
- banner is that a *worker* never merges or releases, and that PRs land one at a time
437
- with a freshness re-check. Delegating a release to the orchestrator does not touch
438
- either of those, so it needs no negotiation with the shipped half.
446
+ config the two `authority` answers — and it defaults to none. What is fixed
447
+ above the banner is that a *worker* never merges or releases, and that PRs land
448
+ one at a time with a freshness re-check. Delegating a release to the
449
+ orchestrator does not touch either of those, so it needs no negotiation with the
450
+ shipped half.
439
451
 
440
452
  **Below the banner — rewrite from the interview.**
441
453
 
442
- - **Releases.** Replace the section with the procedure you scaffolded in Step 4. If
443
- humans-release, say so in one short paragraph and stop; do not leave the "replace
444
- this paragraph if you are delegating" instructions in a finished brief, because
445
- a standing prompt full of alternatives it did not choose is a standing prompt
446
- the session has to guess its way through. If they chose a boundary, write the
447
- boundary as a sentence with an end *"your leg ends at the merged pin PR; you
448
- never deploy it"*then the scaffolded steps, then the forbidden list with its
449
- citations. State the merge authority explicitly either way: a brief that
450
- prescribes landing a release PR without ever saying the orchestrator may merge is
451
- a brief the session has to infer permission from, and it will infer wrong in one
452
- direction or the other.
454
+ - **Releases.** The section's opening paragraph is rendered from the `authority`
455
+ answers, and Duty 1's "the PR is green" branch is rendered from the same
456
+ place. Leave both alone: they are the config speaking, and re-running
457
+ `/conductor setup` is how they change. What you write is everything under that
458
+ paragraph — the procedure you scaffolded in Step 4. If humans-release, there is
459
+ nothing to scaffold and you stop. If they chose a boundary, write the boundary
460
+ as a sentence with an end *"your leg ends at the merged pin PR; you never
461
+ deploy it"* — then the scaffolded steps, then the forbidden list with its
462
+ citations. Do not restate the merge grant in your own words: a second spelling
463
+ of it is exactly the disagreement the rendered paragraph exists to prevent.
453
464
  - **Reporting.** Rewrite it as the one scope they chose, in the second person,
454
465
  concretely. Delete the description of the scope they did not choose: it is
455
466
  useful in a template and noise in a live prompt. Keep the closing constraint