omp-conductor 0.15.4 → 0.15.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -251,10 +251,20 @@ command exits without changing anything when all three surfaces already use that
251
251
  release, the Herdr plugin is pinned to its exact `gitHead`, and the brief is
252
252
  current.
253
253
 
254
- When a daemon is live, the command drains and restarts its recorded project.
255
- An explicit `--project` that names a different project is rejected before pause
256
- or installation, so an operator cannot drain one queue and kill another queue's
257
- workers.
254
+ The upgrade is host-wide, because everything it replaces is: one daemon serves
255
+ every configured project, so its restart lands on all of them at once. A bare
256
+ `omp-conductor upgrade` therefore drains **every** project's workers and
257
+ refreshes **every** project's brief, and pauses them with the fleet-wide
258
+ sentinel — which leaves any per-project `hold` you set standing when it
259
+ restores dispatch.
260
+
261
+ `--project` narrows that only when it is truthful to do so. When a live daemon
262
+ recorded a single project, the command drains and restarts that project, and an
263
+ explicit `--project` naming a different one is rejected before pause or
264
+ installation. When the daemon serves every configured project and there is more
265
+ than one, `--project` is rejected too: draining one queue and then restarting
266
+ the shared daemon would kill another queue's workers without ever counting
267
+ them. Re-run without the flag.
258
268
 
259
269
  Ticks remain in their existing armed or disarmed state, so an ordinary update
260
270
  does not halt the exact pane or require another Telegram arm challenge. Progress
@@ -2334,7 +2344,7 @@ omp-conductor help
2334
2344
  | `start` | Start `herdr-fleet.service` when that optional unit is installed, clearing a previous pane-recovery pin, then spawn the dispatch loop in the background and wait until it answers `GET /healthz` on `:8787`. Without systemd or that unit it keeps the standalone daemon behaviour. It never clears pause or arms ticks. Refuses if a daemon is already live, naming its pid; if the process dies or never serves, it cleans up and quotes the tail of `daemon.log`. |
2335
2345
  | `stop` | Prefer `systemctl stop omp-conductor.service` when that unit's MainPID is the live daemon — systemd then owns the stop and will not schedule a restart for the exit it just requested. Otherwise `SIGTERM`, then `SIGKILL` after a 10-second grace period. Prints `not running` when there is nothing to stop, and tags the confirmation with `(via systemctl)` when the unit path was used. |
2336
2346
  | `restart [--now] [--timeout SECONDS] [--port N] [--project NAME]` | Drains the fleet by default: pause new claims, wait until live workers reach `0 / N` (bounded by `--timeout SECONDS`, default 1800 = 30 min), restart, then restore the prior dispatch state. Prefer `systemctl restart` when the unit owns the live pid so the replacement stays supervised; otherwise `stop` then `start`, inheriting the running daemon's port and project unless a flag overrides them. `--now` skips the drain and restarts immediately, orphaning any live runs (old behaviour). A drain that hits `--timeout` restarts nothing and leaves dispatch paused — `omp-conductor resume` lifts it, or re-run `restart` to keep waiting. The new process **salvages dirty live worktrees before orphaning** those rows — see [Deploying a new package onto a busy fleet](#deploying-a-new-package-onto-a-busy-fleet). |
2337
- | `upgrade [--to VERSION] [--project NAME]` | Deterministically update the Bun-global CLI, omp plugin, Herdr recovery plugin, and managed brief as one release. Resolves the npm version and exact `gitHead`, pauses only new claims, drains active workers, installs all surfaces, reloads Herdr and the daemon, waits for pane recovery, verifies identities and fleet health twice, then restores the original dispatch state. A no-op when already current. Failure leaves dispatch paused. Must run outside a Herdr-managed session. |
2347
+ | `upgrade [--to VERSION] [--project NAME]` | Deterministically update the Bun-global CLI, omp plugin, Herdr recovery plugin, and managed brief as one release. Resolves the npm version and exact `gitHead`, pauses only new claims, drains active workers, installs all surfaces, reloads Herdr and the daemon, waits for pane recovery, verifies identities and fleet health twice, then restores the original dispatch state. Host-wide by default: one daemon serves every configured project, so a bare run drains all of them and refreshes every brief. `--project` is rejected when the live daemon serves several projects — draining one queue and restarting the shared daemon would kill another's workers. A no-op when already current. Failure leaves dispatch paused. Must run outside a Herdr-managed session. |
2338
2348
  | `status [--project NAME]` | Layered fleet report first: `dispatch` / `ticks` / next scheduled tick / `pane` / `recovery` / `herdr` / `telegram` / `brief` / `decisions` / optional `failure classes` and `code graph` / `daemon`, then the project body. The project body includes the latest completed dispatch timestamp, ready/routed/admitted counts, bounded hold groups, and the GitHub API budget (`graphql` / `core` remaining and reset, in the caps block); API failures are marked `DEGRADED` so queue starvation cannot look idle. Active-run lines overlay cooperative worker `paused`/`pausing` from `/healthz` without changing SQLite `running` state or the live worker count. The next tick comes from the live heartbeat process, not a guess from log timestamps. Telegram health uses `getMe` to prove API authentication without sending a message and reports inbound bridge configuration separately. Configured graphs report prerequisites, indexed repos, timer state, and refresh freshness without blocking dispatch. A `reports` block lists everything the outbox has not delivered, with its age, and prints `pending` (nobody has it) differently from `SENDING` (outcome unknown, it may already have arrived) — see [Report delivery](#report-delivery-the-outbox). The daemon block includes `rss` from `/healthz`; live workers add a busy-deploy warning. A `.conductor-stalled` marker adds an `orchestrator STALLED since …` line. |
2339
2349
  | `ledger [--issue N] [--limit N]` | The action audit: every [mediated-verb](#the-mediated-verbs-126) mutation and every next-attempt turn budget. Verb entries include the arguments, decision, named refusal, and resulting SHA. Turn-budget entries remain after an override is replaced or consumed. Reads (`conductor_pr_status`) are absent so polling cannot bury the signal. `--issue` narrows both histories; `--limit` defaults to 50. Recent verb refusals and pending turn overrides also appear in `status`. |
2340
2350
  | `board [--project NAME]` | Live keyboard-driven kanban over the same SQLite and `/healthz` truth as `status`, plus the tracker's current labels: Queue, Claimed, Running, Green, Blocked, Failed, Orphaned, the last 24 hours of Merged and Settled, and Parked (an issue the tracker has not confirmed closed — still open, or a label read that failed — so nothing dispatches it until a human labels it). Columns are mutually exclusive and describe current state, not the newest run row, so a requeued issue is queued rather than failed and a closed issue is neither. Refreshes run/spend/turn values every second, and health plus the label read every ten seconds. `Enter` follows the selected transcript in place; `u` invokes the existing unblock workflow on a Blocked, Failed, or Orphaned card; `i` / `p` open the issue / PR; `r` refreshes health; `?` shows all keys. Requires an interactive terminal of at least 50×20. |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "omp-conductor",
3
- "version": "0.15.4",
3
+ "version": "0.15.6",
4
4
  "type": "module",
5
5
  "license": "MIT",
6
6
  "description": "A 24/7 dispatcher that takes ready GitHub issues to green, mergeable PRs using omp coding sessions, with tiered escalation first to an orchestrator session and then to a human.",
package/src/board.ts CHANGED
@@ -4,6 +4,7 @@ import { emitKeypressEvents } from "node:readline";
4
4
  import { findProject, loadConfig, resolveCaps } from "./config.ts";
5
5
  import { statusSnapshotFromStore, type StatusSnapshot } from "./daemon.ts";
6
6
  import {
7
+ classifyDaemonProjectHealth,
7
8
  codeGraphFromHealthz,
8
9
  fleetLayers,
9
10
  probeTelegramHealth,
@@ -926,39 +927,16 @@ export function renderBoard(
926
927
  async function probeBoardHealth(project: ProjectConfig): Promise<BoardHealthProbe> {
927
928
  const layers = fleetLayers(project.name);
928
929
  const record = livingDaemon();
930
+ // Same gate as status (#379): never health-check a record pinned to another
931
+ // project, and never invent a second membership convention beside
932
+ // classifyDaemonProjectHealth.
929
933
  const wrongRecord = record?.project !== undefined && record.project !== project.name;
930
934
  const [telegram, health] = await Promise.all([
931
935
  probeTelegramHealth(project.name),
932
936
  record === undefined || wrongRecord ? undefined : healthCheck(record.port),
933
937
  ]);
934
- let daemon: DaemonBoardState;
935
- if (record === undefined) daemon = "stopped";
936
- else if (wrongRecord) daemon = "other-project";
937
- else if (health?.ok !== true) daemon = "unreachable";
938
- else {
939
- try {
940
- const payload = JSON.parse(health.body ?? "null") as unknown;
941
- if (payload === null || typeof payload !== "object") {
942
- daemon = "unreachable";
943
- } else if (Reflect.get(payload, "project") === project.name) {
944
- daemon = "ok";
945
- } else {
946
- const projects = Reflect.get(payload, "projects");
947
- daemon =
948
- Array.isArray(projects) &&
949
- projects.some(
950
- (entry) =>
951
- entry !== null &&
952
- typeof entry === "object" &&
953
- Reflect.get(entry, "project") === project.name,
954
- )
955
- ? "ok"
956
- : "other-project";
957
- }
958
- } catch {
959
- daemon = "unreachable";
960
- }
961
- }
938
+ const classified = classifyDaemonProjectHealth(record, health, project.name);
939
+ const daemon: DaemonBoardState = classified.kind;
962
940
  const cachedGraph = daemon === "ok" ? codeGraphFromHealthz(health?.body, project.name) : undefined;
963
941
  return {
964
942
  health: {
package/src/fleet.ts CHANGED
@@ -55,8 +55,10 @@ import {
55
55
  healthCheck,
56
56
  isAlive,
57
57
  livingDaemon,
58
+ probeUnit,
58
59
  stopDaemon,
59
60
  type StopResult,
61
+ type UnitOwnership,
60
62
  SYSTEMD_UNIT,
61
63
  } from "./lifecycle.ts";
62
64
  import { formatRss, rssBytesFromHealthz } from "./host.ts";
@@ -99,7 +101,7 @@ export const ARM_CHALLENGE_TIMEOUT_MS = 300_000;
99
101
  const LEGACY_HERDR_SESSION = "fleet";
100
102
 
101
103
  export const LEGACY_HERDR_SESSION_HINT =
102
- 'herdr session "fleet" found — rename it to "conductor" or set HERDR_SESSION=fleet (remove this bridge next minor)';
104
+ 'herdr session "fleet" found — rename it to "conductor" or run omp-conductor setup host to pin HERDR_SESSION=fleet (remove this bridge next minor)';
103
105
 
104
106
  let legacyHerdrSessionHintPrinted = false;
105
107
 
@@ -1270,10 +1272,132 @@ export function formatCodeGraphHealth(graph: CodeGraphHealth, now = Date.now()):
1270
1272
  }
1271
1273
 
1272
1274
 
1275
+ /**
1276
+ * Project-scoped reading of a living daemon record + `/healthz` body.
1277
+ *
1278
+ * Board and `status` share this so a host running one daemon for several
1279
+ * projects cannot be called healthy for a project it does not serve (#379).
1280
+ * A third interpretation convention is forbidden.
1281
+ */
1282
+ export type DaemonProjectHealth =
1283
+ | { kind: "stopped" }
1284
+ | { kind: "ok" }
1285
+ | { kind: "unreachable" }
1286
+ | { kind: "other-project"; serves?: string };
1287
+
1288
+ /**
1289
+ * Facts `formatFleetStatus` needs about the living daemon. Pure input — the
1290
+ * caller probes; the formatter never shells out (#379).
1291
+ */
1292
+ export type FleetDaemonProbe = {
1293
+ project: DaemonProjectHealth;
1294
+ /** `/healthz` body when the probe is project-ok (rss / overlays). */
1295
+ body?: string;
1296
+ /** systemd ownership of the living record pid; omit when unprobed. */
1297
+ unit?: UnitOwnership;
1298
+ };
1299
+
1300
+ /**
1301
+ * Decide whether a living record + optional `/healthz` answer serve `project`.
1302
+ *
1303
+ * Mirrors the board gate: a record pinned to another project is never probed
1304
+ * further; a multi-project payload is ok only when `project` appears in
1305
+ * `projects[]` (or as the top-level single-project `project` field).
1306
+ */
1307
+ export function classifyDaemonProjectHealth(
1308
+ record: { project?: string } | undefined,
1309
+ health: { ok: boolean; body?: string } | undefined,
1310
+ project: string,
1311
+ ): DaemonProjectHealth {
1312
+ if (record === undefined) return { kind: "stopped" };
1313
+ // Record project is authoritative when set — a single-project leftover from
1314
+ // #376 must not be presented as this project's daemon just because its port
1315
+ // still answers.
1316
+ if (record.project !== undefined && record.project !== project) {
1317
+ return { kind: "other-project", serves: record.project };
1318
+ }
1319
+ if (health?.ok !== true) return { kind: "unreachable" };
1320
+ try {
1321
+ const payload = JSON.parse(health.body ?? "null") as unknown;
1322
+ if (payload === null || typeof payload !== "object") {
1323
+ return { kind: "unreachable" };
1324
+ }
1325
+ if (Reflect.get(payload, "project") === project) return { kind: "ok" };
1326
+ const projects = Reflect.get(payload, "projects");
1327
+ if (
1328
+ Array.isArray(projects) &&
1329
+ projects.some(
1330
+ (entry) =>
1331
+ entry !== null &&
1332
+ typeof entry === "object" &&
1333
+ Reflect.get(entry, "project") === project,
1334
+ )
1335
+ ) {
1336
+ return { kind: "ok" };
1337
+ }
1338
+ return { kind: "other-project", serves: servedProjectName(payload) };
1339
+ } catch {
1340
+ return { kind: "unreachable" };
1341
+ }
1342
+ }
1343
+
1344
+ /** Best-effort name of who a foreign `/healthz` payload is actually serving. */
1345
+ function servedProjectName(payload: object): string | undefined {
1346
+ const top = Reflect.get(payload, "project");
1347
+ if (typeof top === "string" && top.length > 0) return top;
1348
+ const projects = Reflect.get(payload, "projects");
1349
+ if (!Array.isArray(projects)) return undefined;
1350
+ for (const entry of projects) {
1351
+ if (entry === null || typeof entry !== "object") continue;
1352
+ const name = Reflect.get(entry, "project");
1353
+ if (typeof name === "string" && name.length > 0) return name;
1354
+ }
1355
+ return undefined;
1356
+ }
1357
+
1358
+ function formatDaemonHealthz(probe: FleetDaemonProbe | undefined): string {
1359
+ if (probe === undefined) return "unprobed";
1360
+ switch (probe.project.kind) {
1361
+ case "stopped":
1362
+ return "stopped";
1363
+ case "ok":
1364
+ return "ok";
1365
+ case "unreachable":
1366
+ return "unreachable — the process is up but not serving";
1367
+ case "other-project":
1368
+ return probe.project.serves === undefined
1369
+ ? "other-project"
1370
+ : `other-project — serves ${probe.project.serves}`;
1371
+ }
1372
+ }
1373
+
1374
+ /**
1375
+ * Unit line for a living record. The bare unit name alone used to imply
1376
+ * systemd ownership that was never checked (#379); every branch states the
1377
+ * probe result, and `unknown` is never rendered as owned.
1378
+ */
1379
+ function formatDaemonUnit(unit: UnitOwnership | undefined, recordPid: number): string {
1380
+ if (unit === undefined) return ` unit ${SYSTEMD_UNIT} unprobed`;
1381
+ switch (unit.kind) {
1382
+ case "active":
1383
+ if (unit.pid === recordPid) {
1384
+ return ` unit ${SYSTEMD_UNIT} systemd-owned`;
1385
+ }
1386
+ // Live unit, different MainPID — the record pid is not systemd's.
1387
+ return ` unit ${SYSTEMD_UNIT} unmanaged — MainPID ${unit.pid}`;
1388
+ case "failed":
1389
+ return ` unit ${SYSTEMD_UNIT} unmanaged — unit failed`;
1390
+ case "inactive":
1391
+ return ` unit ${SYSTEMD_UNIT} inactive`;
1392
+ case "unknown":
1393
+ return ` unit ${SYSTEMD_UNIT} unknown — ${unit.reason}`;
1394
+ }
1395
+ }
1396
+
1273
1397
  export function formatFleetStatus(
1274
1398
  s: StatusSnapshot,
1275
1399
  layers: FleetLayers,
1276
- daemonHealth?: { ok: boolean; body?: string },
1400
+ daemon: FleetDaemonProbe | undefined = undefined,
1277
1401
  telegram: TelegramHealth = { kind: "unprobed" },
1278
1402
  now = Date.now(),
1279
1403
  codeGraph: CodeGraphHealth = { configured: false },
@@ -1315,20 +1439,17 @@ export function formatFleetStatus(
1315
1439
  if (!layers.daemon.running || layers.daemon.pid === undefined) {
1316
1440
  daemonBlock = "daemon not running";
1317
1441
  } else {
1318
- const hz =
1319
- daemonHealth === undefined
1320
- ? "unprobed"
1321
- : daemonHealth.ok
1322
- ? "ok"
1323
- : "unreachable — the process is up but not serving";
1324
- const rss = rssBytesFromHealthz(daemonHealth?.body);
1442
+ // Only trust rss from a project-ok body — a foreign payload's bytes are
1443
+ // not this project's daemon facts (#379).
1444
+ const rss =
1445
+ daemon?.project.kind === "ok" ? rssBytesFromHealthz(daemon.body) : undefined;
1325
1446
  daemonBlock = [
1326
1447
  "daemon",
1327
1448
  ` pid ${layers.daemon.pid}`,
1328
1449
  ` port ${layers.daemon.port ?? "?"}`,
1329
1450
  ...(rss === undefined ? [] : [` rss ${formatRss(rss)}`]),
1330
- ` healthz ${hz}`,
1331
- ` unit ${SYSTEMD_UNIT}`,
1451
+ ` healthz ${formatDaemonHealthz(daemon)}`,
1452
+ formatDaemonUnit(daemon?.unit, layers.daemon.pid),
1332
1453
  ].join("\n");
1333
1454
  }
1334
1455
 
@@ -1504,8 +1625,11 @@ export async function renderStatus(projectName?: string): Promise<string> {
1504
1625
  const layers = fleetLayers(projectName);
1505
1626
  const project = findProject(loadConfig(), projectName);
1506
1627
  const rec = livingDaemon();
1507
- const [health, telegram, planUsage, github] = await Promise.all([
1508
- rec === undefined ? undefined : healthCheck(rec.port),
1628
+ // Gate the port probe on record membership first — a leftover single-project
1629
+ // daemon from #376 must not be health-checked as if it served us (#379).
1630
+ const wrongRecord = rec?.project !== undefined && rec.project !== project.name;
1631
+ const [rawHealth, telegram, planUsage, github] = await Promise.all([
1632
+ rec === undefined || wrongRecord ? undefined : healthCheck(rec.port),
1509
1633
  probeTelegramHealth(projectName),
1510
1634
  // Read here rather than in `statusSnapshot`, which is synchronous and used
1511
1635
  // by callers that must not shell out. An unmetered project never spawns
@@ -1516,13 +1640,27 @@ export async function renderStatus(projectName?: string): Promise<string> {
1516
1640
  // broken report (#188).
1517
1641
  fetchRateLimit(),
1518
1642
  ]);
1519
- const cached = codeGraphFromHealthz(health?.body, project.name);
1643
+ const projectHealth = classifyDaemonProjectHealth(rec, rawHealth, project.name);
1644
+ // Ownership is probed, never assumed from the unit name constant (#379).
1645
+ const unit = rec === undefined ? undefined : probeUnit();
1646
+ const daemon: FleetDaemonProbe | undefined =
1647
+ rec === undefined
1648
+ ? undefined
1649
+ : {
1650
+ project: projectHealth,
1651
+ ...(projectHealth.kind === "ok" && rawHealth?.body !== undefined
1652
+ ? { body: rawHealth.body }
1653
+ : {}),
1654
+ ...(unit === undefined ? {} : { unit }),
1655
+ };
1656
+ const healthBody = projectHealth.kind === "ok" ? rawHealth?.body : undefined;
1657
+ const cached = codeGraphFromHealthz(healthBody, project.name);
1520
1658
  const codeGraph = cached ?? (await probeCodeGraph(project));
1521
- const workerPhases = workerPhasesFromHealthz(health?.body, project.name);
1659
+ const workerPhases = workerPhasesFromHealthz(healthBody, project.name);
1522
1660
  return formatFleetStatus(
1523
1661
  { ...s, planUsage, github },
1524
1662
  layers,
1525
- health,
1663
+ daemon,
1526
1664
  telegram,
1527
1665
  Date.now(),
1528
1666
  codeGraph,
package/src/setup-host.ts CHANGED
@@ -4,7 +4,7 @@ import { homedir, userInfo } from "node:os";
4
4
  import { dirname, join } from "node:path";
5
5
  import { configPath, loadConfig, resolveCaps, stateDir } from "./config.ts";
6
6
  import { isPaused, runDaemon, statusSnapshot, type StatusSnapshot } from "./daemon.ts";
7
- import { DEFAULT_FLEET_AGENT_NAME } from "./fleet.ts";
7
+ import { DEFAULT_FLEET_AGENT_NAME, DEFAULT_HERDR_SESSION, resolveHerdrSessionWithBridge } from "./fleet.ts";
8
8
  import {
9
9
  DEFAULT_PORT,
10
10
  healthCheck,
@@ -65,6 +65,12 @@ export interface ServiceRuntime {
65
65
  packageCli: string;
66
66
  conductorHome: string;
67
67
  telegramStateDir: string;
68
+ /**
69
+ * Non-default Herdr session to pin into the unit (#394). Omitted when the
70
+ * host uses {@link DEFAULT_HERDR_SESSION}: an always-present line would show
71
+ * as drift on every existing host and add a redundant default.
72
+ */
73
+ herdrSession?: string;
68
74
  }
69
75
 
70
76
  function systemdQuote(value: string): string {
@@ -237,7 +243,12 @@ function refusal(
237
243
  }
238
244
 
239
245
 
240
- function defaultServiceRuntime(telegramStateDir: string): ServiceRuntime {
246
+ function defaultServiceRuntime(
247
+ telegramStateDir: string,
248
+ // Bridge-resolved session name. Passed in so tests stay hermetic (no herdr
249
+ // spawn) and the renderer never shells out — same threading as telegramStateDir.
250
+ herdrSession: string = resolveHerdrSessionWithBridge(),
251
+ ): ServiceRuntime {
241
252
  const home = homedir();
242
253
  const bun = process.execPath;
243
254
  const globalCli = Bun.which("omp-conductor");
@@ -253,6 +264,9 @@ function defaultServiceRuntime(telegramStateDir: string): ServiceRuntime {
253
264
  packageCli: join(import.meta.dir, "cli.ts"),
254
265
  conductorHome: dirname(configPath()),
255
266
  telegramStateDir,
267
+ // Only pin a non-default session. Default hosts keep a unit with no
268
+ // HERDR_SESSION line so setup/upgrade do not invent drift (#394).
269
+ ...(herdrSession !== DEFAULT_HERDR_SESSION ? { herdrSession } : {}),
256
270
  };
257
271
  }
258
272
 
@@ -283,6 +297,11 @@ export function renderDaemonService(runtime: ServiceRuntime, totalWorkers: numbe
283
297
  `Environment=${systemdQuote(`PATH=${runtime.path}`)}`,
284
298
  `Environment=${systemdQuote(`OMP_CONDUCTOR_HOME=${runtime.conductorHome}`)}`,
285
299
  `Environment=${systemdQuote(`OMP_TELEGRAM_STATE_DIR=${runtime.telegramStateDir}`)}`,
300
+ // Escape hatch for hosts still on the pre-#320 "fleet" session (#394). Only
301
+ // when set: an always-on line would drift every default host on every setup.
302
+ ...(runtime.herdrSession === undefined
303
+ ? []
304
+ : [`Environment=${systemdQuote(`HERDR_SESSION=${runtime.herdrSession}`)}`]),
286
305
  `WorkingDirectory=${systemdPath(stateDir())}`,
287
306
  `ExecStart=${command.map(systemdQuote).join(" ")}`,
288
307
  "Restart=on-failure",
package/src/upgrade.ts CHANGED
@@ -52,6 +52,13 @@ export interface UpgradeDeps {
52
52
  snapshot(project?: string): { liveWorkers: number };
53
53
  layers(project?: string): FleetLayers;
54
54
  brief(project?: string): { kind: BriefLayout["kind"]; current: boolean };
55
+ /**
56
+ * Every configured project name, in config order. One daemon serves all of
57
+ * them (the generated unit's ExecStart carries no `--project`), so a
58
+ * transaction that restarts that daemon has to know the whole set rather
59
+ * than the one project a caller happened to name (#389).
60
+ */
61
+ projectNames(): readonly string[];
55
62
  /**
56
63
  * The daemon the transaction targets, when one is running. `generation`
57
64
  * identifies the exact instance (pid + start time) so a later restart cannot
@@ -103,6 +110,7 @@ export const DEFAULT_DEPS: UpgradeDeps = {
103
110
  const live = readFileSync(layout.orchestratorPath, "utf8");
104
111
  return { kind: layout.kind, current: live === renderBriefForProject(project, policy) };
105
112
  },
113
+ projectNames: () => loadConfig().projects.map((p) => p.name),
106
114
  daemonIdentity: () => {
107
115
  const daemon = livingDaemon();
108
116
  if (daemon === undefined) return { running: false };
@@ -268,9 +276,95 @@ function previousHerdrInstall(source: string): readonly [string, readonly string
268
276
  ];
269
277
  }
270
278
 
279
+ /**
280
+ * The projects one lifecycle transaction covers, and the single pause sentinel
281
+ * that represents them.
282
+ *
283
+ * Everything an upgrade or a draining restart replaces is host-wide: the
284
+ * Bun-global CLI, the omp plugin, the Herdr plugin, and the one dispatch
285
+ * daemon. That daemon serves *every* configured project — the generated unit's
286
+ * ExecStart carries no `--project` — so its restart lands on all of them at
287
+ * once. The scope therefore names every project whose workers must drain and
288
+ * whose brief must be refreshed, instead of the single project a caller
289
+ * happened to name (#389).
290
+ *
291
+ * `selectors` are what the rest of the module passes back to the project-aware
292
+ * deps. A selector of `undefined` means "the one configured project", which is
293
+ * how a single-project host has always addressed itself: keeping it verbatim
294
+ * means the single-project path still emits exactly the same `brief-upgrade`
295
+ * command and the same one `snapshot` call it did before.
296
+ *
297
+ * `pauseKey` is the sentinel this transaction owns. It is `undefined` for a
298
+ * host-wide scope, which selects the legacy *global* sentinel — and that is
299
+ * precisely the right instrument: `isPaused(name)` consults the global file
300
+ * for every project, so one write pauses the whole host, and removing it lifts
301
+ * exactly this transaction's pause while leaving any per-project operator hold
302
+ * standing.
303
+ */
304
+ interface UpgradeScope {
305
+ /** Project selectors to address the project-aware deps with, never empty. */
306
+ selectors: readonly (string | undefined)[];
307
+ /** The sentinel this transaction pauses and resumes. */
308
+ pauseKey: string | undefined;
309
+ }
310
+
311
+ /**
312
+ * Decides which projects a transaction covers, and refuses the requests that
313
+ * cannot be honoured truthfully.
314
+ *
315
+ * Two refusals, both about a caller naming a project the daemon does not agree
316
+ * with:
317
+ *
318
+ * - the daemon records a *different* project — it serves that one, so a
319
+ * request aimed at another is simply about a different fleet;
320
+ * - the daemon records *no* project while several are configured. That is not
321
+ * "unknown", it is the multi-project daemon: it serves them all. Narrowing
322
+ * is refused rather than obeyed, because draining one project and then
323
+ * restarting the shared daemon would kill the other projects' live workers
324
+ * without ever counting them (#389, and the shared-daemon rule in #378).
325
+ *
326
+ * A single-configured-project host keeps the historical bare selector, so its
327
+ * behaviour is unchanged whether or not the daemon recorded its name.
328
+ */
329
+ function resolveScope(deps: UpgradeDeps, verb: "upgrade" | "restart", project?: string): UpgradeScope {
330
+ let configured: readonly string[] = [];
331
+ try {
332
+ configured = deps.projectNames();
333
+ } catch {
334
+ // No readable config is not this function's error to raise: the caller
335
+ // reaches a project-aware dep moments later and fails with the real
336
+ // problem. Treat it as the historical bare case.
337
+ }
338
+ const daemon = deps.daemonIdentity();
339
+
340
+ if (project !== undefined) {
341
+ if (daemon.running && daemon.project !== undefined && daemon.project !== project) {
342
+ throw new Error(
343
+ `active daemon serves ${daemon.project}; refusing to ${verb} --project ${project} while it is running`,
344
+ );
345
+ }
346
+ if (daemon.running && daemon.project === undefined && configured.length > 1) {
347
+ throw new Error(
348
+ `the running daemon serves all ${String(configured.length)} configured projects ` +
349
+ `(${configured.join(", ")}); ${verb} replaces host-wide packages and restarts that one daemon, ` +
350
+ `so it cannot be narrowed to --project ${project} — re-run \`omp-conductor ${verb}\` without --project`,
351
+ );
352
+ }
353
+ return { selectors: [project], pauseKey: project };
354
+ }
355
+
356
+ // A daemon that recorded its project serves that project alone; the rest of
357
+ // the config is somebody else's fleet as far as this restart is concerned.
358
+ if (daemon.running && daemon.project !== undefined) {
359
+ return { selectors: [daemon.project], pauseKey: daemon.project };
360
+ }
361
+ if (configured.length > 1) return { selectors: configured, pauseKey: undefined };
362
+ return { selectors: [undefined], pauseKey: undefined };
363
+ }
364
+
271
365
  async function waitForDrain(
272
366
  deps: UpgradeDeps,
273
- project?: string,
367
+ scope: UpgradeScope,
274
368
  deadlineAt?: number,
275
369
  stale?: () => string | undefined,
276
370
  ): Promise<void> {
@@ -282,7 +376,10 @@ async function waitForDrain(
282
376
  // and nothing it may stop (#377).
283
377
  const cancelled = stale?.();
284
378
  if (cancelled !== undefined) throw new Error(cancelled);
285
- const workers = deps.snapshot(project).liveWorkers;
379
+ // Every project the scope covers: a host-wide drain that counted only one
380
+ // would report zero while another project's worker was still writing to a
381
+ // worktree the restart is about to orphan (#389).
382
+ const workers = scope.selectors.reduce((n, s) => n + deps.snapshot(s).liveWorkers, 0);
286
383
  if (workers === 0) return;
287
384
  if (deadlineAt !== undefined && Date.now() >= deadlineAt) {
288
385
  throw new Error(
@@ -321,13 +418,13 @@ interface RestartBegun {
321
418
  */
322
419
  function restartFenceProblem(
323
420
  deps: UpgradeDeps,
324
- project: string | undefined,
421
+ scope: UpgradeScope,
325
422
  begun: RestartBegun,
326
423
  ): string | undefined {
327
- if (!deps.layers(project).paused) {
424
+ if (!deps.layers(scope.pauseKey).paused) {
328
425
  return "restart cancelled: dispatch was resumed while the drain was in progress — nothing was restarted";
329
426
  }
330
- const owned = deps.pauseState(project);
427
+ const owned = deps.pauseState(scope.pauseKey);
331
428
  if (
332
429
  owned === undefined ||
333
430
  owned.since !== begun.pauseToken.since ||
@@ -370,13 +467,17 @@ export async function drainAndRestart(
370
467
  deps: UpgradeDeps,
371
468
  o: { project?: string; timeoutMs: number },
372
469
  ): Promise<void> {
373
- const initial = deps.layers(o.project);
374
- if (!initial.paused) deps.setPaused(true, o.project);
470
+ // Host-wide by default (#389): the daemon this restarts serves every
471
+ // configured project, so the drain counts every project's workers and the
472
+ // pause covers all of them.
473
+ const scope = resolveScope(deps, "restart", o.project);
474
+ const initial = deps.layers(scope.pauseKey);
475
+ if (!initial.paused) deps.setPaused(true, scope.pauseKey);
375
476
  // The lock must be provable NOW. A pause that cannot be read as an instance
376
477
  // — sentinel malformed or unreadable — would make the whole fence fail open
377
478
  // (nothing to compare against), so the transaction refuses before waiting
378
479
  // on anything it cannot act upon either way.
379
- const pauseToken = deps.pauseState(o.project);
480
+ const pauseToken = deps.pauseState(scope.pauseKey);
380
481
  if (pauseToken === undefined) {
381
482
  throw new Error(
382
483
  "restart cancelled: cannot prove the restart-owned pause — the active pause sentinel is unreadable or malformed; nothing was restarted",
@@ -386,14 +487,14 @@ export async function drainAndRestart(
386
487
  pauseToken,
387
488
  daemon: deps.daemonIdentity(),
388
489
  };
389
- const stale = () => restartFenceProblem(deps, o.project, begun);
390
- await waitForDrain(deps, o.project, Date.now() + o.timeoutMs, stale);
490
+ const stale = () => restartFenceProblem(deps, scope, begun);
491
+ await waitForDrain(deps, scope, Date.now() + o.timeoutMs, stale);
391
492
  // The drain completed; the world may have moved on while it did. Re-prove
392
493
  // the pause and the generation an instant before the destructive call.
393
494
  const cancelled = stale();
394
495
  if (cancelled !== undefined) throw new Error(cancelled);
395
496
  await deps.restartDaemon();
396
- if (!initial.paused) deps.setPaused(false, o.project);
497
+ if (!initial.paused) deps.setPaused(false, scope.pauseKey);
397
498
  }
398
499
 
399
500
  function recoveryProblem(
@@ -419,19 +520,32 @@ function recoveryProblem(
419
520
  async function waitForRecovery(
420
521
  deps: UpgradeDeps,
421
522
  initial: FleetLayers,
422
- project?: string,
523
+ scope: UpgradeScope,
423
524
  ): Promise<void> {
424
525
  let problem = "recovery did not settle";
425
526
  for (let attempt = 0; attempt < RECOVERY_ATTEMPTS; attempt += 1) {
426
- problem = recoveryProblem(deps.layers(project), initial, deps.snapshot(project).liveWorkers) ?? "";
527
+ const live = scope.selectors.reduce((n, s) => n + deps.snapshot(s).liveWorkers, 0);
528
+ problem = recoveryProblem(deps.layers(scope.pauseKey), initial, live) ?? "";
427
529
  if (problem === "") return;
428
530
  await deps.sleep(RECOVERY_POLL_MS);
429
531
  }
430
532
  throw new Error(`upgrade verification failed: ${problem}`);
431
533
  }
432
534
 
535
+ /** One project's brief, as the transaction found it. */
536
+ interface ScopedBrief {
537
+ /** The selector to address the project-aware deps and the CLI with. */
538
+ selector: string | undefined;
539
+ kind: BriefLayout["kind"];
540
+ current: boolean;
541
+ }
542
+
433
543
  /**
434
- * Brings the brief up with the freshly installed package.
544
+ * Brings every in-scope brief up with the freshly installed package.
545
+ *
546
+ * Each project owns its own ORCHESTRATOR.md, so a host-wide upgrade migrates
547
+ * them one at a time rather than leaving every project but one on a floor the
548
+ * installed package no longer emits (#389).
435
549
  *
436
550
  * `brief-upgrade --migrate --apply` is the cross-version ABI: this code runs
437
551
  * from the *old* CLI while the new one is already installed, so the verb has to
@@ -445,29 +559,27 @@ async function waitForRecovery(
445
559
  * fleet on a single-file brief whose floor no longer tracks the package —
446
560
  * the exact drift this verb exists to end — so it fails loudly and rolls back.
447
561
  */
448
- async function upgradeBrief(
449
- deps: UpgradeDeps,
450
- kind: BriefLayout["kind"],
451
- project?: string,
452
- ): Promise<void> {
453
- const selected = project === undefined ? [] : ["--project", project];
454
- if (kind === "legacy-handwritten") {
455
- // Ungated on purpose: a hand-written brief cannot be migrated without it.
456
- await mustRun(deps, "omp-conductor", ["brief-upgrade", "--retrofit", "--apply", ...selected]);
457
- }
458
- try {
459
- await mustRun(deps, "omp-conductor", ["brief-upgrade", "--migrate", "--apply", ...selected]);
460
- } catch (err) {
461
- if (kind !== "overlay") throw err;
462
- const msg = err instanceof Error ? err.message : String(err);
463
- deps.log(`brief: brief-upgrade unavailable in the target CLI (${msg}) — overlay recomposes each tick, continuing`);
562
+ async function upgradeBriefs(deps: UpgradeDeps, briefs: readonly ScopedBrief[]): Promise<void> {
563
+ for (const brief of briefs) {
564
+ const selected = brief.selector === undefined ? [] : ["--project", brief.selector];
565
+ if (brief.kind === "legacy-handwritten") {
566
+ // Ungated on purpose: a hand-written brief cannot be migrated without it.
567
+ await mustRun(deps, "omp-conductor", ["brief-upgrade", "--retrofit", "--apply", ...selected]);
568
+ }
569
+ try {
570
+ await mustRun(deps, "omp-conductor", ["brief-upgrade", "--migrate", "--apply", ...selected]);
571
+ } catch (err) {
572
+ if (brief.kind !== "overlay") throw err;
573
+ const msg = err instanceof Error ? err.message : String(err);
574
+ deps.log(`brief: brief-upgrade unavailable in the target CLI (${msg}) — overlay recomposes each tick, continuing`);
575
+ }
464
576
  }
465
577
  }
466
578
 
467
579
  async function rollbackUpgrade(
468
580
  deps: UpgradeDeps,
469
581
  previous: InstalledSurfaces,
470
- project: string | undefined,
582
+ scope: UpgradeScope,
471
583
  installTouched: boolean,
472
584
  briefChanged: boolean,
473
585
  herdrReloadStarted: boolean,
@@ -577,7 +689,10 @@ async function rollbackUpgrade(
577
689
  if (briefChanged) {
578
690
  try {
579
691
  deps.log("rollback: ORCHESTRATOR.md package floor");
580
- await upgradeBrief(deps, "overlay", project);
692
+ await upgradeBriefs(
693
+ deps,
694
+ scope.selectors.map((selector) => ({ selector, kind: "overlay" as const, current: false })),
695
+ );
581
696
  } catch (err) {
582
697
  failures.push(err instanceof Error ? err.message : String(err));
583
698
  }
@@ -626,23 +741,21 @@ export async function upgradeConductor(
626
741
  }
627
742
 
628
743
  const requested = options.version === undefined ? `${PACKAGE}@latest` : `${PACKAGE}@${options.version}`;
629
- const daemon = deps.daemonIdentity();
630
- if (daemon.running && options.project !== undefined && daemon.project !== options.project) {
631
- const active = daemon.project === undefined ? "an unrecorded project" : daemon.project;
632
- throw new Error(
633
- `active daemon serves ${active}; refusing to upgrade --project ${options.project} while it is running`,
634
- );
635
- }
636
- const project = options.project ?? daemon.project;
744
+ // Host-wide by default (#389): one daemon serves every configured project,
745
+ // so an upgrade that restarts it drains them all and refreshes every brief.
746
+ const scope = resolveScope(deps, "upgrade", options.project);
637
747
 
638
748
  const release = parseRegistry(
639
749
  (await mustRun(deps, "npm", ["view", requested, "version", "gitHead", "--json"])).stdout,
640
750
  );
641
- const initial = deps.layers(project);
751
+ const initial = deps.layers(scope.pauseKey);
642
752
  const surfaces = await inspectSurfaces(deps);
643
- const brief = deps.brief(project);
753
+ const briefs: ScopedBrief[] = scope.selectors.map((selector) => ({
754
+ selector,
755
+ ...deps.brief(selector),
756
+ }));
644
757
  const installNeeded = !surfacesCurrent(surfaces, release.version, release.gitHead);
645
- if (!installNeeded && brief.current) {
758
+ if (!installNeeded && briefs.every((b) => b.current)) {
646
759
  return {
647
760
  previousVersion: surfaces.cliVersion,
648
761
  ...release,
@@ -673,17 +786,24 @@ export async function upgradeConductor(
673
786
  // either — the in-memory restore below still covers a clean rollback.
674
787
  }
675
788
 
676
- if (brief.kind === "missing") throw new Error("no ORCHESTRATOR.md exists for the configured project");
789
+ const missing = briefs.find((b) => b.kind === "missing");
790
+ if (missing !== undefined) {
791
+ throw new Error(
792
+ missing.selector === undefined
793
+ ? "no ORCHESTRATOR.md exists for the configured project"
794
+ : `no ORCHESTRATOR.md exists for project ${missing.selector}`,
795
+ );
796
+ }
677
797
  if (initial.herdr === "unknown") throw new Error("cannot determine whether herdr-fleet.service is active");
678
798
 
679
799
  deps.log(`target release: omp-conductor@${release.version} (${release.gitHead})`);
680
800
  if (!initial.paused) {
681
801
  deps.log("safety: pausing new issue claims");
682
- deps.setPaused(true, project);
802
+ deps.setPaused(true, scope.pauseKey);
683
803
  }
684
804
  deps.log("drain: waiting for live omp worker sessions");
685
805
  try {
686
- await waitForDrain(deps, project);
806
+ await waitForDrain(deps, scope);
687
807
  } catch (err) {
688
808
  const failure = err instanceof Error ? err.message : String(err);
689
809
  throw new Error(
@@ -721,7 +841,7 @@ export async function upgradeConductor(
721
841
 
722
842
  deps.log("brief: refreshing managed ORCHESTRATOR.md floor while preserving POLICY.md");
723
843
  briefChanged = true;
724
- await upgradeBrief(deps, brief.kind, project);
844
+ await upgradeBriefs(deps, briefs);
725
845
 
726
846
  if (initial.herdr === "active") {
727
847
  herdrReloadStarted = true;
@@ -746,13 +866,13 @@ export async function upgradeConductor(
746
866
  const active = await mustRun(deps, "systemctl", ["is-active", HERDR_UNIT]);
747
867
  if (active.stdout.trim() !== "active") throw new Error(`${HERDR_UNIT} is not active after restart`);
748
868
  }
749
- await waitForRecovery(deps, initial, project);
869
+ await waitForRecovery(deps, initial, scope);
750
870
  deps.log("verify 2/2: recovered fleet remains stable");
751
871
  await deps.sleep(1_000);
752
- await waitForRecovery(deps, initial, project);
872
+ await waitForRecovery(deps, initial, scope);
753
873
 
754
- if (!initial.paused) deps.setPaused(false, project);
755
- const restoredDispatch = deps.layers(project).dispatch;
874
+ if (!initial.paused) deps.setPaused(false, scope.pauseKey);
875
+ const restoredDispatch = deps.layers(scope.pauseKey).dispatch;
756
876
  if (restoredDispatch !== initial.dispatch) {
757
877
  throw new Error(
758
878
  `dispatch state is ${restoredDispatch}; expected to restore ${initial.dispatch}`,
@@ -760,9 +880,9 @@ export async function upgradeConductor(
760
880
  }
761
881
  } catch (err) {
762
882
  try {
763
- if (!deps.layers(project).paused) deps.setPaused(true, project);
883
+ if (!deps.layers(scope.pauseKey).paused) deps.setPaused(true, scope.pauseKey);
764
884
  } catch {
765
- deps.setPaused(true, project);
885
+ deps.setPaused(true, scope.pauseKey);
766
886
  }
767
887
  const failure = err instanceof Error ? err.message : String(err);
768
888
  deps.log(`upgrade failed: ${failure}`);
@@ -770,7 +890,7 @@ export async function upgradeConductor(
770
890
  await rollbackUpgrade(
771
891
  deps,
772
892
  surfaces,
773
- project,
893
+ scope,
774
894
  installTouched,
775
895
  briefChanged,
776
896
  herdrReloadStarted,