johns-harness 2026.9.28 → 2026.9.29

package/CHANGELOG.md

@@ -1,5 +1,9 @@
 # Changelog
 
+## 2026.9.29
+
+- Family 76.1: the registration-safe reload worker waits for `launchctl bootout` teardown to finish before bootstrapping the staged plist. A gateway that shuts down gracefully on SIGTERM stays registered for a moment after `bootout`; 2026.9.28 mistook that lingering registration for the replacement, skipped `bootstrap`, and reported `Service removed while waiting for health` with the job left unloaded (observed on the first live fleet reload). The wait is bounded at 30 seconds and a timeout is a retryable worker error, not an advanced checkpoint. Ordinary `kickstart -k` restarts were unaffected. The real macOS self-restart fixture now exits gracefully so the reload cases reproduce the race; two unit cases cover the wait and the timeout.
+
 ## 2026.9.28
 
 - Family 76: `johnness gateway restart` from inside a gateway's own launchd process group no longer unregisters its supervisor. The old `bootout -> wait -> bootstrap` sequence killed the caller before it could bootstrap, which caused a multi-hour outage when a cron issued the restart. All three service implementations, both internal restart bundles and both updater bundles now share `dist/launchd-lifecycle.js`: a loaded service restarts with `kickstart -k` only (no bootout, unload or bootstrap fallback), a missing service may bootstrap its explicitly selected plist, and real plist replacement (`gateway install --force`) stages validated bytes and hands off to an independently registered, durably armed completer with an fsynced phase journal, retry under launchd, and prior-plist restore after five failed bootstraps. Explicit stop/uninstall cancels any pending completer. Updaters snapshot the helper before npm replaces the installed tree. Success requires a changed launchd-owned PID and, when a port is declared, `/healthz` plus listener ownership by that PID; receipts live in `<stateDir>/lifecycle/<uuid>/`. Ten browser timeout messages now suggest a non-browser alternative instead of a whole-gateway restart. Covered by lifecycle unit, fault-injection, replay and packed-artifact tests plus five real disposable LaunchAgent self-restart/reload cases (`JOHNNESS_TEST_LAUNCHD=1`); verifier check 76.1.

package/README.md

@@ -85,7 +85,7 @@ node johnness.mjs --help
 ### Verify
 
 ```bash
-npm test                        # 990 tests against the installed dependency tree
+npm test                        # 992 tests against the installed dependency tree
 bash scripts/verify-patches.sh  # 468 checks across every patch family
 ```
 
@@ -199,7 +199,7 @@ The log is a [`RULES.md`](RULES.md) workspace convention enforced through the ac
 
 ## Patch families
 
-The runtime carries 76 patch families plus the 65.1 safe-update amendment. Each is a production fix applied at the source with a regression test, a durable patch marker, and an explicit recovery path. A verifier runs 468 checks against the package tree and installed dependencies, and 990 tests run against the installed dependency tree.
+The runtime carries 76 patch families plus the 65.1 safe-update amendment. Each is a production fix applied at the source with a regression test, a durable patch marker, and an explicit recovery path. A verifier runs 468 checks against the package tree and installed dependencies, and 992 tests run against the installed dependency tree.
 
 Every family follows the same six steps: reproduce the failure, trace the exact runtime path, make the smallest source-level change that restores the invariant, add a regression test and a patch marker, run the verifier, and retain rollback artifacts. The full index: [`docs/PATCHES.md`](docs/PATCHES.md).
 

package/dist/launchd-lifecycle.js

@@ -169,6 +169,17 @@ async function healthy(p, current, deps) {
   if (!pids.length || !pids.every((pid) => descendant(pid, current.pid))) return false;
   try { return (await fetch(`http://127.0.0.1:${p.port}/healthz`, { signal: AbortSignal.timeout(2000) })).ok; } catch { return false; }
 }
+/** After bootout, launchd finishes SIGTERM teardown asynchronously; only the previous pid can
+ * still be registered here. A different registered pid means the plist was already re-bootstrapped. */
+async function unloaded(run, p, deps) {
+  const deadline = Date.now() + (deps.unloadTimeoutMs ?? 30000);
+  for (;;) {
+    const current = runtime(run, p.target);
+    if (!current.loaded || (current.pid && p.previousPid && current.pid !== p.previousPid)) return current;
+    if (Date.now() >= deadline) throw new Error("Previous process still registered after bootout; independent worker will retry");
+    await (deps.pause || pause)(200);
+  }
+}
 /** Restartable state machine. Checkpoints precede effects; real supervisor state
  * reconciles a missing effect acknowledgement after SIGKILL. Exported for fault tests. */
 export async function executeLaunchdOperation(p, receiptFile, deps = {}) {
@@ -195,6 +206,11 @@ export async function executeLaunchdOperation(p, receiptFile, deps = {}) {
       const r = effect(["bootout", p.target]);
       if (r.code && !missing(r)) throw new Error(`launchctl bootout failed (${r.code})`);
     }
+    // bootout is asynchronous for a job that handles SIGTERM: launchd keeps it registered (old pid)
+    // until graceful shutdown completes. Wait for the teardown before recording the intent to
+    // bootstrap, or the still-registered job would be mistaken for the replacement and later
+    // vanish, producing "Service removed while waiting for health" with no gateway loaded.
+    await unloaded(run, p, deps);
     save("bootstrap-intent"); receipt = read(receiptFile);
     current = runtime(run, p.target);
   }
@@ -220,6 +236,8 @@ export async function executeLaunchdOperation(p, receiptFile, deps = {}) {
     save("health-pending", { rolledBack: true, healthStartedAt: Date.now() }); receipt = read(receiptFile);
   }
   if (receipt.status === "bootstrap-intent") {
+    // Resume after a crash between bootout and this checkpoint: the old job may still be tearing down.
+    if (p.mode === "reload") await unloaded(run, p, deps);
     current = runtime(run, p.target);
     if (!current.loaded) {
       const r = effect(["bootstrap", p.domain, p.plistPath]);

package/docs/PATCHES.md

@@ -2741,6 +2741,18 @@ self-restart can disappear; the independent receipt is the source of truth.
 Ten browser error copies now report a browser blocker and suggest a non-browser
 alternative, never a whole-gateway restart. Exec process-group behavior is unchanged.
 
+**76.1 (2026.9.29), graceful unload before bootstrap.** The first live reload of a real
+gateway (Trisha, `gateway install --force` semantics) failed with `Service removed while
+waiting for health` and left the job unloaded. `launchctl bootout` returns before a job
+that handles SIGTERM has exited; the worker's next `print` still saw the old registered
+PID, treated the job as loaded, skipped `bootstrap`, and then watched launchd finish the
+teardown. The worker now waits after `bootout` (and again on resume at `bootstrap-intent`)
+until the target is unregistered or a different PID is registered, with a 30 s bound that
+surfaces as a retryable worker error instead of an advanced checkpoint. The real macOS
+fixture now performs a 700 ms graceful SIGTERM shutdown so the reload cases exercise the
+race; two unit cases cover the wait and the timeout. `kickstart -k` restarts were never
+affected.
+
 **Verification:** `tests/launchd-safe-restart.test.mjs` covers lifecycle phases,
 fault injection, error classification, scoping, all copies and exact patch replay.
 `JOHNNESS_TEST_LAUNCHD=1 node --test --test-concurrency=1 tests/launchd-self-restart.macos.test.mjs`

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "johns-harness",
-  "version": "2026.9.28",
+  "version": "2026.9.29",
   "description": "John's Harness: a production agent harness that runs autonomous coding agents as one cooperative swarm.",
   "keywords": [
     "ai-agents",