@botiverse/k-carrier 0.1.7 → 0.2.0

package/NOTICE

@@ -5,10 +5,12 @@ This product is licensed under the Apache License, Version 2.0 (see LICENSE).
 
 Design influences — concepts studied, no code copied
 -----------------------------------------------------
-k-carrier's design was informed by studying two mature, unrelated updaters.
+k-carrier's design was informed by studying other installers and updaters.
 Only architectural concepts were borrowed; no source code was copied, and the
 implementation here is original.
 
+- External installer boundary and self-update execution model are informed by Rustup's rustup-init and self-update flow (rust-lang/rustup, client installer/self-update). Concept only; no source code copied.
+
 - Two-slot upgrade transaction (stable/experiment slots with promote/rollback)
   and the "the installer is itself a managed package" fleet model are inspired
   by Datadog's fleet installer (DataDog/datadog-agent, pkg/fleet). Concept only.
@@ -19,6 +21,7 @@ implementation here is original.
   signature verification — it verifies artifact integrity via sha256 + size
   only. No distsign / signing code is present in this repository.
 
-See docs/prior-art.md for the full analysis, including what each project does
+See docs/prior-art/design-influences.md and
+docs/prior-art/external-runner-research.md for the full analysis, including what each project does
 and where k-carrier deliberately differs (fail-closed restart, post-upgrade
 convergence read-back, and rollback).

package/README.md

@@ -1,38 +1,84 @@
 # K (k-carrier)
 
-**Self-upgrade framework for programs that must prove they came back up.**
+**Reliable application upgrades, run by an independent installer.**
 
-CLI self-update libraries stop at replacing bytes; fleet updaters assume a machine someone else administers. K covers what neither does: **an upgrade that is a transaction and can prove it happened** — two slots with rollback, crash-safe at every step, handoff of a live process with its workloads intact, and convergence proven from the live process and named OS surfaces (a version string is never accepted as proof). Consent and notification are built in, because on a machine someone owns personally, changing behaviour silently is not acceptable — but nothing here is limited to personal machines.
+K is for applications you distribute yourself to end users' machines, across
+operating systems, outside any package manager: desktop agents, background
+services installed by `curl | sh`, CLIs that update themselves. Nobody
+operates those machines. A failed upgrade means the product silently stops
+working and nobody can log in to repair it. K combines a rustup-style external
+installer with recoverable, verified service upgrades.
 
-**Two process models, defined by how many live incarnations K manages** — `swap` (**0**: K replaces bytes and touches no process; a one-shot CLI and an hours-long agent session are the same case) and `service` (**1**: K stops the old, starts the new, and proves it). OS lifecycle convergence and fleet drive are capabilities you opt into on top, not a third model. Proof is executable: a runnable example per case, and a claim without a green example does not exist.
+If a package manager, container image or fleet orchestrator already owns your
+installation, that manager owns upgrades too. Your adapter identifies that
+ownership so K can defer; you probably do not need it.
 
-## What K owns
+## Why upgrading is non-trivial
 
-K does not replace platform packaging or artifact delivery. It wraps an
-addressable release in a transaction with rollback and convergence readback.
-Because the process driving an upgrade may die on the success path, the
-successor proves the handoff from live evidence rather than trusting a flag.
+Downloading a new executable is only the beginning. The application may still
+be running, power may fail halfway through replacement, or the new version may
+fail to start on a particular machine. Replacing the file does not prove the
+service came back healthy. The installed updater may itself be too old or
+broken to help.
 
-## Start here
+K stages verified bytes in a second slot, stops the service, starts and probes
+the candidate, then commits or restores the previous executable. A durable
+journal lets a subsequent installer recover interrupted work. An upgrade ends
+promoted, rolled back, or explicitly unresolved with the evidence preserved;
+it is never reported as a success K did not observe.
 
-- **[`docs/integration.md`](docs/integration.md)** — from-zero guide: the problem in plain words, concept primer, tiered adoption with code.
-- [`docs/design-v1.md`](docs/design-v1.md) — full design: six layers, architecture, decision record.
-- [`docs/harness-design.md`](docs/harness-design.md) — the test framework, designed first: harness as executable spec (teeth registry, real-process crash injection, adversarial self-verification).
-- [`docs/test-plan.md`](docs/test-plan.md) — executable test plan (M0–M6, must-red per cell).
-- [`docs/prior-art.md`](docs/prior-art.md) — the source-level survey this design stands on (Tailscale / Datadog), and the license-defense record behind `NOTICE`.
+## One installer, multiple entrypoints
 
-## Repo layout
+`install.sh` and your application's `self upgrade` launch the same external
+installer, built from K and your product adapter. The installer owns the
+upgrade transaction and exits when finished. It can be updated independently
+of the application and run even when the installed application cannot start.
 
-```
-core/       the framework — zero host-specific concepts (shells live in their
-            product's repo and consume core as a dependency)
-harness/    generic acceptance bed: fake-host daemon + profile-tiered teeth
-examples/   one runnable demo per profile (swap-tool / service-daemon / hosted-service)
-docs/       guides + design + test plan + prior art
-```
+You distribute **three things**: the bootstrap script, the installer, and the
+application release. They can share a hosting location. You supply the
+release source and service lifecycle operations; K supplies the transaction
+machinery. The runner uses Node 24, either as an external runtime or bundled
+into a Node single executable. A fully runtime-independent installer must also
+package its supervisor and controller dependencies.
 
-**Platform support today:** Linux and macOS gate CI. Windows platform
-operations are implemented; its acceptance harness and CI gate are still in
-progress.
+The installer must survive stopping the application. If a worker crashes,
+the temporary supervisor runs bounded recovery. After reboot, an operator or
+OS startup hook must start installation again. K rolls back executables,
+not application data; data migration compatibility remains the product's job.
+Artifact hashes check integrity; your distribution channel establishes trust.
 
-Status: incubating. TypeScript first. License: **Apache-2.0**.
+## Status
+
+| Area | State |
+|---|---|
+| Two-slot transaction, journal, lock, receipts | Done; generated crash matrix, seeded simulation, and a Lean model of all phases, rollback and crash/recovery interleavings (host honesty assumed) |
+| External runner protocol, supervisor, bounded recovery | Done; Linux/macOS process tests |
+| Command controller boundary | Done; demo controller only |
+| Verified download with resume and gzip | Done |
+| Single-executable (SEA) runner | Verified manually; build flag provided, no CI |
+| Windows | Core is platform-seamed; harness port incomplete, CI informational |
+| Product-ready `install.sh` template | Not yet |
+| Publisher signing of installer or release metadata | Not provided; hash and size only |
+| Automatic restart after machine reboot | Not provided; product OS hook |
+| Receipt archive garbage collection | Not provided |
+
+## Documentation
+
+Guides, written to be read in order:
+
+- [How an upgrade works](docs/guide.md): the processes involved, one upgrade start to finish, what breaks, how to read the result
+- [Runnable example](examples/external-service/README.md)
+- [Integration and distribution guide](docs/integration.md)
+
+Contracts, written to be looked up:
+
+- [Design](docs/design.md): normative execution model, transaction, supervision and controller obligations
+- [Reference](docs/reference.md): protocol, exit codes, budgets, on-disk layout
+- [Test plan](docs/test-plan.md) and [harness design](docs/harness-design.md)
+- [Formal model](formal/README.md)
+
+Background:
+
+- [Prior art](docs/prior-art/design-influences.md) and [external installer research](docs/prior-art/external-runner-research.md)
+
+Incubating · TypeScript / Node 24 · Apache-2.0. Contributions welcome.

package/core/src/artifact/download.ts

@@ -32,6 +32,7 @@ import { ArtifactError } from "./errors.ts";
 import type { Release } from "./source.ts";
 import { collectStream } from "./collectStream.ts";
 import { partialPathFor } from "./partialPath.ts";
+import { decodeGzipArtifact } from "./gzip.ts";
 import type { DownloadOptions } from "./transferPolicy.ts";
 export { partialPathFor } from "./partialPath.ts";
 export type { DownloadOptions } from "./transferPolicy.ts";
@@ -41,6 +42,27 @@ export async function downloadVerified(
   release: Release,
   opts: DownloadOptions = {},
 ): Promise<Uint8Array> {
+  if (release.gzip !== undefined) {
+    const gzip = release.gzip;
+    if (!gzip || typeof gzip.url !== "string" || !gzip.url ||
+        !/^[a-f0-9]{64}$/.test(gzip.sha256) || !Number.isSafeInteger(gzip.size) || gzip.size <= 0 ||
+        !Number.isSafeInteger(release.size) || release.size < 0 ||
+        !/^[a-f0-9]{64}$/.test(release.sha256)) {
+      throw new ArtifactError("MANIFEST_INVALID", "invalid gzip artifact identity");
+    }
+    // Range and partial files refer to compressed bytes, never decoded offsets.
+    const compressed = await downloadVerified({
+      version: release.version, url: gzip.url, size: gzip.size, sha256: gzip.sha256,
+    }, opts);
+    try {
+      return await decodeGzipArtifact(compressed, release);
+    } catch (error) {
+      if (opts.resumeDir) {
+        await fs.rm(partialPathFor(opts.resumeDir, gzip.url), { force: true }).catch(() => {});
+      }
+      throw error;
+    }
+  }
   const url = release.url;
   const clock = opts.clock ?? systemClock;
   const timeoutMs = opts.timeoutMs ?? 10000;
@@ -147,7 +169,7 @@ async function fetchAndAppend(
     });
   };
   try {
-    const headers: Record<string, string> = {};
+    const headers: Record<string, string> = { "Accept-Encoding": "identity" };
     if (partialSize > 0) headers["Range"] = `bytes=${partialSize}-`;
     let res: Response;
     armResponse();

package/core/src/artifact/gzip.ts

@@ -0,0 +1,26 @@
+import { createHash } from "node:crypto";
+import { gunzip } from "node:zlib";
+import { ArtifactError } from "./errors.ts";
+import type { Release } from "./source.ts";
+
+/** Decode only verified transport bytes; bound expansion by the canonical size. */
+export async function decodeGzipArtifact(bytes: Uint8Array, release: Release): Promise<Uint8Array> {
+  let decoded: Uint8Array;
+  try {
+    decoded = await new Promise<Buffer>((resolve, reject) => {
+      gunzip(bytes, { maxOutputLength: Math.max(1, release.size) }, (error, output) => {
+        if (error) reject(error);
+        else resolve(output);
+      });
+    });
+  } catch (cause) {
+    throw new ArtifactError("DOWNLOAD_FAILED", "gzip is invalid or exceeds the declared decoded size", { cause });
+  }
+  if (decoded.length !== release.size) {
+    throw new ArtifactError("SIZE_MISMATCH", "decoded gzip size does not match the release");
+  }
+  if (createHash("sha256").update(decoded).digest("hex") !== release.sha256) {
+    throw new ArtifactError("SHA256_MISMATCH", "decoded gzip SHA-256 does not match the release");
+  }
+  return decoded;
+}

package/core/src/artifact/source.ts

@@ -39,6 +39,8 @@ export interface Release {
   url: string;
   sha256: string;
   size: number;
+  /** Optional explicit gzip representation. Outer size/hash always describe installed bytes. */
+  gzip?: { url: string; sha256: string; size: number };
 }
 
 export interface ReleaseSource {

package/core/src/{createUpgrader.ts → createRunner.ts}

@@ -1,5 +1,6 @@
+import { HostCallUncertain } from "./txn/hostCallBudget.ts";
 /**
- * createUpgrader — the one construction every entrypoint uses.
+ * createRunner — the one construction every entrypoint uses.
  *
  * Order of gates: lock -> ownership -> source -> policy -> compat ->
  * download -> engine. Everything before the engine can only produce `held`
@@ -27,9 +28,11 @@ import type { OperationDescriptor } from "./operation.ts";
 import { createOperationLifecycle } from "./operationLifecycle.ts";
 import { driveUpgrade } from "./upgrade/drive.ts";
 import type { ArtifactTransferPolicy } from "./artifact/transferPolicy.ts";
+import { quarantineState } from "./quarantine.ts";
 
-export interface CreateUpgraderOptions extends UpgraderConfig {
+export interface RunnerOptions extends UpgraderConfig {
   clock?: Clock;
+  hostCallBudgetMs?: number;
   /** Reports who owns this install; default: we own it. */
   installOwnership?: () => "self" | "managed-elsewhere";
   /** Optional host semantic gate; a string result refuses the transition. */
@@ -67,7 +70,7 @@ export interface CreateUpgraderOptions extends UpgraderConfig {
   artifactTransferPolicy?: ArtifactTransferPolicy;
 }
 
-export function createUpgrader(opts: CreateUpgraderOptions): Upgrader {
+export function createRunner(opts: RunnerOptions): Upgrader {
   const clock = opts.clock ?? systemClock;
   const effects = fileEffects(opts.stateDir);
   const ownership = opts.installOwnership ?? ((): "self" => "self");
@@ -91,6 +94,7 @@ export function createUpgrader(opts: CreateUpgraderOptions): Upgrader {
     effects,
     host: opts.host,
     clock,
+    ...(opts.hostCallBudgetMs === undefined ? {} : { hostCallBudgetMs: opts.hostCallBudgetMs }),
     evaluatePredicates: async (evidence: ProcessEvidence, targetVersion: string) => {
       lastEvidence = evidence;
       if (evidence.version !== targetVersion) {
@@ -178,9 +182,9 @@ export function createUpgrader(opts: CreateUpgraderOptions): Upgrader {
     }, request);
 
   return {
-    recover: async () => {
+    recover: async (expected) => {
       operationLifecycle.reset();
-      await recoverUpgrade(opts.stateDir, clock, engine, operationLifecycle.settleRecovery);
+      await recoverUpgrade(opts.stateDir, clock, engine, operationLifecycle.settleRecovery, expected);
     },
 
     async check(): Promise<{ current: string; target: string | null }> {
@@ -224,6 +228,7 @@ export function createUpgrader(opts: CreateUpgraderOptions): Upgrader {
 
     async rollback(reason: string): Promise<"rolled-back" | { held: string }> {
       const lock = await acquireUpgradeLock(opts.stateDir, clock.nowMs());
+      let release = true;
       try {
         // Gate on the action's nature: settling K's own in-flight
         // transaction is ALWAYS allowed (a held mid-transaction is a
@@ -239,8 +244,11 @@ export function createUpgrader(opts: CreateUpgraderOptions): Upgrader {
         await effects.slots.clearExperiment();
         await opts.notificationSink({ kind: "rolled-back", detail: { reason } });
         return "rolled-back";
+      } catch (error) {
+        if (error instanceof HostCallUncertain) release = false;
+        throw error;
       } finally {
-        await lock.release();
+        if (release) await lock.release();
       }
     },
 
@@ -259,13 +267,6 @@ export function createUpgrader(opts: CreateUpgraderOptions): Upgrader {
 
     operation: operationLifecycle.read,
 
-    async acknowledgeOperation(operationId) {
-      const lock = await acquireUpgradeLock(opts.stateDir, clock.nowMs());
-      try {
-        return await operationLifecycle.acknowledge(operationId);
-      } finally {
-        await lock.release();
-      }
-    },
+    quarantineState: (options) => quarantineState(opts.stateDir, options),
   };
 }

package/core/src/index.ts

@@ -3,8 +3,9 @@
 // This barrel is the single supported entry point for the core framework;
 // deep imports into ./core/src/** are internal and not part of the public API.
 
-// The upgrader factory and its configuration.
-export * from "./createUpgrader.ts";
+// Compose the transaction engine inside a runner; no forwarding factory.
+export { createRunner } from "./createRunner.ts";
+export type { RunnerOptions } from "./createRunner.ts";
 
 // One-time adoption of an already-running trusted binary into K's stable
 // slot, plus the K-owned slot resolver host adapters use to launch it.
@@ -14,9 +15,10 @@ export * from "./bootstrap.ts";
 // NotificationEvent.
 export * from "./upgrader.ts";
 export * from "./operation.ts";
+export * from "./quarantine.ts";
 
 // The release-source boundary applications implement and the durable
-// provenance journal they wire into createUpgrader.
+// provenance journal they wire into createRunner.
 export * from "./artifact/source.ts";
 export * from "./artifact/transferPolicy.ts";
 export * from "./provenance/journal.ts";
@@ -26,3 +28,10 @@ export * from "./lifecycle/hostAdapter.ts";
 
 // The built-in invariants and their types (WorldSnapshot, Invariant, ...).
 export * from "./invariants.ts";
+export * from "./protocol/runner.ts";
+export * from "./runner/execute.ts";
+export * from "./runner/cli.ts";
+export * from "./launcher/launch.ts";
+export * from "./lifecycle/commandHost.ts";
+
+export { HostCallTimeout, HostCallUncertain } from "./txn/hostCallBudget.ts";

package/core/src/launcher/launch.ts

@@ -0,0 +1,15 @@
+import { superviseRunner, type RunnerLaunch } from "./supervise.ts";
+export { superviseRunner, resumeRunner } from "./supervise.ts";
+export type { RunnerLaunch, RunnerLaunchResult } from "./supervise.ts";
+
+/** Installer entrypoint: one final response and an honest recovery exit code. */
+export async function launchRunner(input: RunnerLaunch): Promise<number> {
+  const result = await superviseRunner(input);
+  const response = result.error ? { protocolVersion: 1, action: input.request.action,
+    exitCode: 3, result: "recovery-required", error: result.error,
+    operation: result.response?.operation ?? { kind: "unreadable", reason: "worker returned no usable receipt" },
+    recoveryFile: result.recoveryFile } : result.response;
+  if (response) process.stdout.write(`${JSON.stringify(response)}\n`);
+  if (result.error) process.stderr.write(`${result.error}; recovery: ${result.recoveryFile}\n`);
+  return result.exitCode;
+}

package/core/src/launcher/supervise.ts

@@ -0,0 +1,170 @@
+import { spawn } from "node:child_process";
+import { promises as fs } from "node:fs";
+import path from "node:path";
+import { createHash } from "node:crypto";
+import type { Release } from "../artifact/source.ts";
+import { downloadVerified } from "../artifact/download.ts";
+import { artifactTransferTimeouts } from "../artifact/transferPolicy.ts";
+import { parseRunnerRequest, parseRunnerResponse, type RunnerRequest, type RunnerResponse } from "../protocol/runner.ts";
+import { platformOpsFor } from "../platform/index.ts";
+import { systemClock } from "../clock.ts";
+
+export interface RunnerLaunch {
+  release: Release;
+  request: RunnerRequest;
+  scratchDir: string;
+  interpreter?: string;
+  /** Finite worker, recovery-attempt and overall execution budgets. */
+  executionTimeoutMs?: number;
+  recoveryTimeoutMs?: number;
+  totalTimeoutMs?: number;
+  recoveryAttempts?: number;
+}
+
+export interface RunnerLaunchResult {
+  exitCode: number;
+  response: RunnerResponse | null;
+  /** Retained verified helper + invocation; no transaction state is copied. */
+  recoveryFile: string | null;
+  error: string | null;
+  attempts: number;
+}
+
+interface Attempt { code: number; response: RunnerResponse | null; fenced: boolean }
+
+function budget(value: number | undefined, fallback: number): number {
+  const result = value ?? fallback;
+  if (!Number.isSafeInteger(result) || result <= 0 || result > 2_147_483_647) throw new Error("invalid runner budget");
+  return result;
+}
+
+/** A worker is reaped before a successor is allowed. Controller effects are
+ * separately fenced by the adapter under K's lock, including after a restart. */
+async function run(file: string, interpreter: string | undefined, request: RunnerRequest, timeout: number): Promise<Attempt> {
+  return new Promise((resolve) => {
+    const child = spawn(interpreter ?? file, interpreter ? [file] : [], { stdio: ["pipe", "pipe", "inherit"] });
+    let output = "";
+    let exited = false;
+    let expired = false;
+    let finished = false;
+    let cancelGrace: (() => void) | undefined;
+    const finish = (code: number, fenced: boolean): void => {
+      if (finished) return;
+      finished = true;
+      cancel(); cancelGrace?.();
+      child.stdin.destroy(); child.stdout.destroy(); child.unref();
+      let response: RunnerResponse | null = null;
+      if (!expired) {
+        try { response = parseRunnerResponse(JSON.parse(output)); }
+        catch { /* a missing/malformed response cannot settle a transaction */ }
+      }
+      if (response && (response.action !== request.action || response.exitCode !== code)) response = null;
+      resolve({ code, response, fenced });
+    };
+    const terminate = (): void => {
+      if (expired || finished) return;
+      expired = true;
+      if (child.pid && !exited) {
+        try { platformOpsFor().killProcess(child.pid); } catch { /* require observed exit */ }
+      }
+      child.stdin.destroy(); child.stdout.destroy();
+      cancelGrace = systemClock.after(1000, () => finish(3, exited));
+    };
+    const cancel = systemClock.after(timeout, terminate);
+    child.on("error", () => finish(1, child.pid === undefined));
+    child.on("exit", () => { exited = true; });
+    child.on("close", (code) => finish(expired ? 3 : code ?? 1, exited || child.pid === undefined));
+    child.stdout.on("data", (chunk: Buffer) => {
+      output += chunk.toString();
+      if (output.length > 64 * 1024) terminate();
+    });
+    child.stdin.on("error", () => {});
+    child.stdin.end(JSON.stringify(request));
+  });
+}
+
+function settled(response: RunnerResponse | null, request: RunnerRequest): boolean {
+  if (!response || response.error !== null) return false;
+  const expected = request.action === "upgrade" ? request : request.action === "recover" ? request.expected : undefined;
+  if (!expected) return response.exitCode !== 3 && response.operation.kind !== "unreadable" &&
+    (response.operation.kind === "genesis" || response.operation.operation.outcome !== null);
+  if (response.operation.kind !== "observed") return false;
+  const op = response.operation.operation;
+  const code = op.outcome === "promoted" || op.outcome === "up-to-date" ? 0 : op.outcome === "held" ? 2 : 1;
+  return op.id === expected.id && op.targetVersion === expected.targetVersion && op.outcome !== null && response.exitCode === code;
+}
+
+/** Own one install until a bound receipt or explicit unresolved recovery. */
+export async function superviseRunner(input: RunnerLaunch): Promise<RunnerLaunchResult> {
+  const request = parseRunnerRequest(input.request);
+  const execution = budget(input.executionTimeoutMs, 600_000);
+  const recovery = budget(input.recoveryTimeoutMs, 120_000);
+  const total = budget(input.totalTimeoutMs, execution + recovery * 2);
+  const retries = input.recoveryAttempts ?? 2;
+  if (!Number.isSafeInteger(retries) || retries < 0 || retries > 10) throw new Error("invalid recovery attempts");
+  const transfer = artifactTransferTimeouts(input.release.size);
+  const bytes = await downloadVerified(input.release, { timeoutMs: transfer.overallTimeoutMs,
+    responseTimeoutMs: transfer.responseTimeoutMs, idleTimeoutMs: transfer.idleTimeoutMs });
+  await fs.mkdir(input.scratchDir, { recursive: true });
+  const dir = await fs.mkdtemp(path.join(input.scratchDir, "k-runner-"));
+  const file = path.join(dir, input.interpreter ? "runner.mjs" : "runner.bin");
+  const recoveryFile = path.join(dir, "recovery.json");
+  const expected = request.action === "upgrade" ? { id: request.id, targetVersion: request.targetVersion } :
+    request.action === "recover" ? request.expected : undefined;
+  const recoverRequest: RunnerRequest = { protocolVersion: 1, action: "recover", ...(expected ? { expected } : {}) };
+  // Retain enough trusted distribution metadata to verify and recover offline.
+  // This is an invocation descriptor, never another transaction log.
+  const artifact = await fs.open(file, "wx", 0o700);
+  try { await artifact.writeFile(bytes); await artifact.sync(); }
+  finally { await artifact.close(); }
+  await platformOpsFor().makeExecutable(file);
+  const handle = await fs.open(recoveryFile, "w", 0o600);
+  try { await handle.writeFile(JSON.stringify({ file, interpreter: input.interpreter,
+    sha256: createHash("sha256").update(bytes).digest("hex"), size: bytes.length, request: recoverRequest })); await handle.sync(); }
+  finally { await handle.close(); }
+  const deadline = systemClock.nowMs() + total;
+  let response: RunnerResponse | null = null;
+  let attempts = 0;
+  for (; attempts <= retries; attempts++) {
+    const remaining = deadline - systemClock.nowMs();
+    if (remaining <= 0) break;
+    const attempt = await run(file, input.interpreter, attempts === 0 ? request : recoverRequest,
+      Math.min(remaining, attempts === 0 ? execution : recovery));
+    response = attempt.response;
+    if (attempt.fenced && (request.action === "status" || settled(response, request))) {
+      await fs.rm(dir, { force: true, recursive: true });
+      const exitCode = request.action === "status" && !response && attempt.code === 0 ? 1 : attempt.code;
+      return { exitCode, response, recoveryFile: null, error: null, attempts: attempts + 1 };
+    }
+    if (!attempt.fenced || !expected) { attempts++; break; }
+  }
+  return { exitCode: 3, response, recoveryFile, attempts,
+    error: "RECOVERY_UNRESOLVED: retained helper and invocation; transaction evidence is unchanged" };
+}
+
+/** Resume the retained, hash-verified helper without distribution access. */
+export async function resumeRunner(recoveryFile: string, options: Pick<RunnerLaunch,
+  "executionTimeoutMs" | "recoveryTimeoutMs" | "totalTimeoutMs" | "recoveryAttempts"> = {}): Promise<RunnerLaunchResult> {
+  const descriptor = JSON.parse(await fs.readFile(recoveryFile, "utf8")) as {
+    file: string; interpreter?: string; sha256: string; size: number; request: RunnerRequest;
+  };
+  const request = parseRunnerRequest(descriptor.request);
+  if (request.action !== "recover" || !request.expected ||
+      typeof descriptor.file !== "string" || (descriptor.interpreter !== undefined && typeof descriptor.interpreter !== "string")) {
+    throw new Error("INVALID_RECOVERY_DESCRIPTOR");
+  }
+  const bytes = await fs.readFile(descriptor.file);
+  if (bytes.length !== descriptor.size || createHash("sha256").update(bytes).digest("hex") !== descriptor.sha256) {
+    throw new Error("RECOVERY_ARTIFACT_MISMATCH");
+  }
+  const result = await superviseRunner({ ...options, request,
+    ...(descriptor.interpreter ? { interpreter: descriptor.interpreter } : {}),
+    scratchDir: path.dirname(path.dirname(recoveryFile)), release: { version: "retained-runner",
+      url: `data:application/octet-stream;base64,${bytes.toString("base64")}`, size: descriptor.size, sha256: descriptor.sha256 } });
+  if (!result.recoveryFile) {
+    // Remove only the files this descriptor owns, never its caller's directory.
+    await fs.rm(descriptor.file, { force: true });
+    await fs.rm(recoveryFile, { force: true });
+  }
+  return result;
+}

package/core/src/lifecycle/commandHost.ts

@@ -0,0 +1,111 @@
+import { spawn } from "node:child_process";
+import { promises as fs } from "node:fs";
+import path from "node:path";
+import { randomUUID } from "node:crypto";
+import { HostCallTimeout, HostCallUncertain } from "../txn/hostCallBudget.ts";
+import type { HostAdapter, ProcessEvidence, Slot } from "./hostAdapter.ts";
+import { slotArtifactPath } from "../bootstrap.ts";
+import { systemClock } from "../clock.ts";
+import { platformOpsFor } from "../platform/index.ts";
+function controllerObject(value: unknown): Record<string, unknown> {
+  if (value === null || typeof value !== "object" || Array.isArray(value)) {
+    throw new Error("HOST_PROTOCOL_INVALID: expected an object");
+  }
+  return value as Record<string, unknown>;
+}
+
+export interface CommandHostOptions {
+  /** Trusted external controller argv. Never a shell expression. */
+  command: [string, ...string[]];
+  stateDir: string;
+  cwd?: string;
+  timeoutMs?: number;
+}
+
+/** Controller stdin is one JSON request; stdout is one JSON response. */
+export function createCommandHost(options: CommandHostOptions): HostAdapter {
+  const timeout = options.timeoutMs ?? 30_000;
+  if (!Number.isSafeInteger(timeout) || timeout <= 0 || timeout > 120_000) throw new Error("invalid controller timeout");
+  const [file, ...args] = options.command;
+  const controllersDir = path.join(options.stateDir, "controllers");
+  async function drainControllers(): Promise<void> {
+    await fs.mkdir(controllersDir, { recursive: true });
+    const deadline = systemClock.nowMs() + timeout;
+    for (const name of await fs.readdir(controllersDir)) {
+      const controllerPath = path.join(controllersDir, name);
+      const pid = Number(name.split("-")[0]);
+      if (!Number.isSafeInteger(pid) || pid <= 0) throw new Error("HOST_FENCE_UNRESOLVED: invalid controller evidence");
+      while (platformOpsFor().isProcessAlive(pid)) {
+        if (systemClock.nowMs() >= deadline) throw new HostCallTimeout("fence", timeout);
+        await new Promise<void>((resolve) => { systemClock.after(Math.min(50, timeout), resolve); });
+      }
+      await fs.rm(controllerPath, { force: true });
+    }
+  }
+  async function call(action: string, slot?: Slot): Promise<Record<string, unknown>> {
+    const input = { protocolVersion: 1, action,
+      ...(slot ? { slot, artifactPath: slotArtifactPath(options.stateDir, slot) } : {}) };
+    const child = spawn(file, args, { stdio: ["pipe", "pipe", "pipe"],
+      ...(options.cwd ? { cwd: options.cwd } : {}) });
+    const controllerPath = path.join(controllersDir, `${child.pid}-${randomUUID()}.json`);
+    let exited = false;
+    let cancel: (() => void) | undefined;
+    let stdout = "";
+    const done = new Promise<string>((resolve, reject) => {
+      child.on("error", reject);
+      child.on("exit", () => { exited = true; });
+      child.stdout.on("data", (chunk: Buffer) => {
+        stdout += chunk.toString();
+        if (stdout.length > 64 * 1024) reject(new HostCallUncertain("HOST_RESPONSE_TOO_LARGE"));
+      });
+      child.stderr.resume();
+      child.on("close", (code) => code === 0 ? resolve(stdout) : reject(new Error(`HOST_COMMAND_FAILED: ${action} (${code})`)));
+    });
+    // Install rejection handlers before filesystem IO or an early spawn failure.
+    void done.catch(() => {});
+    child.stdin.on("error", () => {});
+    try {
+      if (!child.pid) return await done.then(() => { throw new Error("HOST_SPAWN_FAILED"); });
+      await fs.mkdir(controllersDir, { recursive: true });
+      const handle = await fs.open(controllerPath, "wx", 0o600);
+      try { await handle.writeFile(JSON.stringify({ pid: child.pid })); await handle.sync(); }
+      finally { await handle.close(); }
+      // No controller gets a command before its lifetime is recorded. A worker
+      // dying before this point closes stdin without authorizing any effect.
+      child.stdin.end(JSON.stringify(input));
+      stdout = await Promise.race([done, new Promise<never>((_resolve, reject) => {
+        cancel = systemClock.after(timeout, () => reject(new HostCallTimeout(action, timeout)));
+      })]);
+    } finally {
+      cancel?.();
+      if (!exited && child.pid) {
+        try { platformOpsFor().killProcess(child.pid); } catch { /* fence will verify */ }
+      }
+      child.stdin.destroy(); child.stdout.destroy(); child.stderr.destroy(); child.unref();
+      // On uncertain termination retain the process evidence for the next fence.
+      if (exited) await fs.rm(controllerPath, { force: true });
+    }
+    const value = controllerObject(JSON.parse(stdout));
+    if (value.protocolVersion !== 1 || value.ok !== true) throw new Error(`HOST_PROTOCOL_INVALID: ${action}`);
+    return value;
+  }
+  return {
+    fence: async () => {
+      await drainControllers();
+      // The controller must also fence detached/queued service-manager effects;
+      // absence of its old PID alone does not establish that condition.
+      await call("fence");
+    },
+    quiesce: async () => { await call("quiesce"); },
+    stop: async (slot) => { await call("stop", slot); },
+    start: async (slot) => { await call("start", slot); },
+    resume: async () => { await call("resume"); },
+    healthProbe: async (): Promise<ProcessEvidence> => {
+      const value = controllerObject((await call("probe")).evidence);
+      if (typeof value.version !== "string" || !value.version || typeof value.startId !== "string" ||
+          !value.startId || !Number.isSafeInteger(value.pid) || typeof value.pid !== "number" || value.pid <= 0 ||
+          value.pid === process.pid) throw new Error("HOST_EVIDENCE_INVALID");
+      return { version: value.version, pid: value.pid, startId: value.startId };
+    },
+  };
+}