@catalyst-cloud/cli 0.16.43 → 0.16.44

package/CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## 0.16.44
+
+`catalyst runner update` now adds the log setting to a self-hosted machine that was set up before it existed. If the machine's `compose.yaml` has no `CATALYST_NODE_LOG_SHIPPING` line, the update adds the line a new setup writes and prints a notice saying so. With that line, which is on by default, Catalyst's own service containers on the machine send their logs to Catalyst, with secrets removed first, once Catalyst turns log collection on for your workspace. To keep it off, set `CATALYST_NODE_LOG_SHIPPING=off` in the runner's `.env` and run `docker compose up -d`; a value you set there always wins. An update that rolls back leaves `compose.yaml` as it was, and a file that already names the setting, or that the update does not recognise, is left alone.
+
+A runner release that comes up while the machine is pausing work to prepare is no longer rolled back for that pause alone. An automatic update keeps the new release for up to 45 minutes, until the machine is ready, and reports it as waiting. If the machine is still not ready once the pause ends, the update rolls back as before. `catalyst runner update` also prints its outcome and releases its lock when Docker exits before reading its input, where it used to stop without a result.
+
 ## 0.16.43
 
 `catalyst ask accept` now waits for an answer that has not reached Catalyst yet, instead of failing. If you accept an ask a few seconds after answering it, Catalyst may not have your answer comment yet. The command used to stop with a refusal, and the same command worked when you ran it again. Now it waits as long as Catalyst asks and tries again, for up to 30 seconds. If the answer still has not arrived, it says so and tells you when to run the same command again. A refusal that will not change on retry is reported at once, with its reason.

package/dist/args.js

@@ -125,7 +125,7 @@ export const FLAG_TABLES = {
         },
         "coding-account": {
             value: true,
-            help: "verify this coding-account slot (catalyst accounts lists them; or CATALYST_ONBOARD_CODING_ACCOUNT)",
+            help: "verify this AI account identifier (catalyst accounts lists them; or CATALYST_ONBOARD_CODING_ACCOUNT)",
         },
     },
     status: {},
@@ -235,7 +235,7 @@ export const FLAG_TABLES = {
     queue: {
         team: {
             value: true,
-            help: "team key (default: every team the tenant contract names)",
+            help: "team key (default: every team the workspace contract names)",
         },
     },
     watch: {
@@ -344,11 +344,11 @@ export const FLAG_TABLES = {
         },
         "gates-repo": {
             value: true,
-            help: "raise: the --gates-pr repository as owner/name (default: the account's only repository)",
+            help: "raise: the --gates-pr repository as owner/name (default: the workspace's only repository)",
         },
         anyone: {
             value: false,
-            help: "list: every open ask in the tenant, not only the ones assigned to you",
+            help: "list: every open ask in the workspace, not only the ones assigned to you",
         },
         "ask-key": { value: true, help: "raise: idempotency key" },
         answer: { value: true, help: "accept: the answering comment id" },
@@ -433,7 +433,7 @@ export const FLAG_TABLES = {
     var: {
         repo: {
             value: true,
-            help: "repository scope (owner/name); omit for tenant scope",
+            help: "repository scope (owner/name); omit for workspace scope",
         },
         names: {
             value: true,
@@ -554,7 +554,7 @@ export const FLAG_TABLES = {
     "codex-env": {
         print: {
             value: false,
-            help: "print the OTEL_RESOURCE_ATTRIBUTES value a Codex launch should carry: the attributes already set, plus the coding account's slot and this machine's short name",
+            help: "print the OTEL_RESOURCE_ATTRIBUTES value a Codex launch should carry: the attributes already set, plus the AI account identifier and this computer's short name",
         },
         apply: {
             value: false,
@@ -562,7 +562,7 @@ export const FLAG_TABLES = {
         },
         refresh: {
             value: false,
-            help: "fetch this workspace's coding accounts now and update the cache --print and --apply read",
+            help: "fetch this workspace's AI accounts now and update the cache --print and --apply read",
         },
         "codex-home": { value: true, help: "print: the Codex home to read (default: $CODEX_HOME, else ~/.codex)" },
     },
@@ -655,7 +655,7 @@ export const VERB_USAGE = {
     flow: "flow [--repo <owner/name>] [--period 24h|2d|3d|1w|2w|4w|12w|26w] [--granularity hour|day] [--json]   (a repository's open pull requests over time, one row per hour or day; needs login)",
     project: "project <list [--json] | wip-limit <get|set <n>|set default> [--team K] [--json]>",
     lease: "lease <take <ticket> --agent <seat> [--ttl <minutes>] | renew <ticket> (--commit <sha>|--artifact <path or id>|--finding <id>|--pr <number>) [--note <text>] [--agent <seat>] [--generation <n>] | release <ticket> [--agent <seat>] [--generation <n>] | show <ticket>> [--json]   (a seat's ticket lease: take, renew with new progress, release, read; needs login)",
-    "codex-env": "codex-env --print [--codex-home <dir>] | --apply | --refresh [--json]   (the telemetry attributes naming the coding account and machine a Codex launch runs as; only --refresh waits on the network)",
+    "codex-env": "codex-env --print [--codex-home <dir>] | --apply | --refresh [--json]   (the telemetry attributes naming the AI account and computer a Codex launch runs as; only --refresh waits on the network)",
     work: "work available [--team K] [--json]   (per team: tickets nobody holds, with the last holder and progress, and open pull requests no live lease covers, drafts included)",
     repo: "repo <status|pause <owner/name> --reason <text>|resume <owner/name> [--reason <text>]|agents-block <path> [--write]|agent-setup <path> [--apply] [--with-check]> [--json]   (status: visible workspace repositories; agents-block/agent-setup: local checkout)",
     legacy: "legacy [--remove [--data] [--yes]] [--json]",

package/dist/capabilities.js

@@ -48,8 +48,8 @@ export const CAPABILITIES = [
     { verb: "secret set|import", does: "enter a repository secret's value from this terminal, never echoed", needs: "admin", routes: [], since: "0.8.0" },
     { verb: "var set|import", does: "enter plain environment variable values from this terminal, never echoed", needs: "admin", routes: [], since: "0.13.2" },
     { verb: "env import", does: "store this computer's values for the names .catalyst/catalyst.toml declares, never printed", needs: "admin", routes: [], since: "0.16.20" },
-    { verb: "accounts", does: "read the workspace's coding accounts and which can take work", needs: "member", routes: [], since: "0.5.0" },
-    { verb: "codex-env", does: "name the coding account and the machine on every Codex launch's telemetry: print the attributes, set them for apps on macOS, or refresh the account cache", needs: "member", routes: [], since: "0.16.33" },
+    { verb: "accounts", does: "read the workspace's AI accounts and which can take work", needs: "member", routes: [], since: "0.5.0" },
+    { verb: "codex-env", does: "name the AI account and the computer on every Codex launch's telemetry: print the attributes, set them for apps on macOS, or refresh the AI account cache", needs: "member", routes: [], since: "0.16.33" },
     { verb: "setup --remove-codex-attribution", does: "take out the Codex launch hooks setup adds: the codex function in the shell rc files, and on macOS the login agent and the attributes it set", needs: "member", routes: [], since: "0.16.34" },
     { verb: "explain|running|queue|history", does: "read why a ticket runs or waits, what runs now, and what is queued", needs: "member", routes: [], since: "0.1.0" },
     { verb: "write comment|state|label|create|reaction|attachment|session", does: "act on a ticket as the person, through the agent proxy", needs: "member", routes: [agent("POST", "issue-comment"), agent("POST", "issue-state"), agent("POST", "issue-label"), agent("POST", "issue-create")], since: "0.1.0" },

package/dist/cli.js

@@ -110,7 +110,7 @@ export function usageText() {
         "  catalyst write <comment|state|label|create|reaction|attachment|session> ...",
         "  catalyst ask <raise|accept|answer --proxy|proxy [status|on|off]|disown|list> ...",
         "  catalyst release <ticket> --because <what changed> [--retry-unchanged] [--dry-run] | release --class <c> --team <K> ...",
-        "  catalyst environment [read] | environment propose --file <path>|--stdin [--approve] | environment approve   (your account; needs login)",
+        "  catalyst environment [read] | environment propose --file <path>|--stdin [--approve] | environment approve   (your workspace; needs login)",
         "  catalyst env <inventory [path] | check <file> | migrate [catalyst.env.json] | draft [--root DIR] [--diff] [--write] | shell [--root DIR] [--env-file FILE]... [--image REF] [--check]> [--json]   (this repository; no login — shell needs Docker and downloads Catalyst's image once)",
         "  catalyst secret set <NAME> --repo <owner/name> [--command '<cmd>'] | secret import <file> --repo <owner/name> [--names NAME[,NAME...]]",
         "  catalyst var set <NAME> | var import <file> [--repo <owner/name>] [--names NAME[,NAME...]]   (plain environment variables)",
@@ -125,7 +125,7 @@ export function usageText() {
         "  catalyst repo <agents-block <path> [--write]|agent-setup <path> [--apply] [--with-check]>   (a checkout's AGENTS.md block and portable agent layout; working tree only)",
         "  catalyst lease <take <ticket> --agent <seat> [--ttl <minutes>]|renew <ticket> --commit|--artifact|--finding|--pr <ref> [--note <text>]|release <ticket>|show <ticket>> [--json]   (a seat's ticket lease; needs login)",
         "  catalyst work available [--team K] [--json]   (tickets nobody holds and open pull requests no lease covers)",
-        "  catalyst codex-env --print [--codex-home <dir>] | --apply | --refresh   (the telemetry attributes naming the coding account and machine a Codex launch runs as)",
+        "  catalyst codex-env --print [--codex-home <dir>] | --apply | --refresh   (the telemetry attributes naming the AI account and computer a Codex launch runs as)",
         "  catalyst legacy [--remove [--data] [--yes]]   (leftovers of the old local Catalyst runtime: list them; remove them only on a yes)",
         "  catalyst identity linear <status|options|set> [<linearUserId>] [--json]",
         "  catalyst messages read --ticket CTC-N [--since SEQUENCE] [--json]   (messages sent to a ticket on your teams)",

package/dist/execution.js

@@ -35,7 +35,7 @@ export const EXCLUSION_REASONS = {
     runner_image_breaker: "fleet-wide: the live runner image is failing every phase at startup; dispatch resumes when the pin moves",
     no_branch_to_remediate: "a remediate round is queued on a ticket with no recorded branch",
     retry_backoff: "the failed phase is retrying in place and waiting out its backoff",
-    routing_unavailable: "the unit was claimed and refused at kickoff (no route, no eligible coding-account slot, or the provider is unavailable)",
+    routing_unavailable: "the unit was claimed and refused at kickoff (no route, no eligible AI account, or the provider is unavailable)",
     repo_paused: "an operator paused the repository",
     remediate_parked: "the remediate phase is parked, so the failing phase has nowhere to be repaired; `catalyst release <ticket>` releases the park once its cause is fixed",
     phase_parked: "the offered phase is parked after repeated failures or the repair-round cap; `catalyst release <ticket>` releases it once its cause is fixed",
@@ -374,7 +374,7 @@ export async function cmdAccounts(args, ctx) {
         accept: [404],
     });
     if (res.status === 404)
-        throw needsNewerCloud("coding-account status", cfg);
+        throw needsNewerCloud("AI account status", cfg);
     const accounts = Array.isArray(res.body) ? res.body : (res.body?.accounts ?? []);
     // CTC-6204: removed accounts are history, so they are left out unless --all asks for them. Listed,
     // they read as accounts to repair, and agents kept trying to revive them.
@@ -388,7 +388,7 @@ export async function cmdAccounts(args, ctx) {
     const addAt = `${normalizeBaseUrl(cfg.baseUrl)}/settings/coding-accounts`;
     const hiddenLine = `${hidden} removed account${hidden === 1 ? " is" : "s are"} kept for history and take no work; catalyst accounts --all lists ${hidden === 1 ? "it" : "them"}.`;
     if (accounts.length === 0) {
-        ctx.stdout(`No coding accounts are enrolled in this workspace. A workspace owner or admin adds one at ${addAt}; there is no command for that yet.`);
+        ctx.stdout(`No AI accounts are enrolled in this workspace. A workspace owner or admin adds one at ${addAt}; there is no command for that yet.`);
         return 0;
     }
     if (shown.length === 0) {

package/dist/onboard-runner.js

@@ -96,6 +96,9 @@ export const spawnDocker = (args, options) => new Promise((resolve, reject) => {
     });
     child.on("error", reject);
     child.on("close", (code) => resolve({ code: code ?? 1, stdout, stderr }));
+    // Listen before writing (src/watch.ts:96-121): a docker that exits before reading its stdin
+    // raises EPIPE here, and the exit code from `close` is still the verdict.
+    child.stdin.on("error", () => { });
     child.stdin.end(options.input ?? "");
 });
 /** Compose's own error lines, reduced to short printable text a person can act on. */

package/dist/runner-command.js

@@ -193,7 +193,7 @@ export async function cmdRunner(args, ctx, deps = {}) {
             : await runRunnerUpdate(engine, {
                 dir: dir(), check,
                 ...(ownerWindows ? { window: windowBounds(ctx.now().getTime(), ownerWindows) } : {}),
-                ...(auto && !check ? { drain: true, held: () => imageHold(ctx), ...(args.json ? {} : { onPass: (pass) => ctx.stdout(autoLine(pass, ctx.now())) }) } : {}),
+                ...(auto && !check ? { scheduled: true, drain: true, held: () => imageHold(ctx), ...(args.json ? {} : { onPass: (pass) => ctx.stdout(autoLine(pass, ctx.now())) }) } : {}),
             });
         // `runner update --check` also lists every installed component against the approved release, as
         // `catalyst update --check` does. The image fields stay where scripts already read them.

package/dist/runner-update.js

@@ -1,4 +1,4 @@
-import { closeSync, existsSync, openSync, readFileSync, renameSync, unlinkSync, writeFileSync } from "node:fs";
+import { chmodSync, closeSync, existsSync, openSync, readFileSync, renameSync, statSync, unlinkSync, writeFileSync } from "node:fs";
 import { join } from "node:path";
 import { randomUUID } from "node:crypto";
 import { readManifest } from "./config.js";
@@ -44,6 +44,11 @@ const LOCK_FILE = "update.lock";
 const PENDING_REMOVAL_KEPT = 20;
 const READY_DEADLINE_MS = 5 * 60_000;
 const READY_POLL_MS = 5_000;
+/** CTC-8745: the supervisor is up and its one unmet prerequisite is its own preparation stop. */
+const WAITING_PREPARATION_STOP = "waiting_preparation_stop";
+/** How long a release is kept through a preparation stop. Stops that end by a quiet period lift at 30
+ *  minutes; a workspace stop can hold forever, and the release can be what holds it. */
+const PREPARATION_STOP_WAIT_MS = 45 * 60_000;
 const ATTEMPT_HOST_LABEL = "dev.catalystcloud.attempt.host-id";
 const ATTEMPT_ID_LABEL = "dev.catalystcloud.attempt.id";
 const DESIRED_RELEASE_PATH = "/api/v1/hosts/desired-release";
@@ -173,7 +178,7 @@ export function decideRelease(appliedGeneration, read) {
     return { action: "update", release };
 }
 export function readReleaseState(dir) {
-    const empty = { version: 1, generation: null, releaseId: null, appliedAtMs: null, previous: null, journal: null, pendingRemoval: [], windowSettledMs: null, deferral: null, drain: null, last: null };
+    const empty = { version: 1, kept: null, generation: null, releaseId: null, appliedAtMs: null, previous: null, journal: null, pendingRemoval: [], windowSettledMs: null, deferral: null, drain: null, last: null };
     try {
         const parsed = JSON.parse(readFileSync(join(dir, RELEASE_STATE_FILE), "utf8"));
         if (!isRecord(parsed) || parsed.version !== 1)
@@ -191,6 +196,8 @@ export function readReleaseState(dir) {
                 ? parsed.pendingRemoval.filter((ref) => typeof ref === "string").slice(0, PENDING_REMOVAL_KEPT)
                 : [],
             windowSettledMs: posInt(parsed.windowSettledMs) ? parsed.windowSettledMs : null,
+            kept: isRecord(parsed.kept) && posInt(parsed.kept.generation) && posInt(parsed.kept.sinceMs)
+                ? { generation: parsed.kept.generation, sinceMs: parsed.kept.sinceMs } : null,
             deferral: isRecord(parsed.deferral) && posInt(parsed.deferral.sinceMs) && posInt(parsed.deferral.offered) && posInt(parsed.deferral.offeredSinceMs)
                 ? { sinceMs: parsed.deferral.sinceMs, offered: parsed.deferral.offered, offeredSinceMs: parsed.deferral.offeredSinceMs } : null,
             drain: isRecord(parsed.drain) && typeof parsed.drain.requestId === "string" && UUID.test(parsed.drain.requestId) &&
@@ -210,9 +217,34 @@ export function writePrivateAtomic(path, text) {
     writeFileSync(temp, text, { mode: 0o600 });
     renameSync(temp, path);
 }
+/** Write a file by rename with the mode it already has, so an edit keeps what setup chose. */
+function writeKeepingMode(path, text) {
+    const mode = statSync(path).mode & 0o777;
+    const temp = `${path}.${process.pid}.tmp`;
+    writeFileSync(temp, text, { mode });
+    chmodSync(temp, mode);
+    renameSync(temp, path);
+}
 export function writeReleaseState(dir, state) {
     writePrivateAtomic(join(dir, RELEASE_STATE_FILE), `${JSON.stringify(state, null, 2)}\n`);
 }
+const NODE_LOG_SETTING = /^\s*CATALYST_NODE_LOG_SHIPPING\s*:/m;
+const NODE_LOG_LINE = "      CATALYST_NODE_LOG_SHIPPING: ${CATALYST_NODE_LOG_SHIPPING-on}\n";
+const NODE_LOG_LINE_FOLLOWS = /^ {6}CATALYST_GIT_CLONE_TIMEOUT_MS: \$\{CATALYST_GIT_CLONE_TIMEOUT_MS:-\}\n/m;
+export const NODE_LOG_NOTICE = "Catalyst's own service containers on this machine now send their logs to Catalyst, with secrets removed first. To stop it, set CATALYST_NODE_LOG_SHIPPING=off in this runner's .env and run docker compose up -d.";
+/** CTC-8674: an update replaces image references and nothing else, so a machine set up earlier has no
+ *  log setting in its Compose file and ships nothing. This adds the one line, exactly as setup writes
+ *  it. Null when there is nothing to add: the setting is already there, or the file is not one we
+ *  recognise. `.env` is never touched here, so a value the owner set there always wins. */
+export function withNodeLogSetting(composeText) {
+    if (NODE_LOG_SETTING.test(composeText))
+        return null;
+    const follows = NODE_LOG_LINE_FOLLOWS.exec(composeText);
+    if (follows === null)
+        return null;
+    const at = follows.index + follows[0].length;
+    return `${composeText.slice(0, at)}${NODE_LOG_LINE}${composeText.slice(at)}`;
+}
 /** `.env` with each `KEY=` line for `values` replaced in place (or appended), every other line kept. */
 export function rewriteEnvText(text, values) {
     const pending = new Map(Object.entries(values));
@@ -419,6 +451,40 @@ async function running(engine, dir, images, signal) {
         return false;
     return engine.supervisorReadySinceStart(dir, signal);
 }
+/** CTC-8745: these images are running and only a preparation stop keeps them from being ready. A
+ *  stop survives a restart and lifts by itself, so going back would not help. */
+async function inPreparationStop(engine, dir, images, signal) {
+    const live = await engine.services(dir, signal);
+    if (!live?.supervisor?.running || live.supervisor.image !== images.supervisor)
+        return false;
+    if (!live.watchdog?.running || live.watchdog.image !== images.watchdog)
+        return false;
+    return engine.supervisorInPreparationStop(dir, signal);
+}
+/** CTC-8745: whether the latest readiness line in this log names a preparation stop and nothing
+ *  else, from a supervisor that writes a line whenever what is failing changes. */
+export function preparationStopOnly(logs) {
+    let latest = null;
+    for (const line of logs.split("\n")) {
+        if (!line.includes('"event":"host_'))
+            continue;
+        try {
+            const parsed = JSON.parse(line);
+            if (isRecord(parsed) && (parsed.event === "host_ready" || parsed.event === "host_not_ready"))
+                latest = parsed;
+        }
+        catch {
+            // Not a log line this reads.
+        }
+    }
+    // An older supervisor writes one line per flip, so its latest line can be stale: it does not count.
+    if (latest?.event !== "host_not_ready" || latest.linePerChange !== true)
+        return false;
+    if (!Array.isArray(latest.failing) || latest.failing.length !== 1)
+        return false;
+    const [only] = latest.failing;
+    return isRecord(only) && only.id === "host.preparation-failed" && only.reason === "failed";
+}
 const REPORT_DETAIL = /[^a-z0-9_:,.-]/g;
 /** The previous run's outcome as the release read reports it (CTC-6924), or undefined before the
  *  first run. Only the fixed outcome vocabulary leaves the machine: anything outside the cloud's
@@ -557,6 +623,7 @@ async function update(engine, options, now, fresh = false, onHostId) {
     const { dir, signal } = options;
     const envPath = join(dir, ".env");
     let state = readReleaseState(dir);
+    const notices = [];
     const done = (outcome, detail, extra = {}) => ({ outcome, detail, generation: state.generation, exitCode: 0, ...extra });
     const read = await engine.desiredRelease(dir, signal, options.check ? undefined : updateReport(state, readManifest().version), fresh);
     const hostId = read.kind === "no_credential" ? "" : read.hostId;
@@ -565,6 +632,10 @@ async function update(engine, options, now, fresh = false, onHostId) {
     const offered = read.kind === "answered" && read.status === 200 && isRecord(read.body) && read.body.ok === true
         ? parseDesiredRelease(read.body.release) : null;
     if (decision.action === "retain") {
+        // CTC-8745: a kept release ends on local files alone, so a read that offers no release (a hold,
+        // an outage, an older generation) cannot keep it on past its wait.
+        if (!options.check && state.kept !== null && (options.scheduled !== true || now() - state.kept.sinceMs >= PREPARATION_STOP_WAIT_MS))
+            return returnFromKept(engine, options, state, hostId, now);
         if (!options.check && state.drain?.acknowledgedAtMs != null && decision.reason !== "not_newer")
             return drainingResult(dir, state, now(), null, undefined, `read:${decision.reason}`);
         return decision.reason === "not_newer"
@@ -615,18 +686,37 @@ async function update(engine, options, now, fresh = false, onHostId) {
     const record = (outcome, detail, previous, unremoved = []) => {
         const journal = release.journalFormat === null ? null : { format: release.journalFormat, supervisor: target.supervisor };
         const pendingRemoval = [...new Set([...state.pendingRemoval, ...unremoved])].slice(-PENDING_REMOVAL_KEPT);
-        writeReleaseState(dir, { ...state, generation: release.generation, releaseId: release.releaseId, appliedAtMs: now(), previous, journal, pendingRemoval });
-        return { outcome, detail, generation: release.generation, exitCode: 0, ...tagged };
+        writeReleaseState(dir, { ...state, generation: release.generation, releaseId: release.releaseId, appliedAtMs: now(), previous, kept: null, journal, pendingRemoval });
+        return { outcome, detail, generation: release.generation, exitCode: 0, ...tagged, ...(notices.length > 0 ? { notices } : {}) };
     };
     if (options.check)
         return done("planned", changed.length > 0 ? `would_update:${changed.join(",")}` : "already_configured", { ...tagged, generation: release.generation });
     // `.env` naming the release proves nothing on its own: an earlier run may have stopped after writing
     // it. Only the running containers, ready, prove the release is applied.
-    if (changed.length === 0 && (await running(engine, dir, target, signal)))
-        return record("recorded", "already_running_release", state.previous);
+    if (changed.length === 0 && (await running(engine, dir, target, signal))) {
+        if (state.kept === null)
+            return record("recorded", "already_running_release", state.previous);
+        // CTC-8745: the stop lifted and the kept release came up ready, so what it replaced can go now.
+        const kept = existsSync(join(dir, PREVIOUS_ENV_FILE)) ? imagesOf(parseEnvText(readFileSync(join(dir, PREVIOUS_ENV_FILE), "utf8"))) : null;
+        const replaced = RELEASE_COMPONENTS.filter((c) => kept !== null && kept[c] !== "" && kept[c] !== target[c]).map((c) => kept[c]);
+        return record("recorded", "kept_through_preparation_stop", kept ?? state.previous, replaced.length > 0 ? await engine.removeImages(replaced, signal) : []);
+    }
+    // CTC-8745: a kept release has until its deadline, counted from its first wait. Past that it gets
+    // one restart and the normal deadline, and then goes back, whatever the stop says. A run no schedule
+    // follows has nothing to end a wait, so it goes back at the normal deadline.
+    const waitOver = options.scheduled !== true || (state.kept?.generation === release.generation && now() - state.kept.sinceMs >= PREPARATION_STOP_WAIT_MS);
+    const keepWaiting = () => {
+        if (state.kept?.generation !== release.generation)
+            writeReleaseState(dir, { ...state, kept: { generation: release.generation, sinceMs: now() } });
+        return done("retained", WAITING_PREPARATION_STOP, tagged);
+    };
+    // Starting a kept release again inside its wait would change nothing.
+    if (changed.length === 0 && !waitOver && (await inPreparationStop(engine, dir, target, signal)))
+        return keepWaiting();
     // What a failure goes back to: today's file, or for a half-applied file the one saved before it.
+    // CTC-8745: a kept release is unproven, so a newer offer goes back past it too.
     let rollbackText = envText;
-    if (changed.length === 0) {
+    if (changed.length === 0 || state.kept !== null) {
         try {
             rollbackText = readFileSync(join(dir, PREVIOUS_ENV_FILE), "utf8");
         }
@@ -686,26 +776,106 @@ async function update(engine, options, now, fresh = false, onHostId) {
             : deferBusy(dir, state, release, now(), busyAfterStop, done, tagged);
     }
     // From here the machine changes. The previous file is the rollback.
-    if (changed.length > 0)
+    if (changed.length > 0 && state.kept === null)
         writePrivateAtomic(join(dir, PREVIOUS_ENV_FILE), envText);
+    // The log setting goes in with the release and comes out with it: a rollback puts the file back as
+    // it was, so the run whose release stays adds the line and says so.
+    const composePath = join(dir, "compose.yaml");
+    let composeBefore = null;
+    try {
+        const text = readFileSync(composePath, "utf8");
+        const withSetting = withNodeLogSetting(text);
+        if (withSetting !== null) {
+            writeKeepingMode(composePath, withSetting);
+            composeBefore = text;
+        }
+    }
+    catch {
+        // The setting is not worth a failed update: with no line the machine ships nothing, and the next one tries again.
+    }
+    /** True when the line could not be taken out, so the rollback runs with it and must say so. */
+    const undoSetting = () => {
+        if (composeBefore === null)
+            return false;
+        try {
+            writeKeepingMode(composePath, composeBefore);
+            return false;
+        }
+        catch {
+            return true;
+        }
+    };
     writePrivateAtomic(envPath, rewriteEnvText(envText, Object.fromEntries(RELEASE_COMPONENTS.map((c) => [RELEASE_ENV_KEYS[c], target[c]]))));
     if ((await engine.composeUp(dir, signal)) && (await waitReady(engine, dir, target, options, now))) {
         // Only once the new release is running and ready: the replaced images free their disk (a runner
         // image is several GB, against a free-space floor that decides whether the machine takes work).
         // A rollback or failure never reaches here, so the previous images stay for it.
         // An image still in use, such as the runner image under phases left running, is kept for a later run.
-        const unremoved = changed.length > 0
-            ? await engine.removeImages(changed.map((c) => current[c]).filter((ref) => ref !== ""), signal)
-            : [];
+        // CTC-8745: so do the proven images a kept release left in place for its rollback.
+        const proven = state.kept === null ? null : rollbackImages;
+        const replaced = [...new Set([...changed.map((c) => current[c]), ...(proven === null ? [] : RELEASE_COMPONENTS.map((c) => proven[c]))])]
+            .filter((ref) => ref !== "" && !RELEASE_COMPONENTS.some((c) => target[c] === ref));
+        const unremoved = replaced.length > 0 ? await engine.removeImages(replaced, signal) : [];
         const applied = changed.length > 0 ? `updated:${changed.join(",")}` : "reconciled";
-        return record("updated", inPlace ? `${applied}:in_place` : applied, changed.length > 0 ? current : state.previous, unremoved);
+        if (composeBefore !== null)
+            notices.push(NODE_LOG_NOTICE);
+        return record("updated", inPlace ? `${applied}:in_place` : applied, proven ?? (changed.length > 0 ? current : state.previous), unremoved);
     }
+    // CTC-8745: not recorded as applied, and the previous images stay, until it comes up ready.
+    // The new supervisor is up with the line, so a release kept waiting keeps it and says so.
+    if (!waitOver && (await inPreparationStop(engine, dir, target, signal)))
+        return composeBefore === null ? keepWaiting() : { ...keepWaiting(), notices: [NODE_LOG_NOTICE] };
     if (rollbackText === null || rollbackImages === null)
-        return done("failed", "new_supervisor_not_ready_no_previous", { ...tagged, exitCode: 1 });
+        return done("failed", "new_supervisor_not_ready_no_previous", { ...tagged, exitCode: 1, ...(composeBefore === null ? {} : { notices: [NODE_LOG_NOTICE] }) });
     writePrivateAtomic(envPath, rollbackText);
+    const settingStays = undoSetting();
     const custodyRefused = await restoreCustody(rollbackImages);
-    const restored = (await engine.composeUp(dir, signal)) && (await waitReady(engine, dir, rollbackImages, options, now));
-    return done(restored && custodyRefused === null ? "rolled_back" : "failed", !restored ? "rollback_not_ready" : custodyRefused === null ? "new_supervisor_not_ready" : `custody_restore_failed:${custodyRefused}`, { ...tagged, exitCode: 1 });
+    // CTC-8745: a stop the release did not cause holds the release it went back to as well. That is a
+    // rollback that worked, not a failed one.
+    const restored = (await engine.composeUp(dir, signal))
+        && ((await waitReady(engine, dir, rollbackImages, options, now)) || (await inPreparationStop(engine, dir, rollbackImages, signal)));
+    writeReleaseState(dir, { ...state, kept: null });
+    return done(restored && custodyRefused === null ? "rolled_back" : "failed", !restored ? "rollback_not_ready" : custodyRefused === null ? "new_supervisor_not_ready" : `custody_restore_failed:${custodyRefused}`, { ...tagged, exitCode: 1, ...(settingStays ? { notices: [NODE_LOG_NOTICE] } : {}) });
+}
+/** CTC-8745: puts back the proven release a kept one replaced, with no release read. The phase
+ *  boundary and the outcome are the update's own: stop the supervisor, go back only if no phase is
+ *  running, and report it as the rollback it is. */
+async function returnFromKept(engine, options, state, hostId, now) {
+    const { dir, signal } = options;
+    const envPath = join(dir, ".env");
+    const result = (outcome, detail, exitCode) => ({ outcome, detail, generation: state.generation, exitCode });
+    let rollbackText;
+    try {
+        rollbackText = readFileSync(join(dir, PREVIOUS_ENV_FILE), "utf8");
+    }
+    catch {
+        return result("failed", "new_supervisor_not_ready_no_previous", 1);
+    }
+    const rollbackImages = imagesOf(parseEnvText(rollbackText));
+    const env = readEnvFile(envPath) ?? {};
+    const restoreCustody = async () => {
+        if (persistedCustody(dir, env) !== "installed")
+            return null;
+        const attested = await engine.custody(dir, rollbackImages, signal);
+        if ("refused" in attested)
+            return attested.refused;
+        return attested.authority === env.CATALYST_THOUGHTS_CUSTODY_AUTHORITY ? null : "authority_changed";
+    };
+    if (!(await engine.stopSupervisor(dir, signal))) {
+        await engine.composeUp(dir, signal);
+        return result("retained", "supervisor_stop_failed", 0);
+    }
+    const busy = await engine.attemptContainers(dir, hostId, signal);
+    if (busy !== 0) {
+        await engine.composeUp(dir, signal);
+        return result("retained", busy === null ? "docker_unavailable" : "phases_running", 0);
+    }
+    writePrivateAtomic(envPath, rollbackText);
+    const custodyRefused = await restoreCustody();
+    const restored = (await engine.composeUp(dir, signal))
+        && ((await waitReady(engine, dir, rollbackImages, options, now)) || (await inPreparationStop(engine, dir, rollbackImages, signal)));
+    writeReleaseState(dir, { ...state, kept: null });
+    return result(restored && custodyRefused === null ? "rolled_back" : "failed", !restored ? "rollback_not_ready" : custodyRefused === null ? "new_supervisor_not_ready" : `custody_restore_failed:${custodyRefused}`, 1);
 }
 async function waitReady(engine, dir, images, options, now) {
     const sleep = options.sleep ?? ((ms) => new Promise((resolve) => setTimeout(resolve, ms)));
@@ -853,6 +1023,20 @@ export function dockerRunnerUpdateEngine(deps = {}) {
             return { code: 1, stdout: "" };
         }
     };
+    /** The supervisor container's log since it last started, or null when that cannot be read. */
+    const logsSinceStart = async (dir, signal) => {
+        const compose = runnerComposeArgs(dir);
+        if (!compose)
+            return null;
+        const id = (await docker([...compose, "ps", "--quiet", "supervisor"], { cwd: dir, signal })).stdout.trim();
+        if (!/^[0-9a-f]{12,64}$/.test(id))
+            return null;
+        const started = (await docker(["inspect", "--format", "{{.State.StartedAt}}", id], { signal })).stdout.trim();
+        if (!Number.isFinite(Date.parse(started)))
+            return null;
+        const logs = await docker(["logs", "--since", started, id], { signal, timeoutMs: 30_000 });
+        return `${logs.stdout}\n${logs.stderr ?? ""}`;
+    };
     return {
         async desiredRelease(dir, signal, report) {
             const compose = runnerComposeArgs(dir);
@@ -943,17 +1127,12 @@ export function dockerRunnerUpdateEngine(deps = {}) {
             return kept;
         },
         async supervisorReadySinceStart(dir, signal) {
-            const compose = runnerComposeArgs(dir);
-            if (!compose)
-                return false;
-            const id = (await docker([...compose, "ps", "--quiet", "supervisor"], { cwd: dir, signal })).stdout.trim();
-            if (!/^[0-9a-f]{12,64}$/.test(id))
-                return false;
-            const started = (await docker(["inspect", "--format", "{{.State.StartedAt}}", id], { signal })).stdout.trim();
-            if (!Number.isFinite(Date.parse(started)))
-                return false;
-            const logs = await docker(["logs", "--since", started, id], { signal, timeoutMs: 30_000 });
-            return `${logs.stdout}\n${logs.stderr ?? ""}`.includes('"event":"host_ready"');
+            const logs = await logsSinceStart(dir, signal);
+            return logs !== null && logs.includes('"event":"host_ready"');
+        },
+        async supervisorInPreparationStop(dir, signal) {
+            const logs = await logsSinceStart(dir, signal);
+            return logs !== null && preparationStopOnly(logs);
         },
     };
 }
@@ -1000,8 +1179,11 @@ export const CUSTODY_UNATTESTED_LINE = "Private thoughts storage on this Mac is
     "If Catalyst had stopped sending this machine new phases for an update, it still sends none. To repair storage: catalyst onboard --runner";
 export function updateResultLines(result) {
     return [
-        OUTCOME_TEXT[result.outcome],
+        result.detail === WAITING_PREPARATION_STOP
+            ? "The new release is running. This machine stopped asking for phases after their preparation kept failing, so the update finishes once it asks again, or goes back to the previous release after 45 minutes."
+            : OUTCOME_TEXT[result.outcome],
         `Release: ${result.releaseId ?? "none"}, generation ${result.generation ?? "none"} (${result.detail}).`,
         ...(custodyUnattested(result) ? [CUSTODY_UNATTESTED_LINE] : []),
+        ...(result.notices ?? []),
     ];
 }

package/dist/runner.js

@@ -51,6 +51,8 @@ const spawnCommand = (file, args, options = {}) => new Promise((resolve) => {
         code: error ? (typeof error.code === "number" ? error.code : 127) : 0,
         stdout: String(stdout), stderr: String(stderr),
     }));
+    // As in spawnDocker: a broken pipe is not the verdict, the exit code is.
+    child.stdin?.on("error", () => { });
     child.stdin?.end(options.input ?? "");
 });
 function owned(path, uid, kind) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@catalyst-cloud/cli",
-  "version": "0.16.43",
+  "version": "0.16.44",
   "description": "Set up Catalyst Cloud, check your tickets and manage your account from the command line.",
   "license": "MIT",
   "type": "module",

package/skills/catalyst-github/SKILL.md

@@ -4,7 +4,7 @@ description: >-
   Catalyst's GitHub: show a ticket's pull request with its checks, reviews and review threads, say whether it is mergeable under this repository's policy, and explain what a PR accumulates as the ticket moves (the branch, the draft, the rewrite, the force-pushes, the labels, the queue). Use when someone asks "show me the PR", "what are the checks saying", "why hasn't it merged", "what does the queue need", "what does that label mean", or wants the branch and merge conventions Catalyst follows.
 allowed-tools: Bash(catalyst:*) Bash(npx -p @catalyst-cloud/cli catalyst:*)
 ---
-<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.43 — written in this repository for customer accounts -->
+<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.44 — written in this repository for customer accounts -->
 
 # Catalyst's GitHub
 

package/skills/catalyst-linear/SKILL.md

@@ -4,7 +4,7 @@ description: >-
   Catalyst's view of Linear on the customer's own cloud account. Reads a ticket with its comments, relations, labels, linked pull requests and agent sessions inline, from the cloud API by default, with local replica reads available as an explicit opt-in, always naming the source; searches tickets, PRs, projects and initiatives; writes comments, card moves, labels and new tickets through the account's agent proxy as the app actor, with every route, stage id, label id and marker read from the account's contract. Knows what a ticket accumulates as Catalyst works it (phase-outcome comments, the document per phase, the agent session, the labels, the single blocks relation, the bookkeeping marker, the eyes acknowledgement). Use when a person says "show me the ticket", "what did Catalyst write on it", "comment on it", "move it", "label it" or "file a ticket". Not for raising a decision for a human (what-needs-me) and not for pull requests (catalyst-github).
 allowed-tools: Bash(catalyst:*) Bash(npx -p @catalyst-cloud/cli catalyst:*)
 ---
-<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.43 — written in this repository for customer accounts -->
+<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.44 — written in this repository for customer accounts -->
 
 # Catalyst Linear
 

package/skills/catalyst-onboard/SKILL.md

@@ -4,7 +4,7 @@ description: >-
   Set up Catalyst Cloud and say whether setup is working. Walks a person from nothing to their first ticket running, one step at a time: the coding account, the project, the integrations and connected accounts, each repository's settings file, then one real ticket moving. Also logs this machine in, reads the readiness verdict with who can fix each failure, and checks the optional local replica. Use when someone says "set me up", "onboard me", "I just signed up", "get me started", "what do I do first", "log me in", "which account is this machine on", "am I set up", "what is missing", "why does nothing happen" or "is the replica running", when a login expired, or when a skill script exits 2 saying this machine is not connected. Does every step a key can do through the catalyst CLI, hands over the exact page for the steps only a browser can do, and never claims a step it did not watch succeed.
 allowed-tools: Bash(catalyst:*) Bash(npx -p @catalyst-cloud/cli catalyst:*)
 ---
-<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.43 — written in this repository for customer accounts -->
+<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.44 — written in this repository for customer accounts -->
 
 # Onboard me
 

package/skills/run-this-project/SKILL.md

@@ -4,7 +4,7 @@ description: >-
   Own one Catalyst Cloud project end to end until it closes. Use when the person says "run this project for me", "own this until it ships", "keep this moving", or hands you a project id or a set of tickets to drive. Subscribes to the account's event stream for the scope through the catalyst CLI, reacts to each change in the same turn, makes tickets ready and moves them to dispatch, parks what should stop, chases stalls, escalates inward, and keeps one status summary current. Writes to Linear as the app actor; never polls.
 allowed-tools: Bash(catalyst:*) Bash(npx -p @catalyst-cloud/cli catalyst:*)
 ---
-<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.43 — written in this repository for customer accounts -->
+<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.44 — written in this repository for customer accounts -->
 
 # Run this project
 

package/skills/unstick/SKILL.md

@@ -4,7 +4,7 @@ description: >-
   Get a stuck Catalyst Cloud ticket moving again. Use when the person asks "why is this parked and can you release it?", "unpark this", "get things flowing again", "the outage is over, retry what failed", or hands you a ticket that is not moving. Reads why nothing runs and what holds the ticket through the catalyst CLI, decides whether the recorded cause is fixed, previews the release, releases every governor holding the ticket the right way (or one failure class across a team), and raises an ask only for what a person has to do. Never releases a cause it cannot show changed without saying so.
 allowed-tools: Bash(catalyst:*) Bash(npx -p @catalyst-cloud/cli catalyst:*)
 ---
-<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.43 — written in this repository for customer accounts -->
+<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.44 — written in this repository for customer accounts -->
 
 # Unstick
 

package/skills/what-needs-me/SKILL.md

@@ -4,7 +4,7 @@ description: >-
   The human's decision inbox on Catalyst Cloud, and the one way an agent raises a decision on their behalf. Use when the person asks "what needs me?", "what am I blocking?", or when active work is gated on a choice only they can make. Lists open asks ranked by how much open work each one holds, files an ask through the cloud's ask route with the account's own template, and records the answer so the held work releases. Never answers as the human, never duplicates an open ask.
 allowed-tools: Bash(catalyst:*) Bash(npx -p @catalyst-cloud/cli catalyst:*)
 ---
-<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.43 — written in this repository for customer accounts -->
+<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.44 — written in this repository for customer accounts -->
 
 # What needs me
 

package/skills/what-this-repo-needs/SKILL.md

@@ -4,7 +4,7 @@ description: >-
   What environment variable names does this repository need, and where does each one come from? Scans the repository offline (no login, no network) and lists the names in three groups (build/test, deploy-only, bindings), each with where it was found, what uses it, and where a local value would come from. Never reads a value or prints one. Also checks the environment variable table in `.catalyst/catalyst.toml` offline. Use when someone asks "what does catalyst.env.json mean", "what env vars does this repo need", "why does the container need this", or before reviewing or writing the declaration.
 allowed-tools: Bash(catalyst:*) Bash(npx -p @catalyst-cloud/cli catalyst:*)
 ---
-<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.43 — written in this repository for customer accounts -->
+<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.44 — written in this repository for customer accounts -->
 
 # What this repository needs
 

package/skills/whats-happening/SKILL.md

@@ -4,7 +4,7 @@ description: >-
   Use for "what's happening?", "what machines do we have?", "how much capacity do we have?", "where are we?", "why is that stuck?", "what closed?", "what's next?", "how does this work?", "why did it do that?", "how does it prioritise?" and "what does this setting do, where is it set?" in Catalyst Cloud, or asks for something to be done rather than known. Reads the contract via catalyst CLI. Read-only: route work/decisions, never answer as the human.
 allowed-tools: Bash(catalyst:*) Bash(npx -p @catalyst-cloud/cli catalyst:*)
 ---
-<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.43 — written in this repository for customer accounts -->
+<!-- vendored-from: @catalyst-cloud/catalyst-skills@0.16.44 — written in this repository for customer accounts -->
 
 # What's happening