@celilo/cli 5.1.1 → 5.2.0

package/CELILO_SUBSYSTEMS.md

@@ -375,6 +375,7 @@ Run any celilo command on celilo-mgr over SSH instead of screen-scraping `ssh <h
 - **Client** — `packages/core/src/remote-client.ts` (`@celilo/core`) — `resolveRemote` (`--remote <dest>` / `CELILO_REMOTE`), `runRemoteClient` (`ssh -T`, renders progress via the local ProgressDisplay, answers interviews via the `@celilo/cli-display` prompts). Refuses to prompt on a non-TTY stdin, replying `unanswerable` rather than submitting a default as if a human had chosen it.
 - **Access control** — `apps/celilo/src/services/api-access.ts` — `grantPrincipal`, `isAuthorized` (deny-by-default, `command:subcommand` grants), `renderAuthorizedKeys`. Table: `api_principals` (`apps/celilo/src/db/schema.ts`). CLI: `apps/celilo/src/cli/commands/api.ts` (`api grant|list|revoke|authorized-keys|key new`).
 - **Principal enrolment for a module (`control_plane_api`)** — `apps/celilo/src/services/api-principal-enrolment.ts` — `enrolControlPlanePrincipal`, `revokeControlPlanePrincipal`, and `buildControlPlaneApi`, the method table a consuming module's hooks receive. The consumer generates an ed25519 pair on its own system and presents the public half; nothing here accepts a private key. Grants are DERIVED from `readOnlyGrants(COMMANDS)` and are not a parameter, so a caller cannot ask for more, and a write verb is never granted however it is named. **Framework-granted, so no module provides it** — enrolment writes celilo's own `api_principals` row and a module script may import nothing but `@celilo/capabilities`, which rules out celilo-mgmt as much as anyone else (`web-ui-console` D7b). Injected by `capability-loader.ts` ONLY for a module whose stored manifest declares it under `requires`/`optional`, unlike every other capability the loader hands out, and scoped to that module: a caller may not name a neighbour's principal. Contract: `packages/capabilities/src/control-plane-api.ts`.
+- **Parked-session registry** — `apps/celilo/src/api/sessions.ts` — the on-disk handoff that lets a command parked on an unanswerable question outlive the ssh session that started it (`api-sessions/<id>/session.json` + `output.ndjson`; each connection is its own process, so the registry cannot be in memory). `SessionWriter` owns a live session; `reapExpiredSessions` abandons parked sessions past their 30-minute TTL by ANSWERING the bus query `{abandoned}` rather than killing the child, which is what releases the module-operation lock on the way out. `pruneRetiredSessions` applies `SESSION_RETENTION_MS` (24h) to RETIRED records only — `running` and `parked` are left alone at any age, because a long command is not garbage and an old parked record can still be live. **The reap reads an INDEX of parked sessions (`api-parked/<id>`, one empty marker per parked session, maintained inside `write()` so every state transition already goes through it), NOT every record.** It is on the connection path on purpose — it is the only reaper that runs on a host with no dispatcher — and that is only defensible while it is O(parked): scanning every record there is what made connection cost track session count (celilo#1440's 94,706 records and 11.4s in front of a 240ms command; the shape itself is celilo#1454). The index is a cache, never the truth — `listParkedSessions` verifies each marker against its record and drops the ones that disagree, and an absent index is rebuilt from one scan. Retention pruning is O(every record) and nothing waits on it, so it belongs to `runApiSessionSweep` on the hourly `celilo-api-session-sweep` subscriber (`timer.tick.1h` → `celilo api sweep`, armed by `ensureApiSessionSweepSubscriber` from module registration and `celilo system migrate`, exactly like the backup and operations sweeps); `api-serve` keeps a stamp-gated fallback prune that runs AFTER a command is served, so a dispatcher-less host stays bounded without paying on connection setup. Gates: `sessions-reap-cost.test.ts` (the cost ratio across two magnitudes of backlog is what fails if O(records) work returns to the path), `sessions-retention.test.ts`, `sessions.test.ts`.
 - **Mid-run interview bridge (`kind:daemon` responder)** — `apps/celilo/src/services/remote-responder.ts` — `startRemoteResponder` bridges bus `interview.required.*` ↔ wire.
 - **Server provisioning** — the `celilo-bootstrap` deb (`packaging/celilo-bootstrap/scripts/postinst`) creates the non-root `celilo-api` landing account + sshd; membership in the `celilo` group + `/etc/sudoers.d/celilo` (`!use_pty`) gives api-serve DB access via the wrapper's sudo-drop.
 - **Self-upgrade (apt)** — `celilo apt-upgrade` (`apps/celilo/src/cli/commands/apt-upgrade.ts`) upgrades the deb-installed `celilo`/`celilo-bootstrap` packages (`apt-get update` → `--only-upgrade install`) then spawns a fresh `celilo system migrate` (ISS-0100), then `celilo events restart-daemon` so the dispatcher actually runs the code just installed — a failure there fails the whole command and names which steps DID complete, because "upgraded" while the dispatcher serves stale code is the silent state celilo#604 documents. It's the RW target behind the MCP's registry-derived `celilo_apt_upgrade` tool; the celilo user's two apt invocations are scoped-sudo'd by `/etc/sudoers.d/celilo-apt-upgrade`, shipped by `celilo-bootstrap`. **This upgrades celilo ITSELF — not the modules it manages. For those, see Module auto-upgrade below; the two are routinely confused.**

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@celilo/cli",
-  "version": "5.1.1",
+  "version": "5.2.0",
   "description": "Celilo — home lab orchestration CLI",
   "type": "module",
   "bin": {
@@ -61,7 +61,7 @@
     "@aws-sdk/lib-storage": "^3.1101.0",
     "@celilo/capabilities": "^6.1.0",
     "@celilo/cli-display": "^0.2.0",
-    "@celilo/core": "^0.14.0",
+    "@celilo/core": "^0.15.0",
     "@celilo/event-bus": "^0.7.0",
     "ajv": "^8.18.0",
     "drizzle-orm": "^0.36.4",

package/src/api/serve.ts

@@ -17,6 +17,8 @@
  * answers by event id; a later `attach` collects the outcome.
  */
 
+import { existsSync, statSync, writeFileSync } from 'node:fs';
+import { join } from 'node:path';
 import { createInterface } from 'node:readline';
 import {
   API_PROTOCOL_VERSION,
@@ -26,7 +28,7 @@ import {
   translateOutputLine,
 } from '@celilo/core';
 import { parseArguments } from '../cli/parser';
-import { getEventBusPath } from '../config/paths';
+import { getDataDir, getEventBusPath } from '../config/paths';
 import { getDb } from '../db/client';
 import { isAuthorized } from '../services/api-access';
 import { EVENT_TYPES } from '../services/bus-interview';
@@ -36,6 +38,7 @@ import {
   SessionWriter,
   abandonSession,
   expiryReason,
+  pruneRetiredSessions,
   readSession,
   reapExpiredSessions,
   replayOutput,
@@ -234,12 +237,61 @@ function handleCancel(principal: string, sessionId: string): void {
   send(resultMessage(true, 0));
 }
 
+/**
+ * How often a connection will pay for the retention prune when nothing else is
+ * pruning. Far finer than the 24h retention it enforces.
+ */
+const RETENTION_FALLBACK_INTERVAL_MS = 60 * 60 * 1000;
+
+function retentionStampPath(): string {
+  return join(getDataDir(), 'api-sessions-pruned-at');
+}
+
+/**
+ * The daemonless safety net for retention.
+ *
+ * The hourly `celilo-api-session-sweep` subscriber owns pruning now, but a host
+ * with no dispatcher running has no subscribers at all — and unbounded retention
+ * is precisely what celilo#1440 was: 94,706 records, 1.2 GB, every connection
+ * paying for all of it. Moving the prune to a timer without this would re-open
+ * that on exactly the hosts least likely to notice.
+ *
+ * Two things keep it off the critical path, and both matter:
+ *
+ *  1. It runs AFTER the command's result has been sent, not before the first
+ *     command is read. The caller is already served; this is teardown.
+ *  2. A stamp file gates it to once an hour, so the cost is not per-connection.
+ *     One `statSync` is what a connection pays in the common case.
+ *
+ * The stamp is written FIRST, so a prune that throws still backs off rather than
+ * having every subsequent connection retry a sweep that cannot work.
+ */
+function sweepRetentionIfDue(): void {
+  try {
+    const stamp = retentionStampPath();
+    const lastRun = existsSync(stamp) ? statSync(stamp).mtimeMs : 0;
+    if (Date.now() - lastRun < RETENTION_FALLBACK_INTERVAL_MS) return;
+    writeFileSync(stamp, '');
+    pruneRetiredSessions();
+  } catch {
+    // Housekeeping. Nothing about this is worth failing a served command over,
+    // and the hourly subscriber is the primary path regardless.
+  }
+}
+
 export async function apiServeMode(principal: string): Promise<void> {
   send({ type: 'ready', protocolVersion: API_PROTOCOL_VERSION });
 
   const busDbPath = getEventBusPath();
   // A session whose owning process died (reboot, OOM) would otherwise stay
-  // parked forever, holding whatever its command holds.
+  // parked forever, holding whatever its command holds. This is the only reaper
+  // that runs on a host with no dispatcher, which is why it is still here and
+  // not only on the hourly sweep.
+  //
+  // It reads the parked index rather than every session record, so its cost
+  // tracks parked sessions (usually none) and not how many have ever run — the
+  // property celilo#1454 is about. Nothing O(records) may go back in front of
+  // this line.
   reapExpiredSessions({ busDbPath });
 
   // When the ssh client goes away, sshd hangs up its forced command. Ignoring
@@ -405,5 +457,6 @@ export async function apiServeMode(principal: string): Promise<void> {
   // command finishes or the reaper abandons it.
   live.clientAttached = false;
   if (commandRunning) await commandRunning;
+  sweepRetentionIfDue();
   process.exit(0);
 }

package/src/api/sessions-reap-cost.test.ts

@@ -0,0 +1,218 @@
+/**
+ * Opening an API connection must not cost one read per session that has ever run.
+ *
+ * `api-serve` calls `reapExpiredSessions` after sending `ready` and before it
+ * will read a command, so whatever that call scans is charged to every caller of
+ * the remote API — the web console twice per panel. It used to scan every
+ * record: celilo#1440 was the extreme form (94,706 records, 11.4s before a
+ * 240ms command), and bounding retention to 24h fixed the magnitude while
+ * leaving the shape, which is celilo#1454.
+ *
+ * The reap now reads the parked INDEX instead, so its working set is the number
+ * of PARKED sessions (usually none) rather than the number of records.
+ *
+ * Two gates here, deliberately different in kind:
+ *
+ *   MECHANISM — the index holds one entry per parked session and nothing else,
+ *   so what the reap opens is bounded by parked count. Deterministic.
+ *
+ *   COST — the wall-clock of the reap across two magnitudes of backlog, which is
+ *   what the issue asks for and what fails if any O(records) work returns to
+ *   this path, by whatever route.
+ *
+ * WATCHED RED, and the result is worth recording because it says which of these
+ * tests is actually the gate. Restoring `listSessions()` as the reap's source —
+ * the pre-fix shape — fails the COST test at a ratio of 114.4 against a
+ * threshold of 10, and fails the daemonless test. It does NOT fail the mechanism
+ * test, which keeps passing: a full scan still finds the one parked session, so
+ * asserting on the reap's RESULT cannot see how much it read to get there. The
+ * mechanism test documents the index and guards its self-healing; the ratio is
+ * the only thing here that fails when the cost comes back.
+ *
+ * Restoring the `pruneRetiredSessions` call inside `reapExpiredSessions` fails
+ * 'the reap does not prune' at 0 records remaining against 20.
+ */
+
+import { afterEach, beforeEach, expect, test } from 'bun:test';
+import { mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import {
+  SESSION_RETENTION_MS,
+  type SessionState,
+  SessionWriter,
+  getParkedDir,
+  getSessionsDir,
+  listParkedSessions,
+  reapExpiredSessions,
+  runApiSessionSweep,
+} from './sessions';
+
+const NOW = 1_800_000_000_000;
+const OLD = NOW - SESSION_RETENTION_MS - 60_000;
+
+let dataDir: string;
+let busDbPath: string;
+let previous: string | undefined;
+
+/**
+ * Write a record the way a retired session leaves one behind: on disk, with no
+ * process attached. `expiresAt` in the past is what makes a parked one reapable.
+ */
+function seed(sessionId: string, state: SessionState, startedAt = OLD): void {
+  const dir = join(getSessionsDir(), sessionId);
+  mkdirSync(dir, { recursive: true });
+  writeFileSync(
+    join(dir, 'session.json'),
+    JSON.stringify({
+      sessionId,
+      principal: 'celilo-web-console',
+      argv: ['module', 'list'],
+      startedAt,
+      expiresAt: startedAt + 30 * 60 * 1000,
+      state,
+      parkedEventId: null,
+      parkedEventType: null,
+      question: null,
+      questionKey: null,
+    }),
+  );
+}
+
+/** A parked record AND its index entry — what `write()` produces together. */
+function seedParked(sessionId: string, startedAt = OLD): void {
+  seed(sessionId, 'parked', startedAt);
+  mkdirSync(getParkedDir(), { recursive: true });
+  writeFileSync(join(getParkedDir(), sessionId), '');
+}
+
+/** Wall-clock of the exact call `api-serve` makes before reading a command. */
+function timeReap(): number {
+  const started = performance.now();
+  reapExpiredSessions({ busDbPath, now: NOW });
+  return performance.now() - started;
+}
+
+beforeEach(() => {
+  previous = process.env.CELILO_DATA_DIR;
+  dataDir = mkdtempSync(join(tmpdir(), 'celilo-reap-cost-'));
+  process.env.CELILO_DATA_DIR = dataDir;
+  busDbPath = join(dataDir, 'bus.db');
+  mkdirSync(getSessionsDir(), { recursive: true });
+  mkdirSync(getParkedDir(), { recursive: true });
+});
+
+afterEach(() => {
+  if (previous === undefined) delete process.env.CELILO_DATA_DIR;
+  else process.env.CELILO_DATA_DIR = previous;
+  rmSync(dataDir, { recursive: true, force: true });
+});
+
+test('the reap opens the parked index, not every record', () => {
+  for (let i = 0; i < 4000; i++) seed(`retired-${i}`, 'finished');
+  seedParked('the-only-parked-one');
+
+  // The premise, stated so a reader can see the two numbers that matter.
+  expect(readdirSync(getSessionsDir())).toHaveLength(4001);
+  expect(readdirSync(getParkedDir())).toHaveLength(1);
+
+  // What the reap can even consider is the index, so 1 — not 4001.
+  expect(listParkedSessions().map((s) => s.sessionId)).toEqual(['the-only-parked-one']);
+  expect(reapExpiredSessions({ busDbPath, now: NOW }).map((r) => r.sessionId)).toEqual([
+    'the-only-parked-one',
+  ]);
+});
+
+test('the reap does not prune — retention is the hourly sweep, not the caller', () => {
+  for (let i = 0; i < 20; i++) seed(`retired-${i}`, 'finished');
+
+  reapExpiredSessions({ busDbPath, now: NOW });
+  // Still all there: nothing O(records) ran in front of the caller's command.
+  expect(readdirSync(getSessionsDir())).toHaveLength(20);
+
+  // The sweep is what reclaims them, and it is the same records.
+  expect(runApiSessionSweep({ busDbPath, now: NOW })).toEqual({ reaped: 0, pruned: 20 });
+  expect(readdirSync(getSessionsDir())).toEqual([]);
+});
+
+test('connection cost does not track the number of session records', () => {
+  const SMALL = 50;
+  const LARGE = 5000; // 100x
+
+  for (let i = 0; i < SMALL; i++) seed(`small-${i}`, 'finished');
+  timeReap(); // discard: first call pays for cold caches, not for N
+  const small = Math.min(timeReap(), timeReap(), timeReap());
+
+  for (let i = SMALL; i < LARGE; i++) seed(`large-${i}`, 'finished');
+  expect(readdirSync(getSessionsDir())).toHaveLength(LARGE);
+  const large = Math.min(timeReap(), timeReap(), timeReap());
+
+  // A floor, because at these speeds the measurement is mostly timer noise and a
+  // ratio against ~0ms means nothing.
+  const ratio = large / Math.max(small, 0.05);
+
+  // O(records) would put this near 100. Ten is a wide margin that still cannot
+  // be reached by anything proportional to a 100x change in N.
+  expect(ratio).toBeLessThan(10);
+});
+
+test('a parked session past its TTL is reaped with no dispatcher running', () => {
+  // The property the inline call buys and the whole reason it stays on the
+  // connection path: nothing else reaps on a host with no bus dispatcher. There
+  // is deliberately no subscriber, no responder and no daemon in this test.
+  seedParked('owner-died-in-a-reboot');
+  for (let i = 0; i < 500; i++) seed(`retired-${i}`, 'finished');
+
+  const reaped = reapExpiredSessions({ busDbPath, now: NOW });
+
+  expect(reaped.map((r) => r.sessionId)).toEqual(['owner-died-in-a-reboot']);
+  // Abandoned, and retained as history rather than deleted.
+  const record = JSON.parse(
+    readFileSync(join(getSessionsDir(), 'owner-died-in-a-reboot', 'session.json'), 'utf-8'),
+  );
+  expect(record.state).toBe('abandoned');
+  // And it leaves the index, so the next connection does not reconsider it.
+  expect(readdirSync(getParkedDir())).toEqual([]);
+});
+
+test('a marker whose session is gone is dropped, not paid for forever', () => {
+  // A prune removes the directory and the marker outlives it. The index is a
+  // cache, so it self-heals rather than making every future reap read a hole.
+  mkdirSync(getParkedDir(), { recursive: true });
+  writeFileSync(join(getParkedDir(), 'pruned-out-from-under-us'), '');
+
+  expect(listParkedSessions()).toEqual([]);
+  expect(readdirSync(getParkedDir())).toEqual([]);
+});
+
+test('an index that does not exist yet is built from one scan, then reused', () => {
+  // The upgrade path: a host coming into this code may already hold a parked
+  // session, and it must still be reaped. Pay for one full scan, once.
+  rmSync(getParkedDir(), { recursive: true, force: true });
+  seed('parked-before-the-upgrade', 'parked');
+  for (let i = 0; i < 100; i++) seed(`retired-${i}`, 'finished');
+
+  expect(listParkedSessions().map((s) => s.sessionId)).toEqual(['parked-before-the-upgrade']);
+  // The index now exists and holds exactly that session, so the scan was once.
+  expect(readdirSync(getParkedDir())).toEqual(['parked-before-the-upgrade']);
+});
+
+test('the index follows the real lifecycle, not a separate bookkeeping call', () => {
+  // The index is maintained inside `write()`, which every state transition
+  // already goes through. That is what makes it hard to drift: there is no
+  // second call for a future edit to forget.
+  const session = SessionWriter.create({ principal: 'tester', argv: ['module', 'list'] });
+  expect(readdirSync(getParkedDir())).toEqual([]);
+
+  session.park({ eventId: '1', eventType: 'interview.required.x.y', question: 'well?' });
+  expect(readdirSync(getParkedDir())).toEqual([session.id]);
+
+  session.unpark();
+  expect(readdirSync(getParkedDir())).toEqual([]);
+
+  session.park({ eventId: '2', eventType: 'interview.required.x.y', question: 'again?' });
+  expect(readdirSync(getParkedDir())).toEqual([session.id]);
+
+  session.finish('finished');
+  expect(readdirSync(getParkedDir())).toEqual([]);
+});

package/src/api/sessions-retention.test.ts

@@ -0,0 +1,120 @@
+/**
+ * Session records are retained as history on purpose. The bug was that the
+ * retention had no bound.
+ *
+ * Nothing in `sessions.ts` ever deleted a directory, so the set grew by one per
+ * API connection and never shrank — and `reapExpiredSessions` reads EVERY
+ * record, on the path `api-serve` takes before it will accept a command. So the
+ * cost of every session that had ever run was charged to every caller, on every
+ * connection. Measured on celilo-mgr 2026-09-27: 94,706 directories, 1.2 GB,
+ * and 11.4s between `ready` and the first command being read, for a command
+ * that then took 240ms. celilo#1440.
+ *
+ * WATCHED RED: with the `running`/`parked` guard removed, the live-session tests
+ * fail.
+ *
+ * The prune no longer runs from `reapExpiredSessions` — that coupling was the
+ * other half of the same defect, since it put O(every record) in front of every
+ * caller's first command. `runApiSessionSweep` owns it on an hourly subscriber
+ * now, and `sessions-reap-cost.test.ts` is what holds that line. celilo#1454.
+ */
+
+import { afterEach, beforeEach, expect, test } from 'bun:test';
+import { mkdirSync, mkdtempSync, readdirSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import {
+  SESSION_RETENTION_MS,
+  type SessionState,
+  listSessions,
+  pruneRetiredSessions,
+} from './sessions';
+
+const NOW = 1_800_000_000_000;
+const OLD = NOW - SESSION_RETENTION_MS - 60_000; // comfortably past the bound
+const RECENT = NOW - 60_000;
+
+let dataDir: string;
+let previous: string | undefined;
+
+function seed(sessionId: string, state: SessionState, startedAt: number): void {
+  const dir = join(dataDir, 'api-sessions', sessionId);
+  mkdirSync(dir, { recursive: true });
+  writeFileSync(
+    join(dir, 'session.json'),
+    JSON.stringify({
+      sessionId,
+      principal: 'celilo-web-console',
+      argv: ['api', 'list'],
+      startedAt,
+      expiresAt: startedAt + 30 * 60 * 1000,
+      state,
+      parkedEventId: null,
+      parkedEventType: null,
+      question: null,
+      questionKey: null,
+    }),
+  );
+}
+
+const remaining = () => readdirSync(join(dataDir, 'api-sessions')).sort();
+
+beforeEach(() => {
+  previous = process.env.CELILO_DATA_DIR;
+  dataDir = mkdtempSync(join(tmpdir(), 'celilo-sessions-'));
+  process.env.CELILO_DATA_DIR = dataDir;
+  mkdirSync(join(dataDir, 'api-sessions'), { recursive: true });
+});
+
+afterEach(() => {
+  if (previous === undefined) delete process.env.CELILO_DATA_DIR;
+  else process.env.CELILO_DATA_DIR = previous;
+  rmSync(dataDir, { recursive: true, force: true });
+});
+
+test('removes retired sessions past the retention bound', () => {
+  seed('old-finished', 'finished', OLD);
+  seed('old-abandoned', 'abandoned', OLD);
+
+  expect(pruneRetiredSessions({ now: NOW })).toBe(2);
+  expect(remaining()).toEqual([]);
+});
+
+test('keeps a retired session that is still inside the bound', () => {
+  seed('just-finished', 'finished', RECENT);
+
+  expect(pruneRetiredSessions({ now: NOW })).toBe(0);
+  expect(remaining()).toEqual(['just-finished']);
+});
+
+test('never removes a live session, however old it is', () => {
+  // A long command is not garbage, and an old `parked` record can still be
+  // live — abandonSession declines to abandon one whose question was answered.
+  seed('ancient-running', 'running', OLD);
+  seed('ancient-parked', 'parked', OLD);
+
+  expect(pruneRetiredSessions({ now: NOW })).toBe(0);
+  expect(remaining()).toEqual(['ancient-parked', 'ancient-running']);
+});
+
+test('removes a corrupt directory, which listSessions cannot even see', () => {
+  const dir = join(dataDir, 'api-sessions', 'corrupt');
+  mkdirSync(dir, { recursive: true });
+  writeFileSync(join(dir, 'session.json'), '{ not json');
+
+  // The premise: it is invisible to every other function here, so nothing else
+  // could ever reclaim it — which is why it is pruned on its directory mtime.
+  expect(listSessions().map((s) => s.sessionId)).not.toContain('corrupt');
+  expect(pruneRetiredSessions({ now: NOW, retentionMs: -1 })).toBe(1);
+  expect(remaining()).toEqual([]);
+});
+
+test('a backlog collapses to just the live sessions', () => {
+  for (let i = 0; i < 500; i++) seed(`retired-${i}`, 'finished', OLD);
+  seed('live', 'running', OLD);
+
+  expect(pruneRetiredSessions({ now: NOW })).toBe(500);
+  expect(remaining()).toEqual(['live']);
+  // The point of the bound: the next caller's scan is one record, not 501.
+  expect(listSessions()).toHaveLength(1);
+});

package/src/api/sessions.ts

@@ -25,6 +25,8 @@ import {
   mkdirSync,
   readFileSync,
   readdirSync,
+  rmSync,
+  statSync,
   writeFileSync,
 } from 'node:fs';
 import { join } from 'node:path';
@@ -32,6 +34,7 @@ import type { ServerMessage } from '@celilo/core';
 import { defineEvents, openBus } from '@celilo/event-bus';
 import { z } from 'zod';
 import { getDataDir } from '../config/paths';
+import type { SubscriberRegistrar } from '../services/module-operations';
 
 const NO_SCHEMAS = defineEvents({});
 
@@ -82,7 +85,21 @@ export function readSession(sessionId: string): SessionRecord | null {
   if (!existsSync(path)) return null;
   // Untrusted only in the sense of "written by another process" — validate it
   // rather than trusting the shape (Rule 3.7).
-  const parsed = SessionRecordSchema.safeParse(JSON.parse(readFileSync(path, 'utf-8')));
+  //
+  // The SHAPE was validated and the SYNTAX was not, so a record that is not
+  // JSON at all threw straight out of here. `listSessions` maps this over every
+  // directory, and `api-serve` calls that before it will accept a command, so
+  // one truncated write — a crash mid-write, a full disk — took down every API
+  // connection on the box, not just the session it belonged to. Unparseable and
+  // malformed are the same event to every caller here, and both already mean
+  // null.
+  let raw: unknown;
+  try {
+    raw = JSON.parse(readFileSync(path, 'utf-8'));
+  } catch {
+    return null;
+  }
+  const parsed = SessionRecordSchema.safeParse(raw);
   return parsed.success ? parsed.data : null;
 }
 
@@ -103,6 +120,94 @@ export function sessionParkedOn(eventId: string): SessionRecord | null {
 function write(record: SessionRecord): void {
   mkdirSync(sessionDir(record.sessionId), { recursive: true });
   writeFileSync(recordPath(record.sessionId), `${JSON.stringify(record, null, 2)}\n`);
+  syncParkedMarker(record);
+}
+
+/**
+ * The index of PARKED sessions: one empty file per parked session, named by its
+ * id, in a directory of its own.
+ *
+ * It exists so the inline reap at `api-serve` startup costs O(parked) instead of
+ * O(every record that has ever run). Reaping has to find parked sessions, and
+ * finding them by reading every `session.json` is what made connection setup
+ * scale with session count (celilo#1454; celilo#1440 was its extreme form, at
+ * 94,706 records and 11.4s per connection). Parked is a rare state — usually
+ * none — so a `readdir` of this directory is the O(1) the path should always
+ * have been.
+ *
+ * ponytail: a directory of empty marker files rather than one index file. A
+ * single JSON index needs a lock — every `api-serve` process is a separate
+ * writer — while create/unlink of a named file is atomic and needs none.
+ * Upgrade to an index only if the parked set ever gets big enough to readdir
+ * slowly, which would mean something else is very wrong.
+ *
+ * It is a CACHE, never the truth: `listParkedSessions` verifies every marker
+ * against the record and drops the ones that no longer agree, so a marker lost
+ * to a crash or left behind by a prune self-heals on the next pass.
+ *
+ * ⚠️ It lives BESIDE `api-sessions`, not inside it. `listSessions` and
+ * `pruneRetiredSessions` both `readdir` that directory and treat every entry as
+ * a session id, so a marker directory nested in there would read as a corrupt
+ * session — and the prune would delete the whole index on its own mtime.
+ */
+export function getParkedDir(): string {
+  return join(getDataDir(), 'api-parked');
+}
+
+function parkedMarkerPath(sessionId: string): string {
+  return join(getParkedDir(), sessionId);
+}
+
+function syncParkedMarker(record: SessionRecord): void {
+  if (record.state === 'parked') {
+    mkdirSync(getParkedDir(), { recursive: true });
+    // Deliberately unguarded. A parked session that fails to enter the index is
+    // one nothing will reap, which is the outage this index has to not cause —
+    // so it fails the park, loudly, exactly as a failed record write does.
+    writeFileSync(parkedMarkerPath(record.sessionId), '');
+    return;
+  }
+  try {
+    rmSync(parkedMarkerPath(record.sessionId), { force: true });
+  } catch {
+    // A marker we cannot remove is only a stale entry, and `listParkedSessions`
+    // drops those on sight. Nothing here is worth failing an unpark over.
+  }
+}
+
+/**
+ * Every session the index says is parked, confirmed against its record.
+ *
+ * A marker whose record is gone, unreadable, or no longer parked is stale — the
+ * session retired, or a prune removed it — so it is removed here rather than
+ * paid for on every future pass.
+ */
+export function listParkedSessions(): SessionRecord[] {
+  const markerDir = getParkedDir();
+  if (!existsSync(markerDir)) {
+    // No index yet: a host upgrading into this code may already hold parked
+    // sessions, and they must still be reaped. Pay for one full scan, build the
+    // index from it, and never scan again.
+    const parked = listSessions().filter((s) => s.state === 'parked');
+    mkdirSync(markerDir, { recursive: true });
+    for (const record of parked) writeFileSync(parkedMarkerPath(record.sessionId), '');
+    return parked;
+  }
+
+  const parked: SessionRecord[] = [];
+  for (const sessionId of readdirSync(markerDir)) {
+    const record = readSession(sessionId);
+    if (record?.state === 'parked') {
+      parked.push(record);
+      continue;
+    }
+    try {
+      rmSync(parkedMarkerPath(sessionId), { force: true });
+    } catch {
+      // Same as above: a stale marker costs one failed read on the next pass.
+    }
+  }
+  return parked;
 }
 
 /**
@@ -255,16 +360,93 @@ export function expiryReason(record: SessionRecord): string {
  * parked child on its own timer, and again at `api-serve` startup so a session
  * whose owner died (reboot, OOM) doesn't sit parked forever — the failure mode
  * that left a `module deploy` holding every backup lock on the fleet for 20 days.
+ *
+ * Reads the parked INDEX, not every record, so its cost tracks the number of
+ * parked sessions (usually none) rather than the number that have ever run. It
+ * stays on the connection path for the reason above — it is the only thing that
+ * reaps on a host with no dispatcher running — and staying there is only
+ * defensible because it is now O(parked). celilo#1454.
+ *
+ * It no longer runs the retention prune. Pruning is housekeeping, is O(every
+ * record) by nature, and nothing waits on it, so it belongs on the hourly sweep
+ * (`runApiSessionSweep`) rather than in front of a caller's first command.
  */
 export function reapExpiredSessions(opts: { busDbPath: string; now?: number }): SessionRecord[] {
   const now = opts.now ?? Date.now();
-  const expired = listSessions().filter((s) => s.state === 'parked' && s.expiresAt <= now);
+  const expired = listParkedSessions().filter((s) => s.expiresAt <= now);
   for (const record of expired) {
     abandonSession(record, { busDbPath: opts.busDbPath, reason: expiryReason(record) });
   }
   return expired;
 }
 
+/**
+ * How long a RETIRED session's directory is kept.
+ *
+ * A finished record is retained as history on purpose (see `finish`), but the
+ * retention was unbounded: nothing in this file ever deleted a directory, so
+ * the set grew by one per API connection and never shrank. That is expensive in
+ * a way nothing about it looks expensive, because `reapExpiredSessions` reads
+ * EVERY record, and `api-serve` calls it before it will accept a command — so
+ * every caller pays for every session that has ever run, on every connection.
+ *
+ * Measured on celilo-mgr, 2026-09-27: 94,706 directories, 1.2 GB, and an 11.4s
+ * gap between `api-serve` sending `ready` and reading its first command, on a
+ * box whose root is an SD card. The command itself took 240ms. The web console
+ * makes two such connections per panel, so every panel cost 23-36s against a
+ * shorter client timeout and read as "unavailable — Request Timeout".
+ * celilo#1440.
+ *
+ * A day is far past the 30-minute TTL, so nothing live can be caught by it.
+ */
+export const SESSION_RETENTION_MS = 24 * 60 * 60 * 1000;
+
+/** A directory's own mtime, for records too corrupt to carry their own. */
+function dirModifiedAt(sessionId: string): number | null {
+  try {
+    return statSync(sessionDir(sessionId)).mtimeMs;
+  } catch {
+    return null;
+  }
+}
+
+/**
+ * Remove the directories of sessions that retired longer than `retentionMs`
+ * ago, and return how many went.
+ *
+ * Only RETIRED sessions: `running` and `parked` are left alone at any age. A
+ * long-running command is not garbage, and a parked one is the reaper's
+ * business — `abandonSession` deliberately declines to abandon a session whose
+ * question has since been answered, so an old parked record can be a live one.
+ *
+ * A directory whose record will not parse is pruned on its own mtime. Such a
+ * directory is invisible to `listSessions`, so nothing else here can ever see
+ * it, and it would sit on disk forever while still costing every scan a
+ * `readdir` entry and a failed read.
+ */
+export function pruneRetiredSessions(opts: { now?: number; retentionMs?: number } = {}): number {
+  const dir = getSessionsDir();
+  if (!existsSync(dir)) return 0;
+  const cutoff = (opts.now ?? Date.now()) - (opts.retentionMs ?? SESSION_RETENTION_MS);
+
+  let removed = 0;
+  for (const sessionId of readdirSync(dir)) {
+    const record = readSession(sessionId);
+    if (record && (record.state === 'running' || record.state === 'parked')) continue;
+    const retiredAt = record ? record.startedAt : dirModifiedAt(sessionId);
+    if (retiredAt === null || retiredAt > cutoff) continue;
+    try {
+      rmSync(sessionDir(sessionId), { recursive: true, force: true });
+      removed++;
+    } catch {
+      // A directory we cannot remove (permissions, a racing writer) is not a
+      // reason to abandon the sweep: the next caller pays for it, and that is
+      // strictly better than leaving the rest of the backlog in place.
+    }
+  }
+  return removed;
+}
+
 /** Buffered messages an attaching client should be replayed. */
 export function replayOutput(
   sessionId: string,
@@ -276,3 +458,45 @@ export function replayOutput(
   const start = Math.max(fromLine, lines.length - REPLAY_LIMIT);
   return { messages: lines.slice(start), next: lines.length };
 }
+
+/**
+ * The hourly pass: reap what the TTL has expired, then apply retention.
+ *
+ * This is where the O(every record) prune lives now. It used to run in front of
+ * every API connection's first command, which is what made connection cost track
+ * session count (celilo#1454).
+ *
+ * Registered as an ordinary bus subscriber whose handler is an existing CLI
+ * command, exactly like `celilo-backup-sweep` and `celilo-operations-sweep` — no
+ * new scheduler. Hourly is far finer than the 24h retention, and the pass is
+ * idempotent: with nothing expired and nothing retired it is one `readdir` of an
+ * empty index plus one of the record set.
+ *
+ * The reap stays duplicated between here and `api-serve` startup on purpose.
+ * This subscriber is the efficient path; the inline call is the one that still
+ * works when no dispatcher is running, and a host in that state is exactly the
+ * host whose owning process just died.
+ */
+export function runApiSessionSweep(opts: {
+  busDbPath: string;
+  now?: number;
+  retentionMs?: number;
+}): { reaped: number; pruned: number } {
+  const now = opts.now ?? Date.now();
+  const reaped = reapExpiredSessions({ busDbPath: opts.busDbPath, now });
+  const pruned = pruneRetiredSessions({ now, retentionMs: opts.retentionMs });
+  return { reaped: reaped.length, pruned };
+}
+
+export const API_SESSION_SWEEP_SUBSCRIBER = 'celilo-api-session-sweep';
+export const API_SESSION_SWEEP_PATTERN = 'timer.tick.1h';
+
+/** Idempotent: `bus.subscribe` upserts by name. */
+export function ensureApiSessionSweepSubscriber(bus: SubscriberRegistrar): void {
+  bus.subscribe({
+    name: API_SESSION_SWEEP_SUBSCRIBER,
+    pattern: API_SESSION_SWEEP_PATTERN,
+    handler: 'celilo api sweep',
+    registeredBy: 'celilo-api-sessions',
+  });
+}

package/src/cli/commands/api.ts

@@ -245,3 +245,25 @@ export async function handleApiKeyNew(args: string[]): Promise<CommandResult> {
     return { success: false, error: `Failed to generate API key: ${errMsg(error)}` };
   }
 }
+
+/**
+ * `celilo api sweep` — the hourly session pass: reap expired parked sessions,
+ * then apply retention to retired ones.
+ *
+ * This is the handler behind the `celilo-api-session-sweep` subscriber. It holds
+ * the O(every record) prune that used to run in front of every API connection's
+ * first command (celilo#1454).
+ */
+export async function handleApiSweep(): Promise<CommandResult> {
+  try {
+    const { runApiSessionSweep } = await import('../../api/sessions');
+    const { getEventBusPath } = await import('../../config/paths');
+    const { reaped, pruned } = runApiSessionSweep({ busDbPath: getEventBusPath() });
+    return {
+      success: true,
+      message: `Session sweep: ${reaped} expired session(s) abandoned, ${pruned} retired record(s) removed`,
+    };
+  } catch (error) {
+    return { success: false, error: `Session sweep failed: ${errMsg(error)}` };
+  }
+}

package/src/cli/commands/system-migrate.ts

@@ -9,6 +9,7 @@
 
 import type { Database } from 'bun:sqlite';
 import { defineEvents, openBus } from '@celilo/event-bus';
+import { ensureApiSessionSweepSubscriber } from '../../api/sessions';
 import { getEventBusPath } from '../../config/paths';
 import { createDbClient, findMigrationsFolder, getDb } from '../../db/client';
 import { runMigrationsOn } from '../../db/migrate';
@@ -45,6 +46,11 @@ function ensureCoreSubscribers(): void {
       // pause exists only because staging leaked, and the reaper that stops it
       // leaking ships in this same binary.
       ensureBackupSweepSubscriber(bus);
+      // Core, not module-dependent: any fleet that answers the remote API at all
+      // accumulates session records. This is the hourly pass that retires them,
+      // and arming it here is what gets it onto fleets that upgrade without
+      // installing anything (celilo#1454).
+      ensureApiSessionSweepSubscriber(bus);
     } finally {
       bus.close();
     }

package/src/cli/completion.ts

@@ -409,7 +409,7 @@ export async function getCompletions(words: string[], current: number): Promise<
 
   // API subcommands
   if (command === 'api' && currentIndex === 1) {
-    const subcommands = ['grant', 'list', 'revoke', 'authorized-keys', 'key'];
+    const subcommands = ['grant', 'list', 'revoke', 'authorized-keys', 'key', 'sweep'];
     return filterSuggestions(subcommands, args[1] || '');
   }
 

package/src/cli/index.ts

@@ -19,6 +19,7 @@ import {
   handleApiKeyNew,
   handleApiList,
   handleApiRevoke,
+  handleApiSweep,
 } from './commands/api';
 import { handleAptUpgrade } from './commands/apt-upgrade';
 import { handleCapabilityInfo } from './commands/capability-info';
@@ -2176,6 +2177,7 @@ export async function runCli(argv: string[]): Promise<CommandResult> {
           '  celilo api revoke <principal>',
           '  celilo api authorized-keys',
           '  celilo api key new <name>',
+          '  celilo api sweep',
           '',
           'Grants are command:subcommand (module:deploy), command:* (service:*), or * (all).',
         ].join('\n'),
@@ -2208,6 +2210,10 @@ export async function runCli(argv: string[]): Promise<CommandResult> {
       return handleApiAuthorizedKeys();
     }
 
+    if (parsed.subcommand === 'sweep') {
+      return handleApiSweep();
+    }
+
     if (parsed.subcommand === 'key') {
       if (parsed.args[0] === 'new') {
         return handleApiKeyNew(parsed.args.slice(1));

package/src/services/module-subscriptions.test.ts

@@ -218,6 +218,29 @@ describe('register / unregister roundtrip', () => {
     }
   });
 
+  // Every module deploy goes through an API session, so the same argument
+  // applies: registering ANY module arms the pass that retires session records.
+  // Asserted as a ROW, because "we called ensure…" is not the same fact
+  // (celilo#1454).
+  it('arms the api-session sweep for any module', () => {
+    registerModuleSubscriptions(baseManifest({}), '/p');
+
+    const bus = openBus({ dbPath, events: defineEvents({}) });
+    try {
+      const row = bus.db
+        .query<{ pattern: string; handler: string }, []>(
+          "SELECT pattern, handler FROM subscribers WHERE name = 'celilo-api-session-sweep'",
+        )
+        .get();
+      expect(row).toEqual({
+        pattern: 'timer.tick.1h',
+        handler: 'celilo api sweep',
+      });
+    } finally {
+      bus.close();
+    }
+  });
+
   it('registers each subscription as a row, names scoped to module id', () => {
     const result = registerModuleSubscriptions(
       baseManifest({

package/src/services/module-subscriptions.ts

@@ -17,6 +17,7 @@
 import { join } from 'node:path';
 import { defineEvents, openBus } from '@celilo/event-bus';
 import { inArray } from 'drizzle-orm';
+import { ensureApiSessionSweepSubscriber } from '../api/sessions';
 import { getEventBusPath, getModuleStoragePath } from '../config/paths';
 import { getDb } from '../db/client';
 import { modules } from '../db/schema';
@@ -98,6 +99,11 @@ export function registerModuleSubscriptions(
     // reclaims abandoned rows. Idempotent.
     ensureOperationsSweepSubscriber(bus);
 
+    // Also unconditional, and for the same reason: every module deploy goes
+    // through an API session, so the first module on a fleet is what arms the
+    // pass that retires session records. Idempotent.
+    ensureApiSessionSweepSubscriber(bus);
+
     // A module that can be backed up is also what switches the scheduled
     // backup sweep on. Registering here rather than at system init means the
     // sweep appears the moment the fleet has something to back up, and — since