@celilo/cli 4.0.0 → 4.1.0

package/CELILO_SUBSYSTEMS.md

@@ -212,6 +212,7 @@ runner seam (`execRunner` real / `createMockRunner` for tests) lives in
   - **Inbound** (`reconcileAspectsForSystems`): every approved aspect applied to systems that have just come into existence. Called from `module-deploy.ts` between `waitForSSH` and `executeAnsible` (so a module's playbook and `on_install` see a correctly configured host) and from `machine-add.ts`. Eligibility is `applicable_zones` + approval; the aspect's `triggers` list is deliberately NOT consulted, so convergence is not opt-in per manifest. A failure here IS fatal to the deploy that created the system — the aspect is a prerequisite of that host — while a failure on `machine add` is only a warning. PAUSED providers are skipped, which is the escape hatch for a wedged aspect. Nothing is rolled back: rows, guest and IPAM allocation persist and a re-run converges on the same system. Fixes celilo#902, where a system provisioned after its provider deployed silently never received the aspect. See `openspec/changes/aspect-fanout-new-systems/`.
   - **Coverage verification** — `verifyAspectCoverage` + `celilo system doctor --deep [--fix]`. Answers "is any system missing an aspect its zone entitles it to" WITHOUT stored state: the entitled set is `planAspectFanOut` itself, and coverage is measured by evaluating the role against the host in Ansible check mode (`executeAnsible({check:true})` + `parseAnsibleRecap`). Three outcomes, not two — `changed=0,skipped=0` applied, `changed>0` missing, **`skipped>0` unknown**, because check mode SKIPS a task it cannot evaluate and a `command`/`shell` role would otherwise report clean having never run.
 - **In-flight operation lock** — `apps/celilo/src/services/module-operations.ts` — `startOperation`/`completeOperation`/`failOperation` record deploy/uninstall/backup/restore in `module_operations`; `refuseIfInFlight`/`checkInFlight` are what backup and restore consult. Deploy and uninstall REGISTER but never check: it is a one-way guard protecting backup/restore consistency, not a general mutex (`openspec/specs/management-server-backup/spec.md` "In-flight operation refusal"). A row stops holding the lock once it is GONE, STOPPED/zombie (`isPidRunnable`, `ps -o state=` — `kill(pid,0)` calls a Ctrl-Z'd process alive), or older than `OPERATION_TTL_MS` (2h). The TTL is not redundancy: a pid is a recycled number, and once the pid space wraps an old row names an unrelated healthy process. Operator surface: `celilo module operations [list|clear] [--abandoned] [--all]` (`apps/celilo/src/cli/commands/module-operations.ts`); `list` shows only what holds the lock, abandoned rows are summarised unless `--abandoned`. Abandoned rows are reclaimed hourly by the `celilo-operations-sweep` bus subscriber (`timer.tick.1h` → `celilo module operations clear`, armed by `ensureOperationsSweepSubscriber` from module registration and `celilo system migrate`). `clear` MARKS rows failed rather than deleting them, and that is load-bearing: the `abandoned_operations` audit reads exactly those released rows to notice one module's operation dying over and over.
+- **Management-server reboot** — `apps/celilo/src/services/system-reboot.ts`, surfaced as `celilo system reboot [--allow-reboot] [--verify]` (`apps/celilo/src/cli/commands/system-reboot.ts`) and reachable over MCP like any other registry leaf. Rebooting THIS box takes down the dispatcher, the MCP server and the remote API, so `requestReboot` refuses while `checkInFlight` reports a module operation running and names it, and consent follows the flag-or-interview rule (`--allow-reboot` wins, otherwise an event-bus question — never stdin). **Verification is a second call, by construction**: the process asking for the reboot dies in it, so no single invocation can both reboot the host and report that it came back. `requestReboot` records `state/reboot-requested.json` carrying the boot id at request time; `verifyReturn` (`--verify`) passes only once that boot id has CHANGED and the dispatcher heartbeat and installed CLI both answer, and names which of the three is missing when it has not. It refuses up front when the boot id is unreadable, because a reboot whose return could never be verified is the out-of-band `sudo reboot` this replaces. `judgeRebootPending` compares the running kernel against the newest installed one and reads Debian's `/var/run/reboot-required`; either tell is enough, neither readable is `unmeasured` rather than clean. Both facts surface as the `reboot_pending` audit category (`apps/celilo/src/services/audit/reboot-pending.ts`): a pending reboot is `drift`, an UNVERIFIED one is `blocked` — that is a control plane that may not have come back. celilo#1378.
 - **Module pause / unpause (control-plane quiescence)** — `apps/celilo/src/services/module-pause.ts` — pure `planPause`/`planUnpause` producing an ordered plan, `executePause`/`executeUnpause` performing it, plus `listPausedModules`/`pausedAmong`/`formatPausedDuration`/`describePausedModule` (the ONE place an age is formatted). CLI: `apps/celilo/src/cli/commands/module-pause.ts` (`celilo module pause|unpause <id> [--cascade] [--stop-infra] [--reason] [--dry-run] [--yes]`). Pausing takes a module out of the CONTROL plane — no dispatched events, no timer hooks, no health checks, alerts suppressed — while leaving the DATA plane running, because capability consumption is deploy-time: every consumer calls `firewall`/`dhcp_server` from `on_install` and nothing calls it while serving. Config, secrets, IPAM/VMID and placement are preserved; `on_uninstall` does NOT run. Quiescence is enforced in two places: pausing drops the module's bus subscriptions (`unregisterModuleSubscriptions`), and `run-named-hook.ts` refuses any non-lifecycle hook for a PAUSED module (`skippedPaused`), which catches the paths that skip the bus — `events resync-subscriptions`, a restore that starts events.db empty, aspect fan-out, public-web republish. `on_install`/`on_uninstall` are exempt by hook NAME (not a caller flag): unpause redeploys through `on_install`, and removing a paused provider needs `on_uninstall`. Unpause always REDEPLOYS (`deployModule`) — that is what rebinds a consumer to a replacement provider and recreates provider-local state from the consumers that own it — and re-registers subscriptions, which a plain deploy does not do. A failed unpause restores `PAUSED` rather than leaving the module live and mis-bound. Cascade order reuses `services/update/dep-graph.ts` unchanged (pause = consumers first, unpause = providers first) and is computed from the GRAPH, never from which modules are currently paused, so a cascade walks THROUGH already-done members and is resumable. See `openspec/specs/module-pause/spec.md`.
 - **Provider-removal guard** — `apps/celilo/src/services/remove-guard.ts` — `findRemovalBlockers`/`describeRemovalRefusal`, called from `apps/celilo/src/cli/commands/module-remove.ts`. A PAUSED module is not a dependent (unpause cannot return it to service without a redeploy, and a redeploy re-resolves capabilities), which is what makes a provider swap possible at all. A dependent is one declaring the capability under `requires` **or** `optional` — the same relation `dep-graph.ts` uses, so the guard and the cascade agree on the set; the guard previously read `requires` alone, which let a removal silently orphan `technitium`'s `optional` `dhcp_server`. Refusals name each blocker AND which declaration makes it one. It deliberately does NOT exempt a dependent because another provider of the same capability exists (celilo#683).
 - **Consumer-removal cleanup (every provider is told)** — `apps/celilo/src/services/consumer-cleanup.ts` — pure `planConsumerCleanup` + `loadConsumerCleanupPlan` + `runConsumerCleanup`, called from `apps/celilo/src/cli/commands/module-remove.ts` after `on_uninstall` and before `terraform destroy`. A capability is two-sided: the consumer asks, the provider mints something in ITS world (a caddy site block, a DNAT rule, an OIDC client at authentik, a registered CI runner), and removal only ever touched one side — the FK cascade dropped celilo's row, so the provider's next converge had no way to learn the thing existed. This dispatches the `on_consumer_removed` hook to every provider of every capability the departing module declared under `requires` OR `optional` (the same relation `remove-guard.ts` counts as a dependency edge), **once per provider** rather than once per capability, sorted by provider id. The hook receives one input, `consumer`, and NOTHING else: a provider that cannot answer "what do I hold for this module" without being told has a different defect — the consumer's id was never recorded at mint time. It replaces `services/web-route-cleanup.ts`, which did the same job for exactly one capability, by name, from core. Semantics that are easy to get backwards: **a failed withdrawal never blocks the removal** — the consumer goes and the failing PROVIDER is marked `ERROR` with the departing consumer named in `error_message` (surfaced as a `blocked` finding by `services/audit/undeployed-modules.ts`), because the hook is a full converge and after a failure the provider's state is unknown rather than "one thing missed". **Dispatch continues past a failure**, so one broken provider cannot leave the others holding state. A PAUSED provider is skipped with a warning naming what it keeps (`run-named-hook.ts` refuses non-lifecycle hooks on a paused module, and `on_consumer_removed` must NOT join `LIFECYCLE_HOOKS`), and a never-deployed one is skipped silently. Providers implementing it: caddy, caddy-internal, iptables, greenwave, axon, authentik, forgejo, generic-cpanel-hosting-provider. See `openspec/changes/consumer-removal-cleanup/`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@celilo/cli",
-  "version": "4.0.0",
+  "version": "4.1.0",
   "description": "Celilo — home lab orchestration CLI",
   "type": "module",
   "bin": {
@@ -35,6 +35,7 @@
     "dev": "bun run src/cli/index.ts",
     "test": "bun test --timeout 60000 src/",
     "test:integration": "bun test --timeout 60000 test-integration/",
+    "test:integration:pr": "bun test --timeout 60000 test-integration/alerting test-integration/ansible test-integration/api test-integration/cli test-integration/completion test-integration/deploy test-integration/golden test-integration/machine test-integration/module test-integration/publish test-integration/secrets test-integration/service test-integration/sizing test-integration/system test-integration/values test-integration/module-integrity-baseline.test.ts",
     "test:integration-slow": "bun test --timeout 300000 test-integration/",
     "test:completion": "bun test --timeout 60000 test-integration/completion/",
     "test:all": "bun test --timeout 60000 src/ test-integration/",
@@ -58,10 +59,10 @@
   "dependencies": {
     "@aws-sdk/client-s3": "^3.1109.0",
     "@aws-sdk/lib-storage": "^3.1101.0",
-    "@celilo/capabilities": "^5.1.0",
+    "@celilo/capabilities": "^5.1.1",
     "@celilo/cli-display": "^0.2.0",
-    "@celilo/core": "^0.13.0",
-    "@celilo/event-bus": "^0.6.0",
+    "@celilo/core": "^0.14.0",
+    "@celilo/event-bus": "^0.7.0",
     "ajv": "^8.18.0",
     "drizzle-orm": "^0.36.4",
     "ink": "^7.0.1",

package/src/cli/command-tree-parser.test.ts

@@ -162,6 +162,24 @@ Commands:
       expect(commands.filter((c: string) => c === 'module')).toHaveLength(1);
     });
 
+    test('treats wrapped description lines as continuations, not subcommands', () => {
+      // celilo#1409: a wrapped help description's continuation lines matched the
+      // subcommand regex, so prose words were reported as missing registry entries.
+      const helpText = `
+Commands:
+  reboot                   Reboot the system if an operation
+                           dies, and it
+                           came back
+  status                   Show system status
+`;
+
+      // biome-ignore lint/suspicious/noExplicitAny: accessing private method for testing
+      const parseMethod = (parser as any).parseHelpText.bind(parser);
+      const commands = parseMethod(helpText);
+
+      expect(commands).toEqual(['reboot', 'status']);
+    });
+
     test('handles hyphenated command names', () => {
       const helpText = `
 Commands:

package/src/cli/command-tree-parser.ts

@@ -100,12 +100,16 @@ export class CommandTreeParser {
 
     // Track whether we're in a Commands/Subcommands section
     let inCommandSection = false;
+    // Column where the last parsed entry's description starts; anything indented
+    // that far is wrapped prose, not a subcommand.
+    let descriptionColumn: number | null = null;
 
     for (const line of lines) {
       // Check if we're entering a Commands section
       // Match "  Commands:" or "│  Commands:"
       if (/^[│\s]*(Commands|Subcommands):\s*$/i.test(line)) {
         inCommandSection = true;
+        descriptionColumn = null;
         continue;
       }
 
@@ -134,8 +138,16 @@ export class CommandTreeParser {
       // "    command-name   Description"
       const match = line.match(/^[│\s]*([a-z][a-z0-9-]*)(?:\s{2,}|\s+[<a-z])/);
 
+      // A wrapped description's continuation lines are indented to the previous
+      // entry's description column and match the pattern above (celilo#1409).
+      const indent = line.search(/[^│\s]/);
+      if (descriptionColumn !== null && indent >= descriptionColumn) {
+        continue;
+      }
+
       if (match) {
         const cmdName = match[1];
+        descriptionColumn = match[0].length;
 
         // Skip examples (lines starting with "celilo")
         if (cmdName === 'celilo') {

package/src/cli/commands/api.ts

@@ -9,6 +9,7 @@ import { existsSync, mkdirSync, readFileSync } from 'node:fs';
 import { homedir } from 'node:os';
 import { dirname, join } from 'node:path';
 import {
+  getPrincipalByName,
   grantPrincipal,
   listPrincipals,
   renderAuthorizedKeys,
@@ -22,7 +23,25 @@ function errMsg(error: unknown): string {
   return error instanceof Error ? error.message : String(error);
 }
 
-/** `celilo api grant <principal> --key <pubkey|path> --can <grant[,grant...]>` */
+function grantList(value: unknown): string[] {
+  return typeof value === 'string'
+    ? value
+        .split(',')
+        .map((g) => g.trim())
+        .filter(Boolean)
+    : [];
+}
+
+/**
+ * `celilo api grant <principal> [--key <pubkey|path>] --can|--add|--remove <grant[,...]>`
+ *
+ * `--can` REPLACES the grant set; `--add`/`--remove` amend it. Replace-only was
+ * the whole surface, so adding one permission meant re-supplying the full list
+ * plus the key, and a typo in either silently narrowed the principal or changed
+ * its key (celilo#1404). So: `--key` is required only when CREATING a principal,
+ * and a `--can` that drops grants an existing principal holds is refused unless
+ * `--force` says the narrowing is intended.
+ */
 export async function handleApiGrant(
   args: string[],
   flags: Record<string, boolean | string> = {},
@@ -32,37 +51,71 @@ export async function handleApiGrant(
     if (!name) {
       return {
         success: false,
-        error: 'Usage: celilo api grant <principal> --key <pubkey|path> --can <grant[,grant...]>',
+        error:
+          'Usage: celilo api grant <principal> [--key <pubkey|path>] --can|--add|--remove <grant[,grant...]>',
       };
     }
 
-    const keyArg = typeof flags.key === 'string' ? flags.key : '';
-    if (!keyArg) {
+    const replace = grantList(flags.can);
+    const added = grantList(flags.add);
+    const removed = grantList(flags.remove);
+    if (replace.length > 0 && (added.length > 0 || removed.length > 0)) {
       return {
         success: false,
-        error: '--key <public-key-or-path> is required (e.g. ~/.ssh/id_ed25519.pub)',
+        error: '--can replaces the grant set; use it OR --add/--remove, not both.',
+      };
+    }
+    if (replace.length === 0 && added.length === 0 && removed.length === 0) {
+      return {
+        success: false,
+        error:
+          'One of --can <grants> (replace), --add <grants>, or --remove <grants> is required (e.g. --add module:journal).',
       };
     }
 
-    const canArg = typeof flags.can === 'string' ? flags.can : '';
-    if (!canArg) {
+    const existing = await getPrincipalByName(name);
+    if (!existing && (added.length > 0 || removed.length > 0)) {
       return {
         success: false,
-        error: '--can <grant[,grant...]> is required (e.g. --can module:deploy,service:*)',
+        error: `No API principal named "${name}" — --add/--remove amend an existing one. Create it with --key and --can.`,
       };
     }
 
-    const publicKey = existsSync(keyArg) ? readFileSync(keyArg, 'utf8').trim() : keyArg.trim();
-    const grants = canArg
-      .split(',')
-      .map((g) => g.trim())
-      .filter(Boolean);
+    const keyArg = typeof flags.key === 'string' ? flags.key : '';
+    if (!keyArg && !existing) {
+      return {
+        success: false,
+        error: '--key <public-key-or-path> is required (e.g. ~/.ssh/id_ed25519.pub)',
+      };
+    }
+    // Keep the enrolled key when none is supplied, so amending grants can never
+    // rotate the key by omission.
+    const publicKey = keyArg
+      ? existsSync(keyArg)
+        ? readFileSync(keyArg, 'utf8').trim()
+        : keyArg.trim()
+      : (existing?.publicKey ?? '');
+
+    const current = existing?.grants ?? [];
+    let grants: string[];
+    if (replace.length > 0) {
+      const dropped = current.filter((g) => !replace.includes(g));
+      if (dropped.length > 0 && flags.force !== true) {
+        return {
+          success: false,
+          error: `Refusing to narrow "${name}": --can would drop ${dropped.join(', ')}. Use --add/--remove to amend, or --force to replace anyway.`,
+        };
+      }
+      grants = replace;
+    } else {
+      grants = [...new Set([...current, ...added])].filter((g) => !removed.includes(g));
+    }
 
     const { principal, created } = await grantPrincipal({ name, publicKey, grants });
 
     return {
       success: true,
-      message: `${created ? 'Granted' : 'Updated'} API access for "${principal.name}" → ${grants.join(', ')}\n\nInstall the forced-command line on celilo-mgr with:\n  celilo api authorized-keys >> ~celilo-api/.ssh/authorized_keys`,
+      message: `${created ? 'Granted' : 'Updated'} API access for "${principal.name}" → ${grants.join(', ') || '(none)'}\n\nInstall the forced-command line on celilo-mgr with:\n  celilo api authorized-keys >> ~celilo-api/.ssh/authorized_keys`,
     };
   } catch (error) {
     return { success: false, error: `Failed to grant API access: ${errMsg(error)}` };

package/src/cli/commands/events.ts

@@ -6,7 +6,7 @@
  * second tool.
  *
  * Subcommands:
- *   status               bus.health() as JSON
+ *   status               bus.healthConfirmed() as JSON
  *   tail                 recent events
  *   list-subscribers     persistent subscribers
  *   list-pending         pending deliveries (subscriber fan-out — NOT questions)
@@ -80,7 +80,10 @@ function jsonResult(data: unknown): CommandResult {
 export async function handleEventsStatus(): Promise<CommandResult> {
   const bus = openCliBus();
   try {
-    return jsonResult(bus.health());
+    // Confirmed: the ansible task and the fleet check both read this verdict and
+    // act on it, so it must not say `wedged` about a dispatcher the reconcile path
+    // calls `lagging` (celilo#1399). One threshold, one answer.
+    return jsonResult(await bus.healthConfirmed());
   } finally {
     bus.close();
   }

package/src/cli/commands/machine-reclassify.ts

@@ -0,0 +1,79 @@
+/**
+ * Machine Reclassify Command
+ *
+ * Re-runs zone detection over a machine's STORED interface labels and reports
+ * old → new. Dry-run by default: an operator sees every relabel before any of
+ * them is written. `--apply` writes.
+ *
+ * See services/machine-interface-zones.ts for why stale labels happen.
+ */
+
+import { eq } from 'drizzle-orm';
+import { getDb } from '../../db/client';
+import { machines } from '../../db/schema';
+import { applyReclassification, diffInterfaceZones } from '../../services/machine-interface-zones';
+import { getMachineByHostname, getMachineByIp } from '../../services/machine-pool';
+import { detectZoneFromIp } from '../../services/zone-detector';
+import { celiloIntro, celiloOutro } from '../prompts';
+import type { CommandResult } from '../types';
+
+const USAGE =
+  'Usage:\n  celilo machine reclassify <hostname|ip>          # show what would change\n  celilo machine reclassify <hostname|ip> --apply  # write the new labels';
+
+export async function handleMachineReclassify(
+  args: string[],
+  flags: Record<string, boolean | string> = {},
+): Promise<CommandResult> {
+  try {
+    celiloIntro('Re-classify Machine Interfaces');
+
+    const identifier = args[0];
+    if (!identifier) {
+      return { success: false, error: `Machine hostname or IP is required\n\n${USAGE}` };
+    }
+
+    const isIp = /^\d+\.\d+\.\d+\.\d+$/.test(identifier);
+    const machine = isIp
+      ? await getMachineByIp(identifier)
+      : await getMachineByHostname(identifier);
+
+    if (!machine) {
+      return { success: false, error: `Machine not found: ${identifier}` };
+    }
+
+    const interfaces = machine.interfaces ?? [];
+    const changes = await diffInterfaceZones(interfaces, detectZoneFromIp);
+
+    if (changes.length === 0) {
+      celiloOutro(
+        `${machine.hostname}: all ${interfaces.length} interface zone label(s) already match the current zone model`,
+      );
+      return { success: true, message: `${machine.hostname}: no re-classification needed` };
+    }
+
+    const lines = changes.map((c) => `  ${c.name} (${c.ipAddress}): ${c.from} → ${c.to}`);
+    const summary = `${machine.hostname}: ${changes.length} interface(s) would be re-classified\n${lines.join('\n')}`;
+
+    if (!flags.apply) {
+      celiloOutro(`${summary}\n\nRe-run with --apply to write these labels.`);
+      return { success: true, message: summary };
+    }
+
+    getDb()
+      .update(machines)
+      .set({ interfaces: applyReclassification(interfaces, changes) })
+      .where(eq(machines.id, machine.id))
+      .run();
+
+    celiloOutro(`${summary}\n\nWritten.`);
+    return {
+      success: true,
+      message: `${machine.hostname}: re-classified ${changes.length} interface(s)`,
+    };
+  } catch (error) {
+    return {
+      success: false,
+      error: `Failed to re-classify machine interfaces: ${error instanceof Error ? error.message : String(error)}`,
+    };
+  }
+}

package/src/cli/commands/system-audit.ts

@@ -26,6 +26,7 @@ import { testProxmoxConnection } from '../../api-clients/proxmox';
 import { getDb } from '../../db/client';
 import {
   capabilitySecrets,
+  machines,
   moduleConfigs as moduleConfigsTbl,
   modules,
   secrets,
@@ -52,6 +53,7 @@ import {
 } from '../../services/audit/public-dns-source';
 import { makeJournalReader, readAppliedMigrations } from '../../services/audit/schema';
 import type { SecretCheckResult } from '../../services/audit/secrets-decryptable';
+import { collectServerBunPinDeps } from '../../services/audit/server-bun-pin';
 import type { ServiceCredentialsResult } from '../../services/audit/services-credentials';
 import type { ServiceReachableResult } from '../../services/audit/services-reachable';
 import type { TerraformPlanRunner } from '../../services/audit/terraform-plan';
@@ -63,6 +65,7 @@ import { collectJailExemptions } from '../../services/jail-exemptions';
 import { probeMachines } from '../../services/machine-probe';
 import { parseStoredConfigValue } from '../../services/module-config';
 import { createPublicDnsProbe, loadPublicDnsProbeSettings } from '../../services/public-dns-probe';
+import { collectRebootPendingDeps } from '../../services/system-reboot';
 import { buildTerraformEnvForModule } from '../../services/terraform-env';
 import { hasFlag } from '../parser';
 import type { CommandResult } from '../types';
@@ -368,6 +371,22 @@ async function buildAuditDeps(
     probeDiskUsage(db),
   ]);
 
+  // Stale interface zone labels. Reads only what is already stored — the
+  // detector matches an address against the declared subnets, so this costs
+  // no SSH and cannot be confused by an unreachable machine.
+  const { diffInterfaceZones } = await import('../../services/machine-interface-zones');
+  const { detectZoneFromIp } = await import('../../services/zone-detector');
+  const machineZoneLabels = await Promise.all(
+    db
+      .select()
+      .from(machines)
+      .all()
+      .map(async (m) => ({
+        hostname: m.hostname,
+        changes: await diffInterfaceZones(m.interfaces, detectZoneFromIp),
+      })),
+  );
+
   const migrationsFolder = findMigrationsFolderSafe();
 
   return {
@@ -388,6 +407,8 @@ async function buildAuditDeps(
       })),
     },
     browserPin: collectBrowserPinDeps(deployedModules),
+    serverBunPin: collectServerBunPinDeps(),
+    rebootPending: collectRebootPendingDeps(),
     terraformPlan: {
       modules: terraformModules,
       run: realTerraformPlan,
@@ -429,6 +450,7 @@ async function buildAuditDeps(
     secretsDecryptable: { results: secretResults },
     servicesReachable: { results: serviceReachableResults },
     machinesReachable: { results: machineReachableResults },
+    machineInterfaceZones: { machines: machineZoneLabels },
     // The only check here whose vantage point is OUTSIDE the fleet. Its
     // undetermined counters are read but not written from this path: an
     // operator-run audit is not a run of a schedule, so counting it towards

package/src/cli/commands/system-reboot.ts

@@ -0,0 +1,126 @@
+/**
+ * `celilo system reboot` — reboot the management server (celilo#1378).
+ *
+ * Thin adapter (Rule 10.5): parse flags, obtain consent, compose the real
+ * host's dependencies, call the service, format the result. Every judgement —
+ * the in-flight refusal, the boot-id bookkeeping, what counts as "came back" —
+ * lives in `services/system-reboot.ts` and is tested against a fake host.
+ *
+ * Consent follows the repo's flag-or-interview rule: `--allow-reboot` (or
+ * `--yes`) wins, otherwise the question goes on the event bus so a headless
+ * responder can answer it. No stdin prompt.
+ *
+ * `--verify` is the other half, and it exists because the process asking for a
+ * reboot dies in it: no single call can both reboot this host and report that
+ * it came back. See the service's header.
+ */
+
+import { spawnSync } from 'node:child_process';
+import { readFileSync } from 'node:fs';
+import { defineEvents, openBus } from '@celilo/event-bus';
+import { getEventBusPath } from '../../config/paths';
+import { askConfirm, withInterviewSession } from '../../services/bus-interview';
+import {
+  BOOT_ID_PATH,
+  collectRebootPendingDeps,
+  requestReboot,
+  verifyReturn,
+} from '../../services/system-reboot';
+import { hasFlag } from '../parser';
+import type { CommandResult } from '../types';
+
+/** The deb's wrapper — the same binary `apt-upgrade` re-execs for migrations. */
+const CELILO_BIN = '/usr/local/bin/celilo';
+
+/** This command publishes nothing; it only reads the bus's health. */
+const NO_SCHEMAS = defineEvents({});
+
+function readBootId(): string | null {
+  try {
+    const id = readFileSync(BOOT_ID_PATH, 'utf-8').trim();
+    return id.length > 0 ? id : null;
+  } catch {
+    return null;
+  }
+}
+
+/** Does the installed CLI answer? Its version, or null. */
+function installedCliVersion(): string | null {
+  try {
+    const { status, stdout } = spawnSync(CELILO_BIN, ['--version'], { encoding: 'utf-8' });
+    if (status !== 0) return null;
+    const version = (stdout ?? '').trim();
+    return version.length > 0 ? version : null;
+  } catch {
+    return null;
+  }
+}
+
+/**
+ * Is the dispatcher alive? Confirmed rather than sampled — a dispatcher
+ * starved on a box that just booted looks exactly like a dead one for one
+ * sample (celilo#1399).
+ */
+async function dispatcherHeartbeatLive(): Promise<boolean> {
+  let bus: ReturnType<typeof openBus> | undefined;
+  try {
+    bus = openBus({ dbPath: getEventBusPath(), events: NO_SCHEMAS });
+    const health = await bus.healthConfirmed();
+    return health.status !== 'no_dispatcher' && health.status !== 'wedged';
+  } catch {
+    return false;
+  } finally {
+    bus?.close?.();
+  }
+}
+
+export async function handleSystemReboot(
+  _args: string[],
+  flags: Record<string, string | boolean>,
+): Promise<CommandResult> {
+  if (hasFlag(flags, 'verify')) {
+    const result = await verifyReturn({
+      bootId: readBootId,
+      heartbeatLive: dispatcherHeartbeatLive,
+      cliVersion: installedCliVersion,
+    });
+    return result.success
+      ? { success: true, message: result.message ?? 'Reboot verified.' }
+      : { success: false, error: result.error ?? 'Reboot could not be verified.' };
+  }
+
+  // Report the condition before asking, so the operator is answering about
+  // the host in front of them rather than about the word "reboot".
+  const { verdict } = collectRebootPendingDeps();
+  const why = verdict.pending
+    ? verdict.reasons.join('; ')
+    : 'no pending kernel or deferred restart was detected';
+
+  const consented =
+    flags['allow-reboot'] === true ||
+    hasFlag(flags, 'yes') ||
+    (await withInterviewSession(() =>
+      askConfirm({
+        scope: 'system-reboot',
+        key: 'reboot',
+        message: `Reboot the management server? This takes down the dispatcher, the MCP server and the remote API. Condition: ${why}.`,
+        defaultValue: false,
+      }),
+    ));
+
+  if (!consented) {
+    return { success: false, error: 'Reboot not approved — re-run with --allow-reboot.' };
+  }
+
+  const result = requestReboot(
+    {
+      bootId: readBootId,
+      issueReboot: () => spawnSync('sudo', ['systemctl', 'reboot'], { stdio: 'inherit' }),
+    },
+    why,
+  );
+
+  return result.success
+    ? { success: true, message: result.message ?? 'Reboot issued.' }
+    : { success: false, error: result.error ?? 'Reboot refused.' };
+}

package/src/cli/commands/system-update.ts

@@ -31,12 +31,14 @@ import { collectBrowserPinDeps } from '../../services/audit/browser-pin';
 import { fetchLatestCliVersion } from '../../services/audit/cli-version';
 import { unusedPublicDnsProbe } from '../../services/audit/public-dns';
 import { makeJournalReader, readAppliedMigrations } from '../../services/audit/schema';
+import { collectServerBunPinDeps } from '../../services/audit/server-bun-pin';
 import { createModuleBackup, createSystemStateBackup } from '../../services/backup-create';
 import { BACKUP_SCHEDULE_CONFIG_KEY } from '../../services/backup-schedule';
 import { runAllHealthChecks, runModuleHealthCheck } from '../../services/health-runner';
 import { collectJailExemptions } from '../../services/jail-exemptions';
 import { configOverride, parseStoredConfigValue } from '../../services/module-config';
 import { deployModule } from '../../services/module-deploy';
+import { collectRebootPendingDeps } from '../../services/system-reboot';
 import { buildModuleGraph } from '../../services/update/dep-graph';
 import {
   type ModuleSnapshot,
@@ -549,6 +551,8 @@ export async function handleSystemUpdate(
       })),
     },
     browserPin: collectBrowserPinDeps(upgradableModules),
+    serverBunPin: collectServerBunPinDeps(),
+    rebootPending: collectRebootPendingDeps(),
     terraformPlan: {
       modules: upgradableModules.map((m) => ({
         id: m.id,
@@ -616,6 +620,7 @@ export async function handleSystemUpdate(
     secretsDecryptable: { results: [] },
     servicesReachable: { results: [] },
     machinesReachable: { results: [] },
+    machineInterfaceZones: { machines: [] },
     // Public reachability needs a network round trip per name; the update
     // flow's partial audit does no probing. `system audit` and the scheduled
     // monitor own this check.

package/src/cli/completion.ts

@@ -439,14 +439,14 @@ export async function getCompletions(words: string[], current: number): Promise<
   }
 
   if (command === 'machine' && currentIndex === 1) {
-    const subcommands = ['add', 'list', 'status', 'remove', 'earmark', 'detect'];
+    const subcommands = ['add', 'list', 'status', 'remove', 'earmark', 'reclassify', 'detect'];
     return filterSuggestions(subcommands, args[1] || '');
   }
 
   // Machine remove/earmark - complete with machine hostnames and IPs
   if (
     command === 'machine' &&
-    (args[1] === 'remove' || args[1] === 'earmark') &&
+    (args[1] === 'remove' || args[1] === 'earmark' || args[1] === 'reclassify') &&
     currentIndex === 2
   ) {
     const machineList = await listMachines();
@@ -704,6 +704,7 @@ export async function getCompletions(words: string[], current: number): Promise<
       'update',
       'doctor',
       'migrate',
+      'reboot',
     ];
     return filterSuggestions(subcommands, args[1] || '');
   }

package/src/cli/index.ts

@@ -65,6 +65,7 @@ import {
 import { handleMachineAdd } from './commands/machine-add';
 import { handleMachineEarmark } from './commands/machine-earmark';
 import { handleMachineList } from './commands/machine-list';
+import { handleMachineReclassify } from './commands/machine-reclassify';
 import { handleMachineRemove } from './commands/machine-remove';
 import { handleMachineStatus } from './commands/machine-status';
 import { moduleAudit } from './commands/module-audit';
@@ -132,6 +133,7 @@ import { handleSystemDoctor } from './commands/system-doctor';
 import { handleSystemEnsureFleetKey } from './commands/system-ensure-fleet-key';
 import { handleSystemInit } from './commands/system-init';
 import { handleSystemMigrate } from './commands/system-migrate';
+import { handleSystemReboot } from './commands/system-reboot';
 import { handleSystemSecretGet } from './commands/system-secret-get';
 import { handleSystemSecretSet } from './commands/system-secret-set';
 import { handleSystemUpdate } from './commands/system-update';
@@ -1097,6 +1099,10 @@ Examples:
   celilo machine earmark 10.0.10.10 caddy
   celilo machine earmark pi-01 --clear
 
+  # Re-classify stored interface zone labels after a zone-model change
+  celilo machine reclassify celilo-mgr
+  celilo machine reclassify celilo-mgr --apply
+
   # Remove machine (by hostname or IP)
   celilo machine remove pi-01
   celilo machine remove 10.0.10.10 --force
@@ -1214,6 +1220,11 @@ Subcommands:
 
   doctor [--fix]                       Diagnose system prerequisites and @celilo/* version drift
 
+  reboot [--allow-reboot] [--verify]   Reboot the management server. Refuses while a module
+                                       operation is in flight and names it. The process asking
+                                       dies in the reboot, so --verify is what confirms the host
+                                       came back: boot id changed, dispatcher live, CLI answering.
+
 Runbook:
   https://celilo.computer/docs/system-update
   (offline summary in apps/celilo/docs/INDEX.md)
@@ -2269,6 +2280,10 @@ export async function runCli(argv: string[]): Promise<CommandResult> {
       return handleMachineEarmark(parsed.args, parsed.flags);
     }
 
+    if (parsed.subcommand === 'reclassify') {
+      return handleMachineReclassify(parsed.args, parsed.flags);
+    }
+
     return {
       success: false,
       error: `Unknown machine subcommand: ${parsed.subcommand}\n\nRun "celilo machine --help" for usage`,
@@ -2369,6 +2384,10 @@ export async function runCli(argv: string[]): Promise<CommandResult> {
       return handleSystemMigrate(parsed.args, parsed.flags);
     }
 
+    if (parsed.subcommand === 'reboot') {
+      return handleSystemReboot(parsed.args, parsed.flags);
+    }
+
     return {
       success: false,
       error: `Unknown system subcommand: ${parsed.subcommand}\n\nRun "celilo system --help" for usage`,