@celilo/cli 0.22.0 → 0.24.0

package/CELILO_CORE_MODULES.md

@@ -21,13 +21,13 @@ Each entry: `module id` — what it is — **provides** / **requires** capabilit
 
 ## Network fabric (DNS / firewall / DHCP)
 
-- **axon** — Axon Networks Q1000K ISP router driver (Brightspeed-branded); port-forwarding + public-IP discovery + DHCP DNS via the TR-181 CGI API. **provides:** `firewall`, `dhcp_server`. Fork of **greenwave** — identical protocol, differing only in the vendor extension prefix (`X_AXON_` vs `X_GWS_`/`X_LANTIQ_COM_`). Pick by device: Q1000K → **axon**, C4000XG → **greenwave**.
+- **axon** — Axon Networks Q1000K ISP router driver (Brightspeed-branded); port-forwarding + public-IP discovery + DHCP DNS + DHCP address-pool bounds via the TR-181 CGI API. **provides:** `firewall`, `dhcp_server`. Fork of **greenwave** — identical protocol, differing only in the vendor extension prefix (`X_AXON_` vs `X_GWS_`/`X_LANTIQ_COM_`). Pick by device: Q1000K → **axon**, C4000XG → **greenwave**. Two device behaviours shape this driver and are not shared by **greenwave** (celilo#739): the router **regenerates** `Pool.1.DNSServers` from its own upstream resolver list every ~15 min and on any config commit, so `setDhcpDns` also writes `Device.DNS.X_AXON_CustomServer` — the *input* to that computation, which survives — and **technitium re-asserts** the pool value on `timer.tick.1m`. Optional `dhcp_pool_start`/`dhcp_pool_end` bound the leased range; set them below the addresses IPAM allocates, because **IPAM has no knowledge of the router's DHCP pool** and nothing otherwise stops the router leasing an address celilo already assigned.
 - **greenwave** — GreenWave C4000XG ISP router driver; port-forwarding + public-IP discovery via REST. **provides:** `firewall`, `dhcp_server`. Legacy device; new deployments on Axon hardware want **axon**.
 - **iptables** — iptables firewall + NAT; cross-VLAN port exposure with recursive upstream delegation. Converge model: `exposeService` registers into the shared-core port-forward registry, then renders the complete ruleset and applies it atomically via `iptables-restore` (default-DROP FORWARD + coarse zone-tier matrix; SSH-free). **provides:** `firewall`.
 - **knot-unbound-internal** — split-horizon internal DNS via Knot (authoritative) + Unbound (recursive); lightweight, plain apt, no .NET. **provides:** `dns_internal`. Ships a base-module-aspect (`modules/knot-unbound-internal/base-module-aspect/`).
 - **technitium** — internal split-horizon DNS resolver + authoritative server (web UI + HTTP API); heavier alternative to knot-unbound. **provides:** `dns_internal`. Ships a base-module-aspect (`modules/technitium/base-module-aspect/`).
 - **namecheap** — public DNS A-record management via Namecheap Dynamic DNS API (HTTP, no browser automation). A caller supplies a NAME and nothing else: the address is the source IP of celilo's own update, re-derived on every assert. Registering `<domain>` also claims `www.<domain>` and vice versa (best effort, reported back as `outputs.companion_fqdn` so the framework's `public_dns` check watches it — Namecheap answers `ErrCount 0` for `www` updates it does not apply). DDNS passwords are keyed by the **registrable domain**, never the FQDN. **provides:** `dns_registrar`.
-- **wireguard** — owns the admin WireGuard tunnel on the firewall host: interface, listen port, peers (as records), and client subnet are module config rather than hand-maintained state. Exposes the listen port and **registers the client subnet as a trusted source**, so VPN reach into the managed zones is in the firewall registry and every converge re-emits it. Records `network.control-plane-vpn.subnet`, which the internal resolver's split-horizon view also consumes. Adopts a running tunnel in place (existing key and peers retained; `wg syncconf`, never `wg-quick down`) because that tunnel is the operator's recovery path. **requires:** `firewall` (and the provider must support trusted-source registration — `iptables` does; the ISP-router drivers `greenwave` and `axon` do not).
+- **wireguard** — owns the admin WireGuard tunnel on the firewall host: interface, listen port, peers (as records), and client subnet are module config rather than hand-maintained state. Exposes the listen port and **registers the client subnet as a trusted source**, so VPN reach into the managed zones is in the firewall registry and every converge re-emits it. **Declares `network.control-plane-vpn.subnet` BEFORE it creates the interface** — the first side effect of `on_install`, before any command touches the box. The ordering is load-bearing, not tidiness: the firewall classifies every interface on its host and one no declaration accounts for is `alien`, so creating `wg0` first would leave a window in which a converge — a timer, the registry-poll CD, any unrelated module deploy — could shut the admin VPN down as an intruder's interface. The module declares the NETWORK, never the device: only the firewall decides what an interface is for. `health_check` asserts the declaration still holds, because the other checks all pass while it is missing. That same key is what the internal resolver's split-horizon view consumes. Adopts a running tunnel in place (existing key and peers retained; `wg syncconf`, never `wg-quick down`) because that tunnel is the operator's recovery path. **requires:** `firewall` (and the provider must support trusted-source registration — `iptables` does; the ISP-router drivers `greenwave` and `axon` do not).
 
 ## Public edge (ingress / identity)
 

package/CELILO_SUBSYSTEMS.md

@@ -25,7 +25,10 @@ see `openspec/specs/`. Companion doc: [CELILO_CORE_MODULES.md](./CELILO_CORE_MOD
 
 - **IPAM (IP/VMID allocation)** — `apps/celilo/src/ipam/allocator.ts` — `allocateIPFromSubnet`, `allocateVMID`, `reserveIP`/`unreserveIP`, `inferZoneFromIP`, `getAllocation`. Auto-wrapper: `apps/celilo/src/ipam/auto-allocator.ts` — `allocateForModule` / `deallocateForModule`.
 - **Infrastructure selection (container-service vs machine pool)** — `apps/celilo/src/services/machine-pool.ts` (`getMachineByHostname`, `addMachine`, `assignModuleToMachine`) and `apps/celilo/src/services/container-service.ts` (`getContainerServiceByName`, `addContainerService`, `verifyContainerService`). Provider API clients: `apps/celilo/src/api-clients/proxmox.ts`, `apps/celilo/src/api-clients/digitalocean.ts`.
-- **Zone detection / system config** — `apps/celilo/src/services/zone-detector.ts` — `detectZoneFromIp` reads `network.<zone>.subnet` from the `systemConfig` table.
+- **Zone detection / system config** — `apps/celilo/src/services/zone-detector.ts` — `detectZoneFromIp` reads `network.<zone>.subnet` from the `systemConfig` table and returns `NetworkZone | 'unknown'`. It answers CONTAINMENT ONLY. It used to return `'external'` on no-match, which conflated "no declared subnet contains this" with "the internet can route to this" — on a firewall with five RFC1918 legs that reported four of them as facing the internet. `'unknown'` is the honest answer; the caller resolves it (see `machine add`: publicly routable → `external`, otherwise fail asking for `--zone`). The subnet-backed zone list is derived from `NETWORK_ZONES` minus `external`, which has no subnet and must never be given one.
+- **Interface classification** — `packages/capabilities/src/interface-classification.ts` — THE shared classifier, used by the backend and every firewall provider module so the two cannot drift apart again. `isPubliclyRoutable(ip)` is a property of the address alone (false for RFC 1918, RFC 6598 carrier-grade NAT, loopback, link-local, multicast, reserved). `classifyInterfaces(interfaces, zones)` assigns each interface `zone → external → alien`, first match winning, where `external` is the RESIDUAL — routable and claimed by no declared zone — and is never subnet-matched. `externalEdge()` returns none/single/**ambiguous** rather than silently picking the first public address. `defaultRouteFinding()` enforces the invariant that the default route leaves through `internal` or `external`. **`subnetContains(cidr, ip)` lives here and is the ONLY implementation** — three existed and disagreed (the backend's mishandled `/0`); the other two are deleted, not aliased. Design: `openspec/changes/firewall-interface-classification/design.md`.
+- **Declared networks (classification input)** — `readDeclaredNetworks(db)` in `apps/celilo/src/hooks/capability-loader.ts` — every `network.<name>.subnet` in system config, which is what an interface is attributed against. Read from the CONFIG, not from `NETWORK_ZONES`: celilo declares networks that are not placement zones (`network.control-plane-vpn.subnet`, written by `wireguard` before it brings `wg0` up). Injected into the firewall capability as a LIVE reader (`declaredNetworks`), alongside the trusted-source store and for the same reason — a declaration written during a hook run must be visible to the converge that follows it in that same run, which a snapshot taken at capability-build time cannot be.
+- **Firewall interface audit** — `apps/celilo/src/services/audit/interface-classification.ts` — reports per-firewall classification in `celilo audit`: alien interfaces by name and address (drift), and the blocking findings a converge refuses on — a carrier-grade NAT leg, an ambiguous external edge, a default route on the wrong leg.
 - **Zone taxonomy (canonical list)** — `apps/celilo/src/db/schema.ts` — `NETWORK_ZONES` is the single array; `NetworkZone` is DERIVED from it. Never hand-maintain a second copy: a duplicate that dropped a member made zone validation return null and silently fall back to a wrong-but-valid zone.
 - **Control-plane network (`secure-mgmt`)** — `apps/celilo/src/hooks/capability-loader.ts` — `loadControlPlaneSubnet` returns the subnet of the zone `celilo-mgmt` is deployed in. `secure-mgmt` is a placement zone AND the control-plane tier, deliberately NOT in `ZONE_TIER_ORDER` (it is not part of the `dmz → app → secure` data-plane chain; it reaches every tier by trust). The firewall's `trustedSubnets` derives from this rather than assuming celilo-mgr sits on `internal`. Reported as an actionable gap by `checkControlPlaneNetwork` in `apps/celilo/src/services/fleet-checks.ts` when the management address matches no configured subnet.
 - **A deployed system's zone** — `apps/celilo/src/services/deployed-systems.ts` — for machine-pool deploys the zone recorded is the ZONE OF THE MACHINE, not `requires.system.zone` (which is only the minimum used to *select* a host, as with sizing). Three writers must agree: `recordDeployedSystemForModule`, `backfillModuleSystems`, and `apps/celilo/src/variables/context.ts` — the last runs latest and will overwrite the others.
@@ -170,8 +173,26 @@ runner seam (`execRunner` real / `createMockRunner` for tests) lives in
 - **DNS provider backfill** (re-emit registrations when a provider deploys) — `apps/celilo/src/services/dns-provider-backfill.ts` — `isDnsInternalProvider`, `backfillProviderDns`.
 - **Base-module aspects (fan-out across the fleet)** — `apps/celilo/src/services/aspect-runner.ts` — `planAspectFanOut`, `runAspectFanOut`, `maybeRunAspectForTrigger`. Aspect content lives in `modules/<m>/base-module-aspect/` (e.g. knot-unbound-internal, technitium).
 - **In-flight operation lock** — `apps/celilo/src/services/module-operations.ts` — `startOperation`/`completeOperation`/`failOperation` record deploy/uninstall/backup/restore in `module_operations`; `refuseIfInFlight`/`checkInFlight` are what backup and restore consult. Deploy and uninstall REGISTER but never check: it is a one-way guard protecting backup/restore consistency, not a general mutex (`openspec/specs/management-server-backup/spec.md` "In-flight operation refusal"). A row stops holding the lock once it is GONE, STOPPED/zombie (`isPidRunnable`, `ps -o state=` — `kill(pid,0)` calls a Ctrl-Z'd process alive), or older than `OPERATION_TTL_MS` (2h). The TTL is not redundancy: a pid is a recycled number, and once the pid space wraps an old row names an unrelated healthy process. Operator surface: `celilo module operations [list|clear] [--abandoned] [--all]` (`apps/celilo/src/cli/commands/module-operations.ts`); `list` shows only what holds the lock, abandoned rows are summarised unless `--abandoned`. Abandoned rows are reclaimed hourly by the `celilo-operations-sweep` bus subscriber (`timer.tick.1h` → `celilo module operations clear`, armed by `ensureOperationsSweepSubscriber` from module registration and `celilo system migrate`). `clear` MARKS rows failed rather than deleting them, and that is load-bearing: the `abandoned_operations` audit reads exactly those released rows to notice one module's operation dying over and over.
+- **Module pause / unpause (control-plane quiescence)** — `apps/celilo/src/services/module-pause.ts` — pure `planPause`/`planUnpause` producing an ordered plan, `executePause`/`executeUnpause` performing it, plus `listPausedModules`/`pausedAmong`/`formatPausedDuration`/`describePausedModule` (the ONE place an age is formatted). CLI: `apps/celilo/src/cli/commands/module-pause.ts` (`celilo module pause|unpause <id> [--cascade] [--stop-infra] [--reason] [--dry-run] [--yes]`). Pausing takes a module out of the CONTROL plane — no dispatched events, no timer hooks, no health checks, alerts suppressed — while leaving the DATA plane running, because capability consumption is deploy-time: every consumer calls `firewall`/`dhcp_server` from `on_install` and nothing calls it while serving. Config, secrets, IPAM/VMID and placement are preserved; `on_uninstall` does NOT run. Quiescence is enforced in two places: pausing drops the module's bus subscriptions (`unregisterModuleSubscriptions`), and `run-named-hook.ts` refuses any non-lifecycle hook for a PAUSED module (`skippedPaused`), which catches the paths that skip the bus — `events resync-subscriptions`, a restore that starts events.db empty, aspect fan-out, public-web republish. `on_install`/`on_uninstall` are exempt by hook NAME (not a caller flag): unpause redeploys through `on_install`, and removing a paused provider needs `on_uninstall`. Unpause always REDEPLOYS (`deployModule`) — that is what rebinds a consumer to a replacement provider and recreates provider-local state from the consumers that own it — and re-registers subscriptions, which a plain deploy does not do. A failed unpause restores `PAUSED` rather than leaving the module live and mis-bound. Cascade order reuses `services/update/dep-graph.ts` unchanged (pause = consumers first, unpause = providers first) and is computed from the GRAPH, never from which modules are currently paused, so a cascade walks THROUGH already-done members and is resumable. See `openspec/changes/module-pause-lifecycle/`.
+- **Provider-removal guard** — `apps/celilo/src/services/remove-guard.ts` — `findRemovalBlockers`/`describeRemovalRefusal`, called from `apps/celilo/src/cli/commands/module-remove.ts`. A PAUSED module is not a dependent (unpause cannot return it to service without a redeploy, and a redeploy re-resolves capabilities), which is what makes a provider swap possible at all. A dependent is one declaring the capability under `requires` **or** `optional` — the same relation `dep-graph.ts` uses, so the guard and the cascade agree on the set; the guard previously read `requires` alone, which let a removal silently orphan `technitium`'s `optional` `dhcp_server`. Refusals name each blocker AND which declaration makes it one. It deliberately does NOT exempt a dependent because another provider of the same capability exists (celilo#683).
 - **Backup artifact encryption** — `apps/celilo/src/services/backup-cipher.ts` — `encryptFileToFile`/`decryptFileToFile`, file-in/file-out and streamed, used by every backup writer (`backup-create.ts`) and reader (`backup-restore.ts`, `restore-from-file.ts`). Do NOT route artifacts through `secrets/encryption.ts`: that API is string-in/string-out for short DB values, and feeding it a tar cost base64 (1.33x) then hex (2x) then `JSON.stringify` — ~9x the artifact in memory, which OOM-killed forgejo's 774 MB backup, and a hard ~805 MB ceiling from the max string length that no amount of RAM raises. On-disk format is `magic "CELILOBK" (8) | version (1) | iv (16) | ciphertext | GCM tag (16)`; the tag is a trailer because it does not exist until the last byte is encrypted. `decryptFileToFile` still reads the pre-2026-07 JSON-envelope artifacts, discriminating on the magic bytes — the envelope's own `schemaVersion` cannot serve, as it lives inside the encrypted tar.
 
+### Paused modules are conspicuous
+
+A pause switches OFF the alerting that would otherwise report the module as
+down, so the paused-ness itself is the signal. Four surfaces, none optional:
+`celilo module list` / `celilo status` render `PAUSED (3d)` with the reason;
+`checkPausedModules` in `apps/celilo/src/services/fleet-checks.ts` makes ANY
+paused module a `system doctor` FAILURE (no threshold — a pause is a degraded
+state, and a threshold is just something to tune until the detector stops
+firing); `findSuppressor` in `apps/celilo/src/services/alerting/suppression.ts`
+attributes the module's suppressed alerts to the pause rather than dropping them
+anonymously; and `fleetWarnings()` in `apps/celilo/src/api/serve.ts` stamps a
+warning naming every paused module and its age onto EVERY management-API result
+(`ResultMessage.warnings`), including on commands unrelated to those modules.
+That last one is the important half: it does not depend on the operator choosing
+to look, which is how a forgotten pause actually gets found.
+
 ## Generation & templating
 
 - **Generator** — `apps/celilo/src/templates/generator.ts` — `generateTemplates` (orchestration), plus Terraform/Ansible file handling.
@@ -199,7 +220,10 @@ is currently wrong, and routes carry the message to a person's phone. Design:
 - **Alert identity** — `apps/celilo/src/services/alerting/keys.ts` — the key grammar (`module:<id>[/check:<name>]`, `builtin:<check>[/<kind>:<target>]`) that makes "the same problem" the same alert across runs. `moduleAlertKey`, `moduleCheckAlertKey`, `builtinAlertKey`, `parseAlertKey`.
 - **Reconciliation** — `apps/celilo/src/services/alerting/reconcile.ts` (`reconcile`) — a successful run's failing-key set is authoritative and resolution is by SET DIFFERENCE (absent ⇒ resolved). A run whose outcome is `error` resolves NOTHING and fires a module-level alert instead: the false-all-clear guard.
 - **Monitor execution** — `apps/celilo/src/services/alerting/run-monitor.ts` (`runOneMonitor`) + `sweep.ts` (`selectDueMonitors`) + `builtin-monitors.ts` / `health-coverage.ts` (the built-in checks and the "module with no health check" coverage check).
-- **Scheduled audit categories (`builtin_check` monitors)** — `apps/celilo/src/services/alerting/builtin-source.ts` — `SCHEDULABLE_BUILTIN_CHECKS` is the list of `celilo system audit` categories cheap enough to run every sweep: `machines_reachable`, `backups`, `disk_space`, `abandoned_operations`, and `public_dns` (`apps/celilo/src/services/audit/abandoned-operations.ts` — ≥3 abandonments of the same (module, operation) in 7d, the fingerprint of an operation being killed mid-flight). Everything else in the audit needs the whole world injected (proxmox, terraform, registry) and is not schedulable. Enable one with `celilo monitor add backups --interval 1h`. Targets are tab-completable — `completion.ts` reads `SCHEDULABLE_BUILTIN_CHECKS` directly rather than a hand-copied list, so a newly-schedulable check is completable immediately.
+- **Health-check cadence (one accessor)** — `apps/celilo/src/services/alerting/health-cadence.ts` — `effectiveHealthCheckCadence(manifest, override)`, `isScheduled`, `loadModuleHealthCadences`, `reconcileModuleWatchState`. The manifest's `hooks.health_check.interval` SUGGESTS; the operator's `health_check_interval` decides; both resolve at read time. `null` means nobody named a cadence (a coverage gap); `'manual'` means the operator opted out (a decision — raises no coverage finding, and resolves the module's live alerts on the way down, since nothing will report on them again).
+- **⚠️ `monitors.intervalMinutes` and `monitors.enabled` are `builtin_check`-only.** A `module_hook` row carries severity, escalation policy and `lastRunAt`; its cadence and whether it is watched resolve through the accessor above. The columns' meaning depending on `kind` is a named smell (design.md D8) — the alternatives are a cached resolved value that rots, or splitting the table, which needs a synthetic monitor identity for `alerts.monitorId`. Gates: `sweep-runner.test.ts` asserts a module row's stored values are NOT consulted; `cadence-migration.test.ts` asserts the same through `loadModuleHealthCadences`.
+- **Carrying an existing fleet over** — `apps/celilo/src/services/alerting/cadence-migration.ts` (`migrateMonitorCadences`), run from `celilo system migrate` (the `.deb` postinst runs it on every apt upgrade). A monitor row whose cadence diverges from its manifest gets that cadence written as an override; a disabled one gets `manual`. Not bookkeeping: without it the upgrade that ships read-time resolution silently reverts every hand-set cadence to the author's suggestion and resumes watching modules an operator deliberately disabled. Idempotent — writes only where no override exists.
+- **Scheduled audit categories (`builtin_check` monitors)** — `apps/celilo/src/services/alerting/builtin-source.ts` — `SCHEDULABLE_BUILTIN_CHECKS` is the list of `celilo system audit` categories cheap enough to run every sweep: `machines_reachable`, `backups`, `disk_space`, `abandoned_operations`, and `public_dns` (`apps/celilo/src/services/audit/abandoned-operations.ts` — ≥3 abandonments of the same (module, operation) in 7d, the fingerprint of an operation being killed mid-flight). Everything else in the audit needs the whole world injected (proxmox, terraform, registry) and is not schedulable. Enable one with `celilo monitor add backups --interval 1h`, and re-cadence it later with `celilo monitor set-interval backups 6h` (in place, because the monitor id owns the alert history). `monitor set-interval`/`enable`/`disable` REFUSE a module target and name `celilo module config set <m> health_check_interval` — two ways to set one module's cadence would disagree about what `module status` shows. Targets are tab-completable — `completion.ts` reads `SCHEDULABLE_BUILTIN_CHECKS` directly rather than a hand-copied list, so a newly-schedulable check is completable immediately.
 - **Disk-space check** — `apps/celilo/src/services/audit/disk-space.ts` (`auditDiskSpace`, pure over measurements) + `apps/celilo/src/services/disk-probe.ts` (`probeDiskUsage`). Thresholds: `drift` at 85%, `blocked` at 95% — early enough to act on, since a check that fires at exhaustion reports an outage rather than preventing one. ⚠️ **The local management box is MEASURED, not exempted.** `probeMachines()` deliberately reports the local box reachable without probing it (celilo has no SSH key for itself, and the question is meaningless there); copying that shortcut into a disk check would skip the host most likely to fill — the one that stages backups, caches modules and writes the logs, and the one that DID fill. Local reads `statfs`; remote runs `df -P /` over the same bounded SSH. `percentUsed` matches `df`'s capacity semantics (excludes root-reserved blocks) so an alert and an operator's own `df` agree. An unmeasurable host yields a `todo` finding — recorded, never paged, because `machines_reachable` is already paging for that host. Findings are subjected on the **hostname**, not the machine UUID, because suppression resolves a machine's ancestor key from the hostname (see #596, where `machines_reachable` gets this wrong and its alerts therefore never suppress anything). The `backups` roster comes from `apps/celilo/src/services/audit/backup-source.ts` (`loadBackupAuditInfo`), shared with `celilo system audit` so both judge the same fleet.
 - **Public-DNS reachability check (the only check with an OFF-FLEET vantage)** — `apps/celilo/src/services/audit/public-dns.ts` (`auditPublicDns`, pure over an injected probe and the previous run's counters) + `apps/celilo/src/services/public-dns-probe.ts` (the probe) + `audit/public-dns-source.ts` (ledger names + the `public_dns_evidence` counters). Every other check in celilo looks from INSIDE, behind a split-horizon resolver that deliberately answers with an in-zone address — correct for its purpose, and why all of them reported healthy for the nine days of celilo#626. This one resolves every `dns_registrations` FQDN through an **off-fleet resolver** (`public_dns.resolver`, default `1.1.1.1`) and compares it against the address the fleet appears to come from per an independent **echo service** (`public_dns.echo_url`, default `https://api.ipify.org`). Three properties are load-bearing: `assertOffFleetResolver` REFUSES a resolver matching `dns.primary`/`dns.fallback` (a check that quietly used the fleet's resolver would pass forever — the original bug one layer up); the expectation never comes from the registrar's own response (self-agreement, and Namecheap returns `ErrCount 0` for `www` updates it does not apply); and a divergence is a finding only once it OUTLIVES the record's own TTL, measured from the last assert, or it would page on every ISP re-lease. Missing evidence is counted rather than read as success — one undetermined run is silent, N consecutive ones are their own finding (`public_dns_evidence`), which is the hole celilo-website's isitup.org probe demonstrated live. Codes: `public_dns_stale`, `public_dns_missing`, `public_dns_companion_unclaimed`, `public_dns_unverifiable`. Spec: `openspec/specs/public-dns-reachability/spec.md`.
 - **The sweep** — `apps/celilo/src/services/alerting/sweep-runner.ts` (`runSweep`) — the ordered pass that makes alerting run by itself: run due monitors → promote past-grace alerts → re-evaluate suppression → flush quiet-hours deferrals → notify. Driven by `celilo alerts sweep` on `timer.tick.5m`. Never throws for one bad monitor.
@@ -216,18 +240,32 @@ is currently wrong, and routes carry the message to a person's phone. Design:
 
 ## Backups
 
-Creation, scheduling and freshness. A module declares an `on_backup` hook and a
-`backup.schedule`; celilo runs it on that cadence and alerts when it stops.
+Creation, scheduling and freshness. A module declares an `on_backup` hook and
+SUGGESTS a `backup.schedule`; the operator's override decides; celilo runs it on
+the resolved cadence and alerts when it stops.
 
-- **Creation** — `apps/celilo/src/services/backup-create.ts` — `createModuleBackup` (invokes the module's `on_backup` hook into an encrypted envelope), `createSystemStateBackup` (celilo.db), `findBackupEligibleModules`, `isBackupDue`. Storage destinations: `backup-storage.ts`. Retention: `backup-retention.ts` (`pruneBackupsForModule`). Restore: `backup-restore.ts`.
-- **Cadence (one accessor)** — `apps/celilo/src/services/backup-schedule.ts` — `effectiveBackupSchedule(manifest)`. An absent `backup.schedule` means `daily`, NOT `manual`; opting out takes an explicit `manual`. Both the freshness audit and the backup sweep must read cadence through this one function, or a module can be alerted-on but never backed up.
+- **Creation** — `apps/celilo/src/services/backup-create.ts` — `createModuleBackup` (invokes the module's `on_backup` hook into an encrypted envelope), `createSystemStateBackup` (celilo.db), `findBackupEligibleModules` (returns each module's operator config alongside its manifest, because every caller has to resolve a policy out of the two together), `isBackupDue`. Storage destinations: `backup-storage.ts`. Restore: `backup-restore.ts`.
+- **Cadence (one accessor)** — `apps/celilo/src/services/backup-schedule.ts` — `effectiveBackupSchedule(manifest, override)`. The manifest SUGGESTS; the operator's `backup_schedule` row in `module_configs` decides; absent from both means `daily`, NOT `manual` (opting out takes an explicit `manual`). Resolution happens at READ time — nothing is materialised at install or deploy — so a corrected manifest reaches every install that has not overridden. The one-argument form was DELETED rather than kept as an overload (Rule 3.9): it would let an un-updated reader compile clean while silently ignoring overrides. Both the freshness audit and the backup sweep must read cadence through this one function, or a module can be alerted-on but never backed up.
+- **The cadence type** — `apps/celilo/src/services/cadence.ts` — `Cadence` (`{minutes}` | `'manual'`), `parseCadence` / `formatCadence` / `cadenceMs`, and `cadenceSchema({floorMinutes})`. One spelling set for every cadence in celilo: a named period (`hourly`/`daily`/`weekly`/`monthly`), a duration (`6h`, `90m`, `3d`), or `manual`. ⚠️ **The floors are DERIVED from the sweep ticks, never written down**: this file owns `BACKUP_SWEEP_PATTERN` / `ALERTING_SWEEP_PATTERN` and computes `BACKUP_CADENCE_FLOOR_MINUTES` / `MONITOR_INTERVAL_FLOOR_MINUTES` from them (the sweeps import their pattern from here), so changing a tick moves what it can serve in the same edit. A cadence finer than its sweep's tick is REFUSED, not coerced — accepting it leaves the operator believing they configured something that can silently never happen.
+- **Retention (one accessor)** — `apps/celilo/src/services/backup-retention.ts` — `effectiveBackupRetention(manifest, configs)`, `prunesNothing`, `identifyExpiredBackups`, `pruneBackupsForModule`. Two INDEPENDENT dimensions (copies, age), each resolved override → manifest → **unbounded**. ⚠️ **An unset dimension is unbounded, never a default bound.** `backup.retention` is an optional block, so a manifest omitting it prunes nothing at all and its inner `count: 7` / `max_age_days: 30` defaults never apply; if setting one dimension let the other fall back to those, an operator asking to keep 3 copies would silently arm a 30-day deletion on a module that had been keeping everything. Unbounded is `Infinity`, which `identifyExpiredBackups` needs no special case for. Gate: `services/backup-retention.test.ts`. The four sites that used to read `manifest.backup.retention` directly (`cli/commands/backup-sweep.ts`, `backup-create.ts`, `backup-prune.ts` twice) all go through the accessor — that duplication is what let the schedule readers drift.
 - **The sweep** — `apps/celilo/src/services/backup-sweep.ts` — `runBackupSweep` (the pass that makes backups run by themselves: reclaim orphaned staging → for each eligible module, is its declared cadence due → back it up → apply declared retention) + `ensureBackupSweepSubscriber`. Driven by `celilo backup sweep` on `timer.tick.1h` — the coarsest tick that can still serve an `hourly` cadence. Armed from BOTH `registerModuleSubscriptions` (any module declaring an `on_backup` hook, so it appears on install or `module update`) AND `celilo system migrate`, which the `.deb` postinst runs on every apt upgrade — registering only from module install/update meant a corrected budget never reached an existing fleet, since the row already exists and module updates can be weeks apart. A run refused by the in-flight operation lock is a skip retried next tick, never a failure.
 - **⚠️ The sweep's budget is stated, never inherited** — `BACKUP_SWEEP_TIMEOUT_MS` (4h) and `BACKUP_SWEEP_MAX_ATTEMPTS` (1) in `backup-sweep.ts`. The event bus defaults to `timeout_ms: 60000` / `max_attempts: 3`, and inheriting them made scheduled backups structurally impossible: one forgejo backup measured ~5.5 minutes (1.3 GB result, 3.9 GB peak staging), so the dispatcher SIGTERMed it at 60s — three times an hour, for days. The retry count is half the bug, not a detail: an impossible pass retried 3x strands 3x the staging (27 GB in 5.7 hours on celilo-mgr). The hourly tick IS the retry.
 - **Staging reclamation** — `apps/celilo/src/services/backup-staging.ts` — `reapOrphanedStaging`, `stagingDirFor` (the single source of truth for `/tmp/celilo-backup-<record.id>`, shared with `backup-create.ts` so writer and reaper cannot drift). `backup-create.ts` removes its staging in a `finally`, which is correct and NOT enough: a `finally` never runs when the process is killed by a signal — dispatcher timeout, OOM, Ctrl-C, reboot — and those strand the LARGEST directories. So reclamation is kill-mode agnostic by construction: it asks "is anyone still using this?", answered from the `backups.pid` column plus `isPidRunnable`, both of which outlive the process. A directory is removed only when its owner is provably gone (record absent, record terminal, pid dead, or past `STAGING_TTL_MS` = 6h — the TTL is the only check surviving pid reuse). A live backup is always kept; an unrecognised name is ignored, never deleted. Reclaiming a record that still claimed `in_progress` also marks it failed with `ABANDONED_BACKUP_MESSAGE`, so `celilo backup list` stops showing phantom in-flight backups and the `backups` drift check cannot read a dead attempt as a fresh backup.
-- **Freshness audit** — `apps/celilo/src/services/audit/backups.ts` (`auditBackups`) — `backup_missing` / `backup_stale` drift findings against the same declared cadence.
+- **Freshness audit** — `apps/celilo/src/services/audit/backups.ts` (`auditBackups`, `backupStaleThresholdMs`) — `backup_missing` / `backup_stale` drift findings against the same EFFECTIVE cadence, loaded (with each module's override) by `audit/backup-source.ts`. The stale threshold is a formula, `cadence + max(1h, cadence × 0.1)`, not a lookup table: a table cannot answer for `6h`. An effective cadence of `manual` short-circuits BEFORE the never-backed-up check — a module the operator opted out of used to be reported as missing a backup forever, remediated only by the thing they declined.
+- **⚠️ Both age measurements read `backups.completedAt`** — `loadBackupHistory` (`backup-metadata.ts`) and `latestSuccessfulBackupByModule` (`audit/backup-source.ts`). The run path used to measure from `startedAt` and the audit from `completedAt`, so the two disagreed by the duration of the backup itself. Gate: `services/backup-age-agreement.test.ts`.
 - **CLI** — `apps/celilo/src/cli/commands/` — `backup-sweep.ts`, `backup-create.ts` (also `celilo module backup`), `backup-list.ts`, `backup-restore.ts`, `backup-prune.ts`, `backup-delete.ts`, `backup-import.ts`, `backup-pull.ts`, `backup-name.ts`.
 - **Storage destinations CLI** — `storage-add-local.ts`, `storage-add-s3.ts`, `storage-list.ts`, `storage-verify.ts`, `storage-set-default.ts`, `storage-set-path.ts` (relocate a local destination, migrating existing archives unless `--no-migrate`), `storage-remove.ts`. Any credential change goes through `updateStorageCredentials` in `backup-storage.ts`, which clears the verification stamp — a `✓ Verified` must never describe a destination it was not measured against (#566).
 
+## Per-module operator policy
+
+How celilo TREATS a module — its backup cadence, its health-check cadence, its
+upgrade controls — as opposed to how the module configures itself.
+
+- **The keys** — `FRAMEWORK_CONFIG_KEYS` in `apps/celilo/src/cli/commands/module-config.ts` — keys EVERY module accepts whether or not its manifest declares them: `auto_upgrade`, `upgrade_policy`, `backup_schedule`, `health_check_interval`, `backup_retention_count`, `backup_retention_max_age_days`. Each carries `{schema, why}`: a Zod schema, and the reason a wrong value is REFUSED rather than coerced. That reason is per key on purpose — the substance is what a wrong value would silently do (a cadence typo falls back to the manifest's suggestion; an `upgrade_policy` typo falls back to `by-semver`, which skips the pre-deploy backup on a patch), and no validator knows that.
+- **Storage** — rows in `module_configs`, unique on `(module_id, key)`, cascading on module removal. Nothing deletes them on `module update`, and `backup-create.ts` archives them into backup envelopes while `backup-restore.ts` restores them — so an override survives update, upgrade and restore with no new table and no migration.
+- **Operator surface** — `celilo module config set|get|unset <module> <key> [value]`. `unset` (→ `deleteModuleConfig` in `services/module-config.ts`) is what returns a module to following its manifest; without it an operator who once set a value could never go back, and later manifest corrections would stop reaching them permanently. Unsetting an absent key reports that and SUCCEEDS — it states a desired end state. All three are MCP tools with no bespoke code, generated from `packages/core/src/command-registry.ts`.
+- **Display** — `formatCadencePolicy` in `apps/celilo/src/cli/commands/module-status.ts` shows the effective value AND its source, naming the manifest's suggestion when an override is in effect. The stored override alone does not say what it was changed from; the effective value alone does not say who chose it.
+
 ## Persistence
 
 - **DB schema** — `apps/celilo/src/db/schema.ts`. Client: `apps/celilo/src/db/client.ts`. Migration runner: `apps/celilo/src/db/migrate.ts`. Migrations: `apps/celilo/drizzle/`.
@@ -238,6 +276,20 @@ Creation, scheduling and freshness. A module declares an `on_backup` hook and a
 - **Event bus** — `packages/event-bus/src/index.ts` — `Bus`, `openBus`, `defineEvents`, `defineHandler`, `runDispatcher`, pattern matching + timer ticks (`emitDueTimerTicks`, `retentionSweep`). **Exactly one dispatcher per bus**: `runDispatcher` refuses to start while another dispatcher's process is alive (`assertSoleDispatcher` in `dispatcher.ts`, liveness via `bus.liveDispatchers()` — `kill(pid,0)`, not heartbeat age, since a tick blocks for as long as its slowest handler). The exclusion is here rather than in the systemd unit because a stranded dispatcher can sit outside the unit's cgroup where `KillMode` cannot reach it (#580). `bus.health()` reports `dispatcherCount`/`dispatchers` and a `duplicate_dispatcher` status; `checkDispatcher` (`services/fleet-checks.ts`) fails on more than one. **Supervision** (`services/events-daemon.ts`): the unit is named `celilo-events.service` in BOTH the user and system scope, so the two are indistinguishable in every operator-facing string — `install-daemon` (which defaults to **user** scope) therefore refuses when the other scope's unit exists, and `checkDispatcher` compares the live pid against each installed unit's `MainPID` (`unitMainPid`) rather than merely testing that a unit file exists, since a file nobody is running is not supervision (#610).
 - **Dispatcher supervision (install / restart)** — `apps/celilo/src/services/events-daemon.ts` — `installDaemon`/`planDaemonInstall`/`uninstallDaemon`/`readInstalledUnit` write the systemd unit or launchd plist without touching supervisor state, and `restartDaemon` (+ pure `orphanDispatcherPids`, `resolveRestartScope`, `supervisorCommands`) is the one verb that DOES cycle it. CLI: `celilo events install-daemon|uninstall-daemon|show-daemon|restart-daemon [--system]`. `restart-daemon` exists because the dispatcher runs the code it LOADED: celilo-mgr sat 9 days on event-bus v0.1.8 after apt installed v0.2.0, running the very bug the release fixed (celilo#604). Two things make it non-trivial and both are load-bearing: (1) it stops any live dispatcher the supervisor does not own first — an ORPHAN (PPID 1) is invisible to `systemctl restart`, and `assertSoleDispatcher` then crash-loops the unit while the old code keeps serving; (2) it verifies on the BUS that a NEW pid is live reporting `BUS_VERSION`, never off systemctl's exit code, which returns 0 into exactly that crash loop. System scope shells `sudo systemctl` (the unit is root-owned; celilo runs unprivileged), covered by the scoped `/etc/sudoers.d/celilo-events-restart` conffile that `celilo-bootstrap` ships — a unit test asserts the argv and the grant cannot drift apart.
 
+## Terminal UI (`@celilo/cli-display`)
+
+Every CLI-facing primitive celilo renders with. Lives in `packages/cli-display/src/` so module code running in-process (capability functions, hook scripts) can reach it via `@celilo/capabilities`' re-export, and so `@celilo/core` can use it without depending on `@celilo/cli`. `@clack/prompts` was removed tree-wide in celilo#699; nothing here wraps a third-party prompt library.
+
+**The stream contract:** stdout carries a command's RESULT and nothing else — no glyph, no box-drawing prefix, no ANSI. Everything else (prompts, progress, banners, log lines, diagnostics) goes to **stderr**. This is what lets `celilo module list | grep '^caddy '` and `celilo events tail | jq` work without a repair step; the previous renderer put chrome on stdout, which cost a full e2e run when a `│  ` prefix read as a missing module (celilo#695) and made ten JSON commands unparseable (celilo#698).
+
+- **Prompts** — `packages/cli-display/src/prompt.ts` — `text`, `password`, `confirm`, `select`, `multiselect`, plus `CANCEL`/`isCancel` and `isInteractive`. Raw-mode keypress handling with cursor movement, submit-time validation, and per-keystroke tokenization (one stdin chunk can carry a whole paste). **Refuses a non-TTY stdin** with `PromptUnavailableError` instead of hanging on it or taking the next newline as a considered answer — both real past failures. New interactive decisions must NOT be added here; they belong on the event-bus interview, which a headless responder can answer.
+- **Message chrome** — `packages/cli-display/src/messages.ts` — `intro`, `outro`, `note`, `cancel`, and `log.{success,error,warn,info,message,step}`. All to stderr; routed into the active `ProgressDisplay` when one is set, so the two renderers never fight over the cursor.
+- **Colour** — `packages/cli-display/src/colors.ts` — `colorEnabled()`, `paint()`, and the `colors` token object. Gated per use on `NO_COLOR` → `FORCE_COLOR` → `stderr.isTTY`, so colour never reaches a pipe, a file, or protocol mode.
+- **Progress display** — `packages/cli-display/src/progress-display.ts` — `ProgressDisplay`: a flat append-only stream with a time gutter and one status-glyph vocabulary in `render` mode, or structured `[progress:start|done|fail|sub]` markers in `protocol` mode (what the Remote API server emits and the client re-renders locally).
+- **Active-display singleton** — `packages/cli-display/src/active-display.ts` — `getActiveDisplay`/`setActiveDisplay`; how a long-running command lets everything down its call chain route output through one display.
+- **celilo-side wrapper** — `apps/celilo/src/cli/prompts.ts` — `celiloIntro`/`celiloOutro`, `promptText`/`promptPassword`/`promptConfirm`, `showNote`, `log`. Roughly 160 `log.*` call sites route through this one module, which is why swapping the underlying implementation was a one-file change.
+- **FuelGauge** — `apps/celilo/src/cli/fuel-gauge.ts` — the Cylon-style indicator for operations over ~3s (ESC to background, ^C to cancel); delegates to the active `ProgressDisplay` when one is set.
+
 ## Remote API (drive the CLI over the wire)
 
 Run any celilo command on celilo-mgr over SSH instead of screen-scraping `ssh <host> celilo …`. Typed, streamed, per-operation authz, mid-run interviews. Design: `openspec/changes/replace-ssh-cli-api/proposal.md`.
@@ -246,7 +298,7 @@ Run any celilo command on celilo-mgr over SSH instead of screen-scraping `ssh <h
 - **Wire protocol** — `packages/core/src/protocol.ts` (`@celilo/core`) — versioned NDJSON tagged union (`command`/`progress`/`log`/`result`/`error`/`interview`/`answer`) + `translateOutputLine`.
 - **Registry serialization** — `apps/celilo/src/cli/commands/commands-json.ts` — `celilo commands --json` prints the full `COMMANDS` tree as JSON; the live source of truth the MCP fetches to generate its tool surface (so it mirrors whatever celilo version the server runs). `service list --json` similarly exposes configured providers for MCP auto-detect. `module list --json` prints the module roster (id/version/state) as stable JSON — the backbone the MCP composite troubleshooting tools correlate `audit --json` findings against.
 - **Server** — `apps/celilo/src/api/serve.ts` (`apiServeMode`); the `celilo api-serve --principal=<id>` sshd forced-command entry point (dispatched in `apps/celilo/src/cli/index.ts`). Authorizes per principal, runs the command as a protocol-mode child, streams output, audits to stderr.
-- **Client** — `packages/core/src/remote-client.ts` (`@celilo/core`) — `resolveRemote` (`--remote <dest>` / `CELILO_REMOTE`), `runRemoteClient` (`ssh -T`, renders progress via the local ProgressDisplay, answers interviews via clack).
+- **Client** — `packages/core/src/remote-client.ts` (`@celilo/core`) — `resolveRemote` (`--remote <dest>` / `CELILO_REMOTE`), `runRemoteClient` (`ssh -T`, renders progress via the local ProgressDisplay, answers interviews via the `@celilo/cli-display` prompts). Refuses to prompt on a non-TTY stdin, replying `unanswerable` rather than submitting a default as if a human had chosen it.
 - **Access control** — `apps/celilo/src/services/api-access.ts` — `grantPrincipal`, `isAuthorized` (deny-by-default, `command:subcommand` grants), `renderAuthorizedKeys`. Table: `api_principals` (`apps/celilo/src/db/schema.ts`). CLI: `apps/celilo/src/cli/commands/api.ts` (`api grant|list|revoke|authorized-keys|key new`).
 - **Mid-run interview bridge (`kind:daemon` responder)** — `apps/celilo/src/services/remote-responder.ts` — `startRemoteResponder` bridges bus `interview.required.*` ↔ wire.
 - **Server provisioning** — the `celilo-bootstrap` deb (`packaging/celilo-bootstrap/scripts/postinst`) creates the non-root `celilo-api` landing account + sshd; membership in the `celilo` group + `/etc/sudoers.d/celilo` (`!use_pty`) gives api-serve DB access via the wrapper's sudo-drop.
@@ -259,6 +311,6 @@ Run any celilo command on celilo-mgr over SSH instead of screen-scraping `ssh <h
 - **cele2e harness** — `packages/e2e/src/` — `runner.ts`, `container-manager.ts` (`startNetwork`, `reconnectNetwork`), `network-builder.ts` (`NetworkBuilder`).
 - **Run wrapper** — `infra/scripts/cele2e-run.sh` lives in the separate `infra/` clone, **not** in this repo. See the cele2e section of the repo-root `CLAUDE.md` for the operator workflow.
 - **Signal simulators** — three containers, three jobs. `docker/Dockerfile.signal-cli` runs the REAL unlinked daemon (`network().withSignalCli()`, reachable at `signal-cli.lab`) so `e2e/tests/signal-contract.test.ts` can re-check celilo's understanding of the JSON-RPC surface against the actual binary — the pebble pattern. `docker/Dockerfile.signal-sim` runs `simulators/signal-cli/server.ts` (`withSignalSim()`, `signal-sim.lab`), the drivable stand-in with a control surface (`/_control/inbound`, `/_control/sent`, `/_control/unlink`). `/_control/inbound` from the LINKED account's own number emits a `syncMessage.sentMessage` transcript rather than a `dataMessage`, because that is the only shape the real daemon delivers a note-to-self in — and a note-to-self is what every reply is in the default single-operator setup (#460). `docker/Dockerfile.signal-release` (`withSignalRelease()`, `signal-release.lab`) serves the signal-cli release tarball so the module's deploy-time download resolves inside the sealed network — the download is served, never skipped. The libsignal aarch64 native is compiled at `build-infra` time from the module's own recipe (`packages/e2e/scripts/stage-libsignal.ts` → `modules/signal/build/`) and staged into the apt-repo pool, so the recipe is exercised for real on every rebuild while the deploy stays fast and the network stays sealed.
-- **Public-boundary NAT model** — `packages/e2e/config/routing/` — exactly ONE NAT sits between the fleet and the simulated internet: the customer firewall (`fw-main` in `direct-internet`, `fw-isp` in the two-layer default), which MASQUERADEs to `100.100.0.100`. The ISP edge `fw-ext` ROUTES the customer's `100.100.0.0/24` and must never re-NAT it (`-s 100.100.0.0/24 -j RETURN` ahead of its MASQUERADE) — an ISP does not NAT a subscriber that already holds a public address. This is load-bearing, not cosmetic: Namecheap-style DDNS registers the SOURCE address when the caller omits `ip=`, which is how celilo registers public names since #464/#466, so a second NAT here publishes the simulator's own address for every public hostname and quietly breaks ACME, inbound reach, and seeded apex records. Constants: `externalWanIp()` / `externalWanSubnet()` in `src/types.ts`. The corollary: every simulator on `internet-external` must default-route via fw-ext (`100.64.0.1`) so it can reply to the customer's public address — Docker's bridge gateway has no path across networks. `config/routing/public-sim-entrypoint.sh` is the shared two-liner; the DNS hierarchy, pebble, isitup and celilo-website-sim already carried it, and npm-registry / registry / apt-repo / minio / cpanel-host only appeared to work because the double NAT put fw-ext's on-link address in the source field.
+- **Public-boundary NAT model** — `packages/e2e/config/routing/` — exactly ONE NAT sits between the fleet and the simulated internet: the customer firewall (`fw-main` in `direct-internet`, `fw-isp` in the two-layer default), which MASQUERADEs to `203.0.113.100`. The ISP edge `fw-ext` ROUTES the customer's `203.0.113.0/24` and must never re-NAT it (`-s 203.0.113.0/24 -j RETURN` ahead of its MASQUERADE) — an ISP does not NAT a subscriber that already holds a public address. This is load-bearing, not cosmetic: Namecheap-style DDNS registers the SOURCE address when the caller omits `ip=`, which is how celilo registers public names since #464/#466, so a second NAT here publishes the simulator's own address for every public hostname and quietly breaks ACME, inbound reach, and seeded apex records. Constants: `externalWanIp()` / `externalWanSubnet()` in `src/types.ts`. The corollary: every simulator on `internet-external` must default-route via fw-ext (`100.64.0.1`) so it can reply to the customer's public address — Docker's bridge gateway has no path across networks. `config/routing/public-sim-entrypoint.sh` is the shared two-liner; the DNS hierarchy, pebble, isitup and celilo-website-sim already carried it, and npm-registry / registry / apt-repo / minio / cpanel-host only appeared to work because the double NAT put fw-ext's on-link address in the source field.
 - **Simulated address plan** — `packages/e2e/src/types.ts` — `SIM_PRIVATE_SUPERNET` (`10.226.0.0/16`), `zoneIp(zone, host)`, `ZONE_SUBNETS`, `ZONE_GATEWAYS`. Every simulated PRIVATE zone is derived from this one table; the compose generator, `zone-classifier.ts` and the harness's `system init` all read it, so renumbering the whole sim is a one-line change. The sim deliberately does NOT reuse a real fleet's zone /24s (#539): the previous plan was byte-identical to production's, so a stack leaked on celilo's own forgejo-builder — which lives in the real dmz — claimed the builder's own subnet and blackholed every containerized CI job's route to the forge for ~135s. Teardown cannot prevent that (a SIGKILL runs no handler), so the addresses moved instead; the second, better reason is that a suite which only passes on production's exact octets is asserting one site's address plan rather than celilo's behaviour. `src/address-plan.test.ts` is the recurrence gate — it fails if a zone leaves the supernet, or if any retired fleet prefix reappears anywhere in `packages/e2e`, `e2e/tests` or `modules/*/e2e`.
 - **MCP server (agent-driven e2e)** — `packages/mcp-server/src/index.ts` — stdio MCP server exposing `start_run`/`run_status`/`run_result`/`stop_run` (detached cele2e runs read off the event bus, no ANSI scraping) + `env_check` (docker VM, run-lock, shared-infra, mgmt image CLI version, netapps). Dev/ops tool, `private`, not shipped to consumers.

package/drizzle/0024_module_pause.sql

@@ -0,0 +1,20 @@
+-- Module pause/unpause (openspec/changes/module-pause-lifecycle).
+--
+-- Purely additive: `state` gains a new legal value 'PAUSED' (no constraint to
+-- alter — the column is free text typed only in TypeScript), plus two nullable
+-- columns. Every existing module is simply not paused, so there is nothing to
+-- backfill.
+--
+-- `paused_at` is not decoration. A pause suppresses the alerting that would
+-- otherwise report the module as down, so the paused-ness itself has to be
+-- loud; the DURATION is what distinguishes a maintenance window from an outage
+-- someone forgot about, and `state` alone cannot answer it (design D7).
+--
+-- Note for rollback: an older CLI reading state='PAUSED' hits a value outside
+-- its union and fails to recognise it, rather than reading the module as live.
+-- Loud beats silent — but it still has no `unpause`, so unpause everything
+-- before downgrading.
+
+ALTER TABLE `modules` ADD `paused_at` integer;--> statement-breakpoint
+ALTER TABLE `modules` ADD `pause_reason` text;--> statement-breakpoint
+CREATE INDEX `modules_state_idx` ON `modules` (`state`);

package/drizzle/meta/_journal.json

@@ -169,6 +169,13 @@
       "when": 1783900000000,
       "tag": "0023_public_dns_evidence",
       "breakpoints": true
+    },
+    {
+      "idx": 24,
+      "version": "6",
+      "when": 1784000000000,
+      "tag": "0024_module_pause",
+      "breakpoints": true
     }
   ]
-}
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@celilo/cli",
-  "version": "0.22.0",
+  "version": "0.24.0",
   "description": "Celilo — home lab orchestration CLI",
   "type": "module",
   "bin": {
@@ -56,12 +56,12 @@
     "check:schema": "bun run scripts/export-manifest-schema.ts --check"
   },
   "dependencies": {
-    "@aws-sdk/client-s3": "^3.1024.0",
-    "@celilo/capabilities": "^1.0.1",
-    "@celilo/cli-display": "^0.1.10",
-    "@celilo/core": "^0.6.0",
-    "@celilo/event-bus": "^0.4.0",
-    "@clack/prompts": "^1.1.0",
+    "@aws-sdk/client-s3": "^3.1109.0",
+    "@aws-sdk/lib-storage": "^3.1101.0",
+    "@celilo/capabilities": "^1.2.0",
+    "@celilo/cli-display": "^0.2.0",
+    "@celilo/core": "^0.8.0",
+    "@celilo/event-bus": "^0.5.0",
     "ajv": "^8.18.0",
     "drizzle-orm": "^0.36.4",
     "ink": "^7.0.1",

package/src/__integration__/container-services-cli.integration.test.ts

@@ -111,7 +111,11 @@ describe('Container Services and Machine Pool CLI Integration', () => {
       // Remove service with --force flag to skip confirmation
       const output = runCli(ctx.cli, 'service remove service-to-remove --force');
 
-      expect(output).toContain('removed');
+      // The command's RESULT, which is what stdout carries. This used to assert
+      // on the lowercase "removed" of the `celiloOutro` banner, which clack put
+      // on stdout while putting the result on stderr; celilo#699 swapped those
+      // so stdout is the data channel and the banner is chrome on stderr.
+      expect(output).toContain('Removed service: service-to-remove');
 
       // Verify deletion
       const services = await db.select().from(containerServices).all();
@@ -236,7 +240,9 @@ describe('Container Services and Machine Pool CLI Integration', () => {
       // Remove machine with --force flag to skip confirmation
       const output = runCli(ctx.cli, 'machine remove remove-me --force');
 
-      expect(output).toContain('removed');
+      // See the service-remove case above: stdout now carries the result, not
+      // the outro banner (celilo#699).
+      expect(output).toContain('Removed machine: remove-me');
 
       // Verify deletion
       const machinesList = await db.select().from(machines).all();

package/src/api/remote-client.test.ts

@@ -93,11 +93,12 @@ test('renders a forwarded interview and sends the answer back', async () => {
 /**
  * Regression for the fabricated "operator declined".
  *
- * The default renderer prompts with clack, which reads keypresses off stdin
- * whether or not stdin is a terminal. Driven over the MCP (stdin = the JSON-RPC
- * stream) the next newline submitted the prompt at its `initialValue` — the
- * question's `defaultValue` — so a breaking update nobody saw came back as a
- * considered "no". With no terminal the client must say it cannot answer.
+ * The default renderer prompts on the terminal. The renderer it replaced read
+ * keypresses off stdin whether or not stdin was a terminal, so driven over the
+ * MCP (stdin = the JSON-RPC stream) the next newline submitted the prompt at
+ * its `initialValue` — the question's `defaultValue` — and a breaking update
+ * nobody saw came back as a considered "no". With no terminal the client must
+ * say it cannot answer.
  *
  * And it must say so as `unanswerable`, NOT as an `answer` of any shape: an
  * `answer` is what consumes the query and destroys a question nobody decided

package/src/api/serve.ts

@@ -27,8 +27,10 @@ import {
 } from '@celilo/core';
 import { parseArguments } from '../cli/parser';
 import { getEventBusPath } from '../config/paths';
+import { getDb } from '../db/client';
 import { isAuthorized } from '../services/api-access';
 import { EVENT_TYPES } from '../services/bus-interview';
+import { describePausedModule, listPausedModules } from '../services/module-pause';
 import { type WireInterview, startRemoteResponder } from '../services/remote-responder';
 import {
   SessionWriter,
@@ -50,6 +52,38 @@ function send(msg: ServerMessage): void {
   process.stdout.write(`${JSON.stringify(msg)}\n`);
 }
 
+/**
+ * The fleet-level warnings stamped onto every terminal result (design D7).
+ *
+ * Deliberately unconditional and deliberately on UNRELATED commands: a pause
+ * suppresses its module's alerting, so the paused-ness itself is the only
+ * remaining signal, and a forgotten pause is found incidentally rather than by
+ * someone choosing to look. One indexed query (`modules_state_idx`), so this
+ * stays cheap enough to run on every call.
+ *
+ * Never throws: a broken or missing DB must not turn a working command into a
+ * failure over a warning.
+ */
+function fleetWarnings(): string[] {
+  try {
+    const paused = listPausedModules(getDb());
+    if (paused.length === 0) return [];
+    return [
+      `${paused.length} module(s) PAUSED: ${paused.map((m) => describePausedModule(m)).join(', ')}. A paused module is quiesced and its alerts are suppressed. Unpause with "celilo module unpause <id>".`,
+    ];
+  } catch {
+    return [];
+  }
+}
+
+/** Terminal result, with the fleet warnings attached. */
+function resultMessage(success: boolean, exitCode: number): ServerMessage {
+  const warnings = fleetWarnings();
+  return warnings.length > 0
+    ? { type: 'result', success, exitCode, warnings }
+    : { type: 'result', success, exitCode };
+}
+
 function audit(principal: string, op: string, decision: string, exitCode?: number): void {
   const suffix = exitCode === undefined ? '' : ` exit=${exitCode}`;
   process.stderr.write(
@@ -107,7 +141,7 @@ async function runCommand(argv: string[], forward: (msg: ServerMessage) => void)
 
   await Promise.all([pumpLines(child.stdout, forward), pumpLines(child.stderr, forward)]);
   const exitCode = await child.exited;
-  forward({ type: 'result', success: exitCode === 0, exitCode });
+  forward(resultMessage(exitCode === 0, exitCode));
   return exitCode;
 }
 
@@ -119,7 +153,7 @@ async function handleAttach(principal: string, sessionId: string): Promise<void>
   const record = readSession(sessionId);
   if (!record) {
     send({ type: 'error', error: `no such session: ${sessionId}` });
-    send({ type: 'result', success: false, exitCode: 1 });
+    send(resultMessage(false, 1));
     return;
   }
   if (record.principal !== principal) {
@@ -127,7 +161,7 @@ async function handleAttach(principal: string, sessionId: string): Promise<void>
       type: 'error',
       error: `permission denied: session ${sessionId} belongs to another principal`,
     });
-    send({ type: 'result', success: false, exitCode: EXIT_PERMISSION_DENIED });
+    send(resultMessage(false, EXIT_PERMISSION_DENIED));
     audit(principal, `attach:${sessionId}`, 'deny');
     return;
   }
@@ -167,7 +201,7 @@ function handleCancel(principal: string, sessionId: string): void {
   const record = readSession(sessionId);
   if (!record) {
     send({ type: 'error', error: `no such session: ${sessionId}` });
-    send({ type: 'result', success: false, exitCode: 1 });
+    send(resultMessage(false, 1));
     return;
   }
   if (record.principal !== principal) {
@@ -175,7 +209,7 @@ function handleCancel(principal: string, sessionId: string): void {
       type: 'error',
       error: `permission denied: session ${sessionId} belongs to another principal`,
     });
-    send({ type: 'result', success: false, exitCode: EXIT_PERMISSION_DENIED });
+    send(resultMessage(false, EXIT_PERMISSION_DENIED));
     audit(principal, `cancel:${sessionId}`, 'deny');
     return;
   }
@@ -185,7 +219,7 @@ function handleCancel(principal: string, sessionId: string): void {
     emittedBy: `api:${principal}`,
   });
   audit(principal, `cancel:${sessionId}`, 'allow');
-  send({ type: 'result', success: true, exitCode: 0 });
+  send(resultMessage(true, 0));
 }
 
 export async function apiServeMode(principal: string): Promise<void> {
@@ -251,7 +285,7 @@ export async function apiServeMode(principal: string): Promise<void> {
 
     if (!(await isAuthorized(principal, command, subcommand))) {
       send({ type: 'error', error: `permission denied: "${principal}" is not granted "${label}"` });
-      send({ type: 'result', success: false, exitCode: EXIT_PERMISSION_DENIED });
+      send(resultMessage(false, EXIT_PERMISSION_DENIED));
       audit(principal, label, 'deny');
       return;
     }

package/src/api-clients/proxmox.ts

@@ -636,6 +636,40 @@ export class ProxmoxClient {
     if (!result.success) return result;
     return { success: true, data: findNodeForVmid(result.data, vmid) };
   }
+
+  /**
+   * Power a guest down or up. Used by `module pause --stop-infra` (design D2),
+   * where stopping the box is opt-in rather than what a pause means.
+   *
+   * `shutdown` rather than `stop` on the way down: it asks the guest to go
+   * quietly instead of pulling its power, and a pause is a planned operation,
+   * not a fault. Proxmox returns a task id; callers that need to know it
+   * finished poll with `pollTaskUntilDone`.
+   */
+  async setGuestPower(
+    vmid: number,
+    kind: 'lxc' | 'qemu',
+    power: 'shutdown' | 'start',
+  ): Promise<ProxmoxResult<string>> {
+    const node = await this.nodeForVmid(vmid);
+    if (!node.success) return node;
+    if (!node.data) {
+      return { success: false, message: `No Proxmox node hosts vmid ${vmid}` };
+    }
+    return makeProxmoxPost<string>(
+      this.credentials,
+      `/nodes/${node.data}/${kind}/${vmid}/status/${power}`,
+      {},
+    );
+  }
+
+  /** Current run state of a guest (`running`, `stopped`, …). */
+  async guestStatus(vmid: number): Promise<ProxmoxResult<string | null>> {
+    const result = await this.clusterResources();
+    if (!result.success) return result;
+    const guest = result.data.find((r) => r.vmid === vmid);
+    return { success: true, data: guest?.status ?? null };
+  }
 }
 
 /**

package/src/cli/commands/alerts-act.ts

@@ -7,7 +7,6 @@
  */
 
 import { getDb } from '../../db/client';
-import { parseIntervalMinutes } from '../../manifest/schema';
 import {
   acknowledgeAlert,
   findLiveAlertByKey,
@@ -15,6 +14,7 @@ import {
   silenceAlert,
 } from '../../services/alerting/ack';
 import { findPerson, listPeople } from '../../services/alerting/people';
+import { parseIntervalMinutes } from '../../services/cadence';
 import type { CommandResult } from '../types';
 
 function resolveKey(key: string | undefined) {

package/src/cli/commands/alerts-sweep.ts

@@ -21,6 +21,7 @@ import type { SuppressionTopology } from '../../services/alerting/suppression';
 import { runSweep } from '../../services/alerting/sweep-runner';
 import { getModuleSystems } from '../../services/deployed-systems';
 import { runModuleHealthCheck } from '../../services/health-runner';
+import { listPausedModules } from '../../services/module-pause';
 import type { CommandResult } from '../types';
 
 /** Grace window before a newly-fired alert may notify. */
@@ -65,6 +66,7 @@ export async function handleAlertsSweep(): Promise<CommandResult> {
     },
     loadTopology: () => loadTopology(db),
     loadDeployWindowModules: () => modulesInDeployWindow(db),
+    loadPausedModules: () => new Set(listPausedModules(db).map((m) => m.id)),
     isSuppressible: (alert) => suppressibleByMonitor.get(alert.monitorId) ?? true,
     notifyDepsFor: (alert) => buildNotifyDeps(db, alert, new Date()),
     now: () => new Date(),

package/src/cli/commands/backup-create.ts

@@ -10,7 +10,17 @@ import {
   isBackupDue,
 } from '../../services/backup-create';
 import { formatSize } from '../../services/backup-metadata';
-import { pruneBackupsForModule } from '../../services/backup-retention';
+import {
+  effectiveBackupRetention,
+  pruneBackupsForModule,
+  prunesNothing,
+} from '../../services/backup-retention';
+import {
+  BACKUP_SCHEDULE_CONFIG_KEY,
+  effectiveBackupSchedule,
+} from '../../services/backup-schedule';
+import { formatCadence } from '../../services/cadence';
+import { configOverride } from '../../services/module-config';
 import { celiloIntro, celiloOutro } from '../prompts';
 import type { CommandResult } from '../types';
 
@@ -56,12 +66,20 @@ export async function handleBackupCreate(
       if (eligible.length > 0) {
         console.log('\nChecking module backup schedules...');
 
-        for (const { module: mod, manifest } of eligible) {
-          const schedule = manifest.backup?.schedule ?? 'manual';
+        for (const { module: mod, manifest, configs } of eligible) {
+          // Through the shared accessor, like every other reader. This site
+          // used to read the manifest directly AND default an absent cadence
+          // to `manual` — the opposite of what the accessor has always said —
+          // so `celilo backup create` quietly skipped modules the hourly sweep
+          // was backing up daily.
+          const schedule = effectiveBackupSchedule(
+            manifest,
+            configOverride(configs, BACKUP_SCHEDULE_CONFIG_KEY),
+          );
           const due = force || isBackupDue(mod.id, schedule);
 
           if (!due) {
-            console.log(`  ${mod.id}: skipping (not due, schedule: ${schedule})`);
+            console.log(`  ${mod.id}: skipping (not due, schedule: ${formatCadence(schedule)})`);
             moduleSkipCount++;
             continue;
           }
@@ -77,13 +95,10 @@ export async function handleBackupCreate(
           console.log(`  → ${result.storagePath}`);
           moduleBackupCount++;
 
-          // Auto-prune per retention policy
-          const retention = manifest.backup?.retention;
-          if (retention) {
-            const pruneResult = await pruneBackupsForModule(mod.id, {
-              count: retention.count,
-              maxAgeDays: retention.max_age_days,
-            });
+          // Auto-prune per the effective retention policy
+          const policy = effectiveBackupRetention(manifest, configs);
+          if (!prunesNothing(policy)) {
+            const pruneResult = await pruneBackupsForModule(mod.id, policy);
             if (pruneResult.deleted > 0) {
               console.log(`  ✓ Pruned ${pruneResult.deleted} old backup(s)`);
             }