@celilo/cli 0.13.3 → 0.14.0

package/CELILO_CORE_MODULES.md

@@ -26,10 +26,12 @@ Each entry: `module id` — what it is — **provides** / **requires** capabilit
 - **knot-unbound-internal** — split-horizon internal DNS via Knot (authoritative) + Unbound (recursive); lightweight, plain apt, no .NET. **provides:** `dns_internal`. Ships a base-module-aspect (`modules/knot-unbound-internal/base-module-aspect/`).
 - **technitium** — internal split-horizon DNS resolver + authoritative server (web UI + HTTP API); heavier alternative to knot-unbound. **provides:** `dns_internal`. Ships a base-module-aspect (`modules/technitium/base-module-aspect/`).
 - **namecheap** — public DNS A-record management via Namecheap Dynamic DNS API (HTTP, no browser automation). **provides:** `dns_registrar`.
+- **wireguard** — owns the admin WireGuard tunnel on the firewall host: interface, listen port, peers (as records), and client subnet are module config rather than hand-maintained state. Exposes the listen port and **registers the client subnet as a trusted source**, so VPN reach into the managed zones is in the firewall registry and every converge re-emits it. Records `network.vpn.subnet`, which the internal resolver's split-horizon view also consumes. Adopts a running tunnel in place (existing key and peers retained; `wg syncconf`, never `wg-quick down`) because that tunnel is the operator's recovery path. **requires:** `firewall` (and the provider must support trusted-source registration — `iptables` does, `greenwave` does not).
 
 ## Public edge (ingress / identity)
 
 - **caddy** — reverse proxy with automatic HTTPS (HTTP-01 ACME); the standard HTTPS ingress. **provides:** `public_web`. **requires:** `dns_registrar`, `firewall`.
+- **generic-cpanel-hosting-provider** — publishes static sites into a subfolder of a domain on a cPanel/SSH web host celilo does **not** govern (the host owns the domain, DNS, TLS and web server). Systemless like **namecheap** — no `requires.system`, no zone, no IPAM; it holds credentials for an external party. One provider serves many accounts, resolved by hostname; onboarding takes the account password once, `ssh-copy-id`s celilo's key, then discards it. **provides:** `external_web`.
 - **authentik** — Authentik identity provider with OIDC (Docker Compose: server, worker, Postgres, Redis). **provides:** `idp`. **requires:** `public_web`, `dns_registrar`, `firewall`.
 
 ## Celilo's own infrastructure (self-hosted)
@@ -37,6 +39,7 @@ Each entry: `module id` — what it is — **provides** / **requires** capabilit
 - **celilo-mgmt** — the celilo management server itself, deployed as a module (replaces install.sh + `system init`; ships daemon, runs migrations, self-registers). **provides:** `celilo_event_bus`, `celilo_module_deploy_worker`. **requires:** `cross_module_read`. See `openspec/specs/management-as-module/spec.md`.
 - **celilo-registry** — module registry server (Cargo sparse protocol); stores `.netapp` files, serves index + search/download API. On install it provisions a confidential introspection OIDC client via `idp.create_oidc_client` (SECURE_MODULE_PUBLISH.md §5[D-A]) and converges its issuer + introspection endpoint + creds onto the box for RFC 7662 token verification. **provides:** `registry_publish`. **requires:** `public_web`, `dns_registrar`, `idp`.
 - **celilo-apt-repo** — Debian apt repository (reprepro + Bun HTTP server) serving the celilo `.deb` at apt.celilo.computer. **provides:** `apt_publish`. **requires:** `public_web`, `dns_registrar`.
+- **signal** — bidirectional Signal transport for alerts and deploy-interview questions; runs signal-cli in daemon mode with its JSON-RPC socket bound to the host's own address (never public). Enrolled as a SECONDARY DEVICE of an existing Signal account rather than registering its own number — Signal blocks most VOIP ranges and bans bot-ish registrations. Recipient addresses live on celilo routes, not in module config, so adding a person never requires a redeploy. Runs on x86_64 and aarch64. `libsignal-client` ships no linux-aarch64 native, so celilo builds one (`modules/signal/build/`) and installs it as a `libsignal-jni` .deb on ARM hosts; x86_64 uses the JAR's bundled native. **provides:** `notification` (`send`, `receive`). **requires:** nothing — a transport that depended on the proxy, registrar or firewall could not tell you those were broken. See `openspec/changes/add-alerting/`.
 - **celilo-website** — public docs site (static Astro) served via Caddy on celilo.computer. **requires:** `public_web`, `dns_registrar`.
 
 ## Git forge & CI pipeline

package/CELILO_SUBSYSTEMS.md

@@ -45,13 +45,39 @@ see `openspec/specs/`. Companion doc: [CELILO_CORE_MODULES.md](./CELILO_CORE_MOD
 | `public_web` | `packages/capabilities/src/public-web.ts` (`createPublicWeb`). **Route lifecycle** is framework-owned at both ends, not per-module: removing a consumer runs `apps/celilo/src/services/web-route-cleanup.ts` (delete rows → reclaim `/srv/www/<slug>` → emit `routes_changed`) whether or not the module has an `on_uninstall`; deploying a provider runs `apps/celilo/src/services/public-web-republish.ts` (re-run every static consumer's `on_install`) so a provider rebuild refills the web roots it destroyed. | caddy |
 | `idp` | `packages/capabilities/src/idp.ts` | authentik |
 | `dns_registrar` | `packages/capabilities/src/dns-registrar.ts` (`registerHost`) | namecheap |
+| `external_web` | `packages/capabilities/src/external-web.ts` (`publishStaticSite`). **OFF-FLEET static publishing** — the counterpart to `public_web`. See the in-fleet vs off-fleet note below. | generic-cpanel-hosting-provider |
 | `firewall` | `packages/capabilities/src/firewall.ts` (`exposeService`, `unexposeService`, `listExposedServices`). **Converge model**: register into the shared-core port-forward registry → render the complete ruleset → apply atomically (see the firewall converge note below). | greenwave, iptables |
 | `dns_internal` | `packages/capabilities/src/dns-internal.ts` | knot-unbound-internal, technitium |
 | `dhcp_server` | `packages/capabilities/src/dhcp-server.ts` | greenwave |
 | `source_forge` | `packages/capabilities/src/source-forge.ts` | forgejo |
 | `registry_publish` | `packages/capabilities/src/registry-publish.ts` | celilo-registry |
+| `notification` | `packages/capabilities/src/notification.ts` (`send`, optional `receive`) | signal (planned) — transports are ordinary modules, both self-hosted and credential-only |
 | `cross_module_read` | `packages/capabilities/src/cross-module-read.ts` | framework (read other modules' capability data) |
 
+### In-fleet (`public_web`) vs off-fleet (`external_web`)
+
+Both serve a static site; the difference is **control, not file serving**. A
+system is *in-fleet* when celilo governs it — it can run a managed daemon, bind
+a port, drive the firewall, put the box on the VPN. It is **off-fleet** when
+celilo can only place files there: an unprivileged, jailed cPanel account is the
+canonical case. Off-fleet is a governance boundary, not a deploy boundary.
+
+| | `public_web` (caddy) | `external_web` |
+|---|---|---|
+| TLS certificate | celilo obtains it (ACME) | already installed by the host |
+| Public + split-horizon DNS | celilo registers | the host's |
+| Firewall ingress | celilo opens | not celilo's to open |
+| `idp` / reverse-proxy composition | yes | no |
+| Route | any path, incl. `/` | a subfolder only; `/` is REJECTED (the docroot holds the host's own site) |
+
+`external_web` is module-owned (loaded through `CAPABILITY_MODULE_MAP` like
+`dns_registrar`), not framework-owned like `createPublicWeb` — there is no
+celilo-side machinery to own, only a provider holding credentials. Its
+`publishStaticSite` signature is a subset of `public_web`'s on purpose, so a
+content module targets either with
+`capabilities.external_web ?? capabilities.public_web`. Both are
+**authoritative**: a publish that doesn't reach clients throws (#328).
+
 ## Remote-ops primitives (the SSH seam — modules never hand-build SSH)
 
 Module hooks reach a remote box ONLY through these typed primitives
@@ -59,16 +85,17 @@ Module hooks reach a remote box ONLY through these typed primitives
 `modules/**/scripts/` is a defect. Impl: `packages/capabilities/src/remote.ts`
 (exported from `packages/capabilities/src/index.ts`).
 
-- **`remoteExec`** — the ONE ssh seam (`ssh root@<target> <cmd>`); everything else builds on it.
+- **`remoteExec`** — the ONE ssh seam (`ssh <user>@<target> <cmd>`); everything else builds on it. `user`/`port`/`identityFile` on the target default to `root`/22/the agent key, so a fleet call is unchanged; an OFF-FLEET account (`external_web`) sets them.
 - **`probe`** — read-only health checks: `systemd` / `http` / `command`.
 - **`serviceCtl`** — systemctl start/stop/restart/reload/enable/disable.
 - **`runAppCommand` / `runAppCommandWithSecret`** — escape-hatch on-box command; the secret variant feeds the secret on **stdin** (`$SECRET`), never argv.
 - **`streamBackup` / `streamRestore` / `fetchFile` / `pushFile`** — binary-safe streaming via local shell redirect/pipe.
 - **`waitFor`** — predicate-poll combinator.
 - **`applyRenderedConfig`** — converge: write rendered config → validate → apply → rollback (one round-trip). Used by caddy (Caddyfile), knot (views), iptables (ruleset).
+- **`installAuthorizedKey`** — ONE-TIME credential bootstrap for an off-fleet account: `ssh-copy-id` under a password taken from the child ENV (`SSH_ASKPASS_REQUIRE=force`, so no `sshpass` dependency and the password never lands in a command string). Idempotent; used by `external_web` onboarding so every later publish is key-based.
 - **`tailLog` / `grepLog`** — journald reads (regex / ignoreCase).
 
-Target = `RemoteTarget` (`{ ipv4_address }`; `DeployedSystem` satisfies it). The
+Target = `RemoteTarget` (`{ ipv4_address }` plus optional `user` / `port` / `identityFile`; `DeployedSystem` satisfies it). The
 runner seam (`execRunner` real / `createMockRunner` for tests) lives in
 `packages/capabilities/src/testing.ts`. Authoring guide: `MODULE_PRIMITIVES.md`
 (ships in `@celilo/cli`).
@@ -88,6 +115,27 @@ runner seam (`execRunner` real / `createMockRunner` for tests) lives in
   `exposeService`/`unexposeService` register into the store, then render + apply
   atomically via `applyRenderedConfig` (`iptables-restore`). Replaces the old
   per-rule `iptables -A`; the registry (not `iptables -L`) is the source of truth.
+- **Trusted-source registry (desired state)** — `trusted_sources` DB table
+  (migration `0016`) + `apps/celilo/src/services/trusted-sources.ts`
+  (`buildTrustedSourceStore`, `composeTrustedSubnets`). The sibling primitive to
+  port forwards, for the case a port forward cannot express: an ORIGIN subnet
+  permitted to initiate into every managed zone. Registered through the OPTIONAL
+  `registerTrustedSource` / `withdrawTrustedSource` / `listTrustedSources` trio on
+  `packages/capabilities/src/firewall.ts`; `iptables` implements it, `greenwave`
+  deliberately does not. A consumer detects support with `supportsTrustedSources`
+  and fails loudly via `requireTrustedSources` — a silent no-op is prohibited by
+  the contract, because a discarded registration yields a network that believes
+  it has reach it does not have.
+- **Composed trusted subnets** — `apps/celilo/src/hooks/capability-loader.ts`
+  (`loadTrustedSubnets`): derived control plane + module registrations + the
+  `firewall.trusted_subnets` operator override, deduped and origin-tagged. With
+  no contributors it equals the derived subnet alone, so the rendered ruleset is
+  byte-identical to the pre-composition output.
+- **Unowned-trusted-network audit** — `apps/celilo/src/services/audit/trusted-sources.ts`
+  + `apps/celilo/src/services/firewall-reach.ts`: reads each firewall's LIVE table
+  (`iptables-save`) and reports reach granted to a network celilo does not
+  recognise, instead of removing it silently at the next converge. Surfaces under
+  `celilo system audit` as the `trusted_sources` category.
 
 ## Hooks & deploy
 
@@ -114,6 +162,26 @@ runner seam (`execRunner` real / `createMockRunner` for tests) lives in
 - **Registry token verification (opaque + idp introspection)** — `packages/registry-server/src/auth.ts` (`TokenAuth` — opaque SHA-256 publish tokens, admin/per-package scope) + `packages/registry-server/src/introspection.ts` (`IntrospectionVerifier` — RFC 7662 verify-bridge, SECURE_MODULE_PUBLISH.md §5[D-A]). `authorizePackage()` in `server.ts` tries the opaque set first (unchanged), then, for a token unknown to it, `identify()`s it via the idp introspection endpoint using the registry's confidential OIDC client creds (`OIDC_INTROSPECTION_ENDPOINT`/`OIDC_CLIENT_ID`/`OIDC_CLIENT_SECRET`, provisioned on install — ce-7aa), reading `{active, sub, groups, exp}`. Fails CLOSED on any introspection error; never logs tokens/secrets. Instant revocation: revoke at the idp → next publish sees `active:false` → 401.
 - **Registry module-owner table (hybrid group + owner authz — ce-1ch, D-C)** — `packages/registry-server/src/module-owner-store.ts` (`ModuleOwnerStore` — JSON-persisted `{moduleName, ownerSub, claimedAt, sourceGroup}`, `REGISTRY_OWNERS_FILE`/`dataDir/module-owners.json`). The verified `groups` claim gates *whether* an identity may publish (`REGISTRY_ADMIN_GROUP`→publish/reassign anything; `REGISTRY_PUBLISHER_GROUP`, default `celilo-authors`, configurable→claim+publish owned); the owner table gates *which names*. First-publish-claims: the first verified publisher of an unclaimed name owns it; a *different* publisher is then DENIED (confused-deputy defense — Author-A cannot publish Author-B's module). Admin HTTP endpoints `GET /api/v1/modules/owners`, `GET|POST /api/v1/modules/owners/{name}` (reassign). Operator front door: `celilo registry owner list|show|set` (`apps/celilo/src/cli/commands/registry-owner.ts`), admin token resolved from the local `publish_tokens` bootstrap list.
 
+## Alerting & notifications
+
+Observation and delivery: monitors run checks on a schedule, alerts hold what
+is currently wrong, and routes carry the message to a person's phone. Design:
+`openspec/changes/add-alerting/design.md`.
+
+- **Alert identity** — `apps/celilo/src/services/alerting/keys.ts` — the key grammar (`module:<id>[/check:<name>]`, `builtin:<check>[/<kind>:<target>]`) that makes "the same problem" the same alert across runs. `moduleAlertKey`, `moduleCheckAlertKey`, `builtinAlertKey`, `parseAlertKey`.
+- **Reconciliation** — `apps/celilo/src/services/alerting/reconcile.ts` (`reconcile`) — a successful run's failing-key set is authoritative and resolution is by SET DIFFERENCE (absent ⇒ resolved). A run whose outcome is `error` resolves NOTHING and fires a module-level alert instead: the false-all-clear guard.
+- **Monitor execution** — `apps/celilo/src/services/alerting/run-monitor.ts` (`runOneMonitor`) + `sweep.ts` (`selectDueMonitors`) + `builtin-monitors.ts` / `health-coverage.ts` (the built-in checks and the "module with no health check" coverage check).
+- **The sweep** — `apps/celilo/src/services/alerting/sweep-runner.ts` (`runSweep`) — the ordered pass that makes alerting run by itself: run due monitors → promote past-grace alerts → re-evaluate suppression → flush quiet-hours deferrals → notify. Driven by `celilo alerts sweep` on `timer.tick.5m`. Never throws for one bad monitor.
+- **Suppression (topology-derived, never configured)** — `apps/celilo/src/services/alerting/suppression.ts` — `ancestorKeysFor`/`findSuppressor`/`machineAlertKey`. A firing machine explains its modules' failures; a firing capability provider explains its zone's consumers. Derived from `module_systems`, so it cannot drift from reality. Deploy windows: `deploy-hooks.ts` (`openDeployWindow`/`closeDeployWindows` — closed by module, so a crashed deploy self-heals).
+- **Escalation & quiet hours** — `escalation.ts` (`decideEscalation`, every reason to stay silent enumerated) + `quiet-hours.ts` (`isWithinQuietHours`, Intl-based and DST-safe). Quiet hours defer the MESSAGE while the escalation clock keeps running.
+- **Delivery** — `notifier.ts` (`notifyAlert`, `deliverDeferred`, `notifyResolved`, body composition) + `transport-loader.ts` (`loadNotificationTransport` — resolves a route's transport module's `notification` capability).
+- **Inbound replies** — `tokens.ts` (per-DELIVERY reply tokens, so a reply identifies WHO) + `inbound.ts` (`interpretInbound` — two-factor: valid unexpired token AND sender matching the issuing route) + `inbound-poller.ts` (`pollInbound`, `makeReceiver`). celilo-mgr polls the transport; nothing calls in, so the ack path never depends on the public HTTPS it may be paging about.
+- **Ack / silence / resolve** — `ack.ts` — three deliberately distinct operations. An ack is broadcast to every other paged route.
+- **Interviews over a transport** — `interview-responder.ts` (delivery policy; `secret.*` families are REFUSED) + `notification-responder.ts` (queries the bus's events table for unanswered questions: a watch only sees events emitted after it registers, and `alerts poll` is one-shot. Leaves a question alone for 90s so an attached responder — terminal, or the `api-serve` wire bridge — answers first). Alerts and interviews share the send path, the token table, and the inbound path; only `targetId` differs.
+- **People, routes, policies** — `people.ts` — `createPerson`/`createRoute`/`createPolicy`/`addPolicyStep`.
+- **Persistence** — `apps/celilo/src/services/alerting/store.ts`; tables `people`, `routes`, `escalation_policies`, `escalation_steps`, `monitors`, `monitor_runs`, `suppression_windows`, `alerts`, `notification_deliveries` (`apps/celilo/src/db/schema.ts`, migration `drizzle/0017_alerting.sql`).
+- **CLI** — `apps/celilo/src/cli/commands/` — `alerts-list.ts`, `alerts-act.ts` (ack/silence/resolve), `alerts-sweep.ts`, `alerts-poll.ts`, `monitor.ts`, `notify-config.ts` (`person`/`route`/`escalation-policy`).
+
 ## Persistence
 
 - **DB schema** — `apps/celilo/src/db/schema.ts`. Client: `apps/celilo/src/db/client.ts`. Migration runner: `apps/celilo/src/db/migrate.ts`. Migrations: `apps/celilo/drizzle/`.
@@ -141,4 +209,5 @@ Run any celilo command on celilo-mgr over SSH instead of screen-scraping `ssh <h
 
 - **cele2e harness** — `packages/e2e/src/` — `runner.ts`, `container-manager.ts` (`startNetwork`, `reconnectNetwork`), `network-builder.ts` (`NetworkBuilder`).
 - **Run wrapper** — `infra/scripts/cele2e-run.sh` lives in the separate `infra/` clone, **not** in this repo. See the cele2e section of the repo-root `CLAUDE.md` for the operator workflow.
+- **Signal simulators** — three containers, three jobs. `docker/Dockerfile.signal-cli` runs the REAL unlinked daemon (`network().withSignalCli()`, reachable at `signal-cli.lab`) so `e2e/tests/signal-contract.test.ts` can re-check celilo's understanding of the JSON-RPC surface against the actual binary — the pebble pattern. `docker/Dockerfile.signal-sim` runs `simulators/signal-cli/server.ts` (`withSignalSim()`, `signal-sim.lab`), the drivable stand-in with a control surface (`/_control/inbound`, `/_control/sent`, `/_control/unlink`). `docker/Dockerfile.signal-release` (`withSignalRelease()`, `signal-release.lab`) serves the signal-cli release tarball so the module's deploy-time download resolves inside the sealed network — the download is served, never skipped. The libsignal aarch64 native is compiled at `build-infra` time from the module's own recipe (`packages/e2e/scripts/stage-libsignal.ts` → `modules/signal/build/`) and staged into the apt-repo pool, so the recipe is exercised for real on every rebuild while the deploy stays fast and the network stays sealed.
 - **MCP server (agent-driven e2e)** — `packages/mcp-server/src/index.ts` — stdio MCP server exposing `start_run`/`run_status`/`run_result`/`stop_run` (detached cele2e runs read off the event bus, no ANSI scraping) + `env_check` (docker VM, run-lock, shared-infra, mgmt image CLI version, netapps). Dev/ops tool, `private`, not shipped to consumers.

package/docs/ALERTING.md

@@ -0,0 +1,298 @@
+# Alerting — operator guide
+
+Getting celilo to tell you when something is wrong, on a channel you actually
+read, without teaching you to ignore it.
+
+This guide is task-ordered: set up who gets told, how they get told, and in
+what order; then watch a module; then handle an alert when it arrives.
+
+---
+
+## The shape of it
+
+```
+  monitor ──runs a check every N minutes──▶ alert ──escalation policy──▶ route ──▶ your phone
+     │                                        │                                      │
+     └─ a module's health_check hook,          └─ one per PROBLEM, not per run,       └─ you reply
+        or a built-in (machines_reachable)        identified by a stable key             `ack`
+```
+
+Four nouns, and they are worth keeping straight:
+
+| noun | what it is |
+|------|-----------|
+| **person** | a human, with a timezone and optional quiet hours |
+| **route** | one way to reach a person (a Signal number). A person may have several |
+| **escalation policy** | an ordered list of *steps*: which route, how many minutes in |
+| **monitor** | a check that runs on a schedule and produces alerts |
+
+An alert's identity is its **key** — `module:caddy/check:cert-validity`. The
+same problem across a hundred runs is one alert, so a slow-burning failure
+does not become a hundred messages.
+
+---
+
+## 1. Add yourself
+
+```bash
+celilo person add peba --timezone America/Los_Angeles
+```
+
+Quiet hours are optional and per-person. They defer the *message*, never the
+escalation clock — steps keep advancing while you sleep, so if nobody
+acknowledges by 03:00 the chain has already reached whoever is awake:
+
+```bash
+celilo person add peba --timezone America/Los_Angeles \
+  --quiet-hours 22:00-07:00
+```
+
+A policy may be marked to page through quiet hours anyway
+(`escalation_policies.bypass_quiet_hours`); `celilo escalation-policy list`
+shows which do. Use it sparingly — a channel that wakes you for a warning is a
+channel you will mute.
+
+## 2. Add a route
+
+A route needs a transport module. Deploy `signal` first (see
+[the signal module's README](../../../modules/signal/README.md)); then:
+
+```bash
+celilo route add peba signal --address +15551234567 --can-ack
+```
+
+`--can-ack` says replies from this address are trusted to acknowledge alerts.
+A route without it is send-only — it still gets paged, it just gets no reply
+token, because a reply instruction nobody can follow is worse than none.
+
+`--severity-floor critical` keeps a route out of everything below critical.
+That is how "hazel's phone is for real emergencies only" is expressed.
+
+```bash
+celilo route list
+```
+
+## 3. Build an escalation policy
+
+Steps are `--after` minutes from when escalation *begins*, not from the
+previous step — so the numbers read as a timeline rather than an accumulation.
+A step names a person and a transport, which together identify the route.
+
+```bash
+celilo escalation-policy add oncall
+celilo escalation-policy step oncall peba signal --after 0
+celilo escalation-policy step oncall hazel signal --after 15
+```
+
+That is: page me immediately; if nobody has acknowledged 15 minutes later,
+page hazel. A step whose route's severity floor excludes the alert is
+skipped *immediately* rather than costing its delay in silence.
+
+## 4. Watch something
+
+A module that declares `hooks.health_check` gets a monitor automatically on
+its first deploy. To see what is watched:
+
+```bash
+celilo monitor list
+```
+
+To add one by hand, and to point it at a policy:
+
+```bash
+celilo monitor add caddy --interval 5m
+celilo monitor add machines_reachable --interval 5m
+celilo escalation-policy assign oncall caddy
+```
+
+The target says which kind it is: an audit category (`machines_reachable`,
+`health_coverage`) is a built-in check, anything else is a module's
+`health_check` hook. `celilo monitor run <target>` runs one now.
+
+`machines_reachable` is deliberately **unsuppressible**: it is how you find
+out the fleet is unreachable, so nothing may explain it away.
+
+Modules with no health check at all show up under the built-in
+`health_coverage` check — "not observed" is a state worth being able to see.
+
+---
+
+## Living with it
+
+### What you will see
+
+```bash
+celilo alerts list
+celilo alerts list --json     # for scripts
+celilo module list            # health column, per module
+```
+
+The health column reads `healthy` / `failing(n)` / `suppressed` / `not observed`.
+"not observed" is not "healthy" and is not printed as if it were.
+
+### When one arrives
+
+The message names the key first — on a phone that is what shows in the
+notification, and half-awake you need to know *what* before *why*.
+
+```
+🔴 module:caddy/check:cert-validity
+certificate for www.example.com expires in 2 days
+
+reply K3M9QX
+```
+
+Reply with the token, or just `ack` if only one alert is outstanding for you.
+Everyone else who was paged is told **who** took it, so the secondary can go
+back to bed.
+
+From a terminal:
+
+```bash
+celilo alerts ack module:caddy/check:cert-validity --as peba
+```
+
+### ack vs silence vs resolve
+
+These look similar and mean entirely different things. Collapsing any two is
+how an alerting system stops being trusted.
+
+| | means | escalation | alert state |
+|---|---|---|---|
+| **ack** | "I have this" | stops | still **firing** — the problem is still happening |
+| **silence** | "I know, stop telling me until X" | stops until expiry | firing, silenced |
+| **resolve** | "the condition is gone" | stops | resolved |
+
+```bash
+celilo alerts silence module:caddy/check:cert-validity --for 2h
+celilo alerts resolve module:caddy/check:cert-validity
+```
+
+`celilo monitor disable <target>` is the bigger hammer: it stops the check
+running at all, rather than silencing what it finds.
+
+Silence always expires — that is what makes it safe to offer.
+
+### What `resolve` is actually for
+
+Reasonable question: if the check is still failing, what does resolving it
+achieve? Nothing — and that is the point. **Resolve does not override the
+monitor.** The next successful run is authoritative, so if the condition is
+still there the alert comes straight back on the next sweep.
+
+Its use is narrower than it looks: **you have just fixed something, and the
+monitor's interval is fifteen minutes.** Without resolve you stare at a stale
+`firing` row — and keep escalating against it — until the next run notices.
+Resolve says "I know this one is done, stop counting" and costs nothing if you
+are wrong, because the monitor corrects you.
+
+If you find yourself resolving the same key repeatedly, the check interval is
+too long or the check is wrong. That is the signal, not the resolve.
+
+### Replies can only acknowledge
+
+The reply grammar is deliberately one verb. A bare token acknowledges; so does
+`ack`, `ok`, `k`, or 👍:
+
+```
+K3M9QX            → acknowledged
+K3M9QX ack        → acknowledged
+K3M9QX resolve    → REJECTED, and nothing happens
+```
+
+`resolve` and `silence` are terminal operations, not reply verbs — every extra
+verb is another thing to mistype at 3am, and a reply that silently did the wrong
+thing would be worse than one that did nothing. An unrecognised verb is
+rejected and counted, so `celilo alerts poll` reports it rather than swallowing
+it.
+
+### Why didn't I get paged?
+
+Every reason to stay silent is recorded rather than being an early return.
+
+```bash
+celilo alerts list --json | jq '.[] | {key, state, suppressedBy: .suppressedByAlertId}'
+celilo monitor run caddy        # run one check now, see what it reports
+```
+
+The usual answers:
+
+- **within grace** — a new failure waits out a grace window before it may page,
+  so a check that flaps for thirty seconds never reaches you.
+- **suppressed** — something upstream explains it. A firing machine explains
+  every module on it; a firing capability provider explains its zone's
+  consumers. This is derived from the deployment topology, never configured,
+  so it cannot drift from reality.
+- **deploy window** — a deploy suppresses its own module. Restarting a service
+  fails its own health check, and paging you about the deploy you are
+  personally running is how a channel gets muted.
+- **awaiting confirmation** — suppression just lifted. The alert waits for one
+  more successful run before paging: if it recovered along with its cause, it
+  resolves quietly.
+- **quiet hours** — deferred, not dropped. It arrives when the window ends,
+  *if* it is still a problem then.
+
+### What happens when the check itself breaks
+
+A check that could not **run** tells you nothing about what it was watching.
+So an errored run resolves nothing — every alert it owns is frozen exactly as
+it was — and raises one module-level alert saying the check is broken.
+
+The alternative is the single most dangerous failure an alerting system can
+have: reading "could not run" as "found nothing wrong", clearing every alert,
+and telling you everything is fine at the moment you can see least.
+
+---
+
+## Running it
+
+Two commands, both dispatcher-driven, both safe to run by hand:
+
+```bash
+celilo alerts sweep    # timer.tick.5m — run due monitors, escalate, notify
+celilo alerts poll     # read replies off every bidirectional transport
+```
+
+`poll` runs far more often than the sweep: "I texted ack and nothing happened"
+is the experience the whole design exists to prevent.
+
+celilo-mgr polls the transport; nothing calls in. That direction is
+deliberate — an inbound webhook would need public HTTPS, which means caddy,
+DNS and a certificate, so the ack path would depend on the very infrastructure
+being paged about.
+
+---
+
+## Answering deploy questions from your phone
+
+Because an alert and an interview question are the same message with different
+reply semantics, the same transport carries both. A headless deploy that needs
+a value can ask you:
+
+```
+❓ caddy needs: admin_email
+
+reply K3M9QX <value>
+```
+
+Two rules, both enforced before anything is sent:
+
+1. **A terminal wins.** If stdin is a TTY, the question is answered there — a
+   deploy started from your laptop must not appear to hang while silently
+   waiting for a text.
+2. **Secrets are refused.** `secret.*` questions never travel over a messaging
+   transport, on a headless box or anywhere else. Typing a credential into a
+   chat puts it in the transport's message store and the device's history
+   before the responder ever sees it. Answer those at a terminal.
+
+---
+
+## See also
+
+- **Rolling this out to a live fleet**: `openspec/changes/add-alerting/ROLLOUT.md`
+  — ordered stages, what to verify at each, and how to back out.
+- **Building the libsignal aarch64 native** (needed on ARM hosts):
+  `modules/signal/build/README.md`
+- Design and the decisions behind it: `openspec/changes/add-alerting/design.md`
+- Subsystem map: [CELILO_SUBSYSTEMS.md](../CELILO_SUBSYSTEMS.md) § Alerting & notifications
+- The Signal transport: `modules/signal/README.md`

package/docs/INDEX.md

@@ -0,0 +1,103 @@
+# Celilo CLI Docs
+
+Canonical user-facing docs live at https://celilo.computer/docs. This
+file is the offline-readable summary for engineers working in the repo.
+
+## Alerting
+
+See [ALERTING.md](./ALERTING.md) — people, routes, escalation policies,
+watching a module, and what to do when an alert arrives.
+
+## System update runbook
+
+Canonical: https://celilo.computer/docs/system-update
+
+### Philosophy
+
+`celilo system update` is the single command for bringing a deployment
+back to "READY" — the audit-determined state where every drift category
+is clean. It is built on three guarantees:
+
+1. **Audit first.** `update` runs `system audit` as its first step and
+   refuses to proceed if the verdict is `BLOCKED`. The audit is also
+   exposed as a standalone read-only command (`system audit`) for use
+   in cron, CI, or interactive checks.
+2. **Backups before mutation.** Every module that runs through the
+   update flow gets a fresh backup tagged with the run's `update_id`
+   before any change is applied. `--no-backup` skips this and is the
+   only way to opt out.
+3. **Failures isolate to the dependency subtree.** The orchestrator
+   walks modules in topological order. A failure on a provider skips
+   only its transitive consumers — peers and unrelated modules continue.
+   Consumers may also proceed if their declared `requires` version
+   still satisfies the running provider (version-aware skip).
+
+### Recommended sequence
+
+```
+celilo system audit            # cheap, read-only — run any time
+celilo system update --dry-run # preview the plan
+celilo system update           # do it
+```
+
+For a single module:
+
+```
+celilo system update --module <id>
+```
+
+For an audit that returns `BLOCKED` on a destructive Terraform plan:
+
+```
+celilo system update --allow-destructive   # only after reviewing the plan
+```
+
+### Rollback
+
+Pre-update backups are tagged with the `update_id` printed in the
+`system update` output. To roll back a single module:
+
+```
+celilo backup list --module <id>
+celilo backup restore <backup-id>
+```
+
+System-state snapshots (the celilo SQLite DB) are taken before the
+first module mutation in a run; restore them via the same flow with
+the system-scoped backup id.
+
+### FAQ
+
+**Q. Why did `system update` skip module X?**
+Likely one of:
+- `system audit` reported no drift for X (already at latest).
+- A provider that X depends on failed earlier in the run, AND X's
+  `requires` declares a version newer than the still-running provider.
+  Output line `skipReason` makes the cause explicit.
+- `--module <id>` was passed and X wasn't the named module.
+
+**Q. The audit says `capability_abi: BLOCKED` — what now?**
+A module's manifest declares a capability version that the running
+`@celilo/capabilities` SDK no longer supports (or vice versa). Either
+upgrade the framework (`bun update -g @celilo/cli` then re-run `system
+update`) or rebuild the module against the new SDK and republish.
+`CAPABILITY_CONTRACT_VERSIONS` in `packages/capabilities/src/capability-contract.ts`
+is the source of truth.
+
+### Troubleshooting
+
+**Schema migrations fail on update.** `system audit` checks pending
+migrations as part of `schema` drift. If a migration is failing on
+your DB, `bun run drizzle-kit migrate` from `apps/celilo/` will print
+the error directly.
+
+**Backups storage is full.** Pre-update backups live in their own
+retention pool tagged with the `update_id`. They are pruned by
+`backup retention` policy independently from regular backups. Inspect
+with `celilo backup list --type pre-update`.
+
+## Related
+
+- Design doc: `apps/celilo/designs/CELILO_UPDATE.md`
+- Audit categories: `apps/celilo/src/services/audit/`
+- Orchestrator: `apps/celilo/src/services/update/`

package/drizzle/0016_trusted_sources.sql

@@ -0,0 +1,10 @@
+CREATE TABLE `trusted_sources` (
+	`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
+	`firewall_ip` text NOT NULL,
+	`subnet` text NOT NULL,
+	`description` text DEFAULT '' NOT NULL,
+	`registered_by` text NOT NULL,
+	`created_at` integer DEFAULT (unixepoch()) NOT NULL
+);
+--> statement-breakpoint
+CREATE UNIQUE INDEX `trusted_sources_unique_idx` ON `trusted_sources` (`firewall_ip`,`subnet`);