@celilo/cli 1.8.0 → 1.9.1

package/CELILO_CORE_MODULES.md

@@ -39,6 +39,7 @@ Each entry: `module id` — what it is — **provides** / **requires** capabilit
 - **iptables** — iptables firewall + NAT; cross-VLAN port exposure with recursive upstream delegation. Converge model: `exposeService` registers into the shared-core port-forward registry, then renders the complete ruleset and applies it atomically via `iptables-restore` (default-DROP FORWARD + coarse zone-tier matrix; SSH-free). **REQUIRES the networks it has legs on and defines none of them** (`requires.networks: [{from: $self:zones}]`) — the required set is resolved from the same `zones` value that says which NICs the box has, so the legs and the declared networks are ONE list. They used to be two: `zones` and a `provided_networks` array this module wrote into system config from its install hook, with nothing forcing them to agree. A leg whose network has no declared subnet classifies `alien`, and on a firewall with a recorded baseline an alien interface is DISABLED — so under-declaring was never cosmetic. `firewall-interface-classification` §3/D8 reached that conclusion for the test harness; `networks-are-declared-not-written` applies it in production and deletes `provided_networks` outright (breaking: iptables 3.0.0). celilo supplies each range, asking the operator when it holds none. **provides:** `firewall`.
 - **dnsmasq-dhcp** — a DHCP-only dnsmasq in **`internal`**, providing the zone-scoped `dhcp_server` capability for a network with no upstream router to serve it. **provides:** `dhcp_server` (zone `internal`). The other two `dhcp_server` providers, **axon** and **greenwave**, do not run a DHCP server at all — they drive an ISP router's built-in one over its API, which works only while the home network shares a broadcast domain with that router. Put `internal` behind the celilo firewall and DHCP discovery, being a broadcast, no longer reaches it: celilo could describe that topology and not make it work. Importing this module at all depends on well-known-provider uniqueness being scoped PER ZONE (celilo#1028), since axon/greenwave already claim `dhcp_server` fleet-wide. **`port=0` is load-bearing**: dnsmasq ships a resolver as well as a DHCP server, and `dns_internal` is a different capability with its own providers — two resolvers answering on one network is a split-brain that is very hard to see from outside. celilo's transactional artifacts go to `/var/lib/celilo/config-state/` via `applyRenderedConfig`'s `statePath`, NOT beside the managed fragment, because dnsmasq parses every file in `/etc/dnsmasq.d` and a `.celilo-bak` there is configuration the daemon reads. It declares a `requires.system` and is placed by celilo like any other module; an earlier shape declared none, which meant no host in the inventory, an `apt-get` shelled over SSH from a hook, and an operator-typed address — and which also evaded `validateMachineRoleForModule`, the check that refuses to place a non-firewall module on a multi-interface router.
 - **knot-unbound-internal** — split-horizon internal DNS via Knot (authoritative) + Unbound (recursive); lightweight, plain apt, no .NET. **provides:** `dns_internal`. Ships a base-module-aspect (`modules/knot-unbound-internal/base-module-aspect/`).
+- **knot-unbound-secondary** — a read-only secondary for **knot-unbound-internal**: Knot transfers every managed zone from the primary and Unbound serves the same split-horizon view. Lives in **`dmz`**, symmetric with the primary (design D5e) — NOT in `internal` as first written. That placement was chosen deliberately and the reasoning is worth keeping: design D5a records the secondary's justification as weak on purpose, since steady state is already a single resolver address and the one recurring non-bug case a secondary covers is redeploying the primary, which blanks fleet DNS for the duration. Same-zone covers that completely. A split placement buys zone-level independence, which D5a explicitly declined, and would not deliver it anyway — celilo has no host anti-affinity, so both LXCs land on the container service's `default_target_node`. **provides:** `dns_internal_secondary`, carrying its OWN addresses rather than being advertised by the primary: if the primary carried them its published data would be stale from the moment this module deployed until something recomputed it, and recomputing means redeploying the primary — the manual step D5 exists to remove. **requires:** `dns_internal` (hard, and honestly declared: it sat under `optional` until celilo#854, because the import-time secret gate refused a consumer over a `tsig_key` it never reads). Its protected-zone ingress is a `*_ingress_ip` / `source: infrastructure` variable that `ensureIngressIps` allocates from `network.internal.subnet`, so the operator types none of the three addresses the fork's version asked for. Registering it as a transfer peer on the primary is `registerTransferPeer` (design D5f) — ⚠️ the empty peer state MUST render an explicit `deny: on`, because a knot acl with `action: transfer` and no `address:` matches ANY source and `knotc conf-check` accepts that silently.
 - **technitium** — internal split-horizon DNS resolver + authoritative server (web UI + HTTP API); heavier alternative to knot-unbound. **provides:** `dns_internal`. Ships a base-module-aspect (`modules/technitium/base-module-aspect/`).
 - **namecheap** — public DNS A-record management via Namecheap Dynamic DNS API (HTTP, no browser automation). A caller supplies a NAME and nothing else: the address is the source IP of celilo's own update, re-derived on every assert. Registering `<domain>` also claims `www.<domain>` and vice versa (best effort, reported back as `outputs.companion_fqdn` so the framework's `public_dns` check watches it — Namecheap answers `ErrCount 0` for `www` updates it does not apply). DDNS passwords are keyed by the **registrable domain**, never the FQDN. **provides:** `dns_registrar`.
 - **wireguard** — owns the admin WireGuard tunnel on the firewall host: interface, listen port, peers (as records), and client subnet are module config rather than hand-maintained state. Exposes the listen port and **registers the client subnet as a trusted source**, so VPN reach into the managed zones is in the firewall registry and every converge re-emits it. **REQUIRES the `control-plane-vpn` network and READS its range** (`requires.networks`; `client_subnet` is a `source: system` derive of `network.control-plane-vpn.subnet`). It does not write that network and has no way to — celilo owns the namespace, and the deploy will not reach any hook until the network is defined, asking for a range if one is missing. So `wg0` is attributable the moment it exists. This replaces a declare-before-you-create ordering inside `on_install`, which could only narrow the window and not close it: a consumer that captured config before the hook started could not see a value the hook wrote, whatever order it wrote it in, and that is exactly what left `wg0` unattributable (celilo#759). `health_check` still asserts the declaration matches what the tunnel serves — what it catches now is divergence, celilo's network having changed since the module resolved its config. That same key is what the internal resolver's split-horizon view consumes. Adopts a running tunnel in place (existing key retained; `wg syncconf`, never `wg-quick down`) because that tunnel is the operator's recovery path. **Adoption is a one-time IMPORT and it ENDS.** The marker is `registered_peers` — celilo's own config key, separate from the operator's `peers` so a machine can never rewrite what an operator typed. While that key has never been written the tunnel is not yet celilo's; the deploy that writes it (even as `[]`) CLAIMS the tunnel, importing whatever `[Peer]` blocks are running, and from then on the render is closed-world (`peers` ∪ `registered_peers`) with a `[Peer]` on the box in neither reported as drift by `on_install` and by `health_check`'s `unknown_peers`. An ABSENT `peers` declaration and a declared-EMPTY one are deliberately different (no `default: []` on either variable): they used to be the same value, so the module read every empty list as "adopt what is running", the last peer could not be revoked, and a hand-added peer rode along on every deploy unreported (celilo#765). On a tunnel celilo has not yet claimed, a declared-empty `peers` is REFUSED rather than obeyed — celilo's own variable-default seeding wrote a real `peers = []` row for every install of the previous version that never set one, so on the installed base an empty list cannot be told from a stored default, and obeying it would `wg syncconf` the admin tunnel down to zero peers. **PROVIDES `control_plane_vpn`** so another module can enrol clients without an operator editing YAML: `registerClient` / `revokeClient` write to `registered_peers` — the same key the adoption claim uses — and `getEndpoint` reads the server's public key LIVE off the host rather than storing a copy that could be republished after the key changed. `client_pool` is the range a consumer may allocate from, a strict subset of the client subnet, and `validate_config` REFUSES any operator-declared peer inside it: the two allocators cannot see each other, so the range is divided rather than negotiated, and that check is the only place a bad division is caught. ⚠️ Granting this capability grants fleet-wide reach — the tunnel's client subnet is a registered trusted source, so every client enrolled through it reaches every managed zone. **requires:** `firewall` (and the provider must support trusted-source registration — `iptables` does; the ISP-router drivers `greenwave` and `axon` do not).
@@ -74,6 +75,7 @@ Each entry: `module id` — what it is — **provides** / **requires** capabilit
 ## E2E fixtures & probes (not production apps)
 
 - **hello-foo** / **hello-bar** — hostless single-page sites (one hand-written `site/index.html`, no build step) published at a configurable `domain` + `path`. They exist so the caddy route-lifecycle e2e can put several independent routes on one caddy and watch them interact — celilo-website is the only other static-site module, and one site can't answer "does removing `/` break `/foo`?". **requires:** `public_web`, `dns_registrar`.
+- **hello-trespass** — TEST FIXTURE, never deploy. Its `on_install` hook attempts three things a hook has no business doing (read the master key at its default location, write into a sibling module's install tree, read the SSH private key / `remoteExec` another module's system) and REPORTS which succeeded rather than failing. It is the recurrence gate for `openspec/changes/hook-process-boundary`: each stage flips exactly one row and the test asserts the rows that stage does NOT claim still succeed, so a gate cannot quietly grow to cover work that was never done. **requires:** nothing.
 - **hello-private-foo** / **hello-private-bar** — the `private_web` twins of the pair above: hostless single-page sites published through the fleet-only ingress. Two of them for the same reason — one consumer cannot answer "does withdrawing `/foo` leave `/bar` alone?", and per-consumer scoping is exactly what `unregisterRoutes()` promises. Their hook is `private_web ?? public_web` with no branching after it, which is the spec's "same consumer code, either ingress" requirement made executable. **hello-private-foo** additionally calls `getCaCertificate()` and writes the PEM into its own site before publishing — the CA-bundling pattern a real consumer uses, and the only way to observe that the capability returned something usable. **requires:** `private_web`.
 - **source-forge-probe** — hostless e2e probe that exercises the `source_forge` capability (issue/revoke runner token) against a deployed Forgejo. **requires:** `source_forge`.
 - **test-ansible-output** — e2e fixture that runs 25×1s sleep tasks to assert Ansible output streams in real time. Not a real app.

package/CELILO_SUBSYSTEMS.md

@@ -47,6 +47,7 @@ see `openspec/specs/`. Companion doc: [CELILO_CORE_MODULES.md](./CELILO_CORE_MOD
 - **Define a capability function** — `packages/capabilities/src/define-capability-function.ts` — `defineCapabilityFunction`.
 - **Canonical capability registry** — `packages/capabilities/src/capability-registry.ts` — `KNOWN_CAPABILITY_NAMES` (the authoritative list), `CapabilityRegistry` type. Public surface: `packages/capabilities/src/index.ts`.
 - **Loader (wires provider factories into hook contexts)** — `apps/celilo/src/hooks/capability-loader.ts` — `loadCapabilityFunctions`, `resolveFirewallNatIp`.
+- **Which provider a consumer actually resolved to (`capability_bindings`)** — `apps/celilo/src/services/capability-bindings.ts` — `recordCapabilityBinding`, `listCapabilityBindings`, `withBindingRecord`, written from the single return of `loadCapabilityFunctions`. The `capabilities` table is provider-side only, so celilo could answer "who COULD provide this" and never "who does this module actually use"; every caller needing the second question rebuilt the same approximation (the consumer's `requires` + `optional` crossed with `capabilities`), which names every provider a module MIGHT have bound to. `tango-nexus` declares four optional capabilities, is deployed against one off-fleet host, and that approximation carries three edges that do not exist. **The CALL is the binding, not the resolution** — the loader injects every registered capability regardless of what the consumer declared (its own comment: "not just required ones"), so recording what it resolves is a SUPERSET of even the permissive set. A consumer's hook invoking a method is the only event that separates the optional it uses from the ones it merely declares, which is why this matters most for the capabilities that mint nothing (`external_web`, `idp`, `source_forge`, `registry_publish`, `control_plane_vpn`, `apt_publish`) and therefore leave no artefact anywhere else. Recorded on ATTEMPT rather than success: a consumer that called a provider and got an error is standing on that provider, and hiding the edge exactly when the call fails hides it when it matters most. Unique on (consumer, capability) so a redeploy re-asserts and a provider swap rewrites in place; `bound_at` is LAST-SEEN, so a binding last exercised forty deploys ago is a queryable fact rather than a silent lie. Cascades on the consumer. It runs BESIDE `planConsumerCleanup`, never narrowing it — cleanup must still reach every provider that might hold minted state, including one whose binding row was never written. See celilo#1072.
 - **Ledger wrappers (stateful capabilities)** — `apps/celilo/src/services/dns-registrations.ts` (`withDnsRegistrationLedger`), `apps/celilo/src/services/dns-internal-records.ts` (`withDnsInternalLedger`).
 - **Public DNS ledger (`dns_registrations` + `dns_registration_consumers`)** — `apps/celilo/src/services/dns-registrations.ts` — records WHICH MODULE ASKED FOR WHICH NAME, and deliberately **no address**: the published value is source-detected on every assert and observable on demand from public DNS, and a stored copy is one careless read away from becoming an instruction again (celilo#626 — the 15-minute `refresh_registrations` replayed addresses written by an older celilo and took five public names dark for nine days while every in-fleet check stayed green). Consumers are a SET, not a column: the FK cascade on a single `consumer_module_id` decided when a LIVE record was forgotten, so a row now survives until its LAST consumer is removed (`openspec/changes/public-dns-reachability/design.md` D5). Orphaned rows are pruned on read (`pruneOrphanedRegistrations`) rather than by trigger — SQLite fires delete triggers for FK-cascaded deletes only with `recursive_triggers` on. `companion: true` marks a name celilo claimed on a module's behalf (`www.<domain>` ↔ `<domain>`) rather than one a module asked for; the provider reports it as `outputs.companion_fqdn` whether or not the claim reported success, because a provider's success response is not evidence of publication. Operator surface: `celilo dns registrations`.
 
@@ -179,6 +180,7 @@ runner seam (`execRunner` real / `createMockRunner` for tests) lives in
 ## Hooks & deploy
 
 - **Hook executor / ABI** — `apps/celilo/src/hooks/executor.ts` (`invokeHook`, `executeHookScript`, `checkRequiredCapabilities`), types in `apps/celilo/src/hooks/types.ts` (`HookContext`, `HookDefinition`, `HookName`). Named-hook runner: `apps/celilo/src/hooks/run-named-hook.ts`. Manifest hook config: `apps/celilo/src/hooks/load-hook-config.ts`.
+- **Hook process boundary (a hook is a program celilo RUNS)** — `apps/celilo/src/hooks/hook-protocol.ts` (the NDJSON frame union, `HOOK_PROTOCOL_VERSION`, `serializeError`/`deserializeError`), `apps/celilo/src/hooks/broker.ts` (`startBroker`, `capabilityShape`), `apps/celilo/src/hooks/hook-runner.ts` (the spawned shim — the ONLY thing that `import()`s module code). `executeHookScript` spawns `bun hook-runner.ts` over a Unix socket instead of importing; the nine `invokeHook` call sites and `defineHook` are unchanged. **The broker does not know what a capability is**: it sends a shape descriptor built by the same own-string-key walk `wrapWithLogging` does (functions → `methods`, everything else → `data`, which is where `stampProvider`'s `providerModuleId` lives), and the shim rebuilds forwarding proxies from it — so an optional method a provider did not implement is absent rather than present-and-throwing, and `if (cap.registerTrustedSource)` keeps answering correctly. A socket rather than stdout because module scripts spawn subprocesses and a grandchild writing to fd 1 would corrupt the frame stream. Two consequences worth knowing: the child's environment is an **allow-list** (`hookChildEnv` — `PATH`, `HOME`, `LANG`, `TZ`, `TMPDIR`, `CELILO_HOOK_*`, `CELILO_DEBUG`), so a hook reading any other operator variable now gets `undefined`; and a timeout is a real SIGTERM-then-SIGKILL with the broker refusing further capability calls, replacing a `Promise.race` that cancelled nothing and let a "timed out" hook go on writing DNS and firewall state (celilo#1003). Capability PROVIDER factories still load in-process — they ARE the broker's implementation. Stage 1 of `openspec/changes/hook-process-boundary`; it claims the environment and the process, not the filesystem (a hook can still compute the default master-key path) and not the network.
 - **Deploy pipeline** — `apps/celilo/src/services/module-deploy.ts`.
 - **DNS provider backfill** (re-emit registrations when a provider deploys) — `apps/celilo/src/services/dns-provider-backfill.ts` — `isDnsInternalProvider`, `backfillProviderDns`.
 - **Base-module aspects (fan-out across the fleet)** — `apps/celilo/src/services/aspect-runner.ts` — `planAspectFanOut`, `runAspectFanOut`, `maybeRunAspectForTrigger`. Aspect content lives in `modules/<m>/base-module-aspect/` (e.g. knot-unbound-internal, technitium). Two directions, with DELIBERATELY OPPOSITE failure semantics:

package/drizzle/0028_capability_bindings.sql

@@ -0,0 +1,26 @@
+-- Which provider a consumer actually resolved to.
+--
+-- `capabilities` is provider-side only. It answers "who could provide this" and
+-- has never answered "who does this module actually use", so every caller that
+-- needed the second question rebuilt the same approximation: the consumer's
+-- `requires` + `optional` crossed with `capabilities`. That names every provider
+-- a module MIGHT have bound to. A module declaring four optional capabilities
+-- and using one gets three false edges and no way to tell which is real.
+--
+-- A row is written when a consumer's hook first calls a method on an injected
+-- capability. Unique on (consumer, capability) so a redeploy re-asserts rather
+-- than duplicates, and a provider swap rewrites the provider in place.
+--
+-- `provider_module_id` is a plain column with no foreign key, matching
+-- `dns_internal_records` after 0027: removing a provider must not erase the
+-- record that a consumer was bound to it.
+CREATE TABLE `capability_bindings` (
+	`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
+	`consumer_module_id` text NOT NULL,
+	`capability_name` text NOT NULL,
+	`provider_module_id` text NOT NULL,
+	`bound_at` integer DEFAULT (unixepoch()) NOT NULL,
+	FOREIGN KEY (`consumer_module_id`) REFERENCES `modules`(`id`) ON UPDATE no action ON DELETE cascade
+);
+--> statement-breakpoint
+CREATE UNIQUE INDEX `capability_bindings_consumer_capability_idx` ON `capability_bindings` (`consumer_module_id`,`capability_name`);

package/drizzle/0029_module_instances.sql

@@ -0,0 +1,58 @@
+-- Instances of a submodule (openspec/changes/submodules, D2 and D4).
+--
+-- An instance IS a row in `modules`, under a celilo-derived id. That is the
+-- decision this table exists to support rather than to work around: every
+-- deployment-state table in celilo keys on `module_id` and only on `module_id`
+-- (`module_systems` is PRIMARY KEY (module_id, name), `module_configs` is
+-- UNIQUE (module_id, key), `module_infrastructure` references it), so making an
+-- instance a module row means health, backup, fleet status, `module list` and
+-- the removal path all needed no change at all.
+--
+-- The alternative was adding an `instance_id` column to each of those tables and
+-- widening every key. It touches every reader and it fails QUIETLY: a query that
+-- still filters on module_id alone compiles clean and returns N rows where it
+-- expected one.
+--
+-- `module_id` is both the primary key and the FK, so an instance cannot exist
+-- without the module row it names, and dropping the module row drops this.
+--
+-- IDENTITY IS THE TRIPLE (parent_id, submodule, instance_key), never
+-- instance_key alone. Two parents may use the same key string and one parent may
+-- use the same key across two of its submodules; both are legal and neither
+-- collides. `instance_key` is opaque: celilo stores it and never interprets it,
+-- because a key derived from a display name orphans a running system the first
+-- time somebody is renamed at the identity provider.
+--
+-- `failure_reason` and `retryable` are NULL unless state = 'failed'. That is the
+-- Rule 3.4 exception (null meaning this whole section is absent) rather than a
+-- missing default. `retryable` is a column rather than something a caller infers
+-- from the reason text: a parent's reconcile loop reads it to decide whether to
+-- rebuild, and a missing interview answer will fail identically forever while a
+-- timed-out provision is worth another go. Forcing a caller to string-match the
+-- reason is how a reporter gets trusted by its shape and quietly answers wrong.
+
+CREATE TABLE `module_instances` (
+	`module_id` text PRIMARY KEY NOT NULL,
+	`parent_id` text NOT NULL,
+	`submodule` text NOT NULL,
+	`instance_key` text NOT NULL,
+	`label` text,
+	`state` text DEFAULT 'pending' NOT NULL,
+	`failure_reason` text,
+	`retryable` integer,
+	`created_at` integer DEFAULT (unixepoch()) NOT NULL,
+	`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
+	FOREIGN KEY (`module_id`) REFERENCES `modules`(`id`) ON UPDATE no action ON DELETE cascade,
+	FOREIGN KEY (`parent_id`) REFERENCES `modules`(`id`) ON UPDATE no action ON DELETE cascade
+);
+--> statement-breakpoint
+-- The real identity. A database fact rather than a check somebody has to
+-- remember, so "this parent already has an instance under this key" cannot be
+-- got wrong by a caller.
+CREATE UNIQUE INDEX `module_instances_identity_idx` ON `module_instances` (`parent_id`,`submodule`,`instance_key`);--> statement-breakpoint
+-- Every reconcile call lists one parent's instances. That must not become a
+-- table scan once a fleet carries tens of them.
+CREATE INDEX `module_instances_parent_idx` ON `module_instances` (`parent_id`);--> statement-breakpoint
+-- `list` filters by state, and the collapsed operator view rolls health up per
+-- parent, which reads the same way.
+CREATE INDEX `module_instances_state_idx` ON `module_instances` (`state`);

package/drizzle/meta/_journal.json

@@ -197,6 +197,20 @@
       "when": 1784300000000,
       "tag": "0027_dns_internal_records_consumer_cascade",
       "breakpoints": true
+    },
+    {
+      "idx": 28,
+      "version": "6",
+      "when": 1784400000000,
+      "tag": "0028_capability_bindings",
+      "breakpoints": true
+    },
+    {
+      "idx": 29,
+      "version": "6",
+      "when": 1784400000000,
+      "tag": "0029_module_instances",
+      "breakpoints": true
     }
   ]
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@celilo/cli",
-  "version": "1.8.0",
+  "version": "1.9.1",
   "description": "Celilo — home lab orchestration CLI",
   "type": "module",
   "bin": {
@@ -58,7 +58,7 @@
   "dependencies": {
     "@aws-sdk/client-s3": "^3.1109.0",
     "@aws-sdk/lib-storage": "^3.1101.0",
-    "@celilo/capabilities": "^2.6.0",
+    "@celilo/capabilities": "^3.1.0",
     "@celilo/cli-display": "^0.2.0",
     "@celilo/core": "^0.9.1",
     "@celilo/event-bus": "^0.6.0",

package/src/cli/commands/module-show.ts

@@ -165,6 +165,7 @@ export async function handleModuleShowZone(args: string[]): Promise<CommandResul
     // compile error here rather than silently rendering as "Unknown zone". Both
     // `internal` and `secure-mgmt` were missing from the previous string-keyed map.
     const zoneDescriptions: Record<NetworkZone, string> = {
+      'isp-transit': 'ISP transit (Private segment to the router upstream of the firewall)',
       internal: 'Internal (Semi-trusted network behind the firewall)',
       dmz: 'DMZ (Public-facing services)',
       app: 'Application (Internal services)',

package/src/db/foreign-keys.test.ts

@@ -0,0 +1,101 @@
+/**
+ * Recurrence gate for celilo#1074.
+ *
+ * `db/client.ts` runs `PRAGMA foreign_keys = ON` before handing back the real
+ * database. The test helpers did not, and SQLite defaults the pragma OFF per
+ * connection, so every one of the schema's `onDelete: 'cascade'` declarations
+ * was enforced in production and inert in the suite.
+ *
+ * The direction is what made it invisible. An unenforced cascade can only ever
+ * make a test pass that should have failed, so nothing has ever gone red over
+ * it and nothing ever could. `consumer-cleanup.ts`'s whole design reasons about
+ * what the cascade removes and when, and no test could observe any of it.
+ *
+ * Two assertions, because the pragma buys two different things: rows go away
+ * when their parent does, and a row referencing an absent parent is refused.
+ * The second is what stops a structurally impossible fixture from reading as
+ * valid.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from 'bun:test';
+import { mkdtemp, rm } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import {
+  cleanupTestDatabase,
+  setupTestDatabase,
+  setupTestDatabaseAt,
+  setupTestDatabaseFile,
+} from '../test-utils/database';
+import type { DbClient } from './client';
+
+describe('test databases enforce foreign keys', () => {
+  let db: DbClient;
+
+  beforeEach(async () => {
+    db = await setupTestDatabase();
+  });
+
+  afterEach(async () => {
+    await cleanupTestDatabase(db);
+  });
+
+  test('deleting a module takes its dependent rows with it', () => {
+    db.$client.run(
+      `INSERT INTO modules (id, name, version, source_path, manifest_data) VALUES ('caddy-consumer', 'consumer', '1.0.0', '/tmp/c', '{}')`,
+    );
+    db.$client.run(
+      `INSERT INTO web_routes (slug, module_id, type, path, hostname) VALUES ('s', 'caddy-consumer', 'reverse_proxy', '/', 'example.com')`,
+    );
+
+    db.$client.run(`DELETE FROM modules WHERE id = 'caddy-consumer'`);
+
+    const rows = db.$client
+      .query(`SELECT id FROM web_routes WHERE module_id = 'caddy-consumer'`)
+      .all();
+    expect(rows).toHaveLength(0);
+  });
+
+  test('a row referencing a module that does not exist is refused', () => {
+    expect(() =>
+      db.$client.run(
+        `INSERT INTO capabilities (module_id, capability_name, version, data) VALUES ('nonexistent', 'public_web', '1.0.0', '{}')`,
+      ),
+    ).toThrow();
+  });
+});
+
+/**
+ * Every way to get a test database, checked directly.
+ *
+ * `test-utils/database.ts` is now the only module that hands one out — the
+ * second one collided with it on two exported names and disagreed with it on
+ * this pragma, which is why the gap was invisible from any call site. A helper
+ * added here later must be added to this list; there is no longer another module
+ * for it to hide in.
+ */
+describe('every test-database helper enforces foreign keys', () => {
+  const enforced = (client: DbClient): boolean =>
+    (client.$client.query('PRAGMA foreign_keys').get() as { foreign_keys: number }).foreign_keys ===
+    1;
+
+  test('setupTestDatabase (in memory)', async () => {
+    const memory = await setupTestDatabase();
+    expect(enforced(memory)).toBe(true);
+    await cleanupTestDatabase(memory);
+  });
+
+  test('setupTestDatabaseFile (temp directory it owns)', async () => {
+    const { db: file, cleanup } = await setupTestDatabaseFile();
+    expect(enforced(file)).toBe(true);
+    await cleanup();
+  });
+
+  test('setupTestDatabaseAt (path the caller chooses)', async () => {
+    const dir = await mkdtemp(join(tmpdir(), 'celilo-fk-'));
+    const at = await setupTestDatabaseAt(join(dir, 'celilo.db'));
+    expect(enforced(at)).toBe(true);
+    at.$client.close();
+    await rm(dir, { recursive: true, force: true });
+  });
+});

package/src/db/schema.ts

@@ -153,6 +153,54 @@ export const capabilities = sqliteTable('capabilities', {
     .default(sql`(unixepoch())`),
 });
 
+/**
+ * Capability bindings — which provider a consumer actually resolved to.
+ *
+ * `capabilities` is provider-side only: it answers "who COULD provide this",
+ * never "who does this module actually use". Resolution happened at hook time
+ * and vanished into the generated project. Every consumer of that fact
+ * reconstructed the same approximation — the consumer's `requires` + `optional`
+ * crossed with `capabilities` — which names every provider a module MIGHT have
+ * bound to. For a module declaring four optional capabilities and using one,
+ * that is three false edges and no way to tell which.
+ *
+ * A row is written when a consumer's hook first CALLS a method on an injected
+ * capability, not when the loader resolves one. The loader deliberately injects
+ * every registered capability regardless of what the consumer declared
+ * (`loadCapabilityFunctions`, "not just required ones"), so resolution is a
+ * superset of the declared set and recording it would restate the
+ * approximation. The call is the binding.
+ *
+ * Unique on (consumer, capability): a redeploy re-asserts the row rather than
+ * duplicating it, and a provider swap rewrites `provider_module_id` in place.
+ * Cascaded on the consumer: the binding dies with the module that made it.
+ *
+ * This does NOT replace the permissive set. `planConsumerCleanup` must still
+ * notify every provider that might hold minted state, including ones this table
+ * has no row for.
+ *
+ * @owner celilo — cross-module bookkeeping of who resolved to whom (T8, beside `capabilities`)
+ */
+export const capabilityBindings = sqliteTable(
+  'capability_bindings',
+  {
+    id: integer('id').primaryKey({ autoIncrement: true }),
+    consumerModuleId: text('consumer_module_id')
+      .notNull()
+      .references(() => modules.id, { onDelete: 'cascade' }),
+    capabilityName: text('capability_name').notNull(),
+    providerModuleId: text('provider_module_id').notNull(),
+    /** Last time the consumer called into this provider. */
+    boundAt: integer('bound_at', { mode: 'timestamp' }).notNull().default(sql`(unixepoch())`),
+  },
+  (table) => ({
+    consumerCapabilityUnique: uniqueIndex('capability_bindings_consumer_capability_idx').on(
+      table.consumerModuleId,
+      table.capabilityName,
+    ),
+  }),
+);
+
 /**
  * Capability secrets table - stores encrypted secrets owned by capabilities
  * Values are encrypted with AES-256-GCM using master key
@@ -251,6 +299,100 @@ export const moduleIntegrity = sqliteTable('module_integrity', {
   updatedAt: integer('updated_at', { mode: 'timestamp' }).notNull().default(sql`(unixepoch())`),
 });
 
+/**
+ * Lifecycle states an instance moves through, reported by the deploy worker's
+ * `list` so a parent can compare desired against observed (design D6).
+ *
+ * `failed` and `pending` must stay distinguishable: a parent that cannot tell
+ * "has not started" from "will never work" either retries a bug forever or
+ * gives up on work that was merely queued.
+ */
+export const INSTANCE_STATES = [
+  'pending',
+  'provisioning',
+  'ready',
+  'failed',
+  'destroying',
+] as const;
+
+export type InstanceState = (typeof INSTANCE_STATES)[number];
+
+/**
+ * Instances of a submodule (openspec/changes/submodules, D2 and D4).
+ *
+ * An instance IS a row in `modules`, under a celilo-derived id, which is what
+ * lets every table and reader keyed on `moduleId` keep working unchanged:
+ * `module_systems`, `module_configs`, `module_infrastructure`, health, backup,
+ * fleet status and the removal path all needed no change. `moduleId` here is
+ * both this table's key and the FK to that row, so an instance cannot exist
+ * without the module row it names.
+ *
+ * This table carries only what a module row cannot express: who owns it, which
+ * submodule it came from, and the opaque key its parent knows it by.
+ *
+ * IDENTITY IS THE TRIPLE (parent, submodule, instanceKey), not `instanceKey`
+ * alone (D2). Two parents may use the same key string, and one parent may use
+ * the same key across two of its submodules; both are legal and neither
+ * collides. `instanceKey` is opaque — celilo stores it and never interprets
+ * it, because a key derived from a display name orphans a running system the
+ * first time somebody is renamed at the identity provider.
+ * @owner celilo — instance ownership and lifecycle state
+ */
+export const moduleInstances = sqliteTable(
+  'module_instances',
+  {
+    /**
+     * The derived `modules.id` this instance runs as, e.g. `byoi-lab-sdf82c1e`.
+     * Both this table's key and the FK, so the two cannot drift apart.
+     */
+    moduleId: text('module_id')
+      .primaryKey()
+      .references(() => modules.id, { onDelete: 'cascade' }),
+    /** The module that declared the submodule and created this instance. */
+    parentId: text('parent_id')
+      .notNull()
+      .references(() => modules.id, { onDelete: 'cascade' }),
+    /** The submodule name the parent declared — its directory under `submodules/`. */
+    submodule: text('submodule').notNull(),
+    /** Opaque, stable id supplied by the parent. Never interpreted. */
+    instanceKey: text('instance_key').notNull(),
+    /**
+     * Non-authoritative label for operator display. NOTHING may key on it, and
+     * changing it does not change identity — that is the whole point of D2.
+     */
+    label: text('label'),
+    state: text('state').$type<InstanceState>().notNull().default('pending'),
+    /**
+     * Why a `failed` instance failed. Null unless `state = 'failed'` — the
+     * legitimate "this section is absent" case rather than a missing default.
+     */
+    failureReason: text('failure_reason'),
+    /**
+     * Whether retrying could succeed. Set with `failureReason`.
+     *
+     * A parent's reconcile loop reads this to decide whether to rebuild. A
+     * missing interview answer (D7) is deterministic and will fail identically
+     * forever, while a timed-out provision is worth another go. Without the
+     * flag a caller has to string-match `failureReason`, which is the class of
+     * bug where a reporter is trusted by its shape and quietly answers wrong.
+     */
+    retryable: integer('retryable', { mode: 'boolean' }),
+    createdAt: integer('created_at', { mode: 'timestamp' }).notNull().default(sql`(unixepoch())`),
+    updatedAt: integer('updated_at', { mode: 'timestamp' }).notNull().default(sql`(unixepoch())`),
+  },
+  (table) => ({
+    // The real identity (D2). Makes "this parent already has an instance under
+    // this key" a database fact rather than a check somebody has to remember.
+    identity: unique().on(table.parentId, table.submodule, table.instanceKey),
+    // Every reconcile call lists one parent's instances, so that must not be a
+    // table scan once a fleet carries tens of them.
+    parentIdx: index('module_instances_parent_idx').on(table.parentId),
+    // `list` filters by state, and the collapsed operator view rolls up health
+    // per parent, which reads the same way.
+    stateIdx: index('module_instances_state_idx').on(table.state),
+  }),
+);
+
 /**
  * IPAM (IP Address Management) allocations table
  * Tracks VMID and IP address assignments per module
@@ -327,6 +469,12 @@ export const moduleBuilds = sqliteTable('module_builds', {
  * something declares `network.<zone>.subnet` — normally the module that supplies
  * the network. Defining a zone here costs nothing and activates nothing.
  *
+ * - isp-transit: the private segment between a celilo firewall and the router
+ *   upstream of it. Non-placement and non-allocatable, because the upstream
+ *   router is the address authority. It exists so a deployment CAN separate the
+ *   wire to that router from the workload network, not so every deployment
+ *   must: a firewall whose egress leg IS the workload network declares
+ *   `default_route_zone: internal` and behaves exactly as it does today.
  * - internal: the semi-trusted LAN the management server sits on
  * - dmz: public-facing services
  * - app: internal application services
@@ -345,6 +493,7 @@ export const moduleBuilds = sqliteTable('module_builds', {
  *   holds a leg on and must translate for.
  */
 export const NETWORK_ZONES = [
+  'isp-transit',
   'internal',
   'dmz',
   'app',
@@ -366,8 +515,9 @@ export type NetworkZone = (typeof NETWORK_ZONES)[number];
  * Zones an IP allocation or reservation can name: every NetworkZone whose
  * addresses celilo hands out.
  *
- * Two are excluded, for the same underlying reason — somebody else is the
+ * Three are excluded, for the same underlying reason — somebody else is the
  * address authority:
+ *   - `isp-transit`, whose addresses the upstream router assigns;
  *   - `external`, whose systems are addressed by the cloud/VPS provider;
  *   - `vpn`, whose client addresses are assigned by the module terminating the
  *     tunnel. celilo allocating into that subnet would collide with the VPN
@@ -378,17 +528,21 @@ export type NetworkZone = (typeof NETWORK_ZONES)[number];
  * cast in machine-pool.ts, and the cast had already drifted (it was missing
  * `secure-mgmt`, and its comment claimed the only difference was `external`).
  */
-export type AllocatableZone = Exclude<NetworkZone, 'external' | 'control-plane-vpn'>;
+export type AllocatableZone = Exclude<
+  NetworkZone,
+  'isp-transit' | 'external' | 'control-plane-vpn'
+>;
 
 /** The zones whose addresses celilo hands out, as a runtime list. */
 export const ALLOCATABLE_ZONES: AllocatableZone[] = NETWORK_ZONES.filter(
-  (zone): zone is AllocatableZone => zone !== 'external' && zone !== 'control-plane-vpn',
+  (zone): zone is AllocatableZone =>
+    zone !== 'isp-transit' && zone !== 'external' && zone !== 'control-plane-vpn',
 );
 
 /**
  * Is this a zone celilo allocates addresses in? Use this rather than testing
- * `zone !== 'external'` by hand — that check predates `vpn` and read as "the one
- * externally-addressed zone" when there are now two.
+ * `zone !== 'external'` by hand — that check predates the other externally
+ * addressed roles and silently makes them allocatable.
  */
 export function isAllocatableZone(zone: NetworkZone): zone is AllocatableZone {
   return (ALLOCATABLE_ZONES as string[]).includes(zone);
@@ -1409,3 +1563,5 @@ export type Alert = typeof alerts.$inferSelect;
 export type NewAlert = typeof alerts.$inferInsert;
 export type NotificationDelivery = typeof notificationDeliveries.$inferSelect;
 export type NewNotificationDelivery = typeof notificationDeliveries.$inferInsert;
+export type CapabilityBinding = typeof capabilityBindings.$inferSelect;
+export type NewCapabilityBinding = typeof capabilityBindings.$inferInsert;