@celilo/cli 1.7.0 → 1.8.0

package/CELILO_CORE_MODULES.md

@@ -37,6 +37,7 @@ Each entry: `module id` — what it is — **provides** / **requires** capabilit
 - **axon** — Axon Networks Q1000K ISP router driver (Brightspeed-branded); port-forwarding + public-IP discovery + DHCP DNS + DHCP address-pool bounds via the TR-181 CGI API. **provides:** `firewall`, `dhcp_server`. Fork of **greenwave** — identical protocol, differing only in the vendor extension prefix (`X_AXON_` vs `X_GWS_`/`X_LANTIQ_COM_`). Pick by device: Q1000K → **axon**, C4000XG → **greenwave**. Two device behaviours shape this driver and are not shared by **greenwave** (celilo#739): the router **regenerates** `Pool.1.DNSServers` from its own upstream resolver list every ~15 min and on any config commit, so `setDhcpDns` also writes `Device.DNS.X_AXON_CustomServer` — the *input* to that computation, which survives — and **technitium re-asserts** the pool value on `timer.tick.1m`. Optional `dhcp_pool_start`/`dhcp_pool_end` bound the leased range; set them below the addresses IPAM allocates, because **IPAM has no knowledge of the router's DHCP pool** and nothing otherwise stops the router leasing an address celilo already assigned.
 - **greenwave** — GreenWave C4000XG ISP router driver; port-forwarding + public-IP discovery via REST. **provides:** `firewall`, `dhcp_server`. Legacy device; new deployments on Axon hardware want **axon**.
 - **iptables** — iptables firewall + NAT; cross-VLAN port exposure with recursive upstream delegation. Converge model: `exposeService` registers into the shared-core port-forward registry, then renders the complete ruleset and applies it atomically via `iptables-restore` (default-DROP FORWARD + coarse zone-tier matrix; SSH-free). **REQUIRES the networks it has legs on and defines none of them** (`requires.networks: [{from: $self:zones}]`) — the required set is resolved from the same `zones` value that says which NICs the box has, so the legs and the declared networks are ONE list. They used to be two: `zones` and a `provided_networks` array this module wrote into system config from its install hook, with nothing forcing them to agree. A leg whose network has no declared subnet classifies `alien`, and on a firewall with a recorded baseline an alien interface is DISABLED — so under-declaring was never cosmetic. `firewall-interface-classification` §3/D8 reached that conclusion for the test harness; `networks-are-declared-not-written` applies it in production and deletes `provided_networks` outright (breaking: iptables 3.0.0). celilo supplies each range, asking the operator when it holds none. **provides:** `firewall`.
+- **dnsmasq-dhcp** — a DHCP-only dnsmasq in **`internal`**, providing the zone-scoped `dhcp_server` capability for a network with no upstream router to serve it. **provides:** `dhcp_server` (zone `internal`). The other two `dhcp_server` providers, **axon** and **greenwave**, do not run a DHCP server at all — they drive an ISP router's built-in one over its API, which works only while the home network shares a broadcast domain with that router. Put `internal` behind the celilo firewall and DHCP discovery, being a broadcast, no longer reaches it: celilo could describe that topology and not make it work. Importing this module at all depends on well-known-provider uniqueness being scoped PER ZONE (celilo#1028), since axon/greenwave already claim `dhcp_server` fleet-wide. **`port=0` is load-bearing**: dnsmasq ships a resolver as well as a DHCP server, and `dns_internal` is a different capability with its own providers — two resolvers answering on one network is a split-brain that is very hard to see from outside. celilo's transactional artifacts go to `/var/lib/celilo/config-state/` via `applyRenderedConfig`'s `statePath`, NOT beside the managed fragment, because dnsmasq parses every file in `/etc/dnsmasq.d` and a `.celilo-bak` there is configuration the daemon reads. It declares a `requires.system` and is placed by celilo like any other module; an earlier shape declared none, which meant no host in the inventory, an `apt-get` shelled over SSH from a hook, and an operator-typed address — and which also evaded `validateMachineRoleForModule`, the check that refuses to place a non-firewall module on a multi-interface router.
 - **knot-unbound-internal** — split-horizon internal DNS via Knot (authoritative) + Unbound (recursive); lightweight, plain apt, no .NET. **provides:** `dns_internal`. Ships a base-module-aspect (`modules/knot-unbound-internal/base-module-aspect/`).
 - **technitium** — internal split-horizon DNS resolver + authoritative server (web UI + HTTP API); heavier alternative to knot-unbound. **provides:** `dns_internal`. Ships a base-module-aspect (`modules/technitium/base-module-aspect/`).
 - **namecheap** — public DNS A-record management via Namecheap Dynamic DNS API (HTTP, no browser automation). A caller supplies a NAME and nothing else: the address is the source IP of celilo's own update, re-derived on every assert. Registering `<domain>` also claims `www.<domain>` and vice versa (best effort, reported back as `outputs.companion_fqdn` so the framework's `public_dns` check watches it — Namecheap answers `ErrCount 0` for `www` updates it does not apply). DDNS passwords are keyed by the **registrable domain**, never the FQDN. **provides:** `dns_registrar`.

package/CELILO_SUBSYSTEMS.md

@@ -32,7 +32,11 @@ see `openspec/specs/`. Companion doc: [CELILO_CORE_MODULES.md](./CELILO_CORE_MOD
 - **Network requirement + ensure (celilo owns the namespace)** — `apps/celilo/src/services/network-ensure.ts` (`ensureRequiredNetworks`), `NetworkRequirementSchema` / `getRequiredNetworkNames` in `apps/celilo/src/manifest/schema.ts`. A module declares `requires.networks: [{name}]` — a NAME, never a value; the schema is `.strict()` so a `subnet:` on the requirement is rejected with a message saying why. The deploy calls `ensureRequiredNetworks` in its interview phase, BEFORE generation and before any hook, and asks over the generic bus interview (`askText`, so it is answerable headless) for anything undefined. Which attributes a network has is celilo's answer, taken from `apps/celilo/schemas/system_config.json`: that file declares `network.<n>.gateway` for the routed segments and omits it for `control-plane-vpn`, so a gateway is never asked for a network that has none. Well-known names carry a `suggested` range there — deliberately NOT `default`, which `getDefaultConfiguration()` would seed at `system init`. Spec: `openspec/specs/network-declaration/spec.md`.
 - **The network write path (closed) + celilo's own discovery** — `celilo system apply-config` (`apps/celilo/src/cli/commands/system-apply-config.ts`) REFUSES the whole `network.` namespace, `network.bridge` excepted (a Proxmox bridge name is not addressing, and it is the one network key with a schema default). That is the automation surface a module hook shells out to, so closing it there is what makes "networks are celilo's" an authority rather than a convention every module has to remember. The refusal names the alternative — declare the network, read it with `$system:` — because a bare rejection sends a module author hunting for a typo. The one write that legitimately needed the surface moved INTO celilo: `celilo system discover-network` (`apps/celilo/src/services/network-discovery.ts`, `cli/commands/system-discover-network.ts`) parses the box's own `ip route` and records `network.internal.*` — or `network.secure-mgmt.*` when the box is off the internal LAN (#300). `celilo-mgmt` calls it and decides nothing; it used to parse and write this itself. Idempotent and never overwrites addressing already set. Recurrence gate: `test-integration/module/no-module-writes-networks.test.ts`.
 - **Firewall interface audit** — `apps/celilo/src/services/audit/interface-classification.ts` — reports per-firewall classification in `celilo audit`: alien interfaces by name and address (drift), and the blocking findings a converge refuses on — a carrier-grade NAT leg, an ambiguous external edge, a default route on the wrong leg.
-- **Zone taxonomy (canonical list)** — `apps/celilo/src/db/schema.ts` — `NETWORK_ZONES` is the single array; `NetworkZone` is DERIVED from it. Never hand-maintain a second copy: a duplicate that dropped a member made zone validation return null and silently fall back to a wrong-but-valid zone.
+- **Zone taxonomy (canonical list)** — `apps/celilo/src/db/schema.ts` — `NETWORK_ZONES` is the single array; `NetworkZone` is DERIVED from it. Never hand-maintain a second copy: a duplicate that dropped a member made zone validation return null and silently fall back to a wrong-but-valid zone. Two copies are unavoidable and BOTH are guarded: `modules/iptables`' `zones` picker (`test-integration/module/zone-options-cover-network-zones.test.ts` fails on any drift, and did when `isp-transit` was added to core alone) and the generated `schemas/module-manifest.schema.json` (`bun run check:schema`, run from `apps/celilo` — a FOURTH gate CI runs that the "Validation gates" list in CLAUDE.md does not name).
+  - **`isp-transit`** — the private segment between a celilo firewall and the router upstream of it. Non-placement and non-allocatable (it joins `external` and `control-plane-vpn` in the excluded set) because that router owns the addressing. It exists because on a DOWNSTREAM firewall the default route and the workload network share one interface, so `internal` meant both at once and every egress rule also reasoned about the workload network. A firewall with no separate transit leg declares `default_route_zone: internal` and behaves as it always did.
+- **Declared egress zone** — `modules/iptables` `default_route_zone`, consumed by `defaultRouteFinding` in `packages/capabilities/src/interface-classification.ts` — celilo already classifies whichever interface carries the default route; what it could not do is tell an INTENDED egress leg from a drifted one, so the comparison was hardcoded to "internal or external". Declaring the intent makes a wandered default route a finding. Defaults to `internal`.
+- **Capability-secret access is reference-driven** — `apps/celilo/src/capabilities/validation.ts` — `validateCapabilityAccess` fires on the `$capability:<name>.<secret>` paths the CONSUMER references, collected from its manifest AND from the templates `validateModuleTemplates` already parses at import. It used to iterate the secrets the PROVIDER declares, so requiring a capability that merely carries a secret put a module on the hook for that secret's `readable_by` list (celilo#854, celilo#1038). Reading a secret still goes through `checkCapabilitySecretAccess` at resolution time.
+- **Zone-first capability provider selection** — `apps/celilo/src/capabilities/lookup.ts` — one shared selection function used by both the direct lookup and the hook capability loader, which had drifted apart. Well-known-provider uniqueness is scoped per zone, which is what lets a second `dhcp_server` provider exist at all while `axon`/`greenwave` claim it fleet-wide.
 - **Control-plane network (`secure-mgmt`)** — `apps/celilo/src/hooks/capability-loader.ts` — `loadControlPlaneSubnet` returns the subnet of the zone `celilo-mgmt` is deployed in. `secure-mgmt` is a placement zone AND the control-plane tier, deliberately NOT in `ZONE_TIER_ORDER` (it is not part of the `dmz → app → secure` data-plane chain; it reaches every tier by trust). The firewall's `trustedSubnets` derives from this rather than assuming celilo-mgr sits on `internal`. Reported as an actionable gap by `checkControlPlaneNetwork` in `apps/celilo/src/services/fleet-checks.ts` when the management address matches no configured subnet.
 - **A deployed system's zone** — `apps/celilo/src/services/deployed-systems.ts` — for machine-pool deploys the zone recorded is the ZONE OF THE MACHINE, not `requires.system.zone` (which is only the minimum used to *select* a host, as with sizing). Three writers must agree: `recordDeployedSystemForModule`, `backfillModuleSystems`, and `apps/celilo/src/variables/context.ts` — the last runs latest and will overwrite the others.
 - **Host discovery ("which host serves module X?")** — `apps/celilo/src/cli/commands/module-where.ts` (`celilo module where <id> [--json]`, MCP `celilo_module_where`) — reads deployed hosts from `module_systems` via `getModuleSystems`, reconciles the live Proxmox node via `reconcilePlacement`, and adds a role-based reachability hint per zone. CI/build infra (builder VM, Forgejo runners) is out of scope (not in `module_systems`).

package/drizzle/0027_dns_internal_records_consumer_cascade.sql

@@ -0,0 +1,43 @@
+-- `dns_internal_records` cascades on its PROVIDER as well as its consumer, so
+-- swapping the internal DNS provider deletes the fleet's entire internal DNS
+-- ledger — including the `zone_routable_ip` view overrides the resolver's
+-- split-horizon config is reconciled from (celilo#1010).
+--
+-- `web_routes` does not do this. Its rows cascade on the consumer only, which is
+-- why a `public_web` provider swap keeps every route. The two tables' docblocks
+-- claim to be siblings and nothing marks the difference as a decision, because it
+-- was not one.
+--
+-- The claim on a capability-owned table is the CONSUMER
+-- (openspec/changes/capability-owned-tables D3/D8), and `dns_internal`'s
+-- declaration cannot express anything else. This makes the table agree with it.
+--
+-- `provider_module_id` STAYS. It is how a record is attributed to the resolver
+-- serving it and it is half the uniqueness key; only its ON DELETE action is
+-- wrong. It becomes a plain column with no foreign key rather than a cascade,
+-- so removing a provider leaves the ledger intact for the next one to reconcile
+-- from — which is the behaviour stage 1's provider-arrival backfill assumes.
+--
+-- SQLite cannot alter a foreign key, so the table is rebuilt. Order matters:
+-- `foreign_keys` is ON during migration, so the copy is made BEFORE the drop and
+-- the index is recreated after the rename. Same shape as
+-- `0021_dns_registration_consumers`.
+
+CREATE TABLE `__new_dns_internal_records` (
+	`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
+	`provider_module_id` text NOT NULL,
+	`consumer_module_id` text NOT NULL,
+	`host` text NOT NULL,
+	`ip` text NOT NULL,
+	`zone_routable_ip` text,
+	`registered_at` integer DEFAULT (unixepoch()) NOT NULL,
+	FOREIGN KEY (`consumer_module_id`) REFERENCES `modules`(`id`) ON UPDATE no action ON DELETE cascade
+);
+--> statement-breakpoint
+INSERT INTO `__new_dns_internal_records` SELECT `id`, `provider_module_id`, `consumer_module_id`, `host`, `ip`, `zone_routable_ip`, `registered_at` FROM `dns_internal_records`;
+--> statement-breakpoint
+DROP TABLE `dns_internal_records`;
+--> statement-breakpoint
+ALTER TABLE `__new_dns_internal_records` RENAME TO `dns_internal_records`;
+--> statement-breakpoint
+CREATE UNIQUE INDEX `dns_internal_records_provider_host_idx` ON `dns_internal_records` (`provider_module_id`,`host`);

package/drizzle/meta/_journal.json

@@ -190,6 +190,13 @@
       "when": 1784200000000,
       "tag": "0026_module_integrity_version",
       "breakpoints": true
+    },
+    {
+      "idx": 27,
+      "version": "6",
+      "when": 1784300000000,
+      "tag": "0027_dns_internal_records_consumer_cascade",
+      "breakpoints": true
     }
   ]
-}
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@celilo/cli",
-  "version": "1.7.0",
+  "version": "1.8.0",
   "description": "Celilo — home lab orchestration CLI",
   "type": "module",
   "bin": {
@@ -58,7 +58,7 @@
   "dependencies": {
     "@aws-sdk/client-s3": "^3.1109.0",
     "@aws-sdk/lib-storage": "^3.1101.0",
-    "@celilo/capabilities": "^2.4.0",
+    "@celilo/capabilities": "^2.6.0",
     "@celilo/cli-display": "^0.2.0",
     "@celilo/core": "^0.9.1",
     "@celilo/event-bus": "^0.6.0",

package/src/capabilities/validation.test.ts

@@ -824,5 +824,56 @@ describe('Capability Access Validation', () => {
       expect(refused.success).toBe(false);
       expect(refused.error).toContain('tsig_key');
     });
+
+    /**
+     * A reference in a TEMPLATE counts too. `resolver.ts` does refuse one at
+     * the point of use, so nothing is unsafe without this, but the refusal
+     * lands at generation rather than at import.
+     *
+     * That gap sits exactly where the repo sends people. CLAUDE.md's module
+     * Definition of Done says "Capability variable usage — Templates use
+     * `$capability:` syntax" and gives a `.tf.tpl` example, so the first author
+     * who follows that instruction with a RESTRICTED secret is the one who
+     * finds out late. No module in the tree references a capability from a
+     * template today, which is why the gap is currently invisible rather than
+     * absent.
+     */
+    test('a secret named only in a template is refused at import', async () => {
+      const manifest: ModuleManifest = {
+        celilo_contract: '1.0',
+        id: 'nosy-app',
+        name: 'Nosy App',
+        version: '1.0.0',
+        description: 'Test',
+        requires: { capabilities: [{ name: 'dns_internal', version: '1.0.0' }] },
+        provides: { capabilities: [{ name: 'private_web', version: '1.0.0', data: {} }] },
+        variables: { owns: [], imports: [] },
+      };
+
+      // The manifest names nothing. Only the .tf.tpl does.
+      const result = await validateCapabilityAccess(manifest, knotDb, ['dns_internal.tsig_key']);
+
+      expect(result.success).toBe(false);
+      expect(result.error).toContain("cannot access secret 'tsig_key'");
+    });
+
+    test('a non-secret template reference is still not a secret reference', async () => {
+      const manifest: ModuleManifest = {
+        celilo_contract: '1.0',
+        id: 'caddy-internal',
+        name: 'Caddy (fleet-only ingress)',
+        version: '1.0.0',
+        description: 'Test',
+        requires: { capabilities: [{ name: 'dns_internal', version: '1.0.0' }] },
+        provides: { capabilities: [{ name: 'private_web', version: '1.0.0', data: {} }] },
+        variables: { owns: [], imports: [] },
+      };
+
+      const result = await validateCapabilityAccess(manifest, knotDb, [
+        'dns_internal.server.ip.primary',
+      ]);
+
+      expect(result.success).toBe(true);
+    });
   });
 });

package/src/capabilities/validation.ts

@@ -21,11 +21,16 @@ export interface ValidationResult {
  *
  * @param manifest - Consumer module manifest
  * @param db - Database connection
+ * @param templateReferences - `<capability>.<path>` references found outside
+ *   the manifest, i.e. in the module's templates. Empty is the honest default
+ *   for a caller holding no template context; `module import` passes what the
+ *   template validator already parsed.
  * @returns Validation result
  */
 export async function validateCapabilityAccess(
   manifest: ModuleManifest,
   db: Database,
+  templateReferences: readonly string[] = [],
 ): Promise<ValidationResult> {
   // If module doesn't require capabilities, validation passes
   if (!manifest.requires?.capabilities || manifest.requires.capabilities.length === 0) {
@@ -35,7 +40,7 @@ export async function validateCapabilityAccess(
   // Get list of capabilities this module provides
   const consumerCapabilities = (manifest.provides?.capabilities || []).map((cap) => cap.name);
 
-  const references = collectCapabilityReferences(manifest);
+  const references = collectCapabilityReferences(manifest, templateReferences);
 
   // Check each required capability
   for (const requiredCapability of manifest.requires.capabilities) {
@@ -105,16 +110,25 @@ export async function validateCapabilityAccess(
  * lives — a `variables.owns[].derive_from`, a default, a capability data block —
  * without this having to track which fields may hold one.
  *
- * References made from a module's TEMPLATES are deliberately not collected here.
- * `src/variables/resolver.ts` checks access at the point of use, which is the
- * only place a template reference can be seen, and it refuses there.
+ * Template references are included too, supplied by the caller. `resolver.ts`
+ * does refuse a template reference at the point of use, so nothing is unsafe
+ * without them, but the refusal lands at generation rather than at import. That
+ * matters because a module's TEMPLATES are where this repo tells authors to put
+ * these: the Definition of Done in CLAUDE.md says "Capability variable usage —
+ * Templates use `$capability:` syntax" and gives a `.tf.tpl` example. A gate
+ * that does not fire where the documentation sends people is a gate with a hole
+ * in the shape of the instructions.
  */
-function collectCapabilityReferences(manifest: ModuleManifest): Set<string> {
-  return new Set(
-    parseVariables(JSON.stringify(manifest))
+function collectCapabilityReferences(
+  manifest: ModuleManifest,
+  templateReferences: readonly string[] = [],
+): Set<string> {
+  return new Set([
+    ...parseVariables(JSON.stringify(manifest))
       .filter((variable) => variable.type === 'capability')
       .map((variable) => variable.path),
-  );
+    ...templateReferences,
+  ]);
 }
 
 /**

package/src/db/dns-internal-cascade-migration.test.ts

@@ -0,0 +1,184 @@
+/**
+ * Migration gate for celilo#1010, over a database that already holds records.
+ *
+ * `0027` rebuilds `dns_internal_records` to drop the foreign key on
+ * `provider_module_id`. A rebuild is copy, drop, rename — the most destructive
+ * shape a migration takes — and it will run on celilo-mgr, which holds the live
+ * internal DNS ledger including the `zone_routable_ip` view overrides the
+ * resolver's split-horizon config is reconciled from.
+ *
+ * Every other test in the suite starts from an empty database and can only prove
+ * the forward invariant. This one builds a database at the schema BEFORE the
+ * change, puts real rows in it, and runs the real migrator over it — the same
+ * shape as `dns-registrations-migration.test.ts`, and for the same reason: the
+ * installed base is where the risk is, and a clean-start suite cannot see it.
+ *
+ * Task 2.6 asks for backup and restore proven before this runs on celilo-mgr.
+ * This is the stronger half of that: it proves the migration PRESERVES the rows,
+ * so a restore is the fallback rather than the plan.
+ */
+
+import { Database } from 'bun:sqlite';
+import { afterEach, beforeEach, describe, expect, test } from 'bun:test';
+import { cpSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { drizzle } from 'drizzle-orm/bun-sqlite';
+import { migrate } from 'drizzle-orm/bun-sqlite/migrator';
+import { findMigrationsFolder } from './client';
+
+/** The last migration before this change — the schema celilo-mgr is on today. */
+const LAST_LEGACY_TAG = '0026_module_integrity_version';
+
+interface JournalEntry {
+  idx: number;
+  version: string;
+  when: number;
+  tag: string;
+  breakpoints: boolean;
+}
+
+/** A migrations folder truncated at `LAST_LEGACY_TAG`, built from the real files. */
+function legacyMigrationsFolder(into: string): string {
+  const source = findMigrationsFolder();
+  const journal = JSON.parse(readFileSync(join(source, 'meta', '_journal.json'), 'utf8')) as {
+    version: string;
+    dialect: string;
+    entries: JournalEntry[];
+  };
+  const cutoff = journal.entries.findIndex((e) => e.tag === LAST_LEGACY_TAG);
+  if (cutoff === -1) throw new Error(`Journal has no entry for ${LAST_LEGACY_TAG}`);
+  const kept = journal.entries.slice(0, cutoff + 1);
+
+  mkdirSync(join(into, 'meta'), { recursive: true });
+  for (const entry of kept) {
+    cpSync(join(source, `${entry.tag}.sql`), join(into, `${entry.tag}.sql`));
+  }
+  writeFileSync(
+    join(into, 'meta', '_journal.json'),
+    JSON.stringify({ ...journal, entries: kept }, null, 2),
+  );
+  return into;
+}
+
+describe('0027 over a database that already holds an internal DNS ledger', () => {
+  let tempDir: string;
+  let dbPath: string;
+
+  beforeEach(() => {
+    tempDir = mkdtempSync(join(tmpdir(), 'celilo-dnsint-'));
+    dbPath = join(tempDir, 'legacy.db');
+  });
+
+  afterEach(() => {
+    rmSync(tempDir, { recursive: true, force: true });
+  });
+
+  function openLegacyDatabase(): Database {
+    const sqlite = new Database(dbPath, { create: true });
+    sqlite.run('PRAGMA foreign_keys = ON');
+    migrate(drizzle(sqlite), {
+      migrationsFolder: legacyMigrationsFolder(join(tempDir, 'legacy-migrations')),
+    });
+    return sqlite;
+  }
+
+  function seedModule(sqlite: Database, id: string): void {
+    sqlite.run(
+      'INSERT INTO modules (id, name, source_path, version, manifest_data) VALUES (?, ?, ?, ?, ?)',
+      [id, id, `/srv/${id}`, '1.0.0', JSON.stringify({ id })],
+    );
+  }
+
+  function seedLedger(sqlite: Database): void {
+    seedModule(sqlite, 'technitium');
+    seedModule(sqlite, 'knot-unbound-internal');
+    seedModule(sqlite, 'caddy');
+    seedModule(sqlite, 'forgejo');
+    sqlite.run(
+      'INSERT INTO dns_internal_records (provider_module_id, consumer_module_id, host, ip, zone_routable_ip) VALUES (?, ?, ?, ?, ?)',
+      ['technitium', 'caddy', 'auth.example.org', '192.168.0.253', '10.0.10.14'],
+    );
+    sqlite.run(
+      'INSERT INTO dns_internal_records (provider_module_id, consumer_module_id, host, ip, zone_routable_ip) VALUES (?, ?, ?, ?, NULL)',
+      ['technitium', 'forgejo', 'git.example.org', '192.168.0.253'],
+    );
+  }
+
+  function runCurrentMigrations(sqlite: Database): void {
+    migrate(drizzle(sqlite), { migrationsFolder: findMigrationsFolder() });
+  }
+
+  test('the rebuild carries every row across, values intact', () => {
+    const sqlite = openLegacyDatabase();
+    seedLedger(sqlite);
+
+    runCurrentMigrations(sqlite);
+
+    const rows = sqlite
+      .query<
+        { host: string; ip: string; zone_routable_ip: string | null; provider_module_id: string },
+        []
+      >(
+        'SELECT host, ip, zone_routable_ip, provider_module_id FROM dns_internal_records ORDER BY host',
+      )
+      .all();
+    expect(rows).toHaveLength(2);
+    expect(rows[0]?.host).toBe('auth.example.org');
+    // The override is the value whose loss is silent: the resolver keeps
+    // answering, just with an address in-zone clients cannot route to.
+    expect(rows[0]?.zone_routable_ip).toBe('10.0.10.14');
+    expect(rows[1]?.zone_routable_ip).toBeNull();
+    // Attribution survives; only its ON DELETE action changed.
+    expect(rows[0]?.provider_module_id).toBe('technitium');
+    sqlite.close();
+  });
+
+  test('after migrating, removing the PROVIDER no longer empties the ledger', () => {
+    const sqlite = openLegacyDatabase();
+    seedLedger(sqlite);
+    runCurrentMigrations(sqlite);
+
+    sqlite.run('DELETE FROM modules WHERE id = ?', ['technitium']);
+
+    expect(
+      sqlite.query<{ c: number }, []>('SELECT COUNT(*) AS c FROM dns_internal_records').get()?.c,
+    ).toBe(2);
+    sqlite.close();
+  });
+
+  /**
+   * The behaviour that did NOT change. Dropping a foreign key is an easy way to
+   * lose the one you meant to keep, and nothing else would notice.
+   */
+  test('the consumer cascade still fires after the rebuild', () => {
+    const sqlite = openLegacyDatabase();
+    seedLedger(sqlite);
+    runCurrentMigrations(sqlite);
+
+    sqlite.run('DELETE FROM modules WHERE id = ?', ['caddy']);
+
+    const left = sqlite
+      .query<{ consumer_module_id: string }, []>(
+        'SELECT consumer_module_id FROM dns_internal_records',
+      )
+      .all();
+    expect(left.map((r) => r.consumer_module_id)).toEqual(['forgejo']);
+    sqlite.close();
+  });
+
+  /** The unique index is recreated by the rebuild, not left behind with the old table. */
+  test('the provider+host uniqueness survives the rebuild', () => {
+    const sqlite = openLegacyDatabase();
+    seedLedger(sqlite);
+    runCurrentMigrations(sqlite);
+
+    expect(() =>
+      sqlite.run(
+        'INSERT INTO dns_internal_records (provider_module_id, consumer_module_id, host, ip) VALUES (?, ?, ?, ?)',
+        ['technitium', 'forgejo', 'auth.example.org', '192.168.0.253'],
+      ),
+    ).toThrow();
+    sqlite.close();
+  });
+});

package/src/db/schema.ts

@@ -801,16 +801,33 @@ export const publicDnsEvidence = sqliteTable('public_dns_evidence', {
  *
  * `celilo system doctor` reads this to assert service hostnames resolve to
  * the firewall natIp (LAN-reachable) and not a zone-side container IP that
- * a LAN device can't route to. Rows die with either module via FK cascade.
+ * a LAN device can't route to. Rows die with their CONSUMER via FK cascade, and
+ * NOT with their provider (celilo#1010 — see `providerModuleId`).
  * @owner capability:dns_internal — resolver configuration; migrates to the provider (T6)
  */
 export const dnsInternalRecords = sqliteTable(
   'dns_internal_records',
   {
     id: integer('id').primaryKey({ autoIncrement: true }),
-    providerModuleId: text('provider_module_id')
-      .notNull()
-      .references(() => modules.id, { onDelete: 'cascade' }),
+    /**
+     * The resolver serving this record. A PLAIN column with no foreign key, and
+     * that is the celilo#1010 correction rather than an oversight.
+     *
+     * It used to cascade, so swapping `technitium` for `knot-unbound-internal`
+     * deleted the fleet's entire internal DNS ledger, `zone_routable_ip` view
+     * overrides included. `web_routes` cascades on its consumer only and the two
+     * docblocks claimed to be siblings, so the divergence read as intent and was
+     * not. The claim on a capability-owned table is the CONSUMER
+     * (openspec/changes/capability-owned-tables D3/D8), and `dns_internal`'s
+     * declaration cannot express anything else.
+     *
+     * Attribution is still real and still enforced — it is half
+     * `dns_internal_records_provider_host_idx` — it just no longer decides when a
+     * LIVE record is forgotten. A provider leaving now leaves the ledger for the
+     * next one to reconcile from, which is what stage 1's provider-arrival
+     * backfill assumes. Migration `0027`.
+     */
+    providerModuleId: text('provider_module_id').notNull(),
     consumerModuleId: text('consumer_module_id')
       .notNull()
       .references(() => modules.id, { onDelete: 'cascade' }),

package/src/manifest/template-validator.test.ts

@@ -258,4 +258,51 @@ resource "local_file" "test" {
       expect(errors[0]?.variable).toBe('$self:invalid_var');
     });
   });
+
+  /**
+   * The capability-secret gate at import consumes these, so what happens when
+   * templates cannot be read is a security property rather than a detail. It
+   * has to FAIL CLOSED: an unreadable tree must abort the import, never hand
+   * the gate an empty reference set that reads as "this module references no
+   * secrets". It was already true here — but true by inspection, which is the
+   * weakest way for a security property to be true.
+   */
+  describe('capability references, and failing closed', () => {
+    test('collects $capability: references from templates', async () => {
+      const dir = await mkdtemp(join(tmpdir(), 'celilo-tplrefs-'));
+      try {
+        await mkdir(join(dir, 'terraform'), { recursive: true });
+        await writeFile(
+          join(dir, 'terraform', 'main.tf.tpl'),
+          'acme_dns = "$capability:dns_internal.tsig_key"\nzone = "$capability:dns_internal.dns.domain"\n',
+        );
+        const manifest = createTestManifest({
+          requires: { capabilities: [{ name: 'dns_internal', version: '1.0.0' }] },
+        });
+
+        const result = await validateModuleTemplates(dir, manifest);
+
+        expect(result.capabilityReferences.sort()).toEqual([
+          'dns_internal.dns.domain',
+          'dns_internal.tsig_key',
+        ]);
+      } finally {
+        await rm(dir, { recursive: true, force: true });
+      }
+    });
+
+    test('an unreadable module tree fails, and yields no references', async () => {
+      // A path that does not exist stands in for any read failure. The PAIRING
+      // is the property: success:false travels WITH the empty array, and
+      // `import.ts` returns on !success before the access gate runs, so the
+      // empty set can never be mistaken for "nothing referenced".
+      const result = await validateModuleTemplates(
+        join(tmpdir(), 'celilo-no-such-module-tree-9f3a2b'),
+        createTestManifest(),
+      );
+
+      expect(result.success).toBe(false);
+      expect(result.capabilityReferences).toEqual([]);
+    });
+  });
 });

package/src/manifest/template-validator.ts

@@ -19,6 +19,12 @@ export interface TemplateValidationError {
 export interface TemplateValidationResult {
   success: boolean;
   errors: TemplateValidationError[];
+  /**
+   * `<capability>.<path>` paths the templates reference. The capability-access
+   * check at import consumes these, so a secret referenced only from a template
+   * is refused at import rather than later at generation.
+   */
+  capabilityReferences: string[];
 }
 
 /**
@@ -277,10 +283,16 @@ export async function validateModuleTemplates(
     // Find all .tpl files
     const templateFiles = await findTemplateFiles(modulePath, modulePath);
 
-    // Validate each template file
+    // Validate each template file. Every one is parsed here anyway, so the
+    // capability references fall out of work already being done — no second
+    // walk of the module tree.
+    const capabilityReferences = new Set<string>();
     for (const relativePath of templateFiles) {
       const fullPath = join(modulePath, relativePath);
       const content = await readFile(fullPath, 'utf-8');
+      for (const variable of parseVariables(content)) {
+        if (variable.type === 'capability') capabilityReferences.add(variable.path);
+      }
       const errors = validateTemplateContent(content, manifest, relativePath);
       allErrors.push(...errors);
     }
@@ -288,6 +300,7 @@ export async function validateModuleTemplates(
     return {
       success: allErrors.length === 0,
       errors: allErrors,
+      capabilityReferences: [...capabilityReferences],
     };
   } catch (error) {
     return {
@@ -299,6 +312,10 @@ export async function validateModuleTemplates(
           error: `Failed to validate templates: ${error instanceof Error ? error.message : 'Unknown error'}`,
         },
       ],
+      // Unreadable templates cannot yield references, and import returns on
+      // !success before the access check runs, so this can never reach it as a
+      // silent pass.
+      capabilityReferences: [],
     };
   }
 }

package/src/module/import.ts

@@ -607,7 +607,25 @@ export async function importModule(options: ModuleImportOptions): Promise<Module
     // Execution: Validate capability access if module requires capabilities
     if (manifest.requires?.capabilities && manifest.requires.capabilities.length > 0) {
       const { validateCapabilityAccess } = await import('../capabilities/validation');
-      const accessResult = await validateCapabilityAccess(manifest, db.$client);
+      // Templates are where CLAUDE.md's Definition of Done tells module authors
+      // to put `$capability:` references, so the import-time gate has to see
+      // them (celilo#854, celilo#1027).
+      //
+      // This adds no parser. `validateModuleTemplates` above already reads and
+      // parses every `.tpl` on every import, roughly 25 lines before the gate
+      // runs — it was discarding the references it saw. So there is no new file
+      // walk, no new failure mode and no new ordering question: the data is
+      // already in scope and the gate simply was not looking at it.
+      //
+      // Fail-closed by that same ordering. An unreadable template makes
+      // `validateModuleTemplates` return success:false with no references, and
+      // the early return above fires BEFORE this check, so an empty reference
+      // set can never reach the gate as a silent pass.
+      const accessResult = await validateCapabilityAccess(
+        manifest,
+        db.$client,
+        templateValidation.capabilityReferences,
+      );
 
       if (!accessResult.success) {
         if (tempDir) await cleanupTempDir(tempDir);