@celilo/cli 1.3.0 → 1.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -29,7 +29,7 @@ see `openspec/specs/`. Companion doc: [CELILO_CORE_MODULES.md](./CELILO_CORE_MOD
29
29
  - **Zone detection / system config** — `apps/celilo/src/services/zone-detector.ts` — `detectZoneFromIp` reads `network.<zone>.subnet` from the `systemConfig` table and returns `NetworkZone | 'unknown'`. It answers CONTAINMENT ONLY. It used to return `'external'` on no-match, which conflated "no declared subnet contains this" with "the internet can route to this" — on a firewall with five RFC1918 legs that reported four of them as facing the internet. `'unknown'` is the honest answer; the caller resolves it (see `machine add`: publicly routable → `external`, otherwise fail asking for `--zone`). The subnet-backed zone list is derived from `NETWORK_ZONES` minus `external`, which has no subnet and must never be given one.
30
30
  - **Interface classification** — `packages/capabilities/src/interface-classification.ts` — THE shared classifier, used by the backend and every firewall provider module so the two cannot drift apart again. `isPubliclyRoutable(ip)` is a property of the address alone (false for RFC 1918, RFC 6598 carrier-grade NAT, loopback, link-local, multicast, reserved). `classifyInterfaces(interfaces, zones)` assigns each interface `zone → external → alien`, first match winning, where `external` is the RESIDUAL — routable and claimed by no declared zone — and is never subnet-matched. `externalEdge()` returns none/single/**ambiguous** rather than silently picking the first public address. `defaultRouteFinding()` enforces the invariant that the default route leaves through `internal` or `external`. **`subnetContains(cidr, ip)` lives here and is the ONLY implementation** — three existed and disagreed (the backend's mishandled `/0`); the other two are deleted, not aliased. Design: `openspec/changes/firewall-interface-classification/design.md`.
31
31
  - **Declared networks (classification input)** — `readDeclaredNetworks(db)` in `apps/celilo/src/hooks/capability-loader.ts` — every `network.<name>.subnet` in system config, which is what an interface is attributed against. Read from the CONFIG, not from `NETWORK_ZONES`: celilo holds networks that are not placement zones (`network.control-plane-vpn.subnet`, which `wireguard` requires and reads). Injected into the firewall capability as a LIVE reader (`declaredNetworks`) — that liveness was a mitigation for values written mid-run by a module hook, which `network-declaration` removes; see that spec before assuming a snapshot is still unsafe.
32
- - **Network requirement + ensure (celilo owns the namespace)** — `apps/celilo/src/services/network-ensure.ts` (`ensureRequiredNetworks`), `NetworkRequirementSchema` / `getRequiredNetworkNames` in `apps/celilo/src/manifest/schema.ts`. A module declares `requires.networks: [{name}]` — a NAME, never a value; the schema is `.strict()` so a `subnet:` on the requirement is rejected with a message saying why. The deploy calls `ensureRequiredNetworks` in its interview phase, BEFORE generation and before any hook, and asks over the generic bus interview (`askText`, so it is answerable headless) for anything undefined. Which attributes a network has is celilo's answer, taken from `apps/celilo/schemas/system_config.json`: that file declares `network.<n>.gateway` for the routed segments and omits it for `control-plane-vpn`, so a gateway is never asked for a network that has none. Well-known names carry a `suggested` range there — deliberately NOT `default`, which `getDefaultConfiguration()` would seed at `system init`. Spec: `openspec/changes/networks-are-declared-not-written/specs/network-declaration/spec.md`.
32
+ - **Network requirement + ensure (celilo owns the namespace)** — `apps/celilo/src/services/network-ensure.ts` (`ensureRequiredNetworks`), `NetworkRequirementSchema` / `getRequiredNetworkNames` in `apps/celilo/src/manifest/schema.ts`. A module declares `requires.networks: [{name}]` — a NAME, never a value; the schema is `.strict()` so a `subnet:` on the requirement is rejected with a message saying why. The deploy calls `ensureRequiredNetworks` in its interview phase, BEFORE generation and before any hook, and asks over the generic bus interview (`askText`, so it is answerable headless) for anything undefined. Which attributes a network has is celilo's answer, taken from `apps/celilo/schemas/system_config.json`: that file declares `network.<n>.gateway` for the routed segments and omits it for `control-plane-vpn`, so a gateway is never asked for a network that has none. Well-known names carry a `suggested` range there — deliberately NOT `default`, which `getDefaultConfiguration()` would seed at `system init`. Spec: `openspec/specs/network-declaration/spec.md`.
33
33
  - **The network write path (closed) + celilo's own discovery** — `celilo system apply-config` (`apps/celilo/src/cli/commands/system-apply-config.ts`) REFUSES the whole `network.` namespace, `network.bridge` excepted (a Proxmox bridge name is not addressing, and it is the one network key with a schema default). That is the automation surface a module hook shells out to, so closing it there is what makes "networks are celilo's" an authority rather than a convention every module has to remember. The refusal names the alternative — declare the network, read it with `$system:` — because a bare rejection sends a module author hunting for a typo. The one write that legitimately needed the surface moved INTO celilo: `celilo system discover-network` (`apps/celilo/src/services/network-discovery.ts`, `cli/commands/system-discover-network.ts`) parses the box's own `ip route` and records `network.internal.*` — or `network.secure-mgmt.*` when the box is off the internal LAN (#300). `celilo-mgmt` calls it and decides nothing; it used to parse and write this itself. Idempotent and never overwrites addressing already set. Recurrence gate: `test-integration/module/no-module-writes-networks.test.ts`.
34
34
  - **Firewall interface audit** — `apps/celilo/src/services/audit/interface-classification.ts` — reports per-firewall classification in `celilo audit`: alien interfaces by name and address (drift), and the blocking findings a converge refuses on — a carrier-grade NAT leg, an ambiguous external edge, a default route on the wrong leg.
35
35
  - **Zone taxonomy (canonical list)** — `apps/celilo/src/db/schema.ts` — `NETWORK_ZONES` is the single array; `NetworkZone` is DERIVED from it. Never hand-maintain a second copy: a duplicate that dropped a member made zone validation return null and silently fall back to a wrong-but-valid zone.
@@ -181,7 +181,7 @@ runner seam (`execRunner` real / `createMockRunner` for tests) lives in
181
181
  - **Inbound** (`reconcileAspectsForSystems`): every approved aspect applied to systems that have just come into existence. Called from `module-deploy.ts` between `waitForSSH` and `executeAnsible` (so a module's playbook and `on_install` see a correctly configured host) and from `machine-add.ts`. Eligibility is `applicable_zones` + approval; the aspect's `triggers` list is deliberately NOT consulted, so convergence is not opt-in per manifest. A failure here IS fatal to the deploy that created the system — the aspect is a prerequisite of that host — while a failure on `machine add` is only a warning. PAUSED providers are skipped, which is the escape hatch for a wedged aspect. Nothing is rolled back: rows, guest and IPAM allocation persist and a re-run converges on the same system. Fixes celilo#902, where a system provisioned after its provider deployed silently never received the aspect. See `openspec/changes/aspect-fanout-new-systems/`.
182
182
  - **Coverage verification** — `verifyAspectCoverage` + `celilo system doctor --deep [--fix]`. Answers "is any system missing an aspect its zone entitles it to" WITHOUT stored state: the entitled set is `planAspectFanOut` itself, and coverage is measured by evaluating the role against the host in Ansible check mode (`executeAnsible({check:true})` + `parseAnsibleRecap`). Three outcomes, not two — `changed=0,skipped=0` applied, `changed>0` missing, **`skipped>0` unknown**, because check mode SKIPS a task it cannot evaluate and a `command`/`shell` role would otherwise report clean having never run.
183
183
  - **In-flight operation lock** — `apps/celilo/src/services/module-operations.ts` — `startOperation`/`completeOperation`/`failOperation` record deploy/uninstall/backup/restore in `module_operations`; `refuseIfInFlight`/`checkInFlight` are what backup and restore consult. Deploy and uninstall REGISTER but never check: it is a one-way guard protecting backup/restore consistency, not a general mutex (`openspec/specs/management-server-backup/spec.md` "In-flight operation refusal"). A row stops holding the lock once it is GONE, STOPPED/zombie (`isPidRunnable`, `ps -o state=` — `kill(pid,0)` calls a Ctrl-Z'd process alive), or older than `OPERATION_TTL_MS` (2h). The TTL is not redundancy: a pid is a recycled number, and once the pid space wraps an old row names an unrelated healthy process. Operator surface: `celilo module operations [list|clear] [--abandoned] [--all]` (`apps/celilo/src/cli/commands/module-operations.ts`); `list` shows only what holds the lock, abandoned rows are summarised unless `--abandoned`. Abandoned rows are reclaimed hourly by the `celilo-operations-sweep` bus subscriber (`timer.tick.1h` → `celilo module operations clear`, armed by `ensureOperationsSweepSubscriber` from module registration and `celilo system migrate`). `clear` MARKS rows failed rather than deleting them, and that is load-bearing: the `abandoned_operations` audit reads exactly those released rows to notice one module's operation dying over and over.
184
- - **Module pause / unpause (control-plane quiescence)** — `apps/celilo/src/services/module-pause.ts` — pure `planPause`/`planUnpause` producing an ordered plan, `executePause`/`executeUnpause` performing it, plus `listPausedModules`/`pausedAmong`/`formatPausedDuration`/`describePausedModule` (the ONE place an age is formatted). CLI: `apps/celilo/src/cli/commands/module-pause.ts` (`celilo module pause|unpause <id> [--cascade] [--stop-infra] [--reason] [--dry-run] [--yes]`). Pausing takes a module out of the CONTROL plane — no dispatched events, no timer hooks, no health checks, alerts suppressed — while leaving the DATA plane running, because capability consumption is deploy-time: every consumer calls `firewall`/`dhcp_server` from `on_install` and nothing calls it while serving. Config, secrets, IPAM/VMID and placement are preserved; `on_uninstall` does NOT run. Quiescence is enforced in two places: pausing drops the module's bus subscriptions (`unregisterModuleSubscriptions`), and `run-named-hook.ts` refuses any non-lifecycle hook for a PAUSED module (`skippedPaused`), which catches the paths that skip the bus — `events resync-subscriptions`, a restore that starts events.db empty, aspect fan-out, public-web republish. `on_install`/`on_uninstall` are exempt by hook NAME (not a caller flag): unpause redeploys through `on_install`, and removing a paused provider needs `on_uninstall`. Unpause always REDEPLOYS (`deployModule`) — that is what rebinds a consumer to a replacement provider and recreates provider-local state from the consumers that own it — and re-registers subscriptions, which a plain deploy does not do. A failed unpause restores `PAUSED` rather than leaving the module live and mis-bound. Cascade order reuses `services/update/dep-graph.ts` unchanged (pause = consumers first, unpause = providers first) and is computed from the GRAPH, never from which modules are currently paused, so a cascade walks THROUGH already-done members and is resumable. See `openspec/changes/module-pause-lifecycle/`.
184
+ - **Module pause / unpause (control-plane quiescence)** — `apps/celilo/src/services/module-pause.ts` — pure `planPause`/`planUnpause` producing an ordered plan, `executePause`/`executeUnpause` performing it, plus `listPausedModules`/`pausedAmong`/`formatPausedDuration`/`describePausedModule` (the ONE place an age is formatted). CLI: `apps/celilo/src/cli/commands/module-pause.ts` (`celilo module pause|unpause <id> [--cascade] [--stop-infra] [--reason] [--dry-run] [--yes]`). Pausing takes a module out of the CONTROL plane — no dispatched events, no timer hooks, no health checks, alerts suppressed — while leaving the DATA plane running, because capability consumption is deploy-time: every consumer calls `firewall`/`dhcp_server` from `on_install` and nothing calls it while serving. Config, secrets, IPAM/VMID and placement are preserved; `on_uninstall` does NOT run. Quiescence is enforced in two places: pausing drops the module's bus subscriptions (`unregisterModuleSubscriptions`), and `run-named-hook.ts` refuses any non-lifecycle hook for a PAUSED module (`skippedPaused`), which catches the paths that skip the bus — `events resync-subscriptions`, a restore that starts events.db empty, aspect fan-out, public-web republish. `on_install`/`on_uninstall` are exempt by hook NAME (not a caller flag): unpause redeploys through `on_install`, and removing a paused provider needs `on_uninstall`. Unpause always REDEPLOYS (`deployModule`) — that is what rebinds a consumer to a replacement provider and recreates provider-local state from the consumers that own it — and re-registers subscriptions, which a plain deploy does not do. A failed unpause restores `PAUSED` rather than leaving the module live and mis-bound. Cascade order reuses `services/update/dep-graph.ts` unchanged (pause = consumers first, unpause = providers first) and is computed from the GRAPH, never from which modules are currently paused, so a cascade walks THROUGH already-done members and is resumable. See `openspec/specs/module-pause/spec.md`.
185
185
  - **Provider-removal guard** — `apps/celilo/src/services/remove-guard.ts` — `findRemovalBlockers`/`describeRemovalRefusal`, called from `apps/celilo/src/cli/commands/module-remove.ts`. A PAUSED module is not a dependent (unpause cannot return it to service without a redeploy, and a redeploy re-resolves capabilities), which is what makes a provider swap possible at all. A dependent is one declaring the capability under `requires` **or** `optional` — the same relation `dep-graph.ts` uses, so the guard and the cascade agree on the set; the guard previously read `requires` alone, which let a removal silently orphan `technitium`'s `optional` `dhcp_server`. Refusals name each blocker AND which declaration makes it one. It deliberately does NOT exempt a dependent because another provider of the same capability exists (celilo#683).
186
186
  - **Consumer-removal cleanup (every provider is told)** — `apps/celilo/src/services/consumer-cleanup.ts` — pure `planConsumerCleanup` + `loadConsumerCleanupPlan` + `runConsumerCleanup`, called from `apps/celilo/src/cli/commands/module-remove.ts` after `on_uninstall` and before `terraform destroy`. A capability is two-sided: the consumer asks, the provider mints something in ITS world (a caddy site block, a DNAT rule, an OIDC client at authentik, a registered CI runner), and removal only ever touched one side — the FK cascade dropped celilo's row, so the provider's next converge had no way to learn the thing existed. This dispatches the `on_consumer_removed` hook to every provider of every capability the departing module declared under `requires` OR `optional` (the same relation `remove-guard.ts` counts as a dependency edge), **once per provider** rather than once per capability, sorted by provider id. The hook receives one input, `consumer`, and NOTHING else: a provider that cannot answer "what do I hold for this module" without being told has a different defect — the consumer's id was never recorded at mint time. It replaces `services/web-route-cleanup.ts`, which did the same job for exactly one capability, by name, from core. Semantics that are easy to get backwards: **a failed withdrawal never blocks the removal** — the consumer goes and the failing PROVIDER is marked `ERROR` with the departing consumer named in `error_message` (surfaced as a `blocked` finding by `services/audit/undeployed-modules.ts`), because the hook is a full converge and after a failure the provider's state is unknown rather than "one thing missed". **Dispatch continues past a failure**, so one broken provider cannot leave the others holding state. A PAUSED provider is skipped with a warning naming what it keeps (`run-named-hook.ts` refuses non-lifecycle hooks on a paused module, and `on_consumer_removed` must NOT join `LIFECYCLE_HOOKS`), and a never-deployed one is skipped silently. Providers implementing it: caddy, caddy-internal, iptables, greenwave, axon, authentik, forgejo, generic-cpanel-hosting-provider. See `openspec/changes/consumer-removal-cleanup/`.
187
187
  - **Firewall registry ownership** — `apps/celilo/src/services/port-forwards.ts` + `apps/celilo/src/services/trusted-sources.ts`. Both stores are bound to the CONSUMING module and stamp `registered_by` themselves; a caller cannot supply it, so a registration is never attributed to the wrong module. Both writes are **declarative**: `replace()` states a consumer's COMPLETE set for a target, so a port or subnet it previously registered and now omits is withdrawn (celilo#855 — before this, a module that exposed `:8080` and redeployed exposing `:9090` kept both, forever). The owner is IN both unique indexes, not merely beside them: two consumers wanting the same forward are two ROWS, so one leaving cannot delete a rule the other still needs; `modules/iptables/scripts/ruleset-renderer.ts` dedupes on the rule tuple so the pair renders once. Neither column is a FK, so `runConsumerCleanup` deletes these rows explicitly after every provider has converged without them.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@celilo/cli",
3
- "version": "1.3.0",
3
+ "version": "1.5.0",
4
4
  "description": "Celilo — home lab orchestration CLI",
5
5
  "type": "module",
6
6
  "bin": {
@@ -58,7 +58,7 @@
58
58
  "dependencies": {
59
59
  "@aws-sdk/client-s3": "^3.1109.0",
60
60
  "@aws-sdk/lib-storage": "^3.1101.0",
61
- "@celilo/capabilities": "^2.1.0",
61
+ "@celilo/capabilities": "^2.2.0",
62
62
  "@celilo/cli-display": "^0.2.0",
63
63
  "@celilo/core": "^0.9.0",
64
64
  "@celilo/event-bus": "^0.6.0",
@@ -6,70 +6,22 @@
6
6
  * (e.g., container_created triggers web automation for API key creation).
7
7
  */
8
8
 
9
- import type { DeployedSystem } from '@celilo/capabilities';
9
+ import type { HookDefinition, HookResult as ModuleHookResult } from '@celilo/capabilities';
10
10
 
11
11
  /**
12
- * Hook definition from module manifest.
12
+ * `HookDefinition`, `HookLogger` and `HookContext` are re-exported from
13
+ * `@celilo/capabilities`, which owns them. All three were declared a second
14
+ * time here and the `HookLogger` copy said so in its own comment ("kept in
15
+ * sync") — a duplicate that announces it needs syncing is one that will
16
+ * eventually not be.
13
17
  *
14
- * Inputs and outputs are no longer carried by the manifest they live in the
15
- * contract registry (see `apps/celilo/src/manifest/contracts/v1.ts`),
16
- * keyed by the manifest's `celilo_contract` version. The executor resolves
17
- * the signature at runtime.
18
+ * A second declaration of a shared shape does not fail loudly when it drifts:
19
+ * both sides keep compiling and the two meanings diverge silently. `HookName`
20
+ * was the same duplication and HAD already drifted — it was missing
21
+ * `reconcile_routes`, so caddy's reconcile hook was a hook the celilo side of
22
+ * the codebase did not believe in (celilo#821).
18
23
  */
19
- export interface HookDefinition {
20
- /** Path to hook script relative to module directory */
21
- script: string;
22
- /** Total timeout in milliseconds (default: 60000) */
23
- timeout?: number;
24
- /** If true, hook needs interactive terminal (no FuelGauge wrapping) */
25
- interactive?: boolean;
26
- }
27
-
28
- /**
29
- * Logger interface provided to hook scripts.
30
- *
31
- * Mirrors `HookLogger` in `@celilo/capabilities/types` — kept in sync so
32
- * concrete loggers we build here (createGaugeLogger, createConsoleLogger,
33
- * createCapturingLogger) satisfy both the in-process consumers and the
34
- * cross-package contract used by `wrapWithLogging`.
35
- */
36
- export interface HookLogger {
37
- info(message: string): void;
38
- warn(message: string): void;
39
- error(message: string): void;
40
- success(message: string): void;
41
- /** Begin a logical span. Subsequent log calls nest under it visually. */
42
- beginStep?(name: string): void;
43
- /** End the current span (success). */
44
- endStep?(name: string): void;
45
- /** End the current span (failure) with an error message. */
46
- failStep?(name: string, error: string): void;
47
- }
48
-
49
- /**
50
- * Context passed to hook scripts
51
- */
52
- export interface HookContext {
53
- /** Module configuration values */
54
- config: Record<string, unknown>;
55
- /** Module secret values (decrypted) */
56
- secrets: Record<string, string>;
57
- /**
58
- * The 0..N systems this module has deployed onto (openspec/specs/module-systems-addressing/spec.md).
59
- * Always an array — no singular convenience, so the 0/1/N reality stays visible.
60
- */
61
- systems: DeployedSystem[];
62
- /** Logger for reporting progress */
63
- logger: HookLogger;
64
- /** Run in debug mode (e.g., Playwright headless: false) */
65
- debug: boolean;
66
- /** Directory for saving screenshots on failure */
67
- screenshotDir: string;
68
- /** Capability function interfaces from provider modules (e.g., context.capabilities.dns_registrar) */
69
- capabilities: Record<string, unknown>;
70
- /** Dynamic inputs specified in manifest (e.g., vps_ip) */
71
- [key: string]: unknown;
72
- }
24
+ export type { HookDefinition, HookLogger, HookContext } from '@celilo/capabilities';
73
25
 
74
26
  /**
75
27
  * Surfaced by `invokeHook` when a capability call inside the hook threw
@@ -85,21 +37,15 @@ export interface MissingProviderInputDetails {
85
37
  }
86
38
 
87
39
  /**
88
- * Result returned from hook execution
40
+ * Result returned from hook execution.
41
+ *
42
+ * The shape a module's hook produces is `@celilo/capabilities`' `HookResult`;
43
+ * this adds the one field only the framework sets. A module never writes
44
+ * `missingProviderInput` — the executor sets it when a capability call throws
45
+ * `MissingProviderInputError` — so it belongs on the framework's view of the
46
+ * result rather than in the module-facing contract.
89
47
  */
90
- export interface HookResult {
91
- success: boolean;
92
- outputs: Record<string, unknown>;
93
- error?: string;
94
- /**
95
- * Every file the hook wrote to its per-run artifact directory.
96
- * All of them, not just the newest image: a screenshot without the
97
- * DOM and the observed requests is the least useful third of a
98
- * post-mortem.
99
- */
100
- artifactPaths?: string[];
101
- /** Duration in milliseconds */
102
- duration: number;
48
+ export interface HookResult extends ModuleHookResult {
103
49
  /**
104
50
  * Set when the hook failed because a capability call needs the
105
51
  * framework to extend another module's config. Mutually exclusive
@@ -261,6 +261,41 @@ export const V1_HOOKS: ContractHooks = {
261
261
  inputs: {},
262
262
  outputs: {},
263
263
  },
264
+ /**
265
+ * Report what a tunnel/interface is ACTUALLY carrying, diffed against what
266
+ * celilo's config says it should carry.
267
+ *
268
+ * Deliberately a read of the live thing rather than of celilo's own record.
269
+ * celilo#928 was exactly a divergence between the two — a device present in
270
+ * config, rendered as active by a UI, and absent from the interface — and
271
+ * diagnosing it needed a screenshot and a root SSH because nothing celilo
272
+ * offered could tell the two apart. A hook that reported the record would
273
+ * have reproduced the misleading signal rather than exposing it.
274
+ *
275
+ * No structured outputs: it reports through the logger, so an operator and an
276
+ * agent read the same thing.
277
+ */
278
+ list_peers: {
279
+ inputs: {},
280
+ outputs: {},
281
+ },
282
+ /**
283
+ * Re-assert a module's desired peer set onto the thing carrying it, on a
284
+ * timer, without an operator.
285
+ *
286
+ * The counterpart to applying on change: apply-on-change bounds latency, this
287
+ * bounds how long ANY divergence can persist — including drift nobody
288
+ * predicted (a hand-edit on the box, an apply that failed, a record written
289
+ * before the applying code existed). celilo#934, where the module could
290
+ * already SEE the drift in `health_check` and had nowhere to act on it.
291
+ *
292
+ * A converge that changes nothing must be silent, so the log stays a record
293
+ * of things happening.
294
+ */
295
+ reconcile_peers: {
296
+ inputs: {},
297
+ outputs: {},
298
+ },
264
299
  /**
265
300
  * Build-bus upstream publish hook. The executor passes the
266
301
  * PublishEvent fields as env vars (CELILO_EVENT_PAYLOAD,
@@ -364,6 +364,8 @@ const LIFECYCLE_HOOK_SCHEMAS = {
364
364
  refresh_registrations: LifecycleHookSchema.optional(),
365
365
  reassert_dhcp_dns: LifecycleHookSchema.optional(),
366
366
  reconcile_clients: LifecycleHookSchema.optional(),
367
+ list_peers: LifecycleHookSchema.optional(),
368
+ reconcile_peers: LifecycleHookSchema.optional(),
367
369
  } satisfies Record<HookName, z.ZodTypeAny>;
368
370
 
369
371
  /**
@@ -6,6 +6,7 @@ import { basename, join, relative } from 'node:path';
6
6
  import { create as tarCreate } from 'tar';
7
7
  import { parse as parseYaml } from 'yaml';
8
8
  import { log } from '../../cli/prompts';
9
+ import { formatViolations, scanModuleDirectory } from '../../policy/module-script-scan';
9
10
  import { validateModuleDirectory } from '../import';
10
11
  import { computeFileChecksum } from './checksum';
11
12
  import { includeNodeModulesPath } from './package-rules';
@@ -175,6 +176,34 @@ export async function buildModule(options: ModuleBuildOptions): Promise<ModuleBu
175
176
  return { success: false, error: dirError };
176
177
  }
177
178
 
179
+ // Refuse to package a module whose hook scripts hand-build SSH or take the
180
+ // raw-exec escape hatch without justifying it
181
+ // (openspec/changes/unified-management-no-ssh/proposal.md).
182
+ //
183
+ // The same rules run as a `bun test` gate over this repo's modules. This is
184
+ // the enforcement point that catches what that one cannot: a module built
185
+ // outside CI. `bun run publish` is a documented escape hatch for when the
186
+ // runners are down and it runs no tests, and a module authored outside this
187
+ // repo never passes through the suite at all — in both cases packaging is the
188
+ // last place anything looks at the code before it becomes an artifact the
189
+ // fleet installs.
190
+ //
191
+ // Scans the SOURCE scripts, before staging: it fails in under a second rather
192
+ // than after a `bun pm pack` and a full module build, and the staged copy
193
+ // bundles `@celilo/capabilities` — whose `remote.ts` builds the very
194
+ // `ssh … root@` string these rules exist to keep out of module code — so
195
+ // scanning the bundle would fail every module in the fleet on the
196
+ // implementation of the primitives they were told to use.
197
+ const policyViolations = scanModuleDirectory(sourceDir);
198
+ if (policyViolations.length > 0) {
199
+ return {
200
+ success: false,
201
+ error: `Refusing to package ${basename(sourceDir)}: module script policy violations.\n${formatViolations(
202
+ policyViolations,
203
+ )}\n\nSee apps/celilo/MODULE_PRIMITIVES.md.`,
204
+ };
205
+ }
206
+
178
207
  // Copy source to a temp dir for building. Strategy:
179
208
  // - If the source has a package.json, use `bun pm pack` to respect the
180
209
  // `files` field (or .npmignore), copying only what the build needs.
@@ -11,6 +11,19 @@
11
11
  * refer to that document's sections 1-4 — read the row before changing a line
12
12
  * here.
13
13
  *
14
+ * **Every debt entry names the issue tracking its removal.** The audit is a
15
+ * ruling; the issues are the work. If you are here because the gate went red,
16
+ * the issue in the entry's `why` is where the fix belongs:
17
+ *
18
+ * #937 the hardcoded well-known registry + its duplicate zone table
19
+ * #938 the loader's capName branches, CAPABILITY_MODULE_MAP, isFirewallModule
20
+ * #939 the four provider-domain tables
21
+ * #940 the Caddyfile generator + interface classification in the shared package
22
+ * #941 core shelling iptables-save
23
+ * #943 the three capability-named CLI verbs
24
+ * #944 core shelling wg pubkey
25
+ * #945 three core primitives gated on one capability (generalise, don't move)
26
+ *
14
27
  * Two kinds of entry live here and they must not be confused:
15
28
  *
16
29
  * - **Debt** — the default. It is here because it is wrong and not yet fixed.
@@ -25,6 +38,9 @@
25
38
  * each belongs to. Scan A asserts the tagged set EQUALS this map — not that it
26
39
  * is empty. These four exist today (audit T1, T2, T3, T6) and Phase 1 migrates
27
40
  * nothing; an assertion that core holds none would be red the day it landed.
41
+ *
42
+ * Tracked by #939. Migrating them keeps the ownership CLAIM in core — dropping
43
+ * it is the mistake that made `dns_registration_consumers` necessary (#626).
28
44
  */
29
45
  export const CAPABILITY_OWNED_TABLES: Readonly<Record<string, string>> = {
30
46
  web_routes: 'public_web',
@@ -68,25 +84,25 @@ export const CAPABILITY_NAME_BASELINE: readonly CapabilityNameRow[] = [
68
84
  file: 'apps/celilo/src/capabilities/well-known.ts',
69
85
  capability: 'dhcp_server',
70
86
  count: 1,
71
- why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2',
87
+ why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2 (#937)',
72
88
  },
73
89
  {
74
90
  file: 'apps/celilo/src/capabilities/well-known.ts',
75
91
  capability: 'dns_internal',
76
92
  count: 1,
77
- why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2',
93
+ why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2 (#937)',
78
94
  },
79
95
  {
80
96
  file: 'apps/celilo/src/capabilities/well-known.ts',
81
97
  capability: 'dns_registrar',
82
98
  count: 1,
83
- why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2',
99
+ why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2 (#937)',
84
100
  },
85
101
  {
86
102
  file: 'apps/celilo/src/capabilities/well-known.ts',
87
103
  capability: 'public_web',
88
104
  count: 1,
89
- why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2',
105
+ why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2 (#937)',
90
106
  },
91
107
  {
92
108
  file: 'apps/celilo/src/cli/commands/notify-config.ts',
@@ -98,91 +114,91 @@ export const CAPABILITY_NAME_BASELINE: readonly CapabilityNameRow[] = [
98
114
  file: 'apps/celilo/src/cli/commands/token.ts',
99
115
  capability: 'idp',
100
116
  count: 1,
101
- why: 'X7 — a top-level CLI verb backed by the idp capability; Phase 5',
117
+ why: 'X7 — a top-level CLI verb backed by the idp capability; Phase 5 (#943)',
102
118
  },
103
119
  {
104
120
  file: 'apps/celilo/src/cli/completion.ts',
105
121
  capability: 'firewall',
106
122
  count: 3,
107
- why: 'X6 — tab completion for the capability-named CLI verbs; falls out with X6',
123
+ why: 'X6 — tab completion for the capability-named CLI verbs; falls out with X6 (#943)',
108
124
  },
109
125
  {
110
126
  file: 'apps/celilo/src/cli/index.ts',
111
127
  capability: 'firewall',
112
128
  count: 2,
113
- why: 'X5/X6/X7 — the command tree wiring the capability-named CLI verbs; Phase 5',
129
+ why: 'X5/X6/X7 — the command tree wiring the capability-named CLI verbs; Phase 5 (#943)',
114
130
  },
115
131
  {
116
132
  file: 'apps/celilo/src/hooks/capability-loader.ts',
117
133
  capability: 'control_plane_vpn',
118
134
  count: 1,
119
- why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
135
+ why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
120
136
  },
121
137
  {
122
138
  file: 'apps/celilo/src/hooks/capability-loader.ts',
123
139
  capability: 'dhcp_server',
124
140
  count: 1,
125
- why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
141
+ why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
126
142
  },
127
143
  {
128
144
  file: 'apps/celilo/src/hooks/capability-loader.ts',
129
145
  capability: 'dns_internal',
130
146
  count: 3,
131
- why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
147
+ why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
132
148
  },
133
149
  {
134
150
  file: 'apps/celilo/src/hooks/capability-loader.ts',
135
151
  capability: 'dns_registrar',
136
152
  count: 4,
137
- why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
153
+ why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
138
154
  },
139
155
  {
140
156
  file: 'apps/celilo/src/hooks/capability-loader.ts',
141
157
  capability: 'external_web',
142
158
  count: 1,
143
- why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
159
+ why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
144
160
  },
145
161
  {
146
162
  file: 'apps/celilo/src/hooks/capability-loader.ts',
147
163
  capability: 'firewall',
148
164
  count: 12,
149
- why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
165
+ why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
150
166
  },
151
167
  {
152
168
  file: 'apps/celilo/src/hooks/capability-loader.ts',
153
169
  capability: 'idp',
154
170
  count: 1,
155
- why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
171
+ why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
156
172
  },
157
173
  {
158
174
  file: 'apps/celilo/src/hooks/capability-loader.ts',
159
175
  capability: 'notification',
160
176
  count: 1,
161
- why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
177
+ why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
162
178
  },
163
179
  {
164
180
  file: 'apps/celilo/src/hooks/capability-loader.ts',
165
181
  capability: 'private_web',
166
182
  count: 1,
167
- why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
183
+ why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
168
184
  },
169
185
  {
170
186
  file: 'apps/celilo/src/hooks/capability-loader.ts',
171
187
  capability: 'public_web',
172
188
  count: 8,
173
- why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
189
+ why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
174
190
  },
175
191
  {
176
192
  file: 'apps/celilo/src/hooks/capability-loader.ts',
177
193
  capability: 'registry_publish',
178
194
  count: 1,
179
- why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
195
+ why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
180
196
  },
181
197
  {
182
198
  file: 'apps/celilo/src/hooks/capability-loader.ts',
183
199
  capability: 'source_forge',
184
200
  count: 1,
185
- why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
201
+ why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
186
202
  },
187
203
  {
188
204
  file: 'apps/celilo/src/manifest/validate.ts',
@@ -212,49 +228,49 @@ export const CAPABILITY_NAME_BASELINE: readonly CapabilityNameRow[] = [
212
228
  file: 'apps/celilo/src/services/dns-provider-backfill.ts',
213
229
  capability: 'dns_internal',
214
230
  count: 2,
215
- why: 'S3 — the generic event-replay half is core; the dns_internal gate is not',
231
+ why: 'S3 — the generic event-replay half is core; the dns_internal gate is not (#945)',
216
232
  },
217
233
  {
218
234
  file: 'apps/celilo/src/services/firewall-reach.ts',
219
235
  capability: 'firewall',
220
236
  count: 1,
221
- why: 'S8 — core shells iptables-save at a remote box and parses the output',
237
+ why: 'S8 — core shells iptables-save at a remote box and parses the output (#941)',
222
238
  },
223
239
  {
224
240
  file: 'apps/celilo/src/services/infrastructure-selector.ts',
225
241
  capability: 'firewall',
226
242
  count: 1,
227
- why: 'S15 — isFirewallModule hardcodes a placement rule the manifest should declare',
243
+ why: 'S15 — isFirewallModule hardcodes a placement rule the manifest should declare (#938)',
228
244
  },
229
245
  {
230
246
  file: 'apps/celilo/src/services/module-deploy.ts',
231
247
  capability: 'firewall',
232
248
  count: 2,
233
- why: "S16 — two more copies of S15's decision; fixing S15 removes all three",
249
+ why: "S16 — two more copies of S15's decision; fixing S15 removes all three (#938)",
234
250
  },
235
251
  {
236
252
  file: 'apps/celilo/src/services/public-web-republish.ts',
237
253
  capability: 'public_web',
238
254
  count: 1,
239
- why: "S2 — encodes caddy's redeploy behaviour; generalise to a provider-declared re-assert signal",
255
+ why: "S2 — encodes caddy's redeploy behaviour; generalise to a provider-declared re-assert signal (#945)",
240
256
  },
241
257
  {
242
258
  file: 'apps/celilo/src/services/zone-policy.ts',
243
259
  capability: 'public_web',
244
260
  count: 1,
245
- why: 'S13 — ZONE_REQUIREMENTS, a second hand-maintained copy of well-known.ts; Phase 2',
261
+ why: 'S13 — ZONE_REQUIREMENTS, a second hand-maintained copy of well-known.ts; Phase 2 (#937)',
246
262
  },
247
263
  {
248
264
  file: 'apps/celilo/src/templates/generator.ts',
249
265
  capability: 'dns_internal',
250
266
  count: 1,
251
- why: 'X4 — declaration-driven and the model for the rest; only the error string names the capability',
267
+ why: 'X4 — declaration-driven and the model for the rest; only the error string names the capability (#945)',
252
268
  },
253
269
  {
254
270
  file: 'apps/celilo/src/variables/context.ts',
255
271
  capability: 'dns_internal',
256
272
  count: 1,
257
- why: 'X3 — reaches capabilitiesMap.dns_internal by name; should read a declared field (X4 is the model)',
273
+ why: 'X3 — reaches capabilitiesMap.dns_internal by name; should read a declared field (X4 is the model) (#945)',
258
274
  },
259
275
  {
260
276
  file: 'packages/capabilities/src/capability-contract.ts',
@@ -344,13 +360,13 @@ export const CAPABILITY_NAME_BASELINE: readonly CapabilityNameRow[] = [
344
360
  file: 'packages/capabilities/src/public-web.ts',
345
361
  capability: 'public_web',
346
362
  count: 1,
347
- why: "X8 — generateCaddyfile renders caddy's complete config from the shared package; Phase 5",
363
+ why: "X8 — generateCaddyfile renders caddy's complete config from the shared package; Phase 5 (#940)",
348
364
  },
349
365
  {
350
366
  file: 'packages/capabilities/src/utils.ts',
351
367
  capability: 'public_web',
352
368
  count: 3,
353
- why: 'X8 — shared helpers that name the capability they serve; moves with X8 in Phase 5',
369
+ why: 'X8 — shared helpers that name the capability they serve; moves with X8 in Phase 5 (#940)',
354
370
  },
355
371
  ];
356
372
 
@@ -368,17 +384,17 @@ export const SERVICE_FILENAME_BASELINE: readonly ServiceFilenameRow[] = [
368
384
  {
369
385
  file: 'apps/celilo/src/services/dns-internal-records.ts',
370
386
  capability: 'dns_internal',
371
- why: 'S5 — the store for T6; moves with the table in Phase 4',
387
+ why: 'S5 — the store for T6; moves with the table in Phase 4 (#939)',
372
388
  },
373
389
  {
374
390
  file: 'apps/celilo/src/services/firewall-reach.ts',
375
391
  capability: 'firewall',
376
- why: 'S8 — core reaching into one provider implementation',
392
+ why: 'S8 — core reaching into one provider implementation (#941)',
377
393
  },
378
394
  {
379
395
  file: 'apps/celilo/src/services/public-web-republish.ts',
380
396
  capability: 'public_web',
381
- why: "S2 — named for one provider's redeploy behaviour",
397
+ why: "S2 — named for one provider's redeploy behaviour (#945)",
382
398
  },
383
399
  ];
384
400
 
@@ -387,18 +403,18 @@ export const PROVIDER_LITERAL_BASELINE: readonly ProviderLiteralRow[] = [
387
403
  file: 'apps/celilo/src/secrets/generators.ts',
388
404
  literal: 'wg pubkey',
389
405
  count: 1,
390
- why: 'X11 — core shells `wg pubkey` to derive a key, and tells the operator to brew install wireguard-tools',
406
+ why: 'X11 — core shells `wg pubkey` to derive a key, and tells the operator to brew install wireguard-tools (#944)',
391
407
  },
392
408
  {
393
409
  file: 'apps/celilo/src/services/firewall-reach.ts',
394
410
  literal: 'iptables-save',
395
411
  count: 1,
396
- why: "S8 — core parses a firewall's live ruleset; implementation-specific, not merely capability-specific",
412
+ why: "S8 — core parses a firewall's live ruleset; implementation-specific, not merely capability-specific (#941)",
397
413
  },
398
414
  {
399
415
  file: 'packages/capabilities/src/public-web.ts',
400
416
  literal: '/srv/www',
401
417
  count: 4,
402
- why: "X8 — the Caddyfile generator knows caddy's on-disk asset layout",
418
+ why: "X8 — the Caddyfile generator knows caddy's on-disk asset layout (#940)",
403
419
  },
404
420
  ];
@@ -0,0 +1,164 @@
1
+ /**
2
+ * The scan rules themselves, and the proof that packaging refuses a module that
3
+ * breaks them.
4
+ *
5
+ * `no-hand-built-ssh.test.ts` asserts the CURRENT tree is clean, which is a
6
+ * different claim: it passes both when the rules work and when they match
7
+ * nothing. These tests are the ones that fail if a rule stops catching things.
8
+ */
9
+
10
+ import { describe, expect, it } from 'bun:test';
11
+ import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
12
+ import { tmpdir } from 'node:os';
13
+ import { join } from 'node:path';
14
+ import { buildModule } from '../module/packaging/build';
15
+ import { scanModuleDirectory, scanModuleScriptSource } from './module-script-scan';
16
+
17
+ const rules = (src: string) => scanModuleScriptSource('f.ts', src).map((v) => v.rule);
18
+
19
+ describe('module script scan — SSH rules', () => {
20
+ it('catches a hand-built ssh string', () => {
21
+ expect(rules("run(`ssh root@${ip} 'systemctl restart x'`);")).toContain(
22
+ 'raw ssh invocation (ssh … root@)',
23
+ );
24
+ });
25
+
26
+ it('catches StrictHostKeyChecking however it is invoked', () => {
27
+ expect(rules("const c = 'ssh -o StrictHostKeyChecking=no host';")).toContain(
28
+ 'raw ssh invocation (StrictHostKeyChecking)',
29
+ );
30
+ });
31
+
32
+ it('catches an ssh2 import', () => {
33
+ expect(rules("import { Client } from 'ssh2';")).toContain("'ssh2' import");
34
+ });
35
+
36
+ it('does not fire on ordinary module code', () => {
37
+ expect(rules('const x = probe(system, { kind: "systemd", unit: "caddy" }, run);')).toEqual([]);
38
+ });
39
+ });
40
+
41
+ describe('module script scan — raw-exec escape hatch', () => {
42
+ it('flags a runAppCommand call with no justification', () => {
43
+ expect(rules('const r = runAppCommand(system, "rm -f /tmp/x", run);')).toContain(
44
+ 'unjustified raw-exec escape hatch',
45
+ );
46
+ });
47
+
48
+ it('flags runAppCommandWithSecret too', () => {
49
+ expect(rules('const r = runAppCommandWithSecret(system, cli, secret, run);')).toContain(
50
+ 'unjustified raw-exec escape hatch',
51
+ );
52
+ });
53
+
54
+ it('accepts a call justified immediately above', () => {
55
+ const src = [
56
+ '// escape-hatch: forgejo admin user create is CLI-only, no HTTP API path.',
57
+ 'const r = runAppCommand(system, cmd, run);',
58
+ ].join('\n');
59
+ expect(rules(src)).toEqual([]);
60
+ });
61
+
62
+ it('accepts a call wrapped in waitFor, justified above the enclosing statement', () => {
63
+ // The real shape in modules/caddy-internal — the justification sits above
64
+ // `const ready = await waitFor(`, a few lines up from the call itself.
65
+ const src = [
66
+ '// escape-hatch: the command output IS the payload; no API to ask.',
67
+ 'const ready = await waitFor(',
68
+ ' () =>',
69
+ ' runAppCommand(target, CMD, run, {',
70
+ ' timeoutMs: 10_000,',
71
+ ' }).ok,',
72
+ ');',
73
+ ].join('\n');
74
+ expect(rules(src)).toEqual([]);
75
+ });
76
+
77
+ it('does not let a distant hatch launder a later call', () => {
78
+ const src = [
79
+ '// escape-hatch: justifies the call directly below it, and nothing else.',
80
+ 'const a = runAppCommand(system, one, run);',
81
+ ...Array(12).fill('doSomethingElse();'),
82
+ 'const b = runAppCommand(system, two, run);',
83
+ ].join('\n');
84
+ // Exactly one violation: the second call, which has no hatch in reach.
85
+ expect(rules(src)).toEqual(['unjustified raw-exec escape hatch']);
86
+ });
87
+
88
+ it('does not flag the import of runAppCommand', () => {
89
+ expect(rules("import { runAppCommand, probe } from '@celilo/capabilities';")).toEqual([]);
90
+ });
91
+ });
92
+
93
+ describe('module script scan — what it deliberately does not scan', () => {
94
+ let dir: string;
95
+
96
+ function write(rel: string, content: string): void {
97
+ const full = join(dir, rel);
98
+ mkdirSync(join(full, '..'), { recursive: true });
99
+ writeFileSync(full, content);
100
+ }
101
+
102
+ it('ignores bundled node_modules and test files', () => {
103
+ dir = mkdtempSync(join(tmpdir(), 'celilo-scan-test-'));
104
+ try {
105
+ write('manifest.yml', 'id: demo\n');
106
+ // @celilo/capabilities legitimately BUILDS the ssh string these rules ban.
107
+ // Scanning the bundled closure would fail every module in the fleet.
108
+ write(
109
+ 'scripts/node_modules/@celilo/capabilities/src/remote.ts',
110
+ 'const c = `ssh -o StrictHostKeyChecking=no root@${ip} ${cmd}`;',
111
+ );
112
+ write('scripts/setup.test.ts', "run('ssh root@host uptime');");
113
+ write('scripts/setup.ts', 'export const fine = 1;\n');
114
+ expect(scanModuleDirectory(dir)).toEqual([]);
115
+ } finally {
116
+ rmSync(dir, { recursive: true, force: true });
117
+ }
118
+ });
119
+ });
120
+
121
+ describe('packaging refuses a module that breaks the policy', () => {
122
+ let dir: string;
123
+
124
+ function write(rel: string, content: string): void {
125
+ const full = join(dir, rel);
126
+ mkdirSync(join(full, '..'), { recursive: true });
127
+ writeFileSync(full, content);
128
+ }
129
+
130
+ it('fails the build, naming the file, line and rule', async () => {
131
+ dir = mkdtempSync(join(tmpdir(), 'celilo-package-policy-'));
132
+ try {
133
+ write('manifest.yml', 'id: demo\nversion: 0.1.0\n');
134
+ write(
135
+ 'scripts/setup.ts',
136
+ ['export function bad(ip: string) {', ' return `ssh root@${ip} uptime`;', '}'].join('\n'),
137
+ );
138
+
139
+ const result = await buildModule({ sourceDir: dir });
140
+
141
+ expect(result.success).toBe(false);
142
+ expect(result.error).toContain('scripts/setup.ts:2');
143
+ expect(result.error).toContain('raw ssh invocation');
144
+ expect(result.error).toContain('MODULE_PRIMITIVES.md');
145
+ } finally {
146
+ rmSync(dir, { recursive: true, force: true });
147
+ }
148
+ });
149
+
150
+ it('fails the build for an unjustified escape hatch', async () => {
151
+ dir = mkdtempSync(join(tmpdir(), 'celilo-package-policy-'));
152
+ try {
153
+ write('manifest.yml', 'id: demo\nversion: 0.1.0\n');
154
+ write('scripts/setup.ts', 'const r = runAppCommand(system, "rm -rf /srv", run);\n');
155
+
156
+ const result = await buildModule({ sourceDir: dir });
157
+
158
+ expect(result.success).toBe(false);
159
+ expect(result.error).toContain('unjustified raw-exec escape hatch');
160
+ } finally {
161
+ rmSync(dir, { recursive: true, force: true });
162
+ }
163
+ });
164
+ });
@@ -0,0 +1,143 @@
1
+ /**
2
+ * The recurrence gate for openspec/changes/unified-management-no-ssh/proposal.md:
3
+ * **modules never hand-build SSH, and the one sanctioned raw-exec path is
4
+ * always justified in writing.**
5
+ *
6
+ * ONE definition of the rules, used by both enforcement points:
7
+ *
8
+ * - `apps/celilo/src/policy/no-hand-built-ssh.test.ts` — every in-repo module,
9
+ * on every `bun test`.
10
+ * - `apps/celilo/src/module/packaging/build.ts` — every `.netapp` at the
11
+ * moment it is packaged, including modules that never pass through this
12
+ * repo's CI (`bun run publish` is a documented escape hatch and runs no
13
+ * tests).
14
+ *
15
+ * Deliberately not two copies. The types this repo keeps re-learning that
16
+ * lesson on — `HookName` (celilo#821), `HookContext` — were duplicated
17
+ * declarations that drifted silently because nothing fails when two copies
18
+ * disagree. A scan rule is worse: the duplicate that drifts is the one that
19
+ * stops catching things, and a gate that stops catching things looks exactly
20
+ * like a gate with nothing to catch.
21
+ */
22
+
23
+ import { existsSync, readFileSync, readdirSync, statSync } from 'node:fs';
24
+ import { join, relative } from 'node:path';
25
+
26
+ export interface ScanViolation {
27
+ /** Path as the operator should see it — relative to the scanned root. */
28
+ file: string;
29
+ /** 1-based line number of the offending line. */
30
+ line: number;
31
+ /** Short rule name, e.g. `raw ssh invocation`. */
32
+ rule: string;
33
+ /** What to do instead. */
34
+ hint: string;
35
+ }
36
+
37
+ /**
38
+ * How far above a `runAppCommand*` call the justification may sit.
39
+ *
40
+ * Calibrated against the real call sites rather than guessed: the convention is
41
+ * a comment block directly above the call, but a call wrapped in `waitFor(() =>
42
+ * …)` puts the justification above the ENCLOSING statement, a few lines up. A
43
+ * window covers both without needing to parse TypeScript. Eight lines is the
44
+ * widest real gap plus headroom; wide enough to miss nothing legitimate, narrow
45
+ * enough that an unrelated hatch elsewhere in the function cannot launder a
46
+ * fresh call.
47
+ */
48
+ const ESCAPE_HATCH_LOOKBACK_LINES = 8;
49
+
50
+ const PATTERN_RULES: Array<{ rule: string; re: RegExp; hint: string }> = [
51
+ {
52
+ rule: 'raw ssh invocation (StrictHostKeyChecking)',
53
+ re: /StrictHostKeyChecking/,
54
+ hint: 'Use a remote-ops primitive (remoteExec/probe/serviceCtl/…). See MODULE_PRIMITIVES.md.',
55
+ },
56
+ {
57
+ rule: 'raw ssh invocation (ssh … root@)',
58
+ re: /\bssh\s+(?:-\S+\s+|\S*root@)/,
59
+ hint: 'Use a remote-ops primitive, not a hand-built ssh string. See MODULE_PRIMITIVES.md.',
60
+ },
61
+ {
62
+ rule: "'ssh2' import",
63
+ re: /(?:from|require\()\s*['"]ssh2['"]/,
64
+ hint: 'Modules never open their own SSH connection — use the primitives. See MODULE_PRIMITIVES.md.',
65
+ },
66
+ ];
67
+
68
+ /** A `runAppCommand(` / `runAppCommandWithSecret(` CALL — not the import. */
69
+ const RAW_EXEC_CALL = /\brunAppCommand(?:WithSecret)?\s*\(/;
70
+
71
+ const ESCAPE_HATCH_MARKER = /escape-hatch:/;
72
+
73
+ /**
74
+ * Scan one file's source. Pure — takes the text, returns the violations, so it
75
+ * is testable without a filesystem and reusable over a staged package.
76
+ */
77
+ export function scanModuleScriptSource(file: string, source: string): ScanViolation[] {
78
+ const violations: ScanViolation[] = [];
79
+ const lines = source.split('\n');
80
+
81
+ lines.forEach((text, i) => {
82
+ for (const { rule, re, hint } of PATTERN_RULES) {
83
+ if (re.test(text)) violations.push({ file, line: i + 1, rule, hint });
84
+ }
85
+
86
+ if (!RAW_EXEC_CALL.test(text)) return;
87
+ const from = Math.max(0, i - ESCAPE_HATCH_LOOKBACK_LINES);
88
+ const justified = lines.slice(from, i).some((l) => ESCAPE_HATCH_MARKER.test(l));
89
+ if (!justified) {
90
+ violations.push({
91
+ file,
92
+ line: i + 1,
93
+ rule: 'unjustified raw-exec escape hatch',
94
+ hint:
95
+ 'runAppCommand* is the ONLY sanctioned raw-exec path and every call site must say why ' +
96
+ 'no capability, HTTP or converge path exists. Add an `// escape-hatch: …` comment ' +
97
+ 'immediately above the call. See MODULE_PRIMITIVES.md.',
98
+ });
99
+ }
100
+ });
101
+
102
+ return violations;
103
+ }
104
+
105
+ /**
106
+ * Every production `.ts` under a module's `scripts/` — excluding tests and
107
+ * `node_modules`.
108
+ *
109
+ * The exclusion is load-bearing, not tidiness: the shipped closure bundles
110
+ * `@celilo/capabilities`, whose `remote.ts` builds the `ssh … root@` string
111
+ * that every one of these rules exists to keep OUT of module code. Scanning it
112
+ * would fail every module in the fleet on the implementation of the primitives
113
+ * they were told to use.
114
+ */
115
+ export function moduleScriptFiles(scriptsDir: string): string[] {
116
+ if (!existsSync(scriptsDir) || !statSync(scriptsDir).isDirectory()) return [];
117
+ const out: string[] = [];
118
+ const walk = (dir: string) => {
119
+ for (const entry of readdirSync(dir)) {
120
+ if (entry === 'node_modules') continue;
121
+ const p = join(dir, entry);
122
+ if (statSync(p).isDirectory()) walk(p);
123
+ else if (p.endsWith('.ts') && !p.endsWith('.test.ts')) out.push(p);
124
+ }
125
+ };
126
+ walk(scriptsDir);
127
+ return out;
128
+ }
129
+
130
+ /**
131
+ * Scan a module directory (the one holding `manifest.yml`). Returns every
132
+ * violation in its `scripts/`, with paths relative to `moduleDir`.
133
+ */
134
+ export function scanModuleDirectory(moduleDir: string): ScanViolation[] {
135
+ return moduleScriptFiles(join(moduleDir, 'scripts')).flatMap((f) =>
136
+ scanModuleScriptSource(relative(moduleDir, f), readFileSync(f, 'utf-8')),
137
+ );
138
+ }
139
+
140
+ /** Render violations for a test failure message or a refused publish. */
141
+ export function formatViolations(violations: ScanViolation[]): string {
142
+ return violations.map((v) => ` ${v.file}:${v.line}\n → ${v.rule}. ${v.hint}`).join('\n');
143
+ }
@@ -1,24 +1,20 @@
1
1
  /**
2
- * Recurrence gate for openspec/changes/unified-management-no-ssh/proposal.md: **modules never hand-build SSH.**
2
+ * Recurrence gate for openspec/changes/unified-management-no-ssh/proposal.md: **modules never hand-build SSH,
3
+ * and every raw-exec escape hatch is justified in writing.**
3
4
  *
4
- * Module hooks reach a remote box through the typed primitives in
5
- * `@celilo/capabilities` (probe / serviceCtl / applyRenderedConfig / see
6
- * apps/celilo/MODULE_PRIMITIVES.md), never a raw `ssh root@…` string, `ssh2`, or
7
- * an ad-hoc `child_process` shell-out. This test scans every production module
8
- * script and fails if a banned pattern reappears, so the SSH-elimination work
9
- * can't silently erode.
5
+ * The rules themselves live in `./module-script-scan`, because this is not the
6
+ * only place they run `.netapp` packaging applies the same scan at publish
7
+ * time, which is the enforcement point that also covers a module built outside
8
+ * this repo's CI. One definition, two callers.
10
9
  *
11
- * Precise on purpose: `noRestrictedImports` can't see raw ssh *strings* (the real
12
- * risk). We ban the SSH shapes themselves — an `ssh root@…` string catches
13
- * hand-built SSH however it's invoked (child_process, execSync, or the Runner),
14
- * and `ssh2` catches the library route. We do NOT ban `child_process` outright:
15
- * many modules' on_install legitimately shell LOCAL `celilo`/system commands
16
- * (`celilo system apply-config`), which isn't remote SSH.
10
+ * This file is the in-repo half: every production module script, on every
11
+ * `bun test`.
17
12
  */
18
13
 
19
14
  import { describe, expect, test } from 'bun:test';
20
- import { existsSync, readFileSync, readdirSync, statSync } from 'node:fs';
15
+ import { existsSync, readdirSync, statSync } from 'node:fs';
21
16
  import { join, resolve } from 'node:path';
17
+ import { formatViolations, moduleScriptFiles, scanModuleDirectory } from './module-script-scan';
22
18
 
23
19
  /** Walk up from this test to the repo root (the dir holding both modules/ and apps/). */
24
20
  function repoRoot(): string {
@@ -30,61 +26,25 @@ function repoRoot(): string {
30
26
  throw new Error('could not locate repo root (no ancestor with modules/ + apps/)');
31
27
  }
32
28
 
33
- /** Every production `.ts` under modules/<m>/scripts/ (excludes node_modules + tests). */
34
- function moduleScripts(): string[] {
29
+ function moduleDirs(): string[] {
35
30
  const modulesRoot = join(repoRoot(), 'modules');
36
- const out: string[] = [];
37
- const walk = (dir: string) => {
38
- for (const entry of readdirSync(dir)) {
39
- if (entry === 'node_modules') continue;
40
- const p = join(dir, entry);
41
- if (statSync(p).isDirectory()) walk(p);
42
- else if (p.endsWith('.ts') && !p.endsWith('.test.ts')) out.push(p);
43
- }
44
- };
45
- for (const mod of readdirSync(modulesRoot)) {
46
- const scripts = join(modulesRoot, mod, 'scripts');
47
- if (existsSync(scripts) && statSync(scripts).isDirectory()) walk(scripts);
48
- }
49
- return out;
31
+ return readdirSync(modulesRoot)
32
+ .map((m) => join(modulesRoot, m))
33
+ .filter((d) => statSync(d).isDirectory() && existsSync(join(d, 'scripts')));
50
34
  }
51
35
 
52
- const BANNED = [
53
- {
54
- name: 'raw ssh invocation (StrictHostKeyChecking)',
55
- re: /StrictHostKeyChecking/,
56
- hint: 'Use a remote-ops primitive (remoteExec/probe/serviceCtl/…). See MODULE_PRIMITIVES.md.',
57
- },
58
- {
59
- name: 'raw ssh invocation (ssh … root@)',
60
- re: /\bssh\s+(?:-\S+\s+|\S*root@)/,
61
- hint: 'Use a remote-ops primitive, not a hand-built ssh string. See MODULE_PRIMITIVES.md.',
62
- },
63
- {
64
- name: "'ssh2' import",
65
- re: /(?:from|require\()\s*['"]ssh2['"]/,
66
- hint: 'Modules never open their own SSH connection — use the primitives. See MODULE_PRIMITIVES.md.',
67
- },
68
- ];
69
-
70
36
  describe('recurrence gate: modules never hand-build SSH', () => {
71
- const scripts = moduleScripts();
37
+ const dirs = moduleDirs();
72
38
 
73
39
  test('scans a non-trivial set of module scripts (sanity — the scan actually ran)', () => {
74
- expect(scripts.length).toBeGreaterThan(10);
40
+ const scanned = dirs.flatMap((d) => moduleScriptFiles(join(d, 'scripts')));
41
+ expect(scanned.length).toBeGreaterThan(10);
75
42
  });
76
43
 
77
- test('no production module script hand-builds SSH (ssh string / ssh2)', () => {
78
- const violations: string[] = [];
79
- for (const file of scripts) {
80
- const src = readFileSync(file, 'utf-8');
81
- for (const b of BANNED) {
82
- if (b.re.test(src)) violations.push(`${file}\n → ${b.name}. ${b.hint}`);
83
- }
84
- }
85
- expect(
86
- violations,
87
- `Hand-built SSH found in module scripts:\n ${violations.join('\n ')}`,
88
- ).toEqual([]);
44
+ test('no production module script hand-builds SSH, and every runAppCommand* is justified', () => {
45
+ const violations = dirs.flatMap((d) => scanModuleDirectory(d));
46
+ expect(violations, `Module script policy violations:\n${formatViolations(violations)}`).toEqual(
47
+ [],
48
+ );
89
49
  });
90
50
  });
@@ -0,0 +1,69 @@
1
+ /**
2
+ * `copyAnsibleRoleFilesDirs` had no test, and celilo#925 is what that cost.
3
+ *
4
+ * A module's Ansible role `files/` directory holds static assets — most often a
5
+ * compiled binary — that cannot go through the utf-8 template pipeline. They are
6
+ * copied verbatim on every generate. Except they were not: the copy omitted
7
+ * `force`, which Node defaults to true and bun does not, so the FIRST generate
8
+ * populated `generated/` and no later one ever replaced it.
9
+ *
10
+ * Nothing surfaced it. `cp` reports no error, so the caller's try/catch caught
11
+ * nothing; Ansible then installed the first binary forever and reported `ok`,
12
+ * unchanged, while the module's version field advanced past it. On the live
13
+ * fleet that read as a successful deploy of code that never shipped.
14
+ *
15
+ * The overwrite case is therefore the point of this file. A test that only
16
+ * copied into an empty directory would have passed throughout.
17
+ */
18
+ import { describe, expect, test } from 'bun:test';
19
+ import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from 'node:fs';
20
+ import { tmpdir } from 'node:os';
21
+ import { join } from 'node:path';
22
+ import { copyAnsibleRoleFilesDirs } from './generator';
23
+
24
+ function moduleWithRoleFile(binary: string): string {
25
+ const root = mkdtempSync(join(tmpdir(), 'celilo-rolefiles-'));
26
+ const filesDir = join(root, 'ansible', 'roles', 'demo', 'files');
27
+ mkdirSync(filesDir, { recursive: true });
28
+ writeFileSync(join(filesDir, 'demo-linux-x86_64'), binary);
29
+ return root;
30
+ }
31
+
32
+ const generatedBinary = (out: string): string =>
33
+ readFileSync(join(out, 'ansible', 'roles', 'demo', 'files', 'demo-linux-x86_64'), 'utf-8');
34
+
35
+ describe('copyAnsibleRoleFilesDirs', () => {
36
+ test('populates an empty generated tree', async () => {
37
+ const modulePath = moduleWithRoleFile('v1');
38
+ const outputPath = mkdtempSync(join(tmpdir(), 'celilo-out-'));
39
+
40
+ await copyAnsibleRoleFilesDirs(modulePath, outputPath);
41
+
42
+ expect(generatedBinary(outputPath)).toBe('v1');
43
+ });
44
+
45
+ /**
46
+ * celilo#925 in one assertion. This is the case that regressed, and the only
47
+ * one that can catch it: the destination already exists, and a rebuilt
48
+ * artifact has to replace it.
49
+ */
50
+ test('OVERWRITES an artifact a previous generate already placed', async () => {
51
+ const outputPath = mkdtempSync(join(tmpdir(), 'celilo-out-'));
52
+
53
+ await copyAnsibleRoleFilesDirs(moduleWithRoleFile('v1'), outputPath);
54
+ expect(generatedBinary(outputPath)).toBe('v1');
55
+
56
+ // The module is rebuilt at a new version; generate runs again.
57
+ await copyAnsibleRoleFilesDirs(moduleWithRoleFile('v2'), outputPath);
58
+
59
+ expect(generatedBinary(outputPath)).toBe('v2');
60
+ });
61
+
62
+ test('a module with no role files/ directory is not an error', async () => {
63
+ const root = mkdtempSync(join(tmpdir(), 'celilo-norole-'));
64
+ mkdirSync(join(root, 'ansible', 'roles', 'demo', 'tasks'), { recursive: true });
65
+ const outputPath = mkdtempSync(join(tmpdir(), 'celilo-out-'));
66
+
67
+ await copyAnsibleRoleFilesDirs(root, outputPath);
68
+ });
69
+ });
@@ -500,7 +500,29 @@ export async function copyAnsibleRoleFilesDirs(
500
500
  if (!existsSync(srcFilesDir)) continue;
501
501
  const destFilesDir = join(outputPath, 'ansible', 'roles', role.name, 'files');
502
502
  await mkdir(dirname(destFilesDir), { recursive: true });
503
- await cp(srcFilesDir, destFilesDir, { recursive: true, preserveTimestamps: true });
503
+ // `force: true` is LOAD-BEARING on bun, and its absence was celilo#925.
504
+ //
505
+ // Node defaults `force` to true, so this looked correct and is correct
506
+ // under Node. Bun 1.3.3 does not, on this path specifically — measured,
507
+ // copying "NEW" over an existing "OLD":
508
+ //
509
+ // recursive only -> NEW
510
+ // recursive + force -> NEW
511
+ // recursive + preserveTimestamps -> OLD <- what this was
512
+ // recursive + force + preserveTimestamps -> NEW
513
+ //
514
+ // celilo runs on bun. So a module's built binary landed in `generated/`
515
+ // exactly once, at first generate, and no later version ever replaced it —
516
+ // silently, because `cp` reports no error, so the caller's try/catch has
517
+ // nothing to catch. Ansible then copies that first binary forever and
518
+ // reports `ok`, unchanged, while the module's version field advances.
519
+ //
520
+ // The sibling call in `storage-set-path.ts:153` already passes `force`.
521
+ await cp(srcFilesDir, destFilesDir, {
522
+ recursive: true,
523
+ force: true,
524
+ preserveTimestamps: true,
525
+ });
504
526
  }
505
527
  }
506
528