@celilo/cli 1.3.0 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CELILO_SUBSYSTEMS.md +2 -2
- package/package.json +2 -2
- package/src/hooks/types.ts +20 -74
- package/src/manifest/contracts/v1.ts +35 -0
- package/src/manifest/schema.ts +2 -0
- package/src/module/packaging/build.ts +29 -0
- package/src/policy/module-business-baseline.ts +51 -35
- package/src/policy/module-script-scan.test.ts +164 -0
- package/src/policy/module-script-scan.ts +143 -0
- package/src/policy/no-hand-built-ssh.test.ts +22 -62
- package/src/templates/copy-role-files.test.ts +69 -0
- package/src/templates/generator.ts +23 -1
package/CELILO_SUBSYSTEMS.md
CHANGED
|
@@ -29,7 +29,7 @@ see `openspec/specs/`. Companion doc: [CELILO_CORE_MODULES.md](./CELILO_CORE_MOD
|
|
|
29
29
|
- **Zone detection / system config** — `apps/celilo/src/services/zone-detector.ts` — `detectZoneFromIp` reads `network.<zone>.subnet` from the `systemConfig` table and returns `NetworkZone | 'unknown'`. It answers CONTAINMENT ONLY. It used to return `'external'` on no-match, which conflated "no declared subnet contains this" with "the internet can route to this" — on a firewall with five RFC1918 legs that reported four of them as facing the internet. `'unknown'` is the honest answer; the caller resolves it (see `machine add`: publicly routable → `external`, otherwise fail asking for `--zone`). The subnet-backed zone list is derived from `NETWORK_ZONES` minus `external`, which has no subnet and must never be given one.
|
|
30
30
|
- **Interface classification** — `packages/capabilities/src/interface-classification.ts` — THE shared classifier, used by the backend and every firewall provider module so the two cannot drift apart again. `isPubliclyRoutable(ip)` is a property of the address alone (false for RFC 1918, RFC 6598 carrier-grade NAT, loopback, link-local, multicast, reserved). `classifyInterfaces(interfaces, zones)` assigns each interface `zone → external → alien`, first match winning, where `external` is the RESIDUAL — routable and claimed by no declared zone — and is never subnet-matched. `externalEdge()` returns none/single/**ambiguous** rather than silently picking the first public address. `defaultRouteFinding()` enforces the invariant that the default route leaves through `internal` or `external`. **`subnetContains(cidr, ip)` lives here and is the ONLY implementation** — three existed and disagreed (the backend's mishandled `/0`); the other two are deleted, not aliased. Design: `openspec/changes/firewall-interface-classification/design.md`.
|
|
31
31
|
- **Declared networks (classification input)** — `readDeclaredNetworks(db)` in `apps/celilo/src/hooks/capability-loader.ts` — every `network.<name>.subnet` in system config, which is what an interface is attributed against. Read from the CONFIG, not from `NETWORK_ZONES`: celilo holds networks that are not placement zones (`network.control-plane-vpn.subnet`, which `wireguard` requires and reads). Injected into the firewall capability as a LIVE reader (`declaredNetworks`) — that liveness was a mitigation for values written mid-run by a module hook, which `network-declaration` removes; see that spec before assuming a snapshot is still unsafe.
|
|
32
|
-
- **Network requirement + ensure (celilo owns the namespace)** — `apps/celilo/src/services/network-ensure.ts` (`ensureRequiredNetworks`), `NetworkRequirementSchema` / `getRequiredNetworkNames` in `apps/celilo/src/manifest/schema.ts`. A module declares `requires.networks: [{name}]` — a NAME, never a value; the schema is `.strict()` so a `subnet:` on the requirement is rejected with a message saying why. The deploy calls `ensureRequiredNetworks` in its interview phase, BEFORE generation and before any hook, and asks over the generic bus interview (`askText`, so it is answerable headless) for anything undefined. Which attributes a network has is celilo's answer, taken from `apps/celilo/schemas/system_config.json`: that file declares `network.<n>.gateway` for the routed segments and omits it for `control-plane-vpn`, so a gateway is never asked for a network that has none. Well-known names carry a `suggested` range there — deliberately NOT `default`, which `getDefaultConfiguration()` would seed at `system init`. Spec: `openspec/
|
|
32
|
+
- **Network requirement + ensure (celilo owns the namespace)** — `apps/celilo/src/services/network-ensure.ts` (`ensureRequiredNetworks`), `NetworkRequirementSchema` / `getRequiredNetworkNames` in `apps/celilo/src/manifest/schema.ts`. A module declares `requires.networks: [{name}]` — a NAME, never a value; the schema is `.strict()` so a `subnet:` on the requirement is rejected with a message saying why. The deploy calls `ensureRequiredNetworks` in its interview phase, BEFORE generation and before any hook, and asks over the generic bus interview (`askText`, so it is answerable headless) for anything undefined. Which attributes a network has is celilo's answer, taken from `apps/celilo/schemas/system_config.json`: that file declares `network.<n>.gateway` for the routed segments and omits it for `control-plane-vpn`, so a gateway is never asked for a network that has none. Well-known names carry a `suggested` range there — deliberately NOT `default`, which `getDefaultConfiguration()` would seed at `system init`. Spec: `openspec/specs/network-declaration/spec.md`.
|
|
33
33
|
- **The network write path (closed) + celilo's own discovery** — `celilo system apply-config` (`apps/celilo/src/cli/commands/system-apply-config.ts`) REFUSES the whole `network.` namespace, `network.bridge` excepted (a Proxmox bridge name is not addressing, and it is the one network key with a schema default). That is the automation surface a module hook shells out to, so closing it there is what makes "networks are celilo's" an authority rather than a convention every module has to remember. The refusal names the alternative — declare the network, read it with `$system:` — because a bare rejection sends a module author hunting for a typo. The one write that legitimately needed the surface moved INTO celilo: `celilo system discover-network` (`apps/celilo/src/services/network-discovery.ts`, `cli/commands/system-discover-network.ts`) parses the box's own `ip route` and records `network.internal.*` — or `network.secure-mgmt.*` when the box is off the internal LAN (#300). `celilo-mgmt` calls it and decides nothing; it used to parse and write this itself. Idempotent and never overwrites addressing already set. Recurrence gate: `test-integration/module/no-module-writes-networks.test.ts`.
|
|
34
34
|
- **Firewall interface audit** — `apps/celilo/src/services/audit/interface-classification.ts` — reports per-firewall classification in `celilo audit`: alien interfaces by name and address (drift), and the blocking findings a converge refuses on — a carrier-grade NAT leg, an ambiguous external edge, a default route on the wrong leg.
|
|
35
35
|
- **Zone taxonomy (canonical list)** — `apps/celilo/src/db/schema.ts` — `NETWORK_ZONES` is the single array; `NetworkZone` is DERIVED from it. Never hand-maintain a second copy: a duplicate that dropped a member made zone validation return null and silently fall back to a wrong-but-valid zone.
|
|
@@ -181,7 +181,7 @@ runner seam (`execRunner` real / `createMockRunner` for tests) lives in
|
|
|
181
181
|
- **Inbound** (`reconcileAspectsForSystems`): every approved aspect applied to systems that have just come into existence. Called from `module-deploy.ts` between `waitForSSH` and `executeAnsible` (so a module's playbook and `on_install` see a correctly configured host) and from `machine-add.ts`. Eligibility is `applicable_zones` + approval; the aspect's `triggers` list is deliberately NOT consulted, so convergence is not opt-in per manifest. A failure here IS fatal to the deploy that created the system — the aspect is a prerequisite of that host — while a failure on `machine add` is only a warning. PAUSED providers are skipped, which is the escape hatch for a wedged aspect. Nothing is rolled back: rows, guest and IPAM allocation persist and a re-run converges on the same system. Fixes celilo#902, where a system provisioned after its provider deployed silently never received the aspect. See `openspec/changes/aspect-fanout-new-systems/`.
|
|
182
182
|
- **Coverage verification** — `verifyAspectCoverage` + `celilo system doctor --deep [--fix]`. Answers "is any system missing an aspect its zone entitles it to" WITHOUT stored state: the entitled set is `planAspectFanOut` itself, and coverage is measured by evaluating the role against the host in Ansible check mode (`executeAnsible({check:true})` + `parseAnsibleRecap`). Three outcomes, not two — `changed=0,skipped=0` applied, `changed>0` missing, **`skipped>0` unknown**, because check mode SKIPS a task it cannot evaluate and a `command`/`shell` role would otherwise report clean having never run.
|
|
183
183
|
- **In-flight operation lock** — `apps/celilo/src/services/module-operations.ts` — `startOperation`/`completeOperation`/`failOperation` record deploy/uninstall/backup/restore in `module_operations`; `refuseIfInFlight`/`checkInFlight` are what backup and restore consult. Deploy and uninstall REGISTER but never check: it is a one-way guard protecting backup/restore consistency, not a general mutex (`openspec/specs/management-server-backup/spec.md` "In-flight operation refusal"). A row stops holding the lock once it is GONE, STOPPED/zombie (`isPidRunnable`, `ps -o state=` — `kill(pid,0)` calls a Ctrl-Z'd process alive), or older than `OPERATION_TTL_MS` (2h). The TTL is not redundancy: a pid is a recycled number, and once the pid space wraps an old row names an unrelated healthy process. Operator surface: `celilo module operations [list|clear] [--abandoned] [--all]` (`apps/celilo/src/cli/commands/module-operations.ts`); `list` shows only what holds the lock, abandoned rows are summarised unless `--abandoned`. Abandoned rows are reclaimed hourly by the `celilo-operations-sweep` bus subscriber (`timer.tick.1h` → `celilo module operations clear`, armed by `ensureOperationsSweepSubscriber` from module registration and `celilo system migrate`). `clear` MARKS rows failed rather than deleting them, and that is load-bearing: the `abandoned_operations` audit reads exactly those released rows to notice one module's operation dying over and over.
|
|
184
|
-
- **Module pause / unpause (control-plane quiescence)** — `apps/celilo/src/services/module-pause.ts` — pure `planPause`/`planUnpause` producing an ordered plan, `executePause`/`executeUnpause` performing it, plus `listPausedModules`/`pausedAmong`/`formatPausedDuration`/`describePausedModule` (the ONE place an age is formatted). CLI: `apps/celilo/src/cli/commands/module-pause.ts` (`celilo module pause|unpause <id> [--cascade] [--stop-infra] [--reason] [--dry-run] [--yes]`). Pausing takes a module out of the CONTROL plane — no dispatched events, no timer hooks, no health checks, alerts suppressed — while leaving the DATA plane running, because capability consumption is deploy-time: every consumer calls `firewall`/`dhcp_server` from `on_install` and nothing calls it while serving. Config, secrets, IPAM/VMID and placement are preserved; `on_uninstall` does NOT run. Quiescence is enforced in two places: pausing drops the module's bus subscriptions (`unregisterModuleSubscriptions`), and `run-named-hook.ts` refuses any non-lifecycle hook for a PAUSED module (`skippedPaused`), which catches the paths that skip the bus — `events resync-subscriptions`, a restore that starts events.db empty, aspect fan-out, public-web republish. `on_install`/`on_uninstall` are exempt by hook NAME (not a caller flag): unpause redeploys through `on_install`, and removing a paused provider needs `on_uninstall`. Unpause always REDEPLOYS (`deployModule`) — that is what rebinds a consumer to a replacement provider and recreates provider-local state from the consumers that own it — and re-registers subscriptions, which a plain deploy does not do. A failed unpause restores `PAUSED` rather than leaving the module live and mis-bound. Cascade order reuses `services/update/dep-graph.ts` unchanged (pause = consumers first, unpause = providers first) and is computed from the GRAPH, never from which modules are currently paused, so a cascade walks THROUGH already-done members and is resumable. See `openspec/
|
|
184
|
+
- **Module pause / unpause (control-plane quiescence)** — `apps/celilo/src/services/module-pause.ts` — pure `planPause`/`planUnpause` producing an ordered plan, `executePause`/`executeUnpause` performing it, plus `listPausedModules`/`pausedAmong`/`formatPausedDuration`/`describePausedModule` (the ONE place an age is formatted). CLI: `apps/celilo/src/cli/commands/module-pause.ts` (`celilo module pause|unpause <id> [--cascade] [--stop-infra] [--reason] [--dry-run] [--yes]`). Pausing takes a module out of the CONTROL plane — no dispatched events, no timer hooks, no health checks, alerts suppressed — while leaving the DATA plane running, because capability consumption is deploy-time: every consumer calls `firewall`/`dhcp_server` from `on_install` and nothing calls it while serving. Config, secrets, IPAM/VMID and placement are preserved; `on_uninstall` does NOT run. Quiescence is enforced in two places: pausing drops the module's bus subscriptions (`unregisterModuleSubscriptions`), and `run-named-hook.ts` refuses any non-lifecycle hook for a PAUSED module (`skippedPaused`), which catches the paths that skip the bus — `events resync-subscriptions`, a restore that starts events.db empty, aspect fan-out, public-web republish. `on_install`/`on_uninstall` are exempt by hook NAME (not a caller flag): unpause redeploys through `on_install`, and removing a paused provider needs `on_uninstall`. Unpause always REDEPLOYS (`deployModule`) — that is what rebinds a consumer to a replacement provider and recreates provider-local state from the consumers that own it — and re-registers subscriptions, which a plain deploy does not do. A failed unpause restores `PAUSED` rather than leaving the module live and mis-bound. Cascade order reuses `services/update/dep-graph.ts` unchanged (pause = consumers first, unpause = providers first) and is computed from the GRAPH, never from which modules are currently paused, so a cascade walks THROUGH already-done members and is resumable. See `openspec/specs/module-pause/spec.md`.
|
|
185
185
|
- **Provider-removal guard** — `apps/celilo/src/services/remove-guard.ts` — `findRemovalBlockers`/`describeRemovalRefusal`, called from `apps/celilo/src/cli/commands/module-remove.ts`. A PAUSED module is not a dependent (unpause cannot return it to service without a redeploy, and a redeploy re-resolves capabilities), which is what makes a provider swap possible at all. A dependent is one declaring the capability under `requires` **or** `optional` — the same relation `dep-graph.ts` uses, so the guard and the cascade agree on the set; the guard previously read `requires` alone, which let a removal silently orphan `technitium`'s `optional` `dhcp_server`. Refusals name each blocker AND which declaration makes it one. It deliberately does NOT exempt a dependent because another provider of the same capability exists (celilo#683).
|
|
186
186
|
- **Consumer-removal cleanup (every provider is told)** — `apps/celilo/src/services/consumer-cleanup.ts` — pure `planConsumerCleanup` + `loadConsumerCleanupPlan` + `runConsumerCleanup`, called from `apps/celilo/src/cli/commands/module-remove.ts` after `on_uninstall` and before `terraform destroy`. A capability is two-sided: the consumer asks, the provider mints something in ITS world (a caddy site block, a DNAT rule, an OIDC client at authentik, a registered CI runner), and removal only ever touched one side — the FK cascade dropped celilo's row, so the provider's next converge had no way to learn the thing existed. This dispatches the `on_consumer_removed` hook to every provider of every capability the departing module declared under `requires` OR `optional` (the same relation `remove-guard.ts` counts as a dependency edge), **once per provider** rather than once per capability, sorted by provider id. The hook receives one input, `consumer`, and NOTHING else: a provider that cannot answer "what do I hold for this module" without being told has a different defect — the consumer's id was never recorded at mint time. It replaces `services/web-route-cleanup.ts`, which did the same job for exactly one capability, by name, from core. Semantics that are easy to get backwards: **a failed withdrawal never blocks the removal** — the consumer goes and the failing PROVIDER is marked `ERROR` with the departing consumer named in `error_message` (surfaced as a `blocked` finding by `services/audit/undeployed-modules.ts`), because the hook is a full converge and after a failure the provider's state is unknown rather than "one thing missed". **Dispatch continues past a failure**, so one broken provider cannot leave the others holding state. A PAUSED provider is skipped with a warning naming what it keeps (`run-named-hook.ts` refuses non-lifecycle hooks on a paused module, and `on_consumer_removed` must NOT join `LIFECYCLE_HOOKS`), and a never-deployed one is skipped silently. Providers implementing it: caddy, caddy-internal, iptables, greenwave, axon, authentik, forgejo, generic-cpanel-hosting-provider. See `openspec/changes/consumer-removal-cleanup/`.
|
|
187
187
|
- **Firewall registry ownership** — `apps/celilo/src/services/port-forwards.ts` + `apps/celilo/src/services/trusted-sources.ts`. Both stores are bound to the CONSUMING module and stamp `registered_by` themselves; a caller cannot supply it, so a registration is never attributed to the wrong module. Both writes are **declarative**: `replace()` states a consumer's COMPLETE set for a target, so a port or subnet it previously registered and now omits is withdrawn (celilo#855 — before this, a module that exposed `:8080` and redeployed exposing `:9090` kept both, forever). The owner is IN both unique indexes, not merely beside them: two consumers wanting the same forward are two ROWS, so one leaving cannot delete a rule the other still needs; `modules/iptables/scripts/ruleset-renderer.ts` dedupes on the rule tuple so the pair renders once. Neither column is a FK, so `runConsumerCleanup` deletes these rows explicitly after every provider has converged without them.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@celilo/cli",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.5.0",
|
|
4
4
|
"description": "Celilo — home lab orchestration CLI",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -58,7 +58,7 @@
|
|
|
58
58
|
"dependencies": {
|
|
59
59
|
"@aws-sdk/client-s3": "^3.1109.0",
|
|
60
60
|
"@aws-sdk/lib-storage": "^3.1101.0",
|
|
61
|
-
"@celilo/capabilities": "^2.
|
|
61
|
+
"@celilo/capabilities": "^2.2.0",
|
|
62
62
|
"@celilo/cli-display": "^0.2.0",
|
|
63
63
|
"@celilo/core": "^0.9.0",
|
|
64
64
|
"@celilo/event-bus": "^0.6.0",
|
package/src/hooks/types.ts
CHANGED
|
@@ -6,70 +6,22 @@
|
|
|
6
6
|
* (e.g., container_created triggers web automation for API key creation).
|
|
7
7
|
*/
|
|
8
8
|
|
|
9
|
-
import type {
|
|
9
|
+
import type { HookDefinition, HookResult as ModuleHookResult } from '@celilo/capabilities';
|
|
10
10
|
|
|
11
11
|
/**
|
|
12
|
-
*
|
|
12
|
+
* `HookDefinition`, `HookLogger` and `HookContext` are re-exported from
|
|
13
|
+
* `@celilo/capabilities`, which owns them. All three were declared a second
|
|
14
|
+
* time here and the `HookLogger` copy said so in its own comment ("kept in
|
|
15
|
+
* sync") — a duplicate that announces it needs syncing is one that will
|
|
16
|
+
* eventually not be.
|
|
13
17
|
*
|
|
14
|
-
*
|
|
15
|
-
*
|
|
16
|
-
*
|
|
17
|
-
* the
|
|
18
|
+
* A second declaration of a shared shape does not fail loudly when it drifts:
|
|
19
|
+
* both sides keep compiling and the two meanings diverge silently. `HookName`
|
|
20
|
+
* was the same duplication and HAD already drifted — it was missing
|
|
21
|
+
* `reconcile_routes`, so caddy's reconcile hook was a hook the celilo side of
|
|
22
|
+
* the codebase did not believe in (celilo#821).
|
|
18
23
|
*/
|
|
19
|
-
export
|
|
20
|
-
/** Path to hook script relative to module directory */
|
|
21
|
-
script: string;
|
|
22
|
-
/** Total timeout in milliseconds (default: 60000) */
|
|
23
|
-
timeout?: number;
|
|
24
|
-
/** If true, hook needs interactive terminal (no FuelGauge wrapping) */
|
|
25
|
-
interactive?: boolean;
|
|
26
|
-
}
|
|
27
|
-
|
|
28
|
-
/**
|
|
29
|
-
* Logger interface provided to hook scripts.
|
|
30
|
-
*
|
|
31
|
-
* Mirrors `HookLogger` in `@celilo/capabilities/types` — kept in sync so
|
|
32
|
-
* concrete loggers we build here (createGaugeLogger, createConsoleLogger,
|
|
33
|
-
* createCapturingLogger) satisfy both the in-process consumers and the
|
|
34
|
-
* cross-package contract used by `wrapWithLogging`.
|
|
35
|
-
*/
|
|
36
|
-
export interface HookLogger {
|
|
37
|
-
info(message: string): void;
|
|
38
|
-
warn(message: string): void;
|
|
39
|
-
error(message: string): void;
|
|
40
|
-
success(message: string): void;
|
|
41
|
-
/** Begin a logical span. Subsequent log calls nest under it visually. */
|
|
42
|
-
beginStep?(name: string): void;
|
|
43
|
-
/** End the current span (success). */
|
|
44
|
-
endStep?(name: string): void;
|
|
45
|
-
/** End the current span (failure) with an error message. */
|
|
46
|
-
failStep?(name: string, error: string): void;
|
|
47
|
-
}
|
|
48
|
-
|
|
49
|
-
/**
|
|
50
|
-
* Context passed to hook scripts
|
|
51
|
-
*/
|
|
52
|
-
export interface HookContext {
|
|
53
|
-
/** Module configuration values */
|
|
54
|
-
config: Record<string, unknown>;
|
|
55
|
-
/** Module secret values (decrypted) */
|
|
56
|
-
secrets: Record<string, string>;
|
|
57
|
-
/**
|
|
58
|
-
* The 0..N systems this module has deployed onto (openspec/specs/module-systems-addressing/spec.md).
|
|
59
|
-
* Always an array — no singular convenience, so the 0/1/N reality stays visible.
|
|
60
|
-
*/
|
|
61
|
-
systems: DeployedSystem[];
|
|
62
|
-
/** Logger for reporting progress */
|
|
63
|
-
logger: HookLogger;
|
|
64
|
-
/** Run in debug mode (e.g., Playwright headless: false) */
|
|
65
|
-
debug: boolean;
|
|
66
|
-
/** Directory for saving screenshots on failure */
|
|
67
|
-
screenshotDir: string;
|
|
68
|
-
/** Capability function interfaces from provider modules (e.g., context.capabilities.dns_registrar) */
|
|
69
|
-
capabilities: Record<string, unknown>;
|
|
70
|
-
/** Dynamic inputs specified in manifest (e.g., vps_ip) */
|
|
71
|
-
[key: string]: unknown;
|
|
72
|
-
}
|
|
24
|
+
export type { HookDefinition, HookLogger, HookContext } from '@celilo/capabilities';
|
|
73
25
|
|
|
74
26
|
/**
|
|
75
27
|
* Surfaced by `invokeHook` when a capability call inside the hook threw
|
|
@@ -85,21 +37,15 @@ export interface MissingProviderInputDetails {
|
|
|
85
37
|
}
|
|
86
38
|
|
|
87
39
|
/**
|
|
88
|
-
* Result returned from hook execution
|
|
40
|
+
* Result returned from hook execution.
|
|
41
|
+
*
|
|
42
|
+
* The shape a module's hook produces is `@celilo/capabilities`' `HookResult`;
|
|
43
|
+
* this adds the one field only the framework sets. A module never writes
|
|
44
|
+
* `missingProviderInput` — the executor sets it when a capability call throws
|
|
45
|
+
* `MissingProviderInputError` — so it belongs on the framework's view of the
|
|
46
|
+
* result rather than in the module-facing contract.
|
|
89
47
|
*/
|
|
90
|
-
export interface HookResult {
|
|
91
|
-
success: boolean;
|
|
92
|
-
outputs: Record<string, unknown>;
|
|
93
|
-
error?: string;
|
|
94
|
-
/**
|
|
95
|
-
* Every file the hook wrote to its per-run artifact directory.
|
|
96
|
-
* All of them, not just the newest image: a screenshot without the
|
|
97
|
-
* DOM and the observed requests is the least useful third of a
|
|
98
|
-
* post-mortem.
|
|
99
|
-
*/
|
|
100
|
-
artifactPaths?: string[];
|
|
101
|
-
/** Duration in milliseconds */
|
|
102
|
-
duration: number;
|
|
48
|
+
export interface HookResult extends ModuleHookResult {
|
|
103
49
|
/**
|
|
104
50
|
* Set when the hook failed because a capability call needs the
|
|
105
51
|
* framework to extend another module's config. Mutually exclusive
|
|
@@ -261,6 +261,41 @@ export const V1_HOOKS: ContractHooks = {
|
|
|
261
261
|
inputs: {},
|
|
262
262
|
outputs: {},
|
|
263
263
|
},
|
|
264
|
+
/**
|
|
265
|
+
* Report what a tunnel/interface is ACTUALLY carrying, diffed against what
|
|
266
|
+
* celilo's config says it should carry.
|
|
267
|
+
*
|
|
268
|
+
* Deliberately a read of the live thing rather than of celilo's own record.
|
|
269
|
+
* celilo#928 was exactly a divergence between the two — a device present in
|
|
270
|
+
* config, rendered as active by a UI, and absent from the interface — and
|
|
271
|
+
* diagnosing it needed a screenshot and a root SSH because nothing celilo
|
|
272
|
+
* offered could tell the two apart. A hook that reported the record would
|
|
273
|
+
* have reproduced the misleading signal rather than exposing it.
|
|
274
|
+
*
|
|
275
|
+
* No structured outputs: it reports through the logger, so an operator and an
|
|
276
|
+
* agent read the same thing.
|
|
277
|
+
*/
|
|
278
|
+
list_peers: {
|
|
279
|
+
inputs: {},
|
|
280
|
+
outputs: {},
|
|
281
|
+
},
|
|
282
|
+
/**
|
|
283
|
+
* Re-assert a module's desired peer set onto the thing carrying it, on a
|
|
284
|
+
* timer, without an operator.
|
|
285
|
+
*
|
|
286
|
+
* The counterpart to applying on change: apply-on-change bounds latency, this
|
|
287
|
+
* bounds how long ANY divergence can persist — including drift nobody
|
|
288
|
+
* predicted (a hand-edit on the box, an apply that failed, a record written
|
|
289
|
+
* before the applying code existed). celilo#934, where the module could
|
|
290
|
+
* already SEE the drift in `health_check` and had nowhere to act on it.
|
|
291
|
+
*
|
|
292
|
+
* A converge that changes nothing must be silent, so the log stays a record
|
|
293
|
+
* of things happening.
|
|
294
|
+
*/
|
|
295
|
+
reconcile_peers: {
|
|
296
|
+
inputs: {},
|
|
297
|
+
outputs: {},
|
|
298
|
+
},
|
|
264
299
|
/**
|
|
265
300
|
* Build-bus upstream publish hook. The executor passes the
|
|
266
301
|
* PublishEvent fields as env vars (CELILO_EVENT_PAYLOAD,
|
package/src/manifest/schema.ts
CHANGED
|
@@ -364,6 +364,8 @@ const LIFECYCLE_HOOK_SCHEMAS = {
|
|
|
364
364
|
refresh_registrations: LifecycleHookSchema.optional(),
|
|
365
365
|
reassert_dhcp_dns: LifecycleHookSchema.optional(),
|
|
366
366
|
reconcile_clients: LifecycleHookSchema.optional(),
|
|
367
|
+
list_peers: LifecycleHookSchema.optional(),
|
|
368
|
+
reconcile_peers: LifecycleHookSchema.optional(),
|
|
367
369
|
} satisfies Record<HookName, z.ZodTypeAny>;
|
|
368
370
|
|
|
369
371
|
/**
|
|
@@ -6,6 +6,7 @@ import { basename, join, relative } from 'node:path';
|
|
|
6
6
|
import { create as tarCreate } from 'tar';
|
|
7
7
|
import { parse as parseYaml } from 'yaml';
|
|
8
8
|
import { log } from '../../cli/prompts';
|
|
9
|
+
import { formatViolations, scanModuleDirectory } from '../../policy/module-script-scan';
|
|
9
10
|
import { validateModuleDirectory } from '../import';
|
|
10
11
|
import { computeFileChecksum } from './checksum';
|
|
11
12
|
import { includeNodeModulesPath } from './package-rules';
|
|
@@ -175,6 +176,34 @@ export async function buildModule(options: ModuleBuildOptions): Promise<ModuleBu
|
|
|
175
176
|
return { success: false, error: dirError };
|
|
176
177
|
}
|
|
177
178
|
|
|
179
|
+
// Refuse to package a module whose hook scripts hand-build SSH or take the
|
|
180
|
+
// raw-exec escape hatch without justifying it
|
|
181
|
+
// (openspec/changes/unified-management-no-ssh/proposal.md).
|
|
182
|
+
//
|
|
183
|
+
// The same rules run as a `bun test` gate over this repo's modules. This is
|
|
184
|
+
// the enforcement point that catches what that one cannot: a module built
|
|
185
|
+
// outside CI. `bun run publish` is a documented escape hatch for when the
|
|
186
|
+
// runners are down and it runs no tests, and a module authored outside this
|
|
187
|
+
// repo never passes through the suite at all — in both cases packaging is the
|
|
188
|
+
// last place anything looks at the code before it becomes an artifact the
|
|
189
|
+
// fleet installs.
|
|
190
|
+
//
|
|
191
|
+
// Scans the SOURCE scripts, before staging: it fails in under a second rather
|
|
192
|
+
// than after a `bun pm pack` and a full module build, and the staged copy
|
|
193
|
+
// bundles `@celilo/capabilities` — whose `remote.ts` builds the very
|
|
194
|
+
// `ssh … root@` string these rules exist to keep out of module code — so
|
|
195
|
+
// scanning the bundle would fail every module in the fleet on the
|
|
196
|
+
// implementation of the primitives they were told to use.
|
|
197
|
+
const policyViolations = scanModuleDirectory(sourceDir);
|
|
198
|
+
if (policyViolations.length > 0) {
|
|
199
|
+
return {
|
|
200
|
+
success: false,
|
|
201
|
+
error: `Refusing to package ${basename(sourceDir)}: module script policy violations.\n${formatViolations(
|
|
202
|
+
policyViolations,
|
|
203
|
+
)}\n\nSee apps/celilo/MODULE_PRIMITIVES.md.`,
|
|
204
|
+
};
|
|
205
|
+
}
|
|
206
|
+
|
|
178
207
|
// Copy source to a temp dir for building. Strategy:
|
|
179
208
|
// - If the source has a package.json, use `bun pm pack` to respect the
|
|
180
209
|
// `files` field (or .npmignore), copying only what the build needs.
|
|
@@ -11,6 +11,19 @@
|
|
|
11
11
|
* refer to that document's sections 1-4 — read the row before changing a line
|
|
12
12
|
* here.
|
|
13
13
|
*
|
|
14
|
+
* **Every debt entry names the issue tracking its removal.** The audit is a
|
|
15
|
+
* ruling; the issues are the work. If you are here because the gate went red,
|
|
16
|
+
* the issue in the entry's `why` is where the fix belongs:
|
|
17
|
+
*
|
|
18
|
+
* #937 the hardcoded well-known registry + its duplicate zone table
|
|
19
|
+
* #938 the loader's capName branches, CAPABILITY_MODULE_MAP, isFirewallModule
|
|
20
|
+
* #939 the four provider-domain tables
|
|
21
|
+
* #940 the Caddyfile generator + interface classification in the shared package
|
|
22
|
+
* #941 core shelling iptables-save
|
|
23
|
+
* #943 the three capability-named CLI verbs
|
|
24
|
+
* #944 core shelling wg pubkey
|
|
25
|
+
* #945 three core primitives gated on one capability (generalise, don't move)
|
|
26
|
+
*
|
|
14
27
|
* Two kinds of entry live here and they must not be confused:
|
|
15
28
|
*
|
|
16
29
|
* - **Debt** — the default. It is here because it is wrong and not yet fixed.
|
|
@@ -25,6 +38,9 @@
|
|
|
25
38
|
* each belongs to. Scan A asserts the tagged set EQUALS this map — not that it
|
|
26
39
|
* is empty. These four exist today (audit T1, T2, T3, T6) and Phase 1 migrates
|
|
27
40
|
* nothing; an assertion that core holds none would be red the day it landed.
|
|
41
|
+
*
|
|
42
|
+
* Tracked by #939. Migrating them keeps the ownership CLAIM in core — dropping
|
|
43
|
+
* it is the mistake that made `dns_registration_consumers` necessary (#626).
|
|
28
44
|
*/
|
|
29
45
|
export const CAPABILITY_OWNED_TABLES: Readonly<Record<string, string>> = {
|
|
30
46
|
web_routes: 'public_web',
|
|
@@ -68,25 +84,25 @@ export const CAPABILITY_NAME_BASELINE: readonly CapabilityNameRow[] = [
|
|
|
68
84
|
file: 'apps/celilo/src/capabilities/well-known.ts',
|
|
69
85
|
capability: 'dhcp_server',
|
|
70
86
|
count: 1,
|
|
71
|
-
why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2',
|
|
87
|
+
why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2 (#937)',
|
|
72
88
|
},
|
|
73
89
|
{
|
|
74
90
|
file: 'apps/celilo/src/capabilities/well-known.ts',
|
|
75
91
|
capability: 'dns_internal',
|
|
76
92
|
count: 1,
|
|
77
|
-
why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2',
|
|
93
|
+
why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2 (#937)',
|
|
78
94
|
},
|
|
79
95
|
{
|
|
80
96
|
file: 'apps/celilo/src/capabilities/well-known.ts',
|
|
81
97
|
capability: 'dns_registrar',
|
|
82
98
|
count: 1,
|
|
83
|
-
why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2',
|
|
99
|
+
why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2 (#937)',
|
|
84
100
|
},
|
|
85
101
|
{
|
|
86
102
|
file: 'apps/celilo/src/capabilities/well-known.ts',
|
|
87
103
|
capability: 'public_web',
|
|
88
104
|
count: 1,
|
|
89
|
-
why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2',
|
|
105
|
+
why: 'X1 — hardcoded canonical hostnames, required zones and literal ports; Phase 2 (#937)',
|
|
90
106
|
},
|
|
91
107
|
{
|
|
92
108
|
file: 'apps/celilo/src/cli/commands/notify-config.ts',
|
|
@@ -98,91 +114,91 @@ export const CAPABILITY_NAME_BASELINE: readonly CapabilityNameRow[] = [
|
|
|
98
114
|
file: 'apps/celilo/src/cli/commands/token.ts',
|
|
99
115
|
capability: 'idp',
|
|
100
116
|
count: 1,
|
|
101
|
-
why: 'X7 — a top-level CLI verb backed by the idp capability; Phase 5',
|
|
117
|
+
why: 'X7 — a top-level CLI verb backed by the idp capability; Phase 5 (#943)',
|
|
102
118
|
},
|
|
103
119
|
{
|
|
104
120
|
file: 'apps/celilo/src/cli/completion.ts',
|
|
105
121
|
capability: 'firewall',
|
|
106
122
|
count: 3,
|
|
107
|
-
why: 'X6 — tab completion for the capability-named CLI verbs; falls out with X6',
|
|
123
|
+
why: 'X6 — tab completion for the capability-named CLI verbs; falls out with X6 (#943)',
|
|
108
124
|
},
|
|
109
125
|
{
|
|
110
126
|
file: 'apps/celilo/src/cli/index.ts',
|
|
111
127
|
capability: 'firewall',
|
|
112
128
|
count: 2,
|
|
113
|
-
why: 'X5/X6/X7 — the command tree wiring the capability-named CLI verbs; Phase 5',
|
|
129
|
+
why: 'X5/X6/X7 — the command tree wiring the capability-named CLI verbs; Phase 5 (#943)',
|
|
114
130
|
},
|
|
115
131
|
{
|
|
116
132
|
file: 'apps/celilo/src/hooks/capability-loader.ts',
|
|
117
133
|
capability: 'control_plane_vpn',
|
|
118
134
|
count: 1,
|
|
119
|
-
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
|
|
135
|
+
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
|
|
120
136
|
},
|
|
121
137
|
{
|
|
122
138
|
file: 'apps/celilo/src/hooks/capability-loader.ts',
|
|
123
139
|
capability: 'dhcp_server',
|
|
124
140
|
count: 1,
|
|
125
|
-
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
|
|
141
|
+
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
|
|
126
142
|
},
|
|
127
143
|
{
|
|
128
144
|
file: 'apps/celilo/src/hooks/capability-loader.ts',
|
|
129
145
|
capability: 'dns_internal',
|
|
130
146
|
count: 3,
|
|
131
|
-
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
|
|
147
|
+
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
|
|
132
148
|
},
|
|
133
149
|
{
|
|
134
150
|
file: 'apps/celilo/src/hooks/capability-loader.ts',
|
|
135
151
|
capability: 'dns_registrar',
|
|
136
152
|
count: 4,
|
|
137
|
-
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
|
|
153
|
+
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
|
|
138
154
|
},
|
|
139
155
|
{
|
|
140
156
|
file: 'apps/celilo/src/hooks/capability-loader.ts',
|
|
141
157
|
capability: 'external_web',
|
|
142
158
|
count: 1,
|
|
143
|
-
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
|
|
159
|
+
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
|
|
144
160
|
},
|
|
145
161
|
{
|
|
146
162
|
file: 'apps/celilo/src/hooks/capability-loader.ts',
|
|
147
163
|
capability: 'firewall',
|
|
148
164
|
count: 12,
|
|
149
|
-
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
|
|
165
|
+
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
|
|
150
166
|
},
|
|
151
167
|
{
|
|
152
168
|
file: 'apps/celilo/src/hooks/capability-loader.ts',
|
|
153
169
|
capability: 'idp',
|
|
154
170
|
count: 1,
|
|
155
|
-
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
|
|
171
|
+
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
|
|
156
172
|
},
|
|
157
173
|
{
|
|
158
174
|
file: 'apps/celilo/src/hooks/capability-loader.ts',
|
|
159
175
|
capability: 'notification',
|
|
160
176
|
count: 1,
|
|
161
|
-
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
|
|
177
|
+
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
|
|
162
178
|
},
|
|
163
179
|
{
|
|
164
180
|
file: 'apps/celilo/src/hooks/capability-loader.ts',
|
|
165
181
|
capability: 'private_web',
|
|
166
182
|
count: 1,
|
|
167
|
-
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
|
|
183
|
+
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
|
|
168
184
|
},
|
|
169
185
|
{
|
|
170
186
|
file: 'apps/celilo/src/hooks/capability-loader.ts',
|
|
171
187
|
capability: 'public_web',
|
|
172
188
|
count: 8,
|
|
173
|
-
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
|
|
189
|
+
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
|
|
174
190
|
},
|
|
175
191
|
{
|
|
176
192
|
file: 'apps/celilo/src/hooks/capability-loader.ts',
|
|
177
193
|
capability: 'registry_publish',
|
|
178
194
|
count: 1,
|
|
179
|
-
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
|
|
195
|
+
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
|
|
180
196
|
},
|
|
181
197
|
{
|
|
182
198
|
file: 'apps/celilo/src/hooks/capability-loader.ts',
|
|
183
199
|
capability: 'source_forge',
|
|
184
200
|
count: 1,
|
|
185
|
-
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3',
|
|
201
|
+
why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
|
|
186
202
|
},
|
|
187
203
|
{
|
|
188
204
|
file: 'apps/celilo/src/manifest/validate.ts',
|
|
@@ -212,49 +228,49 @@ export const CAPABILITY_NAME_BASELINE: readonly CapabilityNameRow[] = [
|
|
|
212
228
|
file: 'apps/celilo/src/services/dns-provider-backfill.ts',
|
|
213
229
|
capability: 'dns_internal',
|
|
214
230
|
count: 2,
|
|
215
|
-
why: 'S3 — the generic event-replay half is core; the dns_internal gate is not',
|
|
231
|
+
why: 'S3 — the generic event-replay half is core; the dns_internal gate is not (#945)',
|
|
216
232
|
},
|
|
217
233
|
{
|
|
218
234
|
file: 'apps/celilo/src/services/firewall-reach.ts',
|
|
219
235
|
capability: 'firewall',
|
|
220
236
|
count: 1,
|
|
221
|
-
why: 'S8 — core shells iptables-save at a remote box and parses the output',
|
|
237
|
+
why: 'S8 — core shells iptables-save at a remote box and parses the output (#941)',
|
|
222
238
|
},
|
|
223
239
|
{
|
|
224
240
|
file: 'apps/celilo/src/services/infrastructure-selector.ts',
|
|
225
241
|
capability: 'firewall',
|
|
226
242
|
count: 1,
|
|
227
|
-
why: 'S15 — isFirewallModule hardcodes a placement rule the manifest should declare',
|
|
243
|
+
why: 'S15 — isFirewallModule hardcodes a placement rule the manifest should declare (#938)',
|
|
228
244
|
},
|
|
229
245
|
{
|
|
230
246
|
file: 'apps/celilo/src/services/module-deploy.ts',
|
|
231
247
|
capability: 'firewall',
|
|
232
248
|
count: 2,
|
|
233
|
-
why: "S16 — two more copies of S15's decision; fixing S15 removes all three",
|
|
249
|
+
why: "S16 — two more copies of S15's decision; fixing S15 removes all three (#938)",
|
|
234
250
|
},
|
|
235
251
|
{
|
|
236
252
|
file: 'apps/celilo/src/services/public-web-republish.ts',
|
|
237
253
|
capability: 'public_web',
|
|
238
254
|
count: 1,
|
|
239
|
-
why: "S2 — encodes caddy's redeploy behaviour; generalise to a provider-declared re-assert signal",
|
|
255
|
+
why: "S2 — encodes caddy's redeploy behaviour; generalise to a provider-declared re-assert signal (#945)",
|
|
240
256
|
},
|
|
241
257
|
{
|
|
242
258
|
file: 'apps/celilo/src/services/zone-policy.ts',
|
|
243
259
|
capability: 'public_web',
|
|
244
260
|
count: 1,
|
|
245
|
-
why: 'S13 — ZONE_REQUIREMENTS, a second hand-maintained copy of well-known.ts; Phase 2',
|
|
261
|
+
why: 'S13 — ZONE_REQUIREMENTS, a second hand-maintained copy of well-known.ts; Phase 2 (#937)',
|
|
246
262
|
},
|
|
247
263
|
{
|
|
248
264
|
file: 'apps/celilo/src/templates/generator.ts',
|
|
249
265
|
capability: 'dns_internal',
|
|
250
266
|
count: 1,
|
|
251
|
-
why: 'X4 — declaration-driven and the model for the rest; only the error string names the capability',
|
|
267
|
+
why: 'X4 — declaration-driven and the model for the rest; only the error string names the capability (#945)',
|
|
252
268
|
},
|
|
253
269
|
{
|
|
254
270
|
file: 'apps/celilo/src/variables/context.ts',
|
|
255
271
|
capability: 'dns_internal',
|
|
256
272
|
count: 1,
|
|
257
|
-
why: 'X3 — reaches capabilitiesMap.dns_internal by name; should read a declared field (X4 is the model)',
|
|
273
|
+
why: 'X3 — reaches capabilitiesMap.dns_internal by name; should read a declared field (X4 is the model) (#945)',
|
|
258
274
|
},
|
|
259
275
|
{
|
|
260
276
|
file: 'packages/capabilities/src/capability-contract.ts',
|
|
@@ -344,13 +360,13 @@ export const CAPABILITY_NAME_BASELINE: readonly CapabilityNameRow[] = [
|
|
|
344
360
|
file: 'packages/capabilities/src/public-web.ts',
|
|
345
361
|
capability: 'public_web',
|
|
346
362
|
count: 1,
|
|
347
|
-
why: "X8 — generateCaddyfile renders caddy's complete config from the shared package; Phase 5",
|
|
363
|
+
why: "X8 — generateCaddyfile renders caddy's complete config from the shared package; Phase 5 (#940)",
|
|
348
364
|
},
|
|
349
365
|
{
|
|
350
366
|
file: 'packages/capabilities/src/utils.ts',
|
|
351
367
|
capability: 'public_web',
|
|
352
368
|
count: 3,
|
|
353
|
-
why: 'X8 — shared helpers that name the capability they serve; moves with X8 in Phase 5',
|
|
369
|
+
why: 'X8 — shared helpers that name the capability they serve; moves with X8 in Phase 5 (#940)',
|
|
354
370
|
},
|
|
355
371
|
];
|
|
356
372
|
|
|
@@ -368,17 +384,17 @@ export const SERVICE_FILENAME_BASELINE: readonly ServiceFilenameRow[] = [
|
|
|
368
384
|
{
|
|
369
385
|
file: 'apps/celilo/src/services/dns-internal-records.ts',
|
|
370
386
|
capability: 'dns_internal',
|
|
371
|
-
why: 'S5 — the store for T6; moves with the table in Phase 4',
|
|
387
|
+
why: 'S5 — the store for T6; moves with the table in Phase 4 (#939)',
|
|
372
388
|
},
|
|
373
389
|
{
|
|
374
390
|
file: 'apps/celilo/src/services/firewall-reach.ts',
|
|
375
391
|
capability: 'firewall',
|
|
376
|
-
why: 'S8 — core reaching into one provider implementation',
|
|
392
|
+
why: 'S8 — core reaching into one provider implementation (#941)',
|
|
377
393
|
},
|
|
378
394
|
{
|
|
379
395
|
file: 'apps/celilo/src/services/public-web-republish.ts',
|
|
380
396
|
capability: 'public_web',
|
|
381
|
-
why: "S2 — named for one provider's redeploy behaviour",
|
|
397
|
+
why: "S2 — named for one provider's redeploy behaviour (#945)",
|
|
382
398
|
},
|
|
383
399
|
];
|
|
384
400
|
|
|
@@ -387,18 +403,18 @@ export const PROVIDER_LITERAL_BASELINE: readonly ProviderLiteralRow[] = [
|
|
|
387
403
|
file: 'apps/celilo/src/secrets/generators.ts',
|
|
388
404
|
literal: 'wg pubkey',
|
|
389
405
|
count: 1,
|
|
390
|
-
why: 'X11 — core shells `wg pubkey` to derive a key, and tells the operator to brew install wireguard-tools',
|
|
406
|
+
why: 'X11 — core shells `wg pubkey` to derive a key, and tells the operator to brew install wireguard-tools (#944)',
|
|
391
407
|
},
|
|
392
408
|
{
|
|
393
409
|
file: 'apps/celilo/src/services/firewall-reach.ts',
|
|
394
410
|
literal: 'iptables-save',
|
|
395
411
|
count: 1,
|
|
396
|
-
why: "S8 — core parses a firewall's live ruleset; implementation-specific, not merely capability-specific",
|
|
412
|
+
why: "S8 — core parses a firewall's live ruleset; implementation-specific, not merely capability-specific (#941)",
|
|
397
413
|
},
|
|
398
414
|
{
|
|
399
415
|
file: 'packages/capabilities/src/public-web.ts',
|
|
400
416
|
literal: '/srv/www',
|
|
401
417
|
count: 4,
|
|
402
|
-
why: "X8 — the Caddyfile generator knows caddy's on-disk asset layout",
|
|
418
|
+
why: "X8 — the Caddyfile generator knows caddy's on-disk asset layout (#940)",
|
|
403
419
|
},
|
|
404
420
|
];
|
|
@@ -0,0 +1,164 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The scan rules themselves, and the proof that packaging refuses a module that
|
|
3
|
+
* breaks them.
|
|
4
|
+
*
|
|
5
|
+
* `no-hand-built-ssh.test.ts` asserts the CURRENT tree is clean, which is a
|
|
6
|
+
* different claim: it passes both when the rules work and when they match
|
|
7
|
+
* nothing. These tests are the ones that fail if a rule stops catching things.
|
|
8
|
+
*/
|
|
9
|
+
|
|
10
|
+
import { describe, expect, it } from 'bun:test';
|
|
11
|
+
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
|
12
|
+
import { tmpdir } from 'node:os';
|
|
13
|
+
import { join } from 'node:path';
|
|
14
|
+
import { buildModule } from '../module/packaging/build';
|
|
15
|
+
import { scanModuleDirectory, scanModuleScriptSource } from './module-script-scan';
|
|
16
|
+
|
|
17
|
+
const rules = (src: string) => scanModuleScriptSource('f.ts', src).map((v) => v.rule);
|
|
18
|
+
|
|
19
|
+
describe('module script scan — SSH rules', () => {
|
|
20
|
+
it('catches a hand-built ssh string', () => {
|
|
21
|
+
expect(rules("run(`ssh root@${ip} 'systemctl restart x'`);")).toContain(
|
|
22
|
+
'raw ssh invocation (ssh … root@)',
|
|
23
|
+
);
|
|
24
|
+
});
|
|
25
|
+
|
|
26
|
+
it('catches StrictHostKeyChecking however it is invoked', () => {
|
|
27
|
+
expect(rules("const c = 'ssh -o StrictHostKeyChecking=no host';")).toContain(
|
|
28
|
+
'raw ssh invocation (StrictHostKeyChecking)',
|
|
29
|
+
);
|
|
30
|
+
});
|
|
31
|
+
|
|
32
|
+
it('catches an ssh2 import', () => {
|
|
33
|
+
expect(rules("import { Client } from 'ssh2';")).toContain("'ssh2' import");
|
|
34
|
+
});
|
|
35
|
+
|
|
36
|
+
it('does not fire on ordinary module code', () => {
|
|
37
|
+
expect(rules('const x = probe(system, { kind: "systemd", unit: "caddy" }, run);')).toEqual([]);
|
|
38
|
+
});
|
|
39
|
+
});
|
|
40
|
+
|
|
41
|
+
describe('module script scan — raw-exec escape hatch', () => {
|
|
42
|
+
it('flags a runAppCommand call with no justification', () => {
|
|
43
|
+
expect(rules('const r = runAppCommand(system, "rm -f /tmp/x", run);')).toContain(
|
|
44
|
+
'unjustified raw-exec escape hatch',
|
|
45
|
+
);
|
|
46
|
+
});
|
|
47
|
+
|
|
48
|
+
it('flags runAppCommandWithSecret too', () => {
|
|
49
|
+
expect(rules('const r = runAppCommandWithSecret(system, cli, secret, run);')).toContain(
|
|
50
|
+
'unjustified raw-exec escape hatch',
|
|
51
|
+
);
|
|
52
|
+
});
|
|
53
|
+
|
|
54
|
+
it('accepts a call justified immediately above', () => {
|
|
55
|
+
const src = [
|
|
56
|
+
'// escape-hatch: forgejo admin user create is CLI-only, no HTTP API path.',
|
|
57
|
+
'const r = runAppCommand(system, cmd, run);',
|
|
58
|
+
].join('\n');
|
|
59
|
+
expect(rules(src)).toEqual([]);
|
|
60
|
+
});
|
|
61
|
+
|
|
62
|
+
it('accepts a call wrapped in waitFor, justified above the enclosing statement', () => {
|
|
63
|
+
// The real shape in modules/caddy-internal — the justification sits above
|
|
64
|
+
// `const ready = await waitFor(`, a few lines up from the call itself.
|
|
65
|
+
const src = [
|
|
66
|
+
'// escape-hatch: the command output IS the payload; no API to ask.',
|
|
67
|
+
'const ready = await waitFor(',
|
|
68
|
+
' () =>',
|
|
69
|
+
' runAppCommand(target, CMD, run, {',
|
|
70
|
+
' timeoutMs: 10_000,',
|
|
71
|
+
' }).ok,',
|
|
72
|
+
');',
|
|
73
|
+
].join('\n');
|
|
74
|
+
expect(rules(src)).toEqual([]);
|
|
75
|
+
});
|
|
76
|
+
|
|
77
|
+
it('does not let a distant hatch launder a later call', () => {
|
|
78
|
+
const src = [
|
|
79
|
+
'// escape-hatch: justifies the call directly below it, and nothing else.',
|
|
80
|
+
'const a = runAppCommand(system, one, run);',
|
|
81
|
+
...Array(12).fill('doSomethingElse();'),
|
|
82
|
+
'const b = runAppCommand(system, two, run);',
|
|
83
|
+
].join('\n');
|
|
84
|
+
// Exactly one violation: the second call, which has no hatch in reach.
|
|
85
|
+
expect(rules(src)).toEqual(['unjustified raw-exec escape hatch']);
|
|
86
|
+
});
|
|
87
|
+
|
|
88
|
+
it('does not flag the import of runAppCommand', () => {
|
|
89
|
+
expect(rules("import { runAppCommand, probe } from '@celilo/capabilities';")).toEqual([]);
|
|
90
|
+
});
|
|
91
|
+
});
|
|
92
|
+
|
|
93
|
+
describe('module script scan — what it deliberately does not scan', () => {
|
|
94
|
+
let dir: string;
|
|
95
|
+
|
|
96
|
+
function write(rel: string, content: string): void {
|
|
97
|
+
const full = join(dir, rel);
|
|
98
|
+
mkdirSync(join(full, '..'), { recursive: true });
|
|
99
|
+
writeFileSync(full, content);
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
it('ignores bundled node_modules and test files', () => {
|
|
103
|
+
dir = mkdtempSync(join(tmpdir(), 'celilo-scan-test-'));
|
|
104
|
+
try {
|
|
105
|
+
write('manifest.yml', 'id: demo\n');
|
|
106
|
+
// @celilo/capabilities legitimately BUILDS the ssh string these rules ban.
|
|
107
|
+
// Scanning the bundled closure would fail every module in the fleet.
|
|
108
|
+
write(
|
|
109
|
+
'scripts/node_modules/@celilo/capabilities/src/remote.ts',
|
|
110
|
+
'const c = `ssh -o StrictHostKeyChecking=no root@${ip} ${cmd}`;',
|
|
111
|
+
);
|
|
112
|
+
write('scripts/setup.test.ts', "run('ssh root@host uptime');");
|
|
113
|
+
write('scripts/setup.ts', 'export const fine = 1;\n');
|
|
114
|
+
expect(scanModuleDirectory(dir)).toEqual([]);
|
|
115
|
+
} finally {
|
|
116
|
+
rmSync(dir, { recursive: true, force: true });
|
|
117
|
+
}
|
|
118
|
+
});
|
|
119
|
+
});
|
|
120
|
+
|
|
121
|
+
describe('packaging refuses a module that breaks the policy', () => {
|
|
122
|
+
let dir: string;
|
|
123
|
+
|
|
124
|
+
function write(rel: string, content: string): void {
|
|
125
|
+
const full = join(dir, rel);
|
|
126
|
+
mkdirSync(join(full, '..'), { recursive: true });
|
|
127
|
+
writeFileSync(full, content);
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
it('fails the build, naming the file, line and rule', async () => {
|
|
131
|
+
dir = mkdtempSync(join(tmpdir(), 'celilo-package-policy-'));
|
|
132
|
+
try {
|
|
133
|
+
write('manifest.yml', 'id: demo\nversion: 0.1.0\n');
|
|
134
|
+
write(
|
|
135
|
+
'scripts/setup.ts',
|
|
136
|
+
['export function bad(ip: string) {', ' return `ssh root@${ip} uptime`;', '}'].join('\n'),
|
|
137
|
+
);
|
|
138
|
+
|
|
139
|
+
const result = await buildModule({ sourceDir: dir });
|
|
140
|
+
|
|
141
|
+
expect(result.success).toBe(false);
|
|
142
|
+
expect(result.error).toContain('scripts/setup.ts:2');
|
|
143
|
+
expect(result.error).toContain('raw ssh invocation');
|
|
144
|
+
expect(result.error).toContain('MODULE_PRIMITIVES.md');
|
|
145
|
+
} finally {
|
|
146
|
+
rmSync(dir, { recursive: true, force: true });
|
|
147
|
+
}
|
|
148
|
+
});
|
|
149
|
+
|
|
150
|
+
it('fails the build for an unjustified escape hatch', async () => {
|
|
151
|
+
dir = mkdtempSync(join(tmpdir(), 'celilo-package-policy-'));
|
|
152
|
+
try {
|
|
153
|
+
write('manifest.yml', 'id: demo\nversion: 0.1.0\n');
|
|
154
|
+
write('scripts/setup.ts', 'const r = runAppCommand(system, "rm -rf /srv", run);\n');
|
|
155
|
+
|
|
156
|
+
const result = await buildModule({ sourceDir: dir });
|
|
157
|
+
|
|
158
|
+
expect(result.success).toBe(false);
|
|
159
|
+
expect(result.error).toContain('unjustified raw-exec escape hatch');
|
|
160
|
+
} finally {
|
|
161
|
+
rmSync(dir, { recursive: true, force: true });
|
|
162
|
+
}
|
|
163
|
+
});
|
|
164
|
+
});
|
|
@@ -0,0 +1,143 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The recurrence gate for openspec/changes/unified-management-no-ssh/proposal.md:
|
|
3
|
+
* **modules never hand-build SSH, and the one sanctioned raw-exec path is
|
|
4
|
+
* always justified in writing.**
|
|
5
|
+
*
|
|
6
|
+
* ONE definition of the rules, used by both enforcement points:
|
|
7
|
+
*
|
|
8
|
+
* - `apps/celilo/src/policy/no-hand-built-ssh.test.ts` — every in-repo module,
|
|
9
|
+
* on every `bun test`.
|
|
10
|
+
* - `apps/celilo/src/module/packaging/build.ts` — every `.netapp` at the
|
|
11
|
+
* moment it is packaged, including modules that never pass through this
|
|
12
|
+
* repo's CI (`bun run publish` is a documented escape hatch and runs no
|
|
13
|
+
* tests).
|
|
14
|
+
*
|
|
15
|
+
* Deliberately not two copies. The types this repo keeps re-learning that
|
|
16
|
+
* lesson on — `HookName` (celilo#821), `HookContext` — were duplicated
|
|
17
|
+
* declarations that drifted silently because nothing fails when two copies
|
|
18
|
+
* disagree. A scan rule is worse: the duplicate that drifts is the one that
|
|
19
|
+
* stops catching things, and a gate that stops catching things looks exactly
|
|
20
|
+
* like a gate with nothing to catch.
|
|
21
|
+
*/
|
|
22
|
+
|
|
23
|
+
import { existsSync, readFileSync, readdirSync, statSync } from 'node:fs';
|
|
24
|
+
import { join, relative } from 'node:path';
|
|
25
|
+
|
|
26
|
+
export interface ScanViolation {
|
|
27
|
+
/** Path as the operator should see it — relative to the scanned root. */
|
|
28
|
+
file: string;
|
|
29
|
+
/** 1-based line number of the offending line. */
|
|
30
|
+
line: number;
|
|
31
|
+
/** Short rule name, e.g. `raw ssh invocation`. */
|
|
32
|
+
rule: string;
|
|
33
|
+
/** What to do instead. */
|
|
34
|
+
hint: string;
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
/**
|
|
38
|
+
* How far above a `runAppCommand*` call the justification may sit.
|
|
39
|
+
*
|
|
40
|
+
* Calibrated against the real call sites rather than guessed: the convention is
|
|
41
|
+
* a comment block directly above the call, but a call wrapped in `waitFor(() =>
|
|
42
|
+
* …)` puts the justification above the ENCLOSING statement, a few lines up. A
|
|
43
|
+
* window covers both without needing to parse TypeScript. Eight lines is the
|
|
44
|
+
* widest real gap plus headroom; wide enough to miss nothing legitimate, narrow
|
|
45
|
+
* enough that an unrelated hatch elsewhere in the function cannot launder a
|
|
46
|
+
* fresh call.
|
|
47
|
+
*/
|
|
48
|
+
const ESCAPE_HATCH_LOOKBACK_LINES = 8;
|
|
49
|
+
|
|
50
|
+
const PATTERN_RULES: Array<{ rule: string; re: RegExp; hint: string }> = [
|
|
51
|
+
{
|
|
52
|
+
rule: 'raw ssh invocation (StrictHostKeyChecking)',
|
|
53
|
+
re: /StrictHostKeyChecking/,
|
|
54
|
+
hint: 'Use a remote-ops primitive (remoteExec/probe/serviceCtl/…). See MODULE_PRIMITIVES.md.',
|
|
55
|
+
},
|
|
56
|
+
{
|
|
57
|
+
rule: 'raw ssh invocation (ssh … root@)',
|
|
58
|
+
re: /\bssh\s+(?:-\S+\s+|\S*root@)/,
|
|
59
|
+
hint: 'Use a remote-ops primitive, not a hand-built ssh string. See MODULE_PRIMITIVES.md.',
|
|
60
|
+
},
|
|
61
|
+
{
|
|
62
|
+
rule: "'ssh2' import",
|
|
63
|
+
re: /(?:from|require\()\s*['"]ssh2['"]/,
|
|
64
|
+
hint: 'Modules never open their own SSH connection — use the primitives. See MODULE_PRIMITIVES.md.',
|
|
65
|
+
},
|
|
66
|
+
];
|
|
67
|
+
|
|
68
|
+
/** A `runAppCommand(` / `runAppCommandWithSecret(` CALL — not the import. */
|
|
69
|
+
const RAW_EXEC_CALL = /\brunAppCommand(?:WithSecret)?\s*\(/;
|
|
70
|
+
|
|
71
|
+
const ESCAPE_HATCH_MARKER = /escape-hatch:/;
|
|
72
|
+
|
|
73
|
+
/**
|
|
74
|
+
* Scan one file's source. Pure — takes the text, returns the violations, so it
|
|
75
|
+
* is testable without a filesystem and reusable over a staged package.
|
|
76
|
+
*/
|
|
77
|
+
export function scanModuleScriptSource(file: string, source: string): ScanViolation[] {
|
|
78
|
+
const violations: ScanViolation[] = [];
|
|
79
|
+
const lines = source.split('\n');
|
|
80
|
+
|
|
81
|
+
lines.forEach((text, i) => {
|
|
82
|
+
for (const { rule, re, hint } of PATTERN_RULES) {
|
|
83
|
+
if (re.test(text)) violations.push({ file, line: i + 1, rule, hint });
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
if (!RAW_EXEC_CALL.test(text)) return;
|
|
87
|
+
const from = Math.max(0, i - ESCAPE_HATCH_LOOKBACK_LINES);
|
|
88
|
+
const justified = lines.slice(from, i).some((l) => ESCAPE_HATCH_MARKER.test(l));
|
|
89
|
+
if (!justified) {
|
|
90
|
+
violations.push({
|
|
91
|
+
file,
|
|
92
|
+
line: i + 1,
|
|
93
|
+
rule: 'unjustified raw-exec escape hatch',
|
|
94
|
+
hint:
|
|
95
|
+
'runAppCommand* is the ONLY sanctioned raw-exec path and every call site must say why ' +
|
|
96
|
+
'no capability, HTTP or converge path exists. Add an `// escape-hatch: …` comment ' +
|
|
97
|
+
'immediately above the call. See MODULE_PRIMITIVES.md.',
|
|
98
|
+
});
|
|
99
|
+
}
|
|
100
|
+
});
|
|
101
|
+
|
|
102
|
+
return violations;
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
/**
|
|
106
|
+
* Every production `.ts` under a module's `scripts/` — excluding tests and
|
|
107
|
+
* `node_modules`.
|
|
108
|
+
*
|
|
109
|
+
* The exclusion is load-bearing, not tidiness: the shipped closure bundles
|
|
110
|
+
* `@celilo/capabilities`, whose `remote.ts` builds the `ssh … root@` string
|
|
111
|
+
* that every one of these rules exists to keep OUT of module code. Scanning it
|
|
112
|
+
* would fail every module in the fleet on the implementation of the primitives
|
|
113
|
+
* they were told to use.
|
|
114
|
+
*/
|
|
115
|
+
export function moduleScriptFiles(scriptsDir: string): string[] {
|
|
116
|
+
if (!existsSync(scriptsDir) || !statSync(scriptsDir).isDirectory()) return [];
|
|
117
|
+
const out: string[] = [];
|
|
118
|
+
const walk = (dir: string) => {
|
|
119
|
+
for (const entry of readdirSync(dir)) {
|
|
120
|
+
if (entry === 'node_modules') continue;
|
|
121
|
+
const p = join(dir, entry);
|
|
122
|
+
if (statSync(p).isDirectory()) walk(p);
|
|
123
|
+
else if (p.endsWith('.ts') && !p.endsWith('.test.ts')) out.push(p);
|
|
124
|
+
}
|
|
125
|
+
};
|
|
126
|
+
walk(scriptsDir);
|
|
127
|
+
return out;
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
/**
|
|
131
|
+
* Scan a module directory (the one holding `manifest.yml`). Returns every
|
|
132
|
+
* violation in its `scripts/`, with paths relative to `moduleDir`.
|
|
133
|
+
*/
|
|
134
|
+
export function scanModuleDirectory(moduleDir: string): ScanViolation[] {
|
|
135
|
+
return moduleScriptFiles(join(moduleDir, 'scripts')).flatMap((f) =>
|
|
136
|
+
scanModuleScriptSource(relative(moduleDir, f), readFileSync(f, 'utf-8')),
|
|
137
|
+
);
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
/** Render violations for a test failure message or a refused publish. */
|
|
141
|
+
export function formatViolations(violations: ScanViolation[]): string {
|
|
142
|
+
return violations.map((v) => ` ${v.file}:${v.line}\n → ${v.rule}. ${v.hint}`).join('\n');
|
|
143
|
+
}
|
|
@@ -1,24 +1,20 @@
|
|
|
1
1
|
/**
|
|
2
|
-
* Recurrence gate for openspec/changes/unified-management-no-ssh/proposal.md: **modules never hand-build SSH
|
|
2
|
+
* Recurrence gate for openspec/changes/unified-management-no-ssh/proposal.md: **modules never hand-build SSH,
|
|
3
|
+
* and every raw-exec escape hatch is justified in writing.**
|
|
3
4
|
*
|
|
4
|
-
*
|
|
5
|
-
*
|
|
6
|
-
*
|
|
7
|
-
*
|
|
8
|
-
* script and fails if a banned pattern reappears, so the SSH-elimination work
|
|
9
|
-
* can't silently erode.
|
|
5
|
+
* The rules themselves live in `./module-script-scan`, because this is not the
|
|
6
|
+
* only place they run — `.netapp` packaging applies the same scan at publish
|
|
7
|
+
* time, which is the enforcement point that also covers a module built outside
|
|
8
|
+
* this repo's CI. One definition, two callers.
|
|
10
9
|
*
|
|
11
|
-
*
|
|
12
|
-
*
|
|
13
|
-
* hand-built SSH however it's invoked (child_process, execSync, or the Runner),
|
|
14
|
-
* and `ssh2` catches the library route. We do NOT ban `child_process` outright:
|
|
15
|
-
* many modules' on_install legitimately shell LOCAL `celilo`/system commands
|
|
16
|
-
* (`celilo system apply-config`), which isn't remote SSH.
|
|
10
|
+
* This file is the in-repo half: every production module script, on every
|
|
11
|
+
* `bun test`.
|
|
17
12
|
*/
|
|
18
13
|
|
|
19
14
|
import { describe, expect, test } from 'bun:test';
|
|
20
|
-
import { existsSync,
|
|
15
|
+
import { existsSync, readdirSync, statSync } from 'node:fs';
|
|
21
16
|
import { join, resolve } from 'node:path';
|
|
17
|
+
import { formatViolations, moduleScriptFiles, scanModuleDirectory } from './module-script-scan';
|
|
22
18
|
|
|
23
19
|
/** Walk up from this test to the repo root (the dir holding both modules/ and apps/). */
|
|
24
20
|
function repoRoot(): string {
|
|
@@ -30,61 +26,25 @@ function repoRoot(): string {
|
|
|
30
26
|
throw new Error('could not locate repo root (no ancestor with modules/ + apps/)');
|
|
31
27
|
}
|
|
32
28
|
|
|
33
|
-
|
|
34
|
-
function moduleScripts(): string[] {
|
|
29
|
+
function moduleDirs(): string[] {
|
|
35
30
|
const modulesRoot = join(repoRoot(), 'modules');
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
if (entry === 'node_modules') continue;
|
|
40
|
-
const p = join(dir, entry);
|
|
41
|
-
if (statSync(p).isDirectory()) walk(p);
|
|
42
|
-
else if (p.endsWith('.ts') && !p.endsWith('.test.ts')) out.push(p);
|
|
43
|
-
}
|
|
44
|
-
};
|
|
45
|
-
for (const mod of readdirSync(modulesRoot)) {
|
|
46
|
-
const scripts = join(modulesRoot, mod, 'scripts');
|
|
47
|
-
if (existsSync(scripts) && statSync(scripts).isDirectory()) walk(scripts);
|
|
48
|
-
}
|
|
49
|
-
return out;
|
|
31
|
+
return readdirSync(modulesRoot)
|
|
32
|
+
.map((m) => join(modulesRoot, m))
|
|
33
|
+
.filter((d) => statSync(d).isDirectory() && existsSync(join(d, 'scripts')));
|
|
50
34
|
}
|
|
51
35
|
|
|
52
|
-
const BANNED = [
|
|
53
|
-
{
|
|
54
|
-
name: 'raw ssh invocation (StrictHostKeyChecking)',
|
|
55
|
-
re: /StrictHostKeyChecking/,
|
|
56
|
-
hint: 'Use a remote-ops primitive (remoteExec/probe/serviceCtl/…). See MODULE_PRIMITIVES.md.',
|
|
57
|
-
},
|
|
58
|
-
{
|
|
59
|
-
name: 'raw ssh invocation (ssh … root@)',
|
|
60
|
-
re: /\bssh\s+(?:-\S+\s+|\S*root@)/,
|
|
61
|
-
hint: 'Use a remote-ops primitive, not a hand-built ssh string. See MODULE_PRIMITIVES.md.',
|
|
62
|
-
},
|
|
63
|
-
{
|
|
64
|
-
name: "'ssh2' import",
|
|
65
|
-
re: /(?:from|require\()\s*['"]ssh2['"]/,
|
|
66
|
-
hint: 'Modules never open their own SSH connection — use the primitives. See MODULE_PRIMITIVES.md.',
|
|
67
|
-
},
|
|
68
|
-
];
|
|
69
|
-
|
|
70
36
|
describe('recurrence gate: modules never hand-build SSH', () => {
|
|
71
|
-
const
|
|
37
|
+
const dirs = moduleDirs();
|
|
72
38
|
|
|
73
39
|
test('scans a non-trivial set of module scripts (sanity — the scan actually ran)', () => {
|
|
74
|
-
|
|
40
|
+
const scanned = dirs.flatMap((d) => moduleScriptFiles(join(d, 'scripts')));
|
|
41
|
+
expect(scanned.length).toBeGreaterThan(10);
|
|
75
42
|
});
|
|
76
43
|
|
|
77
|
-
test('no production module script hand-builds SSH
|
|
78
|
-
const violations
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
if (b.re.test(src)) violations.push(`${file}\n → ${b.name}. ${b.hint}`);
|
|
83
|
-
}
|
|
84
|
-
}
|
|
85
|
-
expect(
|
|
86
|
-
violations,
|
|
87
|
-
`Hand-built SSH found in module scripts:\n ${violations.join('\n ')}`,
|
|
88
|
-
).toEqual([]);
|
|
44
|
+
test('no production module script hand-builds SSH, and every runAppCommand* is justified', () => {
|
|
45
|
+
const violations = dirs.flatMap((d) => scanModuleDirectory(d));
|
|
46
|
+
expect(violations, `Module script policy violations:\n${formatViolations(violations)}`).toEqual(
|
|
47
|
+
[],
|
|
48
|
+
);
|
|
89
49
|
});
|
|
90
50
|
});
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* `copyAnsibleRoleFilesDirs` had no test, and celilo#925 is what that cost.
|
|
3
|
+
*
|
|
4
|
+
* A module's Ansible role `files/` directory holds static assets — most often a
|
|
5
|
+
* compiled binary — that cannot go through the utf-8 template pipeline. They are
|
|
6
|
+
* copied verbatim on every generate. Except they were not: the copy omitted
|
|
7
|
+
* `force`, which Node defaults to true and bun does not, so the FIRST generate
|
|
8
|
+
* populated `generated/` and no later one ever replaced it.
|
|
9
|
+
*
|
|
10
|
+
* Nothing surfaced it. `cp` reports no error, so the caller's try/catch caught
|
|
11
|
+
* nothing; Ansible then installed the first binary forever and reported `ok`,
|
|
12
|
+
* unchanged, while the module's version field advanced past it. On the live
|
|
13
|
+
* fleet that read as a successful deploy of code that never shipped.
|
|
14
|
+
*
|
|
15
|
+
* The overwrite case is therefore the point of this file. A test that only
|
|
16
|
+
* copied into an empty directory would have passed throughout.
|
|
17
|
+
*/
|
|
18
|
+
import { describe, expect, test } from 'bun:test';
|
|
19
|
+
import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from 'node:fs';
|
|
20
|
+
import { tmpdir } from 'node:os';
|
|
21
|
+
import { join } from 'node:path';
|
|
22
|
+
import { copyAnsibleRoleFilesDirs } from './generator';
|
|
23
|
+
|
|
24
|
+
function moduleWithRoleFile(binary: string): string {
|
|
25
|
+
const root = mkdtempSync(join(tmpdir(), 'celilo-rolefiles-'));
|
|
26
|
+
const filesDir = join(root, 'ansible', 'roles', 'demo', 'files');
|
|
27
|
+
mkdirSync(filesDir, { recursive: true });
|
|
28
|
+
writeFileSync(join(filesDir, 'demo-linux-x86_64'), binary);
|
|
29
|
+
return root;
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
const generatedBinary = (out: string): string =>
|
|
33
|
+
readFileSync(join(out, 'ansible', 'roles', 'demo', 'files', 'demo-linux-x86_64'), 'utf-8');
|
|
34
|
+
|
|
35
|
+
describe('copyAnsibleRoleFilesDirs', () => {
|
|
36
|
+
test('populates an empty generated tree', async () => {
|
|
37
|
+
const modulePath = moduleWithRoleFile('v1');
|
|
38
|
+
const outputPath = mkdtempSync(join(tmpdir(), 'celilo-out-'));
|
|
39
|
+
|
|
40
|
+
await copyAnsibleRoleFilesDirs(modulePath, outputPath);
|
|
41
|
+
|
|
42
|
+
expect(generatedBinary(outputPath)).toBe('v1');
|
|
43
|
+
});
|
|
44
|
+
|
|
45
|
+
/**
|
|
46
|
+
* celilo#925 in one assertion. This is the case that regressed, and the only
|
|
47
|
+
* one that can catch it: the destination already exists, and a rebuilt
|
|
48
|
+
* artifact has to replace it.
|
|
49
|
+
*/
|
|
50
|
+
test('OVERWRITES an artifact a previous generate already placed', async () => {
|
|
51
|
+
const outputPath = mkdtempSync(join(tmpdir(), 'celilo-out-'));
|
|
52
|
+
|
|
53
|
+
await copyAnsibleRoleFilesDirs(moduleWithRoleFile('v1'), outputPath);
|
|
54
|
+
expect(generatedBinary(outputPath)).toBe('v1');
|
|
55
|
+
|
|
56
|
+
// The module is rebuilt at a new version; generate runs again.
|
|
57
|
+
await copyAnsibleRoleFilesDirs(moduleWithRoleFile('v2'), outputPath);
|
|
58
|
+
|
|
59
|
+
expect(generatedBinary(outputPath)).toBe('v2');
|
|
60
|
+
});
|
|
61
|
+
|
|
62
|
+
test('a module with no role files/ directory is not an error', async () => {
|
|
63
|
+
const root = mkdtempSync(join(tmpdir(), 'celilo-norole-'));
|
|
64
|
+
mkdirSync(join(root, 'ansible', 'roles', 'demo', 'tasks'), { recursive: true });
|
|
65
|
+
const outputPath = mkdtempSync(join(tmpdir(), 'celilo-out-'));
|
|
66
|
+
|
|
67
|
+
await copyAnsibleRoleFilesDirs(root, outputPath);
|
|
68
|
+
});
|
|
69
|
+
});
|
|
@@ -500,7 +500,29 @@ export async function copyAnsibleRoleFilesDirs(
|
|
|
500
500
|
if (!existsSync(srcFilesDir)) continue;
|
|
501
501
|
const destFilesDir = join(outputPath, 'ansible', 'roles', role.name, 'files');
|
|
502
502
|
await mkdir(dirname(destFilesDir), { recursive: true });
|
|
503
|
-
|
|
503
|
+
// `force: true` is LOAD-BEARING on bun, and its absence was celilo#925.
|
|
504
|
+
//
|
|
505
|
+
// Node defaults `force` to true, so this looked correct and is correct
|
|
506
|
+
// under Node. Bun 1.3.3 does not, on this path specifically — measured,
|
|
507
|
+
// copying "NEW" over an existing "OLD":
|
|
508
|
+
//
|
|
509
|
+
// recursive only -> NEW
|
|
510
|
+
// recursive + force -> NEW
|
|
511
|
+
// recursive + preserveTimestamps -> OLD <- what this was
|
|
512
|
+
// recursive + force + preserveTimestamps -> NEW
|
|
513
|
+
//
|
|
514
|
+
// celilo runs on bun. So a module's built binary landed in `generated/`
|
|
515
|
+
// exactly once, at first generate, and no later version ever replaced it —
|
|
516
|
+
// silently, because `cp` reports no error, so the caller's try/catch has
|
|
517
|
+
// nothing to catch. Ansible then copies that first binary forever and
|
|
518
|
+
// reports `ok`, unchanged, while the module's version field advances.
|
|
519
|
+
//
|
|
520
|
+
// The sibling call in `storage-set-path.ts:153` already passes `force`.
|
|
521
|
+
await cp(srcFilesDir, destFilesDir, {
|
|
522
|
+
recursive: true,
|
|
523
|
+
force: true,
|
|
524
|
+
preserveTimestamps: true,
|
|
525
|
+
});
|
|
504
526
|
}
|
|
505
527
|
}
|
|
506
528
|
|