@celilo/cli 5.0.0 → 5.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -201,6 +201,7 @@ runner seam (`execRunner` real / `createMockRunner` for tests) lives in
201
201
 
202
202
  - **Hook executor / ABI** — `apps/celilo/src/hooks/executor.ts` (`invokeHook`, `executeHookScript`, `checkRequiredCapabilities`), types in `apps/celilo/src/hooks/types.ts` (`HookContext`, `HookDefinition`, `HookName`). Named-hook runner: `apps/celilo/src/hooks/run-named-hook.ts`. Manifest hook config: `apps/celilo/src/hooks/load-hook-config.ts`.
203
203
  - **Hook process boundary (a hook is a program celilo RUNS)** — `apps/celilo/src/hooks/hook-protocol.ts` (the NDJSON frame union, `HOOK_PROTOCOL_VERSION`, `serializeError`/`deserializeError`), `apps/celilo/src/hooks/broker.ts` (`startBroker`, `capabilityShape`), `apps/celilo/src/hooks/hook-runner-entry.ts` + `apps/celilo/src/hooks/hook-runner.ts` (the spawned shim — the ONLY thing that `import()`s module code; the entry exists to install the advisory lint before the shim's import graph can ESM-load `node:fs`, see `unjailed-lint.ts`). `executeHookScript` spawns `bun hook-runner-entry.ts` over a Unix socket instead of importing; the nine `invokeHook` call sites and `defineHook` are unchanged. **The broker does not know what a capability is**: it sends a shape descriptor built by the same own-string-key walk `wrapWithLogging` does (functions → `methods`, everything else → `data`, which is where `stampProvider`'s `providerModuleId` lives), and the shim rebuilds forwarding proxies from it — so an optional method a provider did not implement is absent rather than present-and-throwing, and `if (cap.registerTrustedSource)` keeps answering correctly. A socket rather than stdout because module scripts spawn subprocesses and a grandchild writing to fd 1 would corrupt the frame stream. Two consequences worth knowing: the child's environment is an **allow-list** (`hookChildEnv` — how to run: `PATH`, `HOME`, `LANG`, `TZ`, `TMPDIR`; whom to trust: `NODE_EXTRA_CA_CERTS`, `SSL_CERT_FILE`, `SSL_CERT_DIR`; the proxy variables; the `CELILO_HOOK_*` channels; `CELILO_DEBUG` — `FORWARDED_ENV` in executor.ts is the source of truth), so a hook reading any other operator variable now gets `undefined`; and a timeout is a real SIGTERM-then-SIGKILL with the broker refusing further capability calls, replacing a `Promise.race` that cancelled nothing and let a "timed out" hook go on writing DNS and firewall state (celilo#1003). Capability PROVIDER factories still load in-process — they ARE the broker's implementation. Stages 1 and 2 of `openspec/changes/hook-process-boundary`; the filesystem is claimed by the jail below, and SSH reachability by the remote-ops broker next.
204
+ - **Hook-owned state (a hook WRITES what it discovers; it never returns it)** — `apps/celilo/src/hooks/hook-store.ts` (broker side, over `(db, moduleId, manifest, masterKey)`) and `apps/celilo/src/hooks/hook-store-proxy.ts` (the child-side buffering proxy). Two `HookStore` instances reach every hook as `context.config` and `context.secrets`, each with async `get`/`set`/`delete`/`transaction` — async because a hook is a subprocess and every call is a broker round trip over the `store` RPC family (hook protocol version 2). `config` validates against the `variables.owns` entries declared `source: hook` and writes plaintext via `upsertModuleConfig`; `secrets` validates against `secrets.declares` and encrypts. **An undeclared name THROWS, naming the module and the declared set** (design D2) — a silent drop is the failure this whole subsystem exists to delete. `transaction(fn)` buffers CHILD-side, so a throwing `fn` sends no frame at all and the broker applies a finished op list inside one `db.transaction`. The executor injects both lazily through a `hookStores` provider at all nine `invokeHook` sites; a provider-less run REFUSES store writes loudly rather than dropping them. **The counterpart fact is an absence**: contract 1.0 swept a hook's RETURNED record into secrets and config from three capture sites in the deploy path, and all three are gone — nothing anywhere persists a hook return value, `'1.0'` is refused by name as a RETIRED contract, and the recurrence gate is `apps/celilo/test-integration/module/hook-returns-are-never-persisted.test.ts` (behavioural for `validate_config`/`on_install`, reach-measured static over every `invokeHook` caller for `container_created`). A framework-owned provider secret goes through the named `recordProviderSecret` in `secrets/storage.ts`, which no hook can reach. See `openspec/changes/hook-owned-state/`.
204
205
  - **Unjailed advisory lint (task 4.7, NOT a security boundary)** — `apps/celilo/src/hooks/unjailed-lint.ts`. Where there is no jail backend the executor passes the run's derived mount set to the shim in the environment (`CELILO_HOOK_MOUNT_SET`), and the shim wraps the path-taking `node:fs` / `node:fs/promises` functions so an access outside the set (or a write to a read-only row) warns through the hook's own logger: "on a jailed host this would fail", and that it is advisory. It is the mount-set derivation's SECOND consumer, so it cannot drift from what the jail enforces. It observes JS-level `node:fs` calls only — module code bypasses it trivially — and it must never be described as a boundary, in code or output.
205
206
  - **Remote-ops broker (reachability scoped by the credential, stage 3 / D12)** — `apps/celilo/src/hooks/remote-broker.ts` (`startRemoteBroker`, `RemoteAccessPolicy`) answering a SECOND socket beside the capability one, with the policy in `apps/celilo/src/services/remote-access.ts` (`remoteAccessPolicy`) and the asking half inside `@celilo/capabilities`' own remote primitives (`packages/capabilities/src/remote.ts`, "The hook remote-ops bridge"). The jail binds no `~/.ssh`, so a hand-built `ssh` cannot authenticate; the primitives detect `CELILO_HOOK_REMOTE_SOCKET`, send each operation as a STRUCTURED request (never a shell string — the broker rebuilds the ssh line itself), and the broker checks the target against `ownedSystemModuleIds` + `getModuleSystems` before running anything, refusing with the module, the target, and the capability route named. Requests carrying the module's OWN credential (an `identityFile` crossing as content and materialised per call, or `installAuthorizedKey`'s password — the cPanel case) are scoped by that credential instead; an explicit non-root user likewise, because the fleet key's authority is root on fleet systems. Stream primitives' LOCAL paths are confined to the run's granted roots (stateDir, screenshots, generated/, declared path inputs), or `streamBackup` would be a write-as-celilo oracle. Attribution is by the module that PERFORMS the operation: a hook's request to the hook's module (the nine `invokeHook` sites build the policy), a provider's transport to the provider's module — providers run in-process and do not cross this socket, and `public_web`'s hand-built upload was replaced by the Ansible static-content converge (capability-owned-tables stage 4, celilo#1014). Residual, recorded rather than papered over: a hook can still `fetch()` any HTTP endpoint directly; only `probeHttp` consults the target check.
206
207
  - **Hook jail (a hook sees the paths it was given, and nothing else)** — `apps/celilo/src/hooks/mount-set.ts` (`deriveMountSet`, `toBwrapArgs`, `forbiddenPaths` — PURE, computes a filesystem view and touches nothing) and `apps/celilo/src/hooks/jail.ts` (`detectJailBackend`, `planJailedSpawn`, `realpathRequest`, `runtimeModulePathsFor`, `recordJailMode` — the half that touches the machine). `executeHookScript` spawns the shim under `bwrap` with the module's tree read-only, `state/` + `generated/` + this run's `screenshots/<run>` read-write on top of it, each contract-declared path input at its declared access, the broker's socket directory, the runtime and the `node_modules` the shim resolves through, and `/tmp` a fresh tmpfs FIRST so it cannot erase the socket or a staged input. `~/.ssh` is deliberately NOT bound (stage 3, D12): withholding the credential is what makes the remote-ops broker's target check a boundary. **The acceptance criterion is absence, not a check**: the module store and the data directory are simply not bound, so `master.key` and a sibling module give `ENOENT`. The set is DERIVED — a module cannot ask for more — and `bwrap` itself is never in it, because the AppArmor profile grants `userns` to `/usr/bin/bwrap` for anyone on the box (design D9). Backend detection RUNS bubblewrap rather than looking for it (four different denials all leave the binary in place), and the resulting mode is written to `hook-jail-mode.json` beside celilo's other per-machine state rather than logged, so a host that stops jailing is readable. The policy is `auto` / `required` / `off`, resolved in a fixed precedence: the `CELILO_HOOK_JAIL` environment variable, then the stored `hooks.jail_policy` system config key (`celilo system config set hooks.jail_policy <value>`, which asks an interview question before writing `off` unless `--force`), then the default, which is **`off`** (ce-rez7). Jailing begins because an operator set a policy, never because an upgrade installed a backend — celilo-mgr's 2026-09-07 move to 1:2.2.1 installed bubblewrap and loaded the AppArmor profile and hooks kept running unjailed, which is the ruling working. `resolveJailPolicy` returns the source alongside the policy and `system doctor` renders it, so an effective value is always locatable. macOS has a `sandbox-exec` backend in the union, but `auto` defers on it (ce-29z: sandbox-exec DENIES undeclared writes where bubblewrap masks them, and D14's declared-path mechanism is not built), so the hook runs unjailed with the mode recorded; `required` bypasses the deferral for an operator who opts in. An `unjailed` record carries `lastJailed` (the jailed record it replaced on the same host), which is what the `hook_jail` self-monitor reads (`services/alerting/hook-jail.ts`, created unsuppressible by `celilo monitor add hook_jail`) to alert on a host that used to jail and has stopped. `celilo system doctor`'s "Hook execution" section (`renderHookExecutionSection`) reports the live mode and, when unjailed, why.
@@ -252,6 +253,7 @@ to look, which is how a forgotten pause actually gets found.
252
253
 
253
254
  - **Generator** — `apps/celilo/src/templates/generator.ts` — `generateTemplates` (orchestration), plus Terraform/Ansible file handling.
254
255
  - **Variable resolution** — `apps/celilo/src/variables/resolver.ts` (parser: `apps/celilo/src/variables/parser.ts`). Supported prefixes: `$self`, `$system`, `$secret`, `$system_secret`, `$capability`, `$infra`.
256
+ - **Where a config value came from (`module_configs.source`)** — the `source` column (migration `0031_module_config_source.sql`, nullable, no backfill — nothing in an existing row proves who wrote it). The variable `source:` enum in `apps/celilo/src/manifest/schema.ts` and `schemas/module-manifest.schema.json` carries **`hook`** alongside `user`/`system`/`capability`/`infrastructure`/`terraform`: the value is written by the owning module's own hook through `context.config.set`, so the configuration interview never prompts for it (`deploy-validation.ts`, `deploy-preflight.ts`, `templates/generator.ts` all skip it the way they skip `infrastructure`) and `celilo module config set` refuses it — `isDerivedVariable` refuses every non-`user` source, and `describeDerivedSource`/`explainNotSettable` name the accessor. An operator and a hook writing the same key would otherwise overwrite each other, which is what `source: user` on four wireguard keys actually did.
255
257
 
256
258
  ## Secrets
257
259
 
@@ -51,6 +51,7 @@ expect(probe(sys, { kind: 'systemd', unit: 'caddy' }, run).healthy).toBe(true);
51
51
  | Read a unit's journal | `tailLog` / `grepLog` | `ssh … journalctl` |
52
52
  | Run a secret on a command line | `runAppCommandWithSecret` | secret on argv |
53
53
  | Anything else with no capability/HTTP/converge path | `runAppCommand` (+ `// escape-hatch:`) | a raw ssh string |
54
+ | Persist state the hook discovered | `context.config.set` / `context.secrets.set` | returning it; `execFileSync('celilo', …)` |
54
55
 
55
56
  **Reach for a capability method first.** If another module already provides the
56
57
  thing (open a port → `firewall.exposeService`; register a route →
@@ -177,6 +178,14 @@ directly — prefer the specific primitive. `opts` is `{ input?, timeoutMs? }`.
177
178
  - **No `sleep`** — `waitFor` on the real condition.
178
179
  - **Computed config → `applyRenderedConfig`**, not `ssh cat >`. It validates and
179
180
  rolls back; a broken config never goes live.
181
+ - **State a hook discovered is WRITTEN, never returned.** `context.config.set`
182
+ if it is not sensitive, `context.secrets.set` if it is — sensitivity decides
183
+ the store, and nothing else does. Declare the name first (`source: hook` in
184
+ `variables.owns`, or `secrets.declares`) or the write throws. A returned
185
+ record is persisted nowhere, and the hook jail has no `celilo` binary, so
186
+ shelling out to `celilo module config set` reports success and writes
187
+ nothing. Details: MODULE_DEVELOPMENT_GUIDE.md, "Persisting state a hook
188
+ discovers".
180
189
 
181
190
  See also: `CELILO_SUBSYSTEMS.md` (the primitive impls + the firewall converge),
182
191
  `openspec/changes/unified-management-no-ssh/proposal.md` (the design), `reference/MODULE_DEVELOPMENT_GUIDE.md`.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@celilo/cli",
3
- "version": "5.0.0",
3
+ "version": "5.1.0",
4
4
  "description": "Celilo — home lab orchestration CLI",
5
5
  "type": "module",
6
6
  "bin": {
@@ -59,7 +59,7 @@
59
59
  "dependencies": {
60
60
  "@aws-sdk/client-s3": "^3.1109.0",
61
61
  "@aws-sdk/lib-storage": "^3.1101.0",
62
- "@celilo/capabilities": "^6.0.0",
62
+ "@celilo/capabilities": "^6.1.0",
63
63
  "@celilo/cli-display": "^0.2.0",
64
64
  "@celilo/core": "^0.14.0",
65
65
  "@celilo/event-bus": "^0.7.0",
@@ -0,0 +1,158 @@
1
+ /**
2
+ * `orchestrator_state` reaches a module through the loader, and only when the
3
+ * manifest declares it.
4
+ *
5
+ * INV-2 (module-orchestrator-primitives design.md): the declaration is the
6
+ * authorization. Like `control_plane_api`, this capability is framework-granted
7
+ * — no provider module exists — so the ordinary "inject everything buildable"
8
+ * loop does not apply and the loader must gate the injection on the `requires`
9
+ * line. Modeled on `capability-loader-control-plane-api.test.ts`, which proves
10
+ * the same property for `control_plane_api` through the loader rather than on
11
+ * the builder, because the loader is where a wrong argument would be passed.
12
+ */
13
+
14
+ import { afterEach, beforeEach, describe, expect, test } from 'bun:test';
15
+ import type { HookLogger } from '@celilo/capabilities';
16
+ import type { DbClient } from '../db/client';
17
+ import { cleanupTestDatabase, setupTestDatabase } from '../test-utils/database';
18
+ import { loadCapabilityFunctions } from './capability-loader';
19
+
20
+ const noopLogger: HookLogger = {
21
+ info() {},
22
+ warn() {},
23
+ error() {},
24
+ success() {},
25
+ };
26
+
27
+ function installModule(db: DbClient, moduleId: string, manifest: Record<string, unknown>): void {
28
+ db.$client.run(
29
+ `INSERT INTO modules (id, name, version, source_path, manifest_data) VALUES (?, ?, '1.0.0', '/tmp/${moduleId}', ?)`,
30
+ [moduleId, moduleId, JSON.stringify(manifest)],
31
+ );
32
+ }
33
+
34
+ const DECLARES = {
35
+ requires: { capabilities: [{ name: 'orchestrator_state', version: '1.0.0' }] },
36
+ };
37
+
38
+ describe('orchestrator_state injection', () => {
39
+ let db: DbClient;
40
+
41
+ beforeEach(async () => {
42
+ db = await setupTestDatabase();
43
+ });
44
+
45
+ afterEach(async () => {
46
+ await cleanupTestDatabase(db);
47
+ });
48
+
49
+ test('a module that declares it gets it, with no provider module deployed', async () => {
50
+ installModule(db, 'wireguard', DECLARES);
51
+
52
+ const capabilities = await loadCapabilityFunctions('wireguard', db, noopLogger);
53
+
54
+ expect(capabilities.orchestrator_state).toBeTruthy();
55
+ const state = capabilities.orchestrator_state as Record<string, unknown>;
56
+ expect(typeof state.get_system_config).toBe('function');
57
+ });
58
+
59
+ test('a module that declares it as optional gets it too', async () => {
60
+ installModule(db, 'someday-consumer', {
61
+ optional: { capabilities: [{ name: 'orchestrator_state', version: '1.0.0' }] },
62
+ });
63
+
64
+ const capabilities = await loadCapabilityFunctions('someday-consumer', db, noopLogger);
65
+
66
+ expect(capabilities.orchestrator_state).toBeTruthy();
67
+ });
68
+
69
+ test('a module that declares nothing does NOT get it', async () => {
70
+ installModule(db, 'hello-foo', { requires: { capabilities: [] } });
71
+
72
+ const capabilities = await loadCapabilityFunctions('hello-foo', db, noopLogger);
73
+
74
+ expect(capabilities.orchestrator_state).toBeUndefined();
75
+ });
76
+
77
+ test('a module that requires something ELSE does NOT get it', async () => {
78
+ installModule(db, 'hello-bar', {
79
+ requires: { capabilities: [{ name: 'firewall', version: '1.0.0' }] },
80
+ });
81
+
82
+ const capabilities = await loadCapabilityFunctions('hello-bar', db, noopLogger);
83
+
84
+ expect(capabilities.orchestrator_state).toBeUndefined();
85
+ });
86
+
87
+ test('the injected read works end to end through the loader', async () => {
88
+ // Through the loader, not only on the builder: the loader is where a
89
+ // wrong db handle or a wrong module id would be passed.
90
+ installModule(db, 'wireguard', DECLARES);
91
+ db.$client.run(
92
+ `INSERT INTO system_config (key, value) VALUES ('network.control-plane-vpn.subnet', '10.255.255.0/24')`,
93
+ );
94
+
95
+ const capabilities = await loadCapabilityFunctions('wireguard', db, noopLogger);
96
+ const state = capabilities.orchestrator_state as {
97
+ get_system_config(request: { key: string }): Promise<{ value: string | null }>;
98
+ };
99
+
100
+ await expect(
101
+ state.get_system_config({ key: 'network.control-plane-vpn.subnet' }),
102
+ ).resolves.toEqual({ value: '10.255.255.0/24' });
103
+ });
104
+
105
+ test('the injected table refuses a request naming a module (INV-1 through the loader)', async () => {
106
+ installModule(db, 'wireguard', DECLARES);
107
+
108
+ const capabilities = await loadCapabilityFunctions('wireguard', db, noopLogger);
109
+ const state = capabilities.orchestrator_state as {
110
+ get_system_config(request: object): Promise<unknown>;
111
+ };
112
+ const smuggled = { key: 'k', module: 'celilo-mgmt' } as never;
113
+
114
+ await expect(state.get_system_config(smuggled)).rejects.toThrow(/'wireguard'/);
115
+ });
116
+
117
+ test('an allow-listed module gets the privileged list_machines verb', async () => {
118
+ // list_machines is a per-module trust decision (design INV-2): the pool
119
+ // goes only to the modules the allow-list names, even though both modules
120
+ // here declare the same capability for their config reads.
121
+ installModule(db, 'celilo-mgmt', DECLARES);
122
+
123
+ const capabilities = await loadCapabilityFunctions('celilo-mgmt', db, noopLogger);
124
+ const state = capabilities.orchestrator_state as Record<string, unknown>;
125
+
126
+ expect(typeof state.get_system_config).toBe('function');
127
+ expect(typeof state.list_machines).toBe('function');
128
+ });
129
+
130
+ test('a module declaring the capability but not allow-listed does NOT get list_machines', async () => {
131
+ installModule(db, 'wireguard', DECLARES);
132
+
133
+ const capabilities = await loadCapabilityFunctions('wireguard', db, noopLogger);
134
+ const state = capabilities.orchestrator_state as Record<string, unknown>;
135
+
136
+ expect(typeof state.get_system_config).toBe('function');
137
+ expect(state.list_machines).toBeUndefined();
138
+ });
139
+
140
+ test('the privileged list reads end to end through the loader', async () => {
141
+ // Through the loader, not only on the builder: the loader is where a
142
+ // wrong allow-list lookup or a wrong db handle would be passed.
143
+ installModule(db, 'celilo-mgmt', DECLARES);
144
+ db.$client.run(
145
+ `INSERT INTO machines (id, hostname, zone, ip_address, ssh_user, ssh_key_encrypted, hardware, role, interfaces)
146
+ VALUES ('m-1', 'pibox-1', 'dmz', '10.0.20.11', 'peba', 'enc:x', '{"cpu_cores":4,"memory_mb":8192,"disk_gb":64}', 'host', '[]')`,
147
+ );
148
+
149
+ const capabilities = await loadCapabilityFunctions('celilo-mgmt', db, noopLogger);
150
+ const state = capabilities.orchestrator_state as {
151
+ list_machines(request: object): Promise<{ machines: Array<Record<string, unknown>> }>;
152
+ };
153
+
154
+ const { machines } = await state.list_machines({});
155
+ expect(machines).toHaveLength(1);
156
+ expect(machines[0]).toMatchObject({ hostname: 'pibox-1', zone: 'dmz', role: 'host' });
157
+ });
158
+ });
@@ -41,6 +41,7 @@ import {
41
41
  systemConfig,
42
42
  webRoutes,
43
43
  } from '../db/schema';
44
+ import { allowedPrivilegedVerbs } from '../manifest/validate';
44
45
  import { resolveModuleStateWebRoot, resolveModuleWebRoot } from '../module/web-root';
45
46
  import { decryptSecret } from '../secrets/encryption';
46
47
  import { getOrCreateMasterKey } from '../secrets/master-key';
@@ -53,6 +54,7 @@ import {
53
54
  import { CONTROL_PLANE_MODULE_ID, getModuleSystems } from '../services/deployed-systems';
54
55
  import { withDnsInternalLedger } from '../services/dns-internal-records';
55
56
  import { withDnsRegistrationLedger } from '../services/dns-registrations';
57
+ import { buildOrchestratorState } from '../services/orchestrator-state';
56
58
  import { buildPortForwardStore } from '../services/port-forwards';
57
59
  import {
58
60
  TRUSTED_SUBNETS_CONFIG_KEY,
@@ -940,6 +942,28 @@ export async function loadCapabilityFunctions(
940
942
  debugLog(`control_plane_api: framework-granted, injected for ${consumingModuleId}`);
941
943
  }
942
944
 
945
+ // orchestrator_state: framework-granted like control_plane_api, and gated on
946
+ // the DECLARATION for the same reason (INV-2). The verbs read celilo's own
947
+ // tables — system_config, and for allow-listed modules the machine pool — so
948
+ // the requires line is the authorization, and a module that never declared
949
+ // the capability holds no table at all. Built for the calling module (INV-1):
950
+ // the db this function already holds is the store the verbs read, passed
951
+ // explicitly so the service tests can pin the closure against an isolated
952
+ // database. The privileged verbs come from the allow-list map, not the
953
+ // declaration: declaring the capability buys the config reads, not the pool.
954
+ if (consumerDeclares(db, consumingModuleId, 'orchestrator_state')) {
955
+ result.orchestrator_state = wrapWithLogging(
956
+ buildOrchestratorState(
957
+ consumingModuleId,
958
+ db,
959
+ allowedPrivilegedVerbs('orchestrator_state', consumingModuleId),
960
+ ),
961
+ logger,
962
+ 'orchestrator_state',
963
+ );
964
+ debugLog(`orchestrator_state: framework-granted, injected for ${consumingModuleId}`);
965
+ }
966
+
943
967
  // celilo#1072: the CALL is the binding, not the resolution. Everything above
944
968
  // is injected whether or not the consumer declared it — the loop's own
945
969
  // comment says "not just required ones" — so recording what was resolved
@@ -177,6 +177,34 @@ describe('~/.ssh is never in the mount set (stage 3, D12)', () => {
177
177
  });
178
178
  });
179
179
 
180
+ describe('a bound browser without fonts is a browser that cannot render', () => {
181
+ // celilo#1422. Binding BROWSER_ROOT makes the browser START; it does not make
182
+ // it RENDER. With no font configuration in the jail, Blink's remote font face
183
+ // path hits a fatal NOTREACHED on any page that loads a web font and the
184
+ // renderer dies mid navigation — surfacing as a Playwright selector timeout
185
+ // that names neither fonts nor the jail. Measured on the management host,
186
+ // same bwrap invocation, only these rows differing: 12 NOTREACHED without,
187
+ // 0 with.
188
+ //
189
+ // Tied to BROWSER_ROOT deliberately. Fonts are only interesting BECAUSE a
190
+ // browser is bound, so if the browser bind is ever dropped this test should
191
+ // be reconsidered with it rather than left asserting an unrelated directory.
192
+ test('every font directory the renderer needs is bound alongside the browser', () => {
193
+ const paths = pathsOf(BASE);
194
+ expect(paths).toContain(BROWSER_ROOT);
195
+ for (const dir of ['/usr/share/fonts', '/usr/share/fontconfig', '/etc/fonts']) {
196
+ expect(paths).toContain(dir);
197
+ }
198
+ });
199
+
200
+ test('fonts are read-only — a hook has no business writing to them', () => {
201
+ for (const dir of ['/usr/share/fonts', '/usr/share/fontconfig', '/etc/fonts']) {
202
+ const row = deriveMountSet(BASE).entries.find((e) => e.path === dir);
203
+ expect(row?.mode).toBe('ro');
204
+ }
205
+ });
206
+ });
207
+
180
208
  describe('the fleet browser is reachable, and only read-only (task 4.10)', () => {
181
209
  test('BROWSER_ROOT is bound', () => {
182
210
  // Task 4.10 names `~/.cache/ms-playwright`. That path is stale:
@@ -112,6 +112,34 @@ export interface MountSetRequest {
112
112
  /** Directories whose contents the runtime needs in order to start at all. */
113
113
  const RUNTIME_SUPPORT_DIRS = ['/usr/lib', '/lib', '/lib64', '/etc/ssl'] as const;
114
114
 
115
+ /**
116
+ * What RENDERING needs, as distinct from what running needs.
117
+ *
118
+ * Omitting these does not stop the browser starting, which is why it survived
119
+ * the bind added for `BROWSER_ROOT` below. It kills the RENDERER, and only on a
120
+ * page that loads a web font. Blink's remote font face path hits a fatal
121
+ * `NOTREACHED` with no font configuration present, the renderer dies mid
122
+ * navigation, and Playwright reports `Target page, context or browser has been
123
+ * closed` while awaiting a selector — which reads as a slow page or a flaky
124
+ * selector and names neither fonts nor the jail.
125
+ *
126
+ * Measured 2026-09-25 on the management host, same bwrap invocation, only these
127
+ * three rows differing (celilo#1422):
128
+ *
129
+ * without: 12x ERROR ... remote_font_face_source.cc:365] NOTREACHED hit.
130
+ * with: 0
131
+ *
132
+ * celilo PROVISIONS these deliberately — `celilo-mgmt`'s browser block installs
133
+ * `fontconfig` and `fonts-dejavu-core` because "a screenshot with no glyphs is
134
+ * not worth retaining" (D7). So the fleet already pays for fonts and the jail
135
+ * was hiding them from the one process that needs them: the provisioning and
136
+ * the mount set disagreed, and the browser lost.
137
+ *
138
+ * Read-only, and absent rows are dropped like any other, so a host with no
139
+ * fonts installed is exactly as it was rather than newly fatal.
140
+ */
141
+ const FONT_DIRS = ['/usr/share/fonts', '/usr/share/fontconfig', '/etc/fonts'] as const;
142
+
115
143
  /**
116
144
  * What resolving a hostname needs. Read-only, and absent ones are dropped.
117
145
  *
@@ -220,6 +248,11 @@ export function deriveMountSet(request: MountSetRequest): MountSet {
220
248
  for (const dir of RUNTIME_SUPPORT_DIRS) {
221
249
  entries.push(entry(dir, 'ro', 'shared libraries and trust store', 'runtime'));
222
250
  }
251
+ for (const dir of FONT_DIRS) {
252
+ entries.push(
253
+ entry(dir, 'ro', 'fonts — the renderer dies without them, see FONT_DIRS', 'runtime'),
254
+ );
255
+ }
223
256
  for (const file of RESOLVER_FILES) {
224
257
  entries.push(entry(file, 'ro', 'name resolution — see RESOLVER_FILES', 'runtime'));
225
258
  }
@@ -860,6 +860,20 @@ export const ModuleManifestSchema = z
860
860
  command: z.string().min(1).optional(),
861
861
  /** Path to a build script (relative to module directory). Mutually exclusive with command. */
862
862
  script: z.string().min(1).optional(),
863
+ /**
864
+ * Wall-clock budget for the build, in seconds (celilo#1252).
865
+ *
866
+ * The packager used to kill every build at a hardcoded 300s. A module
867
+ * whose build command declares its own retry budget — forgejo fetches
868
+ * two release archives from codeberg with `curl --retry` — could
869
+ * express far more wall clock than that cap allowed, and the retries
870
+ * written for a flaky upstream never ran. The cap is a property of the
871
+ * fetch the module performs, so the module names it.
872
+ *
873
+ * `packages/e2e/tests/module-build-budget-fits-timeout.test.ts` fails
874
+ * when a build command's worst-case curl budget exceeds this value.
875
+ */
876
+ timeout_seconds: z.number().int().positive().max(7200).default(300),
863
877
  artifacts: z
864
878
  .array(
865
879
  z
@@ -6,7 +6,7 @@
6
6
 
7
7
  import { describe, expect, it } from 'bun:test';
8
8
  import type { ModuleManifest } from './schema';
9
- import { validatePrivilegedCapabilities } from './validate';
9
+ import { isPrivilegedCapability, validatePrivilegedCapabilities } from './validate';
10
10
 
11
11
  function buildManifest(overrides: Partial<ModuleManifest>): ModuleManifest {
12
12
  return {
@@ -81,4 +81,43 @@ describe('validatePrivilegedCapabilities', () => {
81
81
  );
82
82
  expect(result?.errors).toHaveLength(2);
83
83
  });
84
+
85
+ describe('framework-granted capabilities that any module may declare', () => {
86
+ // `control_plane_api` and `orchestrator_state` are framework-granted but NOT
87
+ // allow-listed: the `requires` line is the authorization (web-ui-console
88
+ // D7b, module-orchestrator-primitives D3). The registration that matters is
89
+ // `isPrivilegedCapability` — when it returns false for one of these, three
90
+ // call sites (`capabilities/validation.ts:50`, `services/deploy-
91
+ // validation.ts:182`, `hooks/executor.ts:720`) go looking for a provider
92
+ // module that can never exist and refuse the module with "no module
93
+ // provides this capability". `validatePrivilegedCapabilities` alone cannot
94
+ // see that failure, which is why the predicate is asserted directly.
95
+ it('treats orchestrator_state as framework-granted', () => {
96
+ expect(isPrivilegedCapability('orchestrator_state')).toBe(true);
97
+ });
98
+
99
+ it('treats control_plane_api as framework-granted', () => {
100
+ expect(isPrivilegedCapability('control_plane_api')).toBe(true);
101
+ });
102
+
103
+ it('accepts any module requiring orchestrator_state', () => {
104
+ const result = validatePrivilegedCapabilities(
105
+ buildManifest({
106
+ id: 'wireguard',
107
+ requires: { capabilities: [{ name: 'orchestrator_state', version: '1.0.0' }] },
108
+ }),
109
+ );
110
+ expect(result).toBeNull();
111
+ });
112
+
113
+ it('still refuses a privilege that IS allow-listed, so the two categories stay distinct', () => {
114
+ const result = validatePrivilegedCapabilities(
115
+ buildManifest({
116
+ id: 'wireguard',
117
+ requires: { capabilities: [{ name: 'cross_module_read', version: '1.0.0' }] },
118
+ }),
119
+ );
120
+ expect(result).not.toBeNull();
121
+ });
122
+ });
84
123
  });
@@ -213,8 +213,32 @@ export function validateCapabilityNames(manifest: ModuleManifest): ValidationErr
213
213
  */
214
214
  const PRIVILEGED_CAPABILITY_ALLOW_LIST: Record<string, readonly string[]> = {
215
215
  cross_module_read: ['celilo-mgmt'],
216
+ // A PRIVILEGED VERB, not a capability: `orchestrator_state` itself stays in
217
+ // FRAMEWORK_GRANTED_CAPABILITIES (any module may read system_config), and
218
+ // this verb-scoped key names the only module trusted with the machine pool
219
+ // (design INV-2). The key never matches a declared capability name, so
220
+ // import-time validation is untouched; the capability loader consults it
221
+ // through allowedPrivilegedVerbs when it builds the table.
222
+ 'orchestrator_state.list_machines': ['celilo-mgmt'],
216
223
  };
217
224
 
225
+ /**
226
+ * The privileged verbs of one capability a module may hold, from the
227
+ * allow-list above. Verb-scoped keys are `<capability>.<verb>`; a module not
228
+ * named for a verb gets no entry, and the builder omits the method.
229
+ */
230
+ export function allowedPrivilegedVerbs(capability: string, moduleId: string): string[] {
231
+ const verbs: string[] = [];
232
+ for (const key of Object.keys(PRIVILEGED_CAPABILITY_ALLOW_LIST)) {
233
+ const [keyCapability, keyVerb] = key.split('.');
234
+ if (keyCapability === capability && keyVerb) {
235
+ const allowed = PRIVILEGED_CAPABILITY_ALLOW_LIST[key];
236
+ if (allowed.includes(moduleId)) verbs.push(keyVerb);
237
+ }
238
+ }
239
+ return verbs;
240
+ }
241
+
218
242
  /**
219
243
  * Framework-granted capabilities that any module may declare.
220
244
  *
@@ -235,7 +259,17 @@ const PRIVILEGED_CAPABILITY_ALLOW_LIST: Record<string, readonly string[]> = {
235
259
  * ids in core, so every new console-shaped consumer would need a core change
236
260
  * and a `.deb` release before it could be imported at all.
237
261
  */
238
- const FRAMEWORK_GRANTED_CAPABILITIES: ReadonlySet<string> = new Set(['control_plane_api']);
262
+ const FRAMEWORK_GRANTED_CAPABILITIES: ReadonlySet<string> = new Set([
263
+ 'control_plane_api',
264
+ // module-orchestrator-primitives slice 2: reads of celilo's own state
265
+ // (`system_config`, and the privileged `list_machines` verb for
266
+ // allow-listed modules). The table is built for the calling module and the
267
+ // declaration is the authorization, like `control_plane_api` — no
268
+ // capability-level allow-list, because the config reads are not a per-module
269
+ // trust decision. The pool snapshot verb is, and its allow-list lives in
270
+ // PRIVILEGED_CAPABILITY_ALLOW_LIST as a verb-scoped key.
271
+ 'orchestrator_state',
272
+ ]);
239
273
 
240
274
  /**
241
275
  * Whether `name` is a framework-granted privilege rather than a normal
@@ -348,12 +348,19 @@ export async function buildModule(options: ModuleBuildOptions): Promise<ModuleBu
348
348
  // variable expansion would strip shell-only bash variables like
349
349
  // $STAGE before bash ever sees them.
350
350
  const buildEnv = { ...process.env, CELILO_MODULE_SOURCE_DIR: sourceDir };
351
+ // celilo#1252: the budget is the MODULE's, not a constant three layers
352
+ // away. forgejo's fetch declares `curl --retry 10 --max-time 900` and was
353
+ // killed at 300s having emitted five of its ten retries, reporting only
354
+ // "spawnSync bash ETIMEDOUT" — which names neither the cap, the elapsed
355
+ // time, nor the module.
356
+ const timeoutMs = (manifest.build.timeout_seconds ?? 300) * 1000;
357
+ const startedAt = Date.now();
351
358
  try {
352
359
  if (manifest.build.command) {
353
360
  execFileSync('bash', ['-c', manifest.build.command], {
354
361
  cwd: buildDir,
355
362
  stdio: 'inherit',
356
- timeout: 300_000,
363
+ timeout: timeoutMs,
357
364
  env: buildEnv,
358
365
  });
359
366
  } else {
@@ -362,12 +369,19 @@ export async function buildModule(options: ModuleBuildOptions): Promise<ModuleBu
362
369
  execFileSync(cmd, [script], {
363
370
  cwd: buildDir,
364
371
  stdio: 'inherit',
365
- timeout: 300_000,
372
+ timeout: timeoutMs,
366
373
  env: buildEnv,
367
374
  });
368
375
  }
369
376
  } catch (buildError) {
370
377
  const msg = buildError instanceof Error ? buildError.message : String(buildError);
378
+ if (msg.includes('ETIMEDOUT')) {
379
+ const elapsed = Math.round((Date.now() - startedAt) / 1000);
380
+ return {
381
+ success: false,
382
+ error: `Auto-build failed: ${moduleId}'s build exceeded its ${timeoutMs / 1000}s timeout (killed after ${elapsed}s). Raise build.timeout_seconds in manifest.yml if the build legitimately needs longer, or shrink what it fetches.`,
383
+ };
384
+ }
371
385
  return { success: false, error: `Auto-build failed: Build exited with code ${msg}` };
372
386
  }
373
387
 
@@ -155,4 +155,10 @@ export const CAPABILITY_SHAPE_BASELINE: Readonly<Record<string, CapabilityShape>
155
155
  requests:
156
156
  '[{kind:"interface",members:[{name:"all",optional:false,type:"Promise<PrivateWebRoute[]>"},{name:"replaceAll",optional:false,type:"Promise<void>"}],name:"PrivateRouteView"},{kind:"interface",members:[{name:"hostname",optional:false,type:"string"},{name:"module_id",optional:false,type:"string"},{name:"path",optional:false,type:"string"},{name:"slug",optional:false,type:"string"},{name:"target_host",optional:true,type:"string"},{name:"target_port",optional:true,type:"number"},{name:"type",optional:false,type:"\'static\' | \'reverse_proxy\'"},{name:"websocket",optional:true,type:"boolean"}],name:"PrivateWebRoute"}]',
157
157
  },
158
+ orchestrator_state: {
159
+ version: '1.0.0',
160
+ hash: '5127c0aa524cdd03bbcfa080a220307dfe6742143d76f3a50d7fa4870519b0c4',
161
+ requests:
162
+ '[{kind:"interface",members:[{name:"key",optional:false,type:"string"}],name:"GetSystemConfigRequest"},{kind:"interface",members:[{name:"value",optional:false,type:"string | null"}],name:"GetSystemConfigResult"},{kind:"alias",members:[{name:"_self",optional:false,type:"Record<string, never>"}],name:"ListMachinesRequest"},{kind:"interface",members:[{name:"machines",optional:false,type:"MachineSnapshot[]"}],name:"ListMachinesResult"},{kind:"interface",members:[{name:"arch",optional:true,type:"string"},{name:"cpu_cores",optional:false,type:"number"},{name:"disk_gb",optional:false,type:"number"},{name:"memory_mb",optional:false,type:"number"}],name:"MachineHardwareSnapshot"},{kind:"interface",members:[{name:"ip_address",optional:false,type:"string"},{name:"name",optional:false,type:"string"},{name:"zone",optional:false,type:"string"}],name:"MachineInterfaceSnapshot"},{kind:"interface",members:[{name:"earmarked_module",optional:false,type:"string | null"},{name:"hardware",optional:false,type:"MachineHardwareSnapshot"},{name:"hostname",optional:false,type:"string"},{name:"id",optional:false,type:"string"},{name:"interfaces",optional:false,type:"MachineInterfaceSnapshot[]"},{name:"ip_address",optional:false,type:"string"},{name:"role",optional:false,type:"\'host\' | \'router\'"},{name:"ssh_user",optional:false,type:"string"},{name:"zone",optional:false,type:"string"}],name:"MachineSnapshot"},{kind:"interface",members:[{name:"get_system_config",optional:false,type:"Promise<GetSystemConfigResult>"},{name:"list_machines",optional:true,type:"Promise<ListMachinesResult>"}],name:"OrchestratorStateCapability"}]',
163
+ },
158
164
  };
@@ -261,6 +261,12 @@ export const CAPABILITY_NAME_BASELINE: readonly CapabilityNameRow[] = [
261
261
  count: 1,
262
262
  why: 'L1-L12 — core branches on the capability NAME; becomes a provider declaration in Phase 3 (#938)',
263
263
  },
264
+ {
265
+ file: 'apps/celilo/src/manifest/validate.ts',
266
+ capability: 'orchestrator_state',
267
+ count: 1,
268
+ why: "PERMANENT — orchestrator_state is framework-granted, not module-provided: its verbs read celilo's own tables, so celilo builds the method table and no module can (module-orchestrator-primitives)",
269
+ },
264
270
  {
265
271
  file: 'apps/celilo/src/manifest/validate.ts',
266
272
  capability: 'cross_module_read',
@@ -273,6 +279,12 @@ export const CAPABILITY_NAME_BASELINE: readonly CapabilityNameRow[] = [
273
279
  count: 1,
274
280
  why: 'PERMANENT — framework-granted, so celilo satisfies it and no module provides it; the resolver has to be told that by name (web-ui-console D7b)',
275
281
  },
282
+ {
283
+ file: 'apps/celilo/src/hooks/capability-loader.ts',
284
+ capability: 'orchestrator_state',
285
+ count: 5,
286
+ why: "PERMANENT — orchestrator_state is framework-granted, not module-provided: its verbs read celilo's own tables, so celilo builds the method table and no module can (module-orchestrator-primitives)",
287
+ },
276
288
  {
277
289
  file: 'apps/celilo/src/hooks/capability-loader.ts',
278
290
  capability: 'control_plane_api',
@@ -375,6 +387,12 @@ export const CAPABILITY_NAME_BASELINE: readonly CapabilityNameRow[] = [
375
387
  count: 1,
376
388
  why: 'PERMANENT — X10, a contract declaration with no implementation; this is the package doing its job',
377
389
  },
390
+ {
391
+ file: 'packages/capabilities/src/capability-contract.ts',
392
+ capability: 'orchestrator_state',
393
+ count: 1,
394
+ why: 'PERMANENT — X10, a contract declaration with no implementation; this is the package doing its job',
395
+ },
378
396
  {
379
397
  file: 'packages/capabilities/src/capability-contract.ts',
380
398
  capability: 'cross_module_read',
@@ -507,6 +525,11 @@ export const SERVICE_FILENAME_BASELINE: readonly ServiceFilenameRow[] = [
507
525
  capability: 'cross_module_read',
508
526
  why: 'PERMANENT — cross_module_read is a framework privilege, not a module capability',
509
527
  },
528
+ {
529
+ file: 'apps/celilo/src/services/orchestrator-state.ts',
530
+ capability: 'orchestrator_state',
531
+ why: "PERMANENT — orchestrator_state is framework-granted, not module-provided: its verbs read celilo's own tables, so celilo builds the method table and no module can (module-orchestrator-primitives)",
532
+ },
510
533
  {
511
534
  file: 'apps/celilo/src/services/provider-converge.ts',
512
535
  capability: 'provider_converge',
@@ -513,10 +513,11 @@ describe('RegistryClient.download integrity', () => {
513
513
  });
514
514
 
515
515
  test('a non-digest cksum sentinel skips verification instead of failing', async () => {
516
- // The registry's bootstrap path publishes `cksum: 'bootstrap'`
517
- // (packages/registry-server/src/bootstrap.ts:114) because those modules are
518
- // packaged on demand and have no stable digest. Treating a sentinel as a
519
- // digest rejects a perfectly good 2.7MB package on every import.
516
+ // `bootstrap` is the sentinel an index entry carries when it has no
517
+ // integrity data at all. The registry's bootstrap path no longer emits it
518
+ // — it hashes the .netapp it packages (celilo#1262) — but the sentinel
519
+ // branch stays, because treating one as a digest would reject a perfectly
520
+ // good 2.7MB package on every import.
520
521
  fetchSpy.mockImplementation(() => new Response(body, { status: 200 }));
521
522
 
522
523
  const client = new RegistryClient('https://reg.example.com');
@@ -272,6 +272,52 @@ describe('infrastructure-selector', () => {
272
272
  });
273
273
  });
274
274
 
275
+ describe('the no-infrastructure error names what IS configured', () => {
276
+ it("names the service and the zones it covers when none covers the module's zone", async () => {
277
+ const db = createDbClient({ path: testDbPath });
278
+ await db.insert(modules).values({
279
+ id: 'needs-dmz',
280
+ name: 'Needs DMZ',
281
+ version: '1.0.0',
282
+ manifestData: {
283
+ requires: { system: { cpu: 1, memory: 512, disk: 5, zone: 'dmz' } },
284
+ },
285
+ sourcePath: '/test',
286
+ state: 'CONFIGURED',
287
+ });
288
+ const module = (await db.select().from(modules).limit(1))[0] as Module;
289
+
290
+ // A service exists — it just does not list 'dmz'. This is the case the
291
+ // old message described as having no service at all.
292
+ const service = await addContainerService({
293
+ name: 'Proxmox E2E',
294
+ providerName: 'proxmox',
295
+ zones: ['app', 'secure-mgmt'],
296
+ providerConfig: {},
297
+ apiCredentials: { api_url: 'https://test' },
298
+ });
299
+ const { updateVerificationStatus } = await import('./container-service');
300
+ await updateVerificationStatus(service.id, { success: true, message: 'ok' });
301
+
302
+ let message = '';
303
+ try {
304
+ await selectInfrastructure(module);
305
+ throw new Error('expected selectInfrastructure to throw');
306
+ } catch (err) {
307
+ expect(err).toBeInstanceOf(InfrastructureError);
308
+ message = (err as Error).message;
309
+ }
310
+
311
+ expect(message).toContain("zone 'dmz'");
312
+ expect(message).toContain('Proxmox E2E');
313
+ expect(message).toContain("'app'");
314
+ expect(message).toContain("'secure-mgmt'");
315
+ // The misleading half: do not tell the operator to configure a service
316
+ // they already have.
317
+ expect(message).not.toContain('celilo service add');
318
+ });
319
+ });
320
+
275
321
  describe('hardware validation', () => {
276
322
  it('rejects machine with insufficient CPU', async () => {
277
323
  const db = createDbClient({ path: testDbPath });
@@ -1,6 +1,7 @@
1
1
  import type { Module } from '../db/schema';
2
2
  import { type ModuleManifest, getSingularSystemSpec } from '../manifest/schema';
3
3
  import type {
4
+ ContainerService,
4
5
  InfrastructureSelection,
5
6
  Machine,
6
7
  MachineRole,
@@ -79,9 +80,30 @@ interface RejectionReason {
79
80
  /**
80
81
  * Build a detailed error message explaining why no infrastructure was selected
81
82
  */
82
- function buildInfrastructureErrorMessage(zone: string, rejections: RejectionReason[]): string {
83
+ function buildInfrastructureErrorMessage(
84
+ zone: string,
85
+ rejections: RejectionReason[],
86
+ allServices: ContainerService[],
87
+ ): string {
83
88
  if (rejections.length === 0) {
84
- return `No infrastructure available for zone '${zone}'. Configure a container service with 'celilo service add' or add a machine with 'celilo machine add'.`;
89
+ // Naming the services that DO exist is the whole value of this branch.
90
+ // "Configure a container service" is actively misleading when one is
91
+ // already configured and simply does not list this zone — it sends the
92
+ // reader to `machine add`, which is the wrong fix and, in an e2e fixture,
93
+ // a forbidden one (a machine-pool deploy never renders `main.tf.tpl`,
94
+ // celilo#879). Measured 2026-09-25: a suite whose Proxmox service covered
95
+ // `app` and `secure-mgmt` failed to place caddy in `dmz`, and this
96
+ // sentence described the situation as having no service at all.
97
+ const covering = allServices
98
+ .map(
99
+ (svc) =>
100
+ `'${svc.name}' covers ${svc.zones.length > 0 ? svc.zones.map((z) => `'${z}'`).join(', ') : 'no zones'}`,
101
+ )
102
+ .join('; ');
103
+ const have = covering
104
+ ? ` Configured container services do not cover it: ${covering}. Add '${zone}' to one with 'celilo service reconfigure', or add a machine with 'celilo machine add'.`
105
+ : ` Configure a container service with 'celilo service add' or add a machine with 'celilo machine add'.`;
106
+ return `No infrastructure available for zone '${zone}'.${have}`;
85
107
  }
86
108
 
87
109
  const details = rejections
@@ -246,5 +268,7 @@ export async function selectInfrastructure(module: Module): Promise<Infrastructu
246
268
  }
247
269
 
248
270
  // 3. No infrastructure available
249
- throw new InfrastructureError(buildInfrastructureErrorMessage(zone, rejections));
271
+ throw new InfrastructureError(
272
+ buildInfrastructureErrorMessage(zone, rejections, await listContainerServices()),
273
+ );
250
274
  }
@@ -0,0 +1,211 @@
1
+ /**
2
+ * `orchestrator_state` — the framework-built method table.
3
+ *
4
+ * INV-1 (module-orchestrator-primitives design.md) is what these tests exist to
5
+ * pin: the module a verb acts on is a CONSTRUCTION argument, never a field in
6
+ * the request. The jail cannot be trusted to name itself honestly, so a request
7
+ * that arrives carrying a module field is refused even though the contract type
8
+ * has no such field — the broker hands the implementation whatever JSON the
9
+ * hook process sent, and the runtime check is the only thing standing between a
10
+ * hook and a neighbour's data once the shape has crossed the process boundary.
11
+ *
12
+ * The refusal message names the calling module, which is also how the closure
13
+ * itself is observable: two instances built for different modules read the same
14
+ * table but each names its OWN module when it refuses.
15
+ */
16
+
17
+ import { afterEach, beforeEach, describe, expect, test } from 'bun:test';
18
+ import type { DbClient } from '../db/client';
19
+ import { machines, systemConfig } from '../db/schema';
20
+ import { cleanupTestDatabase, setupTestDatabase } from '../test-utils/database';
21
+ import { buildOrchestratorState } from './orchestrator-state';
22
+
23
+ const VPN_SUBNET_KEY = 'network.control-plane-vpn.subnet';
24
+
25
+ /**
26
+ * Pull the privileged verb off the table with a guard that fails clearly.
27
+ * The method is optional on the contract (the allow-list decides), so a test
28
+ * that calls it unguarded reads as possibly-undefined rather than as the
29
+ * assertion it is: these tests granted the verb, and its absence is what the
30
+ * test should report.
31
+ */
32
+ function requireListMachines(state: ReturnType<typeof buildOrchestratorState>) {
33
+ const { list_machines } = state;
34
+ if (!list_machines) {
35
+ throw new Error('list_machines was not injected; the verbs argument did not grant it');
36
+ }
37
+ return list_machines;
38
+ }
39
+
40
+ describe('buildOrchestratorState', () => {
41
+ let db: DbClient;
42
+
43
+ beforeEach(async () => {
44
+ db = await setupTestDatabase();
45
+ });
46
+
47
+ afterEach(async () => {
48
+ await cleanupTestDatabase(db);
49
+ });
50
+
51
+ test('reads the value of a set system-config key', async () => {
52
+ db.insert(systemConfig).values({ key: VPN_SUBNET_KEY, value: '10.255.255.0/24' }).run();
53
+ const state = buildOrchestratorState('wireguard', db);
54
+
55
+ await expect(state.get_system_config({ key: VPN_SUBNET_KEY })).resolves.toEqual({
56
+ value: '10.255.255.0/24',
57
+ });
58
+ });
59
+
60
+ test('an unset key is value null, not an error', async () => {
61
+ // The case the replaced CLI call mangled: `celilo system config get`
62
+ // exits non-zero for an unset key, so the caller had to catch the spawn
63
+ // failure AND parse stdout. Here the normal absence is a typed result.
64
+ const state = buildOrchestratorState('wireguard', db);
65
+
66
+ await expect(state.get_system_config({ key: 'no.such.key' })).resolves.toEqual({
67
+ value: null,
68
+ });
69
+ });
70
+
71
+ test('a request carrying a module field is refused (INV-1)', async () => {
72
+ // The contract type has no `module` field, so this only arrives as a
73
+ // smuggled extra property across the broker. Refused, not ignored: an
74
+ // ignored field means a hook testing the water learns nothing.
75
+ const state = buildOrchestratorState('wireguard', db);
76
+ const smuggled = { key: VPN_SUBNET_KEY, module: 'celilo-mgmt' } as never;
77
+
78
+ await expect(state.get_system_config(smuggled)).rejects.toThrow(/module/);
79
+ });
80
+
81
+ test('the refusal names the module the table was built for (the closure, observed)', async () => {
82
+ // Two instances over the same db, built for different modules. The reads
83
+ // are identical; the refusals are not, and the difference is the proof
84
+ // that each instance closed over its own module id.
85
+ const wireguard = buildOrchestratorState('wireguard', db);
86
+ const impostor = buildOrchestratorState('celilo-mgmt', db);
87
+
88
+ const smuggled = { key: 'k', module: 'someone-else' } as never;
89
+ await expect(wireguard.get_system_config(smuggled)).rejects.toThrow(/'wireguard'/);
90
+ await expect(impostor.get_system_config(smuggled)).rejects.toThrow(/'celilo-mgmt'/);
91
+ });
92
+
93
+ test('a request carrying any unexpected field is refused, not just module-named ones', async () => {
94
+ // The general shape of INV-1: the request is exactly the contract, and
95
+ // anything extra is a jail trying to widen the surface. Refusing on the
96
+ // general rule covers the next smuggled field nobody thought to name.
97
+ const state = buildOrchestratorState('wireguard', db);
98
+ const widened = { key: 'k', value: '10.0.0.0/8' } as never;
99
+
100
+ await expect(state.get_system_config(widened)).rejects.toThrow(/value/);
101
+ });
102
+ });
103
+
104
+ describe('buildOrchestratorState list_machines', () => {
105
+ let db: DbClient;
106
+
107
+ beforeEach(async () => {
108
+ db = await setupTestDatabase();
109
+ });
110
+
111
+ afterEach(async () => {
112
+ await cleanupTestDatabase(db);
113
+ });
114
+
115
+ function insertMachine(overrides: Partial<typeof machines.$inferInsert> = {}): void {
116
+ db.insert(machines)
117
+ .values({
118
+ id: 'm-1',
119
+ hostname: 'pibox-1',
120
+ zone: 'dmz',
121
+ ipAddress: '10.0.20.11',
122
+ sshUser: 'peba',
123
+ sshKeyEncrypted: 'enc:never-read-back',
124
+ hardware: { cpu_cores: 4, memory_mb: 8192, disk_gb: 64 },
125
+ role: 'host',
126
+ interfaces: [{ name: 'eth0', ipAddress: '10.0.20.11', zone: 'dmz' }],
127
+ ...overrides,
128
+ })
129
+ .run();
130
+ }
131
+
132
+ test('an allowed module gets the pool, as the snapshot type carries it', async () => {
133
+ // The list_machines allow-list (design INV-2) is the loader's decision;
134
+ // the builder takes the allowed verbs as an argument so the service tests
135
+ // can pin the closure without importing the trust map.
136
+ insertMachine({ earmarkedModule: null });
137
+ insertMachine({
138
+ id: 'm-2',
139
+ hostname: 'gw-1',
140
+ role: 'router',
141
+ earmarkedModule: 'celilo-mgmt',
142
+ });
143
+ const state = buildOrchestratorState('celilo-mgmt', db, ['list_machines']);
144
+
145
+ const { machines: pool } = await requireListMachines(state)({});
146
+ expect(pool).toEqual([
147
+ {
148
+ id: 'm-1',
149
+ hostname: 'pibox-1',
150
+ zone: 'dmz',
151
+ ip_address: '10.0.20.11',
152
+ ssh_user: 'peba',
153
+ role: 'host',
154
+ interfaces: [{ name: 'eth0', ip_address: '10.0.20.11', zone: 'dmz' }],
155
+ hardware: { cpu_cores: 4, memory_mb: 8192, disk_gb: 64 },
156
+ earmarked_module: null,
157
+ },
158
+ {
159
+ id: 'm-2',
160
+ hostname: 'gw-1',
161
+ zone: 'dmz',
162
+ ip_address: '10.0.20.11',
163
+ ssh_user: 'peba',
164
+ role: 'router',
165
+ interfaces: [{ name: 'eth0', ip_address: '10.0.20.11', zone: 'dmz' }],
166
+ hardware: { cpu_cores: 4, memory_mb: 8192, disk_gb: 64 },
167
+ earmarked_module: 'celilo-mgmt',
168
+ },
169
+ ]);
170
+ });
171
+
172
+ test('an empty pool is an empty list, not an error', async () => {
173
+ // A fresh management box has no machines yet. The backup writes an empty
174
+ // inventory because the pool IS empty — a real result, not the swallow
175
+ // the replaced CLI spawn produced.
176
+ const state = buildOrchestratorState('celilo-mgmt', db, ['list_machines']);
177
+
178
+ const { machines: pool } = await requireListMachines(state)({});
179
+ expect(pool).toEqual([]);
180
+ });
181
+
182
+ test('the encrypted key column never reaches the snapshot', async () => {
183
+ // The one column the snapshot must not carry. Asserting on the VALUE (not
184
+ // just the type) because a pass-through of the row would ship the secret
185
+ // inside a correctly-shaped envelope.
186
+ insertMachine();
187
+ const state = buildOrchestratorState('celilo-mgmt', db, ['list_machines']);
188
+
189
+ const { machines: pool } = await requireListMachines(state)({});
190
+ expect(JSON.stringify(pool)).not.toContain('enc:never-read-back');
191
+ });
192
+
193
+ test('a request carrying any field is refused (INV-1)', async () => {
194
+ // The list_machines contract is empty, so ANY field is a smuggled
195
+ // widening — including the module id the jail cannot be trusted to name.
196
+ const state = buildOrchestratorState('celilo-mgmt', db, ['list_machines']);
197
+ const smuggled = { module: 'wireguard' } as never;
198
+
199
+ await expect(requireListMachines(state)(smuggled)).rejects.toThrow(/module/);
200
+ });
201
+
202
+ test('a table built without the verb allowed does not carry it', async () => {
203
+ // Deny by default (Rule 6.4): the loader passes the allow-list decision,
204
+ // and a builder that defaulted to generous would hand the pool to any
205
+ // module declaring orchestrator_state for the config reads.
206
+ insertMachine();
207
+ const state = buildOrchestratorState('wireguard', db);
208
+
209
+ expect((state as unknown as Record<string, unknown>).list_machines).toBeUndefined();
210
+ });
211
+ });
@@ -0,0 +1,122 @@
1
+ /**
2
+ * The `orchestrator_state` method table celilo builds for one consuming module.
3
+ *
4
+ * Framework-granted like `control_plane_api` (`api-principal-enrolment.ts`,
5
+ * whose shape this copies): the verbs answer questions about celilo's own
6
+ * tables, so no module script can implement them, and a module script may
7
+ * import nothing but `@celilo/capabilities`. celilo builds the table for the
8
+ * calling module and `capability-loader.ts` injects it when the manifest
9
+ * declares the capability — the `requires` line is the authorization.
10
+ *
11
+ * ## INV-1: the module is a construction argument, never a request field
12
+ *
13
+ * `buildOrchestratorState(moduleId)` closes over the module id, exactly as
14
+ * `buildControlPlaneApi` does. A request that arrives carrying any field the
15
+ * contract does not declare is REFUSED, not ignored: the broker hands this
16
+ * table whatever JSON the hook process sent, the contract type has no way to
17
+ * reject extra properties at compile time, and an ignored field would let a
18
+ * jail probe the surface while learning nothing. The refusal message names the
19
+ * calling module, so the log shows who tried to widen what.
20
+ *
21
+ * ## Reads through the existing store
22
+ *
23
+ * Both verbs read the tables the CLI reads, through drizzle, with the same
24
+ * injectable `db` (defaulting to the process-wide `getDb()`) so the INV-1
25
+ * tests can pin the closure against an isolated database instead of the
26
+ * operator's real fleet. `get_system_config` reads `system_config`;
27
+ * `list_machines` reads `machines` — the same rows `celilo machine list`
28
+ * renders, minus the encrypted key column, which no snapshot carries (the
29
+ * fleet keypair is staged into the backup envelope by the framework).
30
+ *
31
+ * `list_machines` is a PRIVILEGED verb: handing out the pool is a per-module
32
+ * trust decision (design INV-2), so the loader passes the verbs the calling
33
+ * module is allow-listed for and the table omits the rest. Deny by default:
34
+ * a builder that defaulted to generous would hand the pool to every module
35
+ * declaring `orchestrator_state` for its config reads.
36
+ */
37
+
38
+ import type { OrchestratorStateCapability } from '@celilo/capabilities';
39
+ import { eq } from 'drizzle-orm';
40
+ import { type DbClient, getDb } from '../db/client';
41
+ import { machines, systemConfig } from '../db/schema';
42
+
43
+ /**
44
+ * The fields `get_system_config`'s contract declares, and therefore the only
45
+ * fields a request may carry. Anything else is a smuggled widening and is
46
+ * refused with the offending field named.
47
+ */
48
+ const GET_SYSTEM_CONFIG_FIELDS = ['key'] as const;
49
+
50
+ /**
51
+ * Refuse a request carrying any field the contract does not declare.
52
+ *
53
+ * INV-1's runtime half. `as never` call sites are expected: a hook process
54
+ * sends JSON, and JSON has no compile-time shape once it crosses the broker.
55
+ */
56
+ function requireExactRequest(
57
+ moduleId: string,
58
+ verb: string,
59
+ request: object,
60
+ allowedFields: readonly string[],
61
+ ): void {
62
+ const unexpected = Object.keys(request).filter((field) => !allowedFields.includes(field));
63
+ if (unexpected.length === 0) return;
64
+ throw new Error(
65
+ `Module '${moduleId}' called orchestrator_state.${verb} with unexpected field(s): ${unexpected.join(', ')}. The calling module is a construction argument of this capability, never a request field; allowed fields for ${verb}: ${allowedFields.join(', ')}.`,
66
+ );
67
+ }
68
+
69
+ /**
70
+ * Build the `orchestrator_state` table for one module.
71
+ *
72
+ * `db` is injectable for tests; production callers (the capability loader)
73
+ * let it default to the process-wide client, the same way the enrolment
74
+ * service resolves its store.
75
+ *
76
+ * `allowPrivilegedVerbs` names the privileged verbs the calling module may
77
+ * hold, as computed by the loader from the allow-list in
78
+ * `manifest/validate.ts`. It defaults to empty (deny by default): a module
79
+ * declaring `orchestrator_state` gets `get_system_config` and nothing else
80
+ * unless the trust map names it.
81
+ */
82
+ export function buildOrchestratorState(
83
+ moduleId: string,
84
+ db: DbClient = getDb(),
85
+ allowPrivilegedVerbs: readonly string[] = [],
86
+ ): OrchestratorStateCapability {
87
+ const table: OrchestratorStateCapability = {
88
+ async get_system_config(request) {
89
+ requireExactRequest(moduleId, 'get_system_config', request, GET_SYSTEM_CONFIG_FIELDS);
90
+ const row = db.select().from(systemConfig).where(eq(systemConfig.key, request.key)).get();
91
+ return { value: row?.value ?? null };
92
+ },
93
+ };
94
+
95
+ if (allowPrivilegedVerbs.includes('list_machines')) {
96
+ table.list_machines = async (request) => {
97
+ // The contract is empty, so the allowed-field set is empty and any
98
+ // field at all — including a module id — is a smuggled widening.
99
+ requireExactRequest(moduleId, 'list_machines', request, []);
100
+ const rows = db.select().from(machines).all();
101
+ return {
102
+ machines: rows.map((row) => ({
103
+ id: row.id,
104
+ hostname: row.hostname,
105
+ zone: row.zone,
106
+ ip_address: row.ipAddress,
107
+ ssh_user: row.sshUser,
108
+ role: row.role,
109
+ interfaces: row.interfaces.map((iface) => ({
110
+ name: iface.name,
111
+ ip_address: iface.ipAddress,
112
+ zone: iface.zone,
113
+ })),
114
+ hardware: row.hardware,
115
+ earmarked_module: row.earmarkedModule ?? null,
116
+ })),
117
+ };
118
+ };
119
+ }
120
+
121
+ return table;
122
+ }