@celilo/cli 5.0.0 → 5.0.1

package/CELILO_SUBSYSTEMS.md

@@ -201,6 +201,7 @@ runner seam (`execRunner` real / `createMockRunner` for tests) lives in
 
 - **Hook executor / ABI** — `apps/celilo/src/hooks/executor.ts` (`invokeHook`, `executeHookScript`, `checkRequiredCapabilities`), types in `apps/celilo/src/hooks/types.ts` (`HookContext`, `HookDefinition`, `HookName`). Named-hook runner: `apps/celilo/src/hooks/run-named-hook.ts`. Manifest hook config: `apps/celilo/src/hooks/load-hook-config.ts`.
 - **Hook process boundary (a hook is a program celilo RUNS)** — `apps/celilo/src/hooks/hook-protocol.ts` (the NDJSON frame union, `HOOK_PROTOCOL_VERSION`, `serializeError`/`deserializeError`), `apps/celilo/src/hooks/broker.ts` (`startBroker`, `capabilityShape`), `apps/celilo/src/hooks/hook-runner-entry.ts` + `apps/celilo/src/hooks/hook-runner.ts` (the spawned shim — the ONLY thing that `import()`s module code; the entry exists to install the advisory lint before the shim's import graph can ESM-load `node:fs`, see `unjailed-lint.ts`). `executeHookScript` spawns `bun hook-runner-entry.ts` over a Unix socket instead of importing; the nine `invokeHook` call sites and `defineHook` are unchanged. **The broker does not know what a capability is**: it sends a shape descriptor built by the same own-string-key walk `wrapWithLogging` does (functions → `methods`, everything else → `data`, which is where `stampProvider`'s `providerModuleId` lives), and the shim rebuilds forwarding proxies from it — so an optional method a provider did not implement is absent rather than present-and-throwing, and `if (cap.registerTrustedSource)` keeps answering correctly. A socket rather than stdout because module scripts spawn subprocesses and a grandchild writing to fd 1 would corrupt the frame stream. Two consequences worth knowing: the child's environment is an **allow-list** (`hookChildEnv` — how to run: `PATH`, `HOME`, `LANG`, `TZ`, `TMPDIR`; whom to trust: `NODE_EXTRA_CA_CERTS`, `SSL_CERT_FILE`, `SSL_CERT_DIR`; the proxy variables; the `CELILO_HOOK_*` channels; `CELILO_DEBUG` — `FORWARDED_ENV` in executor.ts is the source of truth), so a hook reading any other operator variable now gets `undefined`; and a timeout is a real SIGTERM-then-SIGKILL with the broker refusing further capability calls, replacing a `Promise.race` that cancelled nothing and let a "timed out" hook go on writing DNS and firewall state (celilo#1003). Capability PROVIDER factories still load in-process — they ARE the broker's implementation. Stages 1 and 2 of `openspec/changes/hook-process-boundary`; the filesystem is claimed by the jail below, and SSH reachability by the remote-ops broker next.
+- **Hook-owned state (a hook WRITES what it discovers; it never returns it)** — `apps/celilo/src/hooks/hook-store.ts` (broker side, over `(db, moduleId, manifest, masterKey)`) and `apps/celilo/src/hooks/hook-store-proxy.ts` (the child-side buffering proxy). Two `HookStore` instances reach every hook as `context.config` and `context.secrets`, each with async `get`/`set`/`delete`/`transaction` — async because a hook is a subprocess and every call is a broker round trip over the `store` RPC family (hook protocol version 2). `config` validates against the `variables.owns` entries declared `source: hook` and writes plaintext via `upsertModuleConfig`; `secrets` validates against `secrets.declares` and encrypts. **An undeclared name THROWS, naming the module and the declared set** (design D2) — a silent drop is the failure this whole subsystem exists to delete. `transaction(fn)` buffers CHILD-side, so a throwing `fn` sends no frame at all and the broker applies a finished op list inside one `db.transaction`. The executor injects both lazily through a `hookStores` provider at all nine `invokeHook` sites; a provider-less run REFUSES store writes loudly rather than dropping them. **The counterpart fact is an absence**: contract 1.0 swept a hook's RETURNED record into secrets and config from three capture sites in the deploy path, and all three are gone — nothing anywhere persists a hook return value, `'1.0'` is refused by name as a RETIRED contract, and the recurrence gate is `apps/celilo/test-integration/module/hook-returns-are-never-persisted.test.ts` (behavioural for `validate_config`/`on_install`, reach-measured static over every `invokeHook` caller for `container_created`). A framework-owned provider secret goes through the named `recordProviderSecret` in `secrets/storage.ts`, which no hook can reach. See `openspec/changes/hook-owned-state/`.
 - **Unjailed advisory lint (task 4.7, NOT a security boundary)** — `apps/celilo/src/hooks/unjailed-lint.ts`. Where there is no jail backend the executor passes the run's derived mount set to the shim in the environment (`CELILO_HOOK_MOUNT_SET`), and the shim wraps the path-taking `node:fs` / `node:fs/promises` functions so an access outside the set (or a write to a read-only row) warns through the hook's own logger: "on a jailed host this would fail", and that it is advisory. It is the mount-set derivation's SECOND consumer, so it cannot drift from what the jail enforces. It observes JS-level `node:fs` calls only — module code bypasses it trivially — and it must never be described as a boundary, in code or output.
 - **Remote-ops broker (reachability scoped by the credential, stage 3 / D12)** — `apps/celilo/src/hooks/remote-broker.ts` (`startRemoteBroker`, `RemoteAccessPolicy`) answering a SECOND socket beside the capability one, with the policy in `apps/celilo/src/services/remote-access.ts` (`remoteAccessPolicy`) and the asking half inside `@celilo/capabilities`' own remote primitives (`packages/capabilities/src/remote.ts`, "The hook remote-ops bridge"). The jail binds no `~/.ssh`, so a hand-built `ssh` cannot authenticate; the primitives detect `CELILO_HOOK_REMOTE_SOCKET`, send each operation as a STRUCTURED request (never a shell string — the broker rebuilds the ssh line itself), and the broker checks the target against `ownedSystemModuleIds` + `getModuleSystems` before running anything, refusing with the module, the target, and the capability route named. Requests carrying the module's OWN credential (an `identityFile` crossing as content and materialised per call, or `installAuthorizedKey`'s password — the cPanel case) are scoped by that credential instead; an explicit non-root user likewise, because the fleet key's authority is root on fleet systems. Stream primitives' LOCAL paths are confined to the run's granted roots (stateDir, screenshots, generated/, declared path inputs), or `streamBackup` would be a write-as-celilo oracle. Attribution is by the module that PERFORMS the operation: a hook's request to the hook's module (the nine `invokeHook` sites build the policy), a provider's transport to the provider's module — providers run in-process and do not cross this socket, and `public_web`'s hand-built upload was replaced by the Ansible static-content converge (capability-owned-tables stage 4, celilo#1014). Residual, recorded rather than papered over: a hook can still `fetch()` any HTTP endpoint directly; only `probeHttp` consults the target check.
 - **Hook jail (a hook sees the paths it was given, and nothing else)** — `apps/celilo/src/hooks/mount-set.ts` (`deriveMountSet`, `toBwrapArgs`, `forbiddenPaths` — PURE, computes a filesystem view and touches nothing) and `apps/celilo/src/hooks/jail.ts` (`detectJailBackend`, `planJailedSpawn`, `realpathRequest`, `runtimeModulePathsFor`, `recordJailMode` — the half that touches the machine). `executeHookScript` spawns the shim under `bwrap` with the module's tree read-only, `state/` + `generated/` + this run's `screenshots/<run>` read-write on top of it, each contract-declared path input at its declared access, the broker's socket directory, the runtime and the `node_modules` the shim resolves through, and `/tmp` a fresh tmpfs FIRST so it cannot erase the socket or a staged input. `~/.ssh` is deliberately NOT bound (stage 3, D12): withholding the credential is what makes the remote-ops broker's target check a boundary. **The acceptance criterion is absence, not a check**: the module store and the data directory are simply not bound, so `master.key` and a sibling module give `ENOENT`. The set is DERIVED — a module cannot ask for more — and `bwrap` itself is never in it, because the AppArmor profile grants `userns` to `/usr/bin/bwrap` for anyone on the box (design D9). Backend detection RUNS bubblewrap rather than looking for it (four different denials all leave the binary in place), and the resulting mode is written to `hook-jail-mode.json` beside celilo's other per-machine state rather than logged, so a host that stops jailing is readable. The policy is `auto` / `required` / `off`, resolved in a fixed precedence: the `CELILO_HOOK_JAIL` environment variable, then the stored `hooks.jail_policy` system config key (`celilo system config set hooks.jail_policy <value>`, which asks an interview question before writing `off` unless `--force`), then the default, which is **`off`** (ce-rez7). Jailing begins because an operator set a policy, never because an upgrade installed a backend — celilo-mgr's 2026-09-07 move to 1:2.2.1 installed bubblewrap and loaded the AppArmor profile and hooks kept running unjailed, which is the ruling working. `resolveJailPolicy` returns the source alongside the policy and `system doctor` renders it, so an effective value is always locatable. macOS has a `sandbox-exec` backend in the union, but `auto` defers on it (ce-29z: sandbox-exec DENIES undeclared writes where bubblewrap masks them, and D14's declared-path mechanism is not built), so the hook runs unjailed with the mode recorded; `required` bypasses the deferral for an operator who opts in. An `unjailed` record carries `lastJailed` (the jailed record it replaced on the same host), which is what the `hook_jail` self-monitor reads (`services/alerting/hook-jail.ts`, created unsuppressible by `celilo monitor add hook_jail`) to alert on a host that used to jail and has stopped. `celilo system doctor`'s "Hook execution" section (`renderHookExecutionSection`) reports the live mode and, when unjailed, why.
@@ -252,6 +253,7 @@ to look, which is how a forgotten pause actually gets found.
 
 - **Generator** — `apps/celilo/src/templates/generator.ts` — `generateTemplates` (orchestration), plus Terraform/Ansible file handling.
 - **Variable resolution** — `apps/celilo/src/variables/resolver.ts` (parser: `apps/celilo/src/variables/parser.ts`). Supported prefixes: `$self`, `$system`, `$secret`, `$system_secret`, `$capability`, `$infra`.
+- **Where a config value came from (`module_configs.source`)** — the `source` column (migration `0031_module_config_source.sql`, nullable, no backfill — nothing in an existing row proves who wrote it). The variable `source:` enum in `apps/celilo/src/manifest/schema.ts` and `schemas/module-manifest.schema.json` carries **`hook`** alongside `user`/`system`/`capability`/`infrastructure`/`terraform`: the value is written by the owning module's own hook through `context.config.set`, so the configuration interview never prompts for it (`deploy-validation.ts`, `deploy-preflight.ts`, `templates/generator.ts` all skip it the way they skip `infrastructure`) and `celilo module config set` refuses it — `isDerivedVariable` refuses every non-`user` source, and `describeDerivedSource`/`explainNotSettable` name the accessor. An operator and a hook writing the same key would otherwise overwrite each other, which is what `source: user` on four wireguard keys actually did.
 
 ## Secrets
 

package/MODULE_PRIMITIVES.md

@@ -51,6 +51,7 @@ expect(probe(sys, { kind: 'systemd', unit: 'caddy' }, run).healthy).toBe(true);
 | Read a unit's journal | `tailLog` / `grepLog` | `ssh … journalctl` |
 | Run a secret on a command line | `runAppCommandWithSecret` | secret on argv |
 | Anything else with no capability/HTTP/converge path | `runAppCommand` (+ `// escape-hatch:`) | a raw ssh string |
+| Persist state the hook discovered | `context.config.set` / `context.secrets.set` | returning it; `execFileSync('celilo', …)` |
 
 **Reach for a capability method first.** If another module already provides the
 thing (open a port → `firewall.exposeService`; register a route →
@@ -177,6 +178,14 @@ directly — prefer the specific primitive. `opts` is `{ input?, timeoutMs? }`.
 - **No `sleep`** — `waitFor` on the real condition.
 - **Computed config → `applyRenderedConfig`**, not `ssh cat >`. It validates and
   rolls back; a broken config never goes live.
+- **State a hook discovered is WRITTEN, never returned.** `context.config.set`
+  if it is not sensitive, `context.secrets.set` if it is — sensitivity decides
+  the store, and nothing else does. Declare the name first (`source: hook` in
+  `variables.owns`, or `secrets.declares`) or the write throws. A returned
+  record is persisted nowhere, and the hook jail has no `celilo` binary, so
+  shelling out to `celilo module config set` reports success and writes
+  nothing. Details: MODULE_DEVELOPMENT_GUIDE.md, "Persisting state a hook
+  discovers".
 
 See also: `CELILO_SUBSYSTEMS.md` (the primitive impls + the firewall converge),
 `openspec/changes/unified-management-no-ssh/proposal.md` (the design), `reference/MODULE_DEVELOPMENT_GUIDE.md`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@celilo/cli",
-  "version": "5.0.0",
+  "version": "5.0.1",
   "description": "Celilo — home lab orchestration CLI",
   "type": "module",
   "bin": {

package/src/hooks/mount-set.test.ts

@@ -177,6 +177,34 @@ describe('~/.ssh is never in the mount set (stage 3, D12)', () => {
   });
 });
 
+describe('a bound browser without fonts is a browser that cannot render', () => {
+  // celilo#1422. Binding BROWSER_ROOT makes the browser START; it does not make
+  // it RENDER. With no font configuration in the jail, Blink's remote font face
+  // path hits a fatal NOTREACHED on any page that loads a web font and the
+  // renderer dies mid navigation — surfacing as a Playwright selector timeout
+  // that names neither fonts nor the jail. Measured on the management host,
+  // same bwrap invocation, only these rows differing: 12 NOTREACHED without,
+  // 0 with.
+  //
+  // Tied to BROWSER_ROOT deliberately. Fonts are only interesting BECAUSE a
+  // browser is bound, so if the browser bind is ever dropped this test should
+  // be reconsidered with it rather than left asserting an unrelated directory.
+  test('every font directory the renderer needs is bound alongside the browser', () => {
+    const paths = pathsOf(BASE);
+    expect(paths).toContain(BROWSER_ROOT);
+    for (const dir of ['/usr/share/fonts', '/usr/share/fontconfig', '/etc/fonts']) {
+      expect(paths).toContain(dir);
+    }
+  });
+
+  test('fonts are read-only — a hook has no business writing to them', () => {
+    for (const dir of ['/usr/share/fonts', '/usr/share/fontconfig', '/etc/fonts']) {
+      const row = deriveMountSet(BASE).entries.find((e) => e.path === dir);
+      expect(row?.mode).toBe('ro');
+    }
+  });
+});
+
 describe('the fleet browser is reachable, and only read-only (task 4.10)', () => {
   test('BROWSER_ROOT is bound', () => {
     // Task 4.10 names `~/.cache/ms-playwright`. That path is stale:

package/src/hooks/mount-set.ts

@@ -112,6 +112,34 @@ export interface MountSetRequest {
 /** Directories whose contents the runtime needs in order to start at all. */
 const RUNTIME_SUPPORT_DIRS = ['/usr/lib', '/lib', '/lib64', '/etc/ssl'] as const;
 
+/**
+ * What RENDERING needs, as distinct from what running needs.
+ *
+ * Omitting these does not stop the browser starting, which is why it survived
+ * the bind added for `BROWSER_ROOT` below. It kills the RENDERER, and only on a
+ * page that loads a web font. Blink's remote font face path hits a fatal
+ * `NOTREACHED` with no font configuration present, the renderer dies mid
+ * navigation, and Playwright reports `Target page, context or browser has been
+ * closed` while awaiting a selector — which reads as a slow page or a flaky
+ * selector and names neither fonts nor the jail.
+ *
+ * Measured 2026-09-25 on the management host, same bwrap invocation, only these
+ * three rows differing (celilo#1422):
+ *
+ *   without: 12x ERROR ... remote_font_face_source.cc:365] NOTREACHED hit.
+ *   with:     0
+ *
+ * celilo PROVISIONS these deliberately — `celilo-mgmt`'s browser block installs
+ * `fontconfig` and `fonts-dejavu-core` because "a screenshot with no glyphs is
+ * not worth retaining" (D7). So the fleet already pays for fonts and the jail
+ * was hiding them from the one process that needs them: the provisioning and
+ * the mount set disagreed, and the browser lost.
+ *
+ * Read-only, and absent rows are dropped like any other, so a host with no
+ * fonts installed is exactly as it was rather than newly fatal.
+ */
+const FONT_DIRS = ['/usr/share/fonts', '/usr/share/fontconfig', '/etc/fonts'] as const;
+
 /**
  * What resolving a hostname needs. Read-only, and absent ones are dropped.
  *
@@ -220,6 +248,11 @@ export function deriveMountSet(request: MountSetRequest): MountSet {
   for (const dir of RUNTIME_SUPPORT_DIRS) {
     entries.push(entry(dir, 'ro', 'shared libraries and trust store', 'runtime'));
   }
+  for (const dir of FONT_DIRS) {
+    entries.push(
+      entry(dir, 'ro', 'fonts — the renderer dies without them, see FONT_DIRS', 'runtime'),
+    );
+  }
   for (const file of RESOLVER_FILES) {
     entries.push(entry(file, 'ro', 'name resolution — see RESOLVER_FILES', 'runtime'));
   }

package/src/manifest/schema.ts

@@ -860,6 +860,20 @@ export const ModuleManifestSchema = z
         command: z.string().min(1).optional(),
         /** Path to a build script (relative to module directory). Mutually exclusive with command. */
         script: z.string().min(1).optional(),
+        /**
+         * Wall-clock budget for the build, in seconds (celilo#1252).
+         *
+         * The packager used to kill every build at a hardcoded 300s. A module
+         * whose build command declares its own retry budget — forgejo fetches
+         * two release archives from codeberg with `curl --retry` — could
+         * express far more wall clock than that cap allowed, and the retries
+         * written for a flaky upstream never ran. The cap is a property of the
+         * fetch the module performs, so the module names it.
+         *
+         * `packages/e2e/tests/module-build-budget-fits-timeout.test.ts` fails
+         * when a build command's worst-case curl budget exceeds this value.
+         */
+        timeout_seconds: z.number().int().positive().max(7200).default(300),
         artifacts: z
           .array(
             z

package/src/module/packaging/build.ts

@@ -348,12 +348,19 @@ export async function buildModule(options: ModuleBuildOptions): Promise<ModuleBu
       // variable expansion would strip shell-only bash variables like
       // $STAGE before bash ever sees them.
       const buildEnv = { ...process.env, CELILO_MODULE_SOURCE_DIR: sourceDir };
+      // celilo#1252: the budget is the MODULE's, not a constant three layers
+      // away. forgejo's fetch declares `curl --retry 10 --max-time 900` and was
+      // killed at 300s having emitted five of its ten retries, reporting only
+      // "spawnSync bash ETIMEDOUT" — which names neither the cap, the elapsed
+      // time, nor the module.
+      const timeoutMs = (manifest.build.timeout_seconds ?? 300) * 1000;
+      const startedAt = Date.now();
       try {
         if (manifest.build.command) {
           execFileSync('bash', ['-c', manifest.build.command], {
             cwd: buildDir,
             stdio: 'inherit',
-            timeout: 300_000,
+            timeout: timeoutMs,
             env: buildEnv,
           });
         } else {
@@ -362,12 +369,19 @@ export async function buildModule(options: ModuleBuildOptions): Promise<ModuleBu
           execFileSync(cmd, [script], {
             cwd: buildDir,
             stdio: 'inherit',
-            timeout: 300_000,
+            timeout: timeoutMs,
             env: buildEnv,
           });
         }
       } catch (buildError) {
         const msg = buildError instanceof Error ? buildError.message : String(buildError);
+        if (msg.includes('ETIMEDOUT')) {
+          const elapsed = Math.round((Date.now() - startedAt) / 1000);
+          return {
+            success: false,
+            error: `Auto-build failed: ${moduleId}'s build exceeded its ${timeoutMs / 1000}s timeout (killed after ${elapsed}s). Raise build.timeout_seconds in manifest.yml if the build legitimately needs longer, or shrink what it fetches.`,
+          };
+        }
         return { success: false, error: `Auto-build failed: Build exited with code ${msg}` };
       }
 

package/src/registry/client.test.ts

@@ -513,10 +513,11 @@ describe('RegistryClient.download integrity', () => {
   });
 
   test('a non-digest cksum sentinel skips verification instead of failing', async () => {
-    // The registry's bootstrap path publishes `cksum: 'bootstrap'`
-    // (packages/registry-server/src/bootstrap.ts:114) because those modules are
-    // packaged on demand and have no stable digest. Treating a sentinel as a
-    // digest rejects a perfectly good 2.7MB package on every import.
+    // `bootstrap` is the sentinel an index entry carries when it has no
+    // integrity data at all. The registry's bootstrap path no longer emits it
+    // — it hashes the .netapp it packages (celilo#1262) — but the sentinel
+    // branch stays, because treating one as a digest would reject a perfectly
+    // good 2.7MB package on every import.
     fetchSpy.mockImplementation(() => new Response(body, { status: 200 }));
 
     const client = new RegistryClient('https://reg.example.com');