@rizom/ops 0.2.0-alpha.44 → 0.2.0-alpha.440

package/README.md

@@ -2,10 +2,19 @@
 
 Operator CLI package for managing pilot brain fleet registry repos.
 
+## Runtime images
+
+The fleet builds one immutable `brain-${brainVersion}` image per effective Brain version. Every new image includes the union of exact site and theme package pins across the entire fleet, including users still running older Brain versions. Smoke-to-fleet promotion therefore reuses the same image without changing package pins or publishing another Brain release. Conflicting pins fail before build, even across cohorts on different Brain versions.
+
+Before reusing an existing image, Build verifies its actual installed Brain and package versions against its current adopters. Deploy independently verifies the selected instance's requirements before provisioning or replacing containers, including manual deploys. Verification runs only a manifest-reading command in a read-only, network-disabled container on the CI runner; it never starts the Brain or mounts fleet data. Missing packages, mismatched versions, and verification failures stop the workflow. Existing images are not required to serve cohorts that have not adopted them yet.
+
+Explicit builds also load the fleet registry and include its complete package union; `site_packages` adds extra exact pins rather than replacing the union. Package-set changes after an image was published may make that immutable image unsuitable: the gates reject reuse rather than silently rebuilding it. Recovery of an already deployed incomplete image requires a separately reviewed artifact recovery plan.
+
 ## Commands
 
 - `brains-ops init <repo>`
-- `brains-ops render <repo>` — regenerates `views/users.md` and fills status columns from built-in live probes (`DNS`, `/health`, unauthenticated `/mcp`)
+- `brains-ops crossover:stage <source-repo> <output-dir> <bundle-review.yaml> [site-pins.yaml]` — creates a separate secret-free capability-bundle review copy without mutating the source; the bundle review must match every explicit source selection, and a reviewed pins file is required when hosted site overrides exist
+- `brains-ops render <repo>` — regenerates `views/users.md` and fills status columns from built-in live probes (`DNS`, `/health/ready`, unauthenticated `/mcp`)
 - `brains-ops user:add <repo> <handle> --cohort <cohort>` — scaffolds a user file, per-user secrets template, and cohort membership
 - `brains-ops onboard <repo> <handle>` — creates/seeds the user's content repo using `CONTENT_REPO_ADMIN_TOKEN` for GitHub repo administration and `GIT_SYNC_TOKEN` for git clone/push
 - `brains-ops age-key:bootstrap <repo>`
@@ -13,13 +22,85 @@ Operator CLI package for managing pilot brain fleet registry repos.
 - `brains-ops cert:bootstrap <repo>`
 - `brains-ops secrets:push <repo>`
 - `brains-ops secrets:encrypt <repo> <handle>`
+- `brains-ops verify-user <repo> <handle>` — checks `/health/operate`, unauthenticated `/mcp`, and site-enabled browser/Studio routes
+- `brains-ops recover:retire-legacy-projection-job <database> <job-id> --type <legacy-type> --dry-run` — verifies that one exact pre-scheduler projection job has no owner or partial progress; replace `--dry-run` with `--confirm retire:<job-id>` only after reviewing the result
+- `brains-ops stress:directory-sync <repo> <handle> --profile <regression|load|stress> --confirm stress:<handle>` — runs a smoke-only, reversible directory-sync workload and writes structured evidence
+- `brains-ops stress:directory-sync:verify-access <repo> <handle> --confirm stress:<handle>` — clones the smoke content repository and verifies push authorization with `git push --dry-run`, without creating a ref or starting a workload
+- `brains-ops stress:directory-sync:cleanup <repo> <handle> --confirm stress:<handle>` — idempotently removes residual stress probes
+- `brains-ops smoke:health-watchdog <repo> <handle> --confirm watchdog-smoke:<handle> --run-id <id>` — verifies fleet watchdog selection and restart-budget isolation on the smoke host
+- `brains-ops smoke:health-watchdog:cleanup <repo> <handle> --confirm watchdog-smoke:<handle> --run-id <id>` — removes deterministic residual watchdog smoke fixtures
 - `brains-ops reconcile-cohort <repo> <cohort>`
 - `brains-ops reconcile-all <repo>`
+- `brains-ops reconcile-all <repo> --dry-run` — reconciles an isolated copy twice with external content-repository access blocked, lists both passes' changed files, and requires second-pass zero drift
+
+`render` owns the observational `views/users.md` projection. Onboard and reconcile commands own generated per-user config and never rewrite live observed status in that view.
+
+### Per-user plugin configuration
+
+Use canonical plugin configuration in `users/<handle>.yaml`, not edits to generated `brain.yaml`:
+
+```yaml
+plugins:
+  dashboard:
+    ask: true
+```
+
+The generator composes an object before serializing once. Explicit per-user fields override generated defaults; nested maps merge, arrays replace, and `false` is preserved. Null deletion markers remain in the generated document until the app resolves runtime defaults. Both stages use the same shared merger. The operator validates the configuration map shape; individual plugin schemas remain authoritative in the app.
+
+The example enables only the dashboard Ask tab. It does not activate guest access, change allowances, or supply authored content. Users without `plugins` retain the same configuration values. New or changed configuration uses the standard YAML formatter. Reconciliation compares YAML values before writing an existing `brain.yaml`, so formatting-only differences are left untouched and cannot schedule unrelated deployments. Malformed or substantively changed configuration is regenerated.
+
+## Directory-sync stress profiles
+
+Directory-sync stress is deliberately smoke-only. The handle, domain, and content repository must each identify smoke, and the operator must pass the exact `stress:<handle>` confirmation. The runner refuses production-like targets. The smoke user must also declare the hermetic posture below; reconcile and deploy it before running the workload:
+
+```yaml
+embeddingEnabled: false
+topicExtractionEnabled: false
+skillDerivationEnabled: false
+swotDerivationEnabled: false
+```
+
+- `regression`: add 20 files, update all 20, then delete them;
+- `load`: ramp through 50, 150, and 350 files, update 350, rename 100, update again, then delete all probes;
+- `stress`: continue the same deterministic ramp to 700 files and 200 renames.
+
+Each run creates a rollback branch, gates on health failures during the monitored workload window, preserves warmup and cleanup health samples as evidence without poisoning that gate, samples container CPU/memory/PIDs, detects watchdog restarts even when Docker's restart counter remains zero, rejects external AI usage, waits for Git and entity persistence, and always attempts cleanup. JSON, Markdown, runtime logs, and samples are written under `.brains-ops/stress/` unless `--artifacts-dir` is supplied. The scaffolded `Directory Sync Stress` workflow runs a weekly regression and supports manual profiles. Its manual `verify_only` mode exercises the Bitwarden/Varlock content credential path through clone and a dry-run push, skips the workload, and does not schedule cleanup. Workload runs retain a separate `always()` cleanup job. Successful idempotent cleanup also prunes retained stress backup branches; branches remain available when probes remain for recovery. The workflow never deploys or targets a non-smoke user.
+
+## Health watchdog smoke
+
+The scaffolded manual `Health Watchdog Smoke` workflow uses the same pilot desired state, Hetzner lookup, Bitwarden/Varlock deploy key, and `deploy-<handle>` concurrency boundary as fleet deployment. It refuses non-smoke handles/domains and requires `watchdog-smoke:<handle>` confirmation. The runner requires the installed watchdog to match the packaged canonical payload, exercises that installed script, and verifies the deployed image label, active timer, exact selector, liveness healthcheck, diagnostics, negative container isolation, and three-per-hour restart budget without restarting the deployed rover. Evidence is uploaded from `RUNNER_TEMP`; an independent `always()` cleanup job removes deterministic temporary containers and remote files.
 
 ## Scope
 
 `brains-ops` lives in the `brains` monorepo and is consumed as a separate package.
 
+The active loader accepts one schema requiring `bundleContract: capability-bundles-v1` and the nine capability/policy bundle IDs. Offline parsing of prior desired state is isolated to `crossover:stage` and is never used by render, verify, or reconciliation paths.
+
+Crossover staging excludes source `.git`, `.operator`, `.brains-ops`, `.turbo`, `dist`, `node_modules`, plaintext `.env`/`.env.local`, and `*.secrets.yaml` artifacts. It generates fresh per-user `.env` selector files in the review copy without copying source secret values. A review manifest must name both sides of the pilot selection and every explicit cohort selection; source drift fails staging:
+
+```yaml
+bundleContract: capability-bundles-v1
+pilot:
+  sourceBundles: [core]
+  targetBundles: [core, media, web, chat]
+cohorts:
+  sites:
+    sourceBundles: [core, site, publishing]
+    targetBundles:
+      [core, media, automation, web, chat, site, publishing, federation]
+```
+
+Hosted sites must also be enumerated in a separate reviewed file; staging rejects missing, extra, or identity-mismatched pins instead of inferring versions:
+
+```yaml
+sites:
+  example:
+    package: "@rizom/site-example"
+    version: 0.2.0-alpha.1
+    theme: "@rizom/theme-example"
+    themeVersion: 0.2.0-alpha.1
+```
+
 It operates on a separate private data repo, such as `rover-pilot/`, which stores:
 
 - `pilot.yaml`

package/dist/age-key-bootstrap.d.ts

@@ -1,4 +1,4 @@
-import { type RunCommand } from "./run-subprocess";
+import { type RunCommand } from "@brains/deploy-support/run-subprocess";
 export interface AgeKeyBootstrapOptions {
     logger?: (message: string) => void;
     pushTo?: string | undefined;