create-op-node 0.0.1 → 0.2.0

package/README.md

@@ -24,13 +24,261 @@ npx create-op-node bootstrap
 
 Configures macOS power settings, installs Homebrew + the CLI tool list, sets up Docker Desktop + Tailscale + Ollama, clones the node repo you created, materializes the pgsodium key from 1Password, writes the LaunchAgent plist, logs into ghcr.io, pulls + warms the LLM model, and finally `docker compose pull && up -d` brings the whole stack online. Health-check loop waits until all 10 containers are `(healthy)`.
 
-And at any time after that:
+## Resetting the Studio
+
+To start over (e.g. before rerunning `bootstrap` against a different
+region, or after a misconfiguration), reverse the Studio-side state:
+
+```bash
+npx create-op-node reset --region us-ca
+```
+
+Three phases run in reverse-bootstrap order. By default volumes are
+preserved — the database survives so you can bring the stack back up
+with `bootstrap` without losing data.
+
+1. **Stop the stack** — `docker compose down`. Pass `--wipe-data` to
+   add `-v` (destroys named volumes including the database). The wipe
+   mode requires retyping the region label as confirmation — and the
+   prompt deliberately doesn't pre-fill the answer, so you have to type
+   it from memory. `y` won't do it.
+2. **Unload + remove the LaunchAgent** — `launchctl unload` then `rm`
+   the plist and the pgsodium key file. `--keep-key-file` leaves the
+   key in place as a belt-and-suspenders backup before a wipe-data run.
+3. **`docker logout`** — clears the registry-credentials store entry
+   for `ghcr.io` (or override with `--registry`). This only clears the
+   store entry; if your credential helper caches the token elsewhere
+   (or `~/.docker/config.json` has stale entries from another host),
+   those need separate cleanup.
+
+Reset does **not** touch cloud-side state: the Cloudflare resources,
+the GitHub repo, the TFC workspace, and the 1Password items remain.
+`init` is idempotent against existing cloud setup, so re-running it
+won't duplicate anything.
+
+Useful flags:
+
+- `--dry-run` — print the plan without acting. Phases that would run
+  show with a `?` icon; phases that are skipped show with `·`.
+- `--skip-stack` / `--skip-launch-agent` / `--skip-docker-logout` —
+  surgical resets when only one piece needs cleaning.
+- `--no-remove-orphans` — drop `--remove-orphans` from `compose down`.
+  Useful when you ran bootstrap with a custom `--compose-file` set and
+  reset without it.
+- `--repo-dir <path>` — explicit path to the cloned node repo when
+  reset is run from outside the checkout. Passing a path that doesn't
+  look like a node repo is a hard error, not a silent skip.
+- `--registry <reg>` — log out of a registry other than `ghcr.io`.
+
+```bash
+# Try-before-you-buy: preview every step.
+npx create-op-node reset --region us-ca --dry-run
+
+# Nuke from orbit: containers + volumes + LaunchAgent + ghcr credentials.
+npx create-op-node reset --region us-ca --wipe-data
+```
+
+## Verifying a live node
 
 ```bash
 npx create-op-node verify --domain your-domain.example
 ```
 
-Off-LAN health probe of a live node — TLS, GraphQL reachability, cosign signature check on the running images.
+Off-LAN health probe of a live node, runnable from anywhere with internet
+access. Five phases:
+
+1. **TLS handshake** to `api.<domain>:443` — surfaces cert subject, issuer,
+   and days-to-expiry. Warns when the cert is within `--cert-warn-days`
+   of expiring (default 14d). Negative expiries render as
+   `expired Nd ago`.
+2. **`GET https://api.<domain>/health`** must return 200.
+3. **`POST https://api.<domain>/api`** with `{ __typename }` must return a
+   valid GraphQL envelope (catches the "TLS green, but a misconfigured
+   proxy returns HTML" case).
+4. **Cloudflare Tunnel status** (optional) — looks up `connections` via
+   the CF API. Zero connectors registered → warning that cloudflared on
+   the Studio is offline. Requires all three of `--cf-token` (or
+   `--cf-token-file`), `--cf-account-id`, `--tunnel-id`; partial
+   configuration warns + names the missing flag.
+5. **`cosign verify`** (optional, repeatable `--image`) — keyless
+   verification against the GitHub Actions OIDC issuer + Fulcio +
+   the Rekor transparency log. Silently skipped when `cosign` isn't on
+   `PATH` (install with `brew install cosign` to enable).
+
+No phase short-circuits the others — verify always runs the full pass so
+the operator sees the whole landscape in one report. Exits non-zero only
+when at least one phase failed; warnings are reported but don't fail the
+run. Skipped phases are hidden by default; add `--show-skipped` to see them.
+
+Full flag set:
+
+```bash
+npx create-op-node verify \
+  --domain yournode.example.org \
+  --cf-token-file ~/.config/opuspopuli/cf-token \
+  --cf-account-id $CF_ACCOUNT_ID \
+  --tunnel-id $TUNNEL_ID \
+  --image ghcr.io/opuspopuli/api:latest \
+  --image ghcr.io/opuspopuli/users:latest \
+  --cert-warn-days 21
+```
+
+`--cf-token-file` is preferred over `--cf-token` for cron / systemd
+invocations — the latter ends up in `ps` output, the former doesn't.
+Use `--api-host <host>` to override the default `api.<domain>`
+construction when your node exposes the API at a different subdomain.
+
+## Bootstrapping a region config
+
+A node serves data; **what** data it serves is defined by a declarative region
+config in [`OpusPopuli/opuspopuli-regions`](https://github.com/OpusPopuli/opuspopuli-regions).
+Hand-writing one of those JSON files against the schema is the same kind of
+fiddly, error-prone step the rest of this CLI exists to remove — so there's a
+subcommand for it. Run it from the root of your `opuspopuli-regions` checkout:
+
+```bash
+npx create-op-node region
+```
+
+The wizard walks you through level (state or county), names, the two-letter
+state code, FIPS code, timezone, and at least one data source (URL, data type,
+source type, content goal). It then:
+
+1. Derives the `regionId` and keeps `name === config.regionId` (the invariant
+   the regions repo enforces).
+2. Validates the generated file against the **vendored copy of
+   `region-plugin.schema.json`** (the canonical contract) using an ESM-native
+   JSON Schema validator, then layers on the cross-field rules the repo's
+   `pnpm test` adds in code: semver shape, FIPS length per level
+   (2 digits for state, 5 for county), county-id-prefixed-by-parent, no
+   duplicate data sources keyed by `(dataType, url)`. All checks run **before**
+   writing, so the file lands green instead of bouncing off CI.
+3. Writes it to the canonical path
+   (`regions/<state>/<state>.json` or
+   `regions/<state>/counties/<county>/<county>.json`).
+
+> **Conventions you may not expect**
+>
+> - New configs are stamped at **version `0.1.0`** — the documented starting
+>   point in `opuspopuli-regions/CLAUDE.md`. Bump manually as the config
+>   matures (additions → minor, breaking changes → major).
+> - `boundarySources` is **not** prompted for. It's optional per the schema, so
+>   the scaffolded file is valid without it — but if your region has TIGER /
+>   ArcGIS boundary coverage and you want PostGIS point-in-polygon district
+>   lookups, you'll need to add the block by hand after scaffolding (see
+>   `regions/california/california.json` for a worked example).
+> - `civics_blocks` is **not** part of the region config schema. It's a
+>   per-region taxonomy that lives elsewhere in the platform — don't look for
+>   a prompt for it here.
+
+Then it's just `pnpm test` + a PR. Non-interactive flags (`--level`, `--name`,
+`--parent`, `--state-code`, `--fips`, `--timezone`, `--out-dir`, `--force`) are
+available for scripting; run `create-op-node region --help` for the list.
+
+### What it looks like
+
+```text
+┌  create-op-node region
+
+◇  What level is this region?
+│  County
+
+◇  County name?
+│  Alameda
+
+◇  Parent state slug?
+│  california
+
+◇  Display name?
+│  Alameda County
+
+◇  One-line description of the data coverage?
+│  Civic data for Alameda County, California
+
+◇  Two-letter state code?
+│  CA
+
+◇  County FIPS (5 digits)?
+│  06001
+
+◇  IANA timezone?
+│  America/Los_Angeles
+
+◇  Data source #1 — URL?
+│  https://bos.acgov.org/
+
+◇  Data type?
+│  meetings
+
+◇  Source type?
+│  html_scrape
+
+◇  Content goal (what should the scraper extract)?
+│  Fetch Board of Supervisors agendas, minutes, and votes
+
+◇  Category label (optional)?
+│  Board of Supervisors
+
+◇  Add another data source?
+│  No
+
+◆  regions/california/counties/alameda/alameda.json (preview) ──────────╮
+│  {                                                                     │
+│    "name": "california-alameda",                                       │
+│    "displayName": "Alameda County",                                    │
+│    "description": "Civic data for Alameda County, California",         │
+│    "version": "0.1.0",                                                 │
+│    "config": {                                                         │
+│      "regionId": "california-alameda",                                 │
+│      "regionName": "Alameda County",                                   │
+│      "description": "Civic data for Alameda County, California",       │
+│      "timezone": "America/Los_Angeles",                                │
+│      "stateCode": "CA",                                                │
+│      "fipsCode": "06001",                                              │
+│      "dataSources": [                                                  │
+│        {                                                               │
+│          "url": "https://bos.acgov.org/",                              │
+│          "dataType": "meetings",                                       │
+│          "sourceType": "html_scrape",                                  │
+│          "contentGoal": "Fetch Board of Supervisors agendas, ...",     │
+│          "category": "Board of Supervisors"                            │
+│        }                                                               │
+│      ]                                                                 │
+│    },                                                                  │
+│    "parentRegionId": "california"                                      │
+│  }                                                                     │
+├────────────────────────────────────────────────────────────────────────╯
+
+◇  Write regions/california/counties/alameda/alameda.json?
+│  Yes
+
+◆  Done ────────────────────────────────────────────────────╮
+│  ✓ Wrote regions/california/counties/alameda/alameda.json  │
+│                                                            │
+│  Next steps in your opuspopuli-regions checkout:           │
+│    pnpm test                 # schema + hierarchy          │
+│    pnpm test:connectivity    # URL reachability            │
+│    git add … && git commit && open a PR                    │
+├──────────────────────────────────────────────────────────────╯
+
+└  Region scaffolded: california-alameda
+```
+
+### Caveats
+
+A couple of honest sharp edges, since this command lives in the *node* CLI
+rather than in the regions repo itself:
+
+- **Run it from a `opuspopuli-regions` checkout.** The file is written relative
+  to `--out-dir` (default: current directory) at the canonical `regions/…`
+  path. Run it anywhere else and the file lands in the wrong tree.
+- **The validation here mirrors the regions schema; it does not import it.**
+  `create-op-node` can't see `region-plugin.schema.json` at runtime, so its
+  pre-write checks are a hand-maintained copy of the rules. They can drift if
+  the schema changes. The regions repo's own `pnpm test` is the source of
+  truth — **always run it after scaffolding**; treat a green run there, not a
+  green run here, as the real signal. If the two ever disagree, the schema
+  wins and this command needs updating.
 
 ## Why this exists
 
@@ -48,8 +296,9 @@ The CLI itself never holds any credentials beyond the scope of a single command
                        │                                      │
    npx create-op-node ─┤  ┌─ init  ──────► Cloudflare API     │
                        │  ├─ bootstrap     GitHub API         │
-                       │  └─ verify        Terraform Cloud    │
-                       │                   `op` CLI (optional)│
+                       │  ├─ verify        Terraform Cloud    │
+                       │  └─ region        `op` CLI (optional)│
+                       │     (writes a regions repo config)   │
                        └──────────────────────────────────────┘
                                             │
                                             ▼
@@ -73,12 +322,18 @@ The CLI itself never holds any credentials beyond the scope of a single command
 
 ## Status
 
-**v0.0.1 — scaffold only.** The 3 subcommands exist with type-safe argument parsing and the Cloudflare 5-scope probe. The interactive prompts collect inputs but stop before doing destructive work. Iterating in the open against the OpusPopuli/opuspopuli-node template as it stabilizes.
+**`init` — fully wired.** Full Phase 1 of the runbook: prompts → Cloudflare 5-scope probe → Terraform Cloud verify → GitHub template clone → 5 repo secrets seeded → branch + prod.tfvars committed → PR opened → pgsodium key generated → (after operator merges PR) Terraform apply polled → Tunnel token retrieved + saved to 1Password.
+
+**`bootstrap` — fully wired.** Phase 2 on the Mac Studio: macOS sanity (auto-restart, disk sleep), Homebrew + tool installs (gh, pnpm, jq, cloudflared, rclone, ollama, docker, tailscale), GitHub + Tailscale signin prompts, pgsodium key + Tunnel token read from 1Password, LaunchAgent written + loaded, ghcr.io login, Ollama models pulled + warmed, region repo located or cloned, `docker compose pull && up -d`, health-check loop until everything reports `(healthy)`.
+
+**`verify` — scaffold stub.** Type-safe argument parsing only; prints a roadmap-style message and exits.
+
+**`region`** — fully wired. Scaffolds schema-valid region configs for the `OpusPopuli/opuspopuli-regions` repo.
 
 ### Roadmap
 
-- **v0.1.0** — `init` fully wired: GitHub template clone, secret seeding, prod.tfvars generation, pgsodium key flow, first PR, Tunnel token retrieval.
-- **v0.2.0** — `bootstrap` fully wired: macOS config, brew installs, Docker Desktop setup, Tailscale, repo clone, LaunchAgent, ghcr.io login, Ollama, `docker compose up`.
+- **v0.1.0** ✅ `init` end-to-end + `region` scaffolder.
+- **v0.2.0** ✅ `bootstrap` fully wired on the Studio side.
 - **v0.3.0** — `verify` fully wired: TLS + GraphQL + cosign signature checks.
 - **v0.4.0** — Resend domain + DKIM automation, drift detection, automated backup-restore drill.
 
@@ -117,5 +372,6 @@ node dist/cli.js --help     # test the built binary
 ## Related
 
 - [`OpusPopuli/opuspopuli-node`](https://github.com/OpusPopuli/opuspopuli-node) — the per-region deployment template this CLI creates from.
+- [`OpusPopuli/opuspopuli-regions`](https://github.com/OpusPopuli/opuspopuli-regions) — declarative region configs; `create-op-node region` scaffolds one.
 - [`OpusPopuli/opuspopuli`](https://github.com/OpusPopuli/opuspopuli) — the central monorepo that builds + publishes `ghcr.io/opuspopuli/*` images.
 - [`OpusPopuli/prompt-service`](https://github.com/OpusPopuli/prompt-service) — private prompt-template service consumed by every node.