rigscope 0.0.0-stage → 0.1.0

package/docs/operations.md

@@ -0,0 +1,515 @@
+# Production operations
+
+This guide covers the supported single-host, two-process deployment:
+
+- `dist/hub.cjs` — Node 26 custom server: listener, WebSocket collectors,
+  browser SSE, authentication and the production Next.js handler from `.next`.
+- `dist/collector.cjs` — outbound-only collector with durable per-destination
+  outboxes.
+- `dist/ops.cjs` — inspect, migrate, backup, restore, package, install,
+  activate, rollback and bounded synthetic measurement.
+
+There is no Next.js standalone output and no `next start`. The hub runs the
+custom server from the installed release directory; the production UI must be
+present in `.next`.
+
+All paths below are examples. Choose a prefix and state directory owned by the
+service user. Paths containing spaces are supported by the generated units.
+
+## Managed CLI operations
+
+For a per-user installation created by `rigscope install` (default state
+`~/.local/state/rigscope`), the CLI owns the runtime, component units and release pairing:
+
+```sh
+rigscope status
+rigscope doctor [--json]
+rigscope upgrade [--from PACKAGE] [--timeout SECONDS]
+rigscope rollback [--backup PATH] [--timeout SECONDS]
+rigscope recover-password [--user owner] [--password-file PATH] [--enable]
+```
+
+`doctor` is read-only. It checks Node, private configuration and state
+permissions, the installed and previous runtimes, the component units, the database
+and schema, free space, hub port/Unix-socket state, role-appropriate listener
+health, configured destinations with TLS certificate validation, and procfs or
+RAPL readability for a collector. A hub without a collector is valid, and a
+collector without a local hub database is valid. Output contains counts and
+paths, never password hashes, collector tokens or environment values.
+
+`upgrade` stages the package without touching the active runtime, stops the
+running components, takes a consistent pre-upgrade hub database backup when one
+exists, and activates the new runtime. It restarts only components that were
+running before the upgrade and verifies them within `--timeout` seconds (default
+25). Verification follows the installation's enabled components:
+
+- the active package reports the staged version;
+- restarted component services are active;
+- a restarted hub answers `status: "ok"` with the expected version;
+- an existing hub database passes integrity and ledger checks and reaches the
+  staged runtime's supported schema;
+- a restarted collector produces a fresh sampling health marker.
+
+A failed start, migration or health wait restores the previous runtime and the
+paired pre-upgrade database when the old runtime cannot read the new schema. The
+replaced database is preserved first; credentials, collector identities and
+outboxes are not rewritten. `release-state.json` in the state directory records
+which runtime each backup belongs to. An inactive installation is staged and
+activated without starting; live health is then reported as skipped.
+
+`rollback` swaps to `runtime-previous` and validates it with the same bounded
+health and database checks. If the current schema is compatible it keeps the
+database unchanged, including post-upgrade data. If the previous runtime cannot
+read it, rollback restores the paired pre-upgrade backup, refuses when no
+compatible backup is recorded, and prints where the replaced newer database was
+preserved. For installations created before release state existed, pass an
+explicit compatible copy with `--backup PATH`. Rollback never silently deletes
+the newer database and never runs old code against a newer schema.
+
+`recover-password` is offline recovery for the durable managed user store. It
+requires the hub service to be stopped, takes the hub database lease and
+refuses a live or ambiguous lock. Enter the password at the private prompt, or
+pass a private `0600` file with `--password-file`. It updates only that managed
+user's hash in the database, clears `disabled` only with `--enable`, writes a
+bounded `user.password-recovery` audit row and bumps a durable session-epoch
+marker. It never edits `config.json` or `HUB_BROWSER_USERS`, so bootstrap
+password changes do not overwrite managed users. Restarting the hub invalidates
+all in-memory browser sessions; sign in again with the new password.
+
+## Low-level manifest archive workflow
+
+The remaining archive deployment examples assume a source checkout with built
+artifacts. They use a manifest `.tar.gz`, which is different from the npm `.tgz`
+used by `rigscope install` and `rigscope upgrade`. For ordinary installations,
+follow [installation](installation.md) and the managed commands above. The npm
+package excludes source helper scripts and does not require users to build it.
+
+### Build and package
+
+```sh
+npm ci
+npm run build:ui          # production .next
+npm run build:runtime     # dist/hub.cjs, dist/collector.cjs, dist/ops.cjs
+node scripts/package-release.cjs --output /var/tmp/rigscope-release.tar.gz
+```
+
+The archive contains the built runtime, `.next` without its cache, package and
+lock files, the repository's `next.config.js`, `tsconfig.json`,
+operational docs and the `packaging/` examples. It deliberately excludes source, tests, git data,
+`node_modules`, private environment files, SQLite databases, keys and logs.
+`manifest.json` lists every packaged file with size and SHA-256; the installer
+re-verifies it before activation.
+
+Production dependencies are installed inside each staged release:
+
+```sh
+cd /home/operator/.local/opt/rigscope/releases/<version>
+npm ci --omit=dev
+```
+
+`install --install-deps` can run that command explicitly, but the installer
+never runs npm implicitly for activation. If a package mirror is unavailable,
+stage without activation and install from an approved mirror; do not weaken npm
+security policy just to make installation pass. Isolated tests use a synthetic
+dependency tree and do not perform remote installs.
+
+## Stage and activate
+
+```sh
+node scripts/install-release.cjs \
+  --source /var/tmp/rigscope-release.tar.gz \
+  --prefix /home/operator/.local/opt/rigscope \
+  --state-dir /home/operator/.local/state/rigscope \
+  --units-dir /home/operator/.config/systemd/user
+```
+
+The installer creates:
+
+```text
+<prefix>/
+  releases/<version>/          immutable built release + manifest
+  releases/<version>/node_modules/
+  current -> releases/<version>
+  previous -> releases/<previous>   (retained for rollback)
+<state>/
+  hub/telemetry.sqlite          hub database (0700 dir, 0600 file)
+  hub.env                       private EnvironmentFile (0600, no secrets printed)
+  collector.env                 private EnvironmentFile (0600)
+  collector/collector.json      collector identity/enrollment (created by collector)
+  outbox/<sha256(url)[0:16]>/   per-destination collector outboxes
+  backups/                      private pre-upgrade/pre-activation database copies
+  install-state.json            versions, schema backups, no credentials
+  run/                          private runtime directory for a Unix socket
+<units-dir>/
+  rigscope-hub.service
+  rigscope-collector.service
+```
+
+Staging does not switch `current` or start anything. After production
+dependencies are installed and the old hub is stopped, activate explicitly:
+
+```sh
+node scripts/install-release.cjs activate \
+  --prefix /home/operator/.local/opt/rigscope \
+  --state-dir /home/operator/.local/state/rigscope \
+  --version <version> --confirm-stopped
+```
+
+Activation order is fixed:
+
+1. Validate the release manifest, hashes and production dependencies.
+2. Verify no hub lock exists and the operator confirmed the hub is stopped.
+3. Inspect the current database schema.
+4. If migration is needed: write a consistent online backup, then migrate, then
+   verify integrity and the target schema. A failed migration restores the
+   pre-migration backup and leaves `current` unchanged.
+5. Atomically switch `current`, retaining `previous` and a schema-backup record.
+6. Print (never execute) systemd commands.
+
+If validation or migration fails, a good database is never silently replaced.
+The pre-migration backup path is printed. First installation with no database
+creates and migrates a new private database.
+
+## systemd user units
+
+The generated units use an absolute Node 26 executable, absolute paths,
+`WorkingDirectory=<prefix>/current`, a private `EnvironmentFile`, journald
+logging, `Restart=on-failure`, `NoNewPrivileges=yes`, `PrivateTmp=yes`,
+`UMask=0077`, `ProtectSystem=strict`, and explicit writable state/database/cache paths.
+
+Startup is explicit and is not performed by the installer:
+
+```sh
+systemctl --user daemon-reload
+systemctl --user enable --now rigscope-hub.service rigscope-collector.service
+```
+
+Automatic startup after logout/host reboot requires an explicit administrator
+choice to enable user lingering for that account:
+
+```sh
+sudo loginctl enable-linger operator
+```
+
+Review the generated units and private env files first. Health and observation:
+
+```sh
+curl --fail --silent http://127.0.0.1:8081/api/health
+systemctl --user status rigscope-hub.service rigscope-collector.service
+journalctl --user -u rigscope-hub.service -f
+journalctl --user -u rigscope-collector.service -f
+```
+
+Shutdown is bounded: the hub targets five seconds and the collector three
+seconds; the generated `TimeoutStopSec` leaves a ten- and seven-second slack
+respectively. Keep those times at or above the process deadlines.
+
+## Configuration
+
+`RIGSCOPE_OPS_DEBUG` enables operation diagnostics; keep diagnostic output private. `RIGSCOPE_HUB_LOCK_FILE` overrides the hub lease path when needed; use the same path for the hub and operations commands.
+
+The generated `hub.env` and `collector.env` contain only comments. Empty
+configuration fails closed: no `HUB_BROWSER_USERS` means browser data is
+unavailable, and no collector credentials/destinations means no fleet data.
+Never copy the public demo credentials from the examples to a real host.
+
+Loopback development:
+
+```sh
+HUB_HOST=127.0.0.1
+HUB_PORT=8081
+HUB_DATABASE_PATH=/home/operator/.local/state/rigscope/hub/telemetry.sqlite
+```
+
+Browser users and collectors belong in the private environment file. Generate
+browser password hashes locally with `scripts/hash-browser-password.cjs`; never
+store plaintext passwords or hashes in shared logs.
+
+Remote mode uses a private Unix socket behind an external TLS terminator:
+
+```sh
+HUB_PUBLIC_ORIGIN=https://monitor.example.invalid
+HUB_SOCKET_PATH=/home/operator/.local/state/rigscope/run/hub.sock
+```
+
+The parent directory must be private and owned by the service user. The
+installer's state `run/` directory is already writable by the unit. Remote
+health checks go through the socket with the exact Host:
+
+```sh
+curl --fail --silent --unix-socket <socket> \
+  -H 'Host: monitor.example.invalid' http://localhost/api/health
+```
+
+This example hostname and every `demo-token-*` value are public fixtures, not a
+deployment. See `development.md` for the exact origin/Host/cookie
+boundary. Never expose the loopback HTTP listener remotely without the private
+socket/TLS boundary and authentication.
+
+Collector enrollment remains separate from browser sessions:
+
+```sh
+COLLECTOR_ENROLL_URL=ws://127.0.0.1:8081/api/collector/connect
+COLLECTOR_ENROLL_TOKEN=<one-use grant>
+COLLECTOR_MODE=real
+COLLECTOR_STATE_FILE=/home/operator/.local/state/rigscope/collector/collector.json
+```
+
+Remove the one-use grant after enrollment and start or restart the collector. The
+hub does not need a restart. Enrollment state
+and outboxes live outside the repository.
+
+For an isolated local check without touching the real deployment, the public demo
+grant from the packaging examples can drive a loopback enrollment and collection
+run. These are deliberate demo strings only, never a real deployment:
+
+```sh
+HUB_STATE_FILE="$PWD/.rigscope-local/hub/enrollment.json" \
+HUB_ENROLLMENT_GRANTS="$(node -e 'console.log(JSON.stringify([{token:"demo-token-enrollment-local",expiresAt:new Date(Date.now()+300000).toISOString()}]))')" \
+npm start
+
+# In another terminal:
+COLLECTOR_STATE_FILE="$PWD/.rigscope-local/collector/state.json" \
+COLLECTOR_ENROLL_URL='ws://127.0.0.1:8081/api/collector/connect' \
+COLLECTOR_ENROLL_TOKEN='demo-token-enrollment-local' \
+npm run collector:enroll
+
+# Start real local collection with the saved credential:
+COLLECTOR_STATE_FILE="$PWD/.rigscope-local/collector/state.json" npm run collector:start
+```
+
+Enrollment posts `{collectorId}` to `/api/collector/enroll`; the bearer grant travels
+only in the Authorization header. Grants must expire within fifteen minutes and are
+consumed across restarts. Plaintext is allowed only for literal loopback addresses;
+WSS enrollment uses HTTPS with ordinary certificate validation.
+
+## Database commands
+
+All commands take explicit paths. They print no credentials, user hashes,
+webhook URLs, tokens or record payloads.
+
+```sh
+# Read-only schema/integrity/size/table summary.
+node dist/ops.cjs inspect --database /path/telemetry.sqlite
+
+# Consistent single-file online backup. Does NOT migrate the source.
+node dist/ops.cjs backup \
+  --database /path/telemetry.sqlite \
+  --output /path/private-backups/telemetry-$(date +%Y%m%dT%H%M%S).sqlite
+
+# Offline guarded migration. Existing databases require --confirm-stopped.
+node dist/ops.cjs migrate \
+  --database /path/telemetry.sqlite \
+  --backup-dir /path/private-backups --confirm-stopped
+
+# Guarded restore over an existing database: both flags are required.
+node dist/ops.cjs restore \
+  --input /path/private-backups/telemetry-....sqlite \
+  --database /path/telemetry.sqlite \
+  --replace --confirm-stopped
+
+# Offline managed-user recovery; password comes from a private 0600 file.
+node dist/ops.cjs recover-admin \
+  --database /path/telemetry.sqlite \
+  --user owner --password-file /path/private-recovery-password --enable
+```
+
+Safety rules enforced by `backup`/`restore`:
+
+- The hub database is one SQLite file. The online backup copies all tables in
+  that file: telemetry, history rollups, receipts/gaps, machine latest and
+  inventory, the collector registry/grants/audit, admin users/audit, and alert
+  rules/incidents/jobs/webhook state. There is no partial table selection.
+- Backup opens the source read-only, runs `integrity_check`, stages the copy,
+  converts it to a self-contained DELETE-journal file, verifies the staged copy,
+  sets mode 0600 and atomically renames it. The source schema is never migrated.
+- Restore validates the input integrity and migration ledger, refuses a schema
+  newer than this operations build, refuses to overwrite the target without
+  `--replace --confirm-stopped`, and refuses if a hub lock is present.
+- It stages and verifies a copy first. For an existing target it checkpoints the
+  old database, takes a pre-restore backup, then removes only the old WAL/SHM
+  sidecars after proof that the hub is stopped, and atomically replaces the
+  main file. A failed validation never overwrites the target.
+- The hub process lease is `<databasePath>.hub.lock` (JSON `{pid,instance}`).
+  Operations refuses while it exists, even if the PID looks stale, because only
+  the hub's own startup proof may recover a proven-dead lease. The hub removes
+  it on clean shutdown/exit; operators must inspect a lock and never delete it
+  blindly.
+- Restoring an older-schema backup is allowed; the matching release migrates it
+  on the next hub start. Never start an old release against a newer schema.
+
+Schema versions are derived from the build's migration ledger. The current
+ledger includes telemetry, latest-session identity, admin, alerts and collector
+registry migrations. A newer database must be opened by newer code; an old
+release must never be pointed at a newer schema. `inspect` reports this state.
+
+## Consistent backup set
+
+The database backup is not the whole installation. A restorable set is:
+
+1. Hub SQLite database: `telemetry.sqlite` (includes registry/admin/alerts).
+   Use `ops backup`; do not copy the main file alone while the hub is running.
+2. Enrollment/config state when configured: `HUB_STATE_FILE` (for
+   example `hub/enrollment.json`, the import source for previously enrolled
+   collectors) and the private hub environment file. These
+   contain credentials: protect and copy them only between private locations.
+3. Collector identity/enrollment: `collector/collector.json`.
+4. Collector outboxes: `outbox/<destination-hash>/outbox.sqlite` per hub URL,
+   which may contain unacknowledged telemetry. Replay is at-least-once; the hub
+   deduplicates by collector/run/sequence.
+5. Optional operational metadata: `install-state.json`, units and the built
+   release. Keep the release matching the database schema for rollback.
+
+For a consistent set, prefer stopping both services cleanly, then run the online
+database backup and copy the other private files. For an online backup, the SQLite
+copy is internally consistent, but JSON identity/outbox files should be copied in
+a quiescent moment; outbox copies may lag and replay duplicates are harmless.
+
+Restore the database with `ops restore`; copy non-database state back separately
+with the same ownership and modes. Never silently copy another installation's private keys, browser credentials or unrelated files.
+
+## Upgrade and rollback
+
+For an upgrade:
+
+1. Package and stage the new release; install `npm ci --omit=dev` there.
+2. Stop the old hub/collector and verify the lock is gone.
+3. Run `install ... --activate --confirm-stopped` (or `activate`).
+4. The installer backs up, migrates and verifies before switching `current`.
+5. Start/enable services explicitly and check health. If the services were
+   already installed:
+   `systemctl --user restart rigscope-hub.service rigscope-collector.service`.
+
+Rollback:
+
+```sh
+node dist/ops.cjs rollback \
+  --prefix /home/operator/.local/opt/rigscope \
+  --state-dir /home/operator/.local/state/rigscope \
+  --confirm-stopped
+```
+
+Rollback is refused while the hub lock exists. If the database schema is newer
+than the previous release's manifest, rollback restores the recorded matching
+pre-upgrade backup (verified against that release's schema) before switching
+`current`. If no matching backup exists, it refuses rather than claiming an old
+binary can read a newer schema. The replaced database is itself backed up first.
+
+### Upgrade ordering
+
+Within a user installation or this manifest archive layout, hub and collector
+share one release: upgrade the installation together. Stop the hub and
+collector, activate the new release (which migrates and verifies the database),
+then start both services from the same release. Avoid running an old collector
+against a newer hub or vice versa for longer than the restart itself. Do not roll
+a hub back below the schema its database already uses; `rollback` enforces that
+boundary, and collector rollback keeps its private state and outboxes. Process
+detail remains request-scoped and is never persisted or included in the backup
+set. Separate machines and hubs are upgraded independently. System installations
+use the component controls described in [installation](installation.md#system-component-lifecycle).
+
+On upgrade from the previous collector endpoint, saved `/api/collector/v1/connect`
+URLs normalize to `/api/collector/connect` without changing identity, credentials,
+sharing policy or pending enrollment recovery proofs. The collector recovers the
+old URL's outbox before connecting. If both queue directories exist, it imports
+missing record identities in a transaction before removing the old directory.
+Stored metrics and gaps with the previous `protocolVersion: 2` envelope are
+decoded into the current wire format at the outbox boundary; record identities
+and previously offered gap payloads are preserved. Wire parsers remain strict.
+
+## Measurement
+
+`ops measure` is a bounded synthetic runner, not a capacity guarantee:
+
+```sh
+node dist/ops.cjs measure --machines 2 --batches 30 --records-per-batch 5 \
+  --gpus 2 --viewers 2 --json
+```
+
+It writes synthetic records through the real `TelemetryStore` into a
+temporary private database and reports accepted/rejected/busy counts, ingestion
+latency, event-loop delay/utilization, RSS, database/WAL size, store queue and
+flush counters, and one history-query latency. The output is labelled synthetic.
+Defaults are intentionally small; upper bounds are 1,000 machines, 2,000 batches,
+32 records per batch and 100,000 records total. It never opens a service socket
+or contacts a hub. `--keep` retains the temporary directory for inspection. The
+library accepts `historyConcurrency` for overlapping History request rounds;
+the CLI exposes it as `--history-concurrency N` (default 1).
+
+Current integration notes: storage commits batches of at most 32 records and the
+history query has a 25,000-row/100 ms budget; a 512-record synthetic benchmark
+showed roughly 407 ms transactions. Measure on the actual deployment before
+choosing cadence, retention or viewer counts. No latency or capacity guarantee
+is implied by these development numbers.
+
+For sustained concurrent viewers, collector reconnects/replay, slow SSE
+consumers and a fleet beyond 50 machines, use the isolated harness described in
+[scale-validation.md](scale-validation.md). It spawns a temporary loopback hub
+and never contacts the live service.
+
+## Verification limits
+
+Default repository tests exercise backup/restore, newer-schema refusal, corrupted input,
+lock refusal, installer staging/activation/rollback and failed-upgrade recovery
+in isolated temporary directories. CLI fixture tests additionally cover doctor
+redaction and role separation, inactive and live upgrade validation, paired
+rollback restore/refusal, compatible rollback without data replacement, and
+offline password recovery with lock refusal and audit. They do not start
+services, enable units, touch Tailscale, use real credentials, modify developer
+state, or perform npm network installs; fixture dependency trees stand in for
+production `npm ci`.
+Production visual/render verification and integrated runtime tests are separate.
+The opt-in real-service checks in [releasing](releasing.md#verify-the-exact-archive)
+do install disposable systemd user services and production npm dependencies;
+they remove their test units afterward and use separate state and launchers.
+
+Pass `--node /absolute/path/to/node` only when the unit must use a Node 26
+executable different from the one running the installer. The path must be an
+absolute regular file; use the release's `node_modules` installed with
+`npm ci --omit=dev`, never a development-only tree.
+
+## Storage budgets and pressure recovery
+
+Each Collector Hub connection owns policy filtering, cadence, inventory and its
+durable outbox submission. On restart, recovered metrics are sanitized with the
+current sharing policy before the connection starts; capture-time cadence is not
+reapplied to retained records. Endpoint upgrade recovery and aggregate budgets
+continue to use the existing outboxes. Hub startup restores stored telemetry,
+cadence, inventory and metadata through Sample ingestion while retaining a
+disconnected state until an authenticated Collector reconnects.
+
+The default hub main-file plus WAL budget is 2 GiB, shared by every SQLite table.
+The main file has a page-count limit; WAL reserves up to 128 MiB inside that budget.
+Telemetry writes return busy on pressure or SQLITE_FULL, with no receipt/ack for a
+failed transaction. Retention reclaims old raw/one-minute points in bounded chunks
+while preserving the newest hour; hourly rollups retain their configured age window.
+A full newest-hour working set must fit the deployment's budget. Outboxes retry
+while the operator frees space or provisions a larger supported configuration.
+
+Collector defaults are 64 MiB logical JSON and 80 MiB physical main+WAL per hub,
+256 MiB logical and 640 MiB physical across eight destinations. Physical pressure
+evicts only pending metrics, records dropped intervals, and protects in-flight
+records. Failed gap writes are held in bounded memory and retried after recovery;
+an abrupt power loss can still lose that in-memory notice.
+
+New databases use incremental page reclamation. Existing databases created without
+auto-vacuum may keep allocated pages after retention deletes rows. Startup reports
+pressure and limits growth; it never runs an unbounded VACUUM. Back up the database,
+stop the hub and inspect file/WAL sizes before offline maintenance. These are
+application/SQLite budgets rather than an operating-system filesystem quota. An
+atomic transaction can briefly increase WAL before its post-commit checkpoint;
+blocked checkpoints leave pressure visible and prevent continued ingestion. Keep
+free disk space for backups, SQLite sidecars and release artifacts.
+
+Collector state and enrollment file locks now contain an owner PID and instance.
+A proven-dead owner is recovered with inode checks. Legacy empty `.lock` files have
+no owner proof and intentionally require one manual inspection: verify no collector
+or enrollment writer is running before removing that specific abandoned file.
+
+
+Generated user units omit `ProtectKernelModules=yes`: on the target user manager,
+that directive fails before Node starts with `218/CAPABILITIES`. Normal user
+privileges and `NoNewPrivileges=yes` already prevent gaining module-loading
+privileges. PrivateTmp, ProtectSystem=strict, explicit writable paths and the other
+validated user-unit hardening remain enabled. Do not copy root-only hardening into
+user units without starting and stopping an isolated unit on that host.

package/docs/releasing.md

@@ -0,0 +1,152 @@
+# Releasing Rigscope
+
+The first public version is `0.1.0`. Source and issue links target
+`https://github.com/Triet1402/rigscope`. Create that repository and push the
+release source before npm publication. The package includes MIT licensing,
+dependency notices, the security policy and [release notes](../CHANGELOG.md).
+
+## Recommended release order
+
+1. Finish the version, public docs, release notes and repository metadata. Commit
+   those changes. Keep the installation examples in the package ready for the
+   version being published.
+2. Push the source to GitHub and run the manual CI workflow on that revision.
+   This repository deliberately has no push, pull-request or scheduled CI
+   triggers. A local result is not a GitHub Actions result.
+3. Build one npm archive from the final revision. Retain it, its SHA-256 checksum
+   and verification results. Test and publish that archive without rebuilding it.
+4. Run the publish dry run, check the npm account, and publish the verified archive.
+5. Verify the registry version and install it into a disposable test installation.
+   Check sign-in, fresh telemetry and the installed command.
+6. Tag the exact source revision as `v0.1.0`, push the tag, and create a GitHub
+   Release with the release notes, verified archive and checksum. Announce it
+   after the registry installation works.
+
+For subsequent releases, update the package and lockfile versions together
+(for example, `npm version patch --no-git-tag-version`), update the release notes,
+and repeat the same sequence. A published name/version cannot be overwritten,
+so a correction after publication needs a new version. Prereleases should use
+an explicit tag such as `next`; ordinary `rigscope upgrade` follows `latest`.
+
+## Build the final archive
+
+Run from the source checkout with Linux and Node.js 26+:
+
+```sh
+npm ci
+npm run typecheck
+mkdir -p out/release
+npm pack --pack-destination out/release
+```
+
+`npm pack` runs `prepack`, which regenerates dependency notices, builds fresh
+runtime/UI artifacts and verifies them. If notice generation changes a tracked
+file, include that change in the release commit before producing the final
+archive. Reuse the archive for every check below. Replace `0.1.0` in these
+commands when releasing another version.
+
+## Verify the exact archive
+
+```sh
+node scripts/verify-package.cjs ./out/release/rigscope-0.1.0.tgz
+npm run verify:installed -- --archive ./out/release/rigscope-0.1.0.tgz
+npm test
+node scripts/test-runtime.cjs
+node scripts/test.cjs --suite development
+```
+
+Package inspection verifies the built CLI/runtime/UI and excludes source tests,
+Git data, private environment files and caches. Inspect the file list for
+unexpected state, databases, credentials or logs. Installed-package verification
+uses production dependencies in a temporary directory, audits the resolved
+dependency tree and checks the CLI version plus authenticated sign-in/History.
+
+The following opt-in checks require a working Linux systemd user manager. They
+create disposable installations with separate state, launchers, ports and unit
+names, then remove their services. They do not select an existing installation.
+
+```sh
+RIGSCOPE_TEST_ARCHIVE="$PWD/out/release/rigscope-0.1.0.tgz" \
+  node --import tsx --test tests/cli-install.integration.ts
+node scripts/verify-operations.cjs ./out/release/rigscope-0.1.0.tgz
+```
+
+These cover installation, pairing, independent component controls, bootstrap
+removal, retained-data reinstall, upgrade/rollback, failed-start recovery and
+offline password recovery. Runtime tests additionally cover authenticated HTTPS,
+SSE, collector connections, persistence and reconnect behavior in isolated setups.
+
+On the reference host with 48 logical cores, two RTX 3090s and an ASPEED adapter,
+run the hardware-specific test explicitly:
+
+```sh
+RIGSCOPE_REAL_HW=1 node --import tsx --test tests/sampling-real.test.ts
+```
+
+That test reads the actual host and is skipped by default. Its exact inventory
+assertions are specific to that rig; it is not a portable hardware test.
+
+Record the final checksum after verification:
+
+```sh
+cd out/release
+sha256sum rigscope-0.1.0.tgz > SHA256SUMS
+sha256sum --check SHA256SUMS
+```
+
+Keep the source revision, archive checksum, check results and any skips together.
+Record boot enablement separately from an actual reboot. Physical laptop testing
+and synthetic scale workloads are separate evidence; see
+[hardware support](installation.md#hardware) and [scale validation](scale-validation.md).
+
+## Publish dry run and publication
+
+Run these commands from the source checkout. Sign in interactively if needed,
+then verify the account and preview the exact archive:
+
+```sh
+npm login --registry=https://registry.npmjs.org
+npm whoami --registry=https://registry.npmjs.org
+npm publish ./out/release/rigscope-0.1.0.tgz --access public --tag latest \
+  --registry=https://registry.npmjs.org --dry-run
+```
+
+The dry run does not publish, reserve the name, or prove that registry
+authorization and two-factor authentication will succeed. The repository
+metadata does not establish npm package ownership.
+
+When releasing, this command actually publishes the verified archive publicly:
+
+```sh
+npm publish ./out/release/rigscope-0.1.0.tgz --access public --tag latest \
+  --registry=https://registry.npmjs.org
+npm view rigscope@0.1.0 version dist.integrity --registry=https://registry.npmjs.org
+```
+
+Then use `npx --yes rigscope@0.1.0 install` with a disposable `--state-dir`, an
+unused `--port` and an isolated `XDG_BIN_HOME` for the registry installation
+check. Uninstall that test installation afterward. Use the same isolation
+settings for all test commands so the check does not select a real installation.
+
+The public quick start uses `npx --yes rigscope@latest install --boot` and the
+installed `rigscope upgrade`. Archive commands remain available before npm
+publication and for explicit version selection. Users do not need a global npm
+installation of Rigscope.
+
+GitHub Actions trusted publishing can automate later releases. Configure it for
+the actual repository and publishing workflow; the current CI workflow only
+verifies and does not publish. See npm's
+[publishing documentation](https://docs.npmjs.com/cli/commands/npm-publish/) and
+[trusted publishing guide](https://docs.npmjs.com/trusted-publishers/).
+
+## Alternative manifest archives
+
+`npm run package:release` creates an operations `.tar.gz` with a hashed manifest.
+It is a separate format from the npm `.tgz`. The low-level installer validates
+size/hash and required artifacts before activation. Production dependencies must
+be installed in the staged release; activation never silently runs npm or service
+commands. See [operations](operations.md#low-level-manifest-archive-workflow) and
+[system installation](installation.md#system-installation).
+
+Configuration, database schema and wire-protocol revisions are internal
+compatibility checks, independent of the application package version.