rigscope 0.0.0-stage → 0.1.0

package/docs/scale-validation.md

@@ -0,0 +1,263 @@
+# Sustained scale validation
+
+Status: implementation and local sustained measurement, 2026-11-04.
+
+This document records the isolated sustained workload added after the earlier
+`ops measure` burst. It is deliberately **not** a capacity guarantee. The
+numbers below describe one host, one payload shape, one database and the
+concurrency listed for each case. They do not prove that a larger fleet,
+different hardware, slower storage or a different browser workload will behave
+the same way.
+
+## What was missing
+
+The earlier `ops measure` command was a bounded synthetic burst capped at 50
+synthetic machines; it queues a fixed number of records and by default runs
+History query iterations sequentially unless --history-concurrency is supplied. It never opens a service socket, so it
+cannot show concurrent browser requests, SSE delivery, slow consumers,
+collector reconnects or replay. Those paths are now measured by a standalone
+harness instead of being inferred from the burst runner.
+
+`src/operations/measure.ts` was also extended:
+
+- the synthetic machine bound is now 1,000 instead of 50,
+- `historyConcurrency` initiates overlapping History request rounds and reports
+  their wall time, p95 and query count,
+- the existing CLI arguments and report fields remain compatible.
+
+## Harness and isolation
+
+The harness is `scripts/scale-harness.ts`. It is a standalone source-checkout
+tool, separate from the installed operational CLI. With development dependencies
+installed, run it with `tsx`:
+
+```sh
+npx tsx scripts/scale-harness.ts --suite \
+  --json-out docs/scale-validation.json
+```
+
+The parent process:
+
+1. creates a fresh `0700` temporary directory and a `0600` hub configuration,
+2. writes a throwaway SQLite database path,
+3. spawns the same script in `hub` child mode on an ephemeral `127.0.0.1`
+   port,
+4. logs in real browser sessions through `/api/auth/login` (including the
+   normal scrypt work and login token bucket),
+5. connects real collector WebSockets through `/api/collector/connect`,
+6. reads authenticated `/api/live` SSE streams and `/api/history` responses,
+7. stops the child, closes the database and removes the temporary directory
+   unless `--keep` is set.
+
+The child is composed from the production modules: `HubState`,
+`TelemetryStore`, `SampleIngestion`, `CollectorGateway`, `LiveFeed`,
+`BrowserHttpAuth`, `BrowserSessionStore` and `MonitorHttp`. The database uses
+WAL, `synchronous=FULL` and the real commit-before-ack path. The only synthetic
+parts are the metric values and the cadence driver. No live hub, service
+socket, production database or Tailscale endpoint is contacted.
+
+The harness exposes a private `GET /__scale/stats` control endpoint on the
+isolated child. It returns event-loop delay percentiles, event-loop
+utilization, process CPU/RSS, database/WAL bytes, telemetry store counters,
+connected collector count and SSE client count. The endpoint is not part of
+the hub and is never reachable from a deployment.
+
+## Workload
+
+Each case uses representative metrics: eight CPU cores, two GPUs, two
+filesystems, two disk devices, two network interfaces, plus the optional
+bottleneck fields (temperature, package power, I/O wait, pressure, inode use,
+disk latency and utilization). One record carries roughly 50 stored series
+points before rollups.
+
+| Case | Machines | Measured phase | Live cadence | History viewers | SSE viewers | Special workload |
+| --- | ---: | ---: | ---: | ---: | ---: | --- |
+| `small-5m` | 5 | 60 s | 1 s | 4 | 2 | small-fleet baseline |
+| `fleet-75m` | 75 | 60 s | 1 s | 8 | 4 | beyond the old 50-machine cap |
+| `fleet-75m-churn` | 75 | 60 s | 1 s | 8 | 4 | 15 collectors reconnect every 15 s with 20 replay records; SSE viewers reconnect with `Last-Event-ID` |
+| `fleet-75m-slow-sse` | 75 | 60 s | 1 s | 8 | 6 | two SSE viewers pause reading for 15 s, measuring stale delivery and any buffer drop/reconnect |
+| `fleet-75m-history` | 75 | 60 s | 1 s | 8 | 2 | broad History metric set every 300 ms per viewer |
+
+Before each measured phase the harness connects all collectors and waits
+1.5 seconds. The measured phase starts after that warmup. Records are counted
+from collector startup, so the ingestion counters include the warmup and bounded drain tail; the
+event-loop, CPU, RSS and database-growth windows are reset at the measured
+phase boundary.
+
+### Bounds
+
+- machines: 1–1,000,
+- measured phase: up to 1 hour,
+- sample cadence: at least 100 ms (the protocol rate limit),
+- records per live batch: 1–32,
+- viewers: 1–50 authenticated sessions,
+- SSE clients: 1–64,
+- replay: at most 128 records per reconnect,
+- estimated live plus replay records: at most 500,000 per case,
+- database: 512 MiB main+WAL by default (`--database-budget-mb 0` disables the
+  cap for inspection only),
+- temporary database/config directory: removed after the case unless
+  `--keep` is used.
+
+## Host
+
+The measurement host was:
+
+- AMD EPYC 7402P 24-Core Processor, 48 logical CPUs,
+- 121 GiB usable memory (130,351,214,592 bytes reported by the OS),
+- Linux 7.3.0-9-generic, Ubuntu 26.10, x86_64,
+- Node.js 26.13.0,
+- local NVMe root filesystem (the harness database lived under `/tmp`),
+- load average 0.92 / 0.76 / 0.84 around the run; unrelated user services were
+  active, so this is not a quiet benchmark host.
+
+`monitorEventLoopDelay` used a 10 ms resolution. The reported p50 therefore
+floors near 10 ms even when the process is mostly idle; p95/p99/max are the
+meaningful tail values.
+
+## Results
+
+All five cases completed with zero rejected records, zero `retry` acknowledgments,
+zero unacknowledged live records, zero write failures and no disk-budget
+pressure. The event-loop window excludes the 1.5 s collector warmup. "History /
+SSE" is authenticated History sessions / SSE clients. "Sample→commit" in the original report is capture-to-durable-ack latency: an
+upper bound on commit lag that also includes the acknowledgment return path.
+It is measured on the same host clock, not across uncalibrated remote clocks.
+"Normal SSE lag" is the
+age of the newest captured sample when an `update`/`snapshot` arrives at a
+normal viewer; it includes the hub's 500 ms coalescing window. Hub CPU is the
+hub child's process CPU as a percentage of one core, not the whole host.
+
+| Case | Machines | History / SSE | Records accepted | Ack p95 | Sample→commit p95 | History p95 / max | Normal SSE lag p95 | Event-loop p99 / max | Hub CPU | RSS peak | DB+WAL growth |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| `small-5m` | 5 | 4 / 2 | 317 / 317 | 26.7 ms | 27 ms | 12.9 / 20.0 ms | 526 ms | 12.5 / 18.5 ms | 2.6% | 135.9 MiB | 4.63 MiB |
+| `fleet-75m` | 75 | 8 / 4 | 4,789 / 4,789 | 55.6 ms | 56 ms | 20.9 / 35.2 ms | 439 ms | 30.4 / 44.6 ms | 12.4% | 188.7 MiB | 12.08 MiB |
+| `fleet-75m-churn` | 75 | 8 / 4 | 4,845 / 4,845 | 52.2 ms | 52 ms | 23.4 / 35.0 ms | 438 ms | 29.0 / 42.4 ms | 14.2% | 203.2 MiB | 14.22 MiB |
+| `fleet-75m-slow-sse` | 75 | 8 / 6 | 4,791 / 4,791 | 64.4 ms | 64 ms | 23.0 / 36.6 ms | 502 ms | 38.5 / 76.2 ms | 12.0% | 190.2 MiB | 12.95 MiB |
+| `fleet-75m-history` | 75 | 8 / 2 | 4,785 / 4,785 | 70.7 ms | 71 ms | 50.3 / 60.5 ms | 453 ms | 48.0 / 72.7 ms | 31.2% | 229.7 MiB | 12.98 MiB |
+
+Additional observations:
+
+- **History authorization budget held.** The broad case ran 1,608
+  authenticated queries (8 sessions polling every 300 ms) with 1,608 `200`
+  responses, zero `429` responses and zero errors. Each response returned
+  about 25 series and 800 points; the 25,000-row/100 ms budget was not
+  truncated.
+- **Collector reconnect/replay recovered.** The churn case performed 45
+  collector reconnects and replayed 900 historical records; all 900 were
+  accepted. First live acknowledgment after reconnect was 167.9 ms p95
+  (175.8 ms max). The durable queue finished empty.
+- **SSE reconnect/reset recovered.** The churn case reconnected 48 SSE streams
+  and received 48 `reset` snapshots; all resets followed a reconnect. First
+  event after reconnect was 10.1 ms p95.
+- **Slow consumers stayed connected but stale.** The two paused viewers
+  received no new bytes for 15 s and then read buffered snapshots. Their
+  delivery lag was 11.6 s p95 while normal viewers stayed at 502 ms p95. On
+  this loopback path no slow-viewer drop or reconnect was observed, so the
+  hub's 1 MiB writable-buffer guard did not fire; kernel socket buffering
+  likely absorbed the pause. The deliberate disconnect/reconnect path is
+  covered by the churn case instead.
+- **Database/WAL stayed bounded.** The final main+WAL size was 14.8 MiB for
+  the 75-machine baseline, 16.9 MiB with churn, and 15.7 MiB with broad
+  History. The 512 MiB measurement budget was never approached, no checkpoint
+  pressure appeared, and the store queue was empty at the end of every case.
+- **Small fleet stayed cheap.** Five machines, four History viewers and two SSE
+  viewers used 2.6% of one core and stayed at 12.5 ms event-loop p99.
+
+The full machine-readable report, including per-case store counters, database
+sizes, SSE byte counts, tier counts and host load averages, is
+[scale-validation.json](scale-validation.json). The generated JSON is the raw
+artifact; this page is the summary.
+
+## Interpretation
+
+These results support the following limited statements on this host:
+
+- A 75-machine, 1-second-cadence synthetic fleet can be ingested with
+  commit-before-ack p95 under 100 ms while eight authenticated History viewers
+  and four full-fleet SSE viewers run concurrently.
+- Collector reconnect plus 20-record replay per affected machine recovered
+  within a few hundred milliseconds in this run, without rejected or unacked
+  records.
+- SSE reconnect with `Last-Event-ID` produced a fresh `reset` snapshot within
+  about 10 ms on loopback, and the next update was fresh again.
+- Broad History polling near the per-session token budget raised the
+  event-loop tail to about 48 ms p99 and used about 31% of one core, but
+  produced no 429s and no truncated responses at this data size.
+
+They do **not** support claims that:
+
+- 75 machines is a capacity limit or a safe upper bound,
+- a larger fleet, more viewers, slower storage or a different payload will meet
+  the same latencies,
+- the hub is multi-core, clustered or horizontally scalable,
+- slow consumers are always disconnected before they fall behind,
+- the default 100 concurrent collector socket budget should be raised without
+  a deployment-specific measurement.
+
+The 75-machine cases used the default 100-connection budget. A deployment that
+raises `HUB_MAX_COLLECTOR_CONNECTIONS` must re-run the harness with its own
+payloads, storage and viewer counts.
+
+## Reproduction
+
+```sh
+# full five-case suite (about five minutes)
+npx tsx scripts/scale-harness.ts --suite \
+  --json-out docs/scale-validation.json
+
+# one case with overrides
+npx tsx scripts/scale-harness.ts --case fleet-75m \
+  --machines 75 --viewers 8 --sse-clients 4 --duration-ms 60000 \
+  --json-out /tmp/rigscope-scale-75m.json
+
+# retain the throwaway database and configuration for inspection
+npx tsx scripts/scale-harness.ts --case small-5m --keep
+```
+
+The short `tests/scale-harness.test.ts` smoke runs the same child with two
+machines, collector reconnect/replay and an SSE reset. It is not a capacity
+test; it proves the harness paths work in CI.
+
+## Limits
+
+- The hub child and all load generators share one host. The hub event loop uses
+  one thread; the load processes can run on other cores. This is
+  not a distributed-load test and the generator competes with the hub.
+- History queries are synchronous SQLite work on the hub event loop.
+  "Concurrent viewers" means overlapping authenticated HTTP requests, not
+  parallel SQLite readers.
+- The payload is representative but not captured from a real fleet. Different
+  process counts, GPUs, filesystems or optional fields change the stored row
+  count.
+- The host may run unrelated services. Load averages and CPU are recorded in
+  the JSON artifact, but they are not a controlled benchmark environment.
+- Passing these cases does not remove the concurrent collector socket budget
+  (default 100, configurable with `HUB_MAX_COLLECTOR_CONNECTIONS`) or the
+  history row/elapsed/frame/storage budgets.
+- Authentication and machine scopes use the production modules with loopback
+  origin rules. This harness does not exercise the deployed HTTPS/private-socket
+  boundary; its separate runtime and actual-path checks remain distinct.
+- A result is evidence for this host and workload only. It is not a claim of
+  unlimited machine, viewer or History capacity.
+
+
+## Root integration acceptance
+
+After integration, a separate 75-machine, eight-second churn run used eight
+History sessions and four SSE consumers. Its authorization canary required
+anonymous SSE 401, an allowed scoped History query 200, a forbidden machine query
+403, and correctly filtered initial and reconnect/reset SSE snapshots. All passed.
+The run accepted 930 live records plus 600 replay records, recorded 30 collector
+reconnects, finished with zero unacked live records and zero write failures.
+`scale-acceptance.json` records this shorter integration check. It is not another
+60-second capacity measurement; the five sustained results above remain the
+worker's original raw artifact from before the authorization canary was added.
+
+The original `sampleToCommitLagMs` field names are retained for compatibility.
+Those numbers are capture-to-durable-ack upper bounds on commit latency, including
+the return path; they are not a server-side SQL commit timestamp. Queue/backpressure
+counters and end-of-run queue emptiness are recorded, but the original report does
+not sample an exact queue high-water mark. Slow viewers did not reach the server's
+buffer-disconnect threshold in these runs, so that threshold is not a measured
+fleet-capacity limit.

package/next.config.js

@@ -0,0 +1,38 @@
+const { PHASE_DEVELOPMENT_SERVER } = require("next/constants");
+const { availableParallelism } = require("node:os");
+
+const workers = Number(process.env.RIGSCOPE_BUILD_WORKERS ?? Math.min(4, availableParallelism()));
+if (!Number.isInteger(workers) || workers < 1 || workers > 64) {
+  throw new Error(
+    `RIGSCOPE_BUILD_WORKERS must be an integer from 1 to 64 (received ${JSON.stringify(process.env.RIGSCOPE_BUILD_WORKERS)}). ` +
+      "Set it to the number of build workers, or leave it unset to auto-select up to four.",
+  );
+}
+
+/** @param {string} phase @returns {import("next").NextConfig} */
+const nextConfig = (phase) => ({
+  // Keep dev writes separate from the packaged hub's production artifact.
+  // Use a small static-generation worker pool for this six-page scaffold.
+  distDir: phase === PHASE_DEVELOPMENT_SERVER ? ".next-dev" : ".next",
+  experimental: { cpus: workers },
+  // Remote development goes through an outer TLS terminator and a Unix
+  // socket, so the browser's Origin hostname is the configured public origin,
+  // not localhost. Keep an explicit hostname allowlist for /_next assets and
+  // HMR (Next also adds the createNextApp hostname). Configuring this list
+  // makes Next's dev guard block unlisted origins instead of only warning.
+  // The hub validates the full exact origin before delegation.
+  allowedDevOrigins: phase === PHASE_DEVELOPMENT_SERVER ? publicDevOrigin() : undefined,
+});
+
+function publicDevOrigin() {
+  const value = process.env.HUB_PUBLIC_ORIGIN;
+  if (value === undefined || value === "") return undefined;
+  try {
+    const origin = new URL(value);
+    return origin.protocol === "https:" ? [origin.hostname] : undefined;
+  } catch {
+    return undefined;
+  }
+}
+
+module.exports = nextConfig;

package/package.json

@@ -1,6 +1,111 @@
 {
   "name": "rigscope",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.1.0",
+  "rigscopeConfigVersion": 2,
+  "private": false,
+  "engines": {
+    "node": ">=26.0.0"
+  },
+  "os": [
+    "linux"
+  ],
+  "scripts": {
+    "dev": "NODE_ENV=development tsx src/hub/main.ts",
+    "build": "npm run build:ui && npm run build:runtime",
+    "build:runtime": "node scripts/build-runtime.cjs",
+    "start": "node dist/hub.cjs",
+    "collector:mock": "COLLECTOR_MODE=mock tsx collector/main.ts",
+    "collector:enroll": "tsx collector/main.ts --enroll",
+    "collector:start": "node dist/collector.cjs",
+    "ops": "node dist/ops.cjs",
+    "package:release": "npm run build && node scripts/package-release.cjs",
+    "typecheck": "tsc --noEmit",
+    "test": "node scripts/test.cjs",
+    "test:runtime": "npm run build:runtime && node scripts/test-runtime.cjs",
+    "test:dev": "npm run build:runtime && node scripts/test.cjs --suite development",
+    "validate:https": "node scripts/validate-https.cjs",
+    "build:ui": "next build --webpack",
+    "rigscope": "node bin/rigscope.cjs",
+    "verify:build": "node scripts/verify-build.cjs",
+    "verify:package": "node scripts/verify-package.cjs",
+    "verify:installed": "node scripts/verify-installed-package.cjs",
+    "prepack": "node scripts/prepack.cjs",
+    "package:npm": "npm pack",
+    "notices": "node scripts/third-party-notices.cjs"
+  },
+  "dependencies": {
+    "lucide-react": "^0.475.0",
+    "next": "^16.4.5",
+    "next-themes": "^0.4.6",
+    "react": "^19.1.0",
+    "react-dom": "^19.1.0",
+    "ws": "^8.22.1"
+  },
+  "devDependencies": {
+    "@tailwindcss/postcss": "^4.1.0",
+    "@types/node": "^22.13.0",
+    "@types/react": "^19.1.0",
+    "@types/react-dom": "^19.1.0",
+    "@types/ws": "^8.18.2",
+    "esbuild": "^0.28.2",
+    "postcss": "^8.5.23",
+    "tailwindcss": "^4.1.0",
+    "tsx": "^4.23.15",
+    "typescript": "^5.7.0"
+  },
+  "description": "Your machines, in view. A self-hosted Linux system and process monitor.",
+  "bin": {
+    "rigscope": "bin/rigscope.cjs"
+  },
+  "files": [
+    "bin/",
+    "dist/",
+    ".next/BUILD_ID",
+    ".next/*.json",
+    ".next/server/",
+    ".next/static/",
+    "next.config.js",
+    "README.md",
+    "CHANGELOG.md",
+    "docs/installation.md",
+    "docs/operations.md",
+    "docs/bottlenecks.md",
+    "docs/exports.md",
+    "docs/development.md",
+    "docs/releasing.md",
+    "docs/scale-validation.md",
+    "docs/scale-validation.json",
+    "docs/scale-acceptance.json",
+    "ARCHITECTURE.md",
+    "DESIGN.md",
+    "LICENSE",
+    "SECURITY.md",
+    "THIRD_PARTY_NOTICES.md",
+    "packaging/"
+  ],
+  "license": "MIT",
+  "author": {
+    "name": "Triet (@Triet1402)",
+    "url": "https://github.com/Triet1402"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/Triet1402/rigscope.git"
+  },
+  "homepage": "https://github.com/Triet1402/rigscope#readme",
+  "bugs": {
+    "url": "https://github.com/Triet1402/rigscope/issues"
+  },
+  "keywords": [
+    "linux",
+    "monitoring",
+    "system-monitor",
+    "process-monitor",
+    "self-hosted",
+    "gpu"
+  ],
+  "publishConfig": {
+    "access": "public",
+    "registry": "https://registry.npmjs.org/"
+  }
+}

package/packaging/README.md

@@ -0,0 +1,5 @@
+# Packaging reference files
+
+The `.env.example` files contain public examples, never deployment credentials. Copy configuration to private files (0600) outside the source tree and generate fresh credentials. No configuration starts a production deployment by itself.
+
+Use [installation](../docs/installation.md) for managed user/system installs, [operations](../docs/operations.md) for manifest archives and recovery, and [releasing](../docs/releasing.md) for package validation. System reference units are in [system](system/README.md). Generated units in the installer source are authoritative.

package/packaging/collector.env.example

@@ -0,0 +1,24 @@
+# Collector environment example
+#
+# Copy what you need into the private collector.env created by the installer
+# (mode 0600). The generated file starts with every value commented out: with
+# no destinations the collector fails closed and delivers nothing.
+#
+# PUBLIC DEMO VALUES ONLY. demo-token-local is a deliberately public local test
+# string, not a credential for any real deployment.
+
+# COLLECTOR_MODE=real
+# COLLECTOR_STATE_FILE=/home/operator/.local/state/rigscope/collector/collector.json
+# COLLECTOR_INTERVAL_MS=1000
+# COLLECTOR_HOSTNAME=rig-01
+
+# Enroll once against the hub, then remove the one-use grant from the hub.
+# COLLECTOR_ENROLL_URL=ws://127.0.0.1:8081/api/collector/connect
+# COLLECTOR_ENROLL_TOKEN=<one-use grant from the hub>
+
+# Or supply destinations explicitly (enrollment writes these into private state
+# for real installations). Full websocket URL including the connect path:
+# COLLECTOR_HUBS=[{"url":"ws://127.0.0.1:8081/api/collector/connect","token":"demo-token-local"}]
+
+# Per-destination sharing policy can withhold sensitive families. Defaults are
+# CPU/memory only; process details remain request-scoped and are never spooled.

package/packaging/hub.env.example

@@ -0,0 +1,31 @@
+# Hub environment example
+#
+# Copy the values you need into the private hub.env created by the installer
+# (mode 0600), not into a repository file. The generated hub.env starts with
+# every value commented out: no browser users and no collector credentials
+# means protected browser data fails closed.
+#
+# PUBLIC DEMO VALUES ONLY below. Never use demo-token-local for a real host,
+# never reuse a demo browser password, and never put credentials in URLs.
+
+# Loopback-only development listener.
+# HUB_HOST=127.0.0.1
+# HUB_PORT=8081
+
+# Remote mode is selected only when both are set. The socket must live under a
+# private directory (the installer's state directory is writable by the unit).
+# HUB_PUBLIC_ORIGIN=https://monitor.example.invalid
+# HUB_SOCKET_PATH=/home/operator/.local/state/rigscope/run/hub.sock
+
+# Browser access fails closed without this. Generate the real hash locally with
+# scripts/hash-browser-password.cjs; never paste a plaintext password here.
+# HUB_BROWSER_USERS=[{"username":"operator","passwordHash":"<private hash>","machineIds":"*"}]
+
+# Static collector credentials (public demo shape/token).
+# HUB_COLLECTORS=[{"collectorId":"demo-collector","machineId":"demo-machine","token":"demo-token-local"}]
+
+# Enrollment/registry state. The installer sets HUB_DATABASE_PATH in the unit;
+# keep enrollment state outside the repository, and HUB_STATE_FILE also serves
+# as the recovery import source for previously enrolled collectors.
+# HUB_STATE_FILE=/home/operator/.local/state/rigscope/hub/enrollment.json
+# HUB_ENROLLMENT_GRANTS=[{"token":"<one-use grant>","expiresAt":"<ISO timestamp>"}]

package/packaging/system/90-rigscope-rapl.rules

@@ -0,0 +1,7 @@
+# Managed by Rigscope: read-only package energy/range counters for the collector group.
+# Top-level package devices only; subdomains and constraint files are not touched.
+SUBSYSTEM=="powercap", ACTION=="add|change", KERNEL=="intel-rapl:[0-9]*", KERNEL!="intel-rapl:*:*", ATTR{name}=="package*", \
+  RUN+="/usr/bin/chgrp rigscope-rapl /sys%p/energy_uj", \
+  RUN+="/usr/bin/chmod 0440 /sys%p/energy_uj", \
+  RUN+="/usr/bin/chgrp rigscope-rapl /sys%p/max_energy_range_uj", \
+  RUN+="/usr/bin/chmod 0440 /sys%p/max_energy_range_uj"

package/packaging/system/README.md

@@ -0,0 +1,5 @@
+# System installation references
+
+These files illustrate dedicated `rigscope-hub` and `rigscope-collector` accounts, systemd units and read-only package-energy access. The renderer in `src/cli/system.ts` is authoritative; do not install examples blindly.
+
+See [system installation](../../docs/installation.md#system-installation) for planning, `--apply`, independent component controls and RAPL permission limits. The collector needs GPU device access and does not use `PrivateDevices=yes`; the hub does. Both run without root privileges, with private state and bounded writable paths.

package/packaging/system/rigscope-collector.service.example

@@ -0,0 +1,53 @@
+# Managed by `rigscope system`. Enable explicitly after enrollment.
+# Independent component: enabling or restarting this unit never touches the hub.
+# The collector is nologin and has no capability grants.
+# Package power reads use the private group rule installed separately.
+# Node receives no capabilities, no root, and no device-control access.
+[Unit]
+Description=Rigscope collector (system scope)
+Documentation=file:/opt/rigscope/current/README.md
+After=network-online.target
+Wants=network-online.target
+StartLimitIntervalSec=60
+StartLimitBurst=5
+
+[Service]
+Type=simple
+User=rigscope-collector
+Group=rigscope-collector
+WorkingDirectory=/opt/rigscope/current
+EnvironmentFile=/etc/rigscope/collector.env
+Environment=NODE_ENV=production
+Restart=on-failure
+RestartSec=2s
+KillSignal=SIGTERM
+UMask=0077
+NoNewPrivileges=yes
+PrivateTmp=yes
+ProtectSystem=strict
+ProtectHome=yes
+ProtectKernelTunables=yes
+ProtectKernelModules=yes
+ProtectKernelLogs=yes
+ProtectControlGroups=yes
+RestrictRealtime=yes
+RestrictSUIDSGID=yes
+RestrictNamespaces=yes
+LockPersonality=yes
+RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
+CapabilityBoundingSet=
+AmbientCapabilities=
+ExecStart="/opt/rigscope/node/node" "/opt/rigscope/current/dist/collector.cjs" --await-connection
+SupplementaryGroups=rigscope-rapl
+Environment="COLLECTOR_MODE=real"
+Environment="COLLECTOR_STATE_FILE=/var/lib/rigscope/collector/state.json"
+Environment="COLLECTOR_HEALTH_FILE=/var/lib/rigscope/collector/health.json"
+ReadWritePaths="/var/lib/rigscope/collector"
+ReadOnlyPaths="/opt/rigscope/current"
+TimeoutStopSec=10s
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=rigscope-collector
+
+[Install]
+WantedBy=multi-user.target

package/packaging/system/rigscope-hub.service.example

@@ -0,0 +1,50 @@
+# Managed by `rigscope system`. Enable explicitly; do not edit in place.
+# Independent component: enabling or restarting this unit never touches the collector.
+[Unit]
+Description=Rigscope hub (system scope)
+Documentation=file:/opt/rigscope/current/README.md
+After=network-online.target
+Wants=network-online.target
+StartLimitIntervalSec=60
+StartLimitBurst=5
+
+[Service]
+Type=simple
+User=rigscope-hub
+Group=rigscope-hub
+WorkingDirectory=/opt/rigscope/current
+EnvironmentFile=/etc/rigscope/hub.env
+Environment=NODE_ENV=production
+Restart=on-failure
+RestartSec=2s
+KillSignal=SIGTERM
+UMask=0077
+NoNewPrivileges=yes
+PrivateTmp=yes
+ProtectSystem=strict
+ProtectHome=yes
+ProtectKernelTunables=yes
+ProtectKernelModules=yes
+ProtectKernelLogs=yes
+ProtectControlGroups=yes
+RestrictRealtime=yes
+RestrictSUIDSGID=yes
+RestrictNamespaces=yes
+LockPersonality=yes
+RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
+CapabilityBoundingSet=
+AmbientCapabilities=
+ExecStart="/opt/rigscope/node/node" "/opt/rigscope/current/dist/hub.cjs"
+Environment="HUB_DATABASE_PATH=/var/lib/rigscope/hub/telemetry.sqlite"
+Environment="HUB_STATE_FILE=/var/lib/rigscope/hub/enrollment.json"
+Environment="HUB_SOCKET_PATH=/var/lib/rigscope/hub/run/hub.sock"
+ReadWritePaths="/var/lib/rigscope/hub"
+ReadOnlyPaths="/opt/rigscope/current"
+PrivateDevices=yes
+TimeoutStopSec=15s
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=rigscope-hub
+
+[Install]
+WantedBy=multi-user.target

package/packaging/system/rigscope-rapl.sysusers

@@ -0,0 +1,5 @@
+# Reference only. Generated by `rigscope system install --apply`
+# or `rigscope system power grant --apply`.
+# Only the collector account joins the narrow power group.
+g rigscope-rapl - -
+m rigscope-collector rigscope-rapl

package/packaging/system/rigscope.sysusers

@@ -0,0 +1,7 @@
+# Reference only. Generated by `rigscope system install --apply`.
+# One complete package: both dedicated, unprivileged, non-login accounts exist
+# even when one component is disabled, so it can be enabled later.
+g rigscope-hub - -
+u rigscope-hub - "Rigscope hub" /nonexistent /usr/sbin/nologin
+g rigscope-collector - -
+u rigscope-collector - "Rigscope collector" /nonexistent /usr/sbin/nologin