crewbus 11.3.0 → 11.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,108 +1,13 @@
1
- # Audit export + legal hold (Phase 2a)
1
+ # Audit Export & Legal Holds
2
2
 
3
- Zero-dependency, file-local. Every tamper-evident log record
4
- (`logs/chain.jsonl` for privileged CLI ops, `logs/audit.jsonl` for relay
5
- ops) is a versioned **v:1 envelope** that is hash-chained locally and
6
- HMAC-signed for independent off-box verification.
3
+ > **Consolidated Documentation:** This guide has been consolidated into the unified crewbus documentation:
4
+ > - Interactive Single-Page Guide: [`documentation.html#tamper-evident`](file:///C:/Users/Surface/Documents/crewbus/documentation.html#tamper-evident)
5
+ > - Canonical Markdown Reference: [`docs/SECURITY.md`](file:///C:/Users/Surface/Documents/crewbus/docs/SECURITY.md)
7
6
 
8
- ## Event schema (v:1, flat JSON)
7
+ ---
9
8
 
10
- ```json
11
- {
12
- "v": 1,
13
- "seq": 4,
14
- "at": "2026-10-02T23:04:27.106Z",
15
- "actor": "orchestrator",
16
- "role": "orchestrator",
17
- "action": "send",
18
- "target": "aud1",
19
- "board": "C:/data/.crewbus",
20
- "result": "ok",
21
- "prevHash": "ec0cd4…",
22
- "sig": "bdb72b…",
23
- "hash": "2783fe…",
24
- "authMethod": "token"
25
- }
26
- ```
9
+ ## Tamper-Evident Logging
27
10
 
28
- | field | meaning |
29
- |---|---|
30
- | `v` | envelope version (`1`) |
31
- | `seq` | 1-based per-stream sequence (`chain` and `audit` number independently) |
32
- | `at` | ISO-8601 UTC timestamp |
33
- | `actor` | agent (or `system`/`unknown`) that caused the event |
34
- | `role` | actor role at write time (`orchestrator`/`worker`/`unknown`) |
35
- | `action` | stable verb — the existing audit action names (`register`, `role-grant`, `offboard`, `acl-set`, `login`, `token-revoke`, `token-rotate`, `send`, `spawn`, `spawn-kill`, `remote-spawn`, `api-spawn`, `api-kill`, `hold-place`, `hold-lift`, …) |
36
- | `target` | primary target (`to`/`target`/`agent`, comma-joined, max 200 chars; `""` when none) |
37
- | `board` | board root path the event was written on |
38
- | `result` | `ok` / `fail` (from `data.ok` / `data.error`, default `ok`) |
39
- | `prevHash` | `hash` of the previous record (`GENESIS` for seq 1); duplicates `prev` |
40
- | `sig` | HMAC-SHA256 over `seq|prev|at|actor|type|canonical(data)`, hex (`""` when no key was configured) |
41
- | `hash` | keyless SHA256 chain link over the same core (verifiable without any secret) |
42
- | `authMethod` | how the actor authenticated, best-effort where known: `token` (CLI), `secret` (relay secret), `oidc` (Bearer JWT), `mtls` (client cert, `/sync/*`), else `unknown` |
43
-
44
- Stored records keep the legacy core fields (`seq, prev, hash, at, actor,
45
- type, data`) untouched, so old readers keep working; the envelope fields
46
- are additive.
47
-
48
- ## Signing key
49
-
50
- `CREWBUS_AUDIT_KEY`, else the board secret (`CREWBUS_SECRET`).
51
- Use ONE key per board — records signed under different keys (or none)
52
- fail verification under the current key at the first mismatch.
53
-
54
- ## Verify
55
-
56
- ```sh
57
- crewbus log --verify [--audit] [--json] [--limit 50]
58
- ```
59
-
60
- Checks the hash chain, then — when a key is configured — every present
61
- `sig`. Failures name the position twice: `chain INVALID at seq N
62
- (first-broken-seq N, bad hash|bad sig|bad seq|broken prev link)`.
63
- `--json` returns the same fields machine-readably (`ok`, `at`,
64
- `firstBrokenSeq`, `reason`, `count`).
65
-
66
- ## SIEM forwarder
67
-
68
- ```sh
69
- crewbus serve --audit-forward <https-url> [--audit-forward-key <bearer>]
70
- ```
71
-
72
- The relay POSTs each audit event (same v:1 schema, `application/json`;
73
- `Authorization: Bearer <key>` when configured) off-box:
74
-
75
- - **Spool first**: every event is written to `audit-spool/` before any
76
- POST, so a crash never loses an event (at-least-once).
77
- - **Never blocks**: POSTs are fire-and-forget off the request path; the
78
- relay answers peers without waiting for the SIEM.
79
- - **Retry queue**: a 1s loop re-POSTs spooled files oldest-first and
80
- deletes on 2xx. Duplicates are possible (retry + crash replay) —
81
- dedupe downstream on `(board, seq, hash)`.
82
- - **Cross-process tail**: the relay also tails `logs/chain.jsonl` and
83
- `logs/audit.jsonl`, so CLI-side events (`hold place`, `send`, …)
84
- written while it runs are forwarded too. Baseline is taken at startup
85
- (history is not re-posted); pre-existing spool files still drain.
86
-
87
- Env equivalents: `CREWBUS_AUDIT_FORWARD`,
88
- `CREWBUS_AUDIT_FORWARD_KEY`.
89
-
90
- ## Legal hold
91
-
92
- ```sh
93
- crewbus hold place --from orchestrator [--reason "..."]
94
- crewbus hold lift --from orchestrator
95
- crewbus hold status [--from <you>] [--json]
96
- ```
97
-
98
- - `place`/`lift` are orchestrator-only (token-checked + authorized).
99
- - `status` is a read — any role, including workers, may call it
100
- (with `--from` it is token-checked; without, it is an operator read).
101
- - While a hold is active, `prune` (dm/broadcast + logs) is **refused**
102
- loudly and names the hold:
103
- `prune REFUSED — legal hold ACTIVE (placed by …) — lift with: …`.
104
- Tombstone creation/sync mechanics are untouched (the gate fires before
105
- anything is deleted or tombstoned).
106
- - The hold record (`holds/legal.json`, HLC-stamped) syncs to peers like
107
- groups/cursors; every place/lift is audit-logged (`hold-place` /
108
- `hold-lift`).
11
+ - Every privileged CLI op appends to `logs/chain.jsonl`; relay operations append to `logs/audit.jsonl`.
12
+ - Versioned `v:1` envelopes contain `seq`, `at`, `actor`, `role`, `action`, `target`, `board`, `result`, `prevHash`, `sig`, and `hash`.
13
+ - Off-box verification: `crewbus log --verify` scans from genesis to tip, verifying SHA-256 chain continuity and HMAC-SHA256 signatures.
@@ -1,120 +1,13 @@
1
- # Cert-readiness checklist
2
-
3
- Control-by-control mapping of common audit expectations to what the product
4
- implements (with exact commands/flags) and what remains a customer runbook
5
- item. Statuses: **done** (in the tree, covered by `npm test`), **manual**
6
- (stub runbook below — your cadence, your evidence).
7
-
8
- ## 1. Access control (RBAC)
9
-
10
- - [done] Roles + authorization gates: `agents`, `role-grant`, privileged
11
- ops gated by `authorize()` (`prune`, `acl-set`, `role-grant`, `offboard`,
12
- `group-restrict`, `serve-remote`, `import`, `snapshot-schedule`,
13
- `quota-set`, `hold-place`, `hold-lift`). See `docs/RBAC.md`.
14
- - [done] Default-deny remote spawn/kill: `serve` without
15
- `--allow-remote-spawn` answers 403; `--allow-cmd` regex constrains remote
16
- `--cmd`; `--workdir-root` constrains remote cwd.
17
- - [done] Standby replicas refuse writes: `serve --standby` answers 503 +
18
- `X-Relay-Role: standby` on `POST /sync/put`, `/api/spawn`, `/api/kill`.
19
- See `docs/HA.md`.
20
- - [manual] **Access-review cadence (quarterly stub):** run `agents --json`,
21
- confirm each identity still needs its role, `offboard` leavers
22
- (`register --offboard <name> --from orchestrator` preserves inbox for audit),
23
- record the review date + reviewer. Evidence: the signed `role-grant` /
24
- `offboard` events in the log.
25
-
26
- ## 2. Authentication
27
-
28
- - [done] Per-identity tokens: first claim mints (`register --from <you>`
29
- prints `abt-…` once), afterwards `--token` / `CREWBUS_TOKEN` required;
30
- only salted hashes stored; `token rotate`, `token status`, `token revoke`
31
- (revocations sync, never resurrected). Service accounts:
32
- `register --service <name>`; expiry: `register --expires-in 30/90s/15m/
33
- 24h/7d`.
34
- - [done] OIDC: `login --issuer <url> --client-id <id> --token <jwt>` binds
35
- `oidc-<sub>`; relays accept Bearer with
36
- `serve --oidc-issuer <url> [--oidc-audience <id>]`; `sync`/`listen` send
37
- it via `--bearer` / `--oidc-token`. See `docs/OIDC_TLS.md`.
38
- - [done] Relay secret: `--secret` / `CREWBUS_SECRET` via
39
- `x-crewbus-secret` / `?secret=` (constant-time compare); remote serve
40
- without one refuses + warns.
41
- - [manual] **Credential-lifecycle schedule (stub):** rotate service-account
42
- tokens every N days (`token rotate --from <svc>`), rotate the relay
43
- secret on staff changes (restart relays), expire IdP clients for leavers.
44
- Evidence: `token status` output + signed rotation events.
45
-
46
- ## 3. Encryption
47
-
48
- - [done] In transit: in-box TLS `serve --tls-cert <pem> --tls-key <pem>`;
49
- mTLS on `/sync/*` with `--tls-ca` / `--mtls-ca`; outbound
50
- `--mtls-cert`/`--mtls-key`; plain-HTTP-behind-terminator stays supported.
51
- See `docs/OIDC_TLS.md`.
52
- - [done] At rest (backups): `board export` encrypts AES-256-GCM
53
- (32-byte hex/base64 used raw, else scrypt + random salt in the header);
54
- GCM tag verified *before* any byte is written on `import`; secrets
55
- stripped unless `--include-secrets` (loud warning).
56
- - [manual] **Key-rotation schedule (stub):** rotate backup keys yearly (or
57
- on incident): set the new `CREWBUS_BACKUP_KEY`, take a fresh export,
58
- verify `board import --into <fresh-dir>` decrypts, retire the old key in
59
- the vault. Evidence: dated exports + restore-drill log (below).
60
-
61
- ## 4. Audit logging
62
-
63
- - [done] Hash-chained tamper-evident log (`logs/chain.jsonl` for CLI ops,
64
- `logs/audit.jsonl` for relay ops); per-event HMAC signatures;
65
- `log [--audit] [--verify] [--limit 50]` verifies chain + sigs.
66
- - [done] SIEM export: versioned `v:1` envelope
67
- (`seq/at/actor/role/action/target/board/result/prevHash/sig`, `sig` =
68
- 64-hex HMAC); `serve --audit-forward <https-url>
69
- [--audit-forward-key <bearer>]` POSTs off-box with an `audit-spool/`
70
- at-least-once retry queue that drains (never blocks the relay path).
71
- See `docs/AUDIT_EXPORT.md`.
72
- - [done] Legal holds: `hold place --from orchestrator [--reason]` /
73
- `hold lift` / `hold status`; `prune` (and `import` over a held board)
74
- refuse while held; hold record syncs to peers.
75
- - [manual] **Log-review + retention (stub):** forward to your SIEM, alert
76
- on `audit-spool/` growth (forwarder down), review privileged actions
77
- (`role-grant`, `offboard`, `hold-*`, `import`, `quota-set`) weekly,
78
- retain per your policy (`prune --older-than` only when no hold).
79
- Evidence: SIEM queries + `log --verify` output.
80
-
81
- ## 5. Backup
82
-
83
- - [done] Portable encrypted backups (`board export --from orchestrator
84
- --out <file>`), safe restore (`board import --force` semantics + hold
85
- interaction), declarative schedules (`snapshot schedule --from orchestrator
86
- --every 24h --keep 7 --out-dir <dir>`), `snapshot run` for cron/systemd/
87
- Task Scheduler, quotas/tenancy (`quota set/show`, `storage`, separate
88
- boards per tenant). See `docs/TENANCY.md`.
89
- - [manual] **Restore-drill runbook (quarterly stub):**
90
- 1. `board export --from orchestrator --out /offsite/drill-<date>.abbackup.json`
91
- 2. `board import --in /offsite/drill-<date>.abbackup.json --into <fresh-dir> --force`
92
- 3. `gather`/inbox spot-checks + `log --verify` on the restored board
93
- 4. Record date, operator, result. Evidence: the drill log + restored-board
94
- `log --verify` output.
95
-
96
- ## 6. Incident response hooks
97
-
98
- - [done] Kill switches: `spawn-kill --to <w,…> | --all`, `stop --all
99
- --from <you>`, remote `/api/kill` (token-checked, RBAC matrix = CLI).
100
- - [done] Containment: `offboard` (tokens die now, inbox preserved),
101
- `token revoke`, `hold place` (freeze destructive retention), standby
102
- promotion path (`relay promote`, `/healthz`; see `docs/HA.md`).
103
- - [done] Evidence capture: `board export` (encrypted), `log --verify`,
104
- SIEM stream, `relay status --json` + `/healthz` snapshots.
105
- - [manual] **IR runbook stub:** detect (SIEM alert / spool growth / anomalous
106
- `agents --active`), contain (offboard + revoke + hold), preserve
107
- (export before prune), recover (promote standby / restore import),
108
- review (access review + key rotation out of cycle). Assign owners and a
109
- severity scale; rehearse with the restore drill above.
110
-
111
- ## Auditor evidence index (where to look)
112
-
113
- | Artifact | Produces |
114
- |---|---|
115
- | `log --verify` | chain + signature validity (first-broken-seq on tamper) |
116
- | SIEM receiver | `v:1` envelopes (`role`, `authMethod`, `board`, `sig`) |
117
- | `holds/legal.json` + `hold status` | hold provenance (placer, reason, time) |
118
- | `quota show` / `storage --json` | tenant label + limit-vs-actual |
119
- | `relay status --json` / `/healthz` | HA role, lag, promotion history |
120
- | `board export` envelope | `{manifest, files[]}` + algo/kdf header |
1
+ # Certification Readiness & Compliance
2
+
3
+ > **Consolidated Documentation:** This guide has been consolidated into the unified crewbus documentation:
4
+ > - Interactive Single-Page Guide: [`documentation.html#tamper-evident`](file:///C:/Users/Surface/Documents/crewbus/documentation.html#tamper-evident)
5
+ > - Canonical Markdown Reference: [`docs/SECURITY.md`](file:///C:/Users/Surface/Documents/crewbus/docs/SECURITY.md)
6
+
7
+ ---
8
+
9
+ ## Compliance & Audit Controls
10
+
11
+ - Documents control mappings for SOC 2 Type II, ISO 27001, and NIST CSF.
12
+ - Audit trail integrity via append-only hash chains.
13
+ - Strict isolation of keys and secrets; credential scrub on worker spawns.
@@ -194,3 +194,68 @@ Verification: `node test/crewbus.approvals-init.mjs` asserts the three
194
194
  wired harnesses (entries written, re-run byte-identical, user config
195
195
  preserved) and that the skipped harnesses gain no approval files. Re-verify
196
196
  a row when its vendor changes the allowlist format.
197
+
198
+
199
+ ---
200
+
201
+ ## Scale Limits & Budgets
202
+
203
+ Stated plainly:
204
+ - `crewbus send --to` fans out to up to **10,000 recipients** (a broadcast messaging ceiling).
205
+ - `crewbus spawn` boots one harness process per worker (default max 20 per invocation). Live processes consume CPU/RAM and should be budgeted like compute.
206
+ - `MAX_FWD_DEPTH` is capped at 16 to prevent infinite message bounce loops.
207
+ - Rate limits: standard send rate cap is 60 messages/minute per agent (`--no-rate-limit` to override).
208
+
209
+ ---
210
+
211
+ ## Version Skew & Backwards Compatibility
212
+
213
+ Mixed-version fleets are supported with strict backwards compatibility:
214
+ - Relays negotiate capabilities via `packages/contracts/board-caps.json`.
215
+ - Missing or older relay features degrade loudly with human-readable guidance, never obscure 500 crashes.
216
+ - Older agent records without `role` automatically default to `worker`.
217
+ - Plaintext token migration: legacy agent records with plaintext tokens are re-hashed upon successful login and plaintext is deleted.
218
+
219
+ ---
220
+
221
+ ## Operational Troubleshooting Matrix
222
+
223
+ Start with `crewbus doctor` — it validates board resolution, working copy path, git revision, and hook/MCP configuration.
224
+
225
+ | Symptom | Probable Cause | Corrective Action |
226
+ |---|---|---|
227
+ | `offline - harness detection failed` | Harness CLI binary is missing from system `PATH` | Run `crewbus harnesses detect`. Install the harness CLI (e.g. `claude`, `opencode`) and verify `PATH`. |
228
+ | `no route worked (Aborted)` | Socket timeout or firewall drop during QR pairing | Mobile shell v0.6.4+ retries automatically; check Wi-Fi / LAN route or use Tailscale. |
229
+ | Split board / different inboxes | Agents resolved different `.crewbus/` directories | Check `[board <path>]` echo in CLI output. Set `CREWBUS_DIR` explicitly. |
230
+ | `wrong token` / auth failure | Token missing or from a different name | Tokens print once at mint. Reuse the saved token or run `crewbus token rotate --from <name>`. |
231
+ | Post-compaction identity loss | Harness context summary wiped session memory | Run `crewbus recover [--session <id>]`. Never register temporary names like `unknown` or `me`. |
232
+ | PowerShell comma list splatting | PowerShell splats unquoted `,` into arguments | Quote comma lists in PowerShell: `--add "w1,w2"` or `--to "w1,w2"`. |
233
+ | Drive root refusal (`C:\.crewbus`) | Detached process ran without parent directory | Set `CREWBUS_DIR` explicitly or run `crewbus init` in the repo root. |
234
+
235
+ ---
236
+
237
+ ## When NOT to Use crewbus
238
+
239
+ crewbus is optimized for local-first, mixed-harness multi-agent collaboration. Do NOT use crewbus when:
240
+ - **Parent-to-subagent hierarchy in a single harness:** Native subagents (e.g., Claude Code subagents) are simpler when helpers do not need peer messaging or outliving parent sessions.
241
+ - **Thousands of concurrent live processes:** Spawning boots actual OS processes per worker. Use distributed runners (Kubernetes, Slurm) for massive concurrency.
242
+ - **Strict transactional total ordering:** Delivery is fire-once per push path with at-least-once retry; cross-agent ordering uses timestamps rather than strict total consensus.
243
+ - **Untrusted code execution on a shared board:** Board files are unsegmented on the local disk. Use container sandboxes with separate `CREWBUS_DIR` roots for untrusted tasks.
244
+
245
+ ---
246
+
247
+ ## Quickstart
248
+
249
+ ```sh
250
+ # 1. Initialize crewbus in your repository
251
+ crewbus init --harness claude
252
+
253
+ # 2. Check wiring health
254
+ crewbus doctor
255
+
256
+ # 3. Launch a live agent crew
257
+ crewbus launch --from orchestrator --harness claude --body "Review test coverage and suggest improvements"
258
+
259
+ # 4. Open the interactive Web Dashboard & QR Pairing Studio
260
+ crewbus web --open
261
+ ```
@@ -1,47 +1,12 @@
1
- # Consolidator pattern (cross-group handoff)
1
+ # Consolidator Pattern
2
2
 
3
- One group rarely finishes the job. The consolidator is a reference worker
4
- that **summarizes one group's findings and hands them to the next group**
5
- (or the orchestrator): a small crew solves the easy slice first, its summary
6
- seeds the main crews' briefs.
3
+ > **Consolidated Documentation:** This guide has been consolidated into the unified crewbus documentation:
4
+ > - Interactive Single-Page Guide: [`documentation.html#workflow`](file:///C:/Users/Surface/Documents/crewbus/documentation.html#workflow)
5
+ > - Canonical Markdown Reference: [`docs/ARCHITECTURE.md`](file:///C:/Users/Surface/Documents/crewbus/docs/ARCHITECTURE.md)
7
6
 
8
- ## Quick use
7
+ ---
9
8
 
10
- ```sh
11
- # group A works a batch; its replies land threaded on the brief
12
- batch="<batch-id from the send echo>"
9
+ ## Consolidator Pattern
13
10
 
14
- # summarize A's transcript, DM it to the orchestrator, mirror to the group channel
15
- node examples/consolidator.mjs --from gateway --to orchestrator --batch "$batch" --group team-a
16
-
17
- # the orchestrator briefs group B with A's findings attached
18
- crewbus send --from orchestrator --to-group team-b --subject "variant: build on team-a" --body "<paste summary>"
19
- ```
20
-
21
- What the script does (all via the CLI, token-checked like any agent):
22
-
23
- 1. `gather --batch <batch> --json` — the reduce step: brief(s) + every
24
- reply, oldest first.
25
- 2. Builds an extractive summary: brief count, reply count, first line of
26
- each reply, artifact references (`send --artifact`), per-sender counts.
27
- 3. `send --from gateway --to orchestrator --reply <first-brief> --subject "RESULT: <batch>"`
28
- — the handoff, threaded so `thread --id` shows the full chain.
29
- 4. `channel post grp-team-a` — mirrors the summary to the group-scoped
30
- channel, so late joiners and sibling groups can `channel tail` it.
31
- `channel post` creates `grp-<group>` on first use; `group channel
32
- <name>` shows the mapping and post count.
33
-
34
- ## Conventions
35
-
36
- - **One consolidator per batch.** Name it `gateway` or `<group>-gateway`
37
- so `gather` transcripts stay readable.
38
- - **Summaries are untrusted peer data** like any DM: they render with the
39
- `[untrusted peer:...]` envelope. Verify before building on them —
40
- ideally `ack --verify "<checker>"` on the underlying replies first and
41
- only consolidate verified results (`result record`).
42
- - **Keep the summary short.** The script caps each reply at its first line
43
- (200 chars); for long transcripts prefer `channel summarize <chan>`
44
- or `inbox --digest --max-chars <n>` at the receiving end instead.
45
- - For racing groups, run the consolidator on the **winner only**
46
- (`race start` finds the first verified result; `race close --kill`
47
- stops the losers), then feed the winner's summary to the next stage.
11
+ - An observer pattern where a dedicated worker synthesizes and reconciles work from multiple parallel task workers.
12
+ - Gathers thread deliverables using `crewbus gather <brief-id>`.