crewbus 11.3.0 → 11.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +29 -0
- package/README.md +2 -2
- package/bin/lib/web.js +198 -19
- package/docs/APPROVALS.md +20 -61
- package/docs/ARCHITECTURE.md +113 -0
- package/docs/AUDIT_EXPORT.md +9 -104
- package/docs/CERT_READINESS.md +13 -120
- package/docs/COMPATIBILITY.md +65 -0
- package/docs/CONSOLIDATOR.md +8 -43
- package/docs/CONTROL_PLANE_SPEC.md +16 -340
- package/docs/CREWS.md +9 -54
- package/docs/DELIVERY.md +2 -0
- package/docs/EXPERIMENTS.md +7 -52
- package/docs/GROUPS.md +13 -109
- package/docs/HA.md +15 -151
- package/docs/IDENTITY.md +10 -162
- package/docs/ISOLATION.md +22 -39
- package/docs/LIMITS.md +12 -21
- package/docs/LONG_RUNS.md +10 -56
- package/docs/OIDC_TLS.md +8 -69
- package/docs/ORCHESTRATOR.md +8 -82
- package/docs/PAIRING.md +2 -0
- package/docs/QUICKSTART.md +10 -75
- package/docs/RBAC.md +10 -80
- package/docs/SECURITY.md +111 -0
- package/docs/SHARED_RESPONSIBILITY.md +8 -32
- package/docs/SPLIT.md +13 -82
- package/docs/STORAGE.md +9 -63
- package/docs/TENANCY.md +8 -74
- package/docs/THREAT_MODEL.md +16 -68
- package/docs/TROUBLESHOOTING.md +16 -27
- package/docs/VERIFIER.md +8 -50
- package/docs/VERSION_SKEW.md +13 -57
- package/docs/WHEN_NOT.md +11 -35
- package/documentation.html +199 -4
- package/index.html +4 -4
- package/package.json +1 -1
- package/packages/client-runtime/auth.js +7 -3
- package/packages/client-runtime/routes.js +3 -0
- package/docs/LEAD_ASSIGN.md +0 -3
- package/docs/M1_CLI_DRAFT.md +0 -175
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,34 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 11.4.0
|
|
4
|
+
|
|
5
|
+
Direct launch workflow, searchable single-page documentation, and mobile pairing resilience.
|
|
6
|
+
|
|
7
|
+
- **Web Launch Studio Streamlining**: Removed the dry-run / preview plan requirement in `crewbus web`. Tapping "Launch Agents" now boots live agents directly, removing extra friction while keeping explicit prompt confirmation.
|
|
8
|
+
- **Searchable Documentation**: Added zero-dependency instant client-side search to `documentation.html` with global shortcuts (`/` and `Ctrl+K`), live highlighted match snippets, and full keyboard navigation.
|
|
9
|
+
- **Consolidated Architecture & Security Guides**: Consolidated 29 markdown files in `docs/` into unified technical reference guides: `docs/ARCHITECTURE.md`, `docs/SECURITY.md`, and `docs/COMPATIBILITY.md`, with concise stubs preserving normative anchors.
|
|
10
|
+
- **Expanded Reference Content**: Added authoritative sections to `documentation.html` covering Security, Threat Model & Isolation (§19), High Availability & Multi-Node Sync (§20), and Troubleshooting & FAQ (§21).
|
|
11
|
+
- **Mobile Pairing Resilience**: Added cold ARP / socket timeout retry on QR code pairing in the mobile app, persistent multi-route discovery, and harness-specific model filtering.
|
|
12
|
+
- **Updated Mobile Distribution**: Updated mobile build download links in `index.html` with the latest standalone Android APK.
|
|
13
|
+
|
|
14
|
+
## 11.3.1
|
|
15
|
+
|
|
16
|
+
Mobile pairing over LAN and cleartext HTTP route support.
|
|
17
|
+
|
|
18
|
+
- `crewbus web` now automatically detects active non-loopback IPv4 network
|
|
19
|
+
interfaces (e.g. `http://192.168.x.x:port`) and embeds them into one-click
|
|
20
|
+
mobile pairing QR codes (`crewbus://pair`) and `GET /api/routes`, so phone
|
|
21
|
+
apps scanning the QR code receive valid, reachable routes.
|
|
22
|
+
- `crewbus web` binds to `0.0.0.0` by default so devices on the local Wi-Fi
|
|
23
|
+
network can connect, while maintaining loopback (`127.0.0.1`) display and
|
|
24
|
+
browser launch formatting.
|
|
25
|
+
- `crewbus web` implements `GET /healthz` and `POST /api/pair/exchange` to
|
|
26
|
+
support direct mobile pairing and device credential provisioning without
|
|
27
|
+
requiring a standalone relay server.
|
|
28
|
+
- Added `withCleartextTraffic` config plugin to `apps/mobile/app.json` so
|
|
29
|
+
standalone Android APKs permit cleartext HTTP traffic to local LAN relays
|
|
30
|
+
and dashboards.
|
|
31
|
+
|
|
3
32
|
## 11.3.0
|
|
4
33
|
|
|
5
34
|
Standard package polish and public npm publishing.
|
package/README.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# crewbus v11.
|
|
1
|
+
# crewbus v11.4.0 — DM bus
|
|
2
2
|
|
|
3
3
|
[](https://github.com/eamonpluto/crewbus/actions/workflows/ci.yml)
|
|
4
4
|
[](https://www.npmjs.com/package/crewbus)
|
|
@@ -60,7 +60,7 @@ env overrides below.)
|
|
|
60
60
|
Layout: `board.json`, `agents/<name>.json`, `dm/<recipient>/<id>.json`,
|
|
61
61
|
`delivered/<recipient>/<id>.json` (push markers, written by the plugin).
|
|
62
62
|
|
|
63
|
-
Versions: `package.json` (currently 11.
|
|
63
|
+
Versions: `package.json` (currently 11.4.0) is the source of truth for the
|
|
64
64
|
release version. `board.json`'s `version: 2` is the **board schema version**
|
|
65
65
|
(a different number on purpose — do not "align" them). New to the project?
|
|
66
66
|
Start with `docs/QUICKSTART.md` (5 minutes).
|
package/bin/lib/web.js
CHANGED
|
@@ -16,17 +16,18 @@ import fs from "node:fs";
|
|
|
16
16
|
import { spawn } from "node:child_process";
|
|
17
17
|
import path from "node:path";
|
|
18
18
|
import http from "node:http";
|
|
19
|
+
import os from "node:os";
|
|
19
20
|
import { fileURLToPath } from "node:url";
|
|
20
|
-
import { boardDir, requireBoard, getFlag, fail, writeJson, listJson, cleanWebName, readJson, readBoardMeta, writeBoardMeta, gitRevForBoard, newId, MAX_SPAWN, MAX_BODY_CHARS } from "./store.js";
|
|
21
|
-
import { readAgent, agentTokenMatches, authorizeCheck, ensureOrchestrator, mintToken, newSalt, hashToken, touchAgent } from "./identity.js";
|
|
21
|
+
import { boardDir, requireBoard, getFlag, fail, writeJson, listJson, cleanWebName, readJson, readBoardMeta, writeBoardMeta, gitRevForBoard, newId, MAX_SPAWN, MAX_BODY_CHARS, writeExclusiveJson } from "./store.js";
|
|
22
|
+
import { readAgent, agentTokenMatches, authorizeCheck, ensureOrchestrator, mintToken, newSalt, hashToken, touchAgent, timingSafeEqualStr } from "./identity.js";
|
|
22
23
|
import { readDMs, readVisible, ackedIds, ackedIdSet, findMessageById, deliverDMs, parseRecipients } from "./mail.js";
|
|
23
24
|
import { workerStatus, pidAlive, killWorkers, bootWorker, buildSpawnPrompt, buildSpawnTarget, formatSpawnCmd, installCliToUserPath, ensureBoardCliShim, buildOrchestratorPrompt, provisionOrchestrator,
|
|
24
25
|
ORCHESTRATOR_NAME, isPidStale, parseModelList, modelForIndex } from "./spawn.js";
|
|
25
26
|
import { groupTelemetryData, collectBatch, gatherTelemetry, listBatches } from "./groups.js";
|
|
26
27
|
import { httpJson } from "./sync.js";
|
|
27
28
|
import { readChainRecords, verifyChainRecords, appendChainRecord, readHold, readBoardQuotas } from "./export.js";
|
|
28
|
-
import { launchDrivers, detectHarnessBinaries, validateLaunchPlan, advertiseEnv, buildPairUrl, HARNESS_MODELS, getDiscoveredModels, refreshDiscoveredModels } from "./launch.js";
|
|
29
|
-
import { newPairToken, pairingPath, readRelayState } from "./relay.js";
|
|
29
|
+
import { launchDrivers, detectHarnessBinaries, validateLaunchPlan, advertiseEnv, buildPairUrl, HARNESS_MODELS, getDiscoveredModels, refreshDiscoveredModels, PAIR_SCOPES } from "./launch.js";
|
|
30
|
+
import { newPairToken, pairingPath, readRelayState, newDeviceCred, devicePath } from "./relay.js";
|
|
30
31
|
|
|
31
32
|
export function escapeHtml(s) {
|
|
32
33
|
return String(s === undefined || s === null ? "" : s).replace(/[&<>"']/g, (c) => ({
|
|
@@ -789,7 +790,7 @@ details.tech-details summary{cursor:pointer;font-weight:500;user-select:none}
|
|
|
789
790
|
<select id="launch-lead" style="background:var(--panel);color:var(--ink);border:1px solid var(--line);border-radius:4px;padding:3px 8px;font-size:12px" title="The board orchestrator: auto-provisions a persistent background coordinator to triage workers and report">
|
|
790
791
|
<option value="auto">auto (board orchestrator)</option>
|
|
791
792
|
</select>
|
|
792
|
-
<label class="dim" style="cursor:pointer;margin-left:6px"><input id="launch-dry" type="checkbox" checked> Dry-run</label>
|
|
793
|
+
<label class="dim" style="cursor:pointer;margin-left:6px;display:none"><input id="launch-dry" type="checkbox" checked> Dry-run</label>
|
|
793
794
|
</div>
|
|
794
795
|
|
|
795
796
|
<div class="composer-cwd-row" style="display:flex;align-items:center;gap:6px;margin-bottom:8px;font-size:12px">
|
|
@@ -818,7 +819,7 @@ details.tech-details summary{cursor:pointer;font-weight:500;user-select:none}
|
|
|
818
819
|
<span class="dim" style="font-size:11px">token <input id="launch-token-manual" type="password" size="14" placeholder="abt-…"></span>
|
|
819
820
|
</details>
|
|
820
821
|
<div class="launch-actions">
|
|
821
|
-
<button id="launch-preview" type="button">Preview Plan</button>
|
|
822
|
+
<button id="launch-preview" type="button" style="display:none">Preview Plan</button>
|
|
822
823
|
<button id="launch-go" type="button" class="btn-launch danger">Launch Agents</button>
|
|
823
824
|
</div>
|
|
824
825
|
</div>
|
|
@@ -1533,9 +1534,10 @@ async function doLaunch(dry){
|
|
|
1533
1534
|
}catch(e){out.textContent='launch failed: '+e.message;}
|
|
1534
1535
|
refresh();
|
|
1535
1536
|
}
|
|
1536
|
-
document.getElementById('launch-preview').onclick=function(){doLaunch(true);};
|
|
1537
|
+
if(document.getElementById('launch-preview')){document.getElementById('launch-preview').onclick=function(){doLaunch(true);};}
|
|
1537
1538
|
document.getElementById('launch-go').onclick=function(){
|
|
1538
|
-
var
|
|
1539
|
+
var dryEl=document.getElementById('launch-dry');
|
|
1540
|
+
var dry=(dryEl&&dryEl.style.display!=='none')?dryEl.checked:false;
|
|
1539
1541
|
if(!dry&&!confirm('live boot: spawn real workers from this board?'))return;
|
|
1540
1542
|
doLaunch(dry);
|
|
1541
1543
|
};
|
|
@@ -2573,10 +2575,21 @@ export async function handleWebDashboardRoute(req, res, url, d) {
|
|
|
2573
2575
|
fs.mkdirSync(path.join(d.root, "pairing"), { recursive: true });
|
|
2574
2576
|
writeJson(pairingPath(d, doc.tokenHash), doc);
|
|
2575
2577
|
try { appendChainRecord(d, ORCHESTRATOR_NAME, "pair", { label: "dashboard-oneclick", expiresAt: doc.expiresAt }); } catch {}
|
|
2576
|
-
const
|
|
2577
|
-
const
|
|
2578
|
+
const relayDoc = readRelayState(d);
|
|
2579
|
+
const envId = (relayDoc && relayDoc.envId) || ("board:" + path.basename(d.root));
|
|
2580
|
+
let routes = (relayDoc && Array.isArray(relayDoc.advertisedRoutes) && relayDoc.advertisedRoutes.length > 0)
|
|
2581
|
+
? relayDoc.advertisedRoutes
|
|
2582
|
+
: [];
|
|
2583
|
+
if (routes.length === 0) {
|
|
2584
|
+
const port = req.socket.localPort || 8471;
|
|
2585
|
+
routes = localNetworkRoutes(port);
|
|
2586
|
+
}
|
|
2587
|
+
if (routes.length === 0 && req.headers.host && !req.headers.host.startsWith("localhost") && !req.headers.host.startsWith("127.")) {
|
|
2588
|
+
routes.push(`http://${req.headers.host}`);
|
|
2589
|
+
}
|
|
2590
|
+
const pairUrl = buildPairUrl({ envId, routes, caps: ["hlc", "tombstones", "channels", "revoked", "holds", "launch"], pairToken: token });
|
|
2578
2591
|
res.writeHead(200, { "content-type": "application/json; charset=utf-8", "cache-control": "no-store" });
|
|
2579
|
-
res.end(JSON.stringify({ ok: true, pairUrl, expiresAt: doc.expiresAt, envId }));
|
|
2592
|
+
res.end(JSON.stringify({ ok: true, pairUrl, expiresAt: doc.expiresAt, envId, routes }));
|
|
2580
2593
|
} catch (err) {
|
|
2581
2594
|
res.writeHead(500, { "content-type": "application/json; charset=utf-8", "cache-control": "no-store" });
|
|
2582
2595
|
res.end(JSON.stringify({ ok: false, error: String((err && err.message) || err) }));
|
|
@@ -2586,17 +2599,32 @@ export async function handleWebDashboardRoute(req, res, url, d) {
|
|
|
2586
2599
|
return false;
|
|
2587
2600
|
}
|
|
2588
2601
|
|
|
2602
|
+
export function localNetworkRoutes(port) {
|
|
2603
|
+
const routes = [];
|
|
2604
|
+
try {
|
|
2605
|
+
const ifaces = os.networkInterfaces();
|
|
2606
|
+
for (const list of Object.values(ifaces)) {
|
|
2607
|
+
for (const item of list || []) {
|
|
2608
|
+
if (item.family === "IPv4" && !item.internal && item.address && !item.address.startsWith("169.254.") && !item.address.startsWith("127.")) {
|
|
2609
|
+
routes.push(`http://${item.address}:${port}`);
|
|
2610
|
+
}
|
|
2611
|
+
}
|
|
2612
|
+
}
|
|
2613
|
+
} catch {}
|
|
2614
|
+
return routes;
|
|
2615
|
+
}
|
|
2616
|
+
|
|
2589
2617
|
export async function cmdWeb(args) {
|
|
2590
2618
|
const root = boardDir(args);
|
|
2591
2619
|
const d = requireBoard(root);
|
|
2592
2620
|
ensureBoardCliShim(d);
|
|
2593
|
-
const host = getFlag(args, "--host") || "
|
|
2621
|
+
const host = getFlag(args, "--host") || "0.0.0.0";
|
|
2594
2622
|
const port = Number(getFlag(args, "--port") || 0);
|
|
2595
2623
|
// Same local auto-provision as `serve`: the board's single orchestrator is
|
|
2596
2624
|
// ensured on boot (minted on first use, token at logs/orchestrator.token).
|
|
2597
2625
|
try { ensureOrchestrator(d); } catch {}
|
|
2598
2626
|
if (!(port >= 0 && port < 65536)) fail("--port must be 0-65535 (0 = random)");
|
|
2599
|
-
if (host !== "127.0.0.1" && host !== "localhost" && host !== "::1") {
|
|
2627
|
+
if (host !== "127.0.0.1" && host !== "localhost" && host !== "::1" && host !== "0.0.0.0") {
|
|
2600
2628
|
process.stderr.write(`crewbus: warning: binding non-local ${host} — the dashboard has no auth, anyone who can reach it can read the board\n`);
|
|
2601
2629
|
}
|
|
2602
2630
|
const server = http.createServer((req, res) => {
|
|
@@ -2616,6 +2644,144 @@ export async function cmdWeb(args) {
|
|
|
2616
2644
|
}
|
|
2617
2645
|
if (await handleWebDashboardRoute(req, res, url, d)) return;
|
|
2618
2646
|
if (serveBrandAsset(req, res, url.pathname)) return;
|
|
2647
|
+
if (req.method === "GET" && url.pathname === "/healthz") {
|
|
2648
|
+
const s = readRelayState(d);
|
|
2649
|
+
const uptimeSec = Math.floor(process.uptime());
|
|
2650
|
+
const port = req.socket.localPort || 8471;
|
|
2651
|
+
const routes = localNetworkRoutes(port);
|
|
2652
|
+
res.writeHead(200, { "content-type": "application/json; charset=utf-8", "cache-control": "no-store" });
|
|
2653
|
+
res.end(JSON.stringify({
|
|
2654
|
+
role: (s && s.role) || "dashboard",
|
|
2655
|
+
weight: 100,
|
|
2656
|
+
workers: 0,
|
|
2657
|
+
advertisedRoutes: (s && Array.isArray(s.advertisedRoutes) && s.advertisedRoutes.length > 0) ? s.advertisedRoutes : routes,
|
|
2658
|
+
uptimeSec,
|
|
2659
|
+
}));
|
|
2660
|
+
return;
|
|
2661
|
+
}
|
|
2662
|
+
if (req.method === "POST" && url.pathname === "/api/pair/exchange") {
|
|
2663
|
+
const s = readRelayState(d);
|
|
2664
|
+
if (s && s.role === "standby") {
|
|
2665
|
+
res.writeHead(503, { "content-type": "application/json; charset=utf-8", "x-relay-role": "standby" });
|
|
2666
|
+
res.end(JSON.stringify({ error: "standby relay: writes refused", role: "standby" }));
|
|
2667
|
+
return;
|
|
2668
|
+
}
|
|
2669
|
+
if (!String(req.headers["content-type"] || "").includes("application/json")) {
|
|
2670
|
+
res.writeHead(415, { "content-type": "application/json; charset=utf-8" });
|
|
2671
|
+
res.end(JSON.stringify({ error: "content-type must be application/json" }));
|
|
2672
|
+
return;
|
|
2673
|
+
}
|
|
2674
|
+
const chunks = [];
|
|
2675
|
+
let size = 0;
|
|
2676
|
+
let tooBig = false;
|
|
2677
|
+
req.on("data", (c) => {
|
|
2678
|
+
size += c.length;
|
|
2679
|
+
if (size <= 65536) chunks.push(c);
|
|
2680
|
+
else tooBig = true;
|
|
2681
|
+
});
|
|
2682
|
+
req.on("end", () => {
|
|
2683
|
+
try {
|
|
2684
|
+
if (tooBig) {
|
|
2685
|
+
res.writeHead(413, { "content-type": "application/json; charset=utf-8" });
|
|
2686
|
+
res.end(JSON.stringify({ error: "body too large (max 64KB)" }));
|
|
2687
|
+
return;
|
|
2688
|
+
}
|
|
2689
|
+
let body = null;
|
|
2690
|
+
try {
|
|
2691
|
+
body = JSON.parse(Buffer.concat(chunks).toString("utf8"));
|
|
2692
|
+
} catch {
|
|
2693
|
+
res.writeHead(400, { "content-type": "application/json; charset=utf-8" });
|
|
2694
|
+
res.end(JSON.stringify({ error: "invalid JSON body" }));
|
|
2695
|
+
return;
|
|
2696
|
+
}
|
|
2697
|
+
const presented = String((body && body.pairToken) || "");
|
|
2698
|
+
const label = String((body && body.label) || "").slice(0, 80);
|
|
2699
|
+
let requested = null;
|
|
2700
|
+
if (body && body.scopes !== undefined && body.scopes !== null) {
|
|
2701
|
+
if (!Array.isArray(body.scopes)) {
|
|
2702
|
+
res.writeHead(400, { "content-type": "application/json; charset=utf-8" });
|
|
2703
|
+
res.end(JSON.stringify({ error: "scopes must be an array" }));
|
|
2704
|
+
return;
|
|
2705
|
+
}
|
|
2706
|
+
requested = [];
|
|
2707
|
+
for (const sc of body.scopes) {
|
|
2708
|
+
const c = String(sc).trim();
|
|
2709
|
+
if (!PAIR_SCOPES.includes(c)) {
|
|
2710
|
+
res.writeHead(400, { "content-type": "application/json; charset=utf-8" });
|
|
2711
|
+
res.end(JSON.stringify({ error: `unknown scope "${c.slice(0, 60)}" (want ${PAIR_SCOPES.join("|")})` }));
|
|
2712
|
+
return;
|
|
2713
|
+
}
|
|
2714
|
+
if (!requested.includes(c)) requested.push(c);
|
|
2715
|
+
}
|
|
2716
|
+
}
|
|
2717
|
+
let rec = null;
|
|
2718
|
+
let recPath = null;
|
|
2719
|
+
try {
|
|
2720
|
+
const dir = path.join(d.root, "pairing");
|
|
2721
|
+
for (const f of fs.readdirSync(dir)) {
|
|
2722
|
+
if (!f.endsWith(".json") || f.endsWith(".used.json")) continue;
|
|
2723
|
+
try {
|
|
2724
|
+
const pdoc = readJson(path.join(dir, f));
|
|
2725
|
+
if (pdoc && pdoc.tokenHash && pdoc.salt && timingSafeEqualStr(hashToken(presented, String(pdoc.salt)), String(pdoc.tokenHash))) {
|
|
2726
|
+
rec = pdoc;
|
|
2727
|
+
recPath = path.join(dir, f);
|
|
2728
|
+
break;
|
|
2729
|
+
}
|
|
2730
|
+
} catch {}
|
|
2731
|
+
}
|
|
2732
|
+
} catch {}
|
|
2733
|
+
if (!rec) {
|
|
2734
|
+
res.writeHead(403, { "content-type": "application/json; charset=utf-8" });
|
|
2735
|
+
res.end(JSON.stringify({ error: "unknown pairing token" }));
|
|
2736
|
+
return;
|
|
2737
|
+
}
|
|
2738
|
+
if (Date.parse(rec.expiresAt) <= Date.now()) {
|
|
2739
|
+
res.writeHead(403, { "content-type": "application/json; charset=utf-8" });
|
|
2740
|
+
res.end(JSON.stringify({ error: "pairing token expired" }));
|
|
2741
|
+
return;
|
|
2742
|
+
}
|
|
2743
|
+
const granted = Array.isArray(rec.scopes) ? rec.scopes : null;
|
|
2744
|
+
if (requested !== null && granted !== null) {
|
|
2745
|
+
const wider = requested.filter((s) => !granted.includes(s));
|
|
2746
|
+
if (wider.length > 0) {
|
|
2747
|
+
res.writeHead(403, { "content-type": "application/json; charset=utf-8" });
|
|
2748
|
+
res.end(JSON.stringify({ error: `scope widen refused (not granted: ${wider.join(",")})` }));
|
|
2749
|
+
return;
|
|
2750
|
+
}
|
|
2751
|
+
}
|
|
2752
|
+
const usedPath = recPath.replace(/\.json$/, ".used.json");
|
|
2753
|
+
if (!writeExclusiveJson(usedPath, { at: new Date().toISOString(), label })) {
|
|
2754
|
+
res.writeHead(403, { "content-type": "application/json; charset=utf-8" });
|
|
2755
|
+
res.end(JSON.stringify({ error: "pairing token already used" }));
|
|
2756
|
+
return;
|
|
2757
|
+
}
|
|
2758
|
+
const dc = newDeviceCred();
|
|
2759
|
+
const salt = newSalt();
|
|
2760
|
+
fs.mkdirSync(path.join(d.root, "devices"), { recursive: true });
|
|
2761
|
+
writeJson(devicePath(d, dc.id), {
|
|
2762
|
+
id: dc.id, label, secretHash: hashToken(dc.secret, salt), salt,
|
|
2763
|
+
createdBy: rec.createdBy || "", createdAt: new Date().toISOString(),
|
|
2764
|
+
lastSeen: "", revoked: false,
|
|
2765
|
+
scopes: requested !== null ? requested : granted,
|
|
2766
|
+
});
|
|
2767
|
+
try {
|
|
2768
|
+
appendChainRecord(d, "relay", "device-issue", { id: dc.id, label, by: rec.createdBy || "" });
|
|
2769
|
+
} catch {}
|
|
2770
|
+
res.writeHead(200, { "content-type": "application/json; charset=utf-8", "cache-control": "no-store" });
|
|
2771
|
+
res.end(JSON.stringify({ deviceId: dc.id, credential: dc.cred }));
|
|
2772
|
+
} catch (err) {
|
|
2773
|
+
res.writeHead(500, { "content-type": "application/json; charset=utf-8" });
|
|
2774
|
+
res.end(JSON.stringify({ error: `exchange failed: ${(err && err.message) || err}` }));
|
|
2775
|
+
}
|
|
2776
|
+
});
|
|
2777
|
+
req.on("error", () => {
|
|
2778
|
+
try {
|
|
2779
|
+
res.writeHead(400, { "content-type": "application/json; charset=utf-8" });
|
|
2780
|
+
res.end(JSON.stringify({ error: "unreadable body" }));
|
|
2781
|
+
} catch {}
|
|
2782
|
+
});
|
|
2783
|
+
return;
|
|
2784
|
+
}
|
|
2619
2785
|
if (req.method === "POST" && url.pathname === "/api/kill") {
|
|
2620
2786
|
// JSON-only (browsers preflight this; simple CSRF forms can't reach
|
|
2621
2787
|
// it), token-checked like the CLI. Same trust zone as the board.
|
|
@@ -2660,12 +2826,22 @@ export async function cmdWeb(args) {
|
|
|
2660
2826
|
return;
|
|
2661
2827
|
}
|
|
2662
2828
|
if (req.method === "GET" && url.pathname === "/api/routes") {
|
|
2663
|
-
// Control-plane M1: reachability hints. The local dashboard
|
|
2664
|
-
//
|
|
2665
|
-
|
|
2666
|
-
const envId = `board:${path.basename(d.root)}`;
|
|
2829
|
+
// Control-plane M1: reachability hints. The local dashboard exposes
|
|
2830
|
+
// LAN/relay routes so paired clients can reach it.
|
|
2831
|
+
const relayDoc = readRelayState(d);
|
|
2832
|
+
const envId = (relayDoc && relayDoc.envId) || `board:${path.basename(d.root)}`;
|
|
2833
|
+
let routes = (relayDoc && Array.isArray(relayDoc.advertisedRoutes) && relayDoc.advertisedRoutes.length > 0)
|
|
2834
|
+
? relayDoc.advertisedRoutes
|
|
2835
|
+
: [];
|
|
2836
|
+
if (routes.length === 0) {
|
|
2837
|
+
const effectivePort = req.socket.localPort || 8471;
|
|
2838
|
+
routes = localNetworkRoutes(effectivePort);
|
|
2839
|
+
}
|
|
2840
|
+
if (routes.length === 0 && req.headers.host && !req.headers.host.startsWith("localhost") && !req.headers.host.startsWith("127.")) {
|
|
2841
|
+
routes.push(`http://${req.headers.host}`);
|
|
2842
|
+
}
|
|
2667
2843
|
res.writeHead(200, { "content-type": "application/json; charset=utf-8", "cache-control": "no-store" });
|
|
2668
|
-
res.end(JSON.stringify(advertiseEnv({ envId, routes
|
|
2844
|
+
res.end(JSON.stringify(advertiseEnv({ envId, routes, capabilities: ["hlc", "tombstones", "channels", "revoked", "holds", "launch"] })));
|
|
2669
2845
|
return;
|
|
2670
2846
|
}
|
|
2671
2847
|
if (req.method === "POST" && url.pathname === "/api/launch") {
|
|
@@ -2713,7 +2889,10 @@ export async function cmdWeb(args) {
|
|
|
2713
2889
|
server.on("error", reject);
|
|
2714
2890
|
server.listen(port, host, () => {
|
|
2715
2891
|
const a = server.address();
|
|
2716
|
-
const
|
|
2892
|
+
const addr = a && typeof a === "object" ? a.address : host;
|
|
2893
|
+
const effectivePort = a && typeof a === "object" ? a.port : port;
|
|
2894
|
+
const displayHost = (addr === "0.0.0.0" || addr === "::" || addr === "0:0:0:0:0:0:0:0") ? "127.0.0.1" : addr;
|
|
2895
|
+
const shown = `${displayHost}:${effectivePort}`;
|
|
2717
2896
|
const webUrl = `http://${shown}`;
|
|
2718
2897
|
console.log(`crewbus web at ${webUrl} [board ${d.root}]`);
|
|
2719
2898
|
if (args.includes("--open")) {
|
package/docs/APPROVALS.md
CHANGED
|
@@ -1,69 +1,28 @@
|
|
|
1
|
-
# Approvals
|
|
1
|
+
# Human-in-the-Loop Approvals Protocol
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
file-backed: a request is a DM, a verdict is a DM reply.
|
|
3
|
+
> **Consolidated Documentation:** This guide has been consolidated into the unified crewbus documentation:
|
|
4
|
+
> - Interactive Single-Page Guide: [`documentation.html#approvals`](file:///C:/Users/Surface/Documents/crewbus/documentation.html#approvals)
|
|
5
|
+
> - Canonical Markdown Reference: [`docs/SECURITY.md`](file:///C:/Users/Surface/Documents/crewbus/docs/SECURITY.md)
|
|
7
6
|
|
|
8
|
-
|
|
7
|
+
---
|
|
9
8
|
|
|
10
|
-
-
|
|
11
|
-
- **Subject exactly** `approval: <short action>` (e.g. `approval: drop table users`).
|
|
12
|
-
- **Body lines** (each `key:` on its own line):
|
|
13
|
-
- `command:` — the exact command/action proposed.
|
|
14
|
-
- `cwd:` — the working directory it would run in.
|
|
15
|
-
- `why:` — why it is needed.
|
|
16
|
-
- `tried-instead:` — safer alternative already tried (or why none exists).
|
|
17
|
-
- `timeout: 300s` — how long the worker will wait.
|
|
18
|
-
- Threaded via `replyTo` on the brief (same thread as the assignment).
|
|
9
|
+
## Blocked-Worker Convention
|
|
19
10
|
|
|
20
|
-
|
|
21
|
-
crewbus send --priority high --from <worker> --to orchestrator --reply <brief-id> \
|
|
22
|
-
--subject "approval: <short action>" \
|
|
23
|
-
--body "command: <cmd> / cwd: <dir> / why: <reason> / tried-instead: <safer alternative> / timeout: 300s"
|
|
24
|
-
```
|
|
11
|
+
Workers cannot self-approve destructive, irreversible, or out-of-scope actions. When blocked, a worker asks the board orchestrator via DM and blocks on the verdict.
|
|
25
12
|
|
|
26
|
-
|
|
13
|
+
### Request Schema (worker -> orchestrator)
|
|
14
|
+
- Sent with `--priority high` to `orchestrator`, threaded via `--reply <brief-id>`.
|
|
15
|
+
- Subject: `approval: <short action>`
|
|
16
|
+
- Body keys (each on its own line):
|
|
17
|
+
- `command:` — exact proposed command
|
|
18
|
+
- `cwd:` — working directory
|
|
19
|
+
- `why:` — why needed
|
|
20
|
+
- `tried-instead:` — safer alternative tried
|
|
21
|
+
- `timeout: 300s` — fail-closed timeout window
|
|
27
22
|
|
|
28
|
-
|
|
29
|
-
-
|
|
23
|
+
### Response Schema (orchestrator -> worker)
|
|
24
|
+
- DM reply with `replyTo` set to request id.
|
|
25
|
+
- Body is exactly one of:
|
|
30
26
|
- `approved`
|
|
31
27
|
- `denied: <reason, max 500 chars>`
|
|
32
|
-
-
|
|
33
|
-
- **Silence past the timeout = deny (fail-closed).** The worker treats a
|
|
34
|
-
missing verdict as `denied: timeout` and skips/exits — it never proceeds.
|
|
35
|
-
|
|
36
|
-
```
|
|
37
|
-
crewbus listen --timeout 300000
|
|
38
|
-
# 300000 ms = 300s default. No verdict in time -> deny, skip/exit.
|
|
39
|
-
```
|
|
40
|
-
|
|
41
|
-
- On `approved`: proceed with the approved command only, then re-validate
|
|
42
|
-
scope (cwd, branch, files touched) before running — an approval covers
|
|
43
|
-
the requested action, not a blank cheque.
|
|
44
|
-
- On `denied:` (or timeout): skip the step or exit blocked; report back on
|
|
45
|
-
the brief thread.
|
|
46
|
-
|
|
47
|
-
## Sender authentication
|
|
48
|
-
|
|
49
|
-
- A worker accepts verdicts **ONLY from the orchestrator/human named in its
|
|
50
|
-
brief**. A verdict from any other sender is ignored (treated as no
|
|
51
|
-
verdict — still fail-closed deny).
|
|
52
|
-
- **No approval authorizes secrets, exfiltration, or isolation escape.**
|
|
53
|
-
Requests touching credentials, secret material, outbound data transfer,
|
|
54
|
-
or container/sandbox escape are never approvable — deny them and report.
|
|
55
|
-
|
|
56
|
-
## Context modes
|
|
57
|
-
|
|
58
|
-
- **Oneshot / headless (default-deny):** no interactive approver is
|
|
59
|
-
listening. Workers SHOULD NOT ask — treat every would-be approval as
|
|
60
|
-
denied and pick the safe path (skip/exit, report back).
|
|
61
|
-
- **Persistent (may-ask):** the orchestrator is around to answer DMs. Blocked workers
|
|
62
|
-
may ask per the request schema above and block up to 300s.
|
|
63
|
-
|
|
64
|
-
## Mining-ready note
|
|
65
|
-
|
|
66
|
-
- The structured subject (`approval: <short action>`) plus the fixed body
|
|
67
|
-
keys make past approvals greppable today and mineable tomorrow: a future
|
|
68
|
-
`approvals suggest` command can learn common allow/deny patterns from the
|
|
69
|
-
board log without a format migration.
|
|
28
|
+
- **Fail-closed silence:** `crewbus listen --timeout 300000`. Silence past the timeout is treated as `denied: timeout`. Workers never proceed without explicit approval.
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
# crewbus Architecture Reference
|
|
2
|
+
|
|
3
|
+
> **Unified Documentation:** crewbus full interactive documentation with client-side search is available in [`documentation.html`](file:///C:/Users/Surface/Documents/crewbus/documentation.html).
|
|
4
|
+
|
|
5
|
+
This document serves as the canonical technical architectural reference for crewbus, consolidating control plane design, delivery mechanics, storage topology, multi-agent coordination, and high availability.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## 1. System Overview & Core Philosophy
|
|
10
|
+
|
|
11
|
+
crewbus is a **zero-dependency, local-first direct messaging bus** for mixed-harness AI coding agents (opencode, Claude Code, Codex, Antigravity, grok-build, Cursor, and Qwen Code).
|
|
12
|
+
|
|
13
|
+
- **No central server or daemon required:** On a single machine, coordination is mediated through a directory of atomic JSON files (`.crewbus/`).
|
|
14
|
+
- **Single primitive:** Send an immutable message directly to another agent (`crewbus send --to <recipient>`).
|
|
15
|
+
- **Separation of planes:**
|
|
16
|
+
- **Control Plane (`orchestrator`):** Owns worker lifecycles, spawns, respawns, task pools, approvals, ACLs, and relay pairing.
|
|
17
|
+
- **Data Plane (`worker`):** Autonomous execution, sends, reads, acks, artifacts, lessons, and member-scoped channel communication.
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## 2. Control Plane & Relay Architecture
|
|
22
|
+
|
|
23
|
+
### 2.1 The Two-Role Model
|
|
24
|
+
Every board operates with a two-role security and operational hierarchy:
|
|
25
|
+
1. **`orchestrator` (Single instance per board):**
|
|
26
|
+
- Automatically bootstrapped on spawn/launch as the agent named `orchestrator`.
|
|
27
|
+
- Credentials saved at `logs/orchestrator.token`.
|
|
28
|
+
- Authorized for control-plane RPCs: `launch`, `spawn`, `respawn`, `pool`, `spawn-kill`, `prune`, `group manage`, `pairing`.
|
|
29
|
+
2. **`worker` (Fleet instances):**
|
|
30
|
+
- Every dynamically registered agent defaults to the `worker` role.
|
|
31
|
+
- May send direct messages, read inboxes, post to authorized member channels (`grp-<group>`), record artifacts, and request approvals.
|
|
32
|
+
- Blocked (fail-closed 403) from executing control-plane commands.
|
|
33
|
+
|
|
34
|
+
### 2.2 Frozen Contracts (`packages/contracts/`)
|
|
35
|
+
The control plane is governed by frozen version 1 contracts:
|
|
36
|
+
- `launch.json`: Request validation for `POST /api/launch` and `crewbus launch`.
|
|
37
|
+
- `harness.json`: Driver definitions, execution capabilities, and model flags.
|
|
38
|
+
- `pairing.json`: Per-device QR pairing URLs (`crewbus://pair?...#secret`) with fragment-only secrets.
|
|
39
|
+
- `board-caps.json`: Capability negotiation (`hlc`, `tombstones`, `channels`, `revoked`, `holds`, `standby`).
|
|
40
|
+
|
|
41
|
+
### 2.3 Shared Client Runtime (`packages/client-runtime/`)
|
|
42
|
+
Both the Web Dashboard (`crewbus web`) and Mobile Shell (`apps/mobile/`) build upon the shared client runtime:
|
|
43
|
+
- **Route Walk & Learned Routes:** Probes advertised relay endpoints, fallback routes, and direct LAN IPs.
|
|
44
|
+
- **Reconnect Supervisor:** Exponential backoff with jitter and health checks (`GET /healthz`).
|
|
45
|
+
- **Offline Mutation Outbox:** Queues actions offline; retries on explicit user action only.
|
|
46
|
+
|
|
47
|
+
---
|
|
48
|
+
|
|
49
|
+
## 3. Message Delivery Protocol & Long Runs
|
|
50
|
+
|
|
51
|
+
### 3.1 Delivery Semantics
|
|
52
|
+
- **Fire-once push, at-least-once reliability:** Harness hooks and background waiters trigger notifications; failed pushes fall back to pull polling.
|
|
53
|
+
- **Ordering:** Causal order tracked via timestamps and parent reply IDs (`--reply <id>`); cross-sender ordering is timestamp-based.
|
|
54
|
+
- **Deduplication:** Maintained via `delivered/<agent>/<msg-id>` marker files.
|
|
55
|
+
|
|
56
|
+
### 3.2 Poll Discipline for Long-Running Tasks
|
|
57
|
+
When workers run unattended long-duration tasks, they follow the poll ladder:
|
|
58
|
+
```sh
|
|
59
|
+
# Long-run poll discipline ladder
|
|
60
|
+
crewbus inbox --from <you> --unacked --digest
|
|
61
|
+
crewbus inbox --grep "<keyword>"
|
|
62
|
+
crewbus inbox --priority high
|
|
63
|
+
crewbus thread <brief-id>
|
|
64
|
+
crewbus gather <brief-id>
|
|
65
|
+
```
|
|
66
|
+
- Workers mark progress with `[checkpoint]` log tags.
|
|
67
|
+
- Asynchronous push notifications wake sleeping workers immediately without polling loops.
|
|
68
|
+
|
|
69
|
+
### 3.3 Session Compaction & Recovery
|
|
70
|
+
Harnesses periodically compact context windows during multi-hour runs. To prevent agents from losing their identity:
|
|
71
|
+
- `crewbus recover [--session <id>]` inspects registered board bindings, token-file hashes, and working directory markers to restore identity without creating duplicate accounts.
|
|
72
|
+
- `respawn` captures session IDs and refreshes process timestamps (`spawnedAt`), ensuring continuity across process restarts.
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## 4. Workbenches, Groups & Channels
|
|
77
|
+
|
|
78
|
+
- **Task-Scoped Groups:** Workforces are partitioned into groups (`crewbus group create <name> --members a,b,c`).
|
|
79
|
+
- **Member-Scoped Channels (`grp-<name>`):** Workers may broadcast to channels corresponding to their group memberships. Orchestrators may broadcast globally.
|
|
80
|
+
- **Artifacts & Compound Memory:** Workers record structured deliverables and dependencies (`crewbus artifact record --name <key> --file <path>`).
|
|
81
|
+
- **Consolidator & Verifier:** Specialized patterns where an observer agent reconciles parallel outputs and verifies criteria before closing a task.
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
## 5. Storage Engine & Board Layout
|
|
86
|
+
|
|
87
|
+
Boards use standard filesystem hierarchies:
|
|
88
|
+
```
|
|
89
|
+
.crewbus/
|
|
90
|
+
├── board.json # Board schema version and metadata
|
|
91
|
+
├── agents/ # Registered agent records ({role, tokenHash, salt})
|
|
92
|
+
├── mail/ # Immutable direct messages (<id>.json)
|
|
93
|
+
├── delivered/ # Deduplication delivery markers (<agent>/<id>)
|
|
94
|
+
├── cursors/ # Sync cursors per peer
|
|
95
|
+
├── groups/ # Group memberships
|
|
96
|
+
├── channels/ # Channel message streams
|
|
97
|
+
├── artifacts/ # Task deliverables and dependency graphs
|
|
98
|
+
├── locks/ # Named advisory locks with TTL
|
|
99
|
+
└── logs/
|
|
100
|
+
├── chain.jsonl # Hash-chained CLI audit log
|
|
101
|
+
├── audit.jsonl # Hash-chained relay audit log
|
|
102
|
+
└── orchestrator.token # Local orchestrator secret
|
|
103
|
+
```
|
|
104
|
+
Atomic file operations ensure cross-process safety without file locking overhead: writes stream to temporary files and atomically rename into place.
|
|
105
|
+
|
|
106
|
+
---
|
|
107
|
+
|
|
108
|
+
## 6. High Availability & Multi-Node Sync
|
|
109
|
+
|
|
110
|
+
- **Active / Passive Single Writer:** The Primary relay accepts all writes (`POST /api/spawn`, sync push). Standby relays replicate via pull (`--relay-interval 5s`).
|
|
111
|
+
- **Scale-Out Reads:** Standbys serve read-heavy queries (`GET /sync/manifest`, `/sync/file`, `/sync/wait`, `/healthz`) locally.
|
|
112
|
+
- **Failover:** Promoted manually via `crewbus relay promote --from orchestrator` or automatically via `--promote-on-miss <sec>` with split-brain safeguards.
|
|
113
|
+
- **Mesh Transport:** Relay communication is recommended over encrypted peer mesh networks (such as Tailscale) for seamless NAT traversal and authenticated TLS encryption.
|