ruvnet-brain 2.5.1 β†’ 2.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -4,7 +4,7 @@
4
4
 
5
5
  # 🧠 RuvNet Brain
6
6
 
7
- ### 🧠 RuvNet Brain β€” [![RuvNet Brain version 2.5.1 β€” updated 2026-07-12 21:11 EDT](https://img.shields.io/badge/version_2.5.1-updated_2026--07--12_21:11_EDT-1E90FF?style=for-the-badge&labelColor=0757BA)](https://github.com/stuinfla/ruvnet-brain/blob/main/plugin/.claude-plugin/plugin.json)
7
+ ### 🧠 RuvNet Brain β€” [![RuvNet Brain version 2.8.0 β€” updated 2026-07-14 05:59 EDT](https://img.shields.io/badge/version_2.8.0-updated_2026--07--14_05:59_EDT-1E90FF?style=for-the-badge&labelColor=0757BA)](https://github.com/stuinfla/ruvnet-brain/blob/main/plugin/.claude-plugin/plugin.json)
8
8
 
9
9
  **A portable, source-grounded brain over Reuven Cohen's (rUv's) RuvNet stack β€” delivered as a Claude Code plugin that makes Claude _use_ the stack instead of fighting it.**
10
10
 
@@ -77,7 +77,7 @@ So 2.5.1 makes it a **wall, not advice**: a `PreToolUse` gate that **blocks any
77
77
 
78
78
  | | v1 (0.x–1.x) | v2.0 |
79
79
  |---|---|---|
80
- | **Corpus** | 24 repos built | **32 repos** built (of 197 live ruvnet repos), each verified by a live retrieval query |
80
+ | **Corpus** | 24 repos built | **36 repos** built (of 197 live ruvnet repos), each verified by a live retrieval query |
81
81
  | **Depth** (flagship `ruvector`) | 18,491 passages Β· **0** full source bodies | **28,018 passages Β· 2,996 full bodies** β€” depth also restored to `agent-harness-generator` (8,896/715), `ruview` (7,434/765), `open-claude-code` (195/69) |
82
82
  | **Corpus QA gate** | none | every store must prove *embeds correctly + reads correctly* β€” vector count == passage count, depth floors, a 3-passage self-retrieval round-trip per store β€” **72/72 store-variants PASS**, wired fail-closed into the nightly publish |
83
83
  | **Retrieval eval** | 12 frozen questions | **120 frozen, hash-pinned questions** across 5 strata; promotion gated on Wilson lower bounds, fail-closed β€” it blocked a real release this morning, which is the feature working |
@@ -212,7 +212,7 @@ Plus: the **β€œtake the wheel” behavioral pipeline** (below), a **4-level beha
212
212
 
213
213
  ## How it works
214
214
 
215
- The expensive work happens **once, at build time**: every covered repo is deep-walked (whole files, full function bodies, plus a symbol index), embedded into **two** vector variants (MiniLM-384 for edge/portability, bge-768 for depth) stored on-disk in **RVF / HNSW**, and distilled into a concepts + capability layer of per-repo primers and cards. That's **129,037 source chunks**. At **query time**, `search_ruvnet` searches every repo's store at once, pools the hits, and runs them through **one cross-encoder rerank** on a common scale β€” so the truly relevant file wins regardless of which repo it lives in β€” then returns whole source files, each labeled by repo and path.
215
+ The expensive work happens **once, at build time**: every covered repo is deep-walked (whole files, full function bodies, plus a symbol index), embedded into **two** vector variants (MiniLM-384 for edge/portability, bge-768 for depth) stored on-disk in **RVF / HNSW**, and distilled into a concepts + capability layer of per-repo primers and cards. That's **129,685 source chunks**. At **query time**, `search_ruvnet` searches every repo's store at once, pools the hits, and runs them through **one cross-encoder rerank** on a common scale β€” so the truly relevant file wins regardless of which repo it lives in β€” then returns whole source files, each labeled by repo and path.
216
216
 
217
217
  ![RuvNet Brain architecture pipeline](assets/diagrams/architecture-pipeline.svg)
218
218
 
@@ -246,7 +246,7 @@ The brain answers **both** kinds of questions. **Name the repo or ask something
246
246
 
247
247
  ## What it covers
248
248
 
249
- 32 of rUv's repos in the [ruvnet](https://github.com/ruvnet) org β€” the reusable **building blocks** you'd actually compose into a system β€” each deep-walked and embedded in both variants. The core blocks below also carry symbol indexes and capability cards (the 8 newest repos are findable by name; their capability cards are coming).
249
+ 36 of rUv's repos in the [ruvnet](https://github.com/ruvnet) org β€” the reusable **building blocks** you'd actually compose into a system β€” each deep-walked and embedded in both variants. The core blocks below also carry symbol indexes and capability cards (the 8 newest repos are findable by name; their capability cards are coming).
250
250
 
251
251
  ![The RuvNet stack the brain covers](primer/assets/diagrams/ruvnet-stack.svg)
252
252
 
@@ -322,7 +322,7 @@ node forge-ask-all.mjs --dir . --q "How does RuVector implement HNSW vector sear
322
322
 
323
323
  This project versions in the open (see the live badge up top for the exact plugin version; the downloadable knowledge bundle is a separate track) β€” we don't claim β€œdone,” β€œcomplete,” or β€œzero hallucinations.” Where it stands:
324
324
 
325
- - βœ… **The grounding brain is real and proven** β€” 32 repos, 129,037 chunks, dual embeddings, cross-encoder rerank, plugin (MCP tool + enforcement hook + skill), all re-runnable.
325
+ - βœ… **The grounding brain is real and proven** β€” 36 repos, 129,685 chunks, dual embeddings, cross-encoder rerank, plugin (MCP tool + enforcement hook + skill), all re-runnable.
326
326
  - βœ… **Code-level depth** β€” the code-rich repos are indexed to full function bodies; β€œhow is it implemented?” returns the implementation. Verified in the shipped bundle (clean-room 3/3).
327
327
  - βœ… **Routing holds** β€” named 47/48, described 26/28, scenario 7/8; behavioral L1–L4 all pass; private stores fenced out of the public bundle (zero-leak verified).
328
328
  - ⚠️ **Two routing residuals** (above) β€” surfaced, not hidden.
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "ruvnet-brain",
3
- "version": "2.5.1",
4
- "description": "One-command installer for RuvNet Brain \u2014 a portable, source-grounded brain over rUv's RuvNet building blocks, delivered as a Claude Code plugin so Claude uses the stack instead of fighting it.",
3
+ "version": "2.8.0",
4
+ "description": "One-command installer for RuvNet Brain β€” a portable, source-grounded brain over rUv's RuvNet building blocks, delivered as a Claude Code plugin so Claude uses the stack instead of fighting it.",
5
5
  "type": "module",
6
6
  "bin": {
7
7
  "ruvnet-brain": "bin/install.mjs"
@@ -42,7 +42,12 @@
42
42
  "scripts/route-cheap.mjs",
43
43
  "scripts/dispatch-receipt.mjs",
44
44
  "scripts/metaharness-receipts.mjs",
45
- "scripts/codex-routed.sh"
45
+ "scripts/codex-routed.sh",
46
+ "scripts/metaharness-router.mjs",
47
+ "scripts/no-silent-substitution.mjs",
48
+ "scripts/falsify.mjs",
49
+ "scripts/nightly-watchdog.mjs",
50
+ "scripts/job-heartbeat.sh"
46
51
  ],
47
52
  "engines": {
48
53
  "node": ">=18"
@@ -0,0 +1,231 @@
1
+ #!/usr/bin/env node
2
+ // scripts/falsify.mjs β€” THE ADVERSARY. Ask the questions Stuart has to keep asking me.
3
+ //
4
+ // ─────────────────────────────────────────────────────────────────────────────────────────────────
5
+ // WHY (2026-07-13). Stuart: "Why the fuck do you still keep needing me to call you on these things?
6
+ // Ru would not miss stuff like this. You are supposed to be acting like Ru."
7
+ //
8
+ // He is right, and the defect is mechanical, not motivational:
9
+ //
10
+ // I VERIFY WHAT I BUILT. I DO NOT VERIFY WHETHER IT SHOULD EXIST.
11
+ //
12
+ // Tests I wrote passing is circular evidence β€” I wrote them to pass. Every miss tonight has that
13
+ // shape: the router worked (but rUv already shipped one); the jobs were "watched" (but nothing checked
14
+ // they ran); the gong was "complete" (but it never covered liveness). In each case my own artifacts
15
+ // said green, and the only thing that said otherwise was Stuart.
16
+ //
17
+ // Look at how rUv actually writes: every ADR carries an "Honest guardrail" and a measured number, and
18
+ // ADR-043 literally reports a TIE against the baseline rather than dressing it as a win. He starts from
19
+ // "this is probably wrong" and hunts for the thing that would prove it. I start from "this works."
20
+ //
21
+ // So this file is the missing step: BEFORE declaring done, run the questions an adversary would ask.
22
+ // Not a linter for code β€” a linter for CLAIMS.
23
+ //
24
+ // Usage: node scripts/falsify.mjs # run every check; exit 1 if any claim is unproven
25
+ // ─────────────────────────────────────────────────────────────────────────────────────────────────
26
+
27
+ import { spawnSync } from 'node:child_process';
28
+ import fs from 'node:fs';
29
+ import path from 'node:path';
30
+ import { fileURLToPath, pathToFileURL } from 'node:url';
31
+
32
+ const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
33
+ const sh = (cmd, args) => spawnSync(cmd, args, { cwd: ROOT, encoding: 'utf8' });
34
+
35
+ /**
36
+ * Each check is a question Stuart had to ask me, turned into code so he never has to ask it again.
37
+ * A check FAILS when the claim is unproven β€” not when the code is broken. That distinction is the
38
+ * whole point: "it works" and "it should exist / it actually ran / it is really rUv's" are different
39
+ * claims, and only the first one had any gate on it.
40
+ */
41
+ export const CHECKS = [
42
+ {
43
+ id: 'am-i-impersonating-ruv',
44
+ asked: '"You wrote a bunch of code and told me it\'s Ruv\'s code?"',
45
+ why: 'I hand-rolled a router and called it MetaHarness while @metaharness/router sat on npm.',
46
+ run: () => {
47
+ const r = sh('node', ['scripts/no-silent-substitution.mjs']);
48
+ return { ok: r.status === 0, detail: r.status === 0 ? 'no local code wears a rUv tool\'s name' : r.stderr.trim().split('\n').slice(-2).join(' ') };
49
+ },
50
+ },
51
+ {
52
+ id: 'did-the-jobs-actually-run',
53
+ asked: '"Is the nightly actually running, or are you just telling me it is?"',
54
+ why: 'launchd reports exit 0 for a job that NEVER RAN. Silence was being read as health.',
55
+ run: () => {
56
+ const r = sh('node', ['scripts/nightly-watchdog.mjs', '--json']);
57
+ try {
58
+ const { results } = JSON.parse(r.stdout);
59
+ const bad = results.filter((x) => x.state !== 'OK');
60
+ return { ok: bad.length === 0, detail: bad.length ? `${bad.length} job(s) unproven: ${bad.map((b) => `${b.label}=${b.state}`).join(', ')}` : `all ${results.length} jobs produced a fresh successful receipt` };
61
+ } catch { return { ok: false, detail: 'watchdog produced no readable verdict β€” that is itself a failure' }; }
62
+ },
63
+ },
64
+ {
65
+ id: 'is-the-router-actually-routing',
66
+ asked: '"Why am I still burning my Fable/Opus quota if the router exists?"',
67
+ why: 'The router\'s entire lifetime output was 3 test pings and $0.018 β€” because the rule was advisory.',
68
+ run: () => {
69
+ const f = path.join(process.env.HOME, '.claude', 'metaharness', 'routing-receipts.jsonl');
70
+ let rows = [];
71
+ try { rows = fs.readFileSync(f, 'utf8').trim().split('\n').filter(Boolean).map((l) => JSON.parse(l)); } catch { /* none */ }
72
+ const subagent = rows.filter((r) => r.source === 'claude-subagent');
73
+ const saved = rows.reduce((s, r) => s + (r.saved || 0), 0);
74
+ // The honest bar: routing must be VISIBLY happening, not merely possible.
75
+ return {
76
+ ok: subagent.length > 0,
77
+ detail: subagent.length
78
+ ? `${rows.length} routed task(s), ${subagent.length} subagent, est. $${saved.toFixed(2)} saved`
79
+ : 'ZERO subagent dispatches routed β€” the router is decorative again',
80
+ };
81
+ },
82
+ },
83
+ {
84
+ id: 'does-the-repo-ship-what-it-claims',
85
+ asked: '"Are people actually getting the current version?"',
86
+ why: 'Version drift across surfaces is a visible, credibility-destroying mistake.',
87
+ run: () => {
88
+ const r = sh('node', ['scripts/sync-version.mjs', '--check']);
89
+ return { ok: r.status === 0, detail: (r.stdout + r.stderr).trim().split('\n').pop() };
90
+ },
91
+ },
92
+ {
93
+ id: 'does-the-package-ship-what-it-promises',
94
+ asked: '"Are users actually GETTING the thing you told them shipped?"',
95
+ why: 'v2.5.1 headline was "it uses rUv\'s real router" β€” and scripts/metaharness-router.mjs was NOT in the npm tarball. The dependency shipped; the code that calls it did not. Every gate was green and this adversary itself missed it, because it only ever checked whether the REPO was honest β€” never whether the ARTIFACT USERS RECEIVE contains what the README promises.',
96
+ run: () => {
97
+ // The npm `files` whitelist is opt-IN, so a NEW file is invisible to users unless someone
98
+ // remembers to list it. A shipped script that relative-imports an UNSHIPPED one is broken for
99
+ // every user while working perfectly in the repo β€” green CI, green tests, broken product.
100
+ //
101
+ // My FIRST version of this check derived the file list from bin/install.mjs and therefore did
102
+ // NOT catch metaharness-router.mjs β€” the very bug it was written for. It reported βœ… on the
103
+ // broken state. A check that passes on the bug it was written for is worse than no check, so:
104
+ // follow the ACTUAL relative imports of every shipped file, transitively.
105
+ const pkg = JSON.parse(fs.readFileSync(path.join(ROOT, 'package.json'), 'utf8'));
106
+ const shipped = new Set(pkg.files || []);
107
+ const covered = (f) => shipped.has(f) || [...shipped].some((s) => s.endsWith('/') && f.startsWith(s));
108
+
109
+ const seeds = [...shipped].filter((f) => f.endsWith('.mjs') && fs.existsSync(path.join(ROOT, f)));
110
+ const missing = new Set();
111
+ const seen = new Set();
112
+ const walkImports = (rel) => {
113
+ if (seen.has(rel)) return;
114
+ seen.add(rel);
115
+ let src;
116
+ try { src = fs.readFileSync(path.join(ROOT, rel), 'utf8'); } catch { return; }
117
+ for (const m of src.matchAll(/(?:from|import)\s*\(?\s*['"](\.[^'"]+)['"]/g)) {
118
+ const dep = path.posix.normalize(path.posix.join(path.posix.dirname(rel), m[1]));
119
+ if (!fs.existsSync(path.join(ROOT, dep))) continue; // not a real file (or an npm pkg) β€” skip
120
+ if (!covered(dep)) missing.add(`${dep} (imported by ${rel})`);
121
+ walkImports(dep);
122
+ }
123
+ };
124
+ seeds.forEach(walkImports);
125
+
126
+ return {
127
+ ok: missing.size === 0,
128
+ detail: missing.size
129
+ ? `${missing.size} file(s) are IMPORTED BY SHIPPED CODE but are NOT in the npm package β€” the feature is broken for every user: ${[...missing].join('; ')}`
130
+ : `every file imported by shipped code is itself shipped (${seen.size} files reachable from ${shipped.size} whitelist entries)`,
131
+ };
132
+ },
133
+ },
134
+ {
135
+ id: 'does-the-engine-really-use-ruvs-router',
136
+ asked: '"Is the router users actually RUN really rUv\'s, or still your hand-roll?"',
137
+ why: 'v2.5 shipped with the wrapper written, tested, and CI-gated β€” while model-router-engine.mjs, the file that actually executes, never imported it. An honest artifact sitting next to a lying claim is still a lie.',
138
+ run: () => {
139
+ const engine = fs.readFileSync(path.join(ROOT, 'scripts/model-router-engine.mjs'), 'utf8');
140
+ const wired = /metaharness-router\.mjs/.test(engine);
141
+ const declares = /routedBy/.test(engine); // and it must SAY who decided, every time
142
+ return {
143
+ ok: wired && declares,
144
+ detail: wired && declares
145
+ ? 'the engine consults @metaharness/router first and reports routedBy on every decision'
146
+ : `the ENGINE USERS RUN does not ${!wired ? 'import rUv\'s router' : 'declare who decided'} β€” the README\'s claim is false`,
147
+ };
148
+ },
149
+ },
150
+ {
151
+ id: 'does-memory-actually-recall',
152
+ asked: '"Is AgentDB actually storing AND recalling, or is it just small bits?"',
153
+ why: 'I reported AgentDB as broken THREE times. It was never broken. I called `memory search` positionally when the CLI wants -q, then broke my own canary test with a bad grep, and reported MY test defects as PRODUCT defects. This check does the round trip so I can never again mistake my own sloppiness for a broken tool.',
154
+ run: () => {
155
+ // MEMORY_DB override exists so this check can be TESTED AGAINST A BROKEN STATE. A check nobody
156
+ // has watched fail is not a check. (My first two attempts to test it were themselves broken β€”
157
+ // which is the entire reason this check exists.)
158
+ const db = process.env.MEMORY_DB || path.join(ROOT, '.swarm', 'memory.db');
159
+ if (!fs.existsSync(db)) return { ok: false, detail: 'no memory.db β€” memory is not initialised' };
160
+
161
+ // 1. Is it COLLECTING? (rows, embedded, current)
162
+ const q = (sql) => (spawnSync('sqlite3', [db, sql], { encoding: 'utf8' }).stdout || '').trim();
163
+ const total = Number(q('SELECT COUNT(*) FROM memory_entries;'));
164
+ const embedded = Number(q("SELECT SUM(embedding IS NOT NULL AND LENGTH(embedding)>50) FROM memory_entries;"));
165
+ const ageH = (Date.now() - Number(q('SELECT MAX(created_at) FROM memory_entries;'))) / 3600_000;
166
+
167
+ // 2. Does it SURVIVE compaction / session end? (the whole point)
168
+ const pre = Number(q("SELECT COUNT(*) FROM memory_entries WHERE key LIKE 'session-precompact%';"));
169
+ const end = Number(q("SELECT COUNT(*) FROM memory_entries WHERE key LIKE 'session-sessionend%';"));
170
+
171
+ // 3. Does it RECALL? A real semantic round-trip through the REAL CLI β€” not a DB peek.
172
+ const r = spawnSync('npx', ['ruflo@latest', 'memory', 'search', '-q', 'nightly job supervision heartbeat receipts'], {
173
+ cwd: ROOT, encoding: 'utf8', timeout: 120_000,
174
+ });
175
+ const recalled = /Found\s+\d+\s+results/.test(r.stdout || '') && !/Found\s+0\s+results/.test(r.stdout || '');
176
+
177
+ const problems = [];
178
+ if (total < 10) problems.push(`only ${total} entries stored`);
179
+ if (embedded / Math.max(total, 1) < 0.9) problems.push(`only ${embedded}/${total} entries are embedded β€” semantic recall will be partial`);
180
+ if (ageH > 48) problems.push(`newest entry is ${ageH.toFixed(0)}h old β€” capture may have stopped`);
181
+ if (pre === 0) problems.push('ZERO PreCompact snapshots β€” context will NOT survive a compaction');
182
+ if (end === 0) problems.push('ZERO SessionEnd snapshots β€” context will NOT survive a session restart');
183
+ if (!recalled) problems.push('semantic search returned NOTHING β€” recall is dead');
184
+
185
+ return {
186
+ ok: problems.length === 0,
187
+ detail: problems.length
188
+ ? problems.join('; ')
189
+ : `${total} entries (${embedded} embedded), ${pre} PreCompact + ${end} SessionEnd snapshots, semantic recall returns results`,
190
+ };
191
+ },
192
+ },
193
+ {
194
+ id: 'is-ci-actually-green',
195
+ asked: '"Why am I getting failure notifications from GitHub?"',
196
+ why: 'I declared green from LOCAL gates while CI had never passed β€” 25 straight failures.',
197
+ run: () => {
198
+ const r = sh('gh', ['run', 'list', '--workflow', 'ci.yml', '--limit', '1', '--json', 'conclusion,status']);
199
+ try {
200
+ const [run] = JSON.parse(r.stdout);
201
+ if (!run) return { ok: false, detail: 'could not read CI status β€” do not claim green' };
202
+ if (run.status !== 'completed') return { ok: false, detail: `CI is still ${run.status} β€” "pending" is not "green"` };
203
+ return { ok: run.conclusion === 'success', detail: `latest ci run: ${run.conclusion}` };
204
+ } catch { return { ok: false, detail: 'gh unavailable β€” CI state UNKNOWN, which is not the same as green' }; }
205
+ },
206
+ },
207
+ ];
208
+
209
+ export function runAll(checks = CHECKS) {
210
+ return checks.map((c) => ({ ...c, ...c.run() }));
211
+ }
212
+
213
+ function main() {
214
+ console.log('falsify β€” the questions Stuart should not have to keep asking\n');
215
+ const results = runAll();
216
+ for (const r of results) {
217
+ console.log(`${r.ok ? 'βœ…' : '❌'} ${r.asked}`);
218
+ console.log(` ${r.detail}`);
219
+ if (!r.ok) console.log(` why this check exists: ${r.why}`);
220
+ console.log('');
221
+ }
222
+ const failed = results.filter((r) => !r.ok);
223
+ if (failed.length) {
224
+ console.error(`${failed.length} claim(s) UNPROVEN. Do not report success. Fix or say plainly what is not verified.`);
225
+ process.exit(1);
226
+ }
227
+ console.log('Every claim above is proven by something other than my own opinion.');
228
+ process.exit(0);
229
+ }
230
+
231
+ if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) main();
@@ -0,0 +1,77 @@
1
+ #!/bin/sh
2
+ # job-heartbeat.sh β€” wrap a scheduled job so it CANNOT run without leaving proof, and CANNOT fail quietly.
3
+ #
4
+ # WHY (2026-07-13): every scheduled job on this machine was trusted to report on itself, and they
5
+ # didn't. launchd's own exit status is useless as proof β€” a job that has NEVER RUN reports exit 0,
6
+ # identical to one that ran and succeeded. That ambiguity let com.ruvnet.brain-nightly sit unfired
7
+ # and look healthy. Per-job good intentions rot; a wrapper cannot forget.
8
+ #
9
+ # Usage (from a LaunchAgent's ProgramArguments):
10
+ # /bin/sh /path/to/job-heartbeat.sh <label> -- <command> [args...]
11
+ #
12
+ # Guarantees:
13
+ # 1. A "start" receipt is written BEFORE the command runs.
14
+ # 2. An "end" receipt with the REAL exit code is written even if the command dies, is killed, or
15
+ # the machine yanks it away β€” the trap fires on EXIT/INT/TERM. There is no silent death.
16
+ # 3. A non-zero exit pushes an URGENT ntfy alert immediately (topic: $NTFY_TOPIC, or the file
17
+ # ~/.cache/ruvnet-brain/ntfy-topic). No topic = no push, but the receipt is still written.
18
+ # 4. The wrapper's own exit code is the job's exit code β€” launchd still sees the truth.
19
+
20
+ set -u
21
+
22
+ LABEL="${1:?usage: job-heartbeat.sh <label> -- <command...>}"
23
+ shift
24
+ [ "${1:-}" = "--" ] && shift
25
+ [ $# -gt 0 ] || { echo "job-heartbeat: no command given for $LABEL" >&2; exit 2; }
26
+
27
+ HB_DIR="${JOB_HEARTBEAT_DIR:-$HOME/.cache/ruvnet-brain/heartbeats}"
28
+ mkdir -p "$HB_DIR"
29
+ HB="$HB_DIR/$LABEL.json"
30
+
31
+ ts() { date -u +%Y-%m-%dT%H:%M:%SZ; }
32
+ STARTED="$(ts)"
33
+ START_EPOCH="$(date +%s)"
34
+
35
+ # Start receipt. If the job vanishes without ever writing an end receipt, THIS is the evidence that
36
+ # it started and never finished β€” a state the watchdog reports as FAILING, not as silence.
37
+ cat > "$HB" <<EOF
38
+ {"label":"$LABEL","started_at":"$STARTED","state":"running","pid":$$,"command":"$(echo "$@" | sed 's/"/\\"/g')"}
39
+ EOF
40
+
41
+ notify() { # notify <title> <body> <priority>
42
+ topic="${NTFY_TOPIC:-}"
43
+ [ -z "$topic" ] && [ -f "$HOME/.cache/ruvnet-brain/ntfy-topic" ] && topic="$(cat "$HOME/.cache/ruvnet-brain/ntfy-topic")"
44
+ [ -z "$topic" ] && return 0
45
+ curl -sS -m 10 -H "Title: $1" -H "Priority: $3" -H "Tags: rotating_light" -d "$2" "https://ntfy.sh/$topic" >/dev/null 2>&1 || true
46
+ }
47
+
48
+ finish() {
49
+ code=${FORCED_CODE:-$?}
50
+ ended="$(ts)"
51
+ dur=$(( $(date +%s) - START_EPOCH ))
52
+ if [ "$code" -eq 0 ]; then state="ok"; else state="failed"; fi
53
+ cat > "$HB" <<EOF
54
+ {"label":"$LABEL","started_at":"$STARTED","ended_at":"$ended","state":"$state","exit_code":$code,"duration_sec":$dur}
55
+ EOF
56
+ # Gong on failure, immediately β€” not at the next watchdog sweep. A failing nightly should reach the
57
+ # phone while it is still tonight's problem.
58
+ if [ "$code" -ne 0 ]; then
59
+ notify "πŸ”΄ SCHEDULED JOB FAILED: $LABEL" "exit $code after ${dur}s β€” see the job's log. Receipt: $HB" "urgent"
60
+ fi
61
+ exit "$code"
62
+ }
63
+ trap finish EXIT
64
+ # A signal handler must KILL THE CHILD, then exit β€” letting the EXIT trap write the receipt once.
65
+ trap 'FORCED_CODE=143; kill -TERM "$CHILD" 2>/dev/null; exit 143' TERM
66
+ trap 'FORCED_CODE=130; kill -TERM "$CHILD" 2>/dev/null; exit 130' INT
67
+
68
+ # Run the job in the BACKGROUND and `wait` for it β€” do NOT run it in the foreground.
69
+ # Break-test finding (2026-07-13): a POSIX shell blocked on a FOREGROUND child does not run its trap
70
+ # when signalled β€” it dies with the receipt still saying "running", which is precisely the silent
71
+ # death this wrapper exists to prevent. `wait` is interruptible, so the trap fires immediately.
72
+ # The one death nothing can catch is SIGKILL (-9) / power loss, by definition: no handler runs. That
73
+ # case is caught one level up β€” nightly-watchdog.mjs reports a receipt stuck in "running" as FAILING
74
+ # ("started and never finished"). Trap for catchable deaths, watchdog for uncatchable ones.
75
+ "$@" &
76
+ CHILD=$!
77
+ wait "$CHILD"
@@ -43,22 +43,29 @@ export function loadReceipts(file) {
43
43
  }
44
44
 
45
45
  const fmt$ = (n) => `$${n < 0.01 ? n.toFixed(5) : n.toFixed(4)}`;
46
+ // Measured wall-clock, human-readable. '-' when the receipt has no duration β€” never invented.
47
+ const fmtT = (ms) => {
48
+ if (typeof ms !== 'number' || !(ms > 0)) return '-';
49
+ if (ms < 60_000) return `${(ms / 1000).toFixed(1)}s`;
50
+ return `${Math.floor(ms / 60_000)}m${String(Math.round((ms % 60_000) / 1000)).padStart(2, '0')}s`;
51
+ };
46
52
 
47
53
  export function formatTable(rows) {
48
54
  if (!rows.length) return 'No routing receipts yet.\nRoute something cheap first: node scripts/route-cheap.mjs --task "<text>"';
49
55
 
50
56
  // `channel` + `instead of` (2026-07-13): subagent receipts arrived with a per-row baseline β€” the
51
57
  // model that agent WOULD have inherited β€” so a single global "frontier" column would misreport them.
52
- const header = ['date', 'channel', 'task class', 'model used', 'instead of', 'est. cost', 'est. baseline', 'saved'];
58
+ const header = ['date', 'channel', 'task class', 'model used', 'instead of', 'est. cost', 'est. baseline', 'saved', 'time'];
53
59
  const body = rows.map((r) => [
54
60
  (r.ts || '').replace('T', ' ').slice(0, 16),
55
- r.source === 'claude-subagent' ? 'subagent' : 'openrouter',
61
+ r.source === 'claude-subagent' ? 'subagent' : r.source === 'calibration' ? 'calibrate' : 'openrouter',
56
62
  r.task_class || '?',
57
63
  r.model,
58
64
  r.frontier_ref || 'claude-opus-4.8',
59
65
  fmt$(r.est_cost ?? 0),
60
66
  fmt$(r.est_frontier_cost ?? 0),
61
67
  fmt$(r.saved),
68
+ fmtT(r.duration_ms), // measured, never estimated β€” '-' when absent
62
69
  ]);
63
70
  const widths = header.map((h, i) => Math.max(h.length, ...body.map((row) => row[i].length)));
64
71
  const line = (cells) => cells.map((cell, i) => cell.padEnd(widths[i])).join(' ');
@@ -67,19 +74,58 @@ export function formatTable(rows) {
67
74
  const totalFrontier = rows.reduce((s, r) => s + (r.est_frontier_cost || 0), 0);
68
75
  const totalSaved = rows.reduce((s, r) => s + r.saved, 0);
69
76
  const ratio = totalCost > 0 ? (totalFrontier / totalCost).toFixed(1) : '?';
77
+ // Lead with the PERCENTAGE β€” "$1.83" reads as pocket change; "68% cheaper" is the actual
78
+ // message (Stuart, 2026-07-13). Dollars stay for auditability; percent carries the story.
79
+ const pct = totalFrontier > 0 ? Math.round((totalSaved / totalFrontier) * 100) : 0;
70
80
 
71
81
  // Baselines now vary per row; name them all rather than picking one and implying it covers everything.
72
82
  const baselines = [...new Set(rows.map((r) => r.frontier_ref || 'claude-opus-4.8'))].join(', ');
83
+ const timedTotal = rows.reduce((s, r) => s + (typeof r.duration_ms === 'number' && r.duration_ms > 0 ? r.duration_ms : 0), 0);
84
+ const timedRows = rows.filter((r) => typeof r.duration_ms === 'number' && r.duration_ms > 0).length;
85
+ // TIME as a percentage, like cost β€” "20 seconds" is trivia; "~40% faster" is the message
86
+ // (Stuart, 2026-07-13: cheaper AND faster = fundamentally more efficient). Only rows carrying a
87
+ // MEASURED baseline_duration_ms (written by the calibration harness) enter this comparison.
88
+ const paired = rows.filter((r) => r.duration_ms > 0 && r.baseline_duration_ms > 0);
89
+ const pairedRouted = paired.reduce((s, r) => s + r.duration_ms, 0);
90
+ const pairedBase = paired.reduce((s, r) => s + r.baseline_duration_ms, 0);
91
+ const fasterPct = pairedBase > 0 ? Math.round((1 - pairedRouted / pairedBase) * 100) : null;
92
+ // Say what the measurement actually says β€” FASTER, SLOWER, or parity. The 2026-07-13
93
+ // calibration measured cheap tiers at speed PARITY on micro-tasks (startup dominates);
94
+ // a card that only knows how to say "faster" would have lied.
95
+ const speedBadge = fasterPct === null ? ''
96
+ : fasterPct >= 5 ? ` · ⚑ ~${fasterPct}% FASTER (measured, n=${paired.length})`
97
+ : fasterPct <= -5 ? ` · ⏱ ~${-fasterPct}% slower on routed tier (measured, n=${paired.length})`
98
+ : ` · ⚑ speed parity (measured, n=${paired.length})`;
73
99
  const subagents = rows.filter((r) => r.source === 'claude-subagent').length;
74
100
 
101
+ // The card leads with the PERCENTAGE and a spent-vs-unrouted bar β€” "$1.83" reads as pocket
102
+ // change; "68% cheaper", drawn, is the message (Stuart, 2026-07-13). The dollar table stays
103
+ // below for auditability; every number still traces to a receipt row.
104
+ const BAR = 30;
105
+ const withBar = totalFrontier > 0 ? Math.min(BAR, Math.max(1, Math.round(BAR * (totalCost / totalFrontier)))) : BAR;
106
+ const drawBar = (n) => 'β–ˆ'.repeat(n) + 'β–‘'.repeat(BAR - n);
107
+ const rule = '─'.repeat(70);
75
108
  return [
109
+ rule,
110
+ ` πŸ’° SAVED ~${pct}%${speedBadge} Β· ~${ratio}Γ— cheaper Β· ${rows.length} routed task(s) (${subagents} subagent, ${rows.length - subagents} openrouter/calibration)`,
111
+ '',
112
+ ` without routing ${drawBar(BAR)} ${fmt$(totalFrontier)}`,
113
+ ` with MetaHarness ${drawBar(withBar)} ${fmt$(totalCost)} β†’ ~${fmt$(totalSaved)} kept`,
114
+ rule,
76
115
  line(header),
77
116
  line(widths.map((w) => '-'.repeat(w))),
78
117
  ...body.map(line),
79
118
  '',
80
- `${rows.length} routed task(s) (${subagents} subagent, ${rows.length - subagents} openrouter) Β· est. spent ${fmt$(totalCost)} vs ${fmt$(totalFrontier)} unrouted (${baselines}) Β· saved ~${fmt$(totalSaved)} (~${ratio}x cheaper)`,
119
+ `Baselines: ${baselines} β€” the model each task would have run on if it had not been routed.`,
81
120
  'Pricing is live-verified. Token counts are measured OR estimated per row (each row records which, in token_source).',
82
- 'Baseline = the model the task would have run on if it had not been routed.',
121
+ // Time honesty: durations are MEASURED wall-clock of the routed run. A "time saved vs baseline"
122
+ // number requires a measured per-tier speed baseline (the calibration batch) β€” until that exists
123
+ // we state the gap instead of inventing the comparison.
124
+ fasterPct !== null
125
+ ? `Time, measured: routed ${fmtT(pairedRouted)} vs baseline ${fmtT(pairedBase)} on ${paired.length} calibrated task(s) β†’ ${fasterPct >= 5 ? `~${fasterPct}% faster` : fasterPct <= -5 ? `~${-fasterPct}% slower (startup-dominated micro-tasks β€” speed wins come from parallel fan-out and long generations, measured from real dispatches)` : 'speed parity'}. Uncalibrated rows show routed time only.`
126
+ : timedTotal > 0
127
+ ? `Measured time on routed models: ${fmtT(timedTotal)} across ${timedRows} timed task(s). Time-SAVED vs baseline: not yet measured β€” appears after the per-tier calibration run.`
128
+ : 'No measured durations in these receipts yet. Time-saved reporting activates after the per-tier calibration run.',
83
129
  ].join('\n');
84
130
  }
85
131
 
@@ -0,0 +1,158 @@
1
+ #!/usr/bin/env node
2
+ // scripts/metaharness-router.mjs β€” routing through rUv's REAL router: @metaharness/router.
3
+ //
4
+ // ─────────────────────────────────────────────────────────────────────────────────────────────────
5
+ // WHY THIS FILE EXISTS (2026-07-13). I hand-rolled scripts/model-router-engine.mjs β€” 216 lines of my
6
+ // own heuristic with a "documented placeholder policy" β€” and called it "the MetaHarness router
7
+ // engine" in SKILL.md. rUv had ALREADY built and shipped the real thing:
8
+ //
9
+ // @metaharness/router@0.3.2 (ruvnet/agent-harness-generator, packages/router)
10
+ // ADR-040 (DRACO Phase-2) + ADR-043 (KRR training pipeline) β€” status ACCEPTED / IMPLEMENTED.
11
+ // "Cost-optimal model router β€” route each query to the cheapest model that's good enough
12
+ // (k-NN over labelled embeddings). The productized DRACO Phase-2 finding."
13
+ //
14
+ // Building a Claude fake and giving it rUv's name is the single behaviour the brain's own playbook
15
+ // forbids ("NEVER quietly build a Claude fake, call it by the real tool's name, and hide that it's a
16
+ // hand-roll"). This file is the correction: the ROUTING DECISION now comes from rUv's code.
17
+ // ─────────────────────────────────────────────────────────────────────────────────────────────────
18
+ //
19
+ // WHAT COMPOSES, AND WHY IT IS NOT DUPLICATE WORK.
20
+ // @metaharness/router minimises COST subject to predicted QUALITY. It has no concept of "this model
21
+ // is already paid for by THIS user's subscription" β€” nor should it; that is deployment truth, not
22
+ // routing math. So the local piece collapses to exactly one honest job: a PRICE TRANSFORM.
23
+ //
24
+ // a model covered by this user's subscription β†’ costPerMTok = 0
25
+ //
26
+ // Feed that price table to Router.fromExamples() and its cost-optimal logic does the rest, natively:
27
+ // a $0 model that clears the quality bar IS the cheapest candidate. The subscription overlay stops
28
+ // being a competing engine and becomes four lines of price arithmetic. That is the whole fix.
29
+ //
30
+ // THE HONEST CONSTRAINT (stated, not buried): Router predicts quality by k-NN over LABELLED examples
31
+ // (query embedding β†’ the quality that candidate achieved). Those labels come from real routed
32
+ // outcomes. Until enough accumulate, k-NN has nothing to average and its prediction is not
33
+ // meaningful β€” the ADR-040/043 learning curve starts at the bottom. So:
34
+ // β€’ labels < MIN_LABELS β†’ we say COLD-START out loud and fall back, rather than dressing up a
35
+ // guess as a learned prediction. That dressing-up is the original sin.
36
+ // β€’ labels >= MIN_LABELS β†’ rUv's Router makes the call, and we report predictedQuality/metBar.
37
+ // Every routed task appends a label, so this improves with use rather than with my opinions.
38
+
39
+ import fs from 'node:fs';
40
+ import path from 'node:path';
41
+ import os from 'node:os';
42
+ import { fileURLToPath } from 'node:url';
43
+
44
+ const __dirname = path.dirname(fileURLToPath(import.meta.url));
45
+
46
+ // k-NN needs neighbours to average. Below this, a "prediction" is noise wearing a lab coat.
47
+ export const MIN_LABELS = 5;
48
+
49
+ export const OUTCOMES = process.env.MODEL_ROUTER_DECISIONS
50
+ || path.join(os.homedir(), '.claude', 'metaharness', 'routing-outcomes.jsonl');
51
+
52
+ /** Load rUv's router. Returns null (never a fake) if the real package isn't installed. */
53
+ export async function loadRealRouter() {
54
+ try {
55
+ return await import('@metaharness/router');
56
+ } catch {
57
+ return null; // caller MUST say so out loud β€” never silently substitute a hand-roll
58
+ }
59
+ }
60
+
61
+ /**
62
+ * Effective price table for THIS user. The ONLY thing the local layer legitimately contributes:
63
+ * a model the user's subscription already pays for costs $0 at the margin, so the cost-optimal
64
+ * router should treat it as free. Everything else keeps its real blended price.
65
+ */
66
+ export function effectivePrices(candidates, profile) {
67
+ const prices = {};
68
+ for (const c of candidates) {
69
+ const covered = (c.subscription || []).some((h) => profile?.harnesses?.[h]?.subscription === true);
70
+ const blended = typeof c.costPerMTok === 'number'
71
+ ? c.costPerMTok
72
+ : ((c.costIn ?? 0) + (c.costOut ?? 0)) / 2; // blended $/Mtok β€” the axis Router minimises
73
+ prices[c.id] = covered ? 0 : blended;
74
+ }
75
+ return prices;
76
+ }
77
+
78
+ /**
79
+ * Labelled rows in the exact shape Router.fromExamples() consumes β€” the DRACO row shape:
80
+ * { embedding: number[], scores: { [modelId]: quality 0..1 } }
81
+ * Rows without an embedding are unusable for k-NN and are dropped (counted, never faked).
82
+ */
83
+ export function loadLabelledRows(file = OUTCOMES) {
84
+ let raw;
85
+ try { raw = fs.readFileSync(file, 'utf8'); } catch { return { rows: [], unusable: 0 }; }
86
+ const rows = [];
87
+ let unusable = 0;
88
+ for (const line of raw.split('\n')) {
89
+ if (!line.trim()) continue;
90
+ try {
91
+ const o = JSON.parse(line);
92
+ if (Array.isArray(o.embedding) && o.embedding.length && o.scores && Object.keys(o.scores).length) {
93
+ rows.push({ embedding: o.embedding, scores: o.scores });
94
+ } else unusable++;
95
+ } catch { unusable++; }
96
+ }
97
+ return { rows, unusable };
98
+ }
99
+
100
+ /** Embed a prompt with the same pinned local MiniLM-384 the brain uses. No network, no API cost. */
101
+ export async function embed(text) {
102
+ const { pipeline, env } = await import(path.join(__dirname, '..', 'kb', 'node_modules', '@xenova', 'transformers', 'src', 'transformers.js'));
103
+ if (process.env.KB_MODEL_CACHE) { env.cacheDir = process.env.KB_MODEL_CACHE; env.allowRemoteModels = true; }
104
+ const fe = await pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2', {
105
+ quantized: true,
106
+ revision: '751bff37182d3f1213fa05d7196b954e230abad9', // pinned β€” same weights the KB was built with
107
+ });
108
+ const out = await fe(text, { pooling: 'mean', normalize: true });
109
+ return Array.from(out.data);
110
+ }
111
+
112
+ /**
113
+ * THE ROUTE CALL. rUv's Router makes the decision; we supply candidates, this user's prices, and the
114
+ * query embedding. We report exactly what it decided β€” including when it could NOT decide.
115
+ */
116
+ export async function route(prompt, candidates, profile, { qualityBar = 0.7, k = 5 } = {}) {
117
+ const mod = await loadRealRouter();
118
+ if (!mod) {
119
+ return { routedBy: 'UNAVAILABLE', reason: '@metaharness/router is not installed β€” install it rather than hand-rolling a substitute (npm i -g @metaharness/router)' };
120
+ }
121
+
122
+ const { rows, unusable } = loadLabelledRows();
123
+ if (rows.length < MIN_LABELS) {
124
+ // Say it plainly. A k-NN with 1 neighbour is not a learned prediction, and pretending otherwise
125
+ // is precisely the failure this whole file exists to correct.
126
+ return {
127
+ routedBy: 'COLD-START',
128
+ labels: rows.length,
129
+ needed: MIN_LABELS,
130
+ unusable,
131
+ reason: `only ${rows.length} labelled example(s); @metaharness/router needs β‰₯${MIN_LABELS} to predict quality by k-NN. Falling back to the local heuristic β€” and SAYING SO. Every routed task appends a label; this stops being a fallback with use, not with opinions.`,
132
+ };
133
+ }
134
+
135
+ const prices = effectivePrices(candidates, profile);
136
+ const router = mod.Router.fromExamples(rows, prices, { k, qualityBar });
137
+ const pick = router.route(await embed(prompt));
138
+
139
+ return {
140
+ routedBy: '@metaharness/router', // rUv's code made this call. Not mine.
141
+ version: '0.3.2',
142
+ model: pick.id,
143
+ predictedQuality: pick.predictedQuality,
144
+ metBar: pick.metBar,
145
+ costPerMTok: pick.costPerMTok,
146
+ subscriptionCovered: prices[pick.id] === 0,
147
+ labels: rows.length,
148
+ qualityBar,
149
+ };
150
+ }
151
+
152
+ /** Append a label from a real routed outcome β€” the fuel k-NN runs on. */
153
+ export async function recordOutcome(prompt, scores, file = OUTCOMES) {
154
+ fs.mkdirSync(path.dirname(file), { recursive: true });
155
+ const row = { ts: new Date().toISOString(), embedding: await embed(prompt), scores };
156
+ fs.appendFileSync(file, JSON.stringify(row) + '\n');
157
+ return row;
158
+ }
@@ -171,14 +171,44 @@ async function main() {
171
171
  const policy = await loadPolicy(args.policy);
172
172
  const features = extractFeatures(prompt, harness);
173
173
 
174
+ // ── rUv's REAL router gets FIRST REFUSAL. ────────────────────────────────────────────────────────
175
+ // 2026-07-13: this is the fix for a lie I shipped. v2.5's headline was "it uses @metaharness/router",
176
+ // I wrote the wrapper, tested it, gated CI against faking β€” and NEVER WIRED IT INTO THIS FILE. The
177
+ // engine users actually run stayed 100% hand-rolled while the README said otherwise. An honest
178
+ // artifact sitting next to a lying claim is still a lie. The decision now comes from rUv's code
179
+ // whenever it CAN decide, and the local heuristic is a fallback that must ANNOUNCE ITSELF.
174
180
  let decision;
175
- if (!policy) {
176
- // No policy at all: pick cheapest priced candidate for the harness as a safe floor, and SAY SO.
177
- const pool = candidates.filter((m) => (m.harness || []).includes(harness));
178
- const pick = pool.slice().sort((x, y) => (x.costPerMTok?.out ?? Infinity) - (y.costPerMTok?.out ?? Infinity))[0] || candidates[0];
179
- decision = { model: pick?.id ?? null, provider: pick?.provider ?? null, tier: pick?.tier ?? null, reason: 'NO POLICY FOUND β€” fell back to cheapest priced candidate for the harness', confidence: 0 };
180
- } else {
181
- decision = policy.choose({ features, candidates, harness });
181
+ let routedBy;
182
+ const pool = candidates.filter((m) => (m.harness || []).includes(harness));
183
+ try {
184
+ const mh = await import('./metaharness-router.mjs');
185
+ const r = await mh.route(prompt, pool.length ? pool : candidates, profile);
186
+ if (r.routedBy === '@metaharness/router') {
187
+ const pick = candidates.find((m) => m.id === r.model);
188
+ decision = {
189
+ model: r.model,
190
+ provider: pick?.provider ?? null,
191
+ tier: pick?.tier ?? null,
192
+ reason: `@metaharness/router (rUv's learned cost-optimal router): predicted quality ${r.predictedQuality?.toFixed(2)}, ${r.metBar ? 'clears' : 'BELOW'} the bar, ${r.subscriptionCovered ? '$0 (your subscription)' : `$${r.costPerMTok}/Mtok`}, from ${r.labels} labelled example(s)`,
193
+ confidence: r.predictedQuality ?? 0,
194
+ };
195
+ routedBy = '@metaharness/router';
196
+ } else {
197
+ // COLD-START or the package is absent. Say which, out loud β€” never pass the fallback off as the
198
+ // learned router. That substitution is the entire sin this wiring exists to end.
199
+ routedBy = `local-heuristic (${r.routedBy}: ${r.reason})`;
200
+ }
201
+ } catch (e) {
202
+ routedBy = `local-heuristic (@metaharness/router unavailable: ${e.message})`;
203
+ }
204
+
205
+ if (!decision) {
206
+ if (!policy) {
207
+ const pick = pool.slice().sort((x, y) => (x.costPerMTok?.out ?? Infinity) - (y.costPerMTok?.out ?? Infinity))[0] || candidates[0];
208
+ decision = { model: pick?.id ?? null, provider: pick?.provider ?? null, tier: pick?.tier ?? null, reason: 'NO POLICY FOUND β€” fell back to cheapest priced candidate for the harness', confidence: 0 };
209
+ } else {
210
+ decision = policy.choose({ features, candidates, harness });
211
+ }
182
212
  }
183
213
 
184
214
  const chosen = candidates.find((m) => m.id === decision.model) || null;
@@ -190,6 +220,8 @@ async function main() {
190
220
  tier: decision.tier,
191
221
  reason: decision.reason,
192
222
  confidence: decision.confidence,
223
+ // WHO decided. Never let a caller assume the learned router made a call the heuristic made.
224
+ routedBy,
193
225
  policy_source: policy ? policy.source.replace(os.homedir(), '~') : 'none',
194
226
  profile: profile ? PROFILE_PATH.replace(os.homedir(), '~') : 'none (catalog taken as-is β€” run model-router-setup.mjs)',
195
227
  price_verified: chosen ? chosen.verified : null,
@@ -0,0 +1,190 @@
1
+ #!/usr/bin/env node
2
+ // scripts/nightly-watchdog.mjs β€” WHO WATCHES THE WATCHERS.
3
+ //
4
+ // WHY THIS EXISTS (2026-07-13). Stuart: "Every time I ask you, you tell me one thing, and then three
5
+ // days later you're like, 'oh by the way, it hasn't been running for three days.'" He was describing a
6
+ // real structural hole. The audit that night found com.ruvnet.brain-nightly's launchd trigger had NEVER
7
+ // fired β€” and everything reported healthy, because:
8
+ //
9
+ // THE TRAP: launchd reports LAST EXIT STATUS 0 for a job that has NEVER RUN. That is byte-identical
10
+ // to a job that ran and succeeded. So "check the exit code" β€” the obvious design, and the one that was
11
+ // in place β€” literally cannot distinguish triumph from total absence. Silence read as health.
12
+ //
13
+ // THE FIX, and the rule the whole file is built on: EVIDENCE OR IT DIDN'T HAPPEN. Every watched job is
14
+ // declared in config/scheduled-jobs.json and must produce a timestamped receipt (written by
15
+ // scripts/job-heartbeat.sh, trap-protected so a dying job still writes one). Reality is then compared
16
+ // against the registry. Anything less than fresh, successful, positive proof is a VIOLATION:
17
+ //
18
+ // MISSING β€” declared in the registry but not loaded in launchd (it can never fire; this is how
19
+ // com.ruvnet.issue4-verify sat dead on disk, invisible)
20
+ // NEVER-RAN β€” loaded, but no receipt has EVER been written (the brain-nightly case)
21
+ // STALE β€” a receipt exists but is older than the job's schedule allows (it stopped)
22
+ // FAILING β€” it ran and reported a non-zero exit, or it started and never finished
23
+ // OK β€” a fresh receipt says it ran and exited 0. The ONLY state that counts as working.
24
+ //
25
+ // Shape confirmed against prior art in the ecosystem: agentic-qe/src/workers/workers/heartbeat-scheduler.ts
26
+ // (token-free periodic liveness + stale detection). No launchd supervision existed to reuse.
27
+ //
28
+ // Alerts are TRANSITION-ONLY (the discipline proven in kb/brain-alarm.mjs): a state change pages once.
29
+ // Constant redness must never become constant noise, or the gong trains you to ignore it.
30
+ //
31
+ // Usage:
32
+ // node scripts/nightly-watchdog.mjs # verdict + exit 1 if ANY job is not OK
33
+ // node scripts/nightly-watchdog.mjs --json
34
+ // node scripts/nightly-watchdog.mjs --quiet # cron mode: speak only when something is wrong
35
+
36
+ import fs from 'node:fs';
37
+ import path from 'node:path';
38
+ import os from 'node:os';
39
+ import { spawnSync } from 'node:child_process';
40
+ import { fileURLToPath, pathToFileURL } from 'node:url';
41
+
42
+ const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
43
+ const REGISTRY = process.env.WATCHDOG_REGISTRY || path.join(ROOT, 'config', 'scheduled-jobs.json');
44
+ const STATE = process.env.WATCHDOG_STATE || path.join(os.homedir(), '.cache', 'ruvnet-brain', 'watchdog-state.json');
45
+ const HB_DIR = process.env.JOB_HEARTBEAT_DIR || path.join(os.homedir(), '.cache', 'ruvnet-brain', 'heartbeats');
46
+ const HOUR = 3600_000;
47
+
48
+ export const OK = 'OK';
49
+ export const MISSING = 'MISSING';
50
+ export const NEVER_RAN = 'NEVER-RAN';
51
+ export const STALE = 'STALE';
52
+ export const FAILING = 'FAILING';
53
+
54
+ /** Labels currently loaded in launchd. A job that isn't here CANNOT fire, whatever its plist says. */
55
+ export function loadedLabels(run = () => spawnSync('launchctl', ['list'], { encoding: 'utf8' }).stdout || '') {
56
+ return new Set(
57
+ run()
58
+ .split('\n')
59
+ .slice(1)
60
+ .map((l) => l.trim().split(/\s+/)[2])
61
+ .filter(Boolean),
62
+ );
63
+ }
64
+
65
+ /**
66
+ * Judge one job from its receipt. `hb === null` means NO RECEIPT EXISTS β€” which is NEVER-RAN, never OK.
67
+ * This single line is the whole lesson of 2026-07-13: absence of evidence is not evidence of health.
68
+ */
69
+ export function judge(job, hb, loaded, now) {
70
+ if (!loaded) {
71
+ return { state: MISSING, detail: 'declared in the registry but NOT LOADED in launchd β€” it can never fire' };
72
+ }
73
+ if (!hb) {
74
+ return { state: NEVER_RAN, detail: 'no run has EVER been recorded β€” the schedule has not fired once' };
75
+ }
76
+ const stamp = new Date(hb.ended_at || hb.started_at);
77
+ if (Number.isNaN(stamp.getTime())) {
78
+ return { state: FAILING, detail: 'receipt exists but its timestamp is unreadable' };
79
+ }
80
+ const ageHours = (now - stamp) / HOUR;
81
+
82
+ // Started and never finished: the receipt is stuck in "running". Either it hung or it was killed
83
+ // hard enough to skip its own trap. Both are failures β€” and both used to look like silence.
84
+ if (hb.state === 'running' && ageHours > 6) {
85
+ return { state: FAILING, ageHours, detail: `started ${ageHours.toFixed(1)}h ago and NEVER FINISHED (hung or killed)` };
86
+ }
87
+ if (ageHours > job.maxAgeHours) {
88
+ return { state: STALE, ageHours, detail: `last ran ${ageHours.toFixed(1)}h ago β€” its schedule allows ${job.maxAgeHours}h. It stopped.` };
89
+ }
90
+ if (hb.state === 'failed' || (typeof hb.exit_code === 'number' && hb.exit_code !== 0)) {
91
+ return { state: FAILING, ageHours, detail: `last run FAILED with exit ${hb.exit_code} (${ageHours.toFixed(1)}h ago)` };
92
+ }
93
+ if (hb.state === 'running') {
94
+ return { state: OK, ageHours, detail: `running right now (started ${ageHours.toFixed(1)}h ago)` };
95
+ }
96
+ return { state: OK, ageHours, detail: `ran ${ageHours.toFixed(1)}h ago, exit 0` };
97
+ }
98
+
99
+ export function readHeartbeat(label, dir = HB_DIR) {
100
+ try { return JSON.parse(fs.readFileSync(path.join(dir, `${label}.json`), 'utf8')); } catch { return null; }
101
+ }
102
+
103
+ /**
104
+ * TIER 2 EVIDENCE. A heartbeat (tier 1) is the real proof: start, end, exit code, trap-protected.
105
+ * But receipts only exist from the moment a job is wrapped, and jobs that have genuinely been running
106
+ * for weeks would read NEVER-RAN on day one β€” a false alarm that would poison the gong immediately.
107
+ *
108
+ * A job's own log, written WHILE IT WORKED, is still evidence it ran (it satisfies the rule: a job that
109
+ * died silently cannot produce it). It is weaker β€” no exit code, so a job that logs and then dies looks
110
+ * alive β€” so it is accepted, LABELLED as second-class, and superseded the moment a real receipt lands.
111
+ * Absence of BOTH remains NEVER-RAN. This is a bootstrap ramp, not a loophole.
112
+ */
113
+ export function readLegacyLog(job, root = ROOT) {
114
+ if (!job.legacyLog) return null;
115
+ try {
116
+ const p = path.join(root, job.legacyLog);
117
+ const { mtime } = fs.statSync(p);
118
+ const tail = fs.readFileSync(p, 'utf8').slice(-4000);
119
+ return {
120
+ started_at: mtime.toISOString(),
121
+ ended_at: mtime.toISOString(),
122
+ state: /FATAL|VERIFIED FAILURE/.test(tail) ? 'failed' : 'ok',
123
+ exit_code: /FATAL|VERIFIED FAILURE/.test(tail) ? 1 : 0,
124
+ _tier2: true, // surfaced in the verdict so nobody mistakes this for a real receipt
125
+ };
126
+ } catch { return null; }
127
+ }
128
+
129
+ export function checkAll(now, { registry = REGISTRY, loaded = loadedLabels(), hbDir = HB_DIR, root = ROOT } = {}) {
130
+ const { jobs } = JSON.parse(fs.readFileSync(registry, 'utf8'));
131
+ return jobs.map((job) => {
132
+ const hb = readHeartbeat(job.label, hbDir) || readLegacyLog(job, root);
133
+ const verdict = judge(job, hb, loaded.has(job.label), now);
134
+ if (hb?._tier2 && verdict.state === OK) verdict.detail += ' (via its log β€” no receipt yet; the next run writes one)';
135
+ return { ...job, ...verdict };
136
+ });
137
+ }
138
+
139
+ const loadState = () => { try { return JSON.parse(fs.readFileSync(STATE, 'utf8')); } catch { return {}; } };
140
+ const saveState = (s) => { fs.mkdirSync(path.dirname(STATE), { recursive: true }); fs.writeFileSync(STATE, JSON.stringify(s, null, 2)); };
141
+
142
+ /** Page only on a CHANGE of state. Repeating the same alarm nightly is how alarms get ignored. */
143
+ export function transitions(results, prev) {
144
+ return results.filter((r) => (prev[r.label] ?? OK) !== r.state);
145
+ }
146
+
147
+ async function push(title, body, priority) {
148
+ const topic = process.env.NTFY_TOPIC
149
+ || (() => { try { return fs.readFileSync(path.join(os.homedir(), '.cache', 'ruvnet-brain', 'ntfy-topic'), 'utf8').trim(); } catch { return null; } })();
150
+ if (!topic) return false;
151
+ try {
152
+ await fetch(`https://ntfy.sh/${topic}`, {
153
+ method: 'POST',
154
+ headers: { Title: title, Priority: priority, Tags: priority === 'urgent' ? 'rotating_light' : 'white_check_mark' },
155
+ body,
156
+ });
157
+ return true;
158
+ } catch { return false; }
159
+ }
160
+
161
+ async function main() {
162
+ const json = process.argv.includes('--json');
163
+ const quiet = process.argv.includes('--quiet');
164
+ const results = checkAll(new Date());
165
+ const bad = results.filter((r) => r.state !== OK);
166
+
167
+ const prev = loadState();
168
+ for (const c of transitions(results, prev)) {
169
+ if (c.state === OK) await push(`βœ… ${c.label} is healthy again`, c.detail, 'default');
170
+ else await push(`πŸ”΄ ${c.state}: ${c.label}`, `${c.what}\n\n${c.detail}\n\nschedule: ${c.schedule}`, 'urgent');
171
+ }
172
+ saveState(Object.fromEntries(results.map((r) => [r.label, r.state])));
173
+
174
+ if (json) console.log(JSON.stringify({ results, checkedAt: new Date().toISOString() }, null, 2));
175
+ else if (!quiet || bad.length) {
176
+ console.log('Scheduled-job watchdog β€” proof each job RAN, not just that it exists\n');
177
+ for (const r of results) {
178
+ const icon = r.state === OK ? 'βœ…' : 'πŸ”΄';
179
+ console.log(`${icon} ${r.state.padEnd(10)} ${r.label}`);
180
+ console.log(` ${r.detail}`);
181
+ console.log(` ${r.what} Β· ${r.schedule}\n`);
182
+ }
183
+ console.log(bad.length
184
+ ? `${bad.length} of ${results.length} job(s) are NOT confirmed working. NEVER-RAN means the schedule has never fired β€” it does not mean "probably fine".`
185
+ : `All ${results.length} jobs produced a fresh, successful receipt.`);
186
+ }
187
+ process.exit(bad.length ? 1 : 0);
188
+ }
189
+
190
+ if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) await main();
@@ -0,0 +1,176 @@
1
+ #!/usr/bin/env node
2
+ // scripts/no-silent-substitution.mjs β€” THE GATE THAT WOULD HAVE CAUGHT ME.
3
+ //
4
+ // WHY (2026-07-13, Stuart: "you're still doing this crap where you fucking lie to me and write a
5
+ // bunch of code and then tell me it's Ruv's code?"):
6
+ //
7
+ // I wrote scripts/model-router-engine.mjs β€” 216 lines of my own heuristic with a self-described
8
+ // "placeholder policy" β€” and SKILL.md called it "the MetaHarness router engine". Meanwhile
9
+ // @metaharness/router@0.3.2 (ADR-040/043, Accepted/implemented) was sitting on npm: rUv's real
10
+ // learned cost-optimal router, the productized DRACO Phase-2 finding. I built a Claude fake and
11
+ // gave it his name. Every test passed. Every gate was green. NOTHING CHECKED FOR THE ONE THING THAT
12
+ // ACTUALLY MATTERED.
13
+ //
14
+ // That is a QA hole, not a slip. Tests check "does my code work"; nothing checked "should this code
15
+ // exist at all, or does rUv already ship it?" This gate closes that hole and runs in CI.
16
+ //
17
+ // THE RULE IT ENFORCES:
18
+ // If this repo contains code that implements a capability rUv already ships as a package, then
19
+ // EITHER
20
+ // (a) the real package is a declared dependency AND is actually imported somewhere, OR
21
+ // (b) the local file carries an explicit, un-missable disclosure:
22
+ // HAND-ROLLED: <reason>. REAL TOOL: <package>
23
+ // Silence is not an option. You may hand-roll β€” you may NEVER hand-roll silently, and you may
24
+ // never call your hand-roll by rUv's name.
25
+ //
26
+ // Usage: node scripts/no-silent-substitution.mjs # exit 1 on any violation (CI gate)
27
+
28
+ import fs from 'node:fs';
29
+ import path from 'node:path';
30
+ import { fileURLToPath, pathToFileURL } from 'node:url';
31
+
32
+ const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
33
+
34
+ /**
35
+ * The capability map. Each entry: a capability rUv ALREADY SHIPS, the package that provides it, and
36
+ * the signals that this repo is implementing it locally.
37
+ *
38
+ * Add to this list whenever the ecosystem ships something we might be tempted to rebuild. The cost of
39
+ * a missing entry is exactly the bug above: a hand-roll wearing rUv's name, with green tests.
40
+ */
41
+ export const CAPABILITIES = [
42
+ {
43
+ capability: 'cost-optimal model routing',
44
+ pkg: '@metaharness/router',
45
+ // Naming your file/docs after rUv's product is the tell. If you use the NAME, you use the TOOL.
46
+ claimsTheName: /metaharness[ -]?router|the metaharness router engine/i,
47
+ localImpl: /cost.?optimal|qualityBar|route.*cheapest|model.?router/i,
48
+ },
49
+ {
50
+ capability: 'test generation / QE fleet',
51
+ pkg: 'agentic-qe',
52
+ claimsTheName: /agentic[ -]?qe/i,
53
+ localImpl: /generate.*tests?.*automatically|coverage.?gap.?analysis/i,
54
+ },
55
+ {
56
+ capability: 'vector store / HNSW',
57
+ pkg: '@ruvector/rvf',
58
+ claimsTheName: /\bRVF\b|ruvector/i,
59
+ localImpl: /hand.?rolled.*cosine|own.*hnsw.*implementation/i,
60
+ },
61
+ {
62
+ capability: 'prompt-injection / PII defence',
63
+ pkg: '@claude-flow/aidefence',
64
+ claimsTheName: /aimds|aidefence/i,
65
+ localImpl: /prompt.?injection.*scanner|pii.*detector/i,
66
+ },
67
+ ];
68
+
69
+ // Scan CODE and the SKILL (the two places a substitution can actually deceive someone), not prose.
70
+ // A README that says "we use agentic-qe" is a claim about usage β€” checking it belongs in claims-verify.
71
+ // A gate that fires on every doc mentioning a tool is a gate everyone learns to ignore (see the
72
+ // windows-unit lesson: a permanently-red required job trains people to stop reading CI).
73
+ const SCAN_DIRS = ['scripts', 'bin', 'plugin/scripts', 'plugin/skills'];
74
+ const SKIP_DIRS = new Set(['node_modules', '.git', 'clones', 'dist', 'coverage', 'kb']);
75
+ const SCAN_EXT = new Set(['.mjs', '.js', '.ts', '.md']);
76
+ // This file NAMES every tool in order to police them β€” exempting it is not a loophole, it is the
77
+ // difference between the rulebook and a violation.
78
+ const EXEMPT = new Set(['scripts/no-silent-substitution.mjs', 'tests/unit/no-silent-substitution.test.mjs']);
79
+
80
+ // The disclosure that makes a hand-roll legitimate. Explicit, greppable, impossible to write by accident.
81
+ const DISCLOSURE = /HAND-ROLLED:.*REAL TOOL:\s*(\S+)/is;
82
+
83
+ export function walk(dir, out = []) {
84
+ for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
85
+ if (e.isDirectory()) { if (!SKIP_DIRS.has(e.name)) walk(path.join(dir, e.name), out); }
86
+ else if (SCAN_EXT.has(path.extname(e.name))) out.push(path.join(dir, e.name));
87
+ }
88
+ return out;
89
+ }
90
+
91
+ /**
92
+ * Is the real package genuinely used? Checks EVERY manifest in the repo, not just the root β€” the
93
+ * first version of this gate only read the root package.json and cried wolf on @ruvector/rvf, which
94
+ * is declared in kb/package.json and used all over kb/. A gate with false positives gets switched
95
+ * off, and then it protects nothing.
96
+ */
97
+ export function packageIsReallyUsed(pkg, root = ROOT) {
98
+ const manifests = ['package.json', 'kb/package.json', 'plugin/package.json'];
99
+ let declared = false;
100
+ for (const m of manifests) {
101
+ try {
102
+ const p = JSON.parse(fs.readFileSync(path.join(root, m), 'utf8'));
103
+ if ({ ...p.dependencies, ...p.devDependencies, ...p.optionalDependencies }[pkg]) { declared = true; break; }
104
+ } catch { /* manifest absent β€” keep looking */ }
105
+ }
106
+ if (!declared) return { declared: false, imported: false };
107
+
108
+ // Match ANY real code reference to the package, not just a static `import ... from`. kb/ loads
109
+ // @ruvector/rvf through a lazy dynamic resolver (createRequire/resolve-deps), so a static-import
110
+ // regex reports the most heavily-used dependency in the repo as "never imported". A gate that
111
+ // flags real usage as fraud is worse than no gate.
112
+ const nameRe = new RegExp(`['"\`]${pkg.replace(/[/@.]/g, '\\$&')}['"\`/]`);
113
+ const codeDirs = ['scripts', 'kb', 'bin'].map((d) => path.join(root, d)).filter((d) => fs.existsSync(d));
114
+ const imported = codeDirs
115
+ .flatMap((d) => walk(d, []))
116
+ .filter((f) => ['.mjs', '.js', '.ts'].includes(path.extname(f))) // code only β€” a doc mentioning it is not usage
117
+ .some((f) => nameRe.test(fs.readFileSync(f, 'utf8')));
118
+ return { declared, imported };
119
+ }
120
+
121
+ export function audit(root = ROOT) {
122
+ const violations = [];
123
+ const files = SCAN_DIRS.flatMap((d) => {
124
+ const abs = path.join(root, d);
125
+ return fs.existsSync(abs) ? walk(abs, []) : [];
126
+ }).filter((f) => !EXEMPT.has(path.relative(root, f).split(path.sep).join('/')));
127
+
128
+ for (const cap of CAPABILITIES) {
129
+ const use = packageIsReallyUsed(cap.pkg, root);
130
+ for (const abs of files) {
131
+ const rel = path.relative(root, abs).split(path.sep).join('/');
132
+ const src = fs.readFileSync(abs, 'utf8');
133
+
134
+ // BOTH signals are required, and that conjunction IS the crime:
135
+ // buildsIt β€” this file implements the capability
136
+ // namesIt β€” and calls it by rUv's name
137
+ // Merely mentioning a tool is not a substitution. Merely implementing something is not either
138
+ // (you are allowed to write code). Implementing it AND wearing rUv's name is the deception.
139
+ if (!(cap.claimsTheName.test(src) && cap.localImpl.test(src))) continue;
140
+
141
+ // Two ways to be innocent: actually use the real tool, or openly admit you did not.
142
+ if (use.imported || DISCLOSURE.test(src)) continue;
143
+
144
+ violations.push({
145
+ file: rel,
146
+ capability: cap.capability,
147
+ pkg: cap.pkg,
148
+ why: use.declared
149
+ ? `implements "${cap.capability}" and calls itself ${cap.pkg}, but the package is declared and NEVER IMPORTED β€” a hand-roll wearing rUv's name`
150
+ : `implements "${cap.capability}" and calls itself ${cap.pkg}, but the package is NOT a dependency β€” a Claude fake with rUv's label on it`,
151
+ fix: `Either USE ${cap.pkg} (npm i ${cap.pkg}; import it), or add the disclosure: "HAND-ROLLED: <reason>. REAL TOOL: ${cap.pkg}"`,
152
+ });
153
+ }
154
+ }
155
+ return violations;
156
+ }
157
+
158
+ function main() {
159
+ const violations = audit();
160
+ console.log('no-silent-substitution β€” is any local code impersonating a real rUv tool?\n');
161
+ if (!violations.length) {
162
+ console.log('βœ… none. Every RuvNet capability this repo names is either genuinely used or openly disclosed as a hand-roll.');
163
+ process.exit(0);
164
+ }
165
+ for (const v of violations) {
166
+ console.error(`❌ ${v.file}`);
167
+ console.error(` capability : ${v.capability}`);
168
+ console.error(` problem : ${v.why}`);
169
+ console.error(` fix : ${v.fix}\n`);
170
+ }
171
+ console.error(`${violations.length} silent substitution(s). You may hand-roll β€” you may NEVER hand-roll SILENTLY,`);
172
+ console.error("and you may never call your hand-roll by rUv's name.");
173
+ process.exit(1);
174
+ }
175
+
176
+ if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) main();
@@ -82,7 +82,9 @@ const fmt$ = (n) => `$${n < 0.01 ? n.toFixed(5) : n.toFixed(4)}`;
82
82
 
83
83
  export function receiptLine(model, costs) {
84
84
  const ref = costs.ref && costs.ref !== FRONTIER.name ? costs.ref : 'frontier';
85
- return `\x1b[2m⚑ MetaHarness: routed to ${model} (est. ${fmt$(costs.cost)} vs ${fmt$(costs.frontier)} ${ref} β€” saved ~${fmt$(costs.saved)})\x1b[0m`;
85
+ // Percentage leads β€” "saved ~$0.005" reads as noise, "~97% cheaper" is the message (Stuart, 2026-07-13).
86
+ const pct = costs.frontier > 0 ? Math.round((costs.saved / costs.frontier) * 100) : 0;
87
+ return `\x1b[2m⚑ MetaHarness: routed to ${model} β€” ~${pct}% cheaper (est. ${fmt$(costs.cost)} vs ${fmt$(costs.frontier)} ${ref}, saved ~${fmt$(costs.saved)})\x1b[0m`;
86
88
  }
87
89
 
88
90
  // Load OPENROUTER_API_KEY from ruvnet-brain/.env if not already in env. Value never printed/logged.