model-orchestrator 1.0.4 → 1.0.5

package/AGENTS.md

@@ -33,6 +33,6 @@ When using the Claude Code plugin, follow [plugin/README.md](plugin/README.md).
 - **Installer safety:** writes remain inside `--dir` and `--project`; preserve user edits according to manifest hashes and explicit flags. Run no third-party installer.
 - **Runner safety:** preserve exit codes and the log schema. Before changing an output judge, add a failing case in `test/judges.test.js`.
 - **Secrets:** use environment-variable names only. Never add a credential value to code, examples or tests.
-- **Proof:** measure through `proof/scripts/`, store results in `proof/results.json` and regenerate the proof page. The suite rejects expired entries.
+- **Proof:** measure through `proof/scripts/`, store results in `proof/results.json` and regenerate the proof page. An expired entry warns and is re-measured; it never blocks tests or releases.
 - **Verify:** run `npm test` and report tests, pass, fail, skipped and exit code. The suite prints current counts. Use `npm pack --dry-run` to inspect publication contents.
 - **Style:** use short condition-to-action instructions and no em dashes. `test/prose.test.js` checks public vocabulary and examples.

package/CHANGELOG.md

@@ -4,6 +4,13 @@ All notable changes to this project are documented here. The format follows [Kee
 
 ## [Unreleased]
 
+## [1.0.5] - 2026-09-28
+
+### Changed
+
+- The proof page keeps only figures that measure this package. The two browser token figures measured the author's own browser subagent, which model-orchestrator does not ship, and are removed from the proof data, the site and the trailer.
+- An expired proof figure is a signal to re-measure: it warns and never fails tests, CI or a release. The product site leaves expired figures off.
+
 ## [1.0.4] - 2026-09-28
 
 ### Security
@@ -550,7 +557,8 @@ First release.
 - Tests: a case per fix, judges proven to go red, mutation checks; `npm test` prints the current count.
 - Adversarial audit: two Codex rounds plus a two-engine review (Codex, Antigravity); findings and fixes in `docs/audit-brief.md`. After the review: subagents go to the project root (`--project`), snippet paths computed from `--dir`, lane sections rendered from the selection, a primary agent required, level 3 asks for API keys separately from CLIs, images and CLI installs pinned, an activation summary at the end of every install.
 
-[Unreleased]: https://github.com/aunysillyme/model-orchestrator/compare/v1.0.4...HEAD
+[Unreleased]: https://github.com/aunysillyme/model-orchestrator/compare/v1.0.5...HEAD
+[1.0.5]: https://github.com/aunysillyme/model-orchestrator/compare/v1.0.4...v1.0.5
 [1.0.4]: https://github.com/aunysillyme/model-orchestrator/compare/v1.0.3...v1.0.4
 [1.0.3]: https://github.com/aunysillyme/model-orchestrator/compare/v1.0.2...v1.0.3
 [1.0.2]: https://github.com/aunysillyme/model-orchestrator/compare/v1.0.1...v1.0.2

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "model-orchestrator",
-  "version": "1.0.4",
+  "version": "1.0.5",
   "description": "Model router for AI coding agents: installs routing rules, 8 subagents, hooks and a CLI runner so your AI picks model and effort per task and saves tokens",
   "type": "module",
   "bin": {

package/proof/README.md

@@ -10,8 +10,6 @@ Environment: Node v22.22.3, darwin arm64. Timing varies with startup caches and
 | Lane runner overhead | 63.74 ms median difference | 7 | 2026-09-27 | 2026-10-11 | [script](../proof/scripts/runner-overhead.js) |
 | Empty results flagged | 10 fixtures rejected | 10 | 2026-09-27 | 2026-10-11 | [script](../proof/scripts/missing-results.js) |
 | Acceptance failures blocked | 4 fixtures rejected | 4 | 2026-09-27 | 2026-10-11 | [script](../proof/scripts/check-gate.js) |
-| Main conversation browser tokens | 408147 tokens per browser step (median) | 9775 | 2026-09-26 | 2026-10-27 | author setup, re-measured locally |
-| Small browser subagent tokens | 17197 tokens per step (highest of 5 runs) | 5 | 2026-09-27 | 2026-10-27 | author setup, re-measured locally |
 
 ## Run the proof scripts
 
@@ -21,7 +19,7 @@ node proof/scripts/render.js
 npm test
 ```
 
-The measurement command refreshes reproducible entries and preserves separately sourced author-setup entries. The test suite rejects expired, future-dated or incomplete entries and checks this page against the data. The weekly [refresh workflow](../.github/workflows/proof.yml) reruns the scripts and commits their data and generated page.
+The measurement command refreshes the reproducible entries. An expired entry is a signal to re-measure; the test suite rejects future-dated or incomplete entries and checks this page against the data. The weekly [refresh workflow](../.github/workflows/proof.yml) reruns the scripts and commits their data and generated page.
 
 ## Try the acceptance gate
 
@@ -70,22 +68,6 @@ Kind: reproducible local measurement. Sample size: 4. Measured: 2026-09-27. Expi
 
 Source: [proof/scripts/check-gate.js](../proof/scripts/check-gate.js).
 
-### Main conversation browser tokens
-
-Measured on the author's Claude Code sessions: every browser tool call in the transcripts, counting tokens re-read by the main conversation per step. Median: 408,147 tokens per browser step across 145 sessions and 9,775 browser steps. Sample size counts browser steps.
-
-Kind: measured on the author's setup. Sample size: 9775. Measured: 2026-09-26. Expires: 2026-10-27.
-
-Source: author setup, re-measured locally.
-
-### Small browser subagent tokens
-
-At least 23x fewer tokens per step in this sample: the main conversation median of 408,147 divided by the highest subagent run of 17,197 is 23.73x. Measured on the author's Claude Code sessions, with the same kind of work handed to a small browser subagent. Five runs, tokens divided by steps or tool calls per run: 17,197 (206,369 tokens / 12 steps, 2026-09-26); 4,077 (93,773 / 23), 4,201 (105,034 / 25), 4,585 (91,709 / 20), and 4,489 (94,263 / 21), all four on 2026-09-27. Range: 4,077 to 17,197; median of 5 runs: 4,489. Typical context, using the median of 5 runs: about 91x fewer tokens per step. These are measurements from the author's own sessions, not a controlled comparison or a guarantee for other setups. The four 2026-09-27 runs shared one browser pane, so some steps were spent recovering a drifting tab, which raises the step count and lowers per-step tokens. The first run counted steps; the later runs counted tool calls. Sample size counts runs.
-
-Kind: measured on the author's setup. Sample size: 5. Measured: 2026-09-27. Expires: 2026-10-27.
-
-Source: author setup, re-measured locally.
-
 ## Operation and verification
 
 - **What and why:** executable measurements keep public figures traceable to current output.
@@ -95,6 +77,6 @@ Source: author setup, re-measured locally.
 - **Reads:** package scripts, the installer, runner and acceptance-check runner. Fixture tests use an isolated home and PATH.
 - **Writes:** results.json, this generated page, temporary fixture directories and local fixture logs. The recorder writes gate-demo.cast and gate-demo.gif.
 - **Closed loop:** the workflow fails when measurement or tests fail. GitHub Actions records the failure; repository notification settings decide who receives it. No separate alert service is configured.
-- **Failure modes:** runner behavior changes, an expired catalog snapshot, missing runtime, unavailable write permission, or timing noise. Review the failed job, rerun locally, and send a reproducible issue to the repository maintainers.
-- **Run and verify:** run the commands above, inspect sample arrays and fixture exit codes in results.json, and require npm test to pass. A future-time unit test proves expiry can fail.
-- **Source of truth:** results.json and the scripts it names. Author-setup evidence is added separately by its owner.
+- **Failure modes:** runner behavior changes, missing runtime, unavailable write permission, or timing noise. Review the failed job, rerun locally, and send a reproducible issue to the repository maintainers.
+- **Run and verify:** run the commands above, inspect sample arrays and fixture exit codes in results.json, and require npm test to pass.
+- **Source of truth:** results.json and the scripts it names.

package/proof/results.json

@@ -169,30 +169,6 @@
           "exitCode": 1
         }
       ]
-    },
-    {
-      "id": "author-main-browser-tokens",
-      "label": "Main conversation browser tokens",
-      "kind": "author-setup",
-      "value": 408147,
-      "unit": "tokens per browser step (median)",
-      "measuredAt": "2026-09-26",
-      "method": "Measured on the author's Claude Code sessions: every browser tool call in the transcripts, counting tokens re-read by the main conversation per step. Median: 408,147 tokens per browser step across 145 sessions and 9,775 browser steps. Sample size counts browser steps.",
-      "sampleSize": 9775,
-      "script": "author setup, re-measured locally",
-      "expiresAt": "2026-10-27"
-    },
-    {
-      "id": "author-subagent-browser-tokens",
-      "label": "Small browser subagent tokens",
-      "kind": "author-setup",
-      "value": 17197,
-      "unit": "tokens per step (highest of 5 runs)",
-      "measuredAt": "2026-09-27",
-      "method": "At least 23x fewer tokens per step in this sample: the main conversation median of 408,147 divided by the highest subagent run of 17,197 is 23.73x. Measured on the author's Claude Code sessions, with the same kind of work handed to a small browser subagent. Five runs, tokens divided by steps or tool calls per run: 17,197 (206,369 tokens / 12 steps, 2026-09-26); 4,077 (93,773 / 23), 4,201 (105,034 / 25), 4,585 (91,709 / 20), and 4,489 (94,263 / 21), all four on 2026-09-27. Range: 4,077 to 17,197; median of 5 runs: 4,489. Typical context, using the median of 5 runs: about 91x fewer tokens per step. These are measurements from the author's own sessions, not a controlled comparison or a guarantee for other setups. The four 2026-09-27 runs shared one browser pane, so some steps were spent recovering a drifting tab, which raises the step count and lowers per-step tokens. The first run counted steps; the later runs counted tool calls. Sample size counts runs.",
-      "sampleSize": 5,
-      "script": "author setup, re-measured locally",
-      "expiresAt": "2026-10-27"
     }
   ]
 }

package/proof/scripts/lib.js

@@ -64,7 +64,9 @@ export function validateResults(data, now = new Date()) {
     else {
       if (item.measuredAt > today) errors.push(`${item.id}: measurement is in the future`);
       if (item.expiresAt < item.measuredAt) errors.push(`${item.id}: expiry precedes measurement`);
-      if (item.expiresAt < today) errors.push(`${item.id}: expired ${item.expiresAt}`);
+      // Expiry is a refresh signal, never a gate: an expired figure is reported and
+      // left off the pages, and tests, CI and releases keep running.
+      if (item.expiresAt < today) console.warn(`proof: ${item.id} expired ${item.expiresAt}; rerun node proof/scripts/measure.js`);
     }
     const localAuthorSource = item.kind === 'author-setup' && item.script === 'author setup, re-measured locally';
     if (!localAuthorSource && (typeof item.script !== 'string' || !/^proof\/scripts\/[A-Za-z0-9_-]+\.js$/.test(item.script))) errors.push(`${item.id}: script must name a proof script`);

package/proof/scripts/render.js

@@ -7,7 +7,7 @@ export function proofMarkdown(data) {
   const source = (e, label) => e.kind === 'author-setup' && e.script === 'author setup, re-measured locally' ? e.script : `[${label}](../${e.script})`;
   const rows = data.entries.map(e => `| ${e.label} | ${e.value.toFixed(Number.isInteger(e.value) ? 0 : 2)} ${e.unit} | ${e.sampleSize} | ${e.measuredAt} | ${e.expiresAt} | ${source(e, 'script')} |`);
   const methods = data.entries.map(e => `### ${e.label}\n\n${e.method}\n\nKind: ${e.kind === 'author-setup' ? "measured on the author's setup" : 'reproducible local measurement'}. Sample size: ${e.sampleSize}. Measured: ${e.measuredAt}. Expires: ${e.expiresAt}.\n\nSource: ${source(e, e.script)}.`).join('\n\n');
-  return `# Reproduce the measurements\n\nGenerated from [results.json](results.json). Each figure has a method, sample size, measurement date and expiry. Run the scripts on your own machine to compare.\n\nEnvironment: Node ${data.environment.node}, ${data.environment.platform} ${data.environment.arch}. Timing varies with startup caches and other work on the machine. Synthetic cases show what those fixtures exercise.\n\n| Measurement | Result | Sample size | Measured | Expires | Reproduce |\n|---|---|---|---|---|---|\n${rows.join('\n')}\n\n## Run the proof scripts\n\n\`\`\`sh\nnode proof/scripts/measure.js\nnode proof/scripts/render.js\nnpm test\n\`\`\`\n\nThe measurement command refreshes reproducible entries and preserves separately sourced author-setup entries. The test suite rejects expired, future-dated or incomplete entries and checks this page against the data. The weekly [refresh workflow](../.github/workflows/proof.yml) reruns the scripts and commits their data and generated page.\n\n## Try the acceptance gate\n\n\`\`\`sh\naunx checks ACCEPTANCE_CHECKS.json\naunx checks run ACCEPTANCE_CHECKS.json\n\`\`\`\n\nThe scaffold starts red. Replace the sample with commands that prove your requirements, then put \`aunx checks run ACCEPTANCE_CHECKS.json && <your-release-command>\` in your own release sequence. Commands are local code you review before running. Manual evidence stays UNVERIFIED and blocks the gate.\n\n![Acceptance gate rejects a missing output, then passes after the output exists](gate-demo.gif)\n\nThe [recording script](scripts/record-gate.js) captures real command output into an asciicast, then renders it with an already installed agg. Companion tools are installed by their users.\n\n## Measurement methods\n\n${methods}\n\n## Operation and verification\n\n- **What and why:** executable measurements keep public figures traceable to current output.\n- **Trigger:** weekly schedule, workflow dispatch, or \`node proof/scripts/measure.js\`.\n- **Invocation chain:** workflow -> measurement functions -> isolated Node fixtures -> results.json -> this page -> npm test.\n- **Dependencies:** Node and the repository. The optional GIF recorder uses agg from the asciinema project.\n- **Reads:** package scripts, the installer, runner and acceptance-check runner. Fixture tests use an isolated home and PATH.\n- **Writes:** results.json, this generated page, temporary fixture directories and local fixture logs. The recorder writes gate-demo.cast and gate-demo.gif.\n- **Closed loop:** the workflow fails when measurement or tests fail. GitHub Actions records the failure; repository notification settings decide who receives it. No separate alert service is configured.\n- **Failure modes:** runner behavior changes, an expired catalog snapshot, missing runtime, unavailable write permission, or timing noise. Review the failed job, rerun locally, and send a reproducible issue to the repository maintainers.\n- **Run and verify:** run the commands above, inspect sample arrays and fixture exit codes in results.json, and require npm test to pass. A future-time unit test proves expiry can fail.\n- **Source of truth:** results.json and the scripts it names. Author-setup evidence is added separately by its owner.\n`;
+  return `# Reproduce the measurements\n\nGenerated from [results.json](results.json). Each figure has a method, sample size, measurement date and expiry. Run the scripts on your own machine to compare.\n\nEnvironment: Node ${data.environment.node}, ${data.environment.platform} ${data.environment.arch}. Timing varies with startup caches and other work on the machine. Synthetic cases show what those fixtures exercise.\n\n| Measurement | Result | Sample size | Measured | Expires | Reproduce |\n|---|---|---|---|---|---|\n${rows.join('\n')}\n\n## Run the proof scripts\n\n\`\`\`sh\nnode proof/scripts/measure.js\nnode proof/scripts/render.js\nnpm test\n\`\`\`\n\nThe measurement command refreshes the reproducible entries. An expired entry is a signal to re-measure; the test suite rejects future-dated or incomplete entries and checks this page against the data. The weekly [refresh workflow](../.github/workflows/proof.yml) reruns the scripts and commits their data and generated page.\n\n## Try the acceptance gate\n\n\`\`\`sh\naunx checks ACCEPTANCE_CHECKS.json\naunx checks run ACCEPTANCE_CHECKS.json\n\`\`\`\n\nThe scaffold starts red. Replace the sample with commands that prove your requirements, then put \`aunx checks run ACCEPTANCE_CHECKS.json && <your-release-command>\` in your own release sequence. Commands are local code you review before running. Manual evidence stays UNVERIFIED and blocks the gate.\n\n![Acceptance gate rejects a missing output, then passes after the output exists](gate-demo.gif)\n\nThe [recording script](scripts/record-gate.js) captures real command output into an asciicast, then renders it with an already installed agg. Companion tools are installed by their users.\n\n## Measurement methods\n\n${methods}\n\n## Operation and verification\n\n- **What and why:** executable measurements keep public figures traceable to current output.\n- **Trigger:** weekly schedule, workflow dispatch, or \`node proof/scripts/measure.js\`.\n- **Invocation chain:** workflow -> measurement functions -> isolated Node fixtures -> results.json -> this page -> npm test.\n- **Dependencies:** Node and the repository. The optional GIF recorder uses agg from the asciinema project.\n- **Reads:** package scripts, the installer, runner and acceptance-check runner. Fixture tests use an isolated home and PATH.\n- **Writes:** results.json, this generated page, temporary fixture directories and local fixture logs. The recorder writes gate-demo.cast and gate-demo.gif.\n- **Closed loop:** the workflow fails when measurement or tests fail. GitHub Actions records the failure; repository notification settings decide who receives it. No separate alert service is configured.\n- **Failure modes:** runner behavior changes, missing runtime, unavailable write permission, or timing noise. Review the failed job, rerun locally, and send a reproducible issue to the repository maintainers.\n- **Run and verify:** run the commands above, inspect sample arrays and fixture exit codes in results.json, and require npm test to pass.\n- **Source of truth:** results.json and the scripts it names.\n`;
 }
 export function renderPage() {
   const data = readResults();