model-orchestrator 1.0.4 → 1.0.6

package/AGENTS.md

@@ -33,6 +33,6 @@ When using the Claude Code plugin, follow [plugin/README.md](plugin/README.md).
 - **Installer safety:** writes remain inside `--dir` and `--project`; preserve user edits according to manifest hashes and explicit flags. Run no third-party installer.
 - **Runner safety:** preserve exit codes and the log schema. Before changing an output judge, add a failing case in `test/judges.test.js`.
 - **Secrets:** use environment-variable names only. Never add a credential value to code, examples or tests.
-- **Proof:** measure through `proof/scripts/`, store results in `proof/results.json` and regenerate the proof page. The suite rejects expired entries.
+- **Proof:** measure through `proof/scripts/`, store results in `proof/results.json` and regenerate the proof page. An expired entry warns and is re-measured; it never blocks tests or releases.
 - **Verify:** run `npm test` and report tests, pass, fail, skipped and exit code. The suite prints current counts. Use `npm pack --dry-run` to inspect publication contents.
 - **Style:** use short condition-to-action instructions and no em dashes. `test/prose.test.js` checks public vocabulary and examples.

package/CHANGELOG.md

@@ -4,6 +4,25 @@ All notable changes to this project are documented here. The format follows [Kee
 
 ## [Unreleased]
 
+## [1.0.6] - 2026-09-29
+
+### Added
+
+- The hermes lane takes `--provider` and `"provider"` in `bin/lanes.json` defaults, so a pinned model goes to a provider that serves it. A provider needs a model with it (a flag, a default or `HERMES_INFERENCE_MODEL`); otherwise cli-run refuses before the lane starts. Runs log `provider_requested` and `provider_source`.
+- `--doctor` notes a hermes model pinned without a provider, and a provider pinned without a model. The terminal route line shows the provider.
+
+### Fixed
+
+- A hermes model/provider mismatch (an upstream "model is not supported" or `model_not_found`, for example `grok-4.6` sent to `openai-codex`) is now class `rejected`, exit 16, with a problem line naming the mismatch and a fix pointing at `hermes model`. It was reported as a vague nonzero exit.
+- A pinned hermes route (`--model` or `--effort`, by flag or `lanes.json` default) no longer fails every run. cli-run placed those flags between `-z` and the prompt, and `-z` takes the prompt as its value, so Hermes refused the arguments ("argument -z/--oneshot: expected one argument"). Route flags now come before `-z`, and a Hermes argument error is reported as such rather than as a toolsets error.
+
+## [1.0.5] - 2026-09-28
+
+### Changed
+
+- The proof page keeps only figures that measure this package. The two browser token figures measured the author's own browser subagent, which model-orchestrator does not ship, and are removed from the proof data, the site and the trailer.
+- An expired proof figure is a signal to re-measure: it warns and never fails tests, CI or a release. The product site leaves expired figures off.
+
 ## [1.0.4] - 2026-09-28
 
 ### Security
@@ -550,7 +569,8 @@ First release.
 - Tests: a case per fix, judges proven to go red, mutation checks; `npm test` prints the current count.
 - Adversarial audit: two Codex rounds plus a two-engine review (Codex, Antigravity); findings and fixes in `docs/audit-brief.md`. After the review: subagents go to the project root (`--project`), snippet paths computed from `--dir`, lane sections rendered from the selection, a primary agent required, level 3 asks for API keys separately from CLIs, images and CLI installs pinned, an activation summary at the end of every install.
 
-[Unreleased]: https://github.com/aunysillyme/model-orchestrator/compare/v1.0.4...HEAD
+[Unreleased]: https://github.com/aunysillyme/model-orchestrator/compare/v1.0.5...HEAD
+[1.0.5]: https://github.com/aunysillyme/model-orchestrator/compare/v1.0.4...v1.0.5
 [1.0.4]: https://github.com/aunysillyme/model-orchestrator/compare/v1.0.3...v1.0.4
 [1.0.3]: https://github.com/aunysillyme/model-orchestrator/compare/v1.0.2...v1.0.3
 [1.0.2]: https://github.com/aunysillyme/model-orchestrator/compare/v1.0.1...v1.0.2

package/bin/cli-run.mjs

@@ -53,6 +53,7 @@
 // problem/fix lines go to your terminal only, redacted.
 //
 // ROUTE: which model and reasoning effort a lane ran with.
+// Hermes also accepts --provider to select the provider serving that model.
 // A lane with no --model and no lanes.json default inherits whatever its own
 // config file says, which is invisible from here and is how a documented route
 // silently stops being the route that runs. --model / --effort pin it per call,
@@ -199,7 +200,11 @@ export function judgeHermes(rc, out, err) {
     // hermes collapses every upstream failure into one exit code; its stderr
     // is the only place the cause is named.
     const blob = String(err || '').toLowerCase(); // stderr only: stdout is the agent's own prose
-    if (hermesQuotaText(blob)) why += ': upstream free tier degraded or limited, a retry is reasonable';
+    // A mismatch is checked first: a retry cannot fix it, and an unrelated
+    // "limit" warning on stderr must not turn it into a quota report.
+    if (hermesMismatchLine(err) || hermesMismatchLine(out, { vendorShape: true })) why += ': model/provider mismatch, the provider does not serve this model';
+    else if (hermesQuotaText(blob)) why += ': upstream free tier degraded or limited, a retry is reasonable';
+    else if (hermesArgError(blob)) why += ': hermes rejected its arguments, a caller bug and not a lane fault';
     else if (blob.includes('toolset')) why += ': invalid --toolsets value, a caller bug and not a lane fault';
     return fail('exit_nonzero', `hermes exit ${rc}: ${why}`);
   }
@@ -253,14 +258,18 @@ export function judgeQwen(rc, out) {
 //   grok    -m MODEL   --reasoning-effort EFFORT
 //   codex   -m MODEL   -c model_reasoning_effort="EFFORT"   (a TOML override, hence the quotes)
 //   agy     --model M  --effort EFFORT                      (low|medium|high)
-//   hermes  -m MODEL   --reasoning LEVEL                    (none|minimal|...)
+//   hermes  -m MODEL   --reasoning LEVEL   --provider ID    (none|minimal|...)
 //   qwen    -m MODEL   no reasoning flag
+// Only hermes takes a provider: one hermes install signs in to many providers,
+// and a model id sent to a provider that does not serve it is an HTTP 400
+// (grok-4.6 on openai-codex). `hermes -z --provider` without a model exits 2,
+// so cli-run refuses that pairing before the lane starts.
 export const LANE_FLAGS = {
-  grok: { model: (v) => ['-m', v], effort: (v) => ['--reasoning-effort', v] },
-  codex: { model: (v) => ['-m', v], effort: (v) => ['-c', `model_reasoning_effort="${v}"`] },
-  agy: { model: (v) => ['--model', v], effort: (v) => ['--effort', v] },
-  hermes: { model: (v) => ['-m', v], effort: (v) => ['--reasoning', v] },
-  qwen: { model: (v) => ['-m', v], effort: null }
+  grok: { model: (v) => ['-m', v], effort: (v) => ['--reasoning-effort', v], provider: null },
+  codex: { model: (v) => ['-m', v], effort: (v) => ['-c', `model_reasoning_effort="${v}"`], provider: null },
+  agy: { model: (v) => ['--model', v], effort: (v) => ['--effort', v], provider: null },
+  hermes: { model: (v) => ['-m', v], effort: (v) => ['--reasoning', v], provider: (v) => ['--provider', v] },
+  qwen: { model: (v) => ['-m', v], effort: null, provider: null }
 };
 
 // Auto is deliberately a small, static ladder. It is not a vendor capability
@@ -353,13 +362,16 @@ export function badRouteValue(kind, v) {
 }
 
 // --- adapters: build argv for a lane -------------------------------------
-// Route flags go in front of the prompt for every lane, because two lanes
-// (hermes, codex) take the prompt as a positional argument and a flag after it
-// is either ignored or read as part of it.
+// Route flags go in front of the prompt for every lane, because codex takes the
+// prompt as a positional argument and a flag after it is either ignored or read
+// as part of it. hermes needs them in front of -z itself: -z takes the prompt as
+// its own value, so `-z -m X prompt` is an argparse error ("argument -z/--oneshot:
+// expected one argument") and every pinned hermes route failed that way.
 function routeFlags(lane, opts) {
   const spec = LANE_FLAGS[lane];
   const out = [];
   if (!spec) return out;
+  if (opts.provider && spec.provider) out.push(...spec.provider(opts.provider));
   if (opts.model) out.push(...spec.model(opts.model));
   if (opts.effort && spec.effort) out.push(...spec.effort(opts.effort));
   return out;
@@ -384,7 +396,7 @@ export function buildArgv(lane, binary, prompt, opts, tmp) {
       return { argv: [binary, '--print-timeout', `${mins}m`, '--output-format', 'stream-json', ...route, '-p', prompt] };
     }
     case 'hermes':
-      return { argv: [binary, '-z', ...route, prompt, '--usage-file', join(tmp, 'usage.json')] };
+      return { argv: [binary, ...route, '-z', prompt, '--usage-file', join(tmp, 'usage.json')] };
     case 'qwen': {
       const argv = [binary, '-o', 'json', ...route];
       if (opts.safeMode) argv.push('--safe-mode');
@@ -527,6 +539,30 @@ function hermesQuotaText(lower) {
   return lower.includes('no usable content') || lower.includes('limit') || lower.includes('degraded');
 }
 
+// argparse's own refusal. Its usage dump always lists `-t TOOLSETS`, so this is
+// checked before the toolset match or every bad argument reads as a toolset error.
+function hermesArgError(lower) {
+  return lower.includes('hermes: error:');
+}
+
+// A model id the chosen provider does not serve. `hermes -z` prints a failed
+// turn's provider error on stdout and exits 2, so this reads both streams, but
+// returns only the matching line: the rest of stdout is the agent's own prose.
+// The wording is the upstream's own ("The 'grok-4.6' model is not supported
+// when using Codex with a ChatGPT account"; OpenAI's model_not_found code).
+export function hermesMismatchLine(text, { vendorShape = false } = {}) {
+  for (const line of String(text || '').split('\n')) {
+    const l = line.toLowerCase();
+    if (!(l.includes('model is not supported') || l.includes('model_not_found'))) continue;
+    // stdout is agent prose, so there only the vendor's own shape counts: an HTTP
+    // status, an error prefix, the Codex wording, or the OpenAI error code.
+    if (vendorShape && !(/^\s*(http [45]\d\d|error)\b/.test(l) || l.includes('model is not supported when using') || l.includes('model_not_found'))) continue;
+    // Terminal-safe: stdout can carry web-derived text, so no control characters.
+    return line.replace(/[\x00-\x08\x0b-\x1f\x7f]/g, '').trim().slice(0, 300);
+  }
+  return null;
+}
+
 // qwen: the terminal event's full error text, and a result that is an API error.
 // Never the display detail, which is clipped and can carry a model name.
 export function qwenErrorText(out) {
@@ -548,7 +584,7 @@ export function qwenErrorText(out) {
 function authoritativeBlob(lane, out, err, detail, rc) {
   if (lane === 'codex') return `${codexErrorEventsText(out)}\n${err || ''}`;
   if (lane === 'agy') return `${agyResultFieldsText(out)}\n${err || ''}`;
-  if (lane === 'hermes') return rc !== 0 ? String(err || '') : '';
+  if (lane === 'hermes') return rc !== 0 ? `${err || ''}\n${hermesMismatchLine(out, { vendorShape: true }) || ''}` : '';
   if (lane === 'qwen') return `${qwenErrorText(out)}\n${err || ''}`;
   return `${detail || ''}\n${err || ''}`; // grok: no auth, quota or rejected signal is defined
 }
@@ -572,7 +608,7 @@ export function sigRejected(lane, blob) {
   const b = blob.toLowerCase();
   if (lane === 'qwen') return b.includes('[api error: 400') || b.includes('no endpoints found') || b.includes('failed to parse grammar');
   if (lane === 'codex') return blob.includes('invalid_request_error');
-  if (lane === 'hermes') return b.includes('toolset');
+  if (lane === 'hermes') return hermesArgError(b) || b.includes('toolset') || hermesMismatchLine(blob) !== null;
   return false;
 }
 
@@ -752,6 +788,7 @@ const FIX = {
   auth: 'set the credential the message above names (its environment variable, or the lane\'s own login command), then rerun',
   quota: 'switch to another lane, or wait for the reset time if the message gave one',
   rejected: 'correct the model id, flag or request the upstream message names',
+  mismatch: 'pair the model with a provider that serves it: run `hermes model`, or pin both with --provider and --model (or "defaults": {"hermes": {"provider": ..., "model": ...}} in bin/lanes.json)',
   refused: 'adjust the hook or deny rule named above, or give this lane the tool it needs',
   cut_short: 'rerun once; if it recurs, run without --quiet and read the lane\'s stderr on the terminal',
   empty: 'rerun once, or use another lane',
@@ -769,7 +806,11 @@ export function problemAndFix(lane, cls, { out = '', err = '', detail = '', refu
   switch (cls) {
     case 'auth': return { problem: `${tag} auth: ${cause || d || 'missing or invalid credentials'}`, fix: FIX.auth };
     case 'quota': return { problem: `${tag} quota: ${cause || d || 'rate limit or credits exhausted'}`, fix: FIX.quota };
-    case 'rejected': return { problem: `${tag} rejected: ${cause || d || 'the upstream rejected the request'}`, fix: FIX.rejected };
+    case 'rejected': {
+      const mismatch = lane === 'hermes' ? hermesMismatchLine(err) || hermesMismatchLine(out, { vendorShape: true }) : null;
+      if (mismatch) return { problem: `${tag} rejected: model/provider mismatch: ${redact(mismatch)}`, fix: FIX.mismatch };
+      return { problem: `${tag} rejected: ${cause || d || 'the upstream rejected the request'}`, fix: FIX.rejected };
+    }
     case 'refused': return { problem: `${tag} refused: ${denial() || 'a hook or deny rule blocked the call'}`, fix: FIX.refused };
     case 'cut_short': return { problem: `${tag} cut short: ${d || 'no terminal success event, cause not identifiable'}`, fix: FIX.cut_short };
     case 'empty': return { problem: `${tag} empty: ${d || 'completed but delivered nothing'}`, fix: FIX.empty };
@@ -801,6 +842,7 @@ export function classifyRun(lane, { rc = 0, out = '', err = '', reason = '', det
     else {
       const blob = authoritativeBlob(lane, out, err, detail, rc);
       if (sigAuth(lane, blob)) cls = 'auth';
+      else if (lane === 'hermes' && hermesMismatchLine(blob) !== null) cls = 'rejected'; // before quota: see judgeHermes
       else if (sigQuota(lane, blob)) cls = 'quota';
       else if (sigRejected(lane, blob)) cls = 'rejected';
       else if (Number.isInteger(refused) && refused > 0) cls = 'refused';
@@ -1080,15 +1122,19 @@ export function laneConfig(here = dirname(fileURLToPath(import.meta.url))) {
       for (const [lane, d] of Object.entries(j.defaults)) {
         if (!LANES.includes(lane)) return null;
         if (!d || typeof d !== 'object' || Array.isArray(d)) return null;
-        const { model, effort, ...rest } = d;
+        const { model, effort, provider, ...rest } = d;
         if (Object.keys(rest).length) return null;
         if (model !== undefined && badRouteValue('model', model)) return null;
+        if (provider !== undefined) {
+          if (badRouteValue('provider', provider)) return null;
+          if (!LANE_FLAGS[lane] || !LANE_FLAGS[lane].provider) return null; // only hermes routes by provider
+        }
         if (effort !== undefined) {
           if (badRouteValue('effort', effort)) return null;
           if (effort.startsWith('auto') && effort !== 'auto') return null;
           if (!LANE_FLAGS[lane] || !LANE_FLAGS[lane].effort) return null; // a lane with no reasoning flag cannot have one pinned
         }
-        defaults[lane] = { model: model ?? null, effort: effort ?? null };
+        defaults[lane] = { model: model ?? null, effort: effort ?? null, provider: provider ?? null };
       }
     }
     return { enabled: j.enabled, defaults };
@@ -1109,20 +1155,22 @@ export function resolveRoute(lane, opts, defaults) {
   const d = (defaults && defaults[lane]) || {};
   const model = opts.model ?? d.model ?? null;
   const effort = opts.effort ?? d.effort ?? null;
+  const provider = opts.provider ?? d.provider ?? null;
   const src = (flag, def) => (flag != null ? 'flag' : def != null ? 'lanes.json' : 'lane_default');
-  return { model, effort, model_source: src(opts.model, d.model), effort_source: src(opts.effort, d.effort) };
+  return { model, effort, provider, model_source: src(opts.model, d.model), effort_source: src(opts.effort, d.effort), provider_source: src(opts.provider, d.provider) };
 }
 
 function usage(msg) {
   if (msg) console.error('cli-run: ' + msg);
   console.error(`usage: cli-run <${LANES.join('|')}> "<prompt>" [--brief FILE] [--timeout SECS] [--quiet]
-                [--model ID] [--effort LEVEL] [--expect-file PATH] [--expect-json]
+                [--model ID] [--effort LEVEL] [--provider ID] [--expect-file PATH] [--expect-json]
        cli-run codex --audit "<prompt>"          read-only sandbox (audit shape)
        cli-run qwen [--safe-mode] "<prompt>"     qwen-only flag
        cli-run --doctor [--run]                  enabled lanes, binaries, and the route each one is pinned to
 
   --model / --effort pin what a lane runs with, instead of letting it inherit its
   own config. Every lane takes --model; every lane except qwen takes --effort.
+  Hermes also takes --provider; it needs --model, a defaults model, or HERMES_INFERENCE_MODEL.
   Levels are the vendor's own (agy low|medium|high, hermes none|minimal|...): an
   unknown level is rejected by the lane, and reported by class (codex: rejected, 16).
   Exit codes: 0 ok, 10 empty, 11 no output, 12 timeout, 13 unavailable, 14 auth,
@@ -1179,7 +1227,8 @@ export async function doctor(run, { here = dirname(fileURLToPath(import.meta.url
     const d = defaults[lane] || {};
     // A disabled lane has no route worth reporting; saying "not pinned" there
     // reads as a finding about a lane that is not going to run.
-    const route = !on ? '' : d.effort === 'auto' ? `route ${d.model || 'lane default'}/auto (sized per call)` : d.model || d.effort ? `route ${d.model || 'lane default'}/${d.effort || 'lane default'}` : 'route not pinned (inherits the lane\'s own config)';
+    const model = `${d.provider ? d.provider + ':' : ''}${d.model || 'lane default'}`;
+    const route = !on ? '' : d.effort === 'auto' ? `route ${model}/auto (sized per call)` : d.model || d.effort || d.provider ? `route ${model}/${d.effort || 'lane default'}` : 'route not pinned (inherits the lane\'s own config)';
     let line = `  ${lane.padEnd(7)} ${on ? 'enabled ' : 'disabled'} ${bin ? 'binary ok' : 'binary MISSING'}${route ? '  ' + route : ''}`;
     if (on && !bin) bad++;
     if (on && bin && run) {
@@ -1188,6 +1237,8 @@ export async function doctor(run, { here = dirname(fileURLToPath(import.meta.url
       if (rc !== OK) bad++;
     }
     console.log(compact ? `  ${lane}: ${bin ? 'present' : 'MISSING'}` : line);
+    if (on && lane === 'hermes' && d.model && !d.provider) console.log('    note: model pinned with no provider: Hermes sends it to its default provider. A model that provider does not serve fails with HTTP 400; pin "provider" beside "model".');
+    if (on && lane === 'hermes' && d.provider && !d.model) console.log('    note: provider pinned with no model: every run without --model will be refused unless HERMES_INFERENCE_MODEL supplies a model; pin "model" beside "provider".');
   }
   console.log(bad ? `doctor: ${bad} problem(s)` : 'doctor: all enabled lanes ' + (run ? 'answered' : 'present'));
   if (!compact) {
@@ -1236,10 +1287,10 @@ export function checkContracts(opts, text, before) {
 }
 
 export async function main(argv) {
-  const VALUE = new Set(['--brief', '--timeout', '--model', '--effort', '--expect-file']);
+  const VALUE = new Set(['--brief', '--timeout', '--model', '--effort', '--provider', '--expect-file']);
   const BOOL = new Set(['--quiet', '--audit', '--safe-mode', '--doctor', '--run', '--expect-json']);
   const args = [...argv];
-  const opts = { timeout: 900, quiet: false, audit: false, model: null, effort: null, safeMode: false, brief: null, doctor: false, run: false, expectFile: null, expectJson: false };
+  const opts = { timeout: 900, quiet: false, audit: false, model: null, effort: null, provider: null, safeMode: false, brief: null, doctor: false, run: false, expectFile: null, expectJson: false };
   const positional = [];
   while (args.length) {
     const a = args.shift();
@@ -1250,6 +1301,7 @@ export async function main(argv) {
       else if (a === '--timeout') opts.timeout = Number(v);
       else if (a === '--expect-file') opts.expectFile = v;
       else if (a === '--effort') opts.effort = v;
+      else if (a === '--provider') opts.provider = v;
       else opts.model = v;
     } else if (BOOL.has(a)) {
       if (a === '--quiet') opts.quiet = true;
@@ -1283,7 +1335,7 @@ export async function main(argv) {
   if (!Number.isFinite(opts.timeout) || opts.timeout <= 0) return usage('--timeout must be a positive number of seconds');
   if (opts.audit && lane !== 'codex') return usage('--audit is codex-only');
   if (opts.safeMode && lane !== 'qwen') return usage('--safe-mode is qwen-only');
-  for (const [kind, v] of [['model', opts.model], ['effort', opts.effort]]) {
+  for (const [kind, v] of [['model', opts.model], ['effort', opts.effort], ['provider', opts.provider]]) {
     if (v == null) continue;
     const bad = badRouteValue(kind, v);
     if (bad) return usage(bad);
@@ -1292,6 +1344,7 @@ export async function main(argv) {
   // qwen has no reasoning flag. Dropping --effort silently would leave the caller
   // believing a route that never happened, which is the defect this feature fixes.
   if (opts.effort && !(LANE_FLAGS[lane] && LANE_FLAGS[lane].effort)) return usage(`${lane} has no reasoning-effort flag; --effort is not available on this lane`);
+  if (opts.provider && !LANE_FLAGS[lane].provider) return usage(`${lane} has no provider flag; --provider is hermes-only`);
 
   const digest = createHash('sha256').update(prompt).digest('hex').slice(0, 12);
   const base = { lane, prompt_sha256_12: digest, prompt_chars: prompt.length };
@@ -1301,11 +1354,15 @@ export async function main(argv) {
   // gap this feature exists to close. A malformed lanes.json has no usable
   // defaults, so the flags stand alone and say so.
   const route = resolveRoute(lane, opts, cfg === null ? {} : cfg.defaults);
+  if (route.provider && !route.model && !(process.env.HERMES_INFERENCE_MODEL || '').trim()) return usage('--provider <p> needs a model: pass --model, or set "model" beside "provider" in lanes.json "defaults"');
   const sizing = resolveAutoEffort(lane, route.effort, prompt, opts.audit);
   opts.model = route.model;
   opts.effort = sizing.resolved;
+  opts.provider = route.provider;
   Object.assign(base, {
     model_requested: route.model,
+    provider_requested: route.provider,
+    provider_source: route.provider_source,
     effort_requested: route.effort,
     model_source: route.model_source,
     effort_source: route.effort_source,
@@ -1384,7 +1441,7 @@ export async function main(argv) {
     }
     const code = cls === 'interrupted' ? 128 + (r.interrupted === 'SIGINT' ? 2 : 15) : CLASS_CODES[cls];
     if (text && code === OK) process.stdout.write(text + '\n');
-    const routeNote = route.model || route.effort ? `${route.model || 'lane default'}/${route.effort || 'lane default'}` : 'lane default';
+    const routeNote = (route.provider ? `${route.provider}:` : '') + (route.model || route.effort ? `${route.model || 'lane default'}/${route.effort || 'lane default'}` : 'lane default');
     if (!opts.quiet) {
       console.error(`cli-run[${lane}] ${verdict} rc=${code} class=${cls} refused=${refused === null ? 'null' : refused} ${r.seconds.toFixed(1)}s raw=${r.outBytes || 0}B route=${routeNote} :: ${redact(detail)}`);
       let authoritative = null;

package/docs/how-it-routes.md

@@ -60,6 +60,8 @@ aunx cli-run --doctor
 # Direct form: node bin/cli-run.mjs --doctor
 ```
 
+Hermes also takes `--provider '<provider-id>'` (or `"provider"` in its `lanes.json` defaults), always together with a model, because one Hermes install can reach several providers and a model sent to the wrong one fails with HTTP 400. `--doctor` notes a Hermes model pinned without a provider.
+
 Explicit flags override defaults in `bin/lanes.json`. Without either, the vendor CLI uses its own configuration. The runner records what was requested and the source of each request: `flag`, `lanes.json` or `lane_default`. These fields describe requested settings; the vendor's own reporting is the place to verify the actual model used.
 
 ## Share facts once, then scope each worker

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "model-orchestrator",
-  "version": "1.0.4",
+  "version": "1.0.6",
   "description": "Model router for AI coding agents: installs routing rules, 8 subagents, hooks and a CLI runner so your AI picks model and effort per task and saves tokens",
   "type": "module",
   "bin": {

package/proof/README.md

@@ -10,8 +10,6 @@ Environment: Node v22.22.3, darwin arm64. Timing varies with startup caches and
 | Lane runner overhead | 63.74 ms median difference | 7 | 2026-09-27 | 2026-10-11 | [script](../proof/scripts/runner-overhead.js) |
 | Empty results flagged | 10 fixtures rejected | 10 | 2026-09-27 | 2026-10-11 | [script](../proof/scripts/missing-results.js) |
 | Acceptance failures blocked | 4 fixtures rejected | 4 | 2026-09-27 | 2026-10-11 | [script](../proof/scripts/check-gate.js) |
-| Main conversation browser tokens | 408147 tokens per browser step (median) | 9775 | 2026-09-26 | 2026-10-27 | author setup, re-measured locally |
-| Small browser subagent tokens | 17197 tokens per step (highest of 5 runs) | 5 | 2026-09-27 | 2026-10-27 | author setup, re-measured locally |
 
 ## Run the proof scripts
 
@@ -21,7 +19,7 @@ node proof/scripts/render.js
 npm test
 ```
 
-The measurement command refreshes reproducible entries and preserves separately sourced author-setup entries. The test suite rejects expired, future-dated or incomplete entries and checks this page against the data. The weekly [refresh workflow](../.github/workflows/proof.yml) reruns the scripts and commits their data and generated page.
+The measurement command refreshes the reproducible entries. An expired entry is a signal to re-measure; the test suite rejects future-dated or incomplete entries and checks this page against the data. The weekly [refresh workflow](../.github/workflows/proof.yml) reruns the scripts and commits their data and generated page.
 
 ## Try the acceptance gate
 
@@ -70,22 +68,6 @@ Kind: reproducible local measurement. Sample size: 4. Measured: 2026-09-27. Expi
 
 Source: [proof/scripts/check-gate.js](../proof/scripts/check-gate.js).
 
-### Main conversation browser tokens
-
-Measured on the author's Claude Code sessions: every browser tool call in the transcripts, counting tokens re-read by the main conversation per step. Median: 408,147 tokens per browser step across 145 sessions and 9,775 browser steps. Sample size counts browser steps.
-
-Kind: measured on the author's setup. Sample size: 9775. Measured: 2026-09-26. Expires: 2026-10-27.
-
-Source: author setup, re-measured locally.
-
-### Small browser subagent tokens
-
-At least 23x fewer tokens per step in this sample: the main conversation median of 408,147 divided by the highest subagent run of 17,197 is 23.73x. Measured on the author's Claude Code sessions, with the same kind of work handed to a small browser subagent. Five runs, tokens divided by steps or tool calls per run: 17,197 (206,369 tokens / 12 steps, 2026-09-26); 4,077 (93,773 / 23), 4,201 (105,034 / 25), 4,585 (91,709 / 20), and 4,489 (94,263 / 21), all four on 2026-09-27. Range: 4,077 to 17,197; median of 5 runs: 4,489. Typical context, using the median of 5 runs: about 91x fewer tokens per step. These are measurements from the author's own sessions, not a controlled comparison or a guarantee for other setups. The four 2026-09-27 runs shared one browser pane, so some steps were spent recovering a drifting tab, which raises the step count and lowers per-step tokens. The first run counted steps; the later runs counted tool calls. Sample size counts runs.
-
-Kind: measured on the author's setup. Sample size: 5. Measured: 2026-09-27. Expires: 2026-10-27.
-
-Source: author setup, re-measured locally.
-
 ## Operation and verification
 
 - **What and why:** executable measurements keep public figures traceable to current output.
@@ -95,6 +77,6 @@ Source: author setup, re-measured locally.
 - **Reads:** package scripts, the installer, runner and acceptance-check runner. Fixture tests use an isolated home and PATH.
 - **Writes:** results.json, this generated page, temporary fixture directories and local fixture logs. The recorder writes gate-demo.cast and gate-demo.gif.
 - **Closed loop:** the workflow fails when measurement or tests fail. GitHub Actions records the failure; repository notification settings decide who receives it. No separate alert service is configured.
-- **Failure modes:** runner behavior changes, an expired catalog snapshot, missing runtime, unavailable write permission, or timing noise. Review the failed job, rerun locally, and send a reproducible issue to the repository maintainers.
-- **Run and verify:** run the commands above, inspect sample arrays and fixture exit codes in results.json, and require npm test to pass. A future-time unit test proves expiry can fail.
-- **Source of truth:** results.json and the scripts it names. Author-setup evidence is added separately by its owner.
+- **Failure modes:** runner behavior changes, missing runtime, unavailable write permission, or timing noise. Review the failed job, rerun locally, and send a reproducible issue to the repository maintainers.
+- **Run and verify:** run the commands above, inspect sample arrays and fixture exit codes in results.json, and require npm test to pass.
+- **Source of truth:** results.json and the scripts it names.

package/proof/results.json

@@ -169,30 +169,6 @@
           "exitCode": 1
         }
       ]
-    },
-    {
-      "id": "author-main-browser-tokens",
-      "label": "Main conversation browser tokens",
-      "kind": "author-setup",
-      "value": 408147,
-      "unit": "tokens per browser step (median)",
-      "measuredAt": "2026-09-26",
-      "method": "Measured on the author's Claude Code sessions: every browser tool call in the transcripts, counting tokens re-read by the main conversation per step. Median: 408,147 tokens per browser step across 145 sessions and 9,775 browser steps. Sample size counts browser steps.",
-      "sampleSize": 9775,
-      "script": "author setup, re-measured locally",
-      "expiresAt": "2026-10-27"
-    },
-    {
-      "id": "author-subagent-browser-tokens",
-      "label": "Small browser subagent tokens",
-      "kind": "author-setup",
-      "value": 17197,
-      "unit": "tokens per step (highest of 5 runs)",
-      "measuredAt": "2026-09-27",
-      "method": "At least 23x fewer tokens per step in this sample: the main conversation median of 408,147 divided by the highest subagent run of 17,197 is 23.73x. Measured on the author's Claude Code sessions, with the same kind of work handed to a small browser subagent. Five runs, tokens divided by steps or tool calls per run: 17,197 (206,369 tokens / 12 steps, 2026-09-26); 4,077 (93,773 / 23), 4,201 (105,034 / 25), 4,585 (91,709 / 20), and 4,489 (94,263 / 21), all four on 2026-09-27. Range: 4,077 to 17,197; median of 5 runs: 4,489. Typical context, using the median of 5 runs: about 91x fewer tokens per step. These are measurements from the author's own sessions, not a controlled comparison or a guarantee for other setups. The four 2026-09-27 runs shared one browser pane, so some steps were spent recovering a drifting tab, which raises the step count and lowers per-step tokens. The first run counted steps; the later runs counted tool calls. Sample size counts runs.",
-      "sampleSize": 5,
-      "script": "author setup, re-measured locally",
-      "expiresAt": "2026-10-27"
     }
   ]
 }

package/proof/scripts/lib.js

@@ -64,7 +64,9 @@ export function validateResults(data, now = new Date()) {
     else {
       if (item.measuredAt > today) errors.push(`${item.id}: measurement is in the future`);
       if (item.expiresAt < item.measuredAt) errors.push(`${item.id}: expiry precedes measurement`);
-      if (item.expiresAt < today) errors.push(`${item.id}: expired ${item.expiresAt}`);
+      // Expiry is a refresh signal, never a gate: an expired figure is reported and
+      // left off the pages, and tests, CI and releases keep running.
+      if (item.expiresAt < today) console.warn(`proof: ${item.id} expired ${item.expiresAt}; rerun node proof/scripts/measure.js`);
     }
     const localAuthorSource = item.kind === 'author-setup' && item.script === 'author setup, re-measured locally';
     if (!localAuthorSource && (typeof item.script !== 'string' || !/^proof\/scripts\/[A-Za-z0-9_-]+\.js$/.test(item.script))) errors.push(`${item.id}: script must name a proof script`);

package/proof/scripts/render.js

@@ -7,7 +7,7 @@ export function proofMarkdown(data) {
   const source = (e, label) => e.kind === 'author-setup' && e.script === 'author setup, re-measured locally' ? e.script : `[${label}](../${e.script})`;
   const rows = data.entries.map(e => `| ${e.label} | ${e.value.toFixed(Number.isInteger(e.value) ? 0 : 2)} ${e.unit} | ${e.sampleSize} | ${e.measuredAt} | ${e.expiresAt} | ${source(e, 'script')} |`);
   const methods = data.entries.map(e => `### ${e.label}\n\n${e.method}\n\nKind: ${e.kind === 'author-setup' ? "measured on the author's setup" : 'reproducible local measurement'}. Sample size: ${e.sampleSize}. Measured: ${e.measuredAt}. Expires: ${e.expiresAt}.\n\nSource: ${source(e, e.script)}.`).join('\n\n');
-  return `# Reproduce the measurements\n\nGenerated from [results.json](results.json). Each figure has a method, sample size, measurement date and expiry. Run the scripts on your own machine to compare.\n\nEnvironment: Node ${data.environment.node}, ${data.environment.platform} ${data.environment.arch}. Timing varies with startup caches and other work on the machine. Synthetic cases show what those fixtures exercise.\n\n| Measurement | Result | Sample size | Measured | Expires | Reproduce |\n|---|---|---|---|---|---|\n${rows.join('\n')}\n\n## Run the proof scripts\n\n\`\`\`sh\nnode proof/scripts/measure.js\nnode proof/scripts/render.js\nnpm test\n\`\`\`\n\nThe measurement command refreshes reproducible entries and preserves separately sourced author-setup entries. The test suite rejects expired, future-dated or incomplete entries and checks this page against the data. The weekly [refresh workflow](../.github/workflows/proof.yml) reruns the scripts and commits their data and generated page.\n\n## Try the acceptance gate\n\n\`\`\`sh\naunx checks ACCEPTANCE_CHECKS.json\naunx checks run ACCEPTANCE_CHECKS.json\n\`\`\`\n\nThe scaffold starts red. Replace the sample with commands that prove your requirements, then put \`aunx checks run ACCEPTANCE_CHECKS.json && <your-release-command>\` in your own release sequence. Commands are local code you review before running. Manual evidence stays UNVERIFIED and blocks the gate.\n\n![Acceptance gate rejects a missing output, then passes after the output exists](gate-demo.gif)\n\nThe [recording script](scripts/record-gate.js) captures real command output into an asciicast, then renders it with an already installed agg. Companion tools are installed by their users.\n\n## Measurement methods\n\n${methods}\n\n## Operation and verification\n\n- **What and why:** executable measurements keep public figures traceable to current output.\n- **Trigger:** weekly schedule, workflow dispatch, or \`node proof/scripts/measure.js\`.\n- **Invocation chain:** workflow -> measurement functions -> isolated Node fixtures -> results.json -> this page -> npm test.\n- **Dependencies:** Node and the repository. The optional GIF recorder uses agg from the asciinema project.\n- **Reads:** package scripts, the installer, runner and acceptance-check runner. Fixture tests use an isolated home and PATH.\n- **Writes:** results.json, this generated page, temporary fixture directories and local fixture logs. The recorder writes gate-demo.cast and gate-demo.gif.\n- **Closed loop:** the workflow fails when measurement or tests fail. GitHub Actions records the failure; repository notification settings decide who receives it. No separate alert service is configured.\n- **Failure modes:** runner behavior changes, an expired catalog snapshot, missing runtime, unavailable write permission, or timing noise. Review the failed job, rerun locally, and send a reproducible issue to the repository maintainers.\n- **Run and verify:** run the commands above, inspect sample arrays and fixture exit codes in results.json, and require npm test to pass. A future-time unit test proves expiry can fail.\n- **Source of truth:** results.json and the scripts it names. Author-setup evidence is added separately by its owner.\n`;
+  return `# Reproduce the measurements\n\nGenerated from [results.json](results.json). Each figure has a method, sample size, measurement date and expiry. Run the scripts on your own machine to compare.\n\nEnvironment: Node ${data.environment.node}, ${data.environment.platform} ${data.environment.arch}. Timing varies with startup caches and other work on the machine. Synthetic cases show what those fixtures exercise.\n\n| Measurement | Result | Sample size | Measured | Expires | Reproduce |\n|---|---|---|---|---|---|\n${rows.join('\n')}\n\n## Run the proof scripts\n\n\`\`\`sh\nnode proof/scripts/measure.js\nnode proof/scripts/render.js\nnpm test\n\`\`\`\n\nThe measurement command refreshes the reproducible entries. An expired entry is a signal to re-measure; the test suite rejects future-dated or incomplete entries and checks this page against the data. The weekly [refresh workflow](../.github/workflows/proof.yml) reruns the scripts and commits their data and generated page.\n\n## Try the acceptance gate\n\n\`\`\`sh\naunx checks ACCEPTANCE_CHECKS.json\naunx checks run ACCEPTANCE_CHECKS.json\n\`\`\`\n\nThe scaffold starts red. Replace the sample with commands that prove your requirements, then put \`aunx checks run ACCEPTANCE_CHECKS.json && <your-release-command>\` in your own release sequence. Commands are local code you review before running. Manual evidence stays UNVERIFIED and blocks the gate.\n\n![Acceptance gate rejects a missing output, then passes after the output exists](gate-demo.gif)\n\nThe [recording script](scripts/record-gate.js) captures real command output into an asciicast, then renders it with an already installed agg. Companion tools are installed by their users.\n\n## Measurement methods\n\n${methods}\n\n## Operation and verification\n\n- **What and why:** executable measurements keep public figures traceable to current output.\n- **Trigger:** weekly schedule, workflow dispatch, or \`node proof/scripts/measure.js\`.\n- **Invocation chain:** workflow -> measurement functions -> isolated Node fixtures -> results.json -> this page -> npm test.\n- **Dependencies:** Node and the repository. The optional GIF recorder uses agg from the asciinema project.\n- **Reads:** package scripts, the installer, runner and acceptance-check runner. Fixture tests use an isolated home and PATH.\n- **Writes:** results.json, this generated page, temporary fixture directories and local fixture logs. The recorder writes gate-demo.cast and gate-demo.gif.\n- **Closed loop:** the workflow fails when measurement or tests fail. GitHub Actions records the failure; repository notification settings decide who receives it. No separate alert service is configured.\n- **Failure modes:** runner behavior changes, missing runtime, unavailable write permission, or timing noise. Review the failed job, rerun locally, and send a reproducible issue to the repository maintainers.\n- **Run and verify:** run the commands above, inspect sample arrays and fixture exit codes in results.json, and require npm test to pass.\n- **Source of truth:** results.json and the scripts it names.\n`;
 }
 export function renderPage() {
   const data = readResults();

package/src/install.js

@@ -823,7 +823,7 @@ export function planFiles(opts) {
           enabled: selected.filter((a) => a.facts.cliRun).map((a) => a.id),
           defaults: Object.fromEntries((opts.effortAuto || []).map((lane) => [lane, { effort: 'auto' }])),
           note: 'Lanes cli-run may call. Edit to enable or disable a lane. A lane not listed here exits 13 (unavailable).',
-          defaultsNote: 'Pin what a lane runs with, so the route in your docs is the route that runs: "defaults": {"' + (selected.find(a => a.facts.cliRun)?.id || '<lane>') + '": ' + JSON.stringify({ model: '<model-id>', ...(LANE_FLAGS[selected.find(a => a.facts.cliRun)?.id]?.effort ? { effort: 'high' } : {}) }) + '}. Left empty, a lane inherits its own config file, which cli-run cannot see and does not guess. `--model` and `--effort` override this per call, and `--doctor` prints what each lane is pinned to. Every enabled lane takes a model; the runner reports which lanes support an effort flag.'
+          defaultsNote: 'Pin what a lane runs with, so the route in your docs is the route that runs: "defaults": {"' + (selected.find(a => a.facts.cliRun)?.id || '<lane>') + '": ' + JSON.stringify({ model: '<model-id>', ...(LANE_FLAGS[selected.find(a => a.facts.cliRun)?.id]?.effort ? { effort: 'high' } : {}) }) + '}. Left empty, a lane inherits its own config file, which cli-run cannot see and does not guess. `--model` and `--effort` override this per call, and `--doctor` prints what each lane is pinned to. Every enabled lane takes a model; the runner reports which lanes support an effort flag. Hermes also takes "provider" beside "model", so the model goes to a provider that serves it.'
         },
         null,
         2

package/templates/intermediate/CLI-RUN.md

@@ -97,13 +97,15 @@ Replace the placeholder with a current vendor model ID before using this example
 
 The runner supports these lanes whether or not you selected them.
 
-| Lane | Model flag | Effort flag |
-|---|---|---|
-| grok | `-m` | `--reasoning-effort` |
-| codex | `-m` | `-c model_reasoning_effort="LEVEL"` |
-| agy | `--model` | `--effort` |
-| hermes | `-m` | `--reasoning` |
-| qwen | `-m` | Unsupported; an effort request is a usage error |
+| Lane | Model flag | Effort flag | Provider flag |
+|---|---|---|---|
+| grok | `-m` | `--reasoning-effort` | Unsupported |
+| codex | `-m` | `-c model_reasoning_effort="LEVEL"` | Unsupported |
+| agy | `--model` | `--effort` | Unsupported |
+| hermes | `-m` | `--reasoning` | `--provider` |
+| qwen | `-m` | Unsupported; an effort request is a usage error | Unsupported |
+
+Hermes signs in to many providers, and a model sent to a provider that does not serve it fails with HTTP 400. Pin both with `--provider` and `--model`, or with `"provider"` beside `"model"` under `"hermes"` in `bin/lanes.json` defaults. A provider with no model is a usage error. That failure is reported as `rejected` (exit 16) with a model/provider mismatch line; `hermes model` repairs the pairing in Hermes itself.
 
 When using `--effort auto`, treat its medium/high selection as a bounded heuristic; an audit has a high floor. Use explicit high for builds and xhigh where supported for security-critical or irreversible work. The vendor validates its own effort names and reports unsupported values through the failure class.