@alexeiled/claude-router 1.23.1 → 1.25.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (81) hide show
  1. package/.claude-plugin/plugin.json +1 -1
  2. package/CHANGELOG.md +62 -1
  3. package/README.md +39 -7
  4. package/docs/architecture.md +27 -4
  5. package/docs/auto-engine.md +152 -0
  6. package/docs/configuration.md +47 -3
  7. package/docs/evaluation.md +222 -3
  8. package/docs/native-router.md +11 -3
  9. package/docs/providers.md +28 -0
  10. package/docs/router-band.svg +124 -133
  11. package/docs/router-pane-classifier.svg +24 -23
  12. package/docs/router-pane-now.svg +9 -7
  13. package/docs/router-pane-policy.svg +4 -4
  14. package/docs/router-pane-routes.svg +8 -8
  15. package/docs/router-pane-usage-whatif.svg +14 -14
  16. package/docs/router-pane-usage.svg +20 -20
  17. package/docs/routing-modes.md +13 -4
  18. package/docs/user-guide.md +79 -19
  19. package/hooks/native-router.mjs +1194 -215
  20. package/lib/activity-stats.mjs +23 -4
  21. package/lib/activity.mjs +7 -5
  22. package/lib/auto-engine.mjs +414 -0
  23. package/lib/auto-route.mjs +253 -0
  24. package/lib/band.mjs +35 -440
  25. package/lib/classifier-client.mjs +12 -8
  26. package/lib/config-file.mjs +7 -2
  27. package/lib/config.mjs +64 -15
  28. package/lib/cost.mjs +5 -1
  29. package/lib/costs.mjs +4 -2
  30. package/lib/decisions.mjs +80 -5
  31. package/lib/display.mjs +69 -20
  32. package/lib/facts.mjs +30 -0
  33. package/lib/modes.mjs +8 -4
  34. package/lib/panel.mjs +75 -2163
  35. package/lib/policy.mjs +7 -6
  36. package/lib/providers.mjs +7 -0
  37. package/lib/replay-trace.mjs +550 -0
  38. package/lib/replay.mjs +718 -0
  39. package/lib/route-line.mjs +46 -0
  40. package/lib/route.mjs +35 -21
  41. package/lib/savings.mjs +14 -3
  42. package/lib/subagents.mjs +41 -11
  43. package/lib/subcommands.mjs +60 -0
  44. package/lib/ui/fallback.mjs +22 -0
  45. package/lib/ui/graphical/band.mjs +256 -0
  46. package/lib/ui/graphical/charts.mjs +163 -0
  47. package/lib/ui/graphical/classifier.mjs +153 -0
  48. package/lib/ui/graphical/frame.mjs +227 -0
  49. package/lib/ui/graphical/kit.mjs +156 -0
  50. package/lib/ui/graphical/meter.mjs +25 -0
  51. package/lib/ui/graphical/now.mjs +353 -0
  52. package/lib/ui/graphical/policy.mjs +56 -0
  53. package/lib/ui/graphical/routes.mjs +264 -0
  54. package/lib/ui/graphical/squares.mjs +29 -0
  55. package/lib/ui/graphical/usage.mjs +376 -0
  56. package/lib/ui/model/band.mjs +439 -0
  57. package/lib/ui/model/classifier.mjs +308 -0
  58. package/lib/ui/model/draft.mjs +62 -0
  59. package/lib/ui/model/format.mjs +41 -0
  60. package/lib/ui/model/frame.mjs +359 -0
  61. package/lib/ui/model/now.mjs +390 -0
  62. package/lib/ui/model/policy.mjs +194 -0
  63. package/lib/ui/model/routes.mjs +392 -0
  64. package/lib/ui/model/usage-agents.mjs +311 -0
  65. package/lib/ui/model/usage-breakdown.mjs +237 -0
  66. package/lib/ui/model/usage-shared.mjs +58 -0
  67. package/lib/ui/model/usage.mjs +330 -0
  68. package/lib/ui/terminal/band.mjs +220 -0
  69. package/lib/ui/terminal/classifier.mjs +142 -0
  70. package/lib/ui/terminal/frame.mjs +169 -0
  71. package/lib/ui/terminal/kit.mjs +133 -0
  72. package/lib/ui/terminal/now.mjs +300 -0
  73. package/lib/ui/terminal/policy.mjs +52 -0
  74. package/lib/ui/terminal/routes.mjs +167 -0
  75. package/lib/ui/terminal/usage.mjs +363 -0
  76. package/lib/ui/terminal/wrap.mjs +37 -0
  77. package/lib/ui/tokens.mjs +106 -0
  78. package/lib/usage.mjs +5 -5
  79. package/lib/view.mjs +5 -2
  80. package/package.json +3 -2
  81. package/types/index.d.ts +15 -0
@@ -4,6 +4,225 @@ The native Mod has no measured savings result. The pane reports Claude's own usa
4
4
 
5
5
  This page separates an older gateway experiment from a shadow replay of native policy. Neither result measures answer quality or proves lower real spend. [Activity routing](#activity-routing) adds two checks of the activity label, with no savings result either.
6
6
 
7
+ ## Automatic-routing shadow proposals
8
+
9
+ Opt-in `routingEngine: "shadow"` records a demand and target proposal against the real table incumbent for each
10
+ applicable decision. It does not run the proposed route or maintain a simulated route history. These rows do not
11
+ establish savings or answer quality; a next-turn cost estimate is not an observed cost or a counterfactual saving.
12
+ The bounded decision records are not replay traces. See [Configuration](configuration.md#routing-engine).
13
+
14
+ ## Replay trace recorder
15
+
16
+ In `routingEngine: "shadow"`, the main-turn hooks also keep text-free replay inputs separately from decision rows:
17
+ the initial configured and mode-transformed routing policy, provider/cache semantics and model prices; classifier
18
+ tier/activity/phase vectors; failure fingerprints and occurrence indexes; pins, modes, resets, context/availability
19
+ facts and observed moves; and correlated request/reply model, effort and token counters. Checkpoint evidence uses
20
+ message indexes and success flags, not commands, tool results or filesystem-path fields, so a later replay can use
21
+ its own switch cursor. Validated routing aliases and model IDs are preserved exactly, including path-shaped IDs;
22
+ classifier endpoints, credentials and the config file's nativePath are not copied.
23
+
24
+ Records use `trace:v1:<session id>`: 30-day retention, 128 KiB per session, 1 MiB total trace budget and a measured
25
+ 3 MiB guard on the shared plugin store. Pending turns are bounded too. A cap discards the entire pending turn,
26
+ keeps the earlier contiguous prefix usable and counts excluded turns/events. Late starts, resume/reload, gaps,
27
+ conflicts and lost writes are incomplete. A failed write taints memory and persists that taint on the next success;
28
+ an unavailable store cannot guarantee a durable failure marker. A clean session close is not required.
29
+ Freshness requires the first main request (index 0), no saved loop or trace, and readable normalized request
30
+ history with no assistant messages. Injected startup reminders are user rows and do not make a trace incomplete;
31
+ `classic.SessionStart` is not required. Unreadable or malformed history, reload and late enable are conservative.
32
+ Compaction preserves an existing trace with an explicit reset; `/clear` permits fresh history under a new session ID.
33
+
34
+ Run `/router trace export` to merge retained traces into `router-traces.json` beside the selected profile's
35
+ router config. The command also works after switching back to `table`; ordinary table turns do no trace I/O.
36
+ Exports overwrite their previous JSON after merging; near 3 MiB, a new dated `router-traces-*.json` file receives
37
+ the retained snapshot instead. Keep all export files for offline replay. Known taint is never erased by a longer
38
+ or newer clean prefix. **Delete history** clears stored traces in any mode, but exported files remain user-owned:
39
+ the host filesystem API has no unlink operation.
40
+
41
+ Recent active sessions have a one-hour last-write lease against budget eviction; stale activity cannot defeat
42
+ 30-day retention. Store scans, quota checks and deletion are best effort across processes, not atomic reservations.
43
+ There is no live exposure, routing-quality result or savings claim from this recorder yet.
44
+
45
+ ## Offline automatic-engine replay
46
+
47
+ Run the tool from a checkout with **all** incremental exports (shell-expanded file arguments, no implicit profile
48
+ access). It reads only those files and writes anonymous JSON to stdout; it never changes a router or provider profile.
49
+
50
+ ```sh
51
+ node scripts/replay-traces.mjs /exports/router-traces.json /exports/router-traces-*.json > replay-report.json
52
+ node scripts/replay-traces.mjs --reversal-window 3 test/fixtures/replay-traces.json
53
+ node --test test/replay.test.mjs
54
+ ```
55
+
56
+ Omit the dated-file argument when there are no rollover files. The second command uses only synthetic data.
57
+ The implementation follows [design §5.1 and §6 step 5](plans/2026-11-16-auto-routing-design.md#51-replay-traces-and-offline-replay).
58
+
59
+ 1. Merge sessions with the recorder's `mergeTraceExport`: preserve lost-write taint, overlap conflicts and capped
60
+ counts. Keep usable capped contiguous prefixes; report their omitted turns/events separately.
61
+ 2. Replay the table chain from **only** its initial header, applying config, controls, history preparation/resets,
62
+ choose/observed/continue operations and responses. Compare every recorded decision and request model/effort.
63
+ An unexplained mismatch on any otherwise usable prefix returns `status: "blocked"`, exits 1 and prints **no**
64
+ auto/session totals. It remains in the fidelity denominator. Taint/gaps/incompleteness are explained exclusions.
65
+ 3. Only after fidelity passes, validate independent-replay evidence. Missing checkpoints, required boolean facts,
66
+ unconditional continuation modality or activity indexes produce `auto-input-incomplete` and exit 1, without
67
+ silently excluding a usable fidelity session. Earlier exports can lack these fields.
68
+ 4. Simulate auto with its own demand/votes, incumbent, cache, routed-demand/provenance and switch/dwell cursors.
69
+ Classifier decisions advance votes; continuations reevaluate constraints and pending moves without adding votes.
70
+ Checkpoints and observed activity are reconstructed from text-free indexes against its own cursor. Only an
71
+ accepted bounded execute phase discounts demand; observed edits can remove it. No activity route table is used.
72
+
73
+ **Mode meaning:** fidelity and both comparisons follow the recorded mode-change history. Output is partitioned by
74
+ the mode active on each request, not three artificial reruns. Table uses the recorded transformed config; auto
75
+ uses the recorded base policy, with mode only as the capability margin. There is no forced-mode option.
76
+
77
+ The report has `fidelity`, `coverage`, `exposureDiagnostic` and anonymous numbered `sessions[].modes`. Each mode
78
+ contains both engines' conditional cost range, priced/unpriced request counts, effort mix, switches and disagreement
79
+ counts grouped by effective demand/activity. Unknown catalog capabilities are counted explicitly. Anthropic seeds four balanced base-route pairs as
80
+ user-derived policy judgments; other pairs and provider presets remain unknown. These seeds and their replay are not
81
+ quality-calibration evidence.
82
+
83
+ Switches are computed on the full session before mode partitioning and attributed to the **destination** mode.
84
+ A reversal closes A→B→A within the explicit `--reversal-window N` main turns (default 3); `reversals` counts closing
85
+ legs, `reversalMoves` counts participating legs, and `oneWay` counts unpaired moves. Rates are per 100 captured turns
86
+ in that mode. Pins do not count as switches and restore the pre-pin route/demand/dwell baseline; every pinned
87
+ continuation still checks eligibility. Manual intervals, history resets and external native interventions break the
88
+ policy-switch chain, including reversal pairing.
89
+
90
+ ### Coverage and estimate limits
91
+
92
+ - `coverage.exportCohort` ratios describe **only captured/exported sessions**, a lower-bound cohort. Exports do not
93
+ count evicted or untraced sessions; `coverage.allSessions` denominators and its 80% gate stay unavailable.
94
+ Sufficient future metadata would include capture-eligible session starts (including untraced/evicted sessions),
95
+ total main turns and per-mode eligible routed turns over the same capture window. This tool invents none of it.
96
+ - Exposure thresholds (30 complete sessions, 1,000 main turns, 200 per evaluated mode) are diagnostics only.
97
+ Captured main turns are separate from eligible unpinned classifier-decision turns; manual-only sessions/turns
98
+ do not count as engine exposure. Freshness is unverified and the exposure gate remains `not-established`.
99
+ - Cost freezes observed total prompt/output lengths, timestamps, failures and tool calls. Auto simulates its own
100
+ full-prefix cache scenario, never the table's cache-hit tokens. Explicit cache warmth uses the existing conservative
101
+ five-minute helper; automatic caches retain hit/miss bounds, with no invented TTL guarantee. Effort-specific,
102
+ disabled, long-context and off-peak pricing use the existing pure cost helpers. These are **conditional estimates**,
103
+ not counterfactual savings, actual bills or paired quality results.
104
+ - Missing usage/prices and substituted replies make the affected total range `null`, not free. An explicitly named
105
+ `knownPricedRange` is only the priced subtotal. No transcript-length/quality change from a hypothetical request
106
+ is modeled.
107
+ - Fidelity distinguishes table echoes from genuine third-model/native fallbacks. Auto freezes actual external
108
+ interventions, response substitutions and routed null-stop suspension, rather than misreading a table echo as
109
+ its own fallback. It cannot predict a provider's fallback for a different hypothetical auto request.
110
+ - Default output includes no session IDs, prompts, tool text, model identifiers or paths. Input errors name the
111
+ numbered file/structural field, not its path. The CLI cannot discover omitted exports or verify their provenance.
112
+
113
+ ## Paired benchmark proposal
114
+
115
+ The [automatic-routing design §5.3](plans/2026-11-16-auto-routing-design.md#53-paired-benchmark-smoke-gate-for-quality)
116
+ requires paired `table`/`auto` runs, three modes and two repeats, with per-task checks and discordances. The
117
+ 24 synthetic expected-output cases in `test/fixtures/benchmark-routing.json` are harness fixtures, not calibrated
118
+ engineering tasks or a quality result. Their activity labels are fixture metadata, not reference classifications.
119
+ Eight are marked held out in the fixture split. The separate engineering proposal below needs fixtures and review
120
+ before any evaluation; shadow-disagreement sources are pending.
121
+
122
+ ```sh
123
+ node scripts/benchmark-routing.mjs --dry-run
124
+ node scripts/benchmark-routing.mjs --dry-run --smoke
125
+ node --test test/benchmark-routing.test.mjs
126
+ ```
127
+
128
+ The dry run prints a SHA-256, run counts, serialized token/rate assumptions and a configured-price scenario.
129
+ Dry-run planning uses the built-in catalog maximum prices through `ratesAt`; a live smoke preflight reads the selected
130
+ work profile's catalog without changing it. Cold-write multipliers apply only to explicit caches. The 10,000-input
131
+ assumption is not a limit; live smoke sets `CLAUDE_CODE_MAX_OUTPUT_TOKENS=512`, one agentic turn and a $0.40
132
+ CLI budget per call. These are not a proxy-price quotation or a bill. A run estimates before launching and reserves
133
+ $1.20 for three calls. An exclusive reservation file under `.pi/` refuses automatic reruns; further spending needs
134
+ review, not a hidden retry. The 288 planned task runs are for the toy mechanics fixtures, not an engineering benchmark cost estimate.
135
+ An engineering task run may use multiple inference calls, tools and retries; it needs new token, request, effort
136
+ and provider budgets plus lead approval before any run. No full matrix is authorized or run.
137
+
138
+ `--live --smoke` is limited to the three development tasks, `table`/`balanced`, one repeat, using the existing
139
+ installed release through `zsh -ic` and `claude-mp-work -p`. It loads no checkout plugin. The selected profile must
140
+ still be `table`; `--setting-sources user`, strict empty MCP and no tools limit the surface. Init metadata must
141
+ identify exactly one router at the installed user path. Profile contents are hashed before/after each call. The
142
+ wrapper's normal MCP housekeeping is permitted; the original active-environment bytes/existence are restored and
143
+ verified after success, failure, timeout or handled interrupt. SIGKILL, power loss and independently detached
144
+ processes are not covered. Process-group cleanup is best effort; Windows cannot kill a descendant tree via Node
145
+ alone. The smoke establishes harness mechanics only, not evaluation of this checkout or either engine.
146
+
147
+ Offline system-boundary tests exercise independent directories, output checks, deadlines, output limits, budgets,
148
+ unknown costs and paired discordance reporting. The harness checks its own and the manifest's hashes around child
149
+ execution; this detects persistent changes, not transient same-user writes or an OS sandbox. The checks run in the
150
+ parent outside the child's working directory. Unknown/zero total or per-model CLI prices stop further calls rather
151
+ than implying free usage. Reports sum every supplied attempt, including failures, and leave missing/unpriced
152
+ totals unknown. Duplicate attempts within a paired cell are reported as ambiguous, never silently replaced by a
153
+ passing rerun.
154
+ CLI-reported USD can use rates different from the selected catalog and may omit proxy retries, classifier spend or
155
+ upstream failed calls: actual billing remains unknown.
156
+ No comparison of quality, savings or measured routing performance follows from these checks.
157
+
158
+ The checked-in [installed-release smoke record](../experiments/mod-router/results/benchmark-harness-smoke.json)
159
+ contains three passing external checks, 19,298 ms summed subprocess wall time and $0.0098816 CLI-reported price
160
+ estimates. It records the manifest/config hashes and successful restoration/provenance checks without prompts,
161
+ profile paths or credentials. CLI usage metadata names DeepSeek Flash for two calls and GLM Flash for one; backend
162
+ identity is not independently verified.
163
+ These are harness-validation observations on installed Router 1.23.1, not a paired result for this checkout.
164
+
165
+ ### Engineering task set: draft only
166
+
167
+ The following 24 synthetic engineering tasks are a proposed **separate** evaluation set, not implemented fixtures.
168
+ Difficulty and activity are author hypotheses, not classifier references. Each row needs a self-contained repository
169
+ fixture and an independently held-out automatic oracle outside model control. Freeze the fixture/manifest hash and
170
+ split before capability or margin tuning; do not run or inspect held-out outcomes during tuning. All rows are authored
171
+ examples: shadow-disagreement sourcing is **pending** and must be added or documented before the set is finalized.
172
+
173
+ | ID | Split | Activity / demand hypothesis | Task | Independent automatic oracle |
174
+ | --- | --- | --- | --- | --- |
175
+ | E01 | dev | code / low | Add integer clamping to a small library | Boundary and type tests |
176
+ | E02 | dev | debug / medium | Repair a parser that mishandles escaped delimiters | Existing tests plus hidden escaped and empty cases |
177
+ | E03 | dev | code / medium | Add pagination to a small service across handler and storage modules | Fixture API and deterministic pagination tests |
178
+ | E04 | heldout | code / medium | Implement stable deduplication across a CLI and helper | Output and duplicate-order tests |
179
+ | E05 | dev | security / high | Reject traversal and symlink escapes in an archive extractor | Traversal, symlink and valid-path tests |
180
+ | E06 | dev | debug / high | Fix a cross-module stale cache invalidation defect | Concurrent update and expiry regression tests |
181
+ | E07 | heldout | debug / medium | Fix time boundary logic for expiry at exact deadline | Fake-clock boundary tests |
182
+ | E08 | dev | ops / medium | Repair CI script so test and lint failures reach the exit status | Controlled subprocess exit-code matrix |
183
+ | E09 | dev | review / medium | Identify seeded unsafe defaults in a short configuration | Exact structured findings against injected defects |
184
+ | E10 | heldout | debug / high | Repair nondeterministic retry handling after partial network failures | Deterministic mocked-service request and retry tests |
185
+ | E11 | dev | code / high | Add idempotent transaction handling across API and persistence | Duplicate/concurrent request and rollback tests |
186
+ | E12 | dev | debug / medium | Diagnose and fix an off-by-one token budget after a tool continuation | Transcript fixture and boundary tests |
187
+ | E13 | heldout | code / low | Add case-insensitive header lookup preserving original values | Mixed-case and missing-header tests |
188
+ | E14 | dev | code / medium | Migrate a versioned configuration while preserving unknown fields | Old/new config round-trip and rejection tests |
189
+ | E15 | dev | debug / high | Fix matching failure signatures after an edit and repeated tool failure | Scripted transcript with two failures and edit tests |
190
+ | E16 | heldout | security / high | Correct authorization checks across two endpoints sharing a store | Cross-user read/write and allowed-access tests |
191
+ | E17 | dev | refactor / medium | Replace duplicate parsing in two modules without changing behavior | Characterization and randomized input tests |
192
+ | E18 | dev | ops / low | Correct shell quoting for filenames containing spaces | Subprocess tests on spaces and metacharacters |
193
+ | E19 | heldout | code / high | Implement bounded parallel workers with cancellation and cleanup | Deterministic concurrency and child-cleanup tests |
194
+ | E20 | dev | debug / medium | Fix a multi-file regression after two failed test attempts and an edit | Scripted tool history plus external final tests |
195
+ | E21 | dev | docs / low | Correct a documented CLI example for a real edge-case flag | Execute example against fixture CLI |
196
+ | E22 | heldout | ops / medium | Repair a build pipeline that masks errors in a chained command | Failure-injection exit-code tests |
197
+ | E23 | dev | review / high | Find seeded race and missing transaction rollback | Machine-readable findings with location and defect checks |
198
+ | E24 | heldout | debug / medium | Fix Unicode normalization inconsistency across reader and index | Composed/decomposed matching and preservation tests |
199
+
200
+ These are not yet the §5.3 benchmark: the tasks, initial repositories, test oracles, tool-history presentation,
201
+ permissions and expected output need independent implementation and review. No engineering benchmark or held-out
202
+ outcome has been collected. Difficulty does not establish a capability value.
203
+
204
+ ### Full paired-run isolation: proposal only
205
+
206
+ Auto routing is available by opt-in in 1.24.0. The full paired benchmark adapter remains disabled pending checkout/profile
207
+ isolation and approval; a full live matrix always refuses. A native pass-through or table
208
+ run is never relabeled `auto`. The current hook reads `routingEngine` only from the selected profile file; there is
209
+ **no** run-scoped environment/settings override. Project `.claude/router.json` can set mode only.
210
+
211
+ Before the real paired evaluation, choose and implement one code-provenance path:
212
+
213
+ - Add an explicit supplied-router option to `ce`'s model-proxy path, matching its normal launch behavior. This can
214
+ preserve existing implicit authentication but needs wrapper maintenance and must disable the marketplace copy.
215
+ - Launch `claude-proxy` directly with per-run settings that disable the marketplace router and a checkout
216
+ `--plugin-dir`. This gives precise code provenance but must reuse an approved authentication interface, never
217
+ copy credentials or silently bypass account policy.
218
+ - Run after a release that includes the evaluated implementation. This avoids two plugin copies but does not
219
+ itself isolate the engine selection or establish checkout provenance.
220
+
221
+ Every path **also** needs a separately approved run-scoped engine override or safely isolated profile configuration
222
+ that keeps existing implicit authentication without copying secrets. Only then can each child select its own
223
+ `table` or `auto` without editing the user's real router file. No wrapper modification, new engine override,
224
+ authentication mechanism, live auto wiring or cutover is implemented here.
225
+
7
226
  ## Provider profiles
8
227
 
9
228
  [Provider defaults](providers.md#built-in-defaults) are starting policies, not measured per-activity quality or savings results.
@@ -178,7 +397,7 @@ Method, per main reply that routing chose:
178
397
 
179
398
  Checks:
180
399
 
181
- - Unit tests (`test/savings.test.mjs`) cover a session that saves, one that costs more with Haiku as your model, equal routes at exactly 0, cache expiry at 5m and 1h, a compaction, a change of your model or effort, the 100K surcharge, an unpriced model, and a corrupted store. `test/hook-routing.test.mjs` covers two sessions finishing turns at the same moment, a `/model` change mid-session that keeps each reply against its own model, and a running turn's replies counted once before and after the turn is recorded. `test/usage.test.mjs` covers calendar days across month ends and daylight saving changes, the periods, the record cap and expiry, the trend buckets and what is worth a look; `test/usage-view.test.mjs` covers the Usage tab.
400
+ - Unit tests (`test/savings.test.mjs`) cover a session that saves, one that costs more with Haiku as your model, equal routes at exactly 0, cache expiry at 5m and 1h, a compaction, a change of your model or effort, the 100K surcharge, an unpriced model, and a corrupted store. `test/hook-usage.test.mjs` covers two sessions finishing turns at the same moment, a `/model` change mid-session that keeps each reply against its own model, and a running turn's replies counted once before and after the turn is recorded. `test/usage.test.mjs` covers calendar days across month ends and daylight saving changes, the periods, the record cap and expiry, the trend buckets and what is worth a look; `test/usage-view.test.mjs` covers the Usage tab.
182
401
  - A one-off replay, not checked in, of this machine's main-conversation transcripts from 2026-10-01 to 2026-10-10 (nine sessions with 10 or more replies) gave $0.00 for every session that ran only on the model it was compared with: three on Opus 5.5, one on Haiku 5.5. Every main-conversation cache write in them was a one-hour write, which is what the lifetime rule gives on this Max plan.
183
402
  - A live Claude Code 2.1.296 session on 2026-10-10, Opus 5.5 at `high` with turns pinned to Haiku 5.5, showed **Routed replies $0.48** beside **Cost $0.480 reported by Claude** for the same 16 replies; its last Haiku reply had a 114K-token prompt, which agrees only at the 5x rate. A Haiku 5.5 session with a pinned Opus turn showed $0.45 routed against $0.446 reported.
184
403
 
@@ -208,7 +427,7 @@ Method:
208
427
  Checks:
209
428
 
210
429
  - `test/subagents.test.mjs` covers each pass reason, the pin rule, the classify mapping, core's model estimate, the router's move, the subagent lifetime, Haiku's long-context price, and the store's days, merge, reset, bad values and records from before 1.14.0. `test/hook-subagents.test.mjs` covers `on` (the routed model), a fork, an explicit model, a teammate, a workflow agent and a pinned project agent (each passes), a classifier failure (core's model), parallel spawns, a spawn-path pause that leaves main routing alone, respawns, session end, Delete history, and the Usage and Now tabs. `test/usage-view.test.mjs` covers the block by period, an open row, a narrow pane and undated records; `test/hook-band.test.mjs` the band in a subagent's transcript.
211
- - `experiments/mod-router/results/agent-spawn-probe.json` records what Claude Code 2.1.296 does with a spawn's model, frontmatter and workflow agents.
430
+ - `experiments/mod-router/results/agent-spawn-probe.json` records what Claude Code 2.1.296 does with a spawn's model, frontmatter and workflow agents. On 2.1.321 a live probe showed that a model rewrite applies to workflow agents, and that a resumed subagent raises `agent.spawn` with `e.resume` and keeps its first model.
212
431
  - `experiments/mod-router/results/subagent-routing-live.json` is a live `on` run on 2026-10-10 with Sonnet as the parent. Explore was sent Haiku and ran on it. general-purpose was classified `standard`. A pinned project agent and an explicit `opus` call passed through. The store counted each one.
213
432
 
214
433
  When to turn it `on`:
@@ -222,7 +441,7 @@ Keep it if requests per agent and respawns per type stay no higher than before y
222
441
  Limits:
223
442
 
224
443
  - Answer quality is not measured.
225
- - Workflow agents cannot be routed. In the maintainer's sessions they were 22 of 24 Opus subagents (`workflow-subagent-models.json`), so most inherited Opus spend there is out of reach. `CLAUDE_CODE_SUBAGENT_MODEL` is the lever for them.
444
+ - Workflow agents are classified by default (`subagents.types['workflow-subagent']`). In the maintainer's sessions they were 22 of 24 Opus subagents (`workflow-subagent-models.json`), so this reaches most of the inherited Opus spend. The routed effect is not measured yet, and answer quality is not either.
226
445
  - `on` estimates core's model. It does not apply the family-alias rule or the `availableModels` substitution.
227
446
  - Respawns are a proxy. A parent may spawn the same type twice on purpose.
228
447
  - Records follow the savings records' storage, with the same gaps.
@@ -1,11 +1,13 @@
1
1
  # Native router: behavior and evidence
2
2
 
3
- This guide describes the Mod shipped in the plugin. Claude Code 2.1.289 is the minimum tested version. The router changes `model` and `effort` on main-conversation steps. Claude Code handles the Anthropic request, credentials, stream, tool execution, and cost ledger. Subagent steps keep their model and effort; the router chooses an agent's model once, at spawn, unless its definition pins one ([configuration](configuration.md#subagent-routing)).
3
+ This guide describes the Mod shipped in the plugin. Claude Code 2.1.321 is the minimum tested version. The router changes `model` and `effort` on main-conversation steps. Claude Code handles the Anthropic request, credentials, stream, tool execution, and cost ledger. Subagent steps keep their model and effort; the router chooses an agent's model once, at spawn, unless its definition pins one ([configuration](configuration.md#subagent-routing)).
4
4
 
5
5
  ## Turn behavior
6
6
 
7
7
  At the first main step of a logical turn, the Mod reads recent message text and asks the active classifier (Jev, Clef, Clef Flash, Clef Omni, OpenAI, or Ollama) for a tier, and for the turn's activity unless `activityRouting` is `off`. It applies local policy and saves the route. Tool continuations reuse that route, except that a turn whose tool calls edit code moves up to the `code` route, and every `policy.reviewEveryRequests` requests a background classifier review may move it either way ([Changes during a task](user-guide.md#changes-during-a-task)). Before each continuation, context fit is checked again. A large tool result can move the step to a model with a larger window.
8
8
 
9
+ Claude Code sends a model change on every step, but an effort change on a tool-result step only when `CLAUDE_CODE_MODEL_CAPABILITIES=per_turn_effort` is set in its environment (the router does not set it). Without it a turn keeps the effort of its first request, so the band and the decision record can name an effort that did not run; see [what we do not know yet](activity-routing.md#what-we-do-not-know-yet).
10
+
9
11
  The classifier receives the current prompt and at most six preceding user or assistant text messages. Each item is limited to 1,200 characters. It does not receive system messages, tool inputs, or tool results. Claude Code's Anthropic credentials never enter the classifier request. Jev, Clef, Clef Flash, Clef Omni, and OpenAI receive the text on their services; Ollama receives it on this machine. [Configuration](configuration.md#classifiers) lists each host.
10
12
 
11
13
  The default tiers are defined in [Configuration](configuration.md#built-in-defaults). Policy uses vote hysteresis, repeated tool errors, context fit, model availability, and switching-cost estimates. The [architecture](architecture.md#request-flow) explains the event boundary and failure handling.
@@ -23,15 +25,20 @@ The activity is what the turn produces: `code`, `debug`, `explore`, `plan`, `rev
23
25
  | `/router activities <off\|on>` | Set the activity routing mode. Written to `router.json` at once; **Undo** in the pane reverts it. |
24
26
  | `/router mode <cost\|balanced\|quality\|default>` | Set or clear this session's [routing mode](routing-modes.md), from the next turn. Kept for a resume in `$.store` `routing-mode:<session id>`. Never aborts a turn. |
25
27
  | `/router decisions [n]` | Print this session's last `n` routing decisions (default 20, at most 60): origin, classifier answer, task, tier, route, reason with the mass and bar it compared, and each turn's end. No text. |
28
+ | `/router line <off\|moves\|all>` | Set when a turn leaves a route line (default `moves`: only when its route moved). Kept in `$.store` as `route:line`, across sessions. |
26
29
  | `/model <name>` | Select a model and turn routing off. `/router auto` turns it back on. |
27
30
 
28
- The pane's **Set up** and **Credentials → Edit** buttons open Claude Code's secure plugin configuration. Claude Code refuses `$.command.run` from inside a `command.run` hook, so the Mod starts that dialog from a timer after the press returns.
31
+ The prompt's typeahead (`prompt.autocomplete`) offers these words after `/router ` and their values after `pin `, `mode `, `activities ` and `line `. It starts with the first typed character, because the engine passes a token that is never empty. The command, the typeahead and the argument hint all read one table, `lib/subcommands.mjs`, and a test runs the command with every word and value in it.
32
+
33
+ The route line is one `$.ui.place` row per routed turn, drawn by a `ui.render` hook on `Placed`. At `turn.complete` of the main loop, skipping an aborted turn, a subagent's turn and a turn with routing off, the router writes the route that ran (model, effort, tier, reason, activity and the policy's estimate, never text) to `$.state` under `router.lines` with the row's id (`rl-` and the turn id, made safe for `$.ui.place`), then places the row. The hook draws only rows whose `plugin` is the router's and leaves the rest to `next(e)`. A surface that cannot draw the row at that moment (`isPlaced: false`) is logged to the debug log and the turn goes on. Claude Code does not bring rows back on `--resume` and ends them at a compaction, so the records are not read again.
34
+
35
+ In a terminal, the pane's **Set up** and **Credentials → Edit** buttons open Claude Code's secure plugin configuration. The desktop app, VS Code and mobile cannot open that dialog, so there both buttons show a card with the command to run in a terminal (the band's **Set up** does the same). Claude Code refuses `$.command.run` from inside a `command.run` hook, so the Mod starts that dialog from a timer after the press returns.
29
36
 
30
37
  Mode belongs to a Claude Code session. Clear starts a new session with routing on. Resume restores the saved mode for that session. A new or forked session has a new ID: it starts with routing on for a model that one of the tiers routes to and with routing off for any other model, as a fresh launch does. With activity routing `on`, a model that only an activity override uses also starts the session with routing on. A pin applies to one logical turn, then the prior route resumes. A history reset clears votes and cache evidence; the current turn keeps its route, pin and any native fallback.
31
38
 
32
39
  ## Router pane
33
40
 
34
- The status band is one line above the prompt: a tier meter, the model, the activity, a short reason, classifier support and the switch cost, and context and cache use. Segments drop by priority when the band is narrow: classifier support and the switch cost first, then context and cache, then the activity, then the reason. Hovering reveals a pin per tier, **Turn off**, and a two-row view whose second row shows the plan windows on a Claude plan and the session's cost against no routing; while routing is off, **Turn on** sits on the band itself, so the band carries one routing control at a time. A fresh session that starts with routing off because of its model shows `not a routing tier` on the band until the mode changes. The band yields to surveys. In a subagent's transcript it names the agent's model and who picked it, such as `agent ■ Haiku 5.5 ↓ from Opus 5.5 (Claude Code’s pick)`; without a record of the agent it reads `agent · model chosen at spawn`, or `agent · keeps its own model (subagent routing off)`. A model change between turns raises a toast. A pin, or a step that changes only the effort, such as `medium` to `high` on the defaults, raises none. While the classifier runs, the turn's spinner says `Choosing model`. In Manual mode or when routing is unavailable, the prompt footer carries `routing off` or `routing unavailable`. The **Router** button opens the pane. The pane has five numbered tabs, Now, Routes, Policy, Classifier, and Usage, under one `ROUTER` line with the routing on/off pair and the classifier state. The [user guide](user-guide.md#open-the-router-pane) describes each tab.
41
+ In the terminal, the status band is one line above the prompt: a tier meter, the model, the activity, a short reason, classifier support and the switch cost, and context and cache use. Segments drop by priority when the band is narrow: classifier support and the switch cost first, then context and cache, then the activity, then the reason. Hovering shows, on the same row, a pin per tier, **Turn off**, and a two-row view whose second row shows the plan windows on a Claude plan and the session's cost against no routing; while routing is off, **Turn on** sits on the band itself, so the band carries one routing control at a time. A fresh session that starts with routing off because of its model shows `not a routing tier` on the band until the mode changes. The band yields to surveys. While subagents that the router routed are running, a row under the band lists them (`agents 2 ▸ ■ Haiku 5.5 Explore · retry usages ●`, at most three and then `+N`, never a teammate); pressing one opens that agent's transcript through `$.ui.view`, which Claude Code allows only as an answer to the person's own input, and a refusal is shown as a toast. The row is read from `$.agent.list()` joined with the router's spawn records, and redraws when the set of running agents changes (`subagentIds` in the view). In an agent's transcript the band starts with `◂ main`, which returns to the main conversation. In a subagent's transcript it names the agent's model and who picked it, such as `agent ■ Haiku 5.5 ↓ from Opus 5.5 (Claude Code’s pick)`; without a record of the agent it reads `agent · model chosen at spawn`, or `agent · keeps its own model (subagent routing off)`. A model change between turns raises a toast. A pin, or a step that changes only the effort, such as `medium` to `high` on the defaults, raises none. While the classifier runs, the turn's spinner says `Choosing model`. In Manual mode or when routing is unavailable, the prompt footer carries `routing off` or `routing unavailable`. The **Router** button opens the pane. In the terminal the pane has five numbered tabs, Now, Routes, Policy, Classifier, and Usage, under one `ROUTER` line with the routing on/off pair and the classifier state. The desktop app, VS Code and mobile draw a fixed one-row band, with nothing revealed on hover, and tabs without digits; see [the user guide](user-guide.md#in-the-desktop-app-vs-code-and-mobile). The [user guide](user-guide.md#open-the-router-pane) describes each tab.
35
42
 
36
43
  | Reading | Meaning and limits |
37
44
  | ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
@@ -84,6 +91,7 @@ The pane refuses to write through a symlink. A failed validation names the setti
84
91
  - Three launched classifier failures open a 60-second pause for that classifier. One in-flight request is allowed per classifier in each active Mod instance. Pending host HTTP work blocks another request until it settles.
85
92
  - A `router.json` with v0.8 gateway keys or the 1.1 `jev` section marks the router unavailable until the [migration command](configuration.md#convert-an-older-routerjson) converts it.
86
93
  - Unsupported Claude Code versions and leftover v0.8 gateway settings (the `jev-router` model or the `127.0.0.1:43170` base URL) mark the router unavailable. `/router` lists the settings to remove. A local gateway does not run as part of this Mod.
94
+ - A session that starts with the router unavailable shows one toast with the reason (`$.ui.toast`, not `$.ui.notify`, which is a native terminal notification). It repeats at most once a day per cause: the time is kept in `$.store` as `notice:unavailable:<version|gateway|config>`. A `claude -p` run shows none.
87
95
  - While the router is unavailable, the pane header reads `ROUTER unavailable Claude’s model is kept` with the reason, and offers no on/off pair or pins. `/router` without a surface starts with `Routing unavailable: <reason>. Claude’s model is kept.`, and `/router auto`, `off`, `pin`, and `activities` reply the same and change nothing.
88
96
  - A context-window error makes that model ineligible until the next history reset. The Mod does not retry an Anthropic request after a stream begins.
89
97
  - Unknown context retains the current model. There is no characters-to-tokens fallback.
package/docs/providers.md CHANGED
@@ -9,6 +9,34 @@ Custom profiles need their own model settings. They do not inherit Claude models
9
9
 
10
10
  This guide assumes Claude Code already works with your provider. Endpoint and credential setup are outside its scope.
11
11
 
12
+ ## Capability evidence
13
+
14
+ The Anthropic preset seeds `claude-haiku-5-5@medium` at 0, `claude-haiku-5-5@high` at 1,
15
+ `claude-opus-5-5@medium` at 2, and `claude-opus-5-5@xhigh` at 3. These are user-derived policy judgments
16
+ from the built-in balanced base routes, not model-card benchmarks or quality guarantees. All other pairs, including
17
+ Haiku at low and Opus at high, remain unknown, not zero. User-router PR #79 seed values are profile overrides layered
18
+ over these built-ins. Other built-in provider presets have no seeds. No rating is inferred from price, ordinal strength, benchmark rank,
19
+ or an effort name.
20
+ This metadata does not affect the current table router. See [configuration](configuration.md#model-capability-declarations)
21
+ for declarations and overrides.
22
+
23
+ The model-card IDs below identify the reviewed records, not verified served backends. The sources do not give
24
+ numeric values on this router's demand scale.
25
+
26
+ | Preset model / model-card ID | Evidence and gap |
27
+ | --- | --- |
28
+ | Fable 5.2 / `anthropic-claude-fable-5-2` | [Prompting guide](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-fable-5-2): effort guidance, not demand-tier calibration. |
29
+ | Opus 5.5 / `anthropic-claude-opus-5-5` | [Prompting guide](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-opus-5-5): effort recommendations, not tier adequacy. |
30
+ | Sonnet 5.5 / `anthropic-claude-sonnet-5-5` | [Prompting guide](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-sonnet-5-5): well-specified agentic coding at medium does not establish all medium-tier work. |
31
+ | Haiku 5.5 / `anthropic-claude-haiku-5-5` | [Prompting guide](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-haiku-5-5): simple work at low does not calibrate reliability to micro demand. |
32
+ | DeepSeek Flash / `deepseek-deepseek-v4.1-flash-1017` | [Provider evaluation](evaluation.md#provider-profiles): earlier benchmark results do not establish the hosted snapshot's per-effort adequacy. |
33
+ | GLM-5.3 / `zai-glm-5.3`; GLM-5.3-Flash / `zai-glm-5.3-flash` | [Provider evaluation](evaluation.md#provider-profiles): max-effort results do not establish low/high performance or tier adequacy. |
34
+ | MiniMax M3.1 Flash / `minimax-minimax-m3.1-flash`; Kimi K3.1 Pro / `moonshot-kimi-k3.1-pro` | [Provider experiments](provider-experiments.md): small synthetic checks do not calibrate repository tasks per effort. |
35
+ | MiMo V2.6 Flash and Pro / no model-card record | [Provider experiments](provider-experiments.md): no granular effort setting; small synthetic checks do not establish demand-tier adequacy. |
36
+
37
+ The auto engine selects the cheapest eligible unknown when no eligible declaration is known. Neither the seeds nor
38
+ unknown-only selections establish quality or escalation-headroom calibration; users can configure their own maps.
39
+
12
40
  ## Select a provider
13
41
 
14
42
  For DeepSeek, create `deepseek-router.json` in your Claude Code profile directory with this content: