@alexeiled/claude-router 0.8.0 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/.claude-plugin/plugin.json +11 -7
  2. package/CHANGELOG.md +43 -0
  3. package/README.md +55 -98
  4. package/docs/architecture.md +55 -419
  5. package/docs/configuration.md +88 -182
  6. package/docs/evaluation.md +62 -81
  7. package/docs/native-router.md +80 -0
  8. package/docs/router-band.svg +119 -0
  9. package/docs/router-pane-now.svg +85 -0
  10. package/docs/router-pane-tiers.svg +62 -0
  11. package/docs/user-guide.md +127 -172
  12. package/hooks/hooks.json +1 -26
  13. package/hooks/native-router.mjs +701 -0
  14. package/lib/config.mjs +97 -63
  15. package/lib/{facts.mjs → facts-pure.mjs} +2 -23
  16. package/lib/{jev.mjs → jev-contract.mjs} +17 -38
  17. package/lib/native-band.mjs +206 -0
  18. package/lib/native-cost.mjs +88 -0
  19. package/lib/native-display.mjs +122 -0
  20. package/lib/native-jev.mjs +125 -0
  21. package/lib/native-panel.mjs +492 -0
  22. package/lib/native-router.mjs +221 -0
  23. package/lib/policy.mjs +17 -21
  24. package/package.json +10 -8
  25. package/scripts/migrate-config.mjs +57 -0
  26. package/types/index.d.ts +112 -0
  27. package/lib/cost.mjs +0 -93
  28. package/lib/gateway.mjs +0 -221
  29. package/lib/idle.mjs +0 -41
  30. package/lib/rewrite.mjs +0 -130
  31. package/lib/router.mjs +0 -276
  32. package/lib/runtime.mjs +0 -78
  33. package/lib/sse.mjs +0 -57
  34. package/lib/status.mjs +0 -152
  35. package/lib/store.mjs +0 -76
  36. package/scripts/ensure-gateway.mjs +0 -143
  37. package/scripts/gateway.mjs +0 -81
  38. package/scripts/git-hooks/pre-commit +0 -41
  39. package/scripts/git-hooks/pre-push +0 -34
  40. package/scripts/status.mjs +0 -20
  41. package/scripts/statusline.mjs +0 -45
  42. package/scripts/transcript-models.sh +0 -4
  43. package/skills/high/SKILL.md +0 -9
  44. package/skills/low/SKILL.md +0 -7
  45. package/skills/medium/SKILL.md +0 -8
  46. package/skills/micro/SKILL.md +0 -7
  47. package/skills/setup/SKILL.md +0 -31
  48. package/skills/status/SKILL.md +0 -12
@@ -1,203 +1,109 @@
1
1
  # Configuration
2
2
 
3
- ## Where the settings are
4
-
5
- | Setting | Location | Written by |
6
- | -------------------- | --------------------------------------------------------- | --------------------------------------- |
7
- | Jev API key | macOS Keychain, plugin option `typesafe_api_key` | Claude Code, when you enable the plugin |
8
- | Claude Code settings | `~/.claude/settings.json` | `/router:setup` |
9
- | Router configuration | `~/.claude/router.json` | You |
10
- | Data and logs | `~/.claude/plugins/data/router-alexei-led-claude-router/` | The gateway |
11
-
12
- - The hooks give the key to the gateway as `TYPESAFE_API_KEY`. The key is
13
- never in a file.
14
- - The gateway that the hooks start reads `~/.claude/router.json`, or the file
15
- that `ROUTER_CONFIG` names when it is under `~/.claude/`. A project can set
16
- environment variables for the plugin hooks, so the hooks take your home
17
- directory from your OS user account, not from `$HOME`, and ignore
18
- `ROUTER_CONFIG` for a path outside `~/.claude/`. So a cloned repository
19
- cannot choose the router configuration (and with it the port, the routes or
20
- the Jev endpoint) through `router.json`, `ROUTER_CONFIG` or `$HOME`. The
21
- gateway still inherits the other environment variables of the session that
22
- started it.
23
- - The gateway reads `router.json` at start. After a change, run
24
- `pkill -f scripts/gateway.mjs`. The next prompt starts a new gateway.
25
-
26
- ## Claude Code settings
27
-
28
- `/router:setup` writes these keys to `~/.claude/settings.json`:
3
+ The router works with its built-in defaults. Configure a Jev key to enable advice. Change `router.json` only when you need to change routes, model prices, cache assumptions, or policy.
4
+
5
+ ## Key and profile settings
6
+
7
+ | Item | Where it lives | How to change it |
8
+ | ---------------------------------- | ----------------------------------------------- | ------------------------------------------------------------------------------ |
9
+ | Jev API key | Sensitive plugin option `typesafe_api_key` | Run `/plugin configure router`, or use **Set Jev key** in the router pane. |
10
+ | Router settings | `router.json` in the active Claude Code profile | Edit the file directly, or use the pane's Tiers and Tuning tabs. |
11
+ | Anthropic credentials and API cost | Claude Code | The Mod leaves these to Claude Code. |
12
+
13
+ The active profile directory is `CLAUDE_CONFIG_DIR` when set. Otherwise it is `~/.claude`. The Mod does not need an Anthropic proxy URL, model alias, hint header, daemon, or custom status line.
14
+
15
+ The Mod reads the key from the sensitive plugin option. It also accepts `TYPESAFE_API_KEY` from the process environment. Without a key, it keeps Claude's current model and reports degraded routing.
16
+
17
+ Project and local settings cannot set `HOME` or `CLAUDE_CONFIG_DIR` for this Mod. Those overrides make the router unavailable for the session, so a project cannot redirect the router to another profile's key or configuration.
18
+
19
+ ## Built-in defaults
20
+
21
+ The default baseline is tier `low`. Each route refers to an alias in `models`.
22
+
23
+ | Tier | Model alias | Model ID | Effort |
24
+ | -------- | ----------- | ------------------- | ------------------------------------ |
25
+ | `micro` | `haiku` | `claude-haiku-4-5` | None |
26
+ | `low` | `sonnet` | `claude-sonnet-5-5` | Keeps the effort sent by Claude Code |
27
+ | `medium` | `opus` | `claude-opus-5-5` | `medium` |
28
+ | `high` | `opus` | `claude-opus-5-5` | `xhigh` |
29
+
30
+ Default model settings:
31
+
32
+ | Alias | Input / output / cache read, USD per million tokens | Context window | Billing | Effort levels |
33
+ | -------- | --------------------------------------------------- | -------------- | ------- | --------------------------------------- |
34
+ | `opus` | 4 / 20 / 0.2 | 1,000,000 | `plan` | `low`, `medium`, `high`, `xhigh`, `max` |
35
+ | `sonnet` | 2 / 10 / 0.2 | 1,000,000 | `plan` | `low`, `medium`, `high`, `xhigh`, `max` |
36
+ | `haiku` | 1 / 5 / 0.1 | 200,000 | `plan` | none |
37
+
38
+ Prices are configured list-price inputs for estimates. They are not a subscription bill or a claim of savings. Haiku receives no effort field. A route without an effort keeps the session effort, clamped to what the model supports. To make a tier whose default names an effort keep the session effort instead, set `"effort": null`, for example `"routes": { "high": { "model": "opus", "effort": null } }`.
39
+
40
+ ## Optional `router.json`
41
+
42
+ Start with the setting you need and leave the rest at defaults. For example, require three votes before moving to a cheaper tier and allow Jev up to two seconds:
29
43
 
30
44
  ```json
31
45
  {
32
- "model": "jev-router[1m]",
33
- "env": {
34
- "ANTHROPIC_BASE_URL": "http://127.0.0.1:43170",
35
- "ENABLE_TOOL_SEARCH": "true",
36
- "CLAUDE_CODE_GATEWAY_HINT_HEADERS": "1"
37
- },
38
- "modelPicker": {
39
- "options": [
40
- {
41
- "model": "jev-router[1m]",
42
- "label": "Jev Router (auto)",
43
- "description": "Auto-selects the model and effort for each turn",
44
- "behavesAs": "claude-opus-5-5"
45
- }
46
- ]
47
- }
46
+ "jev": { "timeoutMs": 2000 },
47
+ "policy": { "downgradeVotes": 3 }
48
48
  }
49
49
  ```
50
50
 
51
- | Key | Why |
52
- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
53
- | `model` | The `[1m]` suffix tells Claude Code that the window is 1M tokens. Claude Code removes the suffix before the request. |
54
- | `env.ANTHROPIC_BASE_URL` | Sends the traffic of Claude Code to the gateway. |
55
- | `env.ENABLE_TOOL_SEARCH` | Keeps MCP tool schemas deferred. Without it, each request carries all schemas: about 50K tokens with the claude.ai connectors. |
56
- | `env.CLAUDE_CODE_GATEWAY_HINT_HEADERS` | Claude Code sends the [class of each request](architecture.md#request-classes). |
57
- | `modelPicker` row | Adds `Jev Router (auto)` to `/model`. `behavesAs` names a known model. Without it, Claude Code rejects the alias. |
58
- | `statusLine` (optional) | Runs your status line command, then adds the route of the last turn. |
51
+ The Mod validates the whole file. Unknown keys and invalid values make routing unavailable with a general configuration error. Retired gateway keys trigger a migration instruction. The migration command reports an invalid setting path and does not print its value.
59
52
 
60
- Setup writes the file once, as its last step. Restart Claude Code after it.
61
- Until the restart, the session shows "There's an issue with the selected
62
- model (jev-router[1m])".
53
+ | Section | Supported fields | Defaults |
54
+ | ---------------- | ------------------------------------------------------------------------------------ | --------------------------------------------------------------- |
55
+ | `baselineTier` | `micro`, `low`, `medium`, `high` | `low` |
56
+ | `routes.<tier>` | `model`, optional `effort` (a level, or `null` for the session effort) | As in the route table above |
57
+ | `models.<alias>` | `id`, `input`, optional `output`, `cacheRead`, `contextWindow`, `billing`, `efforts` | Three defaults above |
58
+ | `cache` | `writeMultiplier`, `ttlMs`, `warmMarginMs` | `5m: 1.25`, `1h: 2`; `300000` / `3600000` ms; `30000` ms |
59
+ | `policy` | Fields below | Values below |
60
+ | `jev` | `endpoint`, `model`, `timeoutMs` | `https://api.typesafe.ai/v1/systemone`, `jev-1.13.0`, `1500` ms |
61
+ | `context` | `recentTurns`, `maxTextChars` | `6`, `1200` |
63
62
 
64
- ## Router configuration
63
+ Policy defaults:
65
64
 
66
- Each key in `~/.claude/router.json` is optional. A key replaces the default
67
- at the same path, and objects merge.
65
+ | Field | Default | Meaning |
66
+ | ----------------------- | ------: | -------------------------------------------------------------------------------------------------- |
67
+ | `upgradeVotes` | `2` | Consecutive supporting votes before an upgrade. |
68
+ | `upgradeBase` | `0.75` | Minimum probability mass required for an upgrade. |
69
+ | `upgradeSlope` | `0.15` | Maximum increase to the upgrade bar from estimated switching cost. |
70
+ | `upgradePivotUsd` | `0.5` | Cost scale used by the upgrade bar. Must be positive. |
71
+ | `jumpConfidence` | `0.95` | Support for a two-tier jump without waiting for votes. |
72
+ | `downgradeVotes` | `2` | Consecutive supporting votes before a downgrade. |
73
+ | `downgradeMass` | `0.9` | Minimum probability mass required for a downgrade. |
74
+ | `downgradeSlope` | `0.08` | Maximum increase to the downgrade bar from estimated switching cost. |
75
+ | `downgradePivotUsd` | `0.5` | Cost scale used by the downgrade bar. Must be positive. |
76
+ | `downgradeHorizonTurns` | `5` | Later turns included in a downgrade payback estimate. |
77
+ | `continuationMass` | `0.7` | Advice probability that keeps the current route for a continuation. |
78
+ | `escalationHoldTurns` | `2` | Turns held after a repeated tool error escalates the route. |
79
+ | `cashCapUsd` | `2` | Maximum estimated cold cache write for a `credits` model. It does not cap output or session spend. |
68
80
 
69
- The file is strict. Text that is not valid JSON, an unknown key (a typo such
70
- as `routs`), a value of the wrong type, or a number out of range stops a new
71
- gateway from starting. The error names the file and the field, never the
72
- value. A gateway that already runs keeps serving. When no gateway runs, every
73
- request fails to connect until you fix the file. The SessionStart hook prints
74
- the error; the prompt hook is quiet. `/router:status` also shows it: it reads
75
- the file and starts nothing.
81
+ The pane writes a subset of these settings. The **Tiers** tab sets `routes` and `baselineTier`: it writes only tiers that differ from the defaults and removes the rest. The **Tuning** tab sets `jev.timeoutMs`, `policy.downgradeVotes`, `policy.downgradeHorizonTurns`, and `policy.cashCapUsd`. Each save validates the whole file, keeps other keys, and applies from the next turn. A failed check names the setting and leaves the file unchanged. The pane refuses to write through a symlink. Model aliases and the other settings require a direct edit.
76
82
 
77
- For example, to run `low` on Sonnet at `high` effort and give Jev more time:
83
+ Native cache freshness is unknown after 270 seconds or after a history reset. Claude usage does not report the cache TTL. The policy evaluates price bounds for five-minute and one-hour writes, but the one-hour case is not observed fact. See the [architecture](architecture.md#cache-and-cost) for the estimate rules.
78
84
 
79
- ```json
80
- {
81
- "routes": { "low": { "model": "sonnet", "effort": "high" } },
82
- "jev": { "timeoutMs": 2500 }
83
- }
85
+ ## Convert a v0.8 configuration
86
+
87
+ The `router.json` converter changes only that file. It does not remove Claude Code gateway settings. Remove them before the first native session.
88
+
89
+ A marketplace install keeps the plugin ID `router@alexei-led-claude-router`: update it and keep it enabled. If you load v1 from a checkout with `--plugin-dir` instead, disable the marketplace install so only one router loads. Set `CLAUDE_CONFIG_DIR` to the intended profile directory first:
90
+
91
+ ```sh
92
+ CLAUDE_CONFIG_DIR=/path/to/profile claude plugin disable router@alexei-led-claude-router --scope user
84
93
  ```
85
94
 
86
- ### gateway
87
-
88
- | Key | Default | Meaning |
89
- | ---------------- | ------------ | ------------------------------------------------------------------------- |
90
- | `port` | `43170` | The loopback port of the gateway. |
91
- | `alias` | `jev-router` | The model name that Claude Code sends. |
92
- | `baselineTier` | `low` | The tier when nothing else decides: a new session, or no Jev answer. |
93
- | `auxiliaryTier` | `low` | The tier for side requests, for example session titles. |
94
- | `idleShutdownMs` | `7200000` | The gateway exits after this time without requests. `0` keeps it running. |
95
-
96
- ### routes
97
-
98
- | Tier | Default |
99
- | -------- | ------------------------------------------ |
100
- | `high` | `{ "model": "opus", "effort": "xhigh" }` |
101
- | `medium` | `{ "model": "sonnet", "effort": "xhigh" }` |
102
- | `low` | `{ "model": "sonnet" }` |
103
- | `micro` | `{ "model": "haiku" }` |
104
-
105
- `model` is a key of `models`. `effort` is `low`, `medium`, `high`, `xhigh` or
106
- `max`. Without `effort`, the gateway keeps the effort that Claude Code sent.
107
- If you change a route, also change the frontmatter of `skills/<tier>/SKILL.md`.
108
- A test makes sure that they agree.
109
-
110
- ### models
111
-
112
- | Alias | `id` | `input` | `output` | `cacheRead` | `contextWindow` | `maxOutput` | `billing` | `efforts` | `features` |
113
- | -------- | ------------------- | ------- | -------- | ----------- | --------------- | ----------- | --------- | --------- | ---------- |
114
- | `opus` | `claude-opus-5-5` | 4 | 20 | 0.2 | 1,000,000 | — | `plan` | all five | all three |
115
- | `sonnet` | `claude-sonnet-5-5` | 2 | 10 | 0.2 | 1,000,000 | — | `plan` | all five | all three |
116
- | `haiku` | `claude-haiku-4-5` | 1 | 5 | 0.1 | 200,000 | 64,000 | `plan` | none | none |
117
-
118
- | Field | Meaning |
119
- | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
120
- | `id` | The model id that the gateway sends to Anthropic. |
121
- | `input`, `output`, `cacheRead` | List prices in USD per million tokens. `output` is optional. The `shadow` estimate and the downgrade tax use it. |
122
- | `contextWindow` | The window in tokens. A turn goes only to a model that holds the context at 80% fill. |
123
- | `maxOutput` | Optional limit for `max_tokens`. Haiku 4.5 rejects more than 64K. |
124
- | `billing` | `plan` for the subscription limits. `credits` for models that bill usage credits. |
125
- | `efforts` | The effort levels that the model accepts. An empty list removes effort and thinking. |
126
- | `features` | The request features of Claude Code that the model accepts: `mid-conversation-system`, `per-turn-control`, `mid-conversation-tool-changes`. The gateway removes the others ([architecture](architecture.md#system-context)). Without the field: none. |
127
-
128
- The default prices match `test/fixtures/list-prices.json`, which names its
129
- source and date. The default `efforts` match `test/fixtures/effort-support.json`,
130
- from a probe against the real API. A test fails when they differ.
131
-
132
- ### policy
133
-
134
- The [switching policy](architecture.md#switching-policy) uses these values.
135
-
136
- | Key | Default | Meaning |
137
- | ----------------------- | ------- | ----------------------------------------------------------------------------------------------------- |
138
- | `upgradeVotes` | `2` | Consecutive votes above the current tier before an upgrade. |
139
- | `upgradeBase` | `0.75` | The upgrade bar when a switch costs nothing. |
140
- | `upgradeSlope` | `0.15` | How much a switching cost can raise the bar: `base + slope × tax / (tax + pivot)`. |
141
- | `upgradePivotUsd` | `0.5` | The switching cost that adds half of the slope. |
142
- | `jumpConfidence` | `0.95` | The confidence for a jump of two tiers without the vote delay. |
143
- | `downgradeVotes` | `2` | Consecutive votes for a lower tier before a downgrade. |
144
- | `downgradeMass` | `0.9` | The downgrade bar when the candidate's cache is already warm. |
145
- | `downgradeSlope` | `0.08` | How much a cold candidate's cache write can raise the bar, same formula as upgrade. |
146
- | `downgradePivotUsd` | `0.5` | The switching cost that adds half of the slope. |
147
- | `downgradeHorizonTurns` | `5` | The turns of output and read savings that a downgrade's tax nets. Input only without `output` prices. |
148
- | `continuationMass` | `0.7` | The Jev probability for "continues the task" that keeps the route. |
149
- | `escalationHoldTurns` | `2` | Turns that the route stays up after an escalation. |
150
- | `cashCapUsd` | `2` | The cold-write guard: the largest first cache write for a switch to a `credits` model. |
151
-
152
- ### cache, jev, context, log
153
-
154
- | Key | Default | Meaning |
155
- | ----------------------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------- |
156
- | `cache.writeMultiplier` | `{ "5m": 1.25, "1h": 2 }` | The price of a cache write, as a multiple of `input`. |
157
- | `cache.ttlMs` | `{ "5m": 300000, "1h": 3600000 }` | The lifetime of each cache TTL. |
158
- | `cache.warmMarginMs` | `30000` | A cache counts as cold this long before it expires. |
159
- | `jev.endpoint` | `https://api.typesafe.ai/v1/systemone` | The Jev API. |
160
- | `jev.model` | `jev-1.13.0` | The Jev model. |
161
- | `jev.timeoutMs` | `1500` | The total time for one Jev answer, one retry included. |
162
- | `context.recentTurns` | `6` | The number of recent turns that Jev receives. |
163
- | `context.maxTextChars` | `1200` | The characters of the prompt and of each turn that Jev receives. A longer text keeps its start and end. |
164
- | `log` | `true` | Write `decisions.jsonl`. |
165
-
166
- ## Environment variables
167
-
168
- | Variable | Effect |
169
- | -------------------- | ------------------------------------------------------------------------------------------------------------ |
170
- | `TYPESAFE_API_KEY` | The Jev key. The hooks set it from the plugin option. |
171
- | `ROUTER_CONFIG` | Another path for the router configuration, under `~/.claude/` only. Another path is ignored, with a warning. |
172
- | `CLAUDE_PLUGIN_DATA` | The data directory. Claude Code sets it for the plugin hooks. |
173
-
174
- ## Command-line flags
175
-
176
- A project cannot change these: they are part of the command, not of the
177
- environment.
178
-
179
- | Flag | Script | Effect |
180
- | --------------------- | ------------------------------------------------- | ------------------------------------------------------------------------ |
181
- | `--config <path>` | `gateway.mjs`, `ensure-gateway.mjs`, `status.mjs` | The router configuration, at any path. |
182
- | `--force-tier <tier>` | `gateway.mjs` | `micro`, `low`, `medium` or `high`. Skips Jev and the policy. For tests. |
183
-
184
- The flags are for a gateway that you start by hand. The hooks and the status
185
- line do not pass them. To use another file with the hooks, keep it under
186
- `~/.claude/` and set `ROUTER_CONFIG`.
187
-
188
- To force a tier, stop the gateway and start it by hand with the data directory
189
- of the plugin, so the log stays in its place:
95
+ Then inspect that profile's `settings.json`. Remove the old `jev-router[1m]` model-picker row and any `model` or `env.ANTHROPIC_MODEL` value set to that alias. Remove `env.ANTHROPIC_BASE_URL` when it points to the router's loopback port. Clear `env.CLAUDE_CODE_GATEWAY_HINT_HEADERS`. Restore or remove a router-specific `statusLine` that points to the deleted script. Remove `env.ENABLE_TOOL_SEARCH` only if the old router setup added it and you do not need it for another reason. Preserve unrelated settings. Set the normal full baseline model, for example `claude-sonnet-5-5`.
96
+
97
+ No conversion is needed if you never created `router.json`. For an existing file, run the migration command that ships with the installed plugin:
190
98
 
191
99
  ```sh
192
- pkill -f scripts/gateway.mjs
193
- CLAUDE_PLUGIN_DATA=~/.claude/plugins/data/router-alexei-led-claude-router \
194
- node <plugin>/scripts/gateway.mjs --force-tier high
100
+ node ~/.claude/plugins/cache/alexei-led-claude-router/router/1.0.0/scripts/migrate-config.mjs ~/.claude/router.json
195
101
  ```
196
102
 
197
- ## Data directory
103
+ Use the installed version in the path. From a checkout, run `node scripts/migrate-config.mjs ~/.claude/router.json`.
104
+
105
+ Pass the file under your active profile instead when `CLAUDE_CONFIG_DIR` points elsewhere. The command validates the converted configuration before writing it and creates an exact-byte `router.json.v0.8.backup`. It refuses to overwrite an existing backup. Route, model-price, cache, and policy overrides remain in the converted file.
106
+
107
+ The converter removes old gateway and logging settings, request-feature fields, and model output caps. It moves `gateway.baselineTier` to top-level `baselineTier`. It stops without writing when it finds an unsupported old field or invalid setting.
198
108
 
199
- | File | Content | Retention |
200
- | ----------------- | --------------------------------------------- | -------------------------------------------- |
201
- | `decisions.jsonl` | One line for each decision and each response. | Moves to `.1` above 20 MB. |
202
- | `gateway.log` | Gateway events with timestamps. | Moves to `.1` above 20 MB, at gateway start. |
203
- | `sessions/` | The routing memory of each session. | Removed after 30 days without use. |
109
+ Node.js 22 or later is needed only for this migration command and project development. Claude Code runs the Mod. Node is not a routing service.
@@ -1,83 +1,64 @@
1
1
  # Evaluation
2
2
 
3
- A snapshot of real work by one developer on one machine. The router ran in
4
- Claude Code (claude-router) and in Pi (pi-model-router). Both use Jev.
5
-
6
- ![jev-router in Claude Code: where the requests went, and what the work cost](tier-share.svg)
7
-
8
- ## Claude Code
9
-
10
- The window starts at the first request on the current models (Opus 5.5,
11
- Sonnet 5, Haiku 4.5): 2026-09-23 09:05 UTC. It ends at 2026-09-24 06:00 UTC.
12
- The data has 26 sessions and 1,611 requests. Jev advised 111 prompts.
13
-
14
- | Tier | Model and effort | Requests | Share |
15
- | -------- | ------------------ | -------- | ----- |
16
- | `micro` | Haiku 4.5 | 154 | 9.6% |
17
- | `low` | Sonnet 5 | 1,239 | 76.9% |
18
- | `medium` | Opus 5.5, `high` | 73 | 4.5% |
19
- | `high` | Opus 5.5, `xhigh` | 145 | 9.0% |
20
-
21
- The cost of the same work at list prices:
22
-
23
- | Scenario | Cost | Saving with the router |
24
- | ------------------------- | ------- | ---------------------- |
25
- | Always Opus at `xhigh` | $166.26 | 15.5% |
26
- | Best model per session | $146.54 | 4.1% |
27
- | jev-router | $140.55 | — |
28
-
29
- "Best model per session" is a user who knows in advance the strongest tier
30
- that Jev selected in each session, and uses it for the full session. The
31
- result for each group of sessions:
32
-
33
- | Strongest tier in the session | Sessions | Requests | Router | Baseline | Saving |
34
- | ----------------------------- | -------- | -------- | ------- | -------- | ------ |
35
- | `low` (Sonnet) | 23 | 573 | $39.74 | $39.74 | 0% |
36
- | `medium` (Opus, `high`) | 2 | 847 | $76.89 | $84.57 | 9.1% |
37
- | `high` (Opus, `xhigh`) | 1 | 191 | $23.92 | $22.23 | −7.6% |
38
-
39
- ## Output size
40
-
41
- The baselines keep the output tokens of the router. A stronger model at a
42
- higher effort writes more, so the real savings are larger. If the stronger
43
- route writes more output on the requests that the router sent lower, the
44
- savings are:
45
-
46
- | Output of the stronger route | vs. always Opus at `xhigh` | vs. best model per session |
47
- | ---------------------------- | -------------------------- | -------------------------- |
48
- | Same (1×) | 15.5% | 4.1% |
49
- | 1.5× | 21.1% | 7.6% |
50
- | 2× | 26.0% | 10.8% |
51
-
52
- ## Pi
53
-
54
- Pi used two OpenAI profiles from 2026-09-22 21:00 to 2026-09-24 06:00 UTC,
55
- with 25 sessions and 1,964 requests. Jev advised 148 decisions. The profiles
56
- put correctness first: `high` served 83.6% of the requests and `medium` served
57
- 16.4%. The router cost $347.90. The best model per session cost $348.37. With
58
- this policy, routing is cost-neutral.
59
-
60
- ## Method
61
-
62
- - **Data.** Claude Code: `decisions.jsonl`, one `observed` line with the
63
- usage of each response. Pi: the session files, a router decision and the
64
- usage of each message. Neither source has prompt text.
65
- - **Prices.** List prices in USD per million tokens. Claude: the defaults in
66
- `lib/config.mjs`. Pi: the Pi model registry, with long-context prices. For
67
- subscription plans, the dollars show the relative cost only.
68
- - **Cache writes.** In Claude Code, the input that is not a cache read counts
69
- as a cache write at the granted TTL: 1.25× input for 5 minutes, 2× for 1
70
- hour.
71
- - **Baselines.** Each baseline uses the tokens and the cache reads of the
72
- router. After each change of route in a conversation, the baseline also
73
- reads the previous context from the cache, if the cache is still warm. Thus
74
- the baseline does not pay for the cache writes that routing causes.
75
- - **Tiers with one model.** Opus at `high` and at `xhigh` have the same price.
76
- Subagents count toward the strongest tier of their session.
77
-
78
- ## Limits
79
-
80
- - One developer and two days make a snapshot, not a benchmark.
81
- - Answer quality was not measured.
82
- - For Pi, the baseline assumes that the OpenAI cache stays warm for 10
83
- minutes.
3
+ The native Mod has no measured savings result. The pane reports Claude's own usage and configured-price scenarios. It does not maintain a counterfactual bill or a cumulative savings total.
4
+
5
+ This page separates an older gateway experiment from a shadow replay of native policy. Neither result measures answer quality or proves lower real spend.
6
+
7
+ ## Historical gateway experiment
8
+
9
+ These figures are retained from the earlier gateway implementation. They do not evaluate the native Mod or its current routes. The snapshot covers 26 Claude Code sessions and 1,611 requests from 2026-09-23 09:05 UTC through 2026-09-24 06:00 UTC. Jev advised on 111 prompts.
10
+
11
+ ![Historical gateway model share and configured-price costs](tier-share.svg)
12
+
13
+ The original list-price comparison was:
14
+
15
+ | Scenario | Estimated cost | Difference from router |
16
+ | ----------------------------------------------- | -------------: | ---------------------: |
17
+ | Always Opus at `xhigh` | $166.26 | Router was 15.5% lower |
18
+ | Strongest model selected for each whole session | $146.54 | Router was 4.1% lower |
19
+ | Earlier gateway router | $140.55 | — |
20
+
21
+ The session-level baseline assumes a user knows the strongest tier needed by each session and uses it for every request. All scenarios reuse the router's token and output counts. The estimate holds stronger-model output constant, so it does not capture any change in output length. The [original evaluation method](#historical-method-and-limits) lists the limits.
22
+
23
+ This is one developer's short trace at configured list prices. It does not include a quality review. Do not use these values as expected results for the native Mod.
24
+
25
+ ## Native shadow replay
26
+
27
+ The native replay uses an aggregate Team-session trace from 2026-09-23 through 2026-10-04. The checked-in result is [`trace-evaluation.json`](../experiments/mod-router/results/trace-evaluation.json), produced by [`trace-eval.mjs`](../experiments/mod-router/scripts/trace-eval.mjs).
28
+
29
+ | Coverage item | Result |
30
+ | ----------------------------------------------------------- | --------------------------------: |
31
+ | Team rows | 24,408 across 112 sessions |
32
+ | Main-conversation decision rows | 11,619 |
33
+ | Main decisions with Jev advice | 37 |
34
+ | Advised decisions replayed at both cache-prefix bounds | 37 of 37 |
35
+ | Decisions with a different route or reason between policies | 0 of 37 |
36
+ | Differing switches and estimated switch cost | 0; no switch cost can be computed |
37
+
38
+ The legacy replay matched the logged tier and reason for 1,704 of 1,704 replayable decisions. On the 37 advised main decisions, both policies produced the same route and reason. This replay therefore shows no native routing difference and cannot support a savings claim.
39
+
40
+ Eight advised decisions reached a switching-cost threshold calculation. The native configured-price tax ranged from $0.028660 to $3.471293 and the threshold from 0.860483 to 0.953675. These are shadow estimates for individual next requests, not bills or additive savings.
41
+
42
+ The old log records cache reads but not uncached input or cache-creation tokens. The native replay tests two prefix bounds: cache reads alone and the full observed input. All 37 decisions matched at both bounds. That is a bound-stability check, not an exact reconstruction of native cache state.
43
+
44
+ ### Reproduce the native replay
45
+
46
+ Run this command with the Team `decisions.jsonl` log available in its usual Claude Code profile:
47
+
48
+ ```sh
49
+ node experiments/mod-router/scripts/trace-eval.mjs
50
+ node --test test/trace-eval.test.mjs
51
+ ```
52
+
53
+ The script writes aggregate counts, reason histograms, route transitions, and shadow estimates. It excludes prompts, session identifiers, agent names, error text, headers, keys, and filesystem paths from the result. It reads the local profile log and writes the JSON result file.
54
+
55
+ ## Historical method and limits
56
+
57
+ - Costs use configured USD list prices per million tokens. They are not subscription cash charges.
58
+ - The older gateway baseline uses the router's observed tokens and cache reads. After a route change, the baseline assumes the prior context can be read from cache.
59
+ - The native replay uses the current default routes and prices, with recorded model IDs substituted. It compares the native cost bounds with a legacy cost model using the same policy state and trace facts.
60
+ - The replay cannot reconstruct exact native cache prefixes or one-hour cache evidence from the old log. The native cache model treats TTL as unknown.
61
+ - The replay applies the legacy state chain to both policy arms. It does not simulate later state changes after a different decision.
62
+ - Failure signatures and pins are not present in the trace. The relevant route decisions are excluded from comparison.
63
+ - Most main rows are tool continuations or have no Jev advice. Only 37 advised decisions support the native comparison.
64
+ - Costs cover the next request. They use the last observed output size and do not model effort-dependent output length or future answer quality.
@@ -0,0 +1,80 @@
1
+ # Native router: behavior and evidence
2
+
3
+ This guide describes the Mod shipped in the plugin. Claude Code 2.1.289 is the minimum tested version. The router changes `model` and `effort` on main-conversation steps. Claude Code handles the Anthropic request, credentials, stream, tool execution, and cost ledger. Subagent steps keep their original model and effort.
4
+
5
+ ## Turn behavior
6
+
7
+ At the first main step of a logical turn, the Mod reads recent message text and asks Jev for a tier. It applies local policy and saves the route. Tool continuations reuse that route. Before each continuation, context fit is checked again. A large tool result can move the step to a model with a larger window.
8
+
9
+ Jev receives the current prompt and at most six preceding user or assistant text messages. Each item is limited to 1,200 characters. It does not receive system messages, tool inputs, or tool results. Claude Code's Anthropic credentials never enter the Jev request.
10
+
11
+ The default tiers are defined in [Configuration](configuration.md#built-in-defaults). Policy uses vote hysteresis, repeated tool errors, context fit, model availability, and switching-cost estimates. The [architecture](architecture.md#request-flow) explains the event boundary and failure handling.
12
+
13
+ ## Controls and session modes
14
+
15
+ | Control | Behavior |
16
+ | -------------------- | ----------------------------------------------------------------------- |
17
+ | `/router` | Open the pane. Without a UI surface, print short status. |
18
+ | `/router auto` | Enable automatic routing. |
19
+ | `/router off` | Enter Manual mode and preserve Claude's selected model. |
20
+ | `/router pin <tier>` | Pin the next turn and tool continuations. Auto must already be enabled. |
21
+ | `/model <name>` | Select a model and enter Manual mode. `/router auto` resumes routing. |
22
+
23
+ The pane's **Set Jev key** button opens Claude Code's secure plugin configuration. Claude Code refuses `$.command.run` from inside a `command.run` hook, so the Mod starts that dialog from a timer after the press returns.
24
+
25
+ Mode belongs to a Claude Code session. Clear starts a new session in Auto. Resume restores the saved mode for that session. A new or forked session has a new ID: it starts in Auto on the baseline model and in Manual on another model, as a fresh launch does. A pin applies to one logical turn, then the prior Auto incumbent resumes. A history reset clears votes and cache evidence; the current turn keeps its route, pin and any native fallback.
26
+
27
+ ## Router pane
28
+
29
+ The status band is one line above the prompt: a tier meter, the route, a short reason, classifier support, and context and cache use. Segments drop by priority when the band is narrow. Hovering reveals pins, Manual, and a two-row view. The band yields to surveys and does not describe subagent transcripts. A model change between turns raises a toast. A pin, or a step that changes only the effort, such as `medium` to `high` on the defaults, raises none. While the classifier runs, the turn's spinner says `Choosing model`. In Manual mode or when routing is unavailable, the prompt footer carries `router off` or `router unavailable`. The **Router** button opens the pane. The pane has four tabs, Now, Tiers, Tuning, and Usage, with Auto and Manual on every tab. The [user guide](user-guide.md#open-the-router-pane) describes each tab.
30
+
31
+ | Reading | Meaning and limits |
32
+ | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
33
+ | Now / Served | The requested model and effort. **Served** appears only when the engine answered with another model. Actual effort is not reported. |
34
+ | Jev support | Per-tier probabilities from the last classification, and the switch bar the policy required. Pins and skipped calls leave it empty. |
35
+ | Replies | The tier that served each of the last 30 main-conversation replies; a dot where the router did not choose. |
36
+ | Cost | Claude Code's native usage value. A plan's list-price estimates are not subscription cash charges. |
37
+ | Context bar | Maximum available observed and local estimated context, compared with the routed model's window. The fit rule reserves 20%. |
38
+ | Observed / estimated input | Claude's local context reading and the router's best current context estimate. `unknown` blocks a move to a smaller window. |
39
+ | Cache reuse | Last response's cache-read tokens divided by its reported input counters. |
40
+ | Cache read / written; Output | Response usage counters. Cache warmth does not prove a particular TTL. |
41
+ | Input per reply | Up to 30 observed main-conversation input sizes, scaled from the lowest to the highest reading. |
42
+ | Jev / Last classification | Classifier state and elapsed time. Missing-key, busy, and paused skips have no network latency reading. |
43
+ | Next-turn difference | Configured-price range for the suggested tier versus the incumbent on the next request. Negative means estimated lower cost. |
44
+ | Conservative payback | Conditional later-turn estimate based on configured cache prices, future reads, and the last output size. It is not measured savings. |
45
+ | Cache read benefit | Estimated price difference for observed cache reads before cache writes. It is not net savings. |
46
+
47
+ There is no router-owned cost ledger or cumulative savings counter. Jev charges are not included. See [Evaluation](evaluation.md) for what the available trace can support.
48
+
49
+ ## Tuning
50
+
51
+ The Tiers tab edits each tier's model alias and effort and the baseline tier. The Tuning tab has four controls. Drafts do not affect the active turn. **Save routes** and **Save tuning** validate the whole file with the same loader the Mod starts with, write `router.json` for future turns, and keep unrelated keys. A route equal to the built-in default is removed from the file, so later default changes still reach it.
52
+
53
+ | Control | Values in the pane | Effect |
54
+ | ---------------- | --------------------------- | ---------------------------------------------------- |
55
+ | Jev deadline | 500, 1,000, 1,500, 3,000 ms | Total advice time, including any transient retry. |
56
+ | Votes to go down | 1, 2, 3 | Consecutive votes required for a downgrade. |
57
+ | Payback horizon | 1, 3, 5, 10 turns | Later turns included in downgrade economics. |
58
+ | Credits cap | $0.50, $1, $2, $5 | Largest estimated cold write on a `credits` model. |
59
+
60
+ The pane refuses to write through a symlink. A failed validation names the setting and leaves the file unchanged. New model aliases and the other supported fields need an edit to the active profile's `router.json`.
61
+
62
+ ## Safety and failure states
63
+
64
+ - Missing Jev key, a Jev failure, or a policy refusal keeps the current model. Network refusal is never bypassed with a helper process.
65
+ - Three launched Jev failures open a 60-second pause. One in-flight request is allowed per active Mod instance. Pending host HTTP work blocks another request until it settles.
66
+ - Unsupported Claude Code versions and leftover v0.8 gateway settings (the `jev-router` model or the `127.0.0.1:43170` base URL) mark the router unavailable. `/router` lists the settings to remove. A local gateway does not run as part of this Mod.
67
+ - A context-window error makes that model ineligible until the next history reset. The Mod does not retry an Anthropic request after a stream begins.
68
+ - Unknown context retains the current model. There is no characters-to-tokens fallback.
69
+ - Unknown model substitutions receive no cache credit for the requested model.
70
+ - Clear, rewind, branch, or committed compaction clears cache evidence and policy votes. Resume restores the selected session's mode, not its old cache evidence.
71
+
72
+ ## Verified checks
73
+
74
+ The Team canary loaded the published 1.0.0 package with a full Sonnet baseline, reached the real Jev endpoint, and kept Sonnet on a `downgrade-pending` result. This was a routing smoke test, not a measured-savings trial.
75
+
76
+ Recorded acceptance checks cover the local HTTP deadline and one-request admission, session clear/resume, circuit-breaker pause, reload, replacement, unload, engine fallback, and interrupt during Jev advice. The interrupt check closed the loopback socket within 4 ms and discarded late advice without changing the active route or UI. Results are stored under [`experiments/mod-router/results`](../experiments/mod-router/results/).
77
+
78
+ A billed chain of pinned turns moved across Opus, Haiku, and Sonnet with a tool call on every turn, and no request was rejected. Claude Code builds each model's request itself; Haiku receives its own thinking mode and a 32K output cap. Sonnet and Opus accepted histories above 589K input tokens. These probes validate the tested path only. Claude Code and organization policy control other model and request combinations.
79
+
80
+ To stop routing in a session, run `/router off`. To stop loading the Mod, run `claude plugin disable router@alexei-led-claude-router` and restart.