@alexeiled/claude-router 0.8.0 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/.claude-plugin/plugin.json +10 -6
  2. package/CHANGELOG.md +19 -0
  3. package/README.md +51 -100
  4. package/docs/architecture.md +55 -419
  5. package/docs/configuration.md +86 -182
  6. package/docs/evaluation.md +62 -81
  7. package/docs/native-router.md +79 -0
  8. package/docs/user-guide.md +103 -172
  9. package/hooks/hooks.json +1 -26
  10. package/hooks/native-router.mjs +635 -0
  11. package/lib/config.mjs +20 -58
  12. package/lib/{facts.mjs → facts-pure.mjs} +2 -23
  13. package/lib/{jev.mjs → jev-contract.mjs} +17 -38
  14. package/lib/native-cost.mjs +88 -0
  15. package/lib/native-display.mjs +75 -0
  16. package/lib/native-jev.mjs +125 -0
  17. package/lib/native-panel.mjs +150 -0
  18. package/lib/native-router.mjs +216 -0
  19. package/lib/policy.mjs +17 -21
  20. package/package.json +9 -8
  21. package/scripts/migrate-config.mjs +57 -0
  22. package/types/index.d.ts +95 -0
  23. package/lib/cost.mjs +0 -93
  24. package/lib/gateway.mjs +0 -221
  25. package/lib/idle.mjs +0 -41
  26. package/lib/rewrite.mjs +0 -130
  27. package/lib/router.mjs +0 -276
  28. package/lib/runtime.mjs +0 -78
  29. package/lib/sse.mjs +0 -57
  30. package/lib/status.mjs +0 -152
  31. package/lib/store.mjs +0 -76
  32. package/scripts/ensure-gateway.mjs +0 -143
  33. package/scripts/gateway.mjs +0 -81
  34. package/scripts/git-hooks/pre-commit +0 -41
  35. package/scripts/git-hooks/pre-push +0 -34
  36. package/scripts/status.mjs +0 -20
  37. package/scripts/statusline.mjs +0 -45
  38. package/scripts/transcript-models.sh +0 -4
  39. package/skills/high/SKILL.md +0 -9
  40. package/skills/low/SKILL.md +0 -7
  41. package/skills/medium/SKILL.md +0 -8
  42. package/skills/micro/SKILL.md +0 -7
  43. package/skills/setup/SKILL.md +0 -31
  44. package/skills/status/SKILL.md +0 -12
@@ -1,203 +1,107 @@
1
1
  # Configuration
2
2
 
3
- ## Where the settings are
4
-
5
- | Setting | Location | Written by |
6
- | -------------------- | --------------------------------------------------------- | --------------------------------------- |
7
- | Jev API key | macOS Keychain, plugin option `typesafe_api_key` | Claude Code, when you enable the plugin |
8
- | Claude Code settings | `~/.claude/settings.json` | `/router:setup` |
9
- | Router configuration | `~/.claude/router.json` | You |
10
- | Data and logs | `~/.claude/plugins/data/router-alexei-led-claude-router/` | The gateway |
11
-
12
- - The hooks give the key to the gateway as `TYPESAFE_API_KEY`. The key is
13
- never in a file.
14
- - The gateway that the hooks start reads `~/.claude/router.json`, or the file
15
- that `ROUTER_CONFIG` names when it is under `~/.claude/`. A project can set
16
- environment variables for the plugin hooks, so the hooks take your home
17
- directory from your OS user account, not from `$HOME`, and ignore
18
- `ROUTER_CONFIG` for a path outside `~/.claude/`. So a cloned repository
19
- cannot choose the router configuration (and with it the port, the routes or
20
- the Jev endpoint) through `router.json`, `ROUTER_CONFIG` or `$HOME`. The
21
- gateway still inherits the other environment variables of the session that
22
- started it.
23
- - The gateway reads `router.json` at start. After a change, run
24
- `pkill -f scripts/gateway.mjs`. The next prompt starts a new gateway.
25
-
26
- ## Claude Code settings
27
-
28
- `/router:setup` writes these keys to `~/.claude/settings.json`:
3
+ The router works with its built-in defaults. Configure a Jev key to enable advice. Change `router.json` only when you need to change routes, model prices, cache assumptions, or policy.
4
+
5
+ ## Key and profile settings
6
+
7
+ | Item | Where it lives | How to change it |
8
+ | ---------------------------------- | ----------------------------------------------- | ------------------------------------------------------------------------------ |
9
+ | Jev API key | Sensitive plugin option `typesafe_api_key` | Run `/plugin configure router`, or use **Set Jev API key** in the router pane. |
10
+ | Router settings | `router.json` in the active Claude Code profile | Edit the file directly, or use the pane's tuning controls. |
11
+ | Anthropic credentials and API cost | Claude Code | The Mod leaves these to Claude Code. |
12
+
13
+ The active profile directory is `CLAUDE_CONFIG_DIR` when set. Otherwise it is `~/.claude`. The Mod does not need an Anthropic proxy URL, model alias, hint header, daemon, or custom status line.
14
+
15
+ The Mod reads the key from the sensitive plugin option. It also accepts `TYPESAFE_API_KEY` from the process environment. Without a key, it keeps Claude's current model and reports degraded routing.
16
+
17
+ Project and local settings cannot set `HOME` or `CLAUDE_CONFIG_DIR` for this Mod. Those overrides make the router unavailable for the session, so a project cannot redirect the router to another profile's key or configuration.
18
+
19
+ ## Built-in defaults
20
+
21
+ The default baseline is tier `low`. Each route refers to an alias in `models`.
22
+
23
+ | Tier | Model alias | Model ID | Effort |
24
+ | -------- | ----------- | ------------------- | ------------------------------------ |
25
+ | `micro` | `haiku` | `claude-haiku-4-5` | None |
26
+ | `low` | `sonnet` | `claude-sonnet-5-5` | Keeps the effort sent by Claude Code |
27
+ | `medium` | `sonnet` | `claude-sonnet-5-5` | `xhigh` |
28
+ | `high` | `opus` | `claude-opus-5-5` | `xhigh` |
29
+
30
+ Default model settings:
31
+
32
+ | Alias | Input / output / cache read, USD per million tokens | Context window | Billing | Effort levels |
33
+ | -------- | --------------------------------------------------- | -------------- | ------- | --------------------------------------- |
34
+ | `opus` | 4 / 20 / 0.2 | 1,000,000 | `plan` | `low`, `medium`, `high`, `xhigh`, `max` |
35
+ | `sonnet` | 2 / 10 / 0.2 | 1,000,000 | `plan` | `low`, `medium`, `high`, `xhigh`, `max` |
36
+ | `haiku` | 1 / 5 / 0.1 | 200,000 | `plan` | none |
37
+
38
+ Prices are configured list-price inputs for estimates. They are not a subscription bill or a claim of savings. Haiku receives no effort field. A route without an effort keeps the session effort, clamped to what the model supports.
39
+
40
+ ## Optional `router.json`
41
+
42
+ Start with the setting you need and leave the rest at defaults. For example, require three votes before moving to a cheaper tier and allow Jev up to two seconds:
29
43
 
30
44
  ```json
31
45
  {
32
- "model": "jev-router[1m]",
33
- "env": {
34
- "ANTHROPIC_BASE_URL": "http://127.0.0.1:43170",
35
- "ENABLE_TOOL_SEARCH": "true",
36
- "CLAUDE_CODE_GATEWAY_HINT_HEADERS": "1"
37
- },
38
- "modelPicker": {
39
- "options": [
40
- {
41
- "model": "jev-router[1m]",
42
- "label": "Jev Router (auto)",
43
- "description": "Auto-selects the model and effort for each turn",
44
- "behavesAs": "claude-opus-5-5"
45
- }
46
- ]
47
- }
46
+ "jev": { "timeoutMs": 2000 },
47
+ "policy": { "downgradeVotes": 3 }
48
48
  }
49
49
  ```
50
50
 
51
- | Key | Why |
52
- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
53
- | `model` | The `[1m]` suffix tells Claude Code that the window is 1M tokens. Claude Code removes the suffix before the request. |
54
- | `env.ANTHROPIC_BASE_URL` | Sends the traffic of Claude Code to the gateway. |
55
- | `env.ENABLE_TOOL_SEARCH` | Keeps MCP tool schemas deferred. Without it, each request carries all schemas: about 50K tokens with the claude.ai connectors. |
56
- | `env.CLAUDE_CODE_GATEWAY_HINT_HEADERS` | Claude Code sends the [class of each request](architecture.md#request-classes). |
57
- | `modelPicker` row | Adds `Jev Router (auto)` to `/model`. `behavesAs` names a known model. Without it, Claude Code rejects the alias. |
58
- | `statusLine` (optional) | Runs your status line command, then adds the route of the last turn. |
51
+ The Mod validates the whole file. Unknown keys and invalid values make routing unavailable with a general configuration error. Retired gateway keys trigger a migration instruction. The migration command reports an invalid setting path and does not print its value.
59
52
 
60
- Setup writes the file once, as its last step. Restart Claude Code after it.
61
- Until the restart, the session shows "There's an issue with the selected
62
- model (jev-router[1m])".
53
+ | Section | Supported fields | Defaults |
54
+ | ---------------- | ------------------------------------------------------------------------------------ | --------------------------------------------------------------- |
55
+ | `baselineTier` | `micro`, `low`, `medium`, `high` | `low` |
56
+ | `routes.<tier>` | `model`, optional `effort` | As in the route table above |
57
+ | `models.<alias>` | `id`, `input`, optional `output`, `cacheRead`, `contextWindow`, `billing`, `efforts` | Three defaults above |
58
+ | `cache` | `writeMultiplier`, `ttlMs`, `warmMarginMs` | `5m: 1.25`, `1h: 2`; `300000` / `3600000` ms; `30000` ms |
59
+ | `policy` | Fields below | Values below |
60
+ | `jev` | `endpoint`, `model`, `timeoutMs` | `https://api.typesafe.ai/v1/systemone`, `jev-1.13.0`, `1500` ms |
61
+ | `context` | `recentTurns`, `maxTextChars` | `6`, `1200` |
63
62
 
64
- ## Router configuration
63
+ Policy defaults:
65
64
 
66
- Each key in `~/.claude/router.json` is optional. A key replaces the default
67
- at the same path, and objects merge.
65
+ | Field | Default | Meaning |
66
+ | ----------------------- | ------: | -------------------------------------------------------------------------------------------------- |
67
+ | `upgradeVotes` | `2` | Consecutive supporting votes before an upgrade. |
68
+ | `upgradeBase` | `0.75` | Minimum probability mass required for an upgrade. |
69
+ | `upgradeSlope` | `0.15` | Maximum increase to the upgrade bar from estimated switching cost. |
70
+ | `upgradePivotUsd` | `0.5` | Cost scale used by the upgrade bar. Must be positive. |
71
+ | `jumpConfidence` | `0.95` | Support for a two-tier jump without waiting for votes. |
72
+ | `downgradeVotes` | `2` | Consecutive supporting votes before a downgrade. |
73
+ | `downgradeMass` | `0.9` | Minimum probability mass required for a downgrade. |
74
+ | `downgradeSlope` | `0.08` | Maximum increase to the downgrade bar from estimated switching cost. |
75
+ | `downgradePivotUsd` | `0.5` | Cost scale used by the downgrade bar. Must be positive. |
76
+ | `downgradeHorizonTurns` | `5` | Later turns included in a downgrade payback estimate. |
77
+ | `continuationMass` | `0.7` | Advice probability that keeps the current route for a continuation. |
78
+ | `escalationHoldTurns` | `2` | Turns held after a repeated tool error escalates the route. |
79
+ | `cashCapUsd` | `2` | Maximum estimated cold cache write for a `credits` model. It does not cap output or session spend. |
68
80
 
69
- The file is strict. Text that is not valid JSON, an unknown key (a typo such
70
- as `routs`), a value of the wrong type, or a number out of range stops a new
71
- gateway from starting. The error names the file and the field, never the
72
- value. A gateway that already runs keeps serving. When no gateway runs, every
73
- request fails to connect until you fix the file. The SessionStart hook prints
74
- the error; the prompt hook is quiet. `/router:status` also shows it: it reads
75
- the file and starts nothing.
81
+ The pane exposes only three tuning controls: Jev deadline, downgrade vote count, and downgrade horizon. Select **Save tuning** to preserve other keys and apply the change to future turns. The pane refuses to write through a symlink. Other supported settings require a direct edit.
76
82
 
77
- For example, to run `low` on Sonnet at `high` effort and give Jev more time:
83
+ Native cache freshness is unknown after 270 seconds or after a history reset. Claude usage does not report the cache TTL. The policy evaluates price bounds for five-minute and one-hour writes, but the one-hour case is not observed fact. See the [architecture](architecture.md#cache-and-cost) for the estimate rules.
78
84
 
79
- ```json
80
- {
81
- "routes": { "low": { "model": "sonnet", "effort": "high" } },
82
- "jev": { "timeoutMs": 2500 }
83
- }
85
+ ## Convert a v0.8 configuration
86
+
87
+ The `router.json` converter changes only that file. It does not remove Claude Code gateway settings. Remove them before the first native session.
88
+
89
+ A marketplace install keeps the plugin ID `router@alexei-led-claude-router`: update it and keep it enabled. If you load v1 from a checkout with `--plugin-dir` instead, disable the marketplace v0.8 install so only one router loads. Set `CLAUDE_CONFIG_DIR` to the intended profile directory first:
90
+
91
+ ```sh
92
+ CLAUDE_CONFIG_DIR=/path/to/profile claude plugin disable router@alexei-led-claude-router --scope user
84
93
  ```
85
94
 
86
- ### gateway
87
-
88
- | Key | Default | Meaning |
89
- | ---------------- | ------------ | ------------------------------------------------------------------------- |
90
- | `port` | `43170` | The loopback port of the gateway. |
91
- | `alias` | `jev-router` | The model name that Claude Code sends. |
92
- | `baselineTier` | `low` | The tier when nothing else decides: a new session, or no Jev answer. |
93
- | `auxiliaryTier` | `low` | The tier for side requests, for example session titles. |
94
- | `idleShutdownMs` | `7200000` | The gateway exits after this time without requests. `0` keeps it running. |
95
-
96
- ### routes
97
-
98
- | Tier | Default |
99
- | -------- | ------------------------------------------ |
100
- | `high` | `{ "model": "opus", "effort": "xhigh" }` |
101
- | `medium` | `{ "model": "sonnet", "effort": "xhigh" }` |
102
- | `low` | `{ "model": "sonnet" }` |
103
- | `micro` | `{ "model": "haiku" }` |
104
-
105
- `model` is a key of `models`. `effort` is `low`, `medium`, `high`, `xhigh` or
106
- `max`. Without `effort`, the gateway keeps the effort that Claude Code sent.
107
- If you change a route, also change the frontmatter of `skills/<tier>/SKILL.md`.
108
- A test makes sure that they agree.
109
-
110
- ### models
111
-
112
- | Alias | `id` | `input` | `output` | `cacheRead` | `contextWindow` | `maxOutput` | `billing` | `efforts` | `features` |
113
- | -------- | ------------------- | ------- | -------- | ----------- | --------------- | ----------- | --------- | --------- | ---------- |
114
- | `opus` | `claude-opus-5-5` | 4 | 20 | 0.2 | 1,000,000 | — | `plan` | all five | all three |
115
- | `sonnet` | `claude-sonnet-5-5` | 2 | 10 | 0.2 | 1,000,000 | — | `plan` | all five | all three |
116
- | `haiku` | `claude-haiku-4-5` | 1 | 5 | 0.1 | 200,000 | 64,000 | `plan` | none | none |
117
-
118
- | Field | Meaning |
119
- | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
120
- | `id` | The model id that the gateway sends to Anthropic. |
121
- | `input`, `output`, `cacheRead` | List prices in USD per million tokens. `output` is optional. The `shadow` estimate and the downgrade tax use it. |
122
- | `contextWindow` | The window in tokens. A turn goes only to a model that holds the context at 80% fill. |
123
- | `maxOutput` | Optional limit for `max_tokens`. Haiku 4.5 rejects more than 64K. |
124
- | `billing` | `plan` for the subscription limits. `credits` for models that bill usage credits. |
125
- | `efforts` | The effort levels that the model accepts. An empty list removes effort and thinking. |
126
- | `features` | The request features of Claude Code that the model accepts: `mid-conversation-system`, `per-turn-control`, `mid-conversation-tool-changes`. The gateway removes the others ([architecture](architecture.md#system-context)). Without the field: none. |
127
-
128
- The default prices match `test/fixtures/list-prices.json`, which names its
129
- source and date. The default `efforts` match `test/fixtures/effort-support.json`,
130
- from a probe against the real API. A test fails when they differ.
131
-
132
- ### policy
133
-
134
- The [switching policy](architecture.md#switching-policy) uses these values.
135
-
136
- | Key | Default | Meaning |
137
- | ----------------------- | ------- | ----------------------------------------------------------------------------------------------------- |
138
- | `upgradeVotes` | `2` | Consecutive votes above the current tier before an upgrade. |
139
- | `upgradeBase` | `0.75` | The upgrade bar when a switch costs nothing. |
140
- | `upgradeSlope` | `0.15` | How much a switching cost can raise the bar: `base + slope × tax / (tax + pivot)`. |
141
- | `upgradePivotUsd` | `0.5` | The switching cost that adds half of the slope. |
142
- | `jumpConfidence` | `0.95` | The confidence for a jump of two tiers without the vote delay. |
143
- | `downgradeVotes` | `2` | Consecutive votes for a lower tier before a downgrade. |
144
- | `downgradeMass` | `0.9` | The downgrade bar when the candidate's cache is already warm. |
145
- | `downgradeSlope` | `0.08` | How much a cold candidate's cache write can raise the bar, same formula as upgrade. |
146
- | `downgradePivotUsd` | `0.5` | The switching cost that adds half of the slope. |
147
- | `downgradeHorizonTurns` | `5` | The turns of output and read savings that a downgrade's tax nets. Input only without `output` prices. |
148
- | `continuationMass` | `0.7` | The Jev probability for "continues the task" that keeps the route. |
149
- | `escalationHoldTurns` | `2` | Turns that the route stays up after an escalation. |
150
- | `cashCapUsd` | `2` | The cold-write guard: the largest first cache write for a switch to a `credits` model. |
151
-
152
- ### cache, jev, context, log
153
-
154
- | Key | Default | Meaning |
155
- | ----------------------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------- |
156
- | `cache.writeMultiplier` | `{ "5m": 1.25, "1h": 2 }` | The price of a cache write, as a multiple of `input`. |
157
- | `cache.ttlMs` | `{ "5m": 300000, "1h": 3600000 }` | The lifetime of each cache TTL. |
158
- | `cache.warmMarginMs` | `30000` | A cache counts as cold this long before it expires. |
159
- | `jev.endpoint` | `https://api.typesafe.ai/v1/systemone` | The Jev API. |
160
- | `jev.model` | `jev-1.13.0` | The Jev model. |
161
- | `jev.timeoutMs` | `1500` | The total time for one Jev answer, one retry included. |
162
- | `context.recentTurns` | `6` | The number of recent turns that Jev receives. |
163
- | `context.maxTextChars` | `1200` | The characters of the prompt and of each turn that Jev receives. A longer text keeps its start and end. |
164
- | `log` | `true` | Write `decisions.jsonl`. |
165
-
166
- ## Environment variables
167
-
168
- | Variable | Effect |
169
- | -------------------- | ------------------------------------------------------------------------------------------------------------ |
170
- | `TYPESAFE_API_KEY` | The Jev key. The hooks set it from the plugin option. |
171
- | `ROUTER_CONFIG` | Another path for the router configuration, under `~/.claude/` only. Another path is ignored, with a warning. |
172
- | `CLAUDE_PLUGIN_DATA` | The data directory. Claude Code sets it for the plugin hooks. |
173
-
174
- ## Command-line flags
175
-
176
- A project cannot change these: they are part of the command, not of the
177
- environment.
178
-
179
- | Flag | Script | Effect |
180
- | --------------------- | ------------------------------------------------- | ------------------------------------------------------------------------ |
181
- | `--config <path>` | `gateway.mjs`, `ensure-gateway.mjs`, `status.mjs` | The router configuration, at any path. |
182
- | `--force-tier <tier>` | `gateway.mjs` | `micro`, `low`, `medium` or `high`. Skips Jev and the policy. For tests. |
183
-
184
- The flags are for a gateway that you start by hand. The hooks and the status
185
- line do not pass them. To use another file with the hooks, keep it under
186
- `~/.claude/` and set `ROUTER_CONFIG`.
187
-
188
- To force a tier, stop the gateway and start it by hand with the data directory
189
- of the plugin, so the log stays in its place:
95
+ Then inspect that profile's `settings.json`. Remove the old `jev-router[1m]` model-picker row and any `model` or `env.ANTHROPIC_MODEL` value set to that alias. Remove `env.ANTHROPIC_BASE_URL` when it points to the router's loopback port. Clear `env.CLAUDE_CODE_GATEWAY_HINT_HEADERS`. Restore or remove a router-specific `statusLine` that points to the deleted script. Remove `env.ENABLE_TOOL_SEARCH` only if the old router setup added it and you do not need it for another reason. Preserve unrelated settings. Set the normal full baseline model, for example `claude-sonnet-5-5`.
96
+
97
+ No conversion is needed if you never created `router.json`. For an existing file, run the migration command from the v1 plugin directory, a checkout or the installed plugin cache:
190
98
 
191
99
  ```sh
192
- pkill -f scripts/gateway.mjs
193
- CLAUDE_PLUGIN_DATA=~/.claude/plugins/data/router-alexei-led-claude-router \
194
- node <plugin>/scripts/gateway.mjs --force-tier high
100
+ node scripts/migrate-config.mjs ~/.claude/router.json
195
101
  ```
196
102
 
197
- ## Data directory
103
+ Pass the file under your active profile instead when `CLAUDE_CONFIG_DIR` points elsewhere. The command validates the converted configuration before writing it and creates an exact-byte `router.json.v0.8.backup`. It refuses to overwrite an existing backup. Route, model-price, cache, and policy overrides remain in the converted file.
104
+
105
+ The converter removes old gateway and logging settings, request-feature fields, and model output caps. It moves `gateway.baselineTier` to top-level `baselineTier`. It stops without writing when it finds an unsupported old field or invalid setting.
198
106
 
199
- | File | Content | Retention |
200
- | ----------------- | --------------------------------------------- | -------------------------------------------- |
201
- | `decisions.jsonl` | One line for each decision and each response. | Moves to `.1` above 20 MB. |
202
- | `gateway.log` | Gateway events with timestamps. | Moves to `.1` above 20 MB, at gateway start. |
203
- | `sessions/` | The routing memory of each session. | Removed after 30 days without use. |
107
+ Node.js 22 or later is needed only for this migration command and project development. Claude Code runs the Mod. Node is not a routing service.
@@ -1,83 +1,64 @@
1
1
  # Evaluation
2
2
 
3
- A snapshot of real work by one developer on one machine. The router ran in
4
- Claude Code (claude-router) and in Pi (pi-model-router). Both use Jev.
5
-
6
- ![jev-router in Claude Code: where the requests went, and what the work cost](tier-share.svg)
7
-
8
- ## Claude Code
9
-
10
- The window starts at the first request on the current models (Opus 5.5,
11
- Sonnet 5, Haiku 4.5): 2026-09-23 09:05 UTC. It ends at 2026-09-24 06:00 UTC.
12
- The data has 26 sessions and 1,611 requests. Jev advised 111 prompts.
13
-
14
- | Tier | Model and effort | Requests | Share |
15
- | -------- | ------------------ | -------- | ----- |
16
- | `micro` | Haiku 4.5 | 154 | 9.6% |
17
- | `low` | Sonnet 5 | 1,239 | 76.9% |
18
- | `medium` | Opus 5.5, `high` | 73 | 4.5% |
19
- | `high` | Opus 5.5, `xhigh` | 145 | 9.0% |
20
-
21
- The cost of the same work at list prices:
22
-
23
- | Scenario | Cost | Saving with the router |
24
- | ------------------------- | ------- | ---------------------- |
25
- | Always Opus at `xhigh` | $166.26 | 15.5% |
26
- | Best model per session | $146.54 | 4.1% |
27
- | jev-router | $140.55 | — |
28
-
29
- "Best model per session" is a user who knows in advance the strongest tier
30
- that Jev selected in each session, and uses it for the full session. The
31
- result for each group of sessions:
32
-
33
- | Strongest tier in the session | Sessions | Requests | Router | Baseline | Saving |
34
- | ----------------------------- | -------- | -------- | ------- | -------- | ------ |
35
- | `low` (Sonnet) | 23 | 573 | $39.74 | $39.74 | 0% |
36
- | `medium` (Opus, `high`) | 2 | 847 | $76.89 | $84.57 | 9.1% |
37
- | `high` (Opus, `xhigh`) | 1 | 191 | $23.92 | $22.23 | −7.6% |
38
-
39
- ## Output size
40
-
41
- The baselines keep the output tokens of the router. A stronger model at a
42
- higher effort writes more, so the real savings are larger. If the stronger
43
- route writes more output on the requests that the router sent lower, the
44
- savings are:
45
-
46
- | Output of the stronger route | vs. always Opus at `xhigh` | vs. best model per session |
47
- | ---------------------------- | -------------------------- | -------------------------- |
48
- | Same (1×) | 15.5% | 4.1% |
49
- | 1.5× | 21.1% | 7.6% |
50
- | 2× | 26.0% | 10.8% |
51
-
52
- ## Pi
53
-
54
- Pi used two OpenAI profiles from 2026-09-22 21:00 to 2026-09-24 06:00 UTC,
55
- with 25 sessions and 1,964 requests. Jev advised 148 decisions. The profiles
56
- put correctness first: `high` served 83.6% of the requests and `medium` served
57
- 16.4%. The router cost $347.90. The best model per session cost $348.37. With
58
- this policy, routing is cost-neutral.
59
-
60
- ## Method
61
-
62
- - **Data.** Claude Code: `decisions.jsonl`, one `observed` line with the
63
- usage of each response. Pi: the session files, a router decision and the
64
- usage of each message. Neither source has prompt text.
65
- - **Prices.** List prices in USD per million tokens. Claude: the defaults in
66
- `lib/config.mjs`. Pi: the Pi model registry, with long-context prices. For
67
- subscription plans, the dollars show the relative cost only.
68
- - **Cache writes.** In Claude Code, the input that is not a cache read counts
69
- as a cache write at the granted TTL: 1.25× input for 5 minutes, 2× for 1
70
- hour.
71
- - **Baselines.** Each baseline uses the tokens and the cache reads of the
72
- router. After each change of route in a conversation, the baseline also
73
- reads the previous context from the cache, if the cache is still warm. Thus
74
- the baseline does not pay for the cache writes that routing causes.
75
- - **Tiers with one model.** Opus at `high` and at `xhigh` have the same price.
76
- Subagents count toward the strongest tier of their session.
77
-
78
- ## Limits
79
-
80
- - One developer and two days make a snapshot, not a benchmark.
81
- - Answer quality was not measured.
82
- - For Pi, the baseline assumes that the OpenAI cache stays warm for 10
83
- minutes.
3
+ The native Mod has no measured savings result. The pane reports Claude's own usage and configured-price scenarios. It does not maintain a counterfactual bill or a cumulative savings total.
4
+
5
+ This page separates an older gateway experiment from a shadow replay of native policy. Neither result measures answer quality or proves lower real spend.
6
+
7
+ ## Historical gateway experiment
8
+
9
+ These figures are retained from the earlier gateway implementation. They do not evaluate the native Mod or its current routes. The snapshot covers 26 Claude Code sessions and 1,611 requests from 2026-09-23 09:05 UTC through 2026-09-24 06:00 UTC. Jev advised on 111 prompts.
10
+
11
+ ![Historical gateway model share and configured-price costs](tier-share.svg)
12
+
13
+ The original list-price comparison was:
14
+
15
+ | Scenario | Estimated cost | Difference from router |
16
+ | ----------------------------------------------- | -------------: | ---------------------: |
17
+ | Always Opus at `xhigh` | $166.26 | Router was 15.5% lower |
18
+ | Strongest model selected for each whole session | $146.54 | Router was 4.1% lower |
19
+ | Earlier gateway router | $140.55 | — |
20
+
21
+ The session-level baseline assumes a user knows the strongest tier needed by each session and uses it for every request. All scenarios reuse the router's token and output counts. The estimate holds stronger-model output constant, so it does not capture any change in output length. The [original evaluation method](#historical-method-and-limits) lists the limits.
22
+
23
+ This is one developer's short trace at configured list prices. It does not include a quality review. Do not use these values as expected results for the native Mod.
24
+
25
+ ## Native shadow replay
26
+
27
+ The native replay uses an aggregate Team-session trace from 2026-09-23 through 2026-10-04. The checked-in result is [`trace-evaluation.json`](../experiments/mod-router/results/trace-evaluation.json), produced by [`trace-eval.mjs`](../experiments/mod-router/scripts/trace-eval.mjs).
28
+
29
+ | Coverage item | Result |
30
+ | ----------------------------------------------------------- | --------------------------------: |
31
+ | Team rows | 24,408 across 112 sessions |
32
+ | Main-conversation decision rows | 11,619 |
33
+ | Main decisions with Jev advice | 37 |
34
+ | Advised decisions replayed at both cache-prefix bounds | 37 of 37 |
35
+ | Decisions with a different route or reason between policies | 0 of 37 |
36
+ | Differing switches and estimated switch cost | 0; no switch cost can be computed |
37
+
38
+ The legacy replay matched the logged tier and reason for 1,704 of 1,704 replayable decisions. On the 37 advised main decisions, both policies produced the same route and reason. This replay therefore shows no native routing difference and cannot support a savings claim.
39
+
40
+ Eight advised decisions reached a switching-cost threshold calculation. The native configured-price tax ranged from $0.028660 to $3.471293 and the threshold from 0.860483 to 0.953675. These are shadow estimates for individual next requests, not bills or additive savings.
41
+
42
+ The old log records cache reads but not uncached input or cache-creation tokens. The native replay tests two prefix bounds: cache reads alone and the full observed input. All 37 decisions matched at both bounds. That is a bound-stability check, not an exact reconstruction of native cache state.
43
+
44
+ ### Reproduce the native replay
45
+
46
+ Run this command with the Team `decisions.jsonl` log available in its usual Claude Code profile:
47
+
48
+ ```sh
49
+ node experiments/mod-router/scripts/trace-eval.mjs
50
+ node --test test/trace-eval.test.mjs
51
+ ```
52
+
53
+ The script writes aggregate counts, reason histograms, route transitions, and shadow estimates. It excludes prompts, session identifiers, agent names, error text, headers, keys, and filesystem paths from the result. It reads the local profile log and writes the JSON result file.
54
+
55
+ ## Historical method and limits
56
+
57
+ - Costs use configured USD list prices per million tokens. They are not subscription cash charges.
58
+ - The older gateway baseline uses the router's observed tokens and cache reads. After a route change, the baseline assumes the prior context can be read from cache.
59
+ - The native replay uses the current default routes and prices, with recorded model IDs substituted. It compares the native cost bounds with a legacy cost model using the same policy state and trace facts.
60
+ - The replay cannot reconstruct exact native cache prefixes or one-hour cache evidence from the old log. The native cache model treats TTL as unknown.
61
+ - The replay applies the legacy state chain to both policy arms. It does not simulate later state changes after a different decision.
62
+ - Failure signatures and pins are not present in the trace. The relevant route decisions are excluded from comparison.
63
+ - Most main rows are tool continuations or have no Jev advice. Only 37 advised decisions support the native comparison.
64
+ - Costs cover the next request. They use the last observed output size and do not model effort-dependent output length or future answer quality.
@@ -0,0 +1,79 @@
1
+ # Native router: behavior and evidence
2
+
3
+ This guide describes the Mod shipped by this checkout. Claude Code 2.1.289 is the minimum tested version. The router changes `model` and `effort` on main-conversation steps. Claude Code handles the Anthropic request, credentials, stream, tool execution, and cost ledger. Subagent steps keep their original model and effort.
4
+
5
+ ## Turn behavior
6
+
7
+ At the first main step of a logical turn, the Mod reads recent message text and asks Jev for a tier. It applies local policy and saves the route. Tool continuations reuse that route. Before each continuation, context fit is checked again. A large tool result can move the step to a model with a larger window.
8
+
9
+ Jev receives the current prompt and at most six preceding user or assistant text messages. Each item is limited to 1,200 characters. It does not receive system messages, tool inputs, or tool results. Claude Code's Anthropic credentials never enter the Jev request.
10
+
11
+ The default tiers are defined in [Configuration](configuration.md#built-in-defaults). Policy uses vote hysteresis, repeated tool errors, context fit, model availability, and switching-cost estimates. The [architecture](architecture.md#request-flow) explains the event boundary and failure handling.
12
+
13
+ ## Controls and session modes
14
+
15
+ | Control | Behavior |
16
+ | -------------------- | ----------------------------------------------------------------------- |
17
+ | `/router` | Open the details pane. Without a UI surface, print short status. |
18
+ | `/router status` | Print short status. |
19
+ | `/router auto` | Enable automatic routing. |
20
+ | `/router off` | Enter Manual mode and preserve Claude's selected model. |
21
+ | `/router pin <tier>` | Pin the next turn and tool continuations. Auto must already be enabled. |
22
+ | `/router setup` | Open Claude Code's secure plugin configuration for the Jev key. |
23
+ | `/model <name>` | Select a model and enter Manual mode. `/router auto` resumes routing. |
24
+
25
+ Mode belongs to a Claude Code session. Clear starts a new session in Auto. Resume restores the saved mode for that session. A new or forked session has a new ID: it starts in Auto on the baseline model and in Manual on another model, as a fresh launch does. A pin applies to one logical turn, then the prior Auto incumbent resumes. A history reset clears votes and cache evidence; the current turn keeps its route, pin and any native fallback.
26
+
27
+ ## Router pane
28
+
29
+ The status band stays compact above the prompt. The **Router** button opens a pane with model choice, actual response, reason, metrics, estimates, controls, and tuning.
30
+
31
+ | Reading | Meaning and limits |
32
+ | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
33
+ | Last reply / Selected | Actual response model versus the model requested for that step. The engine does not report actual effort. |
34
+ | API cost reported by Claude | Claude Code's native usage value. A plan's list-price estimates are not subscription cash charges. |
35
+ | Context bar | Maximum available observed and local estimated context, compared with the routed model's window. The fit rule reserves 20%. |
36
+ | Observed / estimated input | Claude's local context reading and the router's best current context estimate. `unknown` blocks a move to a smaller window. |
37
+ | Cache reuse | Last response's cache-read tokens divided by its reported input counters. |
38
+ | Cache read / written; Output | Response usage counters. Cache warmth does not prove a particular TTL. |
39
+ | Input trend | Up to ten observed main-conversation input sizes, scaled to the largest one. |
40
+ | Jev / Last classification | Classifier state and elapsed time. Missing-key, busy, and paused skips have no network latency reading. |
41
+ | Next-turn difference | Configured-price range for the suggested tier versus the incumbent on the next request. Negative means estimated lower cost. |
42
+ | Conservative payback | Conditional later-turn estimate based on configured cache prices, future reads, and the last output size. It is not measured savings. |
43
+ | Cache read benefit | Estimated price difference for observed cache reads before cache writes. It is not net savings. |
44
+
45
+ There is no router-owned cost ledger or cumulative savings counter. Jev charges are not included. See [Evaluation](evaluation.md) for what the available trace can support.
46
+
47
+ ## Tuning
48
+
49
+ The pane exposes three controls. Draft changes do not affect the active turn. **Save tuning** validates and writes them to `router.json` for future turns, preserving unrelated keys.
50
+
51
+ | Control | Values in the pane | Effect |
52
+ | ------------------------- | --------------------------- | ------------------------------------------------- |
53
+ | Jev deadline | 500, 1,000, 1,500, 3,000 ms | Total advice time, including any transient retry. |
54
+ | Votes before cheaper tier | 1, 2, 3 | Consecutive votes required for a downgrade. |
55
+ | Payback horizon | 1, 3, 5, 10 turns | Later turns included in downgrade economics. |
56
+
57
+ The panel refuses to write through a symlink. Other supported configuration fields need an edit to the active profile's `router.json`.
58
+
59
+ ## Safety and failure states
60
+
61
+ - Missing Jev key, a Jev failure, or a policy refusal keeps the current model. Network refusal is never bypassed with a helper process.
62
+ - Three launched Jev failures open a 60-second pause. One in-flight request is allowed per active Mod instance. Pending host HTTP work blocks another request until it settles.
63
+ - Unsupported Claude Code versions and detected gateway aliases/base URLs mark the router unavailable. A local gateway does not run as part of this Mod.
64
+ - A context-window error makes that model ineligible until the next history reset. The Mod does not retry an Anthropic request after a stream begins.
65
+ - Unknown context retains the current model. There is no characters-to-tokens fallback.
66
+ - Unknown model substitutions receive no cache credit for the requested model.
67
+ - Clear, rewind, branch, or committed compaction clears cache evidence and policy votes. Resume restores the selected session's mode, not its old cache evidence.
68
+
69
+ ## Verified checks
70
+
71
+ The Team canary loaded the isolated native plugin with a full Sonnet baseline, reached the real Jev endpoint, and kept Sonnet on a `downgrade-pending` result. This was a routing smoke test, not a measured-savings trial.
72
+
73
+ Recorded acceptance checks cover the local HTTP deadline and one-request admission, session clear/resume, circuit-breaker pause, reload, replacement, unload, engine fallback, and interrupt during Jev advice. The interrupt check closed the loopback socket within 4 ms and discarded late advice without changing the active route or UI. Results are stored under [`experiments/mod-router/results`](../experiments/mod-router/results/).
74
+
75
+ The model probes accepted signed-thinking history and tool results across Sonnet, Opus, and Haiku. Sonnet and Opus accepted histories above 589K input tokens. Haiku deferred-tool use passed with a 128K output limit. These probes validate the tested path only. Claude Code and organization policy control other model and request combinations.
76
+
77
+ The Team launcher loads the Mod from a Team-owned directory passed with `--plugin-dir`. Its disable switch is tested: when that directory is absent, `ce team` starts on native Sonnet with no router plugin. Rename the directory to a sibling `.disabled` name and restart to disable the router. Rename it back to re-enable the Mod. The destination name must be free.
78
+
79
+ That switch disables routing. It does not restore the v0.8 gateway. The [user guide](user-guide.md#roll-back-to-v080) gives the v0.8.0 rollback. A rehearsal loaded the v0.8.0 tag with `--plugin-dir` after disabling v1, and started its gateway from that checkout on an isolated port.