@alexeiled/claude-router 0.7.0 → 0.7.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +33 -1
- package/docs/architecture.md +58 -46
- package/docs/configuration.md +72 -69
- package/docs/user-guide.md +1 -1
- package/lib/config.mjs +10 -5
- package/lib/cost.mjs +12 -3
- package/lib/policy.mjs +16 -3
- package/lib/router.mjs +2 -1
- package/package.json +1 -1
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "router",
|
|
3
3
|
"displayName": "jev-router",
|
|
4
|
-
"version": "0.7.
|
|
4
|
+
"version": "0.7.2",
|
|
5
5
|
"description": "Auto-picks the right Claude model for each turn — small models for quick edits, mid-tier for code, frontier for hard problems. Uses Jev to classify each request.",
|
|
6
6
|
"author": { "name": "Alexei Ledenev", "url": "https://github.com/alexei-led" },
|
|
7
7
|
"repository": "https://github.com/alexei-led/claude-router",
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,37 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.7.2 (2026-09-25)
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
|
|
7
|
+
- `router.json` accepted a `policy.downgradePivotUsd` of 0 and a
|
|
8
|
+
`policy.downgradeSlope` that is not a number. A zero pivot made the
|
|
9
|
+
downgrade bar `NaN` whenever a switch cost nothing, and then no downgrade
|
|
10
|
+
ever passed. Both values now stop a new gateway from starting, as the
|
|
11
|
+
upgrade keys already did; `/router:status` names the field.
|
|
12
|
+
|
|
13
|
+
### Changed
|
|
14
|
+
|
|
15
|
+
- The downgrade bar counts what a cheaper model saves, not only its cache
|
|
16
|
+
write. The tax used to count the input of the next request alone. It now
|
|
17
|
+
nets the output and cache-read savings of the next
|
|
18
|
+
`policy.downgradeHorizonTurns` turns (default 5). From a warm Opus at
|
|
19
|
+
`xhigh` to a cold Sonnet, with 104k of context and 4k of output, the bar
|
|
20
|
+
drops from 0.935 to 0.922 (1-hour TTL). It never goes below 0.90. A model
|
|
21
|
+
without an `output` price keeps the input-only tax.
|
|
22
|
+
|
|
23
|
+
## 0.7.1 (2026-09-24)
|
|
24
|
+
|
|
25
|
+
### Fixed
|
|
26
|
+
|
|
27
|
+
- A downgrade paid no attention to its own cost. The switching policy
|
|
28
|
+
checked confidence and vote count for a lower tier, never the price of
|
|
29
|
+
getting there: a candidate whose cache was cold next to a warm incumbent
|
|
30
|
+
could downgrade into a full cache write bigger than what the smaller
|
|
31
|
+
model saved. The downgrade bar now grows with the switching tax the same
|
|
32
|
+
way the upgrade bar does: `0.90 + 0.08 × tax / (tax + $0.50)`, up to
|
|
33
|
+
0.98. A downgrade to an already-warm candidate is unaffected.
|
|
34
|
+
|
|
3
35
|
## 0.7.0 (2026-09-24)
|
|
4
36
|
|
|
5
37
|
### Upgrade requirements
|
|
@@ -71,7 +103,7 @@
|
|
|
71
103
|
asked, a tool continuation looked like a new turn, and a request that Claude
|
|
72
104
|
Code resent after a 400 was decided again.
|
|
73
105
|
- The first Haiku turn of a session failed with `400 role 'system' is not
|
|
74
|
-
|
|
106
|
+
supported on this model`, and a later one made Claude Code flatten the
|
|
75
107
|
history for the rest of the session: Opus then lost its features, and the
|
|
76
108
|
router saw a shorter history and reset its votes. Sonnet answered the first
|
|
77
109
|
turn with two 400s before Claude Code retried without the features.
|
package/docs/architecture.md
CHANGED
|
@@ -67,8 +67,8 @@ flowchart TB
|
|
|
67
67
|
`output_config` on messages (the effort of each turn) go;
|
|
68
68
|
- no `mid-conversation-system` (Haiku): each `system` message becomes a
|
|
69
69
|
`<system-reminder>` text in the user message next to it.
|
|
70
|
-
|
|
71
|
-
|
|
70
|
+
The same history adapts the same way on every turn, so the cached prefix
|
|
71
|
+
holds.
|
|
72
72
|
- On a model with `per-turn-control`, the effort of each turn overrides the
|
|
73
73
|
request's effort. A route with its own effort (`medium`, `high`) therefore
|
|
74
74
|
sets it in every message's `output_config` too.
|
|
@@ -152,17 +152,17 @@ flowchart TD
|
|
|
152
152
|
class RESET,KEEP,POL,FIT act
|
|
153
153
|
```
|
|
154
154
|
|
|
155
|
-
| Check | How the gateway decides
|
|
156
|
-
| --------------- |
|
|
157
|
-
| Pass through | Any other model: `/model`, subagents with their own model.
|
|
158
|
-
| Pin | The current message starts with the `/router:<tier>` command block. That tier serves the turn and its tool calls, reason `pinned`. No Jev call.
|
|
159
|
-
| Not a turn | `count_tokens` and other endpoints get the last route of the session.
|
|
160
|
-
| Side request | Header `x-claude-code-request-class` is `auxiliary` or `compaction`. Without the header: thinking disabled or an output format in the body.
|
|
161
|
-
| History break | Fewer messages than the last main request (a compaction or a rewind), or the header `x-claude-code-context-compacted`.
|
|
155
|
+
| Check | How the gateway decides |
|
|
156
|
+
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
157
|
+
| Pass through | Any other model: `/model`, subagents with their own model. |
|
|
158
|
+
| Pin | The current message starts with the `/router:<tier>` command block. That tier serves the turn and its tool calls, reason `pinned`. No Jev call. |
|
|
159
|
+
| Not a turn | `count_tokens` and other endpoints get the last route of the session. |
|
|
160
|
+
| Side request | Header `x-claude-code-request-class` is `auxiliary` or `compaction`. Without the header: thinking disabled or an output format in the body. |
|
|
161
|
+
| History break | Fewer messages than the last main request (a compaction or a rewind), or the header `x-claude-code-context-compacted`. |
|
|
162
162
|
| Resent request | Same last user message at the same position. Claude Code resends after a 429, a 529, a dropped stream, or a 400 that it answers by dropping a feature. No Jev call, no vote. |
|
|
163
|
-
| System messages | Claude Code sends hook output and tool additions as `system` messages after the prompt. The turn is the last user or assistant message; Jev never receives `system` text.
|
|
164
|
-
| Context fit | The route moves to a model whose window holds the next context at 80% fill. Reason `context-fit`.
|
|
165
|
-
| Session memory | Key `x-claude-code-session-id`. A subagent adds its agent id: `<session>.<agent id>`. Its turns do not change the route of the main conversation.
|
|
163
|
+
| System messages | Claude Code sends hook output and tool additions as `system` messages after the prompt. The turn is the last user or assistant message; Jev never receives `system` text. |
|
|
164
|
+
| Context fit | The route moves to a model whose window holds the next context at 80% fill. Reason `context-fit`. |
|
|
165
|
+
| Session memory | Key `x-claude-code-session-id`. A subagent adds its agent id: `<session>.<agent id>`. Its turns do not change the route of the main conversation. |
|
|
166
166
|
|
|
167
167
|
## Switching policy
|
|
168
168
|
|
|
@@ -185,7 +185,7 @@ flowchart TD
|
|
|
185
185
|
JMP -- no --> UPQ{"2 votes,<br/>U ≥ bar?"}
|
|
186
186
|
UPQ -- yes --> UP["upgrade"]
|
|
187
187
|
UPQ -- no --> R5["upgrade-pending"]
|
|
188
|
-
DIR -- lower --> DNQ{"2 votes,<br/>D ≥
|
|
188
|
+
DIR -- lower --> DNQ{"2 votes,<br/>D ≥ bar?"}
|
|
189
189
|
DNQ -- yes --> DOWN["downgrade"]
|
|
190
190
|
DNQ -- no --> R6["downgrade-pending"]
|
|
191
191
|
|
|
@@ -205,9 +205,21 @@ only when that tier is above the current route.
|
|
|
205
205
|
or below the candidate. The `uncertain` mass supports neither.
|
|
206
206
|
- The upgrade bar grows with the switching tax:
|
|
207
207
|
`bar = 0.75 + 0.15 × tax / (tax + $0.50)`. It stays between 0.75 and 0.90.
|
|
208
|
+
- The downgrade bar grows the same way, from the other side: a candidate
|
|
209
|
+
colder than the incumbent pays a cache write, not just a smaller model.
|
|
210
|
+
`bar = 0.90 + 0.08 × tax / (tax + $0.50)`. It stays between 0.90 and 0.98.
|
|
211
|
+
- The downgrade tax nets the savings of the next `H` turns
|
|
212
|
+
(`policy.downgradeHorizonTurns`, default 5):
|
|
213
|
+
`tax = max(0, next + (H − 1) × later)`. `next` is the cost difference of
|
|
214
|
+
the next turn, input and output. `later` is the difference of each later
|
|
215
|
+
turn, with both caches warm. Both come from the
|
|
216
|
+
[shadow estimate](#cache-and-cost-model). Example: warm Opus at `xhigh` to
|
|
217
|
+
a cold Sonnet, 104k of context, 4k of output, 1-hour TTL. The input tax is
|
|
218
|
+
$0.40 (bar 0.935); net of five turns it is $0.20 (bar 0.922).
|
|
219
|
+
- When either model has no `output` price, or a zero one, the downgrade tax
|
|
220
|
+
is the input tax alone. A missing price is unknown, not free.
|
|
208
221
|
- A prompt continues the task when the Jev continuation answer is 0.7 or
|
|
209
222
|
more. A continuing prompt never votes for a downgrade.
|
|
210
|
-
- A downgrade has no tax term.
|
|
211
223
|
- Upgrades have no cooldown. Plan, then execute, then hard work again is a
|
|
212
224
|
valid sequence.
|
|
213
225
|
- A failure signal is two `tool_result` errors with the same text and an edit
|
|
@@ -239,11 +251,11 @@ input_cost = P_read × W + P_write × (N − W)
|
|
|
239
251
|
tax = max(0, input_cost(candidate) − input_cost(current))
|
|
240
252
|
```
|
|
241
253
|
|
|
242
|
-
| Symbol | Source
|
|
243
|
-
| --------- |
|
|
244
|
-
| `N` | Next context: tokens and output of the last response.
|
|
245
|
-
| `W` | Warm prefix of the candidate cache. Zero when the cache is `expired` or `unknown`.
|
|
246
|
-
| `P_read` | List price for cache reads.
|
|
254
|
+
| Symbol | Source |
|
|
255
|
+
| --------- | ------------------------------------------------------------------------------------------------------ |
|
|
256
|
+
| `N` | Next context: tokens and output of the last response. |
|
|
257
|
+
| `W` | Warm prefix of the candidate cache. Zero when the cache is `expired` or `unknown`. |
|
|
258
|
+
| `P_read` | List price for cache reads. |
|
|
247
259
|
| `P_write` | Input list price × 1.25 (5-minute TTL) or × 2 (1-hour TTL). The TTL comes from `usage.cache_creation`. |
|
|
248
260
|
|
|
249
261
|
- **Upper bound.** An effort change counts the whole prefix as lost. On some
|
|
@@ -258,8 +270,8 @@ tax = max(0, input_cost(candidate) − input_cost(current))
|
|
|
258
270
|
`test/fixtures/list-prices.json`, which names its source and date.
|
|
259
271
|
- **Shadow estimate.** When the Jev choice differs from the current route, the
|
|
260
272
|
log records the cost of the next turn, the cost of each later turn with
|
|
261
|
-
output, and the turns until a switch repays its cost. The
|
|
262
|
-
|
|
273
|
+
output, and the turns until a switch repays its cost. The downgrade tax
|
|
274
|
+
uses the same arithmetic. The upgrade tax counts input only.
|
|
263
275
|
|
|
264
276
|
## Modules
|
|
265
277
|
|
|
@@ -314,19 +326,19 @@ so tests run without a network. `gateway` imports one pure function from
|
|
|
314
326
|
`router`, the side-request test, not the router itself. Modules that need
|
|
315
327
|
settings or tier names import `config.mjs` (not drawn).
|
|
316
328
|
|
|
317
|
-
| Module | Responsibility
|
|
318
|
-
| --------- |
|
|
319
|
-
| `gateway` | HTTP server, loopback checks, relay with `pipeline()`, status endpoint.
|
|
320
|
-
| `router` | One request: class, facts, advice, policy, rewrite, session memory, log.
|
|
321
|
-
| `facts` | Prompt, recent turns, continuation, failure signal, resend key.
|
|
322
|
-
| `jev` | One bounded Jev request with a Choice (tier) and a Noul (continuation).
|
|
323
|
-
| `policy` | Switching policy, context fit, cold-write guard.
|
|
324
|
-
| `cost` | Cache key, warmth, input cost, switching tax, shadow estimate.
|
|
325
|
-
| `rewrite` | Model, effort, thinking and output limit for each model family.
|
|
326
|
-
| `sse` | `usage` from SSE and JSON responses.
|
|
327
|
-
| `idle` | When the daemon can exit.
|
|
328
|
-
| `status` | Status snapshot, status line segment, `/router:status` report.
|
|
329
|
-
| `store` | Session memory, `decisions.jsonl`, rotation.
|
|
329
|
+
| Module | Responsibility |
|
|
330
|
+
| --------- | ---------------------------------------------------------------------------------------------- |
|
|
331
|
+
| `gateway` | HTTP server, loopback checks, relay with `pipeline()`, status endpoint. |
|
|
332
|
+
| `router` | One request: class, facts, advice, policy, rewrite, session memory, log. |
|
|
333
|
+
| `facts` | Prompt, recent turns, continuation, failure signal, resend key. |
|
|
334
|
+
| `jev` | One bounded Jev request with a Choice (tier) and a Noul (continuation). |
|
|
335
|
+
| `policy` | Switching policy, context fit, cold-write guard. |
|
|
336
|
+
| `cost` | Cache key, warmth, input cost, switching tax, shadow estimate. |
|
|
337
|
+
| `rewrite` | Model, effort, thinking and output limit for each model family. |
|
|
338
|
+
| `sse` | `usage` from SSE and JSON responses. |
|
|
339
|
+
| `idle` | When the daemon can exit. |
|
|
340
|
+
| `status` | Status snapshot, status line segment, `/router:status` report. |
|
|
341
|
+
| `store` | Session memory, `decisions.jsonl`, rotation. |
|
|
330
342
|
| `runtime` | Configuration path (anchored under `~/.claude`), data directory, flags, gateway process check. |
|
|
331
343
|
|
|
332
344
|
## Gateway lifecycle
|
|
@@ -353,16 +365,16 @@ stateDiagram-v2
|
|
|
353
365
|
|
|
354
366
|
## Failure handling
|
|
355
367
|
|
|
356
|
-
| Failure
|
|
357
|
-
|
|
|
358
|
-
| Anthropic 429, 529, 5xx
|
|
359
|
-
| Upstream reset during a stream
|
|
360
|
-
| Client leaves (Esc)
|
|
361
|
-
| Jev error or timeout
|
|
362
|
-
| Three Jev failures in a row
|
|
363
|
-
| Routing error
|
|
364
|
-
| Disk error
|
|
365
|
-
| Unexpected exception
|
|
368
|
+
| Failure | Behavior |
|
|
369
|
+
| ------------------------------ | ---------------------------------------------------------------- |
|
|
370
|
+
| Anthropic 429, 529, 5xx | Pass through with `Retry-After`. Claude Code owns retries. |
|
|
371
|
+
| Upstream reset during a stream | The client connection resets, and Claude Code retries at once. |
|
|
372
|
+
| Client leaves (Esc) | The gateway stops the upstream request. |
|
|
373
|
+
| Jev error or timeout | One retry inside the 1.5 s budget. Then the current route stays. |
|
|
374
|
+
| Three Jev failures in a row | New turns skip Jev for one minute, then try once. |
|
|
375
|
+
| Routing error | The baseline tier serves. The alias never goes to Anthropic. |
|
|
376
|
+
| Disk error | Logged. The routing decision stays. |
|
|
377
|
+
| Unexpected exception | Logged. The daemon continues. |
|
|
366
378
|
|
|
367
379
|
## Security and privacy
|
|
368
380
|
|
|
@@ -429,7 +441,7 @@ claude --plugin-dir . --model jev-router # with ANTHROPIC_BASE_URL
|
|
|
429
441
|
- Without the hint headers, the gateway guesses side requests. A missed side
|
|
430
442
|
request with a short history counts as a history break.
|
|
431
443
|
- `x-claude-code-agent-type` goes to the log only.
|
|
432
|
-
- The shadow estimate
|
|
433
|
-
changes how much a model writes.
|
|
444
|
+
- The shadow estimate and the downgrade tax use the last output size for
|
|
445
|
+
both routes, but effort changes how much a model writes.
|
|
434
446
|
|
|
435
447
|
A day of real usage is in [Evaluation](evaluation.md).
|
package/docs/configuration.md
CHANGED
|
@@ -2,12 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
## Where the settings are
|
|
4
4
|
|
|
5
|
-
| Setting
|
|
6
|
-
|
|
|
7
|
-
| Jev API key
|
|
8
|
-
| Claude Code settings
|
|
9
|
-
| Router configuration
|
|
10
|
-
| Data and logs
|
|
5
|
+
| Setting | Location | Written by |
|
|
6
|
+
| -------------------- | --------------------------------------------------------- | --------------------------------------- |
|
|
7
|
+
| Jev API key | macOS Keychain, plugin option `typesafe_api_key` | Claude Code, when you enable the plugin |
|
|
8
|
+
| Claude Code settings | `~/.claude/settings.json` | `/router:setup` |
|
|
9
|
+
| Router configuration | `~/.claude/router.json` | You |
|
|
10
|
+
| Data and logs | `~/.claude/plugins/data/router-alexei-led-claude-router/` | The gateway |
|
|
11
11
|
|
|
12
12
|
- The hooks give the key to the gateway as `TYPESAFE_API_KEY`. The key is
|
|
13
13
|
never in a file.
|
|
@@ -48,14 +48,14 @@
|
|
|
48
48
|
}
|
|
49
49
|
```
|
|
50
50
|
|
|
51
|
-
| Key | Why
|
|
52
|
-
| -------------------------------------- |
|
|
53
|
-
| `model` | The `[1m]` suffix tells Claude Code that the window is 1M tokens. Claude Code removes the suffix before the request.
|
|
54
|
-
| `env.ANTHROPIC_BASE_URL` | Sends the traffic of Claude Code to the gateway.
|
|
51
|
+
| Key | Why |
|
|
52
|
+
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
|
|
53
|
+
| `model` | The `[1m]` suffix tells Claude Code that the window is 1M tokens. Claude Code removes the suffix before the request. |
|
|
54
|
+
| `env.ANTHROPIC_BASE_URL` | Sends the traffic of Claude Code to the gateway. |
|
|
55
55
|
| `env.ENABLE_TOOL_SEARCH` | Keeps MCP tool schemas deferred. Without it, each request carries all schemas: about 50K tokens with the claude.ai connectors. |
|
|
56
|
-
| `env.CLAUDE_CODE_GATEWAY_HINT_HEADERS` | Claude Code sends the [class of each request](architecture.md#request-classes).
|
|
57
|
-
| `modelPicker` row | Adds `Jev Router (auto)` to `/model`. `behavesAs` names a known model. Without it, Claude Code rejects the alias.
|
|
58
|
-
| `statusLine` (optional) | Runs your status line command, then adds the route of the last turn.
|
|
56
|
+
| `env.CLAUDE_CODE_GATEWAY_HINT_HEADERS` | Claude Code sends the [class of each request](architecture.md#request-classes). |
|
|
57
|
+
| `modelPicker` row | Adds `Jev Router (auto)` to `/model`. `behavesAs` names a known model. Without it, Claude Code rejects the alias. |
|
|
58
|
+
| `statusLine` (optional) | Runs your status line command, then adds the route of the last turn. |
|
|
59
59
|
|
|
60
60
|
Setup writes the file once, as its last step. Restart Claude Code after it.
|
|
61
61
|
Until the restart, the session shows "There's an issue with the selected
|
|
@@ -85,22 +85,22 @@ For example, to run `low` on Sonnet at `high` effort and give Jev more time:
|
|
|
85
85
|
|
|
86
86
|
### gateway
|
|
87
87
|
|
|
88
|
-
| Key | Default | Meaning
|
|
89
|
-
| ---------------- | ------------ |
|
|
90
|
-
| `port` | `43170` | The loopback port of the gateway.
|
|
91
|
-
| `alias` | `jev-router` | The model name that Claude Code sends.
|
|
92
|
-
| `baselineTier` | `low` | The tier when nothing else decides: a new session, or no Jev answer.
|
|
93
|
-
| `auxiliaryTier` | `low` | The tier for side requests, for example session titles.
|
|
94
|
-
| `idleShutdownMs` | `7200000` | The gateway exits after this time without requests. `0` keeps it running.
|
|
88
|
+
| Key | Default | Meaning |
|
|
89
|
+
| ---------------- | ------------ | ------------------------------------------------------------------------- |
|
|
90
|
+
| `port` | `43170` | The loopback port of the gateway. |
|
|
91
|
+
| `alias` | `jev-router` | The model name that Claude Code sends. |
|
|
92
|
+
| `baselineTier` | `low` | The tier when nothing else decides: a new session, or no Jev answer. |
|
|
93
|
+
| `auxiliaryTier` | `low` | The tier for side requests, for example session titles. |
|
|
94
|
+
| `idleShutdownMs` | `7200000` | The gateway exits after this time without requests. `0` keeps it running. |
|
|
95
95
|
|
|
96
96
|
### routes
|
|
97
97
|
|
|
98
|
-
| Tier | Default
|
|
99
|
-
| -------- |
|
|
98
|
+
| Tier | Default |
|
|
99
|
+
| -------- | ---------------------------------------- |
|
|
100
100
|
| `high` | `{ "model": "opus", "effort": "xhigh" }` |
|
|
101
101
|
| `medium` | `{ "model": "opus", "effort": "high" }` |
|
|
102
|
-
| `low` | `{ "model": "sonnet" }`
|
|
103
|
-
| `micro` | `{ "model": "haiku" }`
|
|
102
|
+
| `low` | `{ "model": "sonnet" }` |
|
|
103
|
+
| `micro` | `{ "model": "haiku" }` |
|
|
104
104
|
|
|
105
105
|
`model` is a key of `models`. `effort` is `low`, `medium`, `high`, `xhigh` or
|
|
106
106
|
`max`. Without `effort`, the gateway keeps the effort that Claude Code sent.
|
|
@@ -115,15 +115,15 @@ A test makes sure that they agree.
|
|
|
115
115
|
| `sonnet` | `claude-sonnet-5` | 2 | 10 | 0.2 | 1,000,000 | — | `plan` | all five | `mid-conversation-system` |
|
|
116
116
|
| `haiku` | `claude-haiku-4-5` | 1 | 5 | 0.1 | 200,000 | 64,000 | `plan` | none | none |
|
|
117
117
|
|
|
118
|
-
| Field
|
|
119
|
-
|
|
|
120
|
-
| `id`
|
|
121
|
-
| `input`, `output`, `cacheRead` | List prices in USD per million tokens. `output` is optional.
|
|
122
|
-
| `contextWindow`
|
|
123
|
-
| `maxOutput`
|
|
124
|
-
| `billing`
|
|
125
|
-
| `efforts`
|
|
126
|
-
| `features`
|
|
118
|
+
| Field | Meaning |
|
|
119
|
+
| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
120
|
+
| `id` | The model id that the gateway sends to Anthropic. |
|
|
121
|
+
| `input`, `output`, `cacheRead` | List prices in USD per million tokens. `output` is optional. The `shadow` estimate and the downgrade tax use it. |
|
|
122
|
+
| `contextWindow` | The window in tokens. A turn goes only to a model that holds the context at 80% fill. |
|
|
123
|
+
| `maxOutput` | Optional limit for `max_tokens`. Haiku 4.5 rejects more than 64K. |
|
|
124
|
+
| `billing` | `plan` for the subscription limits. `credits` for models that bill usage credits. |
|
|
125
|
+
| `efforts` | The effort levels that the model accepts. An empty list removes effort and thinking. |
|
|
126
|
+
| `features` | The request features of Claude Code that the model accepts: `mid-conversation-system`, `per-turn-control`, `mid-conversation-tool-changes`. The gateway removes the others ([architecture](architecture.md#system-context)). Without the field: none. |
|
|
127
127
|
|
|
128
128
|
The default prices match `test/fixtures/list-prices.json`, which names its
|
|
129
129
|
source and date. The default `efforts` match `test/fixtures/effort-support.json`,
|
|
@@ -133,50 +133,53 @@ from a probe against the real API. A test fails when they differ.
|
|
|
133
133
|
|
|
134
134
|
The [switching policy](architecture.md#switching-policy) uses these values.
|
|
135
135
|
|
|
136
|
-
| Key
|
|
137
|
-
|
|
|
138
|
-
| `upgradeVotes`
|
|
139
|
-
| `upgradeBase`
|
|
140
|
-
| `upgradeSlope`
|
|
141
|
-
| `upgradePivotUsd`
|
|
142
|
-
| `jumpConfidence`
|
|
143
|
-
| `downgradeVotes`
|
|
144
|
-
| `downgradeMass`
|
|
145
|
-
| `
|
|
146
|
-
| `
|
|
147
|
-
| `
|
|
136
|
+
| Key | Default | Meaning |
|
|
137
|
+
| ----------------------- | ------- | ----------------------------------------------------------------------------------------------------- |
|
|
138
|
+
| `upgradeVotes` | `2` | Consecutive votes above the current tier before an upgrade. |
|
|
139
|
+
| `upgradeBase` | `0.75` | The upgrade bar when a switch costs nothing. |
|
|
140
|
+
| `upgradeSlope` | `0.15` | How much a switching cost can raise the bar: `base + slope × tax / (tax + pivot)`. |
|
|
141
|
+
| `upgradePivotUsd` | `0.5` | The switching cost that adds half of the slope. |
|
|
142
|
+
| `jumpConfidence` | `0.95` | The confidence for a jump of two tiers without the vote delay. |
|
|
143
|
+
| `downgradeVotes` | `2` | Consecutive votes for a lower tier before a downgrade. |
|
|
144
|
+
| `downgradeMass` | `0.9` | The downgrade bar when the candidate's cache is already warm. |
|
|
145
|
+
| `downgradeSlope` | `0.08` | How much a cold candidate's cache write can raise the bar, same formula as upgrade. |
|
|
146
|
+
| `downgradePivotUsd` | `0.5` | The switching cost that adds half of the slope. |
|
|
147
|
+
| `downgradeHorizonTurns` | `5` | The turns of output and read savings that a downgrade's tax nets. Input only without `output` prices. |
|
|
148
|
+
| `continuationMass` | `0.7` | The Jev probability for "continues the task" that keeps the route. |
|
|
149
|
+
| `escalationHoldTurns` | `2` | Turns that the route stays up after an escalation. |
|
|
150
|
+
| `cashCapUsd` | `2` | The cold-write guard: the largest first cache write for a switch to a `credits` model. |
|
|
148
151
|
|
|
149
152
|
### cache, jev, context, log
|
|
150
153
|
|
|
151
|
-
| Key
|
|
152
|
-
|
|
|
153
|
-
| `cache.writeMultiplier`
|
|
154
|
-
| `cache.ttlMs`
|
|
155
|
-
| `cache.warmMarginMs`
|
|
156
|
-
| `jev.endpoint`
|
|
157
|
-
| `jev.model`
|
|
158
|
-
| `jev.timeoutMs`
|
|
159
|
-
| `context.recentTurns`
|
|
160
|
-
| `context.maxTextChars`
|
|
161
|
-
| `log`
|
|
154
|
+
| Key | Default | Meaning |
|
|
155
|
+
| ----------------------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------- |
|
|
156
|
+
| `cache.writeMultiplier` | `{ "5m": 1.25, "1h": 2 }` | The price of a cache write, as a multiple of `input`. |
|
|
157
|
+
| `cache.ttlMs` | `{ "5m": 300000, "1h": 3600000 }` | The lifetime of each cache TTL. |
|
|
158
|
+
| `cache.warmMarginMs` | `30000` | A cache counts as cold this long before it expires. |
|
|
159
|
+
| `jev.endpoint` | `https://api.typesafe.ai/v1/systemone` | The Jev API. |
|
|
160
|
+
| `jev.model` | `jev-1.13.0` | The Jev model. |
|
|
161
|
+
| `jev.timeoutMs` | `1500` | The total time for one Jev answer, one retry included. |
|
|
162
|
+
| `context.recentTurns` | `6` | The number of recent turns that Jev receives. |
|
|
163
|
+
| `context.maxTextChars` | `1200` | The characters of the prompt and of each turn that Jev receives. A longer text keeps its start and end. |
|
|
164
|
+
| `log` | `true` | Write `decisions.jsonl`. |
|
|
162
165
|
|
|
163
166
|
## Environment variables
|
|
164
167
|
|
|
165
|
-
| Variable | Effect
|
|
166
|
-
| -------------------- |
|
|
167
|
-
| `TYPESAFE_API_KEY` | The Jev key. The hooks set it from the plugin option.
|
|
168
|
+
| Variable | Effect |
|
|
169
|
+
| -------------------- | ------------------------------------------------------------------------------------------------------------ |
|
|
170
|
+
| `TYPESAFE_API_KEY` | The Jev key. The hooks set it from the plugin option. |
|
|
168
171
|
| `ROUTER_CONFIG` | Another path for the router configuration, under `~/.claude/` only. Another path is ignored, with a warning. |
|
|
169
|
-
| `CLAUDE_PLUGIN_DATA` | The data directory. Claude Code sets it for the plugin hooks.
|
|
172
|
+
| `CLAUDE_PLUGIN_DATA` | The data directory. Claude Code sets it for the plugin hooks. |
|
|
170
173
|
|
|
171
174
|
## Command-line flags
|
|
172
175
|
|
|
173
176
|
A project cannot change these: they are part of the command, not of the
|
|
174
177
|
environment.
|
|
175
178
|
|
|
176
|
-
| Flag | Script
|
|
177
|
-
| --------------------- |
|
|
178
|
-
| `--config <path>` | `gateway.mjs`, `ensure-gateway.mjs`, `status.mjs` | The router configuration, at any path.
|
|
179
|
-
| `--force-tier <tier>` | `gateway.mjs`
|
|
179
|
+
| Flag | Script | Effect |
|
|
180
|
+
| --------------------- | ------------------------------------------------- | ------------------------------------------------------------------------ |
|
|
181
|
+
| `--config <path>` | `gateway.mjs`, `ensure-gateway.mjs`, `status.mjs` | The router configuration, at any path. |
|
|
182
|
+
| `--force-tier <tier>` | `gateway.mjs` | `micro`, `low`, `medium` or `high`. Skips Jev and the policy. For tests. |
|
|
180
183
|
|
|
181
184
|
The flags are for a gateway that you start by hand. The hooks and the status
|
|
182
185
|
line do not pass them. To use another file with the hooks, keep it under
|
|
@@ -193,8 +196,8 @@ CLAUDE_PLUGIN_DATA=~/.claude/plugins/data/router-alexei-led-claude-router \
|
|
|
193
196
|
|
|
194
197
|
## Data directory
|
|
195
198
|
|
|
196
|
-
| File | Content
|
|
197
|
-
| ----------------- |
|
|
198
|
-
| `decisions.jsonl` | One line for each decision and each response. | Moves to `.1` above 20 MB.
|
|
199
|
-
| `gateway.log` | Gateway events with timestamps.
|
|
200
|
-
| `sessions/` | The routing memory of each session.
|
|
199
|
+
| File | Content | Retention |
|
|
200
|
+
| ----------------- | --------------------------------------------- | -------------------------------------------- |
|
|
201
|
+
| `decisions.jsonl` | One line for each decision and each response. | Moves to `.1` above 20 MB. |
|
|
202
|
+
| `gateway.log` | Gateway events with timestamps. | Moves to `.1` above 20 MB, at gateway start. |
|
|
203
|
+
| `sessions/` | The routing memory of each session. | Removed after 30 days without use. |
|
package/docs/user-guide.md
CHANGED
|
@@ -143,7 +143,7 @@ log has no prompt text.
|
|
|
143
143
|
| `reason` | The rule that decided. Tool calls show `tool-continuation`. `error`: routing failed and the default tier served. |
|
|
144
144
|
| `advice` | The Jev probabilities for each tier, and for "continues the task". |
|
|
145
145
|
| `estimate` | The numbers of a vote: confidence, bar, switching tax, cache state. |
|
|
146
|
-
| `shadow` | The cost of the Jev choice against the current route. The
|
|
146
|
+
| `shadow` | The cost of the Jev choice against the current route. The downgrade tax uses the same arithmetic. |
|
|
147
147
|
| `observed` | The usage of the response: model, effort, context tokens, cache reads, output. |
|
|
148
148
|
| `failed` | Anthropic answered a routed turn with an error: status, tier, model, effort. |
|
|
149
149
|
| `historyBreak` | A compaction or a rewind. The votes and the cache estimates reset. |
|
package/lib/config.mjs
CHANGED
|
@@ -24,7 +24,7 @@ export const DEFAULTS = {
|
|
|
24
24
|
},
|
|
25
25
|
// `id` is sent upstream verbatim. List prices in USD per million tokens; `cacheRead` is absolute,
|
|
26
26
|
// not a multiplier (Opus 5.5 reads at 0.05x input, the rest at the standard 0.1x). A price change must also change
|
|
27
|
-
// test/fixtures/list-prices.json, with its source and date. `output` feeds
|
|
27
|
+
// test/fixtures/list-prices.json, with its source and date. `output` feeds the shadow estimate and the downgrade tax.
|
|
28
28
|
// `efforts` lists what the model accepts; an empty list means no effort field and no adaptive thinking. A change
|
|
29
29
|
// must also change test/fixtures/effort-support.json, from a new probe against the real API.
|
|
30
30
|
// `features` lists the FEATURES the model accepts; the gateway removes the others from the request. A model
|
|
@@ -77,6 +77,9 @@ export const DEFAULTS = {
|
|
|
77
77
|
jumpConfidence: 0.95,
|
|
78
78
|
downgradeVotes: 2,
|
|
79
79
|
downgradeMass: 0.9,
|
|
80
|
+
downgradeSlope: 0.08, // a cold candidate raises the bar toward ~0.98, same shape as the upgrade bar
|
|
81
|
+
downgradePivotUsd: 0.5,
|
|
82
|
+
downgradeHorizonTurns: 5, // turns whose output and read savings offset a downgrade's cache write
|
|
80
83
|
continuationMass: 0.7,
|
|
81
84
|
escalationHoldTurns: 2,
|
|
82
85
|
// Ceiling on a cold cache write to a `credits` model. No default model bills credits, so this is
|
|
@@ -211,12 +214,14 @@ function validate(config) {
|
|
|
211
214
|
for (const field of ['upgradeBase', 'jumpConfidence', 'downgradeMass', 'continuationMass']) {
|
|
212
215
|
if (!(isNumber(p[field], 0) && p[field] <= 1)) throw new Error(`policy.${field} must be between 0 and 1`);
|
|
213
216
|
}
|
|
214
|
-
for (const field of ['upgradeVotes', 'downgradeVotes', 'escalationHoldTurns']) {
|
|
217
|
+
for (const field of ['upgradeVotes', 'downgradeVotes', 'downgradeHorizonTurns', 'escalationHoldTurns']) {
|
|
215
218
|
if (!isNumber(p[field], 1, { integer: true })) throw new Error(`policy.${field} must be a positive integer`);
|
|
216
219
|
}
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
+
// The pivot divides: tax / (tax + pivot). Zero makes a free switch NaN and silently stops that switch.
|
|
221
|
+
for (const side of ['upgrade', 'downgrade']) {
|
|
222
|
+
if (!isNumber(p[`${side}Slope`], 0)) throw new Error(`policy.${side}Slope must be a non-negative number`);
|
|
223
|
+
if (!isNumber(p[`${side}PivotUsd`], 0, { above: true })) throw new Error(`policy.${side}PivotUsd must be positive`);
|
|
224
|
+
}
|
|
220
225
|
if (!isNumber(p.cashCapUsd, 0)) throw new Error('policy.cashCapUsd must be a non-negative number');
|
|
221
226
|
const { cache, jev, context } = config;
|
|
222
227
|
for (const map of ['writeMultiplier', 'ttlMs']) {
|
package/lib/cost.mjs
CHANGED
|
@@ -62,9 +62,9 @@ export function switchingTaxUsd(config, candidateTier, incumbentTier, facts, now
|
|
|
62
62
|
);
|
|
63
63
|
}
|
|
64
64
|
|
|
65
|
-
// Shadow estimate for the decision log; the
|
|
66
|
-
// list-price equivalent, not a charge. Output uses the last observed output size for both routes, although
|
|
67
|
-
// changes how much a model writes.
|
|
65
|
+
// Shadow estimate for the decision log; the downgrade tax nets it over a horizon. At list prices, so for `plan` models
|
|
66
|
+
// it is a list-price equivalent, not a charge. Output uses the last observed output size for both routes, although
|
|
67
|
+
// effort changes how much a model writes.
|
|
68
68
|
// - nextTurnUsd: candidate minus incumbent for the next request, input at the current cache state plus output.
|
|
69
69
|
// - laterTurnUsd: the same difference for each later turn, once both caches are warm.
|
|
70
70
|
// - paybackTurns: 0 when the switch is cheaper at once, n when later turns repay it after n turns, null when never.
|
|
@@ -82,3 +82,12 @@ export function shadowEconomics(config, candidateTier, incumbentTier, facts, now
|
|
|
82
82
|
else if (nextTurnUsd > 0 && laterTurnUsd < 0) paybackTurns = Math.ceil(nextTurnUsd / -laterTurnUsd);
|
|
83
83
|
return { nextTurnUsd, laterTurnUsd, paybackTurns, outputTokens };
|
|
84
84
|
}
|
|
85
|
+
|
|
86
|
+
// A downgrade's tax over `turns` turns: the next one plus the later ones, output included. A missing or zero output
|
|
87
|
+
// price is unknown, not free, so the tax then falls back to the next request's input alone.
|
|
88
|
+
export function downgradeTaxUsd(config, candidateTier, incumbentTier, facts, now, turns) {
|
|
89
|
+
const priced = [candidateTier, incumbentTier].every((tier) => modelOf(config, tier).output > 0);
|
|
90
|
+
if (!priced) return Math.max(0, switchingTaxUsd(config, candidateTier, incumbentTier, facts, now));
|
|
91
|
+
const { nextTurnUsd, laterTurnUsd } = shadowEconomics(config, candidateTier, incumbentTier, facts, now);
|
|
92
|
+
return Math.max(0, nextTurnUsd + (turns - 1) * laterTurnUsd);
|
|
93
|
+
}
|
package/lib/policy.mjs
CHANGED
|
@@ -1,6 +1,14 @@
|
|
|
1
1
|
// Switching policy v0 (docs/architecture.md): stickiness, escalation floor, cost-gated votes.
|
|
2
2
|
import { rank, TIERS } from './config.mjs';
|
|
3
|
-
import {
|
|
3
|
+
import {
|
|
4
|
+
cacheState,
|
|
5
|
+
coldWriteUsd,
|
|
6
|
+
downgradeTaxUsd,
|
|
7
|
+
isWarm,
|
|
8
|
+
nextContextTokens,
|
|
9
|
+
routeCacheKey,
|
|
10
|
+
switchingTaxUsd,
|
|
11
|
+
} from './cost.mjs';
|
|
4
12
|
|
|
5
13
|
export function initialState() {
|
|
6
14
|
return { turn: 0, votes: [], holdUntilTurn: 0, escalatedSignature: null };
|
|
@@ -48,8 +56,13 @@ export function decide({ config, facts, advice, state, baseline, now }) {
|
|
|
48
56
|
|
|
49
57
|
const support = massAtOrBelow(advice.probabilities, choice);
|
|
50
58
|
const streak = trailing(next.votes, (v) => rank(v.tier) <= rank(choice));
|
|
51
|
-
|
|
52
|
-
|
|
59
|
+
// A downgrade is not always cheaper this turn: a cold candidate next to a warm incumbent can pay a bigger cache
|
|
60
|
+
// write than it saves. The bar rises with that cost, net of what the next turns save, the way the upgrade bar does.
|
|
61
|
+
const tax = downgradeTaxUsd(config, choice, incumbent, facts, now, config.policy.downgradeHorizonTurns);
|
|
62
|
+
const threshold =
|
|
63
|
+
config.policy.downgradeMass + config.policy.downgradeSlope * (tax / (tax + config.policy.downgradePivotUsd));
|
|
64
|
+
const estimate = { downgradeMass: support, streak, taxUsd: tax, threshold };
|
|
65
|
+
if (support >= threshold && streak >= config.policy.downgradeVotes)
|
|
53
66
|
return result(choice, 'downgrade', next, estimate);
|
|
54
67
|
return stay('downgrade-pending', estimate);
|
|
55
68
|
}
|
package/lib/router.mjs
CHANGED
|
@@ -185,7 +185,8 @@ export class Router {
|
|
|
185
185
|
}
|
|
186
186
|
const now = this.now();
|
|
187
187
|
const decision = decide({ config: this.config, facts, advice, state, baseline: gateway.baselineTier, now });
|
|
188
|
-
// Shadow only: what following Jev's choice instead of staying would cost at list prices. The policy
|
|
188
|
+
// Shadow only: what following Jev's choice instead of staying would cost at list prices. The policy does not read
|
|
189
|
+
// this record; the downgrade tax computes its own from the same function.
|
|
189
190
|
const incumbent = TIERS.includes(facts.lastRoute) ? facts.lastRoute : gateway.baselineTier;
|
|
190
191
|
const shadow =
|
|
191
192
|
TIERS.includes(advice?.choice) && advice.choice !== incumbent
|
package/package.json
CHANGED