@alexeiled/claude-router 1.21.2 → 1.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +19 -0
- package/docs/activity-routing.md +28 -13
- package/docs/architecture.md +4 -4
- package/docs/configuration.md +1 -1
- package/docs/evaluation.md +1 -1
- package/docs/native-router.md +1 -1
- package/docs/user-guide.md +6 -4
- package/hooks/native-router.mjs +4 -0
- package/lib/decisions.mjs +35 -3
- package/lib/route.mjs +12 -2
- package/package.json +1 -1
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "router",
|
|
3
3
|
"displayName": "Router",
|
|
4
|
-
"version": "1.
|
|
4
|
+
"version": "1.22.0",
|
|
5
5
|
"description": "Native model and effort routing within your provider. Anthropic, DeepSeek, z.ai, MiniMax, MiMo and Kimi defaults, custom provider profiles, classifier advice and one model per subagent.",
|
|
6
6
|
"author": {
|
|
7
7
|
"name": "Alexei Ledenev",
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,24 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.22.0 (2026-11-11)
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
|
|
7
|
+
- A decision row keeps the probability mass the policy compared and the bar it compared it with (`mass`, `bar`), for a tier move and an activity move up, taken or held. `/router decisions` shows them after the reason: `(upgrade-pending 52% vs bar 79%)`. A held move can show a mass above its bar: it waits for a second vote or a checkpoint. A `jump`, a move on the turn's own tool calls and a context or availability override show neither. Rows saved before this change still read.
|
|
8
|
+
|
|
9
|
+
### Changed
|
|
10
|
+
|
|
11
|
+
- The docs say what transcripts show about an effort change on the same model: the cache was kept in 102 of 104 changes, which supports pricing an effort-only move as no cache write. On Claude Code 2.1.322 an effort changed inside a turn was not sent in 6 of 6 cases, so the band and the decision record can name an effort that did not run. Counts and limits: [What we do not know yet](docs/activity-routing.md#what-we-do-not-know-yet).
|
|
12
|
+
- The activity routing guide now says Sonnet 5.5 reads cached context at $0.10 per million tokens, half of Opus 5.5's $0.20, as it has since 1.21.0. Its 350K-token examples are recomputed: a move from Opus to Sonnet saves about $0.07 a request and takes about 22 requests to repay its cache write, and Sonnet to Haiku repays after about 9.
|
|
13
|
+
|
|
14
|
+
### Fixed
|
|
15
|
+
|
|
16
|
+
- Repeated tool failures no longer move a session to a weaker model when routing is turned on over a model no route names, such as Fable 5.2 on the defaults. That model was counted as the start tier's route, so failures on Fable at xhigh "escalated" to Haiku at high. The session's model is now the one in use: it stays unless a stronger route exists, and otherwise leaves for the start tier's route through the same checks as a routing-mode change. A model chosen with `/model` after routed turns still resumes the last routed route.
|
|
17
|
+
|
|
18
|
+
### Upgrade
|
|
19
|
+
|
|
20
|
+
Restart Claude Code after updating.
|
|
21
|
+
|
|
3
22
|
## 1.21.2 (2026-11-10)
|
|
4
23
|
|
|
5
24
|
### Added
|
package/docs/activity-routing.md
CHANGED
|
@@ -64,7 +64,8 @@ Why each row:
|
|
|
64
64
|
|
|
65
65
|
The matrix uses five routes on three models. Opus 5.5, Sonnet 5.5 and Haiku 5.5 keep one cache across efforts, so the
|
|
66
66
|
five routes use three caches, one more (Sonnet) than the base routes alone. On a provider that caches each effort apart,
|
|
67
|
-
each (model, effort) pair is its own cache.
|
|
67
|
+
each (model, effort) pair is its own cache. [What we do not know yet](#what-we-do-not-know-yet) has what transcripts
|
|
68
|
+
show about that shared cache, and its limits.
|
|
68
69
|
|
|
69
70
|
## Which model fits which work
|
|
70
71
|
|
|
@@ -76,7 +77,7 @@ each (model, effort) pair is its own cache.
|
|
|
76
77
|
- Avoid `xhigh` in multi-turn chats. Anthropic reports that Haiku then sometimes puts the whole answer in its thinking
|
|
77
78
|
and shows no reply.
|
|
78
79
|
- Above 100,000 tokens of context, Anthropic charges 5 times Haiku's list price. Even then it costs at most a quarter
|
|
79
|
-
of what
|
|
80
|
+
of what Opus costs per token, and against Sonnet a quarter for input and output and half for cache reads.
|
|
80
81
|
|
|
81
82
|
**Sonnet 5.5**: small, well-defined code changes.
|
|
82
83
|
|
|
@@ -84,8 +85,8 @@ each (model, effort) pair is its own cache.
|
|
|
84
85
|
on FrontierCode Main.
|
|
85
86
|
- At `medium` it scored about 37% and 39.2%, below Haiku at `high` on FrontierCode Main. That is why the defaults use
|
|
86
87
|
`high`.
|
|
87
|
-
-
|
|
88
|
-
Opus.
|
|
88
|
+
- Every Sonnet rate is half of Opus's: it reads cached context at $0.10 per million tokens against $0.20. In long
|
|
89
|
+
sessions a request costs about half of one on Opus.
|
|
89
90
|
|
|
90
91
|
**Opus 5.5**: ordinary and hard engineering, design, review, and research answers.
|
|
91
92
|
|
|
@@ -101,18 +102,19 @@ These are Anthropic's results, mostly at `max` effort and in other harnesses. Th
|
|
|
101
102
|
Each model keeps its own cache of the conversation. A model change writes the whole conversation into the new model's
|
|
102
103
|
cache. In the sample below Claude Code always wrote a one-hour cache, which costs twice the input price.
|
|
103
104
|
|
|
104
|
-
At 350,000 tokens of context, the typical size in that sample, at list prices
|
|
105
|
+
At 350,000 tokens of context, the typical size in that sample, at list prices, for a request that reads the whole
|
|
106
|
+
context from the cache and writes about 3,000 output tokens:
|
|
105
107
|
|
|
106
108
|
| Cost | Haiku 5.5 | Sonnet 5.5 | Opus 5.5 |
|
|
107
109
|
| ---------------------------- | ----------- | ----------- | ----------- |
|
|
108
110
|
| Write the whole context once | $0.35 | $1.40 | $2.80 |
|
|
109
|
-
| One request on a warm cache | about $0.03 | about $0.
|
|
111
|
+
| One request on a warm cache | about $0.03 | about $0.07 | about $0.13 |
|
|
110
112
|
|
|
111
|
-
- A move from Opus to Haiku pays back after about 4 requests, from Sonnet after about
|
|
112
|
-
requests at the median, so the `ops` move pays back. Question-and-answer runs lasted 1 to 3,
|
|
113
|
-
does not.
|
|
114
|
-
- A move from Opus to Sonnet saves about $0.
|
|
115
|
-
the defaults do not move `medium` or `high` work to Sonnet.
|
|
113
|
+
- A move from Opus to Haiku pays back after about 4 requests, from Sonnet after about 9. Command runs lasted 6 to 11
|
|
114
|
+
requests at the median, so the `ops` move at `medium`, from Opus, pays back. Question-and-answer runs lasted 1 to 3,
|
|
115
|
+
so moving `explore` does not.
|
|
116
|
+
- A move from Opus to Sonnet saves about $0.07 per request and costs $1.40 to start: about 22 requests to pay back.
|
|
117
|
+
It rarely does. That is why the defaults do not move `medium` or `high` work to Sonnet.
|
|
116
118
|
|
|
117
119
|
The router does this sum for every move with the prices in `router.json`. A move to a cheaper route inside a tier is
|
|
118
120
|
taken only when it pays back within the payback horizon. A move down a tier weighs the same cost against the
|
|
@@ -248,8 +250,21 @@ Limits:
|
|
|
248
250
|
- How well the classifier labels real turns. On 70 synthetic prompts, Jev and OpenAI labelled all 70 right; see
|
|
249
251
|
[Evaluation](evaluation.md#activity-probe-set). The **Agreement** lines, on the
|
|
250
252
|
Classifier tab for this session and across sessions, check your own turns against the tools they used.
|
|
251
|
-
- Whether an effort change on the same model keeps the cache.
|
|
252
|
-
|
|
253
|
+
- Whether an effort change on the same model always keeps the cache. Claude Code's prompt-caching docs say it does on
|
|
254
|
+
the 5.5 models and Fable 5.1 with an API key or a subscription, and the router prices it so. On 2026-11-11 we counted
|
|
255
|
+
104 such changes with under 270 seconds between requests in 30 days of Claude Code transcripts, main thread only, up
|
|
256
|
+
or down, inside a turn or at a turn start. The request after the change still read the cache in 102 of 104. The
|
|
257
|
+
median share of the prompt written to the cache after a change was 0.4 to 0.5% on Opus and Haiku and 0.1 to 0.7% on
|
|
258
|
+
Sonnet; the same models without a change wrote 0.2 to 0.4%. Limits: these are transcripts, not a controlled test.
|
|
259
|
+
Opus had 29 changes up and 33 down and Haiku 18 up and 15 down, but Sonnet had 7 up and 1 down, and Fable 1 up,
|
|
260
|
+
which is too small to count. Two changes lost the cache, and we did not look into why.
|
|
261
|
+
- Whether Claude Code sends an effort that the router changes inside a turn. The router changes it in the same
|
|
262
|
+
`turn.step` hook that changes the model. On Claude Code 2.1.322, in transcripts of 2026-11-10 and 2026-11-11, the
|
|
263
|
+
router did so in 6 cases: all 6 requests still carried the turn's first effort, while the changed model was sent in
|
|
264
|
+
every case. At the start of a turn the router's effort was sent in 145 of 145 decisions. Transcripts from older
|
|
265
|
+
versions, 2.1.296 to 2.1.319, show the effort changing between tool results inside a turn. We have not reproduced this
|
|
266
|
+
in isolation. So the band and the decision record can name an effort that did not run; the request parameters in the
|
|
267
|
+
transcript show what ran.
|
|
253
268
|
- How many requests a turn needs on each model.
|
|
254
269
|
|
|
255
270
|
## Sources
|
package/docs/architecture.md
CHANGED
|
@@ -98,9 +98,9 @@ The router decides on a _cell_: a tier and an activity. The route is `{ ...route
|
|
|
98
98
|
|
|
99
99
|
**Observed move.** On each later step of the turn, `observedMove` reads the turn's tool calls since its prompt, or since the transcript snapshot of the last review applied inside the turn (`loop.observedFrom`), since the review already judged the earlier ones, (`observedActivity`: `code` after an edit to a non-doc file, `docs` after edits to doc files only, else `ops`, `read` or `talk`). When that activity's route at the decision's tier is stronger than the running one, the remaining steps take it, with reason `activity-up` and `lastActivity` set, so the next turn starts there. The tier is fitted to the context first, so the gates price the route that will run. It goes through `lateralMove` with probability 1: the tools are the evidence, so the switching tax never holds it back (the upgrade bar tops out near 0.9), and only the cash gate can. The turn's reply label (`turnActivities.label`) follows the move; the classifier's answer stays as recorded. It never moves down, and skips pinned turns and `off`. `canMoveUp` skips the transcript read when no activity's route at the tier is stronger, so a step on the strongest route costs nothing. A move to a smaller window waits for a measured context, as in `routeTurn`. Context fit and model availability then apply as for any continuation.
|
|
100
100
|
|
|
101
|
-
**Review inside a turn.** Every `reviewEveryRequests` requests after the last classifier answer (`loop.reviewedAt`, counted in `loop.requests`), `reviewMidTurn` reads the transcript and asks the active classifier in the background, with the turn's prompt and recent dialogue and `facts.midTurn` set. The classifier reads text only: the dialogue holds user and assistant text, not tool calls, results or edits. The policy reads those itself (`observedMove`, repeated failures, `completedSince`). The step does not wait. The first step after the answer arrives runs `chooseRoute` on it, before the observed move, with only the advice taken from the review: the checkpoint, context and cache facts are read at that step. Tier and activity move by the same rules as a continuation prompt: up on votes, down only past `downAllowed`. A pinned turn, a failed read or ask, an answer without advice, and an answer about a replaced history (another `generation`) change nothing; turn completion and session end abort a review in flight. `observeResponse` counts each request and records the request and message count at which the
|
|
101
|
+
**Review inside a turn.** Every `reviewEveryRequests` requests after the last classifier answer (`loop.reviewedAt`, counted in `loop.requests`), `reviewMidTurn` reads the transcript and asks the active classifier in the background, with the turn's prompt and recent dialogue and `facts.midTurn` set. The classifier reads text only: the dialogue holds user and assistant text, not tool calls, results or edits. The policy reads those itself (`observedMove`, repeated failures, `completedSince`). The step does not wait. The first step after the answer arrives runs `chooseRoute` on it, before the observed move, with only the advice taken from the review: the checkpoint, context and cache facts are read at that step. Tier and activity move by the same rules as a continuation prompt: up on votes, down only past `downAllowed`. A pinned turn, a failed read or ask, an answer without advice, and an answer about a replaced history (another `generation`) change nothing; turn completion and session end abort a review in flight. `observeResponse` counts each request and records the request and message count at which the requested (model, effort) last changed (`switchedAt`), which the dwell and the verification window start from. A failed transcript read keeps the running route. A verification is a Bash command whose status is the check's: a checker (`pytest`, `tsc`, `eslint`, …) or a launcher whose subcommand names a check (`npm test`, `npm run lint`, `go test`, `make lint`, `uv run pytest`), alone or in an `&&` list. A pipeline, `;`, `||` or background job does not count, since its status can hide a failed check; neither do a `--version` or `--help` run, an option before the subcommand (`npm --prefix test install`), package names, paths and file names, nor a command Claude Code sent to the background, by request or after a timeout. The first response a loop serves also sets `switchedAt`, so checks from before the router chose a route do not count. When compaction or a rewind shortens the history, `prepareLoop` moves `switchedAt.messageCount` and `observedFrom` back to its end and keeps the request-based dwell.
|
|
102
102
|
|
|
103
|
-
At session start the incumbent is the session's model. `cellForModel` looks for it in the base routes, baseline first, and only with `activityRouting: on` in the activity overrides too, so a session that starts on an override route keeps its cell until the classifier says otherwise. The start mode uses the same lookup: with `on`, a start model that only an override uses (Sonnet 5.5 on the defaults) starts with routing on.
|
|
103
|
+
At session start the incumbent is the session's model. `cellForModel` looks for it in the base routes, baseline first, and only with `activityRouting: on` in the activity overrides too, so a session that starts on an override route keeps its cell until the classifier says otherwise. The start mode uses the same lookup: with `on`, a start model that only an override uses (Sonnet 5.5 on the defaults) starts with routing on. A session model with no cell (Fable 5.2 on the defaults, turned on with `/router auto`) starts at `baselineTier`, but `emptyLoop` records the model itself, at the session's effort, as the route in use (`ownRoute`). So escalation, the switching tax and cache state compare against what ran, and the session leaves it through a gate: failure escalation to a stronger route, a tier switch the classifier supports (a jump or the upgrade votes, which carry no dwell), or `lateralMove` for a cheaper route, which waits for the dwell and a checkpoint on the same task. Until the first routed decision, a manual reply replaces that record with the model that served it (a model with a cell records none), so `/model` before `/router auto` starts from the model in use. After routed turns, `/model` then `/router auto` resumes the last routed route, as for any model.
|
|
104
104
|
|
|
105
105
|
## Routing modes
|
|
106
106
|
|
|
@@ -114,7 +114,7 @@ The turn-start decision (`decideTurn`) writes `router decision {mode, modeSource
|
|
|
114
114
|
|
|
115
115
|
**Prompt origin.** `turn.start` carries the prompt text but not its origin; `prompt.submit` carries `origin` (`PromptOrigin`), stamped by the engine. The hook keeps the kind (`originKind` in `lib/decisions.mjs`). An idle submission starts its turn inside `next(e)`, so that turn takes it; a submission queued behind a running turn waits (`queueOrigin`, at most 16 for at most 10 minutes) and names the later turn that starts with its text (`takeOrigin`), or expires when it was delivered into the running turn. Unmatched turns are `unknown` and route as before. A progress origin (`task-notification`, `observer`, `observer-activity`, `auto-continuation`) sets `facts.progress`, so the turn is the same task (`sameTaskOf`), and `progressInput` asks the classifier about the session's active goal, with the notification as the newest dialogue line. The goal is the last prompt from a goal origin (`composer`, `peer`, `plugin`, …), held in memory for the session only: after a resume or a reload a notification is asked about as it reads. `unclassified` and kinds a later engine adds route as before and leave the goal.
|
|
116
116
|
|
|
117
|
-
**Decision record.** Each routed turn collects rows (`decisionRow`): its start, each applied review, each observed move, at most 10 per turn (`addTurnRow`), and at `turn.complete` an end row (`endRow`) with the turn's tool bucket, routed requests, priced requests and unpriced ones by cause. `flushDecisions` adds them to the session's own `decisions:v1:<session id>` record (newest 60 rows) under the reset marker, like the usage records. Session start deletes other sessions' records past 30 days or beyond the 20 newest (`prunedDecisions`), each read again before its delete. Rows hold labels, shares and counts, never prompt, tool or path text. `/router decisions [n]` prints them; **Delete history** deletes them. `experiments/mod-router/scripts/trace-eval.mjs` replays the gateway trace once per mode at tier level; it cannot replay moves inside a tier.
|
|
117
|
+
**Decision record.** Each routed turn collects rows (`decisionRow`): its start, each applied review, each observed move, at most 10 per turn (`addTurnRow`), and at `turn.complete` an end row (`endRow`) with the turn's tool bucket, routed requests, priced requests and unpriced ones by cause. `flushDecisions` adds them to the session's own `decisions:v1:<session id>` record (newest 60 rows) under the reset marker, like the usage records. Session start deletes other sessions' records past 30 days or beyond the 20 newest (`prunedDecisions`), each read again before its delete. A start or review row whose reason is `upgrade`, `upgrade-pending`, `downgrade`, `downgrade-pending`, `downgrade-held`, `activity-up` or `activity-pending` also keeps the estimate's mass and threshold as `mass` and `bar`, rounded to two decimals, when the estimate has both. The mass is the one above the incumbent for a move up; for a move down the one at or below the target, and on `downgrade-pending` at or below the classifier's choice (`decide` reports that estimate, though a tier between the choice and the incumbent can be closer to its bar); for an activity move up the label's share, or 1 when the move needs no label (back to the tier's route, a route the config changed). A pending or held row can show a mass at or above its bar: the values are rounded, and the move can wait for its vote streak or its checkpoint. Other rows keep neither: a `jump` is weighed against `jumpConfidence`, a context or availability override keeps the estimate of the decision it replaced, an observed move takes no classifier mass, and a hold, a cash gate and a cheaper activity route judged on its cache write compare none. A lateral outcome replaces the reason and the estimate of a tier stay (`decideCell`), so a tier move held on a turn that also weighed an activity route shows the activity's reason and numbers, not its own. Rows hold labels, shares and counts, never prompt, tool or path text. A row names the route the router chose, not the request that ran: on Claude Code 2.1.322 an effort changed inside a turn was not sent in the 6 cases found (see [What we do not know yet](activity-routing.md#what-we-do-not-know-yet)). `/router decisions [n]` prints them; **Delete history** deletes them. `experiments/mod-router/scripts/trace-eval.mjs` replays the gateway trace once per mode at tier level; it cannot replay moves inside a tier.
|
|
118
118
|
|
|
119
119
|
## State and controls
|
|
120
120
|
|
|
@@ -172,7 +172,7 @@ Execution profile selection is fixed at session start by `CLAUDE_ROUTER_PROVIDER
|
|
|
172
172
|
|
|
173
173
|
The explicit-cache rules below describe Anthropic. Automatic prefix-cache providers instead use observed hit/miss input rates, no write premium and no guaranteed TTL. Their future price bounds allow a full miss; counterfactual cache comparisons are omitted unless the observed route is identical. Same-model effort downgrades skip input-only payback, which cannot value generated thinking-token changes, but retain the vote, dwell and cold-input guards. Their cache observations remain effort-specific until sharing is established. `disabled` cache mode uses the uncached input rate throughout.
|
|
174
174
|
|
|
175
|
-
Cache identity uses the response model ID with the `[1m]` context hint removed, plus the effective effort on models that keep one cache per effort. Opus 5.5, Sonnet 5.5, Haiku 5.5, Fable 5.1 and Fable 5.2 keep one cache across efforts with an API key or a Claude subscription (code.claude.com/docs/en/prompt-caching, "Changing effort level"), so an effort-only move costs no cache write there. The hook gives every effort its own cache when the session runs on Amazon Bedrock, Google Cloud, Microsoft Foundry, a custom `ANTHROPIC_BASE_URL`, or with `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS` (`effortSplitsCache`). A HIPAA configuration is not visible to a Mod; there the router prices effort moves as free. The Haiku 4.5 dated snapshot has an explicit mapping; Haiku 5.5 has no dated ID. An unrecognized substituted model gets no cache credit for the requested route.
|
|
175
|
+
Cache identity uses the response model ID with the `[1m]` context hint removed, plus the effective effort on models that keep one cache per effort. Opus 5.5, Sonnet 5.5, Haiku 5.5, Fable 5.1 and Fable 5.2 keep one cache across efforts with an API key or a Claude subscription (code.claude.com/docs/en/prompt-caching, "Changing effort level"), so an effort-only move costs no cache write there. Transcripts show the cache kept in 102 of 104 same-model effort changes, with limits; see [What we do not know yet](activity-routing.md#what-we-do-not-know-yet). The hook gives every effort its own cache when the session runs on Amazon Bedrock, Google Cloud, Microsoft Foundry, a custom `ANTHROPIC_BASE_URL`, or with `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS` (`effortSplitsCache`). A HIPAA configuration is not visible to a Mod; there the router prices effort moves as free. The Haiku 4.5 dated snapshot has an explicit mapping; Haiku 5.5 has no dated ID. An unrecognized substituted model gets no cache credit for the requested route.
|
|
176
176
|
|
|
177
177
|
The cached prefix is observed `cache_read + cache_creation`. Output tokens count only toward the next context size. Freshness uses the configured five-minute lifetime with its safety margin. Claude's usage counters do not prove one-hour cache lifetime.
|
|
178
178
|
|
package/docs/configuration.md
CHANGED
|
@@ -377,7 +377,7 @@ The pane writes a subset of these settings. The **Routes** and **Policy** tabs s
|
|
|
377
377
|
|
|
378
378
|
A project can set the routing mode, and only that, in `.claude/router.json` in the directory Claude Code runs in: `{ "mode": "cost" }`. Any other content is ignored with a notice; it never makes routing unavailable. A session's `/router mode` beats it, and it beats the profile's `mode`. See [Routing modes](routing-modes.md).
|
|
379
379
|
|
|
380
|
-
Opus 5.5, Sonnet 5.5, Haiku 5.5, Fable 5.1 and Fable 5.2 keep one prompt cache across effort levels, so the policy prices an effort-only move, such as Opus at `medium` to Opus at `high`, as no cache write. On Amazon Bedrock, Google Cloud, Microsoft Foundry, a custom `ANTHROPIC_BASE_URL`, or with `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS`, each effort has its own cache and the move pays its write. A HIPAA configuration cannot be detected; the router then prices those moves as free.
|
|
380
|
+
Opus 5.5, Sonnet 5.5, Haiku 5.5, Fable 5.1 and Fable 5.2 keep one prompt cache across effort levels, so the policy prices an effort-only move, such as Opus at `medium` to Opus at `high`, as no cache write. Transcripts support that price, with limits; see [What we do not know yet](activity-routing.md#what-we-do-not-know-yet). On Amazon Bedrock, Google Cloud, Microsoft Foundry, a custom `ANTHROPIC_BASE_URL`, or with `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS`, each effort has its own cache and the move pays its write. A HIPAA configuration cannot be detected; the router then prices those moves as free.
|
|
381
381
|
|
|
382
382
|
Native cache freshness is unknown after 270 seconds or after a history reset. Claude usage does not report the cache TTL. The policy evaluates price bounds for five-minute and one-hour writes, but the one-hour case is not observed fact. See the [architecture](architecture.md#cache-and-cost) for the estimate rules.
|
|
383
383
|
|
package/docs/evaluation.md
CHANGED
|
@@ -172,7 +172,7 @@ Limits:
|
|
|
172
172
|
- Output length is held the same. A stronger model may write less, or finish in fewer tool steps; a cheaper one may need more. The readout cannot see either.
|
|
173
173
|
- Answer quality is not measured. A saving bought with worse answers would read the same.
|
|
174
174
|
- Your model's cache is simulated, and a five-minute session that the lifetime rule takes for one hour (a plan subscriber billed as overage) would read too much cache on your side.
|
|
175
|
-
- A route that changes only the effort costs the same per token. On Opus 5.5, Sonnet 5.5, Haiku 5.5, Fable 5.1 and Fable 5.2 it shares your cache and reads $0.00; on a provider that caches each effort apart, it shows its cache writes.
|
|
175
|
+
- A route that changes only the effort costs the same per token. On Opus 5.5, Sonnet 5.5, Haiku 5.5, Fable 5.1 and Fable 5.2 it shares your cache and reads $0.00; on a provider that caches each effort apart, it shows its cache writes. [Activity routing](activity-routing.md#what-we-do-not-know-yet) has what transcripts show about the shared cache.
|
|
176
176
|
- Classifier charges, replies with routing off, and the side calls the router does not see (titles, compaction, server-side tools) are not counted; Claude Code's `/cost` counts them. On a Claude plan the dollars stand for quota, not cash, and the pane marks them `≈`.
|
|
177
177
|
- The usage records keep one record per session, by local calendar day, so sessions finishing turns at the same moment both count. A turn's switch counts on the day it ended, its replies on the days they arrived. Claude Code's store has no atomic update, which leaves these gaps: the same session open in two Claude Code processes at once can lose one's turn; a turn finishing in another process during **Delete history** can be lost; and a session's total from before a reset comes back if its first saved reply has the same millisecond as the reset, or if the system clock moved back across the reset. The router reads every session's record at session start and when the pane opens, so that read grows with the number of sessions kept. A session drops its days older than 400 days when it writes its record; on that read, another session's record with no day in the last 400 days is deleted, and past 1,000 records of other sessions those with the oldest last day go too. A record that does not read whole, such as one a later version wrote, counts toward the 1,000 but is never deleted. Totals kept before the dated usage records have no dates: All adds them, and the other periods leave them out.
|
|
178
178
|
|
package/docs/native-router.md
CHANGED
|
@@ -22,7 +22,7 @@ The activity is what the turn produces: `code`, `debug`, `explore`, `plan`, `rev
|
|
|
22
22
|
| `/router pin <tier>` | Pin the next turn and tool continuations. Routing on must already be enabled. |
|
|
23
23
|
| `/router activities <off\|on>` | Set the activity routing mode. Written to `router.json` at once; **Undo** in the pane reverts it. |
|
|
24
24
|
| `/router mode <cost\|balanced\|quality\|default>` | Set or clear this session's [routing mode](routing-modes.md), from the next turn. Kept for a resume in `$.store` `routing-mode:<session id>`. Never aborts a turn. |
|
|
25
|
-
| `/router decisions [n]` | Print this session's last `n` routing decisions (default 20, at most 60): origin, classifier answer, task, tier, route, reason, and each turn's end. No text. |
|
|
25
|
+
| `/router decisions [n]` | Print this session's last `n` routing decisions (default 20, at most 60): origin, classifier answer, task, tier, route, reason with the mass and bar it compared, and each turn's end. No text. |
|
|
26
26
|
| `/model <name>` | Select a model and turn routing off. `/router auto` turns it back on. |
|
|
27
27
|
|
|
28
28
|
The pane's **Set up** and **Credentials → Edit** buttons open Claude Code's secure plugin configuration. Claude Code refuses `$.command.run` from inside a `command.run` hook, so the Mod starts that dialog from a timer after the press returns.
|
package/docs/user-guide.md
CHANGED
|
@@ -27,7 +27,7 @@ In Claude Code:
|
|
|
27
27
|
- **OpenAI**: an OpenAI API key. The prompt text goes to OpenAI.
|
|
28
28
|
- **Ollama**: no credential. Run `ollama serve` and pull the model; the prompt text stays on this machine.
|
|
29
29
|
3. To use another classifier, open the pane's **Classifier** tab and select its row. See [Choose the classifier](#choose-the-classifier).
|
|
30
|
-
4. Check the band above the prompt: `Routing on · ready`. On a model that no tier routes to, and that no activity override uses with activity routing `on`, the session starts with routing off and the band says why; run `/router auto` to route.
|
|
30
|
+
4. Check the band above the prompt: `Routing on · ready`. On a model that no tier routes to, and that no activity override uses with activity routing `on`, the session starts with routing off and the band says why; run `/router auto` to route. That model then keeps running until a decision moves it: a stronger tier the classifier supports (two votes, or one very sure answer) moves at once; a cheaper route goes at once on a new task when it pays back its cache write, and waits for the dwell and a passed check on a task in progress.
|
|
31
31
|
|
|
32
32
|
Keys are stored as sensitive plugin options. Do not paste them into a model conversation. The [configuration guide](configuration.md) covers settings and migrations.
|
|
33
33
|
|
|
@@ -43,7 +43,7 @@ Router draws one line above the prompt. The line ends with a **Router** button t
|
|
|
43
43
|
Read the first line from left to right:
|
|
44
44
|
|
|
45
45
|
- **Tier meter** `▂▄▆█`: one bar per tier, lit up to the current tier in its color, like signal strength. The same bars name the tiers everywhere: `▂` micro, `▂▄` low, `▂▄▆` medium, `▂▄▆█` high.
|
|
46
|
-
- **Model**: the model and effort
|
|
46
|
+
- **Model**: the model and effort the router chose for this turn, such as `Opus 5.5 · xhigh`; see [Changes during a task](#changes-during-a-task) for when the effort is not the one sent. **fallback** after it means Claude Code answered with another model.
|
|
47
47
|
- **Activity**: the kind of work, in orange italic: `code`, `debug`, `explore`, `plan`, `review`, `ops` (commands: git, builds, tests, CI) or `docs`. See [Route by activity](#route-by-activity). When the reason already names the activity, the band does not repeat it.
|
|
48
48
|
- **Reason**: a glyph and a few words. `↑` moved up, `↓` moved down, `=` stayed, `…` waits, `!` a problem, `⏵` a pin: `↑ clearly stronger`, `↓ cheaper fits`, `= fits`, `… cheaper, waiting`, `↑ code` for an activity move.
|
|
49
49
|
- **Classifier support**: what the active classifier, such as Jev or Clef Flash, gave the switch against the bar the policy required, such as `Jev 88% ≥ 82%`. When whole percents would read as equal, it shows tenths: `Jev 81.0% < 81.4%`.
|
|
@@ -286,6 +286,8 @@ On the running task:
|
|
|
286
286
|
|
|
287
287
|
These thresholds are starting values. They are not tuned on data yet.
|
|
288
288
|
|
|
289
|
+
On Claude Code 2.1.322 an effort that the router changed inside a turn was not sent in the 6 cases found, while a changed model was, so the band and the decision record can name an effort that did not run. See [What we do not know yet](activity-routing.md#what-we-do-not-know-yet).
|
|
290
|
+
|
|
289
291
|
### Read the activity on the pane
|
|
290
292
|
|
|
291
293
|
- **Now**: the first line of **NEXT TURN** names the classifier's activity with its share and the next answer, such as `code 84% debug 8%`. An `uncertain` answer is shown dim: it applies no activity. In `on`, a line under it names the cell when the route is an override, such as `code at low runs on Sonnet 5.5 · high · tier route Haiku 5.5 · high`, and **why** reads `↑ code` or `↓ ops` when the activity moved the route; a refused move, a hold, or a tier change keeps its reason.
|
|
@@ -318,7 +320,7 @@ A routing mode moves each tier's route one step toward cost or quality; pins and
|
|
|
318
320
|
|
|
319
321
|
### Routing decisions
|
|
320
322
|
|
|
321
|
-
Each routed turn adds rows to the session's decision record in `$.store` (`decisions:v1:<session id>`): one when the turn starts, one per classifier review or tool-based move inside it, and one when it ends. A row holds the prompt's origin (`composer`, `task-notification`, …, or `unknown`), the classifier's tier and activity with their shares, whether it counted as the running task, the tier, activity, route and reason chosen, and at the end the turn's tool bucket and how many replies were priced. It holds no prompt, tool or file text. `/router decisions [n]` prints them. Each session keeps its newest 60 rows; records of the 20 most recent other sessions stay, for at most 30 days. **Delete history** deletes them all.
|
|
323
|
+
Each routed turn adds rows to the session's decision record in `$.store` (`decisions:v1:<session id>`): one when the turn starts, one per classifier review or tool-based move inside it, and one when it ends. A row holds the prompt's origin (`composer`, `task-notification`, …, or `unknown`), the classifier's tier and activity with their shares, whether it counted as the running task, the tier, activity, route and reason chosen, and at the end the turn's tool bucket and how many replies were priced. It holds no prompt, tool or file text. The route is the one chosen, which is not always the one sent; see [Changes during a task](#changes-during-a-task). `/router decisions [n]` prints them. For a tier move or an activity move up, taken or held, the row also holds the probability mass the policy compared and the bar it compared it with, printed after the reason: `(upgrade-pending 52% vs bar 79%)`. A held move can show a mass above its bar: it then waits for a second vote or for a checkpoint. On `downgrade-pending` the mass is the support for the classifier's own choice; a tier between it and the running one can be closer to its bar. A `jump`, a move on the turn's own tool calls and a context or availability override show neither number. Each session keeps its newest 60 rows; records of the 20 most recent other sessions stay, for at most 30 days. **Delete history** deletes them all.
|
|
322
324
|
|
|
323
325
|
A pin does not change the next turn after the pinned turn finishes. A fresh session on a model that a tier routes to, or with activity routing `on` a model an activity override uses (Sonnet 5.5 on the defaults), starts with routing on. A fresh session on any other model starts with routing off. `/clear` starts the new session with routing on. Resuming a saved session restores its saved routing on or off.
|
|
324
326
|
|
|
@@ -353,7 +355,7 @@ A new model ID needs an entry in `router.json` `models` with its price, context
|
|
|
353
355
|
|
|
354
356
|
The **Policy** tab writes each control inside the sentence it completes, and names the default beside a value that differs from it. One **Save** writes them with any route changes:
|
|
355
357
|
|
|
356
|
-
- **Start** at a tier: the tier a session starts from when its model is no tier's model (`baselineTier`).
|
|
358
|
+
- **Start** at a tier: the tier a session starts from when its model is no tier's model (`baselineTier`). The session's model runs until a decision moves it to that tier's route or another tier.
|
|
357
359
|
- **Going down** after 1, 2, or 3 agreeing turns in a row, and only if a cache write pays back within 1, 3, 5, or 10 later turns. A new or reset history needs one vote.
|
|
358
360
|
- **Going up** needs the classifier's support at 75%, raised up to 90% as the switch costs more. Read-only here; see [Configuration](configuration.md).
|
|
359
361
|
- **Activities** apply at 50%, 60%, 70%, or 80% certainty or more. Below it a new task runs its tier's model; the running task follows [Changes during a task](#changes-during-a-task). See [Activity routing](activity-routing.md#change-a-cell).
|
package/hooks/native-router.mjs
CHANGED
|
@@ -92,6 +92,7 @@ import {
|
|
|
92
92
|
nativeFacts,
|
|
93
93
|
observedMove,
|
|
94
94
|
observeResponse,
|
|
95
|
+
ownRoute,
|
|
95
96
|
prepareLoop,
|
|
96
97
|
resetHistory,
|
|
97
98
|
} from '../lib/route.mjs';
|
|
@@ -780,6 +781,9 @@ async function* passMain($, e, next, loop, version, nativeModel, reason, router)
|
|
|
780
781
|
stopReason: response.stopReason,
|
|
781
782
|
now: Date.now(),
|
|
782
783
|
});
|
|
784
|
+
// Before routing has chosen a route, the one in use is the model serving now: /model can change it with routing
|
|
785
|
+
// off. Once a decision exists, the last routed route is kept, as for every model.
|
|
786
|
+
if (reason === 'manual' && !loop.decision) observed.route = ownRoute(router.config, e.model);
|
|
783
787
|
// Routing chose nothing here, so the reply is not counted; your model's cache still follows it.
|
|
784
788
|
const compared = await compareYours($, router, loop, response, e.effort ?? null, nativeModel);
|
|
785
789
|
if (compared) observed.yoursBy = compared.states;
|
package/lib/decisions.mjs
CHANGED
|
@@ -6,8 +6,8 @@
|
|
|
6
6
|
import { afterResetOf, isRecord } from './records.mjs';
|
|
7
7
|
|
|
8
8
|
export const DECISIONS_PREFIX = 'decisions:v1:';
|
|
9
|
-
// The newest rows a session keeps, and the sessions kept: $.store holds 4 MiB for every key together, and a row is
|
|
10
|
-
//
|
|
9
|
+
// The newest rows a session keeps, and the sessions kept: $.store holds 4 MiB for every key together, and a row is at
|
|
10
|
+
// most about 350 bytes, so the records stay under 450 KB. Upgrade trigger: a review needs more history than this, or the store nears its limit; then roll rows into
|
|
11
11
|
// per-day counts.
|
|
12
12
|
export const MAX_ROWS = 60;
|
|
13
13
|
export const MAX_RECORDS = 20;
|
|
@@ -63,6 +63,32 @@ const round = (n, digits = 2) => (Number.isFinite(n) ? Math.round(n * 10 ** digi
|
|
|
63
63
|
const label = (value) => (typeof value === 'string' && value.length > 0 && value.length <= 80 ? value : null);
|
|
64
64
|
const count = (n) => (Number.isSafeInteger(n) && n > 0 ? n : 0);
|
|
65
65
|
|
|
66
|
+
// The reasons whose estimate holds a mass the policy compared with its threshold. Any other reason can carry an
|
|
67
|
+
// estimate that is not its own: a context or availability override keeps the one of the decision it replaced, and a
|
|
68
|
+
// jump compared its mass with jumpConfidence, not the threshold.
|
|
69
|
+
const WEIGHED = [
|
|
70
|
+
'upgrade',
|
|
71
|
+
'upgrade-pending',
|
|
72
|
+
'downgrade',
|
|
73
|
+
'downgrade-pending',
|
|
74
|
+
'downgrade-held',
|
|
75
|
+
'activity-up',
|
|
76
|
+
'activity-pending',
|
|
77
|
+
];
|
|
78
|
+
|
|
79
|
+
// The mass the policy compared and the bar it had to clear, from the decision's estimate (decide, lateralMove): mass
|
|
80
|
+
// above the incumbent for a move up, at or below the classifier's choice for a pending move down (the target for a
|
|
81
|
+
// taken or held one), the label's share for an activity move up, 1 when that move needs no label. Both or neither: a
|
|
82
|
+
// decision that weighed no mass (a hold, a cash gate, a cheaper route judged on its cache write) adds no field, so its
|
|
83
|
+
// row stays as small as before. Nor does an observed move: its mass is 1 by rule, the tools being the evidence.
|
|
84
|
+
function compared(kind, decision) {
|
|
85
|
+
if (kind === 'observed' || !WEIGHED.includes(decision?.reason)) return {};
|
|
86
|
+
const { upgradeMass, downgradeMass, threshold } = decision.estimate ?? {};
|
|
87
|
+
const mass = round(upgradeMass ?? downgradeMass);
|
|
88
|
+
const bar = round(threshold);
|
|
89
|
+
return mass === null || bar === null ? {} : { mass, bar };
|
|
90
|
+
}
|
|
91
|
+
|
|
66
92
|
// One routing decision as a row: `kind` start (the turn's first step), review (a classifier review applied inside the
|
|
67
93
|
// turn) or observed (a move on the turn's own tool calls). `advice` is the classifier's answer or null, `error` why
|
|
68
94
|
// there is none, `decision` the route chosen (chooseRoute's), `facts` what it was chosen on, `sameTask` sameTaskOf.
|
|
@@ -99,6 +125,7 @@ export function decisionRow({
|
|
|
99
125
|
same: typeof sameTask === 'boolean' ? sameTask : null,
|
|
100
126
|
down: typeof downAllowed === 'boolean' ? downAllowed : null,
|
|
101
127
|
ctx: Number.isFinite(contextTokens) ? Math.round(contextTokens / 1000) : null,
|
|
128
|
+
...compared(kind, decision),
|
|
102
129
|
};
|
|
103
130
|
}
|
|
104
131
|
|
|
@@ -159,6 +186,8 @@ const ROW_FIELDS = {
|
|
|
159
186
|
same: nullable((v) => typeof v === 'boolean'),
|
|
160
187
|
down: nullable((v) => typeof v === 'boolean'),
|
|
161
188
|
ctx: nullable(Number.isFinite),
|
|
189
|
+
mass: Number.isFinite,
|
|
190
|
+
bar: Number.isFinite,
|
|
162
191
|
observed: nullable(label),
|
|
163
192
|
requests: Number.isSafeInteger,
|
|
164
193
|
priced: Number.isSafeInteger,
|
|
@@ -213,6 +242,9 @@ const clock = (ms) => {
|
|
|
213
242
|
return `${pad2(d.getHours())}:${pad2(d.getMinutes())}:${pad2(d.getSeconds())}`;
|
|
214
243
|
};
|
|
215
244
|
const share = (p) => (Number.isFinite(p) ? ` ${Math.round(p * 100)}%` : '');
|
|
245
|
+
// No comparison sign: the stored values are rounded, and a move can clear its bar and still wait for a second vote.
|
|
246
|
+
const weighed = (row) =>
|
|
247
|
+
Number.isFinite(row.mass) && Number.isFinite(row.bar) ? `${share(row.mass)} vs bar${share(row.bar)}` : '';
|
|
216
248
|
|
|
217
249
|
function endLine(row) {
|
|
218
250
|
const unpriced = Object.entries(row.unpriced ?? {})
|
|
@@ -246,7 +278,7 @@ export function decisionLines(rows, n = 20) {
|
|
|
246
278
|
`asked ${asked}`,
|
|
247
279
|
task,
|
|
248
280
|
`-> ${row.tier ?? '-'}${row.activity ? ` ${row.activity}` : ''} ${row.route ?? ''}`.trimEnd(),
|
|
249
|
-
`(${row.reason ?? '?'})`,
|
|
281
|
+
`(${row.reason ?? '?'}${weighed(row)})`,
|
|
250
282
|
row.pinned ? 'pinned' : null,
|
|
251
283
|
]
|
|
252
284
|
.filter(Boolean)
|
package/lib/route.mjs
CHANGED
|
@@ -40,14 +40,24 @@ export function cellForModel(config, id) {
|
|
|
40
40
|
return null;
|
|
41
41
|
}
|
|
42
42
|
|
|
43
|
+
// The route a session model runs when no route names it: that model at the session's effort. Null for a model with a
|
|
44
|
+
// cell, and for one the config does not list.
|
|
45
|
+
export function ownRoute(config, model) {
|
|
46
|
+
if (cellForModel(config, model)) return null;
|
|
47
|
+
const key = Object.keys(config.models).find((k) => isSameModel(config.models[k].id, model));
|
|
48
|
+
return key ? { model: key, effort: null } : null;
|
|
49
|
+
}
|
|
50
|
+
|
|
43
51
|
export function emptyLoop(config, model) {
|
|
44
52
|
const cell = cellForModel(config, model);
|
|
45
53
|
return {
|
|
46
54
|
lastRoute: cell?.tier ?? config.baselineTier,
|
|
47
55
|
lastActivity: cell?.activity ?? null,
|
|
48
56
|
// The route (models key, effort) the last routed decision ran for `lastRoute`, kept as it ran: a routing mode or
|
|
49
|
-
// a route edit changes the config, never the route in use. Null until the first decision
|
|
50
|
-
route:
|
|
57
|
+
// a route edit changes the config, never the route in use. Null until the first decision, except on a session
|
|
58
|
+
// model no route names (ownRoute): it starts at the baseline tier but keeps running until a decision moves it.
|
|
59
|
+
// Taken as the baseline's route, repeated failures on it "escalated" to a weaker model.
|
|
60
|
+
route: ownRoute(config, model),
|
|
51
61
|
state: initialState(),
|
|
52
62
|
models: {},
|
|
53
63
|
resolutions: {},
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@alexeiled/claude-router",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.22.0",
|
|
4
4
|
"description": "Native Claude Code model and effort routing with Anthropic, DeepSeek, z.ai, MiniMax, MiMo and Kimi defaults, custom provider profiles, and classifier advice.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "Alexei Ledenev",
|