@alexeiled/claude-router 1.18.0 → 1.19.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +2 -2
- package/CHANGELOG.md +34 -0
- package/README.md +4 -1
- package/docs/architecture.md +9 -0
- package/docs/configuration.md +20 -6
- package/docs/evaluation.md +4 -0
- package/docs/native-router.md +4 -0
- package/docs/provider-experiments.md +141 -0
- package/docs/providers.md +83 -5
- package/docs/routing-modes.md +3 -1
- package/docs/user-guide.md +8 -2
- package/hooks/native-router.mjs +7 -5
- package/lib/band.mjs +8 -2
- package/lib/display.mjs +9 -1
- package/lib/modes.mjs +3 -16
- package/lib/panel.mjs +92 -27
- package/lib/providers.mjs +88 -0
- package/lib/savings.mjs +6 -2
- package/lib/usage.mjs +77 -16
- package/package.json +2 -2
- package/types/index.d.ts +3 -1
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "router",
|
|
3
3
|
"displayName": "Router",
|
|
4
|
-
"version": "1.
|
|
5
|
-
"description": "Native model and effort routing within your provider. Anthropic, DeepSeek
|
|
4
|
+
"version": "1.19.1",
|
|
5
|
+
"description": "Native model and effort routing within your provider. Anthropic, DeepSeek, z.ai, MiniMax, MiMo and Kimi defaults, custom provider profiles, classifier advice and one model per subagent.",
|
|
6
6
|
"author": {
|
|
7
7
|
"name": "Alexei Ledenev",
|
|
8
8
|
"url": "https://github.com/alexei-led"
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,39 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.19.1 (2026-10-31)
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
|
|
7
|
+
- Keep observed spend, replies, tokens, activity breakdowns and trends when provider cache behavior prevents a savings comparison. The band and pane say `comparison unavailable` instead of `no replies yet`. Mixed periods label the comparable subset; unknown savings remain unknown.
|
|
8
|
+
- Normalize unsupported effort when comparing MiMo routes. An inherited session effort no longer makes the same no-effort model look like a different cache identity.
|
|
9
|
+
- Fit the Routes grid to narrow panes. Long labels are shortened without losing their built-in or unsaved markers; the cell editor retains full details.
|
|
10
|
+
|
|
11
|
+
### Upgrade
|
|
12
|
+
|
|
13
|
+
Restart Claude Code after updating. No router-file changes are needed. Usage records migrate automatically to version 3, preserving older comparisons; replies omitted by older releases cannot be recovered. Earlier releases cannot display version 3 records.
|
|
14
|
+
|
|
15
|
+
## 1.19.0 (2026-10-31)
|
|
16
|
+
|
|
17
|
+
MiniMax, MiMo and Kimi gain built-in provider profiles alongside DeepSeek and z.ai.
|
|
18
|
+
|
|
19
|
+
### Added
|
|
20
|
+
|
|
21
|
+
- `minimax` defaults use MiniMax-M3.1-Flash with low, medium, high and max effort. Legacy M3 ignores effort and is not used as a deeper tier.
|
|
22
|
+
- `mimo` defaults combine MiMo-V2.6-Flash and Pro. The router selects models without claiming undocumented effort levels; provider-default thinking stays on.
|
|
23
|
+
- `kimi` defaults use Kimi-K3.1-Pro throughout the conversation, with low, high and max effort. They do not mix K2/K3 thinking histories.
|
|
24
|
+
- Each profile includes activity and subagent routes, image/context metadata, API list prices and conservative cache accounting. [Provider tables](docs/providers.md#new-provider-modes) show every cost, balanced and quality combination.
|
|
25
|
+
- [Research and experiments](docs/provider-experiments.md) record synthetic activity checks and native streaming tools. MiMo Flash's two format failures remain visible; no measured quality or savings guarantee is claimed.
|
|
26
|
+
|
|
27
|
+
### Fixed
|
|
28
|
+
|
|
29
|
+
- Cost mode skips unsupported effort levels when stepping down, rather than projecting back to the current level. Sparse ladders such as DeepSeek, z.ai and Kimi can now move max → high → low.
|
|
30
|
+
|
|
31
|
+
### Upgrade
|
|
32
|
+
|
|
33
|
+
- After upgrading, create `minimax-router.json`, `mimo-router.json` or `kimi-router.json` containing `{}` in the active Claude Code profile, then set `CLAUDE_ROUTER_PROVIDER` before launch. Missing or invalid files keep the native model unchanged.
|
|
34
|
+
- Start a fresh conversation when moving from another Kimi model to K3.1. Do not resume old model-specific thinking/tool history into it. Remove a fixed `CLAUDE_CODE_EFFORT_LEVEL` override to allow per-turn effort routing.
|
|
35
|
+
- Existing Anthropic users need no configuration change. Endpoint credentials and billing remain Claude Code's responsibility.
|
|
36
|
+
|
|
3
37
|
## 1.18.0 (2026-10-30)
|
|
4
38
|
|
|
5
39
|
The band and the pane now show one session cost readout: the same number, the same words, at the same moment.
|
package/README.md
CHANGED
|
@@ -23,12 +23,15 @@ With `CLAUDE_CONFIG_DIR`, put the file in that directory instead.
|
|
|
23
23
|
| Anthropic, the default | `router.json` | Haiku 5.5, Sonnet 5.5, Opus 5.5 |
|
|
24
24
|
| `deepseek` | `deepseek-router.json` | DeepSeek Flash, routed by effort |
|
|
25
25
|
| `zai` | `zai-router.json` | GLM-5.3-Flash and GLM-5.3 |
|
|
26
|
+
| `minimax` | `minimax-router.json` | MiniMax-M3.1-Flash, routed by effort |
|
|
27
|
+
| `mimo` | `mimo-router.json` | MiMo-V2.6-Flash and MiMo-V2.6-Pro, no routed effort |
|
|
28
|
+
| `kimi` | `kimi-router.json` | Kimi-K3.1-Pro, routed by effort |
|
|
26
29
|
|
|
27
30
|
OpenRouter, Ollama, other local servers and other compatible hosted providers use custom `<provider>-router.json` files.
|
|
28
31
|
They have no built-in model defaults. A selected file must exist and validate, or the router leaves Claude Code's model unchanged.
|
|
29
32
|
|
|
30
33
|
See [Providers](docs/providers.md) for **default routes and activities**, **custom profiles**, and the **`--settings` flag alternative**.
|
|
31
|
-
Execution provider selection is separate from classifier selection.
|
|
34
|
+
Execution provider selection is separate from classifier selection. Observed usage stays visible even when a provider's cache behavior makes savings comparisons unavailable.
|
|
32
35
|
|
|
33
36
|
## Supported classifiers
|
|
34
37
|
|
package/docs/architecture.md
CHANGED
|
@@ -135,6 +135,15 @@ Nothing is recorded with `activityRouting: off`. Stats run inside a guard: a fai
|
|
|
135
135
|
|
|
136
136
|
After each main reply, where the response is observed, `savings.mjs` prices the reply on the model that served it and the same tokens on "your model": the session's model and the effort Claude Code sent on the turn's first request. `compareModels` prices each reply against every model in `router.json` as "your model", at the session effort. Each keeps a simulated single cache in the loop under its models key in `yoursBy` (`{ total, at, onYours, model, effort }`), which a history reset clears; after an effort change the next reply finds it cold. Each reply also becomes a `replyEntry` in `usage.mjs`: its time, its cell `mode|activity|tier|modelId@effort` with the model that served it, its prices against the model selected then, its input, cache-read and output tokens, and what the same tokens cost on each model as yours. Models are kept by id, not by models key, because a key can later name another model. Until the turn ends its entries are also the view's `usageLive`; every readout (the two-row band, the Now and Usage tabs, `/router`) adds them to the record with `withReplies`, so they count the same replies at the same moment. At turn completion the entries leave `turnUsage` in the same tick they enter the held record, and every view write of the records carries `usageLive` with them, so a pane opened mid-write, a failed store write or a Reset counts a turn once or not at all, never twice. At turn completion `recordUsage` adds the turn's entries to the session's own `$.store` key `usage:v1:<session id>`, by local calendar day: per day the cells' nine sums, the what-if sums, the replies per your model, the routed cost per hour, the turns, and the route switches by tier and by activity, with `since`, the record's first write. Days older than 400 days leave the record when it is written. Each key has one writer, and Claude Code 2.1.296 writes one key under a file lock (observed in its code; the `$.store` types do not promise it), so sessions that finish turns at the same moment do not overwrite each other. The view's `usageStore` adds up two parts: every other session's key, read at session start and when the pane opens, and this session's record (`usageSession`), which the router reads once, holds in memory, adds each finished turn to and then writes, so a pane opened during that write counts the turn once. On that read, `prunedUsage` names other sessions' records with no day in the last 400 days and, past 1,000 records, those whose last day is oldest; a record that does not read whole, such as one a later version wrote, is never named. Each named record is read again and deleted only if it has not changed, so a session that wrote its record since keeps it; with no conditional delete in `$.store`, a write between that read and the delete is still lost. A day that does not read is dropped, and the others stay. The Usage tab sums the days of the chosen period (`summarize`); the period is kept across sessions in `usage:period`. Before these records each session kept undated totals in `savings:v1:<session id>`; they are no longer written, and the view's `savingsStore`, their sum, is added to the All period. **Delete history** first writes the time in `savings:reset:v1`, then deletes every usage, savings, activity stats, metrics and subagent record. A record whose first reply is older than that time reads as empty, and each session reads the time before it adds a turn, so a session that still holds its record from before a Reset in another session drops it instead of writing it back. The comparison is by milliseconds: a record whose first reply has the reset's millisecond, or one written while the clock had moved back, still counts. A stored value of any other shape reads as empty. A reply with routing off or after an engine fallback is not counted, but it still moves your model's cache. A reply whose cache lifetime cannot be read loses its prices only, and your model's cache still follows it; a usage without rate limits prices writes at five minutes. No failure here fails the turn. [Evaluation](evaluation.md#routing-vs-your-model) gives the method and its limits.
|
|
137
137
|
|
|
138
|
+
Usage storage is version 3 under the same `usage:v1:<session id>` keys. Each day separates comparable `cells`
|
|
139
|
+
from observed-only `uncompared` cells. Both retain reply counts, served-model cost and tokens; only comparable cells
|
|
140
|
+
contribute to savings and What if. The unused comparison fields in observed-only arrays are zero placeholders,
|
|
141
|
+
never a zero-cost baseline. `summarize().observed` combines both lanes for spend, counts and breakdowns while the
|
|
142
|
+
existing top-level comparison totals remain separate. Live replies, daily/hourly trends, periods, merges and resets
|
|
143
|
+
use the same separation. Version 1/2 records migrate on read without losing their comparison history. Previously
|
|
144
|
+
omitted observed-only replies cannot be reconstructed. Older releases do not read version 3 records; they do not
|
|
145
|
+
prune unreadable newer records.
|
|
146
|
+
|
|
138
147
|
### Subagents
|
|
139
148
|
|
|
140
149
|
A subagent's model is chosen once, in the `agent.spawn` hook, before its cache exists. Its `turn.step` events pass through unchanged: a subagent runs on the 5-minute cache lifetime by default, and a switch between its steps would rewrite a 50,000–300,000-token context cold. The facts below are from the Claude Code 2.1.296 types (`AgentSpawnInput`, `HookBudget`) and a probe Mod's observed events, `experiments/mod-router/results/agent-spawn-probe.json`. The [plan](plans/2026-10-10-subagent-routing-plan.md#2-verified-facts) cites each one.
|
package/docs/configuration.md
CHANGED
|
@@ -39,18 +39,24 @@ A non-Anthropic file must exist and validate. It never falls back to `router.jso
|
|
|
39
39
|
|
|
40
40
|
### Provider prices
|
|
41
41
|
|
|
42
|
-
|
|
42
|
+
All non-Anthropic built-in model windows are at least 1M tokens.
|
|
43
43
|
Price estimates use the [DeepSeek peak API rates](https://api-docs.deepseek.com/quick_start/pricing) and [z.ai API list rates](https://docs.z.ai/guides/overview/pricing.md).
|
|
44
|
-
|
|
44
|
+
DeepSeek and z.ai rates were checked on 2026-10-28; the new provider rates on 2026-10-31.
|
|
45
45
|
|
|
46
46
|
| Model | Input / output / cached input, USD per million tokens |
|
|
47
47
|
| --- | --- |
|
|
48
48
|
| DeepSeek Flash | 0.30 / 1.20 / 0.006 |
|
|
49
49
|
| GLM-5.3-Flash | 0.15 / 0.50 / 0.03 |
|
|
50
50
|
| GLM-5.3 | 1.40 / 4.40 / 0.26 |
|
|
51
|
+
| MiniMax-M3.1-Flash | 0.15 / 0.60 / 0.015 |
|
|
52
|
+
| MiMo-V2.6-Flash | 0.14 / 0.28 / 0.0028 |
|
|
53
|
+
| MiMo-V2.6-Pro | 0.435 / 0.87 / 0.0036 |
|
|
54
|
+
| Kimi-K3.1-Pro | 2.00 / 10.00 / 0.20 |
|
|
51
55
|
|
|
52
56
|
These are fixed estimate inputs, not an invoice. DeepSeek off-peak rates differ, and z.ai plan quota uses its own multipliers.
|
|
53
|
-
|
|
57
|
+
New provider sources: [MiniMax pricing](https://platform.minimax.io/docs/pricing/overview),
|
|
58
|
+
[MiMo pricing](https://mimo.mi.com/docs/en-US/pricing), and [Kimi pricing](https://platform.kimi.ai/docs/pricing/chat).
|
|
59
|
+
All non-Anthropic built-in presets use `billing: "credits"`.
|
|
54
60
|
For an included-plan model, set `billing: "plan"` to disable the cold-input cash cap.
|
|
55
61
|
A provider name or URL does not establish its billing product.
|
|
56
62
|
|
|
@@ -61,7 +67,7 @@ A provider name or URL does not establish its billing product.
|
|
|
61
67
|
| Mode | Use | Estimate |
|
|
62
68
|
| --- | --- | --- |
|
|
63
69
|
| `explicit` | Anthropic default | Configured write multipliers and TTLs |
|
|
64
|
-
| `automatic` |
|
|
70
|
+
| `automatic` | All other built-in profiles and custom-profile default | Reported hit/miss input rates, no separate write premium |
|
|
65
71
|
| `disabled` | No cache discount to model | All input at the uncached rate, never marked warm |
|
|
66
72
|
|
|
67
73
|
Automatic-cache bounds allow a future hit or a full miss, including after an observed miss.
|
|
@@ -83,11 +89,19 @@ An unknown cache state is not proof of a miss.
|
|
|
83
89
|
|
|
84
90
|
[DeepSeek caching](https://api-docs.deepseek.com/guides/kv_cache) is automatic and best-effort. Its Anthropic endpoint ignores explicit cache controls.
|
|
85
91
|
[z.ai caching](https://docs.z.ai/guides/capabilities/cache.md) has no verified model-specific TTL.
|
|
92
|
+
MiniMax and MiMo expose automatic prefix caching; no retention guarantee is assumed here.
|
|
93
|
+
Kimi's default five-minute cache fills cost the ordinary input rate; the preset does not select top-level one-hour
|
|
94
|
+
cache control, whose writes cost twice as much. Its observed default fills/reads are priced, but TTL is not used
|
|
95
|
+
to promise future hits. A custom client that selects one-hour Kimi caching must configure its write premium.
|
|
96
|
+
MiMo's currently free cache-write promotion is not treated as free cold input.
|
|
86
97
|
Claude Code's `DISABLE_PROMPT_CACHING` does not establish whether an automatic server cache is disabled.
|
|
87
98
|
|
|
88
|
-
|
|
99
|
+
Known observed usage is retained in Usage even when savings cannot be compared.
|
|
89
100
|
For automatic caches, routing-versus-your-model comparisons require an identical observed route.
|
|
90
|
-
Otherwise the
|
|
101
|
+
Otherwise the band and pane show observed spend, replies, tokens and activity costs with `comparison unavailable`.
|
|
102
|
+
A partially comparable period labels the savings subset; rows without full comparison coverage show `—` under `vs yours`.
|
|
103
|
+
The router never substitutes zero savings or invents another route's cache hit rate.
|
|
104
|
+
Models with `efforts: []`, including MiMo, ignore session effort in the comparison identity.
|
|
91
105
|
Missing counters remain unknown. See [provider acceptance checks](evaluation.md#provider-profiles).
|
|
92
106
|
|
|
93
107
|
## Built-in defaults
|
package/docs/evaluation.md
CHANGED
|
@@ -21,6 +21,10 @@ Terminal engineering and SRE results differ. Neither GLM model leads every activ
|
|
|
21
21
|
Measured max-effort results do not establish low/high performance or an optimal routing matrix.
|
|
22
22
|
The original DeepSeek benchmark profile does not establish the quality of its newer hosted 1017 snapshot.
|
|
23
23
|
|
|
24
|
+
MiniMax, MiMo and Kimi have [separate sourced research and live checks](provider-experiments.md), including
|
|
25
|
+
all activity/mode routes, retained output-format failures and native streaming tool continuations.
|
|
26
|
+
Those small synthetic checks do not establish repository-quality rankings or optimal efforts.
|
|
27
|
+
|
|
24
28
|
### Compatibility checks
|
|
25
29
|
|
|
26
30
|
On 2026-10-28, six synthetic live requests per provider returned HTTP 200.
|
package/docs/native-router.md
CHANGED
|
@@ -71,6 +71,10 @@ The Routes tab starts with the **Activity routing** selector, `off · on`, and a
|
|
|
71
71
|
|
|
72
72
|
The Classifier tab lists every configured classifier as a row with its service host (`this machine` for a local one), its activity probe and p95 wait, and whether its key and endpoint settings are complete. Selecting a row writes `classifier` to `router.json` at once, or removes it for the default, through the same validation, and resets the new classifier's failure count. A turn already being classified finishes with its own classifier; its readings stay off the pane. **Undo** returns to the previous classifier. The deadline saves at once too, and a built-in classifier's default deadline is written as no override. **Sends** names the host that receives prompt text. The **Deadline** line shows the active classifier's p95 wait across sessions, or its probe's before any turn, in yellow when it is slower than the deadline.
|
|
73
73
|
|
|
74
|
+
The Routes grid sizes its cells to the available pane width. Long labels use an ellipsis while retaining their
|
|
75
|
+
built-in/unsaved marker; select a cell to inspect its complete model and effort. Observed-only Usage rows retain
|
|
76
|
+
spend and tokens but show no fabricated savings. See [comparison coverage](user-guide.md#open-the-router-pane).
|
|
77
|
+
|
|
74
78
|
The pane refuses to write through a symlink. A failed validation names the setting and leaves the file unchanged. New model aliases and the other supported fields need an edit to the active profile's `router.json`.
|
|
75
79
|
|
|
76
80
|
## Safety and failure states
|
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
# MiniMax, MiMo and Kimi evidence
|
|
2
|
+
|
|
3
|
+
Checked 2026-10-31. The [default routes](providers.md#built-in-defaults) are conservative starting policies,
|
|
4
|
+
not an optimal activity matrix or measured savings. These execution providers are separate from the classifier.
|
|
5
|
+
|
|
6
|
+
## Contracts and selection
|
|
7
|
+
|
|
8
|
+
| Provider | Selected model(s) | Documented controls used | Why |
|
|
9
|
+
| --- | --- | --- | --- |
|
|
10
|
+
| MiniMax | MiniMax-M3.1-Flash | low, medium, high, max | Current effort-capable model; cheaper than M3. M3 ignores effort. |
|
|
11
|
+
| MiMo | MiMo-V2.6-Flash / Pro | No routed effort | Flash for volume, Pro for harder work. Anthropic docs establish binary thinking, not depth levels. |
|
|
12
|
+
| Kimi | Kimi-K3.1-Pro | low, high, max | Cheaper successor to K3; one model avoids mixing preserved-thinking histories. |
|
|
13
|
+
|
|
14
|
+
MiniMax also documents `xhigh`, but some Claude Code versions serialize it as `high`. The preset omits it
|
|
15
|
+
rather than claim an extra distinct compute setting. Raw `xhigh` acceptance was checked, not native normalization.
|
|
16
|
+
Kimi K2.7 is not used as a cheap tier: its lifecycle notice closes access for new users on 2026-10-31
|
|
17
|
+
and plans retirement by the end of December 2026. Existing K3 users must start a fresh conversation to migrate to K3.1.
|
|
18
|
+
|
|
19
|
+
The local model-cards catalog was an initial reference: MiniMax M3 verified 2026-07-14 and Kimi K3/K2.7
|
|
20
|
+
verified 2026-09-13. It has no M3.1 Flash, K3.1 or MiMo record. Current official endpoint docs supersede
|
|
21
|
+
older catalog controls and K2 context claims.
|
|
22
|
+
|
|
23
|
+
### Primary sources
|
|
24
|
+
|
|
25
|
+
- MiniMax [Messages](https://platform.minimax.io/docs/api-reference/text-chat-anthropic),
|
|
26
|
+
[SDK compatibility](https://platform.minimax.io/docs/api-reference/text-anthropic-api),
|
|
27
|
+
[pricing](https://platform.minimax.io/docs/pricing/overview),
|
|
28
|
+
[automatic caching](https://platform.minimax.io/docs/api-reference/text-prompt-caching),
|
|
29
|
+
[Claude Code](https://platform.minimax.io/docs/m-plan/claude-code),
|
|
30
|
+
[Flash release](https://www.minimax.io/blog/minimax-m31f).
|
|
31
|
+
- MiMo [Anthropic API](https://mimo.mi.com/docs/en-US/api/chat/anthropic-api),
|
|
32
|
+
[model selection](https://mimo.mi.com/docs/quick-start/summary/model),
|
|
33
|
+
[pricing](https://mimo.mi.com/docs/en-US/pricing),
|
|
34
|
+
[Claude Code](https://mimo.mi.com/docs/integration/claudecode),
|
|
35
|
+
[deep thinking](https://mimo.mi.com/docs/en-US/quick-start/usage-guide/text-generation/deep-thinking).
|
|
36
|
+
- Kimi [Messages](https://platform.kimi.ai/docs/api/messages),
|
|
37
|
+
[models and lifecycle](https://platform.kimi.ai/docs/models),
|
|
38
|
+
[K3.1 guide](https://platform.kimi.ai/docs/guide/kimi-k3-1-pro-quickstart),
|
|
39
|
+
[pricing](https://platform.kimi.ai/docs/pricing/chat),
|
|
40
|
+
[caching](https://platform.kimi.ai/docs/guide/context-caching),
|
|
41
|
+
[K3 session guidance](https://forum.moonshot.ai/t/kimi-k3-is-here-our-most-capable-model/480).
|
|
42
|
+
|
|
43
|
+
Vendor results motivate the model choices, not the exact activity overrides. No independent head-to-head
|
|
44
|
+
repository benchmark establishes which effort each activity needs. Token prices are not prices per successful task.
|
|
45
|
+
|
|
46
|
+
## Live checks
|
|
47
|
+
|
|
48
|
+
All prompts and fixtures were synthetic. No real repository code, private conversations or credentials
|
|
49
|
+
are included in artifacts. Credential files stay local. JSON checks save final answers and usage, never reasoning text.
|
|
50
|
+
|
|
51
|
+
### Raw controls
|
|
52
|
+
|
|
53
|
+
[`provider-controls.json`](../experiments/mod-router/results/provider-controls.json) records 20 HTTP 200 requests:
|
|
54
|
+
old/new MiniMax, both MiMo models, and old/new Kimi. The body used a prime-number JSON prompt,
|
|
55
|
+
`max_tokens: 2048`, `thinking: {type: "adaptive"}`, and the recorded `output_config.effort`.
|
|
56
|
+
All responses named the requested model.
|
|
57
|
+
|
|
58
|
+
HTTP acceptance alone cannot distinguish honored fields from ignored fields. In particular, M3 success does not
|
|
59
|
+
make it effort-controllable, and MiMo low/high/max success does not establish different reasoning depths.
|
|
60
|
+
The production MiMo preset therefore lists no effort levels. Xiaomi's documented default thinking remains on;
|
|
61
|
+
Claude Code may still send its session effort and adaptive thinking, accepted in the native smoke test.
|
|
62
|
+
|
|
63
|
+
### Activity and mode microchecks
|
|
64
|
+
|
|
65
|
+
`node scripts/probe-provider-routes.mjs <provider> <budget-usd>` is billed and opt-in.
|
|
66
|
+
It asks one deterministic JSON task for each distinct activity/model/effort route across all tiers and modes,
|
|
67
|
+
using [the checked-in tasks](../test/fixtures/provider-tasks.json). Every mode/tier cell references its measured route.
|
|
68
|
+
The tests check small code-contract outputs, a boundary bug, an import graph, topological planning, SQL injection,
|
|
69
|
+
replica status and a configuration reference. They are **not code editing or repository-quality benchmarks**.
|
|
70
|
+
|
|
71
|
+
| Provider | Distinct calls | Exact JSON checks passed | Estimated API-list cost |
|
|
72
|
+
| --- | ---: | ---: | ---: |
|
|
73
|
+
| MiniMax | 28 | 28 | $0.001472 |
|
|
74
|
+
| MiMo | 14 | 12 | $0.001001 |
|
|
75
|
+
| Kimi | 21 | 21 | $0.024376 |
|
|
76
|
+
|
|
77
|
+
Results: [MiniMax](../experiments/mod-router/results/provider-routes-minimax.json),
|
|
78
|
+
[MiMo](../experiments/mod-router/results/provider-routes-mimo.json),
|
|
79
|
+
[Kimi](../experiments/mod-router/results/provider-routes-kimi.json).
|
|
80
|
+
|
|
81
|
+
MiMo Flash returned objects instead of module-name strings for `explore`, and added prose to an otherwise correct
|
|
82
|
+
`plan` array. Those are retained failures, not retried or hidden by a permissive parser. Pro passed both.
|
|
83
|
+
One sample does not justify moving every exploration or planning task to Pro.
|
|
84
|
+
MiniMax/Kimi low effort also passed these easy tasks, so the sample does not prove high/max necessary for harder work.
|
|
85
|
+
|
|
86
|
+
### Native streaming and tools
|
|
87
|
+
|
|
88
|
+
Claude Code **2.1.316** loaded the production Mod in isolated homes. Each provider ran a pinned high turn,
|
|
89
|
+
a Read tool continuation, a pinned micro turn, and another Read continuation: four upstream requests.
|
|
90
|
+
The test forwarded only these synthetic requests to fixed provider endpoints, capped output at 2,048 tokens,
|
|
91
|
+
and verified the requested/served model, supported routed effort and final marker.
|
|
92
|
+
|
|
93
|
+
| Provider | Observed transition | Result |
|
|
94
|
+
| --- | --- | --- |
|
|
95
|
+
| MiniMax | M3.1 Flash max → low | Passed |
|
|
96
|
+
| MiMo | Pro → Flash, session effort retained | Passed |
|
|
97
|
+
| Kimi | K3.1 Pro max → low, same model | Passed |
|
|
98
|
+
|
|
99
|
+
Artifacts: [MiniMax](../experiments/mod-router/results/provider-native-minimax.json),
|
|
100
|
+
[MiMo](../experiments/mod-router/results/provider-native-mimo.json),
|
|
101
|
+
[Kimi](../experiments/mod-router/results/provider-native-kimi.json).
|
|
102
|
+
Repeat with `node experiments/mod-router/scripts/provider-live.mjs <provider> <budget-usd>` (billed).
|
|
103
|
+
This temporary diagnostic forwarder is **not** part of the shipped router.
|
|
104
|
+
|
|
105
|
+
All three reported positive cache reads on tool continuations. MiniMax/MiMo streamed usage omitted cache-creation
|
|
106
|
+
counts; the engine compatibility check covers omitted-counter normalization. Kimi reported ordinary-price five-minute
|
|
107
|
+
cache fills and cache reads; the native request contained no top-level one-hour cache control.
|
|
108
|
+
No cross-effort hit-rate or TTL guarantee follows from these few requests.
|
|
109
|
+
|
|
110
|
+
The operator approved a $5 ceiling. Raw controls, microchecks and two native smoke passes together accounted for
|
|
111
|
+
approximately **$0.0625** at published list prices, including the superseded first native pass.
|
|
112
|
+
The [spend record](../experiments/mod-router/results/provider-experiment-spend.json) includes the superseded pass's
|
|
113
|
+
captured console totals so the aggregate can be reproduced. This is not a provider invoice or subscription-quota estimate.
|
|
114
|
+
The scripts enforce **per-invocation** ceilings only; the supervisor tracked the aggregate manually, not with a shared
|
|
115
|
+
runtime ledger. Each repeat needs a fresh spending authorization.
|
|
116
|
+
|
|
117
|
+
## Offline checks
|
|
118
|
+
|
|
119
|
+
`npm run test:providers` uses Claude Code and a loopback-only API with dummy credentials, not paid calls.
|
|
120
|
+
It covers all five non-Anthropic built-ins, profile selection, high/micro transitions, tools, omitted usage,
|
|
121
|
+
missing files and invalid configuration. Unit tests cover every new tier × activity × mode, pinning,
|
|
122
|
+
file isolation, reset/undo, metadata, lack of MiMo effort controls and the single-model Kimi catalog.
|
|
123
|
+
The tables in [Providers](providers.md#new-provider-modes) are generated and checked against the actual route transform.
|
|
124
|
+
|
|
125
|
+
Cost mode now steps to the next supported lower effort, skipping gaps: `max → high → low` on Kimi,
|
|
126
|
+
DeepSeek and z.ai. Previously upward projection could return the original effort and make cost mode a no-op.
|
|
127
|
+
MiMo has no effort lever; adjacent equal routes intentionally remain equal.
|
|
128
|
+
|
|
129
|
+
## Limits and upgrade triggers
|
|
130
|
+
|
|
131
|
+
Not tested: native images/video, near-1M context, compaction or rewind with provider thinking signatures,
|
|
132
|
+
forced tool choice, sustained rate limits, subscription eligibility, or real per-activity task quality and savings.
|
|
133
|
+
Raw API vision support is documented, not an end-to-end screenshot result.
|
|
134
|
+
|
|
135
|
+
MiMo Token Plan terms restrict general API/backend automation; use the documented PAYG endpoint and prices
|
|
136
|
+
for these experiments. A Claude Code integration does not establish that every non-coding automation qualifies
|
|
137
|
+
for a subscription. Check your plan terms and endpoint separately.
|
|
138
|
+
|
|
139
|
+
Keep these small presets until a representative coding/debug/review workload measures task success, retries,
|
|
140
|
+
total input/output, latency and cost per accepted change. Expand activity overrides only when that evidence
|
|
141
|
+
supports a difference. Do not add K2/K3 mixing or unsupported effort controls to make the tables look richer.
|
package/docs/providers.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
Claude Router routes models and effort within the provider that Claude Code already uses.
|
|
4
4
|
Set a provider name and give the router a matching JSON file. No provider-specific plugin or proxy is needed.
|
|
5
5
|
|
|
6
|
-
**Built-in profiles:** Anthropic, DeepSeek
|
|
6
|
+
**Built-in profiles:** Anthropic, DeepSeek, z.ai, MiniMax, MiMo and Kimi.
|
|
7
7
|
**Custom profiles:** OpenRouter, Ollama, other local servers, and other hosted providers with a Claude Code-compatible Anthropic endpoint.
|
|
8
8
|
Custom profiles need their own model settings. They do not inherit Claude models.
|
|
9
9
|
|
|
@@ -42,12 +42,15 @@ The directory is `CLAUDE_CONFIG_DIR` when set, otherwise `~/.claude`.
|
|
|
42
42
|
| Unset, empty, or `anthropic` | `router.json` | Haiku 5.5, Sonnet 5.5, Opus 5.5 |
|
|
43
43
|
| `deepseek` | `deepseek-router.json` | DeepSeek Flash |
|
|
44
44
|
| `zai` | `zai-router.json` | GLM-5.3-Flash, GLM-5.3 |
|
|
45
|
+
| `minimax` | `minimax-router.json` | MiniMax-M3.1-Flash |
|
|
46
|
+
| `mimo` | `mimo-router.json` | MiMo-V2.6-Flash, MiMo-V2.6-Pro |
|
|
47
|
+
| `kimi` | `kimi-router.json` | Kimi-K3.1-Pro |
|
|
45
48
|
| `openrouter` | `openrouter-router.json` | None. Supply a complete profile. |
|
|
46
49
|
| `ollama` | `ollama-router.json` | None. Supply a complete profile. |
|
|
47
50
|
| Any other valid name | `<provider>-router.json` | None. Supply a complete profile. |
|
|
48
51
|
|
|
49
52
|
Names are local profile identifiers, not a provider registry or an endpoint lookup.
|
|
50
|
-
For example, `
|
|
53
|
+
For example, `local` or `my-gateway` can name a custom profile.
|
|
51
54
|
Names must start with a lowercase letter, contain only lowercase letters, digits and hyphens, and have at most 64 characters.
|
|
52
55
|
|
|
53
56
|
**Setting the name does not change Claude Code's endpoint, credentials or billing product.**
|
|
@@ -68,8 +71,20 @@ For an all-local setup, [configure the Ollama classifier](user-guide.md#use-olla
|
|
|
68
71
|
## Built-in defaults
|
|
69
72
|
|
|
70
73
|
Anthropic is the default when no provider name is set.
|
|
71
|
-
|
|
72
|
-
A file containing `{}` is enough
|
|
74
|
+
Every non-Anthropic built-in requires its named file, even when you want only defaults.
|
|
75
|
+
A file containing `{}` is enough. MiniMax, MiMo and Kimi require Router 1.19.0 or newer.
|
|
76
|
+
|
|
77
|
+
```sh
|
|
78
|
+
# First create the selected profile file containing {}, then launch on a configured model:
|
|
79
|
+
CLAUDE_ROUTER_PROVIDER=minimax claude --model 'MiniMax-M3.1-Flash[1m]'
|
|
80
|
+
CLAUDE_ROUTER_PROVIDER=mimo claude --model 'mimo-v2.6-flash[1m]'
|
|
81
|
+
CLAUDE_ROUTER_PROVIDER=kimi claude --model 'kimi-k3.1-pro[1m]'
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
Remove a fixed `CLAUDE_CODE_EFFORT_LEVEL` override if you want the router's per-turn efforts.
|
|
85
|
+
When migrating Kimi, start a **fresh conversation**. Its preserved thinking/signatures are model-specific;
|
|
86
|
+
resuming another model's history into K3-family models is not a supported migration.
|
|
87
|
+
The default Kimi ladder uses one model throughout, not a K2/K3 mixture.
|
|
73
88
|
|
|
74
89
|
### Model IDs
|
|
75
90
|
|
|
@@ -81,6 +96,10 @@ A file containing `{}` is enough for either.
|
|
|
81
96
|
| DeepSeek | `flash` | `deepseek-flash[1m]` |
|
|
82
97
|
| z.ai | `flash` | `glm-5.3-flash[1m]` |
|
|
83
98
|
| z.ai | `glm` | `glm-5.3[1m]` |
|
|
99
|
+
| MiniMax | `flash` | `MiniMax-M3.1-Flash[1m]` |
|
|
100
|
+
| MiMo | `flash` | `mimo-v2.6-flash[1m]` |
|
|
101
|
+
| MiMo | `pro` | `mimo-v2.6-pro[1m]` |
|
|
102
|
+
| Kimi | `k3` | `kimi-k3.1-pro[1m]` |
|
|
84
103
|
|
|
85
104
|
`[1m]` is a Claude Code context hint, not a different served model.
|
|
86
105
|
Changing Claude Code's model-slot environment variables does not update this catalog.
|
|
@@ -95,8 +114,13 @@ Each cell is `model alias · effort`. All profiles start at tier `low`.
|
|
|
95
114
|
| Anthropic | haiku · medium | haiku · high | opus · medium | opus · xhigh |
|
|
96
115
|
| DeepSeek | flash · low | flash · high | flash · high | flash · max |
|
|
97
116
|
| z.ai | flash · low | flash · high | glm · high | glm · max |
|
|
117
|
+
| MiniMax | flash · low | flash · medium | flash · high | flash · max |
|
|
118
|
+
| MiMo | flash · none | flash · none | pro · none | pro · none |
|
|
119
|
+
| Kimi | k3 · low | k3 · high | k3 · high | k3 · max |
|
|
98
120
|
|
|
99
121
|
A provider can use one model at several efforts. Two tiers can also share the same model and effort.
|
|
122
|
+
`none` for MiMo means **no routed effort**, not thinking off. Claude Code may still send its session effort;
|
|
123
|
+
Xiaomi does not document granular reasoning levels for its Anthropic endpoint. The preset lists `efforts: []`.
|
|
100
124
|
|
|
101
125
|
### Activity overrides
|
|
102
126
|
|
|
@@ -110,9 +134,13 @@ With activity routing on, these cells replace the base route. Other cells keep t
|
|
|
110
134
|
| z.ai | `code`, `debug`, `plan`, `review` | `medium` | glm · max |
|
|
111
135
|
| z.ai | `ops`, `explore`, `docs` | `medium` | flash · high |
|
|
112
136
|
| z.ai | `ops`, `explore`, `docs` | `high` | flash · max |
|
|
137
|
+
| MiniMax | `code`, `debug`, `plan`, `review` | `medium` | flash · max |
|
|
138
|
+
| MiMo | `ops`, `explore`, `docs` | `medium` | flash · none |
|
|
139
|
+
| Kimi | `code`, `debug`, `plan`, `review` | `medium` | k3 · max |
|
|
113
140
|
|
|
114
141
|
GLM-5.3 is text-only. Image-bearing history excludes it when the router can select a compatible route.
|
|
115
|
-
The default models all have a
|
|
142
|
+
The default models all have a context window of at least 1M tokens.
|
|
143
|
+
The four new model entries support image input at the provider; native image and near-limit context paths have not been tested.
|
|
116
144
|
|
|
117
145
|
Subagent routing is on for built-in profiles:
|
|
118
146
|
|
|
@@ -121,6 +149,9 @@ Subagent routing is on for built-in profiles:
|
|
|
121
149
|
| Anthropic | haiku | sonnet | opus |
|
|
122
150
|
| DeepSeek | flash | flash | flash |
|
|
123
151
|
| z.ai | flash | glm | glm |
|
|
152
|
+
| MiniMax | flash | flash | flash |
|
|
153
|
+
| MiMo | flash | pro | pro |
|
|
154
|
+
| Kimi | k3 | k3 | k3 |
|
|
124
155
|
|
|
125
156
|
Subagents receive a model at spawn, not a routed effort. Their own effort setting still applies.
|
|
126
157
|
|
|
@@ -128,6 +159,53 @@ These defaults are starting policies, not measured quality or savings guarantees
|
|
|
128
159
|
See [the evidence and limits](evaluation.md#provider-profiles).
|
|
129
160
|
For prices and cache settings, see [the configuration reference](configuration.md#provider-profiles).
|
|
130
161
|
|
|
162
|
+
## New provider modes
|
|
163
|
+
|
|
164
|
+
These are configured routes by tier and activity, before vote, context, failure, dwell and cache-cost guards.
|
|
165
|
+
`base/uncertain` means the tier's base route; an uncertain classifier tier keeps the incumbent tier.
|
|
166
|
+
A pin bypasses modes and activities. The ordinary [mode rules](routing-modes.md) apply without provider-specific code.
|
|
167
|
+
Generate these tables with `node scripts/provider-matrix.mjs`.
|
|
168
|
+
|
|
169
|
+
<!-- provider-matrix:start -->
|
|
170
|
+
**minimax**
|
|
171
|
+
|
|
172
|
+
| Tier | Activities | cost | balanced | quality |
|
|
173
|
+
| --- | --- | --- | --- | --- |
|
|
174
|
+
| micro | base/uncertain, code, debug, explore, plan, review, ops, docs | flash · low | flash · low | flash · low |
|
|
175
|
+
| low | base/uncertain, code, debug, explore, plan, review, ops, docs | flash · low | flash · medium | flash · high |
|
|
176
|
+
| medium | base/uncertain, explore, ops, docs | flash · medium | flash · high | flash · max |
|
|
177
|
+
| medium | code, debug, plan, review | flash · high | flash · max | flash · max |
|
|
178
|
+
| high | base/uncertain, code, debug, explore, plan, review, ops, docs | flash · high | flash · max | flash · max |
|
|
179
|
+
|
|
180
|
+
**mimo**
|
|
181
|
+
|
|
182
|
+
| Tier | Activities | cost | balanced | quality |
|
|
183
|
+
| --- | --- | --- | --- | --- |
|
|
184
|
+
| micro | base/uncertain, code, debug, explore, plan, review, ops, docs | flash · no routed effort | flash · no routed effort | flash · no routed effort |
|
|
185
|
+
| low | base/uncertain, code, debug, plan, review | flash · no routed effort | flash · no routed effort | pro · no routed effort |
|
|
186
|
+
| low | explore, ops, docs | flash · no routed effort | flash · no routed effort | flash · no routed effort |
|
|
187
|
+
| medium | base/uncertain, code, debug, plan, review | flash · no routed effort | pro · no routed effort | pro · no routed effort |
|
|
188
|
+
| medium | explore, ops, docs | flash · no routed effort | flash · no routed effort | pro · no routed effort |
|
|
189
|
+
| high | base/uncertain, code, debug, plan, review | pro · no routed effort | pro · no routed effort | pro · no routed effort |
|
|
190
|
+
| high | explore, ops, docs | flash · no routed effort | pro · no routed effort | pro · no routed effort |
|
|
191
|
+
|
|
192
|
+
**kimi**
|
|
193
|
+
|
|
194
|
+
| Tier | Activities | cost | balanced | quality |
|
|
195
|
+
| --- | --- | --- | --- | --- |
|
|
196
|
+
| micro | base/uncertain, code, debug, explore, plan, review, ops, docs | k3 · low | k3 · low | k3 · low |
|
|
197
|
+
| low | base/uncertain, code, debug, explore, plan, review, ops, docs | k3 · low | k3 · high | k3 · max |
|
|
198
|
+
| medium | base/uncertain, explore, ops, docs | k3 · low | k3 · high | k3 · max |
|
|
199
|
+
| medium | code, debug, plan, review | k3 · high | k3 · max | k3 · max |
|
|
200
|
+
| high | base/uncertain, code, debug, explore, plan, review, ops, docs | k3 · high | k3 · max | k3 · max |
|
|
201
|
+
<!-- provider-matrix:end -->
|
|
202
|
+
|
|
203
|
+
MiniMax uses all four preset effort levels. `xhigh` is omitted because some Claude Code versions send it as `high`;
|
|
204
|
+
`max` supplies the deeper setting. Legacy M3 ignores effort and is not a stronger tier by default.
|
|
205
|
+
MiMo's cost/quality modes change models where a neighbouring tier permits it; otherwise the route stays the same.
|
|
206
|
+
Kimi uses only `low`, `high`, `max`, the documented controls, and never promotes a session into another model.
|
|
207
|
+
These are starting policies, not an optimal activity matrix. See [research and experiments](provider-experiments.md).
|
|
208
|
+
|
|
131
209
|
## Add another provider
|
|
132
210
|
|
|
133
211
|
No code change or provider registration is needed.
|
package/docs/routing-modes.md
CHANGED
|
@@ -15,7 +15,9 @@ In every mode:
|
|
|
15
15
|
- A pin (`/router pin <tier>`) runs that tier's configured route. With routing off (`/router off`, `/model`),
|
|
16
16
|
Claude's model runs, as before.
|
|
17
17
|
- A route never loses its effort: an effort level moves only within what the model lists in `models.<alias>.efforts`,
|
|
18
|
-
|
|
18
|
+
skipping unsupported levels in the direction of the move. For example, `cost` moves `max → high → low`
|
|
19
|
+
on a model listing only those three. A route with a `null` effort (the session's) or a model without efforts has no effort step.
|
|
20
|
+
- [MiniMax, MiMo and Kimi tables](providers.md#new-provider-modes) show every new preset's mode/activity combination.
|
|
19
21
|
|
|
20
22
|
## Choose a mode
|
|
21
23
|
|
package/docs/user-guide.md
CHANGED
|
@@ -104,11 +104,17 @@ Every write applies from the next turn and can be undone. **Undo** (`u`) puts ba
|
|
|
104
104
|
|
|
105
105
|

|
|
106
106
|
|
|
107
|
-
The Usage tab
|
|
107
|
+
The Usage tab keeps observed routed spend and, when the cache history permits it, compares that spend against your model. Your model is the model `/model` (or `--model`) selected, at the session effort, when a reply ran. The router prices observed tokens at the list prices in `router.json`. A comparison also prices the same tokens on your model when its cache history can be estimated.
|
|
108
108
|
|
|
109
109
|
The top row picks the period: **Session**, **Today**, **7d** (today and the six days before), **30d**, **Month** (since the first of this month) and **All**. Press `p` for the next one. The pane remembers the choice across sessions. Session holds this session's replies, and the line under the row names your model, or every model when `/model` changed during the session. The other periods add up the replies of every session on those calendar days. Each session counts against its own model, and the line reads `all projects · Oct 7 – Oct 13 · each session vs its own model`.
|
|
110
110
|
|
|
111
|
-
|
|
111
|
+
When comparisons are unavailable, the band, Now tab and Usage tab say `observed $… · N replies · comparison unavailable`.
|
|
112
|
+
Counts, tokens, activity/model/tier breakdowns and cost trends still include these replies. A mixed period labels the
|
|
113
|
+
comparable subset separately; its savings cover only those replies. `vs yours` is `—` for a row with incomplete
|
|
114
|
+
comparison coverage, and **What if** stays unavailable rather than inventing a baseline. This is common with automatic
|
|
115
|
+
provider caches, whose future hit rates are unknown. No changes are needed in existing router files.
|
|
116
|
+
|
|
117
|
+
When the whole period is comparable, the answer comes first, in the band's words: `routing −$1.84 (−22%) vs Opus 5.5 · xhigh` for Session, `routing −$7.96 (−46%) vs each session’s own model` for the other periods, or `routing +$2.48 (×5.0) vs Haiku 5.5 · high` when routing cost more. Below 10 routed replies it adds `early sample`, because one switch and its cache write can decide a short run. The ledger under it adds up to the routed cost in whole cents:
|
|
112
118
|
|
|
113
119
|
| Line | What it is |
|
|
114
120
|
| --------------------- | ---------------------------------------------------------------------------------------------- |
|
package/hooks/native-router.mjs
CHANGED
|
@@ -1176,14 +1176,15 @@ async function observeRouted($, router, { turnId, cfg, loop, loopVersion, reques
|
|
|
1176
1176
|
// Kept by model id, under the model that served it: a substitute's cost is not the requested model's, and a models
|
|
1177
1177
|
// key can name another model later.
|
|
1178
1178
|
const idOf = (model) => modelSpec(cfg, model)?.id ?? model;
|
|
1179
|
-
if (compared)
|
|
1179
|
+
if (compared || observedUsd !== null)
|
|
1180
1180
|
await addUsageEntry($, router, {
|
|
1181
1181
|
turnId,
|
|
1182
|
-
compared,
|
|
1182
|
+
compared: compared ?? { reply: null },
|
|
1183
|
+
observedUsd,
|
|
1183
1184
|
tier,
|
|
1184
1185
|
route: `${idOf(response.usage?.model ?? request.model)}@${request.effort ?? 'session'}`,
|
|
1185
1186
|
yours: idOf(nativeModel),
|
|
1186
|
-
byId: Object.fromEntries(Object.entries(compared
|
|
1187
|
+
byId: Object.fromEntries(Object.entries(compared?.replies ?? {}).map(([alias, r]) => [cfg.models[alias].id, r])),
|
|
1187
1188
|
usage: response.usage,
|
|
1188
1189
|
at,
|
|
1189
1190
|
});
|
|
@@ -1234,10 +1235,10 @@ async function compareYours($, router, loop, response, servedEffort, nativeModel
|
|
|
1234
1235
|
// A routed reply, priced against your model now, added to the turn's running replies, which the view counts at once
|
|
1235
1236
|
// and turn.complete adds to the usage record. `route` is the served model's `id@effort`, `yours` your model's id,
|
|
1236
1237
|
// `byId` the reply priced against each model as yours, keyed by model id; `at` when the reply arrived (ms).
|
|
1237
|
-
async function addUsageEntry($, router, { turnId, compared, tier, route, yours, byId, usage, at }) {
|
|
1238
|
+
async function addUsageEntry($, router, { turnId, compared, observedUsd, tier, route, yours, byId, usage, at }) {
|
|
1238
1239
|
try {
|
|
1239
1240
|
const view = router.view ?? (await readView($, router));
|
|
1240
|
-
if (compared.reply) {
|
|
1241
|
+
if (compared.reply || Number.isFinite(observedUsd)) {
|
|
1241
1242
|
const turn = router.turnActivities.get(turnId);
|
|
1242
1243
|
const entry = replyEntry({
|
|
1243
1244
|
at,
|
|
@@ -1247,6 +1248,7 @@ async function addUsageEntry($, router, { turnId, compared, tier, route, yours,
|
|
|
1247
1248
|
route,
|
|
1248
1249
|
yours,
|
|
1249
1250
|
reply: compared.reply,
|
|
1251
|
+
observedUsd,
|
|
1250
1252
|
replies: byId,
|
|
1251
1253
|
usage,
|
|
1252
1254
|
});
|
package/lib/band.mjs
CHANGED
|
@@ -10,6 +10,7 @@ import {
|
|
|
10
10
|
modelColor,
|
|
11
11
|
modelName,
|
|
12
12
|
NOT_A_TIER_REASON,
|
|
13
|
+
observedUsageText,
|
|
13
14
|
percentPair,
|
|
14
15
|
reasonActivity,
|
|
15
16
|
routeLabel,
|
|
@@ -218,10 +219,15 @@ export function bandSegments(config, view, usage, actions) {
|
|
|
218
219
|
// parts.
|
|
219
220
|
export function sessionRow(config, view, columns, now = Date.now()) {
|
|
220
221
|
const label = part('session ', { dimColor: true });
|
|
221
|
-
if (!modelSpec(config, view.nativeModel))
|
|
222
|
-
return [label, part(`${modelName(view.nativeModel)} has no list price`, { dimColor: true })];
|
|
223
222
|
const summary = sessionUsage(view, now);
|
|
224
223
|
const r = readout(summary);
|
|
224
|
+
if (summary.observed.uncomparedReplies)
|
|
225
|
+
return fitSegments(
|
|
226
|
+
[segment(0, label, part(observedUsageText(summary.observed), { dimColor: true }))],
|
|
227
|
+
columns,
|
|
228
|
+
).flatMap((s) => s.parts);
|
|
229
|
+
if (!modelSpec(config, view.nativeModel))
|
|
230
|
+
return [label, part(`${modelName(view.nativeModel)} has no list price`, { dimColor: true })];
|
|
225
231
|
if (!r.replies) return [part('no replies yet', { dimColor: true })];
|
|
226
232
|
const count = `${r.replies} repl${r.replies === 1 ? 'y' : 'ies'}`;
|
|
227
233
|
const answer = [
|
package/lib/display.mjs
CHANGED
|
@@ -51,7 +51,7 @@ export function unpricedLine(metrics) {
|
|
|
51
51
|
const missing = UNPRICED_CAUSES.filter((cause) => cause !== 'comparison');
|
|
52
52
|
const total = missing.reduce((n, cause) => n + unpriced[cause], 0);
|
|
53
53
|
const comparison = unpriced.comparison
|
|
54
|
-
? `Comparison unavailable: ${unpriced.comparison} routed repl${unpriced.comparison === 1 ? 'y' : 'ies'}; observed costs are in
|
|
54
|
+
? `Comparison unavailable: ${unpriced.comparison} routed repl${unpriced.comparison === 1 ? 'y' : 'ies'}; observed costs are in Usage`
|
|
55
55
|
: null;
|
|
56
56
|
if (!total) return comparison;
|
|
57
57
|
const causes = missing
|
|
@@ -345,6 +345,10 @@ export const liveUsage = (view, record) => withReplies(readUsage(record), view.u
|
|
|
345
345
|
// This session's routed replies, the running turns' included.
|
|
346
346
|
export const sessionUsage = (view, now) => summarize(liveUsage(view, view.usageSession), periodRange('session', now));
|
|
347
347
|
|
|
348
|
+
// Observed spend is valid even when a counterfactual cache history cannot be priced.
|
|
349
|
+
export const observedUsageText = (observed) =>
|
|
350
|
+
`observed ${usd(observed.routedUsd)} · ${observed.replies} repl${observed.replies === 1 ? 'y' : 'ies'} · comparison unavailable`;
|
|
351
|
+
|
|
348
352
|
// The models a session's replies were priced against, when they are not the one `/model` selects now: after a
|
|
349
353
|
// `/model` change the record keeps each reply against the model selected when it ran. Null when they are.
|
|
350
354
|
export function recordedModels(config, view, summary) {
|
|
@@ -378,10 +382,14 @@ export const switchesLost = (r) => r.switchUsd > 0 && r.switchUsd > -r.cheaperUs
|
|
|
378
382
|
export function savingsLine(config, view, now) {
|
|
379
383
|
const summary = sessionUsage(view, now);
|
|
380
384
|
const session = readout(summary);
|
|
385
|
+
if (summary.observed.uncomparedReplies)
|
|
386
|
+
return `Session: ${observedUsageText(summary.observed)}${session.replies ? `; ${replyCount(session.replies)} comparable only: ${routingAnswer(session)}` : ''}, est. at list prices`;
|
|
381
387
|
const label = `vs ${comparedModel(config, view, summary)}`;
|
|
382
388
|
if (!session.replies) {
|
|
383
389
|
const kept = allTime(view.usageStore, view.savingsStore);
|
|
384
390
|
const across = readout(kept.totals);
|
|
391
|
+
if (kept.observed.uncomparedReplies)
|
|
392
|
+
return `Session ${label}: no routed replies yet; across sessions ${observedUsageText(kept.observed)}, est. at list prices`;
|
|
385
393
|
const saved = across.early
|
|
386
394
|
? ''
|
|
387
395
|
: `; ${routingAnswer(across)} since ${sinceDate(kept.since)}, each session against its own model`;
|
package/lib/modes.mjs
CHANGED
|
@@ -3,19 +3,6 @@
|
|
|
3
3
|
import { ACTIVITIES, DEFAULTS, EFFORTS, ROUTING_MODES, rank, resolveRoute, sameRoute, TIERS } from './config.mjs';
|
|
4
4
|
import { clampEffort } from './cost.mjs';
|
|
5
5
|
|
|
6
|
-
// `wanted` on a model that lists `supported`: itself, else the next higher listed, else the highest lower one.
|
|
7
|
-
export function projectEffort(supported, wanted) {
|
|
8
|
-
if (supported.includes(wanted)) return wanted;
|
|
9
|
-
const at = EFFORTS.indexOf(wanted);
|
|
10
|
-
return (
|
|
11
|
-
EFFORTS.slice(at + 1).find((e) => supported.includes(e)) ??
|
|
12
|
-
EFFORTS.slice(0, at)
|
|
13
|
-
.reverse()
|
|
14
|
-
.find((e) => supported.includes(e)) ??
|
|
15
|
-
null
|
|
16
|
-
);
|
|
17
|
-
}
|
|
18
|
-
|
|
19
6
|
// The route's effort one level up (`step` 1) or down (-1), on what the model lists; null when it cannot move that
|
|
20
7
|
// way: a null effort (the session's), a model without efforts, or no listed level in that direction. The model's
|
|
21
8
|
// lowest listed level is the floor, so a route never loses its effort.
|
|
@@ -23,9 +10,9 @@ function steppedEffort(config, route, step) {
|
|
|
23
10
|
const supported = config.models[route.model].efforts;
|
|
24
11
|
const from = clampEffort(route.effort, supported);
|
|
25
12
|
if (!from) return null;
|
|
26
|
-
const
|
|
27
|
-
const
|
|
28
|
-
return
|
|
13
|
+
const at = EFFORTS.indexOf(from);
|
|
14
|
+
const choices = step < 0 ? EFFORTS.slice(0, at).reverse() : EFFORTS.slice(at + 1);
|
|
15
|
+
return choices.find((effort) => supported.includes(effort)) ?? null;
|
|
29
16
|
}
|
|
30
17
|
|
|
31
18
|
// The route `mode` runs for tier and activity, always from the configured routes. `cost` takes the tier below (never
|
package/lib/panel.mjs
CHANGED
|
@@ -30,6 +30,7 @@ import {
|
|
|
30
30
|
missingText,
|
|
31
31
|
modelColor,
|
|
32
32
|
modelName,
|
|
33
|
+
observedUsageText,
|
|
33
34
|
percent,
|
|
34
35
|
percentPair,
|
|
35
36
|
probeOf,
|
|
@@ -350,7 +351,8 @@ function nowTab(ui, config, view, usage, actions, _modelOptions, columns, now) {
|
|
|
350
351
|
out.push(ui.text(' '), ...sessionLines(ui, config, view, session, actions, columns, now));
|
|
351
352
|
// Today repeats this session's part of it until another session adds to it.
|
|
352
353
|
const ownToday = summarize(liveUsage(view, view.usageSession), periodRange('today', now));
|
|
353
|
-
if (today.replies > ownToday.replies)
|
|
354
|
+
if (today.observed.replies > ownToday.observed.replies)
|
|
355
|
+
out.push(ui.text(' '), ...todayLines(ui, today, actions, columns));
|
|
354
356
|
out.push(
|
|
355
357
|
...sessionAgentLines(ui, view, actions, columns),
|
|
356
358
|
...worthLines(ui, today, actions, columns),
|
|
@@ -488,17 +490,26 @@ function sessionLines(ui, config, view, summary, actions, columns, now) {
|
|
|
488
490
|
const switches = summary.switches.tier + summary.switches.activity;
|
|
489
491
|
const head = [
|
|
490
492
|
Number.isFinite(since) ? `since ${sinceText(since, now)}` : null,
|
|
491
|
-
summary.replies
|
|
492
|
-
? `${plural(summary.replies, 'routed reply', 'routed replies')}${view.usageLive?.length ? ` (${view.usageLive.length} live)` : ''}`
|
|
493
|
+
summary.observed.replies
|
|
494
|
+
? `${plural(summary.observed.replies, 'routed reply', 'routed replies')}${view.usageLive?.length ? ` (${view.usageLive.length} live)` : ''}`
|
|
493
495
|
: replies
|
|
494
496
|
? plural(replies, 'reply', 'replies')
|
|
495
497
|
: null,
|
|
496
498
|
summary.turns ? plural(switches, 'route switch', 'route switches') : null,
|
|
497
499
|
].filter(Boolean);
|
|
498
|
-
if (!strip && !summary.replies) return [ui.line(ui.head('SESSION'), ui.dim(' · no replies yet'))];
|
|
500
|
+
if (!strip && !summary.observed.replies) return [ui.line(ui.head('SESSION'), ui.dim(' · no replies yet'))];
|
|
499
501
|
const out = [ui.line(ui.head('SESSION'), head.length ? ui.dim(` · ${head.join(' · ')}`) : null)];
|
|
500
502
|
// The band's session row, then the parts that add up to it.
|
|
501
|
-
if (summary.
|
|
503
|
+
if (summary.observed.uncomparedReplies) {
|
|
504
|
+
out.push(
|
|
505
|
+
ui.line(
|
|
506
|
+
' ',
|
|
507
|
+
ui.text(observedUsageText(summary.observed)),
|
|
508
|
+
' ',
|
|
509
|
+
ui.Button({ key: 'usage-session', label: 'Usage', onPress: () => actions.usageFor('session') }),
|
|
510
|
+
),
|
|
511
|
+
);
|
|
512
|
+
} else if (summary.replies) {
|
|
502
513
|
const r = readout(summary);
|
|
503
514
|
out.push(
|
|
504
515
|
answerLine(ui, r, comparedModel(config, view, summary), () => actions.usageFor('session'), 'session', columns),
|
|
@@ -524,7 +535,7 @@ function sessionLines(ui, config, view, summary, actions, columns, now) {
|
|
|
524
535
|
),
|
|
525
536
|
);
|
|
526
537
|
}
|
|
527
|
-
const used = breakdown(summary.cells, 'activity')
|
|
538
|
+
const used = breakdown(summary.observed.cells, 'activity')
|
|
528
539
|
.filter((group) => ACTIVITIES.includes(group.name))
|
|
529
540
|
.sort((a, b) => b.sums[0] - a.sums[0])
|
|
530
541
|
.map((group) => [group.name, `${activityVerb(group.name)} ${group.sums[0]}`]);
|
|
@@ -595,16 +606,22 @@ function todayLines(ui, summary, actions, columns) {
|
|
|
595
606
|
ui.line(
|
|
596
607
|
ui.head('TODAY'),
|
|
597
608
|
ui.dim(
|
|
598
|
-
` · ${plural(summary.replies, 'routed reply', 'routed replies')} · ${plural(switches, 'route switch', 'route switches')}`,
|
|
609
|
+
` · ${plural(summary.observed.replies, 'routed reply', 'routed replies')} · ${plural(switches, 'route switch', 'route switches')}`,
|
|
599
610
|
),
|
|
600
611
|
),
|
|
601
|
-
|
|
612
|
+
summary.observed.uncomparedReplies
|
|
613
|
+
? ui.line(
|
|
614
|
+
ui.text(observedUsageText(summary.observed)),
|
|
615
|
+
' ',
|
|
616
|
+
ui.Button({ key: 'usage-today', label: 'Usage', onPress: () => actions.usageFor('today') }),
|
|
617
|
+
)
|
|
618
|
+
: answerLine(ui, readout(summary), 'each session’s own model', () => actions.usageFor('today'), 'today', columns),
|
|
602
619
|
];
|
|
603
620
|
}
|
|
604
621
|
|
|
605
622
|
// Today's first finding from Usage's WORTH A LOOK, with the tab that changes it; nothing when there is none.
|
|
606
623
|
function worthLines(ui, summary, actions, columns) {
|
|
607
|
-
const [first] = summary.replies >= INSIGHT_REPLIES ? insights(summary) : [];
|
|
624
|
+
const [first] = !summary.observed.uncomparedReplies && summary.replies >= INSIGHT_REPLIES ? insights(summary) : [];
|
|
608
625
|
if (!first) return [];
|
|
609
626
|
return [ui.text(' '), ui.line(ui.head('WORTH A LOOK'), ui.dim(' · today')), insightLine(ui, first, actions, columns)];
|
|
610
627
|
}
|
|
@@ -708,6 +725,15 @@ function routeSelects(ui, config, key, route, modelOptions, onModel, onEffort) {
|
|
|
708
725
|
];
|
|
709
726
|
}
|
|
710
727
|
|
|
728
|
+
const GRID_LABEL = 13;
|
|
729
|
+
|
|
730
|
+
function fitRouteCell(label, width) {
|
|
731
|
+
if (label.length <= width) return label;
|
|
732
|
+
const mark = /[°●]+$/.exec(label)?.[0] ?? '';
|
|
733
|
+
if (mark.length >= width) return mark.slice(-width);
|
|
734
|
+
return `${label.slice(0, Math.max(0, width - 1 - mark.length))}…${mark}`;
|
|
735
|
+
}
|
|
736
|
+
|
|
711
737
|
// Routes per cache: the tiers' routes, and the overrides when `overrides` is set. The count shows the overrides once
|
|
712
738
|
// activity routing is on; the one-more-cache warning always counts them, so it reads the same in every mode. `shared`
|
|
713
739
|
// says, per counted route, whether its model keeps one cache across efforts.
|
|
@@ -735,7 +761,9 @@ function routesTab(ui, config, view, _usage, actions, modelOptions, columns) {
|
|
|
735
761
|
const draft = { ...config, ...changes.draft };
|
|
736
762
|
const mode = draft.activityRouting;
|
|
737
763
|
const narrow = Number.isFinite(columns) && columns < 80;
|
|
738
|
-
const width =
|
|
764
|
+
const width = Number.isFinite(columns)
|
|
765
|
+
? Math.max(1, Math.min(narrow ? 14 : 16, Math.floor((columns - GRID_LABEL) / TIERS.length)))
|
|
766
|
+
: 16;
|
|
739
767
|
const { uses, shared } = cacheUses(draft, mode === 'on');
|
|
740
768
|
const selected = view.routeCell ?? null;
|
|
741
769
|
const cellButton = (row, tier, text, dim) =>
|
|
@@ -743,13 +771,13 @@ function routesTab(ui, config, view, _usage, actions, modelOptions, columns) {
|
|
|
743
771
|
width,
|
|
744
772
|
ui.Button({
|
|
745
773
|
key: `cell-${row}-${tier}`,
|
|
746
|
-
label: `${selected === `${row}.${tier}` ? '▸' : ' '}${text}`,
|
|
774
|
+
label: fitRouteCell(`${selected === `${row}.${tier}` ? '▸' : ' '}${text}`, width),
|
|
747
775
|
plain: true,
|
|
748
776
|
...(dim ? { dimColor: true } : {}),
|
|
749
777
|
onPress: () => actions.selectCell(`${row}.${tier}`),
|
|
750
778
|
}),
|
|
751
779
|
);
|
|
752
|
-
const gridRow = (name, cells) => ui.line(ui.cell(
|
|
780
|
+
const gridRow = (name, cells) => ui.line(ui.cell(GRID_LABEL, name), ...cells);
|
|
753
781
|
const out = [
|
|
754
782
|
ui.line(ui.head('ROUTES'), ui.dim(' · which model runs, by tier and activity · edit, then Save')),
|
|
755
783
|
ui.line(
|
|
@@ -770,7 +798,10 @@ function routesTab(ui, config, view, _usage, actions, modelOptions, columns) {
|
|
|
770
798
|
),
|
|
771
799
|
...overridesState(ui, view, config.activityRouting, mode),
|
|
772
800
|
ui.text(' '),
|
|
773
|
-
ui.line(
|
|
801
|
+
ui.line(
|
|
802
|
+
ui.cell(GRID_LABEL, ''),
|
|
803
|
+
...TIERS.map((tier) => ui.cell(width, ui.tier(tier, fitRouteCell(` ${tier}`, width)))),
|
|
804
|
+
),
|
|
774
805
|
gridRow(
|
|
775
806
|
ui.text('every turn', { bold: true }),
|
|
776
807
|
TIERS.map((tier) =>
|
|
@@ -1694,7 +1725,26 @@ function scopeLine(ui, config, view, period, summary, now) {
|
|
|
1694
1725
|
}
|
|
1695
1726
|
|
|
1696
1727
|
// The answer first, in the band's words: "routing −$1.84 (−22%) vs Opus 5.5 · xhigh", or why there is nothing to compare.
|
|
1697
|
-
function headline(ui, config, view, period, r, summary) {
|
|
1728
|
+
function headline(ui, config, view, period, r, summary, observed = summary.observed) {
|
|
1729
|
+
if (observed.uncomparedReplies)
|
|
1730
|
+
return [
|
|
1731
|
+
ui.text(observedUsageText(observed), { bold: true }),
|
|
1732
|
+
ui.dim(
|
|
1733
|
+
`${observed.uncomparedReplies} of ${observed.replies} replies have no comparison; savings are not estimated for them.`,
|
|
1734
|
+
),
|
|
1735
|
+
...(r.replies
|
|
1736
|
+
? [
|
|
1737
|
+
ui.dim(`${plural(r.replies, 'comparable reply', 'comparable replies')} only:`),
|
|
1738
|
+
ui.line(
|
|
1739
|
+
...answerParts(
|
|
1740
|
+
ui,
|
|
1741
|
+
r,
|
|
1742
|
+
period === 'session' ? comparedModel(config, view, summary) : 'each session’s own model',
|
|
1743
|
+
),
|
|
1744
|
+
),
|
|
1745
|
+
]
|
|
1746
|
+
: []),
|
|
1747
|
+
];
|
|
1698
1748
|
if (!r.replies) {
|
|
1699
1749
|
// A reply is priced against your model, so on a model without a price none is counted.
|
|
1700
1750
|
if (period === 'session' && !modelSpec(config, view.nativeModel))
|
|
@@ -1830,7 +1880,9 @@ function breakdownLines(ui, view, actions, summary, columns) {
|
|
|
1830
1880
|
ui.dim(' g next'),
|
|
1831
1881
|
),
|
|
1832
1882
|
];
|
|
1833
|
-
const
|
|
1883
|
+
const cells = summary.observed.cells;
|
|
1884
|
+
const groups = breakdown(cells, by);
|
|
1885
|
+
const missing = new Map(breakdown(summary.observed.uncompared, by).map((g) => [g.name, g]));
|
|
1834
1886
|
if (!groups.length) return [...out, ui.dim(' no routed replies in this period')];
|
|
1835
1887
|
const tail = !Number.isFinite(columns) || columns >= BREAKDOWN + TAIL;
|
|
1836
1888
|
const tailHead = by === 'model' ? 'share' : 'mostly on';
|
|
@@ -1839,14 +1891,14 @@ function breakdownLines(ui, view, actions, summary, columns) {
|
|
|
1839
1891
|
` ${''.padEnd(NAME)}${'replies'.padStart(8)}${'cost'.padStart(10)}${'vs yours'.padStart(11)}${tail ? ` ${tailHead}` : ''}`,
|
|
1840
1892
|
),
|
|
1841
1893
|
);
|
|
1842
|
-
const numbers = (sums) => [
|
|
1894
|
+
const numbers = (sums, unavailable) => [
|
|
1843
1895
|
`${String(sums[0]).padStart(8)}${usd(sums[1]).padStart(10)}`,
|
|
1844
|
-
differenceText(ui, sums[1] - sums[2], 11),
|
|
1896
|
+
unavailable ? ui.dim('—'.padStart(11)) : differenceText(ui, sums[1] - sums[2], 11),
|
|
1845
1897
|
];
|
|
1846
1898
|
for (const group of groups) {
|
|
1847
1899
|
const id = `${by}:${group.name}`;
|
|
1848
1900
|
const open = view.usageOpen === id;
|
|
1849
|
-
const route = by === 'model' ? null : mostlyOn(cellsWhere(
|
|
1901
|
+
const route = by === 'model' ? null : mostlyOn(cellsWhere(cells, by, group.name));
|
|
1850
1902
|
out.push(
|
|
1851
1903
|
ui.line(
|
|
1852
1904
|
' ',
|
|
@@ -1858,11 +1910,11 @@ function breakdownLines(ui, view, actions, summary, columns) {
|
|
|
1858
1910
|
}),
|
|
1859
1911
|
' ',
|
|
1860
1912
|
ui.cell(NAME, groupName(ui, by, group.name)),
|
|
1861
|
-
...numbers(group.sums),
|
|
1913
|
+
...numbers(group.sums, missing.has(group.name)),
|
|
1862
1914
|
!tail
|
|
1863
1915
|
? null
|
|
1864
1916
|
: by === 'model'
|
|
1865
|
-
? ui.dim(` ${percent(summary.routedUsd > 0 ? group.sums[1] / summary.routedUsd : 0)}`)
|
|
1917
|
+
? ui.dim(` ${percent(summary.observed.routedUsd > 0 ? group.sums[1] / summary.observed.routedUsd : 0)}`)
|
|
1866
1918
|
: route
|
|
1867
1919
|
? ui.dim(` ${routeText(route)}`)
|
|
1868
1920
|
: null,
|
|
@@ -1870,12 +1922,23 @@ function breakdownLines(ui, view, actions, summary, columns) {
|
|
|
1870
1922
|
);
|
|
1871
1923
|
if (!open) continue;
|
|
1872
1924
|
for (const child of group.children)
|
|
1873
|
-
out.push(
|
|
1925
|
+
out.push(
|
|
1926
|
+
ui.line(
|
|
1927
|
+
' ',
|
|
1928
|
+
ui.cell(NAME - 2, groupName(ui, CHILD_BY[by], child.name)),
|
|
1929
|
+
...numbers(
|
|
1930
|
+
child.sums,
|
|
1931
|
+
missing.get(group.name)?.children.some((c) => c.name === child.name),
|
|
1932
|
+
),
|
|
1933
|
+
),
|
|
1934
|
+
);
|
|
1874
1935
|
const [, , , cheaper, stronger, switched, input, , output] = group.sums;
|
|
1875
1936
|
out.push(
|
|
1876
1937
|
...dimLines(
|
|
1877
1938
|
ui,
|
|
1878
|
-
|
|
1939
|
+
missing.has(group.name)
|
|
1940
|
+
? ` comparison unavailable · input ${formatTokens(input)} · output ${formatTokens(output)}`
|
|
1941
|
+
: ` cheaper ${signedUsd(cents(cheaper))} · stronger ${signedUsd(cents(stronger))} · switch ${signedUsd(cents(switched))} · input ${formatTokens(input)} · output ${formatTokens(output)}`,
|
|
1879
1942
|
columns,
|
|
1880
1943
|
),
|
|
1881
1944
|
);
|
|
@@ -1886,6 +1949,8 @@ function breakdownLines(ui, view, actions, summary, columns) {
|
|
|
1886
1949
|
// What the period's tokens would cost on each model alone, as your model for every reply (compareModels), beside what
|
|
1887
1950
|
// routing spent and your own models; a model some reply had no price on is left out.
|
|
1888
1951
|
function compareLines(ui, summary) {
|
|
1952
|
+
if (summary.observed.uncomparedReplies)
|
|
1953
|
+
return [ui.dim('What if unavailable: some replies have no comparable cache history.')];
|
|
1889
1954
|
const yours = Object.keys(summary.yours);
|
|
1890
1955
|
const single = yours.length === 1 ? yours[0] : null;
|
|
1891
1956
|
const rows = Object.entries(summary.whatIf)
|
|
@@ -1947,7 +2012,7 @@ function trendLines(ui, view, actions, summary, period, now) {
|
|
|
1947
2012
|
// The evidence worth a look, each with the tab that changes it: the cell that carried most of the cost, the switch
|
|
1948
2013
|
// writes against the saving, the cell that ran above your model. Never which model to use.
|
|
1949
2014
|
function insightLines(ui, summary, actions, columns) {
|
|
1950
|
-
if (summary.replies < INSIGHT_REPLIES) return [];
|
|
2015
|
+
if (summary.observed.uncomparedReplies || summary.replies < INSIGHT_REPLIES) return [];
|
|
1951
2016
|
const list = insights(summary);
|
|
1952
2017
|
if (!list.length) return [];
|
|
1953
2018
|
return [
|
|
@@ -2021,12 +2086,12 @@ function usageTab(ui, config, view, _usage, actions, _modelOptions, columns, now
|
|
|
2021
2086
|
const legacy = kept?.legacy ?? null;
|
|
2022
2087
|
const r = readout(kept?.totals ?? summary);
|
|
2023
2088
|
const out = [periodRow(ui, period, actions), scopeLine(ui, config, view, period, summary, now), ui.text(' ')];
|
|
2024
|
-
out.push(...headline(ui, config, view, period, r, summary));
|
|
2089
|
+
out.push(...headline(ui, config, view, period, r, summary, kept?.observed ?? summary.observed));
|
|
2025
2090
|
if (r.replies) {
|
|
2026
2091
|
out.push(ui.text(' '), ...ledger(ui, config, view, { period, r, summary, legacy, columns }));
|
|
2027
|
-
if (summary.replies) out.push(ui.text(' '), statsLine(ui, summary));
|
|
2028
2092
|
}
|
|
2029
|
-
if (summary.replies) {
|
|
2093
|
+
if (summary.observed.replies) out.push(ui.text(' '), statsLine(ui, { ...summary.observed, turns: summary.turns }));
|
|
2094
|
+
if (summary.observed.replies) {
|
|
2030
2095
|
out.push(ui.text(' '), ...breakdownLines(ui, view, actions, summary, columns), ui.text(' '));
|
|
2031
2096
|
out.push(
|
|
2032
2097
|
...(view.usageCompare
|
|
@@ -2061,6 +2126,6 @@ function helpLines(ui) {
|
|
|
2061
2126
|
ui.dim('Routing vs your model holds tokens and output length the same'),
|
|
2062
2127
|
ui.dim('and does not measure answer quality. Cache benefit is before writes.'),
|
|
2063
2128
|
ui.dim('The context bar uses the routed model’s window, 20% in reserve.'),
|
|
2064
|
-
ui.dim('
|
|
2129
|
+
ui.dim('Cache estimates follow the selected provider profile; minus means cheaper.'),
|
|
2065
2130
|
];
|
|
2066
2131
|
}
|
package/lib/providers.mjs
CHANGED
|
@@ -18,6 +18,94 @@ const agents = (light, heavy) => ({
|
|
|
18
18
|
});
|
|
19
19
|
|
|
20
20
|
export const PROVIDER_DEFAULTS = {
|
|
21
|
+
minimax: {
|
|
22
|
+
models: {
|
|
23
|
+
flash: {
|
|
24
|
+
id: 'MiniMax-M3.1-Flash[1m]',
|
|
25
|
+
input: 0.15,
|
|
26
|
+
output: 0.6,
|
|
27
|
+
cacheRead: 0.015,
|
|
28
|
+
contextWindow: 1_000_000,
|
|
29
|
+
billing: 'credits',
|
|
30
|
+
// xhigh is documented but some Claude Code versions serialize it as high; use max instead.
|
|
31
|
+
efforts: ['low', 'medium', 'high', 'max'],
|
|
32
|
+
strength: 1,
|
|
33
|
+
imageInput: true,
|
|
34
|
+
},
|
|
35
|
+
},
|
|
36
|
+
routes: {
|
|
37
|
+
micro: route('flash', 'low'),
|
|
38
|
+
low: route('flash', 'medium'),
|
|
39
|
+
medium: route('flash', 'high'),
|
|
40
|
+
high: route('flash', 'max'),
|
|
41
|
+
},
|
|
42
|
+
activities: Object.fromEntries(
|
|
43
|
+
['code', 'debug', 'plan', 'review'].map((a) => [a, { medium: route('flash', 'max') }]),
|
|
44
|
+
),
|
|
45
|
+
subagents: agents('flash', 'flash'),
|
|
46
|
+
cache: AUTOMATIC,
|
|
47
|
+
},
|
|
48
|
+
mimo: {
|
|
49
|
+
models: {
|
|
50
|
+
flash: {
|
|
51
|
+
id: 'mimo-v2.6-flash[1m]',
|
|
52
|
+
input: 0.14,
|
|
53
|
+
output: 0.28,
|
|
54
|
+
cacheRead: 0.0028,
|
|
55
|
+
contextWindow: 1_000_000,
|
|
56
|
+
billing: 'credits',
|
|
57
|
+
// The Anthropic endpoint documents binary thinking, not granular effort. Keep its default thinking.
|
|
58
|
+
efforts: [],
|
|
59
|
+
strength: 1,
|
|
60
|
+
imageInput: true,
|
|
61
|
+
},
|
|
62
|
+
pro: {
|
|
63
|
+
id: 'mimo-v2.6-pro[1m]',
|
|
64
|
+
input: 0.435,
|
|
65
|
+
output: 0.87,
|
|
66
|
+
cacheRead: 0.0036,
|
|
67
|
+
contextWindow: 1_000_000,
|
|
68
|
+
billing: 'credits',
|
|
69
|
+
efforts: [],
|
|
70
|
+
strength: 2,
|
|
71
|
+
imageInput: true,
|
|
72
|
+
},
|
|
73
|
+
},
|
|
74
|
+
routes: {
|
|
75
|
+
micro: route('flash', null),
|
|
76
|
+
low: route('flash', null),
|
|
77
|
+
medium: route('pro', null),
|
|
78
|
+
high: route('pro', null),
|
|
79
|
+
},
|
|
80
|
+
activities: Object.fromEntries(['ops', 'explore', 'docs'].map((a) => [a, { medium: route('flash', null) }])),
|
|
81
|
+
subagents: agents('flash', 'pro'),
|
|
82
|
+
cache: AUTOMATIC,
|
|
83
|
+
},
|
|
84
|
+
kimi: {
|
|
85
|
+
// One model throughout the conversation: K3-family thinking/signatures must survive tool continuations.
|
|
86
|
+
models: {
|
|
87
|
+
k3: {
|
|
88
|
+
id: 'kimi-k3.1-pro[1m]',
|
|
89
|
+
input: 2,
|
|
90
|
+
output: 10,
|
|
91
|
+
cacheRead: 0.2,
|
|
92
|
+
contextWindow: 1_048_576,
|
|
93
|
+
billing: 'credits',
|
|
94
|
+
efforts: EFFORTS,
|
|
95
|
+
strength: 1,
|
|
96
|
+
imageInput: true,
|
|
97
|
+
},
|
|
98
|
+
},
|
|
99
|
+
routes: {
|
|
100
|
+
micro: route('k3', 'low'),
|
|
101
|
+
low: route('k3', 'high'),
|
|
102
|
+
medium: route('k3', 'high'),
|
|
103
|
+
high: route('k3', 'max'),
|
|
104
|
+
},
|
|
105
|
+
activities: Object.fromEntries(['code', 'debug', 'plan', 'review'].map((a) => [a, { medium: route('k3', 'max') }])),
|
|
106
|
+
subagents: agents('k3', 'k3'),
|
|
107
|
+
cache: AUTOMATIC,
|
|
108
|
+
},
|
|
21
109
|
deepseek: {
|
|
22
110
|
models: {
|
|
23
111
|
flash: {
|
package/lib/savings.mjs
CHANGED
|
@@ -116,10 +116,14 @@ export function compareReply(config, state, { usage, served, yours, ttl, now })
|
|
|
116
116
|
const total = counts.input + counts.cacheRead + counts.cacheWrite;
|
|
117
117
|
const [servedSpec, yoursSpec] = [modelSpec(config, served.model), modelSpec(config, yours.model)];
|
|
118
118
|
const sameModel = servedSpec ? servedSpec === yoursSpec : served.model === yours.model;
|
|
119
|
-
const identity = {
|
|
119
|
+
const identity = {
|
|
120
|
+
model: yoursSpec?.id ?? yours.model,
|
|
121
|
+
effort: yoursSpec?.efforts.length === 0 ? null : (yours.effort ?? null),
|
|
122
|
+
};
|
|
123
|
+
const servedEffort = servedSpec?.efforts.length === 0 ? null : (served.effort ?? null);
|
|
120
124
|
// On a model that keeps one cache across efforts (effortSharesCache), an effort-only difference is the same cache.
|
|
121
125
|
const sharedEffort = effortSharesCache(config, identity.model);
|
|
122
|
-
const onYours = sameModel && (sharedEffort ||
|
|
126
|
+
const onYours = sameModel && (sharedEffort || servedEffort === identity.effort);
|
|
123
127
|
const next = { total, at: now, onYours, ...identity };
|
|
124
128
|
// A cache simulated for another model or effort of yours is none of this one's: the new one starts cold.
|
|
125
129
|
const prior = state?.model === identity.model && (sharedEffort || state.effort === identity.effort) ? state : null;
|
package/lib/usage.mjs
CHANGED
|
@@ -36,7 +36,7 @@ const IS_USD = CELL_SUMS.map((sum) => USD.has(sum));
|
|
|
36
36
|
export const DAY = /^\d{4}-\d{2}-\d{2}$/;
|
|
37
37
|
// `mode|activity|tier|id@effort`: mode, activity and tier never hold a `|`, and an effort never an `@`, so a model id
|
|
38
38
|
// may hold either. Version 1 records, from before routing modes, had no mode: every reply then ran `balanced`.
|
|
39
|
-
export const USAGE_VERSION =
|
|
39
|
+
export const USAGE_VERSION = 3;
|
|
40
40
|
const CELL_KEY = /^[^|]+\|[^|]+\|[^|]+\|.+@[^@]+$/;
|
|
41
41
|
const V1_CELL_KEY = /^[^|]+\|[^|]+\|.+@[^@]+$/;
|
|
42
42
|
const V1_MODE = 'balanced';
|
|
@@ -66,31 +66,54 @@ export function emptyUsage() {
|
|
|
66
66
|
return { version: USAGE_VERSION, since: null, days: {} };
|
|
67
67
|
}
|
|
68
68
|
|
|
69
|
-
const emptyDay = () => ({
|
|
69
|
+
const emptyDay = () => ({
|
|
70
|
+
cells: {},
|
|
71
|
+
uncompared: {},
|
|
72
|
+
whatIf: {},
|
|
73
|
+
yours: {},
|
|
74
|
+
hours: {},
|
|
75
|
+
turns: 0,
|
|
76
|
+
switches: { tier: 0, activity: 0 },
|
|
77
|
+
});
|
|
70
78
|
|
|
71
79
|
// One routed main reply as its turn collects it. `at` is when it arrived (ms); `mode` the turn's routing mode;
|
|
72
80
|
// `activity` the turn's activity and
|
|
73
81
|
// `tier` the tier that served it (null: none); `route` the model id that served it and the request's effort,
|
|
74
82
|
// `id@effort` (`session` for no effort); `yours` your model's id; `reply` its prices against your model, and `replies`
|
|
75
83
|
// against each model as yours keyed by model id, from compareModels; `usage` the API usage, whose counts compareReply
|
|
76
|
-
// already checked
|
|
77
|
-
|
|
84
|
+
// already checked, or observedReplyUsd checked them when only observedUsd is available. A false `compared` entry
|
|
85
|
+
// goes in the uncompared lane; its zero comparison fields are never savings. Model ids, not models keys.
|
|
86
|
+
export function replyEntry({
|
|
87
|
+
at,
|
|
88
|
+
mode,
|
|
89
|
+
activity,
|
|
90
|
+
tier,
|
|
91
|
+
route,
|
|
92
|
+
yours,
|
|
93
|
+
reply,
|
|
94
|
+
replies,
|
|
95
|
+
usage,
|
|
96
|
+
observedUsd = /** @type {number | null} */ (null),
|
|
97
|
+
}) {
|
|
98
|
+
if (!reply && !(Number.isFinite(observedUsd) && observedUsd >= 0))
|
|
99
|
+
throw new Error('Observed usage requires a known price');
|
|
78
100
|
const input = usage.input_tokens + usage.cache_read_input_tokens + usage.cache_creation_input_tokens;
|
|
79
101
|
return {
|
|
80
102
|
at,
|
|
103
|
+
...(!reply ? { compared: false } : {}),
|
|
81
104
|
cell: `${mode}|${activity ?? 'none'}|${tier ?? 'none'}|${route}`,
|
|
82
105
|
sums: [
|
|
83
106
|
1,
|
|
84
|
-
reply
|
|
85
|
-
reply
|
|
86
|
-
reply
|
|
87
|
-
reply
|
|
88
|
-
reply
|
|
107
|
+
reply?.routedUsd ?? observedUsd,
|
|
108
|
+
reply?.yoursUsd ?? 0,
|
|
109
|
+
reply?.cheaperUsd ?? 0,
|
|
110
|
+
reply?.strongerUsd ?? 0,
|
|
111
|
+
reply?.switchUsd ?? 0,
|
|
89
112
|
input,
|
|
90
113
|
usage.cache_read_input_tokens,
|
|
91
114
|
usage.output_tokens,
|
|
92
115
|
],
|
|
93
|
-
whatIf: Object.fromEntries(Object.entries(replies).map(([model, r]) => [model, r.yoursUsd])),
|
|
116
|
+
whatIf: reply ? Object.fromEntries(Object.entries(replies).map(([model, r]) => [model, r.yoursUsd])) : {},
|
|
94
117
|
yours,
|
|
95
118
|
};
|
|
96
119
|
}
|
|
@@ -132,7 +155,8 @@ function addEntries(recordDays, entries) {
|
|
|
132
155
|
copied.add(key);
|
|
133
156
|
}
|
|
134
157
|
const day = days[key];
|
|
135
|
-
|
|
158
|
+
const cells = entry.compared === false ? day.uncompared : day.cells;
|
|
159
|
+
cells[entry.cell] = cells[entry.cell] ? addSums(cells[entry.cell], entry.sums) : [...entry.sums];
|
|
136
160
|
for (const [model, usd] of Object.entries(entry.whatIf)) {
|
|
137
161
|
const [n, sum] = own(day.whatIf, model) ?? [0, 0];
|
|
138
162
|
put(day.whatIf, model, [n + 1, microUsd(sum + usd)]);
|
|
@@ -161,6 +185,15 @@ function readDay(day, version) {
|
|
|
161
185
|
isRecord(day) &&
|
|
162
186
|
isRecord(day.cells) &&
|
|
163
187
|
Object.entries(day.cells).every(([cell, sums]) => key.test(cell) && isSums(sums)) &&
|
|
188
|
+
(version < 3 ||
|
|
189
|
+
(isRecord(day.uncompared) &&
|
|
190
|
+
Object.entries(day.uncompared).every(
|
|
191
|
+
([cell, sums]) =>
|
|
192
|
+
key.test(cell) &&
|
|
193
|
+
isSums(sums) &&
|
|
194
|
+
sums.slice(2, 6).every((n) => n === 0) &&
|
|
195
|
+
[0, 6, 7, 8].every((i) => isCount(sums[i])),
|
|
196
|
+
))) &&
|
|
164
197
|
isRecord(day.whatIf) &&
|
|
165
198
|
Object.values(day.whatIf).every(isPair) &&
|
|
166
199
|
isRecord(day.yours) &&
|
|
@@ -177,15 +210,16 @@ function readDay(day, version) {
|
|
|
177
210
|
? {
|
|
178
211
|
...day,
|
|
179
212
|
cells: Object.fromEntries(Object.entries(day.cells).map(([cell, sums]) => [`${V1_MODE}|${cell}`, sums])),
|
|
213
|
+
uncompared: {},
|
|
180
214
|
}
|
|
181
|
-
: day;
|
|
215
|
+
: { ...day, uncompared: version < 3 ? {} : day.uncompared };
|
|
182
216
|
}
|
|
183
217
|
|
|
184
218
|
// The stored value read day by day: a missing or unknown record is empty, and a day that does not read is dropped
|
|
185
|
-
// while the others stay, so a hand edit never breaks a turn or the pane. A version 1 record
|
|
186
|
-
// replies under `balanced
|
|
219
|
+
// while the others stay, so a hand edit never breaks a turn or the pane. A version 1 record keeps its
|
|
220
|
+
// replies under `balanced`. Versions 1 and 2 gain an empty observed-only lane on the next version 3 write.
|
|
187
221
|
export function readUsage(value) {
|
|
188
|
-
if (!isRecord(value) || ![1, USAGE_VERSION].includes(value.version) || !isRecord(value.days)) return emptyUsage();
|
|
222
|
+
if (!isRecord(value) || ![1, 2, USAGE_VERSION].includes(value.version) || !isRecord(value.days)) return emptyUsage();
|
|
189
223
|
if (!(value.since === null || Number.isFinite(value.since))) return emptyUsage();
|
|
190
224
|
const days = {};
|
|
191
225
|
for (const [key, day] of Object.entries(value.days)) {
|
|
@@ -232,6 +266,8 @@ function mergeDay(a, b) {
|
|
|
232
266
|
const day = structuredClone(a);
|
|
233
267
|
for (const [key, sums] of Object.entries(b.cells))
|
|
234
268
|
day.cells[key] = day.cells[key] ? addSums(day.cells[key], sums) : [...sums];
|
|
269
|
+
for (const [key, sums] of Object.entries(b.uncompared ?? {}))
|
|
270
|
+
day.uncompared[key] = day.uncompared[key] ? addSums(day.uncompared[key], sums) : [...sums];
|
|
235
271
|
for (const [model, [n, usd]] of Object.entries(b.whatIf)) {
|
|
236
272
|
const [m, sum] = own(day.whatIf, model) ?? [0, 0];
|
|
237
273
|
put(day.whatIf, model, [m + n, microUsd(sum + usd)]);
|
|
@@ -280,7 +316,24 @@ export function summarize(record, { from, to }) {
|
|
|
280
316
|
addSums,
|
|
281
317
|
CELL_SUMS.map(() => 0),
|
|
282
318
|
);
|
|
319
|
+
const cells = { ...total.cells };
|
|
320
|
+
for (const [key, sums] of Object.entries(total.uncompared))
|
|
321
|
+
cells[key] = cells[key] ? addSums(cells[key], sums) : [...sums];
|
|
322
|
+
const observed = Object.values(cells).reduce(
|
|
323
|
+
addSums,
|
|
324
|
+
CELL_SUMS.map(() => 0),
|
|
325
|
+
);
|
|
283
326
|
return {
|
|
327
|
+
observed: {
|
|
328
|
+
replies: observed[0],
|
|
329
|
+
routedUsd: observed[1],
|
|
330
|
+
inputTokens: observed[6],
|
|
331
|
+
cacheReadTokens: observed[7],
|
|
332
|
+
outputTokens: observed[8],
|
|
333
|
+
cells,
|
|
334
|
+
uncompared: total.uncompared,
|
|
335
|
+
uncomparedReplies: Object.values(total.uncompared).reduce((n, sums) => n + sums[0], 0),
|
|
336
|
+
},
|
|
284
337
|
replies,
|
|
285
338
|
routedUsd,
|
|
286
339
|
yoursUsd,
|
|
@@ -310,6 +363,11 @@ export function allTime(usageStore, savingsStore) {
|
|
|
310
363
|
return {
|
|
311
364
|
totals: Object.fromEntries(READOUT_SUMS.map((key) => [key, summary[key] + legacy[key]])),
|
|
312
365
|
since: earlierSince(record.since, legacy.since),
|
|
366
|
+
observed: {
|
|
367
|
+
...summary.observed,
|
|
368
|
+
replies: summary.observed.replies + legacy.replies,
|
|
369
|
+
routedUsd: summary.observed.routedUsd + legacy.routedUsd,
|
|
370
|
+
},
|
|
313
371
|
legacy,
|
|
314
372
|
};
|
|
315
373
|
}
|
|
@@ -375,7 +433,10 @@ export function trend(summary, period, now) {
|
|
|
375
433
|
const first = summary.days[0][0];
|
|
376
434
|
const start = periodRange(period, now).from ?? first;
|
|
377
435
|
const routed = new Map(
|
|
378
|
-
summary.days.map(([key, day]) => [
|
|
436
|
+
summary.days.map(([key, day]) => [
|
|
437
|
+
key,
|
|
438
|
+
[...Object.values(day.cells), ...Object.values(day.uncompared ?? {})].reduce((s, c) => s + c[1], 0),
|
|
439
|
+
]),
|
|
379
440
|
);
|
|
380
441
|
const values = [];
|
|
381
442
|
const [y, m, d] = start.split('-').map(Number);
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@alexeiled/claude-router",
|
|
3
|
-
"version": "1.
|
|
4
|
-
"description": "Native Claude Code model and effort routing with Anthropic, DeepSeek
|
|
3
|
+
"version": "1.19.1",
|
|
4
|
+
"description": "Native Claude Code model and effort routing with Anthropic, DeepSeek, z.ai, MiniMax, MiMo and Kimi defaults, custom provider profiles, and classifier advice.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "Alexei Ledenev",
|
|
7
7
|
"repository": {
|
package/types/index.d.ts
CHANGED
|
@@ -164,6 +164,8 @@ declare module 'claude-code' {
|
|
|
164
164
|
// `hours` routed USD by local hour; `turns` and `switches` count turns on the day they ended.
|
|
165
165
|
interface RouterUsageDay {
|
|
166
166
|
cells: Record<string, number[]>;
|
|
167
|
+
// Observed-only replies in version 3: same sums, with comparison fields 2..5 unused (zero).
|
|
168
|
+
uncompared?: Record<string, number[]>;
|
|
167
169
|
whatIf: Record<string, [number, number]>;
|
|
168
170
|
yours: Record<string, number>;
|
|
169
171
|
hours: Record<string, number>;
|
|
@@ -173,7 +175,7 @@ declare module 'claude-code' {
|
|
|
173
175
|
// Kept across sessions, one per session under store key 'usage:v1:<session id>' or their sum in the view, by day
|
|
174
176
|
// ('2026-10-13'), since its first turn written after the last reset (ms).
|
|
175
177
|
interface RouterUsageStore {
|
|
176
|
-
version: 1 | 2;
|
|
178
|
+
version: 1 | 2 | 3;
|
|
177
179
|
since: number | null;
|
|
178
180
|
days: Record<string, RouterUsageDay>;
|
|
179
181
|
}
|