@alexeiled/claude-router 1.19.0 → 1.19.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +23 -0
- package/README.md +3 -1
- package/docs/architecture.md +9 -0
- package/docs/configuration.md +5 -2
- package/docs/evaluation.md +8 -0
- package/docs/native-router.md +4 -0
- package/docs/providers.md +96 -2
- package/docs/user-guide.md +8 -2
- package/hooks/native-router.mjs +7 -5
- package/lib/band.mjs +8 -2
- package/lib/display.mjs +9 -1
- package/lib/panel.mjs +92 -27
- package/lib/savings.mjs +6 -2
- package/lib/usage.mjs +77 -16
- package/package.json +1 -1
- package/types/index.d.ts +3 -1
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "router",
|
|
3
3
|
"displayName": "Router",
|
|
4
|
-
"version": "1.19.
|
|
4
|
+
"version": "1.19.2",
|
|
5
5
|
"description": "Native model and effort routing within your provider. Anthropic, DeepSeek, z.ai, MiniMax, MiMo and Kimi defaults, custom provider profiles, classifier advice and one model per subagent.",
|
|
6
6
|
"author": {
|
|
7
7
|
"name": "Alexei Ledenev",
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,28 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.19.2 (2026-11-01)
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
|
|
7
|
+
- Document the explicit `model-proxy` custom profile for mixed Anthropic and DeepSeek sessions through a separately started native proxy. No built-in mixed-provider defaults, endpoint switching or credential handling are added to the Mod.
|
|
8
|
+
- Verify two-provider route switching, response-ID normalization, profile isolation and selected-file saves with the existing native hook harness.
|
|
9
|
+
|
|
10
|
+
### Upgrade
|
|
11
|
+
|
|
12
|
+
Restart Claude Code after updating. Existing profiles need no changes. To opt in, create a complete `model-proxy-router.json` in the selected account directory and follow the proxy setup guide. Every selected provider receives the unchanged conversation history.
|
|
13
|
+
|
|
14
|
+
## 1.19.1 (2026-10-31)
|
|
15
|
+
|
|
16
|
+
### Fixed
|
|
17
|
+
|
|
18
|
+
- Keep observed spend, replies, tokens, activity breakdowns and trends when provider cache behavior prevents a savings comparison. The band and pane say `comparison unavailable` instead of `no replies yet`. Mixed periods label the comparable subset; unknown savings remain unknown.
|
|
19
|
+
- Normalize unsupported effort when comparing MiMo routes. An inherited session effort no longer makes the same no-effort model look like a different cache identity.
|
|
20
|
+
- Fit the Routes grid to narrow panes. Long labels are shortened without losing their built-in or unsaved markers; the cell editor retains full details.
|
|
21
|
+
|
|
22
|
+
### Upgrade
|
|
23
|
+
|
|
24
|
+
Restart Claude Code after updating. No router-file changes are needed. Usage records migrate automatically to version 3, preserving older comparisons; replies omitted by older releases cannot be recovered. Earlier releases cannot display version 3 records.
|
|
25
|
+
|
|
3
26
|
## 1.19.0 (2026-10-31)
|
|
4
27
|
|
|
5
28
|
MiniMax, MiMo and Kimi gain built-in provider profiles alongside DeepSeek and z.ai.
|
package/README.md
CHANGED
|
@@ -31,7 +31,7 @@ OpenRouter, Ollama, other local servers and other compatible hosted providers us
|
|
|
31
31
|
They have no built-in model defaults. A selected file must exist and validate, or the router leaves Claude Code's model unchanged.
|
|
32
32
|
|
|
33
33
|
See [Providers](docs/providers.md) for **default routes and activities**, **custom profiles**, and the **`--settings` flag alternative**.
|
|
34
|
-
Execution provider selection is separate from classifier selection.
|
|
34
|
+
Execution provider selection is separate from classifier selection. Observed usage stays visible even when a provider's cache behavior makes savings comparisons unavailable.
|
|
35
35
|
|
|
36
36
|
## Supported classifiers
|
|
37
37
|
|
|
@@ -109,6 +109,8 @@ The Anthropic profile uses these activity defaults. Other profiles have [their o
|
|
|
109
109
|
|
|
110
110
|
A [routing mode](docs/routing-modes.md) trades cost against answer quality without new routes: `cost` takes a cheaper model or one effort level less where a turn allows it, `quality` one effort level more or the tier above, and `balanced`, the default, runs the routes as configured. Set it per session with `/router mode cost`, per project in `.claude/router.json`, or for the profile with `"mode"` in `router.json`.
|
|
111
111
|
|
|
112
|
+
For mixed Anthropic and DeepSeek sessions, use the explicit [model-proxy profile](docs/providers.md#mix-anthropic-and-deepseek-with-model-proxy) with a separately started local proxy. No built-in mixed-provider defaults are installed.
|
|
113
|
+
|
|
112
114
|
## Install
|
|
113
115
|
|
|
114
116
|
You need Claude Code 2.1.289 or newer and credentials for one [supported classifier](#supported-classifiers).
|
package/docs/architecture.md
CHANGED
|
@@ -135,6 +135,15 @@ Nothing is recorded with `activityRouting: off`. Stats run inside a guard: a fai
|
|
|
135
135
|
|
|
136
136
|
After each main reply, where the response is observed, `savings.mjs` prices the reply on the model that served it and the same tokens on "your model": the session's model and the effort Claude Code sent on the turn's first request. `compareModels` prices each reply against every model in `router.json` as "your model", at the session effort. Each keeps a simulated single cache in the loop under its models key in `yoursBy` (`{ total, at, onYours, model, effort }`), which a history reset clears; after an effort change the next reply finds it cold. Each reply also becomes a `replyEntry` in `usage.mjs`: its time, its cell `mode|activity|tier|modelId@effort` with the model that served it, its prices against the model selected then, its input, cache-read and output tokens, and what the same tokens cost on each model as yours. Models are kept by id, not by models key, because a key can later name another model. Until the turn ends its entries are also the view's `usageLive`; every readout (the two-row band, the Now and Usage tabs, `/router`) adds them to the record with `withReplies`, so they count the same replies at the same moment. At turn completion the entries leave `turnUsage` in the same tick they enter the held record, and every view write of the records carries `usageLive` with them, so a pane opened mid-write, a failed store write or a Reset counts a turn once or not at all, never twice. At turn completion `recordUsage` adds the turn's entries to the session's own `$.store` key `usage:v1:<session id>`, by local calendar day: per day the cells' nine sums, the what-if sums, the replies per your model, the routed cost per hour, the turns, and the route switches by tier and by activity, with `since`, the record's first write. Days older than 400 days leave the record when it is written. Each key has one writer, and Claude Code 2.1.296 writes one key under a file lock (observed in its code; the `$.store` types do not promise it), so sessions that finish turns at the same moment do not overwrite each other. The view's `usageStore` adds up two parts: every other session's key, read at session start and when the pane opens, and this session's record (`usageSession`), which the router reads once, holds in memory, adds each finished turn to and then writes, so a pane opened during that write counts the turn once. On that read, `prunedUsage` names other sessions' records with no day in the last 400 days and, past 1,000 records, those whose last day is oldest; a record that does not read whole, such as one a later version wrote, is never named. Each named record is read again and deleted only if it has not changed, so a session that wrote its record since keeps it; with no conditional delete in `$.store`, a write between that read and the delete is still lost. A day that does not read is dropped, and the others stay. The Usage tab sums the days of the chosen period (`summarize`); the period is kept across sessions in `usage:period`. Before these records each session kept undated totals in `savings:v1:<session id>`; they are no longer written, and the view's `savingsStore`, their sum, is added to the All period. **Delete history** first writes the time in `savings:reset:v1`, then deletes every usage, savings, activity stats, metrics and subagent record. A record whose first reply is older than that time reads as empty, and each session reads the time before it adds a turn, so a session that still holds its record from before a Reset in another session drops it instead of writing it back. The comparison is by milliseconds: a record whose first reply has the reset's millisecond, or one written while the clock had moved back, still counts. A stored value of any other shape reads as empty. A reply with routing off or after an engine fallback is not counted, but it still moves your model's cache. A reply whose cache lifetime cannot be read loses its prices only, and your model's cache still follows it; a usage without rate limits prices writes at five minutes. No failure here fails the turn. [Evaluation](evaluation.md#routing-vs-your-model) gives the method and its limits.
|
|
137
137
|
|
|
138
|
+
Usage storage is version 3 under the same `usage:v1:<session id>` keys. Each day separates comparable `cells`
|
|
139
|
+
from observed-only `uncompared` cells. Both retain reply counts, served-model cost and tokens; only comparable cells
|
|
140
|
+
contribute to savings and What if. The unused comparison fields in observed-only arrays are zero placeholders,
|
|
141
|
+
never a zero-cost baseline. `summarize().observed` combines both lanes for spend, counts and breakdowns while the
|
|
142
|
+
existing top-level comparison totals remain separate. Live replies, daily/hourly trends, periods, merges and resets
|
|
143
|
+
use the same separation. Version 1/2 records migrate on read without losing their comparison history. Previously
|
|
144
|
+
omitted observed-only replies cannot be reconstructed. Older releases do not read version 3 records; they do not
|
|
145
|
+
prune unreadable newer records.
|
|
146
|
+
|
|
138
147
|
### Subagents
|
|
139
148
|
|
|
140
149
|
A subagent's model is chosen once, in the `agent.spawn` hook, before its cache exists. Its `turn.step` events pass through unchanged: a subagent runs on the 5-minute cache lifetime by default, and a switch between its steps would rewrite a 50,000–300,000-token context cold. The facts below are from the Claude Code 2.1.296 types (`AgentSpawnInput`, `HookBudget`) and a probe Mod's observed events, `experiments/mod-router/results/agent-spawn-probe.json`. The [plan](plans/2026-10-10-subagent-routing-plan.md#2-verified-facts) cites each one.
|
package/docs/configuration.md
CHANGED
|
@@ -96,9 +96,12 @@ to promise future hits. A custom client that selects one-hour Kimi caching must
|
|
|
96
96
|
MiMo's currently free cache-write promotion is not treated as free cold input.
|
|
97
97
|
Claude Code's `DISABLE_PROMPT_CACHING` does not establish whether an automatic server cache is disabled.
|
|
98
98
|
|
|
99
|
-
|
|
99
|
+
Known observed usage is retained in Usage even when savings cannot be compared.
|
|
100
100
|
For automatic caches, routing-versus-your-model comparisons require an identical observed route.
|
|
101
|
-
Otherwise the
|
|
101
|
+
Otherwise the band and pane show observed spend, replies, tokens and activity costs with `comparison unavailable`.
|
|
102
|
+
A partially comparable period labels the savings subset; rows without full comparison coverage show `—` under `vs yours`.
|
|
103
|
+
The router never substitutes zero savings or invents another route's cache hit rate.
|
|
104
|
+
Models with `efforts: []`, including MiMo, ignore session effort in the comparison identity.
|
|
102
105
|
Missing counters remain unknown. See [provider acceptance checks](evaluation.md#provider-profiles).
|
|
103
106
|
|
|
104
107
|
## Built-in defaults
|
package/docs/evaluation.md
CHANGED
|
@@ -222,3 +222,11 @@ Limits:
|
|
|
222
222
|
- Failure signatures and pins are not present in the trace. The relevant route decisions are excluded from comparison.
|
|
223
223
|
- Most main rows are tool continuations or have no Jev advice. Only 37 advised decisions support the native comparison.
|
|
224
224
|
- Costs cover the next request. They use the last observed output size and do not model effort-dependent output length or future answer quality.
|
|
225
|
+
|
|
226
|
+
## model-proxy integration
|
|
227
|
+
|
|
228
|
+
The explicit two-provider profile is covered by custom-profile validation, native hook harness switching, bare response-ID normalization, selected-file saves and automatic-advice cases. There are no built-in mixed-provider model defaults.
|
|
229
|
+
|
|
230
|
+
A scoped native check on 2026-11-01 used Router 1.19.1 with Claude Code 2.1.316, the new proxy helper and an explicitly selected work account directory. A synthetic conversation completed pinned DeepSeek → Opus → DeepSeek with Read-tool continuation, followed by a successful automatic turn with Jev healthy. The proxy's [verification report](https://github.com/alexei-led/claude-proxy/blob/main/docs/verification.md#two-provider-integration-for-v013) owns this transport evidence.
|
|
231
|
+
|
|
232
|
+
This is not a routing-quality, savings, full-context or universal provider-history benchmark. Automatic switching in both directions remains covered by controlled harness cases rather than that pinned live sequence.
|
package/docs/native-router.md
CHANGED
|
@@ -71,6 +71,10 @@ The Routes tab starts with the **Activity routing** selector, `off · on`, and a
|
|
|
71
71
|
|
|
72
72
|
The Classifier tab lists every configured classifier as a row with its service host (`this machine` for a local one), its activity probe and p95 wait, and whether its key and endpoint settings are complete. Selecting a row writes `classifier` to `router.json` at once, or removes it for the default, through the same validation, and resets the new classifier's failure count. A turn already being classified finishes with its own classifier; its readings stay off the pane. **Undo** returns to the previous classifier. The deadline saves at once too, and a built-in classifier's default deadline is written as no override. **Sends** names the host that receives prompt text. The **Deadline** line shows the active classifier's p95 wait across sessions, or its probe's before any turn, in yellow when it is slower than the deadline.
|
|
73
73
|
|
|
74
|
+
The Routes grid sizes its cells to the available pane width. Long labels use an ellipsis while retaining their
|
|
75
|
+
built-in/unsaved marker; select a cell to inspect its complete model and effort. Observed-only Usage rows retain
|
|
76
|
+
spend and tokens but show no fabricated savings. See [comparison coverage](user-guide.md#open-the-router-pane).
|
|
77
|
+
|
|
74
78
|
The pane refuses to write through a symlink. A failed validation names the setting and leaves the file unchanged. New model aliases and the other supported fields need an edit to the active profile's `router.json`.
|
|
75
79
|
|
|
76
80
|
## Safety and failure states
|
package/docs/providers.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Providers
|
|
2
2
|
|
|
3
|
-
Claude Router routes models and effort
|
|
4
|
-
Set a provider name and give the router a matching JSON file.
|
|
3
|
+
Claude Router routes models and effort through the Anthropic-compatible endpoint that Claude Code already uses.
|
|
4
|
+
Set a provider name and give the router a matching JSON file. Direct provider profiles need no proxy. The optional `model-proxy` profile uses a separately started local proxy.
|
|
5
5
|
|
|
6
6
|
**Built-in profiles:** Anthropic, DeepSeek, z.ai, MiniMax, MiMo and Kimi.
|
|
7
7
|
**Custom profiles:** OpenRouter, Ollama, other local servers, and other hosted providers with a Claude Code-compatible Anthropic endpoint.
|
|
@@ -45,6 +45,7 @@ The directory is `CLAUDE_CONFIG_DIR` when set, otherwise `~/.claude`.
|
|
|
45
45
|
| `minimax` | `minimax-router.json` | MiniMax-M3.1-Flash |
|
|
46
46
|
| `mimo` | `mimo-router.json` | MiMo-V2.6-Flash, MiMo-V2.6-Pro |
|
|
47
47
|
| `kimi` | `kimi-router.json` | Kimi-K3.1-Pro |
|
|
48
|
+
| `model-proxy` | `model-proxy-router.json` | None. Supply an explicit mixed-provider profile. |
|
|
48
49
|
| `openrouter` | `openrouter-router.json` | None. Supply a complete profile. |
|
|
49
50
|
| `ollama` | `ollama-router.json` | None. Supply a complete profile. |
|
|
50
51
|
| Any other valid name | `<provider>-router.json` | None. Supply a complete profile. |
|
|
@@ -206,6 +207,99 @@ MiMo's cost/quality modes change models where a neighbouring tier permits it; ot
|
|
|
206
207
|
Kimi uses only `low`, `high`, `max`, the documented controls, and never promotes a session into another model.
|
|
207
208
|
These are starting policies, not an optimal activity matrix. See [research and experiments](provider-experiments.md).
|
|
208
209
|
|
|
210
|
+
## Mix Anthropic and DeepSeek with model-proxy
|
|
211
|
+
|
|
212
|
+
`model-proxy` is a custom execution profile, not another classifier or routing engine.
|
|
213
|
+
Claude Code sends every request to one local `claude-proxy` endpoint. The proxy selects an explicitly configured upstream from the native model ID.
|
|
214
|
+
|
|
215
|
+
1. Install and configure [claude-proxy](https://github.com/alexei-led/claude-proxy/blob/main/docs/integration.md).
|
|
216
|
+
2. Save the following complete profile as `model-proxy-router.json` in the selected Claude Code configuration directory.
|
|
217
|
+
3. Start the proxy manually, then use its `connect` command from v0.1.3 or newer.
|
|
218
|
+
|
|
219
|
+
```json
|
|
220
|
+
{
|
|
221
|
+
"models": {
|
|
222
|
+
"deepseek": {
|
|
223
|
+
"id": "deepseek-flash[1m]",
|
|
224
|
+
"input": 0.3,
|
|
225
|
+
"output": 1.2,
|
|
226
|
+
"cacheRead": 0.006,
|
|
227
|
+
"contextWindow": 1000000,
|
|
228
|
+
"billing": "credits",
|
|
229
|
+
"efforts": [
|
|
230
|
+
"low",
|
|
231
|
+
"high",
|
|
232
|
+
"max"
|
|
233
|
+
],
|
|
234
|
+
"strength": 3,
|
|
235
|
+
"imageInput": true
|
|
236
|
+
},
|
|
237
|
+
"claude": {
|
|
238
|
+
"id": "claude-opus-5-5",
|
|
239
|
+
"input": 4,
|
|
240
|
+
"output": 20,
|
|
241
|
+
"cacheRead": 0.2,
|
|
242
|
+
"contextWindow": 1000000,
|
|
243
|
+
"billing": "plan",
|
|
244
|
+
"efforts": [
|
|
245
|
+
"low",
|
|
246
|
+
"medium",
|
|
247
|
+
"high",
|
|
248
|
+
"xhigh",
|
|
249
|
+
"max"
|
|
250
|
+
],
|
|
251
|
+
"strength": 4
|
|
252
|
+
}
|
|
253
|
+
},
|
|
254
|
+
"routes": {
|
|
255
|
+
"micro": {
|
|
256
|
+
"model": "deepseek",
|
|
257
|
+
"effort": "low"
|
|
258
|
+
},
|
|
259
|
+
"low": {
|
|
260
|
+
"model": "deepseek",
|
|
261
|
+
"effort": "low"
|
|
262
|
+
},
|
|
263
|
+
"medium": {
|
|
264
|
+
"model": "deepseek",
|
|
265
|
+
"effort": "high"
|
|
266
|
+
},
|
|
267
|
+
"high": {
|
|
268
|
+
"model": "claude",
|
|
269
|
+
"effort": "xhigh"
|
|
270
|
+
}
|
|
271
|
+
},
|
|
272
|
+
"activityRouting": "off",
|
|
273
|
+
"subagentRouting": "off",
|
|
274
|
+
"cache": {
|
|
275
|
+
"mode": "disabled"
|
|
276
|
+
}
|
|
277
|
+
}
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
These are experimental starting routes, not measured quality rankings. Both lower tiers use DeepSeek at low effort; medium uses high effort; high uses Opus at xhigh.
|
|
281
|
+
The price fields are API list-price estimates, not subscription charges. DeepSeek values use peak prices.
|
|
282
|
+
Cache-cost assumptions, activity routing and subagent routing are off initially; upstream caching remains active.
|
|
283
|
+
The default classifier is Jev and needs its existing plugin credential. Select another configured classifier explicitly if needed; no local classifier server is required.
|
|
284
|
+
|
|
285
|
+
```sh
|
|
286
|
+
# Terminal 1: the selected account's proxy configuration
|
|
287
|
+
claude-proxy --config /path/to/proxy.json
|
|
288
|
+
|
|
289
|
+
# Terminal 2: preserve that account's existing CLAUDE_CONFIG_DIR
|
|
290
|
+
claude-proxy connect --config /path/to/proxy.json -- claude --model 'deepseek-flash[1m]'
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
`connect` supplies `CLAUDE_ROUTER_PROVIDER=model-proxy`, the local URL and the separate local access header only to the child process.
|
|
294
|
+
It does not change saved OAuth, configure provider keys, start the proxy, or install this Mod.
|
|
295
|
+
Without the helper, the proxy guide documents equivalent process-local environment setup.
|
|
296
|
+
Endpoint and credential fields do not belong in this router JSON.
|
|
297
|
+
|
|
298
|
+
Open `/router` and check the selected file, classifier health and routing state before use.
|
|
299
|
+
The catalog must contain the bare native IDs; `[1m]` is client metadata, not a proxy alias.
|
|
300
|
+
Every selected provider receives the unchanged history. Approve each destination; do not mix company context into an unapproved provider.
|
|
301
|
+
Native Anthropic support does not guarantee compatibility with every tool or thinking history. Start with a fresh, non-sensitive conversation.
|
|
302
|
+
|
|
209
303
|
## Add another provider
|
|
210
304
|
|
|
211
305
|
No code change or provider registration is needed.
|
package/docs/user-guide.md
CHANGED
|
@@ -104,11 +104,17 @@ Every write applies from the next turn and can be undone. **Undo** (`u`) puts ba
|
|
|
104
104
|
|
|
105
105
|

|
|
106
106
|
|
|
107
|
-
The Usage tab
|
|
107
|
+
The Usage tab keeps observed routed spend and, when the cache history permits it, compares that spend against your model. Your model is the model `/model` (or `--model`) selected, at the session effort, when a reply ran. The router prices observed tokens at the list prices in `router.json`. A comparison also prices the same tokens on your model when its cache history can be estimated.
|
|
108
108
|
|
|
109
109
|
The top row picks the period: **Session**, **Today**, **7d** (today and the six days before), **30d**, **Month** (since the first of this month) and **All**. Press `p` for the next one. The pane remembers the choice across sessions. Session holds this session's replies, and the line under the row names your model, or every model when `/model` changed during the session. The other periods add up the replies of every session on those calendar days. Each session counts against its own model, and the line reads `all projects · Oct 7 – Oct 13 · each session vs its own model`.
|
|
110
110
|
|
|
111
|
-
|
|
111
|
+
When comparisons are unavailable, the band, Now tab and Usage tab say `observed $… · N replies · comparison unavailable`.
|
|
112
|
+
Counts, tokens, activity/model/tier breakdowns and cost trends still include these replies. A mixed period labels the
|
|
113
|
+
comparable subset separately; its savings cover only those replies. `vs yours` is `—` for a row with incomplete
|
|
114
|
+
comparison coverage, and **What if** stays unavailable rather than inventing a baseline. This is common with automatic
|
|
115
|
+
provider caches, whose future hit rates are unknown. No changes are needed in existing router files.
|
|
116
|
+
|
|
117
|
+
When the whole period is comparable, the answer comes first, in the band's words: `routing −$1.84 (−22%) vs Opus 5.5 · xhigh` for Session, `routing −$7.96 (−46%) vs each session’s own model` for the other periods, or `routing +$2.48 (×5.0) vs Haiku 5.5 · high` when routing cost more. Below 10 routed replies it adds `early sample`, because one switch and its cache write can decide a short run. The ledger under it adds up to the routed cost in whole cents:
|
|
112
118
|
|
|
113
119
|
| Line | What it is |
|
|
114
120
|
| --------------------- | ---------------------------------------------------------------------------------------------- |
|
package/hooks/native-router.mjs
CHANGED
|
@@ -1176,14 +1176,15 @@ async function observeRouted($, router, { turnId, cfg, loop, loopVersion, reques
|
|
|
1176
1176
|
// Kept by model id, under the model that served it: a substitute's cost is not the requested model's, and a models
|
|
1177
1177
|
// key can name another model later.
|
|
1178
1178
|
const idOf = (model) => modelSpec(cfg, model)?.id ?? model;
|
|
1179
|
-
if (compared)
|
|
1179
|
+
if (compared || observedUsd !== null)
|
|
1180
1180
|
await addUsageEntry($, router, {
|
|
1181
1181
|
turnId,
|
|
1182
|
-
compared,
|
|
1182
|
+
compared: compared ?? { reply: null },
|
|
1183
|
+
observedUsd,
|
|
1183
1184
|
tier,
|
|
1184
1185
|
route: `${idOf(response.usage?.model ?? request.model)}@${request.effort ?? 'session'}`,
|
|
1185
1186
|
yours: idOf(nativeModel),
|
|
1186
|
-
byId: Object.fromEntries(Object.entries(compared
|
|
1187
|
+
byId: Object.fromEntries(Object.entries(compared?.replies ?? {}).map(([alias, r]) => [cfg.models[alias].id, r])),
|
|
1187
1188
|
usage: response.usage,
|
|
1188
1189
|
at,
|
|
1189
1190
|
});
|
|
@@ -1234,10 +1235,10 @@ async function compareYours($, router, loop, response, servedEffort, nativeModel
|
|
|
1234
1235
|
// A routed reply, priced against your model now, added to the turn's running replies, which the view counts at once
|
|
1235
1236
|
// and turn.complete adds to the usage record. `route` is the served model's `id@effort`, `yours` your model's id,
|
|
1236
1237
|
// `byId` the reply priced against each model as yours, keyed by model id; `at` when the reply arrived (ms).
|
|
1237
|
-
async function addUsageEntry($, router, { turnId, compared, tier, route, yours, byId, usage, at }) {
|
|
1238
|
+
async function addUsageEntry($, router, { turnId, compared, observedUsd, tier, route, yours, byId, usage, at }) {
|
|
1238
1239
|
try {
|
|
1239
1240
|
const view = router.view ?? (await readView($, router));
|
|
1240
|
-
if (compared.reply) {
|
|
1241
|
+
if (compared.reply || Number.isFinite(observedUsd)) {
|
|
1241
1242
|
const turn = router.turnActivities.get(turnId);
|
|
1242
1243
|
const entry = replyEntry({
|
|
1243
1244
|
at,
|
|
@@ -1247,6 +1248,7 @@ async function addUsageEntry($, router, { turnId, compared, tier, route, yours,
|
|
|
1247
1248
|
route,
|
|
1248
1249
|
yours,
|
|
1249
1250
|
reply: compared.reply,
|
|
1251
|
+
observedUsd,
|
|
1250
1252
|
replies: byId,
|
|
1251
1253
|
usage,
|
|
1252
1254
|
});
|
package/lib/band.mjs
CHANGED
|
@@ -10,6 +10,7 @@ import {
|
|
|
10
10
|
modelColor,
|
|
11
11
|
modelName,
|
|
12
12
|
NOT_A_TIER_REASON,
|
|
13
|
+
observedUsageText,
|
|
13
14
|
percentPair,
|
|
14
15
|
reasonActivity,
|
|
15
16
|
routeLabel,
|
|
@@ -218,10 +219,15 @@ export function bandSegments(config, view, usage, actions) {
|
|
|
218
219
|
// parts.
|
|
219
220
|
export function sessionRow(config, view, columns, now = Date.now()) {
|
|
220
221
|
const label = part('session ', { dimColor: true });
|
|
221
|
-
if (!modelSpec(config, view.nativeModel))
|
|
222
|
-
return [label, part(`${modelName(view.nativeModel)} has no list price`, { dimColor: true })];
|
|
223
222
|
const summary = sessionUsage(view, now);
|
|
224
223
|
const r = readout(summary);
|
|
224
|
+
if (summary.observed.uncomparedReplies)
|
|
225
|
+
return fitSegments(
|
|
226
|
+
[segment(0, label, part(observedUsageText(summary.observed), { dimColor: true }))],
|
|
227
|
+
columns,
|
|
228
|
+
).flatMap((s) => s.parts);
|
|
229
|
+
if (!modelSpec(config, view.nativeModel))
|
|
230
|
+
return [label, part(`${modelName(view.nativeModel)} has no list price`, { dimColor: true })];
|
|
225
231
|
if (!r.replies) return [part('no replies yet', { dimColor: true })];
|
|
226
232
|
const count = `${r.replies} repl${r.replies === 1 ? 'y' : 'ies'}`;
|
|
227
233
|
const answer = [
|
package/lib/display.mjs
CHANGED
|
@@ -51,7 +51,7 @@ export function unpricedLine(metrics) {
|
|
|
51
51
|
const missing = UNPRICED_CAUSES.filter((cause) => cause !== 'comparison');
|
|
52
52
|
const total = missing.reduce((n, cause) => n + unpriced[cause], 0);
|
|
53
53
|
const comparison = unpriced.comparison
|
|
54
|
-
? `Comparison unavailable: ${unpriced.comparison} routed repl${unpriced.comparison === 1 ? 'y' : 'ies'}; observed costs are in
|
|
54
|
+
? `Comparison unavailable: ${unpriced.comparison} routed repl${unpriced.comparison === 1 ? 'y' : 'ies'}; observed costs are in Usage`
|
|
55
55
|
: null;
|
|
56
56
|
if (!total) return comparison;
|
|
57
57
|
const causes = missing
|
|
@@ -345,6 +345,10 @@ export const liveUsage = (view, record) => withReplies(readUsage(record), view.u
|
|
|
345
345
|
// This session's routed replies, the running turns' included.
|
|
346
346
|
export const sessionUsage = (view, now) => summarize(liveUsage(view, view.usageSession), periodRange('session', now));
|
|
347
347
|
|
|
348
|
+
// Observed spend is valid even when a counterfactual cache history cannot be priced.
|
|
349
|
+
export const observedUsageText = (observed) =>
|
|
350
|
+
`observed ${usd(observed.routedUsd)} · ${observed.replies} repl${observed.replies === 1 ? 'y' : 'ies'} · comparison unavailable`;
|
|
351
|
+
|
|
348
352
|
// The models a session's replies were priced against, when they are not the one `/model` selects now: after a
|
|
349
353
|
// `/model` change the record keeps each reply against the model selected when it ran. Null when they are.
|
|
350
354
|
export function recordedModels(config, view, summary) {
|
|
@@ -378,10 +382,14 @@ export const switchesLost = (r) => r.switchUsd > 0 && r.switchUsd > -r.cheaperUs
|
|
|
378
382
|
export function savingsLine(config, view, now) {
|
|
379
383
|
const summary = sessionUsage(view, now);
|
|
380
384
|
const session = readout(summary);
|
|
385
|
+
if (summary.observed.uncomparedReplies)
|
|
386
|
+
return `Session: ${observedUsageText(summary.observed)}${session.replies ? `; ${replyCount(session.replies)} comparable only: ${routingAnswer(session)}` : ''}, est. at list prices`;
|
|
381
387
|
const label = `vs ${comparedModel(config, view, summary)}`;
|
|
382
388
|
if (!session.replies) {
|
|
383
389
|
const kept = allTime(view.usageStore, view.savingsStore);
|
|
384
390
|
const across = readout(kept.totals);
|
|
391
|
+
if (kept.observed.uncomparedReplies)
|
|
392
|
+
return `Session ${label}: no routed replies yet; across sessions ${observedUsageText(kept.observed)}, est. at list prices`;
|
|
385
393
|
const saved = across.early
|
|
386
394
|
? ''
|
|
387
395
|
: `; ${routingAnswer(across)} since ${sinceDate(kept.since)}, each session against its own model`;
|
package/lib/panel.mjs
CHANGED
|
@@ -30,6 +30,7 @@ import {
|
|
|
30
30
|
missingText,
|
|
31
31
|
modelColor,
|
|
32
32
|
modelName,
|
|
33
|
+
observedUsageText,
|
|
33
34
|
percent,
|
|
34
35
|
percentPair,
|
|
35
36
|
probeOf,
|
|
@@ -350,7 +351,8 @@ function nowTab(ui, config, view, usage, actions, _modelOptions, columns, now) {
|
|
|
350
351
|
out.push(ui.text(' '), ...sessionLines(ui, config, view, session, actions, columns, now));
|
|
351
352
|
// Today repeats this session's part of it until another session adds to it.
|
|
352
353
|
const ownToday = summarize(liveUsage(view, view.usageSession), periodRange('today', now));
|
|
353
|
-
if (today.replies > ownToday.replies)
|
|
354
|
+
if (today.observed.replies > ownToday.observed.replies)
|
|
355
|
+
out.push(ui.text(' '), ...todayLines(ui, today, actions, columns));
|
|
354
356
|
out.push(
|
|
355
357
|
...sessionAgentLines(ui, view, actions, columns),
|
|
356
358
|
...worthLines(ui, today, actions, columns),
|
|
@@ -488,17 +490,26 @@ function sessionLines(ui, config, view, summary, actions, columns, now) {
|
|
|
488
490
|
const switches = summary.switches.tier + summary.switches.activity;
|
|
489
491
|
const head = [
|
|
490
492
|
Number.isFinite(since) ? `since ${sinceText(since, now)}` : null,
|
|
491
|
-
summary.replies
|
|
492
|
-
? `${plural(summary.replies, 'routed reply', 'routed replies')}${view.usageLive?.length ? ` (${view.usageLive.length} live)` : ''}`
|
|
493
|
+
summary.observed.replies
|
|
494
|
+
? `${plural(summary.observed.replies, 'routed reply', 'routed replies')}${view.usageLive?.length ? ` (${view.usageLive.length} live)` : ''}`
|
|
493
495
|
: replies
|
|
494
496
|
? plural(replies, 'reply', 'replies')
|
|
495
497
|
: null,
|
|
496
498
|
summary.turns ? plural(switches, 'route switch', 'route switches') : null,
|
|
497
499
|
].filter(Boolean);
|
|
498
|
-
if (!strip && !summary.replies) return [ui.line(ui.head('SESSION'), ui.dim(' · no replies yet'))];
|
|
500
|
+
if (!strip && !summary.observed.replies) return [ui.line(ui.head('SESSION'), ui.dim(' · no replies yet'))];
|
|
499
501
|
const out = [ui.line(ui.head('SESSION'), head.length ? ui.dim(` · ${head.join(' · ')}`) : null)];
|
|
500
502
|
// The band's session row, then the parts that add up to it.
|
|
501
|
-
if (summary.
|
|
503
|
+
if (summary.observed.uncomparedReplies) {
|
|
504
|
+
out.push(
|
|
505
|
+
ui.line(
|
|
506
|
+
' ',
|
|
507
|
+
ui.text(observedUsageText(summary.observed)),
|
|
508
|
+
' ',
|
|
509
|
+
ui.Button({ key: 'usage-session', label: 'Usage', onPress: () => actions.usageFor('session') }),
|
|
510
|
+
),
|
|
511
|
+
);
|
|
512
|
+
} else if (summary.replies) {
|
|
502
513
|
const r = readout(summary);
|
|
503
514
|
out.push(
|
|
504
515
|
answerLine(ui, r, comparedModel(config, view, summary), () => actions.usageFor('session'), 'session', columns),
|
|
@@ -524,7 +535,7 @@ function sessionLines(ui, config, view, summary, actions, columns, now) {
|
|
|
524
535
|
),
|
|
525
536
|
);
|
|
526
537
|
}
|
|
527
|
-
const used = breakdown(summary.cells, 'activity')
|
|
538
|
+
const used = breakdown(summary.observed.cells, 'activity')
|
|
528
539
|
.filter((group) => ACTIVITIES.includes(group.name))
|
|
529
540
|
.sort((a, b) => b.sums[0] - a.sums[0])
|
|
530
541
|
.map((group) => [group.name, `${activityVerb(group.name)} ${group.sums[0]}`]);
|
|
@@ -595,16 +606,22 @@ function todayLines(ui, summary, actions, columns) {
|
|
|
595
606
|
ui.line(
|
|
596
607
|
ui.head('TODAY'),
|
|
597
608
|
ui.dim(
|
|
598
|
-
` · ${plural(summary.replies, 'routed reply', 'routed replies')} · ${plural(switches, 'route switch', 'route switches')}`,
|
|
609
|
+
` · ${plural(summary.observed.replies, 'routed reply', 'routed replies')} · ${plural(switches, 'route switch', 'route switches')}`,
|
|
599
610
|
),
|
|
600
611
|
),
|
|
601
|
-
|
|
612
|
+
summary.observed.uncomparedReplies
|
|
613
|
+
? ui.line(
|
|
614
|
+
ui.text(observedUsageText(summary.observed)),
|
|
615
|
+
' ',
|
|
616
|
+
ui.Button({ key: 'usage-today', label: 'Usage', onPress: () => actions.usageFor('today') }),
|
|
617
|
+
)
|
|
618
|
+
: answerLine(ui, readout(summary), 'each session’s own model', () => actions.usageFor('today'), 'today', columns),
|
|
602
619
|
];
|
|
603
620
|
}
|
|
604
621
|
|
|
605
622
|
// Today's first finding from Usage's WORTH A LOOK, with the tab that changes it; nothing when there is none.
|
|
606
623
|
function worthLines(ui, summary, actions, columns) {
|
|
607
|
-
const [first] = summary.replies >= INSIGHT_REPLIES ? insights(summary) : [];
|
|
624
|
+
const [first] = !summary.observed.uncomparedReplies && summary.replies >= INSIGHT_REPLIES ? insights(summary) : [];
|
|
608
625
|
if (!first) return [];
|
|
609
626
|
return [ui.text(' '), ui.line(ui.head('WORTH A LOOK'), ui.dim(' · today')), insightLine(ui, first, actions, columns)];
|
|
610
627
|
}
|
|
@@ -708,6 +725,15 @@ function routeSelects(ui, config, key, route, modelOptions, onModel, onEffort) {
|
|
|
708
725
|
];
|
|
709
726
|
}
|
|
710
727
|
|
|
728
|
+
const GRID_LABEL = 13;
|
|
729
|
+
|
|
730
|
+
function fitRouteCell(label, width) {
|
|
731
|
+
if (label.length <= width) return label;
|
|
732
|
+
const mark = /[°●]+$/.exec(label)?.[0] ?? '';
|
|
733
|
+
if (mark.length >= width) return mark.slice(-width);
|
|
734
|
+
return `${label.slice(0, Math.max(0, width - 1 - mark.length))}…${mark}`;
|
|
735
|
+
}
|
|
736
|
+
|
|
711
737
|
// Routes per cache: the tiers' routes, and the overrides when `overrides` is set. The count shows the overrides once
|
|
712
738
|
// activity routing is on; the one-more-cache warning always counts them, so it reads the same in every mode. `shared`
|
|
713
739
|
// says, per counted route, whether its model keeps one cache across efforts.
|
|
@@ -735,7 +761,9 @@ function routesTab(ui, config, view, _usage, actions, modelOptions, columns) {
|
|
|
735
761
|
const draft = { ...config, ...changes.draft };
|
|
736
762
|
const mode = draft.activityRouting;
|
|
737
763
|
const narrow = Number.isFinite(columns) && columns < 80;
|
|
738
|
-
const width =
|
|
764
|
+
const width = Number.isFinite(columns)
|
|
765
|
+
? Math.max(1, Math.min(narrow ? 14 : 16, Math.floor((columns - GRID_LABEL) / TIERS.length)))
|
|
766
|
+
: 16;
|
|
739
767
|
const { uses, shared } = cacheUses(draft, mode === 'on');
|
|
740
768
|
const selected = view.routeCell ?? null;
|
|
741
769
|
const cellButton = (row, tier, text, dim) =>
|
|
@@ -743,13 +771,13 @@ function routesTab(ui, config, view, _usage, actions, modelOptions, columns) {
|
|
|
743
771
|
width,
|
|
744
772
|
ui.Button({
|
|
745
773
|
key: `cell-${row}-${tier}`,
|
|
746
|
-
label: `${selected === `${row}.${tier}` ? '▸' : ' '}${text}`,
|
|
774
|
+
label: fitRouteCell(`${selected === `${row}.${tier}` ? '▸' : ' '}${text}`, width),
|
|
747
775
|
plain: true,
|
|
748
776
|
...(dim ? { dimColor: true } : {}),
|
|
749
777
|
onPress: () => actions.selectCell(`${row}.${tier}`),
|
|
750
778
|
}),
|
|
751
779
|
);
|
|
752
|
-
const gridRow = (name, cells) => ui.line(ui.cell(
|
|
780
|
+
const gridRow = (name, cells) => ui.line(ui.cell(GRID_LABEL, name), ...cells);
|
|
753
781
|
const out = [
|
|
754
782
|
ui.line(ui.head('ROUTES'), ui.dim(' · which model runs, by tier and activity · edit, then Save')),
|
|
755
783
|
ui.line(
|
|
@@ -770,7 +798,10 @@ function routesTab(ui, config, view, _usage, actions, modelOptions, columns) {
|
|
|
770
798
|
),
|
|
771
799
|
...overridesState(ui, view, config.activityRouting, mode),
|
|
772
800
|
ui.text(' '),
|
|
773
|
-
ui.line(
|
|
801
|
+
ui.line(
|
|
802
|
+
ui.cell(GRID_LABEL, ''),
|
|
803
|
+
...TIERS.map((tier) => ui.cell(width, ui.tier(tier, fitRouteCell(` ${tier}`, width)))),
|
|
804
|
+
),
|
|
774
805
|
gridRow(
|
|
775
806
|
ui.text('every turn', { bold: true }),
|
|
776
807
|
TIERS.map((tier) =>
|
|
@@ -1694,7 +1725,26 @@ function scopeLine(ui, config, view, period, summary, now) {
|
|
|
1694
1725
|
}
|
|
1695
1726
|
|
|
1696
1727
|
// The answer first, in the band's words: "routing −$1.84 (−22%) vs Opus 5.5 · xhigh", or why there is nothing to compare.
|
|
1697
|
-
function headline(ui, config, view, period, r, summary) {
|
|
1728
|
+
function headline(ui, config, view, period, r, summary, observed = summary.observed) {
|
|
1729
|
+
if (observed.uncomparedReplies)
|
|
1730
|
+
return [
|
|
1731
|
+
ui.text(observedUsageText(observed), { bold: true }),
|
|
1732
|
+
ui.dim(
|
|
1733
|
+
`${observed.uncomparedReplies} of ${observed.replies} replies have no comparison; savings are not estimated for them.`,
|
|
1734
|
+
),
|
|
1735
|
+
...(r.replies
|
|
1736
|
+
? [
|
|
1737
|
+
ui.dim(`${plural(r.replies, 'comparable reply', 'comparable replies')} only:`),
|
|
1738
|
+
ui.line(
|
|
1739
|
+
...answerParts(
|
|
1740
|
+
ui,
|
|
1741
|
+
r,
|
|
1742
|
+
period === 'session' ? comparedModel(config, view, summary) : 'each session’s own model',
|
|
1743
|
+
),
|
|
1744
|
+
),
|
|
1745
|
+
]
|
|
1746
|
+
: []),
|
|
1747
|
+
];
|
|
1698
1748
|
if (!r.replies) {
|
|
1699
1749
|
// A reply is priced against your model, so on a model without a price none is counted.
|
|
1700
1750
|
if (period === 'session' && !modelSpec(config, view.nativeModel))
|
|
@@ -1830,7 +1880,9 @@ function breakdownLines(ui, view, actions, summary, columns) {
|
|
|
1830
1880
|
ui.dim(' g next'),
|
|
1831
1881
|
),
|
|
1832
1882
|
];
|
|
1833
|
-
const
|
|
1883
|
+
const cells = summary.observed.cells;
|
|
1884
|
+
const groups = breakdown(cells, by);
|
|
1885
|
+
const missing = new Map(breakdown(summary.observed.uncompared, by).map((g) => [g.name, g]));
|
|
1834
1886
|
if (!groups.length) return [...out, ui.dim(' no routed replies in this period')];
|
|
1835
1887
|
const tail = !Number.isFinite(columns) || columns >= BREAKDOWN + TAIL;
|
|
1836
1888
|
const tailHead = by === 'model' ? 'share' : 'mostly on';
|
|
@@ -1839,14 +1891,14 @@ function breakdownLines(ui, view, actions, summary, columns) {
|
|
|
1839
1891
|
` ${''.padEnd(NAME)}${'replies'.padStart(8)}${'cost'.padStart(10)}${'vs yours'.padStart(11)}${tail ? ` ${tailHead}` : ''}`,
|
|
1840
1892
|
),
|
|
1841
1893
|
);
|
|
1842
|
-
const numbers = (sums) => [
|
|
1894
|
+
const numbers = (sums, unavailable) => [
|
|
1843
1895
|
`${String(sums[0]).padStart(8)}${usd(sums[1]).padStart(10)}`,
|
|
1844
|
-
differenceText(ui, sums[1] - sums[2], 11),
|
|
1896
|
+
unavailable ? ui.dim('—'.padStart(11)) : differenceText(ui, sums[1] - sums[2], 11),
|
|
1845
1897
|
];
|
|
1846
1898
|
for (const group of groups) {
|
|
1847
1899
|
const id = `${by}:${group.name}`;
|
|
1848
1900
|
const open = view.usageOpen === id;
|
|
1849
|
-
const route = by === 'model' ? null : mostlyOn(cellsWhere(
|
|
1901
|
+
const route = by === 'model' ? null : mostlyOn(cellsWhere(cells, by, group.name));
|
|
1850
1902
|
out.push(
|
|
1851
1903
|
ui.line(
|
|
1852
1904
|
' ',
|
|
@@ -1858,11 +1910,11 @@ function breakdownLines(ui, view, actions, summary, columns) {
|
|
|
1858
1910
|
}),
|
|
1859
1911
|
' ',
|
|
1860
1912
|
ui.cell(NAME, groupName(ui, by, group.name)),
|
|
1861
|
-
...numbers(group.sums),
|
|
1913
|
+
...numbers(group.sums, missing.has(group.name)),
|
|
1862
1914
|
!tail
|
|
1863
1915
|
? null
|
|
1864
1916
|
: by === 'model'
|
|
1865
|
-
? ui.dim(` ${percent(summary.routedUsd > 0 ? group.sums[1] / summary.routedUsd : 0)}`)
|
|
1917
|
+
? ui.dim(` ${percent(summary.observed.routedUsd > 0 ? group.sums[1] / summary.observed.routedUsd : 0)}`)
|
|
1866
1918
|
: route
|
|
1867
1919
|
? ui.dim(` ${routeText(route)}`)
|
|
1868
1920
|
: null,
|
|
@@ -1870,12 +1922,23 @@ function breakdownLines(ui, view, actions, summary, columns) {
|
|
|
1870
1922
|
);
|
|
1871
1923
|
if (!open) continue;
|
|
1872
1924
|
for (const child of group.children)
|
|
1873
|
-
out.push(
|
|
1925
|
+
out.push(
|
|
1926
|
+
ui.line(
|
|
1927
|
+
' ',
|
|
1928
|
+
ui.cell(NAME - 2, groupName(ui, CHILD_BY[by], child.name)),
|
|
1929
|
+
...numbers(
|
|
1930
|
+
child.sums,
|
|
1931
|
+
missing.get(group.name)?.children.some((c) => c.name === child.name),
|
|
1932
|
+
),
|
|
1933
|
+
),
|
|
1934
|
+
);
|
|
1874
1935
|
const [, , , cheaper, stronger, switched, input, , output] = group.sums;
|
|
1875
1936
|
out.push(
|
|
1876
1937
|
...dimLines(
|
|
1877
1938
|
ui,
|
|
1878
|
-
|
|
1939
|
+
missing.has(group.name)
|
|
1940
|
+
? ` comparison unavailable · input ${formatTokens(input)} · output ${formatTokens(output)}`
|
|
1941
|
+
: ` cheaper ${signedUsd(cents(cheaper))} · stronger ${signedUsd(cents(stronger))} · switch ${signedUsd(cents(switched))} · input ${formatTokens(input)} · output ${formatTokens(output)}`,
|
|
1879
1942
|
columns,
|
|
1880
1943
|
),
|
|
1881
1944
|
);
|
|
@@ -1886,6 +1949,8 @@ function breakdownLines(ui, view, actions, summary, columns) {
|
|
|
1886
1949
|
// What the period's tokens would cost on each model alone, as your model for every reply (compareModels), beside what
|
|
1887
1950
|
// routing spent and your own models; a model some reply had no price on is left out.
|
|
1888
1951
|
function compareLines(ui, summary) {
|
|
1952
|
+
if (summary.observed.uncomparedReplies)
|
|
1953
|
+
return [ui.dim('What if unavailable: some replies have no comparable cache history.')];
|
|
1889
1954
|
const yours = Object.keys(summary.yours);
|
|
1890
1955
|
const single = yours.length === 1 ? yours[0] : null;
|
|
1891
1956
|
const rows = Object.entries(summary.whatIf)
|
|
@@ -1947,7 +2012,7 @@ function trendLines(ui, view, actions, summary, period, now) {
|
|
|
1947
2012
|
// The evidence worth a look, each with the tab that changes it: the cell that carried most of the cost, the switch
|
|
1948
2013
|
// writes against the saving, the cell that ran above your model. Never which model to use.
|
|
1949
2014
|
function insightLines(ui, summary, actions, columns) {
|
|
1950
|
-
if (summary.replies < INSIGHT_REPLIES) return [];
|
|
2015
|
+
if (summary.observed.uncomparedReplies || summary.replies < INSIGHT_REPLIES) return [];
|
|
1951
2016
|
const list = insights(summary);
|
|
1952
2017
|
if (!list.length) return [];
|
|
1953
2018
|
return [
|
|
@@ -2021,12 +2086,12 @@ function usageTab(ui, config, view, _usage, actions, _modelOptions, columns, now
|
|
|
2021
2086
|
const legacy = kept?.legacy ?? null;
|
|
2022
2087
|
const r = readout(kept?.totals ?? summary);
|
|
2023
2088
|
const out = [periodRow(ui, period, actions), scopeLine(ui, config, view, period, summary, now), ui.text(' ')];
|
|
2024
|
-
out.push(...headline(ui, config, view, period, r, summary));
|
|
2089
|
+
out.push(...headline(ui, config, view, period, r, summary, kept?.observed ?? summary.observed));
|
|
2025
2090
|
if (r.replies) {
|
|
2026
2091
|
out.push(ui.text(' '), ...ledger(ui, config, view, { period, r, summary, legacy, columns }));
|
|
2027
|
-
if (summary.replies) out.push(ui.text(' '), statsLine(ui, summary));
|
|
2028
2092
|
}
|
|
2029
|
-
if (summary.replies) {
|
|
2093
|
+
if (summary.observed.replies) out.push(ui.text(' '), statsLine(ui, { ...summary.observed, turns: summary.turns }));
|
|
2094
|
+
if (summary.observed.replies) {
|
|
2030
2095
|
out.push(ui.text(' '), ...breakdownLines(ui, view, actions, summary, columns), ui.text(' '));
|
|
2031
2096
|
out.push(
|
|
2032
2097
|
...(view.usageCompare
|
|
@@ -2061,6 +2126,6 @@ function helpLines(ui) {
|
|
|
2061
2126
|
ui.dim('Routing vs your model holds tokens and output length the same'),
|
|
2062
2127
|
ui.dim('and does not measure answer quality. Cache benefit is before writes.'),
|
|
2063
2128
|
ui.dim('The context bar uses the routed model’s window, 20% in reserve.'),
|
|
2064
|
-
ui.dim('
|
|
2129
|
+
ui.dim('Cache estimates follow the selected provider profile; minus means cheaper.'),
|
|
2065
2130
|
];
|
|
2066
2131
|
}
|
package/lib/savings.mjs
CHANGED
|
@@ -116,10 +116,14 @@ export function compareReply(config, state, { usage, served, yours, ttl, now })
|
|
|
116
116
|
const total = counts.input + counts.cacheRead + counts.cacheWrite;
|
|
117
117
|
const [servedSpec, yoursSpec] = [modelSpec(config, served.model), modelSpec(config, yours.model)];
|
|
118
118
|
const sameModel = servedSpec ? servedSpec === yoursSpec : served.model === yours.model;
|
|
119
|
-
const identity = {
|
|
119
|
+
const identity = {
|
|
120
|
+
model: yoursSpec?.id ?? yours.model,
|
|
121
|
+
effort: yoursSpec?.efforts.length === 0 ? null : (yours.effort ?? null),
|
|
122
|
+
};
|
|
123
|
+
const servedEffort = servedSpec?.efforts.length === 0 ? null : (served.effort ?? null);
|
|
120
124
|
// On a model that keeps one cache across efforts (effortSharesCache), an effort-only difference is the same cache.
|
|
121
125
|
const sharedEffort = effortSharesCache(config, identity.model);
|
|
122
|
-
const onYours = sameModel && (sharedEffort ||
|
|
126
|
+
const onYours = sameModel && (sharedEffort || servedEffort === identity.effort);
|
|
123
127
|
const next = { total, at: now, onYours, ...identity };
|
|
124
128
|
// A cache simulated for another model or effort of yours is none of this one's: the new one starts cold.
|
|
125
129
|
const prior = state?.model === identity.model && (sharedEffort || state.effort === identity.effort) ? state : null;
|
package/lib/usage.mjs
CHANGED
|
@@ -36,7 +36,7 @@ const IS_USD = CELL_SUMS.map((sum) => USD.has(sum));
|
|
|
36
36
|
export const DAY = /^\d{4}-\d{2}-\d{2}$/;
|
|
37
37
|
// `mode|activity|tier|id@effort`: mode, activity and tier never hold a `|`, and an effort never an `@`, so a model id
|
|
38
38
|
// may hold either. Version 1 records, from before routing modes, had no mode: every reply then ran `balanced`.
|
|
39
|
-
export const USAGE_VERSION =
|
|
39
|
+
export const USAGE_VERSION = 3;
|
|
40
40
|
const CELL_KEY = /^[^|]+\|[^|]+\|[^|]+\|.+@[^@]+$/;
|
|
41
41
|
const V1_CELL_KEY = /^[^|]+\|[^|]+\|.+@[^@]+$/;
|
|
42
42
|
const V1_MODE = 'balanced';
|
|
@@ -66,31 +66,54 @@ export function emptyUsage() {
|
|
|
66
66
|
return { version: USAGE_VERSION, since: null, days: {} };
|
|
67
67
|
}
|
|
68
68
|
|
|
69
|
-
const emptyDay = () => ({
|
|
69
|
+
const emptyDay = () => ({
|
|
70
|
+
cells: {},
|
|
71
|
+
uncompared: {},
|
|
72
|
+
whatIf: {},
|
|
73
|
+
yours: {},
|
|
74
|
+
hours: {},
|
|
75
|
+
turns: 0,
|
|
76
|
+
switches: { tier: 0, activity: 0 },
|
|
77
|
+
});
|
|
70
78
|
|
|
71
79
|
// One routed main reply as its turn collects it. `at` is when it arrived (ms); `mode` the turn's routing mode;
|
|
72
80
|
// `activity` the turn's activity and
|
|
73
81
|
// `tier` the tier that served it (null: none); `route` the model id that served it and the request's effort,
|
|
74
82
|
// `id@effort` (`session` for no effort); `yours` your model's id; `reply` its prices against your model, and `replies`
|
|
75
83
|
// against each model as yours keyed by model id, from compareModels; `usage` the API usage, whose counts compareReply
|
|
76
|
-
// already checked
|
|
77
|
-
|
|
84
|
+
// already checked, or observedReplyUsd checked them when only observedUsd is available. A false `compared` entry
|
|
85
|
+
// goes in the uncompared lane; its zero comparison fields are never savings. Model ids, not models keys.
|
|
86
|
+
export function replyEntry({
|
|
87
|
+
at,
|
|
88
|
+
mode,
|
|
89
|
+
activity,
|
|
90
|
+
tier,
|
|
91
|
+
route,
|
|
92
|
+
yours,
|
|
93
|
+
reply,
|
|
94
|
+
replies,
|
|
95
|
+
usage,
|
|
96
|
+
observedUsd = /** @type {number | null} */ (null),
|
|
97
|
+
}) {
|
|
98
|
+
if (!reply && !(Number.isFinite(observedUsd) && observedUsd >= 0))
|
|
99
|
+
throw new Error('Observed usage requires a known price');
|
|
78
100
|
const input = usage.input_tokens + usage.cache_read_input_tokens + usage.cache_creation_input_tokens;
|
|
79
101
|
return {
|
|
80
102
|
at,
|
|
103
|
+
...(!reply ? { compared: false } : {}),
|
|
81
104
|
cell: `${mode}|${activity ?? 'none'}|${tier ?? 'none'}|${route}`,
|
|
82
105
|
sums: [
|
|
83
106
|
1,
|
|
84
|
-
reply
|
|
85
|
-
reply
|
|
86
|
-
reply
|
|
87
|
-
reply
|
|
88
|
-
reply
|
|
107
|
+
reply?.routedUsd ?? observedUsd,
|
|
108
|
+
reply?.yoursUsd ?? 0,
|
|
109
|
+
reply?.cheaperUsd ?? 0,
|
|
110
|
+
reply?.strongerUsd ?? 0,
|
|
111
|
+
reply?.switchUsd ?? 0,
|
|
89
112
|
input,
|
|
90
113
|
usage.cache_read_input_tokens,
|
|
91
114
|
usage.output_tokens,
|
|
92
115
|
],
|
|
93
|
-
whatIf: Object.fromEntries(Object.entries(replies).map(([model, r]) => [model, r.yoursUsd])),
|
|
116
|
+
whatIf: reply ? Object.fromEntries(Object.entries(replies).map(([model, r]) => [model, r.yoursUsd])) : {},
|
|
94
117
|
yours,
|
|
95
118
|
};
|
|
96
119
|
}
|
|
@@ -132,7 +155,8 @@ function addEntries(recordDays, entries) {
|
|
|
132
155
|
copied.add(key);
|
|
133
156
|
}
|
|
134
157
|
const day = days[key];
|
|
135
|
-
|
|
158
|
+
const cells = entry.compared === false ? day.uncompared : day.cells;
|
|
159
|
+
cells[entry.cell] = cells[entry.cell] ? addSums(cells[entry.cell], entry.sums) : [...entry.sums];
|
|
136
160
|
for (const [model, usd] of Object.entries(entry.whatIf)) {
|
|
137
161
|
const [n, sum] = own(day.whatIf, model) ?? [0, 0];
|
|
138
162
|
put(day.whatIf, model, [n + 1, microUsd(sum + usd)]);
|
|
@@ -161,6 +185,15 @@ function readDay(day, version) {
|
|
|
161
185
|
isRecord(day) &&
|
|
162
186
|
isRecord(day.cells) &&
|
|
163
187
|
Object.entries(day.cells).every(([cell, sums]) => key.test(cell) && isSums(sums)) &&
|
|
188
|
+
(version < 3 ||
|
|
189
|
+
(isRecord(day.uncompared) &&
|
|
190
|
+
Object.entries(day.uncompared).every(
|
|
191
|
+
([cell, sums]) =>
|
|
192
|
+
key.test(cell) &&
|
|
193
|
+
isSums(sums) &&
|
|
194
|
+
sums.slice(2, 6).every((n) => n === 0) &&
|
|
195
|
+
[0, 6, 7, 8].every((i) => isCount(sums[i])),
|
|
196
|
+
))) &&
|
|
164
197
|
isRecord(day.whatIf) &&
|
|
165
198
|
Object.values(day.whatIf).every(isPair) &&
|
|
166
199
|
isRecord(day.yours) &&
|
|
@@ -177,15 +210,16 @@ function readDay(day, version) {
|
|
|
177
210
|
? {
|
|
178
211
|
...day,
|
|
179
212
|
cells: Object.fromEntries(Object.entries(day.cells).map(([cell, sums]) => [`${V1_MODE}|${cell}`, sums])),
|
|
213
|
+
uncompared: {},
|
|
180
214
|
}
|
|
181
|
-
: day;
|
|
215
|
+
: { ...day, uncompared: version < 3 ? {} : day.uncompared };
|
|
182
216
|
}
|
|
183
217
|
|
|
184
218
|
// The stored value read day by day: a missing or unknown record is empty, and a day that does not read is dropped
|
|
185
|
-
// while the others stay, so a hand edit never breaks a turn or the pane. A version 1 record
|
|
186
|
-
// replies under `balanced
|
|
219
|
+
// while the others stay, so a hand edit never breaks a turn or the pane. A version 1 record keeps its
|
|
220
|
+
// replies under `balanced`. Versions 1 and 2 gain an empty observed-only lane on the next version 3 write.
|
|
187
221
|
export function readUsage(value) {
|
|
188
|
-
if (!isRecord(value) || ![1, USAGE_VERSION].includes(value.version) || !isRecord(value.days)) return emptyUsage();
|
|
222
|
+
if (!isRecord(value) || ![1, 2, USAGE_VERSION].includes(value.version) || !isRecord(value.days)) return emptyUsage();
|
|
189
223
|
if (!(value.since === null || Number.isFinite(value.since))) return emptyUsage();
|
|
190
224
|
const days = {};
|
|
191
225
|
for (const [key, day] of Object.entries(value.days)) {
|
|
@@ -232,6 +266,8 @@ function mergeDay(a, b) {
|
|
|
232
266
|
const day = structuredClone(a);
|
|
233
267
|
for (const [key, sums] of Object.entries(b.cells))
|
|
234
268
|
day.cells[key] = day.cells[key] ? addSums(day.cells[key], sums) : [...sums];
|
|
269
|
+
for (const [key, sums] of Object.entries(b.uncompared ?? {}))
|
|
270
|
+
day.uncompared[key] = day.uncompared[key] ? addSums(day.uncompared[key], sums) : [...sums];
|
|
235
271
|
for (const [model, [n, usd]] of Object.entries(b.whatIf)) {
|
|
236
272
|
const [m, sum] = own(day.whatIf, model) ?? [0, 0];
|
|
237
273
|
put(day.whatIf, model, [m + n, microUsd(sum + usd)]);
|
|
@@ -280,7 +316,24 @@ export function summarize(record, { from, to }) {
|
|
|
280
316
|
addSums,
|
|
281
317
|
CELL_SUMS.map(() => 0),
|
|
282
318
|
);
|
|
319
|
+
const cells = { ...total.cells };
|
|
320
|
+
for (const [key, sums] of Object.entries(total.uncompared))
|
|
321
|
+
cells[key] = cells[key] ? addSums(cells[key], sums) : [...sums];
|
|
322
|
+
const observed = Object.values(cells).reduce(
|
|
323
|
+
addSums,
|
|
324
|
+
CELL_SUMS.map(() => 0),
|
|
325
|
+
);
|
|
283
326
|
return {
|
|
327
|
+
observed: {
|
|
328
|
+
replies: observed[0],
|
|
329
|
+
routedUsd: observed[1],
|
|
330
|
+
inputTokens: observed[6],
|
|
331
|
+
cacheReadTokens: observed[7],
|
|
332
|
+
outputTokens: observed[8],
|
|
333
|
+
cells,
|
|
334
|
+
uncompared: total.uncompared,
|
|
335
|
+
uncomparedReplies: Object.values(total.uncompared).reduce((n, sums) => n + sums[0], 0),
|
|
336
|
+
},
|
|
284
337
|
replies,
|
|
285
338
|
routedUsd,
|
|
286
339
|
yoursUsd,
|
|
@@ -310,6 +363,11 @@ export function allTime(usageStore, savingsStore) {
|
|
|
310
363
|
return {
|
|
311
364
|
totals: Object.fromEntries(READOUT_SUMS.map((key) => [key, summary[key] + legacy[key]])),
|
|
312
365
|
since: earlierSince(record.since, legacy.since),
|
|
366
|
+
observed: {
|
|
367
|
+
...summary.observed,
|
|
368
|
+
replies: summary.observed.replies + legacy.replies,
|
|
369
|
+
routedUsd: summary.observed.routedUsd + legacy.routedUsd,
|
|
370
|
+
},
|
|
313
371
|
legacy,
|
|
314
372
|
};
|
|
315
373
|
}
|
|
@@ -375,7 +433,10 @@ export function trend(summary, period, now) {
|
|
|
375
433
|
const first = summary.days[0][0];
|
|
376
434
|
const start = periodRange(period, now).from ?? first;
|
|
377
435
|
const routed = new Map(
|
|
378
|
-
summary.days.map(([key, day]) => [
|
|
436
|
+
summary.days.map(([key, day]) => [
|
|
437
|
+
key,
|
|
438
|
+
[...Object.values(day.cells), ...Object.values(day.uncompared ?? {})].reduce((s, c) => s + c[1], 0),
|
|
439
|
+
]),
|
|
379
440
|
);
|
|
380
441
|
const values = [];
|
|
381
442
|
const [y, m, d] = start.split('-').map(Number);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@alexeiled/claude-router",
|
|
3
|
-
"version": "1.19.
|
|
3
|
+
"version": "1.19.2",
|
|
4
4
|
"description": "Native Claude Code model and effort routing with Anthropic, DeepSeek, z.ai, MiniMax, MiMo and Kimi defaults, custom provider profiles, and classifier advice.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "Alexei Ledenev",
|
package/types/index.d.ts
CHANGED
|
@@ -164,6 +164,8 @@ declare module 'claude-code' {
|
|
|
164
164
|
// `hours` routed USD by local hour; `turns` and `switches` count turns on the day they ended.
|
|
165
165
|
interface RouterUsageDay {
|
|
166
166
|
cells: Record<string, number[]>;
|
|
167
|
+
// Observed-only replies in version 3: same sums, with comparison fields 2..5 unused (zero).
|
|
168
|
+
uncompared?: Record<string, number[]>;
|
|
167
169
|
whatIf: Record<string, [number, number]>;
|
|
168
170
|
yours: Record<string, number>;
|
|
169
171
|
hours: Record<string, number>;
|
|
@@ -173,7 +175,7 @@ declare module 'claude-code' {
|
|
|
173
175
|
// Kept across sessions, one per session under store key 'usage:v1:<session id>' or their sum in the view, by day
|
|
174
176
|
// ('2026-10-13'), since its first turn written after the last reset (ms).
|
|
175
177
|
interface RouterUsageStore {
|
|
176
|
-
version: 1 | 2;
|
|
178
|
+
version: 1 | 2 | 3;
|
|
177
179
|
since: number | null;
|
|
178
180
|
days: Record<string, RouterUsageDay>;
|
|
179
181
|
}
|