@alexeiled/claude-router 1.17.1 → 1.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +16 -0
- package/docs/architecture.md +1 -1
- package/docs/evaluation.md +2 -2
- package/docs/native-router.md +1 -1
- package/docs/router-band.svg +9 -13
- package/docs/router-pane-now.svg +130 -120
- package/docs/router-pane-routes.svg +1 -1
- package/docs/router-pane-usage-whatif.svg +6 -8
- package/docs/router-pane-usage.svg +6 -8
- package/docs/user-guide.md +12 -12
- package/hooks/native-router.mjs +33 -27
- package/lib/band.mjs +42 -37
- package/lib/display.mjs +47 -22
- package/lib/panel.mjs +82 -78
- package/lib/savings.mjs +5 -37
- package/lib/usage.mjs +29 -9
- package/lib/view.mjs +2 -4
- package/package.json +1 -1
package/docs/user-guide.md
CHANGED
|
@@ -46,7 +46,7 @@ Read the first line of the picture from left to right:
|
|
|
46
46
|
- **Tier, activity, and model**: the route for this turn. The activity sits between the tier and the model as a verb in orange italic, such as `coding →`: `coding`, `debugging`, `exploring`, `planning`, `reviewing`, `running` (commands: git, builds, tests, CI) or `documenting`. See [Route by activity](#route-by-activity). **fallback** means Claude Code answered with another model.
|
|
47
47
|
- **Reason**: one or two words with a direction, such as `↑ jump`, `↓ downgrade`, `= fits`, or `… waiting to go down`. An activity move reads `↗ coding` or `↘ running`, and a refused one `… running: not worth a switch`.
|
|
48
48
|
- **Classifier support**: what the active classifier, such as Jev or Clef Flash, gave the switch against the bar the policy required, such as `Jev 88% ≥ 82%`. When whole percents would read as equal, it shows tenths: `Jev 81.0% < 81.4%`.
|
|
49
|
-
- **
|
|
49
|
+
- **Next switch**: `next switch ≈ $0.37`, the policy's price for the switch it weighed this turn, mostly the prompt cache the other model must write. For a move to a cheaper model it is net of what that model saves over the next turns. The higher the cost, the higher the bar a stronger model must clear.
|
|
50
50
|
- **ctx** and **cache hit**: context use and the share of input read from the cache on the last reply. They turn yellow at 60% context and red at 80%.
|
|
51
51
|
|
|
52
52
|
| Band shows | Meaning |
|
|
@@ -61,7 +61,7 @@ Read the first line of the picture from left to right:
|
|
|
61
61
|
| `✕ Routing unavailable` | Nothing routes: a missing/invalid provider file, an old Claude Code, or leftover v0.8 settings. **Fix** opens the pane. |
|
|
62
62
|
| `○ Router · ■ Haiku 5.5 · moved down by the router` | You are viewing a subagent's transcript. The model it runs on, and who picked it; see below. |
|
|
63
63
|
|
|
64
|
-
The band shows one routing control at a time. While routing is on, hover over the band for a second row: pin a tier for the next turn, **Turn off** routing, or show two rows. While routing is off, **Turn on** sits on the first line next to `Routing off` and stays at any width: a narrow band drops the explanation and the model first. On a narrow band the hover row drops `routing on`, then **2 rows**, then `pin next turn`, then the pins; it keeps **Turn off**, and **1 row** when the band shows two. The hover row needs a pointer: from the keyboard, run `/router off`, or open the pane and press `f`. The two-row band adds this session against your model, the
|
|
64
|
+
The band shows one routing control at a time. While routing is on, hover over the band for a second row: pin a tier for the next turn, **Turn off** routing, or show two rows. While routing is off, **Turn on** sits on the first line next to `Routing off` and stays at any width: a narrow band drops the explanation and the model first. On a narrow band the hover row drops `routing on`, then **2 rows**, then `pin next turn`, then the pins; it keeps **Turn off**, and **1 row** when the band shows two. The hover row needs a pointer: from the keyboard, run `/router off`, or open the pane and press `f`. The two-row band adds this session against your model, and the parts that add up to it: `session routing −$1.84 (−22%) vs Opus 5.5 · xhigh = −$2.31 cheaper models +$0.47 cache rebuilds · 64 replies`, or `session routing +$2.48 (×5.0) vs Haiku 5.5 · high = +$2.31 stronger models +$0.17 cache rebuilds · 41 replies` when routing cost more. Minus is less than your model, plus is more. **Cache rebuilds** is what the model switches cost: the next model must write the prompt cache again. The row is the first line of the pane's SESSION block, with the same words and the same number, and both count the replies of the turn still running. After a `/model` change the row names the models the replies ran against, as the Usage tab does. Before the tenth routed reply it leaves out the parts and adds `early sample`, as the pane does, because one switch can decide a short run: `session routing −$0.05 (−19%) vs Opus 5.5 · xhigh · 3 replies · early sample`. When the row is short, the parts lose their words (`= −$2.31 + $0.47`), then the reply count drops, then the parts. Router remembers that choice across sessions. When Router changes the model between turns, a toast reads `Model changed: old → new · reason`. A pin does not raise one, and neither does a change of effort alone.
|
|
65
65
|
|
|
66
66
|
The band steps aside while a survey needs the space.
|
|
67
67
|
|
|
@@ -83,11 +83,11 @@ Run `/router` or select **Router** above the prompt. Without a UI surface, as in
|
|
|
83
83
|
|
|
84
84
|
The top line of every tab, `ROUTING [ ◉ on ] [ ○ off ] Jev ready · 347 ms`, shows the routing mode, switches it, and says whether the classifier is ready. Press `o` or `f` while the pane has focus. **?** shows what the estimates leave out. When routing is unavailable, the top line reads `ROUTING unavailable Claude’s model is kept` with the reason under it, and offers no on or off: see [Troubleshooting](#troubleshooting).
|
|
85
85
|
|
|
86
|
-

|
|
87
87
|
|
|
88
|
-
The **Now** tab answers "what runs next turn, and why". **Next turn** names the tier, the model and effort, and, when an activity override runs, the tier's own model beside it. **Activity** names the classifier's activity as a verb, such as `coding 84%`, with the two next answers and the threshold at which an activity applies. **Why** gives the reason; under it, `Jev gave stronger tiers 88% (needs 82%) · switch ≈ $0.48` compares the classifier's support for the switch with the bar the policy set, and gives the estimated switch
|
|
88
|
+
The **Now** tab answers "what runs next turn, and why". **Next turn** names the tier, the model and effort, and, when an activity override runs, the tier's own model beside it. **Activity** names the classifier's activity as a verb, such as `coding 84%`, with the two next answers and the threshold at which an activity applies. **Why** gives the reason; under it, `Jev gave stronger tiers 88% (needs 82%) · next switch ≈ $0.48` compares the classifier's support for the switch with the bar the policy set, and gives the estimated cost of the next switch. **Pin next** has one button per tier, strongest first, with the support the classifier gave it on the last turn; `▶` marks the tier that runs now. A press forces that tier for the next turn only.
|
|
89
89
|
|
|
90
|
-
**SESSION** sums this session from its first routed reply: `since 14:18 · 86 routed replies · 9 route switches
|
|
90
|
+
**SESSION** sums this session from its first routed reply: `since 14:18 · 86 routed replies (4 live) · 9 route switches`, where `live` counts the replies of the turn still running. Its first line is the band's session row, `routing −$1.84 (−22%) vs Opus 5.5 · xhigh`, with **Usage** to open the Usage tab on this session. The ledger under it gives the parts that add up to that answer, the same as the Usage tab's but without the $0.00 lines. When cache rebuilds cost more than the cheaper models saved, `▲ switches cost more than they saved` follows, with **Policy** to raise the bar for a switch. The strip under the ledger colors every reply since Claude Code started the session by the tier that served it; a dot marks replies routing did not serve, such as those with routing off. Each cell is one reply up to 48 cells. Past that, and on a pane too narrow for the cells, pairs of cells merge and the strip says how many replies a cell holds, so it spans all those replies on one line. After Claude Code restarts a resumed session, the strip starts again and says since when, such as `since 15:02`, while the heading still counts the whole session. The replies per tier, `■ low 41 ■ medium 12 ■ high 33`, double as the color key, and the routed replies per activity follow. **last switch** says when the route last changed and between which tiers, or between which routes when a switch stayed inside a tier, with the activity it ran for.
|
|
91
91
|
|
|
92
92
|
**TODAY** appears once another session added to today, with the same line for every session today and its own **Usage** link. **SUBAGENTS** appears once this session started a subagent: how many are in the middle of a turn now (`● 1 running`; an agent that waits on its own background command is not counted until it resumes), the same squares as on the Usage tab, and what routing saved on them; **Usage** opens the Usage tab on this session. **WORTH A LOOK** shows today's first finding from the Usage tab, from 20 routed replies, with the tab that changes it. The last lines give the context gauge, the cache reuse and the cost Claude reports; **details** adds the counters and estimates in [Understand usage and estimates](#understand-usage-and-estimates).
|
|
93
93
|
|
|
@@ -102,20 +102,20 @@ Every write applies from the next turn and can be undone. **Undo** (`u`) puts ba
|
|
|
102
102
|
|
|
103
103
|
### Compare routing with your model
|
|
104
104
|
|
|
105
|
-

|
|
106
106
|
|
|
107
107
|
The Usage tab answers what routing cost against your model. Your model is the model `/model` (or `--model`) selected, at the session effort, when a reply ran. The router prices each routed reply's tokens twice at the list prices in `router.json`: on the model that served it, and on your model.
|
|
108
108
|
|
|
109
|
-
The top row picks the period: **Session**, **Today**, **7d** (today and the six days before), **30d**, **Month** (since the first of this month) and **All**. Press `p` for the next one. The pane remembers the choice across sessions. Session holds this session's replies, and the line under the row names your model, or every model when `/model` changed during the session. The other periods add up the replies of every session on those calendar days. Each session counts against its own model, and the line reads `all
|
|
109
|
+
The top row picks the period: **Session**, **Today**, **7d** (today and the six days before), **30d**, **Month** (since the first of this month) and **All**. Press `p` for the next one. The pane remembers the choice across sessions. Session holds this session's replies, and the line under the row names your model, or every model when `/model` changed during the session. The other periods add up the replies of every session on those calendar days. Each session counts against its own model, and the line reads `all projects · Oct 7 – Oct 13 · each session vs its own model`.
|
|
110
110
|
|
|
111
|
-
The answer comes first: `
|
|
111
|
+
The answer comes first, in the band's words: `routing −$1.84 (−22%) vs Opus 5.5 · xhigh` for Session, `routing −$7.96 (−46%) vs each session’s own model` for the other periods, or `routing +$2.48 (×5.0) vs Haiku 5.5 · high` when routing cost more. Below 10 routed replies it adds `early sample`, because one switch and its cache write can decide a short run. The ledger under it adds up to the routed cost in whole cents:
|
|
112
112
|
|
|
113
113
|
| Line | What it is |
|
|
114
114
|
| --------------------- | ---------------------------------------------------------------------------------------------- |
|
|
115
115
|
| Your model | What the same input and output tokens cost on your model, with one cache that never switched. |
|
|
116
116
|
| − Cheaper models | What routing saved on replies that ran on models cheaper than yours. |
|
|
117
117
|
| + Stronger models | What replies on models stronger than yours cost on top. |
|
|
118
|
-
| +
|
|
118
|
+
| + Cache rebuilds | What rewriting the prompt cache after a model switch cost, and its share of the saving. |
|
|
119
119
|
| = Routed | What the routed replies cost, with a bar against your model: `54% of your model`, or `×2.1`. |
|
|
120
120
|
|
|
121
121
|
A line of counts follows: replies, turns, input tokens, the share of the input read from the cache, and output tokens. With routing off or unavailable and no routed reply this session, the tab says the replies use your model and are not counted.
|
|
@@ -127,7 +127,7 @@ The next line draws the routed cost as blocks: **COST BY HOUR** over today's hou
|
|
|
127
127
|
From 20 routed replies in the period, **WORTH A LOOK** lists up to three findings:
|
|
128
128
|
|
|
129
129
|
- the activity and tier that carried a quarter or more of the cost
|
|
130
|
-
- the
|
|
130
|
+
- the cache rebuilds, when they used 15% or more of the cheaper-model saving
|
|
131
131
|
- the activity and tier that ran most above your model
|
|
132
132
|
|
|
133
133
|
Each line has a button to the tab that changes it. **Routes** opens that cell in the grid, and **Policy** the switching thresholds. The tab never says which model to use, because it does not measure answer quality.
|
|
@@ -152,7 +152,7 @@ These are estimates, not a bill. They hold the tokens and the output length the
|
|
|
152
152
|
- **Next-turn difference** compares the candidate with the current model under five-minute and one-hour cache-write scenarios. A negative number means the candidate is estimated to cost less for that request.
|
|
153
153
|
- **Payback** estimates later turns to recover an initial difference under assumed future cache reads and the last observed output size. It does not guarantee savings.
|
|
154
154
|
|
|
155
|
-
On the Usage tab, **SUBAGENTS** covers the same period as the rest of the tab: this session, or every session in every project that uses this router install (`all projects`), since the last **Delete history**. Each agent is one square, colored by the model it ran on: filled (`■`) when the router moved it off the model Claude Code would have picked, hollow (`□`) when it ran on Claude Code's pick, because the router agreed or did not route it. The models and their agents follow the squares, then how many the router moved, and what routing saved or cost on the routed agents against Claude Code's picks. 7d, 30d, Month and All add the agents per day. **ROUTED** lists each agent type the router picked a model for: its squares, agents, cost, and **vs CC**, the same tokens on Claude Code's pick. A row opens to its models, each with how many agents the router kept, moved up or moved down, and to its requests per agent, respawns and classifier failures. **NOT ROUTED** lists the agents Claude Code picked a model for, by reason, with why: `the workflow sets it`, `your model` for a fork, `its agent file sets it`. A row without any priced request says `no price`. Records from before 1.14.0 have no date or model: Session and All add them as one line, and an empty Today or 7d says so. See [Subagent routing](configuration.md#subagent-routing). Plan quota is not on the pane: Claude Code shows it in `/usage`. On the Now tab, **details** adds the context and cache counters, the output of the last reply, the input per reply, and the estimates above.
|
|
155
|
+
On the Usage tab, **SUBAGENTS** covers the same period as the rest of the tab: this session, or every session in every project that uses this router install (`all projects`), since the last **Delete history**. Each agent is one square, colored by the model it ran on: filled (`■`) when the router moved it off the model Claude Code would have picked, hollow (`□`) when it ran on Claude Code's pick, because the router agreed or did not route it. The models and their agents follow the squares, then how many the router moved, and what routing saved or cost on the routed agents against Claude Code's picks, in the session readout's words: `routing −$0.10 (−13%) vs Claude Code’s picks on 3 routed agents`. 7d, 30d, Month and All add the agents per day. **ROUTED** lists each agent type the router picked a model for: its squares, agents, cost, and **vs CC**, the same tokens on Claude Code's pick. A row opens to its models, each with how many agents the router kept, moved up or moved down, and to its requests per agent, respawns and classifier failures. **NOT ROUTED** lists the agents Claude Code picked a model for, by reason, with why: `the workflow sets it`, `your model` for a fork, `its agent file sets it`. A row without any priced request says `no price`. Records from before 1.14.0 have no date or model: Session and All add them as one line, and an empty Today or 7d says so. See [Subagent routing](configuration.md#subagent-routing). Plan quota is not on the pane: Claude Code shows it in `/usage`. On the Now tab, **details** adds the context and cache counters, the output of the last reply, the input per reply, and the estimates above.
|
|
156
156
|
|
|
157
157
|
The router keeps no bill. Routing vs your model is an estimate at configured list prices. The pane does not include classifier charges. Claude's native `/cost` is the source for its reported API cost.
|
|
158
158
|
|
|
@@ -226,7 +226,7 @@ These thresholds are starting values. They are not tuned on data yet.
|
|
|
226
226
|
|
|
227
227
|
On the **Classifier** tab, **ACROSS SESSIONS · since the last reset** shows the counts kept across sessions, once there are any: the turns labelled from their tools, **Agreement** over them, **Code/ops/explore**, the same agreement over the turns the classifier answered `code`, `ops` or `explore` (omitted when there are none), the two most frequent mismatches as `predicted → observed count` (`code → read 3`), **Activity moves**, the moves to another route inside a tier, taken and refused for any reason, **Escalations**, such as `after a cheaper activity move: 1 in 12 moves`, the tool-error escalations that followed a move to a cheaper route in `on` while the turns stayed on it in `on` (turning routing off, a turn with routing off, or a turn in `off` ends the watch, a pin in between does not, and an escalation the credits cap held counts). Each of these lines appears only when it has data. The active classifier's p95 wait across sessions shows on its **Deadline** line instead (not recorded with activity routing `off`). This agreement compares the classifier's raw answer, so it can differ from the session's. The pane reads these counts when the session starts and when it opens, and adds this session's turns as they finish; another session's new latency and escalation readings show the next time the pane opens. **Delete history** on the Usage tab deletes these counts with the usage and subagent records, for this session and across sessions.
|
|
228
228
|
|
|
229
|
-
`/router` without a UI surface adds an `Activity: ops (81%)` line, the resolved route, and `Agreement across sessions: 19 of 22 turns (86%) · code/ops/explore 9 of 10 (90%)`. It also prints the session
|
|
229
|
+
`/router` without a UI surface adds an `Activity: ops (81%)` line, the resolved route, and `Agreement across sessions: 19 of 22 turns (86%) · code/ops/explore 9 of 10 (90%)`. It also prints the session in the band's words, such as `Session: routing −$1.84 (−22%) vs Opus 5.5 · xhigh = −$2.31 cheaper models +$0.47 cache rebuilds, 64 replies, est. at list prices`.
|
|
230
230
|
|
|
231
231
|
## Change routing
|
|
232
232
|
|
package/hooks/native-router.mjs
CHANGED
|
@@ -96,13 +96,10 @@ import {
|
|
|
96
96
|
resetHistory,
|
|
97
97
|
} from '../lib/route.mjs';
|
|
98
98
|
import {
|
|
99
|
-
addReplies,
|
|
100
99
|
afterReset,
|
|
101
100
|
compareModels,
|
|
102
101
|
emptySavingsStore,
|
|
103
102
|
mergeSavingsStores,
|
|
104
|
-
modelAlias,
|
|
105
|
-
modelEntry,
|
|
106
103
|
modelSpec,
|
|
107
104
|
observedReplyUsd,
|
|
108
105
|
readSavingsStore,
|
|
@@ -424,10 +421,6 @@ async function readProject($, profilePath) {
|
|
|
424
421
|
async function updateView($, router, patch) {
|
|
425
422
|
const value = router.view ?? (await readView($, router));
|
|
426
423
|
router.view = { ...value, ...patch };
|
|
427
|
-
// The session readout is against the model /model selects now, so a /model change shows the whole session against
|
|
428
|
-
// the new one.
|
|
429
|
-
if ('nativeModel' in patch || 'savingsBy' in patch)
|
|
430
|
-
router.view.savings = modelEntry(router.view.savingsBy, modelAlias(router.config, router.view.nativeModel)) ?? null;
|
|
431
424
|
await $.state.set(VIEW, router.view);
|
|
432
425
|
}
|
|
433
426
|
|
|
@@ -484,9 +477,14 @@ async function readOthers($, router, kind, { sessionId, resetAt, keys }) {
|
|
|
484
477
|
|
|
485
478
|
// The view's sums of `kinds`, this session's record and the others', from what the router holds; a kind whose others
|
|
486
479
|
// were never read gives nothing.
|
|
480
|
+
// The replies of turns still running, in the view, so the band and the pane count them before the turn is recorded.
|
|
481
|
+
const liveUsage = (router) => [...router.turnUsage.values()].flat();
|
|
482
|
+
|
|
487
483
|
function ledgerView(router, kinds = LEDGER_KINDS) {
|
|
488
|
-
|
|
489
|
-
|
|
484
|
+
// The running turns' replies always go with the usage sums: a turn moves from one to the other, so a view written
|
|
485
|
+
// with only one of them would count it twice or miss it.
|
|
486
|
+
const patch = { usageLive: liveUsage(router) };
|
|
487
|
+
for (const kind of new Set([...kinds, 'usage'])) {
|
|
490
488
|
const { others, own } = router.ledgers[kind];
|
|
491
489
|
const { field, ownField, merge, empty } = LEDGERS[kind];
|
|
492
490
|
if (!others) continue;
|
|
@@ -734,7 +732,7 @@ function detailText(config, view) {
|
|
|
734
732
|
...activityDetailLines(config, view),
|
|
735
733
|
storeAgreementLine(view),
|
|
736
734
|
...[unpricedLine(view.activityMetrics)].filter(Boolean),
|
|
737
|
-
savingsLine(view),
|
|
735
|
+
savingsLine(config, view, Date.now()),
|
|
738
736
|
view.error || missing ? classifierStatus(config, view.error ?? missing) : `${activeClassifier(config).label} ready`,
|
|
739
737
|
`Context: ${view.contextKnown ? `${view.contextTokens} tokens (estimate)` : 'unknown'}`,
|
|
740
738
|
`Observed cache: ${view.cacheRead ?? 'unknown'} read, ${view.cacheWrite ?? 'unknown'} written tokens`,
|
|
@@ -1138,8 +1136,8 @@ async function publishReply($, router, turnId, response, tier, route, routedUsd,
|
|
|
1138
1136
|
});
|
|
1139
1137
|
}
|
|
1140
1138
|
|
|
1141
|
-
// A routed reply written to the loop at `loopVersion`, then published: the view's metrics and the
|
|
1142
|
-
//
|
|
1139
|
+
// A routed reply written to the loop at `loopVersion`, then published: the view's metrics and the reply's usage entry
|
|
1140
|
+
// for routing vs your model. A lost write publishes nothing.
|
|
1143
1141
|
async function observeRouted($, router, { turnId, cfg, loop, loopVersion, request, response, nativeModel }) {
|
|
1144
1142
|
// When the reply arrived: the awaits below can cross midnight or the hour it is kept under.
|
|
1145
1143
|
const at = Date.now();
|
|
@@ -1179,7 +1177,7 @@ async function observeRouted($, router, { turnId, cfg, loop, loopVersion, reques
|
|
|
1179
1177
|
// key can name another model later.
|
|
1180
1178
|
const idOf = (model) => modelSpec(cfg, model)?.id ?? model;
|
|
1181
1179
|
if (compared)
|
|
1182
|
-
await
|
|
1180
|
+
await addUsageEntry($, router, {
|
|
1183
1181
|
turnId,
|
|
1184
1182
|
compared,
|
|
1185
1183
|
tier,
|
|
@@ -1213,7 +1211,7 @@ async function cacheTtlOf($) {
|
|
|
1213
1211
|
// A main reply against every configured model (compareModels): the loop's next states, the reply's prices by models
|
|
1214
1212
|
// key, and `reply`, its prices against your model now, or null. The readout never breaks a turn. A reply whose cache
|
|
1215
1213
|
// lifetime cannot be read still moves the models' caches and is not counted; any other failure leaves the loop's
|
|
1216
|
-
// states
|
|
1214
|
+
// states as they were and counts nothing.
|
|
1217
1215
|
async function compareYours($, router, loop, response, servedEffort, nativeModel, cfg = router.config) {
|
|
1218
1216
|
try {
|
|
1219
1217
|
if (!response.usage?.model) return null;
|
|
@@ -1233,10 +1231,10 @@ async function compareYours($, router, loop, response, servedEffort, nativeModel
|
|
|
1233
1231
|
}
|
|
1234
1232
|
}
|
|
1235
1233
|
|
|
1236
|
-
// A routed reply
|
|
1237
|
-
//
|
|
1234
|
+
// A routed reply, priced against your model now, added to the turn's running replies, which the view counts at once
|
|
1235
|
+
// and turn.complete adds to the usage record. `route` is the served model's `id@effort`, `yours` your model's id,
|
|
1238
1236
|
// `byId` the reply priced against each model as yours, keyed by model id; `at` when the reply arrived (ms).
|
|
1239
|
-
async function
|
|
1237
|
+
async function addUsageEntry($, router, { turnId, compared, tier, route, yours, byId, usage, at }) {
|
|
1240
1238
|
try {
|
|
1241
1239
|
const view = router.view ?? (await readView($, router));
|
|
1242
1240
|
if (compared.reply) {
|
|
@@ -1253,17 +1251,28 @@ async function addSavings($, router, { turnId, compared, tier, route, yours, byI
|
|
|
1253
1251
|
usage,
|
|
1254
1252
|
});
|
|
1255
1253
|
router.turnUsage.set(turnId, [...(router.turnUsage.get(turnId) ?? []), entry]);
|
|
1254
|
+
await updateView($, router, ledgerView(router, ['usage']));
|
|
1256
1255
|
}
|
|
1257
|
-
await updateView($, router, { savingsBy: addReplies(view.savingsBy ?? {}, compared.replies) });
|
|
1258
1256
|
} catch {}
|
|
1259
1257
|
}
|
|
1260
1258
|
|
|
1261
|
-
// A finished turn's replies, and the switch that started it, added to this session's usage record.
|
|
1262
|
-
|
|
1259
|
+
// A finished turn's replies, and the switch that started it, added to this session's usage record. The turn leaves
|
|
1260
|
+
// the running replies in the tick it enters the held record, so a view written at any point counts it once. A failed
|
|
1261
|
+
// store write keeps it in the held record, which the next turn writes; a failed read or a Reset before that drops it
|
|
1262
|
+
// from both.
|
|
1263
|
+
async function flushUsage($, router, turnId, switched) {
|
|
1263
1264
|
try {
|
|
1264
|
-
|
|
1265
|
-
|
|
1266
|
-
|
|
1265
|
+
await addToLedger($, router, 'usage', (record, now) => {
|
|
1266
|
+
// Read here, not before the awaits: a reply can still join the turn while the record is read.
|
|
1267
|
+
const entries = router.turnUsage.get(turnId) ?? [];
|
|
1268
|
+
router.turnUsage.delete(turnId);
|
|
1269
|
+
return recordUsage(record, entries, { switched, now });
|
|
1270
|
+
});
|
|
1271
|
+
} catch {
|
|
1272
|
+
} finally {
|
|
1273
|
+
router.turnUsage.delete(turnId);
|
|
1274
|
+
await updateView($, router, ledgerView(router, ['usage'])).catch(() => {});
|
|
1275
|
+
}
|
|
1267
1276
|
}
|
|
1268
1277
|
|
|
1269
1278
|
// A turn's prompt, its origin and the classifier's request text. A progress turn asks about the goal it serves, with
|
|
@@ -1865,7 +1874,6 @@ function paneActions($, router, view) {
|
|
|
1865
1874
|
}
|
|
1866
1875
|
await updateView($, router, {
|
|
1867
1876
|
activityStats: null,
|
|
1868
|
-
savingsBy: {},
|
|
1869
1877
|
confirmDelete: false,
|
|
1870
1878
|
usageOpen: null,
|
|
1871
1879
|
...ledgerView(router),
|
|
@@ -2186,9 +2194,7 @@ export function register(on, options) {
|
|
|
2186
2194
|
if (turn?.mode === 'off') {
|
|
2187
2195
|
if ((await $.session.id()) === turn.sessionId) router.afterDown = advanceDown(router.afterDown, turn).after;
|
|
2188
2196
|
} else if (turn) await recordActivity($, router, turn);
|
|
2189
|
-
|
|
2190
|
-
router.turnUsage.delete(e.turnId);
|
|
2191
|
-
if (entries) await flushUsage($, router, entries, turn?.switched ?? null);
|
|
2197
|
+
if (router.turnUsage.has(e.turnId)) await flushUsage($, router, e.turnId, turn?.switched ?? null);
|
|
2192
2198
|
await flushDecisions($, router, e.turnId, turn);
|
|
2193
2199
|
return next(e);
|
|
2194
2200
|
});
|
package/lib/band.mjs
CHANGED
|
@@ -4,7 +4,8 @@ import {
|
|
|
4
4
|
classifierLabel,
|
|
5
5
|
classifierStatus,
|
|
6
6
|
classifierWarns,
|
|
7
|
-
|
|
7
|
+
comparedModel,
|
|
8
|
+
costParts,
|
|
8
9
|
modeLever,
|
|
9
10
|
modelColor,
|
|
10
11
|
modelName,
|
|
@@ -12,17 +13,18 @@ import {
|
|
|
12
13
|
percentPair,
|
|
13
14
|
reasonActivity,
|
|
14
15
|
routeLabel,
|
|
16
|
+
routingAnswer,
|
|
15
17
|
routingModeText,
|
|
16
|
-
|
|
18
|
+
savingsColor,
|
|
19
|
+
sessionUsage,
|
|
17
20
|
shortReason,
|
|
18
21
|
TIER_COLOR,
|
|
19
22
|
unavailableReason,
|
|
20
23
|
usageMetrics,
|
|
21
24
|
usd,
|
|
22
|
-
yourModel,
|
|
23
25
|
} from './display.mjs';
|
|
24
26
|
import { isSameModel } from './route.mjs';
|
|
25
|
-
import {
|
|
27
|
+
import { modelSpec, readout } from './savings.mjs';
|
|
26
28
|
import { PASS_WHY } from './subagents.mjs';
|
|
27
29
|
|
|
28
30
|
const BARS = '▂▄▆█';
|
|
@@ -202,7 +204,7 @@ export function bandSegments(config, view, usage, actions) {
|
|
|
202
204
|
segment(5, part([routingModeText(view), modeLever(config, view)].filter(Boolean).join(' · '), { dimColor: true })),
|
|
203
205
|
support ? segment(4, part(support, { dimColor: true })) : null,
|
|
204
206
|
Number.isFinite(switchUsd) && switchUsd >= 0.005
|
|
205
|
-
? segment(4, part(`switch
|
|
207
|
+
? segment(4, part(`next switch ≈ ${usd(switchUsd)}`, { dimColor: true }))
|
|
206
208
|
: null,
|
|
207
209
|
metrics.contextBar.percent === null && metrics.cacheBar.percent === null
|
|
208
210
|
? null
|
|
@@ -210,42 +212,45 @@ export function bandSegments(config, view, usage, actions) {
|
|
|
210
212
|
].filter(Boolean);
|
|
211
213
|
}
|
|
212
214
|
|
|
213
|
-
// The
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
}
|
|
219
|
-
|
|
220
|
-
// The second row of the detailed band: this session against the model /model selects now, one scope end to end, so a
|
|
221
|
-
// /model change shows the whole session against the new one. The answer stays; the reply count and the cost behind it
|
|
222
|
-
// drop first when the band is narrow.
|
|
223
|
-
export function sessionRow(config, view, columns) {
|
|
215
|
+
// The second row of the detailed band: this session against your model, as the pane's SESSION block starts, and the
|
|
216
|
+
// parts that add up to it: "routing +$0.18 (+4%) vs Opus 5.5 · high = −$0.42 cheaper models +$0.60 cache
|
|
217
|
+
// rebuilds · 34 replies". When the band is narrow the parts lose their words, then the reply count goes, then the
|
|
218
|
+
// parts.
|
|
219
|
+
export function sessionRow(config, view, columns, now = Date.now()) {
|
|
224
220
|
const label = part('session ', { dimColor: true });
|
|
225
221
|
if (!modelSpec(config, view.nativeModel))
|
|
226
222
|
return [label, part(`${modelName(view.nativeModel)} has no list price`, { dimColor: true })];
|
|
227
|
-
const
|
|
223
|
+
const summary = sessionUsage(view, now);
|
|
224
|
+
const r = readout(summary);
|
|
228
225
|
if (!r.replies) return [part('no replies yet', { dimColor: true })];
|
|
229
|
-
const
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
226
|
+
const count = `${r.replies} repl${r.replies === 1 ? 'y' : 'ies'}`;
|
|
227
|
+
const answer = [
|
|
228
|
+
label,
|
|
229
|
+
part(routingAnswer(r), { color: savingsColor(r.differenceUsd), bold: true }),
|
|
230
|
+
part(` vs ${comparedModel(config, view, summary)}`),
|
|
231
|
+
];
|
|
232
|
+
const width = (row) => row.reduce((n, p) => n + p.text.length, 0);
|
|
233
|
+
const fit = (rows) =>
|
|
234
|
+
fitSegments(
|
|
235
|
+
[segment(0, ...(rows.find((row) => !Number.isFinite(columns) || width(row) <= columns) ?? answer))],
|
|
236
|
+
columns,
|
|
237
|
+
).flatMap((s) => s.parts);
|
|
238
|
+
// Below EARLY_REPLIES one switch decides the sum, so the parts wait, as on the pane.
|
|
239
|
+
if (r.early) return fit([[...answer, part(`${SEPARATOR}${count} · early sample`, { dimColor: true })], answer]);
|
|
240
|
+
const replies = part(`${SEPARATOR}${count}`, { dimColor: true });
|
|
241
|
+
const parts = costParts(r);
|
|
242
|
+
const named = parts.flatMap(([amount, words, value], i) => [
|
|
243
|
+
part(i ? ' ' : ' = ', { dimColor: true }),
|
|
244
|
+
part(`${amount} ${words}`, { color: savingsColor(value) }),
|
|
245
|
+
]);
|
|
246
|
+
const bare = parts.length
|
|
247
|
+
? [
|
|
248
|
+
part(` = ${parts.map(([amount], i) => (i ? amount.replace(/^([−+])/, '$1 ') : amount)).join(' ')}`, {
|
|
249
|
+
dimColor: true,
|
|
250
|
+
}),
|
|
251
|
+
]
|
|
252
|
+
: [];
|
|
253
|
+
return fit([[...answer, ...named, replies], [...answer, ...bare, replies], [...answer, ...bare], answer]);
|
|
249
254
|
}
|
|
250
255
|
|
|
251
256
|
// The row revealed under the pointer: pins and the mode control while routing is on, and the one- or two-row choice.
|
package/lib/display.mjs
CHANGED
|
@@ -13,8 +13,8 @@ import { ratesAt } from './cost.mjs';
|
|
|
13
13
|
import { applyMode, modeLabel, modeRoute } from './modes.mjs';
|
|
14
14
|
import { PROBE_RESULTS } from './probe-results.mjs';
|
|
15
15
|
import { cellForModel } from './route.mjs';
|
|
16
|
-
import {
|
|
17
|
-
import { allTime } from './usage.mjs';
|
|
16
|
+
import { modelSpec, readout } from './savings.mjs';
|
|
17
|
+
import { allTime, periodRange, readUsage, summarize, withReplies } from './usage.mjs';
|
|
18
18
|
|
|
19
19
|
// Cost order, cheapest first, so the colors read as a scale. Hex values: every surface draws them.
|
|
20
20
|
export const TIER_COLOR = { micro: '#5fd7af', low: '#5f9fe0', medium: '#d7af5f', high: '#e0708a' };
|
|
@@ -339,33 +339,58 @@ export const yourModel = (view) => routeLabel(view.nativeModel, view.nativeEffor
|
|
|
339
339
|
export const sinceDate = (ms) => new Date(ms).toLocaleDateString('en-US', { month: 'short', day: 'numeric' });
|
|
340
340
|
const replyCount = (n, what = 'replies') => `${n} ${n === 1 ? what.replace(/ies$/, 'y') : what}`;
|
|
341
341
|
|
|
342
|
-
//
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
342
|
+
// A usage record with the running turns' replies (view.usageLive) added, which every readout counts: the band, the
|
|
343
|
+
// pane and /router show the same replies at the same moment.
|
|
344
|
+
export const liveUsage = (view, record) => withReplies(readUsage(record), view.usageLive ?? []);
|
|
345
|
+
// This session's routed replies, the running turns' included.
|
|
346
|
+
export const sessionUsage = (view, now) => summarize(liveUsage(view, view.usageSession), periodRange('session', now));
|
|
347
|
+
|
|
348
|
+
// The models a session's replies were priced against, when they are not the one `/model` selects now: after a
|
|
349
|
+
// `/model` change the record keeps each reply against the model selected when it ran. Null when they are.
|
|
350
|
+
export function recordedModels(config, view, summary) {
|
|
351
|
+
const ids = Object.keys(summary.yours ?? {});
|
|
352
|
+
if (!ids.length || (ids.length === 1 && ids[0] === modelSpec(config, view.nativeModel)?.id)) return null;
|
|
353
|
+
return ids.map(modelName).join(', ');
|
|
347
354
|
}
|
|
355
|
+
// The model a session's replies were priced against: the one /model selects now, with its effort, or the models the
|
|
356
|
+
// record kept them against.
|
|
357
|
+
export const comparedModel = (config, view, summary) => recordedModels(config, view, summary) ?? yourModel(view);
|
|
358
|
+
|
|
359
|
+
// The words every readout uses. Routing against your model: "routing −$1.84 (−22%)", minus less than your model, plus
|
|
360
|
+
// more; then the parts that add up to it.
|
|
361
|
+
export const routingAnswer = (r) => `routing ${savingsAmount(r)}`;
|
|
362
|
+
export const COST_PART = { cheaper: 'cheaper models', stronger: 'stronger models', switches: 'cache rebuilds' };
|
|
363
|
+
// The parts of a readout that are not zero, as [signed amount, words]: "−$2.31", "cheaper models".
|
|
364
|
+
export function costParts(r) {
|
|
365
|
+
return [
|
|
366
|
+
[r.cheaperUsd, COST_PART.cheaper],
|
|
367
|
+
[r.strongerUsd, COST_PART.stronger],
|
|
368
|
+
[r.switchUsd, COST_PART.switches],
|
|
369
|
+
]
|
|
370
|
+
.filter(([value]) => value !== 0)
|
|
371
|
+
.map(([value, words]) => [signedUsd(value), words, value]);
|
|
372
|
+
}
|
|
373
|
+
// Cache rebuilds cost more than the cheaper models saved: the switches lost money.
|
|
374
|
+
export const switchesLost = (r) => r.switchUsd > 0 && r.switchUsd > -r.cheaperUsd;
|
|
348
375
|
|
|
349
|
-
// The `/router` line without a surface: this session's
|
|
376
|
+
// The `/router` line without a surface: this session's readout, or why there is none yet, with the saved totals
|
|
350
377
|
// when this session has no routed reply.
|
|
351
|
-
export function savingsLine(view) {
|
|
352
|
-
const
|
|
353
|
-
const session = readout(
|
|
354
|
-
const
|
|
355
|
-
const across = readout(kept.totals);
|
|
378
|
+
export function savingsLine(config, view, now) {
|
|
379
|
+
const summary = sessionUsage(view, now);
|
|
380
|
+
const session = readout(summary);
|
|
381
|
+
const label = `vs ${comparedModel(config, view, summary)}`;
|
|
356
382
|
if (!session.replies) {
|
|
383
|
+
const kept = allTime(view.usageStore, view.savingsStore);
|
|
384
|
+
const across = readout(kept.totals);
|
|
357
385
|
const saved = across.early
|
|
358
386
|
? ''
|
|
359
|
-
: `; ${
|
|
360
|
-
return
|
|
387
|
+
: `; ${routingAnswer(across)} since ${sinceDate(kept.since)}, each session against its own model`;
|
|
388
|
+
return `Session ${label}: no routed replies yet${saved}`;
|
|
361
389
|
}
|
|
362
|
-
if (session.early)
|
|
363
|
-
|
|
364
|
-
const
|
|
365
|
-
|
|
366
|
-
? `${stronger.count} of ${replyCount(session.replies)} on ${stronger.several ? 'stronger models' : modelName(stronger.model)}`
|
|
367
|
-
: replyCount(session.replies);
|
|
368
|
-
return `${label}: ${savingsAmount(session)} this session, ${replies}, est. at list prices`;
|
|
390
|
+
if (session.early)
|
|
391
|
+
return `Session: ${routingAnswer(session)} ${label}, ${replyCount(session.replies)}, early sample, est. at list prices`;
|
|
392
|
+
const parts = costParts(session).map(([amount, words]) => `${amount} ${words}`);
|
|
393
|
+
return `Session: ${routingAnswer(session)} ${label}${parts.length ? ` = ${parts.join(' ')}` : ''}, ${replyCount(session.replies)}, est. at list prices`;
|
|
369
394
|
}
|
|
370
395
|
|
|
371
396
|
export const GATEWAY_SETTINGS = 'v0.8 gateway settings remain · see /router';
|