@alexeiled/claude-router 1.21.2 → 1.22.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +30 -0
- package/README.md +7 -7
- package/docs/activity-routing.md +56 -24
- package/docs/architecture.md +32 -20
- package/docs/configuration.md +4 -4
- package/docs/evaluation.md +21 -6
- package/docs/native-router.md +1 -1
- package/docs/user-guide.md +17 -15
- package/hooks/native-router.mjs +113 -8
- package/lib/activity-stats.mjs +5 -3
- package/lib/activity.mjs +29 -3
- package/lib/band.mjs +7 -2
- package/lib/classifier-apis.mjs +79 -24
- package/lib/classifier-client.mjs +16 -5
- package/lib/classifier-contract.mjs +47 -11
- package/lib/config.mjs +1 -1
- package/lib/decisions.mjs +51 -4
- package/lib/display.mjs +10 -6
- package/lib/facts.mjs +32 -1
- package/lib/panel.mjs +5 -3
- package/lib/route.mjs +32 -8
- package/lib/view.mjs +11 -1
- package/package.json +1 -1
package/docs/user-guide.md
CHANGED
|
@@ -27,7 +27,7 @@ In Claude Code:
|
|
|
27
27
|
- **OpenAI**: an OpenAI API key. The prompt text goes to OpenAI.
|
|
28
28
|
- **Ollama**: no credential. Run `ollama serve` and pull the model; the prompt text stays on this machine.
|
|
29
29
|
3. To use another classifier, open the pane's **Classifier** tab and select its row. See [Choose the classifier](#choose-the-classifier).
|
|
30
|
-
4. Check the band above the prompt: `Routing on · ready`. On a model that no tier routes to, and that no activity override uses with activity routing `on`, the session starts with routing off and the band says why; run `/router auto` to route.
|
|
30
|
+
4. Check the band above the prompt: `Routing on · ready`. On a model that no tier routes to, and that no activity override uses with activity routing `on`, the session starts with routing off and the band says why; run `/router auto` to route. That model then keeps running until a decision moves it: a stronger tier the classifier supports (two votes, or one very sure answer) moves at once; a cheaper route goes at once on a new task when it pays back its cache write, and waits for the dwell and a passed check on a task in progress.
|
|
31
31
|
|
|
32
32
|
Keys are stored as sensitive plugin options. Do not paste them into a model conversation. The [configuration guide](configuration.md) covers settings and migrations.
|
|
33
33
|
|
|
@@ -43,7 +43,7 @@ Router draws one line above the prompt. The line ends with a **Router** button t
|
|
|
43
43
|
Read the first line from left to right:
|
|
44
44
|
|
|
45
45
|
- **Tier meter** `▂▄▆█`: one bar per tier, lit up to the current tier in its color, like signal strength. The same bars name the tiers everywhere: `▂` micro, `▂▄` low, `▂▄▆` medium, `▂▄▆█` high.
|
|
46
|
-
- **Model**: the model and effort
|
|
46
|
+
- **Model**: the model and effort the router chose for this turn, such as `Opus 5.5 · xhigh`; see [Changes during a task](#changes-during-a-task) for when the effort is not the one sent. **fallback** after it means Claude Code answered with another model.
|
|
47
47
|
- **Activity**: the kind of work, in orange italic: `code`, `debug`, `explore`, `plan`, `review`, `ops` (commands: git, builds, tests, CI) or `docs`. See [Route by activity](#route-by-activity). When the reason already names the activity, the band does not repeat it.
|
|
48
48
|
- **Reason**: a glyph and a few words. `↑` moved up, `↓` moved down, `=` stayed, `…` waits, `!` a problem, `⏵` a pin: `↑ clearly stronger`, `↓ cheaper fits`, `= fits`, `… cheaper, waiting`, `↑ code` for an activity move.
|
|
49
49
|
- **Classifier support**: what the active classifier, such as Jev or Clef Flash, gave the switch against the bar the policy required, such as `Jev 88% ≥ 82%`. When whole percents would read as equal, it shows tenths: `Jev 81.0% < 81.4%`.
|
|
@@ -227,9 +227,9 @@ The router keeps no bill. Its dollars are estimates at configured list prices an
|
|
|
227
227
|
|
|
228
228
|
## Route by activity
|
|
229
229
|
|
|
230
|
-
A tier says how hard the work is. The
|
|
230
|
+
A tier says how hard the work is. The goal says what the active task must deliver, while phase describes this step. The existing activity names describe goals:
|
|
231
231
|
|
|
232
|
-
|
|
|
232
|
+
| Goal | The task produces |
|
|
233
233
|
| --------- | --------------------------------------------------------------- |
|
|
234
234
|
| `code` | Changed code or config, including tests |
|
|
235
235
|
| `debug` | A found cause: failing tests, stack traces, "why does X..." |
|
|
@@ -239,7 +239,7 @@ A tier says how hard the work is. The activity says what kind of work it is. The
|
|
|
239
239
|
| `ops` | Executed commands: git, builds, test runs, CI, deploys |
|
|
240
240
|
| `docs` | Prose for people: README, docs, comments, commit messages |
|
|
241
241
|
|
|
242
|
-
A mixed turn such as "fix it and commit"
|
|
242
|
+
A mixed turn such as "fix it and commit" keeps goal `code`. Verification and delivery steps can retain that goal while changing phase. A positively supported verification-only or execution-only phase can use the `ops` cell; mixed checks and fixes cannot. The router then picks the route by tier and activity. With the default overrides, a `low` coding turn runs on Sonnet 5.5 at `high`, a `low` git turn stays on Haiku 5.5 at `high`, and a `medium` git turn moves from Opus 5.5 to Haiku 5.5 at `high`. [Activity routing](activity-routing.md) explains the default matrix, why each cell is there, and how to change it.
|
|
243
243
|
|
|
244
244
|
The mode decides what happens with the label:
|
|
245
245
|
|
|
@@ -255,13 +255,13 @@ When an override runs, **NEXT TURN** on the Now tab adds a line with the tier's
|
|
|
255
255
|
How the router moves inside a tier:
|
|
256
256
|
|
|
257
257
|
- It compares routes, not labels. If the new activity resolves to the route already running, nothing switches.
|
|
258
|
-
- A move to a stronger route needs the classifier's support, as an upgrade between tiers does. A move to a cheaper one must pay back its cache write within the payback horizon. One vote is enough for either, because the
|
|
259
|
-
-
|
|
258
|
+
- A move to a stronger route needs the classifier's support, as an upgrade between tiers does. A move to a cheaper one must pay back its cache write within the payback horizon. One vote is enough for either, because this proposes a cell for the current step, not a new task goal.
|
|
259
|
+
- A goal applies only when the classifier gives it at least 60% probability (`policy.activityMass`). Phase must independently meet that same bar before a verification/execution step can propose `ops`; its support is capped at the lower of goal and phase support, not a multiplied probability. Below that, or on `uncertain`, a new task goes back to its base route if the move passes the same checks. On the running task the activity can move either way while the turn runs; see [Changes during a task](#changes-during-a-task).
|
|
260
260
|
- A pinned turn uses the pin's base route and shows no activity.
|
|
261
261
|
- When the classifier gives no answer at all (a timeout, a failure, a pause, or a missing key), the running route stays. When it answers the tier but gives no usable activity, only the running activity stays: the tier answer still applies, so the turn can change tier and keeps its activity there. In `off` the base route always runs.
|
|
262
262
|
- A fresh session starts with routing on when the session model is a tier's route. With activity routing `on`, a model that only an activity override uses also counts: Sonnet 5.5 on the defaults. In `off` it does not.
|
|
263
263
|
|
|
264
|
-
The band
|
|
264
|
+
The band names the accepted goal and phase when available, and distinguishes the effective route activity: `code / verify → ops`. These are classifier estimates, not observed proof of the work done. The two-row band and the **Replies** strip show the activities of recent replies.
|
|
265
265
|
|
|
266
266
|
### Changes during a task
|
|
267
267
|
|
|
@@ -269,7 +269,7 @@ The classifier labels a turn from its prompt, and the work can change after that
|
|
|
269
269
|
|
|
270
270
|
- **On every later step of a turn** it reads the turn's tool calls. A turn labelled `ops` or `explore` that edits a code file moves at once to the tier's `code` route (`docs` when it edits only Markdown or `docs/` files). The tools are the evidence, so this needs no classifier support, and the cache cost does not hold it back; only the cash cap on a credits-billed model can. This move only goes up. The toast and the band show it as `↗ coding`.
|
|
271
271
|
- **Every 4 requests inside a turn** (`policy.reviewEveryRequests`) it asks the classifier again, in the background, with the turn's prompt and the recent dialogue. A step never waits for the answer: it applies at the first step after it arrives. Each review sends the classifier the same kind of text a new turn does. `0` turns reviews off.
|
|
272
|
-
- **A new prompt that continues the task** is decided the same way as a review. Claude Code stamps each prompt with where it came from. A task notification,
|
|
272
|
+
- **A new prompt that continues the task** is decided the same way as a review. Claude Code stamps each prompt with where it came from. A task notification, observer report or automatic continuation is treated as progress, not as a new goal. It is asked about the known goal with the report as dialogue. Such a report may belong to an older task, so it cannot justify a downgrade. A message from you or another agent, or "go on", counts as the running task when the classifier scores it as a continuation. After a resume the router does not know your last request yet, so a notification is judged as it reads.
|
|
273
273
|
|
|
274
274
|
On the running task:
|
|
275
275
|
|
|
@@ -282,20 +282,22 @@ On the running task:
|
|
|
282
282
|
|
|
283
283
|
A failed check blocks every one of these, and so does a check whose result is not the check's own (chained, redirected, piped, after `;` or `||`, or in the background). While no checkpoint is met, the route stays: for a tier the band shows `… down after a checkpoint`; a held activity keeps the running one and shows `= fits`. A cheaper activity also needs 90% probability (`policy.downgradeMass`; 65% in `cost`).
|
|
284
284
|
- A cheaper tier lands where the answer as a whole points: when the classifier says `micro` 63% and `low` 30%, the turn moves to `low`, which 93% of the answer supports, instead of staying until `micro` alone reaches 90%.
|
|
285
|
-
- A pinned turn does not move. A
|
|
285
|
+
- A pinned turn does not move. A prompt not scored as a continuation is routed as a fresh turn. The bounded in-memory goal anchor is created or replaced only on positive new-task evidence, retained only on positive continuation, and dropped on ambiguity, history replacement or session end. A continuation without an anchor cannot seed a short reply as a new goal. Ollama provides neither phase nor continuity, so it does not retain this anchor; recent dialogue can still help its goal answer.
|
|
286
286
|
|
|
287
287
|
These thresholds are starting values. They are not tuned on data yet.
|
|
288
288
|
|
|
289
|
+
On Claude Code 2.1.322 an effort that the router changed inside a turn was not sent in the 6 cases found, while a changed model was, so the band and the decision record can name an effort that did not run. See [What we do not know yet](activity-routing.md#what-we-do-not-know-yet).
|
|
290
|
+
|
|
289
291
|
### Read the activity on the pane
|
|
290
292
|
|
|
291
293
|
- **Now**: the first line of **NEXT TURN** names the classifier's activity with its share and the next answer, such as `code 84% debug 8%`. An `uncertain` answer is shown dim: it applies no activity. In `on`, a line under it names the cell when the route is an override, such as `code at low runs on Sonnet 5.5 · high · tier route Haiku 5.5 · high`, and **why** reads `↑ code` or `↓ ops` when the activity moved the route; a refused move, a hold, or a tier change keeps its reason.
|
|
292
294
|
- **Routes**: see [Edit routes and policy](#edit-routes-and-policy). Each activity is a row of the grid, and a cell shows the model the activity runs at that tier.
|
|
293
295
|
- **Usage**: **BY** with **Activity** lists the period's replies, cost and difference against no routing per activity, and `mostly on`, the route most of its replies used, such as `Sonnet 5.5 · high`; opening a row lists its routes. Replies whose turn had no applied activity count as `no activity`. The **cache rebuilds** row counts the route switches by tier and by activity together. See [Compare routing with no routing](#compare-routing-with-no-routing).
|
|
294
|
-
- **Classifier**: **THIS SESSION** gives **
|
|
296
|
+
- **Classifier**: **THIS SESSION** gives **Tool compatibility (not accuracy)**, how often the tools fit the classifier's goal (see [Evaluation](evaluation.md#tool-agreement)).
|
|
295
297
|
|
|
296
|
-
On the **Classifier** tab, **ACROSS SESSIONS · since the last reset** shows the counts kept across sessions, once there are any: the turns labelled from their tools, **
|
|
298
|
+
On the **Classifier** tab, **ACROSS SESSIONS · since the last reset** shows the counts kept across sessions, once there are any: the turns labelled from their tools, **Tool compatibility** over them, **Code/ops/explore**, the same agreement over the turns the classifier answered `code`, `ops` or `explore` (omitted when there are none), the two most frequent mismatches as `predicted → observed count` (`code → read 3`), **Activity moves**, the moves to another route inside a tier, taken and refused for any reason, **Escalations**, such as `after a cheaper activity move: 1 in 12 moves`, the tool-error escalations that followed a move to a cheaper route in `on` while the turns stayed on it in `on` (turning routing off, a turn with routing off, or a turn in `off` ends the watch, a pin in between does not, and an escalation the credits cap held counts). Each of these lines appears only when it has data. The active classifier's p95 wait across sessions shows on its **Deadline** line instead (not recorded with activity routing `off`). This agreement compares the classifier's raw answer, so it can differ from the session's. The pane reads these counts when the session starts and when it opens, and adds this session's turns as they finish; another session's new latency and escalation readings show the next time the pane opens. **Delete history** on the Usage tab deletes these counts with the usage and subagent records, for this session and across sessions.
|
|
297
299
|
|
|
298
|
-
`/router` without a UI surface
|
|
300
|
+
`/router` without a UI surface distinguishes `Goal: code (95%)`, `Phase: verify` and `Route activity: ops`, followed by the resolved route. It labels tool compatibility explicitly as not accuracy. Goal-contract counters start a separate series; pre-change turn-label counters remain stored but are excluded until **Delete history** removes them. It also prints the session in the band's words, such as `Session: $6.40 billed · −$1.84 (−22%) vs Opus 5.5 $8.24 · 64 replies`, after a `Plan: 5h 48% · 7d 43%` line on a Claude plan.
|
|
299
301
|
|
|
300
302
|
## Change routing
|
|
301
303
|
|
|
@@ -318,7 +320,7 @@ A routing mode moves each tier's route one step toward cost or quality; pins and
|
|
|
318
320
|
|
|
319
321
|
### Routing decisions
|
|
320
322
|
|
|
321
|
-
Each routed turn adds rows to the session's decision record in `$.store` (`decisions:v1:<session id>`): one when the turn starts, one per classifier review or tool-based move inside it, and one when it ends. A row holds the prompt's origin (`composer`, `task-notification`, …, or `unknown`), the classifier's tier and activity with their shares, whether it counted as the running task, the tier, activity, route and reason chosen, and at the end the turn's tool bucket and how many replies were priced. It holds no prompt, tool or file text. `/router decisions [n]` prints them. Each session keeps its newest 60 rows; records of the 20 most recent other sessions stay, for at most 30 days. **Delete history** deletes them all.
|
|
323
|
+
Each routed turn adds rows to the session's decision record in `$.store` (`decisions:v1:<session id>`): one when the turn starts, one per classifier review or tool-based move inside it, and one when it ends. A row holds the prompt's origin (`composer`, `task-notification`, …, or `unknown`), the classifier's tier and activity with their shares, whether it counted as the running task, the tier, activity, route and reason chosen, and at the end the turn's tool bucket and how many replies were priced. It holds no prompt, tool or file text. The route is the one chosen, which is not always the one sent; see [Changes during a task](#changes-during-a-task). `/router decisions [n]` prints them. For a tier move or an activity move up, taken or held, the row also holds the probability mass the policy compared and the bar it compared it with, printed after the reason: `(upgrade-pending 52% vs bar 79%)`. A held move can show a mass above its bar: it then waits for a second vote or for a checkpoint. On `downgrade-pending` the mass is the support for the classifier's own choice; a tier between it and the running one can be closer to its bar. A `jump`, a move on the turn's own tool calls and a context or availability override show neither number. Each session keeps its newest 60 rows; records of the 20 most recent other sessions stay, for at most 30 days. **Delete history** deletes them all.
|
|
322
324
|
|
|
323
325
|
A pin does not change the next turn after the pinned turn finishes. A fresh session on a model that a tier routes to, or with activity routing `on` a model an activity override uses (Sonnet 5.5 on the defaults), starts with routing on. A fresh session on any other model starts with routing off. `/clear` starts the new session with routing on. Resuming a saved session restores its saved routing on or off.
|
|
324
326
|
|
|
@@ -353,7 +355,7 @@ A new model ID needs an entry in `router.json` `models` with its price, context
|
|
|
353
355
|
|
|
354
356
|
The **Policy** tab writes each control inside the sentence it completes, and names the default beside a value that differs from it. One **Save** writes them with any route changes:
|
|
355
357
|
|
|
356
|
-
- **Start** at a tier: the tier a session starts from when its model is no tier's model (`baselineTier`).
|
|
358
|
+
- **Start** at a tier: the tier a session starts from when its model is no tier's model (`baselineTier`). The session's model runs until a decision moves it to that tier's route or another tier.
|
|
357
359
|
- **Going down** after 1, 2, or 3 agreeing turns in a row, and only if a cache write pays back within 1, 3, 5, or 10 later turns. A new or reset history needs one vote.
|
|
358
360
|
- **Going up** needs the classifier's support at 75%, raised up to 90% as the switch costs more. Read-only here; see [Configuration](configuration.md).
|
|
359
361
|
- **Activities** apply at 50%, 60%, 70%, or 80% certainty or more. Below it a new task runs its tier's model; the running task follows [Changes during a task](#changes-during-a-task). See [Activity routing](activity-routing.md#change-a-cell).
|
package/hooks/native-router.mjs
CHANGED
|
@@ -1,9 +1,11 @@
|
|
|
1
1
|
import {
|
|
2
|
+
ACTIVITY_CONTRACT,
|
|
2
3
|
advanceDown,
|
|
3
4
|
advanceRun,
|
|
4
5
|
emptyMetrics,
|
|
5
6
|
emptySession,
|
|
6
7
|
emptyStore,
|
|
8
|
+
LEGACY_STORE_PREFIX,
|
|
7
9
|
METRICS_PREFIX,
|
|
8
10
|
mergeMetrics,
|
|
9
11
|
mergeStores,
|
|
@@ -47,6 +49,7 @@ import { changedLeaves, notSaved, restored, rewrittenConfig } from '../lib/confi
|
|
|
47
49
|
import {
|
|
48
50
|
addTurnRow,
|
|
49
51
|
appendDecisions,
|
|
52
|
+
continuedGoal,
|
|
50
53
|
DECISIONS_PREFIX,
|
|
51
54
|
decisionLines,
|
|
52
55
|
decisionRow,
|
|
@@ -75,7 +78,16 @@ import {
|
|
|
75
78
|
unavailableReason,
|
|
76
79
|
unpricedLine,
|
|
77
80
|
} from '../lib/display.mjs';
|
|
78
|
-
import {
|
|
81
|
+
import {
|
|
82
|
+
clip,
|
|
83
|
+
currentStepText,
|
|
84
|
+
hasImages,
|
|
85
|
+
historySignature,
|
|
86
|
+
observedActivity,
|
|
87
|
+
promptIndex,
|
|
88
|
+
routingBucket,
|
|
89
|
+
stripReminders,
|
|
90
|
+
} from '../lib/facts.mjs';
|
|
79
91
|
import { applyMode, modeLabel, readProjectMode, resolveRoutingMode } from '../lib/modes.mjs';
|
|
80
92
|
import { renderPanel, routeDraftOf, routingChanges } from '../lib/panel.mjs';
|
|
81
93
|
import { sameTaskOf } from '../lib/policy.mjs';
|
|
@@ -92,6 +104,7 @@ import {
|
|
|
92
104
|
nativeFacts,
|
|
93
105
|
observedMove,
|
|
94
106
|
observeResponse,
|
|
107
|
+
ownRoute,
|
|
95
108
|
prepareLoop,
|
|
96
109
|
resetHistory,
|
|
97
110
|
} from '../lib/route.mjs';
|
|
@@ -238,6 +251,9 @@ function createRouter(options) {
|
|
|
238
251
|
// The session's active goal, the last prompt from a goal origin, clipped, `{ sessionId, text }`, in memory only;
|
|
239
252
|
// and each progress turn's report (`progressNotes`), which the classifier reads as dialogue (progressInput).
|
|
240
253
|
goal: null,
|
|
254
|
+
goalTurnId: null,
|
|
255
|
+
goalEpoch: 0,
|
|
256
|
+
turnGoals: new Map(),
|
|
241
257
|
progressNotes: new Map(),
|
|
242
258
|
// Each turn's decision rows until turn.complete writes them, with the turn's end, to the session's record, held as
|
|
243
259
|
// `{ sessionId, record }` once read (`decisionRecord`).
|
|
@@ -780,6 +796,9 @@ async function* passMain($, e, next, loop, version, nativeModel, reason, router)
|
|
|
780
796
|
stopReason: response.stopReason,
|
|
781
797
|
now: Date.now(),
|
|
782
798
|
});
|
|
799
|
+
// Before routing has chosen a route, the one in use is the model serving now: /model can change it with routing
|
|
800
|
+
// off. Once a decision exists, the last routed route is kept, as for every model.
|
|
801
|
+
if (reason === 'manual' && !loop.decision) observed.route = ownRoute(router.config, e.model);
|
|
783
802
|
// Routing chose nothing here, so the reply is not counted; your model's cache still follows it.
|
|
784
803
|
const compared = await compareYours($, router, loop, response, e.effort ?? null, nativeModel);
|
|
785
804
|
if (compared) observed.yoursBy = compared.states;
|
|
@@ -812,6 +831,7 @@ async function decideTurn($, router, e, signal, step) {
|
|
|
812
831
|
$.session.messages({ as: 'api' }),
|
|
813
832
|
]);
|
|
814
833
|
if (!(await live())) return null;
|
|
834
|
+
validateGoal(router, e.turnId, messages);
|
|
815
835
|
const origin = router.turnOrigins.get(e.turnId) ?? 'unknown';
|
|
816
836
|
// A progress turn is the same task (sameTaskOf).
|
|
817
837
|
const facts = {
|
|
@@ -821,6 +841,7 @@ async function decideTurn($, router, e, signal, step) {
|
|
|
821
841
|
effort: e.effort,
|
|
822
842
|
turnId: e.turnId,
|
|
823
843
|
contextTokens: context.tokens,
|
|
844
|
+
progress: isProgressOrigin(origin),
|
|
824
845
|
}),
|
|
825
846
|
progress: isProgressOrigin(origin),
|
|
826
847
|
};
|
|
@@ -846,6 +867,8 @@ async function decideTurn($, router, e, signal, step) {
|
|
|
846
867
|
endpoint: credentials.endpoint,
|
|
847
868
|
prompt: asked.prompt,
|
|
848
869
|
turns: asked.turns,
|
|
870
|
+
activeGoal: router.turnGoals.get(e.turnId)?.goal,
|
|
871
|
+
currentStep: router.progressNotes.get(e.turnId) ?? facts.prompt,
|
|
849
872
|
signal: controller.signal,
|
|
850
873
|
});
|
|
851
874
|
// The wait for advice alone, comparable with the activity probe's p95; the routing after it is not the classifier's.
|
|
@@ -870,6 +893,7 @@ async function decideTurn($, router, e, signal, step) {
|
|
|
870
893
|
if (written.isSet && pin) router.turnPins.set(e.turnId, pinPresses);
|
|
871
894
|
// Manual, /clear and turn completion abort the controller; one may land during the write.
|
|
872
895
|
if (!written.isSet || controller.signal.aborted) return null;
|
|
896
|
+
rememberGoal(router, e.turnId, result.advice, cfg.policy.continuationMass, messages);
|
|
873
897
|
if (previous?.model && previous.model !== selected.decision.model && !selected.decision.pinned)
|
|
874
898
|
$.ui.toast(switchToast(cfg, previous, selected.decision, selected.decision.estimate));
|
|
875
899
|
const activity = turnActivity(cfg, result.advice, previous, selected.decision);
|
|
@@ -935,6 +959,10 @@ async function decideTurn($, router, e, signal, step) {
|
|
|
935
959
|
effort: selected.decision.effort,
|
|
936
960
|
reason: selected.decision.reason,
|
|
937
961
|
activity: activity.label,
|
|
962
|
+
goal: selected.decision.goal,
|
|
963
|
+
taskPhase: selected.decision.taskPhase,
|
|
964
|
+
phaseChoice: result.advice?.phase?.choice ?? null,
|
|
965
|
+
phaseProbabilities: result.advice?.phase?.probabilities ?? null,
|
|
938
966
|
contextTokens: context.tokens,
|
|
939
967
|
contextKnown: context.known,
|
|
940
968
|
comparison: selected.decision.comparison,
|
|
@@ -969,6 +997,7 @@ function stepFacts(cfg, router, loop, e, messages, context) {
|
|
|
969
997
|
effort: e.effort,
|
|
970
998
|
turnId: e.turnId,
|
|
971
999
|
contextTokens: context.tokens,
|
|
1000
|
+
progress: isProgressOrigin(router.turnOrigins.get(e.turnId)),
|
|
972
1001
|
});
|
|
973
1002
|
}
|
|
974
1003
|
|
|
@@ -1033,8 +1062,10 @@ async function reviewMidTurn($, router, cfg, loop, e, { context, nativeModel, av
|
|
|
1033
1062
|
if (!review.answer.advice || loop.decision?.pinned || review.generation !== loop.generation) return loop;
|
|
1034
1063
|
// Only the advice is from the review; the checkpoint, context and cache facts are read now.
|
|
1035
1064
|
const messages = await transcript($);
|
|
1036
|
-
if (!messages) return loop;
|
|
1037
|
-
|
|
1065
|
+
if (!messages || historySignature(messages, review.observedUpTo) !== review.prefix) return loop;
|
|
1066
|
+
// Appends preserve the task evidence, but a command-only phase cannot authorize work after its snapshot.
|
|
1067
|
+
const advice =
|
|
1068
|
+
messages.length === review.observedUpTo ? review.answer.advice : { ...review.answer.advice, phase: null };
|
|
1038
1069
|
const facts = { ...stepFacts(cfg, router, loop, e, messages, context), midTurn: true };
|
|
1039
1070
|
const selected = chooseRoute(cfg, loop, {
|
|
1040
1071
|
facts,
|
|
@@ -1079,6 +1110,10 @@ async function reviewMidTurn($, router, cfg, loop, e, { context, nativeModel, av
|
|
|
1079
1110
|
estimate: selected.decision.estimate ?? null,
|
|
1080
1111
|
activityChoice: advice.activity?.choice ?? null,
|
|
1081
1112
|
activityProbabilities: advice.activity?.probabilities ?? null,
|
|
1113
|
+
goal: selected.decision.goal,
|
|
1114
|
+
taskPhase: selected.decision.taskPhase,
|
|
1115
|
+
phaseChoice: advice.phase?.choice ?? null,
|
|
1116
|
+
phaseProbabilities: advice.phase?.probabilities ?? null,
|
|
1082
1117
|
});
|
|
1083
1118
|
// Tool calls up to the review's snapshot were judged by it; later ones can still move the route up.
|
|
1084
1119
|
return { ...selected, observedFrom: review.observedUpTo };
|
|
@@ -1086,7 +1121,13 @@ async function reviewMidTurn($, router, cfg, loop, e, { context, nativeModel, av
|
|
|
1086
1121
|
const every = cfg.policy.reviewEveryRequests;
|
|
1087
1122
|
if (review || every === 0 || !loop.decision?.tier || loop.decision.pinned) return loop;
|
|
1088
1123
|
if ((loop.requests ?? 0) - (loop.reviewedAt ?? 0) < every) return loop;
|
|
1089
|
-
const job = {
|
|
1124
|
+
const job = {
|
|
1125
|
+
controller: new AbortController(),
|
|
1126
|
+
generation: loop.generation,
|
|
1127
|
+
observedUpTo: 0,
|
|
1128
|
+
prefix: null,
|
|
1129
|
+
answer: null,
|
|
1130
|
+
};
|
|
1090
1131
|
router.reviews.set(e.turnId, job);
|
|
1091
1132
|
try {
|
|
1092
1133
|
const [credentials, messages] = await Promise.all([
|
|
@@ -1094,6 +1135,7 @@ async function reviewMidTurn($, router, cfg, loop, e, { context, nativeModel, av
|
|
|
1094
1135
|
$.session.messages({ as: 'api' }),
|
|
1095
1136
|
]);
|
|
1096
1137
|
job.observedUpTo = messages.length;
|
|
1138
|
+
job.prefix = historySignature(messages);
|
|
1097
1139
|
const facts = stepFacts(cfg, router, loop, e, messages, context);
|
|
1098
1140
|
clientOf(router, cfg.classifier)
|
|
1099
1141
|
.ask({
|
|
@@ -1104,6 +1146,8 @@ async function reviewMidTurn($, router, cfg, loop, e, { context, nativeModel, av
|
|
|
1104
1146
|
endpoint: credentials.endpoint,
|
|
1105
1147
|
prompt: facts.prompt,
|
|
1106
1148
|
turns: facts.turns,
|
|
1149
|
+
activeGoal: router.turnGoals.get(e.turnId)?.goal,
|
|
1150
|
+
currentStep: currentStepText(messages, facts.prompt, cfg.context.maxTextChars),
|
|
1107
1151
|
signal: job.controller.signal,
|
|
1108
1152
|
})
|
|
1109
1153
|
.then((answer) => {
|
|
@@ -1282,17 +1326,70 @@ async function flushUsage($, router, turnId, switched) {
|
|
|
1282
1326
|
// its report as dialogue (progressInput); without a goal from this session it reads as before. A goal origin's prompt
|
|
1283
1327
|
// becomes the goal.
|
|
1284
1328
|
async function startTurn($, router, e) {
|
|
1285
|
-
const
|
|
1329
|
+
const epoch = ++router.goalEpoch;
|
|
1330
|
+
router.goalTurnId = e.turnId;
|
|
1331
|
+
await fixTurnConfig($, router, e.turnId);
|
|
1332
|
+
const text = clip(stripReminders(e.text), router.config.context.maxTextChars);
|
|
1286
1333
|
const origin = originOfTurn(router, e.text);
|
|
1287
1334
|
router.turnOrigins.set(e.turnId, origin);
|
|
1288
1335
|
const sessionId = await $.session.id();
|
|
1289
|
-
if (
|
|
1336
|
+
if (router.goalEpoch !== epoch || router.goalTurnId !== e.turnId) return;
|
|
1337
|
+
const previous = router.goal?.sessionId === sessionId ? router.goal.text : null;
|
|
1338
|
+
const candidate = isGoalOrigin(origin) ? text : null;
|
|
1339
|
+
router.turnGoals.set(e.turnId, {
|
|
1340
|
+
sessionId,
|
|
1341
|
+
previous,
|
|
1342
|
+
previousRecord: previous ? router.goal : null,
|
|
1343
|
+
candidate,
|
|
1344
|
+
goal: previous,
|
|
1345
|
+
});
|
|
1346
|
+
if (isProgressOrigin(origin) && previous) {
|
|
1290
1347
|
router.prompts.set(e.turnId, router.goal.text);
|
|
1291
1348
|
router.progressNotes.set(e.turnId, text);
|
|
1292
1349
|
return;
|
|
1293
1350
|
}
|
|
1294
1351
|
router.prompts.set(e.turnId, text);
|
|
1295
|
-
|
|
1352
|
+
}
|
|
1353
|
+
|
|
1354
|
+
function validateGoal(router, turnId, messages) {
|
|
1355
|
+
const pending = router.turnGoals.get(turnId);
|
|
1356
|
+
const record = pending?.previousRecord;
|
|
1357
|
+
if (!record || historySignature(messages, record.prefixCount) === record.prefix) return;
|
|
1358
|
+
if (router.goal === record) router.goal = null;
|
|
1359
|
+
const note = router.progressNotes.get(turnId);
|
|
1360
|
+
if (note) {
|
|
1361
|
+
router.prompts.set(turnId, note);
|
|
1362
|
+
router.progressNotes.delete(turnId);
|
|
1363
|
+
}
|
|
1364
|
+
router.turnGoals.set(turnId, { ...pending, previous: null, previousRecord: null, goal: null });
|
|
1365
|
+
}
|
|
1366
|
+
|
|
1367
|
+
function rememberGoal(router, turnId, advice, threshold, messages) {
|
|
1368
|
+
const pending = router.turnGoals.get(turnId);
|
|
1369
|
+
if (!pending?.candidate || router.goalTurnId !== turnId) return;
|
|
1370
|
+
const text = continuedGoal(pending.previous, pending.candidate, advice?.continuation, threshold);
|
|
1371
|
+
const retained = text && text === pending.previous && advice?.continuation > 0.5;
|
|
1372
|
+
router.goal = text
|
|
1373
|
+
? retained
|
|
1374
|
+
? pending.previousRecord
|
|
1375
|
+
: {
|
|
1376
|
+
sessionId: pending.sessionId,
|
|
1377
|
+
text,
|
|
1378
|
+
prefixCount: messages.length,
|
|
1379
|
+
prefix: historySignature(messages),
|
|
1380
|
+
}
|
|
1381
|
+
: null;
|
|
1382
|
+
router.turnGoals.set(turnId, { ...pending, goal: text });
|
|
1383
|
+
}
|
|
1384
|
+
|
|
1385
|
+
function clearGoal(router) {
|
|
1386
|
+
router.goal = null;
|
|
1387
|
+
router.goalTurnId = null;
|
|
1388
|
+
router.goalEpoch += 1;
|
|
1389
|
+
router.turnGoals.clear();
|
|
1390
|
+
// A notification must not keep a goal that a rewind/compaction invalidated.
|
|
1391
|
+
for (const [turnId, note] of router.progressNotes) router.prompts.set(turnId, note);
|
|
1392
|
+
router.progressNotes.clear();
|
|
1296
1393
|
}
|
|
1297
1394
|
|
|
1298
1395
|
// The engine stamps each prompt with where it came from (PromptOrigin); turn.start does not carry it. An idle
|
|
@@ -1338,6 +1435,7 @@ async function flushDecisions($, router, turnId, turn) {
|
|
|
1338
1435
|
router.turnDecisions.delete(turnId);
|
|
1339
1436
|
router.turnOrigins.delete(turnId);
|
|
1340
1437
|
router.progressNotes.delete(turnId);
|
|
1438
|
+
router.turnGoals.delete(turnId);
|
|
1341
1439
|
if (!rows.length) return;
|
|
1342
1440
|
try {
|
|
1343
1441
|
const resets = router.statsResets;
|
|
@@ -1864,6 +1962,8 @@ function paneActions($, router, view) {
|
|
|
1864
1962
|
const keys = (await $.store.keys()).filter(
|
|
1865
1963
|
(key) =>
|
|
1866
1964
|
key.startsWith(DECISIONS_PREFIX) ||
|
|
1965
|
+
key === LEGACY_STORE_PREFIX ||
|
|
1966
|
+
key.startsWith(`${LEGACY_STORE_PREFIX}:`) ||
|
|
1867
1967
|
Object.values(LEDGERS).some(({ prefix, legacy }) => key.startsWith(prefix) || key === legacy),
|
|
1868
1968
|
);
|
|
1869
1969
|
await Promise.all(keys.map((key) => $.store.delete(key)));
|
|
@@ -1949,6 +2049,7 @@ function paneActions($, router, view) {
|
|
|
1949
2049
|
async function endSession($, router) {
|
|
1950
2050
|
for (const controller of router.controllers) controller.abort();
|
|
1951
2051
|
router.decisions.clear();
|
|
2052
|
+
clearGoal(router);
|
|
1952
2053
|
router.prompts.clear();
|
|
1953
2054
|
router.turnOrigins.clear();
|
|
1954
2055
|
router.progressNotes.clear();
|
|
@@ -2033,6 +2134,9 @@ export function register(on, options) {
|
|
|
2033
2134
|
Boolean(base?.includes('127.0.0.1:43170') || base?.includes('localhost:43170'));
|
|
2034
2135
|
await updateView($, router, {
|
|
2035
2136
|
nativeModel: model,
|
|
2137
|
+
...(existing.activityContract !== ACTIVITY_CONTRACT
|
|
2138
|
+
? { activityContract: ACTIVITY_CONTRACT, activityStats: null }
|
|
2139
|
+
: {}),
|
|
2036
2140
|
...(await startView($, router, model)),
|
|
2037
2141
|
phase: gateway || !supported ? 'unavailable' : 'ready',
|
|
2038
2142
|
error: !supported ? 'requires Claude Code 2.1.289 or newer' : gateway ? GATEWAY_SETTINGS : null,
|
|
@@ -2081,7 +2185,6 @@ export function register(on, options) {
|
|
|
2081
2185
|
on('command.run', { command: 'router' }, ($, e) => routerCommand($, router, e.args));
|
|
2082
2186
|
|
|
2083
2187
|
on('turn.start', async ($, e, next) => {
|
|
2084
|
-
await fixTurnConfig($, router, e.turnId);
|
|
2085
2188
|
await startTurn($, router, e);
|
|
2086
2189
|
return next(e);
|
|
2087
2190
|
});
|
|
@@ -2113,6 +2216,7 @@ export function register(on, options) {
|
|
|
2113
2216
|
let loopVersion = loaded.version;
|
|
2114
2217
|
// The session's model names its tier in the configured routes: a mode's table can run it at another tier.
|
|
2115
2218
|
let loop = prepareLoop(loaded.value ?? emptyLoop(turn.configured, e.model), e.messageCount);
|
|
2219
|
+
if (loaded.value && loop.generation !== loaded.value.generation) clearGoal(router);
|
|
2116
2220
|
if (view.phase === 'unavailable' || mode === 'manual') {
|
|
2117
2221
|
return yield* passMain($, e, next, loop, loopVersion, nativeModel, 'manual', router);
|
|
2118
2222
|
}
|
|
@@ -2212,6 +2316,7 @@ export function register(on, options) {
|
|
|
2212
2316
|
const ref = { ...LOOP, id: 'main' };
|
|
2213
2317
|
const loop = (await $.state.get(ref)).value;
|
|
2214
2318
|
if (loop) await $.state.set(ref, resetHistory(loop));
|
|
2319
|
+
clearGoal(router);
|
|
2215
2320
|
for (const controller of router.controllers) controller.abort();
|
|
2216
2321
|
}
|
|
2217
2322
|
return result;
|
package/lib/activity-stats.mjs
CHANGED
|
@@ -9,10 +9,12 @@ import { afterResetOf, earlierSince, isRecord } from './records.mjs';
|
|
|
9
9
|
// A turn's bucket from its tools (observedActivity in facts.mjs). `shell` and `check` came in 1.16: a version before
|
|
10
10
|
// reads a store that holds them as empty.
|
|
11
11
|
export const OBSERVED = ['code', 'docs', 'shell', 'ops', 'check', 'read', 'talk'];
|
|
12
|
-
//
|
|
13
|
-
export const
|
|
12
|
+
// Goal labels must not pool with the old turn-activity contract. Legacy records remain until Delete history.
|
|
13
|
+
export const ACTIVITY_CONTRACT = 'goal-phase-v1';
|
|
14
|
+
export const LEGACY_STORE_PREFIX = 'activity:stats:v1';
|
|
15
|
+
export const STORE_KEY = 'activity:goals:v1';
|
|
14
16
|
// Each session keeps its own stats under this prefix and its session id, as the metrics do.
|
|
15
|
-
export const STORE_PREFIX = 'activity:
|
|
17
|
+
export const STORE_PREFIX = 'activity:goals:v1:';
|
|
16
18
|
// Each session keeps its own metrics under this prefix and its session id, as savings does (SAVINGS_PREFIX): sessions
|
|
17
19
|
// finishing turns at the same moment do not overwrite each other. The readout is their sum (mergeMetrics).
|
|
18
20
|
export const METRICS_PREFIX = 'activity:metrics:v1:';
|
package/lib/activity.mjs
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
// Activity routing (docs/plans/2026-10-09-activity-routing-plan.md §5): which activity a turn applies, how two routes
|
|
2
2
|
// compare, and when a route change inside one tier is worth it. Pure.
|
|
3
|
+
|
|
4
|
+
import { PHASE_VALUES } from './classifier-contract.mjs';
|
|
3
5
|
import { ACTIVITIES, EFFORTS, resolveRoute, routeSpec } from './config.mjs';
|
|
4
6
|
import { routeEffort } from './cost.mjs';
|
|
5
7
|
import { cashGate, sameTaskOf } from './policy.mjs';
|
|
@@ -12,6 +14,29 @@ export function acceptActivity(config, advice) {
|
|
|
12
14
|
return answer.probabilities?.[answer.choice] >= config.policy.activityMass ? answer.choice : null;
|
|
13
15
|
}
|
|
14
16
|
|
|
17
|
+
export function acceptPhase(config, advice) {
|
|
18
|
+
const answer = advice?.phase;
|
|
19
|
+
if (!answer || answer.choice === 'uncertain' || !PHASE_VALUES.includes(answer.choice)) return null;
|
|
20
|
+
return answer.probabilities?.[answer.choice] >= config.policy.activityMass ? answer.choice : null;
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
// The routing signal differs from the task goal only for positively identified bounded command phases.
|
|
24
|
+
// The minimum is a conservative support floor, not a multiplied or calibrated joint probability.
|
|
25
|
+
export function routingActivity(config, advice) {
|
|
26
|
+
const answer = advice?.activity;
|
|
27
|
+
if (!answer) return null;
|
|
28
|
+
const phase = acceptPhase(config, advice);
|
|
29
|
+
const mass = answer.probabilities?.[answer.choice] ?? 0;
|
|
30
|
+
if (acceptActivity(config, advice) && (phase === 'verify' || phase === 'execute'))
|
|
31
|
+
return { choice: 'ops', mass: Math.min(mass, advice.phase.probabilities[phase]) };
|
|
32
|
+
return { choice: answer.choice, mass };
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
const acceptRoutingActivity = (config, advice) => {
|
|
36
|
+
const signal = routingActivity(config, advice);
|
|
37
|
+
return ACTIVITIES.includes(signal?.choice) && signal.mass >= config.policy.activityMass ? signal.choice : null;
|
|
38
|
+
};
|
|
39
|
+
|
|
15
40
|
// 1 when route `a` is stronger than `b`, -1 when cheaper, 0 when equal in strength, null when they cannot be ordered.
|
|
16
41
|
// Models with explicit strength order by that rank; otherwise by configured output price, then effort. A null effort is `sentEffort`, the one Claude Code sends, as
|
|
17
42
|
// the request would carry it; without a named effort on both sides (none sent, or a numeric budget) two different
|
|
@@ -45,12 +70,13 @@ export const sameRequest = (config, a, b, sentEffort) =>
|
|
|
45
70
|
// (completedSince) since the last model change. An unclear label keeps the running activity.
|
|
46
71
|
export function chooseActivity(config, advice, incumbent, sentEffort, facts = {}) {
|
|
47
72
|
if (!advice?.activity) return incumbent.activity;
|
|
48
|
-
if (!sameTaskOf(config, advice, facts)) return
|
|
49
|
-
const
|
|
73
|
+
if (!sameTaskOf(config, advice, facts)) return acceptRoutingActivity(config, advice);
|
|
74
|
+
const signal = routingActivity(config, advice);
|
|
75
|
+
const label = signal.choice;
|
|
50
76
|
if (!ACTIVITIES.includes(label)) return incumbent.activity;
|
|
51
77
|
const order = compareRoutes(config, resolveRoute(config, incumbent.tier, label), incumbent.route, sentEffort);
|
|
52
78
|
if (order > 0) return label;
|
|
53
|
-
const mass =
|
|
79
|
+
const mass = signal.mass;
|
|
54
80
|
if (order < 0 && facts.downAllowed && mass >= config.policy.downgradeMass) return label;
|
|
55
81
|
return incumbent.activity;
|
|
56
82
|
}
|
package/lib/band.mjs
CHANGED
|
@@ -230,9 +230,14 @@ export function bandSegments(config, view, usage, actions) {
|
|
|
230
230
|
routeParts(view.actualModel ?? view.selectedModel ?? view.nativeModel, view.effort),
|
|
231
231
|
served(view) ? part(' fallback', { color: 'yellow' }) : null,
|
|
232
232
|
);
|
|
233
|
+
const activityText = view.goal
|
|
234
|
+
? `${view.goal}${view.taskPhase ? ` / ${view.taskPhase}` : ''}${view.activity && view.activity !== view.goal ? ` → ${view.activity}` : ''}`
|
|
235
|
+
: view.activity;
|
|
233
236
|
const activity =
|
|
234
|
-
|
|
235
|
-
|
|
237
|
+
activityText &&
|
|
238
|
+
config.activityRouting !== 'off' &&
|
|
239
|
+
((view.goal && (view.taskPhase || view.goal !== view.activity)) || !reasonNamesActivity(view.reason))
|
|
240
|
+
? segment(2, part(activityText, { color: COLOR.activity, italic: true }))
|
|
236
241
|
: null;
|
|
237
242
|
const reason = segment(1, dim(shortReason(view.reason, reasonActivity(config, view))));
|
|
238
243
|
// The warning leads and the route yields: when the band is narrow, what to fix matters more than the model kept.
|