@alexeiled/claude-router 0.6.1 → 0.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +102 -0
- package/README.md +94 -78
- package/docs/architecture.md +416 -291
- package/docs/configuration.md +153 -162
- package/docs/evaluation.md +83 -0
- package/docs/tier-share.svg +73 -44
- package/docs/user-guide.md +176 -114
- package/lib/config.mjs +100 -13
- package/lib/facts.mjs +35 -6
- package/lib/gateway.mjs +33 -30
- package/lib/jev.mjs +7 -4
- package/lib/policy.mjs +7 -2
- package/lib/rewrite.mjs +93 -3
- package/lib/router.mjs +41 -8
- package/lib/runtime.mjs +72 -7
- package/lib/status.mjs +2 -1
- package/lib/store.mjs +2 -1
- package/package.json +3 -2
- package/scripts/ensure-gateway.mjs +34 -8
- package/scripts/gateway.mjs +12 -2
- package/scripts/status.mjs +16 -3
- package/scripts/statusline.mjs +6 -2
package/docs/user-guide.md
CHANGED
|
@@ -1,140 +1,202 @@
|
|
|
1
1
|
# User guide
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
After the [install](../README.md#install), work in Claude Code as usual. The
|
|
4
|
+
router selects the model and the effort for each turn. This guide shows how
|
|
5
|
+
to read what it does, and how to control it.
|
|
6
|
+
|
|
7
|
+
- [Check the setup](#check-the-setup)
|
|
8
|
+
- [A session, step by step](#a-session-step-by-step)
|
|
9
|
+
- [Read the status line](#read-the-status-line)
|
|
10
|
+
- [Read `/router:status`](#read-routerstatus)
|
|
11
|
+
- [Pin a tier](#pin-a-tier)
|
|
12
|
+
- [Read the decision log](#read-the-decision-log)
|
|
13
|
+
- [Update](#update)
|
|
14
|
+
- [Stop using the router](#stop-using-the-router)
|
|
15
|
+
- [Troubleshooting](#troubleshooting)
|
|
16
|
+
|
|
17
|
+
## Check the setup
|
|
18
|
+
|
|
19
|
+
1. Start Claude Code. `/model` shows `Jev Router (auto)` as the model.
|
|
20
|
+
2. Send a prompt. If setup added the status line, it shows the route, for
|
|
21
|
+
example `jev-router ▸ sonnet-5 · high · same-tier`.
|
|
22
|
+
3. Run `/router:status`. It shows `Jev routing: active.`
|
|
23
|
+
|
|
24
|
+
If a step fails, read [Troubleshooting](#troubleshooting).
|
|
25
|
+
|
|
26
|
+
## A session, step by step
|
|
27
|
+
|
|
28
|
+
An example of four turns. Each box is one prompt with its route and reason.
|
|
29
|
+
|
|
30
|
+
```mermaid
|
|
31
|
+
flowchart LR
|
|
32
|
+
T1["<b>1</b> · design a cache layer<br/>Opus · xhigh<br/><i>jump</i>"]
|
|
33
|
+
T2["<b>2</b> · yes, go on<br/>Opus · xhigh<br/><i>continuation</i>"]
|
|
34
|
+
T3["<b>3</b> · rename foo to bar<br/>Opus · xhigh<br/><i>downgrade-pending</i>"]
|
|
35
|
+
T4["<b>4</b> · fix the typo<br/>Sonnet<br/><i>downgrade</i>"]
|
|
36
|
+
T1 --> T2 --> T3 --> T4
|
|
37
|
+
|
|
38
|
+
classDef opus fill:#0d366b,stroke:#0d366b,color:#ffffff
|
|
39
|
+
classDef sonnet fill:#cde2fb,stroke:#2a78d6,color:#0b0b0b
|
|
40
|
+
class T1,T2,T3 opus
|
|
41
|
+
class T4 sonnet
|
|
42
|
+
```
|
|
4
43
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
The
|
|
9
|
-
requests it exits, unless a turn waits for a tool result, for example a
|
|
10
|
-
permission prompt that you have not answered. The next prompt starts it again
|
|
11
|
-
(`gateway.idleShutdownMs` in [configuration](configuration.md)).
|
|
44
|
+
1. A confident vote for hard work goes up two tiers at once.
|
|
45
|
+
2. A prompt that continues the task keeps the route.
|
|
46
|
+
3. One vote for a lower tier is not enough. The route stays.
|
|
47
|
+
4. The second vote in a row moves the route down.
|
|
12
48
|
|
|
13
|
-
|
|
49
|
+
The tool calls in a turn keep the route of the turn, so the cache stays warm.
|
|
50
|
+
The [README](../README.md#how-jev-selects-a-tier) gives all the rules.
|
|
14
51
|
|
|
15
|
-
|
|
16
|
-
TYPESAFE_API_KEY=… node scripts/gateway.mjs &
|
|
17
|
-
ANTHROPIC_BASE_URL=http://127.0.0.1:43170 claude --plugin-dir . --model jev-router
|
|
18
|
-
```
|
|
52
|
+
## Read the status line
|
|
19
53
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
- A prompt that continues the task keeps the route. Examples: "continue",
|
|
27
|
-
"yes", "now fix the tests".
|
|
28
|
-
- After two failed repairs of the same error, the route goes up one tier. The
|
|
29
|
-
route stays there for two turns. A repair is an edit between the two errors.
|
|
30
|
-
- An upgrade needs two consecutive votes for a higher tier. A jump of two
|
|
31
|
-
tiers with high confidence happens at once. The required confidence goes up
|
|
32
|
-
with the cost to read the context again on the new model. Opus at `high` and
|
|
33
|
-
Opus at `xhigh` are two caches: a change of effort rewrites the cached
|
|
34
|
-
conversation, so `medium` to `high` is not free.
|
|
35
|
-
- A downgrade needs two consecutive confident votes.
|
|
36
|
-
- When the conversation gets shorter, after a compaction or a rewind, the
|
|
37
|
-
gateway forgets the cached prefixes, the pending votes and the escalation
|
|
38
|
-
hold. They were about turns that are no longer in the conversation.
|
|
39
|
-
- The cold-write guard: an automatic switch to a model that bills usage
|
|
40
|
-
credits is refused when that model's cache is not warm and the first cache
|
|
41
|
-
write would cost more than `policy.cashCapUsd`. Then the strongest plan tier
|
|
42
|
-
serves the turn. The guard limits that one estimated write, not the cost of
|
|
43
|
-
the turn: output is not counted. Behind the gateway, Claude Code does not
|
|
44
|
-
show its consent prompt for these credits. No default model bills credits;
|
|
45
|
-
this applies once you add one in `router.json`.
|
|
46
|
-
- When Jev fails or times out, or when there is no key, the baseline tier
|
|
47
|
-
serves the turn. After three failures in a row, new turns skip Jev for one
|
|
48
|
-
minute and then try it once, so an outage costs one slow turn a minute, not
|
|
49
|
-
a second on every turn. `/router:status` shows the pause.
|
|
50
|
-
- When Claude Code sends the same request again (after a 429, a 529 or a
|
|
51
|
-
dropped connection), the gateway keeps the route of that turn. A retry is
|
|
52
|
-
not a new vote and makes no Jev call.
|
|
53
|
-
- Jev receives the prompt and the last six turns of text. Tool results are not
|
|
54
|
-
sent. No other data leaves the machine, except the usual Anthropic request.
|
|
55
|
-
|
|
56
|
-
A Claude Code turn starts at about 100k tokens of system prompt and tool
|
|
57
|
-
definitions, so the first switch to a model is the expensive one.
|
|
58
|
-
|
|
59
|
-
## Pin a tier by hand
|
|
60
|
-
|
|
61
|
-
Type the tier skill as a command. The turn runs on the model of that skill.
|
|
62
|
-
The gateway sends the real model id unchanged.
|
|
54
|
+
```text
|
|
55
|
+
jev-router ▸ opus-5-5 · xhigh · upgrade
|
|
56
|
+
│ │ └─ reason for the route
|
|
57
|
+
│ └─ effort
|
|
58
|
+
└─ model of the last turn
|
|
59
|
+
```
|
|
63
60
|
|
|
61
|
+
| Line | Meaning |
|
|
62
|
+
| ------------------------------------------------ | --------------------------------------------------- |
|
|
63
|
+
| `jev-router ▸ …` | The route of the last turn in this session. |
|
|
64
|
+
| `jev-router: no turn yet` | This session has no routed turn. |
|
|
65
|
+
| `router: gateway off, the next prompt starts it` | The gateway stopped after idle time, or it crashed. |
|
|
66
|
+
|
|
67
|
+
The line changes after each message.
|
|
68
|
+
|
|
69
|
+
**The route changed:**
|
|
70
|
+
|
|
71
|
+
| Reason | Meaning |
|
|
72
|
+
| ------------- | ---------------------------------------------------------------------------- |
|
|
73
|
+
| `upgrade` | Two votes in a row for a higher tier, with enough confidence. |
|
|
74
|
+
| `jump` | One vote confident enough to go up two tiers. |
|
|
75
|
+
| `downgrade` | Two votes in a row for a lower tier, with enough confidence. |
|
|
76
|
+
| `escalation` | The same error came back after an edit. The route went up one tier. |
|
|
77
|
+
| `context-fit` | The selected model cannot hold the context. A larger window serves the turn. |
|
|
78
|
+
| `cash-gate` | The cold-write guard stopped a switch to a `credits` model. |
|
|
79
|
+
|
|
80
|
+
**The route stayed:**
|
|
81
|
+
|
|
82
|
+
| Reason | Meaning |
|
|
83
|
+
| ------------------- | ------------------------------------------------------------------- |
|
|
84
|
+
| `same-tier` | Jev agreed with the current tier. |
|
|
85
|
+
| `continuation` | The prompt continues the task. |
|
|
86
|
+
| `upgrade-pending` | One vote for a higher tier. The next vote decides. |
|
|
87
|
+
| `downgrade-pending` | One vote for a lower tier. The next vote decides. |
|
|
88
|
+
| `hold` | The route stays up for two turns after an escalation. |
|
|
89
|
+
| `uncertain` | Jev did not select a tier. |
|
|
90
|
+
| `no-advice` | No Jev answer: no key, an error, a pause, or a prompt without text. |
|
|
91
|
+
|
|
92
|
+
`forced` means that the gateway was started with `--force-tier`.
|
|
93
|
+
|
|
94
|
+
## Read `/router:status`
|
|
95
|
+
|
|
96
|
+
`/router:status` shows the gateway, the routes and the last turn. For a
|
|
97
|
+
vote, it also shows the numbers:
|
|
98
|
+
|
|
99
|
+
```text
|
|
100
|
+
Router v0.7.0, gateway: http://127.0.0.1:43170, alias `jev-router`.
|
|
101
|
+
Jev routing: active.
|
|
102
|
+
Default tier: low.
|
|
103
|
+
…
|
|
104
|
+
Last turn: low → claude-sonnet-5 at high, reason upgrade-pending, context 84210 tokens, cache reads 83904.
|
|
105
|
+
Why: Jev asked for a higher tier; the route stays until the votes and the mass are enough.
|
|
106
|
+
Estimate: upgrade mass 0.78 against a bar of 0.84, switching tax $0.66 at list prices, 1 vote(s) in a row, cache: candidate unknown, current warm.
|
|
64
107
|
```
|
|
108
|
+
|
|
109
|
+
- **Upgrade mass** is the Jev confidence in a higher tier. It must reach the
|
|
110
|
+
bar.
|
|
111
|
+
- **The bar** grows with the switching tax: the cost to write the context
|
|
112
|
+
into a cold cache. Here Opus has no warm cache.
|
|
113
|
+
- **Cache** is `warm`, `expired` or `unknown`. `unknown` means no response
|
|
114
|
+
for that cache since the session started or the conversation got shorter.
|
|
115
|
+
- **Dollars** are list prices. On a subscription plan, they show the relative
|
|
116
|
+
cost only.
|
|
117
|
+
|
|
118
|
+
## Pin a tier
|
|
119
|
+
|
|
120
|
+
To run one turn on a fixed tier, type the tier skill before the prompt:
|
|
121
|
+
|
|
122
|
+
```text
|
|
65
123
|
/router:high redesign the auth flow
|
|
66
124
|
/router:micro rename foo to bar in this file
|
|
67
125
|
```
|
|
68
126
|
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
## See the current route
|
|
127
|
+
The status line shows the reason `pinned`. Tool calls in that turn stay on the
|
|
128
|
+
tier; the next prompt goes back to Jev.
|
|
72
129
|
|
|
73
|
-
`/
|
|
74
|
-
|
|
75
|
-
a vote it shows the estimate: the probability mass, the bar it had to reach,
|
|
76
|
-
the switching tax and whether each cache was `warm`, `expired` or `unknown`.
|
|
77
|
-
`unknown` means no response for that cache since the session started or the
|
|
78
|
-
conversation got shorter. The dollars are list prices. For plan models they
|
|
79
|
-
are a list-price equivalent, not a charge.
|
|
130
|
+
`/model <name>` also works, but it stops the routing for the rest of the
|
|
131
|
+
session.
|
|
80
132
|
|
|
81
|
-
|
|
82
|
-
while the session uses the router:
|
|
133
|
+
## Read the decision log
|
|
83
134
|
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
- `router: no turn yet`: the session has no routed turn.
|
|
88
|
-
- `router: gateway off, the next prompt starts it`: the gateway stopped after
|
|
89
|
-
idle time, or it crashed. Either way the next prompt starts it.
|
|
135
|
+
The gateway writes one JSON line for each routed request to
|
|
136
|
+
`~/.claude/plugins/data/router-alexei-led-claude-router/decisions.jsonl`. The
|
|
137
|
+
log has no prompt text.
|
|
90
138
|
|
|
91
|
-
|
|
139
|
+
| Field | Content |
|
|
140
|
+
| -------------- | ---------------------------------------------------------------------------------- |
|
|
141
|
+
| `tier` | The tier of the request. |
|
|
142
|
+
| `model`, `effort` | The model id and the effort that the gateway sent. `effort` is `null` for a model without effort. |
|
|
143
|
+
| `reason` | The rule that decided. Tool calls show `tool-continuation`. `error`: routing failed and the default tier served. |
|
|
144
|
+
| `advice` | The Jev probabilities for each tier, and for "continues the task". |
|
|
145
|
+
| `estimate` | The numbers of a vote: confidence, bar, switching tax, cache state. |
|
|
146
|
+
| `shadow` | The cost of the Jev choice against the current route. The policy does not use it. |
|
|
147
|
+
| `observed` | The usage of the response: model, effort, context tokens, cache reads, output. |
|
|
148
|
+
| `failed` | Anthropic answered a routed turn with an error: status, tier, model, effort. |
|
|
149
|
+
| `historyBreak` | A compaction or a rewind. The votes and the cache estimates reset. |
|
|
150
|
+
| `cacheReset` | The context shrank by more than 20%. The cache estimates reset. |
|
|
92
151
|
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
The gateway writes one line for each routed request to `decisions.jsonl` in
|
|
96
|
-
the plugin data directory. The directory is
|
|
97
|
-
`~/.claude/plugins/data/router-<marketplace>/`. For a gateway that you started
|
|
98
|
-
by hand, the directory is `$TMPDIR/router/`.
|
|
152
|
+
Two useful queries:
|
|
99
153
|
|
|
100
154
|
```sh
|
|
101
|
-
|
|
155
|
+
LOG=~/.claude/plugins/data/router-alexei-led-claude-router/decisions.jsonl
|
|
156
|
+
jq -r 'select(.observed) | .observed.model' $LOG | sort | uniq -c # requests by model
|
|
157
|
+
jq -r 'select(.reason) | .reason' $LOG | sort | uniq -c | sort -rn # decisions by reason
|
|
158
|
+
jq -c 'select(.failed) | .failed' $LOG # failed turns
|
|
102
159
|
```
|
|
103
160
|
|
|
104
|
-
`
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
161
|
+
`scripts/transcript-models.sh <transcript.jsonl>` shows the model of each
|
|
162
|
+
assistant message in a Claude Code transcript.
|
|
163
|
+
|
|
164
|
+
## Update
|
|
108
165
|
|
|
109
|
-
|
|
110
|
-
cost, at list prices: the next turn, each later turn, and the number of turns
|
|
111
|
-
until a cheaper route repays its cache write. The policy does not read it.
|
|
112
|
-
`historyBreak` lines mark a compaction or a rewind; `cacheReset` lines mark a
|
|
113
|
-
context that shrank by more than a fifth.
|
|
166
|
+
1. Update the marketplace and the plugin:
|
|
114
167
|
|
|
115
|
-
|
|
116
|
-
|
|
168
|
+
```sh
|
|
169
|
+
claude plugin marketplace update alexei-led-claude-router
|
|
170
|
+
claude plugin update router@alexei-led-claude-router
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
2. Restart Claude Code. The next prompt replaces the running gateway.
|
|
174
|
+
3. Run `/router:setup` once. The status line command contains the plugin
|
|
175
|
+
version.
|
|
176
|
+
|
|
177
|
+
## Stop using the router
|
|
178
|
+
|
|
179
|
+
1. In `~/.claude/settings.json`, remove `model`, `env.ANTHROPIC_BASE_URL`,
|
|
180
|
+
`env.ENABLE_TOOL_SEARCH`, `env.CLAUDE_CODE_GATEWAY_HINT_HEADERS` and the
|
|
181
|
+
`jev-router[1m]` row of `modelPicker.options`.
|
|
182
|
+
2. If setup changed your status line, restore the old command.
|
|
183
|
+
3. Restart Claude Code.
|
|
184
|
+
4. To remove the plugin, run
|
|
185
|
+
`claude plugin uninstall router@alexei-led-claude-router`.
|
|
186
|
+
|
|
187
|
+
The gateway stops by itself after two hours without requests.
|
|
117
188
|
|
|
118
189
|
## Troubleshooting
|
|
119
190
|
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
- To stop the gateway, run `pkill -f scripts/gateway.mjs`. It releases the port
|
|
133
|
-
at once and finishes the responses it is streaming. The next prompt starts it
|
|
134
|
-
again.
|
|
135
|
-
- Rate limits and overload errors from Anthropic (429, 529) reach Claude Code
|
|
136
|
-
unchanged. Claude Code waits and retries; the gateway does not add a second
|
|
137
|
-
layer of retries.
|
|
138
|
-
- Updating from 0.3.0 or earlier: those gateways cannot hand over to a newer
|
|
139
|
-
one. Stop the old one once with `pkill -f scripts/gateway.mjs`; the next
|
|
140
|
-
prompt starts the new version.
|
|
191
|
+
| Symptom | Cause | Fix |
|
|
192
|
+
| ----------------------------------------------------------------- | --------------------------------------------- | ------------------------------------------------------------------------------------ |
|
|
193
|
+
| "There's an issue with the selected model (jev-router[1m])" | The session started before the restart. | Restart Claude Code. |
|
|
194
|
+
| Claude Code rejects `jev-router` | The session does not use the gateway. | Run `/router:setup`, then restart Claude Code. |
|
|
195
|
+
| The status line shows `gateway off` after the next prompt | The gateway cannot start, for example because `router.json` has an unknown key. | Run `/router:status` to see the error. Read `gateway.log` next to `decisions.jsonl`. |
|
|
196
|
+
| Every turn has the reason `no-advice` | No Jev key, or Jev fails. | Run `/router:status`. Read the `router:` lines in `gateway.log`. |
|
|
197
|
+
| The effort is not the effort that you set | The model does not accept that effort. | The gateway uses the nearest lower level. Haiku has no effort and no thinking. |
|
|
198
|
+
| 429 or 529 errors | Anthropic rate limits or overload. | Claude Code waits and tries again. The gateway sends these errors through unchanged. |
|
|
199
|
+
| A `router.json` change has no effect | The gateway reads the configuration at start. | Run `pkill -f scripts/gateway.mjs`. The next prompt starts a new gateway. |
|
|
200
|
+
|
|
201
|
+
To see if the gateway runs, open `http://127.0.0.1:43170/v1/models`. The list
|
|
202
|
+
shows the alias.
|
package/lib/config.mjs
CHANGED
|
@@ -2,6 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
export const TIERS = ['micro', 'low', 'medium', 'high'];
|
|
4
4
|
export const EFFORTS = ['low', 'medium', 'high', 'xhigh', 'max'];
|
|
5
|
+
// Request features that Claude Code uses with its own models and that only some models accept (a beta each):
|
|
6
|
+
// `system` messages after the prompt, per-turn control, and tool additions or removals in the history.
|
|
7
|
+
export const FEATURES = ['mid-conversation-system', 'per-turn-control', 'mid-conversation-tool-changes'];
|
|
5
8
|
|
|
6
9
|
// The alias before 0.4.2. Settings written by an older /router:setup still send it; drop it once they are rare.
|
|
7
10
|
export const LEGACY_ALIAS = 'router';
|
|
@@ -22,7 +25,11 @@ export const DEFAULTS = {
|
|
|
22
25
|
// `id` is sent upstream verbatim. List prices in USD per million tokens; `cacheRead` is absolute,
|
|
23
26
|
// not a multiplier (Opus 5.5 reads at 0.05x input, the rest at the standard 0.1x). A price change must also change
|
|
24
27
|
// test/fixtures/list-prices.json, with its source and date. `output` feeds only the shadow estimate in the log.
|
|
25
|
-
// `efforts` lists what the model accepts; an empty list means no effort field and no adaptive thinking.
|
|
28
|
+
// `efforts` lists what the model accepts; an empty list means no effort field and no adaptive thinking. A change
|
|
29
|
+
// must also change test/fixtures/effort-support.json, from a new probe against the real API.
|
|
30
|
+
// `features` lists the FEATURES the model accepts; the gateway removes the others from the request. A model
|
|
31
|
+
// without the field gets none: a removed feature is lost, a feature the model rejects fails the turn. A change
|
|
32
|
+
// must also change test/fixtures/model-features.json, from a new probe.
|
|
26
33
|
models: {
|
|
27
34
|
opus: {
|
|
28
35
|
id: 'claude-opus-5-5',
|
|
@@ -32,6 +39,7 @@ export const DEFAULTS = {
|
|
|
32
39
|
contextWindow: 1_000_000,
|
|
33
40
|
billing: 'plan',
|
|
34
41
|
efforts: EFFORTS,
|
|
42
|
+
features: FEATURES,
|
|
35
43
|
},
|
|
36
44
|
sonnet: {
|
|
37
45
|
id: 'claude-sonnet-5',
|
|
@@ -41,6 +49,7 @@ export const DEFAULTS = {
|
|
|
41
49
|
contextWindow: 1_000_000,
|
|
42
50
|
billing: 'plan',
|
|
43
51
|
efforts: EFFORTS,
|
|
52
|
+
features: ['mid-conversation-system'],
|
|
44
53
|
},
|
|
45
54
|
// `maxOutput` caps `max_tokens`: Claude Code asks for 128K, Haiku 4.5 answers more than 64K with 400.
|
|
46
55
|
haiku: {
|
|
@@ -52,6 +61,7 @@ export const DEFAULTS = {
|
|
|
52
61
|
maxOutput: 64_000,
|
|
53
62
|
billing: 'plan',
|
|
54
63
|
efforts: [],
|
|
64
|
+
features: [],
|
|
55
65
|
},
|
|
56
66
|
},
|
|
57
67
|
cache: {
|
|
@@ -67,6 +77,8 @@ export const DEFAULTS = {
|
|
|
67
77
|
jumpConfidence: 0.95,
|
|
68
78
|
downgradeVotes: 2,
|
|
69
79
|
downgradeMass: 0.9,
|
|
80
|
+
downgradeSlope: 0.08, // a cold candidate raises the bar toward ~0.98, same shape as the upgrade bar
|
|
81
|
+
downgradePivotUsd: 0.5,
|
|
70
82
|
continuationMass: 0.7,
|
|
71
83
|
escalationHoldTurns: 2,
|
|
72
84
|
// Ceiling on a cold cache write to a `credits` model. No default model bills credits, so this is
|
|
@@ -80,13 +92,31 @@ export const DEFAULTS = {
|
|
|
80
92
|
|
|
81
93
|
const API_KEY_VAR = 'TYPESAFE_API_KEY';
|
|
82
94
|
|
|
83
|
-
|
|
95
|
+
// Keys a route or a model entry may carry. The other closed sections take their key sets from DEFAULTS.
|
|
96
|
+
const ROUTE_KEYS = ['model', 'effort'];
|
|
97
|
+
const MODEL_KEYS = [
|
|
98
|
+
'id',
|
|
99
|
+
'input',
|
|
100
|
+
'output',
|
|
101
|
+
'cacheRead',
|
|
102
|
+
'contextWindow',
|
|
103
|
+
'maxOutput',
|
|
104
|
+
'billing',
|
|
105
|
+
'efforts',
|
|
106
|
+
'features',
|
|
107
|
+
];
|
|
108
|
+
// A JSON file can hold these as own keys; merged into a plain object they reach the prototype chain.
|
|
109
|
+
const FORBIDDEN_KEYS = new Set(['__proto__', 'constructor', 'prototype']);
|
|
110
|
+
|
|
111
|
+
// `forcedTier` skips Jev and the policy. It comes from the daemon's --force-tier flag, never from env: a project's
|
|
112
|
+
// settings can set a hook's env, and a forced tier changes spend.
|
|
113
|
+
export function loadConfig({ env = {}, userFile = null, forcedTier = null } = {}) {
|
|
114
|
+
if (userFile !== null) checkShape(userFile);
|
|
84
115
|
const config = merge(DEFAULTS, userFile ?? {});
|
|
85
116
|
validate(config);
|
|
86
117
|
const apiKey = env[API_KEY_VAR]?.trim() || null;
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
throw new Error(`ROUTER_FORCE_TIER must be one of ${TIERS.join(', ')}`);
|
|
118
|
+
if (forcedTier !== null && !TIERS.includes(forcedTier))
|
|
119
|
+
throw new Error(`the forced tier must be one of ${TIERS.join(', ')}`);
|
|
90
120
|
return { ...config, apiKey, forcedTier };
|
|
91
121
|
}
|
|
92
122
|
|
|
@@ -99,21 +129,57 @@ function merge(base, override) {
|
|
|
99
129
|
const out = { ...base };
|
|
100
130
|
for (const [key, value] of Object.entries(override)) {
|
|
101
131
|
out[key] =
|
|
102
|
-
key
|
|
132
|
+
Object.hasOwn(base, key) && typeof value === 'object' && value !== null && !Array.isArray(value)
|
|
103
133
|
? merge(base[key], value)
|
|
104
134
|
: value;
|
|
105
135
|
}
|
|
106
136
|
return out;
|
|
107
137
|
}
|
|
108
138
|
|
|
139
|
+
// The raw file, before the merge: only known keys, and every section an object. Messages name the path, never the
|
|
140
|
+
// value. An open map (model aliases, cache TTL labels) takes any key but the forbidden ones.
|
|
141
|
+
function checkShape(file) {
|
|
142
|
+
checkObject(file, 'router.json', Object.keys(DEFAULTS));
|
|
143
|
+
const { gateway, routes, models, cache, policy, jev, context } = file;
|
|
144
|
+
if (gateway !== undefined) checkObject(gateway, 'gateway', Object.keys(DEFAULTS.gateway));
|
|
145
|
+
if (routes !== undefined) {
|
|
146
|
+
checkObject(routes, 'routes', TIERS);
|
|
147
|
+
for (const [tier, route] of Object.entries(routes)) checkObject(route, `routes.${tier}`, ROUTE_KEYS);
|
|
148
|
+
}
|
|
149
|
+
if (models !== undefined) {
|
|
150
|
+
checkObject(models, 'models');
|
|
151
|
+
for (const [alias, model] of Object.entries(models)) checkObject(model, `models.${alias}`, MODEL_KEYS);
|
|
152
|
+
}
|
|
153
|
+
if (cache !== undefined) {
|
|
154
|
+
checkObject(cache, 'cache', Object.keys(DEFAULTS.cache));
|
|
155
|
+
for (const map of ['writeMultiplier', 'ttlMs'])
|
|
156
|
+
if (cache[map] !== undefined) checkObject(cache[map], `cache.${map}`);
|
|
157
|
+
}
|
|
158
|
+
if (policy !== undefined) checkObject(policy, 'policy', Object.keys(DEFAULTS.policy));
|
|
159
|
+
if (jev !== undefined) checkObject(jev, 'jev', Object.keys(DEFAULTS.jev));
|
|
160
|
+
if (context !== undefined) checkObject(context, 'context', Object.keys(DEFAULTS.context));
|
|
161
|
+
}
|
|
162
|
+
|
|
163
|
+
// `allowed` null: an open map.
|
|
164
|
+
function checkObject(value, path, allowed = null) {
|
|
165
|
+
if (typeof value !== 'object' || value === null || Array.isArray(value)) throw new Error(`${path} must be an object`);
|
|
166
|
+
for (const key of Object.keys(value)) {
|
|
167
|
+
const at = path === 'router.json' ? key : `${path}.${key}`;
|
|
168
|
+
if (FORBIDDEN_KEYS.has(key)) throw new Error(`${at} is not allowed`);
|
|
169
|
+
if (allowed && !allowed.includes(key)) throw new Error(`${at} is not a known key`);
|
|
170
|
+
}
|
|
171
|
+
}
|
|
172
|
+
|
|
173
|
+
const isNumber = (value, min, { above = false, integer = false } = {}) =>
|
|
174
|
+
Number.isFinite(value) && (above ? value > min : value >= min) && (!integer || Number.isInteger(value));
|
|
175
|
+
|
|
109
176
|
function validate(config) {
|
|
110
177
|
for (const tier of TIERS) {
|
|
111
178
|
const route = config.routes[tier];
|
|
112
179
|
if (!route || typeof route.model !== 'string') throw new Error(`routes.${tier}.model is required`);
|
|
113
|
-
|
|
114
|
-
if (!model) throw new Error(`routes.${tier}.model "${route.model}" is not in models`);
|
|
180
|
+
if (!Object.hasOwn(config.models, route.model)) throw new Error(`routes.${tier}.model is not in models`);
|
|
115
181
|
if (route.effort !== undefined && !EFFORTS.includes(route.effort))
|
|
116
|
-
throw new Error(`routes.${tier}.effort
|
|
182
|
+
throw new Error(`routes.${tier}.effort must be one of ${EFFORTS.join(', ')}`);
|
|
117
183
|
}
|
|
118
184
|
for (const [alias, model] of Object.entries(config.models)) {
|
|
119
185
|
for (const field of ['input', 'cacheRead', 'contextWindow']) {
|
|
@@ -129,6 +195,11 @@ function validate(config) {
|
|
|
129
195
|
throw new Error(`models.${alias}.billing must be plan or credits`);
|
|
130
196
|
if (!Array.isArray(model.efforts) || model.efforts.some((e) => !EFFORTS.includes(e)))
|
|
131
197
|
throw new Error(`models.${alias}.efforts must list valid effort levels`);
|
|
198
|
+
if (
|
|
199
|
+
model.features !== undefined &&
|
|
200
|
+
!(Array.isArray(model.features) && model.features.every((f) => FEATURES.includes(f)))
|
|
201
|
+
)
|
|
202
|
+
throw new Error(`models.${alias}.features must list known features (${FEATURES.join(', ')})`);
|
|
132
203
|
}
|
|
133
204
|
for (const key of ['baselineTier', 'auxiliaryTier']) {
|
|
134
205
|
if (!TIERS.includes(config.gateway[key])) throw new Error(`gateway.${key} must be one of ${TIERS.join(', ')}`);
|
|
@@ -140,11 +211,27 @@ function validate(config) {
|
|
|
140
211
|
throw new Error('gateway.idleShutdownMs must be a non-negative number (0 keeps the gateway running)');
|
|
141
212
|
const p = config.policy;
|
|
142
213
|
for (const field of ['upgradeBase', 'jumpConfidence', 'downgradeMass', 'continuationMass']) {
|
|
143
|
-
if (!(p[field]
|
|
214
|
+
if (!(isNumber(p[field], 0) && p[field] <= 1)) throw new Error(`policy.${field} must be between 0 and 1`);
|
|
144
215
|
}
|
|
145
216
|
for (const field of ['upgradeVotes', 'downgradeVotes', 'escalationHoldTurns']) {
|
|
146
|
-
if (!
|
|
217
|
+
if (!isNumber(p[field], 1, { integer: true })) throw new Error(`policy.${field} must be a positive integer`);
|
|
218
|
+
}
|
|
219
|
+
if (!isNumber(p.upgradeSlope, 0)) throw new Error('policy.upgradeSlope must be a non-negative number');
|
|
220
|
+
// The pivot divides: tax / (tax + pivot). Zero makes a free switch NaN and silently stops upgrades.
|
|
221
|
+
if (!isNumber(p.upgradePivotUsd, 0, { above: true })) throw new Error('policy.upgradePivotUsd must be positive');
|
|
222
|
+
if (!isNumber(p.cashCapUsd, 0)) throw new Error('policy.cashCapUsd must be a non-negative number');
|
|
223
|
+
const { cache, jev, context } = config;
|
|
224
|
+
for (const map of ['writeMultiplier', 'ttlMs']) {
|
|
225
|
+
for (const [ttl, value] of Object.entries(cache[map]))
|
|
226
|
+
if (!isNumber(value, 0, { above: true })) throw new Error(`cache.${map}.${ttl} must be positive`);
|
|
227
|
+
}
|
|
228
|
+
if (!isNumber(cache.warmMarginMs, 0)) throw new Error('cache.warmMarginMs must be a non-negative number');
|
|
229
|
+
if (!isNumber(jev.timeoutMs, 0, { above: true })) throw new Error('jev.timeoutMs must be positive');
|
|
230
|
+
for (const field of ['endpoint', 'model']) {
|
|
231
|
+
if (typeof jev[field] !== 'string' || !jev[field]) throw new Error(`jev.${field} is required`);
|
|
232
|
+
}
|
|
233
|
+
for (const field of ['recentTurns', 'maxTextChars']) {
|
|
234
|
+
if (!isNumber(context[field], 1, { integer: true })) throw new Error(`context.${field} must be a positive integer`);
|
|
147
235
|
}
|
|
148
|
-
if (
|
|
149
|
-
if (!(config.jev.timeoutMs > 0)) throw new Error('jev.timeoutMs must be positive');
|
|
236
|
+
if (typeof config.log !== 'boolean') throw new Error('log must be true or false');
|
|
150
237
|
}
|
package/lib/facts.mjs
CHANGED
|
@@ -5,11 +5,20 @@ import { createHash } from 'node:crypto';
|
|
|
5
5
|
const EDIT_TOOLS = new Set(['Edit', 'Write', 'MultiEdit', 'NotebookEdit', 'Bash']);
|
|
6
6
|
const FAILURE_WINDOW = 40;
|
|
7
7
|
const REMINDER = /<system-reminder>[\s\S]*?<\/system-reminder>/g;
|
|
8
|
+
const CLIP_MARKER = ' […] ';
|
|
9
|
+
// A typed /router:<tier> reaches the gateway as a command block at the start of a text block. Anchored, so a pasted
|
|
10
|
+
// file or quoted log that mentions the marker cannot pin a tier.
|
|
11
|
+
const PIN = /^<command-message>router:([a-z]+)<\/command-message>/;
|
|
8
12
|
|
|
9
13
|
export function factsFromRequest(body, memory, { recentTurns, maxTextChars }) {
|
|
10
14
|
const messages = Array.isArray(body.messages) ? body.messages : [];
|
|
11
|
-
|
|
15
|
+
// Claude Code puts hook output and tool additions in `system` messages after the user message, and keeps them in
|
|
16
|
+
// the history: the turn is the last user or assistant message.
|
|
17
|
+
const lastIndex = messages.findLastIndex((m) => m?.role !== 'system');
|
|
18
|
+
const last = messages[lastIndex];
|
|
12
19
|
const lastBlocks = blocks(last?.content);
|
|
20
|
+
// The current user message goes to Jev as the prompt; `turns` holds only the dialogue before it.
|
|
21
|
+
const current = last?.role === 'user' ? lastIndex : -1;
|
|
13
22
|
const turns = [];
|
|
14
23
|
const errors = [];
|
|
15
24
|
const edits = [];
|
|
@@ -22,15 +31,18 @@ export function factsFromRequest(body, memory, { recentTurns, maxTextChars }) {
|
|
|
22
31
|
for (const block of content) if (block.type === 'tool_use' && EDIT_TOOLS.has(block.name)) edits.push(index);
|
|
23
32
|
}
|
|
24
33
|
const text = textOf(content);
|
|
25
|
-
if (text && (message.role === 'user' || message.role === 'assistant'))
|
|
26
|
-
turns.push({ role: message.role, text: text
|
|
34
|
+
if (index !== current && text && (message.role === 'user' || message.role === 'assistant'))
|
|
35
|
+
turns.push({ role: message.role, text: clip(text, maxTextChars) });
|
|
27
36
|
});
|
|
28
37
|
return {
|
|
29
38
|
turns: turns.slice(-recentTurns),
|
|
30
|
-
prompt:
|
|
39
|
+
prompt: current === -1 ? '' : clip(textOf(lastBlocks), maxTextChars),
|
|
31
40
|
continuation: last?.role === 'user' && lastBlocks.some((b) => b.type === 'tool_result'),
|
|
32
|
-
// The
|
|
33
|
-
|
|
41
|
+
// The tier named by a /router:<tier> in the current message; the router checks that it is a tier.
|
|
42
|
+
pin: current === -1 ? null : pinOf(lastBlocks),
|
|
43
|
+
// The same position and the same last user message: Claude Code resent the request (429, 529, a dropped stream, or
|
|
44
|
+
// a 400 it answers by dropping a feature from the system message after it).
|
|
45
|
+
turnKey: last?.role === 'user' ? `${lastIndex + 1}:${hash(JSON.stringify(last.content))}` : null,
|
|
34
46
|
failure: repeatedFailure(errors, edits, messages.length - 1),
|
|
35
47
|
// The effort Claude Code sent: a route without its own effort keeps it, and it is part of the cache key.
|
|
36
48
|
effort: body.output_config?.effort ?? null,
|
|
@@ -53,6 +65,15 @@ function repeatedFailure(errors, edits, lastIndex) {
|
|
|
53
65
|
return null;
|
|
54
66
|
}
|
|
55
67
|
|
|
68
|
+
// At most `max` characters: the head and the tail of a long text, where a request and its question usually are.
|
|
69
|
+
function clip(text, max) {
|
|
70
|
+
if (text.length <= max) return text;
|
|
71
|
+
if (max <= CLIP_MARKER.length) return text.slice(0, max);
|
|
72
|
+
const room = max - CLIP_MARKER.length;
|
|
73
|
+
const head = Math.ceil(room / 2);
|
|
74
|
+
return `${text.slice(0, head)}${CLIP_MARKER}${text.slice(text.length - (room - head))}`;
|
|
75
|
+
}
|
|
76
|
+
|
|
56
77
|
function signatureOf(block) {
|
|
57
78
|
return textOf(blocks(block.content)).toLowerCase().replace(/\d+/g, '#').replace(/\s+/g, ' ').slice(0, 120);
|
|
58
79
|
}
|
|
@@ -73,3 +94,11 @@ function textOf(content) {
|
|
|
73
94
|
.join('\n')
|
|
74
95
|
.trim();
|
|
75
96
|
}
|
|
97
|
+
|
|
98
|
+
function pinOf(content) {
|
|
99
|
+
for (const block of content) {
|
|
100
|
+
const match = block.type === 'text' && typeof block.text === 'string' ? PIN.exec(block.text) : null;
|
|
101
|
+
if (match) return match[1];
|
|
102
|
+
}
|
|
103
|
+
return null;
|
|
104
|
+
}
|