@alexeiled/claude-router 0.6.0 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,117 +1,202 @@
1
1
  # User guide
2
2
 
3
- ## Start a session
3
+ After the [install](../README.md#install), work in Claude Code as usual. The
4
+ router selects the model and the effort for each turn. This guide shows how
5
+ to read what it does, and how to control it.
6
+
7
+ - [Check the setup](#check-the-setup)
8
+ - [A session, step by step](#a-session-step-by-step)
9
+ - [Read the status line](#read-the-status-line)
10
+ - [Read `/router:status`](#read-routerstatus)
11
+ - [Pin a tier](#pin-a-tier)
12
+ - [Read the decision log](#read-the-decision-log)
13
+ - [Update](#update)
14
+ - [Stop using the router](#stop-using-the-router)
15
+ - [Troubleshooting](#troubleshooting)
16
+
17
+ ## Check the setup
18
+
19
+ 1. Start Claude Code. `/model` shows `Jev Router (auto)` as the model.
20
+ 2. Send a prompt. If setup added the status line, it shows the route, for
21
+ example `jev-router ▸ sonnet-5 · high · same-tier`.
22
+ 3. Run `/router:status`. It shows `Jev routing: active.`
23
+
24
+ If a step fails, read [Troubleshooting](#troubleshooting).
25
+
26
+ ## A session, step by step
27
+
28
+ An example of four turns. Each box is one prompt with its route and reason.
29
+
30
+ ```mermaid
31
+ flowchart LR
32
+ T1["<b>1</b> · design a cache layer<br/>Opus · xhigh<br/><i>jump</i>"]
33
+ T2["<b>2</b> · yes, go on<br/>Opus · xhigh<br/><i>continuation</i>"]
34
+ T3["<b>3</b> · rename foo to bar<br/>Opus · xhigh<br/><i>downgrade-pending</i>"]
35
+ T4["<b>4</b> · fix the typo<br/>Sonnet<br/><i>downgrade</i>"]
36
+ T1 --> T2 --> T3 --> T4
37
+
38
+ classDef opus fill:#0d366b,stroke:#0d366b,color:#ffffff
39
+ classDef sonnet fill:#cde2fb,stroke:#2a78d6,color:#0b0b0b
40
+ class T1,T2,T3 opus
41
+ class T4 sonnet
42
+ ```
4
43
 
5
- After the install steps in the README, run `claude` as usual. Claude Code shows
6
- `Jev Router (auto)` as the model. The transcript records the model that answered
7
- each message. The first session starts the gateway, and all sessions share it.
8
- The gateway continues to run after a session ends. After two hours without
9
- requests it exits, unless a turn waits for a tool result, for example a
10
- permission prompt that you have not answered. The next prompt starts it again
11
- (`gateway.idleShutdownMs` in [configuration](configuration.md)).
44
+ 1. A confident vote for hard work goes up two tiers at once.
45
+ 2. A prompt that continues the task keeps the route.
46
+ 3. One vote for a lower tier is not enough. The route stays.
47
+ 4. The second vote in a row moves the route down.
12
48
 
13
- To try the gateway in one session without a change to the configuration:
49
+ The tool calls in a turn keep the route of the turn, so the cache stays warm.
50
+ The [README](../README.md#how-jev-selects-a-tier) gives all the rules.
14
51
 
15
- ```sh
16
- TYPESAFE_API_KEY=… node scripts/gateway.mjs &
17
- ANTHROPIC_BASE_URL=http://127.0.0.1:43170 claude --plugin-dir . --model jev-router
18
- ```
52
+ ## Read the status line
19
53
 
20
- ## What happens at each prompt
21
-
22
- - A new prompt causes one Jev call. The call takes about one second and gives
23
- a tier. Then the policy decides if it acts on the tier.
24
- - Tool calls inside the turn keep the route. The gateway does not ask Jev, and
25
- the cache of the model continues to hit.
26
- - A prompt that continues the task keeps the route. Examples: "continue",
27
- "yes", "now fix the tests".
28
- - After two failed repairs of the same error, the route goes up one tier. The
29
- route stays there for two turns. A repair is an edit between the two errors.
30
- - An upgrade needs two consecutive votes for a higher tier. A jump of two
31
- tiers with high confidence happens at once. The required confidence goes up
32
- with the cost to read the context again on the new model.
33
- - A downgrade needs two consecutive confident votes.
34
- - A cold switch to a model that bills usage credits is refused when the cache
35
- write costs more than `policy.cashCapUsd`. Then the strongest plan tier
36
- serves the turn. Behind the gateway, Claude Code does not show its consent
37
- prompt for these credits, so the cap is the only guard. No default model
38
- bills credits; this applies once you add one in `router.json`.
39
- - When Jev fails or times out, or when there is no key, the baseline tier
40
- serves the turn. After three failures in a row, new turns skip Jev for one
41
- minute and then try it once, so an outage costs one slow turn a minute, not
42
- a second on every turn. `/router:status` shows the pause.
43
- - When Claude Code sends the same request again (after a 429, a 529 or a
44
- dropped connection), the gateway keeps the route of that turn. A retry is
45
- not a new vote and makes no Jev call.
46
- - Jev receives the prompt and the last six turns of text. Tool results are not
47
- sent. No other data leaves the machine, except the usual Anthropic request.
48
-
49
- A Claude Code turn starts at about 100k tokens of system prompt and tool
50
- definitions, so the first switch to a model is the expensive one.
51
-
52
- ## Pin a tier by hand
53
-
54
- Type the tier skill as a command. The turn runs on the model of that skill.
55
- The gateway sends the real model id unchanged.
54
+ ```text
55
+ jev-router ▸ opus-5-5 · xhigh · upgrade
56
+ │ │ └─ reason for the route
57
+ │ └─ effort
58
+ └─ model of the last turn
59
+ ```
56
60
 
61
+ | Line | Meaning |
62
+ | ------------------------------------------------ | --------------------------------------------------- |
63
+ | `jev-router ▸ …` | The route of the last turn in this session. |
64
+ | `jev-router: no turn yet` | This session has no routed turn. |
65
+ | `router: gateway off, the next prompt starts it` | The gateway stopped after idle time, or it crashed. |
66
+
67
+ The line changes after each message.
68
+
69
+ **The route changed:**
70
+
71
+ | Reason | Meaning |
72
+ | ------------- | ---------------------------------------------------------------------------- |
73
+ | `upgrade` | Two votes in a row for a higher tier, with enough confidence. |
74
+ | `jump` | One vote confident enough to go up two tiers. |
75
+ | `downgrade` | Two votes in a row for a lower tier, with enough confidence. |
76
+ | `escalation` | The same error came back after an edit. The route went up one tier. |
77
+ | `context-fit` | The selected model cannot hold the context. A larger window serves the turn. |
78
+ | `cash-gate` | The cold-write guard stopped a switch to a `credits` model. |
79
+
80
+ **The route stayed:**
81
+
82
+ | Reason | Meaning |
83
+ | ------------------- | ------------------------------------------------------------------- |
84
+ | `same-tier` | Jev agreed with the current tier. |
85
+ | `continuation` | The prompt continues the task. |
86
+ | `upgrade-pending` | One vote for a higher tier. The next vote decides. |
87
+ | `downgrade-pending` | One vote for a lower tier. The next vote decides. |
88
+ | `hold` | The route stays up for two turns after an escalation. |
89
+ | `uncertain` | Jev did not select a tier. |
90
+ | `no-advice` | No Jev answer: no key, an error, a pause, or a prompt without text. |
91
+
92
+ `forced` means that the gateway was started with `--force-tier`.
93
+
94
+ ## Read `/router:status`
95
+
96
+ `/router:status` shows the gateway, the routes and the last turn. For a
97
+ vote, it also shows the numbers:
98
+
99
+ ```text
100
+ Router v0.7.0, gateway: http://127.0.0.1:43170, alias `jev-router`.
101
+ Jev routing: active.
102
+ Default tier: low.
103
+ …
104
+ Last turn: low → claude-sonnet-5 at high, reason upgrade-pending, context 84210 tokens, cache reads 83904.
105
+ Why: Jev asked for a higher tier; the route stays until the votes and the mass are enough.
106
+ Estimate: upgrade mass 0.78 against a bar of 0.84, switching tax $0.66 at list prices, 1 vote(s) in a row, cache: candidate unknown, current warm.
57
107
  ```
108
+
109
+ - **Upgrade mass** is the Jev confidence in a higher tier. It must reach the
110
+ bar.
111
+ - **The bar** grows with the switching tax: the cost to write the context
112
+ into a cold cache. Here Opus has no warm cache.
113
+ - **Cache** is `warm`, `expired` or `unknown`. `unknown` means no response
114
+ for that cache since the session started or the conversation got shorter.
115
+ - **Dollars** are list prices. On a subscription plan, they show the relative
116
+ cost only.
117
+
118
+ ## Pin a tier
119
+
120
+ To run one turn on a fixed tier, type the tier skill before the prompt:
121
+
122
+ ```text
58
123
  /router:high redesign the auth flow
59
124
  /router:micro rename foo to bar in this file
60
125
  ```
61
126
 
62
- `/model <name>` also works. It stops the routing for the rest of the session.
63
-
64
- ## See the current route
127
+ The status line shows the reason `pinned`. Tool calls in that turn stay on the
128
+ tier; the next prompt goes back to Jev.
65
129
 
66
- `/router:status` shows the gateway, the routes, and the model, effort and
67
- reason of the last turn in this session.
130
+ `/model <name>` also works, but it stops the routing for the rest of the
131
+ session.
68
132
 
69
- The status line wrapper from `/router:setup` adds one line to your status line
70
- while the session uses the router:
133
+ ## Read the decision log
71
134
 
72
- - `jev-router ▸ opus-5-5 · xhigh`: the model and the effort of the last turn.
73
- - `router: no turn yet`: the session has no routed turn.
74
- - `router: gateway off, the next prompt starts it`: the gateway stopped after
75
- idle time, or it crashed. Either way the next prompt starts it.
135
+ The gateway writes one JSON line for each routed request to
136
+ `~/.claude/plugins/data/router-alexei-led-claude-router/decisions.jsonl`. The
137
+ log has no prompt text.
76
138
 
77
- The line updates when Claude Code redraws the status line, after each message.
139
+ | Field | Content |
140
+ | -------------- | ---------------------------------------------------------------------------------- |
141
+ | `tier` | The tier of the request. |
142
+ | `model`, `effort` | The model id and the effort that the gateway sent. `effort` is `null` for a model without effort. |
143
+ | `reason` | The rule that decided. Tool calls show `tool-continuation`. `error`: routing failed and the default tier served. |
144
+ | `advice` | The Jev probabilities for each tier, and for "continues the task". |
145
+ | `estimate` | The numbers of a vote: confidence, bar, switching tax, cache state. |
146
+ | `shadow` | The cost of the Jev choice against the current route. The policy does not use it. |
147
+ | `observed` | The usage of the response: model, effort, context tokens, cache reads, output. |
148
+ | `failed` | Anthropic answered a routed turn with an error: status, tier, model, effort. |
149
+ | `historyBreak` | A compaction or a rewind. The votes and the cache estimates reset. |
150
+ | `cacheReset` | The context shrank by more than 20%. The cache estimates reset. |
78
151
 
79
- ## Read the decisions
80
-
81
- The gateway writes one line for each routed request to `decisions.jsonl` in
82
- the plugin data directory. The directory is
83
- `~/.claude/plugins/data/router-<marketplace>/`. For a gateway that you started
84
- by hand, the directory is `$TMPDIR/router/`.
152
+ Two useful queries:
85
153
 
86
154
  ```sh
87
- tail -n 20 ~/.claude/plugins/data/router-*/decisions.jsonl | jq -c '{tier, reason, estimate, observed}'
155
+ LOG=~/.claude/plugins/data/router-alexei-led-claude-router/decisions.jsonl
156
+ jq -r 'select(.observed) | .observed.model' $LOG | sort | uniq -c # requests by model
157
+ jq -r 'select(.reason) | .reason' $LOG | sort | uniq -c | sort -rn # decisions by reason
158
+ jq -c 'select(.failed) | .failed' $LOG # failed turns
88
159
  ```
89
160
 
90
- `tier` is the selected tier. `reason` is the rule that decided. The `observed`
91
- lines carry the model that answered, the context tokens and the cache reads
92
- from the response. Compare the estimated and the observed cache reads to tune
93
- the thresholds.
94
-
95
- `scripts/transcript-models.sh <transcript.jsonl>` shows the model for each
161
+ `scripts/transcript-models.sh <transcript.jsonl>` shows the model of each
96
162
  assistant message in a Claude Code transcript.
97
163
 
164
+ ## Update
165
+
166
+ 1. Update the marketplace and the plugin:
167
+
168
+ ```sh
169
+ claude plugin marketplace update alexei-led-claude-router
170
+ claude plugin update router@alexei-led-claude-router
171
+ ```
172
+
173
+ 2. Restart Claude Code. The next prompt replaces the running gateway.
174
+ 3. Run `/router:setup` once. The status line command contains the plugin
175
+ version.
176
+
177
+ ## Stop using the router
178
+
179
+ 1. In `~/.claude/settings.json`, remove `model`, `env.ANTHROPIC_BASE_URL`,
180
+ `env.ENABLE_TOOL_SEARCH`, `env.CLAUDE_CODE_GATEWAY_HINT_HEADERS` and the
181
+ `jev-router[1m]` row of `modelPicker.options`.
182
+ 2. If setup changed your status line, restore the old command.
183
+ 3. Restart Claude Code.
184
+ 4. To remove the plugin, run
185
+ `claude plugin uninstall router@alexei-led-claude-router`.
186
+
187
+ The gateway stops by itself after two hours without requests.
188
+
98
189
  ## Troubleshooting
99
190
 
100
- - If Claude Code does not accept `jev-router` as a model, make sure that the
101
- gateway runs and that `ANTHROPIC_BASE_URL` is set. The command
102
- `curl http://127.0.0.1:43170/v1/models` lists the alias.
103
- - If Claude Code reports that it does not use the gateway, run
104
- `/router:setup` and restart Claude Code.
105
- - If each turn runs on the default tier, make sure that the key is set, and
106
- read `/router:status`: it shows when Jev is paused after failures. The
107
- timestamped `router:` lines in `gateway.log` next to `decisions.jsonl` give
108
- the reason, for example `jev http 401` for a rejected key.
109
- - If the effort is not what you set, read the `efforts` list of the model. The
110
- gateway lowers the effort to a level that the model accepts. Haiku has no
111
- effort and no thinking.
112
- - To stop the gateway, run `pkill -f scripts/gateway.mjs`. It releases the port
113
- at once and finishes the responses it is streaming. The next prompt starts it
114
- again.
115
- - Rate limits and overload errors from Anthropic (429, 529) reach Claude Code
116
- unchanged. Claude Code waits and retries; the gateway does not add a second
117
- layer of retries.
191
+ | Symptom | Cause | Fix |
192
+ | ----------------------------------------------------------------- | --------------------------------------------- | ------------------------------------------------------------------------------------ |
193
+ | "There's an issue with the selected model (jev-router[1m])" | The session started before the restart. | Restart Claude Code. |
194
+ | Claude Code rejects `jev-router` | The session does not use the gateway. | Run `/router:setup`, then restart Claude Code. |
195
+ | The status line shows `gateway off` after the next prompt | The gateway cannot start, for example because `router.json` has an unknown key. | Run `/router:status` to see the error. Read `gateway.log` next to `decisions.jsonl`. |
196
+ | Every turn has the reason `no-advice` | No Jev key, or Jev fails. | Run `/router:status`. Read the `router:` lines in `gateway.log`. |
197
+ | The effort is not the effort that you set | The model does not accept that effort. | The gateway uses the nearest lower level. Haiku has no effort and no thinking. |
198
+ | 429 or 529 errors | Anthropic rate limits or overload. | Claude Code waits and tries again. The gateway sends these errors through unchanged. |
199
+ | A `router.json` change has no effect | The gateway reads the configuration at start. | Run `pkill -f scripts/gateway.mjs`. The next prompt starts a new gateway. |
200
+
201
+ To see if the gateway runs, open `http://127.0.0.1:43170/v1/models`. The list
202
+ shows the alias.
package/lib/config.mjs CHANGED
@@ -2,6 +2,9 @@
2
2
 
3
3
  export const TIERS = ['micro', 'low', 'medium', 'high'];
4
4
  export const EFFORTS = ['low', 'medium', 'high', 'xhigh', 'max'];
5
+ // Request features that Claude Code uses with its own models and that only some models accept (a beta each):
6
+ // `system` messages after the prompt, per-turn control, and tool additions or removals in the history.
7
+ export const FEATURES = ['mid-conversation-system', 'per-turn-control', 'mid-conversation-tool-changes'];
5
8
 
6
9
  // The alias before 0.4.2. Settings written by an older /router:setup still send it; drop it once they are rare.
7
10
  export const LEGACY_ALIAS = 'router';
@@ -20,34 +23,45 @@ export const DEFAULTS = {
20
23
  micro: { model: 'haiku' },
21
24
  },
22
25
  // `id` is sent upstream verbatim. List prices in USD per million tokens; `cacheRead` is absolute,
23
- // not a multiplier (Opus 5.5 reads at 0.05x input, the rest at the standard 0.1x — verify when prices move).
24
- // `efforts` lists what the model accepts; an empty list means no effort field and no adaptive thinking.
26
+ // not a multiplier (Opus 5.5 reads at 0.05x input, the rest at the standard 0.1x). A price change must also change
27
+ // test/fixtures/list-prices.json, with its source and date. `output` feeds only the shadow estimate in the log.
28
+ // `efforts` lists what the model accepts; an empty list means no effort field and no adaptive thinking. A change
29
+ // must also change test/fixtures/effort-support.json, from a new probe against the real API.
30
+ // `features` lists the FEATURES the model accepts; the gateway removes the others from the request. A model
31
+ // without the field gets none: a removed feature is lost, a feature the model rejects fails the turn. A change
32
+ // must also change test/fixtures/model-features.json, from a new probe.
25
33
  models: {
26
34
  opus: {
27
35
  id: 'claude-opus-5-5',
28
36
  input: 4,
37
+ output: 20,
29
38
  cacheRead: 0.2,
30
39
  contextWindow: 1_000_000,
31
40
  billing: 'plan',
32
41
  efforts: EFFORTS,
42
+ features: FEATURES,
33
43
  },
34
44
  sonnet: {
35
45
  id: 'claude-sonnet-5',
36
46
  input: 2,
47
+ output: 10,
37
48
  cacheRead: 0.2,
38
49
  contextWindow: 1_000_000,
39
50
  billing: 'plan',
40
51
  efforts: EFFORTS,
52
+ features: ['mid-conversation-system'],
41
53
  },
42
54
  // `maxOutput` caps `max_tokens`: Claude Code asks for 128K, Haiku 4.5 answers more than 64K with 400.
43
55
  haiku: {
44
56
  id: 'claude-haiku-4-5',
45
57
  input: 1,
58
+ output: 5,
46
59
  cacheRead: 0.1,
47
60
  contextWindow: 200_000,
48
61
  maxOutput: 64_000,
49
62
  billing: 'plan',
50
63
  efforts: [],
64
+ features: [],
51
65
  },
52
66
  },
53
67
  cache: {
@@ -76,13 +90,31 @@ export const DEFAULTS = {
76
90
 
77
91
  const API_KEY_VAR = 'TYPESAFE_API_KEY';
78
92
 
79
- export function loadConfig({ env = {}, userFile = null } = {}) {
93
+ // Keys a route or a model entry may carry. The other closed sections take their key sets from DEFAULTS.
94
+ const ROUTE_KEYS = ['model', 'effort'];
95
+ const MODEL_KEYS = [
96
+ 'id',
97
+ 'input',
98
+ 'output',
99
+ 'cacheRead',
100
+ 'contextWindow',
101
+ 'maxOutput',
102
+ 'billing',
103
+ 'efforts',
104
+ 'features',
105
+ ];
106
+ // A JSON file can hold these as own keys; merged into a plain object they reach the prototype chain.
107
+ const FORBIDDEN_KEYS = new Set(['__proto__', 'constructor', 'prototype']);
108
+
109
+ // `forcedTier` skips Jev and the policy. It comes from the daemon's --force-tier flag, never from env: a project's
110
+ // settings can set a hook's env, and a forced tier changes spend.
111
+ export function loadConfig({ env = {}, userFile = null, forcedTier = null } = {}) {
112
+ if (userFile !== null) checkShape(userFile);
80
113
  const config = merge(DEFAULTS, userFile ?? {});
81
114
  validate(config);
82
115
  const apiKey = env[API_KEY_VAR]?.trim() || null;
83
- const forcedTier = env.ROUTER_FORCE_TIER ?? null;
84
- if (forcedTier && !TIERS.includes(forcedTier))
85
- throw new Error(`ROUTER_FORCE_TIER must be one of ${TIERS.join(', ')}`);
116
+ if (forcedTier !== null && !TIERS.includes(forcedTier))
117
+ throw new Error(`the forced tier must be one of ${TIERS.join(', ')}`);
86
118
  return { ...config, apiKey, forcedTier };
87
119
  }
88
120
 
@@ -95,27 +127,65 @@ function merge(base, override) {
95
127
  const out = { ...base };
96
128
  for (const [key, value] of Object.entries(override)) {
97
129
  out[key] =
98
- key in base && typeof value === 'object' && value !== null && !Array.isArray(value)
130
+ Object.hasOwn(base, key) && typeof value === 'object' && value !== null && !Array.isArray(value)
99
131
  ? merge(base[key], value)
100
132
  : value;
101
133
  }
102
134
  return out;
103
135
  }
104
136
 
137
+ // The raw file, before the merge: only known keys, and every section an object. Messages name the path, never the
138
+ // value. An open map (model aliases, cache TTL labels) takes any key but the forbidden ones.
139
+ function checkShape(file) {
140
+ checkObject(file, 'router.json', Object.keys(DEFAULTS));
141
+ const { gateway, routes, models, cache, policy, jev, context } = file;
142
+ if (gateway !== undefined) checkObject(gateway, 'gateway', Object.keys(DEFAULTS.gateway));
143
+ if (routes !== undefined) {
144
+ checkObject(routes, 'routes', TIERS);
145
+ for (const [tier, route] of Object.entries(routes)) checkObject(route, `routes.${tier}`, ROUTE_KEYS);
146
+ }
147
+ if (models !== undefined) {
148
+ checkObject(models, 'models');
149
+ for (const [alias, model] of Object.entries(models)) checkObject(model, `models.${alias}`, MODEL_KEYS);
150
+ }
151
+ if (cache !== undefined) {
152
+ checkObject(cache, 'cache', Object.keys(DEFAULTS.cache));
153
+ for (const map of ['writeMultiplier', 'ttlMs'])
154
+ if (cache[map] !== undefined) checkObject(cache[map], `cache.${map}`);
155
+ }
156
+ if (policy !== undefined) checkObject(policy, 'policy', Object.keys(DEFAULTS.policy));
157
+ if (jev !== undefined) checkObject(jev, 'jev', Object.keys(DEFAULTS.jev));
158
+ if (context !== undefined) checkObject(context, 'context', Object.keys(DEFAULTS.context));
159
+ }
160
+
161
+ // `allowed` null: an open map.
162
+ function checkObject(value, path, allowed = null) {
163
+ if (typeof value !== 'object' || value === null || Array.isArray(value)) throw new Error(`${path} must be an object`);
164
+ for (const key of Object.keys(value)) {
165
+ const at = path === 'router.json' ? key : `${path}.${key}`;
166
+ if (FORBIDDEN_KEYS.has(key)) throw new Error(`${at} is not allowed`);
167
+ if (allowed && !allowed.includes(key)) throw new Error(`${at} is not a known key`);
168
+ }
169
+ }
170
+
171
+ const isNumber = (value, min, { above = false, integer = false } = {}) =>
172
+ Number.isFinite(value) && (above ? value > min : value >= min) && (!integer || Number.isInteger(value));
173
+
105
174
  function validate(config) {
106
175
  for (const tier of TIERS) {
107
176
  const route = config.routes[tier];
108
177
  if (!route || typeof route.model !== 'string') throw new Error(`routes.${tier}.model is required`);
109
- const model = config.models[route.model];
110
- if (!model) throw new Error(`routes.${tier}.model "${route.model}" is not in models`);
178
+ if (!Object.hasOwn(config.models, route.model)) throw new Error(`routes.${tier}.model is not in models`);
111
179
  if (route.effort !== undefined && !EFFORTS.includes(route.effort))
112
- throw new Error(`routes.${tier}.effort "${route.effort}" is invalid`);
180
+ throw new Error(`routes.${tier}.effort must be one of ${EFFORTS.join(', ')}`);
113
181
  }
114
182
  for (const [alias, model] of Object.entries(config.models)) {
115
183
  for (const field of ['input', 'cacheRead', 'contextWindow']) {
116
184
  if (!Number.isFinite(model[field]) || model[field] < 0)
117
185
  throw new Error(`models.${alias}.${field} must be a non-negative number`);
118
186
  }
187
+ if (model.output !== undefined && !(Number.isFinite(model.output) && model.output >= 0))
188
+ throw new Error(`models.${alias}.output must be a non-negative number`);
119
189
  if (typeof model.id !== 'string' || !model.id) throw new Error(`models.${alias}.id is required`);
120
190
  if (model.maxOutput !== undefined && !(Number.isInteger(model.maxOutput) && model.maxOutput > 0))
121
191
  throw new Error(`models.${alias}.maxOutput must be a positive integer`);
@@ -123,6 +193,11 @@ function validate(config) {
123
193
  throw new Error(`models.${alias}.billing must be plan or credits`);
124
194
  if (!Array.isArray(model.efforts) || model.efforts.some((e) => !EFFORTS.includes(e)))
125
195
  throw new Error(`models.${alias}.efforts must list valid effort levels`);
196
+ if (
197
+ model.features !== undefined &&
198
+ !(Array.isArray(model.features) && model.features.every((f) => FEATURES.includes(f)))
199
+ )
200
+ throw new Error(`models.${alias}.features must list known features (${FEATURES.join(', ')})`);
126
201
  }
127
202
  for (const key of ['baselineTier', 'auxiliaryTier']) {
128
203
  if (!TIERS.includes(config.gateway[key])) throw new Error(`gateway.${key} must be one of ${TIERS.join(', ')}`);
@@ -134,11 +209,27 @@ function validate(config) {
134
209
  throw new Error('gateway.idleShutdownMs must be a non-negative number (0 keeps the gateway running)');
135
210
  const p = config.policy;
136
211
  for (const field of ['upgradeBase', 'jumpConfidence', 'downgradeMass', 'continuationMass']) {
137
- if (!(p[field] >= 0 && p[field] <= 1)) throw new Error(`policy.${field} must be between 0 and 1`);
212
+ if (!(isNumber(p[field], 0) && p[field] <= 1)) throw new Error(`policy.${field} must be between 0 and 1`);
138
213
  }
139
214
  for (const field of ['upgradeVotes', 'downgradeVotes', 'escalationHoldTurns']) {
140
- if (!Number.isInteger(p[field]) || p[field] < 1) throw new Error(`policy.${field} must be a positive integer`);
215
+ if (!isNumber(p[field], 1, { integer: true })) throw new Error(`policy.${field} must be a positive integer`);
216
+ }
217
+ if (!isNumber(p.upgradeSlope, 0)) throw new Error('policy.upgradeSlope must be a non-negative number');
218
+ // The pivot divides: tax / (tax + pivot). Zero makes a free switch NaN and silently stops upgrades.
219
+ if (!isNumber(p.upgradePivotUsd, 0, { above: true })) throw new Error('policy.upgradePivotUsd must be positive');
220
+ if (!isNumber(p.cashCapUsd, 0)) throw new Error('policy.cashCapUsd must be a non-negative number');
221
+ const { cache, jev, context } = config;
222
+ for (const map of ['writeMultiplier', 'ttlMs']) {
223
+ for (const [ttl, value] of Object.entries(cache[map]))
224
+ if (!isNumber(value, 0, { above: true })) throw new Error(`cache.${map}.${ttl} must be positive`);
225
+ }
226
+ if (!isNumber(cache.warmMarginMs, 0)) throw new Error('cache.warmMarginMs must be a non-negative number');
227
+ if (!isNumber(jev.timeoutMs, 0, { above: true })) throw new Error('jev.timeoutMs must be positive');
228
+ for (const field of ['endpoint', 'model']) {
229
+ if (typeof jev[field] !== 'string' || !jev[field]) throw new Error(`jev.${field} is required`);
230
+ }
231
+ for (const field of ['recentTurns', 'maxTextChars']) {
232
+ if (!isNumber(context[field], 1, { integer: true })) throw new Error(`context.${field} must be a positive integer`);
141
233
  }
142
- if (!(p.cashCapUsd >= 0)) throw new Error('policy.cashCapUsd must be a non-negative number');
143
- if (!(config.jev.timeoutMs > 0)) throw new Error('jev.timeoutMs must be positive');
234
+ if (typeof config.log !== 'boolean') throw new Error('log must be true or false');
144
235
  }
package/lib/cost.mjs CHANGED
@@ -1,22 +1,46 @@
1
- // Cache-aware input cost of the next request. Pure arithmetic over configured prices and the gateway's memory.
1
+ // Cache-aware cost of the next request. Pure arithmetic over configured prices and the gateway's memory.
2
+ import { clampEffort } from './rewrite.mjs';
3
+
4
+ // The prompt cache belongs to a model, and its messages part to the effort too: a top-level effort change rewrites
5
+ // the messages cache. Opus at `high` and Opus at `xhigh` are two caches, not one.
6
+ export function cacheKey(modelId, effort) {
7
+ return effort ? `${modelId}@${effort}` : modelId;
8
+ }
9
+
10
+ // The cache a request routed to `tier` uses. `sentEffort` is what Claude Code sent, for a route that keeps it.
11
+ export function routeCacheKey(config, tier, sentEffort) {
12
+ const model = modelOf(config, tier);
13
+ return cacheKey(model.id, clampEffort(config.routes[tier].effort ?? sentEffort, model.efforts));
14
+ }
2
15
 
3
16
  export function isWarm(modelState, now, cache) {
4
17
  if (!modelState) return false;
5
18
  return now < modelState.lastAt + cache.ttlMs[modelState.ttl] - cache.warmMarginMs;
6
19
  }
7
20
 
21
+ // For the logs: `unknown` means no response for this cache since the session started or its history broke. The cost
22
+ // arithmetic treats `unknown` and `expired` alike, as a full write.
23
+ export function cacheState(modelState, now, cache) {
24
+ if (!modelState) return 'unknown';
25
+ return isWarm(modelState, now, cache) ? 'warm' : 'expired';
26
+ }
27
+
28
+ function modelOf(config, tier) {
29
+ return config.models[config.routes[tier].model];
30
+ }
31
+
8
32
  function ttlFor(config, alias, facts) {
9
33
  if (config.models[alias].billing === 'credits') return '5m';
10
34
  return facts.lastRequest?.ttl ?? '5m';
11
35
  }
12
36
 
13
- // USD for the input side of one request on `alias` with `tokens` of context.
14
- export function inputCostUsd(config, alias, tokens, facts, now) {
37
+ // USD for the input side of one request routed to `tier` with `tokens` of context.
38
+ export function inputCostUsd(config, tier, tokens, facts, now) {
39
+ const alias = config.routes[tier].model;
15
40
  const model = config.models[alias];
16
- const state = facts.models[model.id];
41
+ const state = facts.models[routeCacheKey(config, tier, facts.effort)];
17
42
  const reusable = isWarm(state, now, config.cache) ? Math.min(state.prefixTokens, tokens) : 0;
18
- const ttl = ttlFor(config, alias, facts);
19
- const write = model.input * config.cache.writeMultiplier[ttl];
43
+ const write = model.input * config.cache.writeMultiplier[ttlFor(config, alias, facts)];
20
44
  return (model.cacheRead * reusable + write * (tokens - reusable)) / 1e6;
21
45
  }
22
46
 
@@ -31,9 +55,30 @@ export function nextContextTokens(facts) {
31
55
  return last ? last.tokens + last.outputTokens : 0;
32
56
  }
33
57
 
34
- export function switchingTaxUsd(config, candidateAlias, incumbentAlias, facts, now) {
58
+ export function switchingTaxUsd(config, candidateTier, incumbentTier, facts, now) {
35
59
  const tokens = nextContextTokens(facts);
36
60
  return (
37
- inputCostUsd(config, candidateAlias, tokens, facts, now) - inputCostUsd(config, incumbentAlias, tokens, facts, now)
61
+ inputCostUsd(config, candidateTier, tokens, facts, now) - inputCostUsd(config, incumbentTier, tokens, facts, now)
38
62
  );
39
63
  }
64
+
65
+ // Shadow estimate for the decision log; the policy does not read it. At list prices, so for `plan` models it is a
66
+ // list-price equivalent, not a charge. Output uses the last observed output size for both routes, although effort
67
+ // changes how much a model writes.
68
+ // - nextTurnUsd: candidate minus incumbent for the next request, input at the current cache state plus output.
69
+ // - laterTurnUsd: the same difference for each later turn, once both caches are warm.
70
+ // - paybackTurns: 0 when the switch is cheaper at once, n when later turns repay it after n turns, null when never.
71
+ export function shadowEconomics(config, candidateTier, incumbentTier, facts, now) {
72
+ const candidate = modelOf(config, candidateTier);
73
+ const incumbent = modelOf(config, incumbentTier);
74
+ if (candidate.output === undefined || incumbent.output === undefined) return null;
75
+ const tokens = nextContextTokens(facts);
76
+ const outputTokens = facts.lastRequest?.outputTokens ?? 0;
77
+ const outputUsd = ((candidate.output - incumbent.output) * outputTokens) / 1e6;
78
+ const nextTurnUsd = switchingTaxUsd(config, candidateTier, incumbentTier, facts, now) + outputUsd;
79
+ const laterTurnUsd = ((candidate.cacheRead - incumbent.cacheRead) * tokens) / 1e6 + outputUsd;
80
+ let paybackTurns = null;
81
+ if (nextTurnUsd <= 0 && laterTurnUsd <= 0) paybackTurns = 0;
82
+ else if (nextTurnUsd > 0 && laterTurnUsd < 0) paybackTurns = Math.ceil(nextTurnUsd / -laterTurnUsd);
83
+ return { nextTurnUsd, laterTurnUsd, paybackTurns, outputTokens };
84
+ }