@alexeiled/claude-router 0.6.0 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,435 @@
1
+ # Architecture
2
+
3
+ claude-router selects a model and an effort level for each Claude Code turn.
4
+ A local gateway changes each request before it goes to Anthropic. TypeSafe Jev
5
+ advises a tier, and a local policy makes the decision.
6
+
7
+ - [System context](#system-context)
8
+ - [Tiers](#tiers)
9
+ - [Request flow](#request-flow)
10
+ - [Request classes](#request-classes)
11
+ - [Switching policy](#switching-policy)
12
+ - [Cache and cost model](#cache-and-cost-model)
13
+ - [Modules](#modules)
14
+ - [Gateway lifecycle](#gateway-lifecycle)
15
+ - [Failure handling](#failure-handling)
16
+ - [Security and privacy](#security-and-privacy)
17
+ - [Observability](#observability)
18
+ - [Release](#release)
19
+ - [Known limits](#known-limits)
20
+
21
+ ## System context
22
+
23
+ ```mermaid
24
+ flowchart TB
25
+ subgraph machine["Your machine"]
26
+ direction LR
27
+ CC["Claude Code<br/>model: jev-router"]
28
+ HOOK["Plugin hooks"]
29
+ UI["Status line<br/>/router:status"]
30
+ GW["Gateway<br/>127.0.0.1:43170"]
31
+ DATA[("Data directory<br/>sessions · decisions.jsonl")]
32
+ end
33
+ subgraph remote["Remote services"]
34
+ direction LR
35
+ API["api.anthropic.com"]
36
+ JEV["TypeSafe Jev"]
37
+ end
38
+
39
+ CC <-->|"Messages API"| GW
40
+ HOOK -.->|"start or replace"| GW
41
+ UI -.->|"GET /router/status"| GW
42
+ GW -->|"prompt and<br/>recent turns"| JEV
43
+ GW <-->|"changed request,<br/>unchanged response"| API
44
+ GW --> DATA
45
+
46
+ classDef ext fill:#eef2ff,stroke:#6366f1,color:#1e1b4b
47
+ classDef core fill:#ecfdf5,stroke:#059669,color:#064e3b
48
+ classDef store fill:#fff7ed,stroke:#ea580c,color:#431407
49
+ class JEV,API ext
50
+ class GW core
51
+ class DATA store
52
+ style machine fill:#f8fafc,stroke:#94a3b8,color:#0f172a
53
+ style remote fill:#f8fafc,stroke:#94a3b8,color:#0f172a
54
+ ```
55
+
56
+ - `/router:setup` points `ANTHROPIC_BASE_URL` at the gateway and sets the
57
+ model to `jev-router[1m]`. One gateway serves all sessions on the machine.
58
+ - The gateway changes only requests for the alias. It sets `model`,
59
+ `output_config.effort` and `thinking`. For a small model, it also limits
60
+ `max_tokens` and removes the 1M context beta header and the thinking edits.
61
+ - Claude Code builds each request for the model that the alias behaves as
62
+ (Opus). A model without one of its [request features](configuration.md#models)
63
+ gets the request without it, the way Claude Code retries after a 400:
64
+ - no `mid-conversation-tool-changes` (Sonnet, Haiku): `tool_addition` and
65
+ `tool_removal` blocks go; their cache breakpoint moves to the block before;
66
+ - no `per-turn-control` (Sonnet, Haiku): the beta header and the
67
+ `output_config` on messages (the effort of each turn) go;
68
+ - no `mid-conversation-system` (Haiku): each `system` message becomes a
69
+ `<system-reminder>` text in the user message next to it.
70
+ The same history adapts the same way on every turn, so the cached prefix
71
+ holds.
72
+ - On a model with `per-turn-control`, the effort of each turn overrides the
73
+ request's effort. A route with its own effort (`medium`, `high`) therefore
74
+ sets it in every message's `output_config` too.
75
+ - The gateway never changes `system` or `tools`, and never changes `messages`
76
+ for a model that takes all the features. Prompt caching and a claude.ai
77
+ login work as with a direct connection.
78
+ - Responses go back unchanged. The gateway reads `usage` from them.
79
+
80
+ **Why a gateway.** Only a gateway can route every turn. The tier skills are
81
+ manual pins. Claude Code 2.1.x switches the model from their `model:`
82
+ frontmatter for `sonnet` and `opus`, and that turn passes through. For `haiku`
83
+ it sends the turn to the alias; the gateway reads the `/router:micro` command
84
+ in the prompt and serves the turn on `micro`, reason `pinned`. After a pin, the
85
+ next prompt is decided from the route before the pin.
86
+
87
+ ## Tiers
88
+
89
+ Four tiers, from `micro` (Haiku) to `high` (Opus at `xhigh`), map to a model
90
+ and an effort. The [README](../README.md#how-jev-selects-a-tier) shows the
91
+ work that Jev selects each tier for. The [routes](configuration.md#routes)
92
+ set the mapping.
93
+
94
+ - `low` is the baseline.
95
+ - `medium` and `high` use the same model at two efforts.
96
+ - The gateway lowers an effort to a level that the model accepts. Haiku
97
+ accepts no effort, so it runs without thinking.
98
+
99
+ ## Request flow
100
+
101
+ A new user turn:
102
+
103
+ ```mermaid
104
+ sequenceDiagram
105
+ autonumber
106
+ participant CC as Claude Code
107
+ participant GW as Gateway
108
+ participant R as Router
109
+ participant J as Jev
110
+ participant A as Anthropic
111
+
112
+ CC->>GW: POST /v1/messages (jev-router)
113
+ GW->>R: route(body, session, hints)
114
+ Note over R: facts from body and session memory
115
+ R->>J: prompt and last 6 turns
116
+ J-->>R: tier probabilities, continuation
117
+ Note over R: switching policy selects the tier
118
+ R-->>GW: body with model, effort, thinking
119
+ GW->>A: request
120
+ A-->>GW: response stream
121
+ GW-->>CC: same bytes
122
+ GW->>R: usage: tokens, cache reads, TTL
123
+ Note over R: session memory, decisions.jsonl
124
+ ```
125
+
126
+ A tool continuation and a resent request skip the Jev call. They keep the
127
+ route of their turn.
128
+
129
+ ## Request classes
130
+
131
+ ```mermaid
132
+ flowchart TD
133
+ IN(["Request"]) --> ALIAS{"model is<br/>jev-router?"}
134
+ ALIAS -- no --> PASS["Pass through unchanged"]
135
+ ALIAS -- yes --> MSG{"POST<br/>/v1/messages?"}
136
+ MSG -- no --> LAST["Last route of the session"]
137
+ MSG -- yes --> SIDE{"Side request?"}
138
+ SIDE -- yes --> AUX["auxiliaryTier<br/>memory unchanged"]
139
+ SIDE -- no --> BRK{"History break?"}
140
+ BRK -- yes --> RESET["Reset cache estimates,<br/>votes and hold"]
141
+ BRK -- no --> TOOL
142
+ RESET --> TOOL{"tool_result or<br/>resent request?"}
143
+ TOOL -- yes --> KEEP["Route of the turn"]
144
+ TOOL -- no --> POL["Jev and switching policy"]
145
+ KEEP --> FIT["Fit the context window"]
146
+ POL --> FIT
147
+ FIT --> OUT(["Change and forward"])
148
+
149
+ classDef stop fill:#f1f5f9,stroke:#64748b,color:#0f172a
150
+ classDef act fill:#ecfdf5,stroke:#059669,color:#064e3b
151
+ class PASS,LAST,AUX stop
152
+ class RESET,KEEP,POL,FIT act
153
+ ```
154
+
155
+ | Check | How the gateway decides |
156
+ | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
157
+ | Pass through | Any other model: `/model`, subagents with their own model. |
158
+ | Pin | The current message starts with the `/router:<tier>` command block. That tier serves the turn and its tool calls, reason `pinned`. No Jev call. |
159
+ | Not a turn | `count_tokens` and other endpoints get the last route of the session. |
160
+ | Side request | Header `x-claude-code-request-class` is `auxiliary` or `compaction`. Without the header: thinking disabled or an output format in the body. |
161
+ | History break | Fewer messages than the last main request (a compaction or a rewind), or the header `x-claude-code-context-compacted`. |
162
+ | Resent request | Same last user message at the same position. Claude Code resends after a 429, a 529, a dropped stream, or a 400 that it answers by dropping a feature. No Jev call, no vote. |
163
+ | System messages | Claude Code sends hook output and tool additions as `system` messages after the prompt. The turn is the last user or assistant message; Jev never receives `system` text. |
164
+ | Context fit | The route moves to a model whose window holds the next context at 80% fill. Reason `context-fit`. |
165
+ | Session memory | Key `x-claude-code-session-id`. A subagent adds its agent id: `<session>.<agent id>`. Its turns do not change the route of the main conversation. |
166
+
167
+ ## Switching policy
168
+
169
+ ```mermaid
170
+ flowchart TD
171
+ S(["New user turn"]) --> ERR{"Same error twice,<br/>edit between?"}
172
+ ERR -- yes --> ESC["escalation<br/>one tier up for 2 turns"]
173
+ ERR -- no --> HOLD{"Hold active?"}
174
+ HOLD -- yes --> R1["hold"]
175
+ HOLD -- no --> ADV{"Jev answered?"}
176
+ ADV -- no --> R2["no-advice"]
177
+ ADV -- yes --> CONT{"Continues<br/>the task?"}
178
+ CONT -- yes --> R3["continuation"]
179
+ CONT -- no --> DIR{"Jev choice vs<br/>current tier"}
180
+ DIR -- "same or uncertain" --> R4["same-tier<br/>uncertain"]
181
+ DIR -- higher --> GUARD{"Cold-write<br/>guard blocks?"}
182
+ GUARD -- yes --> CG["cash-gate<br/>strongest plan tier"]
183
+ GUARD -- no --> JMP{"2 tiers up,<br/>U ≥ 0.95?"}
184
+ JMP -- yes --> JUMP["jump"]
185
+ JMP -- no --> UPQ{"2 votes,<br/>U ≥ bar?"}
186
+ UPQ -- yes --> UP["upgrade"]
187
+ UPQ -- no --> R5["upgrade-pending"]
188
+ DIR -- lower --> DNQ{"2 votes,<br/>D ≥ 0.90?"}
189
+ DNQ -- yes --> DOWN["downgrade"]
190
+ DNQ -- no --> R6["downgrade-pending"]
191
+
192
+ classDef stay fill:#f1f5f9,stroke:#64748b,color:#0f172a
193
+ classDef up fill:#fef3c7,stroke:#d97706,color:#451a03
194
+ classDef down fill:#e0f2fe,stroke:#0284c7,color:#082f49
195
+ class R1,R2,R3,R4,R5,R6 stay
196
+ class ESC,CG,JUMP,UP up
197
+ class DOWN down
198
+ ```
199
+
200
+ Each box is the `reason` that the log and the status line show. Grey boxes
201
+ keep the current route. The cold-write guard serves the strongest plan tier
202
+ only when that tier is above the current route.
203
+
204
+ - `U` is the Jev probability mass above the current tier. `D` is the mass at
205
+ or below the candidate. The `uncertain` mass supports neither.
206
+ - The upgrade bar grows with the switching tax:
207
+ `bar = 0.75 + 0.15 × tax / (tax + $0.50)`. It stays between 0.75 and 0.90.
208
+ - A prompt continues the task when the Jev continuation answer is 0.7 or
209
+ more. A continuing prompt never votes for a downgrade.
210
+ - A downgrade has no tax term.
211
+ - Upgrades have no cooldown. Plan, then execute, then hard work again is a
212
+ valid sequence.
213
+ - A failure signal is two `tool_result` errors with the same text and an edit
214
+ between them, in the last 40 messages. Each signal escalates once.
215
+ - The thresholds are defaults in [configuration](configuration.md#policy).
216
+
217
+ ## Cache and cost model
218
+
219
+ The gateway keeps one cache record for each model and effort, for example
220
+ `claude-opus-5-5@xhigh`. A change of effort replaces the cached messages, so
221
+ `medium` and `high` are two caches.
222
+
223
+ ```mermaid
224
+ stateDiagram-v2
225
+ direction LR
226
+ [*] --> unknown
227
+ unknown --> warm: response recorded
228
+ warm --> expired: TTL minus 30 s passed
229
+ expired --> warm: response recorded
230
+ warm --> unknown: history break or context shrank
231
+ expired --> unknown: history break or context shrank
232
+ ```
233
+
234
+ The switching tax compares the input cost of the next request on the
235
+ candidate and on the current route:
236
+
237
+ ```text
238
+ input_cost = P_read × W + P_write × (N − W)
239
+ tax = max(0, input_cost(candidate) − input_cost(current))
240
+ ```
241
+
242
+ | Symbol | Source |
243
+ | --------- | ------------------------------------------------------------------------------------------------------- |
244
+ | `N` | Next context: tokens and output of the last response. |
245
+ | `W` | Warm prefix of the candidate cache. Zero when the cache is `expired` or `unknown`. |
246
+ | `P_read` | List price for cache reads. |
247
+ | `P_write` | Input list price × 1.25 (5-minute TTL) or × 2 (1-hour TTL). The TTL comes from `usage.cache_creation`. |
248
+
249
+ - **Upper bound.** An effort change counts the whole prefix as lost. On some
250
+ models, the tools and the system prompt stay in the cache.
251
+ - **Context editing.** A context that shrinks by more than 20% with no fewer
252
+ messages resets the cache records only. The votes stay.
253
+ - **Cold-write guard.** A model with `billing: "credits"` bills cash. When its
254
+ cache is not warm, `policy.cashCapUsd` limits the first cache write. It is
255
+ not a budget for the turn. No default model bills credits.
256
+ - **Prices.** List prices in USD per million tokens. For plan models, the
257
+ dollars only give an order between models. A test pins the defaults to
258
+ `test/fixtures/list-prices.json`, which names its source and date.
259
+ - **Shadow estimate.** When the Jev choice differs from the current route, the
260
+ log records the cost of the next turn, the cost of each later turn with
261
+ output, and the turns until a switch repays its cost. The policy does not
262
+ read it.
263
+
264
+ ## Modules
265
+
266
+ ```mermaid
267
+ flowchart TD
268
+ subgraph scripts["scripts/ · entry points"]
269
+ D["gateway.mjs<br/>daemon"]
270
+ E["ensure-gateway.mjs<br/>hook"]
271
+ SL["statusline.mjs<br/>status line"]
272
+ SC["status.mjs<br/>/router:status"]
273
+ end
274
+ subgraph lib["lib/"]
275
+ GW["gateway<br/>HTTP, relay"]
276
+ IDLE["idle"]
277
+ SSE["sse"]
278
+ RT["router<br/>orchestration"]
279
+ FA["facts"]
280
+ JEV["jev"]
281
+ POL["policy"]
282
+ COST["cost"]
283
+ RW["rewrite"]
284
+ STA["status"]
285
+ STORE[("store")]
286
+ RUN["runtime"]
287
+ end
288
+
289
+ D -->|"creates, injects router"| GW
290
+ D --> RT & RUN & IDLE & STA & STORE
291
+ GW --> IDLE & SSE & STA
292
+ GW -.->|"side-request test"| RT
293
+ RT --> FA & JEV & POL & COST & RW & STORE
294
+ POL --> COST --> RW
295
+ STA --> RW
296
+ E --> RUN & STA & STORE
297
+ SL --> RUN & STA
298
+ SC --> RUN & STA
299
+ RUN --> STORE
300
+
301
+ classDef pure fill:#ecfdf5,stroke:#059669,color:#064e3b
302
+ classDef io fill:#fff7ed,stroke:#ea580c,color:#431407
303
+ class FA,POL,COST,RW,SSE pure
304
+ class STORE,GW,JEV io
305
+ style scripts fill:#f8fafc,stroke:#94a3b8,color:#0f172a
306
+ style lib fill:#f8fafc,stroke:#94a3b8,color:#0f172a
307
+ ```
308
+
309
+ Green modules are pure functions. Orange modules do I/O: `gateway` serves
310
+ HTTP, `jev` calls Jev, and in `lib/` `store` is the only module that writes
311
+ files. Outside `lib/`, the hook `ensure-gateway.mjs` also opens `gateway.log`
312
+ for the daemon that it starts. The Jev transport and the router are injected,
313
+ so tests run without a network. `gateway` imports one pure function from
314
+ `router`, the side-request test, not the router itself. Modules that need
315
+ settings or tier names import `config.mjs` (not drawn).
316
+
317
+ | Module | Responsibility |
318
+ | --------- | --------------------------------------------------------------------------- |
319
+ | `gateway` | HTTP server, loopback checks, relay with `pipeline()`, status endpoint. |
320
+ | `router` | One request: class, facts, advice, policy, rewrite, session memory, log. |
321
+ | `facts` | Prompt, recent turns, continuation, failure signal, resend key. |
322
+ | `jev` | One bounded Jev request with a Choice (tier) and a Noul (continuation). |
323
+ | `policy` | Switching policy, context fit, cold-write guard. |
324
+ | `cost` | Cache key, warmth, input cost, switching tax, shadow estimate. |
325
+ | `rewrite` | Model, effort, thinking and output limit for each model family. |
326
+ | `sse` | `usage` from SSE and JSON responses. |
327
+ | `idle` | When the daemon can exit. |
328
+ | `status` | Status snapshot, status line segment, `/router:status` report. |
329
+ | `store` | Session memory, `decisions.jsonl`, rotation. |
330
+ | `runtime` | Configuration path (anchored under `~/.claude`), data directory, flags, gateway process check. |
331
+
332
+ ## Gateway lifecycle
333
+
334
+ ```mermaid
335
+ stateDiagram-v2
336
+ direction LR
337
+ [*] --> Running: hook starts it, detached
338
+ Running --> Draining: SIGTERM
339
+ Draining --> Stopped: streams done, max 10 min
340
+ Running --> Stopped: 2 h idle
341
+ Stopped --> Running: next prompt
342
+ ```
343
+
344
+ - The `SessionStart` and `UserPromptSubmit` hooks run `ensure-gateway.mjs`.
345
+ It starts a missing gateway and sends `SIGTERM` to an older version. It
346
+ never replaces a newer version.
347
+ - On `SIGTERM`, the gateway releases the port at once. A new gateway can
348
+ start while the old one finishes its streams.
349
+ - The idle exit waits for two hours without requests. It also waits while a
350
+ request is open or a turn waits for a tool result, for example a
351
+ permission prompt. A waiting turn stops counting after one day.
352
+ - A second daemon on a busy port exits quietly.
353
+
354
+ ## Failure handling
355
+
356
+ | Failure | Behavior |
357
+ | ---------------------------------- | --------------------------------------------------------------------------------------------- |
358
+ | Anthropic 429, 529, 5xx | Pass through with `Retry-After`. Claude Code owns retries. |
359
+ | Upstream reset during a stream | The client connection resets, and Claude Code retries at once. |
360
+ | Client leaves (Esc) | The gateway stops the upstream request. |
361
+ | Jev error or timeout | One retry inside the 1.5 s budget. Then the current route stays. |
362
+ | Three Jev failures in a row | New turns skip Jev for one minute, then try once. |
363
+ | Routing error | The baseline tier serves. The alias never goes to Anthropic. |
364
+ | Disk error | Logged. The routing decision stays. |
365
+ | Unexpected exception | Logged. The daemon continues. |
366
+
367
+ ## Security and privacy
368
+
369
+ - The gateway listens on `127.0.0.1` only. It refuses a non-loopback `Host`
370
+ (DNS rebinding) and any request with an `Origin` header, also from a page on
371
+ another loopback port: Claude Code sends none.
372
+ - Jev receives the prompt and the text of the six turns before it, up to
373
+ 1,200 characters each. A longer text keeps about 600 characters from its
374
+ start and from its end. Tool results and system reminders are not sent.
375
+ - The Jev key is in the macOS Keychain. The status endpoint shows only whether
376
+ a key is set.
377
+ - `decisions.jsonl` has no prompt text.
378
+
379
+ ## Observability
380
+
381
+ - `GET /router/status?session=<id>` gives the routes and the last turn of a
382
+ session. The status line and `/router:status` read it.
383
+ - `decisions.jsonl` has one line for each decision (with the model and the
384
+ effort sent), each response, each failed routed turn (`failed`, with the
385
+ status) and each routing error (`reason: error`, without the message). It
386
+ has no prompt text.
387
+ - `gateway.log` has the daemon events.
388
+
389
+ The [user guide](user-guide.md#read-the-status-line) explains how to read
390
+ them. The [data directory](configuration.md#data-directory) gives their
391
+ retention.
392
+
393
+ ## Release
394
+
395
+ ```mermaid
396
+ flowchart LR
397
+ TAG["Signed tag<br/>vX.Y.Z on main"] --> VER["Check signature,<br/>tag = package.json"]
398
+ VER --> TEST["Lint, tests,<br/>pack dry run"]
399
+ TEST --> NPM["npm publish<br/>with provenance"]
400
+ NPM --> GH["GitHub release<br/>named by the tag"]
401
+
402
+ classDef step fill:#eef2ff,stroke:#6366f1,color:#1e1b4b
403
+ class TAG,VER,TEST,NPM,GH step
404
+ ```
405
+
406
+ - The repository root is the plugin and the npm package
407
+ `@alexeiled/claude-router`.
408
+ - Claude Code installs the plugin from Git: the marketplace source is
409
+ `github`. An npm source fails with npm 12 (`EALLOWREMOTE`).
410
+ - `claude plugin update` compares the version in `.claude-plugin/plugin.json`.
411
+ Each release changes it together with `package.json`.
412
+ - `ci.yml` runs the checks and the tests for each push and pull request.
413
+
414
+ To work on the code:
415
+
416
+ ```sh
417
+ npm install
418
+ git config --local core.hooksPath scripts/git-hooks # Biome, tests and Gitleaks before commit and push
419
+ npm test && npm run check # node:test, Biome
420
+ claude --plugin-dir . --model jev-router # with ANTHROPIC_BASE_URL and TYPESAFE_API_KEY
421
+ ```
422
+
423
+ ## Known limits
424
+
425
+ - The request body does not show the exit from plan mode.
426
+ - The thresholds are start values. The `observed` and `shadow` lines in
427
+ `decisions.jsonl` are the input for tuning.
428
+ - The gateway reads its configuration once. A change needs a restart.
429
+ - Without the hint headers, the gateway guesses side requests. A missed side
430
+ request with a short history counts as a history break.
431
+ - `x-claude-code-agent-type` goes to the log only.
432
+ - The shadow estimate uses the last output size for both routes, but effort
433
+ changes how much a model writes.
434
+
435
+ A day of real usage is in [Evaluation](evaluation.md).