@alexeiled/claude-router 0.8.0 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +10 -6
- package/CHANGELOG.md +19 -0
- package/README.md +51 -100
- package/docs/architecture.md +55 -419
- package/docs/configuration.md +86 -182
- package/docs/evaluation.md +62 -81
- package/docs/native-router.md +79 -0
- package/docs/user-guide.md +103 -172
- package/hooks/hooks.json +1 -26
- package/hooks/native-router.mjs +635 -0
- package/lib/config.mjs +20 -58
- package/lib/{facts.mjs → facts-pure.mjs} +2 -23
- package/lib/{jev.mjs → jev-contract.mjs} +17 -38
- package/lib/native-cost.mjs +88 -0
- package/lib/native-display.mjs +75 -0
- package/lib/native-jev.mjs +125 -0
- package/lib/native-panel.mjs +150 -0
- package/lib/native-router.mjs +216 -0
- package/lib/policy.mjs +17 -21
- package/package.json +9 -8
- package/scripts/migrate-config.mjs +57 -0
- package/types/index.d.ts +95 -0
- package/lib/cost.mjs +0 -93
- package/lib/gateway.mjs +0 -221
- package/lib/idle.mjs +0 -41
- package/lib/rewrite.mjs +0 -130
- package/lib/router.mjs +0 -276
- package/lib/runtime.mjs +0 -78
- package/lib/sse.mjs +0 -57
- package/lib/status.mjs +0 -152
- package/lib/store.mjs +0 -76
- package/scripts/ensure-gateway.mjs +0 -143
- package/scripts/gateway.mjs +0 -81
- package/scripts/git-hooks/pre-commit +0 -41
- package/scripts/git-hooks/pre-push +0 -34
- package/scripts/status.mjs +0 -20
- package/scripts/statusline.mjs +0 -45
- package/scripts/transcript-models.sh +0 -4
- package/skills/high/SKILL.md +0 -9
- package/skills/low/SKILL.md +0 -7
- package/skills/medium/SKILL.md +0 -8
- package/skills/micro/SKILL.md +0 -7
- package/skills/setup/SKILL.md +0 -31
- package/skills/status/SKILL.md +0 -12
package/docs/architecture.md
CHANGED
|
@@ -1,452 +1,88 @@
|
|
|
1
1
|
# Architecture
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
A local gateway changes each request before it goes to Anthropic. TypeSafe Jev
|
|
5
|
-
advises a tier, and a local policy makes the decision.
|
|
6
|
-
|
|
7
|
-
- [System context](#system-context)
|
|
8
|
-
- [Tiers](#tiers)
|
|
9
|
-
- [Request flow](#request-flow)
|
|
10
|
-
- [Request classes](#request-classes)
|
|
11
|
-
- [Switching policy](#switching-policy)
|
|
12
|
-
- [Cache and cost model](#cache-and-cost-model)
|
|
13
|
-
- [Modules](#modules)
|
|
14
|
-
- [Gateway lifecycle](#gateway-lifecycle)
|
|
15
|
-
- [Failure handling](#failure-handling)
|
|
16
|
-
- [Security and privacy](#security-and-privacy)
|
|
17
|
-
- [Observability](#observability)
|
|
18
|
-
- [Release](#release)
|
|
19
|
-
- [Known limits](#known-limits)
|
|
20
|
-
|
|
21
|
-
## System context
|
|
22
|
-
|
|
23
|
-
```mermaid
|
|
24
|
-
flowchart TB
|
|
25
|
-
subgraph machine["Your machine"]
|
|
26
|
-
direction LR
|
|
27
|
-
CC["Claude Code<br/>model: jev-router"]
|
|
28
|
-
HOOK["Plugin hooks"]
|
|
29
|
-
UI["Status line<br/>/router:status"]
|
|
30
|
-
GW["Gateway<br/>127.0.0.1:43170"]
|
|
31
|
-
DATA[("Data directory<br/>sessions · decisions.jsonl")]
|
|
32
|
-
end
|
|
33
|
-
subgraph remote["Remote services"]
|
|
34
|
-
direction LR
|
|
35
|
-
API["api.anthropic.com"]
|
|
36
|
-
JEV["TypeSafe Jev"]
|
|
37
|
-
end
|
|
38
|
-
|
|
39
|
-
CC <-->|"Messages API"| GW
|
|
40
|
-
HOOK -.->|"start or replace"| GW
|
|
41
|
-
UI -.->|"GET /router/status"| GW
|
|
42
|
-
GW -->|"prompt and<br/>recent turns"| JEV
|
|
43
|
-
GW <-->|"changed request,<br/>unchanged response"| API
|
|
44
|
-
GW --> DATA
|
|
45
|
-
|
|
46
|
-
classDef ext fill:#eef2ff,stroke:#6366f1,color:#1e1b4b
|
|
47
|
-
classDef core fill:#ecfdf5,stroke:#059669,color:#064e3b
|
|
48
|
-
classDef store fill:#fff7ed,stroke:#ea580c,color:#431407
|
|
49
|
-
class JEV,API ext
|
|
50
|
-
class GW core
|
|
51
|
-
class DATA store
|
|
52
|
-
style machine fill:#f8fafc,stroke:#94a3b8,color:#0f172a
|
|
53
|
-
style remote fill:#f8fafc,stroke:#94a3b8,color:#0f172a
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
- `/router:setup` points `ANTHROPIC_BASE_URL` at the gateway and sets the
|
|
57
|
-
model to `jev-router[1m]`. One gateway serves all sessions on the machine.
|
|
58
|
-
- The gateway changes only requests for the alias. It sets `model`,
|
|
59
|
-
`output_config.effort` and `thinking`. For a small model, it also limits
|
|
60
|
-
`max_tokens` and removes the 1M context beta header and the thinking edits.
|
|
61
|
-
- Claude Code builds each request for the model that the alias behaves as
|
|
62
|
-
(Opus). A model without one of its [request features](configuration.md#models)
|
|
63
|
-
gets the request without it, the way Claude Code retries after a 400:
|
|
64
|
-
- no `mid-conversation-tool-changes` (Haiku): `tool_addition` and
|
|
65
|
-
`tool_removal` blocks go; their cache breakpoint moves to the block before;
|
|
66
|
-
- no `per-turn-control` (Haiku): the beta header and the
|
|
67
|
-
`output_config` on messages (the effort of each turn) go;
|
|
68
|
-
- no `mid-conversation-system` (Haiku): each `system` message becomes a
|
|
69
|
-
`<system-reminder>` text in the user message next to it.
|
|
70
|
-
The same history adapts the same way on every turn, so the cached prefix
|
|
71
|
-
holds.
|
|
72
|
-
- On a model with `per-turn-control`, the effort of each turn overrides the
|
|
73
|
-
request's effort. A route with its own effort (`medium`, `high`) therefore
|
|
74
|
-
sets it in every message's `output_config` too.
|
|
75
|
-
- The gateway never changes `system` or `tools`, and never changes `messages`
|
|
76
|
-
for a model that takes all the features. Prompt caching and a claude.ai
|
|
77
|
-
login work as with a direct connection.
|
|
78
|
-
- Responses go back unchanged. The gateway reads `usage` from them.
|
|
79
|
-
|
|
80
|
-
**Why a gateway.** Only a gateway can route every turn. The tier skills are
|
|
81
|
-
manual pins. Claude Code 2.1.x switches the model from their `model:`
|
|
82
|
-
frontmatter for `sonnet` and `opus`, and that turn passes through. For `haiku`
|
|
83
|
-
it sends the turn to the alias; the gateway reads the `/router:micro` command
|
|
84
|
-
in the prompt and serves the turn on `micro`, reason `pinned`. After a pin, the
|
|
85
|
-
next prompt is decided from the route before the pin.
|
|
86
|
-
|
|
87
|
-
## Tiers
|
|
88
|
-
|
|
89
|
-
Four tiers, from `micro` (Haiku) to `high` (Opus at `xhigh`), map to a model
|
|
90
|
-
and an effort. The [README](../README.md#how-jev-selects-a-tier) shows the
|
|
91
|
-
work that Jev selects each tier for. The [routes](configuration.md#routes)
|
|
92
|
-
set the mapping.
|
|
93
|
-
|
|
94
|
-
- `low` is the baseline.
|
|
95
|
-
- `medium` and `high` use the same model at two efforts.
|
|
96
|
-
- The gateway lowers an effort to a level that the model accepts. Haiku
|
|
97
|
-
accepts no effort, so it runs without thinking.
|
|
3
|
+
Claude Code owns the model request. The router Mod asks Jev for tier advice, applies local routing policy, and changes only the next main-conversation step's model and effort. Every subagent step passes through with Claude's model selection unchanged.
|
|
98
4
|
|
|
99
5
|
## Request flow
|
|
100
6
|
|
|
101
|
-
A new user turn:
|
|
102
|
-
|
|
103
7
|
```mermaid
|
|
104
8
|
sequenceDiagram
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
9
|
+
autonumber
|
|
10
|
+
actor Dev as Developer
|
|
11
|
+
participant Code as Claude Code
|
|
12
|
+
participant Mod as Router Mod
|
|
13
|
+
participant Jev
|
|
14
|
+
participant API as Anthropic
|
|
15
|
+
Dev->>Code: Prompt
|
|
16
|
+
Code->>Mod: turn.start and turn.step
|
|
17
|
+
alt Main conversation, Auto
|
|
18
|
+
Mod->>Jev: Current prompt and bounded dialogue
|
|
19
|
+
Jev-->>Mod: Tier advice
|
|
20
|
+
Note over Mod: Apply votes, context fit, cache costs and failure rules
|
|
21
|
+
Mod-->>Code: Forward model and effort
|
|
22
|
+
else Manual, subagent, or unavailable
|
|
23
|
+
Mod-->>Code: Forward model and effort unchanged
|
|
24
|
+
end
|
|
25
|
+
Code->>API: Native authenticated request
|
|
26
|
+
API-->>Code: Response stream and usage
|
|
27
|
+
Code-->>Mod: Main step result
|
|
28
|
+
Mod-->>Dev: Status band and details pane
|
|
124
29
|
```
|
|
125
30
|
|
|
126
|
-
|
|
127
|
-
route of their turn.
|
|
31
|
+
The Mod does not construct an Anthropic request or proxy its response. Claude Code owns authentication, request fields, streaming, tools, model fallback, and its native usage ledger. No local gateway process or port is part of the runtime.
|
|
128
32
|
|
|
129
|
-
|
|
33
|
+
Jev receives only the current prompt and up to six recent user or assistant text messages. Each text is capped at 1,200 characters. System messages, tool inputs, and tool results are not included. The Jev key is a sensitive plugin option. The Mod does not receive Claude's Anthropic credentials.
|
|
130
34
|
|
|
131
|
-
|
|
132
|
-
flowchart TD
|
|
133
|
-
IN(["Request"]) --> ALIAS{"model is<br/>jev-router?"}
|
|
134
|
-
ALIAS -- no --> PASS["Pass through unchanged"]
|
|
135
|
-
ALIAS -- yes --> MSG{"POST<br/>/v1/messages?"}
|
|
136
|
-
MSG -- no --> LAST["Last route of the session"]
|
|
137
|
-
MSG -- yes --> SIDE{"Side request?"}
|
|
138
|
-
SIDE -- yes --> AUX["auxiliaryTier<br/>memory unchanged"]
|
|
139
|
-
SIDE -- no --> BRK{"History break?"}
|
|
140
|
-
BRK -- yes --> RESET["Reset cache estimates,<br/>votes and hold"]
|
|
141
|
-
BRK -- no --> TOOL
|
|
142
|
-
RESET --> TOOL{"tool_result or<br/>resent request?"}
|
|
143
|
-
TOOL -- yes --> KEEP["Route of the turn"]
|
|
144
|
-
TOOL -- no --> POL["Jev and switching policy"]
|
|
145
|
-
KEEP --> FIT["Fit the context window"]
|
|
146
|
-
POL --> FIT
|
|
147
|
-
FIT --> OUT(["Change and forward"])
|
|
35
|
+
Direct Claude Code 2.1.289 probes accepted signed-thinking history and tool results across Sonnet, Opus, and Haiku switches. Sonnet and Opus accepted histories above 589K input tokens. Deferred MCP tools worked on rewritten Haiku with a 128K output limit. These checks cover tested requests. They do not prove every Claude Code feature combination.
|
|
148
36
|
|
|
149
|
-
|
|
150
|
-
classDef act fill:#ecfdf5,stroke:#059669,color:#064e3b
|
|
151
|
-
class PASS,LAST,AUX stop
|
|
152
|
-
class RESET,KEEP,POL,FIT act
|
|
153
|
-
```
|
|
37
|
+
## Responsibilities
|
|
154
38
|
|
|
155
|
-
|
|
|
156
|
-
|
|
|
157
|
-
|
|
|
158
|
-
|
|
|
159
|
-
|
|
|
160
|
-
|
|
|
161
|
-
|
|
|
162
|
-
|
|
|
163
|
-
|
|
|
164
|
-
|
|
|
165
|
-
| Session memory | Key `x-claude-code-session-id`. A subagent adds its agent id: `<session>.<agent id>`. Its turns do not change the route of the main conversation. |
|
|
39
|
+
| Component | Responsibility |
|
|
40
|
+
| --------------------------------------------- | -------------------------------------------------------------------------------- |
|
|
41
|
+
| [Native hooks](../hooks/native-router.mjs) | Turn identity, controls, lifecycle, per-step rewrites, and response observation. |
|
|
42
|
+
| [Jev contract](../lib/jev-contract.mjs) | Bounded request payload, answer validation, and retry rules. |
|
|
43
|
+
| [Native Jev](../lib/native-jev.mjs) | Native HTTP, deadline, in-flight admission, and circuit breaker. |
|
|
44
|
+
| [Router controller](../lib/native-router.mjs) | Context fit, model identity, cache observations, and one-turn pins. |
|
|
45
|
+
| [Policy](../lib/policy.mjs) | Tier votes, escalation, and switching gates. |
|
|
46
|
+
| [Native costs](../lib/native-cost.mjs) | Cache uncertainty bounds and switching estimates. |
|
|
47
|
+
| [Native panel](../lib/native-panel.mjs) | Route status, usage metrics, controls, and validated tuning. |
|
|
48
|
+
| Claude Code | API requests, tools, authentication, stream output, and usage ledger. |
|
|
166
49
|
|
|
167
|
-
|
|
50
|
+
Node.js is not part of routing. It is used for development and the optional v0.8 configuration migration.
|
|
168
51
|
|
|
169
|
-
|
|
170
|
-
flowchart TD
|
|
171
|
-
S(["New user turn"]) --> ERR{"Same error twice,<br/>edit between?"}
|
|
172
|
-
ERR -- yes --> ESC["escalation<br/>one tier up for 2 turns"]
|
|
173
|
-
ERR -- no --> HOLD{"Hold active?"}
|
|
174
|
-
HOLD -- yes --> R1["hold"]
|
|
175
|
-
HOLD -- no --> ADV{"Jev answered?"}
|
|
176
|
-
ADV -- no --> R2["no-advice"]
|
|
177
|
-
ADV -- yes --> CONT{"Continues<br/>the task?"}
|
|
178
|
-
CONT -- yes --> R3["continuation"]
|
|
179
|
-
CONT -- no --> DIR{"Jev choice vs<br/>current tier"}
|
|
180
|
-
DIR -- "same or uncertain" --> R4["same-tier<br/>uncertain"]
|
|
181
|
-
DIR -- higher --> GUARD{"Cold-write<br/>guard blocks?"}
|
|
182
|
-
GUARD -- yes --> CG["cash-gate<br/>strongest plan tier"]
|
|
183
|
-
GUARD -- no --> JMP{"2 tiers up,<br/>U ≥ 0.95?"}
|
|
184
|
-
JMP -- yes --> JUMP["jump"]
|
|
185
|
-
JMP -- no --> UPQ{"2 votes,<br/>U ≥ bar?"}
|
|
186
|
-
UPQ -- yes --> UP["upgrade"]
|
|
187
|
-
UPQ -- no --> R5["upgrade-pending"]
|
|
188
|
-
DIR -- lower --> DNQ{"2 votes,<br/>D ≥ bar?"}
|
|
189
|
-
DNQ -- yes --> DOWN["downgrade"]
|
|
190
|
-
DNQ -- no --> R6["downgrade-pending"]
|
|
52
|
+
## Decision and context
|
|
191
53
|
|
|
192
|
-
|
|
193
|
-
classDef up fill:#fef3c7,stroke:#d97706,color:#451a03
|
|
194
|
-
classDef down fill:#e0f2fe,stroke:#0284c7,color:#082f49
|
|
195
|
-
class R1,R2,R3,R4,R5,R6 stay
|
|
196
|
-
class ESC,CG,JUMP,UP up
|
|
197
|
-
class DOWN down
|
|
198
|
-
```
|
|
54
|
+
One logical turn gets one Jev classification. Tool continuations reuse the decision and add no votes. Before each step, the controller checks whether the selected model still fits after new tool output.
|
|
199
55
|
|
|
200
|
-
|
|
201
|
-
keep the current route. The cold-write guard serves the strongest plan tier
|
|
202
|
-
only when that tier is above the current route.
|
|
56
|
+
A move to a smaller context window needs a main-response measurement from the current history and Claude's local context estimate. The router uses the larger available reading and reserves 20% of the model window. If it cannot establish fit, it keeps the current model with `context-unknown`. It does not infer token count from character count.
|
|
203
57
|
|
|
204
|
-
-
|
|
205
|
-
or below the candidate. The `uncertain` mass supports neither.
|
|
206
|
-
- The upgrade bar grows with the switching tax:
|
|
207
|
-
`bar = 0.75 + 0.15 × tax / (tax + $0.50)`. It stays between 0.75 and 0.90.
|
|
208
|
-
- The downgrade bar grows the same way, from the other side: a candidate
|
|
209
|
-
colder than the incumbent pays a cache write, not just a smaller model.
|
|
210
|
-
`bar = 0.90 + 0.08 × tax / (tax + $0.50)`. It stays between 0.90 and 0.98.
|
|
211
|
-
- The downgrade tax nets the savings of the next `H` turns
|
|
212
|
-
(`policy.downgradeHorizonTurns`, default 5):
|
|
213
|
-
`tax = max(0, next + (H − 1) × later)`. `next` is the cost difference of
|
|
214
|
-
the next turn, input and output. `later` is the difference of each later
|
|
215
|
-
turn, with both caches warm. Both come from the
|
|
216
|
-
[shadow estimate](#cache-and-cost-model). Example: warm Opus at `xhigh` to
|
|
217
|
-
a cold Sonnet, 104k of context, 4k of output, 1-hour TTL. The input tax is
|
|
218
|
-
$0.40 (bar 0.935); net of five turns it is $0.20 (bar 0.922).
|
|
219
|
-
- When either model has no `output` price, or a zero one, the downgrade tax
|
|
220
|
-
is the input tax alone. A missing price is unknown, not free.
|
|
221
|
-
- A prompt continues the task when the Jev continuation answer is 0.7 or
|
|
222
|
-
more. A continuing prompt never votes for a downgrade.
|
|
223
|
-
- Upgrades have no cooldown. Plan, then execute, then hard work again is a
|
|
224
|
-
valid sequence.
|
|
225
|
-
- A failure signal is two `tool_result` errors with the same text and an edit
|
|
226
|
-
between them, in the last 40 messages. Each signal escalates once.
|
|
227
|
-
- The thresholds are defaults in [configuration](configuration.md#policy).
|
|
58
|
+
An explicit context-window failure marks that model ineligible until the next history reset. An engine fallback suspends rewriting for the rest of the turn. The router does not retry an Anthropic request.
|
|
228
59
|
|
|
229
|
-
##
|
|
60
|
+
## State and controls
|
|
230
61
|
|
|
231
|
-
|
|
232
|
-
`claude-opus-5-5@xhigh`. A change of effort replaces the cached messages, so
|
|
233
|
-
`medium` and `high` are two caches.
|
|
62
|
+
`$.state` stores route decisions, response observations, and UI state for the active session. Version-checked writes reject stale turn results. A private in-memory view merges state writes during one dispatch because state reads are frozen for that dispatch.
|
|
234
63
|
|
|
235
|
-
|
|
236
|
-
stateDiagram-v2
|
|
237
|
-
direction LR
|
|
238
|
-
[*] --> unknown
|
|
239
|
-
unknown --> warm: response recorded
|
|
240
|
-
warm --> expired: TTL minus 30 s passed
|
|
241
|
-
expired --> warm: response recorded
|
|
242
|
-
warm --> unknown: history break or context shrank
|
|
243
|
-
expired --> unknown: history break or context shrank
|
|
244
|
-
```
|
|
64
|
+
Clear, branch, rewind, and committed compaction clear cache evidence and votes. A fresh session on the configured baseline starts in Auto. A fresh session on another model starts in Manual. Resume restores the selected session's saved mode. Hot reload retains valid `$.state`. Running `/model` enters Manual, including an explicit choice of the baseline. `/router auto` resumes routing. `/router off` also enters Manual. Pins apply to the next turn and its tool continuations. The prior Auto route resumes afterward.
|
|
245
65
|
|
|
246
|
-
|
|
247
|
-
candidate and on the current route:
|
|
66
|
+
Subagent events pass directly to Claude Code. They do not create Jev requests or change main-conversation metrics. This is the selected scope because Mods do not expose whether a subagent model was explicit or inherited.
|
|
248
67
|
|
|
249
|
-
|
|
250
|
-
input_cost = P_read × W + P_write × (N − W)
|
|
251
|
-
tax = max(0, input_cost(candidate) − input_cost(current))
|
|
252
|
-
```
|
|
68
|
+
## Jev deadline and failure handling
|
|
253
69
|
|
|
254
|
-
|
|
255
|
-
| --------- | ------------------------------------------------------------------------------------------------------ |
|
|
256
|
-
| `N` | Next context: tokens and output of the last response. |
|
|
257
|
-
| `W` | Warm prefix of the candidate cache. Zero when the cache is `expired` or `unknown`. |
|
|
258
|
-
| `P_read` | List price for cache reads. |
|
|
259
|
-
| `P_write` | Input list price × 1.25 (5-minute TTL) or × 2 (1-hour TTL). The TTL comes from `usage.cache_creation`. |
|
|
70
|
+
Each advice attempt has a default 1,500 ms total deadline, including any transient retry. A timeout does not retry. `Retry-After` is honored only when it fits the remaining budget. Three launched failures pause Jev for 60 seconds. Missing-key, busy, and paused skips do not count as failures.
|
|
260
71
|
|
|
261
|
-
-
|
|
262
|
-
models, the tools and the system prompt stay in the cache.
|
|
263
|
-
- **Context editing.** A context that shrinks by more than 20% with no fewer
|
|
264
|
-
messages resets the cache records only. The votes stay.
|
|
265
|
-
- **Cold-write guard.** A model with `billing: "credits"` bills cash. When its
|
|
266
|
-
cache is not warm, `policy.cashCapUsd` limits the first cache write. It is
|
|
267
|
-
not a budget for the turn. No default model bills credits.
|
|
268
|
-
- **Prices.** List prices in USD per million tokens. For plan models, the
|
|
269
|
-
dollars only give an order between models. A test pins the defaults to
|
|
270
|
-
`test/fixtures/list-prices.json`, which names its source and date.
|
|
271
|
-
- **Shadow estimate.** When the Jev choice differs from the current route, the
|
|
272
|
-
log records the cost of the next turn, the cost of each later turn with
|
|
273
|
-
output, and the turns until a switch repays its cost. The downgrade tax
|
|
274
|
-
uses the same arithmetic. The upgrade tax counts input only.
|
|
72
|
+
The pinned host API does not expose a per-request abort option for `$.http.fetch`. The Mod returns to the incumbent at its own deadline and blocks another advice request while that wire call remains pending. Claude Code's observed host timeout is about 30 seconds. Recorded reload and unload checks closed pending sockets after 24.7 and 28.5 seconds. Replacement closed one after 31.6 seconds. These runs do not show immediate per-call cancellation. A plugin network-policy refusal degrades routing and is never bypassed with a subprocess.
|
|
275
73
|
|
|
276
|
-
|
|
74
|
+
In a 20-process loopback test, timeout fallback had 1,504 ms p95 and 1,505 ms worst case. A 20-call burst made one wire request. The other 19 calls skipped as busy. The live Team canary reached Jev and received advice. See the [native acceptance records](../experiments/mod-router/results/native-http-acceptance.json) and [interruption record](../experiments/mod-router/results/native-interrupt-acceptance.json).
|
|
277
75
|
|
|
278
|
-
|
|
279
|
-
flowchart TD
|
|
280
|
-
subgraph scripts["scripts/ · entry points"]
|
|
281
|
-
D["gateway.mjs<br/>daemon"]
|
|
282
|
-
E["ensure-gateway.mjs<br/>hook"]
|
|
283
|
-
SL["statusline.mjs<br/>status line"]
|
|
284
|
-
SC["status.mjs<br/>/router:status"]
|
|
285
|
-
end
|
|
286
|
-
subgraph lib["lib/"]
|
|
287
|
-
GW["gateway<br/>HTTP, relay"]
|
|
288
|
-
IDLE["idle"]
|
|
289
|
-
SSE["sse"]
|
|
290
|
-
RT["router<br/>orchestration"]
|
|
291
|
-
FA["facts"]
|
|
292
|
-
JEV["jev"]
|
|
293
|
-
POL["policy"]
|
|
294
|
-
COST["cost"]
|
|
295
|
-
RW["rewrite"]
|
|
296
|
-
STA["status"]
|
|
297
|
-
STORE[("store")]
|
|
298
|
-
RUN["runtime"]
|
|
299
|
-
end
|
|
76
|
+
## Cache and cost
|
|
300
77
|
|
|
301
|
-
|
|
302
|
-
D --> RT & RUN & IDLE & STA & STORE
|
|
303
|
-
GW --> IDLE & SSE & STA
|
|
304
|
-
GW -.->|"side-request test"| RT
|
|
305
|
-
RT --> FA & JEV & POL & COST & RW & STORE
|
|
306
|
-
POL --> COST --> RW
|
|
307
|
-
STA --> RW
|
|
308
|
-
E --> RUN & STA & STORE
|
|
309
|
-
SL --> RUN & STA
|
|
310
|
-
SC --> RUN & STA
|
|
311
|
-
RUN --> STORE
|
|
78
|
+
Cache identity uses the exact response model ID and effective effort. The known Haiku dated snapshot has an explicit mapping. An unrecognized substituted model gets no cache credit for the requested route.
|
|
312
79
|
|
|
313
|
-
|
|
314
|
-
classDef io fill:#fff7ed,stroke:#ea580c,color:#431407
|
|
315
|
-
class FA,POL,COST,RW,SSE pure
|
|
316
|
-
class STORE,GW,JEV io
|
|
317
|
-
style scripts fill:#f8fafc,stroke:#94a3b8,color:#0f172a
|
|
318
|
-
style lib fill:#f8fafc,stroke:#94a3b8,color:#0f172a
|
|
319
|
-
```
|
|
80
|
+
The cached prefix is observed `cache_read + cache_creation`. Output tokens count only toward the next context size. Freshness uses the configured five-minute lifetime with its safety margin. Claude's usage counters do not prove one-hour cache lifetime.
|
|
320
81
|
|
|
321
|
-
|
|
322
|
-
HTTP, `jev` calls Jev, and in `lib/` `store` is the only module that writes
|
|
323
|
-
files. Outside `lib/`, the hook `ensure-gateway.mjs` also opens `gateway.log`
|
|
324
|
-
for the daemon that it starts. The Jev transport and the router are injected,
|
|
325
|
-
so tests run without a network. `gateway` imports one pure function from
|
|
326
|
-
`router`, the side-request test, not the router itself. Modules that need
|
|
327
|
-
settings or tier names import `config.mjs` (not drawn).
|
|
328
|
-
|
|
329
|
-
| Module | Responsibility |
|
|
330
|
-
| --------- | ---------------------------------------------------------------------------------------------- |
|
|
331
|
-
| `gateway` | HTTP server, loopback checks, relay with `pipeline()`, status endpoint. |
|
|
332
|
-
| `router` | One request: class, facts, advice, policy, rewrite, session memory, log. |
|
|
333
|
-
| `facts` | Prompt, recent turns, continuation, failure signal, resend key. |
|
|
334
|
-
| `jev` | One bounded Jev request with a Choice (tier) and a Noul (continuation). |
|
|
335
|
-
| `policy` | Switching policy, context fit, cold-write guard. |
|
|
336
|
-
| `cost` | Cache key, warmth, input cost, switching tax, shadow estimate. |
|
|
337
|
-
| `rewrite` | Model, effort, thinking and output limit for each model family. |
|
|
338
|
-
| `sse` | `usage` from SSE and JSON responses. |
|
|
339
|
-
| `idle` | When the daemon can exit. |
|
|
340
|
-
| `status` | Status snapshot, status line segment, `/router:status` report. |
|
|
341
|
-
| `store` | Session memory, `decisions.jsonl`, rotation. |
|
|
342
|
-
| `runtime` | Configuration path (anchored under `~/.claude`), data directory, flags, gateway process check. |
|
|
343
|
-
|
|
344
|
-
## Gateway lifecycle
|
|
345
|
-
|
|
346
|
-
```mermaid
|
|
347
|
-
stateDiagram-v2
|
|
348
|
-
direction LR
|
|
349
|
-
[*] --> Running: hook starts it, detached
|
|
350
|
-
Running --> Draining: SIGTERM
|
|
351
|
-
Draining --> Stopped: streams done, max 10 min
|
|
352
|
-
Running --> Stopped: 2 h idle
|
|
353
|
-
Stopped --> Running: next prompt
|
|
354
|
-
```
|
|
355
|
-
|
|
356
|
-
- The `SessionStart` and `UserPromptSubmit` hooks run `ensure-gateway.mjs`.
|
|
357
|
-
It starts a missing gateway and sends `SIGTERM` to an older version. It
|
|
358
|
-
never replaces a newer version.
|
|
359
|
-
- On `SIGTERM`, the gateway releases the port at once. A new gateway can
|
|
360
|
-
start while the old one finishes its streams.
|
|
361
|
-
- The idle exit waits for two hours without requests. It also waits while a
|
|
362
|
-
request is open or a turn waits for a tool result, for example a
|
|
363
|
-
permission prompt. A waiting turn stops counting after one day.
|
|
364
|
-
- A second daemon on a busy port exits quietly.
|
|
365
|
-
|
|
366
|
-
## Failure handling
|
|
367
|
-
|
|
368
|
-
| Failure | Behavior |
|
|
369
|
-
| ------------------------------ | ---------------------------------------------------------------- |
|
|
370
|
-
| Anthropic 429, 529, 5xx | Pass through with `Retry-After`. Claude Code owns retries. |
|
|
371
|
-
| Upstream reset during a stream | The client connection resets, and Claude Code retries at once. |
|
|
372
|
-
| Client leaves (Esc) | The gateway stops the upstream request. |
|
|
373
|
-
| Jev error or timeout | One retry inside the 1.5 s budget. Then the current route stays. |
|
|
374
|
-
| Three Jev failures in a row | New turns skip Jev for one minute, then try once. |
|
|
375
|
-
| Routing error | The baseline tier serves. The alias never goes to Anthropic. |
|
|
376
|
-
| Disk error | Logged. The routing decision stays. |
|
|
377
|
-
| Unexpected exception | Logged. The daemon continues. |
|
|
378
|
-
|
|
379
|
-
## Security and privacy
|
|
380
|
-
|
|
381
|
-
- The gateway listens on `127.0.0.1` only. It refuses a non-loopback `Host`
|
|
382
|
-
(DNS rebinding) and any request with an `Origin` header, also from a page on
|
|
383
|
-
another loopback port: Claude Code sends none.
|
|
384
|
-
- Jev receives the prompt and the text of the six turns before it, up to
|
|
385
|
-
1,200 characters each. A longer text keeps about 600 characters from its
|
|
386
|
-
start and from its end. Tool results and system reminders are not sent.
|
|
387
|
-
- The Jev key is in the macOS Keychain. The status endpoint shows only whether
|
|
388
|
-
a key is set.
|
|
389
|
-
- `decisions.jsonl` has no prompt text.
|
|
390
|
-
|
|
391
|
-
## Observability
|
|
392
|
-
|
|
393
|
-
- `GET /router/status?session=<id>` gives the routes and the last turn of a
|
|
394
|
-
session. The status line and `/router:status` read it.
|
|
395
|
-
- `decisions.jsonl` has one line for each decision (with the model and the
|
|
396
|
-
effort sent), each response, each failed routed turn (`failed`, with the
|
|
397
|
-
status) and each routing error (`reason: error`, without the message). It
|
|
398
|
-
has no prompt text.
|
|
399
|
-
- `gateway.log` has the daemon events.
|
|
400
|
-
|
|
401
|
-
The [user guide](user-guide.md#read-the-status-line) explains how to read
|
|
402
|
-
them. The [data directory](configuration.md#data-directory) gives their
|
|
403
|
-
retention.
|
|
404
|
-
|
|
405
|
-
## Release
|
|
406
|
-
|
|
407
|
-
```mermaid
|
|
408
|
-
flowchart LR
|
|
409
|
-
TAG["Signed tag<br/>vX.Y.Z on main"] --> VER["Check signature,<br/>tag = package.json"]
|
|
410
|
-
VER --> TEST["Lint, tests,<br/>pack dry run"]
|
|
411
|
-
TEST --> NPM["npm publish<br/>with provenance"]
|
|
412
|
-
NPM --> GH["GitHub release<br/>named by the tag"]
|
|
413
|
-
|
|
414
|
-
classDef step fill:#eef2ff,stroke:#6366f1,color:#1e1b4b
|
|
415
|
-
class TAG,VER,TEST,NPM,GH step
|
|
416
|
-
```
|
|
417
|
-
|
|
418
|
-
- The repository root is the plugin and the npm package
|
|
419
|
-
`@alexeiled/claude-router`.
|
|
420
|
-
- Claude Code installs the plugin from Git: the marketplace source is
|
|
421
|
-
`github`. An npm source fails with npm 12 (`EALLOWREMOTE`).
|
|
422
|
-
- `claude plugin update` compares the version in `.claude-plugin/plugin.json`.
|
|
423
|
-
Each release changes it together with `package.json`.
|
|
424
|
-
- `ci.yml` runs the checks and the tests for each push and pull request.
|
|
425
|
-
|
|
426
|
-
To work on the code:
|
|
427
|
-
|
|
428
|
-
```sh
|
|
429
|
-
npm install
|
|
430
|
-
git config --local core.hooksPath scripts/git-hooks # Biome, ShellCheck, tests and Gitleaks before commit and push
|
|
431
|
-
npm test && npm run check # node:test, Biome
|
|
432
|
-
claude --plugin-dir . --model jev-router # with ANTHROPIC_BASE_URL and TYPESAFE_API_KEY
|
|
433
|
-
```
|
|
82
|
+
The switching policy prices the candidate at a conservative cache-write bound and the incumbent at its best supported input bound. The pane shows the configured-price range and a conditional payback estimate. These are estimates, not measured savings. Claude's native `/cost` ledger owns total API cost. Jev spend is not included.
|
|
434
83
|
|
|
435
|
-
##
|
|
84
|
+
## Deployment
|
|
436
85
|
|
|
437
|
-
- The
|
|
438
|
-
- The thresholds are start values. The `observed` and `shadow` lines in
|
|
439
|
-
`decisions.jsonl` are the input for tuning.
|
|
440
|
-
- The gateway reads its configuration once. A change needs a restart.
|
|
441
|
-
- Without the hint headers, the gateway guesses side requests. A missed side
|
|
442
|
-
request with a short history counts as a history break.
|
|
443
|
-
- `x-claude-code-agent-type` goes to the log only.
|
|
444
|
-
- The shadow estimate and the downgrade tax use the last output size for
|
|
445
|
-
both routes, but effort changes how much a model writes.
|
|
446
|
-
- Opus 5.5 and Sonnet 5.5 bind thinking blocks to the model and the
|
|
447
|
-
conversation. Anthropic answers a replay after a prefix change with 400 for
|
|
448
|
-
accounts created on or after 2026-08-31. A switch between the two, with
|
|
449
|
-
thinking blocks in the history, passed on one older account (2026-10-01);
|
|
450
|
-
no newer account was tested.
|
|
86
|
+
Load the plugin directory with Claude Code's `--plugin-dir` option. The Team launcher uses an isolated `~/.claude-team/mods/router` directory, so updating the Mod does not replace another profile's plugin cache. The Team canary passed with the native plugin, a full Sonnet baseline, and a real Jev classification.
|
|
451
87
|
|
|
452
|
-
|
|
88
|
+
The v0.8.0 git version remains available as the rollback source. Native routing does not require the proxy. Use the [configuration guide](configuration.md) for optional settings and migration, and the [native guide](native-router.md) for panel and control behavior.
|