@kdejaeger/pi-model-router 0.4.1 → 0.4.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,41 +1,32 @@
1
1
  # pi-model-router
2
2
 
3
3
  [![npm version](https://img.shields.io/npm/v/@kdejaeger/pi-model-router)](https://www.npmjs.com/package/@kdejaeger/pi-model-router)
4
- [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
5
4
 
6
- **Intelligent per-turn model router extension** (forked from [yeliu84/pi-model-router](https://github.com/yeliu84/pi-model-router)) for the [pi-coding-agent](https://github.com/earendil-works/pi/tree/main/packages/coding-agent). Automatically selects between high, medium, and low-tier LLMs on every turn based on task intent and context size with automatic fallbacks, image-aware rerouting, context truncation, and classifier-based tier selection.
5
+ > Think of pi-model-router as an automatic transmission for your LLM it shifts gears up or down depending on what you're doing, so you never waste compute on a trivial task or run out of reasoning power on a complex one.
7
6
 
8
- > Think of it as an automatic transmission for your LLM -- it shifts gears up or down depending on what you're doing, so you never waste compute on a trivial task or run out of reasoning power on a complex one.
9
-
10
- ---
7
+ This extension (forked from [yeliu84/pi-model-router](https://github.com/yeliu84/pi-model-router)) for the [pi-coding-agent](https://github.com/earendil-works/pi/tree/main/packages/coding-agent) automatically selects between high, medium, and low-tier LLMs on every few turns. It considers conversation history, context size and thresholds, image presence, manual tier pins, tool-result continuation patterns, previous routing decisions, and automatic fallbacks — with context truncation as a last resort.
11
8
 
12
9
  ## Table of Contents
13
10
 
14
- - [How It Works](#how-it-works)
11
+ - [Overview](#overview)
15
12
  - [Installation](#installation)
16
13
  - [Quick Start](#quick-start)
17
14
  - [Configuration](#configuration)
15
+ - [Activating the Router](#activating-the-router)
18
16
  - [Commands](#commands)
19
17
  - [How Routing Works](#how-routing-works)
20
- - [Example Configurations](#example-configurations)
21
18
  - [Architecture](#architecture)
22
- - [License](#license)
23
-
24
- ---
19
+ - [Documentation](#documentation)
25
20
 
26
- ## How It Works
27
21
 
28
- The pi-model-router registers itself as a **custom logical provider** (`router`) via `pi.registerProvider`. Each profile becomes a stable model (e.g., `router/balanced`). **The model shown in your footer stays fixed** while the underlying LLM changes per turn based on task complexity:
22
+ ## Overview
29
23
 
30
- ```
31
- footer shows: router/balanced
24
+ The pi-model-router registers itself as a **custom logical provider** (`router`) via `pi.registerProvider`. Each profile becomes a stable model (e.g., `router/cheap`). The model shown in your footer on the left stays fixed, while the underlying LLM changes per turn based on task complexity.
32
25
 
33
- Turn 1: "Plan the architecture for the new API" → openai/gpt-5.4-pro (high)
34
- Turn 2: "Yes, go ahead" → openai/gpt-5.4-pro (high)
35
- Turn 3: "Implement the API handlers" → google/gemini-flash (med)
36
- ```
26
+ ![Router footer](docs/footer.png)
27
+ ![Router screenshot](docs/screenshot.png)
37
28
 
38
- ---
29
+ For the full decision pipeline, see [How Routing Works](#how-routing-works).
39
30
 
40
31
  ## Installation
41
32
 
@@ -59,30 +50,95 @@ Or load directly for a single run:
59
50
  pi -e ./extensions/index.ts
60
51
  ```
61
52
 
62
- ---
63
-
64
53
  ## Quick Start
65
54
 
66
55
  1. **Install** the package (see above).
67
56
 
68
- 2. **Create a config file** at `.pi/model-router.json` in your project (bigger example configs later in this document):
69
-
70
- ```json
71
- {
72
- "profiles": {
73
- "balanced": {
74
- "high": { "model": "openai/gpt-5.4-pro", "thinking": "high" },
75
- "medium": { "model": "google/gemini-flash-latest", "thinking": "medium" },
76
- "low": { "model": "openai/gpt-5.4-nano", "thinking": "low" }
77
- },
78
- "cheap": {
79
- "high": { "model": "google/gemini-flash-latest", "thinking": "low" },
80
- "medium": { "model": "openai/gpt-5.4-nano", "thinking": "off" },
81
- "low": { "model": "google/gemini-flash-lite-latest", "thinking": "off" }
82
- }
83
- }
84
- }
85
- ```
57
+ 2. **Create a config file** at `.pi/model-router.json`. Here is an example.
58
+
59
+ ```json
60
+ {
61
+ "debug": true,
62
+ "classifierModels": ["openrouter/deepseek/deepseek-v4-flash"],
63
+ "classifierModelThinking": "high",
64
+ "defaultContextThresholdPercent": 80,
65
+ "contextThresholdPercentOverrides": {
66
+ "opencode/deepseek-v4-flash-free": 60,
67
+ "openrouter/deepseek/deepseek-v4-flash": 60,
68
+ "openrouter/deepseek/deepseek-v4-pro": 50,
69
+ "openrouter/google/gemma-4-31b-it": 50,
70
+ "openrouter/google/gemini-3-flash-preview": 80,
71
+ "openrouter/google/gemini-3.5-flash": 80,
72
+ "openrouter/google/gemini-3.1-pro-preview": 18,
73
+ "openrouter/moonshotai/kimi-k2.6": 70,
74
+ "openrouter/moonshotai/kimi-k2.7-code": 60,
75
+ "openrouter/minimax/minimax-m3": 50,
76
+ "openrouter/z-ai/glm-5.2": 50,
77
+ "openai-codex/gpt-5.4-mini": 60,
78
+ "openai-codex/gpt-5.5": 60,
79
+ "openai-codex/gpt-5.6-luna": 65,
80
+ "openai-codex/gpt-5.6-terra": 65,
81
+ "openai-codex/gpt-5.6-sol": 65,
82
+ "openrouter/openai/gpt-5.4-nano": 30,
83
+ "openrouter/anthropic/claude-opus-4.8": 65,
84
+ "openrouter/x-ai/grok-4.5": 50
85
+ },
86
+ "profiles": {
87
+ "cheap": {
88
+ "high": {
89
+ "model": "openrouter/deepseek/deepseek-v4-flash", "thinking": "xhigh",
90
+ "fallbacks": ["openrouter/minimax/minimax-m3"]
91
+ },
92
+ "medium": {
93
+ "model": "openrouter/deepseek/deepseek-v4-flash", "thinking": "high",
94
+ "fallbacks": ["openrouter/minimax/minimax-m3"]
95
+ },
96
+ "low": {
97
+ "model": "openrouter/deepseek/deepseek-v4-flash", "thinking": "off",
98
+ "fallbacks": ["openrouter/minimax/minimax-m3"]
99
+ }
100
+ },
101
+ "balanced": {
102
+ "high": {
103
+ "model": "openrouter/z-ai/glm-5.2", "thinking": "xhigh",
104
+ "fallbacks": ["openrouter/minimax/minimax-m3"]
105
+ },
106
+ "medium": {
107
+ "model": "openrouter/deepseek/deepseek-v4-flash", "thinking": "xhigh",
108
+ "fallbacks": ["openrouter/minimax/minimax-m3"]
109
+ },
110
+ "low": {
111
+ "model": "openrouter/deepseek/deepseek-v4-flash", "thinking": "off",
112
+ "fallbacks": ["openrouter/minimax/minimax-m3"]
113
+ }
114
+ },
115
+ "expensive": {
116
+ "high": {
117
+ "model": "openrouter/z-ai/glm-5.2", "thinking": "max",
118
+ "fallbacks": ["openrouter/x-ai/grok-4.5", "openai/gpt-5.6-luna"]
119
+ },
120
+ "medium": {
121
+ "model": "openrouter/z-ai/glm-5.2", "thinking": "max",
122
+ "fallbacks": ["openai/gpt-5.6-luna", "openrouter/x-ai/grok-4.5"]
123
+ },
124
+ "low": {
125
+ "model": "openrouter/minimax/minimax-m3", "thinking": "off",
126
+ "fallbacks": ["openrouter/z-ai/glm-5.2"]
127
+ }
128
+ },
129
+ "gpt-5.6": {
130
+ "high": { "model": "openai/gpt-5.6-sol", "thinking": "medium" },
131
+ "medium": { "model": "openai/gpt-5.6-terra", "thinking": "high" },
132
+ "low": { "model": "openai/gpt-5.6-luna", "thinking": "high" }
133
+ },
134
+ "opus-4.8": {
135
+ "high": { "model": "openrouter/anthropic/claude-opus-4.8", "thinking": "medium" },
136
+ "medium": { "model": "openrouter/anthropic/claude-opus-4.8", "thinking": "low" },
137
+ "low": { "model": "openrouter/anthropic/claude-opus-4.8", "thinking": "off" }
138
+ }
139
+ }
140
+ }
141
+ ```
86
142
 
87
143
  3. **Activate the router** -- choose one approach:
88
144
 
@@ -99,7 +155,127 @@ pi -e ./extensions/index.ts
99
155
  /router
100
156
  ```
101
157
 
102
- ---
158
+ ## How Routing Works
159
+
160
+ For every turn, the router executes this ordered pipeline:
161
+
162
+ ```
163
+ GATE 0: GOOGLE LOCK
164
+ Google thinking used last turn AND this is a tool continuation?
165
+ → Freeze the exact model/tier (skip everything below)
166
+
167
+ GATE 1: CLASSIFIER GATING
168
+ Manual pin active? → Use pinned tier (skip classifier)
169
+ New user message? → Run classifier from scratch
170
+ Tool-result continuation?
171
+ ├─ Initial tool results back after user prompt? → Run classifier
172
+ ├─ Multiple tool failures in a row? → Run classifier
173
+ ├─ Periodic check due? → Run classifier
174
+ └─ Otherwise → Reuse previous routing decision
175
+
176
+ POST-ROUTE CORRECTIONS (always apply)
177
+ Image doesn't fit? → Search current and higher tiers for a model supporting images
178
+ Context too full? → Search current and higher tiers for a model with a bigger context window
179
+
180
+ EXECUTION
181
+ Context window too small? → Trim oldest messages to fit the target model
182
+ Model failed? → Try fallbacks in order, then surface the error
183
+ Model got swapped? → Re-assert the router model after the turn
184
+ ```
185
+
186
+ ### Classifier Gating
187
+
188
+ When the router has an LLM classifier configured (`classifierModels`), it doesn't run on every turn. Instead, the classifier is gated by smart triggers that avoid waste while catching real tier mismatches. The classifier has final say on tier (post-route corrections like image escalation).
189
+
190
+ | Gate | Trigger | Reason |
191
+ |---|---------------------------------------------------------------------------------------------|---|
192
+ | **Google lock** | Previous model was Google with thinking, and this is a tool-result continuation | Freezes to the same model — any change breaks thought-signature replay |
193
+ | **Fresh feedback** | Tool-continuation count first reaches or exceeds `classifierRunOnceAfterToolCount` (default: 3) | Captures the assistant's first response + tool result after a new user message |
194
+ | **Tool failures** | Consecutive failed tool results (from the tail) ≥ `classifierRunAfterToolFailures` (default: 2) | Model is struggling — reclassify to potentially upgrade tier |
195
+ | **Interval** | Tool-continuation count % `classifierInterval` === 0 (default: 10) | Periodic re-check for long-running turns where initial assessment might be stale |
196
+
197
+ All counters reset per user turn — each new user message is treated as a fresh task. The tool failures gate counts **consecutive** failures from the tail: one successful tool result resets the count to 0, so it only fires when failures are actually piling up.
198
+
199
+ When the classifier is skipped, the **previous routing decision** is reused directly. Post-route corrections (image escalation) still apply regardless.
200
+
201
+ ### Context Controls
202
+
203
+ Modern models offer large context windows, but filling them to 100% before a tier switch is rarely desirable. Some models also increase pricing after a certain token threshold (e.g., double the input/output cost beyond 250K tokens). The context threshold lets you define the percentage of a model's window at which the router considers escalating to a higher tier.
204
+
205
+ **Context & Image Requirements** (`defaultContextThresholdPercent`): When the conversation context exceeds this percentage of a model's window, the router searches for a suitable model in the current or higher tiers.
206
+
207
+ **Per-Model Overrides** (`contextThresholdPercentOverrides`): Tune the context threshold for specific models. Keys are canonical model refs in `"provider/model"` format. Values are percentages; lower values cause the router to consider upgrading sooner.
208
+
209
+ ```json
210
+ {
211
+ "defaultContextThresholdPercent": 80,
212
+ "contextThresholdPercentOverrides": {
213
+ "openrouter/deepseek/deepseek-v4-flash": 60,
214
+ "openrouter/deepseek/deepseek-v4-pro": 50,
215
+ "openai-codex/gpt-5.5": 60,
216
+ "openrouter/google/gemini-3.1-pro-preview": 18
217
+ }
218
+ }
219
+ ```
220
+
221
+ ### Fallback Chains
222
+
223
+ Each tier can define `fallbacks` -- an ordered list of alternative models. If the primary model fails, the router retries each fallback in sequence before surfacing an error.
224
+
225
+ ```json
226
+ {
227
+ "profiles": {
228
+ "auto": {
229
+ "high": {
230
+ "model": "openai/gpt-5.4-pro",
231
+ "fallbacks": ["anthropic/claude-3-5-sonnet-20241022", "google/gemini-2.5-pro-latest"]
232
+ }
233
+ }
234
+ }
235
+ }
236
+ ```
237
+
238
+ When a fallback is used, `decision.isFallback` is set to `true` and shown in the status. The tier's configured thinking level (or runtime override) applies to all fallback models -- if a fallback doesn't support the requested level, pi silently clamps it.
239
+
240
+ If a model fails during a turn, the router retries it **once** (2 total attempts) before moving to the next fallback in the chain.
241
+
242
+ ### Image-Aware Auto-Routing
243
+
244
+ When the user attaches an image, the router checks whether the routed model supports image inputs. If not, it searches for another model in the same tier that does. If none are found, it escalates to higher tiers until a suitable model is found.
245
+
246
+ > **Note:** The search ensures that the selected model also fits the current context window requirements. To avoid landing on a model that can see images but not the full conversation, make sure your models have adequate context windows and image support configured.
247
+
248
+ ### Google Thinking Tool Continuation
249
+
250
+ When using Google models with thinking enabled, tool-result continuations require the **same model** to avoid thought-signature replay errors. The router detects this pattern and preserves the exact model/tier for the continuation turn.
251
+
252
+ ### Auto-Context Truncation
253
+
254
+ The router reports the **largest context window across all models in a profile** (scanning all tiers and their fallbacks for the maximum). When routing to a model with a smaller window, the router trims oldest messages (preserving the system prompt and the most recent message) to fit within the target model's limit.
255
+
256
+ Conservative estimation: **4 characters = 1 token**.
257
+
258
+ This is a rough last-resort cut, not a replacement for pi's built-in session compaction (`/compact`).
259
+
260
+ ### Session & Debugging
261
+
262
+ **Persistent State:** Router state persists across agent restarts AND conversation branches via `pi.appendEntry` with a custom `router-state` entry type. Pins, debug mode, the last routing decision, and the last non-router model are all preserved. State is **branch-safe** -- different conversation branches maintain independent state using `sessionManager.getBranch()`.
263
+
264
+ **Status Line:** The router shows its status as a single line in the pi TUI status bar, e.g.:
265
+ ```
266
+ ⇋ medium -> google/gemini-flash-latest
267
+ ```
268
+
269
+ The status line may show decision flags in brackets when applicable:
270
+ - `[fallback]` — a fallback model was used
271
+ - `[context]` — context threshold triggered an upgrade
272
+
273
+ **Debug Notifications:** With `/router debug on`, routing decisions and classifier runs are logged as notifications with timestamps. The classifier notification shows why the classifier was triggered (`cont` = tool-result continuations since the last user message, `fail` = consecutive recent tool failures).
274
+ ```
275
+ [10:32:15 AM] high -> openai/gpt-5.4-pro (high) - Classifier: multi-file architecture change across 4 services requires careful trade-off analysis.
276
+ [10:33:42 AM] medium -> google/gemini-flash-latest (medium) - Classifier: implementing a well-defined feature with clear acceptance criteria.
277
+ [10:34:10 AM] low -> openai/gpt-5.4-nano (low) - Classifier: simple field rename with no behavioral changes.
278
+ ```
103
279
 
104
280
  ## Configuration
105
281
 
@@ -112,15 +288,7 @@ Config is loaded from two locations and **merged**:
112
288
  | Global | User-wide | `~/.pi/agent/model-router.json` |
113
289
  | Project | Per-project | `.pi/model-router.json` |
114
290
 
115
- ### Config Merging Order
116
-
117
- Configs are merged: **Base defaults <- Global config <- Project config**.
118
-
119
- Project config values override global values, which override built-in defaults. Profiles are merged **deeply** -- if you define only a `high` tier override for a profile in your project config, the `medium` and `low` tiers are inherited from the global config.
120
-
121
- **When no config file exists**, the router loads with an empty profile list and no active models. Create a `.pi/model-router.json` with at least one profile to use the router.
122
-
123
-
291
+ For details on merging order and validation, see [Config Merging Order](docs/ARCHITECTURE.md#config-merging-order) in the architecture doc.
124
292
 
125
293
  ### Configuration Fields
126
294
 
@@ -133,7 +301,6 @@ Project config values override global values, which override built-in defaults.
133
301
  | `classifierRunAfterToolFailures` | `number` | `2` | Run the classifier after this many consecutive tool failures (counting from the tail of the current turn). Default: 2. |
134
302
  | `classifierInterval` | `number` | `10` | Run the classifier every N tool continuations as a periodic re-check (crossed interval buckets). Default: 10. Set to 0 to disable. |
135
303
  | `defaultContextThresholdPercent` | `number` | `90` | Default percentage threshold of a model's context window. If session context usage exceeds this percentage, the router searches for a suitable model in the current or higher tiers. |
136
- | `contextThresholdPercentOverrides` | `Record<string, number>` | -- | **Optional.** Per-model context threshold overrides. Keys are canonical model refs in `"provider/model"` format. Values are the percentage of that model's context window that triggers an upgrade search. These take precedence over `defaultContextThresholdPercent`. Unknown keys produce a warning on provider registration. See [Context Threshold Overrides](#context-threshold-overrides). |
137
304
  | `profiles` | `object` | _(required)_ | Map of profile definitions. |
138
305
 
139
306
  ### Profile Definitions
@@ -148,35 +315,6 @@ Each profile defines three **tiers** (`high`, `medium`, `low`). Each tier config
148
315
 
149
316
  **Valid thinking levels** (from least to most reasoning): `off`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`
150
317
 
151
- ### Context Threshold Overrides
152
-
153
- Use `contextThresholdPercentOverrides` to tune the context threshold for specific models. Keys must be canonical model refs in `"provider/model"` format. Values are percentages; lower values cause the router to consider upgrading sooner.
154
-
155
- ```json
156
- {
157
- "contextThresholdPercentOverrides": {
158
- "openrouter/deepseek/deepseek-v4-flash": 60,
159
- "openrouter/deepseek/deepseek-v4-pro": 50,
160
- "openai-codex/gpt-5.5": 60,
161
- "openrouter/google/gemini-3.1-pro-preview": 18
162
- }
163
- }
164
- ```
165
-
166
-
167
-
168
- ### Config Validation
169
-
170
- The config system performs thorough validation on reload/startup and surfaces warnings via the notification system:
171
-
172
- - Validates all profile model refs are in `provider/model` format
173
- - Validates thinking levels against allowed values
174
- - Reports missing/invalid profiles with fallback resolution
175
- - Normalizes `defaultContextThresholdPercent` to positive values only
176
- - Validates `contextThresholdPercentOverrides` keys against known models; unknown keys produce a warning on provider registration
177
-
178
- ---
179
-
180
318
  ## Activating the Router
181
319
 
182
320
  When you define profiles in your config, each profile is registered as a model with the `router` provider — e.g., `router/balanced`, `router/cheap`. These models appear in pi's model list and are available for selection in pi's model switcher, just like any other model.
@@ -233,15 +371,13 @@ Valid pin values: `high`, `medium`, `low`, `clear`.
233
371
 
234
372
 
235
373
 
236
- ### `/router debug <on|off|show|clear>`
374
+ ### `/router debug <on|off>`
237
375
 
238
- Control turn-by-turn routing debug notifications and history. Debug history stores the last 12 routing decisions.
376
+ Control turn-by-turn routing debug notifications.
239
377
 
240
378
  ```bash
241
379
  /router debug on # Enable
242
380
  /router debug off # Disable
243
- /router debug show # Show the last 12 routing decisions
244
- /router debug clear # Clear history
245
381
  /router debug # Toggle
246
382
  ```
247
383
 
@@ -261,229 +397,8 @@ Show a comprehensive help listing of all subcommands.
261
397
 
262
398
  ```bash
263
399
  /router help
264
- /router ?
265
- ```
266
-
267
- ---
268
-
269
- ## How Routing Works
270
-
271
- For every turn, the router executes this ordered pipeline:
272
-
273
- ```
274
- GATE 0: GOOGLE LOCK
275
- - Google thinking tool continuation? → preserve exact model/tier
276
- (skips EVERYTHING below)
277
-
278
- GATE 1: CLASSIFIER GATING (only when classifierModels is configured)
279
- - Manual pin set → use pinned tier, classifier is skipped entirely
280
- - No classifier result yet → default to `medium`
281
- - New user message → run classifier
282
- - Tool-result continuation?
283
- ├─ contCount >= classifierRunOnceAfterToolCount (first crossing)? → run once
284
- ├─ Consecutive failures >= classifierRunAfterToolFailures? → run (crisis)
285
- ├─ contCount % classifierInterval === 0? → run (periodic)
286
- └─ Otherwise → reuse previous decision
287
-
288
- POST-ROUTE CORRECTIONS (always apply)
289
- - Image-aware escalation: upgrade tier if routed model
290
- doesn't support image attachments
291
- - Context threshold enforcement: if context usage exceeds
292
- threshold, re-search for a suitable model in current or
293
- higher tiers (strict pass first, truncation pass as fallback)
294
-
295
- EXECUTION
296
- - Auto-context truncation: trim oldest messages if target
297
- model's window is smaller than the profile's maximum window
298
- - Fallback chain: retry fallback models if primary fails
299
- - Post-turn re-assert: re-select the router model after each turn if it was changed
300
- ```
301
-
302
-
303
-
304
- ### Classifier Gating
305
-
306
- When the router has an LLM classifier configured (`classifierModels`), it doesn't run on every turn. Instead, the classifier is gated by smart triggers that avoid waste while catching real tier mismatches. **The classifier has final say on tier** (post-route corrections like image escalation).
307
-
308
- | Gate | Trigger | Reason |
309
- |---|---------------------------------------------------------------------------------------------|---|
310
- | **Google lock** | Previous model was Google with thinking, and this is a tool-result continuation | Freezes to the same model — any change breaks thought-signature replay |
311
- | **Fresh feedback** | Tool-continuation count first reaches or exceeds `classifierRunOnceAfterToolCount` (default: 3) | Captures the assistant's first response + tool result after a new user message |
312
- | **Crisis** | Consecutive failed tool results (from the tail) ≥ `classifierRunAfterToolFailures` (default: 2) | Model is struggling — reclassify to potentially upgrade tier |
313
- | **Interval** | Tool-continuation count % `classifierInterval` === 0 (default: 10) | Periodic re-check for long-running turns where initial assessment might be stale |
314
-
315
- All counters reset per user turn — each new user message is treated as a fresh task. The crisis gate counts **consecutive** failures from the tail: one successful tool result resets the count to 0, so it only fires when failures are actually piling up.
316
-
317
- When the classifier is skipped, the **previous routing decision** is reused directly. Post-route corrections (image escalation) still apply regardless.
318
-
319
- ### Context Controls
320
-
321
- **Context & Image Requirements** (`defaultContextThresholdPercent`): When the conversation context exceeds this percentage of a model's window, or when images are detected, the router searches for a suitable model in the current or higher tiers. This check happens on every turn, but since context usage usually only grows across turns (unless compaction reduces it), the router will often stay in higher tiers once pushed there.
322
-
323
-
324
-
325
- ### Fallback Chains
326
-
327
- Each tier can define `fallbacks` -- an ordered list of alternative models. If the primary model fails, the router retries each fallback in sequence before surfacing an error.
328
-
329
- ```json
330
- {
331
- "profiles": {
332
- "auto": {
333
- "high": {
334
- "model": "openai/gpt-5.4-pro",
335
- "fallbacks": ["anthropic/claude-3-5-sonnet-20241022", "google/gemini-2.5-pro-latest"]
336
- }
337
- }
338
- }
339
- }
340
- ```
341
-
342
- When a fallback is used, `decision.isFallback` is set to `true` and shown in the status. The tier's configured thinking level (or runtime override) applies to all fallback models -- if a fallback doesn't support the requested level, pi silently clamps it.
343
-
344
- If a model fails during a turn, the router retries it **once** (2 total attempts) before moving to the next fallback in the chain.
345
-
346
- ### Image-Aware Auto-Routing
347
-
348
- When the user attaches an image, the router checks whether the routed model supports image inputs. If not, it searches for another model in the same tier that does. If none are found, it escalates to higher tiers until a suitable model is found.
349
-
350
- > **Note:** The search ensures that the selected model also fits the current context window requirements. To avoid landing on a model that can see images but not the full conversation, make sure your models have adequate context windows and image support configured.
351
-
352
- ### Google Thinking Tool Continuation
353
-
354
- When using Google models with thinking enabled, tool-result continuations require the **same model** to avoid thought-signature replay errors. The router detects this pattern and preserves the exact model/tier for the continuation turn.
355
-
356
- ### Auto-Context Truncation
357
-
358
- The router reports the **largest context window across all models in a profile** (scanning all tiers and their fallbacks for the maximum). When routing to a model with a smaller window, the router trims oldest messages (preserving the system prompt and the most recent message) to fit within the target model's limit.
359
-
360
- Conservative estimation: **4 characters = 1 token**.
361
-
362
- This is a rough last-resort cut, not a replacement for pi's built-in session compaction (`/compact`).
363
-
364
- ### Session & Debugging
365
-
366
- **Persistent State:** Router state persists across agent restarts AND conversation branches via `pi.appendEntry` with a custom `router-state` entry type. Pins, debug mode, debug history, the last routing decision, and the last non-router model are all preserved. State is **branch-safe** -- different conversation branches maintain independent state using `sessionManager.getBranch()`.
367
-
368
- **Status Line:** The router shows its status as a single line in the pi TUI status bar, e.g.:
369
- ```
370
- ⇋ medium -> google/gemini-flash-latest
371
- ```
372
-
373
- The status line may show decision flags in brackets when applicable:
374
- - `[fallback]` — a fallback model was used
375
- - `[context]` — context threshold triggered an upgrade
376
-
377
- **Debug History:** With `/router debug on`, routing decisions and classifier runs are logged with timestamps. View with `/router debug show` to see the routing history:
378
-
379
- ```
380
- Running router classifier — init(≥3), interval(%10) (cont:5) ...
381
400
  ```
382
401
 
383
- The classifier notification shows why the classifier was triggered (`cont` = tool-result continuations since the last user message, `fail` = consecutive recent tool failures).
384
- ```
385
- [10:32:15 AM] high -> openai/gpt-5.4-pro (high) - Classifier: multi-file architecture change across 4 services requires careful trade-off analysis.
386
- [10:33:42 AM] medium -> google/gemini-flash-latest (medium) - Classifier: implementing a well-defined feature with clear acceptance criteria.
387
- [10:34:10 AM] low -> openai/gpt-5.4-nano (low) - Classifier: simple field rename with no behavioral changes.
388
- ```
389
-
390
- ---
391
-
392
- ## Example Configuration
393
-
394
- ```json
395
- {
396
- "debug": true,
397
- "classifierModels": ["openrouter/deepseek/deepseek-v4-flash"],
398
- "classifierModelThinking": "high",
399
- "defaultContextThresholdPercent": 80,
400
- "contextThresholdPercentOverrides": {
401
- "opencode/deepseek-v4-flash-free": 60,
402
- "openrouter/deepseek/deepseek-v4-flash": 60,
403
- "openrouter/deepseek/deepseek-v4-pro": 50,
404
-
405
- "openrouter/google/gemma-4-31b-it": 50,
406
- "openrouter/google/gemini-3-flash-preview": 80,
407
- "openrouter/google/gemini-3.5-flash": 80,
408
- "openrouter/google/gemini-3.1-pro-preview": 18,
409
-
410
- "openrouter/moonshotai/kimi-k2.6": 70,
411
- "openrouter/moonshotai/kimi-k2.7-code": 60,
412
-
413
- "openrouter/minimax/minimax-m3": 50,
414
-
415
- "openrouter/z-ai/glm-5.2": 50,
416
-
417
- "openai-codex/gpt-5.4-mini": 60,
418
- "openai-codex/gpt-5.5": 60,
419
- "openai-codex/gpt-5.6-luna": 65,
420
- "openai-codex/gpt-5.6-terra": 65,
421
- "openai-codex/gpt-5.6-sol": 65,
422
- "openrouter/openai/gpt-5.4-nano": 30,
423
-
424
- "openrouter/anthropic/claude-opus-4.8": 65,
425
-
426
- "openrouter/x-ai/grok-4.5": 50
427
- },
428
- "profiles": {
429
- "cheap": {
430
- "high": {
431
- "model": "openrouter/deepseek/deepseek-v4-flash", "thinking": "xhigh",
432
- "fallbacks": ["openrouter/minimax/minimax-m3"]
433
- },
434
- "medium": {
435
- "model": "openrouter/deepseek/deepseek-v4-flash", "thinking": "high",
436
- "fallbacks": ["openrouter/minimax/minimax-m3"]
437
- },
438
- "low": {
439
- "model": "openrouter/deepseek/deepseek-v4-flash", "thinking": "off",
440
- "fallbacks": ["openrouter/minimax/minimax-m3"]
441
- }
442
- },
443
- "balanced": {
444
- "high": {
445
- "model": "openrouter/z-ai/glm-5.2", "thinking": "xhigh",
446
- "fallbacks": ["openrouter/minimax/minimax-m3"]
447
- },
448
- "medium": {
449
- "model": "openrouter/deepseek/deepseek-v4-flash", "thinking": "xhigh",
450
- "fallbacks": ["openrouter/minimax/minimax-m3"]
451
- },
452
- "low": {
453
- "model": "openrouter/deepseek/deepseek-v4-flash", "thinking": "off",
454
- "fallbacks": ["openrouter/minimax/minimax-m3"]
455
- }
456
- },
457
- "expensive": {
458
- "high": {
459
- "model": "openrouter/z-ai/glm-5.2", "thinking": "max",
460
- "fallbacks": ["openrouter/x-ai/grok-4.5", "openai/gpt-5.6-luna"]
461
- },
462
- "medium": {
463
- "model": "openrouter/z-ai/glm-5.2", "thinking": "max",
464
- "fallbacks": ["openai/gpt-5.6-luna", "openrouter/x-ai/grok-4.5"]
465
- },
466
- "low": {
467
- "model": "openrouter/minimax/minimax-m3", "thinking": "off",
468
- "fallbacks": ["openrouter/z-ai/glm-5.2"]
469
- }
470
- },
471
- "gpt-5.6": {
472
- "high": { "model": "openai/gpt-5.6-sol", "thinking": "medium" },
473
- "medium": { "model": "openai/gpt-5.6-terra", "thinking": "high" },
474
- "low": { "model": "openai/gpt-5.6-luna", "thinking": "high" }
475
- },
476
- "opus-4.8": {
477
- "high": { "model": "openrouter/anthropic/claude-opus-4.8", "thinking": "medium" },
478
- "medium": { "model": "openrouter/anthropic/claude-opus-4.8", "thinking": "low" },
479
- "low": { "model": "openrouter/anthropic/claude-opus-4.8", "thinking": "off" }
480
- }
481
- }
482
- }
483
- ```
484
-
485
- ---
486
-
487
402
  ## Architecture
488
403
 
489
404
  See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for a detailed architectural deep dive, including the decision flow, module responsibilities, state persistence, and fallback chain design.
@@ -49,6 +49,36 @@ session_start / model_select / turn_end (index.ts)
49
49
  they do not perform routing themselves.
50
50
  ```
51
51
 
52
+ ## Config Merging Order
53
+
54
+ Config is loaded from two locations and **merged**:
55
+
56
+ | Location | Scope | Path |
57
+ |---|---|---|
58
+ | Global | User-wide | `~/.pi/agent/model-router.json` |
59
+ | Project | Per-project | `.pi/model-router.json` |
60
+
61
+ The merge order is: **Base defaults ← Global config ← Project config**.
62
+
63
+ Project config values override global values, which override built-in defaults. Profiles are merged **deeply** — if you define only a `high` tier override for a profile in your project config, the `medium` and `low` tiers are inherited from the global config.
64
+
65
+ **When no config file exists**, the router loads with an empty profile list and no active models.
66
+
67
+ ### Why Two Config Locations?
68
+
69
+ - **Global config** (`~/.pi/agent/model-router.json`) lets you set cross-project defaults — profile definitions, classifier preferences, and context thresholds you want everywhere.
70
+ - **Project config** (`.pi/model-router.json`) lets you override per-project — different profile choices, tighter context budgets, or debug flags for development.
71
+
72
+ This is handled by `extensions/config.ts`, which normalizes and validates the merged result.
73
+
74
+ ### Config Validation
75
+
76
+ On reload/startup, the config system validates:
77
+ - Profile model refs are in `provider/model` format
78
+ - Thinking levels are valid (`off` → `max`)
79
+ - `defaultContextThresholdPercent` is positive
80
+ - `contextThresholdPercentOverrides` keys match known models (unknown keys produce a warning on provider registration)
81
+
52
82
  ## State & Persistence
53
83
 
54
84
  Router state is persisted using `pi.appendEntry` with a custom type `router-state`. This allows the router to:
@@ -66,10 +96,8 @@ Router state is persisted using `pi.appendEntry` with a custom type `router-stat
66
96
  | `debugEnabled` | `boolean` | Debug mode state |
67
97
  | `lastDecision` | `RoutingDecision` | Most recent routing decision |
68
98
  | `lastNonRouterModel` | `string` | Last model used before switching to router |
69
- | `debugHistory` | `RoutingDecision[]` | Recent routing decisions |
70
-
71
- > **Branch safety**: Because state is saved via `pi.appendEntry`, each conversation branch gets its own independent state. Switching branches restores the pins and history that were active on that branch.
99
+ > **Branch safety**: Because state is saved via `pi.appendEntry`, each conversation branch gets its own independent state. Switching branches restores the pins and state that were active on that branch.
72
100
 
73
- ### Debug History
101
+ ### Debug Mode
74
102
 
75
- The debug history stores the last 12 routing decisions. When debug mode is enabled (`/router debug on`), each decision is appended to `debugHistory` and `/router debug show` prints the full history.
103
+ Use `/router debug on` to enable debug notifications for routing decisions and classifier runs.
@@ -16,7 +16,6 @@ import {
16
16
  } from './config';
17
17
  import {
18
18
  formatPinSummary,
19
- formatDecision,
20
19
  } from './ui';
21
20
 
22
21
  export const registerCommands = (
@@ -29,7 +28,6 @@ export const registerCommands = (
29
28
  readonly lastDecision: RoutingDecision | undefined;
30
29
  lastNonRouterModel: string | undefined;
31
30
  debugEnabled: boolean;
32
- readonly debugHistory: RoutingDecision[];
33
31
  readonly lastConfigWarnings: string[];
34
32
  },
35
33
  actions: {
@@ -53,7 +51,7 @@ export const registerCommands = (
53
51
  { name: 'pin', desc: 'Pin routing for a profile to a tier, or clear' },
54
52
  { name: 'disable', desc: 'Disable the router and restore last model' },
55
53
 
56
- { name: 'debug', desc: 'Toggle or clear router debug history' },
54
+ { name: 'debug', desc: 'Toggle router debug notifications on/off' },
57
55
  { name: 'reload', desc: 'Reload the model router configuration' },
58
56
  { name: 'help', desc: 'Show usage help for subcommands' },
59
57
  ];
@@ -120,7 +118,6 @@ export const registerCommands = (
120
118
  `Available profiles: ${profileNames(state.currentConfig).join(', ')}`,
121
119
  `Last non-router model: ${state.lastNonRouterModel ?? 'none'}`,
122
120
  `Debug: ${state.debugEnabled ? 'on' : 'off'}`,
123
- `Debug history: ${state.debugHistory.length} decisions`,
124
121
  ];
125
122
  if (state.lastDecision) {
126
123
  lines.push(
@@ -261,22 +258,13 @@ export const registerCommands = (
261
258
 
262
259
  const handleDebug = async (args: string[], ctx: ExtensionContext) => {
263
260
  if (args.length > 1) {
264
- ctx.ui.notify('Usage: /router debug <on|off|show|clear>', 'error');
261
+ ctx.ui.notify('Usage: /router debug <on|off>', 'error');
265
262
  return;
266
263
  }
267
264
  const cmd = args[0]?.toLowerCase();
268
265
  if (cmd === 'on') state.debugEnabled = true;
269
266
  else if (cmd === 'off') state.debugEnabled = false;
270
- else if (cmd === 'clear') state.debugHistory.length = 0;
271
- else if (cmd === 'show') {
272
- if (state.debugHistory.length === 0) {
273
- ctx.ui.notify('No recent routing decisions.', 'info');
274
- } else {
275
- const history = state.debugHistory.map(formatDecision).join('\n');
276
- ctx.ui.notify(`Recent Routing Decisions:\n${history}`, 'info');
277
- }
278
- return;
279
- } else {
267
+ else {
280
268
  state.debugEnabled = !state.debugEnabled;
281
269
  }
282
270
  actions.persistState();
@@ -363,7 +351,7 @@ export const registerCommands = (
363
351
  }
364
352
  case 'debug': {
365
353
  const debugPrefix = subArgs[0] ?? '';
366
- const items = ['on', 'off', 'toggle', 'clear', 'show']
354
+ const items = ['on', 'off', 'toggle']
367
355
  .filter((v) => v.startsWith(debugPrefix))
368
356
  .map((v) => ({
369
357
  value: `debug ${v}`,
@@ -401,7 +389,6 @@ export const registerCommands = (
401
389
  await handleStatus(subArgs, ctx);
402
390
  break;
403
391
  case 'help':
404
- case '?':
405
392
  if (subArgs.length > 0) {
406
393
  ctx.ui.notify('Usage: /router help (no arguments)', 'error');
407
394
  return;
@@ -413,9 +400,9 @@ export const registerCommands = (
413
400
  ' profile [name] Switch to a profile (enables router if off). Lists available if no name.',
414
401
  ' pin [profile] <tier|clear> Pin to a tier (high|medium|low) or clear the pin.',
415
402
  ' disable Disable the router and restore the last used non-router model.',
416
- ' debug <on|off|show|clear> Control routing debug logging to notifications and history.',
403
+ ' debug <on|off> Enable/disable routing debug notifications.',
417
404
  ' reload Hot-reload the configuration JSON from .pi/model-router.json.',
418
- ' help, ? Show this help message.',
405
+ ' help Show this help message.',
419
406
  ].join('\n'),
420
407
  'info',
421
408
  );
@@ -28,7 +28,6 @@ const routerExtension = (pi: ExtensionAPI) => {
28
28
  let selectedProfile: string | undefined;
29
29
  let lastLoadedModelKeys = '';
30
30
  let pinnedTierByProfile: RouterPinByProfile = {};
31
- let debugHistory: RoutingDecision[] = [];
32
31
  let lastNonRouterModel: string | undefined;
33
32
  let lastExtensionContext: ExtensionContext | undefined;
34
33
  let lastConfigWarnings: string[] = [];
@@ -47,18 +46,12 @@ const routerExtension = (pi: ExtensionAPI) => {
47
46
  }
48
47
  };
49
48
 
50
- const MAX_DEBUG_HISTORY = 12;
51
- const recordDebugDecision = (decision: RoutingDecision) => {
52
- debugHistory = [...debugHistory, decision].slice(-MAX_DEBUG_HISTORY);
53
- };
54
-
55
49
  const persistState = () => {
56
50
  const state = buildPersistedState(
57
51
  routerEnabled,
58
52
  selectedProfile,
59
53
  pinnedTierByProfile,
60
54
  debugEnabled,
61
- debugHistory,
62
55
  lastDecision,
63
56
  lastNonRouterModel,
64
57
  );
@@ -68,10 +61,6 @@ const routerExtension = (pi: ExtensionAPI) => {
68
61
  lastDecision: state.lastDecision
69
62
  ? { ...state.lastDecision, timestamp: 0 }
70
63
  : undefined,
71
- debugHistory: state.debugHistory?.map((decision) => ({
72
- ...decision,
73
- timestamp: 0,
74
- })),
75
64
  });
76
65
  if (snapshot === lastPersistedSnapshot) {
77
66
  return;
@@ -210,7 +199,6 @@ const routerExtension = (pi: ExtensionAPI) => {
210
199
  },
211
200
  {
212
201
  persistState,
213
- recordDebugDecision,
214
202
  updateStatus: actions.updateStatus,
215
203
  },
216
204
  );
@@ -234,7 +222,6 @@ const routerExtension = (pi: ExtensionAPI) => {
234
222
  ctx.model?.provider === 'router' ? ctx.model.id : selectedProfile,
235
223
  );
236
224
  pinnedTierByProfile = {};
237
- debugHistory = [];
238
225
  lastNonRouterModel =
239
226
  ctx.model && ctx.model.provider !== 'router'
240
227
  ? `${ctx.model.provider}/${ctx.model.id}`
@@ -267,9 +254,6 @@ const routerExtension = (pi: ExtensionAPI) => {
267
254
  pinnedTierByProfile[selectedProfile] = savedState.pinTier;
268
255
  }
269
256
  debugEnabled = savedState.debugEnabled ?? debugEnabled;
270
- debugHistory = savedState.debugHistory
271
- ? [...savedState.debugHistory].slice(-MAX_DEBUG_HISTORY)
272
- : [];
273
257
  lastNonRouterModel = savedState.lastNonRouterModel ?? lastNonRouterModel;
274
258
  }
275
259
 
@@ -335,9 +319,7 @@ const routerExtension = (pi: ExtensionAPI) => {
335
319
  set debugEnabled(v) {
336
320
  debugEnabled = v;
337
321
  },
338
- get debugHistory() {
339
- return debugHistory;
340
- },
322
+
341
323
  get lastConfigWarnings() {
342
324
  return lastConfigWarnings;
343
325
  },
@@ -85,7 +85,6 @@ export const registerRouterProvider = (
85
85
  },
86
86
  actions: {
87
87
  persistState: () => void;
88
- recordDebugDecision: (decision: RoutingDecision) => void;
89
88
  updateStatus: (ctx: ExtensionContext) => void;
90
89
  },
91
90
  ) => {
@@ -413,8 +412,6 @@ export const registerRouterProvider = (
413
412
  }
414
413
  }
415
414
 
416
- actions.recordDebugDecision(decision);
417
-
418
415
  if (!success) {
419
416
  const errorMsg = `Failed to delegate to any model in the chain.${failureReasons.length > 0 ? ' Reasons: ' + failureReasons.filter(Boolean).join('; ') + '.' : ''}`;
420
417
  const combinedError = lastError ? new Error(`${(lastError as Error).message} — ${errorMsg}`) : new Error(errorMsg);
@@ -23,7 +23,6 @@ export const buildPersistedState = (
23
23
  selectedProfile: string | undefined,
24
24
  pinnedTierByProfile: RouterPinByProfile,
25
25
  debugEnabled: boolean,
26
- debugHistory: RoutingDecision[],
27
26
  lastDecision?: RoutingDecision,
28
27
  lastNonRouterModel?: string,
29
28
  ): RouterPersistedState => ({
@@ -32,7 +31,6 @@ export const buildPersistedState = (
32
31
  pinTier: selectedProfile ? pinnedTierByProfile[selectedProfile] : undefined,
33
32
  pinByProfile: { ...pinnedTierByProfile },
34
33
  debugEnabled,
35
- debugHistory,
36
34
  lastDecision,
37
35
  lastNonRouterModel,
38
36
  timestamp: Date.now(),
@@ -51,7 +51,6 @@ export interface RouterPersistedState {
51
51
  pinTier?: RouterTier;
52
52
  pinByProfile?: RouterPinByProfile;
53
53
  debugEnabled?: boolean;
54
- debugHistory?: RoutingDecision[];
55
54
  lastDecision?: RoutingDecision;
56
55
  lastNonRouterModel?: string;
57
56
  timestamp: number;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@kdejaeger/pi-model-router",
3
- "version": "0.4.1",
3
+ "version": "0.4.2",
4
4
  "type": "module",
5
5
  "description": "Intelligent per-turn model router extension for the pi coding agent",
6
6
  "keywords": [
@@ -33,7 +33,8 @@
33
33
  "pi": {
34
34
  "extensions": [
35
35
  "./extensions/index.ts"
36
- ]
36
+ ],
37
+ "image": "https://raw.githubusercontent.com/kdejaeger/pi-model-router/main/docs/icon.svg"
37
38
  },
38
39
  "scripts": {
39
40
  "tsc": "tsc --noEmit",
@@ -1,48 +0,0 @@
1
- {
2
- "debug": false,
3
- "classifierModels": ["google/gemini-flash-lite-latest", "openrouter/deepseek/deepseek-v4-flash"],
4
- "classifierModelThinking": "off",
5
- "classifierRunOnceAfterToolCount": 3,
6
- "classifierRunAfterToolFailures": 2,
7
- "classifierInterval": 10,
8
- "defaultContextThresholdPercent": 70,
9
- "contextThresholdPercentOverrides": {
10
- "openrouter/deepseek/deepseek-v4-flash": 60,
11
- "openrouter/deepseek/deepseek-v4-pro": 50,
12
- "openrouter/google/gemma-4-31b-it": 50,
13
- "openrouter/google/gemini-3-flash-preview": 80,
14
- "openrouter/openai/gpt-5.4-nano": 30
15
- },
16
- "profiles": {
17
- "auto": {
18
- "high": {
19
- "model": "openai/gpt-5.4-pro",
20
- "thinking": "high",
21
- "fallbacks": ["anthropic/claude-3-5-sonnet-20241022"]
22
- },
23
- "medium": { "model": "google/gemini-flash-latest", "thinking": "medium" },
24
- "low": { "model": "openai/gpt-5.4-nano", "thinking": "low" }
25
- },
26
- "cheap": {
27
- "high": { "model": "google/gemini-flash-latest", "thinking": "low" },
28
- "medium": { "model": "openai/gpt-5.4-nano", "thinking": "off" },
29
- "low": { "model": "google/gemini-flash-lite-latest", "thinking": "off" }
30
- },
31
- "deep": {
32
- "high": { "model": "openai/o1-preview", "thinking": "xhigh" },
33
- "medium": { "model": "openai/gpt-5.4-pro", "thinking": "medium" },
34
- "low": { "model": "google/gemini-flash-latest", "thinking": "low" }
35
- },
36
- "anthropic": {
37
- "high": {
38
- "model": "anthropic/claude-3-5-sonnet-20241022",
39
- "thinking": "high"
40
- },
41
- "medium": {
42
- "model": "anthropic/claude-3-5-sonnet-20241022",
43
- "thinking": "medium"
44
- },
45
- "low": { "model": "anthropic/claude-3-haiku-20240307", "thinking": "low" }
46
- }
47
- }
48
- }