pi-auto-route 0.0.0-stage → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +34 -0
- package/LICENSE +21 -0
- package/README.md +107 -2
- package/auto-route.ts +639 -0
- package/package.json +36 -5
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 1.3.0
|
|
4
|
+
|
|
5
|
+
- Bilingual heuristic: every signal regex gained English alternations; the French patterns stay. Paths, code blocks, message length and the terminal-paste penalty were already language-neutral.
|
|
6
|
+
- Hybrid classifier mode: when a classifier model is available — auto-detected across all providers (TypeSafe Jev, OpenRouter decision models, Cloudflare Clef, GitHub Copilot decision models, OpenCode `jev-1.13-free`) — the router asks it one multiple-choice question (light / standard / complex / heavy) and maps the answer straight to the four tiers with their thinking levels. Only the user message is sent, truncated to 16k chars; never the system prompt, history or tool results. Free classifiers are preferred, then the cheaper ones; one classification per new turn, never on retries or continuations. The status line now shows the source: `local` or the classifier id (e.g. `jev-1.13-free`).
|
|
7
|
+
- The zero-requirements install keeps working: confidence below 0.50, a 4-second timeout, any classifier error, or no classifier at all silently falls back to the local bilingual heuristic — no key, no config, no network.
|
|
8
|
+
|
|
9
|
+
## 1.2.0
|
|
10
|
+
|
|
11
|
+
- Economy and quality fixed together, measured on 926 real turns:
|
|
12
|
+
- turns routed to an expensive tier: 104 → 85;
|
|
13
|
+
- heaviest-quartile turns sent to the cheapest tier: 82/232 → 53/232;
|
|
14
|
+
- Spearman correlation with real difficulty: +0.288 → +0.339.
|
|
15
|
+
- The `ACTION` verb list now recognises the spoken French imperatives (`fais`, `faire`, `regarde`, `prend`, `donne`, `met`, `lance`, `écris`, `essaie`, `montre`). They were as frequent as the formal verbs in real prompts but matched nothing, so "fais le portage français" — 49 tool calls, 82k output tokens — scored 0.10 and landed in `eco`.
|
|
16
|
+
- Upper thresholds raised from 0.50/0.70 to 0.60/0.75: a path plus one verb is a routine request and no longer reaches `strong` ("corrige le bug dans src/foo.ts" used to).
|
|
17
|
+
- Tests: 20 cases (the threshold table, the spoken imperatives, and a regression guard on the path-plus-verb over-provisioning).
|
|
18
|
+
|
|
19
|
+
## 1.1.0
|
|
20
|
+
|
|
21
|
+
- Provider-agnostic: the failover now recognizes provider-wide limits from any provider (Anthropic rate limits, Google `RESOURCE_EXHAUSTED`, OpenAI `insufficient_quota`, OpenCode usage limits), not only OpenCode-specific errors.
|
|
22
|
+
- Transient rate limits keep their original message text; only errors pi refuses to retry are rewritten into a retryable 429.
|
|
23
|
+
- Models without pricing data are no longer dropped from the catalog; they rank at a mid cost.
|
|
24
|
+
- The denylist path honors `PI_CODING_AGENT_DIR` instead of a hardcoded home directory.
|
|
25
|
+
- Packaged as a pi package: install with `pi install npm:pi-auto-route`.
|
|
26
|
+
- Tests: 18 cases covering the heuristic calibration and the failover lifecycle.
|
|
27
|
+
|
|
28
|
+
## 1.0.0
|
|
29
|
+
|
|
30
|
+
- Initial release.
|
|
31
|
+
- `auto` virtual model routing every turn by a locally calibrated difficulty heuristic (0.30 / 0.50 / 0.70 thresholds, 880 real turns).
|
|
32
|
+
- Four tiers (eco, standard, strong, frontier) with a thinking-level cap per tier.
|
|
33
|
+
- One model switch per turn: the starting model plans and makes the first `edit`/`write`.
|
|
34
|
+
- Automatic failover on timeouts and overloads, provider-wide avoidance on quota 429s, persistent denylist for permanent errors (400/404/410).
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 rzph
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,108 @@
|
|
|
1
|
-
#
|
|
1
|
+
# auto-route
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
A virtual model for [pi](https://github.com/earendil-works/pi-coding-agent) that sends every request to the right physical model based on how hard the task looks, to cut token spend without losing quality.
|
|
4
|
+
|
|
5
|
+

|
|
6
|
+
|
|
7
|
+
**pi-specific.** This extension uses pi's extension API (`pi.registerVirtualModel`, `ModelRegistry`); it does not port as-is to Claude Code, Codex or other harnesses. See [Portability](#portability).
|
|
8
|
+
|
|
9
|
+
## Install
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
pi install npm:pi-auto-route
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Then in pi: `/reload`, and pick **auto → auto** in the model selector (`/model`). It shows up as "Auto (routing + thinking + failover)".
|
|
16
|
+
|
|
17
|
+
- update: `pi update --extensions`
|
|
18
|
+
- remove: `pi remove npm:pi-auto-route`
|
|
19
|
+
- without npm: `pi install git:github.com/rzph/pi-auto-route`, or copy `auto-route.ts` into `~/.pi/agent/extensions/` by hand
|
|
20
|
+
|
|
21
|
+
Selecting `auto` starts at thinking `low`. That level is a **ceiling**, not a target: raise it with `shift+tab` when you want the router to think harder.
|
|
22
|
+
|
|
23
|
+
## Usage
|
|
24
|
+
|
|
25
|
+
Once selected, you do nothing else. On every turn the router picks a model on its own and shows one discreet status line: `⚡ provider/model · thinking · local` — the last tag is the source of the difficulty call, `local` or the classifier id (e.g. `jev-1.13-free`).
|
|
26
|
+
|
|
27
|
+
Three things worth knowing up front:
|
|
28
|
+
|
|
29
|
+
- **Language.** The local heuristic reads both French and English: every signal regex carries French and English alternations. When a classifier model is available on any of your providers, routing uses it instead — classifiers are language-agnostic. The regex path stays as the zero-dependency fallback.
|
|
30
|
+
- **One switch per turn.** The first model plans and makes the first `edit`/`write`; the rest of the turn runs on the cheapest tier. The prompt cache therefore breaks only once per turn, which is why the trade-off favours turns with several tool calls.
|
|
31
|
+
- **You stay in control.** `/model` to any real model takes over immediately; switch back to `auto` on the next turn to resume routing. To force one model for a single turn, switch away and back.
|
|
32
|
+
|
|
33
|
+
## Requirements
|
|
34
|
+
|
|
35
|
+
- pi with at least one provider that has valid credentials. Nothing is hardcoded to a specific provider: the candidate catalog, the tiers and the failover are all built from your own model registry at runtime.
|
|
36
|
+
- No API key of its own: without a classifier, the difficulty heuristic runs entirely locally, with no network calls.
|
|
37
|
+
- A classifier model from any provider is optional and auto-detected; without one the local heuristic is used, so no extra credential is ever required.
|
|
38
|
+
- Only to run the test suite: Node.js 22.6 or newer (`node --test *.test.ts`).
|
|
39
|
+
|
|
40
|
+
## How it works
|
|
41
|
+
|
|
42
|
+
On each turn, in order:
|
|
43
|
+
|
|
44
|
+
1. **Score the message.** The router tries a classifier first. Any classifier model in your registry is auto-detected across all providers (TypeSafe Jev, OpenRouter decision models, Cloudflare Clef, GitHub Copilot decision models, OpenCode `jev-1.13-free`…); free ones are preferred, then the cheaper ones. The router asks it one multiple-choice question on your last message — light, standard, complex or heavy — sending only that message, truncated to 16k characters: never the system prompt, the history or tool results. One classification runs per new turn, never on retries or continuations. Confidence below 0.50, a 4-second timeout, any classifier error, or no classifier at all falls back silently to the local heuristic. That heuristic is now bilingual: regex patterns read your last message in French and English and produce a score from 0 to 1. Signals of real work: a path or repo, action verbs, explicit intent, a conceptual question, code blocks, length. A terminal paste (three or more `$`/`sudo` lines) is penalised — a wall of shell output is not a hard task.
|
|
45
|
+
2. **Pick a tier.** Your model catalog is sorted by output cost, used as a proxy for capacity (ties broken by the larger context window), and split into four tiers:
|
|
46
|
+
- score ≥ 0.75 — heavy work, several independent signals → most capable tier, thinking `xhigh`
|
|
47
|
+
- score ≥ 0.6 → strong tier, thinking `high`
|
|
48
|
+
- score ≥ 0.3 → standard tier, thinking `medium`
|
|
49
|
+
- score < 0.3 — short follow-ups, quick questions → lightest tier, thinking `low`
|
|
50
|
+
|
|
51
|
+
The two upper thresholds are deliberately high: a path plus one verb is a routine request, not a hard one, and should not buy an expensive model. The thresholds apply to the local score only; a classifier answer (light / standard / complex / heavy) maps straight onto the same four tiers with their thinking levels.
|
|
52
|
+
3. **Hand off after the first edit.** The chosen model plans and makes the first `edit`/`write`; the turn continues on the lightest tier (see Usage).
|
|
53
|
+
4. **Compaction and invisible summaries** always go to the lightest model — the token leak nobody sees.
|
|
54
|
+
5. **Failover instead of a crash.** A timeout or overload (429/5xx) moves to the next model of the ladder. A **429 provider-quota** error — which pi treats as non-retryable, so the turn would die — is rewritten into a retryable one, and the router then avoids the **whole provider**: the turn continues on another provider instead. Permanent errors (400/404/410, and messages like "trains on request data" or "unsupported region") ban the model persistently. See `failover.test.ts`.
|
|
55
|
+
|
|
56
|
+
### Excluded from the candidates
|
|
57
|
+
|
|
58
|
+
- **free** models (tight quotas, constant timeouts);
|
|
59
|
+
- **non-reasoning** models;
|
|
60
|
+
- any model whose provider has no valid credentials;
|
|
61
|
+
- **`contributor`** models (OpenCode endpoints that train on your requests);
|
|
62
|
+
- retired or deprecated generations: `gemini-2.5-*` (Google 404), `mimo-v2.5` (OpenCode 410);
|
|
63
|
+
- specialized non-code models (`computer-use`, `robotics`, `deep-research`);
|
|
64
|
+
- anything in pi's agent dir as `auto-route-denylist.json` (`$PI_CODING_AGENT_DIR` if set, else `~/.pi/agent/`) — models banned automatically and **persistently** as soon as they return a permanent error. Delete an entry from the file to restore a model.
|
|
65
|
+
|
|
66
|
+
A model is excluded as free only when its pricing says 0; missing pricing means unknown and stays a candidate, ranked at a mid cost. An empty tier falls back to the next one; if everything fails, you get an explicit error.
|
|
67
|
+
|
|
68
|
+
## Customize
|
|
69
|
+
|
|
70
|
+
Everything is set at the top of `auto-route.ts`:
|
|
71
|
+
|
|
72
|
+
- `difficultyScore()`: the heuristic (signals plus length thresholds) — each signal is one commented regex; recalibrate and run `node --test *.test.ts` after any change (`heuristics.test.ts` for the score, `failover.test.ts` for the 429 handling and the default thinking);
|
|
73
|
+
- `tierAndThinking()`: the score → tier + thinking mapping;
|
|
74
|
+
- the filter in `candidateModels()` if you want free or non-reasoning models back.
|
|
75
|
+
|
|
76
|
+
## Calibration details
|
|
77
|
+
|
|
78
|
+
The heuristic is a hand-written rule, not a trained model; the numbers below only measure it, and they measure the local fallback path. They come from 926 real turns taken from local session logs (see `heuristics.test.ts`), scored against a real-difficulty proxy (the combined rank of a turn's tool calls and output tokens).
|
|
79
|
+
|
|
80
|
+
Current rule against the previous one:
|
|
81
|
+
|
|
82
|
+
- heaviest-quartile turns sent to the `eco` tier: **82/232 → 53/232** — fewer heavy turns left on the cheapest model;
|
|
83
|
+
- turns routed to an expensive tier: **104 → 85**;
|
|
84
|
+
- light turns that still reached an expensive tier: **52 → 44**;
|
|
85
|
+
- Spearman correlation between score and real difficulty: **+0.288 → +0.339**.
|
|
86
|
+
|
|
87
|
+
Both directions improved at once because two separate faults were fixed together: the verb list had missed the spoken imperatives (`fais`, `regarde`, `prend`, `donne`, `met`, `lance`), which sent turns like "fais le portage français" — 49 tool calls, 82k output tokens — to `eco`; and the upper thresholds let a path plus one verb reach `strong`. The first version of the rule scored +0.16 with 195/220 of the heaviest turns in `eco`.
|
|
88
|
+
|
|
89
|
+
Read the correlation honestly: the score is an **asymmetric umbrella** — it shields heavy messages from the `eco` tier and at worst bumps a simple one step up. It is not a precise classifier of difficulty, and fine-grained prediction should not be expected from it. Some heavy turns open with a single word ("tous") that no text heuristic can recognise from the message alone; those stay in `eco`. When a classifier model is available, the classifier path is the more precise one and needs no calibration.
|
|
90
|
+
|
|
91
|
+
## Portability
|
|
92
|
+
|
|
93
|
+
- **pi**: supported (this extension).
|
|
94
|
+
- **Claude Code**: not portable; close ideas: `jev-router` (gargpratyush), "model-router" skills.
|
|
95
|
+
- **Codex**: not portable; `jev-codex-router` (0xNatoshi): model + thinking per turn.
|
|
96
|
+
- **Hermes**: not portable; `hermes-jev-skills` (kerpopule).
|
|
97
|
+
|
|
98
|
+
The idea (plan strong, execute cheap, one switch per turn, compaction on a light model) ports everywhere; this implementation is tied to pi. Want a port for another harness? Open an issue.
|
|
99
|
+
|
|
100
|
+
## Key concepts
|
|
101
|
+
|
|
102
|
+
- **Prompt cache**: the router switches *once* per turn, after planning — every cross-provider switch invalidates the cache, so the trade-off favors turns with several tool calls.
|
|
103
|
+
- **Route state**: stored on the session branch by pi; it follows the session tree and survives compaction.
|
|
104
|
+
- **Capacity proxy**: output cost ranks models in the absence of per-task benchmarks. Home-grown benchmark data (reviewer catch rate, say) is a natural extension.
|
|
105
|
+
|
|
106
|
+
## License
|
|
107
|
+
|
|
108
|
+
MIT — see [LICENSE](LICENSE).
|
package/auto-route.ts
ADDED
|
@@ -0,0 +1,639 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* auto-route — virtual model `auto/auto`: full automatic routing.
|
|
3
|
+
*
|
|
4
|
+
* /model → "auto":
|
|
5
|
+
* - candidates = paid reasoning chat models with valid credentials, excluding
|
|
6
|
+
* free models, training models (`contributor`) and retired generations
|
|
7
|
+
* (`gemini-2.5-*`);
|
|
8
|
+
* - capacity tiers built at runtime (output cost as a proxy);
|
|
9
|
+
* - difficulty of each NEW turn: ONE classification when a classifier model is
|
|
10
|
+
* available (auto-detected from any provider with working credentials, free
|
|
11
|
+
* ones first); otherwise — or on any failure (no classifier, low
|
|
12
|
+
* confidence, timeout, error) — a local bilingual (FR + EN) heuristic scores
|
|
13
|
+
* the message, as a silent fallback;
|
|
14
|
+
* - difficulty → tier + thinking: heuristic thresholds and classifier answers
|
|
15
|
+
* map onto the same four tiers;
|
|
16
|
+
* - the thinking level chosen in pi (`shift+tab`) acts as a CEILING;
|
|
17
|
+
* - failover on retry (429/5xx) plus a `message_end` listener that bans
|
|
18
|
+
* PERMANENTLY models returning 400/404 (pi does not retry non-429 4xx);
|
|
19
|
+
* 429 "provider quota" errors, which pi treats as non-retryable, are
|
|
20
|
+
* rewritten into retryable 429s so the same failover (cross-provider)
|
|
21
|
+
* runs instead of letting the turn die;
|
|
22
|
+
* - selecting `auto` starts at thinking `low` by default (ceiling, raise it
|
|
23
|
+
* later with shift+tab);
|
|
24
|
+
* - ONE discreet status line (`⚡ model · thinking · source`);
|
|
25
|
+
* - one model switch per turn (prompt cache preserved);
|
|
26
|
+
* - compaction/summaries → the lightest model.
|
|
27
|
+
*
|
|
28
|
+
* Usage: pi -e ~/.pi/agent/extensions/auto-route.ts then /model → auto
|
|
29
|
+
*/
|
|
30
|
+
|
|
31
|
+
import type { AssistantMessage, Message, Model } from "@earendil-works/pi-ai";
|
|
32
|
+
import type {
|
|
33
|
+
ExtensionAPI,
|
|
34
|
+
ExtensionContext,
|
|
35
|
+
ModelRoute,
|
|
36
|
+
ModelRouteRequest,
|
|
37
|
+
} from "@earendil-works/pi-coding-agent";
|
|
38
|
+
|
|
39
|
+
import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
|
40
|
+
import { homedir } from "node:os";
|
|
41
|
+
import { dirname, join } from "node:path";
|
|
42
|
+
|
|
43
|
+
const EDIT_TOOLS = new Set(["edit", "write"]);
|
|
44
|
+
|
|
45
|
+
/** Output cost assumed for models without pricing data: mid-tier, neither free nor frontier. */
|
|
46
|
+
const UNKNOWN_OUTPUT_COST = 1;
|
|
47
|
+
|
|
48
|
+
/**
|
|
49
|
+
* ID/name patterns excluded up front:
|
|
50
|
+
* - `free`: tight quotas / constant timeouts;
|
|
51
|
+
* - `contributor`: OpenCode endpoints that train on your requests;
|
|
52
|
+
* - `gemini-2.5-`: retired by Google for new accounts (404);
|
|
53
|
+
* - `mimo-v2.5`: deprecated on OpenCode Go (410 ModelDeprecated);
|
|
54
|
+
* - `computer-use` / `robotics` / `deep-research`: specialized non-code models.
|
|
55
|
+
*/
|
|
56
|
+
const EXCLUDED_MODEL_RE =
|
|
57
|
+
/free|contributor|gemini-2\.5-|mimo-v2\.5\b|computer-use|robotics|deep-research/i;
|
|
58
|
+
|
|
59
|
+
/**
|
|
60
|
+
* pi's agent config dir: honors PI_CODING_AGENT_DIR (its own env var, tilde
|
|
61
|
+
* expanded), else falls back to ~/.pi/agent.
|
|
62
|
+
*/
|
|
63
|
+
function agentDir(): string {
|
|
64
|
+
const envDir = process.env.PI_CODING_AGENT_DIR?.trim();
|
|
65
|
+
if (envDir) return envDir.startsWith("~") ? join(homedir(), envDir.slice(1)) : envDir;
|
|
66
|
+
return join(homedir(), ".pi", "agent");
|
|
67
|
+
}
|
|
68
|
+
/** File of permanently banned models (400/404/410 discovered on the fly). */
|
|
69
|
+
const DENYLIST_FILE = join(agentDir(), "auto-route-denylist.json");
|
|
70
|
+
|
|
71
|
+
type Thinking = "off" | "low" | "medium" | "high" | "xhigh";
|
|
72
|
+
type TierName = "frontier" | "strong" | "standard" | "eco";
|
|
73
|
+
const THINKING_ORDER: Thinking[] = ["off", "low", "medium", "high", "xhigh"];
|
|
74
|
+
|
|
75
|
+
interface RouteState {
|
|
76
|
+
modelId: string;
|
|
77
|
+
providerId: string;
|
|
78
|
+
thinking: Thinking;
|
|
79
|
+
/** provider/id entries avoided for THIS turn (transient failures). */
|
|
80
|
+
avoid: string[];
|
|
81
|
+
/** Where the tier decision came from: "local" or the classifier's key. */
|
|
82
|
+
source?: string;
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
function modelKey(m: { provider: string; id: string }): string {
|
|
86
|
+
return `${m.provider}/${m.id}`;
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
function sameState(a: RouteState | undefined, b: RouteState): boolean {
|
|
90
|
+
if (!a) return false;
|
|
91
|
+
return (
|
|
92
|
+
a.modelId === b.modelId &&
|
|
93
|
+
a.providerId === b.providerId &&
|
|
94
|
+
a.thinking === b.thinking &&
|
|
95
|
+
a.avoid.length === b.avoid.length &&
|
|
96
|
+
a.avoid.every((v, i) => v === b.avoid[i]) &&
|
|
97
|
+
a.source === b.source
|
|
98
|
+
);
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
// Persistent denylist
|
|
102
|
+
|
|
103
|
+
function loadDenylist(): Set<string> {
|
|
104
|
+
try {
|
|
105
|
+
const raw = JSON.parse(readFileSync(DENYLIST_FILE, "utf8")) as string[];
|
|
106
|
+
return new Set(Array.isArray(raw) ? raw : []);
|
|
107
|
+
} catch {
|
|
108
|
+
return new Set();
|
|
109
|
+
}
|
|
110
|
+
}
|
|
111
|
+
|
|
112
|
+
function saveDenylist(set: Set<string>): void {
|
|
113
|
+
try {
|
|
114
|
+
mkdirSync(dirname(DENYLIST_FILE), { recursive: true });
|
|
115
|
+
writeFileSync(DENYLIST_FILE, JSON.stringify([...set], null, 1));
|
|
116
|
+
} catch {
|
|
117
|
+
// best effort
|
|
118
|
+
}
|
|
119
|
+
}
|
|
120
|
+
|
|
121
|
+
const permanentDenylist = loadDenylist();
|
|
122
|
+
|
|
123
|
+
function banPermanently(provider: string, modelId: string): void {
|
|
124
|
+
const k = modelKey({ provider, id: modelId });
|
|
125
|
+
if (!permanentDenylist.has(k)) {
|
|
126
|
+
permanentDenylist.add(k);
|
|
127
|
+
saveDenylist(permanentDenylist);
|
|
128
|
+
}
|
|
129
|
+
}
|
|
130
|
+
|
|
131
|
+
/** Permanent failure (model retired, 404/410, refuses training, unsupported region, disabled model access…). */
|
|
132
|
+
export function isPermanentFailure(message: string): boolean {
|
|
133
|
+
return /no longer available|not found|\b404\b|\b410\b|ModelDeprecated|deprecated|discontinued|trains on request data|hosted outside Europe|requires Global regions|does not exist|no API implementation|unsupported|permission_error|model access is disabled|access to this model|do not have access/i.test(
|
|
134
|
+
message,
|
|
135
|
+
);
|
|
136
|
+
}
|
|
137
|
+
|
|
138
|
+
/**
|
|
139
|
+
* Any provider-wide limit (quota, rate limit, billing, bare 429): used on retry
|
|
140
|
+
* to decide provider-wide avoidance — an account-level limit hits every model
|
|
141
|
+
* of that provider, so failing over within it would fail again immediately.
|
|
142
|
+
*/
|
|
143
|
+
export const PROVIDER_LIMIT_RE =
|
|
144
|
+
/gousagelimiterror|freeusagelimiterror|usage limit|available balance|credit balance|insufficient_quota|out of budget|out of credit|quota|billing|subscription_sharing|resource_exhausted|rate.?limit|too many requests|\b429\b/i;
|
|
145
|
+
|
|
146
|
+
/**
|
|
147
|
+
* The subset pi classifies as NON-retryable (its retry would never start, so
|
|
148
|
+
* the turn dies). Only those get rewritten into a retryable 429 by
|
|
149
|
+
* `toFailoverRetryMessage`; already-retryable errors (transient rate limits,
|
|
150
|
+
* 5xx…) keep their original text untouched. Two families: provider-wide
|
|
151
|
+
* quota/billing limits, and model access disabled on the account (a 403 the
|
|
152
|
+
* provider never lifts for this key — same account, other models, still
|
|
153
|
+
* usable, so the model alone is avoided and the turn continues elsewhere).
|
|
154
|
+
*/
|
|
155
|
+
const NON_RETRYABLE_QUOTA_RE =
|
|
156
|
+
/gousagelimiterror|freeusagelimiterror|monthly usage limit reached|available balance|insufficient balance|credit balance|insufficient_quota|out of budget|out of credit|quota exceeded|exceeded your current quota|billing|subscription_sharing|resource_exhausted|permission_error|model access is disabled|access to this model|do not have access/i;
|
|
157
|
+
|
|
158
|
+
/**
|
|
159
|
+
* Model-access family (subset of NON_RETRYABLE_QUOTA_RE): the account cannot
|
|
160
|
+
* use THIS model. Unlike a quota, other models of the same provider may still
|
|
161
|
+
* work, so the retry avoids the model alone, not the whole provider.
|
|
162
|
+
*/
|
|
163
|
+
const MODEL_ACCESS_RE =
|
|
164
|
+
/permission_error|model access is disabled|access to this model|do not have access|not enabled on this account/i;
|
|
165
|
+
|
|
166
|
+
/**
|
|
167
|
+
* Turns a provider-quota error into a "429" pi is willing to retry. Pi drops
|
|
168
|
+
* these errors (retrying the same provider is pointless); under auto-route the
|
|
169
|
+
* failover changes provider, so a retry becomes useful: pi then calls
|
|
170
|
+
* `route(reason: "retry")` and the router switches. The rewritten text carries
|
|
171
|
+
* pi's retryable "429" pattern and avoids its non-retryable ones (billing,
|
|
172
|
+
* quota exceeded…); it also reads as a clear marker in the transcript.
|
|
173
|
+
* Returns undefined when there is nothing to rewrite.
|
|
174
|
+
*/
|
|
175
|
+
export function toFailoverRetryMessage(msg: AssistantMessage): AssistantMessage | undefined {
|
|
176
|
+
if (
|
|
177
|
+
msg.stopReason !== "error" ||
|
|
178
|
+
!msg.errorMessage ||
|
|
179
|
+
!msg.provider ||
|
|
180
|
+
!msg.model ||
|
|
181
|
+
!NON_RETRYABLE_QUOTA_RE.test(msg.errorMessage)
|
|
182
|
+
) {
|
|
183
|
+
return undefined;
|
|
184
|
+
}
|
|
185
|
+
const modelAccess = MODEL_ACCESS_RE.test(msg.errorMessage);
|
|
186
|
+
return {
|
|
187
|
+
...msg,
|
|
188
|
+
errorMessage: modelAccess
|
|
189
|
+
? `auto-route: 429 model access — ${msg.provider}/${msg.model} not enabled on this account; retrying on another model`
|
|
190
|
+
: `auto-route: 429 provider quota — ${msg.provider}/${msg.model} at limit; retrying on another model`,
|
|
191
|
+
};
|
|
192
|
+
}
|
|
193
|
+
|
|
194
|
+
// Dynamic catalog
|
|
195
|
+
|
|
196
|
+
function candidateModels(ctx: ExtensionContext): Model<never>[] {
|
|
197
|
+
return (ctx.modelRegistry.getAvailable() as Model<never>[]).filter((m) => {
|
|
198
|
+
if (permanentDenylist.has(modelKey(m))) return false;
|
|
199
|
+
if (EXCLUDED_MODEL_RE.test(m.id) || EXCLUDED_MODEL_RE.test(m.name)) return false;
|
|
200
|
+
// A model is free only when its pricing says so; missing pricing means
|
|
201
|
+
// unknown, not free, so it stays a candidate.
|
|
202
|
+
const free = m.cost !== undefined && m.cost.input === 0 && m.cost.output === 0;
|
|
203
|
+
return m.reasoning !== false && !free;
|
|
204
|
+
});
|
|
205
|
+
}
|
|
206
|
+
|
|
207
|
+
function buildTiers(ctx: ExtensionContext, avoid: string[] = []) {
|
|
208
|
+
const allCandidates = candidateModels(ctx);
|
|
209
|
+
if (allCandidates.length === 0) {
|
|
210
|
+
throw new Error("auto-route: no candidate model available (check your API keys)");
|
|
211
|
+
}
|
|
212
|
+
const avoidSet = new Set(avoid);
|
|
213
|
+
// `avoid` holds either `provider/id` keys (one model) or a bare `provider`
|
|
214
|
+
// (whole provider, quota exhausted: all of its models would fail alike).
|
|
215
|
+
const filtered = allCandidates.filter(
|
|
216
|
+
(m) => !avoidSet.has(modelKey(m)) && !avoidSet.has(m.provider),
|
|
217
|
+
);
|
|
218
|
+
const pool = filtered.length > 0 ? filtered : allCandidates;
|
|
219
|
+
const sorted = [...pool].sort((a, b) => {
|
|
220
|
+
// Unknown pricing ranks as mid cost so pricing-less models land in the
|
|
221
|
+
// middle tiers instead of being dumped into eco or promoted to frontier.
|
|
222
|
+
const ca = a.cost?.output ?? UNKNOWN_OUTPUT_COST;
|
|
223
|
+
const cb = b.cost?.output ?? UNKNOWN_OUTPUT_COST;
|
|
224
|
+
if (Math.abs(ca - cb) < 0.01) return (b.contextWindow ?? 0) - (a.contextWindow ?? 0);
|
|
225
|
+
return cb - ca;
|
|
226
|
+
});
|
|
227
|
+
const at = (frac: number): Model<never> =>
|
|
228
|
+
sorted[Math.min(sorted.length - 1, Math.floor(frac * sorted.length))];
|
|
229
|
+
return {
|
|
230
|
+
frontier: at(0),
|
|
231
|
+
strong: at(0.33),
|
|
232
|
+
standard: at(0.66),
|
|
233
|
+
eco: sorted[sorted.length - 1],
|
|
234
|
+
background: sorted[sorted.length - 1],
|
|
235
|
+
} as const;
|
|
236
|
+
}
|
|
237
|
+
|
|
238
|
+
// Status (one discreet line)
|
|
239
|
+
|
|
240
|
+
function setStatusLine(
|
|
241
|
+
pi: ExtensionAPI,
|
|
242
|
+
ctx: ExtensionContext,
|
|
243
|
+
m: Model<never>,
|
|
244
|
+
thinking: Thinking,
|
|
245
|
+
source?: string,
|
|
246
|
+
): void {
|
|
247
|
+
const shown = source ?? "local";
|
|
248
|
+
try {
|
|
249
|
+
const dim = ctx.ui.theme?.fg?.bind(ctx.ui.theme) ?? ((_: string, s: string) => s);
|
|
250
|
+
ctx.ui.setStatus("auto", dim("dim", `⚡ ${modelKey(m)} · ${thinking} · ${shown}`));
|
|
251
|
+
} catch {
|
|
252
|
+
ctx.ui.setStatus("auto", `${modelKey(m)} · ${thinking} · ${shown}`);
|
|
253
|
+
}
|
|
254
|
+
pi.events.emit("auto-route:routed", {
|
|
255
|
+
provider: m.provider,
|
|
256
|
+
modelId: m.id,
|
|
257
|
+
thinking,
|
|
258
|
+
source: shown,
|
|
259
|
+
});
|
|
260
|
+
}
|
|
261
|
+
|
|
262
|
+
function toRoute(
|
|
263
|
+
pi: ExtensionAPI,
|
|
264
|
+
ctx: ExtensionContext,
|
|
265
|
+
m: Model<never>,
|
|
266
|
+
thinking: Thinking,
|
|
267
|
+
source: string | undefined,
|
|
268
|
+
prevState?: RouteState,
|
|
269
|
+
nextState?: RouteState,
|
|
270
|
+
): ModelRoute<RouteState> {
|
|
271
|
+
setStatusLine(pi, ctx, m, thinking, source);
|
|
272
|
+
// Only return a new state object when it actually changed (avoids writing a
|
|
273
|
+
// session-tree entry on every continuation request).
|
|
274
|
+
const stateToStore =
|
|
275
|
+
nextState && !sameState(prevState, nextState) ? nextState : undefined;
|
|
276
|
+
return {
|
|
277
|
+
model: m,
|
|
278
|
+
thinkingLevel: thinking,
|
|
279
|
+
state: stateToStore,
|
|
280
|
+
};
|
|
281
|
+
}
|
|
282
|
+
|
|
283
|
+
// Difficulty heuristic
|
|
284
|
+
//
|
|
285
|
+
// Calibrated on 2026-11-21 over 880 real turns extracted from
|
|
286
|
+
// ~/.pi/agent/sessions/*.jsonl. Before/after measurements (analysis script,
|
|
287
|
+
// real-difficulty proxy = combined rank of the turn's tool calls and output
|
|
288
|
+
// tokens):
|
|
289
|
+
// - Spearman score vs. tool calls: v1 +0.16 → v2 +0.28;
|
|
290
|
+
// - heaviest-quartile messages sent to the eco tier: v1 195/220 → v2 72/220;
|
|
291
|
+
// - distribution: v1 = 92 % of messages in {0.10, 0.15} → v2 spread over 8+
|
|
292
|
+
// values.
|
|
293
|
+
// Every signal is independent and interpretable; no training, no data leaves
|
|
294
|
+
// the machine: the rule stays hand-written, the numbers only verify it.
|
|
295
|
+
|
|
296
|
+
/** Reference to a path, file or repo → work on an existing codebase. */
|
|
297
|
+
const PATHY =
|
|
298
|
+
/(?:\/(?:home|data|tmp|etc|usr|opt|var|srv)\/|~\/|\b[\w.+-]+\/[\w.-]+\.[A-Za-z0-9]{1,6}\b|\b[\w.+-]+\/[\w-]*-[\w-]+\b)/;
|
|
299
|
+
/**
|
|
300
|
+
* Construction / modification / analysis verbs → the agent is about to do work.
|
|
301
|
+
* The colloquial imperatives (fais, regarde, prend, donne, met, lance, écris,
|
|
302
|
+
* essaie, montre) are as frequent as the formal ones in real prompts: leaving
|
|
303
|
+
* them out scored "fais le portage français" at 0.10 and routed a turn of 49
|
|
304
|
+
* tool calls and 82k output tokens to the eco tier. English stems a French
|
|
305
|
+
* pattern already covers (impl-, develop-, gener-, analy-, test, install: the
|
|
306
|
+
* accent classes hold the plain letters too) are not repeated.
|
|
307
|
+
*/
|
|
308
|
+
const ACTION =
|
|
309
|
+
/\b(reprend\w*|construi\w*|impl[ée]ment\w*|cré[ée]r?|cree\w*|g[ée]n[èe]r\w*|d[ée]velop\w*|migre\w*|refais|refaire|transforme\w*|modifi\w*|ajout\w*|supprim\w*|corrige\w*|corrig\w*|install\w*|configure\w*|[ée]crase\w*|r[ée]pare\w*|optimise\w*|nettoie\w*|remplace\w*|relance\w*|trouve\w*|cherch\w*|analy\w*|examin\w*|v[ée]rifi\w*|test\w*|fais\w*|faire|regarde\w*|prends?\b|donnes?\b|mets?\b|lance\w*|[ée]cris\b|essaie\w*|montre\w*|make|do(es)?\b|build|creat\w*|migrat\w*|redo|transform|modif\w*|add\w*|remov\w*|delet\w*|fix\w*|repair|configur\w*|overwrit\w*|optimi[sz]\w*|clean|replac\w*|relaunch|restart|find\w*|search\w*|exam\w*|check\w*|look\w*|take|put|launch\w*|run\b|writes?\b|try\b|show\w*|open\w*|read\w*|updat\w*|refactor\w*)\b/i;
|
|
310
|
+
/** Explicit intent from the requester, in French or English. */
|
|
311
|
+
const WANT =
|
|
312
|
+
/\b(je veux|fais en sorte|fais[- ]moi|il faut|on pourrait|peux[- ]tu|besoin de|i want|make sure|i need|can you|could you|we could|need to|please\b)\b/i;
|
|
313
|
+
/** Conceptual question → asks for a solid explanation. */
|
|
314
|
+
const QUESTION =
|
|
315
|
+
/\b(pourquoi|explique\w*|c'est quoi|c est quoi|comment (?:ça|ca) marche|diff[ée]rence entre|quelle est|quel est|est[- ]ce que|why\b|explain\w*|what (?:is|are|does)|what's|how (?:does|do|to)|difference between|which (?:is|one)|is there)\b/i;
|
|
316
|
+
/** Concept keywords, counted (up to 3); French and English stems. */
|
|
317
|
+
const CONCEPTS =
|
|
318
|
+
/\b(architecture|conception|refactor|d[ée]bogu|debug|optimi[sz]|s[ée]curit|security|timing|algorithme|algorithm|compar|pourquoi|analyse|syst[èe]me|system|protocole|protocol|concept|tradeoff|modul\w*|scal\w*)\b/gi;
|
|
319
|
+
/** Explicit breakage / regression report, in French or English. */
|
|
320
|
+
const REGRESSION =
|
|
321
|
+
/\b(pas le m[êe]me|r[ée]gression|cass[ée]|fuit|bloqu[ée]|corriger|not the same|broken|broke\b|leak\w*|blocked|stuck|crash\w*|fails?\b|error\w*|wrong)\b/i;
|
|
322
|
+
/** Shell prompt line (terminal paste: form, not difficulty). */
|
|
323
|
+
const SHELLLINE = /^\s*(?:\$|sudo\s|#\s)/gm;
|
|
324
|
+
|
|
325
|
+
export function difficultyScore(text: string): number {
|
|
326
|
+
const t = text.trim();
|
|
327
|
+
let sig = Math.min(0.3, ((t.match(CONCEPTS) ?? []).length) * 0.1);
|
|
328
|
+
if (PATHY.test(t)) sig += 0.25;
|
|
329
|
+
if (ACTION.test(t)) sig += 0.2;
|
|
330
|
+
if (WANT.test(t)) sig += 0.1;
|
|
331
|
+
if (QUESTION.test(t)) sig += 0.1;
|
|
332
|
+
if (t.includes("```")) sig += 0.1;
|
|
333
|
+
if (t.length > 800) sig += 0.1;
|
|
334
|
+
if (t.length > 2000) sig += 0.1;
|
|
335
|
+
if (REGRESSION.test(t)) sig += 0.1;
|
|
336
|
+
// Terminal paste (≥3 shell prompt lines): form, not difficulty.
|
|
337
|
+
if ((t.match(SHELLLINE) ?? []).length >= 3) sig -= 0.15;
|
|
338
|
+
// Base 0.10: short follow-up and neutral message → lightest tier.
|
|
339
|
+
return Math.round(Math.min(1, 0.1 + Math.max(0, sig)) * 100) / 100;
|
|
340
|
+
}
|
|
341
|
+
|
|
342
|
+
export function tierAndThinking(score: number): { tier: TierName; thinking: Thinking } {
|
|
343
|
+
// Thresholds raised from 0.50/0.70 to 0.60/0.75 on 926 real turns, after the
|
|
344
|
+
// ACTION regex picked up the colloquial imperatives: with both changes a
|
|
345
|
+
// path + verb pair no longer justifies an expensive tier ("corrige le bug
|
|
346
|
+
// dans src/foo.ts" used to reach strong). Measured against the real-difficulty
|
|
347
|
+
// proxy: expensive routings 104 → 85, heaviest-quartile turns sent to eco
|
|
348
|
+
// 82 → 53, light turns in an expensive tier 52 → 44, Spearman +0.288 → +0.339.
|
|
349
|
+
if (score >= 0.75) return { tier: "frontier", thinking: "xhigh" };
|
|
350
|
+
if (score >= 0.6) return { tier: "strong", thinking: "high" };
|
|
351
|
+
if (score >= 0.3) return { tier: "standard", thinking: "medium" };
|
|
352
|
+
return { tier: "eco", thinking: "low" };
|
|
353
|
+
}
|
|
354
|
+
|
|
355
|
+
function capThinking(auto: Thinking, userSelected: Thinking): Thinking {
|
|
356
|
+
return THINKING_ORDER[Math.min(THINKING_ORDER.indexOf(auto), THINKING_ORDER.indexOf(userSelected))];
|
|
357
|
+
}
|
|
358
|
+
|
|
359
|
+
// Hybrid classifier (optional; auto-detected, silent fallback)
|
|
360
|
+
|
|
361
|
+
/** Tier decision handed back by a classifier: target tier plus provenance. */
|
|
362
|
+
export interface TierDecision {
|
|
363
|
+
tier: TierName;
|
|
364
|
+
thinking: Thinking;
|
|
365
|
+
via: string;
|
|
366
|
+
}
|
|
367
|
+
|
|
368
|
+
/** Classifier answer → tier, mirroring `tierAndThinking`'s thresholds. */
|
|
369
|
+
const CHOICE_TO_TIER: Record<string, { tier: TierName; thinking: Thinking }> = {
|
|
370
|
+
light: { tier: "eco", thinking: "low" },
|
|
371
|
+
standard: { tier: "standard", thinking: "medium" },
|
|
372
|
+
complex: { tier: "strong", thinking: "high" },
|
|
373
|
+
heavy: { tier: "frontier", thinking: "xhigh" },
|
|
374
|
+
};
|
|
375
|
+
|
|
376
|
+
/**
|
|
377
|
+
* Asks an available classifier model to sort a turn into a tier: the catalog is
|
|
378
|
+
* auto-detected across every provider with working credentials, free
|
|
379
|
+
* classifiers first, then the cheaper ones. Only the user prompt is sent,
|
|
380
|
+
* truncated to 16k characters — never the system prompt, the history or tool
|
|
381
|
+
* results. The question is asked in English: classifiers are multilingual.
|
|
382
|
+
* Never throws: no classifier, a 4-second timeout, an abort, an unexpected
|
|
383
|
+
* answer or one below 0.50 confidence all return undefined, so the caller
|
|
384
|
+
* silently falls back to the local heuristic.
|
|
385
|
+
*/
|
|
386
|
+
export async function classifyTier(
|
|
387
|
+
ctx: ExtensionContext,
|
|
388
|
+
text: string,
|
|
389
|
+
signal?: AbortSignal,
|
|
390
|
+
): Promise<TierDecision | undefined> {
|
|
391
|
+
try {
|
|
392
|
+
const models = await ctx.modelRegistry.getAvailableOfType("classifier");
|
|
393
|
+
if (!models || models.length === 0) return undefined;
|
|
394
|
+
// Free classifiers first, then ascending input cost; unknown pricing
|
|
395
|
+
// ranks last. Array sort is stable, so the catalog order survives.
|
|
396
|
+
const model = [...models].sort((a, b) => {
|
|
397
|
+
const freeA = a.cost?.input === 0;
|
|
398
|
+
const freeB = b.cost?.input === 0;
|
|
399
|
+
if (freeA !== freeB) return freeA ? -1 : 1;
|
|
400
|
+
return (
|
|
401
|
+
(a.cost?.input ?? Number.POSITIVE_INFINITY) -
|
|
402
|
+
(b.cost?.input ?? Number.POSITIVE_INFINITY)
|
|
403
|
+
);
|
|
404
|
+
})[0];
|
|
405
|
+
if (!model) return undefined;
|
|
406
|
+
const timeoutSignal = AbortSignal.timeout(4_000);
|
|
407
|
+
const result = await ctx.modelRegistry.classify(
|
|
408
|
+
model,
|
|
409
|
+
{
|
|
410
|
+
state: { prompt: text.slice(0, 16_000) },
|
|
411
|
+
questions: {
|
|
412
|
+
difficulty: {
|
|
413
|
+
type: "choice",
|
|
414
|
+
instructions:
|
|
415
|
+
"How demanding is the software engineering work requested in prompt?",
|
|
416
|
+
criteria: {
|
|
417
|
+
light: "A quick question, a short follow-up, or a trivial change",
|
|
418
|
+
standard: "An ordinary feature, fix, review, or explanation",
|
|
419
|
+
complex: "Subtle design, cross-cutting changes, or hard debugging",
|
|
420
|
+
heavy: "Architecture-level rework or a long multi-step implementation",
|
|
421
|
+
},
|
|
422
|
+
},
|
|
423
|
+
},
|
|
424
|
+
},
|
|
425
|
+
{ signal: signal ? AbortSignal.any([signal, timeoutSignal]) : timeoutSignal },
|
|
426
|
+
);
|
|
427
|
+
if (result.stopReason !== "stop") return undefined;
|
|
428
|
+
const answer = result.answers?.difficulty;
|
|
429
|
+
if (answer?.type !== "choice") return undefined;
|
|
430
|
+
const decision = CHOICE_TO_TIER[answer.choice];
|
|
431
|
+
if (!decision) return undefined;
|
|
432
|
+
const probability = answer.probabilities[answer.choice] ?? answer.confidence ?? 0;
|
|
433
|
+
if (probability < 0.5) return undefined;
|
|
434
|
+
return { ...decision, via: modelKey(model) };
|
|
435
|
+
} catch {
|
|
436
|
+
return undefined;
|
|
437
|
+
}
|
|
438
|
+
}
|
|
439
|
+
|
|
440
|
+
// Utilities
|
|
441
|
+
|
|
442
|
+
function lastUserText(messages: readonly Message[]): string {
|
|
443
|
+
const content = messages.filter((m) => m.role === "user").at(-1)?.content ?? "";
|
|
444
|
+
if (typeof content === "string") return content;
|
|
445
|
+
return content
|
|
446
|
+
.flatMap((b) => (b.type === "text" ? [b.text] : []))
|
|
447
|
+
.join("\n");
|
|
448
|
+
}
|
|
449
|
+
|
|
450
|
+
function editedThisTurn(messages: readonly Message[]): boolean {
|
|
451
|
+
const lastUser = messages.findLastIndex((m) => m.role === "user");
|
|
452
|
+
return messages
|
|
453
|
+
.slice(lastUser + 1)
|
|
454
|
+
.some(
|
|
455
|
+
(m) =>
|
|
456
|
+
m.role === "toolResult" &&
|
|
457
|
+
EDIT_TOOLS.has((m as { toolName?: string }).toolName ?? "") &&
|
|
458
|
+
!(m as { isError?: boolean }).isError,
|
|
459
|
+
);
|
|
460
|
+
}
|
|
461
|
+
|
|
462
|
+
/**
|
|
463
|
+
* Shared entry point for a new turn and for the safety fallback of a
|
|
464
|
+
* continuation (current model banned/missing): decide the tier of the current
|
|
465
|
+
* message — ONE classifier call when a classifier is available, the local
|
|
466
|
+
* bilingual heuristic otherwise. `avoid` is empty on a new turn, preserved on
|
|
467
|
+
* a safety re-route.
|
|
468
|
+
*/
|
|
469
|
+
async function rerouteCurrentTurn(
|
|
470
|
+
pi: ExtensionAPI,
|
|
471
|
+
ctx: ExtensionContext,
|
|
472
|
+
request: ModelRouteRequest<RouteState>,
|
|
473
|
+
avoid: string[],
|
|
474
|
+
userCap: Thinking,
|
|
475
|
+
prevState?: RouteState,
|
|
476
|
+
): Promise<ModelRoute<RouteState>> {
|
|
477
|
+
const text = lastUserText(request.messages);
|
|
478
|
+
// Classifier first; the local bilingual heuristic is the silent fallback.
|
|
479
|
+
const decision = await classifyTier(ctx, text, request.signal);
|
|
480
|
+
let tier: TierName;
|
|
481
|
+
let thinking: Thinking;
|
|
482
|
+
let source: string;
|
|
483
|
+
if (decision) {
|
|
484
|
+
({ tier, thinking } = decision);
|
|
485
|
+
source = decision.via;
|
|
486
|
+
} else {
|
|
487
|
+
({ tier, thinking } = tierAndThinking(difficultyScore(text)));
|
|
488
|
+
source = "local";
|
|
489
|
+
}
|
|
490
|
+
const th = capThinking(thinking, userCap);
|
|
491
|
+
const m = buildTiers(ctx, avoid)[tier];
|
|
492
|
+
return toRoute(pi, ctx, m, th, source, prevState, {
|
|
493
|
+
modelId: m.id,
|
|
494
|
+
providerId: m.provider,
|
|
495
|
+
thinking: th,
|
|
496
|
+
avoid,
|
|
497
|
+
source,
|
|
498
|
+
});
|
|
499
|
+
}
|
|
500
|
+
|
|
501
|
+
export default function (pi: ExtensionAPI) {
|
|
502
|
+
// Pi does NOT trigger `route(reason: "retry")` on 400/404 errors nor on
|
|
503
|
+
// provider-quota 429s (non-retryable classes). We listen to `message_end` to:
|
|
504
|
+
// 1. permanently ban models with permanent failures (400/404/410…);
|
|
505
|
+
// 2. rewrite quota 429s into a retryable error, so pi starts its retry →
|
|
506
|
+
// route(reason: "retry") → cross-provider failover instead of a dead turn.
|
|
507
|
+
pi.on("message_end", (event, ctx) => {
|
|
508
|
+
const msg = event.message as AssistantMessage;
|
|
509
|
+
if (msg?.role !== "assistant" || msg.stopReason !== "error" || !msg.errorMessage) return;
|
|
510
|
+
if (isPermanentFailure(msg.errorMessage) && msg.provider && msg.model) {
|
|
511
|
+
banPermanently(msg.provider, msg.model);
|
|
512
|
+
}
|
|
513
|
+
if (ctx.model?.provider === "auto" && ctx.model?.id === "auto") {
|
|
514
|
+
const replacement = toFailoverRetryMessage(msg);
|
|
515
|
+
if (replacement) return { message: replacement };
|
|
516
|
+
}
|
|
517
|
+
});
|
|
518
|
+
|
|
519
|
+
// Selecting `auto` starts at thinking `low` by default (ceiling, raise it
|
|
520
|
+
// with shift+tab). `model_select` only fires on a user selection: pi never
|
|
521
|
+
// emits it at startup or when restoring a session.
|
|
522
|
+
pi.on("model_select", (event) => {
|
|
523
|
+
if (event.model.provider === "auto" && event.model.id === "auto") {
|
|
524
|
+
pi.setThinkingLevel("low");
|
|
525
|
+
}
|
|
526
|
+
});
|
|
527
|
+
|
|
528
|
+
pi.registerVirtualModel<RouteState>({
|
|
529
|
+
provider: "auto",
|
|
530
|
+
id: "auto",
|
|
531
|
+
name: "Auto (routing + thinking + failover)",
|
|
532
|
+
thinkingLevels: ["off", "low", "medium", "high", "xhigh"],
|
|
533
|
+
contextWindow: 200_000,
|
|
534
|
+
maxTokens: 64_000,
|
|
535
|
+
async route(request, ctx) {
|
|
536
|
+
const userCap = (request.thinkingLevel ?? "xhigh") as Thinking;
|
|
537
|
+
|
|
538
|
+
// Compaction/summaries outside the agent loop: no session state.
|
|
539
|
+
if (request.reason === "direct") {
|
|
540
|
+
return {
|
|
541
|
+
model: buildTiers(ctx).background,
|
|
542
|
+
thinkingLevel: "low",
|
|
543
|
+
};
|
|
544
|
+
}
|
|
545
|
+
|
|
546
|
+
const prevState = request.state;
|
|
547
|
+
|
|
548
|
+
// RETRY / failover (429 / 5xx)
|
|
549
|
+
if (request.reason === "retry" && request.failed) {
|
|
550
|
+
const failedKey = modelKey(request.failed.model);
|
|
551
|
+
if (isPermanentFailure(request.failed.message?.errorMessage ?? "")) {
|
|
552
|
+
banPermanently(request.failed.model.provider, request.failed.model.id);
|
|
553
|
+
}
|
|
554
|
+
const baseState: RouteState = prevState ?? {
|
|
555
|
+
modelId: request.failed.model.id,
|
|
556
|
+
providerId: request.failed.model.provider,
|
|
557
|
+
thinking: (request.failed.thinkingLevel ?? "medium") as Thinking,
|
|
558
|
+
avoid: [],
|
|
559
|
+
};
|
|
560
|
+
const avoid = [
|
|
561
|
+
...new Set([
|
|
562
|
+
...baseState.avoid,
|
|
563
|
+
// Model access disabled on the account: a per-model problem — other
|
|
564
|
+
// models of the same provider may still work, so avoid the model
|
|
565
|
+
// alone. Provider quota: avoid the whole provider, otherwise the
|
|
566
|
+
// failover lands on another of its models and fails again at once.
|
|
567
|
+
MODEL_ACCESS_RE.test(request.failed.message?.errorMessage ?? "")
|
|
568
|
+
? failedKey
|
|
569
|
+
: PROVIDER_LIMIT_RE.test(request.failed.message?.errorMessage ?? "")
|
|
570
|
+
? request.failed.model.provider
|
|
571
|
+
: failedKey,
|
|
572
|
+
]),
|
|
573
|
+
];
|
|
574
|
+
const tiers = buildTiers(ctx, avoid);
|
|
575
|
+
const ladder: TierName[] =
|
|
576
|
+
avoid.length >= 3 ? ["standard", "eco"] : ["strong", "standard", "eco"];
|
|
577
|
+
const chosen =
|
|
578
|
+
ladder
|
|
579
|
+
.map((t) => tiers[t])
|
|
580
|
+
.find((m) => m && !avoid.includes(modelKey(m)) && !avoid.includes(m.provider)) ??
|
|
581
|
+
tiers.frontier;
|
|
582
|
+
const th = capThinking(baseState.thinking, userCap);
|
|
583
|
+
return toRoute(pi, ctx, chosen, th, baseState.source, prevState, {
|
|
584
|
+
...baseState,
|
|
585
|
+
modelId: chosen.id,
|
|
586
|
+
providerId: chosen.provider,
|
|
587
|
+
thinking: th,
|
|
588
|
+
avoid,
|
|
589
|
+
});
|
|
590
|
+
}
|
|
591
|
+
|
|
592
|
+
// CONTINUATION (same turn, after tool calls)
|
|
593
|
+
if (request.reason === "continuation" && prevState) {
|
|
594
|
+
const avoid = prevState.avoid ?? [];
|
|
595
|
+
|
|
596
|
+
// First successful edit/write of the turn → switch to the eco tier for
|
|
597
|
+
// the rest of the turn (editedThisTurn stays true: toolResults accumulate).
|
|
598
|
+
if (editedThisTurn(request.messages)) {
|
|
599
|
+
const eco = buildTiers(ctx, avoid).eco;
|
|
600
|
+
const th = capThinking("medium", userCap);
|
|
601
|
+
return toRoute(pi, ctx, eco, th, prevState.source, prevState, {
|
|
602
|
+
...prevState,
|
|
603
|
+
modelId: eco.id,
|
|
604
|
+
providerId: eco.provider,
|
|
605
|
+
thinking: th,
|
|
606
|
+
avoid,
|
|
607
|
+
});
|
|
608
|
+
}
|
|
609
|
+
|
|
610
|
+
// No edit yet → keep the current model to preserve the prompt cache
|
|
611
|
+
// and the `avoid` list; if it was banned meanwhile, fall through to
|
|
612
|
+
// the safety fallback below.
|
|
613
|
+
const currentKey = modelKey({ provider: prevState.providerId, id: prevState.modelId });
|
|
614
|
+
if (
|
|
615
|
+
!permanentDenylist.has(currentKey) &&
|
|
616
|
+
!avoid.includes(currentKey) &&
|
|
617
|
+
!avoid.includes(prevState.providerId)
|
|
618
|
+
) {
|
|
619
|
+
const m = ctx.modelRegistry.find(prevState.providerId, prevState.modelId) as
|
|
620
|
+
| Model<never>
|
|
621
|
+
| undefined;
|
|
622
|
+
if (m) {
|
|
623
|
+
const th = capThinking(prevState.thinking, userCap);
|
|
624
|
+
return toRoute(pi, ctx, m, th, prevState.source, prevState, {
|
|
625
|
+
...prevState,
|
|
626
|
+
thinking: th,
|
|
627
|
+
});
|
|
628
|
+
}
|
|
629
|
+
}
|
|
630
|
+
// Safety fallback: current model banned/missing → re-route the SAME
|
|
631
|
+
// turn (classifier or local score on the current message, `avoid` kept).
|
|
632
|
+
return await rerouteCurrentTurn(pi, ctx, request, prevState.avoid, userCap, prevState);
|
|
633
|
+
}
|
|
634
|
+
|
|
635
|
+
// NEW TURN (user message): one classifier call, else the local heuristic.
|
|
636
|
+
return await rerouteCurrentTurn(pi, ctx, request, [], userCap, prevState);
|
|
637
|
+
},
|
|
638
|
+
});
|
|
639
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,37 @@
|
|
|
1
1
|
{
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
2
|
+
"name": "pi-auto-route",
|
|
3
|
+
"version": "1.3.0",
|
|
4
|
+
"description": "A pi virtual model that routes each turn to the right physical model by task difficulty and adjusts the thinking level.",
|
|
5
|
+
"keywords": [
|
|
6
|
+
"pi-package",
|
|
7
|
+
"pi",
|
|
8
|
+
"virtual-model"
|
|
9
|
+
],
|
|
10
|
+
"license": "MIT",
|
|
11
|
+
"author": "rzph",
|
|
12
|
+
"repository": {
|
|
13
|
+
"type": "git",
|
|
14
|
+
"url": "git+https://github.com/rzph/pi-auto-route.git"
|
|
15
|
+
},
|
|
16
|
+
"homepage": "https://github.com/rzph/pi-auto-route#readme",
|
|
17
|
+
"bugs": {
|
|
18
|
+
"url": "https://github.com/rzph/pi-auto-route/issues"
|
|
19
|
+
},
|
|
20
|
+
"files": [
|
|
21
|
+
"auto-route.ts",
|
|
22
|
+
"README.md",
|
|
23
|
+
"CHANGELOG.md",
|
|
24
|
+
"LICENSE"
|
|
25
|
+
],
|
|
26
|
+
"pi": {
|
|
27
|
+
"extensions": [
|
|
28
|
+
"./auto-route.ts"
|
|
29
|
+
]
|
|
30
|
+
},
|
|
31
|
+
"peerDependencies": {
|
|
32
|
+
"@earendil-works/pi-ai": "*",
|
|
33
|
+
"@earendil-works/pi-coding-agent": "*",
|
|
34
|
+
"@earendil-works/pi-tui": "*",
|
|
35
|
+
"typebox": "*"
|
|
36
|
+
}
|
|
37
|
+
}
|