@alexeiled/pi-model-router 0.6.5 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +27 -0
- package/README.md +56 -251
- package/extensions/classifier.ts +4 -5
- package/extensions/commands.ts +278 -576
- package/extensions/config.ts +52 -8
- package/extensions/constants.ts +7 -0
- package/extensions/context.ts +6 -2
- package/extensions/jev.ts +325 -66
- package/extensions/provider.ts +30 -4
- package/extensions/state.ts +29 -2
- package/extensions/types.ts +50 -1
- package/extensions/ui.ts +49 -7
- package/model-router.example.json +2 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,32 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.7.0] - 2026-09-22
|
|
4
|
+
|
|
5
|
+
### Behavior change
|
|
6
|
+
|
|
7
|
+
- A Choice below `confidenceThreshold` is not discarded. The router selects the lowest tier whose cumulative probability reaches `probabilityThreshold` (default 0.8). Abstention mass counts for the baseline tier.
|
|
8
|
+
- Each Jev tier is a structured Choice option with `covers`, `notFor` and `examples`. The instructions are a structured object that names the state fields.
|
|
9
|
+
- The router retries one transient Jev status (`408`, `429`, `5xx`) inside the existing total budget. The backoff honors `Retry-After`. Permanent statuses and cancellation are not retried.
|
|
10
|
+
- The response validator accepts omitted zero-mass options and two-decimal rounding.
|
|
11
|
+
- The request token estimate adds 400 tokens of headroom instead of 200.
|
|
12
|
+
|
|
13
|
+
### Removed
|
|
14
|
+
|
|
15
|
+
- The `low-confidence` outcome. Debug output shows `selected`, `basis`, `route-p` and `route-threshold` instead.
|
|
16
|
+
- The `/router` verbs `status`, `profile`, `fix`, `disable`, `debug`, `?` and per-tier `thinking`. A removed verb prints its replacement and does nothing.
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
|
|
20
|
+
- `jev.probabilityThreshold`, `jev.retry.maxAttempts`, `jev.retry.backoffMs` and `classifierModel.timeoutMs`. Defaults are unchanged.
|
|
21
|
+
- The `/router` verbs `off`, `log [on|off|clear]` and argument-free `widget`. Top-level completion lists verbs and profile names.
|
|
22
|
+
- A skipped advisor records `bypassReason` and the footer shows it: `advice skipped: pinned high`, `over budget`, `only high eligible`, `tool turn`.
|
|
23
|
+
- An `invalid-response` names the failing local check. Remote text is not retained.
|
|
24
|
+
|
|
25
|
+
### Documentation
|
|
26
|
+
|
|
27
|
+
- `docs/README.md` is the index. `docs/jev-advisor.md` holds the Jev guide. `docs/research/` holds dated experiment reports. `docs/archive/` holds superseded reports. File names are lowercase.
|
|
28
|
+
- Parallel Noul questions were tested and not added. See `docs/research/jev-routing-policy.md`.
|
|
29
|
+
|
|
3
30
|
## [0.6.5] - 2026-09-22
|
|
4
31
|
|
|
5
32
|
- The router now sends structured Jev state for the current request, recent dialogue, and optional tool evidence.
|
package/README.md
CHANGED
|
@@ -95,12 +95,12 @@ pi -e ./extensions/index.ts
|
|
|
95
95
|
|
|
96
96
|
- Generation and classification use Pi's provider registry, including native/custom providers and credential-specific URLs. Only the optional Jev advisor uses separate HTTPS transport.
|
|
97
97
|
- Fallbacks run only before content is emitted; cancellation does not retry. Every target must support the requested input and exact thinking level; explicit unsupported effort is not silently reduced. Omitted thinking defaults to `off` for non-reasoning targets, including fallbacks.
|
|
98
|
-
- Jev
|
|
98
|
+
- Jev has one total advisory budget (`jev.timeoutMs`, 1500 ms by default) that includes its single transient retry. The classifier-only compatibility path has its own budget (`classifierModel.timeoutMs`, 10 s by default) and never retries. Failure or uncertainty means the eligible baseline; caller cancellation stops generation.
|
|
99
99
|
- Valid same-turn tool continuations reuse the actual prior route before either advisor. Pins, budget policy and a single eligible primary candidate also bypass advisors. Invalid continuations choose a compatible local route without advice; incompatible Google thought-signature replay fails plainly.
|
|
100
100
|
- Pi owns tool execution permissions and per-request authentication. The router checks configured provider/profile identity, not which backend login is currently behind a provider. No private authentication storage is read.
|
|
101
101
|
- Context trimming preserves system instructions and whole active tool turns. It is a text estimate, not a guarantee that images or a large active turn fit.
|
|
102
102
|
|
|
103
|
-
See [architecture](
|
|
103
|
+
See [docs/architecture.md](docs/architecture.md) and [docs/releasing.md](docs/releasing.md).
|
|
104
104
|
|
|
105
105
|
## Configuration
|
|
106
106
|
|
|
@@ -136,8 +136,8 @@ The extension stores the last selected profile in `~/.pi/agent/model-router-stat
|
|
|
136
136
|
|
|
137
137
|
| Field | Description |
|
|
138
138
|
| ----------------------- | --------------------------------------------------------------------------------- |
|
|
139
|
-
| `classifierModel` | (Optional) Pi model used for four-tier semantic advice only when Jev is not active (disabled, not opted in or missing a key). Supports model aliases. Failure means baseline. |
|
|
140
|
-
| `jev` | (Optional, user config only) External advisor settings; requires global enablement, a key and an explicit `profiles.<name>.jev.enabled` opt-in. Disabled by default. |
|
|
139
|
+
| `classifierModel` | (Optional) Pi model used for four-tier semantic advice only when Jev is not active (disabled, not opted in or missing a key). A string, or `{ "model", "thinking", "timeoutMs" }` (`timeoutMs` defaults to 10000). Supports model aliases. Failure means baseline. |
|
|
140
|
+
| `jev` | (Optional, user config only) External advisor settings; requires global enablement, a key and an explicit `profiles.<name>.jev.enabled` opt-in. Disabled by default. Reference: [docs/jev-advisor.md](docs/jev-advisor.md). |
|
|
141
141
|
| `ui.statusLine` | `compact` (default) or `detailed`. Display only; project config may override it. Widget/debug always include full diagnostics. |
|
|
142
142
|
| `maxSessionBudget` | (Optional) Soft generation-cost threshold in USD. Unpinned requests prefer eligible medium-or-lower tiers and skip advisors. Not a spending cap; classifier and Jev costs are excluded. |
|
|
143
143
|
| `phaseBias`, `rules` | Deprecated and ignored, with a fixed value-free warning. Remove these fields; there is no legacy keyword mode. |
|
|
@@ -174,264 +174,69 @@ remains, and model tier never grants or restricts tool permissions.
|
|
|
174
174
|
|
|
175
175
|
### Optional Jev advisor: user config only
|
|
176
176
|
|
|
177
|
-
Jev
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
skip Jev and the classifier.
|
|
182
|
-
|
|
183
|
-
Malformed responses, `uncertain`, low confidence, timeout and HTTP errors go
|
|
184
|
-
directly to the eligible baseline, without a classifier cascade. When Jev is not
|
|
185
|
-
active (including a missing key), the optional Pi classifier is a separate
|
|
186
|
-
compatibility path; without it, the router uses baseline directly. Jev cannot select another profile or an arbitrary model,
|
|
187
|
-
provider account or thinking level. Explicit generation fallback chains may
|
|
188
|
-
still cross providers, as configured by you.
|
|
189
|
-
|
|
190
|
-
Configure Jev **only** in `~/.pi/agent/model-router.json` (or the agent directory
|
|
191
|
-
selected by Pi). Both global enablement and an explicit user-level profile opt-in
|
|
192
|
-
are required. Work profiles remain disabled unless you explicitly approve sending
|
|
193
|
-
their bounded recent conversation text externally. All project-level `jev` settings, including
|
|
194
|
-
profile opt-ins, are ignored with a warning, before merging user credentials.
|
|
177
|
+
Jev is TypeSafe's System One model. When enabled, the router asks it once per
|
|
178
|
+
new user turn to pick one of the active profile's eligible tier/model/thinking
|
|
179
|
+
pairs from bounded recent conversation text. It cannot choose another profile,
|
|
180
|
+
an unconfigured model or thinking level, or grant tool permissions.
|
|
195
181
|
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
"apiKey": "<rendered by chezmoi/1Password>",
|
|
201
|
-
"endpoint": "https://api.typesafe.ai/v1/systemone",
|
|
202
|
-
"model": "jev-1.13.0",
|
|
203
|
-
"timeoutMs": 1500,
|
|
204
|
-
"confidenceThreshold": 0.65,
|
|
205
|
-
"maxStateTokens": 3000,
|
|
206
|
-
"context": {
|
|
207
|
-
"previousTurns": 2,
|
|
208
|
-
"maxHistoryTokens": 500,
|
|
209
|
-
"toolResults": "last-error",
|
|
210
|
-
"maxToolTokens": 250
|
|
211
|
-
},
|
|
212
|
-
"mode": "advisory"
|
|
213
|
-
},
|
|
214
|
-
"profiles": {
|
|
215
|
-
"personal": {
|
|
216
|
-
"jev": { "enabled": true },
|
|
217
|
-
"high": { "model": "openai/gpt-5.4-pro", "thinking": "high" },
|
|
218
|
-
"medium": { "model": "google/gemini-flash-latest", "thinking": "medium" },
|
|
219
|
-
"low": { "model": "openai/gpt-5.4-nano", "thinking": "off" },
|
|
220
|
-
"micro": { "model": "openai/gpt-5.4-nano", "thinking": "off" }
|
|
221
|
-
}
|
|
222
|
-
}
|
|
223
|
-
}
|
|
224
|
-
```
|
|
225
|
-
|
|
226
|
-
The endpoint, model, timeout, confidence threshold, state limit and mode shown
|
|
227
|
-
above are defaults. Only HTTPS endpoints without embedded credentials, query
|
|
228
|
-
parameters or fragments are accepted. `timeoutMs` defaults to 1500 ms and must
|
|
229
|
-
be a positive finite number within Node's timer range (at most 2147483647 ms).
|
|
230
|
-
There is no product-level cap: 4000 or 5000 ms are valid if you prefer waiting
|
|
231
|
-
longer before falling back. It sets the total Jev advisory budget, including
|
|
232
|
-
request and response-body time; there is no separate 750 ms cap. Confidence must
|
|
233
|
-
be 0–1. `maxStateTokens` is an estimated preflight budget from 1–24000. The separate classifier-only
|
|
234
|
-
path keeps a 10-second bound. Neither path retries or starts generation after
|
|
235
|
-
caller cancellation.
|
|
236
|
-
|
|
237
|
-
If Jev frequently falls back because requests time out, try `"timeoutMs": 3000`
|
|
238
|
-
in your user config. Existing explicit values such as 750 remain unchanged;
|
|
239
|
-
remove the field or set it to 1500 to use the new default. Increasing the timeout
|
|
240
|
-
does not lower the confidence threshold or guarantee a different route. After
|
|
241
|
-
upgrading, start a new Pi session; use `/router thinking auto` to clear any
|
|
242
|
-
unwanted effort override in an existing session.
|
|
243
|
-
|
|
244
|
-
**External data:** Jev receives bounded text in three named JSON fields:
|
|
245
|
-
`currentRequest`, `recentDialogue` and `recentToolEvidence`, plus candidate
|
|
246
|
-
tier/model/thinking identifiers. The default includes up to two prior user turns
|
|
247
|
-
with their last text replies, and at most the last tool result of the immediately
|
|
248
|
-
previous turn **if Pi marks that result as an error**. Selection and head/tail
|
|
249
|
-
truncation are deterministic, with no keyword scoring or summarizer call. System
|
|
250
|
-
prompts, raw config, credentials from config, thinking blocks, tool-call arguments
|
|
251
|
-
and image/binary blocks are not extracted.
|
|
252
|
-
This is not a redaction service: text itself may contain secrets or private data,
|
|
253
|
-
including tool output. Approve this external-data handling before enabling a
|
|
254
|
-
profile, especially work. Short replies, other languages and imperfect sentences
|
|
255
|
-
are advisor input, not local intent branches. Semantic classification and confidence
|
|
256
|
-
are probabilistic, not a security sandbox; Pi owns tool permissions.
|
|
257
|
-
|
|
258
|
-
Jev classifies the **latest user request**, using earlier messages only as
|
|
259
|
-
context. Criteria describe the reasoning each tier supports, not just its name.
|
|
260
|
-
The objective is **quality-first**: prefer frontier reasoning when it can materially
|
|
261
|
-
improve correctness, completeness or reduce rework, even if a smaller model could
|
|
262
|
-
probably complete the task. Direct retrieval and mechanical work still favor
|
|
263
|
-
micro/low. This is semantic advice, not a local keyword or complexity heuristic.
|
|
264
|
-
Do not increase the context limit or lower the threshold just to raise confidence.
|
|
265
|
-
Confidence measures decisiveness across choices, **not** the chance that the
|
|
266
|
-
selected generation model will succeed. It is distinct from the selected option's
|
|
267
|
-
probability. See [Jev Choice](https://docs.typesafe.ai/primitives/choice).
|
|
268
|
-
|
|
269
|
-
Concurrent calls for the same turn share one Jev request and its original deadline.
|
|
270
|
-
A repeated same-turn call reuses the validated decision rather than reverting to
|
|
271
|
-
baseline. Cancelling one waiter does not cancel another; the transport is aborted
|
|
272
|
-
when no waiters remain. Each new user turn can choose a different backend and
|
|
273
|
-
thinking level. Tool continuations keep their validated route. The logical
|
|
274
|
-
`router/<profile>` stays selected throughout; this is not conversation-wide pinning.
|
|
275
|
-
|
|
276
|
-
### Context selection and tuning
|
|
277
|
-
|
|
278
|
-
Configure `jev.context` only in user config. Project Jev settings remain ignored;
|
|
279
|
-
profile privacy opt-in is still required. Partial context settings inherit defaults.
|
|
280
|
-
Invalid values or unknown context keys reject the Jev config with a value-free warning.
|
|
281
|
-
|
|
282
|
-
| Setting | Default | Meaning |
|
|
283
|
-
| --- | --- | --- |
|
|
284
|
-
| `maxStateTokens` | `3000` | Estimated selected-state token budget, including excerpt markers. Range 1–24000. |
|
|
285
|
-
| `context.previousTurns` | `2` | Previous user turns, each with its last non-empty assistant text reply. Integer 0–20; not transport-message count. |
|
|
286
|
-
| `context.maxHistoryTokens` | `500` | Shared estimated-token ceiling for prior dialogue, integer 0–24000. |
|
|
287
|
-
| `context.toolResults` | `"last-error"` | `"none"`, `"last"` or `"last-error"`. The latter includes the last result only when its native `isError` flag is true. |
|
|
288
|
-
| `context.maxToolTokens` | `250` | Estimated-token ceiling for that one tool result, integer 0–24000. |
|
|
289
|
-
|
|
290
|
-
Priority is current request → recent dialogue → tool evidence. Individual ceilings
|
|
291
|
-
never expand the total estimated-token budget. The full current request wins when
|
|
292
|
-
it fits; otherwise its beginning and end are kept. Prior turns also use head/tail
|
|
293
|
-
excerpts when needed, with `truncated: true`. Unused space need not be filled. The
|
|
294
|
-
20-turn cap also bounds JSON metadata overhead.
|
|
295
|
-
|
|
296
|
-
TypeSafe publishes Jev's post-response `usage.input_tokens`, but no tokenizer or
|
|
297
|
-
preflight count API. OpenAI tokenizers are not compatible substitutes: in a small
|
|
298
|
-
EN/RU/code/emoji calibration, `cl100k`/`o200k` underestimated actual Jev requests by
|
|
299
|
-
26–49%. The router therefore uses a documented conservative estimate: ASCII/4,
|
|
300
|
-
non-ASCII UTF-8 bytes/2, then a 10% margin. The serialized request adds 200 tokens
|
|
301
|
-
of measured envelope headroom. A 28000 estimated-request safety gate leaves room
|
|
302
|
-
below Jev's stricter 32k `state + longest question` limit; its 64k whole-request
|
|
303
|
-
limit is not binding for this single Choice question. See TypeSafe's
|
|
304
|
-
[model limits](https://docs.typesafe.ai/models) and
|
|
305
|
-
[long-context guidance](https://docs.typesafe.ai/model-jaggedness/jev-1.13).
|
|
306
|
-
|
|
307
|
-
Only the last tool result of the immediately previous user turn is eligible, even
|
|
308
|
-
when more dialogue turns are selected. `last-error` does not parse stdout for words
|
|
309
|
-
such as `ERROR`, search backwards for an old failure, or resurrect a failure after
|
|
310
|
-
a later successful result. A tool can report a meaningful failure as ordinary text
|
|
311
|
-
with `isError: false`; choose `last` when that distinction matters. Tool arguments
|
|
312
|
-
are always excluded. Empty/thinking/tool-call-only assistant messages cannot consume
|
|
313
|
-
dialogue slots. Older intermediate assistant narration is not selected.
|
|
314
|
-
|
|
315
|
-
Suggested overrides (merge into `jev.context`):
|
|
316
|
-
|
|
317
|
-
- **Independent tasks:** `{"previousTurns": 0, "toolResults": "none"}`.
|
|
318
|
-
- **Dialogue only:** `{"previousTurns": 2, "toolResults": "none"}`.
|
|
319
|
-
- **Tool-heavy diagnosis:** `{"toolResults": "last", "maxToolTokens": 500}`.
|
|
320
|
-
- **Longer follow-ups:** `{"previousTurns": 4, "maxHistoryTokens": 1000}`.
|
|
321
|
-
|
|
322
|
-
The default is a conservative data-selection compromise from live experiments,
|
|
323
|
-
not a guarantee of higher confidence. Larger windows did not consistently help;
|
|
324
|
-
the most useful clear improvement was retaining a request at the end of long text.
|
|
325
|
-
Some short/ambiguous follow-ups still use the configured baseline. See
|
|
326
|
-
[context experiments](docs/JEV-CONTEXT-VALIDATION.md) for results and limitations.
|
|
327
|
-
|
|
328
|
-
`/router status` shows effective settings. Widget/debug show estimated current/
|
|
329
|
-
history/tool tokens, included turns/results, truncated blocks, the estimated full
|
|
330
|
-
request and Jev's actual post-response input usage. Character counts may remain in
|
|
331
|
-
persisted diagnostics for compatibility, but are not configuration budgets. No
|
|
332
|
-
selected text is added to these metrics. This affects **Jev only**: generation still receives
|
|
333
|
-
Pi's normal context. The separate Pi-classifier compatibility path is unchanged.
|
|
334
|
-
|
|
335
|
-
### Quality-first fallback
|
|
336
|
-
|
|
337
|
-
If avoiding underpowered answers matters more than extra cost/latency, set
|
|
338
|
-
`"baselineTier": "high"` in an existing profile with a configured high tier:
|
|
182
|
+
Enable it only in the user config, with a global `jev.enabled`, an API key and an
|
|
183
|
+
explicit per-profile opt-in. Project-level `jev` settings are ignored with a
|
|
184
|
+
warning. Enabling a profile is approval to send its bounded conversation text,
|
|
185
|
+
including tool output, to TypeSafe; the router does not redact it.
|
|
339
186
|
|
|
340
187
|
```json
|
|
341
188
|
{
|
|
342
|
-
"
|
|
343
|
-
|
|
344
|
-
"baselineTier": "high"
|
|
345
|
-
}
|
|
346
|
-
}
|
|
189
|
+
"jev": { "enabled": true, "apiKey": "<rendered by chezmoi/1Password>" },
|
|
190
|
+
"profiles": { "personal": { "jev": { "enabled": true } } }
|
|
347
191
|
}
|
|
348
192
|
```
|
|
349
193
|
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
194
|
+
Everything else has defaults: `jev-1.13.0`, a 1500 ms total budget with one
|
|
195
|
+
transient retry, a 0.65 confidence threshold, a 0.8 probability threshold and a
|
|
196
|
+
3000-token context budget of the current request, two prior turns and the last
|
|
197
|
+
native-error tool result. A Choice below the confidence threshold is resolved
|
|
198
|
+
from its full probability distribution rather than discarded; an `uncertain`
|
|
199
|
+
answer, a timeout or an error uses the local baseline directly. See
|
|
200
|
+
[docs/jev-advisor.md](docs/jev-advisor.md) for the full configuration reference,
|
|
201
|
+
context tuning, the acceptance policy, footer/widget diagnostics and
|
|
202
|
+
troubleshooting.
|
|
203
|
+
|
|
204
|
+
### Routing diagnostics
|
|
205
|
+
|
|
206
|
+
The footer shows profile, tier, model/thinking and the advisor outcome, for
|
|
207
|
+
example `🧭 Jev → high c91% · 807ms`. `ui.statusLine: "detailed"` adds the
|
|
208
|
+
top option's probability and the request start time. When a configured
|
|
209
|
+
advisor is not asked, the footer says why: `advice skipped: pinned high`,
|
|
210
|
+
`over budget`, `only high eligible`, `tool turn`. `/router` and
|
|
211
|
+
`/router widget` show full metrics; `/router log on` keeps the last 50
|
|
212
|
+
decisions in branch-safe session state and `/router log` summarizes them.
|
|
213
|
+
Field meanings, outcome codes and fixes are in
|
|
214
|
+
[docs/jev-advisor.md](docs/jev-advisor.md#diagnostics).
|
|
358
215
|
|
|
359
|
-
|
|
360
|
-
{
|
|
361
|
-
"ui": { "statusLine": "compact" }
|
|
362
|
-
}
|
|
363
|
-
```
|
|
216
|
+
## Commands
|
|
364
217
|
|
|
365
|
-
|
|
366
|
-
and latency. Omits the repeated provider prefix to fit split panes.
|
|
367
|
-
Examples: `🧭 Jev → high c91% · 807ms`,
|
|
368
|
-
`🧭 Jev high c35% <65% → baseline · 764ms`,
|
|
369
|
-
`🧭 Jev: no tier chosen → baseline · 860ms`,
|
|
370
|
-
`🧭 Jev: timeout → baseline · 5.0s`.
|
|
371
|
-
- **`detailed`:** adds selected probability and local request-start time. Example:
|
|
372
|
-
`🧭 Jev high c35% <65% → baseline · 764ms · p48% @18:34:49`.
|
|
373
|
-
Use this on wide terminals; long model/profile names can truncate a footer.
|
|
374
|
-
- **Widget / status:** `/router widget on` or `/router status` shows full metrics,
|
|
375
|
-
including the Jev model label, HTTP status, candidate count, estimated context/request tokens and actual server input usage.
|
|
376
|
-
- **History:** `/router debug on`, then `/router debug show`. The last 50 decisions
|
|
377
|
-
are saved in branch-safe `router-state` session entries and restored on resume.
|
|
378
|
-
Debug off stops collecting history; the latest decision still persists.
|
|
379
|
-
- **Statistics:** `/router debug stats` reports unique HTTP requests, advised tiers,
|
|
380
|
-
outcome counts/rates and median latency. Statistics cover only the retained
|
|
381
|
-
decision window, **not session lifetime**. Locally generated request IDs deduplicate
|
|
382
|
-
shared requests, cached routes and tool continuations, including after resume.
|
|
383
|
-
Older decisions without IDs are excluded. `/router debug clear` clears the window.
|
|
384
|
-
|
|
385
|
-
`c` is confidence; `p` is the selected option's probability. `<65%` explains a
|
|
386
|
-
confidence rejection. `ms`/`s` is local request-to-validated-result time, not pure model inference
|
|
387
|
-
time. `@` is the original request's local start time. `reuse` / `tool route` means
|
|
388
|
-
no new Jev request: the displayed metrics belong to the original routing attempt.
|
|
389
|
-
`baseline` (or `base` in older traces) means deterministic local baseline, not necessarily the medium tier.
|
|
390
|
-
`no tier chosen` means Jev could not judge the required capability from the supplied
|
|
391
|
-
context. It does not prove that context was missing. Compact mode omits abstention
|
|
392
|
-
scores; widget/debug label them `abstention-confidence` and `abstention-p`, not
|
|
393
|
-
confidence in the generation model. The acceptance threshold is not applied to abstention.
|
|
394
|
-
`local baseline` / `advice bypassed` distinguishes no advisor from a rejected answer.
|
|
395
|
-
|
|
396
|
-
Failures are distinguished as `low-confidence`, `uncertain`, `invalid-response`,
|
|
397
|
-
`http-error`, `network-error`, `deadline`, `cancelled` or `unavailable`. A quick
|
|
398
|
-
low-confidence rejection is **not a timeout**; increasing timeout will not fix it.
|
|
399
|
-
Only validated choices, numeric diagnostics, recognized version labels and locally
|
|
400
|
-
generated request IDs are retained. State/debug never
|
|
401
|
-
retain the Jev key, endpoint, request text, raw response or remote explanations.
|
|
402
|
-
Older explanations are discarded as non-rendered `legacy` metadata; Pi's own
|
|
403
|
-
conversation transcript is separate from router state.
|
|
404
|
-
|
|
405
|
-
For chezmoi, use a **private template**, for example
|
|
406
|
-
`private_model-router.json.tmpl` under your agent-directory source path. Render
|
|
407
|
-
only the `apiKey` value using a reference such as
|
|
408
|
-
`{{ onepasswordRead "op://Personal/TypeSafe/apiKey" | toJson }}` (unquoted in the
|
|
409
|
-
JSON template). Adapt the vault/item reference locally. Keep the rendered file
|
|
410
|
-
out of Git and restrict permissions to `0600` (`chmod 600` on Unix); verify the
|
|
411
|
-
mode without printing the file. Never commit rendered credentials or 1Password
|
|
412
|
-
output. No environment variable is required, and the extension never executes a
|
|
413
|
-
secret-lookup command. The repository example contains only a placeholder and
|
|
414
|
-
keeps Jev disabled.
|
|
218
|
+
One verb per concern. A verb without an argument shows the state it controls.
|
|
415
219
|
|
|
416
|
-
|
|
220
|
+
| Command | Description |
|
|
221
|
+
| --- | --- |
|
|
222
|
+
| `/router` | Status: profile, pin, thinking override, cost, Jev settings, last decision. |
|
|
223
|
+
| `/router <profile>` | Switch profile and enable the router. Same as `/model router/<profile>`. |
|
|
224
|
+
| `/router off` | Leave the router and restore the previous non-router model. |
|
|
225
|
+
| `/router pin <tier\|auto>` | Pin the active profile to `high`, `medium`, `low` or `micro`; `auto` clears. |
|
|
226
|
+
| `/router thinking <level\|auto>` | Override thinking for every tier; `auto` clears. Pi's own thinking selector applies the same override. Per-tier levels belong in the profile config. An override that leaves no eligible route is rejected. |
|
|
227
|
+
| `/router log [on\|off\|clear]` | Recent decisions and Jev statistics for the retained window; `on`/`off` control collection (last 50, saved in session state), `clear` forgets them. |
|
|
228
|
+
| `/router widget` | Toggle the status widget. |
|
|
229
|
+
| `/router reload` | Reload `model-router.json`. |
|
|
230
|
+
| `/router help` | Usage. |
|
|
417
231
|
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
| `/router` | Show detailed status, current profile, spend, and settings. |
|
|
421
|
-
| `/router status` | Alias for `/router` (show current status). |
|
|
422
|
-
| `/router profile [name]` | Switch to a profile or list available ones (enables router if off). |
|
|
423
|
-
| `/router pin <t\|a>` | Pin a tier (high/medium/low/micro/auto) for the active profile. |
|
|
424
|
-
| `/router fix <tier>` | Correct the _last_ decision and pin that tier for the current profile. |
|
|
425
|
-
| `/router thinking <level>` | Override thinking level for all tiers (e.g. `/router thinking max`). Unsupported tiers are skipped; an override that leaves no eligible route is rejected without changing any tier. |
|
|
426
|
-
| `/router thinking <tier> <level>` | Override thinking level for a specific tier (e.g. `/router thinking low off`). |
|
|
427
|
-
| `/router disable` | Disable the router and switch back to the last non-router model. |
|
|
428
|
-
| `/router widget <on\|off>` | Toggle the persistent state widget (supports `toggle`). |
|
|
429
|
-
| `/router debug <on\|off>` | Toggle router debug state; use `show` or `clear` for local decision history. |
|
|
430
|
-
| `/router debug stats` | Deduplicated Jev counts, advised tiers, fallback rates and median latency for retained history. |
|
|
431
|
-
| `/router reload` | Hot-reload the configuration JSON. |
|
|
432
|
-
| `/router help` | Show usage help for all subcommands. |
|
|
232
|
+
Removed verbs (`status`, `profile`, `fix`, `disable`, `debug`, `?`, per-tier
|
|
233
|
+
`thinking`) answer with their replacement and do nothing.
|
|
433
234
|
|
|
434
235
|
## Documentation
|
|
435
236
|
|
|
436
|
-
- [
|
|
437
|
-
- [
|
|
237
|
+
- [docs/README.md](docs/README.md): index of all project documents.
|
|
238
|
+
- [docs/architecture.md](docs/architecture.md): routing flow, module boundaries, state and persistence.
|
|
239
|
+
- [docs/jev-advisor.md](docs/jev-advisor.md): Jev configuration, context tuning, acceptance policy, diagnostics and troubleshooting.
|
|
240
|
+
- [docs/releasing.md](docs/releasing.md): release procedure.
|
|
241
|
+
- [docs/research/](docs/research/): experiments behind the current defaults.
|
|
242
|
+
- [model-router.example.json](model-router.example.json): sample profiles.
|
package/extensions/classifier.ts
CHANGED
|
@@ -5,10 +5,10 @@ import type {
|
|
|
5
5
|
} from '@earendil-works/pi-ai';
|
|
6
6
|
import type { ExtensionContext } from '@earendil-works/pi-coding-agent';
|
|
7
7
|
import { isRouterTier, parseCanonicalModelRef } from './config';
|
|
8
|
+
import { DEFAULT_CLASSIFIER_TIMEOUT_MS } from './constants';
|
|
8
9
|
import { extractTextFromContent, getBoundedRecentContext } from './context';
|
|
9
10
|
import type { ClassifierTier, RouterPhase } from './types';
|
|
10
11
|
|
|
11
|
-
const CLASSIFIER_TIMEOUT_MS = 10_000;
|
|
12
12
|
const CLASSIFIER_MAX_TOKENS = 256;
|
|
13
13
|
|
|
14
14
|
export const runClassifier = async (
|
|
@@ -18,7 +18,7 @@ export const runClassifier = async (
|
|
|
18
18
|
currentPhase?: RouterPhase,
|
|
19
19
|
thinking?: ThinkingLevel,
|
|
20
20
|
signal?: AbortSignal,
|
|
21
|
-
routingDeadline = performance.now() +
|
|
21
|
+
routingDeadline = performance.now() + DEFAULT_CLASSIFIER_TIMEOUT_MS,
|
|
22
22
|
): Promise<{ tier: ClassifierTier } | undefined> => {
|
|
23
23
|
try {
|
|
24
24
|
const remaining = routingDeadline - performance.now();
|
|
@@ -51,9 +51,8 @@ export const runClassifier = async (
|
|
|
51
51
|
},
|
|
52
52
|
],
|
|
53
53
|
};
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
);
|
|
54
|
+
// The caller's deadline already reflects the configured classifier budget.
|
|
55
|
+
const timeout = AbortSignal.timeout(Math.max(1, Math.ceil(remaining)));
|
|
57
56
|
const classifierSignal = signal
|
|
58
57
|
? AbortSignal.any([signal, timeout])
|
|
59
58
|
: timeout;
|