@alexeiled/pi-model-router 0.6.0 → 0.6.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +14 -0
- package/README.md +24 -9
- package/extensions/commands.ts +3 -0
- package/extensions/config.ts +6 -7
- package/extensions/index.ts +32 -1
- package/extensions/provider.ts +107 -84
- package/extensions/state.ts +3 -1
- package/extensions/types.ts +12 -0
- package/extensions/ui.ts +49 -10
- package/model-router.example.json +1 -1
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.6.2 - 2026-09-21
|
|
4
|
+
|
|
5
|
+
- Increase the default Jev timeout from 750 ms to 1500 ms. User-level `jev.timeoutMs` now sets the total advisory budget without the previous hidden 750 ms cap or an arbitrary upper cap. Positive finite values within Node's timer range are accepted, including 4000 and 5000 ms. Existing explicit shorter timeouts remain valid.
|
|
6
|
+
- Fix delayed Pi thinking-display events being mistaken for user changes and overriding every routing tier. Internal display updates now preserve configured per-tier effort; explicit user overrides still apply.
|
|
7
|
+
- Keep privacy opt-in, confidence validation, caller cancellation, and direct baseline fallback unchanged. No additional advisory requests or retries.
|
|
8
|
+
- Add regression coverage for delayed valid Jev responses, configurable deadlines, internal thinking events, and Astra/Luna candidate selection. Align configuration examples and operator documentation.
|
|
9
|
+
|
|
10
|
+
## 0.6.1 — 2026-09-21
|
|
11
|
+
|
|
12
|
+
- Add compact human-readable route provenance to the Pi footer, widget and `/router status`: `🧭 Jev ✓` means Jev selected the route, `🧭 Jev ↪ base` means Jev ran but the local baseline was used, and no marker means Jev was not involved.
|
|
13
|
+
- Track closed route-guidance outcomes across Jev, classifier, bypass and baseline paths, including reusable tool continuations and explicit generation fallbacks without persisting remote text or credentials.
|
|
14
|
+
- Keep the normal footer uncluttered for turns without external route guidance; retain latency only as a short widget/status diagnostic.
|
|
15
|
+
- Add regression coverage for UI rendering, persistence compatibility, advisor fallback, continuation inheritance and privacy boundaries.
|
|
16
|
+
|
|
3
17
|
## 0.6.0 — 2026-09-21
|
|
4
18
|
|
|
5
19
|
- Add the optional `micro` tier with `off` thinking by default; all four tiers are configured model/effort choices, not security permissions. Existing three-tier and partial profiles remain supported.
|
package/README.md
CHANGED
|
@@ -95,7 +95,7 @@ pi -e ./extensions/index.ts
|
|
|
95
95
|
|
|
96
96
|
- Generation and classification use Pi's provider registry, including native/custom providers and credential-specific URLs. Only the optional Jev advisor uses separate HTTPS transport.
|
|
97
97
|
- Fallbacks run only before content is emitted; cancellation does not retry. Every target must support the requested input and exact thinking level; explicit unsupported effort is not silently reduced. Omitted thinking defaults to `off` for non-reasoning targets, including fallbacks.
|
|
98
|
-
- Jev gets
|
|
98
|
+
- Jev gets a configurable total advisory budget via `jev.timeoutMs`: 1500 ms by default, with no additional routing cap or retry. The separate classifier-only compatibility path retains its 10-second bound and 256-token output limit. Failure or uncertainty means eligible baseline; caller cancellation stops generation.
|
|
99
99
|
- Valid same-turn tool continuations reuse the actual prior route before either advisor. Pins, budget policy and a single eligible primary candidate also bypass advisors. Invalid continuations choose a compatible local route without advice; incompatible Google thought-signature replay fails plainly.
|
|
100
100
|
- Pi owns tool execution permissions and per-request authentication. The router checks configured provider/profile identity, not which backend login is currently behind a provider. No private authentication storage is read.
|
|
101
101
|
- Context trimming preserves system instructions and whole active tool turns. It is a text estimate, not a guarantee that images or a large active turn fit.
|
|
@@ -199,7 +199,7 @@ profile opt-ins, are ignored with a warning, before merging user credentials.
|
|
|
199
199
|
"apiKey": "<rendered by chezmoi/1Password>",
|
|
200
200
|
"endpoint": "https://api.typesafe.ai/v1/systemone",
|
|
201
201
|
"model": "jev-1.13.0",
|
|
202
|
-
"timeoutMs":
|
|
202
|
+
"timeoutMs": 1500,
|
|
203
203
|
"confidenceThreshold": 0.65,
|
|
204
204
|
"maxStateChars": 12000,
|
|
205
205
|
"mode": "advisory"
|
|
@@ -218,14 +218,22 @@ profile opt-ins, are ignored with a warning, before merging user credentials.
|
|
|
218
218
|
|
|
219
219
|
The endpoint, model, timeout, confidence threshold, state limit and mode shown
|
|
220
220
|
above are defaults. Only HTTPS endpoints without embedded credentials, query
|
|
221
|
-
parameters or fragments are accepted.
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
221
|
+
parameters or fragments are accepted. `timeoutMs` defaults to 1500 ms and must
|
|
222
|
+
be a positive finite number within Node's timer range (at most 2147483647 ms).
|
|
223
|
+
There is no product-level cap: 4000 or 5000 ms are valid if you prefer waiting
|
|
224
|
+
longer before falling back. It sets the total Jev advisory budget, including
|
|
225
|
+
request and response-body time; there is no separate 750 ms cap. Confidence must
|
|
226
|
+
be 0–1, and the context limit must be 1–12000 characters. The separate classifier-only
|
|
226
227
|
path keeps a 10-second bound. Neither path retries or starts generation after
|
|
227
228
|
caller cancellation.
|
|
228
229
|
|
|
230
|
+
If Jev frequently falls back because requests time out, try `"timeoutMs": 3000`
|
|
231
|
+
in your user config. Existing explicit values such as 750 remain unchanged;
|
|
232
|
+
remove the field or set it to 1500 to use the new default. Increasing the timeout
|
|
233
|
+
does not lower the confidence threshold or guarantee a different route. After
|
|
234
|
+
upgrading, start a new Pi session; use `/router thinking auto` to clear any
|
|
235
|
+
unwanted effort override in an existing session.
|
|
236
|
+
|
|
229
237
|
**External data:** Jev receives bounded, role-labelled recent user/assistant/tool
|
|
230
238
|
text, prioritizing the latest user request within `maxStateChars`, plus candidate
|
|
231
239
|
tier/model/thinking identifiers. Truncation is deterministic, with no keyword
|
|
@@ -244,6 +252,13 @@ request text, raw response or remote explanations. Older saved explanations are
|
|
|
244
252
|
discarded as non-rendered `legacy` metadata; Pi's own conversation transcript is
|
|
245
253
|
separate from router state.
|
|
246
254
|
|
|
255
|
+
**Footer and widget:** The footer stays in its normal route-only form when Jev is
|
|
256
|
+
not involved. When Jev selects the route it adds `· 🧭 Jev ✓`; when Jev is used
|
|
257
|
+
but its advice is rejected, it adds `· 🧭 Jev ↪ base`. `base` means the local
|
|
258
|
+
deterministic baseline; the selected tier and model remain visible in the route
|
|
259
|
+
text. `/router widget on` and `/router status` show the same marker and a short
|
|
260
|
+
latency value. No marker means no external route guidance was used.
|
|
261
|
+
|
|
247
262
|
For chezmoi, use a **private template**, for example
|
|
248
263
|
`private_model-router.json.tmpl` under your agent-directory source path. Render
|
|
249
264
|
only the `apiKey` value using a reference such as
|
|
@@ -268,11 +283,11 @@ keeps Jev disabled.
|
|
|
268
283
|
| `/router thinking <tier> <level>` | Override thinking level for a specific tier (e.g. `/router thinking low off`). |
|
|
269
284
|
| `/router disable` | Disable the router and switch back to the last non-router model. |
|
|
270
285
|
| `/router widget <on\|off>` | Toggle the persistent state widget (supports `toggle`). |
|
|
271
|
-
| `/router debug <on\|off>` | Toggle
|
|
286
|
+
| `/router debug <on\|off>` | Toggle router debug state; use `show` or `clear` for local decision history. |
|
|
272
287
|
| `/router reload` | Hot-reload the configuration JSON. |
|
|
273
288
|
| `/router help` | Show usage help for all subcommands. |
|
|
274
289
|
|
|
275
290
|
## Documentation
|
|
276
291
|
|
|
277
292
|
- [Architecture Guide](docs/ARCHITECTURE.md): Deep dive into the routing logic and modular design.
|
|
278
|
-
- [Sample Configuration](model-router.example.json):
|
|
293
|
+
- [Sample Configuration](model-router.example.json): Profile examples (`auto`, `cheap`, `deep`, `anthropic`).
|
package/extensions/commands.ts
CHANGED
|
@@ -24,6 +24,7 @@ import type {
|
|
|
24
24
|
RoutingDecision,
|
|
25
25
|
} from './types';
|
|
26
26
|
import {
|
|
27
|
+
formatAdvisorDetail,
|
|
27
28
|
formatDecision,
|
|
28
29
|
formatDecisionSource,
|
|
29
30
|
formatModelRef,
|
|
@@ -183,6 +184,7 @@ export const registerCommands = (
|
|
|
183
184
|
`Debug history: ${state.debugHistory.length} decisions`,
|
|
184
185
|
];
|
|
185
186
|
if (state.lastDecision) {
|
|
187
|
+
const advisorDetail = formatAdvisorDetail(state.lastDecision);
|
|
186
188
|
lines.push(
|
|
187
189
|
`Last routed tier: ${state.lastDecision.tier}`,
|
|
188
190
|
`Last phase: ${state.lastDecision.phase}`,
|
|
@@ -190,6 +192,7 @@ export const registerCommands = (
|
|
|
190
192
|
...(formatDecisionSource(state.lastDecision)
|
|
191
193
|
? [`Reason: ${formatDecisionSource(state.lastDecision)}`]
|
|
192
194
|
: []),
|
|
195
|
+
...(advisorDetail ? [advisorDetail] : []),
|
|
193
196
|
);
|
|
194
197
|
}
|
|
195
198
|
if (state.lastConfigWarnings && state.lastConfigWarnings.length > 0) {
|
package/extensions/config.ts
CHANGED
|
@@ -376,10 +376,13 @@ export const normalizeTierConfig = (
|
|
|
376
376
|
};
|
|
377
377
|
};
|
|
378
378
|
|
|
379
|
+
// Node turns larger setTimeout delays into 1 ms rather than waiting longer.
|
|
380
|
+
const MAX_TIMER_DELAY_MS = 2_147_483_647;
|
|
381
|
+
|
|
379
382
|
export const DEFAULT_JEV_CONFIG = {
|
|
380
383
|
endpoint: 'https://api.typesafe.ai/v1/systemone',
|
|
381
384
|
model: 'jev-1.13.0',
|
|
382
|
-
timeoutMs:
|
|
385
|
+
timeoutMs: 1500,
|
|
383
386
|
confidenceThreshold: 0.65,
|
|
384
387
|
maxStateChars: 12000,
|
|
385
388
|
mode: 'advisory',
|
|
@@ -420,7 +423,7 @@ export const normalizeJevConfig = (
|
|
|
420
423
|
typeof value.timeoutMs !== 'number' ||
|
|
421
424
|
!Number.isFinite(value.timeoutMs) ||
|
|
422
425
|
value.timeoutMs <= 0 ||
|
|
423
|
-
value.timeoutMs >
|
|
426
|
+
value.timeoutMs > MAX_TIMER_DELAY_MS ||
|
|
424
427
|
typeof value.confidenceThreshold !== 'number' ||
|
|
425
428
|
!Number.isFinite(value.confidenceThreshold) ||
|
|
426
429
|
value.confidenceThreshold < 0 ||
|
|
@@ -434,10 +437,6 @@ export const normalizeJevConfig = (
|
|
|
434
437
|
(typeof value.apiKey !== 'string' || /[\r\n]/.test(value.apiKey)))
|
|
435
438
|
)
|
|
436
439
|
return invalid();
|
|
437
|
-
if (value.timeoutMs > 750)
|
|
438
|
-
warnings.push(
|
|
439
|
-
'Jev timeoutMs clamped to the effective 750 ms provider cap.',
|
|
440
|
-
);
|
|
441
440
|
const apiKey = typeof value.apiKey === 'string' ? value.apiKey.trim() : '';
|
|
442
441
|
if (value.enabled === true && !apiKey) {
|
|
443
442
|
warnings.push('Jev disabled: missing user-config API key.');
|
|
@@ -447,7 +446,7 @@ export const normalizeJevConfig = (
|
|
|
447
446
|
apiKey,
|
|
448
447
|
endpoint: value.endpoint,
|
|
449
448
|
model: value.model,
|
|
450
|
-
timeoutMs:
|
|
449
|
+
timeoutMs: value.timeoutMs,
|
|
451
450
|
confidenceThreshold: value.confidenceThreshold,
|
|
452
451
|
maxStateChars: value.maxStateChars,
|
|
453
452
|
mode: 'advisory',
|
package/extensions/index.ts
CHANGED
|
@@ -57,6 +57,10 @@ const routerExtension = (pi: ExtensionAPI) => {
|
|
|
57
57
|
let isInitialized = false;
|
|
58
58
|
let isInternalModelSwitch = false;
|
|
59
59
|
let isInternalThinkingChange = false;
|
|
60
|
+
const pendingThinkingChanges: Array<{
|
|
61
|
+
previousLevel: ThinkingLevel;
|
|
62
|
+
level: ThinkingLevel;
|
|
63
|
+
}> = [];
|
|
60
64
|
let ignoreStartupThinkingEvent = false;
|
|
61
65
|
|
|
62
66
|
const runtimeState = {
|
|
@@ -142,13 +146,27 @@ const routerExtension = (pi: ExtensionAPI) => {
|
|
|
142
146
|
};
|
|
143
147
|
|
|
144
148
|
const setThinkingLevelInternally = (level: ThinkingLevel) => {
|
|
149
|
+
let previousLevel: ThinkingLevel;
|
|
150
|
+
try {
|
|
151
|
+
previousLevel = pi.getThinkingLevel();
|
|
152
|
+
} catch {
|
|
153
|
+
return; // The session runtime may already be torn down.
|
|
154
|
+
}
|
|
155
|
+
const change = { previousLevel, level };
|
|
156
|
+
pendingThinkingChanges.push(change);
|
|
145
157
|
isInternalThinkingChange = true;
|
|
146
158
|
try {
|
|
147
159
|
pi.setThinkingLevel(level);
|
|
160
|
+
change.level = pi.getThinkingLevel();
|
|
148
161
|
} catch {
|
|
149
162
|
// Extension context may be stale after session teardown.
|
|
163
|
+
change.level = change.previousLevel;
|
|
150
164
|
} finally {
|
|
151
165
|
isInternalThinkingChange = false;
|
|
166
|
+
if (change.level === change.previousLevel) {
|
|
167
|
+
const index = pendingThinkingChanges.indexOf(change);
|
|
168
|
+
if (index >= 0) pendingThinkingChanges.splice(index, 1);
|
|
169
|
+
}
|
|
152
170
|
}
|
|
153
171
|
};
|
|
154
172
|
|
|
@@ -513,8 +531,21 @@ const routerExtension = (pi: ExtensionAPI) => {
|
|
|
513
531
|
|
|
514
532
|
pi.on('thinking_level_select', (event, ctx) => {
|
|
515
533
|
ensureInitializedFromContext(ctx);
|
|
534
|
+
if (isInternalThinkingChange) {
|
|
535
|
+
pendingThinkingChanges.pop();
|
|
536
|
+
return;
|
|
537
|
+
}
|
|
538
|
+
// Pi emits without awaiting extension handlers; earlier handlers can delay this echo.
|
|
539
|
+
const internalChange = pendingThinkingChanges.findIndex(
|
|
540
|
+
(change) =>
|
|
541
|
+
change.previousLevel === event.previousLevel &&
|
|
542
|
+
change.level === event.level,
|
|
543
|
+
);
|
|
544
|
+
if (internalChange >= 0) {
|
|
545
|
+
pendingThinkingChanges.splice(internalChange, 1);
|
|
546
|
+
return;
|
|
547
|
+
}
|
|
516
548
|
if (!isInitialized || !routerEnabled || !selectedProfile) return;
|
|
517
|
-
if (isInternalThinkingChange) return;
|
|
518
549
|
if (ignoreStartupThinkingEvent) {
|
|
519
550
|
ignoreStartupThinkingEvent = false;
|
|
520
551
|
return;
|
package/extensions/provider.ts
CHANGED
|
@@ -39,6 +39,7 @@ import {
|
|
|
39
39
|
selectBaselineRoute,
|
|
40
40
|
} from './routing';
|
|
41
41
|
import type {
|
|
42
|
+
AdvisorOutcome,
|
|
42
43
|
RouterConfig,
|
|
43
44
|
RouterPinByProfile,
|
|
44
45
|
RouterThinkingByProfile,
|
|
@@ -279,7 +280,7 @@ export const registerRouterProvider = (
|
|
|
279
280
|
// Streams can complete out of order. Keep a small turn-keyed history rather
|
|
280
281
|
// than letting the latest stream replace another stream's continuation.
|
|
281
282
|
const continuations = new Map<string, ContinuationRecord>();
|
|
282
|
-
const advisedTurns = new Map<string,
|
|
283
|
+
const advisedTurns = new Map<string, AdvisorOutcome>();
|
|
283
284
|
const rememberContinuation = (record: ContinuationRecord) => {
|
|
284
285
|
continuations.delete(record.turn);
|
|
285
286
|
continuations.set(record.turn, record);
|
|
@@ -289,9 +290,9 @@ export const registerRouterProvider = (
|
|
|
289
290
|
continuations.delete(oldest);
|
|
290
291
|
}
|
|
291
292
|
};
|
|
292
|
-
const rememberAdvisedTurn = (turn: string) => {
|
|
293
|
+
const rememberAdvisedTurn = (turn: string, outcome: AdvisorOutcome) => {
|
|
293
294
|
advisedTurns.delete(turn);
|
|
294
|
-
advisedTurns.set(turn,
|
|
295
|
+
advisedTurns.set(turn, outcome);
|
|
295
296
|
while (advisedTurns.size > 16) {
|
|
296
297
|
const oldest = advisedTurns.keys().next().value;
|
|
297
298
|
if (oldest === undefined) break;
|
|
@@ -391,6 +392,14 @@ export const registerRouterProvider = (
|
|
|
391
392
|
]);
|
|
392
393
|
const toolContinuation =
|
|
393
394
|
context.messages.at(-1)?.role === 'toolResult';
|
|
395
|
+
const jev = state.currentConfig.jev;
|
|
396
|
+
const useJev = Boolean(
|
|
397
|
+
jev?.enabled &&
|
|
398
|
+
profile.jev?.enabled &&
|
|
399
|
+
jev.apiKey.trim().length > 0,
|
|
400
|
+
);
|
|
401
|
+
const advisorConfigured =
|
|
402
|
+
useJev || Boolean(state.currentConfig.classifierModel);
|
|
394
403
|
const continuationRecord =
|
|
395
404
|
toolContinuation && turn ? continuations.get(turn) : undefined;
|
|
396
405
|
const continuationDecision = continuationRecord?.decision;
|
|
@@ -459,6 +468,11 @@ export const registerRouterProvider = (
|
|
|
459
468
|
baseline.reasonCode,
|
|
460
469
|
);
|
|
461
470
|
decision.isBudgetForced = baseline.isBudgetForced;
|
|
471
|
+
decision.advisor = advisorConfigured ? 'bypassed' : 'none';
|
|
472
|
+
if (!toolContinuation && turn) {
|
|
473
|
+
const previousAdvisor = advisedTurns.get(turn);
|
|
474
|
+
if (previousAdvisor) decision.advisor = previousAdvisor;
|
|
475
|
+
}
|
|
462
476
|
}
|
|
463
477
|
|
|
464
478
|
// Tool results never invoke advisors, even when their prior route cannot be reused.
|
|
@@ -469,99 +483,105 @@ export const registerRouterProvider = (
|
|
|
469
483
|
user &&
|
|
470
484
|
turn &&
|
|
471
485
|
!advisedTurns.has(turn) &&
|
|
472
|
-
|
|
473
|
-
(state.currentConfig.jev?.enabled && profile.jev?.enabled))
|
|
486
|
+
advisorConfigured
|
|
474
487
|
) {
|
|
475
|
-
rememberAdvisedTurn(turn);
|
|
476
488
|
const started = performance.now();
|
|
477
|
-
const
|
|
478
|
-
|
|
479
|
-
jev?.enabled &&
|
|
480
|
-
profile.jev?.enabled &&
|
|
481
|
-
jev.apiKey.trim().length > 0;
|
|
482
|
-
const routingDeadline = started + (useJev ? 1500 : 10_000);
|
|
489
|
+
const routingDeadline =
|
|
490
|
+
started + (useJev && jev ? jev.timeoutMs : 10_000);
|
|
483
491
|
const candidates = primaryRoutePairs(profile, pairs).map(
|
|
484
492
|
createJevCandidate,
|
|
485
493
|
);
|
|
486
494
|
// A single primary bypasses advice, not a baseline's eligible fallback.
|
|
487
|
-
if (candidates.length
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
taskSummary: getBoundedRecentContext(
|
|
496
|
-
context,
|
|
497
|
-
jev.maxStateChars,
|
|
498
|
-
),
|
|
499
|
-
candidates,
|
|
500
|
-
profile: profile.jev,
|
|
501
|
-
routingDeadline,
|
|
502
|
-
signal: options?.signal,
|
|
503
|
-
},
|
|
504
|
-
).catch(() => undefined);
|
|
505
|
-
options?.signal?.throwIfAborted();
|
|
506
|
-
// Re-read registry capabilities after the network boundary.
|
|
507
|
-
pairs = available();
|
|
508
|
-
const candidate = candidates.find(
|
|
509
|
-
(entry) => entry.id === advice?.candidateId,
|
|
510
|
-
);
|
|
511
|
-
if (
|
|
512
|
-
candidate &&
|
|
513
|
-
performance.now() < routingDeadline &&
|
|
514
|
-
pairs.some(
|
|
515
|
-
(pair) =>
|
|
516
|
-
pair.model === candidate.model &&
|
|
517
|
-
pair.tier === candidate.tier &&
|
|
518
|
-
pair.thinking === candidate.thinking,
|
|
519
|
-
)
|
|
520
|
-
) {
|
|
521
|
-
decision = decisionForPair(model.id, candidate, 'jev');
|
|
522
|
-
} else {
|
|
523
|
-
const baseline = selectBaselineRoute(
|
|
524
|
-
model.id,
|
|
525
|
-
profile,
|
|
526
|
-
pairs,
|
|
527
|
-
);
|
|
528
|
-
decision = decisionForPair(
|
|
529
|
-
model.id,
|
|
530
|
-
baseline.pair,
|
|
531
|
-
baseline.reasonCode,
|
|
532
|
-
);
|
|
533
|
-
decision.errorClass = 'advisor-unavailable';
|
|
534
|
-
}
|
|
535
|
-
} else if (state.currentConfig.classifierModel) {
|
|
536
|
-
const classifier = state.currentConfig.classifierModel;
|
|
537
|
-
const result = await runClassifier(
|
|
538
|
-
classifier.model,
|
|
539
|
-
registry,
|
|
495
|
+
if (candidates.length <= 1) {
|
|
496
|
+
decision.advisor = 'bypassed';
|
|
497
|
+
rememberAdvisedTurn(turn, 'bypassed');
|
|
498
|
+
} else if (useJev && jev) {
|
|
499
|
+
decision.advisor = 'jev';
|
|
500
|
+
rememberAdvisedTurn(turn, 'jev');
|
|
501
|
+
const advice = await runJev(jev, {
|
|
502
|
+
taskSummary: getBoundedRecentContext(
|
|
540
503
|
context,
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
options?.signal
|
|
547
|
-
|
|
504
|
+
jev.maxStateChars,
|
|
505
|
+
),
|
|
506
|
+
candidates,
|
|
507
|
+
profile: profile.jev,
|
|
508
|
+
routingDeadline,
|
|
509
|
+
signal: options?.signal,
|
|
510
|
+
}).catch(() => undefined);
|
|
511
|
+
options?.signal?.throwIfAborted();
|
|
512
|
+
// Re-read registry capabilities after the network boundary.
|
|
513
|
+
pairs = available();
|
|
514
|
+
const candidate = candidates.find(
|
|
515
|
+
(entry) => entry.id === advice?.candidateId,
|
|
516
|
+
);
|
|
517
|
+
if (
|
|
518
|
+
candidate &&
|
|
519
|
+
performance.now() < routingDeadline &&
|
|
520
|
+
pairs.some(
|
|
521
|
+
(pair) =>
|
|
522
|
+
pair.model === candidate.model &&
|
|
523
|
+
pair.tier === candidate.tier &&
|
|
524
|
+
pair.thinking === candidate.thinking,
|
|
525
|
+
)
|
|
526
|
+
) {
|
|
527
|
+
decision = {
|
|
528
|
+
...decisionForPair(model.id, candidate, 'jev'),
|
|
529
|
+
advisor: 'jev',
|
|
530
|
+
};
|
|
531
|
+
} else {
|
|
548
532
|
const baseline = selectBaselineRoute(model.id, profile, pairs);
|
|
549
533
|
decision = decisionForPair(
|
|
550
534
|
model.id,
|
|
551
535
|
baseline.pair,
|
|
552
536
|
baseline.reasonCode,
|
|
553
537
|
);
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
|
|
559
|
-
|
|
560
|
-
|
|
561
|
-
|
|
562
|
-
|
|
563
|
-
|
|
564
|
-
|
|
538
|
+
decision.advisor = 'jev-fallback';
|
|
539
|
+
rememberAdvisedTurn(turn, 'jev-fallback');
|
|
540
|
+
decision.errorClass = 'advisor-unavailable';
|
|
541
|
+
}
|
|
542
|
+
decision.routingLatencyMs = Math.max(
|
|
543
|
+
0,
|
|
544
|
+
performance.now() - started,
|
|
545
|
+
);
|
|
546
|
+
if (performance.now() >= routingDeadline)
|
|
547
|
+
decision.errorClass = 'deadline';
|
|
548
|
+
} else if (state.currentConfig.classifierModel) {
|
|
549
|
+
const classifier = state.currentConfig.classifierModel;
|
|
550
|
+
const result = await runClassifier(
|
|
551
|
+
classifier.model,
|
|
552
|
+
registry,
|
|
553
|
+
context,
|
|
554
|
+
undefined,
|
|
555
|
+
classifier.thinking,
|
|
556
|
+
options?.signal,
|
|
557
|
+
routingDeadline,
|
|
558
|
+
).catch(() => undefined);
|
|
559
|
+
options?.signal?.throwIfAborted();
|
|
560
|
+
pairs = available();
|
|
561
|
+
const baseline = selectBaselineRoute(model.id, profile, pairs);
|
|
562
|
+
decision = decisionForPair(
|
|
563
|
+
model.id,
|
|
564
|
+
baseline.pair,
|
|
565
|
+
baseline.reasonCode,
|
|
566
|
+
);
|
|
567
|
+
if (result && performance.now() < routingDeadline) {
|
|
568
|
+
const pair = pairs.find((entry) => entry.tier === result.tier);
|
|
569
|
+
if (pair) {
|
|
570
|
+
decision = {
|
|
571
|
+
...decisionForPair(model.id, pair, 'classifier'),
|
|
572
|
+
isClassifier: true,
|
|
573
|
+
advisor: 'classifier',
|
|
574
|
+
};
|
|
575
|
+
rememberAdvisedTurn(turn, 'classifier');
|
|
576
|
+
} else {
|
|
577
|
+
decision.advisor = 'classifier-fallback';
|
|
578
|
+
rememberAdvisedTurn(turn, 'classifier-fallback');
|
|
579
|
+
decision.errorClass = 'advisor-unavailable';
|
|
580
|
+
}
|
|
581
|
+
} else {
|
|
582
|
+
decision.advisor = 'classifier-fallback';
|
|
583
|
+
rememberAdvisedTurn(turn, 'classifier-fallback');
|
|
584
|
+
decision.errorClass = 'advisor-unavailable';
|
|
565
585
|
}
|
|
566
586
|
decision.routingLatencyMs = Math.max(
|
|
567
587
|
0,
|
|
@@ -603,7 +623,10 @@ export const registerRouterProvider = (
|
|
|
603
623
|
throw new Error(
|
|
604
624
|
'No compatible route for Google tool continuation.',
|
|
605
625
|
);
|
|
606
|
-
decision =
|
|
626
|
+
decision = {
|
|
627
|
+
...decisionForPair(model.id, priorPair, 'continuation'),
|
|
628
|
+
advisor: decision.advisor,
|
|
629
|
+
};
|
|
607
630
|
}
|
|
608
631
|
|
|
609
632
|
state.lastDecision = decision;
|
package/extensions/state.ts
CHANGED
|
@@ -14,7 +14,7 @@ import type {
|
|
|
14
14
|
RouterPinByProfile,
|
|
15
15
|
RoutingDecision,
|
|
16
16
|
} from './types';
|
|
17
|
-
import { isRoutingReasonCode } from './types';
|
|
17
|
+
import { isAdvisorOutcome, isRoutingReasonCode } from './types';
|
|
18
18
|
|
|
19
19
|
const LAST_PROFILE_STATE_FILE = 'model-router-state.json';
|
|
20
20
|
|
|
@@ -57,6 +57,7 @@ const isDecision = (value: unknown): value is RoutingDecision =>
|
|
|
57
57
|
(value.reasonCode === undefined
|
|
58
58
|
? typeof value.reasoning === 'string'
|
|
59
59
|
: isPersistedReasonCode(value.reasonCode)) &&
|
|
60
|
+
(value.advisor === undefined || isAdvisorOutcome(value.advisor)) &&
|
|
60
61
|
['isClassifier', 'isFallback', 'isBudgetForced'].every(
|
|
61
62
|
(key) => value[key] === undefined || typeof value[key] === 'boolean',
|
|
62
63
|
);
|
|
@@ -164,6 +165,7 @@ export const snapshotDecision = (
|
|
|
164
165
|
decision.errorClass === 'deadline'
|
|
165
166
|
? decision.errorClass
|
|
166
167
|
: undefined,
|
|
168
|
+
advisor: isAdvisorOutcome(decision.advisor) ? decision.advisor : undefined,
|
|
167
169
|
thinking: decision.thinking,
|
|
168
170
|
timestamp: decision.timestamp,
|
|
169
171
|
isClassifier: decision.isClassifier,
|
package/extensions/types.ts
CHANGED
|
@@ -131,6 +131,17 @@ export const isRoutingReasonCode = (
|
|
|
131
131
|
): value is RoutingReasonCode =>
|
|
132
132
|
ROUTING_REASON_CODES.some((code) => code === value);
|
|
133
133
|
export type RoutingErrorClass = 'advisor-unavailable' | 'deadline';
|
|
134
|
+
export const ADVISOR_OUTCOMES = [
|
|
135
|
+
'none',
|
|
136
|
+
'bypassed',
|
|
137
|
+
'jev',
|
|
138
|
+
'jev-fallback',
|
|
139
|
+
'classifier',
|
|
140
|
+
'classifier-fallback',
|
|
141
|
+
] as const;
|
|
142
|
+
export type AdvisorOutcome = (typeof ADVISOR_OUTCOMES)[number];
|
|
143
|
+
export const isAdvisorOutcome = (value: unknown): value is AdvisorOutcome =>
|
|
144
|
+
ADVISOR_OUTCOMES.some((outcome) => outcome === value);
|
|
134
145
|
|
|
135
146
|
export interface RoutingDecision {
|
|
136
147
|
profile: string;
|
|
@@ -142,6 +153,7 @@ export interface RoutingDecision {
|
|
|
142
153
|
reasonCode: RoutingReasonCode;
|
|
143
154
|
routingLatencyMs?: number | undefined;
|
|
144
155
|
errorClass?: RoutingErrorClass | undefined;
|
|
156
|
+
advisor?: AdvisorOutcome | undefined;
|
|
145
157
|
thinking: ThinkingLevel;
|
|
146
158
|
timestamp: number;
|
|
147
159
|
isClassifier?: boolean | undefined;
|
package/extensions/ui.ts
CHANGED
|
@@ -5,7 +5,7 @@ import type {
|
|
|
5
5
|
RouterThinkingByProfile,
|
|
6
6
|
RoutingDecision,
|
|
7
7
|
} from './types';
|
|
8
|
-
import { isRoutingReasonCode } from './types';
|
|
8
|
+
import { isAdvisorOutcome, isRoutingReasonCode } from './types';
|
|
9
9
|
|
|
10
10
|
const getDecisionFlags = (decision: RoutingDecision): string[] => {
|
|
11
11
|
const flags: string[] = [];
|
|
@@ -19,9 +19,53 @@ export const formatDecisionSource = (decision: RoutingDecision): string =>
|
|
|
19
19
|
? decision.reasonCode
|
|
20
20
|
: '';
|
|
21
21
|
|
|
22
|
+
export const formatAdvisorLabel = (
|
|
23
|
+
decision: RoutingDecision,
|
|
24
|
+
): string | undefined => {
|
|
25
|
+
if (!isAdvisorOutcome(decision.advisor)) return undefined;
|
|
26
|
+
switch (decision.advisor) {
|
|
27
|
+
case 'none':
|
|
28
|
+
case 'bypassed':
|
|
29
|
+
return undefined;
|
|
30
|
+
case 'jev':
|
|
31
|
+
return '🧭 Jev ✓';
|
|
32
|
+
case 'jev-fallback':
|
|
33
|
+
return '🧭 Jev ↪ base';
|
|
34
|
+
case 'classifier':
|
|
35
|
+
return '🧠 Classifier ✓';
|
|
36
|
+
case 'classifier-fallback':
|
|
37
|
+
return '🧠 Classifier ↪ base';
|
|
38
|
+
default:
|
|
39
|
+
return undefined;
|
|
40
|
+
}
|
|
41
|
+
};
|
|
42
|
+
|
|
43
|
+
export const formatAdvisorDetail = (
|
|
44
|
+
decision: RoutingDecision,
|
|
45
|
+
): string | undefined => {
|
|
46
|
+
const label = formatAdvisorLabel(decision);
|
|
47
|
+
if (!label) return undefined;
|
|
48
|
+
const latency = Number.isFinite(decision.routingLatencyMs)
|
|
49
|
+
? ` · ${Math.round(decision.routingLatencyMs ?? 0)}ms`
|
|
50
|
+
: '';
|
|
51
|
+
return `${label}${latency}`;
|
|
52
|
+
};
|
|
53
|
+
|
|
54
|
+
export const formatAdvisorFooter = (decision: RoutingDecision): string => {
|
|
55
|
+
if (
|
|
56
|
+
!decision.advisor ||
|
|
57
|
+
decision.advisor === 'none' ||
|
|
58
|
+
decision.advisor === 'bypassed'
|
|
59
|
+
)
|
|
60
|
+
return '';
|
|
61
|
+
const label = formatAdvisorLabel(decision);
|
|
62
|
+
return label ? ` · ${label}` : '';
|
|
63
|
+
};
|
|
64
|
+
|
|
22
65
|
export const formatDecision = (decision: RoutingDecision): string => {
|
|
23
66
|
const source = formatDecisionSource(decision);
|
|
24
|
-
|
|
67
|
+
const advisor = formatAdvisorLabel(decision);
|
|
68
|
+
return `${decision.profile}: ${decision.tier} -> ${decision.targetProvider}/${decision.targetModelId} [${decision.thinking}]${source ? ` (${source})` : ''}${advisor ? ` [${advisor}]` : ''}`;
|
|
25
69
|
};
|
|
26
70
|
|
|
27
71
|
export const formatPinSummary = (
|
|
@@ -79,7 +123,7 @@ export const updateStatus = (
|
|
|
79
123
|
|
|
80
124
|
let statusText: string;
|
|
81
125
|
if (lastDecision && matchesProfile && matchesPin) {
|
|
82
|
-
statusText = `router:${activeRouterProfile}${pinLabel} -> ${lastDecision.tier} -> ${lastDecision.targetProvider}/${lastDecision.targetModelId} (${lastDecision.thinking})`;
|
|
126
|
+
statusText = `router:${activeRouterProfile}${pinLabel} -> ${lastDecision.tier} -> ${lastDecision.targetProvider}/${lastDecision.targetModelId} (${lastDecision.thinking})${formatAdvisorFooter(lastDecision)}`;
|
|
83
127
|
} else {
|
|
84
128
|
statusText = `router:${activeRouterProfile}${pinLabel} -> waiting`;
|
|
85
129
|
}
|
|
@@ -103,18 +147,13 @@ export const updateStatus = (
|
|
|
103
147
|
if (lastDecision && lastDecision.profile === statusProfile) {
|
|
104
148
|
const flags = getDecisionFlags(lastDecision);
|
|
105
149
|
const flagsStr = flags.length > 0 ? ` [${flags.join(',')}]` : '';
|
|
150
|
+
const advisorDetail = formatAdvisorDetail(lastDecision);
|
|
106
151
|
|
|
107
152
|
widgetLines.push(
|
|
108
153
|
`Route: ${lastDecision.tier}${flagsStr} -> ${lastDecision.targetProvider}/${lastDecision.targetModelId} (${lastDecision.thinking})`,
|
|
109
154
|
`Phase: ${lastDecision.phase}`,
|
|
110
155
|
`Source: ${formatDecisionSource(lastDecision) || 'unknown'}`,
|
|
111
|
-
...(
|
|
112
|
-
? [`Routing: ${Math.round(lastDecision.routingLatencyMs ?? 0)}ms`]
|
|
113
|
-
: []),
|
|
114
|
-
...(lastDecision.errorClass === 'deadline' ||
|
|
115
|
-
lastDecision.errorClass === 'advisor-unavailable'
|
|
116
|
-
? [`Routing error: ${lastDecision.errorClass}`]
|
|
117
|
-
: []),
|
|
156
|
+
...(advisorDetail ? [advisorDetail] : []),
|
|
118
157
|
);
|
|
119
158
|
} else if (!routerEnabled && lastNonRouterModel) {
|
|
120
159
|
widgetLines.push(`Fallback: ${lastNonRouterModel}`);
|
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
"apiKey": "<rendered by chezmoi/1Password in user config only>",
|
|
6
6
|
"endpoint": "https://api.typesafe.ai/v1/systemone",
|
|
7
7
|
"model": "jev-1.13.0",
|
|
8
|
-
"timeoutMs":
|
|
8
|
+
"timeoutMs": 1500,
|
|
9
9
|
"confidenceThreshold": 0.65,
|
|
10
10
|
"maxStateChars": 12000,
|
|
11
11
|
"mode": "advisory"
|