omp-llm-role 1.3.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -1
- package/docs/dev/architecture.md +32 -12
- package/docs/dev/current-state.md +82 -11
- package/docs/dev/explorer.md +21 -5
- package/docs/dev/releasing.md +36 -3
- package/docs/dev/scoring.md +21 -4
- package/docs/dev/testing.md +62 -12
- package/docs/dev/writes-and-state.md +131 -26
- package/package.json +9 -1
- package/src/availability.ts +40 -7
- package/src/explorer/explain.ts +11 -6
- package/src/settings.ts +19 -0
- package/src/updater.ts +126 -32
- package/web/app.js +7 -1
package/README.md
CHANGED
|
@@ -126,6 +126,7 @@ Global knobs:
|
|
|
126
126
|
| `priceSwitchFraction` | `0.5` | Inside the margin, adopt a challenger at least this fraction cheaper (0.5 = half the $/M); `0` disables |
|
|
127
127
|
| `writeFallbackChains` | `true` | Write per-role fallback chains into `config.yml` |
|
|
128
128
|
| `fallbackChainDepth` | `2` | Fallback models per role chain |
|
|
129
|
+
| `fallbackQualityFloor` | `0.9` | A cheaper-or-equal fallback must retain at least this fraction of the chosen model's quality; `0` disables |
|
|
129
130
|
| `activateDefaultOnEmptySession` | `true` | Reapply the new `default` selector to an empty session's active model |
|
|
130
131
|
| `tokenProfile` | unset | `{ inputTokens, outputTokens }` per task for the explorer's `$/task` column, used when a role has no recorded usage. Display-only; the recorded usage takes precedence. Unset = the column reads `—` |
|
|
131
132
|
| `errorReporting` | `off` | Opt-in reporting of **unexpected** plugin errors as GitHub issues on `Gott50/omp-llm-role`; one of `off`, `ask`, `auto` (see below) |
|
|
@@ -160,7 +161,7 @@ defaults.
|
|
|
160
161
|
|---|---|---|
|
|
161
162
|
| `endpointCeilings` | `filters.tools: true`, `filters.minOutputTokens: 16384` | Routes that cannot call tools and routes whose output ceiling is below 16 384 tokens leave the pool — a reasoning model is not cut off mid-thought |
|
|
162
163
|
| `cachePricing` | `cacheHitRate: 0.5` | The role is priced on a cache-heavy agent loop's invoice (the endpoint's `cacheReadPrice` blended in) rather than the sticker price |
|
|
163
|
-
| `providerPinning` | `preferOwnProvider: true`, `filters.preferredMinThroughput: 50` | Each request is bound to
|
|
164
|
+
| `providerPinning` | `preferOwnProvider: true`, `filters.preferredMinThroughput: 50` | Each request is bound to a provider route **that OpenRouter actually serves**: the updater walks the model's value-for-money route order (the `vfm` score, with a soft 50 tok/s floor that deprioritizes a pathologically slow route without dropping it) and **probes each candidate with a one-token request bound to it** (`provider: { only: [slug] }`), pinning the first route the request does not reject and **dropping the pin** (bare selector, OpenRouter's own routing) when no candidate is routable — so a role never 404s on a route the endpoints data still called healthy. The selector gains `@<slug>`, and the fallback chain carries the primary model on every gate-passing route except the pinned one, then each of the next `fallbackChainDepth` fallback models on every gate-passing route — each drawn from the role's own ranking below the primary, the cheaper-or-equal ones that keep at least `fallbackQualityFloor` of its quality first, then the pricier — so a down provider fails over to another provider of the same model before the chain switches models, and a failover never silently multiplies the spend or drops the role onto a much weaker model. The model is priced **and measured** on its score-max route (its billed price and p50 throughput, so `q` moves too), falling back to the default blend only when it has no candidate route |
|
|
164
165
|
| `costCap` | `filters.maxPriceUsdPerM: 10` | The priciest tail of the eligible pool leaves it, so the pick cannot be an expensive outlier |
|
|
165
166
|
|
|
166
167
|
Precedence is one rule:
|
package/docs/dev/architecture.md
CHANGED
|
@@ -29,12 +29,12 @@ the extension's `/refresh-roles`, `/explore-roles` and `/create-agent`.
|
|
|
29
29
|
| `src/benchmark-sources.ts` | The benchmark-source registry: the single source of truth for which sources exist, what metric each feeds, and how to fetch/parse/join each. Exports `BENCHMARK_SOURCES` (the shipped static sources), `resolveBenchmarkSource`, `declaredSourceForLink`, `sourceForMetric`, `parseBenchmarkPayload`, `parseBenchmarkPayloadMeta`, `loadBenchmarkScores`, `applyBenchmarkScores`, `joinBenchmarkScores`, `normalizeMetricKey`, `externalMetricKey`, `normalizeDesignId`, `parseWritingEvidence`, `parseLlmStatsBenchmark`, `parseBenchmarkCatalog`, `loadBenchmarkCatalog`, `cachedSourceInfo`, `catalogMetric`, `SHIPPED_CATALOG_METRICS`, the declarative-source surface (`SourceDeclaration`, `validateDeclaration`, `parseSourceDeclaration`, `loadDeclaredSources`, `saveDeclaredSource`, `declaredSourcesPath`, `declarationToSource`, `executeDeclaration`, `llmStatsBenchmarkDeclaration`, `genericBenchmarkSource`, `dryRunDeclaration`), the fetch helpers (`fetchJson`, `fetchText`), and the `BenchmarkSource`/`JoinRule`/`BenchmarkScores`/`BenchmarkPayloadMeta`/`FocusSourceInfo`/`BenchmarkCatalogEntry`/`DeclarationDryRun` types. Imports `engine.ts` for types only, so there is no runtime cycle. |
|
|
30
30
|
| `src/settings.ts` | Shipped defaults and the settings merge/validate path. Exports `DEFAULT_ROLES`, `DEFAULT_SETTINGS`, `SHIPPED_AGENTS`, `KNOWN_METRICS`, `isKnownMetric` (a shipped key or an external `<ns>:<local>`), `PLUGIN_SETTINGS_PATH`, `ACTIVATE_DEFAULT_KEY`, `deriveSettingsSchema`, `readPluginSettingsMap`, `resolveSettings`, `roleUniverse`, `deepMergeInto`, `findProjectAnchor`, `projectLockPath`, and the `ResolvedSettings`/`PluginSettingSchema`/`UniverseEntry`/`RoleKind` types. `ResolvedSettings` carries `tokenProfile` (`{ inputTokens, outputTokens } | null`, default `null`) — the user-set per-task token profile the explorer's per-task price falls back to when a role has no recorded usage; validated to finite numbers ≥ 0, schema rows `tokenProfile.inputTokens`/`tokenProfile.outputTokens`. `readPluginSettingsMap` merges the project lock file (`projectLockPath(cwd)` = `<cwd>/.omp/plugins/omp-plugins.lock.json`, omp's project dir with **no walk-up**) over the user-level one; `findProjectAnchor` (the walk-up `.omp`/`.git` anchor) is kept only for the `/project-roles` divergence warning. |
|
|
31
31
|
| `src/agent-pins.ts` | Agent → pinned-role derivation. Exports `parseAgentPin` (first `@<role>` in the `model:` frontmatter) and `discoverAgentPins` (scans the shipped, user and project agent dirs; project > user > plugin). Drives the `task.disabledAgents` sync. |
|
|
32
|
-
| `src/availability.ts` | Key tier gate, catalog filter, variant resolution,
|
|
32
|
+
| `src/availability.ts` | Key tier gate, catalog filter, variant resolution, the model probe, the auto-pin's route probe, the keyed-catalog availability primitive, and the allowed-providers whitelist harvest. The keyed source is the Bearer-authenticated `GET /api/v1/models` (the account setting "Filter the model catalog for API keys" makes it a per-key allowlist; set membership is the only signal), shared by the explorer and the updater's companion fast path. Exports `fetchKeyMeta`, `tierGate`, `filterCatalog`, `rankingIdOf`, `currentRankingId`, `resolveVariant`, `enrichThinkingLevels`, `probeModel`, `probeRoute` (the same one-token completion plus `provider: { only: [slug] }` — the auto-pin's routability gate; spec #90), `fetchAllowedProviders`, `providerAllowed`, `computeKeyAvailability`, `fetchKeyAvailability`, `catalogFromOmpModelsJson`, `THINKING_LEVELS`, and the `CatalogEntry`/`KeyMeta`/`Tier`/`ProbeVerdict`/`KeyAvailability` types. |
|
|
33
33
|
| `src/config-edit.ts` | Surgical line-oriented YAML patch for `modelRoles` + `retry.fallbackChains` + `task.disabledAgents`, plus the atomic writer. Exports `parseConfig`, `patchConfig`, `writeConfigAtomic`, `ConfigEditError`, and the `ConfigPatch` type. No runtime YAML dependency. |
|
|
34
34
|
| `src/state.ts` | State/history/lock files under the agent dir, and agent-dir resolution. Exports `agentDir`, `loadState`, `saveState`, `appendHistory`, `acquireLock`, `releaseLock`, `freshState`, and the `PluginState` type (which carries `managedDisabledAgents` — the `task.disabledAgents` names the plugin added). |
|
|
35
35
|
| `src/usage-ledger.ts` | The per-role usage ledger (spec #81): an append-only jsonl file (`llm-role-usage.jsonl`, cap `USAGE_LEDGER_CAP = 2000` rows) beside the plugin's other state, one row per assistant turn. Exports `UsageRow`, `TokenProfile`, `USAGE_LEDGER_FILE`, `USAGE_LEDGER_CAP`, `appendUsageRow` (best-effort, never throws), `readUsageRows` (tolerant: missing/malformed → `[]`), `roleTokenProfile` (the role's median per-turn input/output counts via `agent-create.ts`'s `quantile` — the repo's one quantile convention — or `null`), and `resolveTokenProfile` (recorded usage → the `tokenProfile` setting → `null`). The caller owns the dir; the extension records into `resolveProjectDir() ?? agentDir()`, the explorer reads the active scope's dir. |
|
|
36
36
|
| `src/error-report.ts` | Opt-in reporting of **unexpected** plugin errors as GitHub issues on `Gott50/omp-llm-role` (the `errorReporting` setting). The plugin's only GitHub client and the only home for a fingerprint, a redaction rule, the agent-dir ledger (`ERROR_REPORT_LEDGER_FILE`) and the issue body. **Entry point:** `reportUnexpectedError(input, deps)` — the feature's one test seam (per-call input + fully injected `deps`; tests supply a fake `fetch`, clock, env, dir and token resolver). Also exports `defaultErrorReportDeps()` (production deps: resolved settings, global `fetch`, real clock, `agentDir`, the shipped manifest, `gh auth token`, the OS opener) and `resetErrorReportGuard()` (the in-process dedupe guard's test hook). Never throws, never rejects; a no-op when `errorReporting` is `off`. |
|
|
37
|
-
| `src/updater.ts` | Orchestration: rank → tier gate → probe → hysteresis → chains → agent-disable sync → config write. Exports `runUpdater`, `resolveProjectDir` (the project scope's `<cwd>/.omp` when it carries a non-empty `omp-llm-role` settings entry, else `null` — the same test the explorer's scope resolution and the extension's usage-ledger dir use), the `Deps`/`Decision`/`RunResult`/`Trigger`/`DecisionReason` types. Resolves the project scope (`<cwd>/.omp` when it carries an `omp-llm-role` settings entry) and scopes the config/state/history/lock to it. `agentDisablePatch` also removes a previously-managed `task.disabledAgents` name whose agent file is gone. `RunResult.defect` is set on the exception-abort paths only (a `ConfigEditError`/self-check mismatch and the catch-all throw); the enumerated environment aborts leave it unset. |
|
|
37
|
+
| `src/updater.ts` | Orchestration: rank → tier gate → model probe → hysteresis → auto-pin route probe → chains → agent-disable sync → config write. Exports `runUpdater`, `resolveProjectDir` (the project scope's `<cwd>/.omp` when it carries a non-empty `omp-llm-role` settings entry, else `null` — the same test the explorer's scope resolution and the extension's usage-ledger dir use), the `Deps`/`Decision`/`RunResult`/`Trigger`/`DecisionReason` types. Resolves the project scope (`<cwd>/.omp` when it carries an `omp-llm-role` settings entry) and scopes the config/state/history/lock to it. `agentDisablePatch` also removes a previously-managed `task.disabledAgents` name whose agent file is gone. `RunResult.defect` is set on the exception-abort paths only (a `ConfigEditError`/self-check mismatch and the catch-all throw); the enumerated environment aborts leave it unset. |
|
|
38
38
|
| `src/extension.ts` | omp extension entry (default export). Registers `session_start` (awaited day-gated run), `/refresh-roles`, `/explore-roles` (in-process explorer), `/create-agent` (architect + benchmark-link resolution + catalog discovery), `/remove-agent`, `/project-roles` (profile architect + `setupProject` + in-process updater), and `session_shutdown`. Also subscribes to `session_start`, `retry_fallback_applied`, `retry_fallback_succeeded` and `turn_start` to keep the live model's display name suffixed with its provider pin (`src/provider-display.ts`); each handler mutates the live model's `name` in place and never calls `pi.setModel` (which would wipe omp's in-flight fallback state and clobber the thinking level). Owns `extDeps` (the extension's `Deps`) and the live-session model hook. Records the per-role usage ledger (spec #81) on `turn_end` and `session_shutdown` — one row per assistant turn read from `sessionManager.getBranch()`, into `resolveProjectDir() ?? agentDir()`, deduped by `ts|model` so the two hooks cannot double-record. Every unexpected command-handler throw and every `runUpdater` defect is routed through `src/error-report.ts`: `reportCommandError` for the six command-handler catch sites and `reportDefect` for a `runUpdater` result whose `defect` is set — each keeps the pre-existing one-line warning and adds the reporting module's own lines. |
|
|
39
39
|
| `src/guards.ts` | The package's one type guard: `isRecord`. |
|
|
40
40
|
| `src/provider-display.ts` | The provider display (spec #77): keep the live session model's `name` suffixed with its upstream provider pin, so omp's model segment renders e.g. `DeepSeek V4.1 Flash @coreweave/fp8`. Pure and dual-runtime (no SDK import). Exports `providerPinOf` (the single `compat.openRouterRouting.only[0]`, else `compat.vercelGatewayRouting.only[0]`; null unless exactly one non-empty string), `providerPinOfSelector` (the trailing `@<slug>` of a selector, after stripping a thinking suffix), `withProviderSuffix` (strip any previous ` @<slug>` then append, idempotent), and `applyProviderDisplay` (mutate a record's `name`; no-op otherwise). |
|
|
@@ -145,20 +145,39 @@ The omp extension injects the production `Deps`; tests inject fakes.
|
|
|
145
145
|
6. **Hysteresis.** `no-current` → adopt; ineligible current → `adopted`; best
|
|
146
146
|
beats current by `switchMargin` → `switched`; inside the margin but
|
|
147
147
|
`cheaperInsideMargin` → `switched-cost`; else `kept-margin` / `kept-eligible`.
|
|
148
|
-
7. **
|
|
148
|
+
7. **Route pin (auto-pin).** With the `providerPinning` capability on and no
|
|
149
|
+
manual `providerPin`, the chosen model's `rankedRoutes` order is walked and
|
|
150
|
+
each candidate is probed with `probeRoute` (a one-token completion bound to
|
|
151
|
+
the slug; optional seam `deps.probeRoute?`, DI like `probeModel`); the first
|
|
152
|
+
route that is not `blocked` becomes the `@<slug>` pin. `unknown` counts as
|
|
153
|
+
usable; a model whose every candidate is rejected (or whose probe budget ran
|
|
154
|
+
out) drops the pin — the selector stays bare and `Decision.pinDropped` is set.
|
|
155
|
+
Verdicts are cached per run by `catalogId@slug`; the walk spends at most
|
|
156
|
+
`ROUTE_PROBE_BUDGET = 12` new probes per role. It never runs for a manual pin,
|
|
157
|
+
a locked or disabled role, a non-OpenRouter selector or a model with no
|
|
158
|
+
candidate route. Contract:
|
|
159
|
+
[`writes-and-state.md`](writes-and-state.md), *The provider-pin
|
|
160
|
+
auto-configuration*.
|
|
161
|
+
8. **Suffix.** Append `roles.<role>.thinking` only when the chosen catalog row's
|
|
149
162
|
`thinking[]` includes the level (meta levels `off`/`auto` need only a
|
|
150
163
|
non-empty list).
|
|
151
|
-
|
|
152
|
-
`fallbackChainDepth`
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
`
|
|
157
|
-
|
|
158
|
-
|
|
164
|
+
9. **Chains.** For every managed role, `chainUpserts` (bare key → the next
|
|
165
|
+
`fallbackChainDepth` fallback models drawn from the role's ranked candidate
|
|
166
|
+
list below the chosen model — the cheaper-or-equal candidates that retain at
|
|
167
|
+
least `fallbackQualityFloor` of its `q` first, then the pricier ones, each
|
|
168
|
+
bucket in value order, truncated to the depth — suffixed per target) and
|
|
169
|
+
`chainPrunes` (plugin-written keys no longer referenced). The list is the
|
|
170
|
+
ranked candidate list minus the key-blocked and probe-`blocked` candidates; an
|
|
171
|
+
unprobed candidate stays eligible, so the chain can reach a cheaper-or-equal
|
|
172
|
+
model below the probe window (writes-and-state.md, *Which models are
|
|
173
|
+
chained*). A key claimed by more than one role stays level-free.
|
|
174
|
+
10. **Config patch.** `writePatch` acquires the refresh lock, reads `config.yml`,
|
|
175
|
+
`patchConfig` (surgical, self-checked), `writeConfigAtomic` (mtime-guarded
|
|
176
|
+
tmp+rename, 3 conflict retries). Zero changes → no write, no mtime change.
|
|
177
|
+
11. **State + history.** `saveState` (day gate, managed roles, last selectors,
|
|
159
178
|
plugin-written chain keys, managed `task.disabledAgents` names, previous
|
|
160
179
|
`modelRoles` snapshot) and `appendHistory` (one row per completed run).
|
|
161
|
-
|
|
180
|
+
12. **Live-session coupling.** When `default` actually changed and
|
|
162
181
|
`activateDefaultOnEmptySession` is set, `deps.applySessionModel?.(selector,
|
|
163
182
|
previous)` hands the new selector to the host (extension only).
|
|
164
183
|
|
|
@@ -221,6 +240,7 @@ type Deps = {
|
|
|
221
240
|
getSettings?(): Promise<Record<string, unknown>>;
|
|
222
241
|
getKeyMeta?(token: string): Promise<KeyMeta>;
|
|
223
242
|
probeModel?(token: string, catalogId: string): Promise<ProbeVerdict>;
|
|
243
|
+
probeRoute?(token: string, catalogId: string, providerSlug: string): Promise<ProbeVerdict>;
|
|
224
244
|
getAllowedProviders?(token: string, modelId: string): Promise<Set<string> | null>;
|
|
225
245
|
// Host coupling (extension only): apply a concrete selector to the live session model.
|
|
226
246
|
applySessionModel?(selector: string, previous: string | null): Promise<void>;
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Current state (2026-10-
|
|
1
|
+
# Current state (2026-10-19)
|
|
2
2
|
|
|
3
3
|
Dated snapshot of the ranking and the landed work as of the date in the heading —
|
|
4
4
|
matched/priced/eligible counts, per-role leaders, the open defects, and the
|
|
@@ -6,12 +6,77 @@ per-issue work log below. It is refreshed by
|
|
|
6
6
|
regenerating the report (`node src/cli/llm-role-rank.ts --out docs/llm-role-rankings.md`) and
|
|
7
7
|
copying the new numbers here; release prose belongs in
|
|
8
8
|
[`../../CHANGELOG.md`](../../CHANGELOG.md), not in this file. The model-level
|
|
9
|
-
ranking numbers below are the 2026-10-
|
|
10
|
-
|
|
9
|
+
ranking numbers below are the 2026-10-19 measurement (regenerated at the 1.3.0
|
|
10
|
+
cut; byte-identical to the 2026-10-18 one apart from the report's date line —
|
|
11
|
+
the input-price blend basis #21, the #17/#18/#19/#20 wave, the #34–#40
|
|
12
|
+
focus-axis/capability-flag work, and
|
|
11
13
|
spec #81's explorer surfaces — the ranking CLI itself is unchanged); the
|
|
12
14
|
provider-pin numbers are the 2026-10-11 measurement (spec #71; see the top
|
|
13
15
|
bullet). Neither is re-derived on read.
|
|
14
16
|
|
|
17
|
+
- Fallback chain quality floor (2026-10-19, spec #95): a role's fallback chain no
|
|
18
|
+
longer only reorders the probe window — that window is `1 + fallbackChainDepth`
|
|
19
|
+
models, the same size as the chain, so the reorder could never reach a cheaper
|
|
20
|
+
model at rank 4 or deeper. The chain's fallback **models** now come from the
|
|
21
|
+
role's whole ranked candidate list below the chosen model: the
|
|
22
|
+
**cheaper-or-equal** ones first — a candidate qualifies only if it retains at
|
|
23
|
+
least `fallbackQualityFloor` (a new setting, default `0.9`; `0` disables the
|
|
24
|
+
floor, `1` requires an exact quality match) of the chosen model's own `q`, so a
|
|
25
|
+
failover never silently drops the role onto a much weaker model — then the
|
|
26
|
+
pricier ones, each bucket in value order, truncated to `fallbackChainDepth`.
|
|
27
|
+
Key-blocked and probe-`blocked` candidates are excluded; an **unprobed**
|
|
28
|
+
candidate stays eligible, so the chain can reach a cheaper-or-equal model ranked
|
|
29
|
+
below the probe window (chain entries are never probe-verified — spec #90). A
|
|
30
|
+
candidate priced exactly at the chosen model's price whose `q` is below the
|
|
31
|
+
floor lands in neither bucket and is dropped. The floor gates **chain
|
|
32
|
+
membership only**: the primary choice, the hysteresis decision, the ranking, the
|
|
33
|
+
probe walk's stop condition and budget, the chain key, the route expansion, the
|
|
34
|
+
thinking levels and the prune-on-pin-change are all unchanged, as are the
|
|
35
|
+
manual-`providerPin` and no-pin paths. Measured on the 2026-10-19 caches with
|
|
36
|
+
the live resolved defs (`fallbackQualityFloor` 0.9, depth 2): `slow`, `vision`
|
|
37
|
+
and `task` gain cheaper fallbacks, the other seven roles are byte-identical, and
|
|
38
|
+
no role is degraded. That is the **per-role** effect: in the written config only
|
|
39
|
+
the `vision` key changes, because `deepseek-v4.1-flash@coreweave/fp8` is claimed
|
|
40
|
+
by seven roles and carries the last claimer's chain (the shared-key behaviour
|
|
41
|
+
spec #95 leaves out of scope). The ranking numbers are unchanged — the chain is a
|
|
42
|
+
write-side shape, not a ranking input — so `docs/llm-role-rankings.md` is not
|
|
43
|
+
regenerated and the dated numbers below stand as written; the suite grew
|
|
44
|
+
652 → 666.
|
|
45
|
+
|
|
46
|
+
- Routability-verified auto-pin (2026-10-19, spec #90): the `providerPinning`
|
|
47
|
+
auto-pin no longer writes the score-max route blind. The daily update walks the
|
|
48
|
+
chosen model's `vfm` route order and probes each candidate with a one-token
|
|
49
|
+
completion bound to it (`provider: { only: [slug] }`), because OpenRouter's
|
|
50
|
+
endpoints API can list a route as healthy — non-degraded, a full
|
|
51
|
+
`supported_parameters` list, 100 % uptime — while its routing table excludes it:
|
|
52
|
+
the live config's `deepseek/deepseek-v4.1-flash@relace` passed every gate and
|
|
53
|
+
then 404'd on every request, including every fallback revert. Only the narrow
|
|
54
|
+
no-allowed-providers 404 skips a route; a timeout, a 5xx, a 429 and a 404 with
|
|
55
|
+
any other body are `unknown` and count as usable, so a transient blip never
|
|
56
|
+
unpins a role. Verdicts are cached per run by `catalogId@slug`, and the walk is
|
|
57
|
+
capped at `ROUTE_PROBE_BUDGET = 12` probes per role. When no candidate is
|
|
58
|
+
routable the pin is **dropped** — the selector stays bare and OpenRouter routes
|
|
59
|
+
the request itself — and the run notes name the rejected slugs
|
|
60
|
+
(`; route-blocked: …`), plus the dropped pin (`; pin dropped (no routable
|
|
61
|
+
route)`) when none survived. The fallback chain keeps its shape
|
|
62
|
+
(the primary contributes every gate-passing route except the pinned **slug** —
|
|
63
|
+
see the chain bullet below), and **chain entries are not probed**: one
|
|
64
|
+
unroutable entry is survivable, omp fails over. A manual
|
|
65
|
+
`roles.<role>.providerPin` stays a hard gate: never probed, never replaced. The
|
|
66
|
+
explorer's auto-pin line now reads as a **candidate** — the first route the walk
|
|
67
|
+
tries, not a promise of the written pin. The ranking numbers are unchanged — the
|
|
68
|
+
probe is a write-side gate, not a ranking input, and neither the ranking CLI nor
|
|
69
|
+
`docs/llm-role-rankings.md` changed; the suite grew 631 → 652.
|
|
70
|
+
|
|
71
|
+
- Release 1.3.0 (2026-10-19, cut `55416d4`, tag `v1.3.0`): the spec #81 explorer
|
|
72
|
+
surfaces ship. The report was regenerated at the cut and is **unchanged** from
|
|
73
|
+
the 2026-10-18 measurement — matched 149/408, priced 148, eligible 140
|
|
74
|
+
(vision 73, designer 87), the same ten leaders — so the numbers below stand as
|
|
75
|
+
written. The cut also carried the #89 fix: the pre-commit hook and
|
|
76
|
+
`tests/ci.test.ts` now scrub `GIT_*` from every child process, so the suite's
|
|
77
|
+
temp-repo git calls can no longer target the real repository (suite 630 →
|
|
78
|
+
631).
|
|
79
|
+
|
|
15
80
|
- Explorer as-written pool, per-task price, and price levers (2026-10-18, spec
|
|
16
81
|
#81): the explorer gains a **pool toggle** (raw vs as-written), a **per-task
|
|
17
82
|
price column**, and two **price-lever readouts**. The as-written pool applies
|
|
@@ -66,15 +131,21 @@ bullet). Neither is re-derived on read.
|
|
|
66
131
|
|
|
67
132
|
- Auto-pin chain expansion (2026-10-10, issue #69): with `providerPinning` on,
|
|
68
133
|
the fallback chain no longer stops at three providers per model — the primary
|
|
69
|
-
model contributes every gate-passing route except the pinned
|
|
70
|
-
|
|
71
|
-
score
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
134
|
+
model contributes every gate-passing route **except the pinned route** (the
|
|
135
|
+
exclusion is by pinned slug, not by position: spec #90's probe can pin a route
|
|
136
|
+
other than the score leader, so the leader rides as a chain entry, and a dropped
|
|
137
|
+
pin excludes nothing), then each `fallbackChainDepth` fallback model contributes
|
|
138
|
+
every gate-passing route, in score order (the `vfm` route score — spec #71
|
|
139
|
+
replaced the original route-value order) and deduped by provider slug, so a down
|
|
140
|
+
provider fails over to another provider of the same model before the chain
|
|
141
|
+
switches models. Chain length is data-driven (`(R_primary − 1) + Σ R_fallback`,
|
|
142
|
+
or `R_primary + Σ R_fallback` with a dropped pin) instead of the fixed
|
|
75
143
|
`2 + 3·fallbackChainDepth`; the manual-`providerPin` and no-pin paths are
|
|
76
|
-
unchanged.
|
|
77
|
-
|
|
144
|
+
unchanged. Which `fallbackChainDepth` **models** each chain falls back to was
|
|
145
|
+
the probe window until spec #95, which draws them from the role's ranked
|
|
146
|
+
candidate list below the chosen model with the quality floor (see the top
|
|
147
|
+
bullet) — the route expansion above is unchanged. The ranking numbers are
|
|
148
|
+
unchanged — the chain is a write-side shape, not a ranking input.
|
|
78
149
|
- Opt-in error reporting (2026-10-08, issue #48): a new `errorReporting` setting
|
|
79
150
|
(`off` — the default — `ask`, `auto`) turns a genuinely unexpected failure — a
|
|
80
151
|
command handler throwing, or a `runUpdater` abort whose `RunResult.defect` is
|
package/docs/dev/explorer.md
CHANGED
|
@@ -191,10 +191,14 @@ Pure, no I/O; all ranking math is delegated to `src/engine.ts`.
|
|
|
191
191
|
per-metric raw target that would close the gap, via `inverseCardinal`, with
|
|
192
192
|
unreachable/extrapolated notes), `dominators` (models both cheaper and at
|
|
193
193
|
least as good on `q`, top 3), `ownProvider` (true when the model was priced on
|
|
194
|
-
the soft best route rather than the blend), `autoPin` (the
|
|
195
|
-
|
|
194
|
+
the soft best route rather than the blend), `autoPin` (the **candidate** route's
|
|
195
|
+
provider slug — `bestRoute(model, def)?.providerSlug`, the max-`vfm`
|
|
196
196
|
route (scoring.md, *Route ordering*), when the role sets
|
|
197
|
-
`preferOwnProvider` and carries no manual `providerPin`, else `null`
|
|
197
|
+
`preferOwnProvider` and carries no manual `providerPin`, else `null` — the
|
|
198
|
+
first route the updater's daily walk tries, **not** a promise of the written
|
|
199
|
+
pin: the updater probes it and skips to the next route when OpenRouter does not
|
|
200
|
+
serve it, or drops the pin entirely when none is routable; writes-and-state.md,
|
|
201
|
+
*The provider-pin auto-configuration*), and the
|
|
198
202
|
availability overlay `key`/`keyReason`
|
|
199
203
|
(the same verdict as `rankRows`, plus the reason it reads that way). It mirrors
|
|
200
204
|
`rankRole`'s capability fill so the contributions sum exactly to `q`.
|
|
@@ -369,8 +373,20 @@ a scraped third-party site, so the DOM is built with
|
|
|
369
373
|
line). The cost line appends `· best route`
|
|
370
374
|
when the model was priced on the soft best route (`ownProvider`), and a
|
|
371
375
|
**pin line** shows the pin in force: `pin: <slug> (manual)` when the edited def
|
|
372
|
-
carries a `providerPin`, else `auto pin: <slug>` from the
|
|
373
|
-
`autoPin` (read-only — the manual field is the only way to set
|
|
376
|
+
carries a `providerPin`, else `auto pin (candidate): <slug>` from the
|
|
377
|
+
explanation's `autoPin` (read-only — the manual field is the only way to set
|
|
378
|
+
one). The label says *candidate* on purpose: the value is the first route the
|
|
379
|
+
updater's value-ordered walk *tries*, not a promise of the written pin — the
|
|
380
|
+
updater probes that route and skips to the next when OpenRouter does not serve
|
|
381
|
+
it, or drops the pin entirely (bare selector) when none is routable, so the
|
|
382
|
+
written pin can be a later route or absent altogether (spec #90;
|
|
383
|
+
writes-and-state.md, *The provider-pin auto-configuration*). The explorer is
|
|
384
|
+
read-only and offline and cannot probe, so it names the candidate and says so
|
|
385
|
+
in the tooltip. The candidate equals the written pin when every route probes
|
|
386
|
+
clean; with the account's allowed-providers whitelist active, the panel and the
|
|
387
|
+
written pin agree only on the **as-written** pool, which applies the same prune
|
|
388
|
+
the updater applies before walking (in the raw pool the panel can name a route
|
|
389
|
+
the prune removes). The
|
|
374
390
|
per-task line shows the breakdown (`<in> in × $<p_in>/M + <out> out ×
|
|
375
391
|
$<p_out>/M × <factor> = $<total>`), or `— (no token profile — record usage or
|
|
376
392
|
set tokenProfile)` when `perTask` is `null`. The **Price levers** section
|
package/docs/dev/releasing.md
CHANGED
|
@@ -122,6 +122,39 @@ path, plus `marketplace update` + `upgrade` to a bumped catalog version), and
|
|
|
122
122
|
push publishes to npm over OIDC and creates the GitHub Release. The guard was
|
|
123
123
|
verified with a negative control — tag `v0.0.1` against version `1.1.0` failed
|
|
124
124
|
in `notes` before any publish.
|
|
125
|
+
- **1.3.0 published by CD 2026-10-19** (run 39222193609, tag push): the guard
|
|
126
|
+
and the npm floor passed, `npm publish` completed over OIDC trusted publishing
|
|
127
|
+
with a provenance attestation (`npm audit signatures`: 1 package has a
|
|
128
|
+
verified attestation), and the GitHub Release was created with the 1.3.0
|
|
129
|
+
CHANGELOG section as its body
|
|
130
|
+
(`https://github.com/Gott50/omp-llm-role/releases/tag/v1.3.0`). Verified:
|
|
131
|
+
`dist-tags.latest` = `1.3.0`, npm's `gitHead` = `55416d4` (the tag's commit),
|
|
132
|
+
and the publish step's log carries the direct-publish signature (`Publishing
|
|
133
|
+
to https://registry.npmjs.org/ with tag latest and default access`, `Signed
|
|
134
|
+
provenance statement`, `+ omp-llm-role@1.3.0`). The cut (`55416d4`, tagged
|
|
135
|
+
`v1.3.0`) bumped the four lockstep fields, renamed `## [Unreleased]` to
|
|
136
|
+
`## [1.3.0] - 2026-10-19`, and regenerated the report (408 models, matched
|
|
137
|
+
149/408, priced 148 — byte-identical to the 2026-10-18 report apart from its
|
|
138
|
+
date line; `docs/dev/current-state.md` refreshed to the same measurement).
|
|
139
|
+
The cut also carried the #89 fix (the `GIT_*` scrub in the pre-commit hook and
|
|
140
|
+
`tests/ci.test.ts`), so the release commit went through the hook with the real
|
|
141
|
+
repository untouched.
|
|
142
|
+
- **1.3.0 install routes smoke-tested** (2026-10-19, omp 18.12.2, throwaway
|
|
143
|
+
`HOME`s, cwd outside the real home tree): npm
|
|
144
|
+
(`omp plugin install omp-llm-role`), git
|
|
145
|
+
(`omp plugin install github:Gott50/omp-llm-role`) and marketplace
|
|
146
|
+
(`omp plugin marketplace add Gott50/omp-llm-role` +
|
|
147
|
+
`omp plugin install omp-llm-role@gott50-plugins`) each installed 1.3.0
|
|
148
|
+
(`omp plugin doctor`: 4 ok / 0 warnings / 0 errors for npm and git; the
|
|
149
|
+
marketplace install's one warning is the expected
|
|
150
|
+
`package_manifest: Not created yet`, and it landed at
|
|
151
|
+
`…/cache/plugins/gott50-plugins___omp-llm-role___1.3.0`). Each post-install
|
|
152
|
+
session run (`omp -p "say ok"` with `OPENROUTER_API_KEY` from
|
|
153
|
+
`omp token openrouter`) wrote `modelRoles` (9 roles),
|
|
154
|
+
`retry.fallbackChains` and `task.disabledAgents: [designer]` into that HOME's
|
|
155
|
+
`config.yml`, landed `llm-role-state.json` + `llm-role-history.jsonl` +
|
|
156
|
+
`llm-role-usage.jsonl` (the spec #81 ledger, live for the first time), and
|
|
157
|
+
answered.
|
|
125
158
|
- **1.2.0 published by CD 2026-10-12** (run 38299141569, tag push): the guard
|
|
126
159
|
and the npm floor passed, `npm publish` completed over OIDC trusted publishing
|
|
127
160
|
with a provenance attestation (`npm audit signatures`: 1 package has a
|
|
@@ -200,7 +233,7 @@ path, plus `marketplace update` + `upgrade` to a bumped catalog version), and
|
|
|
200
233
|
- **Stray registry placeholder.** The registry also carries a `0.0.0-stage`
|
|
201
234
|
version ("Temporary package placeholder for staged publishing", 334 B,
|
|
202
235
|
published 2026-10-03). It is not from this tree and does not affect installs
|
|
203
|
-
(`latest` is `1.
|
|
204
|
-
- **Next release: none owed.** `## [Unreleased]` is empty — the 1.
|
|
205
|
-
its entries under `## [1.
|
|
236
|
+
(`latest` is `1.3.0`).
|
|
237
|
+
- **Next release: none owed.** `## [Unreleased]` is empty — the 1.3.0 cut moved
|
|
238
|
+
its entries under `## [1.3.0] - 2026-10-19`; the number comes from the
|
|
206
239
|
Unreleased scope, not from the open issues.
|
package/docs/dev/scoring.md
CHANGED
|
@@ -152,9 +152,22 @@ while every other weighted metric stays the model's own.
|
|
|
152
152
|
### Route ordering (the value-for-money score)
|
|
153
153
|
|
|
154
154
|
`rankedRoutes(model, def)` (`src/engine.ts`) is the one function that orders a
|
|
155
|
-
model's candidate routes; `bestRoute` is `rankedRoutes[0]`, and `roleRoute`
|
|
156
|
-
route `rankRole` prices the model on) and the updater's auto-pin
|
|
157
|
-
|
|
155
|
+
model's candidate routes; `bestRoute` is `rankedRoutes[0]`, and both `roleRoute`
|
|
156
|
+
(the route `rankRole` prices the model on) and the updater's auto-pin read that
|
|
157
|
+
one order. The priced route is the score-max candidate; the **written pin is the
|
|
158
|
+
first route in that order the route probe does not reject** (writes-and-state.md,
|
|
159
|
+
*The provider-pin auto-configuration*), so the two can differ — the probe is a
|
|
160
|
+
routability check on the write side only and never feeds the ranking. The
|
|
161
|
+
fallback chain follows the pin: it excludes the route the pin names (by slug, not
|
|
162
|
+
by position) and, with a dropped pin, excludes none. Which **models** the chain
|
|
163
|
+
falls back to is a separate choice (spec #95): its `fallbackChainDepth` fallback
|
|
164
|
+
models come from the role's **ranked candidate list** below the chosen model —
|
|
165
|
+
the cheaper-or-equal candidates that retain at least `fallbackQualityFloor` of
|
|
166
|
+
its `q` first, then the pricier ones, each bucket in value order, truncated to
|
|
167
|
+
the depth — **not** from the probe window, so the chain can reach a
|
|
168
|
+
cheaper-or-equal model ranked below the `1 + fallbackChainDepth` window the walk
|
|
169
|
+
probes (key-blocked and probe-`blocked` candidates are excluded; unprobed ones
|
|
170
|
+
stay eligible — writes-and-state.md, *The provider-pin auto-configuration*).
|
|
158
171
|
|
|
159
172
|
- **Candidates.** A route is a candidate when it is non-degraded (`status === 0`),
|
|
160
173
|
passes `routePassesEndpointFilters(route, def.filters)`, carries a usable billed
|
|
@@ -283,7 +296,11 @@ on the 2026-10-11 cache):
|
|
|
283
296
|
routes that pass the role's endpoint filters and the account's
|
|
284
297
|
allowed-providers whitelist (see *Whitelist-aware route selection*) — falling
|
|
285
298
|
back to
|
|
286
|
-
the default `1/price²` blend only when the model has no candidate route.
|
|
299
|
+
the default `1/price²` blend only when the model has no candidate route. The
|
|
300
|
+
**written** pin is the first route in that same order the route probe does not
|
|
301
|
+
reject (spec #90; writes-and-state.md, *The provider-pin auto-configuration*) —
|
|
302
|
+
the probe is a routability gate on the write side, so the priced (score-max)
|
|
303
|
+
route and the written pin can differ. It is
|
|
287
304
|
a **per-model policy, not a scalar** — the sensible pin differs per model, so
|
|
288
305
|
the bundle carries a policy field evaluated against the model's own routes at
|
|
289
306
|
rank time, not a slug baked into the lock file. `bestRoute` (`src/engine.ts`)
|
package/docs/dev/testing.md
CHANGED
|
@@ -34,17 +34,38 @@ passing.
|
|
|
34
34
|
best; cost override (`switched-cost`); `priceSwitchFraction: 0` keeps the
|
|
35
35
|
margin in charge; an inside-margin challenger short of the fraction stays
|
|
36
36
|
kept; the day gate.
|
|
37
|
-
6. **Chain pruning** (`chain-pruning.test.ts
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
the chosen
|
|
37
|
+
6. **Chain pruning + the quality floor** (`chain-pruning.test.ts`,
|
|
38
|
+
`chain-quality-floor.test.ts`) — plugin-written stale keys removed;
|
|
39
|
+
owner-written keys preserved; `writeFallbackChains: false`; a role that left
|
|
40
|
+
the managed set is pruned; the chain prefers candidates at or below the chosen
|
|
41
|
+
price. The floor (spec #95): a cheaper rank-4 candidate that clears the floor
|
|
42
|
+
becomes the first fallback where the pricier rank 2/3 do not, even though the
|
|
43
|
+
probe walk never reaches it; a cheap primary whose cheaper-or-equal candidates
|
|
44
|
+
are all below the floor keeps the next-best-by-value models;
|
|
45
|
+
`fallbackQualityFloor: 0` admits any cheaper-or-equal candidate and `1` only an
|
|
46
|
+
exactly-equal-quality one; an equal-priced sub-floor candidate is in neither
|
|
47
|
+
bucket and is dropped; a key-blocked cheaper candidate never enters the chain;
|
|
48
|
+
the chain stops at `fallbackChainDepth` in the cheaper set's value order; the
|
|
49
|
+
manual-pin path and the auto-pin chain shape are unchanged; an out-of-range
|
|
50
|
+
floor is a settings error that writes nothing; and a `kept-margin` role's chain
|
|
51
|
+
is built from the kept model's `q` and the candidates below it, with the
|
|
52
|
+
selector and the decision reason unaffected.
|
|
41
53
|
7. **Chain suffixes** (`chain-suffix.test.ts`) — chain values carry the role's
|
|
42
54
|
suffix, the key stays bare; an entry without thinking support gets no suffix;
|
|
43
55
|
two roles on one model share a level-free chain.
|
|
44
56
|
8. **Probe gate** (`probe-gate.test.ts`) — 200 → ok; the no-allowed-providers 404
|
|
45
57
|
→ blocked; any other failure → usable; blocked best → next adopted; blocked
|
|
46
|
-
current → replaced;
|
|
47
|
-
|
|
58
|
+
current → replaced; a probe-`blocked` candidate never enters a chain
|
|
59
|
+
(`fallback chains contain only probe-clean candidates`); verdicts cached across
|
|
60
|
+
roles; all-blocked → role untouched with a note. Spec #95 changed what the
|
|
61
|
+
chain's model set is built from: "blocked candidates beyond a kept current do
|
|
62
|
+
not count toward chain depth" was **deleted** (it encoded the removed
|
|
63
|
+
probe-window membership rule), "walk fills a kept role's chain to full depth
|
|
64
|
+
with probe-clean entries" was **renamed** to "a kept role's chain is filled to
|
|
65
|
+
full depth from the candidates below the kept model", and a new "the chain
|
|
66
|
+
reaches a cheaper candidate the walk never probes" pins that an **unprobed**
|
|
67
|
+
cheap candidate is chained while the walk still stops at
|
|
68
|
+
`1 + fallbackChainDepth` clean candidates.
|
|
48
69
|
9. **Keyed catalog** (`keyed-catalog.test.ts`) — key-blocked candidates pruned
|
|
49
70
|
from the walk; only candidates in neither catalog are probed; key-blocked
|
|
50
71
|
never in a chain; no-filter/unavailable degrade to the probe walk.
|
|
@@ -78,7 +99,8 @@ passing.
|
|
|
78
99
|
floor sorts a below-floor route last **without dropping it**; a route with no
|
|
79
100
|
p50 uses the model's blended throughput (for both the score and the floor
|
|
80
101
|
check); equal scores break by `providerSlug` then `id`; `explainModel`'s
|
|
81
|
-
`autoPin` is the score-max route
|
|
102
|
+
`autoPin` is the score-max route — the auto-pin's first **candidate**, which
|
|
103
|
+
the updater probes before writing it (spec #90).
|
|
82
104
|
14. **Thinking price** (`thinking-price.test.ts`) — bare/off unadjusted; a level
|
|
83
105
|
scales only thinking-capable models; the frontier uses the effective price;
|
|
84
106
|
`resolveSettings` validates thinking and rejects the legacy `suffixes` map;
|
|
@@ -149,6 +171,10 @@ passing.
|
|
|
149
171
|
`POST /api/pool` toggle returns the gated/pruned dataset in `as-written` mode
|
|
150
172
|
and the full one in `raw`; the bootstrap payload carries `poolMode` and
|
|
151
173
|
`tokenProfile`; the explain response carries `leaderFlip`/`thinkingLevels`.
|
|
174
|
+
The auto-pin indicator (spec #90): `explainModel`'s `autoPin` is the route the
|
|
175
|
+
updater's walk pins when every route probes clean, and it deliberately differs
|
|
176
|
+
from the written pin when the walk skips a blocked leader — the panel names
|
|
177
|
+
the *candidate*.
|
|
152
178
|
24. **Project scope** (`project-updater.test.ts`, `project-setup.test.ts`,
|
|
153
179
|
`project-roles-command.test.ts`) — project mode scopes config/state/history/
|
|
154
180
|
lock to `<cwd>/.omp`; the day gate is per scope; the registry write;
|
|
@@ -208,13 +234,30 @@ passing.
|
|
|
208
234
|
and `cut` bumping the four fields, cutting the changelog, committing and
|
|
209
235
|
tagging while refusing a dirty tree, an existing tag, an empty
|
|
210
236
|
`## [Unreleased]` and a HEAD that is not `origin/main`.
|
|
211
|
-
31. **Auto-pin** (`auto-pin.test.ts`) — the selector gains the
|
|
212
|
-
|
|
213
|
-
|
|
237
|
+
31. **Auto-pin** (`auto-pin.test.ts`) — the selector gains the `@<slug>` of the
|
|
238
|
+
first route in the role's `vfm` order that the route probe does not reject
|
|
239
|
+
(spec #90): a `blocked` leader pins the next route and records the skipped
|
|
240
|
+
slug (`routeBlockedSlugs`); `unknown` is usable and stays pinned; every route
|
|
241
|
+
blocked writes a bare selector and sets `pinDropped`; the notify lines carry
|
|
242
|
+
`; route-blocked: …` and `; pin dropped (no routable route)`; an all-blocked
|
|
243
|
+
pool issues at most `ROUTE_PROBE_BUDGET` probes; a verdict cached for one role
|
|
244
|
+
is reused by another; two blocked routes sharing a provider slug are recorded
|
|
245
|
+
once; and the capability off, a manual pin, a locked role, a disabled role and
|
|
246
|
+
a model with no gate-passing route never consult the probe (a throwing stub
|
|
247
|
+
proves it) and leave the config byte-identical. The chain carries every
|
|
248
|
+
gate-passing route of the primary except the pinned **slug** (not the pinned
|
|
249
|
+
position) then of each fallback model, in score order (the `vfm` route
|
|
214
250
|
score desc, spec #71), deduped by
|
|
215
251
|
provider slug; a degraded / over-`maxPriceUsdPerM` / endpoint-filter-failing /
|
|
216
|
-
whitelist-blocked route is excluded; the manual pin wins
|
|
217
|
-
|
|
252
|
+
whitelist-blocked route is excluded; the manual pin wins (its chain is the
|
|
253
|
+
fallback models only, every value carrying the manual slug) and the no-pin
|
|
254
|
+
path is bare (fallback models only); a re-run is idempotent and a pin change
|
|
255
|
+
prunes the old key. Since spec #95 the chain's fallback **models** come from
|
|
256
|
+
the ranked candidate list below the chosen model with the quality floor
|
|
257
|
+
(item 6) rather than the probe window, so the fixtures price each route at its
|
|
258
|
+
model's own price plus its position — routes all priced `1..n` would collapse
|
|
259
|
+
every model to one effective price and the floor would drop the weaker
|
|
260
|
+
fallbacks at equal price.
|
|
218
261
|
32. **Provider display** (`provider-display.test.ts`,
|
|
219
262
|
`provider-display-wiring.test.ts`) — `providerPinOf` reads the single
|
|
220
263
|
`openRouterRouting.only[0]` (else `vercelGatewayRouting`), preferring
|
|
@@ -253,6 +296,13 @@ passing.
|
|
|
253
296
|
catalog/tier gate (`resolveVariant`) and the allowed-providers prune
|
|
254
297
|
(`providerAllowed`), leaves the routes untouched on a `null` whitelist,
|
|
255
298
|
yields an empty pool on `tier: "none"`, and never mutates its input.
|
|
299
|
+
36. **Route probe** (`route-probe.test.ts`) — `probeRoute` sends the exact
|
|
300
|
+
one-token completion shape (POST, bearer, `max_tokens: 1`, `provider: { only:
|
|
301
|
+
[slug] }`): 200 → `ok`; the no-allowed-providers 404 → `blocked`; a 404 with
|
|
302
|
+
any other body, a 5xx, a 429 and a transport throw → `unknown` (usable). The
|
|
303
|
+
walk that consumes it (skip a `blocked` route, accept `unknown`, drop the pin
|
|
304
|
+
when nothing is clean, the per-role budget, the cross-role verdict cache) is
|
|
305
|
+
covered by the auto-pin cases above.
|
|
256
306
|
|
|
257
307
|
## Live checks
|
|
258
308
|
|
|
@@ -145,41 +145,128 @@ feeds the ranking, the reported `$/M` price, or any write. The recorded usage
|
|
|
145
145
|
takes precedence over it (`resolveTokenProfile`), so once a role has ledger rows
|
|
146
146
|
the setting is ignored for that role.
|
|
147
147
|
|
|
148
|
-
## The provider-pin auto-configuration (spec #58)
|
|
148
|
+
## The provider-pin auto-configuration (spec #58, routability-verified by #90)
|
|
149
149
|
|
|
150
150
|
When a role's **expanded** def sets `preferOwnProvider: true` (the
|
|
151
151
|
`providerPinning` capability's recommended bundle) and carries **no** manual
|
|
152
|
-
`providerPin`, the updater binds the role's requests to
|
|
153
|
-
|
|
152
|
+
`providerPin`, the updater binds the role's requests to one provider route by
|
|
153
|
+
writing that route's slug as the selector's `@<slug>` suffix:
|
|
154
154
|
|
|
155
155
|
```
|
|
156
156
|
openrouter/<id>@<slug>:<level>
|
|
157
157
|
```
|
|
158
158
|
|
|
159
159
|
omp's `splitUpstreamRouting` parses the trailing `@<slug>` and applies
|
|
160
|
-
`compat.openRouterRouting = { only: [slug] }
|
|
161
|
-
is the
|
|
162
|
-
|
|
163
|
-
*
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
160
|
+
`compat.openRouterRouting = { only: [slug] }`. The pin is the best route **the
|
|
161
|
+
probe accepts**, which is the route the role was priced on — the route the walk
|
|
162
|
+
tries first — so the price the ranking assumed is the price the request pays
|
|
163
|
+
*only while the walk does not skip that candidate*; a skipped candidate means the
|
|
164
|
+
route that replaced it pays, at its own price. The pin is re-derived on every
|
|
165
|
+
run, so it follows the ranking when the chosen model changes.
|
|
166
|
+
|
|
167
|
+
**The walk — the pin is probed before it is written.** The slug is **not** the
|
|
168
|
+
score-max route unconditionally: the updater walks the role's route order
|
|
169
|
+
(`rankedRoutes(chosenModel, def)` — the `vfm` order the chain also uses; see
|
|
170
|
+
scoring.md, *Route ordering*) and probes each candidate with `probeRoute`: a
|
|
171
|
+
one-token completion carrying `provider: { only: [slug] }`, the same request
|
|
172
|
+
skeleton and the same narrow no-allowed-providers pattern as `probeModel`. The
|
|
173
|
+
verdict vocabulary is `probeModel`'s:
|
|
174
|
+
|
|
175
|
+
- `blocked` — the narrow no-allowed-providers 404 — skips the route and the walk
|
|
176
|
+
continues to the next candidate;
|
|
177
|
+
- `ok` and **`unknown` are both usable**. A non-404 status (a 429, a 5xx), a 404
|
|
178
|
+
whose body does not carry the pattern, and a transport throw (a timeout, a
|
|
179
|
+
connection reset) are all `unknown`: a transient blip must never silently unpin
|
|
180
|
+
a role, and a rate limit is omp's runtime-fallback domain, not this plugin's.
|
|
181
|
+
|
|
182
|
+
Verdicts are cached per run by `catalogId@slug`, so a route shared by several
|
|
183
|
+
roles is probed once, and the walk is capped per role at **`ROUTE_PROBE_BUDGET =
|
|
184
|
+
12`** *new* probes (cached verdicts are free) — a pathological pool must not turn
|
|
185
|
+
the daily update into a long probe walk.
|
|
186
|
+
|
|
187
|
+
**Why a real request.** OpenRouter's endpoints API and its routing table
|
|
188
|
+
disagree: an endpoint can list `status: 0` with a full `supported_parameters`
|
|
189
|
+
list and 100 % uptime, pass the whitelist gate and carry a usable billed price,
|
|
190
|
+
and still 404 for `provider.only` — and the exclusion is dynamic (the same
|
|
191
|
+
request can succeed minutes later). The routing table is visible only in the 404
|
|
192
|
+
body of a request that failed, so a request is the only reliable signal. This is
|
|
193
|
+
exactly the failure the endpoint gates cannot see (spec #90: the live config held
|
|
194
|
+
`deepseek/deepseek-v4.1-flash@relace`, which 404'd on every revert).
|
|
195
|
+
|
|
196
|
+
**No routable candidate → no pin.** If every candidate is `blocked` (or the
|
|
197
|
+
budget runs out with no clean candidate), no pin is written: the selector stays
|
|
198
|
+
**bare** and OpenRouter routes the request itself, instead of omp failing on an
|
|
199
|
+
unroutable `@<slug>`. `Decision.pinDropped` is true in that case (see *State,
|
|
200
|
+
history, lock*), and the run notes it.
|
|
201
|
+
|
|
202
|
+
A manual `roles.<role>.providerPin` **wins** over the automatic choice and stays
|
|
203
|
+
a **hard** gate (a model with no matching route is ineligible). It is **never
|
|
204
|
+
probed and never replaced** — the probe applies only to the route the capability
|
|
205
|
+
itself chooses; clearing the manual pin resumes the automatic choice.
|
|
167
206
|
|
|
168
207
|
**Chain shape.** The primary's chain key is the selector minus the trailing
|
|
169
|
-
`:level` — so it **includes** the pin (`openrouter/<id>@<slug
|
|
170
|
-
order:
|
|
208
|
+
`:level` — so it **includes** the pin (`openrouter/<id>@<slug>`, or the bare
|
|
209
|
+
`openrouter/<id>` when the pin was dropped). Its values, in order:
|
|
171
210
|
|
|
172
|
-
1. the same model on **every** gate-passing route except the pinned
|
|
211
|
+
1. the same model on **every** gate-passing route except the pinned route, in
|
|
173
212
|
score order (the `vfm` route score desc — see scoring.md, *Route ordering*),
|
|
174
213
|
2. then, for each of the `fallbackChainDepth` fallback models, that model on
|
|
175
214
|
**every** gate-passing route, in score order.
|
|
176
215
|
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
216
|
+
**Which models are chained (spec #95).** The `fallbackChainDepth` fallback
|
|
217
|
+
**models** come from the role's whole **ranked candidate list** — the
|
|
218
|
+
tier-eligible ranking rows, in value order — not from the probe window (which is
|
|
219
|
+
`1 + fallbackChainDepth` models, the same size as the chain, so a reorder of it
|
|
220
|
+
can never reach a cheaper model at rank 4 or deeper). Two classes of candidate
|
|
221
|
+
are excluded, with the same predicates the probe walk uses: **key-blocked**
|
|
222
|
+
models (the account's key cannot use them) and models whose probe verdict is
|
|
223
|
+
`blocked`. An **unprobed** candidate stays — the chain's deeper entries are
|
|
224
|
+
never probe-verified (they are not a new class: chain entries are already
|
|
225
|
+
route-level unprobed, and omp fails over on an unroutable entry). `after` is
|
|
226
|
+
that list **below the chosen model** — the leader on a fresh pick, the kept
|
|
227
|
+
current candidate on a kept role (`kept-eligible` / `kept-margin`) — so the
|
|
228
|
+
`after` set and the floor both key off the model the selector actually names —
|
|
229
|
+
the previous `pool.slice(chosenIdx + 1)` keyed off the same index in the
|
|
230
|
+
probe-clean pool — and candidates ranked above the chosen are never chained. The
|
|
231
|
+
set is then split:
|
|
232
|
+
|
|
233
|
+
```
|
|
234
|
+
cheaper = after where priceEff <= chosen.priceEff AND q >= chosen.q · fallbackQualityFloor
|
|
235
|
+
pricier = after where priceEff > chosen.priceEff (never floor-filtered)
|
|
236
|
+
chain = [...cheaper, ...pricier].slice(0, fallbackChainDepth)
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
so a cheaper-or-equal fallback keeps the role's quality (the floor is a fraction
|
|
240
|
+
of the chosen model's **own** `q`, so it reads the same on every role's quality
|
|
241
|
+
scale) and a cheap-but-weak candidate set cannot starve the chain (the pricier
|
|
242
|
+
bucket always fills any remaining depth with the next-best by value). Both
|
|
243
|
+
buckets keep value order, so a cheaper candidate deep in the ranking is reached
|
|
244
|
+
only when fewer than `fallbackChainDepth` cheaper-or-equal candidates clear the
|
|
245
|
+
floor — a bounded search, not a cheapest-first sort. A candidate priced exactly
|
|
246
|
+
at the chosen model's price whose `q` is below the floor is in **neither** bucket
|
|
247
|
+
and drops out. `fallbackQualityFloor` (a number in `[0, 1]`, default `0.9`; `0`
|
|
248
|
+
disables the floor, `1` requires an exact quality match — validated, never
|
|
249
|
+
silently clamped) is documented in the README's settings table. The floor gates
|
|
250
|
+
**chain membership only**: the primary choice, the hysteresis decision, the
|
|
251
|
+
ranking, the probe walk's stop condition and budget, the chain key, the route
|
|
252
|
+
expansion, the thinking levels, the dedup and the prune-on-pin-change are all
|
|
253
|
+
unchanged.
|
|
254
|
+
|
|
255
|
+
The exclusion is **by pinned slug, not by position**: the probe can pin a route
|
|
256
|
+
other than the score leader (so the leader rides as a fallback entry while the
|
|
257
|
+
pinned route does not), and with a **dropped pin** (the auto-pin applied, but no
|
|
258
|
+
candidate was routable) **no** route is excluded — the primary contributes every
|
|
259
|
+
gate-passing route. Length = `(R_primary − 1) + Σ R_fallback`, or `R_primary + Σ
|
|
260
|
+
R_fallback` with a dropped pin, where `R` is the number of distinct gate-passing
|
|
261
|
+
provider slugs for that model — data-driven, not a fixed cap. Entries dedup by
|
|
262
|
+
chain value, so two routes sharing a provider slug collapse to one. Every entry
|
|
263
|
+
carries the same `@<slug>` suffix (and the role's thinking level where the target
|
|
264
|
+
advertises it).
|
|
265
|
+
**Chain entries are not probed** — only the primary pin is: the chain carries
|
|
266
|
+
many routes and one unroutable entry is survivable (omp fails over), so probing
|
|
267
|
+
it would multiply the request cost for no benefit (out of scope, spec #90). A
|
|
268
|
+
model with no gate-passing route contributes nothing. The gates are the engine's
|
|
269
|
+
`rankedRoutes` set: non-degraded
|
|
183
270
|
(`status === 0`), the endpoint capability filters (`tools` / `minContextTokens` /
|
|
184
271
|
`minOutputTokens`), a usable billed price, and the `maxPriceUsdPerM` cap
|
|
185
272
|
(thinking-adjusted). The soft `filters.preferredMinThroughput` floor is a
|
|
@@ -190,16 +277,28 @@ pool to allowed providers before ranking, which is what makes `rankedRoutes`
|
|
|
190
277
|
whitelist-aware. The probe walk target is unchanged (`1 + fallbackChainDepth` —
|
|
191
278
|
it counts models, not routes).
|
|
192
279
|
|
|
280
|
+
**Outside the auto-pin.** The chosen model contributes nothing when the auto-pin
|
|
281
|
+
does not apply: with a manual `providerPin` (`autoPinApplies` is false, so the
|
|
282
|
+
route probe does not run either) or with the capability off, the chain carries
|
|
283
|
+
**one entry per `fallbackChainDepth` fallback model** —
|
|
284
|
+
`openrouter/<fallbackId>@<manualSlug>` with a manual pin, the bare
|
|
285
|
+
`openrouter/<fallbackId>` with none — each with the role's thinking level where
|
|
286
|
+
the target advertises it, and no route of the chosen model at all.
|
|
287
|
+
|
|
193
288
|
**Prune on a pin change.** Because the chain key includes the pin, a pin change
|
|
194
289
|
changes the key; the updater's chain prune (`state.pluginWrittenChainKeys`) drops
|
|
195
290
|
the old key so a stale pinned chain does not linger.
|
|
196
291
|
|
|
197
|
-
**No-ops.** The auto-pin writes nothing
|
|
198
|
-
(`preferOwnProvider` not `true`); a
|
|
199
|
-
|
|
200
|
-
route data; no route passes the role's
|
|
201
|
-
|
|
202
|
-
to
|
|
292
|
+
**No-ops.** The auto-pin writes nothing — and **the route probe is never
|
|
293
|
+
consulted** — when: the capability is off (`preferOwnProvider` not `true`); a
|
|
294
|
+
manual `providerPin` is set; the role is locked or disabled; the selector is not
|
|
295
|
+
an OpenRouter selector; the model has no route data; no route passes the role's
|
|
296
|
+
endpoint filters; or no candidate has a usable billed price (the last three leave
|
|
297
|
+
`rankedRoutes` empty, so there is no candidate to probe, and `pinDropped` stays
|
|
298
|
+
false — the pin is simply absent). The case where the model *has* candidates but
|
|
299
|
+
none is routable is different: that is the **dropped pin** above — still no pin,
|
|
300
|
+
but recorded (`pinDropped`). With the capability off the written config is
|
|
301
|
+
byte-identical to before.
|
|
203
302
|
|
|
204
303
|
## State, history, lock
|
|
205
304
|
|
|
@@ -294,7 +393,13 @@ settings entry the global behavior is unchanged.
|
|
|
294
393
|
ids the role's endpoint filters dropped (no capable standard-tier route), rendered
|
|
295
394
|
`; endpoint-blocked: …` on the decision line; `pinBlocked[]` records the ranking ids
|
|
296
395
|
the role's provider pin dropped (no route matching the pin), rendered
|
|
297
|
-
`; pin-blocked: …`.
|
|
396
|
+
`; pin-blocked: …`. `routeBlockedSlugs[]` records the provider slugs the auto-pin's
|
|
397
|
+
route probe rejected on the chosen model, in walk order and deduped by slug
|
|
398
|
+
(two routes of one model can share a slug), rendered `; route-blocked: …`;
|
|
399
|
+
`pinDropped` is `true` when the auto-pin applies and the model has gate-passing
|
|
400
|
+
routes but no candidate survived the probe (or the `ROUTE_PROBE_BUDGET` ran
|
|
401
|
+
out), rendered `; pin dropped (no routable route)` — the selector is bare in
|
|
402
|
+
that case. Both are empty/false when no probe ran.
|
|
298
403
|
- Role removal (`/remove-agent`) goes through `removeRoleSettings` in
|
|
299
404
|
`src/role-settings.ts` — the same validate/backup/atomic-write path as
|
|
300
405
|
`writeRoleSettings`, deleting the role's flat dotted keys and any nested entry.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "omp-llm-role",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.4.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "omp plugin: daily model-role updater driven by llm-stats/OpenRouter rankings",
|
|
6
6
|
"license": "MIT",
|
|
@@ -63,6 +63,14 @@
|
|
|
63
63
|
"min": 0,
|
|
64
64
|
"step": 1
|
|
65
65
|
},
|
|
66
|
+
"fallbackQualityFloor": {
|
|
67
|
+
"type": "number",
|
|
68
|
+
"description": "Quality floor for a cheaper-or-equal fallback: the fraction of the chosen model's quality it must retain (0 disables)",
|
|
69
|
+
"default": 0.9,
|
|
70
|
+
"min": 0,
|
|
71
|
+
"max": 1,
|
|
72
|
+
"step": 0.001
|
|
73
|
+
},
|
|
66
74
|
"activateDefaultOnEmptySession": {
|
|
67
75
|
"type": "boolean",
|
|
68
76
|
"description": "Reapply the new default selector to an empty session's active model",
|
package/src/availability.ts
CHANGED
|
@@ -89,8 +89,8 @@ export type ProbeVerdict = "ok" | "blocked" | "unknown";
|
|
|
89
89
|
const OPENROUTER_COMPLETIONS_URL = "https://openrouter.ai/api/v1/chat/completions";
|
|
90
90
|
|
|
91
91
|
/** POST a one-token completion to OpenRouter with the shared request skeleton
|
|
92
|
-
* (Bearer auth, JSON body, 20s timeout). Shared by `probeModel`
|
|
93
|
-
* `fetchAllowedProviders`. */
|
|
92
|
+
* (Bearer auth, JSON body, 20s timeout). Shared by `probeModel`, `probeRoute`
|
|
93
|
+
* and `fetchAllowedProviders`. */
|
|
94
94
|
function completionRequest(token: string, body: unknown, fetchImpl: typeof fetch): Promise<Response> {
|
|
95
95
|
return fetchImpl(OPENROUTER_COMPLETIONS_URL, {
|
|
96
96
|
method: "POST",
|
|
@@ -110,6 +110,20 @@ function errorMessage(body: unknown): string {
|
|
|
110
110
|
* whitelist (or a data-policy toggle) leaves a model without a runnable endpoint. */
|
|
111
111
|
const BLOCKED_RE = /no allowed providers/i;
|
|
112
112
|
|
|
113
|
+
/**
|
|
114
|
+
* The verdict a probe response carries, shared by `probeModel` and `probeRoute`
|
|
115
|
+
* so the model probe and the route probe cannot drift apart: a 2xx is `ok`, the
|
|
116
|
+
* narrow no-allowed-providers 404 is `blocked`, and everything else is
|
|
117
|
+
* `unknown` (usable) — any other status (a 429, a 5xx), a 404 with any other
|
|
118
|
+
* body, an unparseable body. Never throws.
|
|
119
|
+
*/
|
|
120
|
+
async function verdictOf(res: Response): Promise<ProbeVerdict> {
|
|
121
|
+
if (res.ok) return "ok";
|
|
122
|
+
if (res.status !== 404) return "unknown";
|
|
123
|
+
const body: unknown = await res.json().catch(() => null);
|
|
124
|
+
return BLOCKED_RE.test(errorMessage(body)) ? "blocked" : "unknown";
|
|
125
|
+
}
|
|
126
|
+
|
|
113
127
|
/**
|
|
114
128
|
* One-token completion probe: the only reliable signal for
|
|
115
129
|
* account-level provider restrictions — neither `/api/v1/key` nor the catalog
|
|
@@ -121,11 +135,30 @@ const BLOCKED_RE = /no allowed providers/i;
|
|
|
121
135
|
export async function probeModel(token: string, catalogId: string, fetchImpl: typeof fetch = fetch): Promise<ProbeVerdict> {
|
|
122
136
|
try {
|
|
123
137
|
const res = await completionRequest(token, { model: catalogId, messages: [{ role: "user", content: "ok" }], max_tokens: 1 }, fetchImpl);
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
138
|
+
return await verdictOf(res);
|
|
139
|
+
} catch {
|
|
140
|
+
return "unknown";
|
|
141
|
+
}
|
|
142
|
+
}
|
|
143
|
+
|
|
144
|
+
/**
|
|
145
|
+
* One-token completion bound to a single provider route (`provider.only`), the
|
|
146
|
+
* same request skeleton as `probeModel` plus the route pin. The endpoints API
|
|
147
|
+
* and OpenRouter's routing table disagree — an endpoint can list `status: 0`
|
|
148
|
+
* with 100% uptime and still 404 — so a real request is the only reliable
|
|
149
|
+
* signal for whether a route is routable. Only the narrow no-allowed-providers
|
|
150
|
+
* 404 blocks a route; a non-404 status, a 404 with any other body and a
|
|
151
|
+
* transport throw are all "unknown" and count as usable, exactly as
|
|
152
|
+
* `probeModel` does. Never throws.
|
|
153
|
+
*/
|
|
154
|
+
export async function probeRoute(token: string, catalogId: string, providerSlug: string, fetchImpl: typeof fetch = fetch): Promise<ProbeVerdict> {
|
|
155
|
+
try {
|
|
156
|
+
const res = await completionRequest(
|
|
157
|
+
token,
|
|
158
|
+
{ model: catalogId, messages: [{ role: "user", content: "ok" }], max_tokens: 1, provider: { only: [providerSlug] } },
|
|
159
|
+
fetchImpl,
|
|
160
|
+
);
|
|
161
|
+
return await verdictOf(res);
|
|
129
162
|
} catch {
|
|
130
163
|
return "unknown";
|
|
131
164
|
}
|
package/src/explorer/explain.ts
CHANGED
|
@@ -347,9 +347,11 @@ export type Explanation =
|
|
|
347
347
|
* `preferOwnProvider` preference, no hard pin) rather than the 1/price²
|
|
348
348
|
* blend — the panel shows WHY the price is what it is. */
|
|
349
349
|
ownProvider: boolean;
|
|
350
|
-
/** the automatic pin: the model's best route's provider slug
|
|
351
|
-
*
|
|
352
|
-
*
|
|
350
|
+
/** the automatic pin's candidate: the model's best route's provider slug —
|
|
351
|
+
* the first route the updater's walk tries — when the role prefers a bound
|
|
352
|
+
* route and carries no manual pin, else null. The probe can skip to a later
|
|
353
|
+
* route or drop the pin, so this is the candidate, not the written pin.
|
|
354
|
+
* Read-only in the UI; a manual `providerPin` overrides it. */
|
|
353
355
|
autoPin: string | null;
|
|
354
356
|
role: { name: string; lambda: number; derivedLambda: number; wPrice: number; qW: number; thinking: SuffixLevel | undefined; cacheHitRate: number };
|
|
355
357
|
contributions: Contribution[];
|
|
@@ -389,9 +391,12 @@ export function explainModel(def: RoleDef, models: Model[], modelId: string, rol
|
|
|
389
391
|
const route = roleRoute(model, def);
|
|
390
392
|
// The model is priced on the soft best route (not a hard pin, not the blend).
|
|
391
393
|
const ownProvider = pinRoute === null && route !== null;
|
|
392
|
-
// The automatic pin
|
|
393
|
-
//
|
|
394
|
-
//
|
|
394
|
+
// The automatic pin's CANDIDATE route: the model's best route's slug — the
|
|
395
|
+
// first route the updater's walk tries — only when the role prefers a bound
|
|
396
|
+
// route and carries no manual pin. The updater probes that candidate and can
|
|
397
|
+
// pin a later route (or drop the pin entirely), so this is what the walk tries
|
|
398
|
+
// first, not a promise of the written pin. Reuses the already-computed `route`
|
|
399
|
+
// (the same best route) instead of recomputing it.
|
|
395
400
|
const autoPin = autoPinApplies(def) ? route?.providerSlug ?? null : null;
|
|
396
401
|
const missingRequired = def.required.filter((k) => roleMetricValue(eff, k, pinRoute, cacheHitRate) == null);
|
|
397
402
|
if (missingRequired.length > 0) reasons.push(`missing required metric(s): ${missingRequired.join(", ")}`);
|
package/src/settings.ts
CHANGED
|
@@ -235,6 +235,13 @@ export type ResolvedSettings = {
|
|
|
235
235
|
priceSwitchFraction: number;
|
|
236
236
|
writeFallbackChains: boolean;
|
|
237
237
|
fallbackChainDepth: number;
|
|
238
|
+
/**
|
|
239
|
+
* Quality floor for the fallback chain's cheaper-or-equal candidates: a
|
|
240
|
+
* candidate priced at or below the chosen model qualifies only if it retains
|
|
241
|
+
* at least this fraction of the chosen model's `q`. 0 disables the floor (any
|
|
242
|
+
* cheaper-or-equal candidate qualifies); 1 requires an exact quality match.
|
|
243
|
+
*/
|
|
244
|
+
fallbackQualityFloor: number;
|
|
238
245
|
roles: Record<string, RoleDef>;
|
|
239
246
|
activateDefaultOnEmptySession: boolean;
|
|
240
247
|
/** Opt-in reporting of unexpected plugin errors as GitHub issues on the
|
|
@@ -255,6 +262,7 @@ export const DEFAULT_SETTINGS: ResolvedSettings = {
|
|
|
255
262
|
priceSwitchFraction: 0.5,
|
|
256
263
|
writeFallbackChains: true,
|
|
257
264
|
fallbackChainDepth: 2,
|
|
265
|
+
fallbackQualityFloor: 0.9,
|
|
258
266
|
roles: DEFAULT_ROLES,
|
|
259
267
|
activateDefaultOnEmptySession: true,
|
|
260
268
|
errorReporting: "off",
|
|
@@ -306,6 +314,14 @@ export function deriveSettingsSchema(): Record<string, PluginSettingSchema> {
|
|
|
306
314
|
min: 0,
|
|
307
315
|
step: 1,
|
|
308
316
|
},
|
|
317
|
+
fallbackQualityFloor: {
|
|
318
|
+
type: "number",
|
|
319
|
+
description: "Quality floor for a cheaper-or-equal fallback: the fraction of the chosen model's quality it must retain (0 disables)",
|
|
320
|
+
default: DEFAULT_SETTINGS.fallbackQualityFloor,
|
|
321
|
+
min: 0,
|
|
322
|
+
max: 1,
|
|
323
|
+
step: 0.001,
|
|
324
|
+
},
|
|
309
325
|
activateDefaultOnEmptySession: {
|
|
310
326
|
type: "boolean",
|
|
311
327
|
description: "Reapply the new default selector to an empty session's active model",
|
|
@@ -579,6 +595,9 @@ export function resolveSettings(raw: Record<string, unknown>): { settings: Resol
|
|
|
579
595
|
if (!Number.isInteger(merged.fallbackChainDepth) || merged.fallbackChainDepth < 0) {
|
|
580
596
|
errors.push(`fallbackChainDepth: must be an integer >= 0, got ${JSON.stringify(merged.fallbackChainDepth)}`);
|
|
581
597
|
}
|
|
598
|
+
if (typeof merged.fallbackQualityFloor !== "number" || !Number.isFinite(merged.fallbackQualityFloor) || merged.fallbackQualityFloor < 0 || merged.fallbackQualityFloor > 1) {
|
|
599
|
+
errors.push(`fallbackQualityFloor: must be a number in [0, 1], got ${JSON.stringify(merged.fallbackQualityFloor)}`);
|
|
600
|
+
}
|
|
582
601
|
if (typeof merged.activateDefaultOnEmptySession !== "boolean") {
|
|
583
602
|
errors.push(
|
|
584
603
|
`${ACTIVATE_DEFAULT_KEY}: must be a boolean, got ${JSON.stringify(merged.activateDefaultOnEmptySession)}`,
|
package/src/updater.ts
CHANGED
|
@@ -15,8 +15,8 @@
|
|
|
15
15
|
|
|
16
16
|
import { readFileSync, statSync } from "node:fs";
|
|
17
17
|
import { dirname, join } from "node:path";
|
|
18
|
-
import { autoPinApplies,
|
|
19
|
-
import { currentRankingId, enrichThinkingLevels, fetchAllowedProviders, fetchKeyAvailability, fetchKeyMeta, filterCatalog, probeModel, providerAllowed, resolveVariant, THINKING_LEVELS, tierGate, type CatalogEntry, type KeyAvailability, type KeyMeta, type ProbeVerdict } from "./availability.ts";
|
|
18
|
+
import { autoPinApplies, computeRankings, endpointFilterDrops, loadRankData, META_LEVELS, providerPinDrops, rankedRoutes, type Ranked, type RankData, type RoleDef } from "./engine.ts";
|
|
19
|
+
import { currentRankingId, enrichThinkingLevels, fetchAllowedProviders, fetchKeyAvailability, fetchKeyMeta, filterCatalog, probeModel, probeRoute, providerAllowed, resolveVariant, THINKING_LEVELS, tierGate, type CatalogEntry, type KeyAvailability, type KeyMeta, type ProbeVerdict } from "./availability.ts";
|
|
20
20
|
import { ConfigEditError, parseConfig, patchConfig, writeConfigAtomic, type ConfigPatch } from "./config-edit.ts";
|
|
21
21
|
import { PLUGIN_SETTINGS_PATH, projectLockPath, readPluginSettingsMap, resolveSettings, roleUniverse, type ResolvedSettings } from "./settings.ts";
|
|
22
22
|
import { discoverAgentPins } from "./agent-pins.ts";
|
|
@@ -41,6 +41,15 @@ export type Decision = {
|
|
|
41
41
|
/** Ranking ids dropped by the role's provider pin (no route matching the pin) —
|
|
42
42
|
* recorded like a probe-blocked candidate. */
|
|
43
43
|
pinBlocked: string[];
|
|
44
|
+
/** Provider slugs the auto-pin's route probe rejected on the chosen model, in
|
|
45
|
+
* walk order — rendered `; route-blocked: …`. Slugs, not ids: the sibling
|
|
46
|
+
* `blocked` holds catalog ids and `pinBlocked` ranking ids. Empty when no
|
|
47
|
+
* probe ran. */
|
|
48
|
+
routeBlockedSlugs: string[];
|
|
49
|
+
/** True when the auto-pin applies and the model has gate-passing routes but
|
|
50
|
+
* every candidate was rejected (or the route-probe budget ran out): the
|
|
51
|
+
* selector stays bare and OpenRouter routes the request itself. */
|
|
52
|
+
pinDropped: boolean;
|
|
44
53
|
/** Availability source for this role's selection: the keyed-catalog fast path or the probe walk. */
|
|
45
54
|
availabilitySource: "keyed-catalog" | "probe";
|
|
46
55
|
/** Candidates pruned as key-blocked (present in the public catalog, absent from the keyed one). */
|
|
@@ -68,10 +77,18 @@ export type Trigger = "session-start" | "manual";
|
|
|
68
77
|
/** A ranked model resolved to a concrete catalog id. */
|
|
69
78
|
type Candidate = { ranked: Ranked; catalogId: string };
|
|
70
79
|
|
|
71
|
-
/** Per managed role: its bare chain key, role suffix, provider pin, the
|
|
72
|
-
*
|
|
73
|
-
*
|
|
74
|
-
|
|
80
|
+
/** Per managed role: its bare chain key, role suffix, provider pin, the
|
|
81
|
+
* candidates its chain's fallback models are drawn from, the chosen model's
|
|
82
|
+
* index in that list, and the inputs the auto-pin chain builder needs (the
|
|
83
|
+
* expanded def for `rankedRoutes`, and whether the pin was auto-derived).
|
|
84
|
+
* `candidates` is the role's ranked candidate list minus the candidates no
|
|
85
|
+
* chain entry may name (key-blocked, probe-blocked) — not the probe-clean pool
|
|
86
|
+
* — so the chain can reach a cheaper-or-equal model ranked below the probe
|
|
87
|
+
* window. `pinnedSlug` is the route the auto-pin wrote — undefined for a manual
|
|
88
|
+
* pin, for a dropped pin and with the capability off — so the chain builder can
|
|
89
|
+
* exclude the pinned route by slug rather than by its position in the route
|
|
90
|
+
* order. */
|
|
91
|
+
type ChainPlan = { key: string; suffix: string | undefined; pin: string; pinnedSlug: string | undefined; candidates: Candidate[]; chosenIdx: number; def: RoleDef; autoPin: boolean };
|
|
75
92
|
|
|
76
93
|
export type Deps = {
|
|
77
94
|
getToken(): Promise<string>;
|
|
@@ -83,6 +100,13 @@ export type Deps = {
|
|
|
83
100
|
getSettings?(): Promise<Record<string, unknown>>;
|
|
84
101
|
getKeyMeta?(token: string): Promise<KeyMeta>;
|
|
85
102
|
probeModel?(token: string, catalogId: string): Promise<ProbeVerdict>;
|
|
103
|
+
/**
|
|
104
|
+
* DI seam like `probeModel`; production defaults to the real `probeRoute`.
|
|
105
|
+
* One-token completion bound to a single provider route (`provider.only`), the
|
|
106
|
+
* auto-pin's routability gate: OpenRouter's routing table can exclude a route
|
|
107
|
+
* its endpoints API still lists as healthy.
|
|
108
|
+
*/
|
|
109
|
+
probeRoute?(token: string, catalogId: string, providerSlug: string): Promise<ProbeVerdict>;
|
|
86
110
|
/** DI seam like `getKeyMeta`/`probeModel`; production defaults to `fetchKeyAvailability` (never throws). */
|
|
87
111
|
getKeyAvailability?(token: string): Promise<KeyAvailability>;
|
|
88
112
|
/**
|
|
@@ -106,6 +130,12 @@ const CONFLICT_RETRIES = 3;
|
|
|
106
130
|
/** Per-role cap on allowlist probe requests (budget against pathological all-blocked pools). */
|
|
107
131
|
const PROBE_BUDGET = 12;
|
|
108
132
|
|
|
133
|
+
/** Per-role cap on route-probe requests for the auto-pin walk (budget against a
|
|
134
|
+
* pathological route pool: a fully blocked model with dozens of routes must not
|
|
135
|
+
* turn the daily update into a long probe walk). Hitting the cap with no clean
|
|
136
|
+
* candidate drops the pin, exactly as an all-blocked pool does. */
|
|
137
|
+
export const ROUTE_PROBE_BUDGET = 12;
|
|
138
|
+
|
|
109
139
|
function mtimeOf(path: string): number {
|
|
110
140
|
try {
|
|
111
141
|
return statSync(path).mtimeMs;
|
|
@@ -120,9 +150,11 @@ function decisionLine(d: Decision): string {
|
|
|
120
150
|
const blocked = d.blocked.length > 0 ? `; blocked: ${d.blocked.join(", ")}` : "";
|
|
121
151
|
const endpoint = d.endpointBlocked.length > 0 ? `; endpoint-blocked: ${d.endpointBlocked.join(", ")}` : "";
|
|
122
152
|
const pin = d.pinBlocked.length > 0 ? `; pin-blocked: ${d.pinBlocked.join(", ")}` : "";
|
|
153
|
+
const routes = d.routeBlockedSlugs.length > 0 ? `; route-blocked: ${d.routeBlockedSlugs.join(", ")}` : "";
|
|
154
|
+
const dropped = d.pinDropped ? "; pin dropped (no routable route)" : "";
|
|
123
155
|
return d.from === d.to
|
|
124
|
-
? `@${d.role}: kept ${d.to} (${d.reason}; ${scores}${blocked}${endpoint}${pin})`
|
|
125
|
-
: `@${d.role}: ${from} -> ${d.to} (${d.reason}; ${scores}${blocked}${endpoint}${pin})`;
|
|
156
|
+
? `@${d.role}: kept ${d.to} (${d.reason}; ${scores}${blocked}${endpoint}${pin}${routes}${dropped})`
|
|
157
|
+
: `@${d.role}: ${from} -> ${d.to} (${d.reason}; ${scores}${blocked}${endpoint}${pin}${routes}${dropped})`;
|
|
126
158
|
}
|
|
127
159
|
|
|
128
160
|
/**
|
|
@@ -372,6 +404,10 @@ export async function runUpdater(trigger: Trigger, deps: Deps, opts?: { force?:
|
|
|
372
404
|
const pinUnmatched: string[] = [];
|
|
373
405
|
const probe = deps.probeModel ?? probeModel;
|
|
374
406
|
const probeVerdicts = new Map<string, ProbeVerdict>();
|
|
407
|
+
const routeProbe = deps.probeRoute ?? probeRoute;
|
|
408
|
+
/** Route verdicts, cached by `catalogId@slug` across roles: a route shared by
|
|
409
|
+
* several roles is probed once per run. */
|
|
410
|
+
const routeVerdicts = new Map<string, ProbeVerdict>();
|
|
375
411
|
for (const [role, def] of Object.entries(settings.roles)) {
|
|
376
412
|
// Locked: no ranking, no probe, no decision, no selector/chain write.
|
|
377
413
|
if (lockedRoles.has(role)) continue;
|
|
@@ -483,14 +519,50 @@ export async function runUpdater(trigger: Trigger, deps: Deps, opts?: { force?:
|
|
|
483
519
|
// `splitUpstreamRouting` parses the trailing `@<slug>` and applies
|
|
484
520
|
// `compat.openRouterRouting = { only: [slug] }`.
|
|
485
521
|
//
|
|
486
|
-
// Auto-pin (spec #58): with the soft
|
|
487
|
-
// no manual pin, the
|
|
488
|
-
// rides as that suffix
|
|
489
|
-
//
|
|
490
|
-
//
|
|
522
|
+
// Auto-pin (spec #58, routability-verified by spec #90): with the soft
|
|
523
|
+
// `preferOwnProvider` capability on and no manual pin, the best route the
|
|
524
|
+
// probe accepts rides as that suffix. That candidate is the route the role
|
|
525
|
+
// was priced on, so the price the ranking assumed is the price the request
|
|
526
|
+
// pays — but only while the walk does not skip the candidate; a skipped
|
|
527
|
+
// candidate means the route that replaced it pays, at its own price. A
|
|
528
|
+
// manual `providerPin` is a hard gate and always wins; a model with no
|
|
529
|
+
// gate-passing route keeps default routing (bare selector).
|
|
530
|
+
//
|
|
531
|
+
// The route order is the one the fallback chain already uses
|
|
532
|
+
// (`rankedRoutes`). Every candidate is probed with `provider: { only:
|
|
533
|
+
// [slug] }`, because OpenRouter's endpoints API can list a route as
|
|
534
|
+
// healthy while its routing table excludes it — the failure the endpoints
|
|
535
|
+
// data cannot see. A `blocked` route is skipped, `unknown` counts as
|
|
536
|
+
// usable (a transient blip must never unpin a role), and a pool with no
|
|
537
|
+
// clean candidate drops the pin: the selector stays bare and OpenRouter
|
|
538
|
+
// routes the request itself instead of 404ing on an unroutable `@<slug>`.
|
|
491
539
|
const manualPin = def.providerPin;
|
|
492
|
-
const
|
|
493
|
-
|
|
540
|
+
const autoRoutes = autoPinApplies(def) ? rankedRoutes(chosen.ranked.model, def) : [];
|
|
541
|
+
// No gate-passing route means nothing to pin and nothing to drop: the bare
|
|
542
|
+
// selector is default routing, not a dropped pin.
|
|
543
|
+
const hasAutoRoutes = autoRoutes.length > 0;
|
|
544
|
+
const routeBlockedSlugs: string[] = [];
|
|
545
|
+
let autoSlug: string | undefined;
|
|
546
|
+
let routeProbesSpent = 0;
|
|
547
|
+
for (const r of autoRoutes) {
|
|
548
|
+
const key = `${chosen.catalogId}@${r.providerSlug}`;
|
|
549
|
+
let verdict = routeVerdicts.get(key);
|
|
550
|
+
if (verdict === undefined) {
|
|
551
|
+
if (routeProbesSpent >= ROUTE_PROBE_BUDGET) break;
|
|
552
|
+
verdict = await routeProbe(token, chosen.catalogId, r.providerSlug);
|
|
553
|
+
routeVerdicts.set(key, verdict);
|
|
554
|
+
routeProbesSpent++;
|
|
555
|
+
}
|
|
556
|
+
if (verdict === "blocked") {
|
|
557
|
+
// Deduped: two routes can share a provider slug (the chain collapses
|
|
558
|
+
// them too), and the decision line should not list one slug twice.
|
|
559
|
+
if (!routeBlockedSlugs.includes(r.providerSlug)) routeBlockedSlugs.push(r.providerSlug);
|
|
560
|
+
continue;
|
|
561
|
+
}
|
|
562
|
+
autoSlug = r.providerSlug;
|
|
563
|
+
break;
|
|
564
|
+
}
|
|
565
|
+
const pinSlug = manualPin ?? autoSlug;
|
|
494
566
|
const pin = pinSlug !== undefined ? `@${pinSlug}` : "";
|
|
495
567
|
const chosenRow = rowById.get(chosen.catalogId);
|
|
496
568
|
// Append only a level the model's catalog thinking[] actually supports —
|
|
@@ -509,16 +581,28 @@ export async function runUpdater(trigger: Trigger, deps: Deps, opts?: { force?:
|
|
|
509
581
|
blocked: blockedForRole,
|
|
510
582
|
endpointBlocked: endpointFilterDrops(def, rank.models),
|
|
511
583
|
pinBlocked: providerPinDrops(def, rank.models),
|
|
584
|
+
routeBlockedSlugs,
|
|
585
|
+
pinDropped: hasAutoRoutes && autoSlug === undefined,
|
|
512
586
|
availabilitySource: availability.active ? "keyed-catalog" : "probe",
|
|
513
587
|
keyBlockedCount,
|
|
514
588
|
});
|
|
515
589
|
|
|
516
590
|
// Fallback chain inputs for every managed role: the bare key (a chain key
|
|
517
|
-
// matches the active model id, never a level) plus the
|
|
518
|
-
//
|
|
591
|
+
// matches the active model id, never a level) plus the candidate list the
|
|
592
|
+
// chain's fallback models are drawn from. Values are built after the loop,
|
|
593
|
+
// once every key claim is known.
|
|
519
594
|
if (settings.writeFallbackChains) {
|
|
520
595
|
const key = suffix !== undefined && finalSelector.endsWith(`:${suffix}`) ? finalSelector.slice(0, -(suffix.length + 1)) : finalSelector;
|
|
521
|
-
|
|
596
|
+
// The chain's fallback models come from the role's whole ranked
|
|
597
|
+
// candidate list, so a cheaper-or-equal model ranked below the probe
|
|
598
|
+
// window can be chained. Only the candidates the probe walk would never
|
|
599
|
+
// accept are dropped — key-blocked (the account's key cannot use them)
|
|
600
|
+
// and probe-verdict-blocked (present in the public catalog but not for
|
|
601
|
+
// this key). An unprobed candidate stays: the chain's deeper entries are
|
|
602
|
+
// no longer required to be probe-verified. The chosen model passed both
|
|
603
|
+
// predicates, so it is always in the list and `chosenIdx` is never -1.
|
|
604
|
+
const chainCandidates = candidates.filter((c) => !keyBlocked(c) && probeVerdicts.get(c.catalogId) !== "blocked");
|
|
605
|
+
chainPlanByRole[role] = { key, suffix, pin, pinnedSlug: autoSlug, candidates: chainCandidates, chosenIdx: chainCandidates.indexOf(chosen), def, autoPin: hasAutoRoutes };
|
|
522
606
|
}
|
|
523
607
|
}
|
|
524
608
|
|
|
@@ -539,13 +623,18 @@ export async function runUpdater(trigger: Trigger, deps: Deps, opts?: { force?:
|
|
|
539
623
|
for (const d of decisions) {
|
|
540
624
|
const plan = chainPlanByRole[d.role];
|
|
541
625
|
const suffix = claims[plan.key] === 1 ? plan.suffix : undefined;
|
|
542
|
-
// Cost-aware ordering: prefer fallbacks priced at or
|
|
543
|
-
//
|
|
544
|
-
//
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
|
|
626
|
+
// Cost-aware ordering with a quality floor: prefer fallbacks priced at or
|
|
627
|
+
// below the chosen model that still retain `fallbackQualityFloor` of its
|
|
628
|
+
// quality (a failover must not silently drop the role onto a much weaker
|
|
629
|
+
// model), then fill any remaining depth with the next-best by value. The
|
|
630
|
+
// floor and the `after` set both key off the chosen model — the leader on
|
|
631
|
+
// a fresh pick, the kept current candidate on a kept role — and the floor
|
|
632
|
+
// never filters the pricier set, so a cheap-but-weak candidate set cannot
|
|
633
|
+
// starve the chain.
|
|
634
|
+
const after = plan.candidates.slice(plan.chosenIdx + 1);
|
|
635
|
+
const chosen = plan.candidates[plan.chosenIdx];
|
|
636
|
+
const cheaper = after.filter((p) => p.ranked.priceEff <= chosen.ranked.priceEff && p.ranked.q >= chosen.ranked.q * settings.fallbackQualityFloor);
|
|
637
|
+
const pricier = after.filter((p) => p.ranked.priceEff > chosen.ranked.priceEff);
|
|
549
638
|
const chainPool = [...cheaper, ...pricier].slice(0, settings.fallbackChainDepth);
|
|
550
639
|
// A chain value's thinking level is appended only when the target's
|
|
551
640
|
// catalog entry advertises support (or the level is a meta level).
|
|
@@ -556,7 +645,7 @@ export async function runUpdater(trigger: Trigger, deps: Deps, opts?: { force?:
|
|
|
556
645
|
const values: string[] = [];
|
|
557
646
|
if (plan.autoPin) {
|
|
558
647
|
// Auto-pin chain (spec #58, expanded by #69): every gate-passing route
|
|
559
|
-
// of the primary model
|
|
648
|
+
// of the primary model except the pinned route, then every
|
|
560
649
|
// gate-passing route of each fallback model, in the role's
|
|
561
650
|
// value-for-money score order (`rankedRoutes`, spec #71) — every entry
|
|
562
651
|
// bound to its route's provider, so a down primary
|
|
@@ -564,12 +653,17 @@ export async function runUpdater(trigger: Trigger, deps: Deps, opts?: { force?:
|
|
|
564
653
|
// than a different model. `rankedRoutes` already applies every gate
|
|
565
654
|
// (non-degraded, endpoint filters, usable billed price, the
|
|
566
655
|
// maxPriceUsdPerM cap, the allowed-providers whitelist), so the chain
|
|
567
|
-
// carries exactly the routes the role can use. The
|
|
568
|
-
//
|
|
569
|
-
//
|
|
570
|
-
//
|
|
571
|
-
|
|
572
|
-
|
|
656
|
+
// carries exactly the routes the role can use. The pin is excluded by
|
|
657
|
+
// slug, not by position: the route probe can pin a route other than
|
|
658
|
+
// the score leader, and with a dropped pin no route is excluded at
|
|
659
|
+
// all. The dedup below collapses two routes that share a provider slug
|
|
660
|
+
// (same chain value), so the chain length is (R_primary − 1) + Σ
|
|
661
|
+
// R_fallback over distinct gate-passing provider slugs (R_primary with
|
|
662
|
+
// no pin). Only the pinned route is probe-verified: one unroutable
|
|
663
|
+
// chain entry is survivable, omp fails over.
|
|
664
|
+
const chosenCandidate = plan.candidates[plan.chosenIdx];
|
|
665
|
+
for (const r of rankedRoutes(chosenCandidate.ranked.model, plan.def)) {
|
|
666
|
+
if (r.providerSlug === plan.pinnedSlug) continue;
|
|
573
667
|
values.push(`openrouter/${chosenCandidate.catalogId}@${r.providerSlug}${levelFor(chosenCandidate.catalogId)}`);
|
|
574
668
|
}
|
|
575
669
|
for (const p of chainPool) {
|
package/web/app.js
CHANGED
|
@@ -185,6 +185,7 @@ const TIPS = {
|
|
|
185
185
|
focusCrossSource: "Cross-source: the zeroeval per-entry price/context compared against the plugin's own price/context for the models carrying the metric (the median relative divergence, within a 0.5 tolerance). The speed figure is informational only (rps vs tok/s). An annotation, never a gate — it never feeds the 1/price² blend; unknown means no model joined or the payload carried none of the four fields.",
|
|
186
186
|
imageFilter: "Require image input (filters.image) — the gate that shrinks the vision role's eligible set.",
|
|
187
187
|
providerPin: "Hard provider pin: the OpenRouter provider slug this role's requests are routed to (a tiered slug like deepinfra/fp8 is accepted verbatim). Setting it opts the role out of the automatic best-route pin; empty = default OpenRouter routing.",
|
|
188
|
+
autoPin: "Automatic provider pin (candidate): this model's best route by the role's value-for-money score — the first route the updater's daily run tries. The updater probes that route with a one-token request bound to it (provider: { only: [slug] }) and skips to the next route when OpenRouter does not serve it, dropping the pin entirely (bare selector, OpenRouter's own routing) when none is routable — so the pin that gets written can be a later route, or absent.",
|
|
188
189
|
features: "Opt-in capability presets (issue #40). Each flag applies the plugin's recommended settings for that capability to this role; an explicit knob value always wins. Export writes the flag, not the expanded values, so a future change to the recommendation keeps reaching the role.",
|
|
189
190
|
featureState: "inherit = use the global flag (features.<id> in the lock file); on/off = a per-role override (roles.<role>.features.<id>) that beats the global in either direction.",
|
|
190
191
|
featureKnob: "An individual knob the capability's recommended bundle fills. Leave it at inherit to take the recommendation; set a value to override it (an explicit value always wins over the flag).",
|
|
@@ -562,7 +563,12 @@ function renderExplain() {
|
|
|
562
563
|
if (manualPin) {
|
|
563
564
|
panel.append(el("p", { class: "sub pin-line", "data-tip": TIPS.providerPin, text: "pin: " + manualPin + " (manual)" }));
|
|
564
565
|
} else if (ex.autoPin) {
|
|
565
|
-
|
|
566
|
+
// The candidate route — the first route the updater's value-ordered walk
|
|
567
|
+
// tries — not a promise of the written pin: the updater probes it and skips
|
|
568
|
+
// to the next route when OpenRouter does not serve it, or drops the pin
|
|
569
|
+
// entirely when none is routable (spec #90). Read-only, offline: the
|
|
570
|
+
// explorer cannot probe, so it names the candidate and says so in the tip.
|
|
571
|
+
panel.append(el("p", { class: "sub pin-line", "data-tip": TIPS.autoPin, text: "auto pin (candidate): " + ex.autoPin }));
|
|
566
572
|
}
|
|
567
573
|
|
|
568
574
|
panel.append(el("h3", { "data-tip": TIPS.composition, text: "Value composition" }));
|