omp-llm-role 1.3.0 → 1.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -126,6 +126,7 @@ Global knobs:
126
126
  | `priceSwitchFraction` | `0.5` | Inside the margin, adopt a challenger at least this fraction cheaper (0.5 = half the $/M); `0` disables |
127
127
  | `writeFallbackChains` | `true` | Write per-role fallback chains into `config.yml` |
128
128
  | `fallbackChainDepth` | `2` | Fallback models per role chain |
129
+ | `fallbackQualityFloor` | `0.9` | A cheaper-or-equal fallback must retain at least this fraction of the chosen model's quality; `0` disables |
129
130
  | `activateDefaultOnEmptySession` | `true` | Reapply the new `default` selector to an empty session's active model |
130
131
  | `tokenProfile` | unset | `{ inputTokens, outputTokens }` per task for the explorer's `$/task` column, used when a role has no recorded usage. Display-only; the recorded usage takes precedence. Unset = the column reads `—` |
131
132
  | `errorReporting` | `off` | Opt-in reporting of **unexpected** plugin errors as GitHub issues on `Gott50/omp-llm-role`; one of `off`, `ask`, `auto` (see below) |
@@ -160,7 +161,7 @@ defaults.
160
161
  |---|---|---|
161
162
  | `endpointCeilings` | `filters.tools: true`, `filters.minOutputTokens: 16384` | Routes that cannot call tools and routes whose output ceiling is below 16 384 tokens leave the pool — a reasoning model is not cut off mid-thought |
162
163
  | `cachePricing` | `cacheHitRate: 0.5` | The role is priced on a cache-heavy agent loop's invoice (the endpoint's `cacheReadPrice` blended in) rather than the sticker price |
163
- | `providerPinning` | `preferOwnProvider: true`, `filters.preferredMinThroughput: 50` | Each request is bound to the model's **best provider route** — the max-value-for-money route (the `vfm` score, with a soft 50 tok/s floor that deprioritizes a pathologically slow route without dropping it) — the selector gains `@<slug>` — and the fallback chain carries the primary model on every gate-passing route except the pinned best, then each fallback model on every gate-passing route, so a down provider fails over to another provider of the same model before the chain switches models. The model is priced **and measured** on that route (its billed price and p50 throughput, so `q` moves too), falling back to the default blend only when it has no candidate route |
164
+ | `providerPinning` | `preferOwnProvider: true`, `filters.preferredMinThroughput: 50` | Each request is bound to a provider route **that OpenRouter actually serves**: the updater walks the model's value-for-money route order (the `vfm` score, with a soft 50 tok/s floor that deprioritizes a pathologically slow route without dropping it) and **probes each candidate with a one-token request bound to it** (`provider: { only: [slug] }`), pinning the first route the request does not reject and **dropping the pin** (bare selector, OpenRouter's own routing) when no candidate is routable — so a role never 404s on a route the endpoints data still called healthy. The selector gains `@<slug>`, and the fallback chain carries the primary model on every gate-passing route except the pinned one, then each of the next `fallbackChainDepth` fallback models on every gate-passing route — each drawn from the role's own ranking below the primary, the cheaper-or-equal ones that keep at least `fallbackQualityFloor` of its quality first, then the pricier — so a down provider fails over to another provider of the same model before the chain switches models, and a failover never silently multiplies the spend or drops the role onto a much weaker model. The model is priced **and measured** on its score-max route (its billed price and p50 throughput, so `q` moves too), falling back to the default blend only when it has no candidate route |
164
165
  | `costCap` | `filters.maxPriceUsdPerM: 10` | The priciest tail of the eligible pool leaves it, so the pick cannot be an expensive outlier |
165
166
 
166
167
  Precedence is one rule:
@@ -29,12 +29,12 @@ the extension's `/refresh-roles`, `/explore-roles` and `/create-agent`.
29
29
  | `src/benchmark-sources.ts` | The benchmark-source registry: the single source of truth for which sources exist, what metric each feeds, and how to fetch/parse/join each. Exports `BENCHMARK_SOURCES` (the shipped static sources), `resolveBenchmarkSource`, `declaredSourceForLink`, `sourceForMetric`, `parseBenchmarkPayload`, `parseBenchmarkPayloadMeta`, `loadBenchmarkScores`, `applyBenchmarkScores`, `joinBenchmarkScores`, `normalizeMetricKey`, `externalMetricKey`, `normalizeDesignId`, `parseWritingEvidence`, `parseLlmStatsBenchmark`, `parseBenchmarkCatalog`, `loadBenchmarkCatalog`, `cachedSourceInfo`, `catalogMetric`, `SHIPPED_CATALOG_METRICS`, the declarative-source surface (`SourceDeclaration`, `validateDeclaration`, `parseSourceDeclaration`, `loadDeclaredSources`, `saveDeclaredSource`, `declaredSourcesPath`, `declarationToSource`, `executeDeclaration`, `llmStatsBenchmarkDeclaration`, `genericBenchmarkSource`, `dryRunDeclaration`), the fetch helpers (`fetchJson`, `fetchText`), and the `BenchmarkSource`/`JoinRule`/`BenchmarkScores`/`BenchmarkPayloadMeta`/`FocusSourceInfo`/`BenchmarkCatalogEntry`/`DeclarationDryRun` types. Imports `engine.ts` for types only, so there is no runtime cycle. |
30
30
  | `src/settings.ts` | Shipped defaults and the settings merge/validate path. Exports `DEFAULT_ROLES`, `DEFAULT_SETTINGS`, `SHIPPED_AGENTS`, `KNOWN_METRICS`, `isKnownMetric` (a shipped key or an external `<ns>:<local>`), `PLUGIN_SETTINGS_PATH`, `ACTIVATE_DEFAULT_KEY`, `deriveSettingsSchema`, `readPluginSettingsMap`, `resolveSettings`, `roleUniverse`, `deepMergeInto`, `findProjectAnchor`, `projectLockPath`, and the `ResolvedSettings`/`PluginSettingSchema`/`UniverseEntry`/`RoleKind` types. `ResolvedSettings` carries `tokenProfile` (`{ inputTokens, outputTokens } | null`, default `null`) — the user-set per-task token profile the explorer's per-task price falls back to when a role has no recorded usage; validated to finite numbers ≥ 0, schema rows `tokenProfile.inputTokens`/`tokenProfile.outputTokens`. `readPluginSettingsMap` merges the project lock file (`projectLockPath(cwd)` = `<cwd>/.omp/plugins/omp-plugins.lock.json`, omp's project dir with **no walk-up**) over the user-level one; `findProjectAnchor` (the walk-up `.omp`/`.git` anchor) is kept only for the `/project-roles` divergence warning. |
31
31
  | `src/agent-pins.ts` | Agent → pinned-role derivation. Exports `parseAgentPin` (first `@<role>` in the `model:` frontmatter) and `discoverAgentPins` (scans the shipped, user and project agent dirs; project > user > plugin). Drives the `task.disabledAgents` sync. |
32
- | `src/availability.ts` | Key tier gate, catalog filter, variant resolution, provider-allowlist probe, the keyed-catalog availability primitive, and the allowed-providers whitelist harvest. The keyed source is the Bearer-authenticated `GET /api/v1/models` (the account setting "Filter the model catalog for API keys" makes it a per-key allowlist; set membership is the only signal), shared by the explorer and the updater's companion fast path. Exports `fetchKeyMeta`, `tierGate`, `filterCatalog`, `rankingIdOf`, `currentRankingId`, `resolveVariant`, `enrichThinkingLevels`, `probeModel`, `fetchAllowedProviders`, `providerAllowed`, `computeKeyAvailability`, `fetchKeyAvailability`, `catalogFromOmpModelsJson`, `THINKING_LEVELS`, and the `CatalogEntry`/`KeyMeta`/`Tier`/`ProbeVerdict`/`KeyAvailability` types. |
32
+ | `src/availability.ts` | Key tier gate, catalog filter, variant resolution, the model probe, the auto-pin's route probe, the keyed-catalog availability primitive, and the allowed-providers whitelist harvest. The keyed source is the Bearer-authenticated `GET /api/v1/models` (the account setting "Filter the model catalog for API keys" makes it a per-key allowlist; set membership is the only signal), shared by the explorer and the updater's companion fast path. Exports `fetchKeyMeta`, `tierGate`, `filterCatalog`, `rankingIdOf`, `currentRankingId`, `resolveVariant`, `enrichThinkingLevels`, `probeModel`, `probeRoute` (the same one-token completion plus `provider: { only: [slug] }` — the auto-pin's routability gate; spec #90), `fetchAllowedProviders`, `providerAllowed`, `computeKeyAvailability`, `fetchKeyAvailability`, `catalogFromOmpModelsJson`, `THINKING_LEVELS`, and the `CatalogEntry`/`KeyMeta`/`Tier`/`ProbeVerdict`/`KeyAvailability` types. |
33
33
  | `src/config-edit.ts` | Surgical line-oriented YAML patch for `modelRoles` + `retry.fallbackChains` + `task.disabledAgents`, plus the atomic writer. Exports `parseConfig`, `patchConfig`, `writeConfigAtomic`, `ConfigEditError`, and the `ConfigPatch` type. No runtime YAML dependency. |
34
34
  | `src/state.ts` | State/history/lock files under the agent dir, and agent-dir resolution. Exports `agentDir`, `loadState`, `saveState`, `appendHistory`, `acquireLock`, `releaseLock`, `freshState`, and the `PluginState` type (which carries `managedDisabledAgents` — the `task.disabledAgents` names the plugin added). |
35
35
  | `src/usage-ledger.ts` | The per-role usage ledger (spec #81): an append-only jsonl file (`llm-role-usage.jsonl`, cap `USAGE_LEDGER_CAP = 2000` rows) beside the plugin's other state, one row per assistant turn. Exports `UsageRow`, `TokenProfile`, `USAGE_LEDGER_FILE`, `USAGE_LEDGER_CAP`, `appendUsageRow` (best-effort, never throws), `readUsageRows` (tolerant: missing/malformed → `[]`), `roleTokenProfile` (the role's median per-turn input/output counts via `agent-create.ts`'s `quantile` — the repo's one quantile convention — or `null`), and `resolveTokenProfile` (recorded usage → the `tokenProfile` setting → `null`). The caller owns the dir; the extension records into `resolveProjectDir() ?? agentDir()`, the explorer reads the active scope's dir. |
36
36
  | `src/error-report.ts` | Opt-in reporting of **unexpected** plugin errors as GitHub issues on `Gott50/omp-llm-role` (the `errorReporting` setting). The plugin's only GitHub client and the only home for a fingerprint, a redaction rule, the agent-dir ledger (`ERROR_REPORT_LEDGER_FILE`) and the issue body. **Entry point:** `reportUnexpectedError(input, deps)` — the feature's one test seam (per-call input + fully injected `deps`; tests supply a fake `fetch`, clock, env, dir and token resolver). Also exports `defaultErrorReportDeps()` (production deps: resolved settings, global `fetch`, real clock, `agentDir`, the shipped manifest, `gh auth token`, the OS opener) and `resetErrorReportGuard()` (the in-process dedupe guard's test hook). Never throws, never rejects; a no-op when `errorReporting` is `off`. |
37
- | `src/updater.ts` | Orchestration: rank → tier gate → probe → hysteresis → chains → agent-disable sync → config write. Exports `runUpdater`, `resolveProjectDir` (the project scope's `<cwd>/.omp` when it carries a non-empty `omp-llm-role` settings entry, else `null` — the same test the explorer's scope resolution and the extension's usage-ledger dir use), the `Deps`/`Decision`/`RunResult`/`Trigger`/`DecisionReason` types. Resolves the project scope (`<cwd>/.omp` when it carries an `omp-llm-role` settings entry) and scopes the config/state/history/lock to it. `agentDisablePatch` also removes a previously-managed `task.disabledAgents` name whose agent file is gone. `RunResult.defect` is set on the exception-abort paths only (a `ConfigEditError`/self-check mismatch and the catch-all throw); the enumerated environment aborts leave it unset. |
37
+ | `src/updater.ts` | Orchestration: rank → tier gate → model probe → hysteresis → auto-pin route probe → chains → agent-disable sync → config write. Exports `runUpdater`, `resolveProjectDir` (the project scope's `<cwd>/.omp` when it carries a non-empty `omp-llm-role` settings entry, else `null` — the same test the explorer's scope resolution and the extension's usage-ledger dir use), the `Deps`/`Decision`/`RunResult`/`Trigger`/`DecisionReason` types. Resolves the project scope (`<cwd>/.omp` when it carries an `omp-llm-role` settings entry) and scopes the config/state/history/lock to it. `agentDisablePatch` also removes a previously-managed `task.disabledAgents` name whose agent file is gone. `RunResult.defect` is set on the exception-abort paths only (a `ConfigEditError`/self-check mismatch and the catch-all throw); the enumerated environment aborts leave it unset. |
38
38
  | `src/extension.ts` | omp extension entry (default export). Registers `session_start` (awaited day-gated run), `/refresh-roles`, `/explore-roles` (in-process explorer), `/create-agent` (architect + benchmark-link resolution + catalog discovery), `/remove-agent`, `/project-roles` (profile architect + `setupProject` + in-process updater), and `session_shutdown`. Also subscribes to `session_start`, `retry_fallback_applied`, `retry_fallback_succeeded` and `turn_start` to keep the live model's display name suffixed with its provider pin (`src/provider-display.ts`); each handler mutates the live model's `name` in place and never calls `pi.setModel` (which would wipe omp's in-flight fallback state and clobber the thinking level). Owns `extDeps` (the extension's `Deps`) and the live-session model hook. Records the per-role usage ledger (spec #81) on `turn_end` and `session_shutdown` — one row per assistant turn read from `sessionManager.getBranch()`, into `resolveProjectDir() ?? agentDir()`, deduped by `ts|model` so the two hooks cannot double-record. Every unexpected command-handler throw and every `runUpdater` defect is routed through `src/error-report.ts`: `reportCommandError` for the six command-handler catch sites and `reportDefect` for a `runUpdater` result whose `defect` is set — each keeps the pre-existing one-line warning and adds the reporting module's own lines. |
39
39
  | `src/guards.ts` | The package's one type guard: `isRecord`. |
40
40
  | `src/provider-display.ts` | The provider display (spec #77): keep the live session model's `name` suffixed with its upstream provider pin, so omp's model segment renders e.g. `DeepSeek V4.1 Flash @coreweave/fp8`. Pure and dual-runtime (no SDK import). Exports `providerPinOf` (the single `compat.openRouterRouting.only[0]`, else `compat.vercelGatewayRouting.only[0]`; null unless exactly one non-empty string), `providerPinOfSelector` (the trailing `@<slug>` of a selector, after stripping a thinking suffix), `withProviderSuffix` (strip any previous ` @<slug>` then append, idempotent), and `applyProviderDisplay` (mutate a record's `name`; no-op otherwise). |
@@ -145,20 +145,39 @@ The omp extension injects the production `Deps`; tests inject fakes.
145
145
  6. **Hysteresis.** `no-current` → adopt; ineligible current → `adopted`; best
146
146
  beats current by `switchMargin` → `switched`; inside the margin but
147
147
  `cheaperInsideMargin` → `switched-cost`; else `kept-margin` / `kept-eligible`.
148
- 7. **Suffix.** Append `roles.<role>.thinking` only when the chosen catalog row's
148
+ 7. **Route pin (auto-pin).** With the `providerPinning` capability on and no
149
+ manual `providerPin`, the chosen model's `rankedRoutes` order is walked and
150
+ each candidate is probed with `probeRoute` (a one-token completion bound to
151
+ the slug; optional seam `deps.probeRoute?`, DI like `probeModel`); the first
152
+ route that is not `blocked` becomes the `@<slug>` pin. `unknown` counts as
153
+ usable; a model whose every candidate is rejected (or whose probe budget ran
154
+ out) drops the pin — the selector stays bare and `Decision.pinDropped` is set.
155
+ Verdicts are cached per run by `catalogId@slug`; the walk spends at most
156
+ `ROUTE_PROBE_BUDGET = 12` new probes per role. It never runs for a manual pin,
157
+ a locked or disabled role, a non-OpenRouter selector or a model with no
158
+ candidate route. Contract:
159
+ [`writes-and-state.md`](writes-and-state.md), *The provider-pin
160
+ auto-configuration*.
161
+ 8. **Suffix.** Append `roles.<role>.thinking` only when the chosen catalog row's
149
162
  `thinking[]` includes the level (meta levels `off`/`auto` need only a
150
163
  non-empty list).
151
- 8. **Chains.** For every managed role, `chainUpserts` (bare key → next
152
- `fallbackChainDepth` probe-clean candidates, suffixed per target) and
153
- `chainPrunes` (plugin-written keys no longer referenced). A key claimed by
154
- more than one role stays level-free.
155
- 9. **Config patch.** `writePatch` acquires the refresh lock, reads `config.yml`,
156
- `patchConfig` (surgical, self-checked), `writeConfigAtomic` (mtime-guarded
157
- tmp+rename, 3 conflict retries). Zero changes → no write, no mtime change.
158
- 10. **State + history.** `saveState` (day gate, managed roles, last selectors,
164
+ 9. **Chains.** For every managed role, `chainUpserts` (bare key → the next
165
+ `fallbackChainDepth` fallback models drawn from the role's ranked candidate
166
+ list below the chosen model — the cheaper-or-equal candidates that retain at
167
+ least `fallbackQualityFloor` of its `q` first, then the pricier ones, each
168
+ bucket in value order, truncated to the depth — suffixed per target) and
169
+ `chainPrunes` (plugin-written keys no longer referenced). The list is the
170
+ ranked candidate list minus the key-blocked and probe-`blocked` candidates; an
171
+ unprobed candidate stays eligible, so the chain can reach a cheaper-or-equal
172
+ model below the probe window (writes-and-state.md, *Which models are
173
+ chained*). A key claimed by more than one role stays level-free.
174
+ 10. **Config patch.** `writePatch` acquires the refresh lock, reads `config.yml`,
175
+ `patchConfig` (surgical, self-checked), `writeConfigAtomic` (mtime-guarded
176
+ tmp+rename, 3 conflict retries). Zero changes → no write, no mtime change.
177
+ 11. **State + history.** `saveState` (day gate, managed roles, last selectors,
159
178
  plugin-written chain keys, managed `task.disabledAgents` names, previous
160
179
  `modelRoles` snapshot) and `appendHistory` (one row per completed run).
161
- 11. **Live-session coupling.** When `default` actually changed and
180
+ 12. **Live-session coupling.** When `default` actually changed and
162
181
  `activateDefaultOnEmptySession` is set, `deps.applySessionModel?.(selector,
163
182
  previous)` hands the new selector to the host (extension only).
164
183
 
@@ -221,6 +240,7 @@ type Deps = {
221
240
  getSettings?(): Promise<Record<string, unknown>>;
222
241
  getKeyMeta?(token: string): Promise<KeyMeta>;
223
242
  probeModel?(token: string, catalogId: string): Promise<ProbeVerdict>;
243
+ probeRoute?(token: string, catalogId: string, providerSlug: string): Promise<ProbeVerdict>;
224
244
  getAllowedProviders?(token: string, modelId: string): Promise<Set<string> | null>;
225
245
  // Host coupling (extension only): apply a concrete selector to the live session model.
226
246
  applySessionModel?(selector: string, previous: string | null): Promise<void>;
@@ -1,4 +1,4 @@
1
- # Current state (2026-10-18)
1
+ # Current state (2026-10-19)
2
2
 
3
3
  Dated snapshot of the ranking and the landed work as of the date in the heading —
4
4
  matched/priced/eligible counts, per-role leaders, the open defects, and the
@@ -6,12 +6,77 @@ per-issue work log below. It is refreshed by
6
6
  regenerating the report (`node src/cli/llm-role-rank.ts --out docs/llm-role-rankings.md`) and
7
7
  copying the new numbers here; release prose belongs in
8
8
  [`../../CHANGELOG.md`](../../CHANGELOG.md), not in this file. The model-level
9
- ranking numbers below are the 2026-10-18 measurement (the input-price blend basis
10
- #21, the #17/#18/#19/#20 wave, the #34–#40 focus-axis/capability-flag work, and
9
+ ranking numbers below are the 2026-10-19 measurement (regenerated at the 1.3.0
10
+ cut; byte-identical to the 2026-10-18 one apart from the report's date line —
11
+ the input-price blend basis #21, the #17/#18/#19/#20 wave, the #34–#40
12
+ focus-axis/capability-flag work, and
11
13
  spec #81's explorer surfaces — the ranking CLI itself is unchanged); the
12
14
  provider-pin numbers are the 2026-10-11 measurement (spec #71; see the top
13
15
  bullet). Neither is re-derived on read.
14
16
 
17
+ - Fallback chain quality floor (2026-10-19, spec #95): a role's fallback chain no
18
+ longer only reorders the probe window — that window is `1 + fallbackChainDepth`
19
+ models, the same size as the chain, so the reorder could never reach a cheaper
20
+ model at rank 4 or deeper. The chain's fallback **models** now come from the
21
+ role's whole ranked candidate list below the chosen model: the
22
+ **cheaper-or-equal** ones first — a candidate qualifies only if it retains at
23
+ least `fallbackQualityFloor` (a new setting, default `0.9`; `0` disables the
24
+ floor, `1` requires an exact quality match) of the chosen model's own `q`, so a
25
+ failover never silently drops the role onto a much weaker model — then the
26
+ pricier ones, each bucket in value order, truncated to `fallbackChainDepth`.
27
+ Key-blocked and probe-`blocked` candidates are excluded; an **unprobed**
28
+ candidate stays eligible, so the chain can reach a cheaper-or-equal model ranked
29
+ below the probe window (chain entries are never probe-verified — spec #90). A
30
+ candidate priced exactly at the chosen model's price whose `q` is below the
31
+ floor lands in neither bucket and is dropped. The floor gates **chain
32
+ membership only**: the primary choice, the hysteresis decision, the ranking, the
33
+ probe walk's stop condition and budget, the chain key, the route expansion, the
34
+ thinking levels and the prune-on-pin-change are all unchanged, as are the
35
+ manual-`providerPin` and no-pin paths. Measured on the 2026-10-19 caches with
36
+ the live resolved defs (`fallbackQualityFloor` 0.9, depth 2): `slow`, `vision`
37
+ and `task` gain cheaper fallbacks, the other seven roles are byte-identical, and
38
+ no role is degraded. That is the **per-role** effect: in the written config only
39
+ the `vision` key changes, because `deepseek-v4.1-flash@coreweave/fp8` is claimed
40
+ by seven roles and carries the last claimer's chain (the shared-key behaviour
41
+ spec #95 leaves out of scope). The ranking numbers are unchanged — the chain is a
42
+ write-side shape, not a ranking input — so `docs/llm-role-rankings.md` is not
43
+ regenerated and the dated numbers below stand as written; the suite grew
44
+ 652 → 666.
45
+
46
+ - Routability-verified auto-pin (2026-10-19, spec #90): the `providerPinning`
47
+ auto-pin no longer writes the score-max route blind. The daily update walks the
48
+ chosen model's `vfm` route order and probes each candidate with a one-token
49
+ completion bound to it (`provider: { only: [slug] }`), because OpenRouter's
50
+ endpoints API can list a route as healthy — non-degraded, a full
51
+ `supported_parameters` list, 100 % uptime — while its routing table excludes it:
52
+ the live config's `deepseek/deepseek-v4.1-flash@relace` passed every gate and
53
+ then 404'd on every request, including every fallback revert. Only the narrow
54
+ no-allowed-providers 404 skips a route; a timeout, a 5xx, a 429 and a 404 with
55
+ any other body are `unknown` and count as usable, so a transient blip never
56
+ unpins a role. Verdicts are cached per run by `catalogId@slug`, and the walk is
57
+ capped at `ROUTE_PROBE_BUDGET = 12` probes per role. When no candidate is
58
+ routable the pin is **dropped** — the selector stays bare and OpenRouter routes
59
+ the request itself — and the run notes name the rejected slugs
60
+ (`; route-blocked: …`), plus the dropped pin (`; pin dropped (no routable
61
+ route)`) when none survived. The fallback chain keeps its shape
62
+ (the primary contributes every gate-passing route except the pinned **slug** —
63
+ see the chain bullet below), and **chain entries are not probed**: one
64
+ unroutable entry is survivable, omp fails over. A manual
65
+ `roles.<role>.providerPin` stays a hard gate: never probed, never replaced. The
66
+ explorer's auto-pin line now reads as a **candidate** — the first route the walk
67
+ tries, not a promise of the written pin. The ranking numbers are unchanged — the
68
+ probe is a write-side gate, not a ranking input, and neither the ranking CLI nor
69
+ `docs/llm-role-rankings.md` changed; the suite grew 631 → 652.
70
+
71
+ - Release 1.3.0 (2026-10-19, cut `55416d4`, tag `v1.3.0`): the spec #81 explorer
72
+ surfaces ship. The report was regenerated at the cut and is **unchanged** from
73
+ the 2026-10-18 measurement — matched 149/408, priced 148, eligible 140
74
+ (vision 73, designer 87), the same ten leaders — so the numbers below stand as
75
+ written. The cut also carried the #89 fix: the pre-commit hook and
76
+ `tests/ci.test.ts` now scrub `GIT_*` from every child process, so the suite's
77
+ temp-repo git calls can no longer target the real repository (suite 630 →
78
+ 631).
79
+
15
80
  - Explorer as-written pool, per-task price, and price levers (2026-10-18, spec
16
81
  #81): the explorer gains a **pool toggle** (raw vs as-written), a **per-task
17
82
  price column**, and two **price-lever readouts**. The as-written pool applies
@@ -66,15 +131,21 @@ bullet). Neither is re-derived on read.
66
131
 
67
132
  - Auto-pin chain expansion (2026-10-10, issue #69): with `providerPinning` on,
68
133
  the fallback chain no longer stops at three providers per model — the primary
69
- model contributes every gate-passing route except the pinned best, then each
70
- `fallbackChainDepth` fallback model contributes every gate-passing route, in
71
- score order (the `vfm` route score — spec #71 replaced the original
72
- route-value order) and deduped by provider slug, so a down provider fails over
73
- to another provider of the same model before the chain switches models. Chain
74
- length is data-driven (`(R_primary − 1) + Σ R_fallback`) instead of the fixed
134
+ model contributes every gate-passing route **except the pinned route** (the
135
+ exclusion is by pinned slug, not by position: spec #90's probe can pin a route
136
+ other than the score leader, so the leader rides as a chain entry, and a dropped
137
+ pin excludes nothing), then each `fallbackChainDepth` fallback model contributes
138
+ every gate-passing route, in score order (the `vfm` route score — spec #71
139
+ replaced the original route-value order) and deduped by provider slug, so a down
140
+ provider fails over to another provider of the same model before the chain
141
+ switches models. Chain length is data-driven (`(R_primary − 1) + Σ R_fallback`,
142
+ or `R_primary + Σ R_fallback` with a dropped pin) instead of the fixed
75
143
  `2 + 3·fallbackChainDepth`; the manual-`providerPin` and no-pin paths are
76
- unchanged. The ranking numbers are unchanged — the chain is a write-side
77
- shape, not a ranking input.
144
+ unchanged. Which `fallbackChainDepth` **models** each chain falls back to was
145
+ the probe window until spec #95, which draws them from the role's ranked
146
+ candidate list below the chosen model with the quality floor (see the top
147
+ bullet) — the route expansion above is unchanged. The ranking numbers are
148
+ unchanged — the chain is a write-side shape, not a ranking input.
78
149
  - Opt-in error reporting (2026-10-08, issue #48): a new `errorReporting` setting
79
150
  (`off` — the default — `ask`, `auto`) turns a genuinely unexpected failure — a
80
151
  command handler throwing, or a `runUpdater` abort whose `RunResult.defect` is
@@ -191,10 +191,14 @@ Pure, no I/O; all ranking math is delegated to `src/engine.ts`.
191
191
  per-metric raw target that would close the gap, via `inverseCardinal`, with
192
192
  unreachable/extrapolated notes), `dominators` (models both cheaper and at
193
193
  least as good on `q`, top 3), `ownProvider` (true when the model was priced on
194
- the soft best route rather than the blend), `autoPin` (the provider slug the
195
- updater would write — `bestRoute(model, def)?.providerSlug`, the max-`vfm`
194
+ the soft best route rather than the blend), `autoPin` (the **candidate** route's
195
+ provider slug — `bestRoute(model, def)?.providerSlug`, the max-`vfm`
196
196
  route (scoring.md, *Route ordering*), when the role sets
197
- `preferOwnProvider` and carries no manual `providerPin`, else `null`), and the
197
+ `preferOwnProvider` and carries no manual `providerPin`, else `null` — the
198
+ first route the updater's daily walk tries, **not** a promise of the written
199
+ pin: the updater probes it and skips to the next route when OpenRouter does not
200
+ serve it, or drops the pin entirely when none is routable; writes-and-state.md,
201
+ *The provider-pin auto-configuration*), and the
198
202
  availability overlay `key`/`keyReason`
199
203
  (the same verdict as `rankRows`, plus the reason it reads that way). It mirrors
200
204
  `rankRole`'s capability fill so the contributions sum exactly to `q`.
@@ -369,8 +373,20 @@ a scraped third-party site, so the DOM is built with
369
373
  line). The cost line appends `· best route`
370
374
  when the model was priced on the soft best route (`ownProvider`), and a
371
375
  **pin line** shows the pin in force: `pin: <slug> (manual)` when the edited def
372
- carries a `providerPin`, else `auto pin: <slug>` from the explanation's
373
- `autoPin` (read-only — the manual field is the only way to set one). The
376
+ carries a `providerPin`, else `auto pin (candidate): <slug>` from the
377
+ explanation's `autoPin` (read-only — the manual field is the only way to set
378
+ one). The label says *candidate* on purpose: the value is the first route the
379
+ updater's value-ordered walk *tries*, not a promise of the written pin — the
380
+ updater probes that route and skips to the next when OpenRouter does not serve
381
+ it, or drops the pin entirely (bare selector) when none is routable, so the
382
+ written pin can be a later route or absent altogether (spec #90;
383
+ writes-and-state.md, *The provider-pin auto-configuration*). The explorer is
384
+ read-only and offline and cannot probe, so it names the candidate and says so
385
+ in the tooltip. The candidate equals the written pin when every route probes
386
+ clean; with the account's allowed-providers whitelist active, the panel and the
387
+ written pin agree only on the **as-written** pool, which applies the same prune
388
+ the updater applies before walking (in the raw pool the panel can name a route
389
+ the prune removes). The
374
390
  per-task line shows the breakdown (`<in> in × $<p_in>/M + <out> out ×
375
391
  $<p_out>/M × <factor> = $<total>`), or `— (no token profile — record usage or
376
392
  set tokenProfile)` when `perTask` is `null`. The **Price levers** section
@@ -122,6 +122,39 @@ path, plus `marketplace update` + `upgrade` to a bumped catalog version), and
122
122
  push publishes to npm over OIDC and creates the GitHub Release. The guard was
123
123
  verified with a negative control — tag `v0.0.1` against version `1.1.0` failed
124
124
  in `notes` before any publish.
125
+ - **1.3.0 published by CD 2026-10-19** (run 39222193609, tag push): the guard
126
+ and the npm floor passed, `npm publish` completed over OIDC trusted publishing
127
+ with a provenance attestation (`npm audit signatures`: 1 package has a
128
+ verified attestation), and the GitHub Release was created with the 1.3.0
129
+ CHANGELOG section as its body
130
+ (`https://github.com/Gott50/omp-llm-role/releases/tag/v1.3.0`). Verified:
131
+ `dist-tags.latest` = `1.3.0`, npm's `gitHead` = `55416d4` (the tag's commit),
132
+ and the publish step's log carries the direct-publish signature (`Publishing
133
+ to https://registry.npmjs.org/ with tag latest and default access`, `Signed
134
+ provenance statement`, `+ omp-llm-role@1.3.0`). The cut (`55416d4`, tagged
135
+ `v1.3.0`) bumped the four lockstep fields, renamed `## [Unreleased]` to
136
+ `## [1.3.0] - 2026-10-19`, and regenerated the report (408 models, matched
137
+ 149/408, priced 148 — byte-identical to the 2026-10-18 report apart from its
138
+ date line; `docs/dev/current-state.md` refreshed to the same measurement).
139
+ The cut also carried the #89 fix (the `GIT_*` scrub in the pre-commit hook and
140
+ `tests/ci.test.ts`), so the release commit went through the hook with the real
141
+ repository untouched.
142
+ - **1.3.0 install routes smoke-tested** (2026-10-19, omp 18.12.2, throwaway
143
+ `HOME`s, cwd outside the real home tree): npm
144
+ (`omp plugin install omp-llm-role`), git
145
+ (`omp plugin install github:Gott50/omp-llm-role`) and marketplace
146
+ (`omp plugin marketplace add Gott50/omp-llm-role` +
147
+ `omp plugin install omp-llm-role@gott50-plugins`) each installed 1.3.0
148
+ (`omp plugin doctor`: 4 ok / 0 warnings / 0 errors for npm and git; the
149
+ marketplace install's one warning is the expected
150
+ `package_manifest: Not created yet`, and it landed at
151
+ `…/cache/plugins/gott50-plugins___omp-llm-role___1.3.0`). Each post-install
152
+ session run (`omp -p "say ok"` with `OPENROUTER_API_KEY` from
153
+ `omp token openrouter`) wrote `modelRoles` (9 roles),
154
+ `retry.fallbackChains` and `task.disabledAgents: [designer]` into that HOME's
155
+ `config.yml`, landed `llm-role-state.json` + `llm-role-history.jsonl` +
156
+ `llm-role-usage.jsonl` (the spec #81 ledger, live for the first time), and
157
+ answered.
125
158
  - **1.2.0 published by CD 2026-10-12** (run 38299141569, tag push): the guard
126
159
  and the npm floor passed, `npm publish` completed over OIDC trusted publishing
127
160
  with a provenance attestation (`npm audit signatures`: 1 package has a
@@ -200,7 +233,7 @@ path, plus `marketplace update` + `upgrade` to a bumped catalog version), and
200
233
  - **Stray registry placeholder.** The registry also carries a `0.0.0-stage`
201
234
  version ("Temporary package placeholder for staged publishing", 334 B,
202
235
  published 2026-10-03). It is not from this tree and does not affect installs
203
- (`latest` is `1.2.0`).
204
- - **Next release: none owed.** `## [Unreleased]` is empty — the 1.2.0 cut moved
205
- its entries under `## [1.2.0] - 2026-10-12`; the number comes from the
236
+ (`latest` is `1.3.0`).
237
+ - **Next release: none owed.** `## [Unreleased]` is empty — the 1.3.0 cut moved
238
+ its entries under `## [1.3.0] - 2026-10-19`; the number comes from the
206
239
  Unreleased scope, not from the open issues.
@@ -152,9 +152,22 @@ while every other weighted metric stays the model's own.
152
152
  ### Route ordering (the value-for-money score)
153
153
 
154
154
  `rankedRoutes(model, def)` (`src/engine.ts`) is the one function that orders a
155
- model's candidate routes; `bestRoute` is `rankedRoutes[0]`, and `roleRoute` (the
156
- route `rankRole` prices the model on) and the updater's auto-pin both read it, so
157
- the priced route, the written pin and the fallback chain move together.
155
+ model's candidate routes; `bestRoute` is `rankedRoutes[0]`, and both `roleRoute`
156
+ (the route `rankRole` prices the model on) and the updater's auto-pin read that
157
+ one order. The priced route is the score-max candidate; the **written pin is the
158
+ first route in that order the route probe does not reject** (writes-and-state.md,
159
+ *The provider-pin auto-configuration*), so the two can differ — the probe is a
160
+ routability check on the write side only and never feeds the ranking. The
161
+ fallback chain follows the pin: it excludes the route the pin names (by slug, not
162
+ by position) and, with a dropped pin, excludes none. Which **models** the chain
163
+ falls back to is a separate choice (spec #95): its `fallbackChainDepth` fallback
164
+ models come from the role's **ranked candidate list** below the chosen model —
165
+ the cheaper-or-equal candidates that retain at least `fallbackQualityFloor` of
166
+ its `q` first, then the pricier ones, each bucket in value order, truncated to
167
+ the depth — **not** from the probe window, so the chain can reach a
168
+ cheaper-or-equal model ranked below the `1 + fallbackChainDepth` window the walk
169
+ probes (key-blocked and probe-`blocked` candidates are excluded; unprobed ones
170
+ stay eligible — writes-and-state.md, *The provider-pin auto-configuration*).
158
171
 
159
172
  - **Candidates.** A route is a candidate when it is non-degraded (`status === 0`),
160
173
  passes `routePassesEndpointFilters(route, def.filters)`, carries a usable billed
@@ -283,7 +296,11 @@ on the 2026-10-11 cache):
283
296
  routes that pass the role's endpoint filters and the account's
284
297
  allowed-providers whitelist (see *Whitelist-aware route selection*) — falling
285
298
  back to
286
- the default `1/price²` blend only when the model has no candidate route. It is
299
+ the default `1/price²` blend only when the model has no candidate route. The
300
+ **written** pin is the first route in that same order the route probe does not
301
+ reject (spec #90; writes-and-state.md, *The provider-pin auto-configuration*) —
302
+ the probe is a routability gate on the write side, so the priced (score-max)
303
+ route and the written pin can differ. It is
287
304
  a **per-model policy, not a scalar** — the sensible pin differs per model, so
288
305
  the bundle carries a policy field evaluated against the model's own routes at
289
306
  rank time, not a slug baked into the lock file. `bestRoute` (`src/engine.ts`)
@@ -34,17 +34,38 @@ passing.
34
34
  best; cost override (`switched-cost`); `priceSwitchFraction: 0` keeps the
35
35
  margin in charge; an inside-margin challenger short of the fraction stays
36
36
  kept; the day gate.
37
- 6. **Chain pruning** (`chain-pruning.test.ts`) — plugin-written stale keys
38
- removed; owner-written keys preserved; `writeFallbackChains: false`; a role
39
- that left the managed set is pruned; the chain prefers candidates at or below
40
- the chosen price.
37
+ 6. **Chain pruning + the quality floor** (`chain-pruning.test.ts`,
38
+ `chain-quality-floor.test.ts`) — plugin-written stale keys removed;
39
+ owner-written keys preserved; `writeFallbackChains: false`; a role that left
40
+ the managed set is pruned; the chain prefers candidates at or below the chosen
41
+ price. The floor (spec #95): a cheaper rank-4 candidate that clears the floor
42
+ becomes the first fallback where the pricier rank 2/3 do not, even though the
43
+ probe walk never reaches it; a cheap primary whose cheaper-or-equal candidates
44
+ are all below the floor keeps the next-best-by-value models;
45
+ `fallbackQualityFloor: 0` admits any cheaper-or-equal candidate and `1` only an
46
+ exactly-equal-quality one; an equal-priced sub-floor candidate is in neither
47
+ bucket and is dropped; a key-blocked cheaper candidate never enters the chain;
48
+ the chain stops at `fallbackChainDepth` in the cheaper set's value order; the
49
+ manual-pin path and the auto-pin chain shape are unchanged; an out-of-range
50
+ floor is a settings error that writes nothing; and a `kept-margin` role's chain
51
+ is built from the kept model's `q` and the candidates below it, with the
52
+ selector and the decision reason unaffected.
41
53
  7. **Chain suffixes** (`chain-suffix.test.ts`) — chain values carry the role's
42
54
  suffix, the key stays bare; an entry without thinking support gets no suffix;
43
55
  two roles on one model share a level-free chain.
44
56
  8. **Probe gate** (`probe-gate.test.ts`) — 200 → ok; the no-allowed-providers 404
45
57
  → blocked; any other failure → usable; blocked best → next adopted; blocked
46
- current → replaced; chains contain only probe-clean candidates; verdicts
47
- cached across roles; all-blocked → role untouched.
58
+ current → replaced; a probe-`blocked` candidate never enters a chain
59
+ (`fallback chains contain only probe-clean candidates`); verdicts cached across
60
+ roles; all-blocked → role untouched with a note. Spec #95 changed what the
61
+ chain's model set is built from: "blocked candidates beyond a kept current do
62
+ not count toward chain depth" was **deleted** (it encoded the removed
63
+ probe-window membership rule), "walk fills a kept role's chain to full depth
64
+ with probe-clean entries" was **renamed** to "a kept role's chain is filled to
65
+ full depth from the candidates below the kept model", and a new "the chain
66
+ reaches a cheaper candidate the walk never probes" pins that an **unprobed**
67
+ cheap candidate is chained while the walk still stops at
68
+ `1 + fallbackChainDepth` clean candidates.
48
69
  9. **Keyed catalog** (`keyed-catalog.test.ts`) — key-blocked candidates pruned
49
70
  from the walk; only candidates in neither catalog are probed; key-blocked
50
71
  never in a chain; no-filter/unavailable degrade to the probe walk.
@@ -78,7 +99,8 @@ passing.
78
99
  floor sorts a below-floor route last **without dropping it**; a route with no
79
100
  p50 uses the model's blended throughput (for both the score and the floor
80
101
  check); equal scores break by `providerSlug` then `id`; `explainModel`'s
81
- `autoPin` is the score-max route.
102
+ `autoPin` is the score-max route — the auto-pin's first **candidate**, which
103
+ the updater probes before writing it (spec #90).
82
104
  14. **Thinking price** (`thinking-price.test.ts`) — bare/off unadjusted; a level
83
105
  scales only thinking-capable models; the frontier uses the effective price;
84
106
  `resolveSettings` validates thinking and rejects the legacy `suffixes` map;
@@ -149,6 +171,10 @@ passing.
149
171
  `POST /api/pool` toggle returns the gated/pruned dataset in `as-written` mode
150
172
  and the full one in `raw`; the bootstrap payload carries `poolMode` and
151
173
  `tokenProfile`; the explain response carries `leaderFlip`/`thinkingLevels`.
174
+ The auto-pin indicator (spec #90): `explainModel`'s `autoPin` is the route the
175
+ updater's walk pins when every route probes clean, and it deliberately differs
176
+ from the written pin when the walk skips a blocked leader — the panel names
177
+ the *candidate*.
152
178
  24. **Project scope** (`project-updater.test.ts`, `project-setup.test.ts`,
153
179
  `project-roles-command.test.ts`) — project mode scopes config/state/history/
154
180
  lock to `<cwd>/.omp`; the day gate is per scope; the registry write;
@@ -208,13 +234,30 @@ passing.
208
234
  and `cut` bumping the four fields, cutting the changelog, committing and
209
235
  tagging while refusing a dirty tree, an existing tag, an empty
210
236
  `## [Unreleased]` and a HEAD that is not `origin/main`.
211
- 31. **Auto-pin** (`auto-pin.test.ts`) — the selector gains the best route's
212
- `@<slug>`; the chain carries every gate-passing route of the primary (all but
213
- the pinned best) then of each fallback model, in score order (the `vfm` route
237
+ 31. **Auto-pin** (`auto-pin.test.ts`) — the selector gains the `@<slug>` of the
238
+ first route in the role's `vfm` order that the route probe does not reject
239
+ (spec #90): a `blocked` leader pins the next route and records the skipped
240
+ slug (`routeBlockedSlugs`); `unknown` is usable and stays pinned; every route
241
+ blocked writes a bare selector and sets `pinDropped`; the notify lines carry
242
+ `; route-blocked: …` and `; pin dropped (no routable route)`; an all-blocked
243
+ pool issues at most `ROUTE_PROBE_BUDGET` probes; a verdict cached for one role
244
+ is reused by another; two blocked routes sharing a provider slug are recorded
245
+ once; and the capability off, a manual pin, a locked role, a disabled role and
246
+ a model with no gate-passing route never consult the probe (a throwing stub
247
+ proves it) and leave the config byte-identical. The chain carries every
248
+ gate-passing route of the primary except the pinned **slug** (not the pinned
249
+ position) then of each fallback model, in score order (the `vfm` route
214
250
  score desc, spec #71), deduped by
215
251
  provider slug; a degraded / over-`maxPriceUsdPerM` / endpoint-filter-failing /
216
- whitelist-blocked route is excluded; the manual pin wins and the no-pin path
217
- is bare; a re-run is idempotent and a pin change prunes the old key.
252
+ whitelist-blocked route is excluded; the manual pin wins (its chain is the
253
+ fallback models only, every value carrying the manual slug) and the no-pin
254
+ path is bare (fallback models only); a re-run is idempotent and a pin change
255
+ prunes the old key. Since spec #95 the chain's fallback **models** come from
256
+ the ranked candidate list below the chosen model with the quality floor
257
+ (item 6) rather than the probe window, so the fixtures price each route at its
258
+ model's own price plus its position — routes all priced `1..n` would collapse
259
+ every model to one effective price and the floor would drop the weaker
260
+ fallbacks at equal price.
218
261
  32. **Provider display** (`provider-display.test.ts`,
219
262
  `provider-display-wiring.test.ts`) — `providerPinOf` reads the single
220
263
  `openRouterRouting.only[0]` (else `vercelGatewayRouting`), preferring
@@ -253,6 +296,13 @@ passing.
253
296
  catalog/tier gate (`resolveVariant`) and the allowed-providers prune
254
297
  (`providerAllowed`), leaves the routes untouched on a `null` whitelist,
255
298
  yields an empty pool on `tier: "none"`, and never mutates its input.
299
+ 36. **Route probe** (`route-probe.test.ts`) — `probeRoute` sends the exact
300
+ one-token completion shape (POST, bearer, `max_tokens: 1`, `provider: { only:
301
+ [slug] }`): 200 → `ok`; the no-allowed-providers 404 → `blocked`; a 404 with
302
+ any other body, a 5xx, a 429 and a transport throw → `unknown` (usable). The
303
+ walk that consumes it (skip a `blocked` route, accept `unknown`, drop the pin
304
+ when nothing is clean, the per-role budget, the cross-role verdict cache) is
305
+ covered by the auto-pin cases above.
256
306
 
257
307
  ## Live checks
258
308
 
@@ -145,41 +145,128 @@ feeds the ranking, the reported `$/M` price, or any write. The recorded usage
145
145
  takes precedence over it (`resolveTokenProfile`), so once a role has ledger rows
146
146
  the setting is ignored for that role.
147
147
 
148
- ## The provider-pin auto-configuration (spec #58)
148
+ ## The provider-pin auto-configuration (spec #58, routability-verified by #90)
149
149
 
150
150
  When a role's **expanded** def sets `preferOwnProvider: true` (the
151
151
  `providerPinning` capability's recommended bundle) and carries **no** manual
152
- `providerPin`, the updater binds the role's requests to the chosen model's best
153
- route by writing that route's provider slug as the selector's `@<slug>` suffix:
152
+ `providerPin`, the updater binds the role's requests to one provider route by
153
+ writing that route's slug as the selector's `@<slug>` suffix:
154
154
 
155
155
  ```
156
156
  openrouter/<id>@<slug>:<level>
157
157
  ```
158
158
 
159
159
  omp's `splitUpstreamRouting` parses the trailing `@<slug>` and applies
160
- `compat.openRouterRouting = { only: [slug] }`, so the price the ranking assumed
161
- is the price the request pays. The slug is `bestRoute(chosenModel, def)`'s
162
- `providerSlug` — the max-value-for-money candidate route (see scoring.md,
163
- *Route ordering*). The pin is re-derived on every run, so it follows the ranking when the
164
- chosen model changes. A manual `roles.<role>.providerPin` **wins** over the
165
- automatic choice and stays a **hard** gate (a model with no matching route is
166
- ineligible); clearing it resumes the automatic choice.
160
+ `compat.openRouterRouting = { only: [slug] }`. The pin is the best route **the
161
+ probe accepts**, which is the route the role was priced on — the route the walk
162
+ tries first — so the price the ranking assumed is the price the request pays
163
+ *only while the walk does not skip that candidate*; a skipped candidate means the
164
+ route that replaced it pays, at its own price. The pin is re-derived on every
165
+ run, so it follows the ranking when the chosen model changes.
166
+
167
+ **The walk — the pin is probed before it is written.** The slug is **not** the
168
+ score-max route unconditionally: the updater walks the role's route order
169
+ (`rankedRoutes(chosenModel, def)` — the `vfm` order the chain also uses; see
170
+ scoring.md, *Route ordering*) and probes each candidate with `probeRoute`: a
171
+ one-token completion carrying `provider: { only: [slug] }`, the same request
172
+ skeleton and the same narrow no-allowed-providers pattern as `probeModel`. The
173
+ verdict vocabulary is `probeModel`'s:
174
+
175
+ - `blocked` — the narrow no-allowed-providers 404 — skips the route and the walk
176
+ continues to the next candidate;
177
+ - `ok` and **`unknown` are both usable**. A non-404 status (a 429, a 5xx), a 404
178
+ whose body does not carry the pattern, and a transport throw (a timeout, a
179
+ connection reset) are all `unknown`: a transient blip must never silently unpin
180
+ a role, and a rate limit is omp's runtime-fallback domain, not this plugin's.
181
+
182
+ Verdicts are cached per run by `catalogId@slug`, so a route shared by several
183
+ roles is probed once, and the walk is capped per role at **`ROUTE_PROBE_BUDGET =
184
+ 12`** *new* probes (cached verdicts are free) — a pathological pool must not turn
185
+ the daily update into a long probe walk.
186
+
187
+ **Why a real request.** OpenRouter's endpoints API and its routing table
188
+ disagree: an endpoint can list `status: 0` with a full `supported_parameters`
189
+ list and 100 % uptime, pass the whitelist gate and carry a usable billed price,
190
+ and still 404 for `provider.only` — and the exclusion is dynamic (the same
191
+ request can succeed minutes later). The routing table is visible only in the 404
192
+ body of a request that failed, so a request is the only reliable signal. This is
193
+ exactly the failure the endpoint gates cannot see (spec #90: the live config held
194
+ `deepseek/deepseek-v4.1-flash@relace`, which 404'd on every revert).
195
+
196
+ **No routable candidate → no pin.** If every candidate is `blocked` (or the
197
+ budget runs out with no clean candidate), no pin is written: the selector stays
198
+ **bare** and OpenRouter routes the request itself, instead of omp failing on an
199
+ unroutable `@<slug>`. `Decision.pinDropped` is true in that case (see *State,
200
+ history, lock*), and the run notes it.
201
+
202
+ A manual `roles.<role>.providerPin` **wins** over the automatic choice and stays
203
+ a **hard** gate (a model with no matching route is ineligible). It is **never
204
+ probed and never replaced** — the probe applies only to the route the capability
205
+ itself chooses; clearing the manual pin resumes the automatic choice.
167
206
 
168
207
  **Chain shape.** The primary's chain key is the selector minus the trailing
169
- `:level` — so it **includes** the pin (`openrouter/<id>@<slug>`). Its values, in
170
- order:
208
+ `:level` — so it **includes** the pin (`openrouter/<id>@<slug>`, or the bare
209
+ `openrouter/<id>` when the pin was dropped). Its values, in order:
171
210
 
172
- 1. the same model on **every** gate-passing route except the pinned best, in
211
+ 1. the same model on **every** gate-passing route except the pinned route, in
173
212
  score order (the `vfm` route score desc — see scoring.md, *Route ordering*),
174
213
  2. then, for each of the `fallbackChainDepth` fallback models, that model on
175
214
  **every** gate-passing route, in score order.
176
215
 
177
- Length = `(R_primary − 1) + Σ R_fallback`, where `R` is the number of distinct
178
- gate-passing provider slugs for that model — data-driven, not a fixed cap.
179
- Entries dedup by chain value, so two routes sharing a provider slug collapse to
180
- one. Every entry carries the same `@<slug>` suffix (and the role's thinking level
181
- where the target advertises it). A model with no gate-passing route contributes
182
- nothing. The gates are the engine's `rankedRoutes` set: non-degraded
216
+ **Which models are chained (spec #95).** The `fallbackChainDepth` fallback
217
+ **models** come from the role's whole **ranked candidate list** — the
218
+ tier-eligible ranking rows, in value order — not from the probe window (which is
219
+ `1 + fallbackChainDepth` models, the same size as the chain, so a reorder of it
220
+ can never reach a cheaper model at rank 4 or deeper). Two classes of candidate
221
+ are excluded, with the same predicates the probe walk uses: **key-blocked**
222
+ models (the account's key cannot use them) and models whose probe verdict is
223
+ `blocked`. An **unprobed** candidate stays — the chain's deeper entries are
224
+ never probe-verified (they are not a new class: chain entries are already
225
+ route-level unprobed, and omp fails over on an unroutable entry). `after` is
226
+ that list **below the chosen model** — the leader on a fresh pick, the kept
227
+ current candidate on a kept role (`kept-eligible` / `kept-margin`) — so the
228
+ `after` set and the floor both key off the model the selector actually names —
229
+ the previous `pool.slice(chosenIdx + 1)` keyed off the same index in the
230
+ probe-clean pool — and candidates ranked above the chosen are never chained. The
231
+ set is then split:
232
+
233
+ ```
234
+ cheaper = after where priceEff <= chosen.priceEff AND q >= chosen.q · fallbackQualityFloor
235
+ pricier = after where priceEff > chosen.priceEff (never floor-filtered)
236
+ chain = [...cheaper, ...pricier].slice(0, fallbackChainDepth)
237
+ ```
238
+
239
+ so a cheaper-or-equal fallback keeps the role's quality (the floor is a fraction
240
+ of the chosen model's **own** `q`, so it reads the same on every role's quality
241
+ scale) and a cheap-but-weak candidate set cannot starve the chain (the pricier
242
+ bucket always fills any remaining depth with the next-best by value). Both
243
+ buckets keep value order, so a cheaper candidate deep in the ranking is reached
244
+ only when fewer than `fallbackChainDepth` cheaper-or-equal candidates clear the
245
+ floor — a bounded search, not a cheapest-first sort. A candidate priced exactly
246
+ at the chosen model's price whose `q` is below the floor is in **neither** bucket
247
+ and drops out. `fallbackQualityFloor` (a number in `[0, 1]`, default `0.9`; `0`
248
+ disables the floor, `1` requires an exact quality match — validated, never
249
+ silently clamped) is documented in the README's settings table. The floor gates
250
+ **chain membership only**: the primary choice, the hysteresis decision, the
251
+ ranking, the probe walk's stop condition and budget, the chain key, the route
252
+ expansion, the thinking levels, the dedup and the prune-on-pin-change are all
253
+ unchanged.
254
+
255
+ The exclusion is **by pinned slug, not by position**: the probe can pin a route
256
+ other than the score leader (so the leader rides as a fallback entry while the
257
+ pinned route does not), and with a **dropped pin** (the auto-pin applied, but no
258
+ candidate was routable) **no** route is excluded — the primary contributes every
259
+ gate-passing route. Length = `(R_primary − 1) + Σ R_fallback`, or `R_primary + Σ
260
+ R_fallback` with a dropped pin, where `R` is the number of distinct gate-passing
261
+ provider slugs for that model — data-driven, not a fixed cap. Entries dedup by
262
+ chain value, so two routes sharing a provider slug collapse to one. Every entry
263
+ carries the same `@<slug>` suffix (and the role's thinking level where the target
264
+ advertises it).
265
+ **Chain entries are not probed** — only the primary pin is: the chain carries
266
+ many routes and one unroutable entry is survivable (omp fails over), so probing
267
+ it would multiply the request cost for no benefit (out of scope, spec #90). A
268
+ model with no gate-passing route contributes nothing. The gates are the engine's
269
+ `rankedRoutes` set: non-degraded
183
270
  (`status === 0`), the endpoint capability filters (`tools` / `minContextTokens` /
184
271
  `minOutputTokens`), a usable billed price, and the `maxPriceUsdPerM` cap
185
272
  (thinking-adjusted). The soft `filters.preferredMinThroughput` floor is a
@@ -190,16 +277,28 @@ pool to allowed providers before ranking, which is what makes `rankedRoutes`
190
277
  whitelist-aware. The probe walk target is unchanged (`1 + fallbackChainDepth` —
191
278
  it counts models, not routes).
192
279
 
280
+ **Outside the auto-pin.** The chosen model contributes nothing when the auto-pin
281
+ does not apply: with a manual `providerPin` (`autoPinApplies` is false, so the
282
+ route probe does not run either) or with the capability off, the chain carries
283
+ **one entry per `fallbackChainDepth` fallback model** —
284
+ `openrouter/<fallbackId>@<manualSlug>` with a manual pin, the bare
285
+ `openrouter/<fallbackId>` with none — each with the role's thinking level where
286
+ the target advertises it, and no route of the chosen model at all.
287
+
193
288
  **Prune on a pin change.** Because the chain key includes the pin, a pin change
194
289
  changes the key; the updater's chain prune (`state.pluginWrittenChainKeys`) drops
195
290
  the old key so a stale pinned chain does not linger.
196
291
 
197
- **No-ops.** The auto-pin writes nothing when: the capability is off
198
- (`preferOwnProvider` not `true`); a manual `providerPin` is set; the role is
199
- locked or disabled; the selector is not an OpenRouter selector; the model has no
200
- route data; no route passes the role's endpoint filters; or the best route has no
201
- usable billed price. With the capability off the written config is byte-identical
202
- to before.
292
+ **No-ops.** The auto-pin writes nothing — and **the route probe is never
293
+ consulted** — when: the capability is off (`preferOwnProvider` not `true`); a
294
+ manual `providerPin` is set; the role is locked or disabled; the selector is not
295
+ an OpenRouter selector; the model has no route data; no route passes the role's
296
+ endpoint filters; or no candidate has a usable billed price (the last three leave
297
+ `rankedRoutes` empty, so there is no candidate to probe, and `pinDropped` stays
298
+ false — the pin is simply absent). The case where the model *has* candidates but
299
+ none is routable is different: that is the **dropped pin** above — still no pin,
300
+ but recorded (`pinDropped`). With the capability off the written config is
301
+ byte-identical to before.
203
302
 
204
303
  ## State, history, lock
205
304
 
@@ -294,7 +393,13 @@ settings entry the global behavior is unchanged.
294
393
  ids the role's endpoint filters dropped (no capable standard-tier route), rendered
295
394
  `; endpoint-blocked: …` on the decision line; `pinBlocked[]` records the ranking ids
296
395
  the role's provider pin dropped (no route matching the pin), rendered
297
- `; pin-blocked: …`.
396
+ `; pin-blocked: …`. `routeBlockedSlugs[]` records the provider slugs the auto-pin's
397
+ route probe rejected on the chosen model, in walk order and deduped by slug
398
+ (two routes of one model can share a slug), rendered `; route-blocked: …`;
399
+ `pinDropped` is `true` when the auto-pin applies and the model has gate-passing
400
+ routes but no candidate survived the probe (or the `ROUTE_PROBE_BUDGET` ran
401
+ out), rendered `; pin dropped (no routable route)` — the selector is bare in
402
+ that case. Both are empty/false when no probe ran.
298
403
  - Role removal (`/remove-agent`) goes through `removeRoleSettings` in
299
404
  `src/role-settings.ts` — the same validate/backup/atomic-write path as
300
405
  `writeRoleSettings`, deleting the role's flat dotted keys and any nested entry.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "omp-llm-role",
3
- "version": "1.3.0",
3
+ "version": "1.4.0",
4
4
  "type": "module",
5
5
  "description": "omp plugin: daily model-role updater driven by llm-stats/OpenRouter rankings",
6
6
  "license": "MIT",
@@ -63,6 +63,14 @@
63
63
  "min": 0,
64
64
  "step": 1
65
65
  },
66
+ "fallbackQualityFloor": {
67
+ "type": "number",
68
+ "description": "Quality floor for a cheaper-or-equal fallback: the fraction of the chosen model's quality it must retain (0 disables)",
69
+ "default": 0.9,
70
+ "min": 0,
71
+ "max": 1,
72
+ "step": 0.001
73
+ },
66
74
  "activateDefaultOnEmptySession": {
67
75
  "type": "boolean",
68
76
  "description": "Reapply the new default selector to an empty session's active model",
@@ -89,8 +89,8 @@ export type ProbeVerdict = "ok" | "blocked" | "unknown";
89
89
  const OPENROUTER_COMPLETIONS_URL = "https://openrouter.ai/api/v1/chat/completions";
90
90
 
91
91
  /** POST a one-token completion to OpenRouter with the shared request skeleton
92
- * (Bearer auth, JSON body, 20s timeout). Shared by `probeModel` and
93
- * `fetchAllowedProviders`. */
92
+ * (Bearer auth, JSON body, 20s timeout). Shared by `probeModel`, `probeRoute`
93
+ * and `fetchAllowedProviders`. */
94
94
  function completionRequest(token: string, body: unknown, fetchImpl: typeof fetch): Promise<Response> {
95
95
  return fetchImpl(OPENROUTER_COMPLETIONS_URL, {
96
96
  method: "POST",
@@ -110,6 +110,20 @@ function errorMessage(body: unknown): string {
110
110
  * whitelist (or a data-policy toggle) leaves a model without a runnable endpoint. */
111
111
  const BLOCKED_RE = /no allowed providers/i;
112
112
 
113
+ /**
114
+ * The verdict a probe response carries, shared by `probeModel` and `probeRoute`
115
+ * so the model probe and the route probe cannot drift apart: a 2xx is `ok`, the
116
+ * narrow no-allowed-providers 404 is `blocked`, and everything else is
117
+ * `unknown` (usable) — any other status (a 429, a 5xx), a 404 with any other
118
+ * body, an unparseable body. Never throws.
119
+ */
120
+ async function verdictOf(res: Response): Promise<ProbeVerdict> {
121
+ if (res.ok) return "ok";
122
+ if (res.status !== 404) return "unknown";
123
+ const body: unknown = await res.json().catch(() => null);
124
+ return BLOCKED_RE.test(errorMessage(body)) ? "blocked" : "unknown";
125
+ }
126
+
113
127
  /**
114
128
  * One-token completion probe: the only reliable signal for
115
129
  * account-level provider restrictions — neither `/api/v1/key` nor the catalog
@@ -121,11 +135,30 @@ const BLOCKED_RE = /no allowed providers/i;
121
135
  export async function probeModel(token: string, catalogId: string, fetchImpl: typeof fetch = fetch): Promise<ProbeVerdict> {
122
136
  try {
123
137
  const res = await completionRequest(token, { model: catalogId, messages: [{ role: "user", content: "ok" }], max_tokens: 1 }, fetchImpl);
124
- if (res.ok) return "ok";
125
- if (res.status !== 404) return "unknown";
126
- const body: unknown = await res.json().catch(() => null);
127
- const message = errorMessage(body);
128
- return BLOCKED_RE.test(message) ? "blocked" : "unknown";
138
+ return await verdictOf(res);
139
+ } catch {
140
+ return "unknown";
141
+ }
142
+ }
143
+
144
+ /**
145
+ * One-token completion bound to a single provider route (`provider.only`), the
146
+ * same request skeleton as `probeModel` plus the route pin. The endpoints API
147
+ * and OpenRouter's routing table disagree — an endpoint can list `status: 0`
148
+ * with 100% uptime and still 404 — so a real request is the only reliable
149
+ * signal for whether a route is routable. Only the narrow no-allowed-providers
150
+ * 404 blocks a route; a non-404 status, a 404 with any other body and a
151
+ * transport throw are all "unknown" and count as usable, exactly as
152
+ * `probeModel` does. Never throws.
153
+ */
154
+ export async function probeRoute(token: string, catalogId: string, providerSlug: string, fetchImpl: typeof fetch = fetch): Promise<ProbeVerdict> {
155
+ try {
156
+ const res = await completionRequest(
157
+ token,
158
+ { model: catalogId, messages: [{ role: "user", content: "ok" }], max_tokens: 1, provider: { only: [providerSlug] } },
159
+ fetchImpl,
160
+ );
161
+ return await verdictOf(res);
129
162
  } catch {
130
163
  return "unknown";
131
164
  }
@@ -347,9 +347,11 @@ export type Explanation =
347
347
  * `preferOwnProvider` preference, no hard pin) rather than the 1/price²
348
348
  * blend — the panel shows WHY the price is what it is. */
349
349
  ownProvider: boolean;
350
- /** the automatic pin: the model's best route's provider slug when the role
351
- * prefers a bound route and carries no manual pin, else null. Read-only in
352
- * the UI; a manual `providerPin` overrides it. */
350
+ /** the automatic pin's candidate: the model's best route's provider slug —
351
+ * the first route the updater's walk tries — when the role prefers a bound
352
+ * route and carries no manual pin, else null. The probe can skip to a later
353
+ * route or drop the pin, so this is the candidate, not the written pin.
354
+ * Read-only in the UI; a manual `providerPin` overrides it. */
353
355
  autoPin: string | null;
354
356
  role: { name: string; lambda: number; derivedLambda: number; wPrice: number; qW: number; thinking: SuffixLevel | undefined; cacheHitRate: number };
355
357
  contributions: Contribution[];
@@ -389,9 +391,12 @@ export function explainModel(def: RoleDef, models: Model[], modelId: string, rol
389
391
  const route = roleRoute(model, def);
390
392
  // The model is priced on the soft best route (not a hard pin, not the blend).
391
393
  const ownProvider = pinRoute === null && route !== null;
392
- // The automatic pin the updater would write: the model's best route's slug,
393
- // only when the role prefers a bound route and carries no manual pin. Reuses the
394
- // already-computed `route` (the same best route) instead of recomputing it.
394
+ // The automatic pin's CANDIDATE route: the model's best route's slug — the
395
+ // first route the updater's walk tries — only when the role prefers a bound
396
+ // route and carries no manual pin. The updater probes that candidate and can
397
+ // pin a later route (or drop the pin entirely), so this is what the walk tries
398
+ // first, not a promise of the written pin. Reuses the already-computed `route`
399
+ // (the same best route) instead of recomputing it.
395
400
  const autoPin = autoPinApplies(def) ? route?.providerSlug ?? null : null;
396
401
  const missingRequired = def.required.filter((k) => roleMetricValue(eff, k, pinRoute, cacheHitRate) == null);
397
402
  if (missingRequired.length > 0) reasons.push(`missing required metric(s): ${missingRequired.join(", ")}`);
package/src/settings.ts CHANGED
@@ -235,6 +235,13 @@ export type ResolvedSettings = {
235
235
  priceSwitchFraction: number;
236
236
  writeFallbackChains: boolean;
237
237
  fallbackChainDepth: number;
238
+ /**
239
+ * Quality floor for the fallback chain's cheaper-or-equal candidates: a
240
+ * candidate priced at or below the chosen model qualifies only if it retains
241
+ * at least this fraction of the chosen model's `q`. 0 disables the floor (any
242
+ * cheaper-or-equal candidate qualifies); 1 requires an exact quality match.
243
+ */
244
+ fallbackQualityFloor: number;
238
245
  roles: Record<string, RoleDef>;
239
246
  activateDefaultOnEmptySession: boolean;
240
247
  /** Opt-in reporting of unexpected plugin errors as GitHub issues on the
@@ -255,6 +262,7 @@ export const DEFAULT_SETTINGS: ResolvedSettings = {
255
262
  priceSwitchFraction: 0.5,
256
263
  writeFallbackChains: true,
257
264
  fallbackChainDepth: 2,
265
+ fallbackQualityFloor: 0.9,
258
266
  roles: DEFAULT_ROLES,
259
267
  activateDefaultOnEmptySession: true,
260
268
  errorReporting: "off",
@@ -306,6 +314,14 @@ export function deriveSettingsSchema(): Record<string, PluginSettingSchema> {
306
314
  min: 0,
307
315
  step: 1,
308
316
  },
317
+ fallbackQualityFloor: {
318
+ type: "number",
319
+ description: "Quality floor for a cheaper-or-equal fallback: the fraction of the chosen model's quality it must retain (0 disables)",
320
+ default: DEFAULT_SETTINGS.fallbackQualityFloor,
321
+ min: 0,
322
+ max: 1,
323
+ step: 0.001,
324
+ },
309
325
  activateDefaultOnEmptySession: {
310
326
  type: "boolean",
311
327
  description: "Reapply the new default selector to an empty session's active model",
@@ -579,6 +595,9 @@ export function resolveSettings(raw: Record<string, unknown>): { settings: Resol
579
595
  if (!Number.isInteger(merged.fallbackChainDepth) || merged.fallbackChainDepth < 0) {
580
596
  errors.push(`fallbackChainDepth: must be an integer >= 0, got ${JSON.stringify(merged.fallbackChainDepth)}`);
581
597
  }
598
+ if (typeof merged.fallbackQualityFloor !== "number" || !Number.isFinite(merged.fallbackQualityFloor) || merged.fallbackQualityFloor < 0 || merged.fallbackQualityFloor > 1) {
599
+ errors.push(`fallbackQualityFloor: must be a number in [0, 1], got ${JSON.stringify(merged.fallbackQualityFloor)}`);
600
+ }
582
601
  if (typeof merged.activateDefaultOnEmptySession !== "boolean") {
583
602
  errors.push(
584
603
  `${ACTIVATE_DEFAULT_KEY}: must be a boolean, got ${JSON.stringify(merged.activateDefaultOnEmptySession)}`,
package/src/updater.ts CHANGED
@@ -15,8 +15,8 @@
15
15
 
16
16
  import { readFileSync, statSync } from "node:fs";
17
17
  import { dirname, join } from "node:path";
18
- import { autoPinApplies, bestRoute, computeRankings, endpointFilterDrops, loadRankData, META_LEVELS, providerPinDrops, rankedRoutes, type Ranked, type RankData, type RoleDef } from "./engine.ts";
19
- import { currentRankingId, enrichThinkingLevels, fetchAllowedProviders, fetchKeyAvailability, fetchKeyMeta, filterCatalog, probeModel, providerAllowed, resolveVariant, THINKING_LEVELS, tierGate, type CatalogEntry, type KeyAvailability, type KeyMeta, type ProbeVerdict } from "./availability.ts";
18
+ import { autoPinApplies, computeRankings, endpointFilterDrops, loadRankData, META_LEVELS, providerPinDrops, rankedRoutes, type Ranked, type RankData, type RoleDef } from "./engine.ts";
19
+ import { currentRankingId, enrichThinkingLevels, fetchAllowedProviders, fetchKeyAvailability, fetchKeyMeta, filterCatalog, probeModel, probeRoute, providerAllowed, resolveVariant, THINKING_LEVELS, tierGate, type CatalogEntry, type KeyAvailability, type KeyMeta, type ProbeVerdict } from "./availability.ts";
20
20
  import { ConfigEditError, parseConfig, patchConfig, writeConfigAtomic, type ConfigPatch } from "./config-edit.ts";
21
21
  import { PLUGIN_SETTINGS_PATH, projectLockPath, readPluginSettingsMap, resolveSettings, roleUniverse, type ResolvedSettings } from "./settings.ts";
22
22
  import { discoverAgentPins } from "./agent-pins.ts";
@@ -41,6 +41,15 @@ export type Decision = {
41
41
  /** Ranking ids dropped by the role's provider pin (no route matching the pin) —
42
42
  * recorded like a probe-blocked candidate. */
43
43
  pinBlocked: string[];
44
+ /** Provider slugs the auto-pin's route probe rejected on the chosen model, in
45
+ * walk order — rendered `; route-blocked: …`. Slugs, not ids: the sibling
46
+ * `blocked` holds catalog ids and `pinBlocked` ranking ids. Empty when no
47
+ * probe ran. */
48
+ routeBlockedSlugs: string[];
49
+ /** True when the auto-pin applies and the model has gate-passing routes but
50
+ * every candidate was rejected (or the route-probe budget ran out): the
51
+ * selector stays bare and OpenRouter routes the request itself. */
52
+ pinDropped: boolean;
44
53
  /** Availability source for this role's selection: the keyed-catalog fast path or the probe walk. */
45
54
  availabilitySource: "keyed-catalog" | "probe";
46
55
  /** Candidates pruned as key-blocked (present in the public catalog, absent from the keyed one). */
@@ -68,10 +77,18 @@ export type Trigger = "session-start" | "manual";
68
77
  /** A ranked model resolved to a concrete catalog id. */
69
78
  type Candidate = { ranked: Ranked; catalogId: string };
70
79
 
71
- /** Per managed role: its bare chain key, role suffix, provider pin, the pool it
72
- * was chosen from, and the inputs the auto-pin chain builder needs (the expanded
73
- * def for `rankedRoutes`, and whether the pin was auto-derived). */
74
- type ChainPlan = { key: string; suffix: string | undefined; pin: string; pool: Candidate[]; chosenIdx: number; def: RoleDef; autoPin: boolean };
80
+ /** Per managed role: its bare chain key, role suffix, provider pin, the
81
+ * candidates its chain's fallback models are drawn from, the chosen model's
82
+ * index in that list, and the inputs the auto-pin chain builder needs (the
83
+ * expanded def for `rankedRoutes`, and whether the pin was auto-derived).
84
+ * `candidates` is the role's ranked candidate list minus the candidates no
85
+ * chain entry may name (key-blocked, probe-blocked) — not the probe-clean pool
86
+ * — so the chain can reach a cheaper-or-equal model ranked below the probe
87
+ * window. `pinnedSlug` is the route the auto-pin wrote — undefined for a manual
88
+ * pin, for a dropped pin and with the capability off — so the chain builder can
89
+ * exclude the pinned route by slug rather than by its position in the route
90
+ * order. */
91
+ type ChainPlan = { key: string; suffix: string | undefined; pin: string; pinnedSlug: string | undefined; candidates: Candidate[]; chosenIdx: number; def: RoleDef; autoPin: boolean };
75
92
 
76
93
  export type Deps = {
77
94
  getToken(): Promise<string>;
@@ -83,6 +100,13 @@ export type Deps = {
83
100
  getSettings?(): Promise<Record<string, unknown>>;
84
101
  getKeyMeta?(token: string): Promise<KeyMeta>;
85
102
  probeModel?(token: string, catalogId: string): Promise<ProbeVerdict>;
103
+ /**
104
+ * DI seam like `probeModel`; production defaults to the real `probeRoute`.
105
+ * One-token completion bound to a single provider route (`provider.only`), the
106
+ * auto-pin's routability gate: OpenRouter's routing table can exclude a route
107
+ * its endpoints API still lists as healthy.
108
+ */
109
+ probeRoute?(token: string, catalogId: string, providerSlug: string): Promise<ProbeVerdict>;
86
110
  /** DI seam like `getKeyMeta`/`probeModel`; production defaults to `fetchKeyAvailability` (never throws). */
87
111
  getKeyAvailability?(token: string): Promise<KeyAvailability>;
88
112
  /**
@@ -106,6 +130,12 @@ const CONFLICT_RETRIES = 3;
106
130
  /** Per-role cap on allowlist probe requests (budget against pathological all-blocked pools). */
107
131
  const PROBE_BUDGET = 12;
108
132
 
133
+ /** Per-role cap on route-probe requests for the auto-pin walk (budget against a
134
+ * pathological route pool: a fully blocked model with dozens of routes must not
135
+ * turn the daily update into a long probe walk). Hitting the cap with no clean
136
+ * candidate drops the pin, exactly as an all-blocked pool does. */
137
+ export const ROUTE_PROBE_BUDGET = 12;
138
+
109
139
  function mtimeOf(path: string): number {
110
140
  try {
111
141
  return statSync(path).mtimeMs;
@@ -120,9 +150,11 @@ function decisionLine(d: Decision): string {
120
150
  const blocked = d.blocked.length > 0 ? `; blocked: ${d.blocked.join(", ")}` : "";
121
151
  const endpoint = d.endpointBlocked.length > 0 ? `; endpoint-blocked: ${d.endpointBlocked.join(", ")}` : "";
122
152
  const pin = d.pinBlocked.length > 0 ? `; pin-blocked: ${d.pinBlocked.join(", ")}` : "";
153
+ const routes = d.routeBlockedSlugs.length > 0 ? `; route-blocked: ${d.routeBlockedSlugs.join(", ")}` : "";
154
+ const dropped = d.pinDropped ? "; pin dropped (no routable route)" : "";
123
155
  return d.from === d.to
124
- ? `@${d.role}: kept ${d.to} (${d.reason}; ${scores}${blocked}${endpoint}${pin})`
125
- : `@${d.role}: ${from} -> ${d.to} (${d.reason}; ${scores}${blocked}${endpoint}${pin})`;
156
+ ? `@${d.role}: kept ${d.to} (${d.reason}; ${scores}${blocked}${endpoint}${pin}${routes}${dropped})`
157
+ : `@${d.role}: ${from} -> ${d.to} (${d.reason}; ${scores}${blocked}${endpoint}${pin}${routes}${dropped})`;
126
158
  }
127
159
 
128
160
  /**
@@ -372,6 +404,10 @@ export async function runUpdater(trigger: Trigger, deps: Deps, opts?: { force?:
372
404
  const pinUnmatched: string[] = [];
373
405
  const probe = deps.probeModel ?? probeModel;
374
406
  const probeVerdicts = new Map<string, ProbeVerdict>();
407
+ const routeProbe = deps.probeRoute ?? probeRoute;
408
+ /** Route verdicts, cached by `catalogId@slug` across roles: a route shared by
409
+ * several roles is probed once per run. */
410
+ const routeVerdicts = new Map<string, ProbeVerdict>();
375
411
  for (const [role, def] of Object.entries(settings.roles)) {
376
412
  // Locked: no ranking, no probe, no decision, no selector/chain write.
377
413
  if (lockedRoles.has(role)) continue;
@@ -483,14 +519,50 @@ export async function runUpdater(trigger: Trigger, deps: Deps, opts?: { force?:
483
519
  // `splitUpstreamRouting` parses the trailing `@<slug>` and applies
484
520
  // `compat.openRouterRouting = { only: [slug] }`.
485
521
  //
486
- // Auto-pin (spec #58): with the soft `preferOwnProvider` capability on and
487
- // no manual pin, the chosen model's best route by the role's own value
488
- // rides as that suffix, so the price the ranking assumed is the price the
489
- // request pays. A manual `providerPin` is a hard gate and always wins; a
490
- // model with no candidate route keeps default routing (bare selector).
522
+ // Auto-pin (spec #58, routability-verified by spec #90): with the soft
523
+ // `preferOwnProvider` capability on and no manual pin, the best route the
524
+ // probe accepts rides as that suffix. That candidate is the route the role
525
+ // was priced on, so the price the ranking assumed is the price the request
526
+ // pays — but only while the walk does not skip the candidate; a skipped
527
+ // candidate means the route that replaced it pays, at its own price. A
528
+ // manual `providerPin` is a hard gate and always wins; a model with no
529
+ // gate-passing route keeps default routing (bare selector).
530
+ //
531
+ // The route order is the one the fallback chain already uses
532
+ // (`rankedRoutes`). Every candidate is probed with `provider: { only:
533
+ // [slug] }`, because OpenRouter's endpoints API can list a route as
534
+ // healthy while its routing table excludes it — the failure the endpoints
535
+ // data cannot see. A `blocked` route is skipped, `unknown` counts as
536
+ // usable (a transient blip must never unpin a role), and a pool with no
537
+ // clean candidate drops the pin: the selector stays bare and OpenRouter
538
+ // routes the request itself instead of 404ing on an unroutable `@<slug>`.
491
539
  const manualPin = def.providerPin;
492
- const autoRoute = autoPinApplies(def) ? bestRoute(chosen.ranked.model, def) : null;
493
- const pinSlug = manualPin ?? autoRoute?.providerSlug;
540
+ const autoRoutes = autoPinApplies(def) ? rankedRoutes(chosen.ranked.model, def) : [];
541
+ // No gate-passing route means nothing to pin and nothing to drop: the bare
542
+ // selector is default routing, not a dropped pin.
543
+ const hasAutoRoutes = autoRoutes.length > 0;
544
+ const routeBlockedSlugs: string[] = [];
545
+ let autoSlug: string | undefined;
546
+ let routeProbesSpent = 0;
547
+ for (const r of autoRoutes) {
548
+ const key = `${chosen.catalogId}@${r.providerSlug}`;
549
+ let verdict = routeVerdicts.get(key);
550
+ if (verdict === undefined) {
551
+ if (routeProbesSpent >= ROUTE_PROBE_BUDGET) break;
552
+ verdict = await routeProbe(token, chosen.catalogId, r.providerSlug);
553
+ routeVerdicts.set(key, verdict);
554
+ routeProbesSpent++;
555
+ }
556
+ if (verdict === "blocked") {
557
+ // Deduped: two routes can share a provider slug (the chain collapses
558
+ // them too), and the decision line should not list one slug twice.
559
+ if (!routeBlockedSlugs.includes(r.providerSlug)) routeBlockedSlugs.push(r.providerSlug);
560
+ continue;
561
+ }
562
+ autoSlug = r.providerSlug;
563
+ break;
564
+ }
565
+ const pinSlug = manualPin ?? autoSlug;
494
566
  const pin = pinSlug !== undefined ? `@${pinSlug}` : "";
495
567
  const chosenRow = rowById.get(chosen.catalogId);
496
568
  // Append only a level the model's catalog thinking[] actually supports —
@@ -509,16 +581,28 @@ export async function runUpdater(trigger: Trigger, deps: Deps, opts?: { force?:
509
581
  blocked: blockedForRole,
510
582
  endpointBlocked: endpointFilterDrops(def, rank.models),
511
583
  pinBlocked: providerPinDrops(def, rank.models),
584
+ routeBlockedSlugs,
585
+ pinDropped: hasAutoRoutes && autoSlug === undefined,
512
586
  availabilitySource: availability.active ? "keyed-catalog" : "probe",
513
587
  keyBlockedCount,
514
588
  });
515
589
 
516
590
  // Fallback chain inputs for every managed role: the bare key (a chain key
517
- // matches the active model id, never a level) plus the pool the chosen model
518
- // came from. Values are built after the loop, once every key claim is known.
591
+ // matches the active model id, never a level) plus the candidate list the
592
+ // chain's fallback models are drawn from. Values are built after the loop,
593
+ // once every key claim is known.
519
594
  if (settings.writeFallbackChains) {
520
595
  const key = suffix !== undefined && finalSelector.endsWith(`:${suffix}`) ? finalSelector.slice(0, -(suffix.length + 1)) : finalSelector;
521
- chainPlanByRole[role] = { key, suffix, pin, pool, chosenIdx: pool.indexOf(chosen), def, autoPin: autoRoute !== null };
596
+ // The chain's fallback models come from the role's whole ranked
597
+ // candidate list, so a cheaper-or-equal model ranked below the probe
598
+ // window can be chained. Only the candidates the probe walk would never
599
+ // accept are dropped — key-blocked (the account's key cannot use them)
600
+ // and probe-verdict-blocked (present in the public catalog but not for
601
+ // this key). An unprobed candidate stays: the chain's deeper entries are
602
+ // no longer required to be probe-verified. The chosen model passed both
603
+ // predicates, so it is always in the list and `chosenIdx` is never -1.
604
+ const chainCandidates = candidates.filter((c) => !keyBlocked(c) && probeVerdicts.get(c.catalogId) !== "blocked");
605
+ chainPlanByRole[role] = { key, suffix, pin, pinnedSlug: autoSlug, candidates: chainCandidates, chosenIdx: chainCandidates.indexOf(chosen), def, autoPin: hasAutoRoutes };
522
606
  }
523
607
  }
524
608
 
@@ -539,13 +623,18 @@ export async function runUpdater(trigger: Trigger, deps: Deps, opts?: { force?:
539
623
  for (const d of decisions) {
540
624
  const plan = chainPlanByRole[d.role];
541
625
  const suffix = claims[plan.key] === 1 ? plan.suffix : undefined;
542
- // Cost-aware ordering: prefer fallbacks priced at or below the chosen
543
- // model (a fallback should not cost more than the primary), then fill any
544
- // remaining depth with the next-best by value.
545
- const after = plan.pool.slice(plan.chosenIdx + 1);
546
- const chosenPrice = plan.pool[plan.chosenIdx].ranked.priceEff;
547
- const cheaper = after.filter((p) => p.ranked.priceEff <= chosenPrice);
548
- const pricier = after.filter((p) => p.ranked.priceEff > chosenPrice);
626
+ // Cost-aware ordering with a quality floor: prefer fallbacks priced at or
627
+ // below the chosen model that still retain `fallbackQualityFloor` of its
628
+ // quality (a failover must not silently drop the role onto a much weaker
629
+ // model), then fill any remaining depth with the next-best by value. The
630
+ // floor and the `after` set both key off the chosen model — the leader on
631
+ // a fresh pick, the kept current candidate on a kept role — and the floor
632
+ // never filters the pricier set, so a cheap-but-weak candidate set cannot
633
+ // starve the chain.
634
+ const after = plan.candidates.slice(plan.chosenIdx + 1);
635
+ const chosen = plan.candidates[plan.chosenIdx];
636
+ const cheaper = after.filter((p) => p.ranked.priceEff <= chosen.ranked.priceEff && p.ranked.q >= chosen.ranked.q * settings.fallbackQualityFloor);
637
+ const pricier = after.filter((p) => p.ranked.priceEff > chosen.ranked.priceEff);
549
638
  const chainPool = [...cheaper, ...pricier].slice(0, settings.fallbackChainDepth);
550
639
  // A chain value's thinking level is appended only when the target's
551
640
  // catalog entry advertises support (or the level is a meta level).
@@ -556,7 +645,7 @@ export async function runUpdater(trigger: Trigger, deps: Deps, opts?: { force?:
556
645
  const values: string[] = [];
557
646
  if (plan.autoPin) {
558
647
  // Auto-pin chain (spec #58, expanded by #69): every gate-passing route
559
- // of the primary model (all but the pinned best), then every
648
+ // of the primary model except the pinned route, then every
560
649
  // gate-passing route of each fallback model, in the role's
561
650
  // value-for-money score order (`rankedRoutes`, spec #71) — every entry
562
651
  // bound to its route's provider, so a down primary
@@ -564,12 +653,17 @@ export async function runUpdater(trigger: Trigger, deps: Deps, opts?: { force?:
564
653
  // than a different model. `rankedRoutes` already applies every gate
565
654
  // (non-degraded, endpoint filters, usable billed price, the
566
655
  // maxPriceUsdPerM cap, the allowed-providers whitelist), so the chain
567
- // carries exactly the routes the role can use. The dedup below
568
- // collapses two routes that share a provider slug (same chain value),
569
- // so the chain length is (R_primary − 1) + Σ R_fallback over distinct
570
- // gate-passing provider slugs.
571
- const chosenCandidate = plan.pool[plan.chosenIdx];
572
- for (const r of rankedRoutes(chosenCandidate.ranked.model, plan.def).slice(1)) {
656
+ // carries exactly the routes the role can use. The pin is excluded by
657
+ // slug, not by position: the route probe can pin a route other than
658
+ // the score leader, and with a dropped pin no route is excluded at
659
+ // all. The dedup below collapses two routes that share a provider slug
660
+ // (same chain value), so the chain length is (R_primary − 1) + Σ
661
+ // R_fallback over distinct gate-passing provider slugs (R_primary with
662
+ // no pin). Only the pinned route is probe-verified: one unroutable
663
+ // chain entry is survivable, omp fails over.
664
+ const chosenCandidate = plan.candidates[plan.chosenIdx];
665
+ for (const r of rankedRoutes(chosenCandidate.ranked.model, plan.def)) {
666
+ if (r.providerSlug === plan.pinnedSlug) continue;
573
667
  values.push(`openrouter/${chosenCandidate.catalogId}@${r.providerSlug}${levelFor(chosenCandidate.catalogId)}`);
574
668
  }
575
669
  for (const p of chainPool) {
package/web/app.js CHANGED
@@ -185,6 +185,7 @@ const TIPS = {
185
185
  focusCrossSource: "Cross-source: the zeroeval per-entry price/context compared against the plugin's own price/context for the models carrying the metric (the median relative divergence, within a 0.5 tolerance). The speed figure is informational only (rps vs tok/s). An annotation, never a gate — it never feeds the 1/price² blend; unknown means no model joined or the payload carried none of the four fields.",
186
186
  imageFilter: "Require image input (filters.image) — the gate that shrinks the vision role's eligible set.",
187
187
  providerPin: "Hard provider pin: the OpenRouter provider slug this role's requests are routed to (a tiered slug like deepinfra/fp8 is accepted verbatim). Setting it opts the role out of the automatic best-route pin; empty = default OpenRouter routing.",
188
+ autoPin: "Automatic provider pin (candidate): this model's best route by the role's value-for-money score — the first route the updater's daily run tries. The updater probes that route with a one-token request bound to it (provider: { only: [slug] }) and skips to the next route when OpenRouter does not serve it, dropping the pin entirely (bare selector, OpenRouter's own routing) when none is routable — so the pin that gets written can be a later route, or absent.",
188
189
  features: "Opt-in capability presets (issue #40). Each flag applies the plugin's recommended settings for that capability to this role; an explicit knob value always wins. Export writes the flag, not the expanded values, so a future change to the recommendation keeps reaching the role.",
189
190
  featureState: "inherit = use the global flag (features.<id> in the lock file); on/off = a per-role override (roles.<role>.features.<id>) that beats the global in either direction.",
190
191
  featureKnob: "An individual knob the capability's recommended bundle fills. Leave it at inherit to take the recommendation; set a value to override it (an explicit value always wins over the flag).",
@@ -562,7 +563,12 @@ function renderExplain() {
562
563
  if (manualPin) {
563
564
  panel.append(el("p", { class: "sub pin-line", "data-tip": TIPS.providerPin, text: "pin: " + manualPin + " (manual)" }));
564
565
  } else if (ex.autoPin) {
565
- panel.append(el("p", { class: "sub pin-line", "data-tip": TIPS.providerPin, text: "auto pin: " + ex.autoPin }));
566
+ // The candidate route — the first route the updater's value-ordered walk
567
+ // tries — not a promise of the written pin: the updater probes it and skips
568
+ // to the next route when OpenRouter does not serve it, or drops the pin
569
+ // entirely when none is routable (spec #90). Read-only, offline: the
570
+ // explorer cannot probe, so it names the candidate and says so in the tip.
571
+ panel.append(el("p", { class: "sub pin-line", "data-tip": TIPS.autoPin, text: "auto pin (candidate): " + ex.autoPin }));
566
572
  }
567
573
 
568
574
  panel.append(el("h3", { "data-tip": TIPS.composition, text: "Value composition" }));