omp-llm-role 1.2.0 → 1.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -63,6 +63,13 @@ omp plugin install omp-llm-role@gott50-plugins
63
63
  unknown keys stay byte-identical, and a no-change run writes nothing.
64
64
  `~/.omp/agent/llm-role-state.json` snapshots the previous `modelRoles` block
65
65
  on every write as a rollback aid.
66
+ - **Usage ledger.** The plugin appends one row per assistant turn (the model,
67
+ the role, the provider, the token counts and the cost) to
68
+ `llm-role-usage.jsonl` beside its other state — `<project>/.omp` in project
69
+ mode, `~/.omp/agent` otherwise. It is append-only, capped at the newest 2000
70
+ rows, and best-effort (a recording failure never breaks a session). The
71
+ explorer's `$/task` column reads it to learn your own token mix; nothing else
72
+ uses it.
66
73
 
67
74
  ## Roles
68
75
 
@@ -119,7 +126,9 @@ Global knobs:
119
126
  | `priceSwitchFraction` | `0.5` | Inside the margin, adopt a challenger at least this fraction cheaper (0.5 = half the $/M); `0` disables |
120
127
  | `writeFallbackChains` | `true` | Write per-role fallback chains into `config.yml` |
121
128
  | `fallbackChainDepth` | `2` | Fallback models per role chain |
129
+ | `fallbackQualityFloor` | `0.9` | A cheaper-or-equal fallback must retain at least this fraction of the chosen model's quality; `0` disables |
122
130
  | `activateDefaultOnEmptySession` | `true` | Reapply the new `default` selector to an empty session's active model |
131
+ | `tokenProfile` | unset | `{ inputTokens, outputTokens }` per task for the explorer's `$/task` column, used when a role has no recorded usage. Display-only; the recorded usage takes precedence. Unset = the column reads `—` |
123
132
  | `errorReporting` | `off` | Opt-in reporting of **unexpected** plugin errors as GitHub issues on `Gott50/omp-llm-role`; one of `off`, `ask`, `auto` (see below) |
124
133
 
125
134
  Per-role knobs (`roles.<name>.*`):
@@ -152,7 +161,7 @@ defaults.
152
161
  |---|---|---|
153
162
  | `endpointCeilings` | `filters.tools: true`, `filters.minOutputTokens: 16384` | Routes that cannot call tools and routes whose output ceiling is below 16 384 tokens leave the pool — a reasoning model is not cut off mid-thought |
154
163
  | `cachePricing` | `cacheHitRate: 0.5` | The role is priced on a cache-heavy agent loop's invoice (the endpoint's `cacheReadPrice` blended in) rather than the sticker price |
155
- | `providerPinning` | `preferOwnProvider: true`, `filters.preferredMinThroughput: 50` | Each request is bound to the model's **best provider route** — the max-value-for-money route (the `vfm` score, with a soft 50 tok/s floor that deprioritizes a pathologically slow route without dropping it) — the selector gains `@<slug>` — and the fallback chain carries the primary model on every gate-passing route except the pinned best, then each fallback model on every gate-passing route, so a down provider fails over to another provider of the same model before the chain switches models. The model is priced **and measured** on that route (its billed price and p50 throughput, so `q` moves too), falling back to the default blend only when it has no candidate route |
164
+ | `providerPinning` | `preferOwnProvider: true`, `filters.preferredMinThroughput: 50` | Each request is bound to a provider route **that OpenRouter actually serves**: the updater walks the model's value-for-money route order (the `vfm` score, with a soft 50 tok/s floor that deprioritizes a pathologically slow route without dropping it) and **probes each candidate with a one-token request bound to it** (`provider: { only: [slug] }`), pinning the first route the request does not reject and **dropping the pin** (bare selector, OpenRouter's own routing) when no candidate is routable — so a role never 404s on a route the endpoints data still called healthy. The selector gains `@<slug>`, and the fallback chain carries the primary model on every gate-passing route except the pinned one, then each of the next `fallbackChainDepth` fallback models on every gate-passing route — each drawn from the role's own ranking below the primary, the cheaper-or-equal ones that keep at least `fallbackQualityFloor` of its quality first, then the pricier — so a down provider fails over to another provider of the same model before the chain switches models, and a failover never silently multiplies the spend or drops the role onto a much weaker model. The model is priced **and measured** on its score-max route (its billed price and p50 throughput, so `q` moves too), falling back to the default blend only when it has no candidate route |
156
165
  | `costCap` | `filters.maxPriceUsdPerM: 10` | The priciest tail of the eligible pool leaves it, so the pick cannot be an expensive outlier |
157
166
 
158
167
  Precedence is one rule:
@@ -421,21 +430,31 @@ editing source and re-running the CLI.
421
430
  that scope's lock file — a project scope resolves as the project lock merged
422
431
  over the user-level lock, exactly what the updater ranks. A project whose lock
423
432
  file is gone is shown disabled. Switching with unsaved edits asks first.
433
+ - **Pool toggle** — the header switches the ranked pool between **raw** (the
434
+ account-agnostic fetch) and **as-written** (the pool the updater actually
435
+ writes from: the catalog/tier gate plus your account's allowed-providers
436
+ whitelist). The as-written pool is what you will get, so its leader is the
437
+ model the next `/refresh-roles` will pick; the raw pool shows the
438
+ account-agnostic ranking. The toggle is best-effort (a failed whitelist fetch
439
+ degrades to the raw pool) and per-scope, and it never affects **Export** —
440
+ Export always writes the role you authored, from either pool.
424
441
  - **Role tabs** — every role the plugin knows: the nine omp built-in roles, the
425
442
  shipped opt-in `designer`, and any role present only in the lock file. Each
426
443
  tab is tinted by provenance — `default` (omp built-in), `plugin` (shipped but
427
444
  opt-in), `user` (lock-file only) — and a disabled role is muted and dashed.
428
445
  Selecting a disabled role still ranks its models.
429
446
  - **Rank table** — every eligible model for the selected role, with `value`,
430
- `q`, `$/M` (effective, thinking-adjusted), `tok/s`, `ctx`, a `★` Pareto
447
+ `q`, `$/M` (effective, thinking-adjusted), **`$/task`** (what you actually pay
448
+ per task under your own token mix — see below), `tok/s`, `ctx`, a `★` Pareto
431
449
  marker, and a `Δ` column showing the rank delta against the role's effective
432
450
  def (what the plugin does today).
433
451
  - **Explain panel** — click a row for the full decomposition: each weighted
434
452
  metric's raw value → cardinal transform → renormalized weight → contribution,
435
453
  the cost block (`λ`, effective vs billed price, `penalty = λ·price`,
436
- `value = q − penalty`), "why not higher" (the value gap to the model above
437
- plus the per-metric target that would close it), and the models that dominate
438
- it on (price, q).
454
+ `value = q − penalty`), the **per-task price** breakdown, "why not higher"
455
+ (the value gap to the model above plus the per-metric target that would close
456
+ it), the models that dominate it on (price, q), and the **Price levers**
457
+ section (below).
439
458
  - **Weight editor** — edit the role's description, weights, `required` set,
440
459
  `filters.image`, `thinking` level, and `λ` override live; the table re-ranks
441
460
  on every change. Weights are edited freely (no implicit rescaling): the `Σ`
@@ -457,8 +476,30 @@ editing source and re-running the CLI.
457
476
  - **Hover explanations** — every column header, metric name, role tab, and
458
477
  control carries a tooltip explaining its semantics.
459
478
 
479
+ **The per-task price (`$/task`).** The `$/M` column is the reported price — the
480
+ billed 3:1 input:output blend OpenRouter charges, scaled by the role's thinking
481
+ factor. The `$/task` column is what *you* pay per task: your own input:output
482
+ token mix applied to the role's priced route. The mix comes from the plugin's
483
+ own recorded usage (the role's median per-turn counts, from the
484
+ `llm-role-usage.jsonl` ledger the plugin appends to as you work), falling back
485
+ to the `tokenProfile` setting when a role has no recorded usage. It is
486
+ **display-only** — the ranking still sorts on `$/M` — and it reads `—` when no
487
+ profile is available (no recorded usage and no `tokenProfile` set) or the model
488
+ has no usable route price. A model that is cheaper per token but
489
+ more verbose can cost more per task; this column is where that shows up.
490
+
491
+ **The price levers.** The explain panel's **Price levers** section makes the
492
+ price posture visible. The **leader flip** is the price weight `w*` at which a
493
+ cheaper challenger overtakes the current leader, with the challenger named and
494
+ the distance from your current weight — so you can see whether your price
495
+ posture binds (a small bump flips the pick) or is slack. It is absent when no
496
+ cheaper challenger exists. The **thinking level** table shows which model wins
497
+ at each level, with the level's price factor — the factor multiplies the price
498
+ axis, so it penalizes expensive models more, which is why `auto` favours the
499
+ cheap model. Both readouts are computed on the pool you are viewing.
500
+
460
501
  The UI never reimplements the ranking math: the numbers on screen are exactly
461
- the numbers the plugin would use. The server binds `127.0.0.1` only (no auth)
502
+ the numbers the plugin would use. The server binds `[IP_ADDRESS]` only (no auth)
462
503
  and serves the SPA with no build step and no external requests.
463
504
 
464
505
  ## Availability
@@ -25,16 +25,17 @@ the extension's `/refresh-roles`, `/explore-roles` and `/create-agent`.
25
25
 
26
26
  | File | Role / key exports |
27
27
  |---|---|
28
- | `src/engine.ts` | Ranking engine shared by CLI and plugin. Fetch/cache chain per source, cardinal transforms, value scoring. Exports `loadRankData`, `computeRankings`, `buildModels`, `rankRole`, `paretoFrontier`, `cardinalMetric`, `roleLambda`, `thinkingPriceFactor`, `CAPABILITY_FILL`, `SUFFIX_LEVELS`, `META_LEVELS`, `THINKING_TOKEN_OVERHEAD`, `parseFindData`, `parseModelPage`, `narrowEndpointRecord`, `buildOpenRouterEnrichment`, `applyOpenRouterData`, `hasEndpointFilters`, `routePassesEndpointFilters`, `modelPassesEndpointFilters`, `endpointFilterDrops`, `applyDesignPercentiles`, `applyWritingScores`, `extractFlight`, `extractJsonArray`, the cache readers/writers, and the `Model`/`RoleDef`/`Ranked`/`RankData`/`OpenRouterEndpointRecord` types. Re-exports `normalizeDesignId` and `parseWritingEvidence` from the benchmark-source registry. |
28
+ | `src/engine.ts` | Ranking engine shared by CLI and plugin. Fetch/cache chain per source, cardinal transforms, value scoring. Exports `loadRankData`, `computeRankings`, `buildModels`, `rankRole`, `paretoFrontier`, `cardinalMetric`, `roleLambda`, `thinkingPriceFactor`, `CAPABILITY_FILL`, `SUFFIX_LEVELS`, `META_LEVELS`, `THINKING_TOKEN_OVERHEAD`, `parseFindData`, `parseModelPage`, `narrowEndpointRecord`, `buildOpenRouterEnrichment`, `applyOpenRouterData`, `hasEndpointFilters`, `routePassesEndpointFilters`, `modelPassesEndpointFilters`, `endpointFilterDrops`, the route-price accessors (`routeInputPrice`, `routeEffectivePrice`, `routeOutputPrice`, `blendOutputPrice`), `applyDesignPercentiles`, `applyWritingScores`, `extractFlight`, `extractJsonArray`, the cache readers/writers, and the `Model`/`RoleDef`/`Ranked`/`RankData`/`OpenRouterEndpointRecord` types. Re-exports `normalizeDesignId` and `parseWritingEvidence` from the benchmark-source registry. |
29
29
  | `src/benchmark-sources.ts` | The benchmark-source registry: the single source of truth for which sources exist, what metric each feeds, and how to fetch/parse/join each. Exports `BENCHMARK_SOURCES` (the shipped static sources), `resolveBenchmarkSource`, `declaredSourceForLink`, `sourceForMetric`, `parseBenchmarkPayload`, `parseBenchmarkPayloadMeta`, `loadBenchmarkScores`, `applyBenchmarkScores`, `joinBenchmarkScores`, `normalizeMetricKey`, `externalMetricKey`, `normalizeDesignId`, `parseWritingEvidence`, `parseLlmStatsBenchmark`, `parseBenchmarkCatalog`, `loadBenchmarkCatalog`, `cachedSourceInfo`, `catalogMetric`, `SHIPPED_CATALOG_METRICS`, the declarative-source surface (`SourceDeclaration`, `validateDeclaration`, `parseSourceDeclaration`, `loadDeclaredSources`, `saveDeclaredSource`, `declaredSourcesPath`, `declarationToSource`, `executeDeclaration`, `llmStatsBenchmarkDeclaration`, `genericBenchmarkSource`, `dryRunDeclaration`), the fetch helpers (`fetchJson`, `fetchText`), and the `BenchmarkSource`/`JoinRule`/`BenchmarkScores`/`BenchmarkPayloadMeta`/`FocusSourceInfo`/`BenchmarkCatalogEntry`/`DeclarationDryRun` types. Imports `engine.ts` for types only, so there is no runtime cycle. |
30
- | `src/settings.ts` | Shipped defaults and the settings merge/validate path. Exports `DEFAULT_ROLES`, `DEFAULT_SETTINGS`, `SHIPPED_AGENTS`, `KNOWN_METRICS`, `isKnownMetric` (a shipped key or an external `<ns>:<local>`), `PLUGIN_SETTINGS_PATH`, `ACTIVATE_DEFAULT_KEY`, `deriveSettingsSchema`, `readPluginSettingsMap`, `resolveSettings`, `roleUniverse`, `deepMergeInto`, `findProjectAnchor`, `projectLockPath`, and the `ResolvedSettings`/`PluginSettingSchema`/`UniverseEntry`/`RoleKind` types. `readPluginSettingsMap` merges the project lock file (`projectLockPath(cwd)` = `<cwd>/.omp/plugins/omp-plugins.lock.json`, omp's project dir with **no walk-up**) over the user-level one; `findProjectAnchor` (the walk-up `.omp`/`.git` anchor) is kept only for the `/project-roles` divergence warning. |
30
+ | `src/settings.ts` | Shipped defaults and the settings merge/validate path. Exports `DEFAULT_ROLES`, `DEFAULT_SETTINGS`, `SHIPPED_AGENTS`, `KNOWN_METRICS`, `isKnownMetric` (a shipped key or an external `<ns>:<local>`), `PLUGIN_SETTINGS_PATH`, `ACTIVATE_DEFAULT_KEY`, `deriveSettingsSchema`, `readPluginSettingsMap`, `resolveSettings`, `roleUniverse`, `deepMergeInto`, `findProjectAnchor`, `projectLockPath`, and the `ResolvedSettings`/`PluginSettingSchema`/`UniverseEntry`/`RoleKind` types. `ResolvedSettings` carries `tokenProfile` (`{ inputTokens, outputTokens } | null`, default `null`) — the user-set per-task token profile the explorer's per-task price falls back to when a role has no recorded usage; validated to finite numbers ≥ 0, schema rows `tokenProfile.inputTokens`/`tokenProfile.outputTokens`. `readPluginSettingsMap` merges the project lock file (`projectLockPath(cwd)` = `<cwd>/.omp/plugins/omp-plugins.lock.json`, omp's project dir with **no walk-up**) over the user-level one; `findProjectAnchor` (the walk-up `.omp`/`.git` anchor) is kept only for the `/project-roles` divergence warning. |
31
31
  | `src/agent-pins.ts` | Agent → pinned-role derivation. Exports `parseAgentPin` (first `@<role>` in the `model:` frontmatter) and `discoverAgentPins` (scans the shipped, user and project agent dirs; project > user > plugin). Drives the `task.disabledAgents` sync. |
32
- | `src/availability.ts` | Key tier gate, catalog filter, variant resolution, provider-allowlist probe, the keyed-catalog availability primitive, and the allowed-providers whitelist harvest. The keyed source is the Bearer-authenticated `GET /api/v1/models` (the account setting "Filter the model catalog for API keys" makes it a per-key allowlist; set membership is the only signal), shared by the explorer and the updater's companion fast path. Exports `fetchKeyMeta`, `tierGate`, `filterCatalog`, `rankingIdOf`, `currentRankingId`, `resolveVariant`, `enrichThinkingLevels`, `probeModel`, `fetchAllowedProviders`, `providerAllowed`, `computeKeyAvailability`, `fetchKeyAvailability`, `catalogFromOmpModelsJson`, `THINKING_LEVELS`, and the `CatalogEntry`/`KeyMeta`/`Tier`/`ProbeVerdict`/`KeyAvailability` types. |
32
+ | `src/availability.ts` | Key tier gate, catalog filter, variant resolution, the model probe, the auto-pin's route probe, the keyed-catalog availability primitive, and the allowed-providers whitelist harvest. The keyed source is the Bearer-authenticated `GET /api/v1/models` (the account setting "Filter the model catalog for API keys" makes it a per-key allowlist; set membership is the only signal), shared by the explorer and the updater's companion fast path. Exports `fetchKeyMeta`, `tierGate`, `filterCatalog`, `rankingIdOf`, `currentRankingId`, `resolveVariant`, `enrichThinkingLevels`, `probeModel`, `probeRoute` (the same one-token completion plus `provider: { only: [slug] }` — the auto-pin's routability gate; spec #90), `fetchAllowedProviders`, `providerAllowed`, `computeKeyAvailability`, `fetchKeyAvailability`, `catalogFromOmpModelsJson`, `THINKING_LEVELS`, and the `CatalogEntry`/`KeyMeta`/`Tier`/`ProbeVerdict`/`KeyAvailability` types. |
33
33
  | `src/config-edit.ts` | Surgical line-oriented YAML patch for `modelRoles` + `retry.fallbackChains` + `task.disabledAgents`, plus the atomic writer. Exports `parseConfig`, `patchConfig`, `writeConfigAtomic`, `ConfigEditError`, and the `ConfigPatch` type. No runtime YAML dependency. |
34
34
  | `src/state.ts` | State/history/lock files under the agent dir, and agent-dir resolution. Exports `agentDir`, `loadState`, `saveState`, `appendHistory`, `acquireLock`, `releaseLock`, `freshState`, and the `PluginState` type (which carries `managedDisabledAgents` — the `task.disabledAgents` names the plugin added). |
35
+ | `src/usage-ledger.ts` | The per-role usage ledger (spec #81): an append-only jsonl file (`llm-role-usage.jsonl`, cap `USAGE_LEDGER_CAP = 2000` rows) beside the plugin's other state, one row per assistant turn. Exports `UsageRow`, `TokenProfile`, `USAGE_LEDGER_FILE`, `USAGE_LEDGER_CAP`, `appendUsageRow` (best-effort, never throws), `readUsageRows` (tolerant: missing/malformed → `[]`), `roleTokenProfile` (the role's median per-turn input/output counts via `agent-create.ts`'s `quantile` — the repo's one quantile convention — or `null`), and `resolveTokenProfile` (recorded usage → the `tokenProfile` setting → `null`). The caller owns the dir; the extension records into `resolveProjectDir() ?? agentDir()`, the explorer reads the active scope's dir. |
35
36
  | `src/error-report.ts` | Opt-in reporting of **unexpected** plugin errors as GitHub issues on `Gott50/omp-llm-role` (the `errorReporting` setting). The plugin's only GitHub client and the only home for a fingerprint, a redaction rule, the agent-dir ledger (`ERROR_REPORT_LEDGER_FILE`) and the issue body. **Entry point:** `reportUnexpectedError(input, deps)` — the feature's one test seam (per-call input + fully injected `deps`; tests supply a fake `fetch`, clock, env, dir and token resolver). Also exports `defaultErrorReportDeps()` (production deps: resolved settings, global `fetch`, real clock, `agentDir`, the shipped manifest, `gh auth token`, the OS opener) and `resetErrorReportGuard()` (the in-process dedupe guard's test hook). Never throws, never rejects; a no-op when `errorReporting` is `off`. |
36
- | `src/updater.ts` | Orchestration: rank → tier gate → probe → hysteresis → chains → agent-disable sync → config write. Exports `runUpdater`, the `Deps`/`Decision`/`RunResult`/`Trigger`/`DecisionReason` types. Resolves the project scope (`<cwd>/.omp` when it carries an `omp-llm-role` settings entry) and scopes the config/state/history/lock to it. `agentDisablePatch` also removes a previously-managed `task.disabledAgents` name whose agent file is gone. `RunResult.defect` is set on the exception-abort paths only (a `ConfigEditError`/self-check mismatch and the catch-all throw); the enumerated environment aborts leave it unset. |
37
- | `src/extension.ts` | omp extension entry (default export). Registers `session_start` (awaited day-gated run), `/refresh-roles`, `/explore-roles` (in-process explorer), `/create-agent` (architect + benchmark-link resolution + catalog discovery), `/remove-agent`, `/project-roles` (profile architect + `setupProject` + in-process updater), and `session_shutdown`. Also subscribes to `session_start`, `retry_fallback_applied`, `retry_fallback_succeeded` and `turn_start` to keep the live model's display name suffixed with its provider pin (`src/provider-display.ts`); each handler mutates the live model's `name` in place and never calls `pi.setModel` (which would wipe omp's in-flight fallback state and clobber the thinking level). Owns `extDeps` (the extension's `Deps`) and the live-session model hook. Every unexpected command-handler throw and every `runUpdater` defect is routed through `src/error-report.ts`: `reportCommandError` for the six command-handler catch sites and `reportDefect` for a `runUpdater` result whose `defect` is set — each keeps the pre-existing one-line warning and adds the reporting module's own lines. |
37
+ | `src/updater.ts` | Orchestration: rank → tier gate → model probe → hysteresis → auto-pin route probe → chains → agent-disable sync → config write. Exports `runUpdater`, `resolveProjectDir` (the project scope's `<cwd>/.omp` when it carries a non-empty `omp-llm-role` settings entry, else `null` — the same test the explorer's scope resolution and the extension's usage-ledger dir use), the `Deps`/`Decision`/`RunResult`/`Trigger`/`DecisionReason` types. Resolves the project scope (`<cwd>/.omp` when it carries an `omp-llm-role` settings entry) and scopes the config/state/history/lock to it. `agentDisablePatch` also removes a previously-managed `task.disabledAgents` name whose agent file is gone. `RunResult.defect` is set on the exception-abort paths only (a `ConfigEditError`/self-check mismatch and the catch-all throw); the enumerated environment aborts leave it unset. |
38
+ | `src/extension.ts` | omp extension entry (default export). Registers `session_start` (awaited day-gated run), `/refresh-roles`, `/explore-roles` (in-process explorer), `/create-agent` (architect + benchmark-link resolution + catalog discovery), `/remove-agent`, `/project-roles` (profile architect + `setupProject` + in-process updater), and `session_shutdown`. Also subscribes to `session_start`, `retry_fallback_applied`, `retry_fallback_succeeded` and `turn_start` to keep the live model's display name suffixed with its provider pin (`src/provider-display.ts`); each handler mutates the live model's `name` in place and never calls `pi.setModel` (which would wipe omp's in-flight fallback state and clobber the thinking level). Owns `extDeps` (the extension's `Deps`) and the live-session model hook. Records the per-role usage ledger (spec #81) on `turn_end` and `session_shutdown` — one row per assistant turn read from `sessionManager.getBranch()`, into `resolveProjectDir() ?? agentDir()`, deduped by `ts|model` so the two hooks cannot double-record. Every unexpected command-handler throw and every `runUpdater` defect is routed through `src/error-report.ts`: `reportCommandError` for the six command-handler catch sites and `reportDefect` for a `runUpdater` result whose `defect` is set — each keeps the pre-existing one-line warning and adds the reporting module's own lines. |
38
39
  | `src/guards.ts` | The package's one type guard: `isRecord`. |
39
40
  | `src/provider-display.ts` | The provider display (spec #77): keep the live session model's `name` suffixed with its upstream provider pin, so omp's model segment renders e.g. `DeepSeek V4.1 Flash @coreweave/fp8`. Pure and dual-runtime (no SDK import). Exports `providerPinOf` (the single `compat.openRouterRouting.only[0]`, else `compat.vercelGatewayRouting.only[0]`; null unless exactly one non-empty string), `providerPinOfSelector` (the trailing `@<slug>` of a selector, after stripping a thinking suffix), `withProviderSuffix` (strip any previous ` @<slug>` then append, idempotent), and `applyProviderDisplay` (mutate a record's `name`; no-op otherwise). |
40
41
  | `src/role-settings.ts` | The one validated role write path (validate → merge → backup → atomic write) and its removal half. Exports `validateRole`, `mergeExport`, `writeRoleSettings`, `mergeRemove`, `removeRoleSettings`, `timestamp`. Shared by the explorer's Export, `src/cli/create-role.ts`, `/create-agent` and `/remove-agent`. |
@@ -43,7 +44,7 @@ the extension's `/refresh-roles`, `/explore-roles` and `/create-agent`.
43
44
  | `src/agent-architect.ts` | omp's in-process architects: the agent-creation architect (`/create-agent`, `/agents` hub) and the project-profile architect (`/project-roles`). Exports `generateAgentSpec`, `generateProjectProfile`, `parseAgentSpec`, `parseProjectProfile`, `extractJsonObject`, `extractAssistantText`, and the `ArchitectOptions`/`ProjectArchitectOptions` types. Imports `@oh-my-pi/pi-coding-agent` **dynamically** (inside the runner), so the module is importable under plain Node. |
44
45
  | `src/benchmark-author.ts` | Benchmark-source authoring for an unknown link: fetch the link, run an in-process architect (`src/prompts/benchmark-source-architect.md`) to propose a declarative source spec, and validate it. Imports `@oh-my-pi/pi-coding-agent` **dynamically** (inside the function). Exports `authorBenchmarkSource`, `parseSourceDeclarationJson`, and the `BenchmarkAuthorOptions` type. |
45
46
  | `src/benchmark-discovery.ts` | Judge-backed benchmark relevance for `/create-agent` discovery: one `noul` (yes/no probability) question per catalog candidate, batched into a single judgment through omp's configured `judge` role (TypeSafe jev or its fallback chain). Extension-only (imports `@oh-my-pi/pi-coding-agent` and its `./judgment` subpath — the deeper `./judgment/standalone` path does not resolve under omp's SDK injection). The SDK is imported dynamically so a resolution failure degrades to "no discovery" instead of failing the whole extension load. Exports `judgeBenchmarkRelevance`. |
46
- | `src/agent-create.ts` | `/create-agent` core: purpose → archetype → validated role → agent `.md`, plus the shared input parser and report formatter both hosts use. Exports `createAgent`, `resolveRole`, `parseCreateAgentInput` (flag form or free text), `parseCreateAgentArgs`, `tokenizeArgs`, `extractBenchmarks`, `extractBenchmarkLinks`, `applyFocusBenchmarks`, `assessFocusMetric`, `belowBarReason`, `focusCoverageOk`, `countMetricCoverage`, `discoverBenchmarks`, `checkWeightMath`, `formatBenchmarks`, `formatArchetypes`, `formatCreateAgentReport`, `CREATE_AGENT_USAGE`, and the request/result types (incl. `FocusMetricAssessment`/`FocusCoverageEntry`/`FocusAssessor`). |
47
+ | `src/agent-create.ts` | `/create-agent` core: purpose → archetype → validated role → agent `.md`, plus the shared input parser and report formatter both hosts use. Exports `createAgent`, `resolveRole`, `parseCreateAgentInput` (flag form or free text), `parseCreateAgentArgs`, `tokenizeArgs`, `extractBenchmarks`, `extractBenchmarkLinks`, `applyFocusBenchmarks`, `assessFocusMetric`, `belowBarReason`, `focusCoverageOk`, `countMetricCoverage`, `quantile` (the repo's one quantile convention — the focus-axis dispersion stats and the usage ledger's per-turn median), `discoverBenchmarks`, `checkWeightMath`, `formatBenchmarks`, `formatArchetypes`, `formatCreateAgentReport`, `CREATE_AGENT_USAGE`, and the request/result types (incl. `FocusMetricAssessment`/`FocusCoverageEntry`/`FocusAssessor`). |
47
48
  | `src/agent-remove.ts` | `/remove-agent` core: delete an agent `.md` and its role, plus the shared flag parser and report formatter. Exports `removeAgent`, `parseRemoveAgentArgs`, `formatRemoveAgentReport`, `REMOVE_AGENT_USAGE`, and the request/result types. Refuses shipped default roles. |
48
49
  | `src/project-setup.ts` | `/project-roles` core: a discovered `ProjectProfile` → a project-scoped role set (kept shipped roles, dropped ones, new fitted roles) written to the project plugin settings lock file, plus the project agents for the new roles. All-or-nothing (every check — names, metrics, weight math, the `--force` gate, agent-file collisions — before any write), `--dry-run`. Exports `setupProject`, `parseProjectRolesArgs`, `parseRolesSpec`, `applyProfileOverrides`, `formatProjectRolesReport`, `PROJECT_ROLES_USAGE`, and the `ProjectProfile`/`ProposedRole`/`SetupProjectOpts`/`SetupProjectResult` types (the result carries an `archetypes` map). The plan report prints the capability flags each new role will author (`features: <id>=<on>`). Dual-runtime safe (no SDK import); the extension command does the LLM work and calls it. |
49
50
 
@@ -51,9 +52,11 @@ the extension's `/refresh-roles`, `/explore-roles` and `/create-agent`.
51
52
 
52
53
  | File | Role / key exports |
53
54
  |---|---|
54
- | `src/explorer/boot.ts` | Shared explorer launcher: bind/port fallback, lock-file roles, browser open, close. Exports `startExplorer`, `EXPLORER_DEFAULT_PORT`, and the `ExplorerHandle`/`ExplorerBootOpts` types. Used by `/explore-roles`. |
55
- | `src/explorer/server.ts` | Zero-dependency HTTP surface (static SPA + JSON API). Exports `createExplorerServer` and the `ExplorerOpts` type. Endpoints: `GET /api/bootstrap` (carries `focusAssessments`), `POST /api/rank`, `POST /api/explain`, `POST /api/export`, `POST /api/refresh`. |
56
- | `src/explorer/explain.ts` | Pure explanation layer: rank rows with baseline deltas, per-model decomposition, inverse-cardinal targets, and the nine-axis focus assessment. Exports `rankRows`, `explainModel`, `inverseCardinal`, `focusMetricsOf`, `focusAssessments`, `METRIC_META`, `metricMeta`, `metricMetaFor`, `weightableMetrics`, and the `RankRow`/`Explanation`/`Contribution`/`Closing`/`MetricMeta` types. |
55
+ | `src/explorer/boot.ts` | Shared explorer launcher: bind/port fallback, lock-file roles, browser open, close. Exports `startExplorer`, `EXPLORER_DEFAULT_PORT`, and the `ExplorerHandle`/`ExplorerBootOpts` types. Used by `/explore-roles`. `ExplorerBootOpts` carries the as-written pool's inputs (`tier`, `allowedProviders`, `reloadAllowedProviders`), the `tokenProfile` fallback, and `usageDir` (the user-level ledger dir; a project scope reads its own `<root>/.omp` ledger). |
56
+ | `src/explorer/server.ts` | Zero-dependency HTTP surface (static SPA + JSON API). Exports `createExplorerServer` and the `ExplorerOpts` type. Endpoints: `GET /api/bootstrap` (carries `focusAssessments`, `poolMode`, `tokenProfile`), `POST /api/scope`, `POST /api/pool` (switch the ranked pool, raw vs as-written), `POST /api/rank` (rows carry the display-only `perTask`), `POST /api/explain` (carries `leaderFlip`/`thinkingLevels`), `POST /api/export`, `POST /api/refresh`. `ExplorerOpts` gains `getPoolInputs()` and the optional `getUsageRows(scope)`/`tokenProfile` seams. |
57
+ | `src/explorer/explain.ts` | Pure explanation layer: rank rows with baseline deltas, per-model decomposition, inverse-cardinal targets, the nine-axis focus assessment, and the display-only per-task price. Exports `rankRows`, `explainModel`, `perTaskPrice`, `inverseCardinal`, `focusMetricsOf`, `focusAssessments`, `METRIC_META`, `metricMeta`, `metricMetaFor`, `weightableMetrics`, and the `RankRow`/`Explanation`/`Contribution`/`Closing`/`MetricMeta`/`PerTaskPrice` types. `rankRows`/`explainModel` take a trailing optional `profile?: TokenProfile \| null`; `RankRow.perTask` and `Explanation.perTask` are `null` when no profile resolves or the model has no usable route price. |
58
+ | `src/explorer/levers.ts` | Pure price-lever readouts (spec #81): the leader-flip threshold and the thinking level's effect on the leader, both computed on the pool they are handed. Exports `leaderFlip`, `thinkingLevelEffect`, and the `LeaderFlip`/`ThinkingLevelRow` types. Reuses the engine's `roleLambda`/`rankRole`/`SUFFIX_LEVELS`/`thinkingPriceFactor` — never re-derives the math. |
59
+ | `src/explorer/pool.ts` | The as-written pool (spec #81): the explorer's dataset filtered exactly as the updater filters it before ranking — the catalog/tier gate (`filterCatalog` + `resolveVariant`) and the allowed-providers whitelist prune (`providerAllowed`). Pure (never mutates its input). Exports `asWrittenPool`, and the `PoolMode`/`PoolInputs` types. |
57
60
 
58
61
  ### Assets and non-code
59
62
 
@@ -142,20 +145,39 @@ The omp extension injects the production `Deps`; tests inject fakes.
142
145
  6. **Hysteresis.** `no-current` → adopt; ineligible current → `adopted`; best
143
146
  beats current by `switchMargin` → `switched`; inside the margin but
144
147
  `cheaperInsideMargin` → `switched-cost`; else `kept-margin` / `kept-eligible`.
145
- 7. **Suffix.** Append `roles.<role>.thinking` only when the chosen catalog row's
148
+ 7. **Route pin (auto-pin).** With the `providerPinning` capability on and no
149
+ manual `providerPin`, the chosen model's `rankedRoutes` order is walked and
150
+ each candidate is probed with `probeRoute` (a one-token completion bound to
151
+ the slug; optional seam `deps.probeRoute?`, DI like `probeModel`); the first
152
+ route that is not `blocked` becomes the `@<slug>` pin. `unknown` counts as
153
+ usable; a model whose every candidate is rejected (or whose probe budget ran
154
+ out) drops the pin — the selector stays bare and `Decision.pinDropped` is set.
155
+ Verdicts are cached per run by `catalogId@slug`; the walk spends at most
156
+ `ROUTE_PROBE_BUDGET = 12` new probes per role. It never runs for a manual pin,
157
+ a locked or disabled role, a non-OpenRouter selector or a model with no
158
+ candidate route. Contract:
159
+ [`writes-and-state.md`](writes-and-state.md), *The provider-pin
160
+ auto-configuration*.
161
+ 8. **Suffix.** Append `roles.<role>.thinking` only when the chosen catalog row's
146
162
  `thinking[]` includes the level (meta levels `off`/`auto` need only a
147
163
  non-empty list).
148
- 8. **Chains.** For every managed role, `chainUpserts` (bare key → next
149
- `fallbackChainDepth` probe-clean candidates, suffixed per target) and
150
- `chainPrunes` (plugin-written keys no longer referenced). A key claimed by
151
- more than one role stays level-free.
152
- 9. **Config patch.** `writePatch` acquires the refresh lock, reads `config.yml`,
153
- `patchConfig` (surgical, self-checked), `writeConfigAtomic` (mtime-guarded
154
- tmp+rename, 3 conflict retries). Zero changes → no write, no mtime change.
155
- 10. **State + history.** `saveState` (day gate, managed roles, last selectors,
164
+ 9. **Chains.** For every managed role, `chainUpserts` (bare key → the next
165
+ `fallbackChainDepth` fallback models drawn from the role's ranked candidate
166
+ list below the chosen model — the cheaper-or-equal candidates that retain at
167
+ least `fallbackQualityFloor` of its `q` first, then the pricier ones, each
168
+ bucket in value order, truncated to the depth — suffixed per target) and
169
+ `chainPrunes` (plugin-written keys no longer referenced). The list is the
170
+ ranked candidate list minus the key-blocked and probe-`blocked` candidates; an
171
+ unprobed candidate stays eligible, so the chain can reach a cheaper-or-equal
172
+ model below the probe window (writes-and-state.md, *Which models are
173
+ chained*). A key claimed by more than one role stays level-free.
174
+ 10. **Config patch.** `writePatch` acquires the refresh lock, reads `config.yml`,
175
+ `patchConfig` (surgical, self-checked), `writeConfigAtomic` (mtime-guarded
176
+ tmp+rename, 3 conflict retries). Zero changes → no write, no mtime change.
177
+ 11. **State + history.** `saveState` (day gate, managed roles, last selectors,
156
178
  plugin-written chain keys, managed `task.disabledAgents` names, previous
157
179
  `modelRoles` snapshot) and `appendHistory` (one row per completed run).
158
- 11. **Live-session coupling.** When `default` actually changed and
180
+ 12. **Live-session coupling.** When `default` actually changed and
159
181
  `activateDefaultOnEmptySession` is set, `deps.applySessionModel?.(selector,
160
182
  previous)` hands the new selector to the host (extension only).
161
183
 
@@ -218,6 +240,7 @@ type Deps = {
218
240
  getSettings?(): Promise<Record<string, unknown>>;
219
241
  getKeyMeta?(token: string): Promise<KeyMeta>;
220
242
  probeModel?(token: string, catalogId: string): Promise<ProbeVerdict>;
243
+ probeRoute?(token: string, catalogId: string, providerSlug: string): Promise<ProbeVerdict>;
221
244
  getAllowedProviders?(token: string, modelId: string): Promise<Set<string> | null>;
222
245
  // Host coupling (extension only): apply a concrete selector to the live session model.
223
246
  applySessionModel?(selector: string, previous: string | null): Promise<void>;
package/docs/dev/ci.md CHANGED
@@ -21,6 +21,15 @@ ref must be green first. `test` runs **exactly** the command the local
21
21
  pre-commit hook runs (`.githooks/pre-commit`), and `tests/ci.test.ts` fails when
22
22
  the two drift apart — "green locally" and "green in CI" cannot diverge.
23
23
 
24
+ The hook scrubs `GIT_DIR`, `GIT_WORK_TREE`, `GIT_INDEX_FILE` and
25
+ `GIT_OBJECT_DIRECTORY` before running the suite, and `tests/ci.test.ts` scrubs
26
+ the same variables from every child process it spawns. Git exports `GIT_DIR` to
27
+ every hook, and `GIT_DIR` overrides `cwd` — so without the scrub the test file's
28
+ temp-repo `git init`/`config`/`add`/`commit` calls target the **real**
29
+ repository, rewriting its config and moving a branch (issue #89). The test file
30
+ carries the scrub because a test that shells out to git must not depend on its
31
+ caller's environment; the hook carries it as defence in depth.
32
+
24
33
  The Node version comes from `.node-version` at the repo root (the exact local
25
34
  toolchain; bumping it is a normal commit). `engines` is deliberately not in
26
35
  `package.json`: the plugin executes inside omp's runtime, so an `engines` field
@@ -1,4 +1,4 @@
1
- # Current state (2026-10-11)
1
+ # Current state (2026-10-19)
2
2
 
3
3
  Dated snapshot of the ranking and the landed work as of the date in the heading —
4
4
  matched/priced/eligible counts, per-role leaders, the open defects, and the
@@ -6,11 +6,111 @@ per-issue work log below. It is refreshed by
6
6
  regenerating the report (`node src/cli/llm-role-rank.ts --out docs/llm-role-rankings.md`) and
7
7
  copying the new numbers here; release prose belongs in
8
8
  [`../../CHANGELOG.md`](../../CHANGELOG.md), not in this file. The model-level
9
- ranking numbers below are the 2026-10-09 measurement (the input-price blend basis
10
- #21, the #17/#18/#19/#20 wave and the #34–#40 focus-axis/capability-flag work);
11
- the provider-pin numbers are the 2026-10-11 measurement (spec #71; see the top
9
+ ranking numbers below are the 2026-10-19 measurement (regenerated at the 1.3.0
10
+ cut; byte-identical to the 2026-10-18 one apart from the report's date line —
11
+ the input-price blend basis #21, the #17/#18/#19/#20 wave, the #34–#40
12
+ focus-axis/capability-flag work, and
13
+ spec #81's explorer surfaces — the ranking CLI itself is unchanged); the
14
+ provider-pin numbers are the 2026-10-11 measurement (spec #71; see the top
12
15
  bullet). Neither is re-derived on read.
13
16
 
17
+ - Fallback chain quality floor (2026-10-19, spec #95): a role's fallback chain no
18
+ longer only reorders the probe window — that window is `1 + fallbackChainDepth`
19
+ models, the same size as the chain, so the reorder could never reach a cheaper
20
+ model at rank 4 or deeper. The chain's fallback **models** now come from the
21
+ role's whole ranked candidate list below the chosen model: the
22
+ **cheaper-or-equal** ones first — a candidate qualifies only if it retains at
23
+ least `fallbackQualityFloor` (a new setting, default `0.9`; `0` disables the
24
+ floor, `1` requires an exact quality match) of the chosen model's own `q`, so a
25
+ failover never silently drops the role onto a much weaker model — then the
26
+ pricier ones, each bucket in value order, truncated to `fallbackChainDepth`.
27
+ Key-blocked and probe-`blocked` candidates are excluded; an **unprobed**
28
+ candidate stays eligible, so the chain can reach a cheaper-or-equal model ranked
29
+ below the probe window (chain entries are never probe-verified — spec #90). A
30
+ candidate priced exactly at the chosen model's price whose `q` is below the
31
+ floor lands in neither bucket and is dropped. The floor gates **chain
32
+ membership only**: the primary choice, the hysteresis decision, the ranking, the
33
+ probe walk's stop condition and budget, the chain key, the route expansion, the
34
+ thinking levels and the prune-on-pin-change are all unchanged, as are the
35
+ manual-`providerPin` and no-pin paths. Measured on the 2026-10-19 caches with
36
+ the live resolved defs (`fallbackQualityFloor` 0.9, depth 2): `slow`, `vision`
37
+ and `task` gain cheaper fallbacks, the other seven roles are byte-identical, and
38
+ no role is degraded. That is the **per-role** effect: in the written config only
39
+ the `vision` key changes, because `deepseek-v4.1-flash@coreweave/fp8` is claimed
40
+ by seven roles and carries the last claimer's chain (the shared-key behaviour
41
+ spec #95 leaves out of scope). The ranking numbers are unchanged — the chain is a
42
+ write-side shape, not a ranking input — so `docs/llm-role-rankings.md` is not
43
+ regenerated and the dated numbers below stand as written; the suite grew
44
+ 652 → 666.
45
+
46
+ - Routability-verified auto-pin (2026-10-19, spec #90): the `providerPinning`
47
+ auto-pin no longer writes the score-max route blind. The daily update walks the
48
+ chosen model's `vfm` route order and probes each candidate with a one-token
49
+ completion bound to it (`provider: { only: [slug] }`), because OpenRouter's
50
+ endpoints API can list a route as healthy — non-degraded, a full
51
+ `supported_parameters` list, 100 % uptime — while its routing table excludes it:
52
+ the live config's `deepseek/deepseek-v4.1-flash@relace` passed every gate and
53
+ then 404'd on every request, including every fallback revert. Only the narrow
54
+ no-allowed-providers 404 skips a route; a timeout, a 5xx, a 429 and a 404 with
55
+ any other body are `unknown` and count as usable, so a transient blip never
56
+ unpins a role. Verdicts are cached per run by `catalogId@slug`, and the walk is
57
+ capped at `ROUTE_PROBE_BUDGET = 12` probes per role. When no candidate is
58
+ routable the pin is **dropped** — the selector stays bare and OpenRouter routes
59
+ the request itself — and the run notes name the rejected slugs
60
+ (`; route-blocked: …`), plus the dropped pin (`; pin dropped (no routable
61
+ route)`) when none survived. The fallback chain keeps its shape
62
+ (the primary contributes every gate-passing route except the pinned **slug** —
63
+ see the chain bullet below), and **chain entries are not probed**: one
64
+ unroutable entry is survivable, omp fails over. A manual
65
+ `roles.<role>.providerPin` stays a hard gate: never probed, never replaced. The
66
+ explorer's auto-pin line now reads as a **candidate** — the first route the walk
67
+ tries, not a promise of the written pin. The ranking numbers are unchanged — the
68
+ probe is a write-side gate, not a ranking input, and neither the ranking CLI nor
69
+ `docs/llm-role-rankings.md` changed; the suite grew 631 → 652.
70
+
71
+ - Release 1.3.0 (2026-10-19, cut `55416d4`, tag `v1.3.0`): the spec #81 explorer
72
+ surfaces ship. The report was regenerated at the cut and is **unchanged** from
73
+ the 2026-10-18 measurement — matched 149/408, priced 148, eligible 140
74
+ (vision 73, designer 87), the same ten leaders — so the numbers below stand as
75
+ written. The cut also carried the #89 fix: the pre-commit hook and
76
+ `tests/ci.test.ts` now scrub `GIT_*` from every child process, so the suite's
77
+ temp-repo git calls can no longer target the real repository (suite 630 →
78
+ 631).
79
+
80
+ - Explorer as-written pool, per-task price, and price levers (2026-10-18, spec
81
+ #81): the explorer gains a **pool toggle** (raw vs as-written), a **per-task
82
+ price column**, and two **price-lever readouts**. The as-written pool applies
83
+ the same two filters the updater applies before ranking — the catalog/tier gate
84
+ and the allowed-providers whitelist prune — so the leader the user sees is the
85
+ model the updater will write (the raw pool's leader differed from the config's
86
+ for 6 of 10 roles). The per-task price is `inputTokens·p_in +
87
+ outputTokens·p_out·thinkingFactor` over the role's priced route, using the
88
+ plugin's own recorded per-role usage (the median per-turn counts) or the
89
+ `tokenProfile` setting; it is **display-only** — the price axis stays `$/M`
90
+ (issue #32's verdict stands) and the reported `$/M` keeps its billed 3:1 blend.
91
+ The readouts are the leader-flip threshold (`λ* = min over cheaper challengers
92
+ X of (q_L − q_X)/(pEff_L − pEff_X)`, `w* = 20λ*/(1+20λ*)`) and the thinking
93
+ level's effect on the leader. A new append-only, capped, best-effort
94
+ **usage ledger** (`llm-role-usage.jsonl`, one row per assistant turn) follows
95
+ the plugin's state dir (`<project>/.omp` in project mode, `agentDir()`
96
+ otherwise). The ranking numbers are unchanged — the ranking CLI does not apply
97
+ the pool toggle, and the per-task price never feeds the sort key.
98
+ - Provider display (2026-10-12, spec #77): when the live session model carries a
99
+ provider pin (a hand-set `providerPin`, or the automatic pin the
100
+ `providerPinning` capability writes), the plugin keeps the model object's
101
+ `name` suffixed with the route, so omp's status-line model segment renders
102
+ e.g. `DeepSeek V4.1 Flash @coreweave/fp8`. The suffix is applied on session
103
+ start, refreshed on the two fallback events, and re-derived at every
104
+ `turn_start` — the re-derive is what heals the display after omp silently
105
+ restores the primary model following a fallback cooldown (no session event is
106
+ emitted), and it also covers a manual `/model` change with a one-turn lag. An
107
+ event's selector is trusted only when it names the live session model, so an
108
+ advisor fallback (the same events, the advisor's model) re-derives from the
109
+ live model instead of stamping the advisor's pin onto the session.
110
+ Display-only: the suffix rides the model object's `name`, so attribution,
111
+ usage recording and model comparison are unaffected. The ranking numbers are
112
+ unchanged.
113
+
14
114
  - Provider ordering: value-for-money route score (2026-10-11, spec #71): the
15
115
  auto-pin route basis changed from the route's own value (`q_route −
16
116
  λ·priceEff_route`) to a min/max-normalized **value-for-money score** `vfm =
@@ -31,15 +131,21 @@ bullet). Neither is re-derived on read.
31
131
 
32
132
  - Auto-pin chain expansion (2026-10-10, issue #69): with `providerPinning` on,
33
133
  the fallback chain no longer stops at three providers per model — the primary
34
- model contributes every gate-passing route except the pinned best, then each
35
- `fallbackChainDepth` fallback model contributes every gate-passing route, in
36
- score order (the `vfm` route score — spec #71 replaced the original
37
- route-value order) and deduped by provider slug, so a down provider fails over
38
- to another provider of the same model before the chain switches models. Chain
39
- length is data-driven (`(R_primary − 1) + Σ R_fallback`) instead of the fixed
134
+ model contributes every gate-passing route **except the pinned route** (the
135
+ exclusion is by pinned slug, not by position: spec #90's probe can pin a route
136
+ other than the score leader, so the leader rides as a chain entry, and a dropped
137
+ pin excludes nothing), then each `fallbackChainDepth` fallback model contributes
138
+ every gate-passing route, in score order (the `vfm` route score — spec #71
139
+ replaced the original route-value order) and deduped by provider slug, so a down
140
+ provider fails over to another provider of the same model before the chain
141
+ switches models. Chain length is data-driven (`(R_primary − 1) + Σ R_fallback`,
142
+ or `R_primary + Σ R_fallback` with a dropped pin) instead of the fixed
40
143
  `2 + 3·fallbackChainDepth`; the manual-`providerPin` and no-pin paths are
41
- unchanged. The ranking numbers are unchanged — the chain is a write-side
42
- shape, not a ranking input.
144
+ unchanged. Which `fallbackChainDepth` **models** each chain falls back to was
145
+ the probe window until spec #95, which draws them from the role's ranked
146
+ candidate list below the chosen model with the quality floor (see the top
147
+ bullet) — the route expansion above is unchanged. The ranking numbers are
148
+ unchanged — the chain is a write-side shape, not a ranking input.
43
149
  - Opt-in error reporting (2026-10-08, issue #48): a new `errorReporting` setting
44
150
  (`off` — the default — `ask`, `auto`) turns a genuinely unexpected failure — a
45
151
  command handler throwing, or a `runUpdater` abort whose `RunResult.defect` is
@@ -653,18 +759,18 @@ bullet). Neither is re-derived on read.
653
759
  0.663, margin 0.039 > `switchMargin`), because the incumbent has no Design
654
760
  Arena data and takes the neutral fill while MiMo carries a measured 0.97.
655
761
  The other nine roles are untouched (only `designer` weights `website`).
656
- - 408 llm-stats models; OpenRouter matched 155/408 (throughput), 153 priced (the
657
- 2026-10-08 snapshot read 155/407 — the daily cache moved, not the code; the
762
+ - 408 llm-stats models; OpenRouter matched 149/408 (throughput), 148 priced (the
763
+ 2026-10-09 snapshot read 155/408 — the daily cache moved, not the code; the
658
764
  #34–#40 axes and flags leave every ranking byte-identical until a flag is
659
765
  turned on).
660
- - Eligible per role: 146 (vision 75, designer 89, image-input filter).
766
+ - Eligible per role: 140 (vision 73, designer 87, image-input filter).
661
767
  - Value-ranking leaders (this report, thinking-adjusted prices; the *ranking*
662
768
  leaders, before the account's provider whitelist drops the blocked ones):
663
- `default` DeepSeek-V4.1-Flash (0.810), `smol` DeepSeek-V4.1-Flash (0.729),
664
- `slow` DeepSeek-V4.1-Flash (0.832), `vision` DeepSeek-V4.1-Flash (0.730),
665
- `plan` Muse Spark 1.3 (0.793), `commit` DeepSeek-V4.1-Flash (0.762), `tiny`
666
- Muse Spark 1.1 (0.791), `task` DeepSeek-V4.1-Flash (0.754), `advisor`
667
- Muse Spark 1.3 (0.787), `designer` DeepSeek-V4.1-Flash (0.787).
769
+ `default` DeepSeek-V4.1-Flash (0.807), `smol` DeepSeek-V4.1-Flash (0.706),
770
+ `slow` Gemini 4 Argon (0.834), `vision` Gemini 4 Argon (0.732),
771
+ `plan` Muse Spark 1.3 (0.789), `commit` DeepSeek-V4.1-Flash (0.736), `tiny`
772
+ Mercury 2 (0.797), `task` DeepSeek-V4.1-Flash (0.750), `advisor`
773
+ Muse Spark 1.3 (0.785), `designer` DeepSeek-V4.1-Flash (0.772).
668
774
  - Thinking-adjusted pricing landed (2026-09-27): the suffix table moved into
669
775
  `DEFAULT_ROLES` as a per-role `thinking` field, and the price axis scales by
670
776
  the level's factor for thinking-capable models — `slow` (`:max`, ×7.86)