model-compare-opencode-plugin 0.4.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -10,7 +10,7 @@ Each priority is a live alias inside the existing OpenRouter provider — no new
10
10
  | `openrouter/model-compare/price` | best price pick |
11
11
  | `openrouter/model-compare/quality` | best quality pick |
12
12
 
13
- The alias re-points at the current pick while OpenCode runs (refresh every 30 minutes, data cache 6 hours). OpenCode's model picker shows it as `Model Compare: <priority> → <pick>`, with `(fallback)` or `(unavailable)` when the fallback is serving or the alias is disabled.
13
+ The alias re-points at the current pick while OpenCode runs (refresh every 30 minutes, data cache 6 hours). It also pins the catalog's scored reasoning effort on each request, so the model runs at the depth its `quality` score was measured at. OpenCode's model picker shows it as `Model Compare: <priority> → <pick> · <effort> [alt <effort> <index>]`, with `(fallback)` while the fallback is serving, `(unavailable)` when the alias is disabled, and the `alt` part naming the best measured effort above the current one that OpenRouter can actually select.
14
14
 
15
15
 
16
16
  ## Install
@@ -69,7 +69,7 @@ opencode run --model "openrouter/model-compare/price" "msg"
69
69
  Commands:
70
70
 
71
71
  - `/use-best [priority]` — switch the session to `openrouter/model-compare/<p>` (defaults to `defaultPriority`). Refuses a disabled alias.
72
- - `/model-compare` — top 5 per priority with scores and data age; priorities whose `weights` override actually takes effect are labelled `(custom weights)` with recomputed scores, and each alias's current pick is marked `← current pick`. The output is a synthetic message (`resume: false`): it costs no model turn and, in a headless session, surfaces in the transcript at the next turn. In the TUI it opens the report dialog instead.
72
+ - `/model-compare` — top 5 per priority with scores and data age; each line also shows the reasoning effort behind `quality` (`· high`) and the best selectable measured alternative (`· alt max 56.0`). Priorities whose `weights` override actually takes effect are labelled `(custom weights)` with recomputed scores, and each alias's current pick is marked `← current pick`. The output is a synthetic message (`resume: false`): it costs no model turn and, in a headless session, surfaces in the transcript at the next turn. In the TUI it opens the report dialog instead, with `effort` and `alt` columns.
73
73
 
74
74
 
75
75
  ### Keyboard shortcuts
@@ -85,7 +85,7 @@ Commands:
85
85
 
86
86
  Both dialogs open anywhere — including the opening screen, where no session exists yet. There, picking a row (a priority in the picks dialog, a model in the report) creates a new session already bound to that model and opens it; merely opening a dialog never creates a session. With a session open, a pick switches that session as before. The TUI has no `/use-best` keybinding: the slash command already covers switching.
87
87
 
88
- Status: the footer shows `MC: balanced → <pick>` (balanced only — `disabled`/`unavailable`/`(fallback)` as applicable; all three priorities live in the picks dialog). Toasts announce `/use-best` outcomes and pick changes (never for the first observation after load), the `alt+e` served endpoint, and the cannot-steer notice — see [Steering and served endpoints](#steering-and-served-endpoints). Headless, command feedback is synthetic transcript messages at the next turn; the steering toasts have no headless surface.
88
+ Status: the footer shows `MC: balanced → <pick>` (balanced only — `disabled`/`unavailable`/`(fallback)` as applicable; all three priorities live in the picks dialog). The picker's alias name carries the effort and alternative; the footer stays a compact served-model line. Toasts announce `/use-best` outcomes and pick changes (never for the first observation after load), the `alt+e` served endpoint, and the cannot-steer notice — see [Steering and served endpoints](#steering-and-served-endpoints). Headless, command feedback is synthetic transcript messages at the next turn; the steering toasts have no headless surface.
89
89
 
90
90
 
91
91
  ## Configuration
@@ -155,18 +155,18 @@ The alt+e capture is best-effort by design, and alias-scoped: only a session who
155
155
 
156
156
  The last validated catalog is persisted, so ranking and `/model-compare` work offline after a restart. The cache is keyed by source (URL or script path), so changing the source never serves the previous source's data. Script exit code 2 means "no candidates" *or* an argparse usage error — never read it as "no candidates" alone.
157
157
 
158
- **Vendored pin:** `vendor/model_compare.py` is byte-identical to upstream `v0.5.0` (`vendor/UPSTREAM.json` records repo, tag, commit, sha256). The catalog is schema v4 (since upstream 0.5.0): v3's `pricing` tier/schedule structure and published ranking `parameters.weights` are unchanged, and v4 adds per-model quality provenance (`quality_as_of`, `quality_drift`) plus a reshaped `sources.aa` block — nothing `validateCatalog` reads changed shape. The plugin accepts schema v4 ONLY — `validateCatalog` rejects anything else — and releases are gated on the live remote serving v4 (the release-gate check pipes `https://model-compare.dev/catalog.json` through `scripts/canary-check.mjs`, which fails closed when the served document is not the expected schema). Effective-tier pricing semantics still apply: `pricing` scalars hold effective-tier values — the tier in effect at 1M-token prompts — so models with OpenRouter price overrides are ranked at what a long session actually pays.
158
+ **Vendored pin:** `vendor/model_compare.py` is byte-identical to upstream `v0.6.0` (`vendor/UPSTREAM.json` records repo, tag, commit, sha256). The catalog is schema v5 (since upstream 0.6.0): `quality` is the AA intelligence index at the model's effective OpenRouter default reasoning effort, and every entry carries `quality_effort` plus a `reasoning` block (`default_effort`, the expanded `supported_efforts`, the raw flags, and the measured `depths` with AA `$/task`). The plugin accepts schema v5 ONLY — `validateCatalog` rejects anything else — and releases are gated on the live remote serving v5 (the release-gate check pipes `https://model-compare.dev/catalog.json` through `scripts/canary-check.mjs`, which fails closed when the served document is not the expected schema). Each alias request pins exactly the pick's `quality_effort` in its body (`reasoning: { effort }`), so the alias runs at the depth its score represents; an explicit `#variant` selection, or a body that already sets `reasoning`/`reasoning_effort`, wins and is never overwritten. The catalog's `reasoning` sentinel is display-only and never sent to the gateway. Effective-tier pricing semantics still apply: `pricing` scalars hold effective-tier values — the tier in effect at 1M-token prompts — so models with OpenRouter price overrides are ranked at what a long session actually pays.
159
159
 
160
- **Vendored 0.5.0 cold-cache network profile:** since 0.4.0 the vendored script performs per-model endpoint-pricing fetches when building a fresh catalog (74 candidates in the recorded run). Each fetch fails closed per model — after the script's internal retries, a failed endpoint fetch drops that model and the catalog shrinks silently (429-class responses raise without retry) — so a degraded network costs pool coverage, not pricing correctness. Since 0.5.0, unkeyed runs also hydrate quality from the published catalog (one fetch of `https://model-compare.dev/catalog.json`, cached under `published-catalog`; keyed runs consult the AA API — now the primary quality source — and never fetch the published catalog); every quality hydration fails closed to "no quality data" per model. A cold cache under a slow network can exceed the plugin's 60-second local budget, in which case the plugin warns and falls back (`auto` → remote, `local` → its cached document).
160
+ **Vendored 0.6.0 cold-cache network profile:** since 0.4.0 the vendored script performs per-model endpoint-pricing fetches when building a fresh catalog (76 candidates in the recorded run). Each fetch fails closed per model — after the script's internal retries, a failed endpoint fetch drops that model and the catalog shrinks silently (429-class responses raise without retry) — so a degraded network costs pool coverage, not pricing correctness. Unkeyed runs hydrate quality from the published catalog (one fetch of `https://model-compare.dev/catalog.json`, cached under `published-catalog`; keyed runs consult the AA API — the primary quality source — and never fetch the published catalog); every quality hydration fails closed to "no quality data" per model, and a missing record at the selected effort shows `-` rather than borrowing another effort's score. A cold cache under a slow network can exceed the plugin's 60-second local budget, in which case the plugin warns and falls back (`auto` → remote, `local` → its cached document).
161
161
 
162
162
  **Upgrading the pin** — reviewed PRs only; nothing merges automatically:
163
163
 
164
164
  1. `node scripts/sync-vendor.mjs <tag>` — fetches the script at that commit, reports the sha256 change, re-pins `UPSTREAM.json`, prints the matching CHANGELOG section (`--check` verifies offline).
165
- 2. Conformance tests check `--version`, schema v4, required fields, recomputed `overall` scores, and the standard-library-only import allowlist.
165
+ 2. Conformance tests check `--version`, schema v5, required fields, recomputed `overall` scores, and the standard-library-only import allowlist.
166
166
  3. The upgrade PR body lists script lines touching imports, URLs, environment variables, subprocess calls and file writes, for reviewer attention.
167
167
  4. Release with a changelog line naming the upstream tag; users run `opencode plugin update` (OpenCode checks but doesn't apply).
168
168
 
169
- A scheduled workflow opens a draft upgrade PR when upstream publishes a newer release (never a downgrade); a nightly canary exercises the vendored script against live upstream data and asserts the generated catalog is schema v4 (`scripts/canary-check.mjs` pins `EXPECTED_SCHEMA_VERSION = 4` and exits nonzero on any other version); releases additionally gate on the live remote itself (see the publishing gates below); a weekly drift check compares the vendored sha256 against upstream's pinned commit and the published rankings against the published scores.
169
+ A scheduled workflow opens a draft upgrade PR when upstream publishes a newer release (never a downgrade); a nightly canary exercises the vendored script against live upstream data and asserts the generated catalog is schema v5 (`scripts/canary-check.mjs` pins `EXPECTED_SCHEMA_VERSION = 5` and exits nonzero on any other version); releases additionally gate on the live remote itself (see the publishing gates below); a weekly drift check compares the vendored sha256 against upstream's pinned commit and the published rankings against the published scores.
170
170
 
171
171
 
172
172
  ## Troubleshooting
@@ -210,7 +210,7 @@ MC_INTEGRATION=1 npx vitest run tests/integration/ # live-server suite (opt-i
210
210
  - The integration files skip unless `MC_INTEGRATION=1`, so plain `npm test` stays hermetic. The suite drives a real `opencode serve` on 127.0.0.1:14721 and needs the `opencode` CLI and `python3` (skips cleanly when either is missing); one case needs network access and the `sqlite3` CLI.
211
211
  - `node scripts/pack-smoke.mjs` (also in CI, after `node scripts/build-tui.mjs`) packs the plugin and checks the tarball against the packaging list (files, `exports`, relative imports, and that the TUI artifact is precompiled — no react import). It does **not** load the package in OpenCode — install the tarball into a scratch project and confirm OpenCode loads it before publishing.
212
212
  - Publishing: cut a GitHub release tagged `vX.Y.Z` and `.github/workflows/release.yaml` stages the package on npm through trusted publishing (tokenless OIDC — no token secret; the npmjs.com trusted-publisher entry must name `release.yaml`, the `canonical/model-compare-opencode-plugin` repository and the `release` environment, with "Allow npm publish" and "Allow npm dist-tag" unchecked — stage-only, npm's recommended posture). Nothing goes live until a maintainer approves it with 2FA on npmjs.com (package page → Staged Packages) or via `npm stage approve`. The workflow fails closed when the tag does not match `package.json`'s version, and the gates (build, typecheck, tests, pack-smoke, live-remote canary) run before staging.
213
- - `fixtures/catalog-v0.5.0.json` is the sole contract fixture — a recorded schema-v4 catalog; `fixtures/catalog-no-quality.json` is derived from it by `scripts/make-no-quality-fixture.py` (the vendored script's own quality-blind weights, so its scores are internally consistent). No pre-v4 fixture exists; the v4-only gate rejects one. `node scripts/sync-vendor.mjs --check` verifies the pin's sha256 offline.
213
+ - `fixtures/catalog-v0.6.0.json` is the sole contract fixture — a recorded schema-v5 catalog; `fixtures/catalog-no-quality.json` is derived from it by `scripts/make-no-quality-fixture.py` (the vendored script's own quality-blind weights, so its scores are internally consistent). No pre-v5 fixture exists; the v5-only gate rejects one. `node scripts/sync-vendor.mjs --check` verifies the pin's sha256 offline.
214
214
  - Repo layout: `src/core/` (pure ranking/pinning), `src/sources/` (local + remote acquisition), `src/opencode/` (aliases, guard, commands, storage, refresh), `dist/` (generated `tui.js` artifact, gitignored), `vendor/` (pinned upstream script), `tests/{unit,conformance,integration}`.
215
215
 
216
216
 
package/dist/tui.js CHANGED
@@ -157,6 +157,8 @@ function toReportEntries(raw) {
157
157
  value: entry.value,
158
158
  score: typeof entry.score === "string" ? entry.score : "-",
159
159
  quality: typeof entry.quality === "string" ? entry.quality : "-",
160
+ effort: typeof entry.effort === "string" ? entry.effort : "",
161
+ alternative: typeof entry.alternative === "string" ? entry.alternative : "",
160
162
  price: typeof entry.price === "string" ? entry.price : "-",
161
163
  current: entry.current === true,
162
164
  fromFallback: entry.fromFallback === true
@@ -708,8 +710,15 @@ export default Plugin.define({
708
710
  const labels = data.entries.map(entry => `${entry.rank === null ? "·" : entry.rank}. ${entry.id}${entry.fromFallback ? " (fallback)" : ""}`);
709
711
  const labelOf = new Map(data.entries.map((entry, index) => [entry, labels[index]]));
710
712
  const colW = (values, floor, cap) => Math.min(cap, Math.max(floor, ...values.map(value => value.length)));
711
- const qualW = colW(data.entries.map(entry => entry.quality), "qual".length, 8);
712
- const priceW = colW(data.entries.map(entry => entry.price), "$ in/out".length, 16);
713
+ // Quality is a one-decimal AA index (max "100.0"); the caps leave the
714
+ // 52-char row budget intact (see `fixed` below) while fitting real
715
+ // values: "minimal" (effort), "minimal 100.0" (alt), "10.00/100.00".
716
+ const qualW = colW(data.entries.map(entry => entry.quality), "qual".length, 5);
717
+ // Catalog v5: the effort the quality was measured at and the best
718
+ // selectable measured alternative ("max 56.0"). A missing value is "-".
719
+ const effortW = colW(data.entries.map(entry => entry.effort || "-"), "effort".length, 7);
720
+ const altW = colW(data.entries.map(entry => entry.alternative || "-"), "alt".length, 13);
721
+ const priceW = colW(data.entries.map(entry => entry.price), "$ in/out".length, 13);
713
722
  // A cell wider than its clamped column is truncated to fit.
714
723
  const cell = (value, width) => value.length > width ? value.slice(0, width - 1) + "…" : value;
715
724
  // Prices use padEnd (not padStart): padded to a common width the
@@ -717,10 +726,13 @@ export default Plugin.define({
717
726
  // stays exactly 2 (padStart pushed each value right by its slack).
718
727
  // The default dialog is 60 wide: 2 list insets + 6 row insets leave 52
719
728
  // for row content; the header shares that budget (plus its 3-indent).
720
- const fixed = 2 + qualW + 2 + priceW;
721
- const nameW = Math.max(8, Math.min(52 - fixed, Math.max(...labels.map(label => label.length))));
729
+ // Column caps keep the budget even for hostile strings: 5+7+13+13 plus
730
+ // the four 2-char paddings and the 2-char row indent = 46, leaving the
731
+ // name floor of 6 (the name column shrinks first for real content).
732
+ const fixed = 2 + qualW + 2 + effortW + 2 + altW + 2 + priceW;
733
+ const nameW = Math.max(6, Math.min(52 - fixed, Math.max(...labels.map(label => label.length))));
722
734
  const trim = label => label.length > nameW ? label.slice(0, nameW - 1) + "…" : label;
723
- const headerLine = " " + " ".repeat(nameW) + " " + "qual".padStart(qualW) + " " + "$ in/out".padEnd(priceW);
735
+ const headerLine = " " + " ".repeat(nameW) + " " + "qual".padStart(qualW) + " " + "effort".padStart(effortW) + " " + "alt".padEnd(altW) + " " + "$ in/out".padEnd(priceW);
724
736
  const rows = [{
725
737
  label: "",
726
738
  headerLine
@@ -735,7 +747,7 @@ export default Plugin.define({
735
747
  for (const entry of section) {
736
748
  rows.push({
737
749
  label: trim(labelOf.get(entry) ?? "").padEnd(nameW),
738
- columns: " " + cell(entry.quality, qualW).padStart(qualW) + " " + cell(entry.price, priceW).padEnd(priceW),
750
+ columns: " " + cell(entry.quality, qualW).padStart(qualW) + " " + cell(entry.effort || "-", effortW).padStart(effortW) + " " + cell(entry.alternative || "-", altW).padEnd(altW) + " " + cell(entry.price, priceW).padEnd(priceW),
739
751
  // F-20 id split: the row DISPLAYS the sanitized id (in `label`)
740
752
  // but picks `value`, the raw catalog id switchToModel needs.
741
753
  value: entry.value,
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "model-compare-opencode-plugin",
3
- "version": "0.4.0",
3
+ "version": "0.6.0",
4
4
  "description": "OpenCode plugin that switches OpenRouter model aliases to the best value-for-money pick from canonical/model-compare's ranking (balanced, price, quality).",
5
5
  "keywords": [
6
6
  "opencode",
@@ -44,16 +44,17 @@
44
44
  "prepublishOnly": "npm run build && npm run typecheck && npm test && node scripts/pack-smoke.mjs"
45
45
  },
46
46
  "dependencies": {
47
- "@opencode/plugin": "2.0.16"
47
+ "@opencode/plugin": "2.1.4"
48
48
  },
49
49
  "peerDependencies": {
50
- "@opentui/core": "^0.5.8",
51
- "@opentui/solid": "^0.5.8",
50
+ "@opentui/core": "^0.6.8",
51
+ "@opentui/solid": "^0.6.8",
52
52
  "solid-js": "^1.9.0"
53
53
  },
54
54
  "devDependencies": {
55
- "@opentui/core": "0.5.17",
56
- "@opentui/solid": "0.5.17",
55
+ "@opencode/theme": "2.1.4",
56
+ "@opentui/core": "0.6.8",
57
+ "@opentui/solid": "0.6.8",
57
58
  "@types/node": "^22.18.13",
58
59
  "solid-js": "1.9.12",
59
60
  "typescript": "^7.0.2",
@@ -1,9 +1,16 @@
1
1
  // src/core/catalog.ts
2
- // Catalog document validation (design §5). Accepts schema v4 documents (only) from
2
+ // Catalog document validation (design §5). Accepts schema v5 documents (only) from
3
3
  // either source (vendored script or remote catalog) and projects them onto
4
4
  // the plugin's Catalog type. Validation only — ranking lives elsewhere.
5
5
 
6
- import type { Catalog, ModelEntry, Priority, Weights } from "./types"
6
+ import type {
7
+ Catalog,
8
+ ModelEntry,
9
+ Priority,
10
+ Reasoning,
11
+ ReasoningDepth,
12
+ Weights,
13
+ } from "./types"
7
14
 
8
15
  const PRIORITIES: readonly Priority[] = ["balanced", "price", "quality"]
9
16
  const SCORE_KEYS = ["quality", "price", "context", "age"] as const
@@ -44,18 +51,67 @@ function steer(raw: unknown): ModelEntry["steering"] {
44
51
  return steering
45
52
  }
46
53
 
54
+ function finiteOrNull(value: unknown): number | null {
55
+ return typeof value === "number" && Number.isFinite(value) ? value : null
56
+ }
57
+
58
+ function stringOrNull(value: unknown): string | null {
59
+ return typeof value === "string" ? value : null
60
+ }
61
+
62
+ function boolOrNull(value: unknown): boolean | null {
63
+ return typeof value === "boolean" ? value : null
64
+ }
65
+
66
+ // The v5 reasoning block (display + request injection). Field-by-field
67
+ // projection, never a spread of the raw object: a hostile catalogURL serves
68
+ // arbitrary JSON here, and only the fields the plugin reads survive. A
69
+ // non-object block reads as absent; a malformed field reads as its neutral
70
+ // value (empty list / null) rather than rejecting the whole catalog — the
71
+ // block is advisory metadata, not ranking input. Depth records without a
72
+ // non-empty string effort are dropped; every other depth field is nullable.
73
+ function projectReasoning(raw: unknown): Reasoning | undefined {
74
+ if (!isObject(raw)) return undefined
75
+ const supported =
76
+ Array.isArray(raw.supported_efforts) && raw.supported_efforts.every((effort) => typeof effort === "string")
77
+ ? [...(raw.supported_efforts as string[])]
78
+ : []
79
+ const depths: ReasoningDepth[] = []
80
+ if (Array.isArray(raw.depths)) {
81
+ for (const record of raw.depths) {
82
+ if (!isObject(record)) continue
83
+ if (typeof record.effort !== "string" || record.effort === "") continue
84
+ depths.push({
85
+ effort: record.effort,
86
+ index: finiteOrNull(record.index),
87
+ cost_per_task_usd: finiteOrNull(record.cost_per_task_usd),
88
+ source: stringOrNull(record.source),
89
+ as_of: stringOrNull(record.as_of),
90
+ })
91
+ }
92
+ }
93
+ return {
94
+ default_effort: stringOrNull(raw.default_effort),
95
+ supported_efforts: supported,
96
+ default_enabled: boolOrNull(raw.default_enabled),
97
+ mandatory: boolOrNull(raw.mandatory),
98
+ supports_max_tokens: boolOrNull(raw.supports_max_tokens),
99
+ depths,
100
+ }
101
+ }
102
+
47
103
  export function validateCatalog(
48
104
  raw: unknown,
49
105
  ): { ok: true; catalog: Catalog } | { ok: false; reason: string } {
50
106
  if (!isObject(raw)) return { ok: false, reason: "catalog must be a JSON object" }
51
107
 
52
- // Supported schema: v4 only — upstream's current contract (0.5.0's
53
- // schema_version 4). The v4 changes are additive provenance (per-model
54
- // quality_as_of/quality_drift, a reshaped sources.aa block) and change
55
- // nothing this validator reads; every other version is rejected with the
56
- // reason string the canary and the stored-document path rely on.
57
- if (raw.schema_version !== 4) {
58
- return { ok: false, reason: `schema_version must be 4, got ${JSON.stringify(raw.schema_version)}` }
108
+ // Supported schema: v5 only — upstream's current contract (0.6.0's
109
+ // schema_version 5). v5 changes the meaning of `quality` (the AA score at
110
+ // the selected/default reasoning effort) and adds per-model
111
+ // `quality_effort` and `reasoning`; every other version is rejected with
112
+ // the reason string the canary and the stored-document path rely on.
113
+ if (raw.schema_version !== 5) {
114
+ return { ok: false, reason: `schema_version must be 5, got ${JSON.stringify(raw.schema_version)}` }
59
115
  }
60
116
  if (typeof raw.generated_at !== "string") {
61
117
  return { ok: false, reason: "generated_at must be a string" }
@@ -160,12 +216,17 @@ export function validateCatalog(
160
216
  const input = pricing.input_per_1m
161
217
  const output = pricing.output_per_1m
162
218
  // The steering block projects (or reads as absent) per the steer() rules.
163
- // The assignment sits AFTER the spread so it explicitly overrides the raw
164
- // block that `...entry` would otherwise carry through verbatim — a
165
- // hostile steering block must never survive onto the projected entry.
219
+ // The v5 quality_effort/reasoning fields project the same way: strip the
220
+ // raw keys before the spread so the camelCase projection is the only
221
+ // representation on the entry (a hostile block never survives verbatim).
222
+ const { steering: _rawSteering, quality_effort: _rawQualityEffort, reasoning: _rawReasoning, ...passthrough } =
223
+ entry
166
224
  const steering = steer(entry.steering)
225
+ const qualityEffort =
226
+ typeof entry.quality_effort === "string" && entry.quality_effort !== "" ? entry.quality_effort : null
227
+ const reasoning = projectReasoning(entry.reasoning)
167
228
  models.push({
168
- ...entry,
229
+ ...passthrough,
169
230
  id: entry.id,
170
231
  name: entry.name,
171
232
  pricing: {
@@ -177,6 +238,8 @@ export function validateCatalog(
177
238
  context: entry.context,
178
239
  scores: { ...components, overall },
179
240
  quality,
241
+ qualityEffort,
242
+ reasoning,
180
243
  steering,
181
244
  })
182
245
  }
@@ -220,14 +283,33 @@ export function validateStoredCatalog(
220
283
  stored: unknown,
221
284
  ): { ok: true; catalog: Catalog } | { ok: false; reason: string } {
222
285
  if (!isObject(stored)) return { ok: false, reason: "stored catalog must be an object" }
286
+ // v5 projections always carry qualityEffort (null or a string). A record
287
+ // written by a pre-v5 plugin lacks it: accepting it would mix v4's max-row
288
+ // quality with v5's default-effort semantics, so it is a cache miss.
289
+ if (Array.isArray(stored.models)) {
290
+ for (const [i, entry] of stored.models.entries()) {
291
+ if (isObject(entry) && !Object.hasOwn(entry, "qualityEffort")) {
292
+ return { ok: false, reason: `models[${i}].qualityEffort is missing (not a v5 catalog)` }
293
+ }
294
+ }
295
+ }
223
296
  const models = Array.isArray(stored.models)
224
297
  ? stored.models.map((entry) => {
225
- if (!isObject(entry) || entry.steering === undefined) return entry
226
- return { ...entry, steering: { model: entry.id, provider: entry.steering } }
298
+ if (!isObject(entry)) return entry
299
+ // The projection renamed quality_effort to qualityEffort and unwrapped
300
+ // steering; rebuild the raw v5 shape. A hand-edited record re-projects
301
+ // (or reads as absent) exactly like a raw one would.
302
+ const raw: Record<string, unknown> = {
303
+ ...entry,
304
+ quality_effort: entry.qualityEffort ?? null,
305
+ }
306
+ if (entry.reasoning !== undefined) raw.reasoning = entry.reasoning
307
+ if (entry.steering !== undefined) raw.steering = { model: entry.id, provider: entry.steering }
308
+ return raw
227
309
  })
228
310
  : stored.models
229
311
  return validateCatalog({
230
- schema_version: 4,
312
+ schema_version: 5,
231
313
  generated_at: stored.generatedAt,
232
314
  parameters: { weights: stored.weights },
233
315
  models,
@@ -0,0 +1,96 @@
1
+ // src/core/reasoning.ts
2
+ // Reasoning-depth helpers (catalog v5): gateway-effort validation, the
3
+ // selectable-depth rule, and the best measured alternative to the effort a
4
+ // model was scored at. Pure functions over a projected ModelEntry — no I/O,
5
+ // no OpenCode imports.
6
+ //
7
+ // The selectable rule is upstream's site rule (web/site/app.js depthRows):
8
+ // OpenRouter can select an effort only when the model's expanded
9
+ // `supported_efforts` lists it (or the model is a single-state toggle that
10
+ // exposes `#none`). An AA row for an unlisted effort is display-only and is
11
+ // never advertised as an alternative. The catalog's `reasoning` sentinel is a
12
+ // producer label for a single-state reasoning model, not a gateway value.
13
+
14
+ import type { ModelEntry, Reasoning } from "./types"
15
+
16
+ // What OpenRouter accepts as a discrete effort. Mirrors upstream's
17
+ // REASONING_GATEWAY_EFFORTS. Deliberately excludes the catalog sentinels
18
+ // "reasoning" and "default".
19
+ export const GATEWAY_EFFORTS = ["max", "xhigh", "high", "medium", "low", "minimal", "none"] as const
20
+
21
+ // Deepest-first display order, mirroring upstream's AA_EFFORT_ORDER.
22
+ const EFFORT_ORDER: readonly string[] = [...GATEWAY_EFFORTS, "reasoning"]
23
+
24
+ export function gatewayEffort(value: string | null | undefined): string | null {
25
+ return value !== null && value !== undefined && (GATEWAY_EFFORTS as readonly string[]).includes(value)
26
+ ? value
27
+ : null
28
+ }
29
+
30
+ function effortOrder(effort: string): number {
31
+ const index = EFFORT_ORDER.indexOf(effort)
32
+ return index === -1 ? EFFORT_ORDER.length : index
33
+ }
34
+
35
+ // Upstream's v5 non-reasoning rule (the producer's is_reasoning flag is not
36
+ // published): no reasoning metadata at all, so the bare id is the only form.
37
+ function isNonReasoningBlock(reasoning: Reasoning): boolean {
38
+ return (
39
+ reasoning.default_effort === "none" &&
40
+ reasoning.default_enabled !== false &&
41
+ reasoning.mandatory !== true &&
42
+ reasoning.supports_max_tokens !== true
43
+ )
44
+ }
45
+
46
+ // A single-state reasoning model with reasoning off by default exposes only
47
+ // the "#none" toggle.
48
+ function isSingleStateToggle(reasoning: Reasoning): boolean {
49
+ return reasoning.mandatory !== true && reasoning.default_enabled === false
50
+ }
51
+
52
+ export function isSelectableDepth(model: ModelEntry, effort: string): boolean {
53
+ if (effort === "reasoning") return false
54
+ const reasoning = model.reasoning
55
+ if (reasoning === undefined) return false
56
+ if (reasoning.supported_efforts.includes(effort)) return true
57
+ if (effort !== "none" || reasoning.supported_efforts.length > 0) return false
58
+ if (isNonReasoningBlock(reasoning)) return false
59
+ return isSingleStateToggle(reasoning)
60
+ }
61
+
62
+ export type DepthAlternative = {
63
+ effort: string
64
+ index: number
65
+ cost: number | null
66
+ }
67
+
68
+ // The best measured, selectable effort that scores strictly higher than the
69
+ // effort `quality` already represents. Unselectable rows never qualify; ties
70
+ // resolve to the deeper effort (the display order).
71
+ export function bestDepthAlternative(model: ModelEntry): DepthAlternative | null {
72
+ const base = model.quality
73
+ const selected = model.qualityEffort
74
+ const reasoning = model.reasoning
75
+ if (base == null || selected == null || reasoning === undefined) return null
76
+
77
+ const depths = [...reasoning.depths].sort((a, b) => effortOrder(a.effort) - effortOrder(b.effort))
78
+ let best: DepthAlternative | null = null
79
+ for (const depth of depths) {
80
+ if (depth.index === null || depth.effort === selected) continue
81
+ if (!isSelectableDepth(model, depth.effort)) continue
82
+ if (depth.index <= base) continue
83
+ if (best === null || depth.index > best.index) {
84
+ best = { effort: depth.effort, index: depth.index, cost: depth.cost_per_task_usd }
85
+ }
86
+ }
87
+ return best
88
+ }
89
+
90
+ // The report's compact alternative label: "<effort> <index>" ("max 56.0"),
91
+ // the same shape upstream's site shows under QUAL. Empty when there is none.
92
+ export function formatDepthAlternative(model: ModelEntry | undefined): string {
93
+ if (model === undefined) return ""
94
+ const best = bestDepthAlternative(model)
95
+ return best === null ? "" : `${best.effort} ${best.index.toFixed(1)}`
96
+ }
@@ -12,6 +12,7 @@
12
12
  // and the steeringFor cases here); a semantic change to one must be mirrored
13
13
  // in the other.
14
14
 
15
+ import { gatewayEffort } from "./reasoning"
15
16
  import type { ModelEntry, Priority } from "./types"
16
17
 
17
18
  // A model reference as the hook event carries it (`event.model`), mirroring
@@ -75,6 +76,30 @@ export function steeringFor(
75
76
  return models.get(pick.id)?.steering ?? null
76
77
  }
77
78
 
79
+ // The reasoning effort an alias request must pin, or null when it must not:
80
+ // the same alias/body/pick/model gates as steeringFor, plus the explicit
81
+ // variant rule. `#variant` is OpenCode's per-request selection (the event
82
+ // carries it on `model.variant`); "default" is the no-selection sentinel
83
+ // OpenCode stamps on `--model` refs (spike fact 1), so it does NOT count as an
84
+ // explicit variant. The catalog's `quality_effort` is the effort the score was
85
+ // measured at; the "reasoning" sentinel is a producer label and is never sent
86
+ // to the gateway.
87
+ export function reasoningEffortFor(
88
+ eventModel: ModelRef,
89
+ variant: string | undefined,
90
+ bodyModel: string | undefined,
91
+ picks: Partial<Record<Priority, AliasTarget>>,
92
+ models: Map<string, ModelEntry>,
93
+ ): string | null {
94
+ const p = aliasPriority(eventModel)
95
+ if (p === null) return null
96
+ if (variant !== undefined && variant !== "default") return null
97
+ const pick = picks[p]
98
+ if (pick === undefined || pick.id === null) return null
99
+ if (bodyModel !== undefined && bodyModel !== pick.id) return null
100
+ return gatewayEffort(models.get(pick.id)?.qualityEffort)
101
+ }
102
+
78
103
  function isPlainObject(value: unknown): value is Record<string, unknown> {
79
104
  return typeof value === "object" && value !== null && !Array.isArray(value)
80
105
  }
@@ -103,6 +128,17 @@ export function injectSteering(body: unknown, steering: Steering): unknown {
103
128
  }
104
129
  }
105
130
 
131
+ // The body with `reasoning: { effort }` added — or the input untouched when
132
+ // it already carries `reasoning` or the legacy `reasoning_effort` field (an
133
+ // explicit #variant, user config or another plugin wins; the plugin never
134
+ // clobbers a reasoning choice it did not make) or is not a JSON object.
135
+ // Reference inequality is the wiring's "re-create the Request" signal.
136
+ export function injectReasoning(body: unknown, effort: string): unknown {
137
+ if (!isPlainObject(body)) return body
138
+ if (Object.hasOwn(body, "reasoning") || Object.hasOwn(body, "reasoning_effort")) return body
139
+ return { ...body, reasoning: { effort } }
140
+ }
141
+
106
142
  // normalize_provider (vendor/model_compare.py): casefold the name and strip
107
143
  // ALL non-alphanumeric characters — "Z.AI", "z-ai" and "z ai" all normalize
108
144
  // to "zai". JS toLowerCase stands in for Python's casefold (provider display
package/src/core/types.ts CHANGED
@@ -6,6 +6,31 @@ export type Priority = "balanced" | "price" | "quality"
6
6
 
7
7
  export type Weights = { quality: number; price: number; context: number; age: number }
8
8
 
9
+ // One measured reasoning effort (catalog v5): the AA intelligence index at
10
+ // that effort, AA's reference-price benchmark cost per task (display-only,
11
+ // never scored), and the record's provenance. `null` marks a missing value;
12
+ // the record itself survives so the effort can still be offered as a variant.
13
+ export type ReasoningDepth = {
14
+ effort: string
15
+ index: number | null
16
+ cost_per_task_usd: number | null
17
+ source: string | null
18
+ as_of: string | null
19
+ }
20
+
21
+ // The v5 per-model reasoning block. `supported_efforts` is what OpenRouter
22
+ // accepts as a discrete effort (expanded upstream, never null); `depths` is
23
+ // what some source measured (empty when nothing did). The raw flags are kept
24
+ // for provenance and for the single-state/non-reasoning display rules.
25
+ export type Reasoning = {
26
+ default_effort: string | null
27
+ supported_efforts: string[]
28
+ default_enabled: boolean | null
29
+ mandatory: boolean | null
30
+ supports_max_tokens: boolean | null
31
+ depths: ReasoningDepth[]
32
+ }
33
+
9
34
  export type ModelEntry = {
10
35
  id: string
11
36
  name: string
@@ -20,6 +45,14 @@ export type ModelEntry = {
20
45
  }
21
46
  context: number
22
47
  quality: number | null
48
+ // The effort `quality` was measured at (catalog v5 `quality_effort`). The
49
+ // alias request pins exactly this effort; the sentinel "reasoning" is a
50
+ // catalog label, never a gateway value. Optional only for hand-built /
51
+ // legacy projected entries: validateCatalog always writes it (null or a
52
+ // string), and validateStoredCatalog requires it on cached records.
53
+ qualityEffort?: string | null
54
+ // Present when the raw block projects cleanly (absent stays absent).
55
+ reasoning?: Reasoning
23
56
  // The steering block (F-11/F-12): upstream records the provider's
24
57
  // endpoint-routing hints as { model, provider: { order, ignore,
25
58
  // quantizations?, allow_fallbacks } }; the projection unwraps the provider
@@ -47,7 +80,7 @@ export type Catalog = {
47
80
  models: ModelEntry[]
48
81
  }
49
82
 
50
- // Upstream.s PRIORITY_WEIGHTS at the pinned tag (v0.5.0; unchanged since v0.3.0). Used when a
83
+ // Upstream's PRIORITY_WEIGHTS at the pinned tag (v0.6.0; unchanged since v0.3.0). Used when a
51
84
  // document carries no `parameters.weights` at all (design §5); individual
52
85
  // missing keys fall back to weight 0 instead.
53
86
  export const DEFAULT_WEIGHTS: Record<Priority, Weights> = {
package/src/index.ts CHANGED
@@ -19,7 +19,13 @@ import { parseConfig } from "./config"
19
19
  import { pinKey, type Pin } from "./core/pins"
20
20
  import { effectiveWeights, rankModels } from "./core/rank"
21
21
  import type { Catalog, EndpointCapture, ModelEntry, Priority, Sourced } from "./core/types"
22
- import { captureEndpoint, injectSteering, steeringFor } from "./core/steering"
22
+ import {
23
+ captureEndpoint,
24
+ injectReasoning,
25
+ injectSteering,
26
+ reasoningEffortFor,
27
+ steeringFor,
28
+ } from "./core/steering"
23
29
  import { formatReport, humanizeAge, reportEntries, steeringNote } from "./opencode/commands"
24
30
  import { ModelCompare } from "./rpc"
25
31
  import { planAlias, type AliasTarget } from "./opencode/aliases"
@@ -329,7 +335,11 @@ function applyAliases(state: WireState, editor: AliasEditor): void {
329
335
  // plan: planAlias only checks for undefined (it never reads model
330
336
  // internals — the applier does, below).
331
337
  const source = target.id === null ? undefined : editor.get(ALIAS_PROVIDER, target.id)
332
- const plan = planAlias(p, target, source as unknown as ModelEntry | undefined)
338
+ // The concrete editor model proves presence; the catalog entry supplies
339
+ // the v5 effort/alternative labels (the editor model has no ranking data).
340
+ const catalogModel =
341
+ target.id === null ? undefined : state.sourced?.doc.models.find((entry) => entry.id === target.id)
342
+ const plan = planAlias(p, target, source as unknown as ModelEntry | undefined, catalogModel)
333
343
 
334
344
  if (plan.enabled && plan.modelID !== null && source !== undefined) {
335
345
  editor.update(ALIAS_PROVIDER, aliasId(p), (model) => {
@@ -558,21 +568,36 @@ export async function wire(ctx: PluginCtx, deps: WireDeps = {}): Promise<() => v
558
568
  state.appliedSteering[event.sessionID] = null
559
569
  return
560
570
  }
561
- const steering = steeringFor(
571
+ const byId = new Map(
572
+ state.sourced.doc.models.map((entry): [string, ModelEntry] => [entry.id, entry]),
573
+ )
574
+ const bodyModel = bodyModelOf(body)
575
+ const steering = steeringFor(event.model, bodyModel, state.picks, byId)
576
+ // Two independent injections share one re-created Request: the
577
+ // provider-routing block and the catalog's scored reasoning effort.
578
+ // Each helper preserves a value the body already sets, so an explicit
579
+ // #variant (either signalled by the event's variant or by the body's
580
+ // own reasoning field) wins without withholding the other block.
581
+ let modified: unknown = body
582
+ let providerApplied = false
583
+ if (steering !== null) {
584
+ const next = injectSteering(modified, steering)
585
+ if (next !== modified) {
586
+ modified = next
587
+ providerApplied = true
588
+ }
589
+ }
590
+ const effort = reasoningEffortFor(
562
591
  event.model,
563
- bodyModelOf(body),
592
+ event.model.variant,
593
+ bodyModel,
564
594
  state.picks,
565
- new Map(state.sourced.doc.models.map((entry): [string, ModelEntry] => [entry.id, entry])),
595
+ byId,
566
596
  )
567
- if (steering === null) {
568
- // No steering data for the pick: untouched, fail open.
569
- state.appliedSteering[event.sessionID] = null
570
- return
571
- }
572
- const modified = injectSteering(body, steering)
597
+ if (effort !== null) modified = injectReasoning(modified, effort)
573
598
  if (modified === body) {
574
- // The body already carries a `provider` key (or is not an object):
575
- // never clobber routing the user or a provider configured.
599
+ // Neither block had anything to add: no steering data, no scored
600
+ // effort, or the body already carries the corresponding field.
576
601
  state.appliedSteering[event.sessionID] = null
577
602
  return
578
603
  }
@@ -592,7 +617,9 @@ export async function wire(ctx: PluginCtx, deps: WireDeps = {}): Promise<() => v
592
617
  })
593
618
  // Record the order actually applied — the response hook reads it as
594
619
  // captureEndpoint's order (the `!` flag fires only on a non-empty one).
595
- state.appliedSteering[event.sessionID] = steering.order
620
+ // A reasoning-only injection (or a body that already carried routing)
621
+ // applies no routing, so the record is null.
622
+ state.appliedSteering[event.sessionID] = providerApplied ? steering!.order : null
596
623
  } catch {
597
624
  // Fail open: any error leaves the original request proceeding
598
625
  // untouched, and records that nothing was injected.