model-compare-opencode-plugin 0.1.3 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -77,6 +77,7 @@ Commands:
77
77
  | Key | Action |
78
78
  |---|---|
79
79
  | `alt+m` | picks dialog — all three priorities with their current picks |
80
+ | `alt+e` | toast the last response's served endpoint, alias responses only (`!` prefix flags a downgrade — not in the steering order) |
80
81
  | `alt+shift+m` | report dialog — the top-5 table; `return` switches the session to the highlighted model |
81
82
  | `up` / `down` | move the dialog cursor |
82
83
  | `return` | select the highlighted row (picks dialog: switch to that priority) |
@@ -84,7 +85,7 @@ Commands:
84
85
 
85
86
  Both dialogs open anywhere — including the opening screen, where no session exists yet. There, picking a row (a priority in the picks dialog, a model in the report) creates a new session already bound to that model and opens it; merely opening a dialog never creates a session. With a session open, a pick switches that session as before. The TUI has no `/use-best` keybinding: the slash command already covers switching.
86
87
 
87
- Status: the footer shows `MC: balanced → <pick>` (balanced only — `disabled`/`unavailable`/`(fallback)` as applicable; all three priorities live in the picks dialog). Toasts announce `/use-best` outcomes and pick changes — never for the first observation after load. Headless, all feedback is synthetic transcript messages at the next turn.
88
+ Status: the footer shows `MC: balanced → <pick>` (balanced only — `disabled`/`unavailable`/`(fallback)` as applicable; all three priorities live in the picks dialog). Toasts announce `/use-best` outcomes and pick changes (never for the first observation after load), the `alt+e` served endpoint, and the cannot-steer notice — see [Steering and served endpoints](#steering-and-served-endpoints). Headless, command feedback is synthetic transcript messages at the next turn; the steering toasts have no headless surface.
88
89
 
89
90
 
90
91
  ## Configuration
@@ -99,7 +100,7 @@ Options live under `plugins[].options`:
99
100
  | `weights` | upstream | Partial overrides per named priority (`quality`, `price`, `context`, `age`), merged with upstream's weights and renormalized to sum 1 |
100
101
  | `pinDays` | `14` | Age in days after which a pick is re-evaluated |
101
102
  | `minLead` | `0.03` | Score lead needed to replace a pinned pick |
102
- | `catalogURL` | published catalog | Remote source URL (`https://canonical.github.io/model-compare/catalog.json`). Only point it at a source you trust: it decides which models can be picked |
103
+ | `catalogURL` | published catalog | Remote source URL. The catalog is published at `https://model-compare.dev/catalog.json` — the plugin's built-in default, fetched directly over https. (The old `https://canonical.github.io/model-compare/catalog.json` home redirects to a plaintext `http://` target; the plugin fails closed on any redirect that leaves https.) Only point it at a source you trust: it decides which models can be picked |
103
104
  | `scriptPath` | vendored copy | Development override for a local `model_compare.py` |
104
105
 
105
106
  A present-but-invalid value logs a warning and applies the default (`weights` drop invalid entries individually; the valid remainder applies). Absent options apply defaults silently. Setup never throws; the aliases stay functional.
@@ -117,6 +118,35 @@ Per priority the plugin keeps one pin/last-known-good record in plugin storage,
117
118
  Resolution order: pin (if runnable) → best runnable candidate → `fallback` (if runnable) → the alias is disabled. The fallback is never stored as a pin. Pins are advisory: concurrent OpenCode processes refresh independently, last writer wins.
118
119
 
119
120
 
121
+ ## Steering and served endpoints
122
+
123
+ Requests made through one of the three aliases carry the resolved pick's routing: the catalog's per-model `steering` block is injected into the request body under OpenRouter's `provider` key, so the call is routed onto the endpoints whose pricing the ranking actually ranked — not onto whatever OpenRouter's default routing picks, which can be a sub-default-quantization endpoint at a price the ranking never saw.
124
+
125
+ The block comes from the catalog, whichever source produced it: the vendored script derives it from the same per-model endpoint data its pricing uses, and the published catalog carries it ready-made. Only the block's `provider` sub-object is injected — its top-level `model` field never reaches the body — and the routing object is built field by field:
126
+
127
+ - `order` — the endpoints the ranking priced, cheapest blended price first; eligible endpoints whose price can't be computed are appended last rather than dropped.
128
+ - `ignore` — the live endpoints the ranking excluded, so OpenRouter cannot route onto pricing it never ranked (sub-default quantizations, schedule-gate failures, variants).
129
+ - `quantizations` — the precision floor: the model's default precision and anything more precise among its live endpoints. The key is omitted entirely when the model has no default precision — never sent as an empty list, which would exclude every endpoint.
130
+ - `allow_fallbacks` — always `true`: when nothing in `order` can serve, OpenRouter may still route outside the list.
131
+
132
+ The rules, and the exceptions:
133
+
134
+ - **Aliases only.** Steering fires only when the session's model is one of the three aliases and the request's model is that alias's current pick. What decides is the request's model reference: alias references steer, concrete references never do — a model you picked directly is untouched even when it is the exact model an alias currently resolves to. Primary turns and compaction alike are steered; auxiliary calls that run a different model (title generation) are not.
135
+ - **A pre-existing `provider` key wins.** If the request body already carries a `provider` key — routing you or a provider integration configured — the plugin does not inject; it never clobbers routing it did not choose.
136
+ - **No steering data, no injection.** When the plugin has no catalog source, or the current pick's catalog entry carries no `steering` block, the request proceeds untouched: OpenRouter's default routing applies and the run completes normally — unsteered, not failed. With no catalog source at all, the TUI says so once per session, on the first alias request that has a pick to steer: `cannot steer this session; OpenRouter default routing applies` (skipped when that request already carries its own `provider` — then default routing is not what applies). Headless runs render no toasts; the routing behavior is identical. A disabled alias steers nothing and toasts nothing; what serves then is the binding's own fallback behavior (see [Troubleshooting](#troubleshooting)).
137
+ - **Always-on.** No option turns steering off. It is what makes the plugin's value promise — the ranked price is what you pay — true.
138
+
139
+ `order` is a preference list, not a pinned destination: OpenRouter works down the list and serves from whichever ranked endpoint can take the call — which one serves is OpenRouter's decision, and with fallbacks allowed it can even land outside the list.
140
+
141
+ **Which endpoint served a call.** Three ways to check:
142
+
143
+ - The `alt+e` toast (see [Keyboard shortcuts](#keyboard-shortcuts)) — the current session's last primary alias response's served endpoint: `last response served by <endpoint>`, or `! served by <endpoint> — not in steering order` when the served provider matched no entry of the steering order actually applied. The report dialog's footer carries the same capture as `last response: <model> @ <endpoint>`, under the note `routing steered to <quant> endpoints (aliases only)`.
144
+ - The response body — every OpenRouter response names its serving endpoint in the `provider` field.
145
+ - OpenRouter's dashboard — per-request endpoint records.
146
+
147
+ The alt+e capture is best-effort by design, and alias-scoped: only a session whose model is one of the three aliases has responses peeked and recorded — a direct pick's or another agent's response is never read, let alone modified. The plugin peeks at only the first 4 KB of the response body — a pass-through read; the body itself flows byte-for-byte. Streaming responses carry `provider` in the first chunk, so the peek normally catches them; a non-streaming response whose `provider` field sits beyond the peek window is forwarded uncaptured, and `alt+e` then toasts nothing. The response body's `provider` field and OpenRouter's dashboard remain the authoritative verification paths.
148
+
149
+
120
150
  ## Data sources
121
151
 
122
152
  - `auto` — vendored `vendor/model_compare.py` run with `python3` (no shell, 60 s timeout, minimal environment); degrades to remote when python3 is missing or the run fails.
@@ -125,16 +155,18 @@ Resolution order: pin (if runnable) → best runnable candidate → `fallback` (
125
155
 
126
156
  The last validated catalog is persisted, so ranking and `/model-compare` work offline after a restart. The cache is keyed by source (URL or script path), so changing the source never serves the previous source's data. Script exit code 2 means "no candidates" *or* an argparse usage error — never read it as "no candidates" alone.
127
157
 
128
- **Vendored pin:** `vendor/model_compare.py` is byte-identical to upstream `v0.3.1` (`vendor/UPSTREAM.json` records repo, tag, commit, sha256). Since upstream 0.3.0 the catalog is schema v2: `pricing` scalars hold effective-tier values — the tier in effect at 1M-token prompts — so models with OpenRouter price overrides are ranked at what a long session actually pays.
158
+ **Vendored pin:** `vendor/model_compare.py` is byte-identical to upstream `v0.4.0` (`vendor/UPSTREAM.json` records repo, tag, commit, sha256). Since upstream 0.4.0 the catalog is schema v3: `pricing` carries tier and schedule structure alongside the flat scalars, and `parameters` carries the published ranking weights. The plugin accepts schema v3 ONLY — `validateCatalog` rejects anything else — and releases are gated on the live remote serving v3 (the release-gate check pipes `https://model-compare.dev/catalog.json` through `scripts/canary-check.mjs`, which fails closed when the served document is not the expected schema). Effective-tier pricing semantics still apply: `pricing` scalars hold effective-tier values — the tier in effect at 1M-token prompts — so models with OpenRouter price overrides are ranked at what a long session actually pays.
159
+
160
+ **Vendored 0.4.0 cold-cache network profile:** since 0.4.0 the vendored script performs per-model endpoint-pricing fetches when building a fresh catalog (71 in the recorded run). Each fetch fails closed per model — after the script's internal retries, a failed endpoint fetch drops that model and the catalog shrinks silently (429-class responses raise without retry) — so a degraded network costs pool coverage, not pricing correctness. A cold cache under a slow network can exceed the plugin's 60-second local budget, in which case the plugin warns and falls back (`auto` → remote, `local` → its cached document).
129
161
 
130
162
  **Upgrading the pin** — reviewed PRs only; nothing merges automatically:
131
163
 
132
164
  1. `node scripts/sync-vendor.mjs <tag>` — fetches the script at that commit, reports the sha256 change, re-pins `UPSTREAM.json`, prints the matching CHANGELOG section (`--check` verifies offline).
133
- 2. Conformance tests check `--version`, schema v2, required fields, recomputed `overall` scores, and the standard-library-only import allowlist.
165
+ 2. Conformance tests check `--version`, schema v3, required fields, recomputed `overall` scores, and the standard-library-only import allowlist.
134
166
  3. The upgrade PR body lists script lines touching imports, URLs, environment variables, subprocess calls and file writes, for reviewer attention.
135
167
  4. Release with a changelog line naming the upstream tag; users run `opencode plugin update` (OpenCode checks but doesn't apply).
136
168
 
137
- A scheduled workflow opens a draft upgrade PR when upstream publishes a newer release (never a downgrade); a nightly canary exercises the script live; a weekly drift check compares the vendored sha256 against upstream's pinned commit and the published rankings against the published scores.
169
+ A scheduled workflow opens a draft upgrade PR when upstream publishes a newer release (never a downgrade); a nightly canary exercises the vendored script against live upstream data and asserts the generated catalog is schema v3 (`scripts/canary-check.mjs` pins `EXPECTED_SCHEMA_VERSION = 3` and exits nonzero on any other version); releases additionally gate on the live remote itself (see the publishing gates below); a weekly drift check compares the vendored sha256 against upstream's pinned commit and the published rankings against the published scores.
138
170
 
139
171
 
140
172
  ## Troubleshooting
@@ -177,8 +209,8 @@ MC_INTEGRATION=1 npx vitest run tests/integration/ # live-server suite (opt-i
177
209
  - Local development loads `src/` directly and needs no build; the *published* package does — OpenCode's Solid JSX transform skips `node_modules` paths, so the npm-installed raw TSX cannot load (see [Install](#install) and the troubleshooting entry). `npm run build` precompiles `src/tui.tsx` through the same Babel pipeline as the host's loader (taken from the pinned `@opentui/solid`) into `dist/tui.js`, rebasing relative imports from `src/`-relative to `dist/`-relative; `prepublishOnly` runs it, and CI builds before the packaging check. `dist/` is regenerated, never committed.
178
210
  - The integration files skip unless `MC_INTEGRATION=1`, so plain `npm test` stays hermetic. The suite drives a real `opencode serve` on 127.0.0.1:14721 and needs the `opencode` CLI and `python3` (skips cleanly when either is missing); one case needs network access and the `sqlite3` CLI.
179
211
  - `node scripts/pack-smoke.mjs` (also in CI, after `node scripts/build-tui.mjs`) packs the plugin and checks the tarball against the packaging list (files, `exports`, relative imports, and that the TUI artifact is precompiled — no react import). It does **not** load the package in OpenCode — install the tarball into a scratch project and confirm OpenCode loads it before publishing.
180
- - Publishing: cut a GitHub release tagged `vX.Y.Z` and `.github/workflows/release.yaml` stages the package on npm through trusted publishing (tokenless OIDC — no token secret; the npmjs.com trusted-publisher entry must name `release.yaml`, the `canonical/model-compare-opencode-plugin` repository and the `release` environment, with "Allow npm publish" and "Allow npm dist-tag" unchecked — stage-only, npm's recommended posture). Nothing goes live until a maintainer approves it with 2FA on npmjs.com (package page → Staged Packages) or via `npm stage approve`. The workflow fails closed when the tag does not match `package.json`'s version, and the gates (build, typecheck, tests, pack-smoke) run before staging.
181
- - `fixtures/catalog-v0.3.0.json` is a recorded schema-v2 catalog; `fixtures/catalog-no-quality.json` is generated from it by `scripts/make-no-quality-fixture.py` (the vendored script's own quality-blind weights, so its scores are internally consistent). `node scripts/sync-vendor.mjs --check` verifies the pin's sha256 offline.
212
+ - Publishing: cut a GitHub release tagged `vX.Y.Z` and `.github/workflows/release.yaml` stages the package on npm through trusted publishing (tokenless OIDC — no token secret; the npmjs.com trusted-publisher entry must name `release.yaml`, the `canonical/model-compare-opencode-plugin` repository and the `release` environment, with "Allow npm publish" and "Allow npm dist-tag" unchecked — stage-only, npm's recommended posture). Nothing goes live until a maintainer approves it with 2FA on npmjs.com (package page → Staged Packages) or via `npm stage approve`. The workflow fails closed when the tag does not match `package.json`'s version, and the gates (build, typecheck, tests, pack-smoke, live-remote canary) run before staging.
213
+ - `fixtures/catalog-v0.4.0.json` is the sole contract fixture — a recorded schema-v3 catalog; `fixtures/catalog-no-quality.json` is derived from it by `scripts/make-no-quality-fixture.py` (the vendored script's own quality-blind weights, so its scores are internally consistent). No v2 fixture exists; the v3-only gate rejects one. `node scripts/sync-vendor.mjs --check` verifies the pin's sha256 offline.
182
214
  - Repo layout: `src/core/` (pure ranking/pinning), `src/sources/` (local + remote acquisition), `src/opencode/` (aliases, guard, commands, storage, refresh), `dist/` (generated `tui.js` artifact, gitignored), `vendor/` (pinned upstream script), `tests/{unit,conformance,integration}`.
183
215
 
184
216
 
package/dist/tui.js CHANGED
@@ -57,7 +57,7 @@ import { TextAttributes, RGBA } from "@opentui/core";
57
57
 
58
58
  import { createEffect, createMemo, createRoot, createSignal } from "solid-js";
59
59
  import { ALIAS_PROVIDER, PRIORITIES, alignedPriority, aliasId, pickOf, pickFromName, readAliasViews, statusLine, useBestDecision, useBestNotice, useBestRefusal } from "../src/opencode/usebest";
60
- import { sanitizeCatalogText } from "../src/opencode/commands";
60
+ import { lastResponseLine, sanitizeCatalogText } from "../src/opencode/commands";
61
61
  import { ModelCompare } from "../src/rpc";
62
62
  const TOAST_MS = 6_000;
63
63
  const RESYNC_MS = 30_000;
@@ -69,6 +69,11 @@ const PACKAGE_FRAGMENT = "model-compare-opencode-plugin";
69
69
  // The RPC event the /model-compare command emits (see src/rpc.ts; the wire
70
70
  // name is prefixed with the RPC's id).
71
71
  const REPORT_EVENT = "rpc.model-compare.report";
72
+ // The event the runtime hook emits once per session when it would have
73
+ // steered a request but has no steering data (no catalog source) — the Q2
74
+ // fallback's runtime notification. The toast copy below is the whole message.
75
+ const CANNOT_STEER_EVENT = "rpc.model-compare.cannotSteer";
76
+ const CANNOT_STEER_COPY = "cannot steer this session; OpenRouter default routing applies";
72
77
 
73
78
  // The list dialogs' lifecycle token (see showListDialog): a module-scope
74
79
  // generation counter, bumped every time a dialog is armed, plus the armed
@@ -158,6 +163,30 @@ function toReportEntries(raw) {
158
163
  }];
159
164
  });
160
165
  }
166
+
167
+ // The report payload's footer capture (the last-response line and the alt+e
168
+ // toast's data), defensively normalized at the trust boundary like the
169
+ // entries: anything but a well-shaped object — or one without an endpoint —
170
+ // degrades to null (no capture), never a placeholder line.
171
+ function toCapture(raw) {
172
+ if (typeof raw !== "object" || raw === null) return null;
173
+ const capture = raw;
174
+ if (typeof capture.endpoint !== "string" || capture.endpoint === "") return null;
175
+ return {
176
+ endpoint: capture.endpoint,
177
+ // The Task-5 handoff: a capture may carry an empty model (a response
178
+ // whose provider had no model id) — carried as-is, and the footer omits
179
+ // the model part rather than rendering an empty slot.
180
+ model: typeof capture.model === "string" ? capture.model : "",
181
+ downgrade: capture.downgrade === true
182
+ };
183
+ }
184
+
185
+ // The footer's steering note, defensively normalized: anything but a
186
+ // non-empty string means the note does not render (no steering data).
187
+ function toSteeringNote(raw) {
188
+ return typeof raw === "string" && raw !== "" ? raw : null;
189
+ }
161
190
  export default Plugin.define({
162
191
  id: "model-compare-tui",
163
192
  async setup(context) {
@@ -347,7 +376,11 @@ export default Plugin.define({
347
376
  // dialog is open, cleared on close; the app slot's in-tree effect maps
348
377
  // it to a modal layer.
349
378
  const [listKeys, setListKeys] = createSignal(null);
350
- function showListDialog(title, rows, onPick) {
379
+ function showListDialog(title, rows, onPick,
380
+ // The footer's extra lines (the report dialog's steering note and
381
+ // last-response line), rendered muted ABOVE the Details line. Absent
382
+ // for the other dialogs (the picks dialog carries no footer lines).
383
+ footerLines = []) {
351
384
  const selectable = rows.filter(row => row.value !== undefined);
352
385
  if (selectable.length === 0) {
353
386
  notify("nothing to select", "error");
@@ -594,9 +627,18 @@ export default Plugin.define({
594
627
  })();
595
628
  });
596
629
  });
630
+ _$insert(_el$2, () => footerLines.map(line => (() => {
631
+ var _el$18 = _$createElement("box"),
632
+ _el$19 = _$createElement("text");
633
+ _$insertNode(_el$18, _el$19);
634
+ _$setProp(_el$18, "paddingLeft", 4);
635
+ _$setProp(_el$19, "fg", muted);
636
+ _$insert(_el$19, line);
637
+ return _el$18;
638
+ })()), _el$8);
597
639
  _$insertNode(_el$8, _el$9);
598
640
  _$setProp(_el$8, "paddingLeft", 4);
599
- _$insertNode(_el$9, _$createTextNode(`Details: https://canonical.github.io/model-compare`));
641
+ _$insertNode(_el$9, _$createTextNode(`Details: https://model-compare.dev`));
600
642
  _$setProp(_el$9, "fg", muted);
601
643
  _$effect(_p$ => {
602
644
  var _v$ = TextAttributes.BOLD,
@@ -701,9 +743,17 @@ export default Plugin.define({
701
743
  });
702
744
  }
703
745
  }
746
+ // The report dialog's footer (design §Decisions, option C): the
747
+ // steering note ABOVE the last-response line, both omitted when their
748
+ // data is absent — never a placeholder line.
749
+ const footerLines = [];
750
+ if (data.steeringNote !== null) footerLines.push(data.steeringNote);
751
+ const lastResponse = lastResponseLine(data.capture);
752
+ if (lastResponse !== null) footerLines.push(lastResponse);
704
753
  showListDialog(`Model Compare — top 5 (data age: ${data.age ?? "unknown"})`, rows, id => {
705
- if (data.sessionID !== null) {
706
- void switchToModel(data.sessionID, id);
754
+ const sessionID = data.sessionID;
755
+ if (sessionID !== null) {
756
+ void switchToModel(sessionID, id);
707
757
  return;
708
758
  }
709
759
  // The opening screen: the pick becomes a NEW session's model —
@@ -712,7 +762,7 @@ export default Plugin.define({
712
762
  providerID: ALIAS_PROVIDER,
713
763
  id
714
764
  }, `switched to openrouter/${sanitizeCatalogText(id)}`);
715
- });
765
+ }, footerLines);
716
766
  }
717
767
 
718
768
  // The report dialog rides the /model-compare command's RPC event: ONE
@@ -732,13 +782,38 @@ export default Plugin.define({
732
782
  const entries = toReportEntries(data.entries);
733
783
  const eventSessionID = data.sessionID;
734
784
  // Ownerless callback: park the report on the signal — the app slot's
735
- // in-tree effect opens the dialog (see pendingReport above).
785
+ // in-tree effect opens the dialog (see pendingReport above). The
786
+ // footer's fields ride the payload like the entries (defensively
787
+ // normalized at this trust boundary).
736
788
  setPendingReport({
737
789
  sessionID: eventSessionID,
738
790
  age: typeof data.age === "string" ? data.age : null,
739
- entries
791
+ entries,
792
+ capture: toCapture(data.capture),
793
+ steeringNote: toSteeringNote(data.steeringNote)
740
794
  });
741
795
  });
796
+
797
+ // The cannot-steer fallback (Task 6): the runtime hook emits this event
798
+ // once per session when a steerable alias request finds no steering data
799
+ // (no catalog source). The TUI surfaces it as a toast with the exact
800
+ // fallback copy; headless shows nothing — like every TUI-coupled surface.
801
+ // Scoped exactly like the report listener: a foreign directory's session
802
+ // is never viewable here, and only the CURRENTLY viewed session toasts.
803
+ // Trade-off: a cannotSteer notice for a same-project background session
804
+ // (or a session the user isn't currently viewing) is dropped forever —
805
+ // the emit is once-per-session — but noise from unopened sessions is the
806
+ // worse failure; the notice is about the session the user interacts with.
807
+ const offCannotSteerEvent = context.data.listen(({
808
+ details
809
+ }) => {
810
+ const event = details;
811
+ if (event.type !== CANNOT_STEER_EVENT) return;
812
+ if (typeof event.location?.directory === "string" && here !== undefined && event.location.directory !== here) return;
813
+ const data = event.data;
814
+ if (typeof data?.sessionID !== "string" || data.sessionID !== currentSessionID()) return;
815
+ notify(CANNOT_STEER_COPY);
816
+ });
742
817
  createRoot(dispose => {
743
818
  disposeRoot = dispose;
744
819
 
@@ -867,7 +942,9 @@ export default Plugin.define({
867
942
  showReportDialog({
868
943
  sessionID: null,
869
944
  age: typeof data?.age === "string" ? data.age : null,
870
- entries: toReportEntries(data?.entries)
945
+ entries: toReportEntries(data?.entries),
946
+ capture: toCapture(data?.capture),
947
+ steeringNote: toSteeringNote(data?.steeringNote)
871
948
  });
872
949
  } catch (error) {
873
950
  notify(`report failed: ${error instanceof Error ? error.message : String(error)}`, "error");
@@ -875,6 +952,41 @@ export default Plugin.define({
875
952
  }
876
953
  };
877
954
 
955
+ // The served-endpoint toast (design §Steering visibility): alt+e toasts
956
+ // the CURRENT session's last response's served endpoint, looked up
957
+ // through the report RPC's footer fields (the capture record rides the
958
+ // established report payload path). The copy is exact — normal:
959
+ // "last response served by <endpoint>"; downgrade (the served provider
960
+ // matched none of the applied steering order): "! served by <endpoint> —
961
+ // not in steering order"; nothing toasts when nothing was captured for
962
+ // the session (and with no session open there is nothing to look up).
963
+ const endpointCommand = {
964
+ id: "model-compare.endpoint",
965
+ title: "Last served endpoint",
966
+ group: "Model Compare",
967
+ bind: "alt+e",
968
+ palette: true,
969
+ run: async () => {
970
+ const sessionID = currentSessionID();
971
+ if (sessionID === null) return;
972
+ try {
973
+ const report = await context.client.rpc(ModelCompare).report({
974
+ sessionID
975
+ }, here === undefined ? undefined : {
976
+ location: {
977
+ directory: here
978
+ }
979
+ });
980
+ const capture = toCapture(report?.capture);
981
+ if (capture === null) return;
982
+ // F-20: the endpoint is response data — flattened for the toast.
983
+ notify(capture.downgrade ? `! served by ${sanitizeCatalogText(capture.endpoint)} — not in steering order` : `last response served by ${sanitizeCatalogText(capture.endpoint)}`);
984
+ } catch (error) {
985
+ notify(`endpoint lookup failed: ${error instanceof Error ? error.message : String(error)}`, "error");
986
+ }
987
+ }
988
+ };
989
+
878
990
  // The palette-only use-best: the immediate-toast variant. The SLASH
879
991
  // surface is left to the server command (one entry everywhere); the
880
992
  // palette is TUI-layer territory, where this variant's instant feedback
@@ -950,7 +1062,7 @@ export default Plugin.define({
950
1062
  try {
951
1063
  context.keymap.layer(() => ({
952
1064
  mode: "global",
953
- commands: [useBestCommand, picksCommand, reportCommand]
1065
+ commands: [useBestCommand, picksCommand, reportCommand, endpointCommand]
954
1066
  }));
955
1067
  } catch (error) {
956
1068
  layerFailure("the model-compare keymap layer", error);
@@ -967,6 +1079,7 @@ export default Plugin.define({
967
1079
  if (resyncTimer !== undefined) clearInterval(resyncTimer);
968
1080
  context.renderer.off("resize", onRendererResize);
969
1081
  offReportEvent();
1082
+ offCannotSteerEvent();
970
1083
  disposeRoot();
971
1084
  disposeAppSlot();
972
1085
  disposeHomeFooter();
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "model-compare-opencode-plugin",
3
- "version": "0.1.3",
3
+ "version": "0.3.0",
4
4
  "description": "OpenCode plugin that switches OpenRouter model aliases to the best value-for-money pick from canonical/model-compare's ranking (balanced, price, quality).",
5
5
  "keywords": [
6
6
  "opencode",
package/src/config.ts CHANGED
@@ -21,7 +21,7 @@ export type Config = {
21
21
  weights: Partial<Record<Priority, Partial<Weights>>>
22
22
  pinDays: number // default 14
23
23
  minLead: number // default 0.03
24
- catalogURL: string // default "https://canonical.github.io/model-compare/catalog.json"
24
+ catalogURL: string // default "https://model-compare.dev/catalog.json"
25
25
  scriptPath: string | null // null → vendored copy
26
26
  }
27
27
 
@@ -29,7 +29,10 @@ const SOURCES = ["auto", "local", "remote"] as const
29
29
  const PRIORITIES = ["balanced", "price", "quality"] as const
30
30
  const WEIGHT_KEYS = ["quality", "price", "context", "age"] as const
31
31
 
32
- const DEFAULT_CATALOG_URL = "https://canonical.github.io/model-compare/catalog.json"
32
+ // The published catalog's direct https URL — no redirect involved (the old
33
+ // canonical.github.io home redirected to a plaintext http target, which
34
+ // fetchRemote now refuses; see Ruling 9).
35
+ const DEFAULT_CATALOG_URL = "https://model-compare.dev/catalog.json"
33
36
  const FALLBACK_PREFIX = "openrouter/"
34
37
 
35
38
  // True for plain config objects. Arrays are excluded: their index keys are
@@ -1,5 +1,5 @@
1
1
  // src/core/catalog.ts
2
- // Catalog document validation (design §5). Accepts schema v2 documents (only) from
2
+ // Catalog document validation (design §5). Accepts schema v3 documents (only) from
3
3
  // either source (vendored script or remote catalog) and projects them onto
4
4
  // the plugin's Catalog type. Validation only — ranking lives elsewhere.
5
5
 
@@ -12,19 +12,48 @@ function isObject(value: unknown): value is Record<string, unknown> {
12
12
  return typeof value === "object" && value !== null && !Array.isArray(value)
13
13
  }
14
14
 
15
+ function isNonEmptyStringList(value: unknown): value is string[] {
16
+ return Array.isArray(value) && value.length > 0 && value.every((id) => typeof id === "string")
17
+ }
18
+
19
+ // The steering block (F-11/F-12): the raw upstream entry carries
20
+ // { model, provider: { order, ignore, quantizations?, allow_fallbacks } }.
21
+ // The projection unwraps `provider` as ModelEntry.steering and never projects
22
+ // the top-level `model`. Per-field absence (the pricing-split posture), no
23
+ // throw, no partial field: order/ignore must be non-empty string arrays (an
24
+ // empty list would filter out every endpoint) and allow_fallbacks a boolean —
25
+ // any miss drops the whole block; quantizations is optional and kept only as
26
+ // a non-empty string array, otherwise that field alone is dropped and the
27
+ // block survives.
28
+ function steer(raw: unknown): ModelEntry["steering"] {
29
+ if (!isObject(raw)) return undefined
30
+ const provider = raw.provider
31
+ if (!isObject(provider)) return undefined
32
+ const order = provider.order
33
+ const ignore = provider.ignore
34
+ const fallbacks = provider.allow_fallbacks
35
+ if (!isNonEmptyStringList(order) || !isNonEmptyStringList(ignore) || typeof fallbacks !== "boolean") {
36
+ return undefined
37
+ }
38
+ const steering: NonNullable<ModelEntry["steering"]> = {
39
+ order: [...order],
40
+ ignore: [...ignore],
41
+ allow_fallbacks: fallbacks,
42
+ }
43
+ if (isNonEmptyStringList(provider.quantizations)) steering.quantizations = [...provider.quantizations]
44
+ return steering
45
+ }
46
+
15
47
  export function validateCatalog(
16
48
  raw: unknown,
17
49
  ): { ok: true; catalog: Catalog } | { ok: false; reason: string } {
18
50
  if (!isObject(raw)) return { ok: false, reason: "catalog must be a JSON object" }
19
51
 
20
- // Supported schema: v2 only — upstream's current contract (0.3.0: pricing
21
- // scalars are effective-tier values; pricing.base/tiers/schedule and a
22
- // top-level schedules index were added — ignored here, see the roadmap
23
- // for consumer ideas). The plugin has never been released, so no stored
24
- // v1 documents exist anywhere; the published catalog has served v2 since
25
- // 2026-10-10.
26
- if (raw.schema_version !== 2) {
27
- return { ok: false, reason: `schema_version must be 2, got ${JSON.stringify(raw.schema_version)}` }
52
+ // Supported schema: v3 only — upstream's current contract (0.4.0's
53
+ // schema_version 3). Every other version is rejected with the reason string
54
+ // the canary and the stored-document path rely on.
55
+ if (raw.schema_version !== 3) {
56
+ return { ok: false, reason: `schema_version must be 3, got ${JSON.stringify(raw.schema_version)}` }
28
57
  }
29
58
  if (typeof raw.generated_at !== "string") {
30
59
  return { ok: false, reason: "generated_at must be a string" }
@@ -128,6 +157,11 @@ export function validateCatalog(
128
157
  // stays absent, present-but-invalid behaves as absent).
129
158
  const input = pricing.input_per_1m
130
159
  const output = pricing.output_per_1m
160
+ // The steering block projects (or reads as absent) per the steer() rules.
161
+ // The assignment sits AFTER the spread so it explicitly overrides the raw
162
+ // block that `...entry` would otherwise carry through verbatim — a
163
+ // hostile steering block must never survive onto the projected entry.
164
+ const steering = steer(entry.steering)
131
165
  models.push({
132
166
  ...entry,
133
167
  id: entry.id,
@@ -141,6 +175,7 @@ export function validateCatalog(
141
175
  context: entry.context,
142
176
  scores: { ...components, overall },
143
177
  quality,
178
+ steering,
144
179
  })
145
180
  }
146
181
 
@@ -174,16 +209,26 @@ export function validateCatalog(
174
209
  // `Catalog`, not the raw upstream document) by rebuilding the raw shape and
175
210
  // running it through validateCatalog: a corrupt or hand-edited record is
176
211
  // rejected instead of reaching ranking code. The round trip is lossless for a
177
- // catalog validateCatalog produced.
212
+ // catalog validateCatalog produced. A model carrying a projected steering
213
+ // block is re-wrapped into the raw { model, provider } shape first — the
214
+ // top-level `model` is never projected, so entry.id stands in for it (the
215
+ // projection ignores it either way). A hand-edited steering block re-projects
216
+ // or reads as absent, exactly like a raw one would.
178
217
  export function validateStoredCatalog(
179
218
  stored: unknown,
180
219
  ): { ok: true; catalog: Catalog } | { ok: false; reason: string } {
181
220
  if (!isObject(stored)) return { ok: false, reason: "stored catalog must be an object" }
221
+ const models = Array.isArray(stored.models)
222
+ ? stored.models.map((entry) => {
223
+ if (!isObject(entry) || entry.steering === undefined) return entry
224
+ return { ...entry, steering: { model: entry.id, provider: entry.steering } }
225
+ })
226
+ : stored.models
182
227
  return validateCatalog({
183
- schema_version: 2,
228
+ schema_version: 3,
184
229
  generated_at: stored.generatedAt,
185
230
  parameters: { weights: stored.weights },
186
- models: stored.models,
231
+ models,
187
232
  rankings: stored.rankings,
188
233
  })
189
234
  }
@@ -0,0 +1,145 @@
1
+ // src/core/steering.ts
2
+ // The steering decision core (design §Steering injection + served-endpoint
3
+ // visibility): whether a request may carry the catalog's steering block, the
4
+ // injection shape, and the served-endpoint membership verdict. Pure functions
5
+ // over the data the wiring hooks supply — no I/O, no OpenCode imports.
6
+ //
7
+ // Layering note: the guard module (src/opencode/guard.ts) owns the canonical
8
+ // alias-ref parse, but core modules never import from src/opencode (this
9
+ // repo's rule: only opencode → core). The identification below mirrors the
10
+ // guard's parseAliasRef exactly — variant-blind, exact-priority remainder —
11
+ // and both copies are pinned to the same vectors (tests/unit/guard.test.ts
12
+ // and the steeringFor cases here); a semantic change to one must be mirrored
13
+ // in the other.
14
+
15
+ import type { ModelEntry, Priority } from "./types"
16
+
17
+ // A model reference as the hook event carries it (`event.model`), mirroring
18
+ // the guard's ModelRef (src/opencode/guard.ts) structurally — the wiring
19
+ // passes the event's ref straight through, and any `variant` key the event
20
+ // adds is ignored (spike fact 1: the alias ref's variant is not stable across
21
+ // binding styles — absent for root-default/frontmatter, "default" for
22
+ // `--model` — so identification must stay variant-blind).
23
+ export type ModelRef = { providerID: string; id: string }
24
+
25
+ // The resolved pick for one priority, mirroring the alias registry's
26
+ // AliasTarget (src/opencode/aliases.ts) structurally for the same layering
27
+ // reason: the wiring passes its plan's picks directly.
28
+ export type AliasTarget = { id: string | null; fromFallback: boolean }
29
+
30
+ // The steering block exactly as ModelEntry.steering projects it (Task 2):
31
+ // OpenRouter's provider routing sub-object. The raw upstream block's
32
+ // top-level `model` field is never part of it.
33
+ export type Steering = NonNullable<ModelEntry["steering"]>
34
+
35
+ const ALIAS_PREFIX = "model-compare/"
36
+
37
+ const PRIORITIES: readonly Priority[] = ["balanced", "price", "quality"]
38
+
39
+ // Alias identification over the event's model ref — the guard's parseAliasRef
40
+ // rule mirrored (see the layering note above): ONLY openrouter plus
41
+ // `model-compare/<p>` whose remainder is exactly a priority name; ids may
42
+ // carry more slashes in OpenRouter's vendor/model style, so anything after
43
+ // the priority disqualifies. Accepts undefined for runtime robustness (the
44
+ // caller's ref is runtime data).
45
+ function aliasPriority(ref: ModelRef | undefined): Priority | null {
46
+ if (ref === undefined || ref.providerID !== "openrouter") return null
47
+ if (!ref.id.startsWith(ALIAS_PREFIX)) return null
48
+ const remainder = ref.id.slice(ALIAS_PREFIX.length)
49
+ return PRIORITIES.includes(remainder as Priority) ? (remainder as Priority) : null
50
+ }
51
+
52
+ // The steering block to inject for a request, or null when the request must
53
+ // not be steered: a non-alias ref (including a DIRECTLY-PICKED model — even
54
+ // when its id equals the current pick's; matching on that equality would
55
+ // INVERT the rule, since an alias-bound ref is never equal to a concrete pick
56
+ // id while a directly-picked one is — and steer requests outside the aliases,
57
+ // the anti-goal), a pick that resolved to nothing, a body model that is not
58
+ // the pick's id, or a pick whose catalog entry carries no steering block.
59
+ //
60
+ // `bodyModel` is the request body's `model` field when the wiring has read it
61
+ // (the live body carries the pick id verbatim — spike fact 1); undefined
62
+ // skips that gate and lets the alias signal decide. The `models` map is keyed
63
+ // by bare catalog id — the pick ids' form.
64
+ export function steeringFor(
65
+ eventModel: ModelRef,
66
+ bodyModel: string | undefined,
67
+ picks: Partial<Record<Priority, AliasTarget>>,
68
+ models: Map<string, ModelEntry>,
69
+ ): Steering | null {
70
+ const p = aliasPriority(eventModel)
71
+ if (p === null) return null
72
+ const pick = picks[p]
73
+ if (pick === undefined || pick.id === null) return null
74
+ if (bodyModel !== undefined && bodyModel !== pick.id) return null
75
+ return models.get(pick.id)?.steering ?? null
76
+ }
77
+
78
+ function isPlainObject(value: unknown): value is Record<string, unknown> {
79
+ return typeof value === "object" && value !== null && !Array.isArray(value)
80
+ }
81
+
82
+ // The body with the steering block's provider sub-object injected under the
83
+ // `provider` key — or the input returned untouched when it already carries a
84
+ // `provider` key (the plugin never clobbers routing the user or a provider
85
+ // configured) or is not a JSON object. The injected routing object is built
86
+ // field-by-field, never spread from the steering block, so the raw block's
87
+ // top-level `model` field (or any unknown key) cannot reach the body; an
88
+ // absent `quantizations` stays absent (OpenRouter reads the key's presence).
89
+ //
90
+ // Contract the wiring relies on: the SAME reference comes back when nothing
91
+ // was injected, a NEW object when it was — reference inequality is the
92
+ // "re-create the Request" signal.
93
+ export function injectSteering(body: unknown, steering: Steering): unknown {
94
+ if (!isPlainObject(body) || Object.hasOwn(body, "provider")) return body
95
+ return {
96
+ ...body,
97
+ provider: {
98
+ order: steering.order,
99
+ ignore: steering.ignore,
100
+ ...(steering.quantizations === undefined ? {} : { quantizations: steering.quantizations }),
101
+ allow_fallbacks: steering.allow_fallbacks,
102
+ },
103
+ }
104
+ }
105
+
106
+ // normalize_provider (vendor/model_compare.py): casefold the name and strip
107
+ // ALL non-alphanumeric characters — "Z.AI", "z-ai" and "z ai" all normalize
108
+ // to "zai". JS toLowerCase stands in for Python's casefold (provider display
109
+ // names are ASCII in practice; a non-ASCII residue the casefold would keep is
110
+ // stripped by the character class either way).
111
+ function normalizeProvider(name: string): string {
112
+ return name.toLowerCase().replace(/[^a-z0-9]+/g, "")
113
+ }
114
+
115
+ // The provider part of a routing reference: the text before the first `/`
116
+ // (`novita/fp8`, `deepinfra/fp8/flex` → `novita`/`deepinfra`), or the whole
117
+ // string when it carries none — a served display name (`Novita`) is therefore
118
+ // its own provider part. Applied to BOTH sides symmetrically (spike fact 3:
119
+ // the served string is a display name, never the script's tagged-slug form).
120
+ function providerPart(ref: string): string {
121
+ const slash = ref.indexOf("/")
122
+ return slash === -1 ? ref : ref.slice(0, slash)
123
+ }
124
+
125
+ // The served-endpoint verdict for one response: compare the response's
126
+ // `provider` string against the steering `order` that was actually applied to
127
+ // the request. `endpoint` is the served string verbatim (the alt+e toast's
128
+ // text) — null only when there is nothing to record (no provider at all, or a
129
+ // name with no provider part). `downgrade` is true ONLY when a parseable
130
+ // provider matches no entry of a NON-EMPTY order; an absent or empty order
131
+ // (no steering data — offline, or a model without a steering block) records
132
+ // the endpoint with no verdict, because `!` may only assert a fallback
133
+ // "despite the steering" when a non-empty order was actually applied.
134
+ export function captureEndpoint(
135
+ provider: string | null,
136
+ order: string[] | null,
137
+ ): { endpoint: string | null; downgrade: boolean } {
138
+ if (provider === null) return { endpoint: null, downgrade: false }
139
+ const served = normalizeProvider(providerPart(provider))
140
+ if (served === "") return { endpoint: null, downgrade: false }
141
+ const entries = order ?? []
142
+ if (entries.length === 0) return { endpoint: provider, downgrade: false }
143
+ const matched = entries.some((entry) => normalizeProvider(providerPart(entry)) === served)
144
+ return { endpoint: provider, downgrade: !matched }
145
+ }