omp-llm-role 1.2.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +45 -5
- package/docs/dev/architecture.md +11 -8
- package/docs/dev/ci.md +9 -0
- package/docs/dev/current-state.md +47 -12
- package/docs/dev/explorer.md +106 -19
- package/docs/dev/releasing.md +33 -3
- package/docs/dev/scoring.md +55 -4
- package/docs/dev/testing.md +43 -2
- package/docs/dev/writes-and-state.md +30 -0
- package/docs/llm-role-rankings.md +115 -115
- package/package.json +11 -1
- package/src/agent-create.ts +4 -2
- package/src/engine.ts +33 -2
- package/src/explorer/boot.ts +43 -1
- package/src/explorer/explain.ts +90 -5
- package/src/explorer/levers.ts +100 -0
- package/src/explorer/pool.ts +46 -0
- package/src/explorer/server.ts +92 -21
- package/src/extension.ts +132 -2
- package/src/settings.ts +31 -0
- package/src/updater.ts +6 -1
- package/src/usage-ledger.ts +127 -0
- package/web/app.js +132 -0
- package/web/style.css +3 -0
package/docs/dev/testing.md
CHANGED
|
@@ -137,11 +137,18 @@ passing.
|
|
|
137
137
|
`filters.preferredMinThroughput: 50` (the registry row and the expanded def
|
|
138
138
|
agree), an explicit `filters.preferredMinThroughput` beats the preset, and
|
|
139
139
|
the schema carries the `roles.<role>.filters.preferredMinThroughput` row
|
|
140
|
-
(number, min 0, default 0).
|
|
140
|
+
(number, min 0, default 0). The `tokenProfile` setting (spec #81) defaults to
|
|
141
|
+
`null`, accepts a record with finite `inputTokens`/`outputTokens` ≥ 0, and
|
|
142
|
+
rejects a non-record or a negative/non-finite field; the schema carries the
|
|
143
|
+
`tokenProfile.inputTokens`/`tokenProfile.outputTokens` rows (number, min 0).
|
|
141
144
|
23. **Explorer** (`explorer.test.ts`, `explorer-scopes.test.ts`) — rank deltas;
|
|
142
145
|
contributions sum to `q`; `inverseCardinal` round-trips; Export preserves
|
|
143
146
|
siblings and normalizes nested roles; the scope switcher; the availability
|
|
144
|
-
overlay.
|
|
147
|
+
overlay. The per-task price (spec #81): `rankRows`/`explainModel` fill
|
|
148
|
+
`perTask` from a supplied profile and leave it `null` without one; the
|
|
149
|
+
`POST /api/pool` toggle returns the gated/pruned dataset in `as-written` mode
|
|
150
|
+
and the full one in `raw`; the bootstrap payload carries `poolMode` and
|
|
151
|
+
`tokenProfile`; the explain response carries `leaderFlip`/`thinkingLevels`.
|
|
145
152
|
24. **Project scope** (`project-updater.test.ts`, `project-setup.test.ts`,
|
|
146
153
|
`project-roles-command.test.ts`) — project mode scopes config/state/history/
|
|
147
154
|
lock to `<cwd>/.omp`; the day gate is per scope; the registry write;
|
|
@@ -224,6 +231,28 @@ passing.
|
|
|
224
231
|
from the live model on `turn_start` (the silent-restore case), strips when
|
|
225
232
|
neither source carries a pin, never calls `pi.setModel`, and contains a
|
|
226
233
|
throwing live-model read.
|
|
234
|
+
33. **Usage ledger** (`usage-ledger.test.ts`, `usage-ledger-wiring.test.ts`) —
|
|
235
|
+
`appendUsageRow` writes one row with the expected fields; `readUsageRows`
|
|
236
|
+
degrades to `[]` on a missing/malformed file and skips an unparseable or
|
|
237
|
+
wrongly-shaped line; the cap holds (the newest `USAGE_LEDGER_CAP` rows
|
|
238
|
+
survive a compaction); `roleTokenProfile` is the role's median per-turn
|
|
239
|
+
counts (or `null` with no rows) and `resolveTokenProfile` prefers the
|
|
240
|
+
recorded usage over the setting. The wiring (a fake `pi`/`ctx` with a
|
|
241
|
+
`sessionManager.getBranch()`) records one row per assistant turn on
|
|
242
|
+
`turn_end` and `session_shutdown`, dedupes by `ts|model` so the two hooks
|
|
243
|
+
cannot double-record, writes into `resolveProjectDir() ?? agentDir()`, and
|
|
244
|
+
contains a throwing branch read.
|
|
245
|
+
34. **Explorer price levers** (`explorer-levers.test.ts`) — `leaderFlip` returns
|
|
246
|
+
the min-λ threshold over the strictly cheaper challengers, the current
|
|
247
|
+
`wPrice`, the distance and the challenger; `null` when no cheaper challenger
|
|
248
|
+
exists; the degenerate case (value leader = quality leader) is handled by the
|
|
249
|
+
same min formula, not `0/0`. `thinkingLevelEffect` returns one row per level
|
|
250
|
+
with the engine's own factor and the leader at that level, and `null` for a
|
|
251
|
+
bare role.
|
|
252
|
+
35. **As-written pool** (`explorer-pool.test.ts`) — `asWrittenPool` applies the
|
|
253
|
+
catalog/tier gate (`resolveVariant`) and the allowed-providers prune
|
|
254
|
+
(`providerAllowed`), leaves the routes untouched on a `null` whitelist,
|
|
255
|
+
yields an empty pool on `tier: "none"`, and never mutates its input.
|
|
227
256
|
|
|
228
257
|
## Live checks
|
|
229
258
|
|
|
@@ -244,6 +273,18 @@ passing.
|
|
|
244
273
|
derived metadata.
|
|
245
274
|
- **Agent removal** — `/remove-agent --name <n>` in a session deletes both
|
|
246
275
|
artifacts and the updater pass removes the config entries.
|
|
276
|
+
- **Explorer pool / per-task / levers** — `/explore-roles` in a session with a
|
|
277
|
+
recorded usage ledger: confirm the header's pool toggle switches the table
|
|
278
|
+
between the raw and as-written pools (the model count and the leader change),
|
|
279
|
+
the `$/task` column renders a value (or `—` with no profile), and the explain
|
|
280
|
+
panel shows the per-task breakdown and the **Price levers** section (the
|
|
281
|
+
leader-flip line and the per-level leader table). Verify in headless Chromium
|
|
282
|
+
(the SPA is not unit-tested).
|
|
283
|
+
- **Usage ledger** — run a real omp turn, then confirm `llm-role-usage.jsonl`
|
|
284
|
+
gained one row per assistant turn (the model, the role, the provider, the
|
|
285
|
+
token counts, the cost) in the plugin's state dir (`<project>/.omp` in project
|
|
286
|
+
mode, `agentDir()` otherwise), and that the explorer's `$/task` column then
|
|
287
|
+
fills for that role. A session with no completed turn writes no rows.
|
|
247
288
|
- **CI/CD** — push a branch and open a PR (or push to `main`) and `gh run watch`
|
|
248
289
|
the `test` job green; push a tag that disagrees with the package version and
|
|
249
290
|
confirm the `release` job fails in the `notes` guard *before* any publish;
|
|
@@ -134,6 +134,17 @@ gating — that this plugin never reads (zero occurrences in `src/`, `web/`,
|
|
|
134
134
|
`tests/`). The plugin's `features.<id>` keys are plugin settings, not omp
|
|
135
135
|
feature flags; do not merge the two.
|
|
136
136
|
|
|
137
|
+
**`tokenProfile` (spec #81).** A global setting (`tokenProfile.inputTokens` /
|
|
138
|
+
`tokenProfile.outputTokens`, flat dotted keys like the rest) — the per-task
|
|
139
|
+
token profile the explorer's `$/task` column falls back to when a role has no
|
|
140
|
+
recorded usage. `null` (the default) = unset, and the column reads `—`. It is
|
|
141
|
+
validated to `null` or a record whose two fields are finite numbers ≥ 0
|
|
142
|
+
(`tokenProfile: must be null or an object with inputTokens/outputTokens` /
|
|
143
|
+
`tokenProfile.<key>: must be a number >= 0`). It is **display-only**: it never
|
|
144
|
+
feeds the ranking, the reported `$/M` price, or any write. The recorded usage
|
|
145
|
+
takes precedence over it (`resolveTokenProfile`), so once a role has ledger rows
|
|
146
|
+
the setting is ignored for that role.
|
|
147
|
+
|
|
137
148
|
## The provider-pin auto-configuration (spec #58)
|
|
138
149
|
|
|
139
150
|
When a role's **expanded** def sets `preferOwnProvider: true` (the
|
|
@@ -209,6 +220,7 @@ settings entry the global behavior is unchanged.
|
|
|
209
220
|
|---|---|---|
|
|
210
221
|
| `llm-role-state.json` | `{ lastRunDay, managedRoles, roleLastSelector, pluginWrittenChainKeys, previousModelRoles, managedDisabledAgents }` | day gate, managed-role set, last selectors, plugin-owned chain keys, pre-write `modelRoles` snapshot, **managed `task.disabledAgents` names** |
|
|
211
222
|
| `llm-role-history.jsonl` | one JSON row per completed run: `{ ts, trigger, keyMeta{isFreeTier, limitRemaining, creditsRemaining}, decisions[] }` | append-only decision log |
|
|
223
|
+
| `llm-role-usage.jsonl` | one JSON row per assistant turn: `{ ts, role, model, provider, input, output, cacheRead, cacheWrite, cost }` | the per-role usage ledger (spec #81) the explorer's per-task price resolves its token profile from |
|
|
212
224
|
| `llm-role-projects.json` | `{ projects: [{ root, lastUsed }] }` | the plugin's memory of the projects where `/project-roles` was used (the explorer's scope list) |
|
|
213
225
|
| `.llm-role-refresh.lock` | `O_EXCL` create, holds `"<pid> <iso>"` | serializes concurrent session starts; stale (> 60 s) locks are unlinked and retried once |
|
|
214
226
|
| `llm-role-error-reports.json` | `{ reports: [{ fingerprint, label, title, firstSeen, lastSeen, createdDay?, offeredDay?, count, action, issueNumber?, issueUrl? }], dayCount: { day, count } }` | what the opt-in `errorReporting` path filed, offered or suppressed |
|
|
@@ -241,6 +253,24 @@ settings entry the global behavior is unchanged.
|
|
|
241
253
|
that feeds the 3-creations-per-day cap. Nothing is written when
|
|
242
254
|
`errorReporting` is `off`.
|
|
243
255
|
|
|
256
|
+
- `llm-role-usage.jsonl` is the per-role usage ledger (spec #81,
|
|
257
|
+
`src/usage-ledger.ts`). Unlike the two ledgers above it is **not** global: it
|
|
258
|
+
follows the plugin's state location — `<cwd>/.omp` in project mode,
|
|
259
|
+
`agentDir()` otherwise — so a project's usage stays separate from the
|
|
260
|
+
user-level profile. The extension records one row per assistant turn on
|
|
261
|
+
`turn_end` and `session_shutdown` (the backstop for a headless one-turn run),
|
|
262
|
+
read from `sessionManager.getBranch()` — the same entries the session jsonl
|
|
263
|
+
persists — into `resolveProjectDir() ?? agentDir()`, deduped by `ts|model` so
|
|
264
|
+
the two hooks cannot double-record. The file is **append-only** (a crash never
|
|
265
|
+
corrupts it) and **bounded**: `appendUsageRow` compacts to the newest
|
|
266
|
+
`USAGE_LEDGER_CAP = 2000` rows via a temp file + atomic rename, so a crash
|
|
267
|
+
mid-compaction leaves the original intact. Recording is **best-effort** — a
|
|
268
|
+
failure never breaks a session, and `appendUsageRow` never throws. Reading is
|
|
269
|
+
tolerant: a missing, unreadable or malformed file degrades to `[]`, and an
|
|
270
|
+
unparseable or wrongly-shaped line is skipped. The explorer reads the active
|
|
271
|
+
scope's ledger (a project scope its own `<root>/.omp`), so the per-task column
|
|
272
|
+
follows the scope.
|
|
273
|
+
|
|
244
274
|
- `previousModelRoles` is the full snapshot of the last `modelRoles` block
|
|
245
275
|
**before** the plugin changed it — a manual rollback aid; there is no rollback
|
|
246
276
|
command.
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
llm-stats.com best-fit ranking per omp model role — 408 models, 2026-10-
|
|
1
|
+
llm-stats.com best-fit ranking per omp model role — 408 models, 2026-10-19
|
|
2
2
|
Value ranking per role: each metric is cardinal-normalized with fixed anchors
|
|
3
3
|
(no ranks): llm-stats index_* affine (v+20)/80 (interval scale, observed −16..+60),
|
|
4
4
|
benchmarks chance-anchored pass rates, throughput log-anchored 10..300 tok/s.
|
|
@@ -21,179 +21,179 @@ agon=Design Arena `agents/agon_webapps` Elo from the same endpoint: context
|
|
|
21
21
|
only, unweighted (22/87 of the designer pool, below the ~35–40% coverage
|
|
22
22
|
bar at which a metric earns weight).
|
|
23
23
|
★ = Pareto-frontier: no eligible model is both cheaper and better (q).
|
|
24
|
-
Throughput + price: OpenRouter per-provider routes under the default price-based routing (weight 1/price² over stable standard-tier providers; p50 tok/s of routed traffic; $/M 3:1 in:out), throughput
|
|
24
|
+
Throughput + price: OpenRouter per-provider routes under the default price-based routing (weight 1/price² over stable standard-tier providers; p50 tok/s of routed traffic; $/M 3:1 in:out), throughput 149/408, priced 148; models without OpenRouter throughput or a billed route are not ranked.
|
|
25
25
|
|
|
26
26
|
## @default — Main workhorse: strong general coding-agent quality, sane cost
|
|
27
27
|
eligible: 140 — λ 0.00556 $/quality-point (thinking ×1.857)
|
|
28
28
|
|
|
29
29
|
| # | value | q | model | org | $/M | tok/s | ctx | gen 54.7% | code 18.9% | ag 12.6% | tool 10.5% | tput 3.2% |
|
|
30
30
|
| -: | ----: | ----: | --------------------- | ----------- | ----: | ----: | ---: | --------: | ---------: | -------: | ---------: | --------: |
|
|
31
|
-
| 1 | 0.
|
|
32
|
-
| 2 | 0.
|
|
33
|
-
| 3 | 0.
|
|
34
|
-
| 4 | 0.
|
|
35
|
-
| 5 | 0.776 | 0.
|
|
36
|
-
| 6 | 0.
|
|
37
|
-
| 7 | 0.
|
|
38
|
-
| 8 | 0.
|
|
39
|
-
| 9 | 0.
|
|
40
|
-
| 10 | 0.
|
|
31
|
+
| 1 | 0.807 | 0.809 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.37 | 75 | 1.0M | 0.483 | 0.146 | 0.093 | 0.069 | 0.019 |
|
|
32
|
+
| 2 | 0.799 | 0.841 | ★ Gemini 4 Argon | Google | $7.65 | 38 | — | 0.516 | 0.149 | 0.096 | 0.068 | 0.012 |
|
|
33
|
+
| 3 | 0.794 | 0.806 | GLM-5.3 | Zhipu AI | $2.09 | 73 | 1.0M | 0.488 | 0.142 | 0.090 | 0.067 | 0.018 |
|
|
34
|
+
| 4 | 0.789 | 0.810 | ★ Muse Spark 1.3 | Meta | $3.71 | 70 | 1.0M | 0.495 | 0.139 | 0.091 | 0.067 | 0.018 |
|
|
35
|
+
| 5 | 0.776 | 0.793 | DeepSeek-V4-Pro-0813 | DeepSeek | $3.09 | 78 | 1.0M | 0.484 | 0.139 | 0.088 | 0.063 | 0.019 |
|
|
36
|
+
| 6 | 0.776 | 0.781 | MiMo-V2.6-Pro | Xiaomi | $1.01 | 29 | 1.0M | 0.476 | 0.141 | 0.089 | 0.066 | 0.010 |
|
|
37
|
+
| 7 | 0.773 | 0.811 | ★ Kimi K3 | Moonshot AI | $6.81 | 52 | 1.0M | 0.494 | 0.144 | 0.091 | 0.067 | 0.015 |
|
|
38
|
+
| 8 | 0.766 | 0.809 | GPT-6.1 Sol | OpenAI | $7.89 | 40 | 1.1M | 0.506 | 0.141 | 0.092 | 0.057 | 0.013 |
|
|
39
|
+
| 9 | 0.764 | 0.777 | Hy4 preview | Tencent | $2.32 | 37 | — | 0.478 | 0.137 | 0.087 | 0.063 | 0.012 |
|
|
40
|
+
| 10 | 0.763 | 0.765 | ★ GLM-5.3-Flash | Zhipu AI | $0.33 | 66 | 1.0M | 0.473 | 0.123 | 0.087 | 0.065 | 0.018 |
|
|
41
41
|
|
|
42
42
|
## @smol — Fast lightweight model: cheap and quick, still competent
|
|
43
43
|
eligible: 140 — λ 0.04091 $/quality-point
|
|
44
44
|
|
|
45
|
-
| # | value | q | model | org
|
|
46
|
-
| -: | ----: | ----: | ---------------------------- |
|
|
47
|
-
| 1 | 0.
|
|
48
|
-
| 2 | 0.
|
|
49
|
-
| 3 | 0.
|
|
50
|
-
| 4 | 0.
|
|
51
|
-
| 5 | 0.
|
|
52
|
-
| 6 | 0.
|
|
53
|
-
| 7 | 0.
|
|
54
|
-
| 8 | 0.
|
|
55
|
-
| 9 | 0.
|
|
56
|
-
| 10 | 0.
|
|
45
|
+
| # | value | q | model | org | $/M | tok/s | ctx | tput 40% | gen 28.6% | code 17.1% | tool 14.3% |
|
|
46
|
+
| -: | ----: | ----: | ---------------------------- | --------------------- | ----: | ----: | ---: | -------: | --------: | ---------: | ---------: |
|
|
47
|
+
| 1 | 0.706 | 0.714 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.20 | 75 | 1.0M | 0.237 | 0.252 | 0.132 | 0.094 |
|
|
48
|
+
| 2 | 0.687 | 0.713 | Inkling-Small | Thinking Machines Lab | $0.64 | 167 | 524k | 0.331 | 0.208 | 0.104 | 0.070 |
|
|
49
|
+
| 3 | 0.669 | 0.671 | ★ Ling 3.0 Flash | InclusionAI | $0.04 | 140 | 131k | 0.310 | 0.200 | 0.096 | 0.065 |
|
|
50
|
+
| 4 | 0.662 | 0.744 | ★ Muse Spark 1.1 | Meta | $2.00 | 127 | 1.0M | 0.298 | 0.245 | 0.113 | 0.087 |
|
|
51
|
+
| 5 | 0.661 | 0.669 | GLM-5.3-Flash | Zhipu AI | $0.18 | 66 | 1.0M | 0.223 | 0.247 | 0.112 | 0.088 |
|
|
52
|
+
| 6 | 0.661 | 0.707 | GLM-5.3 | Zhipu AI | $1.12 | 73 | 1.0M | 0.233 | 0.255 | 0.129 | 0.091 |
|
|
53
|
+
| 7 | 0.652 | 0.670 | DeepSeek-V4-Flash-Vision-Exp | DeepSeek | $0.43 | 83 | 1.0M | 0.249 | 0.231 | 0.115 | 0.075 |
|
|
54
|
+
| 8 | 0.652 | 0.733 | Muse Spark 1.2 | Meta | $2.00 | 224 | 1.0M | 0.366 | 0.204 | 0.102 | 0.062 |
|
|
55
|
+
| 9 | 0.647 | 0.667 | GPT-5.6 Luna | OpenAI | $0.47 | 80 | 1.1M | 0.244 | 0.229 | 0.117 | 0.076 |
|
|
56
|
+
| 10 | 0.643 | 0.707 | Gemini 3.8 Flash | Google | $1.57 | 95 | 1.0M | 0.264 | 0.247 | 0.119 | 0.076 |
|
|
57
57
|
|
|
58
58
|
## @slow — Most capable model for hard problems; cost and speed as tiebreakers
|
|
59
59
|
eligible: 140 — λ 0.00263 $/quality-point (thinking ×2.714)
|
|
60
60
|
|
|
61
61
|
| # | value | q | model | org | $/M | tok/s | ctx | gen 63.2% | code 18.9% | ag 13.7% | tput 4.2% |
|
|
62
62
|
| -: | ----: | ----: | --------------------- | ----------- | ----: | ----: | ---: | --------: | ---------: | -------: | --------: |
|
|
63
|
-
| 1 | 0.
|
|
64
|
-
| 2 | 0.
|
|
65
|
-
| 3 | 0.
|
|
66
|
-
| 4 | 0.
|
|
67
|
-
| 5 | 0.812 | 0.
|
|
68
|
-
| 6 | 0.
|
|
69
|
-
| 7 | 0.
|
|
70
|
-
| 8 | 0.796 | 0.
|
|
71
|
-
| 9 | 0.
|
|
72
|
-
| 10 | 0.
|
|
63
|
+
| 1 | 0.835 | 0.865 | ★ Gemini 4 Argon | Google | $11.2 | 38 | — | 0.595 | 0.149 | 0.104 | 0.016 |
|
|
64
|
+
| 2 | 0.827 | 0.829 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.54 | 75 | 1.0M | 0.558 | 0.146 | 0.100 | 0.025 |
|
|
65
|
+
| 3 | 0.819 | 0.827 | GLM-5.3 | Zhipu AI | $3.05 | 73 | 1.0M | 0.563 | 0.142 | 0.098 | 0.025 |
|
|
66
|
+
| 4 | 0.818 | 0.833 | ★ Muse Spark 1.3 | Meta | $5.43 | 70 | 1.0M | 0.572 | 0.139 | 0.099 | 0.024 |
|
|
67
|
+
| 5 | 0.812 | 0.842 | GPT-6.1 Sol | OpenAI | $11.5 | 40 | 1.1M | 0.584 | 0.141 | 0.100 | 0.017 |
|
|
68
|
+
| 6 | 0.806 | 0.833 | Kimi K3 | Moonshot AI | $9.95 | 52 | 1.0M | 0.570 | 0.144 | 0.098 | 0.020 |
|
|
69
|
+
| 7 | 0.806 | 0.818 | DeepSeek-V4-Pro-0813 | DeepSeek | $4.51 | 78 | 1.0M | 0.559 | 0.139 | 0.095 | 0.025 |
|
|
70
|
+
| 8 | 0.796 | 0.800 | MiMo-V2.6-Pro | Xiaomi | $1.47 | 29 | 1.0M | 0.549 | 0.141 | 0.097 | 0.013 |
|
|
71
|
+
| 9 | 0.792 | 0.852 | GPT-5.6 Sol | OpenAI | $22.9 | 54 | 1.1M | 0.580 | 0.150 | 0.101 | 0.021 |
|
|
72
|
+
| 10 | 0.790 | 0.798 | Hy4 preview | Tencent | $3.39 | 37 | — | 0.552 | 0.137 | 0.094 | 0.016 |
|
|
73
73
|
|
|
74
74
|
## @vision — Image understanding: vision index dominates
|
|
75
|
-
eligible:
|
|
75
|
+
eligible: 73 — λ 0.00682 $/quality-point (thinking ×1.857)
|
|
76
76
|
|
|
77
77
|
| # | value | q | model | org | $/M | tok/s | ctx | vis 54.2% | gen 20.8% | rea 15.6% | code 6.3% | tput 3.1% |
|
|
78
78
|
| -: | ----: | ----: | --------------------- | ------------------------- | ----: | ----: | ---: | --------: | --------: | --------: | --------: | --------: |
|
|
79
|
-
| 1 | 0.
|
|
79
|
+
| 1 | 0.733 | 0.786 | ★ Gemini 4 Argon | Google | $7.65 | 38 | — | 0.385 | 0.196 | 0.143 | 0.049 | 0.012 |
|
|
80
80
|
| 2 | 0.728 | 0.731 | ★ Qwen3.8 Flash | Alibaba Cloud / Qwen Team | $0.43 | 47 | 1.0M | 0.363 | 0.178 | 0.133 | 0.042 | 0.014 |
|
|
81
|
-
| 3 | 0.
|
|
82
|
-
| 4 | 0.
|
|
83
|
-
| 5 | 0.
|
|
84
|
-
| 6 | 0.
|
|
85
|
-
| 7 | 0.
|
|
86
|
-
| 8 | 0.
|
|
87
|
-
| 9 | 0.
|
|
88
|
-
| 10 | 0.
|
|
81
|
+
| 3 | 0.727 | 0.730 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.37 | 75 | 1.0M | 0.345 | 0.184 | 0.134 | 0.048 | 0.018 |
|
|
82
|
+
| 4 | 0.724 | 0.750 | ★ Muse Spark 1.1 | Meta | $3.71 | 127 | 1.0M | 0.371 | 0.179 | 0.136 | 0.041 | 0.023 |
|
|
83
|
+
| 5 | 0.722 | 0.768 | ★ Kimi K3 | Moonshot AI | $6.81 | 52 | 1.0M | 0.378 | 0.188 | 0.139 | 0.048 | 0.015 |
|
|
84
|
+
| 6 | 0.715 | 0.769 | GPT-6.1 Sol | OpenAI | $7.89 | 40 | 1.1M | 0.377 | 0.193 | 0.140 | 0.047 | 0.013 |
|
|
85
|
+
| 7 | 0.714 | 0.734 | ★ Gemini 3.8 Flash | Google | $2.91 | 95 | 1.0M | 0.361 | 0.180 | 0.129 | 0.043 | 0.021 |
|
|
86
|
+
| 8 | 0.713 | 0.715 | ★ GLM-5.3-Flash | Zhipu AI | $0.33 | 66 | 1.0M | 0.344 | 0.180 | 0.133 | 0.041 | 0.017 |
|
|
87
|
+
| 9 | 0.702 | 0.707 | Qwen3.8-27B | Alibaba Cloud / Qwen Team | $0.72 | 55 | 262k | 0.358 | 0.168 | 0.126 | 0.039 | 0.016 |
|
|
88
|
+
| 10 | 0.697 | 0.717 | Gemini 3.7 Flash | Google | $2.87 | 86 | 1.0M | 0.353 | 0.176 | 0.127 | 0.041 | 0.020 |
|
|
89
89
|
|
|
90
90
|
## @plan — Planning: reasoning and long-context coherence, priced so the cheapest capable planner wins
|
|
91
91
|
eligible: 140 — λ 0.01667 $/quality-point (thinking ×1.857)
|
|
92
92
|
|
|
93
|
-
| # | value | q | model
|
|
94
|
-
| -: | ----: | ----: |
|
|
95
|
-
| 1 | 0.
|
|
96
|
-
| 2 | 0.
|
|
97
|
-
| 3 | 0.
|
|
98
|
-
| 4 | 0.
|
|
99
|
-
| 5 | 0.
|
|
100
|
-
| 6 | 0.
|
|
101
|
-
| 7 | 0.
|
|
102
|
-
| 8 | 0.
|
|
103
|
-
| 9 | 0.735 | 0.
|
|
104
|
-
| 10 | 0.
|
|
93
|
+
| # | value | q | model | org | $/M | tok/s | ctx | gen 80% | lc 15.6% | tput 4.4% |
|
|
94
|
+
| -: | ----: | ----: | --------------------- | ------------------------- | ----: | ----: | ---: | ------: | -------: | --------: |
|
|
95
|
+
| 1 | 0.789 | 0.851 | ★ Muse Spark 1.3 | Meta | $3.71 | 70 | 1.0M | 0.724 | 0.101 | 0.025 |
|
|
96
|
+
| 2 | 0.774 | 0.781 | ★ Qwen3.8 Flash | Alibaba Cloud / Qwen Team | $0.43 | 47 | 1.0M | 0.682 | 0.079 | 0.020 |
|
|
97
|
+
| 3 | 0.764 | 0.813 | ★ Gemini 3.8 Flash | Google | $2.91 | 95 | 1.0M | 0.692 | 0.091 | 0.029 |
|
|
98
|
+
| 4 | 0.757 | 0.763 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.37 | 75 | 1.0M | 0.707 | 0.030 | 0.026 |
|
|
99
|
+
| 5 | 0.755 | 0.802 | ★ Gemini 3.7 Flash | Google | $2.87 | 86 | 1.0M | 0.676 | 0.098 | 0.028 |
|
|
100
|
+
| 6 | 0.752 | 0.790 | ★ Hy4 preview | Tencent | $2.32 | 37 | — | 0.699 | 0.075 | 0.017 |
|
|
101
|
+
| 7 | 0.742 | 0.870 | ★ Gemini 4 Argon | Google | $7.65 | 38 | — | 0.754 | 0.098 | 0.017 |
|
|
102
|
+
| 8 | 0.740 | 0.746 | ★ GLM-5.3-Flash | Zhipu AI | $0.33 | 66 | 1.0M | 0.691 | 0.030 | 0.025 |
|
|
103
|
+
| 9 | 0.735 | 0.749 | GPT-5.6 Luna | OpenAI | $0.88 | 80 | 1.1M | 0.642 | 0.080 | 0.027 |
|
|
104
|
+
| 10 | 0.735 | 0.769 | GLM-5.3 | Zhipu AI | $2.09 | 73 | 1.0M | 0.713 | 0.030 | 0.026 |
|
|
105
105
|
|
|
106
106
|
## @commit — Commit messages: cheap and fast with decent general quality
|
|
107
107
|
eligible: 140 — λ 0.04091 $/quality-point
|
|
108
108
|
|
|
109
|
-
| # | value | q | model | org
|
|
110
|
-
| -: | ----: | ----: | ---------------------------- |
|
|
111
|
-
| 1 | 0.
|
|
112
|
-
| 2 | 0.
|
|
113
|
-
| 3 | 0.
|
|
114
|
-
| 4 | 0.
|
|
115
|
-
| 5 | 0.
|
|
116
|
-
| 6 | 0.
|
|
117
|
-
| 7 | 0.
|
|
118
|
-
| 8 | 0.
|
|
119
|
-
| 9 | 0.
|
|
120
|
-
| 10 | 0.
|
|
109
|
+
| # | value | q | model | org | $/M | tok/s | ctx | tput 41.5% | gen 43.1% | code 15.4% |
|
|
110
|
+
| -: | ----: | ----: | ---------------------------- | --------------------- | ----: | ----: | ---: | ---------: | --------: | ---------: |
|
|
111
|
+
| 1 | 0.736 | 0.745 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.20 | 75 | 1.0M | 0.246 | 0.380 | 0.118 |
|
|
112
|
+
| 2 | 0.724 | 0.750 | ★ Inkling-Small | Thinking Machines Lab | $0.64 | 167 | 524k | 0.344 | 0.313 | 0.094 |
|
|
113
|
+
| 3 | 0.709 | 0.710 | ★ Ling 3.0 Flash | InclusionAI | $0.04 | 140 | 131k | 0.322 | 0.302 | 0.086 |
|
|
114
|
+
| 4 | 0.699 | 0.781 | ★ Muse Spark 1.1 | Meta | $2.00 | 127 | 1.0M | 0.310 | 0.370 | 0.101 |
|
|
115
|
+
| 5 | 0.697 | 0.779 | Muse Spark 1.2 | Meta | $2.00 | 224 | 1.0M | 0.380 | 0.308 | 0.091 |
|
|
116
|
+
| 6 | 0.696 | 0.703 | GLM-5.3-Flash | Zhipu AI | $0.18 | 66 | 1.0M | 0.231 | 0.372 | 0.100 |
|
|
117
|
+
| 7 | 0.695 | 0.741 | GLM-5.3 | Zhipu AI | $1.12 | 73 | 1.0M | 0.242 | 0.384 | 0.115 |
|
|
118
|
+
| 8 | 0.692 | 0.710 | DeepSeek-V4-Flash-Vision-Exp | DeepSeek | $0.43 | 83 | 1.0M | 0.259 | 0.348 | 0.103 |
|
|
119
|
+
| 9 | 0.690 | 0.754 | ★ Gemini 3.8 Flash | Google | $1.57 | 95 | 1.0M | 0.275 | 0.373 | 0.106 |
|
|
120
|
+
| 10 | 0.685 | 0.704 | GPT-5.6 Luna | OpenAI | $0.47 | 80 | 1.1M | 0.254 | 0.346 | 0.105 |
|
|
121
121
|
|
|
122
122
|
## @tiny — Background tasks (titles, memory): cheapest and fastest wins
|
|
123
123
|
eligible: 140 — λ 0.03333 $/quality-point
|
|
124
124
|
|
|
125
|
-
| # | value | q | model
|
|
126
|
-
| -: | ----: | ----: |
|
|
127
|
-
| 1 | 0.
|
|
128
|
-
| 2 | 0.
|
|
129
|
-
| 3 | 0.
|
|
130
|
-
| 4 | 0.
|
|
131
|
-
| 5 | 0.
|
|
132
|
-
| 6 | 0.
|
|
133
|
-
| 7 | 0.
|
|
134
|
-
| 8 | 0.
|
|
135
|
-
| 9 | 0.
|
|
136
|
-
| 10 | 0.
|
|
125
|
+
| # | value | q | model | org | $/M | tok/s | ctx | tput 58.3% | gen 41.7% |
|
|
126
|
+
| -: | ----: | ----: | ----------------------------- | ------------------------- | ----: | ----: | ---: | ---------: | --------: |
|
|
127
|
+
| 1 | 0.769 | 0.782 | ★ Mercury 2 | Inception | $0.38 | 255 | 128k | 0.555 | 0.227 |
|
|
128
|
+
| 2 | 0.765 | 0.831 | ★ Muse Spark 1.2 | Meta | $2.00 | 224 | 1.0M | 0.533 | 0.298 |
|
|
129
|
+
| 3 | 0.764 | 0.786 | ★ Inkling-Small | Thinking Machines Lab | $0.64 | 167 | 524k | 0.483 | 0.303 |
|
|
130
|
+
| 4 | 0.743 | 0.745 | ★ Ling 3.0 Flash | InclusionAI | $0.04 | 140 | 131k | 0.453 | 0.292 |
|
|
131
|
+
| 5 | 0.738 | 0.756 | Gemini 3.1 Flash-Lite | Google | $0.56 | 176 | 1.0M | 0.492 | 0.265 |
|
|
132
|
+
| 6 | 0.726 | 0.793 | Muse Spark 1.1 | Meta | $2.00 | 127 | 1.0M | 0.435 | 0.358 |
|
|
133
|
+
| 7 | 0.717 | 0.768 | Gemini 3.6 Flash | Google | $1.54 | 132 | 1.0M | 0.443 | 0.325 |
|
|
134
|
+
| 8 | 0.707 | 0.713 | DeepSeek-V4.1-Flash | DeepSeek | $0.20 | 75 | 1.0M | 0.345 | 0.368 |
|
|
135
|
+
| 9 | 0.705 | 0.707 | Laguna XS 2.1 | Poolside | $0.07 | 161 | 262k | 0.477 | 0.231 |
|
|
136
|
+
| 10 | 0.702 | 0.743 | Qwen3-235B-A22B-Thinking-2507 | Alibaba Cloud / Qwen Team | $1.21 | 177 | — | 0.493 | 0.250 |
|
|
137
137
|
|
|
138
138
|
## @task — Subagents: agentic + tool calling, moderate cost sensitivity
|
|
139
139
|
eligible: 140 — λ 0.00747 $/quality-point (thinking ×1.857)
|
|
140
140
|
|
|
141
141
|
| # | value | q | model | org | $/M | tok/s | ctx | ag 29.9% | tool 20.7% | code 18.4% | gen 25.3% | tput 5.7% |
|
|
142
142
|
| -: | ----: | ----: | --------------------- | ------------------------- | ----: | ----: | ---: | -------: | ---------: | ---------: | --------: | --------: |
|
|
143
|
-
| 1 | 0.
|
|
144
|
-
| 2 | 0.
|
|
145
|
-
| 3 | 0.
|
|
146
|
-
| 4 | 0.
|
|
147
|
-
| 5 | 0.
|
|
148
|
-
| 6 | 0.
|
|
149
|
-
| 7 | 0.
|
|
150
|
-
| 8 | 0.
|
|
151
|
-
| 9 | 0.
|
|
152
|
-
| 10 | 0.
|
|
143
|
+
| 1 | 0.750 | 0.753 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.37 | 75 | 1.0M | 0.219 | 0.136 | 0.141 | 0.223 | 0.034 |
|
|
144
|
+
| 2 | 0.726 | 0.742 | GLM-5.3 | Zhipu AI | $2.09 | 73 | 1.0M | 0.213 | 0.131 | 0.138 | 0.225 | 0.033 |
|
|
145
|
+
| 3 | 0.716 | 0.743 | Muse Spark 1.3 | Meta | $3.71 | 70 | 1.0M | 0.215 | 0.132 | 0.134 | 0.229 | 0.033 |
|
|
146
|
+
| 4 | 0.710 | 0.767 | ★ Gemini 4 Argon | Google | $7.65 | 38 | — | 0.228 | 0.134 | 0.145 | 0.238 | 0.022 |
|
|
147
|
+
| 5 | 0.707 | 0.715 | MiMo-V2.6-Pro | Xiaomi | $1.01 | 29 | 1.0M | 0.211 | 0.129 | 0.137 | 0.220 | 0.018 |
|
|
148
|
+
| 6 | 0.702 | 0.725 | DeepSeek-V4-Pro-0813 | DeepSeek | $3.09 | 78 | 1.0M | 0.208 | 0.124 | 0.135 | 0.224 | 0.035 |
|
|
149
|
+
| 7 | 0.700 | 0.703 | ★ GLM-5.3-Flash | Zhipu AI | $0.33 | 66 | 1.0M | 0.206 | 0.127 | 0.120 | 0.218 | 0.032 |
|
|
150
|
+
| 8 | 0.692 | 0.743 | Kimi K3 | Moonshot AI | $6.81 | 52 | 1.0M | 0.215 | 0.132 | 0.140 | 0.228 | 0.028 |
|
|
151
|
+
| 9 | 0.688 | 0.691 | Qwen3.8 Flash | Alibaba Cloud / Qwen Team | $0.43 | 47 | 1.0M | 0.198 | 0.127 | 0.124 | 0.216 | 0.026 |
|
|
152
|
+
| 10 | 0.687 | 0.705 | Hy4 preview | Tencent | $2.32 | 37 | — | 0.205 | 0.124 | 0.133 | 0.221 | 0.022 |
|
|
153
153
|
|
|
154
154
|
## @advisor — Advisor/watchdog: deep reasoning over long context
|
|
155
155
|
eligible: 140 — λ 0.01250 $/quality-point (thinking ×1.857)
|
|
156
156
|
|
|
157
157
|
| # | value | q | model | org | $/M | tok/s | ctx | gen 74.3% | lc 12.6% | tput 13.1% |
|
|
158
158
|
| -: | ----: | ----: | --------------------- | ------------------------- | ----: | ----: | ---: | --------: | -------: | ---------: |
|
|
159
|
-
| 1 | 0.
|
|
160
|
-
| 2 | 0.
|
|
161
|
-
| 3 | 0.
|
|
162
|
-
| 4 | 0.
|
|
163
|
-
| 5 | 0.
|
|
164
|
-
| 6 | 0.
|
|
165
|
-
| 7 | 0.
|
|
166
|
-
| 8 | 0.
|
|
167
|
-
| 9 | 0.
|
|
168
|
-
| 10 | 0.
|
|
159
|
+
| 1 | 0.783 | 0.830 | ★ Muse Spark 1.3 | Meta | $3.71 | 70 | 1.0M | 0.673 | 0.082 | 0.075 |
|
|
160
|
+
| 2 | 0.767 | 0.803 | ★ Gemini 3.8 Flash | Google | $2.91 | 95 | 1.0M | 0.643 | 0.073 | 0.087 |
|
|
161
|
+
| 3 | 0.755 | 0.790 | ★ Gemini 3.7 Flash | Google | $2.87 | 86 | 1.0M | 0.628 | 0.079 | 0.083 |
|
|
162
|
+
| 4 | 0.754 | 0.759 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.37 | 75 | 1.0M | 0.657 | 0.024 | 0.078 |
|
|
163
|
+
| 5 | 0.752 | 0.757 | Qwen3.8 Flash | Alibaba Cloud / Qwen Team | $0.43 | 47 | 1.0M | 0.634 | 0.063 | 0.060 |
|
|
164
|
+
| 6 | 0.737 | 0.764 | ★ GLM-5.3 | Zhipu AI | $2.09 | 73 | 1.0M | 0.663 | 0.024 | 0.076 |
|
|
165
|
+
| 7 | 0.735 | 0.831 | ★ Gemini 4 Argon | Google | $7.65 | 38 | — | 0.700 | 0.079 | 0.051 |
|
|
166
|
+
| 8 | 0.735 | 0.739 | ★ GLM-5.3-Flash | Zhipu AI | $0.33 | 66 | 1.0M | 0.642 | 0.024 | 0.073 |
|
|
167
|
+
| 9 | 0.731 | 0.760 | Hy4 preview | Tencent | $2.32 | 37 | — | 0.649 | 0.060 | 0.050 |
|
|
168
|
+
| 10 | 0.730 | 0.741 | GPT-5.6 Luna | OpenAI | $0.88 | 80 | 1.1M | 0.597 | 0.065 | 0.080 |
|
|
169
169
|
|
|
170
170
|
## @designer — Design work: visual/UX judgement on image-capable models
|
|
171
|
-
eligible:
|
|
171
|
+
eligible: 87 — λ 0.01024 $/quality-point (thinking ×1.857)
|
|
172
172
|
|
|
173
173
|
| # | value | q | model | org | $/M | tok/s | ctx | gen 31.3% | code 7.2% | vis 21.7% | tput 18.1% | web 21.7% | agon |
|
|
174
174
|
| -: | ----: | ----: | --------------------- | ----------- | ----: | ----: | ---: | --------: | --------: | --------: | ---------: | --------: | ---: |
|
|
175
|
-
| 1 | 0.
|
|
176
|
-
| 2 | 0.
|
|
177
|
-
| 3 | 0.
|
|
178
|
-
| 4 | 0.
|
|
179
|
-
| 5 | 0.
|
|
180
|
-
| 6 | 0.
|
|
181
|
-
| 7 | 0.
|
|
182
|
-
| 8 | 0.
|
|
183
|
-
| 9 | 0.
|
|
184
|
-
| 10 | 0.
|
|
185
|
-
focus: website — coverage
|
|
175
|
+
| 1 | 0.771 | 0.775 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.37 | 75 | 1.0M | 0.277 | 0.056 | 0.138 | 0.107 | 0.197 | — |
|
|
176
|
+
| 2 | 0.744 | 0.774 | Gemini 3.8 Flash | Google | $2.91 | 95 | 1.0M | 0.271 | 0.050 | 0.144 | 0.119 | 0.189 | 1236 |
|
|
177
|
+
| 3 | 0.741 | 0.779 | ★ Muse Spark 1.3 | Meta | $3.71 | 70 | 1.0M | 0.283 | 0.053 | 0.125 | 0.103 | 0.214 | 1296 |
|
|
178
|
+
| 4 | 0.730 | 0.760 | Gemini 3.7 Flash | Google | $2.87 | 86 | 1.0M | 0.265 | 0.047 | 0.141 | 0.114 | 0.192 | 1223 |
|
|
179
|
+
| 5 | 0.727 | 0.765 | Muse Spark 1.1 | Meta | $3.71 | 127 | 1.0M | 0.269 | 0.048 | 0.148 | 0.135 | 0.165 | — |
|
|
180
|
+
| 6 | 0.725 | 0.755 | Gemini 3.6 Flash | Google | $2.87 | 132 | 1.0M | 0.244 | 0.042 | 0.140 | 0.137 | 0.190 | 1194 |
|
|
181
|
+
| 7 | 0.721 | 0.724 | ★ GLM-5.3-Flash | Zhipu AI | $0.33 | 66 | 1.0M | 0.270 | 0.047 | 0.138 | 0.101 | 0.168 | — |
|
|
182
|
+
| 8 | 0.718 | 0.788 | ★ Kimi K3 | Moonshot AI | $6.81 | 52 | 1.0M | 0.283 | 0.055 | 0.152 | 0.088 | 0.211 | 1292 |
|
|
183
|
+
| 9 | 0.702 | 0.781 | Gemini 4 Argon | Google | $7.65 | 38 | — | 0.295 | 0.057 | 0.154 | 0.070 | 0.204 | — |
|
|
184
|
+
| 10 | 0.698 | 0.709 | MiMo-V2.6-Pro | Xiaomi | $1.01 | 29 | 1.0M | 0.272 | 0.054 | 0.127 | 0.056 | 0.199 | — |
|
|
185
|
+
focus: website — coverage 127/408 (31.1%) ok; dispersion 0.992 ok; composition ok; freshness 0.0mo ok; trust unknown; modality unknown; maintenance unknown; provenance unknown; cross-source unknown
|
|
186
186
|
|
|
187
187
|
# Suggested settings.modelRoles (best-fit #1 per role, resolved via the omp catalog).
|
|
188
188
|
# Selectors are openrouter/<id>; the omp-llm-role plugin writes this block daily.
|
|
189
189
|
modelRoles:
|
|
190
190
|
default: "openrouter/deepseek/deepseek-v4.1-flash"
|
|
191
191
|
smol: "openrouter/deepseek/deepseek-v4.1-flash"
|
|
192
|
-
slow: "openrouter/
|
|
193
|
-
vision: "openrouter/
|
|
192
|
+
slow: "openrouter/google/gemini-4-argon"
|
|
193
|
+
vision: "openrouter/google/gemini-4-argon"
|
|
194
194
|
plan: "openrouter/meta/muse-spark-1.3"
|
|
195
195
|
commit: "openrouter/deepseek/deepseek-v4.1-flash"
|
|
196
|
-
tiny: "openrouter/
|
|
196
|
+
tiny: "openrouter/inception/mercury-2"
|
|
197
197
|
task: "openrouter/deepseek/deepseek-v4.1-flash"
|
|
198
198
|
advisor: "openrouter/meta/muse-spark-1.3"
|
|
199
199
|
designer: "openrouter/deepseek/deepseek-v4.1-flash"
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "omp-llm-role",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.3.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "omp plugin: daily model-role updater driven by llm-stats/OpenRouter rankings",
|
|
6
6
|
"license": "MIT",
|
|
@@ -78,6 +78,16 @@
|
|
|
78
78
|
"auto"
|
|
79
79
|
]
|
|
80
80
|
},
|
|
81
|
+
"tokenProfile.inputTokens": {
|
|
82
|
+
"type": "number",
|
|
83
|
+
"description": "Input tokens per task for the per-task price (used when the role has no recorded usage)",
|
|
84
|
+
"min": 0
|
|
85
|
+
},
|
|
86
|
+
"tokenProfile.outputTokens": {
|
|
87
|
+
"type": "number",
|
|
88
|
+
"description": "Output tokens per task for the per-task price (used when the role has no recorded usage)",
|
|
89
|
+
"min": 0
|
|
90
|
+
},
|
|
81
91
|
"features.endpointCeilings": {
|
|
82
92
|
"type": "boolean",
|
|
83
93
|
"description": "Drop routes that cannot call tools and routes whose output ceiling is below 16 384 tokens",
|
package/src/agent-create.ts
CHANGED
|
@@ -424,8 +424,10 @@ function coveredValues(models: readonly Model[], metric: string, fill: number):
|
|
|
424
424
|
return values;
|
|
425
425
|
}
|
|
426
426
|
|
|
427
|
-
/** Linear-interpolated quantile (R type 7) of a value list
|
|
428
|
-
|
|
427
|
+
/** Linear-interpolated quantile (R type 7) of a value list — the repo's one
|
|
428
|
+
* quantile convention: the focus-axis dispersion stats and the usage ledger's
|
|
429
|
+
* per-turn median (`quantile(values, 0.5)`) both read it. */
|
|
430
|
+
export function quantile(values: readonly number[], p: number): number {
|
|
429
431
|
const sorted = [...values].sort((a, b) => a - b);
|
|
430
432
|
const h = (sorted.length - 1) * p;
|
|
431
433
|
const lo = Math.floor(h);
|
package/src/engine.ts
CHANGED
|
@@ -313,6 +313,19 @@ export type OpenRouterEndpointPages = Record<string, OpenRouterEndpointRecord[]>
|
|
|
313
313
|
|
|
314
314
|
export type OrEnrichment = { tput: number; latency: number | null; price: number | null; thinking: boolean; routes: OpenRouterEndpointRecord[] };
|
|
315
315
|
|
|
316
|
+
/** The route's input (prompt) $/M under an assumed cache-hit rate: the input term
|
|
317
|
+
* `routeEffectivePrice` blends into the billed price. A hit rate `h` blends the
|
|
318
|
+
* cache-read input in, `h·cacheRead + (1−h)·input`; with `h = 0` or no cache-read
|
|
319
|
+
* price this is the listed input price, and a route with no input price has none —
|
|
320
|
+
* missing data keeps the full input price, never a silent discount. The single
|
|
321
|
+
* accessor for the input term: the billed blend, the output-price inverse
|
|
322
|
+
* (`routeOutputPrice`) and the explorer's per-task price all read it. */
|
|
323
|
+
export function routeInputPrice(route: OpenRouterEndpointRecord, cacheHitRate = 0): number | null {
|
|
324
|
+
if (route.weightPrice == null) return null;
|
|
325
|
+
if (cacheHitRate <= 0 || route.cacheReadPrice == null) return route.weightPrice;
|
|
326
|
+
return cacheHitRate * route.cacheReadPrice + (1 - cacheHitRate) * route.weightPrice;
|
|
327
|
+
}
|
|
328
|
+
|
|
316
329
|
/** The effective billed $/M of one route under an assumed cache-hit rate. The route's
|
|
317
330
|
* billed 3:1 price is `price = (3·input + output)/4` on the listed input price; a hit
|
|
318
331
|
* rate `h` blends the cache-read input into the input term,
|
|
@@ -320,15 +333,33 @@ export type OrEnrichment = { tput: number; latency: number | null; price: number
|
|
|
320
333
|
* (the output term is unchanged). With `h = 0`, no cache-read price, or no input price
|
|
321
334
|
* this is the route's listed `price` — missing data keeps the full input price, never a
|
|
322
335
|
* silent discount. Shared by the blend (`blendRoutePool`), the pin path (`rankRole`) and
|
|
323
|
-
* `roleMetricValue`. */
|
|
336
|
+
* `roleMetricValue`; the input term itself is `routeInputPrice`. */
|
|
324
337
|
export function routeEffectivePrice(route: OpenRouterEndpointRecord, cacheHitRate = 0): number | null {
|
|
325
338
|
if (route.price == null) return null;
|
|
326
339
|
if (cacheHitRate <= 0 || route.cacheReadPrice == null || route.weightPrice == null) return route.price;
|
|
327
340
|
const input = route.weightPrice;
|
|
328
|
-
const effInput =
|
|
341
|
+
const effInput = routeInputPrice(route, cacheHitRate) as number;
|
|
329
342
|
return route.price + (3 * (effInput - input)) / 4;
|
|
330
343
|
}
|
|
331
344
|
|
|
345
|
+
/** The output (completion) $/M implied by a billed 3:1 blend and the input price
|
|
346
|
+
* behind it: the inverse of `billed = IO_PRICE_RATIO·(3·input + output)`, i.e.
|
|
347
|
+
* `output = billed/IO_PRICE_RATIO − (1−IO_PRICE_RATIO)/IO_PRICE_RATIO · input`.
|
|
348
|
+
* The 3:1 ratio lives here (and in `IO_PRICE_RATIO`), not in the callers; dividing
|
|
349
|
+
* by `IO_PRICE_RATIO` (2⁻²) is exact, so this is the plain `4·billed − 3·input`. */
|
|
350
|
+
export function blendOutputPrice(billed: number, input: number): number {
|
|
351
|
+
return billed / IO_PRICE_RATIO - ((1 - IO_PRICE_RATIO) / IO_PRICE_RATIO) * input;
|
|
352
|
+
}
|
|
353
|
+
|
|
354
|
+
/** The route's output (completion) $/M: the inverse blend of its own billed and
|
|
355
|
+
* input prices (see `blendOutputPrice`). Null when either of them is missing. */
|
|
356
|
+
export function routeOutputPrice(route: OpenRouterEndpointRecord, cacheHitRate = 0): number | null {
|
|
357
|
+
const billed = routeEffectivePrice(route, cacheHitRate);
|
|
358
|
+
const input = routeInputPrice(route, cacheHitRate);
|
|
359
|
+
if (billed === null || input === null) return null;
|
|
360
|
+
return blendOutputPrice(billed, input);
|
|
361
|
+
}
|
|
362
|
+
|
|
332
363
|
/** Expected price/throughput/latency of one request over a route pool under the
|
|
333
364
|
* router's default price-based load balancing: P(route i) ∝ 1/input_price_i² — the
|
|
334
365
|
* router sorts on the input (prompt) price alone, so the weight basis is
|