omp-llm-role 1.2.0 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -137,11 +137,18 @@ passing.
137
137
  `filters.preferredMinThroughput: 50` (the registry row and the expanded def
138
138
  agree), an explicit `filters.preferredMinThroughput` beats the preset, and
139
139
  the schema carries the `roles.<role>.filters.preferredMinThroughput` row
140
- (number, min 0, default 0).
140
+ (number, min 0, default 0). The `tokenProfile` setting (spec #81) defaults to
141
+ `null`, accepts a record with finite `inputTokens`/`outputTokens` ≥ 0, and
142
+ rejects a non-record or a negative/non-finite field; the schema carries the
143
+ `tokenProfile.inputTokens`/`tokenProfile.outputTokens` rows (number, min 0).
141
144
  23. **Explorer** (`explorer.test.ts`, `explorer-scopes.test.ts`) — rank deltas;
142
145
  contributions sum to `q`; `inverseCardinal` round-trips; Export preserves
143
146
  siblings and normalizes nested roles; the scope switcher; the availability
144
- overlay.
147
+ overlay. The per-task price (spec #81): `rankRows`/`explainModel` fill
148
+ `perTask` from a supplied profile and leave it `null` without one; the
149
+ `POST /api/pool` toggle returns the gated/pruned dataset in `as-written` mode
150
+ and the full one in `raw`; the bootstrap payload carries `poolMode` and
151
+ `tokenProfile`; the explain response carries `leaderFlip`/`thinkingLevels`.
145
152
  24. **Project scope** (`project-updater.test.ts`, `project-setup.test.ts`,
146
153
  `project-roles-command.test.ts`) — project mode scopes config/state/history/
147
154
  lock to `<cwd>/.omp`; the day gate is per scope; the registry write;
@@ -224,6 +231,28 @@ passing.
224
231
  from the live model on `turn_start` (the silent-restore case), strips when
225
232
  neither source carries a pin, never calls `pi.setModel`, and contains a
226
233
  throwing live-model read.
234
+ 33. **Usage ledger** (`usage-ledger.test.ts`, `usage-ledger-wiring.test.ts`) —
235
+ `appendUsageRow` writes one row with the expected fields; `readUsageRows`
236
+ degrades to `[]` on a missing/malformed file and skips an unparseable or
237
+ wrongly-shaped line; the cap holds (the newest `USAGE_LEDGER_CAP` rows
238
+ survive a compaction); `roleTokenProfile` is the role's median per-turn
239
+ counts (or `null` with no rows) and `resolveTokenProfile` prefers the
240
+ recorded usage over the setting. The wiring (a fake `pi`/`ctx` with a
241
+ `sessionManager.getBranch()`) records one row per assistant turn on
242
+ `turn_end` and `session_shutdown`, dedupes by `ts|model` so the two hooks
243
+ cannot double-record, writes into `resolveProjectDir() ?? agentDir()`, and
244
+ contains a throwing branch read.
245
+ 34. **Explorer price levers** (`explorer-levers.test.ts`) — `leaderFlip` returns
246
+ the min-λ threshold over the strictly cheaper challengers, the current
247
+ `wPrice`, the distance and the challenger; `null` when no cheaper challenger
248
+ exists; the degenerate case (value leader = quality leader) is handled by the
249
+ same min formula, not `0/0`. `thinkingLevelEffect` returns one row per level
250
+ with the engine's own factor and the leader at that level, and `null` for a
251
+ bare role.
252
+ 35. **As-written pool** (`explorer-pool.test.ts`) — `asWrittenPool` applies the
253
+ catalog/tier gate (`resolveVariant`) and the allowed-providers prune
254
+ (`providerAllowed`), leaves the routes untouched on a `null` whitelist,
255
+ yields an empty pool on `tier: "none"`, and never mutates its input.
227
256
 
228
257
  ## Live checks
229
258
 
@@ -244,6 +273,18 @@ passing.
244
273
  derived metadata.
245
274
  - **Agent removal** — `/remove-agent --name <n>` in a session deletes both
246
275
  artifacts and the updater pass removes the config entries.
276
+ - **Explorer pool / per-task / levers** — `/explore-roles` in a session with a
277
+ recorded usage ledger: confirm the header's pool toggle switches the table
278
+ between the raw and as-written pools (the model count and the leader change),
279
+ the `$/task` column renders a value (or `—` with no profile), and the explain
280
+ panel shows the per-task breakdown and the **Price levers** section (the
281
+ leader-flip line and the per-level leader table). Verify in headless Chromium
282
+ (the SPA is not unit-tested).
283
+ - **Usage ledger** — run a real omp turn, then confirm `llm-role-usage.jsonl`
284
+ gained one row per assistant turn (the model, the role, the provider, the
285
+ token counts, the cost) in the plugin's state dir (`<project>/.omp` in project
286
+ mode, `agentDir()` otherwise), and that the explorer's `$/task` column then
287
+ fills for that role. A session with no completed turn writes no rows.
247
288
  - **CI/CD** — push a branch and open a PR (or push to `main`) and `gh run watch`
248
289
  the `test` job green; push a tag that disagrees with the package version and
249
290
  confirm the `release` job fails in the `notes` guard *before* any publish;
@@ -134,6 +134,17 @@ gating — that this plugin never reads (zero occurrences in `src/`, `web/`,
134
134
  `tests/`). The plugin's `features.<id>` keys are plugin settings, not omp
135
135
  feature flags; do not merge the two.
136
136
 
137
+ **`tokenProfile` (spec #81).** A global setting (`tokenProfile.inputTokens` /
138
+ `tokenProfile.outputTokens`, flat dotted keys like the rest) — the per-task
139
+ token profile the explorer's `$/task` column falls back to when a role has no
140
+ recorded usage. `null` (the default) = unset, and the column reads `—`. It is
141
+ validated to `null` or a record whose two fields are finite numbers ≥ 0
142
+ (`tokenProfile: must be null or an object with inputTokens/outputTokens` /
143
+ `tokenProfile.<key>: must be a number >= 0`). It is **display-only**: it never
144
+ feeds the ranking, the reported `$/M` price, or any write. The recorded usage
145
+ takes precedence over it (`resolveTokenProfile`), so once a role has ledger rows
146
+ the setting is ignored for that role.
147
+
137
148
  ## The provider-pin auto-configuration (spec #58)
138
149
 
139
150
  When a role's **expanded** def sets `preferOwnProvider: true` (the
@@ -209,6 +220,7 @@ settings entry the global behavior is unchanged.
209
220
  |---|---|---|
210
221
  | `llm-role-state.json` | `{ lastRunDay, managedRoles, roleLastSelector, pluginWrittenChainKeys, previousModelRoles, managedDisabledAgents }` | day gate, managed-role set, last selectors, plugin-owned chain keys, pre-write `modelRoles` snapshot, **managed `task.disabledAgents` names** |
211
222
  | `llm-role-history.jsonl` | one JSON row per completed run: `{ ts, trigger, keyMeta{isFreeTier, limitRemaining, creditsRemaining}, decisions[] }` | append-only decision log |
223
+ | `llm-role-usage.jsonl` | one JSON row per assistant turn: `{ ts, role, model, provider, input, output, cacheRead, cacheWrite, cost }` | the per-role usage ledger (spec #81) the explorer's per-task price resolves its token profile from |
212
224
  | `llm-role-projects.json` | `{ projects: [{ root, lastUsed }] }` | the plugin's memory of the projects where `/project-roles` was used (the explorer's scope list) |
213
225
  | `.llm-role-refresh.lock` | `O_EXCL` create, holds `"<pid> <iso>"` | serializes concurrent session starts; stale (> 60 s) locks are unlinked and retried once |
214
226
  | `llm-role-error-reports.json` | `{ reports: [{ fingerprint, label, title, firstSeen, lastSeen, createdDay?, offeredDay?, count, action, issueNumber?, issueUrl? }], dayCount: { day, count } }` | what the opt-in `errorReporting` path filed, offered or suppressed |
@@ -241,6 +253,24 @@ settings entry the global behavior is unchanged.
241
253
  that feeds the 3-creations-per-day cap. Nothing is written when
242
254
  `errorReporting` is `off`.
243
255
 
256
+ - `llm-role-usage.jsonl` is the per-role usage ledger (spec #81,
257
+ `src/usage-ledger.ts`). Unlike the two ledgers above it is **not** global: it
258
+ follows the plugin's state location — `<cwd>/.omp` in project mode,
259
+ `agentDir()` otherwise — so a project's usage stays separate from the
260
+ user-level profile. The extension records one row per assistant turn on
261
+ `turn_end` and `session_shutdown` (the backstop for a headless one-turn run),
262
+ read from `sessionManager.getBranch()` — the same entries the session jsonl
263
+ persists — into `resolveProjectDir() ?? agentDir()`, deduped by `ts|model` so
264
+ the two hooks cannot double-record. The file is **append-only** (a crash never
265
+ corrupts it) and **bounded**: `appendUsageRow` compacts to the newest
266
+ `USAGE_LEDGER_CAP = 2000` rows via a temp file + atomic rename, so a crash
267
+ mid-compaction leaves the original intact. Recording is **best-effort** — a
268
+ failure never breaks a session, and `appendUsageRow` never throws. Reading is
269
+ tolerant: a missing, unreadable or malformed file degrades to `[]`, and an
270
+ unparseable or wrongly-shaped line is skipped. The explorer reads the active
271
+ scope's ledger (a project scope its own `<root>/.omp`), so the per-task column
272
+ follows the scope.
273
+
244
274
  - `previousModelRoles` is the full snapshot of the last `modelRoles` block
245
275
  **before** the plugin changed it — a manual rollback aid; there is no rollback
246
276
  command.
@@ -1,4 +1,4 @@
1
- llm-stats.com best-fit ranking per omp model role — 408 models, 2026-10-12
1
+ llm-stats.com best-fit ranking per omp model role — 408 models, 2026-10-19
2
2
  Value ranking per role: each metric is cardinal-normalized with fixed anchors
3
3
  (no ranks): llm-stats index_* affine (v+20)/80 (interval scale, observed −16..+60),
4
4
  benchmarks chance-anchored pass rates, throughput log-anchored 10..300 tok/s.
@@ -21,179 +21,179 @@ agon=Design Arena `agents/agon_webapps` Elo from the same endpoint: context
21
21
  only, unweighted (22/87 of the designer pool, below the ~35–40% coverage
22
22
  bar at which a metric earns weight).
23
23
  ★ = Pareto-frontier: no eligible model is both cheaper and better (q).
24
- Throughput + price: OpenRouter per-provider routes under the default price-based routing (weight 1/price² over stable standard-tier providers; p50 tok/s of routed traffic; $/M 3:1 in:out), throughput 150/408, priced 148; models without OpenRouter throughput or a billed route are not ranked.
24
+ Throughput + price: OpenRouter per-provider routes under the default price-based routing (weight 1/price² over stable standard-tier providers; p50 tok/s of routed traffic; $/M 3:1 in:out), throughput 149/408, priced 148; models without OpenRouter throughput or a billed route are not ranked.
25
25
 
26
26
  ## @default — Main workhorse: strong general coding-agent quality, sane cost
27
27
  eligible: 140 — λ 0.00556 $/quality-point (thinking ×1.857)
28
28
 
29
29
  | # | value | q | model | org | $/M | tok/s | ctx | gen 54.7% | code 18.9% | ag 12.6% | tool 10.5% | tput 3.2% |
30
30
  | -: | ----: | ----: | --------------------- | ----------- | ----: | ----: | ---: | --------: | ---------: | -------: | ---------: | --------: |
31
- | 1 | 0.809 | 0.811 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.43 | 81 | 1.0M | 0.484 | 0.147 | 0.093 | 0.068 | 0.020 |
32
- | 2 | 0.797 | 0.809 | GLM-5.3 | Zhipu AI | $2.24 | 93 | 1.0M | 0.489 | 0.143 | 0.091 | 0.066 | 0.021 |
33
- | 3 | 0.796 | 0.817 | ★ Muse Spark 1.3 | Meta | $3.71 | 76 | 1.0M | 0.498 | 0.143 | 0.092 | 0.065 | 0.019 |
34
- | 4 | 0.778 | 0.784 | MiMo-V2.6-Pro | Xiaomi | $1.01 | 28 | 1.0M | 0.478 | 0.142 | 0.090 | 0.065 | 0.010 |
35
- | 5 | 0.776 | 0.790 | DeepSeek-V4-Pro-0813 | DeepSeek | $2.49 | 69 | 1.0M | 0.483 | 0.139 | 0.088 | 0.062 | 0.018 |
36
- | 6 | 0.766 | 0.829 | ★ GPT-5.6 Sol | OpenAI | $11.4 | 68 | 1.1M | 0.503 | 0.151 | 0.094 | 0.064 | 0.018 |
37
- | 7 | 0.765 | 0.809 | GPT-6.1 Sol | OpenAI | $7.96 | 35 | 1.1M | 0.506 | 0.143 | 0.093 | 0.056 | 0.012 |
38
- | 8 | 0.762 | 0.764 | ★ GLM-5.3-Flash | Zhipu AI | $0.36 | 65 | 1.0M | 0.472 | 0.124 | 0.087 | 0.064 | 0.017 |
39
- | 9 | 0.761 | 0.774 | Hy4 preview | Tencent | $2.32 | 32 | — | 0.477 | 0.137 | 0.087 | 0.062 | 0.011 |
40
- | 10 | 0.759 | 0.808 | Kimi K3 | Moonshot AI | $8.92 | 42 | 1.0M | 0.493 | 0.145 | 0.091 | 0.066 | 0.013 |
31
+ | 1 | 0.807 | 0.809 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.37 | 75 | 1.0M | 0.483 | 0.146 | 0.093 | 0.069 | 0.019 |
32
+ | 2 | 0.799 | 0.841 | ★ Gemini 4 Argon | Google | $7.65 | 38 | — | 0.516 | 0.149 | 0.096 | 0.068 | 0.012 |
33
+ | 3 | 0.794 | 0.806 | GLM-5.3 | Zhipu AI | $2.09 | 73 | 1.0M | 0.488 | 0.142 | 0.090 | 0.067 | 0.018 |
34
+ | 4 | 0.789 | 0.810 | ★ Muse Spark 1.3 | Meta | $3.71 | 70 | 1.0M | 0.495 | 0.139 | 0.091 | 0.067 | 0.018 |
35
+ | 5 | 0.776 | 0.793 | DeepSeek-V4-Pro-0813 | DeepSeek | $3.09 | 78 | 1.0M | 0.484 | 0.139 | 0.088 | 0.063 | 0.019 |
36
+ | 6 | 0.776 | 0.781 | MiMo-V2.6-Pro | Xiaomi | $1.01 | 29 | 1.0M | 0.476 | 0.141 | 0.089 | 0.066 | 0.010 |
37
+ | 7 | 0.773 | 0.811 | ★ Kimi K3 | Moonshot AI | $6.81 | 52 | 1.0M | 0.494 | 0.144 | 0.091 | 0.067 | 0.015 |
38
+ | 8 | 0.766 | 0.809 | GPT-6.1 Sol | OpenAI | $7.89 | 40 | 1.1M | 0.506 | 0.141 | 0.092 | 0.057 | 0.013 |
39
+ | 9 | 0.764 | 0.777 | Hy4 preview | Tencent | $2.32 | 37 | — | 0.478 | 0.137 | 0.087 | 0.063 | 0.012 |
40
+ | 10 | 0.763 | 0.765 | ★ GLM-5.3-Flash | Zhipu AI | $0.33 | 66 | 1.0M | 0.473 | 0.123 | 0.087 | 0.065 | 0.018 |
41
41
 
42
42
  ## @smol — Fast lightweight model: cheap and quick, still competent
43
43
  eligible: 140 — λ 0.04091 $/quality-point
44
44
 
45
- | # | value | q | model | org | $/M | tok/s | ctx | tput 40% | gen 28.6% | code 17.1% | tool 14.3% |
46
- | -: | ----: | ----: | ---------------------------- | ------------------------- | ----: | ----: | ---: | -------: | --------: | ---------: | ---------: |
47
- | 1 | 0.715 | 0.724 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.23 | 81 | 1.0M | 0.247 | 0.253 | 0.133 | 0.092 |
48
- | 2 | 0.691 | 0.692 | ★ Ling 3.0 Flash | InclusionAI | $0.04 | 169 | 131k | 0.332 | 0.200 | 0.095 | 0.064 |
49
- | 3 | 0.687 | 0.736 | ★ GLM-5.3 | Zhipu AI | $1.20 | 93 | 1.0M | 0.262 | 0.255 | 0.130 | 0.090 |
50
- | 4 | 0.672 | 0.690 | DeepSeek-V4-Flash-Vision-Exp | DeepSeek | $0.43 | 100 | 1.0M | 0.270 | 0.231 | 0.115 | 0.074 |
51
- | 5 | 0.658 | 0.666 | GLM-5.3-Flash | Zhipu AI | $0.19 | 65 | 1.0M | 0.221 | 0.246 | 0.113 | 0.086 |
52
- | 6 | 0.650 | 0.676 | Inkling-Small | Thinking Machines Lab | $0.64 | 123 | 524k | 0.295 | 0.207 | 0.104 | 0.070 |
53
- | 7 | 0.650 | 0.674 | Qwen3.8-27B | Alibaba Cloud / Qwen Team | $0.60 | 95 | 262k | 0.265 | 0.231 | 0.108 | 0.071 |
54
- | 8 | 0.648 | 0.667 | GPT-5.6 Luna | OpenAI | $0.47 | 80 | 1.1M | 0.245 | 0.229 | 0.117 | 0.076 |
55
- | 9 | 0.634 | 0.716 | Muse Spark 1.3 | Meta | $2.00 | 76 | 1.0M | 0.239 | 0.260 | 0.129 | 0.089 |
56
- | 10 | 0.634 | 0.689 | DeepSeek-V4-Pro-0813 | DeepSeek | $1.34 | 69 | 1.0M | 0.227 | 0.252 | 0.126 | 0.085 |
45
+ | # | value | q | model | org | $/M | tok/s | ctx | tput 40% | gen 28.6% | code 17.1% | tool 14.3% |
46
+ | -: | ----: | ----: | ---------------------------- | --------------------- | ----: | ----: | ---: | -------: | --------: | ---------: | ---------: |
47
+ | 1 | 0.706 | 0.714 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.20 | 75 | 1.0M | 0.237 | 0.252 | 0.132 | 0.094 |
48
+ | 2 | 0.687 | 0.713 | Inkling-Small | Thinking Machines Lab | $0.64 | 167 | 524k | 0.331 | 0.208 | 0.104 | 0.070 |
49
+ | 3 | 0.669 | 0.671 | ★ Ling 3.0 Flash | InclusionAI | $0.04 | 140 | 131k | 0.310 | 0.200 | 0.096 | 0.065 |
50
+ | 4 | 0.662 | 0.744 | ★ Muse Spark 1.1 | Meta | $2.00 | 127 | 1.0M | 0.298 | 0.245 | 0.113 | 0.087 |
51
+ | 5 | 0.661 | 0.669 | GLM-5.3-Flash | Zhipu AI | $0.18 | 66 | 1.0M | 0.223 | 0.247 | 0.112 | 0.088 |
52
+ | 6 | 0.661 | 0.707 | GLM-5.3 | Zhipu AI | $1.12 | 73 | 1.0M | 0.233 | 0.255 | 0.129 | 0.091 |
53
+ | 7 | 0.652 | 0.670 | DeepSeek-V4-Flash-Vision-Exp | DeepSeek | $0.43 | 83 | 1.0M | 0.249 | 0.231 | 0.115 | 0.075 |
54
+ | 8 | 0.652 | 0.733 | Muse Spark 1.2 | Meta | $2.00 | 224 | 1.0M | 0.366 | 0.204 | 0.102 | 0.062 |
55
+ | 9 | 0.647 | 0.667 | GPT-5.6 Luna | OpenAI | $0.47 | 80 | 1.1M | 0.244 | 0.229 | 0.117 | 0.076 |
56
+ | 10 | 0.643 | 0.707 | Gemini 3.8 Flash | Google | $1.57 | 95 | 1.0M | 0.264 | 0.247 | 0.119 | 0.076 |
57
57
 
58
58
  ## @slow — Most capable model for hard problems; cost and speed as tiebreakers
59
59
  eligible: 140 — λ 0.00263 $/quality-point (thinking ×2.714)
60
60
 
61
61
  | # | value | q | model | org | $/M | tok/s | ctx | gen 63.2% | code 18.9% | ag 13.7% | tput 4.2% |
62
62
  | -: | ----: | ----: | --------------------- | ----------- | ----: | ----: | ---: | --------: | ---------: | -------: | --------: |
63
- | 1 | 0.830 | 0.832 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.63 | 81 | 1.0M | 0.558 | 0.147 | 0.101 | 0.026 |
64
- | 2 | 0.828 | 0.842 | ★ Muse Spark 1.3 | Meta | $5.43 | 76 | 1.0M | 0.575 | 0.143 | 0.099 | 0.025 |
65
- | 3 | 0.824 | 0.833 | ★ GLM-5.3 | Zhipu AI | $3.27 | 93 | 1.0M | 0.564 | 0.143 | 0.098 | 0.028 |
66
- | 4 | 0.812 | 0.856 | ★ GPT-5.6 Sol | OpenAI | $16.6 | 68 | 1.1M | 0.580 | 0.151 | 0.101 | 0.024 |
67
- | 5 | 0.812 | 0.843 | ★ GPT-6.1 Sol | OpenAI | $11.6 | 35 | 1.1M | 0.584 | 0.143 | 0.100 | 0.015 |
68
- | 6 | 0.805 | 0.815 | DeepSeek-V4-Pro-0813 | DeepSeek | $3.64 | 69 | 1.0M | 0.557 | 0.139 | 0.095 | 0.024 |
69
- | 7 | 0.799 | 0.803 | MiMo-V2.6-Pro | Xiaomi | $1.47 | 28 | 1.0M | 0.551 | 0.142 | 0.097 | 0.013 |
70
- | 8 | 0.796 | 0.830 | Kimi K3 | Moonshot AI | $13 | 42 | 1.0M | 0.569 | 0.145 | 0.098 | 0.018 |
71
- | 9 | 0.787 | 0.796 | Hy4 preview | Tencent | $3.39 | 32 | — | 0.551 | 0.137 | 0.094 | 0.014 |
72
- | 10 | 0.785 | 0.786 | ★ GLM-5.3-Flash | Zhipu AI | $0.53 | 65 | 1.0M | 0.544 | 0.124 | 0.094 | 0.023 |
63
+ | 1 | 0.835 | 0.865 | ★ Gemini 4 Argon | Google | $11.2 | 38 | — | 0.595 | 0.149 | 0.104 | 0.016 |
64
+ | 2 | 0.827 | 0.829 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.54 | 75 | 1.0M | 0.558 | 0.146 | 0.100 | 0.025 |
65
+ | 3 | 0.819 | 0.827 | GLM-5.3 | Zhipu AI | $3.05 | 73 | 1.0M | 0.563 | 0.142 | 0.098 | 0.025 |
66
+ | 4 | 0.818 | 0.833 | ★ Muse Spark 1.3 | Meta | $5.43 | 70 | 1.0M | 0.572 | 0.139 | 0.099 | 0.024 |
67
+ | 5 | 0.812 | 0.842 | GPT-6.1 Sol | OpenAI | $11.5 | 40 | 1.1M | 0.584 | 0.141 | 0.100 | 0.017 |
68
+ | 6 | 0.806 | 0.833 | Kimi K3 | Moonshot AI | $9.95 | 52 | 1.0M | 0.570 | 0.144 | 0.098 | 0.020 |
69
+ | 7 | 0.806 | 0.818 | DeepSeek-V4-Pro-0813 | DeepSeek | $4.51 | 78 | 1.0M | 0.559 | 0.139 | 0.095 | 0.025 |
70
+ | 8 | 0.796 | 0.800 | MiMo-V2.6-Pro | Xiaomi | $1.47 | 29 | 1.0M | 0.549 | 0.141 | 0.097 | 0.013 |
71
+ | 9 | 0.792 | 0.852 | GPT-5.6 Sol | OpenAI | $22.9 | 54 | 1.1M | 0.580 | 0.150 | 0.101 | 0.021 |
72
+ | 10 | 0.790 | 0.798 | Hy4 preview | Tencent | $3.39 | 37 | — | 0.552 | 0.137 | 0.094 | 0.016 |
73
73
 
74
74
  ## @vision — Image understanding: vision index dominates
75
- eligible: 72 — λ 0.00682 $/quality-point (thinking ×1.857)
75
+ eligible: 73 — λ 0.00682 $/quality-point (thinking ×1.857)
76
76
 
77
77
  | # | value | q | model | org | $/M | tok/s | ctx | vis 54.2% | gen 20.8% | rea 15.6% | code 6.3% | tput 3.1% |
78
78
  | -: | ----: | ----: | --------------------- | ------------------------- | ----: | ----: | ---: | --------: | --------: | --------: | --------: | --------: |
79
- | 1 | 0.728 | 0.731 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.43 | 81 | 1.0M | 0.346 | 0.184 | 0.133 | 0.048 | 0.019 |
79
+ | 1 | 0.733 | 0.786 | ★ Gemini 4 Argon | Google | $7.65 | 38 | — | 0.385 | 0.196 | 0.143 | 0.049 | 0.012 |
80
80
  | 2 | 0.728 | 0.731 | ★ Qwen3.8 Flash | Alibaba Cloud / Qwen Team | $0.43 | 47 | 1.0M | 0.363 | 0.178 | 0.133 | 0.042 | 0.014 |
81
- | 3 | 0.717 | 0.772 | ★ GPT-6.1 Sol | OpenAI | $7.96 | 35 | 1.1M | 0.380 | 0.192 | 0.140 | 0.047 | 0.011 |
82
- | 4 | 0.715 | 0.741 | ★ Muse Spark 1.1 | Meta | $3.71 | 48 | 1.0M | 0.371 | 0.179 | 0.135 | 0.041 | 0.014 |
83
- | 5 | 0.712 | 0.715 | ★ GLM-5.3-Flash | Zhipu AI | $0.36 | 65 | 1.0M | 0.344 | 0.179 | 0.132 | 0.041 | 0.017 |
84
- | 6 | 0.711 | 0.730 | Gemini 3.8 Flash | Google | $2.79 | 74 | 1.0M | 0.361 | 0.179 | 0.128 | 0.044 | 0.018 |
85
- | 7 | 0.705 | 0.766 | Kimi K3 | Moonshot AI | $8.92 | 42 | 1.0M | 0.379 | 0.188 | 0.139 | 0.048 | 0.013 |
86
- | 8 | 0.705 | 0.712 | Qwen3.8-27B | Alibaba Cloud / Qwen Team | $1.11 | 95 | 262k | 0.358 | 0.168 | 0.126 | 0.039 | 0.021 |
87
- | 9 | 0.695 | 0.714 | Gemini 3.7 Flash | Google | $2.79 | 80 | 1.0M | 0.353 | 0.174 | 0.127 | 0.041 | 0.019 |
88
- | 10 | 0.692 | 0.770 | GPT-5.6 Sol | OpenAI | $11.4 | 68 | 1.1M | 0.369 | 0.191 | 0.142 | 0.050 | 0.018 |
81
+ | 3 | 0.727 | 0.730 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.37 | 75 | 1.0M | 0.345 | 0.184 | 0.134 | 0.048 | 0.018 |
82
+ | 4 | 0.724 | 0.750 | ★ Muse Spark 1.1 | Meta | $3.71 | 127 | 1.0M | 0.371 | 0.179 | 0.136 | 0.041 | 0.023 |
83
+ | 5 | 0.722 | 0.768 | ★ Kimi K3 | Moonshot AI | $6.81 | 52 | 1.0M | 0.378 | 0.188 | 0.139 | 0.048 | 0.015 |
84
+ | 6 | 0.715 | 0.769 | GPT-6.1 Sol | OpenAI | $7.89 | 40 | 1.1M | 0.377 | 0.193 | 0.140 | 0.047 | 0.013 |
85
+ | 7 | 0.714 | 0.734 | ★ Gemini 3.8 Flash | Google | $2.91 | 95 | 1.0M | 0.361 | 0.180 | 0.129 | 0.043 | 0.021 |
86
+ | 8 | 0.713 | 0.715 | ★ GLM-5.3-Flash | Zhipu AI | $0.33 | 66 | 1.0M | 0.344 | 0.180 | 0.133 | 0.041 | 0.017 |
87
+ | 9 | 0.702 | 0.707 | Qwen3.8-27B | Alibaba Cloud / Qwen Team | $0.72 | 55 | 262k | 0.358 | 0.168 | 0.126 | 0.039 | 0.016 |
88
+ | 10 | 0.697 | 0.717 | Gemini 3.7 Flash | Google | $2.87 | 86 | 1.0M | 0.353 | 0.176 | 0.127 | 0.041 | 0.020 |
89
89
 
90
90
  ## @plan — Planning: reasoning and long-context coherence, priced so the cheapest capable planner wins
91
91
  eligible: 140 — λ 0.01667 $/quality-point (thinking ×1.857)
92
92
 
93
- | # | value | q | model | org | $/M | tok/s | ctx | gen 80% | lc 15.6% | tput 4.4% |
94
- | -: | ----: | ----: | ------------------- | ------------------------- | ----: | ----: | ---: | ------: | -------: | --------: |
95
- | 1 | 0.794 | 0.856 | ★ Muse Spark 1.3 | Meta | $3.71 | 76 | 1.0M | 0.728 | 0.101 | 0.026 |
96
- | 2 | 0.775 | 0.782 | ★ Qwen3.8 Flash | Alibaba Cloud / Qwen Team | $0.43 | 47 | 1.0M | 0.683 | 0.079 | 0.020 |
97
- | 3 | 0.758 | 0.765 | DeepSeek-V4.1-Flash | DeepSeek | $0.43 | 81 | 1.0M | 0.707 | 0.030 | 0.027 |
98
- | 4 | 0.758 | 0.804 | ★ Gemini 3.8 Flash | Google | $2.79 | 74 | 1.0M | 0.687 | 0.091 | 0.026 |
99
- | 5 | 0.749 | 0.788 | ★ Hy4 preview | Tencent | $2.32 | 32 | — | 0.698 | 0.075 | 0.015 |
100
- | 6 | 0.747 | 0.794 | Gemini 3.7 Flash | Google | $2.79 | 80 | 1.0M | 0.669 | 0.098 | 0.027 |
101
- | 7 | 0.738 | 0.744 | ★ GLM-5.3-Flash | Zhipu AI | $0.36 | 65 | 1.0M | 0.689 | 0.030 | 0.025 |
102
- | 8 | 0.737 | 0.774 | GLM-5.3 | Zhipu AI | $2.24 | 93 | 1.0M | 0.715 | 0.030 | 0.029 |
103
- | 9 | 0.735 | 0.750 | GPT-5.6 Luna | OpenAI | $0.88 | 80 | 1.1M | 0.642 | 0.080 | 0.027 |
104
- | 10 | 0.725 | 0.742 | MiMo-V2.6-Pro | Xiaomi | $1.01 | 28 | 1.0M | 0.698 | 0.030 | 0.014 |
93
+ | # | value | q | model | org | $/M | tok/s | ctx | gen 80% | lc 15.6% | tput 4.4% |
94
+ | -: | ----: | ----: | --------------------- | ------------------------- | ----: | ----: | ---: | ------: | -------: | --------: |
95
+ | 1 | 0.789 | 0.851 | ★ Muse Spark 1.3 | Meta | $3.71 | 70 | 1.0M | 0.724 | 0.101 | 0.025 |
96
+ | 2 | 0.774 | 0.781 | ★ Qwen3.8 Flash | Alibaba Cloud / Qwen Team | $0.43 | 47 | 1.0M | 0.682 | 0.079 | 0.020 |
97
+ | 3 | 0.764 | 0.813 | ★ Gemini 3.8 Flash | Google | $2.91 | 95 | 1.0M | 0.692 | 0.091 | 0.029 |
98
+ | 4 | 0.757 | 0.763 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.37 | 75 | 1.0M | 0.707 | 0.030 | 0.026 |
99
+ | 5 | 0.755 | 0.802 | ★ Gemini 3.7 Flash | Google | $2.87 | 86 | 1.0M | 0.676 | 0.098 | 0.028 |
100
+ | 6 | 0.752 | 0.790 | ★ Hy4 preview | Tencent | $2.32 | 37 | — | 0.699 | 0.075 | 0.017 |
101
+ | 7 | 0.742 | 0.870 | ★ Gemini 4 Argon | Google | $7.65 | 38 | — | 0.754 | 0.098 | 0.017 |
102
+ | 8 | 0.740 | 0.746 | ★ GLM-5.3-Flash | Zhipu AI | $0.33 | 66 | 1.0M | 0.691 | 0.030 | 0.025 |
103
+ | 9 | 0.735 | 0.749 | GPT-5.6 Luna | OpenAI | $0.88 | 80 | 1.1M | 0.642 | 0.080 | 0.027 |
104
+ | 10 | 0.735 | 0.769 | GLM-5.3 | Zhipu AI | $2.09 | 73 | 1.0M | 0.713 | 0.030 | 0.026 |
105
105
 
106
106
  ## @commit — Commit messages: cheap and fast with decent general quality
107
107
  eligible: 140 — λ 0.04091 $/quality-point
108
108
 
109
- | # | value | q | model | org | $/M | tok/s | ctx | tput 41.5% | gen 43.1% | code 15.4% |
110
- | -: | ----: | ----: | ---------------------------- | ------------------------- | ----: | ----: | ---: | ---------: | --------: | ---------: |
111
- | 1 | 0.747 | 0.756 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.23 | 81 | 1.0M | 0.256 | 0.381 | 0.119 |
112
- | 2 | 0.731 | 0.733 | ★ Ling 3.0 Flash | InclusionAI | $0.04 | 169 | 131k | 0.345 | 0.302 | 0.086 |
113
- | 3 | 0.724 | 0.773 | ★ GLM-5.3 | Zhipu AI | $1.20 | 93 | 1.0M | 0.272 | 0.385 | 0.116 |
114
- | 4 | 0.715 | 0.732 | DeepSeek-V4-Flash-Vision-Exp | DeepSeek | $0.43 | 100 | 1.0M | 0.281 | 0.348 | 0.103 |
115
- | 5 | 0.695 | 0.719 | Qwen3.8-27B | Alibaba Cloud / Qwen Team | $0.60 | 95 | 262k | 0.275 | 0.348 | 0.097 |
116
- | 6 | 0.694 | 0.702 | GLM-5.3-Flash | Zhipu AI | $0.19 | 65 | 1.0M | 0.229 | 0.371 | 0.101 |
117
- | 7 | 0.687 | 0.713 | Inkling-Small | Thinking Machines Lab | $0.64 | 123 | 524k | 0.307 | 0.313 | 0.093 |
118
- | 8 | 0.686 | 0.706 | GPT-5.6 Luna | OpenAI | $0.47 | 80 | 1.1M | 0.254 | 0.346 | 0.105 |
119
- | 9 | 0.678 | 0.693 | Mercury 2 | Inception | $0.38 | 258 | 128k | 0.397 | 0.234 | 0.062 |
120
- | 10 | 0.676 | 0.747 | GLM-5.2 | Zhipu AI | $1.72 | 111 | 1.0M | 0.294 | 0.349 | 0.103 |
109
+ | # | value | q | model | org | $/M | tok/s | ctx | tput 41.5% | gen 43.1% | code 15.4% |
110
+ | -: | ----: | ----: | ---------------------------- | --------------------- | ----: | ----: | ---: | ---------: | --------: | ---------: |
111
+ | 1 | 0.736 | 0.745 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.20 | 75 | 1.0M | 0.246 | 0.380 | 0.118 |
112
+ | 2 | 0.724 | 0.750 | ★ Inkling-Small | Thinking Machines Lab | $0.64 | 167 | 524k | 0.344 | 0.313 | 0.094 |
113
+ | 3 | 0.709 | 0.710 | ★ Ling 3.0 Flash | InclusionAI | $0.04 | 140 | 131k | 0.322 | 0.302 | 0.086 |
114
+ | 4 | 0.699 | 0.781 | ★ Muse Spark 1.1 | Meta | $2.00 | 127 | 1.0M | 0.310 | 0.370 | 0.101 |
115
+ | 5 | 0.697 | 0.779 | Muse Spark 1.2 | Meta | $2.00 | 224 | 1.0M | 0.380 | 0.308 | 0.091 |
116
+ | 6 | 0.696 | 0.703 | GLM-5.3-Flash | Zhipu AI | $0.18 | 66 | 1.0M | 0.231 | 0.372 | 0.100 |
117
+ | 7 | 0.695 | 0.741 | GLM-5.3 | Zhipu AI | $1.12 | 73 | 1.0M | 0.242 | 0.384 | 0.115 |
118
+ | 8 | 0.692 | 0.710 | DeepSeek-V4-Flash-Vision-Exp | DeepSeek | $0.43 | 83 | 1.0M | 0.259 | 0.348 | 0.103 |
119
+ | 9 | 0.690 | 0.754 | ★ Gemini 3.8 Flash | Google | $1.57 | 95 | 1.0M | 0.275 | 0.373 | 0.106 |
120
+ | 10 | 0.685 | 0.704 | GPT-5.6 Luna | OpenAI | $0.47 | 80 | 1.1M | 0.254 | 0.346 | 0.105 |
121
121
 
122
122
  ## @tiny — Background tasks (titles, memory): cheapest and fastest wins
123
123
  eligible: 140 — λ 0.03333 $/quality-point
124
124
 
125
- | # | value | q | model | org | $/M | tok/s | ctx | tput 58.3% | gen 41.7% |
126
- | -: | ----: | ----: | ---------------------------- | ----------- | ----: | ----: | ---: | ---------: | --------: |
127
- | 1 | 0.775 | 0.777 | ★ Ling 3.0 Flash | InclusionAI | $0.04 | 169 | 131k | 0.485 | 0.292 |
128
- | 2 | 0.771 | 0.784 | ★ Mercury 2 | Inception | $0.38 | 258 | 128k | 0.557 | 0.226 |
129
- | 3 | 0.766 | 0.768 | Ling 3.0 Flash Fin | InclusionAI | $0.06 | 132 | 262k | 0.443 | 0.325 |
130
- | 4 | 0.730 | 0.759 | Gemini 3.5 Flash-Lite | Google | $0.89 | 187 | 1.0M | 0.502 | 0.258 |
131
- | 5 | 0.728 | 0.746 | Gemini 3.1 Flash-Lite | Google | $0.56 | 166 | 1.0M | 0.482 | 0.264 |
132
- | 6 | 0.720 | 0.728 | DeepSeek-V4.1-Flash | DeepSeek | $0.23 | 81 | 1.0M | 0.360 | 0.368 |
133
- | 7 | 0.718 | 0.720 | Laguna XS 2.1 | Poolside | $0.07 | 174 | 262k | 0.490 | 0.231 |
134
- | 8 | 0.717 | 0.731 | DeepSeek-V4-Flash-Vision-Exp | DeepSeek | $0.43 | 100 | 1.0M | 0.394 | 0.337 |
135
- | 9 | 0.714 | 0.754 | GLM-5.3 | Zhipu AI | $1.20 | 93 | 1.0M | 0.382 | 0.372 |
136
- | 10 | 0.713 | 0.716 | Nemotron 3 Nano (30B A3B) | NVIDIA | $0.09 | 187 | 262k | 0.502 | 0.213 |
125
+ | # | value | q | model | org | $/M | tok/s | ctx | tput 58.3% | gen 41.7% |
126
+ | -: | ----: | ----: | ----------------------------- | ------------------------- | ----: | ----: | ---: | ---------: | --------: |
127
+ | 1 | 0.769 | 0.782 | ★ Mercury 2 | Inception | $0.38 | 255 | 128k | 0.555 | 0.227 |
128
+ | 2 | 0.765 | 0.831 | ★ Muse Spark 1.2 | Meta | $2.00 | 224 | 1.0M | 0.533 | 0.298 |
129
+ | 3 | 0.764 | 0.786 | ★ Inkling-Small | Thinking Machines Lab | $0.64 | 167 | 524k | 0.483 | 0.303 |
130
+ | 4 | 0.743 | 0.745 | ★ Ling 3.0 Flash | InclusionAI | $0.04 | 140 | 131k | 0.453 | 0.292 |
131
+ | 5 | 0.738 | 0.756 | Gemini 3.1 Flash-Lite | Google | $0.56 | 176 | 1.0M | 0.492 | 0.265 |
132
+ | 6 | 0.726 | 0.793 | Muse Spark 1.1 | Meta | $2.00 | 127 | 1.0M | 0.435 | 0.358 |
133
+ | 7 | 0.717 | 0.768 | Gemini 3.6 Flash | Google | $1.54 | 132 | 1.0M | 0.443 | 0.325 |
134
+ | 8 | 0.707 | 0.713 | DeepSeek-V4.1-Flash | DeepSeek | $0.20 | 75 | 1.0M | 0.345 | 0.368 |
135
+ | 9 | 0.705 | 0.707 | Laguna XS 2.1 | Poolside | $0.07 | 161 | 262k | 0.477 | 0.231 |
136
+ | 10 | 0.702 | 0.743 | Qwen3-235B-A22B-Thinking-2507 | Alibaba Cloud / Qwen Team | $1.21 | 177 | — | 0.493 | 0.250 |
137
137
 
138
138
  ## @task — Subagents: agentic + tool calling, moderate cost sensitivity
139
139
  eligible: 140 — λ 0.00747 $/quality-point (thinking ×1.857)
140
140
 
141
141
  | # | value | q | model | org | $/M | tok/s | ctx | ag 29.9% | tool 20.7% | code 18.4% | gen 25.3% | tput 5.7% |
142
142
  | -: | ----: | ----: | --------------------- | ------------------------- | ----: | ----: | ---: | -------: | ---------: | ---------: | --------: | --------: |
143
- | 1 | 0.752 | 0.755 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.43 | 81 | 1.0M | 0.220 | 0.134 | 0.142 | 0.224 | 0.035 |
144
- | 2 | 0.730 | 0.746 | GLM-5.3 | Zhipu AI | $2.24 | 93 | 1.0M | 0.214 | 0.130 | 0.139 | 0.226 | 0.038 |
145
- | 3 | 0.721 | 0.748 | Muse Spark 1.3 | Meta | $3.71 | 76 | 1.0M | 0.217 | 0.128 | 0.139 | 0.230 | 0.034 |
146
- | 4 | 0.709 | 0.716 | MiMo-V2.6-Pro | Xiaomi | $1.01 | 28 | 1.0M | 0.212 | 0.128 | 0.138 | 0.221 | 0.018 |
147
- | 5 | 0.701 | 0.720 | DeepSeek-V4-Pro-0813 | DeepSeek | $2.49 | 69 | 1.0M | 0.207 | 0.122 | 0.135 | 0.223 | 0.033 |
148
- | 6 | 0.698 | 0.701 | ★ GLM-5.3-Flash | Zhipu AI | $0.36 | 65 | 1.0M | 0.206 | 0.125 | 0.121 | 0.218 | 0.032 |
149
- | 7 | 0.688 | 0.692 | Qwen3.8 Flash | Alibaba Cloud / Qwen Team | $0.43 | 47 | 1.0M | 0.199 | 0.127 | 0.124 | 0.216 | 0.026 |
150
- | 8 | 0.683 | 0.700 | Hy4 preview | Tencent | $2.32 | 32 | — | 0.205 | 0.123 | 0.133 | 0.221 | 0.019 |
151
- | 9 | 0.674 | 0.759 | ★ GPT-5.6 Sol | OpenAI | $11.4 | 68 | 1.1M | 0.222 | 0.126 | 0.146 | 0.232 | 0.032 |
152
- | 10 | 0.671 | 0.738 | Kimi K3 | Moonshot AI | $8.92 | 42 | 1.0M | 0.215 | 0.131 | 0.141 | 0.228 | 0.024 |
143
+ | 1 | 0.750 | 0.753 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.37 | 75 | 1.0M | 0.219 | 0.136 | 0.141 | 0.223 | 0.034 |
144
+ | 2 | 0.726 | 0.742 | GLM-5.3 | Zhipu AI | $2.09 | 73 | 1.0M | 0.213 | 0.131 | 0.138 | 0.225 | 0.033 |
145
+ | 3 | 0.716 | 0.743 | Muse Spark 1.3 | Meta | $3.71 | 70 | 1.0M | 0.215 | 0.132 | 0.134 | 0.229 | 0.033 |
146
+ | 4 | 0.710 | 0.767 | ★ Gemini 4 Argon | Google | $7.65 | 38 | — | 0.228 | 0.134 | 0.145 | 0.238 | 0.022 |
147
+ | 5 | 0.707 | 0.715 | MiMo-V2.6-Pro | Xiaomi | $1.01 | 29 | 1.0M | 0.211 | 0.129 | 0.137 | 0.220 | 0.018 |
148
+ | 6 | 0.702 | 0.725 | DeepSeek-V4-Pro-0813 | DeepSeek | $3.09 | 78 | 1.0M | 0.208 | 0.124 | 0.135 | 0.224 | 0.035 |
149
+ | 7 | 0.700 | 0.703 | ★ GLM-5.3-Flash | Zhipu AI | $0.33 | 66 | 1.0M | 0.206 | 0.127 | 0.120 | 0.218 | 0.032 |
150
+ | 8 | 0.692 | 0.743 | Kimi K3 | Moonshot AI | $6.81 | 52 | 1.0M | 0.215 | 0.132 | 0.140 | 0.228 | 0.028 |
151
+ | 9 | 0.688 | 0.691 | Qwen3.8 Flash | Alibaba Cloud / Qwen Team | $0.43 | 47 | 1.0M | 0.198 | 0.127 | 0.124 | 0.216 | 0.026 |
152
+ | 10 | 0.687 | 0.705 | Hy4 preview | Tencent | $2.32 | 37 | — | 0.205 | 0.124 | 0.133 | 0.221 | 0.022 |
153
153
 
154
154
  ## @advisor — Advisor/watchdog: deep reasoning over long context
155
155
  eligible: 140 — λ 0.01250 $/quality-point (thinking ×1.857)
156
156
 
157
157
  | # | value | q | model | org | $/M | tok/s | ctx | gen 74.3% | lc 12.6% | tput 13.1% |
158
158
  | -: | ----: | ----: | --------------------- | ------------------------- | ----: | ----: | ---: | --------: | -------: | ---------: |
159
- | 1 | 0.790 | 0.837 | ★ Muse Spark 1.3 | Meta | $3.71 | 76 | 1.0M | 0.677 | 0.082 | 0.078 |
160
- | 2 | 0.757 | 0.763 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.43 | 81 | 1.0M | 0.657 | 0.024 | 0.081 |
161
- | 3 | 0.754 | 0.789 | ★ Gemini 3.8 Flash | Google | $2.79 | 74 | 1.0M | 0.638 | 0.073 | 0.077 |
162
- | 4 | 0.752 | 0.758 | ★ Qwen3.8 Flash | Alibaba Cloud / Qwen Team | $0.43 | 47 | 1.0M | 0.635 | 0.063 | 0.060 |
163
- | 5 | 0.746 | 0.774 | ★ GLM-5.3 | Zhipu AI | $2.24 | 93 | 1.0M | 0.664 | 0.024 | 0.086 |
164
- | 6 | 0.746 | 0.780 | Gemini 3.7 Flash | Google | $2.79 | 80 | 1.0M | 0.621 | 0.079 | 0.080 |
165
- | 7 | 0.733 | 0.737 | ★ GLM-5.3-Flash | Zhipu AI | $0.36 | 65 | 1.0M | 0.640 | 0.024 | 0.072 |
166
- | 8 | 0.731 | 0.742 | GPT-5.6 Luna | OpenAI | $0.88 | 80 | 1.1M | 0.597 | 0.065 | 0.080 |
167
- | 9 | 0.724 | 0.753 | Hy4 preview | Tencent | $2.32 | 32 | — | 0.649 | 0.060 | 0.044 |
168
- | 10 | 0.723 | 0.755 | DeepSeek-V4-Pro-0813 | DeepSeek | $2.49 | 69 | 1.0M | 0.656 | 0.024 | 0.074 |
159
+ | 1 | 0.783 | 0.830 | ★ Muse Spark 1.3 | Meta | $3.71 | 70 | 1.0M | 0.673 | 0.082 | 0.075 |
160
+ | 2 | 0.767 | 0.803 | ★ Gemini 3.8 Flash | Google | $2.91 | 95 | 1.0M | 0.643 | 0.073 | 0.087 |
161
+ | 3 | 0.755 | 0.790 | ★ Gemini 3.7 Flash | Google | $2.87 | 86 | 1.0M | 0.628 | 0.079 | 0.083 |
162
+ | 4 | 0.754 | 0.759 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.37 | 75 | 1.0M | 0.657 | 0.024 | 0.078 |
163
+ | 5 | 0.752 | 0.757 | Qwen3.8 Flash | Alibaba Cloud / Qwen Team | $0.43 | 47 | 1.0M | 0.634 | 0.063 | 0.060 |
164
+ | 6 | 0.737 | 0.764 | ★ GLM-5.3 | Zhipu AI | $2.09 | 73 | 1.0M | 0.663 | 0.024 | 0.076 |
165
+ | 7 | 0.735 | 0.831 | ★ Gemini 4 Argon | Google | $7.65 | 38 | — | 0.700 | 0.079 | 0.051 |
166
+ | 8 | 0.735 | 0.739 | ★ GLM-5.3-Flash | Zhipu AI | $0.33 | 66 | 1.0M | 0.642 | 0.024 | 0.073 |
167
+ | 9 | 0.731 | 0.760 | Hy4 preview | Tencent | $2.32 | 37 | — | 0.649 | 0.060 | 0.050 |
168
+ | 10 | 0.730 | 0.741 | GPT-5.6 Luna | OpenAI | $0.88 | 80 | 1.1M | 0.597 | 0.065 | 0.080 |
169
169
 
170
170
  ## @designer — Design work: visual/UX judgement on image-capable models
171
- eligible: 86 — λ 0.01024 $/quality-point (thinking ×1.857)
171
+ eligible: 87 — λ 0.01024 $/quality-point (thinking ×1.857)
172
172
 
173
173
  | # | value | q | model | org | $/M | tok/s | ctx | gen 31.3% | code 7.2% | vis 21.7% | tput 18.1% | web 21.7% | agon |
174
174
  | -: | ----: | ----: | --------------------- | ----------- | ----: | ----: | ---: | --------: | --------: | --------: | ---------: | --------: | ---: |
175
- | 1 | 0.781 | 0.785 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.43 | 81 | 1.0M | 0.277 | 0.056 | 0.139 | 0.111 | 0.202 | — |
176
- | 2 | 0.751 | 0.789 | ★ Muse Spark 1.3 | Meta | $3.71 | 76 | 1.0M | 0.285 | 0.054 | 0.125 | 0.108 | 0.216 | 1289 |
177
- | 3 | 0.735 | 0.763 | Gemini 3.8 Flash | Google | $2.79 | 74 | 1.0M | 0.269 | 0.051 | 0.144 | 0.106 | 0.193 | 1231 |
178
- | 4 | 0.727 | 0.756 | Gemini 3.7 Flash | Google | $2.79 | 80 | 1.0M | 0.262 | 0.048 | 0.141 | 0.110 | 0.195 | 1215 |
179
- | 5 | 0.721 | 0.724 | ★ GLM-5.3-Flash | Zhipu AI | $0.36 | 65 | 1.0M | 0.270 | 0.047 | 0.138 | 0.100 | 0.169 | — |
180
- | 6 | 0.713 | 0.742 | Gemini 3.6 Flash | Google | $2.79 | 101 | 1.0M | 0.245 | 0.042 | 0.140 | 0.123 | 0.192 | 1188 |
181
- | 7 | 0.697 | 0.707 | MiMo-V2.6-Pro | Xiaomi | $1.01 | 28 | 1.0M | 0.273 | 0.054 | 0.127 | 0.055 | 0.197 | — |
182
- | 8 | 0.688 | 0.770 | GPT-6.1 Sol | OpenAI | $7.96 | 35 | 1.1M | 0.289 | 0.055 | 0.152 | 0.066 | 0.207 | — |
183
- | 9 | 0.688 | 0.779 | Kimi K3 | Moonshot AI | $8.92 | 42 | 1.0M | 0.282 | 0.055 | 0.152 | 0.076 | 0.214 | 1285 |
184
- | 10 | 0.678 | 0.795 | ★ GPT-5.6 Sol | OpenAI | $11.4 | 68 | 1.1M | 0.288 | 0.058 | 0.148 | 0.102 | 0.200 | 1184 |
185
- focus: website — coverage 125/408 (30.6%) ok; dispersion 0.992 ok; composition ok; freshness 0.0mo ok; trust unknown; modality unknown; maintenance unknown; provenance unknown; cross-source unknown
175
+ | 1 | 0.771 | 0.775 | ★ DeepSeek-V4.1-Flash | DeepSeek | $0.37 | 75 | 1.0M | 0.277 | 0.056 | 0.138 | 0.107 | 0.197 | — |
176
+ | 2 | 0.744 | 0.774 | Gemini 3.8 Flash | Google | $2.91 | 95 | 1.0M | 0.271 | 0.050 | 0.144 | 0.119 | 0.189 | 1236 |
177
+ | 3 | 0.741 | 0.779 | ★ Muse Spark 1.3 | Meta | $3.71 | 70 | 1.0M | 0.283 | 0.053 | 0.125 | 0.103 | 0.214 | 1296 |
178
+ | 4 | 0.730 | 0.760 | Gemini 3.7 Flash | Google | $2.87 | 86 | 1.0M | 0.265 | 0.047 | 0.141 | 0.114 | 0.192 | 1223 |
179
+ | 5 | 0.727 | 0.765 | Muse Spark 1.1 | Meta | $3.71 | 127 | 1.0M | 0.269 | 0.048 | 0.148 | 0.135 | 0.165 | — |
180
+ | 6 | 0.725 | 0.755 | Gemini 3.6 Flash | Google | $2.87 | 132 | 1.0M | 0.244 | 0.042 | 0.140 | 0.137 | 0.190 | 1194 |
181
+ | 7 | 0.721 | 0.724 | ★ GLM-5.3-Flash | Zhipu AI | $0.33 | 66 | 1.0M | 0.270 | 0.047 | 0.138 | 0.101 | 0.168 | — |
182
+ | 8 | 0.718 | 0.788 | ★ Kimi K3 | Moonshot AI | $6.81 | 52 | 1.0M | 0.283 | 0.055 | 0.152 | 0.088 | 0.211 | 1292 |
183
+ | 9 | 0.702 | 0.781 | Gemini 4 Argon | Google | $7.65 | 38 | — | 0.295 | 0.057 | 0.154 | 0.070 | 0.204 | — |
184
+ | 10 | 0.698 | 0.709 | MiMo-V2.6-Pro | Xiaomi | $1.01 | 29 | 1.0M | 0.272 | 0.054 | 0.127 | 0.056 | 0.199 | — |
185
+ focus: website — coverage 127/408 (31.1%) ok; dispersion 0.992 ok; composition ok; freshness 0.0mo ok; trust unknown; modality unknown; maintenance unknown; provenance unknown; cross-source unknown
186
186
 
187
187
  # Suggested settings.modelRoles (best-fit #1 per role, resolved via the omp catalog).
188
188
  # Selectors are openrouter/<id>; the omp-llm-role plugin writes this block daily.
189
189
  modelRoles:
190
190
  default: "openrouter/deepseek/deepseek-v4.1-flash"
191
191
  smol: "openrouter/deepseek/deepseek-v4.1-flash"
192
- slow: "openrouter/deepseek/deepseek-v4.1-flash"
193
- vision: "openrouter/deepseek/deepseek-v4.1-flash"
192
+ slow: "openrouter/google/gemini-4-argon"
193
+ vision: "openrouter/google/gemini-4-argon"
194
194
  plan: "openrouter/meta/muse-spark-1.3"
195
195
  commit: "openrouter/deepseek/deepseek-v4.1-flash"
196
- tiny: "openrouter/inclusionai/ling-3.0-flash"
196
+ tiny: "openrouter/inception/mercury-2"
197
197
  task: "openrouter/deepseek/deepseek-v4.1-flash"
198
198
  advisor: "openrouter/meta/muse-spark-1.3"
199
199
  designer: "openrouter/deepseek/deepseek-v4.1-flash"
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "omp-llm-role",
3
- "version": "1.2.0",
3
+ "version": "1.3.0",
4
4
  "type": "module",
5
5
  "description": "omp plugin: daily model-role updater driven by llm-stats/OpenRouter rankings",
6
6
  "license": "MIT",
@@ -78,6 +78,16 @@
78
78
  "auto"
79
79
  ]
80
80
  },
81
+ "tokenProfile.inputTokens": {
82
+ "type": "number",
83
+ "description": "Input tokens per task for the per-task price (used when the role has no recorded usage)",
84
+ "min": 0
85
+ },
86
+ "tokenProfile.outputTokens": {
87
+ "type": "number",
88
+ "description": "Output tokens per task for the per-task price (used when the role has no recorded usage)",
89
+ "min": 0
90
+ },
81
91
  "features.endpointCeilings": {
82
92
  "type": "boolean",
83
93
  "description": "Drop routes that cannot call tools and routes whose output ceiling is below 16 384 tokens",
@@ -424,8 +424,10 @@ function coveredValues(models: readonly Model[], metric: string, fill: number):
424
424
  return values;
425
425
  }
426
426
 
427
- /** Linear-interpolated quantile (R type 7) of a value list. */
428
- function quantile(values: readonly number[], p: number): number {
427
+ /** Linear-interpolated quantile (R type 7) of a value list — the repo's one
428
+ * quantile convention: the focus-axis dispersion stats and the usage ledger's
429
+ * per-turn median (`quantile(values, 0.5)`) both read it. */
430
+ export function quantile(values: readonly number[], p: number): number {
429
431
  const sorted = [...values].sort((a, b) => a - b);
430
432
  const h = (sorted.length - 1) * p;
431
433
  const lo = Math.floor(h);
package/src/engine.ts CHANGED
@@ -313,6 +313,19 @@ export type OpenRouterEndpointPages = Record<string, OpenRouterEndpointRecord[]>
313
313
 
314
314
  export type OrEnrichment = { tput: number; latency: number | null; price: number | null; thinking: boolean; routes: OpenRouterEndpointRecord[] };
315
315
 
316
+ /** The route's input (prompt) $/M under an assumed cache-hit rate: the input term
317
+ * `routeEffectivePrice` blends into the billed price. A hit rate `h` blends the
318
+ * cache-read input in, `h·cacheRead + (1−h)·input`; with `h = 0` or no cache-read
319
+ * price this is the listed input price, and a route with no input price has none —
320
+ * missing data keeps the full input price, never a silent discount. The single
321
+ * accessor for the input term: the billed blend, the output-price inverse
322
+ * (`routeOutputPrice`) and the explorer's per-task price all read it. */
323
+ export function routeInputPrice(route: OpenRouterEndpointRecord, cacheHitRate = 0): number | null {
324
+ if (route.weightPrice == null) return null;
325
+ if (cacheHitRate <= 0 || route.cacheReadPrice == null) return route.weightPrice;
326
+ return cacheHitRate * route.cacheReadPrice + (1 - cacheHitRate) * route.weightPrice;
327
+ }
328
+
316
329
  /** The effective billed $/M of one route under an assumed cache-hit rate. The route's
317
330
  * billed 3:1 price is `price = (3·input + output)/4` on the listed input price; a hit
318
331
  * rate `h` blends the cache-read input into the input term,
@@ -320,15 +333,33 @@ export type OrEnrichment = { tput: number; latency: number | null; price: number
320
333
  * (the output term is unchanged). With `h = 0`, no cache-read price, or no input price
321
334
  * this is the route's listed `price` — missing data keeps the full input price, never a
322
335
  * silent discount. Shared by the blend (`blendRoutePool`), the pin path (`rankRole`) and
323
- * `roleMetricValue`. */
336
+ * `roleMetricValue`; the input term itself is `routeInputPrice`. */
324
337
  export function routeEffectivePrice(route: OpenRouterEndpointRecord, cacheHitRate = 0): number | null {
325
338
  if (route.price == null) return null;
326
339
  if (cacheHitRate <= 0 || route.cacheReadPrice == null || route.weightPrice == null) return route.price;
327
340
  const input = route.weightPrice;
328
- const effInput = cacheHitRate * route.cacheReadPrice + (1 - cacheHitRate) * input;
341
+ const effInput = routeInputPrice(route, cacheHitRate) as number;
329
342
  return route.price + (3 * (effInput - input)) / 4;
330
343
  }
331
344
 
345
+ /** The output (completion) $/M implied by a billed 3:1 blend and the input price
346
+ * behind it: the inverse of `billed = IO_PRICE_RATIO·(3·input + output)`, i.e.
347
+ * `output = billed/IO_PRICE_RATIO − (1−IO_PRICE_RATIO)/IO_PRICE_RATIO · input`.
348
+ * The 3:1 ratio lives here (and in `IO_PRICE_RATIO`), not in the callers; dividing
349
+ * by `IO_PRICE_RATIO` (2⁻²) is exact, so this is the plain `4·billed − 3·input`. */
350
+ export function blendOutputPrice(billed: number, input: number): number {
351
+ return billed / IO_PRICE_RATIO - ((1 - IO_PRICE_RATIO) / IO_PRICE_RATIO) * input;
352
+ }
353
+
354
+ /** The route's output (completion) $/M: the inverse blend of its own billed and
355
+ * input prices (see `blendOutputPrice`). Null when either of them is missing. */
356
+ export function routeOutputPrice(route: OpenRouterEndpointRecord, cacheHitRate = 0): number | null {
357
+ const billed = routeEffectivePrice(route, cacheHitRate);
358
+ const input = routeInputPrice(route, cacheHitRate);
359
+ if (billed === null || input === null) return null;
360
+ return blendOutputPrice(billed, input);
361
+ }
362
+
332
363
  /** Expected price/throughput/latency of one request over a route pool under the
333
364
  * router's default price-based load balancing: P(route i) ∝ 1/input_price_i² — the
334
365
  * router sorts on the input (prompt) price alone, so the weight basis is