llm-orchestrator 1.2.7 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/README.md +9 -7
- package/lib/flow-gate.mjs +16 -4
- package/models/model-thinking-data.json +94 -2
- package/models/model-thinking-matrix.md +23 -3
- package/models/top-models.json +278 -7
- package/package.json +1 -1
- package/policies/routing.md +4 -4
- package/registries/routing-matrix.json +17 -16
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "llm-orchestrator",
|
|
3
3
|
"description": "Write /task once — it plans the work, shards it across parallel subagents, gates every phase and verifies before claiming done. Claude Code, Codex, OpenCode, Kilo.",
|
|
4
|
-
"version": "1.
|
|
4
|
+
"version": "1.3.0",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Bogdan-Gabriel Torcescu",
|
|
7
7
|
"url": "https://www.linkedin.com/in/bogdantorcescu/"
|
package/README.md
CHANGED
|
@@ -365,8 +365,8 @@ more instructions:
|
|
|
365
365
|
it, runs opened without a PlanShard count, runs that planned several shards but started no
|
|
366
366
|
subagents, and runs still open.
|
|
367
367
|
- Trivial is the narrow exception — a one-line change such as a typo or a version bump. A run
|
|
368
|
-
declared trivial that then edits a second file, touches tests,
|
|
369
|
-
(an investigation is not trivial) gets one reminder to reopen it as
|
|
368
|
+
declared trivial that then edits a second file, touches tests, queries a live source twice, or keeps
|
|
369
|
+
working past eight calls (an investigation is not trivial) gets one reminder to reopen it as
|
|
370
370
|
a typed run, and counts as `trivial_overreach` in the audit. Trivial runs close at the end of the
|
|
371
371
|
turn (`Stop` hook), so single-prompt sessions still reach the history. Typed runs stay open across
|
|
372
372
|
turns and close when the session ends (`SessionEnd`), so an unclosed run is never lost.
|
|
@@ -429,7 +429,7 @@ This resolves (task type, phase, role, risk, complexity, context size, harness)
|
|
|
429
429
|
|
|
430
430
|
## Cost-aware routing (built in)
|
|
431
431
|
|
|
432
|
-
Routing is data, not prose: `registries/routing-matrix.json` encodes the W/S/X/F tiers, the T0–T5 thinking levels with their per-provider controls (`effort` on Claude, `reasoning_effort` on Codex, budget-only on Haiku), the tier × thinking resolution table, default routing, the per-task-type flows (phase → pair → roles → gate), 21 risk-floor areas with implementation and independent-review floors, agent defaults, both escalation ladders, fan-out minimums (2 MODERATE / 3 COMPLEX / 4 CRITICAL), the target tier distribution and the quota degradation ladder. `models/top-models.json` carries the curated
|
|
432
|
+
Routing is data, not prose: `registries/routing-matrix.json` encodes the W/S/X/F tiers, the T0–T5 thinking levels with their per-provider controls (`effort` on Claude, `reasoning_effort` on Codex, budget-only on Haiku), the tier × thinking resolution table, default routing, the per-task-type flows (phase → pair → roles → gate), 21 risk-floor areas with implementation and independent-review floors, agent defaults, both escalation ladders, fan-out minimums (2 MODERATE / 3 COMPLEX / 4 CRITICAL), the target tier distribution and the quota degradation ladder. `models/top-models.json` carries the curated shortlist — thirteen ladder incumbents (three of them fallbacks for their successors) and ten measured candidates — with their supported thinking levels, prices, measured $/task per effort and thinking-cost indices (Artificial Analysis snapshot, copied verbatim, never invented). `lib/router.mjs` resolves a dispatch; `bin/route.mjs` exposes it:
|
|
433
433
|
|
|
434
434
|
```sh
|
|
435
435
|
node bin/llm-orchestrator.mjs route --list # vocab: task types, phases, roles, areas
|
|
@@ -448,20 +448,22 @@ Resolution order: default routing → agent default → task-flow phase → comp
|
|
|
448
448
|
|
|
449
449
|
The CLI resolves one dispatch. Inside a flow, the same resolution runs **for every PlanShard, at dispatch time, against the live inventory** — `lib/dispatch-contract.mjs` exposes `buildShardRouting(shard, options)` for one shard, `buildShardContracts(flow, { inventory, harness, includeCandidates })` for all of them (returning the flow ledger: tier histogram against the target distribution, mean `$/task`, blocked shards, warnings), and `rerouteRemaining(flow, inventory)` when a model-not-found, a rejected effort or a quota change invalidates the inventory mid-flow — remaining shards only, so the ledger stays true. Each shard carries a `routing` block with exactly these thirteen fields: `pair`, `tier`, `thinking_level`, `model_requested`, `effort_requested`, `model_effective`, `effort_effective`, `review_floor`, `independent_review`, `selection_reason`, `inventory_revision`, `price_source`, `est_usd_per_task`. An inventory that exposes nothing eligible for the resolved tier yields `blocked: "no eligible model"` — the floor is never lowered to fit what happens to be available.
|
|
450
450
|
|
|
451
|
-
###
|
|
451
|
+
### Shortlisted models and thinking levels (snapshot 2026-09-23)
|
|
452
452
|
|
|
453
453
|
Incumbents hold a seat on a provider ladder. Candidates are measured but unseated: their tier is a `tier_bands` placement from the best measured Artificial Analysis score (W ≤ 37, S 38–44, X 45–50, F ≥ 51), and they rank only with `--include-candidates` or an inventory that exposes them.
|
|
454
454
|
|
|
455
455
|
| Model | Provider | Ladder | Admission | Tier | Thinking levels | $ in / out per MTok |
|
|
456
456
|
|---|---|---|---|---|---|---|
|
|
457
|
-
| GPT-
|
|
457
|
+
| GPT-6 Luna | openai | codex | incumbent | W | low, medium, high, xhigh, max (reasoning_effort) | $0.1 / $0.5 |
|
|
458
|
+
| GPT-5.6 Luna | openai | codex | incumbent (fallback for GPT-6 Luna) | W | low, medium, high, xhigh, max (reasoning_effort) | $0.2 / $1.2 |
|
|
458
459
|
| GPT-5.6 Terra | openai | codex | incumbent | S | low, medium, high, xhigh, max (reasoning_effort) | $2 / $12 |
|
|
459
|
-
| GPT-
|
|
460
|
+
| GPT-6 Sol | openai | codex | incumbent | X | low, medium, high, xhigh, max (reasoning_effort) | $2 / $10 |
|
|
461
|
+
| GPT-5.6 Sol | openai | codex | incumbent (fallback for GPT-6 Sol) | X | low, medium, high, xhigh, max (reasoning_effort) | $4 / $20 |
|
|
460
462
|
| GPT-6 Astra | openai | codex | incumbent | F | low, medium, high, xhigh, max (reasoning_effort) | $10 / $50 |
|
|
461
463
|
| Claude Haiku 4.5 | anthropic | claude | incumbent | W | disabled, enabled (budget_tokens) | $1 / $5 |
|
|
462
464
|
| Claude Sonnet 5 | anthropic | claude | incumbent | S | low, medium, high, xhigh, max (effort) | $2 / $10 |
|
|
463
465
|
| Claude Opus 5 | anthropic | claude | incumbent (fallback for Opus 5.5) | X | low, medium, high, xhigh, max (effort) | $5 / $25 |
|
|
464
|
-
| Claude Opus 5.5 | anthropic | claude | incumbent | X |
|
|
466
|
+
| Claude Opus 5.5 | anthropic | claude | incumbent | X | low, medium, high, xhigh, max (effort) | $4 / $20 |
|
|
465
467
|
| Claude Fable 5 | anthropic | claude | incumbent | F | low, medium, high, xhigh, max (effort) | $10 / $50 |
|
|
466
468
|
| Claude Fable 5.1 | anthropic | claude | incumbent | F | low, medium, high, xhigh, max (effort) | $10 / $50 |
|
|
467
469
|
| Grok 4.7 | xai | — | incumbent | — (unrated) | low, medium, high, xhigh (reasoning_effort) | $2 / $6 |
|
package/lib/flow-gate.mjs
CHANGED
|
@@ -52,6 +52,11 @@ export function overreachWorkNudgeFor(cli = 'llm-orchestrator', calls = 9) {
|
|
|
52
52
|
return `This task was declared trivial, but it has now made ${calls} work calls — trivial is a one-line change, not an investigation. Reopen it as a typed run (\`${cli} run start --type <TYPE> --shards <n>\`) so it is classified, planned and verified like one.`;
|
|
53
53
|
}
|
|
54
54
|
|
|
55
|
+
/** A run declared trivial that is querying live sources. */
|
|
56
|
+
export function overreachLiveNudgeFor(cli = 'llm-orchestrator', live = 2) {
|
|
57
|
+
return `This task was declared trivial, but it has now queried live sources ${live} times — an investigation over production evidence is not trivial. Reopen it as a typed run (\`${cli} run start --type INCIDENT|INVESTIGATION --shards <n>\`), where independent live sources are shards for collectors.`;
|
|
58
|
+
}
|
|
59
|
+
|
|
55
60
|
// Edit-shaped tools, per harness. Only a short hash of each path is kept.
|
|
56
61
|
const EDIT_TOOLS = new Set(['edit', 'write', 'multiedit', 'notebookedit', 'str_replace_based_edit_tool', 'apply_patch', 'patch']);
|
|
57
62
|
const TEST_PATH = /(^|\/)(test|tests|__tests__|spec|specs)\/|[._-](test|spec)\.[a-z0-9]+$|(^|\/)test_[^/]+\.py$|_spec\.rb$/i;
|
|
@@ -122,6 +127,8 @@ export const INLINE_REASON = /^stateful:\s*\S/;
|
|
|
122
127
|
|
|
123
128
|
// A: a trivial run is a one-line change, not an investigation.
|
|
124
129
|
export const TRIVIAL_WORK_LIMIT = 8;
|
|
130
|
+
// Querying a live source more than once is an investigation, however few calls it takes.
|
|
131
|
+
export const TRIVIAL_LIVE_LIMIT = 2;
|
|
125
132
|
|
|
126
133
|
// C: sources whose reads cost real time and live remotely — the reads that pay to fan out.
|
|
127
134
|
const LIVE_SOURCE = /(^|[\s;&|(])(rtk\s+)?(ssh|scp|psql|mysql|mongosh|mongo|redis-cli|kubectl|docker\s+(exec|logs)|aws|gcloud|az|curl|wget|http|httpie)(\s|$)/;
|
|
@@ -428,20 +435,24 @@ export function decide(previous, event, now) {
|
|
|
428
435
|
if (session.run) {
|
|
429
436
|
const current = session.run;
|
|
430
437
|
current.main_work_calls = (current.main_work_calls ?? 0) + 1;
|
|
438
|
+
// Live reads count on every run, trivial ones included — they are what fan-out is for.
|
|
439
|
+
if (event.live) current.live_calls = (current.live_calls ?? 0) + 1;
|
|
431
440
|
if (current.trivial) {
|
|
432
441
|
current.edited_files = [...new Set([...(current.edited_files ?? []), ...event.edits.map((entry) => entry.hash)])];
|
|
433
442
|
current.touched_tests = Boolean(current.touched_tests) || event.edits.some((entry) => entry.test);
|
|
434
443
|
const grewByEdits = current.edited_files.length >= 2 || current.touched_tests;
|
|
435
444
|
const grewByWork = current.main_work_calls > TRIVIAL_WORK_LIMIT;
|
|
436
|
-
|
|
445
|
+
const grewByLive = (current.live_calls ?? 0) >= TRIVIAL_LIVE_LIMIT;
|
|
446
|
+
if (!current.overreach_nudged && (grewByEdits || grewByWork || grewByLive)) {
|
|
437
447
|
current.overreach_nudged = true;
|
|
438
448
|
output = grewByEdits
|
|
439
449
|
? { additionalContext: overreachNudgeFor(undefined, current.edited_files.length), kind: 'overreach', files: current.edited_files.length }
|
|
440
|
-
:
|
|
450
|
+
: grewByLive
|
|
451
|
+
? { additionalContext: overreachLiveNudgeFor(undefined, current.live_calls), kind: 'overreach', live: current.live_calls }
|
|
452
|
+
: { additionalContext: overreachWorkNudgeFor(undefined, current.main_work_calls), kind: 'overreach', calls: current.main_work_calls };
|
|
441
453
|
}
|
|
442
454
|
return { session, output, history };
|
|
443
455
|
}
|
|
444
|
-
if (event.live) current.live_calls = (current.live_calls ?? 0) + 1;
|
|
445
456
|
const evidence = EVIDENCE_TYPES.has(current.task_type);
|
|
446
457
|
const step = (evidence ? EVIDENCE_CALLS_PER_SHARD : DISPATCH_CALLS_PER_SHARD) * (current.planned_shards ?? 0);
|
|
447
458
|
const nudges = current.dispatch_nudges ?? (current.dispatch_nudged ? 1 : 0);
|
|
@@ -593,7 +604,8 @@ export async function handleHook({ payload, project, now = Date.now(), cli = 'll
|
|
|
593
604
|
// decide() speaks in the default CLI spelling; the hook swaps in the runnable path.
|
|
594
605
|
const text = result.output.kind === 'dispatch' ? dispatchNudgeFor(cli, result.output.planned, result.output.taskType)
|
|
595
606
|
: result.output.kind === 'dispatch_followup' ? dispatchFollowupFor(cli, result.output.planned, result.output.taskType)
|
|
596
|
-
: result.output.kind === 'overreach' ? (result.output.
|
|
607
|
+
: result.output.kind === 'overreach' ? (result.output.live ? overreachLiveNudgeFor(cli, result.output.live)
|
|
608
|
+
: result.output.calls ? overreachWorkNudgeFor(cli, result.output.calls) : overreachNudgeFor(cli, result.output.files))
|
|
597
609
|
: nudgeFor(cli);
|
|
598
610
|
return { hookSpecificOutput: { hookEventName: 'PreToolUse', additionalContext: text } };
|
|
599
611
|
}
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"_attribution": "llm-orchestrator · created by Bogdan-Gabriel Torcescu · https://www.linkedin.com/in/bogdantorcescu/ · keep this credit when copying or deriving",
|
|
3
3
|
"schema_version": 1,
|
|
4
|
-
"observed_at": "2026-09-
|
|
4
|
+
"observed_at": "2026-09-23",
|
|
5
5
|
"benchmark": {
|
|
6
6
|
"name": "Artificial Analysis Intelligence Index",
|
|
7
7
|
"version": "v4.3.2",
|
|
@@ -54,6 +54,42 @@
|
|
|
54
54
|
]
|
|
55
55
|
]
|
|
56
56
|
},
|
|
57
|
+
{
|
|
58
|
+
"key": "gpt-6-luna",
|
|
59
|
+
"display_name": "GPT-6 Luna",
|
|
60
|
+
"provider": "OpenAI",
|
|
61
|
+
"api_id": "gpt-6-luna",
|
|
62
|
+
"input_usd_per_mtok": 0.1,
|
|
63
|
+
"output_usd_per_mtok": 0.5,
|
|
64
|
+
"source": "https://artificialanalysis.ai/models/gpt-6-luna",
|
|
65
|
+
"measurements": [
|
|
66
|
+
[
|
|
67
|
+
"low",
|
|
68
|
+
21,
|
|
69
|
+
0.0045
|
|
70
|
+
],
|
|
71
|
+
[
|
|
72
|
+
"medium",
|
|
73
|
+
29,
|
|
74
|
+
0.02
|
|
75
|
+
],
|
|
76
|
+
[
|
|
77
|
+
"high",
|
|
78
|
+
32,
|
|
79
|
+
0.03
|
|
80
|
+
],
|
|
81
|
+
[
|
|
82
|
+
"xhigh",
|
|
83
|
+
34,
|
|
84
|
+
0.04
|
|
85
|
+
],
|
|
86
|
+
[
|
|
87
|
+
"max",
|
|
88
|
+
37,
|
|
89
|
+
0.07
|
|
90
|
+
]
|
|
91
|
+
]
|
|
92
|
+
},
|
|
57
93
|
{
|
|
58
94
|
"key": "gpt-5-6-terra",
|
|
59
95
|
"display_name": "GPT-5.6 Terra",
|
|
@@ -126,6 +162,42 @@
|
|
|
126
162
|
]
|
|
127
163
|
]
|
|
128
164
|
},
|
|
165
|
+
{
|
|
166
|
+
"key": "gpt-6-sol",
|
|
167
|
+
"display_name": "GPT-6 Sol",
|
|
168
|
+
"provider": "OpenAI",
|
|
169
|
+
"api_id": "gpt-6-sol",
|
|
170
|
+
"input_usd_per_mtok": 2,
|
|
171
|
+
"output_usd_per_mtok": 10,
|
|
172
|
+
"source": "https://artificialanalysis.ai/models/gpt-6-sol",
|
|
173
|
+
"measurements": [
|
|
174
|
+
[
|
|
175
|
+
"low",
|
|
176
|
+
34,
|
|
177
|
+
0.13
|
|
178
|
+
],
|
|
179
|
+
[
|
|
180
|
+
"medium",
|
|
181
|
+
40,
|
|
182
|
+
0.25
|
|
183
|
+
],
|
|
184
|
+
[
|
|
185
|
+
"high",
|
|
186
|
+
43,
|
|
187
|
+
0.37
|
|
188
|
+
],
|
|
189
|
+
[
|
|
190
|
+
"xhigh",
|
|
191
|
+
44,
|
|
192
|
+
0.53
|
|
193
|
+
],
|
|
194
|
+
[
|
|
195
|
+
"max",
|
|
196
|
+
48,
|
|
197
|
+
1.06
|
|
198
|
+
]
|
|
199
|
+
]
|
|
200
|
+
},
|
|
129
201
|
{
|
|
130
202
|
"key": "gpt-6-astra",
|
|
131
203
|
"display_name": "GPT-6 Astra",
|
|
@@ -242,8 +314,28 @@
|
|
|
242
314
|
"input_usd_per_mtok": 4,
|
|
243
315
|
"output_usd_per_mtok": 20,
|
|
244
316
|
"source": "https://artificialanalysis.ai/models/claude-opus-5-5",
|
|
245
|
-
"measurement_note": "
|
|
317
|
+
"measurement_note": "All displayed scores include the benchmark default fallback and are not standalone scores.",
|
|
246
318
|
"measurements": [
|
|
319
|
+
[
|
|
320
|
+
"low",
|
|
321
|
+
42,
|
|
322
|
+
0.55
|
|
323
|
+
],
|
|
324
|
+
[
|
|
325
|
+
"medium",
|
|
326
|
+
51,
|
|
327
|
+
1.34
|
|
328
|
+
],
|
|
329
|
+
[
|
|
330
|
+
"high",
|
|
331
|
+
54,
|
|
332
|
+
1.82
|
|
333
|
+
],
|
|
334
|
+
[
|
|
335
|
+
"xhigh",
|
|
336
|
+
56,
|
|
337
|
+
3.46
|
|
338
|
+
],
|
|
247
339
|
[
|
|
248
340
|
"max",
|
|
249
341
|
58,
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
<!-- llm-orchestrator · created by Bogdan-Gabriel Torcescu · https://www.linkedin.com/in/bogdantorcescu/ · keep this credit when copying or deriving -->
|
|
2
2
|
# Model thinking matrix — AA v4.3.2 snapshot
|
|
3
3
|
|
|
4
|
-
Observed: 2026-09-
|
|
4
|
+
Observed: 2026-09-23. Scores and weighted USD/task costs come from the Artificial Analysis Intelligence Index v4.3.2 snapshot, not live account tariffs or a guarantee of repository outcomes. [Overall leaderboard](https://artificialanalysis.ai/leaderboards/models).
|
|
5
5
|
|
|
6
6
|
This report compares measurements. It cannot automatically select a model, establish account availability, or replace security, payment, migration, concurrency, or compatibility review floors.
|
|
7
7
|
|
|
@@ -18,6 +18,11 @@ Benchmark cost index: `100 × observed benchmark cost / $0.50`; GPT-5.6 Sol medi
|
|
|
18
18
|
| gpt-5-6-luna | GPT-5.6 Luna | high | 32 | $0.04 | 8 | 6 | +$0.02 (2.00×); +7 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
19
19
|
| gpt-5-6-luna | GPT-5.6 Luna | xhigh | 35 | $0.09 | 18 | 6 | +$0.05 (2.25×); +3 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
20
20
|
| gpt-5-6-luna | GPT-5.6 Luna | max | 37 | $0.18 | 36 | 6 | +$0.09 (2.00×); +2 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
21
|
+
| gpt-6-luna | GPT-6 Luna | low | 21 | $0.00 | 1 | 3 | — | First measured effort for this model; no internal effort comparison. |
|
|
22
|
+
| gpt-6-luna | GPT-6 Luna | medium | 29 | $0.02 | 4 | 3 | +$0.02 (4.44×); +8 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
23
|
+
| gpt-6-luna | GPT-6 Luna | high | 32 | $0.03 | 6 | 3 | +$0.01 (1.50×); +3 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
24
|
+
| gpt-6-luna | GPT-6 Luna | xhigh | 34 | $0.04 | 8 | 3 | +$0.01 (1.33×); +2 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
25
|
+
| gpt-6-luna | GPT-6 Luna | max | 37 | $0.07 | 14 | 3 | +$0.03 (1.75×); +3 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
21
26
|
| gpt-5-6-terra | GPT-5.6 Terra | low | 27 | $0.14 | 28 | 56 | — | First measured effort for this model; no internal effort comparison. |
|
|
22
27
|
| gpt-5-6-terra | GPT-5.6 Terra | medium | 30 | $0.18 | 36 | 56 | +$0.04 (1.29×); +3 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
23
28
|
| gpt-5-6-terra | GPT-5.6 Terra | high | 34 | $0.34 | 68 | 56 | +$0.16 (1.89×); +4 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
@@ -28,6 +33,11 @@ Benchmark cost index: `100 × observed benchmark cost / $0.50`; GPT-5.6 Sol medi
|
|
|
28
33
|
| gpt-5-6-sol | GPT-5.6 Sol | high | 42 | $0.81 | 162 | 100 | +$0.31 (1.62×); +3 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
29
34
|
| gpt-5-6-sol | GPT-5.6 Sol | xhigh | 44 | $1.18 | 236 | 100 | +$0.37 (1.46×); +2 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
30
35
|
| gpt-5-6-sol | GPT-5.6 Sol | max | 47 | $1.99 | 398 | 100 | +$0.81 (1.69×); +3 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
36
|
+
| gpt-6-sol | GPT-6 Sol | low | 34 | $0.13 | 26 | 50 | — | First measured effort for this model; no internal effort comparison. |
|
|
37
|
+
| gpt-6-sol | GPT-6 Sol | medium | 40 | $0.25 | 50 | 50 | +$0.12 (1.92×); +6 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
38
|
+
| gpt-6-sol | GPT-6 Sol | high | 43 | $0.37 | 74 | 50 | +$0.12 (1.48×); +3 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
39
|
+
| gpt-6-sol | GPT-6 Sol | xhigh | 44 | $0.53 | 106 | 50 | +$0.16 (1.43×); +1 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
40
|
+
| gpt-6-sol | GPT-6 Sol | max | 48 | $1.06 | 212 | 50 | +$0.53 (2.00×); +4 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
31
41
|
| gpt-6-astra | GPT-6 Astra | low | 46 | $0.82 | 164 | 250 | — | First measured effort for this model; no internal effort comparison. |
|
|
32
42
|
| gpt-6-astra | GPT-6 Astra | medium | 50 | $1.54 | 308 | 250 | +$0.72 (1.88×); +4 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
33
43
|
| gpt-6-astra | GPT-6 Astra | high | 51 | $1.73 | 346 | 250 | +$0.19 (1.12×); +1 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
@@ -43,7 +53,11 @@ Benchmark cost index: `100 × observed benchmark cost / $0.50`; GPT-5.6 Sol medi
|
|
|
43
53
|
| claude-opus-5 | Claude Opus 5 | high | 48 | $3.61 | 722 | 125 | +$1.42 (1.65×); +3 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
44
54
|
| claude-opus-5 | Claude Opus 5 | xhigh | 50 | $4.88 | 976 | 125 | +$1.27 (1.35×); +2 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
45
55
|
| claude-opus-5 | Claude Opus 5 | max | 51 | $5.86 | 1172 | 125 | +$0.98 (1.20×); +1 score | Compare only with adjacent measured effort; higher effort raises observed cost. |
|
|
46
|
-
| claude-opus-5-5 | Claude Opus 5.5 |
|
|
56
|
+
| claude-opus-5-5 | Claude Opus 5.5 | low | 42 | $0.55 | 110 | 100 | — | Segregate from standalone cross-model comparisons. |
|
|
57
|
+
| claude-opus-5-5 | Claude Opus 5.5 | medium | 51 | $1.34 | 268 | 100 | +$0.79 (2.44×); +9 score | Segregate from standalone cross-model comparisons. |
|
|
58
|
+
| claude-opus-5-5 | Claude Opus 5.5 | high | 54 | $1.82 | 364 | 100 | +$0.48 (1.36×); +3 score | Segregate from standalone cross-model comparisons. |
|
|
59
|
+
| claude-opus-5-5 | Claude Opus 5.5 | xhigh | 56 | $3.46 | 692 | 100 | +$1.64 (1.90×); +2 score | Segregate from standalone cross-model comparisons. |
|
|
60
|
+
| claude-opus-5-5 | Claude Opus 5.5 | max | 58 | $5.98 | 1196 | 100 | +$2.52 (1.73×); +2 score | Segregate from standalone cross-model comparisons. |
|
|
47
61
|
| claude-fable-5-1 | Claude Fable 5.1 | low | 47 | $2.37 | 474 | 250 | — | Segregate from standalone cross-model comparisons. |
|
|
48
62
|
| claude-fable-5-1 | Claude Fable 5.1 | medium | 49 | $2.98 | 596 | 250 | +$0.61 (1.26×); +2 score | Segregate from standalone cross-model comparisons. |
|
|
49
63
|
| claude-fable-5-1 | Claude Fable 5.1 | high | 51 | $3.91 | 782 | 250 | +$0.93 (1.31×); +2 score | Segregate from standalone cross-model comparisons. |
|
|
@@ -84,8 +98,10 @@ Benchmark cost index: `100 × observed benchmark cost / $0.50`; GPT-5.6 Sol medi
|
|
|
84
98
|
| Model family | Input USD/MTok | Output USD/MTok | Price note |
|
|
85
99
|
| --- | ---: | ---: | --- |
|
|
86
100
|
| GPT-5.6 Luna | $0.200 | $1.200 | AA-observed price; not a live account tariff. |
|
|
101
|
+
| GPT-6 Luna | $0.100 | $0.500 | AA-observed price; not a live account tariff. |
|
|
87
102
|
| GPT-5.6 Terra | $2.000 | $12.000 | AA-observed price; not a live account tariff. |
|
|
88
103
|
| GPT-5.6 Sol | $4.000 | $20.000 | AA-observed price; not a live account tariff. |
|
|
104
|
+
| GPT-6 Sol | $2.000 | $10.000 | AA-observed price; not a live account tariff. |
|
|
89
105
|
| GPT-6 Astra | $10.000 | $50.000 | AA-observed price; not a live account tariff. |
|
|
90
106
|
| Claude Sonnet 5 | $2.000 | $10.000 | AA-observed price; not a live account tariff. |
|
|
91
107
|
| Claude Opus 5 | $5.000 | $25.000 | AA-observed price; not a live account tariff. |
|
|
@@ -111,8 +127,10 @@ Benchmark cost index: `100 × observed benchmark cost / $0.50`; GPT-5.6 Sol medi
|
|
|
111
127
|
| Model family | Canonical API ID | Availability evidence |
|
|
112
128
|
| --- | --- | --- |
|
|
113
129
|
| GPT-5.6 Luna | `gpt-5.6-luna` | Canonical ID recorded in this dataset; exposure in a specific harness remains unverified. |
|
|
130
|
+
| GPT-6 Luna | `gpt-6-luna` | Canonical ID recorded in this dataset; exposure in a specific harness remains unverified. |
|
|
114
131
|
| GPT-5.6 Terra | `gpt-5.6-terra` | Canonical ID recorded in this dataset; exposure in a specific harness remains unverified. |
|
|
115
132
|
| GPT-5.6 Sol | `gpt-5.6-sol` | Canonical ID recorded in this dataset; exposure in a specific harness remains unverified. |
|
|
133
|
+
| GPT-6 Sol | `gpt-6-sol` | Canonical ID recorded in this dataset; exposure in a specific harness remains unverified. |
|
|
116
134
|
| GPT-6 Astra | `gpt-6-astra` | Canonical ID recorded in this dataset; exposure in a specific harness remains unverified. |
|
|
117
135
|
| Claude Sonnet 5 | `claude-sonnet-5` | Canonical ID recorded in this dataset; exposure in a specific harness remains unverified. |
|
|
118
136
|
| Claude Opus 5 | `claude-opus-5` | Canonical ID recorded in this dataset; exposure in a specific harness remains unverified. |
|
|
@@ -136,12 +154,14 @@ Benchmark cost index: `100 × observed benchmark cost / $0.50`; GPT-5.6 Sol medi
|
|
|
136
154
|
## Source and interpretation limits
|
|
137
155
|
|
|
138
156
|
- [GPT-5.6 Luna](https://artificialanalysis.ai/models/gpt-5-6-luna)
|
|
157
|
+
- [GPT-6 Luna](https://artificialanalysis.ai/models/gpt-6-luna)
|
|
139
158
|
- [GPT-5.6 Terra](https://artificialanalysis.ai/models/gpt-5-6-terra)
|
|
140
159
|
- [GPT-5.6 Sol](https://artificialanalysis.ai/models/gpt-5-6-sol)
|
|
160
|
+
- [GPT-6 Sol](https://artificialanalysis.ai/models/gpt-6-sol)
|
|
141
161
|
- [GPT-6 Astra](https://artificialanalysis.ai/models/gpt-6-astra)
|
|
142
162
|
- [Claude Sonnet 5](https://artificialanalysis.ai/models/claude-sonnet-5)
|
|
143
163
|
- [Claude Opus 5](https://artificialanalysis.ai/models/claude-opus-5)
|
|
144
|
-
- [Claude Opus 5.5](https://artificialanalysis.ai/models/claude-opus-5-5) —
|
|
164
|
+
- [Claude Opus 5.5](https://artificialanalysis.ai/models/claude-opus-5-5) — All displayed scores include the benchmark default fallback and are not standalone scores.
|
|
145
165
|
- [Claude Fable 5.1](https://artificialanalysis.ai/models/claude-fable-5-1) — All displayed scores include the benchmark default fallback and are not standalone scores.
|
|
146
166
|
- [Claude Fable 5](https://artificialanalysis.ai/models/claude-fable-5) — AA v4.3.2 max result uses Opus 4.8 default fallback. This legacy catalog model is not in the current leaderboard filter; segregate it from standalone cross-model comparisons.
|
|
147
167
|
- [Claude Haiku 4.5](https://artificialanalysis.ai/models/claude-4-5-haiku) — Its API thinking budget is not an effort enum. The disabled score 15 is incomplete and intentionally excluded.
|
package/models/top-models.json
CHANGED
|
@@ -91,6 +91,7 @@
|
|
|
91
91
|
"share_max": null
|
|
92
92
|
},
|
|
93
93
|
"notes": [
|
|
94
|
+
"Fallback for GPT-6 Luna on its Codex seat: dispatched when GPT-6 Luna is not exposed by the inventory or is rolled back.",
|
|
94
95
|
"Default W-tier config is `xhigh`; `max` when it stalls.",
|
|
95
96
|
"Escalate to S when the task needs capacity (judgment, cross-file scope, ambiguity), not merely more reasoning depth."
|
|
96
97
|
],
|
|
@@ -99,6 +100,115 @@
|
|
|
99
100
|
],
|
|
100
101
|
"observed_at": "2026-09-22"
|
|
101
102
|
},
|
|
103
|
+
{
|
|
104
|
+
"key": "gpt-6-luna",
|
|
105
|
+
"display_name": "GPT-6 Luna",
|
|
106
|
+
"provider": "openai",
|
|
107
|
+
"ladder": "codex",
|
|
108
|
+
"admission": "incumbent",
|
|
109
|
+
"tier": "W",
|
|
110
|
+
"eligible_tiers": [
|
|
111
|
+
"W"
|
|
112
|
+
],
|
|
113
|
+
"supersedes": "gpt-5-6-luna",
|
|
114
|
+
"api_ids": {
|
|
115
|
+
"codex": "gpt-6-luna",
|
|
116
|
+
"claude": null,
|
|
117
|
+
"opencode": null,
|
|
118
|
+
"kilo": null
|
|
119
|
+
},
|
|
120
|
+
"context_window": 1050000,
|
|
121
|
+
"max_output_tokens": 128000,
|
|
122
|
+
"thinking": {
|
|
123
|
+
"control": "reasoning_effort",
|
|
124
|
+
"levels": [
|
|
125
|
+
"low",
|
|
126
|
+
"medium",
|
|
127
|
+
"high",
|
|
128
|
+
"xhigh",
|
|
129
|
+
"max"
|
|
130
|
+
],
|
|
131
|
+
"always_on": false,
|
|
132
|
+
"default_for_tier": "xhigh",
|
|
133
|
+
"note": "Same W policy as GPT-5.6 Luna: exempt from the Codex `high` ceiling, thinking is cheap on a small model. The API also accepts `none`, which is not measured and never routed."
|
|
134
|
+
},
|
|
135
|
+
"price": {
|
|
136
|
+
"input_usd_per_mtok": 0.1,
|
|
137
|
+
"output_usd_per_mtok": 0.5,
|
|
138
|
+
"cache_read_usd_per_mtok": 0.01,
|
|
139
|
+
"cache_write_multiplier": 1.25
|
|
140
|
+
},
|
|
141
|
+
"long_context_surcharge": {
|
|
142
|
+
"threshold_input_tokens": 272000,
|
|
143
|
+
"input_multiplier": 2,
|
|
144
|
+
"output_multiplier": 1.5,
|
|
145
|
+
"effective_input_usd_per_mtok": 0.2,
|
|
146
|
+
"effective_output_usd_per_mtok": 0.75,
|
|
147
|
+
"note": "Crossing the threshold reprices the entire request, not just the overflow."
|
|
148
|
+
},
|
|
149
|
+
"measured": {
|
|
150
|
+
"low": {
|
|
151
|
+
"score": 21,
|
|
152
|
+
"usd_per_task": 0.0045
|
|
153
|
+
},
|
|
154
|
+
"medium": {
|
|
155
|
+
"score": 29,
|
|
156
|
+
"usd_per_task": 0.02
|
|
157
|
+
},
|
|
158
|
+
"high": {
|
|
159
|
+
"score": 32,
|
|
160
|
+
"usd_per_task": 0.03
|
|
161
|
+
},
|
|
162
|
+
"xhigh": {
|
|
163
|
+
"score": 34,
|
|
164
|
+
"usd_per_task": 0.04
|
|
165
|
+
},
|
|
166
|
+
"max": {
|
|
167
|
+
"score": 37,
|
|
168
|
+
"usd_per_task": 0.07
|
|
169
|
+
}
|
|
170
|
+
},
|
|
171
|
+
"indices": {
|
|
172
|
+
"thinking_cost_index": {
|
|
173
|
+
"low": 0.9,
|
|
174
|
+
"medium": 4,
|
|
175
|
+
"high": 6,
|
|
176
|
+
"xhigh": 8,
|
|
177
|
+
"max": 14
|
|
178
|
+
},
|
|
179
|
+
"marginal_thinking": {
|
|
180
|
+
"low->medium": {
|
|
181
|
+
"delta_score": 8,
|
|
182
|
+
"cost_multiplier": 4.44
|
|
183
|
+
},
|
|
184
|
+
"medium->high": {
|
|
185
|
+
"delta_score": 3,
|
|
186
|
+
"cost_multiplier": 1.5
|
|
187
|
+
},
|
|
188
|
+
"high->xhigh": {
|
|
189
|
+
"delta_score": 2,
|
|
190
|
+
"cost_multiplier": 1.33
|
|
191
|
+
},
|
|
192
|
+
"xhigh->max": {
|
|
193
|
+
"delta_score": 3,
|
|
194
|
+
"cost_multiplier": 1.75
|
|
195
|
+
}
|
|
196
|
+
}
|
|
197
|
+
},
|
|
198
|
+
"caps": {
|
|
199
|
+
"max_effort": "max",
|
|
200
|
+
"share_max": null
|
|
201
|
+
},
|
|
202
|
+
"notes": [
|
|
203
|
+
"Successor to GPT-5.6 Luna on the Codex W seat: $0.10/$0.50 vs $0.20/$1.20 per MTok, and cheaper per completed task at every measured effort (max 37 @ $0.07 vs 37 @ $0.18). GPT-5.6 Luna stays routable as its fallback.",
|
|
204
|
+
"Default W-tier config is `xhigh`; `max` when it stalls. Escalate to S for capacity, not depth."
|
|
205
|
+
],
|
|
206
|
+
"sources": [
|
|
207
|
+
"https://artificialanalysis.ai/models/gpt-6-luna",
|
|
208
|
+
"https://developers.openai.com/api/docs/models/gpt-6-luna"
|
|
209
|
+
],
|
|
210
|
+
"observed_at": "2026-09-23"
|
|
211
|
+
},
|
|
102
212
|
{
|
|
103
213
|
"key": "gpt-5-6-terra",
|
|
104
214
|
"display_name": "GPT-5.6 Terra",
|
|
@@ -239,6 +349,7 @@
|
|
|
239
349
|
}
|
|
240
350
|
},
|
|
241
351
|
"notes": [
|
|
352
|
+
"Fallback for GPT-6 Sol on its Codex seat: dispatched when GPT-6 Sol is not exposed by the inventory or is rolled back.",
|
|
242
353
|
"Sol never above `high`; `max` is dominated and never correct.",
|
|
243
354
|
"X T4 on Codex = Sol `high` + an independent Sol `high` reviewer (different agent, no forked history)."
|
|
244
355
|
],
|
|
@@ -247,6 +358,119 @@
|
|
|
247
358
|
],
|
|
248
359
|
"observed_at": "2026-09-22"
|
|
249
360
|
},
|
|
361
|
+
{
|
|
362
|
+
"key": "gpt-6-sol",
|
|
363
|
+
"display_name": "GPT-6 Sol",
|
|
364
|
+
"provider": "openai",
|
|
365
|
+
"ladder": "codex",
|
|
366
|
+
"admission": "incumbent",
|
|
367
|
+
"tier": "X",
|
|
368
|
+
"eligible_tiers": [
|
|
369
|
+
"X"
|
|
370
|
+
],
|
|
371
|
+
"supersedes": "gpt-5-6-sol",
|
|
372
|
+
"api_ids": {
|
|
373
|
+
"codex": "gpt-6-sol",
|
|
374
|
+
"claude": null,
|
|
375
|
+
"opencode": null,
|
|
376
|
+
"kilo": null
|
|
377
|
+
},
|
|
378
|
+
"context_window": 1050000,
|
|
379
|
+
"max_output_tokens": 128000,
|
|
380
|
+
"thinking": {
|
|
381
|
+
"control": "reasoning_effort",
|
|
382
|
+
"levels": [
|
|
383
|
+
"low",
|
|
384
|
+
"medium",
|
|
385
|
+
"high",
|
|
386
|
+
"xhigh",
|
|
387
|
+
"max"
|
|
388
|
+
],
|
|
389
|
+
"always_on": false,
|
|
390
|
+
"default_for_tier": "high",
|
|
391
|
+
"note": "Same Codex policy as GPT-5.6 Sol: ceiling `high`; T4 is `high` plus an independent second Sol `high` reviewer with no shared history. The API also accepts `none`, which is not measured and never routed."
|
|
392
|
+
},
|
|
393
|
+
"price": {
|
|
394
|
+
"input_usd_per_mtok": 2,
|
|
395
|
+
"output_usd_per_mtok": 10,
|
|
396
|
+
"cache_read_usd_per_mtok": 0.2,
|
|
397
|
+
"cache_write_multiplier": 1.25
|
|
398
|
+
},
|
|
399
|
+
"long_context_surcharge": {
|
|
400
|
+
"threshold_input_tokens": 272000,
|
|
401
|
+
"input_multiplier": 2,
|
|
402
|
+
"output_multiplier": 1.5,
|
|
403
|
+
"effective_input_usd_per_mtok": 4,
|
|
404
|
+
"effective_output_usd_per_mtok": 15,
|
|
405
|
+
"note": "Crossing the threshold reprices the entire request, not just the overflow."
|
|
406
|
+
},
|
|
407
|
+
"measured": {
|
|
408
|
+
"low": {
|
|
409
|
+
"score": 34,
|
|
410
|
+
"usd_per_task": 0.13
|
|
411
|
+
},
|
|
412
|
+
"medium": {
|
|
413
|
+
"score": 40,
|
|
414
|
+
"usd_per_task": 0.25
|
|
415
|
+
},
|
|
416
|
+
"high": {
|
|
417
|
+
"score": 43,
|
|
418
|
+
"usd_per_task": 0.37
|
|
419
|
+
},
|
|
420
|
+
"xhigh": {
|
|
421
|
+
"score": 44,
|
|
422
|
+
"usd_per_task": 0.53
|
|
423
|
+
},
|
|
424
|
+
"max": {
|
|
425
|
+
"score": 48,
|
|
426
|
+
"usd_per_task": 1.06
|
|
427
|
+
}
|
|
428
|
+
},
|
|
429
|
+
"indices": {
|
|
430
|
+
"thinking_cost_index": {
|
|
431
|
+
"low": 26,
|
|
432
|
+
"medium": 50,
|
|
433
|
+
"high": 74,
|
|
434
|
+
"xhigh": 106,
|
|
435
|
+
"max": 212
|
|
436
|
+
},
|
|
437
|
+
"marginal_thinking": {
|
|
438
|
+
"low->medium": {
|
|
439
|
+
"delta_score": 6,
|
|
440
|
+
"cost_multiplier": 1.92
|
|
441
|
+
},
|
|
442
|
+
"medium->high": {
|
|
443
|
+
"delta_score": 3,
|
|
444
|
+
"cost_multiplier": 1.48
|
|
445
|
+
},
|
|
446
|
+
"high->xhigh": {
|
|
447
|
+
"delta_score": 1,
|
|
448
|
+
"cost_multiplier": 1.43
|
|
449
|
+
},
|
|
450
|
+
"xhigh->max": {
|
|
451
|
+
"delta_score": 4,
|
|
452
|
+
"cost_multiplier": 2.0
|
|
453
|
+
}
|
|
454
|
+
}
|
|
455
|
+
},
|
|
456
|
+
"caps": {
|
|
457
|
+
"max_effort": "high",
|
|
458
|
+
"share_max": null,
|
|
459
|
+
"t4_substitution": {
|
|
460
|
+
"effort": "high",
|
|
461
|
+
"requires_independent_second_reviewer": true
|
|
462
|
+
}
|
|
463
|
+
},
|
|
464
|
+
"notes": [
|
|
465
|
+
"Successor to GPT-5.6 Sol on the Codex X seat at half the per-token price ($2/$10 vs $4/$20, permanent pricing). It is cheaper per completed task at every measured effort and scores equal or higher at each (high 43 @ $0.37 vs 42 @ $0.81); GPT-5.6 Sol stays routable as its fallback.",
|
|
466
|
+
"Its `medium` (40 @ $0.25) also dominates GPT-5.6 Terra `xhigh` (38 @ $0.63); the S seat stays Terra because an incumbent's tier is its ladder seat, not a score — see the routing note."
|
|
467
|
+
],
|
|
468
|
+
"sources": [
|
|
469
|
+
"https://artificialanalysis.ai/models/gpt-6-sol",
|
|
470
|
+
"https://developers.openai.com/api/docs/models/gpt-6-sol"
|
|
471
|
+
],
|
|
472
|
+
"observed_at": "2026-09-23"
|
|
473
|
+
},
|
|
250
474
|
{
|
|
251
475
|
"key": "gpt-6-astra",
|
|
252
476
|
"display_name": "GPT-6 Astra",
|
|
@@ -506,7 +730,9 @@
|
|
|
506
730
|
"ladder": "claude",
|
|
507
731
|
"admission": "incumbent",
|
|
508
732
|
"tier": "X",
|
|
509
|
-
"eligible_tiers": [
|
|
733
|
+
"eligible_tiers": [
|
|
734
|
+
"X"
|
|
735
|
+
],
|
|
510
736
|
"supersedes": "claude-opus-5",
|
|
511
737
|
"api_ids": {
|
|
512
738
|
"codex": null,
|
|
@@ -518,8 +744,13 @@
|
|
|
518
744
|
"max_output_tokens": 128000,
|
|
519
745
|
"thinking": {
|
|
520
746
|
"control": "effort",
|
|
521
|
-
"levels": [
|
|
522
|
-
|
|
747
|
+
"levels": [
|
|
748
|
+
"low",
|
|
749
|
+
"medium",
|
|
750
|
+
"high",
|
|
751
|
+
"xhigh",
|
|
752
|
+
"max"
|
|
753
|
+
],
|
|
523
754
|
"always_on": true,
|
|
524
755
|
"default_for_tier": "high",
|
|
525
756
|
"note": "Thinking is always on: omit `thinking` or send `adaptive`; `disabled` and `budget_tokens` return 400 at every effort. The API default effort is `medium` (one below Opus 5), so X always sets effort explicitly — `high` for T3, `xhigh` for T4. Forced `tool_choice` any/tool returns 400 — use `auto` + instruction, `strict: true`, or structured outputs. Thinking blocks are model-bound; keep transcripts append-only. Computer use only via `computer_toolset_20260801`."
|
|
@@ -533,14 +764,54 @@
|
|
|
533
764
|
},
|
|
534
765
|
"long_context_surcharge": null,
|
|
535
766
|
"measured": {
|
|
536
|
-
"
|
|
767
|
+
"low": {
|
|
768
|
+
"score": 42,
|
|
769
|
+
"usd_per_task": 0.55
|
|
770
|
+
},
|
|
771
|
+
"medium": {
|
|
772
|
+
"score": 51,
|
|
773
|
+
"usd_per_task": 1.34
|
|
774
|
+
},
|
|
775
|
+
"high": {
|
|
776
|
+
"score": 54,
|
|
777
|
+
"usd_per_task": 1.82
|
|
778
|
+
},
|
|
779
|
+
"xhigh": {
|
|
780
|
+
"score": 56,
|
|
781
|
+
"usd_per_task": 3.46
|
|
782
|
+
},
|
|
783
|
+
"max": {
|
|
784
|
+
"score": 58,
|
|
785
|
+
"usd_per_task": 5.98
|
|
786
|
+
}
|
|
537
787
|
},
|
|
538
|
-
"measurement_note": "
|
|
788
|
+
"measurement_note": "All displayed scores include the benchmark default fallback and are not standalone scores.",
|
|
539
789
|
"indices": {
|
|
540
790
|
"thinking_cost_index": {
|
|
791
|
+
"low": 110,
|
|
792
|
+
"medium": 268,
|
|
793
|
+
"high": 364,
|
|
794
|
+
"xhigh": 692,
|
|
541
795
|
"max": 1196
|
|
542
796
|
},
|
|
543
|
-
"marginal_thinking": {
|
|
797
|
+
"marginal_thinking": {
|
|
798
|
+
"low->medium": {
|
|
799
|
+
"delta_score": 9,
|
|
800
|
+
"cost_multiplier": 2.44
|
|
801
|
+
},
|
|
802
|
+
"medium->high": {
|
|
803
|
+
"delta_score": 3,
|
|
804
|
+
"cost_multiplier": 1.36
|
|
805
|
+
},
|
|
806
|
+
"high->xhigh": {
|
|
807
|
+
"delta_score": 2,
|
|
808
|
+
"cost_multiplier": 1.9
|
|
809
|
+
},
|
|
810
|
+
"xhigh->max": {
|
|
811
|
+
"delta_score": 2,
|
|
812
|
+
"cost_multiplier": 1.73
|
|
813
|
+
}
|
|
814
|
+
}
|
|
544
815
|
},
|
|
545
816
|
"caps": {
|
|
546
817
|
"max_effort": "xhigh",
|
|
@@ -548,7 +819,7 @@
|
|
|
548
819
|
},
|
|
549
820
|
"notes": [
|
|
550
821
|
"Successor to Claude Opus 5 on the Claude X seat. Every per-token price is lower (Anthropic list: input $4 vs $5, output $20 vs $25, cache read $0.20 vs $0.50, cache write $5 vs $6.25), with the same 1M context, 128K output and tokenizer, so the router ranks it ahead of Opus 5 at the same effort; Opus 5 stays routable as its fallback.",
|
|
551
|
-
"
|
|
822
|
+
"Measured at every effort (Artificial Analysis, with default fallback): high 54 @ $1.82 vs Opus 5 high 48 @ $3.61 — higher score at half the cost per task. Anthropic's launch table also puts it above Opus 5 on every published benchmark.",
|
|
552
823
|
"Seat stays X: an incumbent tier is never re-derived from a score."
|
|
553
824
|
],
|
|
554
825
|
"sources": [
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "llm-orchestrator",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.3.0",
|
|
4
4
|
"description": "Write /task once — it plans the work, shards it across parallel subagents, gates every phase and verifies before claiming done. Claude Code, Codex, OpenCode, Kilo.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"engines": {
|
package/policies/routing.md
CHANGED
|
@@ -74,9 +74,9 @@ the incumbents below are role mappings, not guarantees of availability.
|
|
|
74
74
|
|
|
75
75
|
| Tier | Role | Claude incumbent | Codex incumbent |
|
|
76
76
|
| --- | --- | --- | --- |
|
|
77
|
-
| **W** worker | local, mechanical, repetitive, well-defined: code search, classification, extraction, small edits, boilerplate, simple tests, consistency checks, scoped transforms | Haiku 4.5 (`claude-haiku-4-5`) | `gpt-5.6-luna` |
|
|
77
|
+
| **W** worker | local, mechanical, repetitive, well-defined: code search, classification, extraction, small edits, boilerplate, simple tests, consistency checks, scoped transforms | Haiku 4.5 (`claude-haiku-4-5`) | `gpt-6-luna`, falling back to `gpt-5.6-luna` |
|
|
78
78
|
| **S** standard | default software-engineering model: normal implementation, frontend/backend, moderate debugging, tests, reasonable multi-file refactors, codebase analysis, tool use | Sonnet 5 (`claude-sonnet-5`) | `gpt-5.6-terra` |
|
|
79
|
-
| **X** senior | hard debugging, architecture, concurrency, migrations, security, auth, payments, billing, backwards compatibility, critical code review, many invariants | Opus 5.5 (`claude-opus-5-5`), falling back to Opus 5 (`claude-opus-5`) | `gpt-5.6-sol` |
|
|
79
|
+
| **X** senior | hard debugging, architecture, concurrency, migrations, security, auth, payments, billing, backwards compatibility, critical code review, many invariants | Opus 5.5 (`claude-opus-5-5`), falling back to Opus 5 (`claude-opus-5`) | `gpt-6-sol`, falling back to `gpt-5.6-sol` |
|
|
80
80
|
| **F** frontier | exceptional escalation: very ambiguous, long-horizon, cross-system, major architecture, very large codebase, planning under heavy constraints, or when X fails to produce a solid solution | Fable 5 (`claude-fable-5`) — default F. Fable 5.1 (`claude-fable-5-1`) is a hard-capped exception: **≤2% of all dispatches**, explicit request or documented F-T4 failure on Fable 5 only | GPT-6 Astra (`gpt-6-astra`) — a real single-agent frontier tier, no decomposition workaround needed |
|
|
81
81
|
|
|
82
82
|
Escalation order within a provider: **W → S → X → F**.
|
|
@@ -85,9 +85,9 @@ Verified prices (Sep 2026, provider pricing pages), USD in/out per MTok:
|
|
|
85
85
|
|
|
86
86
|
| Tier | Claude model | Claude $ in/out | Codex model | Codex $ in/out |
|
|
87
87
|
| --- | --- | --- | --- | --- |
|
|
88
|
-
| W | Haiku 4.5 (200K ctx) | $1 / $5 | `gpt-
|
|
88
|
+
| W | Haiku 4.5 (200K ctx) | $1 / $5 | `gpt-6-luna` (1.05M ctx); fallback `gpt-5.6-luna` $0.20 / $1.20 | $0.10 / $0.50 |
|
|
89
89
|
| S | Sonnet 5 (1M ctx) | $2 / $10 | `gpt-5.6-terra` (1.05M ctx) | $2 / $12 |
|
|
90
|
-
| X | Opus 5.5 (1M ctx); fallback Opus 5 $5 / $25 | $4 / $20 | `gpt-
|
|
90
|
+
| X | Opus 5.5 (1M ctx); fallback Opus 5 $5 / $25 | $4 / $20 | `gpt-6-sol` (1.05M ctx); fallback `gpt-5.6-sol` $4 / $20 | $2 / $10 |
|
|
91
91
|
| F | Fable 5 (1M ctx) | $10 / $50 | `gpt-6-astra` (1.05M ctx) | $10 / $50 list — **but the cheapest F per completed task of any model here** |
|
|
92
92
|
| F+ (≤2%) | Fable 5.1 (1M ctx) | $10 / $50 list — **effective cost significantly higher** (always-on thinking, longer turns, more output tokens per task) | — (Astra covers F) | — |
|
|
93
93
|
|
|
@@ -8,7 +8,8 @@
|
|
|
8
8
|
"W": {
|
|
9
9
|
"name": "worker",
|
|
10
10
|
"responsibility": "Local, mechanical, repetitive, well-defined: code search, classification, extraction, small edits, boilerplate, simple tests, consistency checks, scoped transforms.",
|
|
11
|
-
"incumbents": {"claude": "claude-haiku-4-5", "codex": "gpt-
|
|
11
|
+
"incumbents": {"claude": "claude-haiku-4-5", "codex": "gpt-6-luna"},
|
|
12
|
+
"fallbacks": {"codex": ["gpt-5.6-luna"]}
|
|
12
13
|
},
|
|
13
14
|
"S": {
|
|
14
15
|
"name": "standard",
|
|
@@ -18,8 +19,8 @@
|
|
|
18
19
|
"X": {
|
|
19
20
|
"name": "senior",
|
|
20
21
|
"responsibility": "Hard debugging, architecture, concurrency, migrations, security, auth, payments, billing, backwards compatibility, critical code review, many invariants.",
|
|
21
|
-
"incumbents": {"claude": "claude-opus-5-5", "codex": "gpt-
|
|
22
|
-
"fallbacks": {"claude": ["claude-opus-5"]}
|
|
22
|
+
"incumbents": {"claude": "claude-opus-5-5", "codex": "gpt-6-sol"},
|
|
23
|
+
"fallbacks": {"claude": ["claude-opus-5"], "codex": ["gpt-5.6-sol"]}
|
|
23
24
|
},
|
|
24
25
|
"F": {
|
|
25
26
|
"name": "frontier",
|
|
@@ -74,16 +75,16 @@
|
|
|
74
75
|
"resolution": {
|
|
75
76
|
"W T0-T1": {
|
|
76
77
|
"claude": {"model": "claude-haiku-4-5", "effort": null, "thinking": "off"},
|
|
77
|
-
"codex": {"model": "gpt-
|
|
78
|
+
"codex": {"model": "gpt-6-luna", "effort": "low"}
|
|
78
79
|
},
|
|
79
80
|
"W T2": {
|
|
80
81
|
"claude": {"model": "claude-haiku-4-5", "effort": null, "thinking": "small budget"},
|
|
81
|
-
"codex": {"model": "gpt-
|
|
82
|
+
"codex": {"model": "gpt-6-luna", "effort": "medium"}
|
|
82
83
|
},
|
|
83
84
|
"W T3": {
|
|
84
85
|
"claude": null,
|
|
85
86
|
"claude_note": "No headroom at the Claude W tier — escalate to S.",
|
|
86
|
-
"codex": {"model": "gpt-
|
|
87
|
+
"codex": {"model": "gpt-6-luna", "effort": "xhigh", "alternate_effort": "max"}
|
|
87
88
|
},
|
|
88
89
|
"S T1": {
|
|
89
90
|
"claude": {"model": "claude-sonnet-5", "effort": "low"},
|
|
@@ -99,15 +100,15 @@
|
|
|
99
100
|
},
|
|
100
101
|
"X T2": {
|
|
101
102
|
"claude": {"model": "claude-opus-5-5", "effort": "medium"},
|
|
102
|
-
"codex": {"model": "gpt-
|
|
103
|
+
"codex": {"model": "gpt-6-sol", "effort": "medium"}
|
|
103
104
|
},
|
|
104
105
|
"X T3": {
|
|
105
106
|
"claude": {"model": "claude-opus-5-5", "effort": "high"},
|
|
106
|
-
"codex": {"model": "gpt-
|
|
107
|
+
"codex": {"model": "gpt-6-sol", "effort": "high"}
|
|
107
108
|
},
|
|
108
109
|
"X T4": {
|
|
109
110
|
"claude": {"model": "claude-opus-5-5", "effort": "xhigh"},
|
|
110
|
-
"codex": {"model": "gpt-
|
|
111
|
+
"codex": {"model": "gpt-6-sol", "effort": "high", "independent_second_reviewer": {"model": "gpt-6-sol", "effort": "high"}}
|
|
111
112
|
},
|
|
112
113
|
"F T3": {
|
|
113
114
|
"claude": {"model": "claude-fable-5", "effort": "high"},
|
|
@@ -288,15 +289,15 @@
|
|
|
288
289
|
{"model": "claude-fable-5-1", "effort": "high", "requires_explicit_flag": true, "note": "≤2% cap, explicit request or documented Fable 5 F-T4 failure only."}
|
|
289
290
|
],
|
|
290
291
|
"codex": [
|
|
291
|
-
{"model": "gpt-
|
|
292
|
-
{"model": "gpt-
|
|
293
|
-
{"model": "gpt-
|
|
294
|
-
{"model": "gpt-
|
|
292
|
+
{"model": "gpt-6-luna", "effort": "low"},
|
|
293
|
+
{"model": "gpt-6-luna", "effort": "medium"},
|
|
294
|
+
{"model": "gpt-6-luna", "effort": "xhigh"},
|
|
295
|
+
{"model": "gpt-6-luna", "effort": "max"},
|
|
295
296
|
{"model": "gpt-5.6-terra", "effort": "medium"},
|
|
296
297
|
{"model": "gpt-5.6-terra", "effort": "high"},
|
|
297
|
-
{"model": "gpt-
|
|
298
|
-
{"model": "gpt-
|
|
299
|
-
{"model": "gpt-
|
|
298
|
+
{"model": "gpt-6-sol", "effort": "medium"},
|
|
299
|
+
{"model": "gpt-6-sol", "effort": "high"},
|
|
300
|
+
{"model": "gpt-6-sol", "effort": "high", "independent_second_reviewer": true},
|
|
300
301
|
{"model": "gpt-6-astra", "effort": "high"},
|
|
301
302
|
{"model": "gpt-6-astra", "effort": "xhigh"},
|
|
302
303
|
{"model": "gpt-6-astra", "effort": "max", "note": "Essentially never."}
|