docks-kit 0.22.23 → 0.22.28
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +10 -10
- package/cli/docs/command-guards.md +103 -14
- package/cli/docs/omp-context.md +5 -5
- package/cli/docs/omp-models.md +240 -48
- package/cli/src/engine-native/ompRemovals.ts +14 -0
- package/cli/src/generated/sotPayload.ts +23 -11
- package/package.json +1 -1
package/cli/docs/omp-models.md
CHANGED
|
@@ -18,7 +18,7 @@ against.
|
|
|
18
18
|
| `tiny` | `openai-codex/gpt-6-luna` | low | 21 | $0.0045 | 2.18 s |
|
|
19
19
|
| `fable` | `anthropic/claude-fable-5-2` | medium | 56 (v5.0.1) | $3.80 (v5.0.1) | 11.23 s (v5.0.1) |
|
|
20
20
|
| `switch_fable` | `anthropic/claude-fable-5-2` | medium | 56 (v5.0.1) | $3.80 (v5.0.1) | 11.23 s (v5.0.1) |
|
|
21
|
-
| `astra` | `openai-codex/gpt-6-astra` | xhigh |
|
|
21
|
+
| `astra` | `openai-codex/gpt-6.2-astra` | xhigh | 62 (v5.0.1) | $2.52 (v5.0.1) | 52.89 s (v5.0.1) |
|
|
22
22
|
| `reviewer` | `openai-codex/gpt-6.1-terra` | high | 42 (v5.0.1) | $0.26 (v5.0.1) | 10.91 s (v5.0.1) |
|
|
23
23
|
| `web` | `web/firecrawl` | n/a | n/a | n/a | n/a |
|
|
24
24
|
|
|
@@ -26,19 +26,38 @@ The table reports the measured Artificial Analysis figures for each assigned
|
|
|
26
26
|
model and level. It states no motive that the config or omp's own
|
|
27
27
|
documentation does not establish. AA has not measured output speed or latency
|
|
28
28
|
for GPT-6 Luna medium, so that row carries `n/a` for TTFT. AA measures no web
|
|
29
|
-
search provider, so the `web` row carries no figures. The `
|
|
30
|
-
and `switch_fable` rows come from later captures at Intelligence
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
29
|
+
search provider, so the `web` row carries no figures. The `astra`, `reviewer`,
|
|
30
|
+
`fable`, and `switch_fable` rows come from later captures at Intelligence
|
|
31
|
+
Index v5.0.1: GPT-6.1 Terra from 2026-11-05, Fable 5.2 from 2026-11-12, and
|
|
32
|
+
GPT-6.2 Astra from 2026-11-18. The 2026-09-29 capture predates these models.
|
|
33
|
+
Compare those rows only with other v5.0.1 figures (see GPT-6.2 Astra, GPT-6.1
|
|
34
|
+
Terra, and Claude Fable 5.2 below). A review benchmark chose the model of the
|
|
35
|
+
`reviewer` role (DocksDocks/public#130), not these figures. The tier of that
|
|
36
|
+
role follows the main session (DocksDocks/public#134).
|
|
36
37
|
|
|
37
38
|
The role map carries no Coding Agent Index column. That index measures specific
|
|
38
39
|
agent harnesses and model levels, not omp roles. The snapshot section below
|
|
39
|
-
lists the relevant entries, including Codex with GPT-6.
|
|
40
|
+
lists the relevant entries, including Codex with GPT-6.2 Sol and GPT-6.2 Astra
|
|
41
|
+
at high and xhigh.
|
|
40
42
|
|
|
41
|
-
###
|
|
43
|
+
### AA rows for each role model
|
|
44
|
+
|
|
45
|
+
For each model that an omp role or retry chain uses, this topic records AA
|
|
46
|
+
rows at `high` and `xhigh` at least. Each set of rows names its Intelligence
|
|
47
|
+
Index version and its capture date. A role or chain change adds the missing
|
|
48
|
+
rows in the same commit. On 2026-11-18, the role map and the retry chains use
|
|
49
|
+
six models, and each has both rows:
|
|
50
|
+
|
|
51
|
+
| Model | Index version | Capture date |
|
|
52
|
+
|---|---|---|
|
|
53
|
+
| Claude Opus 5.5 | v4.3.2 | 2026-09-29 |
|
|
54
|
+
| GPT-6 Luna | v4.3.2 | 2026-09-29 |
|
|
55
|
+
| Claude Fable 5.2 | v5.0.1 | 2026-11-12 |
|
|
56
|
+
| GPT-6.1 Terra | v5.0.1 | 2026-11-05, speed re-measured 2026-11-18 |
|
|
57
|
+
| GPT-6.2 Sol | v5.0.1 | 2026-11-18 |
|
|
58
|
+
| GPT-6.2 Astra | v5.0.1 | 2026-11-18 |
|
|
59
|
+
|
|
60
|
+
### OpenAI model availability in omp
|
|
42
61
|
|
|
43
62
|
The `openai-codex` selectors for GPT-6 Sol and GPT-6 Luna were deployed on
|
|
44
63
|
2026-09-22, while the rollout of both models was still in progress. The owner
|
|
@@ -74,6 +93,15 @@ and the request returned HTTP 401 `refresh_token_invalidated`. The Codex
|
|
|
74
93
|
model cache was last fetched on 2026-09-22 and does not list `gpt-6.1-sol`,
|
|
75
94
|
so that Codex run did not establish model availability.
|
|
76
95
|
|
|
96
|
+
On 2026-11-18, omp 18.22.2 listed `gpt-6.2-sol` with a 500K context window
|
|
97
|
+
and `gpt-6.2-astra` with a 272K context window. Both had low, medium, high,
|
|
98
|
+
xhigh, and max. An `omp -p --mode json` run on each selector at `:low`
|
|
99
|
+
recorded that same id as the serving model. omp 18.22.2 listed no GPT-6.2
|
|
100
|
+
Terra and no GPT-6.2 Luna. To recheck, run `omp models openai-codex`; the
|
|
101
|
+
`gpt-6.2-sol` and `gpt-6.2-astra` rows must appear. Sync deploys the selectors
|
|
102
|
+
without a catalog check. On an omp release without these ids, omp can resolve
|
|
103
|
+
them by fuzzy match to another model without a warning, as on 2026-09-22.
|
|
104
|
+
|
|
77
105
|
What omp's settings catalog establishes about these roles:
|
|
78
106
|
|
|
79
107
|
- `cycleOrder` lists the roles the model switcher cycles. `fable` is the
|
|
@@ -94,7 +122,7 @@ provider. "OpenAI fast mode" below describes how that tier changes. The bundled
|
|
|
94
122
|
`reviewer` and `security-reviewer` agents ship with omp, and the
|
|
95
123
|
`plan-lifecycle` plugin ships `code-reviewer` and `plan-reviewer`.
|
|
96
124
|
`retry.fallbackChains.reviewer` holds `anthropic/claude-opus-5-5:high`, and
|
|
97
|
-
`retry.fallbackChains.task` holds `openai-codex/gpt-6.
|
|
125
|
+
`retry.fallbackChains.task` holds `openai-codex/gpt-6.2-sol:high`; both are
|
|
98
126
|
cross-vendor fallbacks.
|
|
99
127
|
|
|
100
128
|
The review benchmark of DocksDocks/public#130 ran on 2026-11-05. It used 24
|
|
@@ -117,10 +145,10 @@ recall difference down to -0.07 (bound -0.05). Both intervals include 0, and
|
|
|
117
145
|
the user judged the difference as noise.
|
|
118
146
|
|
|
119
147
|
`retry.fallbackChains.astra` holds `anthropic/claude-fable-5-2:medium`.
|
|
120
|
-
`retry.fallbackChains.fable` holds `openai-codex/gpt-6-astra:xhigh`.
|
|
148
|
+
`retry.fallbackChains.fable` holds `openai-codex/gpt-6.2-astra:xhigh`.
|
|
121
149
|
Each deliberate cycle stop falls to the other vendor. Without these explicit
|
|
122
150
|
chains, `retry.fallbackChains.default` would send either stop to
|
|
123
|
-
`openai-codex/gpt-6.
|
|
151
|
+
`openai-codex/gpt-6.2-sol:high`.
|
|
124
152
|
Chain entries are concrete selectors, not role aliases, so this pair cannot
|
|
125
153
|
recurse. The hidden `switch_fable` chain stays empty.
|
|
126
154
|
|
|
@@ -175,12 +203,13 @@ comes from the GPT-5.6 Sol high column of the GPT-6.1 Sol high comparison.
|
|
|
175
203
|
Index composition changes between versions; figures from an earlier capture
|
|
176
204
|
cannot be mixed with these.
|
|
177
205
|
|
|
178
|
-
|
|
179
|
-
and level entries. It
|
|
180
|
-
xhigh, and max; Opus 5.5 and Fable 5.1 with Claude Code
|
|
181
|
-
Luna with Codex at max. It also
|
|
182
|
-
Antigravity SDK with Gemini 3.8 Flash at high, and Opencode
|
|
183
|
-
Kimi Code CLI with Kimi K3 without levels. The relevant
|
|
206
|
+
At v1.5 on 2026-09-29, the Coding Agent Index page (`/agents/coding-agents`)
|
|
207
|
+
listed 30 harness, model, and level entries. It covered GPT-6.1 Sol with Codex
|
|
208
|
+
at low, medium, high, xhigh, and max; Opus 5.5 and Fable 5.1 with Claude Code
|
|
209
|
+
at max; and Astra and Luna with Codex at max. It also covered Grok Build with
|
|
210
|
+
Grok 4.7 at xhigh, Antigravity SDK with Gemini 3.8 Flash at high, and Opencode
|
|
211
|
+
with GLM-5.3 and Kimi Code CLI with Kimi K3 without levels. The relevant
|
|
212
|
+
published v1.5 scores are:
|
|
184
213
|
|
|
185
214
|
| Harness and model | Coding Agent Index |
|
|
186
215
|
|---|---:|
|
|
@@ -196,7 +225,8 @@ Kimi Code CLI with Kimi K3 without levels. The relevant published scores are:
|
|
|
196
225
|
| Codex - GPT-6 Luna (max) | 41 |
|
|
197
226
|
|
|
198
227
|
These are coding-agent scores for specific harnesses and levels, not scores
|
|
199
|
-
for the omp role map.
|
|
228
|
+
for the omp role map. The v1.5 table stays as history. Coding Agent Index v2.1
|
|
229
|
+
below has the 2026-11-18 entries for GPT-6.2 Sol and GPT-6.2 Astra.
|
|
200
230
|
|
|
201
231
|
Column meanings:
|
|
202
232
|
|
|
@@ -212,6 +242,29 @@ Column meanings:
|
|
|
212
242
|
- **TB 4.0** - Terminal-Bench 4.0, one of the index evaluations (ten at v4.3.2,
|
|
213
243
|
nine at v5.0.1).
|
|
214
244
|
|
|
245
|
+
### Coding Agent Index v2.1
|
|
246
|
+
|
|
247
|
+
Source: `https://artificialanalysis.ai/agents/coding-agents`, read on
|
|
248
|
+
2026-11-18 at Coding Agent Index v2.1. This version gives equal weight to
|
|
249
|
+
FrontierCode-AA 2.0, Terminal-Bench 4.1, and SWE-Atlas-QnA-AA, and lists 39
|
|
250
|
+
entries. The page draws the scores only in charts. The values below are the
|
|
251
|
+
`indexScore` fields of the page data, on a 0 to 1 scale. The cost and agent
|
|
252
|
+
wall time columns are the mean per task from the same data. This capture did
|
|
253
|
+
not observe the rendered integer values. Do not compare these values with the
|
|
254
|
+
v1.5 integers above.
|
|
255
|
+
|
|
256
|
+
| Harness and model | indexScore | Cost/task | Agent wall time s |
|
|
257
|
+
|---|---:|---:|---:|
|
|
258
|
+
| Codex - GPT-6.2 Astra (max) | 0.5347 | $5.117 | 1506.7 |
|
|
259
|
+
| Codex - GPT-6.2 Astra (xhigh) | 0.5312 | $3.198 | 992.4 |
|
|
260
|
+
| Codex - GPT-6.2 Astra (high) | 0.5228 | $2.372 | 947.4 |
|
|
261
|
+
| Codex - GPT-6.2 Sol (max) | 0.4862 | $1.216 | 1051.4 |
|
|
262
|
+
| Codex - GPT-6.2 Sol (xhigh) | 0.4979 | $0.898 | 755.8 |
|
|
263
|
+
| Codex - GPT-6.2 Sol (high) | 0.4746 | $0.764 | 635.4 |
|
|
264
|
+
|
|
265
|
+
GPT-6.2 Sol scores higher at `xhigh` than at `max`. v2.1 lists GPT-6.1 Sol
|
|
266
|
+
and GPT-6.1 Terra with Codex only at max.
|
|
267
|
+
|
|
215
268
|
### Claude Opus 5.5 (Anthropic) - `anthropic/claude-opus-5-5`
|
|
216
269
|
|
|
217
270
|
| Level | Index | Cost/task | Tokens/task | Index tokens | Speed t/s | TTFT s | TB 4.0 |
|
|
@@ -294,7 +347,74 @@ baseline at v4.3.2.
|
|
|
294
347
|
Price: $10.00 in, $50.00 out, $0.25 cache hit per 1M. Context 1M. All levels
|
|
295
348
|
run with fallback.
|
|
296
349
|
|
|
297
|
-
### GPT-6
|
|
350
|
+
### GPT-6.2 Sol (OpenAI) - `openai-codex/gpt-6.2-sol`
|
|
351
|
+
|
|
352
|
+
Source: `https://artificialanalysis.ai`, read on 2026-11-18 at Intelligence
|
|
353
|
+
Index v5.0.1. The rows come from the rendered metric table of
|
|
354
|
+
`/models/comparisons/gpt-6-2-sol-vs-gpt-6-1-terra`, which shows every level of
|
|
355
|
+
both models. Tokens per task and index tokens come from the data of the model
|
|
356
|
+
page `/models/gpt-6-2-sol`, because the comparison table does not show them.
|
|
357
|
+
The other values in that data match the rendered table. These rows compare
|
|
358
|
+
only with other v5.0.1 figures, not with the v4.3.2 rows.
|
|
359
|
+
|
|
360
|
+
| Level | Index | Cost/task | Tokens/task | Index tokens | Speed t/s | TTFT s | TB 4.0 |
|
|
361
|
+
|---|---:|---:|---:|---:|---:|---:|---:|
|
|
362
|
+
| max | 57 | $1.01 | 40k | 61M | 85 | 66.70 | 60% |
|
|
363
|
+
| xhigh | 56 | $0.65 | 24k | 33M | 81 | 26.63 | 56% |
|
|
364
|
+
| high | 55 | $0.56 | 20k | 23M | 78 | 9.63 | 59% |
|
|
365
|
+
|
|
366
|
+
AA and OpenAI list $2.00 input, $0.10 cached input, $2.50 cache write, and
|
|
367
|
+
$10.00 output per 1M tokens. OpenAI also lists a 1,050,000-token context,
|
|
368
|
+
922,000 maximum input tokens, and 128,000 maximum output tokens
|
|
369
|
+
(<https://developers.openai.com/api/docs/models/gpt-6.2-sol>). Prompts above
|
|
370
|
+
272K input tokens cost 2x input and cache rates and 1.5x output rates for the
|
|
371
|
+
full request. The knowledge cutoff is 2026-06-01. OpenAI released the model on
|
|
372
|
+
2026-11-17 (<https://developers.openai.com/api/docs/changelog>). Codex uses a
|
|
373
|
+
500,000-token default context (<https://developers.openai.com/codex/models>).
|
|
374
|
+
On 2026-11-18, omp 18.22.2 listed `gpt-6.2-sol` with the same 500K context
|
|
375
|
+
window, 128K maximum output, and the levels low, medium, high, xhigh, and max.
|
|
376
|
+
|
|
377
|
+
`retry.fallbackChains.default` and `.task` moved from GPT-6.1 Sol `high` to
|
|
378
|
+
GPT-6.2 Sol `high`, and `.vision` moved at `medium` (DocksDocks/public#175).
|
|
379
|
+
On 2026-11-18, the v5.0.1 data of the model page `/models/gpt-6-1-sol` listed
|
|
380
|
+
GPT-6.1 Sol high at index 50 and $0.59 per task. GPT-6.2 Sol high has 55 and
|
|
381
|
+
$0.56.
|
|
382
|
+
|
|
383
|
+
### GPT-6.2 Astra (OpenAI) - `openai-codex/gpt-6.2-astra`
|
|
384
|
+
|
|
385
|
+
Source: `https://artificialanalysis.ai`, read on 2026-11-18 at Intelligence
|
|
386
|
+
Index v5.0.1. The rows come from the rendered metric table of
|
|
387
|
+
`/models/comparisons/gpt-6-2-astra-vs-gpt-6-2-sol`. Tokens per task and index
|
|
388
|
+
tokens come from the data of the model page `/models/gpt-6-2-astra`. These
|
|
389
|
+
rows compare only with other v5.0.1 figures, not with the v4.3.2 rows.
|
|
390
|
+
|
|
391
|
+
| Level | Index | Cost/task | Tokens/task | Index tokens | Speed t/s | TTFT s | TB 4.0 |
|
|
392
|
+
|---|---:|---:|---:|---:|---:|---:|---:|
|
|
393
|
+
| max | 62 | $4.00 | 41k | 66M | 53 | 170.99 | 64% |
|
|
394
|
+
| xhigh | 62 | $2.52 | 22k | 27M | 54 | 52.89 | 64% |
|
|
395
|
+
| high | 60 | $2.07 | 18k | 20M | 56 | 22.07 | 60% |
|
|
396
|
+
|
|
397
|
+
The `max` and `xhigh` rows both round to 62. The page data gives 62.14 and
|
|
398
|
+
61.56.
|
|
399
|
+
|
|
400
|
+
AA and OpenAI list $8.00 input, $0.40 cached input, $10.00 cache write, and
|
|
401
|
+
$40.00 output per 1M tokens. OpenAI also lists a 1,050,000-token context,
|
|
402
|
+
922,000 maximum input tokens, and 128,000 maximum output tokens
|
|
403
|
+
(<https://developers.openai.com/api/docs/models/gpt-6.2-astra>). The
|
|
404
|
+
long-prompt rule of GPT-6.2 Sol also applies. The knowledge cutoff is
|
|
405
|
+
2026-05-18. OpenAI released the model on 2026-11-17, in the same changelog
|
|
406
|
+
entry as GPT-6.2 Sol. Codex uses a 272,000-token default context. On
|
|
407
|
+
2026-11-18, omp 18.22.2 listed `gpt-6.2-astra` with the same 272K context
|
|
408
|
+
window, 128K maximum output, and the levels low, medium, high, xhigh, and max.
|
|
409
|
+
|
|
410
|
+
`modelRoles.astra` and `retry.fallbackChains.fable` moved from GPT-6 Astra
|
|
411
|
+
`xhigh` to GPT-6.2 Astra `xhigh` (DocksDocks/public#175). GPT-6 Astra below
|
|
412
|
+
has the v5.0.1 figures for a like-for-like comparison.
|
|
413
|
+
|
|
414
|
+
### GPT-6 Astra (OpenAI) - previous generation
|
|
415
|
+
|
|
416
|
+
The role map and the retry chains no longer use GPT-6 Astra. This table stays
|
|
417
|
+
as the v4.3.2 baseline for the previous `astra` role.
|
|
298
418
|
|
|
299
419
|
| Level | Index | Cost/task | Tokens/task | Index tokens | Speed t/s | TTFT s | TB 4.0 |
|
|
300
420
|
|---|---:|---:|---:|---:|---:|---:|---:|
|
|
@@ -307,7 +427,17 @@ run with fallback.
|
|
|
307
427
|
Price: $10.00 in, $50.00 out, $1.00 cache hit per 1M. Context 1M. Knowledge
|
|
308
428
|
cutoff 2026-04-30. AA publishes no non-reasoning Astra row.
|
|
309
429
|
|
|
310
|
-
|
|
430
|
+
On 2026-11-18, the v5.0.1 data of the AA model page `/models/gpt-6-astra`
|
|
431
|
+
listed GPT-6 Astra `xhigh` at index 55 and $3.66 per task, with 44 t/s and an
|
|
432
|
+
83.77 s TTFT. It listed `high` at index 55, $3.24, 44 t/s, and 40.21 s. At
|
|
433
|
+
`xhigh`, GPT-6.2 Astra scores 62 against 55 and costs $2.52 against $3.66 per
|
|
434
|
+
task. It produces 54 against 44 t/s, with a TTFT of 52.89 s against 83.77 s.
|
|
435
|
+
AA marks GPT-6 Astra deprecated, so its comparison pages return HTTP 404.
|
|
436
|
+
|
|
437
|
+
### GPT-6.1 Sol (OpenAI) - previous generation
|
|
438
|
+
|
|
439
|
+
The role map and the retry chains no longer use GPT-6.1 Sol. This table stays
|
|
440
|
+
as the v4.3.2 baseline for the former `task` role and retry fallbacks.
|
|
311
441
|
|
|
312
442
|
| Level | Index | Cost/task | Tokens/task | Index tokens | Speed t/s | TTFT s | TB 4.0 |
|
|
313
443
|
|---|---:|---:|---:|---:|---:|---:|---:|
|
|
@@ -323,7 +453,9 @@ OpenAI also lists $2.50 per 1M cache-write tokens, a 1,050,000-token context,
|
|
|
323
453
|
272K input tokens cost 2x input and cache rates and 1.5x output rates for the
|
|
324
454
|
full request. The knowledge cutoff is 2026-04-30. The API effort ladder is
|
|
325
455
|
`low, medium, high, xhigh, max`, with `medium` as the default. There is no
|
|
326
|
-
`none`, `minimal`, or non-reasoning level.
|
|
456
|
+
`none`, `minimal`, or non-reasoning level. On 2026-11-18, omp 18.22.2 still
|
|
457
|
+
served `gpt-6.1-sol`, and the OpenAI deprecations page did not list it. AA
|
|
458
|
+
marks it deprecated.
|
|
327
459
|
|
|
328
460
|
### GPT-6.1 Terra (OpenAI) - `openai-codex/gpt-6.1-terra`
|
|
329
461
|
|
|
@@ -346,6 +478,14 @@ For comparison at the same index version, AA lists GPT-6.1 Sol high at index
|
|
|
346
478
|
TTFT, and 52% on Terminal-Bench 4.0. The same Sol high row read 50, $0.32, and
|
|
347
479
|
57.26 s at v4.3.2, which shows why the two captures do not mix.
|
|
348
480
|
|
|
481
|
+
On 2026-11-18, the release comparison page
|
|
482
|
+
`/models/comparisons/gpt-6-2-sol-vs-gpt-6-1-terra` showed new speed and TTFT
|
|
483
|
+
figures for GPT-6.1 Terra at v5.0.1. `high` ran at 65 t/s with a 10.50 s
|
|
484
|
+
TTFT, `xhigh` at 65 t/s with 30.33 s, and `max` at 69 t/s with 146.49 s. The
|
|
485
|
+
index, cost, and token figures were unchanged. AA measures speed and latency
|
|
486
|
+
continuously, so the table keeps the 2026-11-05 rows. The speed comparison
|
|
487
|
+
below uses the 2026-11-18 figures.
|
|
488
|
+
|
|
349
489
|
AA and OpenAI list $1.00 input, $5.00 output, and $0.05 cached input per 1M
|
|
350
490
|
tokens. OpenAI also lists a 1,050,000-token context, 922,000 maximum input
|
|
351
491
|
tokens, and 128,000 maximum output tokens
|
|
@@ -353,6 +493,30 @@ tokens, and 128,000 maximum output tokens
|
|
|
353
493
|
Fast mode costs 2x Standard. The knowledge cutoff is 2026-06-01. The API effort
|
|
354
494
|
ladder is `none, low, medium, high, xhigh, max`, with `medium` as the default.
|
|
355
495
|
|
|
496
|
+
### Output speed: GPT-6.2 Sol and GPT-6.1 Terra
|
|
497
|
+
|
|
498
|
+
GPT-6.2 Sol has the higher output speed at `high` and at `xhigh`.
|
|
499
|
+
The figures come from one page,
|
|
500
|
+
`/models/comparisons/gpt-6-2-sol-vs-gpt-6-1-terra`, read on 2026-11-18 at
|
|
501
|
+
Intelligence Index v5.0.1.
|
|
502
|
+
|
|
503
|
+
| Metric | GPT-6.2 Sol | GPT-6.1 Terra |
|
|
504
|
+
|---|---:|---:|
|
|
505
|
+
| `high` output speed | 78 t/s | 65 t/s |
|
|
506
|
+
| `xhigh` output speed | 81 t/s | 65 t/s |
|
|
507
|
+
| `high` TTFT | 9.63 s | 10.50 s |
|
|
508
|
+
| `xhigh` TTFT | 26.63 s | 30.33 s |
|
|
509
|
+
| `high` end-to-end time | 42.65 s | 43.91 s |
|
|
510
|
+
| `xhigh` end-to-end time | 85.03 s | 86.53 s |
|
|
511
|
+
|
|
512
|
+
At `high`, Sol produces about 19% more tokens per second than Terra. At
|
|
513
|
+
`xhigh`, the gain is about 24%. Sol also has the lower TTFT and end-to-end
|
|
514
|
+
time at both levels.
|
|
515
|
+
|
|
516
|
+
The `reviewer` role stays on GPT-6.1 Terra `high`. The #130 benchmark chose
|
|
517
|
+
Terra by escaped defects, not by speed. omp 18.22.2 lists no GPT-6.2 Terra. A
|
|
518
|
+
move of the reviewer to GPT-6.2 Sol needs a new benchmark.
|
|
519
|
+
|
|
356
520
|
### GPT-6 Luna (OpenAI) - `openai-codex/gpt-6-luna`
|
|
357
521
|
|
|
358
522
|
| Level | Index | Cost/task | Tokens/task | Index tokens | Speed t/s | TTFT s | TB 4.0 |
|
|
@@ -373,7 +537,7 @@ has measured both at the other levels.
|
|
|
373
537
|
### GPT-5.6 Sol (OpenAI) - previous generation
|
|
374
538
|
|
|
375
539
|
The role map no longer uses GPT-5.6 Sol. This table stays as the measured
|
|
376
|
-
baseline for the GPT-6.1 Sol `task` retry fallback.
|
|
540
|
+
baseline for the former GPT-6.1 Sol `task` retry fallback.
|
|
377
541
|
|
|
378
542
|
| Level | Index | Cost/task | Tokens/task | Index tokens | Speed t/s | TTFT s | TB 4.0 |
|
|
379
543
|
|---|---:|---:|---:|---:|---:|---:|---:|
|
|
@@ -407,12 +571,16 @@ Price: $0.20 in, $1.20 out, $0.02 cache hit per 1M. Context 1M.
|
|
|
407
571
|
|
|
408
572
|
`task` runs `anthropic/claude-opus-5-5:high`, the same model and level as
|
|
409
573
|
`default`. The owner moved it from `openai-codex/gpt-6.1-sol:high` on
|
|
410
|
-
2026-10-09. GPT-6.1 Sol high
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
574
|
+
2026-10-09. GPT-6.1 Sol high then became the `task` retry fallback. Since
|
|
575
|
+
DocksDocks/public#175, the fallback is GPT-6.2 Sol high
|
|
576
|
+
(`openai-codex/gpt-6.2-sol:high`). The owner uses Astra only for main
|
|
577
|
+
orchestration, so Astra has a dedicated `astra` cycle stop at `xhigh`. This
|
|
578
|
+
comparison keeps the retired Astra-low, GPT-5.6 Sol, and GPT-6.1 Sol choices
|
|
579
|
+
beside the current Opus level. It is the v4.3.2 capture of 2026-09-29, so it
|
|
580
|
+
does not compare with the v5.0.1 GPT-6.2 rows. GPT-6.2 Sol above has the
|
|
581
|
+
figures for the current fallback.
|
|
582
|
+
|
|
583
|
+
| Metric | Astra low, retired | GPT-5.6 Sol high, retired | GPT-6.1 Sol high, former fallback | GPT-6.1 Sol max | Opus 5.5 high, current |
|
|
416
584
|
|---|---:|---:|---:|---:|---:|
|
|
417
585
|
| Intelligence Index | 46 | 42 | 50 | 52 | 54 |
|
|
418
586
|
| Cost per Index task | $0.82 | $0.81 | $0.32 | $0.72 | $1.82 |
|
|
@@ -443,9 +611,9 @@ Opus 5.5 high scores 54 against GPT-6.1 Sol high at 50, 57% against 52% on
|
|
|
443
611
|
Terminal-Bench 4.0, and 1705 against 1471 on AA-Briefcase. It costs $1.82
|
|
444
612
|
against $0.32 per index task. Its answer TTFT is 52.85 s against 57.26 s.
|
|
445
613
|
|
|
446
|
-
The `astra` cycle stop runs xhigh
|
|
447
|
-
|
|
448
|
-
|
|
614
|
+
The `astra` cycle stop runs GPT-6.2 Astra `xhigh`. At v5.0.1 it has index 62,
|
|
615
|
+
$2.52 per index task, and a 52.89 s TTFT (see GPT-6.2 Astra). Coding Agent
|
|
616
|
+
Index v2.1 lists Codex with GPT-6.2 Astra at high, xhigh, and max.
|
|
449
617
|
|
|
450
618
|
## How Astra is selected in practice
|
|
451
619
|
|
|
@@ -456,10 +624,13 @@ No subagent selects Astra automatically.
|
|
|
456
624
|
|
|
457
625
|
`SoT/.omp/models.yml` declares the full `low, medium, high, xhigh, max` ladder
|
|
458
626
|
with `defaultLevel: xhigh` under
|
|
459
|
-
`providers.openai-codex.modelOverrides.gpt-6-astra.thinking`.
|
|
627
|
+
`providers.openai-codex.modelOverrides.gpt-6.2-astra.thinking`.
|
|
460
628
|
The override stays as the kit's worked example of a provider ladder
|
|
461
629
|
redefinition. The in-session thinking control can still select `low` for a
|
|
462
|
-
quick answer.
|
|
630
|
+
quick answer. The block is keyed by model id, so it moved from `gpt-6-astra`
|
|
631
|
+
to `gpt-6.2-astra` with the role. `ompRemovals.ts syncOmpModelRemovals`
|
|
632
|
+
prunes the deployed `gpt-6-astra` block while it equals the shipped block, and
|
|
633
|
+
keeps a block that the user edited.
|
|
463
634
|
|
|
464
635
|
- The bundled `scout` and `sonic` agents carry `model: "@smol"` and
|
|
465
636
|
`thinking-level: medium` in their embedded frontmatter, so they run Luna,
|
|
@@ -493,9 +664,12 @@ The runtime per-agent measurement from fresh `omp -p` runs on 2026-09-09
|
|
|
493
664
|
predates this change. It applies to the retired Astra-low `task` configuration,
|
|
494
665
|
not the current role map.
|
|
495
666
|
|
|
496
|
-
|
|
497
|
-
`/extended-context
|
|
498
|
-
|
|
667
|
+
On 2026-11-18, omp 18.22.2 listed GPT-6.2 Astra with a 272,000-token context
|
|
668
|
+
window. This is the `/extended-context off` window, and it equals the Codex
|
|
669
|
+
default context. OpenAI lists a 1,050,000-token context and 922,000 maximum
|
|
670
|
+
input tokens. For GPT-6 Astra, `/extended-context on` used the 922k input
|
|
671
|
+
window. The kit has not checked the window that `/extended-context on` gives
|
|
672
|
+
for GPT-6.2 Astra.
|
|
499
673
|
|
|
500
674
|
## Free session launcher (`docks-kit omp`)
|
|
501
675
|
|
|
@@ -680,10 +854,13 @@ and the ladder counts in these docs in the same commit.
|
|
|
680
854
|
values, and exits without starting a session. `priority` is the tier omp's
|
|
681
855
|
`/fast on` selects for OpenAI and OpenAI-Codex models. `--fast` over
|
|
682
856
|
`ultrafast` selects `priority`. The kit ships no Ultrafast toggle: omp honors
|
|
683
|
-
`ultrafast` for a Codex model only when its catalog `serviceTiers` lists it
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
857
|
+
`ultrafast` for a Codex model only when its catalog `serviceTiers` lists it.
|
|
858
|
+
The omp 18.4.9 catalog listed none for GPT-6.1 Sol, Astra, or Luna. On
|
|
859
|
+
2026-11-18, `omp models openai-codex --json` on omp 18.22.2 listed no
|
|
860
|
+
`serviceTiers` for any `gpt-6` id, `gpt-6.2-sol` and `gpt-6.2-astra`
|
|
861
|
+
included. The SoT does not declare `tier.openai`, so the additive sync merge
|
|
862
|
+
keeps the toggled value. New sessions read it; a running session changes
|
|
863
|
+
through `/fast`.
|
|
687
864
|
|
|
688
865
|
The four review agents inherit the live tier of the main session
|
|
689
866
|
(`task.agentServiceTierOverrides: inherit`). In a new session after
|
|
@@ -702,12 +879,27 @@ on normal processing, use `docks-kit omp --fast` and a new session, not
|
|
|
702
879
|
|
|
703
880
|
## Maintenance
|
|
704
881
|
|
|
705
|
-
- Refresh each per-level row from the metric table of
|
|
706
|
-
|
|
707
|
-
|
|
708
|
-
|
|
709
|
-
|
|
710
|
-
|
|
882
|
+
- Refresh each per-level row from the rendered metric table of a release
|
|
883
|
+
comparison page, for example
|
|
884
|
+
`/models/comparisons/gpt-6-2-sol-vs-gpt-6-1-terra` and
|
|
885
|
+
`/models/comparisons/gpt-6-2-astra-vs-gpt-6-2-sol`. Each page shows every
|
|
886
|
+
level of both models. Take tokens per task and index tokens from the data
|
|
887
|
+
of the model page, for example `/models/gpt-6-2-sol`, and check that its
|
|
888
|
+
other values match the rendered table. A model-page chart holds only about
|
|
889
|
+
20 models, so a value missing there does not mean that AA has not measured
|
|
890
|
+
it.
|
|
891
|
+
- On 2026-11-18, the per-level pattern
|
|
892
|
+
`/models/comparisons/<level-slug>-vs-gpt-5-6-sol-high` returned HTTP 404. A
|
|
893
|
+
comparison with a model that AA marks deprecated also returned HTTP 404, for
|
|
894
|
+
example `gpt-6-1-sol`, `gpt-6-astra`, or `gpt-5-6-sol`. Read such a model
|
|
895
|
+
from the data of its model page.
|
|
896
|
+
- Keep AA rows at `high` and `xhigh` at least for each model that an omp role
|
|
897
|
+
or retry chain uses, with the index version and the capture date (see AA
|
|
898
|
+
rows for each role model). Add the missing rows in the same commit as a
|
|
899
|
+
role or chain change.
|
|
900
|
+
- After a role or chain change, run `omp models openai-codex` and check that
|
|
901
|
+
it lists every `openai-codex` id that the roles and chains name. Sync does
|
|
902
|
+
not check the omp catalog.
|
|
711
903
|
- Refresh the Coding Agent Index from `/agents/coding-agents`.
|
|
712
904
|
- Record the index version with the numbers. AA changes index composition
|
|
713
905
|
between versions, so a score from another version is not a comparison.
|
|
@@ -104,6 +104,10 @@ const OMP_RETIRED_KEYS: ReadonlyArray<string> = ["providers.webSearchOrder"];
|
|
|
104
104
|
* served the id as an empty stub. The catalog now publishes the same limits,
|
|
105
105
|
* prices, and ladder. The block's `defaultLevel: high` applied only to a bare
|
|
106
106
|
* selector, and every kit selector names its level.
|
|
107
|
+
*
|
|
108
|
+
* `gpt-6-astra` carried the kit's worked ladder override while the `astra`
|
|
109
|
+
* role ran GPT-6 Astra. The role moved to GPT-6.2 Astra, and the override
|
|
110
|
+
* moved with it to the `gpt-6.2-astra` key, so no role uses this block.
|
|
107
111
|
*/
|
|
108
112
|
const OMP_RETIRED_MODEL_BLOCKS: ReadonlyArray<readonly [string, unknown]> = [
|
|
109
113
|
[
|
|
@@ -122,6 +126,16 @@ const OMP_RETIRED_MODEL_BLOCKS: ReadonlyArray<readonly [string, unknown]> = [
|
|
|
122
126
|
},
|
|
123
127
|
},
|
|
124
128
|
],
|
|
129
|
+
[
|
|
130
|
+
"providers.openai-codex.modelOverrides.gpt-6-astra",
|
|
131
|
+
{
|
|
132
|
+
thinking: {
|
|
133
|
+
mode: "effort",
|
|
134
|
+
efforts: ["low", "medium", "high", "xhigh", "max"],
|
|
135
|
+
defaultLevel: "xhigh",
|
|
136
|
+
},
|
|
137
|
+
},
|
|
138
|
+
],
|
|
125
139
|
];
|
|
126
140
|
|
|
127
141
|
/** Each two-letter short-option cluster with a first letter from `first` and a second from `second`. */
|