@open-agent-toolkit/cli 0.2.19 → 0.2.20

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,5 +1,5 @@
1
1
  {
2
- "version": "2026-07-11.1",
2
+ "version": "2026-07-25.1",
3
3
  "providers": {
4
4
  "codex": {
5
5
  "economy": {
@@ -102,23 +102,33 @@
102
102
  "economy": {
103
103
  "candidates": [
104
104
  "composer-2.5",
105
- "claude-sonnet-5-high",
106
105
  "gpt-5.6-luna-high",
107
106
  "gpt-5.6-luna-xhigh"
108
107
  ]
109
108
  },
110
109
  "balanced": {
111
- "candidates": ["cursor-grok-4.5-high", "gpt-5.6-terra-high"]
110
+ "candidates": [
111
+ "cursor-grok-4.5-high",
112
+ "gpt-5.6-terra-high",
113
+ "claude-opus-5-thinking-low"
114
+ ]
112
115
  },
113
116
  "high": {
114
- "candidates": ["gpt-5.6-sol-medium", "gpt-5.6-sol-high"]
117
+ "candidates": [
118
+ "gpt-5.6-sol-medium",
119
+ "gpt-5.6-sol-high",
120
+ "claude-opus-5-thinking-medium",
121
+ "claude-opus-5-thinking-high"
122
+ ]
115
123
  },
116
124
  "frontier": {
117
125
  "candidates": [
118
126
  "claude-fable-5-thinking-high",
119
127
  "claude-fable-5-thinking-xhigh",
120
128
  "gpt-5.6-sol-xhigh",
121
- "gpt-5.6-sol-max"
129
+ "gpt-5.6-sol-max",
130
+ "claude-opus-5-thinking-xhigh",
131
+ "claude-opus-5-thinking-max"
122
132
  ]
123
133
  }
124
134
  }
@@ -361,13 +361,18 @@ even when the reusable ladder is user-owned.
361
361
  }
362
362
  ```
363
363
 
364
+ The example above is illustrative rather than a copy of the bundled ladder; its
365
+ tiers are trimmed for readability.
366
+
364
367
  The bundled recommendation covers 13 Codex model/effort combinations: Luna and
365
368
  Terra at `low`, `medium`, `high`, and `xhigh`, plus Sol at those efforts and
366
- `max`. Claude covers `haiku`, `sonnet`, `opus`, and `fable`. Cursor covers 12
367
- verified multi-family flat IDs across Composer, Claude, GPT, and Grok. An
368
- explicit mapping connects each flat ladder ID to a separate bracket-form
369
- frontmatter model; configuration and skills never derive or normalize either
370
- form.
369
+ `max`. Claude covers `haiku`, `sonnet`, `opus`, and `fable`. Cursor covers 16
370
+ candidates across four tiers, drawn from a materialization catalogue of 18
371
+ verified multi-family flat IDs spanning Composer, Claude (Sonnet, Opus, and
372
+ Fable), GPT, and Grok; the two figures differ because some approved mappings
373
+ stay materializable without being recommended. An explicit mapping connects
374
+ each flat ladder ID to a separate bracket-form frontmatter model; configuration
375
+ and skills never derive or normalize either form.
371
376
 
372
377
  The corresponding pinned Codex variant catalogue includes
373
378
  `gpt-5.6-luna-high`, `gpt-5.6-terra-xhigh`, `gpt-5.6-sol-high`, and
@@ -17,3 +17,4 @@ Use this section when you are changing OAT itself rather than consuming it as a
17
17
  - [Commit Conventions](commit-conventions.md) - Traceable commit patterns for project and docs work.
18
18
  - [Hooks and Safety](hooks-and-safety.md) - Safety and mutation contracts for provider-sync behavior.
19
19
  - [Smoke Testing](smoke-testing.md) - Runbook for the live workflow smoke runner: prerequisites, scenarios, evidence reports, and recovery.
20
+ - [Verifying Cursor Pins](verifying-cursor-pins.md) - Runbook for probe-verifying a Cursor model pin before shipping it.
@@ -0,0 +1,282 @@
1
+ ---
2
+ title: Verifying Cursor Pins
3
+ description: 'Runbook for probe-verifying a Cursor model pin before shipping it: evidence channel, hook capture, subjects and controls, interpretation, and cleanup.'
4
+ ---
5
+
6
+ # Verifying Cursor Pins
7
+
8
+ A Cursor model pin maps an OAT ladder model ID to the frontmatter selector that
9
+ materialized role files carry, for example `claude-opus-5-thinking-medium` to
10
+ `claude-opus-5[effort=medium]`. Adding one to
11
+ `packages/cli/src/providers/cursor/codec/catalog.ts` requires evidence that
12
+ Cursor actually resolves that selector to the intended model.
13
+
14
+ This runbook produces that evidence. Run it before adding any mapping.
15
+
16
+ ## Why a probe is required
17
+
18
+ `DR-260718-explicit-cursor-pin-mapping` requires mapping-specific,
19
+ native-launch evidence before a mapping may be marked `approved`. Two things
20
+ that look like evidence are not:
21
+
22
+ - **Structural validation.** That a mapping is well-formed says nothing about
23
+ what Cursor does with it.
24
+ - **Agent self-report.** A subagent asked to name its own model is not a
25
+ reliable witness, and the probe agents in this runbook deliberately do not
26
+ ask.
27
+
28
+ The catalog alone is also insufficient wherever a family carries both a
29
+ thinking and a non-thinking flat ID at the same rung. Opus 5 carries both at
30
+ low, medium, and high, so the bracket selector is genuinely ambiguous there and
31
+ only a probe can resolve it. At rungs where only one variant exists, the
32
+ catalog settles the question without a probe — but you still need the probe to
33
+ prove the effort parameter was honored at all.
34
+
35
+ ## Evidence channel
36
+
37
+ Cursor's agent lifecycle hooks report resolution from Cursor itself. Two
38
+ independent events agree:
39
+
40
+ - `subagentStart.subagent_model` — the model the subagent will use, and the
41
+ authoritative field per Cursor's hooks reference.
42
+ - `preToolUse.model` — observed from inside the subagent's own tool call.
43
+
44
+ Two channels that do **not** work:
45
+
46
+ - **The `cursor-agent` CLI and remote runtime.** Agent lifecycle hooks do not
47
+ fire there. A Task subagent launched from that runtime produces zero hook
48
+ events, and every row records as not-reported. Run probes from the Cursor
49
+ desktop app's Agent Chat.
50
+ - **The subagent card label.** It is built from the requested bracket
51
+ parameters and drops the thinking qualifier, so it cannot distinguish the
52
+ thinking and non-thinking variants — precisely the distinction a probe exists
53
+ to settle.
54
+
55
+ As of Cursor 3.12.30, `model_params` (which would carry an explicit `thinking`
56
+ flag) is empty and `model_id` is absent. Thinking is therefore established by
57
+ the resolved slug itself, which is sound because the non-thinking IDs are
58
+ distinct.
59
+
60
+ ## Prerequisites
61
+
62
+ - Cursor desktop app, with the repository open as a workspace.
63
+ - The live catalog for the families you intend to pin:
64
+
65
+ ```bash
66
+ cursor-agent models
67
+ ```
68
+
69
+ Read the full output rather than truncating it. You need to see whether a
70
+ thinking and a non-thinking ID both exist at each rung, and which rung carries
71
+ the family's unqualified display label.
72
+
73
+ ## Setting up capture
74
+
75
+ Both files are temporary. Delete them when the probe is done.
76
+
77
+ Create `.cursor/hooks/g01-capture.sh`:
78
+
79
+ ```bash
80
+ #!/bin/bash
81
+ # TEMPORARY pin-probe capture hook. Delete after probing.
82
+ set -u
83
+
84
+ OUT_DIR="/tmp/g01-probe"
85
+ OUT="$OUT_DIR/hooks.jsonl"
86
+ mkdir -p "$OUT_DIR" 2>/dev/null || :
87
+
88
+ input=$(cat)
89
+ ts=$(date -u '+%Y-%m-%dT%H:%M:%SZ')
90
+
91
+ printf '{"captured_at":"%s","payload":%s}\n' "$ts" "$input" >>"$OUT" 2>/dev/null || :
92
+
93
+ event=$(printf '%s' "$input" |
94
+ sed -n 's/.*"hook_event_name"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p')
95
+
96
+ # Observe only. Never block, never fail closed.
97
+ case "$event" in
98
+ subagentStop) printf '{}\n' ;;
99
+ *) printf '{"permission":"allow"}\n' ;;
100
+ esac
101
+
102
+ exit 0
103
+ ```
104
+
105
+ Make it executable, then register it in `.cursor/hooks.json` for
106
+ `subagentStart`, `subagentStop`, and `preToolUse`:
107
+
108
+ ```json
109
+ {
110
+ "version": 1,
111
+ "hooks": {
112
+ "subagentStart": [
113
+ { "command": ".cursor/hooks/g01-capture.sh", "timeout": 10 }
114
+ ],
115
+ "subagentStop": [
116
+ { "command": ".cursor/hooks/g01-capture.sh", "timeout": 10 }
117
+ ],
118
+ "preToolUse": [{ "command": ".cursor/hooks/g01-capture.sh", "timeout": 10 }]
119
+ }
120
+ }
121
+ ```
122
+
123
+ The hook must observe only. A capture hook that can block or fail closed can
124
+ change the behavior it is supposed to measure.
125
+
126
+ Restart Cursor or reload the window so the hooks register.
127
+
128
+ ## Writing probe agents
129
+
130
+ One throwaway agent per selector, under `.cursor/agents/`. Keep them trivial:
131
+ the probe is the hook payload, not anything the agent produces.
132
+
133
+ ````markdown
134
+ ---
135
+ # TEMPORARY pin probe - not oat-managed. Delete after probing.
136
+ # Class: subject - expect claude-opus-5-thinking-medium
137
+ name: zz-pin-probe-opus5-medium
138
+ description: Temporary pin probe (subject) for claude-opus-5[effort=medium]. Delete after use.
139
+ model: claude-opus-5[effort=medium]
140
+ ---
141
+
142
+ ## Role
143
+
144
+ You are a throwaway pin probe. Run exactly one command and then stop:
145
+
146
+ ```bash
147
+ echo PIN-PROBE zz-pin-probe-opus5-medium
148
+ ```
149
+
150
+ Then reply with the single word `done`. Do not read files or use any
151
+ other tool.
152
+
153
+ Cursor's hook payload is the evidence for this probe. Your own claim about
154
+ your identity is not evidence and is deliberately not requested.
155
+ ````
156
+
157
+ The `echo` exists to force a tool call, which is what produces the
158
+ corroborating `preToolUse` event.
159
+
160
+ ### Subjects and controls
161
+
162
+ Probe every rung you intend to pin. Then add controls, which are what make the
163
+ subject results interpretable:
164
+
165
+ | Class | Example selector | Proves |
166
+ | ---------------- | ------------------------------ | ---------------------------------------------------------------- |
167
+ | Subject | `claude-opus-5[effort=medium]` | The mapping resolves as intended |
168
+ | Positive control | `claude-sonnet-5[effort=high]` | Reproduces an already-verified mapping |
169
+ | Negative control | `claude-opus-9[effort=high]` | Unknown family — the channel reports resolution, not the request |
170
+ | Negative control | `claude-opus-5[effort=ultra]` | Unknown effort — exposes rung-level fallback |
171
+
172
+ The negative controls carry most of the interpretive weight. Without them you
173
+ cannot distinguish a hook that reports what Cursor actually resolved from one
174
+ that merely echoes what you asked for. A bogus family that comes back as the
175
+ bogus family would invalidate the entire run.
176
+
177
+ ## Running
178
+
179
+ Launch each probe agent from the Cursor desktop app's Agent Chat, as a subagent
180
+ with no model override, so Cursor resolves from the agent's frontmatter. Let
181
+ each finish before starting the next.
182
+
183
+ Then read the resolutions:
184
+
185
+ ```bash
186
+ python3 - <<'PY'
187
+ import json
188
+ for line in open('/tmp/g01-probe/hooks.jsonl'):
189
+ p = json.loads(line).get('payload', {})
190
+ if p.get('hook_event_name') == 'subagentStart':
191
+ print(p.get('subagent_name'), '->', p.get('subagent_model'))
192
+ PY
193
+ ```
194
+
195
+ ## Interpreting results
196
+
197
+ ### Unresolvable components fall back silently
198
+
199
+ Cursor does not reject a malformed pin. It substitutes a default for whichever
200
+ component it cannot resolve, with no error or warning:
201
+
202
+ - An unknown **family** falls back to the account default model. Probing
203
+ `claude-opus-9[effort=high]` resolved to `cursor-grok-4.5-high-fast`.
204
+ - An unknown **effort** falls back to that family's default rung. Probing
205
+ `claude-opus-5[effort=ultra]` resolved to `claude-opus-5-thinking-high`.
206
+
207
+ This makes a typo in a selector more dangerous than a plain downgrade: the pin
208
+ silently tracks whatever Cursor currently designates as the default, so a
209
+ vendor-side change alters capability with no change in this repository.
210
+
211
+ ### The default rung is family-specific
212
+
213
+ Exactly one rung per family carries an unqualified display label in the live
214
+ catalog, and that marks the default. It is not always `high`:
215
+
216
+ | Family | Default rung | Unqualified label | Contrast |
217
+ | -------- | ------------ | ----------------- | ------------------------ |
218
+ | Opus 5 | high | `Opus 5 1M` | `Opus 5 1M Medium` |
219
+ | Opus 4.8 | high | `Opus 4.8 1M` | `Opus 4.8 1M Extra High` |
220
+ | Opus 4.7 | **xhigh** | `Opus 4.7 1M` | `Opus 4.7 1M High` |
221
+
222
+ Opus 4.7 is the disconfirming case for a naive "high is always the default"
223
+ reading.
224
+
225
+ ### A default-rung result proves nothing on its own
226
+
227
+ If the rung you requested happens to be the family default, the observation is
228
+ equally consistent with the effort parameter being honored and with it being
229
+ ignored. That row cannot stand alone.
230
+
231
+ Always probe at least one non-default rung in the same family. A set of
232
+ non-default rungs that each resolve as requested is what demonstrates the
233
+ parameter is genuinely honored; a default-rung row is then sound by inference
234
+ across the set rather than from its own evidence.
235
+
236
+ ## Recording the result
237
+
238
+ Add the mapping with a probe record, which ties it to the evidence that
239
+ approved it:
240
+
241
+ ```ts
242
+ approvedMapping(
243
+ 'claude-opus-5-thinking-medium',
244
+ 'claude-opus-5[effort=medium]',
245
+ 'claude-effort',
246
+ { probeName: 'zz-pin-probe-opus5-medium', verifiedAt: '2026-07-25', evidencePath: '...' },
247
+ ),
248
+ ```
249
+
250
+ The record asserts that `submittedSelector` equals the mapping's
251
+ `frontmatterModel` and `resolvedModel` equals its `ladderModelId`. A mapping
252
+ edited later without re-probing therefore fails its own consistency test rather
253
+ than inheriting an approval that never covered the new selector.
254
+
255
+ Retain the raw payloads alongside the summary. They are the primary evidence;
256
+ the summary is an interpretation of them.
257
+
258
+ ## Cleanup
259
+
260
+ Delete every temporary artifact:
261
+
262
+ - `.cursor/hooks.json` and `.cursor/hooks/g01-capture.sh`
263
+ - all `zz-pin-probe-*` agent files
264
+ - the capture file under `/tmp`
265
+
266
+ Before committing retained payloads, redact machine and identity fields.
267
+ Cursor hook payloads carry `user_email`, `workspace_roots`, and an absolute
268
+ `transcript_path`. None carries probe evidence — resolution is read from
269
+ `subagent_model` — so removing them costs nothing and keeps private data out of
270
+ the repository.
271
+
272
+ Confirm the workspace is clean:
273
+
274
+ ```bash
275
+ oat sync --scope project
276
+ ```
277
+
278
+ ## Related
279
+
280
+ - [Smoke Testing](smoke-testing.md) - Runbook for the live workflow smoke runner.
281
+ - [Dispatch Policy](../workflows/projects/dispatch-ceiling.md) - How pins reach the recommendation and role files.
282
+ - [Provider Sync](../provider-sync/providers.md) - Provider-specific sync behavior.
@@ -64,6 +64,23 @@ existing cells. Planning shows the complete recommendation before asking which
64
64
  scope should own it. If the resulting ladder is still missing or incomplete,
65
65
  planning remains blocked rather than replacing the user's explicit values.
66
66
 
67
+ ### Upgrading to a newer recommendation version
68
+
69
+ Preservation applies to whole cells, which has a consequence worth stating
70
+ plainly: when a new recommendation version adds candidates to a tier you have
71
+ already populated, re-running adoption will not give them to you. The existing
72
+ cell is kept intact rather than merged candidate by candidate. Removals are not
73
+ propagated either.
74
+
75
+ To pick up a new version, compare your
76
+ `workflow.dispatchCeiling.recommendationVersion` against the bundled version,
77
+ then either edit the affected cells by hand or clear them and re-adopt.
78
+
79
+ Version `2026-07-25.1` is a live example: it adds Opus 5 rungs to the Cursor
80
+ `balanced`, `high`, and `frontier` tiers and drops `claude-sonnet-5-high` from
81
+ `economy`. An adopter still on the prior version keeps their existing Cursor
82
+ tiers untouched until they take one of those actions.
83
+
67
84
  Before offering adoption, planning runs `oat config list --json` once and treats
68
85
  its output as the effective boundary across shared, repo-local, user, and
69
86
  bundled-default precedence. A complete effective ladder skips adoption even
@@ -194,10 +211,14 @@ candidate in each tier:
194
211
  `max`.
195
212
  - **Claude:** `haiku`, `sonnet`, `opus`, and `fable` across the ordered named
196
213
  tiers.
197
- - **Cursor:** 12 verified multi-family flat IDs across Composer, Claude, GPT,
198
- and Grok. The explicit materialization catalogue maps each flat ladder ID to
199
- a separate bracket-form frontmatter model; OAT does not derive or normalize
200
- either value.
214
+ - **Cursor:** verified multi-family flat IDs across Composer, Claude (Sonnet,
215
+ Opus, and Fable), GPT, and Grok. Two counts apply and they differ: the
216
+ bundled recommendation carries 16 Cursor candidates across the four tiers,
217
+ while the materialization catalogue carries 18 flat IDs. The extra entries
218
+ are approved mappings deliberately kept out of the recommendation but still
219
+ materializable. The catalogue maps each flat ladder ID to a separate
220
+ bracket-form frontmatter model; OAT does not derive or normalize either
221
+ value.
201
222
 
202
223
  The final candidate in a named tier defines that tier's reviewer ceiling. Lower
203
224
  reviewer selection requires a separate reviewed contract; a normal reviewer
@@ -305,6 +326,11 @@ separate evidence layers:
305
326
  - The materialized definition uses the mapping's explicit bracket-form
306
327
  frontmatter model.
307
328
  - Mapping-specific native-launch evidence authorizes the shipped mapping data.
329
+ An approved mapping may carry a probe record whose `submittedSelector` must
330
+ equal the mapping's `frontmatterModel` and whose `resolvedModel` must equal
331
+ its `ladderModelId`, so editing a mapping without re-probing fails its own
332
+ test rather than inheriting an approval it was never granted. See
333
+ [Verifying Cursor Pins](../../contributing/verifying-cursor-pins.md).
308
334
  - `oat doctor` checks current flat-ID catalogue availability, which can detect
309
335
  drift but cannot prove a definition pin.
310
336
  - The launcher records the selected variant and mapped model with `configured`
@@ -316,6 +342,26 @@ prevent a requested definition pin. Native variant acceptance is therefore not
316
342
  runtime-model verification, and skills must not promote self-report or
317
343
  catalogue presence into observed identity.
318
344
 
345
+ #### Unresolvable selectors also fall back silently
346
+
347
+ Entitlement is not the only trigger. Cursor does not reject a malformed pin
348
+ either; it substitutes a default for whichever selector component it cannot
349
+ resolve, with no error or warning:
350
+
351
+ - An unknown family falls back to the account default model. Probing
352
+ `claude-opus-9[effort=high]` resolved to `cursor-grok-4.5-high-fast`.
353
+ - An unknown effort falls back to that family's default rung. Probing
354
+ `claude-opus-5[effort=ultra]` resolved to `claude-opus-5-thinking-high`.
355
+
356
+ The default rung is family-specific and is not always `high` — Opus 4.7
357
+ defaults to `xhigh`. A typo in a pinned selector therefore ships a
358
+ working-but-wrong model that silently tracks a vendor-controlled default, so
359
+ capability can change with no corresponding change in the repository.
360
+
361
+ OAT does not currently validate effort rungs at sync time; that is tracked as
362
+ `BL-260726-validate-cursor-pin-effort`. Until it lands, the probe runbook is
363
+ the only guard.
364
+
319
365
  ## Phase and Optional-Worker Layers
320
366
 
321
367
  The phase implementer directly implements the phase tasks from one Phase Scope
@@ -1,6 +1,6 @@
1
1
  {
2
- "cli": "0.2.19",
3
- "docs-config": "0.2.19",
4
- "docs-theme": "0.2.19",
5
- "docs-transforms": "0.2.19"
2
+ "cli": "0.2.20",
3
+ "docs-config": "0.2.20",
4
+ "docs-theme": "0.2.20",
5
+ "docs-transforms": "0.2.20"
6
6
  }
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: subagent-orchestration
3
- version: 1.0.0
3
+ version: 1.0.1
4
4
  description: Use when delegating work to subagents or choosing a model for a task — routing by task class, selecting provider-specific models and effort, and verifying subagent claims. Covers OpenAI/Codex, Anthropic/Claude, and Cursor.
5
5
  compatibility: Self-contained; no OAT installation required.
6
6
  user-invocable: true
@@ -1,10 +1,10 @@
1
1
  # Evidence and Refresh Policy
2
2
 
3
3
  ```yaml
4
- guidance_version: 2026-07-21
5
- last_verified: 2026-07-21
6
- review_after: 2026-09-04
7
- stale_after: 2026-10-19
4
+ guidance_version: 2026-07-25
5
+ last_verified: 2026-07-25
6
+ review_after: 2026-09-08
7
+ stale_after: 2026-10-23
8
8
  ```
9
9
 
10
10
  This file governs dated provider examples. It does not override a live catalog,
@@ -17,8 +17,11 @@ authorization boundary.
17
17
  - `review-required`: older than 45 days, a newer family is observed, or a material control, price, tool, context, alias, or harness change is detected.
18
18
  - `stale`: older than 90 days, named models are unavailable or deprecated, or the launching surface can no longer express the recorded controls.
19
19
 
20
- Provider mappings in this package become review-required on 2026-09-04 and
21
- stale on 2026-10-19 unless re-verified earlier.
20
+ Provider mappings in this package become review-required on 2026-09-08 and
21
+ stale on 2026-10-23 unless re-verified earlier, or on the earlier date a
22
+ reference states in its own header. `provider-cursor.md` carries an earlier
23
+ 2026-09-04 review date because its catalog was not re-verified in the
24
+ 2026-07-25 refresh.
22
25
 
23
26
  ## Immediate Review Triggers
24
27
 
@@ -61,24 +64,61 @@ When evidence is incomplete, retain the incumbent or route one class up.
61
64
 
62
65
  Do not use aggregate leaderboard rank as a universal model order.
63
66
 
67
+ ## Comparable-Rung Analysis
68
+
69
+ For cross-provider analysis, show same-labelled effort rungs first when they
70
+ exist, then show empirically interesting practical substitutions as a separate
71
+ view. Label the model, effort, service tier, harness, benchmark version, and
72
+ retrieval date for every row.
73
+
74
+ Same-labelled rungs are comparable observations, not equivalent controls.
75
+ CursorBench, provider-native coding-agent harnesses, and common-agent harnesses
76
+ produce different Opus-to-Sol relationships. Preserve that disagreement; never
77
+ publish a provider-independent effort conversion.
78
+
64
79
  ## Current Evidence Summary
65
80
 
66
81
  - OpenAI positions GPT-5.6 Sol as frontier, Terra as balanced, and Luna as
67
82
  high-volume. Official guidance starts at medium generally, but independent
68
83
  coding-agent data shows meaningful Luna/Terra quality cliffs below high for
69
84
  repository work. The dated mechanical Codex floor is therefore Luna high.
70
- - Anthropic positions Sonnet 5 as the speed/intelligence workhorse, Opus 4.8 as
71
- a complex coding and enterprise model, and Fable 5 as the most capable widely
72
- released model. Their effort defaults and recommendations differ by model.
85
+ - Post-release Opus 5 evidence supports medium for normal substantive Claude
86
+ work and high for hard reasoning. Use xhigh for a reasoning-depth bottleneck
87
+ or evaluated long-horizon gain; max is selective and requires an effort
88
+ sweep. Consequence adds independent review rather than automatically raising
89
+ effort.
90
+ - Sonnet 5 is a conditional route when measured latency, throughput, access,
91
+ rate limits, or end-to-end workload economics win. Fable 5 is an
92
+ eligibility-gated specialist; its missing-domain-concept or long-horizon
93
+ reviewer instantiation remains provisional, and zero-data-retention
94
+ requirements exclude the current route.
73
95
  - CursorBench supports Composer 2.5 as an economical bounded coding worker.
74
96
  Cursor explicitly documents Composer fast as the same intelligence at a
75
97
  higher latency tier price.
98
+ - Mechanical reconnaissance remains harness-specific: Luna high in Codex,
99
+ Haiku 4.5 in Claude, and Composer 2.5 in Cursor. Direct API mini or nano
100
+ routes are limited to strict extraction, classification, or similarly
101
+ mechanically verified work.
76
102
  - OpenAI's long-context evidence disqualifies Luna for some very large-context
77
- work even when Luna's nominal context window accepts the input.
78
- - Anthropic documents benign false positives from Fable's stronger cyber
79
- classifier. This is operational evidence to consider when routing
80
- cyber-sensitive work, not a capability weakness or a reason to invert the
81
- general Opus-first policy.
103
+ work even when Luna's nominal context window accepts the input. Sol's current
104
+ direct-API requests above 272K input tokens carry a input and 1.5× output
105
+ price step.
106
+ - Sol's measured advantage is trajectory efficiency, not universal list-price
107
+ leadership. Opus output token pricing can be lower while Sol costs less per
108
+ completed task through fewer tokens, steps, turns, or recoveries.
109
+ - Two independent benchmarks show top-end Opus effort is non-monotonic:
110
+ Artificial Analysis Coding Agent Index (xhigh 67, max 66) and Frontier-Bench
111
+ (xhigh 44.4%, max 43.3%), both retrieved 2026-07-25. Frontier-Bench is
112
+ vendor-operated and its figure rests on a single self-reported row with no
113
+ independently verified counterpart, so weight it below the Coding Agent Index
114
+ rather than treating the two as equal evidence. CursorBench 3.2 is monotonic
115
+ over the same range, 69.3% to 70.0%; its contribution is economic, since
116
+ marginal cost per score point is roughly 3.75x worse above high than below
117
+ it. Together these support treating max as a route requiring justification,
118
+ not a strictly better rung.
119
+ - Cyber-sensitive and valid dual-use work should start on Opus 5 with
120
+ documented fallback handling where safeguards block the workflow. Opus 4.8
121
+ remains a compatibility fallback, not the universal primary.
82
122
  - Cursor disclosed that Cursor repository data entered Grok 4.5's training
83
123
  mixture. Its CursorBench ranking is evidence of competitiveness, not proof
84
124
  of superiority over nearby frontier models. Practical use supports Grok 4.5
@@ -86,8 +126,99 @@ Do not use aggregate leaderboard rank as a universal model order.
86
126
  implementation, and hard-reasoning economy routes, with cross-family review
87
127
  retained for consequential conclusions.
88
128
 
129
+ ## Speed and Wall-Clock Contract
130
+
131
+ Speed evidence must satisfy the multi-measure contract defined in
132
+ `model-selection-principles.md`, which is the single authority for which
133
+ measures are required and which are recorded when available. Do not restate a
134
+ divergent list here.
135
+
136
+ Every load-bearing quantitative claim requires claim-level provenance:
137
+
138
+ - a direct result locator, retrieval date, and stable source identity;
139
+ - exact model selector, effort, reasoning mode, service tier, harness, and
140
+ benchmark version;
141
+ - metric definition, aggregation basis, exclusions, and extraction method;
142
+ - absolute input values before any derived ratio or delta;
143
+ - the formula and units for each derived value;
144
+ - cache, fallback, refusal, retry, and completion treatment;
145
+ - unresolved source gaps stated explicitly rather than inferred away.
146
+
147
+ The Coding Agent Index time measure is average active agent wall time per task.
148
+ It excludes environment startup and verifier or judge time, and must not be
149
+ relabeled as total user-observed elapsed time. AA-Briefcase time remains
150
+ definition-unverified and must not be compared directly with it. Mutable
151
+ leaderboards require row-level locators; a homepage or aggregate rank is not a
152
+ sufficient source.
153
+
154
+ This file records reconciled conclusions and the values that drive a rule, not
155
+ the full ledger behind them. Every figure quoted here must name a publicly
156
+ resolvable source: the benchmark, its version, and the retrieval date, so a
157
+ reader outside the authoring environment can check it. Do not cite a private
158
+ research artifact as a locator — an internal packet may hold the working
159
+ arithmetic, but it cannot serve as provenance for published guidance.
160
+
161
+ ## Research Independence and Acceptance
162
+
163
+ A policy refresh requires:
164
+
165
+ 1. a neutral research brief and isolated independent author lanes;
166
+ 2. a source ledger with claim-level provenance for each lane;
167
+ 3. reciprocal or independent review that did not author the claim under review;
168
+ 4. fact reconciliation before routing synthesis;
169
+ 5. a cross-model synthesis that preserves contradictions and uncertainty;
170
+ 6. explicit human acceptance of the update scope;
171
+ 7. canonical-first application;
172
+ 8. downstream parity verification by read-back.
173
+
174
+ The author and reviewer may use the same evidence, but reviewer output must be
175
+ independently produced. Agreement is not proof. Record disagreements and their
176
+ resolution; do not collapse them into an unsupported consensus.
177
+
178
+ ## Live-Catalog and Downstream Gates
179
+
180
+ Before adding or changing a selector, verify the live launching surface,
181
+ resolved identity, effort or thinking control, service tier, and absence of
182
+ silent fallback. Direct-provider evidence can nominate a harness candidate but
183
+ cannot prove its alias.
184
+
185
+ A selector ships only after a live probe verifies those controls for that
186
+ specific mapping. Verification does not generalize: neither across rungs of one
187
+ family nor across families, because a harness may substitute a default for any
188
+ component it cannot resolve. Probe a non-default rung as well, or a result that
189
+ merely matches the family default cannot distinguish an honored parameter from
190
+ an ignored one.
191
+
192
+ The six Cursor `claude-opus-5` and `claude-opus-4-8` mappings shipped on
193
+ 2026-07-25 met this gate; their resolved identities are listed in
194
+ `provider-cursor.md`.
195
+
196
+ After canonical changes:
197
+
198
+ 1. inventory exact mirrors, wrappers, generated views, and application-specific
199
+ consumers before synchronization;
200
+ 2. use each consumer's supported sync or bundle mechanism;
201
+ 3. read back every consumer, including each machine where user-scope copies
202
+ exist;
203
+ 4. require byte-for-byte equality for mirrors and documented semantic parity
204
+ for intentional wrappers;
205
+ 5. search live prompts, rules, code, and configuration for copied historical
206
+ selectors or effort defaults;
207
+ 6. record intentional exceptions and fail the gate while unexplained
208
+ divergence remains.
209
+
89
210
  ## Reverification Record
90
211
 
212
+ One record describes one provider reference. When a single refresh cycle
213
+ updates several provider references, emit one record per provider inside a
214
+ `records:` list and lift the cycle-level fields — `verified_at`, `sources`,
215
+ `claim_provenance`, `independent_author`, `independent_reviewer`,
216
+ `incumbent_changes`, `reason`, `downstream_consumers`, `downstream_parity`, and
217
+ `unresolved_items` — to the parent so they are stated once rather than
218
+ duplicated per provider. Each record then carries only `provider`,
219
+ `harness_context`, `catalog_source`, `models_considered`, `controls_verified`,
220
+ and `eligibility_verified`.
221
+
91
222
  When updating a provider reference, record:
92
223
 
93
224
  ```yaml
@@ -97,7 +228,14 @@ harness_context: exact launching surface
97
228
  catalog_source: tool schema | CLI list | API catalog | UI snapshot
98
229
  models_considered: exact selectors
99
230
  controls_verified: effort, service tier, reasoning mode, context, tools
100
- sources: official docs and relevant benchmarks
231
+ eligibility_verified: retention, access, pricing thresholds, safeguards
232
+ sources: direct locators for official docs and relevant benchmark rows
233
+ claim_provenance: selector, effort, harness, metric, locator, retrieval date
234
+ independent_author: identity and model family
235
+ independent_reviewer: identity and model family
101
236
  incumbent_changes: additions, replacements, removals, or none
102
237
  reason: scheduled review or trigger
238
+ downstream_consumers: mirrors, wrappers, generated views, applications
239
+ downstream_parity: hashes or documented semantic comparison
240
+ unresolved_items: gaps, provisional routes, and deferred probes
103
241
  ```