@open-agent-toolkit/cli 0.2.19 → 0.2.20
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/assets/config/dispatch-matrix-recommendation.json +15 -5
- package/assets/docs/cli-utilities/configuration.md +10 -5
- package/assets/docs/contributing/index.md +1 -0
- package/assets/docs/contributing/verifying-cursor-pins.md +282 -0
- package/assets/docs/workflows/projects/dispatch-ceiling.md +50 -4
- package/assets/public-package-versions.json +4 -4
- package/assets/skills/subagent-orchestration/SKILL.md +1 -1
- package/assets/skills/subagent-orchestration/references/evidence-and-refresh.md +153 -15
- package/assets/skills/subagent-orchestration/references/model-selection-principles.md +75 -3
- package/assets/skills/subagent-orchestration/references/provider-claude.md +95 -35
- package/assets/skills/subagent-orchestration/references/provider-codex.md +36 -10
- package/assets/skills/subagent-orchestration/references/provider-cursor.md +91 -15
- package/dist/providers/cursor/codec/catalog.d.ts +19 -1
- package/dist/providers/cursor/codec/catalog.d.ts.map +1 -1
- package/dist/providers/cursor/codec/catalog.js +54 -0
- package/package.json +2 -2
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
{
|
|
2
|
-
"version": "2026-07-
|
|
2
|
+
"version": "2026-07-25.1",
|
|
3
3
|
"providers": {
|
|
4
4
|
"codex": {
|
|
5
5
|
"economy": {
|
|
@@ -102,23 +102,33 @@
|
|
|
102
102
|
"economy": {
|
|
103
103
|
"candidates": [
|
|
104
104
|
"composer-2.5",
|
|
105
|
-
"claude-sonnet-5-high",
|
|
106
105
|
"gpt-5.6-luna-high",
|
|
107
106
|
"gpt-5.6-luna-xhigh"
|
|
108
107
|
]
|
|
109
108
|
},
|
|
110
109
|
"balanced": {
|
|
111
|
-
"candidates": [
|
|
110
|
+
"candidates": [
|
|
111
|
+
"cursor-grok-4.5-high",
|
|
112
|
+
"gpt-5.6-terra-high",
|
|
113
|
+
"claude-opus-5-thinking-low"
|
|
114
|
+
]
|
|
112
115
|
},
|
|
113
116
|
"high": {
|
|
114
|
-
"candidates": [
|
|
117
|
+
"candidates": [
|
|
118
|
+
"gpt-5.6-sol-medium",
|
|
119
|
+
"gpt-5.6-sol-high",
|
|
120
|
+
"claude-opus-5-thinking-medium",
|
|
121
|
+
"claude-opus-5-thinking-high"
|
|
122
|
+
]
|
|
115
123
|
},
|
|
116
124
|
"frontier": {
|
|
117
125
|
"candidates": [
|
|
118
126
|
"claude-fable-5-thinking-high",
|
|
119
127
|
"claude-fable-5-thinking-xhigh",
|
|
120
128
|
"gpt-5.6-sol-xhigh",
|
|
121
|
-
"gpt-5.6-sol-max"
|
|
129
|
+
"gpt-5.6-sol-max",
|
|
130
|
+
"claude-opus-5-thinking-xhigh",
|
|
131
|
+
"claude-opus-5-thinking-max"
|
|
122
132
|
]
|
|
123
133
|
}
|
|
124
134
|
}
|
|
@@ -361,13 +361,18 @@ even when the reusable ladder is user-owned.
|
|
|
361
361
|
}
|
|
362
362
|
```
|
|
363
363
|
|
|
364
|
+
The example above is illustrative rather than a copy of the bundled ladder; its
|
|
365
|
+
tiers are trimmed for readability.
|
|
366
|
+
|
|
364
367
|
The bundled recommendation covers 13 Codex model/effort combinations: Luna and
|
|
365
368
|
Terra at `low`, `medium`, `high`, and `xhigh`, plus Sol at those efforts and
|
|
366
|
-
`max`. Claude covers `haiku`, `sonnet`, `opus`, and `fable`. Cursor covers
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
369
|
+
`max`. Claude covers `haiku`, `sonnet`, `opus`, and `fable`. Cursor covers 16
|
|
370
|
+
candidates across four tiers, drawn from a materialization catalogue of 18
|
|
371
|
+
verified multi-family flat IDs spanning Composer, Claude (Sonnet, Opus, and
|
|
372
|
+
Fable), GPT, and Grok; the two figures differ because some approved mappings
|
|
373
|
+
stay materializable without being recommended. An explicit mapping connects
|
|
374
|
+
each flat ladder ID to a separate bracket-form frontmatter model; configuration
|
|
375
|
+
and skills never derive or normalize either form.
|
|
371
376
|
|
|
372
377
|
The corresponding pinned Codex variant catalogue includes
|
|
373
378
|
`gpt-5.6-luna-high`, `gpt-5.6-terra-xhigh`, `gpt-5.6-sol-high`, and
|
|
@@ -17,3 +17,4 @@ Use this section when you are changing OAT itself rather than consuming it as a
|
|
|
17
17
|
- [Commit Conventions](commit-conventions.md) - Traceable commit patterns for project and docs work.
|
|
18
18
|
- [Hooks and Safety](hooks-and-safety.md) - Safety and mutation contracts for provider-sync behavior.
|
|
19
19
|
- [Smoke Testing](smoke-testing.md) - Runbook for the live workflow smoke runner: prerequisites, scenarios, evidence reports, and recovery.
|
|
20
|
+
- [Verifying Cursor Pins](verifying-cursor-pins.md) - Runbook for probe-verifying a Cursor model pin before shipping it.
|
|
@@ -0,0 +1,282 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Verifying Cursor Pins
|
|
3
|
+
description: 'Runbook for probe-verifying a Cursor model pin before shipping it: evidence channel, hook capture, subjects and controls, interpretation, and cleanup.'
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Verifying Cursor Pins
|
|
7
|
+
|
|
8
|
+
A Cursor model pin maps an OAT ladder model ID to the frontmatter selector that
|
|
9
|
+
materialized role files carry, for example `claude-opus-5-thinking-medium` to
|
|
10
|
+
`claude-opus-5[effort=medium]`. Adding one to
|
|
11
|
+
`packages/cli/src/providers/cursor/codec/catalog.ts` requires evidence that
|
|
12
|
+
Cursor actually resolves that selector to the intended model.
|
|
13
|
+
|
|
14
|
+
This runbook produces that evidence. Run it before adding any mapping.
|
|
15
|
+
|
|
16
|
+
## Why a probe is required
|
|
17
|
+
|
|
18
|
+
`DR-260718-explicit-cursor-pin-mapping` requires mapping-specific,
|
|
19
|
+
native-launch evidence before a mapping may be marked `approved`. Two things
|
|
20
|
+
that look like evidence are not:
|
|
21
|
+
|
|
22
|
+
- **Structural validation.** That a mapping is well-formed says nothing about
|
|
23
|
+
what Cursor does with it.
|
|
24
|
+
- **Agent self-report.** A subagent asked to name its own model is not a
|
|
25
|
+
reliable witness, and the probe agents in this runbook deliberately do not
|
|
26
|
+
ask.
|
|
27
|
+
|
|
28
|
+
The catalog alone is also insufficient wherever a family carries both a
|
|
29
|
+
thinking and a non-thinking flat ID at the same rung. Opus 5 carries both at
|
|
30
|
+
low, medium, and high, so the bracket selector is genuinely ambiguous there and
|
|
31
|
+
only a probe can resolve it. At rungs where only one variant exists, the
|
|
32
|
+
catalog settles the question without a probe — but you still need the probe to
|
|
33
|
+
prove the effort parameter was honored at all.
|
|
34
|
+
|
|
35
|
+
## Evidence channel
|
|
36
|
+
|
|
37
|
+
Cursor's agent lifecycle hooks report resolution from Cursor itself. Two
|
|
38
|
+
independent events agree:
|
|
39
|
+
|
|
40
|
+
- `subagentStart.subagent_model` — the model the subagent will use, and the
|
|
41
|
+
authoritative field per Cursor's hooks reference.
|
|
42
|
+
- `preToolUse.model` — observed from inside the subagent's own tool call.
|
|
43
|
+
|
|
44
|
+
Two channels that do **not** work:
|
|
45
|
+
|
|
46
|
+
- **The `cursor-agent` CLI and remote runtime.** Agent lifecycle hooks do not
|
|
47
|
+
fire there. A Task subagent launched from that runtime produces zero hook
|
|
48
|
+
events, and every row records as not-reported. Run probes from the Cursor
|
|
49
|
+
desktop app's Agent Chat.
|
|
50
|
+
- **The subagent card label.** It is built from the requested bracket
|
|
51
|
+
parameters and drops the thinking qualifier, so it cannot distinguish the
|
|
52
|
+
thinking and non-thinking variants — precisely the distinction a probe exists
|
|
53
|
+
to settle.
|
|
54
|
+
|
|
55
|
+
As of Cursor 3.12.30, `model_params` (which would carry an explicit `thinking`
|
|
56
|
+
flag) is empty and `model_id` is absent. Thinking is therefore established by
|
|
57
|
+
the resolved slug itself, which is sound because the non-thinking IDs are
|
|
58
|
+
distinct.
|
|
59
|
+
|
|
60
|
+
## Prerequisites
|
|
61
|
+
|
|
62
|
+
- Cursor desktop app, with the repository open as a workspace.
|
|
63
|
+
- The live catalog for the families you intend to pin:
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
cursor-agent models
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
Read the full output rather than truncating it. You need to see whether a
|
|
70
|
+
thinking and a non-thinking ID both exist at each rung, and which rung carries
|
|
71
|
+
the family's unqualified display label.
|
|
72
|
+
|
|
73
|
+
## Setting up capture
|
|
74
|
+
|
|
75
|
+
Both files are temporary. Delete them when the probe is done.
|
|
76
|
+
|
|
77
|
+
Create `.cursor/hooks/g01-capture.sh`:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
#!/bin/bash
|
|
81
|
+
# TEMPORARY pin-probe capture hook. Delete after probing.
|
|
82
|
+
set -u
|
|
83
|
+
|
|
84
|
+
OUT_DIR="/tmp/g01-probe"
|
|
85
|
+
OUT="$OUT_DIR/hooks.jsonl"
|
|
86
|
+
mkdir -p "$OUT_DIR" 2>/dev/null || :
|
|
87
|
+
|
|
88
|
+
input=$(cat)
|
|
89
|
+
ts=$(date -u '+%Y-%m-%dT%H:%M:%SZ')
|
|
90
|
+
|
|
91
|
+
printf '{"captured_at":"%s","payload":%s}\n' "$ts" "$input" >>"$OUT" 2>/dev/null || :
|
|
92
|
+
|
|
93
|
+
event=$(printf '%s' "$input" |
|
|
94
|
+
sed -n 's/.*"hook_event_name"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p')
|
|
95
|
+
|
|
96
|
+
# Observe only. Never block, never fail closed.
|
|
97
|
+
case "$event" in
|
|
98
|
+
subagentStop) printf '{}\n' ;;
|
|
99
|
+
*) printf '{"permission":"allow"}\n' ;;
|
|
100
|
+
esac
|
|
101
|
+
|
|
102
|
+
exit 0
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
Make it executable, then register it in `.cursor/hooks.json` for
|
|
106
|
+
`subagentStart`, `subagentStop`, and `preToolUse`:
|
|
107
|
+
|
|
108
|
+
```json
|
|
109
|
+
{
|
|
110
|
+
"version": 1,
|
|
111
|
+
"hooks": {
|
|
112
|
+
"subagentStart": [
|
|
113
|
+
{ "command": ".cursor/hooks/g01-capture.sh", "timeout": 10 }
|
|
114
|
+
],
|
|
115
|
+
"subagentStop": [
|
|
116
|
+
{ "command": ".cursor/hooks/g01-capture.sh", "timeout": 10 }
|
|
117
|
+
],
|
|
118
|
+
"preToolUse": [{ "command": ".cursor/hooks/g01-capture.sh", "timeout": 10 }]
|
|
119
|
+
}
|
|
120
|
+
}
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
The hook must observe only. A capture hook that can block or fail closed can
|
|
124
|
+
change the behavior it is supposed to measure.
|
|
125
|
+
|
|
126
|
+
Restart Cursor or reload the window so the hooks register.
|
|
127
|
+
|
|
128
|
+
## Writing probe agents
|
|
129
|
+
|
|
130
|
+
One throwaway agent per selector, under `.cursor/agents/`. Keep them trivial:
|
|
131
|
+
the probe is the hook payload, not anything the agent produces.
|
|
132
|
+
|
|
133
|
+
````markdown
|
|
134
|
+
---
|
|
135
|
+
# TEMPORARY pin probe - not oat-managed. Delete after probing.
|
|
136
|
+
# Class: subject - expect claude-opus-5-thinking-medium
|
|
137
|
+
name: zz-pin-probe-opus5-medium
|
|
138
|
+
description: Temporary pin probe (subject) for claude-opus-5[effort=medium]. Delete after use.
|
|
139
|
+
model: claude-opus-5[effort=medium]
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## Role
|
|
143
|
+
|
|
144
|
+
You are a throwaway pin probe. Run exactly one command and then stop:
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
echo PIN-PROBE zz-pin-probe-opus5-medium
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
Then reply with the single word `done`. Do not read files or use any
|
|
151
|
+
other tool.
|
|
152
|
+
|
|
153
|
+
Cursor's hook payload is the evidence for this probe. Your own claim about
|
|
154
|
+
your identity is not evidence and is deliberately not requested.
|
|
155
|
+
````
|
|
156
|
+
|
|
157
|
+
The `echo` exists to force a tool call, which is what produces the
|
|
158
|
+
corroborating `preToolUse` event.
|
|
159
|
+
|
|
160
|
+
### Subjects and controls
|
|
161
|
+
|
|
162
|
+
Probe every rung you intend to pin. Then add controls, which are what make the
|
|
163
|
+
subject results interpretable:
|
|
164
|
+
|
|
165
|
+
| Class | Example selector | Proves |
|
|
166
|
+
| ---------------- | ------------------------------ | ---------------------------------------------------------------- |
|
|
167
|
+
| Subject | `claude-opus-5[effort=medium]` | The mapping resolves as intended |
|
|
168
|
+
| Positive control | `claude-sonnet-5[effort=high]` | Reproduces an already-verified mapping |
|
|
169
|
+
| Negative control | `claude-opus-9[effort=high]` | Unknown family — the channel reports resolution, not the request |
|
|
170
|
+
| Negative control | `claude-opus-5[effort=ultra]` | Unknown effort — exposes rung-level fallback |
|
|
171
|
+
|
|
172
|
+
The negative controls carry most of the interpretive weight. Without them you
|
|
173
|
+
cannot distinguish a hook that reports what Cursor actually resolved from one
|
|
174
|
+
that merely echoes what you asked for. A bogus family that comes back as the
|
|
175
|
+
bogus family would invalidate the entire run.
|
|
176
|
+
|
|
177
|
+
## Running
|
|
178
|
+
|
|
179
|
+
Launch each probe agent from the Cursor desktop app's Agent Chat, as a subagent
|
|
180
|
+
with no model override, so Cursor resolves from the agent's frontmatter. Let
|
|
181
|
+
each finish before starting the next.
|
|
182
|
+
|
|
183
|
+
Then read the resolutions:
|
|
184
|
+
|
|
185
|
+
```bash
|
|
186
|
+
python3 - <<'PY'
|
|
187
|
+
import json
|
|
188
|
+
for line in open('/tmp/g01-probe/hooks.jsonl'):
|
|
189
|
+
p = json.loads(line).get('payload', {})
|
|
190
|
+
if p.get('hook_event_name') == 'subagentStart':
|
|
191
|
+
print(p.get('subagent_name'), '->', p.get('subagent_model'))
|
|
192
|
+
PY
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
## Interpreting results
|
|
196
|
+
|
|
197
|
+
### Unresolvable components fall back silently
|
|
198
|
+
|
|
199
|
+
Cursor does not reject a malformed pin. It substitutes a default for whichever
|
|
200
|
+
component it cannot resolve, with no error or warning:
|
|
201
|
+
|
|
202
|
+
- An unknown **family** falls back to the account default model. Probing
|
|
203
|
+
`claude-opus-9[effort=high]` resolved to `cursor-grok-4.5-high-fast`.
|
|
204
|
+
- An unknown **effort** falls back to that family's default rung. Probing
|
|
205
|
+
`claude-opus-5[effort=ultra]` resolved to `claude-opus-5-thinking-high`.
|
|
206
|
+
|
|
207
|
+
This makes a typo in a selector more dangerous than a plain downgrade: the pin
|
|
208
|
+
silently tracks whatever Cursor currently designates as the default, so a
|
|
209
|
+
vendor-side change alters capability with no change in this repository.
|
|
210
|
+
|
|
211
|
+
### The default rung is family-specific
|
|
212
|
+
|
|
213
|
+
Exactly one rung per family carries an unqualified display label in the live
|
|
214
|
+
catalog, and that marks the default. It is not always `high`:
|
|
215
|
+
|
|
216
|
+
| Family | Default rung | Unqualified label | Contrast |
|
|
217
|
+
| -------- | ------------ | ----------------- | ------------------------ |
|
|
218
|
+
| Opus 5 | high | `Opus 5 1M` | `Opus 5 1M Medium` |
|
|
219
|
+
| Opus 4.8 | high | `Opus 4.8 1M` | `Opus 4.8 1M Extra High` |
|
|
220
|
+
| Opus 4.7 | **xhigh** | `Opus 4.7 1M` | `Opus 4.7 1M High` |
|
|
221
|
+
|
|
222
|
+
Opus 4.7 is the disconfirming case for a naive "high is always the default"
|
|
223
|
+
reading.
|
|
224
|
+
|
|
225
|
+
### A default-rung result proves nothing on its own
|
|
226
|
+
|
|
227
|
+
If the rung you requested happens to be the family default, the observation is
|
|
228
|
+
equally consistent with the effort parameter being honored and with it being
|
|
229
|
+
ignored. That row cannot stand alone.
|
|
230
|
+
|
|
231
|
+
Always probe at least one non-default rung in the same family. A set of
|
|
232
|
+
non-default rungs that each resolve as requested is what demonstrates the
|
|
233
|
+
parameter is genuinely honored; a default-rung row is then sound by inference
|
|
234
|
+
across the set rather than from its own evidence.
|
|
235
|
+
|
|
236
|
+
## Recording the result
|
|
237
|
+
|
|
238
|
+
Add the mapping with a probe record, which ties it to the evidence that
|
|
239
|
+
approved it:
|
|
240
|
+
|
|
241
|
+
```ts
|
|
242
|
+
approvedMapping(
|
|
243
|
+
'claude-opus-5-thinking-medium',
|
|
244
|
+
'claude-opus-5[effort=medium]',
|
|
245
|
+
'claude-effort',
|
|
246
|
+
{ probeName: 'zz-pin-probe-opus5-medium', verifiedAt: '2026-07-25', evidencePath: '...' },
|
|
247
|
+
),
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
The record asserts that `submittedSelector` equals the mapping's
|
|
251
|
+
`frontmatterModel` and `resolvedModel` equals its `ladderModelId`. A mapping
|
|
252
|
+
edited later without re-probing therefore fails its own consistency test rather
|
|
253
|
+
than inheriting an approval that never covered the new selector.
|
|
254
|
+
|
|
255
|
+
Retain the raw payloads alongside the summary. They are the primary evidence;
|
|
256
|
+
the summary is an interpretation of them.
|
|
257
|
+
|
|
258
|
+
## Cleanup
|
|
259
|
+
|
|
260
|
+
Delete every temporary artifact:
|
|
261
|
+
|
|
262
|
+
- `.cursor/hooks.json` and `.cursor/hooks/g01-capture.sh`
|
|
263
|
+
- all `zz-pin-probe-*` agent files
|
|
264
|
+
- the capture file under `/tmp`
|
|
265
|
+
|
|
266
|
+
Before committing retained payloads, redact machine and identity fields.
|
|
267
|
+
Cursor hook payloads carry `user_email`, `workspace_roots`, and an absolute
|
|
268
|
+
`transcript_path`. None carries probe evidence — resolution is read from
|
|
269
|
+
`subagent_model` — so removing them costs nothing and keeps private data out of
|
|
270
|
+
the repository.
|
|
271
|
+
|
|
272
|
+
Confirm the workspace is clean:
|
|
273
|
+
|
|
274
|
+
```bash
|
|
275
|
+
oat sync --scope project
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
## Related
|
|
279
|
+
|
|
280
|
+
- [Smoke Testing](smoke-testing.md) - Runbook for the live workflow smoke runner.
|
|
281
|
+
- [Dispatch Policy](../workflows/projects/dispatch-ceiling.md) - How pins reach the recommendation and role files.
|
|
282
|
+
- [Provider Sync](../provider-sync/providers.md) - Provider-specific sync behavior.
|
|
@@ -64,6 +64,23 @@ existing cells. Planning shows the complete recommendation before asking which
|
|
|
64
64
|
scope should own it. If the resulting ladder is still missing or incomplete,
|
|
65
65
|
planning remains blocked rather than replacing the user's explicit values.
|
|
66
66
|
|
|
67
|
+
### Upgrading to a newer recommendation version
|
|
68
|
+
|
|
69
|
+
Preservation applies to whole cells, which has a consequence worth stating
|
|
70
|
+
plainly: when a new recommendation version adds candidates to a tier you have
|
|
71
|
+
already populated, re-running adoption will not give them to you. The existing
|
|
72
|
+
cell is kept intact rather than merged candidate by candidate. Removals are not
|
|
73
|
+
propagated either.
|
|
74
|
+
|
|
75
|
+
To pick up a new version, compare your
|
|
76
|
+
`workflow.dispatchCeiling.recommendationVersion` against the bundled version,
|
|
77
|
+
then either edit the affected cells by hand or clear them and re-adopt.
|
|
78
|
+
|
|
79
|
+
Version `2026-07-25.1` is a live example: it adds Opus 5 rungs to the Cursor
|
|
80
|
+
`balanced`, `high`, and `frontier` tiers and drops `claude-sonnet-5-high` from
|
|
81
|
+
`economy`. An adopter still on the prior version keeps their existing Cursor
|
|
82
|
+
tiers untouched until they take one of those actions.
|
|
83
|
+
|
|
67
84
|
Before offering adoption, planning runs `oat config list --json` once and treats
|
|
68
85
|
its output as the effective boundary across shared, repo-local, user, and
|
|
69
86
|
bundled-default precedence. A complete effective ladder skips adoption even
|
|
@@ -194,10 +211,14 @@ candidate in each tier:
|
|
|
194
211
|
`max`.
|
|
195
212
|
- **Claude:** `haiku`, `sonnet`, `opus`, and `fable` across the ordered named
|
|
196
213
|
tiers.
|
|
197
|
-
- **Cursor:**
|
|
198
|
-
and
|
|
199
|
-
|
|
200
|
-
|
|
214
|
+
- **Cursor:** verified multi-family flat IDs across Composer, Claude (Sonnet,
|
|
215
|
+
Opus, and Fable), GPT, and Grok. Two counts apply and they differ: the
|
|
216
|
+
bundled recommendation carries 16 Cursor candidates across the four tiers,
|
|
217
|
+
while the materialization catalogue carries 18 flat IDs. The extra entries
|
|
218
|
+
are approved mappings deliberately kept out of the recommendation but still
|
|
219
|
+
materializable. The catalogue maps each flat ladder ID to a separate
|
|
220
|
+
bracket-form frontmatter model; OAT does not derive or normalize either
|
|
221
|
+
value.
|
|
201
222
|
|
|
202
223
|
The final candidate in a named tier defines that tier's reviewer ceiling. Lower
|
|
203
224
|
reviewer selection requires a separate reviewed contract; a normal reviewer
|
|
@@ -305,6 +326,11 @@ separate evidence layers:
|
|
|
305
326
|
- The materialized definition uses the mapping's explicit bracket-form
|
|
306
327
|
frontmatter model.
|
|
307
328
|
- Mapping-specific native-launch evidence authorizes the shipped mapping data.
|
|
329
|
+
An approved mapping may carry a probe record whose `submittedSelector` must
|
|
330
|
+
equal the mapping's `frontmatterModel` and whose `resolvedModel` must equal
|
|
331
|
+
its `ladderModelId`, so editing a mapping without re-probing fails its own
|
|
332
|
+
test rather than inheriting an approval it was never granted. See
|
|
333
|
+
[Verifying Cursor Pins](../../contributing/verifying-cursor-pins.md).
|
|
308
334
|
- `oat doctor` checks current flat-ID catalogue availability, which can detect
|
|
309
335
|
drift but cannot prove a definition pin.
|
|
310
336
|
- The launcher records the selected variant and mapped model with `configured`
|
|
@@ -316,6 +342,26 @@ prevent a requested definition pin. Native variant acceptance is therefore not
|
|
|
316
342
|
runtime-model verification, and skills must not promote self-report or
|
|
317
343
|
catalogue presence into observed identity.
|
|
318
344
|
|
|
345
|
+
#### Unresolvable selectors also fall back silently
|
|
346
|
+
|
|
347
|
+
Entitlement is not the only trigger. Cursor does not reject a malformed pin
|
|
348
|
+
either; it substitutes a default for whichever selector component it cannot
|
|
349
|
+
resolve, with no error or warning:
|
|
350
|
+
|
|
351
|
+
- An unknown family falls back to the account default model. Probing
|
|
352
|
+
`claude-opus-9[effort=high]` resolved to `cursor-grok-4.5-high-fast`.
|
|
353
|
+
- An unknown effort falls back to that family's default rung. Probing
|
|
354
|
+
`claude-opus-5[effort=ultra]` resolved to `claude-opus-5-thinking-high`.
|
|
355
|
+
|
|
356
|
+
The default rung is family-specific and is not always `high` — Opus 4.7
|
|
357
|
+
defaults to `xhigh`. A typo in a pinned selector therefore ships a
|
|
358
|
+
working-but-wrong model that silently tracks a vendor-controlled default, so
|
|
359
|
+
capability can change with no corresponding change in the repository.
|
|
360
|
+
|
|
361
|
+
OAT does not currently validate effort rungs at sync time; that is tracked as
|
|
362
|
+
`BL-260726-validate-cursor-pin-effort`. Until it lands, the probe runbook is
|
|
363
|
+
the only guard.
|
|
364
|
+
|
|
319
365
|
## Phase and Optional-Worker Layers
|
|
320
366
|
|
|
321
367
|
The phase implementer directly implements the phase tasks from one Phase Scope
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: subagent-orchestration
|
|
3
|
-
version: 1.0.
|
|
3
|
+
version: 1.0.1
|
|
4
4
|
description: Use when delegating work to subagents or choosing a model for a task — routing by task class, selecting provider-specific models and effort, and verifying subagent claims. Covers OpenAI/Codex, Anthropic/Claude, and Cursor.
|
|
5
5
|
compatibility: Self-contained; no OAT installation required.
|
|
6
6
|
user-invocable: true
|
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# Evidence and Refresh Policy
|
|
2
2
|
|
|
3
3
|
```yaml
|
|
4
|
-
guidance_version: 2026-07-
|
|
5
|
-
last_verified: 2026-07-
|
|
6
|
-
review_after: 2026-09-
|
|
7
|
-
stale_after: 2026-10-
|
|
4
|
+
guidance_version: 2026-07-25
|
|
5
|
+
last_verified: 2026-07-25
|
|
6
|
+
review_after: 2026-09-08
|
|
7
|
+
stale_after: 2026-10-23
|
|
8
8
|
```
|
|
9
9
|
|
|
10
10
|
This file governs dated provider examples. It does not override a live catalog,
|
|
@@ -17,8 +17,11 @@ authorization boundary.
|
|
|
17
17
|
- `review-required`: older than 45 days, a newer family is observed, or a material control, price, tool, context, alias, or harness change is detected.
|
|
18
18
|
- `stale`: older than 90 days, named models are unavailable or deprecated, or the launching surface can no longer express the recorded controls.
|
|
19
19
|
|
|
20
|
-
Provider mappings in this package become review-required on 2026-09-
|
|
21
|
-
stale on 2026-10-
|
|
20
|
+
Provider mappings in this package become review-required on 2026-09-08 and
|
|
21
|
+
stale on 2026-10-23 unless re-verified earlier, or on the earlier date a
|
|
22
|
+
reference states in its own header. `provider-cursor.md` carries an earlier
|
|
23
|
+
2026-09-04 review date because its catalog was not re-verified in the
|
|
24
|
+
2026-07-25 refresh.
|
|
22
25
|
|
|
23
26
|
## Immediate Review Triggers
|
|
24
27
|
|
|
@@ -61,24 +64,61 @@ When evidence is incomplete, retain the incumbent or route one class up.
|
|
|
61
64
|
|
|
62
65
|
Do not use aggregate leaderboard rank as a universal model order.
|
|
63
66
|
|
|
67
|
+
## Comparable-Rung Analysis
|
|
68
|
+
|
|
69
|
+
For cross-provider analysis, show same-labelled effort rungs first when they
|
|
70
|
+
exist, then show empirically interesting practical substitutions as a separate
|
|
71
|
+
view. Label the model, effort, service tier, harness, benchmark version, and
|
|
72
|
+
retrieval date for every row.
|
|
73
|
+
|
|
74
|
+
Same-labelled rungs are comparable observations, not equivalent controls.
|
|
75
|
+
CursorBench, provider-native coding-agent harnesses, and common-agent harnesses
|
|
76
|
+
produce different Opus-to-Sol relationships. Preserve that disagreement; never
|
|
77
|
+
publish a provider-independent effort conversion.
|
|
78
|
+
|
|
64
79
|
## Current Evidence Summary
|
|
65
80
|
|
|
66
81
|
- OpenAI positions GPT-5.6 Sol as frontier, Terra as balanced, and Luna as
|
|
67
82
|
high-volume. Official guidance starts at medium generally, but independent
|
|
68
83
|
coding-agent data shows meaningful Luna/Terra quality cliffs below high for
|
|
69
84
|
repository work. The dated mechanical Codex floor is therefore Luna high.
|
|
70
|
-
-
|
|
71
|
-
|
|
72
|
-
|
|
85
|
+
- Post-release Opus 5 evidence supports medium for normal substantive Claude
|
|
86
|
+
work and high for hard reasoning. Use xhigh for a reasoning-depth bottleneck
|
|
87
|
+
or evaluated long-horizon gain; max is selective and requires an effort
|
|
88
|
+
sweep. Consequence adds independent review rather than automatically raising
|
|
89
|
+
effort.
|
|
90
|
+
- Sonnet 5 is a conditional route when measured latency, throughput, access,
|
|
91
|
+
rate limits, or end-to-end workload economics win. Fable 5 is an
|
|
92
|
+
eligibility-gated specialist; its missing-domain-concept or long-horizon
|
|
93
|
+
reviewer instantiation remains provisional, and zero-data-retention
|
|
94
|
+
requirements exclude the current route.
|
|
73
95
|
- CursorBench supports Composer 2.5 as an economical bounded coding worker.
|
|
74
96
|
Cursor explicitly documents Composer fast as the same intelligence at a
|
|
75
97
|
higher latency tier price.
|
|
98
|
+
- Mechanical reconnaissance remains harness-specific: Luna high in Codex,
|
|
99
|
+
Haiku 4.5 in Claude, and Composer 2.5 in Cursor. Direct API mini or nano
|
|
100
|
+
routes are limited to strict extraction, classification, or similarly
|
|
101
|
+
mechanically verified work.
|
|
76
102
|
- OpenAI's long-context evidence disqualifies Luna for some very large-context
|
|
77
|
-
work even when Luna's nominal context window accepts the input.
|
|
78
|
-
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
103
|
+
work even when Luna's nominal context window accepts the input. Sol's current
|
|
104
|
+
direct-API requests above 272K input tokens carry a 2× input and 1.5× output
|
|
105
|
+
price step.
|
|
106
|
+
- Sol's measured advantage is trajectory efficiency, not universal list-price
|
|
107
|
+
leadership. Opus output token pricing can be lower while Sol costs less per
|
|
108
|
+
completed task through fewer tokens, steps, turns, or recoveries.
|
|
109
|
+
- Two independent benchmarks show top-end Opus effort is non-monotonic:
|
|
110
|
+
Artificial Analysis Coding Agent Index (xhigh 67, max 66) and Frontier-Bench
|
|
111
|
+
(xhigh 44.4%, max 43.3%), both retrieved 2026-07-25. Frontier-Bench is
|
|
112
|
+
vendor-operated and its figure rests on a single self-reported row with no
|
|
113
|
+
independently verified counterpart, so weight it below the Coding Agent Index
|
|
114
|
+
rather than treating the two as equal evidence. CursorBench 3.2 is monotonic
|
|
115
|
+
over the same range, 69.3% to 70.0%; its contribution is economic, since
|
|
116
|
+
marginal cost per score point is roughly 3.75x worse above high than below
|
|
117
|
+
it. Together these support treating max as a route requiring justification,
|
|
118
|
+
not a strictly better rung.
|
|
119
|
+
- Cyber-sensitive and valid dual-use work should start on Opus 5 with
|
|
120
|
+
documented fallback handling where safeguards block the workflow. Opus 4.8
|
|
121
|
+
remains a compatibility fallback, not the universal primary.
|
|
82
122
|
- Cursor disclosed that Cursor repository data entered Grok 4.5's training
|
|
83
123
|
mixture. Its CursorBench ranking is evidence of competitiveness, not proof
|
|
84
124
|
of superiority over nearby frontier models. Practical use supports Grok 4.5
|
|
@@ -86,8 +126,99 @@ Do not use aggregate leaderboard rank as a universal model order.
|
|
|
86
126
|
implementation, and hard-reasoning economy routes, with cross-family review
|
|
87
127
|
retained for consequential conclusions.
|
|
88
128
|
|
|
129
|
+
## Speed and Wall-Clock Contract
|
|
130
|
+
|
|
131
|
+
Speed evidence must satisfy the multi-measure contract defined in
|
|
132
|
+
`model-selection-principles.md`, which is the single authority for which
|
|
133
|
+
measures are required and which are recorded when available. Do not restate a
|
|
134
|
+
divergent list here.
|
|
135
|
+
|
|
136
|
+
Every load-bearing quantitative claim requires claim-level provenance:
|
|
137
|
+
|
|
138
|
+
- a direct result locator, retrieval date, and stable source identity;
|
|
139
|
+
- exact model selector, effort, reasoning mode, service tier, harness, and
|
|
140
|
+
benchmark version;
|
|
141
|
+
- metric definition, aggregation basis, exclusions, and extraction method;
|
|
142
|
+
- absolute input values before any derived ratio or delta;
|
|
143
|
+
- the formula and units for each derived value;
|
|
144
|
+
- cache, fallback, refusal, retry, and completion treatment;
|
|
145
|
+
- unresolved source gaps stated explicitly rather than inferred away.
|
|
146
|
+
|
|
147
|
+
The Coding Agent Index time measure is average active agent wall time per task.
|
|
148
|
+
It excludes environment startup and verifier or judge time, and must not be
|
|
149
|
+
relabeled as total user-observed elapsed time. AA-Briefcase time remains
|
|
150
|
+
definition-unverified and must not be compared directly with it. Mutable
|
|
151
|
+
leaderboards require row-level locators; a homepage or aggregate rank is not a
|
|
152
|
+
sufficient source.
|
|
153
|
+
|
|
154
|
+
This file records reconciled conclusions and the values that drive a rule, not
|
|
155
|
+
the full ledger behind them. Every figure quoted here must name a publicly
|
|
156
|
+
resolvable source: the benchmark, its version, and the retrieval date, so a
|
|
157
|
+
reader outside the authoring environment can check it. Do not cite a private
|
|
158
|
+
research artifact as a locator — an internal packet may hold the working
|
|
159
|
+
arithmetic, but it cannot serve as provenance for published guidance.
|
|
160
|
+
|
|
161
|
+
## Research Independence and Acceptance
|
|
162
|
+
|
|
163
|
+
A policy refresh requires:
|
|
164
|
+
|
|
165
|
+
1. a neutral research brief and isolated independent author lanes;
|
|
166
|
+
2. a source ledger with claim-level provenance for each lane;
|
|
167
|
+
3. reciprocal or independent review that did not author the claim under review;
|
|
168
|
+
4. fact reconciliation before routing synthesis;
|
|
169
|
+
5. a cross-model synthesis that preserves contradictions and uncertainty;
|
|
170
|
+
6. explicit human acceptance of the update scope;
|
|
171
|
+
7. canonical-first application;
|
|
172
|
+
8. downstream parity verification by read-back.
|
|
173
|
+
|
|
174
|
+
The author and reviewer may use the same evidence, but reviewer output must be
|
|
175
|
+
independently produced. Agreement is not proof. Record disagreements and their
|
|
176
|
+
resolution; do not collapse them into an unsupported consensus.
|
|
177
|
+
|
|
178
|
+
## Live-Catalog and Downstream Gates
|
|
179
|
+
|
|
180
|
+
Before adding or changing a selector, verify the live launching surface,
|
|
181
|
+
resolved identity, effort or thinking control, service tier, and absence of
|
|
182
|
+
silent fallback. Direct-provider evidence can nominate a harness candidate but
|
|
183
|
+
cannot prove its alias.
|
|
184
|
+
|
|
185
|
+
A selector ships only after a live probe verifies those controls for that
|
|
186
|
+
specific mapping. Verification does not generalize: neither across rungs of one
|
|
187
|
+
family nor across families, because a harness may substitute a default for any
|
|
188
|
+
component it cannot resolve. Probe a non-default rung as well, or a result that
|
|
189
|
+
merely matches the family default cannot distinguish an honored parameter from
|
|
190
|
+
an ignored one.
|
|
191
|
+
|
|
192
|
+
The six Cursor `claude-opus-5` and `claude-opus-4-8` mappings shipped on
|
|
193
|
+
2026-07-25 met this gate; their resolved identities are listed in
|
|
194
|
+
`provider-cursor.md`.
|
|
195
|
+
|
|
196
|
+
After canonical changes:
|
|
197
|
+
|
|
198
|
+
1. inventory exact mirrors, wrappers, generated views, and application-specific
|
|
199
|
+
consumers before synchronization;
|
|
200
|
+
2. use each consumer's supported sync or bundle mechanism;
|
|
201
|
+
3. read back every consumer, including each machine where user-scope copies
|
|
202
|
+
exist;
|
|
203
|
+
4. require byte-for-byte equality for mirrors and documented semantic parity
|
|
204
|
+
for intentional wrappers;
|
|
205
|
+
5. search live prompts, rules, code, and configuration for copied historical
|
|
206
|
+
selectors or effort defaults;
|
|
207
|
+
6. record intentional exceptions and fail the gate while unexplained
|
|
208
|
+
divergence remains.
|
|
209
|
+
|
|
89
210
|
## Reverification Record
|
|
90
211
|
|
|
212
|
+
One record describes one provider reference. When a single refresh cycle
|
|
213
|
+
updates several provider references, emit one record per provider inside a
|
|
214
|
+
`records:` list and lift the cycle-level fields — `verified_at`, `sources`,
|
|
215
|
+
`claim_provenance`, `independent_author`, `independent_reviewer`,
|
|
216
|
+
`incumbent_changes`, `reason`, `downstream_consumers`, `downstream_parity`, and
|
|
217
|
+
`unresolved_items` — to the parent so they are stated once rather than
|
|
218
|
+
duplicated per provider. Each record then carries only `provider`,
|
|
219
|
+
`harness_context`, `catalog_source`, `models_considered`, `controls_verified`,
|
|
220
|
+
and `eligibility_verified`.
|
|
221
|
+
|
|
91
222
|
When updating a provider reference, record:
|
|
92
223
|
|
|
93
224
|
```yaml
|
|
@@ -97,7 +228,14 @@ harness_context: exact launching surface
|
|
|
97
228
|
catalog_source: tool schema | CLI list | API catalog | UI snapshot
|
|
98
229
|
models_considered: exact selectors
|
|
99
230
|
controls_verified: effort, service tier, reasoning mode, context, tools
|
|
100
|
-
|
|
231
|
+
eligibility_verified: retention, access, pricing thresholds, safeguards
|
|
232
|
+
sources: direct locators for official docs and relevant benchmark rows
|
|
233
|
+
claim_provenance: selector, effort, harness, metric, locator, retrieval date
|
|
234
|
+
independent_author: identity and model family
|
|
235
|
+
independent_reviewer: identity and model family
|
|
101
236
|
incumbent_changes: additions, replacements, removals, or none
|
|
102
237
|
reason: scheduled review or trigger
|
|
238
|
+
downstream_consumers: mirrors, wrappers, generated views, applications
|
|
239
|
+
downstream_parity: hashes or documented semantic comparison
|
|
240
|
+
unresolved_items: gaps, provisional routes, and deferred probes
|
|
103
241
|
```
|