opencode-harness-kit 0.2.0 → 0.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +301 -0
- package/GLOBAL.md +50 -13
- package/MIGRATION.md +140 -0
- package/README.md +276 -122
- package/bin/oc-harness.mjs +571 -135
- package/dist/index.js +453 -159
- package/docs/KNOWN-LIMITS.md +189 -4
- package/docs/RESEARCH.md +80 -5
- package/docs/ROUTING.md +254 -0
- package/lanes.json +136 -0
- package/lanes.schema.json +55 -0
- package/package.json +13 -11
- package/profiles/beam-free.json +51 -0
- package/profiles/big-pickle.json +45 -0
- package/profiles/deepseek-v4.1-flash-free.json +49 -0
- package/profiles/fleet-all.json +139 -0
- package/profiles/fleet-safe.json +139 -0
- package/profiles/glm-5.3-flash-free.json +49 -0
- package/profiles/index.json +120 -37
- package/profiles/longcat-2.5-preview-free.json +14 -10
- package/profiles/mimo-v2.6-flash-free.json +44 -0
- package/profiles/minimax-m3.1-flash-free.json +51 -0
- package/profiles/nemotron-3-ultra-free.json +44 -0
- package/profiles/nemotron-3.5-lightning-free.json +44 -0
- package/profiles/nemotron-3.5-super-vl-free.json +52 -0
- package/profiles/schema.json +30 -11
- package/{agents → prompts}/build.md +2 -35
- package/prompts/doc.md +42 -0
- package/{agents → prompts}/locate.md +1 -16
- package/prompts/longform.md +34 -0
- package/{agents → prompts}/plan.md +2 -25
- package/{agents → prompts}/reviewer.md +2 -22
- package/prompts/schema.md +30 -0
- package/prompts/sweep.md +29 -0
- package/{agents → prompts}/verify.md +6 -29
- package/prompts/vision.md +33 -0
- package/retention.yml +75 -57
- package/skills/space-build/SKILL.md +16 -0
- package/skills/space-finish/SKILL.md +0 -10
- package/skills/space-review/SKILL.md +8 -0
- package/src/config.js +94 -69
- package/src/driver/run.mjs +237 -0
- package/src/effort.js +59 -22
- package/src/index.ts +15 -2
- package/src/lane-usage.js +142 -0
- package/src/profile.js +615 -221
- package/src/router.ts +148 -39
- package/src/tasks.js +248 -0
- package/src/telemetry.ts +98 -0
- package/docs/demo/demo.gif +0 -0
- package/docs/demo/demo.mp4 +0 -0
- package/docs/demo/render.py +0 -146
- package/examples/muse-spark-1.3-contributor-free.json +0 -41
- package/profiles/deepseek-v4-flash-free.json +0 -41
- package/profiles/muse-split-free.json +0 -64
- package/profiles/space-bunny-free.json +0 -42
- package/profiles/x-preview-f-free.json +0 -41
package/docs/KNOWN-LIMITS.md
CHANGED
|
@@ -84,14 +84,193 @@ session's:
|
|
|
84
84
|
| `config` | Yes, but it runs once at startup. |
|
|
85
85
|
| `provider` | Can add or override models in the catalog, not the active session. |
|
|
86
86
|
|
|
87
|
-
So `router.ts` does
|
|
88
|
-
moves to the next
|
|
89
|
-
|
|
87
|
+
So `router.ts` does three things instead of the obvious fourth: at startup it
|
|
88
|
+
moves a lane to the next rung of its own chain, when the session's own model
|
|
89
|
+
is spent it moves to the next profile in the fallback queue, and when a
|
|
90
|
+
request fails it tells the agent plainly that a new session is the fix. It
|
|
91
|
+
does not pretend to retry.
|
|
90
92
|
|
|
91
93
|
The gap is real and tracked upstream. Provider issue #42977 asks Zen for a
|
|
92
94
|
`scope` field and a remaining-quota figure on 429 responses, precisely because
|
|
93
95
|
a client cannot currently tell a short quota window from a permanent block.
|
|
94
96
|
|
|
97
|
+
## A copied plugin loads and does nothing
|
|
98
|
+
|
|
99
|
+
The bundle finds `profiles/`, `prompts/`, `retention.yml` and `lanes.json` by
|
|
100
|
+
walking up from its own file. A symlink resolves to the real path, so a
|
|
101
|
+
symlinked plugin finds the kit's data. A **copy** does not: it looks beside
|
|
102
|
+
itself, finds nothing, and every profile lookup fails.
|
|
103
|
+
|
|
104
|
+
The failure is silent in the worst way. The plugin loads, its hooks are
|
|
105
|
+
registered, and nothing complains - but there is no guard, no pruning, no
|
|
106
|
+
lane failover and no telemetry. The session looks entirely normal and is
|
|
107
|
+
entirely unguarded. It was found by instrumenting the config hook, which
|
|
108
|
+
reported `no profile named "fleet-safe"` and nothing else.
|
|
109
|
+
|
|
110
|
+
`oc-harness install` symlinks the plugin rather than copying it, and
|
|
111
|
+
`oc-harness verify` now checks that the link still resolves to a bundle whose
|
|
112
|
+
data root is intact.
|
|
113
|
+
|
|
114
|
+
## An agent markdown file replaces the config entry, not the other way round
|
|
115
|
+
|
|
116
|
+
This is the one that decides where a model can live, and it was measured
|
|
117
|
+
against OpenCode 1.19 rather than read off the docs.
|
|
118
|
+
|
|
119
|
+
Two behaviours, both silent:
|
|
120
|
+
|
|
121
|
+
- An agent file whose name matches a config entry **replaces that entry
|
|
122
|
+
wholesale**. A file saying `model: opencode/beam-free` beat a config entry
|
|
123
|
+
saying `model: opencode/minimax-m3.1-flash-free`.
|
|
124
|
+
- An agent file **with no `model:` line** makes the agent inherit the model of
|
|
125
|
+
the agent that called it, not the model the config declares for it. A lane
|
|
126
|
+
configured for one model and defined by a file without a model ran on the
|
|
127
|
+
parent's model, and the config still looked correct.
|
|
128
|
+
|
|
129
|
+
The reproduction needs the plugin out of the way, because the kit's own
|
|
130
|
+
`config` hook re-pins the model:
|
|
131
|
+
|
|
132
|
+
```
|
|
133
|
+
$ opencode run --pure --agent build "..."
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
With `--pure`, the file's model wins. Without it, the plugin re-pins the model
|
|
137
|
+
to the lane's own - but the file's `prompt` and `description` still win
|
|
138
|
+
either way, so a shadowing file still takes the lane's instructions away.
|
|
139
|
+
|
|
140
|
+
So this kit keeps the model in the generated config and the prompt body in
|
|
141
|
+
`prompts/`, referenced with `{file:...}`. `oc-harness verify` fails when an
|
|
142
|
+
installed agent file shadows a lane, and `oc-harness install` removes the agent
|
|
143
|
+
files an earlier version of this kit left behind.
|
|
144
|
+
|
|
145
|
+
There are **four** directories a shadow can hide in, and opencode reads all
|
|
146
|
+
four: `agent/` and `agents/`, in the config dir and in `.opencode/`. When the
|
|
147
|
+
same name is in both `agent/` and `agents/`, the plural directory wins. A
|
|
148
|
+
checker that only looked at `agents/` missed half of them.
|
|
149
|
+
|
|
150
|
+
The reference also has to be absolute or start with `~`: a relative
|
|
151
|
+
`{file:./prompts/x.md}` is silently ignored, and a lane with no instructions
|
|
152
|
+
still runs.
|
|
153
|
+
|
|
154
|
+
## A request is one assistant message, and the counter used to count three
|
|
155
|
+
|
|
156
|
+
The lane counter decides which of the ten lanes earn their place, so it has to
|
|
157
|
+
count the way the work happens.
|
|
158
|
+
|
|
159
|
+
`opencode` emits `message.updated` several times for a single assistant
|
|
160
|
+
message: once when the message is created, once per streaming update, and
|
|
161
|
+
once when it completes. A counter that wrote a record per event reported three
|
|
162
|
+
requests for every request made. The record now carries the message id and the
|
|
163
|
+
reader merges on it, so one assistant message is one request.
|
|
164
|
+
|
|
165
|
+
The `chat.message` hook looked like the natural place to count a request
|
|
166
|
+
before the model answered, so that a call that never came back still counted.
|
|
167
|
+
It cannot: on OpenCode 1.19 its `agent` field is `undefined`, so it cannot
|
|
168
|
+
attribute a request to a lane at all. It is not used for this, because a
|
|
169
|
+
second counter that cannot be correlated with the first double-counts every
|
|
170
|
+
request.
|
|
171
|
+
|
|
172
|
+
The file is also found by **walking up** from the working directory. The plugin
|
|
173
|
+
writes it to the project root, and reading `process.cwd()` alone reported
|
|
174
|
+
"never used" for every lane whenever the command was run from a subdirectory.
|
|
175
|
+
|
|
176
|
+
## The free tier cannot read a PDF, an audio file or a video
|
|
177
|
+
|
|
178
|
+
Three capabilities the catalogue advertises are not reachable, and no lane
|
|
179
|
+
claims them:
|
|
180
|
+
|
|
181
|
+
- **PDF.** The provider's endpoint rejects the request before the model sees
|
|
182
|
+
it: `content part type "file" is not supported; only text and image_url
|
|
183
|
+
are`. A model asked about the PDF instead suggests running `pdftotext`.
|
|
184
|
+
- **Audio and video.** OpenCode 1.19 refuses to read a binary file at all:
|
|
185
|
+
`Cannot read binary file: ...m4a`. The attachment never reaches the model,
|
|
186
|
+
which then answers as if it had heard nothing.
|
|
187
|
+
|
|
188
|
+
So a document is extracted locally and the **text** reaches the `doc` lane.
|
|
189
|
+
MiMo publishes audio input and is in the fleet for its image reading, not its
|
|
190
|
+
audio.
|
|
191
|
+
|
|
192
|
+
## One model hangs on a file attachment, and only on an attachment
|
|
193
|
+
|
|
194
|
+
`nemotron-3.5-super-vl-free` is the catalogue's vision-language model, and a
|
|
195
|
+
live call with an image never returned: three attempts, 350 seconds, no
|
|
196
|
+
answer. It also hung on a plain `.txt` attached to the prompt.
|
|
197
|
+
|
|
198
|
+
It does **not** hang on a tool read. The `sweep` lane on Super VL answered a
|
|
199
|
+
`grep` sweep in 4.2 seconds average over three requests, with no trouble at
|
|
200
|
+
all. The rule is therefore narrower than "Super VL is broken": never hand it a
|
|
201
|
+
file as an attachment. On a plain prompt it is the fastest reasoning-off route
|
|
202
|
+
on the tier, and it is the only model that publishes a `none` effort level,
|
|
203
|
+
which makes "reasoning off" a job of its own.
|
|
204
|
+
|
|
205
|
+
The same call on MiniMax answered in 7 seconds and read the text out of the
|
|
206
|
+
image correctly.
|
|
207
|
+
|
|
208
|
+
## One shared reserve, sized by the narrowest lane
|
|
209
|
+
|
|
210
|
+
`compaction.reserved` is a single number for the whole config, and a fleet
|
|
211
|
+
mixes a 1M model with 200k models. Sizing it by the widest model reserves
|
|
212
|
+
125,829 tokens and leaves every 200k lane with 74,000 usable, which is where
|
|
213
|
+
the lane stops working. It is therefore sized by the **narrowest** lane:
|
|
214
|
+
|
|
215
|
+
| profile | narrowest lane | input cap | reserved | prune at |
|
|
216
|
+
|---|---|---|---|---|
|
|
217
|
+
| `fleet-safe` | DeepSeek V4.1 / GLM 5.3 | 200,000 | 24,000 | 176,000 |
|
|
218
|
+
| `fleet-all` | Big Pickle | 160,000 | 19,200 | 140,800 |
|
|
219
|
+
|
|
220
|
+
The cost is that a lane with a wide window gets a thinner buffer than it would
|
|
221
|
+
like. That is the right direction to be wrong in: a buffer that is too small
|
|
222
|
+
means compaction happens a little earlier, and a buffer that is too large
|
|
223
|
+
means a lane cannot work at all.
|
|
224
|
+
|
|
225
|
+
## Ten lanes is at the edge of what a selector can carry
|
|
226
|
+
|
|
227
|
+
The catalogue of eleven production harnesses in the source study
|
|
228
|
+
(arXiv:2609.00006) tops out at seven agent entries — OpenCode's own four
|
|
229
|
+
named agents plus three hidden utility ones. Claude Code ships three, Codex
|
|
230
|
+
two, OpenHands four, Mistral Vibe one delegable profile.
|
|
231
|
+
|
|
232
|
+
The guidance that decides the number is explicit: a flat catalogue forces the
|
|
233
|
+
planner to select one-of-N at every turn, and selection accuracy degrades as N
|
|
234
|
+
grows. The measured figures are for tool catalogues — 7-85% accuracy drops as
|
|
235
|
+
a catalogue expands, and Anthropic's own deferred tool search lifting Opus 4.5
|
|
236
|
+
from 79.5% to 88.1% selection accuracy by narrowing the choice to 3-5 options
|
|
237
|
+
— and no corpus system measures agent count the same way.
|
|
238
|
+
|
|
239
|
+
Ten lanes sits at the boundary rather than past it, and the reason it is
|
|
240
|
+
tolerable is that the descriptions do not overlap: each lane names a different
|
|
241
|
+
job, and the failure mode of a large catalogue is similar descriptions, not
|
|
242
|
+
many descriptions. `oc-harness lanes` counts the requests so the fleet can be
|
|
243
|
+
cut to what earns its place, rather than defended on principle.
|
|
244
|
+
|
|
245
|
+
## Measuring a lane takes a delegation, not a flag
|
|
246
|
+
|
|
247
|
+
`opencode run --agent <lane>` only runs a **primary** agent. Handed a
|
|
248
|
+
subagent lane, it silently falls back to the default agent, and the run
|
|
249
|
+
succeeds - so the measurement appears to work and says nothing.
|
|
250
|
+
|
|
251
|
+
A subagent lane is exercised by delegating to it from a primary one:
|
|
252
|
+
|
|
253
|
+
```
|
|
254
|
+
opencode run --agent build "Use the task tool with subagent_type=reviewer to ..."
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
The `plan` and `build` lanes are primary, so they are run directly with
|
|
258
|
+
`--agent`. The other eight are delegated to.
|
|
259
|
+
|
|
260
|
+
## A model that inherits the caller's model is not a model at all
|
|
261
|
+
|
|
262
|
+
A lane whose model is spent should move down its own chain, and a fleet has
|
|
263
|
+
ten models, so one dead route must not take the other nine down with it.
|
|
264
|
+
|
|
265
|
+
The first version of the router replaced the **whole profile** when the
|
|
266
|
+
session's own model was spent, which set every lane of the fallback profile
|
|
267
|
+
onto one model and left the other five lanes pointing at routes that were also
|
|
268
|
+
spent. The docstring on that function promised the opposite. A fleet now keeps
|
|
269
|
+
its lanes: the session model moves, each lane moves down its own chain, and
|
|
270
|
+
the router publishes the model it chose so the effort hook pins the session to
|
|
271
|
+
that one rather than back to the base model that is spent.
|
|
272
|
+
|
|
273
|
+
|
|
95
274
|
## Retry on 429 is deliberately not attempted
|
|
96
275
|
|
|
97
276
|
Zen free routes return `429 FreeUsageLimitError`, and #42977 documents models
|
|
@@ -113,9 +292,15 @@ From one install's 20-day history, free model slots turned over in days:
|
|
|
113
292
|
| `nemotron-3-ultra-free` | 3 days |
|
|
114
293
|
| `muse-spark-1.3-contributor-free` | still running at 20 days |
|
|
115
294
|
|
|
295
|
+
The tier turned over again since: four of this kit's five original profiles
|
|
296
|
+
(`space-bunny-free`, `x-preview-f-free`, `deepseek-v4-flash-free`,
|
|
297
|
+
`muse-spark-1.3-contributor-free`) were all withdrawn within a month of
|
|
298
|
+
shipping, and the default profile pointed at a model that no longer existed.
|
|
299
|
+
|
|
116
300
|
Zen states free models are available "for a limited time" with no date. Treat
|
|
117
301
|
profiles as perishable: run `oc-harness doctor`, which re-checks each profile
|
|
118
|
-
against live metadata and reports drift
|
|
302
|
+
against live metadata and reports drift, and which also reports a lane whose
|
|
303
|
+
whole chain has left the catalogue.
|
|
119
304
|
|
|
120
305
|
## A model-driven compressor is not assumed to work
|
|
121
306
|
|
package/docs/RESEARCH.md
CHANGED
|
@@ -146,9 +146,10 @@ unfalsified. `eval` now prints the figure as `PROJECTION` with `measured: no`,
|
|
|
146
146
|
and `baseline` reports per-model-per-variant medians so the claim becomes real
|
|
147
147
|
once the tiering has actually been running.
|
|
148
148
|
|
|
149
|
-
## Why
|
|
149
|
+
## Why the default is a fleet
|
|
150
150
|
|
|
151
|
-
Scanning free models on OpenCode Zen for context, effort, and privacy
|
|
151
|
+
Scanning free models on OpenCode Zen for context, effort, and privacy
|
|
152
|
+
together, in the tier of 2026-10-07:
|
|
152
153
|
|
|
153
154
|
| model | context | effort levels | retention |
|
|
154
155
|
|---|---|---|---|
|
|
@@ -171,6 +172,27 @@ coverage and no public identity. Zero-retention and a `max` effort tier were
|
|
|
171
172
|
judged worth those risks; that judgement is a default, not a fact, and
|
|
172
173
|
`oc-harness use` makes reversing it one flag.
|
|
173
174
|
|
|
175
|
+
**That default did not survive the month.** Four of those five models were
|
|
176
|
+
withdrawn, including the one the default profile named. The tier of
|
|
177
|
+
2026-10-19 has ten free models with ten different strengths, and no single
|
|
178
|
+
one of them is best at everything: on the first pass of measurements Beam
|
|
179
|
+
answered in 4-6 seconds, GLM took 16-27 and was the careful one, and Nemotron
|
|
180
|
+
Super VL was the only model publishing a `none` effort level. The default
|
|
181
|
+
became a **fleet** — ten jobs, ten models, one session — with a privacy
|
|
182
|
+
ceiling rather than a single model. The reasoning and the measurements are in
|
|
183
|
+
[`ROUTING.md`](ROUTING.md); the source study of the eleven production
|
|
184
|
+
harnesses is in the section above.
|
|
185
|
+
|
|
186
|
+
**The first fleet's numbers were wrong, and measuring it again is what found
|
|
187
|
+
out.** Re-measured on 2026-10-20, lane by lane, over live calls: GLM is
|
|
188
|
+
4.4-5.8 seconds and not 16-27 (the old figure came from a much larger
|
|
189
|
+
prompt); MiMo reads an image in 19.2 seconds and not 8; and Nemotron Super VL
|
|
190
|
+
does **not** hang on a tool read, only on a file attached to the prompt, which
|
|
191
|
+
means the sweep lane is sound. The measurement also showed the safe fleet
|
|
192
|
+
giving up more than it needed to: two lanes were on a model the ceiling had
|
|
193
|
+
forced them onto rather than the best one available for that job. The numbers,
|
|
194
|
+
and what changed because of them, are in [`ROUTING.md`](ROUTING.md).
|
|
195
|
+
|
|
174
196
|
## How the privacy list was assembled
|
|
175
197
|
|
|
176
198
|
From the Privacy section of the OpenCode Zen docs, which states a general
|
|
@@ -180,6 +202,17 @@ not named in the exception list and so fall under the general policy. Those two
|
|
|
180
202
|
are marked in `retention.yml` as inferred rather than quoted, because the
|
|
181
203
|
distinction matters if you are handling someone else's code.
|
|
182
204
|
|
|
205
|
+
Both of those models have since been withdrawn, and `retention.yml` was
|
|
206
|
+
rewritten against the page on 2026-10-19 to match the tier that exists now:
|
|
207
|
+
seven new entries, four dead models dropped, and every value quoted rather
|
|
208
|
+
than inferred.
|
|
209
|
+
|
|
210
|
+
The tier of 2026-10-19 needed a third value. `beam-free` does not train on
|
|
211
|
+
prompts but "retains them for 30 days for safety reasons", which is neither
|
|
212
|
+
"keeps nothing" nor "may train". Calling that zero-retention would have been
|
|
213
|
+
the same kind of claim the file exists to prevent, so `retained` was added and
|
|
214
|
+
the fleet's ceiling can be set to either side of it.
|
|
215
|
+
|
|
183
216
|
## What the ecosystem already covers
|
|
184
217
|
|
|
185
218
|
Checked before writing any of the above, so the kit composes instead of
|
|
@@ -199,12 +232,54 @@ duplicating:
|
|
|
199
232
|
The gap is the seam between a provider catalog and a config file. That is what
|
|
200
233
|
this kit is for.
|
|
201
234
|
|
|
235
|
+
## How many lanes a fleet can carry
|
|
236
|
+
|
|
237
|
+
The fleet question was not answered by taste. Six sources, and what each one
|
|
238
|
+
says:
|
|
239
|
+
|
|
240
|
+
| source | what it says |
|
|
241
|
+
|---|---|
|
|
242
|
+
| arXiv:2609.00006, eleven production harnesses | The largest agent catalogue in the corpus is OpenCode's own, four named agents plus three hidden utility ones. Claude Code ships three, Codex two, OpenHands four, Mistral Vibe one delegable profile. **No harness in the corpus ships ten.** |
|
|
243
|
+
| The same paper, Recommendation 12 | "Stay single-agent until you can point to a concrete breadth-first exploration phase where parallel context isolation clearly beats serial search." The evidence cited: multi-agent systems use ~15x more tokens than a chat baseline, and "most coding tasks involve fewer truly parallelizable tasks than research". |
|
|
244
|
+
| The same paper, §7.2 | Per-agent specialization is "keyed on agent profile rather than on model generation" — the corpus keeps the agent catalogue small and expresses model differences as a *field*, not as more agents. |
|
|
245
|
+
| LangChain's multi-agent guide | The explicit threshold: a system prompt enumerating the agents is right for "small, static agent lists (< 10 agents)"; above ten, agent discovery has to become a tool. |
|
|
246
|
+
| `agentpatterns.ai`, on tool catalogues | Selection accuracy degrades as the catalogue grows: 7-85% drops as a tool catalogue expands, and Anthropic's own deferred tool search lifts Opus 4.5 from 79.5% to 88.1% selection accuracy by narrowing the choice to 3-5 options. |
|
|
247
|
+
| `code-yeongyu/oh-my-openagent` (70k★, an OpenCode plugin) | The real counter-example, and the closest precedent: **four curated agents and ten categories**, each with its own ordered model chain, where the main agent picks a category and never a model name. It works because the categories are jobs, not models. |
|
|
248
|
+
|
|
249
|
+
What they add up to: ten lanes sits at the edge rather than past it, and the
|
|
250
|
+
reason it is tolerable is that the descriptions do not overlap. The failure
|
|
251
|
+
mode of a large catalogue is similar descriptions, not many descriptions.
|
|
252
|
+
|
|
253
|
+
So the fleet is built the way `omo` builds it — a job per lane, a chain per
|
|
254
|
+
lane — and the number is treated as a hypothesis rather than a constant.
|
|
255
|
+
`oc-harness lanes` counts the requests, and a lane nobody uses is cut.
|
|
256
|
+
|
|
257
|
+
The other thing the sources settled is where the model lives. OpenCode's Task
|
|
258
|
+
tool takes a free-string `subagent_type` and no model parameter, so the only
|
|
259
|
+
way to put a different model behind a delegation is to define an agent with
|
|
260
|
+
that model. `agent.<lane>` in the generated config does that, and it is also
|
|
261
|
+
the only place the model can live without an agent markdown file shadowing it
|
|
262
|
+
and quietly replacing it.
|
|
263
|
+
|
|
202
264
|
## Sources
|
|
203
265
|
|
|
204
266
|
- arXiv:2609.00006 — Harness Engineering, eleven-harness source study
|
|
205
267
|
- `https://opencode.ai/docs/zen/` — pricing, retention, free-tier terms
|
|
206
|
-
- `https://opencode.ai/config.json` — the resolved config schema
|
|
207
|
-
- `@opencode-ai/plugin
|
|
208
|
-
|
|
268
|
+
- `https://opencode.ai/config.json` — the resolved config schema
|
|
269
|
+
- `@opencode-ai/plugin` `dist/index.d.ts` — the hook surface (the version
|
|
270
|
+
actually installed; re-check against it after any plugin bump)
|
|
209
271
|
- `https://models.dev/api.json` — provider and model metadata
|
|
210
272
|
- `anomalyco/opencode` issue #42977 — free-route 429 behaviour
|
|
273
|
+
- `https://docs.langchain.com/oss/python/langchain/multi-agent/subagents` —
|
|
274
|
+
the agent-catalogue threshold
|
|
275
|
+
- `https://github.com/1lucascb/agentpatterns.ai` — selection accuracy against
|
|
276
|
+
catalogue size
|
|
277
|
+
- `https://github.com/code-yeongyu/oh-my-openagent` — a fleet of categories
|
|
278
|
+
with per-category model chains, in OpenCode
|
|
279
|
+
- OpenCode 1.19.0, measured: agent-file precedence, the four directories it
|
|
280
|
+
reads agents from, `{file:...}` resolution, binary-file refusal, the
|
|
281
|
+
`chat.message` hook's missing `agent` field, how many times
|
|
282
|
+
`message.updated` fires per message, and a two-model delegation inside one
|
|
283
|
+
session
|
|
284
|
+
- The live free tier, measured 2026-10-20: one request per lane on both
|
|
285
|
+
fleets, ten lanes on each, through real delegations
|
package/docs/ROUTING.md
ADDED
|
@@ -0,0 +1,254 @@
|
|
|
1
|
+
# Routing: ten jobs, ten free models
|
|
2
|
+
|
|
3
|
+
The fleet is the point of this kit. One session, ten lanes, ten models, and a
|
|
4
|
+
human who never switches a model by hand.
|
|
5
|
+
|
|
6
|
+
This page records what the lanes are, which model serves each one, and the
|
|
7
|
+
measurements behind those choices. Everything in the tables below was measured
|
|
8
|
+
against the live tier on **2026-10-20** with OpenCode **1.19.0**, and is
|
|
9
|
+
reproducible with `oc-harness doctor` and `oc-harness lanes`.
|
|
10
|
+
|
|
11
|
+
## The lanes
|
|
12
|
+
|
|
13
|
+
A **lane** is a job. `lanes.json` says what each job is for and nothing about
|
|
14
|
+
models. A profile says which lanes it enables and which model serves each one.
|
|
15
|
+
|
|
16
|
+
| lane | job | model in `fleet-all` | model in `fleet-safe` |
|
|
17
|
+
|---|---|---|---|
|
|
18
|
+
| `build` | write and verify the change | minimax-m3.1-flash-free | minimax-m3.1-flash-free |
|
|
19
|
+
| `plan` | turn a request into a plan | beam-free | deepseek-v4.1-flash-free |
|
|
20
|
+
| `reviewer` | read a change in two passes | nemotron-3-ultra-free | glm-5.3-flash-free |
|
|
21
|
+
| `locate` | find where something lives | nemotron-3.5-lightning-free | deepseek-v4.1-flash-free |
|
|
22
|
+
| `verify` | run the project's checks | big-pickle | deepseek-v4.1-flash-free |
|
|
23
|
+
| `vision` | read an image or a screenshot | mimo-v2.6-flash-free | minimax-m3.1-flash-free |
|
|
24
|
+
| `doc` | read a long document or a spec | glm-5.3-flash-free | glm-5.3-flash-free |
|
|
25
|
+
| `schema` | produce JSON or a JSON schema | deepseek-v4.1-flash-free | deepseek-v4.1-flash-free |
|
|
26
|
+
| `sweep` | sweep a lot of code mechanically | nemotron-3.5-super-vl-free | longcat-2.5-preview-free |
|
|
27
|
+
| `longform` | read a megabyte, or write alternatives | longcat-2.5-preview-free | longcat-2.5-preview-free |
|
|
28
|
+
|
|
29
|
+
`fleet-all` gives every lane the model built for it: ten jobs, ten models,
|
|
30
|
+
one to one. `fleet-safe` is the same fleet under a zero-retention ceiling.
|
|
31
|
+
|
|
32
|
+
**Six of the ten lanes run on a second-choice model under that ceiling**:
|
|
33
|
+
plan, reviewer, locate, verify, vision and sweep. That is the price of the
|
|
34
|
+
ceiling, and it is worth paying only when the model that takes over is the
|
|
35
|
+
best one available *for that job*, which is why the two fleets order their
|
|
36
|
+
chains differently (see below).
|
|
37
|
+
|
|
38
|
+
## Measured, per lane
|
|
39
|
+
|
|
40
|
+
Every number here is a live call, not a catalog reading. A **request** is one
|
|
41
|
+
assistant message, which is one round trip to the provider; a lane that makes
|
|
42
|
+
three tool calls makes three requests. `avg ms` covers the requests that
|
|
43
|
+
finished.
|
|
44
|
+
|
|
45
|
+
`fleet-all`, 2026-10-20:
|
|
46
|
+
|
|
47
|
+
| lane | model | requests | avg ms | max ms |
|
|
48
|
+
|---|---|---|---|---|
|
|
49
|
+
| `build` | minimax-m3.1-flash-free | 21 | 10104 | 34478 |
|
|
50
|
+
| `plan` | beam-free | 7 | 2624 | 4186 |
|
|
51
|
+
| `reviewer` | nemotron-3-ultra-free | 2 | 4716 | 5376 |
|
|
52
|
+
| `locate` | nemotron-3.5-lightning-free | 4 | 3680 | 5523 |
|
|
53
|
+
| `verify` | big-pickle | 2 | 6546 | 7964 |
|
|
54
|
+
| `vision` | mimo-v2.6-flash-free | 2 | 19167 | 23813 |
|
|
55
|
+
| `doc` | glm-5.3-flash-free | 2 | 5421 | 7620 |
|
|
56
|
+
| `schema` | deepseek-v4.1-flash-free | 1 | 3390 | 3390 |
|
|
57
|
+
| `sweep` | nemotron-3.5-super-vl-free | 3 | 4221 | 4951 |
|
|
58
|
+
| `longform` | longcat-2.5-preview-free | 2 | 9382 | 11572 |
|
|
59
|
+
|
|
60
|
+
`fleet-safe`, 2026-10-20:
|
|
61
|
+
|
|
62
|
+
| lane | model | requests | avg ms | max ms |
|
|
63
|
+
|---|---|---|---|---|
|
|
64
|
+
| `build` | minimax-m3.1-flash-free | 42 | 8210 | 42459 |
|
|
65
|
+
| `plan` | deepseek-v4.1-flash-free | 6 | 5106 | 7057 |
|
|
66
|
+
| `reviewer` | glm-5.3-flash-free | 2 | 5757 | 7982 |
|
|
67
|
+
| `locate` | deepseek-v4.1-flash-free | 5 | 3612 | 4380 |
|
|
68
|
+
| `verify` | deepseek-v4.1-flash-free | 2 | 3376 | 3997 |
|
|
69
|
+
| `vision` | minimax-m3.1-flash-free | 2 | 7735 | 12190 |
|
|
70
|
+
| `doc` | glm-5.3-flash-free | 7 | 4372 | 11921 |
|
|
71
|
+
| `schema` | deepseek-v4.1-flash-free | 1 | 5195 | 5195 |
|
|
72
|
+
| `sweep` | longcat-2.5-preview-free | 2 | 7291 | 7477 |
|
|
73
|
+
| `longform` | longcat-2.5-preview-free | 2 | 13177 | 13709 |
|
|
74
|
+
|
|
75
|
+
The `build` rows are not comparable with the others: they include the turns
|
|
76
|
+
that did the delegating, so they count the work of the other nine lanes. A
|
|
77
|
+
one-line `build` reply on its own measured **3130 ms** and **21137 ms** on a
|
|
78
|
+
cold start.
|
|
79
|
+
|
|
80
|
+
**What the safe ceiling costs, measured rather than assumed.** Planning is the
|
|
81
|
+
lane that pays: 2624 ms on Beam against 5106 ms on DeepSeek, about twice the
|
|
82
|
+
wait. Reviewing barely pays at all: 4716 ms on Nemotron 3 Ultra against 5757
|
|
83
|
+
ms on GLM. Locating and verifying get *faster*, because DeepSeek publishes a
|
|
84
|
+
`low` effort level and Big Pickle publishes none at all.
|
|
85
|
+
|
|
86
|
+
And one lane gets faster rather than slower. MiMo reads an image in 19.2 s;
|
|
87
|
+
MiniMax reads the same image in 7.7 s. The unrestricted fleet put `vision` on
|
|
88
|
+
the slower model because MiMo is the tier's dedicated image reader, and under
|
|
89
|
+
the ceiling `vision` moves to MiniMax - which turns out to be the better
|
|
90
|
+
choice anyway.
|
|
91
|
+
|
|
92
|
+
## What each model is for
|
|
93
|
+
|
|
94
|
+
| model | measured | why it owns that lane |
|
|
95
|
+
|---|---|---|
|
|
96
|
+
| minimax-m3.1-flash-free | 3.1 s for a one-line reply, 512k output cap, reads an image in 7.7 s | The widest output cap on the tier and a stable tool call, so the lane that writes code fits, and it is also the fastest measured image reader. |
|
|
97
|
+
| beam-free | **2.6 s average over 7 requests, the fastest on the tier**, 1M context, structured output | Planning is a reading-and-reasoning job with a long context and a structured answer. Nothing else on the tier does it faster. |
|
|
98
|
+
| nemotron-3-ultra-free | 4.7 s average, but **34 s on a cold first request** | The largest open MoE available, which is what a two-pass review needs. The cold start is worth knowing about before blaming the route. |
|
|
99
|
+
| nemotron-3.5-lightning-free | 3.7 s average over 4 requests, 5.5 s at worst, no selectable effort | No effort level, so it spends nothing on reasoning, and it can emit a file larger than 128k tokens. |
|
|
100
|
+
| big-pickle | 6.5 s average over 2 requests, structured output, no effort levels, no images | Stable, and it answers in a shape a program can read. |
|
|
101
|
+
| mimo-v2.6-flash-free | **19.2 s average with an image**, 23.8 s at worst - the slowest lane on the tier | It reads images correctly, which is the job. It publishes audio input too, and that path is unreachable (see below). |
|
|
102
|
+
| glm-5.3-flash-free | 4.4-5.8 s, structured output, 3 effort levels | Careful and quick enough. A previous measurement put it at 16-27 s on a larger prompt; on the work a lane actually does it is five seconds. |
|
|
103
|
+
| deepseek-v4.1-flash-free | 3.4-5.2 s, structured output, reasoning + agentic coding | The natural lane for JSON a program has to parse, and the best all-rounder inside a zero-retention ceiling. |
|
|
104
|
+
| nemotron-3.5-super-vl-free | 4.2 s average over 3 requests, **the only model publishing a `none` effort level** | A lane that turns reasoning off entirely is a job of its own, and no other model can do it. |
|
|
105
|
+
| longcat-2.5-preview-free | 9.4-13.2 s, 1M context, reads images, temperature on | The long-context lane. A preview endpoint: it can be withdrawn without a rename. |
|
|
106
|
+
|
|
107
|
+
## What the tier cannot do
|
|
108
|
+
|
|
109
|
+
Three capabilities the catalog advertises are not reachable, and no lane claims
|
|
110
|
+
them:
|
|
111
|
+
|
|
112
|
+
- **No free model reads a PDF.** The provider's endpoint rejects it before the
|
|
113
|
+
model sees it: `content part type "file" is not supported; only text and
|
|
114
|
+
image_url are`. OpenCode 1.19 also refuses to read a binary file at all -
|
|
115
|
+
`Cannot read binary file`. A document is therefore extracted locally
|
|
116
|
+
(`pdftotext`, or a library) and the **text** is what reaches the `doc` lane.
|
|
117
|
+
The `doc` prompt says so, and asks for the text rather than the path.
|
|
118
|
+
- **No free model receives audio or video.** The same binary-file refusal
|
|
119
|
+
applies. MiMo publishes audio input and MiMo-V2.6-Flash is in the fleet for
|
|
120
|
+
its image reading, not its audio.
|
|
121
|
+
- **`nemotron-3.5-super-vl-free` hangs on a file ATTACHMENT, and only on an
|
|
122
|
+
attachment.** A live call with an image never returned: three attempts, 350
|
|
123
|
+
seconds, no answer. It also hung on a plain `.txt` attached to the prompt.
|
|
124
|
+
It does **not** hang on a tool read: the `sweep` lane on Super VL answered a
|
|
125
|
+
`grep` sweep in 4.2 s average over three requests, with no trouble at all.
|
|
126
|
+
So the model is usable, and the rule is narrower than "Super VL is broken":
|
|
127
|
+
never hand it a file as an attachment, and it is the reasoning-off lane.
|
|
128
|
+
|
|
129
|
+
## The chains
|
|
130
|
+
|
|
131
|
+
A lane's model is a chain, not a pin. Free routes get withdrawn in days - four
|
|
132
|
+
of this kit's five original profiles were dead within a month - so every lane
|
|
133
|
+
lists the models that could serve it, best first.
|
|
134
|
+
|
|
135
|
+
```
|
|
136
|
+
build minimax-m3.1-flash-free → deepseek-v4.1-flash-free → longcat-2.5-preview-free
|
|
137
|
+
plan beam-free → deepseek-v4.1-flash-free → longcat-2.5-preview-free
|
|
138
|
+
...
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
A chain is ordered **inside its own profile's ceiling**, not globally. That is
|
|
142
|
+
the one thing the first version of the safe fleet got wrong: it inherited the
|
|
143
|
+
unrestricted fleet's order, so bumping into the zero-retention ceiling dropped
|
|
144
|
+
every lane onto whatever rung happened to be third. Two lanes were genuinely
|
|
145
|
+
mis-served by that:
|
|
146
|
+
|
|
147
|
+
- `locate` asked for `low` effort and landed on a model that publishes no
|
|
148
|
+
effort levels at all, so the request was dropped on the floor. It now lands
|
|
149
|
+
on DeepSeek, which publishes `low`, and measures 3612 ms.
|
|
150
|
+
- `vision` landed on the same model, at 9-13 s, when MiniMax reads the same
|
|
151
|
+
image in 7.7 s.
|
|
152
|
+
|
|
153
|
+
The ceiling is also the **privacy lever**. A profile declares a `maxRetention`
|
|
154
|
+
ceiling, and a lane takes the first rung at or below it:
|
|
155
|
+
|
|
156
|
+
```
|
|
157
|
+
fleet-safe maxRetention: zero-retention → every lane lands on a zero-retention model
|
|
158
|
+
fleet-all maxRetention: may-train → every lane lands on the model built for it
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
That is why one fleet file can be honest about privacy and still cover ten
|
|
162
|
+
jobs: the ceiling changes which rung wins, not which jobs exist. A chain may
|
|
163
|
+
not name a model above its own profile's ceiling at all, not even as a later
|
|
164
|
+
rung - a chain that could reach a training provider is a chain whose failover
|
|
165
|
+
would break the promise the profile makes. A test enforces that.
|
|
166
|
+
|
|
167
|
+
`oc-harness doctor` reports a lane whose whole chain has left the catalogue,
|
|
168
|
+
which is a lane with no model at all - something no amount of `refresh` on the
|
|
169
|
+
base profile would reveal.
|
|
170
|
+
|
|
171
|
+
### What the safe fleet gives up
|
|
172
|
+
|
|
173
|
+
`sweep` is the one lane that loses its reason to exist under the ceiling.
|
|
174
|
+
Super VL is the only model on the tier that publishes a `none` effort level,
|
|
175
|
+
so "reasoning off" is a job of its own; and Super VL is a may-train provider,
|
|
176
|
+
so the ceiling excludes it. Under `fleet-safe` the lane falls back to
|
|
177
|
+
LongCat, which publishes no effort levels at all, and it is a second `locate`
|
|
178
|
+
with a 1M context window. The lane is kept, because `oc-harness lanes` should
|
|
179
|
+
decide that from measurements rather than from a guess - but the docs say out
|
|
180
|
+
loud that the capability is gone.
|
|
181
|
+
|
|
182
|
+
## Measuring, and cutting
|
|
183
|
+
|
|
184
|
+
Ten lanes is a hypothesis. The only way to know which of them earn their place
|
|
185
|
+
is to count the requests:
|
|
186
|
+
|
|
187
|
+
```
|
|
188
|
+
$ oc-harness lanes
|
|
189
|
+
|
|
190
|
+
lane reqs avg ms model
|
|
191
|
+
----------------------------------------------------------------
|
|
192
|
+
plan 7 2624 beam-free
|
|
193
|
+
build 21 10104 minimax-m3.1-flash-free
|
|
194
|
+
reviewer 2 4716 nemotron-3-ultra-free
|
|
195
|
+
...
|
|
196
|
+
8 lane(s) were never used cut candidates, not defects
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
A lane that is never used is a description the model reads on every turn and a
|
|
200
|
+
free model doing nothing. Cut it from the profile's `roles`, and the lane
|
|
201
|
+
leaves the config with it.
|
|
202
|
+
|
|
203
|
+
The counter reads `.space/telemetry.jsonl` in the project, written by the
|
|
204
|
+
plugin: one record per assistant message, with the lane, the model and the
|
|
205
|
+
duration. Two things about it are worth knowing, because both were wrong at
|
|
206
|
+
some point and both change the number:
|
|
207
|
+
|
|
208
|
+
- **A request is one assistant message.** `opencode` emits `message.updated`
|
|
209
|
+
several times per message, so a counter that read every event reported three
|
|
210
|
+
requests for every request made. The record carries the message id and the
|
|
211
|
+
reader merges on it.
|
|
212
|
+
- **The file is found by walking up from the working directory.** The plugin
|
|
213
|
+
writes it to the project root; reading `process.cwd()` alone reported "never
|
|
214
|
+
used" for every lane whenever the command was run from a subdirectory.
|
|
215
|
+
|
|
216
|
+
## Why the model lives in the config
|
|
217
|
+
|
|
218
|
+
An agent file of the same name replaces the config entry wholesale, and a file
|
|
219
|
+
with no `model:` line makes the agent inherit the caller's model. Both were
|
|
220
|
+
observed on OpenCode 1.19, with the plugin out of the way (`--pure`):
|
|
221
|
+
|
|
222
|
+
```
|
|
223
|
+
agent file: model: opencode/beam-free
|
|
224
|
+
config: agent.build.model: opencode/minimax-m3.1-flash-free
|
|
225
|
+
result: the agent file's model wins
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
So the lanes live in `agent.<lane>` in the generated config, their prompt
|
|
229
|
+
bodies live in `prompts/` and are referenced with `{file:...}`, and
|
|
230
|
+
`oc-harness verify` fails when an installed agent file shadows a lane.
|
|
231
|
+
|
|
232
|
+
The path in that reference has to be absolute or start with `~`. A relative
|
|
233
|
+
path is silently ignored, and a lane with no instructions still runs, which is
|
|
234
|
+
the worst of both.
|
|
235
|
+
|
|
236
|
+
`verify` scans four directories, because opencode reads four: `agent/` and
|
|
237
|
+
`agents/`, in the config dir and in `.opencode/`. A shadow in the singular
|
|
238
|
+
directory is exactly as dangerous as one in the plural one.
|
|
239
|
+
|
|
240
|
+
## What the fleet costs
|
|
241
|
+
|
|
242
|
+
- **Ten descriptions in the model's context.** Every lane's name and
|
|
243
|
+
description is read on every turn. The descriptions are deliberately
|
|
244
|
+
non-overlapping, because overlapping ones are what breaks selection: the
|
|
245
|
+
measured failure is a model choosing the wrong lane.
|
|
246
|
+
- **One extra model call per hand-off.** A delegated lane is a fresh context
|
|
247
|
+
window and a fresh request. Delegating a one-line lookup costs more than
|
|
248
|
+
doing it.
|
|
249
|
+
- **A shared reserve.** `compaction.reserved` is one number for the whole
|
|
250
|
+
config, so it is sized by the **narrowest** lane: 24,000 tokens for
|
|
251
|
+
`fleet-safe`, whose narrowest lane has a 200k window, and 19,200 for
|
|
252
|
+
`fleet-all`, whose narrowest lane is Big Pickle at a 160k input cap.
|
|
253
|
+
Sizing it by the widest model instead would reserve 125,829 tokens and
|
|
254
|
+
leave every 200k lane with 74k usable.
|