opencode-harness-kit 0.5.7 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -12,6 +12,336 @@ is not happening.
12
12
 
13
13
  ---
14
14
 
15
+ ## [Unreleased]
16
+
17
+ ## [0.6.0] — 2026-10-22
18
+
19
+ Published. The tag is what cut it: `git tag v0.6.0 && git push origin
20
+ main v0.6.0`, and the publish job watches for exactly that.
21
+
22
+ A minor: a command for the half of the perishable-tier question nobody was
23
+ answering, and the fix that made it answerable.
24
+
25
+ The free tier turns over in days. `doctor` said whether the model you depend
26
+ on was still there; it said nothing about what had come to replace it, so a
27
+ withdrawal left a hole only a person could fill by hand out of a catalogue
28
+ dump. `research` fills the hole with a shortlist, and `refresh` takes the
29
+ dead rung out of the chain it was in.
30
+
31
+ Two of the bugs here were the same bug wearing different clothes: a check
32
+ that asked "is this model still IN the catalogue" and called the answer
33
+ "yes, it can run". models.dev does not delete a model the provider pulls - it
34
+ keeps the record and marks it `status: "deprecated"`. So this kit's own
35
+ default fleet was carrying a dead fallback rung in **all ten lanes** while
36
+ `doctor` called it healthy.
37
+
38
+ ### Added
39
+
40
+ - **`refresh` now takes the dead rung out.** `research` tells you a rung is
41
+ dead; `refresh` removes it. A chain names the models that could serve a
42
+ lane, best first, and a model the provider has withdrawn cannot run - so it
43
+ is dead weight in a chain whose entire job is to survive a withdrawal.
44
+
45
+ A lane whose own primary was withdrawn takes the next live rung of its own
46
+ chain, because a chain has to start with its model. The fallback queue
47
+ loses its dead entries, and the utility route moves to a model the profile
48
+ still reaches.
49
+
50
+ **The refresh only ever REMOVES.** Putting a model into a lane changes what
51
+ that lane does, and that is a person's decision - it is what `research` is
52
+ for. A script that quietly added a candidate would be putting a model whose
53
+ terms nobody has read into the path of every prompt.
54
+
55
+ Two cases a script cannot repair are reported rather than papered over: a
56
+ profile whose own model was withdrawn (there is no chain behind it to fall
57
+ back to, so the honest fix is to retire the profile, and that is a person's
58
+ call), and a lane left with no model at all.
59
+
60
+ - **`oc-harness research` — the half of the perishable-tier question nobody
61
+ was answering.** `doctor` says whether the model you depend on is still
62
+ there. It said nothing about what has come to replace it, so a withdrawal
63
+ left a hole only a person could fill, by hand, out of a catalogue dump.
64
+
65
+ The new command reads the live catalogue and reports the models in use that
66
+ have been withdrawn — split into the ones that are a profile's own base
67
+ model and the ones that are a rung in a lane chain — then the lanes of the
68
+ default profile with a count of the rungs still alive in each, then the
69
+ candidates: live, free models the kit does not use yet, each with its
70
+ context window, its output cap, the effort levels it publishes, and **which
71
+ lanes it could serve**.
72
+
73
+ "Which lanes it could serve" is the part that makes a candidate usable
74
+ rather than merely interesting. Every lane needs tool calls, the `vision`
75
+ lane needs a model that takes an image, and a lane whose role asks for
76
+ `max` needs a model that publishes `max`. A model that publishes no effort
77
+ levels at all counts as a match, because there is nothing to mismatch.
78
+
79
+ The logic lives in one file, [`src/research.js`](src/research.js), with no
80
+ I/O in it, and the command is a thin layer over it.
81
+
82
+ **It writes nothing** — not `retention.yml`, not a profile, not the config.
83
+ That is the line `refresh` draws, for the same reason: `retention.yml` is a
84
+ human judgement about a provider's terms, and no script can read a
85
+ provider's terms and come back with a privacy verdict. A candidate with no
86
+ verdict is reported as `retention: unknown` and carries an explicit blocker
87
+ rather than being quietly filed under whichever tier it is cheapest on. The
88
+ output is a shortlist to argue with, not a decision to accept.
89
+
90
+ `research --json` prints one parseable object and nothing else, so a
91
+ scheduler can read it.
92
+
93
+ ### Fixed
94
+
95
+ - **`doctor` reported a withdrawn model as a healthy fallback.** The chain
96
+ check asked whether the model was still *in* the catalogue. models.dev does
97
+ not delete a model the provider pulls: it keeps the record and marks it
98
+ `status: "deprecated"`. So "is it still there" and "can it still run" are
99
+ two different questions, and the kit's own default fleet carried a dead
100
+ fallback rung in **all ten lanes** while `doctor` called it healthy. The
101
+ message it printed already said "withdrawn" — only the filter was wrong,
102
+ which is exactly why the dead rung stayed invisible.
103
+
104
+ - **The pruner's ceiling on the distinct outputs did not bound what it kept.**
105
+ The rule keeps the newest copy of each distinct output, and the ceiling on
106
+ that is what stops it being a leak. But the ceiling charged the budget for
107
+ the *tool* parts only, while the rule keeps the *whole message*, verbatim,
108
+ every part in it. A message with a 300-character tool output and a
109
+ 4,000-character reasoning part therefore cost the budget 300 and kept
110
+ 4,300, and twenty such messages kept 40,000 characters of reasoning under a
111
+ 5,000-character ceiling — eight times the budget, carrying exactly the
112
+ reasoning the pruner exists to stub. The ceiling now charges what the rule
113
+ actually keeps.
114
+
115
+ - **A profile that could not be loaded turned the pruner off without a word.**
116
+ The lookup sat in an empty `catch`, so a marker naming a renamed or missing
117
+ profile left the pruner disabled with no line in the log, nothing on the
118
+ console, and nothing to show the user why their context kept growing. A
119
+ project in that state could not tell the difference between "the pruner has
120
+ nothing to do" and "the pruner is broken". The failure is now reported once
121
+ per session, by name.
122
+
123
+ ### Removed
124
+
125
+ - **The `longcat-2.5-preview-free` profile.** The provider withdrew the
126
+ model. A withdrawn model is not a profile that still works - it is a route
127
+ that fails on contact - and this one was the *only* model its profile named,
128
+ so there was no chain behind it to fall back to. The profile is retired and
129
+ recorded in `profiles/index.json` under `withdrawn`, with its retention
130
+ verdict removed from `retention.yml` along with it: a verdict about a model
131
+ that cannot run is not a verdict about anything.
132
+
133
+ The two fleets are unaffected in kind: `longcat` was a fallback rung in
134
+ every lane, and each of those lanes now runs on the rung above it. The
135
+ default fleet loses its third rung, so it now runs its ten lanes on the
136
+ three zero-retention models still live on the tier.
137
+
138
+ - **The demo video is not published with the repository.** The newer take —
139
+ thirty seconds, made on a machine with no screen, the terminal in it the
140
+ real output of the real commands — was under `docs/demo/` for the 0.5.8
141
+ release, and it is out of the repository now.
142
+
143
+ A take is about one machine and one moment, and so is the recording of it.
144
+ The `.gitignore` already said as much: "a recording stays on the machine
145
+ that made it until somebody decides, deliberately, to publish it". The
146
+ decision went the other way, so the file went back where that rule keeps
147
+ it — `video/build/`, which is ignored — and the README no longer links a
148
+ video the repository does not carry.
149
+
150
+ What is published is still the old `docs/demo/pitch.mp4`: four minutes
151
+ forty-seven seconds, recorded live from a Mac terminal, with a voice. The
152
+ npm package never carried a video at all.
153
+
154
+ ## [0.5.8] — 2026-10-22
155
+
156
+ Published. The tag is what cut it: `git tag v0.5.8 && git push origin
157
+ main v0.5.8`, and the publish job watches for exactly that.
158
+
159
+ A minor: one new command, and eleven silent failures closed.
160
+
161
+ ### Added
162
+
163
+ - **`oc-harness auto` — the whole reconcile in one command, asking nothing.**
164
+ `refresh`, `install`, `use` and `verify` are each a step somebody has to
165
+ remember, in the right order, with the right flags; an agent that has to
166
+ keep an install working has to know all of that, and has to know which of
167
+ them are allowed to destroy something.
168
+
169
+ The profile is the one the installed config names, or the default. A file
170
+ of yours that the installer would overwrite is **left alone and
171
+ reported**, not overwritten and not asked about — the answer a person
172
+ would give. `--offline` skips the refresh, for a machine that cannot
173
+ reach models.dev. `--json` prints a report a script can read, and nothing
174
+ else on stdout.
175
+
176
+ `--yes` still means consent: a profile whose models may train on your
177
+ prompts is refused without it, exactly as `use` refuses it. The automation
178
+ is about the mechanics, not about the consent.
179
+
180
+ `init` is unchanged. It is the front door for a person, and it still says
181
+ so when there is no terminal to ask on.
182
+
183
+ - **A demo whose numbers are current.** `docs/demo/audit.mp4` — thirty
184
+ seconds, made on a machine with no screen, no camera and no ffmpeg: the
185
+ scene is drawn by a browser, the terminal in it is the real output of the
186
+ real commands, and the narration is on the frame. It names the numbers of
187
+ this release. The older `pitch.mp4` stays where it was — a live recording
188
+ from a Mac, with a voice — and the README now says which is which.
189
+
190
+ ### Fixed
191
+
192
+ - **A refresh overwrote a profile whose backup could not be written.** The
193
+ backup is the only copy of a file the user may have edited by hand — and
194
+ for an npm install that file is inside node_modules, where the next
195
+ `npm install` destroys it anyway. The error was swallowed and the
196
+ profile was overwritten regardless: the user's edits were gone, with no
197
+ backup and no message. A backup that cannot be written now stops the
198
+ refresh.
199
+
200
+ - **`refresh --all` reported profiles it never checked.** A profile that
201
+ could not be parsed was skipped without a word, and the summary counted
202
+ every id it had been given — so a broken profile was reported as a
203
+ refreshed one, and the withdrawn-model check never ran for it. The count
204
+ is of the profiles actually checked, the skipped ones are named, and the
205
+ command exits non-zero: a skipped profile is a failure, not a footnote.
206
+
207
+ - **`generateConfig(profile, { comments: false })` produced a file nothing
208
+ could read.** The whole header is comments, so turning comments off
209
+ emitted it without the `// ` prefix, in front of the JSON body. The
210
+ result was neither JSON nor JSONC, and the kit's own `parseJsonc`
211
+ returned null for it. It is pure JSON now.
212
+
213
+ - **A file named `toString.md` was reported as shadowing a lane.** The
214
+ check was `lane in (actual.agent ?? {})`, and `in` walks the prototype
215
+ chain — so a file named after a property every object has matched a lane
216
+ that does not exist, and `verify` reported a clean install as broken.
217
+
218
+ - **`--role` took the next flag as its value.** `oc-harness run --role
219
+ --max-rounds 5` set the role to the literal string `--max-rounds`, and
220
+ the loop then failed every round and marked every task blocked. A flag
221
+ with no value is refused now, and so is a role that is not a lane —
222
+ while the answer is still "that is not a lane".
223
+
224
+ - **The install deleted a file of yours that only looked like the kit's.**
225
+ The old agent directory was recognised by a substring, and a directory
226
+ called `space-harness/agents-backup` contains that substring. The file
227
+ was deleted. The match is on the parent directory now, segment by
228
+ segment.
229
+
230
+ - **A wrapper the list did not name was a way around the rules.** The
231
+ wrapper list held `env`, `sudo` and a handful of others, so the programs
232
+ that run a command as their argument - `timeout`, `nice`, `ionice`,
233
+ `stdbuf`, `setsid`, `chroot`, `taskset`, `busybox` - were commands the
234
+ rules had never seen. `timeout 5 rm -rf /tmp/x` deleted a tree, and
235
+ `timeout 5 npm publish` published a package, and neither was blocked.
236
+
237
+ The list is a table now, and the flags that take a value are the
238
+ WRAPPER'S OWN: `sudo -n` is a flag and the word after it is the command,
239
+ while `stdbuf -o 0` takes a mode and the word after it is the mode. One
240
+ list for all of them made `sudo -n rm -rf /tmp/x` read `rm` as the value
241
+ of `-n`, so the delete was invisible - which is the same hole in the other
242
+ direction.
243
+
244
+ - **`env -S "rm -rf /tmp/x"` was allowed.** `-S` takes a string and splits
245
+ it into words, so the string is a command line - the same thing `sh -c`
246
+ carries. It used to arrive as ONE token, and the command word was read
247
+ out of it as the text after the last slash, so the guard looked at `x`.
248
+ The payload is read now, and it is read the way `sh -c` is read.
249
+
250
+ - **`git -C/tmp reset --hard` was allowed.** `-C` and `-c` take a value, and
251
+ the value can be written against the flag. The resolver only knew the
252
+ spaced form, so the subcommand it found was the flag itself - and the
253
+ rule that watches for a history rewrite never saw the subcommand at all.
254
+
255
+ - **A public key was refused as a private key.** The rule named the key
256
+ FILES, and `id_rsa.pub` is the public half of `id_rsa` - a file the user
257
+ is meant to publish. Writing one was refused for a reason nobody could
258
+ find.
259
+
260
+ - **A file that was downloaded and then run was ordinary work.** The rule
261
+ caught `curl x | sh` and nothing else, so the same download in two steps
262
+ - fetch it to a file, then run the file - was allowed. It runs exactly the
263
+ same downloaded script.
264
+
265
+ - **A symlink out of the project was a write out of the project.** The
266
+ scope check compared paths as TEXT, so a link inside the project that
267
+ pointed at `/etc` was a path inside the project - and the write landed
268
+ outside it. The comparison is made on the real path now, and the part of
269
+ the path that does not exist yet is put back on the end.
270
+
271
+ - **`oc-harness agents` called a prototype property a lane.** The check was
272
+ `name in (config.agent)`, and `in` walks the prototype chain - so a project
273
+ agent file named `toString.md` or `constructor.md` matched a property every
274
+ object has: the file was reported as shadowing a lane that does not exist,
275
+ and the missing `model:` that was the point of the report was never
276
+ reported either. `verify` had the same check and the same fix; this one
277
+ was left behind.
278
+
279
+ - **`auto --json` dropped everything the refresh had found.** The refresh
280
+ prints what it found - a withdrawn model, a profile it could not read - and
281
+ `auto --json` silences the human output, so those findings went nowhere:
282
+ the report said `"ok": true` while a profile had never been checked
283
+ against the live catalogue, and the exit code said something else
284
+ entirely. The refresh hands its findings back now, and they are in the
285
+ report, in the step's status, and in the exit code.
286
+
287
+ - **`baseline --db=/path` measured the wrong database.** Only the spaced
288
+ form `--db /path` was read, so the `=` spelling - the one every other flag
289
+ in this kit accepts - was ignored, and the default database was measured
290
+ instead. The report then described a database nobody asked about. Both
291
+ spellings work now, and a `--db` with no value is refused instead of
292
+ crashing.
293
+
294
+ - **`doctor --check-models` reported a profile it never checked.** A profile
295
+ that could not be read was skipped without a word, and the run then said
296
+ every shipped model was still in the catalogue - so a profile that was NEVER
297
+ CHECKED was reported as one that was checked and found fine. It is the same
298
+ hole `refresh --all` had, in the one place that runs from a cron. The
299
+ skipped profiles are named now, and the run fails.
300
+
301
+ ### Changed
302
+
303
+ - **The `verify` and `doc` lanes ask nothing.** Both ran the project's own
304
+ commands, and both asked before every one of them that the lane's
305
+ allowlist did not name. A run that stops to be asked is a run that does
306
+ not finish. The mechanical safety is unchanged: the guard plugin still
307
+ refuses the denied commands in code, and the lanes still cannot edit.
308
+
309
+ ### Added
310
+
311
+ - **A recorder for a machine with no screen.** `vrec` drives the real
312
+ keyboard and the real screen, so it needs macOS, the Screen Recording
313
+ permission, and somebody to grant it. A headless box has none of those,
314
+ and it has no `ffmpeg` either, so the usual encode step has nothing to
315
+ encode with. The published demo was made with `vrec`; on such a machine
316
+ it cannot be made at all.
317
+
318
+ The recorder is in the tools repository, next to `vrec` and `btake` — one
319
+ directory up, and there is exactly one of each, because a copy inside
320
+ this repository is how two copies drift. It draws the scene on a canvas
321
+ and lets the browser be the encoder: Chromium's `captureStream` into
322
+ `MediaRecorder` at a fixed clock, which comes out as an `.mp4` with an
323
+ H.264 track and no ffmpeg on the machine at all. The text on screen is
324
+ the real output of the real commands — the take spec names the commands,
325
+ and they are run when the video is built.
326
+
327
+ The gate is the part that matters. A file that exists, has the right
328
+ duration and has the right size can still be thirty seconds of black, so
329
+ the gate decodes the take, pulls frames out of it, and looks at them: a
330
+ frame with no spread in its pixels shows nothing, and the take fails.
331
+ The frames are written out as PNGs, because a check you cannot look at
332
+ is a check you cannot argue with.
333
+
334
+ **The take is local and the video is local.** The take spec lives in
335
+ `video/takes/` and the recording lands in `video/build/`; both are
336
+ gitignored. The repository ships the recorder and not the take, and a
337
+ recording stays on the machine that made it until somebody decides,
338
+ deliberately, to publish it.
339
+
340
+ And the narration is on the screen. A voice is the best way to carry it
341
+ and it is not always there — `say` is macOS, and a headless Linux box
342
+ has no espeak, no festival and no cloud TTS — so the words go on the
343
+ frame, one caption at a time, and the take still says what it is about.
344
+
15
345
  ## [0.5.7] — 2026-10-21
16
346
 
17
347
  Published. The tag is what cut it: `git tag v0.5.7 && git push origin main
package/README.md CHANGED
@@ -101,7 +101,7 @@ The default profile is a **fleet**: ten jobs, ten free models, one session.
101
101
  The table below is the **unrestricted** fleet (`fleet-all`), where every lane
102
102
  gets the model built for it. The default is the **safe** fleet
103
103
  (`fleet-safe`), where every lane resolves to a provider with a zero-retention
104
- policy and six of the ten run on a second-choice model to stay there. Both
104
+ policy and seven of the ten run on a second-choice model to stay there. Both
105
105
  cover the same ten jobs; they differ in what they are willing to promise
106
106
  about your prompts.
107
107
 
@@ -116,7 +116,13 @@ about your prompts.
116
116
  | `doc` | read a long document or a spec | GLM 5.3 Flash | 7.4 s |
117
117
  | `schema` | produce JSON or a JSON schema | DeepSeek V4.1 Flash | 10.7 s |
118
118
  | `sweep` | sweep a lot of code mechanically | Nemotron 3.5 Super VL | 8.4 s |
119
- | `longform` | read a megabyte, or write alternatives | LongCat 2.5 Preview | 12.0 s |
119
+ | `longform` | read a megabyte, or write alternatives | MiniMax M3.1 Flash | 6.9 s |
120
+
121
+ `longform` used to run on `longcat-2.5-preview-free`, and the provider
122
+ withdrew it. A withdrawn model is not a lane, it is a hole in a lane: the
123
+ route falls to the next rung of the same chain, which is MiniMax - the only
124
+ model left on the tier with a million-token context. The `measured` column
125
+ for that lane is therefore MiniMax's, not the withdrawn model's.
120
126
 
121
127
  The `measured` column is the median of five one-line requests to that model
122
128
  on 2026-10-21 - the same prompt for every model, from the process start, so
@@ -241,7 +247,6 @@ it. One file covers ten jobs either way.
241
247
  | `beam-free` | 5 | 1,048,576 | `max` | retained |
242
248
  | `deepseek-v4.1-flash-free` | 5 | 200,000 | `max` | zero-retention |
243
249
  | `glm-5.3-flash-free` | 5 | 200,000 | `max` | zero-retention |
244
- | `longcat-2.5-preview-free` | 5 | 1,000,000 | — | zero-retention |
245
250
  | `nemotron-3.5-super-vl-free` | 5 | 1,048,576 | `xhigh` | **may train** |
246
251
  | `nemotron-3-ultra-free` | 5 | 1,000,000 | — | **may train** |
247
252
  | `nemotron-3.5-lightning-free` | 5 | 262,144 | — | **may train** |
@@ -281,7 +286,7 @@ withdrawn, and one of them was the **default** at the time:
281
286
  The full list, with a reason for each, is in
282
287
  [`profiles/index.json`](profiles/index.json).
283
288
 
284
- ### The four commands, and when each one runs
289
+ ### The five commands, and when each one runs
285
290
 
286
291
  | command | what it checks | when |
287
292
  |---|---|---|
@@ -289,6 +294,7 @@ The full list, with a reason for each, is in
289
294
  | `oc-harness doctor` | every profile against the live model catalogue, and the installed config against the active profile | **before every release**, and whenever a lane starts failing with a 429 or a "model not found" |
290
295
  | `oc-harness doctor --check-models` | one run, non-zero when a shipped model is no longer in the catalogue | from cron: the shape a scheduler wants |
291
296
  | `oc-harness doctor --watch` | keeps watching, and reports only what **changed** | leave it in a terminal for a week |
297
+ | `oc-harness research` | what could **replace** a withdrawn model, and which lanes each candidate could serve | when `doctor` reports a withdrawal, and from cron before it happens |
292
298
  | `oc-harness refresh <id>` | re-derives a profile's machine-derived fields from live metadata | when `doctor` reports drift |
293
299
  | `oc-harness verify` | the installed config still matches the profile, and the kit's own files are still installed | after every `use` / `install` |
294
300
 
@@ -351,6 +357,193 @@ out to be. A lane is never left without one.
351
357
  See [docs/KNOWN-LIMITS.md](docs/KNOWN-LIMITS.md) for the rest of what this
352
358
  tier cannot do.
353
359
 
360
+ ### Research: what can replace the model that just died
361
+
362
+ `doctor` answers one half of the question a perishable tier forces on you:
363
+ **is the model I depend on still there?** The other half is **what is there
364
+ instead** — and nobody was answering that half, so a withdrawal left a hole
365
+ only a person could fill, by hand, out of a catalogue dump.
366
+
367
+ `oc-harness research` is that half, and it is a file:
368
+ [`src/research.js`](src/research.js).
369
+
370
+ ```
371
+ $ oc-harness research
372
+
373
+ research tier opencode, 10 live of 46 free
374
+
375
+ ✓ every model in use is still live
376
+
377
+ lanes
378
+ build 2 live 0 dead
379
+ plan 2 live 0 dead
380
+ reviewer 2 live 0 dead
381
+ ...
382
+
383
+ candidates live, free, and not in use yet
384
+
385
+ flint-40b-free ctx 262,144 out 65,536 effort low,medium,high,xhigh,max
386
+ fits: plan, build, reviewer, locate, verify, doc, schema, sweep, longform
387
+ retention: unknown
388
+ no retention verdict in retention.yml - a person has to read the provider's terms
389
+
390
+ nothing here is written anywhere. A retention verdict is a person reading a provider's terms.
391
+ ```
392
+
393
+ That is the healthy state. When something has been withdrawn, the same command
394
+ says which, and says where it was used:
395
+
396
+ ```
397
+ gone 1 model(s) in use have been withdrawn
398
+
399
+ ✗ longcat-2.5-preview-free
400
+ withdrawn from the provider - the catalogue keeps the record and marks it deprecated
401
+ base model of longcat-2.5-preview-free
402
+ fallback rung in fleet-all: build, plan, locate, vision, doc, sweep, longform
403
+ fallback rung in fleet-safe: build, plan, reviewer, locate, verify, vision, doc, schema, sweep, longform
404
+
405
+ lanes
406
+ build 2 live 1 dead (longcat-2.5-preview-free)
407
+ plan 2 live 1 dead (longcat-2.5-preview-free)
408
+ ...
409
+ ```
410
+
411
+ It reports four things:
412
+
413
+ 1. **The models in use that have been withdrawn**, split into the ones that are
414
+ a profile's own base model and the ones that are a rung in a lane chain. A
415
+ dead base model means that profile does nothing; a dead chain rung is a
416
+ fallback that fails on the day it is needed.
417
+ 2. **The lanes of the default profile**, each with how many rungs of its chain
418
+ are still alive. A lane with zero live rungs is a lane with no model.
419
+ 3. **The candidates** — live, free models the kit does not use yet, each with
420
+ its context window, its output cap, the effort levels it publishes, and
421
+ **which lanes it could serve**.
422
+ 4. **What is still undecided** about each candidate.
423
+
424
+ #### Why the candidate list knows about lanes
425
+
426
+ A model is not "better" or "worse" in the abstract. It is better or worse
427
+ **for a lane**, and the lanes are not asking for the same thing:
428
+
429
+ | what the lane needs | why |
430
+ |---|---|
431
+ | tool calls | every lane calls tools, so every lane needs a model that can |
432
+ | an image in | the `vision` lane reads an image, so it needs a model that takes one |
433
+ | a published effort level | a lane whose role asks for `max` needs a model that publishes `max` |
434
+
435
+ So `research` matches each candidate against each lane, and a model that
436
+ publishes **no** effort levels counts as a match rather than a failure: there
437
+ is nothing to mismatch, and the kit already puts such models in the chains of
438
+ lanes that ask for one.
439
+
440
+ #### The line the script does not cross
441
+
442
+ `research` **writes nothing**. Not `retention.yml`, not a profile, not the
443
+ config.
444
+
445
+ That is the same line `refresh` draws, and it is drawn for the same reason:
446
+ `retention.yml` is a human judgement about a provider's terms — whether your
447
+ prompts are kept, and whether they train on them. A machine cannot read a
448
+ provider's terms of service and come back with a privacy verdict. So a
449
+ candidate with no verdict is reported as `retention: unknown` and carries an
450
+ explicit blocker, rather than being quietly filed under the tier it happens to
451
+ be cheapest on.
452
+
453
+ `refresh` draws the line in the other direction, and it is the one that
454
+ matters: it **only ever removes**. A model that has been withdrawn is removed
455
+ from the chains. A model that might do the job better is never added, because
456
+ putting a model into a lane changes what that lane does, and that is a
457
+ person's decision.
458
+
459
+ The output of `research` is a shortlist to argue with, not a decision to
460
+ accept. When you have read the terms and made the call, `retention.yml` is
461
+ where the call lives, and `doctor` checks the rest.
462
+
463
+ #### Run it on a schedule
464
+
465
+ ```cron
466
+ 0 9 * * * cd ~/.config/opencode && oc-harness doctor --check-models
467
+ 30 9 * * * cd ~/.config/opencode && oc-harness research
468
+ ```
469
+
470
+ The first line says a route died. The second says what to do about it, and it
471
+ runs half an hour later because by then you have had a cup of coffee and the
472
+ answer is a decision, not an alarm.
473
+
474
+ `research --json` prints one parseable object and nothing else, so a script
475
+ can read it:
476
+
477
+ ```json
478
+ {
479
+ "ok": true,
480
+ "tier": { "provider": "opencode", "total": 46, "live": 10, "withdrawn": 36 },
481
+ "shipped": { "count": 9, "gone": [] },
482
+ "lanes": [ { "lane": "build", "dead": [], "live": ["minimax-m3.1-flash-free", "deepseek-v4.1-flash-free"] } ],
483
+ "candidates": [ { "id": "flint-40b-free", "fits": ["build"], "retention": "unknown", "blockers": ["..."] } ]
484
+ }
485
+ ```
486
+
487
+ #### One thing the catalogue does that a naive check misses
488
+
489
+ **A withdrawn model stays in the catalogue.** models.dev does not delete a
490
+ model the provider pulls — it keeps the record and marks it
491
+ `status: "deprecated"`. So "is it still in the catalogue" and "can it still
492
+ run" are two different questions, and a check that asks the first one reports
493
+ a dead model as a healthy fallback.
494
+
495
+ That is not hypothetical. This kit's own default fleet carried a dead fallback
496
+ rung in **all ten lanes**, and `doctor` called it healthy, because the chain
497
+ check asked whether the model was still present. It now asks whether it is
498
+ still alive, and it says so:
499
+
500
+ ```
501
+ ! fleet-safe drift
502
+ lane build: longcat-2.5-preview-free withdrawn, falls back to minimax-m3.1-flash-free
503
+ ```
504
+
505
+ **Free is a price, not a name.** The tier is the set of models the provider
506
+ charges nothing for. `big-pickle` is on it and does not end in `-free`, so
507
+ the suffix is a naming convention and not the test — `research` reads the
508
+ price.
509
+
510
+ ### Refresh: taking the dead rung out
511
+
512
+ `research` tells you a rung is dead. `refresh` takes it out.
513
+
514
+ ```
515
+ $ oc-harness refresh --all
516
+
517
+ refreshed 11 of 11 profile(s) against the live catalogue
518
+ ~ fleet-safe: withdrawn rungs removed: longcat-2.5-preview-free
519
+ lane build: minimax, deepseek, longcat -> minimax, deepseek
520
+ lane longform: longcat, minimax, deepseek -> minimax, deepseek
521
+ ...
522
+ ```
523
+
524
+ A chain names the models that could serve a lane, best first. A model the
525
+ provider has withdrawn is not one of them any more: it cannot run, so it is
526
+ dead weight in a chain whose entire job is to survive a withdrawal. The
527
+ refresh removes it, and if the lane's own primary was the dead one, the lane
528
+ takes the next live rung as its primary — a chain has to start with its
529
+ model. The fallback queue loses its dead entries, and the utility route moves
530
+ to a model the profile still reaches.
531
+
532
+ **The refresh only ever REMOVES.** Putting a model into a lane changes what
533
+ that lane does, and that is a person's decision — it is what `research` is
534
+ for. A script that quietly added a candidate would be putting a model whose
535
+ terms nobody has read into the path of every prompt.
536
+
537
+ Two cases a script cannot repair, and says so instead of pretending:
538
+
539
+ - **A profile whose own model was withdrawn.** There is no chain behind it to
540
+ fall back to, so there is nothing to repair. `refresh` reports it and leaves
541
+ the file alone; the honest fix is to retire the profile, and retiring one is
542
+ a decision a person makes. (That is exactly what happened to
543
+ `longcat-2.5-preview-free`.)
544
+ - **A lane left with no model at all.** Every rung of its chain was dead. It
545
+ is reported, not filled.
546
+
354
547
  ## Requirements
355
548
 
356
549
  | | |
@@ -358,9 +551,9 @@ tier cannot do.
358
551
  | Node | **22 or newer** — `engines` in package.json, and the CLI uses `node:` prefixed builtins. Node 20 fails at import time, not with a friendly message. |
359
552
  | npm | 10 or newer, for `npm ci` when installing from a checkout |
360
553
  | OS | macOS, Linux and Windows. The tests run on `ubuntu-latest`. |
361
- | Shell | none needed to *use* the kit — no `bash`, `zsh` or `fish` setup. `doctor` and `refresh` call `curl` to read models.dev. `baseline` and the measurement scripts read the install database with `sqlite3`, and fall back to python3's own `sqlite3` module when that binary is not installed — so python3 is the one thing to have. Everything else, including `install`, `use` and `verify`, works offline. |
362
- | Network | only `doctor` and `refresh` need it, and only to read the live model catalogue. `list` is offline: it reads the shipped profiles. Everything else works offline. |
363
- | Disk | `use` and `install` write into `~/.config/opencode/` and nothing else. `doctor` writes nothing. `refresh` rewrites a profile in the kit's own
554
+ | Shell | none needed to *use* the kit — no `bash`, `zsh` or `fish` setup. `doctor`, `refresh` and `research` call `curl` to read models.dev. `baseline` and the measurement scripts read the install database with `sqlite3`, and fall back to python3's own `sqlite3` module when that binary is not installed — so python3 is the one thing to have. Everything else, including `install`, `use` and `verify`, works offline. |
555
+ | Network | only `doctor`, `refresh` and `research` need it, and only to read the live model catalogue. `list` is offline: it reads the shipped profiles. Everything else works offline. |
556
+ | Disk | `use` and `install` write into `~/.config/opencode/` and nothing else. `doctor` writes nothing. `research` writes nothing — it is a report, not a migration. `refresh` rewrites a profile in the kit's own
364
557
  directory - `node_modules` for an npm install - and keeps a backup of the
365
558
  one it replaced. |
366
559
 
@@ -375,6 +568,8 @@ keeps a dated copy in `~/.config/opencode/backups/`, including under
375
568
  oc-harness list profiles with retention and caveats
376
569
  oc-harness show <id> resolved lanes, derived numbers, fallback chains, guard rules
377
570
  oc-harness init pick a free profile, then install + use + verify
571
+ oc-harness auto refresh + install + use + verify, asking nothing
572
+ oc-harness auto --json the same reconcile, as a report a script can read
378
573
  oc-harness use <id> compile a profile into ~/.config/opencode/opencode.jsonc
379
574
  oc-harness use <id> --dryrun print a summary of what would be written, write nothing
380
575
  oc-harness refresh <id> re-derive a profile against the live model list
@@ -571,10 +766,23 @@ recent calls was a window on *recency*, and a long session walks out of it:
571
766
  after six more calls, the only copy of a file the agent had read was gone,
572
767
  and the agent read it again. On the long-task A/B that was seventeen
573
768
  re-reads and five times the input tokens. The newest copy of each distinct
574
- output now survives however old it is, and only the repeats are stubbed -
575
- **-69% input tokens and -60% wall time** on that task, at the cost of a
576
- bigger context. It is a setting: `prune.keepDistinctOutputs` in the profile,
577
- `false` to get the old behaviour back.
769
+ output now survives, and only the repeats are stubbed - **-69% input tokens
770
+ and -60% wall time** on that task, at the cost of a bigger context. It is a
771
+ setting: `prune.keepDistinctOutputs` in the profile, `false` to get the old
772
+ behaviour back.
773
+
774
+ **The distinct outputs have a ceiling.** The rule without one is not a
775
+ trade-off but a leak: it keeps every output the session has ever produced,
776
+ and a long session grows without limit. Measured on a real 137-message
777
+ session on 2026-10-22: the request was 5.26 MB, the pruner removed 0.04 MB of
778
+ it (0.8%), and 4.74 MB of it - 92% - was the distinct outputs of the whole
779
+ session. The profile's own budget for that request was 176,000 characters,
780
+ so the request was twenty-nine times over. The ceiling keeps the newest
781
+ distinct outputs and stubs the ones past it, so the property the rule was
782
+ added for - the copy the agent is working with survives, so it does not read
783
+ it again - still holds. It is `prune.keepDistinctBudgetChars` in the profile;
784
+ unset means the profile's own context budget, and `0` keeps no distinct
785
+ output at all.
578
786
 
579
787
  ## Memory
580
788
 
@@ -677,6 +885,28 @@ oc-harness install # writes everything that file points at
677
885
  `GLOBAL.md` into `~/.config/opencode/`, so this repo stays the single source
678
886
  of truth and an edit here takes effect on the next opencode start.
679
887
 
888
+ ### The one command that asks nothing
889
+
890
+ `init` is the front door for a person: it asks which free profile to run on.
891
+ `auto` is the one for a job, an agent, or a script — it reconciles the whole
892
+ install and never waits for an answer:
893
+
894
+ ```bash
895
+ oc-harness auto # refresh + install + use + verify
896
+ oc-harness auto --json # the same, as a report a script can read
897
+ oc-harness auto --offline # skip the refresh, for a machine that cannot reach models.dev
898
+ ```
899
+
900
+ The profile is the one the installed config names, or the default. A file of
901
+ yours that the installer would overwrite is **left alone and reported**, not
902
+ overwritten and not asked about — the answer a person would give.
903
+
904
+ `--yes` still means consent. A profile whose models may train on your prompts
905
+ is refused without it, exactly as `use` refuses it: the automation is about
906
+ the mechanics, not about the consent. `auto` exits non-zero when `verify`
907
+ finds a problem it is not allowed to repair, and `--json` puts the problems in
908
+ the report.
909
+
680
910
  Then restart opencode. Check it took:
681
911
 
682
912
  ```bash