opencode-harness-kit 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/GLOBAL.md +110 -0
- package/LICENSE +21 -0
- package/README.md +421 -2
- package/agents/build.md +66 -0
- package/agents/locate.md +48 -0
- package/agents/plan.md +61 -0
- package/agents/reviewer.md +62 -0
- package/agents/verify.md +54 -0
- package/bin/baseline.mjs +367 -0
- package/bin/git-hooks/pre-commit +30 -0
- package/bin/oc-harness.mjs +977 -0
- package/dist/index.js +976 -0
- package/docs/KNOWN-LIMITS.md +188 -0
- package/docs/RESEARCH.md +210 -0
- package/examples/muse-spark-1.3-contributor-free.json +41 -0
- package/package.json +77 -4
- package/profiles/deepseek-v4-flash-free.json +41 -0
- package/profiles/index.json +60 -0
- package/profiles/longcat-2.5-preview-free.json +41 -0
- package/profiles/schema.json +132 -0
- package/profiles/space-bunny-free.json +42 -0
- package/profiles/x-preview-f-free.json +41 -0
- package/retention.yml +110 -0
- package/skills/space-build/SKILL.md +54 -0
- package/skills/space-finish/SKILL.md +63 -0
- package/skills/space-plan/SKILL.md +101 -0
- package/skills/space-plan/reference/task-template.md +47 -0
- package/skills/space-review/SKILL.md +53 -0
- package/src/config.js +326 -0
- package/src/effort.js +98 -0
- package/src/guard.ts +59 -0
- package/src/index.ts +80 -0
- package/src/profile.js +333 -0
- package/src/prune.ts +258 -0
- package/src/router.ts +192 -0
- package/src/rules.js +496 -0
- package/src/telemetry.ts +98 -0
|
@@ -0,0 +1,188 @@
|
|
|
1
|
+
# Known limits
|
|
2
|
+
|
|
3
|
+
Written down so nobody spends a day rediscovering them.
|
|
4
|
+
|
|
5
|
+
## A global `permission.edit: "ask"` silently un-locks every read-only agent
|
|
6
|
+
|
|
7
|
+
Worth knowing before you hand-edit the generated config, because the failure is
|
|
8
|
+
invisible from the config itself.
|
|
9
|
+
|
|
10
|
+
Two documented OpenCode behaviours combine badly:
|
|
11
|
+
|
|
12
|
+
1. Permission rules are matched with **the last matching rule winning**.
|
|
13
|
+
2. The **global** rule is appended *after* the agent's own rule.
|
|
14
|
+
|
|
15
|
+
So an agent configured `edit: "deny"` under a global `permission.edit: "ask"`
|
|
16
|
+
resolves to **ask**, not deny. The read-only roles can edit files, just not
|
|
17
|
+
without a prompt. Confirmed on 1.18.35 with `opencode agent list`, which prints
|
|
18
|
+
each agent's fully resolved rule list — for `plan` it showed `edit * deny`
|
|
19
|
+
followed by `edit * ask`.
|
|
20
|
+
|
|
21
|
+
The fix is counter-intuitive: set the global to `allow`. `allow` is the
|
|
22
|
+
permissive *implicit default* and emits no rule at all, so each agent's `deny` is
|
|
23
|
+
what decides. Verified after the change:
|
|
24
|
+
|
|
25
|
+
```
|
|
26
|
+
build edit -> allow
|
|
27
|
+
plan edit -> deny
|
|
28
|
+
locate edit -> deny
|
|
29
|
+
reviewer edit -> deny
|
|
30
|
+
verify edit -> deny
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Writes outside the project still ask, via `external_directory`.
|
|
34
|
+
|
|
35
|
+
## Two schema keys this kit used to emit, and what they do instead
|
|
36
|
+
|
|
37
|
+
The 1.18.35 `Config` schema sets `additionalProperties: false`, so an
|
|
38
|
+
unrecognised top-level key is stripped from the resolved config rather than
|
|
39
|
+
rejected. Two keys were being written that OpenCode throws away:
|
|
40
|
+
|
|
41
|
+
- **`variant`** is not a top-level config key at all. It only exists on
|
|
42
|
+
`agent.<name>.variant`. Per-role tiering is delivered by those, and
|
|
43
|
+
`effort.js` sets the session default in the `config` hook where the runtime
|
|
44
|
+
still sees it. A test once asserted this key existed, which meant it was
|
|
45
|
+
asserting that dead config was present.
|
|
46
|
+
- **`harness`** survives in the file but is not visible to the `config` hook, so
|
|
47
|
+
the plugin reads the profile id from the file on disk instead. That behaviour
|
|
48
|
+
is correct and `profile.js` documents it; `variant` had no such workaround and
|
|
49
|
+
was simply doing nothing.
|
|
50
|
+
|
|
51
|
+
`oc-harness verify` fails on a top-level `variant` specifically, because it looks
|
|
52
|
+
exactly like working tiering in a config you are reading.
|
|
53
|
+
|
|
54
|
+
## A generated config is only half an install
|
|
55
|
+
|
|
56
|
+
`use` writes `opencode.jsonc`. That file *points at* three other things: the
|
|
57
|
+
instruction file in `instructions`, the agent files it declares per role, and the
|
|
58
|
+
plugin that enforces the guard. All three can be absent while the config still
|
|
59
|
+
validates and still looks correct — and, on a real install of this kit, all three
|
|
60
|
+
were.
|
|
61
|
+
|
|
62
|
+
The specific failure was invisible because `opencode agent list` simply does not
|
|
63
|
+
print an agent that does not exist. A config declaring `reviewer`, `locate` and
|
|
64
|
+
`verify` produced no error and no warning; the roles were invoked and quietly
|
|
65
|
+
resolved to nothing.
|
|
66
|
+
|
|
67
|
+
That is what `oc-harness install` and `oc-harness verify` exist for. Anything the
|
|
68
|
+
compiler emits a reference to should have an installer and a checker; otherwise
|
|
69
|
+
the generated file is a pointer to nothing.
|
|
70
|
+
|
|
71
|
+
## Model failover happens at the next session, never mid-flight
|
|
72
|
+
|
|
73
|
+
No OpenCode plugin hook can change the model of a running session. Verified by
|
|
74
|
+
reading `Hooks` in `@opencode-ai/plugin@1.18.35`. The surface is 21 hooks; of
|
|
75
|
+
the six that mention `model`, five read it as input and the sixth
|
|
76
|
+
(`experimental.provider.small_model`) writes only the utility model, never the
|
|
77
|
+
session's:
|
|
78
|
+
|
|
79
|
+
| hook | can it change the model? |
|
|
80
|
+
|---|---|
|
|
81
|
+
| `chat.params` | No. Output is `temperature`, `topP`, `topK`, `maxOutputTokens`, `options`. |
|
|
82
|
+
| `chat.message` | No. `model` and `variant` on the input are read-only. |
|
|
83
|
+
| `experimental.provider.small_model` | Only the small model. |
|
|
84
|
+
| `config` | Yes, but it runs once at startup. |
|
|
85
|
+
| `provider` | Can add or override models in the catalog, not the active session. |
|
|
86
|
+
|
|
87
|
+
So `router.ts` does two things instead of the obvious third: at startup it
|
|
88
|
+
moves to the next healthy profile, and when a request fails it tells the agent
|
|
89
|
+
plainly that a new session is the fix. It does not pretend to retry.
|
|
90
|
+
|
|
91
|
+
The gap is real and tracked upstream. Provider issue #42977 asks Zen for a
|
|
92
|
+
`scope` field and a remaining-quota figure on 429 responses, precisely because
|
|
93
|
+
a client cannot currently tell a short quota window from a permanent block.
|
|
94
|
+
|
|
95
|
+
## Retry on 429 is deliberately not attempted
|
|
96
|
+
|
|
97
|
+
Zen free routes return `429 FreeUsageLimitError`, and #42977 documents models
|
|
98
|
+
staying blocked for **10+ days** without recovery. The `Retry-After` header
|
|
99
|
+
measured by the reporter points at 00:00 UTC, so the window is daily. The
|
|
100
|
+
maintainers' own guidance is to mark the route unhealthy rather than retry.
|
|
101
|
+
|
|
102
|
+
Retrying inside a session would burn turns against a wall. Marking it
|
|
103
|
+
unhealthy and failing over at the next start is the honest behaviour.
|
|
104
|
+
|
|
105
|
+
## Free models get withdrawn quickly
|
|
106
|
+
|
|
107
|
+
From one install's 20-day history, free model slots turned over in days:
|
|
108
|
+
|
|
109
|
+
| model | observed lifetime |
|
|
110
|
+
|---|---|
|
|
111
|
+
| `mimo-v2.5-free` | 1 day |
|
|
112
|
+
| `big-pickle` | 1 day |
|
|
113
|
+
| `nemotron-3-ultra-free` | 3 days |
|
|
114
|
+
| `muse-spark-1.3-contributor-free` | still running at 20 days |
|
|
115
|
+
|
|
116
|
+
Zen states free models are available "for a limited time" with no date. Treat
|
|
117
|
+
profiles as perishable: run `oc-harness doctor`, which re-checks each profile
|
|
118
|
+
against live metadata and reports drift.
|
|
119
|
+
|
|
120
|
+
## A model-driven compressor is not assumed to work
|
|
121
|
+
|
|
122
|
+
`@tarquinen/opencode-dcp` (4.3k stars) exposes a `compress` tool so the model
|
|
123
|
+
chooses when to compact. That is a better design in principle, and it is
|
|
124
|
+
AGPL-3.0, and it depends on a model remembering to call a tool.
|
|
125
|
+
|
|
126
|
+
This kit targets models with weak instruction following, so `prune.ts` is
|
|
127
|
+
deterministic and unconditional. If you want both, install DCP for its
|
|
128
|
+
deduplication and purge-errors passes and let this handle the floor. Check
|
|
129
|
+
whether its `compress` tool actually fires on your model before assuming it
|
|
130
|
+
does; if it does not, `prune.ts` is doing the work anyway.
|
|
131
|
+
|
|
132
|
+
## Pruning is an approximation
|
|
133
|
+
|
|
134
|
+
`prune.ts` keeps pointers rather than bodies. Full tool output is not deleted
|
|
135
|
+
from the session record, and OpenCode already writes over-cap output to its own
|
|
136
|
+
truncation directory, so re-reading is possible. But "re-run the command to see
|
|
137
|
+
it again" is more expensive than "scroll up". The stub is one line precisely so
|
|
138
|
+
an agent can decide whether it is worth it, and the stub says so.
|
|
139
|
+
|
|
140
|
+
A `savedBytes` of 0 in `prune.jsonl` is not a second, redundant pass. It means
|
|
141
|
+
`bytesAfter` did not come out below `bytesBefore` for that turn, which happens
|
|
142
|
+
when a stub is no shorter than the output it replaced (the stub's own label
|
|
143
|
+
costs more than a short body), or when a `reasoning` part was under the 200
|
|
144
|
+
byte floor and so was not shortened at all. These lines are still logged
|
|
145
|
+
because a part was stubbed, which is the event worth recording. Savings
|
|
146
|
+
aggregate correctly either way, since each line is measured independently.
|
|
147
|
+
|
|
148
|
+
`baseline.mjs` measures permission pressure with its own glob approximation of
|
|
149
|
+
OpenCode's matcher, not OpenCode's matcher. It handles the cases that matter
|
|
150
|
+
(bare commands, embedded newlines) but a discrepancy in the last few percent is
|
|
151
|
+
possible. Treat the ratio as a guide, not an exact count. It does import the
|
|
152
|
+
real allowlist rather than keeping a copy, so the list cannot drift from what is
|
|
153
|
+
actually shipped.
|
|
154
|
+
|
|
155
|
+
## What this kit does not do
|
|
156
|
+
|
|
157
|
+
It does not implement memory persistence, worktree orchestration, a context
|
|
158
|
+
compressor, or a session UI. Those exist and are good:
|
|
159
|
+
|
|
160
|
+
| need | use |
|
|
161
|
+
|---|---|
|
|
162
|
+
| model-driven compression | `@tarquinen/opencode-dcp` |
|
|
163
|
+
| cross-harness context proxy | `sleev.ai` |
|
|
164
|
+
| persistent memory | `opencode-supermemory` |
|
|
165
|
+
| git worktrees | `kdcokenny/opencode-worktree` |
|
|
166
|
+
| memory in git markdown | `code-yeongyu/oh-my-opencode` (now `omo`) |
|
|
167
|
+
| agent DAGs | `spoons-and-mirrors/subtask2` |
|
|
168
|
+
|
|
169
|
+
The gap this kit fills is the one between a provider catalog and a config file:
|
|
170
|
+
capability and privacy metadata in, a working harness out, with a command to
|
|
171
|
+
re-check that the metadata has not rotted.
|
|
172
|
+
|
|
173
|
+
## Verification methodology
|
|
174
|
+
|
|
175
|
+
Read
|
|
176
|
+
[Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents](https://arxiv.org/abs/2609.00006)
|
|
177
|
+
(arXiv:2609.00006) before extending this. It dissects eleven production
|
|
178
|
+
harnesses along seven subsystems and catalogues 29 recurring patterns. Two of
|
|
179
|
+
its findings shaped the design here:
|
|
180
|
+
|
|
181
|
+
- No harness in the corpus uses embedding-based retrieval over code. Everything
|
|
182
|
+
runs on ripgrep, tree-sitter, glob, and auto-discovered markdown. An earlier
|
|
183
|
+
draft of this project proposed GraphRAG; that was wrong.
|
|
184
|
+
- `syntax-aware command permissioning` is a named pattern, which is why
|
|
185
|
+
`guard.ts` splits on shell separators instead of matching a raw string.
|
|
186
|
+
|
|
187
|
+
The paper's own advice applies: build these components knowing that some will
|
|
188
|
+
stop being necessary as models improve.
|
package/docs/RESEARCH.md
ADDED
|
@@ -0,0 +1,210 @@
|
|
|
1
|
+
# Research notes
|
|
2
|
+
|
|
3
|
+
Where the numbers came from, and what was tried and thrown away. The short
|
|
4
|
+
version: most of the first draft of this project was wrong, and the corrections
|
|
5
|
+
are more useful than the conclusions.
|
|
6
|
+
|
|
7
|
+
## The measurement everything rests on
|
|
8
|
+
|
|
9
|
+
One real install, 14 days, read out of `~/.local/share/opencode/opencode.db`.
|
|
10
|
+
Reproduce with `npm run baseline`.
|
|
11
|
+
|
|
12
|
+
```
|
|
13
|
+
153 sessions · 11,750 assistant messages
|
|
14
|
+
input 66.07M · output 3.00M · reasoning 1.45M · cache read 1358.49M
|
|
15
|
+
cache read = 20.6x input
|
|
16
|
+
recorded cost $0.40 · the same work on a paid route would be $53.36
|
|
17
|
+
|
|
18
|
+
context per request p50 99,648 p90 246,536 p99 482,779 max 547,986
|
|
19
|
+
fresh input per request p50 534
|
|
20
|
+
compaction events 0
|
|
21
|
+
|
|
22
|
+
stored message bytes
|
|
23
|
+
tool 114.78 MB 77.9% (11,817 parts)
|
|
24
|
+
reasoning 22.94 MB 15.6%
|
|
25
|
+
text 3.04 MB 2.1%
|
|
26
|
+
|
|
27
|
+
per model
|
|
28
|
+
muse-spark-1.3-contributor-free xhigh 7,955 msgs reasoning 31.9% 7.8s median
|
|
29
|
+
space-bunny-free max 2,684 msgs reasoning 38.1% 18.7s median
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Three conclusions:
|
|
33
|
+
|
|
34
|
+
**Context is the cost.** Fresh input per request is 534 tokens; context per
|
|
35
|
+
request is 99,648. Almost every request is a re-read. With 78% of stored bytes
|
|
36
|
+
in tool output, the session is carrying command output forward forever.
|
|
37
|
+
|
|
38
|
+
**Effort is the biggest latency lever.** 18.7s versus 7.8s median is 2.4x, and
|
|
39
|
+
76.8% of responses ended on a tool call rather than prose. Running `max` for
|
|
40
|
+
mechanical work is the single largest waste in the setup.
|
|
41
|
+
|
|
42
|
+
**Compaction never ran.** Zero events across 153 sessions. The configured
|
|
43
|
+
`tail_turns: 15` was inert, because compaction is keyed off the full context
|
|
44
|
+
ceiling that a normal session never reaches.
|
|
45
|
+
|
|
46
|
+
## Permission pressure, measured rather than asserted
|
|
47
|
+
|
|
48
|
+
5,239 historical `bash` commands were replayed against the allowlist shapes:
|
|
49
|
+
|
|
50
|
+
| allowlist | prompts | share |
|
|
51
|
+
|---|---|---|
|
|
52
|
+
| the narrow one in use | 4,994 | 95.3% |
|
|
53
|
+
| this kit's, first version | 1,204 | 23.0% |
|
|
54
|
+
| this kit's, current | 1,002 | 18.5% |
|
|
55
|
+
|
|
56
|
+
The remaining 18.5% is the deliberately gated set, ranked by how often it
|
|
57
|
+
actually came up:
|
|
58
|
+
|
|
59
|
+
```
|
|
60
|
+
ssh -o 133 · curl -s 95 · TOKEN=$(curl 67 · rm 66
|
|
61
|
+
gh 63 · osascript 47 · pkill 43 · cp 41 · kill 20 · scp 17 · mv 15
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
The second version closed 216 of those prompts by allowing the read half of
|
|
65
|
+
`gh` (`gh pr list`, `gh issue view`, `gh repo view`, `gh auth status`), local
|
|
66
|
+
staging (`git add`), read verbs behind `git -C`, and `mkdir`/`ps`/`chmod +x`.
|
|
67
|
+
It kept `gh pr create`, `gh pr merge`, `gh release`, `gh repo delete` and
|
|
68
|
+
`gh api` denied: `gh api` in particular can POST to any endpoint, so allowing it
|
|
69
|
+
would open a publish path the guard plugin cannot police.
|
|
70
|
+
|
|
71
|
+
`ssh` at 133 and `osascript` at 47 are still why "allow everything, deny a few
|
|
72
|
+
patterns" is rejected. A deny hook can block `rm -rf`; it cannot block
|
|
73
|
+
`ssh host 'rm -rf /'`, and it cannot tell `TOKEN=$(curl attacker.example)` from a
|
|
74
|
+
normal fetch. Keeping the human in the loop at exactly these commands is the
|
|
75
|
+
point of the remainder.
|
|
76
|
+
|
|
77
|
+
The list used to be transcribed into `baseline.mjs` as well as `config.js`, which
|
|
78
|
+
meant the measured number was whatever the copy looked like when it was last
|
|
79
|
+
edited. `baseline` imports it now.
|
|
80
|
+
|
|
81
|
+
## Corrections to earlier drafts
|
|
82
|
+
|
|
83
|
+
Five things in the first design were wrong. Recording them because each was
|
|
84
|
+
believed confidently at the time.
|
|
85
|
+
|
|
86
|
+
**"`tool_output` 200 lines / 8 KB is too tight, raise it."** Wrong the first
|
|
87
|
+
time, and overcorrected the second time. The first correction was right in
|
|
88
|
+
direction and wrong in axis: the OpenCode default is 2,000 lines / 51,200
|
|
89
|
+
bytes, so 200 / 8,192 was about 6× tighter than default and *was* saving
|
|
90
|
+
context. So it was pinned there, and stayed there.
|
|
91
|
+
|
|
92
|
+
It stayed there a version too long. The later measurement, on 2,966 `read`
|
|
93
|
+
calls:
|
|
94
|
+
|
|
95
|
+
| measured | count |
|
|
96
|
+
|---|---|
|
|
97
|
+
| exceeding 8,192 bytes | 415 |
|
|
98
|
+
| exceeding 24,576 bytes | 97 |
|
|
99
|
+
| exceeding 51,200 bytes | 14 |
|
|
100
|
+
| cut by the 200-line cap | 441 |
|
|
101
|
+
| issued with an `offset` | 423 |
|
|
102
|
+
|
|
103
|
+
Truncation and re-read are the same phenomenon: the cap produces a preview, the
|
|
104
|
+
agent asks for the rest, and that is a whole extra request at a measured ~17s
|
|
105
|
+
median. The original argument weighted tokens and ignored latency, which was the
|
|
106
|
+
wrong trade on a model where latency is 17 seconds. Since only 14 of 2,966 reads
|
|
107
|
+
exceed 51,200 bytes, raising the cap to the default removes ~97% of the
|
|
108
|
+
truncations while barely moving the worst case — and pruning, not truncation,
|
|
109
|
+
is what actually bounds context.
|
|
110
|
+
|
|
111
|
+
`bin/baseline.mjs` now prints all five numbers so this is re-measurable instead
|
|
112
|
+
of being re-argued.
|
|
113
|
+
|
|
114
|
+
**"Compaction never ran, therefore nothing is wrong."** Zero events across 156
|
|
115
|
+
sessions was read as "the setting is fine" rather than "nothing is being
|
|
116
|
+
bounded". Both readings were half right and the compound conclusion was wrong:
|
|
117
|
+
the configured `tail_turns` was inert *and* there was no bound on growth.
|
|
118
|
+
Dropped the dead key rather than tuning it.
|
|
119
|
+
|
|
120
|
+
**"Consider GraphRAG or a vector store for code retrieval."** Wrong, and
|
|
121
|
+
checkable in one step. arXiv:2609.00006 dissects eleven production harnesses at
|
|
122
|
+
source level; **none** uses embedding-based retrieval over code. All of them
|
|
123
|
+
run ripgrep, tree-sitter, glob, and auto-discovered markdown. An earlier draft
|
|
124
|
+
cited a community plugin's GraphRAG claim without checking the primary source.
|
|
125
|
+
|
|
126
|
+
**"Build a pruner."** Four existing projects already had one, the best being
|
|
127
|
+
`@tarquinen/opencode-dcp` at 4.3k stars. Rewriting it would have been the
|
|
128
|
+
third-worst use of a week. What was worth building is the part nobody had: a
|
|
129
|
+
profile compiler, and the retention field.
|
|
130
|
+
|
|
131
|
+
**"The 46.6% that effort tiering saves is a measurement."** It was not. The
|
|
132
|
+
`eval` cost model pairs `max` medians from `space-bunny-free` with lower-tier
|
|
133
|
+
medians from `muse-spark-1.3-contributor-free` — cross-model, because the
|
|
134
|
+
tiering had never actually run on the zero-retention model: `build` had been
|
|
135
|
+
hand-pinned to the muse-spark route, so every recorded message on
|
|
136
|
+
`space-bunny-free` is at `max`. Per the install's own database:
|
|
137
|
+
|
|
138
|
+
| model | variant | messages | median |
|
|
139
|
+
|---|---|---|---|
|
|
140
|
+
| muse-spark-1.3-contributor-free | xhigh | 7,955 | 7.8s |
|
|
141
|
+
| space-bunny-free | **max** | 3,030 | 16.9s |
|
|
142
|
+
| space-bunny-free | high / xhigh / low | **0** | — |
|
|
143
|
+
|
|
144
|
+
Zero data below `max` is not a rounding detail, it is the whole claim being
|
|
145
|
+
unfalsified. `eval` now prints the figure as `PROJECTION` with `measured: no`,
|
|
146
|
+
and `baseline` reports per-model-per-variant medians so the claim becomes real
|
|
147
|
+
once the tiering has actually been running.
|
|
148
|
+
|
|
149
|
+
## Why Space Bunny Free is the default
|
|
150
|
+
|
|
151
|
+
Scanning free models on OpenCode Zen for context, effort, and privacy together:
|
|
152
|
+
|
|
153
|
+
| model | context | effort levels | retention |
|
|
154
|
+
|---|---|---|---|
|
|
155
|
+
| `space-bunny-free` | 1,048,576 | low … **max** | zero-retention |
|
|
156
|
+
| `x-preview-f-free` | 1,000,000 | low, high, max | zero-retention |
|
|
157
|
+
| `longcat-2.5-preview-free` | 1,000,000 | *(none)* | zero-retention |
|
|
158
|
+
| `deepseek-v4-flash-free` | 200,000 | low, high, max | zero-retention |
|
|
159
|
+
| `muse-spark-1.3-contributor-free` | 1,048,576 | minimal … xhigh | **may train** |
|
|
160
|
+
|
|
161
|
+
It is the only free model that is both top-tier on effort and zero-retention.
|
|
162
|
+
The Muse Spark contributor tier is free, fast, and widely used, and it is the
|
|
163
|
+
one whose provider is permitted to train on prompts. The install this replaces
|
|
164
|
+
had that model as its default, which is the specific reason `retention.yml`
|
|
165
|
+
exists in this project.
|
|
166
|
+
|
|
167
|
+
The trade-off this project cannot resolve: Space Bunny is served over
|
|
168
|
+
`/v1/chat/completions`, a weaker cache dialect than the `/v1/responses` route
|
|
169
|
+
Muse Spark uses, and it is an anonymous preview model with thin benchmark
|
|
170
|
+
coverage and no public identity. Zero-retention and a `max` effort tier were
|
|
171
|
+
judged worth those risks; that judgement is a default, not a fact, and
|
|
172
|
+
`oc-harness use` makes reversing it one flag.
|
|
173
|
+
|
|
174
|
+
## How the privacy list was assembled
|
|
175
|
+
|
|
176
|
+
From the Privacy section of the OpenCode Zen docs, which states a general
|
|
177
|
+
zero-retention policy and then names exceptions. Ten models are named as
|
|
178
|
+
retaining data. Two more, `x-preview-f-free` and `deepseek-v4-flash-free`, are
|
|
179
|
+
not named in the exception list and so fall under the general policy. Those two
|
|
180
|
+
are marked in `retention.yml` as inferred rather than quoted, because the
|
|
181
|
+
distinction matters if you are handling someone else's code.
|
|
182
|
+
|
|
183
|
+
## What the ecosystem already covers
|
|
184
|
+
|
|
185
|
+
Checked before writing any of the above, so the kit composes instead of
|
|
186
|
+
duplicating:
|
|
187
|
+
|
|
188
|
+
| need | existing project |
|
|
189
|
+
|---|---|
|
|
190
|
+
| model-driven context compression | `Tarquinen/opencode-dynamic-context-pruning` (4.3k★, AGPL-3.0, development moved to `sleev.ai`) |
|
|
191
|
+
| secret redaction before the LLM | `inkdust2021/opencode-vibeguard` |
|
|
192
|
+
| git worktrees | `kdcokenny/opencode-worktree` |
|
|
193
|
+
| persistent memory | `supermemoryai/opencode-supermemory` |
|
|
194
|
+
| memory in git markdown, model routing | `code-yeongyu/oh-my-opencode` (now `omo`) |
|
|
195
|
+
| bundled multi-agent harness | `kdcokenny/opencode-workspace` (16 components) |
|
|
196
|
+
| distributing skills and agents from git | `jgordijn/opencode-remote-config` |
|
|
197
|
+
| orchestration from slash commands | `spoons-and-mirrors/subtask2` |
|
|
198
|
+
|
|
199
|
+
The gap is the seam between a provider catalog and a config file. That is what
|
|
200
|
+
this kit is for.
|
|
201
|
+
|
|
202
|
+
## Sources
|
|
203
|
+
|
|
204
|
+
- arXiv:2609.00006 — Harness Engineering, eleven-harness source study
|
|
205
|
+
- `https://opencode.ai/docs/zen/` — pricing, retention, free-tier terms
|
|
206
|
+
- `https://opencode.ai/config.json` — the resolved config schema for 1.18.35
|
|
207
|
+
- `@opencode-ai/plugin@1.18.35` `dist/index.d.ts` — the hook surface (the
|
|
208
|
+
version actually installed; re-check against it after any plugin bump)
|
|
209
|
+
- `https://models.dev/api.json` — provider and model metadata
|
|
210
|
+
- `anomalyco/opencode` issue #42977 — free-route 429 behaviour
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$comment": "Contrarian example. Kept out of profiles/index.json on purpose: free, 1M context, and widely used, but its provider is permitted to train on your prompts and it has no `max` effort. `oc-harness list` shows it struck through with the reason. Install it with `oc-harness use muse-spark-1.3-contributor-free` and the warning is repeated on install.",
|
|
3
|
+
"id": "muse-spark-1.3-contributor-free",
|
|
4
|
+
"title": "Muse Spark 1.3 Contributor Free (OpenCode Zen)",
|
|
5
|
+
"provider": "opencode",
|
|
6
|
+
"model": "muse-spark-1.3-contributor-free",
|
|
7
|
+
"family": "muse-free",
|
|
8
|
+
"verified": "2026-10-07",
|
|
9
|
+
"retention": "may-train",
|
|
10
|
+
"notes": "Capability/privacy tradeoff, not a recommendation. Served over /v1/responses, a better cache dialect than chat/completions, and it supports temperature. But: 'Heavily discounted token pricing in exchange for permission to use your prompts and completions to train future Meta models', and its top effort is xhigh rather than max. Do not make this the default for private code.",
|
|
11
|
+
"limits": {
|
|
12
|
+
"context": 1048576,
|
|
13
|
+
"output": 131072
|
|
14
|
+
},
|
|
15
|
+
"supports": {
|
|
16
|
+
"reasoning": true,
|
|
17
|
+
"temperature": true,
|
|
18
|
+
"toolCall": true,
|
|
19
|
+
"structuredOutput": true,
|
|
20
|
+
"attachment": true
|
|
21
|
+
},
|
|
22
|
+
"effort": {
|
|
23
|
+
"levels": ["minimal", "low", "medium", "high", "xhigh"],
|
|
24
|
+
"default": "xhigh",
|
|
25
|
+
"reserveShare": 0.12
|
|
26
|
+
},
|
|
27
|
+
"roles": {
|
|
28
|
+
"plan": "max",
|
|
29
|
+
"build": "high",
|
|
30
|
+
"reviewer": "xhigh",
|
|
31
|
+
"locate": "low",
|
|
32
|
+
"verify": "low"
|
|
33
|
+
},
|
|
34
|
+
"prune": {
|
|
35
|
+
"enabled": true,
|
|
36
|
+
"keepRecentToolCalls": 6,
|
|
37
|
+
"keepRecentTurns": 3,
|
|
38
|
+
"keepUserMessages": true
|
|
39
|
+
},
|
|
40
|
+
"fallback": ["x-preview-f-free", "longcat-2.5-preview-free"]
|
|
41
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,79 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "opencode-harness-kit",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"
|
|
5
|
-
"
|
|
6
|
-
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "Model-agnostic harness kit for OpenCode: capability profiles, role-based reasoning effort, context pruning, and measured baselines.",
|
|
5
|
+
"license": "MIT",
|
|
6
|
+
"repository": {
|
|
7
|
+
"type": "git",
|
|
8
|
+
"url": "https://github.com/crytobot459/space-harness.git"
|
|
9
|
+
},
|
|
10
|
+
"type": "module",
|
|
11
|
+
"main": "./dist/index.js",
|
|
12
|
+
"module": "./dist/index.js",
|
|
13
|
+
"exports": {
|
|
14
|
+
".": {
|
|
15
|
+
"types": "./src/index.ts",
|
|
16
|
+
"import": "./dist/index.js"
|
|
17
|
+
},
|
|
18
|
+
"./profile": "./src/profile.js",
|
|
19
|
+
"./config": "./src/config.js",
|
|
20
|
+
"./rules": "./src/rules.js",
|
|
21
|
+
"./package.json": "./package.json"
|
|
22
|
+
},
|
|
23
|
+
"bin": {
|
|
24
|
+
"oc-harness": "bin/oc-harness.mjs"
|
|
25
|
+
},
|
|
26
|
+
"files": [
|
|
27
|
+
"bin/",
|
|
28
|
+
"dist/",
|
|
29
|
+
"src/",
|
|
30
|
+
"profiles/",
|
|
31
|
+
"examples/",
|
|
32
|
+
"retention.yml",
|
|
33
|
+
"GLOBAL.md",
|
|
34
|
+
"agents/",
|
|
35
|
+
"skills/",
|
|
36
|
+
"docs/",
|
|
37
|
+
"README.md",
|
|
38
|
+
"LICENSE"
|
|
39
|
+
],
|
|
40
|
+
"scripts": {
|
|
41
|
+
"build": "esbuild src/index.ts --bundle --platform=node --format=esm --outfile=dist/index.js",
|
|
42
|
+
"typecheck": "tsc --noEmit",
|
|
43
|
+
"test": "node --test \"test/*.test.js\"",
|
|
44
|
+
"test:all": "npm run build && npm run typecheck && npm test",
|
|
45
|
+
"test:install": "npm run measure:check && npm run attribute",
|
|
46
|
+
"check": "npm run test:all && npm run test:install && node bin/oc-harness.mjs verify",
|
|
47
|
+
"check:online": "npm run check && node bin/oc-harness.mjs doctor",
|
|
48
|
+
"baseline": "node bin/baseline.mjs",
|
|
49
|
+
"eval": "node eval/run.mjs",
|
|
50
|
+
"measure": "node eval/measure-install.mjs",
|
|
51
|
+
"measure:check": "node eval/measure-install.mjs --min-narrow 45 --min-ctx-drop 20",
|
|
52
|
+
"attribute": "node eval/attribute.mjs",
|
|
53
|
+
"attribute:json": "node eval/attribute.mjs --json",
|
|
54
|
+
"thresholds": "node eval/thresholds.mjs",
|
|
55
|
+
"prepublishOnly": "npm run build"
|
|
56
|
+
},
|
|
57
|
+
"keywords": [
|
|
58
|
+
"opencode",
|
|
59
|
+
"opencode-plugin",
|
|
60
|
+
"agent",
|
|
61
|
+
"harness",
|
|
62
|
+
"context-management",
|
|
63
|
+
"reasoning-effort"
|
|
64
|
+
],
|
|
65
|
+
"engines": {
|
|
66
|
+
"node": ">=22"
|
|
67
|
+
},
|
|
68
|
+
"peerDependencies": {
|
|
69
|
+
"@opencode-ai/plugin": ">=1.18.0"
|
|
70
|
+
},
|
|
71
|
+
"devDependencies": {
|
|
72
|
+
"@opencode-ai/plugin": "^1.18.31",
|
|
73
|
+
"@types/node": "^22.0.0",
|
|
74
|
+
"typescript": "^5.6.0"
|
|
75
|
+
},
|
|
76
|
+
"dependencies": {
|
|
77
|
+
"esbuild": "^0.28.2"
|
|
78
|
+
}
|
|
79
|
+
}
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$comment": "Fallback tier 3. The smallest context in the queue at 200k, which is why prune.keepRecentToolCalls drops: the same settings tuned for a 1M model overflow this one.",
|
|
3
|
+
"id": "deepseek-v4-flash-free",
|
|
4
|
+
"title": "DeepSeek V4 Flash Free (OpenCode Zen)",
|
|
5
|
+
"provider": "opencode",
|
|
6
|
+
"model": "deepseek-v4-flash-free",
|
|
7
|
+
"family": "deepseek-flash",
|
|
8
|
+
"verified": "2026-10-07",
|
|
9
|
+
"retention": "zero-retention",
|
|
10
|
+
"notes": "Text only, no attachment, 200k context. Included to show the compiler scaling pruning to the model: reserveShare and retention fall together, so a 1M-tuned profile cannot silently overflow a 200k model.",
|
|
11
|
+
"limits": {
|
|
12
|
+
"context": 200000,
|
|
13
|
+
"output": 128000
|
|
14
|
+
},
|
|
15
|
+
"supports": {
|
|
16
|
+
"reasoning": true,
|
|
17
|
+
"temperature": true,
|
|
18
|
+
"toolCall": true,
|
|
19
|
+
"structuredOutput": true,
|
|
20
|
+
"attachment": false
|
|
21
|
+
},
|
|
22
|
+
"effort": {
|
|
23
|
+
"levels": ["low", "high", "max"],
|
|
24
|
+
"default": "max",
|
|
25
|
+
"reserveShare": 0.12
|
|
26
|
+
},
|
|
27
|
+
"roles": {
|
|
28
|
+
"plan": "max",
|
|
29
|
+
"build": "high",
|
|
30
|
+
"reviewer": "max",
|
|
31
|
+
"locate": "low",
|
|
32
|
+
"verify": "low"
|
|
33
|
+
},
|
|
34
|
+
"prune": {
|
|
35
|
+
"enabled": true,
|
|
36
|
+
"keepRecentToolCalls": 4,
|
|
37
|
+
"keepRecentTurns": 2,
|
|
38
|
+
"keepUserMessages": true
|
|
39
|
+
},
|
|
40
|
+
"fallback": []
|
|
41
|
+
}
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$comment": "Profiles verified against live provider metadata. `retention` must come from retention.yml; a model missing there cannot enter this list.",
|
|
3
|
+
"default": "space-bunny-free",
|
|
4
|
+
"profiles": [
|
|
5
|
+
{
|
|
6
|
+
"id": "space-bunny-free",
|
|
7
|
+
"file": "profiles/space-bunny-free.json",
|
|
8
|
+
"title": "Space Bunny Free",
|
|
9
|
+
"retention": "zero-retention",
|
|
10
|
+
"free": true,
|
|
11
|
+
"topEffort": "max",
|
|
12
|
+
"context": 1048576,
|
|
13
|
+
"default": true,
|
|
14
|
+
"note": "Only free Zen model with a max effort and zero-retention terms."
|
|
15
|
+
},
|
|
16
|
+
{
|
|
17
|
+
"id": "x-preview-f-free",
|
|
18
|
+
"file": "profiles/x-preview-f-free.json",
|
|
19
|
+
"title": "x-preview-f Free",
|
|
20
|
+
"retention": "zero-retention",
|
|
21
|
+
"free": true,
|
|
22
|
+
"topEffort": "max",
|
|
23
|
+
"context": 1000000,
|
|
24
|
+
"default": false,
|
|
25
|
+
"note": "Three effort levels only; no medium or xhigh."
|
|
26
|
+
},
|
|
27
|
+
{
|
|
28
|
+
"id": "longcat-2.5-preview-free",
|
|
29
|
+
"file": "profiles/longcat-2.5-preview-free.json",
|
|
30
|
+
"title": "LongCat 2.5 Preview Free",
|
|
31
|
+
"retention": "zero-retention",
|
|
32
|
+
"free": true,
|
|
33
|
+
"topEffort": null,
|
|
34
|
+
"context": 1000000,
|
|
35
|
+
"default": false,
|
|
36
|
+
"note": "No tunable reasoning effort."
|
|
37
|
+
},
|
|
38
|
+
{
|
|
39
|
+
"id": "deepseek-v4-flash-free",
|
|
40
|
+
"file": "profiles/deepseek-v4-flash-free.json",
|
|
41
|
+
"title": "DeepSeek V4 Flash Free",
|
|
42
|
+
"retention": "zero-retention",
|
|
43
|
+
"free": true,
|
|
44
|
+
"topEffort": "max",
|
|
45
|
+
"context": 200000,
|
|
46
|
+
"default": false,
|
|
47
|
+
"note": "200k context, text only."
|
|
48
|
+
}
|
|
49
|
+
],
|
|
50
|
+
"excluded": [
|
|
51
|
+
{
|
|
52
|
+
"id": "muse-spark-1.3-contributor-free",
|
|
53
|
+
"file": "examples/muse-spark-1.3-contributor-free.json",
|
|
54
|
+
"title": "Muse Spark 1.3 Contributor Free",
|
|
55
|
+
"retention": "may-train",
|
|
56
|
+
"reason": "Provider may train on your prompts and completions (Meta contributor tier). Top effort is xhigh, not max.",
|
|
57
|
+
"installable": true
|
|
58
|
+
}
|
|
59
|
+
]
|
|
60
|
+
}
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$comment": "Fallback tier 2. No tunable reasoning effort at all, so the loader omits `variant` entirely for this profile rather than guessing a level.",
|
|
3
|
+
"id": "longcat-2.5-preview-free",
|
|
4
|
+
"title": "LongCat 2.5 Preview Free (OpenCode Zen)",
|
|
5
|
+
"provider": "opencode",
|
|
6
|
+
"model": "longcat-2.5-preview-free",
|
|
7
|
+
"family": "longcat",
|
|
8
|
+
"verified": "2026-10-07",
|
|
9
|
+
"retention": "zero-retention",
|
|
10
|
+
"notes": "Zero-retention with a 1M context, but reasoning_options is null upstream: the model has one fixed effort. Any role requesting a level resolves to null and the generated config leaves variant unset.",
|
|
11
|
+
"limits": {
|
|
12
|
+
"context": 1000000,
|
|
13
|
+
"output": 131072
|
|
14
|
+
},
|
|
15
|
+
"supports": {
|
|
16
|
+
"reasoning": true,
|
|
17
|
+
"temperature": true,
|
|
18
|
+
"toolCall": true,
|
|
19
|
+
"structuredOutput": false,
|
|
20
|
+
"attachment": true
|
|
21
|
+
},
|
|
22
|
+
"effort": {
|
|
23
|
+
"levels": [],
|
|
24
|
+
"default": null,
|
|
25
|
+
"reserveShare": 0.12
|
|
26
|
+
},
|
|
27
|
+
"roles": {
|
|
28
|
+
"plan": null,
|
|
29
|
+
"build": null,
|
|
30
|
+
"reviewer": null,
|
|
31
|
+
"locate": null,
|
|
32
|
+
"verify": null
|
|
33
|
+
},
|
|
34
|
+
"prune": {
|
|
35
|
+
"enabled": true,
|
|
36
|
+
"keepRecentToolCalls": 6,
|
|
37
|
+
"keepRecentTurns": 3,
|
|
38
|
+
"keepUserMessages": true
|
|
39
|
+
},
|
|
40
|
+
"fallback": ["deepseek-v4-flash-free"]
|
|
41
|
+
}
|