opencode-longrun-harness 1.2.22
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +390 -0
- package/docs/V1.2.20_EVIDENCE.md +114 -0
- package/docs/V1.2.21_EVIDENCE.md +68 -0
- package/docs/V1.2.22_EVIDENCE.md +52 -0
- package/harness/commissioning/README.md +16 -0
- package/harness/commissioning/inspect-copied-run.mjs +25 -0
- package/harness/commissioning/verify-copied-case.mjs +35 -0
- package/harness/plugin/longrun.js +677 -0
- package/harness/src/cli.mjs +40 -0
- package/harness/src/controller.js +1413 -0
- package/harness/src/evidence.mjs +135 -0
- package/harness/src/execution.mjs +217 -0
- package/harness/src/executor.mjs +21 -0
- package/harness/src/install.mjs +435 -0
- package/harness/src/maintenance.mjs +257 -0
- package/harness/src/memory.mjs +472 -0
- package/harness/test/candidates.test.mjs +73 -0
- package/harness/test/checkpoint.test.mjs +65 -0
- package/harness/test/controller.test.mjs +230 -0
- package/harness/test/evidence.test.mjs +57 -0
- package/harness/test/fixtures/durable-host.mjs +27 -0
- package/harness/test/fixtures/example-app-run.json +1375 -0
- package/harness/test/fixtures/notes-budget-exhausted-run.json +2070 -0
- package/harness/test/fixtures/notes-premature-complete-run.json +1496 -0
- package/harness/test/fixtures/notes-recovery-run.json +622 -0
- package/harness/test/fixtures/presets-readout-run.json +825 -0
- package/harness/test/fixtures/routing-worker.mjs +35 -0
- package/harness/test/fixtures/vitest-failed-receipt.json +33 -0
- package/harness/test/helper.mjs +41 -0
- package/harness/test/install.test.mjs +117 -0
- package/harness/test/lifecycle.test.mjs +102 -0
- package/harness/test/maintenance.test.mjs +204 -0
- package/harness/test/memory.test.mjs +145 -0
- package/harness/test/negative-control.test.mjs +91 -0
- package/harness/test/plugin.test.mjs +169 -0
- package/harness/test/recovery-runner.test.mjs +435 -0
- package/harness/test/recovery.test.mjs +68 -0
- package/harness/test/repair-mechanics.test.mjs +122 -0
- package/harness/test/toolbehavior.test.mjs +75 -0
- package/harness/test/v121-commissioning.test.mjs +177 -0
- package/harness/test/v1210-deadline.test.mjs +134 -0
- package/harness/test/v1211-pause.test.mjs +81 -0
- package/harness/test/v1212-maintenance-pause.test.mjs +76 -0
- package/harness/test/v1213-readout.test.mjs +82 -0
- package/harness/test/v1214-durable.test.mjs +121 -0
- package/harness/test/v1215-guidance.test.mjs +57 -0
- package/harness/test/v1216-test-summary.test.mjs +39 -0
- package/harness/test/v1217-discovery.test.mjs +73 -0
- package/harness/test/v1218-completion-review.test.mjs +203 -0
- package/harness/test/v1219-budget-pause.test.mjs +134 -0
- package/harness/test/v122-lifecycle-resolver.test.mjs +218 -0
- package/harness/test/v1220-budget-amendment.test.mjs +343 -0
- package/harness/test/v1221-negative-fixture-anchor.test.mjs +65 -0
- package/harness/test/v1222-default-evidence-class.test.mjs +75 -0
- package/harness/test/v123-plugin-e2e.test.mjs +120 -0
- package/harness/test/v123-receipt-model.test.mjs +185 -0
- package/harness/test/v124-canonical.test.mjs +147 -0
- package/harness/test/v124-installed.test.mjs +48 -0
- package/harness/test/v125-stability.test.mjs +183 -0
- package/harness/test/v126-execution.test.mjs +183 -0
- package/harness/test/v127-reconciliation.test.mjs +139 -0
- package/harness/test/v128-compaction.test.mjs +156 -0
- package/harness/test/v129-routing.test.mjs +165 -0
- package/harness/tools/audit-receipts.mjs +121 -0
- package/harness/tools/recovery-runner.mjs +499 -0
- package/package.json +49 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 shamusj-create
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,390 @@
|
|
|
1
|
+
# OpenCode Long-run Harness
|
|
2
|
+
|
|
3
|
+
**Bounded, evidence-driven long-run workflow infrastructure for [OpenCode](https://opencode.ai).**
|
|
4
|
+
|
|
5
|
+
A *tracked run* is driven by a **contract**: required criteria, each mapped to **declared checks** that
|
|
6
|
+
must actually execute. Progress is measured as **loss** (unverified required criteria). Every result is
|
|
7
|
+
recorded as a **receipt** bound to a **source fingerprint**, and a run can only reach `COMPLETE` when
|
|
8
|
+
every declared check passes on the *current* source **and** an independent **operator review** accepts
|
|
9
|
+
the evidence.
|
|
10
|
+
|
|
11
|
+
The controller is deliberately dependency-free — Node builtins only. No bundler, no second server, no
|
|
12
|
+
always-on daemon, no external service, no cloud model. It is **project-agnostic**: it drives any
|
|
13
|
+
full-stack repository through the same lifecycle, and the local toolchain (model id, CLI path, runtime)
|
|
14
|
+
is configuration rather than code — see [Tested setup](#tested-setup) for the environment it was
|
|
15
|
+
validated on.
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## Why use this instead of a vanilla harness
|
|
20
|
+
|
|
21
|
+
"Vanilla" — plain OpenCode plus a model, no contract layer — is perfectly good for short, interactive
|
|
22
|
+
work. It degrades in a specific, predictable way on long autonomous work: **you cannot tell the
|
|
23
|
+
difference between "finished" and "confidently described as finished".** This harness exists to make
|
|
24
|
+
that difference mechanical.
|
|
25
|
+
|
|
26
|
+
| Concern | Vanilla | This harness |
|
|
27
|
+
| --- | --- | --- |
|
|
28
|
+
| **What "done" means** | The model says so, or its own tests happen to pass | Every *declared* check passed on the frozen source **and** an operator accepted the evidence |
|
|
29
|
+
| **Evidence freshness** | Tests passed at some point; a later edit does not invalidate the claim | Receipts are bound to a source fingerprint — any tracked edit makes them `STALE` and they must be re-earned |
|
|
30
|
+
| **What gets verified** | Whatever command the model chose to run, at whatever moment | Only checks the contract *declared*, run by id; arbitrary commands are refused |
|
|
31
|
+
| **Self-approval** | The agent can declare its own success | The model **cannot** complete a run; completion needs an operator review bound to an evidence hash, and any later change invalidates it |
|
|
32
|
+
| **Long sessions** | Compaction loses the thread; work is silently redone, or claimed without being redone | Run state is canonical on disk: compaction moves the run to `RECOVERY_REQUIRED` and a supervised resume re-binds the session and worktree |
|
|
33
|
+
| **Runaway loops** | Unbounded retries, no memory of what already failed | Budgets (candidates, active check time, absolute deadline, command attempts, same-failure and no-progress limits) are enforced, not advisory |
|
|
34
|
+
| **False-positive tests** | An assertion can pass while the feature is unreachable | Negative controls on a *physically isolated copy* prove an assertion can actually fail, and the receipt auditor flags substituted commands and PASS-with-non-zero-exit |
|
|
35
|
+
| **Auditability** | A chat transcript | Append-only receipts and evidence, contract hash, source fingerprint, amendments, review basis |
|
|
36
|
+
| **Across projects** | Hand-rolled conventions per repo | One lifecycle, one CLI, one contract shape for any repository; the local toolchain is configurable |
|
|
37
|
+
|
|
38
|
+
**What it deliberately does not do.** It is workflow infrastructure, not a correctness guarantee and not
|
|
39
|
+
an OS sandbox: total host activity (model inference, ordinary tool calls) is *not* metered, budgets cover
|
|
40
|
+
declared-check execution, automatic continuation is **off** by default, compaction recovery is
|
|
41
|
+
**supervised** rather than autonomous, and it does not stop a determined process from touching state
|
|
42
|
+
files. It makes each failure mode above **explicit and checkable** — it does not make them impossible.
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## Features
|
|
47
|
+
|
|
48
|
+
- **Contract-bound runs.** Required criteria, each mapped to declared checks. A required criterion with
|
|
49
|
+
no satisfiable check is refused at *start*, so a run that could never reach loss 0 cannot be opened.
|
|
50
|
+
- **Loss as the progress signal.** Weighted count of unverified required criteria, recomputed from
|
|
51
|
+
evidence rather than asserted by the model.
|
|
52
|
+
- **Declared-checks-only verification.** Checks live in a catalogue; verification runs them by id.
|
|
53
|
+
Arbitrary shell commands are rejected, so "verification" cannot drift into a different command.
|
|
54
|
+
- **Fingerprint-bound receipts.** Every result records the source state it measured, plus status, exit
|
|
55
|
+
code, output tail, evidence class and the contract hash.
|
|
56
|
+
- **Automatic staleness.** A tracked-source change invalidates earlier receipts, so green evidence can
|
|
57
|
+
never be inherited by code that was never tested.
|
|
58
|
+
- **Hard gates.** A check marked as a gate blocks completion regardless of loss.
|
|
59
|
+
- **Enforced budgets.** Candidates, active check time, absolute deadline, command attempts, and
|
|
60
|
+
same-failure / no-progress thresholds.
|
|
61
|
+
- **Operator-only, append-only amendments.** Extra candidates or a new deadline for a paused run —
|
|
62
|
+
preserving the original limits, usage, receipts and history. There is no model-side amend action.
|
|
63
|
+
- **Independent completion review.** An accept/reject decision bound to a hash of the contract, source,
|
|
64
|
+
budgets and evidence. Any later change invalidates the approval.
|
|
65
|
+
- **Negative controls.** A run against a physically isolated copy with one deliberate defect, proving the
|
|
66
|
+
unchanged assertion detects it. Fixtures that overlap the real project are refused.
|
|
67
|
+
- **Compaction recovery.** Natural compaction parks the run; `resume-context` re-establishes state and an
|
|
68
|
+
explicit `resume` re-binds the new session and worktree to the *same* run.
|
|
69
|
+
- **Hierarchical memory.** `AGENTS.md` nodes where human prose is preserved and managed blocks are
|
|
70
|
+
regenerated; memory never overrides canonical lifecycle.
|
|
71
|
+
- **Reversible install.** Ownership-manifest based, JSONC-safe, and it never edits provider, model or
|
|
72
|
+
permission configuration. `uninstall` removes exactly what it installed.
|
|
73
|
+
- **Supervised dispatch runner.** Operator-side: discovers the live model endpoint from the running
|
|
74
|
+
server's own listening socket, verifies the single served model *before* every dispatch, resumes a
|
|
75
|
+
stuck run with bounded attempts, refuses to credit a no-op, and treats a terminal run as an ending
|
|
76
|
+
rather than a failure.
|
|
77
|
+
- **Independent receipt auditor.** A second implementation that re-reads receipts looking for substituted
|
|
78
|
+
commands, contract mismatch, PASS with a non-zero exit, backdating and freshness problems.
|
|
79
|
+
- **Project-agnostic by configuration.** Model id, provider model, OpenCode binary and the runtime
|
|
80
|
+
matcher all resolve from the environment (see [Configure for your project](#configure-for-your-project)).
|
|
81
|
+
|
|
82
|
+
---
|
|
83
|
+
|
|
84
|
+
## Usage guide
|
|
85
|
+
|
|
86
|
+
### 1. Install
|
|
87
|
+
|
|
88
|
+
```sh
|
|
89
|
+
npm install -g github:shamusj-create/opencode-longrun-harness
|
|
90
|
+
longrun-harness install
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
The first line pulls straight from this public repository — no npm account, no registry, no clone needed.
|
|
94
|
+
The second copies the plugin, the operator CLI, the agent, the commands and the skills into your OpenCode
|
|
95
|
+
config directory (`~/.config/opencode` by default), writing only Longrun-owned paths alongside an ownership
|
|
96
|
+
and rollback manifest. `longrun-harness dry-run` previews it; `longrun-harness uninstall` reverses it exactly;
|
|
97
|
+
`longrun-harness disable` / `enable` toggle it without removing anything; `--config-dir PATH` targets
|
|
98
|
+
somewhere else.
|
|
99
|
+
|
|
100
|
+
Then **restart OpenCode.** Plugins load at process start, so one installed into an already-running backend
|
|
101
|
+
is not active yet.
|
|
102
|
+
|
|
103
|
+
<details>
|
|
104
|
+
<summary>Other install paths</summary>
|
|
105
|
+
|
|
106
|
+
- **From a clone** (development): `npm run install:global`, i.e. `node harness/src/cli.mjs install`.
|
|
107
|
+
- **Manual or air-gapped**: copy `harness/plugin/longrun.js` to `~/.config/opencode/plugins/longrun.js`.
|
|
108
|
+
OpenCode auto-loads every file in that directory, so no `opencode.json` entry is required.
|
|
109
|
+
- **`opencode plugin <module>` is not a general-purpose installer** — it resolves npm *registry* packages,
|
|
110
|
+
and a `github:` specifier fails with `NpmInstallFailedError`. Use `longrun-harness install` instead.
|
|
111
|
+
|
|
112
|
+
</details>
|
|
113
|
+
|
|
114
|
+
### 2. Check the install
|
|
115
|
+
|
|
116
|
+
```sh
|
|
117
|
+
longrun-harness doctor # managed paths, manifest, local edits
|
|
118
|
+
~/.config/opencode/longrun-harness/longrun doctor --live # has a live host actually loaded it?
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
The two are deliberately different checks. The first confirms every managed path is present and tells you
|
|
122
|
+
which ones you have edited by hand — local edits are reported and preserved, never silently overwritten.
|
|
123
|
+
It is an ownership check, not a cryptographic one.
|
|
124
|
+
|
|
125
|
+
The second reads the load records the plugin writes and requires a live process, so before you restart it
|
|
126
|
+
reports `AWAITING_RESTART` / `NOT_VERIFIED` with the reason. It answers "is this *actually loaded*", not
|
|
127
|
+
"is this on disk" — a `NOT_VERIFIED` immediately after install is expected; a persistent one is not.
|
|
128
|
+
|
|
129
|
+
### 3. Declare a contract and start a run
|
|
130
|
+
|
|
131
|
+
The contract *is* the acceptance definition, so write it before work begins. Inside an OpenCode session
|
|
132
|
+
the model starts the run natively:
|
|
133
|
+
|
|
134
|
+
```jsonc
|
|
135
|
+
// longrun action=start
|
|
136
|
+
{
|
|
137
|
+
"request": "Add password reset to the accounts service",
|
|
138
|
+
"criteria": [
|
|
139
|
+
{ "id": "API", "required": true, "weight": 1, "checks": ["c-api-tests"], "evidenceClass": "INTEGRATION" },
|
|
140
|
+
{ "id": "UI", "required": true, "weight": 1, "checks": ["c-ui-e2e"], "evidenceClass": "BROWSER" },
|
|
141
|
+
{ "id": "ENG", "required": true, "weight": 1, "checks": ["c-typecheck", "c-build"] }
|
|
142
|
+
],
|
|
143
|
+
"checkCatalogue": {
|
|
144
|
+
"c-api-tests": { "command": ["npm", "run", "test:api"], "kind": "cmd", "timeoutMs": 300000 },
|
|
145
|
+
"c-ui-e2e": { "command": ["npx", "playwright", "test"], "kind": "cmd", "timeoutMs": 600000 },
|
|
146
|
+
"c-typecheck": { "command": ["npx", "tsc", "--noEmit"], "kind": "cmd", "timeoutMs": 180000 },
|
|
147
|
+
"c-build": { "command": ["npm", "run", "build"], "kind": "cmd", "timeoutMs": 300000 }
|
|
148
|
+
},
|
|
149
|
+
"budgets": { "candidateBudget": 40, "timeBudgetHours": 6, "deadlineHours": 24 },
|
|
150
|
+
"autoContinue": false
|
|
151
|
+
}
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
Rules worth knowing up front: criteria and checks are treated as **fixed** once the run starts; `loss`
|
|
155
|
+
falls only when a declared check records a PASS **on the current source**; and the model must never
|
|
156
|
+
edit code after recording a receipt without re-recording it.
|
|
157
|
+
|
|
158
|
+
### 4. Work, recording checks as they pass
|
|
159
|
+
|
|
160
|
+
Record **one check at a time**, as soon as it passes, so partial progress survives a compaction — and run
|
|
161
|
+
the broadest gate **last**, because any later edit invalidates every receipt recorded before it:
|
|
162
|
+
|
|
163
|
+
```
|
|
164
|
+
longrun_verify(runId=…, checkId="c-typecheck")
|
|
165
|
+
longrun_verify(runId=…, checkId="c-api-tests")
|
|
166
|
+
longrun_verify(runId=…, checkId="c-ui-e2e")
|
|
167
|
+
longrun_verify(runId=…, checkId="c-build") # or a single all-in-one gate last
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
### 5. Watch progress from the operator side
|
|
171
|
+
|
|
172
|
+
```sh
|
|
173
|
+
longrun status --json --project /path/to/repo --run lr-…
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
Shows canonical lifecycle, loss, per-criterion state, check status, budgets, stale evidence and review
|
|
177
|
+
state. `receipts` pages through history; `audit-receipts.mjs` audits it independently.
|
|
178
|
+
|
|
179
|
+
### 6. If a budget or deadline genuinely runs out
|
|
180
|
+
|
|
181
|
+
An operator (not the model) can grant more, append-only, on a paused run:
|
|
182
|
+
|
|
183
|
+
```sh
|
|
184
|
+
longrun amend --additional-candidates 20 --new-deadline 2026-09-29T12:00:00+01:00 \
|
|
185
|
+
--amendment-id op-amend-01 --expected-basis HASH --expected-revision N \
|
|
186
|
+
--authorization-file auth.txt --reason-file reason.txt
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
This preserves the original limits, usage, receipts and history, and never resumes the run.
|
|
190
|
+
|
|
191
|
+
### 7. Review, then complete
|
|
192
|
+
|
|
193
|
+
Completion is a two-key operation. When every declared check is PASS on one fingerprint, the operator
|
|
194
|
+
records the independent decision:
|
|
195
|
+
|
|
196
|
+
```sh
|
|
197
|
+
longrun review --verdict accept --expected-basis HASH \
|
|
198
|
+
--review-id review-20260926T2015Z --reason-file reason.txt
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
Only then can the model call `longrun action=complete`. Rejecting a premature completion leaves the run
|
|
202
|
+
paused with its history intact.
|
|
203
|
+
|
|
204
|
+
### 8. Optional: prove your tests can fail
|
|
205
|
+
|
|
206
|
+
For a materially risky assertion, run a **negative control**: copy the reviewed source to an isolated
|
|
207
|
+
fixture, introduce exactly one deliberate defect there, and confirm the *unchanged* assertion fails for
|
|
208
|
+
that defect. Fixtures that overlap the real project are refused, and a timeout/launch failure/zero tests
|
|
209
|
+
are classified as invalid execution rather than as the expected FAIL.
|
|
210
|
+
|
|
211
|
+
### Configure for your project
|
|
212
|
+
|
|
213
|
+
The harness is not tied to one local model setup. Everything below defaults to the environment it was
|
|
214
|
+
developed against, and is overridable:
|
|
215
|
+
|
|
216
|
+
| Variable | Meaning | Default |
|
|
217
|
+
| --- | --- | --- |
|
|
218
|
+
| `LONGRUN_REQUIRED_MODEL` | The single model id the served endpoint must expose | `mtplx-flash-next-optimized-speed` |
|
|
219
|
+
| `LONGRUN_REQUIRED_PROVIDER_MODEL` | The provider-qualified model passed to OpenCode | `mtplx/mtplx-flash-next-optimized-speed` |
|
|
220
|
+
| `LONGRUN_OPENCODE_BIN` | The OpenCode binary to dispatch | `/opt/homebrew/bin/opencode` |
|
|
221
|
+
| `LONGRUN_RUNTIME_MATCHER` | Regex identifying the model server's runtime when discovering its listening port | matches the bundled local runtime |
|
|
222
|
+
| `LONGRUN_MODEL_BASE` | Pin the inference base URL explicitly | discovered, then config, then default |
|
|
223
|
+
|
|
224
|
+
`harness/commissioning/` holds **case scripts** from one specific commissioning exercise — they contain
|
|
225
|
+
that case's numbers and are not generic tools.
|
|
226
|
+
|
|
227
|
+
---
|
|
228
|
+
|
|
229
|
+
## Why this exists
|
|
230
|
+
|
|
231
|
+
Long autonomous coding sessions fail in predictable ways:
|
|
232
|
+
|
|
233
|
+
- the agent claims success that its own tests do not support;
|
|
234
|
+
- evidence silently goes **stale** after a source change, while the check table still looks green;
|
|
235
|
+
- a run loops without progress, or burns its budget and keeps going;
|
|
236
|
+
- a model resumes, rewrites or self-approves a run it should not;
|
|
237
|
+
- a conversation is compacted and nobody notices what was actually verified.
|
|
238
|
+
|
|
239
|
+
This harness makes each of those failure modes explicit *and checkable*. It is workflow
|
|
240
|
+
infrastructure, not a correctness guarantee: it constrains and records what happened, and it refuses
|
|
241
|
+
to call work finished on weak evidence.
|
|
242
|
+
|
|
243
|
+
---
|
|
244
|
+
|
|
245
|
+
## Core concepts
|
|
246
|
+
|
|
247
|
+
| Concept | Meaning |
|
|
248
|
+
| --- | --- |
|
|
249
|
+
| **Tracked run** | One contract-bound unit of work with its own budgets, receipts, history and lifecycle. |
|
|
250
|
+
| **Contract** | Required criteria, each mapped to one or more declared checks. A required criterion with no satisfiable check cannot be created — a run that could never reach loss 0 is refused at start. |
|
|
251
|
+
| **Declared checks** | Named commands (`cmd`) declared in a catalogue. Verification runs *these*, by id; arbitrary commands are refused. |
|
|
252
|
+
| **Receipt** | The recorded result of one declared-check execution: status, exit code, output tail, evidence class, contract hash and the source fingerprint it measured. |
|
|
253
|
+
| **Evidence class** | `STATIC` / `UNIT` / `INTEGRATION` / `SYSTEM` / `BROWSER` / `VISION` / `HUMAN/EXTERNAL`, derived from the criterion the check maps to when not passed explicitly. |
|
|
254
|
+
| **Freshness** | A receipt only counts for the source state it measured. Any tracked-file change makes earlier receipts `STALE`, so evidence must be re-established on the frozen source. |
|
|
255
|
+
| **Loss** | Weighted count of required criteria that are not currently verified. `loss 0` is necessary but not sufficient for completion. |
|
|
256
|
+
| **Hard gates** | Checks a contract marks as gates. A failing gate blocks completion regardless of loss. |
|
|
257
|
+
| **Negative control** | A run against a *physically isolated copy* with one deliberate defect, proving an unchanged assertion actually detects that defect. Fixtures are refused if they overlap the real project. |
|
|
258
|
+
| **Lifecycle** | `READY → IMPLEMENTING → VERIFYING → REPAIRING / NEEDS_REPLAN → COMPACTING → RECOVERY_REQUIRED → … → PAUSED / BLOCKED / COMPLETE / CANCELLED`. Only canonical state reports lifecycle. |
|
|
259
|
+
| **Budgets** | Candidates, active check time, absolute deadline, command attempts, same-failure and no-progress limits. Enforced by the harness, not advisory. |
|
|
260
|
+
| **Amendment** | An **operator-only**, append-only grant of extra candidates and/or a new absolute deadline for an already-paused run. It never resumes a run, never resets counters, never grants acceptance, and has no native model tool action. |
|
|
261
|
+
| **Completion review** | A run cannot complete on green checks alone. An operator records an accept/reject bound to the current evidence basis; any later budget or source change invalidates it. |
|
|
262
|
+
| **Memory** | Hierarchical `AGENTS.md` nodes: curated human prose is preserved, managed blocks are regenerated. Memory never overrides canonical lifecycle. |
|
|
263
|
+
| **Recovery** | A natural compaction moves the run to `RECOVERY_REQUIRED`. `resume-context` re-establishes the state, then an explicit `resume` re-binds the session and worktree to the same run. |
|
|
264
|
+
|
|
265
|
+
---
|
|
266
|
+
|
|
267
|
+
## How it is built
|
|
268
|
+
|
|
269
|
+
| Piece | Path | Role |
|
|
270
|
+
| --- | --- | --- |
|
|
271
|
+
| Controller | `harness/src/controller.js` | Canonical state, contract, loss, receipts, views, review, amendment, memory. The single source of truth. |
|
|
272
|
+
| Execution | `harness/src/execution.mjs` | Declared-check execution, budgets, process groups, termination and timing. |
|
|
273
|
+
| Durable executor | `harness/src/executor.mjs` | Runs a check in a finite worker so an owned process is cleaned up and real evidence survives a host disconnect. Not a daemon. |
|
|
274
|
+
| OpenCode plugin | `harness/plugin/longrun.js` | Exposes the native `longrun` and `longrun_verify` tools, the lifecycle guard, routing and compaction handling. |
|
|
275
|
+
| Installer | `harness/src/install.mjs` | Reversible, ownership-manifest-based, JSONC-safe install. Never edits provider/model config. |
|
|
276
|
+
| Operator CLI | `harness/src/maintenance.mjs` | `doctor`, `status`, `pause`, `review`, `amend`, `disable`, `enable`, `uninstall`. |
|
|
277
|
+
| Recovery runner | `harness/tools/recovery-runner.mjs` | Operator-side supervised dispatch: discovers the live model endpoint, verifies the single served model before every dispatch, resumes a stuck run with bounded attempts, refuses to credit a no-op, settles a compaction-ended turn to a controlled `PAUSED`, treats a terminal run as an ending rather than a failure, and hands a run to a fresh reduced-context conversation. |
|
|
278
|
+
| Receipt auditor | `harness/tools/audit-receipts.mjs` | Independently audits receipts for substituted commands, contract mismatch, PASS-with-non-zero-exit, backdating and freshness. |
|
|
279
|
+
|
|
280
|
+
---
|
|
281
|
+
|
|
282
|
+
## Operator CLI
|
|
283
|
+
|
|
284
|
+
Installed as `longrun-harness/longrun` (a thin launcher over `harness/src/maintenance.mjs`):
|
|
285
|
+
|
|
286
|
+
| Command | Purpose |
|
|
287
|
+
| --- | --- |
|
|
288
|
+
| `doctor [--live]` | Self-test the installation and the resolution path. |
|
|
289
|
+
| `status --json --project DIR --run ID` | Canonical lifecycle, loss, criteria, checks, budgets, review state. |
|
|
290
|
+
| `pause --json --project DIR --run ID` | Canonical pause through the same writer lock as native pause. |
|
|
291
|
+
| `review --verdict accept\|reject --expected-basis HASH --review-id ID --reason-file FILE` | Record the independent completion review. |
|
|
292
|
+
| `amend --additional-candidates N --new-deadline ISO --expected-basis HASH --expected-revision R …` | Append-only budget/deadline grant for a paused run. |
|
|
293
|
+
| `disable` / `enable` / `uninstall` | Take the harness out of the loop, or remove it. |
|
|
294
|
+
|
|
295
|
+
## Native model tool surface
|
|
296
|
+
|
|
297
|
+
Inside OpenCode the model sees exactly two tools:
|
|
298
|
+
|
|
299
|
+
- `longrun` — `help`, `start`, `status`, `receipts`, `next` (resume-context), `checkpoint`, `verify`,
|
|
300
|
+
`pause`, `resume`, `complete`, `cancel`, `reconcile`, `memory_init`, `memory_refresh`, `memory_status`
|
|
301
|
+
- `longrun_verify` — run a *declared* check by id in `normal` or `negative` mode (negative mode takes
|
|
302
|
+
an isolated fixture)
|
|
303
|
+
|
|
304
|
+
There is no native amend action and no native self-approval. Ordinary execution, edits and memory
|
|
305
|
+
writes are stopped by the lifecycle guard in any non-eligible state.
|
|
306
|
+
|
|
307
|
+
## Test
|
|
308
|
+
|
|
309
|
+
```sh
|
|
310
|
+
npm test # node --test harness/test/*.test.mjs
|
|
311
|
+
```
|
|
312
|
+
|
|
313
|
+
**287 tests across 38 files, all passing.** These are offline tests against fixtures and mock
|
|
314
|
+
sessions: they are deliberately *not* treated as proof that a real OpenCode host behaves a certain
|
|
315
|
+
way, and they never touch production state.
|
|
316
|
+
|
|
317
|
+
---
|
|
318
|
+
|
|
319
|
+
## Verified status, honestly
|
|
320
|
+
|
|
321
|
+
- **Offline suite:** 287 passing tests covering identity keying, loss integrity, receipt eligibility
|
|
322
|
+
and staleness, single-flight scheduling, resume authorization, stall/replan/pause, budget
|
|
323
|
+
amendment, completion review, negative-control isolation, memory, endpoint discovery, configurable
|
|
324
|
+
toolchain resolution, and the recovery runner.
|
|
325
|
+
- **Real host behaviour** has been exercised in separate, dated commissioning work: lifecycle
|
|
326
|
+
transitions end-to-end, a real compaction with supervised recovery, and full-stack trial runs driven
|
|
327
|
+
to `COMPLETE`. Those are recorded in the release reports below rather than reproduced here.
|
|
328
|
+
- **Used in anger:** this harness drove six `COMPLETE` runs building a real browser game — mouse-only
|
|
329
|
+
interaction, smooth movement, board rotation and panning, ability targeting, audio, combat legibility
|
|
330
|
+
and effect work — each gated by its own declared checks on a frozen source fingerprint plus an
|
|
331
|
+
independent operator review. Several runs were extended only through the operator amendment path when
|
|
332
|
+
a budget or deadline genuinely ran out, and one was refused completion until an operator accepted the
|
|
333
|
+
evidence. The failures it caught included a test that was green while the feature it named was
|
|
334
|
+
unreachable, and a flaky acceptance gate that a lucky green pair would otherwise have hidden.
|
|
335
|
+
- **Known limits, by design:** total host activity (model inference, ordinary tools) is **not**
|
|
336
|
+
metered — budgets cover declared-check execution; automatic continuation is **OFF**; compaction
|
|
337
|
+
recovery is **supervised**, not autonomous; a run parked with no progress is stopped rather than
|
|
338
|
+
looping; and this is an auditable workflow boundary, **not** an OS sandbox against arbitrary
|
|
339
|
+
state-file access.
|
|
340
|
+
|
|
341
|
+
## Release reports
|
|
342
|
+
|
|
343
|
+
- [v1.2.22 — evidence-class derivation](docs/V1.2.22_EVIDENCE.md)
|
|
344
|
+
- [v1.2.21 — negative-fixture anchoring](docs/V1.2.21_EVIDENCE.md)
|
|
345
|
+
- [v1.2.20 — audited operator budget amendment](docs/V1.2.20_EVIDENCE.md)
|
|
346
|
+
|
|
347
|
+
## Tested setup
|
|
348
|
+
|
|
349
|
+
The harness is developed and driven against a **fully local** toolchain — no cloud model is involved at any
|
|
350
|
+
point. This is the configuration it has actually been exercised on:
|
|
351
|
+
|
|
352
|
+
| Component | Tested version | Notes |
|
|
353
|
+
| --- | --- | --- |
|
|
354
|
+
| macOS | 27.0, Apple Silicon | Desktop and CLI OpenCode |
|
|
355
|
+
| Node.js | 24.21.0 | Floor is `>= 22`; no third-party runtime dependencies |
|
|
356
|
+
| OpenCode | 1.18.32 | Both the CLI dispatch path and the desktop plugin host |
|
|
357
|
+
| MTPLX | 2.12.0 | Local OpenAI-compatible inference server (`com.youssofal.mtplx`) |
|
|
358
|
+
| Qwen model | `mtplx-flash-next-optimized-speed` | The one model id MTPLX serves — 262k context |
|
|
359
|
+
|
|
360
|
+
Put concretely: **MTPLX serves a single local Qwen model, and OpenCode is pointed at it as
|
|
361
|
+
`mtplx/mtplx-flash-next-optimized-speed`.** Every implementation, test and patch behind the release reports
|
|
362
|
+
was generated by that local model. The harness gates the *evidence*; the model that produced the code is
|
|
363
|
+
verified at dispatch.
|
|
364
|
+
|
|
365
|
+
Two defaults exist because this setup forced them, and both are worth knowing before you point the harness
|
|
366
|
+
at your own server:
|
|
367
|
+
|
|
368
|
+
- **The endpoint is discovered, not assumed.** A workstation can have more than one server on the usual
|
|
369
|
+
inference ports. During testing the expected port was held by a *different* runtime serving entirely
|
|
370
|
+
different models, while MTPLX listened elsewhere. A pinned port would have dispatched against the wrong
|
|
371
|
+
server — so the harness enumerates candidate listeners, asks each what it serves, and accepts only the
|
|
372
|
+
one exposing the required model id.
|
|
373
|
+
- **The served model is verified.** A mismatch is otherwise silent, so dispatch confirms that the endpoint
|
|
374
|
+
really serves the configured model id *and* that OpenCode is configured for the matching provider model.
|
|
375
|
+
A server that ignores the requested model is a genuine failure mode here, not a hypothetical one.
|
|
376
|
+
|
|
377
|
+
None of this is a requirement on you: MTPLX and Qwen are simply what this was validated with. The model id,
|
|
378
|
+
provider model, OpenCode binary and runtime matcher are all `LONGRUN_*` settings (see above), so any
|
|
379
|
+
OpenAI-compatible local server can take their place.
|
|
380
|
+
|
|
381
|
+
## Requirements
|
|
382
|
+
|
|
383
|
+
- **Node.js >= 22** (validated on 24.21.0) — Node builtins only, no third-party runtime dependencies.
|
|
384
|
+
- **OpenCode**, desktop or CLI, as the plugin host.
|
|
385
|
+
- **A local OpenAI-compatible inference server** exposing a single model id, with that model selected in
|
|
386
|
+
your OpenCode configuration. See [Tested setup](#tested-setup) for what this was validated against.
|
|
387
|
+
|
|
388
|
+
## License
|
|
389
|
+
|
|
390
|
+
[MIT](LICENSE) — use it, fork it, change it, ship it. Contributions and issues are welcome.
|
|
@@ -0,0 +1,114 @@
|
|
|
1
|
+
# v1.2.20 — audited operator-only budget amendment
|
|
2
|
+
|
|
3
|
+
The isolated annotations trial `lr-00000000a1b2` exhausted its original 24-candidate allowance with 37
|
|
4
|
+
genuine receipts/attempts, and its original absolute deadline (`2026-09-21T03:59:32.883Z`) had already
|
|
5
|
+
expired before any extension was requested. Independent review still rejects full-scope acceptance: the
|
|
6
|
+
last browser check genuinely failed reload/draft preservation, one later local-Qwen edit is unverified,
|
|
7
|
+
all six declared checks are STALE against the current source, and the remaining race/fog/cleanup/memory
|
|
8
|
+
requirements are unmet.
|
|
9
|
+
|
|
10
|
+
The user gave one consolidated authorization (goal round 1): **12 additional source candidates
|
|
11
|
+
(cumulative 36) and a new absolute deadline of `2026-09-22T23:30:00.000Z`** for the same run. This
|
|
12
|
+
release implements, reproduces, tests and installs the audited operator-only mechanism that records
|
|
13
|
+
such a grant without erasing the original limits, usage or failure history. It does not itself extend
|
|
14
|
+
anything until an operator applies it.
|
|
15
|
+
|
|
16
|
+
## What changed
|
|
17
|
+
|
|
18
|
+
- `harness/src/execution.mjs`: `originalTiming` (creation + declared duration, never rewritten),
|
|
19
|
+
`grantedDeadlineAt`, `effectiveIterations`, and an amendment-aware `timing`. For a run with **no**
|
|
20
|
+
grant the timing object is byte-for-byte the previous shape, so existing readouts/consumers are
|
|
21
|
+
unchanged; the original/granted split appears only once a grant exists. `budgetGuard` now refuses a
|
|
22
|
+
new source at the **effective** candidate limit.
|
|
23
|
+
- `harness/src/controller.js`: `AMENDMENT_SCHEMA_VERSION`, `budgetAmendmentStatus`,
|
|
24
|
+
`applyBudgetAmendment` and `operatorBudgetAmendment`. A grant is an **append-only** record placed in
|
|
25
|
+
`run.budgetAmendments`; `run.budget`, `createdAt`, `sourceFingerprint`, receipts, candidates and
|
|
26
|
+
`execution` are never modified. Only `budgetAmendments` and `controlGeneration` change.
|
|
27
|
+
- `completionReviewBasis` includes `budgetAmendments` **only when present**, so a grant invalidates a
|
|
28
|
+
prior completion approval while runs without grants keep their historical review basis exactly.
|
|
29
|
+
- `deriveRunView` reports `budgetLimit` (original vs effective), `controlRevision` and `amendmentBasis`;
|
|
30
|
+
the candidate string shows the effective limit (`24/36`). `buildRecoveryPacket` states the original +
|
|
31
|
+
authorized split and the original vs effective deadline.
|
|
32
|
+
- `harness/src/maintenance.mjs`: a new operator `amend` command. It is **not** in `RUN_ACTIONS` or
|
|
33
|
+
`ACTION_PARAMS`, so the native model tool surface has no amendment action. The plugin help now states
|
|
34
|
+
that amendment is operator-only and cannot be self-granted.
|
|
35
|
+
|
|
36
|
+
A grant binds the current canonical record and control revision (`--expected-basis`,
|
|
37
|
+
`--expected-revision`), so stale/conflicting grants are refused. It requires a paused run with no
|
|
38
|
+
in-flight check, a finite positive allowance (1..1000), an authorization and reason text (recorded
|
|
39
|
+
verbatim), a future deadline that strictly extends the current effective deadline, and a unique
|
|
40
|
+
`--amendment-id`. An exact repeat is an idempotent no-op; the same id with different content is a
|
|
41
|
+
conflict. A grant never resumes the run, never implies acceptance and never authorizes COMPLETE.
|
|
42
|
+
|
|
43
|
+
## Reproduction and regression
|
|
44
|
+
|
|
45
|
+
Before implementation, `harness/test/v1220-budget-amendment.test.mjs` failed 7 of its 8 tests against
|
|
46
|
+
the unchanged v1.2.19 code (only the pre-amendment refusal baseline passed). Log:
|
|
47
|
+
`docs/budget-amendment-evidence/before-tests.log`. Six focused areas are covered: real copied-state
|
|
48
|
+
preservation, original-vs-effective reporting, admission after a grant, refusal of invalid/stale/
|
|
49
|
+
conflicting/non-paused/in-flight requests without mutation, idempotency and cumulative grants, and
|
|
50
|
+
invalidation of a prior completion acceptance. The initial full-suite run exposed one genuine
|
|
51
|
+
interaction: `v1210-deadline.test.mjs` pins the exact `timing` shape, which the additive fields broke;
|
|
52
|
+
`timing` was changed to be shape-stable without a grant and all 254 regressions then passed.
|
|
53
|
+
|
|
54
|
+
Final logs: `docs/budget-amendment-evidence/regression-v1220-final.log` (254 tests, 0 failures).
|
|
55
|
+
These are offline deterministic tests, not native OpenCode host evidence.
|
|
56
|
+
|
|
57
|
+
## Installation and preservation
|
|
58
|
+
|
|
59
|
+
The existing reversible installer ran with no conflicts. Unique affected-path backup:
|
|
60
|
+
`/srv/example/.config/opencode-longrun-backups/v1.2.20-20260921T231504Z` (159 Longrun-owned files plus
|
|
61
|
+
`rollback-manifest.json`). Installed `current` = `1.2.20`; the installed controller and maintenance
|
|
62
|
+
entry point hash-match the source; the baked plugin URL points at
|
|
63
|
+
`releases/1.2.20/lib/controller.js`; source-independent `doctor` reports `ok:true`, executed controller
|
|
64
|
+
`1.2.20`, plugin shape `v1-server-plugin` and receipt model `ok`. `opencode.json`, `opencode.jsonc` and
|
|
65
|
+
`mtplx-session-headers.js` were byte-identical before/after, the unrelated config file count was
|
|
66
|
+
unchanged (3652), and the protected production run `lr-00000000e5f6` stayed byte-identical
|
|
67
|
+
(`f713f819…`). Evidence: `install-v1220.json`, `pre-install-config-hashes.txt`,
|
|
68
|
+
`pre-install-unrelated-count.txt`.
|
|
69
|
+
|
|
70
|
+
A copied install/rollback proof on the real 1.2.20 artifact (`copied-install-rollback-proof.json`)
|
|
71
|
+
installed into a temporary config with a foreign provider config and an unrelated file, ran the
|
|
72
|
+
installed release from an unrelated cwd, then uninstalled: all 18 installed files were recognized and
|
|
73
|
+
removed, `longrun-harness/` and `plugins/longrun.js` were gone, and the provider config plus unrelated
|
|
74
|
+
file were byte-preserved. Minor pre-existing cosmetic limitation: uninstall leaves the now-empty
|
|
75
|
+
`skills/longrun-*` directories behind (their files, including any foreign skill, are handled
|
|
76
|
+
correctly); this is unchanged from prior releases and was not treated as a new defect.
|
|
77
|
+
|
|
78
|
+
## Actual operator application
|
|
79
|
+
|
|
80
|
+
The installed operator CLI read the run's basis (`9359660e…`) and revision (`16`) through
|
|
81
|
+
`status --json`, then applied the authorized grant (`grant-annotations-20260921T2320Z`). Result:
|
|
82
|
+
effective limit 36, effective deadline `2026-09-22T23:30:00.000Z`, state PAUSED, usage untouched.
|
|
83
|
+
A structural diff of the canonical record before/after shows **only** `budgetAmendments` and
|
|
84
|
+
`controlGeneration` (16→17) changed: 24 counted candidates, 37 receipts, 1,625,576 measured
|
|
85
|
+
verification ms, creation time, contract hash, source fingerprint and the original budget
|
|
86
|
+
(24 / 7200s / 140 / expired deadline) are identical. An exact repeat returned `alreadyApplied`, and a
|
|
87
|
+
stale grant was refused with `AMENDMENT_BASIS_CHANGED`. Evidence:
|
|
88
|
+
`docs/budget-amendment-evidence/annotations-{amend-authorization.txt,amend-reason.txt,
|
|
89
|
+
run-before-amend.json,amend-result.json,status-before-amend.json}`.
|
|
90
|
+
|
|
91
|
+
## Host loading
|
|
92
|
+
|
|
93
|
+
Codex performed the authorized normal Desktop quit/relaunch. All seven old OpenCode processes exited
|
|
94
|
+
before relaunch. The fresh Desktop host (PID 13040) wrote a genuine `1.2.20` load record
|
|
95
|
+
(`desktop-load-v1220.json`) with a nonce, `toolsBuilt:true`, `test:false` and the real
|
|
96
|
+
`OpenCode Helper` executable. `doctor --live` still reports `STALE_LOAD_RECORD/NOT_VERIFIED` because
|
|
97
|
+
the fresh record has no hook activity yet (idle app) — that is the honest live rule, not a load
|
|
98
|
+
failure.
|
|
99
|
+
|
|
100
|
+
The amended continuation was then dispatched to the **same** session
|
|
101
|
+
`ses_f3f9f0822ffeWuypAPjXEvlUzm` through an actual OpenCode CLI host using the required local
|
|
102
|
+
`mtplx/mtplx-flash-next-optimized-speed` model (served model and selected session model verified
|
|
103
|
+
before dispatch). Evidence: `docs/annotations-continuation-evidence/`.
|
|
104
|
+
|
|
105
|
+
## Limits
|
|
106
|
+
|
|
107
|
+
- The amendment is an auditable workflow boundary, not an OS sandbox: it cannot stop an actor with
|
|
108
|
+
arbitrary state-file access. Like the completion review, the operator CLI is the only supported
|
|
109
|
+
path and models are instructed never to self-amend.
|
|
110
|
+
- A working-tree source change is orthogonal to a budget/deadline grant and stays unverified.
|
|
111
|
+
- The grant is finite; the original limits remain the historical record, and reaching the new limit
|
|
112
|
+
pauses the run again as before. No automatic continuation.
|
|
113
|
+
- This is harness evidence only. It is **not** annotations acceptance, not a negative control, not a
|
|
114
|
+
memory-outcome claim and not the Godot stage.
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# v1.2.21 — negative-fixture isolation must survive a filesystem-root worktree anchor
|
|
2
|
+
|
|
3
|
+
## Defect and reproduction
|
|
4
|
+
|
|
5
|
+
A real OpenCode CLI host supplied `context.worktree = "/"`. The `longrun_verify` negative-control path
|
|
6
|
+
passed that to `validateNegativeFixture` as a project directory, and the overlap test
|
|
7
|
+
(`inside(project, root)`) is true for `/`, so **every** fixture was refused with
|
|
8
|
+
`FIXTURE_NOT_ISOLATED / "fixture and production project overlap"`.
|
|
9
|
+
|
|
10
|
+
This was hit twice on the real annotations run: first with a fixture under the commissioning base
|
|
11
|
+
(`docs/annotations-continuation-evidence/negative-control-attempt-1-refused.json`), then with a clean
|
|
12
|
+
byte-identical copy under `/private/tmp` — the only common ancestor of both locations is `/`, which
|
|
13
|
+
identified the anchor. The existing negative-control tests never caught it because their context sets
|
|
14
|
+
`directory === worktree === project`, so the guard was never exercised with a broad anchor.
|
|
15
|
+
|
|
16
|
+
Reproduced before the fix with `harness/test/v1221-negative-fixture-anchor.test.mjs`: 2 of 4
|
|
17
|
+
assertions failed on unchanged v1.2.20 (`docs/negative-anchor-evidence/before-tests.log`) — the
|
|
18
|
+
root-anchor case and the missing offending-directory detail.
|
|
19
|
+
|
|
20
|
+
## Fix
|
|
21
|
+
|
|
22
|
+
`validateNegativeFixture` now skips a project directory that is a filesystem root
|
|
23
|
+
(`path.parse(project).root === project`), because a root contains every possible path and therefore
|
|
24
|
+
conveys no isolation information. Real nested/overlapping directories are still refused, and the
|
|
25
|
+
refusal detail now names the offending directory (`fixture … is inside project directory …` /
|
|
26
|
+
`fixture … contains project directory …`). No other behaviour changed; escaping symlinks, shared hard
|
|
27
|
+
links, special files and the entry budget are untouched.
|
|
28
|
+
|
|
29
|
+
## Regression and installation
|
|
30
|
+
|
|
31
|
+
All **258** harness regressions pass (254 prior + 4 new):
|
|
32
|
+
`docs/negative-anchor-evidence/regression-v1221.log`. Installed reversibly through the existing
|
|
33
|
+
installer with unique backup `/srv/example/.config/opencode-longrun-backups/v1.2.21-20260922T005647Z`
|
|
34
|
+
(167 owned files + `rollback-manifest.json`). Installed `current` = `1.2.21`; installed controller
|
|
35
|
+
hash-matches the source; the baked plugin URL points at `releases/1.2.21/lib/controller.js`;
|
|
36
|
+
source-independent `doctor` reports `ok:true`, executed controller `1.2.21`, receipt model `ok`, no
|
|
37
|
+
degraded items. `opencode.json`, `opencode.jsonc` and `mtplx-session-headers.js` were byte-identical
|
|
38
|
+
before/after and the protected production run stayed `f713f819…`. These are offline tests, not native
|
|
39
|
+
host evidence.
|
|
40
|
+
|
|
41
|
+
## Actual result on the real run
|
|
42
|
+
|
|
43
|
+
With v1.2.21 loaded by the actual OpenCode CLI host, the same fixture now validates as isolated
|
|
44
|
+
(`validateNegativeFixture(fixture, [project, "/"])` → `ok`, 6029 entries). The model then executed the
|
|
45
|
+
meaningful negative control: it had removed exactly one line in the fixture's
|
|
46
|
+
`packages/server/src/annotations.ts` `updateNote` — the optimistic-concurrency guard
|
|
47
|
+
`if (cur.revision !== input.revision) return 409 stale_revision` — with no test modified. The
|
|
48
|
+
**unchanged** server assertions failed for that exact behavioural defect:
|
|
49
|
+
|
|
50
|
+
- `create -> update -> delete round trip; revision is server-managed` — expected 409, received 200.
|
|
51
|
+
- `concurrency: two updates on one base revision -> exactly one wins, one is 409 (no lost update)` —
|
|
52
|
+
expected `[200,409]`, received `[200,200]`.
|
|
53
|
+
|
|
54
|
+
Recorded evidence: `kind: negative_control`, `mode: isolation`, `expected: FAIL`, `observed: FAIL`,
|
|
55
|
+
`ok: true`, `valid: true`, `mutatedProduction: false`, exit 1, no termination/error. Production
|
|
56
|
+
fingerprint before and after is identical (`1b9bb758…`), so the active project was not mutated; the
|
|
57
|
+
negative control consumed no candidate (30/36). No setup/import/timeout/cleanup error or masked
|
|
58
|
+
success occurred.
|
|
59
|
+
|
|
60
|
+
## Limits
|
|
61
|
+
|
|
62
|
+
- Offline regression tests and the actual CLI-host negative control are separate evidence levels;
|
|
63
|
+
neither is full-stack acceptance by itself.
|
|
64
|
+
- The fix makes the isolation guard ignore filesystem-root anchors only. A genuinely broad worktree
|
|
65
|
+
(e.g. the user's home) still counts as an overlap, which is intentional.
|
|
66
|
+
- Earlier natural compactions in this session each ended with the model stopping while
|
|
67
|
+
`RECOVERY_REQUIRED`, so supervised same-run dispatches were required; automatic continuation
|
|
68
|
+
remains OFF.
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# v1.2.22 — default evidence class for a check mapped from a criterion
|
|
2
|
+
|
|
3
|
+
## Defect and reproduction
|
|
4
|
+
|
|
5
|
+
A whole-suite final round calls `longrun_verify` once per declared check. `evidenceClass` is an
|
|
6
|
+
optional argument, and when it is omitted the receipt is recorded classless. Every criterion that maps
|
|
7
|
+
that check then resolves to `BLOCKED / UNKNOWN_CLASS`, so status reports `currentLoss 1` /
|
|
8
|
+
`required_unverified` **even though all six checks PASSED**. This happened on two real runs in a row:
|
|
9
|
+
the annotations final round (`lr-00000000a1b2`, fixed by an operator dispatch) and the Godot trial
|
|
10
|
+
final round (`lr-00000000c3d4`, reproduced here). In both cases the operator only detected it by
|
|
11
|
+
reading the canonical criterion diagnostics, not the green check table — exactly the "green checks are
|
|
12
|
+
not acceptance" trap the harness is meant to expose rather than create.
|
|
13
|
+
|
|
14
|
+
Reproduced before the fix with `harness/test/v1222-default-evidence-class.test.mjs`: 2 of 4 assertions
|
|
15
|
+
failed on unchanged v1.2.21 (`docs/evidence-class-evidence/before-tests.log`) — the omitted-class case
|
|
16
|
+
and the missing helper.
|
|
17
|
+
|
|
18
|
+
## Fix
|
|
19
|
+
|
|
20
|
+
`controller.defaultEvidenceClass(run, checkId, explicit)` returns the explicit argument when supplied,
|
|
21
|
+
otherwise the evidence class declared by the **single criterion that maps this check**. It returns
|
|
22
|
+
`null` when the check is unmapped or when criteria demand conflicting classes, so nothing is guessed.
|
|
23
|
+
`longrun_verify` now passes `C.defaultEvidenceClass(run, checkId, args.evidenceClass)` to the executor,
|
|
24
|
+
and the plugin help documents the behaviour. An explicit argument always wins; the check still has to
|
|
25
|
+
run and PASS, so this changes only how the receipt's strength is recorded, not whether evidence exists.
|
|
26
|
+
|
|
27
|
+
## Regression and installation
|
|
28
|
+
|
|
29
|
+
All **262** harness regressions pass (258 prior + 4 new):
|
|
30
|
+
`docs/evidence-class-evidence/regression-v1222.log`. Installed reversibly with backup
|
|
31
|
+
`/srv/example/.config/opencode-longrun-backups/v1.2.22-20260922T031958Z` (175 owned files +
|
|
32
|
+
`rollback-manifest.json`). Installed `current` = `1.2.22`; the installed controller hash-matches the
|
|
33
|
+
source; source-independent `doctor` reports `ok:true`, installed/executed `1.2.22`, receipt model `ok`,
|
|
34
|
+
no degraded items. `opencode.json`, `opencode.jsonc` and `mtplx-session-headers.js` were byte-identical
|
|
35
|
+
before/after and the protected production run stayed `f713f819…`. Offline tests, not native host
|
|
36
|
+
evidence.
|
|
37
|
+
|
|
38
|
+
## Actual host confirmation
|
|
39
|
+
|
|
40
|
+
The Godot session then re-ran the four criterion checks on its unchanged source `dcfdc4bae0` through
|
|
41
|
+
the installed 1.2.22 plugin, **omitting** `evidenceClass`. The receipts were recorded with the derived
|
|
42
|
+
classes (`c-godot-export` INTEGRATION, `c-godot-turret` BROWSER, `c-default-unit` UNIT,
|
|
43
|
+
`c-default-browser` BROWSER), no candidate was consumed, and status moved to `currentLoss 0`, all four
|
|
44
|
+
criteria SATISFIED, `declaredChecksReady true`, sole block `completion_review_required`. Evidence:
|
|
45
|
+
`docs/godot-cycle-evidence/godot-status-final.json`, `godot-finalize-events.jsonl`.
|
|
46
|
+
|
|
47
|
+
## Limits
|
|
48
|
+
|
|
49
|
+
- The derivation reads the run's own declared mapping, so it cannot exceed what the operator already
|
|
50
|
+
declared for that check; an ambiguous mapping still requires an explicit class.
|
|
51
|
+
- This is a recording fix. It does not make a failing check pass, does not weaken an evidence-strength
|
|
52
|
+
gate, and does not replace independent review or a negative control.
|