muse-crew 0.7.10 → 0.7.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/API.md +36 -14
- package/docs/guide.md +5 -5
- package/docs/ooda-report.md +150 -0
- package/docs/publish-verification.md +277 -88
- package/docs/visual-verdict.md +81 -67
- package/lib/AGENTS.md +8 -0
- package/lib/append-ooda-step.js +167 -0
- package/lib/build-readback-request.js +130 -0
- package/lib/compose-evidence-caption.js +141 -0
- package/lib/crew-api.js +467 -43
- package/lib/edit-image.py +216 -0
- package/lib/read-ooda-verdict.js +94 -0
- package/lib/readback-disk.js +186 -0
- package/lib/render-html.js +142 -0
- package/lib/see-act.js +327 -0
- package/lib/serve-artifact.js +203 -0
- package/lib/verify-publish.js +323 -0
- package/lib/write-ooda-verdict.js +147 -0
- package/package.json +1 -1
- package/seed/cron-body-template.md +30 -4
- package/seed/crons.json +1 -1
- package/workflows/bugfix.js +512 -220
- package/workflows/chore.js +389 -119
- package/workflows/crew-dispatch.js +72 -13
- package/workflows/docs.js +11 -2
- package/workflows/standard.js +413 -235
|
@@ -1,9 +1,26 @@
|
|
|
1
1
|
# Publish content verification — parent protocol
|
|
2
2
|
|
|
3
|
+
> **UNBLOCKED (2026-09-15):** the platform's `artifact_inspect` is still
|
|
4
|
+
> gone, but no platform tool is needed anymore. The platform's artifact
|
|
5
|
+
> edits land in its on-disk working copy of the artifact source
|
|
6
|
+
> (`~/workspace/ts-spaces/<slug>/` — verified empirically 2026-09-15:
|
|
7
|
+
> added lines present, removed lines absent across real platform commits),
|
|
8
|
+
> so `lib/readback-disk.js` performs the read-back deterministically: it
|
|
9
|
+
> reads the working copy and emits the exact machine-readable findings
|
|
10
|
+
> block `lib/verify-publish.js` already parses. No LLM, no async handoff,
|
|
11
|
+
> no prose to parse. The verifier is unchanged — the sensor changed, the
|
|
12
|
+
> judge didn't.
|
|
13
|
+
>
|
|
14
|
+
> Authority boundary: the sensor reads the platform's working copy — the
|
|
15
|
+
> tree the hosted artifact is built/served from. A working copy that is
|
|
16
|
+
> stale relative to a just-applied edit yields honest ABSENT findings and
|
|
17
|
+
> the verifier fails CLOSED (parked). Staleness can only park a task,
|
|
18
|
+
> never stamp provenance.
|
|
19
|
+
|
|
3
20
|
Provenance is the artifact's claim that its live content came from a specific
|
|
4
|
-
repo commit. The workflow
|
|
5
|
-
|
|
6
|
-
|
|
21
|
+
repo commit. The workflow never stamps it. This document is the parent-side
|
|
22
|
+
protocol. Deterministic code detects, claims, and certifies; the tick worker
|
|
23
|
+
(the live root agent) is only the async ferry for the inspection.
|
|
7
24
|
|
|
8
25
|
## Why the parent stamps
|
|
9
26
|
|
|
@@ -34,37 +51,132 @@ The contract is split on purpose:
|
|
|
34
51
|
|
|
35
52
|
- **Workflow-owned:** carrying the merged diff to the builder, the
|
|
36
53
|
applied-report observation (logged, never a park), the build-completion
|
|
37
|
-
poll, post-deploy cleanup,
|
|
38
|
-
read-back of the changed regions (carrying the observed builder build
|
|
39
|
-
identifier for correlation), recording the Publish session completed, and
|
|
54
|
+
poll, post-deploy cleanup, recording the Publish session completed, and
|
|
40
55
|
parking with `publish: verification-requested <commit>` instead of
|
|
41
|
-
stamping.
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
56
|
+
stamping. The workflow does NOT trigger the read-back inspection — an
|
|
57
|
+
async inspection triggered from inside a workflow run delivers its result
|
|
58
|
+
to the root agent, never back into the run, so a workflow-side trigger is
|
|
59
|
+
an orphan the verifier cannot consume. The parent triggers the one
|
|
60
|
+
inspection it can actually receive.
|
|
61
|
+
- **Parent-owned (deterministic code, ferried by the tick worker):**
|
|
62
|
+
scanning for verification-pending parks, atomically claiming them,
|
|
63
|
+
building the read-back request, triggering the inspection, waiting for the
|
|
64
|
+
result, comparing it mechanically against the merged diff, checking
|
|
65
|
+
build-ID correlation and supersession, stamping provenance only on a
|
|
66
|
+
match, reading the stamp back exactly, logging the terminal verdict, and
|
|
67
|
+
re-queuing the task to `in_progress`.
|
|
49
68
|
|
|
50
69
|
No artifact publish completes without parent-stamped provenance. A missing or
|
|
51
70
|
mismatched read-back never stamps.
|
|
52
71
|
|
|
72
|
+
## The carried diff: BASE..HEAD from the stamped provenance
|
|
73
|
+
|
|
74
|
+
The "merged diff" the workflow carries is `BASE..HEAD` where `BASE` is the
|
|
75
|
+
previously-stamped provenance `source_commit` — the artifact's actual
|
|
76
|
+
content — never `HEAD^1`. (Task `0c53af4e`, 2026-09-14: a push-time
|
|
77
|
+
reconcile merge put the task's own changes behind an intermediate merge, so
|
|
78
|
+
`HEAD^1..HEAD` carried only the reconcile delta and silently omitted the
|
|
79
|
+
task's fix; the artifact built without it. The stamped base is the only
|
|
80
|
+
ground truth for what the artifact already has; `BASE..HEAD` is the complete
|
|
81
|
+
unpublished delta.)
|
|
82
|
+
|
|
83
|
+
The workflow reads the base via `get-provenance` before computing the diff,
|
|
84
|
+
and the computation is guarded mechanically:
|
|
85
|
+
|
|
86
|
+
- Empty base (no provenance stamped) → the empty tree
|
|
87
|
+
`4b825dc642cb6eb9a060e54bf8d69288fbee4904`, and only then. A present but
|
|
88
|
+
malformed base SHA parks fail-closed.
|
|
89
|
+
- `git merge-base --is-ancestor BASE HEAD` must pass; a non-ancestor base
|
|
90
|
+
parks fail-closed (the stamped provenance must lead to the integrated
|
|
91
|
+
commit, otherwise the artifact has drifted or the stamp is wrong).
|
|
92
|
+
- The agent-reported base must equal the stamped base; a mismatch parks.
|
|
93
|
+
- The expected base content hashes (pre-publish observation) are computed at
|
|
94
|
+
the stamped base, not the merge parent — the artifact's tree should match
|
|
95
|
+
the stamp, and the observation is only meaningful against it.
|
|
96
|
+
|
|
97
|
+
The parent verifier (`lib/verify-publish.js`, `--base`) and the read-back
|
|
98
|
+
request builder (`lib/build-readback-request.js`, `--base`) use the identical
|
|
99
|
+
base: the previously-stamped provenance, or the empty tree for a genuine
|
|
100
|
+
first publish. Request builder and verifier never disagree on the base.
|
|
101
|
+
|
|
102
|
+
## The shape: code detects, the tick ferries, code certifies
|
|
103
|
+
|
|
104
|
+
A standalone verification workflow cannot work with the async inspection
|
|
105
|
+
model: async inspection results are delivered to the root agent of the
|
|
106
|
+
agent tree, never into a workflow run — so a verify workflow would wait
|
|
107
|
+
forever for a result it can never receive. The tick worker IS the live
|
|
108
|
+
root agent, so it is the only component that can both trigger an
|
|
109
|
+
inspection and receive its result. (The inspection tool itself,
|
|
110
|
+
`artifact_inspect`, was removed by the platform 2026-09-14 — see the
|
|
111
|
+
BLOCKED notice at the top.)
|
|
112
|
+
|
|
113
|
+
But the tick worker is a generalist LLM, and the certification decision is
|
|
114
|
+
safety-critical: a misjudged "match" stamps unverified content, and nothing
|
|
115
|
+
downstream can ever detect it (QA checks the stamp, not the content). So the
|
|
116
|
+
LLM never judges. The division:
|
|
117
|
+
|
|
118
|
+
1. **Scan (code):** `scan-verification-pending` finds parked tasks whose
|
|
119
|
+
latest parent note is `publish: verification-requested`, with no terminal
|
|
120
|
+
verdict and no unexpired claim. It atomically claims each one by logging
|
|
121
|
+
`publish: verification-claimed <expiry>` (1-hour lease) — the task stays
|
|
122
|
+
parked, so the dispatcher never dispatches QA mid-verification, and a
|
|
123
|
+
second tick cannot start a duplicate verification. It also reconciles the
|
|
124
|
+
verified-but-still-parked gap (verdict recorded, re-queue lost to a crash)
|
|
125
|
+
back to `in_progress`.
|
|
126
|
+
2. **Build (code):** `build-readback-request.js` builds the EXACT inspection
|
|
127
|
+
request from the publish delta — `git diff <base> <commit>` where
|
|
128
|
+
`<base>` is the previously-stamped provenance `source_commit` (or the
|
|
129
|
+
empty tree for a first publish). The tick never hand-writes the request,
|
|
130
|
+
and never uses `commit^1` as the base: push-time reconcile merges put
|
|
131
|
+
the task's own changes behind an intermediate merge, so `commit^1`
|
|
132
|
+
covers only the reconcile delta (2026-09-14, task `0c53af4e`).
|
|
133
|
+
3. **Ferry (tick worker):** runs the deterministic sensor
|
|
134
|
+
`lib/readback-disk.js` (`--repo-path`, `--commit`, `--base` — the same
|
|
135
|
+
base as step 2 — `--slug` from the project's `deploy_slug`, `--task-id`)
|
|
136
|
+
and saves its stdout to the result file. The sensor exits 0 only when it
|
|
137
|
+
actually read the working copy; on a non-zero exit the tick must NOT
|
|
138
|
+
save stdout — log `publish: verification-procedural-error <commit>
|
|
139
|
+
<stderr>` and leave the task parked for the next tick to retry (a sensor
|
|
140
|
+
failure is procedural — the read could not be performed — not a content
|
|
141
|
+
verdict). `build-readback-request.js` is retained for the manual LLM
|
|
142
|
+
fallback below.
|
|
143
|
+
4. **Certify (code):** `verify-publish.js` parses the inspector's
|
|
144
|
+
machine-readable findings block, compares every added/removed diff line
|
|
145
|
+
against the reported present/absent verdicts, checks build-ID correlation
|
|
146
|
+
and supersession via git, and only then stamps provenance, reads the
|
|
147
|
+
stamp back exactly, logs the terminal verdict, and re-queues to
|
|
148
|
+
`in_progress`. Unparseable findings, mismatches, supersession, and stamp
|
|
149
|
+
failures all fail CLOSED with a terminal `publish: verification-failed`
|
|
150
|
+
verdict — never a stamp.
|
|
151
|
+
- **Envelope:** the tick saves the COMPLETE handoff — the full prose
|
|
152
|
+
report AND the full JSON result, both verbatim (raw prose, JSON, or
|
|
153
|
+
both concatenated are all accepted). Observed 2026-09-14: the
|
|
154
|
+
platform's JSON envelope carries NO machine-readable findings
|
|
155
|
+
block; the block lives in the prose handoff. The verifier locates
|
|
156
|
+
the findings block in prose text and JSON string values (including
|
|
157
|
+
double-encoded ones) and prefers the block whose file paths cover
|
|
158
|
+
the expected diff — an echoed request template or stray prose never
|
|
159
|
+
outranks the real block. No covering block => `unreadable-result`,
|
|
160
|
+
fail closed. Saving JSON-only strands verification.
|
|
161
|
+
fail closed.
|
|
162
|
+
- **Release identity:** `scan-verification-pending` resolves
|
|
163
|
+
`crew_release` through the crew home's `current` symlink (the immutable
|
|
164
|
+
active release) and cross-checks it against the running code's own
|
|
165
|
+
realpath. Unresolvable or disputed => the scan throws fail-closed
|
|
166
|
+
BEFORE writing any claim — provenance is never stamped `unknown`, and
|
|
167
|
+
a stale cron body running an old release cannot certify.
|
|
168
|
+
|
|
169
|
+
The tick body (seed/cron-body-template.md, step 4.5) wires these together.
|
|
170
|
+
The publisher never certifies itself, and the LLM never makes the
|
|
171
|
+
safety-critical match decision.
|
|
172
|
+
|
|
53
173
|
## The park
|
|
54
174
|
|
|
55
175
|
When the artifact build lands, the workflow parks the task with the message:
|
|
56
176
|
|
|
57
177
|
```
|
|
58
178
|
publish: verification-requested <commit> (build <agent_id|agent_id unobserved>) — artifact build landed, post-deploy
|
|
59
|
-
finalized, provenance NOT stamped. Parent: run docs/publish-verification.md
|
|
60
|
-
(content read-back inspection <inspection_id> already triggered).
|
|
61
|
-
```
|
|
62
|
-
|
|
63
|
-
or, if the workflow's inspect trigger failed:
|
|
64
|
-
|
|
65
|
-
```
|
|
66
|
-
publish: verification-requested <commit> (build <agent_id|agent_id unobserved>) — ... (read-back inspect trigger
|
|
67
|
-
failed: <reason> — parent: trigger artifact_inspect manually).
|
|
179
|
+
finalized, provenance NOT stamped. Parent: run docs/publish-verification.md.
|
|
68
180
|
```
|
|
69
181
|
|
|
70
182
|
The parked message is stored as `Parked: publish: verification-requested
|
|
@@ -80,71 +192,95 @@ task holds no resources.
|
|
|
80
192
|
|
|
81
193
|
## Parent verification procedure
|
|
82
194
|
|
|
195
|
+
The automated path is the tick body's step 4.5 (scan → build → ferry →
|
|
196
|
+
verify). The manual fallback below is the same protocol run by hand; it
|
|
197
|
+
exists for when the artifact namespace is unavailable to the tick worker.
|
|
198
|
+
|
|
83
199
|
For a task parked with `publish: verification-requested <commit>`:
|
|
84
200
|
|
|
85
201
|
1. **Resolve the project.** Read the task's project via the Crew API
|
|
86
|
-
(`
|
|
202
|
+
(`get-project`); you need `repo_path` (the git checkout) and the artifact
|
|
87
203
|
slug (the project's publish target).
|
|
88
|
-
2. **Expected change.**
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
204
|
+
2. **Expected change.** The publish delta is `git diff <base> <commit>`
|
|
205
|
+
in `repo_path`, where `<base>` is the previously-stamped provenance
|
|
206
|
+
`source_commit` (read it via `get-provenance`; use the empty-tree sha
|
|
207
|
+
`4b825dc642cb6eb9a060e54bf8d69288fbee4904` when no provenance is
|
|
208
|
+
stamped yet — a first publish). Never use `commit^1` as the base and
|
|
209
|
+
never take the expected change from the builder's report: push-time
|
|
210
|
+
reconcile merges violate the `merge^1 == previously-published tree`
|
|
211
|
+
invariant, so `commit^1..commit` can omit the task's own fix
|
|
212
|
+
(2026-09-14, task `0c53af4e`).
|
|
213
|
+
3. **Actual content.** Run `lib/readback-disk.js` with `--repo-path`,
|
|
214
|
+
`--commit`, `--base` (the same base as step 2), `--slug`, and
|
|
215
|
+
`--task-id`, and save its stdout to the result file — this is the
|
|
216
|
+
deterministic read-back; it emits the machine-readable findings block
|
|
217
|
+
directly. (LLM fallback: if the disk working copy is unavailable, call
|
|
218
|
+
the artifact inspector with `repair_authorized: false` and the
|
|
219
|
+
`verbatim_request` built by `lib/build-readback-request.js`, passing
|
|
220
|
+
`--build-agent-id` from the park message's `(build …)` suffix when it
|
|
221
|
+
is not `agent_id unobserved`.) The findings block grammar:
|
|
222
|
+
```
|
|
223
|
+
FILE: <path>
|
|
224
|
+
ADDED: <exact added line> :: PRESENT|ABSENT
|
|
225
|
+
REMOVED: <exact removed line> :: PRESENT|ABSENT
|
|
226
|
+
END_FILE
|
|
227
|
+
```
|
|
228
|
+
If no read-back can be obtained at all, log
|
|
229
|
+
`publish: verification-blocked <commit> <reason>` and leave the task
|
|
230
|
+
parked for human attention. Never stamp without a read-back.
|
|
106
231
|
4. **Compare mechanically.** For every added (`+`) line in the diff, the
|
|
107
|
-
read-back must report it PRESENT in the
|
|
108
|
-
every removed (`-`) line,
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
232
|
+
read-back's machine-readable block must report it PRESENT in the
|
|
233
|
+
artifact's current source. For every removed (`-`) line, it must report
|
|
234
|
+
it ABSENT — with one mechanical exemption: a removed line that also
|
|
235
|
+
occurs verbatim in untouched code has zero discriminating power (its
|
|
236
|
+
presence proves nothing about whether the old block survived), so the
|
|
237
|
+
verifier exempts it instead of failing a good publish. The exemption is
|
|
238
|
+
computed, never judged: a removed line L in file F is exempt iff L
|
|
239
|
+
occurs in F's old tree (at `<base>`) strictly more times than the diff
|
|
240
|
+
removes it (2026-09-15, task `00bca4b8` — a valid publish parked because
|
|
241
|
+
two removed lines occurred identically in the untouched WorkflowSteps
|
|
242
|
+
component). The comparison is computed by `lib/verify-publish.js` —
|
|
243
|
+
never by eyeballing prose. A missing or malformed findings block fails
|
|
244
|
+
closed as `unreadable-result`, never as a pass.
|
|
245
|
+
4b. **Build-ID correlation.** The read-back may have inspected a different
|
|
246
|
+
build's output than this publish attempt's:
|
|
112
247
|
1. **Expected** = the agent_id in the park message's `(build …)` suffix.
|
|
113
248
|
If the suffix says `agent_id unobserved`, look up the workflow's
|
|
114
249
|
durable publish ledger at `$CREW_HOME/.publish-ledger/<slug>.jsonl`
|
|
115
250
|
for the `submitted` entry with this `<commit>` and use its
|
|
116
251
|
`agent_id` field (it may still be null — then this step is vacuous).
|
|
117
|
-
2. **Live** = the
|
|
118
|
-
|
|
119
|
-
that
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
4. Otherwise the content match from step 4 decides
|
|
127
|
-
|
|
128
|
-
verified` note.
|
|
252
|
+
2. **Live** = whether the expected agent_id appears anywhere in the
|
|
253
|
+
read-back result (the live artifact status exposes no durable
|
|
254
|
+
agent_id — only an in-flight correlation ID that expires with the
|
|
255
|
+
publish attempt, so absence is the common case, not evidence of a
|
|
256
|
+
mismatch).
|
|
257
|
+
3. If expected is non-null and the read-back positively reports a
|
|
258
|
+
DIFFERENT live build identity for this attempt's output, log
|
|
259
|
+
`publish: build-mismatch <commit> expected <expected> observed
|
|
260
|
+
<live>`, stay parked, never stamp, never re-queue.
|
|
261
|
+
4. Otherwise the content match from step 4 decides — the stamp certifies
|
|
262
|
+
CONTENT, not the builder's identity. Log the correlation outcome
|
|
263
|
+
(correlated / unobserved) in the `publish: verified` note.
|
|
129
264
|
5. **Supersession check.** Before stamping, prove the inspected live
|
|
130
|
-
artifact still represents the commit being certified
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
the
|
|
137
|
-
6. **Stamp, verify the stamp, then re-queue:**
|
|
265
|
+
artifact still represents the commit being certified: `git rev-parse
|
|
266
|
+
HEAD` in `repo_path` must equal `<commit>`. If HEAD has moved (a later
|
|
267
|
+
Publish landed), the read-back is stale — log
|
|
268
|
+
`publish: superseded <commit> by <head>` and leave the task parked for
|
|
269
|
+
human attention. Never stamp a superseded commit.
|
|
270
|
+
6. **Stamp, verify the stamp, then re-queue** (all in `lib/verify-publish.js`;
|
|
271
|
+
the manual equivalent):
|
|
138
272
|
- **Match** — stamp provenance with the Crew API CLI `set-provenance`
|
|
139
273
|
(the crew-owned store). Do NOT use the artifact's `setprovenance`
|
|
140
274
|
action — it writes a different, non-authoritative store that QA never
|
|
141
275
|
reads, so the stamp would be invisible to every gate:
|
|
142
|
-
`set-provenance --json '{"source_commit":"<commit>","crew_release":"<
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
276
|
+
`set-provenance --json '{"source_commit":"<commit>","crew_release":"<release>","task_id":"<task>"}'`
|
|
277
|
+
(crew_release is the basename of the active release, e.g.
|
|
278
|
+
`pkg-0.7.10`). Then read the stamp back with `get-provenance`
|
|
279
|
+
and confirm source_commit, crew_release, and task_id match exactly
|
|
280
|
+
what was sent — a stamp that cannot be read back is not a stamp. Only
|
|
281
|
+
then log the task note event
|
|
282
|
+
`publish: verified <commit> (<inspection_id>)` and re-queue with
|
|
283
|
+
`update-task` → state `in_progress` (never `todo` — `todo`
|
|
148
284
|
restarts Triage and resets retry accounting). The dispatcher resumes
|
|
149
285
|
at QA from the completed Publish session (standard/bugfix); chore has
|
|
150
286
|
no QA — it proceeds to terminal completion. QA's provenance check
|
|
@@ -161,6 +297,61 @@ For a task parked with `publish: verification-requested <commit>`:
|
|
|
161
297
|
for human attention. Never re-queue an unstamped-but-verified task into
|
|
162
298
|
QA — QA would fail it and burn rework budget on a stamping problem.
|
|
163
299
|
|
|
300
|
+
## Crash recovery
|
|
301
|
+
|
|
302
|
+
- **Tick dies before triggering the inspection:** the claim expires after
|
|
303
|
+
1 hour; the next scan re-claims and re-verifies from scratch. The stamp
|
|
304
|
+
is an idempotent upsert, so a duplicate verification cannot corrupt it.
|
|
305
|
+
- **Tick dies after the inspection but before the stamp:** same as above —
|
|
306
|
+
the next scan re-runs the whole verification (new inspection, new
|
|
307
|
+
comparison). Wasteful but correct.
|
|
308
|
+
- **Crash between stamp and re-queue:** the next scan sees
|
|
309
|
+
`publish: verified` on a still-parked task and reconciles it to
|
|
310
|
+
`in_progress`. Failure verdicts are never reconciled — they stay parked
|
|
311
|
+
for human attention.
|
|
312
|
+
- **Two ticks verify concurrently:** impossible — the atomic claim means the
|
|
313
|
+
second scan sees the unexpired `publish: verification-claimed` note and
|
|
314
|
+
skips. The lease expiry bounds the damage if a claimer dies.
|
|
315
|
+
|
|
316
|
+
## Unknown-outcome recovery (2026-09-14, Gate 1 Journey 3 attempt 7)
|
|
317
|
+
|
|
318
|
+
Attempt 7 parked at Publish with outcome `unknown`: the rebuild trigger's
|
|
319
|
+
child failed structured closeout and the in-flight-only build-state poll
|
|
320
|
+
could not see the completed build — even though the build HAD run (a fresh
|
|
321
|
+
platform audit directory existed). Two mechanisms close this gap.
|
|
322
|
+
|
|
323
|
+
**1. Workflow-side durable evidence.** Before the rebuild trigger, the
|
|
324
|
+
workflow snapshots the artifact's audit-directory listing
|
|
325
|
+
(`~/workspace/ts-spaces/<slug>/audits/` — best-effort, never a gate). On a
|
|
326
|
+
structured-output failure with no in-flight build observed, it re-lists and
|
|
327
|
+
diffs: a timestamped directory that appeared during the trigger window is
|
|
328
|
+
positive evidence the edit went through and the build completed. The
|
|
329
|
+
fallback never re-issues the edit, never stamps provenance, and only routes
|
|
330
|
+
to the parent's independent content read-back. No new directory still parks
|
|
331
|
+
`unknown` fail-closed. The ledger distinguishes the two confirmations: `edit
|
|
332
|
+
confirmed via durable audit evidence …` vs `edit confirmed via build-state
|
|
333
|
+
poll …`.
|
|
334
|
+
|
|
335
|
+
The fallback's known limitation: audit directories are not attributed to
|
|
336
|
+
tasks, so two concurrent publishes to the same artifact could cross-read.
|
|
337
|
+
The consequence is bounded — the fallback only routes to the parent
|
|
338
|
+
read-back, and the parent still certifies the exact commit's content
|
|
339
|
+
mechanically (a wrong build's content fails closed as `publish:
|
|
340
|
+
content-mismatch` / `publish: build-mismatch`, never stamps).
|
|
341
|
+
|
|
342
|
+
**2. `resolve-publish-unknown` (Crew API).** For attempts already parked
|
|
343
|
+
`unknown` before this fix: given a task parked with a latest ledger outcome
|
|
344
|
+
of `unknown`, it derives the publish window (Integrate-completion event →
|
|
345
|
+
unknown-outcome park event) and checks for a timestamped audit build inside
|
|
346
|
+
that window. On evidence, it appends `unknown-resolved` to the ledger
|
|
347
|
+
(never rewriting the original entry), writes `publish: unknown-resolved`
|
|
348
|
+
and a mirrored `publish: verification-requested <commit>` note (the mirror
|
|
349
|
+
is timestamped strictly later so the scan sees it as the latest), and leaves
|
|
350
|
+
the task parked for the normal scan. Still-unknown cases stay parked:
|
|
351
|
+
unparked task, non-`unknown` latest ledger outcome, missing commit, no audit
|
|
352
|
+
build in the window, unobservable window, or an already-resolved attempt
|
|
353
|
+
(idempotent).
|
|
354
|
+
|
|
164
355
|
## Exact note-event prefixes
|
|
165
356
|
|
|
166
357
|
Case-sensitive, exact-prefix matches — match on prefixes, never on English
|
|
@@ -168,20 +359,29 @@ meaning:
|
|
|
168
359
|
|
|
169
360
|
- `publish: verification-requested <commit>` — workflow park; contained in
|
|
170
361
|
the stored `Parked: …` message.
|
|
362
|
+
- `publish: verification-claimed <ISO-expiry>` — parent scan; atomic claim
|
|
363
|
+
with lease. Not a verdict.
|
|
171
364
|
- `publish: verified <commit> (<inspection_id>)` — parent, after stamping
|
|
172
365
|
AND reading the stamp back exactly; re-queued to `in_progress` (never
|
|
173
366
|
`todo`).
|
|
174
367
|
- `publish: content-mismatch <commit> <details>` — parent; exact FAIL
|
|
175
368
|
evidence quoted; stays parked, never stamped, never re-queued to QA.
|
|
176
369
|
- `publish: build-mismatch <commit> expected <expected> observed <live>` —
|
|
177
|
-
parent; the read-back
|
|
178
|
-
|
|
179
|
-
- `publish: superseded <commit> by <
|
|
180
|
-
|
|
370
|
+
parent; the read-back positively identified a different build's output
|
|
371
|
+
(step 4b); stays parked, never stamped, never re-queued.
|
|
372
|
+
- `publish: superseded <commit> by <head>` — parent; HEAD moved past the
|
|
373
|
+
commit; stays parked for human attention.
|
|
181
374
|
- `publish: verification-blocked <commit> <reason>` — parent; no read-back
|
|
182
375
|
obtainable; stays parked for a human.
|
|
376
|
+
- `publish: unknown-resolved <commit>` — recovery; durable build evidence
|
|
377
|
+
found inside the publish window for a previously-unknown attempt (see
|
|
378
|
+
"Unknown-outcome recovery"). The original `unknown` outcome is preserved;
|
|
379
|
+
the mirrored `verification-requested` note (written strictly later) is
|
|
380
|
+
what the scan claims.
|
|
183
381
|
- `publish: stamp-failed <commit> <reason>` — parent; read-back matched but
|
|
184
382
|
the stamp call failed; stays parked for a human.
|
|
383
|
+
- `publish: reconciled verified-but-parked -> in_progress` — parent scan;
|
|
384
|
+
the verified verdict was recorded but the re-queue was lost.
|
|
185
385
|
|
|
186
386
|
## Workflow differences
|
|
187
387
|
|
|
@@ -192,14 +392,3 @@ meaning:
|
|
|
192
392
|
exact stamp read-back), the dispatcher proceeds to terminal completion.
|
|
193
393
|
The parent's stamp read-back is the final gate — no downstream phase
|
|
194
394
|
re-checks it.
|
|
195
|
-
|
|
196
|
-
## Recovery: rebuilding the read-back request
|
|
197
|
-
|
|
198
|
-
If the park message names no inspection and you must trigger the read-back
|
|
199
|
-
manually, the request is deterministic — rebuild it from the workflow source
|
|
200
|
-
(`buildPublishReadbackRequest` in `workflows/standard.js`, identical in
|
|
201
|
-
`bugfix.js` and `chore.js`) with the task id, the `<commit>` from the park
|
|
202
|
-
message, the diff from `git show <commit>` in the project's `repo_path`,
|
|
203
|
-
and the agent_id from the park message's `(build …)` suffix (use the ledger
|
|
204
|
-
lookup from step 4b if the suffix says `agent_id unobserved`; pass null if
|
|
205
|
-
neither is available).
|
package/docs/visual-verdict.md
CHANGED
|
@@ -1,28 +1,85 @@
|
|
|
1
|
-
# The Visual Verdict —
|
|
1
|
+
# The Visual Verdict — Hazel owns it
|
|
2
|
+
|
|
3
|
+
> **UPDATE (2026-09-15):** the parent post-change verdict protocol is
|
|
4
|
+
> **retired**. There is no `visual_verdict:` note contract, no
|
|
5
|
+
> `visualVerdictStatus()` gate, no `visual: pending` marker, and no parent
|
|
6
|
+
> capture/composition step. Hazel (the QA work agent) drives the artifact
|
|
7
|
+
> herself with `lib/see-act.js`, logs every step to the OODA report, and
|
|
8
|
+
> records her verdict machine-readably — the workflow trusts her verdict
|
|
9
|
+
> and never parks for a parent one. The baseline-capture protocol below is
|
|
10
|
+
> unchanged: baselines are still captured before the fix exists, which the
|
|
11
|
+
> workflow cannot do synchronously.
|
|
12
|
+
|
|
13
|
+
> **UPDATE (2026-09-14):** the agent-owned see-act loop superseded the
|
|
14
|
+
> blocked platform path. `artifact_inspect` was removed by the platform and
|
|
15
|
+
> `artifact.inspect` is malfunction diagnosis, not a substitute — but the
|
|
16
|
+
> workflow no longer needs either: the QA/repro agent drives the artifact
|
|
17
|
+
> itself with `lib/see-act.js` (one browser action per invocation, reads
|
|
18
|
+
> its own screenshots) against `lib/serve-artifact.js` (local server for the
|
|
19
|
+
> built artifact). Experiential tasks get agent-run visual inspection in
|
|
20
|
+
> the QA phase (standard, bugfix) and experiential repro (bugfix).
|
|
2
21
|
|
|
3
22
|
QA owns the visual verdict for experiential artifact tasks: does the change
|
|
4
|
-
*look* right, judged by Hazel's artist-eye rubric, from rendered evidence
|
|
5
|
-
|
|
6
|
-
mechanical halves; the parent handles everything that needs eyes.
|
|
23
|
+
*look* right, judged by Hazel's artist-eye rubric, from rendered evidence
|
|
24
|
+
she captured herself.
|
|
7
25
|
|
|
8
|
-
## Why
|
|
26
|
+
## Why Hazel does it herself
|
|
9
27
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
28
|
+
The old model was async: rendered post-change evidence was delivered to
|
|
29
|
+
the root agent — the parent — which then recorded a `visual_verdict:` note
|
|
30
|
+
the workflow gated on. That split the verdict across two systems and left
|
|
31
|
+
tasks parked whenever the parent path stalled.
|
|
14
32
|
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
evidence, generating composites, launching Hazel only after rendered
|
|
20
|
-
evidence arrives, recording `visual_verdict`, and closing, reworking, or
|
|
21
|
-
parking the task.
|
|
33
|
+
The new model is depth-1: the QA work agent IS the inspector. She starts
|
|
34
|
+
the local artifact server, drives it one browser step at a time, reads
|
|
35
|
+
every frame, edits and composes evidence when a raw screenshot is not
|
|
36
|
+
enough, and records her verdict. The contract is:
|
|
22
37
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
38
|
+
- **Hazel-owned:** the experiential loop, the OODA report
|
|
39
|
+
(`ooda-log.jsonl` + archived frames + derivatives), `verdict.json` and
|
|
40
|
+
the append-only `verdicts.jsonl` ledger, the visual verdict itself.
|
|
41
|
+
- **Workflow-owned:** the experiential flag, the Capture phase, the Map
|
|
42
|
+
baseline gate, the mechanical QA checks, trusting Hazel's verdict.
|
|
43
|
+
- **Parent-owned:** baseline capture only (see below) — baselines are
|
|
44
|
+
captured before the fix exists, which the workflow cannot do
|
|
45
|
+
synchronously.
|
|
46
|
+
|
|
47
|
+
No experiential artifact task completes without Hazel's recorded verdict.
|
|
48
|
+
A missing OODA log is a broken report, not a pass. Unknown is neither
|
|
49
|
+
PASS nor FAIL — un-runnable checks are named as explicit evidence gaps.
|
|
50
|
+
|
|
51
|
+
## The OODA report
|
|
52
|
+
|
|
53
|
+
Task evidence lives under `$CREW_HOME/task-evidence/<task-id>/<phase>/`
|
|
54
|
+
(`repro/` for bugfix Reproduce, `postchange/` for QA):
|
|
55
|
+
|
|
56
|
+
- `ooda-log.jsonl` — ordered steps with observations. Every step carries
|
|
57
|
+
an `--attempt` identity; steps are strictly monotonic within an attempt;
|
|
58
|
+
a rerun is a new attempt at step 1 — attempts accumulate, never
|
|
59
|
+
overwrite. Actions: `aria|shot|click|scroll|type` (browser) and
|
|
60
|
+
`crop|zoom|label|nup|compose` (evidence derivatives).
|
|
61
|
+
- Archived frames — `001-shot-desktop.png`, `002-click-mobile.png`, … via
|
|
62
|
+
`SEE_ACT_ARCHIVE_DIR`. A frame Hazel did not read is not evidence.
|
|
63
|
+
- `verdict.json` — the latest machine-readable verdict
|
|
64
|
+
(`{verdict, summary, expected, actual, missing_evidence}`).
|
|
65
|
+
- `verdicts.jsonl` — the append-only ledger: every attempt's verdict is
|
|
66
|
+
preserved with a mechanical sequence number, never overwritten.
|
|
67
|
+
|
|
68
|
+
## Evidence editing and composition
|
|
69
|
+
|
|
70
|
+
When a raw screenshot is not enough evidence, Hazel has two deterministic
|
|
71
|
+
layers:
|
|
72
|
+
|
|
73
|
+
- **Pixel operations** (`lib/edit-image.py`, Pillow): `crop` (pixel-exact,
|
|
74
|
+
rejects out-of-bounds), `zoom` (nearest-neighbor, back to frame size),
|
|
75
|
+
`label` (caption bar), `nup` (side-by-side grid with labels).
|
|
76
|
+
- **Composition** (`lib/render-html.js`, headless Chromium): renders a
|
|
77
|
+
local HTML layout to PNG for rich before/after presentations with real
|
|
78
|
+
typography. Hermetic — remote assets are blocked and fail loudly.
|
|
79
|
+
|
|
80
|
+
Derivatives supplement, never replace: the source frame stays archived
|
|
81
|
+
and is named in the step's args. Every derivative is logged and read —
|
|
82
|
+
an unread image is not evidence.
|
|
26
83
|
|
|
27
84
|
## Evidence layout
|
|
28
85
|
|
|
@@ -31,16 +88,10 @@ Task evidence lives under:
|
|
|
31
88
|
```
|
|
32
89
|
$CREW_HOME/task-evidence/<task-id>/
|
|
33
90
|
baseline/ baseline captures (pre-change)
|
|
34
|
-
postchange/ post-change captures
|
|
35
|
-
|
|
91
|
+
postchange/ post-change captures + OODA report
|
|
92
|
+
repro/ bugfix reproduction captures + OODA report
|
|
36
93
|
```
|
|
37
94
|
|
|
38
|
-
Capture filenames follow the deterministic frame used by the workflow's
|
|
39
|
-
`buildVisualCapturePlan` output: `<kind>-<n>-<viewport>-<state>.png`
|
|
40
|
-
(e.g. `baseline-01-desktop1440x900-top.png`). The composites are named
|
|
41
|
-
`<stem>-sidebyside.png` and `<stem>-overlay.png` by
|
|
42
|
-
`lib/compose-evidence.py`.
|
|
43
|
-
|
|
44
95
|
## Baseline capture protocol
|
|
45
96
|
|
|
46
97
|
Triggered by the Capture phase when the task is experiential and no
|
|
@@ -69,33 +120,6 @@ baseline evidence is recorded yet. The workflow logs
|
|
|
69
120
|
The workflow requests at most two attempts; after two it records
|
|
70
121
|
`baseline: none (capture unavailable after 2 attempts)` itself.
|
|
71
122
|
|
|
72
|
-
## Post-change capture, composition, Hazel, verdict
|
|
73
|
-
|
|
74
|
-
After the QA step completes its mechanical checks, the QA session notes
|
|
75
|
-
carry `visual: pending` and the deterministic `capture_plan:` — the parent
|
|
76
|
-
then:
|
|
77
|
-
|
|
78
|
-
1. Trigger the post-change inspection with the `capture_plan:` frame from
|
|
79
|
-
the QA session notes. Save captures under `postchange/`.
|
|
80
|
-
2. Run `python3 lib/compose-evidence.py <baseline-dir> <postchange-dir>
|
|
81
|
-
<composites-dir>` and keep its `PAIR...` / `SKIP...` manifest output as
|
|
82
|
-
evidence refs.
|
|
83
|
-
3. Launch Hazel with `identities/hazel.md` (artist-eye rubric + verdict
|
|
84
|
-
discipline), the baseline refs, the post-change refs, and the composite
|
|
85
|
-
manifest. She receives rendered evidence only — never a prose
|
|
86
|
-
description of what the change "looks like".
|
|
87
|
-
4. Record her verdict as a task note event:
|
|
88
|
-
`visual_verdict: PASS <composite refs>` or
|
|
89
|
-
`visual_verdict: FAIL <reason> <refs>`.
|
|
90
|
-
5. Routing:
|
|
91
|
-
- **PASS** — the task is done; close it.
|
|
92
|
-
- **FAIL** — send the task back to Build as budgeted rework, with the
|
|
93
|
-
FAIL reason and refs as the rejection notes.
|
|
94
|
-
- **FAIL with reason beginning `rendering impossible:`** — the target
|
|
95
|
-
cannot be rendered at all (never a rework loop). Park the task for
|
|
96
|
-
human attention. A verdict can never be issued without rendered
|
|
97
|
-
evidence; impossibility fails, it never passes.
|
|
98
|
-
|
|
99
123
|
## Exact note-event prefixes
|
|
100
124
|
|
|
101
125
|
Case-sensitive, exact-prefix matches — the workflow matches on prefixes,
|
|
@@ -105,16 +129,6 @@ never on English meaning:
|
|
|
105
129
|
- `baseline: captured <refs>` — parent, after saving baseline captures.
|
|
106
130
|
- `baseline: none (<reason>)` — baseline not capturable; final QA judges
|
|
107
131
|
on the rubric alone and states that explicitly.
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
Build within the shared budget.
|
|
112
|
-
|
|
113
|
-
## Recovery: rebuilding the capture plan
|
|
114
|
-
|
|
115
|
-
If the QA session notes are truncated (summary cap) and `capture_plan:`
|
|
116
|
-
is lost, the frame is deterministic — rebuild it from the workflow
|
|
117
|
-
source: `buildVisualCapturePlan(taskTitle, taskDescription,
|
|
118
|
-
"postchange", captureTargets)` in `workflows/standard.js` (identical in
|
|
119
|
-
`bugfix.js` and `chore.js`). Capture targets come from the Map step's
|
|
120
|
-
`capture_targets:` marker line; fall back to the task description.
|
|
132
|
+
|
|
133
|
+
The `visual_verdict:` prefixes are retired (2026-09-15) — Hazel's verdict
|
|
134
|
+
lives in `verdict.json` / `verdicts.jsonl`, not in note events.
|