muse-crew 0.13.2 → 0.14.0

package/docs/decisions/qa-reproduce.md

@@ -0,0 +1,486 @@
+# Decision history: qa reproduce
+
+Relocated from workflow source comments during H5 (2026-09-18). The workflows keep only the relied-upon invariant inline; the full decision history lives here.
+
+<a id="reproduce-wrong-layer-guard"></a>
+## Reproduce wrong-layer guard
+
+Invariant: for engine/docs tasks the experiential see-act loop is forbidden; the workflow checks the repro dir mechanically for frame artifacts, attempt-scoped; no baseline is inconclusive fail-closed.
+
+Applies to: standard, bugfix, chore.
+
+```
+  // Reproduce wrong-layer guard (2026-09-16): for engine/docs tasks the
+  // experiential see-act loop is forbidden — a reproducer that drives the
+  // browser burns the run and emits a wrong-layer verdict (three tasks
+  // parked on exactly this in the 2026-09-17 UTC shift). The workflow
+  // checks the repro evidence dir mechanically for see-act frame artifacts
+  // (the driver's archive naming: <n>-<action>-<desktop|mobile>.png).
+  // Attempt-scoped (2026-09-17): the after-listing is diffed against the
+  // baseline taken before this phase attempt's dispatch loop — only frames
+  // CREATED BY the current attempt count as a violation; stale frames from
+  // earlier attempts appear in both snapshots and are excluded. Without a
+  // baseline the guard is inconclusive and fails closed for retry (passing
+  // could route a genuinely wrong-layer verdict to Map).
+```
+
+<a id="baseline-evidence-capture"></a>
+## Baseline evidence capture
+
+Invariant: capture baseline evidence for experiential tasks before the QA agent runs; the baseline is the control.
+
+Applies to: standard.
+
+```
+  // ── Capture: baseline evidence for experiential tasks ─────────────
+  // Hazel's QA capture pass runs right after Triage, before Map, for tasks
+  // Sage flagged experiential. The capture itself is parent-driven (the
+  // inspection handoff arrives at the root agent, outside this script), so
+  // when no baseline evidence is recorded yet the script logs a note event
+  // and parks with the exact parent protocol + resume path. Never fails the
+  // task over missing evidence: after two requests, baseline:none is
+  // recorded and final QA judges on the rubric alone.
+```
+
+Applies to: bugfix.
+
+```
+  // ── Capture: baseline evidence for experiential tasks ─────────────
+  // Hazel's QA capture pass runs right after Triage, before Map, for tasks
+  // Sage flagged experiential. The capture itself is parent-driven (the
+  // inspection handoff arrives at the root agent, outside this script), so
+  // when no baseline evidence is recorded yet the script logs a note event
+  // and parks with the exact parent protocol + resume path. Never fails the
+  // task over missing evidence: after two requests, baseline:none is
+  // recorded and final QA judges on the rubric alone.
+```
+
+Applies to: chore.
+
+```
+  // ── Capture: baseline evidence for experiential tasks ─────────────
+  // Hazel's QA capture pass runs right after Triage, before Map, for tasks
+  // Sage flagged experiential. The capture itself is parent-driven (the
+  // inspection handoff arrives at the root agent, outside this script), so
+  // when no baseline evidence is recorded yet the script logs a note event
+  // and parks with the exact parent protocol + resume path. Never fails the
+  // task over missing evidence: after two requests, baseline:none is
+  // recorded and the task continues without baseline comparison.
+```
+
+
+<a id="pre-publish-pin-guard"></a>
+## Pre-Publish pin guard
+
+Invariant: the pin step materializes the required scripts before Publish; the agent cannot self-certify.
+
+Applies to: standard, bugfix, chore.
+
+```
+// The seven basenames the pin step must materialize — asserted mechanically
+// by workflow code from the verbatim listing, never from agent prose.
+// COMPUTE_DIFF is the deterministic publish-diff computer (room #14,
+// 2026-09-17): the diff is computed by this script, never ferried as an
+// agent JSON string. Pinned like the other publish-critical modules so a
+// mid-run release swap cannot change it under the workflow.
+// CLASSIFY_SURFACE is the surface classifier (room #15, 2026-09-18):
+// crew-api.js statically imports it, so the pin must carry it — a pin
+// without it kills every claim with ERR_MODULE_NOT_FOUND.
+```
+
+<a id="qa-verdict-gate"></a>
+## QA verdict.json closeout gate
+
+Invariant: the prose VERDICT: line drives the closeout gate via extractVerdict; a bare FAIL with an all-positive report is a defect.
+
+Applies to: standard, bugfix, chore.
+
+```
+  // QA verdict.json closeout gate (task 30dceb78): the prose VERDICT: line is
+  // the routing signal, but verdict.json is the reason-carrying record the
+  // workflow actually reads. A FAIL verdict must carry a machine-readable
+  // reason; the workflow refuses to route to rework on an unreasoned or
+  // contradictory verdict. The deterministic cross-checker
+  // (lib/read-ooda-verdict.js) runs against the prose verdict: a missing,
+  // corrupt, contradictory, or reason-less record fails the phase for retry
+  // — the dispatcher re-runs QA at the same step under its
+  // consecutive-failure cap — instead of routing to rework. Without this
+  // gate, a bare VERDICT: FAIL with an all-positive report (canary
+  // 2026-09-15, task 1d692d91) rebuilt nothing and parked at Publish on an
+  // unobserved artifact build. The gate applies only to the experiential QA
+  // path (qaArtifact / qaTerminal), the paths that write verdict.json.
+```
+
+<a id="experiential-loop-guard"></a>
+## QA experiential-loop guard
+
+Invariant: a PASS verdict with missing experiential evidence never completes — parks fail-closed with unattributable_reason.
+
+Applies to: standard.
+
+```
+  // QA experiential-loop guard (clean-room defect 2026-09-16): Hazel's verdict.json
+  // is honest about missing experiential evidence, but the closeout treated a PASS
+  // as terminal done even when the experiential loop never ran (playwright-core
+  // was unresolvable from the release layout — the dependency lived in the
+  // npm install dir, severed from the crew home). A PASS verdict with missing
+  // experiential evidence must never be terminal: the task parks fail-closed
+  // with unattributable_reason=qa-visual-loop-unavailable (artifact surface)
+  // or qa-terminal-loop-unavailable (terminal surface) instead of
+  // transitioning to done. Code, not prompt text: the check reads the
+  // machine-readable verdict via lib/read-ooda-verdict.js, which reports
+  // visual_loop_unavailable from the OODA log's NOT POSSIBLE browser steps
+  // and terminal_loop_unavailable from NOT POSSIBLE terminal steps, plus the
+  // verdict's missing_evidence tool-unavailability notes.
+```
+
+Applies to: bugfix.
+
+```
+    // QA experiential-loop guard (clean-room defect 2026-09-16): Hazel's
+    // verdict.json is honest about missing experiential evidence, but the
+    // closeout treated a PASS as terminal done even when the experiential
+    // loop never ran. A PASS verdict with missing experiential evidence must
+    // never be terminal: the task parks fail-closed with
+    // unattributable_reason=qa-visual-loop-unavailable (artifact surface)
+    // or qa-terminal-loop-unavailable (terminal surface) instead of
+    // transitioning to done. Code, not prompt text: the loop-availability
+    // flags come from lib/read-ooda-verdict.js (the OODA log's NOT POSSIBLE
+    // steps and the verdict's missing_evidence tool-unavailability notes),
+    // already parsed into qaVerdictGate above.
+```
+
+
+<a id="reproduce-baseline"></a>
+## Reproduce wrong-layer guard baseline
+
+Invariant: snapshot the repro evidence dir before the Reproduce agent runs; only frames created by the current attempt count; null baseline is inconclusive fail-closed.
+
+Applies to: standard, bugfix, chore.
+
+```
+  // Reproduce wrong-layer guard baseline (2026-09-17): snapshot the repro
+  // evidence dir listing BEFORE the Reproduce work agent runs. The guard
+  // after the run diffs against this baseline — only frames created by the
+  // current attempt count as a wrong-layer violation; stale frames from
+  // pre-fix attempts appear in both snapshots and are excluded (set diff,
+  // no wall-clock, deterministic). One snapshot per phase attempt covers
+  // all transport retries inside the dispatch loop. null = baseline
+  // unavailable; the guard then fails closed for retry.
+```
+
+<a id="visual-verdict-ownership"></a>
+## Visual verdict ownership
+
+Invariant: Hazel owns the visual verdict experientially (see-act loop + verdict.json); her prose VERDICT drives passed; the old parent note gate is deleted.
+
+Applies to: standard, bugfix, chore.
+
+```
+  // Visual verdict ownership (2026-09-15): Hazel owns the visual verdict
+  // experientially — the QA instructions above have her drive the see-act
+  // loop herself and record verdict.json + the append-only verdicts.jsonl.
+  // Her prose VERDICT: line drives `passed` via extractVerdict; a FAIL
+  // bounces to Build through the standard Review/QA rejection path below.
+  // The old parent-recorded note gate is deleted.
+```
+
+<a id="baseline-evidence-status"></a>
+## Baseline evidence status
+
+Invariant: reads the task's note events for baseline status.
+
+Applies to: standard, bugfix, chore.
+
+```
+// Baseline evidence status: reads the task's note events for the exact
+// protocol prefixes (explicit state, never English matching). Returns
+// { baseline_found, baseline_kind, baseline_refs, requested_count, evidence_count }.
+// found = any note starting exactly "baseline: captured" or "baseline:
+// none" (kind/refs come from the LATEST such message); requested_count =
+// the number of notes starting exactly "baseline: requested";
+// evidence_count = the number of notes starting exactly "baseline: captured" or
+// "baseline: none" (the per-attempt evidence chain; state-derived, no wall clock). Each call
+// uses a fresh key: the evidence changes between calls (the parent logs
+// the capture while this run is parked), so a cached replay would lie.
+```
+
+<a id="pin-basenames"></a>
+## Pin basenames
+
+Invariant: the pin step materializes the required scripts.
+
+Applies to: standard, bugfix, chore.
+
+```
+// The seven basenames the pin step must materialize — asserted mechanically
+// by workflow code from the verbatim listing, never from agent prose.
+// COMPUTE_DIFF is the deterministic publish-diff computer (room #14,
+// 2026-09-17): the diff is computed by this script, never ferried as an
+// agent JSON string. Pinned like the other publish-critical modules so a
+// mid-run release swap cannot change it under the workflow.
+// CLASSIFY_SURFACE is the surface classifier (room #15, 2026-09-18):
+// crew-api.js statically imports it, so the pin must carry it — a pin
+// without it kills every claim with ERR_MODULE_NOT_FOUND.
+```
+
+<a id="visual-protocol-avail"></a>
+## Visual protocol availability
+
+Invariant: the workflow parks if the visual protocol is unavailable.
+
+Applies to: standard.
+
+```
+// Visual verdict protocol availability — the workflow parks for parent-run
+// baseline capture and visual verdict ONLY when the protocol is fully
+// shipped. The protocol requires docs/visual-verdict.md in the release AND
+// the parent-side capture tooling (task b309a97d, "QA owns the visual
+// verdict"). Until both exist, the parks would deadlock waiting for a
+// parent who cannot fulfill them.
+// Effective value for this run, resolved by the dispatcher from the
+// project's visual_protocol setting (null=inherits crew default=off).
+// Manual launches without the arg default to off (previous behavior).
+```
+
+Applies to: bugfix.
+
+```
+// Visual verdict protocol availability — the workflow parks for parent-run
+// baseline capture and visual verdict ONLY when the protocol is fully
+// shipped. The protocol requires docs/visual-verdict.md in the release AND
+// the parent-side capture tooling (task b309a97d, "QA owns the visual
+// verdict"). Until both exist, the parks would deadlock waiting for a
+// parent who cannot fulfill them.
+// Effective value for this run, resolved by the dispatcher from the
+// project's visual_protocol setting (null=inherits crew default=off).
+// Manual launches without the arg default to off (previous behavior).
+```
+
+Applies to: chore.
+
+```
+// Visual verdict protocol availability — the workflow parks for parent-run
+// baseline capture ONLY when the protocol is fully shipped. The protocol
+// requires docs/visual-verdict.md in the release AND the parent-side
+// capture tooling (task b309a97d, "QA owns the visual verdict"). Until both
+// exist, the park would deadlock waiting for a parent who cannot fulfill
+// it.
+// Effective value for this run, resolved by the dispatcher from the
+// project's visual_protocol setting (null=inherits crew default=off).
+// Manual launches without the arg default to off (previous behavior).
+```
+
+
+<a id="capture-baseline"></a>
+## Capture baseline
+
+Invariant: baseline evidence for experiential tasks is captured before work begins.
+
+Applies to: standard.
+
+```
+  // ── Capture: baseline evidence for experiential tasks ─────────────
+  // Hazel's QA capture pass runs right after Triage, before Map, for tasks
+  // Sage flagged experiential. The capture itself is parent-driven (the
+  // inspection handoff arrives at the root agent, outside this script), so
+  // when no baseline evidence is recorded yet the script logs a note event
+  // and parks with the exact parent protocol + resume path. Never fails the
+  // task over missing evidence: after two requests, baseline:none is
+  // recorded and final QA judges on the rubric alone.
+```
+
+Applies to: bugfix.
+
+```
+  // ── Capture: baseline evidence for experiential tasks ─────────────
+  // Hazel's QA capture pass runs right after Triage, before Map, for tasks
+  // Sage flagged experiential. The capture itself is parent-driven (the
+  // inspection handoff arrives at the root agent, outside this script), so
+  // when no baseline evidence is recorded yet the script logs a note event
+  // and parks with the exact parent protocol + resume path. Never fails the
+  // task over missing evidence: after two requests, baseline:none is
+  // recorded and final QA judges on the rubric alone.
+```
+
+Applies to: chore.
+
+```
+  // ── Capture: baseline evidence for experiential tasks ─────────────
+  // Hazel's QA capture pass runs right after Triage, before Map, for tasks
+  // Sage flagged experiential. The capture itself is parent-driven (the
+  // inspection handoff arrives at the root agent, outside this script), so
+  // when no baseline evidence is recorded yet the script logs a note event
+  // and parks with the exact parent protocol + resume path. Never fails the
+  // task over missing evidence: after two requests, baseline:none is
+  // recorded and the task continues without baseline comparison.
+```
+
+
+<a id="wrong-layer-guard"></a>
+## Wrong-layer guard
+
+Invariant: reproduce must run at the correct layer.
+
+Applies to: bugfix.
+
+```
+  // Reproduce wrong-layer guard baseline (2026-09-17): snapshot the repro
+  // evidence dir listing BEFORE the Reproduce work agent runs. The guard
+  // after the run diffs against this baseline — only frames created by the
+  // current attempt count as a wrong-layer violation; stale frames from
+  // pre-fix attempts appear in both snapshots and are excluded (set diff,
+  // no wall-clock, deterministic). One snapshot per phase attempt covers
+  // all transport retries inside the dispatch loop. null = baseline
+  // unavailable; the guard then fails closed for retry.
+```
+
+<a id="parent-reader-removed"></a>
+## Parent reader removed
+
+Invariant: the parent visual-verdict reader was removed; Hazel owns visual QA.
+
+Applies to: bugfix.
+
+```
+// The parent visual-verdict reader was removed 2026-09-15: Hazel (the QA
+// work agent) now owns the visual verdict experientially — she drives the
+// now owns the visual verdict experientially — she drives the see-act loop
+// herself and records verdict.json + the append-only verdicts.jsonl. The old
+// see-act loop herself and records verdict.json + the append-only
+// verdicts.jsonl. The old parent note gate is obsolete and has been
+// deleted from the QA closeout below. (See docs/visual-verdict.md.)
+// from the QA closeout below. (See docs/visual-verdict.md.)
+```
+
+<a id="surface-routing"></a>
+## Surface routing
+
+Invariant: the user-facing surface determines experiential QA routing.
+
+Applies to: standard, bugfix.
+
+```
+// User-facing surface for experiential QA routing: 'artifact' (a rendered
+// web UI Hazel drives with the see-act browser loop) | 'terminal' (a CLI
+// Hazel drives herself, keeping attempt-scoped transcripts) | null
+// (unclassified — no experiential QA). environment_type is the canonical
+// UX-surface axis; deploy_type names the deployment target, but
+// deploy_type === "artifact" remains a legacy artifact-surface signal so
+// pre-field projects keep today's experiential QA (the migration does not
+// backfill the column).
+```
+
+<a id="worktree-confinement"></a>
+## Worktree confinement
+
+Invariant: the Build agent must declare its worktree.
+
+Applies to: standard, bugfix, chore.
+
+```
+// Worktree confinement: the Build agent must declare the exact worktree
+// path it built in on a `worktree:` marker line. The workflow compares it
+// against WORKTREE_HINT mechanically (exact string match) — never by
+// reading agent prose. This closes the hole where a builder whose prepare
+// failed freelanced into a different checkout (canary, 2026-09-11): the
+// honest-but-confused case fails here, and a fabricated path is caught one
+// phase later when Review's inspect finds no commits in the configured repo.
+```
+
+<a id="terminal-qa"></a>
+## Terminal QA
+
+Invariant: Hazel drives CLI transcripts for terminal surfaces.
+
+Applies to: standard, bugfix.
+
+```
+      // Terminal-surface experiential QA: Hazel drives the CLI herself —
+      // the terminal counterpart to the see-act loop above. Same OODA
+      // discipline (append-ooda-step with --action terminal and a transcript
+      // per step; verdict via write-ooda-verdict), judged against the
+      // shared bar resolved via UX_DOCTRINE_PATH (the terminal doctrine page here). Transcripts are delivered;
+      // screenshots are never invented for terminal work.
+```
+
+<a id="layer-resolution"></a>
+## Layer resolution
+
+Invariant: the task's layer determines the QA surface.
+
+Applies to: bugfix.
+
+```
+// Layer resolution: the task's layer is "artifact", "engine", or "docs" —
+// the machine-read "layer:" marker Sage's Triage report ends with. Unknown
+// (missing/garbled line, failed lookup) degrades to "artifact": today's
+// single-strategy behavior, never a park. Mirrors resolveExperiential()
+// (same cache shape, same Triage-notes re-read); the layer flag is captured
+// at Triage closeout (bugLayer) so the common path needs no extra agent call.
+```
+
+<a id="ux-doctrine-page"></a>
+## UX doctrine page
+
+Invariant: the shared UX bar for this run's surface.
+
+Applies to: standard, bugfix, chore.
+
+```
+// UX doctrine page: the shared UX bar for this run's surface, resolved
+// mechanically — every phase prompt reads UX_DOCTRINE_PATH, never a
+// hardcoded filename. Canonical map: lib/ux-doctrine.js (mirrored here as a
+// one-liner because the workflow runtime's relative-import support is
+// unverified; tests pin the mirror). Null on unclassified surfaces: no
+// shared page, and prompts say so instead of naming the wrong one.
+```
+
+<a id="visual-ownership"></a>
+## Visual verdict ownership
+
+Invariant: Hazel owns the visual verdict.
+
+Applies to: bugfix.
+
+```
+  // Visual verdict ownership (2026-09-15): Hazel owns the visual verdict
+  // experientially — the QA instructions above have her drive the see-act
+  // loop herself and record verdict.json + the append-only verdicts.jsonl.
+  // Her prose VERDICT: line drives `passed` via extractVerdict; a FAIL
+  // bounces to Build through the standard Review/QA rejection path below.
+  // The old parent-recorded note gate is deleted.
+```
+
+<a id="stale-decision-guard"></a>
+## Stale-decision guard
+
+Invariant: a stale baseline decision parks fail-closed.
+
+Applies to: standard, bugfix, chore.
+
+```
+    // Stale-decision guard: a "baseline: none (visual protocol unavailable)"
+    // note is only durable while the protocol is unavailable. When
+    // VISUAL_PROTOCOL_AVAILABLE is true, that old decision no longer
+    // stands — fall through to the request path for a fresh capture
+    // attempt. Exact-string trim comparison against the workflow's own
+    // written message (explicit state, never English matching).
+```
+
+<a id="pin-lifecycle"></a>
+## Pin lifecycle
+
+Invariant: snapshot the lifecycle scripts in the pin.
+
+Applies to: standard, bugfix, chore.
+
+```
+// pinLifecycle(key) — snapshot the lifecycle scripts into RUN_LIB and return
+// the verbatim `ls -1` listing so WORKFLOW CODE asserts the six pinned
+// basenames; the agent cannot self-certify. (The pin step was the one place
+// the workflows trusted agent prose: task 24be1cd6 walked to Publish on an
+// empty pin dir.) Byte-identical across standard/bugfix/chore — pinned by
+// tests/pin-location.test.js.
+```
+