devlyn-cli 1.15.0 → 2.1.0

package/config/skills/_shared/runtime-principles.md

@@ -0,0 +1,110 @@
+# Runtime principles — sub-agent contract
+
+The runtime contract every sub-agent inside `/devlyn:resolve` (PLAN / IMPLEMENT / BUILD_GATE / CLEANUP / VERIFY) and `/devlyn:ideate` (FRAME / EXPLORE / SPEC / CHALLENGE) must satisfy. Source of truth for sub-agent behavior on user tasks. NOT for autoresearch-loop / harness-developer concerns (see `autoresearch/PRINCIPLES.md`).
+
+The four sections below mirror the corresponding CLAUDE.md sections (Subtractive-first editing, Goal-locked execution, No-workaround discipline, Evidence over claim). Each section is wrapped in `<!-- runtime-principles:section=NAME:begin -->` / `:end -->` markers in BOTH this file and CLAUDE.md; lint Check 12 (added in iter-0019.A Step 5) extracts each named block from both files and diffs to detect drift.
+
+<!-- runtime-principles:contract:begin -->
+## Subtractive-first editing — perfection = nothing left to remove
+<!-- runtime-principles:section=subtractive-first:begin -->
+
+> "Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away." — Saint-Exupéry. **This is the operating definition of "done" in this repo.** A change is finished when no further line, branch, flag, or doc paragraph can be removed without breaking a learned failure mode. Not before.
+
+This rule overrides instinct. LLMs (including you) are trained on corpora that reward elaborate, defensive, "thorough" code — so the default impulse is to add. That impulse is wrong here. Read the rules below as hard tests, not aesthetic preferences. They are not optional, not negotiable, and not satisfiable by writing more careful additions.
+
+**Mandatory pre-edit question.** Before writing any change, you must answer in this order:
+
+1. **What can I delete that makes the addition unnecessary?** If the addition becomes redundant after the deletion, ship the deletion alone.
+2. **What can I delete that makes the addition smaller?** Trim the surrounding accretion before adding.
+3. **Only then**, what is the minimum addition required?
+
+If you skip question 1 or 2, you are violating this rule even if the resulting code looks clean.
+
+**Hard tests every edit must pass:**
+
+- **Net-negative is the default; pure-addition needs a citation.** A diff that adds N lines and removes 0 must point to a specific cause: a previously-observed failure mode (commit hash, fixture ID, finding ID, user-reported incident), OR an explicit user request / spec requirement that demands new user-visible behavior. The latter is a sufficient citation — do not block legitimate requested additions on the absence of a past failure. What is rejected: vague justifications like "it seems clearer," "for future flexibility," "just in case," "to be safe," "for completeness," "to handle edge cases" — these are the exact phrases that produce accretion.
+- **Delete the line that makes the bug impossible, not the line that catches it.** Defensive wrappers, validation layers, error normalizers, and `try/catch` shells are usually evidence that an upstream contract is unclear. Fix the contract upstream and remove the defenses downstream. The trap: adding the wrapper feels like progress because it makes a test pass. The wrapper is debt; the contract fix is the work. **Scope guard**: if the upstream contract fix is outside the user's stated scope, stop and surface the scope expansion to the user before editing — Goal-locked execution overrides this. The right scope-expansion outcome is "user authorizes the upstream fix" or "user accepts a scoped local fix and a follow-up for upstream"; never silently restructure something the user didn't ask you to.
+- **A new flag, branch, or option is admitting two failures**: (a) the default was wrong, (b) every reader pays attention cost forever. Default-fix-and-delete-flag beats add-flag-with-better-default. The bar for adding a configuration knob is "I have observed two real users with genuinely conflicting needs," not "this might be useful someday."
+- **Doc additions are subject to the same rule.** Before adding a section to any `.md` file (CLAUDE.md, SKILL.md, README, references/), find the now-stale sentence or section the new one supersedes — delete that first. A growing instructions file dilutes the instructions that actually need to be followed; readers (human and LLM) skim long files and miss load-bearing rules.
+- **A "cleaner" refactor that grows line count is not cleaner.** It is a sideways move that increases context, parsing, and review cost. **For refactor-only changes**, line count must drop unless a cited observed failure requires the new shape. **Never delete tests, contracts, public API, comments documenting non-obvious WHY, or user-facing behavior just to win the count** — that is gaming the metric, not honoring the principle. The metric serves complexity reduction; if a deletion would lose information not recoverable from code + commit history, it is the wrong deletion.
+- **Stop adding when no further deletion is possible.** This is the Saint-Exupéry test inverted into a stopping rule: if you have made an addition and you cannot identify anything else that can be removed, examine the addition itself — is part of it still removable? Iterate until the diff is irreducible.
+
+**Anti-rationalization clause** — explicitly guarding against LLM-style hedging:
+
+- "More explicit is safer" is **not** a justification. Explicitness has a cost in attention and rot. Required-explicit goes in; nice-to-explicit gets cut.
+- "Adding context for future readers" is **not** a justification. Future readers benefit more from shorter files than from explanatory prose. The code and the commit message together carry the why.
+- "Defense-in-depth" is **not** a justification at the harness layer. Two layers that catch the same bug are evidence one of them should be the only layer.
+- If you find yourself writing the phrase "in case" in a comment, code reviewer note, or doc, **stop and re-evaluate** — that phrase predicts an unjustified addition.
+
+**Stopping rule.** A change is done when (a) all hypotheses it was meant to close are closed, AND (b) you have attempted at least one further deletion and confirmed it would break something. If you have not tried to delete more, you are not done. If nothing can be deleted to justify the current addition, the addition itself is too large — re-scope or surface the conflict to the user before proceeding.
+
+**Never grow surface area silently.** Every accretion-shaped change must be visible: in the commit message, in the iteration file, or in a flagged review. Silent growth is the failure mode this rule exists to prevent.
+<!-- runtime-principles:section=subtractive-first:end -->
+
+## Goal-locked execution — stay on the North Star, do not wander
+<!-- runtime-principles:section=goal-locked:begin -->
+
+Even with a North Star defined, work drifts off-course ("산으로 간다" / "삼천포로 빠진다" — going up the wrong mountain instead of forward). The harness must **actively block** this drift at run time, not merely discourage it. The default is ruler-straight execution toward the user's stated goal; any deviation requires explicit justification, not the inverse.
+
+This rule exists because LLMs (including you) are trained to be helpful, comprehensive, and thorough — and "helpful" easily becomes "did more than asked." Doing more than asked is not helpfulness; it is scope creep. Read the rules below as hard blocks, not soft preferences.
+
+**The five drift patterns you must refuse to execute on:**
+
+1. **Unrequested work.** "While I'm here, I noticed X is broken/ugly/inefficient" → **stop**. The user did not ask for X. If X is a real defect, surface it as a finding, a follow-up suggestion, or an entry in a TODO list — do NOT fix it inside the current change. Mixing unrequested work with requested work is what makes diffs unreviewable and PRs eternal.
+2. **Tangential cleanup.** "This file looks messy, let me also tidy..." → **stop**. The current task is the only task. Unrelated cleanup is a separate change requiring its own justification, scope, and pre-flight 0 check.
+3. **Speculative robustness.** "Just adding a check / fallback / handler for the case where..." → **stop**. If the case has not been observed (in production, in tests, in a finding), it does not belong in this change. Defensive code added for unobserved cases is the most common form of accretion debt — it never gets removed because nobody can prove the case never happens.
+4. **Re-scoping mid-flight.** "Actually, the better way to do this is to also restructure / rename / migrate..." → **stop**. If you discover the requested approach is wrong, surface that to the user with evidence and let them adjudicate. Do NOT silently expand scope. The user's explicit redirect is the only authorization to enlarge a task.
+5. **Curiosity detours.** "Let me also explore how Y works to understand this better..." → **stop**, unless Y is provably on the goal's critical path. Curiosity-driven exploration is creative-mode; default is execution-mode.
+
+**The single drift test before any deviation from the stated goal:** *"Did the user ask for this, OR does the user's stated goal strictly require it?"* If the answer to both is no, do not do it. Surface it as a note (commit message, end-of-turn summary, finding) and continue on the original path.
+
+**Creative-mode is the narrow exception, not the default.** Creative-mode applies only when (a) the user explicitly invoked an ideation/exploration surface (`/devlyn:ideate`, optional `/devlyn:design-system`, "let's brainstorm", "explore options for"), OR (b) the goal is genuinely under-specified and a clarifying question is impossible (extremely rare — usually you should ask). For everything else — bug fixes, feature work, refactors, doc updates, pipeline runs, code review, debugging — execution-mode is the default and drift is a defect, not a feature.
+
+**Anti-rationalization clause** — explicitly guarding against LLM hedging:
+
+- "It's a small extra change" is **not** a justification. Small accretions compound; one of them is always small.
+- "It's related to what they asked for" is **not** a justification. Related ≠ requested. Requested is the only standard.
+- "It would be incomplete without this" is **not** a justification. The user defines completeness, not your sense of it.
+- "I'm being thorough" is **not** a justification. Thoroughness on the requested goal is required; thoroughness extending past the goal is drift.
+
+**When in doubt, ask — outside hands-free pipelines.** In interactive sessions a short clarification ("the requested fix touches the X code path; I notice Y also looks broken — should I fix it in this change or surface it as a follow-up?") is always cheaper than a wrong-scope diff. Asking is not a weakness; silently expanding scope is. **Inside hands-free pipelines** (`/devlyn:resolve`, scheduled remote agents, autonomous skill runs) the contract forbids mid-pipeline prompts — there asking is unsafe because there is no user to answer. The substitute is: stay strictly on the requested goal, do not expand scope, and log the question/assumption explicitly in the final report (or `.devlyn/runs/<run_id>/` artifacts) so the user can adjudicate after the run completes. Choosing scope creep over logging-and-staying-on-path is always wrong.
+
+**Stopping rule.** A task is done when the user's stated goal is closed AND no off-path work was added. If you find yourself hesitating because "I should also do Z" — Z is drift. Note it for follow-up, do not execute.
+<!-- runtime-principles:section=goal-locked:end -->
+
+## No-workaround discipline
+<!-- runtime-principles:section=no-workaround:begin -->
+
+No `any`, no `@ts-ignore`, no silent `catch`, no hardcoded values, no helper scripts that bypass the root cause. Fix root causes; handle errors with user-visible state per the rule above.
+
+**Permitted exceptions** (explicitly carved out):
+- CSS fallback fonts, CDN failover, image placeholders — widely-accepted best practices.
+- Codex CLI availability downgrade — the one documented silent fallback in this repo. Fires when the resolved engine is `auto` or `codex` (either via skill default or explicit `--engine` flag) and the Codex CLI is absent. Banner `engine downgraded: codex-unavailable` always prints; verdict identical to `--engine claude`. Any other silent fallback in skills code is a bug — file it against the skill that introduced it.
+<!-- runtime-principles:section=no-workaround:end -->
+
+## Evidence over claim
+<!-- runtime-principles:section=evidence:begin -->
+
+Every finding cites concrete evidence. Vague claims are speculation; exclude them.
+
+- **Code findings**: `file:line` you have opened.
+- **Missing findings**: explicit "searched X and found no implementation" statement.
+- **Doc findings**: quote of the stale text + section/line reference.
+- **Browser findings**: screenshot reference + URL/route.
+
+A finding without one of these forms is excluded. Vague findings produce vague fixes.
+<!-- runtime-principles:section=evidence:end -->
+<!-- runtime-principles:contract:end -->
+
+<!-- runtime-principles:consumption:begin -->
+## Consumption (as of iter-0019.A)
+
+**Consumers**:
+- `auto-resolve/SKILL.md` `<harness_principles>` block points here as the contract source. Phase prompt bodies (`phase-1-build.md`, `phase-2-evaluate.md`, `phase-3-critic.md`) inline a compact operational excerpt derived from the contract — phase-specific rule_id mappings + the four section names — not the full text.
+- `preflight/SKILL.md` PHASE 3 (Synthesize) and PHASE 3.5 (RND2) reference this file. Auditor prompts (`code-auditor.md`, `browser-auditor.md`) emit `principle.*` rule_ids derived from the rules above.
+
+**Codex routing**: skills that route to Codex (auto-resolve fix-loop on `--engine auto`/`codex`, preflight code-auditor on `--engine auto`/`codex`) MUST inline the contract excerpt directly into the Codex prompt — Codex has no filesystem access under `read-only` sandbox.
+
+**Non-consumers**:
+- `ideate/SKILL.md` does NOT consume this file. Ideate is planning-layer; its CHALLENGE rubric (`references/challenge-rubric.md`) covers analogous concerns at planning scope, with deliberate one-shot Codex critic discipline.
+<!-- runtime-principles:consumption:end -->

package/config/skills/_shared/spec-verify-check.py

@@ -0,0 +1,519 @@
+#!/usr/bin/env python3
+"""Spec literal verification gate (iter-0019.6 + iter-0019.8 + iter-0019.9
+carrier).
+
+Default mode (BUILD_GATE invocation, no args):
+- Resolves the contract carrier in this priority order (iter-0019.8 + Codex
+  R2 + iter-0019.9 Codex R-phaseA fix):
+  (1) **Benchmark mode trust** (iter-0019.9 fix for the F9 regression): when
+      `BENCH_WORKDIR` is set AND `.devlyn/spec-verify.json` already exists
+      at script start, trust it as the run-fixture.sh-staged contract from
+      `expected.json` and skip source-extract entirely. Without this guard,
+      an ideate-generated spec's `## Verification` ```json``` block (e.g.
+      F9 e2e novice flow generates `commitCount`/`topAuthors` while
+      benchmark truth is `commits`/`authors`) silently overwrote the
+      authoritative benchmark contract. For benchmarks, expected.json is
+      canonical.
+  (2) Otherwise, source markdown extract — read `pipeline.state.json:
+      source.{spec_path | criteria_path}` and extract a `## Verification`
+      ```json``` block. If present, overwrite `.devlyn/spec-verify.json`.
+      This is the real-user carrier path; a pre-existing file from a
+      killed prior run is stale and must not be trusted in real-user mode.
+  (3) If no json block in source AND source.type=="generated": emit
+      CRITICAL `correctness.spec-verify-malformed` so the fix-loop reruns
+      BUILD.
+  (4) If no json block in source AND source.type=="spec": benchmark mode
+      with a pre-staged file would have hit branch (1). Without the
+      pre-staged file, benchmark falls through to no-op (rare — fixture
+      mis-config). Real-user mode silent no-op + drops any stale
+      pre-staged file (preserves iter-0019.6 backward compat for
+      handwritten specs without the carrier).
+- For each verification_commands entry, runs the command in the work-dir,
+  captures combined stdout+stderr, and asserts exit_code matches +
+  stdout_contains all required literals + stdout_not_contains none of the
+  forbidden literals. Mirrors run-fixture.sh's post-run verifier semantics.
+
+Check mode (`--check <markdown_path>`):
+- Used by /devlyn:ideate after writing each item spec to validate that the
+  generated `## Verification` ```json``` block parses + matches the schema.
+- Exits 0 if the block is well-formed (or absent — ideate's check applies
+  to both new specs that include the block and pre-carrier handwritten
+  specs that omit it; absence is not failure here, only malformed JSON or
+  shape error is). Exits 2 on malformed json or shape error.
+
+Why: iter-0018.5's prompt-only contract enforcement was empirically dead
+(F9 verify=0.4 across all engines in iter-0019). Same lesson as iter-0008
+prompt-only engine constraint. Mechanical bash-gate enforcement is the
+only working pattern. iter-0019.8 extends iter-0019.6 from benchmark-only
+to real-user runs by extracting the contract from the spec/criteria
+markdown directly — closes NORTH-STAR test #14.
+
+Exit codes:
+- 0: silent no-op (no source carrier, real-user mode) OR --check passed
+  OR all commands passed.
+- 1: at least one command failed OR carrier malformed (generated source
+  required carrier, generated source had invalid json/shape, or pre-staged
+  file failed shape validation). All paths emit a CRITICAL finding to
+  `.devlyn/spec-verify-findings.jsonl`.
+- 2: invocation error (unreadable spec-verify.json, missing markdown in
+  --check mode, etc.)
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import re
+import subprocess
+import sys
+from pathlib import Path
+
+
+VERIFICATION_SECTION_RE = re.compile(
+    r'(?ms)^##[ \t]+Verification\b[^\n]*\n(.*?)(?=^##[ \t]+|\Z)'
+)
+JSON_FENCE_RE = re.compile(r'(?ms)^```json[ \t]*\n(.*?)\n```[ \t]*$')
+
+
+def extract_verification_block(text: str) -> str | None:
+    """Return the contents of the first ```json``` fenced block under the
+    first `## Verification` H2 heading, or None if not found.
+
+    Boundary: the fenced block must appear AFTER the `## Verification`
+    heading and BEFORE the next H2 (`## ...`) heading or end-of-file.
+    """
+    section = VERIFICATION_SECTION_RE.search(text)
+    if not section:
+        return None
+    fence = JSON_FENCE_RE.search(section.group(1))
+    return fence.group(1) if fence else None
+
+
+def validate_shape(data) -> str | None:
+    """Return None if shape matches the canonical verification_commands
+    schema; else a human-readable error string.
+
+    Schema (iter-0019.8): top-level object with a non-empty
+    `verification_commands` list of objects. Each object requires a
+    non-empty string `cmd`; `exit_code` defaults to 0 and must be a
+    non-bool int; `stdout_contains` and `stdout_not_contains` default to
+    empty list and must be lists of strings. Bool is rejected explicitly
+    because Python's `bool` subclasses `int` — `isinstance(True, int) is
+    True` would otherwise let `exit_code: true` slip through.
+    """
+    if not isinstance(data, dict):
+        return "top-level must be a JSON object"
+    cmds = data.get("verification_commands")
+    if not isinstance(cmds, list):
+        return "verification_commands must be a list"
+    if not cmds:
+        return "verification_commands must contain at least one entry"
+    for i, c in enumerate(cmds):
+        if not isinstance(c, dict):
+            return f"verification_commands[{i}] must be an object"
+        cmd = c.get("cmd")
+        if not isinstance(cmd, str) or not cmd.strip():
+            return f"verification_commands[{i}].cmd must be a non-empty string"
+        ec = c.get("exit_code", 0)
+        if isinstance(ec, bool) or not isinstance(ec, int):
+            return f"verification_commands[{i}].exit_code must be int (not bool)"
+        for k in ("stdout_contains", "stdout_not_contains"):
+            v = c.get(k, [])
+            if not isinstance(v, list) or not all(isinstance(s, str) for s in v):
+                return f"verification_commands[{i}].{k} must be a list of strings"
+    return None
+
+
+def read_source(work: Path, devlyn_dir: Path) -> tuple[str | None, Path | None]:
+    """Return (source_type, markdown_path) from .devlyn/pipeline.state.json,
+    or (None, None) if state is absent/unreadable. The markdown path is
+    resolved against `work` when relative.
+    """
+    state_path = devlyn_dir / "pipeline.state.json"
+    if not state_path.is_file():
+        return (None, None)
+    try:
+        state = json.loads(state_path.read_text())
+    except (json.JSONDecodeError, OSError):
+        return (None, None)
+    src = state.get("source") or {}
+    src_type = src.get("type")
+    if src_type == "spec":
+        md_path = src.get("spec_path")
+    elif src_type == "generated":
+        md_path = src.get("criteria_path")
+    else:
+        md_path = None
+    if not md_path:
+        return (src_type, None)
+    md = Path(md_path)
+    if not md.is_absolute():
+        md = work / md
+    return (src_type, md if md.is_file() else None)
+
+
+def stage_from_source(md: Path, devlyn_dir: Path) -> tuple[bool, str | None]:
+    """Materialize .devlyn/spec-verify.json from the json block in `md`.
+
+    Returns (staged, error). staged=True → wrote spec-verify.json. error
+    non-None → carrier was found but malformed (caller emits CRITICAL).
+    staged=False, error=None → no json block in the source (handwritten
+    spec or generated source missing the contract).
+    """
+    block = extract_verification_block(md.read_text())
+    if block is None:
+        return (False, None)
+    try:
+        data = json.loads(block)
+    except json.JSONDecodeError as e:
+        return (False, f"`## Verification` ```json``` block in {md} has invalid JSON: {e}")
+    err = validate_shape(data)
+    if err:
+        return (False, f"`## Verification` ```json``` block in {md}: {err}")
+    normalized = {"verification_commands": data["verification_commands"]}
+    devlyn_dir.mkdir(parents=True, exist_ok=True)
+    (devlyn_dir / "spec-verify.json").write_text(json.dumps(normalized, indent=2) + "\n")
+    return (True, None)
+
+
+def write_malformed_finding(devlyn_dir: Path, error: str, source_path: Path | None) -> None:
+    """Emit a single CRITICAL finding for a malformed verification carrier."""
+    devlyn_dir.mkdir(parents=True, exist_ok=True)
+    findings_path = devlyn_dir / "spec-verify-findings.jsonl"
+    file_ref = str(source_path) if source_path else ".devlyn/pipeline.state.json"
+    finding = {
+        "id": "BGATE-0001",
+        "rule_id": "correctness.spec-verify-malformed",
+        "level": "error",
+        "severity": "CRITICAL",
+        "confidence": 1.0,
+        "message": f"Verification contract carrier is malformed: {error}",
+        "file": file_ref,
+        "line": 1,
+        "phase": "build_gate",
+        "criterion_ref": "spec-verify://carrier",
+        "fix_hint": (
+            "Fix the `## Verification` ```json``` block: a JSON object with "
+            "a non-empty `verification_commands` array of "
+            "{cmd, exit_code?, stdout_contains?, stdout_not_contains?} "
+            "entries. See references/build-gate.md § 'Spec literal check'."
+        ),
+        "blocking": True,
+        "status": "open",
+    }
+    with findings_path.open("w") as fh:
+        fh.write(json.dumps(finding) + "\n")
+
+
+def run_check_mode(md_path: Path) -> int:
+    """`--check <markdown>` — validate the verification carrier without
+    running any commands. Used by /devlyn:ideate after item-spec write.
+
+    Exit 0: section absent OR section present and well-formed.
+    Exit 2: section present but malformed (so ideate can re-prompt).
+    """
+    if not md_path.is_file():
+        print(f"[spec-verify --check] error: {md_path} not found", file=sys.stderr)
+        return 2
+    block = extract_verification_block(md_path.read_text())
+    if block is None:
+        # Section absent or no json block — opt-in nature preserved for
+        # ideate (a spec without machine verification is still valid; it
+        # just won't activate the BUILD_GATE gate).
+        return 0
+    try:
+        data = json.loads(block)
+    except json.JSONDecodeError as e:
+        print(
+            f"[spec-verify --check] {md_path}: invalid JSON in `## Verification` "
+            f"```json``` block: {e}",
+            file=sys.stderr,
+        )
+        return 2
+    err = validate_shape(data)
+    if err:
+        print(f"[spec-verify --check] {md_path}: shape error: {err}", file=sys.stderr)
+        return 2
+    return 0
+
+
+def main() -> int:
+    if len(sys.argv) >= 2 and sys.argv[1] == "--check":
+        if len(sys.argv) != 3:
+            print("usage: spec-verify-check.py --check <markdown-path>", file=sys.stderr)
+            return 2
+        return run_check_mode(Path(sys.argv[2]))
+
+    bench_mode = "BENCH_WORKDIR" in os.environ
+    work = Path(os.environ.get("BENCH_WORKDIR") or os.getcwd())
+    devlyn_dir = work / ".devlyn"
+    spec_path = devlyn_dir / "spec-verify.json"
+
+    # iter-0019.8 + iter-0019.9 (Codex R-phaseA): determine the contract
+    # carrier source for THIS run. Order:
+    #   1. Benchmark mode (BENCH_WORKDIR set) AND a pre-staged
+    #      .devlyn/spec-verify.json exists at script start: TRUST it (this is
+    #      the run-fixture.sh contract staged from expected.json). Skip
+    #      source-extract entirely. iter-0019.9 closes the F9 regression where
+    #      source-extract from an ideate-generated spec overwrote the
+    #      benchmark contract — for benchmarks, expected.json is canonical.
+    #   2. Otherwise, attempt source-extract from
+    #      `pipeline.state.json:source.{spec_path | criteria_path}`. If it has
+    #      a json block, overwrite .devlyn/spec-verify.json with it. This is
+    #      the real-user carrier path; in real-user mode a pre-existing file
+    #      is stale (from a killed prior run) and must NOT be trusted.
+    #   3. If source has no json block AND source.type=="generated":
+    #      CRITICAL spec-verify-malformed — generated criteria must ship a
+    #      verifiable contract per phase-1-build.md <output_contract>.
+    #   4. If source has no json block AND source.type=="spec":
+    #      - Real-user mode: silent no-op (preserves iter-0019.6 backward
+    #        compat for handwritten specs without the carrier). Drop any
+    #        stale pre-staged file.
+    #      - Benchmark mode: fall through to the pre-staged-trust branch
+    #        (covers pre-iter-0019.9 fixtures whose spec.md has prose-only
+    #        Verification — run-fixture.sh staged the contract regardless).
+    pre_staged = spec_path.is_file()  # captured BEFORE any potential write
+    trust_bench_staged = bench_mode and pre_staged
+    src_type, source_md = read_source(work, devlyn_dir)
+    if source_md is not None and not trust_bench_staged:
+        staged, error = stage_from_source(source_md, devlyn_dir)
+        if error is not None:
+            print(f"[spec-verify] carrier malformed: {error}", file=sys.stderr)
+            write_malformed_finding(devlyn_dir, error, source_md)
+            return 1
+        if not staged:
+            if src_type == "generated":
+                msg = (
+                    f"generated {source_md.name} must include a "
+                    "`## Verification` ```json``` block (verification_commands "
+                    "array). PHASE 1 BUILD generated criteria without one."
+                )
+                print(f"[spec-verify] {msg}", file=sys.stderr)
+                write_malformed_finding(devlyn_dir, msg, source_md)
+                return 1
+            # source.type=="spec", no block in spec markdown.
+            if not bench_mode:
+                # Real-user handwritten spec: silent no-op. Drop any stale
+                # pre-staged file so a killed prior run cannot poison this
+                # run's gate.
+                if spec_path.exists():
+                    spec_path.unlink()
+                return 0
+            # Benchmark mode with no source block AND no pre-staged file
+            # (rare — fixture mis-config) falls through to the no-pre-staged
+            # silent no-op branch below.
+
+    # iter-0019.9 (Codex R2 caveat): close the real-user no-source-md
+    # stale-orphan gap. If pipeline.state.json is absent or has no source,
+    # but a stale .devlyn/spec-verify.json exists in real-user mode, drop
+    # it — the only legitimate path that reaches here with a pre-staged
+    # file is benchmark mode (run-fixture.sh staged it).
+    if source_md is None and not bench_mode and spec_path.exists():
+        spec_path.unlink()
+        return 0
+
+    if not spec_path.exists():
+        # No source markdown carrier AND no pre-staged file. Silent no-op
+        # for benchmark misconfigurations (no fixture to gate against) and
+        # for real-user runs without spec/criteria. Generated source case
+        # is handled above.
+        return 0
+
+    try:
+        spec = json.loads(spec_path.read_text())
+    except (json.JSONDecodeError, OSError) as e:
+        print(f"[spec-verify] error: cannot parse {spec_path}: {e}", file=sys.stderr)
+        return 2
+
+    # iter-0019.8 (Codex R2 #2): apply full shape validation to pre-staged
+    # carriers too — bool exit_code, empty list, whitespace-only cmd were
+    # silently accepted on the benchmark path. Empty list is rejected
+    # because "all 0 commands passed" is vacuously true.
+    shape_err = validate_shape(spec)
+    if shape_err:
+        print(f"[spec-verify] error: {spec_path}: {shape_err}", file=sys.stderr)
+        write_malformed_finding(devlyn_dir, f"{spec_path}: {shape_err}", None)
+        return 1
+    commands = spec["verification_commands"]
+
+    devlyn_dir.mkdir(parents=True, exist_ok=True)
+    results_path = devlyn_dir / "spec-verify.results.json"
+    findings_path = devlyn_dir / "spec-verify-findings.jsonl"
+
+    verify_env = os.environ.copy()
+    verify_env["BENCH_WORKDIR"] = str(work)
+
+    results: list[dict] = []
+    findings: list[dict] = []
+    finding_seq = 1
+
+    for idx, vc in enumerate(commands):
+        cmd = vc.get("cmd")
+        if not cmd:
+            results.append({"index": idx, "cmd": None, "pass": False,
+                            "reason": "missing_cmd"})
+            continue
+
+        expected_exit = vc.get("exit_code", 0)
+        stdout_contains = vc.get("stdout_contains", []) or []
+        stdout_not_contains = vc.get("stdout_not_contains", []) or []
+
+        try:
+            proc = subprocess.run(
+                cmd,
+                cwd=str(work),
+                shell=True,
+                env=verify_env,
+                capture_output=True,
+                text=True,
+                timeout=60,
+            )
+            # Mirror run-fixture.sh post-run verifier: combined stdout+stderr.
+            out = (proc.stdout or "") + (proc.stderr or "")
+            ok_exit = proc.returncode == expected_exit
+            ok_contains = all(s in out for s in stdout_contains)
+            ok_not = not any(s in out for s in stdout_not_contains)
+            passed = bool(ok_exit and ok_contains and ok_not)
+
+            if passed:
+                reason = None
+            elif not ok_exit:
+                reason = "exit"
+            elif not ok_contains:
+                reason = "missing_contains"
+            else:
+                reason = "unexpected_text"
+
+            results.append({
+                "index": idx,
+                "cmd": cmd,
+                "expected_exit": expected_exit,
+                "actual_exit": proc.returncode,
+                "stdout_contains": stdout_contains,
+                "stdout_not_contains": stdout_not_contains,
+                "pass": passed,
+                "reason": reason,
+                "stdout_tail": out[-500:],
+            })
+
+            if not passed:
+                # Construct fine-grained message naming the specific failure.
+                if not ok_exit:
+                    msg = (
+                        f"Verification command #{idx + 1} failed: expected exit "
+                        f"{expected_exit}, got {proc.returncode}."
+                    )
+                elif not ok_contains:
+                    missing = [s for s in stdout_contains if s not in out]
+                    msg = (
+                        f"Verification command #{idx + 1} failed: expected "
+                        f"output to contain {missing!r}."
+                    )
+                else:
+                    forbidden = [s for s in stdout_not_contains if s in out]
+                    msg = (
+                        f"Verification command #{idx + 1} failed: output "
+                        f"contained forbidden literal(s) {forbidden!r}."
+                    )
+
+                fix_hint = (
+                    f"See .devlyn/spec-verify.results.json for the captured "
+                    f"output. Update implementation so `{cmd}` matches the "
+                    f"contract (exit_code={expected_exit}, "
+                    f"contains={stdout_contains}, not_contains={stdout_not_contains})."
+                )
+
+                findings.append({
+                    "id": f"BGATE-{finding_seq:04d}",
+                    "rule_id": "correctness.spec-literal-mismatch",
+                    "level": "error",
+                    "severity": "CRITICAL",
+                    "confidence": 1.0,
+                    "message": msg,
+                    "file": ".devlyn/spec-verify.json",
+                    "line": 1,
+                    "phase": "build_gate",
+                    "criterion_ref": f"spec-verify://verification_commands/{idx}",
+                    "fix_hint": fix_hint,
+                    "blocking": True,
+                    "status": "open",
+                })
+                finding_seq += 1
+
+        except subprocess.TimeoutExpired:
+            results.append({"index": idx, "cmd": cmd, "pass": False,
+                            "reason": "timeout"})
+            findings.append({
+                "id": f"BGATE-{finding_seq:04d}",
+                "rule_id": "correctness.spec-literal-mismatch",
+                "level": "error",
+                "severity": "CRITICAL",
+                "confidence": 1.0,
+                "message": (
+                    f"Verification command #{idx + 1} timed out after 60s."
+                ),
+                "file": ".devlyn/spec-verify.json",
+                "line": 1,
+                "phase": "build_gate",
+                "criterion_ref": f"spec-verify://verification_commands/{idx}",
+                "fix_hint": (
+                    f"Command `{cmd}` exceeded 60s. Reduce work or fix a "
+                    f"hang in the implementation."
+                ),
+                "blocking": True,
+                "status": "open",
+            })
+            finding_seq += 1
+        except Exception as e:  # noqa: BLE001 — surface any harness error explicitly
+            results.append({"index": idx, "cmd": cmd, "pass": False,
+                            "reason": f"error:{e.__class__.__name__}:{e}"})
+            findings.append({
+                "id": f"BGATE-{finding_seq:04d}",
+                "rule_id": "correctness.spec-literal-mismatch",
+                "level": "error",
+                "severity": "CRITICAL",
+                "confidence": 1.0,
+                "message": (
+                    f"Verification command #{idx + 1} raised "
+                    f"{e.__class__.__name__}: {e}."
+                ),
+                "file": ".devlyn/spec-verify.json",
+                "line": 1,
+                "phase": "build_gate",
+                "criterion_ref": f"spec-verify://verification_commands/{idx}",
+                "fix_hint": (
+                    f"Command `{cmd}` could not be executed. Check the work-dir "
+                    f"state and any environment setup the command requires."
+                ),
+                "blocking": True,
+                "status": "open",
+            })
+            finding_seq += 1
+
+    results_path.write_text(json.dumps({"commands": results}, indent=2) + "\n")
+
+    # Append findings (jsonl). BUILD_GATE merge step concatenates this onto
+    # build_gate.findings.jsonl; never overwrite the orchestrator's own gate
+    # findings. Truncate this file each run since it is a per-round artifact.
+    with findings_path.open("w") as fh:
+        for f in findings:
+            fh.write(json.dumps(f) + "\n")
+
+    failed = [r for r in results if r.get("pass") is False]
+    if failed:
+        print(
+            f"[spec-verify] {len(failed)}/{len(results)} command(s) failed; "
+            f"{len(findings)} CRITICAL finding(s) written to {findings_path}",
+            file=sys.stderr,
+        )
+        return 1
+
+    print(
+        f"[spec-verify] all {len(results)} command(s) passed",
+        file=sys.stderr,
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())