devlyn-cli 1.15.0 → 2.1.0

package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/spec.md

@@ -0,0 +1,45 @@
+---
+id: "F1-cli-trivial-flag"
+title: "Add --loud flag to hello subcommand"
+status: planned
+complexity: trivial
+depends-on: []
+---
+
+# F1 Add `--loud` flag to `hello`
+
+## Context
+
+The `hello` subcommand in `bin/cli.js` currently prints `Hello, <name>!`. A
+`--loud` flag gives users an emphatic variant without breaking the default.
+This is a low-risk edit used to calibrate trivial-tier fixture difficulty.
+
+## Requirements
+
+- [ ] `node bin/cli.js hello --loud` prints `HELLO, WORLD!!` (everything uppercased, two trailing exclamation marks).
+- [ ] `node bin/cli.js hello --loud --name alice` prints `HELLO, ALICE!!`.
+- [ ] `node bin/cli.js hello` (no flag) still prints `Hello, world!` (unchanged).
+- [ ] `node bin/cli.js hello --name bob` still prints `Hello, bob!` (unchanged).
+- [ ] Existing tests continue to pass. Add at least one test covering the `--loud` path.
+
+## Constraints
+
+- **No new npm dependencies.** Built-ins only.
+- **No silent catches.** If an unknown flag is passed, exit 1 with an informative message (same pattern as the existing `--name` handler).
+- **Surgical diff.** Only touch `bin/cli.js` and `tests/cli.test.js`. Do not reformat unrelated code.
+
+- **Lifecycle note.** The harness's DOCS phase flips this spec's frontmatter `status` after implementation completes — that is benchmark lifecycle bookkeeping, not a scope violation.
+
+## Out of Scope
+
+- Adding unrelated flags (`--quiet`, `--locale`, etc.).
+- Refactoring the existing argument parser.
+- Touching `server/`, `web/`, or `tests/server.test.js`.
+
+## Verification
+
+- `node bin/cli.js hello` prints `Hello, world!` (exit 0).
+- `node bin/cli.js hello --loud` prints `HELLO, WORLD!!` (exit 0).
+- `node bin/cli.js hello --loud --name alice` prints `HELLO, ALICE!!` (exit 0).
+- `node --test tests/` passes all tests including the new `--loud` case.
+- `git diff --stat` shows only `bin/cli.js` and `tests/cli.test.js` touched.

package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/task.txt

@@ -0,0 +1,8 @@
+Add a --loud flag to the `hello` subcommand in bench-test-repo's CLI (bin/cli.js). When --loud is passed, the greeting is uppercased and ends with two exclamation marks.
+
+For example:
+- `node bin/cli.js hello --loud` → `HELLO, WORLD!!`
+- `node bin/cli.js hello --loud --name alice` → `HELLO, ALICE!!`
+- `node bin/cli.js hello` → `Hello, world!` (unchanged default)
+
+Make sure existing tests still pass and add at least one test for the --loud path. Don't touch unrelated files — only `bin/cli.js` and `tests/cli.test.js`. No new npm dependencies.

package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/NOTES.md

@@ -0,0 +1,54 @@
+# F2 — Notes
+
+## Purpose
+
+Canonical **medium-complexity single-file CLI task** in the suite. Tests the
+middle-ground: a task big enough that first-draft implementations often miss
+an edge case (EACCES vs missing-dir distinction, TTY gating, HOME guard),
+small enough that every arm can plausibly finish in < 10 minutes.
+
+## What failure mode does it detect?
+
+- **Silent catches.** The pattern `try { readdirSync(...) } catch { return [] }`
+  is a natural shortcut here. Bare prompt arms tend to take it. The pipeline's
+  EVAL phase catches it as a `correctness.silent-error` or
+  `hygiene.silent-catch` finding.
+- **Edge-case distinction.** ENOENT vs EACCES must be reported differently.
+  Arms that collapse both into a generic FAIL miss a spec Requirement.
+- **Over-engineering.** Since v3.6's CRITIC calibration, hand-rolled
+  mode-bit writable checks are blocked in favor of `fs.accessSync(...,
+  fs.constants.W_OK)`.
+
+## Which pipeline phases does it exercise?
+
+- Phase 0: routing — `permission`, `env` risk keywords in the task body
+  escalate to `strict`.
+- Phase 1 BUILD: main implementation pass.
+- Phase 1.4 BUILD GATE: `node --check` syntax gate.
+- Phase 2 EVAL: catches silent-catch trap if present.
+- Phase 3 CRITIC design: applies stdlib-vs-hand-rolled calibration.
+- Phase 3 CRITIC security (native): minimal — no deps changed.
+- Phase 4 DOCS: spec frontmatter `status: done`.
+
+## Why can't another fixture cover this?
+
+- F1 is trivial (single-line edit, no edge cases).
+- F3 is backend (different idioms, tests run differently).
+- F5 is designed to force fix-loop (not applicable here).
+- F7 is scope-creep (orthogonal concern).
+
+## When should this fixture be retired or replaced?
+
+When both arms score > 95 for two consecutive shipped versions — i.e., the
+fixture saturates and no longer differentiates. Candidate replacement: a
+similar-size CLI task with multiple interacting flags or a subcommand that
+spawns a child process.
+
+## Calibration history
+
+- v3.4   skill 57 / bare 45 / margin +12 (gpt-5.3-codex judge)
+- v3.4.1 skill 59 / bare 43 / margin +16 (gpt-5.3-codex judge)
+- v3.5   skill 92 / bare 81 / margin +11 (gpt-5.4 xhigh judge) — huge absolute jump; bare silent-catch caught
+
+Absolute scores jumped with the stronger judge. Margin stays solid (+11
+after stdlib calibration is expected to open a few points more).

package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/expected-pair-plan-registry.json

@@ -0,0 +1,170 @@
+{
+  "fixture_id": "F2-cli-medium-subcommand",
+  "generated_at": "2026-04-29T09:57:53Z",
+  "generated_from": {
+    "expected_path": "benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/expected.json",
+    "expected_sha256": "ddef8feba49f20b6957e37840bc6a03e78e554776e380d81ad6390944c72fcab",
+    "metadata_path": "benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/metadata.json",
+    "metadata_sha256": "1b8066a7c649eb6baad7a3e056edbdb16cc3b796e154cedee0cf2258c5543b18",
+    "oracle_script_shas": {
+      "scope-tier-a": "baaf21ed4a67f35d2a8af825e72869ef9737b5dfe08d65dd1a11c26fafe297ae",
+      "scope-tier-b": "9349d00a5c7456a4df9142923334e7004407d53f2443f2e210945bb771971e25",
+      "test-fidelity": "401184da51ae500cecfc75a6c5819b0d28acb63a397f788fb628c2913562f903"
+    }
+  },
+  "required_invariants": [
+    {
+      "authority": "expected.json/forbidden_patterns",
+      "id": "forbidden_pattern__silent_catch_returning_a_fallback_value_violates_no_silent_c__bin_cli_js",
+      "operational_check": "variant arm output MUST NOT contain regex pattern \"catch\\\\s*\\\\([^)]*\\\\)\\\\s*\\\\{[^}]*return\\\\s+(\\\\[\\\\]|null|undefined|\\\\{|false|'')\" in files ['bin/cli.js']; rationale: silent catch returning a fallback value — violates no-silent-catches policy",
+      "severity": "disqualifier",
+      "source_field": "expected.json/forbidden_patterns/0",
+      "source_ref": "expected.json:forbidden_patterns[0]"
+    },
+    {
+      "authority": "expected.json/forbidden_patterns",
+      "id": "forbidden_pattern__ts_ignore_escape_hatch__bin_cli_js",
+      "operational_check": "variant arm output MUST NOT contain regex pattern '@ts-ignore' in files ['bin/cli.js']; rationale: @ts-ignore escape hatch",
+      "severity": "disqualifier",
+      "source_field": "expected.json/forbidden_patterns/1",
+      "source_ref": "expected.json:forbidden_patterns[1]"
+    },
+    {
+      "authority": "expected.json/max_deps_added",
+      "id": "max_deps_added__0",
+      "operational_check": "variant arm MUST NOT add more than 0 new npm dependencies (count delta of package.json:dependencies + devDependencies)",
+      "severity": "hard",
+      "source_field": "expected.json/max_deps_added",
+      "source_ref": "expected.json:max_deps_added"
+    },
+    {
+      "authority": "expected.json/required_files",
+      "id": "required_file__bin_cli_js",
+      "operational_check": "variant arm output MUST contain file 'bin/cli.js' (created or preserved)",
+      "severity": "hard",
+      "source_field": "expected.json/required_files",
+      "source_ref": "expected.json:required_files[bin/cli.js]"
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "scope-tier-a:lockfile-deletion",
+      "operational_check": "variant arm MUST NOT delete a scaffold-present lockfile",
+      "severity": "hard",
+      "source_field": "oracle/scope-tier-a/scope-tier-a:lockfile-deletion",
+      "source_ref": "oracle-scope-tier-a.py"
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "scope-tier-a:tier-a-violation",
+      "operational_check": "variant arm MUST NOT add or modify paths matching: docs/roadmap/** | docs/VISION.md | docs/ROADMAP.md | .github/** | node_modules/** | **/node_modules/** | test-results/** | coverage/** | .nyc_output/** | basename suffix .log | basename prefix .env or secrets.",
+      "severity": "hard",
+      "source_field": "oracle/scope-tier-a/scope-tier-a:tier-a-violation",
+      "source_ref": "oracle-scope-tier-a.py"
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "scope-tier-b:scope-unmatched",
+      "operational_check": "every variant-touched file MUST be either inside spec_output_files (Tier C) OR reachable from a Tier C seed via static JS/TS imports OR matched by expected.json:tier_a_waivers",
+      "severity": "warn",
+      "source_field": "oracle/scope-tier-b/scope-tier-b:scope-unmatched",
+      "source_ref": "oracle-scope-tier-b.py"
+    },
+    {
+      "authority": "expected.json/spec_output_files",
+      "id": "spec_output_file__bin_cli_js",
+      "operational_check": "variant-touched files MUST be inside (or reachable via static imports from) the spec_output_files set; 'bin/cli.js' is one Tier C seed",
+      "severity": "warn",
+      "source_field": "expected.json/spec_output_files",
+      "source_ref": "expected.json:spec_output_files[bin/cli.js]"
+    },
+    {
+      "authority": "expected.json/spec_output_files",
+      "id": "spec_output_file__tests_cli_test_js",
+      "operational_check": "variant-touched files MUST be inside (or reachable via static imports from) the spec_output_files set; 'tests/cli.test.js' is one Tier C seed",
+      "severity": "warn",
+      "source_field": "expected.json/spec_output_files",
+      "source_ref": "expected.json:spec_output_files[tests/cli.test.js]"
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "test-fidelity:assertion-regression",
+      "operational_check": "effective assertion count MUST NOT drop and skipped-test count MUST NOT rise; vacuous expect.assertions(0) is treated as a real regression",
+      "severity": "warn",
+      "source_field": "oracle/test-fidelity/test-fidelity:assertion-regression",
+      "source_ref": "oracle-test-fidelity.py"
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "test-fidelity:mock-swap",
+      "operational_check": "post-arm test file MUST NOT swap REAL_PATTERNS hits for MOCK_PATTERNS hits (jest/vi/sinon, nock/msw, app.handle/inject/callback, hand-rolled IncomingMessage/ServerResponse, etc.); a drop in real_calls combined with a rise in mock_calls is a mock-swap flag",
+      "severity": "flag",
+      "source_field": "oracle/test-fidelity/test-fidelity:mock-swap",
+      "source_ref": "oracle-test-fidelity.py"
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "test-fidelity:test-file-deleted",
+      "operational_check": "no scaffold-present test file may be deleted by the variant arm; deletion of an existing tests/*.test.* / *.spec.* / *.e2e.* file is a flag-severity finding",
+      "severity": "flag",
+      "source_field": "oracle/test-fidelity/test-fidelity:test-file-deleted",
+      "source_ref": "oracle-test-fidelity.py"
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "test-fidelity:test-file-renamed",
+      "operational_check": "rename of a scaffold-present test file is warn-severity (content fidelity not verified across renames in step 1)",
+      "severity": "warn",
+      "source_field": "oracle/test-fidelity/test-fidelity:test-file-renamed",
+      "source_ref": "oracle-test-fidelity.py"
+    },
+    {
+      "authority": "expected.json/verification_commands",
+      "id": "verification__3f35982a",
+      "operational_check": "running `node bin/cli.js doctor` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['doctor:']; stdout MUST NOT contain any of ['undefined', 'Error:']",
+      "severity": "hard",
+      "source_field": "expected.json/verification_commands/0",
+      "source_ref": "expected.json:verification_commands[0]"
+    },
+    {
+      "authority": "expected.json/verification_commands",
+      "id": "verification__460fce04",
+      "operational_check": "running `HOME=/nonexistent node bin/cli.js doctor` in the post-arm work dir MUST exit with code 1; stdout MUST contain all of ['/nonexistent']; stdout MUST NOT contain any of []",
+      "severity": "hard",
+      "source_field": "expected.json/verification_commands/1",
+      "source_ref": "expected.json:verification_commands[1]"
+    },
+    {
+      "authority": "expected.json/verification_commands",
+      "id": "verification__973e287e",
+      "operational_check": "running `python3 -c \"import subprocess; r = subprocess.run(['node', 'bin/cli.js', 'doctor'], capture_output=True); n = r.stdout.count(b'\\x1b['); print(n); exit(0 if n == 0 else 1)\"` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['0']; stdout MUST NOT contain any of []",
+      "severity": "hard",
+      "source_field": "expected.json/verification_commands/2",
+      "source_ref": "expected.json:verification_commands[2]"
+    },
+    {
+      "authority": "expected.json/verification_commands",
+      "id": "verification__d6253a97",
+      "operational_check": "running `node bin/cli.js doctor --help` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['doctor']; stdout MUST NOT contain any of []",
+      "severity": "hard",
+      "source_field": "expected.json/verification_commands/3",
+      "source_ref": "expected.json:verification_commands[3]"
+    },
+    {
+      "authority": "expected.json/verification_commands",
+      "id": "verification__e0f149e4",
+      "operational_check": "running `node bin/cli.js --help` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['doctor']; stdout MUST NOT contain any of []",
+      "severity": "hard",
+      "source_field": "expected.json/verification_commands/4",
+      "source_ref": "expected.json:verification_commands[4]"
+    },
+    {
+      "authority": "expected.json/verification_commands",
+      "id": "verification__fdbcd321",
+      "operational_check": "running `node bin/cli.js doctor --verbose` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['doctor:']; stdout MUST NOT contain any of ['Error:']",
+      "severity": "hard",
+      "source_field": "expected.json/verification_commands/5",
+      "source_ref": "expected.json:verification_commands[5]"
+    }
+  ],
+  "schema_version": "1"
+}

package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/expected.json

@@ -0,0 +1,84 @@
+{
+  "verification_commands": [
+    {
+      "cmd": "node bin/cli.js doctor",
+      "exit_code": 0,
+      "stdout_contains": [
+        "doctor:"
+      ],
+      "stdout_not_contains": [
+        "undefined",
+        "Error:"
+      ]
+    },
+    {
+      "cmd": "HOME=/nonexistent node bin/cli.js doctor",
+      "exit_code": 1,
+      "stdout_contains": [
+        "/nonexistent"
+      ],
+      "stdout_not_contains": []
+    },
+    {
+      "cmd": "python3 -c \"import subprocess; r = subprocess.run(['node', 'bin/cli.js', 'doctor'], capture_output=True); n = r.stdout.count(b'\\x1b['); print(n); exit(0 if n == 0 else 1)\"",
+      "exit_code": 0,
+      "stdout_contains": [
+        "0"
+      ],
+      "stdout_not_contains": []
+    },
+    {
+      "cmd": "node bin/cli.js doctor --help",
+      "exit_code": 0,
+      "stdout_contains": [
+        "doctor"
+      ],
+      "stdout_not_contains": []
+    },
+    {
+      "cmd": "node bin/cli.js --help",
+      "exit_code": 0,
+      "stdout_contains": [
+        "doctor"
+      ],
+      "stdout_not_contains": []
+    },
+    {
+      "cmd": "node bin/cli.js doctor --verbose",
+      "exit_code": 0,
+      "stdout_contains": [
+        "doctor:"
+      ],
+      "stdout_not_contains": [
+        "Error:"
+      ]
+    }
+  ],
+  "forbidden_patterns": [
+    {
+      "pattern": "catch\\s*\\([^)]*\\)\\s*\\{[^}]*return\\s+(?:\\[\\]|null|undefined|false|''|\\{\\s*\\})",
+      "description": "silent catch returning a fallback value (null / undefined / [] / false / '' / empty {}) \u2014 violates no-silent-catches policy. Structured error returns like `return { level: 'fail', message }` are NOT silent (they surface a user-visible failure object) and must not match.",
+      "files": [
+        "bin/cli.js"
+      ],
+      "severity": "disqualifier"
+    },
+    {
+      "pattern": "@ts-ignore",
+      "description": "@ts-ignore escape hatch",
+      "files": [
+        "bin/cli.js"
+      ],
+      "severity": "disqualifier"
+    }
+  ],
+  "required_files": [
+    "bin/cli.js"
+  ],
+  "forbidden_files": [],
+  "max_deps_added": 0,
+  "spec_output_files": [
+    "bin/cli.js",
+    "tests/cli.test.js"
+  ]
+}

package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/metadata.json

@@ -0,0 +1,21 @@
+{
+  "id": "F2-cli-medium-subcommand",
+  "category": "medium",
+  "difficulty": "medium",
+  "timeout_seconds": 1500,
+  "required_tools": [
+    "node"
+  ],
+  "browser": false,
+  "deps_change_expected": false,
+  "intent": "Add a `doctor` subcommand to bin/cli.js that diagnoses the local environment: node version check, $HOME/.claude directory check, installed plugins count, installed skills count, TTY-gated ANSI color, summary line, exit code, --verbose flag, help integration. Zero new npm dependencies. No silent error catches.",
+  "pair_plan_oracle_categories": [
+    "scope-tier-a:lockfile-deletion",
+    "scope-tier-a:tier-a-violation",
+    "scope-tier-b:scope-unmatched",
+    "test-fidelity:assertion-regression",
+    "test-fidelity:mock-swap",
+    "test-fidelity:test-file-deleted",
+    "test-fidelity:test-file-renamed"
+  ]
+}

package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/pair-plan.sample-fail.json

@@ -0,0 +1,214 @@
+{
+  "accepted_invariants": [
+    {
+      "authority": "expected.json/forbidden_patterns",
+      "id": "forbidden_pattern__ts_ignore_escape_hatch__bin_cli_js",
+      "operational_check": "variant arm output MUST NOT contain regex pattern '@ts-ignore' in files ['bin/cli.js']; rationale: @ts-ignore escape hatch",
+      "paraphrase": "variant arm output MUST NOT contain regex pattern '@ts-ignore' in files ['bin/cli.js']; rationale: @ts-ignore escape hat",
+      "source_refs": [
+        "expected.json:forbidden_patterns[1]"
+      ]
+    },
+    {
+      "authority": "expected.json/max_deps_added",
+      "id": "max_deps_added__0",
+      "operational_check": "variant arm MUST NOT add more than 0 new npm dependencies (count delta of package.json:dependencies + devDependencies)",
+      "paraphrase": "variant arm MUST NOT add more than 0 new npm dependencies (count delta of package.json:dependencies + devDependencies)",
+      "source_refs": [
+        "expected.json:max_deps_added"
+      ]
+    },
+    {
+      "authority": "expected.json/required_files",
+      "id": "required_file__bin_cli_js",
+      "operational_check": "variant arm output MUST contain file 'bin/cli.js' (created or preserved)",
+      "paraphrase": "variant arm output MUST contain file 'bin/cli.js' (created or preserved)",
+      "source_refs": [
+        "expected.json:required_files[bin/cli.js]"
+      ]
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "scope-tier-a:lockfile-deletion",
+      "operational_check": "variant arm MUST NOT delete a scaffold-present lockfile",
+      "paraphrase": "variant arm MUST NOT delete a scaffold-present lockfile",
+      "source_refs": [
+        "oracle-scope-tier-a.py"
+      ]
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "scope-tier-a:tier-a-violation",
+      "operational_check": "variant arm MUST NOT add or modify paths matching: docs/roadmap/** | docs/VISION.md | docs/ROADMAP.md | .github/** | node_modules/** | **/node_modules/** | test-results/** | coverage/** | .nyc_output/** | basename suffix .log | basename prefix .env or secrets.",
+      "paraphrase": "variant arm MUST NOT add or modify paths matching: docs/roadmap/** | docs/VISION.md | docs/ROADMAP.md | .github/** | nod",
+      "source_refs": [
+        "oracle-scope-tier-a.py"
+      ]
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "scope-tier-b:scope-unmatched",
+      "operational_check": "every variant-touched file MUST be either inside spec_output_files (Tier C) OR reachable from a Tier C seed via static JS/TS imports OR matched by expected.json:tier_a_waivers",
+      "paraphrase": "every variant-touched file MUST be either inside spec_output_files (Tier C) OR reachable from a Tier C seed via static J",
+      "source_refs": [
+        "oracle-scope-tier-b.py"
+      ]
+    },
+    {
+      "authority": "expected.json/spec_output_files",
+      "id": "spec_output_file__bin_cli_js",
+      "operational_check": "variant-touched files MUST be inside (or reachable via static imports from) the spec_output_files set; 'bin/cli.js' is one Tier C seed",
+      "paraphrase": "variant-touched files MUST be inside (or reachable via static imports from) the spec_output_files set; 'bin/cli.js' is o",
+      "source_refs": [
+        "expected.json:spec_output_files[bin/cli.js]"
+      ]
+    },
+    {
+      "authority": "expected.json/spec_output_files",
+      "id": "spec_output_file__tests_cli_test_js",
+      "operational_check": "variant-touched files MUST be inside (or reachable via static imports from) the spec_output_files set; 'tests/cli.test.js' is one Tier C seed",
+      "paraphrase": "variant-touched files MUST be inside (or reachable via static imports from) the spec_output_files set; 'tests/cli.test.j",
+      "source_refs": [
+        "expected.json:spec_output_files[tests/cli.test.js]"
+      ]
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "test-fidelity:assertion-regression",
+      "operational_check": "effective assertion count MUST NOT drop and skipped-test count MUST NOT rise; vacuous expect.assertions(0) is treated as a real regression",
+      "paraphrase": "effective assertion count MUST NOT drop and skipped-test count MUST NOT rise; vacuous expect.assertions(0) is treated as",
+      "source_refs": [
+        "oracle-test-fidelity.py"
+      ]
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "test-fidelity:mock-swap",
+      "operational_check": "post-arm test file MUST NOT swap REAL_PATTERNS hits for MOCK_PATTERNS hits (jest/vi/sinon, nock/msw, app.handle/inject/callback, hand-rolled IncomingMessage/ServerResponse, etc.); a drop in real_calls combined with a rise in mock_calls is a mock-swap flag",
+      "paraphrase": "post-arm test file MUST NOT swap REAL_PATTERNS hits for MOCK_PATTERNS hits (jest/vi/sinon, nock/msw, app.handle/inject/c",
+      "source_refs": [
+        "oracle-test-fidelity.py"
+      ]
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "test-fidelity:test-file-deleted",
+      "operational_check": "no scaffold-present test file may be deleted by the variant arm; deletion of an existing tests/*.test.* / *.spec.* / *.e2e.* file is a flag-severity finding",
+      "paraphrase": "no scaffold-present test file may be deleted by the variant arm; deletion of an existing tests/*.test.* / *.spec.* / *.e",
+      "source_refs": [
+        "oracle-test-fidelity.py"
+      ]
+    },
+    {
+      "authority": "metadata/oracle-allowlist",
+      "id": "test-fidelity:test-file-renamed",
+      "operational_check": "rename of a scaffold-present test file is warn-severity (content fidelity not verified across renames in step 1)",
+      "paraphrase": "rename of a scaffold-present test file is warn-severity (content fidelity not verified across renames in step 1)",
+      "source_refs": [
+        "oracle-test-fidelity.py"
+      ]
+    },
+    {
+      "authority": "expected.json/verification_commands",
+      "id": "verification__3f35982a",
+      "operational_check": "running `node bin/cli.js doctor` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['doctor:']; stdout MUST NOT contain any of ['undefined', 'Error:']",
+      "paraphrase": "running `node bin/cli.js doctor` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['doctor:'];",
+      "source_refs": [
+        "expected.json:verification_commands[0]"
+      ]
+    },
+    {
+      "authority": "expected.json/verification_commands",
+      "id": "verification__460fce04",
+      "operational_check": "running `HOME=/nonexistent node bin/cli.js doctor` in the post-arm work dir MUST exit with code 1; stdout MUST contain all of ['/nonexistent']; stdout MUST NOT contain any of []",
+      "paraphrase": "running `HOME=/nonexistent node bin/cli.js doctor` in the post-arm work dir MUST exit with code 1; stdout MUST contain a",
+      "source_refs": [
+        "expected.json:verification_commands[1]"
+      ]
+    },
+    {
+      "authority": "expected.json/verification_commands",
+      "id": "verification__973e287e",
+      "operational_check": "running `python3 -c \"import subprocess; r = subprocess.run(['node', 'bin/cli.js', 'doctor'], capture_output=True); n = r.stdout.count(b'\\x1b['); print(n); exit(0 if n == 0 else 1)\"` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['0']; stdout MUST NOT contain any of []",
+      "paraphrase": "running `python3 -c \"import subprocess; r = subprocess.run(['node', 'bin/cli.js', 'doctor'], capture_output=True); n = r",
+      "source_refs": [
+        "expected.json:verification_commands[2]"
+      ]
+    },
+    {
+      "authority": "expected.json/verification_commands",
+      "id": "verification__d6253a97",
+      "operational_check": "running `node bin/cli.js doctor --help` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['doctor']; stdout MUST NOT contain any of []",
+      "paraphrase": "running `node bin/cli.js doctor --help` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['doc",
+      "source_refs": [
+        "expected.json:verification_commands[3]"
+      ]
+    },
+    {
+      "authority": "expected.json/verification_commands",
+      "id": "verification__e0f149e4",
+      "operational_check": "running `node bin/cli.js --help` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['doctor']; stdout MUST NOT contain any of []",
+      "paraphrase": "running `node bin/cli.js --help` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['doctor']; ",
+      "source_refs": [
+        "expected.json:verification_commands[4]"
+      ]
+    },
+    {
+      "authority": "expected.json/verification_commands",
+      "id": "verification__fdbcd321",
+      "operational_check": "running `node bin/cli.js doctor --verbose` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['doctor:']; stdout MUST NOT contain any of ['Error:']",
+      "paraphrase": "running `node bin/cli.js doctor --verbose` in the post-arm work dir MUST exit with code 0; stdout MUST contain all of ['",
+      "source_refs": [
+        "expected.json:verification_commands[5]"
+      ]
+    }
+  ],
+  "authority_order": [
+    "spec.md",
+    "expected.json/rubric",
+    "phase prompt",
+    "model preference"
+  ],
+  "escalated_to_user": [],
+  "fixture_id": "F2-cli-medium-subcommand",
+  "model_stamps": {
+    "claude": {
+      "blocked_ids": [],
+      "model": "claude-opus-4-7",
+      "signed_plan_sha256": "05d19dc09a1c8820f58afcd091c2cd20888f7bf1141af4ec451a69723af0588a",
+      "status": "sign",
+      "timestamp": "2026-04-29T18:30:00Z"
+    },
+    "codex": {
+      "blocked_ids": [],
+      "model": "gpt-5.5",
+      "signed_plan_sha256": "05d19dc09a1c8820f58afcd091c2cd20888f7bf1141af4ec451a69723af0588a",
+      "status": "sign",
+      "timestamp": "2026-04-29T18:31:00Z"
+    }
+  },
+  "plan_status": "final",
+  "planning_mode": "pair",
+  "rejected_alternatives": [],
+  "rounds": [
+    {
+      "claude_draft_sha256": "0000000000000000000000000000000000000000000000000000000000000000",
+      "codex_draft_sha256": "1111111111111111111111111111111111111111111111111111111111111111",
+      "merged_sha256": "2222222222222222222222222222222222222222222222222222222222222222",
+      "note": "sample-pass synthetic round (test fixture)",
+      "round": 1
+    }
+  ],
+  "schema_version": "1",
+  "source": {
+    "canonical_id_registry_path": "benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/expected-pair-plan-registry.json",
+    "canonical_id_registry_sha256": "98ac16e4536ea3ef2e51d3c728982c014211c193a742cea74f1331e4fbba76be",
+    "expected_path": "benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/expected.json",
+    "expected_sha256": "ddef8feba49f20b6957e37840bc6a03e78e554776e380d81ad6390944c72fcab",
+    "rubric_path": "benchmark/auto-resolve/RUBRIC.md",
+    "rubric_sha256": "5b5b709a0b57f7e6f4fbc072af91e1edbc8d7910ae16b9b7be7170616aeaa9af",
+    "spec_path": "benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/spec.md",
+    "spec_sha256": "9b0949c2afd4a522de2bdbbf267d93907fd908bf0f1d0dc5e111ee30ba875bb7"
+  },
+  "unresolved": []
+}