devlyn-cli 2.3.0 → 2.3.1

package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/NOTES.md

@@ -0,0 +1,35 @@
+# F30 - Notes
+
+## Failure Mode
+
+This fixture detects payment-style implementations that pass simple happy-path
+tests while double-applying duplicate operations, letting rejected operations
+consume available credit, failing to release holds on capture/release, or
+mutating input files while computing settlement state.
+
+## Pipeline Phases
+
+It stresses IMPLEMENT and VERIFY. The visible spec names the state transition
+rules, idempotency output shape, and exact account summary; hidden verifiers
+combine those rules so a one-axis implementation is not enough.
+
+## Why Existing Fixtures Do Not Cover This
+
+F16 covers quote math, F23 covers warehouse allocation rollback, F25 covers cart
+promotions, and F28 return authorization was rejected after corrected scoring
+showed solo saturation. None focus on duplicate operation idempotency plus
+credit-hold mutation, capture/release transitions, and validation immutability.
+
+## Retirement Criteria
+
+Retire or rotate this fixture if both `solo_claude` and the selected pair arm
+score near the ceiling for two shipped versions, or if another fixture covers
+idempotent financial hold mutation with clearer pair headroom.
+
+## Headroom Status
+
+Retired after headroom run `20260511-f30-headroom-v1`: bare 33 /
+solo_claude 98, headroom FAIL because `solo_claude score 98 > 80`.
+
+Do not count F30 as pair-lift evidence. Rework the visible contract or hidden
+verifiers before spending pair arms on this idea again.

package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/RETIRED.md

@@ -0,0 +1,12 @@
+# F30 retired
+
+Retired from the active golden suite after headroom run
+`20260511-f30-headroom-v1`.
+
+Reason: `solo_claude` scored 98, exceeding the headroom ceiling of 80, while
+bare scored 33. The fixture is useful as a record of an idempotent hold
+settlement candidate that proved too explicit for solo, but it is not
+pair-lift evidence.
+
+Future use: rework the visible contract or hidden verifiers so the task creates
+a fair pair-risk-probe gap without hiding requirements from the spec.

package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/expected.json

@@ -0,0 +1,58 @@
+{
+  "verification_commands": [
+    {
+      "cmd": "node --test tests/cli.test.js",
+      "exit_code": 0,
+      "stdout_contains": [],
+      "stdout_not_contains": ["not ok "]
+    },
+    {
+      "cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/mixed-idempotent-settlement.js\"",
+      "exit_code": 0,
+      "stdout_contains": ["\"ok\":true"],
+      "stdout_not_contains": [],
+      "contract_refs": [
+        "A failed authorization does not reserve credit and does not block a later valid authorization.",
+        "Capture removes the active hold and increases `balance_cents`.",
+        "Release removes the active hold without changing `balance_cents`.",
+        "Duplicate operation ids do not mutate state and report the original status."
+      ]
+    },
+    {
+      "cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/rejection-boundaries.js\"",
+      "exit_code": 0,
+      "stdout_contains": ["\"ok\":true"],
+      "stdout_not_contains": [],
+      "contract_refs": [
+        "Reusing an active `hold_id` is rejected as `\"duplicate_hold\"`.",
+        "Capture or release with a wrong amount is rejected as `\"amount_mismatch\"` and does not mutate state.",
+        "Invalid unknown-account input exits `2`, prints one JSON error object to stderr, and prints no stdout.",
+        "The input file contents are unchanged after settlement."
+      ]
+    }
+  ],
+  "forbidden_patterns": [
+    {
+      "pattern": "catch\\s*\\([^)]*\\)\\s*\\{[^}]*return\\s+(null|undefined|''|\\{\\})",
+      "description": "silent catch returning fallback in credit hold settlement path",
+      "files": ["bin/cli.js"],
+      "severity": "disqualifier"
+    },
+    {
+      "pattern": "catch\\s*\\([^)]*\\)\\s*\\{\\s*\\}",
+      "description": "empty catch block",
+      "files": ["bin/cli.js"],
+      "severity": "disqualifier"
+    },
+    {
+      "pattern": "/\\*\\s*eslint-disable",
+      "description": "eslint-disable without scoped justification",
+      "files": ["bin/cli.js"],
+      "severity": "disqualifier"
+    }
+  ],
+  "required_files": ["bin/cli.js", "tests/cli.test.js"],
+  "forbidden_files": [],
+  "max_deps_added": 0,
+  "spec_output_files": ["bin/cli.js", "tests/cli.test.js"]
+}

package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/metadata.json

@@ -0,0 +1,10 @@
+{
+  "id": "F30-cli-credit-hold-settlement",
+  "category": "high-risk",
+  "difficulty": "high",
+  "timeout_seconds": 1500,
+  "required_tools": ["node"],
+  "browser": false,
+  "deps_change_expected": false,
+  "intent": "Add a bench-cli settle-holds command that reads account credit holds and operations, applies authorization/capture/release with idempotency and rollback, and emits exact cents-based settlement state without mutating input."
+}

package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/setup.sh

@@ -0,0 +1,2 @@
+#!/usr/bin/env bash
+set -e

package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/spec.md

@@ -0,0 +1,73 @@
+---
+id: "F30-cli-credit-hold-settlement"
+title: "Credit hold settlement"
+status: planned
+complexity: high
+depends-on: []
+---
+
+# F30 Credit hold settlement
+
+## Context
+
+`bench-cli` currently has greeting and version commands only. The task:
+add a `settle-holds` command that reads account credit holds and operations,
+applies authorization/capture/release with idempotency and rollback, and emits
+exact cents-based settlement state without mutating input.
+
+Credit holds feed payment and ledger workflows, so duplicate operations,
+failed operations, and available-credit calculations must be deterministic.
+
+## Requirements
+
+- [ ] `bench-cli settle-holds --input <path>` reads JSON shaped as `{ "accounts": Array<Account>, "operations": Array<Operation> }`.
+- [ ] Each account has `{ "id": string, "balance_cents": number, "credit_limit_cents": number }`.
+- [ ] Each operation has `{ "id": string, "account_id": string, "type": "authorize" | "capture" | "release", "hold_id": string, "amount_cents": number }`.
+- [ ] Validate before settlement: ids and hold ids must be non-empty strings, account ids must be unique, balances and credit limits must be non-negative integers, amount cents must be positive integers, operation types must be one of the allowed strings, and every operation account must exist.
+- [ ] Invalid input exits `2`, writes exactly one JSON error object to stderr, and writes nothing to stdout.
+- [ ] Business rejections do not exit non-zero and do not mutate settlement state.
+- [ ] Process operations in input order.
+- [ ] An `authorize` operation creates one active hold when `credit_limit_cents - balance_cents - active_hold_cents >= amount_cents`; otherwise it is rejected with reason `"insufficient_credit"`.
+- [ ] An `authorize` operation for a `hold_id` that is already active is rejected with reason `"duplicate_hold"`.
+- [ ] A `capture` operation requires an active hold for the same account and exactly the requested amount available on that hold; otherwise it is rejected with reason `"unknown_hold"` or `"amount_mismatch"`.
+- [ ] An approved `capture` increases `balance_cents` by `amount_cents` and removes the active hold.
+- [ ] A `release` operation requires an active hold for the same account and exactly the requested amount available on that hold; otherwise it is rejected with reason `"unknown_hold"` or `"amount_mismatch"`.
+- [ ] An approved `release` removes the active hold without changing `balance_cents`.
+- [ ] Duplicate operation ids are idempotent: the first occurrence is processed normally; each later operation with the same `id` must not mutate state and emits `{ "id": string, "status": "duplicate", "original_status": "approved" | "rejected" }`.
+- [ ] Approved rows have keys `id`, `status`, `type`, `account_id`, `hold_id`, `amount_cents`.
+- [ ] Rejected rows have keys `id`, `status`, `reason`.
+- [ ] Duplicate rows have keys `id`, `status`, `original_status`.
+- [ ] On success, write exactly one JSON object to stdout and no stderr. Keys: `results`, `accounts`.
+- [ ] `results` is ordered by input operation order.
+- [ ] `accounts` is sorted by account id. Each account row has keys `id`, `balance_cents`, `active_hold_cents`, `available_cents`.
+- [ ] The command must not modify the input file.
+- [ ] `tests/cli.test.js` is updated. Existing tests still pass AND at least two new tests cover one mixed approved/rejected/duplicate settlement and one validation failure.
+
+## Constraints
+
+- **No new npm dependencies.**
+- **No floating-money output.** All public amounts are integer cents.
+- **No silent catches.** Invalid input and file-read failures must surface as JSON errors with exit `2`.
+- **No extra stdout/stderr text** on the success path; downstream tooling parses stdout as JSON.
+- **Touch only `bin/cli.js` and `tests/cli.test.js`.**
+
+## Out of Scope
+
+- Payment processor calls.
+- Persistence beyond stdout.
+- Partial captures or partial releases.
+- Currencies, interest, fees, or statement generation.
+- Touching `server/`, `web/`, or `tests/server.test.js`.
+
+## Verification
+
+- `node --test tests/cli.test.js` exits 0.
+- A failed authorization does not reserve credit and does not block a later valid authorization.
+- Capture removes the active hold and increases `balance_cents`.
+- Release removes the active hold without changing `balance_cents`.
+- Duplicate operation ids do not mutate state and report the original status.
+- Reusing an active `hold_id` is rejected as `"duplicate_hold"`.
+- Capture or release with a wrong amount is rejected as `"amount_mismatch"` and does not mutate state.
+- Invalid unknown-account input exits `2`, prints one JSON error object to stderr, and prints no stdout.
+- The input file contents are unchanged after settlement.
+- `git diff --stat` shows only `bin/cli.js` and `tests/cli.test.js` touched.

package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/task.txt

@@ -0,0 +1,17 @@
+Add a bench-cli settle-holds command that reads account credit holds and
+operations, applies authorization/capture/release with idempotency and rollback,
+and emits exact cents-based settlement state without mutating input.
+
+The command should read `--input <path>` JSON with accounts and operations.
+Validate the input before settlement. Use integer cents only, write exactly one
+JSON object to stdout on success, and write exactly one JSON error object to
+stderr with exit code 2 for invalid input. Do not add npm dependencies.
+
+Business rejections should stay on exit code 0. Process operations in input
+order. Authorizations create active holds only when available credit is enough.
+Captures and releases require an active hold for the same account and exact
+amount. Failed operations must not mutate state. Duplicate operation ids are
+idempotent: only the first occurrence mutates state, and later duplicates report
+the original approved/rejected status.
+
+Update the CLI tests. Touch only `bin/cli.js` and `tests/cli.test.js`.

package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/verifiers/mixed-idempotent-settlement.js

@@ -0,0 +1,53 @@
+'use strict';
+
+const assert = require('node:assert');
+const { execFileSync } = require('node:child_process');
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+
+const work = process.env.BENCH_WORKDIR || process.cwd();
+const cli = path.join(work, 'bin', 'cli.js');
+const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'f30-mixed-'));
+const input = path.join(tmp, 'holds.json');
+
+fs.writeFileSync(input, JSON.stringify({
+  accounts: [
+    { id: 'acct-a', balance_cents: 2000, credit_limit_cents: 10000 },
+    { id: 'acct-b', balance_cents: 0, credit_limit_cents: 5000 }
+  ],
+  operations: [
+    { id: 'op-auth-1', account_id: 'acct-a', type: 'authorize', hold_id: 'h-1', amount_cents: 5000 },
+    { id: 'op-too-large', account_id: 'acct-a', type: 'authorize', hold_id: 'h-2', amount_cents: 4000 },
+    { id: 'op-release', account_id: 'acct-a', type: 'release', hold_id: 'h-1', amount_cents: 5000 },
+    { id: 'op-auth-3', account_id: 'acct-a', type: 'authorize', hold_id: 'h-3', amount_cents: 3000 },
+    { id: 'op-capture', account_id: 'acct-a', type: 'capture', hold_id: 'h-3', amount_cents: 3000 },
+    { id: 'op-capture', account_id: 'acct-a', type: 'capture', hold_id: 'h-3', amount_cents: 3000 },
+    { id: 'op-auth-b', account_id: 'acct-b', type: 'authorize', hold_id: 'h-b', amount_cents: 5000 }
+  ]
+}), 'utf8');
+
+const stdout = execFileSync('node', [cli, 'settle-holds', '--input', input], {
+  cwd: work,
+  encoding: 'utf8',
+  stdio: ['ignore', 'pipe', 'pipe']
+});
+const parsed = JSON.parse(stdout);
+
+assert.deepStrictEqual(parsed, {
+  results: [
+    { id: 'op-auth-1', status: 'approved', type: 'authorize', account_id: 'acct-a', hold_id: 'h-1', amount_cents: 5000 },
+    { id: 'op-too-large', status: 'rejected', reason: 'insufficient_credit' },
+    { id: 'op-release', status: 'approved', type: 'release', account_id: 'acct-a', hold_id: 'h-1', amount_cents: 5000 },
+    { id: 'op-auth-3', status: 'approved', type: 'authorize', account_id: 'acct-a', hold_id: 'h-3', amount_cents: 3000 },
+    { id: 'op-capture', status: 'approved', type: 'capture', account_id: 'acct-a', hold_id: 'h-3', amount_cents: 3000 },
+    { id: 'op-capture', status: 'duplicate', original_status: 'approved' },
+    { id: 'op-auth-b', status: 'approved', type: 'authorize', account_id: 'acct-b', hold_id: 'h-b', amount_cents: 5000 }
+  ],
+  accounts: [
+    { id: 'acct-a', balance_cents: 5000, active_hold_cents: 0, available_cents: 5000 },
+    { id: 'acct-b', balance_cents: 0, active_hold_cents: 5000, available_cents: 0 }
+  ]
+});
+
+console.log(JSON.stringify({ ok: true }));

package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/verifiers/rejection-boundaries.js

@@ -0,0 +1,74 @@
+'use strict';
+
+const assert = require('node:assert');
+const { spawnSync } = require('node:child_process');
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+
+const work = process.env.BENCH_WORKDIR || process.cwd();
+const cli = path.join(work, 'bin', 'cli.js');
+
+function runPayload(label, payload) {
+  const tmp = fs.mkdtempSync(path.join(os.tmpdir(), `f30-${label}-`));
+  const input = path.join(tmp, 'holds.json');
+  const original = JSON.stringify(payload, null, 2);
+  fs.writeFileSync(input, original, 'utf8');
+  const result = spawnSync('node', [cli, 'settle-holds', '--input', input], {
+    cwd: work,
+    encoding: 'utf8'
+  });
+  assert.strictEqual(fs.readFileSync(input, 'utf8'), original, `${label}: input mutated`);
+  return result;
+}
+
+const boundary = runPayload('boundary', {
+  accounts: [
+    { id: 'acct-a', balance_cents: 1000, credit_limit_cents: 7000 }
+  ],
+  operations: [
+    { id: 'op-auth', account_id: 'acct-a', type: 'authorize', hold_id: 'h-1', amount_cents: 3000 },
+    { id: 'op-dupe-hold', account_id: 'acct-a', type: 'authorize', hold_id: 'h-1', amount_cents: 1000 },
+    { id: 'op-bad-capture', account_id: 'acct-a', type: 'capture', hold_id: 'h-1', amount_cents: 2000 },
+    { id: 'op-release', account_id: 'acct-a', type: 'release', hold_id: 'h-1', amount_cents: 3000 },
+    { id: 'op-bad-release', account_id: 'acct-a', type: 'release', hold_id: 'h-1', amount_cents: 3000 },
+    { id: 'op-after', account_id: 'acct-a', type: 'authorize', hold_id: 'h-2', amount_cents: 6000 },
+    { id: 'op-dupe-reject', account_id: 'acct-a', type: 'authorize', hold_id: 'h-3', amount_cents: 1 },
+    { id: 'op-dupe-reject', account_id: 'acct-a', type: 'authorize', hold_id: 'h-4', amount_cents: 1 }
+  ]
+});
+
+assert.strictEqual(boundary.status, 0);
+assert.strictEqual(boundary.stderr, '');
+assert.deepStrictEqual(JSON.parse(boundary.stdout), {
+  results: [
+    { id: 'op-auth', status: 'approved', type: 'authorize', account_id: 'acct-a', hold_id: 'h-1', amount_cents: 3000 },
+    { id: 'op-dupe-hold', status: 'rejected', reason: 'duplicate_hold' },
+    { id: 'op-bad-capture', status: 'rejected', reason: 'amount_mismatch' },
+    { id: 'op-release', status: 'approved', type: 'release', account_id: 'acct-a', hold_id: 'h-1', amount_cents: 3000 },
+    { id: 'op-bad-release', status: 'rejected', reason: 'unknown_hold' },
+    { id: 'op-after', status: 'approved', type: 'authorize', account_id: 'acct-a', hold_id: 'h-2', amount_cents: 6000 },
+    { id: 'op-dupe-reject', status: 'rejected', reason: 'insufficient_credit' },
+    { id: 'op-dupe-reject', status: 'duplicate', original_status: 'rejected' }
+  ],
+  accounts: [
+    { id: 'acct-a', balance_cents: 1000, active_hold_cents: 6000, available_cents: 0 }
+  ]
+});
+
+const invalid = runPayload('invalid', {
+  accounts: [
+    { id: 'acct-a', balance_cents: 0, credit_limit_cents: 1000 }
+  ],
+  operations: [
+    { id: 'op-missing-account', account_id: 'acct-missing', type: 'authorize', hold_id: 'h-1', amount_cents: 100 }
+  ]
+});
+
+assert.strictEqual(invalid.status, 2);
+assert.strictEqual(invalid.stdout, '');
+const err = JSON.parse(invalid.stderr);
+assert.strictEqual(typeof err.error, 'string');
+assert.notStrictEqual(err.error.length, 0);
+
+console.log(JSON.stringify({ ok: true }));

package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/NOTES.md

@@ -0,0 +1,60 @@
+# F9 — Notes
+
+## Purpose
+
+**Load-bearing for the novice-user contract.** The suite ship-gate requires
+F9 to pass (margin ≥ +5) on every shipped version. If F9 fails, the "type
+`/devlyn:ideate` and ship worldclass software" promise is not being met.
+
+## What the variant arm does
+
+A novice-simulating prompt (`task.txt` is identical to what the user typed)
+is delivered to a fresh Claude session. The session has our skills installed.
+The pipeline arm is expected to:
+
+1. Recognize this is a vague idea, not a spec → invoke `/devlyn:ideate`.
+2. Ideate produces `docs/VISION.md`, `docs/ROADMAP.md`, and
+   `docs/roadmap/phase-1/1.1-gitstats.md` (or similar).
+3. Run `/devlyn:auto-resolve` on the generated spec.
+4. Run `/devlyn:preflight` for verification.
+
+The variant arm's prompt explicitly instructs this chain so we're not
+relying on Claude to invent it. That's fair because the novice contract is
+about the TOOLS being available + discoverable; the user in this benchmark
+is already primed to use them.
+
+## What the bare arm does
+
+Same raw task delivered as a direct prompt. Bare implements `gitstats`
+using its own judgment. Bare does NOT produce VISION/ROADMAP documents
+(and isn't expected to).
+
+## Why margin ≥ +5 is required
+
+The pipeline's whole value prop is that it trades some bare-case tokens for
+quality uplift on novice flows. If this fixture can't show ≥ +5 margin,
+we're paying pipeline cost without delivering on the novice promise.
+
+## Scoring notes
+
+- The variant's `docs/VISION.md` + `ROADMAP.md` + spec files ARE part of
+  the judge's evaluation. The judge sees the full product (code + docs +
+  roadmap state), not just the diff to `bin/cli.js`.
+- Bare doesn't produce roadmap files, so bare's judge payload is
+  code+test only. This asymmetry is INTENTIONAL — the fixture tests
+  total-output quality, not per-file quality.
+
+## Failure modes detected
+
+- **Pipeline skips ideate.** Variant goes straight to auto-resolve with a
+  vague spec → downstream implementation is weak. Caught by judge:
+  `docs/roadmap/` files missing.
+- **Bare over-engineers.** Without a skeleton, bare builds too much,
+  touches wrong files, adds deps. Caught by spec constraints.
+- **Pipeline ships "done" but preflight was a no-op.** If `.devlyn/PREFLIGHT-REPORT.md` exists but shows no commitment audit, something is broken upstream.
+
+## Rotation trigger
+
+F9 is the last fixture we rotate — it's the anchor. If it saturates
+(variant consistently > 95), the whole suite needs a harder novice-flow
+anchor before we retire this one.

package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/RETIRED.md

@@ -0,0 +1,29 @@
+# RETIRED — F9-e2e-ideate-to-preflight
+
+**Retired**: 2026-04-30 (iter-0033a)
+**Replaced by**: `benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/`
+**Source SHA**: `8d4d57f` (commit before the 2-skill-contract rename).
+
+## Why retired
+
+The 2-skill redesign (Phases 1-3, iter-0029 / 0031 / 0032) replaced
+`/devlyn:ideate` (greenfield) and folded `/devlyn:preflight` into
+`/devlyn:resolve`'s VERIFY phase. The OLD F9 fixture's contract assumed
+the 3-skill chain (`/devlyn:ideate` → `/devlyn:auto-resolve` →
+`/devlyn:preflight`), which is unobtainable at HEAD post-iter-0032
+because OLD ideate was deleted.
+
+iter-0033a redesigned F9 to match the shipped 2-skill contract.
+
+## When to consult this archive
+
+- Replaying a regression suspected from the OLD chain.
+- Migrating a pre-2026-04-30 historical run record back to readable shape.
+- Auditing what changed when the new fixture's measurements diverge from
+  pre-redesign baselines.
+
+## What lives here
+
+The exact file contents of the F9 fixture as of `8d4d57f` (the last commit
+before the rename). DO NOT use this directory as a live fixture — it is
+not picked up by `run-suite.sh`. Restore-and-run requires a manual copy.

package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/expected.json

@@ -0,0 +1,73 @@
+{
+  "verification_commands": [
+    {
+      "cmd": "node bin/cli.js gitstats",
+      "exit_code": 0,
+      "stdout_contains": [
+        "Commits:",
+        "Last commit:"
+      ],
+      "stdout_not_contains": [
+        "Error:"
+      ]
+    },
+    {
+      "cmd": "node bin/cli.js gitstats --json",
+      "exit_code": 0,
+      "stdout_contains": [
+        "{",
+        "commits",
+        "authors"
+      ],
+      "stdout_not_contains": []
+    },
+    {
+      "cmd": "cd /tmp && node -e 'const { spawnSync } = require(\"child_process\"); const p = process.env.BENCH_WORKDIR || process.cwd(); console.log(spawnSync(\"node\", [p + \"/bin/cli.js\", \"gitstats\"], { encoding: \"utf8\", cwd: \"/tmp\" }).status)'",
+      "exit_code": 0,
+      "stdout_contains": [
+        "2"
+      ],
+      "stdout_not_contains": [
+        "0"
+      ]
+    },
+    {
+      "cmd": "node bin/cli.js hello",
+      "exit_code": 0,
+      "stdout_contains": [
+        "Hello, world!"
+      ],
+      "stdout_not_contains": []
+    },
+    {
+      "cmd": "node --test tests/",
+      "exit_code": 0,
+      "stdout_contains": [],
+      "stdout_not_contains": []
+    }
+  ],
+  "forbidden_patterns": [
+    {
+      "pattern": "catch\\s*\\([^)]*\\)\\s*\\{\\s*\\}",
+      "description": "empty catch",
+      "files": [
+        "bin/cli.js"
+      ],
+      "severity": "disqualifier"
+    }
+  ],
+  "required_files": [
+    "bin/cli.js"
+  ],
+  "forbidden_files": [],
+  "max_deps_added": 0,
+  "tier_a_waivers": [
+    "docs/VISION.md",
+    "docs/ROADMAP.md",
+    "docs/roadmap/**"
+  ],
+  "spec_output_files": [
+    "bin/**",
+    "tests/**"
+  ]
+}

package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/metadata.json

@@ -0,0 +1,10 @@
+{
+  "id": "F9-e2e-ideate-to-preflight",
+  "category": "e2e",
+  "difficulty": "high",
+  "timeout_seconds": 3600,
+  "required_tools": ["node"],
+  "browser": false,
+  "deps_change_expected": false,
+  "intent": "End-to-end novice flow: from a vague idea ('git stats CLI for the current repo') the variant must run /devlyn:ideate → /devlyn:auto-resolve → /devlyn:preflight to produce Vision/Roadmap + implemented code + preflight sign-off. The bare arm receives the same vague idea as a direct prompt. This fixture gates the novice-user contract."
+}

package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/setup.sh

@@ -0,0 +1,28 @@
+#!/usr/bin/env bash
+# F9 setup — seed a few synthetic commits with different authors so the
+# `gitstats` subcommand's "top 3 authors by commit count" requirement is
+# meaningfully exercised. Without this, every commit author is the runner's
+# default and the ranking test is a no-op.
+set -e
+
+commit_as() {
+  local name="$1" email="$2" file="$3" message="$4"
+  echo "$(date +%s%N) $name" >> "$file"
+  git add "$file"
+  git -c user.name="$name" -c user.email="$email" commit -q -m "$message"
+}
+
+mkdir -p .bench-seed
+
+commit_as "Alpha Author"   "alpha@bench.test"   .bench-seed/log "seed: alpha 1"
+commit_as "Alpha Author"   "alpha@bench.test"   .bench-seed/log "seed: alpha 2"
+commit_as "Alpha Author"   "alpha@bench.test"   .bench-seed/log "seed: alpha 3"
+commit_as "Alpha Author"   "alpha@bench.test"   .bench-seed/log "seed: alpha 4"
+commit_as "Beta Author"    "beta@bench.test"    .bench-seed/log "seed: beta 1"
+commit_as "Beta Author"    "beta@bench.test"    .bench-seed/log "seed: beta 2"
+commit_as "Beta Author"    "beta@bench.test"    .bench-seed/log "seed: beta 3"
+commit_as "Gamma Author"   "gamma@bench.test"   .bench-seed/log "seed: gamma 1"
+commit_as "Gamma Author"   "gamma@bench.test"   .bench-seed/log "seed: gamma 2"
+commit_as "Delta Author"   "delta@bench.test"   .bench-seed/log "seed: delta 1"
+
+echo "F9 setup: seeded 10 commits across 4 authors (Alpha 4 / Beta 3 / Gamma 2 / Delta 1)"

package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/spec.md

@@ -0,0 +1,58 @@
+---
+id: "F9-e2e-ideate-to-preflight"
+title: "End-to-end: idea → shipped CLI feature"
+status: planned
+complexity: high
+depends-on: []
+---
+
+# F9 End-to-End Novice Flow
+
+## Context
+
+A first-time user has a vague idea:
+
+> "I want a CLI subcommand that shows basic stats about the current git repo — commit count, last commit date, top 3 authors. Call it `gitstats`."
+
+The variant arm is expected to use the pipeline: `/devlyn:ideate` to
+produce a VISION/ROADMAP + spec, then `/devlyn:auto-resolve` to implement
+per spec, then `/devlyn:preflight` to verify. The bare arm receives the
+same idea as a direct prompt and implements it without the pipeline.
+
+This fixture is the suite's most important gate for the "novice user contract":
+a first-time user typing `/devlyn:ideate` should land at working, well-structured software.
+
+## Requirements
+
+- [ ] A new `gitstats` subcommand exists in `bin/cli.js`.
+- [ ] `node bin/cli.js gitstats` (run inside a git repo) prints:
+  - Line 1: commit count (e.g., `Commits: 42`).
+  - Line 2: last commit ISO date (e.g., `Last commit: 2026-04-23T12:00:00Z`).
+  - Lines 3-5: top 3 authors by commit count, format `<rank>. <name> <count>`.
+- [ ] Run outside a git repo → stderr message `Error: not a git repository` and exit 2.
+- [ ] `node bin/cli.js gitstats --json` emits valid JSON with the same data.
+- [ ] Existing subcommands (`hello`, `version`) unchanged.
+- [ ] Add at least one test.
+- [ ] For variant: a `docs/VISION.md`, `docs/ROADMAP.md`, and a `docs/roadmap/phase-1/` spec file must exist after the run (evidence the ideate stage happened).
+
+## Constraints
+
+- **No new npm dependencies.** Use `child_process` to shell out to `git`.
+- **No silent catches.**
+- **Non-git-repo handling.** Do not assume the user is always in a repo.
+
+- **Lifecycle note.** The harness's DOCS phase flips this spec's frontmatter `status` after implementation completes — that is benchmark lifecycle bookkeeping, not a scope violation.
+
+## Out of Scope
+
+- Parsing commit messages, tags, branches.
+- Remote API calls.
+- Touching `server/` or `web/`.
+
+## Verification
+
+- Inside this worktree (which IS a git repo): `node bin/cli.js gitstats` exits 0 and prints at least 5 lines of summary.
+- `node bin/cli.js gitstats --json | node -e 'const d=JSON.parse(require("fs").readFileSync(0,"utf8")); console.log(typeof d.commits)'` prints `number`.
+- `cd /tmp && node <worktree>/bin/cli.js gitstats` (from outside a repo — use the worktree's absolute path) exits 2.
+- For variant: `test -f docs/VISION.md && test -f docs/ROADMAP.md && ls docs/roadmap/phase-1/*.md | head -1`.
+- `node --test tests/` passes.

package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/task.txt

@@ -0,0 +1,5 @@
+I want a CLI subcommand that shows basic stats about the current git repo — commit count, last commit date, top 3 authors. Call it `gitstats`.
+
+Should work inside this repo when I run `node bin/cli.js gitstats`, and fail cleanly if I'm not in a git repo. A `--json` flag for machine-readable output would be useful too.
+
+Keep the existing `hello` and `version` subcommands working. Add a test. No new npm dependencies.