devlyn-cli 3.2.0 → 3.3.0

package/benchmark/auto-resolve/RUBRIC.md

@@ -1,177 +0,0 @@
-# Benchmark Judge Rubric
-
-Stable across model upgrades. This file is the single source of truth for how
-arms are scored and how ship gates evaluate a run. Do not change the rubric
-during a benchmarking window — changing it invalidates comparability with
-prior `history/runs/`.
-
-**Outer goal lives in [`autoresearch/NORTH-STAR.md`](../../autoresearch/NORTH-STAR.md).** The release-decision layer (L0 / L1 / L2 contracts, wall-time efficiency, pair-cost justification) sits on top of the per-arm scoring rules below. When NORTH-STAR.md adds a release-gate number that this file did not have, the new number applies — open a doc-fix iter to mirror it here.
-
-## Scoring — 4 axes, 25 points each, 100 total
-
-The blind judge scores all submitted arms on identical axes without knowing
-which label maps to which arm.
-
-### Axis 1 — Spec Compliance (0-25)
-
-Does this implementation satisfy every Requirements bullet in `spec.md`?
-Does every Verification command behave as the spec states?
-
-- **25** — All Requirements satisfied. All Verification commands would pass.
-- **19-24** — 90%+ coverage, minor omissions.
-- **13-18** — Partial implementation or verification gaps.
-- **7-12** — Major requirements missed.
-- **0-6** — Does not address the core task.
-
-Mechanical cap: after the blind judge returns, `judge.sh` caps total score at
-`floor(100 * verify_score)` and caps the Spec Compliance axis at
-`floor(25 * verify_score)`. This makes the machine-readable acceptance
-contract binding when a judge grades prose generously despite failed required
-verification commands.
-
-### Axis 2 — Constraint Respect (0-25)
-
-Zero new npm deps (unless spec allows), no silent catches (`try { } catch { return fallback }`), no `any`/`@ts-ignore`, explicit HOME/env guards where required, EACCES-specific handling, no hardcoded values that should be configurable.
-
-- Each **disqualifier-severity forbidden_pattern match** or explicit constraint violation = −4 points, minimum 0.
-
-### Axis 3 — Scope Discipline (0-25)
-
-Out of Scope respected. No gratuitous refactors of unrelated code. No "while I'm here" additions. No opportunistic upgrades.
-
-- Each out-of-scope change = −5 points, minimum 0.
-
-### Axis 4 — Code Quality (0-25)
-
-Readable, idiomatic for the language/framework, helpful error messages, appropriate abstraction level (not under- or over-engineered), uses standard library primitives where available (e.g., `fs.accessSync` over mode-bit checks per CLAUDE.md `phase-3-critic.md:32` calibration).
-
-- This axis is judge-calibrated; no deterministic grading. The judge looks at
-  naming, function/file decomposition, error handling, and comparability to
-  idiomatic peer code.
-
----
-
-## Judge Disqualifiers (hard floor)
-
-Any of these produces `disqualifier: true` on the arm, overriding score:
-
-- Silent-catch pattern in diff.
-- Fabricated verification (code that claims to pass without actually running).
-- Skipped a required test file that the spec names.
-- Created a file listed in `expected.forbidden_files`.
-- Exceeded `expected.max_deps_added`.
-- `@ts-ignore` / `eslint-disable` without scoped justification comment.
-- Hardcoded paths or values where spec required configurability.
-
-Disqualifier arms automatically lose the fixture regardless of score.
-
----
-
-## Ship Gates
-
-After the judge finishes every fixture, `scripts/ship-gate.py` applies these
-rules to the run's `summary.json`.
-
-This section describes the broad run-suite ship gate. Current solo<pair
-evidence uses the full-pipeline pair gate with an explicit selected pair arm
-(`l2_risk_probes` for proof runs, `l2_gated` for diagnostics), and that gate
-compares the selected pair arm against `solo_claude`.
-
-### Hard floors (any one failure blocks ship)
-
-1. **No disqualifier-level violation** in any gated harness arm (legacy suite `variant`/L2 and `solo_claude`/L1 when present).
-2. **F9 (E2E) must PASS** — novice-flow contract.
-3. **At least 7 gated, headroom-available fixtures** must have the required margin ≥ +5 for each gated contract — legacy `variant`-`bare` (L2-L0) for the suite gate, and `solo_claude`-`bare` (L1-L0) when `solo_claude` is present. This is **headroom-aware** (added 2026-05-02 per iter-0033 R4 + NORTH-STAR amendment): a fixture is excluded from a contract count when the lower arm is ceiling-near and the higher arm is clean at ceiling. Excluded fixtures become fixture-rotation candidates per the policy below if the two-shipped-version rule is met.
-4. **No fixture regression worse than −5** vs. last `baselines/shipped.json` on the same fixture.
-
-### Soft gates (produce WARNING but do not block)
-
-5. Suite average margin drop > 3 vs. last shipped.
-6. A fixture that previously had margin > +5 now has margin ≤ 0.
-7. Critical-finding catch-rate decrease vs. the last shipped gated harness arm.
-
-### Known-limit exception
-
-- **F8-known-limit-ambiguous** is excluded from gates 3 and 4. It exists to
-  document where the harness may not beat bare. Its allowed margin range is
-  [-3, +3]. Margins outside this range trigger a WARNING regardless of sign
-  (too-good means the fixture is no longer a known limit; too-bad means we
-  shipped a regression somewhere else that this fixture caught).
-
----
-
-## Run Record
-
-Every suite run appends an immutable record to `history/runs/<ts>-<label>.json`:
-
-```json
-{
-  "run_id": "2026-04-23T12:00:00Z-v3.6",
-  "version_label": "v3.6",
-  "git_sha": "fdb7428...",
-  "branch": "benchmark/v3.6-ab-...",
-  "n_per_fixture": 1,
-  "judge_model": "<recorded from ~/.codex/config.toml at run time; do not hardcode>",
-  "judge_effort": "xhigh",
-  "fixtures": [
-    {
-      "id": "F2-cli-medium-subcommand",
-      "variant": { "score": 92, "wall_s": 707, "tokens_agg": 108852, "disqualifier": false,
-                   "axes": {"spec": 23, "constraint": 23, "scope": 24, "quality": 22} },
-      "bare":    { "score": 81, "wall_s": 101, "tokens_agg": 55588,  "disqualifier": false,
-                   "axes": {"spec": 19, "constraint": 19, "scope": 20, "quality": 23} },
-      "winner": "variant",
-      "margin": 11,
-      "critical_findings": {
-        "variant": [],
-        "bare": ["silent catch in findSkillMdFiles (no-silent-catches violation)"]
-      }
-    }
-  ],
-  "suite": {
-    "fixtures_run": 9,
-    "variant_avg": 89.3,
-    "bare_avg": 75.0,
-    "margin_avg": 14.3,
-    "hard_floor_violations": 0,
-    "ship_gate": "PASS"
-  }
-}
-```
-
----
-
-## Fixture Rotation Policy
-
-If any fixture has all compared gated arms scoring > 95 for two consecutive
-shipped versions, it's saturated and no longer differentiates. Replace with a
-harder equivalent and record the swap in
-`history/runs/<ts>-fixture-rotation.json`:
-
-```json
-{
-  "retired": "F1-cli-trivial-flag",
-  "retired_reason": "all compared gated arms > 95 on v3.7 and v3.8 (saturation)",
-  "replacement": "F1b-cli-trivial-flag-v2",
-  "replacement_rationale": "adds exit-code precedence requirement that current leaders didn't handle on first try"
-}
-```
-
-Retired fixtures stay in `fixtures/retired/` for replay if a regression is
-suspected in their area.
-
----
-
-## Why These Thresholds
-
-- **+5 margin floor** — below this, the gated harness arm is not reliably
-  beating its lower baseline given judge variance (empirically ~±3 per axis).
-  For the legacy suite that is `variant` over `bare`; for pair evidence it is
-  the selected pair arm over `solo_claude`. Worth paying pipeline cost requires
-  margin clearly above noise.
-- **−5 regression floor** — one-axis regression can look like −5; allowing
-  less would let real regressions slip through.
-- **7-fixture coverage floor** — requires a broad enough set of
-  headroom-available, non-known-limit fixtures to clear the margin floor. This
-  preserves the original core-suite coverage bar without pretending the current
-  extended fixture inventory is still exactly nine fixtures.

package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/NOTES.md

@@ -1,34 +0,0 @@
-# F1 — Notes
-
-## Purpose
-
-Trivial-tier calibration. Every arm should one-shot this; it's here to catch
-catastrophic regressions and to anchor the "saturation" end of the scoring
-scale.
-
-Pair-candidate status: rejected by design. Because every arm is expected to
-one-shot F1, it is a trivial calibration/control fixture and must not be used
-as pair-lift evidence.
-
-## Failure mode
-
-- **Default-behavior regression.** Careless implementations add `--loud`
-  handling but accidentally alter the default case (e.g., always uppercasing
-  because the flag-check is misplaced). Verification commands 1 and 4 guard
-  against that.
-- **Scope creep.** Modifying unrelated code while "here" would be caught by
-  both CRITIC design sub-pass and the `git diff --stat` spec requirement.
-
-## Pipeline exercise
-
-- Phase 0 routing: expected `standard` route (no risk keywords).
-- Phase 1 BUILD: single-file edit.
-- Phase 1.4 BUILD GATE: `node --check` + `node --test` both must pass.
-- Phase 2 EVAL: minimal findings expected.
-- Phase 3 CRITIC design: verifies diff surgical-ness.
-
-## Rotation trigger
-
-When both `bare` and `solo_claude` score > 95 for two consecutive shipped
-versions, replace with a harder trivial fixture (e.g., one that requires
-handling a new flag interacting with existing flag precedence).

package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/expected.json

@@ -1,68 +0,0 @@
-{
-  "verification_commands": [
-    {
-      "cmd": "node bin/cli.js hello",
-      "exit_code": 0,
-      "stdout_contains": [
-        "Hello, world!"
-      ],
-      "stdout_not_contains": [
-        "HELLO"
-      ]
-    },
-    {
-      "cmd": "node bin/cli.js hello --loud",
-      "exit_code": 0,
-      "stdout_contains": [
-        "HELLO, WORLD!!"
-      ],
-      "stdout_not_contains": []
-    },
-    {
-      "cmd": "node bin/cli.js hello --loud --name alice",
-      "exit_code": 0,
-      "stdout_contains": [
-        "HELLO, ALICE!!"
-      ],
-      "stdout_not_contains": []
-    },
-    {
-      "cmd": "node bin/cli.js hello --name bob",
-      "exit_code": 0,
-      "stdout_contains": [
-        "Hello, bob!"
-      ],
-      "stdout_not_contains": [
-        "HELLO"
-      ]
-    },
-    {
-      "cmd": "node --test tests/cli.test.js",
-      "exit_code": 0,
-      "stdout_contains": [],
-      "stdout_not_contains": [
-        "not ok "
-      ]
-    }
-  ],
-  "forbidden_patterns": [
-    {
-      "pattern": "catch\\s*\\([^)]*\\)\\s*\\{[^}]*return\\s+(null|undefined|'')",
-      "description": "silent catch returning fallback",
-      "files": [
-        "bin/cli.js"
-      ],
-      "severity": "disqualifier"
-    }
-  ],
-  "required_files": [
-    "bin/cli.js",
-    "tests/cli.test.js"
-  ],
-  "forbidden_files": [],
-  "max_deps_added": 0,
-  "spec_output_files": [
-    "bin/cli.js",
-    "tests/cli.test.js"
-  ]
-}

package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/metadata.json

@@ -1,10 +0,0 @@
-{
-  "id": "F1-cli-trivial-flag",
-  "category": "trivial",
-  "difficulty": "trivial",
-  "timeout_seconds": 900,
-  "required_tools": ["node"],
-  "browser": false,
-  "deps_change_expected": false,
-  "intent": "Add a boolean --loud flag to bench-test-repo's hello subcommand. When passed, the greeting is uppercased and ends with '!!'. Default behavior unchanged. Update tests."
-}

package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/setup.sh

@@ -1,4 +0,0 @@
-#!/usr/bin/env bash
-# F1 setup — no changes to base test-repo needed.
-set -e
-exit 0

package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/spec.md

@@ -1,44 +0,0 @@
----
-id: "F1-cli-trivial-flag"
-title: "Add --loud flag to hello subcommand"
-status: planned
-complexity: trivial
-depends-on: []
----
-
-# F1 Add `--loud` flag to `hello`
-
-## Context
-
-The `hello` subcommand in `bin/cli.js` currently prints `Hello, <name>!`. A
-`--loud` flag gives users an emphatic variant without breaking the default.
-This is a low-risk edit used to calibrate trivial-tier fixture difficulty.
-
-## Requirements
-
-- [ ] `node bin/cli.js hello --loud` prints `HELLO, WORLD!!` (everything uppercased, two trailing exclamation marks).
-- [ ] `node bin/cli.js hello --loud --name alice` prints `HELLO, ALICE!!`.
-- [ ] `node bin/cli.js hello` (no flag) still prints `Hello, world!` (unchanged).
-- [ ] `node bin/cli.js hello --name bob` still prints `Hello, bob!` (unchanged).
-- [ ] Existing tests continue to pass. Add at least one test covering the `--loud` path.
-
-## Constraints
-
-- **No new npm dependencies.** Built-ins only.
-- **No silent catches.** If an unknown flag is passed, exit 1 with an informative message (same pattern as the existing `--name` handler).
-- **Surgical diff.** Only touch `bin/cli.js` and `tests/cli.test.js`. Do not reformat unrelated code.
-
-
-## Out of Scope
-
-- Adding unrelated flags (`--quiet`, `--locale`, etc.).
-- Refactoring the existing argument parser.
-- Touching `server/`, `web/`, or `tests/server.test.js`.
-
-## Verification
-
-- `node bin/cli.js hello` prints `Hello, world!` (exit 0).
-- `node bin/cli.js hello --loud` prints `HELLO, WORLD!!` (exit 0).
-- `node bin/cli.js hello --loud --name alice` prints `HELLO, ALICE!!` (exit 0).
-- `node --test tests/` passes all tests including the new `--loud` case.
-- `git diff --stat` shows only `bin/cli.js` and `tests/cli.test.js` touched.

package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/task.ko.txt

@@ -1,8 +0,0 @@
-bench-test-repo의 CLI(bin/cli.js)에 있는 `hello` 서브커맨드에 `--loud` 플래그를 추가해주세요. `--loud`가 전달되면 인사말이 대문자로 바뀌고 느낌표 두 개로 끝나야 합니다.
-
-예시:
-- `node bin/cli.js hello --loud` → `HELLO, WORLD!!`
-- `node bin/cli.js hello --loud --name alice` → `HELLO, ALICE!!`
-- `node bin/cli.js hello` → `Hello, world!` (기본값 변경 없음)
-
-기존 테스트가 계속 통과하는지 확인하고, `--loud` 경로에 대한 테스트를 최소 하나 추가해주세요. 관련 없는 파일은 건드리지 마세요 — `bin/cli.js`와 `tests/cli.test.js`만 수정하세요. 새로운 npm 의존성은 추가하지 마세요.

package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/task.txt

@@ -1,8 +0,0 @@
-Add a --loud flag to the `hello` subcommand in bench-test-repo's CLI (bin/cli.js). When --loud is passed, the greeting is uppercased and ends with two exclamation marks.
-
-For example:
-- `node bin/cli.js hello --loud` → `HELLO, WORLD!!`
-- `node bin/cli.js hello --loud --name alice` → `HELLO, ALICE!!`
-- `node bin/cli.js hello` → `Hello, world!` (unchanged default)
-
-Make sure existing tests still pass and add at least one test for the --loud path. Don't touch unrelated files — only `bin/cli.js` and `tests/cli.test.js`. No new npm dependencies.

package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/NOTES.md

@@ -1,68 +0,0 @@
-# F10 — Notes
-
-## Purpose
-
-Pair-discriminating high-risk fixture. Adds a write endpoint with persistent
-storage. The pair-edge mechanism: when an implementation reads-modifies-
-writes a JSON file in response to async requests, multiple in-flight POSTs
-can interleave during the await gap between read and write — producing
-duplicate ids or lost writes. A single-pass implementer typically writes
-the natural shape:
-
-```js
-const data = JSON.parse(await fs.readFile(...));
-data.items.push({ id: data.items.length + 1, ...body });
-await fs.writeFile(...);
-```
-
-This passes single-POST tests trivially. The failure only surfaces when
-multiple POSTs are in flight together. A reviewer with fresh eyes asking
-"what happens if two of these run at once?" catches it; the same model
-reviewing its own work tends to anchor on the "happy path" view.
-
-## Failure modes detected
-
-- **Duplicate ids** from `length + 1` after concurrent reads.
-- **Lost writes** from last-writer-wins overwrite of in-flight POSTs.
-- **No persistence** — implementer keeps in-memory only, ignoring the
-  restart-survival requirement. Caught by `data/items.json` byte check.
-- **Silent catch** wrapping the write path. Caught by forbidden_pattern.
-
-## Pipeline exercise
-
-- Phase 1 BUILD: implementer must derive that an awaited file read-modify-
-  write under parallel requests requires either serialization (mutex /
-  queue) or a unique-id source independent of array length.
-- Phase 2 EVAL: scrutinizes whether the new tests exercise the close-
-  together POST case rather than just single-POST happy path.
-- Phase 3 CRITIC: scope discipline + production-ready invariant on the
-  consistency claim.
-
-## Discrimination expectation
-
-Calibration target (set in pyx-memory project memory 2026-05-05):
-
-- bare arm: 40-65 (passes spec wording verbatim, fails parallel verifier).
-- solo arm: 60-78 (review pass may catch one mode, may miss another).
-- pair arm: 75-90 (cross-perspective derivation of the consistency
-  invariant from the English "close together" wording).
-
-## Public-spec wording — load-bearing
-
-The spec deliberately uses "close together" and "remain consistent"
-instead of trigger keywords. If the spec were to say "race condition",
-"concurrent", "atomic", or "lock", a single-pass solo arm would
-keyword-match the answer pattern and ace the fixture. The English
-prose forces invariant derivation, which is where pair has the edge.
-
-## Rotation trigger
-
-Headroom run `20260507-f10-f11-tier1-full-pipeline` rejected this fixture as
-pair-lift evidence: bare scored 75 and solo_claude scored 94. Keep it as a
-concurrent persistence control unless the visible contract is reworked to
-expose lower bare/solo ceilings.
-
-Retire when both `bare` and `solo_claude` consistently land > 90 across two
-shipped versions, OR when "close-together-write" becomes a recognized pattern
-such that solo arm reliably reaches for a serializing mechanism on first read.
-Whichever comes first.

package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/expected.json

@@ -1,60 +0,0 @@
-{
-  "verification_commands": [
-    {
-      "cmd": "node --test tests/server.test.js",
-      "exit_code": 0,
-      "stdout_contains": [],
-      "stdout_not_contains": ["not ok "]
-    },
-    {
-      "cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/parallel.js\"",
-      "exit_code": 0,
-      "stdout_contains": ["\"ok\":true"],
-      "stdout_not_contains": [],
-      "contract_refs": [
-        "After multiple POSTs (single or many at once), `GET /items` reflects every successful POST exactly once",
-        "Several POSTs sent close together produce a list with every item, each with a distinct id"
-      ]
-    },
-    {
-      "cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/invalid.js\"",
-      "exit_code": 0,
-      "stdout_contains": ["\"ok\":true"],
-      "stdout_not_contains": [],
-      "contract_refs": [
-        "Invalid body (missing `name`, missing `qty`, `qty` not a positive number) → respond `400` with `{ error: 'invalid_body', field: '<name>' }`",
-        "An invalid POST returns `400` and the persisted file is byte-identical to its pre-request contents."
-      ]
-    },
-    {
-      "cmd": "node -e 'const fs=require(\"fs\"); const d=JSON.parse(fs.readFileSync(\"data/items.json\",\"utf8\")); console.log(Array.isArray(d.items));'",
-      "exit_code": 0,
-      "stdout_contains": ["true"],
-      "stdout_not_contains": ["false", "undefined"]
-    }
-  ],
-  "forbidden_patterns": [
-    {
-      "pattern": "catch\\s*\\([^)]*\\)\\s*\\{[^}]*return\\s+(null|undefined|''|\\{\\})",
-      "description": "silent catch returning fallback in write path",
-      "files": ["server/index.js"],
-      "severity": "disqualifier"
-    },
-    {
-      "pattern": "catch\\s*\\([^)]*\\)\\s*\\{\\s*\\}",
-      "description": "empty catch block",
-      "files": ["server/index.js"],
-      "severity": "disqualifier"
-    },
-    {
-      "pattern": "/\\*\\s*eslint-disable",
-      "description": "eslint-disable without scoped justification",
-      "files": ["server/index.js"],
-      "severity": "disqualifier"
-    }
-  ],
-  "required_files": ["server/index.js", "tests/server.test.js", "data/items.json"],
-  "forbidden_files": [],
-  "max_deps_added": 0,
-  "spec_output_files": ["server/index.js", "tests/server.test.js"]
-}

package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/metadata.json

@@ -1,10 +0,0 @@
-{
-  "id": "F10-persist-write-collision",
-  "category": "high-risk",
-  "difficulty": "high",
-  "timeout_seconds": 1500,
-  "required_tools": ["node"],
-  "browser": false,
-  "deps_change_expected": false,
-  "intent": "Add POST /items that appends a new entry and persists the full list to data/items.json. The list and its ids must remain consistent when multiple POSTs arrive close together — every saved item ends up with a unique numeric id and none are lost."
-}

package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/setup.sh

@@ -1,17 +0,0 @@
-#!/usr/bin/env bash
-# F10 setup — seed data/items.json.
-set -e
-
-mkdir -p data
-
-# Seed with the same baseline the in-memory items started with.
-cat > data/items.json <<'JSON'
-{
-  "items": [
-    { "id": 1, "name": "alpha", "qty": 3 },
-    { "id": 2, "name": "beta", "qty": 5 }
-  ]
-}
-JSON
-
-exit 0

package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/spec.md

@@ -1,51 +0,0 @@
----
-id: "F10-persist-write-collision"
-title: "Add POST /items with persistent store"
-status: planned
-complexity: high
-depends-on: []
----
-
-# F10 Add POST /items with persistent store
-
-## Context
-
-`server/index.js` currently keeps `items` in an in-memory array that
-resets on restart. The task: add `POST /items` that appends a new entry
-and persists the full list to `data/items.json` so state survives
-restarts. Read paths (`GET /items`, `GET /items/:id`) load from the same
-persisted file.
-
-The list and every saved item's `id` must remain consistent when
-multiple POSTs arrive close together — no duplicate ids, no lost writes.
-
-## Requirements
-
-- [ ] `POST /items` accepts JSON body `{ name: string, qty: number }` and returns `201` with `{ item: { id, name, qty } }` on success.
-- [ ] `id` is a unique numeric value. Every successful POST returns an id no other item has, past or present.
-- [ ] After multiple POSTs (single or many at once), `GET /items` reflects every successful POST exactly once — `items.length` equals starting count plus successful POST count.
-- [ ] `data/items.json` on disk reflects the same state. Restarting the server preserves the items.
-- [ ] `GET /items` and `GET /items/:id` read from the persisted file (or from state initialized from it at startup, with all writes persisted before the response is returned).
-- [ ] Invalid body (missing `name`, missing `qty`, `qty` not a positive number) → respond `400` with `{ error: 'invalid_body', field: '<name>' }`. The persisted store must be unchanged after a 400 response.
-- [ ] `tests/server.test.js` is updated. Existing assertions still hold AND at least two new tests cover POST behavior, including a test that fires several POST requests close together and verifies the resulting list contains every item with distinct ids.
-
-## Constraints
-
-- **No new npm dependencies.** Use only Express + Node built-ins (`fs/promises`).
-- **No silent catches.** Any `try/catch` in the write path must surface failure as `500` with a clear body, not return a fake success.
-- **No hardcoded ids.** Existing baseline ids (1, 2) remain valid; new ids must not collide with any past or present id.
-- **No breaking change** to `GET /items` shape or `GET /items/:id` semantics.
-
-## Out of Scope
-
-- Authentication, rate limiting, validation libraries.
-- Database integration. The store is a single JSON file by design.
-- Touching `bin/cli.js`, `web/`, or `tests/cli.test.js`.
-
-## Verification
-
-- `node --test tests/server.test.js` exits 0 and the new tests cover POST.
-- Several POSTs sent close together produce a list with every item, each with a distinct id — assertable from a small Node script that POSTs in parallel and reads back `GET /items`.
-- After server restart, items posted before the restart are still present in `GET /items`.
-- An invalid POST returns `400` and the persisted file is byte-identical to its pre-request contents.
-- `git diff --stat` shows only `server/index.js` and `tests/server.test.js` touched (the `data/items.json` seed comes from setup, not the arm).

package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/task.txt

@@ -1,9 +0,0 @@
-Add `POST /items` to `server/index.js` so users can append a new item with `{ name, qty }`. The full list should be persisted to `data/items.json` so state survives restarts — the existing read endpoints (`GET /items`, `GET /items/:id`) should also use the same persisted file going forward.
-
-Every saved item must have a unique numeric id. If two or more POSTs arrive close together, the resulting list must contain every successful one with distinct ids — no duplicates, nothing lost.
-
-Invalid body (missing `name`, missing `qty`, `qty` not a positive number) → respond `400` with `{ error: 'invalid_body', field: '<name>' }`, and the persisted store must be unchanged after the 400.
-
-Update `tests/server.test.js` so existing tests still pass AND add at least two new tests covering POST behavior. One of them must fire several POSTs close together and verify the final list contains every item with distinct ids.
-
-No new npm dependencies. Only touch `server/index.js`, `tests/server.test.js`, and `data/items.json` (which is seeded for you).

package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/verifiers/invalid.js

@@ -1,29 +0,0 @@
-'use strict';
-const fs = require('fs');
-const http = require('http');
-const path = require('path');
-const { app } = require(path.join(process.env.BENCH_WORKDIR, 'server'));
-
-const before = fs.readFileSync('data/items.json');
-
-const s = http.createServer(app).listen(0, () => {
-  const { port } = s.address();
-  const req = http.request(
-    { host: '127.0.0.1', port, method: 'POST', path: '/items',
-      headers: { 'Content-Type': 'application/json' } },
-    (r) => {
-      let b = '';
-      r.on('data', (c) => (b += c));
-      r.on('end', () => {
-        const after = fs.readFileSync('data/items.json');
-        const same = before.equals(after);
-        const ok = r.statusCode === 400 && same;
-        console.log(JSON.stringify({ status: r.statusCode, store_unchanged: same, ok }));
-        s.close();
-        process.exit(ok ? 0 : 1);
-      });
-    }
-  );
-  req.write(JSON.stringify({ name: 'noqty' }));
-  req.end();
-});

package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/verifiers/parallel.js

@@ -1,50 +0,0 @@
-'use strict';
-const http = require('http');
-const path = require('path');
-const { app } = require(path.join(process.env.BENCH_WORKDIR, 'server'));
-
-const s = http.createServer(app).listen(0, () => {
-  const { port } = s.address();
-  const post = (body) => new Promise((resolve) => {
-    const req = http.request(
-      { host: '127.0.0.1', port, method: 'POST', path: '/items',
-        headers: { 'Content-Type': 'application/json' } },
-      (r) => {
-        let b = '';
-        r.on('data', (c) => (b += c));
-        r.on('end', () => resolve({ status: r.statusCode, body: b ? safeJson(b) : null }));
-      }
-    );
-    req.on('error', () => resolve({ status: 0, body: null }));
-    req.write(JSON.stringify(body));
-    req.end();
-  });
-
-  Promise.all([
-    post({ name: 'gamma', qty: 1 }),
-    post({ name: 'delta', qty: 2 }),
-    post({ name: 'epsilon', qty: 3 }),
-  ]).then(async (results) => {
-    await new Promise((r) => setTimeout(r, 250));
-    http.get(`http://127.0.0.1:${port}/items`, (r) => {
-      let b = '';
-      r.on('data', (c) => (b += c));
-      r.on('end', () => {
-        let data;
-        try { data = JSON.parse(b); } catch { data = { items: [] }; }
-        const items = Array.isArray(data.items) ? data.items : [];
-        const ids = items.map((i) => i && i.id);
-        const uniq = new Set(ids).size;
-        const allPosted = results.every((x) => x.status === 201);
-        const ok = allPosted && items.length === 5 && uniq === 5;
-        console.log(JSON.stringify({
-          len: items.length, uniq, status: results.map((r) => r.status), ok,
-        }));
-        s.close();
-        process.exit(ok ? 0 : 1);
-      });
-    });
-  });
-});
-
-function safeJson(s) { try { return JSON.parse(s); } catch { return null; } }