devlyn-cli 2.7.0 → 2.8.0

package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/setup.sh

@@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+# F35 setup — seed inventory and the replay journal.
+set -e
+
+mkdir -p data
+
+cat > data/inventory.json <<'JSON'
+{
+  "widget": 10,
+  "gadget": 4,
+  "gizmo": 0
+}
+JSON
+
+cat > data/journal.json <<'JSON'
+{
+  "applied": ["op-900"]
+}
+JSON
+
+exit 0

package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/spec.md

@@ -0,0 +1,58 @@
+---
+id: "F35-cli-apply-journal"
+title: "Apply command with journal replay and all-or-nothing persistence"
+status: planned
+complexity: high
+depends-on: []
+---
+
+# F35 Apply command with journal replay and all-or-nothing persistence
+
+## Context
+
+`bench-cli` gets an `apply` command that applies a file of inventory
+adjustments to `data/inventory.json`. Runs are re-deliverable: a journal at
+`data/journal.json` records which ops have already been applied, so a
+delivered-again op must not change inventory twice. A batch either fully
+persists or leaves both files untouched.
+
+## Requirements
+
+- [ ] `bench-cli apply --input <path>` reads JSON shaped as `{ "ops": [{ "op_id": string, "sku": string, "delta": number, "priority": number }] }`.
+- [ ] Quantities come from `data/inventory.json`; the applied-op record comes from the journal's `applied` array in `data/journal.json`. Both are seeded by setup.
+- [ ] Validation happens before any application. Invalid JSON, missing or non-string `op_id`/`sku`, unknown `sku`, non-integer `delta` or `priority`, or an `op_id` that appears more than once in the input file exits `2` and writes exactly one JSON error object to stderr with shape `{ "error": "invalid_input", "reason": string }`, printing nothing to stdout and changing neither file.
+- [ ] An op whose `op_id` is already listed in the journal's `applied` array is recorded as skipped and changes nothing, regardless of its other fields.
+- [ ] Non-skipped ops apply in descending `priority`; ties apply in file order.
+- [ ] An op fails when applying its `delta` would take the sku's quantity below zero, evaluated at its turn in that order, with exact error shape `{ "error": "insufficient_stock", "op_id": string, "sku": string, "available": number, "requested": number }` where `available` is the quantity at that op's turn and `requested` is the absolute value of its `delta`.
+- [ ] If any op fails, nothing persists: `data/inventory.json` and `data/journal.json` are byte-for-byte unchanged, the command exits `2`, and exactly one JSON error object is written to stderr with nothing on stdout.
+- [ ] On success, persist the final quantities to `data/inventory.json`, append the applied `op_id`s to the journal's `applied` array in application order, and write exactly one JSON object to stdout with keys `applied` (op_ids in application order), `skipped` (op_ids in file order), and `inventory` (final quantities for every sku). Nothing is written to stderr on success.
+- [ ] `tests/cli.test.js` is updated. Existing tests still pass AND at least two new tests cover `apply`: one success mixing a journaled op with different priorities, and one failing batch that leaves both files unchanged.
+
+## Constraints
+
+- **No new npm dependencies.**
+- **No silent catches.** If parsing or file reading fails, emit a visible JSON error to stderr and exit `2`.
+- **No extra stdout/stderr text** on the success path; downstream tooling parses stdout as JSON.
+- Quantities are integers; no fractional inventory.
+
+## Out of Scope
+
+- Changing server routes, web UI, or journal schema.
+- Concurrency across multiple `apply` processes.
+- Pruning or rewriting existing journal entries.
+- Touching `server/`, `web/`, or `tests/server.test.js`.
+
+## Verification
+
+- `node --test tests/cli.test.js` exits 0.
+- A batch mixing a journaled op, unequal priorities, and a priority tie yields exact `applied` order, exact `skipped`, exact final `inventory`, and the journal grows by exactly the applied op_ids in that order.
+- A batch whose failing op is only discoverable after higher-priority ops have consumed or added stock reports `available` at that op's turn and leaves both files byte-for-byte unchanged.
+- A duplicated `op_id` in the input file exits `2` with one JSON error on stderr and no stdout.
+- `git diff --stat` shows only `bin/cli.js` and `tests/cli.test.js` touched (inventory and journal are seeded by setup, not the arm).
+
+## Solo-headroom hypothesis
+
+A capable solo_claude baseline is expected to miss that each op's
+`available` stock is evaluated at its turn in priority order rather than
+against the pre-batch inventory or file order; observable command
+`node "$BENCH_FIXTURE_DIR/verifiers/rollback.js"` exposes the miss.

package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/task.txt

@@ -0,0 +1,9 @@
+Add an `apply` command to `bench-cli` so users can run `bench-cli apply --input <path>` with an ops JSON file of inventory adjustments. Quantities live in `data/inventory.json` and previously applied op ids live in the `applied` array of `data/journal.json` (both seeded).
+
+Validate the whole file before applying anything: bad JSON, missing/invalid fields, unknown sku, non-integer delta or priority, or the same `op_id` appearing twice in one file exits `2` with exactly one JSON error object `{ "error": "invalid_input", "reason": string }` on stderr, nothing on stdout, and neither file changed.
+
+An op whose `op_id` is already in the journal is recorded as skipped and changes nothing — whatever its other fields say. Everything else applies in descending `priority` (ties in file order). An op fails if its `delta` would push the sku below zero at its turn in that order; the error is the exact object `{ "error": "insufficient_stock", "op_id", "sku", "available", "requested" }` with `available` measured at that op's turn. Any failure means nothing persists — inventory and journal stay byte-for-byte identical and the command exits `2`.
+
+On success, write the final quantities back to `data/inventory.json`, append applied op ids to the journal in application order, and print exactly one JSON object with `applied` (application order), `skipped` (file order), and `inventory` — with nothing on stderr.
+
+Update `tests/cli.test.js`: existing tests still pass, plus at least two apply tests (one success mixing a journaled op with different priorities, one failing batch that leaves both files unchanged). No new npm dependencies. Only touch `bin/cli.js` and `tests/cli.test.js`.

package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/verifiers/dup-op.js

@@ -0,0 +1,54 @@
+'use strict';
+// F35 hidden verifier — an op_id duplicated WITHIN the input file is a
+// validation error (whole file rejected, nothing changes), distinct from a
+// journal replay which is a per-op skip. Exit 2, one JSON error object on
+// stderr, no stdout, both files untouched.
+
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const { spawnSync } = require('node:child_process');
+
+const work = process.env.BENCH_WORKDIR;
+const invPath = path.join(work, 'data', 'inventory.json');
+const jrnPath = path.join(work, 'data', 'journal.json');
+
+fs.writeFileSync(invPath, JSON.stringify({ widget: 10, gadget: 4, gizmo: 0 }, null, 2) + '\n');
+fs.writeFileSync(jrnPath, JSON.stringify({ applied: ['op-900'] }, null, 2) + '\n');
+const invBefore = fs.readFileSync(invPath, 'utf8');
+const jrnBefore = fs.readFileSync(jrnPath, 'utf8');
+
+const inputPath = path.join(os.tmpdir(), `apply-dup-${process.pid}.json`);
+fs.writeFileSync(inputPath, JSON.stringify({
+  ops: [
+    { op_id: 'op-20', sku: 'widget', delta: -1, priority: 3 },
+    { op_id: 'op-21', sku: 'gadget', delta: 1, priority: 2 },
+    { op_id: 'op-20', sku: 'gizmo', delta: 2, priority: 1 }
+  ]
+}));
+
+const cli = path.join(work, 'bin', 'cli.js');
+const result = spawnSync('node', [cli, 'apply', '--input', inputPath], {
+  cwd: work,
+  encoding: 'utf8'
+});
+
+let err;
+try {
+  err = JSON.parse(result.stderr);
+} catch {
+  err = null;
+}
+
+const filesUnchanged = fs.readFileSync(invPath, 'utf8') === invBefore
+  && fs.readFileSync(jrnPath, 'utf8') === jrnBefore;
+
+const ok = result.status === 2
+  && result.stdout === ''
+  && err !== null
+  && err.error === 'invalid_input'
+  && typeof err.reason === 'string'
+  && filesUnchanged;
+
+console.log(JSON.stringify({ ok, status: result.status, stdout: result.stdout, err, filesUnchanged }));
+process.exit(ok ? 0 : 1);

package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/verifiers/priority-apply.js

@@ -0,0 +1,82 @@
+'use strict';
+// F35 hidden verifier — success path mixing a journaled (replayed) op, unequal
+// priorities, and a priority tie. Final quantities are order-independent
+// (addition commutes), so the discriminator is the exact `applied` ORDER and
+// the journal append order: priority-desc with file-order ties, replayed op
+// skipped regardless of its (otherwise failing) fields.
+
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const { spawnSync } = require('node:child_process');
+
+// Structural equality: object key order is NOT part of the visible contract;
+// array order IS. Codex fairness-audit fix (stringify equality made JSON
+// property order a hidden oracle narrower than spec.md).
+function deepEqual(a, b) {
+  if (a === b) return true;
+  if (Array.isArray(a) && Array.isArray(b)) {
+    return a.length === b.length && a.every((v, i) => deepEqual(v, b[i]));
+  }
+  if (a && b && typeof a === 'object' && typeof b === 'object' && !Array.isArray(a) && !Array.isArray(b)) {
+    const ka = Object.keys(a).sort();
+    const kb = Object.keys(b).sort();
+    return deepEqual(ka, kb) && ka.every((k) => deepEqual(a[k], b[k]));
+  }
+  return false;
+}
+
+const work = process.env.BENCH_WORKDIR;
+const invPath = path.join(work, 'data', 'inventory.json');
+const jrnPath = path.join(work, 'data', 'journal.json');
+
+// Reset seeds so this verifier is order-independent from other commands.
+fs.writeFileSync(invPath, JSON.stringify({ widget: 10, gadget: 4, gizmo: 0 }, null, 2) + '\n');
+fs.writeFileSync(jrnPath, JSON.stringify({ applied: ['op-900'] }, null, 2) + '\n');
+
+const inputPath = path.join(os.tmpdir(), `apply-ok-${process.pid}.json`);
+fs.writeFileSync(inputPath, JSON.stringify({
+  ops: [
+    { op_id: 'op-1', sku: 'widget', delta: -4, priority: 1 },
+    { op_id: 'op-900', sku: 'widget', delta: -9, priority: 9 },
+    { op_id: 'op-2', sku: 'widget', delta: -5, priority: 5 },
+    { op_id: 'op-3', sku: 'gadget', delta: -4, priority: 5 },
+    { op_id: 'op-4', sku: 'widget', delta: 3, priority: 0 }
+  ]
+}));
+
+const cli = path.join(work, 'bin', 'cli.js');
+const result = spawnSync('node', [cli, 'apply', '--input', inputPath], {
+  cwd: work,
+  encoding: 'utf8'
+});
+
+let out;
+try {
+  out = JSON.parse(result.stdout);
+} catch {
+  out = null;
+}
+
+const expected = {
+  applied: ['op-2', 'op-3', 'op-1', 'op-4'],
+  skipped: ['op-900'],
+  inventory: { widget: 4, gadget: 0, gizmo: 0 }
+};
+
+let inv; let jrn;
+try {
+  inv = JSON.parse(fs.readFileSync(invPath, 'utf8'));
+  jrn = JSON.parse(fs.readFileSync(jrnPath, 'utf8'));
+} catch {
+  inv = null; jrn = null;
+}
+
+const ok = result.status === 0
+  && result.stderr === ''
+  && deepEqual(out, expected)
+  && deepEqual(inv, expected.inventory)
+  && deepEqual(jrn, { applied: ['op-900', 'op-2', 'op-3', 'op-1', 'op-4'] });
+
+console.log(JSON.stringify({ ok, status: result.status, stderr: result.stderr, out, inv, jrn }));
+process.exit(ok ? 0 : 1);

package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/verifiers/rollback.js

@@ -0,0 +1,77 @@
+'use strict';
+// F35 hidden verifier — the failing op is only discoverable AFTER a
+// higher-priority op has added stock: op-12 (priority 8) raises widget
+// 10 -> 15, then op-11 (priority 5) requests 20 and fails with
+// available 15 — not 10 (pre-batch / file-order would report 10).
+// Both files must be byte-for-byte unchanged afterward.
+
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const { spawnSync } = require('node:child_process');
+
+// Structural equality: object key order is NOT part of the visible contract;
+// array order IS. Codex fairness-audit fix (stringify equality made JSON
+// property order a hidden oracle narrower than spec.md).
+function deepEqual(a, b) {
+  if (a === b) return true;
+  if (Array.isArray(a) && Array.isArray(b)) {
+    return a.length === b.length && a.every((v, i) => deepEqual(v, b[i]));
+  }
+  if (a && b && typeof a === 'object' && typeof b === 'object' && !Array.isArray(a) && !Array.isArray(b)) {
+    const ka = Object.keys(a).sort();
+    const kb = Object.keys(b).sort();
+    return deepEqual(ka, kb) && ka.every((k) => deepEqual(a[k], b[k]));
+  }
+  return false;
+}
+
+const work = process.env.BENCH_WORKDIR;
+const invPath = path.join(work, 'data', 'inventory.json');
+const jrnPath = path.join(work, 'data', 'journal.json');
+
+fs.writeFileSync(invPath, JSON.stringify({ widget: 10, gadget: 4, gizmo: 0 }, null, 2) + '\n');
+fs.writeFileSync(jrnPath, JSON.stringify({ applied: ['op-900'] }, null, 2) + '\n');
+const invBefore = fs.readFileSync(invPath, 'utf8');
+const jrnBefore = fs.readFileSync(jrnPath, 'utf8');
+
+const inputPath = path.join(os.tmpdir(), `apply-fail-${process.pid}.json`);
+fs.writeFileSync(inputPath, JSON.stringify({
+  ops: [
+    { op_id: 'op-10', sku: 'gadget', delta: 2, priority: 9 },
+    { op_id: 'op-11', sku: 'widget', delta: -20, priority: 5 },
+    { op_id: 'op-12', sku: 'widget', delta: 5, priority: 8 }
+  ]
+}));
+
+const cli = path.join(work, 'bin', 'cli.js');
+const result = spawnSync('node', [cli, 'apply', '--input', inputPath], {
+  cwd: work,
+  encoding: 'utf8'
+});
+
+let err;
+try {
+  err = JSON.parse(result.stderr);
+} catch {
+  err = null;
+}
+
+const expectedErr = {
+  error: 'insufficient_stock',
+  op_id: 'op-11',
+  sku: 'widget',
+  available: 15,
+  requested: 20
+};
+
+const filesUnchanged = fs.readFileSync(invPath, 'utf8') === invBefore
+  && fs.readFileSync(jrnPath, 'utf8') === jrnBefore;
+
+const ok = result.status === 2
+  && result.stdout === ''
+  && deepEqual(err, expectedErr)
+  && filesUnchanged;
+
+console.log(JSON.stringify({ ok, status: result.status, stdout: result.stdout, err, filesUnchanged }));
+process.exit(ok ? 0 : 1);

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/NOTES.md

@@ -0,0 +1,99 @@
+# F36 Session admission command with a global concurrency cap
+
+## Failure mode
+
+Detects implementations that get interval-admission logic correct at small,
+hand-written scale but reach for a data structure that stops scaling once
+the number of concurrently active sessions grows large — most naturally, a
+plain array of admitted sessions that gets linearly scanned or filtered on
+every new candidate. That shape is indistinguishable from the fully correct
+answer on any test an agent is likely to hand-write, and only fails once the
+active set stays large across a long run (Tier-2 axis: performance/scale,
+never previously used in this suite — no existing fixture verifier measures
+wall-clock; grepped `verifiers/*.js` across active + retired fixtures for
+`Date.now|hrtime|performance.now|elapsed`, zero hits before this one).
+Compounded with four small-scale correctness sub-traps so a fast-but-wrong
+shortcut and a correct-but-slow shortcut both lose points independently:
+half-open boundary (touching, not overlapping), start-time eligibility
+order vs. file order, per-candidate (not static) `active_at_start`, and
+eviction that must be driven by actual expiry (`end`) rather than admission
+order — the last one specifically closes a FIFO-queue shortcut that would
+otherwise pass the scale case by coincidence, since the scale case uses a
+uniform session duration and a FIFO queue's admission order happens to
+coincide with its expiry order there (Codex R2 catch, see Rounds below).
+
+## Fairness
+
+Every verifier assertion quotes a visible spec.md bullet via `contract_refs`
+(lint-fixtures enforced). The 150,000-session performance requirement is
+stated as a plain, numeric, visible spec bullet — no mechanism vocabulary
+(no "sort", "heap", "priority queue", "efficient", "O(n log n)") appears
+anywhere in `spec.md` or `task.txt`; only the observable behavior (must
+finish within the process timeout, must not change the answer) is named.
+The scale verifier's 15-second hard kill (`spawnSync` `timeout` option) sits
+more than 100x above the efficient reference's actual running time and
+correspondingly far below any realistic degraded-shortcut running time on
+this fixture's parameters, so the gate has a wide, non-flaky margin in both
+directions.
+
+## Pipeline phase(s) tested
+
+BUILD_GATE / VERIFY: an agent's own test suite is overwhelmingly likely to
+be small-N (a handful of hand-written sessions), so nothing in a normal
+iterative dev loop surfaces the scale defect; only the hidden, harness-side
+scale verifier (never visible to the arm) exercises it. This is a
+structurally different discriminator from every prior fixture in this
+suite, which relied entirely on a single hidden semantic trap rather than a
+mechanical, orthogonal performance axis.
+
+## Why another fixture can't cover this
+
+No active or retired fixture in this suite measures wall-clock/algorithmic
+complexity at all (verified by grep, see Failure mode above). Every prior
+headroom attempt (F27, F28, F30, F34, F35, S2-S6 — see
+`pair-rejected-fixtures.sh`) relied on a single hidden ordering/rollback/
+atomicity trap, and solo_claude scored 88-100 on all of them. This fixture
+tests a category of miss (asymptotic complexity blindness under a full
+self-review pipeline) that none of those exercise.
+
+## When to retire or replace
+
+If a headroom run shows solo_claude reliably recognizes and implements the
+required efficient structure even without prompting (i.e. the visible
+numeric scale requirement alone is enough to trigger scale-aware design),
+retire per the same honest-retirement contract used for F27/F28/F30/F34/F35
+and add this id to `pair-rejected-fixtures.sh` with the observed scores.
+
+## Rounds — Codex GPT-5.5 collaboration (read-only, `codex-monitored.sh`)
+
+**Round 1** (initial design: performance stated as a secondary condition on
+top of a single hidden ordering trap, textbook "meeting rooms with
+capacity" shape): Codex verdict **needs-rework** — "200,000 sessions" alone
+is enough for a strong PLAN phase to infer scale pressure and pattern-match
+the well-known interval-scheduling/heap shape; the boundary traps alone
+weren't judged sufficient given solo already solved comparable explicit
+order/rollback traps in F34/F35. Adopted: make performance the dominant
+axis by choosing scale-case parameters (uniform long overlapping durations,
+capacity held well below the overlap window) so that BOTH a brute
+nested-loop AND a "linear-scan a plain active-sessions array" middle-tier
+shortcut fail — only a genuinely sub-linear-per-op structure survives, not
+merely "some awareness of scale."
+
+**Round 2** (revised design presented): Codex verdict **converge, with one
+fix** — flagged that because the scale case uses a uniform session
+duration, admission order and expiry order coincide, so a FIFO
+queue-with-front-eviction (simpler than the general answer, and not the
+"real" discriminator intended) would also pass the scale case correctly and
+fast. Fix adopted: added the fourth small-scale correctness sub-trap
+(eviction driven by actual smallest `end` among currently active sessions,
+not admission order) via a hand-authored 3-session case where a FIFO
+shortcut and the correct rule disagree. This closes the loophole without
+touching the scale case's parameters, since the two verifiers are
+independent files/inputs.
+
+Codex's own words, R2 answer to Q1: "F36 is much stronger now, but I see one
+real shortcut left: a FIFO/sliding-window queue of admitted sessions can
+pass the scale case if ends are monotonic... Add one small correctness case
+where admitted end order differs from start/admission order... That defeats
+FIFO while keeping the scale discriminator intact." Adopted verbatim as
+correctness sub-trap 4.

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/RETIRED.md

@@ -0,0 +1,18 @@
+# F36 retired
+
+Retired from the active golden suite after headroom run
+`iter-0041-headroom-f36-f37`.
+
+Reason: `solo_claude` scored 96, exceeding the headroom ceiling of 80, and
+timed out doing so (full 1500s budget). bare scored 50 — the bare ceiling
+held (headroom 10), and bare failed on the intended traps (wrong deferred-row
+output shape: reason `capacity`/key `active` instead of
+`over_capacity`/`blocking`, plus a validation miss). But solo's pipeline
+still produced a scalable heap-based implementation that cleared the hidden
+scale verifier, so there is no pair-lift headroom to measure. The
+performance/scale axis alone does not hold the solo pipeline under 80.
+
+Future use: the small-scale correctness traps discriminated bare as designed;
+the scale verifier did not discriminate solo. A future rework needs a
+difficulty axis that survives a full PLAN/BUILD_GATE loop, not just a
+single-pass implementation.

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/expected.json

@@ -0,0 +1,64 @@
+{
+  "verification_commands": [
+    {
+      "cmd": "node --test tests/cli.test.js",
+      "exit_code": 0,
+      "stdout_contains": [],
+      "stdout_not_contains": ["not ok "]
+    },
+    {
+      "cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/correctness-small.js\"",
+      "exit_code": 0,
+      "stdout_contains": ["\"ok\":true"],
+      "stdout_not_contains": [],
+      "contract_refs": [
+        "an admitted session whose `end` equals the candidate's `start` has already ended and does not count against it",
+        "Sessions become eligible for admission in ascending order of `start`; ties are broken by the session's position in the input `sessions` array",
+        "Each deferred row reports `blocking`: the ids of the currently-active admitted sessions preventing its admission, in the order those sessions were themselves admitted"
+      ]
+    },
+    {
+      "cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/validation.js\"",
+      "exit_code": 0,
+      "stdout_contains": ["\"ok\":true"],
+      "stdout_not_contains": [],
+      "contract_refs": [
+        "Validation happens before any admission decision.",
+        "a duplicate `id` uses shape `{ \"error\": \"duplicate_session_id\", \"id\": string }`"
+      ]
+    },
+    {
+      "cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/scale.js\"",
+      "exit_code": 0,
+      "stdout_contains": ["\"ok\":true"],
+      "stdout_not_contains": [],
+      "contract_refs": [
+        "`bench-cli admit` must finish processing the 150,000-session verification input (see `## Verification`) within its process timeout, without changing which sessions are admitted or deferred compared to a smaller input processed under the same rule."
+      ]
+    }
+  ],
+  "forbidden_patterns": [
+    {
+      "pattern": "catch\\s*\\([^)]*\\)\\s*\\{[^}]*return\\s+(null|undefined|''|\\{\\})",
+      "description": "silent catch returning fallback in admit path",
+      "files": ["bin/cli.js"],
+      "severity": "disqualifier"
+    },
+    {
+      "pattern": "catch\\s*\\([^)]*\\)\\s*\\{\\s*\\}",
+      "description": "empty catch block",
+      "files": ["bin/cli.js"],
+      "severity": "disqualifier"
+    },
+    {
+      "pattern": "/\\*\\s*eslint-disable",
+      "description": "eslint-disable without scoped justification",
+      "files": ["bin/cli.js"],
+      "severity": "disqualifier"
+    }
+  ],
+  "required_files": ["bin/cli.js", "tests/cli.test.js"],
+  "forbidden_files": [],
+  "max_deps_added": 0,
+  "spec_output_files": ["bin/cli.js", "tests/cli.test.js"]
+}

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/metadata.json

@@ -0,0 +1,10 @@
+{
+  "id": "F36-cli-session-admission",
+  "category": "high-risk",
+  "difficulty": "high",
+  "timeout_seconds": 1500,
+  "required_tools": ["node"],
+  "browser": false,
+  "deps_change_expected": false,
+  "intent": "Add a bench-cli admit command that admits scheduling sessions against a global concurrency capacity in start-time order with file-order tie-break, deferring sessions that would exceed capacity, and that must finish processing a 150,000-session verification input within its process timeout without changing which sessions are admitted or deferred."
+}

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/setup.sh

@@ -0,0 +1,4 @@
+#!/usr/bin/env bash
+# F36 setup — no seed data required; sessions arrive entirely via --input.
+set -e
+exit 0

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/spec.md

@@ -0,0 +1,61 @@
+---
+id: "F36-cli-session-admission"
+title: "Session admission command with a global concurrency cap"
+status: planned
+complexity: high
+depends-on: []
+---
+
+# F36 Session admission command with a global concurrency cap
+
+## Context
+
+`bench-cli` gets an `admit` command that decides which of a batch of
+scheduling sessions can run at all, given a single global limit on how many
+sessions may be concurrently active. Downstream capacity-planning tooling
+parses stdout as JSON and replays the verification input at production
+scale, so the decision must be exact and the command must finish in
+reasonable time even on large batches.
+
+## Requirements
+
+- [ ] `bench-cli admit --input <path>` reads JSON shaped as `{ "capacity": number, "sessions": [{ "id": string, "start": number, "end": number }] }`.
+- [ ] Sessions become eligible for admission in ascending order of `start`; ties are broken by the session's position in the input `sessions` array (the earlier entry is eligible first).
+- [ ] Sessions are considered for admission one at a time, in that eligibility order. A session is admitted unless, at the moment it is considered, the number of already-admitted sessions whose interval has not yet ended is already equal to `capacity` — in that case it is deferred instead. A session's interval `[start, end)` is half-open: an admitted session whose `end` equals the candidate's `start` has already ended and does not count against it.
+- [ ] Each deferred row reports `blocking`: the ids of the currently-active admitted sessions preventing its admission, in the order those sessions were themselves admitted (not a final or static list, and not the list for any other candidate).
+- [ ] `bench-cli admit` must finish processing the 150,000-session verification input (see `## Verification`) within its process timeout, without changing which sessions are admitted or deferred compared to a smaller input processed under the same rule.
+- [ ] Validation happens before any admission decision. `capacity` must be a positive integer; `sessions` must be an array; every session needs a non-empty string `id`, integer `start`, and integer `end` with `start < end`; every `id` must be unique across the whole input. Invalid input exits `2` and writes exactly one JSON error object to stderr, printing nothing to stdout: a malformed capacity/session field uses shape `{ "error": "invalid_input", "reason": string }`; a duplicate `id` uses shape `{ "error": "duplicate_session_id", "id": string }`.
+- [ ] On success, write exactly one JSON object to stdout and no stderr. Keys: `admitted` (array of admitted session ids, in eligibility order) and `deferred` (array of `{ "id": string, "reason": "over_capacity", "blocking": [string, ...] }` rows, in eligibility order).
+- [ ] `tests/cli.test.js` is updated. Existing tests still pass AND at least two new tests cover `admit`: one where a session listed earlier in the file but starting later must not be admitted ahead of a session listed later but starting earlier, and one deferral.
+
+## Constraints
+
+- **No new npm dependencies.**
+- **No silent catches.** If parsing or file reading fails, emit a visible JSON error to stderr and exit `2`.
+- **No extra stdout/stderr text** on the success path; downstream tooling parses stdout as JSON.
+- `data`, `server`, and `web` directories are unrelated to this command; do not add files there.
+
+## Out of Scope
+
+- Persisting admission results between command invocations.
+- Retrying, rescheduling, or suggesting alternate times for deferred sessions.
+- Changing server routes, web UI, or any file outside `bin/cli.js` and `tests/cli.test.js`.
+- Touching `server/`, `web/`, or `tests/server.test.js`.
+
+## Verification
+
+- `node --test tests/cli.test.js` exits 0.
+- Given three sessions where the second-admitted session's interval ends before the first-admitted session's interval ends, a fourth session starting after the second has ended but before the first has ended is correctly admitted (its admission depends on evicting the second, not the first, from the active set) — `node "$BENCH_FIXTURE_DIR/verifiers/correctness-small.js"` exposes misses on this and the eligibility-order, half-open-boundary, and `blocking` rules above.
+- Malformed input, non-unique ids, and a non-positive capacity are all rejected before any admission decision, per the exact error shapes above — `node "$BENCH_FIXTURE_DIR/verifiers/validation.js"` exposes misses.
+- A 150,000-session input with `capacity` set low relative to heavily overlapping session durations completes within the process timeout and produces the exact admitted/deferred sets — `node "$BENCH_FIXTURE_DIR/verifiers/scale.js"` exposes misses on both timing and correctness.
+- `git diff --stat` shows only `bin/cli.js` and `tests/cli.test.js` touched.
+
+## Solo-headroom hypothesis
+
+A capable solo_claude baseline is expected to reach for a data structure
+that is correct on typical hand-written examples but does not stay fast as
+the number of concurrently active sessions grows — for example, keeping
+admitted sessions in a plain list and scanning or filtering it on every new
+candidate — which is correct at small scale but does not finish the
+150,000-session input in time; observable command
+`node "$BENCH_FIXTURE_DIR/verifiers/scale.js"` exposes the miss.

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/task.txt

@@ -0,0 +1,9 @@
+Add an `admit` command to `bench-cli` so users can run `bench-cli admit --input <path>` with a JSON file shaped as `{ "capacity": number, "sessions": [{ "id": string, "start": number, "end": number }] }`. Sessions become eligible for admission in ascending order of `start`; if two sessions share the same `start`, the one listed earlier in the `sessions` array is eligible first.
+
+Consider sessions one at a time in that eligibility order. Admit a session unless the number of already-admitted sessions whose interval hasn't ended yet is already equal to `capacity` — in that case defer it instead. Treat intervals as half-open: a session that ends exactly when another starts is not overlapping it. Each deferred session should report which already-admitted sessions are still active at its own turn (their ids, in the order those sessions were admitted) — this can differ across deferred sessions depending on when each one is considered, since a session admitted earlier isn't necessarily still active later.
+
+This command needs to stay fast: downstream capacity-planning tooling replays inputs with well over a hundred thousand sessions, and the command must still finish within a reasonable process timeout without changing which sessions get admitted or deferred versus a small input processed the same way.
+
+Validate the whole input before making any admission decision: `capacity` must be a positive integer, `sessions` must be an array, every session needs a non-empty string `id` and integer `start`/`end` with `start` less than `end`, and every `id` must be unique. On any violation, exit 2, print exactly one JSON error object to stderr, and print nothing to stdout — use a distinct error for duplicate ids versus other malformed input. On success, print exactly one JSON object with `admitted` (ids in eligibility order) and `deferred` (rows with id/reason/active-count, in eligibility order) — nothing on stderr.
+
+Update `tests/cli.test.js`: existing tests still pass, plus at least two new tests for `admit` (one exercising the start-time-vs-file-order tie-break, one deferral). No new npm dependencies. Only touch `bin/cli.js` and `tests/cli.test.js`.