devlyn-cli 3.2.1 → 3.3.0

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/NOTES.md

@@ -1,99 +0,0 @@
-# F36 Session admission command with a global concurrency cap
-
-## Failure mode
-
-Detects implementations that get interval-admission logic correct at small,
-hand-written scale but reach for a data structure that stops scaling once
-the number of concurrently active sessions grows large — most naturally, a
-plain array of admitted sessions that gets linearly scanned or filtered on
-every new candidate. That shape is indistinguishable from the fully correct
-answer on any test an agent is likely to hand-write, and only fails once the
-active set stays large across a long run (Tier-2 axis: performance/scale,
-never previously used in this suite — no existing fixture verifier measures
-wall-clock; grepped `verifiers/*.js` across active + retired fixtures for
-`Date.now|hrtime|performance.now|elapsed`, zero hits before this one).
-Compounded with four small-scale correctness sub-traps so a fast-but-wrong
-shortcut and a correct-but-slow shortcut both lose points independently:
-half-open boundary (touching, not overlapping), start-time eligibility
-order vs. file order, per-candidate (not static) `active_at_start`, and
-eviction that must be driven by actual expiry (`end`) rather than admission
-order — the last one specifically closes a FIFO-queue shortcut that would
-otherwise pass the scale case by coincidence, since the scale case uses a
-uniform session duration and a FIFO queue's admission order happens to
-coincide with its expiry order there (Codex R2 catch, see Rounds below).
-
-## Fairness
-
-Every verifier assertion quotes a visible spec.md bullet via `contract_refs`
-(lint-fixtures enforced). The 150,000-session performance requirement is
-stated as a plain, numeric, visible spec bullet — no mechanism vocabulary
-(no "sort", "heap", "priority queue", "efficient", "O(n log n)") appears
-anywhere in `spec.md` or `task.txt`; only the observable behavior (must
-finish within the process timeout, must not change the answer) is named.
-The scale verifier's 15-second hard kill (`spawnSync` `timeout` option) sits
-more than 100x above the efficient reference's actual running time and
-correspondingly far below any realistic degraded-shortcut running time on
-this fixture's parameters, so the gate has a wide, non-flaky margin in both
-directions.
-
-## Pipeline phase(s) tested
-
-BUILD_GATE / VERIFY: an agent's own test suite is overwhelmingly likely to
-be small-N (a handful of hand-written sessions), so nothing in a normal
-iterative dev loop surfaces the scale defect; only the hidden, harness-side
-scale verifier (never visible to the arm) exercises it. This is a
-structurally different discriminator from every prior fixture in this
-suite, which relied entirely on a single hidden semantic trap rather than a
-mechanical, orthogonal performance axis.
-
-## Why another fixture can't cover this
-
-No active or retired fixture in this suite measures wall-clock/algorithmic
-complexity at all (verified by grep, see Failure mode above). Every prior
-headroom attempt (F27, F28, F30, F34, F35, S2-S6 — see
-`pair-rejected-fixtures.sh`) relied on a single hidden ordering/rollback/
-atomicity trap, and solo_claude scored 88-100 on all of them. This fixture
-tests a category of miss (asymptotic complexity blindness under a full
-self-review pipeline) that none of those exercise.
-
-## When to retire or replace
-
-If a headroom run shows solo_claude reliably recognizes and implements the
-required efficient structure even without prompting (i.e. the visible
-numeric scale requirement alone is enough to trigger scale-aware design),
-retire per the same honest-retirement contract used for F27/F28/F30/F34/F35
-and add this id to `pair-rejected-fixtures.sh` with the observed scores.
-
-## Rounds — Codex GPT-5.5 collaboration (read-only, `codex-monitored.sh`)
-
-**Round 1** (initial design: performance stated as a secondary condition on
-top of a single hidden ordering trap, textbook "meeting rooms with
-capacity" shape): Codex verdict **needs-rework** — "200,000 sessions" alone
-is enough for a strong PLAN phase to infer scale pressure and pattern-match
-the well-known interval-scheduling/heap shape; the boundary traps alone
-weren't judged sufficient given solo already solved comparable explicit
-order/rollback traps in F34/F35. Adopted: make performance the dominant
-axis by choosing scale-case parameters (uniform long overlapping durations,
-capacity held well below the overlap window) so that BOTH a brute
-nested-loop AND a "linear-scan a plain active-sessions array" middle-tier
-shortcut fail — only a genuinely sub-linear-per-op structure survives, not
-merely "some awareness of scale."
-
-**Round 2** (revised design presented): Codex verdict **converge, with one
-fix** — flagged that because the scale case uses a uniform session
-duration, admission order and expiry order coincide, so a FIFO
-queue-with-front-eviction (simpler than the general answer, and not the
-"real" discriminator intended) would also pass the scale case correctly and
-fast. Fix adopted: added the fourth small-scale correctness sub-trap
-(eviction driven by actual smallest `end` among currently active sessions,
-not admission order) via a hand-authored 3-session case where a FIFO
-shortcut and the correct rule disagree. This closes the loophole without
-touching the scale case's parameters, since the two verifiers are
-independent files/inputs.
-
-Codex's own words, R2 answer to Q1: "F36 is much stronger now, but I see one
-real shortcut left: a FIFO/sliding-window queue of admitted sessions can
-pass the scale case if ends are monotonic... Add one small correctness case
-where admitted end order differs from start/admission order... That defeats
-FIFO while keeping the scale discriminator intact." Adopted verbatim as
-correctness sub-trap 4.

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/RETIRED.md

@@ -1,18 +0,0 @@
-# F36 retired
-
-Retired from the active golden suite after headroom run
-`iter-0041-headroom-f36-f37`.
-
-Reason: `solo_claude` scored 96, exceeding the headroom ceiling of 80, and
-timed out doing so (full 1500s budget). bare scored 50 — the bare ceiling
-held (headroom 10), and bare failed on the intended traps (wrong deferred-row
-output shape: reason `capacity`/key `active` instead of
-`over_capacity`/`blocking`, plus a validation miss). But solo's pipeline
-still produced a scalable heap-based implementation that cleared the hidden
-scale verifier, so there is no pair-lift headroom to measure. The
-performance/scale axis alone does not hold the solo pipeline under 80.
-
-Future use: the small-scale correctness traps discriminated bare as designed;
-the scale verifier did not discriminate solo. A future rework needs a
-difficulty axis that survives a full PLAN/BUILD_GATE loop, not just a
-single-pass implementation.

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/expected.json

@@ -1,64 +0,0 @@
-{
-  "verification_commands": [
-    {
-      "cmd": "node --test tests/cli.test.js",
-      "exit_code": 0,
-      "stdout_contains": [],
-      "stdout_not_contains": ["not ok "]
-    },
-    {
-      "cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/correctness-small.js\"",
-      "exit_code": 0,
-      "stdout_contains": ["\"ok\":true"],
-      "stdout_not_contains": [],
-      "contract_refs": [
-        "an admitted session whose `end` equals the candidate's `start` has already ended and does not count against it",
-        "Sessions become eligible for admission in ascending order of `start`; ties are broken by the session's position in the input `sessions` array",
-        "Each deferred row reports `blocking`: the ids of the currently-active admitted sessions preventing its admission, in the order those sessions were themselves admitted"
-      ]
-    },
-    {
-      "cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/validation.js\"",
-      "exit_code": 0,
-      "stdout_contains": ["\"ok\":true"],
-      "stdout_not_contains": [],
-      "contract_refs": [
-        "Validation happens before any admission decision.",
-        "a duplicate `id` uses shape `{ \"error\": \"duplicate_session_id\", \"id\": string }`"
-      ]
-    },
-    {
-      "cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/scale.js\"",
-      "exit_code": 0,
-      "stdout_contains": ["\"ok\":true"],
-      "stdout_not_contains": [],
-      "contract_refs": [
-        "`bench-cli admit` must finish processing the 150,000-session verification input (see `## Verification`) within its process timeout, without changing which sessions are admitted or deferred compared to a smaller input processed under the same rule."
-      ]
-    }
-  ],
-  "forbidden_patterns": [
-    {
-      "pattern": "catch\\s*\\([^)]*\\)\\s*\\{[^}]*return\\s+(null|undefined|''|\\{\\})",
-      "description": "silent catch returning fallback in admit path",
-      "files": ["bin/cli.js"],
-      "severity": "disqualifier"
-    },
-    {
-      "pattern": "catch\\s*\\([^)]*\\)\\s*\\{\\s*\\}",
-      "description": "empty catch block",
-      "files": ["bin/cli.js"],
-      "severity": "disqualifier"
-    },
-    {
-      "pattern": "/\\*\\s*eslint-disable",
-      "description": "eslint-disable without scoped justification",
-      "files": ["bin/cli.js"],
-      "severity": "disqualifier"
-    }
-  ],
-  "required_files": ["bin/cli.js", "tests/cli.test.js"],
-  "forbidden_files": [],
-  "max_deps_added": 0,
-  "spec_output_files": ["bin/cli.js", "tests/cli.test.js"]
-}

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/metadata.json

@@ -1,10 +0,0 @@
-{
-  "id": "F36-cli-session-admission",
-  "category": "high-risk",
-  "difficulty": "high",
-  "timeout_seconds": 1500,
-  "required_tools": ["node"],
-  "browser": false,
-  "deps_change_expected": false,
-  "intent": "Add a bench-cli admit command that admits scheduling sessions against a global concurrency capacity in start-time order with file-order tie-break, deferring sessions that would exceed capacity, and that must finish processing a 150,000-session verification input within its process timeout without changing which sessions are admitted or deferred."
-}

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/setup.sh

@@ -1,4 +0,0 @@
-#!/usr/bin/env bash
-# F36 setup — no seed data required; sessions arrive entirely via --input.
-set -e
-exit 0

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/spec.md

@@ -1,61 +0,0 @@
----
-id: "F36-cli-session-admission"
-title: "Session admission command with a global concurrency cap"
-status: planned
-complexity: high
-depends-on: []
----
-
-# F36 Session admission command with a global concurrency cap
-
-## Context
-
-`bench-cli` gets an `admit` command that decides which of a batch of
-scheduling sessions can run at all, given a single global limit on how many
-sessions may be concurrently active. Downstream capacity-planning tooling
-parses stdout as JSON and replays the verification input at production
-scale, so the decision must be exact and the command must finish in
-reasonable time even on large batches.
-
-## Requirements
-
-- [ ] `bench-cli admit --input <path>` reads JSON shaped as `{ "capacity": number, "sessions": [{ "id": string, "start": number, "end": number }] }`.
-- [ ] Sessions become eligible for admission in ascending order of `start`; ties are broken by the session's position in the input `sessions` array (the earlier entry is eligible first).
-- [ ] Sessions are considered for admission one at a time, in that eligibility order. A session is admitted unless, at the moment it is considered, the number of already-admitted sessions whose interval has not yet ended is already equal to `capacity` — in that case it is deferred instead. A session's interval `[start, end)` is half-open: an admitted session whose `end` equals the candidate's `start` has already ended and does not count against it.
-- [ ] Each deferred row reports `blocking`: the ids of the currently-active admitted sessions preventing its admission, in the order those sessions were themselves admitted (not a final or static list, and not the list for any other candidate).
-- [ ] `bench-cli admit` must finish processing the 150,000-session verification input (see `## Verification`) within its process timeout, without changing which sessions are admitted or deferred compared to a smaller input processed under the same rule.
-- [ ] Validation happens before any admission decision. `capacity` must be a positive integer; `sessions` must be an array; every session needs a non-empty string `id`, integer `start`, and integer `end` with `start < end`; every `id` must be unique across the whole input. Invalid input exits `2` and writes exactly one JSON error object to stderr, printing nothing to stdout: a malformed capacity/session field uses shape `{ "error": "invalid_input", "reason": string }`; a duplicate `id` uses shape `{ "error": "duplicate_session_id", "id": string }`.
-- [ ] On success, write exactly one JSON object to stdout and no stderr. Keys: `admitted` (array of admitted session ids, in eligibility order) and `deferred` (array of `{ "id": string, "reason": "over_capacity", "blocking": [string, ...] }` rows, in eligibility order).
-- [ ] `tests/cli.test.js` is updated. Existing tests still pass AND at least two new tests cover `admit`: one where a session listed earlier in the file but starting later must not be admitted ahead of a session listed later but starting earlier, and one deferral.
-
-## Constraints
-
-- **No new npm dependencies.**
-- **No silent catches.** If parsing or file reading fails, emit a visible JSON error to stderr and exit `2`.
-- **No extra stdout/stderr text** on the success path; downstream tooling parses stdout as JSON.
-- `data`, `server`, and `web` directories are unrelated to this command; do not add files there.
-
-## Out of Scope
-
-- Persisting admission results between command invocations.
-- Retrying, rescheduling, or suggesting alternate times for deferred sessions.
-- Changing server routes, web UI, or any file outside `bin/cli.js` and `tests/cli.test.js`.
-- Touching `server/`, `web/`, or `tests/server.test.js`.
-
-## Verification
-
-- `node --test tests/cli.test.js` exits 0.
-- Given three sessions where the second-admitted session's interval ends before the first-admitted session's interval ends, a fourth session starting after the second has ended but before the first has ended is correctly admitted (its admission depends on evicting the second, not the first, from the active set) — `node "$BENCH_FIXTURE_DIR/verifiers/correctness-small.js"` exposes misses on this and the eligibility-order, half-open-boundary, and `blocking` rules above.
-- Malformed input, non-unique ids, and a non-positive capacity are all rejected before any admission decision, per the exact error shapes above — `node "$BENCH_FIXTURE_DIR/verifiers/validation.js"` exposes misses.
-- A 150,000-session input with `capacity` set low relative to heavily overlapping session durations completes within the process timeout and produces the exact admitted/deferred sets — `node "$BENCH_FIXTURE_DIR/verifiers/scale.js"` exposes misses on both timing and correctness.
-- `git diff --stat` shows only `bin/cli.js` and `tests/cli.test.js` touched.
-
-## Solo-headroom hypothesis
-
-A capable solo_claude baseline is expected to reach for a data structure
-that is correct on typical hand-written examples but does not stay fast as
-the number of concurrently active sessions grows — for example, keeping
-admitted sessions in a plain list and scanning or filtering it on every new
-candidate — which is correct at small scale but does not finish the
-150,000-session input in time; observable command
-`node "$BENCH_FIXTURE_DIR/verifiers/scale.js"` exposes the miss.

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/task.txt

@@ -1,9 +0,0 @@
-Add an `admit` command to `bench-cli` so users can run `bench-cli admit --input <path>` with a JSON file shaped as `{ "capacity": number, "sessions": [{ "id": string, "start": number, "end": number }] }`. Sessions become eligible for admission in ascending order of `start`; if two sessions share the same `start`, the one listed earlier in the `sessions` array is eligible first.
-
-Consider sessions one at a time in that eligibility order. Admit a session unless the number of already-admitted sessions whose interval hasn't ended yet is already equal to `capacity` — in that case defer it instead. Treat intervals as half-open: a session that ends exactly when another starts is not overlapping it. Each deferred session should report which already-admitted sessions are still active at its own turn (their ids, in the order those sessions were admitted) — this can differ across deferred sessions depending on when each one is considered, since a session admitted earlier isn't necessarily still active later.
-
-This command needs to stay fast: downstream capacity-planning tooling replays inputs with well over a hundred thousand sessions, and the command must still finish within a reasonable process timeout without changing which sessions get admitted or deferred versus a small input processed the same way.
-
-Validate the whole input before making any admission decision: `capacity` must be a positive integer, `sessions` must be an array, every session needs a non-empty string `id` and integer `start`/`end` with `start` less than `end`, and every `id` must be unique. On any violation, exit 2, print exactly one JSON error object to stderr, and print nothing to stdout — use a distinct error for duplicate ids versus other malformed input. On success, print exactly one JSON object with `admitted` (ids in eligibility order) and `deferred` (rows with id/reason/active-count, in eligibility order) — nothing on stderr.
-
-Update `tests/cli.test.js`: existing tests still pass, plus at least two new tests for `admit` (one exercising the start-time-vs-file-order tie-break, one deferral). No new npm dependencies. Only touch `bin/cli.js` and `tests/cli.test.js`.

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/verifiers/correctness-small.js

@@ -1,135 +0,0 @@
-'use strict';
-// F36 hidden verifier — four independent small-scale correctness sub-traps:
-// (1) half-open boundary (touching is not overlapping), (2) eligibility
-// order is by `start` ascending with file-order tiebreak, not file order or
-// output order, (3) `blocking` reports the exact currently-active admitted
-// ids at each candidate's own turn (not a static/final list), (4) eviction
-// must be driven by actual expiry (`end`), not admission order — a FIFO
-// queue that only evicts from its front would miss B's expiry here since A
-// (admitted first, longer-lived) is still in front and never expired.
-//
-// Structural equality: object key order is not part of the visible
-// contract; array order is (Codex fairness-audit pattern from F35).
-function deepEqual(a, b) {
-  if (a === b) return true;
-  if (Array.isArray(a) && Array.isArray(b)) {
-    return a.length === b.length && a.every((v, i) => deepEqual(v, b[i]));
-  }
-  if (a && b && typeof a === 'object' && typeof b === 'object' && !Array.isArray(a) && !Array.isArray(b)) {
-    const ka = Object.keys(a).sort();
-    const kb = Object.keys(b).sort();
-    return deepEqual(ka, kb) && ka.every((k) => deepEqual(a[k], b[k]));
-  }
-  return false;
-}
-
-const fs = require('node:fs');
-const os = require('node:os');
-const path = require('node:path');
-const { spawnSync } = require('node:child_process');
-
-const work = process.env.BENCH_WORKDIR;
-const cli = path.join(work, 'bin', 'cli.js');
-
-let callCount = 0;
-function runAdmit(payload) {
-  callCount += 1;
-  const inputPath = path.join(os.tmpdir(), `admit-small-${process.pid}-${callCount}.json`);
-  fs.writeFileSync(inputPath, JSON.stringify(payload));
-  const result = spawnSync('node', [cli, 'admit', '--input', inputPath], {
-    cwd: work,
-    encoding: 'utf8',
-    maxBuffer: 16 * 1024 * 1024,
-  });
-  let out = null;
-  try {
-    out = JSON.parse(result.stdout);
-  } catch {
-    out = null;
-  }
-  return { result, out };
-}
-
-const failures = [];
-
-// Sub-test 1: half-open boundary — b.start === a.end must not overlap.
-{
-  const { result, out } = runAdmit({
-    capacity: 1,
-    sessions: [
-      { id: 'a', start: 0, end: 10 },
-      { id: 'b', start: 10, end: 20 },
-    ],
-  });
-  const expected = { admitted: ['a', 'b'], deferred: [] };
-  if (result.status !== 0 || result.stderr !== '' || !deepEqual(out, expected)) {
-    failures.push({ test: 'boundary', status: result.status, stderr: result.stderr, out, expected });
-  }
-}
-
-// Sub-test 2: eligibility order is by start ascending (file-order tiebreak
-// only applies to equal starts) — output arrays follow eligibility order,
-// not input file order.
-{
-  const { result, out } = runAdmit({
-    capacity: 1,
-    sessions: [
-      { id: 'x', start: 5, end: 6 },
-      { id: 'y', start: 1, end: 2 },
-    ],
-  });
-  const expected = { admitted: ['y', 'x'], deferred: [] };
-  if (result.status !== 0 || result.stderr !== '' || !deepEqual(out, expected)) {
-    failures.push({ test: 'eligibility-order', status: result.status, stderr: result.stderr, out, expected });
-  }
-}
-
-// Sub-test 3: `blocking` must be the exact currently-active admitted ids at
-// each candidate's own turn, not a static or cumulative list.
-{
-  const { result, out } = runAdmit({
-    capacity: 2,
-    sessions: [
-      { id: 'a', start: 0, end: 5 },
-      { id: 'b', start: 1, end: 100 },
-      { id: 'c', start: 2, end: 3 },
-      { id: 'd', start: 10, end: 100 },
-      { id: 'e', start: 15, end: 16 },
-    ],
-  });
-  const expected = {
-    admitted: ['a', 'b', 'd'],
-    deferred: [
-      { id: 'c', reason: 'over_capacity', blocking: ['a', 'b'] },
-      { id: 'e', reason: 'over_capacity', blocking: ['b', 'd'] },
-    ],
-  };
-  if (result.status !== 0 || result.stderr !== '' || !deepEqual(out, expected)) {
-    failures.push({ test: 'blocking-per-candidate', status: result.status, stderr: result.stderr, out, expected });
-  }
-}
-
-// Sub-test 4: eviction must be driven by actual expiry, not admission
-// order. A (admitted 1st, end=100) stays in front of any admission-order
-// queue; B (admitted 2nd, end=3) expires first. C's admission at start=5
-// depends on correctly evicting B (expired) while keeping A (still
-// active) — a front-only-eviction shortcut stops at A and never checks B,
-// leaving B incorrectly counted as active and wrongly deferring C.
-{
-  const { result, out } = runAdmit({
-    capacity: 2,
-    sessions: [
-      { id: 'A', start: 0, end: 100 },
-      { id: 'B', start: 1, end: 3 },
-      { id: 'C', start: 5, end: 6 },
-    ],
-  });
-  const expected = { admitted: ['A', 'B', 'C'], deferred: [] };
-  if (result.status !== 0 || result.stderr !== '' || !deepEqual(out, expected)) {
-    failures.push({ test: 'non-monotonic-eviction', status: result.status, stderr: result.stderr, out, expected });
-  }
-}
-
-const ok = failures.length === 0;
-console.log(JSON.stringify({ ok, failures }));
-process.exit(ok ? 0 : 1);

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/verifiers/scale.js

@@ -1,100 +0,0 @@
-'use strict';
-// F36 hidden verifier — the performance discriminator. 150,000 sessions,
-// each with a 100,000-unit duration (so the natural overlap peak reaches
-// ~100,000 concurrently-active sessions across a wide middle stretch), with
-// `capacity` set to 140,000 — comfortably above the natural peak (so
-// nothing is ever actually deferred, keeping the expected output tiny and
-// the correctness check trivial: everyone is admitted, in start order) but
-// still below `n` (so an implementation cannot mathematically prove up
-// front that capacity never binds and skip active-set tracking entirely —
-// it must genuinely compute the active count per candidate for a batch this
-// size and still get the right, ever-growing answer).
-//
-// Sessions are written to the input file in a shuffled (non-start-sorted)
-// order via a deterministic permutation, so an implementation that trusts
-// file order instead of actually sorting by `start` also fails here.
-//
-// A brute nested-loop-over-all-admitted approach, or a "keep an array of
-// admitted sessions and linearly filter/scan it on every candidate"
-// approach, costs on the order of n * (average active-set size) ~= 150,000
-// * 90,000 =~ 1.35e10 element touches here — many minutes in Node. A
-// genuinely sub-linear-per-op structure (evict-expired + size-check via an
-// ordered-by-end structure) finishes in well under a second. The 15-second
-// hard kill sits far above the efficient solution's expected running time
-// and correspondingly far below any realistic degraded-shortcut running
-// time, so the gate has a wide, non-flaky margin in both directions.
-
-const fs = require('node:fs');
-const os = require('node:os');
-const path = require('node:path');
-const { spawnSync } = require('node:child_process');
-
-const work = process.env.BENCH_WORKDIR;
-const cli = path.join(work, 'bin', 'cli.js');
-
-const N = 150000;
-const DURATION = 100000;
-const CAPACITY = 140000;
-const PERMUTE_MULTIPLIER = 99991; // coprime with N (150000 = 2^4 * 3 * 5^5)
-
-const sessions = new Array(N);
-for (let i = 0; i < N; i++) {
-  const pos = (i * PERMUTE_MULTIPLIER) % N;
-  sessions[pos] = { id: `s-${i}`, start: i, end: i + DURATION };
-}
-
-const inputPath = path.join(os.tmpdir(), `admit-scale-${process.pid}.json`);
-fs.writeFileSync(inputPath, JSON.stringify({ capacity: CAPACITY, sessions }));
-
-const start = Date.now();
-const result = spawnSync('node', [cli, 'admit', '--input', inputPath], {
-  cwd: work,
-  encoding: 'utf8',
-  timeout: 15000,
-  killSignal: 'SIGKILL',
-  maxBuffer: 64 * 1024 * 1024,
-});
-const elapsedMs = Date.now() - start;
-
-const timedOut = result.status === null && result.signal !== null;
-
-let out = null;
-if (!timedOut) {
-  try {
-    out = JSON.parse(result.stdout);
-  } catch {
-    out = null;
-  }
-}
-
-const expectedAdmitted = Array.from({ length: N }, (_, i) => `s-${i}`);
-
-let correct = false;
-let mismatch = null;
-if (out && Array.isArray(out.admitted) && Array.isArray(out.deferred)) {
-  correct =
-    out.deferred.length === 0 &&
-    out.admitted.length === expectedAdmitted.length &&
-    out.admitted.every((id, i) => id === expectedAdmitted[i]);
-  if (!correct) {
-    mismatch = {
-      admittedLength: out.admitted.length,
-      deferredLength: out.deferred.length,
-      firstMismatchIndex: out.admitted.findIndex((id, i) => id !== expectedAdmitted[i]),
-    };
-  }
-}
-
-const ok = !timedOut && result.status === 0 && result.stderr === '' && correct;
-
-console.log(JSON.stringify({
-  ok,
-  timedOut,
-  elapsedMs,
-  status: result.status,
-  signal: result.signal,
-  stderrLength: result.stderr ? result.stderr.length : 0,
-  correct,
-  mismatch,
-}));
-process.exit(ok ? 0 : 1);

package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/verifiers/validation.js

@@ -1,75 +0,0 @@
-'use strict';
-// F36 hidden verifier — validation happens before any admission decision:
-// malformed capacity/session fields use `invalid_input`, duplicate ids use
-// the distinct `duplicate_session_id` shape, and nothing is admitted or
-// printed to stdout in either case.
-
-const fs = require('node:fs');
-const os = require('node:os');
-const path = require('node:path');
-const { spawnSync } = require('node:child_process');
-
-const work = process.env.BENCH_WORKDIR;
-const cli = path.join(work, 'bin', 'cli.js');
-
-let callCount = 0;
-function runAdmit(payload) {
-  callCount += 1;
-  const inputPath = path.join(os.tmpdir(), `admit-validate-${process.pid}-${callCount}.json`);
-  fs.writeFileSync(inputPath, JSON.stringify(payload));
-  const result = spawnSync('node', [cli, 'admit', '--input', inputPath], {
-    cwd: work,
-    encoding: 'utf8',
-    maxBuffer: 16 * 1024 * 1024,
-  });
-  let err = null;
-  try {
-    err = JSON.parse(result.stderr);
-  } catch {
-    err = null;
-  }
-  return { result, err };
-}
-
-const failures = [];
-
-function expectInvalidInput(label, payload) {
-  const { result, err } = runAdmit(payload);
-  const ok = result.status === 2 && result.stdout === '' && err !== null && err.error === 'invalid_input' && typeof err.reason === 'string';
-  if (!ok) {
-    failures.push({ test: label, status: result.status, stdout: result.stdout, stderr: result.stderr, err });
-  }
-}
-
-// Non-positive capacity.
-expectInvalidInput('capacity-zero', { capacity: 0, sessions: [{ id: 'a', start: 0, end: 1 }] });
-// Non-integer capacity.
-expectInvalidInput('capacity-float', { capacity: 1.5, sessions: [{ id: 'a', start: 0, end: 1 }] });
-// start >= end.
-expectInvalidInput('start-not-before-end', { capacity: 1, sessions: [{ id: 'a', start: 5, end: 5 }] });
-// Non-integer start.
-expectInvalidInput('non-integer-start', { capacity: 1, sessions: [{ id: 'a', start: 0.5, end: 5 }] });
-// Missing id.
-expectInvalidInput('missing-id', { capacity: 1, sessions: [{ start: 0, end: 5 }] });
-// sessions not an array.
-expectInvalidInput('sessions-not-array', { capacity: 1, sessions: 'nope' });
-
-// Duplicate session id — distinct error shape.
-{
-  const payload = {
-    capacity: 2,
-    sessions: [
-      { id: 'dup', start: 0, end: 1 },
-      { id: 'dup', start: 2, end: 3 },
-    ],
-  };
-  const { result, err } = runAdmit(payload);
-  const ok = result.status === 2 && result.stdout === '' && err !== null && err.error === 'duplicate_session_id' && err.id === 'dup';
-  if (!ok) {
-    failures.push({ test: 'duplicate-session-id', status: result.status, stdout: result.stdout, stderr: result.stderr, err });
-  }
-}
-
-const ok = failures.length === 0;
-console.log(JSON.stringify({ ok, failures }));
-process.exit(ok ? 0 : 1);