devlyn-cli 2.6.1 → 2.8.0

package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/setup.sh

@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+# F37 setup — seed a small, deliberately unsorted rule-revisions file.
+set -e
+
+mkdir -p data
+
+cat > data/rule-revisions.json <<'JSON'
+[
+  { "id": "rev-202", "categoryId": "cat-2", "effectiveAt": 2500, "discountPct": 25, "minPrice": 45 },
+  { "id": "rev-101", "categoryId": "cat-1", "effectiveAt": 1000, "discountPct": 10, "minPrice": 50 },
+  { "id": "rev-301", "categoryId": "cat-3", "effectiveAt": 500, "discountPct": 0, "minPrice": 100 },
+  { "id": "rev-103", "categoryId": "cat-1", "effectiveAt": 3000, "discountPct": 15, "minPrice": 55 },
+  { "id": "rev-201", "categoryId": "cat-2", "effectiveAt": 1500, "discountPct": 5, "minPrice": 40 },
+  { "id": "rev-102", "categoryId": "cat-1", "effectiveAt": 2000, "discountPct": 20, "minPrice": 60 }
+]
+JSON
+
+exit 0

package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/spec.md

@@ -0,0 +1,66 @@
+---
+id: "F37-cli-rule-lookup"
+title: "Price events against the rule in effect at their own timestamp"
+status: planned
+complexity: high
+depends-on: []
+---
+
+# F37 Price events against the rule in effect at their own timestamp
+
+## Context
+
+`bench-cli` gets a `price-events` command that prices a batch of purchase
+events using the pricing rule revision that was actually in effect for that
+event's category at the event's own timestamp — not the newest rule, and
+not the same rule for every event in a category. Rule revisions arrive from
+a seeded, unsorted file with many revisions per category. Downstream
+billing tools parse stdout as JSON, and they replay very large event
+batches, so the lookup must stay exact and the command must finish in
+reasonable time.
+
+## Requirements
+
+- [ ] `bench-cli price-events --input <path>` reads JSON shaped as `{ "events": [{ "id": string, "categoryId": string, "timestamp": number, "basePrice": number }] }`.
+- [ ] Pricing rule revisions come from `data/rule-revisions.json`, shaped as an array of `{ "id": string, "categoryId": string, "effectiveAt": number, "discountPct": number, "minPrice": number }`. Revisions in that file are not sorted by category or by time. The command must not modify `data/rule-revisions.json`.
+- [ ] For each event, consider every revision whose `categoryId` equals the event's `categoryId` and whose `effectiveAt` is less than or equal to the event's `timestamp`. Among those, the one with the greatest `effectiveAt` applies — an exact match (`effectiveAt` equal to the event's `timestamp`) counts as eligible, not just strictly earlier ones.
+- [ ] If more than one eligible revision for that category shares the greatest `effectiveAt`, the one with the greatest `id` (plain string comparison) applies.
+- [ ] If the event's `categoryId` has zero revisions anywhere in `data/rule-revisions.json`, the event is rejected with reason `unknown_category`.
+- [ ] If the event's `categoryId` has at least one revision but none with `effectiveAt` less than or equal to the event's `timestamp`, the event is rejected with reason `no_effective_rule`.
+- [ ] An accepted event's price is `Math.max(rule.minPrice, Math.round(event.basePrice * (100 - rule.discountPct) / 100))`.
+- [ ] `bench-cli price-events` must finish processing the 200,000-event verification input (see `## Verification`) against the seeded revision set within its process timeout, without changing the priced or rejected outcome for any event compared to a smaller input processed the same way.
+- [ ] Validation happens before any pricing. `events` must be an array; every event needs a non-empty string `id` and `categoryId`, an integer `timestamp`, and a non-negative integer `basePrice`; every event `id` must be unique. Invalid input exits `2` and writes exactly one JSON error object to stderr, printing nothing to stdout: a malformed field uses shape `{ "error": "invalid_input", "reason": string }`; a duplicate event `id` uses shape `{ "error": "duplicate_event_id", "id": string }`.
+- [ ] On success, write exactly one JSON object to stdout and no stderr. Keys: `priced` (array of `{ "id": string, "ruleId": string, "price": number }` rows) and `rejected` (array of `{ "id": string, "reason": string }` rows). Each event appears in exactly one of the two arrays; within each array, rows keep the original input event order.
+- [ ] `tests/cli.test.js` is updated. Existing tests still pass AND at least two new tests cover `price-events`: one where two revisions for the same category apply to different events depending on each event's own timestamp, and one rejection.
+
+## Constraints
+
+- **No new npm dependencies.**
+- **No silent catches.** If parsing or file reading fails, emit a visible JSON error to stderr and exit `2`.
+- **No extra stdout/stderr text** on the success path; downstream tooling parses stdout as JSON.
+- `server` and `web` directories are unrelated to this command; do not add files there.
+
+## Out of Scope
+
+- Persisting priced results between command invocations.
+- Adding, editing, or reordering rule revisions in `data/rule-revisions.json`.
+- Changing server routes, web UI, or any file outside `bin/cli.js` and `tests/cli.test.js`.
+- Touching `server/`, `web/`, or `tests/server.test.js`.
+
+## Verification
+
+- `node --test tests/cli.test.js` exits 0.
+- Two revisions for the same category with different `effectiveAt` values each apply to the correct event depending on that event's own `timestamp`, including the exact-match boundary — `node "$BENCH_FIXTURE_DIR/verifiers/correctness-small.js"` exposes misses on this, the same-`effectiveAt` tie-break, and the `unknown_category`-versus-`no_effective_rule` distinction above.
+- Malformed input, non-unique event ids, and non-integer/negative fields are all rejected before any pricing decision, per the exact error shapes above — `node "$BENCH_FIXTURE_DIR/verifiers/validation.js"` exposes misses.
+- A 200,000-event input priced against a large seeded revision set completes within the process timeout and produces the exact priced/rejected outcome for every event — `node "$BENCH_FIXTURE_DIR/verifiers/scale.js"` exposes misses on both timing and correctness.
+- `git diff --stat` shows only `bin/cli.js` and `tests/cli.test.js` touched.
+
+## Solo-headroom hypothesis
+
+A capable solo_claude baseline is expected to reach for a lookup approach
+that is correct on typical hand-written examples but does not stay fast as
+the number of revisions per category grows — for example, scanning a
+category's revisions linearly for every event instead of using a structure
+that stays fast as that list grows — which is correct at small scale but
+does not finish the 200,000-event input in time; observable command
+`node "$BENCH_FIXTURE_DIR/verifiers/scale.js"` exposes the miss.

package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/task.txt

@@ -0,0 +1,9 @@
+Add a `price-events` command to `bench-cli` so users can run `bench-cli price-events --input <path>` with a JSON file shaped as `{ "events": [{ "id": string, "categoryId": string, "timestamp": number, "basePrice": number }] }`. Pricing rules come from the seeded `data/rule-revisions.json` (an array of `{ "id", "categoryId", "effectiveAt", "discountPct", "minPrice" }` rows, not sorted by category or by time, and not to be modified by the command) — there can be many revisions for the same category over time, and each event must be priced using whichever revision was actually in effect for its category at its own timestamp, not just the newest one.
+
+For each event, find the revision for its category with the greatest `effectiveAt` that is less than or equal to the event's own `timestamp` (an exact match counts). If more than one revision ties on that `effectiveAt`, use the one with the greater `id`. If the category has no revisions at all, reject the event with reason `unknown_category`; if it has revisions but none apply yet at that timestamp, reject with reason `no_effective_rule`. An accepted event's price is the base price discounted by the rule's percentage and rounded, but never below the rule's minimum price.
+
+This command needs to stay fast: downstream billing tooling replays event batches well over a hundred thousand rows against revision sets with many entries per category, and it must still finish within a reasonable process timeout without changing any event's priced or rejected outcome versus a small input processed the same way.
+
+Validate the whole input before pricing anything: `events` must be an array, every event needs a non-empty string `id`/`categoryId`, an integer `timestamp`, and a non-negative integer `basePrice`, and every `id` must be unique. On any violation, exit 2, print exactly one JSON error object to stderr, and print nothing to stdout — use a distinct error for duplicate ids versus other malformed input. On success, print exactly one JSON object with `priced` (id/ruleId/price rows) and `rejected` (id/reason rows), each array keeping the original event order — nothing on stderr.
+
+Update `tests/cli.test.js`: existing tests still pass, plus at least two new tests for `price-events` (one exercising two revisions applying to different events by timestamp, one rejection). No new npm dependencies. Only touch `bin/cli.js` and `tests/cli.test.js`.

package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/verifiers/correctness-small.js

@@ -0,0 +1,84 @@
+'use strict';
+// F37 hidden verifier — one bundled scenario exercising: (1) different
+// events in the same category resolving to different revisions by their
+// own timestamp, (2) the inclusive effectiveAt<=timestamp boundary (an
+// exact match is eligible), (3) the same-effectiveAt tie-break (greatest
+// id wins), (4) unknown_category vs no_effective_rule are distinct and
+// correctly assigned. Resets the seed so this verifier is order-independent
+// from other verifier commands (F35 pattern).
+//
+// Structural equality: object key order is not part of the visible
+// contract; array order is.
+function deepEqual(a, b) {
+  if (a === b) return true;
+  if (Array.isArray(a) && Array.isArray(b)) {
+    return a.length === b.length && a.every((v, i) => deepEqual(v, b[i]));
+  }
+  if (a && b && typeof a === 'object' && typeof b === 'object' && !Array.isArray(a) && !Array.isArray(b)) {
+    const ka = Object.keys(a).sort();
+    const kb = Object.keys(b).sort();
+    return deepEqual(ka, kb) && ka.every((k) => deepEqual(a[k], b[k]));
+  }
+  return false;
+}
+
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const { spawnSync } = require('node:child_process');
+
+const work = process.env.BENCH_WORKDIR;
+const revPath = path.join(work, 'data', 'rule-revisions.json');
+const cli = path.join(work, 'bin', 'cli.js');
+
+const revisions = [
+  { id: 'rev-a2', categoryId: 'cat-A', effectiveAt: 200, discountPct: 20, minPrice: 10 },
+  { id: 'rev-a9', categoryId: 'cat-A', effectiveAt: 300, discountPct: 90, minPrice: 10 },
+  { id: 'rev-a1', categoryId: 'cat-A', effectiveAt: 100, discountPct: 10, minPrice: 10 },
+  { id: 'rev-b1', categoryId: 'cat-B', effectiveAt: 50, discountPct: 0, minPrice: 5 },
+  { id: 'rev-a3', categoryId: 'cat-A', effectiveAt: 300, discountPct: 30, minPrice: 10 },
+];
+fs.writeFileSync(revPath, JSON.stringify(revisions, null, 2) + '\n');
+
+const inputPath = path.join(os.tmpdir(), `price-events-small-${process.pid}.json`);
+fs.writeFileSync(inputPath, JSON.stringify({
+  events: [
+    { id: 'e1', categoryId: 'cat-A', timestamp: 150, basePrice: 100 },
+    { id: 'e2', categoryId: 'cat-A', timestamp: 250, basePrice: 100 },
+    { id: 'e3', categoryId: 'cat-A', timestamp: 200, basePrice: 50 },
+    { id: 'e4', categoryId: 'cat-A', timestamp: 350, basePrice: 100 },
+    { id: 'e5', categoryId: 'cat-B', timestamp: 10, basePrice: 100 },
+    { id: 'e6', categoryId: 'cat-C', timestamp: 1000, basePrice: 100 },
+  ],
+}));
+
+const result = spawnSync('node', [cli, 'price-events', '--input', inputPath], {
+  cwd: work,
+  encoding: 'utf8',
+  maxBuffer: 16 * 1024 * 1024,
+});
+
+let out = null;
+try {
+  out = JSON.parse(result.stdout);
+} catch {
+  out = null;
+}
+
+const expected = {
+  priced: [
+    { id: 'e1', ruleId: 'rev-a1', price: 90 },
+    { id: 'e2', ruleId: 'rev-a2', price: 80 },
+    { id: 'e3', ruleId: 'rev-a2', price: 40 },
+    { id: 'e4', ruleId: 'rev-a9', price: 10 },
+  ],
+  rejected: [
+    { id: 'e5', reason: 'no_effective_rule' },
+    { id: 'e6', reason: 'unknown_category' },
+  ],
+};
+
+const ok = result.status === 0 && result.stderr === '' && deepEqual(out, expected);
+
+console.log(JSON.stringify({ ok, status: result.status, stderr: result.stderr, out, expected }));
+process.exit(ok ? 0 : 1);

package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/verifiers/scale.js

@@ -0,0 +1,184 @@
+'use strict';
+// F37 hidden verifier — the performance discriminator. 60,000 rule
+// revisions spread over 4 categories (15,000 revisions/category) and
+// 200,000 events. Every event resolves successfully (no rejections),
+// which keeps the expected output small and the correctness check a
+// direct array comparison; the discriminator is entirely in how the
+// lookup is done, not in exercising the reject paths (those are covered
+// by correctness-small.js).
+//
+// A per-event linear scan of a category's revisions costs on the order of
+// events * revisions-per-category =~ 200,000 * 15,000 = 3e9 comparisons —
+// tens of seconds or more in Node. Grouping by category (near-free) and
+// then binary-searching a sorted per-category list is a few million
+// log-steps — trivially fast. The reference below independently
+// implements the efficient shape (not shared code with the CLI) so it can
+// also serve as ground truth without becoming a second bottleneck. The
+// 15-second hard kill sits far above the efficient solution's expected
+// running time and correspondingly far below any realistic
+// linear-scan running time at these parameters.
+//
+// Revisions are written to `data/rule-revisions.json` in a shuffled
+// (non-category-grouped, non-time-sorted) order via a deterministic
+// permutation, matching the visible "not sorted by category or by time"
+// spec fact and closing a "trust file order" shortcut.
+
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const { spawnSync } = require('node:child_process');
+
+const work = process.env.BENCH_WORKDIR;
+const revPath = path.join(work, 'data', 'rule-revisions.json');
+const cli = path.join(work, 'bin', 'cli.js');
+
+const CATS = 4;
+const REVS_PER_CAT = 15000;
+const TOTAL_REVS = CATS * REVS_PER_CAT;
+const EVENTS = 200000;
+const REV_PERMUTE_MULTIPLIER = 39989; // coprime with 60000 (2^5 * 3 * 5^4)
+
+const revisionsInOrder = new Array(TOTAL_REVS);
+let k = 0;
+for (let c = 0; c < CATS; c++) {
+  for (let j = 0; j < REVS_PER_CAT; j++) {
+    revisionsInOrder[k] = {
+      id: `rev-${c}-${j}`,
+      categoryId: `cat-${c}`,
+      effectiveAt: j * 10,
+      discountPct: (j * 7) % 50,
+      minPrice: 100 + ((j * 13) % 900),
+    };
+    k += 1;
+  }
+}
+
+const shuffled = new Array(TOTAL_REVS);
+for (let i = 0; i < TOTAL_REVS; i++) {
+  const pos = (i * REV_PERMUTE_MULTIPLIER) % TOTAL_REVS;
+  shuffled[pos] = revisionsInOrder[i];
+}
+fs.writeFileSync(revPath, JSON.stringify(shuffled));
+
+const events = new Array(EVENTS);
+for (let i = 0; i < EVENTS; i++) {
+  const c = i % CATS;
+  const base = (i * 131) % REVS_PER_CAT;
+  events[i] = {
+    id: `ev-${i}`,
+    categoryId: `cat-${c}`,
+    timestamp: base * 10 + 5,
+    basePrice: 1000 + (i % 500),
+  };
+}
+
+const inputPath = path.join(os.tmpdir(), `price-events-scale-${process.pid}.json`);
+fs.writeFileSync(inputPath, JSON.stringify({ events }));
+
+// Independent reference: group by category, sort each group by
+// (effectiveAt asc, id asc), binary-search the rightmost entry with
+// effectiveAt <= timestamp.
+const groups = new Map();
+for (const rev of revisionsInOrder) {
+  if (!groups.has(rev.categoryId)) groups.set(rev.categoryId, []);
+  groups.get(rev.categoryId).push(rev);
+}
+for (const list of groups.values()) {
+  list.sort((a, b) => a.effectiveAt - b.effectiveAt || (a.id < b.id ? -1 : a.id > b.id ? 1 : 0));
+}
+
+function findRuleIndex(sorted, timestamp) {
+  let lo = 0;
+  let hi = sorted.length - 1;
+  let ans = -1;
+  while (lo <= hi) {
+    const mid = (lo + hi) >> 1;
+    if (sorted[mid].effectiveAt <= timestamp) {
+      ans = mid;
+      lo = mid + 1;
+    } else {
+      hi = mid - 1;
+    }
+  }
+  return ans;
+}
+
+const expectedPriced = new Array(EVENTS);
+let expectedRejectedCount = 0;
+for (let i = 0; i < EVENTS; i++) {
+  const ev = events[i];
+  const group = groups.get(ev.categoryId);
+  if (!group) {
+    expectedRejectedCount += 1;
+    expectedPriced[i] = null;
+    continue;
+  }
+  const idx = findRuleIndex(group, ev.timestamp);
+  if (idx < 0) {
+    expectedRejectedCount += 1;
+    expectedPriced[i] = null;
+    continue;
+  }
+  const rule = group[idx];
+  const price = Math.max(rule.minPrice, Math.round((ev.basePrice * (100 - rule.discountPct)) / 100));
+  expectedPriced[i] = { id: ev.id, ruleId: rule.id, price };
+}
+
+const start = Date.now();
+const result = spawnSync('node', [cli, 'price-events', '--input', inputPath], {
+  cwd: work,
+  encoding: 'utf8',
+  timeout: 15000,
+  killSignal: 'SIGKILL',
+  maxBuffer: 64 * 1024 * 1024,
+});
+const elapsedMs = Date.now() - start;
+
+const timedOut = result.status === null && result.signal !== null;
+
+let out = null;
+if (!timedOut) {
+  try {
+    out = JSON.parse(result.stdout);
+  } catch {
+    out = null;
+  }
+}
+
+let correct = false;
+let mismatch = null;
+if (out && Array.isArray(out.priced) && Array.isArray(out.rejected)) {
+  correct =
+    out.rejected.length === expectedRejectedCount &&
+    out.priced.length === EVENTS - expectedRejectedCount &&
+    out.priced.every((row, i) => {
+      const expected = expectedPriced[i];
+      return expected && row.id === expected.id && row.ruleId === expected.ruleId && row.price === expected.price;
+    });
+  if (!correct) {
+    const firstMismatchIndex = out.priced.findIndex((row, i) => {
+      const expected = expectedPriced[i];
+      return !expected || row.id !== expected.id || row.ruleId !== expected.ruleId || row.price !== expected.price;
+    });
+    mismatch = {
+      pricedLength: out.priced.length,
+      rejectedLength: out.rejected.length,
+      expectedRejectedCount,
+      firstMismatchIndex,
+    };
+  }
+}
+
+const ok = !timedOut && result.status === 0 && result.stderr === '' && correct;
+
+console.log(JSON.stringify({
+  ok,
+  timedOut,
+  elapsedMs,
+  status: result.status,
+  signal: result.signal,
+  stderrLength: result.stderr ? result.stderr.length : 0,
+  correct,
+  mismatch,
+}));
+process.exit(ok ? 0 : 1);

package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/verifiers/validation.js

@@ -0,0 +1,74 @@
+'use strict';
+// F37 hidden verifier — validation happens before any pricing decision:
+// malformed event fields use `invalid_input`, duplicate event ids use the
+// distinct `duplicate_event_id` shape, and nothing is priced or printed to
+// stdout in either case. Resets the seed so this verifier is
+// order-independent from other verifier commands.
+
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const { spawnSync } = require('node:child_process');
+
+const work = process.env.BENCH_WORKDIR;
+const revPath = path.join(work, 'data', 'rule-revisions.json');
+const cli = path.join(work, 'bin', 'cli.js');
+
+const revisions = [
+  { id: 'rev-a1', categoryId: 'cat-A', effectiveAt: 100, discountPct: 10, minPrice: 10 },
+];
+
+let callCount = 0;
+function runPriceEvents(payload) {
+  fs.writeFileSync(revPath, JSON.stringify(revisions, null, 2) + '\n');
+  callCount += 1;
+  const inputPath = path.join(os.tmpdir(), `price-events-validate-${process.pid}-${callCount}.json`);
+  fs.writeFileSync(inputPath, JSON.stringify(payload));
+  const result = spawnSync('node', [cli, 'price-events', '--input', inputPath], {
+    cwd: work,
+    encoding: 'utf8',
+    maxBuffer: 16 * 1024 * 1024,
+  });
+  let err = null;
+  try {
+    err = JSON.parse(result.stderr);
+  } catch {
+    err = null;
+  }
+  return { result, err };
+}
+
+const failures = [];
+
+function expectInvalidInput(label, payload) {
+  const { result, err } = runPriceEvents(payload);
+  const ok = result.status === 2 && result.stdout === '' && err !== null && err.error === 'invalid_input' && typeof err.reason === 'string';
+  if (!ok) {
+    failures.push({ test: label, status: result.status, stdout: result.stdout, stderr: result.stderr, err });
+  }
+}
+
+expectInvalidInput('events-not-array', { events: 'nope' });
+expectInvalidInput('missing-id', { events: [{ categoryId: 'cat-A', timestamp: 100, basePrice: 10 }] });
+expectInvalidInput('non-integer-timestamp', { events: [{ id: 'e1', categoryId: 'cat-A', timestamp: 1.5, basePrice: 10 }] });
+expectInvalidInput('negative-base-price', { events: [{ id: 'e1', categoryId: 'cat-A', timestamp: 100, basePrice: -1 }] });
+expectInvalidInput('non-string-category', { events: [{ id: 'e1', categoryId: 42, timestamp: 100, basePrice: 10 }] });
+
+// Duplicate event id — distinct error shape.
+{
+  const payload = {
+    events: [
+      { id: 'dup', categoryId: 'cat-A', timestamp: 100, basePrice: 10 },
+      { id: 'dup', categoryId: 'cat-A', timestamp: 200, basePrice: 20 },
+    ],
+  };
+  const { result, err } = runPriceEvents(payload);
+  const ok = result.status === 2 && result.stdout === '' && err !== null && err.error === 'duplicate_event_id' && err.id === 'dup';
+  if (!ok) {
+    failures.push({ test: 'duplicate-event-id', status: result.status, stdout: result.stdout, stderr: result.stderr, err });
+  }
+}
+
+const ok = failures.length === 0;
+console.log(JSON.stringify({ ok, failures }));
+process.exit(ok ? 0 : 1);

package/benchmark/auto-resolve/scripts/pair-rejected-fixtures.sh

@@ -69,6 +69,18 @@ rejected_pair_fixture_reason() {
     F32-*|F32)
       echo "bare 33 / solo_claude 98 in 20260512-f32-subscription-renewal-headroom"
       ;;
+    F34-*|F34)
+      echo "bare 92 / solo_claude 88 in iter-0039-headroom-f34-f35"
+      ;;
+    F35-*|F35)
+      echo "bare 50 / solo_claude 97 in iter-0039-headroom-f34-f35"
+      ;;
+    F36-*|F36)
+      echo "bare 50 / solo_claude 96 with solo timeout in iter-0041-headroom-f36-f37"
+      ;;
+    F37-*|F37)
+      echo "bare 75 / solo_claude 96 with solo invoke failure in iter-0041-headroom-f36-f37"
+      ;;
     S2-*|S2)
       echo "bare 33 / solo_claude 99 with solo timeout in 20260513-s2-inventory-headroom"
       ;;

package/benchmark/auto-resolve/scripts/run-fixture.sh

@@ -15,7 +15,7 @@
 set -euo pipefail
 
 usage() {
-  echo "usage: $0 --fixture <FID> --arm <variant|solo_claude|bare|l2_gated|l2_risk_probes|l2_forced> --run-id <ID> [--resolve-skill new] [--dry-run]"
+  echo "usage: $0 --fixture <FID> --arm <variant|solo_claude|bare|l2_gated|l2_risk_probes|l2_forced> --run-id <ID> [--resolve-skill new] [--engines-config '<json>'] [--dry-run]"
   exit 1
 }
 
@@ -47,17 +47,36 @@ kill_worktree_processes() {
 
 FIXTURE=""; ARM=""; RUN_ID=""; DRY_RUN=0
 RESOLVE_SKILL="new"
+ENGINES_CONFIG=""
 while [ $# -gt 0 ]; do
   case "$1" in
     --fixture)        require_value "$1" "${2:-}"; FIXTURE="$2"; shift 2;;
     --arm)            require_value "$1" "${2:-}"; ARM="$2";     shift 2;;
     --run-id)         require_value "$1" "${2:-}"; RUN_ID="$2";  shift 2;;
     --resolve-skill)  require_value "$1" "${2:-}"; RESOLVE_SKILL="$2"; shift 2;;
+    --engines-config) require_value "$1" "${2:-}"; ENGINES_CONFIG="$2"; shift 2;;
     --dry-run)        DRY_RUN=1;    shift;;
     *) usage;;
   esac
 done
 [ -n "$FIXTURE" ] && [ -n "$ARM" ] && [ -n "$RUN_ID" ] || usage
+
+# iter-0038: --engines-config stages a machine-config role pin
+# (.devlyn/engines.json) into the arm worktree so engine COMBINATIONS are
+# runnable as variant-class arms without new hardcoded arm names. Existing
+# arm launch commands are untouched when the flag is absent. Fail fast on
+# malformed JSON, and refuse bare (no skills — the config would be dead
+# weight that silently mis-labels the arm as configured).
+if [ -n "$ENGINES_CONFIG" ]; then
+  if [ "$ARM" = "bare" ]; then
+    echo "--engines-config is invalid for the bare arm (no skills consume it)" >&2
+    exit 1
+  fi
+  if ! printf '%s' "$ENGINES_CONFIG" | python3 -c 'import json,sys; json.load(sys.stdin)' 2>/dev/null; then
+    echo "--engines-config is not valid JSON" >&2
+    exit 1
+  fi
+fi
 # iter-0019/0037: 3 smoke arms — variant (L2: Claude orchestrator + risk-probes pair path),
 # solo_claude (L1: Claude orchestrator, codex blocked by shim+wrapper enforcement),
 # bare (L0: direct claude -p, no skill, no codex).
@@ -164,6 +183,14 @@ if [ "$ARM" = "variant" ] || [ "$ARM" = "solo_claude" ] \
   if [ -f "$REPO_ROOT/CLAUDE.md" ]; then
     cp "$REPO_ROOT/CLAUDE.md" "$WORK_DIR/CLAUDE.md"
   fi
+  # iter-0038: stage the engine role pin (validated above) pre-baseline.
+  # test-repo/.gitignore covers .devlyn, so it never enters scaffold commits
+  # or arm diffs. A verbatim copy lands in RESULT_DIR as run evidence.
+  if [ -n "$ENGINES_CONFIG" ]; then
+    mkdir -p "$WORK_DIR/.devlyn"
+    printf '%s\n' "$ENGINES_CONFIG" > "$WORK_DIR/.devlyn/engines.json"
+    printf '%s\n' "$ENGINES_CONFIG" > "$RESULT_DIR/engines-config.json"
+  fi
   # Stage the codex PATH shim. Required for both variant (route to monitored
   # wrapper) and solo_claude (CODEX_BLOCKED enforcement at PATH layer).
   if command -v codex >/dev/null 2>&1; then
@@ -405,6 +432,17 @@ EOF
     # OLD branch and this is now the only non-F9 path.
     mkdir -p "$WORK_DIR/docs/roadmap/phase-1"
     cp "$SPEC" "$WORK_DIR/docs/roadmap/phase-1/$FIXTURE.md"
+    # iter-0049: fixtures keep spec.md prose-only (expected.json is the real
+    # carrier, validated separately by lint-fixtures.sh) but spec-verify-check.py
+    # now locates the visible ## Verification prose via a <!-- devlyn:verification -->
+    # sentinel rather than the English header text. Inject it into the WORK_DIR
+    # copy only, so risk-probe derived_from substring matching still finds the
+    # section during a run; the source fixture repo stays untouched.
+    if ! grep -q '<!-- devlyn:verification -->' "$WORK_DIR/docs/roadmap/phase-1/$FIXTURE.md"; then
+      sed -i.bak 's/^## Verification\b/<!-- devlyn:verification -->\n## Verification/' \
+        "$WORK_DIR/docs/roadmap/phase-1/$FIXTURE.md"
+      rm -f "$WORK_DIR/docs/roadmap/phase-1/$FIXTURE.md.bak"
+    fi
     cat > "$PROMPT_FILE" <<EOF
 Use the \`/devlyn:resolve --spec docs/roadmap/phase-1/$FIXTURE.md ${ENGINE_CLAUSE}\` skill to implement the spec. ${ENGINE_PROMPT_HINT}
 

package/benchmark/auto-resolve/scripts/test-lint-fixtures.sh

@@ -387,7 +387,7 @@ status=$?
 set -e
 [ "$status" -ne 0 ]
 grep -Fq 'spec-verify-check --check-expected failed' "$TMP/spec-verify-check-expected-fail.out"
-grep -Fq 'unless sibling spec.md declares all Requirements are pure-design' "$TMP/spec-verify-check-expected-fail.out"
+grep -Fq 'unless spec.expected.json declares "pure_design": true' "$TMP/spec-verify-check-expected-fail.out"
 
 write_fixture "Add idempotency handling for duplicate requests."