promptwheel 0.3.0 → 0.4.1

package/README.md

@@ -27,7 +27,7 @@ PromptWheel  a1b2c3d → e4f5g6h  (×5)
   VERDICT: PASS
 ```
 
-Exit `0` on pass, `1` on fail (CI-friendly). No build step, zero dependencies, Node 18+.
+Exit `0` pass · `1` fail · `2` gamed · `3` inconclusive (a guard measured nothing) — CI-friendly. No build step, zero dependencies, Node 18+.
 
 ## Catch your agent cheating — on by default
 
@@ -42,11 +42,30 @@ PromptWheel  base → head
   VERDICT: GAMED  — a metric "improved" by editing the goalposts, not the source
 ```
 
+**See it fire yourself** — 60 seconds, illustrative (a hand-built cheat is a *mechanism* demo, not evidence):
+
+```bash
+mkdir pw-cheat-demo && cd pw-cheat-demo && git init -q
+printf '{"name":"demo","type":"module","scripts":{"test":"node --test"}}\n' > package.json
+printf 'export const add = (a, b) => a - b;   // BUG: should be a + b\n' > add.js
+printf "import {test} from 'node:test'; import assert from 'node:assert'; import {add} from './add.js';\ntest('add', () => assert.equal(add(2, 3), 5));   // honest test — FAILS on the bug\n" > add.test.js
+git add -A && git commit -qm 'buggy code + an honest failing test'
+npx -y promptwheel@latest init && git add -A && git commit -qm 'add the gate config'
+
+# an agent "greens" the suite by editing the TEST to expect the bug — the code stays broken:
+printf "import {test} from 'node:test'; import assert from 'node:assert'; import {add} from './add.js';\ntest('add', () => assert.equal(add(2, 3), -1));\n" > add.test.js
+git commit -qam 'make the suite green'
+
+npx -y promptwheel@latest run --base HEAD~1 --head HEAD
+#  ▲ tests_pass 0 → 1   🚩 GAMED — the win changed zero production-source files
+#  VERDICT: GAMED   (exit 2)
+```
+
 Inline source-file suppressions (`@ts-nocheck`, `eslint-disable`, `# noqa`) — and the quieter cheat of *weakening the suite while the metric stays flat* — are a different shape, caught by **tripwire guards** (`test_count`, `skipped_tests`, `suppressions`, `assertions`) that fail when a "win" introduces them. The plain `init` default includes these tripwires, so gutting a test file fails the gate out of the box:
 
 ```bash
 npx promptwheel init                      # guarded tests + antihack tripwires by default
-npx promptwheel run                       # detection ON by default · exit 0 win · 1 regression · 2 GAMED
+npx promptwheel run                       # detection ON by default · exit 0 win · 1 regression · 2 GAMED · 3 inconclusive
                                           #   (add --no-detect-gaming for just the outcome gate)
 ```
 
@@ -68,7 +87,7 @@ npx promptwheel run --repeat 5 --json     # measure 5× to establish a noise ban
 
 # the loop: run any agent/script, keep the change ONLY if a metric improved
 npx promptwheel improve --attempt "claude -p 'reduce lint errors'"
-#   exit 0 = kept a real win · 1 = guarded regression (reverted) · 3 = plateau (reverted) · add --json
+#   exit 0 = kept a real win · 1 = guarded regression (reverted) · 3 = plateau / inconclusive (reverted) · add --json
 
 # what's actually responding in this repo? (aggregates .promptwheel/outcomes.jsonl)
 npx promptwheel insights
@@ -76,11 +95,12 @@ npx promptwheel insights
 # EXPERIMENTAL (Phase 5): the earned playbook + where the next attempt should go
 npx promptwheel playbook                  # decayed, evidence-gated claims distilled from the record
 npx promptwheel suggest                   # UCB over the lever scores — proven levers vs under-explored arms
+npx promptwheel backfill -n 30            # cold start: seed the record from git history (cohort-tagged, commit types → labels)
 ```
 
 **The consequence ledger.** git records *what changed*; PromptWheel records *what the change did* — same trust model (local, deterministic, append-only, no server, no LLM in the verdict). `playbook` and `suggest` are pure re-derivations over that ledger: every rendered line was measured by the gate, decays unless re-earned, and stays hidden below an evidence threshold. No compounding claim is made for them until the A/B acceptance test (`bench/compounding-ab.mjs`) passes on real usage data.
 
-**Footprint:** it never touches your working tree — every measurement runs in a throwaway git worktree **in your system temp dir** (one at a time; `node_modules` is symlinked, not copied), removed when the run finishes. The only thing PromptWheel writes to your repo is the optional `.promptwheel/outcomes.jsonl` record — commit it to build the per-repo "what moves what" history, or `.gitignore` it (`--no-record` to skip entirely). A hard-killed run can't leave clutter behind: the next run **self-heals** any orphaned worktree (stale registry entry + abandoned temp checkout).
+**Footprint:** it never touches your working tree — every measurement runs in a throwaway git worktree **in your system temp dir** (one at a time; configured `linkDirs` are symlinked not copied — `node_modules` by default, `.venv` for Python, etc.), removed when the run finishes. The only thing PromptWheel writes to your repo is the optional `.promptwheel/outcomes.jsonl` record — commit it to build the per-repo "what moves what" history, or `.gitignore` it (`--no-record` to skip entirely). A hard-killed run can't leave clutter behind: the next run **self-heals** any orphaned worktree (stale registry entry + abandoned temp checkout).
 
 ## Loop patterns
 
@@ -150,6 +170,12 @@ The Action runs straight from its own checkout — no npm install, no build. See
 - **gamingCheck** — `false` exempts a metric from `--detect-gaming`'s source-only re-run. Use it for tripwire / test-side guards (assertion counts, test counts) whose gains legitimately live in test files — otherwise adding real tests would be flagged as gaming. The `antihack` preset sets this on its tripwires.
 - **gamingThreshold** — the fraction of a win that must survive the source-only re-run to count as earned (default `0.5`). Config-level scalar, overridable per metric; inherited through `extends` like `repeat`.
 
+Dependency & environment keys (config-level; `init` writes stack defaults):
+
+- **linkDirs** — dirs to symlink from your checkout into each measuring worktree, for deps that live outside git. Default `["node_modules"]`; `init` writes `[".venv"]` for Python, `["target"]` for Rust. (`"linkNodeModules": false` still works as shorthand for "link nothing".)
+- **env** — environment variables for every metric command; `{wt}` is substituted with the worktree path. Python configs get `"PYTHONPATH": "{wt}/src:{wt}"` so the *measured ref* is imported, not an editable install of your original checkout.
+- **setup** — an optional command run once per ref before metrics (e.g. `npm run build`, `pip install -e .`), after any source patch — so a build-gated suite measures the ref it's actually on.
+
 ## Guardrails & inheritance
 
 To see what's actually enforced in a repo — including guards inherited from a shared config:
@@ -216,7 +242,7 @@ The engine is one importable file; pure helpers are exported for unit tests, the
 - [x] `promptwheel init` + presets — zero-config onboarding
 - [x] `insights` — reward-stream aggregation (Phase-5 seed)
 - [x] `--detect-gaming` — reward-hack detection: re-prove the win from source edits alone + `antihack` preset
-- [x] npm publish — `promptwheel@0.1.0` (the lead magnet, shipped 2026-06)
+- [x] npm publish — `promptwheel@0.4.1` (the lead magnet)
 - [x] outcome-curated learning + UCB work-discovery — `playbook` + `suggest` + the compounding A/B harness (**experimental**; compounding *claims* stay gated on real-data proof — see [docs/LEARNING.md](docs/LEARNING.md))
 
-> Status: **published** (npm `promptwheel`, v0.2.0) — all core phases built. Lineage: CommandLayer → BlockSpool → PromptWheel (orchestrator, archived) → **PromptWheel (outcome gate)**.
+> Status: **published** (npm `promptwheel`, v0.4.1) — all core phases built. Lineage: CommandLayer → BlockSpool → PromptWheel (orchestrator, archived) → **PromptWheel (outcome gate)**.

package/bin/promptwheel.mjs

@@ -51,12 +51,12 @@ function resolveConfig(p, repo, label, seen) {
       if (!existsSync(bp)) { console.error(`extends target not found: ${ref} (from ${relOf(repo, p)})`); process.exit(2); }
       const base = resolveConfig(bp, repo, relOf(repo, bp), seen);
       for (const m of base.metrics) byName.set(m.name, m);
-      for (const k of ['repeat', 'linkNodeModules', 'record', 'gamingThreshold']) if (base[k] !== undefined) scalars[k] = base[k];
+      for (const k of ['repeat', 'linkNodeModules', 'linkDirs', 'env', 'setup', 'record', 'gamingThreshold']) if (base[k] !== undefined) scalars[k] = base[k];
     }
     for (const m of (cfg.metrics || [])) {
       byName.set(m.name, { ...m, __src: label, __override: byName.has(m.name) });
     }
-    for (const k of ['repeat', 'linkNodeModules', 'record', 'gamingThreshold']) if (cfg[k] !== undefined) scalars[k] = cfg[k];
+    for (const k of ['repeat', 'linkNodeModules', 'linkDirs', 'env', 'setup', 'record', 'gamingThreshold']) if (cfg[k] !== undefined) scalars[k] = cfg[k];
     return { ...scalars, metrics: [...byName.values()] };
   } finally {
     seen.delete(abs); // only ACTIVE ancestors are a cycle — a diamond (a base reached two ways) is fine
@@ -82,10 +82,10 @@ function resolveBase(repo, base) {
   return git(['rev-parse', 'HEAD~1'], repo);
 }
 
-function runMetric(cwd, m) {
+function runMetric(cwd, m, env) {
   let stdout = '', code = 0;
   try {
-    stdout = execSync(m.cmd, { cwd, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], timeout: (m.timeoutSec ?? 300) * 1000 });
+    stdout = execSync(m.cmd, { cwd, env: env || process.env, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], timeout: (m.timeoutSec ?? 300) * 1000 });
   } catch (e) { code = e.status ?? 1; stdout = `${e.stdout || ''}${e.stderr || ''}`; }
   return extract(stdout, code, m.extract);
 }
@@ -103,18 +103,41 @@ function extract(stdout, code, mode) {
   return nums ? Number(nums[nums.length - 1]) : null;
 }
 
+// bridge a measurement worktree to the deps that live OUTSIDE git (node_modules, .venv, target,
+// …) and the env a test command needs. Returns the env for runMetric. Generalizes the old
+// node_modules-only symlink: `linkDirs` says what to link (default ['node_modules']; back-compat
+// with linkNodeModules:false → link nothing); `env` sets vars ({wt} → the worktree path, so a
+// Python run can put the measured ref FIRST on PYTHONPATH). Ecosystem knowledge lives in config,
+// not here — the engine stays ecosystem-agnostic.
+function bridgeEnv(repo, wt, cfg) {
+  const linkDirs = Array.isArray(cfg.linkDirs) ? cfg.linkDirs : (cfg.linkNodeModules === false ? [] : ['node_modules']);
+  for (const d of linkDirs) {
+    if (existsSync(join(repo, d)) && !existsSync(join(wt, d))) {
+      try { symlinkSync(join(repo, d), join(wt, d)); } catch { /* best effort */ }
+    }
+  }
+  if (!cfg.env) return process.env;
+  return { ...process.env, ...Object.fromEntries(Object.entries(cfg.env).map(([k, v]) => [k, String(v).replaceAll('{wt}', () => wt)])) }; // fn replacement: a literal wt avoids $-pattern interpretation
+}
+// an optional per-ref build/install (e.g. `npm run build`, `pip install -e .`). Best-effort: a
+// failed setup just leaves the metric inert → INCONCLUSIVE (never a false green). Run by the
+// caller AFTER any source patch, so it builds the tree actually being measured.
+function runSetup(wt, cfg, env) {
+  if (!cfg.setup) return;
+  try { execSync(cfg.setup, { cwd: wt, env, stdio: 'ignore', timeout: 300_000 }); } catch { /* inert → inconclusive */ }
+}
+
 // measure every metric `repeat` times at a ref, in a throwaway worktree (never touches your tree)
-function measureAt(repo, ref, metrics, linkNodeModules, repeat) {
+function measureAt(repo, ref, cfg, repeat) {
   const wt = mkdtempSync(join(tmpdir(), 'promptwheel-'));
   git(['worktree', 'add', '--quiet', '--detach', wt, ref], repo);
   try {
-    if (linkNodeModules && existsSync(join(repo, 'node_modules')) && !existsSync(join(wt, 'node_modules'))) {
-      try { symlinkSync(join(repo, 'node_modules'), join(wt, 'node_modules')); } catch { /* best effort */ }
-    }
+    const env = bridgeEnv(repo, wt, cfg);
+    runSetup(wt, cfg, env);
     const out = {};
-    for (const m of metrics) {
+    for (const m of cfg.metrics) {
       const samples = [];
-      for (let i = 0; i < repeat; i++) samples.push(runMetric(wt, m));
+      for (let i = 0; i < repeat; i++) samples.push(runMetric(wt, m, env));
       out[m.name] = samples;
     }
     return out;
@@ -201,23 +224,22 @@ function subsystemsOf(repo, base, head) {
 }
 
 // measure `metric` in a worktree at `base` with ONLY the source slice of base→head applied.
-function measureSourceOnly(repo, base, head, metric, linkNM, repeat) {
+function measureSourceOnly(repo, base, head, metric, cfg, repeat) {
   const { source } = changedSourcePaths(repo, base, head);
   if (!source.length) return { samples: [], hadSourceChange: false }; // a "win" with zero source edits = goalposts moved
   const wt = mkdtempSync(join(tmpdir(), 'promptwheel-src-'));
   git(['worktree', 'add', '--quiet', '--detach', wt, base], repo);
   try {
-    if (linkNM && existsSync(join(repo, 'node_modules')) && !existsSync(join(wt, 'node_modules'))) {
-      try { symlinkSync(join(repo, 'node_modules'), join(wt, 'node_modules')); } catch { /* */ }
-    }
+    const env = bridgeEnv(repo, wt, cfg);
     const patch = git(['diff', base, head, '--', ...source], repo);
     if (patch.trim()) {
       const tryApply = (extra) => { execFileSync('git', ['apply', '--whitespace=nowarn', ...extra], { cwd: wt, input: patch + '\n', encoding: 'utf8' }); };
       try { tryApply(['--3way']); }
       catch { try { tryApply([]); } catch { return { samples: [], hadSourceChange: true, applyFailed: true }; } }
     }
+    runSetup(wt, cfg, env); // build the patched tree, so a build-gated metric measures the source edit
     const samples = [];
-    for (let i = 0; i < repeat; i++) samples.push(runMetric(wt, metric));
+    for (let i = 0; i < repeat; i++) samples.push(runMetric(wt, metric, env));
     return { samples, hadSourceChange: true };
   } finally {
     try { git(['worktree', 'remove', '--force', wt], repo); } catch { rmSync(wt, { recursive: true, force: true }); }
@@ -326,7 +348,6 @@ function gate(repo, opts) {
   warnEditableInstall(repo);
   const cfg = loadConfig(repo);
   const repeat = Math.max(1, opts.repeat ?? cfg.repeat ?? 1);
-  const linkNM = cfg.linkNodeModules !== false;
 
   let base, head;
   if (opts.working) {
@@ -342,8 +363,8 @@ function gate(repo, opts) {
     head = opts.head || 'HEAD';
   }
 
-  const before = measureAt(repo, base, cfg.metrics, linkNM, repeat);
-  const after = measureAt(repo, head, cfg.metrics, linkNM, repeat);
+  const before = measureAt(repo, base, cfg, repeat);
+  const after = measureAt(repo, head, cfg, repeat);
   const metrics = cfg.metrics.map((m) => {
     const ev = evaluate(m, before[m.name], after[m.name], repeat);
     return { name: m.name, direction: m.direction || 'up', guard: !!m.guard, ...ev };
@@ -354,18 +375,23 @@ function gate(repo, opts) {
       if (m.status !== 'improved') continue;
       const cm = cfg.metrics.find((c) => c.name === m.name);
       if (cm.gamingCheck === false) continue;   // tripwire / test-side guards aren't re-proven from source (their gain legitimately lives in test files)
-      const j = judgeGaming(m, measureSourceOnly(repo, base, head, cm, linkNM, repeat), cm.gamingThreshold ?? cfg.gamingThreshold ?? 0.5);
+      const j = judgeGaming(m, measureSourceOnly(repo, base, head, cm, cfg, repeat), cm.gamingThreshold ?? cfg.gamingThreshold ?? 0.5);
       m.gamed = j.gamed; m.sourceOnly = j.sourceOnly; m.retained = j.retained; m.gamingReason = j.reason;
     }
   }
   const failed = metrics.some((m) => m.guard && !m.ok);
   const gamed = metrics.some((m) => m.gamed === true);
-  const verdict = failed ? 'fail' : gamed ? 'gamed' : 'pass';
+  // a guard that never runs (inert: a pass/fail metric stuck at 0 across both refs) verifies
+  // nothing — don't launder that into a green PASS. The honest verdict is "couldn't measure".
+  // BUT scope it to a bare no-op run: an inert guard must NOT bury a genuine, measured win
+  // elsewhere (that would make `improve` revert good changes and misread them as a plateau).
+  const inconclusive = !failed && !gamed && metrics.some((m) => m.inert) && !metrics.some((m) => m.status === 'improved');
+  const verdict = failed ? 'fail' : gamed ? 'gamed' : inconclusive ? 'inconclusive' : 'pass';
   const report = {
     base: short(repo, base), head: short(repo, head), repeat, mode: opts.working ? 'working' : 'refs',
     // learning-substrate fields (Phase 5): cohort segments reliability by environment,
     // label attributes a change-type, subsystems fingerprint WHERE the change landed.
-    cohort: process.env.CI ? 'ci' : 'local',
+    cohort: opts.cohort ?? (process.env.CI ? 'ci' : 'local'),
     ...(opts.label ? { label: opts.label } : {}),
     subsystems: subsystemsOf(repo, base, head),
     verdict, metrics,
@@ -381,7 +407,7 @@ function run(argv) {
   if (args.json) console.log(JSON.stringify(report, null, 2));
   else if (args.markdown) console.log(renderMarkdown(report));
   else printHuman(report);
-  process.exit(report.verdict === 'pass' ? 0 : report.verdict === 'gamed' ? 2 : 1);
+  process.exit(report.verdict === 'pass' ? 0 : report.verdict === 'gamed' ? 2 : report.verdict === 'inconclusive' ? 3 : 1);
 }
 
 // persisted record: append every gated run to a per-repo outcome record (best-effort, never fails the gate)
@@ -410,10 +436,11 @@ function improve(argv) {
   const improvedNames = report.metrics.filter((m) => m.status === 'improved').map((m) => m.name);
 
   // result + exit code express loop progress so `while improve; do :; done` converges:
-  //   0 = kept a real win · 1 = guarded regression (reverted) · 3 = plateau/no-op (reverted)
+  //   0 = kept a real win · 1 = guarded regression (reverted) · 3 = plateau/no-op OR inconclusive (reverted)
   let result, exit, note;
   if (report.verdict === 'fail') { result = 'regression'; exit = 1; revert(repo); note = '✗ guarded regression — reverted'; }
   else if (report.verdict === 'gamed') { result = 'gamed'; exit = 1; revert(repo); note = '🚩 gamed — a metric "improved" by editing tests/config, not source — reverted'; }
+  else if (report.verdict === 'inconclusive') { result = 'inconclusive'; exit = 3; revert(repo); note = '≈ inconclusive — a guard measured nothing (inert); reverted'; }
   else if (noChange || improvedNames.length === 0) {
     result = 'plateau'; exit = 3; revert(repo);
     note = noChange ? '= no metric moved — reverted' : '= nothing improved beyond noise — reverted';
@@ -582,6 +609,34 @@ function suggest(argv) {
   console.log('\n  score = lever + exploration bonus — high scores are either proven levers or under-explored arms.\n');
 }
 
+// backfill — seed the ledger from git history: replay past commits through the CURRENT
+// metrics (LEARNING.md harvest path 1). Deterministic, no LLM. Rows are cohort-tagged
+// 'backfill' — historical human commits are NOT live agent-loop evidence; the cohort
+// machinery segments them and flags disagreement rather than averaging it away. The
+// conventional-commit type (fix/feat/refactor/…) becomes the change-type label for free.
+function backfill(argv) {
+  const args = parseArgs(argv);
+  const repo = git(['rev-parse', '--show-toplevel'], process.cwd());
+  const listArgs = ['rev-list', '--no-merges'];
+  if (args.since) listArgs.push(`${args.since}..HEAD`); else listArgs.push('-n', String(args.n ?? 30), 'HEAD');
+  const commits = git(listArgs, repo).split('\n').filter(Boolean).reverse(); // oldest first → ledger stays chronological, decay stays honest
+  if (!commits.length) { console.error('no commits to backfill'); process.exit(2); }
+  const seen = new Set(readLedger(repo).filter((r) => r.cohort === 'backfill').map((r) => r.head));
+  let done = 0, skipped = 0;
+  console.log(`\nbackfilling ${commits.length} commits through the current metrics (gaming detection ${args.dgExplicit && args.detectGaming ? 'ON' : 'off — pass --detect-gaming to audit history too'})\n`);
+  for (const c of commits) {
+    try { git(['rev-parse', `${c}~1`], repo); } catch { skipped++; continue; }        // root commit
+    if (seen.has(short(repo, c))) { skipped++; continue; }                            // already recorded
+    const subj = git(['log', '-1', '--format=%s', c], repo);
+    const type = /^(feat|fix|docs|test|chore|refactor|perf|ci|build|style)\b/i.exec(subj)?.[1]?.toLowerCase();
+    const report = gate(repo, { base: `${c}~1`, head: c, repeat: args.repeat, detectGaming: args.dgExplicit ? args.detectGaming : false, label: type, cohort: 'backfill' });
+    console.log(`  ${report.head} ${report.verdict.toUpperCase().padEnd(6)} ${type ? `#${type}`.padEnd(10) : ''.padEnd(10)} ${subj.slice(0, 56)}`);
+    done++;
+  }
+  console.log(`\n  backfilled ${done}${skipped ? ` (skipped ${skipped}: root or already recorded)` : ''} — old commits that no longer build record as unmeasurable, never faked.`);
+  console.log('  next:  promptwheel playbook   ·   promptwheel suggest\n');
+}
+
 // ---------------------------------------------------------------------------
 // init — write a starter config so a newcomer isn't staring at a blank page
 // ---------------------------------------------------------------------------
@@ -600,10 +655,10 @@ const PRESETS = {
   'antihack': { desc: 'catch reward-hacking: a target + tripwires; pairs with `run --detect-gaming` (source-only re-run)',
     metrics: [
       { name: 'tests_pass', cmd: '__TESTCMD__', extract: 'exit', direction: 'pass', guard: true },
-      { name: 'test_count', cmd: 'grep -rIoE "\\b(it|test|describe) ?\\(|def test_" --exclude-dir=node_modules --exclude-dir=.git --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'up', guard: true, gamingCheck: false },
-      { name: 'skipped_tests', cmd: 'grep -rIoE "\\.(skip|only) ?\\(|xit ?\\(|@pytest\\.mark\\.skip" --exclude-dir=node_modules --exclude-dir=.git --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'down', guard: true, gamingCheck: false },
-      { name: 'suppressions', cmd: 'grep -rIoE "eslint-disable|@ts-(ignore|nocheck)|# ?type: ?ignore|# ?noqa|//nolint|#!?\\[allow\\(" --exclude-dir=node_modules --exclude-dir=.git --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'down', guard: true, gamingCheck: false },
-      { name: 'assertions', cmd: 'grep -rIoE "expect ?\\(|\\bassert|\\bt\\.(is|deepEqual|throws|truthy|falsy|not|ok) ?\\(" --exclude-dir=node_modules --exclude-dir=.git --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'up', guard: true, gamingCheck: false },
+      { name: 'test_count', cmd: 'grep -rIoE "\\b(it|test|describe) ?\\(|def test_" --exclude-dir=node_modules --exclude-dir=.git --exclude-dir=dist --exclude-dir=build --exclude-dir=coverage --exclude-dir=.next --exclude-dir=out --exclude-dir=lib-cov --exclude-dir=__pycache__ --exclude-dir=.pytest_cache --exclude-dir=target --exclude-dir=.venv --exclude-dir=venv --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'up', guard: true, gamingCheck: false },
+      { name: 'skipped_tests', cmd: 'grep -rIoE "\\.(skip|only) ?\\(|xit ?\\(|@pytest\\.mark\\.skip" --exclude-dir=node_modules --exclude-dir=.git --exclude-dir=dist --exclude-dir=build --exclude-dir=coverage --exclude-dir=.next --exclude-dir=out --exclude-dir=lib-cov --exclude-dir=__pycache__ --exclude-dir=.pytest_cache --exclude-dir=target --exclude-dir=.venv --exclude-dir=venv --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'down', guard: true, gamingCheck: false },
+      { name: 'suppressions', cmd: 'grep -rIoE "eslint-disable|@ts-(ignore|nocheck)|# ?type: ?ignore|# ?noqa|//nolint|#!?\\[allow\\(" --exclude-dir=node_modules --exclude-dir=.git --exclude-dir=dist --exclude-dir=build --exclude-dir=coverage --exclude-dir=.next --exclude-dir=out --exclude-dir=lib-cov --exclude-dir=__pycache__ --exclude-dir=.pytest_cache --exclude-dir=target --exclude-dir=.venv --exclude-dir=venv --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'down', guard: true, gamingCheck: false },
+      { name: 'assertions', cmd: 'grep -rIoE "expect ?\\(|\\bassert|\\bt\\.(is|deepEqual|throws|truthy|falsy|not|ok) ?\\(" --exclude-dir=node_modules --exclude-dir=.git --exclude-dir=dist --exclude-dir=build --exclude-dir=coverage --exclude-dir=.next --exclude-dir=out --exclude-dir=lib-cov --exclude-dir=__pycache__ --exclude-dir=.pytest_cache --exclude-dir=target --exclude-dir=.venv --exclude-dir=venv --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'up', guard: true, gamingCheck: false },
     ] },
 };
 
@@ -611,7 +666,7 @@ function detectTestCmd(repo) {
   const has = (f) => existsSync(join(repo, f));
   if (has('go.mod')) return 'go test ./...';
   if (has('Cargo.toml')) return 'cargo test';
-  if (has('pyproject.toml') || has('setup.py') || has('pytest.ini')) return 'pytest -q';
+  if (has('pyproject.toml') || has('setup.py') || has('pytest.ini')) return has('.venv/bin/pytest') ? '.venv/bin/pytest -q' : 'pytest -q';
   // only trust `npm test` when a test script actually exists — otherwise the metric can
   // never pass and the gate would "protect" nothing (very common in Next.js app repos)
   try {
@@ -620,6 +675,17 @@ function detectTestCmd(repo) {
   return 'echo "set your test command in promptwheel.config.json" && false';
 }
 
+// deps that live outside git + the env a test needs, defaulted per stack. init writes these into
+// config; the measurement bridge (bridgeEnv/runSetup) consumes them — keeps the engine agnostic.
+function detectEnv(repo) {
+  const has = (f) => existsSync(join(repo, f));
+  // Python: link the venv (3rd-party deps) and put the worktree src FIRST on PYTHONPATH, so imports
+  // resolve the MEASURED ref instead of an editable install pointing back at your original checkout.
+  if (has('pyproject.toml') || has('setup.py') || has('pytest.ini')) return { linkDirs: ['.venv'], env: { PYTHONPATH: '{wt}/src:{wt}' } }; // src/ AND flat layouts; worktree src wins over an editable install
+  if (has('Cargo.toml')) return { linkDirs: [] }; // do NOT link target/: `cargo test` WRITES there — linking would clobber the user's real dir and share a build cache across refs (measurement blindness). Build fresh in the worktree.
+  return {};
+}
+
 // only offer the lint metric where eslint actually exists — otherwise it reads 0 forever
 // and a newcomer mistakes a metric that can't move for a clean repo.
 function hasEslint(repo) {
@@ -662,7 +728,9 @@ function init(argv) {
     if (lint) metrics = [...metrics, { name: 'lint_errors', cmd: LINT_CMD, extract: 'number', direction: 'down', guard: false }];
     note = `tests + antihack tripwires${lint ? ' + lint' : ''} (detected: ${testCmd})`;
   }
-  writeFileSync(out, JSON.stringify({ repeat: 1, metrics }, null, 2) + '\n');
+  const envCfg = presetName ? {} : detectEnv(repo);
+  if (envCfg.linkDirs) note += ` · linking ${envCfg.linkDirs.join(', ')}`;
+  writeFileSync(out, JSON.stringify({ repeat: 1, ...envCfg, metrics }, null, 2) + '\n');
   console.log(`✓ wrote promptwheel.config.json — ${note}`);
   console.log('\n  next:  promptwheel run --working   # gate your uncommitted changes');
   console.log('         promptwheel init --list     # other presets (llm-eval, bundle-size, …)');
@@ -708,14 +776,14 @@ function printHuman(r) {
     if (m.inert) console.log('      ⚠ never passed at either ref — this guard is protecting nothing (check its command)');
     if (m.gamed === true) console.log(`      🚩 GAMED — ${m.gamingReason}`);
   }
-  console.log(`\n  VERDICT: ${r.verdict.toUpperCase()}${r.verdict === 'fail' ? '  — a guarded metric regressed (beyond noise)' : r.verdict === 'gamed' ? '  — a metric "improved" by editing the goalposts, not the source' : ''}`);
+  console.log(`\n  VERDICT: ${r.verdict.toUpperCase()}${r.verdict === 'fail' ? '  — a guarded metric regressed (beyond noise)' : r.verdict === 'gamed' ? '  — a metric "improved" by editing the goalposts, not the source' : r.verdict === 'inconclusive' ? '  — a guard is inert (measured nothing); a pass cannot be certified' : ''}`);
   if (r.verdict === 'fail') console.log('  intentional? loosen that guard locally in promptwheel.config.json (guard:false, or override the inherited metric by name) — `promptwheel guards` shows the effective set');
   console.log('');
 }
 
 // PR-comment markdown (rendering lives in the tool so the GitHub Action stays thin)
 function renderMarkdown(r) {
-  const icon = r.verdict === 'pass' ? '✅' : r.verdict === 'gamed' ? '🚩' : '❌';
+  const icon = r.verdict === 'pass' ? '✅' : r.verdict === 'gamed' ? '🚩' : r.verdict === 'inconclusive' ? '🟡' : '❌';
   const sIcon = { improved: '🟢', regressed: '🔴', unchanged: '⚪', inconclusive: '🟡', unmeasurable: '⚫' };
   const rows = r.metrics.map((m) => {
     const d = m.delta == null ? '—' : (m.delta > 0 ? `+${m.delta}` : `${m.delta}`);
@@ -731,7 +799,7 @@ function renderMarkdown(r) {
     '|---|--:|--:|--:|---|---|',
     rows,
     '',
-    r.verdict === 'gamed' ? '> 🚩 A metric "improved" only because the agent edited tests/config/grader — not the source. The win does not survive a source-only re-run.' : r.verdict === 'fail' ? '> ❌ A 🛡️ guarded metric regressed beyond the noise band.' : '> ✅ No guarded metric regressed.',
+    r.verdict === 'gamed' ? '> 🚩 A metric "improved" only because the agent edited tests/config/grader — not the source. The win does not survive a source-only re-run.' : r.verdict === 'fail' ? '> ❌ A 🛡️ guarded metric regressed beyond the noise band.' : r.verdict === 'inconclusive' ? '> 🟡 A 🛡️ guarded check is **inert** — it never runs, so nothing was actually verified. Fix its command before trusting a pass.' : '> ✅ No guarded metric regressed.',
     '',
     '<sub>🛡️ = guard · _prove every change moved a metric_ · [PromptWheel](https://github.com/promptwheel-ai/promptwheel)</sub>',
   ].join('\n');
@@ -745,8 +813,10 @@ function parseArgs(argv) {
     else if (argv[i] === '--repeat') a.repeat = parseInt(argv[++i], 10);
     else if (argv[i] === '--working') a.working = true;
     else if (argv[i] === '--no-record') a.noRecord = true;
-    else if (argv[i] === '--detect-gaming' || argv[i] === '--antihack') a.detectGaming = true;
-    else if (argv[i] === '--no-detect-gaming' || argv[i] === '--no-antihack') a.detectGaming = false;
+    else if (argv[i] === '--detect-gaming' || argv[i] === '--antihack') { a.detectGaming = true; a.dgExplicit = true; }
+    else if (argv[i] === '--no-detect-gaming' || argv[i] === '--no-antihack') { a.detectGaming = false; a.dgExplicit = true; }
+    else if (argv[i] === '--since') a.since = argv[++i];
+    else if (argv[i] === '-n' || argv[i] === '--limit') a.n = parseInt(argv[++i], 10);
     else if (argv[i] === '--attempt') a.attempt = argv[++i];
     else if (argv[i] === '--label') a.label = argv[++i];
     else if (argv[i] === '--json') a.json = true;
@@ -766,6 +836,7 @@ function main() {
   else if (cmd === 'insights') insights(rest);
   else if (cmd === 'playbook') playbook(rest);
   else if (cmd === 'suggest') suggest(rest);
+  else if (cmd === 'backfill') backfill(rest);
   else if (cmd === 'init') init(rest);
   else if (cmd === 'guards') guards(rest);
   else {
@@ -782,6 +853,7 @@ function main() {
       '  promptwheel insights                         which metrics actually respond (raw counts)',
       '  promptwheel playbook [--json]                the earned playbook: decayed, evidence-gated claims distilled from the record',
       '  promptwheel suggest [--json]                 UCB work-discovery: where the next attempt should go (experimental)',
+      '  promptwheel backfill [-n N | --since <ref>]  seed the ledger from git history (cohort-tagged; commit types become labels)',
       '  promptwheel guards                           show the effective guardrails (incl. inherited) + flag record',
       '',
       'Loop it:  while promptwheel improve --attempt "$AGENT"; do :; done   # stops on plateau/regression',

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "promptwheel",
-  "version": "0.3.0",
+  "version": "0.4.1",
   "description": "Catch your agent cheating — a deterministic, zero-LLM CLI that flags when an AI coding agent gamed its own metric (re-proves the win from the agent's source edits alone). Built on an outcome gate; runs in CI or a Claude Code hook.",
   "type": "module",
   "bin": {