promptwheel 0.1.2 → 0.3.0

package/README.md

@@ -42,23 +42,23 @@ PromptWheel  base → head
   VERDICT: GAMED  — a metric "improved" by editing the goalposts, not the source
 ```
 
-Inline source-file suppressions (`@ts-nocheck`, `eslint-disable`, `# noqa`) are a different shape, caught by the **`antihack` preset** — a target metric plus tripwire guards (`test_count`, `skipped_tests`, `suppressions`, `assertions`) that fail when a "win" introduces them:
+Inline source-file suppressions (`@ts-nocheck`, `eslint-disable`, `# noqa`) — and the quieter cheat of *weakening the suite while the metric stays flat* — are a different shape, caught by **tripwire guards** (`test_count`, `skipped_tests`, `suppressions`, `assertions`) that fail when a "win" introduces them. The plain `init` default includes these tripwires, so gutting a test file fails the gate out of the box:
 
 ```bash
-npx promptwheel init --preset antihack    # target + tripwire guards
+npx promptwheel init                      # guarded tests + antihack tripwires by default
 npx promptwheel run                       # detection ON by default · exit 0 win · 1 regression · 2 GAMED
                                           #   (add --no-detect-gaming for just the outcome gate)
 ```
 
-Deterministic, zero-LLM, zero-network: an LLM judge asking "did you cheat?" is itself gameable; this is a diff partition plus a re-run, so a flag is trustworthy without a human in the loop. The 50%-of-gain-survives threshold is the default and is tunable.
+Deterministic, zero-LLM, zero-network: an LLM judge asking "did you cheat?" is itself gameable; this is a diff partition plus a re-run, so a flag is trustworthy without a human in the loop. The 50%-of-gain-survives threshold is the default and is tunable via `gamingThreshold` (config-level, or per-metric).
 
-This is **one layer**, not a silver bullet: it catches the evaluator-tampering class (test/grader/golden/config edits) deterministically and for free, so the expensive layers — held-out tests for semantically-weak wins, an LLM judge or a human for intent and leakage — are reserved for the calls only they can make. See **[docs/DETECTION-LAYERS.md](docs/DETECTION-LAYERS.md)** for the coverage matrix and honest scope, and **[bench/RESULTS.md](bench/RESULTS.md)** for the measured numbers (`node bench/gaming-bench.mjs` to reproduce).
+This is **one layer**, not a silver bullet: it catches the evaluator-tampering class (test/grader/golden/config edits) deterministically and for free, so the expensive layers — held-out tests for semantically-weak wins, an LLM judge or a human for intent and leakage — are reserved for the calls only they can make. See **[docs/DETECTION-LAYERS.md](docs/DETECTION-LAYERS.md)** for the coverage matrix and honest scope, and **[bench/RESULTS.md](bench/RESULTS.md)** for the measured numbers (`node bench/gaming-bench.mjs` to reproduce — it includes a cross-stack table with a real pytest run + a numeric eval-pass-rate metric). **Gate your own stack** — pytest · tsc · coverage · bundle · llm-eval — copy a config from **[examples/](examples/)**.
 
 ## Use
 
 ```bash
 # 0. write a starter config for your stack (or hand-write promptwheel.config.json)
-npx promptwheel init                      # detects stack → guarded test metric + lint
+npx promptwheel init                      # detects stack → guarded tests + antihack tripwires (+ lint if eslint is set up)
 npx promptwheel init --list               # presets: tests-pass · lint · bundle-size · llm-eval · antihack
 
 # measure a change
@@ -72,8 +72,14 @@ npx promptwheel improve --attempt "claude -p 'reduce lint errors'"
 
 # what's actually responding in this repo? (aggregates .promptwheel/outcomes.jsonl)
 npx promptwheel insights
+
+# EXPERIMENTAL (Phase 5): the earned playbook + where the next attempt should go
+npx promptwheel playbook                  # decayed, evidence-gated claims distilled from the record
+npx promptwheel suggest                   # UCB over the lever scores — proven levers vs under-explored arms
 ```
 
+**The consequence ledger.** git records *what changed*; PromptWheel records *what the change did* — same trust model (local, deterministic, append-only, no server, no LLM in the verdict). `playbook` and `suggest` are pure re-derivations over that ledger: every rendered line was measured by the gate, decays unless re-earned, and stays hidden below an evidence threshold. No compounding claim is made for them until the A/B acceptance test (`bench/compounding-ab.mjs`) passes on real usage data.
+
 **Footprint:** it never touches your working tree — every measurement runs in a throwaway git worktree **in your system temp dir** (one at a time; `node_modules` is symlinked, not copied), removed when the run finishes. The only thing PromptWheel writes to your repo is the optional `.promptwheel/outcomes.jsonl` record — commit it to build the per-repo "what moves what" history, or `.gitignore` it (`--no-record` to skip entirely). A hard-killed run can't leave clutter behind: the next run **self-heals** any orphaned worktree (stale registry entry + abandoned temp checkout).
 
 ## Loop patterns
@@ -142,6 +148,7 @@ The Action runs straight from its own checkout — no npm install, no build. See
 - **direction** — `up` (higher better) · `down` (lower better) · `pass` (boolean 0/1).
 - **guard** — `true` = a *trusted* regression **fails** the gate; `false` = informational.
 - **gamingCheck** — `false` exempts a metric from `--detect-gaming`'s source-only re-run. Use it for tripwire / test-side guards (assertion counts, test counts) whose gains legitimately live in test files — otherwise adding real tests would be flagged as gaming. The `antihack` preset sets this on its tripwires.
+- **gamingThreshold** — the fraction of a win that must survive the source-only re-run to count as earned (default `0.5`). Config-level scalar, overridable per metric; inherited through `extends` like `repeat`.
 
 ## Guardrails & inheritance
 
@@ -186,13 +193,13 @@ The accumulated record of **which change-types move which metrics** is the asset
 - [docs/VISION.md](docs/VISION.md) — why we pivoted from orchestrator to outcome gate, the thesis, the moat, the open-core model.
 - [docs/ROADMAP.md](docs/ROADMAP.md) — the phased plan and the ship-now/stay-thin guardrails.
 - [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — how the engine works: schemas, extract modes, the trust/noise model.
-- [docs/LEARNING.md](docs/LEARNING.md) — the (research-gated) Phase-5 design: ACE-style playbook + UCB work-discovery.
+- [docs/LEARNING.md](docs/LEARNING.md) — the (research-gated) Phase-5 design: outcome-curated playbook (Agentic Context Engineering, Stanford 2510.04618) + UCB work-discovery.
 - [CLAUDE.md](CLAUDE.md) — the constitution for anyone (human or agent) working in this repo.
 
 ## Develop
 
 ```bash
-npm test     # 37 dep-free tests (node:test) — unit + integration, no dependencies
+npm test     # the dep-free suite (node:test) — unit + integration, no dependencies
 ```
 
 The engine is one importable file; pure helpers are exported for unit tests, the CLI runs only when invoked directly. Add a test with every behavior change.
@@ -210,6 +217,6 @@ The engine is one importable file; pure helpers are exported for unit tests, the
 - [x] `insights` — reward-stream aggregation (Phase-5 seed)
 - [x] `--detect-gaming` — reward-hack detection: re-prove the win from source edits alone + `antihack` preset
 - [x] npm publish — `promptwheel@0.1.0` (the lead magnet, shipped 2026-06)
-- [ ] ACE-style learning + UCB work-discovery (**frozen** — gated on data + ≥1 paid engagement; see [docs/LEARNING.md](docs/LEARNING.md))
+- [x] outcome-curated learning + UCB work-discovery — `playbook` + `suggest` + the compounding A/B harness (**experimental**; compounding *claims* stay gated on real-data proof — see [docs/LEARNING.md](docs/LEARNING.md))
 
-> Status: **published v0.1.0** — the consulting lead magnet (npm `promptwheel@0.1.0`), all core phases built. Lineage: CommandLayer → BlockSpool → PromptWheel (orchestrator, archived) → **PromptWheel (outcome gate)**.
+> Status: **published** (npm `promptwheel`, v0.2.0) — all core phases built. Lineage: CommandLayer → BlockSpool → PromptWheel (orchestrator, archived) → **PromptWheel (outcome gate)**.

package/bin/promptwheel.mjs

@@ -51,12 +51,12 @@ function resolveConfig(p, repo, label, seen) {
       if (!existsSync(bp)) { console.error(`extends target not found: ${ref} (from ${relOf(repo, p)})`); process.exit(2); }
       const base = resolveConfig(bp, repo, relOf(repo, bp), seen);
       for (const m of base.metrics) byName.set(m.name, m);
-      for (const k of ['repeat', 'linkNodeModules', 'record']) if (base[k] !== undefined) scalars[k] = base[k];
+      for (const k of ['repeat', 'linkNodeModules', 'record', 'gamingThreshold']) if (base[k] !== undefined) scalars[k] = base[k];
     }
     for (const m of (cfg.metrics || [])) {
       byName.set(m.name, { ...m, __src: label, __override: byName.has(m.name) });
     }
-    for (const k of ['repeat', 'linkNodeModules', 'record']) if (cfg[k] !== undefined) scalars[k] = cfg[k];
+    for (const k of ['repeat', 'linkNodeModules', 'record', 'gamingThreshold']) if (cfg[k] !== undefined) scalars[k] = cfg[k];
     return { ...scalars, metrics: [...byName.values()] };
   } finally {
     seen.delete(abs); // only ACTIVE ancestors are a cycle — a diamond (a base reached two ways) is fine
@@ -152,7 +152,10 @@ function evaluate(m, beforeS, afterS, repeat) {
 
   // guards fail only on a TRUSTED regression (within-noise regressions are not failures)
   const ok = m.guard ? !regressed : true;
-  return { before, after, delta, status, ok, confidence, noise };
+  // a guarded pass-metric that never passes is protecting NOTHING (broken test cmd,
+  // missing script, failed install) — surface it instead of folding into a green verdict.
+  const inert = !!m.guard && dir === 'pass' && before === 0 && after === 0;
+  return { before, after, delta, status, ok, confidence, noise, ...(inert ? { inert: true } : {}) };
 }
 
 // ---------------------------------------------------------------------------
@@ -169,7 +172,11 @@ const NON_SOURCE = [
   /(^|\/)test_[^/]*\.py$/i, /_test\.py$/i, /(^|\/)conftest\.py$/i,
   /\.snap$/i, /(^|\/)golden[^/]*$/i, /\.golden$/i,
   /(^|\/)(eval|grader|score)[^/]*\.[cm]?[jt]s$/i,
-  /(^|\/)(jest|vitest|playwright|cypress|karma|babel|eslint|tsconfig|pytest)[^/]*\.(json|[cm]?[jt]s|ya?ml|ini|cfg|toml)$/i,
+  // config files only — require a .config/.conf/.setup segment so PRODUCTION source that
+  // happens to be named after a tool (e.g. src/installers/eslint.ts) is NOT swept out of
+  // the source slice (a false sweep here can flag an honest win as GAMED).
+  /(^|\/)(jest|vitest|playwright|cypress|karma|babel|eslint|pytest)[^/]*\.(config|conf|setup)\.[^/]+$/i,
+  /(^|\/)tsconfig[^/]*\.json$/i,
   /(^|\/)\.(eslintrc|babelrc|prettierrc)[^/]*$/i,
   /(^|\/)(pytest\.ini|setup\.cfg|\.flake8|tox\.ini|eslint\.config\.[cm]?js)$/i,
 ];
@@ -181,6 +188,18 @@ function changedSourcePaths(repo, base, head) {
   return { source: all.filter((p) => !isNonSource(p)), nonSource: all.filter(isNonSource) };
 }
 
+// coarse change-location fingerprint for the outcome record: top source dirs of the diff
+function subsystemsOf(repo, base, head) {
+  try {
+    const counts = {};
+    for (const p of changedSourcePaths(repo, base, head).source) {
+      const d = p.includes('/') ? p.slice(0, p.indexOf('/')) : '(root)';
+      counts[d] = (counts[d] || 0) + 1;
+    }
+    return Object.entries(counts).sort((a, b) => b[1] - a[1]).slice(0, 3).map(([d]) => d);
+  } catch { return []; }
+}
+
 // measure `metric` in a worktree at `base` with ONLY the source slice of base→head applied.
 function measureSourceOnly(repo, base, head, metric, linkNM, repeat) {
   const { source } = changedSourcePaths(repo, base, head);
@@ -215,17 +234,18 @@ function gain(m, before, val) {
 }
 
 // a metric that improved full-diff: does the win survive when only source edits are applied?
-// gamed = source edits alone reproduce < half the gain (the rest came from editing the goalposts).
-function judgeGaming(m, srcResult) {
+// gamed = source edits alone reproduce less than `threshold` of the gain (default 0.5 — the
+// rest came from editing the goalposts). Tune via `gamingThreshold` (config-level or per-metric).
+function judgeGaming(m, srcResult, threshold = 0.5) {
   if (!srcResult.hadSourceChange) return { gamed: true, sourceOnly: null, retained: 0, reason: 'the "win" changed zero production-source files — only test/config/grader/golden' };
   if (srcResult.applyFailed) return { gamed: null, sourceOnly: null, retained: null, reason: 'source-only patch did not apply cleanly — inconclusive' };
   const sourceOnly = median(srcResult.samples);
   const full = gain(m, m.before, m.after);
   if (full == null || full <= 0) return { gamed: false, sourceOnly, retained: 1, reason: 'no real improvement to re-prove' };
   const retained = +(gain(m, m.before, sourceOnly) / full).toFixed(3);
-  const gamed = retained < 0.5;
+  const gamed = retained < threshold;
   return { gamed, sourceOnly, retained, reason: gamed
-    ? `only ${(retained * 100).toFixed(0)}% of the gain survives when test/config/grader changes are reverted — most of the "win" came from editing the goalposts`
+    ? `only ${(retained * 100).toFixed(0)}% of the gain survives when test/config/grader changes are reverted (threshold ${(threshold * 100).toFixed(0)}%) — most of the "win" came from editing the goalposts`
     : `${(retained * 100).toFixed(0)}% of the gain survives source-only — the source earned it` };
 }
 
@@ -279,9 +299,31 @@ function selfHeal(repo) {
   } catch { /* cleanup must never block the gate */ }
 }
 
+// Python editable installs (src-layout `pip install -e .`) can pin imports to the ORIGINAL
+// checkout: the worktree's tests then import your current tree at BOTH refs and every delta
+// reads 0 — the gate is structurally blind on that repo shape. Undetectable from the verdict
+// (it looks like "unchanged"), so detect the editable install itself and warn. Verified by a
+// controlled repro (CHANGELOG 0.2.2).
+function warnEditableInstall(repo) {
+  if (!existsSync(join(repo, 'pyproject.toml')) && !existsSync(join(repo, 'setup.py'))) return;
+  try {
+    const probe = [
+      'import site,glob,os,sys',
+      'repo=os.path.realpath(sys.argv[1])',
+      'sps=set(site.getsitepackages()+[site.getusersitepackages()])',
+      'files=[f for sp in sps for pat in ("__editable__*","*.egg-link","*.pth") for f in glob.glob(os.path.join(sp,pat)) if os.path.isfile(f)]',
+      'hit=any(repo in open(f,errors="ignore").read() for f in files)',
+      'print("HIT" if hit else "")',
+    ].join('\n');
+    const out = execFileSync('python3', ['-c', probe, repo], { encoding: 'utf8', timeout: 5000 }).trim();
+    if (out === 'HIT') console.error('⚠ editable install of this repo detected (pip install -e): worktree measurements may import your ORIGINAL checkout, not the measured ref — deltas can read 0. Use a non-editable install in the measuring venv, or put the worktree src on PYTHONPATH.');
+  } catch { /* no python / cannot tell — stay quiet */ }
+}
+
 // the shared core: measure a change (base→head) and return the structured report
 function gate(repo, opts) {
   selfHeal(repo);
+  warnEditableInstall(repo);
   const cfg = loadConfig(repo);
   const repeat = Math.max(1, opts.repeat ?? cfg.repeat ?? 1);
   const linkNM = cfg.linkNodeModules !== false;
@@ -312,14 +354,22 @@ function gate(repo, opts) {
       if (m.status !== 'improved') continue;
       const cm = cfg.metrics.find((c) => c.name === m.name);
       if (cm.gamingCheck === false) continue;   // tripwire / test-side guards aren't re-proven from source (their gain legitimately lives in test files)
-      const j = judgeGaming(m, measureSourceOnly(repo, base, head, cm, linkNM, repeat));
+      const j = judgeGaming(m, measureSourceOnly(repo, base, head, cm, linkNM, repeat), cm.gamingThreshold ?? cfg.gamingThreshold ?? 0.5);
       m.gamed = j.gamed; m.sourceOnly = j.sourceOnly; m.retained = j.retained; m.gamingReason = j.reason;
     }
   }
   const failed = metrics.some((m) => m.guard && !m.ok);
   const gamed = metrics.some((m) => m.gamed === true);
   const verdict = failed ? 'fail' : gamed ? 'gamed' : 'pass';
-  const report = { base: short(repo, base), head: short(repo, head), repeat, mode: opts.working ? 'working' : 'refs', verdict, metrics };
+  const report = {
+    base: short(repo, base), head: short(repo, head), repeat, mode: opts.working ? 'working' : 'refs',
+    // learning-substrate fields (Phase 5): cohort segments reliability by environment,
+    // label attributes a change-type, subsystems fingerprint WHERE the change landed.
+    cohort: process.env.CI ? 'ci' : 'local',
+    ...(opts.label ? { label: opts.label } : {}),
+    subsystems: subsystemsOf(repo, base, head),
+    verdict, metrics,
+  };
   if (!opts.noRecord && cfg.record !== false) recordOutcome(repo, report);
   return report;
 }
@@ -327,7 +377,7 @@ function gate(repo, opts) {
 function run(argv) {
   const args = parseArgs(argv);
   const repo = git(['rev-parse', '--show-toplevel'], process.cwd());
-  const report = gate(repo, { base: args.base, head: args.head, working: args.working, repeat: args.repeat, noRecord: args.noRecord, detectGaming: args.detectGaming });
+  const report = gate(repo, { base: args.base, head: args.head, working: args.working, repeat: args.repeat, noRecord: args.noRecord, detectGaming: args.detectGaming, label: args.label });
   if (args.json) console.log(JSON.stringify(report, null, 2));
   else if (args.markdown) console.log(renderMarkdown(report));
   else printHuman(report);
@@ -355,7 +405,7 @@ function improve(argv) {
   try { execSync(args.attempt, { cwd: repo, stdio: 'inherit' }); }
   catch (e) { console.error(`  (attempt exited ${e.status ?? 1} — gating whatever it changed)`); }
 
-  const report = gate(repo, { working: true, repeat: args.repeat, noRecord: args.noRecord, detectGaming: args.detectGaming });
+  const report = gate(repo, { working: true, repeat: args.repeat, noRecord: args.noRecord, detectGaming: args.detectGaming, label: args.label ?? args.attempt });
   const noChange = report.metrics.every((m) => m.delta === 0 || m.delta == null);
   const improvedNames = report.metrics.filter((m) => m.status === 'improved').map((m) => m.name);
 
@@ -389,7 +439,7 @@ function revert(repo) {
 }
 
 // Phase-5 seed: turn the accumulated reward stream into signal. Thin on purpose —
-// this is the substrate a future ACE playbook / UCB work-discovery loop trains on,
+// this is the substrate a future outcome-curated playbook / UCB work-discovery loop trains on,
 // NOT that loop itself. Just honest aggregation, no model.
 function insights(argv) {
   const args = parseArgs(argv);
@@ -417,6 +467,121 @@ function insights(argv) {
   console.log('  highest-lever metrics are where an agent loop should spend its attempts.\n');
 }
 
+// ---------------------------------------------------------------------------
+// Phase 5 — outcome-curated playbook + UCB work-discovery.
+// Unfrozen 2026-07-02 by explicit founder decision overriding D7 (see docs/LEARNING.md).
+// Design constraints that survive the unfreeze:
+//   - PURE VIEW: the append-only ledger (.promptwheel/outcomes.jsonl) is the only store;
+//     the playbook is re-derived on every read (decay at read time — no curator state to rot).
+//   - EVIDENCE-GATED: a key renders a claim only past MIN_MOVED weighted observations;
+//     everything below that is counted, not asserted.
+//   - CLAIM-GATED: no public compounding claim until bench/compounding-ab.mjs passes on
+//     real usage data. The gate on the CLAIM outlived the gate on the CODE.
+// ---------------------------------------------------------------------------
+const HALF_LIFE = 20;   // runs — an entry's weight halves every HALF_LIFE runs unless re-earned
+const MIN_MOVED = 3;    // weighted moved-observations before a key earns a rendered claim
+const EXPLORE_C = 1.0;  // UCB exploration constant
+
+function readLedger(repo) {
+  const f = join(repo, '.promptwheel', 'outcomes.jsonl');
+  if (!existsSync(f)) return [];
+  return readFileSync(f, 'utf8').split('\n').filter(Boolean)
+    .map((l) => { try { return JSON.parse(l); } catch { return null; } }).filter(Boolean);
+}
+
+// fold the ledger into decayed per-key stats. keys: `metric` · `metric @ subsystem` · `metric # label`
+function foldOutcomes(runs) {
+  const keys = new Map();
+  const N = runs.length;
+  runs.forEach((r, idx) => {
+    const w = Math.pow(0.5, (N - 1 - idx) / HALF_LIFE);   // idx N-1 = newest → weight 1
+    for (const m of (r.metrics || [])) {
+      const tags = [m.name];
+      for (const s of (r.subsystems || [])) tags.push(`${m.name} @ ${s}`);
+      if (r.label) tags.push(`${m.name} # ${r.label}`);
+      for (const key of tags) {
+        const k = keys.get(key) ?? { key, metric: m.name, w: 0, n: 0, imp: 0, reg: 0, moved: 0, effects: [], cohorts: {}, last: null };
+        k.n += 1; k.w += w;
+        const coh = (k.cohorts[r.cohort || 'unknown'] ??= { imp: 0, reg: 0 });
+        if (m.status === 'improved') {
+          k.imp += w; k.moved += w; coh.imp += 1;
+          if (typeof m.delta === 'number' && m.delta !== 0) k.effects.push(Math.abs(m.delta));
+        } else if (m.status === 'regressed') { k.reg += w; k.moved += w; coh.reg += 1; }
+        k.last = r.ts || k.last;
+        keys.set(key, k);
+      }
+    }
+  });
+  return [...keys.values()];
+}
+
+const betaMean = (imp, reg) => (1 + imp) / (2 + imp + reg);   // Beta(1,1)-smoothed helpfulness
+
+// composite lever = smoothed helpfulness × responsiveness; UCB adds an exploration bonus
+function scoreKey(k, totalW) {
+  const p = betaMean(k.imp, k.reg);
+  const move = k.w ? k.moved / k.w : 0;
+  const lever = p * move;
+  const ucb = lever + EXPLORE_C * Math.sqrt(Math.log(Math.max(Math.E, totalW)) / Math.max(1e-6, k.w));
+  return { p: +p.toFixed(3), move: +move.toFixed(3), lever: +lever.toFixed(3), ucb: +ucb.toFixed(3), effect: +(median(k.effects) ?? 0).toFixed(6) };
+}
+
+// cohorts that disagree in sign are flagged, never averaged into a lie
+function cohortNote(k) {
+  const cs = Object.entries(k.cohorts).filter(([, c]) => c.imp + c.reg > 0);
+  if (cs.length < 2) return '';
+  const signs = new Set(cs.map(([, c]) => Math.sign(c.imp - c.reg)));
+  return signs.size > 1 ? ' · ⚠ cohort-dependent (ci vs local disagree)' : '';
+}
+
+function loadEntries(repo) {
+  const runs = readLedger(repo);
+  if (!runs.length) { console.error('no outcome record yet — run the gate a few times first (.promptwheel/outcomes.jsonl)'); process.exit(2); }
+  const keys = foldOutcomes(runs);
+  const totalW = keys.filter((k) => k.key === k.metric).reduce((s, k) => s + k.w, 0);
+  const entries = keys.map((k) => ({ ...k, ...scoreKey(k, totalW), sufficient: k.moved >= MIN_MOVED }))
+    .sort((a, b) => b.lever - a.lever || b.w - a.w);
+  return { runs, entries };
+}
+
+// the distilled, execution-earned context an agent (or CLAUDE.md) can consume
+function playbook(argv) {
+  const args = parseArgs(argv);
+  const repo = git(['rev-parse', '--show-toplevel'], process.cwd());
+  const { runs, entries } = loadEntries(repo);
+  if (args.json) { console.log(JSON.stringify({ runs: runs.length, halfLife: HALF_LIFE, minMoved: MIN_MOVED, entries }, null, 2)); return; }
+  const suff = entries.filter((e) => e.sufficient);
+  const pct = (x) => `${Math.round(x * 100)}%`;
+  const out = [];
+  out.push(`## Earned playbook — ${runs.length} gated runs, decay half-life ${HALF_LIFE} runs`);
+  out.push('*(every line below was measured by the gate, not asserted; entries decay unless re-earned)*', '');
+  if (!suff.length) {
+    out.push(`_Nothing has earned a claim yet — a key needs ≥${MIN_MOVED} weighted moved-observations. Keep gating; ${entries.length} keys are accumulating._`);
+  } else {
+    for (const e of suff.slice(0, 20)) {
+      out.push(`- **${e.key}** — helpful ${pct(e.p)} when it moves (${e.imp.toFixed(1)}✓/${e.reg.toFixed(1)}✗ weighted), responds in ${pct(e.move)} of runs, median effect ${e.effect}${cohortNote(e)}`);
+    }
+    const hidden = entries.length - suff.length;
+    if (hidden > 0) out.push('', `_${hidden} more keys below the evidence threshold — counted, not asserted._`);
+  }
+  console.log(out.join('\n'));
+}
+
+// UCB work-discovery: where should the loop spend its NEXT attempt? Proposes measured
+// targets (metric/subsystem arms) — never code advice; the measurement stays the message.
+function suggest(argv) {
+  const args = parseArgs(argv);
+  const repo = git(['rev-parse', '--show-toplevel'], process.cwd());
+  const { runs, entries } = loadEntries(repo);
+  const arms = entries.filter((e) => e.key === e.metric || e.key.includes(' @ ')).sort((a, b) => b.ucb - a.ucb);
+  if (args.json) { console.log(JSON.stringify({ runs: runs.length, thin: runs.length < 10, arms: arms.slice(0, 10) }, null, 2)); return; }
+  console.log(`\nPromptWheel suggest — UCB over ${runs.length} gated runs${runs.length < 10 ? '  (thin record: exploration dominates — treat as a coin with opinions)' : ''}\n`);
+  for (const a of arms.slice(0, 5)) {
+    console.log(`  ${a.ucb.toFixed(3)}  ${a.key.padEnd(28)} lever ${a.lever.toFixed(3)} (helpful ${a.p}, responds ${a.move}) · evidence ${a.moved.toFixed(1)} moved / ${a.n} runs${a.sufficient ? '' : ' · below claim threshold'}`);
+  }
+  console.log('\n  score = lever + exploration bonus — high scores are either proven levers or under-explored arms.\n');
+}
+
 // ---------------------------------------------------------------------------
 // init — write a starter config so a newcomer isn't staring at a blank page
 // ---------------------------------------------------------------------------
@@ -435,10 +600,10 @@ const PRESETS = {
   'antihack': { desc: 'catch reward-hacking: a target + tripwires; pairs with `run --detect-gaming` (source-only re-run)',
     metrics: [
       { name: 'tests_pass', cmd: '__TESTCMD__', extract: 'exit', direction: 'pass', guard: true },
-      { name: 'test_count', cmd: 'grep -rIoE "\\b(it|test|describe) ?\\(|def test_" --exclude-dir=node_modules --exclude-dir=.git . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'up', guard: true, gamingCheck: false },
-      { name: 'skipped_tests', cmd: 'grep -rIoE "\\.(skip|only) ?\\(|xit ?\\(|@pytest\\.mark\\.skip" --exclude-dir=node_modules --exclude-dir=.git . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'down', guard: true, gamingCheck: false },
-      { name: 'suppressions', cmd: 'grep -rIoE "eslint-disable|@ts-(ignore|nocheck)|# ?type: ?ignore|# ?noqa" --exclude-dir=node_modules --exclude-dir=.git . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'down', guard: true, gamingCheck: false },
-      { name: 'assertions', cmd: 'grep -rIoE "expect ?\\(|\\bassert" --exclude-dir=node_modules --exclude-dir=.git . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'up', guard: true, gamingCheck: false },
+      { name: 'test_count', cmd: 'grep -rIoE "\\b(it|test|describe) ?\\(|def test_" --exclude-dir=node_modules --exclude-dir=.git --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'up', guard: true, gamingCheck: false },
+      { name: 'skipped_tests', cmd: 'grep -rIoE "\\.(skip|only) ?\\(|xit ?\\(|@pytest\\.mark\\.skip" --exclude-dir=node_modules --exclude-dir=.git --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'down', guard: true, gamingCheck: false },
+      { name: 'suppressions', cmd: 'grep -rIoE "eslint-disable|@ts-(ignore|nocheck)|# ?type: ?ignore|# ?noqa|//nolint|#!?\\[allow\\(" --exclude-dir=node_modules --exclude-dir=.git --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'down', guard: true, gamingCheck: false },
+      { name: 'assertions', cmd: 'grep -rIoE "expect ?\\(|\\bassert|\\bt\\.(is|deepEqual|throws|truthy|falsy|not|ok) ?\\(" --exclude-dir=node_modules --exclude-dir=.git --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'up', guard: true, gamingCheck: false },
     ] },
 };
 
@@ -447,10 +612,26 @@ function detectTestCmd(repo) {
   if (has('go.mod')) return 'go test ./...';
   if (has('Cargo.toml')) return 'cargo test';
   if (has('pyproject.toml') || has('setup.py') || has('pytest.ini')) return 'pytest -q';
-  if (has('package.json')) return 'npm test --silent';
+  // only trust `npm test` when a test script actually exists — otherwise the metric can
+  // never pass and the gate would "protect" nothing (very common in Next.js app repos)
+  try {
+    if (JSON.parse(readFileSync(join(repo, 'package.json'), 'utf8')).scripts?.test) return 'npm test --silent';
+  } catch { /* no or unparsable package.json — fall through */ }
   return 'echo "set your test command in promptwheel.config.json" && false';
 }
 
+// only offer the lint metric where eslint actually exists — otherwise it reads 0 forever
+// and a newcomer mistakes a metric that can't move for a clean repo.
+function hasEslint(repo) {
+  const files = ['eslint.config.js', 'eslint.config.mjs', 'eslint.config.cjs',
+    '.eslintrc', '.eslintrc.json', '.eslintrc.js', '.eslintrc.cjs', '.eslintrc.yml', '.eslintrc.yaml'];
+  if (files.some((f) => existsSync(join(repo, f)))) return true;
+  try {
+    const pkg = JSON.parse(readFileSync(join(repo, 'package.json'), 'utf8'));
+    return !!(pkg.devDependencies?.eslint || pkg.dependencies?.eslint);
+  } catch { return false; }
+}
+
 function init(argv) {
   if (argv.includes('--list')) {
     console.log('PromptWheel presets (promptwheel init --preset <name>):\n');
@@ -472,13 +653,14 @@ function init(argv) {
     metrics = metrics.map((m) => m.cmd === '__TESTCMD__' ? { ...m, cmd: detectTestCmd(repo) } : m); // fill the preset's target
     note = presetName;
   } else {
-    // sensible default: tests-pass (guarded) + lint (info — can't fail a newcomer's first run)
+    // default = the headline posture: guarded tests + the antihack tripwires, so the FIRST
+    // cheat a newcomer tries (gut the tests while the metric stays flat) fails the gate —
+    // "catch your agent cheating" must be true out of the box, not only under a preset.
     const testCmd = detectTestCmd(repo);
-    metrics = [
-      { name: 'tests_pass', cmd: testCmd, extract: 'exit', direction: 'pass', guard: true },
-      { name: 'lint_errors', cmd: LINT_CMD, extract: 'number', direction: 'down', guard: false },
-    ];
-    note = `tests-pass + lint (detected: ${testCmd})`;
+    const lint = hasEslint(repo);
+    metrics = PRESETS['antihack'].metrics.map((m) => m.cmd === '__TESTCMD__' ? { ...m, cmd: testCmd } : m);
+    if (lint) metrics = [...metrics, { name: 'lint_errors', cmd: LINT_CMD, extract: 'number', direction: 'down', guard: false }];
+    note = `tests + antihack tripwires${lint ? ' + lint' : ''} (detected: ${testCmd})`;
   }
   writeFileSync(out, JSON.stringify({ repeat: 1, metrics }, null, 2) + '\n');
   console.log(`✓ wrote promptwheel.config.json — ${note}`);
@@ -523,9 +705,12 @@ function printHuman(r) {
     const tag = m.guard ? (m.ok ? 'guard✓' : 'GUARD✗') : 'info';
     const d = m.delta == null ? '—' : (m.delta > 0 ? `+${m.delta}` : `${m.delta}`);
     console.log(`  ${arrowFor(m)} ${m.name.padEnd(18)} ${String(m.before).padStart(8)} → ${String(m.after).padStart(8)}  (${d}, ${m.status}) [${tag}, ${m.confidence}]`);
+    if (m.inert) console.log('      ⚠ never passed at either ref — this guard is protecting nothing (check its command)');
     if (m.gamed === true) console.log(`      🚩 GAMED — ${m.gamingReason}`);
   }
-  console.log(`\n  VERDICT: ${r.verdict.toUpperCase()}${r.verdict === 'fail' ? '  — a guarded metric regressed (beyond noise)' : r.verdict === 'gamed' ? '  — a metric "improved" by editing the goalposts, not the source' : ''}\n`);
+  console.log(`\n  VERDICT: ${r.verdict.toUpperCase()}${r.verdict === 'fail' ? '  — a guarded metric regressed (beyond noise)' : r.verdict === 'gamed' ? '  — a metric "improved" by editing the goalposts, not the source' : ''}`);
+  if (r.verdict === 'fail') console.log('  intentional? loosen that guard locally in promptwheel.config.json (guard:false, or override the inherited metric by name) — `promptwheel guards` shows the effective set');
+  console.log('');
 }
 
 // PR-comment markdown (rendering lives in the tool so the GitHub Action stays thin)
@@ -534,7 +719,7 @@ function renderMarkdown(r) {
   const sIcon = { improved: '🟢', regressed: '🔴', unchanged: '⚪', inconclusive: '🟡', unmeasurable: '⚫' };
   const rows = r.metrics.map((m) => {
     const d = m.delta == null ? '—' : (m.delta > 0 ? `+${m.delta}` : `${m.delta}`);
-    const status = m.gamed === true ? `🚩 gamed (${(m.retained * 100).toFixed(0)}% survives source-only)` : `${sIcon[m.status] || ''} ${m.status}`;
+    const status = m.gamed === true ? `🚩 gamed (${(m.retained * 100).toFixed(0)}% survives source-only)` : `${sIcon[m.status] || ''} ${m.status}${m.inert ? ' ⚠ never passes' : ''}`;
     return `| ${m.guard ? '🛡️ ' : ''}${m.name} | ${m.before} | ${m.after} | ${d} | ${status} | ${m.confidence} |`;
   }).join('\n');
   return [
@@ -563,21 +748,24 @@ function parseArgs(argv) {
     else if (argv[i] === '--detect-gaming' || argv[i] === '--antihack') a.detectGaming = true;
     else if (argv[i] === '--no-detect-gaming' || argv[i] === '--no-antihack') a.detectGaming = false;
     else if (argv[i] === '--attempt') a.attempt = argv[++i];
+    else if (argv[i] === '--label') a.label = argv[++i];
     else if (argv[i] === '--json') a.json = true;
-    else if (argv[i] === '--markdown') a.markdown = true;
+    else if (argv[i] === '--markdown' || argv[i] === '--md') a.markdown = true;
   }
   return a;
 }
 
 // pure, side-effect-free helpers are exported for unit testing; the CLI below only
 // runs when this file is executed directly (not when imported by the test suite).
-export { extract, evaluate, median, spread, renderMarkdown, isNonSource, gain, judgeGaming };
+export { extract, evaluate, median, spread, renderMarkdown, isNonSource, gain, judgeGaming, foldOutcomes, betaMean, scoreKey };
 
 function main() {
   const [cmd, ...rest] = process.argv.slice(2);
   if (cmd === 'run') run(rest);
   else if (cmd === 'improve') improve(rest);
   else if (cmd === 'insights') insights(rest);
+  else if (cmd === 'playbook') playbook(rest);
+  else if (cmd === 'suggest') suggest(rest);
   else if (cmd === 'init') init(rest);
   else if (cmd === 'guards') guards(rest);
   else {
@@ -591,7 +779,9 @@ function main() {
       '  promptwheel run --working                    gate uncommitted changes (incl. newly added files)',
       '  promptwheel improve --attempt "<cmd>"        run an agent/script; keep only if a metric improved',
       '                                               exit 0=kept · 1=regression · 3=plateau · add --json',
-      '  promptwheel insights                         which metrics actually respond (loop memory)',
+      '  promptwheel insights                         which metrics actually respond (raw counts)',
+      '  promptwheel playbook [--json]                the earned playbook: decayed, evidence-gated claims distilled from the record',
+      '  promptwheel suggest [--json]                 UCB work-discovery: where the next attempt should go (experimental)',
       '  promptwheel guards                           show the effective guardrails (incl. inherited) + flag record',
       '',
       'Loop it:  while promptwheel improve --attempt "$AGENT"; do :; done   # stops on plateau/regression',

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "promptwheel",
-  "version": "0.1.2",
+  "version": "0.3.0",
   "description": "Catch your agent cheating — a deterministic, zero-LLM CLI that flags when an AI coding agent gamed its own metric (re-proves the win from the agent's source edits alone). Built on an outcome gate; runs in CI or a Claude Code hook.",
   "type": "module",
   "bin": {