promptwheel 0.4.4 → 0.4.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/README.md +61 -59
  2. package/bin/promptwheel.mjs +84 -76
  3. package/package.json +2 -2
package/README.md CHANGED
@@ -2,17 +2,19 @@
2
2
 
3
3
  [![CI](https://github.com/promptwheel-ai/promptwheel/actions/workflows/ci.yml/badge.svg)](https://github.com/promptwheel-ai/promptwheel/actions/workflows/ci.yml) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) ![Node ≥18](https://img.shields.io/badge/node-%E2%89%A518-brightgreen) ![deps: zero](https://img.shields.io/badge/deps-zero-blue)
4
4
 
5
- **Add a referee to your coding loop.** A green test should mean your agent *fixed the code* — not that it edited the test to match the bug. PromptWheel is the referee that tells the difference: deterministic, no LLM, and un-gameable, because a player can't call its own fouls.
5
+ **A deterministic check for your coding loop.** A green test should mean your agent *fixed the code*, not that it edited the test to match the bug. PromptWheel tells the difference: it re-proves every win from the source changes alone, with no LLM, so a win that only survived by editing the test reads `GAMED`. Un-gameable by construction, because the code being checked never gets to grade itself.
6
6
 
7
- PromptWheel re-proves every "win" using the agent's **source edits alone**. If the gate only went green because the agent edited the test, mocked the grader, suppressed the error (`@ts-ignore` / `eslint-disable`), or deleted the feature, the win evaporates when those edits are reverted → **`VERDICT GAMED`, exit 2**. No LLM in the loop — a diff partition plus a re-run, so every flag is reproducible in seconds with a human-readable reason.
7
+ PromptWheel re-proves every "win" using the agent's **source edits alone**. If the gate only went green because the agent edited the test, mocked the grader, suppressed the error (`@ts-ignore` / `eslint-disable`), or deleted the feature, the win evaporates when those edits are reverted → **`VERDICT GAMED`, exit 2**. No LLM in the loop, a diff partition plus a re-run, so every flag is reproducible in seconds with a human-readable reason.
8
8
 
9
- It's built on an **outcome gate**: for any change it re-runs your metric commands (tests, lint, `tsc`, coverage, bundle, eval) in throwaway git worktrees before and after, and refuses to trust a delta inside the measurement noise. The gate everyone ships asks *"did the number move?"* — PromptWheel also asks *"did the agent **earn** it?"*
9
+ It's built on an **outcome gate**: for any change it re-runs your metric commands (tests, lint, `tsc`, coverage, bundle, eval) in throwaway git worktrees before and after, and refuses to trust a delta inside the measurement noise. The gate everyone ships asks *"did the number move?"*, PromptWheel also asks *"did the agent **earn** it?"*
10
10
 
11
- PromptWheel is the **signal, not the loop driver**: wire it as the verifier inside Claude Code `/loop`, a Ralph `while`-loop, or a Beads pull-loop. Each turn it answers one question — *did this turn earn its keep, and did the agent earn it honestly?* — so the loop improves instead of confidently degrading. (In CI it's the **outcome gate for AI code**: the same verdict, as a PR check.)
11
+ PromptWheel is the **signal, not the loop driver**: wire it as the verifier inside Claude Code `/loop`, a Ralph `while`-loop, or a Beads pull-loop. Each turn it answers one question, *did this turn earn its keep, and did the agent earn it honestly?*, so the loop improves instead of confidently degrading. (In CI it's the **outcome gate for AI code**: the same verdict, as a PR check.)
12
12
 
13
- > Same name, new meaning. The "wheel" is the **improvement flywheel**: every turn only counts if it **provably moved a metric without regressing another.** Orchestration (the old "wheel of prompts") *and the outcome gate itself* are now solved, commoditized problems; the open one is **catching when the agent games that gate** — making the reward signal one you can't cheat.
13
+ PromptWheel is the reference check for the [Verified-Outcome Contract](https://github.com/sawbuck-ai/sawbuck/blob/master/CONTRACT.md), the open interface for a verifiable coding loop.
14
14
 
15
- AI coding agents (and humans) declare success when a change *compiles and tests pass*. They almost never verify that a change **improved a real measurable thing without regressing another** — and outcome data rots in dashboards (DORA, Swarmia) disconnected from the change that caused it. PromptWheel closes that loop.
15
+ > Same name, new meaning. The "wheel" is the **improvement flywheel**: every turn only counts if it **provably moved a metric without regressing another.** Orchestration (the old "wheel of prompts") *and the outcome gate itself* are now solved, commoditized problems; the open one is **catching when the agent games that gate**, making the reward signal one you can't cheat.
16
+
17
+ AI coding agents (and humans) declare success when a change *compiles and tests pass*. They almost never verify that a change **improved a real measurable thing without regressing another**, and outcome data rots in dashboards (DORA, Swarmia) disconnected from the change that caused it. PromptWheel closes that loop.
16
18
 
17
19
  For any change (`base` → `head`), it measures each configured metric in an **isolated git worktree** before and after, **refuses to trust a delta inside the measurement noise band**, enforces **regression guards**, and emits a structured verdict:
18
20
 
@@ -27,41 +29,41 @@ PromptWheel a1b2c3d → e4f5g6h (×5)
27
29
  VERDICT: PASS
28
30
  ```
29
31
 
30
- Exit `0` pass · `1` fail · `2` gamed · `3` inconclusive (a guard measured nothing) — CI-friendly. No build step, zero dependencies, Node 18+.
32
+ Exit `0` pass · `1` fail · `2` gamed · `3` inconclusive (a guard measured nothing), CI-friendly. No build step, zero dependencies, Node 18+.
31
33
 
32
- ## Catch your agent cheating — on by default
34
+ ## Catch your agent cheating, on by default
33
35
 
34
- The headline feature, and it runs **by default** (in both `run` and `improve`; pass `--no-detect-gaming` for the bare outcome gate). After the normal gate, every *win* is re-proven using the agent's source edits **alone**: PromptWheel splits the diff into production source vs. `{test, config, grader, golden}` files, rebuilds a clean worktree at the base with **only the source slice** applied, and re-runs the gate. If the win doesn't survive — because it only passed by skipping/deleting a test, mocking the grader, editing a golden, relaxing a config, or the "win" touched zero source files — the verdict is **`GAMED` (exit 2)**:
36
+ The headline feature, and it runs **by default** (in both `run` and `improve`; pass `--no-detect-gaming` for the bare outcome gate). After the normal gate, every *win* is re-proven using the agent's source edits **alone**: PromptWheel splits the diff into production source vs. `{test, config, grader, golden}` files, rebuilds a clean worktree at the base with **only the source slice** applied, and re-runs the gate. If the win doesn't survive, because it only passed by skipping/deleting a test, mocking the grader, editing a golden, relaxing a config, or the "win" touched zero source files, the verdict is **`GAMED` (exit 2)**:
35
37
 
36
38
  ```
37
39
  $ promptwheel run
38
40
 
39
41
  PromptWheel base → head
40
42
  ▲ tests_pass 0 → 1 (+1, improved) [guard✓, high]
41
- 🚩 GAMED — the "win" changed zero production-source files — only test/config/grader/golden
42
- VERDICT: GAMED — a metric "improved" by editing the goalposts, not the source
43
+ 🚩 GAMED, the "win" changed zero production-source files, only test/config/grader/golden
44
+ VERDICT: GAMED, a metric "improved" by editing the goalposts, not the source
43
45
  ```
44
46
 
45
- **See it fire yourself** — 60 seconds, illustrative (a hand-built cheat is a *mechanism* demo, not evidence):
47
+ **See it fire yourself**, 60 seconds, illustrative (a hand-built cheat is a *mechanism* demo, not evidence):
46
48
 
47
49
  ```bash
48
50
  mkdir pw-cheat-demo && cd pw-cheat-demo && git init -q
49
51
  printf '{"name":"demo","type":"module","scripts":{"test":"node --test"}}\n' > package.json
50
52
  printf 'export const add = (a, b) => a - b; // BUG: should be a + b\n' > add.js
51
- printf "import {test} from 'node:test'; import assert from 'node:assert'; import {add} from './add.js';\ntest('add', () => assert.equal(add(2, 3), 5)); // honest test — FAILS on the bug\n" > add.test.js
53
+ printf "import {test} from 'node:test'; import assert from 'node:assert'; import {add} from './add.js';\ntest('add', () => assert.equal(add(2, 3), 5)); // honest test, FAILS on the bug\n" > add.test.js
52
54
  git add -A && git commit -qm 'buggy code + an honest failing test'
53
55
  npx -y promptwheel@latest init && git add -A && git commit -qm 'add the gate config'
54
56
 
55
- # an agent "greens" the suite by editing the TEST to expect the bug — the code stays broken:
57
+ # an agent "greens" the suite by editing the TEST to expect the bug, the code stays broken:
56
58
  printf "import {test} from 'node:test'; import assert from 'node:assert'; import {add} from './add.js';\ntest('add', () => assert.equal(add(2, 3), -1));\n" > add.test.js
57
59
  git commit -qam 'make the suite green'
58
60
 
59
61
  npx -y promptwheel@latest run --base HEAD~1 --head HEAD
60
- # ▲ tests_pass 0 → 1 🚩 GAMED — the win changed zero production-source files
62
+ # ▲ tests_pass 0 → 1 🚩 GAMED, the win changed zero production-source files
61
63
  # VERDICT: GAMED (exit 2)
62
64
  ```
63
65
 
64
- Inline source-file suppressions (`@ts-nocheck`, `eslint-disable`, `# noqa`) — and the quieter cheat of *weakening the suite while the metric stays flat* — are a different shape, caught by **tripwire guards** (`test_count`, `skipped_tests`, `suppressions`, `assertions`) that fail when a "win" introduces them. The plain `init` default includes these tripwires, so gutting a test file fails the gate out of the box:
66
+ Inline source-file suppressions (`@ts-nocheck`, `eslint-disable`, `# noqa`), and the quieter cheat of *weakening the suite while the metric stays flat*, are a different shape, caught by **tripwire guards** (`test_count`, `skipped_tests`, `suppressions`, `assertions`) that fail when a "win" introduces them. The plain `init` default includes these tripwires, so gutting a test file fails the gate out of the box:
65
67
 
66
68
  ```bash
67
69
  npx promptwheel init # guarded tests + antihack tripwires by default
@@ -71,7 +73,7 @@ npx promptwheel run # detection ON by default · exit 0 wi
71
73
 
72
74
  Deterministic, zero-LLM, zero-network: an LLM judge asking "did you cheat?" is itself gameable; this is a diff partition plus a re-run, so a flag is trustworthy without a human in the loop. The 50%-of-gain-survives threshold is the default and is tunable via `gamingThreshold` (config-level, or per-metric).
73
75
 
74
- This is **one layer**, not a silver bullet: it catches the evaluator-tampering class (test/grader/golden/config edits) deterministically and for free, so the expensive layers — held-out tests for semantically-weak wins, an LLM judge or a human for intent and leakage — are reserved for the calls only they can make. See **[docs/DETECTION-LAYERS.md](docs/DETECTION-LAYERS.md)** for the coverage matrix and honest scope, and **[bench/RESULTS.md](bench/RESULTS.md)** for the measured numbers (`node bench/gaming-bench.mjs` to reproduce — it includes a cross-stack table with a real pytest run + a numeric eval-pass-rate metric). **Gate your own stack** — pytest · tsc · coverage · bundle · llm-eval — copy a config from **[examples/](examples/)**.
76
+ This is **one layer**, not a silver bullet: it catches the evaluator-tampering class (test/grader/golden/config edits) deterministically and for free, so the expensive layers, held-out tests for semantically-weak wins, an LLM judge or a human for intent and leakage, are reserved for the calls only they can make. See **[docs/DETECTION-LAYERS.md](docs/DETECTION-LAYERS.md)** for the coverage matrix and honest scope, and **[bench/RESULTS.md](bench/RESULTS.md)** for the measured numbers (`node bench/gaming-bench.mjs` to reproduce, it includes a cross-stack table with a real pytest run + a numeric eval-pass-rate metric). **Gate your own stack**, pytest · tsc · coverage · bundle · llm-eval, copy a config from **[examples/](examples/)**.
75
77
 
76
78
  ## Use
77
79
 
@@ -94,13 +96,13 @@ npx promptwheel insights
94
96
 
95
97
  # EXPERIMENTAL (Phase 5): the earned playbook + where the next attempt should go
96
98
  npx promptwheel playbook # decayed, evidence-gated claims distilled from the record
97
- npx promptwheel suggest # UCB over the lever scores — proven levers vs under-explored arms
99
+ npx promptwheel suggest # UCB over the lever scores, proven levers vs under-explored arms
98
100
  npx promptwheel backfill -n 30 # cold start: seed the record from git history (cohort-tagged, commit types → labels)
99
101
  ```
100
102
 
101
- **The consequence ledger.** git records *what changed*; PromptWheel records *what the change did* — same trust model (local, deterministic, append-only, no server, no LLM in the verdict). `playbook` and `suggest` are pure re-derivations over that ledger: every rendered line was measured by the gate, decays unless re-earned, and stays hidden below an evidence threshold. No compounding claim is made for them until the A/B acceptance test (`bench/compounding-ab.mjs`) passes on real usage data.
103
+ **The consequence ledger.** git records *what changed*; PromptWheel records *what the change did*, same trust model (local, deterministic, append-only, no server, no LLM in the verdict). `playbook` and `suggest` are pure re-derivations over that ledger: every rendered line was measured by the gate, decays unless re-earned, and stays hidden below an evidence threshold. No compounding claim is made for them until the A/B acceptance test (`bench/compounding-ab.mjs`) passes on real usage data.
102
104
 
103
- **Footprint:** it never touches your working tree — every measurement runs in a throwaway git worktree **in your system temp dir** (one at a time; configured `linkDirs` are symlinked not copied — `node_modules` by default, `.venv` for Python, etc.), removed when the run finishes. The only thing PromptWheel writes to your repo is the optional `.promptwheel/outcomes.jsonl` record — commit it to build the per-repo "what moves what" history, or `.gitignore` it (`--no-record` to skip entirely). A hard-killed run can't leave clutter behind: the next run **self-heals** any orphaned worktree (stale registry entry + abandoned temp checkout).
105
+ **Footprint:** it never touches your working tree, every measurement runs in a throwaway git worktree **in your system temp dir** (one at a time; configured `linkDirs` are symlinked not copied, `node_modules` by default, `.venv` for Python, etc.), removed when the run finishes. The only thing PromptWheel writes to your repo is the optional `.promptwheel/outcomes.jsonl` record, commit it to build the per-repo "what moves what" history, or `.gitignore` it (`--no-record` to skip entirely). A hard-killed run can't leave clutter behind: the next run **self-heals** any orphaned worktree (stale registry entry + abandoned temp checkout).
104
106
 
105
107
  ## Loop patterns
106
108
 
@@ -114,11 +116,11 @@ while npx promptwheel improve --attempt "claude -p 'speed up the hot path'"; do
114
116
  npx promptwheel run --working --json # branch on .verdict / per-metric .status
115
117
  ```
116
118
 
117
- The exit code is the contract — `0` kept · `1` regression · `3` plateau — so any driver (`/loop`, a Ralph `while`, a Beads pull-loop) converges without parsing anything. PromptWheel never drives the loop; it only says whether the turn counted.
119
+ The exit code is the contract, `0` kept · `1` regression · `3` plateau, so any driver (`/loop`, a Ralph `while`, a Beads pull-loop) converges without parsing anything. PromptWheel never drives the loop; it only says whether the turn counted.
118
120
 
119
- **Two callers, by design.** Your *agent* consumes the verdict — it's the loop's per-turn reward (`improve` / `run --working --json`; exit `0` kept · `1`/`2` reverted · `3` plateau). The *harness* (a Stop-hook or CI) runs `--detect-gaming` — the audit the agent **can't self-clear**, because a contestant can't referee itself (see [docs/ENFORCEMENT.md](docs/ENFORCEMENT.md)). Same tool; *who calls it* is the difference between a reward and an audit.
121
+ **Two callers, by design.** Your *agent* consumes the verdict as the loop's per-turn reward (`improve` / `run --working --json`; exit `0` kept · `1`/`2` reverted · `3` plateau). The *harness* (a Stop-hook or CI) runs `--detect-gaming`, the audit the agent **can't self-clear**, because the check never grades itself; it re-proves from the source alone (see [docs/ENFORCEMENT.md](docs/ENFORCEMENT.md)). Same tool; *who calls it* is the difference between a reward and an audit.
120
122
 
121
- ## In Claude Code — plugin
123
+ ## In Claude Code, plugin
122
124
 
123
125
  Bring the gate into Claude Code as slash commands:
124
126
 
@@ -129,7 +131,7 @@ Bring the gate into Claude Code as slash commands:
129
131
 
130
132
  Then `/promptwheel:setup` (write a config) · `/promptwheel:gate` (gate uncommitted changes) · `/promptwheel:improve <cmd>` (keep-if-improved) · `/promptwheel:insights`. The plugin wraps the CLI, so install that too (`npm i -g promptwheel`). See [`plugins/promptwheel/`](plugins/promptwheel/).
131
133
 
132
- ## In CI — GitHub Action
134
+ ## In CI, GitHub Action
133
135
 
134
136
  Drop this in your repo (it posts a verdict comment on every PR and fails the check on a guarded regression beyond noise):
135
137
 
@@ -148,9 +150,9 @@ jobs:
148
150
  with: { repeat: '3' }
149
151
  ```
150
152
 
151
- The Action runs straight from its own checkout — no npm install, no build. See [`action.yml`](action.yml).
153
+ The Action runs straight from its own checkout, no npm install, no build. See [`action.yml`](action.yml).
152
154
 
153
- ## Config — `promptwheel.config.json`
155
+ ## Config, `promptwheel.config.json`
154
156
 
155
157
  ```json
156
158
  {
@@ -163,22 +165,22 @@ The Action runs straight from its own checkout — no npm install, no build. See
163
165
  }
164
166
  ```
165
167
 
166
- - **cmd** — any shell command, run inside the worktree.
167
- - **extract** — reduce its output to a number: `number` (last number, default) · `lines` (count non-empty lines) · `exit` (1 if exit 0 else 0) · `{ "regex": "coverage: (\\d+)" }` (first capture).
168
- - **direction** — `up` (higher better) · `down` (lower better) · `pass` (boolean 0/1).
169
- - **guard** — `true` = a *trusted* regression **fails** the gate; `false` = informational.
170
- - **gamingCheck** — `false` exempts a metric from `--detect-gaming`'s source-only re-run. Use it for tripwire / test-side guards (assertion counts, test counts) whose gains legitimately live in test files — otherwise adding real tests would be flagged as gaming. The `antihack` preset sets this on its tripwires.
171
- - **gamingThreshold** — the fraction of a win that must survive the source-only re-run to count as earned (default `0.5`). Config-level scalar, overridable per metric; inherited through `extends` like `repeat`.
168
+ - **cmd**, any shell command, run inside the worktree.
169
+ - **extract**, reduce its output to a number: `number` (last number, default) · `lines` (count non-empty lines) · `exit` (1 if exit 0 else 0) · `{ "regex": "coverage: (\\d+)" }` (first capture).
170
+ - **direction**, `up` (higher better) · `down` (lower better) · `pass` (boolean 0/1).
171
+ - **guard**, `true` = a *trusted* regression **fails** the gate; `false` = informational.
172
+ - **gamingCheck**, `false` exempts a metric from `--detect-gaming`'s source-only re-run. Use it for tripwire / test-side guards (assertion counts, test counts) whose gains legitimately live in test files, otherwise adding real tests would be flagged as gaming. The `antihack` preset sets this on its tripwires.
173
+ - **gamingThreshold**, the fraction of a win that must survive the source-only re-run to count as earned (default `0.5`). Config-level scalar, overridable per metric; inherited through `extends` like `repeat`.
172
174
 
173
175
  Dependency & environment keys (config-level; `init` writes stack defaults):
174
176
 
175
- - **linkDirs** — dirs to symlink from your checkout into each measuring worktree, for deps that live outside git. Default `["node_modules"]`; `init` writes `[".venv"]` for Python, `["target"]` for Rust. (`"linkNodeModules": false` still works as shorthand for "link nothing".)
176
- - **env** — environment variables for every metric command; `{wt}` is substituted with the worktree path. Python configs get `"PYTHONPATH": "{wt}/src:{wt}"` so the *measured ref* is imported, not an editable install of your original checkout.
177
- - **setup** — an optional command run once per ref before metrics (e.g. `npm run build`, `pip install -e .`), after any source patch — so a build-gated suite measures the ref it's actually on.
177
+ - **linkDirs**, dirs to symlink from your checkout into each measuring worktree, for deps that live outside git. Default `["node_modules"]`; `init` writes `[".venv"]` for Python, `["target"]` for Rust. (`"linkNodeModules": false` still works as shorthand for "link nothing".)
178
+ - **env**, environment variables for every metric command; `{wt}` is substituted with the worktree path. Python configs get `"PYTHONPATH": "{wt}/src:{wt}"` so the *measured ref* is imported, not an editable install of your original checkout.
179
+ - **setup**, an optional command run once per ref before metrics (e.g. `npm run build`, `pip install -e .`), after any source patch, so a build-gated suite measures the ref it's actually on.
178
180
 
179
181
  ## Guardrails & inheritance
180
182
 
181
- To see what's actually enforced in a repo — including guards inherited from a shared config:
183
+ To see what's actually enforced in a repo, including guards inherited from a shared config:
182
184
 
183
185
  ```bash
184
186
  promptwheel guards
@@ -192,40 +194,40 @@ Teams keep one shared base config and have every repo inherit it via `extends`:
192
194
  "metrics": [ { "name": "cost_per_run_usd", "guard": false } ] } // loosen one inherited guard, locally
193
195
  ```
194
196
 
195
- `extends` takes a path (or array of paths) to base configs: a repo **inherits** their guardrails, and a local metric of the same name **overrides** the inherited one (tighten, loosen, or disable). `promptwheel guards` shows the effective set with provenance — `inherited ← base.json`, `local`, or `local override` — plus each guard's flag record from the outcome stream.
197
+ `extends` takes a path (or array of paths) to base configs: a repo **inherits** their guardrails, and a local metric of the same name **overrides** the inherited one (tighten, loosen, or disable). `promptwheel guards` shows the effective set with provenance, `inherited ← base.json`, `local`, or `local override`, plus each guard's flag record from the outcome stream.
196
198
 
197
- Read this way, `extends` is **the shared invariants — the "business rules" — your agents inherit and are held to**: enforced as *measured guards* (a trusted regression fails the gate / reverts the commit), not advisory text an agent can quietly ignore. (Natural-language conventions belong in `AGENTS.md`; only deterministic, measurable guards belong here.)
199
+ Read this way, `extends` is **the shared invariants, the "business rules", your agents inherit and are held to**: enforced as *measured guards* (a trusted regression fails the gate / reverts the commit), not advisory text an agent can quietly ignore. (Natural-language conventions belong in `AGENTS.md`; only deterministic, measurable guards belong here.)
198
200
 
199
- ## Trust model — the point of the whole thing
201
+ ## Trust model, the point of the whole thing
200
202
 
201
203
  A number that jumps around between runs is worthless as a signal. PromptWheel won't pretend otherwise:
202
204
 
203
205
  - `--repeat N` measures each metric N times at both refs and uses the **median**; the **noise band** is the observed spread.
204
206
  - A delta **inside the noise band** is reported `inconclusive` with `low` confidence and **does not fail a guard** (no flaky CI failures).
205
- - **Confidence:** `high` (deterministic extract, or zero observed noise) · `medium` (delta clears the noise band) · `low` (delta inside noise) · `unverified` (single read — run `--repeat` to earn trust).
207
+ - **Confidence:** `high` (deterministic extract, or zero observed noise) · `medium` (delta clears the noise band) · `low` (delta inside noise) · `unverified` (single read, run `--repeat` to earn trust).
206
208
 
207
209
  The accumulated record of **which change-types move which metrics** is the asset: a per-repo reward signal a base tool can't replicate, and the spine that lets an agent loop learn what actually helps.
208
210
 
209
211
  ## Where PromptWheel fits (and where it doesn't)
210
212
 
211
- - **vs single-axis CI gates** (Codspeed, Bencher, size-limit, Lighthouse-CI): they own deep statistics on *one* metric; PromptWheel is the **cross-metric gate that composes them** — "did `eval_pass_rate` **and** cost improve without regressing the guards?" in one verdict. Wrap any of them as a metric `cmd` and let `--repeat` handle the noise.
212
- - **vs loop/agent frameworks** (Ralph, GEPA, reward models): PromptWheel is the **execution-grounded reward they lack** — it runs your real suite with zero deps; it does not drive the loop or do test-time search.
213
+ - **vs single-axis CI gates** (Codspeed, Bencher, size-limit, Lighthouse-CI): they own deep statistics on *one* metric; PromptWheel is the **cross-metric gate that composes them**, "did `eval_pass_rate` **and** cost improve without regressing the guards?" in one verdict. Wrap any of them as a metric `cmd` and let `--repeat` handle the noise.
214
+ - **vs loop/agent frameworks** (Ralph, GEPA, reward models): PromptWheel is the **execution-grounded reward they lack**, it runs your real suite with zero deps; it does not drive the loop or do test-time search.
213
215
  - **When NOT to use it:** if you only care that tests pass, your base verifier already has you covered. PromptWheel earns its place when you have a **graded numeric metric** beyond pass/fail (eval score, $/run, latency, size) that a change could quietly move.
214
216
 
215
217
  ## Docs
216
218
 
217
- - [docs/DETECTION-LAYERS.md](docs/DETECTION-LAYERS.md) — how `--detect-gaming` fits as the **deterministic layer** alongside held-out tests, LLM judges, and human review: the coverage matrix, the compose-as-a-pipeline model, and the honest in/out-of-scope boundary.
218
- - [docs/ENFORCEMENT.md](docs/ENFORCEMENT.md) — making "the agent can't skip it" real: the three places to enforce (loop-revert · CI + branch protection · a tested Claude Code Stop-hook), the exact wiring, and the *protect-the-gate's-own-config* caveat.
219
- - [docs/VISION.md](docs/VISION.md) — why we pivoted from orchestrator to outcome gate, the thesis, the moat, the open-core model.
220
- - [docs/ROADMAP.md](docs/ROADMAP.md) — the phased plan and the ship-now/stay-thin guardrails.
221
- - [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — how the engine works: schemas, extract modes, the trust/noise model.
222
- - [docs/LEARNING.md](docs/LEARNING.md) — the (research-gated) Phase-5 design: outcome-curated playbook (Agentic Context Engineering, Stanford 2510.04618) + UCB work-discovery.
223
- - [CLAUDE.md](CLAUDE.md) — the constitution for anyone (human or agent) working in this repo.
219
+ - [docs/DETECTION-LAYERS.md](docs/DETECTION-LAYERS.md), how `--detect-gaming` fits as the **deterministic layer** alongside held-out tests, LLM judges, and human review: the coverage matrix, the compose-as-a-pipeline model, and the honest in/out-of-scope boundary.
220
+ - [docs/ENFORCEMENT.md](docs/ENFORCEMENT.md), making "the agent can't skip it" real: the three places to enforce (loop-revert · CI + branch protection · a tested Claude Code Stop-hook), the exact wiring, and the *protect-the-gate's-own-config* caveat.
221
+ - [docs/VISION.md](docs/VISION.md), why we pivoted from orchestrator to outcome gate, the thesis, the moat, the open-core model.
222
+ - [docs/ROADMAP.md](docs/ROADMAP.md), the phased plan and the ship-now/stay-thin guardrails.
223
+ - [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md), how the engine works: schemas, extract modes, the trust/noise model.
224
+ - [docs/LEARNING.md](docs/LEARNING.md), the (research-gated) Phase-5 design: outcome-curated playbook (Agentic Context Engineering, Stanford 2510.04618) + UCB work-discovery.
225
+ - [CLAUDE.md](CLAUDE.md), the constitution for anyone (human or agent) working in this repo.
224
226
 
225
227
  ## Develop
226
228
 
227
229
  ```bash
228
- npm test # the dep-free suite (node:test) — unit + integration, no dependencies
230
+ npm test # the dep-free suite (node:test), unit + integration, no dependencies
229
231
  ```
230
232
 
231
233
  The engine is one importable file; pure helpers are exported for unit tests, the CLI runs only when invoked directly. Add a test with every behavior change.
@@ -234,15 +236,15 @@ The engine is one importable file; pure helpers are exported for unit tests, the
234
236
 
235
237
  - [x] before/after worktree measurement + regression guards
236
238
  - [x] noise band + confidence (don't trust a delta inside the jitter)
237
- - [x] `--working` mode — measure uncommitted changes (tracked **and** untracked)
238
- - [x] persisted reward stream (`.promptwheel/outcomes.jsonl`) — the compounding "what moves what" record
239
+ - [x] `--working` mode, measure uncommitted changes (tracked **and** untracked)
240
+ - [x] persisted reward stream (`.promptwheel/outcomes.jsonl`), the compounding "what moves what" record
239
241
  - [x] GitHub Action / PR-comment wrapper (open-core distribution surface)
240
- - [x] agent loop: `improve` — propose → gate → keep only if a metric improved
242
+ - [x] agent loop: `improve`, propose → gate → keep only if a metric improved
241
243
  - [x] loop-consumable `improve`: exit `0` kept / `1` regression / `3` plateau + `--json result`
242
- - [x] `promptwheel init` + presets — zero-config onboarding
243
- - [x] `insights` — reward-stream aggregation (Phase-5 seed)
244
- - [x] `--detect-gaming` — reward-hack detection: re-prove the win from source edits alone + `antihack` preset
245
- - [x] npm publish — `promptwheel@0.4.2` (the lead magnet)
246
- - [x] outcome-curated learning + UCB work-discovery — `playbook` + `suggest` + the compounding A/B harness (**experimental**; compounding *claims* stay gated on real-data proof — see [docs/LEARNING.md](docs/LEARNING.md))
244
+ - [x] `promptwheel init` + presets, zero-config onboarding
245
+ - [x] `insights`, reward-stream aggregation (Phase-5 seed)
246
+ - [x] `--detect-gaming`, reward-hack detection: re-prove the win from source edits alone + `antihack` preset
247
+ - [x] npm publish, `promptwheel@0.4.2` (the lead magnet)
248
+ - [x] outcome-curated learning + UCB work-discovery, `playbook` + `suggest` + the compounding A/B harness (**experimental**; compounding *claims* stay gated on real-data proof, see [docs/LEARNING.md](docs/LEARNING.md))
247
249
 
248
- > Status: **published** (npm `promptwheel`, v0.4.2) — all core phases built. Lineage: CommandLayer → BlockSpool → PromptWheel (orchestrator, archived) → **PromptWheel (outcome gate)**.
250
+ > Status: **published** (npm `promptwheel`, v0.4.2), all core phases built. Lineage: CommandLayer → BlockSpool → PromptWheel (orchestrator, archived) → **PromptWheel (outcome gate)**.
package/bin/promptwheel.mjs CHANGED
@@ -1,11 +1,11 @@
1
1
  #!/usr/bin/env node
2
- // PromptWheel — the per-turn reward for AI coding loops (a.k.a. the outcome gate for AI code).
2
+ // PromptWheel, the per-turn reward for AI coding loops (a.k.a. the outcome gate for AI code).
3
3
  //
4
4
  // For any change (base ref → head ref), measure each configured metric in an
5
5
  // isolated git worktree BEFORE and AFTER, enforce regression guards, and refuse
6
6
  // to trust a delta that sits inside the measurement noise band. Emits a
7
7
  // structured {verdict, metric, delta, confidence}. The thing base agents don't do:
8
- // verify a real number moved — not just "the diff applied / tests passed".
8
+ // verify a real number moved, not just "the diff applied / tests passed".
9
9
  //
10
10
  // Zero deps, zero build. Node 18+.
11
11
  // promptwheel run [--base R] [--head R] [--repeat N] [--json]
@@ -59,14 +59,14 @@ function resolveConfig(p, repo, label, seen) {
59
59
  for (const k of ['repeat', 'linkNodeModules', 'linkDirs', 'env', 'setup', 'record', 'gamingThreshold']) if (cfg[k] !== undefined) scalars[k] = cfg[k];
60
60
  return { ...scalars, metrics: [...byName.values()] };
61
61
  } finally {
62
- seen.delete(abs); // only ACTIVE ancestors are a cycle — a diamond (a base reached two ways) is fine
62
+ seen.delete(abs); // only ACTIVE ancestors are a cycle, a diamond (a base reached two ways) is fine
63
63
  }
64
64
  }
65
65
 
66
66
  function loadConfig(repo) {
67
67
  const candidates = ['promptwheel.config.json', 'outcome-gate.config.json']; // back-compat alias
68
68
  const p = candidates.map((c) => join(repo, c)).find(existsSync);
69
- if (!p) { console.error('no promptwheel.config.json — run: promptwheel init (writes one for your stack)'); process.exit(2); }
69
+ if (!p) { console.error('no promptwheel.config.json, run: promptwheel init (writes one for your stack)'); process.exit(2); }
70
70
  const cfg = resolveConfig(p, repo, 'local', new Set());
71
71
  if (!Array.isArray(cfg.metrics) || cfg.metrics.length === 0) {
72
72
  console.error('config.metrics must be a non-empty array'); process.exit(2);
@@ -84,15 +84,19 @@ function resolveBase(repo, base) {
84
84
 
85
85
  function runMetric(cwd, m, env) {
86
86
  let stdout = '', code = 0;
87
+ // Never leak the harness's own node:test context into the measured gate: a metric like
88
+ // `node --test` would see NODE_TEST_CONTEXT, act as a child reporter, and report a bogus
89
+ // exit code (so the gate can't be trusted when PromptWheel itself runs under a test runner).
90
+ const childEnv = { ...(env || process.env) }; delete childEnv.NODE_TEST_CONTEXT;
87
91
  try {
88
- stdout = execSync(m.cmd, { cwd, env: env || process.env, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], timeout: (m.timeoutSec ?? 300) * 1000 });
92
+ stdout = execSync(m.cmd, { cwd, env: childEnv, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], timeout: (m.timeoutSec ?? 300) * 1000 });
89
93
  } catch (e) { code = e.status ?? 1; stdout = `${e.stdout || ''}${e.stderr || ''}`; }
90
94
  return extract(stdout, code, m.extract, m.allowNonzero);
91
95
  }
92
96
 
93
97
  // extract modes: 'number' (last number, default) | 'lines' | 'exit' | {regex}
94
98
  // A FAILED command is unmeasurable (null -> the metric goes inert, guards
95
- // fail loudly) — error text often contains numbers ("Node.js v22") and
99
+ // fail loudly), error text often contains numbers ("Node.js v22") and
96
100
  // silently measuring those produced identical junk on both refs = a false
97
101
  // green. 'exit' mode consumes nonzero by design; anything else needs the
98
102
  // per-metric opt-in `allowNonzero: true`.
@@ -114,7 +118,7 @@ function extract(stdout, code, mode, allowNonzero) {
114
118
  // node_modules-only symlink: `linkDirs` says what to link (default ['node_modules']; back-compat
115
119
  // with linkNodeModules:false → link nothing); `env` sets vars ({wt} → the worktree path, so a
116
120
  // Python run can put the measured ref FIRST on PYTHONPATH). Ecosystem knowledge lives in config,
117
- // not here — the engine stays ecosystem-agnostic.
121
+ // not here, the engine stays ecosystem-agnostic.
118
122
  function bridgeEnv(repo, wt, cfg) {
119
123
  const linkDirs = Array.isArray(cfg.linkDirs) ? cfg.linkDirs : (cfg.linkNodeModules === false ? [] : ['node_modules']);
120
124
  for (const d of linkDirs) {
@@ -165,9 +169,9 @@ function evaluate(m, beforeS, afterS, repeat) {
165
169
 
166
170
  let confidence;
167
171
  if (deterministic) confidence = 'high';
168
- else if (!sampledNoise) confidence = 'unverified'; // single read — noise unknown; run --repeat to trust
172
+ else if (!sampledNoise) confidence = 'unverified'; // single read, noise unknown; run --repeat to trust
169
173
  else if (noise === 0) confidence = 'high';
170
- else if (withinNoise) confidence = 'low'; // delta is inside the jitter — not real
174
+ else if (withinNoise) confidence = 'low'; // delta is inside the jitter, not real
171
175
  else confidence = 'medium';
172
176
 
173
177
  let improved = false, regressed = false;
@@ -182,7 +186,7 @@ function evaluate(m, beforeS, afterS, repeat) {
182
186
  // guards fail only on a TRUSTED regression (within-noise regressions are not failures)
183
187
  const ok = m.guard ? !regressed : true;
184
188
  // a guarded pass-metric that never passes is protecting NOTHING (broken test cmd,
185
- // missing script, failed install) — surface it instead of folding into a green verdict.
189
+ // missing script, failed install), surface it instead of folding into a green verdict.
186
190
  const inert = !!m.guard && dir === 'pass' && before === 0 && after === 0;
187
191
  return { before, after, delta, status, ok, confidence, noise, ...(inert ? { inert: true } : {}) };
188
192
  }
@@ -192,7 +196,7 @@ function evaluate(m, beforeS, afterS, repeat) {
192
196
  // If the improvement evaporates once test/config/grader/golden changes are reverted,
193
197
  // the agent moved the goalposts (edited the test, mocked the grader, suppressed the
194
198
  // lint rule, deleted the feature) instead of earning it. Pure arithmetic, fully
195
- // explainable — the thing the loop-owner structurally won't ship about its own agent.
199
+ // explainable, the thing the loop-owner structurally won't ship about its own agent.
196
200
  // ---------------------------------------------------------------------------
197
201
  const NON_SOURCE = [
198
202
  /(^|\/)(tests?|__tests?__|spec|e2e|fixtures|snapshots|__snapshots__|__mocks__)\//i,
@@ -201,7 +205,7 @@ const NON_SOURCE = [
201
205
  /(^|\/)test_[^/]*\.py$/i, /_test\.py$/i, /(^|\/)conftest\.py$/i,
202
206
  /\.snap$/i, /(^|\/)golden[^/]*$/i, /\.golden$/i,
203
207
  /(^|\/)(eval|grader|score)[^/]*\.[cm]?[jt]s$/i,
204
- // config files only — require a .config/.conf/.setup segment so PRODUCTION source that
208
+ // config files only, require a .config/.conf/.setup segment so PRODUCTION source that
205
209
  // happens to be named after a tool (e.g. src/installers/eslint.ts) is NOT swept out of
206
210
  // the source slice (a false sweep here can flag an honest win as GAMED).
207
211
  /(^|\/)(jest|vitest|playwright|cypress|karma|babel|eslint|pytest)[^/]*\.(config|conf|setup)\.[^/]+$/i,
@@ -210,7 +214,7 @@ const NON_SOURCE = [
210
214
  /(^|\/)(pytest\.ini|setup\.cfg|\.flake8|tox\.ini|eslint\.config\.[cm]?js)$/i,
211
215
  // promptwheel's OWN evaluator surface: the config that defines the gate
212
216
  // and any pack copied into the repo by `init --preset`. Editing these is
213
- // moving the goalposts, never a production improvement — an agent that
217
+ // moving the goalposts, never a production improvement, an agent that
214
218
  // "improved" security_findings by deleting pattern rules must show 0%
215
219
  // retained in the source-only re-run, not 100%.
216
220
  /(^|\/)promptwheel\.config\.json$/i,
@@ -269,26 +273,26 @@ function gain(m, before, val) {
269
273
  }
270
274
 
271
275
  // a metric that improved full-diff: does the win survive when only source edits are applied?
272
- // gamed = source edits alone reproduce less than `threshold` of the gain (default 0.5 — the
276
+ // gamed = source edits alone reproduce less than `threshold` of the gain (default 0.5, the
273
277
  // rest came from editing the goalposts). Tune via `gamingThreshold` (config-level or per-metric).
274
278
  function judgeGaming(m, srcResult, threshold = 0.5) {
275
- if (!srcResult.hadSourceChange) return { gamed: true, sourceOnly: null, retained: 0, reason: 'the "win" changed zero production-source files — only test/config/grader/golden' };
276
- if (srcResult.applyFailed) return { gamed: null, sourceOnly: null, retained: null, reason: 'source-only patch did not apply cleanly — inconclusive' };
279
+ if (!srcResult.hadSourceChange) return { gamed: true, sourceOnly: null, retained: 0, reason: 'the "win" changed zero production-source files, only test/config/grader/golden' };
280
+ if (srcResult.applyFailed) return { gamed: null, sourceOnly: null, retained: null, reason: 'source-only patch did not apply cleanly, inconclusive' };
277
281
  const sourceOnly = median(srcResult.samples);
278
282
  const full = gain(m, m.before, m.after);
279
283
  if (full == null || full <= 0) return { gamed: false, sourceOnly, retained: 1, reason: 'no real improvement to re-prove' };
280
284
  const retained = +(gain(m, m.before, sourceOnly) / full).toFixed(3);
281
285
  const gamed = retained < threshold;
282
286
  return { gamed, sourceOnly, retained, reason: gamed
283
- ? `only ${(retained * 100).toFixed(0)}% of the gain survives when test/config/grader changes are reverted (threshold ${(threshold * 100).toFixed(0)}%) — most of the "win" came from editing the goalposts`
284
- : `${(retained * 100).toFixed(0)}% of the gain survives source-only — the source earned it` };
287
+ ? `only ${(retained * 100).toFixed(0)}% of the gain survives when test/config/grader changes are reverted (threshold ${(threshold * 100).toFixed(0)}%), most of the "win" came from editing the goalposts`
288
+ : `${(retained * 100).toFixed(0)}% of the gain survives source-only, the source earned it` };
285
289
  }
286
290
 
287
291
  // ---------------------------------------------------------------------------
288
292
  // main
289
293
  // ---------------------------------------------------------------------------
290
294
  // snapshot the working tree (tracked + untracked) into a dangling commit WITHOUT
291
- // touching the real index or files — so a gate/improve sees a file an agent just added.
295
+ // touching the real index or files, so a gate/improve sees a file an agent just added.
292
296
  // Returns 'HEAD' when nothing changed.
293
297
  function workingSnapshot(repo) {
294
298
  const idxDir = mkdtempSync(join(tmpdir(), 'pw-idx-'));
@@ -313,7 +317,7 @@ function workingSnapshot(repo) {
313
317
  }
314
318
  }
315
319
 
316
- // self-heal: a hard-killed run (SIGKILL / power loss) can leave an orphaned worktree —
320
+ // self-heal: a hard-killed run (SIGKILL / power loss) can leave an orphaned worktree,
317
321
  // its `finally` never ran. On the next run, drop stale worktree registry entries and any
318
322
  // abandoned /tmp checkout, so nothing accumulates in your repo or temp dir. Never touches a
319
323
  // live worktree (a still-registered dir, or a checkout younger than 1h).
@@ -328,7 +332,7 @@ function selfHeal(repo) {
328
332
  for (const name of readdirSync(tmp)) {
329
333
  if (!name.startsWith('promptwheel-') && !name.startsWith('pw-idx-')) continue; // worktrees + temp-index dirs
330
334
  const p = join(tmp, name);
331
- if (tracked.has(p)) continue; // a live/registered worktree — leave it
335
+ if (tracked.has(p)) continue; // a live/registered worktree, leave it
332
336
  try { if (statSync(p).mtimeMs < cutoff) rmSync(p, { recursive: true, force: true }); } catch { /* */ }
333
337
  }
334
338
  } catch { /* cleanup must never block the gate */ }
@@ -336,7 +340,7 @@ function selfHeal(repo) {
336
340
 
337
341
  // Python editable installs (src-layout `pip install -e .`) can pin imports to the ORIGINAL
338
342
  // checkout: the worktree's tests then import your current tree at BOTH refs and every delta
339
- // reads 0 — the gate is structurally blind on that repo shape. Undetectable from the verdict
343
+ // reads 0, the gate is structurally blind on that repo shape. Undetectable from the verdict
340
344
  // (it looks like "unchanged"), so detect the editable install itself and warn. Verified by a
341
345
  // controlled repro (CHANGELOG 0.2.2).
342
346
  function warnEditableInstall(repo) {
@@ -351,8 +355,8 @@ function warnEditableInstall(repo) {
351
355
  'print("HIT" if hit else "")',
352
356
  ].join('\n');
353
357
  const out = execFileSync('python3', ['-c', probe, repo], { encoding: 'utf8', timeout: 5000 }).trim();
354
- if (out === 'HIT') console.error('⚠ editable install of this repo detected (pip install -e): worktree measurements may import your ORIGINAL checkout, not the measured ref — deltas can read 0. Use a non-editable install in the measuring venv, or put the worktree src on PYTHONPATH.');
355
- } catch { /* no python / cannot tell — stay quiet */ }
358
+ if (out === 'HIT') console.error('⚠ editable install of this repo detected (pip install -e): worktree measurements may import your ORIGINAL checkout, not the measured ref, deltas can read 0. Use a non-editable install in the measuring venv, or put the worktree src on PYTHONPATH.');
359
+ } catch { /* no python / cannot tell, stay quiet */ }
356
360
  }
357
361
 
358
362
  // the shared core: measure a change (base→head) and return the structured report
@@ -364,11 +368,11 @@ function gate(repo, opts) {
364
368
 
365
369
  let base, head;
366
370
  if (opts.working) {
367
- // measure uncommitted changes — tracked AND untracked — via a temp-index snapshot;
371
+ // measure uncommitted changes, tracked AND untracked, via a temp-index snapshot;
368
372
  // never touches the real index or working tree. (A loop agent's most common action
369
373
  // is to ADD a file; `git stash create` omits untracked, which silently reverted them.)
370
374
  try { git(['rev-parse', '--verify', 'HEAD'], repo); }
371
- catch { console.error('no commits yet — make an initial commit before gating --working changes'); process.exit(2); }
375
+ catch { console.error('no commits yet, make an initial commit before gating --working changes'); process.exit(2); }
372
376
  base = 'HEAD';
373
377
  head = workingSnapshot(repo);
374
378
  } else {
@@ -395,7 +399,7 @@ function gate(repo, opts) {
395
399
  const failed = metrics.some((m) => m.guard && !m.ok);
396
400
  const gamed = metrics.some((m) => m.gamed === true);
397
401
  // a guard that never runs (inert: a pass/fail metric stuck at 0 across both refs) verifies
398
- // nothing — don't launder that into a green PASS. The honest verdict is "couldn't measure".
402
+ // nothing, don't launder that into a green PASS. The honest verdict is "couldn't measure".
399
403
  // BUT scope it to a bare no-op run: an inert guard must NOT bury a genuine, measured win
400
404
  // elsewhere (that would make `improve` revert good changes and misread them as a plateau).
401
405
  const inconclusive = !failed && !gamed && metrics.some((m) => m.inert) && !metrics.some((m) => m.status === 'improved');
@@ -438,11 +442,11 @@ function improve(argv) {
438
442
  if (!args.attempt) { console.error('improve requires --attempt "<command that changes the repo>"'); process.exit(2); }
439
443
  const repo = git(['rev-parse', '--show-toplevel'], process.cwd());
440
444
  const dirty = git(['status', '--porcelain'], repo).split('\n').filter((l) => l.trim() && !l.includes('.promptwheel'));
441
- if (dirty.length) { console.error('working tree not clean — commit or stash first (improve needs a clean base to revert to)'); process.exit(2); }
445
+ if (dirty.length) { console.error('working tree not clean, commit or stash first (improve needs a clean base to revert to)'); process.exit(2); }
442
446
 
443
447
  console.error(`▶ attempt: ${args.attempt}`);
444
448
  try { execSync(args.attempt, { cwd: repo, stdio: 'inherit' }); }
445
- catch (e) { console.error(` (attempt exited ${e.status ?? 1} — gating whatever it changed)`); }
449
+ catch (e) { console.error(` (attempt exited ${e.status ?? 1}, gating whatever it changed)`); }
446
450
 
447
451
  const report = gate(repo, { working: true, repeat: args.repeat, noRecord: args.noRecord, detectGaming: args.detectGaming, label: args.label ?? args.attempt });
448
452
  const noChange = report.metrics.every((m) => m.delta === 0 || m.delta == null);
@@ -451,17 +455,17 @@ function improve(argv) {
451
455
  // result + exit code express loop progress so `while improve; do :; done` converges:
452
456
  // 0 = kept a real win · 1 = guarded regression (reverted) · 3 = plateau/no-op OR inconclusive (reverted)
453
457
  let result, exit, note;
454
- if (report.verdict === 'fail') { result = 'regression'; exit = 1; revert(repo); note = '✗ guarded regression — reverted'; }
455
- else if (report.verdict === 'gamed') { result = 'gamed'; exit = 1; revert(repo); note = '🚩 gamed — a metric "improved" by editing tests/config, not source — reverted'; }
456
- else if (report.verdict === 'inconclusive') { result = 'inconclusive'; exit = 3; revert(repo); note = '≈ inconclusive — a guard measured nothing (inert); reverted'; }
458
+ if (report.verdict === 'fail') { result = 'regression'; exit = 1; revert(repo); note = '✗ guarded regression, reverted'; }
459
+ else if (report.verdict === 'gamed') { result = 'gamed'; exit = 1; revert(repo); note = '🚩 gamed, a metric "improved" by editing tests/config, not source, reverted'; }
460
+ else if (report.verdict === 'inconclusive') { result = 'inconclusive'; exit = 3; revert(repo); note = '≈ inconclusive, a guard measured nothing (inert); reverted'; }
457
461
  else if (noChange || improvedNames.length === 0) {
458
462
  result = 'plateau'; exit = 3; revert(repo);
459
- note = noChange ? '= no metric moved — reverted' : '= nothing improved beyond noise — reverted';
463
+ note = noChange ? '= no metric moved, reverted' : '= nothing improved beyond noise, reverted';
460
464
  } else {
461
465
  git(['add', '-A'], repo);
462
466
  execFileSync('git', ['commit', '-qm', `promptwheel: ${args.attempt} [improved ${improvedNames.join(', ')}]`], { cwd: repo });
463
467
  result = 'kept'; exit = 0;
464
- note = `✓ kept — committed ${git(['rev-parse', '--short', 'HEAD'], repo)} (improved ${improvedNames.join(', ')})`;
468
+ note = `✓ kept, committed ${git(['rev-parse', '--short', 'HEAD'], repo)} (improved ${improvedNames.join(', ')})`;
465
469
  }
466
470
 
467
471
  // stdout carries the value (JSON report or the human table); the decision line goes to
@@ -478,14 +482,14 @@ function revert(repo) {
478
482
  try { git(['clean', '-fd', '-e', '.promptwheel'], repo); } catch { /* best effort */ }
479
483
  }
480
484
 
481
- // Phase-5 seed: turn the accumulated reward stream into signal. Thin on purpose —
485
+ // Phase-5 seed: turn the accumulated reward stream into signal. Thin on purpose,
482
486
  // this is the substrate a future outcome-curated playbook / UCB work-discovery loop trains on,
483
487
  // NOT that loop itself. Just honest aggregation, no model.
484
488
  function insights(argv) {
485
489
  const args = parseArgs(argv);
486
490
  const repo = git(['rev-parse', '--show-toplevel'], process.cwd());
487
491
  const f = join(repo, '.promptwheel', 'outcomes.jsonl');
488
- if (!existsSync(f)) { console.error('no outcome record yet — run the gate a few times first (.promptwheel/outcomes.jsonl)'); process.exit(2); }
492
+ if (!existsSync(f)) { console.error('no outcome record yet, run the gate a few times first (.promptwheel/outcomes.jsonl)'); process.exit(2); }
489
493
  const runs = readFileSync(f, 'utf8').split('\n').filter(Boolean).map((l) => { try { return JSON.parse(l); } catch { return null; } }).filter(Boolean);
490
494
  const agg = {};
491
495
  for (const r of runs) for (const m of (r.metrics || [])) {
@@ -498,27 +502,27 @@ function insights(argv) {
498
502
  const rows = Object.entries(agg).map(([name, a]) => ({ name, ...a, lever: a.runs ? a.improved / a.runs : 0 }))
499
503
  .sort((x, y) => y.lever - x.lever);
500
504
  if (args.json) { console.log(JSON.stringify({ runs: runs.length, metrics: rows }, null, 2)); return; }
501
- console.log(`\nPromptWheel insights — ${runs.length} gated runs recorded\n`);
505
+ console.log(`\nPromptWheel insights, ${runs.length} gated runs recorded\n`);
502
506
  console.log(` ${'metric'.padEnd(18)} ${'runs'.padStart(5)} ${'impr'.padStart(5)} ${'regr'.padStart(5)} ${'inconc'.padStart(6)} ${'net Δ'.padStart(9)} lever`);
503
507
  for (const r of rows) {
504
508
  console.log(` ${r.name.padEnd(18)} ${String(r.runs).padStart(5)} ${String(r.improved).padStart(5)} ${String(r.regressed).padStart(5)} ${String(r.inconclusive).padStart(6)} ${String(r.net).padStart(9)} ${(r.lever * 100).toFixed(0)}%`);
505
509
  }
506
- console.log('\n lever = improved/runs — how reliably this metric actually responds. The');
510
+ console.log('\n lever = improved/runs, how reliably this metric actually responds. The');
507
511
  console.log(' highest-lever metrics are where an agent loop should spend its attempts.\n');
508
512
  }
509
513
 
510
514
  // ---------------------------------------------------------------------------
511
- // Phase 5 — outcome-curated playbook + UCB work-discovery.
515
+ // Phase 5, outcome-curated playbook + UCB work-discovery.
512
516
  // Unfrozen 2026-07-02 by explicit founder decision overriding D7 (see docs/LEARNING.md).
513
517
  // Design constraints that survive the unfreeze:
514
518
  // - PURE VIEW: the append-only ledger (.promptwheel/outcomes.jsonl) is the only store;
515
- // the playbook is re-derived on every read (decay at read time — no curator state to rot).
519
+ // the playbook is re-derived on every read (decay at read time, no curator state to rot).
516
520
  // - EVIDENCE-GATED: a key renders a claim only past MIN_MOVED weighted observations;
517
521
  // everything below that is counted, not asserted.
518
522
  // - CLAIM-GATED: no public compounding claim until bench/compounding-ab.mjs passes on
519
523
  // real usage data. The gate on the CLAIM outlived the gate on the CODE.
520
524
  // ---------------------------------------------------------------------------
521
- const HALF_LIFE = 20; // runs — an entry's weight halves every HALF_LIFE runs unless re-earned
525
+ const HALF_LIFE = 20; // runs, an entry's weight halves every HALF_LIFE runs unless re-earned
522
526
  const MIN_MOVED = 3; // weighted moved-observations before a key earns a rendered claim
523
527
  const EXPLORE_C = 1.0; // UCB exploration constant
524
528
 
@@ -576,7 +580,7 @@ function cohortNote(k) {
576
580
 
577
581
  function loadEntries(repo) {
578
582
  const runs = readLedger(repo);
579
- if (!runs.length) { console.error('no outcome record yet — run the gate a few times first (.promptwheel/outcomes.jsonl)'); process.exit(2); }
583
+ if (!runs.length) { console.error('no outcome record yet, run the gate a few times first (.promptwheel/outcomes.jsonl)'); process.exit(2); }
580
584
  const keys = foldOutcomes(runs);
581
585
  const totalW = keys.filter((k) => k.key === k.metric).reduce((s, k) => s + k.w, 0);
582
586
  const entries = keys.map((k) => ({ ...k, ...scoreKey(k, totalW), sufficient: k.moved >= MIN_MOVED }))
@@ -593,38 +597,38 @@ function playbook(argv) {
593
597
  const suff = entries.filter((e) => e.sufficient);
594
598
  const pct = (x) => `${Math.round(x * 100)}%`;
595
599
  const out = [];
596
- out.push(`## Earned playbook — ${runs.length} gated runs, decay half-life ${HALF_LIFE} runs`);
600
+ out.push(`## Earned playbook, ${runs.length} gated runs, decay half-life ${HALF_LIFE} runs`);
597
601
  out.push('*(every line below was measured by the gate, not asserted; entries decay unless re-earned)*', '');
598
602
  if (!suff.length) {
599
- out.push(`_Nothing has earned a claim yet — a key needs ≥${MIN_MOVED} weighted moved-observations. Keep gating; ${entries.length} keys are accumulating._`);
603
+ out.push(`_Nothing has earned a claim yet, a key needs ≥${MIN_MOVED} weighted moved-observations. Keep gating; ${entries.length} keys are accumulating._`);
600
604
  } else {
601
605
  for (const e of suff.slice(0, 20)) {
602
- out.push(`- **${e.key}** — helpful ${pct(e.p)} when it moves (${e.imp.toFixed(1)}✓/${e.reg.toFixed(1)}✗ weighted), responds in ${pct(e.move)} of runs, median effect ${e.effect}${cohortNote(e)}`);
606
+ out.push(`- **${e.key}**, helpful ${pct(e.p)} when it moves (${e.imp.toFixed(1)}✓/${e.reg.toFixed(1)}✗ weighted), responds in ${pct(e.move)} of runs, median effect ${e.effect}${cohortNote(e)}`);
603
607
  }
604
608
  const hidden = entries.length - suff.length;
605
- if (hidden > 0) out.push('', `_${hidden} more keys below the evidence threshold — counted, not asserted._`);
609
+ if (hidden > 0) out.push('', `_${hidden} more keys below the evidence threshold, counted, not asserted._`);
606
610
  }
607
611
  console.log(out.join('\n'));
608
612
  }
609
613
 
610
614
  // UCB work-discovery: where should the loop spend its NEXT attempt? Proposes measured
611
- // targets (metric/subsystem arms) — never code advice; the measurement stays the message.
615
+ // targets (metric/subsystem arms), never code advice; the measurement stays the message.
612
616
  function suggest(argv) {
613
617
  const args = parseArgs(argv);
614
618
  const repo = git(['rev-parse', '--show-toplevel'], process.cwd());
615
619
  const { runs, entries } = loadEntries(repo);
616
620
  const arms = entries.filter((e) => e.key === e.metric || e.key.includes(' @ ')).sort((a, b) => b.ucb - a.ucb);
617
621
  if (args.json) { console.log(JSON.stringify({ runs: runs.length, thin: runs.length < 10, arms: arms.slice(0, 10) }, null, 2)); return; }
618
- console.log(`\nPromptWheel suggest — UCB over ${runs.length} gated runs${runs.length < 10 ? ' (thin record: exploration dominates — treat as a coin with opinions)' : ''}\n`);
622
+ console.log(`\nPromptWheel suggest, UCB over ${runs.length} gated runs${runs.length < 10 ? ' (thin record: exploration dominates, treat as a coin with opinions)' : ''}\n`);
619
623
  for (const a of arms.slice(0, 5)) {
620
624
  console.log(` ${a.ucb.toFixed(3)} ${a.key.padEnd(28)} lever ${a.lever.toFixed(3)} (helpful ${a.p}, responds ${a.move}) · evidence ${a.moved.toFixed(1)} moved / ${a.n} runs${a.sufficient ? '' : ' · below claim threshold'}`);
621
625
  }
622
- console.log('\n score = lever + exploration bonus — high scores are either proven levers or under-explored arms.\n');
626
+ console.log('\n score = lever + exploration bonus, high scores are either proven levers or under-explored arms.\n');
623
627
  }
624
628
 
625
- // backfill — seed the ledger from git history: replay past commits through the CURRENT
629
+ // backfill, seed the ledger from git history: replay past commits through the CURRENT
626
630
  // metrics (LEARNING.md harvest path 1). Deterministic, no LLM. Rows are cohort-tagged
627
- // 'backfill' — historical human commits are NOT live agent-loop evidence; the cohort
631
+ // 'backfill', historical human commits are NOT live agent-loop evidence; the cohort
628
632
  // machinery segments them and flags disagreement rather than averaging it away. The
629
633
  // conventional-commit type (fix/feat/refactor/…) becomes the change-type label for free.
630
634
  function backfill(argv) {
@@ -636,7 +640,7 @@ function backfill(argv) {
636
640
  if (!commits.length) { console.error('no commits to backfill'); process.exit(2); }
637
641
  const seen = new Set(readLedger(repo).filter((r) => r.cohort === 'backfill').map((r) => r.head));
638
642
  let done = 0, skipped = 0;
639
- console.log(`\nbackfilling ${commits.length} commits through the current metrics (gaming detection ${args.dgExplicit && args.detectGaming ? 'ON' : 'off — pass --detect-gaming to audit history too'})\n`);
643
+ console.log(`\nbackfilling ${commits.length} commits through the current metrics (gaming detection ${args.dgExplicit && args.detectGaming ? 'ON' : 'off, pass --detect-gaming to audit history too'})\n`);
640
644
  for (const c of commits) {
641
645
  try { git(['rev-parse', `${c}~1`], repo); } catch { skipped++; continue; } // root commit
642
646
  if (seen.has(short(repo, c))) { skipped++; continue; } // already recorded
@@ -646,12 +650,12 @@ function backfill(argv) {
646
650
  console.log(` ${report.head} ${report.verdict.toUpperCase().padEnd(6)} ${type ? `#${type}`.padEnd(10) : ''.padEnd(10)} ${subj.slice(0, 56)}`);
647
651
  done++;
648
652
  }
649
- console.log(`\n backfilled ${done}${skipped ? ` (skipped ${skipped}: root or already recorded)` : ''} — old commits that no longer build record as unmeasurable, never faked.`);
653
+ console.log(`\n backfilled ${done}${skipped ? ` (skipped ${skipped}: root or already recorded)` : ''}, old commits that no longer build record as unmeasurable, never faked.`);
650
654
  console.log(' next: promptwheel playbook · promptwheel suggest\n');
651
655
  }
652
656
 
653
657
  // ---------------------------------------------------------------------------
654
- // init — write a starter config so a newcomer isn't staring at a blank page
658
+ // init, write a starter config so a newcomer isn't staring at a blank page
655
659
  // ---------------------------------------------------------------------------
656
660
  const LINT_CMD = 'npx eslint . -f unix 2>/dev/null | grep -c " error " || true';
657
661
  const PRESETS = {
@@ -675,6 +679,10 @@ const PRESETS = {
675
679
  { name: 'suppressions', cmd: 'grep -rIoE "eslint-disable|@ts-(ignore|nocheck)|# ?type: ?ignore|# ?noqa|//nolint|#!?\\[allow\\(" --exclude-dir=node_modules --exclude-dir=.git --exclude-dir=dist --exclude-dir=build --exclude-dir=coverage --exclude-dir=.next --exclude-dir=out --exclude-dir=lib-cov --exclude-dir=__pycache__ --exclude-dir=.pytest_cache --exclude-dir=target --exclude-dir=.venv --exclude-dir=venv --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'down', guard: true, gamingCheck: false },
676
680
  { name: 'assertions', cmd: 'grep -rIoE "expect ?\\(|\\bassert|\\bt\\.(is|deepEqual|throws|truthy|falsy|not|ok) ?\\(" --exclude-dir=node_modules --exclude-dir=.git --exclude-dir=dist --exclude-dir=build --exclude-dir=coverage --exclude-dir=.next --exclude-dir=out --exclude-dir=lib-cov --exclude-dir=__pycache__ --exclude-dir=.pytest_cache --exclude-dir=target --exclude-dir=.venv --exclude-dir=venv --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'up', guard: true, gamingCheck: false },
677
681
  ] },
682
+ 'no-ai-tells': { desc: 'opt-in: fail the gate if the em-dash count rises (an AI writing tell); some projects use them on purpose, so it is not a default',
683
+ metrics: [
684
+ { name: 'em_dashes', cmd: 'grep -rIoe "—" --exclude-dir=node_modules --exclude-dir=.git --exclude-dir=dist --exclude-dir=build --exclude-dir=coverage --exclude-dir=.next --exclude-dir=out --exclude-dir=lib-cov --exclude-dir=__pycache__ --exclude-dir=.pytest_cache --exclude-dir=target --exclude-dir=.venv --exclude-dir=venv --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'down', guard: true, gamingCheck: false },
685
+ ] },
678
686
  };
679
687
 
680
688
  function detectTestCmd(repo) {
@@ -682,26 +690,26 @@ function detectTestCmd(repo) {
682
690
  if (has('go.mod')) return 'go test ./...';
683
691
  if (has('Cargo.toml')) return 'cargo test';
684
692
  if (has('pyproject.toml') || has('setup.py') || has('pytest.ini')) return has('.venv/bin/pytest') ? '.venv/bin/pytest -q' : 'pytest -q';
685
- // only trust `npm test` when a test script actually exists — otherwise the metric can
693
+ // only trust `npm test` when a test script actually exists, otherwise the metric can
686
694
  // never pass and the gate would "protect" nothing (very common in Next.js app repos)
687
695
  try {
688
696
  if (JSON.parse(readFileSync(join(repo, 'package.json'), 'utf8')).scripts?.test) return 'npm test --silent';
689
- } catch { /* no or unparsable package.json — fall through */ }
697
+ } catch { /* no or unparsable package.json, fall through */ }
690
698
  return 'echo "set your test command in promptwheel.config.json" && false';
691
699
  }
692
700
 
693
701
  // deps that live outside git + the env a test needs, defaulted per stack. init writes these into
694
- // config; the measurement bridge (bridgeEnv/runSetup) consumes them — keeps the engine agnostic.
702
+ // config; the measurement bridge (bridgeEnv/runSetup) consumes them, keeps the engine agnostic.
695
703
  function detectEnv(repo) {
696
704
  const has = (f) => existsSync(join(repo, f));
697
705
  // Python: link the venv (3rd-party deps) and put the worktree src FIRST on PYTHONPATH, so imports
698
706
  // resolve the MEASURED ref instead of an editable install pointing back at your original checkout.
699
707
  if (has('pyproject.toml') || has('setup.py') || has('pytest.ini')) return { linkDirs: ['.venv'], env: { PYTHONPATH: '{wt}/src:{wt}' } }; // src/ AND flat layouts; worktree src wins over an editable install
700
- if (has('Cargo.toml')) return { linkDirs: [] }; // do NOT link target/: `cargo test` WRITES there — linking would clobber the user's real dir and share a build cache across refs (measurement blindness). Build fresh in the worktree.
708
+ if (has('Cargo.toml')) return { linkDirs: [] }; // do NOT link target/: `cargo test` WRITES there, linking would clobber the user's real dir and share a build cache across refs (measurement blindness). Build fresh in the worktree.
701
709
  return {};
702
710
  }
703
711
 
704
- // only offer the lint metric where eslint actually exists — otherwise it reads 0 forever
712
+ // only offer the lint metric where eslint actually exists, otherwise it reads 0 forever
705
713
  // and a newcomer mistakes a metric that can't move for a clean repo.
706
714
  function hasEslint(repo) {
707
715
  const files = ['eslint.config.js', 'eslint.config.mjs', 'eslint.config.cjs',
@@ -722,18 +730,18 @@ function init(argv) {
722
730
  const repo = (() => { try { return git(['rev-parse', '--show-toplevel'], process.cwd()); } catch { return process.cwd(); } })();
723
731
  const out = join(repo, 'promptwheel.config.json');
724
732
  if (existsSync(out) && !argv.includes('--force')) {
725
- console.error('promptwheel.config.json already exists — pass --force to overwrite'); process.exit(2);
733
+ console.error('promptwheel.config.json already exists, pass --force to overwrite'); process.exit(2);
726
734
  }
727
735
  const pi = argv.indexOf('--preset');
728
736
  const presetName = pi >= 0 ? argv[pi + 1] : null;
729
737
  let metrics, note;
730
738
  if (presetName) {
731
739
  const p = PRESETS[presetName];
732
- if (!p) { console.error(`unknown preset "${presetName}" — try: promptwheel init --list`); process.exit(2); }
740
+ if (!p) { console.error(`unknown preset "${presetName}", try: promptwheel init --list`); process.exit(2); }
733
741
  metrics = p.metrics || [{ name: 'tests_pass', cmd: detectTestCmd(repo), extract: 'exit', direction: 'pass', guard: true }];
734
742
  metrics = metrics.map((m) => m.cmd === '__TESTCMD__' ? { ...m, cmd: detectTestCmd(repo) } : m); // fill the preset's target
735
743
  // pack-backed presets: the referenced scripts live in THIS package, not
736
- // the user's repo — copy them in so the generated config is
744
+ // the user's repo, copy them in so the generated config is
737
745
  // self-contained and committable (a bare reference silently 404'd)
738
746
  metrics = metrics.map((m) => {
739
747
  const pk = /(^|\s)(packs\/[\w-]+)\//.exec(m.cmd);
@@ -748,7 +756,7 @@ function init(argv) {
748
756
  note = presetName;
749
757
  } else {
750
758
  // default = the headline posture: guarded tests + the antihack tripwires, so the FIRST
751
- // cheat a newcomer tries (gut the tests while the metric stays flat) fails the gate —
759
+ // cheat a newcomer tries (gut the tests while the metric stays flat) fails the gate,
752
760
  // "catch your agent cheating" must be true out of the box, not only under a preset.
753
761
  const testCmd = detectTestCmd(repo);
754
762
  const lint = hasEslint(repo);
@@ -759,7 +767,7 @@ function init(argv) {
759
767
  const envCfg = presetName ? {} : detectEnv(repo);
760
768
  if (envCfg.linkDirs) note += ` · linking ${envCfg.linkDirs.join(', ')}`;
761
769
  writeFileSync(out, JSON.stringify({ repeat: 1, ...envCfg, metrics }, null, 2) + '\n');
762
- console.log(`✓ wrote promptwheel.config.json — ${note}`);
770
+ console.log(`✓ wrote promptwheel.config.json, ${note}`);
763
771
  console.log('\n next: promptwheel run --working # gate your uncommitted changes');
764
772
  console.log(' promptwheel init --list # other presets (llm-eval, bundle-size, …)');
765
773
  }
@@ -783,7 +791,7 @@ function guards(argv) {
783
791
  source: m.__src, override: !!m.__override, ...(hist[m.name] || { runs: 0, flagged: 0, last: null }),
784
792
  }));
785
793
  if (args.json) { console.log(JSON.stringify({ guards: rows }, null, 2)); return; }
786
- console.log('\nPromptWheel guardrails — effective set for this repo\n');
794
+ console.log('\nPromptWheel guardrails, effective set for this repo\n');
787
795
  for (const r of rows) {
788
796
  const prov = r.override ? 'local override' : (r.source === 'local' ? 'local' : `inherited ← ${r.source}`);
789
797
  const rec = r.guard ? (r.runs ? `· ${r.flagged} flagged / ${r.runs} runs` : '· no runs yet') : '';
@@ -799,13 +807,13 @@ function printHuman(r) {
799
807
  console.log(`\nPromptWheel ${r.base} → ${r.head}${r.repeat > 1 ? ` (×${r.repeat})` : ''}\n`);
800
808
  for (const m of r.metrics) {
801
809
  const tag = m.guard ? (m.ok ? 'guard✓' : 'GUARD✗') : 'info';
802
- const d = m.delta == null ? '—' : (m.delta > 0 ? `+${m.delta}` : `${m.delta}`);
810
+ const d = m.delta == null ? 'n/a' : (m.delta > 0 ? `+${m.delta}` : `${m.delta}`);
803
811
  console.log(` ${arrowFor(m)} ${m.name.padEnd(18)} ${String(m.before).padStart(8)} → ${String(m.after).padStart(8)} (${d}, ${m.status}) [${tag}, ${m.confidence}]`);
804
- if (m.inert) console.log(' ⚠ never passed at either ref — this guard is protecting nothing (check its command)');
805
- if (m.gamed === true) console.log(` 🚩 GAMED — ${m.gamingReason}`);
812
+ if (m.inert) console.log(' ⚠ never passed at either ref, this guard is protecting nothing (check its command)');
813
+ if (m.gamed === true) console.log(` 🚩 GAMED, ${m.gamingReason}`);
806
814
  }
807
- console.log(`\n VERDICT: ${r.verdict.toUpperCase()}${r.verdict === 'fail' ? ' — a guarded metric regressed (beyond noise)' : r.verdict === 'gamed' ? ' — a metric "improved" by editing the goalposts, not the source' : r.verdict === 'inconclusive' ? ' — a guard is inert (measured nothing); a pass cannot be certified' : ''}`);
808
- if (r.verdict === 'fail') console.log(' intentional? loosen that guard locally in promptwheel.config.json (guard:false, or override the inherited metric by name) — `promptwheel guards` shows the effective set');
815
+ console.log(`\n VERDICT: ${r.verdict.toUpperCase()}${r.verdict === 'fail' ? ', a guarded metric regressed (beyond noise)' : r.verdict === 'gamed' ? ', a metric "improved" by editing the goalposts, not the source' : r.verdict === 'inconclusive' ? ', a guard is inert (measured nothing); a pass cannot be certified' : ''}`);
816
+ if (r.verdict === 'fail') console.log(' intentional? loosen that guard locally in promptwheel.config.json (guard:false, or override the inherited metric by name), `promptwheel guards` shows the effective set');
809
817
  console.log('');
810
818
  }
811
819
 
@@ -814,12 +822,12 @@ function renderMarkdown(r) {
814
822
  const icon = r.verdict === 'pass' ? '✅' : r.verdict === 'gamed' ? '🚩' : r.verdict === 'inconclusive' ? '🟡' : '❌';
815
823
  const sIcon = { improved: '🟢', regressed: '🔴', unchanged: '⚪', inconclusive: '🟡', unmeasurable: '⚫' };
816
824
  const rows = r.metrics.map((m) => {
817
- const d = m.delta == null ? '—' : (m.delta > 0 ? `+${m.delta}` : `${m.delta}`);
825
+ const d = m.delta == null ? 'n/a' : (m.delta > 0 ? `+${m.delta}` : `${m.delta}`);
818
826
  const status = m.gamed === true ? `🚩 gamed (${(m.retained * 100).toFixed(0)}% survives source-only)` : `${sIcon[m.status] || ''} ${m.status}${m.inert ? ' ⚠ never passes' : ''}`;
819
827
  return `| ${m.guard ? '🛡️ ' : ''}${m.name} | ${m.before} | ${m.after} | ${d} | ${status} | ${m.confidence} |`;
820
828
  }).join('\n');
821
829
  return [
822
- `### ${icon} PromptWheel — outcome gate: **${r.verdict.toUpperCase()}**`,
830
+ `### ${icon} PromptWheel, outcome gate: **${r.verdict.toUpperCase()}**`,
823
831
  '',
824
832
  `\`${r.base} → ${r.head}\` · ${r.mode} mode${r.repeat > 1 ? ` · ×${r.repeat}` : ''}`,
825
833
  '',
@@ -827,7 +835,7 @@ function renderMarkdown(r) {
827
835
  '|---|--:|--:|--:|---|---|',
828
836
  rows,
829
837
  '',
830
- r.verdict === 'gamed' ? '> 🚩 A metric "improved" only because the agent edited tests/config/grader — not the source. The win does not survive a source-only re-run.' : r.verdict === 'fail' ? '> ❌ A 🛡️ guarded metric regressed beyond the noise band.' : r.verdict === 'inconclusive' ? '> 🟡 A 🛡️ guarded check is **inert** — it never runs, so nothing was actually verified. Fix its command before trusting a pass.' : '> ✅ No guarded metric regressed.',
838
+ r.verdict === 'gamed' ? '> 🚩 A metric "improved" only because the agent edited tests/config/grader, not the source. The win does not survive a source-only re-run.' : r.verdict === 'fail' ? '> ❌ A 🛡️ guarded metric regressed beyond the noise band.' : r.verdict === 'inconclusive' ? '> 🟡 A 🛡️ guarded check is **inert**, it never runs, so nothing was actually verified. Fix its command before trusting a pass.' : '> ✅ No guarded metric regressed.',
831
839
  '',
832
840
  '<sub>🛡️ = guard · _prove every change moved a metric_ · [PromptWheel](https://github.com/promptwheel-ai/promptwheel)</sub>',
833
841
  ].join('\n');
@@ -869,11 +877,11 @@ function main() {
869
877
  else if (cmd === 'guards') guards(rest);
870
878
  else {
871
879
  console.log([
872
- 'PromptWheel — catch your agent cheating. Prove a change moved a real metric (and that the agent earned it, not gamed it). The per-turn reward + source-only audit for AI coding loops.',
880
+ 'PromptWheel, catch your agent cheating. Prove a change moved a real metric (and that the agent earned it, not gamed it). The per-turn reward + source-only audit for AI coding loops.',
873
881
  '',
874
882
  ' promptwheel init [--preset <name> | --list] write a starter config for your stack',
875
883
  ' promptwheel run [--base R] [--head R] [--repeat N] [--json|--markdown]',
876
- ' catches reward-hacking BY DEFAULT — re-proves each win from SOURCE edits alone (verdict GAMED,',
884
+ ' catches reward-hacking BY DEFAULT, re-proves each win from SOURCE edits alone (verdict GAMED,',
877
885
  ' exit 2, when a metric only moved by editing tests/config/grader). Use --no-detect-gaming for the bare gate.',
878
886
  ' promptwheel run --working gate uncommitted changes (incl. newly added files)',
879
887
  ' promptwheel improve --attempt "<cmd>" run an agent/script; keep only if a metric improved',
@@ -886,7 +894,7 @@ function main() {
886
894
  '',
887
895
  'Loop it: while promptwheel improve --attempt "$AGENT"; do :; done # stops on plateau/regression',
888
896
  'Config: promptwheel.config.json → { extends?, metrics:[{ name, cmd, direction, extract?, guard? }] } (or: promptwheel init)',
889
- ' extends: a path (or array) to a shared base config — repos INHERIT its guardrails; local metrics override by name.',
897
+ ' extends: a path (or array) to a shared base config, repos INHERIT its guardrails; local metrics override by name.',
890
898
  ].join('\n'));
891
899
  process.exit(cmd ? 2 : 0);
892
900
  }
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "promptwheel",
3
- "version": "0.4.4",
4
- "description": "Add a referee to your coding loop \u2014 a deterministic, zero-LLM CLI that checks every change earned its win: it re-proves the result from the agent's source edits alone, so a green that came from editing the test reads GAMED. Runs in CI or a Claude Code hook.",
3
+ "version": "0.4.6",
4
+ "description": "A deterministic, zero-LLM check for your coding loop. It proves every change earned its win by re-running the result from the agent's source edits alone, so a green that came from editing the test reads GAMED. Runs in CI or a Claude Code hook.",
5
5
  "type": "module",
6
6
  "bin": {
7
7
  "promptwheel": "bin/promptwheel.mjs",