promptwheel 0.1.1 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -29,12 +29,12 @@ PromptWheel a1b2c3d → e4f5g6h (×5)
29
29
 
30
30
  Exit `0` on pass, `1` on fail (CI-friendly). No build step, zero dependencies, Node 18+.
31
31
 
32
- ## Catch your agent cheating — `--detect-gaming`
32
+ ## Catch your agent cheating — on by default
33
33
 
34
- The headline feature. After the normal gate, every *win* is re-proven using the agent's source edits **alone**: PromptWheel splits the diff into production source vs. `{test, config, grader, golden}` files, rebuilds a clean worktree at the base with **only the source slice** applied, and re-runs the gate. If the win doesn't survive — because it only passed by skipping/deleting a test, mocking the grader, editing a golden, relaxing a config, or the "win" touched zero source files — the verdict is **`GAMED` (exit 2)**:
34
+ The headline feature, and it runs **by default** (in both `run` and `improve`; pass `--no-detect-gaming` for the bare outcome gate). After the normal gate, every *win* is re-proven using the agent's source edits **alone**: PromptWheel splits the diff into production source vs. `{test, config, grader, golden}` files, rebuilds a clean worktree at the base with **only the source slice** applied, and re-runs the gate. If the win doesn't survive — because it only passed by skipping/deleting a test, mocking the grader, editing a golden, relaxing a config, or the "win" touched zero source files — the verdict is **`GAMED` (exit 2)**:
35
35
 
36
36
  ```
37
- $ promptwheel run --detect-gaming
37
+ $ promptwheel run
38
38
 
39
39
  PromptWheel base → head
40
40
  ▲ tests_pass 0 → 1 (+1, improved) [guard✓, high]
@@ -46,7 +46,8 @@ Inline source-file suppressions (`@ts-nocheck`, `eslint-disable`, `# noqa`) are
46
46
 
47
47
  ```bash
48
48
  npx promptwheel init --preset antihack # target + tripwire guards
49
- npx promptwheel run --detect-gaming # exit 0 = real win · 1 = regression · 2 = GAMED
49
+ npx promptwheel run # detection ON by default · exit 0 win · 1 regression · 2 GAMED
50
+ # (add --no-detect-gaming for just the outcome gate)
50
51
  ```
51
52
 
52
53
  Deterministic, zero-LLM, zero-network: an LLM judge asking "did you cheat?" is itself gameable; this is a diff partition plus a re-run, so a flag is trustworthy without a human in the loop. The 50%-of-gain-survives threshold is the default and is tunable.
@@ -61,7 +62,7 @@ npx promptwheel init # detects stack → guarded test metri
61
62
  npx promptwheel init --list # presets: tests-pass · lint · bundle-size · llm-eval · antihack
62
63
 
63
64
  # measure a change
64
- npx promptwheel run # base = merge-base with main, head = HEAD
65
+ npx promptwheel run # base = merge-base with main, head = HEAD · reward-hack detection ON
65
66
  npx promptwheel run --working # measure UNCOMMITTED changes (incl. newly added files)
66
67
  npx promptwheel run --repeat 5 --json # measure 5× to establish a noise band, emit JSON
67
68
 
@@ -89,6 +90,8 @@ npx promptwheel run --working --json # branch on .verdict / per-metric .statu
89
90
 
90
91
  The exit code is the contract — `0` kept · `1` regression · `3` plateau — so any driver (`/loop`, a Ralph `while`, a Beads pull-loop) converges without parsing anything. PromptWheel never drives the loop; it only says whether the turn counted.
91
92
 
93
+ **Two callers, by design.** Your *agent* consumes the verdict — it's the loop's per-turn reward (`improve` / `run --working --json`; exit `0` kept · `1`/`2` reverted · `3` plateau). The *harness* (a Stop-hook or CI) runs `--detect-gaming` — the audit the agent **can't self-clear**, because a contestant can't referee itself (see [docs/ENFORCEMENT.md](docs/ENFORCEMENT.md)). Same tool; *who calls it* is the difference between a reward and an audit.
94
+
92
95
  ## In Claude Code — plugin
93
96
 
94
97
  Bring the gate into Claude Code as slash commands:
@@ -158,6 +161,8 @@ Teams keep one shared base config and have every repo inherit it via `extends`:
158
161
 
159
162
  `extends` takes a path (or array of paths) to base configs: a repo **inherits** their guardrails, and a local metric of the same name **overrides** the inherited one (tighten, loosen, or disable). `promptwheel guards` shows the effective set with provenance — `inherited ← base.json`, `local`, or `local override` — plus each guard's flag record from the outcome stream.
160
163
 
164
+ Read this way, `extends` is **the shared invariants — the "business rules" — your agents inherit and are held to**: enforced as *measured guards* (a trusted regression fails the gate / reverts the commit), not advisory text an agent can quietly ignore. (Natural-language conventions belong in `AGENTS.md`; only deterministic, measurable guards belong here.)
165
+
161
166
  ## Trust model — the point of the whole thing
162
167
 
163
168
  A number that jumps around between runs is worthless as a signal. PromptWheel won't pretend otherwise:
@@ -334,7 +334,7 @@ function run(argv) {
334
334
  process.exit(report.verdict === 'pass' ? 0 : report.verdict === 'gamed' ? 2 : 1);
335
335
  }
336
336
 
337
- // the moat: append every gated run to a per-repo outcome record (best-effort, never fails the gate)
337
+ // persisted record: append every gated run to a per-repo outcome record (best-effort, never fails the gate)
338
338
  function recordOutcome(repo, report) {
339
339
  try {
340
340
  const dir = join(repo, '.promptwheel');
@@ -343,7 +343,7 @@ function recordOutcome(repo, report) {
343
343
  } catch { /* recording must never break the gate */ }
344
344
  }
345
345
 
346
- // the flywheel: run any agent/script, gate the result, keep the change ONLY if a metric improved
346
+ // keep-if-improved: run any agent/script, gate the result, keep the change ONLY if a metric improved
347
347
  function improve(argv) {
348
348
  const args = parseArgs(argv);
349
349
  if (!args.attempt) { console.error('improve requires --attempt "<command that changes the repo>"'); process.exit(2); }
@@ -553,7 +553,7 @@ function renderMarkdown(r) {
553
553
  }
554
554
 
555
555
  function parseArgs(argv) {
556
- const a = { json: false };
556
+ const a = { json: false, detectGaming: true }; // reward-hack detection is ON by default; --no-detect-gaming opts out
557
557
  for (let i = 0; i < argv.length; i++) {
558
558
  if (argv[i] === '--base') a.base = argv[++i];
559
559
  else if (argv[i] === '--head') a.head = argv[++i];
@@ -561,6 +561,7 @@ function parseArgs(argv) {
561
561
  else if (argv[i] === '--working') a.working = true;
562
562
  else if (argv[i] === '--no-record') a.noRecord = true;
563
563
  else if (argv[i] === '--detect-gaming' || argv[i] === '--antihack') a.detectGaming = true;
564
+ else if (argv[i] === '--no-detect-gaming' || argv[i] === '--no-antihack') a.detectGaming = false;
564
565
  else if (argv[i] === '--attempt') a.attempt = argv[++i];
565
566
  else if (argv[i] === '--json') a.json = true;
566
567
  else if (argv[i] === '--markdown') a.markdown = true;
@@ -581,13 +582,13 @@ function main() {
581
582
  else if (cmd === 'guards') guards(rest);
582
583
  else {
583
584
  console.log([
584
- 'PromptWheel — the per-turn reward for AI coding loops. Prove a change moved a metric.',
585
+ 'PromptWheel — catch your agent cheating. Prove a change moved a real metric (and that the agent earned it, not gamed it). The per-turn reward + source-only audit for AI coding loops.',
585
586
  '',
586
587
  ' promptwheel init [--preset <name> | --list] write a starter config for your stack',
587
588
  ' promptwheel run [--base R] [--head R] [--repeat N] [--json|--markdown]',
589
+ ' catches reward-hacking BY DEFAULT — re-proves each win from SOURCE edits alone (verdict GAMED,',
590
+ ' exit 2, when a metric only moved by editing tests/config/grader). Use --no-detect-gaming for the bare gate.',
588
591
  ' promptwheel run --working gate uncommitted changes (incl. newly added files)',
589
- ' promptwheel run --detect-gaming re-prove each win with SOURCE edits alone — catch reward-hacking',
590
- ' (verdict GAMED, exit 2, when a metric only moved by editing tests/config/grader)',
591
592
  ' promptwheel improve --attempt "<cmd>" run an agent/script; keep only if a metric improved',
592
593
  ' exit 0=kept · 1=regression · 3=plateau · add --json',
593
594
  ' promptwheel insights which metrics actually respond (loop memory)',
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "promptwheel",
3
- "version": "0.1.1",
3
+ "version": "0.1.2",
4
4
  "description": "Catch your agent cheating — a deterministic, zero-LLM CLI that flags when an AI coding agent gamed its own metric (re-proves the win from the agent's source edits alone). Built on an outcome gate; runs in CI or a Claude Code hook.",
5
5
  "type": "module",
6
6
  "bin": {