promptwheel 0.4.3 → 0.4.5

package/README.md

@@ -2,7 +2,7 @@
 
 [![CI](https://github.com/promptwheel-ai/promptwheel/actions/workflows/ci.yml/badge.svg)](https://github.com/promptwheel-ai/promptwheel/actions/workflows/ci.yml) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) ![Node ≥18](https://img.shields.io/badge/node-%E2%89%A518-brightgreen) ![deps: zero](https://img.shields.io/badge/deps-zero-blue)
 
-**Add a referee to your coding loop.** A green test should mean your agent *fixed the code* — not that it edited the test to match the bug. PromptWheel is the referee that tells the difference: deterministic, no LLM, and un-gameable, because a player can't call its own fouls.
+**A deterministic check for your coding loop.** A green test should mean your agent *fixed the code*, not that it edited the test to match the bug. PromptWheel tells the difference: it re-proves every win from the source changes alone, with no LLM, so a win that only survived by editing the test reads `GAMED`. Un-gameable by construction, because the code being checked never gets to grade itself.
 
 PromptWheel re-proves every "win" using the agent's **source edits alone**. If the gate only went green because the agent edited the test, mocked the grader, suppressed the error (`@ts-ignore` / `eslint-disable`), or deleted the feature, the win evaporates when those edits are reverted → **`VERDICT GAMED`, exit 2**. No LLM in the loop — a diff partition plus a re-run, so every flag is reproducible in seconds with a human-readable reason.
 
@@ -10,6 +10,8 @@ It's built on an **outcome gate**: for any change it re-runs your metric command
 
 PromptWheel is the **signal, not the loop driver**: wire it as the verifier inside Claude Code `/loop`, a Ralph `while`-loop, or a Beads pull-loop. Each turn it answers one question — *did this turn earn its keep, and did the agent earn it honestly?* — so the loop improves instead of confidently degrading. (In CI it's the **outcome gate for AI code**: the same verdict, as a PR check.)
 
+PromptWheel is the reference check for the [Verified-Outcome Contract](https://github.com/sawbuck-ai/sawbuck/blob/master/CONTRACT.md), the open interface for a verifiable coding loop.
+
 > Same name, new meaning. The "wheel" is the **improvement flywheel**: every turn only counts if it **provably moved a metric without regressing another.** Orchestration (the old "wheel of prompts") *and the outcome gate itself* are now solved, commoditized problems; the open one is **catching when the agent games that gate** — making the reward signal one you can't cheat.
 
 AI coding agents (and humans) declare success when a change *compiles and tests pass*. They almost never verify that a change **improved a real measurable thing without regressing another** — and outcome data rots in dashboards (DORA, Swarmia) disconnected from the change that caused it. PromptWheel closes that loop.
@@ -116,7 +118,7 @@ npx promptwheel run --working --json    # branch on .verdict / per-metric .statu
 
 The exit code is the contract — `0` kept · `1` regression · `3` plateau — so any driver (`/loop`, a Ralph `while`, a Beads pull-loop) converges without parsing anything. PromptWheel never drives the loop; it only says whether the turn counted.
 
-**Two callers, by design.** Your *agent* consumes the verdict — it's the loop's per-turn reward (`improve` / `run --working --json`; exit `0` kept · `1`/`2` reverted · `3` plateau). The *harness* (a Stop-hook or CI) runs `--detect-gaming` — the audit the agent **can't self-clear**, because a contestant can't referee itself (see [docs/ENFORCEMENT.md](docs/ENFORCEMENT.md)). Same tool; *who calls it* is the difference between a reward and an audit.
+**Two callers, by design.** Your *agent* consumes the verdict as the loop's per-turn reward (`improve` / `run --working --json`; exit `0` kept · `1`/`2` reverted · `3` plateau). The *harness* (a Stop-hook or CI) runs `--detect-gaming`, the audit the agent **can't self-clear**, because the check never grades itself; it re-proves from the source alone (see [docs/ENFORCEMENT.md](docs/ENFORCEMENT.md)). Same tool; *who calls it* is the difference between a reward and an audit.
 
 ## In Claude Code — plugin
 

package/bin/promptwheel.mjs

@@ -84,8 +84,12 @@ function resolveBase(repo, base) {
 
 function runMetric(cwd, m, env) {
   let stdout = '', code = 0;
+  // Never leak the harness's own node:test context into the measured gate: a metric like
+  // `node --test` would see NODE_TEST_CONTEXT, act as a child reporter, and report a bogus
+  // exit code (so the gate can't be trusted when PromptWheel itself runs under a test runner).
+  const childEnv = { ...(env || process.env) }; delete childEnv.NODE_TEST_CONTEXT;
   try {
-    stdout = execSync(m.cmd, { cwd, env: env || process.env, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], timeout: (m.timeoutSec ?? 300) * 1000 });
+    stdout = execSync(m.cmd, { cwd, env: childEnv, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], timeout: (m.timeoutSec ?? 300) * 1000 });
   } catch (e) { code = e.status ?? 1; stdout = `${e.stdout || ''}${e.stderr || ''}`; }
   return extract(stdout, code, m.extract, m.allowNonzero);
 }
@@ -208,6 +212,13 @@ const NON_SOURCE = [
   /(^|\/)tsconfig[^/]*\.json$/i,
   /(^|\/)\.(eslintrc|babelrc|prettierrc)[^/]*$/i,
   /(^|\/)(pytest\.ini|setup\.cfg|\.flake8|tox\.ini|eslint\.config\.[cm]?js)$/i,
+  // promptwheel's OWN evaluator surface: the config that defines the gate
+  // and any pack copied into the repo by `init --preset`. Editing these is
+  // moving the goalposts, never a production improvement — an agent that
+  // "improved" security_findings by deleting pattern rules must show 0%
+  // retained in the source-only re-run, not 100%.
+  /(^|\/)promptwheel\.config\.json$/i,
+  /(^|\/)\.promptwheel\//i,
 ];
 const isNonSource = (p) => NON_SOURCE.some((re) => re.test(p));
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "promptwheel",
-  "version": "0.4.3",
-  "description": "Add a referee to your coding loop \u2014 a deterministic, zero-LLM CLI that checks every change earned its win: it re-proves the result from the agent's source edits alone, so a green that came from editing the test reads GAMED. Runs in CI or a Claude Code hook.",
+  "version": "0.4.5",
+  "description": "A deterministic, zero-LLM check for your coding loop. It proves every change earned its win by re-running the result from the agent's source edits alone: it re-proves the result from the agent's source edits alone, so a green that came from editing the test reads GAMED. Runs in CI or a Claude Code hook.",
   "type": "module",
   "bin": {
     "promptwheel": "bin/promptwheel.mjs",