promptwheel 0.4.4 → 0.4.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -2
- package/bin/promptwheel.mjs +5 -1
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
[](https://github.com/promptwheel-ai/promptwheel/actions/workflows/ci.yml) [](LICENSE)  
|
|
4
4
|
|
|
5
|
-
**
|
|
5
|
+
**A deterministic check for your coding loop.** A green test should mean your agent *fixed the code*, not that it edited the test to match the bug. PromptWheel tells the difference: it re-proves every win from the source changes alone, with no LLM, so a win that only survived by editing the test reads `GAMED`. Un-gameable by construction, because the code being checked never gets to grade itself.
|
|
6
6
|
|
|
7
7
|
PromptWheel re-proves every "win" using the agent's **source edits alone**. If the gate only went green because the agent edited the test, mocked the grader, suppressed the error (`@ts-ignore` / `eslint-disable`), or deleted the feature, the win evaporates when those edits are reverted → **`VERDICT GAMED`, exit 2**. No LLM in the loop — a diff partition plus a re-run, so every flag is reproducible in seconds with a human-readable reason.
|
|
8
8
|
|
|
@@ -10,6 +10,8 @@ It's built on an **outcome gate**: for any change it re-runs your metric command
|
|
|
10
10
|
|
|
11
11
|
PromptWheel is the **signal, not the loop driver**: wire it as the verifier inside Claude Code `/loop`, a Ralph `while`-loop, or a Beads pull-loop. Each turn it answers one question — *did this turn earn its keep, and did the agent earn it honestly?* — so the loop improves instead of confidently degrading. (In CI it's the **outcome gate for AI code**: the same verdict, as a PR check.)
|
|
12
12
|
|
|
13
|
+
PromptWheel is the reference check for the [Verified-Outcome Contract](https://github.com/sawbuck-ai/sawbuck/blob/master/CONTRACT.md), the open interface for a verifiable coding loop.
|
|
14
|
+
|
|
13
15
|
> Same name, new meaning. The "wheel" is the **improvement flywheel**: every turn only counts if it **provably moved a metric without regressing another.** Orchestration (the old "wheel of prompts") *and the outcome gate itself* are now solved, commoditized problems; the open one is **catching when the agent games that gate** — making the reward signal one you can't cheat.
|
|
14
16
|
|
|
15
17
|
AI coding agents (and humans) declare success when a change *compiles and tests pass*. They almost never verify that a change **improved a real measurable thing without regressing another** — and outcome data rots in dashboards (DORA, Swarmia) disconnected from the change that caused it. PromptWheel closes that loop.
|
|
@@ -116,7 +118,7 @@ npx promptwheel run --working --json # branch on .verdict / per-metric .statu
|
|
|
116
118
|
|
|
117
119
|
The exit code is the contract — `0` kept · `1` regression · `3` plateau — so any driver (`/loop`, a Ralph `while`, a Beads pull-loop) converges without parsing anything. PromptWheel never drives the loop; it only says whether the turn counted.
|
|
118
120
|
|
|
119
|
-
**Two callers, by design.** Your *agent* consumes the verdict
|
|
121
|
+
**Two callers, by design.** Your *agent* consumes the verdict as the loop's per-turn reward (`improve` / `run --working --json`; exit `0` kept · `1`/`2` reverted · `3` plateau). The *harness* (a Stop-hook or CI) runs `--detect-gaming`, the audit the agent **can't self-clear**, because the check never grades itself; it re-proves from the source alone (see [docs/ENFORCEMENT.md](docs/ENFORCEMENT.md)). Same tool; *who calls it* is the difference between a reward and an audit.
|
|
120
122
|
|
|
121
123
|
## In Claude Code — plugin
|
|
122
124
|
|
package/bin/promptwheel.mjs
CHANGED
|
@@ -84,8 +84,12 @@ function resolveBase(repo, base) {
|
|
|
84
84
|
|
|
85
85
|
function runMetric(cwd, m, env) {
|
|
86
86
|
let stdout = '', code = 0;
|
|
87
|
+
// Never leak the harness's own node:test context into the measured gate: a metric like
|
|
88
|
+
// `node --test` would see NODE_TEST_CONTEXT, act as a child reporter, and report a bogus
|
|
89
|
+
// exit code (so the gate can't be trusted when PromptWheel itself runs under a test runner).
|
|
90
|
+
const childEnv = { ...(env || process.env) }; delete childEnv.NODE_TEST_CONTEXT;
|
|
87
91
|
try {
|
|
88
|
-
stdout = execSync(m.cmd, { cwd, env:
|
|
92
|
+
stdout = execSync(m.cmd, { cwd, env: childEnv, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], timeout: (m.timeoutSec ?? 300) * 1000 });
|
|
89
93
|
} catch (e) { code = e.status ?? 1; stdout = `${e.stdout || ''}${e.stderr || ''}`; }
|
|
90
94
|
return extract(stdout, code, m.extract, m.allowNonzero);
|
|
91
95
|
}
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "promptwheel",
|
|
3
|
-
"version": "0.4.
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "0.4.5",
|
|
4
|
+
"description": "A deterministic, zero-LLM check for your coding loop. It proves every change earned its win by re-running the result from the agent's source edits alone: it re-proves the result from the agent's source edits alone, so a green that came from editing the test reads GAMED. Runs in CI or a Claude Code hook.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
7
7
|
"promptwheel": "bin/promptwheel.mjs",
|