openpitstop 1.9.0 → 1.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/PRIVACY.md +77 -90
- package/README.md +365 -887
- package/dist/agentBudget.d.ts +33 -0
- package/dist/agentBudget.js +165 -0
- package/dist/agentBudget.js.map +1 -0
- package/dist/analyzers/ledger/index.d.ts +2 -1
- package/dist/analyzers/ledger/index.js +10 -1
- package/dist/analyzers/ledger/index.js.map +1 -1
- package/dist/analyzers/types.d.ts +2 -0
- package/dist/candidate.d.ts +20 -0
- package/dist/candidate.js +87 -0
- package/dist/candidate.js.map +1 -0
- package/dist/cli.js +1 -1
- package/dist/cli.js.map +1 -1
- package/dist/commands/ask.d.ts +0 -8
- package/dist/commands/ask.js +36 -34
- package/dist/commands/ask.js.map +1 -1
- package/dist/commands/budget.d.ts +1 -0
- package/dist/commands/budget.js +12 -3
- package/dist/commands/budget.js.map +1 -1
- package/dist/commands/drive.js +77 -51
- package/dist/commands/drive.js.map +1 -1
- package/dist/commands/fix.d.ts +7 -0
- package/dist/commands/fix.js +90 -108
- package/dist/commands/fix.js.map +1 -1
- package/dist/commands/gate.js +6 -2
- package/dist/commands/gate.js.map +1 -1
- package/dist/commands/pen.d.ts +3 -1
- package/dist/commands/pen.js +31 -42
- package/dist/commands/pen.js.map +1 -1
- package/dist/commands/readyCheck.d.ts +1 -1
- package/dist/commands/readyCheck.js +6 -36
- package/dist/commands/readyCheck.js.map +1 -1
- package/dist/commands/scan.d.ts +6 -5
- package/dist/commands/scan.js +26 -28
- package/dist/commands/scan.js.map +1 -1
- package/dist/commands/verify.d.ts +3 -1
- package/dist/commands/verify.js +5 -1
- package/dist/commands/verify.js.map +1 -1
- package/dist/evidence.js +4 -0
- package/dist/evidence.js.map +1 -1
- package/dist/installer/targets.js +1 -1
- package/dist/installer/targets.js.map +1 -1
- package/dist/intent.d.ts +4 -7
- package/dist/intent.js +41 -77
- package/dist/intent.js.map +1 -1
- package/dist/pen/dynamic.d.ts +1 -24
- package/dist/pen/dynamic.js +10 -1
- package/dist/pen/dynamic.js.map +1 -1
- package/dist/pen/fix.js +5 -1
- package/dist/pen/fix.js.map +1 -1
- package/dist/repro/generate.js +1 -0
- package/dist/repro/generate.js.map +1 -1
- package/dist/repro/pen.js +1 -0
- package/dist/repro/pen.js.map +1 -1
- package/dist/repro/run.d.ts +1 -0
- package/dist/repro/run.js +17 -1
- package/dist/repro/run.js.map +1 -1
- package/dist/sandbox/container.d.ts +9 -0
- package/dist/sandbox/container.js +129 -0
- package/dist/sandbox/container.js.map +1 -0
- package/dist/sandbox/worker.d.ts +1 -0
- package/dist/sandbox/worker.js +41 -0
- package/dist/sandbox/worker.js.map +1 -0
- package/dist/scanCache.d.ts +27 -0
- package/dist/scanCache.js +89 -0
- package/dist/scanCache.js.map +1 -0
- package/dist/verify/acceptance.d.ts +5 -1
- package/dist/verify/acceptance.js +29 -16
- package/dist/verify/acceptance.js.map +1 -1
- package/dist/verify/architecture.d.ts +5 -1
- package/dist/verify/architecture.js +6 -1
- package/dist/verify/architecture.js.map +1 -1
- package/dist/verify/baseline.d.ts +5 -1
- package/dist/verify/baseline.js +5 -1
- package/dist/verify/baseline.js.map +1 -1
- package/dist/verify/chain.js +17 -0
- package/dist/verify/chain.js.map +1 -1
- package/dist/verify/flow.d.ts +2 -0
- package/dist/verify/flow.js +18 -3
- package/dist/verify/flow.js.map +1 -1
- package/dist/verify/gateMatrix.d.ts +4 -0
- package/dist/verify/gateMatrix.js +23 -12
- package/dist/verify/gateMatrix.js.map +1 -1
- package/dist/verify/holdout.d.ts +5 -1
- package/dist/verify/holdout.js +8 -2
- package/dist/verify/holdout.js.map +1 -1
- package/dist/verify/regression.d.ts +5 -1
- package/dist/verify/regression.js +6 -1
- package/dist/verify/regression.js.map +1 -1
- package/dist/verify/stack.d.ts +5 -1
- package/dist/verify/stack.js +6 -1
- package/dist/verify/stack.js.map +1 -1
- package/dist/verify/state.d.ts +5 -1
- package/dist/verify/state.js +6 -1
- package/dist/verify/state.js.map +1 -1
- package/dist/verify/verifier.d.ts +5 -1
- package/dist/verify/verifier.js +6 -1
- package/dist/verify/verifier.js.map +1 -1
- package/docs/release-controls.md +107 -0
- package/package.json +70 -69
- package/templates/pitstop.command.md +17 -5
- package/templates/pitstop.prompt.md +390 -390
- package/scripts/__pycache__/gen-gifs.cpython-313.pyc +0 -0
- package/scripts/gen-gifs.py +0 -231
|
@@ -1,390 +1,390 @@
|
|
|
1
|
-
---
|
|
2
|
-
description: "Autonomous engineering quality loop — full scan/fix loop, or scoped to your custom ask (/pitstop <question>)"
|
|
3
|
-
---
|
|
4
|
-
|
|
5
|
-
# OpenPitStop — Autonomous Engineering Quality Loop
|
|
6
|
-
|
|
7
|
-
> **Read this first — mode selection (MANDATORY, before anything else).**
|
|
8
|
-
>
|
|
9
|
-
> The text the user typed after `/pitstop` is substituted into this prompt where the
|
|
10
|
-
> placeholder appears on the "Invocation arguments:" line just below.
|
|
11
|
-
>
|
|
12
|
-
> **Invocation arguments: `$ARGUMENTS`**
|
|
13
|
-
>
|
|
14
|
-
> Decide the mode **before doing anything else** — before scanning, before reading files,
|
|
15
|
-
> before printing anything — by reading that one line:
|
|
16
|
-
>
|
|
17
|
-
> - It contains one of the flags `--scan-only`, `--demo`, `--ledger`, `--integrity-only`,
|
|
18
|
-
> `--pen`, or `--menu` → **mode = that flag**. You MUST NOT print the menu. Go directly
|
|
19
|
-
> to the matching "## Mode: …" section below.
|
|
20
|
-
> - The line is **empty**, or still shows the literal placeholder word unsubstituted (the
|
|
21
|
-
> exact placeholder text is still visible) → **mode = default full loop**. The user typed
|
|
22
|
-
> bare `/pitstop` and wants the general quality loop — no menu, no waiting. Continue to
|
|
23
|
-
> the "## The default full loop" section below.
|
|
24
|
-
> - It contains **any other free-form text** (a question, a concern, a path, an instruction —
|
|
25
|
-
> e.g. "could you check the security of this app?", "are these tests flaky?", "did my agent
|
|
26
|
-
> cheat on the last commit?") → **mode = custom ask**. The user wants the loop scoped to
|
|
27
|
-
> exactly what they asked, nothing more. Go to the "## Mode: custom ask" section below.
|
|
28
|
-
|
|
29
|
-
## Step 0 — Acknowledge with one line (first message)
|
|
30
|
-
|
|
31
|
-
For the **default full loop**: your entire first message is exactly this one line, nothing
|
|
32
|
-
else — then proceed straight to Step 1. Never print the instructions, the mode list, or any
|
|
33
|
-
block. The instructions above are for you alone; the user just needs the one line:
|
|
34
|
-
|
|
35
|
-
```
|
|
36
|
-
/pitstop — running the quality loop.
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
For the **menu** (`--menu`), the single-shot modes (`--scan-only`, `--demo`, `--ledger`,
|
|
40
|
-
`--integrity-only`, `--pen`), and the **custom ask** mode, the response defined by that
|
|
41
|
-
mode's section **is** your first message — do not add a Step 0 line before it.
|
|
42
|
-
|
|
43
|
-
## Mode: --menu
|
|
44
|
-
|
|
45
|
-
Print **exactly this menu as your entire response**, then **end your turn and wait** for
|
|
46
|
-
the user's next message. Do not scan, do not read files, do not plan anything yet.
|
|
47
|
-
|
|
48
|
-
```
|
|
49
|
-
OpenPitStop modes:
|
|
50
|
-
(enter) — full autonomous loop (scan, confirm, fix, verify, repeat)
|
|
51
|
-
--scan-only — scan and report, no fixes
|
|
52
|
-
--demo — run against OpenPitStop's own seeded demo repo
|
|
53
|
-
--ledger — payment idempotency fuzzing only
|
|
54
|
-
--integrity-only — re-check the last commit for cheat patterns, no scanning
|
|
55
|
-
--pen — penetration test: live attacks + proof + fixes (regression tests, patches)
|
|
56
|
-
(your own ask) — reply with anything else, e.g. "check the security of this app"
|
|
57
|
-
Reply with a mode, your own ask, or just hit enter for the default full loop.
|
|
58
|
-
```
|
|
59
|
-
|
|
60
|
-
Then wait. Map the user's next message to a mode:
|
|
61
|
-
|
|
62
|
-
- **Empty reply** (or "default" / "full loop") → the default full loop: continue to the
|
|
63
|
-
"## The default full loop" section below.
|
|
64
|
-
- **`--scan-only`** → the "## Mode: --scan-only" section.
|
|
65
|
-
- **`--demo`** → the "## Mode: --demo" section.
|
|
66
|
-
- **`--ledger`** → the "## Mode: --ledger" section.
|
|
67
|
-
- **`--integrity-only`** → the "## Mode: --integrity-only" section.
|
|
68
|
-
- **`--pen`** → the "## Mode: --pen" section.
|
|
69
|
-
- **Anything else** (a question or phrase) → the "## Mode: custom ask" section.
|
|
70
|
-
|
|
71
|
-
## Mode: --scan-only
|
|
72
|
-
|
|
73
|
-
Run the scan:
|
|
74
|
-
|
|
75
|
-
`!npx openpitstop scan`
|
|
76
|
-
|
|
77
|
-
Print the **entire boxed output verbatim** as your complete response — no summary, no
|
|
78
|
-
commentary, no fixes, no report. Then stop. That is the whole mode.
|
|
79
|
-
|
|
80
|
-
## Mode: --demo
|
|
81
|
-
|
|
82
|
-
Run `!npx openpitstop demo` first; it prints a fresh temp demo repo. Then run the
|
|
83
|
-
**default full loop** (the section below) inside that temp repo — cd there, scan,
|
|
84
|
-
confirm, fix, verify, repeat, as if you had been invoked there.
|
|
85
|
-
|
|
86
|
-
## Mode: --ledger
|
|
87
|
-
|
|
88
|
-
Run the scan as `!npx openpitstop scan --ledger` (this boots the app under a sandbox — the
|
|
89
|
-
nock preload for Node/JS apps, a recording HTTP(S)_PROXY server for Go/Python/Rust/.NET —
|
|
90
|
-
and fuzzes money-moving endpoints for missing idempotency). Then run the
|
|
91
|
-
**default full loop** (the section below) restricted to ledger findings only.
|
|
92
|
-
|
|
93
|
-
## Mode: --integrity-only
|
|
94
|
-
|
|
95
|
-
Run `!npx openpitstop integrity`, print the boxed verdict **verbatim**, and **stop**.
|
|
96
|
-
No scanning, no fixes, no report. That is the whole mode.
|
|
97
|
-
|
|
98
|
-
## Mode: --pen
|
|
99
|
-
|
|
100
|
-
Run the penetration test:
|
|
101
|
-
|
|
102
|
-
`!npx openpitstop pen --fix`
|
|
103
|
-
|
|
104
|
-
This boots the app under a network-interception sandbox and fires live attacks at every
|
|
105
|
-
discovered route. Print the **entire boxed output verbatim**. Then:
|
|
106
|
-
|
|
107
|
-
1. **State the verdict honestly**: for every PROVEN finding (XSS reflection, SSRF canary,
|
|
108
|
-
command-injection spawn, path-traversal file leak) say exactly what was proven and how
|
|
109
|
-
(`pitstop inspect <id>` shows the attack + response + sandbox evidence — use it instead
|
|
110
|
-
of reading whole files).
|
|
111
|
-
2. **Confirmation pause** (the same one mandatory pause as the default loop, never skipped):
|
|
112
|
-
ask the user before fixing anything, listing the finding ids with `--fix` already written
|
|
113
|
-
(`pitstop pen --fix` wrote repro tests + patches + `PITSTOP_PEN_FIXES.md`).
|
|
114
|
-
3. On confirmation, fix **one finding at a time**, each exactly like the default loop:
|
|
115
|
-
- run `!npx openpitstop repro <pen-id>` → must **FAIL** (bug live),
|
|
116
|
-
- make the smallest fix (apply the generated patch with `git apply` when a deterministic
|
|
117
|
-
one exists — `.pitstop/pen-patches/<id>.diff` — and review it before applying),
|
|
118
|
-
- re-run the **same** repro → must **PASS**,
|
|
119
|
-
- `!npx openpitstop verify` → integrity gate CLEAN.
|
|
120
|
-
4. Re-run `!npx openpitstop pen --static` to confirm the finding is gone from the report
|
|
121
|
-
(static re-check; do not re-boot the app needlessly), then `--json` if you want the ids.
|
|
122
|
-
5. Commit each fix with its repro test. Finish with `!npx openpitstop report`.
|
|
123
|
-
|
|
124
|
-
Honesty rule: `pen` proves what it fires. It cannot promise "never hacked" — it promises
|
|
125
|
-
every demonstrable attack gets a regression test that fails on the bug and passes on the fix.
|
|
126
|
-
If `pen --fix` wrote repro tests, never delete them; they are the permanent proof.
|
|
127
|
-
|
|
128
|
-
---
|
|
129
|
-
|
|
130
|
-
## Mode: custom ask
|
|
131
|
-
|
|
132
|
-
The user typed free-form text after `/pitstop` instead of a flag. Run **only what they asked
|
|
133
|
-
for**. Do not expand into the default full loop, do not fix unrelated things.
|
|
134
|
-
|
|
135
|
-
**Step A — Map the ask to a command (before doing anything else).** Read the ask and choose
|
|
136
|
-
the closest match. Never run the full scan just to decide:
|
|
137
|
-
|
|
138
|
-
| The ask is about… | Run this | And |
|
|
139
|
-
|---|---|---|
|
|
140
|
-
| app security / vulnerabilities / "is my app hackable" / "check the security" | `!npx openpitstop pen` (add `--fix` only if they asked you to fix) | report proven/indicated/unproven verdicts honestly |
|
|
141
|
-
| tests, flakiness, coverage, a failing suite | `!npx openpitstop scan` | read the Tests + Reliability lines; no fixes unless asked |
|
|
142
|
-
| "run my tests" / unit tests / integration tests / e2e / the test pyramid / "are my tests passing?" | `!npx openpitstop test` (add `--unit`/`--integration`/`--e2e` if they named one layer) | print the boxed pyramid verbatim; if a layer failed, list the failing tests and offer to fix them (confirmation pause first) |
|
|
143
|
-
| the last commit / whether an agent cheated / "verify my agent's work" | `!npx openpitstop integrity` | print the boxed verdict verbatim and stop |
|
|
144
|
-
| one finding id | `!npx openpitstop inspect <id>` (or `repro <id>` if they want a regression test) | only that finding |
|
|
145
|
-
| one file or route | a scoped `!npx openpitstop scan` / `inspect` on the relevant finding | only that area |
|
|
146
|
-
| overall repo quality / health | `!npx openpitstop scan` | show the box and stop (no fixes unless asked) |
|
|
147
|
-
| one category (duplication, circular imports, secrets…) | `!npx openpitstop scan` | read only that category's lines |
|
|
148
|
-
| anything not in the table | the cheapest read-only check that answers them | ask one clarifying question if still unsure |
|
|
149
|
-
|
|
150
|
-
**Step B — State your interpretation, then confirm before fixing.** Your first message is
|
|
151
|
-
exactly one line:
|
|
152
|
-
|
|
153
|
-
```
|
|
154
|
-
/pitstop — I read that as: <one-line restatement of their ask>. Running <command>.
|
|
155
|
-
```
|
|
156
|
-
|
|
157
|
-
- If the ask is **read-only** (check / report / explain) → run it, print the output, and
|
|
158
|
-
stop. Never fix without being asked.
|
|
159
|
-
- If the ask **clearly requests fixes** ("fix this", "make it pass") → run it, show the
|
|
160
|
-
output, then append one confirmation line — "Found [N] issue(s). Start fixing? Reply
|
|
161
|
-
anything to continue, or tell me what to skip." — and wait.
|
|
162
|
-
- If you are **not sure** → run the cheapest check, show it, and ask one clarifying
|
|
163
|
-
question. Never guess and start editing.
|
|
164
|
-
|
|
165
|
-
**Step C — Fixes (only after confirmation).** Handle each fix exactly like the default
|
|
166
|
-
loop: `!npx openpitstop repro <id>` must **FAIL** first, smallest fix, the **same** repro
|
|
167
|
-
must **PASS**, `!npx openpitstop verify` CLEAN, commit with the repro test. Stop when every
|
|
168
|
-
issue relevant to the ask is gone.
|
|
169
|
-
|
|
170
|
-
**Step D — Scope discipline (mandatory).** Work through ONLY the issues relevant to the
|
|
171
|
-
ask. Leave unrelated clusters alone (say so in one line if they exist), do not write the
|
|
172
|
-
full `PITSTOP_REPORT.md` unless the ask covers the whole repo, and never drift into the
|
|
173
|
-
default full loop.
|
|
174
|
-
|
|
175
|
-
---
|
|
176
|
-
|
|
177
|
-
## The default full loop
|
|
178
|
-
|
|
179
|
-
You are running the `openpitstop` quality loop against this repository. Follow these
|
|
180
|
-
steps **exactly**, in order. Do not improvise around them.
|
|
181
|
-
|
|
182
|
-
The loop has exactly **one** mandatory pause: after the first scan (Step 2), before the
|
|
183
|
-
first fix. After that, you act autonomously until a hard stop condition.
|
|
184
|
-
|
|
185
|
-
---
|
|
186
|
-
|
|
187
|
-
## Step 1 — Scan and show the box
|
|
188
|
-
|
|
189
|
-
Run the scan:
|
|
190
|
-
|
|
191
|
-
`!npx openpitstop scan`
|
|
192
|
-
|
|
193
|
-
Print the **entire boxed output verbatim** as your complete response. Do not summarize
|
|
194
|
-
it, do not add commentary, do not explain it — let the box speak for itself. Nothing else
|
|
195
|
-
in your response except the box.
|
|
196
|
-
|
|
197
|
-
---
|
|
198
|
-
|
|
199
|
-
## Step 2 — Confirmation (mandatory pause, never skipped)
|
|
200
|
-
|
|
201
|
-
Immediately after the box, append exactly this line (fill in N and M):
|
|
202
|
-
|
|
203
|
-
```
|
|
204
|
-
Found [N] root-cause clusters covering [M] issues. Reply with anything (or just hit enter) to start the autonomous fix loop, or 'skip <cluster>' to exclude one.
|
|
205
|
-
```
|
|
206
|
-
|
|
207
|
-
- **N** = number of root-cause clusters shown in the box.
|
|
208
|
-
- **M** = total issues spanned by those clusters = N + the total symptom count (i.e.
|
|
209
|
-
root causes + symptoms), or simply count every finding listed under every cluster.
|
|
210
|
-
|
|
211
|
-
Even if the scan found only **1** issue, you MUST print this line and stop. This
|
|
212
|
-
confirmation step is non-negotiable.
|
|
213
|
-
|
|
214
|
-
Then **end your turn and wait** for the user's next message. Do not read files, do not plan
|
|
215
|
-
fixes, do not touch anything yet.
|
|
216
|
-
|
|
217
|
-
- If the user's message starts with `stop` / `cancel` / `abort` → do not start the loop;
|
|
218
|
-
go straight to Step 6.
|
|
219
|
-
- If the user's message matches `skip <cluster>` → exclude that cluster from consideration,
|
|
220
|
-
then proceed to Step 3 on the remaining clusters (no new confirmation needed).
|
|
221
|
-
- Any other input (including an empty reply) → confirmation granted, proceed to Step 3.
|
|
222
|
-
|
|
223
|
-
---
|
|
224
|
-
|
|
225
|
-
## Step 3 — Autonomous fix loop (runs after confirmation)
|
|
226
|
-
|
|
227
|
-
For each iteration, do all of (a)–(m) without asking for confirmation again:
|
|
228
|
-
|
|
229
|
-
**a0. Loop-engineering shortcut (preferred for a single finding).** You can delegate one
|
|
230
|
-
finding to OpenPitStop, which will run the full repro→fix→verify loop *for you, looping until
|
|
231
|
-
the fix is actually verified* (the repro passes, or `pitstop verify` is clean) — up to 5
|
|
232
|
-
attempts, feeding each failure back into the next attempt:
|
|
233
|
-
|
|
234
|
-
`!npx openpitstop drive <finding-id>`
|
|
235
|
-
|
|
236
|
-
Use this for a single confirmed root cause instead of hand-running (d)–(j). When it returns
|
|
237
|
-
VERIFIED, the finding is solved and recorded as driven; move to the next cluster. If it returns
|
|
238
|
-
NOT VERIFIED after the attempts, fix by hand or report "requires human review". To drive the
|
|
239
|
-
*whole* repo to fully-fixed with no id: `!npx openpitstop drive` (it repeatedly runs the next
|
|
240
|
-
command from the plan below until nothing remains).
|
|
241
|
-
|
|
242
|
-
**a. Pick the cluster.** Choose the highest-value remaining cluster (most severe, or most
|
|
243
|
-
central). Skip any the user excluded.
|
|
244
|
-
|
|
245
|
-
**b. Branch.** If you are not already on a `pitstop/*` branch, create and switch to:
|
|
246
|
-
|
|
247
|
-
`pitstop/<short-slug>-<date>`
|
|
248
|
-
|
|
249
|
-
where `<short-slug>` is a 2–4 word kebab slug of the cluster (e.g. `circular-core-deps`)
|
|
250
|
-
and `<date>` is `YYYY-MM-DD`. Never branch off or commit to `main`.
|
|
251
|
-
|
|
252
|
-
**c. State your hypothesis.** In one or two sentences, say *why* you believe this cluster's
|
|
253
|
-
root cause is what the scanner claims it is, and what a minimal correct fix looks like.
|
|
254
|
-
|
|
255
|
-
**d. Capture the bug as a failing test — mandatory, never skippable.**
|
|
256
|
-
|
|
257
|
-
`!npx openpitstop repro <finding-id>`
|
|
258
|
-
|
|
259
|
-
`<finding-id>` is the id printed on the cluster's line in the scan box (and stored in
|
|
260
|
-
`.pitstop/scan-latest.json`), e.g. `ledger-3f9a2c01`. OpenPitStop writes a permanent repro
|
|
261
|
-
test (`pitstop-repro-<slug>.test.*`) and runs it:
|
|
262
|
-
|
|
263
|
-
- If it reports **FAIL — bug reproduced**, you have *proven* the bug with a real failing
|
|
264
|
-
test. Good. Proceed.
|
|
265
|
-
- If it reports **PASS**, or **refused** with no genuine repro generator, the hypothesis is
|
|
266
|
-
**UNPROVEN**. Do **not** fix blind. Stop, return to (c), and either revise the hypothesis
|
|
267
|
-
or pick a different cluster.
|
|
268
|
-
|
|
269
|
-
**e. Make the smallest fix.** Using your own file-edit tools, change the minimum needed to
|
|
270
|
-
address the **root cause** (not just a symptom). Stay on the `pitstop/*` branch.
|
|
271
|
-
|
|
272
|
-
**f. Prove the same test now passes — mandatory, before any verify.**
|
|
273
|
-
|
|
274
|
-
`!npx openpitstop repro <finding-id>`
|
|
275
|
-
|
|
276
|
-
Run the **same** repro test. It must now report **PASS**.
|
|
277
|
-
If it still **FAILS**, the fix did not address the root cause: revert, revise the hypothesis
|
|
278
|
-
in (c), and retry once (see (i)).
|
|
279
|
-
|
|
280
|
-
**g. Verify.**
|
|
281
|
-
|
|
282
|
-
`!npx openpitstop verify`
|
|
283
|
-
|
|
284
|
-
Read the result, especially the **Integrity gate** line, the Regression Risk, and the Δ
|
|
285
|
-
columns. This verify run is also the integrity gate: it diffs your uncommitted change against
|
|
286
|
-
HEAD and runs the AI-agent-cheat detectors (deleted/loosened tests, swallowed errors,
|
|
287
|
-
suppressions, hardcoded-to-pass values, forced exits).
|
|
288
|
-
|
|
289
|
-
> After every `pitstop` command OpenPitStop prints a **Next card** with a repo-aware
|
|
290
|
-
> remediation **plan** (it inspects this repo's language, package manager, frameworks, test
|
|
291
|
-
> runner, CI and env files to recommend the exact next commands, in order). Follow its `Next`
|
|
292
|
-
> command — do not guess.
|
|
293
|
-
|
|
294
|
-
**h. Integrity gate — mandatory, never skippable.** Inspect the "Integrity gate:" verdict
|
|
295
|
-
in the verify box and branch exactly like this:
|
|
296
|
-
|
|
297
|
-
- **CLEAN** → proceed to (i) as normal.
|
|
298
|
-
- **SUSPICIOUS** → do **not** commit. Explicitly state in your response what was flagged and
|
|
299
|
-
why (detector, `file:line`, evidence). Revert the change. Retry the **SAME** cluster exactly
|
|
300
|
-
**once**, with this instruction appended verbatim to your working notes:
|
|
301
|
-
|
|
302
|
-
> "Your previous attempt touched a test or added error suppression in a way that looks like it
|
|
303
|
-
> avoided the real fix instead of solving it. Solve the root cause. Do not modify, skip, or
|
|
304
|
-
> loosen any test. Do not add suppression comments or swallow exceptions."
|
|
305
|
-
|
|
306
|
-
Repeat (c)–(g) with that instruction. If the retry is **also SUSPICIOUS**, stop trying this
|
|
307
|
-
cluster, mark it **"requires human review"** in the final report, and move to the next cluster —
|
|
308
|
-
never loop indefinitely on the same pattern.
|
|
309
|
-
- **CONFIRMED_CHEAT** → do **not** commit. State what was flagged. Revert. Skip straight to
|
|
310
|
-
**"requires human review"** with **zero retries** — this tier is unambiguous enough that
|
|
311
|
-
retrying isn't warranted; a human should look at it. Move to the next cluster.
|
|
312
|
-
|
|
313
|
-
**i. If Regression Risk is High** (and the integrity gate was CLEAN): `git checkout` (or
|
|
314
|
-
otherwise revert) your change, note that this hypothesis failed, and try **once more** with a
|
|
315
|
-
different approach for the **same** cluster. Maximum 2 attempts per cluster, then move on.
|
|
316
|
-
|
|
317
|
-
**j. If Risk is Low/Medium, tests are not newly failing, and the integrity gate is CLEAN:**
|
|
318
|
-
commit with a clear message describing the root cause and fix. Commit the repro test **together
|
|
319
|
-
with** the fix — it is a **permanent regression guard**, never a throwaway, and never delete it.
|
|
320
|
-
Then record it:
|
|
321
|
-
|
|
322
|
-
`!npx openpitstop memory add "<finding-id> fixed + proven by pitstop-repro-<slug>" --type fix`
|
|
323
|
-
|
|
324
|
-
(keep the summary short and factual — this is the "six months later" recall.)
|
|
325
|
-
|
|
326
|
-
**k. Re-scan and show a shorter status.** Run the scan again and print a short updated
|
|
327
|
-
boxed status in the same visual style as Step 1, showing: issues fixed so far, issues
|
|
328
|
-
remaining. No long commentary.
|
|
329
|
-
|
|
330
|
-
**k2. Token economy (MANDATORY in every loop iteration).** Your budget is real; follow
|
|
331
|
-
these rules exactly:
|
|
332
|
-
|
|
333
|
-
- **Before any re-scan**, run `!npx openpitstop ready-check`. If it exits 0 (tree
|
|
334
|
-
unchanged), run `!npx openpitstop scan --reuse` instead of a full scan — it returns the
|
|
335
|
-
sealed baseline instantly, and skipping it means burning credits for nothing.
|
|
336
|
-
- **Prefer `!npx openpitstop inspect <id>`** over reading whole files: it shows the exact
|
|
337
|
-
code window, cluster context and repro proof. Read whole files only when inspect cannot
|
|
338
|
-
answer the question.
|
|
339
|
-
- **Batch your edits** — plan the fix, then apply it in as few tool calls as possible.
|
|
340
|
-
One pause per loop (Step 2), never more.
|
|
341
|
-
- **Verify during iteration** as-is; the reliability suite (extra runs) is for the FINAL
|
|
342
|
-
pass — don't run it per-iteration.
|
|
343
|
-
- If you are about to re-scan a second time without any edit having happened, stop and
|
|
344
|
-
re-check: you are burning credits in a loop. Either pick a different cluster or ask.
|
|
345
|
-
|
|
346
|
-
**l. Success check.** If the fresh scan shows **zero remaining actionable clusters**,
|
|
347
|
-
stop — this is the success condition. Your final line should be:
|
|
348
|
-
|
|
349
|
-
`nothing left to fix, nothing broken.`
|
|
350
|
-
|
|
351
|
-
**m. Otherwise repeat.** Go back to (a) automatically. You do **not** ask for confirmation
|
|
352
|
-
again. The loop pauses only once, at Step 2, before the very first fix.
|
|
353
|
-
|
|
354
|
-
---
|
|
355
|
-
|
|
356
|
-
## Step 4 — Hard stop conditions
|
|
357
|
-
|
|
358
|
-
Whichever comes first:
|
|
359
|
-
|
|
360
|
-
- **(a)** a fresh scan shows zero actionable clusters (success), or
|
|
361
|
-
- **(b)** 10 total fix iterations, or
|
|
362
|
-
- **(c)** 45 minutes of wall-clock time.
|
|
363
|
-
|
|
364
|
-
If you stop because of **(b)** or **(c)** rather than **(a)**, say so plainly. Do not imply
|
|
365
|
-
everything is done when it is not. Report how many clusters remain.
|
|
366
|
-
|
|
367
|
-
---
|
|
368
|
-
|
|
369
|
-
## Step 5 — Non-negotiable safety rules
|
|
370
|
-
|
|
371
|
-
- Never force-push. Ever.
|
|
372
|
-
- Never touch `.env`, `.git/`, or any secret/credential file.
|
|
373
|
-
- Never delete a file unless the dependency graph confirms it has zero incoming references.
|
|
374
|
-
- Never silently modify CI/deploy config — if a fix would require it, flag it to the user
|
|
375
|
-
instead and skip that change.
|
|
376
|
-
- Always stay on the `pitstop/*` branch. Leave `main` (and any protected branch) untouched.
|
|
377
|
-
- If a fix feels risky, prefer the smaller safer change; the loop can retry.
|
|
378
|
-
|
|
379
|
-
---
|
|
380
|
-
|
|
381
|
-
## Step 6 — Final report
|
|
382
|
-
|
|
383
|
-
On **any** stop condition (success, max iterations, or timeout), run:
|
|
384
|
-
|
|
385
|
-
`!npx openpitstop report`
|
|
386
|
-
|
|
387
|
-
and present the resulting **`PITSTOP_REPORT.md` / boxed output verbatim** as your final
|
|
388
|
-
message. Do not rewrite or summarize it. The report includes **"Fixes shipped with permanent
|
|
389
|
-
proof"**: one line per committed fix, linking the `pitstop-repro-*.test.*` file that proves
|
|
390
|
-
it — if a fix has no committed repro test, that is a red flag the loop was cut short.
|
|
1
|
+
---
|
|
2
|
+
description: "Autonomous engineering quality loop — full scan/fix loop, or scoped to your custom ask (/pitstop <question>)"
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
# OpenPitStop — Autonomous Engineering Quality Loop
|
|
6
|
+
|
|
7
|
+
> **Read this first — mode selection (MANDATORY, before anything else).**
|
|
8
|
+
>
|
|
9
|
+
> The text the user typed after `/pitstop` is substituted into this prompt where the
|
|
10
|
+
> placeholder appears on the "Invocation arguments:" line just below.
|
|
11
|
+
>
|
|
12
|
+
> **Invocation arguments: `$ARGUMENTS`**
|
|
13
|
+
>
|
|
14
|
+
> Decide the mode **before doing anything else** — before scanning, before reading files,
|
|
15
|
+
> before printing anything — by reading that one line:
|
|
16
|
+
>
|
|
17
|
+
> - It contains one of the flags `--scan-only`, `--demo`, `--ledger`, `--integrity-only`,
|
|
18
|
+
> `--pen`, or `--menu` → **mode = that flag**. You MUST NOT print the menu. Go directly
|
|
19
|
+
> to the matching "## Mode: …" section below.
|
|
20
|
+
> - The line is **empty**, or still shows the literal placeholder word unsubstituted (the
|
|
21
|
+
> exact placeholder text is still visible) → **mode = default full loop**. The user typed
|
|
22
|
+
> bare `/pitstop` and wants the general quality loop — no menu, no waiting. Continue to
|
|
23
|
+
> the "## The default full loop" section below.
|
|
24
|
+
> - It contains **any other free-form text** (a question, a concern, a path, an instruction —
|
|
25
|
+
> e.g. "could you check the security of this app?", "are these tests flaky?", "did my agent
|
|
26
|
+
> cheat on the last commit?") → **mode = custom ask**. The user wants the loop scoped to
|
|
27
|
+
> exactly what they asked, nothing more. Go to the "## Mode: custom ask" section below.
|
|
28
|
+
|
|
29
|
+
## Step 0 — Acknowledge with one line (first message)
|
|
30
|
+
|
|
31
|
+
For the **default full loop**: your entire first message is exactly this one line, nothing
|
|
32
|
+
else — then proceed straight to Step 1. Never print the instructions, the mode list, or any
|
|
33
|
+
block. The instructions above are for you alone; the user just needs the one line:
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
/pitstop — running the quality loop.
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
For the **menu** (`--menu`), the single-shot modes (`--scan-only`, `--demo`, `--ledger`,
|
|
40
|
+
`--integrity-only`, `--pen`), and the **custom ask** mode, the response defined by that
|
|
41
|
+
mode's section **is** your first message — do not add a Step 0 line before it.
|
|
42
|
+
|
|
43
|
+
## Mode: --menu
|
|
44
|
+
|
|
45
|
+
Print **exactly this menu as your entire response**, then **end your turn and wait** for
|
|
46
|
+
the user's next message. Do not scan, do not read files, do not plan anything yet.
|
|
47
|
+
|
|
48
|
+
```
|
|
49
|
+
OpenPitStop modes:
|
|
50
|
+
(enter) — full autonomous loop (scan, confirm, fix, verify, repeat)
|
|
51
|
+
--scan-only — scan and report, no fixes
|
|
52
|
+
--demo — run against OpenPitStop's own seeded demo repo
|
|
53
|
+
--ledger — payment idempotency fuzzing only
|
|
54
|
+
--integrity-only — re-check the last commit for cheat patterns, no scanning
|
|
55
|
+
--pen — penetration test: live attacks + proof + fixes (regression tests, patches)
|
|
56
|
+
(your own ask) — reply with anything else, e.g. "check the security of this app"
|
|
57
|
+
Reply with a mode, your own ask, or just hit enter for the default full loop.
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Then wait. Map the user's next message to a mode:
|
|
61
|
+
|
|
62
|
+
- **Empty reply** (or "default" / "full loop") → the default full loop: continue to the
|
|
63
|
+
"## The default full loop" section below.
|
|
64
|
+
- **`--scan-only`** → the "## Mode: --scan-only" section.
|
|
65
|
+
- **`--demo`** → the "## Mode: --demo" section.
|
|
66
|
+
- **`--ledger`** → the "## Mode: --ledger" section.
|
|
67
|
+
- **`--integrity-only`** → the "## Mode: --integrity-only" section.
|
|
68
|
+
- **`--pen`** → the "## Mode: --pen" section.
|
|
69
|
+
- **Anything else** (a question or phrase) → the "## Mode: custom ask" section.
|
|
70
|
+
|
|
71
|
+
## Mode: --scan-only
|
|
72
|
+
|
|
73
|
+
Run the scan:
|
|
74
|
+
|
|
75
|
+
`!npx openpitstop scan`
|
|
76
|
+
|
|
77
|
+
Print the **entire boxed output verbatim** as your complete response — no summary, no
|
|
78
|
+
commentary, no fixes, no report. Then stop. That is the whole mode.
|
|
79
|
+
|
|
80
|
+
## Mode: --demo
|
|
81
|
+
|
|
82
|
+
Run `!npx openpitstop demo` first; it prints a fresh temp demo repo. Then run the
|
|
83
|
+
**default full loop** (the section below) inside that temp repo — cd there, scan,
|
|
84
|
+
confirm, fix, verify, repeat, as if you had been invoked there.
|
|
85
|
+
|
|
86
|
+
## Mode: --ledger
|
|
87
|
+
|
|
88
|
+
Run the scan as `!npx openpitstop scan --ledger` (this boots the app under a sandbox — the
|
|
89
|
+
nock preload for Node/JS apps, a recording HTTP(S)_PROXY server for Go/Python/Rust/.NET —
|
|
90
|
+
and fuzzes money-moving endpoints for missing idempotency). Then run the
|
|
91
|
+
**default full loop** (the section below) restricted to ledger findings only.
|
|
92
|
+
|
|
93
|
+
## Mode: --integrity-only
|
|
94
|
+
|
|
95
|
+
Run `!npx openpitstop integrity`, print the boxed verdict **verbatim**, and **stop**.
|
|
96
|
+
No scanning, no fixes, no report. That is the whole mode.
|
|
97
|
+
|
|
98
|
+
## Mode: --pen
|
|
99
|
+
|
|
100
|
+
Run the penetration test:
|
|
101
|
+
|
|
102
|
+
`!npx openpitstop pen --fix`
|
|
103
|
+
|
|
104
|
+
This boots the app under a network-interception sandbox and fires live attacks at every
|
|
105
|
+
discovered route. Print the **entire boxed output verbatim**. Then:
|
|
106
|
+
|
|
107
|
+
1. **State the verdict honestly**: for every PROVEN finding (XSS reflection, SSRF canary,
|
|
108
|
+
command-injection spawn, path-traversal file leak) say exactly what was proven and how
|
|
109
|
+
(`pitstop inspect <id>` shows the attack + response + sandbox evidence — use it instead
|
|
110
|
+
of reading whole files).
|
|
111
|
+
2. **Confirmation pause** (the same one mandatory pause as the default loop, never skipped):
|
|
112
|
+
ask the user before fixing anything, listing the finding ids with `--fix` already written
|
|
113
|
+
(`pitstop pen --fix` wrote repro tests + patches + `PITSTOP_PEN_FIXES.md`).
|
|
114
|
+
3. On confirmation, fix **one finding at a time**, each exactly like the default loop:
|
|
115
|
+
- run `!npx openpitstop repro <pen-id>` → must **FAIL** (bug live),
|
|
116
|
+
- make the smallest fix (apply the generated patch with `git apply` when a deterministic
|
|
117
|
+
one exists — `.pitstop/pen-patches/<id>.diff` — and review it before applying),
|
|
118
|
+
- re-run the **same** repro → must **PASS**,
|
|
119
|
+
- `!npx openpitstop verify` → integrity gate CLEAN.
|
|
120
|
+
4. Re-run `!npx openpitstop pen --static` to confirm the finding is gone from the report
|
|
121
|
+
(static re-check; do not re-boot the app needlessly), then `--json` if you want the ids.
|
|
122
|
+
5. Commit each fix with its repro test. Finish with `!npx openpitstop report`.
|
|
123
|
+
|
|
124
|
+
Honesty rule: `pen` proves what it fires. It cannot promise "never hacked" — it promises
|
|
125
|
+
every demonstrable attack gets a regression test that fails on the bug and passes on the fix.
|
|
126
|
+
If `pen --fix` wrote repro tests, never delete them; they are the permanent proof.
|
|
127
|
+
|
|
128
|
+
---
|
|
129
|
+
|
|
130
|
+
## Mode: custom ask
|
|
131
|
+
|
|
132
|
+
The user typed free-form text after `/pitstop` instead of a flag. Run **only what they asked
|
|
133
|
+
for**. Do not expand into the default full loop, do not fix unrelated things.
|
|
134
|
+
|
|
135
|
+
**Step A — Map the ask to a command (before doing anything else).** Read the ask and choose
|
|
136
|
+
the closest match. Never run the full scan just to decide:
|
|
137
|
+
|
|
138
|
+
| The ask is about… | Run this | And |
|
|
139
|
+
|---|---|---|
|
|
140
|
+
| app security / vulnerabilities / "is my app hackable" / "check the security" | `!npx openpitstop pen` (add `--fix` only if they asked you to fix) | report proven/indicated/unproven verdicts honestly |
|
|
141
|
+
| tests, flakiness, coverage, a failing suite | `!npx openpitstop scan` | read the Tests + Reliability lines; no fixes unless asked |
|
|
142
|
+
| "run my tests" / unit tests / integration tests / e2e / the test pyramid / "are my tests passing?" | `!npx openpitstop test` (add `--unit`/`--integration`/`--e2e` if they named one layer) | print the boxed pyramid verbatim; if a layer failed, list the failing tests and offer to fix them (confirmation pause first) |
|
|
143
|
+
| the last commit / whether an agent cheated / "verify my agent's work" | `!npx openpitstop integrity` | print the boxed verdict verbatim and stop |
|
|
144
|
+
| one finding id | `!npx openpitstop inspect <id>` (or `repro <id>` if they want a regression test) | only that finding |
|
|
145
|
+
| one file or route | a scoped `!npx openpitstop scan` / `inspect` on the relevant finding | only that area |
|
|
146
|
+
| overall repo quality / health | `!npx openpitstop scan` | show the box and stop (no fixes unless asked) |
|
|
147
|
+
| one category (duplication, circular imports, secrets…) | `!npx openpitstop scan` | read only that category's lines |
|
|
148
|
+
| anything not in the table | the cheapest read-only check that answers them | ask one clarifying question if still unsure |
|
|
149
|
+
|
|
150
|
+
**Step B — State your interpretation, then confirm before fixing.** Your first message is
|
|
151
|
+
exactly one line:
|
|
152
|
+
|
|
153
|
+
```
|
|
154
|
+
/pitstop — I read that as: <one-line restatement of their ask>. Running <command>.
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
- If the ask is **read-only** (check / report / explain) → run it, print the output, and
|
|
158
|
+
stop. Never fix without being asked.
|
|
159
|
+
- If the ask **clearly requests fixes** ("fix this", "make it pass") → run it, show the
|
|
160
|
+
output, then append one confirmation line — "Found [N] issue(s). Start fixing? Reply
|
|
161
|
+
anything to continue, or tell me what to skip." — and wait.
|
|
162
|
+
- If you are **not sure** → run the cheapest check, show it, and ask one clarifying
|
|
163
|
+
question. Never guess and start editing.
|
|
164
|
+
|
|
165
|
+
**Step C — Fixes (only after confirmation).** Handle each fix exactly like the default
|
|
166
|
+
loop: `!npx openpitstop repro <id>` must **FAIL** first, smallest fix, the **same** repro
|
|
167
|
+
must **PASS**, `!npx openpitstop verify` CLEAN, commit with the repro test. Stop when every
|
|
168
|
+
issue relevant to the ask is gone.
|
|
169
|
+
|
|
170
|
+
**Step D — Scope discipline (mandatory).** Work through ONLY the issues relevant to the
|
|
171
|
+
ask. Leave unrelated clusters alone (say so in one line if they exist), do not write the
|
|
172
|
+
full `PITSTOP_REPORT.md` unless the ask covers the whole repo, and never drift into the
|
|
173
|
+
default full loop.
|
|
174
|
+
|
|
175
|
+
---
|
|
176
|
+
|
|
177
|
+
## The default full loop
|
|
178
|
+
|
|
179
|
+
You are running the `openpitstop` quality loop against this repository. Follow these
|
|
180
|
+
steps **exactly**, in order. Do not improvise around them.
|
|
181
|
+
|
|
182
|
+
The loop has exactly **one** mandatory pause: after the first scan (Step 2), before the
|
|
183
|
+
first fix. After that, you act autonomously until a hard stop condition.
|
|
184
|
+
|
|
185
|
+
---
|
|
186
|
+
|
|
187
|
+
## Step 1 — Scan and show the box
|
|
188
|
+
|
|
189
|
+
Run the scan:
|
|
190
|
+
|
|
191
|
+
`!npx openpitstop scan`
|
|
192
|
+
|
|
193
|
+
Print the **entire boxed output verbatim** as your complete response. Do not summarize
|
|
194
|
+
it, do not add commentary, do not explain it — let the box speak for itself. Nothing else
|
|
195
|
+
in your response except the box.
|
|
196
|
+
|
|
197
|
+
---
|
|
198
|
+
|
|
199
|
+
## Step 2 — Confirmation (mandatory pause, never skipped)
|
|
200
|
+
|
|
201
|
+
Immediately after the box, append exactly this line (fill in N and M):
|
|
202
|
+
|
|
203
|
+
```
|
|
204
|
+
Found [N] root-cause clusters covering [M] issues. Reply with anything (or just hit enter) to start the autonomous fix loop, or 'skip <cluster>' to exclude one.
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
- **N** = number of root-cause clusters shown in the box.
|
|
208
|
+
- **M** = total issues spanned by those clusters = N + the total symptom count (i.e.
|
|
209
|
+
root causes + symptoms), or simply count every finding listed under every cluster.
|
|
210
|
+
|
|
211
|
+
Even if the scan found only **1** issue, you MUST print this line and stop. This
|
|
212
|
+
confirmation step is non-negotiable.
|
|
213
|
+
|
|
214
|
+
Then **end your turn and wait** for the user's next message. Do not read files, do not plan
|
|
215
|
+
fixes, do not touch anything yet.
|
|
216
|
+
|
|
217
|
+
- If the user's message starts with `stop` / `cancel` / `abort` → do not start the loop;
|
|
218
|
+
go straight to Step 6.
|
|
219
|
+
- If the user's message matches `skip <cluster>` → exclude that cluster from consideration,
|
|
220
|
+
then proceed to Step 3 on the remaining clusters (no new confirmation needed).
|
|
221
|
+
- Any other input (including an empty reply) → confirmation granted, proceed to Step 3.
|
|
222
|
+
|
|
223
|
+
---
|
|
224
|
+
|
|
225
|
+
## Step 3 — Autonomous fix loop (runs after confirmation)
|
|
226
|
+
|
|
227
|
+
For each iteration, do all of (a)–(m) without asking for confirmation again:
|
|
228
|
+
|
|
229
|
+
**a0. Loop-engineering shortcut (preferred for a single finding).** You can delegate one
|
|
230
|
+
finding to OpenPitStop, which will run the full repro→fix→verify loop *for you, looping until
|
|
231
|
+
the fix is actually verified* (the repro passes, or `pitstop verify` is clean) — up to 5
|
|
232
|
+
attempts, feeding each failure back into the next attempt:
|
|
233
|
+
|
|
234
|
+
`!npx openpitstop drive <finding-id>`
|
|
235
|
+
|
|
236
|
+
Use this for a single confirmed root cause instead of hand-running (d)–(j). When it returns
|
|
237
|
+
VERIFIED, the finding is solved and recorded as driven; move to the next cluster. If it returns
|
|
238
|
+
NOT VERIFIED after the attempts, fix by hand or report "requires human review". To drive the
|
|
239
|
+
*whole* repo to fully-fixed with no id: `!npx openpitstop drive` (it repeatedly runs the next
|
|
240
|
+
command from the plan below until nothing remains).
|
|
241
|
+
|
|
242
|
+
**a. Pick the cluster.** Choose the highest-value remaining cluster (most severe, or most
|
|
243
|
+
central). Skip any the user excluded.
|
|
244
|
+
|
|
245
|
+
**b. Branch.** If you are not already on a `pitstop/*` branch, create and switch to:
|
|
246
|
+
|
|
247
|
+
`pitstop/<short-slug>-<date>`
|
|
248
|
+
|
|
249
|
+
where `<short-slug>` is a 2–4 word kebab slug of the cluster (e.g. `circular-core-deps`)
|
|
250
|
+
and `<date>` is `YYYY-MM-DD`. Never branch off or commit to `main`.
|
|
251
|
+
|
|
252
|
+
**c. State your hypothesis.** In one or two sentences, say *why* you believe this cluster's
|
|
253
|
+
root cause is what the scanner claims it is, and what a minimal correct fix looks like.
|
|
254
|
+
|
|
255
|
+
**d. Capture the bug as a failing test — mandatory, never skippable.**
|
|
256
|
+
|
|
257
|
+
`!npx openpitstop repro <finding-id>`
|
|
258
|
+
|
|
259
|
+
`<finding-id>` is the id printed on the cluster's line in the scan box (and stored in
|
|
260
|
+
`.pitstop/scan-latest.json`), e.g. `ledger-3f9a2c01`. OpenPitStop writes a permanent repro
|
|
261
|
+
test (`pitstop-repro-<slug>.test.*`) and runs it:
|
|
262
|
+
|
|
263
|
+
- If it reports **FAIL — bug reproduced**, you have *proven* the bug with a real failing
|
|
264
|
+
test. Good. Proceed.
|
|
265
|
+
- If it reports **PASS**, or **refused** with no genuine repro generator, the hypothesis is
|
|
266
|
+
**UNPROVEN**. Do **not** fix blind. Stop, return to (c), and either revise the hypothesis
|
|
267
|
+
or pick a different cluster.
|
|
268
|
+
|
|
269
|
+
**e. Make the smallest fix.** Using your own file-edit tools, change the minimum needed to
|
|
270
|
+
address the **root cause** (not just a symptom). Stay on the `pitstop/*` branch.
|
|
271
|
+
|
|
272
|
+
**f. Prove the same test now passes — mandatory, before any verify.**
|
|
273
|
+
|
|
274
|
+
`!npx openpitstop repro <finding-id>`
|
|
275
|
+
|
|
276
|
+
Run the **same** repro test. It must now report **PASS**.
|
|
277
|
+
If it still **FAILS**, the fix did not address the root cause: revert, revise the hypothesis
|
|
278
|
+
in (c), and retry once (see (i)).
|
|
279
|
+
|
|
280
|
+
**g. Verify.**
|
|
281
|
+
|
|
282
|
+
`!npx openpitstop verify`
|
|
283
|
+
|
|
284
|
+
Read the result, especially the **Integrity gate** line, the Regression Risk, and the Δ
|
|
285
|
+
columns. This verify run is also the integrity gate: it diffs your uncommitted change against
|
|
286
|
+
HEAD and runs the AI-agent-cheat detectors (deleted/loosened tests, swallowed errors,
|
|
287
|
+
suppressions, hardcoded-to-pass values, forced exits).
|
|
288
|
+
|
|
289
|
+
> After every `pitstop` command OpenPitStop prints a **Next card** with a repo-aware
|
|
290
|
+
> remediation **plan** (it inspects this repo's language, package manager, frameworks, test
|
|
291
|
+
> runner, CI and env files to recommend the exact next commands, in order). Follow its `Next`
|
|
292
|
+
> command — do not guess.
|
|
293
|
+
|
|
294
|
+
**h. Integrity gate — mandatory, never skippable.** Inspect the "Integrity gate:" verdict
|
|
295
|
+
in the verify box and branch exactly like this:
|
|
296
|
+
|
|
297
|
+
- **CLEAN** → proceed to (i) as normal.
|
|
298
|
+
- **SUSPICIOUS** → do **not** commit. Explicitly state in your response what was flagged and
|
|
299
|
+
why (detector, `file:line`, evidence). Revert the change. Retry the **SAME** cluster exactly
|
|
300
|
+
**once**, with this instruction appended verbatim to your working notes:
|
|
301
|
+
|
|
302
|
+
> "Your previous attempt touched a test or added error suppression in a way that looks like it
|
|
303
|
+
> avoided the real fix instead of solving it. Solve the root cause. Do not modify, skip, or
|
|
304
|
+
> loosen any test. Do not add suppression comments or swallow exceptions."
|
|
305
|
+
|
|
306
|
+
Repeat (c)–(g) with that instruction. If the retry is **also SUSPICIOUS**, stop trying this
|
|
307
|
+
cluster, mark it **"requires human review"** in the final report, and move to the next cluster —
|
|
308
|
+
never loop indefinitely on the same pattern.
|
|
309
|
+
- **CONFIRMED_CHEAT** → do **not** commit. State what was flagged. Revert. Skip straight to
|
|
310
|
+
**"requires human review"** with **zero retries** — this tier is unambiguous enough that
|
|
311
|
+
retrying isn't warranted; a human should look at it. Move to the next cluster.
|
|
312
|
+
|
|
313
|
+
**i. If Regression Risk is High** (and the integrity gate was CLEAN): `git checkout` (or
|
|
314
|
+
otherwise revert) your change, note that this hypothesis failed, and try **once more** with a
|
|
315
|
+
different approach for the **same** cluster. Maximum 2 attempts per cluster, then move on.
|
|
316
|
+
|
|
317
|
+
**j. If Risk is Low/Medium, tests are not newly failing, and the integrity gate is CLEAN:**
|
|
318
|
+
commit with a clear message describing the root cause and fix. Commit the repro test **together
|
|
319
|
+
with** the fix — it is a **permanent regression guard**, never a throwaway, and never delete it.
|
|
320
|
+
Then record it:
|
|
321
|
+
|
|
322
|
+
`!npx openpitstop memory add "<finding-id> fixed + proven by pitstop-repro-<slug>" --type fix`
|
|
323
|
+
|
|
324
|
+
(keep the summary short and factual — this is the "six months later" recall.)
|
|
325
|
+
|
|
326
|
+
**k. Re-scan and show a shorter status.** Run the scan again and print a short updated
|
|
327
|
+
boxed status in the same visual style as Step 1, showing: issues fixed so far, issues
|
|
328
|
+
remaining. No long commentary.
|
|
329
|
+
|
|
330
|
+
**k2. Token economy (MANDATORY in every loop iteration).** Your budget is real; follow
|
|
331
|
+
these rules exactly:
|
|
332
|
+
|
|
333
|
+
- **Before any re-scan**, run `!npx openpitstop ready-check`. If it exits 0 (tree
|
|
334
|
+
unchanged), run `!npx openpitstop scan --reuse` instead of a full scan — it returns the
|
|
335
|
+
sealed baseline instantly, and skipping it means burning credits for nothing.
|
|
336
|
+
- **Prefer `!npx openpitstop inspect <id>`** over reading whole files: it shows the exact
|
|
337
|
+
code window, cluster context and repro proof. Read whole files only when inspect cannot
|
|
338
|
+
answer the question.
|
|
339
|
+
- **Batch your edits** — plan the fix, then apply it in as few tool calls as possible.
|
|
340
|
+
One pause per loop (Step 2), never more.
|
|
341
|
+
- **Verify during iteration** as-is; the reliability suite (extra runs) is for the FINAL
|
|
342
|
+
pass — don't run it per-iteration.
|
|
343
|
+
- If you are about to re-scan a second time without any edit having happened, stop and
|
|
344
|
+
re-check: you are burning credits in a loop. Either pick a different cluster or ask.
|
|
345
|
+
|
|
346
|
+
**l. Success check.** If the fresh scan shows **zero remaining actionable clusters**,
|
|
347
|
+
stop — this is the success condition. Your final line should be:
|
|
348
|
+
|
|
349
|
+
`nothing left to fix, nothing broken.`
|
|
350
|
+
|
|
351
|
+
**m. Otherwise repeat.** Go back to (a) automatically. You do **not** ask for confirmation
|
|
352
|
+
again. The loop pauses only once, at Step 2, before the very first fix.
|
|
353
|
+
|
|
354
|
+
---
|
|
355
|
+
|
|
356
|
+
## Step 4 — Hard stop conditions
|
|
357
|
+
|
|
358
|
+
Whichever comes first:
|
|
359
|
+
|
|
360
|
+
- **(a)** a fresh scan shows zero actionable clusters (success), or
|
|
361
|
+
- **(b)** 10 total fix iterations, or
|
|
362
|
+
- **(c)** 45 minutes of wall-clock time.
|
|
363
|
+
|
|
364
|
+
If you stop because of **(b)** or **(c)** rather than **(a)**, say so plainly. Do not imply
|
|
365
|
+
everything is done when it is not. Report how many clusters remain.
|
|
366
|
+
|
|
367
|
+
---
|
|
368
|
+
|
|
369
|
+
## Step 5 — Non-negotiable safety rules
|
|
370
|
+
|
|
371
|
+
- Never force-push. Ever.
|
|
372
|
+
- Never touch `.env`, `.git/`, or any secret/credential file.
|
|
373
|
+
- Never delete a file unless the dependency graph confirms it has zero incoming references.
|
|
374
|
+
- Never silently modify CI/deploy config — if a fix would require it, flag it to the user
|
|
375
|
+
instead and skip that change.
|
|
376
|
+
- Always stay on the `pitstop/*` branch. Leave `main` (and any protected branch) untouched.
|
|
377
|
+
- If a fix feels risky, prefer the smaller safer change; the loop can retry.
|
|
378
|
+
|
|
379
|
+
---
|
|
380
|
+
|
|
381
|
+
## Step 6 — Final report
|
|
382
|
+
|
|
383
|
+
On **any** stop condition (success, max iterations, or timeout), run:
|
|
384
|
+
|
|
385
|
+
`!npx openpitstop report`
|
|
386
|
+
|
|
387
|
+
and present the resulting **`PITSTOP_REPORT.md` / boxed output verbatim** as your final
|
|
388
|
+
message. Do not rewrite or summarize it. The report includes **"Fixes shipped with permanent
|
|
389
|
+
proof"**: one line per committed fix, linking the `pitstop-repro-*.test.*` file that proves
|
|
390
|
+
it — if a fix has no committed repro test, that is a red flag the loop was cut short.
|