openpitstop 1.9.0 → 1.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (105) hide show
  1. package/PRIVACY.md +77 -90
  2. package/README.md +365 -887
  3. package/dist/agentBudget.d.ts +33 -0
  4. package/dist/agentBudget.js +165 -0
  5. package/dist/agentBudget.js.map +1 -0
  6. package/dist/analyzers/ledger/index.d.ts +2 -1
  7. package/dist/analyzers/ledger/index.js +10 -1
  8. package/dist/analyzers/ledger/index.js.map +1 -1
  9. package/dist/analyzers/types.d.ts +2 -0
  10. package/dist/candidate.d.ts +20 -0
  11. package/dist/candidate.js +87 -0
  12. package/dist/candidate.js.map +1 -0
  13. package/dist/cli.js +1 -1
  14. package/dist/cli.js.map +1 -1
  15. package/dist/commands/ask.d.ts +0 -8
  16. package/dist/commands/ask.js +36 -34
  17. package/dist/commands/ask.js.map +1 -1
  18. package/dist/commands/budget.d.ts +1 -0
  19. package/dist/commands/budget.js +12 -3
  20. package/dist/commands/budget.js.map +1 -1
  21. package/dist/commands/drive.js +77 -51
  22. package/dist/commands/drive.js.map +1 -1
  23. package/dist/commands/fix.d.ts +7 -0
  24. package/dist/commands/fix.js +90 -108
  25. package/dist/commands/fix.js.map +1 -1
  26. package/dist/commands/gate.js +6 -2
  27. package/dist/commands/gate.js.map +1 -1
  28. package/dist/commands/pen.d.ts +3 -1
  29. package/dist/commands/pen.js +31 -42
  30. package/dist/commands/pen.js.map +1 -1
  31. package/dist/commands/readyCheck.d.ts +1 -1
  32. package/dist/commands/readyCheck.js +6 -36
  33. package/dist/commands/readyCheck.js.map +1 -1
  34. package/dist/commands/scan.d.ts +6 -5
  35. package/dist/commands/scan.js +26 -28
  36. package/dist/commands/scan.js.map +1 -1
  37. package/dist/commands/verify.d.ts +3 -1
  38. package/dist/commands/verify.js +5 -1
  39. package/dist/commands/verify.js.map +1 -1
  40. package/dist/evidence.js +4 -0
  41. package/dist/evidence.js.map +1 -1
  42. package/dist/installer/targets.js +1 -1
  43. package/dist/installer/targets.js.map +1 -1
  44. package/dist/intent.d.ts +4 -7
  45. package/dist/intent.js +41 -77
  46. package/dist/intent.js.map +1 -1
  47. package/dist/pen/dynamic.d.ts +1 -24
  48. package/dist/pen/dynamic.js +10 -1
  49. package/dist/pen/dynamic.js.map +1 -1
  50. package/dist/pen/fix.js +5 -1
  51. package/dist/pen/fix.js.map +1 -1
  52. package/dist/repro/generate.js +1 -0
  53. package/dist/repro/generate.js.map +1 -1
  54. package/dist/repro/pen.js +1 -0
  55. package/dist/repro/pen.js.map +1 -1
  56. package/dist/repro/run.d.ts +1 -0
  57. package/dist/repro/run.js +17 -1
  58. package/dist/repro/run.js.map +1 -1
  59. package/dist/sandbox/container.d.ts +9 -0
  60. package/dist/sandbox/container.js +129 -0
  61. package/dist/sandbox/container.js.map +1 -0
  62. package/dist/sandbox/worker.d.ts +1 -0
  63. package/dist/sandbox/worker.js +41 -0
  64. package/dist/sandbox/worker.js.map +1 -0
  65. package/dist/scanCache.d.ts +27 -0
  66. package/dist/scanCache.js +89 -0
  67. package/dist/scanCache.js.map +1 -0
  68. package/dist/verify/acceptance.d.ts +5 -1
  69. package/dist/verify/acceptance.js +29 -16
  70. package/dist/verify/acceptance.js.map +1 -1
  71. package/dist/verify/architecture.d.ts +5 -1
  72. package/dist/verify/architecture.js +6 -1
  73. package/dist/verify/architecture.js.map +1 -1
  74. package/dist/verify/baseline.d.ts +5 -1
  75. package/dist/verify/baseline.js +5 -1
  76. package/dist/verify/baseline.js.map +1 -1
  77. package/dist/verify/chain.js +17 -0
  78. package/dist/verify/chain.js.map +1 -1
  79. package/dist/verify/flow.d.ts +2 -0
  80. package/dist/verify/flow.js +18 -3
  81. package/dist/verify/flow.js.map +1 -1
  82. package/dist/verify/gateMatrix.d.ts +4 -0
  83. package/dist/verify/gateMatrix.js +23 -12
  84. package/dist/verify/gateMatrix.js.map +1 -1
  85. package/dist/verify/holdout.d.ts +5 -1
  86. package/dist/verify/holdout.js +8 -2
  87. package/dist/verify/holdout.js.map +1 -1
  88. package/dist/verify/regression.d.ts +5 -1
  89. package/dist/verify/regression.js +6 -1
  90. package/dist/verify/regression.js.map +1 -1
  91. package/dist/verify/stack.d.ts +5 -1
  92. package/dist/verify/stack.js +6 -1
  93. package/dist/verify/stack.js.map +1 -1
  94. package/dist/verify/state.d.ts +5 -1
  95. package/dist/verify/state.js +6 -1
  96. package/dist/verify/state.js.map +1 -1
  97. package/dist/verify/verifier.d.ts +5 -1
  98. package/dist/verify/verifier.js +6 -1
  99. package/dist/verify/verifier.js.map +1 -1
  100. package/docs/release-controls.md +107 -0
  101. package/package.json +70 -69
  102. package/templates/pitstop.command.md +17 -5
  103. package/templates/pitstop.prompt.md +390 -390
  104. package/scripts/__pycache__/gen-gifs.cpython-313.pyc +0 -0
  105. package/scripts/gen-gifs.py +0 -231
@@ -1,390 +1,390 @@
1
- ---
2
- description: "Autonomous engineering quality loop — full scan/fix loop, or scoped to your custom ask (/pitstop <question>)"
3
- ---
4
-
5
- # OpenPitStop — Autonomous Engineering Quality Loop
6
-
7
- > **Read this first — mode selection (MANDATORY, before anything else).**
8
- >
9
- > The text the user typed after `/pitstop` is substituted into this prompt where the
10
- > placeholder appears on the "Invocation arguments:" line just below.
11
- >
12
- > **Invocation arguments: `$ARGUMENTS`**
13
- >
14
- > Decide the mode **before doing anything else** — before scanning, before reading files,
15
- > before printing anything — by reading that one line:
16
- >
17
- > - It contains one of the flags `--scan-only`, `--demo`, `--ledger`, `--integrity-only`,
18
- > `--pen`, or `--menu` → **mode = that flag**. You MUST NOT print the menu. Go directly
19
- > to the matching "## Mode: …" section below.
20
- > - The line is **empty**, or still shows the literal placeholder word unsubstituted (the
21
- > exact placeholder text is still visible) → **mode = default full loop**. The user typed
22
- > bare `/pitstop` and wants the general quality loop — no menu, no waiting. Continue to
23
- > the "## The default full loop" section below.
24
- > - It contains **any other free-form text** (a question, a concern, a path, an instruction —
25
- > e.g. "could you check the security of this app?", "are these tests flaky?", "did my agent
26
- > cheat on the last commit?") → **mode = custom ask**. The user wants the loop scoped to
27
- > exactly what they asked, nothing more. Go to the "## Mode: custom ask" section below.
28
-
29
- ## Step 0 — Acknowledge with one line (first message)
30
-
31
- For the **default full loop**: your entire first message is exactly this one line, nothing
32
- else — then proceed straight to Step 1. Never print the instructions, the mode list, or any
33
- block. The instructions above are for you alone; the user just needs the one line:
34
-
35
- ```
36
- /pitstop — running the quality loop.
37
- ```
38
-
39
- For the **menu** (`--menu`), the single-shot modes (`--scan-only`, `--demo`, `--ledger`,
40
- `--integrity-only`, `--pen`), and the **custom ask** mode, the response defined by that
41
- mode's section **is** your first message — do not add a Step 0 line before it.
42
-
43
- ## Mode: --menu
44
-
45
- Print **exactly this menu as your entire response**, then **end your turn and wait** for
46
- the user's next message. Do not scan, do not read files, do not plan anything yet.
47
-
48
- ```
49
- OpenPitStop modes:
50
- (enter) — full autonomous loop (scan, confirm, fix, verify, repeat)
51
- --scan-only — scan and report, no fixes
52
- --demo — run against OpenPitStop's own seeded demo repo
53
- --ledger — payment idempotency fuzzing only
54
- --integrity-only — re-check the last commit for cheat patterns, no scanning
55
- --pen — penetration test: live attacks + proof + fixes (regression tests, patches)
56
- (your own ask) — reply with anything else, e.g. "check the security of this app"
57
- Reply with a mode, your own ask, or just hit enter for the default full loop.
58
- ```
59
-
60
- Then wait. Map the user's next message to a mode:
61
-
62
- - **Empty reply** (or "default" / "full loop") → the default full loop: continue to the
63
- "## The default full loop" section below.
64
- - **`--scan-only`** → the "## Mode: --scan-only" section.
65
- - **`--demo`** → the "## Mode: --demo" section.
66
- - **`--ledger`** → the "## Mode: --ledger" section.
67
- - **`--integrity-only`** → the "## Mode: --integrity-only" section.
68
- - **`--pen`** → the "## Mode: --pen" section.
69
- - **Anything else** (a question or phrase) → the "## Mode: custom ask" section.
70
-
71
- ## Mode: --scan-only
72
-
73
- Run the scan:
74
-
75
- `!npx openpitstop scan`
76
-
77
- Print the **entire boxed output verbatim** as your complete response — no summary, no
78
- commentary, no fixes, no report. Then stop. That is the whole mode.
79
-
80
- ## Mode: --demo
81
-
82
- Run `!npx openpitstop demo` first; it prints a fresh temp demo repo. Then run the
83
- **default full loop** (the section below) inside that temp repo — cd there, scan,
84
- confirm, fix, verify, repeat, as if you had been invoked there.
85
-
86
- ## Mode: --ledger
87
-
88
- Run the scan as `!npx openpitstop scan --ledger` (this boots the app under a sandbox — the
89
- nock preload for Node/JS apps, a recording HTTP(S)_PROXY server for Go/Python/Rust/.NET —
90
- and fuzzes money-moving endpoints for missing idempotency). Then run the
91
- **default full loop** (the section below) restricted to ledger findings only.
92
-
93
- ## Mode: --integrity-only
94
-
95
- Run `!npx openpitstop integrity`, print the boxed verdict **verbatim**, and **stop**.
96
- No scanning, no fixes, no report. That is the whole mode.
97
-
98
- ## Mode: --pen
99
-
100
- Run the penetration test:
101
-
102
- `!npx openpitstop pen --fix`
103
-
104
- This boots the app under a network-interception sandbox and fires live attacks at every
105
- discovered route. Print the **entire boxed output verbatim**. Then:
106
-
107
- 1. **State the verdict honestly**: for every PROVEN finding (XSS reflection, SSRF canary,
108
- command-injection spawn, path-traversal file leak) say exactly what was proven and how
109
- (`pitstop inspect <id>` shows the attack + response + sandbox evidence — use it instead
110
- of reading whole files).
111
- 2. **Confirmation pause** (the same one mandatory pause as the default loop, never skipped):
112
- ask the user before fixing anything, listing the finding ids with `--fix` already written
113
- (`pitstop pen --fix` wrote repro tests + patches + `PITSTOP_PEN_FIXES.md`).
114
- 3. On confirmation, fix **one finding at a time**, each exactly like the default loop:
115
- - run `!npx openpitstop repro <pen-id>` → must **FAIL** (bug live),
116
- - make the smallest fix (apply the generated patch with `git apply` when a deterministic
117
- one exists — `.pitstop/pen-patches/<id>.diff` — and review it before applying),
118
- - re-run the **same** repro → must **PASS**,
119
- - `!npx openpitstop verify` → integrity gate CLEAN.
120
- 4. Re-run `!npx openpitstop pen --static` to confirm the finding is gone from the report
121
- (static re-check; do not re-boot the app needlessly), then `--json` if you want the ids.
122
- 5. Commit each fix with its repro test. Finish with `!npx openpitstop report`.
123
-
124
- Honesty rule: `pen` proves what it fires. It cannot promise "never hacked" — it promises
125
- every demonstrable attack gets a regression test that fails on the bug and passes on the fix.
126
- If `pen --fix` wrote repro tests, never delete them; they are the permanent proof.
127
-
128
- ---
129
-
130
- ## Mode: custom ask
131
-
132
- The user typed free-form text after `/pitstop` instead of a flag. Run **only what they asked
133
- for**. Do not expand into the default full loop, do not fix unrelated things.
134
-
135
- **Step A — Map the ask to a command (before doing anything else).** Read the ask and choose
136
- the closest match. Never run the full scan just to decide:
137
-
138
- | The ask is about… | Run this | And |
139
- |---|---|---|
140
- | app security / vulnerabilities / "is my app hackable" / "check the security" | `!npx openpitstop pen` (add `--fix` only if they asked you to fix) | report proven/indicated/unproven verdicts honestly |
141
- | tests, flakiness, coverage, a failing suite | `!npx openpitstop scan` | read the Tests + Reliability lines; no fixes unless asked |
142
- | "run my tests" / unit tests / integration tests / e2e / the test pyramid / "are my tests passing?" | `!npx openpitstop test` (add `--unit`/`--integration`/`--e2e` if they named one layer) | print the boxed pyramid verbatim; if a layer failed, list the failing tests and offer to fix them (confirmation pause first) |
143
- | the last commit / whether an agent cheated / "verify my agent's work" | `!npx openpitstop integrity` | print the boxed verdict verbatim and stop |
144
- | one finding id | `!npx openpitstop inspect <id>` (or `repro <id>` if they want a regression test) | only that finding |
145
- | one file or route | a scoped `!npx openpitstop scan` / `inspect` on the relevant finding | only that area |
146
- | overall repo quality / health | `!npx openpitstop scan` | show the box and stop (no fixes unless asked) |
147
- | one category (duplication, circular imports, secrets…) | `!npx openpitstop scan` | read only that category's lines |
148
- | anything not in the table | the cheapest read-only check that answers them | ask one clarifying question if still unsure |
149
-
150
- **Step B — State your interpretation, then confirm before fixing.** Your first message is
151
- exactly one line:
152
-
153
- ```
154
- /pitstop — I read that as: <one-line restatement of their ask>. Running <command>.
155
- ```
156
-
157
- - If the ask is **read-only** (check / report / explain) → run it, print the output, and
158
- stop. Never fix without being asked.
159
- - If the ask **clearly requests fixes** ("fix this", "make it pass") → run it, show the
160
- output, then append one confirmation line — "Found [N] issue(s). Start fixing? Reply
161
- anything to continue, or tell me what to skip." — and wait.
162
- - If you are **not sure** → run the cheapest check, show it, and ask one clarifying
163
- question. Never guess and start editing.
164
-
165
- **Step C — Fixes (only after confirmation).** Handle each fix exactly like the default
166
- loop: `!npx openpitstop repro <id>` must **FAIL** first, smallest fix, the **same** repro
167
- must **PASS**, `!npx openpitstop verify` CLEAN, commit with the repro test. Stop when every
168
- issue relevant to the ask is gone.
169
-
170
- **Step D — Scope discipline (mandatory).** Work through ONLY the issues relevant to the
171
- ask. Leave unrelated clusters alone (say so in one line if they exist), do not write the
172
- full `PITSTOP_REPORT.md` unless the ask covers the whole repo, and never drift into the
173
- default full loop.
174
-
175
- ---
176
-
177
- ## The default full loop
178
-
179
- You are running the `openpitstop` quality loop against this repository. Follow these
180
- steps **exactly**, in order. Do not improvise around them.
181
-
182
- The loop has exactly **one** mandatory pause: after the first scan (Step 2), before the
183
- first fix. After that, you act autonomously until a hard stop condition.
184
-
185
- ---
186
-
187
- ## Step 1 — Scan and show the box
188
-
189
- Run the scan:
190
-
191
- `!npx openpitstop scan`
192
-
193
- Print the **entire boxed output verbatim** as your complete response. Do not summarize
194
- it, do not add commentary, do not explain it — let the box speak for itself. Nothing else
195
- in your response except the box.
196
-
197
- ---
198
-
199
- ## Step 2 — Confirmation (mandatory pause, never skipped)
200
-
201
- Immediately after the box, append exactly this line (fill in N and M):
202
-
203
- ```
204
- Found [N] root-cause clusters covering [M] issues. Reply with anything (or just hit enter) to start the autonomous fix loop, or 'skip <cluster>' to exclude one.
205
- ```
206
-
207
- - **N** = number of root-cause clusters shown in the box.
208
- - **M** = total issues spanned by those clusters = N + the total symptom count (i.e.
209
- root causes + symptoms), or simply count every finding listed under every cluster.
210
-
211
- Even if the scan found only **1** issue, you MUST print this line and stop. This
212
- confirmation step is non-negotiable.
213
-
214
- Then **end your turn and wait** for the user's next message. Do not read files, do not plan
215
- fixes, do not touch anything yet.
216
-
217
- - If the user's message starts with `stop` / `cancel` / `abort` → do not start the loop;
218
- go straight to Step 6.
219
- - If the user's message matches `skip <cluster>` → exclude that cluster from consideration,
220
- then proceed to Step 3 on the remaining clusters (no new confirmation needed).
221
- - Any other input (including an empty reply) → confirmation granted, proceed to Step 3.
222
-
223
- ---
224
-
225
- ## Step 3 — Autonomous fix loop (runs after confirmation)
226
-
227
- For each iteration, do all of (a)–(m) without asking for confirmation again:
228
-
229
- **a0. Loop-engineering shortcut (preferred for a single finding).** You can delegate one
230
- finding to OpenPitStop, which will run the full repro→fix→verify loop *for you, looping until
231
- the fix is actually verified* (the repro passes, or `pitstop verify` is clean) — up to 5
232
- attempts, feeding each failure back into the next attempt:
233
-
234
- `!npx openpitstop drive <finding-id>`
235
-
236
- Use this for a single confirmed root cause instead of hand-running (d)–(j). When it returns
237
- VERIFIED, the finding is solved and recorded as driven; move to the next cluster. If it returns
238
- NOT VERIFIED after the attempts, fix by hand or report "requires human review". To drive the
239
- *whole* repo to fully-fixed with no id: `!npx openpitstop drive` (it repeatedly runs the next
240
- command from the plan below until nothing remains).
241
-
242
- **a. Pick the cluster.** Choose the highest-value remaining cluster (most severe, or most
243
- central). Skip any the user excluded.
244
-
245
- **b. Branch.** If you are not already on a `pitstop/*` branch, create and switch to:
246
-
247
- `pitstop/<short-slug>-<date>`
248
-
249
- where `<short-slug>` is a 2–4 word kebab slug of the cluster (e.g. `circular-core-deps`)
250
- and `<date>` is `YYYY-MM-DD`. Never branch off or commit to `main`.
251
-
252
- **c. State your hypothesis.** In one or two sentences, say *why* you believe this cluster's
253
- root cause is what the scanner claims it is, and what a minimal correct fix looks like.
254
-
255
- **d. Capture the bug as a failing test — mandatory, never skippable.**
256
-
257
- `!npx openpitstop repro <finding-id>`
258
-
259
- `<finding-id>` is the id printed on the cluster's line in the scan box (and stored in
260
- `.pitstop/scan-latest.json`), e.g. `ledger-3f9a2c01`. OpenPitStop writes a permanent repro
261
- test (`pitstop-repro-<slug>.test.*`) and runs it:
262
-
263
- - If it reports **FAIL — bug reproduced**, you have *proven* the bug with a real failing
264
- test. Good. Proceed.
265
- - If it reports **PASS**, or **refused** with no genuine repro generator, the hypothesis is
266
- **UNPROVEN**. Do **not** fix blind. Stop, return to (c), and either revise the hypothesis
267
- or pick a different cluster.
268
-
269
- **e. Make the smallest fix.** Using your own file-edit tools, change the minimum needed to
270
- address the **root cause** (not just a symptom). Stay on the `pitstop/*` branch.
271
-
272
- **f. Prove the same test now passes — mandatory, before any verify.**
273
-
274
- `!npx openpitstop repro <finding-id>`
275
-
276
- Run the **same** repro test. It must now report **PASS**.
277
- If it still **FAILS**, the fix did not address the root cause: revert, revise the hypothesis
278
- in (c), and retry once (see (i)).
279
-
280
- **g. Verify.**
281
-
282
- `!npx openpitstop verify`
283
-
284
- Read the result, especially the **Integrity gate** line, the Regression Risk, and the Δ
285
- columns. This verify run is also the integrity gate: it diffs your uncommitted change against
286
- HEAD and runs the AI-agent-cheat detectors (deleted/loosened tests, swallowed errors,
287
- suppressions, hardcoded-to-pass values, forced exits).
288
-
289
- > After every `pitstop` command OpenPitStop prints a **Next card** with a repo-aware
290
- > remediation **plan** (it inspects this repo's language, package manager, frameworks, test
291
- > runner, CI and env files to recommend the exact next commands, in order). Follow its `Next`
292
- > command — do not guess.
293
-
294
- **h. Integrity gate — mandatory, never skippable.** Inspect the "Integrity gate:" verdict
295
- in the verify box and branch exactly like this:
296
-
297
- - **CLEAN** → proceed to (i) as normal.
298
- - **SUSPICIOUS** → do **not** commit. Explicitly state in your response what was flagged and
299
- why (detector, `file:line`, evidence). Revert the change. Retry the **SAME** cluster exactly
300
- **once**, with this instruction appended verbatim to your working notes:
301
-
302
- > "Your previous attempt touched a test or added error suppression in a way that looks like it
303
- > avoided the real fix instead of solving it. Solve the root cause. Do not modify, skip, or
304
- > loosen any test. Do not add suppression comments or swallow exceptions."
305
-
306
- Repeat (c)–(g) with that instruction. If the retry is **also SUSPICIOUS**, stop trying this
307
- cluster, mark it **"requires human review"** in the final report, and move to the next cluster —
308
- never loop indefinitely on the same pattern.
309
- - **CONFIRMED_CHEAT** → do **not** commit. State what was flagged. Revert. Skip straight to
310
- **"requires human review"** with **zero retries** — this tier is unambiguous enough that
311
- retrying isn't warranted; a human should look at it. Move to the next cluster.
312
-
313
- **i. If Regression Risk is High** (and the integrity gate was CLEAN): `git checkout` (or
314
- otherwise revert) your change, note that this hypothesis failed, and try **once more** with a
315
- different approach for the **same** cluster. Maximum 2 attempts per cluster, then move on.
316
-
317
- **j. If Risk is Low/Medium, tests are not newly failing, and the integrity gate is CLEAN:**
318
- commit with a clear message describing the root cause and fix. Commit the repro test **together
319
- with** the fix — it is a **permanent regression guard**, never a throwaway, and never delete it.
320
- Then record it:
321
-
322
- `!npx openpitstop memory add "<finding-id> fixed + proven by pitstop-repro-<slug>" --type fix`
323
-
324
- (keep the summary short and factual — this is the "six months later" recall.)
325
-
326
- **k. Re-scan and show a shorter status.** Run the scan again and print a short updated
327
- boxed status in the same visual style as Step 1, showing: issues fixed so far, issues
328
- remaining. No long commentary.
329
-
330
- **k2. Token economy (MANDATORY in every loop iteration).** Your budget is real; follow
331
- these rules exactly:
332
-
333
- - **Before any re-scan**, run `!npx openpitstop ready-check`. If it exits 0 (tree
334
- unchanged), run `!npx openpitstop scan --reuse` instead of a full scan — it returns the
335
- sealed baseline instantly, and skipping it means burning credits for nothing.
336
- - **Prefer `!npx openpitstop inspect <id>`** over reading whole files: it shows the exact
337
- code window, cluster context and repro proof. Read whole files only when inspect cannot
338
- answer the question.
339
- - **Batch your edits** — plan the fix, then apply it in as few tool calls as possible.
340
- One pause per loop (Step 2), never more.
341
- - **Verify during iteration** as-is; the reliability suite (extra runs) is for the FINAL
342
- pass — don't run it per-iteration.
343
- - If you are about to re-scan a second time without any edit having happened, stop and
344
- re-check: you are burning credits in a loop. Either pick a different cluster or ask.
345
-
346
- **l. Success check.** If the fresh scan shows **zero remaining actionable clusters**,
347
- stop — this is the success condition. Your final line should be:
348
-
349
- `nothing left to fix, nothing broken.`
350
-
351
- **m. Otherwise repeat.** Go back to (a) automatically. You do **not** ask for confirmation
352
- again. The loop pauses only once, at Step 2, before the very first fix.
353
-
354
- ---
355
-
356
- ## Step 4 — Hard stop conditions
357
-
358
- Whichever comes first:
359
-
360
- - **(a)** a fresh scan shows zero actionable clusters (success), or
361
- - **(b)** 10 total fix iterations, or
362
- - **(c)** 45 minutes of wall-clock time.
363
-
364
- If you stop because of **(b)** or **(c)** rather than **(a)**, say so plainly. Do not imply
365
- everything is done when it is not. Report how many clusters remain.
366
-
367
- ---
368
-
369
- ## Step 5 — Non-negotiable safety rules
370
-
371
- - Never force-push. Ever.
372
- - Never touch `.env`, `.git/`, or any secret/credential file.
373
- - Never delete a file unless the dependency graph confirms it has zero incoming references.
374
- - Never silently modify CI/deploy config — if a fix would require it, flag it to the user
375
- instead and skip that change.
376
- - Always stay on the `pitstop/*` branch. Leave `main` (and any protected branch) untouched.
377
- - If a fix feels risky, prefer the smaller safer change; the loop can retry.
378
-
379
- ---
380
-
381
- ## Step 6 — Final report
382
-
383
- On **any** stop condition (success, max iterations, or timeout), run:
384
-
385
- `!npx openpitstop report`
386
-
387
- and present the resulting **`PITSTOP_REPORT.md` / boxed output verbatim** as your final
388
- message. Do not rewrite or summarize it. The report includes **"Fixes shipped with permanent
389
- proof"**: one line per committed fix, linking the `pitstop-repro-*.test.*` file that proves
390
- it — if a fix has no committed repro test, that is a red flag the loop was cut short.
1
+ ---
2
+ description: "Autonomous engineering quality loop — full scan/fix loop, or scoped to your custom ask (/pitstop <question>)"
3
+ ---
4
+
5
+ # OpenPitStop — Autonomous Engineering Quality Loop
6
+
7
+ > **Read this first — mode selection (MANDATORY, before anything else).**
8
+ >
9
+ > The text the user typed after `/pitstop` is substituted into this prompt where the
10
+ > placeholder appears on the "Invocation arguments:" line just below.
11
+ >
12
+ > **Invocation arguments: `$ARGUMENTS`**
13
+ >
14
+ > Decide the mode **before doing anything else** — before scanning, before reading files,
15
+ > before printing anything — by reading that one line:
16
+ >
17
+ > - It contains one of the flags `--scan-only`, `--demo`, `--ledger`, `--integrity-only`,
18
+ > `--pen`, or `--menu` → **mode = that flag**. You MUST NOT print the menu. Go directly
19
+ > to the matching "## Mode: …" section below.
20
+ > - The line is **empty**, or still shows the literal placeholder word unsubstituted (the
21
+ > exact placeholder text is still visible) → **mode = default full loop**. The user typed
22
+ > bare `/pitstop` and wants the general quality loop — no menu, no waiting. Continue to
23
+ > the "## The default full loop" section below.
24
+ > - It contains **any other free-form text** (a question, a concern, a path, an instruction —
25
+ > e.g. "could you check the security of this app?", "are these tests flaky?", "did my agent
26
+ > cheat on the last commit?") → **mode = custom ask**. The user wants the loop scoped to
27
+ > exactly what they asked, nothing more. Go to the "## Mode: custom ask" section below.
28
+
29
+ ## Step 0 — Acknowledge with one line (first message)
30
+
31
+ For the **default full loop**: your entire first message is exactly this one line, nothing
32
+ else — then proceed straight to Step 1. Never print the instructions, the mode list, or any
33
+ block. The instructions above are for you alone; the user just needs the one line:
34
+
35
+ ```
36
+ /pitstop — running the quality loop.
37
+ ```
38
+
39
+ For the **menu** (`--menu`), the single-shot modes (`--scan-only`, `--demo`, `--ledger`,
40
+ `--integrity-only`, `--pen`), and the **custom ask** mode, the response defined by that
41
+ mode's section **is** your first message — do not add a Step 0 line before it.
42
+
43
+ ## Mode: --menu
44
+
45
+ Print **exactly this menu as your entire response**, then **end your turn and wait** for
46
+ the user's next message. Do not scan, do not read files, do not plan anything yet.
47
+
48
+ ```
49
+ OpenPitStop modes:
50
+ (enter) — full autonomous loop (scan, confirm, fix, verify, repeat)
51
+ --scan-only — scan and report, no fixes
52
+ --demo — run against OpenPitStop's own seeded demo repo
53
+ --ledger — payment idempotency fuzzing only
54
+ --integrity-only — re-check the last commit for cheat patterns, no scanning
55
+ --pen — penetration test: live attacks + proof + fixes (regression tests, patches)
56
+ (your own ask) — reply with anything else, e.g. "check the security of this app"
57
+ Reply with a mode, your own ask, or just hit enter for the default full loop.
58
+ ```
59
+
60
+ Then wait. Map the user's next message to a mode:
61
+
62
+ - **Empty reply** (or "default" / "full loop") → the default full loop: continue to the
63
+ "## The default full loop" section below.
64
+ - **`--scan-only`** → the "## Mode: --scan-only" section.
65
+ - **`--demo`** → the "## Mode: --demo" section.
66
+ - **`--ledger`** → the "## Mode: --ledger" section.
67
+ - **`--integrity-only`** → the "## Mode: --integrity-only" section.
68
+ - **`--pen`** → the "## Mode: --pen" section.
69
+ - **Anything else** (a question or phrase) → the "## Mode: custom ask" section.
70
+
71
+ ## Mode: --scan-only
72
+
73
+ Run the scan:
74
+
75
+ `!npx openpitstop scan`
76
+
77
+ Print the **entire boxed output verbatim** as your complete response — no summary, no
78
+ commentary, no fixes, no report. Then stop. That is the whole mode.
79
+
80
+ ## Mode: --demo
81
+
82
+ Run `!npx openpitstop demo` first; it prints a fresh temp demo repo. Then run the
83
+ **default full loop** (the section below) inside that temp repo — cd there, scan,
84
+ confirm, fix, verify, repeat, as if you had been invoked there.
85
+
86
+ ## Mode: --ledger
87
+
88
+ Run the scan as `!npx openpitstop scan --ledger` (this boots the app under a sandbox — the
89
+ nock preload for Node/JS apps, a recording HTTP(S)_PROXY server for Go/Python/Rust/.NET —
90
+ and fuzzes money-moving endpoints for missing idempotency). Then run the
91
+ **default full loop** (the section below) restricted to ledger findings only.
92
+
93
+ ## Mode: --integrity-only
94
+
95
+ Run `!npx openpitstop integrity`, print the boxed verdict **verbatim**, and **stop**.
96
+ No scanning, no fixes, no report. That is the whole mode.
97
+
98
+ ## Mode: --pen
99
+
100
+ Run the penetration test:
101
+
102
+ `!npx openpitstop pen --fix`
103
+
104
+ This boots the app under a network-interception sandbox and fires live attacks at every
105
+ discovered route. Print the **entire boxed output verbatim**. Then:
106
+
107
+ 1. **State the verdict honestly**: for every PROVEN finding (XSS reflection, SSRF canary,
108
+ command-injection spawn, path-traversal file leak) say exactly what was proven and how
109
+ (`pitstop inspect <id>` shows the attack + response + sandbox evidence — use it instead
110
+ of reading whole files).
111
+ 2. **Confirmation pause** (the same one mandatory pause as the default loop, never skipped):
112
+ ask the user before fixing anything, listing the finding ids with `--fix` already written
113
+ (`pitstop pen --fix` wrote repro tests + patches + `PITSTOP_PEN_FIXES.md`).
114
+ 3. On confirmation, fix **one finding at a time**, each exactly like the default loop:
115
+ - run `!npx openpitstop repro <pen-id>` → must **FAIL** (bug live),
116
+ - make the smallest fix (apply the generated patch with `git apply` when a deterministic
117
+ one exists — `.pitstop/pen-patches/<id>.diff` — and review it before applying),
118
+ - re-run the **same** repro → must **PASS**,
119
+ - `!npx openpitstop verify` → integrity gate CLEAN.
120
+ 4. Re-run `!npx openpitstop pen --static` to confirm the finding is gone from the report
121
+ (static re-check; do not re-boot the app needlessly), then `--json` if you want the ids.
122
+ 5. Commit each fix with its repro test. Finish with `!npx openpitstop report`.
123
+
124
+ Honesty rule: `pen` proves what it fires. It cannot promise "never hacked" — it promises
125
+ every demonstrable attack gets a regression test that fails on the bug and passes on the fix.
126
+ If `pen --fix` wrote repro tests, never delete them; they are the permanent proof.
127
+
128
+ ---
129
+
130
+ ## Mode: custom ask
131
+
132
+ The user typed free-form text after `/pitstop` instead of a flag. Run **only what they asked
133
+ for**. Do not expand into the default full loop, do not fix unrelated things.
134
+
135
+ **Step A — Map the ask to a command (before doing anything else).** Read the ask and choose
136
+ the closest match. Never run the full scan just to decide:
137
+
138
+ | The ask is about… | Run this | And |
139
+ |---|---|---|
140
+ | app security / vulnerabilities / "is my app hackable" / "check the security" | `!npx openpitstop pen` (add `--fix` only if they asked you to fix) | report proven/indicated/unproven verdicts honestly |
141
+ | tests, flakiness, coverage, a failing suite | `!npx openpitstop scan` | read the Tests + Reliability lines; no fixes unless asked |
142
+ | "run my tests" / unit tests / integration tests / e2e / the test pyramid / "are my tests passing?" | `!npx openpitstop test` (add `--unit`/`--integration`/`--e2e` if they named one layer) | print the boxed pyramid verbatim; if a layer failed, list the failing tests and offer to fix them (confirmation pause first) |
143
+ | the last commit / whether an agent cheated / "verify my agent's work" | `!npx openpitstop integrity` | print the boxed verdict verbatim and stop |
144
+ | one finding id | `!npx openpitstop inspect <id>` (or `repro <id>` if they want a regression test) | only that finding |
145
+ | one file or route | a scoped `!npx openpitstop scan` / `inspect` on the relevant finding | only that area |
146
+ | overall repo quality / health | `!npx openpitstop scan` | show the box and stop (no fixes unless asked) |
147
+ | one category (duplication, circular imports, secrets…) | `!npx openpitstop scan` | read only that category's lines |
148
+ | anything not in the table | the cheapest read-only check that answers them | ask one clarifying question if still unsure |
149
+
150
+ **Step B — State your interpretation, then confirm before fixing.** Your first message is
151
+ exactly one line:
152
+
153
+ ```
154
+ /pitstop — I read that as: <one-line restatement of their ask>. Running <command>.
155
+ ```
156
+
157
+ - If the ask is **read-only** (check / report / explain) → run it, print the output, and
158
+ stop. Never fix without being asked.
159
+ - If the ask **clearly requests fixes** ("fix this", "make it pass") → run it, show the
160
+ output, then append one confirmation line — "Found [N] issue(s). Start fixing? Reply
161
+ anything to continue, or tell me what to skip." — and wait.
162
+ - If you are **not sure** → run the cheapest check, show it, and ask one clarifying
163
+ question. Never guess and start editing.
164
+
165
+ **Step C — Fixes (only after confirmation).** Handle each fix exactly like the default
166
+ loop: `!npx openpitstop repro <id>` must **FAIL** first, smallest fix, the **same** repro
167
+ must **PASS**, `!npx openpitstop verify` CLEAN, commit with the repro test. Stop when every
168
+ issue relevant to the ask is gone.
169
+
170
+ **Step D — Scope discipline (mandatory).** Work through ONLY the issues relevant to the
171
+ ask. Leave unrelated clusters alone (say so in one line if they exist), do not write the
172
+ full `PITSTOP_REPORT.md` unless the ask covers the whole repo, and never drift into the
173
+ default full loop.
174
+
175
+ ---
176
+
177
+ ## The default full loop
178
+
179
+ You are running the `openpitstop` quality loop against this repository. Follow these
180
+ steps **exactly**, in order. Do not improvise around them.
181
+
182
+ The loop has exactly **one** mandatory pause: after the first scan (Step 2), before the
183
+ first fix. After that, you act autonomously until a hard stop condition.
184
+
185
+ ---
186
+
187
+ ## Step 1 — Scan and show the box
188
+
189
+ Run the scan:
190
+
191
+ `!npx openpitstop scan`
192
+
193
+ Print the **entire boxed output verbatim** as your complete response. Do not summarize
194
+ it, do not add commentary, do not explain it — let the box speak for itself. Nothing else
195
+ in your response except the box.
196
+
197
+ ---
198
+
199
+ ## Step 2 — Confirmation (mandatory pause, never skipped)
200
+
201
+ Immediately after the box, append exactly this line (fill in N and M):
202
+
203
+ ```
204
+ Found [N] root-cause clusters covering [M] issues. Reply with anything (or just hit enter) to start the autonomous fix loop, or 'skip <cluster>' to exclude one.
205
+ ```
206
+
207
+ - **N** = number of root-cause clusters shown in the box.
208
+ - **M** = total issues spanned by those clusters = N + the total symptom count (i.e.
209
+ root causes + symptoms), or simply count every finding listed under every cluster.
210
+
211
+ Even if the scan found only **1** issue, you MUST print this line and stop. This
212
+ confirmation step is non-negotiable.
213
+
214
+ Then **end your turn and wait** for the user's next message. Do not read files, do not plan
215
+ fixes, do not touch anything yet.
216
+
217
+ - If the user's message starts with `stop` / `cancel` / `abort` → do not start the loop;
218
+ go straight to Step 6.
219
+ - If the user's message matches `skip <cluster>` → exclude that cluster from consideration,
220
+ then proceed to Step 3 on the remaining clusters (no new confirmation needed).
221
+ - Any other input (including an empty reply) → confirmation granted, proceed to Step 3.
222
+
223
+ ---
224
+
225
+ ## Step 3 — Autonomous fix loop (runs after confirmation)
226
+
227
+ For each iteration, do all of (a)–(m) without asking for confirmation again:
228
+
229
+ **a0. Loop-engineering shortcut (preferred for a single finding).** You can delegate one
230
+ finding to OpenPitStop, which will run the full repro→fix→verify loop *for you, looping until
231
+ the fix is actually verified* (the repro passes, or `pitstop verify` is clean) — up to 5
232
+ attempts, feeding each failure back into the next attempt:
233
+
234
+ `!npx openpitstop drive <finding-id>`
235
+
236
+ Use this for a single confirmed root cause instead of hand-running (d)–(j). When it returns
237
+ VERIFIED, the finding is solved and recorded as driven; move to the next cluster. If it returns
238
+ NOT VERIFIED after the attempts, fix by hand or report "requires human review". To drive the
239
+ *whole* repo to fully-fixed with no id: `!npx openpitstop drive` (it repeatedly runs the next
240
+ command from the plan below until nothing remains).
241
+
242
+ **a. Pick the cluster.** Choose the highest-value remaining cluster (most severe, or most
243
+ central). Skip any the user excluded.
244
+
245
+ **b. Branch.** If you are not already on a `pitstop/*` branch, create and switch to:
246
+
247
+ `pitstop/<short-slug>-<date>`
248
+
249
+ where `<short-slug>` is a 2–4 word kebab slug of the cluster (e.g. `circular-core-deps`)
250
+ and `<date>` is `YYYY-MM-DD`. Never branch off or commit to `main`.
251
+
252
+ **c. State your hypothesis.** In one or two sentences, say *why* you believe this cluster's
253
+ root cause is what the scanner claims it is, and what a minimal correct fix looks like.
254
+
255
+ **d. Capture the bug as a failing test — mandatory, never skippable.**
256
+
257
+ `!npx openpitstop repro <finding-id>`
258
+
259
+ `<finding-id>` is the id printed on the cluster's line in the scan box (and stored in
260
+ `.pitstop/scan-latest.json`), e.g. `ledger-3f9a2c01`. OpenPitStop writes a permanent repro
261
+ test (`pitstop-repro-<slug>.test.*`) and runs it:
262
+
263
+ - If it reports **FAIL — bug reproduced**, you have *proven* the bug with a real failing
264
+ test. Good. Proceed.
265
+ - If it reports **PASS**, or **refused** with no genuine repro generator, the hypothesis is
266
+ **UNPROVEN**. Do **not** fix blind. Stop, return to (c), and either revise the hypothesis
267
+ or pick a different cluster.
268
+
269
+ **e. Make the smallest fix.** Using your own file-edit tools, change the minimum needed to
270
+ address the **root cause** (not just a symptom). Stay on the `pitstop/*` branch.
271
+
272
+ **f. Prove the same test now passes — mandatory, before any verify.**
273
+
274
+ `!npx openpitstop repro <finding-id>`
275
+
276
+ Run the **same** repro test. It must now report **PASS**.
277
+ If it still **FAILS**, the fix did not address the root cause: revert, revise the hypothesis
278
+ in (c), and retry once (see (i)).
279
+
280
+ **g. Verify.**
281
+
282
+ `!npx openpitstop verify`
283
+
284
+ Read the result, especially the **Integrity gate** line, the Regression Risk, and the Δ
285
+ columns. This verify run is also the integrity gate: it diffs your uncommitted change against
286
+ HEAD and runs the AI-agent-cheat detectors (deleted/loosened tests, swallowed errors,
287
+ suppressions, hardcoded-to-pass values, forced exits).
288
+
289
+ > After every `pitstop` command OpenPitStop prints a **Next card** with a repo-aware
290
+ > remediation **plan** (it inspects this repo's language, package manager, frameworks, test
291
+ > runner, CI and env files to recommend the exact next commands, in order). Follow its `Next`
292
+ > command — do not guess.
293
+
294
+ **h. Integrity gate — mandatory, never skippable.** Inspect the "Integrity gate:" verdict
295
+ in the verify box and branch exactly like this:
296
+
297
+ - **CLEAN** → proceed to (i) as normal.
298
+ - **SUSPICIOUS** → do **not** commit. Explicitly state in your response what was flagged and
299
+ why (detector, `file:line`, evidence). Revert the change. Retry the **SAME** cluster exactly
300
+ **once**, with this instruction appended verbatim to your working notes:
301
+
302
+ > "Your previous attempt touched a test or added error suppression in a way that looks like it
303
+ > avoided the real fix instead of solving it. Solve the root cause. Do not modify, skip, or
304
+ > loosen any test. Do not add suppression comments or swallow exceptions."
305
+
306
+ Repeat (c)–(g) with that instruction. If the retry is **also SUSPICIOUS**, stop trying this
307
+ cluster, mark it **"requires human review"** in the final report, and move to the next cluster —
308
+ never loop indefinitely on the same pattern.
309
+ - **CONFIRMED_CHEAT** → do **not** commit. State what was flagged. Revert. Skip straight to
310
+ **"requires human review"** with **zero retries** — this tier is unambiguous enough that
311
+ retrying isn't warranted; a human should look at it. Move to the next cluster.
312
+
313
+ **i. If Regression Risk is High** (and the integrity gate was CLEAN): `git checkout` (or
314
+ otherwise revert) your change, note that this hypothesis failed, and try **once more** with a
315
+ different approach for the **same** cluster. Maximum 2 attempts per cluster, then move on.
316
+
317
+ **j. If Risk is Low/Medium, tests are not newly failing, and the integrity gate is CLEAN:**
318
+ commit with a clear message describing the root cause and fix. Commit the repro test **together
319
+ with** the fix — it is a **permanent regression guard**, never a throwaway, and never delete it.
320
+ Then record it:
321
+
322
+ `!npx openpitstop memory add "<finding-id> fixed + proven by pitstop-repro-<slug>" --type fix`
323
+
324
+ (keep the summary short and factual — this is the "six months later" recall.)
325
+
326
+ **k. Re-scan and show a shorter status.** Run the scan again and print a short updated
327
+ boxed status in the same visual style as Step 1, showing: issues fixed so far, issues
328
+ remaining. No long commentary.
329
+
330
+ **k2. Token economy (MANDATORY in every loop iteration).** Your budget is real; follow
331
+ these rules exactly:
332
+
333
+ - **Before any re-scan**, run `!npx openpitstop ready-check`. If it exits 0 (tree
334
+ unchanged), run `!npx openpitstop scan --reuse` instead of a full scan — it returns the
335
+ sealed baseline instantly, and skipping it means burning credits for nothing.
336
+ - **Prefer `!npx openpitstop inspect <id>`** over reading whole files: it shows the exact
337
+ code window, cluster context and repro proof. Read whole files only when inspect cannot
338
+ answer the question.
339
+ - **Batch your edits** — plan the fix, then apply it in as few tool calls as possible.
340
+ One pause per loop (Step 2), never more.
341
+ - **Verify during iteration** as-is; the reliability suite (extra runs) is for the FINAL
342
+ pass — don't run it per-iteration.
343
+ - If you are about to re-scan a second time without any edit having happened, stop and
344
+ re-check: you are burning credits in a loop. Either pick a different cluster or ask.
345
+
346
+ **l. Success check.** If the fresh scan shows **zero remaining actionable clusters**,
347
+ stop — this is the success condition. Your final line should be:
348
+
349
+ `nothing left to fix, nothing broken.`
350
+
351
+ **m. Otherwise repeat.** Go back to (a) automatically. You do **not** ask for confirmation
352
+ again. The loop pauses only once, at Step 2, before the very first fix.
353
+
354
+ ---
355
+
356
+ ## Step 4 — Hard stop conditions
357
+
358
+ Whichever comes first:
359
+
360
+ - **(a)** a fresh scan shows zero actionable clusters (success), or
361
+ - **(b)** 10 total fix iterations, or
362
+ - **(c)** 45 minutes of wall-clock time.
363
+
364
+ If you stop because of **(b)** or **(c)** rather than **(a)**, say so plainly. Do not imply
365
+ everything is done when it is not. Report how many clusters remain.
366
+
367
+ ---
368
+
369
+ ## Step 5 — Non-negotiable safety rules
370
+
371
+ - Never force-push. Ever.
372
+ - Never touch `.env`, `.git/`, or any secret/credential file.
373
+ - Never delete a file unless the dependency graph confirms it has zero incoming references.
374
+ - Never silently modify CI/deploy config — if a fix would require it, flag it to the user
375
+ instead and skip that change.
376
+ - Always stay on the `pitstop/*` branch. Leave `main` (and any protected branch) untouched.
377
+ - If a fix feels risky, prefer the smaller safer change; the loop can retry.
378
+
379
+ ---
380
+
381
+ ## Step 6 — Final report
382
+
383
+ On **any** stop condition (success, max iterations, or timeout), run:
384
+
385
+ `!npx openpitstop report`
386
+
387
+ and present the resulting **`PITSTOP_REPORT.md` / boxed output verbatim** as your final
388
+ message. Do not rewrite or summarize it. The report includes **"Fixes shipped with permanent
389
+ proof"**: one line per committed fix, linking the `pitstop-repro-*.test.*` file that proves
390
+ it — if a fix has no committed repro test, that is a red flag the loop was cut short.