openpitstop 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +512 -0
- package/demo-repo/README.md +24 -0
- package/demo-repo/package-lock.json +4483 -0
- package/demo-repo/package.json +17 -0
- package/demo-repo/src/config.js +5 -0
- package/demo-repo/src/db.js +31 -0
- package/demo-repo/src/server.js +13 -0
- package/demo-repo/src/userController.js +22 -0
- package/demo-repo/src/userRepo.js +18 -0
- package/demo-repo/src/userService.js +59 -0
- package/demo-repo/test/user.test.js +12 -0
- package/demo-repo-fintech/README.md +27 -0
- package/demo-repo-fintech/package-lock.json +1023 -0
- package/demo-repo-fintech/package.json +14 -0
- package/demo-repo-fintech/pitstop-repro-ledger-order-ord-conc-1-charged-2.test.mjs +175 -0
- package/demo-repo-fintech/pitstop-repro-ledger-order-ord-retry-2-charged-2.test.mjs +175 -0
- package/demo-repo-fintech/pitstop-repro-ledger-order-ord-webhook-3-charged-2.test.mjs +175 -0
- package/demo-repo-fintech/src/gateway.js +26 -0
- package/demo-repo-fintech/src/server.js +61 -0
- package/demo-repo-generators/package-lock.json +1508 -0
- package/demo-repo-generators/package.json +17 -0
- package/demo-repo-generators/public/index.html +10 -0
- package/demo-repo-generators/scripts/build.js +17 -0
- package/demo-repo-generators/src/counter.js +16 -0
- package/demo-repo-integrity/README.md +49 -0
- package/demo-repo-integrity/package.json +12 -0
- package/demo-repo-integrity/src/money.js +19 -0
- package/demo-repo-integrity/test/money.test.js +17 -0
- package/dist/analyzers/accessibility.d.ts +18 -0
- package/dist/analyzers/accessibility.js +248 -0
- package/dist/analyzers/accessibility.js.map +1 -0
- package/dist/analyzers/dependencyGraph.d.ts +14 -0
- package/dist/analyzers/dependencyGraph.js +206 -0
- package/dist/analyzers/dependencyGraph.js.map +1 -0
- package/dist/analyzers/devex.d.ts +19 -0
- package/dist/analyzers/devex.js +301 -0
- package/dist/analyzers/devex.js.map +1 -0
- package/dist/analyzers/duplication.d.ts +2 -0
- package/dist/analyzers/duplication.js +47 -0
- package/dist/analyzers/duplication.js.map +1 -0
- package/dist/analyzers/index.d.ts +12 -0
- package/dist/analyzers/index.js +38 -0
- package/dist/analyzers/index.js.map +1 -0
- package/dist/analyzers/integrity/assertionLiteralTamper.d.ts +2 -0
- package/dist/analyzers/integrity/assertionLiteralTamper.js +164 -0
- package/dist/analyzers/integrity/assertionLiteralTamper.js.map +1 -0
- package/dist/analyzers/integrity/exceptionSwallow.d.ts +2 -0
- package/dist/analyzers/integrity/exceptionSwallow.js +198 -0
- package/dist/analyzers/integrity/exceptionSwallow.js.map +1 -0
- package/dist/analyzers/integrity/exitCheat.d.ts +2 -0
- package/dist/analyzers/integrity/exitCheat.js +31 -0
- package/dist/analyzers/integrity/exitCheat.js.map +1 -0
- package/dist/analyzers/integrity/git.d.ts +10 -0
- package/dist/analyzers/integrity/git.js +181 -0
- package/dist/analyzers/integrity/git.js.map +1 -0
- package/dist/analyzers/integrity/hardcodedMatch.d.ts +2 -0
- package/dist/analyzers/integrity/hardcodedMatch.js +93 -0
- package/dist/analyzers/integrity/hardcodedMatch.js.map +1 -0
- package/dist/analyzers/integrity/helpers.d.ts +21 -0
- package/dist/analyzers/integrity/helpers.js +66 -0
- package/dist/analyzers/integrity/helpers.js.map +1 -0
- package/dist/analyzers/integrity/index.d.ts +10 -0
- package/dist/analyzers/integrity/index.js +27 -0
- package/dist/analyzers/integrity/index.js.map +1 -0
- package/dist/analyzers/integrity/mockOverreach.d.ts +2 -0
- package/dist/analyzers/integrity/mockOverreach.js +72 -0
- package/dist/analyzers/integrity/mockOverreach.js.map +1 -0
- package/dist/analyzers/integrity/suppressionCreep.d.ts +2 -0
- package/dist/analyzers/integrity/suppressionCreep.js +27 -0
- package/dist/analyzers/integrity/suppressionCreep.js.map +1 -0
- package/dist/analyzers/integrity/testTamper.d.ts +2 -0
- package/dist/analyzers/integrity/testTamper.js +153 -0
- package/dist/analyzers/integrity/testTamper.js.map +1 -0
- package/dist/analyzers/integrity/types.d.ts +63 -0
- package/dist/analyzers/integrity/types.js +13 -0
- package/dist/analyzers/integrity/types.js.map +1 -0
- package/dist/analyzers/ledger/attacks.d.ts +17 -0
- package/dist/analyzers/ledger/attacks.js +135 -0
- package/dist/analyzers/ledger/attacks.js.map +1 -0
- package/dist/analyzers/ledger/discover.d.ts +15 -0
- package/dist/analyzers/ledger/discover.js +122 -0
- package/dist/analyzers/ledger/discover.js.map +1 -0
- package/dist/analyzers/ledger/evidence.d.ts +22 -0
- package/dist/analyzers/ledger/evidence.js +127 -0
- package/dist/analyzers/ledger/evidence.js.map +1 -0
- package/dist/analyzers/ledger/harness.d.ts +4 -0
- package/dist/analyzers/ledger/harness.js +270 -0
- package/dist/analyzers/ledger/harness.js.map +1 -0
- package/dist/analyzers/ledger/index.d.ts +14 -0
- package/dist/analyzers/ledger/index.js +116 -0
- package/dist/analyzers/ledger/index.js.map +1 -0
- package/dist/analyzers/ledger/types.d.ts +31 -0
- package/dist/analyzers/ledger/types.js +2 -0
- package/dist/analyzers/ledger/types.js.map +1 -0
- package/dist/analyzers/perf.d.ts +2 -0
- package/dist/analyzers/perf.js +106 -0
- package/dist/analyzers/perf.js.map +1 -0
- package/dist/analyzers/reliability.d.ts +22 -0
- package/dist/analyzers/reliability.js +339 -0
- package/dist/analyzers/reliability.js.map +1 -0
- package/dist/analyzers/routes.d.ts +17 -0
- package/dist/analyzers/routes.js +115 -0
- package/dist/analyzers/routes.js.map +1 -0
- package/dist/analyzers/security.d.ts +2 -0
- package/dist/analyzers/security.js +302 -0
- package/dist/analyzers/security.js.map +1 -0
- package/dist/analyzers/suiteRunner.d.ts +64 -0
- package/dist/analyzers/suiteRunner.js +381 -0
- package/dist/analyzers/suiteRunner.js.map +1 -0
- package/dist/analyzers/tests.d.ts +2 -0
- package/dist/analyzers/tests.js +215 -0
- package/dist/analyzers/tests.js.map +1 -0
- package/dist/analyzers/types.d.ts +203 -0
- package/dist/analyzers/types.js +2 -0
- package/dist/analyzers/types.js.map +1 -0
- package/dist/analyzers/util.d.ts +35 -0
- package/dist/analyzers/util.js +209 -0
- package/dist/analyzers/util.js.map +1 -0
- package/dist/ciCheckA.d.ts +2 -0
- package/dist/ciCheckA.js +5 -0
- package/dist/ciCheckA.js.map +1 -0
- package/dist/ciCheckB.d.ts +2 -0
- package/dist/ciCheckB.js +6 -0
- package/dist/ciCheckB.js.map +1 -0
- package/dist/cli.d.ts +2 -0
- package/dist/cli.js +65 -0
- package/dist/cli.js.map +1 -0
- package/dist/commands/budget.d.ts +33 -0
- package/dist/commands/budget.js +108 -0
- package/dist/commands/budget.js.map +1 -0
- package/dist/commands/ci.d.ts +14 -0
- package/dist/commands/ci.js +157 -0
- package/dist/commands/ci.js.map +1 -0
- package/dist/commands/demo.d.ts +2 -0
- package/dist/commands/demo.js +163 -0
- package/dist/commands/demo.js.map +1 -0
- package/dist/commands/digest.d.ts +8 -0
- package/dist/commands/digest.js +189 -0
- package/dist/commands/digest.js.map +1 -0
- package/dist/commands/doctor.d.ts +2 -0
- package/dist/commands/doctor.js +98 -0
- package/dist/commands/doctor.js.map +1 -0
- package/dist/commands/drive.d.ts +3 -0
- package/dist/commands/drive.js +170 -0
- package/dist/commands/drive.js.map +1 -0
- package/dist/commands/gate.d.ts +2 -0
- package/dist/commands/gate.js +136 -0
- package/dist/commands/gate.js.map +1 -0
- package/dist/commands/honesty.d.ts +2 -0
- package/dist/commands/honesty.js +182 -0
- package/dist/commands/honesty.js.map +1 -0
- package/dist/commands/inspect.d.ts +2 -0
- package/dist/commands/inspect.js +244 -0
- package/dist/commands/inspect.js.map +1 -0
- package/dist/commands/install.d.ts +2 -0
- package/dist/commands/install.js +81 -0
- package/dist/commands/install.js.map +1 -0
- package/dist/commands/integrity.d.ts +2 -0
- package/dist/commands/integrity.js +74 -0
- package/dist/commands/integrity.js.map +1 -0
- package/dist/commands/memory.d.ts +6 -0
- package/dist/commands/memory.js +90 -0
- package/dist/commands/memory.js.map +1 -0
- package/dist/commands/pen.d.ts +17 -0
- package/dist/commands/pen.js +158 -0
- package/dist/commands/pen.js.map +1 -0
- package/dist/commands/prompt.d.ts +2 -0
- package/dist/commands/prompt.js +37 -0
- package/dist/commands/prompt.js.map +1 -0
- package/dist/commands/readyCheck.d.ts +21 -0
- package/dist/commands/readyCheck.js +68 -0
- package/dist/commands/readyCheck.js.map +1 -0
- package/dist/commands/report.d.ts +2 -0
- package/dist/commands/report.js +48 -0
- package/dist/commands/report.js.map +1 -0
- package/dist/commands/repro.d.ts +3 -0
- package/dist/commands/repro.js +61 -0
- package/dist/commands/repro.js.map +1 -0
- package/dist/commands/scan.d.ts +27 -0
- package/dist/commands/scan.js +348 -0
- package/dist/commands/scan.js.map +1 -0
- package/dist/commands/share.d.ts +2 -0
- package/dist/commands/share.js +163 -0
- package/dist/commands/share.js.map +1 -0
- package/dist/commands/trends.d.ts +2 -0
- package/dist/commands/trends.js +103 -0
- package/dist/commands/trends.js.map +1 -0
- package/dist/commands/try.d.ts +12 -0
- package/dist/commands/try.js +208 -0
- package/dist/commands/try.js.map +1 -0
- package/dist/commands/verify.d.ts +44 -0
- package/dist/commands/verify.js +381 -0
- package/dist/commands/verify.js.map +1 -0
- package/dist/commands/watch.d.ts +10 -0
- package/dist/commands/watch.js +70 -0
- package/dist/commands/watch.js.map +1 -0
- package/dist/evidence.d.ts +37 -0
- package/dist/evidence.js +61 -0
- package/dist/evidence.js.map +1 -0
- package/dist/graph/correlate.d.ts +7 -0
- package/dist/graph/correlate.js +246 -0
- package/dist/graph/correlate.js.map +1 -0
- package/dist/graph/integrity.d.ts +22 -0
- package/dist/graph/integrity.js +39 -0
- package/dist/graph/integrity.js.map +1 -0
- package/dist/installer/targets.d.ts +23 -0
- package/dist/installer/targets.js +162 -0
- package/dist/installer/targets.js.map +1 -0
- package/dist/memory/store.d.ts +33 -0
- package/dist/memory/store.js +131 -0
- package/dist/memory/store.js.map +1 -0
- package/dist/pen/dynamic.d.ts +36 -0
- package/dist/pen/dynamic.js +622 -0
- package/dist/pen/dynamic.js.map +1 -0
- package/dist/pen/fix.d.ts +35 -0
- package/dist/pen/fix.js +239 -0
- package/dist/pen/fix.js.map +1 -0
- package/dist/pen/proof.d.ts +46 -0
- package/dist/pen/proof.js +112 -0
- package/dist/pen/proof.js.map +1 -0
- package/dist/pen/report.d.ts +11 -0
- package/dist/pen/report.js +258 -0
- package/dist/pen/report.js.map +1 -0
- package/dist/pen/static.d.ts +17 -0
- package/dist/pen/static.js +565 -0
- package/dist/pen/static.js.map +1 -0
- package/dist/pen/store.d.ts +11 -0
- package/dist/pen/store.js +36 -0
- package/dist/pen/store.js.map +1 -0
- package/dist/pen/types.d.ts +112 -0
- package/dist/pen/types.js +53 -0
- package/dist/pen/types.js.map +1 -0
- package/dist/report/format.d.ts +126 -0
- package/dist/report/format.js +706 -0
- package/dist/report/format.js.map +1 -0
- package/dist/report/score.d.ts +47 -0
- package/dist/report/score.js +164 -0
- package/dist/report/score.js.map +1 -0
- package/dist/repro/framework.d.ts +26 -0
- package/dist/repro/framework.js +224 -0
- package/dist/repro/framework.js.map +1 -0
- package/dist/repro/generate.d.ts +14 -0
- package/dist/repro/generate.js +669 -0
- package/dist/repro/generate.js.map +1 -0
- package/dist/repro/ids.d.ts +32 -0
- package/dist/repro/ids.js +175 -0
- package/dist/repro/ids.js.map +1 -0
- package/dist/repro/index.d.ts +24 -0
- package/dist/repro/index.js +69 -0
- package/dist/repro/index.js.map +1 -0
- package/dist/repro/pen.d.ts +26 -0
- package/dist/repro/pen.js +321 -0
- package/dist/repro/pen.js.map +1 -0
- package/dist/repro/run.d.ts +18 -0
- package/dist/repro/run.js +36 -0
- package/dist/repro/run.js.map +1 -0
- package/dist/sandbox/proxy.d.ts +41 -0
- package/dist/sandbox/proxy.js +231 -0
- package/dist/sandbox/proxy.js.map +1 -0
- package/dist/sandbox/startCmd.d.ts +24 -0
- package/dist/sandbox/startCmd.js +71 -0
- package/dist/sandbox/startCmd.js.map +1 -0
- package/dist/ui/spinner.d.ts +18 -0
- package/dist/ui/spinner.js +26 -0
- package/dist/ui/spinner.js.map +1 -0
- package/dist/verify/metrics.d.ts +13 -0
- package/dist/verify/metrics.js +72 -0
- package/dist/verify/metrics.js.map +1 -0
- package/package.json +68 -0
- package/scripts/cheat-demo.cjs +102 -0
- package/templates/ledger/preload.cjs +331 -0
- package/templates/pen/preload.cjs +196 -0
- package/templates/pitstop.prompt.md +371 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Krish-1507
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,512 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="docs/media/pitstop-logo.png" alt="OpenPitStop" width="380">
|
|
3
|
+
</p>
|
|
4
|
+
|
|
5
|
+
# OpenPitStop CLI
|
|
6
|
+
|
|
7
|
+
**The agent finally has a referee it can't cheat.** OpenPitStop is a CLI that scans your repo,
|
|
8
|
+
scores it, and checks everything your AI coding agent does — so when it says "done", you
|
|
9
|
+
know it's actually done.
|
|
10
|
+
|
|
11
|
+
[](https://www.npmjs.com/package/openpitstop)
|
|
12
|
+
[](https://github.com/Krish-1507/OpenPitStop/actions/workflows/ci.yml)
|
|
13
|
+
[](LICENSE)
|
|
14
|
+
|
|
15
|
+
> AI coding agents are brilliant at fixing things — and equally brilliant at *saying they
|
|
16
|
+
> did* when they didn't. OpenPitStop measures your repo with scans, seals every number so it
|
|
17
|
+
> can't be edited later, attacks your app with a live penetration test, and checks every
|
|
18
|
+
> change your agent makes. The exit codes tell you the truth: `0` clean · `1` suspicious ·
|
|
19
|
+
> `2` confirmed cheat.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Why I built this
|
|
24
|
+
|
|
25
|
+
I spend my days running coding agents on real repos. They're brilliant at fixing things —
|
|
26
|
+
and equally brilliant at *telling me they did* when they didn't: focusing tests to hide
|
|
27
|
+
failures, deleting the failing test, editing an assertion to match the buggy output. I got
|
|
28
|
+
tired of auditing my agent's work by hand, so I built a referee.
|
|
29
|
+
|
|
30
|
+
OpenPitStop is my own workflow tool, not a showcase: every repo I touch gets the loop, every
|
|
31
|
+
change gets the gate, and the numbers in this README are the same numbers I trust. It's
|
|
32
|
+
dogfooded hard — OpenPitStop's own CI scans a real repo with OpenPitStop on every push (Linux and
|
|
33
|
+
Windows), and the evidence chain is regression-tested because a bug in it once made OpenPitStop
|
|
34
|
+
cry `TAMPERED` at baselines it had just written. If it can referee itself, it can referee
|
|
35
|
+
your agent.
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## Quick Nav
|
|
40
|
+
|
|
41
|
+
| Jump to | |
|
|
42
|
+
|---|---|
|
|
43
|
+
| [Feature tour](#feature-tour) — the 11 demos | [Install](#install) · [Usage](#usage) · [Tool support](#tool-support) |
|
|
44
|
+
| [See it in 90 seconds](#see-it-in-90-seconds) | [What OpenPitStop actually does](#what-openpitstop-actually-does) · [Every command](#every-command) |
|
|
45
|
+
| [Architecture](#architecture) | [Known limitations](#known-limitations) · [Contributing](#contributing) · [License](#license) |
|
|
46
|
+
|
|
47
|
+
**Straight to one feature:** [The scan](#the-scan) · [Verify](#verify) · [Trends](#trends) · [Inspect](#inspect) · [Repro](#repro) · [The pen test](#the-pen-test) · [Report](#report) · [Share](#share) · [Honesty](#honesty) · [Try it on your repo](#try-it-on-your-repo) · [The live shield](#the-live-shield)
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## Feature tour
|
|
52
|
+
|
|
53
|
+
Every clip below is real command output — the only thing that was trimmed is dead time.
|
|
54
|
+
|
|
55
|
+
### The scan
|
|
56
|
+
|
|
57
|
+
`pitstop scan` — every check runs at once, one box, one score.
|
|
58
|
+
|
|
59
|
+
<p align="center">
|
|
60
|
+
<img src="docs/media/pitstop-scan.gif" alt="pitstop scan — boxed report with the OpenPitStop Score" width="700">
|
|
61
|
+
</p>
|
|
62
|
+
|
|
63
|
+
### Verify
|
|
64
|
+
|
|
65
|
+
`pitstop verify` — re-scan after a change and see exactly how the score moved. Also checks your diff for cheat patterns.
|
|
66
|
+
|
|
67
|
+
<p align="center">
|
|
68
|
+
<img src="docs/media/pitstop-verify.gif" alt="pitstop verify — the score moves, points don't lie" width="700">
|
|
69
|
+
</p>
|
|
70
|
+
|
|
71
|
+
### Trends
|
|
72
|
+
|
|
73
|
+
`pitstop trends` — per-category sparklines from your scan history.
|
|
74
|
+
|
|
75
|
+
<p align="center">
|
|
76
|
+
<img src="docs/media/pitstop-trends.gif" alt="pitstop trends — sparklines showing a repo improving" width="700">
|
|
77
|
+
</p>
|
|
78
|
+
|
|
79
|
+
### Inspect
|
|
80
|
+
|
|
81
|
+
`pitstop inspect <finding-id>` — open up one finding: the code snippet, the root cause, whether a repro test exists, and what OpenPitStop remembers about these files.
|
|
82
|
+
|
|
83
|
+
<p align="center">
|
|
84
|
+
<img src="docs/media/pitstop-inspect.gif" alt="pitstop inspect — deep-dive on a single finding" width="700">
|
|
85
|
+
</p>
|
|
86
|
+
|
|
87
|
+
### Repro
|
|
88
|
+
|
|
89
|
+
`pitstop repro <finding-id>` — every fix starts with a failing test. The test is written to fail *now* and pass after the fix.
|
|
90
|
+
|
|
91
|
+
<p align="center">
|
|
92
|
+
<img src="docs/media/pitstop-repro.gif" alt="pitstop repro — a regression test that fails first" width="700">
|
|
93
|
+
</p>
|
|
94
|
+
|
|
95
|
+
### The pen test
|
|
96
|
+
|
|
97
|
+
`pitstop pen --fix` — boots your app in a sandbox, attacks it, and writes PROVEN verdicts — plus repro tests and a patch.
|
|
98
|
+
|
|
99
|
+
<p align="center">
|
|
100
|
+
<img src="docs/media/pitstop-pen.gif" alt="pitstop pen — runtime-proof findings with repros and a patch" width="700">
|
|
101
|
+
</p>
|
|
102
|
+
|
|
103
|
+
### Report
|
|
104
|
+
|
|
105
|
+
`pitstop report --html` — one self-contained HTML report, sealed with an evidence signature.
|
|
106
|
+
|
|
107
|
+
<p align="center">
|
|
108
|
+
<img src="docs/media/pitstop-report.gif" alt="pitstop report — HTML report with evidence signature" width="700">
|
|
109
|
+
</p>
|
|
110
|
+
|
|
111
|
+
### Share
|
|
112
|
+
|
|
113
|
+
`pitstop share` — one-card summary, easy to paste into a PR or a demo chat.
|
|
114
|
+
|
|
115
|
+
<p align="center">
|
|
116
|
+
<img src="docs/media/pitstop-share.gif" alt="pitstop share — a compact shareable summary card" width="700">
|
|
117
|
+
</p>
|
|
118
|
+
|
|
119
|
+
### Honesty
|
|
120
|
+
|
|
121
|
+
`pitstop honesty --html` — an honest assessment of what this tool can't do.
|
|
122
|
+
|
|
123
|
+
<p align="center">
|
|
124
|
+
<img src="docs/media/pitstop-honesty.gif" alt="pitstop honesty — an honest self-assessment certificate" width="700">
|
|
125
|
+
</p>
|
|
126
|
+
|
|
127
|
+
### Try it on your repo
|
|
128
|
+
|
|
129
|
+
`pitstop try .` — score any repo in ~2 seconds of scanning, no setup, no config.
|
|
130
|
+
(First `npx openpitstop …` on a machine downloads the package once — a few seconds;
|
|
131
|
+
`npm i -g openpitstop` makes even that instant.)
|
|
132
|
+
|
|
133
|
+
<p align="center">
|
|
134
|
+
<img src="docs/media/pitstop-try.gif" alt="pitstop try — zero-setup score on any repo" width="700">
|
|
135
|
+
</p>
|
|
136
|
+
|
|
137
|
+
### The live shield
|
|
138
|
+
|
|
139
|
+
`pitstop watch` — re-scans the moment a file changes and prints the score delta.
|
|
140
|
+
|
|
141
|
+
<p align="center">
|
|
142
|
+
<img src="docs/media/pitstop-watch.gif" alt="pitstop watch — live score delta when a file changes" width="700">
|
|
143
|
+
</p>
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
147
|
+
## See it in 90 seconds
|
|
148
|
+
|
|
149
|
+
Two commands. First, a real broken repo — scanned, scored and reported in seconds:
|
|
150
|
+
|
|
151
|
+
```bash
|
|
152
|
+
npx openpitstop@latest demo
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
Then the part that gets the *wow*: a scripted arc where a lazy agent tries to make the
|
|
156
|
+
failing suite green without fixing the bug — and the gate catches both attempts:
|
|
157
|
+
|
|
158
|
+
```bash
|
|
159
|
+
node scripts/cheat-demo.cjs # from a OpenPitStop repo checkout
|
|
160
|
+
node node_modules/openpitstop/scripts/cheat-demo.cjs # from any project that installed it
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
Set `PITSTOP_CLI="node /path/to/dist/cli.js"` to run it against a local build instead
|
|
164
|
+
of the registry.
|
|
165
|
+
|
|
166
|
+
```
|
|
167
|
+
ACT 1 honest baseline → 1 failed test, scanned and sealed
|
|
168
|
+
ACT 2 agent focuses passing tests → GATE: SUSPICIOUS (exit 1) — blocked
|
|
169
|
+
ACT 3 agent deletes the test → GATE: CONFIRMED_CHEAT (exit 2) — blocked
|
|
170
|
+
(tamper-evident evidence chain verifies the whole way)
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
<p align="center">
|
|
174
|
+
<img src="docs/media/pitstop-demo.gif" alt="The scripted cheat-catch demo: a lazy agent focuses the tests (SUSPICIOUS, exit 1), then deletes the failing test (CONFIRMED_CHEAT, exit 2) — the gate blocks both, on real output." width="780">
|
|
175
|
+
</p>
|
|
176
|
+
|
|
177
|
+
Deterministic, safe to run in a live room, and it's the whole product in miniature:
|
|
178
|
+
**OpenPitStop measures, your agent edits, and the numbers can't be cheated.**
|
|
179
|
+
(Or watch the [34-second video](docs/media/pitstop-demo.mp4).)
|
|
180
|
+
|
|
181
|
+
Not even 90 seconds? Point it at **your own repo** — zero setup, no install, no config:
|
|
182
|
+
|
|
183
|
+
```bash
|
|
184
|
+
npx openpitstop try .
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
Two seconds of scanning, your repo, your score (plus a one-time package download on the
|
|
188
|
+
first-ever `npx` run — see the speed tip in [Install](#install)). Everything else can wait.
|
|
189
|
+
|
|
190
|
+
---
|
|
191
|
+
|
|
192
|
+
## Install
|
|
193
|
+
|
|
194
|
+
One command, that's it:
|
|
195
|
+
|
|
196
|
+
```bash
|
|
197
|
+
npx openpitstop@latest install
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
Run it from inside any project directory. It writes the `/pitstop` command into every
|
|
201
|
+
supported tool below — project-level for the current repo, user-level so it works in any
|
|
202
|
+
repo on your machine. Re-run with `-y` to refresh after updates (it's safe to re-run):
|
|
203
|
+
|
|
204
|
+
```bash
|
|
205
|
+
npx openpitstop install -y
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Re-installing overwrites each tool's `/pitstop` command file with the latest prompt
|
|
209
|
+
(say, a new mode or an updated loop) — your tool picks it up on its next use.
|
|
210
|
+
|
|
211
|
+
Speed tip: the `try`/`scan` itself takes ~2 seconds — but the **first** `npx openpitstop …`
|
|
212
|
+
on a machine has to download the package first (a few seconds on a fast connection, more on a
|
|
213
|
+
slow one). For an instant first run on machines you own, install once:
|
|
214
|
+
|
|
215
|
+
```bash
|
|
216
|
+
npm i -g openpitstop
|
|
217
|
+
openpitstop try .
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
Requires Node.js 22+ (npm will warn on older versions).
|
|
221
|
+
|
|
222
|
+
## Usage
|
|
223
|
+
|
|
224
|
+
Open your repo in any supported tool and type:
|
|
225
|
+
|
|
226
|
+
```
|
|
227
|
+
/pitstop
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
Bare `/pitstop` runs the full quality loop **immediately** — scan, one confirmation pause,
|
|
231
|
+
fix, verify, repeat. No menu, no waiting. Everything below is the power paths on top of
|
|
232
|
+
that:
|
|
233
|
+
|
|
234
|
+
| Invocation | Mode | What it does |
|
|
235
|
+
|---|---|---|
|
|
236
|
+
| `/pitstop` (bare) | **default full loop** | Scans right away, prints the boxed report, one confirmation pause, then the autonomous fix loop — repeat until clean. |
|
|
237
|
+
| `/pitstop --menu` | menu | Prints the full mode list below and **waits** — handy if you forgot the flags. |
|
|
238
|
+
| `/pitstop --scan-only` | scan-only | Runs `openpitstop scan`, prints the entire boxed report verbatim, and stops — no fixes, no commentary. |
|
|
239
|
+
| `/pitstop --demo` | demo | Scaffolds OpenPitStop's seeded broken demo repo into a temp dir, then runs the default full loop there. |
|
|
240
|
+
| `/pitstop --ledger` | ledger | Runs `openpitstop scan --ledger` (boots the app with every outbound HTTP call intercepted and replays duplicate-webhook / double-submit / retry traffic), then runs the loop restricted to the payment findings. |
|
|
241
|
+
| `/pitstop --integrity-only` | integrity-only | Runs `openpitstop integrity`, prints the boxed verdict verbatim, and stops — no scanning, no fixes. |
|
|
242
|
+
| `/pitstop --pen` | pen | Live penetration test with proof — see [The pen test](#the-pen-test). |
|
|
243
|
+
| `/pitstop <your question>` | custom ask | Any free-form text (e.g. `check the security of this app`, `are our tests flaky?`, `did my agent cheat on the last commit?`) is scoped to exactly that ask: the agent maps it to the right command (`pen` for security, `integrity` for cheats, `scan` for health/tests…), states its interpretation in one line, confirms before fixing, and fixes only what you asked. |
|
|
244
|
+
|
|
245
|
+
For reference, `/pitstop --menu` shows this list:
|
|
246
|
+
|
|
247
|
+
```
|
|
248
|
+
OpenPitStop modes:
|
|
249
|
+
(enter) — full autonomous loop (scan, confirm, fix, verify, repeat)
|
|
250
|
+
--scan-only — scan and report, no fixes
|
|
251
|
+
--demo — run against OpenPitStop's own seeded demo repo
|
|
252
|
+
--ledger — payment idempotency fuzzing only
|
|
253
|
+
--integrity-only — re-check the last commit for cheat patterns, no scanning
|
|
254
|
+
--pen — penetration test: live attacks + proof + fixes (regression tests, patches)
|
|
255
|
+
(your own ask) — reply with anything else, e.g. "check the security of this app"
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
A flag after `/pitstop` picks a specific mode; any free-form text after it becomes a scoped
|
|
259
|
+
custom ask; bare `/pitstop` is the full loop. If a tool ever fails to substitute arguments,
|
|
260
|
+
`/pitstop` behaves as bare — the default full loop — rather than guessing.
|
|
261
|
+
|
|
262
|
+
No repo handy? `npx openpitstop@latest demo` scaffolds a broken demo repo in a temp dir so
|
|
263
|
+
you can watch the whole loop — self-contained, no installs on the hot path, and it never
|
|
264
|
+
writes into your tool configs (that stays an explicit `pitstop install`).
|
|
265
|
+
|
|
266
|
+
## Tool support
|
|
267
|
+
|
|
268
|
+
| Tool | Installed to | Status |
|
|
269
|
+
|------|--------------|--------|
|
|
270
|
+
| Claude Code | `.claude/commands/pitstop.md` (project + user), plus a Skill at `.claude/skills/pitstop/SKILL.md` | Full support |
|
|
271
|
+
| Cursor | `.cursor/commands/pitstop.md` (project + user) | Full support |
|
|
272
|
+
| OpenCode | `.opencode/commands/pitstop.md` (project), `~/.config/opencode/commands/` (user) | Full support |
|
|
273
|
+
| Kilo Code | `.kilo/commands/pitstop.md` (project), `~/.config/kilo/commands/` (user) | Full support |
|
|
274
|
+
| Antigravity | `.agent/workflows/pitstop.md` (project + user) | Full support |
|
|
275
|
+
| Gemini CLI | `.gemini/commands/pitstop.toml` (project + user) | Full support |
|
|
276
|
+
| Codex CLI | `~/.codex/prompts/pitstop.md` | Full support |
|
|
277
|
+
| Codex App / VS Code extension | — (no file written) | **Not supported** — OpenAI hasn't shipped custom slash commands there; install prints a manual-copy note instead |
|
|
278
|
+
|
|
279
|
+
Legacy/alternate locations are also written where tool docs are inconsistent across versions
|
|
280
|
+
(see `src/installer/targets.ts`). Existing files are never overwritten unless you pass
|
|
281
|
+
`-y`/`--force`; `npx openpitstop install --uninstall` removes everything.
|
|
282
|
+
|
|
283
|
+
## What OpenPitStop actually does
|
|
284
|
+
|
|
285
|
+
### The loop at a glance
|
|
286
|
+
|
|
287
|
+
<p align="center">
|
|
288
|
+
<img src="docs/media/pitstop-loop.png" alt="OpenPitStop's autonomous loop: scan → report → you confirm → repro (must FAIL first) → fix → verify → repeat until a fresh scan shows zero clusters" width="900">
|
|
289
|
+
</p>
|
|
290
|
+
|
|
291
|
+
OpenPitStop never touches your code. It checks, scores, and referees — your AI agent does the
|
|
292
|
+
editing, knowing it's being watched.
|
|
293
|
+
|
|
294
|
+
### The scan
|
|
295
|
+
|
|
296
|
+
`pitstop scan` runs a bunch of checks on your repo: circular imports, known security
|
|
297
|
+
issues (`npm audit`, plus `pip-audit`/`osv-scanner` for Python and other stacks, and
|
|
298
|
+
`gitleaks`/`semgrep` when installed), duplicated code (`jscpd`), test results
|
|
299
|
+
(jest/vitest/pytest plus native suites for Go, Rust, Flutter/Dart, .NET and Java
|
|
300
|
+
(Maven/Gradle) — pass/fail, duration, coverage), build speed, accessibility, flaky-test
|
|
301
|
+
and race-condition heuristics, and developer-experience checks (unused exports, duplicate
|
|
302
|
+
functions).
|
|
303
|
+
|
|
304
|
+
Each check either contributes a real number, or prints `skipped` with a one-line hint on
|
|
305
|
+
how to install the tool it needs — it never makes up a number. Everything adds up to one
|
|
306
|
+
box that always opens with the **OpenPitStop Score**: a single 0–100 health number (with an
|
|
307
|
+
A–F grade) across the categories that actually ran.
|
|
308
|
+
|
|
309
|
+
Scans are fast by design: the checks run **in parallel**, flaky detection runs the suite
|
|
310
|
+
**twice** by default (`--reliability-runs <n>` to tune; `1` disables it), and `npm audit`
|
|
311
|
+
(plus `osv-scanner`) results are cached for 24 hours (keyed on the lockfile hash) so
|
|
312
|
+
repeated scans inside one fix loop never hit the registry again.
|
|
313
|
+
|
|
314
|
+
## Every command
|
|
315
|
+
|
|
316
|
+
All 23 commands, grouped by job. Run them from inside a repo as `pitstop …` (CLI) or
|
|
317
|
+
`npx openpitstop …` (one-off); `/pitstop` in a tool drives the loop, the rest are
|
|
318
|
+
one-shot.
|
|
319
|
+
|
|
320
|
+
**Measure — the numbers**
|
|
321
|
+
|
|
322
|
+
| Command | What it does |
|
|
323
|
+
|---|---|
|
|
324
|
+
| `pitstop scan [path] [--json] [--reuse] [--ledger]` | The big one. Runs every check in parallel and prints one box with a single **OpenPitStop Score** (0–100, A–F). `--json` for scripts and pipelines; `--reuse` returns the saved baseline when nothing changed; `--ledger` also fuzzes payment idempotency. |
|
|
325
|
+
| `pitstop verify` | Re-scans after a change and shows exactly how the score moved — the numbers can't be argued with. Also checks your diff for agent-cheat patterns. |
|
|
326
|
+
| `pitstop try [path]` | Get a score on **any** repo in ~2 seconds of scanning — no install, no config, no setup (the first `npx` run on a machine downloads the package once). Saves a sealed baseline so verify and gate can build on it later. |
|
|
327
|
+
| `pitstop ready-check [path]` | Quick "is it worth scanning again?" — nothing changed → exit 0 and reuse the baseline; something changed → exit 1. |
|
|
328
|
+
| `pitstop watch [path] [--interval ms]` | The live shield. Sits in a terminal and re-checks the moment you save a file, printing how the score moved. |
|
|
329
|
+
| `pitstop trends` | Turns your saved scan history into per-category sparklines and a score trend — watch a repo actually improve. |
|
|
330
|
+
| `pitstop budget [path]` | The token bill: how many scans/verifies/pens/repros you've run and the compute-seconds, plus advice on what to reuse in a fix loop. |
|
|
331
|
+
|
|
332
|
+
**The fix loop — what the agent is told to do**
|
|
333
|
+
|
|
334
|
+
| Command | What it does |
|
|
335
|
+
|---|---|
|
|
336
|
+
| `pitstop drive <finding-id> [path]` | Hands one finding to *your own agent* (`PITSTOP_AGENT` or `--agent '…{prompt}'`) with explicit orders: write a failing repro first, fix it, make the repro pass, then verify. OpenPitStop referees the whole thing and never edits your code. |
|
|
337
|
+
| `pitstop memory add/list/relevant` | A scratchpad inside the repo: record a decision (`add`), see them newest-first (`list`), or pull up anything related to a file (`relevant`) — so past fixes and rejected approaches survive across sessions. |
|
|
338
|
+
| `pitstop inspect <finding-id>` | Opens up one finding: the code snippet, the root cause, whether a repro test exists, and what OpenPitStop remembers about these files. |
|
|
339
|
+
|
|
340
|
+
**Integrity & anti-cheat — the referee**
|
|
341
|
+
|
|
342
|
+
| Command | What it does |
|
|
343
|
+
|---|---|
|
|
344
|
+
| `pitstop gate [--score 60]` | A commit gate for CI or pre-commit: score threshold + regression risk + diff integrity + evidence signature. **Exit 0 = PASS · 1 = FAIL · 2 = CONFIRMED_CHEAT.** |
|
|
345
|
+
| `pitstop integrity [path]` | Checks the latest commit or working tree for cheat patterns *without* a full scan: deleted or neutered tests, swallowed errors, suppression comments, hardcoded-to-pass values, mocked modules, forced exits. **Exit 0 = CLEAN · 1 = SUSPICIOUS · 2 = CONFIRMED_CHEAT.** |
|
|
346
|
+
| `pitstop ci [path]` | CI-friendly scan + verify against the base branch → a PR-ready markdown report — the gate as a PR comment. It only reports; fixes stay local via `/pitstop`. |
|
|
347
|
+
|
|
348
|
+
**Penetration test — attack your own app**
|
|
349
|
+
|
|
350
|
+
| Command | What it does |
|
|
351
|
+
|---|---|
|
|
352
|
+
| `pitstop pen [path] [--fix] [--html] [--json]` | A real pen test of your own app. Static heuristics find candidates (secrets, routes, injection/SSRF/XSS), then it **boots the app in a sandbox** and attacks it live, recording every outbound HTTP call and spawned process. Findings are labeled `PROVEN` only when the sandbox saw real evidence; everything else is honestly `indicated`/`unproven`. Nothing reaches the real network; raw sockets are blocked. `--fix` writes failing-then-passing repro tests + `git apply`-able patches. Exit 0 = clean · 1 = high/critical · 2 = aborted. |
|
|
353
|
+
| `pitstop inspect <pen-id>` | Deep-dives a pen finding: exactly what attack was fired, what the app responded with, the sandbox evidence lines, the fix, the repro. |
|
|
354
|
+
| `pitstop repro <pen-id>` | Turns a pen finding into a regression test that boots the app — fails now, must pass after the fix. |
|
|
355
|
+
|
|
356
|
+
**Proof & reports — what you show people**
|
|
357
|
+
|
|
358
|
+
| Command | What it does |
|
|
359
|
+
|---|---|
|
|
360
|
+
| `pitstop report --html` | One self-contained `PITSTOP_REPORT.html` (inline SVG trends, integrity timeline, zero external assets). Also writes `PITSTOP_BADGE.svg` — a README-ready shield: ``. |
|
|
361
|
+
| `pitstop share` | Renders a 1200×630 share card (`PITSTOP_CARD.html`) — score, trend, integrity/evidence chips, top findings. Screenshot it and post it. |
|
|
362
|
+
| `pitstop digest [--days N] [--md]` | The progress story: how the score moved, what got fixed and what regressed, gate results, cheat catches, flakies, open findings. |
|
|
363
|
+
| `pitstop honesty [--html]` | The proof that the numbers are real: evidence chain + integrity history + verify deltas + committed repro tests → one verdict, or a shareable HTML certificate. |
|
|
364
|
+
|
|
365
|
+
**Setup & transparency**
|
|
366
|
+
|
|
367
|
+
| Command | What it does |
|
|
368
|
+
|---|---|
|
|
369
|
+
| `pitstop install` / `install --uninstall` | Writes `/pitstop` into every supported tool (project + user level). `--uninstall` removes it all. |
|
|
370
|
+
| `pitstop doctor` | Explains why categories show `skipped`: checks your toolchain (Node, git, jscpd, gitleaks, semgrep, pa11y) and prints copy-paste install hints. |
|
|
371
|
+
| `pitstop prompt [--args …]` | Prints the exact prompt your AI tool expands `/pitstop` into, with your arguments filled in — full transparency into what the agent was told. |
|
|
372
|
+
| `pitstop demo [demo]` | Scaffolds an intentionally-broken demo repo into a fresh temp dir (`demo-repo`, `demo-repo-integrity`, `demo-repo-fintech`, `demo-repo-generators`), initializes git, and scans it immediately. |
|
|
373
|
+
|
|
374
|
+
### The score & badge
|
|
375
|
+
|
|
376
|
+
Skipped categories are excluded and the weights re-adjusted, so a missing `jscpd` never
|
|
377
|
+
silently drags the number down. The verify Δ compares against the last scan with the *exact
|
|
378
|
+
same categories measured* — a category skipped on both sides can't move the score. The
|
|
379
|
+
score only moves when your code does.
|
|
380
|
+
|
|
381
|
+
### Tamper-evident evidence
|
|
382
|
+
|
|
383
|
+
Every scan, verify and integrity document OpenPitStop writes gets a `sha256` fingerprint of
|
|
384
|
+
its own contents (`pitstop-sha256-canonical-v1`, deterministic key-sorted JSON). Edit the
|
|
385
|
+
JSON after the fact — inflate a score, delete a finding — and the next
|
|
386
|
+
`pitstop verify`/`pitstop gate` recomputes the fingerprint, sees the mismatch, and
|
|
387
|
+
reports the chain as broken. OpenPitStop can't be tricked into endorsing a baseline it didn't
|
|
388
|
+
write; the `gate` exit code treats a broken chain as a hard fail.
|
|
389
|
+
|
|
390
|
+
### Prompt transparency
|
|
391
|
+
|
|
392
|
+
Some AI tools show you the expanded slash-command prompt in their UI, some don't. OpenPitStop
|
|
393
|
+
keeps your chat clean either way: the `/pitstop` agent acknowledges with a single short line
|
|
394
|
+
and gets straight to work — the full instruction set stays out of your window. And
|
|
395
|
+
`pitstop prompt` lets you preview the raw prompt before anyone types anything.
|
|
396
|
+
|
|
397
|
+
### Root-cause correlation
|
|
398
|
+
|
|
399
|
+
Findings that touch the same files get grouped into one root cause, so the box shows
|
|
400
|
+
`1 root cause → 2 symptoms` instead of a flat list. Every cluster gets a stable id (e.g.
|
|
401
|
+
`security-19c390c6`) that the repro step can reference.
|
|
402
|
+
|
|
403
|
+
### Confirm, then loop
|
|
404
|
+
|
|
405
|
+
The agent prints the boxed summary, then **waits for your one-time OK**. After that it
|
|
406
|
+
works through each cluster on a `pitstop/*` branch: capture the bug as a failing test
|
|
407
|
+
first (`pitstop repro <id>`), make the smallest fix, pass the same repro test, run
|
|
408
|
+
`pitstop verify`, and commit. It re-scans after every fix and stops when a fresh scan
|
|
409
|
+
shows zero clusters (hard limits: 10 fix rounds or 45 minutes), ending with a
|
|
410
|
+
`PITSTOP_REPORT.md`.
|
|
411
|
+
|
|
412
|
+
### Ledger mode (opt-in)
|
|
413
|
+
|
|
414
|
+
`pitstop scan --ledger` boots your app with **every outbound HTTP call rerouted to a mock
|
|
415
|
+
gateway**, then replays the three classic payment bugs: duplicate webhook, concurrent
|
|
416
|
+
double-submit, delayed retry. If the mock gateway's own receipt log shows more than one
|
|
417
|
+
charge per idempotency key, that's a **proven double-charge** — not a guess. The shipped
|
|
418
|
+
`demo-repo-fintech` fixture produces three such PROVEN findings because its charge and
|
|
419
|
+
webhook endpoints have no idempotency guard. If the sandbox can't intercept some traffic,
|
|
420
|
+
the run aborts (`exit 77`); nothing ever reaches a real gateway.
|
|
421
|
+
|
|
422
|
+
**Which stacks are covered?** Node/JS apps run under the nock preload, which intercepts
|
|
423
|
+
every outbound call in-process. Go, Python, Rust and .NET apps run under a recording
|
|
424
|
+
`HTTP_PROXY` sandbox that answers the known payment-gateway hosts with mocked receipts
|
|
425
|
+
and 502s everything else. Java and Dart are refused (their HTTP clients don't honor
|
|
426
|
+
`HTTP_PROXY`, so interception could not be guaranteed). HTTPS stays blocked (502): without
|
|
427
|
+
a trusted CA the proxy cannot terminate a CONNECT tunnel, so an HTTPS double-charge is
|
|
428
|
+
reported as *indicated*, never *proven*. Native binaries and raw sockets bypass the proxy
|
|
429
|
+
and are not observed. Set `PITSTOP_START` to override start-command guessing for
|
|
430
|
+
non-Node repos.
|
|
431
|
+
|
|
432
|
+
### Integrity gate
|
|
433
|
+
|
|
434
|
+
Every `pitstop verify` also diffs your change against HEAD and checks for the classic
|
|
435
|
+
agent-cheat moves: deleted or loosened tests, tests focused to hide failures
|
|
436
|
+
(`fit`/`test.only`), swallowed exceptions, suppression comments, hardcoded-to-pass values,
|
|
437
|
+
a mocked module-under-test, a forced `exit(0)` in app code, or an assertion's expected
|
|
438
|
+
value edited to match the buggy output. A caught cheat looks like this: change
|
|
439
|
+
`assert.equal(round2(8.075), 8.08)` to expect `8.07` with nothing else in the diff →
|
|
440
|
+
`CONFIRMED_CHEAT`, the change is blocked, and a human reviews it (verified against
|
|
441
|
+
`fixtures/assertion-literal-tamper/`). An honest app-side fix sails through `CLEAN`.
|
|
442
|
+
|
|
443
|
+
### Cheat-catch demo
|
|
444
|
+
|
|
445
|
+
Want to *see* it? The scripted arc from **[See it in 90 seconds](#see-it-in-90-seconds)**
|
|
446
|
+
is `scripts/cheat-demo.cjs` — a fully deterministic SUSPICIOUS → CONFIRMED_CHEAT
|
|
447
|
+
sequence against a real repo with a real failing jest test. Point it at a build with
|
|
448
|
+
`PITSTOP_CLI="node /path/to/dist/cli.js"`, or let it use `npx openpitstop`. Great for a
|
|
449
|
+
video or a live judge's demo.
|
|
450
|
+
|
|
451
|
+
## Architecture
|
|
452
|
+
|
|
453
|
+
OpenPitStop is two pieces that never mix: a **CLI that measures**, and **your host agent that
|
|
454
|
+
reasons and edits**. The CLI produces the scan/verify numbers and the gate verdicts; the
|
|
455
|
+
model in whichever tool you're using reads them, decides what to change, and does the
|
|
456
|
+
editing through the `/pitstop` prompt template. This is deliberately *not* one monolithic
|
|
457
|
+
agent — the numbers can't be talked into looking better, and the agent can't silently
|
|
458
|
+
cheat its own referee. That separation is the product.
|
|
459
|
+
|
|
460
|
+
## Known limitations
|
|
461
|
+
|
|
462
|
+
- **Windows** is CI-verified on every push (build + smoke on `ubuntu-latest` and
|
|
463
|
+
`windows-latest`), and the Windows-specific bugs were reproduced and fixed on a real
|
|
464
|
+
Windows host during development — but not every workflow has been exhaustively manually
|
|
465
|
+
tested on Windows.
|
|
466
|
+
- **Codex App / VS Code extension** isn't supported and won't be until OpenAI ships custom
|
|
467
|
+
slash commands; use Codex CLI for `/pitstop`.
|
|
468
|
+
- **Graceful degradation:** duplication (`jscpd`), secrets/code scanning (`gitleaks`,
|
|
469
|
+
`semgrep`), dependency CVEs (`pip-audit`, `osv-scanner`), and accessibility runtime
|
|
470
|
+
checks (`pa11y`/`axe`) run only when that tool is installed locally. The scan reports
|
|
471
|
+
`skipped` for those categories and works fine without them.
|
|
472
|
+
- Requires **Node.js 22+** (the CLI depends on execa 10, which uses ES2024 `Set.union`).
|
|
473
|
+
- **Multi-stack honesty:** test runs (JS, Python, Go, Rust, Flutter, .NET, Java via Maven
|
|
474
|
+
or Gradle), dependency CVEs, and the `pen`/`ledger` sandboxes are real for Node/JS and
|
|
475
|
+
best-effort elsewhere. Go/Rust/Python/.NET apps run under the `HTTP_PROXY` recording
|
|
476
|
+
sandbox (see Ledger mode); Java and Dart are refused for ledger, and proxy-mode results
|
|
477
|
+
are labelled `indicated`, never `proven`, when the proxy cannot observe the traffic. The
|
|
478
|
+
native test runners parse each toolchain's real output (`go test -json`, `cargo test
|
|
479
|
+
--format json`, `flutter test --machine`, dotnet/maven/gradle summaries) and report
|
|
480
|
+
`skipped` when a toolchain isn't on PATH.
|
|
481
|
+
- **Pen-test honesty:** `pitstop pen` reports each finding with a **runtime-proof
|
|
482
|
+
verdict**: **proven** (the live dynamic attack confirmed it under the sandbox),
|
|
483
|
+
**indicated** (static rule fired but the dynamic phase couldn't confirm), **unproven**
|
|
484
|
+
(the dynamic phase ran and found no evidence for that rule), or **not-tested** (dynamic
|
|
485
|
+
phase aborted). Proven findings are real; everything else is a hypothesis until you
|
|
486
|
+
replay the attack yourself. The sandbox records outbound connections and spawned
|
|
487
|
+
processes instead of blocking them (so real bytes never leave your machine for canaries,
|
|
488
|
+
but a compromised app could still run commands locally); raw socket APIs are blocked
|
|
489
|
+
outright. `pen --fix` writes deterministic patches **only** for findings fixable by pure
|
|
490
|
+
insertion (e.g. missing `helmet()`, `x-powered-by` leaks) — anything else gets a failing
|
|
491
|
+
repro test and fix guidance, which is your contract for the fix.
|
|
492
|
+
- **`drive` verdicts** for runtime pen findings come from the repro test (FAIL first, PASS
|
|
493
|
+
after the fix), not from the static score — the static gate has nothing to say about a
|
|
494
|
+
runtime-only finding.
|
|
495
|
+
|
|
496
|
+
## Contributing
|
|
497
|
+
|
|
498
|
+
OpenPitStop is built to be extended — adding a whole new analyzer is a small, well-scoped
|
|
499
|
+
change. See [CONTRIBUTING.md](CONTRIBUTING.md) for the analyzer interface, conventions, and
|
|
500
|
+
how to open a PR. For the launch notes and the "why", read [LAUNCH.md](LAUNCH.md).
|
|
501
|
+
|
|
502
|
+
## License
|
|
503
|
+
|
|
504
|
+
[MIT](LICENSE)
|
|
505
|
+
|
|
506
|
+
---
|
|
507
|
+
|
|
508
|
+
<p align="center">
|
|
509
|
+
<img src="docs/media/pitstop-icon.png" alt="OpenPitStop icon" width="64">
|
|
510
|
+
</p>
|
|
511
|
+
|
|
512
|
+
<p align="center">Built by <b>Krish J</b> — if it can referee itself, it can referee your agent.</p>
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# pitstop-demo-api
|
|
2
|
+
|
|
3
|
+
> **This is OpenPitStop's intentionally-broken demo repository.** Do not use it as a
|
|
4
|
+
> real application. It is seeded with genuine, detectable problems so you can watch
|
|
5
|
+
> `pitstop` scan, pause for confirmation, then autonomously fix and verify in a loop.
|
|
6
|
+
|
|
7
|
+
Seeded problems (so you can check the numbers in `PITSTOP_REPORT.md`):
|
|
8
|
+
|
|
9
|
+
- **Circular dependencies** between `userRepo` ↔ `userService` ↔ `userController` (a real
|
|
10
|
+
3-module import cycle).
|
|
11
|
+
- **Hardcoded secret** in `src/config.js` (a fake AWS-style key + DB password) — `gitleaks`
|
|
12
|
+
flags it when installed.
|
|
13
|
+
- **Outdated dependency with a known CVE**: `lodash@4.17.15` (CVE-2021-23337). `npm audit`
|
|
14
|
+
flags it.
|
|
15
|
+
- **Copy-pasted block** duplicated across `src/userService.js` and `src/userController.js`
|
|
16
|
+
— `jscpd` flags it when installed.
|
|
17
|
+
- **2 failing unit tests** in `test/user.test.js` (assert correct behavior; the implementation
|
|
18
|
+
is buggy on purpose).
|
|
19
|
+
- **N+1 query pattern** spanning `userController` → `userService` → `userRepo` (the service
|
|
20
|
+
loops users and queries posts per-user). Phase 2's clustering groups these three files into
|
|
21
|
+
one root-cause cluster.
|
|
22
|
+
|
|
23
|
+
Run it: `pitstop demo` copies this into a temp dir, installs the slash command, and tells you
|
|
24
|
+
where to open it.
|