openpitstop 1.9.0 → 1.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/PRIVACY.md +77 -90
- package/README.md +365 -887
- package/dist/agentBudget.d.ts +33 -0
- package/dist/agentBudget.js +165 -0
- package/dist/agentBudget.js.map +1 -0
- package/dist/analyzers/ledger/index.d.ts +2 -1
- package/dist/analyzers/ledger/index.js +10 -1
- package/dist/analyzers/ledger/index.js.map +1 -1
- package/dist/analyzers/types.d.ts +2 -0
- package/dist/candidate.d.ts +20 -0
- package/dist/candidate.js +87 -0
- package/dist/candidate.js.map +1 -0
- package/dist/cli.js +1 -1
- package/dist/cli.js.map +1 -1
- package/dist/commands/ask.d.ts +0 -8
- package/dist/commands/ask.js +36 -34
- package/dist/commands/ask.js.map +1 -1
- package/dist/commands/budget.d.ts +1 -0
- package/dist/commands/budget.js +12 -3
- package/dist/commands/budget.js.map +1 -1
- package/dist/commands/drive.js +77 -51
- package/dist/commands/drive.js.map +1 -1
- package/dist/commands/fix.d.ts +7 -0
- package/dist/commands/fix.js +90 -108
- package/dist/commands/fix.js.map +1 -1
- package/dist/commands/gate.js +6 -2
- package/dist/commands/gate.js.map +1 -1
- package/dist/commands/pen.d.ts +3 -1
- package/dist/commands/pen.js +31 -42
- package/dist/commands/pen.js.map +1 -1
- package/dist/commands/readyCheck.d.ts +1 -1
- package/dist/commands/readyCheck.js +6 -36
- package/dist/commands/readyCheck.js.map +1 -1
- package/dist/commands/scan.d.ts +6 -5
- package/dist/commands/scan.js +26 -28
- package/dist/commands/scan.js.map +1 -1
- package/dist/commands/verify.d.ts +3 -1
- package/dist/commands/verify.js +5 -1
- package/dist/commands/verify.js.map +1 -1
- package/dist/evidence.js +4 -0
- package/dist/evidence.js.map +1 -1
- package/dist/installer/targets.js +1 -1
- package/dist/installer/targets.js.map +1 -1
- package/dist/intent.d.ts +4 -7
- package/dist/intent.js +41 -77
- package/dist/intent.js.map +1 -1
- package/dist/pen/dynamic.d.ts +1 -24
- package/dist/pen/dynamic.js +10 -1
- package/dist/pen/dynamic.js.map +1 -1
- package/dist/pen/fix.js +5 -1
- package/dist/pen/fix.js.map +1 -1
- package/dist/repro/generate.js +1 -0
- package/dist/repro/generate.js.map +1 -1
- package/dist/repro/pen.js +1 -0
- package/dist/repro/pen.js.map +1 -1
- package/dist/repro/run.d.ts +1 -0
- package/dist/repro/run.js +17 -1
- package/dist/repro/run.js.map +1 -1
- package/dist/sandbox/container.d.ts +9 -0
- package/dist/sandbox/container.js +129 -0
- package/dist/sandbox/container.js.map +1 -0
- package/dist/sandbox/worker.d.ts +1 -0
- package/dist/sandbox/worker.js +41 -0
- package/dist/sandbox/worker.js.map +1 -0
- package/dist/scanCache.d.ts +27 -0
- package/dist/scanCache.js +89 -0
- package/dist/scanCache.js.map +1 -0
- package/dist/verify/acceptance.d.ts +5 -1
- package/dist/verify/acceptance.js +29 -16
- package/dist/verify/acceptance.js.map +1 -1
- package/dist/verify/architecture.d.ts +5 -1
- package/dist/verify/architecture.js +6 -1
- package/dist/verify/architecture.js.map +1 -1
- package/dist/verify/baseline.d.ts +5 -1
- package/dist/verify/baseline.js +5 -1
- package/dist/verify/baseline.js.map +1 -1
- package/dist/verify/chain.js +17 -0
- package/dist/verify/chain.js.map +1 -1
- package/dist/verify/flow.d.ts +2 -0
- package/dist/verify/flow.js +18 -3
- package/dist/verify/flow.js.map +1 -1
- package/dist/verify/gateMatrix.d.ts +4 -0
- package/dist/verify/gateMatrix.js +23 -12
- package/dist/verify/gateMatrix.js.map +1 -1
- package/dist/verify/holdout.d.ts +5 -1
- package/dist/verify/holdout.js +8 -2
- package/dist/verify/holdout.js.map +1 -1
- package/dist/verify/regression.d.ts +5 -1
- package/dist/verify/regression.js +6 -1
- package/dist/verify/regression.js.map +1 -1
- package/dist/verify/stack.d.ts +5 -1
- package/dist/verify/stack.js +6 -1
- package/dist/verify/stack.js.map +1 -1
- package/dist/verify/state.d.ts +5 -1
- package/dist/verify/state.js +6 -1
- package/dist/verify/state.js.map +1 -1
- package/dist/verify/verifier.d.ts +5 -1
- package/dist/verify/verifier.js +6 -1
- package/dist/verify/verifier.js.map +1 -1
- package/docs/release-controls.md +107 -0
- package/package.json +70 -69
- package/templates/pitstop.command.md +17 -5
- package/templates/pitstop.prompt.md +390 -390
- package/scripts/__pycache__/gen-gifs.cpython-313.pyc +0 -0
- package/scripts/gen-gifs.py +0 -231
package/README.md
CHANGED
|
@@ -1,887 +1,365 @@
|
|
|
1
|
-
<p align="center">
|
|
2
|
-
<img src="docs/media/pitstop-logo.png" alt="OpenPitStop" width="380">
|
|
3
|
-
</p>
|
|
4
|
-
|
|
5
|
-
# OpenPitStop
|
|
6
|
-
|
|
7
|
-
**
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
[.
|
|
378
|
-
|
|
379
|
-
### The GitHub Action
|
|
380
|
-
|
|
381
|
-
`uses: openpitstop/action` (or `Krish-1507/OpenPitStop@main`) puts the gate on every PR
|
|
382
|
-
as a comment and a failing check when it matters. No wiring by hand. See
|
|
383
|
-
[docs/github-action.md](docs/github-action.md).
|
|
384
|
-
|
|
385
|
-
### The pre-commit hook
|
|
386
|
-
|
|
387
|
-
`npx openpitstop install --hooks` installs the gate one step earlier: the commit
|
|
388
|
-
cannot land until the gate passes. See
|
|
389
|
-
[docs/caught-in-the-wild.md](docs/caught-in-the-wild.md).
|
|
390
|
-
|
|
391
|
-
### Ledger mode (payment proof)
|
|
392
|
-
|
|
393
|
-
`pitstop scan --ledger` boots your app with every outbound HTTP call rerouted to a mock
|
|
394
|
-
gateway, then replays the classic payment bugs (duplicate webhook, concurrent
|
|
395
|
-
double-submit, delayed retry). If the mock shows more than one charge per idempotency
|
|
396
|
-
key, that is a **proven double-charge**, not a guess.
|
|
397
|
-
|
|
398
|
-
### CI reports
|
|
399
|
-
|
|
400
|
-
`pitstop ci` runs a CI-friendly scan plus verify against the base branch and writes a
|
|
401
|
-
PR-ready markdown report, the gate as a PR comment. This is the engine behind the
|
|
402
|
-
GitHub Action.
|
|
403
|
-
|
|
404
|
-
### Ready-check and doctor
|
|
405
|
-
|
|
406
|
-
`pitstop ready-check` answers "is it worth scanning again?" and reuses the baseline when
|
|
407
|
-
nothing changed. `pitstop doctor` explains why a category shows `skipped` and prints
|
|
408
|
-
copy-paste install hints for the tools you are missing.
|
|
409
|
-
|
|
410
|
-
### Digest (progress story)
|
|
411
|
-
|
|
412
|
-
`pitstop digest` turns your history into a plain-English progress story: how the score
|
|
413
|
-
moved, what got fixed, what regressed, and every cheat it caught.
|
|
414
|
-
|
|
415
|
-
---
|
|
416
|
-
|
|
417
|
-
## Install
|
|
418
|
-
|
|
419
|
-
One command, that's it:
|
|
420
|
-
|
|
421
|
-
```bash
|
|
422
|
-
npx openpitstop
|
|
423
|
-
```
|
|
424
|
-
|
|
425
|
-
No arguments needed: the CLI detects your AI tools, and asks what you want —
|
|
426
|
-
install `/pitstop` into them, or score *this* repo (`try .`). Pick, and it does it.
|
|
427
|
-
(In a non-interactive terminal it skips the questions and prints the one-line menu
|
|
428
|
-
instead.)
|
|
429
|
-
|
|
430
|
-
Or go straight to the files:
|
|
431
|
-
|
|
432
|
-
```bash
|
|
433
|
-
npx openpitstop@latest install
|
|
434
|
-
```
|
|
435
|
-
|
|
436
|
-
Run it from inside any project directory. It writes the `/pitstop` command into every
|
|
437
|
-
supported tool below — project-level for the current repo, user-level so it works in any
|
|
438
|
-
repo on your machine. Re-run with `-y` to refresh after updates (it's safe to re-run):
|
|
439
|
-
|
|
440
|
-
```bash
|
|
441
|
-
npx openpitstop install -y
|
|
442
|
-
```
|
|
443
|
-
|
|
444
|
-
Re-installing overwrites each tool's `/pitstop` command file with the latest prompt
|
|
445
|
-
(say, a new mode or an updated loop) — your tool picks it up on its next use.
|
|
446
|
-
|
|
447
|
-
Want the gate *before* the commit, not just on the PR? One extra flag installs the
|
|
448
|
-
pre-commit hook — every commit is checked (SUSPICIOUS/CONFIRMED_CHEAT → blocked) before
|
|
449
|
-
it can land:
|
|
450
|
-
|
|
451
|
-
```bash
|
|
452
|
-
npx openpitstop install --hooks
|
|
453
|
-
```
|
|
454
|
-
|
|
455
|
-
The hook runs the same `pitstop gate` (exit 0 = PASS · 1 = FAIL · 2 = CONFIRMED_CHEAT),
|
|
456
|
-
never blocks the first commit of a repo, never jails a repo that hasn't been scanned yet
|
|
457
|
-
(it warns instead), and can be bypassed once with `git commit --no-verify`. Remove it with
|
|
458
|
-
`npx openpitstop install --uninstall --hooks`. To point the hook at a local build, export
|
|
459
|
-
`PITSTOP_CLI` (e.g. `PITSTOP_CLI="node /path/to/dist/cli.js"`).
|
|
460
|
-
|
|
461
|
-
Speed tip: the `try`/`scan` itself takes ~2 seconds — but the **first** `npx openpitstop …`
|
|
462
|
-
on a machine has to download the package first (a few seconds on a fast connection, more on a
|
|
463
|
-
slow one). For an instant first run on machines you own, install once:
|
|
464
|
-
|
|
465
|
-
```bash
|
|
466
|
-
npm i -g openpitstop
|
|
467
|
-
openpitstop try .
|
|
468
|
-
```
|
|
469
|
-
|
|
470
|
-
Requires Node.js 22+ (npm will warn on older versions).
|
|
471
|
-
|
|
472
|
-
## Usage
|
|
473
|
-
|
|
474
|
-
Open your repo in any supported tool and type:
|
|
475
|
-
|
|
476
|
-
```
|
|
477
|
-
/pitstop
|
|
478
|
-
```
|
|
479
|
-
|
|
480
|
-
Bare `/pitstop` runs the full quality loop **immediately** — scan, one confirmation pause,
|
|
481
|
-
fix, verify, repeat. No menu, no waiting. Everything below is the power paths on top of
|
|
482
|
-
that:
|
|
483
|
-
|
|
484
|
-
| Invocation | Mode | What it does |
|
|
485
|
-
|---|---|---|
|
|
486
|
-
| `/pitstop` (bare) | **default full loop** | Scans right away, prints the boxed report, one confirmation pause, then the autonomous fix loop — repeat until clean. |
|
|
487
|
-
| `/pitstop --menu` | menu | Prints the full mode list below and **waits** — handy if you forgot the flags. |
|
|
488
|
-
| `/pitstop --scan-only` | scan-only | Runs `openpitstop scan`, prints the entire boxed report verbatim, and stops — no fixes, no commentary. |
|
|
489
|
-
| `/pitstop --ledger` | ledger | Runs `openpitstop scan --ledger` (boots the app with every outbound HTTP call intercepted and replays duplicate-webhook / double-submit / retry traffic), then runs the loop restricted to the payment findings. |
|
|
490
|
-
| `/pitstop --integrity-only` | integrity-only | Runs `openpitstop integrity`, prints the boxed verdict verbatim, and stops — no scanning, no fixes. |
|
|
491
|
-
| `/pitstop --pen` | pen | Live penetration test with proof — see [The pen test](#the-pen-test). |
|
|
492
|
-
| `/pitstop <your question>` | custom ask | Any free-form text (e.g. `check the security of this app`, `are our tests flaky?`, `did my agent cheat on the last commit?`) is scoped to exactly that ask: the agent maps it to the right command (`pen` for security, `integrity` for cheats, `scan` for health/tests…), states its interpretation in one line, confirms before fixing, and fixes only what you asked. |
|
|
493
|
-
|
|
494
|
-
For reference, `/pitstop --menu` shows this list:
|
|
495
|
-
|
|
496
|
-
```
|
|
497
|
-
OpenPitStop modes:
|
|
498
|
-
(enter) — full autonomous loop (scan, confirm, fix, verify, repeat)
|
|
499
|
-
--scan-only — scan and report, no fixes
|
|
500
|
-
--ledger — payment idempotency fuzzing only
|
|
501
|
-
--integrity-only — re-check the last commit for cheat patterns, no scanning
|
|
502
|
-
--pen — penetration test: live attacks + proof + fixes (regression tests, patches)
|
|
503
|
-
(your own ask) — reply with anything else, e.g. "check the security of this app"
|
|
504
|
-
```
|
|
505
|
-
|
|
506
|
-
A flag after `/pitstop` picks a specific mode; any free-form text after it becomes a scoped
|
|
507
|
-
custom ask; bare `/pitstop` is the full loop. If a tool ever fails to substitute arguments,
|
|
508
|
-
`/pitstop` behaves as bare — the default full loop — rather than guessing.
|
|
509
|
-
|
|
510
|
-
## Tool support
|
|
511
|
-
|
|
512
|
-
| Tool | Installed to | Status |
|
|
513
|
-
|------|--------------|--------|
|
|
514
|
-
| Claude Code | `.claude/commands/pitstop.md` (project + user), plus a Skill at `.claude/skills/pitstop/SKILL.md | Full support |
|
|
515
|
-
| Cursor | `.cursor/commands/pitstop.md` (project + user) | Full support |
|
|
516
|
-
| OpenCode | `.opencode/commands/pitstop.md` (project), `~/.config/opencode/commands/` (user) | Full support |
|
|
517
|
-
| Kilo Code | `.kilo/commands/pitstop.md` (project), `~/.config/kilo/commands/` (user) | Full support |
|
|
518
|
-
| Antigravity | `.agent/workflows/pitstop.md` (project + user) | Full support |
|
|
519
|
-
| Gemini CLI | `.gemini/commands/pitstop.toml` (project + user) | Full support |
|
|
520
|
-
| Codex CLI | `~/.codex/prompts/pitstop.md` | Full support |
|
|
521
|
-
| FreeBuff CLI | portable `pitstop.md` in your tool's commands folder | Full support (portable) |
|
|
522
|
-
| Grok Build CLI | portable `pitstop.md` in your tool's commands folder | Full support (portable) |
|
|
523
|
-
| MUSE Code CLI | portable `pitstop.md` in your tool's commands folder | Full support (portable) |
|
|
524
|
-
| Any other agent CLI | portable `pitstop.md` (drop it in the commands folder) | Full support (portable) — see below |
|
|
525
|
-
| Codex App / VS Code extension | — (no file written) | **Not supported** — OpenAI hasn't shipped custom slash commands there; install prints a manual-copy note instead |
|
|
526
|
-
| GitHub Action (PRs) | `uses: Krish-1507/OpenPitStop@main` | **Full support** — gate verdict as a PR comment + failing check; see [docs/github-action.md](docs/github-action.md) |
|
|
527
|
-
| git pre-commit hook | `.git/hooks/pre-commit` (installed with `--hooks`) | **Full support** — the gate blocks the commit before it lands |
|
|
528
|
-
|
|
529
|
-
**Works with any agent CLI.** OpenPitStop's `/pitstop` is a portable command file:
|
|
530
|
-
run `pitstop prompt` to print the exact instruction text, then paste it as a custom
|
|
531
|
-
slash command in any coding CLI that supports them (FreeBuff, Grok Build, MUSE Code and
|
|
532
|
-
others included above). The CLIs listed by name also get a dedicated path written
|
|
533
|
-
automatically by `pitstop install` when their commands-folder convention is known. Tell
|
|
534
|
-
us your CLI and we'll add it to the auto-install list. Legacy/alternate locations are
|
|
535
|
-
also written where tool docs are inconsistent across versions (see
|
|
536
|
-
`src/installer/targets.ts`). Existing files are never overwritten unless you pass
|
|
537
|
-
`-y`/`--force`; `npx openpitstop install --uninstall` removes everything.
|
|
538
|
-
|
|
539
|
-
## What OpenPitStop actually does
|
|
540
|
-
|
|
541
|
-
### The loop at a glance
|
|
542
|
-
|
|
543
|
-
<p align="center">
|
|
544
|
-
<img src="docs/media/pitstop-loop.png" alt="OpenPitStop's autonomous loop: scan → report → you confirm → repro (must FAIL first) → fix → verify → repeat until a fresh scan shows zero clusters" width="900">
|
|
545
|
-
</p>
|
|
546
|
-
|
|
547
|
-
OpenPitStop never touches your code. It checks, scores, and referees — your AI agent does the
|
|
548
|
-
editing, knowing it's being watched.
|
|
549
|
-
|
|
550
|
-
### The scan
|
|
551
|
-
|
|
552
|
-
`pitstop scan` runs a bunch of checks on your repo: circular imports, known security
|
|
553
|
-
issues, duplicated code (`jscpd`), test results
|
|
554
|
-
(jest/vitest/pytest plus native suites for Go, Rust, Flutter/Dart, .NET and Java
|
|
555
|
-
(Maven/Gradle) — pass/fail, duration, coverage), build speed, accessibility, flaky-test
|
|
556
|
-
and race-condition heuristics, and developer-experience checks (unused exports, duplicate
|
|
557
|
-
functions).
|
|
558
|
-
|
|
559
|
-
Security is two layers:
|
|
560
|
-
|
|
561
|
-
1. **Dependency audits** — `npm audit`, plus `pip-audit`/`osv-scanner` for Python and
|
|
562
|
-
other stacks, and `gitleaks` for committed secrets. A failed audit is reported as
|
|
563
|
-
`skipped` with a repair hint — deps that were never scanned are never reported as clean.
|
|
564
|
-
2. **The static vulnerability pass** (fully offline, every language, no tooling needed) —
|
|
565
|
-
the classic classes plus the full posture: **SQL injection** (concatenated/
|
|
566
|
-
interpolated queries, ORM raw builders, `$where`, Python f-strings), **authentication**
|
|
567
|
-
(cleartext password compares, missing hashing, `Math.random` tokens, inline JWT
|
|
568
|
-
secrets), **authorization** (unprotected data routes, admin routes without role
|
|
569
|
-
checks), **input validation** (unrestricted uploads, unvalidated money fields,
|
|
570
|
-
`eval`, XSS sinks), **secret management** (known credential formats, inline secret
|
|
571
|
-
literals, committed `.env` files) — plus command injection, path traversal, SSRF,
|
|
572
|
-
**rate limiting** (missing limiters on state-changing routes, limits set so high they
|
|
573
|
-
are decorations), **database lockdown** (privileged accounts in committed connection
|
|
574
|
-
strings, `GRANT ALL`/`SUPERUSER`, TLS-free connections, missing row-level security),
|
|
575
|
-
**data exposure** (credentials/PII in API responses, full DB rows shipped to the
|
|
576
|
-
client, PII in logs), **hidden vulnerabilities** (disabled TLS verification, `alg:
|
|
577
|
-
none` JWTs, security TODOs, lint/type bypasses, tokens in `localStorage`, committed
|
|
578
|
-
minified bundles, backup files), CORS+credentials, missing security headers, CSRF
|
|
579
|
-
exposure, stack leaks and sensitive logging.
|
|
580
|
-
|
|
581
|
-
Every static finding is labeled `[indicated]` and ships with its exact **fix**; `scan`
|
|
582
|
-
and `try` print the complete identify-and-solve list under the score box, so the report
|
|
583
|
-
is a worklist, not a scare. The full matrix — every detection and every fix — is in
|
|
584
|
-
[docs/security.md](docs/security.md).
|
|
585
|
-
|
|
586
|
-
**Optional deeper SAST (Semgrep).** OpenPitStop's built-in static pass needs no extra
|
|
587
|
-
tooling. For a second, cross-language engine you can bolt on [Semgrep](https://semgrep.dev)
|
|
588
|
-
— it is **off by default** and only runs when you opt in. Install it (`pip install
|
|
589
|
-
semgrep`) and set one variable:
|
|
590
|
-
|
|
591
|
-
```bash
|
|
592
|
-
export PITSTOP_SEMGREP_CONFIG=auto # the free Semgrep Registry rules
|
|
593
|
-
pitstop scan # Semgrep is picked up automatically
|
|
594
|
-
```
|
|
595
|
-
|
|
596
|
-
Point it at your own rules any time: `PITSTOP_SEMGREP_CONFIG="p/security-audit p/owasp-top-ten ./my-rules"`.
|
|
597
|
-
With nothing set, no Semgrep process ever runs — no surprise network calls, no slow scans.
|
|
598
|
-
|
|
599
|
-
Each check either contributes a real number, or prints `skipped` with a one-line hint on
|
|
600
|
-
how to install the tool it needs — it never makes up a number. Everything adds up to one
|
|
601
|
-
box that always opens with the **OpenPitStop Score**: a single 0–100 health number (with an
|
|
602
|
-
A–F grade) across the categories that actually ran.
|
|
603
|
-
|
|
604
|
-
Scans are fast by design: the checks run **in parallel**, flaky detection runs the suite
|
|
605
|
-
**twice** by default (`--reliability-runs <n>` to tune; `1` disables it), and `npm audit`
|
|
606
|
-
(plus `osv-scanner`) results are cached for 24 hours (keyed on the lockfile hash) so
|
|
607
|
-
repeated scans inside one fix loop never hit the registry again.
|
|
608
|
-
|
|
609
|
-
### The test pyramid
|
|
610
|
-
|
|
611
|
-
`pitstop test [path] [--unit] [--integration] [--e2e]` runs your **unit, integration and
|
|
612
|
-
e2e** layers the way a senior dev would — it discovers each layer (`test`/`test:unit`,
|
|
613
|
-
`test:integration`/`test:it`, `test:e2e`/`e2e` npm scripts first, then vitest/jest/
|
|
614
|
-
pytest/playwright/cypress by config), executes them, and reports per-layer pass/fail
|
|
615
|
-
counts with the **names of the failing tests**, so the fix list is actionable. Layers it
|
|
616
|
-
cannot find are reported as `skipped — no suite discovered`, never invented; the command
|
|
617
|
-
exits 1 the moment any layer fails, so CI can trust it. (Add `"test:e2e": "playwright
|
|
618
|
-
test"` to a repo and it is picked up automatically on the next run.)
|
|
619
|
-
|
|
620
|
-
## Every command
|
|
621
|
-
|
|
622
|
-
All 39 commands, grouped by job. Run them from inside a repo as `pitstop …` (CLI) or
|
|
623
|
-
`npx openpitstop …` (one-off); `/pitstop` in a tool drives the loop, the rest are
|
|
624
|
-
one-shot.
|
|
625
|
-
|
|
626
|
-
**Measure — the numbers**
|
|
627
|
-
|
|
628
|
-
| Command | What it does |
|
|
629
|
-
|---|---|
|
|
630
|
-
| `pitstop scan [path] [--json] [--reuse] [--ledger]` | The big one. Runs every check in parallel and prints one box with a single **OpenPitStop Score** (0–100, A–F). `--json` for scripts and pipelines; `--reuse` returns the saved baseline when nothing changed; `--ledger` also fuzzes payment idempotency. |
|
|
631
|
-
| `pitstop verify` | Re-scans after a change and shows exactly how the score moved — the numbers can't be argued with. Also checks your diff for agent-cheat patterns. |
|
|
632
|
-
| `pitstop try [path]` | Get a score on **any** repo in ~2 seconds of scanning — no install, no config, no setup (the first `npx` run on a machine downloads the package once). Saves a sealed baseline so verify and gate can build on it later. |
|
|
633
|
-
| `pitstop ready-check [path]` | Quick "is it worth scanning again?" — nothing changed → exit 0 and reuse the baseline; something changed → exit 1. |
|
|
634
|
-
| `pitstop watch [path] [--interval ms]` | The live shield. Sits in a terminal and re-checks the moment you save a file, printing how the score moved. |
|
|
635
|
-
| `pitstop trends` | Turns your saved scan history into per-category sparklines and a score trend — watch a repo actually improve. |
|
|
636
|
-
| `pitstop budget [path]` | The token bill: how many scans/verifies/pens/repros you've run and the compute-seconds, plus advice on what to reuse in a fix loop. |
|
|
637
|
-
| `pitstop test [path] [--unit] [--integration] [--e2e]` | The test pyramid: discovers and runs the **unit, integration and e2e** layers, reports per-layer pass/fail with the failing test names. Any failing layer → exit 1. See [The test pyramid](#the-test-pyramid). |
|
|
638
|
-
|
|
639
|
-
**The fix loop — what the agent is told to do**
|
|
640
|
-
|
|
641
|
-
| Command | What it does |
|
|
642
|
-
|---|---|
|
|
643
|
-
| `pitstop drive <finding-id> [path]` | Hands one finding to *your own agent* (`PITSTOP_AGENT` or `--agent '…{prompt}'`) with explicit orders: write a failing repro first, fix it, make the repro pass, then verify. OpenPitStop referees the whole thing and never edits your code. |
|
|
644
|
-
| `pitstop next [path]` | Reads the sealed artifacts and prints the single best next command plus a checklist of everything still open — repo-aware (detects language, framework, tests, CI). |
|
|
645
|
-
| `pitstop fix [path]` | Autopilot without an agent: chains **scan → pen --fix → verify → gate**, showing the `next` card after each hop. |
|
|
646
|
-
| `pitstop ask "<your ask>"` | Maps a plain-English request ("check the security of this app", "are our tests flaky?") to the right command, states its interpretation, and confirms before fixing. |
|
|
647
|
-
| `pitstop memory add/list/relevant` | A scratchpad inside the repo: record a decision (`add`), see them newest-first (`list`), or pull up anything related to a file (`relevant`) — so past fixes and rejected approaches survive across sessions. |
|
|
648
|
-
| `pitstop inspect <finding-id>` | Opens up one finding: the code snippet, the root cause, whether a repro test exists, and what OpenPitStop remembers about these files. |
|
|
649
|
-
|
|
650
|
-
**Integrity & anti-cheat — the referee**
|
|
651
|
-
|
|
652
|
-
| Command | What it does |
|
|
653
|
-
|---|---|
|
|
654
|
-
| `pitstop gate [--score 60]` | A commit gate for CI, pre-commit hooks or PRs: score threshold + regression risk + diff integrity + evidence signature + the newest baseline-verify / state-verify / verifier-check reports. **Exit 0 = PASS · 1 = FAIL · 2 = CONFIRMED_CHEAT.** |
|
|
655
|
-
| `pitstop integrity [path]` | Checks the latest commit or working tree for cheat patterns *without* a full scan: deleted or neutered tests, swallowed errors, suppression comments, hardcoded-to-pass values, mocked modules, forced exits. **Exit 0 = CLEAN · 1 = SUSPICIOUS · 2 = CONFIRMED_CHEAT.** |
|
|
656
|
-
| `pitstop ci [path]` | CI-friendly scan + verify against the base branch → a PR-ready markdown report — the gate as a PR comment. It only reports; fixes stay local via `/pitstop`. Wired into the [GitHub Action](docs/github-action.md), which comments the gate on every PR and fails the check when it fails. |
|
|
657
|
-
|
|
658
|
-
**Deep verification — can the referee say NO?**
|
|
659
|
-
|
|
660
|
-
| Command | What it does |
|
|
661
|
-
|---|---|
|
|
662
|
-
| `pitstop baseline-verify --baseline <ref> --command <cmd> …` | Proves a fix is real: runs the SAME verification on a known-baseline commit (must FAIL) and the candidate (must PASS), in isolated git worktrees, with sealed tamper-evident evidence and a verification-identity hash. `VERIFIED` only when both hold and nothing changed; otherwise `FAILED` / `UNPROVEN` / `INTEGRITY_FAILURE`. Exit 0/1/2/3. See [docs/baseline-verify.md](docs/baseline-verify.md). |
|
|
663
|
-
| `pitstop state-verify --claim modified:src/auth.ts …` | Independent external state check: verifies the agent's structured claims against the actual filesystem + git (existence, content hashes, line counts, porcelain status, HEAD). Catches "HTTP 200 but nothing changed", empty writes, reverts, wrong-file changes, whitespace-only edits. `STATE_VERIFIED` / `STATE_MISMATCH` / `UNPROVEN` / `INTEGRITY_FAILURE`. Exit 0/1/2/3. See [docs/state-verify.md](docs/state-verify.md). |
|
|
664
|
-
| `pitstop verifier-check --command <cmd> --mutate …` | Verifier self-test: runs the verification on a known-good state (must PASS) and a controlled known-bad state (must FAIL) in temp worktrees. `VERIFIER_VALID` = falsifiable; `VERIFIER_WEAK` = the seeded fault sailed through; `VERIFIER_BROKEN` = fails a correct state. Never mutates your working tree. See [docs/verifier-check.md](docs/verifier-check.md). |
|
|
665
|
-
| `pitstop holdout-verify --suite <dir-or-id> [--baseline <ref>]` | Final hidden exam against verifier overfitting: a holdout suite defined OUTSIDE the repo runs once in a fresh isolated worktree of the candidate commit — the agent never saw it and cannot modify it (files hashed before/after); output is redacted to ids + verdicts. With `--baseline` the suite must FAIL there and PASS on the candidate. `HOLDOUT_PASS` / `HOLDOUT_FAIL` / `HOLDOUT_UNPROVEN` / `HOLDOUT_INTEGRITY_FAILURE`. See [docs/holdout-verify.md](docs/holdout-verify.md). |
|
|
666
|
-
| `pitstop acceptance-verify --contract <dir\|file\|id> [--baseline <ref>]` | Requirement verification: a structured acceptance contract (deterministic `command`/`http`/`fileExists`/`fileContains` criteria — never an LLM judge) is the source of truth for "did the agent satisfy the original requirement?". Boots the app when the contract declares a start command; in-repo contracts are hash-pinned so the agent cannot redefine success without `--authorize`; `--baseline` exposes contracts that pass on both sides. `SATISFIED` / `NOT_SATISFIED` / `UNPROVEN` / `INTEGRITY_FAILURE`. See [docs/acceptance-verify.md](docs/acceptance-verify.md). |
|
|
667
|
-
| `pitstop regression-check --command <cmd> --baseline <ref>` | Per-check regression comparison: previously **verified passing** checks that now fail are `REGRESSION` (hard-blocks the gate); already-broken stays `UNCHANGED`; fixes are `FIXED`; new checks are `NEW_PASS`/`NEW_FAILURE`; flaky (`--runs >1`) and vanished checks are `UNPROVEN` rather than guessed. Per-test names parsed from TAP/spec/jest/pytest/go output, suite-level fallback. See [docs/regression-check.md](docs/regression-check.md). |
|
|
668
|
-
| `pitstop explain [--verbose]` | The unified evidence chain: aggregates every sealed verification document, re-verifies each seal, and derives VERIFIED / BLOCKED / UNPROVEN with per-item evidence references, digests and reasons. Never-run components are NOT_CONFIGURED — never rendered as passes. See [docs/explain.md](docs/explain.md). |
|
|
669
|
-
| `pitstop understand [path]` | Repo awareness: builds the sealed understanding artifact — languages, frameworks, verification commands (test/typecheck/lint/build), test layers, CI, module map, entry points, CODEOWNERS ownership, architecture config. Stage 1 of the pipeline. See [docs/repo-discipline.md](docs/repo-discipline.md). |
|
|
670
|
-
| `pitstop plan --goal … --path … --verify-command …` | Plan before patching: a sealed change contract (goal, steps, `expectedPaths` the change may touch, verification commands). `--show` renders the latest plan with live scope status. See [docs/repo-discipline.md](docs/repo-discipline.md). |
|
|
671
|
-
| `pitstop architecture-check [--against-plan] [--approved]` | Does the change FIT THE SYSTEM? Declared import boundaries, protected paths (auth/deploy/CI need explicit `--approved`), forbidden paths, CODEOWNERS routing, shortcut detectors on the diff, and scope-creep detection against the plan. `CONFORMS` / `APPROVAL_REQUIRED` / `VIOLATIONS` / `INTEGRITY_FAILURE`. See [docs/repo-discipline.md](docs/repo-discipline.md). |
|
|
672
|
-
| `pitstop verify-stack [--only …]` | The full verification stack — unit/integration/e2e + typecheck + lint + build, whatever the repo has — with a deterministic failure DIAGNOSIS per layer (type-error TSxxxx, missing-dependency, assertion-failure, lint rule, environment, timeout) so fixes are targeted, not random edits. See [docs/repo-discipline.md](docs/repo-discipline.md). |
|
|
673
|
-
| `pitstop flow [--baseline …] [--contract …] [--suite …]` | The whole pipeline in one command: understand → contract? → plan-scope → verify-stack → architecture → baseline? → regression? → holdout? → GATE. Unconfigured stages are SKIPPED honestly. See [docs/repo-discipline.md](docs/repo-discipline.md). |
|
|
674
|
-
|
|
675
|
-
**Penetration test — attack your own app**
|
|
676
|
-
|
|
677
|
-
| Command | What it does |
|
|
678
|
-
|---|---|
|
|
679
|
-
| `pitstop pen [path] [--fix] [--html] [--json]` | A real pen test of your own app. Static heuristics find candidates (secrets, routes, injection/SSRF/XSS), then it **boots the app in a sandbox** and attacks it live, recording every outbound HTTP call and spawned process. Findings are labeled `PROVEN` only when the sandbox saw real evidence; everything else is honestly `indicated`/`unproven`. Nothing reaches the real network; raw sockets are blocked. `--fix` writes failing-then-passing repro tests + `git apply`-able patches. Exit 0 = clean · 1 = high/critical · 2 = aborted. |
|
|
680
|
-
| `pitstop inspect <pen-id>` | Deep-dives a pen finding: exactly what attack was fired, what the app responded with, the sandbox evidence lines, the fix, the repro. |
|
|
681
|
-
| `pitstop repro <pen-id>` | Turns a pen finding into a regression test that boots the app — fails now, must pass after the fix. |
|
|
682
|
-
|
|
683
|
-
**Proof & reports — what you show people**
|
|
684
|
-
|
|
685
|
-
| Command | What it does |
|
|
686
|
-
|---|---|
|
|
687
|
-
| `pitstop report --html` | One self-contained `PITSTOP_REPORT.html` (inline SVG trends, integrity timeline, zero external assets). Also writes `PITSTOP_BADGE.svg` — a README-ready shield: ``. |
|
|
688
|
-
| `pitstop share` | Renders a 1200×630 share card (`PITSTOP_CARD.html`) — score, trend, integrity/evidence chips, top findings. Screenshot it and post it. |
|
|
689
|
-
| `pitstop digest [--days N] [--md]` | The progress story: how the score moved, what got fixed and what regressed, gate results, cheat catches, flakies, open findings. |
|
|
690
|
-
| `pitstop honesty [--html]` | The proof that the numbers are real: evidence chain + integrity history + verify deltas + committed repro tests → one verdict, or a shareable HTML certificate. |
|
|
691
|
-
|
|
692
|
-
**Setup & transparency**
|
|
693
|
-
|
|
694
|
-
| Command | What it does |
|
|
695
|
-
|---|---|
|
|
696
|
-
| `pitstop install` / `install --uninstall` | Writes `/pitstop` into every supported tool (project + user level). `--uninstall` removes it all. `--hooks` also installs (or with `--uninstall`, removes) the git pre-commit gate. |
|
|
697
|
-
| `pitstop` (no args) | The guided first-run: detects your AI tools and git repo, then offers to install or score this repo (`try .`). Non-TTY prints the one-line menu instead. |
|
|
698
|
-
| `pitstop demo` | Full loop rehearsal in a throwaway temp dir — walk the slash-command loop by hand against a seeded-broken repo, ending with a `PITSTOP_REPORT.md`. |
|
|
699
|
-
| `pitstop doctor` | Explains why categories show `skipped`: checks your toolchain (Node, git, jscpd, gitleaks, pa11y) and prints copy-paste install hints. Semgrep is optional and opt-in, so doctor won't flag its absence. |
|
|
700
|
-
| `pitstop prompt [--args …]` | Prints the exact prompt your AI tool expands `/pitstop` into, with your arguments filled in — full transparency into what the agent was told. |
|
|
701
|
-
|
|
702
|
-
### The score & badge
|
|
703
|
-
|
|
704
|
-
Skipped categories are excluded and the weights re-adjusted, so a missing `jscpd` never
|
|
705
|
-
silently drags the number down. The verify Δ compares against the last scan with the *exact
|
|
706
|
-
same categories measured* — a category skipped on both sides can't move the score. The
|
|
707
|
-
score only moves when your code does.
|
|
708
|
-
|
|
709
|
-
### Tamper-evident evidence
|
|
710
|
-
|
|
711
|
-
Every scan, verify and integrity document OpenPitStop writes gets a `sha256` fingerprint of
|
|
712
|
-
its own contents (`pitstop-sha256-canonical-v1`, deterministic key-sorted JSON). Edit the
|
|
713
|
-
JSON after the fact — inflate a score, delete a finding — and the next
|
|
714
|
-
`pitstop verify`/`pitstop gate` recomputes the fingerprint, sees the mismatch, and
|
|
715
|
-
reports the chain as broken. OpenPitStop can't be tricked into endorsing a baseline it didn't
|
|
716
|
-
write; the `gate` exit code treats a broken chain as a hard fail.
|
|
717
|
-
|
|
718
|
-
### Prompt transparency
|
|
719
|
-
|
|
720
|
-
Some AI tools show you the expanded slash-command prompt in their UI, some don't. OpenPitStop
|
|
721
|
-
keeps your chat clean either way: the `/pitstop` agent acknowledges with a single short line
|
|
722
|
-
and gets straight to work — the full instruction set stays out of your window. And
|
|
723
|
-
`pitstop prompt` lets you preview the raw prompt before anyone types anything.
|
|
724
|
-
|
|
725
|
-
### Root-cause correlation
|
|
726
|
-
|
|
727
|
-
Findings that touch the same files get grouped into one root cause, so the box shows
|
|
728
|
-
`1 root cause → 2 symptoms` instead of a flat list. Every cluster gets a stable id (e.g.
|
|
729
|
-
`security-19c390c6`) that the repro step can reference.
|
|
730
|
-
|
|
731
|
-
### Confirm, then loop
|
|
732
|
-
|
|
733
|
-
The agent prints the boxed summary, then **waits for your one-time OK**. After that it
|
|
734
|
-
works through each cluster on a `pitstop/*` branch: capture the bug as a failing test
|
|
735
|
-
first (`pitstop repro <id>`), make the smallest fix, pass the same repro test, run
|
|
736
|
-
`pitstop verify`, and commit. It re-scans after every fix and stops when a fresh scan
|
|
737
|
-
shows zero clusters (hard limits: 10 fix rounds or 45 minutes), ending with a
|
|
738
|
-
`PITSTOP_REPORT.md`.
|
|
739
|
-
|
|
740
|
-
### Ledger mode (opt-in)
|
|
741
|
-
|
|
742
|
-
`pitstop scan --ledger` boots your app with **every outbound HTTP call rerouted to a mock
|
|
743
|
-
gateway**, then replays the three classic payment bugs: duplicate webhook, concurrent
|
|
744
|
-
double-submit, delayed retry. If the mock gateway's own receipt log shows more than one
|
|
745
|
-
charge per idempotency key, that's a **proven double-charge** — not a guess. The shipped
|
|
746
|
-
If the sandbox can't intercept some traffic,
|
|
747
|
-
the run aborts (`exit 77`); nothing ever reaches a real gateway.
|
|
748
|
-
|
|
749
|
-
**Which stacks are covered?** Node/JS apps run under the nock preload, which intercepts
|
|
750
|
-
every outbound call in-process. Go, Python, Rust and .NET apps run under a recording
|
|
751
|
-
`HTTP_PROXY` sandbox that answers the known payment-gateway hosts with mocked receipts
|
|
752
|
-
and 502s everything else. Java and Dart are refused (their HTTP clients don't honor
|
|
753
|
-
`HTTP_PROXY`, so interception could not be guaranteed). HTTPS stays blocked (502): without
|
|
754
|
-
a trusted CA the proxy cannot terminate a CONNECT tunnel, so an HTTPS double-charge is
|
|
755
|
-
reported as *indicated*, never *proven*. Native binaries and raw sockets bypass the proxy
|
|
756
|
-
and are not observed. Set `PITSTOP_START` to override start-command guessing for
|
|
757
|
-
non-Node repos.
|
|
758
|
-
|
|
759
|
-
### Integrity gate
|
|
760
|
-
|
|
761
|
-
Every `pitstop verify` also diffs your change against HEAD and checks for the classic
|
|
762
|
-
agent-cheat moves: deleted or loosened tests, tests focused to hide failures
|
|
763
|
-
(`fit`/`test.only`), swallowed exceptions, suppression comments, hardcoded-to-pass values,
|
|
764
|
-
a mocked module-under-test, a forced `exit(0)` in app code, or an assertion's expected
|
|
765
|
-
value edited to match the buggy output. A caught cheat looks like this: change
|
|
766
|
-
`assert.equal(round2(8.075), 8.08)` to expect `8.07` with nothing else in the diff →
|
|
767
|
-
`CONFIRMED_CHEAT`, the change is blocked, and a human reviews it (verified against
|
|
768
|
-
`fixtures/assertion-literal-tamper/`). An honest app-side fix sails through `CLEAN`.
|
|
769
|
-
|
|
770
|
-
## Architecture
|
|
771
|
-
|
|
772
|
-
OpenPitStop is two pieces that never mix: a **CLI that measures**, and **your host agent that
|
|
773
|
-
reasons and edits**. The CLI produces the scan/verify numbers and the gate verdicts; the
|
|
774
|
-
model in whichever tool you're using reads them, decides what to change, and does the
|
|
775
|
-
editing through the `/pitstop` prompt template. This is deliberately *not* one monolithic
|
|
776
|
-
agent — the numbers can't be talked into looking better, and the agent can't silently
|
|
777
|
-
cheat its own referee. That separation is the product.
|
|
778
|
-
|
|
779
|
-
## Known limitations
|
|
780
|
-
|
|
781
|
-
- **Windows** is a first-class, CI-verified platform (build + smoke on `ubuntu-latest`
|
|
782
|
-
and `windows-latest` every push). `watch`, `pen`, `pen --fix` and `scan --ledger` were
|
|
783
|
-
each run end-to-end on a real Windows host against sample apps, with a live watch
|
|
784
|
-
delta, PROVEN ledger double-charges (sealed evidence) and honest pen verdicts
|
|
785
|
-
(including the honest "0 patches" case) all verified. The only open caveat is breadth,
|
|
786
|
-
not correctness: not every exotic repo shape has been hand-exercised on Windows yet.
|
|
787
|
-
- **Codex App / VS Code extension** isn't supported and won't be until OpenAI ships custom
|
|
788
|
-
slash commands; use Codex CLI for `/pitstop`.
|
|
789
|
-
- **Graceful degradation:** duplication (`jscpd`), secret scanning (`gitleaks`),
|
|
790
|
-
dependency CVEs (`pip-audit`, `osv-scanner`), and accessibility runtime checks
|
|
791
|
-
(`pa11y`/`axe`) run only when that tool is installed locally. The scan reports
|
|
792
|
-
`skipped` for those categories and works fine without them. **Semgrep is an opt-in
|
|
793
|
-
deeper engine** (see [Security fixes](#security-fixes)): it only runs when you set
|
|
794
|
-
`PITSTOP_SEMGREP_CONFIG`, so it never runs just because the binary happens to be on
|
|
795
|
-
PATH — no surprise network calls or slow scans.
|
|
796
|
-
- Requires **Node.js 22+** (the CLI depends on execa 10, which uses ES2024 `Set.union`).
|
|
797
|
-
- **Multi-stack honesty:** test runs (JS, Python, Go, Rust, Flutter, .NET, Java via Maven
|
|
798
|
-
or Gradle), dependency CVEs, and the `pen`/`ledger` sandboxes are real for Node/JS and
|
|
799
|
-
best-effort elsewhere. Go/Rust/Python/.NET apps run under the `HTTP_PROXY` recording
|
|
800
|
-
sandbox (see Ledger mode); Java and Dart are refused for ledger, and proxy-mode results
|
|
801
|
-
are labelled `indicated`, never `proven`, when the proxy cannot observe the traffic. The
|
|
802
|
-
native test runners parse each toolchain's real output (`go test -json`, `cargo test
|
|
803
|
-
--format json`, `flutter test --machine`, dotnet/maven/gradle summaries) and report
|
|
804
|
-
`skipped` when a toolchain isn't on PATH.
|
|
805
|
-
- **Pen-test honesty:** `pitstop pen` reports each finding with a **runtime-proof
|
|
806
|
-
verdict**: **proven** (the live dynamic attack confirmed it under the sandbox),
|
|
807
|
-
**indicated** (static rule fired but the dynamic phase couldn't confirm), **unproven**
|
|
808
|
-
(the dynamic phase ran and found no evidence for that rule), or **not-tested** (dynamic
|
|
809
|
-
phase aborted). Proven findings are real; everything else is a hypothesis until you
|
|
810
|
-
replay the attack yourself. The sandbox records outbound connections and spawned
|
|
811
|
-
processes instead of blocking them (so real bytes never leave your machine for canaries,
|
|
812
|
-
but a compromised app could still run commands locally); raw socket APIs are blocked
|
|
813
|
-
outright. `pen --fix` writes deterministic patches **only** for findings fixable by pure
|
|
814
|
-
insertion (e.g. missing `helmet()`, `x-powered-by` leaks) — anything else gets a failing
|
|
815
|
-
repro test and fix guidance, which is your contract for the fix.
|
|
816
|
-
- **`drive` verdicts** for runtime pen findings come from the repro test (FAIL first, PASS
|
|
817
|
-
after the fix), not from the static score — the static gate has nothing to say about a
|
|
818
|
-
runtime-only finding.
|
|
819
|
-
- **Baseline-aware verification honesty:** without an explicit `expectedFailure` predicate,
|
|
820
|
-
a non-zero baseline exit cannot prove the failure was the *intended* bug rather than a
|
|
821
|
-
broken environment — such results are downgraded to `UNPROVEN`, never `VERIFIED`. The
|
|
822
|
-
verification identity covers only the files you declare (`--test-file`/`--config`).
|
|
823
|
-
See [docs/baseline-verify.md](docs/baseline-verify.md).
|
|
824
|
-
- **State verification is not semantic verification.** A content hash proves content
|
|
825
|
-
changed, not that the change is correct or complete; untracked files without a snapshot
|
|
826
|
-
have no before-state and are reported `UNPROVEN` rather than guessed. Files over 8 MB are
|
|
827
|
-
recorded but not hashed. See [docs/state-verify.md](docs/state-verify.md).
|
|
828
|
-
- **Verifier health is per fault class.** Passing one known-bad case proves the verifier can
|
|
829
|
-
fail, not that it covers every regression; choosing a meaningful known-bad state is the
|
|
830
|
-
caller's responsibility, and the evidence records exactly what was seeded.
|
|
831
|
-
See [docs/verifier-check.md](docs/verifier-check.md).
|
|
832
|
-
- **A holdout is a sample, not a guarantee.** It proves the candidate satisfies checks it
|
|
833
|
-
could not see or modify — not correctness in general. Suites must live outside the
|
|
834
|
-
agent's modifiable workspace (a hidden file inside the repo is not a holdout), and a
|
|
835
|
-
suite that cannot fail on a known-bad baseline is reported `HOLDOUT_UNPROVEN` rather than
|
|
836
|
-
trusted. See [docs/holdout-verify.md](docs/holdout-verify.md).
|
|
837
|
-
- **Acceptance is only as strong as its contract.** `acceptance-verify` proves the
|
|
838
|
-
candidate satisfies the contract's *observable* criteria — not that the contract captures
|
|
839
|
-
the whole requirement, and not unobservable qualities (UX, load, security beyond the
|
|
840
|
-
asserted properties). It is deterministic by design, never an LLM judge; a contract that
|
|
841
|
-
passes on the baseline is reported `UNPROVEN` rather than trusted.
|
|
842
|
-
See [docs/acceptance-verify.md](docs/acceptance-verify.md).
|
|
843
|
-
- **Regression detection vs flaky tests:** with the default single run, a flaky candidate
|
|
844
|
-
check is indistinguishable from a regression and IS reported as one; `--runs <n>` detects
|
|
845
|
-
candidate-side inconsistency (`UNPROVEN`), but the baseline is a single run, so a flaky
|
|
846
|
-
baseline check can still mislabel a fix or hide a regression. Per-test granularity depends
|
|
847
|
-
on the runner's output; unparseable output degrades to one suite-level check.
|
|
848
|
-
See [docs/regression-check.md](docs/regression-check.md).
|
|
849
|
-
|
|
850
|
-
## Privacy
|
|
851
|
-
|
|
852
|
-
**Zero telemetry, zero SaaS, zero accounts — nothing leaves your machine unless you ask
|
|
853
|
-
it to.** OpenPitStop is a local CLI with no server and no phone-home: scans, gates and
|
|
854
|
-
pen tests run entirely on your machine, and the only network calls in the entire codebase
|
|
855
|
-
are the dependency audits you can see and opt out of (plus the `npx` download you
|
|
856
|
-
initiated). The full, auditable list — every connection, every cache, every file stored —
|
|
857
|
-
is in [PRIVACY.md](PRIVACY.md). The honesty brand is the product; that statement is the
|
|
858
|
-
receipt.
|
|
859
|
-
|
|
860
|
-
## Contributing
|
|
861
|
-
|
|
862
|
-
OpenPitStop is built to be extended — adding a whole new analyzer is a small, well-scoped
|
|
863
|
-
change. See [CONTRIBUTING.md](CONTRIBUTING.md) for the analyzer interface, conventions, and
|
|
864
|
-
how to open a PR. For the launch notes and the "why", read [LAUNCH.md](LAUNCH.md).
|
|
865
|
-
|
|
866
|
-
## License
|
|
867
|
-
|
|
868
|
-
[MIT](LICENSE)
|
|
869
|
-
|
|
870
|
-
## Support the project
|
|
871
|
-
|
|
872
|
-
If OpenPitStop saved you from shipping a bug your agent swore was fixed, the best
|
|
873
|
-
support is a star and a real repo:
|
|
874
|
-
|
|
875
|
-
- **Star** it: https://github.com/Krish-1507/OpenPitStop
|
|
876
|
-
- **Report issues or ideas**: https://github.com/Krish-1507/OpenPitStop/issues
|
|
877
|
-
- **Contribute** an analyzer (small, well-scoped): see [CONTRIBUTING.md](CONTRIBUTING.md)
|
|
878
|
-
|
|
879
|
-
No donation, no paywall, no telemetry.
|
|
880
|
-
|
|
881
|
-
---
|
|
882
|
-
|
|
883
|
-
<p align="center">
|
|
884
|
-
<img src="docs/media/pitstop-icon.png" alt="OpenPitStop icon" width="64">
|
|
885
|
-
</p>
|
|
886
|
-
|
|
887
|
-
<p align="center">Built by <b>Krish J</b> — if it can referee itself, it can referee your agent.</p>
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="docs/media/pitstop-logo.png" alt="OpenPitStop" width="380">
|
|
3
|
+
</p>
|
|
4
|
+
|
|
5
|
+
# OpenPitStop
|
|
6
|
+
|
|
7
|
+
**OpenPitStop gives your AI coding CLI a repo-aware engineering workflow: understand, plan, check architecture, test security, and verify.**
|
|
8
|
+
**Use it to turn fast code generation into reviewable changes, with regression evidence, explicit boundaries, and local checks that use zero model credits.**
|
|
9
|
+
|
|
10
|
+
[](https://www.npmjs.com/package/openpitstop)
|
|
11
|
+
[](https://github.com/Krish-1507/OpenPitStop/actions/workflows/ci.yml)
|
|
12
|
+
[](LICENSE)
|
|
13
|
+
|
|
14
|
+
Working code can still be wrong for the system. A change can pass a test while bypassing
|
|
15
|
+
authentication, crossing a module boundary, weakening an assertion, or breaking a real user
|
|
16
|
+
flow. PitStop makes those questions part of a repeatable workflow, with inspectable evidence
|
|
17
|
+
and explicit `UNPROVEN` results when the checks cannot establish an answer.
|
|
18
|
+
|
|
19
|
+
**Local CLI · Bring your own coding agent · No PitStop account · No telemetry · MIT**
|
|
20
|
+
|
|
21
|
+
## Start with one command:
|
|
22
|
+
|
|
23
|
+
Requires **Node.js 22+** and npm. Git is needed for change comparison and isolated worktrees.
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
npm install -g openpitstop
|
|
27
|
+
pitstop ask "understand this repo"
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Or get a quick first scan without a global install:
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
npx openpitstop try .
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
`try` runs the lighter checks and saves a baseline. It does not run the full test/build
|
|
37
|
+
stack; duration depends on the repository and installed tools.
|
|
38
|
+
|
|
39
|
+
Install the slash command into your coding CLI, then speak naturally:
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
pitstop install
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
```text
|
|
46
|
+
/pitstop find vulnerabilities in this repo
|
|
47
|
+
/pitstop find and fix vulnerabilities in this repo
|
|
48
|
+
/pitstop review my changes
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
PitStop executes the matching workflow internally. The host CLI handles the slash
|
|
52
|
+
command; the terminal equivalent is `pitstop ask "find vulnerabilities in this repo"`.
|
|
53
|
+
Static vulnerability searches need no model call and do not boot the app.
|
|
54
|
+
|
|
55
|
+
Then say what you need:
|
|
56
|
+
|
|
57
|
+
| Your request | PitStop runs internally |
|
|
58
|
+
|---|---|
|
|
59
|
+
| `pitstop ask "check the security of this app"` | Static security review; no app boot |
|
|
60
|
+
| `pitstop ask "check tests types lint and build"` | Discovered verification layers, with failure diagnosis |
|
|
61
|
+
| `pitstop ask "review my changes"` | Repo understanding → architecture → verification stack → evidence gate |
|
|
62
|
+
| `pitstop ask "did my agent cheat"` | Test and verification integrity checks |
|
|
63
|
+
| `pitstop ask "can I ship this"` | Strict gate; `UNPROVEN` returns a failure exit code |
|
|
64
|
+
| `pitstop ask "show my budget"` | Recorded scan/test/build activity |
|
|
65
|
+
| `pitstop ask "what should I do next"` | Next action and remaining work |
|
|
66
|
+
|
|
67
|
+
The router runs locally and makes **no LLM call**. It recognizes common requests; it is
|
|
68
|
+
not a general language model. Unknown requests and unsupported constraints run nothing.
|
|
69
|
+
Use `--dry-run` to preview, `--repo ./app` to select a repository, or `--json` for a
|
|
70
|
+
machine-readable routing preview. The executed command's exit code is preserved.
|
|
71
|
+
|
|
72
|
+
Actions that write source, launch a paid agent, install integrations, or actively attack
|
|
73
|
+
an app are previewed until you add `--execute`:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
pitstop ask "make this safe" --dry-run
|
|
77
|
+
pitstop ask "make this safe" --execute
|
|
78
|
+
pitstop ask "attack my app" --execute
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
`make this safe` runs the supported deterministic fix workflow. It does not promise to
|
|
82
|
+
repair every issue. `attack my app` runs the live security checks without silently applying
|
|
83
|
+
fixes. Test and build checks execute your repository's scripts; use trusted repositories.
|
|
84
|
+
|
|
85
|
+
## The engineering loop
|
|
86
|
+
|
|
87
|
+
```mermaid
|
|
88
|
+
flowchart LR
|
|
89
|
+
A[Understand the repo] --> B[Plan and boundaries]
|
|
90
|
+
B --> C[Make a scoped change]
|
|
91
|
+
C --> D[Tests, types, lint, build]
|
|
92
|
+
D --> E[Architecture and security]
|
|
93
|
+
E --> F[Regression and acceptance evidence]
|
|
94
|
+
F --> G[Gate and next action]
|
|
95
|
+
G -->|More work needed| B
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
| Engineering question | Evidence PitStop provides |
|
|
99
|
+
|---|---|
|
|
100
|
+
| What system am I changing? | Languages, frameworks, entry points, module map, verification commands, CI and CODEOWNERS |
|
|
101
|
+
| What may this change touch? | A sealed plan, declared import boundaries, protected/forbidden paths and scope checks |
|
|
102
|
+
| Why did verification fail? | Per-layer results and diagnoses: assertion, type error, missing dependency, lint, environment or timeout |
|
|
103
|
+
| Does the change fit this repo? | Declared architecture rules, ownership routing and suspicious shortcuts in the diff |
|
|
104
|
+
| Did the fix address the actual bug? | The same verification against a known-bad baseline and the candidate |
|
|
105
|
+
| Did it meet the requirement? | An acceptance contract with command, HTTP and file criteria |
|
|
106
|
+
| Did it break existing behavior? | Check-level baseline/candidate comparison and optional repeated runs |
|
|
107
|
+
| Why trust the verdict? | Inspectable artifacts, content digests, evidence references and explicit missing checks |
|
|
108
|
+
|
|
109
|
+
Repo awareness is structural analysis and declared rules. Semantic design judgment still
|
|
110
|
+
belongs to the developer and host agent. A plan or gate checks behavior within its scope;
|
|
111
|
+
it does not prevent a separately running agent from editing files.
|
|
112
|
+
|
|
113
|
+
<p align="center">
|
|
114
|
+
<img src="docs/media/flow.gif" alt="OpenPitStop verification pipeline" width="880">
|
|
115
|
+
</p>
|
|
116
|
+
|
|
117
|
+
## Feature tour
|
|
118
|
+
|
|
119
|
+
### Understand, plan and protect the architecture
|
|
120
|
+
|
|
121
|
+
`understand` discovers the repository's structure and tools. `plan` records the goal,
|
|
122
|
+
steps, expected paths and verification commands before source patching.
|
|
123
|
+
`architecture-check --against-plan` checks scope, declared import boundaries, protected
|
|
124
|
+
paths, forbidden paths, CODEOWNERS routing and shortcuts in the diff.
|
|
125
|
+
|
|
126
|
+
Configure boundaries in `openpitstop.architecture.json` or `.pitstop/architecture.json`.
|
|
127
|
+
Protected-path approval is recorded through an explicit option; it is not an authenticated
|
|
128
|
+
human identity. [Configuration and examples](docs/repo-discipline.md).
|
|
129
|
+
|
|
130
|
+
### Scan and score
|
|
131
|
+
|
|
132
|
+
`scan` combines dependency graphs, security, duplication, tests, performance, accessibility,
|
|
133
|
+
reliability and developer-experience checks. Findings include locations and fix guidance;
|
|
134
|
+
related findings are grouped. Optional missing tools are reported as skipped.
|
|
135
|
+
|
|
136
|
+
The **0–100 score** summarizes the categories that ran. It is not a probability of
|
|
137
|
+
correctness or security: always read the coverage and skipped categories alongside it.
|
|
138
|
+
|
|
139
|
+
<p align="center">
|
|
140
|
+
<img src="docs/media/scan.gif" alt="Repository score and findings" width="880">
|
|
141
|
+
</p>
|
|
142
|
+
|
|
143
|
+
### Test pyramid and failure diagnosis
|
|
144
|
+
|
|
145
|
+
`test` discovers unit, integration and E2E layers separately and reports failing test names
|
|
146
|
+
when the runner exposes them. `verify-stack` adds typecheck, lint and build with targeted
|
|
147
|
+
failure diagnosis. Missing layers are visible, not fabricated passes. A failing layer
|
|
148
|
+
returns a nonzero exit. [Verification stack](docs/repo-discipline.md).
|
|
149
|
+
|
|
150
|
+
### Security testing and fixes
|
|
151
|
+
|
|
152
|
+
`pen --static` reviews supported patterns without booting the app. `pen` also starts the
|
|
153
|
+
local application and sends supported attack probes, recording requests, responses and
|
|
154
|
+
intercepted outbound activity. Findings distinguish static indications from runtime evidence.
|
|
155
|
+
The [security matrix](docs/security.md) describes the built-in checks; optional Semgrep
|
|
156
|
+
adds a separate SAST engine.
|
|
157
|
+
|
|
158
|
+
`pen --fix` writes repro tests for supported findings and patch files for a small set of
|
|
159
|
+
deterministic fixes, including Express header hardening. It does not apply source patches.
|
|
160
|
+
`fix` runs understand → scan → generate → plan → apply → recheck security → stack and gate.
|
|
161
|
+
Automatic source patching requires a clean git tree, creates a `pitstop/fix-*` branch,
|
|
162
|
+
respects protected paths and the plan, and never commits or pushes. `--no-apply` keeps
|
|
163
|
+
source patches for review but still writes repro tests and reports. Unsupported fixes
|
|
164
|
+
remain explicit work for you or your agent.
|
|
165
|
+
|
|
166
|
+
**Execution boundary:** live pen, payment attacks and their generated repros run in a
|
|
167
|
+
disposable Linux Docker container: no external network, read-only root and input mounts,
|
|
168
|
+
non-root user, no capabilities, and bounded CPU/memory/processes/time. Only a sanitized
|
|
169
|
+
copy is exposed; `.git`, `.env*`, common credential files and host environment secrets
|
|
170
|
+
are excluded. There is no automatic host-execution fallback.
|
|
171
|
+
|
|
172
|
+
Install/start Docker once, then prepare the default Node image:
|
|
173
|
+
|
|
174
|
+
```bash
|
|
175
|
+
docker pull node:22-bookworm-slim
|
|
176
|
+
pitstop ask "attack my app" --execute
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
Missing Docker, missing dependencies or an unbootable app produces an aborted check,
|
|
180
|
+
never a clean security verdict. Non-Node/native dependencies need a prepared Linux image
|
|
181
|
+
selected with `PITSTOP_SANDBOX_IMAGE`; use a trusted image with Node and your runtime.
|
|
182
|
+
Dependencies must already work on Linux; the isolated run cannot download them.
|
|
183
|
+
See [execution and spending controls](docs/release-controls.md).
|
|
184
|
+
|
|
185
|
+
<p align="center">
|
|
186
|
+
<img src="docs/media/pen.gif" alt="Live security probes and evidence" width="880">
|
|
187
|
+
</p>
|
|
188
|
+
|
|
189
|
+
### Verification beyond a green suite
|
|
190
|
+
|
|
191
|
+
| Layer | What it checks | Guide |
|
|
192
|
+
|---|---|---|
|
|
193
|
+
| `baseline-verify` | Verification fails on a known-bad commit and passes on the candidate; verification identity stays consistent. An intended-failure predicate distinguishes the bug from environment failure. | [Baseline](docs/baseline-verify.md) |
|
|
194
|
+
| `state-verify` | Structured file-change claims against actual filesystem and git state | [State](docs/state-verify.md) |
|
|
195
|
+
| `verifier-check` | A known-good case passes and an explicit known-bad case fails | [Falsifiability](docs/verifier-check.md) |
|
|
196
|
+
| `holdout-verify` | An externally maintained suite checks the committed candidate with redacted results | [Holdout](docs/holdout-verify.md) |
|
|
197
|
+
| `acceptance-verify` | Observable requirements defined in a pinned contract | [Acceptance](docs/acceptance-verify.md) |
|
|
198
|
+
| `regression-check` | Previously passing checks versus the candidate; new, missing and flaky checks remain distinguishable | [Regression](docs/regression-check.md) |
|
|
199
|
+
| `integrity` | Deleted/focused/weakened tests, hardcoded passes, suppression creep and other suspicious diff patterns | [Real examples](docs/caught-in-the-wild.md) |
|
|
200
|
+
|
|
201
|
+
These layers need meaningful contracts, baselines or suites. `flow` runs configured
|
|
202
|
+
stages and reports others as skipped. Git worktrees isolate checkouts, not process
|
|
203
|
+
permissions. A holdout is hidden only when the host's filesystem permissions actually
|
|
204
|
+
keep it outside the editing agent's reach.
|
|
205
|
+
|
|
206
|
+
### Gate and evidence chain
|
|
207
|
+
|
|
208
|
+
`gate` combines live measurements and saved verification layers. Its decision order is
|
|
209
|
+
`CHEAT → BLOCKED → FAILED → UNPROVEN → VERIFIED`. `explain` shows the evidence chain;
|
|
210
|
+
`honesty` exposes limitations and supporting measurements.
|
|
211
|
+
|
|
212
|
+
```bash
|
|
213
|
+
pitstop gate --strict --require stack,architecture,acceptance,regression
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
In strict mode only `VERIFIED` exits `0`; insufficient or failed evidence exits `1`,
|
|
217
|
+
and detected integrity manipulation exits `2`. Legacy `gate` without `--strict` can
|
|
218
|
+
exit `0` with `UNPROVEN` for compatibility. Required layers never silently disappear.
|
|
219
|
+
Other commands have their own exit contracts; inspect `--help` and the linked guides.
|
|
220
|
+
|
|
221
|
+
Artifacts use canonical SHA-256 content digests. They detect edits that do not update
|
|
222
|
+
the digest; they are **not cryptographic signatures from a trusted external signer**.
|
|
223
|
+
Someone with write access can recompute them. Keep authoritative verification and
|
|
224
|
+
holdouts in trusted CI or a separate permission boundary. Every passing historical gate
|
|
225
|
+
layer must match the current source snapshot, Git state, policy and PitStop engine.
|
|
226
|
+
Missing/old bindings or changed inputs become `UNPROVEN`; commit-based checks also
|
|
227
|
+
require that exact clean candidate. Contract and holdout changes invalidate their results.
|
|
228
|
+
[Evidence semantics](docs/explain.md).
|
|
229
|
+
|
|
230
|
+
### Payment proofs, drift and repros
|
|
231
|
+
|
|
232
|
+
`scan --ledger` replays duplicate webhooks, concurrent submissions and delayed retries
|
|
233
|
+
through a mock payment gateway. Duplicate charge receipts provide evidence for the
|
|
234
|
+
tested scenario. Proxy-only stacks have weaker observation coverage, while Docker still enforces the
|
|
235
|
+
network boundary. This does not prove all payment behavior.
|
|
236
|
+
|
|
237
|
+
`pen` compares comparable runs to identify new, disappeared and escalated findings.
|
|
238
|
+
A disappeared finding is a signal to investigate; preserve and replay the regression
|
|
239
|
+
test to establish that the fix holds. `repro <id>` generates and runs supported repros;
|
|
240
|
+
unsupported findings need a developer-written check. [Demo](docs/demo.md).
|
|
241
|
+
|
|
242
|
+
### Keep the loop affordable
|
|
243
|
+
|
|
244
|
+
- **No model required** for intent routing, structural discovery, built-in scans and
|
|
245
|
+
verification. Your chosen agent uses its own provider credits when it reasons or edits.
|
|
246
|
+
- **Content-aware reuse:** `scan --reuse` checks the baseline digest, file contents and
|
|
247
|
+
paths, git HEAD, scan options and selected engine/environment settings. Deleted files,
|
|
248
|
+
backdated edits and dot-directory changes invalidate reuse. Quick scans cannot replace
|
|
249
|
+
full scans. Snapshots are checked before and after measurement.
|
|
250
|
+
- **Bounded freshness:** cached scans expire after one hour; live ledger checks are not
|
|
251
|
+
reused. Symlinks, unreadable inputs and snapshot size limits force a fresh scan.
|
|
252
|
+
Generated/dependency directories are excluded. Reuse is an iteration optimization,
|
|
253
|
+
not release evidence: run fresh checks after changing installed tools, dependencies,
|
|
254
|
+
services or environment variables outside the recorded inputs.
|
|
255
|
+
- **One shared agent budget:** `drive` defaults to three total launches, ten minutes
|
|
256
|
+
and 48,000 total prompt characters across every finding. Change these with
|
|
257
|
+
`--max-agent-calls`, `--max-seconds` and `--max-prompt-chars`. Repeated identical
|
|
258
|
+
failures stop retries; concurrent/nested drive sessions are refused.
|
|
259
|
+
- **Optional dollar ceiling:** `drive --max-cost-usd 1` divides one allowance across
|
|
260
|
+
all launches using Claude print mode’s native budget flag. Providers without an
|
|
261
|
+
enforceable adapter refuse this request before an agent starts.
|
|
262
|
+
- **Honest accounting:** `budget` reports recorded test/build time and activity counts.
|
|
263
|
+
It cannot read your agent's token bill, prove cache savings or promise a dollar amount.
|
|
264
|
+
|
|
265
|
+
`memory` keeps decisions and rejected approaches; `ready-check` explains reuse eligibility;
|
|
266
|
+
`doctor` explains missing tools. `watch` rescans during editing, while `trends` and `digest`
|
|
267
|
+
show the repository's recorded progress.
|
|
268
|
+
|
|
269
|
+
## Install into your coding CLI
|
|
270
|
+
|
|
271
|
+
```bash
|
|
272
|
+
pitstop install
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
The installer writes agent instruction files. Check its displayed destinations; use
|
|
276
|
+
`--force`/`-y` deliberately when replacing existing files. `pitstop prompt` prints the
|
|
277
|
+
full workflow for inspection or manual installation. In a host that supports the
|
|
278
|
+
installed command, use `/pitstop` or `/pitstop check the security of this app`.
|
|
279
|
+
|
|
280
|
+
| Bundled target | Command location |
|
|
281
|
+
|---|---|
|
|
282
|
+
| Claude Code | `.claude/commands/pitstop.md`, plus a skill template |
|
|
283
|
+
| OpenCode | `.opencode/commands/pitstop.md` and configured user locations |
|
|
284
|
+
| Kilo | `.kilo/commands/pitstop.md` and legacy workflow locations |
|
|
285
|
+
| Antigravity | `.agent/workflows/pitstop.md` and legacy workflow locations |
|
|
286
|
+
| Gemini CLI | `.gemini/commands/pitstop.toml` |
|
|
287
|
+
| Codex CLI | `~/.codex/prompts/pitstop.md` |
|
|
288
|
+
| Cursor command files | `.cursor/commands/pitstop.md` |
|
|
289
|
+
| Other agent CLIs | Export `pitstop prompt` into their supported custom-command format |
|
|
290
|
+
|
|
291
|
+
Portable instructions can be used with FreeBuff, Grok Build, MUSE Code and other CLIs
|
|
292
|
+
that accept them; those integrations are not automatically validated by the presence of
|
|
293
|
+
a Markdown file. Host versions determine command invocation and argument syntax.
|
|
294
|
+
The shipped installer does not provide a dedicated Codex desktop-app or Codex VS Code
|
|
295
|
+
extension integration. [Installer targets](src/installer/targets.ts).
|
|
296
|
+
|
|
297
|
+
## Every command, by job
|
|
298
|
+
|
|
299
|
+
| Job | Commands |
|
|
300
|
+
|---|---|
|
|
301
|
+
| Start and navigate | `ask`, `next`, `try`, `demo`, bare `pitstop` onboarding |
|
|
302
|
+
| Understand and plan | `understand`, `plan`, `architecture-check`, `memory` |
|
|
303
|
+
| Measure and diagnose | `scan`, `test`, `verify-stack`, `doctor`, `ready-check` |
|
|
304
|
+
| Fix and inspect | `inspect`, `repro`, `drive`, `fix` |
|
|
305
|
+
| Security | `pen`, `pen --static`, `pen --fix`, `scan --ledger` |
|
|
306
|
+
| Verify | `verify`, `flow`, `gate`, `integrity`, `baseline-verify`, `state-verify`, `verifier-check`, `holdout-verify`, `acceptance-verify`, `regression-check` |
|
|
307
|
+
| Explain and share | `explain`, `honesty`, `report`, `share`, `trends`, `digest`, `budget` |
|
|
308
|
+
| Integrate | `install`, `prompt`, `watch`, `ci` |
|
|
309
|
+
|
|
310
|
+
Use `pitstop <command> --help` for exact options. `report --html` produces a self-contained
|
|
311
|
+
HTML report and a README badge; `share` creates a screenshot-ready card. `ci` produces
|
|
312
|
+
PR-ready reports, and `install --hooks` installs a pre-commit gate. See the
|
|
313
|
+
[GitHub Action guide](docs/github-action.md) and [repository workflow guide](docs/repo-discipline.md).
|
|
314
|
+
|
|
315
|
+
## Where OpenPitStop fits
|
|
316
|
+
|
|
317
|
+
OpenPitStop's focus is the engineering workflow around a change. Specialized tools
|
|
318
|
+
remain useful alongside it; the project does not claim universal detection superiority.
|
|
319
|
+
|
|
320
|
+
| Tool | Documented focus | Relationship to OpenPitStop |
|
|
321
|
+
|---|---|---|
|
|
322
|
+
| [Strix](https://github.com/usestrix/strix) | Agentic security investigation, exploit validation and remediation; also documents CI and continuous testing | A deeper security-testing option. PitStop concentrates on local checks, architecture rules, regression contracts and evidence gates across the change lifecycle. |
|
|
323
|
+
| [Semgrep](https://docs.semgrep.dev/getting-started/quickstart) | Rule-based static security analysis and broader security products | Optional SAST integration already exists through `PITSTOP_SEMGREP_CONFIG`. |
|
|
324
|
+
| [CodeQL](https://docs.github.com/en/code-security/concepts/code-scanning/codeql/codeql-code-scanning) | Queries over a code database for security analysis | Complementary analysis; no built-in CodeQL execution integration is claimed. |
|
|
325
|
+
| [ZAP](https://www.zaproxy.org/docs/automate/automation-framework/) | Web security automation, authenticated contexts and active/passive scanning | Broader web-testing infrastructure; a future adapter would complement the built-in probes. |
|
|
326
|
+
| [Aider](https://aider.chat/docs/repomap.html) | AI coding with repository maps and relevant context | A host editing agent; PitStop adds independently executed checks around its changes. |
|
|
327
|
+
|
|
328
|
+
Research checked **2026-09-25** against the linked primary sources. This is a capability
|
|
329
|
+
comparison, not a benchmark. [Detailed audit, research and next milestones](docs/engineering-review.md).
|
|
330
|
+
|
|
331
|
+
## Architecture and limits
|
|
332
|
+
|
|
333
|
+
The TypeScript CLI dispatches commands to analyzers, repository-understanding modules,
|
|
334
|
+
security probes and verification engines. Results live in `.pitstop/`; the report and
|
|
335
|
+
gate layers consume them. Templates instruct the host coding agent, and `drive` can invoke
|
|
336
|
+
that agent. `fix` is the separate, limited deterministic patch path.
|
|
337
|
+
|
|
338
|
+
The strongest built-in coverage is JS/TS and Node application workflows. Native test
|
|
339
|
+
runners and optional tools extend other stacks, but support varies by feature. Architecture
|
|
340
|
+
checks rely on declared rules and supported import patterns; they do not infer a complete
|
|
341
|
+
system design. Static findings can be false positives, and live probes can miss vulnerabilities.
|
|
342
|
+
Acceptance contracts and tests are only as strong as the behavior they assert.
|
|
343
|
+
|
|
344
|
+
Linux and Windows are included in CI. Passing these fixtures does not establish correctness
|
|
345
|
+
for every repository, framework or environment. No tool can promise to remove all engineering
|
|
346
|
+
problems or make every application secure.
|
|
347
|
+
|
|
348
|
+
## Privacy
|
|
349
|
+
|
|
350
|
+
OpenPitStop has no telemetry service or hosted account requirement. Dependency audits,
|
|
351
|
+
opt-in external tools, repository scripts and a host coding agent may access the network.
|
|
352
|
+
Reports can contain code locations and security evidence; review them before sharing.
|
|
353
|
+
See [PRIVACY.md](PRIVACY.md) for the connection and storage details.
|
|
354
|
+
|
|
355
|
+
## Contribute
|
|
356
|
+
|
|
357
|
+
The most useful contributions are reproducible bugs, meaningful verification fixtures,
|
|
358
|
+
framework adapters and measured false-positive reductions. Start with
|
|
359
|
+
[CONTRIBUTING.md](CONTRIBUTING.md) and the [engineering roadmap](docs/engineering-review.md).
|
|
360
|
+
|
|
361
|
+
If PitStop catches a real failure, share the minimal reproduction and its evidence.
|
|
362
|
+
[Star the project](https://github.com/Krish-1507/OpenPitStop) or
|
|
363
|
+
[open an issue](https://github.com/Krish-1507/OpenPitStop/issues).
|
|
364
|
+
|
|
365
|
+
[MIT License](LICENSE) · Built by **Krish J**
|