sniff-qa 0.5.2 → 0.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. package/.claude-plugin/plugin.json +2 -2
  2. package/CHANGELOG.md +60 -0
  3. package/README.md +219 -347
  4. package/SECURITY.md +3 -3
  5. package/dist/chunk-LJYT4MD5.js +1246 -0
  6. package/dist/chunk-LJYT4MD5.js.map +1 -0
  7. package/dist/cli/index.js +41 -12
  8. package/dist/cli/index.js.map +1 -1
  9. package/dist/crawl-666NLYDC.js +11 -0
  10. package/dist/crawl-666NLYDC.js.map +1 -0
  11. package/dist/crawl-command-3OB4IUBU.js +54 -0
  12. package/dist/crawl-command-3OB4IUBU.js.map +1 -0
  13. package/dist/{discover-ENPRVN5A.js → discover-Q363YJ5X.js} +2 -2
  14. package/dist/{domain-UQ34YE4B.js → domain-NNL2PPRK.js} +1 -1
  15. package/dist/index.d.ts +94 -1
  16. package/dist/index.js +9 -1
  17. package/dist/index.js.map +1 -1
  18. package/dist/mcp/server.js +76 -6
  19. package/dist/mcp/server.js.map +1 -1
  20. package/dist/{model-X5J2JR5U.js → model-2DTSAX7T.js} +1 -1
  21. package/dist/{quality-run-6SXCN5CD.js → quality-run-LLBZZI7X.js} +2 -2
  22. package/dist/{repo-analyzer-4OURSITY.js → repo-analyzer-WK5WP7NT.js} +1 -1
  23. package/dist/{report-OAHSB56I.js → report-IKRVVIW4.js} +2 -2
  24. package/dist/{unified-SKXQQX3Y.js → unified-67Z6YA64.js} +4 -4
  25. package/package.json +2 -2
  26. package/skills/sniff/SKILL.md +20 -25
  27. /package/dist/{discover-ENPRVN5A.js.map → discover-Q363YJ5X.js.map} +0 -0
  28. /package/dist/{domain-UQ34YE4B.js.map → domain-NNL2PPRK.js.map} +0 -0
  29. /package/dist/{model-X5J2JR5U.js.map → model-2DTSAX7T.js.map} +0 -0
  30. /package/dist/{quality-run-6SXCN5CD.js.map → quality-run-LLBZZI7X.js.map} +0 -0
  31. /package/dist/{repo-analyzer-4OURSITY.js.map → repo-analyzer-WK5WP7NT.js.map} +0 -0
  32. /package/dist/{report-OAHSB56I.js.map → report-IKRVVIW4.js.map} +0 -0
  33. /package/dist/{unified-SKXQQX3Y.js.map → unified-67Z6YA64.js.map} +0 -0
package/README.md CHANGED
@@ -9,503 +9,375 @@
9
9
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-Apache_2.0-ef4444?style=flat-square" alt="License"></a>
10
10
  <a href="https://github.com/Aboudjem/sniff/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/Aboudjem/sniff/ci.yml?style=flat-square&label=CI" alt="CI"></a>
11
11
  <a href="https://nodejs.org"><img src="https://img.shields.io/badge/node-%E2%89%A522-339933?style=flat-square&logo=node.js&logoColor=white" alt="Node"></a>
12
+ <a href="https://github.com/Aboudjem/10x"><img src="https://img.shields.io/badge/10x-marketplace-ef4444?style=flat-square" alt="10x marketplace"></a>
12
13
  <a href="https://github.com/Aboudjem/sniff/stargazers"><img src="https://img.shields.io/github/stars/Aboudjem/sniff?style=flat-square&color=ef4444" alt="Stars"></a>
13
14
  </p>
14
15
 
15
- <p align="center"><b>Scan your source code, your live site, or both. Finds bugs before your users do.</b></p>
16
+ <p align="center"><b>Point it at your running app. It walks your real user flows in a real browser and tells you what's actually broken — with proof.</b></p>
16
17
 
17
18
  <p align="center">
18
19
  <a href="#get-started">Get started</a> ·
19
20
  <a href="#what-it-finds">What it finds</a> ·
20
- <a href="#commands">Commands</a> ·
21
- <a href="#works-with-any-stack">Stacks</a> ·
22
- <a href="docs/VIDEO-PLAN.md">Demo videos</a>
21
+ <a href="#why-you-can-trust-it">Why trust it</a> ·
22
+ <a href="#how-it-works">How it works</a> ·
23
+ <a href="#demo">Demo</a> ·
24
+ <a href="#faq">FAQ</a>
23
25
  </p>
24
26
 
25
27
  <picture>
26
28
  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/sniff-diagram.svg">
27
- <img alt="Sniff flow: source -> radar scanner -> findings" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/sniff-diagram.svg" width="100%">
29
+ <img alt="Sniff flow: your running app -> headless browser walk -> findings with proof" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/sniff-diagram.svg" width="100%">
28
30
  </picture>
29
31
 
30
32
  ---
31
33
 
32
- ## The 30-second pitch
34
+ ## What is this?
33
35
 
34
- > You ship a feature. A user finds the bug before you do. Sniff is the opposite of that.
36
+ Sniff is an autonomous QA scanner. You point it at a running web app and it **walks your app's real user flows in a real (headless) browser** — clicking buttons, filling forms, following links — and reports what's actually broken.
35
37
 
36
- Sniff is a tiny CLI that reads your source, opens your app in a headless browser, and hunts down bugs across **eight dimensions** — functional, visual, accessibility, performance, dead links, API endpoints, broken imports, and optional AI-assisted exploration. No API key required. No manual Playwright setup. No config.
38
+ It is **not** a linter and **not** a static scanner. It opens your pages, interacts with them like a user would, and watches what happens.
39
+
40
+ Every bug it reports comes with **proof**: the exact page, the ordered steps to reproduce it, a screenshot, and the console or network excerpt that caught it. A finding without reproduction proof is not a finding.
37
41
 
38
42
  ```bash
39
- npx sniff-qa
43
+ npx sniff-qa --url http://localhost:3000
40
44
  ```
41
45
 
42
- That's the whole setup. The rest of this README is detail.
43
-
44
- ### Why developers love it
46
+ One command — no config, no API key, no Playwright setup. Point it at your running app, or just run `npx sniff-qa` from your project and it auto-detects a dev server on common ports.
45
47
 
46
- - **One command.** `npx sniff-qa` auto-detects your framework, your dev server, your test scenarios. If `npm run dev` is running, sniff finds it.
47
- - **Zero API keys by default.** Deterministic local QA is the default. Codex, Claude, Gemini, Ollama, Anthropic, and OpenAI providers are optional enhancements.
48
- - **Eight checks, every time.** Accessibility + visual regression + performance + dead links + API contracts + source scanning + broken imports + browser runtime hooks — out of the box.
49
- - **Talks to your editor.** Ships as an MCP server. Just say *"scan this project"* in Claude / Cursor / Copilot / Continue / Windsurf / Codex / Gemini.
50
- - **CI-ready.** `sniff --ci` emits JUnit, tracks flakes, and fails on severity thresholds you set. `sniff ci` generates the GitHub Actions workflow.
51
- - **Actually explains the fix.** Every finding cites the rule, the file, the line — and `/sniff-fix` generates the patch.
48
+ > Sniff walks a **running app**. If no dev server is running, it falls back to a source-code scan and tells you exactly how to start the real walk. See [Get started](#get-started).
52
49
 
53
50
  ---
54
51
 
55
52
  ## Get started
56
53
 
57
- ### Use from the terminal
58
-
59
- ```bash
60
- cd ~/projects/my-app # go to your project
61
- npx sniff-qa # that's it
62
- ```
54
+ You need **Node.js 22+** and a web app you can run locally (or a URL).
63
55
 
64
- Sniff auto-detects everything: your framework, your dev server port, and your running app. If `npm run dev` is running, sniff finds it and runs browser checks too. No flags needed.
56
+ > **Naming:** the npm package is **`sniff-qa`** — use `npx sniff-qa` or `npm install -D sniff-qa`. Once installed, the binary is **`sniff`** (the `sniff-qa` binary works too). Don't run `npx sniff` — that's a different package.
65
57
 
66
- You can also be explicit:
58
+ ### 1. Start your app
67
59
 
68
60
  ```bash
69
- npx sniff-qa --url http://localhost:3000 # specific local URL
70
- npx sniff-qa --url https://myapp.com # production URL
61
+ npm run dev # or however you start your app
71
62
  ```
72
63
 
73
- > **No API keys. No manual Playwright install. No config files.** Everything works out of the box. Browser checks auto-install the configured Playwright browser projects on first CLI run.
64
+ ### 2. Walk it
74
65
 
75
- ### Use from your AI editor
66
+ In another terminal:
76
67
 
77
- Sniff ships as an MCP server. Add it to your editor, then ask your AI to scan.
68
+ ```bash
69
+ npx sniff-qa --url http://localhost:3000 # point it at your running app
70
+ ```
78
71
 
79
- <details>
80
- <summary><b>Claude Code plugin marketplace</b> (one-command install)</summary>
72
+ That's the reliable one-liner. Sniff also **auto-detects** a dev server on common ports, so from your project folder you can often just run `npx sniff-qa` with no flags. If your app is on a non-standard port (or auto-detect misses it), pass `--url` — that always works. It also walks a deployed URL:
81
73
 
82
74
  ```bash
83
- claude plugin marketplace add Aboudjem/10x
84
- claude plugin install sniff@10x
75
+ npx sniff-qa --url https://staging.myapp.com
85
76
  ```
86
77
 
87
- This installs sniff as a Claude Code plugin from the [10x marketplace](https://github.com/Aboudjem/10x). Skills, commands, and the MCP server are wired up automatically.
88
- </details>
78
+ > **Found bugs? It exits non-zero — on purpose.** A walk that finds issues exits with code `1` so CI fails the build; it is **not** a crash (you'll see a `✓ Scan complete` line). Pass `--fail-on none` to always exit `0`.
89
79
 
90
- <details>
91
- <summary><b>Claude Code (MCP server only)</b></summary>
80
+ > **First run downloads a browser.** The first time Sniff opens a browser it downloads a Chromium build (~165 MB, one-time). You'll see the progress. You need internet access for that first run; after that it's cached.
81
+
82
+ ### 3. Read the report
83
+
84
+ Findings print to your terminal, grouped by severity, each with steps to reproduce. Want a shareable page?
92
85
 
93
86
  ```bash
94
- claude mcp add sniff-qa npx sniff-qa --mcp
87
+ npx sniff-qa --url http://localhost:3000 --report # runs a walk, then writes sniff-reports/sniff-report.html (self-contained)
95
88
  ```
96
- </details>
97
89
 
98
- <details>
99
- <summary><b>Cursor</b></summary>
90
+ **No app running?** Sniff doesn't fail silently. It runs a source-only scan and prints a clear next step — start your dev server or pass `--url` — so you can get to the real flow-walk. You can also run the source scan on purpose:
100
91
 
101
- Add to `~/.cursor/mcp.json`:
102
- ```json
103
- { "mcpServers": { "sniff-qa": { "type": "stdio", "command": "npx", "args": ["sniff-qa", "--mcp"] } } }
92
+ ```bash
93
+ npx sniff-qa scan # source-only scan, no browser
104
94
  ```
105
- </details>
106
95
 
107
- <details>
108
- <summary><b>VS Code + Copilot</b></summary>
96
+ Stuck? Run `npx sniff-qa doctor` to check your environment (Node, browser, dev server).
109
97
 
110
- Add to `.vscode/mcp.json`:
111
- ```json
112
- { "servers": { "sniff-qa": { "type": "stdio", "command": "npx", "args": ["-y", "sniff-qa", "--mcp"] } } }
113
- ```
114
- </details>
98
+ ---
115
99
 
116
- <details>
117
- <summary><b>Codex CLI</b></summary>
100
+ ## What it finds
118
101
 
119
- ```bash
120
- codex mcp add sniff-qa -- npx -y sniff-qa --mcp
121
- ```
122
- </details>
102
+ Sniff walks your app and looks for **12 classes of real bugs**:
103
+
104
+ | # | Class | Examples |
105
+ |:--|:------|:---------|
106
+ | 1 | **Broken pages / routes** | 4xx/5xx responses, blank renders, crash screens |
107
+ | 2 | **Broken links** | Dead internal and external links |
108
+ | 3 | **Console & network errors** | Uncaught exceptions and failed requests *during interaction* |
109
+ | 4 | **Empty & fake data** | Missing data, plus placeholders like `lorem ipsum`, `TODO`, `test@test.com` |
110
+ | 5 | **Broken forms** | Dead submit buttons, validation that never fires |
111
+ | 6 | **State loss** | Fill a form, hit back, and it's wiped |
112
+ | 7 | **Flow regressions / dead-ends** | A journey that can't be completed |
113
+ | 8 | **Bad loading & error states** | Infinite spinners, missing error states |
114
+ | 9 | **Broken async outcomes** | Submitted but no success feedback (flagged "needs out-of-band verification") |
115
+ | 10 | **Responsive issues** | Overflow and tiny tap targets (a 375px mobile pass) |
116
+ | 11 | **Accessibility** | Missing alt text and labels, contrast — via [axe-core](https://github.com/dequelabs/axe-core) |
117
+ | 12 | **Unclear primary actions** | The main call-to-action is buried or ambiguous |
118
+
119
+ Each finding ships with:
120
+
121
+ - **Reproduction proof** — the exact route, the ordered steps, a screenshot, and the console/network excerpt.
122
+ - **A severity** — so you fix the right thing first.
123
+ - **A confidence** — `confirmed`, `likely`, or `uncertain`. Uncertain findings are hidden by default; add `--all` to see them.
124
+ - **A suggested fix.**
123
125
 
124
- <details>
125
- <summary><b>Windsurf</b></summary>
126
+ <picture>
127
+ <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/features-dark.svg">
128
+ <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/features-light.svg">
129
+ <img alt="The bug classes Sniff finds" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/features-light.svg" width="100%">
130
+ </picture>
126
131
 
127
- Add to `~/.codeium/windsurf/mcp_config.json`:
128
- ```json
129
- { "mcpServers": { "sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] } } }
130
- ```
131
- </details>
132
+ ---
132
133
 
133
- <details>
134
- <summary><b>Gemini CLI</b></summary>
134
+ ## Why you can trust it
135
135
 
136
- Add to `~/.gemini/mcp_config.json`:
137
- ```json
138
- { "mcpServers": { "sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] } } }
139
- ```
140
- </details>
136
+ Most scanners drown you in false positives until you stop reading them. Sniff is built the opposite way.
141
137
 
142
- <details>
143
- <summary><b>Continue.dev</b></summary>
138
+ We measured it on a fixture app planted with **21 bugs across all 12 classes**, plus a clean control page that should produce zero findings:
144
139
 
145
- Add to `.continue/mcpServers/sniff-qa.yaml`:
146
- ```yaml
147
- mcpServers:
148
- sniff-qa: { command: npx, args: [sniff-qa, --mcp], type: stdio }
149
- ```
150
- </details>
140
+ | | Old engine | New engine |
141
+ |:--|:--|:--|
142
+ | Bugs found | 9 / 21 (43%) | **21 / 21 (100%)** |
143
+ | Precision | ~13% | **100%** |
144
+ | False positives | 125 | **0** |
145
+ | Findings on the clean page | — | **0** |
146
+ | Flagship command | crashed | works |
151
147
 
152
- <details>
153
- <summary><b>OpenClaw</b></summary>
148
+ Those numbers are locked as a regression test. The full suite is **441 tests**.
154
149
 
155
- ```bash
156
- clawhub install sniff-qa
157
- ```
158
- </details>
150
+ **How it keeps false positives near zero:**
159
151
 
160
- Then ask: *"Scan this project for issues"* or *"Check accessibility on localhost:3000"* or *"Discover and run E2E tests"*
152
+ - A first-party **noise filter** drops the junk that isn't your bug — favicons, analytics, hot-module-reload chatter, expected auth redirects, engine aborts.
153
+ - Accessibility findings are backed by **axe-core**, which is zero-false-positive by design.
154
+ - **Uncertain findings are suppressed by default** (use `--all` to see them).
155
+ - A broken page is reported **once**, not re-flagged on every link that points to it.
161
156
 
162
- **MCP tools:** `sniff({ mode: "scan" | "run" | "discover" | "report" })` — unified entry point. Plus `sniff_install` when Playwright browser binaries are missing. MCP returns a structured `needsSetup` payload instead of doing a long install inside stdio; call `sniff_install` or run the shown `npx playwright install ...` command, then retry. The legacy per-mode tools (`sniff_scan`, `sniff_run`, `sniff_discover`, `sniff_report`) remain available but are deprecated in v0.5 and will be removed in v0.7.
157
+ If Sniff can't prove a bug, it doesn't claim one.
163
158
 
164
- ### Install as a dev dependency
159
+ ---
165
160
 
166
- ```bash
167
- npm install -D sniff-qa
168
- ```
161
+ ## How is it different?
169
162
 
170
- ```json
171
- {
172
- "scripts": {
173
- "qa": "sniff"
174
- }
175
- }
176
- ```
163
+ Linters read your source. End-to-end frameworks make *you* write the tests. Link checkers only check links. Sniff drives your real app and judges the result.
177
164
 
178
- That's all you need. Sniff auto-detects your dev server when it's running.
165
+ | | **Sniff** | linkinator | pa11y | Playwright codegen | QA-Wolf-style services |
166
+ |:--|:--|:--|:--|:--|:--|
167
+ | Walks real user flows in a browser | **Yes** | No | No | You script it | Yes |
168
+ | Zero setup, zero test-writing | **Yes** | Yes | Yes | No (you write tests) | No (onboarding) |
169
+ | Broken links | **Yes** | Yes | No | Manual | Manual |
170
+ | Accessibility (axe-core) | **Yes** | No | **Yes** | Manual | Some |
171
+ | Empty / placeholder / fake data | **Yes** | No | No | No | No |
172
+ | State-loss (back-button wipes a form) | **Yes** | No | No | Manual | Manual |
173
+ | One-shot reproduction proof per finding | **Yes** | No | Partial | No | Varies |
174
+ | Self-contained HTML report | **Yes** | No | Partial | No | Dashboard |
175
+ | Runs locally, no account, no API key | **Yes** | Yes | Yes | Yes | No (service) |
179
176
 
180
- Requires Node.js 22+. Playwright installs automatically on first CLI browser scan; MCP asks you to run `sniff_install` when setup is missing.
177
+ What Sniff uniquely does in one command: catch **empty/placeholder data**, **state-loss**, and **broken async outcomes** — and hand you a single proof report — with **no scripts to write and no service to sign up for**.
181
178
 
182
179
  ---
183
180
 
184
- ## Usage
181
+ ## Commands
185
182
 
186
- ```bash
187
- npx sniff-qa # scan source + auto-detect dev server
188
- npx sniff-qa --url https://myapp.com # scan source + specific URL
189
- npx sniff-qa ./path/to/project # scan a specific directory
190
- npx sniff-qa --ci # CI mode (JUnit output, no AI explorer)
191
- npx sniff-qa --url http://localhost:3000 --explore # opt-in AI exploration
192
- npx sniff-qa --verbose # show classifier top 3 + matched signals
193
- npx sniff-qa discover --dry-run # preview generated scenarios, no browser
194
- npx sniff-qa discover --force-app-type saas # skip classification, treat as SaaS
183
+ ```
184
+ sniff Walk your app (auto-detects the dev server). The default.
185
+ sniff --url <url> Walk a specific URL
186
+ sniff scan Source-only scan, no browser (placeholders, TODOs, dead links, etc.)
187
+ sniff report Show the results from the last run
188
+ sniff doctor Check your environment (Node, browser, config, dev server)
189
+ sniff ci Generate a GitHub Actions workflow
190
+ sniff fix Auto-fix safe issues (console.log, debugger, etc.)
191
+ sniff --help Show every command and flag
192
+ sniff --version Show the version
195
193
  ```
196
194
 
197
- Sniff auto-detects your dev server by reading `package.json` scripts plus `vite.config`, `nuxt.config`, `astro.config`, and `angular.json`, then probing the framework's default port *and* the next 20 after it (to catch auto-increment when the default is taken). If a port responds with framework markers (Next.js, Vite, Nuxt, Astro, Remix, SvelteKit, Angular), browser checks run automatically. Ghost-process ports like macOS AirPlay on `:5000` no longer collide.
195
+ ### Useful flags
196
+
197
+ | Flag | What it does |
198
+ |:-----|:-------------|
199
+ | `--url <url>` | Walk this URL instead of auto-detecting |
200
+ | `--report` | Write a self-contained HTML report to `sniff-reports/sniff-report.html` |
201
+ | `--all` | Also show low-confidence (`uncertain`) findings |
202
+ | `--max-pages <n>` | Cap how many pages to walk (default: 25) |
203
+ | `--no-mobile` | Skip the 375px responsive pass |
204
+ | `--headed` | Show the browser window while it walks |
205
+ | `--json` | Machine-readable JSON output |
206
+ | `--ci` | CI mode (stable output, non-interactive) |
207
+ | `--fail-on <sev>` | Exit non-zero on findings at or above this severity |
198
208
 
199
209
  ---
200
210
 
201
- ## What it finds
211
+ ## Demo
202
212
 
203
- <picture>
204
- <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/features-dark.svg">
205
- <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/features-light.svg">
206
- <img alt="Sniff checks: source bugs, dead links, API endpoints, broken imports, accessibility, visual regression, performance, optional AI explorer" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/features-light.svg" width="100%">
207
- </picture>
213
+ A real run against a buggy app — 21 real issues, zero false positives, every finding with severity, confidence, reproduction steps, and a fix:
208
214
 
209
- **Source checks** run on every scan. **Browser checks** run automatically when sniff detects a running dev server, or when you pass `--url`.
215
+ <img alt="Sniff walking a buggy app and reporting 21 real issues with zero false positives" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/demo.svg" width="100%">
210
216
 
211
217
  ---
212
218
 
213
- ## Example output
219
+ ## Use it from your AI editor
214
220
 
215
- <picture>
216
- <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/report-dark.svg">
217
- <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/report-light.svg">
218
- <img alt="Sniff terminal output" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/report-light.svg" width="100%">
219
- </picture>
221
+ Sniff ships as an MCP server too. Add it once, then just ask your assistant *"scan this project for bugs"* or *"walk my app."* If your app is running, Sniff auto-detects it — you don't have to pass a URL.
220
222
 
221
- ---
223
+ **One unified `sniff` tool, three modes:**
222
224
 
223
- ## Commands
225
+ - `walk` — **recommended.** Walks your running app's real flows (the flow-walk above).
226
+ - `scan` — source-only scan, no browser.
227
+ - `report` — show the last run's results.
224
228
 
225
- ### Slash commands (AI editors)
229
+ (`run` and `discover` are legacy modes kept for back-compat.)
226
230
 
227
- | Command | What it does |
228
- |:--------|:-------------|
229
- | `/sniff` | Scan the project. Auto-detects dev server, runs source + browser checks |
230
- | `/sniff-fix` | Scan and auto-fix safe issues (debugger, console.log, etc.) |
231
- | `/sniff-report` | Show results from the last scan |
231
+ <details>
232
+ <summary><b>Claude Code</b></summary>
232
233
 
233
- ### CLI
234
+ One-command plugin install from the [10x marketplace](https://github.com/Aboudjem/10x):
234
235
 
236
+ ```bash
237
+ claude plugin marketplace add Aboudjem/10x
238
+ claude plugin install sniff@10x
235
239
  ```
236
- sniff Scan source + auto-detect dev server
237
- sniff --url <url> Scan source + test specific URL
238
- sniff --url <url> --ci Full audit for CI pipelines
239
- sniff <path> Scan a specific directory
240
- sniff discover Autonomous E2E discovery (scenarios + edge cases)
241
- sniff discover --regenerate-only Write sniff-scenarios/ tree without running
242
- sniff fix Auto-fix safe issues (debugger, console.log)
243
- sniff fix --check Dry run: show what would be fixed
244
- sniff init Create sniff.config.ts
245
- sniff ci Generate GitHub Actions workflow
246
- sniff report Show last scan results
247
- sniff update-baselines Accept current visual baselines
248
- sniff doctor Check your environment (Node, Playwright, config, dev server)
249
- sniff --help Show all commands and flags
250
- sniff --version Show version
251
- ```
252
-
253
- <details>
254
- <summary><b>All flags</b></summary>
255
240
 
256
- | Flag | What it does |
257
- |:-----|:-------------|
258
- | `--url <url>` | Enable browser checks (accessibility, visual, performance, AI) |
259
- | `--ci` | CI mode: skip AI explorer, add JUnit output, track flaky tests |
260
- | `--explore` | Opt into AI-assisted exploration after browser checks |
261
- | `--no-explore` | Compatibility flag; exploration is off by default |
262
- | `--no-browser` | Source only even if `--url` is set |
263
- | `--max-steps <n>` | Limit AI explorer steps (default: 50) |
264
- | `--no-headless` | Show the browser window |
265
- | `--headed` | Alias for `--no-headless` |
266
- | `--format html,json,junit` | Choose report formats |
267
- | `--fail-on critical,high` | Severities that cause non-zero exit |
268
- | `--track-flakes` | Track test flakiness across runs |
269
- | `--json` | JSON output for scripts |
270
- | `--verbose` | Print classifier top-3 guesses and matched signals per dimension |
271
- | `--dry-run` | Discover: generate scenarios without launching browser or writing reports |
272
- | `--force-app-type <type>` | Discover: bypass the classifier (e.g. `saas`, `ecommerce`, `booking`) |
273
- | `--app-type <types>` | Discover: filter classifier guesses to these app types (comma-separated) |
241
+ Or add just the MCP server:
274
242
 
243
+ ```bash
244
+ claude mcp add sniff-qa npx -- -y sniff-qa --mcp
245
+ ```
275
246
  </details>
276
247
 
277
- ---
278
-
279
- ## Works with any stack
248
+ <details>
249
+ <summary><b>Cursor</b></summary>
280
250
 
281
- Sniff auto-detects your framework. No config needed.
251
+ Add to `~/.cursor/mcp.json`:
282
252
 
283
- <table>
284
- <tr>
285
- <td align="center" width="14%"><b>React</b><br/><sub>JSX / TSX</sub></td>
286
- <td align="center" width="14%"><b>Next.js</b><br/><sub>App + Pages</sub></td>
287
- <td align="center" width="14%"><b>Vue</b><br/><sub>SFC</sub></td>
288
- <td align="center" width="14%"><b>Svelte</b><br/><sub>Components</sub></td>
289
- <td align="center" width="14%"><b>Angular</b><br/><sub>Templates</sub></td>
290
- <td align="center" width="14%"><b>Express</b><br/><sub>Routes</sub></td>
291
- <td align="center" width="14%"><b>Vanilla</b><br/><sub>HTML / CSS</sub></td>
292
- </tr>
293
- </table>
253
+ ```json
254
+ { "mcpServers": { "sniff-qa": { "command": "npx", "args": ["-y", "sniff-qa", "--mcp"] } } }
255
+ ```
256
+ </details>
294
257
 
295
- API discovery also supports **Fastify**, **Hono**, **tRPC**, and **GraphQL**.
258
+ <details>
259
+ <summary><b>VS Code (Copilot)</b></summary>
296
260
 
297
- ---
261
+ Add to `.vscode/mcp.json`:
298
262
 
299
- ## Configuration
263
+ ```json
264
+ { "servers": { "sniff-qa": { "type": "stdio", "command": "npx", "args": ["-y", "sniff-qa", "--mcp"] } } }
265
+ ```
266
+ </details>
300
267
 
301
- Sniff works with zero config. Only create a config file if you want to customize.
268
+ <details>
269
+ <summary><b>Codex CLI</b></summary>
302
270
 
303
271
  ```bash
304
- npx sniff-qa init # auto-detects: .ts if TypeScript, .mjs if ESM, .js otherwise
305
- npx sniff-qa init --ts # force TypeScript config
306
- npx sniff-qa init --js # force JavaScript config (respects package.json "type")
272
+ codex mcp add sniff-qa -- npx -y sniff-qa --mcp
307
273
  ```
274
+ </details>
308
275
 
309
276
  <details>
310
- <summary><b>sniff.config.ts reference</b></summary>
311
-
312
- ```typescript
313
- import { defineConfig } from 'sniff-qa';
314
-
315
- export default defineConfig({
316
- // Save your URL so you can just run `sniff`
317
- browser: {
318
- baseUrl: 'http://localhost:3000',
319
- projects: ['chromium'], // also supports 'firefox' and 'webkit'
320
- },
321
-
322
- // Viewports to test
323
- viewports: [
324
- { name: 'mobile', width: 375, height: 667 },
325
- { name: 'desktop', width: 1280, height: 720 },
326
- ],
327
-
328
- // Toggle scanner groups
329
- scanners: ['source', 'repo-analyzer', 'e2e', 'accessibility', 'visual', 'performance'],
330
- accessibility: { enabled: true, standard: 'wcag21aa' },
331
- visual: { enabled: true, threshold: 0.1 },
332
- performance: { enabled: true, budgets: { lcp: 2500, fcp: 1800, tti: 3800 } },
333
-
334
- // Optional AI providers. Default is deterministic local QA with provider: 'none'.
335
- ai: {
336
- provider: 'none', // codex-cli, claude-code, anthropic-api, openai-api, gemini-cli, ollama
337
- model: undefined,
338
- command: undefined,
339
- baseUrl: undefined,
340
- },
341
-
342
- // Optional AI explorer, run only with --explore
343
- exploration: { enabled: true, maxSteps: 50 },
344
-
345
- // Dead link checker
346
- deadLinks: {
347
- scanCode: false, // docs/HTML by default; opt into JSX/TSX/JS/TS to reduce noise
348
- checkExternal: true,
349
- timeout: 5000,
350
- retries: 2,
351
- ignorePatterns: [],
352
- maxConcurrent: 10,
353
- },
354
-
355
- // API endpoint discovery
356
- apiEndpoints: {
357
- checkErrorHandling: true,
358
- checkValidation: true,
359
- checkAuth: true,
360
- checkSecrets: true,
361
- frameworks: [], // empty = auto-detect all
362
- },
363
-
364
- // Turn off specific rules
365
- rules: {
366
- 'debug-console-log': 'off',
367
- },
368
- });
369
- ```
277
+ <summary><b>Gemini CLI</b></summary>
370
278
 
371
- Provider selection can also come from environment: `SNIFF_AI_PROVIDER`, `SNIFF_AI_MODEL`, `SNIFF_AI_COMMAND`, and `SNIFF_AI_BASE_URL`. API-backed providers use their normal keys: `ANTHROPIC_API_KEY` or `OPENAI_API_KEY`. If the selected provider is missing, Sniff skips AI generation and keeps the deterministic QA run moving.
279
+ Add to `~/.gemini/mcp_config.json`:
372
280
 
281
+ ```json
282
+ { "mcpServers": { "sniff-qa": { "command": "npx", "args": ["-y", "sniff-qa", "--mcp"] } } }
283
+ ```
373
284
  </details>
374
285
 
375
286
  <details>
376
- <summary><b>All rule IDs</b></summary>
377
-
378
- | Rule | Severity | What it checks |
379
- |:-----|:---------|:---------------|
380
- | `debug-console-log` | medium | console.log/debug/info |
381
- | `debug-debugger` | high | debugger statements |
382
- | `placeholder-lorem` | high | Lorem ipsum text |
383
- | `placeholder-todo` | medium | TODO comments |
384
- | `placeholder-fixme` | high | FIXME comments |
385
- | `placeholder-tbd` | medium | TBD markers |
386
- | `hardcoded-localhost` | medium | localhost URLs |
387
- | `hardcoded-127` | medium | 127.0.0.1 URLs |
388
- | `broken-import` | medium | Unresolved imports |
389
- | `dead-link-internal` | high | Broken file links |
390
- | `dead-link-external` | medium | 404 external URLs |
391
- | `dead-link-anchor` | medium | Missing anchors |
392
- | `api-no-error-handling` | medium | Routes without try/catch |
393
- | `api-no-validation` | medium | POST/PUT without validation |
394
- | `api-no-auth` | low | Routes without auth |
395
- | `api-hardcoded-secret` | critical | Hardcoded API keys |
396
-
397
- Set any to `'off'` to disable.
287
+ <summary><b>Windsurf</b></summary>
398
288
 
399
- </details>
289
+ Add to `~/.codeium/windsurf/mcp_config.json`:
400
290
 
401
- ---
291
+ ```json
292
+ { "mcpServers": { "sniff-qa": { "command": "npx", "args": ["-y", "sniff-qa", "--mcp"] } } }
293
+ ```
294
+ </details>
402
295
 
403
- ## CI integration
296
+ <details>
297
+ <summary><b>Continue.dev</b></summary>
404
298
 
405
- Run Sniff directly in CI:
299
+ Add to `.continue/mcpServers/sniff-qa.yaml`:
406
300
 
407
- ```bash
408
- npx sniff-qa --ci --format html,json,junit
301
+ ```yaml
302
+ mcpServers:
303
+ sniff-qa: { command: npx, args: ["-y", "sniff-qa", "--mcp"], type: stdio }
409
304
  ```
305
+ </details>
410
306
 
411
- Generate a GitHub Actions workflow:
307
+ > The first browser-based walk downloads Chromium (~165 MB). Over MCP, Sniff returns a structured `needsSetup` payload instead of blocking the editor on a long download — run the install it shows you, then ask again.
412
308
 
413
- ```bash
414
- npx sniff-qa ci
415
- ```
309
+ ---
416
310
 
417
- This writes `.github/workflows/sniff.yml` with Playwright caching, JUnit output, and report artifacts.
311
+ ## How it works
418
312
 
419
- **Flakiness quarantine:** Tests that fail 3 of 5 runs get quarantined. They still run, still report, but won't block your pipeline.
313
+ 1. **Find the app.** Sniff auto-detects your running dev server (or you pass `--url`).
314
+ 2. **Walk the flows.** It opens pages in a headless browser and interacts with them like a user — clicking, filling forms, following links — across desktop and a 375px mobile pass.
315
+ 3. **Watch everything.** It records console errors, failed network requests, broken renders, missing feedback, and accessibility issues as it goes.
316
+ 4. **Filter the noise.** The first-party noise filter and axe-core drop the false positives; uncertain findings are held back.
317
+ 5. **Report with proof.** Each surviving finding gets a severity, a confidence, reproduction steps, a screenshot, and a suggested fix — in the terminal and an optional HTML report.
318
+
319
+ <img alt="How sniff works: crawl, act, assert, prove, report" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/how-it-works.svg" width="100%">
420
320
 
421
321
  ---
422
322
 
423
- ## Autonomous discovery
323
+ ## CI integration
324
+
325
+ Run Sniff in your pipeline and fail the build on real bugs:
424
326
 
425
327
  ```bash
426
- sniff discover # run against the detected dev server
427
- sniff discover --regenerate-only # write sniff-scenarios/ and exit (no run)
328
+ npx sniff-qa --ci --fail-on high
428
329
  ```
429
330
 
430
- `sniff discover` reads your source (Prisma, Drizzle, TypeORM, Zod, GraphQL, OpenAPI, TS types), classifies what the app is (ecommerce, booking, saas, social, content, crm, auth-only, marketing, admin) with route tokens in **English, French, Spanish, German, Portuguese, and Italian** (a French SaaS on `/fr/tableau-de-bord` classifies correctly), generates happy-path journeys with real personas, enumerates edge variants (invalid email, XSS, payment declined, empty cart, offline, and more), and drives everything through Playwright.
331
+ Generate a ready-to-commit GitHub Actions workflow:
431
332
 
432
- Pass `--verbose` to see the top-3 candidates and every matched signal — useful for understanding *why* a classification happened. Pass `--force-app-type <type>` to skip classification entirely. Pass `--dry-run` to preview scenarios without launching a browser or writing reports.
333
+ ```bash
334
+ npx sniff-qa ci
335
+ ```
433
336
 
434
- **Multilingual projects supported.** The classifier matches French, Spanish, German, Portuguese (BR/PT), and Italian equivalents for every signature token, so a French SaaS on `/fr/tableau-de-bord` classifies the same as an English one on `/dashboard`.
337
+ This writes `.github/workflows/sniff.yml` with browser caching and report artifacts.
435
338
 
436
- **What you get:**
339
+ ---
437
340
 
438
- - `sniff-scenarios/_generated/<app-type>/<journey>.<variant>.scenario.md` — reviewable scenarios with JSON frontmatter. Track them in git. Hand-edit them — sniff detects the change and asks before regenerating.
439
- - `sniff-reports/discovery/` — HTML, JSON, and JUnit reports per run.
440
- - Flakiness quarantine for scenarios that fail 3 of the last 5 runs.
341
+ ## FAQ
441
342
 
442
- **Flags:**
343
+ **Does it work without a dev server?**
344
+ Sniff is built to walk a *running* app, so that's where it shines. If no server is running, it doesn't fail silently — it runs a source-only scan and tells you exactly how to start the real walk (start your dev server or pass `--url`). You can also run `npx sniff-qa scan` to get the source scan on purpose.
443
345
 
444
- | Flag | What it does |
445
- |:-----|:-------------|
446
- | `--url <url>` | Target URL (default: auto-detect dev server) |
447
- | `--max-scenarios <n>` | Cap total scenarios (default: 50) |
448
- | `--max-variants-per-scenario <n>` | Cap edge variants per scenario (default: 3) |
449
- | `--max-variants-per-run <n>` | Cap edge variants per run (default: 40) |
450
- | `--realism <profile>` | `robot`, `careful-user`, `casual-user` (default), `frustrated-user`, `power-user` |
451
- | `--seed <n>` | Replay a specific random seed |
452
- | `--only <filter>` | Filter scenarios by id substring or app type |
453
- | `--app-type <types>` | Filter classifier guesses to these app types (comma-separated, kept for back-compat) |
454
- | `--force-app-type <type>` | Bypass the classifier entirely — treat the app as this type |
455
- | `--verbose` | Print classifier breakdown (top 3 guesses + matched signals per dimension) |
456
- | `--dry-run` | Generate scenarios without launching the browser or writing reports |
457
- | `--regenerate` | Regenerate `sniff-scenarios/` before running |
458
- | `--regenerate-only` | Regenerate and exit (no browser) |
459
- | `--force-regenerate` | Overwrite hand-edits without prompting |
460
- | `--non-interactive` | Skip prompts and the production-URL countdown |
461
- | `--verbose` | Print classification breakdown (top 3 guesses + matched signals per dimension) |
462
- | `--no-llm` | Skip Claude Code CLI polish (deterministic only) |
463
-
464
- **Production-URL safety:** when the target is not a local address, sniff prints a 5-second countdown banner before running. Hit Ctrl+C to cancel. No `--allow-destructive` flag.
465
-
466
- **LLM polish (optional):** discovery can use the `claude` CLI to break close-call app-type classifications. Responses are cached under `.sniff/discover/cache/`. Pass `--no-llm` to skip.
346
+ **What gets downloaded on first run?**
347
+ The first time Sniff opens a browser, it downloads a Chromium build (~165 MB, one-time, then cached). You'll see the progress, and you need internet access for that first run. Nothing else is installed and no account is created.
467
348
 
468
- ---
349
+ **Do I need an API key?**
350
+ No. Sniff runs entirely on your machine with no API key and no signup. Your code and your app never leave your computer.
469
351
 
470
- ## How it works
352
+ **How is it different from a linter?**
353
+ A linter reads your source files and never runs your app, so it can't see a dead submit button, an infinite spinner, a wiped form, or a 500 page. Sniff opens your real app, interacts with it, and reports what actually broke — with a screenshot and steps to reproduce.
471
354
 
472
- <picture>
473
- <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/pipeline-dark.svg">
474
- <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/pipeline-light.svg">
475
- <img alt="Sniff pipeline" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/pipeline-light.svg" width="100%">
476
- </picture>
355
+ **How is it different from Playwright codegen (or writing E2E tests)?**
356
+ Playwright codegen records a script that *you* author and maintain; it tests only the path you clicked. Sniff writes nothing for you to maintain — it explores your flows on its own and judges the outcome, catching things a recorded happy-path never checks (empty/placeholder data, state-loss, missing success feedback).
477
357
 
478
- <details>
479
- <summary><b>Architecture</b></summary>
358
+ **Will it change my code?**
359
+ No, not during a walk. Walking and scanning are read-only. The separate `sniff fix` command applies safe auto-fixes (like stray `console.log`/`debugger`) and only when you run it.
480
360
 
481
- <picture>
482
- <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/architecture-dark.svg">
483
- <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/architecture-light.svg">
484
- <img alt="Sniff architecture" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/architecture-light.svg" width="100%">
485
- </picture>
486
-
487
- </details>
361
+ **What stacks does it work with?**
362
+ Any web app you can open in a browser — React, Next.js, Vue, Svelte, Angular, Remix, SvelteKit, Astro, plain HTML, and more. It walks the rendered app, so the framework doesn't matter for the browser checks.
488
363
 
489
364
  ---
490
365
 
491
- ## Privacy
492
-
493
- No telemetry. No signup. No data collection. No API keys required. Your code stays on your machine unless you explicitly configure an API-backed AI provider.
366
+ ## Works first-class in
494
367
 
495
- > [!NOTE]
496
- > All deterministic checks work without any API key. Optional AI providers are off by default and must be selected through config or environment. Dead link checking validates external URLs but never sends your code.
368
+ Claude Code · Cursor · VS Code (Copilot) · Codex · Gemini CLI · Windsurf · Continue.dev — via the MCP server (command `npx`, args `["-y", "sniff-qa", "--mcp"]`) or the CLI directly.
497
369
 
498
370
  ---
499
371
 
500
372
  ## Contributing
501
373
 
502
- Add a source rule: each rule is a regex + severity in `src/scanners/source/rules/`. See [CONTRIBUTING.md](CONTRIBUTING.md).
374
+ Issues and PRs welcome. See [CONTRIBUTING.md](CONTRIBUTING.md).
503
375
 
504
376
  ---
505
377
 
506
378
  <p align="center">
507
379
  <sub>
508
- Built on <a href="https://playwright.dev">Playwright</a> · <a href="https://github.com/dequelabs/axe-core">axe-core</a> · <a href="https://developer.chrome.com/docs/lighthouse">Lighthouse</a> · <a href="https://github.com/mapbox/pixelmatch">pixelmatch</a> · <a href="https://zod.dev">Zod</a> · <a href="https://github.com/modelcontextprotocol/typescript-sdk">MCP SDK</a> · <a href="https://github.com/anthropics/anthropic-sdk-typescript">Anthropic SDK</a>
380
+ Built on <a href="https://playwright.dev">Playwright</a> · <a href="https://github.com/dequelabs/axe-core">axe-core</a> · <a href="https://developer.chrome.com/docs/lighthouse">Lighthouse</a> · <a href="https://github.com/mapbox/pixelmatch">pixelmatch</a> · <a href="https://zod.dev">Zod</a> · <a href="https://github.com/modelcontextprotocol/typescript-sdk">MCP SDK</a>
509
381
  </sub>
510
382
  </p>
511
383