sniff-qa 0.5.1 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +2 -2
- package/CHANGELOG.md +66 -0
- package/README.md +219 -347
- package/SECURITY.md +3 -3
- package/dist/chunk-LJYT4MD5.js +1246 -0
- package/dist/chunk-LJYT4MD5.js.map +1 -0
- package/dist/cli/index.js +41 -12
- package/dist/cli/index.js.map +1 -1
- package/dist/crawl-666NLYDC.js +11 -0
- package/dist/crawl-666NLYDC.js.map +1 -0
- package/dist/crawl-command-3OB4IUBU.js +54 -0
- package/dist/crawl-command-3OB4IUBU.js.map +1 -0
- package/dist/{discover-ENPRVN5A.js → discover-Q363YJ5X.js} +2 -2
- package/dist/{domain-UQ34YE4B.js → domain-NNL2PPRK.js} +1 -1
- package/dist/index.d.ts +94 -1
- package/dist/index.js +9 -1
- package/dist/index.js.map +1 -1
- package/dist/mcp/server.js +76 -6
- package/dist/mcp/server.js.map +1 -1
- package/dist/{model-X5J2JR5U.js → model-2DTSAX7T.js} +1 -1
- package/dist/{quality-run-6SXCN5CD.js → quality-run-LLBZZI7X.js} +2 -2
- package/dist/{repo-analyzer-4OURSITY.js → repo-analyzer-WK5WP7NT.js} +1 -1
- package/dist/{report-OAHSB56I.js → report-IKRVVIW4.js} +2 -2
- package/dist/{unified-SKXQQX3Y.js → unified-67Z6YA64.js} +4 -4
- package/package.json +3 -2
- package/skills/sniff/SKILL.md +20 -25
- /package/dist/{discover-ENPRVN5A.js.map → discover-Q363YJ5X.js.map} +0 -0
- /package/dist/{domain-UQ34YE4B.js.map → domain-NNL2PPRK.js.map} +0 -0
- /package/dist/{model-X5J2JR5U.js.map → model-2DTSAX7T.js.map} +0 -0
- /package/dist/{quality-run-6SXCN5CD.js.map → quality-run-LLBZZI7X.js.map} +0 -0
- /package/dist/{repo-analyzer-4OURSITY.js.map → repo-analyzer-WK5WP7NT.js.map} +0 -0
- /package/dist/{report-OAHSB56I.js.map → report-IKRVVIW4.js.map} +0 -0
- /package/dist/{unified-SKXQQX3Y.js.map → unified-67Z6YA64.js.map} +0 -0
package/README.md
CHANGED
|
@@ -9,503 +9,375 @@
|
|
|
9
9
|
<a href="LICENSE"><img src="https://img.shields.io/badge/license-Apache_2.0-ef4444?style=flat-square" alt="License"></a>
|
|
10
10
|
<a href="https://github.com/Aboudjem/sniff/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/Aboudjem/sniff/ci.yml?style=flat-square&label=CI" alt="CI"></a>
|
|
11
11
|
<a href="https://nodejs.org"><img src="https://img.shields.io/badge/node-%E2%89%A522-339933?style=flat-square&logo=node.js&logoColor=white" alt="Node"></a>
|
|
12
|
+
<a href="https://github.com/Aboudjem/10x"><img src="https://img.shields.io/badge/10x-marketplace-ef4444?style=flat-square" alt="10x marketplace"></a>
|
|
12
13
|
<a href="https://github.com/Aboudjem/sniff/stargazers"><img src="https://img.shields.io/github/stars/Aboudjem/sniff?style=flat-square&color=ef4444" alt="Stars"></a>
|
|
13
14
|
</p>
|
|
14
15
|
|
|
15
|
-
<p align="center"><b>
|
|
16
|
+
<p align="center"><b>Point it at your running app. It walks your real user flows in a real browser and tells you what's actually broken — with proof.</b></p>
|
|
16
17
|
|
|
17
18
|
<p align="center">
|
|
18
19
|
<a href="#get-started">Get started</a> ·
|
|
19
20
|
<a href="#what-it-finds">What it finds</a> ·
|
|
20
|
-
<a href="#
|
|
21
|
-
<a href="#
|
|
22
|
-
<a href="
|
|
21
|
+
<a href="#why-you-can-trust-it">Why trust it</a> ·
|
|
22
|
+
<a href="#how-it-works">How it works</a> ·
|
|
23
|
+
<a href="#demo">Demo</a> ·
|
|
24
|
+
<a href="#faq">FAQ</a>
|
|
23
25
|
</p>
|
|
24
26
|
|
|
25
27
|
<picture>
|
|
26
28
|
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/sniff-diagram.svg">
|
|
27
|
-
<img alt="Sniff flow:
|
|
29
|
+
<img alt="Sniff flow: your running app -> headless browser walk -> findings with proof" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/sniff-diagram.svg" width="100%">
|
|
28
30
|
</picture>
|
|
29
31
|
|
|
30
32
|
---
|
|
31
33
|
|
|
32
|
-
##
|
|
34
|
+
## What is this?
|
|
33
35
|
|
|
34
|
-
|
|
36
|
+
Sniff is an autonomous QA scanner. You point it at a running web app and it **walks your app's real user flows in a real (headless) browser** — clicking buttons, filling forms, following links — and reports what's actually broken.
|
|
35
37
|
|
|
36
|
-
|
|
38
|
+
It is **not** a linter and **not** a static scanner. It opens your pages, interacts with them like a user would, and watches what happens.
|
|
39
|
+
|
|
40
|
+
Every bug it reports comes with **proof**: the exact page, the ordered steps to reproduce it, a screenshot, and the console or network excerpt that caught it. A finding without reproduction proof is not a finding.
|
|
37
41
|
|
|
38
42
|
```bash
|
|
39
|
-
npx sniff-qa
|
|
43
|
+
npx sniff-qa --url http://localhost:3000
|
|
40
44
|
```
|
|
41
45
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
### Why developers love it
|
|
46
|
+
One command — no config, no API key, no Playwright setup. Point it at your running app, or just run `npx sniff-qa` from your project and it auto-detects a dev server on common ports.
|
|
45
47
|
|
|
46
|
-
|
|
47
|
-
- **Zero API keys by default.** Deterministic local QA is the default. Codex, Claude, Gemini, Ollama, Anthropic, and OpenAI providers are optional enhancements.
|
|
48
|
-
- **Eight checks, every time.** Accessibility + visual regression + performance + dead links + API contracts + source scanning + broken imports + browser runtime hooks — out of the box.
|
|
49
|
-
- **Talks to your editor.** Ships as an MCP server. Just say *"scan this project"* in Claude / Cursor / Copilot / Continue / Windsurf / Codex / Gemini.
|
|
50
|
-
- **CI-ready.** `sniff --ci` emits JUnit, tracks flakes, and fails on severity thresholds you set. `sniff ci` generates the GitHub Actions workflow.
|
|
51
|
-
- **Actually explains the fix.** Every finding cites the rule, the file, the line — and `/sniff-fix` generates the patch.
|
|
48
|
+
> Sniff walks a **running app**. If no dev server is running, it falls back to a source-code scan and tells you exactly how to start the real walk. See [Get started](#get-started).
|
|
52
49
|
|
|
53
50
|
---
|
|
54
51
|
|
|
55
52
|
## Get started
|
|
56
53
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
```bash
|
|
60
|
-
cd ~/projects/my-app # go to your project
|
|
61
|
-
npx sniff-qa # that's it
|
|
62
|
-
```
|
|
54
|
+
You need **Node.js 22+** and a web app you can run locally (or a URL).
|
|
63
55
|
|
|
64
|
-
|
|
56
|
+
> **Naming:** the npm package is **`sniff-qa`** — use `npx sniff-qa` or `npm install -D sniff-qa`. Once installed, the binary is **`sniff`** (the `sniff-qa` binary works too). Don't run `npx sniff` — that's a different package.
|
|
65
57
|
|
|
66
|
-
|
|
58
|
+
### 1. Start your app
|
|
67
59
|
|
|
68
60
|
```bash
|
|
69
|
-
|
|
70
|
-
npx sniff-qa --url https://myapp.com # production URL
|
|
61
|
+
npm run dev # or however you start your app
|
|
71
62
|
```
|
|
72
63
|
|
|
73
|
-
|
|
64
|
+
### 2. Walk it
|
|
74
65
|
|
|
75
|
-
|
|
66
|
+
In another terminal:
|
|
76
67
|
|
|
77
|
-
|
|
68
|
+
```bash
|
|
69
|
+
npx sniff-qa --url http://localhost:3000 # point it at your running app
|
|
70
|
+
```
|
|
78
71
|
|
|
79
|
-
|
|
80
|
-
<summary><b>Claude Code plugin marketplace</b> (one-command install)</summary>
|
|
72
|
+
That's the reliable one-liner. Sniff also **auto-detects** a dev server on common ports, so from your project folder you can often just run `npx sniff-qa` with no flags. If your app is on a non-standard port (or auto-detect misses it), pass `--url` — that always works. It also walks a deployed URL:
|
|
81
73
|
|
|
82
74
|
```bash
|
|
83
|
-
|
|
84
|
-
claude plugin install sniff@10x
|
|
75
|
+
npx sniff-qa --url https://staging.myapp.com
|
|
85
76
|
```
|
|
86
77
|
|
|
87
|
-
|
|
88
|
-
</details>
|
|
78
|
+
> **Found bugs? It exits non-zero — on purpose.** A walk that finds issues exits with code `1` so CI fails the build; it is **not** a crash (you'll see a `✓ Scan complete` line). Pass `--fail-on none` to always exit `0`.
|
|
89
79
|
|
|
90
|
-
|
|
91
|
-
|
|
80
|
+
> **First run downloads a browser.** The first time Sniff opens a browser it downloads a Chromium build (~165 MB, one-time). You'll see the progress. You need internet access for that first run; after that it's cached.
|
|
81
|
+
|
|
82
|
+
### 3. Read the report
|
|
83
|
+
|
|
84
|
+
Findings print to your terminal, grouped by severity, each with steps to reproduce. Want a shareable page?
|
|
92
85
|
|
|
93
86
|
```bash
|
|
94
|
-
|
|
87
|
+
npx sniff-qa --url http://localhost:3000 --report # runs a walk, then writes sniff-reports/sniff-report.html (self-contained)
|
|
95
88
|
```
|
|
96
|
-
</details>
|
|
97
89
|
|
|
98
|
-
|
|
99
|
-
<summary><b>Cursor</b></summary>
|
|
90
|
+
**No app running?** Sniff doesn't fail silently. It runs a source-only scan and prints a clear next step — start your dev server or pass `--url` — so you can get to the real flow-walk. You can also run the source scan on purpose:
|
|
100
91
|
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
{ "mcpServers": { "sniff-qa": { "type": "stdio", "command": "npx", "args": ["sniff-qa", "--mcp"] } } }
|
|
92
|
+
```bash
|
|
93
|
+
npx sniff-qa scan # source-only scan, no browser
|
|
104
94
|
```
|
|
105
|
-
</details>
|
|
106
95
|
|
|
107
|
-
|
|
108
|
-
<summary><b>VS Code + Copilot</b></summary>
|
|
96
|
+
Stuck? Run `npx sniff-qa doctor` to check your environment (Node, browser, dev server).
|
|
109
97
|
|
|
110
|
-
|
|
111
|
-
```json
|
|
112
|
-
{ "servers": { "sniff-qa": { "type": "stdio", "command": "npx", "args": ["-y", "sniff-qa", "--mcp"] } } }
|
|
113
|
-
```
|
|
114
|
-
</details>
|
|
98
|
+
---
|
|
115
99
|
|
|
116
|
-
|
|
117
|
-
<summary><b>Codex CLI</b></summary>
|
|
100
|
+
## What it finds
|
|
118
101
|
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
102
|
+
Sniff walks your app and looks for **12 classes of real bugs**:
|
|
103
|
+
|
|
104
|
+
| # | Class | Examples |
|
|
105
|
+
|:--|:------|:---------|
|
|
106
|
+
| 1 | **Broken pages / routes** | 4xx/5xx responses, blank renders, crash screens |
|
|
107
|
+
| 2 | **Broken links** | Dead internal and external links |
|
|
108
|
+
| 3 | **Console & network errors** | Uncaught exceptions and failed requests *during interaction* |
|
|
109
|
+
| 4 | **Empty & fake data** | Missing data, plus placeholders like `lorem ipsum`, `TODO`, `test@test.com` |
|
|
110
|
+
| 5 | **Broken forms** | Dead submit buttons, validation that never fires |
|
|
111
|
+
| 6 | **State loss** | Fill a form, hit back, and it's wiped |
|
|
112
|
+
| 7 | **Flow regressions / dead-ends** | A journey that can't be completed |
|
|
113
|
+
| 8 | **Bad loading & error states** | Infinite spinners, missing error states |
|
|
114
|
+
| 9 | **Broken async outcomes** | Submitted but no success feedback (flagged "needs out-of-band verification") |
|
|
115
|
+
| 10 | **Responsive issues** | Overflow and tiny tap targets (a 375px mobile pass) |
|
|
116
|
+
| 11 | **Accessibility** | Missing alt text and labels, contrast — via [axe-core](https://github.com/dequelabs/axe-core) |
|
|
117
|
+
| 12 | **Unclear primary actions** | The main call-to-action is buried or ambiguous |
|
|
118
|
+
|
|
119
|
+
Each finding ships with:
|
|
120
|
+
|
|
121
|
+
- **Reproduction proof** — the exact route, the ordered steps, a screenshot, and the console/network excerpt.
|
|
122
|
+
- **A severity** — so you fix the right thing first.
|
|
123
|
+
- **A confidence** — `confirmed`, `likely`, or `uncertain`. Uncertain findings are hidden by default; add `--all` to see them.
|
|
124
|
+
- **A suggested fix.**
|
|
123
125
|
|
|
124
|
-
<
|
|
125
|
-
<
|
|
126
|
+
<picture>
|
|
127
|
+
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/features-dark.svg">
|
|
128
|
+
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/features-light.svg">
|
|
129
|
+
<img alt="The bug classes Sniff finds" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/features-light.svg" width="100%">
|
|
130
|
+
</picture>
|
|
126
131
|
|
|
127
|
-
|
|
128
|
-
```json
|
|
129
|
-
{ "mcpServers": { "sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] } } }
|
|
130
|
-
```
|
|
131
|
-
</details>
|
|
132
|
+
---
|
|
132
133
|
|
|
133
|
-
|
|
134
|
-
<summary><b>Gemini CLI</b></summary>
|
|
134
|
+
## Why you can trust it
|
|
135
135
|
|
|
136
|
-
|
|
137
|
-
```json
|
|
138
|
-
{ "mcpServers": { "sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] } } }
|
|
139
|
-
```
|
|
140
|
-
</details>
|
|
136
|
+
Most scanners drown you in false positives until you stop reading them. Sniff is built the opposite way.
|
|
141
137
|
|
|
142
|
-
|
|
143
|
-
<summary><b>Continue.dev</b></summary>
|
|
138
|
+
We measured it on a fixture app planted with **21 bugs across all 12 classes**, plus a clean control page that should produce zero findings:
|
|
144
139
|
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
140
|
+
| | Old engine | New engine |
|
|
141
|
+
|:--|:--|:--|
|
|
142
|
+
| Bugs found | 9 / 21 (43%) | **21 / 21 (100%)** |
|
|
143
|
+
| Precision | ~13% | **100%** |
|
|
144
|
+
| False positives | 125 | **0** |
|
|
145
|
+
| Findings on the clean page | — | **0** |
|
|
146
|
+
| Flagship command | crashed | works |
|
|
151
147
|
|
|
152
|
-
|
|
153
|
-
<summary><b>OpenClaw</b></summary>
|
|
148
|
+
Those numbers are locked as a regression test. The full suite is **441 tests**.
|
|
154
149
|
|
|
155
|
-
|
|
156
|
-
clawhub install sniff-qa
|
|
157
|
-
```
|
|
158
|
-
</details>
|
|
150
|
+
**How it keeps false positives near zero:**
|
|
159
151
|
|
|
160
|
-
|
|
152
|
+
- A first-party **noise filter** drops the junk that isn't your bug — favicons, analytics, hot-module-reload chatter, expected auth redirects, engine aborts.
|
|
153
|
+
- Accessibility findings are backed by **axe-core**, which is zero-false-positive by design.
|
|
154
|
+
- **Uncertain findings are suppressed by default** (use `--all` to see them).
|
|
155
|
+
- A broken page is reported **once**, not re-flagged on every link that points to it.
|
|
161
156
|
|
|
162
|
-
|
|
157
|
+
If Sniff can't prove a bug, it doesn't claim one.
|
|
163
158
|
|
|
164
|
-
|
|
159
|
+
---
|
|
165
160
|
|
|
166
|
-
|
|
167
|
-
npm install -D sniff-qa
|
|
168
|
-
```
|
|
161
|
+
## How is it different?
|
|
169
162
|
|
|
170
|
-
|
|
171
|
-
{
|
|
172
|
-
"scripts": {
|
|
173
|
-
"qa": "sniff"
|
|
174
|
-
}
|
|
175
|
-
}
|
|
176
|
-
```
|
|
163
|
+
Linters read your source. End-to-end frameworks make *you* write the tests. Link checkers only check links. Sniff drives your real app and judges the result.
|
|
177
164
|
|
|
178
|
-
|
|
165
|
+
| | **Sniff** | linkinator | pa11y | Playwright codegen | QA-Wolf-style services |
|
|
166
|
+
|:--|:--|:--|:--|:--|:--|
|
|
167
|
+
| Walks real user flows in a browser | **Yes** | No | No | You script it | Yes |
|
|
168
|
+
| Zero setup, zero test-writing | **Yes** | Yes | Yes | No (you write tests) | No (onboarding) |
|
|
169
|
+
| Broken links | **Yes** | Yes | No | Manual | Manual |
|
|
170
|
+
| Accessibility (axe-core) | **Yes** | No | **Yes** | Manual | Some |
|
|
171
|
+
| Empty / placeholder / fake data | **Yes** | No | No | No | No |
|
|
172
|
+
| State-loss (back-button wipes a form) | **Yes** | No | No | Manual | Manual |
|
|
173
|
+
| One-shot reproduction proof per finding | **Yes** | No | Partial | No | Varies |
|
|
174
|
+
| Self-contained HTML report | **Yes** | No | Partial | No | Dashboard |
|
|
175
|
+
| Runs locally, no account, no API key | **Yes** | Yes | Yes | Yes | No (service) |
|
|
179
176
|
|
|
180
|
-
|
|
177
|
+
What Sniff uniquely does in one command: catch **empty/placeholder data**, **state-loss**, and **broken async outcomes** — and hand you a single proof report — with **no scripts to write and no service to sign up for**.
|
|
181
178
|
|
|
182
179
|
---
|
|
183
180
|
|
|
184
|
-
##
|
|
181
|
+
## Commands
|
|
185
182
|
|
|
186
|
-
```
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
183
|
+
```
|
|
184
|
+
sniff Walk your app (auto-detects the dev server). The default.
|
|
185
|
+
sniff --url <url> Walk a specific URL
|
|
186
|
+
sniff scan Source-only scan, no browser (placeholders, TODOs, dead links, etc.)
|
|
187
|
+
sniff report Show the results from the last run
|
|
188
|
+
sniff doctor Check your environment (Node, browser, config, dev server)
|
|
189
|
+
sniff ci Generate a GitHub Actions workflow
|
|
190
|
+
sniff fix Auto-fix safe issues (console.log, debugger, etc.)
|
|
191
|
+
sniff --help Show every command and flag
|
|
192
|
+
sniff --version Show the version
|
|
195
193
|
```
|
|
196
194
|
|
|
197
|
-
|
|
195
|
+
### Useful flags
|
|
196
|
+
|
|
197
|
+
| Flag | What it does |
|
|
198
|
+
|:-----|:-------------|
|
|
199
|
+
| `--url <url>` | Walk this URL instead of auto-detecting |
|
|
200
|
+
| `--report` | Write a self-contained HTML report to `sniff-reports/sniff-report.html` |
|
|
201
|
+
| `--all` | Also show low-confidence (`uncertain`) findings |
|
|
202
|
+
| `--max-pages <n>` | Cap how many pages to walk (default: 25) |
|
|
203
|
+
| `--no-mobile` | Skip the 375px responsive pass |
|
|
204
|
+
| `--headed` | Show the browser window while it walks |
|
|
205
|
+
| `--json` | Machine-readable JSON output |
|
|
206
|
+
| `--ci` | CI mode (stable output, non-interactive) |
|
|
207
|
+
| `--fail-on <sev>` | Exit non-zero on findings at or above this severity |
|
|
198
208
|
|
|
199
209
|
---
|
|
200
210
|
|
|
201
|
-
##
|
|
211
|
+
## Demo
|
|
202
212
|
|
|
203
|
-
|
|
204
|
-
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/features-dark.svg">
|
|
205
|
-
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/features-light.svg">
|
|
206
|
-
<img alt="Sniff checks: source bugs, dead links, API endpoints, broken imports, accessibility, visual regression, performance, optional AI explorer" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/features-light.svg" width="100%">
|
|
207
|
-
</picture>
|
|
213
|
+
A real run against a buggy app — 21 real issues, zero false positives, every finding with severity, confidence, reproduction steps, and a fix:
|
|
208
214
|
|
|
209
|
-
|
|
215
|
+
<img alt="Sniff walking a buggy app and reporting 21 real issues with zero false positives" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/demo.svg" width="100%">
|
|
210
216
|
|
|
211
217
|
---
|
|
212
218
|
|
|
213
|
-
##
|
|
219
|
+
## Use it from your AI editor
|
|
214
220
|
|
|
215
|
-
|
|
216
|
-
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/report-dark.svg">
|
|
217
|
-
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/report-light.svg">
|
|
218
|
-
<img alt="Sniff terminal output" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/report-light.svg" width="100%">
|
|
219
|
-
</picture>
|
|
221
|
+
Sniff ships as an MCP server too. Add it once, then just ask your assistant *"scan this project for bugs"* or *"walk my app."* If your app is running, Sniff auto-detects it — you don't have to pass a URL.
|
|
220
222
|
|
|
221
|
-
|
|
223
|
+
**One unified `sniff` tool, three modes:**
|
|
222
224
|
|
|
223
|
-
|
|
225
|
+
- `walk` — **recommended.** Walks your running app's real flows (the flow-walk above).
|
|
226
|
+
- `scan` — source-only scan, no browser.
|
|
227
|
+
- `report` — show the last run's results.
|
|
224
228
|
|
|
225
|
-
|
|
229
|
+
(`run` and `discover` are legacy modes kept for back-compat.)
|
|
226
230
|
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
| `/sniff` | Scan the project. Auto-detects dev server, runs source + browser checks |
|
|
230
|
-
| `/sniff-fix` | Scan and auto-fix safe issues (debugger, console.log, etc.) |
|
|
231
|
-
| `/sniff-report` | Show results from the last scan |
|
|
231
|
+
<details>
|
|
232
|
+
<summary><b>Claude Code</b></summary>
|
|
232
233
|
|
|
233
|
-
|
|
234
|
+
One-command plugin install from the [10x marketplace](https://github.com/Aboudjem/10x):
|
|
234
235
|
|
|
236
|
+
```bash
|
|
237
|
+
claude plugin marketplace add Aboudjem/10x
|
|
238
|
+
claude plugin install sniff@10x
|
|
235
239
|
```
|
|
236
|
-
sniff Scan source + auto-detect dev server
|
|
237
|
-
sniff --url <url> Scan source + test specific URL
|
|
238
|
-
sniff --url <url> --ci Full audit for CI pipelines
|
|
239
|
-
sniff <path> Scan a specific directory
|
|
240
|
-
sniff discover Autonomous E2E discovery (scenarios + edge cases)
|
|
241
|
-
sniff discover --regenerate-only Write sniff-scenarios/ tree without running
|
|
242
|
-
sniff fix Auto-fix safe issues (debugger, console.log)
|
|
243
|
-
sniff fix --check Dry run: show what would be fixed
|
|
244
|
-
sniff init Create sniff.config.ts
|
|
245
|
-
sniff ci Generate GitHub Actions workflow
|
|
246
|
-
sniff report Show last scan results
|
|
247
|
-
sniff update-baselines Accept current visual baselines
|
|
248
|
-
sniff doctor Check your environment (Node, Playwright, config, dev server)
|
|
249
|
-
sniff --help Show all commands and flags
|
|
250
|
-
sniff --version Show version
|
|
251
|
-
```
|
|
252
|
-
|
|
253
|
-
<details>
|
|
254
|
-
<summary><b>All flags</b></summary>
|
|
255
240
|
|
|
256
|
-
|
|
257
|
-
|:-----|:-------------|
|
|
258
|
-
| `--url <url>` | Enable browser checks (accessibility, visual, performance, AI) |
|
|
259
|
-
| `--ci` | CI mode: skip AI explorer, add JUnit output, track flaky tests |
|
|
260
|
-
| `--explore` | Opt into AI-assisted exploration after browser checks |
|
|
261
|
-
| `--no-explore` | Compatibility flag; exploration is off by default |
|
|
262
|
-
| `--no-browser` | Source only even if `--url` is set |
|
|
263
|
-
| `--max-steps <n>` | Limit AI explorer steps (default: 50) |
|
|
264
|
-
| `--no-headless` | Show the browser window |
|
|
265
|
-
| `--headed` | Alias for `--no-headless` |
|
|
266
|
-
| `--format html,json,junit` | Choose report formats |
|
|
267
|
-
| `--fail-on critical,high` | Severities that cause non-zero exit |
|
|
268
|
-
| `--track-flakes` | Track test flakiness across runs |
|
|
269
|
-
| `--json` | JSON output for scripts |
|
|
270
|
-
| `--verbose` | Print classifier top-3 guesses and matched signals per dimension |
|
|
271
|
-
| `--dry-run` | Discover: generate scenarios without launching browser or writing reports |
|
|
272
|
-
| `--force-app-type <type>` | Discover: bypass the classifier (e.g. `saas`, `ecommerce`, `booking`) |
|
|
273
|
-
| `--app-type <types>` | Discover: filter classifier guesses to these app types (comma-separated) |
|
|
241
|
+
Or add just the MCP server:
|
|
274
242
|
|
|
243
|
+
```bash
|
|
244
|
+
claude mcp add sniff-qa npx -- -y sniff-qa --mcp
|
|
245
|
+
```
|
|
275
246
|
</details>
|
|
276
247
|
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
## Works with any stack
|
|
248
|
+
<details>
|
|
249
|
+
<summary><b>Cursor</b></summary>
|
|
280
250
|
|
|
281
|
-
|
|
251
|
+
Add to `~/.cursor/mcp.json`:
|
|
282
252
|
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
<td align="center" width="14%"><b>Vue</b><br/><sub>SFC</sub></td>
|
|
288
|
-
<td align="center" width="14%"><b>Svelte</b><br/><sub>Components</sub></td>
|
|
289
|
-
<td align="center" width="14%"><b>Angular</b><br/><sub>Templates</sub></td>
|
|
290
|
-
<td align="center" width="14%"><b>Express</b><br/><sub>Routes</sub></td>
|
|
291
|
-
<td align="center" width="14%"><b>Vanilla</b><br/><sub>HTML / CSS</sub></td>
|
|
292
|
-
</tr>
|
|
293
|
-
</table>
|
|
253
|
+
```json
|
|
254
|
+
{ "mcpServers": { "sniff-qa": { "command": "npx", "args": ["-y", "sniff-qa", "--mcp"] } } }
|
|
255
|
+
```
|
|
256
|
+
</details>
|
|
294
257
|
|
|
295
|
-
|
|
258
|
+
<details>
|
|
259
|
+
<summary><b>VS Code (Copilot)</b></summary>
|
|
296
260
|
|
|
297
|
-
|
|
261
|
+
Add to `.vscode/mcp.json`:
|
|
298
262
|
|
|
299
|
-
|
|
263
|
+
```json
|
|
264
|
+
{ "servers": { "sniff-qa": { "type": "stdio", "command": "npx", "args": ["-y", "sniff-qa", "--mcp"] } } }
|
|
265
|
+
```
|
|
266
|
+
</details>
|
|
300
267
|
|
|
301
|
-
|
|
268
|
+
<details>
|
|
269
|
+
<summary><b>Codex CLI</b></summary>
|
|
302
270
|
|
|
303
271
|
```bash
|
|
304
|
-
|
|
305
|
-
npx sniff-qa init --ts # force TypeScript config
|
|
306
|
-
npx sniff-qa init --js # force JavaScript config (respects package.json "type")
|
|
272
|
+
codex mcp add sniff-qa -- npx -y sniff-qa --mcp
|
|
307
273
|
```
|
|
274
|
+
</details>
|
|
308
275
|
|
|
309
276
|
<details>
|
|
310
|
-
<summary><b>
|
|
311
|
-
|
|
312
|
-
```typescript
|
|
313
|
-
import { defineConfig } from 'sniff-qa';
|
|
314
|
-
|
|
315
|
-
export default defineConfig({
|
|
316
|
-
// Save your URL so you can just run `sniff`
|
|
317
|
-
browser: {
|
|
318
|
-
baseUrl: 'http://localhost:3000',
|
|
319
|
-
projects: ['chromium'], // also supports 'firefox' and 'webkit'
|
|
320
|
-
},
|
|
321
|
-
|
|
322
|
-
// Viewports to test
|
|
323
|
-
viewports: [
|
|
324
|
-
{ name: 'mobile', width: 375, height: 667 },
|
|
325
|
-
{ name: 'desktop', width: 1280, height: 720 },
|
|
326
|
-
],
|
|
327
|
-
|
|
328
|
-
// Toggle scanner groups
|
|
329
|
-
scanners: ['source', 'repo-analyzer', 'e2e', 'accessibility', 'visual', 'performance'],
|
|
330
|
-
accessibility: { enabled: true, standard: 'wcag21aa' },
|
|
331
|
-
visual: { enabled: true, threshold: 0.1 },
|
|
332
|
-
performance: { enabled: true, budgets: { lcp: 2500, fcp: 1800, tti: 3800 } },
|
|
333
|
-
|
|
334
|
-
// Optional AI providers. Default is deterministic local QA with provider: 'none'.
|
|
335
|
-
ai: {
|
|
336
|
-
provider: 'none', // codex-cli, claude-code, anthropic-api, openai-api, gemini-cli, ollama
|
|
337
|
-
model: undefined,
|
|
338
|
-
command: undefined,
|
|
339
|
-
baseUrl: undefined,
|
|
340
|
-
},
|
|
341
|
-
|
|
342
|
-
// Optional AI explorer, run only with --explore
|
|
343
|
-
exploration: { enabled: true, maxSteps: 50 },
|
|
344
|
-
|
|
345
|
-
// Dead link checker
|
|
346
|
-
deadLinks: {
|
|
347
|
-
scanCode: false, // docs/HTML by default; opt into JSX/TSX/JS/TS to reduce noise
|
|
348
|
-
checkExternal: true,
|
|
349
|
-
timeout: 5000,
|
|
350
|
-
retries: 2,
|
|
351
|
-
ignorePatterns: [],
|
|
352
|
-
maxConcurrent: 10,
|
|
353
|
-
},
|
|
354
|
-
|
|
355
|
-
// API endpoint discovery
|
|
356
|
-
apiEndpoints: {
|
|
357
|
-
checkErrorHandling: true,
|
|
358
|
-
checkValidation: true,
|
|
359
|
-
checkAuth: true,
|
|
360
|
-
checkSecrets: true,
|
|
361
|
-
frameworks: [], // empty = auto-detect all
|
|
362
|
-
},
|
|
363
|
-
|
|
364
|
-
// Turn off specific rules
|
|
365
|
-
rules: {
|
|
366
|
-
'debug-console-log': 'off',
|
|
367
|
-
},
|
|
368
|
-
});
|
|
369
|
-
```
|
|
277
|
+
<summary><b>Gemini CLI</b></summary>
|
|
370
278
|
|
|
371
|
-
|
|
279
|
+
Add to `~/.gemini/mcp_config.json`:
|
|
372
280
|
|
|
281
|
+
```json
|
|
282
|
+
{ "mcpServers": { "sniff-qa": { "command": "npx", "args": ["-y", "sniff-qa", "--mcp"] } } }
|
|
283
|
+
```
|
|
373
284
|
</details>
|
|
374
285
|
|
|
375
286
|
<details>
|
|
376
|
-
<summary><b>
|
|
377
|
-
|
|
378
|
-
| Rule | Severity | What it checks |
|
|
379
|
-
|:-----|:---------|:---------------|
|
|
380
|
-
| `debug-console-log` | medium | console.log/debug/info |
|
|
381
|
-
| `debug-debugger` | high | debugger statements |
|
|
382
|
-
| `placeholder-lorem` | high | Lorem ipsum text |
|
|
383
|
-
| `placeholder-todo` | medium | TODO comments |
|
|
384
|
-
| `placeholder-fixme` | high | FIXME comments |
|
|
385
|
-
| `placeholder-tbd` | medium | TBD markers |
|
|
386
|
-
| `hardcoded-localhost` | medium | localhost URLs |
|
|
387
|
-
| `hardcoded-127` | medium | 127.0.0.1 URLs |
|
|
388
|
-
| `broken-import` | medium | Unresolved imports |
|
|
389
|
-
| `dead-link-internal` | high | Broken file links |
|
|
390
|
-
| `dead-link-external` | medium | 404 external URLs |
|
|
391
|
-
| `dead-link-anchor` | medium | Missing anchors |
|
|
392
|
-
| `api-no-error-handling` | medium | Routes without try/catch |
|
|
393
|
-
| `api-no-validation` | medium | POST/PUT without validation |
|
|
394
|
-
| `api-no-auth` | low | Routes without auth |
|
|
395
|
-
| `api-hardcoded-secret` | critical | Hardcoded API keys |
|
|
396
|
-
|
|
397
|
-
Set any to `'off'` to disable.
|
|
287
|
+
<summary><b>Windsurf</b></summary>
|
|
398
288
|
|
|
399
|
-
|
|
289
|
+
Add to `~/.codeium/windsurf/mcp_config.json`:
|
|
400
290
|
|
|
401
|
-
|
|
291
|
+
```json
|
|
292
|
+
{ "mcpServers": { "sniff-qa": { "command": "npx", "args": ["-y", "sniff-qa", "--mcp"] } } }
|
|
293
|
+
```
|
|
294
|
+
</details>
|
|
402
295
|
|
|
403
|
-
|
|
296
|
+
<details>
|
|
297
|
+
<summary><b>Continue.dev</b></summary>
|
|
404
298
|
|
|
405
|
-
|
|
299
|
+
Add to `.continue/mcpServers/sniff-qa.yaml`:
|
|
406
300
|
|
|
407
|
-
```
|
|
408
|
-
|
|
301
|
+
```yaml
|
|
302
|
+
mcpServers:
|
|
303
|
+
sniff-qa: { command: npx, args: ["-y", "sniff-qa", "--mcp"], type: stdio }
|
|
409
304
|
```
|
|
305
|
+
</details>
|
|
410
306
|
|
|
411
|
-
|
|
307
|
+
> The first browser-based walk downloads Chromium (~165 MB). Over MCP, Sniff returns a structured `needsSetup` payload instead of blocking the editor on a long download — run the install it shows you, then ask again.
|
|
412
308
|
|
|
413
|
-
|
|
414
|
-
npx sniff-qa ci
|
|
415
|
-
```
|
|
309
|
+
---
|
|
416
310
|
|
|
417
|
-
|
|
311
|
+
## How it works
|
|
418
312
|
|
|
419
|
-
**
|
|
313
|
+
1. **Find the app.** Sniff auto-detects your running dev server (or you pass `--url`).
|
|
314
|
+
2. **Walk the flows.** It opens pages in a headless browser and interacts with them like a user — clicking, filling forms, following links — across desktop and a 375px mobile pass.
|
|
315
|
+
3. **Watch everything.** It records console errors, failed network requests, broken renders, missing feedback, and accessibility issues as it goes.
|
|
316
|
+
4. **Filter the noise.** The first-party noise filter and axe-core drop the false positives; uncertain findings are held back.
|
|
317
|
+
5. **Report with proof.** Each surviving finding gets a severity, a confidence, reproduction steps, a screenshot, and a suggested fix — in the terminal and an optional HTML report.
|
|
318
|
+
|
|
319
|
+
<img alt="How sniff works: crawl, act, assert, prove, report" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/how-it-works.svg" width="100%">
|
|
420
320
|
|
|
421
321
|
---
|
|
422
322
|
|
|
423
|
-
##
|
|
323
|
+
## CI integration
|
|
324
|
+
|
|
325
|
+
Run Sniff in your pipeline and fail the build on real bugs:
|
|
424
326
|
|
|
425
327
|
```bash
|
|
426
|
-
sniff
|
|
427
|
-
sniff discover --regenerate-only # write sniff-scenarios/ and exit (no run)
|
|
328
|
+
npx sniff-qa --ci --fail-on high
|
|
428
329
|
```
|
|
429
330
|
|
|
430
|
-
|
|
331
|
+
Generate a ready-to-commit GitHub Actions workflow:
|
|
431
332
|
|
|
432
|
-
|
|
333
|
+
```bash
|
|
334
|
+
npx sniff-qa ci
|
|
335
|
+
```
|
|
433
336
|
|
|
434
|
-
|
|
337
|
+
This writes `.github/workflows/sniff.yml` with browser caching and report artifacts.
|
|
435
338
|
|
|
436
|
-
|
|
339
|
+
---
|
|
437
340
|
|
|
438
|
-
|
|
439
|
-
- `sniff-reports/discovery/` — HTML, JSON, and JUnit reports per run.
|
|
440
|
-
- Flakiness quarantine for scenarios that fail 3 of the last 5 runs.
|
|
341
|
+
## FAQ
|
|
441
342
|
|
|
442
|
-
**
|
|
343
|
+
**Does it work without a dev server?**
|
|
344
|
+
Sniff is built to walk a *running* app, so that's where it shines. If no server is running, it doesn't fail silently — it runs a source-only scan and tells you exactly how to start the real walk (start your dev server or pass `--url`). You can also run `npx sniff-qa scan` to get the source scan on purpose.
|
|
443
345
|
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
| `--url <url>` | Target URL (default: auto-detect dev server) |
|
|
447
|
-
| `--max-scenarios <n>` | Cap total scenarios (default: 50) |
|
|
448
|
-
| `--max-variants-per-scenario <n>` | Cap edge variants per scenario (default: 3) |
|
|
449
|
-
| `--max-variants-per-run <n>` | Cap edge variants per run (default: 40) |
|
|
450
|
-
| `--realism <profile>` | `robot`, `careful-user`, `casual-user` (default), `frustrated-user`, `power-user` |
|
|
451
|
-
| `--seed <n>` | Replay a specific random seed |
|
|
452
|
-
| `--only <filter>` | Filter scenarios by id substring or app type |
|
|
453
|
-
| `--app-type <types>` | Filter classifier guesses to these app types (comma-separated, kept for back-compat) |
|
|
454
|
-
| `--force-app-type <type>` | Bypass the classifier entirely — treat the app as this type |
|
|
455
|
-
| `--verbose` | Print classifier breakdown (top 3 guesses + matched signals per dimension) |
|
|
456
|
-
| `--dry-run` | Generate scenarios without launching the browser or writing reports |
|
|
457
|
-
| `--regenerate` | Regenerate `sniff-scenarios/` before running |
|
|
458
|
-
| `--regenerate-only` | Regenerate and exit (no browser) |
|
|
459
|
-
| `--force-regenerate` | Overwrite hand-edits without prompting |
|
|
460
|
-
| `--non-interactive` | Skip prompts and the production-URL countdown |
|
|
461
|
-
| `--verbose` | Print classification breakdown (top 3 guesses + matched signals per dimension) |
|
|
462
|
-
| `--no-llm` | Skip Claude Code CLI polish (deterministic only) |
|
|
463
|
-
|
|
464
|
-
**Production-URL safety:** when the target is not a local address, sniff prints a 5-second countdown banner before running. Hit Ctrl+C to cancel. No `--allow-destructive` flag.
|
|
465
|
-
|
|
466
|
-
**LLM polish (optional):** discovery can use the `claude` CLI to break close-call app-type classifications. Responses are cached under `.sniff/discover/cache/`. Pass `--no-llm` to skip.
|
|
346
|
+
**What gets downloaded on first run?**
|
|
347
|
+
The first time Sniff opens a browser, it downloads a Chromium build (~165 MB, one-time, then cached). You'll see the progress, and you need internet access for that first run. Nothing else is installed and no account is created.
|
|
467
348
|
|
|
468
|
-
|
|
349
|
+
**Do I need an API key?**
|
|
350
|
+
No. Sniff runs entirely on your machine with no API key and no signup. Your code and your app never leave your computer.
|
|
469
351
|
|
|
470
|
-
|
|
352
|
+
**How is it different from a linter?**
|
|
353
|
+
A linter reads your source files and never runs your app, so it can't see a dead submit button, an infinite spinner, a wiped form, or a 500 page. Sniff opens your real app, interacts with it, and reports what actually broke — with a screenshot and steps to reproduce.
|
|
471
354
|
|
|
472
|
-
|
|
473
|
-
|
|
474
|
-
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/pipeline-light.svg">
|
|
475
|
-
<img alt="Sniff pipeline" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/pipeline-light.svg" width="100%">
|
|
476
|
-
</picture>
|
|
355
|
+
**How is it different from Playwright codegen (or writing E2E tests)?**
|
|
356
|
+
Playwright codegen records a script that *you* author and maintain; it tests only the path you clicked. Sniff writes nothing for you to maintain — it explores your flows on its own and judges the outcome, catching things a recorded happy-path never checks (empty/placeholder data, state-loss, missing success feedback).
|
|
477
357
|
|
|
478
|
-
|
|
479
|
-
|
|
358
|
+
**Will it change my code?**
|
|
359
|
+
No, not during a walk. Walking and scanning are read-only. The separate `sniff fix` command applies safe auto-fixes (like stray `console.log`/`debugger`) and only when you run it.
|
|
480
360
|
|
|
481
|
-
|
|
482
|
-
|
|
483
|
-
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/architecture-light.svg">
|
|
484
|
-
<img alt="Sniff architecture" src="https://raw.githubusercontent.com/Aboudjem/sniff/main/.github/assets/architecture-light.svg" width="100%">
|
|
485
|
-
</picture>
|
|
486
|
-
|
|
487
|
-
</details>
|
|
361
|
+
**What stacks does it work with?**
|
|
362
|
+
Any web app you can open in a browser — React, Next.js, Vue, Svelte, Angular, Remix, SvelteKit, Astro, plain HTML, and more. It walks the rendered app, so the framework doesn't matter for the browser checks.
|
|
488
363
|
|
|
489
364
|
---
|
|
490
365
|
|
|
491
|
-
##
|
|
492
|
-
|
|
493
|
-
No telemetry. No signup. No data collection. No API keys required. Your code stays on your machine unless you explicitly configure an API-backed AI provider.
|
|
366
|
+
## Works first-class in
|
|
494
367
|
|
|
495
|
-
|
|
496
|
-
> All deterministic checks work without any API key. Optional AI providers are off by default and must be selected through config or environment. Dead link checking validates external URLs but never sends your code.
|
|
368
|
+
Claude Code · Cursor · VS Code (Copilot) · Codex · Gemini CLI · Windsurf · Continue.dev — via the MCP server (command `npx`, args `["-y", "sniff-qa", "--mcp"]`) or the CLI directly.
|
|
497
369
|
|
|
498
370
|
---
|
|
499
371
|
|
|
500
372
|
## Contributing
|
|
501
373
|
|
|
502
|
-
|
|
374
|
+
Issues and PRs welcome. See [CONTRIBUTING.md](CONTRIBUTING.md).
|
|
503
375
|
|
|
504
376
|
---
|
|
505
377
|
|
|
506
378
|
<p align="center">
|
|
507
379
|
<sub>
|
|
508
|
-
Built on <a href="https://playwright.dev">Playwright</a> · <a href="https://github.com/dequelabs/axe-core">axe-core</a> · <a href="https://developer.chrome.com/docs/lighthouse">Lighthouse</a> · <a href="https://github.com/mapbox/pixelmatch">pixelmatch</a> · <a href="https://zod.dev">Zod</a> · <a href="https://github.com/modelcontextprotocol/typescript-sdk">MCP SDK</a>
|
|
380
|
+
Built on <a href="https://playwright.dev">Playwright</a> · <a href="https://github.com/dequelabs/axe-core">axe-core</a> · <a href="https://developer.chrome.com/docs/lighthouse">Lighthouse</a> · <a href="https://github.com/mapbox/pixelmatch">pixelmatch</a> · <a href="https://zod.dev">Zod</a> · <a href="https://github.com/modelcontextprotocol/typescript-sdk">MCP SDK</a>
|
|
509
381
|
</sub>
|
|
510
382
|
</p>
|
|
511
383
|
|