sniff-qa 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +220 -411
- package/dist/chunk-VEI2O6AK.js +34 -0
- package/dist/chunk-VEI2O6AK.js.map +1 -0
- package/dist/cli/index.js +67 -4
- package/dist/cli/index.js.map +1 -1
- package/dist/dev-server-detector-NJ5RN4IC.js +150 -0
- package/dist/dev-server-detector-NJ5RN4IC.js.map +1 -0
- package/dist/fix-R33D34NP.js +110 -0
- package/dist/fix-R33D34NP.js.map +1 -0
- package/dist/mcp/server.js +22 -7
- package/dist/mcp/server.js.map +1 -1
- package/dist/{model-SRPWXU63.js → model-2DTSAX7T.js} +6 -6
- package/dist/model-2DTSAX7T.js.map +1 -0
- package/dist/{repo-analyzer-4OURSITY.js → repo-analyzer-WK5WP7NT.js} +1 -1
- package/dist/{report-E2JZMQED.js → report-WLDV72FM.js} +2 -2
- package/dist/{source-D5KLH46F.js → source-4DOO224X.js} +3 -3
- package/dist/source-4DOO224X.js.map +1 -0
- package/dist/{unified-CQU7PSNJ.js → unified-7RKNG3EI.js} +6 -5
- package/dist/unified-7RKNG3EI.js.map +1 -0
- package/dist/version-FLE7ZHDX.js +7 -0
- package/dist/version-FLE7ZHDX.js.map +1 -0
- package/package.json +1 -1
- package/dist/model-SRPWXU63.js.map +0 -1
- package/dist/source-D5KLH46F.js.map +0 -1
- package/dist/unified-CQU7PSNJ.js.map +0 -1
- /package/dist/{repo-analyzer-4OURSITY.js.map → repo-analyzer-WK5WP7NT.js.map} +0 -0
- /package/dist/{report-E2JZMQED.js.map → report-WLDV72FM.js.map} +0 -0
package/README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
<picture>
|
|
2
2
|
<source media="(prefers-color-scheme: dark)" srcset=".github/assets/logo-dark.svg">
|
|
3
3
|
<source media="(prefers-color-scheme: light)" srcset=".github/assets/logo-light.svg">
|
|
4
|
-
<img alt="Sniff
|
|
4
|
+
<img alt="Sniff" src=".github/assets/logo-light.svg" width="100%">
|
|
5
5
|
</picture>
|
|
6
6
|
|
|
7
7
|
<p align="center">
|
|
@@ -12,259 +12,33 @@
|
|
|
12
12
|
<a href="https://github.com/Aboudjem/sniff/stargazers"><img src="https://img.shields.io/github/stars/Aboudjem/sniff?style=flat-square&color=ef4444" alt="Stars"></a>
|
|
13
13
|
</p>
|
|
14
14
|
|
|
15
|
-
<p align="center">
|
|
16
|
-
<b>One command. Eight checks. Zero config.</b><br/>
|
|
17
|
-
<sub>Source bugs, dead links, API endpoints, accessibility, visual regression, performance, AI exploration, and cross-referencing. All in one pass.</sub>
|
|
18
|
-
</p>
|
|
15
|
+
<p align="center">Scan your source code, your live site, or both. Finds bugs before your users do.</p>
|
|
19
16
|
|
|
20
17
|
---
|
|
21
18
|
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
## What is this?
|
|
25
|
-
|
|
26
|
-
You ship features. But did you actually *test* them?
|
|
19
|
+
## Get started
|
|
27
20
|
|
|
28
|
-
|
|
21
|
+
### Use from the terminal
|
|
29
22
|
|
|
30
23
|
```bash
|
|
31
|
-
|
|
24
|
+
cd ~/projects/my-app # go to your project
|
|
25
|
+
npx sniff-qa # that's it
|
|
32
26
|
```
|
|
33
27
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
<br/>
|
|
37
|
-
|
|
38
|
-
### Key facts
|
|
39
|
-
|
|
40
|
-
- **What:** Open source AI-powered QA testing tool that runs 8 checks in a single command: source scanning (6 rule categories + dead links + API endpoint discovery), accessibility audits, visual regression, performance budgets, AI-driven exploration, and source/browser cross-referencing
|
|
41
|
-
- **Who it's for:** Frontend and full-stack developers who want automated QA without configuring multiple tools
|
|
42
|
-
- **How it works:** Uses Playwright for browser automation, axe-core for WCAG accessibility, Lighthouse for performance, pixelmatch for visual diffs, and Anthropic Claude for adversarial AI exploration
|
|
43
|
-
- **Install:** `npx sniff-qa` (zero config, no signup, no API keys required for base checks)
|
|
44
|
-
- **Comparable to:** Combines Lighthouse CI + Pa11y + BackstopJS into one tool, plus source code scanning and AI-driven edge case discovery that no other tool offers
|
|
45
|
-
- **Unique:** First QA tool with native MCP (Model Context Protocol) integration. Flakiness quarantine system isolates non-deterministic test results automatically
|
|
46
|
-
|
|
47
|
-
<br/>
|
|
48
|
-
|
|
49
|
-
## Quick start
|
|
50
|
-
|
|
51
|
-
| Step | Action |
|
|
52
|
-
|:----:|:-------|
|
|
53
|
-
| **1** | Run a source scan: `npx sniff-qa` |
|
|
54
|
-
| **2** | Run the full audit: `npx sniff-qa --url http://localhost:3000` |
|
|
55
|
-
| **3** | Read the report and fix what matters |
|
|
56
|
-
|
|
57
|
-
The source scan finishes in seconds. The full audit adds accessibility, visual regression, performance, and AI exploration on top of it.
|
|
58
|
-
|
|
59
|
-
> [!TIP]
|
|
60
|
-
> Drop your URL in `sniff.config.ts` once and you can just run `sniff` for the full audit every time.
|
|
61
|
-
|
|
62
|
-
<br/>
|
|
63
|
-
|
|
64
|
-
## The three modes
|
|
28
|
+
Sniff auto-detects everything: your framework, your dev server port, and your running app. If `npm run dev` is running, sniff finds it and runs browser checks too. No flags needed.
|
|
65
29
|
|
|
66
|
-
|
|
67
|
-
graph LR
|
|
68
|
-
A["🔍 Quick Scan<br/><sub>npx sniff-qa</sub>"] --> B["Source bugs only<br/><sub>No browser needed</sub>"]
|
|
69
|
-
C["🛡️ Full Audit<br/><sub>npx sniff-qa --url URL</sub>"] --> D["All 8 checks<br/><sub>Source + Browser + Xref</sub>"]
|
|
70
|
-
E["🏗️ CI Mode<br/><sub>npx sniff-qa --url URL --ci</sub>"] --> F["Deterministic<br/><sub>Skips AI explorer</sub>"]
|
|
71
|
-
|
|
72
|
-
style A fill:#fef2f2,stroke:#ef4444,color:#7f1d1d
|
|
73
|
-
style B fill:#fee2e2,stroke:#ef4444,color:#7f1d1d
|
|
74
|
-
style C fill:#fef2f2,stroke:#ef4444,color:#7f1d1d
|
|
75
|
-
style D fill:#fee2e2,stroke:#ef4444,color:#7f1d1d
|
|
76
|
-
style E fill:#fef2f2,stroke:#ef4444,color:#7f1d1d
|
|
77
|
-
style F fill:#fee2e2,stroke:#ef4444,color:#7f1d1d
|
|
78
|
-
```
|
|
30
|
+
You can also be explicit:
|
|
79
31
|
|
|
80
32
|
```bash
|
|
81
|
-
npx sniff-qa
|
|
82
|
-
npx sniff-qa --url
|
|
83
|
-
npx sniff-qa --url http://localhost:3000 --ci # ci mode (skips AI explorer)
|
|
84
|
-
```
|
|
85
|
-
|
|
86
|
-
CI mode auto-skips the AI explorer because it is non-deterministic. Everything else runs exactly the same.
|
|
87
|
-
|
|
88
|
-
<br/>
|
|
89
|
-
|
|
90
|
-
## What it checks
|
|
91
|
-
|
|
92
|
-
| | Check | Example finding | Details |
|
|
93
|
-
|:--|:------|:----------------|:--------|
|
|
94
|
-
| 📄 | **Source code** | `debugger` statement in `handler.ts:42` | Leftover debugger, placeholder text, hardcoded URLs, broken imports, TODO/FIXME |
|
|
95
|
-
| 🔗 | **Dead links** | Broken link `./guide.md` in `README.md:28` | Validates internal file refs, external URLs (HTTP HEAD with retry), and anchor links. Catches 404s before your users do |
|
|
96
|
-
| 🛣️ | **API endpoints** | `POST /users` has no input validation | Discovers routes from Express, Fastify, Hono, Next.js, SvelteKit, tRPC, and GraphQL. Flags missing error handling, auth, and hardcoded secrets |
|
|
97
|
-
| ♿ | **Accessibility** | Missing form label on `/login` | WCAG 2.x violations via [axe-core](https://github.com/dequelabs/axe-core) with exact fix guidance |
|
|
98
|
-
| 🖼 | **Visual regression** | `/pricing` changed 2.3% of pixels | Local pixel diffing via [pixelmatch](https://github.com/mapbox/pixelmatch). Commit baselines to track changes |
|
|
99
|
-
| ⚡ | **Performance** | LCP 4200ms on `/dashboard` (budget: 2500ms) | [Lighthouse](https://developer.chrome.com/docs/lighthouse) budgets for LCP, FCP, TTI |
|
|
100
|
-
| 🤖 | **AI explorer** | XSS in `/signup` email field crashes app | Roams your app, fills forms with adversarial inputs, reports crashes and console errors |
|
|
101
|
-
| 🔀 | **Cross-reference** | `console.log` in source confirmed at runtime | Correlates source findings with browser evidence. Corroborated issues get bumped severity and HIGH confidence tags |
|
|
102
|
-
|
|
103
|
-
<br/>
|
|
104
|
-
|
|
105
|
-
## How it works
|
|
106
|
-
|
|
107
|
-
<picture>
|
|
108
|
-
<source media="(prefers-color-scheme: dark)" srcset=".github/assets/pipeline-dark.svg">
|
|
109
|
-
<source media="(prefers-color-scheme: light)" srcset=".github/assets/pipeline-light.svg">
|
|
110
|
-
<img alt="Sniff pipeline showing 8 checks flowing through source scanner, browser runner, cross-reference engine, and report output" src=".github/assets/pipeline-light.svg" width="100%">
|
|
111
|
-
</picture>
|
|
112
|
-
|
|
113
|
-
Your code goes in. A report with actionable findings comes out. When you provide a URL, sniff also runs browser checks and cross-references source findings with runtime behavior for high-confidence results. You don't configure anything.
|
|
114
|
-
|
|
115
|
-
<br/>
|
|
116
|
-
|
|
117
|
-
## Compared to
|
|
118
|
-
|
|
119
|
-
| | **Sniff** | Lighthouse CI | Pa11y | BackstopJS |
|
|
120
|
-
|:--|:--:|:--:|:--:|:--:|
|
|
121
|
-
| Source scanning | ✅ | ❌ | ❌ | ❌ |
|
|
122
|
-
| Dead link checking | ✅ | ❌ | ❌ | ❌ |
|
|
123
|
-
| API endpoint discovery | ✅ | ❌ | ❌ | ❌ |
|
|
124
|
-
| Accessibility | ✅ | partial | ✅ | ❌ |
|
|
125
|
-
| Visual regression | ✅ | ❌ | ❌ | ✅ |
|
|
126
|
-
| Performance | ✅ | ✅ | ❌ | ❌ |
|
|
127
|
-
| AI exploration | ✅ | ❌ | ❌ | ❌ |
|
|
128
|
-
| Cross-reference engine | ✅ | ❌ | ❌ | ❌ |
|
|
129
|
-
| Flakiness quarantine | ✅ | ❌ | ❌ | ❌ |
|
|
130
|
-
| Single command | ✅ | ❌ | ❌ | ❌ |
|
|
131
|
-
| MCP server | ✅ | ❌ | ❌ | ❌ |
|
|
132
|
-
| Zero config | ✅ | ❌ | ❌ | ❌ |
|
|
133
|
-
|
|
134
|
-
> [!IMPORTANT]
|
|
135
|
-
> Sniff is the **first QA tool with native MCP integration**. Your AI editor can trigger scans, read results, and fix issues without you switching context.
|
|
136
|
-
|
|
137
|
-
<br/>
|
|
138
|
-
|
|
139
|
-
## Source scanning rules
|
|
140
|
-
|
|
141
|
-
Sniff ships 6 rule categories that run on every scan. No browser needed.
|
|
142
|
-
|
|
143
|
-
| Rule | ID | Severity | What it catches |
|
|
144
|
-
|:-----|:---|:---------|:----------------|
|
|
145
|
-
| Debug statements | `debug-console-log`, `debug-debugger` | medium / high | `console.log`, `console.debug`, `debugger` left in production code |
|
|
146
|
-
| Placeholder text | `placeholder-lorem`, `placeholder-todo`, `placeholder-fixme`, `placeholder-tbd` | high / medium | Lorem ipsum, TODO, FIXME, TBD markers |
|
|
147
|
-
| Hardcoded URLs | `hardcoded-localhost`, `hardcoded-127` | medium | `http://localhost:*` and `http://127.0.0.1:*` in non-test files |
|
|
148
|
-
| Broken imports | `broken-import` | medium | Relative imports that don't resolve to a file (with TS extension mapping) |
|
|
149
|
-
| Dead links | `dead-link-internal`, `dead-link-external`, `dead-link-anchor` | high / medium / low | Broken file references, 404 external URLs, missing anchor targets |
|
|
150
|
-
| API endpoints | `api-no-error-handling`, `api-no-validation`, `api-no-auth`, `api-hardcoded-secret` | critical / medium / low | Route handlers missing try/catch, input validation, auth, or containing hardcoded secrets |
|
|
151
|
-
|
|
152
|
-
### Dead link checker
|
|
153
|
-
|
|
154
|
-
Scans `.md`, `.html`, `.jsx`, `.tsx`, `.vue`, `.svelte`, and `.astro` files for links and validates them:
|
|
155
|
-
|
|
156
|
-
- **Internal links:** Checks the target file exists (with extension resolution and index file fallback)
|
|
157
|
-
- **External links:** Sends HTTP HEAD requests with timeout, retry, and GET fallback for servers that reject HEAD
|
|
158
|
-
- **Anchor links:** Validates `#section` references against markdown headings and HTML `id`/`name` attributes
|
|
159
|
-
- **Smart skipping:** Ignores `mailto:`, `tel:`, `data:`, `javascript:`, and template variables
|
|
160
|
-
|
|
161
|
-
```
|
|
162
|
-
HIGH README.md:28 Broken internal link: ./missing-guide.md
|
|
163
|
-
MED docs/api.md:15 HTTP 404: https://example.com/old-endpoint
|
|
164
|
-
MED CONTRIBUTING.md:8 Anchor #setup-guide not found in ./README.md
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
### API endpoint discovery
|
|
168
|
-
|
|
169
|
-
Automatically discovers API routes from 8 frameworks:
|
|
170
|
-
|
|
171
|
-
| Framework | Detection method |
|
|
172
|
-
|:----------|:----------------|
|
|
173
|
-
| Express | `app.get()`, `router.post()`, etc. |
|
|
174
|
-
| Fastify | `fastify.get()`, `fastify.post()`, etc. |
|
|
175
|
-
| Hono | `app.get()` with `new Hono()` detection |
|
|
176
|
-
| Next.js App Router | `app/**/route.ts` with exported `GET`/`POST` handlers |
|
|
177
|
-
| Next.js Pages Router | `pages/api/**/*.ts` |
|
|
178
|
-
| SvelteKit | `routes/**/+server.ts` |
|
|
179
|
-
| tRPC | `publicProcedure.query()` / `.mutation()` |
|
|
180
|
-
| GraphQL | `type Query { }` / `type Mutation { }` definitions |
|
|
181
|
-
|
|
182
|
-
For each endpoint, sniff checks for:
|
|
183
|
-
- Missing error handling (no try/catch or error middleware)
|
|
184
|
-
- Missing input validation on POST/PUT/PATCH routes (no zod, joi, yup, etc.)
|
|
185
|
-
- Missing auth middleware on non-public routes
|
|
186
|
-
- Hardcoded secrets (API keys, tokens, Stripe keys, GitHub tokens)
|
|
187
|
-
|
|
188
|
-
```
|
|
189
|
-
CRIT src/routes.ts:12 Hardcoded Stripe key in POST /checkout handler
|
|
190
|
-
MED src/api.ts:5 POST /users has no input validation
|
|
191
|
-
MED src/api.ts:18 GET /admin/data has no error handling
|
|
192
|
-
LOW src/api.ts:25 GET /admin/data has no visible auth middleware
|
|
193
|
-
INFO Discovered 6 API endpoints (express: 4, nextjs-app: 2)
|
|
194
|
-
```
|
|
195
|
-
|
|
196
|
-
### Cross-reference engine
|
|
197
|
-
|
|
198
|
-
When you run sniff with `--url`, the cross-reference engine correlates source code findings with browser runtime behavior. Findings confirmed in both layers get bumped severity and a HIGH confidence tag.
|
|
199
|
-
|
|
200
|
-
**5 correlation strategies:**
|
|
201
|
-
|
|
202
|
-
| Source finding | Browser evidence | Correlation |
|
|
203
|
-
|:---------------|:-----------------|:------------|
|
|
204
|
-
| Broken import / dead link | 404 network request | `broken-import-to-404` |
|
|
205
|
-
| `console.log` statement | Console output captured | `console-log-to-runtime` |
|
|
206
|
-
| Hardcoded localhost URL | Network request to that URL | `hardcoded-url-to-network` |
|
|
207
|
-
| Missing label / a11y issue | axe-core violation | `source-a11y-to-axe` |
|
|
208
|
-
| Placeholder text in source | Text visible on page | `placeholder-to-runtime` |
|
|
209
|
-
|
|
33
|
+
npx sniff-qa --url http://localhost:3000 # specific local URL
|
|
34
|
+
npx sniff-qa --url https://myapp.com # production URL
|
|
210
35
|
```
|
|
211
|
-
CORROBORATED (source + browser evidence)
|
|
212
|
-
src/handler.ts:42 console.log("debug data")
|
|
213
|
-
Source: debug-console-log rule triggered
|
|
214
|
-
Browser: console.log output captured at /dashboard
|
|
215
|
-
Confidence: HIGH (confirmed in both layers)
|
|
216
|
-
```
|
|
217
|
-
|
|
218
|
-
<br/>
|
|
219
|
-
|
|
220
|
-
## Architecture
|
|
221
|
-
|
|
222
|
-
<picture>
|
|
223
|
-
<source media="(prefers-color-scheme: dark)" srcset=".github/assets/architecture-dark.svg">
|
|
224
|
-
<source media="(prefers-color-scheme: light)" srcset=".github/assets/architecture-light.svg">
|
|
225
|
-
<img alt="Sniff architecture showing source scanner, browser runner, cross-reference engine, and report engine" src=".github/assets/architecture-light.svg" width="100%">
|
|
226
|
-
</picture>
|
|
227
|
-
|
|
228
|
-
<br/>
|
|
229
|
-
|
|
230
|
-
## Example output
|
|
231
|
-
|
|
232
|
-
<picture>
|
|
233
|
-
<source media="(prefers-color-scheme: dark)" srcset=".github/assets/report-dark.svg">
|
|
234
|
-
<source media="(prefers-color-scheme: light)" srcset=".github/assets/report-light.svg">
|
|
235
|
-
<img alt="Sniff terminal output showing source findings, browser results, and cross-referenced corroborated findings" src=".github/assets/report-light.svg" width="100%">
|
|
236
|
-
</picture>
|
|
237
|
-
|
|
238
|
-
<br/>
|
|
239
|
-
|
|
240
|
-
## Works with your stack
|
|
241
|
-
|
|
242
|
-
No config needed. Sniff auto-detects your framework from `package.json` and source files.
|
|
243
|
-
|
|
244
|
-
<table>
|
|
245
|
-
<tr>
|
|
246
|
-
<td align="center" width="16%"><b>⚛️ React</b></td>
|
|
247
|
-
<td align="center" width="16%"><b>▲ Next.js</b></td>
|
|
248
|
-
<td align="center" width="16%"><b>💚 Vue</b></td>
|
|
249
|
-
<td align="center" width="16%"><b>🔶 Svelte</b></td>
|
|
250
|
-
<td align="center" width="16%"><b>🅰️ Angular</b></td>
|
|
251
|
-
<td align="center" width="16%"><b>🌐 Vanilla</b></td>
|
|
252
|
-
</tr>
|
|
253
|
-
<tr>
|
|
254
|
-
<td align="center">JSX / TSX</td>
|
|
255
|
-
<td align="center">App Router</td>
|
|
256
|
-
<td align="center">SFC</td>
|
|
257
|
-
<td align="center">Components</td>
|
|
258
|
-
<td align="center">Templates</td>
|
|
259
|
-
<td align="center">HTML / CSS</td>
|
|
260
|
-
</tr>
|
|
261
|
-
</table>
|
|
262
36
|
|
|
263
|
-
|
|
37
|
+
> **No API keys. No Playwright install. No config files.** Everything works out of the box. Browser checks auto-install Chromium on first run.
|
|
264
38
|
|
|
265
|
-
|
|
39
|
+
### Use from your AI editor
|
|
266
40
|
|
|
267
|
-
Sniff ships an MCP server.
|
|
41
|
+
Sniff ships as an MCP server. Add it to your editor, then ask your AI to scan.
|
|
268
42
|
|
|
269
43
|
<details>
|
|
270
44
|
<summary><b>Claude Code</b></summary>
|
|
@@ -272,51 +46,24 @@ Sniff ships an MCP server. Pick your tool and copy the snippet.
|
|
|
272
46
|
```bash
|
|
273
47
|
claude mcp add sniff-qa npx sniff-qa --mcp
|
|
274
48
|
```
|
|
275
|
-
|
|
276
|
-
Or `.mcp.json`:
|
|
277
|
-
|
|
278
|
-
```json
|
|
279
|
-
{
|
|
280
|
-
"mcpServers": {
|
|
281
|
-
"sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] }
|
|
282
|
-
}
|
|
283
|
-
}
|
|
284
|
-
```
|
|
285
|
-
|
|
286
49
|
</details>
|
|
287
50
|
|
|
288
51
|
<details>
|
|
289
52
|
<summary><b>Cursor</b></summary>
|
|
290
53
|
|
|
291
|
-
|
|
292
|
-
|
|
54
|
+
Add to `~/.cursor/mcp.json`:
|
|
293
55
|
```json
|
|
294
|
-
{
|
|
295
|
-
"mcpServers": {
|
|
296
|
-
"sniff-qa": {
|
|
297
|
-
"type": "stdio",
|
|
298
|
-
"command": "npx",
|
|
299
|
-
"args": ["sniff-qa", "--mcp"]
|
|
300
|
-
}
|
|
301
|
-
}
|
|
302
|
-
}
|
|
56
|
+
{ "mcpServers": { "sniff-qa": { "type": "stdio", "command": "npx", "args": ["sniff-qa", "--mcp"] } } }
|
|
303
57
|
```
|
|
304
|
-
|
|
305
58
|
</details>
|
|
306
59
|
|
|
307
60
|
<details>
|
|
308
|
-
<summary><b>
|
|
309
|
-
|
|
310
|
-
`~/.codeium/windsurf/mcp_config.json`:
|
|
61
|
+
<summary><b>VS Code + Copilot</b></summary>
|
|
311
62
|
|
|
63
|
+
Add to `.vscode/mcp.json`:
|
|
312
64
|
```json
|
|
313
|
-
{
|
|
314
|
-
"mcpServers": {
|
|
315
|
-
"sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] }
|
|
316
|
-
}
|
|
317
|
-
}
|
|
65
|
+
{ "servers": { "sniff-qa": { "type": "stdio", "command": "npx", "args": ["-y", "sniff-qa", "--mcp"] } } }
|
|
318
66
|
```
|
|
319
|
-
|
|
320
67
|
</details>
|
|
321
68
|
|
|
322
69
|
<details>
|
|
@@ -325,64 +72,34 @@ Or `.mcp.json`:
|
|
|
325
72
|
```bash
|
|
326
73
|
codex mcp add sniff-qa -- npx -y sniff-qa --mcp
|
|
327
74
|
```
|
|
75
|
+
</details>
|
|
328
76
|
|
|
329
|
-
|
|
77
|
+
<details>
|
|
78
|
+
<summary><b>Windsurf</b></summary>
|
|
330
79
|
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
args = ["-y", "sniff-qa", "--mcp"]
|
|
80
|
+
Add to `~/.codeium/windsurf/mcp_config.json`:
|
|
81
|
+
```json
|
|
82
|
+
{ "mcpServers": { "sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] } } }
|
|
335
83
|
```
|
|
336
|
-
|
|
337
84
|
</details>
|
|
338
85
|
|
|
339
86
|
<details>
|
|
340
87
|
<summary><b>Gemini CLI</b></summary>
|
|
341
88
|
|
|
342
|
-
`~/.gemini/mcp_config.json`:
|
|
343
|
-
|
|
89
|
+
Add to `~/.gemini/mcp_config.json`:
|
|
344
90
|
```json
|
|
345
|
-
{
|
|
346
|
-
"mcpServers": {
|
|
347
|
-
"sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] }
|
|
348
|
-
}
|
|
349
|
-
}
|
|
91
|
+
{ "mcpServers": { "sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] } } }
|
|
350
92
|
```
|
|
351
|
-
|
|
352
93
|
</details>
|
|
353
94
|
|
|
354
95
|
<details>
|
|
355
96
|
<summary><b>Continue.dev</b></summary>
|
|
356
97
|
|
|
357
|
-
`.continue/mcpServers/sniff-qa.yaml`:
|
|
358
|
-
|
|
98
|
+
Add to `.continue/mcpServers/sniff-qa.yaml`:
|
|
359
99
|
```yaml
|
|
360
100
|
mcpServers:
|
|
361
|
-
sniff-qa:
|
|
362
|
-
command: npx
|
|
363
|
-
args: [sniff-qa, --mcp]
|
|
364
|
-
type: stdio
|
|
101
|
+
sniff-qa: { command: npx, args: [sniff-qa, --mcp], type: stdio }
|
|
365
102
|
```
|
|
366
|
-
|
|
367
|
-
</details>
|
|
368
|
-
|
|
369
|
-
<details>
|
|
370
|
-
<summary><b>VS Code + Copilot</b></summary>
|
|
371
|
-
|
|
372
|
-
Add to `.vscode/mcp.json`:
|
|
373
|
-
|
|
374
|
-
```json
|
|
375
|
-
{
|
|
376
|
-
"servers": {
|
|
377
|
-
"sniff-qa": {
|
|
378
|
-
"type": "stdio",
|
|
379
|
-
"command": "npx",
|
|
380
|
-
"args": ["-y", "sniff-qa", "--mcp"]
|
|
381
|
-
}
|
|
382
|
-
}
|
|
383
|
-
}
|
|
384
|
-
```
|
|
385
|
-
|
|
386
103
|
</details>
|
|
387
104
|
|
|
388
105
|
<details>
|
|
@@ -391,29 +108,13 @@ Add to `.vscode/mcp.json`:
|
|
|
391
108
|
```bash
|
|
392
109
|
clawhub install sniff-qa
|
|
393
110
|
```
|
|
394
|
-
|
|
395
|
-
Or add to `~/.openclaw/openclaw.json`:
|
|
396
|
-
|
|
397
|
-
```json
|
|
398
|
-
{
|
|
399
|
-
"mcpServers": {
|
|
400
|
-
"sniff-qa": {
|
|
401
|
-
"command": "npx",
|
|
402
|
-
"args": ["-y", "sniff-qa", "--mcp"]
|
|
403
|
-
}
|
|
404
|
-
}
|
|
405
|
-
}
|
|
406
|
-
```
|
|
407
|
-
|
|
408
111
|
</details>
|
|
409
112
|
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
**MCP tools exposed:** `sniff_scan` (source), `sniff_run` (browser), `sniff_report` (last results).
|
|
113
|
+
Then ask: *"Scan this project for issues"* or *"Check accessibility on localhost:3000"*
|
|
413
114
|
|
|
414
|
-
|
|
115
|
+
**MCP tools:** `sniff_scan` (source) · `sniff_run` (browser) · `sniff_report` (results)
|
|
415
116
|
|
|
416
|
-
|
|
117
|
+
### Install as a dev dependency
|
|
417
118
|
|
|
418
119
|
```bash
|
|
419
120
|
npm install -D sniff-qa
|
|
@@ -422,154 +123,262 @@ npm install -D sniff-qa
|
|
|
422
123
|
```json
|
|
423
124
|
{
|
|
424
125
|
"scripts": {
|
|
425
|
-
"qa": "sniff
|
|
126
|
+
"qa": "sniff"
|
|
426
127
|
}
|
|
427
128
|
}
|
|
428
129
|
```
|
|
429
130
|
|
|
430
|
-
|
|
131
|
+
That's all you need. Sniff auto-detects your dev server when it's running.
|
|
431
132
|
|
|
432
|
-
|
|
133
|
+
Requires Node.js 22+. Playwright installs automatically on first browser scan.
|
|
134
|
+
|
|
135
|
+
---
|
|
433
136
|
|
|
434
|
-
##
|
|
137
|
+
## Usage
|
|
435
138
|
|
|
436
139
|
```bash
|
|
437
|
-
npx sniff-qa
|
|
140
|
+
npx sniff-qa # scan source + auto-detect dev server
|
|
141
|
+
npx sniff-qa --url https://myapp.com # scan source + specific URL
|
|
142
|
+
npx sniff-qa ./path/to/project # scan a specific directory
|
|
143
|
+
npx sniff-qa --ci # CI mode (JUnit output, no AI explorer)
|
|
438
144
|
```
|
|
439
145
|
|
|
440
|
-
|
|
146
|
+
Sniff auto-detects your dev server by reading `package.json` scripts and probing common ports (3000, 5173, 8080, 4200). If it finds a running server, browser checks run automatically.
|
|
147
|
+
|
|
148
|
+
---
|
|
149
|
+
|
|
150
|
+
## What it finds
|
|
441
151
|
|
|
442
|
-
|
|
152
|
+
<picture>
|
|
153
|
+
<source media="(prefers-color-scheme: dark)" srcset=".github/assets/features-dark.svg">
|
|
154
|
+
<source media="(prefers-color-scheme: light)" srcset=".github/assets/features-light.svg">
|
|
155
|
+
<img alt="Sniff checks: source bugs, dead links, API endpoints, broken imports, accessibility, visual regression, performance, AI explorer" src=".github/assets/features-light.svg" width="100%">
|
|
156
|
+
</picture>
|
|
443
157
|
|
|
444
|
-
|
|
158
|
+
**Source checks** run on every scan. **Browser checks** run automatically when sniff detects a running dev server, or when you pass `--url`.
|
|
445
159
|
|
|
446
|
-
|
|
160
|
+
---
|
|
447
161
|
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
<
|
|
162
|
+
## Example output
|
|
163
|
+
|
|
164
|
+
<picture>
|
|
165
|
+
<source media="(prefers-color-scheme: dark)" srcset=".github/assets/report-dark.svg">
|
|
166
|
+
<source media="(prefers-color-scheme: light)" srcset=".github/assets/report-light.svg">
|
|
167
|
+
<img alt="Sniff terminal output" src=".github/assets/report-light.svg" width="100%">
|
|
168
|
+
</picture>
|
|
169
|
+
|
|
170
|
+
---
|
|
171
|
+
|
|
172
|
+
## Commands
|
|
173
|
+
|
|
174
|
+
### Slash commands (AI editors)
|
|
175
|
+
|
|
176
|
+
| Command | What it does |
|
|
177
|
+
|:--------|:-------------|
|
|
178
|
+
| `/sniff` | Scan the project. Auto-detects dev server, runs source + browser checks |
|
|
179
|
+
| `/sniff-fix` | Scan and auto-fix safe issues (debugger, console.log, etc.) |
|
|
180
|
+
| `/sniff-report` | Show results from the last scan |
|
|
181
|
+
|
|
182
|
+
### CLI
|
|
451
183
|
|
|
452
184
|
```
|
|
453
|
-
sniff
|
|
454
|
-
sniff --url <url>
|
|
455
|
-
sniff --url <url> --ci
|
|
456
|
-
|
|
457
|
-
sniff
|
|
458
|
-
sniff
|
|
459
|
-
sniff
|
|
460
|
-
sniff
|
|
185
|
+
sniff Scan source + auto-detect dev server
|
|
186
|
+
sniff --url <url> Scan source + test specific URL
|
|
187
|
+
sniff --url <url> --ci Full audit for CI pipelines
|
|
188
|
+
sniff <path> Scan a specific directory
|
|
189
|
+
sniff fix Auto-fix safe issues (debugger, console.log)
|
|
190
|
+
sniff fix --check Dry run: show what would be fixed
|
|
191
|
+
sniff init Create sniff.config.ts
|
|
192
|
+
sniff ci Generate GitHub Actions workflow
|
|
193
|
+
sniff report Show last scan results
|
|
194
|
+
sniff update-baselines Accept current visual baselines
|
|
195
|
+
sniff doctor Check your environment (Node, Playwright, config, dev server)
|
|
196
|
+
sniff --help Show all commands and flags
|
|
197
|
+
sniff --version Show version
|
|
461
198
|
```
|
|
462
199
|
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
| `--
|
|
469
|
-
| `--
|
|
200
|
+
<details>
|
|
201
|
+
<summary><b>All flags</b></summary>
|
|
202
|
+
|
|
203
|
+
| Flag | What it does |
|
|
204
|
+
|:-----|:-------------|
|
|
205
|
+
| `--url <url>` | Enable browser checks (accessibility, visual, performance, AI) |
|
|
206
|
+
| `--ci` | CI mode: skip AI explorer, add JUnit output, track flaky tests |
|
|
207
|
+
| `--no-explore` | Browser checks without AI explorer |
|
|
208
|
+
| `--no-browser` | Source only even if `--url` is set |
|
|
209
|
+
| `--max-steps <n>` | Limit AI explorer steps (default: 50) |
|
|
470
210
|
| `--no-headless` | Show the browser window |
|
|
471
211
|
| `--format html,json,junit` | Choose report formats |
|
|
472
|
-
| `--fail-on critical,high` | Severities that
|
|
473
|
-
| `--track-flakes` |
|
|
474
|
-
| `--json` |
|
|
212
|
+
| `--fail-on critical,high` | Severities that cause non-zero exit |
|
|
213
|
+
| `--track-flakes` | Track test flakiness across runs |
|
|
214
|
+
| `--json` | JSON output for scripts |
|
|
475
215
|
|
|
476
216
|
</details>
|
|
477
217
|
|
|
478
|
-
|
|
218
|
+
---
|
|
219
|
+
|
|
220
|
+
## Works with any stack
|
|
221
|
+
|
|
222
|
+
Sniff auto-detects your framework. No config needed.
|
|
223
|
+
|
|
224
|
+
<table>
|
|
225
|
+
<tr>
|
|
226
|
+
<td align="center" width="14%"><b>React</b><br/><sub>JSX / TSX</sub></td>
|
|
227
|
+
<td align="center" width="14%"><b>Next.js</b><br/><sub>App + Pages</sub></td>
|
|
228
|
+
<td align="center" width="14%"><b>Vue</b><br/><sub>SFC</sub></td>
|
|
229
|
+
<td align="center" width="14%"><b>Svelte</b><br/><sub>Components</sub></td>
|
|
230
|
+
<td align="center" width="14%"><b>Angular</b><br/><sub>Templates</sub></td>
|
|
231
|
+
<td align="center" width="14%"><b>Express</b><br/><sub>Routes</sub></td>
|
|
232
|
+
<td align="center" width="14%"><b>Vanilla</b><br/><sub>HTML / CSS</sub></td>
|
|
233
|
+
</tr>
|
|
234
|
+
</table>
|
|
235
|
+
|
|
236
|
+
API discovery also supports **Fastify**, **Hono**, **tRPC**, and **GraphQL**.
|
|
237
|
+
|
|
238
|
+
---
|
|
479
239
|
|
|
480
240
|
## Configuration
|
|
481
241
|
|
|
482
|
-
|
|
483
|
-
<summary><b>⚙️ sniff.config.ts reference</b></summary>
|
|
484
|
-
<br/>
|
|
242
|
+
Sniff works with zero config. Only create a config file if you want to customize.
|
|
485
243
|
|
|
486
|
-
|
|
244
|
+
```bash
|
|
245
|
+
npx sniff-qa init
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
<details>
|
|
249
|
+
<summary><b>sniff.config.ts reference</b></summary>
|
|
487
250
|
|
|
488
251
|
```typescript
|
|
489
252
|
import { defineConfig } from 'sniff-qa';
|
|
490
253
|
|
|
491
254
|
export default defineConfig({
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
255
|
+
// Save your URL so you can just run `sniff`
|
|
256
|
+
browser: { baseUrl: 'http://localhost:3000' },
|
|
257
|
+
|
|
258
|
+
// Viewports to test
|
|
495
259
|
viewports: [
|
|
496
260
|
{ name: 'mobile', width: 375, height: 667 },
|
|
497
261
|
{ name: 'desktop', width: 1280, height: 720 },
|
|
498
262
|
],
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
},
|
|
263
|
+
|
|
264
|
+
// Performance budgets (ms)
|
|
265
|
+
performance: { budgets: { lcp: 2500, fcp: 1800, tti: 3800 } },
|
|
266
|
+
|
|
267
|
+
// Visual regression threshold (0-1)
|
|
502
268
|
visual: { threshold: 0.1 },
|
|
269
|
+
|
|
270
|
+
// AI explorer
|
|
503
271
|
exploration: { maxSteps: 50 },
|
|
504
|
-
flakiness: { windowSize: 5, threshold: 3 },
|
|
505
272
|
|
|
506
|
-
// Dead link checker
|
|
273
|
+
// Dead link checker
|
|
507
274
|
deadLinks: {
|
|
508
|
-
checkExternal: true,
|
|
509
|
-
timeout: 5000,
|
|
510
|
-
retries: 2,
|
|
511
|
-
ignorePatterns: [],
|
|
512
|
-
maxConcurrent: 10,
|
|
275
|
+
checkExternal: true,
|
|
276
|
+
timeout: 5000,
|
|
277
|
+
retries: 2,
|
|
278
|
+
ignorePatterns: [],
|
|
279
|
+
maxConcurrent: 10,
|
|
513
280
|
},
|
|
514
281
|
|
|
515
|
-
// API endpoint discovery
|
|
282
|
+
// API endpoint discovery
|
|
516
283
|
apiEndpoints: {
|
|
517
|
-
checkErrorHandling: true,
|
|
518
|
-
checkValidation: true,
|
|
519
|
-
checkAuth: true,
|
|
520
|
-
checkSecrets: true,
|
|
521
|
-
frameworks: [],
|
|
284
|
+
checkErrorHandling: true,
|
|
285
|
+
checkValidation: true,
|
|
286
|
+
checkAuth: true,
|
|
287
|
+
checkSecrets: true,
|
|
288
|
+
frameworks: [], // empty = auto-detect all
|
|
522
289
|
},
|
|
523
290
|
|
|
524
|
-
//
|
|
291
|
+
// Turn off specific rules
|
|
525
292
|
rules: {
|
|
526
|
-
'debug-console-log': 'off',
|
|
527
|
-
'dead-link-internal': 'off', // example: skip dead link checks
|
|
293
|
+
'debug-console-log': 'off',
|
|
528
294
|
},
|
|
529
295
|
});
|
|
530
296
|
```
|
|
531
297
|
|
|
532
298
|
</details>
|
|
533
299
|
|
|
534
|
-
<
|
|
300
|
+
<details>
|
|
301
|
+
<summary><b>All rule IDs</b></summary>
|
|
302
|
+
|
|
303
|
+
| Rule | Severity | What it checks |
|
|
304
|
+
|:-----|:---------|:---------------|
|
|
305
|
+
| `debug-console-log` | medium | console.log/debug/info |
|
|
306
|
+
| `debug-debugger` | high | debugger statements |
|
|
307
|
+
| `placeholder-lorem` | high | Lorem ipsum text |
|
|
308
|
+
| `placeholder-todo` | medium | TODO comments |
|
|
309
|
+
| `placeholder-fixme` | high | FIXME comments |
|
|
310
|
+
| `placeholder-tbd` | medium | TBD markers |
|
|
311
|
+
| `hardcoded-localhost` | medium | localhost URLs |
|
|
312
|
+
| `hardcoded-127` | medium | 127.0.0.1 URLs |
|
|
313
|
+
| `broken-import` | medium | Unresolved imports |
|
|
314
|
+
| `dead-link-internal` | high | Broken file links |
|
|
315
|
+
| `dead-link-external` | medium | 404 external URLs |
|
|
316
|
+
| `dead-link-anchor` | medium | Missing anchors |
|
|
317
|
+
| `api-no-error-handling` | medium | Routes without try/catch |
|
|
318
|
+
| `api-no-validation` | medium | POST/PUT without validation |
|
|
319
|
+
| `api-no-auth` | low | Routes without auth |
|
|
320
|
+
| `api-hardcoded-secret` | critical | Hardcoded API keys |
|
|
321
|
+
|
|
322
|
+
Set any to `'off'` to disable.
|
|
323
|
+
|
|
324
|
+
</details>
|
|
535
325
|
|
|
536
|
-
|
|
326
|
+
---
|
|
537
327
|
|
|
538
|
-
|
|
539
|
-
|:--|:--|
|
|
540
|
-
| 🚫 | **No telemetry.** Sniff does not phone home. Ever. |
|
|
541
|
-
| 🔑 | **No signup.** No accounts. No API keys for core functionality. |
|
|
542
|
-
| 📦 | **No data collection.** Your code stays on your machine. |
|
|
543
|
-
| 👁️ | **Open source.** Read every line. Apache 2.0. |
|
|
328
|
+
## CI integration
|
|
544
329
|
|
|
545
|
-
|
|
546
|
-
|
|
330
|
+
```bash
|
|
331
|
+
npx sniff-qa ci
|
|
332
|
+
```
|
|
547
333
|
|
|
548
|
-
|
|
334
|
+
Generates `.github/workflows/sniff.yml` with Playwright caching, JUnit output, and report artifacts.
|
|
549
335
|
|
|
550
|
-
|
|
336
|
+
**Flakiness quarantine:** Tests that fail 3 of 5 runs get quarantined. They still run, still report, but won't block your pipeline.
|
|
551
337
|
|
|
552
|
-
|
|
338
|
+
---
|
|
553
339
|
|
|
554
|
-
|
|
340
|
+
## How it works
|
|
555
341
|
|
|
556
|
-
|
|
342
|
+
<picture>
|
|
343
|
+
<source media="(prefers-color-scheme: dark)" srcset=".github/assets/pipeline-dark.svg">
|
|
344
|
+
<source media="(prefers-color-scheme: light)" srcset=".github/assets/pipeline-light.svg">
|
|
345
|
+
<img alt="Sniff pipeline" src=".github/assets/pipeline-light.svg" width="100%">
|
|
346
|
+
</picture>
|
|
557
347
|
|
|
558
|
-
|
|
348
|
+
<details>
|
|
349
|
+
<summary><b>Architecture</b></summary>
|
|
559
350
|
|
|
560
|
-
|
|
561
|
-
-
|
|
562
|
-
-
|
|
563
|
-
|
|
351
|
+
<picture>
|
|
352
|
+
<source media="(prefers-color-scheme: dark)" srcset=".github/assets/architecture-dark.svg">
|
|
353
|
+
<source media="(prefers-color-scheme: light)" srcset=".github/assets/architecture-light.svg">
|
|
354
|
+
<img alt="Sniff architecture" src=".github/assets/architecture-light.svg" width="100%">
|
|
355
|
+
</picture>
|
|
564
356
|
|
|
565
|
-
|
|
357
|
+
</details>
|
|
358
|
+
|
|
359
|
+
---
|
|
566
360
|
|
|
567
|
-
##
|
|
361
|
+
## Privacy
|
|
568
362
|
|
|
569
|
-
|
|
363
|
+
No telemetry. No signup. No data collection. No API keys. Your code stays on your machine.
|
|
364
|
+
|
|
365
|
+
> [!NOTE]
|
|
366
|
+
> All 8 checks work without any API key. The AI explorer uses Claude Code as the AI provider when run through MCP, so no separate Anthropic key is needed. Dead link checking validates external URLs but never sends your code.
|
|
570
367
|
|
|
571
368
|
---
|
|
572
369
|
|
|
370
|
+
## Contributing
|
|
371
|
+
|
|
372
|
+
Add a source rule: each rule is a regex + severity in `src/scanners/source/rules/`. See [CONTRIBUTING.md](CONTRIBUTING.md).
|
|
373
|
+
|
|
374
|
+
---
|
|
375
|
+
|
|
376
|
+
<p align="center">
|
|
377
|
+
<sub>
|
|
378
|
+
Built on <a href="https://playwright.dev">Playwright</a> · <a href="https://github.com/dequelabs/axe-core">axe-core</a> · <a href="https://developer.chrome.com/docs/lighthouse">Lighthouse</a> · <a href="https://github.com/mapbox/pixelmatch">pixelmatch</a> · <a href="https://zod.dev">Zod</a> · <a href="https://github.com/modelcontextprotocol/typescript-sdk">MCP SDK</a> · <a href="https://github.com/anthropics/anthropic-sdk-typescript">Anthropic SDK</a>
|
|
379
|
+
</sub>
|
|
380
|
+
</p>
|
|
381
|
+
|
|
573
382
|
<p align="center">
|
|
574
383
|
<a href="https://www.linkedin.com/in/adam-boudjemaa/"><img src="https://img.shields.io/badge/LinkedIn-0A66C2?style=flat-square&logo=linkedin&logoColor=white" alt="LinkedIn"></a>
|
|
575
384
|
<a href="https://x.com/AdamBoudj"><img src="https://img.shields.io/badge/X-000000?style=flat-square&logo=x&logoColor=white" alt="X"></a>
|
|
@@ -577,5 +386,5 @@ Easiest way in: **add a source rule.** Each rule is a regex pattern with a sever
|
|
|
577
386
|
</p>
|
|
578
387
|
|
|
579
388
|
<p align="center">
|
|
580
|
-
<sub>Built by <a href="https://github.com/Aboudjem">Adam Boudjemaa</a> · Apache 2.0
|
|
389
|
+
<sub>Built by <a href="https://github.com/Aboudjem">Adam Boudjemaa</a> · <a href="LICENSE">Apache 2.0</a></sub>
|
|
581
390
|
</p>
|