sniff-qa 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,7 +1,7 @@
1
1
  <picture>
2
2
  <source media="(prefers-color-scheme: dark)" srcset=".github/assets/logo-dark.svg">
3
3
  <source media="(prefers-color-scheme: light)" srcset=".github/assets/logo-light.svg">
4
- <img alt="Sniff - open source AI-powered QA testing tool with source scanning, accessibility audits, visual regression, performance budgets, and AI exploration" src=".github/assets/logo-light.svg" width="100%">
4
+ <img alt="Sniff" src=".github/assets/logo-light.svg" width="100%">
5
5
  </picture>
6
6
 
7
7
  <p align="center">
@@ -12,259 +12,33 @@
12
12
  <a href="https://github.com/Aboudjem/sniff/stargazers"><img src="https://img.shields.io/github/stars/Aboudjem/sniff?style=flat-square&color=ef4444" alt="Stars"></a>
13
13
  </p>
14
14
 
15
- <p align="center">
16
- <b>One command. Eight checks. Zero config.</b><br/>
17
- <sub>Source bugs, dead links, API endpoints, accessibility, visual regression, performance, AI exploration, and cross-referencing. All in one pass.</sub>
18
- </p>
15
+ <p align="center">Scan your source code, your live site, or both. Finds bugs before your users do.</p>
19
16
 
20
17
  ---
21
18
 
22
- <br/>
23
-
24
- ## What is this?
25
-
26
- You ship features. But did you actually *test* them?
19
+ ## Get started
27
20
 
28
- **sniff** is an AI-powered QA tool that scans your project for real problems: leftover debugger statements, dead links, API endpoint issues, accessibility violations, visual regressions, performance budget breaches, and adversarial edge cases. It runs eight checks in a single command and cross-references source code with browser behavior for high-confidence findings.
21
+ ### Use from the terminal
29
22
 
30
23
  ```bash
31
- npx sniff-qa
24
+ cd ~/projects/my-app # go to your project
25
+ npx sniff-qa # that's it
32
26
  ```
33
27
 
34
- That's it. No config files. No signup. No API keys. It just works.
35
-
36
- <br/>
37
-
38
- ### Key facts
39
-
40
- - **What:** Open source AI-powered QA testing tool that runs 8 checks in a single command: source scanning (6 rule categories + dead links + API endpoint discovery), accessibility audits, visual regression, performance budgets, AI-driven exploration, and source/browser cross-referencing
41
- - **Who it's for:** Frontend and full-stack developers who want automated QA without configuring multiple tools
42
- - **How it works:** Uses Playwright for browser automation, axe-core for WCAG accessibility, Lighthouse for performance, pixelmatch for visual diffs, and Anthropic Claude for adversarial AI exploration
43
- - **Install:** `npx sniff-qa` (zero config, no signup, no API keys required for base checks)
44
- - **Comparable to:** Combines Lighthouse CI + Pa11y + BackstopJS into one tool, plus source code scanning and AI-driven edge case discovery that no other tool offers
45
- - **Unique:** First QA tool with native MCP (Model Context Protocol) integration. Flakiness quarantine system isolates non-deterministic test results automatically
46
-
47
- <br/>
48
-
49
- ## Quick start
50
-
51
- | Step | Action |
52
- |:----:|:-------|
53
- | **1** | Run a source scan: `npx sniff-qa` |
54
- | **2** | Run the full audit: `npx sniff-qa --url http://localhost:3000` |
55
- | **3** | Read the report and fix what matters |
56
-
57
- The source scan finishes in seconds. The full audit adds accessibility, visual regression, performance, and AI exploration on top of it.
58
-
59
- > [!TIP]
60
- > Drop your URL in `sniff.config.ts` once and you can just run `sniff` for the full audit every time.
61
-
62
- <br/>
63
-
64
- ## The three modes
28
+ Sniff auto-detects everything: your framework, your dev server port, and your running app. If `npm run dev` is running, sniff finds it and runs browser checks too. No flags needed.
65
29
 
66
- ```mermaid
67
- graph LR
68
- A["🔍 Quick Scan<br/><sub>npx sniff-qa</sub>"] --> B["Source bugs only<br/><sub>No browser needed</sub>"]
69
- C["🛡️ Full Audit<br/><sub>npx sniff-qa --url URL</sub>"] --> D["All 8 checks<br/><sub>Source + Browser + Xref</sub>"]
70
- E["🏗️ CI Mode<br/><sub>npx sniff-qa --url URL --ci</sub>"] --> F["Deterministic<br/><sub>Skips AI explorer</sub>"]
71
-
72
- style A fill:#fef2f2,stroke:#ef4444,color:#7f1d1d
73
- style B fill:#fee2e2,stroke:#ef4444,color:#7f1d1d
74
- style C fill:#fef2f2,stroke:#ef4444,color:#7f1d1d
75
- style D fill:#fee2e2,stroke:#ef4444,color:#7f1d1d
76
- style E fill:#fef2f2,stroke:#ef4444,color:#7f1d1d
77
- style F fill:#fee2e2,stroke:#ef4444,color:#7f1d1d
78
- ```
30
+ You can also be explicit:
79
31
 
80
32
  ```bash
81
- npx sniff-qa # quick scan (no browser)
82
- npx sniff-qa --url http://localhost:3000 # full audit (everything)
83
- npx sniff-qa --url http://localhost:3000 --ci # ci mode (skips AI explorer)
84
- ```
85
-
86
- CI mode auto-skips the AI explorer because it is non-deterministic. Everything else runs exactly the same.
87
-
88
- <br/>
89
-
90
- ## What it checks
91
-
92
- | | Check | Example finding | Details |
93
- |:--|:------|:----------------|:--------|
94
- | 📄 | **Source code** | `debugger` statement in `handler.ts:42` | Leftover debugger, placeholder text, hardcoded URLs, broken imports, TODO/FIXME |
95
- | 🔗 | **Dead links** | Broken link `./guide.md` in `README.md:28` | Validates internal file refs, external URLs (HTTP HEAD with retry), and anchor links. Catches 404s before your users do |
96
- | 🛣️ | **API endpoints** | `POST /users` has no input validation | Discovers routes from Express, Fastify, Hono, Next.js, SvelteKit, tRPC, and GraphQL. Flags missing error handling, auth, and hardcoded secrets |
97
- | ♿ | **Accessibility** | Missing form label on `/login` | WCAG 2.x violations via [axe-core](https://github.com/dequelabs/axe-core) with exact fix guidance |
98
- | 🖼 | **Visual regression** | `/pricing` changed 2.3% of pixels | Local pixel diffing via [pixelmatch](https://github.com/mapbox/pixelmatch). Commit baselines to track changes |
99
- | ⚡ | **Performance** | LCP 4200ms on `/dashboard` (budget: 2500ms) | [Lighthouse](https://developer.chrome.com/docs/lighthouse) budgets for LCP, FCP, TTI |
100
- | 🤖 | **AI explorer** | XSS in `/signup` email field crashes app | Roams your app, fills forms with adversarial inputs, reports crashes and console errors |
101
- | 🔀 | **Cross-reference** | `console.log` in source confirmed at runtime | Correlates source findings with browser evidence. Corroborated issues get bumped severity and HIGH confidence tags |
102
-
103
- <br/>
104
-
105
- ## How it works
106
-
107
- <picture>
108
- <source media="(prefers-color-scheme: dark)" srcset=".github/assets/pipeline-dark.svg">
109
- <source media="(prefers-color-scheme: light)" srcset=".github/assets/pipeline-light.svg">
110
- <img alt="Sniff pipeline showing 8 checks flowing through source scanner, browser runner, cross-reference engine, and report output" src=".github/assets/pipeline-light.svg" width="100%">
111
- </picture>
112
-
113
- Your code goes in. A report with actionable findings comes out. When you provide a URL, sniff also runs browser checks and cross-references source findings with runtime behavior for high-confidence results. You don't configure anything.
114
-
115
- <br/>
116
-
117
- ## Compared to
118
-
119
- | | **Sniff** | Lighthouse CI | Pa11y | BackstopJS |
120
- |:--|:--:|:--:|:--:|:--:|
121
- | Source scanning | ✅ | ❌ | ❌ | ❌ |
122
- | Dead link checking | ✅ | ❌ | ❌ | ❌ |
123
- | API endpoint discovery | ✅ | ❌ | ❌ | ❌ |
124
- | Accessibility | ✅ | partial | ✅ | ❌ |
125
- | Visual regression | ✅ | ❌ | ❌ | ✅ |
126
- | Performance | ✅ | ✅ | ❌ | ❌ |
127
- | AI exploration | ✅ | ❌ | ❌ | ❌ |
128
- | Cross-reference engine | ✅ | ❌ | ❌ | ❌ |
129
- | Flakiness quarantine | ✅ | ❌ | ❌ | ❌ |
130
- | Single command | ✅ | ❌ | ❌ | ❌ |
131
- | MCP server | ✅ | ❌ | ❌ | ❌ |
132
- | Zero config | ✅ | ❌ | ❌ | ❌ |
133
-
134
- > [!IMPORTANT]
135
- > Sniff is the **first QA tool with native MCP integration**. Your AI editor can trigger scans, read results, and fix issues without you switching context.
136
-
137
- <br/>
138
-
139
- ## Source scanning rules
140
-
141
- Sniff ships 6 rule categories that run on every scan. No browser needed.
142
-
143
- | Rule | ID | Severity | What it catches |
144
- |:-----|:---|:---------|:----------------|
145
- | Debug statements | `debug-console-log`, `debug-debugger` | medium / high | `console.log`, `console.debug`, `debugger` left in production code |
146
- | Placeholder text | `placeholder-lorem`, `placeholder-todo`, `placeholder-fixme`, `placeholder-tbd` | high / medium | Lorem ipsum, TODO, FIXME, TBD markers |
147
- | Hardcoded URLs | `hardcoded-localhost`, `hardcoded-127` | medium | `http://localhost:*` and `http://127.0.0.1:*` in non-test files |
148
- | Broken imports | `broken-import` | medium | Relative imports that don't resolve to a file (with TS extension mapping) |
149
- | Dead links | `dead-link-internal`, `dead-link-external`, `dead-link-anchor` | high / medium / low | Broken file references, 404 external URLs, missing anchor targets |
150
- | API endpoints | `api-no-error-handling`, `api-no-validation`, `api-no-auth`, `api-hardcoded-secret` | critical / medium / low | Route handlers missing try/catch, input validation, auth, or containing hardcoded secrets |
151
-
152
- ### Dead link checker
153
-
154
- Scans `.md`, `.html`, `.jsx`, `.tsx`, `.vue`, `.svelte`, and `.astro` files for links and validates them:
155
-
156
- - **Internal links:** Checks the target file exists (with extension resolution and index file fallback)
157
- - **External links:** Sends HTTP HEAD requests with timeout, retry, and GET fallback for servers that reject HEAD
158
- - **Anchor links:** Validates `#section` references against markdown headings and HTML `id`/`name` attributes
159
- - **Smart skipping:** Ignores `mailto:`, `tel:`, `data:`, `javascript:`, and template variables
160
-
161
- ```
162
- HIGH README.md:28 Broken internal link: ./missing-guide.md
163
- MED docs/api.md:15 HTTP 404: https://example.com/old-endpoint
164
- MED CONTRIBUTING.md:8 Anchor #setup-guide not found in ./README.md
165
- ```
166
-
167
- ### API endpoint discovery
168
-
169
- Automatically discovers API routes from 8 frameworks:
170
-
171
- | Framework | Detection method |
172
- |:----------|:----------------|
173
- | Express | `app.get()`, `router.post()`, etc. |
174
- | Fastify | `fastify.get()`, `fastify.post()`, etc. |
175
- | Hono | `app.get()` with `new Hono()` detection |
176
- | Next.js App Router | `app/**/route.ts` with exported `GET`/`POST` handlers |
177
- | Next.js Pages Router | `pages/api/**/*.ts` |
178
- | SvelteKit | `routes/**/+server.ts` |
179
- | tRPC | `publicProcedure.query()` / `.mutation()` |
180
- | GraphQL | `type Query { }` / `type Mutation { }` definitions |
181
-
182
- For each endpoint, sniff checks for:
183
- - Missing error handling (no try/catch or error middleware)
184
- - Missing input validation on POST/PUT/PATCH routes (no zod, joi, yup, etc.)
185
- - Missing auth middleware on non-public routes
186
- - Hardcoded secrets (API keys, tokens, Stripe keys, GitHub tokens)
187
-
188
- ```
189
- CRIT src/routes.ts:12 Hardcoded Stripe key in POST /checkout handler
190
- MED src/api.ts:5 POST /users has no input validation
191
- MED src/api.ts:18 GET /admin/data has no error handling
192
- LOW src/api.ts:25 GET /admin/data has no visible auth middleware
193
- INFO Discovered 6 API endpoints (express: 4, nextjs-app: 2)
194
- ```
195
-
196
- ### Cross-reference engine
197
-
198
- When you run sniff with `--url`, the cross-reference engine correlates source code findings with browser runtime behavior. Findings confirmed in both layers get bumped severity and a HIGH confidence tag.
199
-
200
- **5 correlation strategies:**
201
-
202
- | Source finding | Browser evidence | Correlation |
203
- |:---------------|:-----------------|:------------|
204
- | Broken import / dead link | 404 network request | `broken-import-to-404` |
205
- | `console.log` statement | Console output captured | `console-log-to-runtime` |
206
- | Hardcoded localhost URL | Network request to that URL | `hardcoded-url-to-network` |
207
- | Missing label / a11y issue | axe-core violation | `source-a11y-to-axe` |
208
- | Placeholder text in source | Text visible on page | `placeholder-to-runtime` |
209
-
33
+ npx sniff-qa --url http://localhost:3000 # specific local URL
34
+ npx sniff-qa --url https://myapp.com # production URL
210
35
  ```
211
- CORROBORATED (source + browser evidence)
212
- src/handler.ts:42 console.log("debug data")
213
- Source: debug-console-log rule triggered
214
- Browser: console.log output captured at /dashboard
215
- Confidence: HIGH (confirmed in both layers)
216
- ```
217
-
218
- <br/>
219
-
220
- ## Architecture
221
-
222
- <picture>
223
- <source media="(prefers-color-scheme: dark)" srcset=".github/assets/architecture-dark.svg">
224
- <source media="(prefers-color-scheme: light)" srcset=".github/assets/architecture-light.svg">
225
- <img alt="Sniff architecture showing source scanner, browser runner, cross-reference engine, and report engine" src=".github/assets/architecture-light.svg" width="100%">
226
- </picture>
227
-
228
- <br/>
229
-
230
- ## Example output
231
-
232
- <picture>
233
- <source media="(prefers-color-scheme: dark)" srcset=".github/assets/report-dark.svg">
234
- <source media="(prefers-color-scheme: light)" srcset=".github/assets/report-light.svg">
235
- <img alt="Sniff terminal output showing source findings, browser results, and cross-referenced corroborated findings" src=".github/assets/report-light.svg" width="100%">
236
- </picture>
237
-
238
- <br/>
239
-
240
- ## Works with your stack
241
-
242
- No config needed. Sniff auto-detects your framework from `package.json` and source files.
243
-
244
- <table>
245
- <tr>
246
- <td align="center" width="16%"><b>⚛️ React</b></td>
247
- <td align="center" width="16%"><b>▲ Next.js</b></td>
248
- <td align="center" width="16%"><b>💚 Vue</b></td>
249
- <td align="center" width="16%"><b>🔶 Svelte</b></td>
250
- <td align="center" width="16%"><b>🅰️ Angular</b></td>
251
- <td align="center" width="16%"><b>🌐 Vanilla</b></td>
252
- </tr>
253
- <tr>
254
- <td align="center">JSX / TSX</td>
255
- <td align="center">App Router</td>
256
- <td align="center">SFC</td>
257
- <td align="center">Components</td>
258
- <td align="center">Templates</td>
259
- <td align="center">HTML / CSS</td>
260
- </tr>
261
- </table>
262
36
 
263
- <br/>
37
+ > **No API keys. No Playwright install. No config files.** Everything works out of the box. Browser checks auto-install Chromium on first run.
264
38
 
265
- ## 🔌 Use with your AI editor
39
+ ### Use from your AI editor
266
40
 
267
- Sniff ships an MCP server. Pick your tool and copy the snippet.
41
+ Sniff ships as an MCP server. Add it to your editor, then ask your AI to scan.
268
42
 
269
43
  <details>
270
44
  <summary><b>Claude Code</b></summary>
@@ -272,51 +46,24 @@ Sniff ships an MCP server. Pick your tool and copy the snippet.
272
46
  ```bash
273
47
  claude mcp add sniff-qa npx sniff-qa --mcp
274
48
  ```
275
-
276
- Or `.mcp.json`:
277
-
278
- ```json
279
- {
280
- "mcpServers": {
281
- "sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] }
282
- }
283
- }
284
- ```
285
-
286
49
  </details>
287
50
 
288
51
  <details>
289
52
  <summary><b>Cursor</b></summary>
290
53
 
291
- `~/.cursor/mcp.json` or `.cursor/mcp.json`:
292
-
54
+ Add to `~/.cursor/mcp.json`:
293
55
  ```json
294
- {
295
- "mcpServers": {
296
- "sniff-qa": {
297
- "type": "stdio",
298
- "command": "npx",
299
- "args": ["sniff-qa", "--mcp"]
300
- }
301
- }
302
- }
56
+ { "mcpServers": { "sniff-qa": { "type": "stdio", "command": "npx", "args": ["sniff-qa", "--mcp"] } } }
303
57
  ```
304
-
305
58
  </details>
306
59
 
307
60
  <details>
308
- <summary><b>Windsurf</b></summary>
309
-
310
- `~/.codeium/windsurf/mcp_config.json`:
61
+ <summary><b>VS Code + Copilot</b></summary>
311
62
 
63
+ Add to `.vscode/mcp.json`:
312
64
  ```json
313
- {
314
- "mcpServers": {
315
- "sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] }
316
- }
317
- }
65
+ { "servers": { "sniff-qa": { "type": "stdio", "command": "npx", "args": ["-y", "sniff-qa", "--mcp"] } } }
318
66
  ```
319
-
320
67
  </details>
321
68
 
322
69
  <details>
@@ -325,64 +72,34 @@ Or `.mcp.json`:
325
72
  ```bash
326
73
  codex mcp add sniff-qa -- npx -y sniff-qa --mcp
327
74
  ```
75
+ </details>
328
76
 
329
- Or add to `~/.codex/config.toml`:
77
+ <details>
78
+ <summary><b>Windsurf</b></summary>
330
79
 
331
- ```toml
332
- [mcp_servers.sniff-qa]
333
- command = "npx"
334
- args = ["-y", "sniff-qa", "--mcp"]
80
+ Add to `~/.codeium/windsurf/mcp_config.json`:
81
+ ```json
82
+ { "mcpServers": { "sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] } } }
335
83
  ```
336
-
337
84
  </details>
338
85
 
339
86
  <details>
340
87
  <summary><b>Gemini CLI</b></summary>
341
88
 
342
- `~/.gemini/mcp_config.json`:
343
-
89
+ Add to `~/.gemini/mcp_config.json`:
344
90
  ```json
345
- {
346
- "mcpServers": {
347
- "sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] }
348
- }
349
- }
91
+ { "mcpServers": { "sniff-qa": { "command": "npx", "args": ["sniff-qa", "--mcp"] } } }
350
92
  ```
351
-
352
93
  </details>
353
94
 
354
95
  <details>
355
96
  <summary><b>Continue.dev</b></summary>
356
97
 
357
- `.continue/mcpServers/sniff-qa.yaml`:
358
-
98
+ Add to `.continue/mcpServers/sniff-qa.yaml`:
359
99
  ```yaml
360
100
  mcpServers:
361
- sniff-qa:
362
- command: npx
363
- args: [sniff-qa, --mcp]
364
- type: stdio
101
+ sniff-qa: { command: npx, args: [sniff-qa, --mcp], type: stdio }
365
102
  ```
366
-
367
- </details>
368
-
369
- <details>
370
- <summary><b>VS Code + Copilot</b></summary>
371
-
372
- Add to `.vscode/mcp.json`:
373
-
374
- ```json
375
- {
376
- "servers": {
377
- "sniff-qa": {
378
- "type": "stdio",
379
- "command": "npx",
380
- "args": ["-y", "sniff-qa", "--mcp"]
381
- }
382
- }
383
- }
384
- ```
385
-
386
103
  </details>
387
104
 
388
105
  <details>
@@ -391,29 +108,13 @@ Add to `.vscode/mcp.json`:
391
108
  ```bash
392
109
  clawhub install sniff-qa
393
110
  ```
394
-
395
- Or add to `~/.openclaw/openclaw.json`:
396
-
397
- ```json
398
- {
399
- "mcpServers": {
400
- "sniff-qa": {
401
- "command": "npx",
402
- "args": ["-y", "sniff-qa", "--mcp"]
403
- }
404
- }
405
- }
406
- ```
407
-
408
111
  </details>
409
112
 
410
- Once configured, ask: *"Scan this project for issues"* or *"Check accessibility on localhost:3000"*.
411
-
412
- **MCP tools exposed:** `sniff_scan` (source), `sniff_run` (browser), `sniff_report` (last results).
113
+ Then ask: *"Scan this project for issues"* or *"Check accessibility on localhost:3000"*
413
114
 
414
- <br/>
115
+ **MCP tools:** `sniff_scan` (source) · `sniff_run` (browser) · `sniff_report` (results)
415
116
 
416
- ## Install
117
+ ### Install as a dev dependency
417
118
 
418
119
  ```bash
419
120
  npm install -D sniff-qa
@@ -422,154 +123,262 @@ npm install -D sniff-qa
422
123
  ```json
423
124
  {
424
125
  "scripts": {
425
- "qa": "sniff --url http://localhost:3000"
126
+ "qa": "sniff"
426
127
  }
427
128
  }
428
129
  ```
429
130
 
430
- Then run `npm run qa`. Requires Node.js 22+. Playwright browsers install automatically the first time.
131
+ That's all you need. Sniff auto-detects your dev server when it's running.
431
132
 
432
- <br/>
133
+ Requires Node.js 22+. Playwright installs automatically on first browser scan.
134
+
135
+ ---
433
136
 
434
- ## 🏗️ CI integration
137
+ ## Usage
435
138
 
436
139
  ```bash
437
- npx sniff-qa ci
140
+ npx sniff-qa # scan source + auto-detect dev server
141
+ npx sniff-qa --url https://myapp.com # scan source + specific URL
142
+ npx sniff-qa ./path/to/project # scan a specific directory
143
+ npx sniff-qa --ci # CI mode (JUnit output, no AI explorer)
438
144
  ```
439
145
 
440
- Generates `.github/workflows/sniff.yml` with Playwright caching, JUnit output, flakiness quarantine, and report artifacts.
146
+ Sniff auto-detects your dev server by reading `package.json` scripts and probing common ports (3000, 5173, 8080, 4200). If it finds a running server, browser checks run automatically.
147
+
148
+ ---
149
+
150
+ ## What it finds
441
151
 
442
- **Flakiness quarantine.** Tests that fail 3 of 5 runs get quarantined. They still run, still appear in reports, but won't block your pipeline.
152
+ <picture>
153
+ <source media="(prefers-color-scheme: dark)" srcset=".github/assets/features-dark.svg">
154
+ <source media="(prefers-color-scheme: light)" srcset=".github/assets/features-light.svg">
155
+ <img alt="Sniff checks: source bugs, dead links, API endpoints, broken imports, accessibility, visual regression, performance, AI explorer" src=".github/assets/features-light.svg" width="100%">
156
+ </picture>
443
157
 
444
- <br/>
158
+ **Source checks** run on every scan. **Browser checks** run automatically when sniff detects a running dev server, or when you pass `--url`.
445
159
 
446
- ## All commands
160
+ ---
447
161
 
448
- <details>
449
- <summary><b>📋 Full CLI reference</b></summary>
450
- <br/>
162
+ ## Example output
163
+
164
+ <picture>
165
+ <source media="(prefers-color-scheme: dark)" srcset=".github/assets/report-dark.svg">
166
+ <source media="(prefers-color-scheme: light)" srcset=".github/assets/report-light.svg">
167
+ <img alt="Sniff terminal output" src=".github/assets/report-light.svg" width="100%">
168
+ </picture>
169
+
170
+ ---
171
+
172
+ ## Commands
173
+
174
+ ### Slash commands (AI editors)
175
+
176
+ | Command | What it does |
177
+ |:--------|:-------------|
178
+ | `/sniff` | Scan the project. Auto-detects dev server, runs source + browser checks |
179
+ | `/sniff-fix` | Scan and auto-fix safe issues (debugger, console.log, etc.) |
180
+ | `/sniff-report` | Show results from the last scan |
181
+
182
+ ### CLI
451
183
 
452
184
  ```
453
- sniff quick scan (source only)
454
- sniff --url <url> full audit (everything)
455
- sniff --url <url> --ci ci mode (no AI explorer)
456
-
457
- sniff init scaffold sniff.config.ts
458
- sniff ci generate .github/workflows/sniff.yml
459
- sniff report show last results
460
- sniff update-baselines accept current screenshots as baselines
185
+ sniff Scan source + auto-detect dev server
186
+ sniff --url <url> Scan source + test specific URL
187
+ sniff --url <url> --ci Full audit for CI pipelines
188
+ sniff <path> Scan a specific directory
189
+ sniff fix Auto-fix safe issues (debugger, console.log)
190
+ sniff fix --check Dry run: show what would be fixed
191
+ sniff init Create sniff.config.ts
192
+ sniff ci Generate GitHub Actions workflow
193
+ sniff report Show last scan results
194
+ sniff update-baselines Accept current visual baselines
195
+ sniff doctor Check your environment (Node, Playwright, config, dev server)
196
+ sniff --help Show all commands and flags
197
+ sniff --version Show version
461
198
  ```
462
199
 
463
- ### Flags (all optional)
464
-
465
- | Flag | Effect |
466
- |:--|:--|
467
- | `--no-explore` | Skip AI explorer in full mode |
468
- | `--no-browser` | Force source-only even when URL is set |
469
- | `--max-steps <n>` | Cap exploration steps (default: 50) |
200
+ <details>
201
+ <summary><b>All flags</b></summary>
202
+
203
+ | Flag | What it does |
204
+ |:-----|:-------------|
205
+ | `--url <url>` | Enable browser checks (accessibility, visual, performance, AI) |
206
+ | `--ci` | CI mode: skip AI explorer, add JUnit output, track flaky tests |
207
+ | `--no-explore` | Browser checks without AI explorer |
208
+ | `--no-browser` | Source only even if `--url` is set |
209
+ | `--max-steps <n>` | Limit AI explorer steps (default: 50) |
470
210
  | `--no-headless` | Show the browser window |
471
211
  | `--format html,json,junit` | Choose report formats |
472
- | `--fail-on critical,high` | Severities that exit non-zero |
473
- | `--track-flakes` | Enable flakiness detection |
474
- | `--json` | Machine-readable output |
212
+ | `--fail-on critical,high` | Severities that cause non-zero exit |
213
+ | `--track-flakes` | Track test flakiness across runs |
214
+ | `--json` | JSON output for scripts |
475
215
 
476
216
  </details>
477
217
 
478
- <br/>
218
+ ---
219
+
220
+ ## Works with any stack
221
+
222
+ Sniff auto-detects your framework. No config needed.
223
+
224
+ <table>
225
+ <tr>
226
+ <td align="center" width="14%"><b>React</b><br/><sub>JSX / TSX</sub></td>
227
+ <td align="center" width="14%"><b>Next.js</b><br/><sub>App + Pages</sub></td>
228
+ <td align="center" width="14%"><b>Vue</b><br/><sub>SFC</sub></td>
229
+ <td align="center" width="14%"><b>Svelte</b><br/><sub>Components</sub></td>
230
+ <td align="center" width="14%"><b>Angular</b><br/><sub>Templates</sub></td>
231
+ <td align="center" width="14%"><b>Express</b><br/><sub>Routes</sub></td>
232
+ <td align="center" width="14%"><b>Vanilla</b><br/><sub>HTML / CSS</sub></td>
233
+ </tr>
234
+ </table>
235
+
236
+ API discovery also supports **Fastify**, **Hono**, **tRPC**, and **GraphQL**.
237
+
238
+ ---
479
239
 
480
240
  ## Configuration
481
241
 
482
- <details>
483
- <summary><b>⚙️ sniff.config.ts reference</b></summary>
484
- <br/>
242
+ Sniff works with zero config. Only create a config file if you want to customize.
485
243
 
486
- Optional. Drop `sniff.config.ts` in your project root:
244
+ ```bash
245
+ npx sniff-qa init
246
+ ```
247
+
248
+ <details>
249
+ <summary><b>sniff.config.ts reference</b></summary>
487
250
 
488
251
  ```typescript
489
252
  import { defineConfig } from 'sniff-qa';
490
253
 
491
254
  export default defineConfig({
492
- browser: {
493
- baseUrl: 'http://localhost:3000',
494
- },
255
+ // Save your URL so you can just run `sniff`
256
+ browser: { baseUrl: 'http://localhost:3000' },
257
+
258
+ // Viewports to test
495
259
  viewports: [
496
260
  { name: 'mobile', width: 375, height: 667 },
497
261
  { name: 'desktop', width: 1280, height: 720 },
498
262
  ],
499
- performance: {
500
- budgets: { lcp: 2500, fcp: 1800, tti: 3800 },
501
- },
263
+
264
+ // Performance budgets (ms)
265
+ performance: { budgets: { lcp: 2500, fcp: 1800, tti: 3800 } },
266
+
267
+ // Visual regression threshold (0-1)
502
268
  visual: { threshold: 0.1 },
269
+
270
+ // AI explorer
503
271
  exploration: { maxSteps: 50 },
504
- flakiness: { windowSize: 5, threshold: 3 },
505
272
 
506
- // Dead link checker (new in v0.2)
273
+ // Dead link checker
507
274
  deadLinks: {
508
- checkExternal: true, // validate external URLs via HTTP
509
- timeout: 5000, // ms per request
510
- retries: 2, // retry failed requests
511
- ignorePatterns: [], // regex patterns to skip
512
- maxConcurrent: 10, // parallel HTTP requests
275
+ checkExternal: true,
276
+ timeout: 5000,
277
+ retries: 2,
278
+ ignorePatterns: [],
279
+ maxConcurrent: 10,
513
280
  },
514
281
 
515
- // API endpoint discovery (new in v0.2)
282
+ // API endpoint discovery
516
283
  apiEndpoints: {
517
- checkErrorHandling: true, // flag missing try/catch
518
- checkValidation: true, // flag missing input validation
519
- checkAuth: true, // flag missing auth middleware
520
- checkSecrets: true, // flag hardcoded secrets
521
- frameworks: [], // empty = auto-detect all
284
+ checkErrorHandling: true,
285
+ checkValidation: true,
286
+ checkAuth: true,
287
+ checkSecrets: true,
288
+ frameworks: [], // empty = auto-detect all
522
289
  },
523
290
 
524
- // Disable specific rules
291
+ // Turn off specific rules
525
292
  rules: {
526
- 'debug-console-log': 'off', // example: allow console.log
527
- 'dead-link-internal': 'off', // example: skip dead link checks
293
+ 'debug-console-log': 'off',
528
294
  },
529
295
  });
530
296
  ```
531
297
 
532
298
  </details>
533
299
 
534
- <br/>
300
+ <details>
301
+ <summary><b>All rule IDs</b></summary>
302
+
303
+ | Rule | Severity | What it checks |
304
+ |:-----|:---------|:---------------|
305
+ | `debug-console-log` | medium | console.log/debug/info |
306
+ | `debug-debugger` | high | debugger statements |
307
+ | `placeholder-lorem` | high | Lorem ipsum text |
308
+ | `placeholder-todo` | medium | TODO comments |
309
+ | `placeholder-fixme` | high | FIXME comments |
310
+ | `placeholder-tbd` | medium | TBD markers |
311
+ | `hardcoded-localhost` | medium | localhost URLs |
312
+ | `hardcoded-127` | medium | 127.0.0.1 URLs |
313
+ | `broken-import` | medium | Unresolved imports |
314
+ | `dead-link-internal` | high | Broken file links |
315
+ | `dead-link-external` | medium | 404 external URLs |
316
+ | `dead-link-anchor` | medium | Missing anchors |
317
+ | `api-no-error-handling` | medium | Routes without try/catch |
318
+ | `api-no-validation` | medium | POST/PUT without validation |
319
+ | `api-no-auth` | low | Routes without auth |
320
+ | `api-hardcoded-secret` | critical | Hardcoded API keys |
321
+
322
+ Set any to `'off'` to disable.
323
+
324
+ </details>
535
325
 
536
- ## 🔒 Trust and privacy
326
+ ---
537
327
 
538
- | | |
539
- |:--|:--|
540
- | 🚫 | **No telemetry.** Sniff does not phone home. Ever. |
541
- | 🔑 | **No signup.** No accounts. No API keys for core functionality. |
542
- | 📦 | **No data collection.** Your code stays on your machine. |
543
- | 👁️ | **Open source.** Read every line. Apache 2.0. |
328
+ ## CI integration
544
329
 
545
- > [!NOTE]
546
- > The AI explorer requires an Anthropic API key only if you want the chaos monkey feature. The other seven checks (source scanning, dead links, API endpoints, accessibility, visual regression, performance, cross-referencing) work completely offline with zero external calls. External link checking in the dead link scanner makes HTTP requests to validate URLs, but never sends your code.
330
+ ```bash
331
+ npx sniff-qa ci
332
+ ```
547
333
 
548
- <br/>
334
+ Generates `.github/workflows/sniff.yml` with Playwright caching, JUnit output, and report artifacts.
549
335
 
550
- ## Built on
336
+ **Flakiness quarantine:** Tests that fail 3 of 5 runs get quarantined. They still run, still report, but won't block your pipeline.
551
337
 
552
- [Playwright](https://playwright.dev) · [axe-core](https://github.com/dequelabs/axe-core) · [Lighthouse](https://developer.chrome.com/docs/lighthouse) · [pixelmatch](https://github.com/mapbox/pixelmatch) · [Zod](https://zod.dev) · [MCP SDK](https://github.com/modelcontextprotocol/typescript-sdk) · [Anthropic SDK](https://github.com/anthropics/anthropic-sdk-typescript)
338
+ ---
553
339
 
554
- <br/>
340
+ ## How it works
555
341
 
556
- ## Contributing
342
+ <picture>
343
+ <source media="(prefers-color-scheme: dark)" srcset=".github/assets/pipeline-dark.svg">
344
+ <source media="(prefers-color-scheme: light)" srcset=".github/assets/pipeline-light.svg">
345
+ <img alt="Sniff pipeline" src=".github/assets/pipeline-light.svg" width="100%">
346
+ </picture>
557
347
 
558
- Easiest way in: **add a source rule.** Each rule is a regex pattern with a severity level in `src/scanners/source/rules/`. See [CONTRIBUTING.md](CONTRIBUTING.md).
348
+ <details>
349
+ <summary><b>Architecture</b></summary>
559
350
 
560
- **Good first contributions:**
561
- - 🔧 Add a new source scanning rule
562
- - 🐛 Report a false positive
563
- - 📚 Improve documentation
351
+ <picture>
352
+ <source media="(prefers-color-scheme: dark)" srcset=".github/assets/architecture-dark.svg">
353
+ <source media="(prefers-color-scheme: light)" srcset=".github/assets/architecture-light.svg">
354
+ <img alt="Sniff architecture" src=".github/assets/architecture-light.svg" width="100%">
355
+ </picture>
564
356
 
565
- <br/>
357
+ </details>
358
+
359
+ ---
566
360
 
567
- ## License
361
+ ## Privacy
568
362
 
569
- [Apache 2.0](LICENSE)
363
+ No telemetry. No signup. No data collection. No API keys. Your code stays on your machine.
364
+
365
+ > [!NOTE]
366
+ > All 8 checks work without any API key. The AI explorer uses Claude Code as the AI provider when run through MCP, so no separate Anthropic key is needed. Dead link checking validates external URLs but never sends your code.
570
367
 
571
368
  ---
572
369
 
370
+ ## Contributing
371
+
372
+ Add a source rule: each rule is a regex + severity in `src/scanners/source/rules/`. See [CONTRIBUTING.md](CONTRIBUTING.md).
373
+
374
+ ---
375
+
376
+ <p align="center">
377
+ <sub>
378
+ Built on <a href="https://playwright.dev">Playwright</a> · <a href="https://github.com/dequelabs/axe-core">axe-core</a> · <a href="https://developer.chrome.com/docs/lighthouse">Lighthouse</a> · <a href="https://github.com/mapbox/pixelmatch">pixelmatch</a> · <a href="https://zod.dev">Zod</a> · <a href="https://github.com/modelcontextprotocol/typescript-sdk">MCP SDK</a> · <a href="https://github.com/anthropics/anthropic-sdk-typescript">Anthropic SDK</a>
379
+ </sub>
380
+ </p>
381
+
573
382
  <p align="center">
574
383
  <a href="https://www.linkedin.com/in/adam-boudjemaa/"><img src="https://img.shields.io/badge/LinkedIn-0A66C2?style=flat-square&logo=linkedin&logoColor=white" alt="LinkedIn"></a>
575
384
  <a href="https://x.com/AdamBoudj"><img src="https://img.shields.io/badge/X-000000?style=flat-square&logo=x&logoColor=white" alt="X"></a>
@@ -577,5 +386,5 @@ Easiest way in: **add a source rule.** Each rule is a regex pattern with a sever
577
386
  </p>
578
387
 
579
388
  <p align="center">
580
- <sub>Built by <a href="https://github.com/Aboudjem">Adam Boudjemaa</a> · Apache 2.0 · No telemetry · No data collection</sub>
389
+ <sub>Built by <a href="https://github.com/Aboudjem">Adam Boudjemaa</a> · <a href="LICENSE">Apache 2.0</a></sub>
581
390
  </p>