sniff-qa 0.3.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +75 -5
- package/dist/classifier-MWQMQ727.js +1701 -0
- package/dist/classifier-MWQMQ727.js.map +1 -0
- package/dist/cli/index.js +92 -27
- package/dist/cli/index.js.map +1 -1
- package/dist/dev-server-detector-XQ5AYPDT.js +319 -0
- package/dist/dev-server-detector-XQ5AYPDT.js.map +1 -0
- package/dist/discover-JRU5E4FR.js +327 -0
- package/dist/discover-JRU5E4FR.js.map +1 -0
- package/dist/domain-UQ34YE4B.js +1817 -0
- package/dist/domain-UQ34YE4B.js.map +1 -0
- package/dist/edge-cases-ZADKDZHL.js +246 -0
- package/dist/edge-cases-ZADKDZHL.js.map +1 -0
- package/dist/ensure-browsers-ACHCEQCT.js +60 -0
- package/dist/ensure-browsers-ACHCEQCT.js.map +1 -0
- package/dist/flakiness-OLJAY7CS.js +53 -0
- package/dist/flakiness-OLJAY7CS.js.map +1 -0
- package/dist/init-IQKT4ANC.js +148 -0
- package/dist/init-IQKT4ANC.js.map +1 -0
- package/dist/llm-VJAI6H6G.js +126 -0
- package/dist/llm-VJAI6H6G.js.map +1 -0
- package/dist/mcp/server.js +219 -1
- package/dist/mcp/server.js.map +1 -1
- package/dist/persistence-KTCRIGHU.js +270 -0
- package/dist/persistence-KTCRIGHU.js.map +1 -0
- package/dist/{repo-analyzer-WK5WP7NT.js → repo-analyzer-4OURSITY.js} +1 -1
- package/dist/{report-WLDV72FM.js → report-IKRVVIW4.js} +1 -1
- package/dist/report-SDZWX25B.js +280 -0
- package/dist/report-SDZWX25B.js.map +1 -0
- package/dist/runner-OYTN2NFD.js +996 -0
- package/dist/runner-OYTN2NFD.js.map +1 -0
- package/dist/scenarios-I7YYVTD2.js +987 -0
- package/dist/scenarios-I7YYVTD2.js.map +1 -0
- package/dist/tiebreak-L5UFNKU2.js +60 -0
- package/dist/tiebreak-L5UFNKU2.js.map +1 -0
- package/dist/{unified-7RKNG3EI.js → unified-JCUBP6UO.js} +2 -2
- package/package.json +2 -2
- package/dist/dev-server-detector-NJ5RN4IC.js +0 -150
- package/dist/dev-server-detector-NJ5RN4IC.js.map +0 -1
- package/dist/init-75MMIW4A.js +0 -66
- package/dist/init-75MMIW4A.js.map +0 -1
- /package/dist/{repo-analyzer-WK5WP7NT.js.map → repo-analyzer-4OURSITY.js.map} +0 -0
- /package/dist/{report-WLDV72FM.js.map → report-IKRVVIW4.js.map} +0 -0
- /package/dist/{unified-7RKNG3EI.js.map → unified-JCUBP6UO.js.map} +0 -0
package/README.md
CHANGED
|
@@ -41,7 +41,18 @@ npx sniff-qa --url https://myapp.com # production URL
|
|
|
41
41
|
Sniff ships as an MCP server. Add it to your editor, then ask your AI to scan.
|
|
42
42
|
|
|
43
43
|
<details>
|
|
44
|
-
<summary><b>Claude Code</b
|
|
44
|
+
<summary><b>Claude Code plugin marketplace</b> (one-command install)</summary>
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
claude plugin marketplace add Aboudjem/10x
|
|
48
|
+
claude plugin install sniff@10x
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
This installs sniff as a Claude Code plugin from the [10x marketplace](https://github.com/Aboudjem/10x). Skills, commands, and the MCP server are wired up automatically.
|
|
52
|
+
</details>
|
|
53
|
+
|
|
54
|
+
<details>
|
|
55
|
+
<summary><b>Claude Code (MCP server only)</b></summary>
|
|
45
56
|
|
|
46
57
|
```bash
|
|
47
58
|
claude mcp add sniff-qa npx sniff-qa --mcp
|
|
@@ -110,9 +121,9 @@ clawhub install sniff-qa
|
|
|
110
121
|
```
|
|
111
122
|
</details>
|
|
112
123
|
|
|
113
|
-
Then ask: *"Scan this project for issues"* or *"Check accessibility on localhost:3000"*
|
|
124
|
+
Then ask: *"Scan this project for issues"* or *"Check accessibility on localhost:3000"* or *"Discover and run E2E tests"*
|
|
114
125
|
|
|
115
|
-
**MCP tools:** `
|
|
126
|
+
**MCP tools:** `sniff({ mode: "scan" | "run" | "discover" | "report" })` — unified entry point. Plus `sniff_install` when Playwright's Chromium is missing. The legacy per-mode tools (`sniff_scan`, `sniff_run`, `sniff_discover`, `sniff_report`) remain available but are deprecated in v0.5 and will be removed in v0.7.
|
|
116
127
|
|
|
117
128
|
### Install as a dev dependency
|
|
118
129
|
|
|
@@ -141,9 +152,12 @@ npx sniff-qa # scan source + auto-detect
|
|
|
141
152
|
npx sniff-qa --url https://myapp.com # scan source + specific URL
|
|
142
153
|
npx sniff-qa ./path/to/project # scan a specific directory
|
|
143
154
|
npx sniff-qa --ci # CI mode (JUnit output, no AI explorer)
|
|
155
|
+
npx sniff-qa --verbose # show classifier top 3 + matched signals
|
|
156
|
+
npx sniff-qa discover --dry-run # preview generated scenarios, no browser
|
|
157
|
+
npx sniff-qa discover --force-app-type saas # skip classification, treat as SaaS
|
|
144
158
|
```
|
|
145
159
|
|
|
146
|
-
Sniff auto-detects your dev server by reading `package.json` scripts and probing
|
|
160
|
+
Sniff auto-detects your dev server by reading `package.json` scripts plus `vite.config`, `nuxt.config`, `astro.config`, and `angular.json`, then probing the framework's default port *and* the next 20 after it (to catch auto-increment when the default is taken). If a port responds with framework markers (Next.js, Vite, Nuxt, Astro, Remix, SvelteKit, Angular), browser checks run automatically. Ghost-process ports like macOS AirPlay on `:5000` no longer collide.
|
|
147
161
|
|
|
148
162
|
---
|
|
149
163
|
|
|
@@ -186,6 +200,8 @@ sniff Scan source + auto-detect dev server
|
|
|
186
200
|
sniff --url <url> Scan source + test specific URL
|
|
187
201
|
sniff --url <url> --ci Full audit for CI pipelines
|
|
188
202
|
sniff <path> Scan a specific directory
|
|
203
|
+
sniff discover Autonomous E2E discovery (scenarios + edge cases)
|
|
204
|
+
sniff discover --regenerate-only Write sniff-scenarios/ tree without running
|
|
189
205
|
sniff fix Auto-fix safe issues (debugger, console.log)
|
|
190
206
|
sniff fix --check Dry run: show what would be fixed
|
|
191
207
|
sniff init Create sniff.config.ts
|
|
@@ -208,10 +224,15 @@ sniff --version Show version
|
|
|
208
224
|
| `--no-browser` | Source only even if `--url` is set |
|
|
209
225
|
| `--max-steps <n>` | Limit AI explorer steps (default: 50) |
|
|
210
226
|
| `--no-headless` | Show the browser window |
|
|
227
|
+
| `--headed` | Alias for `--no-headless` |
|
|
211
228
|
| `--format html,json,junit` | Choose report formats |
|
|
212
229
|
| `--fail-on critical,high` | Severities that cause non-zero exit |
|
|
213
230
|
| `--track-flakes` | Track test flakiness across runs |
|
|
214
231
|
| `--json` | JSON output for scripts |
|
|
232
|
+
| `--verbose` | Print classifier top-3 guesses and matched signals per dimension |
|
|
233
|
+
| `--dry-run` | Discover: generate scenarios without launching browser or writing reports |
|
|
234
|
+
| `--force-app-type <type>` | Discover: bypass the classifier (e.g. `saas`, `ecommerce`, `booking`) |
|
|
235
|
+
| `--app-type <types>` | Discover: filter classifier guesses to these app types (comma-separated) |
|
|
215
236
|
|
|
216
237
|
</details>
|
|
217
238
|
|
|
@@ -242,7 +263,9 @@ API discovery also supports **Fastify**, **Hono**, **tRPC**, and **GraphQL**.
|
|
|
242
263
|
Sniff works with zero config. Only create a config file if you want to customize.
|
|
243
264
|
|
|
244
265
|
```bash
|
|
245
|
-
npx sniff-qa init
|
|
266
|
+
npx sniff-qa init # auto-detects: .ts if TypeScript, .mjs if ESM, .js otherwise
|
|
267
|
+
npx sniff-qa init --ts # force TypeScript config
|
|
268
|
+
npx sniff-qa init --js # force JavaScript config (respects package.json "type")
|
|
246
269
|
```
|
|
247
270
|
|
|
248
271
|
<details>
|
|
@@ -337,6 +360,53 @@ Generates `.github/workflows/sniff.yml` with Playwright caching, JUnit output, a
|
|
|
337
360
|
|
|
338
361
|
---
|
|
339
362
|
|
|
363
|
+
## Autonomous discovery
|
|
364
|
+
|
|
365
|
+
```bash
|
|
366
|
+
sniff discover # run against the detected dev server
|
|
367
|
+
sniff discover --regenerate-only # write sniff-scenarios/ and exit (no run)
|
|
368
|
+
```
|
|
369
|
+
|
|
370
|
+
`sniff discover` reads your source (Prisma, Drizzle, TypeORM, Zod, GraphQL, OpenAPI, TS types), classifies what the app is (ecommerce, booking, saas, social, content, crm, auth-only, marketing, admin) with route tokens in **English, French, Spanish, German, Portuguese, and Italian** (a French SaaS on `/fr/tableau-de-bord` classifies correctly), generates happy-path journeys with real personas, enumerates edge variants (invalid email, XSS, payment declined, empty cart, offline, and more), and drives everything through Playwright.
|
|
371
|
+
|
|
372
|
+
Pass `--verbose` to see the top-3 candidates and every matched signal — useful for understanding *why* a classification happened. Pass `--force-app-type <type>` to skip classification entirely. Pass `--dry-run` to preview scenarios without launching a browser or writing reports.
|
|
373
|
+
|
|
374
|
+
**Multilingual projects supported.** The classifier matches French, Spanish, German, Portuguese (BR/PT), and Italian equivalents for every signature token, so a French SaaS on `/fr/tableau-de-bord` classifies the same as an English one on `/dashboard`.
|
|
375
|
+
|
|
376
|
+
**What you get:**
|
|
377
|
+
|
|
378
|
+
- `sniff-scenarios/_generated/<app-type>/<journey>.<variant>.scenario.md` — reviewable scenarios with JSON frontmatter. Track them in git. Hand-edit them — sniff detects the change and asks before regenerating.
|
|
379
|
+
- `sniff-reports/discovery/` — HTML, JSON, and JUnit reports per run.
|
|
380
|
+
- Flakiness quarantine for scenarios that fail 3 of the last 5 runs.
|
|
381
|
+
|
|
382
|
+
**Flags:**
|
|
383
|
+
|
|
384
|
+
| Flag | What it does |
|
|
385
|
+
|:-----|:-------------|
|
|
386
|
+
| `--url <url>` | Target URL (default: auto-detect dev server) |
|
|
387
|
+
| `--max-scenarios <n>` | Cap total scenarios (default: 50) |
|
|
388
|
+
| `--max-variants-per-scenario <n>` | Cap edge variants per scenario (default: 3) |
|
|
389
|
+
| `--max-variants-per-run <n>` | Cap edge variants per run (default: 40) |
|
|
390
|
+
| `--realism <profile>` | `robot`, `careful-user`, `casual-user` (default), `frustrated-user`, `power-user` |
|
|
391
|
+
| `--seed <n>` | Replay a specific random seed |
|
|
392
|
+
| `--only <filter>` | Filter scenarios by id substring or app type |
|
|
393
|
+
| `--app-type <types>` | Filter classifier guesses to these app types (comma-separated, kept for back-compat) |
|
|
394
|
+
| `--force-app-type <type>` | Bypass the classifier entirely — treat the app as this type |
|
|
395
|
+
| `--verbose` | Print classifier breakdown (top 3 guesses + matched signals per dimension) |
|
|
396
|
+
| `--dry-run` | Generate scenarios without launching the browser or writing reports |
|
|
397
|
+
| `--regenerate` | Regenerate `sniff-scenarios/` before running |
|
|
398
|
+
| `--regenerate-only` | Regenerate and exit (no browser) |
|
|
399
|
+
| `--force-regenerate` | Overwrite hand-edits without prompting |
|
|
400
|
+
| `--non-interactive` | Skip prompts and the production-URL countdown |
|
|
401
|
+
| `--verbose` | Print classification breakdown (top 3 guesses + matched signals per dimension) |
|
|
402
|
+
| `--no-llm` | Skip Claude Code CLI polish (deterministic only) |
|
|
403
|
+
|
|
404
|
+
**Production-URL safety:** when the target is not a local address, sniff prints a 5-second countdown banner before running. Hit Ctrl+C to cancel. No `--allow-destructive` flag.
|
|
405
|
+
|
|
406
|
+
**LLM polish (optional):** if the `claude` CLI is available, sniff uses it to break close-call app-type classifications. Responses are cached under `.sniff/discover/cache/`. Pass `--no-llm` to skip.
|
|
407
|
+
|
|
408
|
+
---
|
|
409
|
+
|
|
340
410
|
## How it works
|
|
341
411
|
|
|
342
412
|
<picture>
|