sniff-qa 0.3.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/README.md +75 -5
  2. package/dist/classifier-MWQMQ727.js +1701 -0
  3. package/dist/classifier-MWQMQ727.js.map +1 -0
  4. package/dist/cli/index.js +92 -27
  5. package/dist/cli/index.js.map +1 -1
  6. package/dist/dev-server-detector-XQ5AYPDT.js +319 -0
  7. package/dist/dev-server-detector-XQ5AYPDT.js.map +1 -0
  8. package/dist/discover-JRU5E4FR.js +327 -0
  9. package/dist/discover-JRU5E4FR.js.map +1 -0
  10. package/dist/domain-UQ34YE4B.js +1817 -0
  11. package/dist/domain-UQ34YE4B.js.map +1 -0
  12. package/dist/edge-cases-ZADKDZHL.js +246 -0
  13. package/dist/edge-cases-ZADKDZHL.js.map +1 -0
  14. package/dist/ensure-browsers-ACHCEQCT.js +60 -0
  15. package/dist/ensure-browsers-ACHCEQCT.js.map +1 -0
  16. package/dist/flakiness-OLJAY7CS.js +53 -0
  17. package/dist/flakiness-OLJAY7CS.js.map +1 -0
  18. package/dist/init-IQKT4ANC.js +148 -0
  19. package/dist/init-IQKT4ANC.js.map +1 -0
  20. package/dist/llm-VJAI6H6G.js +126 -0
  21. package/dist/llm-VJAI6H6G.js.map +1 -0
  22. package/dist/mcp/server.js +219 -1
  23. package/dist/mcp/server.js.map +1 -1
  24. package/dist/persistence-KTCRIGHU.js +270 -0
  25. package/dist/persistence-KTCRIGHU.js.map +1 -0
  26. package/dist/{repo-analyzer-WK5WP7NT.js → repo-analyzer-4OURSITY.js} +1 -1
  27. package/dist/{report-WLDV72FM.js → report-IKRVVIW4.js} +1 -1
  28. package/dist/report-SDZWX25B.js +280 -0
  29. package/dist/report-SDZWX25B.js.map +1 -0
  30. package/dist/runner-OYTN2NFD.js +996 -0
  31. package/dist/runner-OYTN2NFD.js.map +1 -0
  32. package/dist/scenarios-I7YYVTD2.js +987 -0
  33. package/dist/scenarios-I7YYVTD2.js.map +1 -0
  34. package/dist/tiebreak-L5UFNKU2.js +60 -0
  35. package/dist/tiebreak-L5UFNKU2.js.map +1 -0
  36. package/dist/{unified-7RKNG3EI.js → unified-JCUBP6UO.js} +2 -2
  37. package/package.json +2 -2
  38. package/dist/dev-server-detector-NJ5RN4IC.js +0 -150
  39. package/dist/dev-server-detector-NJ5RN4IC.js.map +0 -1
  40. package/dist/init-75MMIW4A.js +0 -66
  41. package/dist/init-75MMIW4A.js.map +0 -1
  42. /package/dist/{repo-analyzer-WK5WP7NT.js.map → repo-analyzer-4OURSITY.js.map} +0 -0
  43. /package/dist/{report-WLDV72FM.js.map → report-IKRVVIW4.js.map} +0 -0
  44. /package/dist/{unified-7RKNG3EI.js.map → unified-JCUBP6UO.js.map} +0 -0
package/README.md CHANGED
@@ -41,7 +41,18 @@ npx sniff-qa --url https://myapp.com # production URL
41
41
  Sniff ships as an MCP server. Add it to your editor, then ask your AI to scan.
42
42
 
43
43
  <details>
44
- <summary><b>Claude Code</b></summary>
44
+ <summary><b>Claude Code plugin marketplace</b> (one-command install)</summary>
45
+
46
+ ```bash
47
+ claude plugin marketplace add Aboudjem/10x
48
+ claude plugin install sniff@10x
49
+ ```
50
+
51
+ This installs sniff as a Claude Code plugin from the [10x marketplace](https://github.com/Aboudjem/10x). Skills, commands, and the MCP server are wired up automatically.
52
+ </details>
53
+
54
+ <details>
55
+ <summary><b>Claude Code (MCP server only)</b></summary>
45
56
 
46
57
  ```bash
47
58
  claude mcp add sniff-qa npx sniff-qa --mcp
@@ -110,9 +121,9 @@ clawhub install sniff-qa
110
121
  ```
111
122
  </details>
112
123
 
113
- Then ask: *"Scan this project for issues"* or *"Check accessibility on localhost:3000"*
124
+ Then ask: *"Scan this project for issues"* or *"Check accessibility on localhost:3000"* or *"Discover and run E2E tests"*
114
125
 
115
- **MCP tools:** `sniff_scan` (source) · `sniff_run` (browser) · `sniff_report` (results)
126
+ **MCP tools:** `sniff({ mode: "scan" | "run" | "discover" | "report" })` — unified entry point. Plus `sniff_install` when Playwright's Chromium is missing. The legacy per-mode tools (`sniff_scan`, `sniff_run`, `sniff_discover`, `sniff_report`) remain available but are deprecated in v0.5 and will be removed in v0.7.
116
127
 
117
128
  ### Install as a dev dependency
118
129
 
@@ -141,9 +152,12 @@ npx sniff-qa # scan source + auto-detect
141
152
  npx sniff-qa --url https://myapp.com # scan source + specific URL
142
153
  npx sniff-qa ./path/to/project # scan a specific directory
143
154
  npx sniff-qa --ci # CI mode (JUnit output, no AI explorer)
155
+ npx sniff-qa --verbose # show classifier top 3 + matched signals
156
+ npx sniff-qa discover --dry-run # preview generated scenarios, no browser
157
+ npx sniff-qa discover --force-app-type saas # skip classification, treat as SaaS
144
158
  ```
145
159
 
146
- Sniff auto-detects your dev server by reading `package.json` scripts and probing common ports (3000, 5173, 8080, 4200). If it finds a running server, browser checks run automatically.
160
+ Sniff auto-detects your dev server by reading `package.json` scripts plus `vite.config`, `nuxt.config`, `astro.config`, and `angular.json`, then probing the framework's default port *and* the next 20 after it (to catch auto-increment when the default is taken). If a port responds with framework markers (Next.js, Vite, Nuxt, Astro, Remix, SvelteKit, Angular), browser checks run automatically. Ghost-process ports like macOS AirPlay on `:5000` no longer collide.
147
161
 
148
162
  ---
149
163
 
@@ -186,6 +200,8 @@ sniff Scan source + auto-detect dev server
186
200
  sniff --url <url> Scan source + test specific URL
187
201
  sniff --url <url> --ci Full audit for CI pipelines
188
202
  sniff <path> Scan a specific directory
203
+ sniff discover Autonomous E2E discovery (scenarios + edge cases)
204
+ sniff discover --regenerate-only Write sniff-scenarios/ tree without running
189
205
  sniff fix Auto-fix safe issues (debugger, console.log)
190
206
  sniff fix --check Dry run: show what would be fixed
191
207
  sniff init Create sniff.config.ts
@@ -208,10 +224,15 @@ sniff --version Show version
208
224
  | `--no-browser` | Source only even if `--url` is set |
209
225
  | `--max-steps <n>` | Limit AI explorer steps (default: 50) |
210
226
  | `--no-headless` | Show the browser window |
227
+ | `--headed` | Alias for `--no-headless` |
211
228
  | `--format html,json,junit` | Choose report formats |
212
229
  | `--fail-on critical,high` | Severities that cause non-zero exit |
213
230
  | `--track-flakes` | Track test flakiness across runs |
214
231
  | `--json` | JSON output for scripts |
232
+ | `--verbose` | Print classifier top-3 guesses and matched signals per dimension |
233
+ | `--dry-run` | Discover: generate scenarios without launching browser or writing reports |
234
+ | `--force-app-type <type>` | Discover: bypass the classifier (e.g. `saas`, `ecommerce`, `booking`) |
235
+ | `--app-type <types>` | Discover: filter classifier guesses to these app types (comma-separated) |
215
236
 
216
237
  </details>
217
238
 
@@ -242,7 +263,9 @@ API discovery also supports **Fastify**, **Hono**, **tRPC**, and **GraphQL**.
242
263
  Sniff works with zero config. Only create a config file if you want to customize.
243
264
 
244
265
  ```bash
245
- npx sniff-qa init
266
+ npx sniff-qa init # auto-detects: .ts if TypeScript, .mjs if ESM, .js otherwise
267
+ npx sniff-qa init --ts # force TypeScript config
268
+ npx sniff-qa init --js # force JavaScript config (respects package.json "type")
246
269
  ```
247
270
 
248
271
  <details>
@@ -337,6 +360,53 @@ Generates `.github/workflows/sniff.yml` with Playwright caching, JUnit output, a
337
360
 
338
361
  ---
339
362
 
363
+ ## Autonomous discovery
364
+
365
+ ```bash
366
+ sniff discover # run against the detected dev server
367
+ sniff discover --regenerate-only # write sniff-scenarios/ and exit (no run)
368
+ ```
369
+
370
+ `sniff discover` reads your source (Prisma, Drizzle, TypeORM, Zod, GraphQL, OpenAPI, TS types), classifies what the app is (ecommerce, booking, saas, social, content, crm, auth-only, marketing, admin) with route tokens in **English, French, Spanish, German, Portuguese, and Italian** (a French SaaS on `/fr/tableau-de-bord` classifies correctly), generates happy-path journeys with real personas, enumerates edge variants (invalid email, XSS, payment declined, empty cart, offline, and more), and drives everything through Playwright.
371
+
372
+ Pass `--verbose` to see the top-3 candidates and every matched signal — useful for understanding *why* a classification happened. Pass `--force-app-type <type>` to skip classification entirely. Pass `--dry-run` to preview scenarios without launching a browser or writing reports.
373
+
374
+ **Multilingual projects supported.** The classifier matches French, Spanish, German, Portuguese (BR/PT), and Italian equivalents for every signature token, so a French SaaS on `/fr/tableau-de-bord` classifies the same as an English one on `/dashboard`.
375
+
376
+ **What you get:**
377
+
378
+ - `sniff-scenarios/_generated/<app-type>/<journey>.<variant>.scenario.md` — reviewable scenarios with JSON frontmatter. Track them in git. Hand-edit them — sniff detects the change and asks before regenerating.
379
+ - `sniff-reports/discovery/` — HTML, JSON, and JUnit reports per run.
380
+ - Flakiness quarantine for scenarios that fail 3 of the last 5 runs.
381
+
382
+ **Flags:**
383
+
384
+ | Flag | What it does |
385
+ |:-----|:-------------|
386
+ | `--url <url>` | Target URL (default: auto-detect dev server) |
387
+ | `--max-scenarios <n>` | Cap total scenarios (default: 50) |
388
+ | `--max-variants-per-scenario <n>` | Cap edge variants per scenario (default: 3) |
389
+ | `--max-variants-per-run <n>` | Cap edge variants per run (default: 40) |
390
+ | `--realism <profile>` | `robot`, `careful-user`, `casual-user` (default), `frustrated-user`, `power-user` |
391
+ | `--seed <n>` | Replay a specific random seed |
392
+ | `--only <filter>` | Filter scenarios by id substring or app type |
393
+ | `--app-type <types>` | Filter classifier guesses to these app types (comma-separated, kept for back-compat) |
394
+ | `--force-app-type <type>` | Bypass the classifier entirely — treat the app as this type |
395
+ | `--verbose` | Print classifier breakdown (top 3 guesses + matched signals per dimension) |
396
+ | `--dry-run` | Generate scenarios without launching the browser or writing reports |
397
+ | `--regenerate` | Regenerate `sniff-scenarios/` before running |
398
+ | `--regenerate-only` | Regenerate and exit (no browser) |
399
+ | `--force-regenerate` | Overwrite hand-edits without prompting |
400
+ | `--non-interactive` | Skip prompts and the production-URL countdown |
401
+ | `--verbose` | Print classification breakdown (top 3 guesses + matched signals per dimension) |
402
+ | `--no-llm` | Skip Claude Code CLI polish (deterministic only) |
403
+
404
+ **Production-URL safety:** when the target is not a local address, sniff prints a 5-second countdown banner before running. Hit Ctrl+C to cancel. No `--allow-destructive` flag.
405
+
406
+ **LLM polish (optional):** if the `claude` CLI is available, sniff uses it to break close-call app-type classifications. Responses are cached under `.sniff/discover/cache/`. Pass `--no-llm` to skip.
407
+
408
+ ---
409
+
340
410
  ## How it works
341
411
 
342
412
  <picture>