@matware/e2e-runner 1.3.0 → 1.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (56) hide show
  1. package/.claude-plugin/marketplace.json +37 -6
  2. package/.claude-plugin/plugin.json +17 -3
  3. package/LICENSE +190 -0
  4. package/README.md +151 -527
  5. package/agents/test-creator.md +4 -2
  6. package/agents/test-improver.md +5 -3
  7. package/bin/cli.js +84 -20
  8. package/commands/capture.md +45 -0
  9. package/package.json +3 -2
  10. package/skills/e2e-testing/SKILL.md +3 -2
  11. package/skills/e2e-testing/references/action-types.md +22 -4
  12. package/skills/e2e-testing/references/test-json-format.md +23 -0
  13. package/src/actions.js +321 -14
  14. package/src/ai-generate.js +81 -0
  15. package/src/app-pool.js +339 -0
  16. package/src/config.js +131 -7
  17. package/src/dashboard.js +209 -11
  18. package/src/db.js +74 -7
  19. package/src/index.js +6 -4
  20. package/src/learner-sqlite.js +154 -0
  21. package/src/learner.js +70 -3
  22. package/src/mcp-tools.js +259 -34
  23. package/src/module-analysis.js +247 -0
  24. package/src/module-resolver.js +35 -2
  25. package/src/narrate.js +42 -1
  26. package/src/pool-manager.js +68 -17
  27. package/src/pool.js +464 -37
  28. package/src/reporter.js +4 -1
  29. package/src/runner.js +410 -63
  30. package/src/visual-diff.js +515 -0
  31. package/src/websocket.js +14 -3
  32. package/src/wizard.js +184 -0
  33. package/templates/build-dashboard.js +3 -0
  34. package/templates/dashboard/js/api.js +62 -3
  35. package/templates/dashboard/js/init.js +46 -0
  36. package/templates/dashboard/js/keyboard.js +8 -7
  37. package/templates/dashboard/js/quicksearch.js +277 -0
  38. package/templates/dashboard/js/state.js +61 -7
  39. package/templates/dashboard/js/toast.js +1 -1
  40. package/templates/dashboard/js/utils.js +20 -0
  41. package/templates/dashboard/js/view-live.js +240 -9
  42. package/templates/dashboard/js/view-runs.js +540 -94
  43. package/templates/dashboard/js/view-tests.js +157 -16
  44. package/templates/dashboard/js/view-tools.js +234 -0
  45. package/templates/dashboard/js/view-watch.js +2 -2
  46. package/templates/dashboard/js/websocket.js +36 -0
  47. package/templates/dashboard/styles/base.css +489 -53
  48. package/templates/dashboard/styles/components.css +719 -77
  49. package/templates/dashboard/styles/view-live.css +463 -59
  50. package/templates/dashboard/styles/view-runs.css +793 -155
  51. package/templates/dashboard/styles/view-tests.css +440 -77
  52. package/templates/dashboard/styles/view-tools.css +206 -0
  53. package/templates/dashboard/styles/view-watch.css +198 -41
  54. package/templates/dashboard/template.html +369 -56
  55. package/templates/dashboard.html +5375 -901
  56. package/templates/docker-compose-lightpanda.yml +7 -0
package/README.md CHANGED
@@ -9,12 +9,16 @@
9
9
  </p>
10
10
 
11
11
  <p align="center">
12
- <img src="https://img.shields.io/npm/v/@matware/e2e-runner?color=blue" alt="npm version" />
12
+ <a href="https://www.npmjs.com/package/@matware/e2e-runner"><img src="https://img.shields.io/npm/v/@matware/e2e-runner?color=blue" alt="npm version" /></a>
13
13
  <img src="https://img.shields.io/node/v/@matware/e2e-runner" alt="node version" />
14
- <img src="https://img.shields.io/npm/l/@matware/e2e-runner" alt="license" />
14
+ <a href="https://www.npmjs.com/package/@matware/e2e-runner"><img src="https://img.shields.io/npm/dm/@matware/e2e-runner" alt="npm downloads" /></a>
15
+ <a href="https://hub.docker.com/r/fastslack/e2e-runner-mcp"><img src="https://img.shields.io/docker/pulls/fastslack/e2e-runner-mcp" alt="Docker pulls" /></a>
16
+ <a href="https://github.com/fastslack/mtw-e2e-runner/stargazers"><img src="https://img.shields.io/github/stars/fastslack/mtw-e2e-runner" alt="GitHub stars" /></a>
17
+ <a href="LICENSE"><img src="https://img.shields.io/npm/l/@matware/e2e-runner" alt="license" /></a>
15
18
  <img src="https://img.shields.io/badge/MCP-compatible-green" alt="MCP compatible" />
16
19
  <img src="https://img.shields.io/badge/AI--native-Claude%20Code-blueviolet" alt="AI native" />
17
20
  <img src="https://img.shields.io/badge/AI--native-OpenCode-orange" alt="OpenCode compatible" />
21
+ <a href="https://skills.sh"><img src="https://img.shields.io/badge/skills.sh-e2e--testing-ff6600" alt="Agent Skills" /></a>
18
22
  </p>
19
23
 
20
24
  <p align="center">
@@ -27,7 +31,7 @@
27
31
 
28
32
  But what makes it truly different is its **deep AI integration**. With a built-in [MCP server](https://modelcontextprotocol.io/), Claude Code can create tests from a conversation, run them, read the results, capture screenshots, and even visually verify that pages look correct — all without leaving the chat. Paste a GitHub issue URL and get a runnable test back. That's the workflow.
29
33
 
30
- ### This is a test
34
+ ### A test is just JSON
31
35
 
32
36
  ```json
33
37
  [
@@ -45,200 +49,89 @@ But what makes it truly different is its **deep AI integration**. With a built-i
45
49
  ]
46
50
  ```
47
51
 
48
- No imports. No `describe`/`it`. No compilation step. Just a JSON file that describes what a user does — and the runner makes it happen.
52
+ You describe what a user does click this, type that, check the page says X — and the runner does it in a real browser. No imports, no `describe`/`it`, no build step. If you can read it, you can write it.
49
53
 
50
54
  ---
51
55
 
52
- ## Getting Started
53
-
54
- ### Prerequisites
55
-
56
- - **Node.js** >= 20
57
- - **Docker** running (for the Chrome pool)
58
- - Your app running on a known port (e.g. `http://localhost:3000`)
59
-
60
- > **Why `host.docker.internal`?**
61
- >
62
- > Chrome runs inside a Docker container. From inside the container, `localhost` refers to the container itself — not your machine. The special hostname `host.docker.internal` resolves to your host machine, so Chrome can reach your locally running app.
63
- >
64
- > The default `baseUrl` is `http://host.docker.internal:3000`. If your app runs on a different port, change it in `e2e.config.js` after init.
65
- >
66
- > **Linux note:** On Docker Engine (not Docker Desktop), you may need to add `--add-host=host.docker.internal:host-gateway` to the Docker run flags, or use your machine's LAN IP directly as the `baseUrl`.
56
+ ## Agent Skills
67
57
 
68
- ---
69
-
70
- ### Path A: With Claude Code
71
-
72
- If you use [Claude Code](https://docs.anthropic.com/en/docs/claude-code), this is the fastest path — Claude handles test creation and debugging for you.
73
-
74
- **1. Install the package**
58
+ Install E2E testing skills for any coding agent (Claude Code, Cursor, Codex, Copilot, and [40+ more](https://github.com/vercel-labs/skills#supported-agents)):
75
59
 
76
60
  ```bash
77
- npm install --save-dev @matware/e2e-runner
78
- ```
79
-
80
- **2. Scaffold the project structure**
81
-
82
- ```bash
83
- npx e2e-runner init
84
- ```
85
-
86
- This creates `e2e/tests/` with a sample test and `e2e/screenshots/` for captures.
87
-
88
- **3. Configure your base URL**
89
-
90
- Edit `e2e.config.js` and set `baseUrl` to match your app's port:
91
-
92
- ```js
93
- export default {
94
- baseUrl: 'http://host.docker.internal:3000', // change 3000 to your port
95
- };
96
- ```
97
-
98
- **4. Start the Chrome pool**
99
-
100
- ```bash
101
- npx e2e-runner pool start
102
- ```
103
-
104
- You should see:
105
-
106
- ```
107
- ✓ Chrome pool started on port 3333 (max 3 sessions)
108
- ```
109
-
110
- **5. Install the Claude Code plugin**
111
-
112
- ```bash
113
- # Add the marketplace (one-time)
114
- claude plugin marketplace add fastslack/mtw-e2e-runner
115
-
116
- # Install the plugin
117
- claude plugin install e2e-runner@matware
61
+ npx skills add fastslack/mtw-e2e-runner
118
62
  ```
119
63
 
120
- The plugin gives Claude 13 MCP tools, a workflow skill, 3 slash commands, and 3 specialized agents.
64
+ This gives your agent the knowledge to create, run, and debug JSON-driven E2E tests no documentation reading required.
121
65
 
122
- **6. Ask Claude to run the sample test**
123
-
124
- In Claude Code, just say:
125
-
126
- > "Run all E2E tests"
127
-
128
- Claude will check the pool, run the sample test, and report back:
129
-
130
- ```
131
- ==================================================
132
- E2E RESULTS
133
- ==================================================
134
- Total: 1
135
- Passed: 1
136
- Failed: 0
137
- Rate: 100.00%
138
- Duration: 1.23s
139
- ==================================================
140
- ```
141
-
142
- From here, you can ask Claude to create new tests ("test the login flow"), debug failures, or verify GitHub issues.
66
+ > Browse all available skills at [skills.sh](https://skills.sh)
143
67
 
144
68
  ---
145
69
 
146
- ### Path B: CLI Only
70
+ ## Getting Started
147
71
 
148
- No AI requireduse the runner directly from your terminal.
72
+ You need just two things: **Node.js 20+** and **Docker running**. You don't install any browser — the runner spins up Chrome in a container for you.
149
73
 
150
- **1. Install the package**
74
+ ### Try it in 60 seconds
151
75
 
152
76
  ```bash
153
77
  npm install --save-dev @matware/e2e-runner
78
+ npx e2e-runner init # scaffolds e2e/ with a sample test + config
79
+ npx e2e-runner run --all # runs it — Chrome starts automatically on first run
154
80
  ```
155
81
 
156
- **2. Scaffold the project structure**
82
+ That's the whole setup. No separate `pool start`, no browser download: the first run boots the Chrome pool for you and reuses it afterwards.
157
83
 
158
- ```bash
159
- npx e2e-runner init
160
- ```
84
+ > Prefer a single command? `curl -fsSL https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/scripts/quickstart.sh | bash`
161
85
 
162
- This creates `e2e/tests/` with a sample test and `e2e/screenshots/` for captures.
86
+ ### Point it at your app
163
87
 
164
- **3. Configure your base URL**
165
-
166
- Edit `e2e.config.js` and set `baseUrl` to match your app's port:
88
+ `init` created `e2e.config.js`. Set your app's URL there:
167
89
 
168
90
  ```js
169
91
  export default {
170
- baseUrl: 'http://host.docker.internal:3000', // change 3000 to your port
92
+ baseUrl: 'http://host.docker.internal:3000', // change 3000 to your app's port
171
93
  };
172
94
  ```
173
95
 
174
- **4. Start the Chrome pool**
96
+ <details>
97
+ <summary><strong>Why <code>host.docker.internal</code> instead of <code>localhost</code>?</strong></summary>
175
98
 
176
- ```bash
177
- npx e2e-runner pool start
178
- ```
99
+ Chrome runs inside Docker, so `localhost` there points at the container, not your machine. `host.docker.internal` bridges to your host. On Linux (Docker Engine, not Docker Desktop) you may need to add `--add-host=host.docker.internal:host-gateway`, or just use your machine's LAN IP.
100
+ </details>
179
101
 
180
- You should see:
102
+ ### Write your first test
181
103
 
182
- ```
183
- ✓ Chrome pool started on port 3333 (max 3 sessions)
184
- ```
185
-
186
- **5. Run the sample test**
187
-
188
- ```bash
189
- npx e2e-runner run --all
190
- ```
191
-
192
- Expected output:
193
-
194
- ```
195
- ==================================================
196
- E2E RESULTS
197
- ==================================================
198
- Total: 1
199
- Passed: 1
200
- Failed: 0
201
- Rate: 100.00%
202
- Duration: 1.23s
203
- ==================================================
204
- ```
205
-
206
- A screenshot is saved at `e2e/screenshots/homepage.png`.
207
-
208
- **6. Write your first real test**
209
-
210
- Create `e2e/tests/my-first-test.json`:
104
+ Open `e2e/tests/sample.json` and describe a flow as a list of actions:
211
105
 
212
106
  ```json
213
107
  [
214
- {
215
- "name": "homepage-visible",
216
- "actions": [
217
- { "type": "goto", "value": "/" },
218
- { "type": "assert_visible", "selector": "body" },
219
- { "type": "screenshot", "value": "my-first-test.png" }
220
- ]
221
- }
108
+ { "name": "homepage loads", "actions": [
109
+ { "type": "goto", "value": "/" },
110
+ { "type": "assert_text", "text": "Welcome" },
111
+ { "type": "screenshot", "value": "home.png" }
112
+ ]}
222
113
  ]
223
114
  ```
224
115
 
225
- Run it:
116
+ Then `npx e2e-runner run --all` again. Pass/fail, timing, screenshots, and network errors print to your terminal — and to the [web dashboard](#web-dashboard) if it's open.
117
+
118
+ ### Add Claude Code (optional)
226
119
 
227
120
  ```bash
228
- npx e2e-runner run --suite my-first-test
121
+ claude plugin marketplace add fastslack/mtw-e2e-runner
122
+ claude plugin install e2e-runner@matware
229
123
  ```
230
124
 
231
- ---
232
-
233
- ### One-liner quickstart
125
+ This gives Claude 17 MCP tools, slash commands, and specialized agents. Just say *"Run all E2E tests"* or *"Create a test for the login flow"*.
234
126
 
235
- If you want to skip the step-by-step and get everything running in one command:
127
+ ### Add OpenCode (optional)
236
128
 
237
129
  ```bash
238
- curl -fsSL https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/scripts/quickstart.sh | bash
130
+ cp node_modules/@matware/e2e-runner/opencode.json ./
131
+ mkdir -p .opencode && cp -r node_modules/@matware/e2e-runner/.opencode/* .opencode/
239
132
  ```
240
133
 
241
- > This installs the package, scaffolds the project, and starts the Chrome pool. You'll still need to configure your `baseUrl` afterwards.
134
+ See [OPENCODE.md](OPENCODE.md) for details.
242
135
 
243
136
  ### What's next?
244
137
 
@@ -254,7 +147,7 @@ curl -fsSL https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/scrip
254
147
 
255
148
  🧪 **Zero-code tests** — JSON files that anyone on your team can read and write. No JavaScript, no compilation, no framework lock-in.
256
149
 
257
- 🤖 **AI-powered testing** — Claude Code creates, executes, and debugs tests natively through 13 MCP tools. Ask it to "test the checkout flow" and it builds the JSON, runs it, and reports back.
150
+ 🤖 **AI-powered testing** — Claude Code creates, executes, and debugs tests natively through 17 MCP tools. Ask it to "test the checkout flow" and it builds the JSON, runs it, and reports back.
258
151
 
259
152
  🐛 **Issue-to-Test pipeline** — Paste a GitHub or GitLab issue URL. The runner fetches it, generates E2E tests, runs them, and tells you: *bug confirmed* or *not reproducible*.
260
153
 
@@ -262,7 +155,9 @@ curl -fsSL https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/scrip
262
155
 
263
156
  🧠 **Learning system** — Tracks test stability across runs. Detects flaky tests, unstable selectors, slow APIs, and error patterns — then surfaces actionable insights.
264
157
 
265
- ⚡ **Parallel execution** — Run N tests simultaneously against a shared Chrome pool (browserless/chrome). Serial mode available for tests that share state.
158
+ ⚡ **Parallel execution** — Run N tests simultaneously against a shared browser pool (browserless, raw CDP, Lightpanda, Obscura, or Steel). Serial mode available for tests that share state.
159
+
160
+ 🎯 **Pluggable browser drivers** — Pick the engine that fits each test: real Chrome via browserless, Lightpanda or Obscura for fast lightweight runs, Steel for managed sessions. Set `driver` per test or override the whole run with `--driver`.
266
161
 
267
162
  📊 **Real-time dashboard** — Live execution view, run history with pass-rate charts, screenshot gallery with hash-based search, expandable network request logs.
268
163
 
@@ -303,9 +198,9 @@ Suite files can have numeric prefixes for ordering (`01-auth.json`, `02-dashboar
303
198
  | Action | Fields | Description |
304
199
  |--------|--------|-------------|
305
200
  | `goto` | `value` | Navigate to URL (relative to `baseUrl` or absolute) |
306
- | `click` | `selector` or `text` | Click by CSS selector or visible text content |
201
+ | `click` | `selector` or `text` | Click by CSS selector or visible text content. Text mode also takes `scope: "dialog"`, `visible: true`, `last: true` |
307
202
  | `type` / `fill` | `selector`, `value` | Clear field and type text |
308
- | `wait` | `selector`, `text`, or `value` (ms) | Wait for element, text, or fixed delay |
203
+ | `wait` | `selector`, `text`, `gone`, or `value` (ms) | Wait for element/text to appear, for `gone` to disappear (spinner/dialog), or fixed delay. Prefer conditions over fixed `value` sleeps |
309
204
  | `screenshot` | `value` (filename) | Capture a screenshot |
310
205
  | `select` | `selector`, `value` | Select a dropdown option |
311
206
  | `clear` | `selector` | Clear an input field |
@@ -352,9 +247,10 @@ These actions handle common patterns in React/MUI apps that normally require ver
352
247
 
353
248
  | Action | Fields | Description |
354
249
  |--------|--------|-------------|
355
- | `type_react` | `selector`, `value` | Type into React controlled inputs using the native value setter. Dispatches `input` + `change` events so React state updates correctly. |
250
+ | `type_react` | `selector`, `value`, optional `blur`, `waitAfter` | Type into React controlled inputs using the native value setter. Dispatches `input` + `change` events so React state updates correctly. `blur: true` commits on blur; `waitAfter: "<ms>"` waits after (debounced autocomplete). |
356
251
  | `click_regex` | `text` (regex), optional `selector`, optional `value: "last"` | Click element whose textContent matches a regex (case-insensitive). Default: first match. Use `value: "last"` for last match. |
357
252
  | `click_option` | `text` | Click a `[role="option"]` element by text — common in autocomplete/select dropdowns. |
253
+ | `select_combobox` | `text`, optional `selector`, `filter`, `openWait`/`filterWait`/`waitAfter` | Open a MUI Autocomplete/Select, optionally type `filter`, then click the option matching `text`. Falls back across `[role="option"]`, `.MuiAutocomplete-option`, `li.MuiMenuItem-root`. |
358
254
  | `focus_autocomplete` | `text` (label text) | Focus an autocomplete input by its label text. Supports MUI and generic `[role="combobox"]`. |
359
255
  | `click_chip` | `text` | Click a chip/tag element by text. Searches `[class*="Chip"]`, `[class*="chip"]`, `[data-chip]`. |
360
256
 
@@ -408,11 +304,7 @@ Serial tests run one at a time **after** all parallel tests finish — preventin
408
304
 
409
305
  ## Testing Authenticated Apps
410
306
 
411
- Most real-world apps require login before tests can interact with protected pages. E2E Runner provides multiple strategies — choose the one that matches your app's auth mechanism.
412
-
413
- ### Strategy 1: UI Login Flow (any app)
414
-
415
- The most universal approach — fill in the login form like a real user. Works with **any** authentication system (session cookies, JWT, OAuth redirect, etc.):
307
+ The simplest approach log in via the UI like a real user:
416
308
 
417
309
  ```json
418
310
  {
@@ -425,279 +317,29 @@ The most universal approach — fill in the login form like a real user. Works w
425
317
  { "type": "wait", "selector": ".dashboard" }
426
318
  ]
427
319
  },
428
- "tests": [
429
- {
430
- "name": "profile-page",
431
- "actions": [
432
- { "type": "goto", "value": "/profile" },
433
- { "type": "assert_text", "text": "My Profile" }
434
- ]
435
- }
436
- ]
437
- }
438
- ```
439
-
440
- > **When to use:** You don't know or care how auth works internally. The browser handles cookies/tokens automatically after login — just like a real user.
441
-
442
- ### Strategy 2: JWT Token Injection (SPAs)
443
-
444
- For single-page apps that store JWT tokens in `localStorage` or `sessionStorage`. Skip the login form entirely by injecting the token directly:
445
-
446
- ```json
447
- {
448
- "hooks": {
449
- "beforeEach": [
450
- { "type": "goto", "value": "/" },
451
- { "type": "set_storage", "value": "accessToken=eyJhbGciOiJIUzI1NiIs..." },
452
- { "type": "goto", "value": "/dashboard" },
453
- { "type": "wait", "selector": ".dashboard-loaded" }
454
- ]
455
- },
456
320
  "tests": [...]
457
321
  }
458
322
  ```
459
323
 
460
- **Common storage key names** (depends on your app):
461
-
462
- | Framework / Library | Typical key | Storage |
463
- |---------------------|-------------|---------|
464
- | Custom JWT | `accessToken`, `token`, `jwt` | localStorage |
465
- | Auth0 SPA SDK | `@@auth0spajs@@::*` | localStorage |
466
- | Firebase Auth | `firebase:authUser:*` | localStorage |
467
- | AWS Amplify | `CognitoIdentityServiceProvider.*` | localStorage |
468
- | Supabase | `sb-<ref>-auth-token` | localStorage |
469
- | NextAuth (client) | `next-auth.session-token` | cookie (see Strategy 4) |
470
-
471
- **Using `sessionStorage` instead:**
472
-
473
- ```json
474
- { "type": "set_storage", "value": "token=eyJhbG...", "selector": "session" }
475
- ```
476
-
477
- **Asserting the token was stored correctly:**
324
+ For SPAs with JWT, skip the login form by injecting the token directly:
478
325
 
479
326
  ```json
480
- { "type": "assert_storage", "value": "accessToken" }
481
- { "type": "assert_storage", "value": "accessToken=eyJhbG..." }
327
+ { "type": "set_storage", "value": "accessToken=eyJhbGciOiJIUzI1NiIs..." }
482
328
  ```
483
329
 
484
- > **When to use:** Your SPA reads auth tokens from browser storage. Fastest strategy — no network round-trip for login.
485
-
486
- ### Strategy 3: Config-Level Auth Token
487
-
488
- For apps where every test needs the same JWT token. Set it once in config — it's injected into `localStorage` before every `e2e_capture` and `e2e_issue --verify` run:
330
+ Or set it globally in config:
489
331
 
490
332
  ```js
491
333
  // e2e.config.js
492
334
  export default {
493
335
  authToken: 'eyJhbGciOiJIUzI1NiIs...',
494
- authStorageKey: 'accessToken', // default
336
+ authStorageKey: 'accessToken',
495
337
  };
496
338
  ```
497
339
 
498
- Or via environment variables:
499
-
500
- ```bash
501
- AUTH_TOKEN="eyJhbGciOiJIUzI1NiIs..." npx e2e-runner run --all
502
- ```
503
-
504
- Or via CLI:
505
-
506
- ```bash
507
- npx e2e-runner run --all --auth-token "eyJhbG..." --auth-storage-key "jwt"
508
- ```
509
-
510
- MCP tools (`e2e_capture`, `e2e_issue`) also accept `authToken` and `authStorageKey` per call.
511
-
512
- > **When to use:** All tests share the same user session and your app uses JWT in localStorage.
513
-
514
- ### Strategy 4: Cookie-Based Auth (server-rendered apps)
515
-
516
- For apps that use HTTP cookies (Rails, Django, Laravel, Express sessions, NextAuth, etc.). Use `evaluate` to set cookies before navigating:
517
-
518
- ```json
519
- {
520
- "hooks": {
521
- "beforeEach": [
522
- { "type": "goto", "value": "/" },
523
- { "type": "evaluate", "value": "document.cookie = 'session_id=abc123; path=/; SameSite=Lax'" },
524
- { "type": "goto", "value": "/dashboard" }
525
- ]
526
- },
527
- "tests": [...]
528
- }
529
- ```
530
-
531
- **Multiple cookies:**
532
-
533
- ```json
534
- { "type": "evaluate", "value": "document.cookie = 'session_id=abc123; path=/'; document.cookie = '_csrf_token=xyz789; path=/'" }
535
- ```
536
-
537
- **For `HttpOnly` cookies** (can't be set via JavaScript), use the UI login strategy instead — the browser will store them automatically.
538
-
539
- > **When to use:** Traditional server-rendered apps, or any app that authenticates via cookies.
540
-
541
- ### Strategy 5: HTTP Header Auth (API tests)
542
-
543
- For API testing where you need to send `Authorization` headers with every request. Use `evaluate` to override `fetch`/`XMLHttpRequest`:
544
-
545
- ```json
546
- {
547
- "hooks": {
548
- "beforeEach": [
549
- { "type": "goto", "value": "/" },
550
- { "type": "evaluate", "value": "const origFetch = window.fetch; window.fetch = (url, opts = {}) => { opts.headers = { ...opts.headers, 'Authorization': 'Bearer eyJhbG...' }; return origFetch(url, opts); }" }
551
- ]
552
- },
553
- "tests": [
554
- {
555
- "name": "api-returns-user",
556
- "actions": [
557
- { "type": "evaluate", "value": "const res = await fetch('/api/me'); const data = await res.json(); if (data.email !== 'test@example.com') throw new Error('Wrong user: ' + data.email)" }
558
- ]
559
- }
560
- ]
561
- }
562
- ```
340
+ Each test runs in a **fresh browser context**, so auth state is automatically clean between tests.
563
341
 
564
- > **When to use:** API-level tests (with `--test-type api`) that need auth headers.
565
-
566
- ### Strategy 6: OAuth / SSO (external provider)
567
-
568
- OAuth flows redirect to external providers (Google, GitHub, Okta, etc.) which can't be automated reliably. Common workarounds:
569
-
570
- **Option A — Test environment bypass:** Most apps have a direct login endpoint for testing that skips OAuth:
571
-
572
- ```json
573
- { "type": "goto", "value": "/auth/test-login?user=test@example.com" }
574
- ```
575
-
576
- **Option B — Pre-authenticated token:** Get a token from your auth provider's API and inject it:
577
-
578
- ```json
579
- {
580
- "hooks": {
581
- "beforeEach": [
582
- { "type": "goto", "value": "/" },
583
- { "type": "set_storage", "value": "oidc.user:https://auth.example.com:client_id={\"access_token\":\"...\"}" }
584
- ]
585
- }
586
- }
587
- ```
588
-
589
- **Option C — Session cookie from CI:** If your CI can authenticate via API, pass the session cookie as an env var:
590
-
591
- ```bash
592
- SESSION=$(curl -s -c - https://api.example.com/auth/login -d '{"email":"test@example.com","password":"secret"}' | grep session_id | awk '{print $NF}')
593
- AUTH_TOKEN="$SESSION" AUTH_STORAGE_KEY="session_id" npx e2e-runner run --all
594
- ```
595
-
596
- > **When to use:** Apps with Google/GitHub/Okta/Auth0 login. You almost always need a test-environment backdoor.
597
-
598
- ### Reusable Auth Modules
599
-
600
- Extract your auth strategy into a module so every test can reference it without duplication:
601
-
602
- ```json
603
- // e2e/modules/login.json — UI login (universal)
604
- {
605
- "$module": "login",
606
- "description": "Log in via the UI login form",
607
- "params": {
608
- "email": { "required": true, "description": "User email" },
609
- "password": { "required": true, "description": "User password" },
610
- "redirectTo": { "default": "/dashboard", "description": "Page to land on after login" }
611
- },
612
- "actions": [
613
- { "type": "goto", "value": "/login" },
614
- { "type": "type", "selector": "#email", "value": "{{email}}" },
615
- { "type": "type", "selector": "#password", "value": "{{password}}" },
616
- { "type": "click", "text": "Sign In" },
617
- { "type": "wait", "selector": "{{redirectTo}}" }
618
- ]
619
- }
620
- ```
621
-
622
- ```json
623
- // e2e/modules/auth-token.json — JWT injection (SPAs)
624
- {
625
- "$module": "auth-token",
626
- "description": "Inject an auth token into browser storage",
627
- "params": {
628
- "token": { "required": true, "description": "JWT or session token" },
629
- "storageKey": { "default": "accessToken", "description": "Storage key name" },
630
- "storage": { "default": "local", "description": "local or session" },
631
- "redirectTo": { "default": "/dashboard", "description": "Page to navigate to after injection" }
632
- },
633
- "actions": [
634
- { "type": "goto", "value": "/" },
635
- { "type": "set_storage", "value": "{{storageKey}}={{token}}", "selector": "{{#storage}}{{storage}}{{/storage}}" },
636
- { "type": "goto", "value": "{{redirectTo}}" }
637
- ]
638
- }
639
- ```
640
-
641
- Use in tests:
642
-
643
- ```json
644
- // UI login
645
- { "$use": "login", "params": { "email": "admin@test.com", "password": "secret" } }
646
-
647
- // Token injection
648
- { "$use": "auth-token", "params": { "token": "eyJhbG..." } }
649
-
650
- // Token in sessionStorage, redirect to /settings
651
- { "$use": "auth-token", "params": { "token": "eyJhbG...", "storage": "session", "redirectTo": "/settings" } }
652
- ```
653
-
654
- ### Testing Different User Roles
655
-
656
- Use separate tests (or the same module with different credentials) to test role-based access:
657
-
658
- ```json
659
- [
660
- {
661
- "name": "admin-sees-settings",
662
- "actions": [
663
- { "$use": "login", "params": { "email": "admin@test.com", "password": "admin-pass" } },
664
- { "type": "goto", "value": "/settings" },
665
- { "type": "assert_visible", "selector": ".admin-panel" }
666
- ]
667
- },
668
- {
669
- "name": "viewer-cannot-access-settings",
670
- "actions": [
671
- { "$use": "login", "params": { "email": "viewer@test.com", "password": "viewer-pass" } },
672
- { "type": "goto", "value": "/settings" },
673
- { "type": "assert_text", "text": "Access Denied" }
674
- ]
675
- }
676
- ]
677
- ```
678
-
679
- ### Clearing Auth State
680
-
681
- Each test runs in a **fresh browser context** (new connection to the Chrome pool), so cookies and storage are automatically clean. If you need to explicitly clear state mid-test:
682
-
683
- ```json
684
- { "type": "clear_cookies" }
685
- ```
686
-
687
- This clears cookies, localStorage, and sessionStorage for the current origin.
688
-
689
- ### Quick Reference
690
-
691
- | Auth type | Strategy | Key actions |
692
- |-----------|----------|-------------|
693
- | Username/password form | UI Login | `goto` + `type` + `click` in `beforeEach` |
694
- | JWT in localStorage | Token Injection | `set_storage` in `beforeEach` |
695
- | JWT in sessionStorage | Token Injection | `set_storage` with `selector: "session"` |
696
- | Session cookies | Cookie | `evaluate` to set `document.cookie` |
697
- | HttpOnly cookies | UI Login | Must go through login form |
698
- | OAuth / SSO | Test bypass | App-specific test login endpoint |
699
- | API auth headers | Header Override | `evaluate` to patch `fetch` |
700
- | Config-level token | Config | `authToken` + `authStorageKey` in config |
342
+ > **More strategies:** Cookie-based auth, HTTP header injection, OAuth/SSO bypasses, reusable auth modules, and role-based testing — see [docs/authentication.md](docs/authentication.md)
701
343
 
702
344
  ---
703
345
 
@@ -892,152 +534,132 @@ Monitor Chrome pool health: available slots, running sessions, memory pressure.
892
534
 
893
535
  ---
894
536
 
895
- ## Screenshot Capture
537
+ ## Browser Drivers
896
538
 
897
- Capture screenshots of any URL on demandno test suite required:
539
+ The runner can talk to multiple browser engines through different drivers. The default is **`auto`** it probes each pool URL and picks the right driver per pool.
898
540
 
899
- ```bash
900
- e2e-runner capture https://example.com
901
- e2e-runner capture https://example.com --full-page --selector ".loaded" --delay 2000
902
- ```
541
+ | Driver | Engine | Detection probe | When to use |
542
+ |--------|--------|-----------------|-------------|
543
+ | `browserless` | Real Chromium via [browserless](https://www.browserless.io/) | `/pressure` returns JSON | Default. Production-grade JS execution, screencast, full Chrome behavior |
544
+ | `cdp` | Generic CDP-compatible (raw Chrome, etc.) | `/json/version` reachable | Fallback for any CDP server that isn't one of the others |
545
+ | `lightpanda` | [Lightpanda](https://lightpanda.io) (Zig) | `/json/version` Browser=lightpanda | ~9× faster, ~16× less memory than headless Chrome — ideal for high-volume scrape-style tests |
546
+ | `obscura` | [Obscura](https://github.com/h4ckf0r0day/obscura) (Rust + V8) | `/json/version` Browser=obscura | ~30 MB RAM footprint, built-in anti-detection (`--stealth`), stays close to real Chrome via Puppeteer |
547
+ | `steel` | [Steel Browser](https://steel.dev) | `/v1/sessions` returns JSON | Managed session lifecycle, REST API for orchestration |
903
548
 
904
- Via MCP, the `e2e_capture` tool supports `authToken` and `authStorageKey` for authenticated pages — it injects the token into localStorage before navigating.
549
+ ### Pick a driver per test
905
550
 
906
- Every screenshot gets a deterministic hash (`ss:a3f2b1c9`). Use `e2e_screenshot` to retrieve any screenshot by hash — it returns the image with metadata (test name, step, type).
551
+ ```json
552
+ {
553
+ "tests": [
554
+ {
555
+ "name": "checkout flow (heavy JS, real Chrome)",
556
+ "driver": "browserless",
557
+ "actions": [...]
558
+ },
559
+ {
560
+ "name": "scrape product page (lightweight)",
561
+ "driver": "obscura",
562
+ "fallbackDriver": "cdp",
563
+ "actions": [...]
564
+ }
565
+ ]
566
+ }
567
+ ```
907
568
 
908
- ---
569
+ `driver` is optional. If set, only pools whose detected driver matches become candidates. `fallbackDriver` is **explicit opt-in** — without it, a missing target driver fails the test with a clear message. Pool busyness does **not** trigger fallback; the runner waits inside the filtered set.
909
570
 
910
- ## Claude Code Integration
571
+ ### Force a driver for a whole run
911
572
 
912
- The package ships as a **Claude Code plugin** — a single install that gives Claude native access to the test runner, teaches it the optimal workflow, and adds slash commands and specialized agents.
573
+ ```bash
574
+ e2e-runner run --all --driver obscura
575
+ e2e-runner run --all --driver obscura --fallback-driver cdp
576
+ ```
913
577
 
914
- ### Install as Plugin (recommended)
578
+ CLI overrides win over per-test fields — useful for A/B benchmarks against the same suite.
579
+
580
+ ### Running each driver locally
915
581
 
916
582
  ```bash
917
- # 1. Add the marketplace (one-time)
918
- claude plugin marketplace add fastslack/mtw-e2e-runner
583
+ # browserless (default) managed by `pool start`
584
+ e2e-runner pool start
919
585
 
920
- # 2. Install the plugin
921
- claude plugin install e2e-runner@matware
922
- ```
586
+ # Lightpanda pool start uses templates/docker-compose-lightpanda.yml
587
+ e2e-runner pool start # with poolDriver: 'lightpanda' in config
923
588
 
924
- **What you get:**
589
+ # Obscura — install the binary and run it yourself
590
+ curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-linux.tar.gz
591
+ tar xzf obscura-x86_64-linux.tar.gz
592
+ ./obscura serve --port 9222 --stealth
593
+ # then point the runner at it: poolUrls: ['http://localhost:9222'], poolDriver: 'obscura'
594
+ ```
925
595
 
926
- | Component | Description |
927
- |-----------|-------------|
928
- | **13 MCP tools** | Run tests, create test files, capture screenshots, query network logs, manage dashboard, verify issues, query learnings |
929
- | **Skill** | Teaches Claude the full e2e-runner workflow — how to combine tools, interpret results, debug failures, create tests |
930
- | **3 Commands** | `/e2e-runner:run` — run & analyze tests<br>`/e2e-runner:create-test` — explore UI and create tests<br>`/e2e-runner:verify-issue <url>` — verify GitHub/GitLab bugs |
931
- | **3 Agents** | **test-analyzer** — diagnoses failures, analyzes flaky tests, drills into network errors<br>**test-creator** — explores UI, discovers selectors, designs and validates tests<br>**test-improver** — refactors verbose evaluate actions, extracts modules, adds waits/retries, eliminates hardcoded delays |
596
+ ---
932
597
 
933
- ### Install MCP-only (alternative)
598
+ ## Screenshot Capture
934
599
 
935
- If you only want the 13 MCP tools without skills, commands, or agents:
600
+ Capture screenshots of any URL on demand no test suite required:
936
601
 
937
602
  ```bash
938
- claude mcp add --transport stdio --scope user e2e-runner \
939
- -- npx -y -p @matware/e2e-runner e2e-runner-mcp
603
+ e2e-runner capture https://example.com
604
+ e2e-runner capture https://example.com --full-page --selector ".loaded" --delay 2000
940
605
  ```
941
606
 
942
- ### Slash Commands
943
-
944
- | Command | Description |
945
- |---------|-------------|
946
- | `/e2e-runner:run` | Check pool, list suites, run tests, analyze results with screenshots and network drill-down |
947
- | `/e2e-runner:create-test` | Explore the UI with screenshots, find selectors in source code, design test actions, create and validate |
948
- | `/e2e-runner:verify-issue <url>` | Fetch a GitHub/GitLab issue, create tests that verify correct behavior, report bug confirmed or not reproducible |
949
-
950
- ### MCP Tools
607
+ Via MCP, the `e2e_capture` tool supports `authToken` and `authStorageKey` for authenticated pages — it injects the token into localStorage before navigating.
951
608
 
952
- | Tool | Description |
953
- |------|-------------|
954
- | `e2e_run` | Run tests: all suites, by name, or by file. Supports `concurrency`, `baseUrl`, `retries`, `failOnNetworkError` overrides. Returns verification results if tests have `expect`. |
955
- | `e2e_list` | List available test suites with test names and counts |
956
- | `e2e_create_test` | Create a new test JSON file with name, tests, and optional hooks |
957
- | `e2e_create_module` | Create a reusable module with parameterized actions |
958
- | `e2e_pool_status` | Check Chrome pool availability, running sessions, capacity |
959
- | `e2e_screenshot` | Retrieve a screenshot by hash (`ss:a3f2b1c9`). Returns image + metadata |
960
- | `e2e_capture` | Capture screenshot of any URL. Supports `authToken`, `fullPage`, `selector`, `delay` |
961
- | `e2e_dashboard_start` | Start the web dashboard |
962
- | `e2e_dashboard_stop` | Stop the web dashboard |
963
- | `e2e_issue` | Fetch GitHub/GitLab issue and generate tests. `mode: "prompt"` or `mode: "verify"` |
964
- | `e2e_network_logs` | Query network request/response logs by `runDbId`. Filter by test name, method, status, URL pattern. Supports headers and bodies |
965
- | `e2e_learnings` | Query the learning system: `summary`, `flaky`, `selectors`, `pages`, `apis`, `errors`, `trends` |
966
- | `e2e_neo4j` | Manage Neo4j knowledge graph container: `start`, `stop`, `status` |
967
-
968
- > **Note:** Pool start/stop are CLI-only (`e2e-runner pool start|stop`) — not exposed via MCP to prevent killing active sessions.
969
-
970
- ### What You Can Ask Claude Code
971
-
972
- > "Run all E2E tests"
973
- > "Create a test that verifies the checkout flow"
974
- > "What tests are flaky? Show me the learning summary"
975
- > "Capture a screenshot of /dashboard with auth"
976
- > "Fetch issue #42 and create tests for it"
977
- > "What's the API error rate for the last 7 days?"
609
+ Every screenshot gets a deterministic hash (`ss:a3f2b1c9`). Use `e2e_screenshot` to retrieve any screenshot by hash — it returns the image with metadata (test name, step, type).
978
610
 
979
611
  ---
980
612
 
981
- ## OpenCode Integration
982
-
983
- The package also supports [OpenCode](https://github.com/anomalyco/opencode) with native MCP server configuration, skills, and commands.
613
+ ## AI Integration
984
614
 
985
- ### Quick Setup
615
+ ### Claude Code
986
616
 
987
617
  ```bash
988
- # 1. Install the package
989
- npm install --save-dev @matware/e2e-runner
990
-
991
- # 2. Copy OpenCode config to your project
992
- cp node_modules/@matware/e2e-runner/opencode.json ./
993
-
994
- # 3. Copy skills and commands (optional)
995
- mkdir -p .opencode
996
- cp -r node_modules/@matware/e2e-runner/.opencode/* .opencode/
997
-
998
- # 4. Start the Chrome pool
999
- npx e2e-runner pool start
618
+ claude plugin marketplace add fastslack/mtw-e2e-runner
619
+ claude plugin install e2e-runner@matware
1000
620
  ```
1001
621
 
1002
- ### What's Included
622
+ This gives Claude 17 MCP tools, a workflow skill, 4 slash commands (`/e2e-runner:run`, `/e2e-runner:create-test`, `/e2e-runner:verify-issue`, `/e2e-runner:capture`), and 3 specialized agents (test-analyzer, test-creator, test-improver).
1003
623
 
1004
- | Component | Description |
1005
- |-----------|-------------|
1006
- | **15 MCP tools** | Same tools as Claude Code — run tests, create files, screenshots, network logs, learnings, etc. |
1007
- | **Skill** | `e2e-testing` — full workflow guidance with references |
1008
- | **3 Commands** | `/run`, `/create-test`, `/verify-issue` |
624
+ **MCP-only install** (tools only, no skill/commands/agents):
1009
625
 
1010
- ### MCP Configuration
626
+ ```bash
627
+ claude mcp add --transport stdio --scope user e2e-runner \
628
+ -- npx -y -p @matware/e2e-runner e2e-runner-mcp
629
+ ```
1011
630
 
1012
- The `opencode.json` configures the MCP server as a local process:
631
+ ### OpenCode
1013
632
 
1014
- ```json
1015
- {
1016
- "mcp": {
1017
- "e2e-runner": {
1018
- "type": "local",
1019
- "command": "node",
1020
- "args": ["node_modules/@matware/e2e-runner/bin/mcp-server.js"],
1021
- "cwd": "${workspaceFolder}"
1022
- }
1023
- }
1024
- }
633
+ ```bash
634
+ cp node_modules/@matware/e2e-runner/opencode.json ./
635
+ mkdir -p .opencode && cp -r node_modules/@matware/e2e-runner/.opencode/* .opencode/
1025
636
  ```
1026
637
 
1027
- For global installation, use the binary directly:
638
+ See [OPENCODE.md](OPENCODE.md) for details.
1028
639
 
1029
- ```json
1030
- {
1031
- "mcp": {
1032
- "e2e-runner": {
1033
- "type": "local",
1034
- "command": "e2e-runner-mcp"
1035
- }
1036
- }
1037
- }
1038
- ```
640
+ ### MCP Tools
1039
641
 
1040
- See [OPENCODE.md](OPENCODE.md) for full documentation on OpenCode integration.
642
+ | Tool | Description |
643
+ |------|-------------|
644
+ | `e2e_run` | Run tests (all, by suite, or by file) |
645
+ | `e2e_list` | List available test suites |
646
+ | `e2e_create_test` | Create a new test JSON file |
647
+ | `e2e_create_module` | Create a reusable module |
648
+ | `e2e_pool_status` | Check Chrome pool health |
649
+ | `e2e_app_pool_status` | Inspect the app environment pool (forks, ports, drivers) |
650
+ | `e2e_screenshot` | Retrieve a screenshot by hash |
651
+ | `e2e_capture` | Capture screenshot of any URL |
652
+ | `e2e_analyze` | Extract page structure (interactive elements, forms, headings) and emit test scaffolds |
653
+ | `e2e_dashboard_start` | Start web dashboard |
654
+ | `e2e_dashboard_stop` | Stop web dashboard |
655
+ | `e2e_dashboard_restart` | Restart the dashboard (new project dir/port, clear stale sessions) |
656
+ | `e2e_issue` | Fetch issue and generate tests |
657
+ | `e2e_network_logs` | Query network logs for a run |
658
+ | `e2e_learnings` | Query stability insights |
659
+ | `e2e_vars` | Manage SQLite-backed `{{var.KEY}}` project variables |
660
+ | `e2e_neo4j` | Manage Neo4j knowledge graph |
661
+
662
+ > Pool start/stop are CLI-only — not exposed via MCP.
1041
663
 
1042
664
  ---
1043
665
 
@@ -1144,6 +766,8 @@ e2e-runner init # Scaffold project
1144
766
  | `--env <name>` | `default` | Environment profile |
1145
767
  | `--fail-on-network-error` | `false` | Fail tests with network errors |
1146
768
  | `--project-name <name>` | dir name | Project display name |
769
+ | `--driver <name>` | _(per-test)_ | Force pool driver for the run: `browserless`, `cdp`, `lightpanda`, `obscura`, `steel` |
770
+ | `--fallback-driver <name>` | _none_ | Explicit fallback if no pool with `--driver` is reachable |
1147
771
 
1148
772
  ---
1149
773