@matware/e2e-runner 1.2.1 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (82) hide show
  1. package/.claude-plugin/marketplace.json +21 -0
  2. package/.mcp.json +2 -2
  3. package/.opencode/commands/create-test.md +63 -0
  4. package/.opencode/commands/run.md +50 -0
  5. package/.opencode/commands/verify-issue.md +62 -0
  6. package/.opencode/skills/e2e-testing/SKILL.md +181 -0
  7. package/.opencode/skills/e2e-testing/references/action-types.md +143 -0
  8. package/.opencode/skills/e2e-testing/references/auth-strategies.md +91 -0
  9. package/.opencode/skills/e2e-testing/references/graphql.md +59 -0
  10. package/.opencode/skills/e2e-testing/references/issue-verification.md +59 -0
  11. package/.opencode/skills/e2e-testing/references/multi-pool.md +60 -0
  12. package/.opencode/skills/e2e-testing/references/network-debugging.md +62 -0
  13. package/.opencode/skills/e2e-testing/references/test-json-format.md +163 -0
  14. package/.opencode/skills/e2e-testing/references/troubleshooting.md +224 -0
  15. package/.opencode/skills/e2e-testing/references/variables.md +41 -0
  16. package/.opencode/skills/e2e-testing/references/visual-verification.md +89 -0
  17. package/OPENCODE.md +166 -0
  18. package/README.md +581 -55
  19. package/agents/test-creator.md +54 -1
  20. package/agents/test-improver.md +37 -0
  21. package/bin/cli.js +408 -16
  22. package/commands/create-test.md +16 -1
  23. package/opencode.json +11 -0
  24. package/package.json +7 -2
  25. package/scripts/setup-opencode.sh +113 -0
  26. package/skills/e2e-testing/SKILL.md +10 -3
  27. package/skills/e2e-testing/references/action-types.md +48 -5
  28. package/skills/e2e-testing/references/auth-strategies.md +91 -0
  29. package/skills/e2e-testing/references/graphql.md +59 -0
  30. package/skills/e2e-testing/references/issue-verification.md +59 -0
  31. package/skills/e2e-testing/references/multi-pool.md +60 -0
  32. package/skills/e2e-testing/references/network-debugging.md +62 -0
  33. package/skills/e2e-testing/references/test-json-format.md +4 -0
  34. package/skills/e2e-testing/references/troubleshooting.md +44 -2
  35. package/skills/e2e-testing/references/variables.md +41 -0
  36. package/skills/e2e-testing/references/visual-verification.md +89 -0
  37. package/src/actions.js +324 -2
  38. package/src/ai-generate.js +58 -8
  39. package/src/config.js +143 -0
  40. package/src/dashboard.js +145 -13
  41. package/src/db.js +130 -2
  42. package/src/index.js +7 -6
  43. package/src/learner-sqlite.js +304 -0
  44. package/src/learner.js +8 -3
  45. package/src/mcp-tools.js +1121 -43
  46. package/src/module-resolver.js +37 -0
  47. package/src/narrate.js +37 -0
  48. package/src/pool-manager.js +223 -0
  49. package/src/reporter.js +82 -1
  50. package/src/runner.js +157 -28
  51. package/src/sync/auth.js +354 -0
  52. package/src/sync/client.js +572 -0
  53. package/src/sync/hub-routes.js +816 -0
  54. package/src/sync/index.js +68 -0
  55. package/src/sync/middleware.js +347 -0
  56. package/src/sync/queue.js +209 -0
  57. package/src/sync/schema.js +540 -0
  58. package/src/verify.js +10 -7
  59. package/src/watch.js +384 -0
  60. package/templates/build-dashboard.js +47 -6
  61. package/templates/dashboard/js/api.js +60 -0
  62. package/templates/dashboard/js/init.js +13 -0
  63. package/templates/dashboard/js/keyboard.js +46 -0
  64. package/templates/dashboard/js/state.js +40 -0
  65. package/templates/dashboard/js/toast.js +41 -0
  66. package/templates/dashboard/js/utils.js +196 -0
  67. package/templates/dashboard/js/view-live.js +143 -0
  68. package/templates/dashboard/js/view-runs.js +572 -0
  69. package/templates/dashboard/js/view-tests.js +294 -0
  70. package/templates/dashboard/js/view-watch.js +242 -0
  71. package/templates/dashboard/js/websocket.js +110 -0
  72. package/templates/dashboard/styles/base.css +69 -0
  73. package/templates/dashboard/styles/components.css +110 -0
  74. package/templates/dashboard/styles/view-live.css +74 -0
  75. package/templates/dashboard/styles/view-runs.css +207 -0
  76. package/templates/dashboard/styles/view-tests.css +96 -0
  77. package/templates/dashboard/styles/view-watch.css +53 -0
  78. package/templates/dashboard/template.html +165 -99
  79. package/templates/dashboard.html +1596 -541
  80. package/templates/sample-test.json +0 -8
  81. package/templates/dashboard/app.js +0 -1152
  82. package/templates/dashboard/styles.css +0 -413
package/README.md CHANGED
@@ -14,6 +14,7 @@
14
14
  <img src="https://img.shields.io/npm/l/@matware/e2e-runner" alt="license" />
15
15
  <img src="https://img.shields.io/badge/MCP-compatible-green" alt="MCP compatible" />
16
16
  <img src="https://img.shields.io/badge/AI--native-Claude%20Code-blueviolet" alt="AI native" />
17
+ <img src="https://img.shields.io/badge/AI--native-OpenCode-orange" alt="OpenCode compatible" />
17
18
  </p>
18
19
 
19
20
  <p align="center">
@@ -26,32 +27,6 @@
26
27
 
27
28
  But what makes it truly different is its **deep AI integration**. With a built-in [MCP server](https://modelcontextprotocol.io/), Claude Code can create tests from a conversation, run them, read the results, capture screenshots, and even visually verify that pages look correct — all without leaving the chat. Paste a GitHub issue URL and get a runnable test back. That's the workflow.
28
29
 
29
- ### What you get
30
-
31
- 🧪 **Zero-code tests** — JSON files that anyone on your team can read and write. No JavaScript, no compilation, no framework lock-in.
32
-
33
- 🤖 **AI-powered testing** — Claude Code creates, executes, and debugs tests natively through 13 MCP tools. Ask it to "test the checkout flow" and it builds the JSON, runs it, and reports back.
34
-
35
- 🐛 **Issue-to-Test pipeline** — Paste a GitHub or GitLab issue URL. The runner fetches it, generates E2E tests, runs them, and tells you: *bug confirmed* or *not reproducible*.
36
-
37
- 👁️ **Visual verification** — Describe what the page should look like in plain English. The AI captures a screenshot and judges pass/fail against your description. No pixel-diffing setup needed.
38
-
39
- 🧠 **Learning system** — Tracks test stability across runs. Detects flaky tests, unstable selectors, slow APIs, and error patterns — then surfaces actionable insights.
40
-
41
- ⚡ **Parallel execution** — Run N tests simultaneously against a shared Chrome pool (browserless/chrome). Serial mode available for tests that share state.
42
-
43
- 📊 **Real-time dashboard** — Live execution view, run history with pass-rate charts, screenshot gallery with hash-based search, expandable network request logs.
44
-
45
- 🔁 **Smart retries** — Test-level and action-level retries with configurable delays. Flaky tests are detected and flagged automatically.
46
-
47
- 📦 **Reusable modules** — Extract common flows (login, navigation, setup) into parameterized modules and reference them with `$use`.
48
-
49
- 🏗️ **CI-ready** — JUnit XML output, exit code 1 on failure, auto-captured error screenshots. Drop-in GitHub Actions example included.
50
-
51
- 🌐 **Multi-project** — One dashboard aggregates test results from all your projects. One Chrome pool serves them all.
52
-
53
- 🐳 **Portable** — Chrome runs in Docker, tests are JSON files in your repo. Works on any machine with Node.js and Docker.
54
-
55
30
  ### This is a test
56
31
 
57
32
  ```json
@@ -74,45 +49,232 @@ No imports. No `describe`/`it`. No compilation step. Just a JSON file that descr
74
49
 
75
50
  ---
76
51
 
77
- ## Quick Start
52
+ ## Getting Started
53
+
54
+ ### Prerequisites
55
+
56
+ - **Node.js** >= 20
57
+ - **Docker** running (for the Chrome pool)
58
+ - Your app running on a known port (e.g. `http://localhost:3000`)
59
+
60
+ > **Why `host.docker.internal`?**
61
+ >
62
+ > Chrome runs inside a Docker container. From inside the container, `localhost` refers to the container itself — not your machine. The special hostname `host.docker.internal` resolves to your host machine, so Chrome can reach your locally running app.
63
+ >
64
+ > The default `baseUrl` is `http://host.docker.internal:3000`. If your app runs on a different port, change it in `e2e.config.js` after init.
65
+ >
66
+ > **Linux note:** On Docker Engine (not Docker Desktop), you may need to add `--add-host=host.docker.internal:host-gateway` to the Docker run flags, or use your machine's LAN IP directly as the `baseUrl`.
67
+
68
+ ---
69
+
70
+ ### Path A: With Claude Code
71
+
72
+ If you use [Claude Code](https://docs.anthropic.com/en/docs/claude-code), this is the fastest path — Claude handles test creation and debugging for you.
78
73
 
79
- **One-liner** (requires Node.js >= 20 and Docker):
74
+ **1. Install the package**
80
75
 
81
76
  ```bash
82
- curl -fsSL https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/scripts/quickstart.sh | bash
77
+ npm install --save-dev @matware/e2e-runner
83
78
  ```
84
79
 
85
- **Step by step:**
80
+ **2. Scaffold the project structure**
81
+
82
+ ```bash
83
+ npx e2e-runner init
84
+ ```
85
+
86
+ This creates `e2e/tests/` with a sample test and `e2e/screenshots/` for captures.
87
+
88
+ **3. Configure your base URL**
89
+
90
+ Edit `e2e.config.js` and set `baseUrl` to match your app's port:
91
+
92
+ ```js
93
+ export default {
94
+ baseUrl: 'http://host.docker.internal:3000', // change 3000 to your port
95
+ };
96
+ ```
97
+
98
+ **4. Start the Chrome pool**
99
+
100
+ ```bash
101
+ npx e2e-runner pool start
102
+ ```
103
+
104
+ You should see:
105
+
106
+ ```
107
+ ✓ Chrome pool started on port 3333 (max 3 sessions)
108
+ ```
109
+
110
+ **5. Install the Claude Code plugin**
111
+
112
+ ```bash
113
+ # Add the marketplace (one-time)
114
+ claude plugin marketplace add fastslack/mtw-e2e-runner
115
+
116
+ # Install the plugin
117
+ claude plugin install e2e-runner@matware
118
+ ```
119
+
120
+ The plugin gives Claude 13 MCP tools, a workflow skill, 3 slash commands, and 3 specialized agents.
121
+
122
+ **6. Ask Claude to run the sample test**
123
+
124
+ In Claude Code, just say:
125
+
126
+ > "Run all E2E tests"
127
+
128
+ Claude will check the pool, run the sample test, and report back:
129
+
130
+ ```
131
+ ==================================================
132
+ E2E RESULTS
133
+ ==================================================
134
+ Total: 1
135
+ Passed: 1
136
+ Failed: 0
137
+ Rate: 100.00%
138
+ Duration: 1.23s
139
+ ==================================================
140
+ ```
141
+
142
+ From here, you can ask Claude to create new tests ("test the login flow"), debug failures, or verify GitHub issues.
143
+
144
+ ---
145
+
146
+ ### Path B: CLI Only
147
+
148
+ No AI required — use the runner directly from your terminal.
149
+
150
+ **1. Install the package**
86
151
 
87
152
  ```bash
88
- # 1. Install
89
153
  npm install --save-dev @matware/e2e-runner
154
+ ```
90
155
 
91
- # 2. Scaffold project structure
156
+ **2. Scaffold the project structure**
157
+
158
+ ```bash
92
159
  npx e2e-runner init
160
+ ```
161
+
162
+ This creates `e2e/tests/` with a sample test and `e2e/screenshots/` for captures.
163
+
164
+ **3. Configure your base URL**
93
165
 
94
- # 3. Start Chrome pool (requires Docker)
166
+ Edit `e2e.config.js` and set `baseUrl` to match your app's port:
167
+
168
+ ```js
169
+ export default {
170
+ baseUrl: 'http://host.docker.internal:3000', // change 3000 to your port
171
+ };
172
+ ```
173
+
174
+ **4. Start the Chrome pool**
175
+
176
+ ```bash
95
177
  npx e2e-runner pool start
178
+ ```
179
+
180
+ You should see:
96
181
 
97
- # 4. Run all tests
182
+ ```
183
+ ✓ Chrome pool started on port 3333 (max 3 sessions)
184
+ ```
185
+
186
+ **5. Run the sample test**
187
+
188
+ ```bash
98
189
  npx e2e-runner run --all
190
+ ```
99
191
 
100
- # 5. Open the dashboard
101
- npx e2e-runner dashboard
192
+ Expected output:
193
+
194
+ ```
195
+ ==================================================
196
+ E2E RESULTS
197
+ ==================================================
198
+ Total: 1
199
+ Passed: 1
200
+ Failed: 0
201
+ Rate: 100.00%
202
+ Duration: 1.23s
203
+ ==================================================
102
204
  ```
103
205
 
104
- **Add to Claude Code** (once, available in all projects):
206
+ A screenshot is saved at `e2e/screenshots/homepage.png`.
207
+
208
+ **6. Write your first real test**
209
+
210
+ Create `e2e/tests/my-first-test.json`:
211
+
212
+ ```json
213
+ [
214
+ {
215
+ "name": "homepage-visible",
216
+ "actions": [
217
+ { "type": "goto", "value": "/" },
218
+ { "type": "assert_visible", "selector": "body" },
219
+ { "type": "screenshot", "value": "my-first-test.png" }
220
+ ]
221
+ }
222
+ ]
223
+ ```
224
+
225
+ Run it:
105
226
 
106
227
  ```bash
107
- # Full plugin: MCP tools + skills + commands + agents
108
- claude plugin install npm:@matware/e2e-runner
228
+ npx e2e-runner run --suite my-first-test
229
+ ```
109
230
 
110
- # Or MCP-only (tools without skills/commands/agents):
111
- claude mcp add --transport stdio --scope user e2e-runner \
112
- -- npx -y -p @matware/e2e-runner e2e-runner-mcp
231
+ ---
232
+
233
+ ### One-liner quickstart
234
+
235
+ If you want to skip the step-by-step and get everything running in one command:
236
+
237
+ ```bash
238
+ curl -fsSL https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/scripts/quickstart.sh | bash
113
239
  ```
114
240
 
115
- The **plugin** is the recommended approach — it installs the 13 MCP tools *plus* a skill that teaches Claude the optimal workflow, 3 slash commands (`/e2e-runner:run`, `/e2e-runner:create-test`, `/e2e-runner:verify-issue`), and 2 specialized agents for test analysis and creation.
241
+ > This installs the package, scaffolds the project, and starts the Chrome pool. You'll still need to configure your `baseUrl` afterwards.
242
+
243
+ ### What's next?
244
+
245
+ - [Test Format](#test-format) — learn the full action vocabulary
246
+ - [Claude Code Integration](#claude-code-integration) — set up AI-powered testing
247
+ - [Visual Verification](#visual-verification) — describe expected pages in plain English
248
+ - [Issue-to-Test](#issue-to-test) — turn bug reports into executable tests
249
+ - [Web Dashboard](#web-dashboard) — monitor tests in real time
250
+
251
+ ---
252
+
253
+ ## What you get
254
+
255
+ 🧪 **Zero-code tests** — JSON files that anyone on your team can read and write. No JavaScript, no compilation, no framework lock-in.
256
+
257
+ 🤖 **AI-powered testing** — Claude Code creates, executes, and debugs tests natively through 13 MCP tools. Ask it to "test the checkout flow" and it builds the JSON, runs it, and reports back.
258
+
259
+ 🐛 **Issue-to-Test pipeline** — Paste a GitHub or GitLab issue URL. The runner fetches it, generates E2E tests, runs them, and tells you: *bug confirmed* or *not reproducible*.
260
+
261
+ 👁️ **Visual verification** — Describe what the page should look like in plain English. The AI captures a screenshot and judges pass/fail against your description. No pixel-diffing setup needed.
262
+
263
+ 🧠 **Learning system** — Tracks test stability across runs. Detects flaky tests, unstable selectors, slow APIs, and error patterns — then surfaces actionable insights.
264
+
265
+ ⚡ **Parallel execution** — Run N tests simultaneously against a shared Chrome pool (browserless/chrome). Serial mode available for tests that share state.
266
+
267
+ 📊 **Real-time dashboard** — Live execution view, run history with pass-rate charts, screenshot gallery with hash-based search, expandable network request logs.
268
+
269
+ 🔁 **Smart retries** — Test-level and action-level retries with configurable delays. Flaky tests are detected and flagged automatically.
270
+
271
+ 📦 **Reusable modules** — Extract common flows (login, navigation, setup) into parameterized modules and reference them with `$use`.
272
+
273
+ 🏗️ **CI-ready** — JUnit XML output, exit code 1 on failure, auto-captured error screenshots. Drop-in GitHub Actions example included.
274
+
275
+ 🌐 **Multi-project** — One dashboard aggregates test results from all your projects. One Chrome pool serves them all.
276
+
277
+ 🐳 **Portable** — Chrome runs in Docker, tests are JSON files in your repo. Works on any machine with Node.js and Docker.
116
278
 
117
279
  ---
118
280
 
@@ -126,8 +288,7 @@ Each `.json` file in `e2e/tests/` contains an array of tests. Each test has a `n
126
288
  "name": "homepage-loads",
127
289
  "actions": [
128
290
  { "type": "goto", "value": "/" },
129
- { "type": "wait", "selector": ".hero" },
130
- { "type": "assert_text", "text": "Welcome" },
291
+ { "type": "assert_visible", "selector": "body" },
131
292
  { "type": "assert_url", "value": "/" },
132
293
  { "type": "screenshot", "value": "homepage.png" }
133
294
  ]
@@ -245,22 +406,320 @@ Serial tests run one at a time **after** all parallel tests finish — preventin
245
406
 
246
407
  ---
247
408
 
409
+ ## Testing Authenticated Apps
410
+
411
+ Most real-world apps require login before tests can interact with protected pages. E2E Runner provides multiple strategies — choose the one that matches your app's auth mechanism.
412
+
413
+ ### Strategy 1: UI Login Flow (any app)
414
+
415
+ The most universal approach — fill in the login form like a real user. Works with **any** authentication system (session cookies, JWT, OAuth redirect, etc.):
416
+
417
+ ```json
418
+ {
419
+ "hooks": {
420
+ "beforeEach": [
421
+ { "type": "goto", "value": "/login" },
422
+ { "type": "type", "selector": "#email", "value": "test@example.com" },
423
+ { "type": "type", "selector": "#password", "value": "test-password" },
424
+ { "type": "click", "text": "Sign In" },
425
+ { "type": "wait", "selector": ".dashboard" }
426
+ ]
427
+ },
428
+ "tests": [
429
+ {
430
+ "name": "profile-page",
431
+ "actions": [
432
+ { "type": "goto", "value": "/profile" },
433
+ { "type": "assert_text", "text": "My Profile" }
434
+ ]
435
+ }
436
+ ]
437
+ }
438
+ ```
439
+
440
+ > **When to use:** You don't know or care how auth works internally. The browser handles cookies/tokens automatically after login — just like a real user.
441
+
442
+ ### Strategy 2: JWT Token Injection (SPAs)
443
+
444
+ For single-page apps that store JWT tokens in `localStorage` or `sessionStorage`. Skip the login form entirely by injecting the token directly:
445
+
446
+ ```json
447
+ {
448
+ "hooks": {
449
+ "beforeEach": [
450
+ { "type": "goto", "value": "/" },
451
+ { "type": "set_storage", "value": "accessToken=eyJhbGciOiJIUzI1NiIs..." },
452
+ { "type": "goto", "value": "/dashboard" },
453
+ { "type": "wait", "selector": ".dashboard-loaded" }
454
+ ]
455
+ },
456
+ "tests": [...]
457
+ }
458
+ ```
459
+
460
+ **Common storage key names** (depends on your app):
461
+
462
+ | Framework / Library | Typical key | Storage |
463
+ |---------------------|-------------|---------|
464
+ | Custom JWT | `accessToken`, `token`, `jwt` | localStorage |
465
+ | Auth0 SPA SDK | `@@auth0spajs@@::*` | localStorage |
466
+ | Firebase Auth | `firebase:authUser:*` | localStorage |
467
+ | AWS Amplify | `CognitoIdentityServiceProvider.*` | localStorage |
468
+ | Supabase | `sb-<ref>-auth-token` | localStorage |
469
+ | NextAuth (client) | `next-auth.session-token` | cookie (see Strategy 4) |
470
+
471
+ **Using `sessionStorage` instead:**
472
+
473
+ ```json
474
+ { "type": "set_storage", "value": "token=eyJhbG...", "selector": "session" }
475
+ ```
476
+
477
+ **Asserting the token was stored correctly:**
478
+
479
+ ```json
480
+ { "type": "assert_storage", "value": "accessToken" }
481
+ { "type": "assert_storage", "value": "accessToken=eyJhbG..." }
482
+ ```
483
+
484
+ > **When to use:** Your SPA reads auth tokens from browser storage. Fastest strategy — no network round-trip for login.
485
+
486
+ ### Strategy 3: Config-Level Auth Token
487
+
488
+ For apps where every test needs the same JWT token. Set it once in config — it's injected into `localStorage` before every `e2e_capture` and `e2e_issue --verify` run:
489
+
490
+ ```js
491
+ // e2e.config.js
492
+ export default {
493
+ authToken: 'eyJhbGciOiJIUzI1NiIs...',
494
+ authStorageKey: 'accessToken', // default
495
+ };
496
+ ```
497
+
498
+ Or via environment variables:
499
+
500
+ ```bash
501
+ AUTH_TOKEN="eyJhbGciOiJIUzI1NiIs..." npx e2e-runner run --all
502
+ ```
503
+
504
+ Or via CLI:
505
+
506
+ ```bash
507
+ npx e2e-runner run --all --auth-token "eyJhbG..." --auth-storage-key "jwt"
508
+ ```
509
+
510
+ MCP tools (`e2e_capture`, `e2e_issue`) also accept `authToken` and `authStorageKey` per call.
511
+
512
+ > **When to use:** All tests share the same user session and your app uses JWT in localStorage.
513
+
514
+ ### Strategy 4: Cookie-Based Auth (server-rendered apps)
515
+
516
+ For apps that use HTTP cookies (Rails, Django, Laravel, Express sessions, NextAuth, etc.). Use `evaluate` to set cookies before navigating:
517
+
518
+ ```json
519
+ {
520
+ "hooks": {
521
+ "beforeEach": [
522
+ { "type": "goto", "value": "/" },
523
+ { "type": "evaluate", "value": "document.cookie = 'session_id=abc123; path=/; SameSite=Lax'" },
524
+ { "type": "goto", "value": "/dashboard" }
525
+ ]
526
+ },
527
+ "tests": [...]
528
+ }
529
+ ```
530
+
531
+ **Multiple cookies:**
532
+
533
+ ```json
534
+ { "type": "evaluate", "value": "document.cookie = 'session_id=abc123; path=/'; document.cookie = '_csrf_token=xyz789; path=/'" }
535
+ ```
536
+
537
+ **For `HttpOnly` cookies** (can't be set via JavaScript), use the UI login strategy instead — the browser will store them automatically.
538
+
539
+ > **When to use:** Traditional server-rendered apps, or any app that authenticates via cookies.
540
+
541
+ ### Strategy 5: HTTP Header Auth (API tests)
542
+
543
+ For API testing where you need to send `Authorization` headers with every request. Use `evaluate` to override `fetch`/`XMLHttpRequest`:
544
+
545
+ ```json
546
+ {
547
+ "hooks": {
548
+ "beforeEach": [
549
+ { "type": "goto", "value": "/" },
550
+ { "type": "evaluate", "value": "const origFetch = window.fetch; window.fetch = (url, opts = {}) => { opts.headers = { ...opts.headers, 'Authorization': 'Bearer eyJhbG...' }; return origFetch(url, opts); }" }
551
+ ]
552
+ },
553
+ "tests": [
554
+ {
555
+ "name": "api-returns-user",
556
+ "actions": [
557
+ { "type": "evaluate", "value": "const res = await fetch('/api/me'); const data = await res.json(); if (data.email !== 'test@example.com') throw new Error('Wrong user: ' + data.email)" }
558
+ ]
559
+ }
560
+ ]
561
+ }
562
+ ```
563
+
564
+ > **When to use:** API-level tests (with `--test-type api`) that need auth headers.
565
+
566
+ ### Strategy 6: OAuth / SSO (external provider)
567
+
568
+ OAuth flows redirect to external providers (Google, GitHub, Okta, etc.) which can't be automated reliably. Common workarounds:
569
+
570
+ **Option A — Test environment bypass:** Most apps have a direct login endpoint for testing that skips OAuth:
571
+
572
+ ```json
573
+ { "type": "goto", "value": "/auth/test-login?user=test@example.com" }
574
+ ```
575
+
576
+ **Option B — Pre-authenticated token:** Get a token from your auth provider's API and inject it:
577
+
578
+ ```json
579
+ {
580
+ "hooks": {
581
+ "beforeEach": [
582
+ { "type": "goto", "value": "/" },
583
+ { "type": "set_storage", "value": "oidc.user:https://auth.example.com:client_id={\"access_token\":\"...\"}" }
584
+ ]
585
+ }
586
+ }
587
+ ```
588
+
589
+ **Option C — Session cookie from CI:** If your CI can authenticate via API, pass the session cookie as an env var:
590
+
591
+ ```bash
592
+ SESSION=$(curl -s -c - https://api.example.com/auth/login -d '{"email":"test@example.com","password":"secret"}' | grep session_id | awk '{print $NF}')
593
+ AUTH_TOKEN="$SESSION" AUTH_STORAGE_KEY="session_id" npx e2e-runner run --all
594
+ ```
595
+
596
+ > **When to use:** Apps with Google/GitHub/Okta/Auth0 login. You almost always need a test-environment backdoor.
597
+
598
+ ### Reusable Auth Modules
599
+
600
+ Extract your auth strategy into a module so every test can reference it without duplication:
601
+
602
+ ```json
603
+ // e2e/modules/login.json — UI login (universal)
604
+ {
605
+ "$module": "login",
606
+ "description": "Log in via the UI login form",
607
+ "params": {
608
+ "email": { "required": true, "description": "User email" },
609
+ "password": { "required": true, "description": "User password" },
610
+ "redirectTo": { "default": "/dashboard", "description": "Page to land on after login" }
611
+ },
612
+ "actions": [
613
+ { "type": "goto", "value": "/login" },
614
+ { "type": "type", "selector": "#email", "value": "{{email}}" },
615
+ { "type": "type", "selector": "#password", "value": "{{password}}" },
616
+ { "type": "click", "text": "Sign In" },
617
+ { "type": "wait", "selector": "{{redirectTo}}" }
618
+ ]
619
+ }
620
+ ```
621
+
622
+ ```json
623
+ // e2e/modules/auth-token.json — JWT injection (SPAs)
624
+ {
625
+ "$module": "auth-token",
626
+ "description": "Inject an auth token into browser storage",
627
+ "params": {
628
+ "token": { "required": true, "description": "JWT or session token" },
629
+ "storageKey": { "default": "accessToken", "description": "Storage key name" },
630
+ "storage": { "default": "local", "description": "local or session" },
631
+ "redirectTo": { "default": "/dashboard", "description": "Page to navigate to after injection" }
632
+ },
633
+ "actions": [
634
+ { "type": "goto", "value": "/" },
635
+ { "type": "set_storage", "value": "{{storageKey}}={{token}}", "selector": "{{#storage}}{{storage}}{{/storage}}" },
636
+ { "type": "goto", "value": "{{redirectTo}}" }
637
+ ]
638
+ }
639
+ ```
640
+
641
+ Use in tests:
642
+
643
+ ```json
644
+ // UI login
645
+ { "$use": "login", "params": { "email": "admin@test.com", "password": "secret" } }
646
+
647
+ // Token injection
648
+ { "$use": "auth-token", "params": { "token": "eyJhbG..." } }
649
+
650
+ // Token in sessionStorage, redirect to /settings
651
+ { "$use": "auth-token", "params": { "token": "eyJhbG...", "storage": "session", "redirectTo": "/settings" } }
652
+ ```
653
+
654
+ ### Testing Different User Roles
655
+
656
+ Use separate tests (or the same module with different credentials) to test role-based access:
657
+
658
+ ```json
659
+ [
660
+ {
661
+ "name": "admin-sees-settings",
662
+ "actions": [
663
+ { "$use": "login", "params": { "email": "admin@test.com", "password": "admin-pass" } },
664
+ { "type": "goto", "value": "/settings" },
665
+ { "type": "assert_visible", "selector": ".admin-panel" }
666
+ ]
667
+ },
668
+ {
669
+ "name": "viewer-cannot-access-settings",
670
+ "actions": [
671
+ { "$use": "login", "params": { "email": "viewer@test.com", "password": "viewer-pass" } },
672
+ { "type": "goto", "value": "/settings" },
673
+ { "type": "assert_text", "text": "Access Denied" }
674
+ ]
675
+ }
676
+ ]
677
+ ```
678
+
679
+ ### Clearing Auth State
680
+
681
+ Each test runs in a **fresh browser context** (new connection to the Chrome pool), so cookies and storage are automatically clean. If you need to explicitly clear state mid-test:
682
+
683
+ ```json
684
+ { "type": "clear_cookies" }
685
+ ```
686
+
687
+ This clears cookies, localStorage, and sessionStorage for the current origin.
688
+
689
+ ### Quick Reference
690
+
691
+ | Auth type | Strategy | Key actions |
692
+ |-----------|----------|-------------|
693
+ | Username/password form | UI Login | `goto` + `type` + `click` in `beforeEach` |
694
+ | JWT in localStorage | Token Injection | `set_storage` in `beforeEach` |
695
+ | JWT in sessionStorage | Token Injection | `set_storage` with `selector: "session"` |
696
+ | Session cookies | Cookie | `evaluate` to set `document.cookie` |
697
+ | HttpOnly cookies | UI Login | Must go through login form |
698
+ | OAuth / SSO | Test bypass | App-specific test login endpoint |
699
+ | API auth headers | Header Override | `evaluate` to patch `fetch` |
700
+ | Config-level token | Config | `authToken` + `authStorageKey` in config |
701
+
702
+ ---
703
+
248
704
  ## Reusable Modules
249
705
 
250
706
  Extract common flows into parameterized modules:
251
707
 
252
708
  ```json
253
- // e2e/modules/auth.json
709
+ // e2e/modules/login.json
254
710
  {
255
- "$module": "auth-jwt",
256
- "description": "Inject JWT token into localStorage",
711
+ "$module": "login",
712
+ "description": "Log in via the UI login form",
257
713
  "params": {
258
- "token": { "required": true, "description": "JWT token" },
259
- "storageKey": { "default": "accessToken" }
714
+ "email": { "required": true, "description": "User email" },
715
+ "password": { "required": true, "description": "User password" }
260
716
  },
261
717
  "actions": [
262
- { "type": "evaluate", "value": "localStorage.setItem('{{storageKey}}', '{{token}}')" },
263
- { "type": "goto", "value": "/dashboard" }
718
+ { "type": "goto", "value": "/login" },
719
+ { "type": "type", "selector": "#email", "value": "{{email}}" },
720
+ { "type": "type", "selector": "#password", "value": "{{password}}" },
721
+ { "type": "click", "text": "Sign In" },
722
+ { "type": "wait", "value": "2000" }
264
723
  ]
265
724
  }
266
725
  ```
@@ -271,7 +730,7 @@ Use in tests:
271
730
  {
272
731
  "name": "dashboard-loads",
273
732
  "actions": [
274
- { "$use": "auth-jwt", "params": { "token": "eyJhbG..." } },
733
+ { "$use": "login", "params": { "email": "user@test.com", "password": "secret" } },
275
734
  { "type": "assert_text", "text": "Dashboard" }
276
735
  ]
277
736
  }
@@ -455,7 +914,11 @@ The package ships as a **Claude Code plugin** — a single install that gives Cl
455
914
  ### Install as Plugin (recommended)
456
915
 
457
916
  ```bash
458
- claude plugin install npm:@matware/e2e-runner
917
+ # 1. Add the marketplace (one-time)
918
+ claude plugin marketplace add fastslack/mtw-e2e-runner
919
+
920
+ # 2. Install the plugin
921
+ claude plugin install e2e-runner@matware
459
922
  ```
460
923
 
461
924
  **What you get:**
@@ -465,7 +928,7 @@ claude plugin install npm:@matware/e2e-runner
465
928
  | **13 MCP tools** | Run tests, create test files, capture screenshots, query network logs, manage dashboard, verify issues, query learnings |
466
929
  | **Skill** | Teaches Claude the full e2e-runner workflow — how to combine tools, interpret results, debug failures, create tests |
467
930
  | **3 Commands** | `/e2e-runner:run` — run & analyze tests<br>`/e2e-runner:create-test` — explore UI and create tests<br>`/e2e-runner:verify-issue <url>` — verify GitHub/GitLab bugs |
468
- | **2 Agents** | **test-analyzer** — diagnoses failures, analyzes flaky tests, drills into network errors<br>**test-creator** — explores UI, discovers selectors, designs and validates tests |
931
+ | **3 Agents** | **test-analyzer** — diagnoses failures, analyzes flaky tests, drills into network errors<br>**test-creator** — explores UI, discovers selectors, designs and validates tests<br>**test-improver** — refactors verbose evaluate actions, extracts modules, adds waits/retries, eliminates hardcoded delays |
469
932
 
470
933
  ### Install MCP-only (alternative)
471
934
 
@@ -515,6 +978,69 @@ claude mcp add --transport stdio --scope user e2e-runner \
515
978
 
516
979
  ---
517
980
 
981
+ ## OpenCode Integration
982
+
983
+ The package also supports [OpenCode](https://github.com/anomalyco/opencode) with native MCP server configuration, skills, and commands.
984
+
985
+ ### Quick Setup
986
+
987
+ ```bash
988
+ # 1. Install the package
989
+ npm install --save-dev @matware/e2e-runner
990
+
991
+ # 2. Copy OpenCode config to your project
992
+ cp node_modules/@matware/e2e-runner/opencode.json ./
993
+
994
+ # 3. Copy skills and commands (optional)
995
+ mkdir -p .opencode
996
+ cp -r node_modules/@matware/e2e-runner/.opencode/* .opencode/
997
+
998
+ # 4. Start the Chrome pool
999
+ npx e2e-runner pool start
1000
+ ```
1001
+
1002
+ ### What's Included
1003
+
1004
+ | Component | Description |
1005
+ |-----------|-------------|
1006
+ | **15 MCP tools** | Same tools as Claude Code — run tests, create files, screenshots, network logs, learnings, etc. |
1007
+ | **Skill** | `e2e-testing` — full workflow guidance with references |
1008
+ | **3 Commands** | `/run`, `/create-test`, `/verify-issue` |
1009
+
1010
+ ### MCP Configuration
1011
+
1012
+ The `opencode.json` configures the MCP server as a local process:
1013
+
1014
+ ```json
1015
+ {
1016
+ "mcp": {
1017
+ "e2e-runner": {
1018
+ "type": "local",
1019
+ "command": "node",
1020
+ "args": ["node_modules/@matware/e2e-runner/bin/mcp-server.js"],
1021
+ "cwd": "${workspaceFolder}"
1022
+ }
1023
+ }
1024
+ }
1025
+ ```
1026
+
1027
+ For global installation, use the binary directly:
1028
+
1029
+ ```json
1030
+ {
1031
+ "mcp": {
1032
+ "e2e-runner": {
1033
+ "type": "local",
1034
+ "command": "e2e-runner-mcp"
1035
+ }
1036
+ }
1037
+ }
1038
+ ```
1039
+
1040
+ See [OPENCODE.md](OPENCODE.md) for full documentation on OpenCode integration.
1041
+
1042
+ ---
1043
+
518
1044
  ## Network Error Handling
519
1045
 
520
1046
  ### Explicit Assertion