@matware/e2e-runner 1.3.0 → 1.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +37 -6
- package/.claude-plugin/plugin.json +17 -3
- package/LICENSE +190 -0
- package/README.md +61 -526
- package/bin/cli.js +5 -4
- package/commands/capture.md +45 -0
- package/package.json +1 -1
- package/src/actions.js +151 -0
- package/src/ai-generate.js +81 -0
- package/src/app-pool.js +339 -0
- package/src/config.js +125 -7
- package/src/dashboard.js +75 -8
- package/src/db.js +63 -7
- package/src/index.js +6 -4
- package/src/learner-sqlite.js +154 -0
- package/src/learner.js +70 -3
- package/src/mcp-tools.js +251 -32
- package/src/narrate.js +28 -0
- package/src/pool-manager.js +22 -16
- package/src/pool.js +301 -31
- package/src/reporter.js +4 -1
- package/src/runner.js +335 -55
- package/src/visual-diff.js +446 -0
- package/templates/dashboard/js/api.js +2 -0
- package/templates/dashboard/js/utils.js +20 -0
- package/templates/dashboard/js/view-live.js +40 -2
- package/templates/dashboard/js/view-runs.js +161 -57
- package/templates/dashboard/js/websocket.js +6 -0
- package/templates/dashboard/styles/components.css +7 -0
- package/templates/dashboard/styles/view-live.css +24 -1
- package/templates/dashboard/styles/view-runs.css +36 -0
- package/templates/dashboard/template.html +24 -9
- package/templates/dashboard.html +322 -310
package/README.md
CHANGED
|
@@ -9,12 +9,16 @@
|
|
|
9
9
|
</p>
|
|
10
10
|
|
|
11
11
|
<p align="center">
|
|
12
|
-
<img src="https://img.shields.io/npm/v/@matware/e2e-runner?color=blue" alt="npm version"
|
|
12
|
+
<a href="https://www.npmjs.com/package/@matware/e2e-runner"><img src="https://img.shields.io/npm/v/@matware/e2e-runner?color=blue" alt="npm version" /></a>
|
|
13
13
|
<img src="https://img.shields.io/node/v/@matware/e2e-runner" alt="node version" />
|
|
14
|
-
<img src="https://img.shields.io/npm/
|
|
14
|
+
<a href="https://www.npmjs.com/package/@matware/e2e-runner"><img src="https://img.shields.io/npm/dm/@matware/e2e-runner" alt="npm downloads" /></a>
|
|
15
|
+
<a href="https://hub.docker.com/r/fastslack/e2e-runner-mcp"><img src="https://img.shields.io/docker/pulls/fastslack/e2e-runner-mcp" alt="Docker pulls" /></a>
|
|
16
|
+
<a href="https://github.com/fastslack/mtw-e2e-runner/stargazers"><img src="https://img.shields.io/github/stars/fastslack/mtw-e2e-runner" alt="GitHub stars" /></a>
|
|
17
|
+
<a href="LICENSE"><img src="https://img.shields.io/npm/l/@matware/e2e-runner" alt="license" /></a>
|
|
15
18
|
<img src="https://img.shields.io/badge/MCP-compatible-green" alt="MCP compatible" />
|
|
16
19
|
<img src="https://img.shields.io/badge/AI--native-Claude%20Code-blueviolet" alt="AI native" />
|
|
17
20
|
<img src="https://img.shields.io/badge/AI--native-OpenCode-orange" alt="OpenCode compatible" />
|
|
21
|
+
<a href="https://skills.sh"><img src="https://img.shields.io/badge/skills.sh-e2e--testing-ff6600" alt="Agent Skills" /></a>
|
|
18
22
|
</p>
|
|
19
23
|
|
|
20
24
|
<p align="center">
|
|
@@ -49,121 +53,40 @@ No imports. No `describe`/`it`. No compilation step. Just a JSON file that descr
|
|
|
49
53
|
|
|
50
54
|
---
|
|
51
55
|
|
|
52
|
-
##
|
|
53
|
-
|
|
54
|
-
### Prerequisites
|
|
55
|
-
|
|
56
|
-
- **Node.js** >= 20
|
|
57
|
-
- **Docker** running (for the Chrome pool)
|
|
58
|
-
- Your app running on a known port (e.g. `http://localhost:3000`)
|
|
59
|
-
|
|
60
|
-
> **Why `host.docker.internal`?**
|
|
61
|
-
>
|
|
62
|
-
> Chrome runs inside a Docker container. From inside the container, `localhost` refers to the container itself — not your machine. The special hostname `host.docker.internal` resolves to your host machine, so Chrome can reach your locally running app.
|
|
63
|
-
>
|
|
64
|
-
> The default `baseUrl` is `http://host.docker.internal:3000`. If your app runs on a different port, change it in `e2e.config.js` after init.
|
|
65
|
-
>
|
|
66
|
-
> **Linux note:** On Docker Engine (not Docker Desktop), you may need to add `--add-host=host.docker.internal:host-gateway` to the Docker run flags, or use your machine's LAN IP directly as the `baseUrl`.
|
|
67
|
-
|
|
68
|
-
---
|
|
69
|
-
|
|
70
|
-
### Path A: With Claude Code
|
|
71
|
-
|
|
72
|
-
If you use [Claude Code](https://docs.anthropic.com/en/docs/claude-code), this is the fastest path — Claude handles test creation and debugging for you.
|
|
73
|
-
|
|
74
|
-
**1. Install the package**
|
|
75
|
-
|
|
76
|
-
```bash
|
|
77
|
-
npm install --save-dev @matware/e2e-runner
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
**2. Scaffold the project structure**
|
|
81
|
-
|
|
82
|
-
```bash
|
|
83
|
-
npx e2e-runner init
|
|
84
|
-
```
|
|
85
|
-
|
|
86
|
-
This creates `e2e/tests/` with a sample test and `e2e/screenshots/` for captures.
|
|
87
|
-
|
|
88
|
-
**3. Configure your base URL**
|
|
56
|
+
## Agent Skills
|
|
89
57
|
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
```js
|
|
93
|
-
export default {
|
|
94
|
-
baseUrl: 'http://host.docker.internal:3000', // change 3000 to your port
|
|
95
|
-
};
|
|
96
|
-
```
|
|
97
|
-
|
|
98
|
-
**4. Start the Chrome pool**
|
|
58
|
+
Install E2E testing skills for any coding agent (Claude Code, Cursor, Codex, Copilot, and [40+ more](https://github.com/vercel-labs/skills#supported-agents)):
|
|
99
59
|
|
|
100
60
|
```bash
|
|
101
|
-
npx e2e-runner
|
|
102
|
-
```
|
|
103
|
-
|
|
104
|
-
You should see:
|
|
105
|
-
|
|
106
|
-
```
|
|
107
|
-
✓ Chrome pool started on port 3333 (max 3 sessions)
|
|
61
|
+
npx skills add fastslack/mtw-e2e-runner
|
|
108
62
|
```
|
|
109
63
|
|
|
110
|
-
|
|
64
|
+
This gives your agent the knowledge to create, run, and debug JSON-driven E2E tests — no documentation reading required.
|
|
111
65
|
|
|
112
|
-
|
|
113
|
-
# Add the marketplace (one-time)
|
|
114
|
-
claude plugin marketplace add fastslack/mtw-e2e-runner
|
|
115
|
-
|
|
116
|
-
# Install the plugin
|
|
117
|
-
claude plugin install e2e-runner@matware
|
|
118
|
-
```
|
|
119
|
-
|
|
120
|
-
The plugin gives Claude 13 MCP tools, a workflow skill, 3 slash commands, and 3 specialized agents.
|
|
121
|
-
|
|
122
|
-
**6. Ask Claude to run the sample test**
|
|
123
|
-
|
|
124
|
-
In Claude Code, just say:
|
|
125
|
-
|
|
126
|
-
> "Run all E2E tests"
|
|
127
|
-
|
|
128
|
-
Claude will check the pool, run the sample test, and report back:
|
|
129
|
-
|
|
130
|
-
```
|
|
131
|
-
==================================================
|
|
132
|
-
E2E RESULTS
|
|
133
|
-
==================================================
|
|
134
|
-
Total: 1
|
|
135
|
-
Passed: 1
|
|
136
|
-
Failed: 0
|
|
137
|
-
Rate: 100.00%
|
|
138
|
-
Duration: 1.23s
|
|
139
|
-
==================================================
|
|
140
|
-
```
|
|
141
|
-
|
|
142
|
-
From here, you can ask Claude to create new tests ("test the login flow"), debug failures, or verify GitHub issues.
|
|
66
|
+
> Browse all available skills at [skills.sh](https://skills.sh)
|
|
143
67
|
|
|
144
68
|
---
|
|
145
69
|
|
|
146
|
-
|
|
70
|
+
## Getting Started
|
|
147
71
|
|
|
148
|
-
|
|
72
|
+
**Prerequisites:** Node.js >= 20, Docker running, your app on a known port.
|
|
149
73
|
|
|
150
|
-
|
|
74
|
+
### Quickstart
|
|
151
75
|
|
|
152
76
|
```bash
|
|
153
77
|
npm install --save-dev @matware/e2e-runner
|
|
78
|
+
npx e2e-runner init # creates e2e/tests/ with a sample test
|
|
79
|
+
npx e2e-runner pool start # starts Chrome in Docker
|
|
80
|
+
npx e2e-runner run --all # runs the sample test
|
|
154
81
|
```
|
|
155
82
|
|
|
156
|
-
|
|
83
|
+
Or do it all in one command:
|
|
157
84
|
|
|
158
85
|
```bash
|
|
159
|
-
|
|
86
|
+
curl -fsSL https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/scripts/quickstart.sh | bash
|
|
160
87
|
```
|
|
161
88
|
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
**3. Configure your base URL**
|
|
165
|
-
|
|
166
|
-
Edit `e2e.config.js` and set `baseUrl` to match your app's port:
|
|
89
|
+
After setup, edit `e2e.config.js` to set your app's port:
|
|
167
90
|
|
|
168
91
|
```js
|
|
169
92
|
export default {
|
|
@@ -171,74 +94,25 @@ export default {
|
|
|
171
94
|
};
|
|
172
95
|
```
|
|
173
96
|
|
|
174
|
-
**
|
|
97
|
+
> **Why `host.docker.internal`?** Chrome runs inside Docker and can't reach `localhost` on your machine. This hostname bridges the gap. On Linux (Docker Engine, not Desktop), you may need `--add-host=host.docker.internal:host-gateway` or use your LAN IP directly.
|
|
175
98
|
|
|
176
|
-
|
|
177
|
-
npx e2e-runner pool start
|
|
178
|
-
```
|
|
179
|
-
|
|
180
|
-
You should see:
|
|
181
|
-
|
|
182
|
-
```
|
|
183
|
-
✓ Chrome pool started on port 3333 (max 3 sessions)
|
|
184
|
-
```
|
|
185
|
-
|
|
186
|
-
**5. Run the sample test**
|
|
99
|
+
### Add Claude Code (optional)
|
|
187
100
|
|
|
188
101
|
```bash
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
Expected output:
|
|
193
|
-
|
|
194
|
-
```
|
|
195
|
-
==================================================
|
|
196
|
-
E2E RESULTS
|
|
197
|
-
==================================================
|
|
198
|
-
Total: 1
|
|
199
|
-
Passed: 1
|
|
200
|
-
Failed: 0
|
|
201
|
-
Rate: 100.00%
|
|
202
|
-
Duration: 1.23s
|
|
203
|
-
==================================================
|
|
204
|
-
```
|
|
205
|
-
|
|
206
|
-
A screenshot is saved at `e2e/screenshots/homepage.png`.
|
|
207
|
-
|
|
208
|
-
**6. Write your first real test**
|
|
209
|
-
|
|
210
|
-
Create `e2e/tests/my-first-test.json`:
|
|
211
|
-
|
|
212
|
-
```json
|
|
213
|
-
[
|
|
214
|
-
{
|
|
215
|
-
"name": "homepage-visible",
|
|
216
|
-
"actions": [
|
|
217
|
-
{ "type": "goto", "value": "/" },
|
|
218
|
-
{ "type": "assert_visible", "selector": "body" },
|
|
219
|
-
{ "type": "screenshot", "value": "my-first-test.png" }
|
|
220
|
-
]
|
|
221
|
-
}
|
|
222
|
-
]
|
|
223
|
-
```
|
|
224
|
-
|
|
225
|
-
Run it:
|
|
226
|
-
|
|
227
|
-
```bash
|
|
228
|
-
npx e2e-runner run --suite my-first-test
|
|
102
|
+
claude plugin marketplace add fastslack/mtw-e2e-runner
|
|
103
|
+
claude plugin install e2e-runner@matware
|
|
229
104
|
```
|
|
230
105
|
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
### One-liner quickstart
|
|
106
|
+
This gives Claude 13 MCP tools, slash commands, and specialized agents. Just say *"Run all E2E tests"* or *"Create a test for the login flow"*.
|
|
234
107
|
|
|
235
|
-
|
|
108
|
+
### Add OpenCode (optional)
|
|
236
109
|
|
|
237
110
|
```bash
|
|
238
|
-
|
|
111
|
+
cp node_modules/@matware/e2e-runner/opencode.json ./
|
|
112
|
+
mkdir -p .opencode && cp -r node_modules/@matware/e2e-runner/.opencode/* .opencode/
|
|
239
113
|
```
|
|
240
114
|
|
|
241
|
-
|
|
115
|
+
See [OPENCODE.md](OPENCODE.md) for details.
|
|
242
116
|
|
|
243
117
|
### What's next?
|
|
244
118
|
|
|
@@ -408,11 +282,7 @@ Serial tests run one at a time **after** all parallel tests finish — preventin
|
|
|
408
282
|
|
|
409
283
|
## Testing Authenticated Apps
|
|
410
284
|
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
### Strategy 1: UI Login Flow (any app)
|
|
414
|
-
|
|
415
|
-
The most universal approach — fill in the login form like a real user. Works with **any** authentication system (session cookies, JWT, OAuth redirect, etc.):
|
|
285
|
+
The simplest approach — log in via the UI like a real user:
|
|
416
286
|
|
|
417
287
|
```json
|
|
418
288
|
{
|
|
@@ -425,279 +295,29 @@ The most universal approach — fill in the login form like a real user. Works w
|
|
|
425
295
|
{ "type": "wait", "selector": ".dashboard" }
|
|
426
296
|
]
|
|
427
297
|
},
|
|
428
|
-
"tests": [
|
|
429
|
-
{
|
|
430
|
-
"name": "profile-page",
|
|
431
|
-
"actions": [
|
|
432
|
-
{ "type": "goto", "value": "/profile" },
|
|
433
|
-
{ "type": "assert_text", "text": "My Profile" }
|
|
434
|
-
]
|
|
435
|
-
}
|
|
436
|
-
]
|
|
437
|
-
}
|
|
438
|
-
```
|
|
439
|
-
|
|
440
|
-
> **When to use:** You don't know or care how auth works internally. The browser handles cookies/tokens automatically after login — just like a real user.
|
|
441
|
-
|
|
442
|
-
### Strategy 2: JWT Token Injection (SPAs)
|
|
443
|
-
|
|
444
|
-
For single-page apps that store JWT tokens in `localStorage` or `sessionStorage`. Skip the login form entirely by injecting the token directly:
|
|
445
|
-
|
|
446
|
-
```json
|
|
447
|
-
{
|
|
448
|
-
"hooks": {
|
|
449
|
-
"beforeEach": [
|
|
450
|
-
{ "type": "goto", "value": "/" },
|
|
451
|
-
{ "type": "set_storage", "value": "accessToken=eyJhbGciOiJIUzI1NiIs..." },
|
|
452
|
-
{ "type": "goto", "value": "/dashboard" },
|
|
453
|
-
{ "type": "wait", "selector": ".dashboard-loaded" }
|
|
454
|
-
]
|
|
455
|
-
},
|
|
456
298
|
"tests": [...]
|
|
457
299
|
}
|
|
458
300
|
```
|
|
459
301
|
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
| Framework / Library | Typical key | Storage |
|
|
463
|
-
|---------------------|-------------|---------|
|
|
464
|
-
| Custom JWT | `accessToken`, `token`, `jwt` | localStorage |
|
|
465
|
-
| Auth0 SPA SDK | `@@auth0spajs@@::*` | localStorage |
|
|
466
|
-
| Firebase Auth | `firebase:authUser:*` | localStorage |
|
|
467
|
-
| AWS Amplify | `CognitoIdentityServiceProvider.*` | localStorage |
|
|
468
|
-
| Supabase | `sb-<ref>-auth-token` | localStorage |
|
|
469
|
-
| NextAuth (client) | `next-auth.session-token` | cookie (see Strategy 4) |
|
|
470
|
-
|
|
471
|
-
**Using `sessionStorage` instead:**
|
|
302
|
+
For SPAs with JWT, skip the login form by injecting the token directly:
|
|
472
303
|
|
|
473
304
|
```json
|
|
474
|
-
{ "type": "set_storage", "value": "
|
|
305
|
+
{ "type": "set_storage", "value": "accessToken=eyJhbGciOiJIUzI1NiIs..." }
|
|
475
306
|
```
|
|
476
307
|
|
|
477
|
-
|
|
478
|
-
|
|
479
|
-
```json
|
|
480
|
-
{ "type": "assert_storage", "value": "accessToken" }
|
|
481
|
-
{ "type": "assert_storage", "value": "accessToken=eyJhbG..." }
|
|
482
|
-
```
|
|
483
|
-
|
|
484
|
-
> **When to use:** Your SPA reads auth tokens from browser storage. Fastest strategy — no network round-trip for login.
|
|
485
|
-
|
|
486
|
-
### Strategy 3: Config-Level Auth Token
|
|
487
|
-
|
|
488
|
-
For apps where every test needs the same JWT token. Set it once in config — it's injected into `localStorage` before every `e2e_capture` and `e2e_issue --verify` run:
|
|
308
|
+
Or set it globally in config:
|
|
489
309
|
|
|
490
310
|
```js
|
|
491
311
|
// e2e.config.js
|
|
492
312
|
export default {
|
|
493
313
|
authToken: 'eyJhbGciOiJIUzI1NiIs...',
|
|
494
|
-
authStorageKey: 'accessToken',
|
|
314
|
+
authStorageKey: 'accessToken',
|
|
495
315
|
};
|
|
496
316
|
```
|
|
497
317
|
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
```bash
|
|
501
|
-
AUTH_TOKEN="eyJhbGciOiJIUzI1NiIs..." npx e2e-runner run --all
|
|
502
|
-
```
|
|
503
|
-
|
|
504
|
-
Or via CLI:
|
|
505
|
-
|
|
506
|
-
```bash
|
|
507
|
-
npx e2e-runner run --all --auth-token "eyJhbG..." --auth-storage-key "jwt"
|
|
508
|
-
```
|
|
509
|
-
|
|
510
|
-
MCP tools (`e2e_capture`, `e2e_issue`) also accept `authToken` and `authStorageKey` per call.
|
|
511
|
-
|
|
512
|
-
> **When to use:** All tests share the same user session and your app uses JWT in localStorage.
|
|
318
|
+
Each test runs in a **fresh browser context**, so auth state is automatically clean between tests.
|
|
513
319
|
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
For apps that use HTTP cookies (Rails, Django, Laravel, Express sessions, NextAuth, etc.). Use `evaluate` to set cookies before navigating:
|
|
517
|
-
|
|
518
|
-
```json
|
|
519
|
-
{
|
|
520
|
-
"hooks": {
|
|
521
|
-
"beforeEach": [
|
|
522
|
-
{ "type": "goto", "value": "/" },
|
|
523
|
-
{ "type": "evaluate", "value": "document.cookie = 'session_id=abc123; path=/; SameSite=Lax'" },
|
|
524
|
-
{ "type": "goto", "value": "/dashboard" }
|
|
525
|
-
]
|
|
526
|
-
},
|
|
527
|
-
"tests": [...]
|
|
528
|
-
}
|
|
529
|
-
```
|
|
530
|
-
|
|
531
|
-
**Multiple cookies:**
|
|
532
|
-
|
|
533
|
-
```json
|
|
534
|
-
{ "type": "evaluate", "value": "document.cookie = 'session_id=abc123; path=/'; document.cookie = '_csrf_token=xyz789; path=/'" }
|
|
535
|
-
```
|
|
536
|
-
|
|
537
|
-
**For `HttpOnly` cookies** (can't be set via JavaScript), use the UI login strategy instead — the browser will store them automatically.
|
|
538
|
-
|
|
539
|
-
> **When to use:** Traditional server-rendered apps, or any app that authenticates via cookies.
|
|
540
|
-
|
|
541
|
-
### Strategy 5: HTTP Header Auth (API tests)
|
|
542
|
-
|
|
543
|
-
For API testing where you need to send `Authorization` headers with every request. Use `evaluate` to override `fetch`/`XMLHttpRequest`:
|
|
544
|
-
|
|
545
|
-
```json
|
|
546
|
-
{
|
|
547
|
-
"hooks": {
|
|
548
|
-
"beforeEach": [
|
|
549
|
-
{ "type": "goto", "value": "/" },
|
|
550
|
-
{ "type": "evaluate", "value": "const origFetch = window.fetch; window.fetch = (url, opts = {}) => { opts.headers = { ...opts.headers, 'Authorization': 'Bearer eyJhbG...' }; return origFetch(url, opts); }" }
|
|
551
|
-
]
|
|
552
|
-
},
|
|
553
|
-
"tests": [
|
|
554
|
-
{
|
|
555
|
-
"name": "api-returns-user",
|
|
556
|
-
"actions": [
|
|
557
|
-
{ "type": "evaluate", "value": "const res = await fetch('/api/me'); const data = await res.json(); if (data.email !== 'test@example.com') throw new Error('Wrong user: ' + data.email)" }
|
|
558
|
-
]
|
|
559
|
-
}
|
|
560
|
-
]
|
|
561
|
-
}
|
|
562
|
-
```
|
|
563
|
-
|
|
564
|
-
> **When to use:** API-level tests (with `--test-type api`) that need auth headers.
|
|
565
|
-
|
|
566
|
-
### Strategy 6: OAuth / SSO (external provider)
|
|
567
|
-
|
|
568
|
-
OAuth flows redirect to external providers (Google, GitHub, Okta, etc.) which can't be automated reliably. Common workarounds:
|
|
569
|
-
|
|
570
|
-
**Option A — Test environment bypass:** Most apps have a direct login endpoint for testing that skips OAuth:
|
|
571
|
-
|
|
572
|
-
```json
|
|
573
|
-
{ "type": "goto", "value": "/auth/test-login?user=test@example.com" }
|
|
574
|
-
```
|
|
575
|
-
|
|
576
|
-
**Option B — Pre-authenticated token:** Get a token from your auth provider's API and inject it:
|
|
577
|
-
|
|
578
|
-
```json
|
|
579
|
-
{
|
|
580
|
-
"hooks": {
|
|
581
|
-
"beforeEach": [
|
|
582
|
-
{ "type": "goto", "value": "/" },
|
|
583
|
-
{ "type": "set_storage", "value": "oidc.user:https://auth.example.com:client_id={\"access_token\":\"...\"}" }
|
|
584
|
-
]
|
|
585
|
-
}
|
|
586
|
-
}
|
|
587
|
-
```
|
|
588
|
-
|
|
589
|
-
**Option C — Session cookie from CI:** If your CI can authenticate via API, pass the session cookie as an env var:
|
|
590
|
-
|
|
591
|
-
```bash
|
|
592
|
-
SESSION=$(curl -s -c - https://api.example.com/auth/login -d '{"email":"test@example.com","password":"secret"}' | grep session_id | awk '{print $NF}')
|
|
593
|
-
AUTH_TOKEN="$SESSION" AUTH_STORAGE_KEY="session_id" npx e2e-runner run --all
|
|
594
|
-
```
|
|
595
|
-
|
|
596
|
-
> **When to use:** Apps with Google/GitHub/Okta/Auth0 login. You almost always need a test-environment backdoor.
|
|
597
|
-
|
|
598
|
-
### Reusable Auth Modules
|
|
599
|
-
|
|
600
|
-
Extract your auth strategy into a module so every test can reference it without duplication:
|
|
601
|
-
|
|
602
|
-
```json
|
|
603
|
-
// e2e/modules/login.json — UI login (universal)
|
|
604
|
-
{
|
|
605
|
-
"$module": "login",
|
|
606
|
-
"description": "Log in via the UI login form",
|
|
607
|
-
"params": {
|
|
608
|
-
"email": { "required": true, "description": "User email" },
|
|
609
|
-
"password": { "required": true, "description": "User password" },
|
|
610
|
-
"redirectTo": { "default": "/dashboard", "description": "Page to land on after login" }
|
|
611
|
-
},
|
|
612
|
-
"actions": [
|
|
613
|
-
{ "type": "goto", "value": "/login" },
|
|
614
|
-
{ "type": "type", "selector": "#email", "value": "{{email}}" },
|
|
615
|
-
{ "type": "type", "selector": "#password", "value": "{{password}}" },
|
|
616
|
-
{ "type": "click", "text": "Sign In" },
|
|
617
|
-
{ "type": "wait", "selector": "{{redirectTo}}" }
|
|
618
|
-
]
|
|
619
|
-
}
|
|
620
|
-
```
|
|
621
|
-
|
|
622
|
-
```json
|
|
623
|
-
// e2e/modules/auth-token.json — JWT injection (SPAs)
|
|
624
|
-
{
|
|
625
|
-
"$module": "auth-token",
|
|
626
|
-
"description": "Inject an auth token into browser storage",
|
|
627
|
-
"params": {
|
|
628
|
-
"token": { "required": true, "description": "JWT or session token" },
|
|
629
|
-
"storageKey": { "default": "accessToken", "description": "Storage key name" },
|
|
630
|
-
"storage": { "default": "local", "description": "local or session" },
|
|
631
|
-
"redirectTo": { "default": "/dashboard", "description": "Page to navigate to after injection" }
|
|
632
|
-
},
|
|
633
|
-
"actions": [
|
|
634
|
-
{ "type": "goto", "value": "/" },
|
|
635
|
-
{ "type": "set_storage", "value": "{{storageKey}}={{token}}", "selector": "{{#storage}}{{storage}}{{/storage}}" },
|
|
636
|
-
{ "type": "goto", "value": "{{redirectTo}}" }
|
|
637
|
-
]
|
|
638
|
-
}
|
|
639
|
-
```
|
|
640
|
-
|
|
641
|
-
Use in tests:
|
|
642
|
-
|
|
643
|
-
```json
|
|
644
|
-
// UI login
|
|
645
|
-
{ "$use": "login", "params": { "email": "admin@test.com", "password": "secret" } }
|
|
646
|
-
|
|
647
|
-
// Token injection
|
|
648
|
-
{ "$use": "auth-token", "params": { "token": "eyJhbG..." } }
|
|
649
|
-
|
|
650
|
-
// Token in sessionStorage, redirect to /settings
|
|
651
|
-
{ "$use": "auth-token", "params": { "token": "eyJhbG...", "storage": "session", "redirectTo": "/settings" } }
|
|
652
|
-
```
|
|
653
|
-
|
|
654
|
-
### Testing Different User Roles
|
|
655
|
-
|
|
656
|
-
Use separate tests (or the same module with different credentials) to test role-based access:
|
|
657
|
-
|
|
658
|
-
```json
|
|
659
|
-
[
|
|
660
|
-
{
|
|
661
|
-
"name": "admin-sees-settings",
|
|
662
|
-
"actions": [
|
|
663
|
-
{ "$use": "login", "params": { "email": "admin@test.com", "password": "admin-pass" } },
|
|
664
|
-
{ "type": "goto", "value": "/settings" },
|
|
665
|
-
{ "type": "assert_visible", "selector": ".admin-panel" }
|
|
666
|
-
]
|
|
667
|
-
},
|
|
668
|
-
{
|
|
669
|
-
"name": "viewer-cannot-access-settings",
|
|
670
|
-
"actions": [
|
|
671
|
-
{ "$use": "login", "params": { "email": "viewer@test.com", "password": "viewer-pass" } },
|
|
672
|
-
{ "type": "goto", "value": "/settings" },
|
|
673
|
-
{ "type": "assert_text", "text": "Access Denied" }
|
|
674
|
-
]
|
|
675
|
-
}
|
|
676
|
-
]
|
|
677
|
-
```
|
|
678
|
-
|
|
679
|
-
### Clearing Auth State
|
|
680
|
-
|
|
681
|
-
Each test runs in a **fresh browser context** (new connection to the Chrome pool), so cookies and storage are automatically clean. If you need to explicitly clear state mid-test:
|
|
682
|
-
|
|
683
|
-
```json
|
|
684
|
-
{ "type": "clear_cookies" }
|
|
685
|
-
```
|
|
686
|
-
|
|
687
|
-
This clears cookies, localStorage, and sessionStorage for the current origin.
|
|
688
|
-
|
|
689
|
-
### Quick Reference
|
|
690
|
-
|
|
691
|
-
| Auth type | Strategy | Key actions |
|
|
692
|
-
|-----------|----------|-------------|
|
|
693
|
-
| Username/password form | UI Login | `goto` + `type` + `click` in `beforeEach` |
|
|
694
|
-
| JWT in localStorage | Token Injection | `set_storage` in `beforeEach` |
|
|
695
|
-
| JWT in sessionStorage | Token Injection | `set_storage` with `selector: "session"` |
|
|
696
|
-
| Session cookies | Cookie | `evaluate` to set `document.cookie` |
|
|
697
|
-
| HttpOnly cookies | UI Login | Must go through login form |
|
|
698
|
-
| OAuth / SSO | Test bypass | App-specific test login endpoint |
|
|
699
|
-
| API auth headers | Header Override | `evaluate` to patch `fetch` |
|
|
700
|
-
| Config-level token | Config | `authToken` + `authStorageKey` in config |
|
|
320
|
+
> **More strategies:** Cookie-based auth, HTTP header injection, OAuth/SSO bypasses, reusable auth modules, and role-based testing — see [docs/authentication.md](docs/authentication.md)
|
|
701
321
|
|
|
702
322
|
---
|
|
703
323
|
|
|
@@ -907,137 +527,52 @@ Every screenshot gets a deterministic hash (`ss:a3f2b1c9`). Use `e2e_screenshot`
|
|
|
907
527
|
|
|
908
528
|
---
|
|
909
529
|
|
|
910
|
-
##
|
|
530
|
+
## AI Integration
|
|
911
531
|
|
|
912
|
-
|
|
913
|
-
|
|
914
|
-
### Install as Plugin (recommended)
|
|
532
|
+
### Claude Code
|
|
915
533
|
|
|
916
534
|
```bash
|
|
917
|
-
# 1. Add the marketplace (one-time)
|
|
918
535
|
claude plugin marketplace add fastslack/mtw-e2e-runner
|
|
919
|
-
|
|
920
|
-
# 2. Install the plugin
|
|
921
536
|
claude plugin install e2e-runner@matware
|
|
922
537
|
```
|
|
923
538
|
|
|
924
|
-
|
|
925
|
-
|
|
926
|
-
| Component | Description |
|
|
927
|
-
|-----------|-------------|
|
|
928
|
-
| **13 MCP tools** | Run tests, create test files, capture screenshots, query network logs, manage dashboard, verify issues, query learnings |
|
|
929
|
-
| **Skill** | Teaches Claude the full e2e-runner workflow — how to combine tools, interpret results, debug failures, create tests |
|
|
930
|
-
| **3 Commands** | `/e2e-runner:run` — run & analyze tests<br>`/e2e-runner:create-test` — explore UI and create tests<br>`/e2e-runner:verify-issue <url>` — verify GitHub/GitLab bugs |
|
|
931
|
-
| **3 Agents** | **test-analyzer** — diagnoses failures, analyzes flaky tests, drills into network errors<br>**test-creator** — explores UI, discovers selectors, designs and validates tests<br>**test-improver** — refactors verbose evaluate actions, extracts modules, adds waits/retries, eliminates hardcoded delays |
|
|
932
|
-
|
|
933
|
-
### Install MCP-only (alternative)
|
|
539
|
+
This gives Claude 13 MCP tools, a workflow skill, 3 slash commands (`/e2e-runner:run`, `/e2e-runner:create-test`, `/e2e-runner:verify-issue`), and 3 specialized agents (test-analyzer, test-creator, test-improver).
|
|
934
540
|
|
|
935
|
-
|
|
541
|
+
**MCP-only install** (tools only, no skill/commands/agents):
|
|
936
542
|
|
|
937
543
|
```bash
|
|
938
544
|
claude mcp add --transport stdio --scope user e2e-runner \
|
|
939
545
|
-- npx -y -p @matware/e2e-runner e2e-runner-mcp
|
|
940
546
|
```
|
|
941
547
|
|
|
942
|
-
###
|
|
943
|
-
|
|
944
|
-
| Command | Description |
|
|
945
|
-
|---------|-------------|
|
|
946
|
-
| `/e2e-runner:run` | Check pool, list suites, run tests, analyze results with screenshots and network drill-down |
|
|
947
|
-
| `/e2e-runner:create-test` | Explore the UI with screenshots, find selectors in source code, design test actions, create and validate |
|
|
948
|
-
| `/e2e-runner:verify-issue <url>` | Fetch a GitHub/GitLab issue, create tests that verify correct behavior, report bug confirmed or not reproducible |
|
|
949
|
-
|
|
950
|
-
### MCP Tools
|
|
951
|
-
|
|
952
|
-
| Tool | Description |
|
|
953
|
-
|------|-------------|
|
|
954
|
-
| `e2e_run` | Run tests: all suites, by name, or by file. Supports `concurrency`, `baseUrl`, `retries`, `failOnNetworkError` overrides. Returns verification results if tests have `expect`. |
|
|
955
|
-
| `e2e_list` | List available test suites with test names and counts |
|
|
956
|
-
| `e2e_create_test` | Create a new test JSON file with name, tests, and optional hooks |
|
|
957
|
-
| `e2e_create_module` | Create a reusable module with parameterized actions |
|
|
958
|
-
| `e2e_pool_status` | Check Chrome pool availability, running sessions, capacity |
|
|
959
|
-
| `e2e_screenshot` | Retrieve a screenshot by hash (`ss:a3f2b1c9`). Returns image + metadata |
|
|
960
|
-
| `e2e_capture` | Capture screenshot of any URL. Supports `authToken`, `fullPage`, `selector`, `delay` |
|
|
961
|
-
| `e2e_dashboard_start` | Start the web dashboard |
|
|
962
|
-
| `e2e_dashboard_stop` | Stop the web dashboard |
|
|
963
|
-
| `e2e_issue` | Fetch GitHub/GitLab issue and generate tests. `mode: "prompt"` or `mode: "verify"` |
|
|
964
|
-
| `e2e_network_logs` | Query network request/response logs by `runDbId`. Filter by test name, method, status, URL pattern. Supports headers and bodies |
|
|
965
|
-
| `e2e_learnings` | Query the learning system: `summary`, `flaky`, `selectors`, `pages`, `apis`, `errors`, `trends` |
|
|
966
|
-
| `e2e_neo4j` | Manage Neo4j knowledge graph container: `start`, `stop`, `status` |
|
|
967
|
-
|
|
968
|
-
> **Note:** Pool start/stop are CLI-only (`e2e-runner pool start|stop`) — not exposed via MCP to prevent killing active sessions.
|
|
969
|
-
|
|
970
|
-
### What You Can Ask Claude Code
|
|
971
|
-
|
|
972
|
-
> "Run all E2E tests"
|
|
973
|
-
> "Create a test that verifies the checkout flow"
|
|
974
|
-
> "What tests are flaky? Show me the learning summary"
|
|
975
|
-
> "Capture a screenshot of /dashboard with auth"
|
|
976
|
-
> "Fetch issue #42 and create tests for it"
|
|
977
|
-
> "What's the API error rate for the last 7 days?"
|
|
978
|
-
|
|
979
|
-
---
|
|
980
|
-
|
|
981
|
-
## OpenCode Integration
|
|
982
|
-
|
|
983
|
-
The package also supports [OpenCode](https://github.com/anomalyco/opencode) with native MCP server configuration, skills, and commands.
|
|
984
|
-
|
|
985
|
-
### Quick Setup
|
|
548
|
+
### OpenCode
|
|
986
549
|
|
|
987
550
|
```bash
|
|
988
|
-
# 1. Install the package
|
|
989
|
-
npm install --save-dev @matware/e2e-runner
|
|
990
|
-
|
|
991
|
-
# 2. Copy OpenCode config to your project
|
|
992
551
|
cp node_modules/@matware/e2e-runner/opencode.json ./
|
|
993
|
-
|
|
994
|
-
# 3. Copy skills and commands (optional)
|
|
995
|
-
mkdir -p .opencode
|
|
996
|
-
cp -r node_modules/@matware/e2e-runner/.opencode/* .opencode/
|
|
997
|
-
|
|
998
|
-
# 4. Start the Chrome pool
|
|
999
|
-
npx e2e-runner pool start
|
|
552
|
+
mkdir -p .opencode && cp -r node_modules/@matware/e2e-runner/.opencode/* .opencode/
|
|
1000
553
|
```
|
|
1001
554
|
|
|
1002
|
-
|
|
1003
|
-
|
|
1004
|
-
| Component | Description |
|
|
1005
|
-
|-----------|-------------|
|
|
1006
|
-
| **15 MCP tools** | Same tools as Claude Code — run tests, create files, screenshots, network logs, learnings, etc. |
|
|
1007
|
-
| **Skill** | `e2e-testing` — full workflow guidance with references |
|
|
1008
|
-
| **3 Commands** | `/run`, `/create-test`, `/verify-issue` |
|
|
1009
|
-
|
|
1010
|
-
### MCP Configuration
|
|
555
|
+
See [OPENCODE.md](OPENCODE.md) for details.
|
|
1011
556
|
|
|
1012
|
-
|
|
1013
|
-
|
|
1014
|
-
```json
|
|
1015
|
-
{
|
|
1016
|
-
"mcp": {
|
|
1017
|
-
"e2e-runner": {
|
|
1018
|
-
"type": "local",
|
|
1019
|
-
"command": "node",
|
|
1020
|
-
"args": ["node_modules/@matware/e2e-runner/bin/mcp-server.js"],
|
|
1021
|
-
"cwd": "${workspaceFolder}"
|
|
1022
|
-
}
|
|
1023
|
-
}
|
|
1024
|
-
}
|
|
1025
|
-
```
|
|
1026
|
-
|
|
1027
|
-
For global installation, use the binary directly:
|
|
1028
|
-
|
|
1029
|
-
```json
|
|
1030
|
-
{
|
|
1031
|
-
"mcp": {
|
|
1032
|
-
"e2e-runner": {
|
|
1033
|
-
"type": "local",
|
|
1034
|
-
"command": "e2e-runner-mcp"
|
|
1035
|
-
}
|
|
1036
|
-
}
|
|
1037
|
-
}
|
|
1038
|
-
```
|
|
557
|
+
### MCP Tools
|
|
1039
558
|
|
|
1040
|
-
|
|
559
|
+
| Tool | Description |
|
|
560
|
+
|------|-------------|
|
|
561
|
+
| `e2e_run` | Run tests (all, by suite, or by file) |
|
|
562
|
+
| `e2e_list` | List available test suites |
|
|
563
|
+
| `e2e_create_test` | Create a new test JSON file |
|
|
564
|
+
| `e2e_create_module` | Create a reusable module |
|
|
565
|
+
| `e2e_pool_status` | Check Chrome pool health |
|
|
566
|
+
| `e2e_screenshot` | Retrieve a screenshot by hash |
|
|
567
|
+
| `e2e_capture` | Capture screenshot of any URL |
|
|
568
|
+
| `e2e_dashboard_start` | Start web dashboard |
|
|
569
|
+
| `e2e_dashboard_stop` | Stop web dashboard |
|
|
570
|
+
| `e2e_issue` | Fetch issue and generate tests |
|
|
571
|
+
| `e2e_network_logs` | Query network logs for a run |
|
|
572
|
+
| `e2e_learnings` | Query stability insights |
|
|
573
|
+
| `e2e_neo4j` | Manage Neo4j knowledge graph |
|
|
574
|
+
|
|
575
|
+
> Pool start/stop are CLI-only — not exposed via MCP.
|
|
1041
576
|
|
|
1042
577
|
---
|
|
1043
578
|
|