@matware/e2e-runner 1.5.0 → 1.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +3 -3
- package/.claude-plugin/plugin.json +1 -1
- package/LICENSE +1 -1
- package/README.md +451 -274
- package/agents/test-improver.md +2 -1
- package/bin/cli.js +13 -2
- package/package.json +2 -2
- package/skills/e2e-testing/SKILL.md +2 -1
- package/skills/e2e-testing/references/action-types.md +17 -18
- package/skills/e2e-testing/references/troubleshooting.md +2 -26
- package/src/actions.js +12 -2
- package/src/dashboard.js +50 -5
- package/src/db.js +15 -0
- package/src/mcp-tools.js +238 -75
- package/src/narrate.js +19 -0
- package/src/runner.js +72 -14
- package/src/visual-diff.js +8 -7
- package/templates/dashboard/js/utils.js +23 -2
- package/templates/dashboard/js/view-runs.js +94 -9
- package/templates/dashboard/styles/components.css +17 -0
- package/templates/dashboard/styles/view-runs.css +51 -4
- package/templates/dashboard/template.html +2 -2
- package/templates/dashboard.html +187 -17
package/README.md
CHANGED
|
@@ -21,87 +21,133 @@
|
|
|
21
21
|
<a href="https://skills.sh"><img src="https://img.shields.io/badge/skills.sh-e2e--testing-ff6600" alt="Agent Skills" /></a>
|
|
22
22
|
</p>
|
|
23
23
|
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
**E2E Runner** lets you test your web app without writing test code. Tests are plain JSON — and you don't even have to write that yourself: **just ask Claude Code.**
|
|
27
|
+
|
|
28
|
+
## 🎬 Write a test by asking — then watch it run
|
|
29
|
+
|
|
24
30
|
<p align="center">
|
|
25
|
-
<img src="https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/docs/screenshots/
|
|
31
|
+
<img src="https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/docs/screenshots/demo-live.gif" alt="Live dashboard streaming screenshots as a test suite runs" width="820" />
|
|
32
|
+
<br/><sub><em>The live dashboard while a suite runs — every step streams a screenshot into the feed, in real time.</em></sub>
|
|
26
33
|
</p>
|
|
27
34
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
**E2E Runner** is a zero-code browser testing framework where tests are plain JSON files — no Playwright scripts, no Cypress boilerplate, no test framework to learn. Define what to click, type, and assert, and the runner executes it in parallel against a shared Chrome pool.
|
|
35
|
+
With the built-in [MCP server](https://modelcontextprotocol.io/), creating a test is a conversation — no docs, no syntax to memorize:
|
|
31
36
|
|
|
32
|
-
|
|
37
|
+
> **You:** *Create an E2E test for the login flow and run it.*
|
|
38
|
+
>
|
|
39
|
+
> **Claude Code:** *writes the test, runs it in a real browser, and reports back —*
|
|
40
|
+
> ✅ `login-flow` passed in 2.3s · screenshot saved · no network errors.
|
|
33
41
|
|
|
34
|
-
|
|
42
|
+
Behind the scenes Claude just wrote and ran this. A test is **just JSON** — an ordered list of what a user does:
|
|
35
43
|
|
|
36
44
|
```json
|
|
37
45
|
[
|
|
38
|
-
{
|
|
39
|
-
"
|
|
40
|
-
"
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
{ "type": "screenshot", "value": "logged-in.png" }
|
|
47
|
-
]
|
|
48
|
-
}
|
|
46
|
+
{ "name": "login-flow", "actions": [
|
|
47
|
+
{ "type": "goto", "value": "/login" },
|
|
48
|
+
{ "type": "type", "selector": "#email", "value": "user@test.com" },
|
|
49
|
+
{ "type": "type", "selector": "#password", "value": "secret" },
|
|
50
|
+
{ "type": "click", "text": "Sign In" },
|
|
51
|
+
{ "type": "assert_text", "text": "Welcome back" },
|
|
52
|
+
{ "type": "screenshot", "value": "logged-in.png" }
|
|
53
|
+
]}
|
|
49
54
|
]
|
|
50
55
|
```
|
|
51
56
|
|
|
52
|
-
|
|
57
|
+
No imports, no `describe`/`it`, no build step. If you can read it you can write it — or just ask.
|
|
53
58
|
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
## Agent Skills
|
|
57
|
-
|
|
58
|
-
Install E2E testing skills for any coding agent (Claude Code, Cursor, Codex, Copilot, and [40+ more](https://github.com/vercel-labs/skills#supported-agents)):
|
|
59
|
+
**Connect it to Claude Code (2 commands):**
|
|
59
60
|
|
|
60
61
|
```bash
|
|
61
|
-
|
|
62
|
+
claude plugin marketplace add fastslack/mtw-e2e-runner
|
|
63
|
+
claude plugin install e2e-runner@matware
|
|
62
64
|
```
|
|
63
65
|
|
|
64
|
-
|
|
66
|
+
Now say *"create a test for X and run it"* — Claude gets 17 MCP tools, slash commands, and specialized agents.
|
|
65
67
|
|
|
66
|
-
>
|
|
68
|
+
> Using a different agent (Cursor, Codex, Copilot, [40+ more](https://github.com/vercel-labs/skills#supported-agents))? Install the skill: `npx skills add fastslack/mtw-e2e-runner`
|
|
67
69
|
|
|
68
70
|
---
|
|
69
71
|
|
|
70
|
-
##
|
|
72
|
+
## 📖 Contents
|
|
73
|
+
|
|
74
|
+
| | Section | What's inside |
|
|
75
|
+
|---|---------|---------------|
|
|
76
|
+
| 🚀 | **[Install & first test](#install)** | npm setup · run with your own Chrome (no Docker), Obscura, or a Docker pool |
|
|
77
|
+
| ✨ | **[What you get](#features)** | feature overview at a glance |
|
|
78
|
+
| ✍️ | **[Writing tests](#writing-tests)** | test format · full action catalog · retries · serial · modules · auth · hooks |
|
|
79
|
+
| 🤖 | **[AI integration](#ai)** | Claude Code · OpenCode · 17 MCP tools · visual verification · issue-to-test |
|
|
80
|
+
| 📊 | **[Dashboard & insights](#dashboard)** | live dashboard · learning system · network logs · screenshot capture |
|
|
81
|
+
| 🌐 | **[Browser drivers](#drivers)** | browserless · cdp · lightpanda · obscura · steel |
|
|
82
|
+
| ⚙️ | **[CLI, config & CI](#reference)** | commands · flags · `e2e.config.js` · GitHub Actions · programmatic API |
|
|
83
|
+
|
|
84
|
+
---
|
|
71
85
|
|
|
72
|
-
|
|
86
|
+
<a name="install"></a>
|
|
73
87
|
|
|
74
|
-
|
|
88
|
+
## 🚀 Install — it's tiny
|
|
75
89
|
|
|
76
90
|
```bash
|
|
77
91
|
npm install --save-dev @matware/e2e-runner
|
|
78
92
|
npx e2e-runner init # scaffolds e2e/ with a sample test + config
|
|
79
|
-
npx e2e-runner run --all # runs it — Chrome starts automatically on first run
|
|
80
93
|
```
|
|
81
94
|
|
|
82
|
-
|
|
95
|
+
Then pick how to run the browser. **You don't need Docker** unless you want the parallel pool:
|
|
96
|
+
|
|
97
|
+
### Option 1 · Use the Chrome you already have — no Docker ⭐
|
|
83
98
|
|
|
84
|
-
|
|
99
|
+
Launch any Chromium browser with a debugging port, then point the runner at it:
|
|
85
100
|
|
|
86
|
-
|
|
101
|
+
```bash
|
|
102
|
+
google-chrome --headless=new --remote-debugging-port=9222 & # or brave / chromium / msedge
|
|
103
|
+
CHROME_POOL_URL=http://localhost:9222 POOL_DRIVER=cdp npx e2e-runner run --all
|
|
104
|
+
```
|
|
87
105
|
|
|
88
|
-
|
|
106
|
+
Or bake it into `e2e.config.js` so you never repeat it:
|
|
89
107
|
|
|
90
108
|
```js
|
|
91
109
|
export default {
|
|
92
|
-
baseUrl: 'http://
|
|
110
|
+
baseUrl: 'http://localhost:3000', // your app — plain localhost, no docker hostname
|
|
111
|
+
poolUrls: ['http://localhost:9222'],
|
|
112
|
+
poolDriver: 'cdp',
|
|
93
113
|
};
|
|
94
114
|
```
|
|
95
115
|
|
|
116
|
+
Nothing to install beyond npm, and `baseUrl` is just `localhost` (the browser is on your machine).
|
|
117
|
+
|
|
118
|
+
### Option 2 · Obscura — one tiny binary, no Docker
|
|
119
|
+
|
|
120
|
+
A single ~30 MB binary with built-in anti-detection. Install once, run it, point the runner at it:
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
obscura serve --port 9222 --stealth &
|
|
124
|
+
CHROME_POOL_URL=http://localhost:9222 POOL_DRIVER=obscura npx e2e-runner run --all
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
`npx e2e-runner pool start` (with `poolDriver: 'obscura'` in your config) prints the exact install command for your OS.
|
|
128
|
+
|
|
129
|
+
### Option 3 · Docker pool — parallel, for CI & big suites
|
|
130
|
+
|
|
131
|
+
A shared, queue-managed Chrome pool that runs many tests at once:
|
|
132
|
+
|
|
133
|
+
```bash
|
|
134
|
+
npx e2e-runner run --all # the first run auto-starts the Docker pool for you
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Requires Docker. Set `baseUrl: 'http://host.docker.internal:3000'` so the containerized Chrome can reach your app.
|
|
138
|
+
|
|
96
139
|
<details>
|
|
97
|
-
<summary><strong>Why <code>host.docker.internal</code>
|
|
140
|
+
<summary><strong>Why <code>host.docker.internal</code> (Docker option only)?</strong></summary>
|
|
141
|
+
|
|
142
|
+
<br/>
|
|
143
|
+
|
|
144
|
+
With the Docker pool, Chrome runs inside a container, so `localhost` there means the container — not your machine. `host.docker.internal` bridges to your host. On Linux (Docker Engine, not Docker Desktop) add `--add-host=host.docker.internal:host-gateway`, or use your LAN IP. Options 1 & 2 don't have this — the browser is local, so plain `localhost` just works.
|
|
98
145
|
|
|
99
|
-
Chrome runs inside Docker, so `localhost` there points at the container, not your machine. `host.docker.internal` bridges to your host. On Linux (Docker Engine, not Docker Desktop) you may need to add `--add-host=host.docker.internal:host-gateway`, or just use your machine's LAN IP.
|
|
100
146
|
</details>
|
|
101
147
|
|
|
102
148
|
### Write your first test
|
|
103
149
|
|
|
104
|
-
Open `e2e/tests/sample.json`
|
|
150
|
+
Open `e2e/tests/sample.json` — a flow is an ordered list of actions:
|
|
105
151
|
|
|
106
152
|
```json
|
|
107
153
|
[
|
|
@@ -113,18 +159,12 @@ Open `e2e/tests/sample.json` and describe a flow as a list of actions:
|
|
|
113
159
|
]
|
|
114
160
|
```
|
|
115
161
|
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
### Add Claude Code (optional)
|
|
119
|
-
|
|
120
|
-
```bash
|
|
121
|
-
claude plugin marketplace add fastslack/mtw-e2e-runner
|
|
122
|
-
claude plugin install e2e-runner@matware
|
|
123
|
-
```
|
|
162
|
+
Run it with `npx e2e-runner run --all`. Results — pass/fail, timing, screenshots, network errors — print to your terminal and to the [web dashboard](#dashboard) if it's open.
|
|
124
163
|
|
|
125
|
-
|
|
164
|
+
<details>
|
|
165
|
+
<summary><strong>Add OpenCode</strong> (optional)</summary>
|
|
126
166
|
|
|
127
|
-
|
|
167
|
+
<br/>
|
|
128
168
|
|
|
129
169
|
```bash
|
|
130
170
|
cp node_modules/@matware/e2e-runner/opencode.json ./
|
|
@@ -133,17 +173,32 @@ mkdir -p .opencode && cp -r node_modules/@matware/e2e-runner/.opencode/* .openco
|
|
|
133
173
|
|
|
134
174
|
See [OPENCODE.md](OPENCODE.md) for details.
|
|
135
175
|
|
|
136
|
-
|
|
176
|
+
</details>
|
|
177
|
+
|
|
178
|
+
### Updating
|
|
179
|
+
|
|
180
|
+
Each install method updates separately — bump the one(s) you use:
|
|
181
|
+
|
|
182
|
+
```bash
|
|
183
|
+
# npm dependency (per project)
|
|
184
|
+
npm install --save-dev @matware/e2e-runner@latest
|
|
185
|
+
|
|
186
|
+
# Claude Code plugin
|
|
187
|
+
claude plugin update e2e-runner@matware
|
|
188
|
+
|
|
189
|
+
# MCP-only install (npx caches the package — pin @latest to force a refresh)
|
|
190
|
+
claude mcp add --transport stdio --scope user e2e-runner \
|
|
191
|
+
-- npx -y -p @matware/e2e-runner@latest e2e-runner-mcp
|
|
192
|
+
```
|
|
137
193
|
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
- [Visual Verification](#visual-verification) — describe expected pages in plain English
|
|
141
|
-
- [Issue-to-Test](#issue-to-test) — turn bug reports into executable tests
|
|
142
|
-
- [Web Dashboard](#web-dashboard) — monitor tests in real time
|
|
194
|
+
> [!NOTE]
|
|
195
|
+
> Two gotchas: **(1)** `npx` prefers a copy found in the project's `node_modules` over its own cache — if a project pins an old version, the MCP server and dashboard run that old version, so update the project dependency too. **(2)** Already-running processes keep the old code in memory: after updating, restart the dashboard and reconnect the MCP server (`/mcp` → `e2e-runner` → Reconnect, or restart your session).
|
|
143
196
|
|
|
144
197
|
---
|
|
145
198
|
|
|
146
|
-
|
|
199
|
+
<a name="features"></a>
|
|
200
|
+
|
|
201
|
+
## ✨ What you get
|
|
147
202
|
|
|
148
203
|
🧪 **Zero-code tests** — JSON files that anyone on your team can read and write. No JavaScript, no compilation, no framework lock-in.
|
|
149
204
|
|
|
@@ -173,7 +228,16 @@ See [OPENCODE.md](OPENCODE.md) for details.
|
|
|
173
228
|
|
|
174
229
|
---
|
|
175
230
|
|
|
176
|
-
|
|
231
|
+
<a name="writing-tests"></a>
|
|
232
|
+
|
|
233
|
+
## ✍️ Writing tests
|
|
234
|
+
|
|
235
|
+
Everything about authoring tests — the file format, the full action vocabulary, retries, state isolation, and reuse. Expand what you need:
|
|
236
|
+
|
|
237
|
+
<details>
|
|
238
|
+
<summary><strong>Test format & file layout</strong></summary>
|
|
239
|
+
|
|
240
|
+
<br/>
|
|
177
241
|
|
|
178
242
|
Each `.json` file in `e2e/tests/` contains an array of tests. Each test has a `name` and sequential `actions`:
|
|
179
243
|
|
|
@@ -193,7 +257,12 @@ Each `.json` file in `e2e/tests/` contains an array of tests. Each test has a `n
|
|
|
193
257
|
|
|
194
258
|
Suite files can have numeric prefixes for ordering (`01-auth.json`, `02-dashboard.json`). The `--suite` flag matches with or without the prefix, so `--suite auth` finds `01-auth.json`.
|
|
195
259
|
|
|
196
|
-
|
|
260
|
+
</details>
|
|
261
|
+
|
|
262
|
+
<details>
|
|
263
|
+
<summary><strong>Action catalog</strong> — navigation, input & interaction</summary>
|
|
264
|
+
|
|
265
|
+
<br/>
|
|
197
266
|
|
|
198
267
|
| Action | Fields | Description |
|
|
199
268
|
|--------|--------|-------------|
|
|
@@ -210,12 +279,33 @@ Suite files can have numeric prefixes for ordering (`01-auth.json`, `02-dashboar
|
|
|
210
279
|
| `evaluate` | `value` | Execute JavaScript in the browser context |
|
|
211
280
|
| `navigate` | `value` | Browser navigation (`back`, `forward`, `reload`) |
|
|
212
281
|
| `clear_cookies` | — | Clear all cookies for the current page |
|
|
282
|
+
| `wait_network_idle` | optional `value` (idle ms, default 500), `timeout` | Wait until the network has been idle for `value` ms — useful after actions that trigger background requests |
|
|
283
|
+
| `set_storage` | `value` (`"key=val"`), optional `selector: "session"` | Set a `localStorage` key (or `sessionStorage` with `selector: "session"`) |
|
|
284
|
+
| `gql` | `value` (query), optional `text` (variables JSON), optional `selector` (assertion) | Run a GraphQL query/mutation via in-page `fetch`, with the auth token read from `localStorage`. Fails on GraphQL errors. `selector` is a JS expression asserted against the response `r` (e.g. `"r.data.users.length > 0"`). Installs `window.__e2eGql` for later `evaluate` steps |
|
|
285
|
+
|
|
286
|
+
**Click by text** — when `click` uses `text` instead of `selector`, it searches across common interactive and content elements:
|
|
287
|
+
|
|
288
|
+
```
|
|
289
|
+
button, a, [role="button"], [role="tab"], [role="menuitem"], [role="option"],
|
|
290
|
+
[role="listitem"], div[class*="cursor"], span, li, td, th, label, p, h1-h6
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
```json
|
|
294
|
+
{ "type": "click", "text": "Sign In" }
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
</details>
|
|
298
|
+
|
|
299
|
+
<details>
|
|
300
|
+
<summary><strong>Assertions</strong> — verify text, elements, URLs, counts & network</summary>
|
|
213
301
|
|
|
214
|
-
|
|
302
|
+
<br/>
|
|
215
303
|
|
|
216
304
|
| Action | Fields | Description |
|
|
217
305
|
|--------|--------|-------------|
|
|
218
306
|
| `assert_text` | `text` | Assert text exists anywhere on the page (substring) |
|
|
307
|
+
| `assert_no_text` | `text` | Assert text does NOT appear anywhere on the page — opposite of `assert_text` |
|
|
308
|
+
| `assert_text_in` | `selector`, `text`, optional `value: "exact"` | Assert text inside a scoped container. `text` is a case-insensitive regex by default; `value: "exact"` switches to case-sensitive substring |
|
|
219
309
|
| `assert_element_text` | `selector`, `text`, optional `value: "exact"` | Assert element's text contains (or exactly matches) the expected text |
|
|
220
310
|
| `assert_url` | `value` | Assert current URL path or full URL. Paths (`/dashboard`) compare against pathname only |
|
|
221
311
|
| `assert_visible` | `selector` | Assert element exists and is visible |
|
|
@@ -226,22 +316,16 @@ Suite files can have numeric prefixes for ordering (`01-auth.json`, `02-dashboar
|
|
|
226
316
|
| `assert_matches` | `selector`, `value` (regex) | Assert element text matches a regex pattern |
|
|
227
317
|
| `assert_count` | `selector`, `value` | Assert element count: exact (`"5"`), or operators (`">3"`, `">=1"`, `"<10"`) |
|
|
228
318
|
| `assert_no_network_errors` | — | Fail if any network requests failed (e.g. `ERR_CONNECTION_REFUSED`) |
|
|
319
|
+
| `assert_storage` | `value` (`"key"` or `"key=expected"`), optional `selector: "session"` | Assert a `localStorage`/`sessionStorage` key exists or has a specific value |
|
|
320
|
+
| `assert_visual` | `value` (golden image), optional `selector`, `text` (max diff, e.g. `"0.02"`), `fullPage`, `maskRegions`, `threshold` | Visual regression: compare a screenshot against a golden reference. The first run saves the golden; later runs fail if more pixels differ than the threshold (default 2%) and write a diff image |
|
|
229
321
|
| `get_text` | `selector` | Extract element text (non-assertion, never fails). Result: `{ value: "..." }` |
|
|
230
322
|
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
When `click` uses `text` instead of `selector`, it searches across common interactive and content elements:
|
|
234
|
-
|
|
235
|
-
```
|
|
236
|
-
button, a, [role="button"], [role="tab"], [role="menuitem"], [role="option"],
|
|
237
|
-
[role="listitem"], div[class*="cursor"], span, li, td, th, label, p, h1-h6
|
|
238
|
-
```
|
|
323
|
+
</details>
|
|
239
324
|
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
```
|
|
325
|
+
<details>
|
|
326
|
+
<summary><strong>Framework-aware actions</strong> — React/MUI without <code>evaluate</code> boilerplate</summary>
|
|
243
327
|
|
|
244
|
-
|
|
328
|
+
<br/>
|
|
245
329
|
|
|
246
330
|
These actions handle common patterns in React/MUI apps that normally require verbose `evaluate` boilerplate:
|
|
247
331
|
|
|
@@ -253,6 +337,9 @@ These actions handle common patterns in React/MUI apps that normally require ver
|
|
|
253
337
|
| `select_combobox` | `text`, optional `selector`, `filter`, `openWait`/`filterWait`/`waitAfter` | Open a MUI Autocomplete/Select, optionally type `filter`, then click the option matching `text`. Falls back across `[role="option"]`, `.MuiAutocomplete-option`, `li.MuiMenuItem-root`. |
|
|
254
338
|
| `focus_autocomplete` | `text` (label text) | Focus an autocomplete input by its label text. Supports MUI and generic `[role="combobox"]`. |
|
|
255
339
|
| `click_chip` | `text` | Click a chip/tag element by text. Searches `[class*="Chip"]`, `[class*="chip"]`, `[data-chip]`. |
|
|
340
|
+
| `click_icon` | `value` (icon id), optional `selector` (scope) | Click an icon by `data-testid`/`data-icon`/`aria-label`/class fragment or SVG `<title>` — MUI, FontAwesome, Heroicons, etc. Clicks the nearest clickable ancestor (button, link, tab). |
|
|
341
|
+
| `click_menu_item` | `text`, optional `selector` (scope) | Click a menu item by text across `[role="menuitem"]`, `.dropdown-item`, `.menu-item`, MUI `MenuItem`. |
|
|
342
|
+
| `click_in_context` | `text` (container text), `selector` (child) | Click a child element inside the smallest container matching `text` — e.g. the delete button of one specific card/row. |
|
|
256
343
|
|
|
257
344
|
```json
|
|
258
345
|
// Before: 5 lines of evaluate boilerplate
|
|
@@ -262,13 +349,38 @@ These actions handle common patterns in React/MUI apps that normally require ver
|
|
|
262
349
|
{ "type": "type_react", "selector": "#search", "value": "term" }
|
|
263
350
|
```
|
|
264
351
|
|
|
265
|
-
|
|
352
|
+
</details>
|
|
353
|
+
|
|
354
|
+
<details>
|
|
355
|
+
<summary><strong>Multi-tab actions</strong> — popups, OAuth windows & cross-tab flows</summary>
|
|
356
|
+
|
|
357
|
+
<br/>
|
|
358
|
+
|
|
359
|
+
| Action | Fields | Description |
|
|
360
|
+
|--------|--------|-------------|
|
|
361
|
+
| `open_tab` | `value` (URL), optional `text` (label) | Open a new tab and navigate to the URL (relative to `baseUrl` or absolute). Label defaults to `tab-<n>` |
|
|
362
|
+
| `switch_tab` | `value` | Switch the active tab by label, numeric index, or title/URL match (regex or substring). `"default"` returns to the original tab |
|
|
363
|
+
| `wait_for_tab` | optional `text` (label), `timeout` | Wait for a new tab/popup opened by the app (`window.open`, `target="_blank"`) and make it the active tab |
|
|
364
|
+
| `assert_tab_count` | `value` | Assert the number of open tabs: exact (`"2"`) or operators (`">=2"`) |
|
|
365
|
+
| `close_tab` | optional `value` (label) | Close the current (or named) tab and switch back to the last remaining one |
|
|
366
|
+
|
|
367
|
+
All subsequent actions run in the active tab:
|
|
368
|
+
|
|
369
|
+
```json
|
|
370
|
+
{ "type": "click", "text": "Open report" }
|
|
371
|
+
{ "type": "wait_for_tab", "text": "report" }
|
|
372
|
+
{ "type": "assert_text", "text": "Quarterly results" }
|
|
373
|
+
{ "type": "close_tab" }
|
|
374
|
+
```
|
|
375
|
+
|
|
376
|
+
</details>
|
|
266
377
|
|
|
267
|
-
|
|
378
|
+
<details>
|
|
379
|
+
<summary><strong>Retries & flaky detection</strong></summary>
|
|
268
380
|
|
|
269
|
-
|
|
381
|
+
<br/>
|
|
270
382
|
|
|
271
|
-
|
|
383
|
+
**Test-level retry** — retry an entire test on failure. Set globally via config or per-test:
|
|
272
384
|
|
|
273
385
|
```json
|
|
274
386
|
{ "name": "flaky-test", "retries": 3, "timeout": 15000, "actions": [...] }
|
|
@@ -276,9 +388,7 @@ Retry an entire test on failure. Set globally via config or per-test:
|
|
|
276
388
|
|
|
277
389
|
Tests that pass after retry are flagged as **flaky** in the report and learning system.
|
|
278
390
|
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
Retry a single action without rerunning the entire test. Useful for timing-sensitive clicks and waits:
|
|
391
|
+
**Action-level retry** — retry a single action without rerunning the entire test. Useful for timing-sensitive clicks and waits:
|
|
282
392
|
|
|
283
393
|
```json
|
|
284
394
|
{ "type": "click", "selector": "#dynamic-btn", "retries": 3 }
|
|
@@ -287,9 +397,12 @@ Retry a single action without rerunning the entire test. Useful for timing-sensi
|
|
|
287
397
|
|
|
288
398
|
Set globally: `actionRetries` in config, `--action-retries <n>` CLI, or `ACTION_RETRIES` env var. Delay between retries: `actionRetryDelay` (default 500ms).
|
|
289
399
|
|
|
290
|
-
|
|
400
|
+
</details>
|
|
291
401
|
|
|
292
|
-
|
|
402
|
+
<details>
|
|
403
|
+
<summary><strong>Serial tests</strong> — for tests that share state</summary>
|
|
404
|
+
|
|
405
|
+
<br/>
|
|
293
406
|
|
|
294
407
|
Tests that share state (e.g., two tests modifying the same record) can race when running in parallel. Mark them as serial:
|
|
295
408
|
|
|
@@ -300,9 +413,12 @@ Tests that share state (e.g., two tests modifying the same record) can race when
|
|
|
300
413
|
|
|
301
414
|
Serial tests run one at a time **after** all parallel tests finish — preventing interference without slowing down independent tests.
|
|
302
415
|
|
|
303
|
-
|
|
416
|
+
</details>
|
|
417
|
+
|
|
418
|
+
<details>
|
|
419
|
+
<summary><strong>Testing authenticated apps</strong></summary>
|
|
304
420
|
|
|
305
|
-
|
|
421
|
+
<br/>
|
|
306
422
|
|
|
307
423
|
The simplest approach — log in via the UI like a real user:
|
|
308
424
|
|
|
@@ -341,9 +457,12 @@ Each test runs in a **fresh browser context**, so auth state is automatically cl
|
|
|
341
457
|
|
|
342
458
|
> **More strategies:** Cookie-based auth, HTTP header injection, OAuth/SSO bypasses, reusable auth modules, and role-based testing — see [docs/authentication.md](docs/authentication.md)
|
|
343
459
|
|
|
344
|
-
|
|
460
|
+
</details>
|
|
461
|
+
|
|
462
|
+
<details>
|
|
463
|
+
<summary><strong>Reusable modules</strong> — extract common flows with <code>$use</code></summary>
|
|
345
464
|
|
|
346
|
-
|
|
465
|
+
<br/>
|
|
347
466
|
|
|
348
467
|
Extract common flows into parameterized modules:
|
|
349
468
|
|
|
@@ -380,9 +499,35 @@ Use in tests:
|
|
|
380
499
|
|
|
381
500
|
Modules support parameter validation (required params fail fast), conditional blocks (`{{#param}}...{{/param}}`), nested composition, and cycle detection.
|
|
382
501
|
|
|
383
|
-
|
|
502
|
+
</details>
|
|
503
|
+
|
|
504
|
+
<details>
|
|
505
|
+
<summary><strong>Hooks</strong> — beforeAll / beforeEach / afterEach / afterAll</summary>
|
|
506
|
+
|
|
507
|
+
<br/>
|
|
508
|
+
|
|
509
|
+
Run actions at lifecycle points. Define globally in config or per-suite:
|
|
510
|
+
|
|
511
|
+
```json
|
|
512
|
+
{
|
|
513
|
+
"hooks": {
|
|
514
|
+
"beforeAll": [{ "type": "goto", "value": "/setup" }],
|
|
515
|
+
"beforeEach": [{ "type": "goto", "value": "/" }],
|
|
516
|
+
"afterEach": [{ "type": "screenshot", "value": "after.png" }],
|
|
517
|
+
"afterAll": []
|
|
518
|
+
},
|
|
519
|
+
"tests": [...]
|
|
520
|
+
}
|
|
521
|
+
```
|
|
522
|
+
|
|
523
|
+
> **Important:** `beforeAll` runs on a separate browser page that is closed before tests start. Use `beforeEach` for state that tests need (cookies, localStorage, auth tokens).
|
|
384
524
|
|
|
385
|
-
|
|
525
|
+
</details>
|
|
526
|
+
|
|
527
|
+
<details>
|
|
528
|
+
<summary><strong>Exclude patterns</strong> — skip drafts from <code>--all</code></summary>
|
|
529
|
+
|
|
530
|
+
<br/>
|
|
386
531
|
|
|
387
532
|
Skip exploratory or draft tests from `--all` runs:
|
|
388
533
|
|
|
@@ -395,9 +540,84 @@ export default {
|
|
|
395
540
|
|
|
396
541
|
Individual suite runs (`--suite`) are not affected by exclude patterns.
|
|
397
542
|
|
|
543
|
+
</details>
|
|
544
|
+
|
|
398
545
|
---
|
|
399
546
|
|
|
400
|
-
|
|
547
|
+
<a name="ai"></a>
|
|
548
|
+
|
|
549
|
+
## 🤖 AI integration
|
|
550
|
+
|
|
551
|
+
The whole point: your agent writes, runs, and verifies tests for you.
|
|
552
|
+
|
|
553
|
+
<details>
|
|
554
|
+
<summary><strong>Claude Code</strong> — plugin install & MCP-only install</summary>
|
|
555
|
+
|
|
556
|
+
<br/>
|
|
557
|
+
|
|
558
|
+
```bash
|
|
559
|
+
claude plugin marketplace add fastslack/mtw-e2e-runner
|
|
560
|
+
claude plugin install e2e-runner@matware
|
|
561
|
+
```
|
|
562
|
+
|
|
563
|
+
This gives Claude 17 MCP tools, a workflow skill, 4 slash commands (`/e2e-runner:run`, `/e2e-runner:create-test`, `/e2e-runner:verify-issue`, `/e2e-runner:capture`), and 3 specialized agents (test-analyzer, test-creator, test-improver).
|
|
564
|
+
|
|
565
|
+
**MCP-only install** (tools only, no skill/commands/agents):
|
|
566
|
+
|
|
567
|
+
```bash
|
|
568
|
+
claude mcp add --transport stdio --scope user e2e-runner \
|
|
569
|
+
-- npx -y -p @matware/e2e-runner e2e-runner-mcp
|
|
570
|
+
```
|
|
571
|
+
|
|
572
|
+
</details>
|
|
573
|
+
|
|
574
|
+
<details>
|
|
575
|
+
<summary><strong>OpenCode</strong></summary>
|
|
576
|
+
|
|
577
|
+
<br/>
|
|
578
|
+
|
|
579
|
+
```bash
|
|
580
|
+
cp node_modules/@matware/e2e-runner/opencode.json ./
|
|
581
|
+
mkdir -p .opencode && cp -r node_modules/@matware/e2e-runner/.opencode/* .opencode/
|
|
582
|
+
```
|
|
583
|
+
|
|
584
|
+
See [OPENCODE.md](OPENCODE.md) for details.
|
|
585
|
+
|
|
586
|
+
</details>
|
|
587
|
+
|
|
588
|
+
<details>
|
|
589
|
+
<summary><strong>The 17 MCP tools</strong></summary>
|
|
590
|
+
|
|
591
|
+
<br/>
|
|
592
|
+
|
|
593
|
+
| Tool | Description |
|
|
594
|
+
|------|-------------|
|
|
595
|
+
| `e2e_run` | Run tests (all, by suite, or by file) |
|
|
596
|
+
| `e2e_list` | List available test suites |
|
|
597
|
+
| `e2e_create_test` | Create a new test JSON file |
|
|
598
|
+
| `e2e_create_module` | Create a reusable module |
|
|
599
|
+
| `e2e_pool_status` | Check Chrome pool health |
|
|
600
|
+
| `e2e_app_pool_status` | Inspect the app environment pool (forks, ports, drivers) |
|
|
601
|
+
| `e2e_screenshot` | Retrieve a screenshot by hash |
|
|
602
|
+
| `e2e_capture` | Capture screenshot of any URL |
|
|
603
|
+
| `e2e_analyze` | Extract page structure (interactive elements, forms, headings) and emit test scaffolds |
|
|
604
|
+
| `e2e_dashboard_start` | Start web dashboard |
|
|
605
|
+
| `e2e_dashboard_stop` | Stop web dashboard |
|
|
606
|
+
| `e2e_dashboard_restart` | Restart the dashboard (new project dir/port, clear stale sessions) |
|
|
607
|
+
| `e2e_issue` | Fetch issue and generate tests |
|
|
608
|
+
| `e2e_network_logs` | Query network logs for a run |
|
|
609
|
+
| `e2e_learnings` | Query stability insights |
|
|
610
|
+
| `e2e_vars` | Manage SQLite-backed `{{var.KEY}}` project variables |
|
|
611
|
+
| `e2e_neo4j` | Manage Neo4j knowledge graph |
|
|
612
|
+
|
|
613
|
+
> Pool start/stop are CLI-only — not exposed via MCP.
|
|
614
|
+
|
|
615
|
+
</details>
|
|
616
|
+
|
|
617
|
+
<details>
|
|
618
|
+
<summary><strong>Visual verification</strong> — describe the page, AI judges it</summary>
|
|
619
|
+
|
|
620
|
+
<br/>
|
|
401
621
|
|
|
402
622
|
Describe what the page should look like — AI judges pass/fail from screenshots:
|
|
403
623
|
|
|
@@ -414,9 +634,12 @@ Describe what the page should look like — AI judges pass/fail from screenshots
|
|
|
414
634
|
|
|
415
635
|
After test actions complete, the runner auto-captures a verification screenshot. The MCP response includes the screenshot hash — Claude Code retrieves it and visually verifies against your `expect` description. No API key required.
|
|
416
636
|
|
|
417
|
-
|
|
637
|
+
</details>
|
|
638
|
+
|
|
639
|
+
<details>
|
|
640
|
+
<summary><strong>Issue-to-test</strong> — turn a bug report into a runnable test</summary>
|
|
418
641
|
|
|
419
|
-
|
|
642
|
+
<br/>
|
|
420
643
|
|
|
421
644
|
Turn GitHub and GitLab issues into executable E2E tests. Paste an issue URL and get runnable tests — automatically.
|
|
422
645
|
|
|
@@ -445,13 +668,68 @@ In Claude Code, just ask:
|
|
|
445
668
|
|
|
446
669
|
**Auth:** GitHub requires `gh` CLI, GitLab requires `glab` CLI. Self-hosted GitLab is supported.
|
|
447
670
|
|
|
671
|
+
</details>
|
|
672
|
+
|
|
448
673
|
---
|
|
449
674
|
|
|
450
|
-
|
|
675
|
+
<a name="dashboard"></a>
|
|
676
|
+
|
|
677
|
+
## 📊 Dashboard & insights
|
|
678
|
+
|
|
679
|
+
```bash
|
|
680
|
+
e2e-runner dashboard # Start on default port 8484
|
|
681
|
+
e2e-runner dashboard --port 9090 # Custom port
|
|
682
|
+
```
|
|
451
683
|
|
|
452
|
-
|
|
684
|
+
<details>
|
|
685
|
+
<summary><strong>Web dashboard tour</strong> — live view, history, gallery, pool status</summary>
|
|
686
|
+
|
|
687
|
+
<br/>
|
|
453
688
|
|
|
454
|
-
|
|
689
|
+
**Live execution** — monitor tests in real-time with step-by-step progress, durations, and active worker count.
|
|
690
|
+
|
|
691
|
+
<p align="center">
|
|
692
|
+
<img src="https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/docs/screenshots/blog-dashboard-live-running.png" alt="Dashboard - Live test execution" width="800" />
|
|
693
|
+
</p>
|
|
694
|
+
|
|
695
|
+
**Test suites** — browse all suites across projects. Run a single suite or all tests with one click.
|
|
696
|
+
|
|
697
|
+
<p align="center">
|
|
698
|
+
<img src="https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/docs/screenshots/blog-dashboard-suites.png" alt="Dashboard - Test suites grid" width="800" />
|
|
699
|
+
</p>
|
|
700
|
+
|
|
701
|
+
**Run history** — track pass-rate trends with the built-in chart. Click any row to expand full detail.
|
|
702
|
+
|
|
703
|
+
<p align="center">
|
|
704
|
+
<img src="https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/docs/screenshots/blog-dashboard-runs.png" alt="Dashboard - Run history" width="800" />
|
|
705
|
+
</p>
|
|
706
|
+
|
|
707
|
+
**Run detail** — PASS/FAIL badges, screenshot thumbnails with copyable hashes (`ss:77c28b5a`), formatted console errors, and network request logs.
|
|
708
|
+
|
|
709
|
+
<p align="center">
|
|
710
|
+
<img src="https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/docs/screenshots/blog-dashboard-run-detail.png" alt="Dashboard - Run detail" width="800" />
|
|
711
|
+
</p>
|
|
712
|
+
|
|
713
|
+
**Screenshot gallery** — browse all captured screenshots with hash search (action, error, and verification captures).
|
|
714
|
+
|
|
715
|
+
<p align="center">
|
|
716
|
+
<img src="https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/docs/screenshots/blog-dashboard-screenshots-gallery.png" alt="Dashboard - Screenshot gallery" width="800" />
|
|
717
|
+
</p>
|
|
718
|
+
|
|
719
|
+
**Pool status** — Chrome pool health: available slots, running sessions, memory pressure.
|
|
720
|
+
|
|
721
|
+
<p align="center">
|
|
722
|
+
<img src="https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/docs/screenshots/blog-dashboard-pool-status.png" alt="Dashboard - Pool status" width="800" />
|
|
723
|
+
</p>
|
|
724
|
+
|
|
725
|
+
</details>
|
|
726
|
+
|
|
727
|
+
<details>
|
|
728
|
+
<summary><strong>Learning system</strong> — flaky tests, unstable selectors, slow APIs</summary>
|
|
729
|
+
|
|
730
|
+
<br/>
|
|
731
|
+
|
|
732
|
+
The runner learns from every test run — building knowledge about your test suite over time. Query insights via the `e2e_learnings` MCP tool:
|
|
455
733
|
|
|
456
734
|
| Query | Returns |
|
|
457
735
|
|-------|---------|
|
|
@@ -466,75 +744,75 @@ Query insights via the `e2e_learnings` MCP tool:
|
|
|
466
744
|
| `page:<path>` | Drill-down history for a specific page |
|
|
467
745
|
| `selector:<value>` | Drill-down history for a specific selector |
|
|
468
746
|
|
|
469
|
-
**Storage & export:**
|
|
747
|
+
**Storage & export:**
|
|
470
748
|
- SQLite (`~/.e2e-runner/dashboard.db`) — default, zero setup
|
|
471
749
|
- Neo4j knowledge graph — optional, for relationship-based analysis. Manage via `e2e_neo4j` MCP tool or `docker compose`
|
|
472
750
|
- Markdown report (`e2e/learnings.md`) — auto-generated after each run
|
|
473
751
|
|
|
474
752
|
**Test narration:** Each test run generates a human-readable narrative of what happened step by step, visible in the CLI output and the dashboard.
|
|
475
753
|
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
## Web Dashboard
|
|
479
|
-
|
|
480
|
-
Real-time UI for running tests, viewing results, screenshots, and network logs.
|
|
754
|
+
</details>
|
|
481
755
|
|
|
482
|
-
|
|
483
|
-
|
|
484
|
-
e2e-runner dashboard --port 9090 # Custom port
|
|
485
|
-
```
|
|
756
|
+
<details>
|
|
757
|
+
<summary><strong>Network error handling</strong> — assertions, global flag, full logging</summary>
|
|
486
758
|
|
|
487
|
-
|
|
759
|
+
<br/>
|
|
488
760
|
|
|
489
|
-
|
|
761
|
+
**Explicit assertion** — place `assert_no_network_errors` after critical page loads:
|
|
490
762
|
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
763
|
+
```json
|
|
764
|
+
{ "type": "goto", "value": "/dashboard" },
|
|
765
|
+
{ "type": "wait", "selector": ".loaded" },
|
|
766
|
+
{ "type": "assert_no_network_errors" }
|
|
767
|
+
```
|
|
494
768
|
|
|
495
|
-
|
|
769
|
+
**Global flag** — set `failOnNetworkError: true` to automatically fail any test with network errors:
|
|
496
770
|
|
|
497
|
-
|
|
771
|
+
```bash
|
|
772
|
+
e2e-runner run --all --fail-on-network-error
|
|
773
|
+
```
|
|
498
774
|
|
|
499
|
-
|
|
500
|
-
<img src="https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/docs/screenshots/blog-dashboard-suites.png" alt="Dashboard - Test suites grid" width="800" />
|
|
501
|
-
</p>
|
|
775
|
+
When disabled (default), the runner still collects and reports network errors — the MCP response includes a warning when tests pass but have network errors.
|
|
502
776
|
|
|
503
|
-
|
|
777
|
+
**Full network logging** — all XHR/fetch requests are captured with URL, method, status, duration, request/response headers, and response body (truncated at 50KB). Viewable in the dashboard with expandable request detail rows.
|
|
504
778
|
|
|
505
|
-
|
|
779
|
+
MCP drill-down flow:
|
|
506
780
|
|
|
507
|
-
|
|
508
|
-
|
|
509
|
-
|
|
781
|
+
```
|
|
782
|
+
1. e2e_run → compact networkSummary + runDbId
|
|
783
|
+
2. e2e_network_logs(runDbId) → all requests (url, method, status, duration)
|
|
784
|
+
3. e2e_network_logs(runDbId, errorsOnly: true) → only failed requests
|
|
785
|
+
4. e2e_network_logs(runDbId, includeHeaders: true) → with headers
|
|
786
|
+
5. e2e_network_logs(runDbId, includeBodies: true) → full request/response bodies
|
|
787
|
+
```
|
|
510
788
|
|
|
511
|
-
|
|
789
|
+
The `e2e_run` response stays compact (~5KB) regardless of how many requests were captured. Use `e2e_network_logs` with the returned `runDbId` to drill into details on demand.
|
|
512
790
|
|
|
513
|
-
|
|
791
|
+
</details>
|
|
514
792
|
|
|
515
|
-
<
|
|
516
|
-
|
|
517
|
-
</p>
|
|
793
|
+
<details>
|
|
794
|
+
<summary><strong>Screenshot capture</strong> — snapshot any URL on demand</summary>
|
|
518
795
|
|
|
519
|
-
|
|
796
|
+
<br/>
|
|
520
797
|
|
|
521
|
-
|
|
798
|
+
Capture screenshots of any URL on demand — no test suite required:
|
|
522
799
|
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
800
|
+
```bash
|
|
801
|
+
e2e-runner capture https://example.com
|
|
802
|
+
e2e-runner capture https://example.com --full-page --selector ".loaded" --delay 2000
|
|
803
|
+
```
|
|
526
804
|
|
|
527
|
-
|
|
805
|
+
Via MCP, the `e2e_capture` tool supports `authToken` and `authStorageKey` for authenticated pages — it injects the token into localStorage before navigating.
|
|
528
806
|
|
|
529
|
-
|
|
807
|
+
Every screenshot gets a deterministic hash (`ss:a3f2b1c9`). Use `e2e_screenshot` to retrieve any screenshot by hash — it returns the image with metadata (test name, step, type).
|
|
530
808
|
|
|
531
|
-
|
|
532
|
-
<img src="https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/docs/screenshots/blog-dashboard-pool-status.png" alt="Dashboard - Pool status" width="800" />
|
|
533
|
-
</p>
|
|
809
|
+
</details>
|
|
534
810
|
|
|
535
811
|
---
|
|
536
812
|
|
|
537
|
-
|
|
813
|
+
<a name="drivers"></a>
|
|
814
|
+
|
|
815
|
+
## 🌐 Browser drivers
|
|
538
816
|
|
|
539
817
|
The runner can talk to multiple browser engines through different drivers. The default is **`auto`** — it probes each pool URL and picks the right driver per pool.
|
|
540
818
|
|
|
@@ -546,7 +824,10 @@ The runner can talk to multiple browser engines through different drivers. The d
|
|
|
546
824
|
| `obscura` | [Obscura](https://github.com/h4ckf0r0day/obscura) (Rust + V8) | `/json/version` Browser=obscura | ~30 MB RAM footprint, built-in anti-detection (`--stealth`), stays close to real Chrome via Puppeteer |
|
|
547
825
|
| `steel` | [Steel Browser](https://steel.dev) | `/v1/sessions` returns JSON | Managed session lifecycle, REST API for orchestration |
|
|
548
826
|
|
|
549
|
-
|
|
827
|
+
<details>
|
|
828
|
+
<summary><strong>Pick a driver per test / force one per run</strong></summary>
|
|
829
|
+
|
|
830
|
+
<br/>
|
|
550
831
|
|
|
551
832
|
```json
|
|
552
833
|
{
|
|
@@ -568,16 +849,19 @@ The runner can talk to multiple browser engines through different drivers. The d
|
|
|
568
849
|
|
|
569
850
|
`driver` is optional. If set, only pools whose detected driver matches become candidates. `fallbackDriver` is **explicit opt-in** — without it, a missing target driver fails the test with a clear message. Pool busyness does **not** trigger fallback; the runner waits inside the filtered set.
|
|
570
851
|
|
|
571
|
-
|
|
852
|
+
Force a driver for a whole run (CLI overrides win over per-test fields — useful for A/B benchmarks):
|
|
572
853
|
|
|
573
854
|
```bash
|
|
574
855
|
e2e-runner run --all --driver obscura
|
|
575
856
|
e2e-runner run --all --driver obscura --fallback-driver cdp
|
|
576
857
|
```
|
|
577
858
|
|
|
578
|
-
|
|
859
|
+
</details>
|
|
860
|
+
|
|
861
|
+
<details>
|
|
862
|
+
<summary><strong>Running each driver locally</strong></summary>
|
|
579
863
|
|
|
580
|
-
|
|
864
|
+
<br/>
|
|
581
865
|
|
|
582
866
|
```bash
|
|
583
867
|
# browserless (default) — managed by `pool start`
|
|
@@ -593,137 +877,18 @@ tar xzf obscura-x86_64-linux.tar.gz
|
|
|
593
877
|
# then point the runner at it: poolUrls: ['http://localhost:9222'], poolDriver: 'obscura'
|
|
594
878
|
```
|
|
595
879
|
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
## Screenshot Capture
|
|
599
|
-
|
|
600
|
-
Capture screenshots of any URL on demand — no test suite required:
|
|
601
|
-
|
|
602
|
-
```bash
|
|
603
|
-
e2e-runner capture https://example.com
|
|
604
|
-
e2e-runner capture https://example.com --full-page --selector ".loaded" --delay 2000
|
|
605
|
-
```
|
|
606
|
-
|
|
607
|
-
Via MCP, the `e2e_capture` tool supports `authToken` and `authStorageKey` for authenticated pages — it injects the token into localStorage before navigating.
|
|
608
|
-
|
|
609
|
-
Every screenshot gets a deterministic hash (`ss:a3f2b1c9`). Use `e2e_screenshot` to retrieve any screenshot by hash — it returns the image with metadata (test name, step, type).
|
|
610
|
-
|
|
611
|
-
---
|
|
612
|
-
|
|
613
|
-
## AI Integration
|
|
614
|
-
|
|
615
|
-
### Claude Code
|
|
616
|
-
|
|
617
|
-
```bash
|
|
618
|
-
claude plugin marketplace add fastslack/mtw-e2e-runner
|
|
619
|
-
claude plugin install e2e-runner@matware
|
|
620
|
-
```
|
|
621
|
-
|
|
622
|
-
This gives Claude 17 MCP tools, a workflow skill, 4 slash commands (`/e2e-runner:run`, `/e2e-runner:create-test`, `/e2e-runner:verify-issue`, `/e2e-runner:capture`), and 3 specialized agents (test-analyzer, test-creator, test-improver).
|
|
623
|
-
|
|
624
|
-
**MCP-only install** (tools only, no skill/commands/agents):
|
|
625
|
-
|
|
626
|
-
```bash
|
|
627
|
-
claude mcp add --transport stdio --scope user e2e-runner \
|
|
628
|
-
-- npx -y -p @matware/e2e-runner e2e-runner-mcp
|
|
629
|
-
```
|
|
630
|
-
|
|
631
|
-
### OpenCode
|
|
632
|
-
|
|
633
|
-
```bash
|
|
634
|
-
cp node_modules/@matware/e2e-runner/opencode.json ./
|
|
635
|
-
mkdir -p .opencode && cp -r node_modules/@matware/e2e-runner/.opencode/* .opencode/
|
|
636
|
-
```
|
|
637
|
-
|
|
638
|
-
See [OPENCODE.md](OPENCODE.md) for details.
|
|
639
|
-
|
|
640
|
-
### MCP Tools
|
|
641
|
-
|
|
642
|
-
| Tool | Description |
|
|
643
|
-
|------|-------------|
|
|
644
|
-
| `e2e_run` | Run tests (all, by suite, or by file) |
|
|
645
|
-
| `e2e_list` | List available test suites |
|
|
646
|
-
| `e2e_create_test` | Create a new test JSON file |
|
|
647
|
-
| `e2e_create_module` | Create a reusable module |
|
|
648
|
-
| `e2e_pool_status` | Check Chrome pool health |
|
|
649
|
-
| `e2e_app_pool_status` | Inspect the app environment pool (forks, ports, drivers) |
|
|
650
|
-
| `e2e_screenshot` | Retrieve a screenshot by hash |
|
|
651
|
-
| `e2e_capture` | Capture screenshot of any URL |
|
|
652
|
-
| `e2e_analyze` | Extract page structure (interactive elements, forms, headings) and emit test scaffolds |
|
|
653
|
-
| `e2e_dashboard_start` | Start web dashboard |
|
|
654
|
-
| `e2e_dashboard_stop` | Stop web dashboard |
|
|
655
|
-
| `e2e_dashboard_restart` | Restart the dashboard (new project dir/port, clear stale sessions) |
|
|
656
|
-
| `e2e_issue` | Fetch issue and generate tests |
|
|
657
|
-
| `e2e_network_logs` | Query network logs for a run |
|
|
658
|
-
| `e2e_learnings` | Query stability insights |
|
|
659
|
-
| `e2e_vars` | Manage SQLite-backed `{{var.KEY}}` project variables |
|
|
660
|
-
| `e2e_neo4j` | Manage Neo4j knowledge graph |
|
|
661
|
-
|
|
662
|
-
> Pool start/stop are CLI-only — not exposed via MCP.
|
|
663
|
-
|
|
664
|
-
---
|
|
665
|
-
|
|
666
|
-
## Network Error Handling
|
|
667
|
-
|
|
668
|
-
### Explicit Assertion
|
|
669
|
-
|
|
670
|
-
Place `assert_no_network_errors` after critical page loads:
|
|
671
|
-
|
|
672
|
-
```json
|
|
673
|
-
{ "type": "goto", "value": "/dashboard" },
|
|
674
|
-
{ "type": "wait", "selector": ".loaded" },
|
|
675
|
-
{ "type": "assert_no_network_errors" }
|
|
676
|
-
```
|
|
677
|
-
|
|
678
|
-
### Global Flag
|
|
679
|
-
|
|
680
|
-
Set `failOnNetworkError: true` to automatically fail any test with network errors:
|
|
681
|
-
|
|
682
|
-
```bash
|
|
683
|
-
e2e-runner run --all --fail-on-network-error
|
|
684
|
-
```
|
|
685
|
-
|
|
686
|
-
When disabled (default), the runner still collects and reports network errors — the MCP response includes a warning when tests pass but have network errors.
|
|
687
|
-
|
|
688
|
-
### Full Network Logging
|
|
689
|
-
|
|
690
|
-
All XHR/fetch requests are captured with: URL, method, status, duration, request/response headers, and response body (truncated at 50KB). Viewable in the dashboard with expandable request detail rows.
|
|
691
|
-
|
|
692
|
-
**MCP drill-down flow:**
|
|
693
|
-
|
|
694
|
-
```
|
|
695
|
-
1. e2e_run → compact networkSummary + runDbId
|
|
696
|
-
2. e2e_network_logs(runDbId) → all requests (url, method, status, duration)
|
|
697
|
-
3. e2e_network_logs(runDbId, errorsOnly: true) → only failed requests
|
|
698
|
-
4. e2e_network_logs(runDbId, includeHeaders: true) → with headers
|
|
699
|
-
5. e2e_network_logs(runDbId, includeBodies: true) → full request/response bodies
|
|
700
|
-
```
|
|
701
|
-
|
|
702
|
-
The `e2e_run` response stays compact (~5KB) regardless of how many requests were captured. Use `e2e_network_logs` with the returned `runDbId` to drill into details on demand.
|
|
880
|
+
</details>
|
|
703
881
|
|
|
704
882
|
---
|
|
705
883
|
|
|
706
|
-
|
|
707
|
-
|
|
708
|
-
Run actions at lifecycle points. Define globally in config or per-suite:
|
|
709
|
-
|
|
710
|
-
```json
|
|
711
|
-
{
|
|
712
|
-
"hooks": {
|
|
713
|
-
"beforeAll": [{ "type": "goto", "value": "/setup" }],
|
|
714
|
-
"beforeEach": [{ "type": "goto", "value": "/" }],
|
|
715
|
-
"afterEach": [{ "type": "screenshot", "value": "after.png" }],
|
|
716
|
-
"afterAll": []
|
|
717
|
-
},
|
|
718
|
-
"tests": [...]
|
|
719
|
-
}
|
|
720
|
-
```
|
|
884
|
+
<a name="reference"></a>
|
|
721
885
|
|
|
722
|
-
|
|
886
|
+
## ⚙️ CLI, config & CI
|
|
723
887
|
|
|
724
|
-
|
|
888
|
+
<details>
|
|
889
|
+
<summary><strong>CLI commands</strong></summary>
|
|
725
890
|
|
|
726
|
-
|
|
891
|
+
<br/>
|
|
727
892
|
|
|
728
893
|
```bash
|
|
729
894
|
# Run tests
|
|
@@ -751,7 +916,12 @@ e2e-runner capture <url> # On-demand screenshot
|
|
|
751
916
|
e2e-runner init # Scaffold project
|
|
752
917
|
```
|
|
753
918
|
|
|
754
|
-
|
|
919
|
+
</details>
|
|
920
|
+
|
|
921
|
+
<details>
|
|
922
|
+
<summary><strong>CLI options</strong></summary>
|
|
923
|
+
|
|
924
|
+
<br/>
|
|
755
925
|
|
|
756
926
|
| Flag | Default | Description |
|
|
757
927
|
|------|---------|-------------|
|
|
@@ -769,9 +939,12 @@ e2e-runner init # Scaffold project
|
|
|
769
939
|
| `--driver <name>` | _(per-test)_ | Force pool driver for the run: `browserless`, `cdp`, `lightpanda`, `obscura`, `steel` |
|
|
770
940
|
| `--fallback-driver <name>` | _none_ | Explicit fallback if no pool with `--driver` is reachable |
|
|
771
941
|
|
|
772
|
-
|
|
942
|
+
</details>
|
|
943
|
+
|
|
944
|
+
<details>
|
|
945
|
+
<summary><strong>Configuration</strong> — <code>e2e.config.js</code> & priority</summary>
|
|
773
946
|
|
|
774
|
-
|
|
947
|
+
<br/>
|
|
775
948
|
|
|
776
949
|
Create `e2e.config.js` in your project root:
|
|
777
950
|
|
|
@@ -797,7 +970,7 @@ export default {
|
|
|
797
970
|
};
|
|
798
971
|
```
|
|
799
972
|
|
|
800
|
-
|
|
973
|
+
**Config priority (highest wins):**
|
|
801
974
|
|
|
802
975
|
1. CLI flags
|
|
803
976
|
2. Environment variables
|
|
@@ -806,18 +979,17 @@ export default {
|
|
|
806
979
|
|
|
807
980
|
When `--env <name>` is set, the matching profile overrides everything.
|
|
808
981
|
|
|
809
|
-
|
|
982
|
+
</details>
|
|
810
983
|
|
|
811
|
-
|
|
984
|
+
<details>
|
|
985
|
+
<summary><strong>CI/CD</strong> — JUnit XML & GitHub Actions</summary>
|
|
812
986
|
|
|
813
|
-
|
|
987
|
+
<br/>
|
|
814
988
|
|
|
815
989
|
```bash
|
|
816
990
|
e2e-runner run --all --output junit
|
|
817
991
|
```
|
|
818
992
|
|
|
819
|
-
### GitHub Actions
|
|
820
|
-
|
|
821
993
|
```yaml
|
|
822
994
|
jobs:
|
|
823
995
|
e2e:
|
|
@@ -836,9 +1008,12 @@ jobs:
|
|
|
836
1008
|
report_paths: e2e/screenshots/junit.xml
|
|
837
1009
|
```
|
|
838
1010
|
|
|
839
|
-
|
|
1011
|
+
</details>
|
|
840
1012
|
|
|
841
|
-
|
|
1013
|
+
<details>
|
|
1014
|
+
<summary><strong>Programmatic API</strong></summary>
|
|
1015
|
+
|
|
1016
|
+
<br/>
|
|
842
1017
|
|
|
843
1018
|
```js
|
|
844
1019
|
import { createRunner } from '@matware/e2e-runner';
|
|
@@ -853,15 +1028,17 @@ const report = await runner.runTests([
|
|
|
853
1028
|
]);
|
|
854
1029
|
```
|
|
855
1030
|
|
|
1031
|
+
</details>
|
|
1032
|
+
|
|
856
1033
|
---
|
|
857
1034
|
|
|
858
1035
|
## Requirements
|
|
859
1036
|
|
|
860
1037
|
- **Node.js** >= 20
|
|
861
|
-
- **Docker**
|
|
1038
|
+
- **Docker** — only for [Option 3](#install) (the parallel Chrome pool). Options 1 & 2 don't need it.
|
|
862
1039
|
|
|
863
1040
|
## License
|
|
864
1041
|
|
|
865
|
-
Copyright
|
|
1042
|
+
Copyright 2026 Matias Aguirre (fastslack) — Matware
|
|
866
1043
|
|
|
867
1044
|
Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for details.
|