@matware/e2e-runner 1.3.1 → 1.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (47) hide show
  1. package/.claude-plugin/marketplace.json +4 -4
  2. package/.claude-plugin/plugin.json +2 -2
  3. package/README.md +110 -21
  4. package/agents/test-creator.md +4 -2
  5. package/agents/test-improver.md +5 -3
  6. package/bin/cli.js +80 -17
  7. package/package.json +3 -2
  8. package/skills/e2e-testing/SKILL.md +3 -2
  9. package/skills/e2e-testing/references/action-types.md +22 -4
  10. package/skills/e2e-testing/references/test-json-format.md +23 -0
  11. package/src/actions.js +170 -14
  12. package/src/config.js +6 -0
  13. package/src/dashboard.js +135 -4
  14. package/src/db.js +11 -0
  15. package/src/mcp-tools.js +8 -2
  16. package/src/module-analysis.js +247 -0
  17. package/src/module-resolver.js +35 -2
  18. package/src/narrate.js +14 -1
  19. package/src/pool-manager.js +46 -1
  20. package/src/pool.js +177 -20
  21. package/src/runner.js +77 -10
  22. package/src/visual-diff.js +69 -0
  23. package/src/websocket.js +14 -3
  24. package/src/wizard.js +184 -0
  25. package/templates/build-dashboard.js +3 -0
  26. package/templates/dashboard/js/api.js +60 -3
  27. package/templates/dashboard/js/init.js +46 -0
  28. package/templates/dashboard/js/keyboard.js +8 -7
  29. package/templates/dashboard/js/quicksearch.js +277 -0
  30. package/templates/dashboard/js/state.js +61 -7
  31. package/templates/dashboard/js/toast.js +1 -1
  32. package/templates/dashboard/js/view-live.js +235 -42
  33. package/templates/dashboard/js/view-runs.js +379 -37
  34. package/templates/dashboard/js/view-tests.js +157 -16
  35. package/templates/dashboard/js/view-tools.js +234 -0
  36. package/templates/dashboard/js/view-watch.js +2 -2
  37. package/templates/dashboard/js/websocket.js +33 -3
  38. package/templates/dashboard/styles/base.css +489 -53
  39. package/templates/dashboard/styles/components.css +719 -84
  40. package/templates/dashboard/styles/view-live.css +459 -78
  41. package/templates/dashboard/styles/view-runs.css +779 -177
  42. package/templates/dashboard/styles/view-tests.css +440 -77
  43. package/templates/dashboard/styles/view-tools.css +206 -0
  44. package/templates/dashboard/styles/view-watch.css +198 -41
  45. package/templates/dashboard/template.html +354 -56
  46. package/templates/dashboard.html +5173 -711
  47. package/templates/docker-compose-lightpanda.yml +7 -0
@@ -6,7 +6,7 @@
6
6
  },
7
7
  "metadata": {
8
8
  "description": "AI-powered E2E testing tools by Matware — JSON-driven browser tests, Chrome pool, visual verification, and stability learning",
9
- "version": "1.3.0"
9
+ "version": "1.5.0"
10
10
  },
11
11
  "plugins": [
12
12
  {
@@ -14,10 +14,10 @@
14
14
  "source": {
15
15
  "source": "npm",
16
16
  "package": "@matware/e2e-runner",
17
- "version": "^1.3.0"
17
+ "version": "^1.5.0"
18
18
  },
19
- "description": "JSON-driven E2E browser test runner — no JavaScript test files needed. Parallel execution against a Chrome pool, 28+ built-in actions, visual verification, network debugging, flaky test detection, reusable modules, and a real-time dashboard. Includes 3 specialized agents (test-creator, test-analyzer, test-improver), 4 slash commands, and 16 MCP tools.",
20
- "version": "1.3.0",
19
+ "description": "JSON-driven E2E browser test runner — no JavaScript test files needed. Parallel execution against a Chrome pool (browserless, CDP, Lightpanda, Obscura, Steel), 28+ built-in actions, visual verification, network debugging, flaky test detection, reusable modules, and a real-time dashboard. Includes 3 specialized agents (test-creator, test-analyzer, test-improver), 4 slash commands, and 17 MCP tools.",
20
+ "version": "1.5.0",
21
21
  "author": { "name": "Matware" },
22
22
  "homepage": "https://www.npmjs.com/package/@matware/e2e-runner",
23
23
  "repository": "https://github.com/fastslack/mtw-e2e-runner",
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "e2e-runner",
3
- "version": "1.3.0",
4
- "description": "JSON-driven E2E browser test runner — no JavaScript test files needed. Parallel execution against a Chrome pool, 28+ built-in actions, visual verification, network debugging, flaky test detection, reusable modules, and a real-time dashboard. Includes 3 specialized agents, 4 slash commands, and 16 MCP tools.",
3
+ "version": "1.5.0",
4
+ "description": "JSON-driven E2E browser test runner — no JavaScript test files needed. Parallel execution against a Chrome pool (browserless, CDP, Lightpanda, Obscura, Steel), 28+ built-in actions, visual verification, network debugging, flaky test detection, reusable modules, and a real-time dashboard. Includes 3 specialized agents, 4 slash commands, and 17 MCP tools.",
5
5
  "author": { "name": "Matware" },
6
6
  "repository": "https://github.com/fastslack/mtw-e2e-runner",
7
7
  "homepage": "https://www.npmjs.com/package/@matware/e2e-runner",
package/README.md CHANGED
@@ -31,7 +31,7 @@
31
31
 
32
32
  But what makes it truly different is its **deep AI integration**. With a built-in [MCP server](https://modelcontextprotocol.io/), Claude Code can create tests from a conversation, run them, read the results, capture screenshots, and even visually verify that pages look correct — all without leaving the chat. Paste a GitHub issue URL and get a runnable test back. That's the workflow.
33
33
 
34
- ### This is a test
34
+ ### A test is just JSON
35
35
 
36
36
  ```json
37
37
  [
@@ -49,7 +49,7 @@ But what makes it truly different is its **deep AI integration**. With a built-i
49
49
  ]
50
50
  ```
51
51
 
52
- No imports. No `describe`/`it`. No compilation step. Just a JSON file that describes what a user does — and the runner makes it happen.
52
+ You describe what a user does click this, type that, check the page says X — and the runner does it in a real browser. No imports, no `describe`/`it`, no build step. If you can read it, you can write it.
53
53
 
54
54
  ---
55
55
 
@@ -69,32 +69,51 @@ This gives your agent the knowledge to create, run, and debug JSON-driven E2E te
69
69
 
70
70
  ## Getting Started
71
71
 
72
- **Prerequisites:** Node.js >= 20, Docker running, your app on a known port.
72
+ You need just two things: **Node.js 20+** and **Docker running**. You don't install any browser — the runner spins up Chrome in a container for you.
73
73
 
74
- ### Quickstart
74
+ ### Try it in 60 seconds
75
75
 
76
76
  ```bash
77
77
  npm install --save-dev @matware/e2e-runner
78
- npx e2e-runner init # creates e2e/tests/ with a sample test
79
- npx e2e-runner pool start # starts Chrome in Docker
80
- npx e2e-runner run --all # runs the sample test
78
+ npx e2e-runner init # scaffolds e2e/ with a sample test + config
79
+ npx e2e-runner run --all # runs it — Chrome starts automatically on first run
81
80
  ```
82
81
 
83
- Or do it all in one command:
82
+ That's the whole setup. No separate `pool start`, no browser download: the first run boots the Chrome pool for you and reuses it afterwards.
84
83
 
85
- ```bash
86
- curl -fsSL https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/scripts/quickstart.sh | bash
87
- ```
84
+ > Prefer a single command? `curl -fsSL https://raw.githubusercontent.com/fastslack/mtw-e2e-runner/main/scripts/quickstart.sh | bash`
85
+
86
+ ### Point it at your app
88
87
 
89
- After setup, edit `e2e.config.js` to set your app's port:
88
+ `init` created `e2e.config.js`. Set your app's URL there:
90
89
 
91
90
  ```js
92
91
  export default {
93
- baseUrl: 'http://host.docker.internal:3000', // change 3000 to your port
92
+ baseUrl: 'http://host.docker.internal:3000', // change 3000 to your app's port
94
93
  };
95
94
  ```
96
95
 
97
- > **Why `host.docker.internal`?** Chrome runs inside Docker and can't reach `localhost` on your machine. This hostname bridges the gap. On Linux (Docker Engine, not Desktop), you may need `--add-host=host.docker.internal:host-gateway` or use your LAN IP directly.
96
+ <details>
97
+ <summary><strong>Why <code>host.docker.internal</code> instead of <code>localhost</code>?</strong></summary>
98
+
99
+ Chrome runs inside Docker, so `localhost` there points at the container, not your machine. `host.docker.internal` bridges to your host. On Linux (Docker Engine, not Docker Desktop) you may need to add `--add-host=host.docker.internal:host-gateway`, or just use your machine's LAN IP.
100
+ </details>
101
+
102
+ ### Write your first test
103
+
104
+ Open `e2e/tests/sample.json` and describe a flow as a list of actions:
105
+
106
+ ```json
107
+ [
108
+ { "name": "homepage loads", "actions": [
109
+ { "type": "goto", "value": "/" },
110
+ { "type": "assert_text", "text": "Welcome" },
111
+ { "type": "screenshot", "value": "home.png" }
112
+ ]}
113
+ ]
114
+ ```
115
+
116
+ Then `npx e2e-runner run --all` again. Pass/fail, timing, screenshots, and network errors print to your terminal — and to the [web dashboard](#web-dashboard) if it's open.
98
117
 
99
118
  ### Add Claude Code (optional)
100
119
 
@@ -103,7 +122,7 @@ claude plugin marketplace add fastslack/mtw-e2e-runner
103
122
  claude plugin install e2e-runner@matware
104
123
  ```
105
124
 
106
- This gives Claude 13 MCP tools, slash commands, and specialized agents. Just say *"Run all E2E tests"* or *"Create a test for the login flow"*.
125
+ This gives Claude 17 MCP tools, slash commands, and specialized agents. Just say *"Run all E2E tests"* or *"Create a test for the login flow"*.
107
126
 
108
127
  ### Add OpenCode (optional)
109
128
 
@@ -128,7 +147,7 @@ See [OPENCODE.md](OPENCODE.md) for details.
128
147
 
129
148
  🧪 **Zero-code tests** — JSON files that anyone on your team can read and write. No JavaScript, no compilation, no framework lock-in.
130
149
 
131
- 🤖 **AI-powered testing** — Claude Code creates, executes, and debugs tests natively through 13 MCP tools. Ask it to "test the checkout flow" and it builds the JSON, runs it, and reports back.
150
+ 🤖 **AI-powered testing** — Claude Code creates, executes, and debugs tests natively through 17 MCP tools. Ask it to "test the checkout flow" and it builds the JSON, runs it, and reports back.
132
151
 
133
152
  🐛 **Issue-to-Test pipeline** — Paste a GitHub or GitLab issue URL. The runner fetches it, generates E2E tests, runs them, and tells you: *bug confirmed* or *not reproducible*.
134
153
 
@@ -136,7 +155,9 @@ See [OPENCODE.md](OPENCODE.md) for details.
136
155
 
137
156
  🧠 **Learning system** — Tracks test stability across runs. Detects flaky tests, unstable selectors, slow APIs, and error patterns — then surfaces actionable insights.
138
157
 
139
- ⚡ **Parallel execution** — Run N tests simultaneously against a shared Chrome pool (browserless/chrome). Serial mode available for tests that share state.
158
+ ⚡ **Parallel execution** — Run N tests simultaneously against a shared browser pool (browserless, raw CDP, Lightpanda, Obscura, or Steel). Serial mode available for tests that share state.
159
+
160
+ 🎯 **Pluggable browser drivers** — Pick the engine that fits each test: real Chrome via browserless, Lightpanda or Obscura for fast lightweight runs, Steel for managed sessions. Set `driver` per test or override the whole run with `--driver`.
140
161
 
141
162
  📊 **Real-time dashboard** — Live execution view, run history with pass-rate charts, screenshot gallery with hash-based search, expandable network request logs.
142
163
 
@@ -177,9 +198,9 @@ Suite files can have numeric prefixes for ordering (`01-auth.json`, `02-dashboar
177
198
  | Action | Fields | Description |
178
199
  |--------|--------|-------------|
179
200
  | `goto` | `value` | Navigate to URL (relative to `baseUrl` or absolute) |
180
- | `click` | `selector` or `text` | Click by CSS selector or visible text content |
201
+ | `click` | `selector` or `text` | Click by CSS selector or visible text content. Text mode also takes `scope: "dialog"`, `visible: true`, `last: true` |
181
202
  | `type` / `fill` | `selector`, `value` | Clear field and type text |
182
- | `wait` | `selector`, `text`, or `value` (ms) | Wait for element, text, or fixed delay |
203
+ | `wait` | `selector`, `text`, `gone`, or `value` (ms) | Wait for element/text to appear, for `gone` to disappear (spinner/dialog), or fixed delay. Prefer conditions over fixed `value` sleeps |
183
204
  | `screenshot` | `value` (filename) | Capture a screenshot |
184
205
  | `select` | `selector`, `value` | Select a dropdown option |
185
206
  | `clear` | `selector` | Clear an input field |
@@ -226,9 +247,10 @@ These actions handle common patterns in React/MUI apps that normally require ver
226
247
 
227
248
  | Action | Fields | Description |
228
249
  |--------|--------|-------------|
229
- | `type_react` | `selector`, `value` | Type into React controlled inputs using the native value setter. Dispatches `input` + `change` events so React state updates correctly. |
250
+ | `type_react` | `selector`, `value`, optional `blur`, `waitAfter` | Type into React controlled inputs using the native value setter. Dispatches `input` + `change` events so React state updates correctly. `blur: true` commits on blur; `waitAfter: "<ms>"` waits after (debounced autocomplete). |
230
251
  | `click_regex` | `text` (regex), optional `selector`, optional `value: "last"` | Click element whose textContent matches a regex (case-insensitive). Default: first match. Use `value: "last"` for last match. |
231
252
  | `click_option` | `text` | Click a `[role="option"]` element by text — common in autocomplete/select dropdowns. |
253
+ | `select_combobox` | `text`, optional `selector`, `filter`, `openWait`/`filterWait`/`waitAfter` | Open a MUI Autocomplete/Select, optionally type `filter`, then click the option matching `text`. Falls back across `[role="option"]`, `.MuiAutocomplete-option`, `li.MuiMenuItem-root`. |
232
254
  | `focus_autocomplete` | `text` (label text) | Focus an autocomplete input by its label text. Supports MUI and generic `[role="combobox"]`. |
233
255
  | `click_chip` | `text` | Click a chip/tag element by text. Searches `[class*="Chip"]`, `[class*="chip"]`, `[data-chip]`. |
234
256
 
@@ -512,6 +534,67 @@ Monitor Chrome pool health: available slots, running sessions, memory pressure.
512
534
 
513
535
  ---
514
536
 
537
+ ## Browser Drivers
538
+
539
+ The runner can talk to multiple browser engines through different drivers. The default is **`auto`** — it probes each pool URL and picks the right driver per pool.
540
+
541
+ | Driver | Engine | Detection probe | When to use |
542
+ |--------|--------|-----------------|-------------|
543
+ | `browserless` | Real Chromium via [browserless](https://www.browserless.io/) | `/pressure` returns JSON | Default. Production-grade JS execution, screencast, full Chrome behavior |
544
+ | `cdp` | Generic CDP-compatible (raw Chrome, etc.) | `/json/version` reachable | Fallback for any CDP server that isn't one of the others |
545
+ | `lightpanda` | [Lightpanda](https://lightpanda.io) (Zig) | `/json/version` Browser=lightpanda | ~9× faster, ~16× less memory than headless Chrome — ideal for high-volume scrape-style tests |
546
+ | `obscura` | [Obscura](https://github.com/h4ckf0r0day/obscura) (Rust + V8) | `/json/version` Browser=obscura | ~30 MB RAM footprint, built-in anti-detection (`--stealth`), stays close to real Chrome via Puppeteer |
547
+ | `steel` | [Steel Browser](https://steel.dev) | `/v1/sessions` returns JSON | Managed session lifecycle, REST API for orchestration |
548
+
549
+ ### Pick a driver per test
550
+
551
+ ```json
552
+ {
553
+ "tests": [
554
+ {
555
+ "name": "checkout flow (heavy JS, real Chrome)",
556
+ "driver": "browserless",
557
+ "actions": [...]
558
+ },
559
+ {
560
+ "name": "scrape product page (lightweight)",
561
+ "driver": "obscura",
562
+ "fallbackDriver": "cdp",
563
+ "actions": [...]
564
+ }
565
+ ]
566
+ }
567
+ ```
568
+
569
+ `driver` is optional. If set, only pools whose detected driver matches become candidates. `fallbackDriver` is **explicit opt-in** — without it, a missing target driver fails the test with a clear message. Pool busyness does **not** trigger fallback; the runner waits inside the filtered set.
570
+
571
+ ### Force a driver for a whole run
572
+
573
+ ```bash
574
+ e2e-runner run --all --driver obscura
575
+ e2e-runner run --all --driver obscura --fallback-driver cdp
576
+ ```
577
+
578
+ CLI overrides win over per-test fields — useful for A/B benchmarks against the same suite.
579
+
580
+ ### Running each driver locally
581
+
582
+ ```bash
583
+ # browserless (default) — managed by `pool start`
584
+ e2e-runner pool start
585
+
586
+ # Lightpanda — pool start uses templates/docker-compose-lightpanda.yml
587
+ e2e-runner pool start # with poolDriver: 'lightpanda' in config
588
+
589
+ # Obscura — install the binary and run it yourself
590
+ curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-linux.tar.gz
591
+ tar xzf obscura-x86_64-linux.tar.gz
592
+ ./obscura serve --port 9222 --stealth
593
+ # then point the runner at it: poolUrls: ['http://localhost:9222'], poolDriver: 'obscura'
594
+ ```
595
+
596
+ ---
597
+
515
598
  ## Screenshot Capture
516
599
 
517
600
  Capture screenshots of any URL on demand — no test suite required:
@@ -536,7 +619,7 @@ claude plugin marketplace add fastslack/mtw-e2e-runner
536
619
  claude plugin install e2e-runner@matware
537
620
  ```
538
621
 
539
- This gives Claude 13 MCP tools, a workflow skill, 3 slash commands (`/e2e-runner:run`, `/e2e-runner:create-test`, `/e2e-runner:verify-issue`), and 3 specialized agents (test-analyzer, test-creator, test-improver).
622
+ This gives Claude 17 MCP tools, a workflow skill, 4 slash commands (`/e2e-runner:run`, `/e2e-runner:create-test`, `/e2e-runner:verify-issue`, `/e2e-runner:capture`), and 3 specialized agents (test-analyzer, test-creator, test-improver).
540
623
 
541
624
  **MCP-only install** (tools only, no skill/commands/agents):
542
625
 
@@ -563,13 +646,17 @@ See [OPENCODE.md](OPENCODE.md) for details.
563
646
  | `e2e_create_test` | Create a new test JSON file |
564
647
  | `e2e_create_module` | Create a reusable module |
565
648
  | `e2e_pool_status` | Check Chrome pool health |
649
+ | `e2e_app_pool_status` | Inspect the app environment pool (forks, ports, drivers) |
566
650
  | `e2e_screenshot` | Retrieve a screenshot by hash |
567
651
  | `e2e_capture` | Capture screenshot of any URL |
652
+ | `e2e_analyze` | Extract page structure (interactive elements, forms, headings) and emit test scaffolds |
568
653
  | `e2e_dashboard_start` | Start web dashboard |
569
654
  | `e2e_dashboard_stop` | Stop web dashboard |
655
+ | `e2e_dashboard_restart` | Restart the dashboard (new project dir/port, clear stale sessions) |
570
656
  | `e2e_issue` | Fetch issue and generate tests |
571
657
  | `e2e_network_logs` | Query network logs for a run |
572
658
  | `e2e_learnings` | Query stability insights |
659
+ | `e2e_vars` | Manage SQLite-backed `{{var.KEY}}` project variables |
573
660
  | `e2e_neo4j` | Manage Neo4j knowledge graph |
574
661
 
575
662
  > Pool start/stop are CLI-only — not exposed via MCP.
@@ -679,6 +766,8 @@ e2e-runner init # Scaffold project
679
766
  | `--env <name>` | `default` | Environment profile |
680
767
  | `--fail-on-network-error` | `false` | Fail tests with network errors |
681
768
  | `--project-name <name>` | dir name | Project display name |
769
+ | `--driver <name>` | _(per-test)_ | Force pool driver for the run: `browserless`, `cdp`, `lightpanda`, `obscura`, `steel` |
770
+ | `--fallback-driver <name>` | _none_ | Explicit fallback if no pool with `--driver` is reachable |
682
771
 
683
772
  ---
684
773
 
@@ -63,11 +63,12 @@ You are a specialist in creating robust E2E tests for web applications. You expl
63
63
 
64
64
  ### Form Interaction
65
65
  - Standard input → `type` (clears first)
66
- - React controlled input → `type_react`
67
- - Dropdown select → `select` (native) or `focus_autocomplete` + `click_option` (MUI)
66
+ - React controlled input → `type_react` (optional `blur`, `waitAfter`)
67
+ - Dropdown select → `select` (native) or `select_combobox` (MUI Autocomplete/Select — opens, optional `filter`, picks `text` in one action)
68
68
  - Checkbox/radio → `click`
69
69
  - Clear field → `clear`
70
70
  - Submit → `click` on submit button or `press` Enter
71
+ - Confirm in a modal → `click` with `text` + `scope: "dialog"` (add `last: true` if multiple matches)
71
72
 
72
73
  ### Storage
73
74
  - Set localStorage key → `set_storage` with `value: "key=val"`
@@ -83,6 +84,7 @@ You are a specialist in creating robust E2E tests for web applications. You expl
83
84
  ### Waiting
84
85
  - Element appears → `wait` with `selector`
85
86
  - Text appears → `wait` with `text`
87
+ - Element/spinner/dialog disappears → `wait` with `gone` (e.g. `{ "type": "wait", "gone": ".MuiBackdrop-root" }`)
86
88
  - Fixed delay (last resort) → `wait` with `value` (ms)
87
89
 
88
90
  ### Assertions
@@ -26,7 +26,7 @@ You are a specialist in refactoring and optimizing existing E2E tests without ch
26
26
  - **Duplication extraction**: Identify repeated action sequences across tests and extract them into reusable modules (`$use`)
27
27
  - **Selector hardening**: Replace brittle selectors (nth-child, deep nesting, generated classes) with stable alternatives (`data-testid`, `id`, text-based)
28
28
  - **Flaky test stabilization**: Add `wait` actions, `retries`, and `serial: true` based on historical failure data from the learning system
29
- - **Fixed delay elimination**: Replace hardcoded `wait` with ms values with proper waits on selectors or text
29
+ - **Fixed delay elimination**: Replace hardcoded `wait` with ms values with condition waits — `wait` on a `selector`/`text` to appear, or `wait` with `gone` to wait for a spinner/backdrop/dialog to disappear (e.g. `{ "type": "wait", "gone": ".MuiBackdrop-root" }`)
30
30
  - **Visual verification**: Add `expect` fields to tests that lack visual verification
31
31
  - **Serial marking**: Mark tests that share mutable state as `serial: true` to prevent race conditions
32
32
  - **Hook extraction**: Move duplicated setup/teardown actions into `beforeEach`/`beforeAll` hooks
@@ -68,9 +68,11 @@ When you find an `evaluate` action, check if it matches one of these patterns
68
68
  | `el.classList.contains(cls)` | `assert_class` with `selector` + `value` |
69
69
  | `el.hasAttribute(attr)` or `el.getAttribute(attr)` | `assert_attribute` with `selector` + `value` |
70
70
  | `document.querySelectorAll(sel).length` | `assert_count` with `selector` + `value` |
71
- | Native value setter + `dispatchEvent(new Event('input'))` | `type_react` with `selector` + `value` |
72
- | `querySelectorAll('[role="option"]')...click()` | `click_option` with `text` |
71
+ | Native value setter + `dispatchEvent(new Event('input'))` (single input) | `type_react` with `selector` + `value` (+ `blur:true` / `waitAfter` if it blurred/slept) |
72
+ | `querySelectorAll('[role="option"]')...click()` (no combobox open first) | `click_option` with `text` |
73
+ | Open combobox (focus/click input) + optional type filter + click matching option | `select_combobox` with `selector` + `text` (+ `filter`) |
73
74
  | `MuiAutocomplete-root...input.focus()` | `focus_autocomplete` with `text` |
75
+ | Find a button by text inside `[role="dialog"]`/`.MuiDialog-root` and click (often the LAST one) | `click` with `text` + `scope: "dialog"` (+ `last: true`) |
74
76
  | `querySelectorAll('button').filter(regex)...click()` | `click_regex` with `text` + optional `selector` + `value` |
75
77
  | `querySelectorAll('[class*="Chip"]')...click()` | `click_chip` with `text` |
76
78
  | `localStorage.setItem(key, val)` or `sessionStorage.setItem(...)` | `set_storage` with `value: "key=val"`, `selector: "session"` for session |
package/bin/cli.js CHANGED
@@ -21,7 +21,8 @@
21
21
  * e2e-runner issue <url> --generate Generate test file via Claude API
22
22
  * e2e-runner issue <url> --verify Generate + run + report bug status
23
23
  * e2e-runner issue <url> --prompt Output the AI prompt (for piping)
24
- * e2e-runner init Scaffold e2e/ in the current project
24
+ * e2e-runner init Interactive wizard to scaffold e2e/
25
+ * e2e-runner init --yes Scaffold with defaults (no prompts)
25
26
  * e2e-runner --help Show help
26
27
  * e2e-runner --version Show version
27
28
  */
@@ -43,6 +44,7 @@ import { verifyIssue } from '../src/verify.js';
43
44
  import { ensureProject, computeScreenshotHash, registerScreenshotHash } from '../src/db.js';
44
45
  import { log, colors as C } from '../src/logger.js';
45
46
  import { listModules } from '../src/module-resolver.js';
47
+ import { runInitWizard, renderConfig, getDefaultAnswers } from '../src/wizard.js';
46
48
  import { getLearningsSummary, getFlakySummary, getSelectorStability, getPageHealth, getApiHealth, getErrorPatterns, getTestTrends } from '../src/learner-sqlite.js';
47
49
  import { startNeo4j, stopNeo4j, getNeo4jStatus } from '../src/neo4j-pool.js';
48
50
  import {
@@ -118,6 +120,8 @@ function parseCLIConfig() {
118
120
  cliArgs.verificationStrictness = val;
119
121
  }
120
122
  }
123
+ if (getFlag('--driver')) cliArgs.cliDriverOverride = getFlag('--driver');
124
+ if (getFlag('--fallback-driver')) cliArgs.cliFallbackDriverOverride = getFlag('--fallback-driver');
121
125
  return cliArgs;
122
126
  }
123
127
 
@@ -175,7 +179,10 @@ ${C.bold}Usage:${C.reset}
175
179
  e2e-runner sync push Process sync queue (agent mode)
176
180
  e2e-runner sync pull Pull runs from hub (agent mode)
177
181
 
178
- e2e-runner init Scaffold e2e/ in the current project
182
+ e2e-runner init Interactive wizard to scaffold e2e/
183
+ e2e-runner init --yes Scaffold with defaults (CI / non-interactive)
184
+ Flags: --name, --base-url, --driver,
185
+ --pool-port, --concurrency, --no-sample
179
186
 
180
187
  ${C.bold}Options:${C.reset}
181
188
  --base-url <url> App base URL (default: http://host.docker.internal:3000)
@@ -199,6 +206,9 @@ ${C.bold}Options:${C.reset}
199
206
  --auth-login-endpoint <url> Auto-login: POST credentials to this URL to get auth token
200
207
  --auth-token-path <path> Dot-path to token in auth response (default: token)
201
208
  --verification-strictness <level> Visual verification: strict, moderate (default), lenient
209
+ --driver <name> Force pool driver for this run: browserless, cdp, lightpanda, obscura, steel
210
+ (overrides per-test "driver" field; useful for A/B benchmarks)
211
+ --fallback-driver <name> Explicit fallback if no pool with --driver is reachable (overrides per-test "fallbackDriver")
202
212
 
203
213
  ${C.bold}Watch Options:${C.reset}
204
214
  --interval <time> Run interval: 15m, 1h, 30s (required for schedule mode)
@@ -220,6 +230,21 @@ async function cmdRun() {
220
230
  const cliArgs = parseCLIConfig();
221
231
  const config = await loadConfig(cliArgs);
222
232
  config.triggeredBy = 'cli';
233
+
234
+ // Validate CLI driver overrides up-front (clearer error than waiting for first test)
235
+ if (config.cliDriverOverride || config.cliFallbackDriverOverride) {
236
+ const allowed = ['browserless', 'cdp', 'lightpanda', 'obscura', 'steel'];
237
+ for (const [flag, val] of [['--driver', config.cliDriverOverride], ['--fallback-driver', config.cliFallbackDriverOverride]]) {
238
+ if (val && !allowed.includes(val)) {
239
+ console.error(`${C.red}Invalid value for ${flag}: "${val}". Allowed: ${allowed.join(', ')}.${C.reset}`);
240
+ process.exit(1);
241
+ }
242
+ }
243
+ if (config.cliFallbackDriverOverride && !config.cliDriverOverride) {
244
+ console.error(`${C.red}--fallback-driver requires --driver.${C.reset}`);
245
+ process.exit(1);
246
+ }
247
+ }
223
248
  let tests = [];
224
249
  let hooks = {};
225
250
 
@@ -262,9 +287,32 @@ async function cmdRun() {
262
287
  process.exit(1);
263
288
  }
264
289
 
265
- // Verify pool connectivity
290
+ // Verify pool connectivity — auto-start the Docker-managed pool if none is
291
+ // reachable, so first-time users don't need a separate `pool start` step.
266
292
  log('🔌', `Checking Chrome Pool${poolUrls.length > 1 ? 's' : ''}...`);
267
- const pressure = await waitForAnyPool(poolUrls, 30000, { poolDriver: config.poolDriver, maxSessions: config.maxSessions });
293
+ const driverOpts = { poolDriver: config.poolDriver, maxSessions: config.maxSessions };
294
+ const _driver = config.poolDriver || 'auto';
295
+ const _dockerManaged = ['auto', 'browserless', 'lightpanda'].includes(_driver);
296
+ const _autoStart = config.autoStartPool !== false;
297
+ let pressure;
298
+ try {
299
+ pressure = await waitForAnyPool(poolUrls, 5000, driverOpts);
300
+ } catch {
301
+ if (_autoStart && _dockerManaged) {
302
+ log('🐳', `${C.dim}No pool detected — starting Chrome pool via Docker...${C.reset}`);
303
+ try {
304
+ startPool(config);
305
+ } catch (se) {
306
+ console.error(`${C.red}Could not auto-start the Chrome pool: ${se.message}${C.reset}`);
307
+ console.error(`${C.dim}Is Docker running? You can also start it manually: ${C.cyan}e2e-runner pool start${C.reset}`);
308
+ process.exit(1);
309
+ }
310
+ pressure = await waitForAnyPool(poolUrls, 45000, driverOpts);
311
+ } else {
312
+ console.error(`${C.red}No Chrome Pool available.${C.reset} Driver "${_driver}" is not Docker-managed — start your browser endpoint, then re-run.`);
313
+ process.exit(1);
314
+ }
315
+ }
268
316
  log('✅', `Pool ready (${pressure.running}/${pressure.maxConcurrent} sessions, queued: ${pressure.queued})`);
269
317
 
270
318
  // Wire up live progress to dashboard if running
@@ -387,10 +435,23 @@ async function cmdPool() {
387
435
  }
388
436
  }
389
437
 
390
- function cmdInit() {
438
+ async function cmdInit() {
391
439
  const cwd = process.cwd();
392
440
  const templatesDir = path.join(__dirname, '..', 'templates');
393
441
 
442
+ const skipWizard = hasFlag('--yes') || hasFlag('-y') || hasFlag('--non-interactive');
443
+ const flagOverrides = {};
444
+ if (getFlag('--name') && typeof getFlag('--name') === 'string') flagOverrides.projectName = getFlag('--name');
445
+ if (getFlag('--base-url') && typeof getFlag('--base-url') === 'string') flagOverrides.baseUrl = getFlag('--base-url');
446
+ if (getFlag('--driver') && typeof getFlag('--driver') === 'string') flagOverrides.driver = getFlag('--driver');
447
+ if (getFlag('--pool-port') && typeof getFlag('--pool-port') === 'string') flagOverrides.poolPort = parseInt(getFlag('--pool-port'), 10);
448
+ if (getFlag('--concurrency') && typeof getFlag('--concurrency') === 'string') flagOverrides.concurrency = parseInt(getFlag('--concurrency'), 10);
449
+ if (hasFlag('--no-sample')) flagOverrides.includeSampleTest = false;
450
+
451
+ const answers = skipWizard
452
+ ? { ...getDefaultAnswers(cwd), ...flagOverrides }
453
+ : await runInitWizard(cwd, flagOverrides);
454
+
394
455
  // Create directory structure
395
456
  const dirs = [
396
457
  path.join(cwd, 'e2e', 'tests'),
@@ -405,22 +466,24 @@ function cmdInit() {
405
466
  }
406
467
  }
407
468
 
408
- // Copy config template
469
+ // Write generated config
409
470
  const configDest = path.join(cwd, 'e2e.config.js');
410
471
  if (!fs.existsSync(configDest)) {
411
- fs.copyFileSync(path.join(templatesDir, 'e2e.config.js'), configDest);
472
+ fs.writeFileSync(configDest, renderConfig(answers));
412
473
  log('📄', 'Created e2e.config.js');
413
474
  } else {
414
475
  log('⏭️', 'e2e.config.js already exists, skipping');
415
476
  }
416
477
 
417
478
  // Copy sample test
418
- const testDest = path.join(cwd, 'e2e', 'tests', 'sample.json');
419
- if (!fs.existsSync(testDest)) {
420
- fs.copyFileSync(path.join(templatesDir, 'sample-test.json'), testDest);
421
- log('📄', 'Created e2e/tests/sample.json');
422
- } else {
423
- log('⏭️', 'e2e/tests/sample.json already exists, skipping');
479
+ if (answers.includeSampleTest) {
480
+ const testDest = path.join(cwd, 'e2e', 'tests', 'sample.json');
481
+ if (!fs.existsSync(testDest)) {
482
+ fs.copyFileSync(path.join(templatesDir, 'sample-test.json'), testDest);
483
+ log('📄', 'Created e2e/tests/sample.json');
484
+ } else {
485
+ log('⏭️', 'e2e/tests/sample.json already exists, skipping');
486
+ }
424
487
  }
425
488
 
426
489
  // Create .gitkeep
@@ -455,9 +518,9 @@ ${C.bold}${C.green}E2E structure created!${C.reset}
455
518
 
456
519
  ${C.bold}Next steps:${C.reset}
457
520
  1. Edit ${C.cyan}e2e.config.js${C.reset} with your app URL
458
- 2. Edit ${C.cyan}e2e/tests/sample.json${C.reset} with your tests
459
- 3. Start the pool: ${C.cyan}e2e-runner pool start${C.reset}
460
- 4. Run your tests: ${C.cyan}e2e-runner run --all${C.reset}
521
+ 2. Run your tests: ${C.cyan}e2e-runner run --all${C.reset} ${C.dim}(starts Chrome automatically)${C.reset}
522
+
523
+ ${C.dim}That's it the runner spins up the Chrome pool for you on first run.${C.reset}
461
524
  `);
462
525
  }
463
526
 
@@ -1138,7 +1201,7 @@ async function main() {
1138
1201
  break;
1139
1202
 
1140
1203
  case 'init':
1141
- cmdInit();
1204
+ await cmdInit();
1142
1205
  break;
1143
1206
 
1144
1207
  default:
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@matware/e2e-runner",
3
- "version": "1.3.1",
3
+ "version": "1.5.0",
4
4
  "mcpName": "io.github.fastslack/e2e-runner",
5
5
  "description": "E2E test runner using Chrome Pool (browserless/chrome) with parallel execution",
6
6
  "type": "module",
@@ -52,7 +52,8 @@
52
52
  "puppeteer-core": "^24.0.0"
53
53
  },
54
54
  "scripts": {
55
- "build:dashboard": "node templates/build-dashboard.js"
55
+ "build:dashboard": "node templates/build-dashboard.js",
56
+ "prepublishOnly": "node templates/build-dashboard.js"
56
57
  },
57
58
  "engines": {
58
59
  "node": ">=20.0.0"
@@ -72,8 +72,9 @@ Use `e2e_create_test` to write test files. Use `e2e_create_module` for reusable
72
72
  ### Key Action Patterns
73
73
 
74
74
  - **Navigation**: `goto` (full page load), `navigate` (SPA-friendly, non-blocking)
75
- - **Interaction**: `click` (selector or text), `type`/`fill`, `select`, `press`, `hover`, `scroll`
76
- - **React/MUI**: `type_react` (controlled inputs), `click_option`, `focus_autocomplete`, `click_chip`, `click_regex`
75
+ - **Interaction**: `click` (selector or text; text mode also takes `scope:"dialog"`, `visible:true`, `last:true`), `type`/`fill`, `select`, `press`, `hover`, `scroll`
76
+ - **React/MUI**: `type_react` (controlled inputs; optional `blur`, `waitAfter`), `click_option`, `select_combobox` (open+filter+pick MUI Autocomplete/Select in one action), `focus_autocomplete`, `click_chip`, `click_regex`
77
+ - **Waiting**: prefer conditions over sleeps — `wait` takes `selector`/`text` (appear), `gone` (disappear, e.g. spinner/closing dialog), or `value` (fixed ms, last resort); `wait_network_idle`
77
78
  - **Assertions**: `assert_text` (page-wide), `assert_element_text` (scoped), `assert_url`, `assert_visible`, `assert_not_visible`, `assert_count`, `assert_attribute`, `assert_class`, `assert_input_value`, `assert_matches`
78
79
  - **Extraction**: `get_text` (non-assertion, returns element text), `screenshot`
79
80
  - **Advanced**: `evaluate` (run JS in browser), `assert_no_network_errors`, `clear_cookies`
@@ -13,7 +13,7 @@ Complete catalog of all action types supported by @matware/e2e-runner.
13
13
 
14
14
  | Action | Fields | Description |
15
15
  |--------|--------|-------------|
16
- | `click` | `selector` OR `text` | Click by CSS selector or by visible text content. Text search covers: `button, a, [role="button"], [role="tab"], [role="menuitem"], [role="option"], [role="listitem"], div[class*="cursor"], span, li, td, th, label, p, h1-h6, dd, dt`. |
16
+ | `click` | `selector` OR `text` | Click by CSS selector or by visible text content. Text search covers: `button, a, [role="button"], [role="tab"], [role="menuitem"], [role="option"], [role="listitem"], div[class*="cursor"], span, li, td, th, label, p, h1-h6, dd, dt`. Optional text-mode refinements: `scope: "dialog"` (only match inside an open `[role="dialog"]`/`.MuiDialog-root`), `visible: true` (skip hidden/zero-size matches — implied by `scope:dialog`), `last: true` (click the LAST match instead of the first). Prefer these over hand-rolled `evaluate` button-by-text scans. |
17
17
  | `type` / `fill` | `selector`, `value` | Triple-clicks to select all, then Backspace to clear, then types with 20ms delay per character. |
18
18
  | `select` | `selector`, `value` | Select an `<option>` value in a `<select>` element. |
19
19
  | `clear` | `selector` | Triple-click + Backspace to clear an input field. |
@@ -25,9 +25,10 @@ Complete catalog of all action types supported by @matware/e2e-runner.
25
25
 
26
26
  | Action | Fields | Description |
27
27
  |--------|--------|-------------|
28
- | `type_react` | `selector`, `value` | Types into React controlled inputs using native value setter. Dispatches `input` + `change` events so React state updates. Supports `<input>` and `<textarea>`. |
28
+ | `type_react` | `selector`, `value`, `blur` (optional), `waitAfter` (optional ms) | Types into React controlled inputs using native value setter. Focuses, then dispatches `input` + `change` events so React state updates. Supports `<input>` and `<textarea>`. `blur: true` commits on blur (for fields that validate on blur); `waitAfter: "<ms>"` waits after (e.g. for debounced autocomplete). Prefer over inline `setNativeValue` evaluates. |
29
29
  | `click_regex` | `text` (regex), `selector` (optional), `value` (`"last"` optional) | Click element whose textContent matches regex (case-insensitive). Default: first match. `value: "last"` for last match. `selector` scopes the search. |
30
30
  | `click_option` | `text` | Click a `[role="option"]` element by text — for autocomplete/select dropdowns. Waits for option to appear. |
31
+ | `select_combobox` | `selector` (optional, default `input[role='combobox']`), `text` (option to pick), `filter` (optional typed text), `openWait`/`filterWait`/`waitAfter` (optional ms) | Open a MUI Autocomplete/Select, optionally type `filter` to narrow, then click the option matching `text` (case-insensitive substring). Falls back across `[role="option"]`, `.MuiAutocomplete-option`, `li.MuiMenuItem-root`. Replaces the verbose open-input + setNativeValue + scan-options `evaluate` pattern. |
31
32
  | `focus_autocomplete` | `text` (label text) | Focus an autocomplete input by label. Supports MUI `.MuiAutocomplete-root` and `[role="combobox"]`. |
32
33
  | `click_chip` | `text` | Click a chip/tag element by text. Searches `[class*="Chip"]`, `[class*="chip"]`, `[data-chip]`. |
33
34
 
@@ -75,7 +76,7 @@ Complete catalog of all action types supported by @matware/e2e-runner.
75
76
  |--------|--------|-------------|
76
77
  | `get_text` | `selector` | Returns `{ value: textContent.trim() }`. Non-assertion — never fails. |
77
78
  | `screenshot` | `value` (filename, optional) | Captures screenshot. Filename gets timestamp suffix for uniqueness. |
78
- | `wait` | `selector` OR `text` OR `value` (ms) | Wait for selector, text on page, or fixed delay. |
79
+ | `wait` | `selector` OR `text` OR `gone` OR `value` (ms) | Prefer **conditions over fixed sleeps**: `{ selector }` waits for it to appear, `{ text }` waits for text to appear, **`{ gone: "<css>" }`** waits until a selector disappears/hides (spinner, closing dialog), `{ gone: true, selector|text }` is the explicit form, `{ value: "<ms>" }` is a fixed delay (last resort). Replacing `wait` sleeps with `gone`/`selector` makes suites faster and less flaky. |
79
80
  | `wait_network_idle` | `value` (idle ms, default 500), `timeout` (max wait ms, default 30000) | Waits for all network requests to complete. Uses Puppeteer's `page.waitForNetworkIdle()`. Useful after SPA page transitions or data loading. |
80
81
  | `evaluate` | `value` (JS code) | Run JavaScript in browser context. See **Strict Evaluate** below. |
81
82
  | `clear_cookies` | `value` (origin, optional) | Clears cookies, localStorage, sessionStorage for origin. |
@@ -107,10 +108,27 @@ Delay between retries: `actionRetryDelay` config (default 500ms).
107
108
  ### React input + autocomplete flow
108
109
  ```json
109
110
  { "type": "focus_autocomplete", "text": "Category" },
110
- { "type": "type_react", "selector": "#category-input", "value": "Electr" },
111
+ { "type": "type_react", "selector": "#category-input", "value": "Electr", "waitAfter": "400" },
111
112
  { "type": "click_option", "text": "Electronics" }
112
113
  ```
113
114
 
115
+ ### MUI combobox in one action (open + filter + pick)
116
+ ```json
117
+ { "type": "select_combobox", "selector": "[data-cy='specialty'] input", "filter": "cardio", "text": "Cardiología" }
118
+ ```
119
+
120
+ ### Condition waits instead of fixed sleeps (faster, less flaky)
121
+ ```json
122
+ { "type": "click", "text": "Guardar" },
123
+ { "type": "wait", "gone": ".MuiBackdrop-root" },
124
+ { "type": "wait", "selector": "[data-testid='saved-banner']" }
125
+ ```
126
+
127
+ ### Click a button inside an open dialog (no evaluate needed)
128
+ ```json
129
+ { "type": "click", "text": "Iniciar encuentro", "scope": "dialog", "last": true }
130
+ ```
131
+
114
132
  ### Regex click (last match)
115
133
  ```json
116
134
  { "type": "click_regex", "text": "add to cart", "selector": "button", "value": "last" }
@@ -113,6 +113,29 @@ Module definition (in `e2e/modules/auth-login.json`):
113
113
  }
114
114
  ```
115
115
 
116
+ ### Composing modules (nested `$use` + parameter forwarding)
117
+
118
+ A module can `$use` other modules, and **forward its own params/defaults** into the
119
+ nested call's `params` block. Placeholders in a nested `params` value are resolved
120
+ against the outer module's scope before the inner module runs:
121
+
122
+ ```json
123
+ {
124
+ "$module": "login-and-open",
125
+ "params": {
126
+ "patientId": { "required": true },
127
+ "email": { "required": false, "default": "admin@test.com" }
128
+ },
129
+ "actions": [
130
+ { "$use": "auth-login", "params": { "email": "{{email}}", "password": "secret" } },
131
+ { "$use": "open-patient", "params": { "id": "{{patientId}}" } }
132
+ ]
133
+ }
134
+ ```
135
+
136
+ Cycles are detected and rejected. Action types are validated **after** all `$use`
137
+ references are expanded.
138
+
116
139
  ## Suite Naming & Ordering
117
140
 
118
141
  Files can have numeric prefixes for execution order: