@matware/e2e-runner 1.3.1 → 1.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (50) hide show
  1. package/.claude-plugin/marketplace.json +4 -4
  2. package/.claude-plugin/plugin.json +2 -2
  3. package/LICENSE +1 -1
  4. package/README.md +491 -225
  5. package/agents/test-creator.md +4 -2
  6. package/agents/test-improver.md +7 -4
  7. package/bin/cli.js +93 -19
  8. package/package.json +4 -3
  9. package/skills/e2e-testing/SKILL.md +5 -3
  10. package/skills/e2e-testing/references/action-types.md +35 -18
  11. package/skills/e2e-testing/references/test-json-format.md +23 -0
  12. package/skills/e2e-testing/references/troubleshooting.md +2 -26
  13. package/src/actions.js +181 -15
  14. package/src/config.js +6 -0
  15. package/src/dashboard.js +185 -9
  16. package/src/db.js +26 -0
  17. package/src/mcp-tools.js +238 -69
  18. package/src/module-analysis.js +247 -0
  19. package/src/module-resolver.js +35 -2
  20. package/src/narrate.js +33 -1
  21. package/src/pool-manager.js +46 -1
  22. package/src/pool.js +177 -20
  23. package/src/runner.js +144 -19
  24. package/src/visual-diff.js +74 -4
  25. package/src/websocket.js +14 -3
  26. package/src/wizard.js +184 -0
  27. package/templates/build-dashboard.js +3 -0
  28. package/templates/dashboard/js/api.js +60 -3
  29. package/templates/dashboard/js/init.js +46 -0
  30. package/templates/dashboard/js/keyboard.js +8 -7
  31. package/templates/dashboard/js/quicksearch.js +277 -0
  32. package/templates/dashboard/js/state.js +61 -7
  33. package/templates/dashboard/js/toast.js +1 -1
  34. package/templates/dashboard/js/utils.js +23 -2
  35. package/templates/dashboard/js/view-live.js +235 -42
  36. package/templates/dashboard/js/view-runs.js +469 -42
  37. package/templates/dashboard/js/view-tests.js +157 -16
  38. package/templates/dashboard/js/view-tools.js +234 -0
  39. package/templates/dashboard/js/view-watch.js +2 -2
  40. package/templates/dashboard/js/websocket.js +33 -3
  41. package/templates/dashboard/styles/base.css +489 -53
  42. package/templates/dashboard/styles/components.css +736 -84
  43. package/templates/dashboard/styles/view-live.css +459 -78
  44. package/templates/dashboard/styles/view-runs.css +826 -177
  45. package/templates/dashboard/styles/view-tests.css +440 -77
  46. package/templates/dashboard/styles/view-tools.css +206 -0
  47. package/templates/dashboard/styles/view-watch.css +198 -41
  48. package/templates/dashboard/template.html +356 -58
  49. package/templates/dashboard.html +5354 -722
  50. package/templates/docker-compose-lightpanda.yml +7 -0
@@ -63,11 +63,12 @@ You are a specialist in creating robust E2E tests for web applications. You expl
63
63
 
64
64
  ### Form Interaction
65
65
  - Standard input → `type` (clears first)
66
- - React controlled input → `type_react`
67
- - Dropdown select → `select` (native) or `focus_autocomplete` + `click_option` (MUI)
66
+ - React controlled input → `type_react` (optional `blur`, `waitAfter`)
67
+ - Dropdown select → `select` (native) or `select_combobox` (MUI Autocomplete/Select — opens, optional `filter`, picks `text` in one action)
68
68
  - Checkbox/radio → `click`
69
69
  - Clear field → `clear`
70
70
  - Submit → `click` on submit button or `press` Enter
71
+ - Confirm in a modal → `click` with `text` + `scope: "dialog"` (add `last: true` if multiple matches)
71
72
 
72
73
  ### Storage
73
74
  - Set localStorage key → `set_storage` with `value: "key=val"`
@@ -83,6 +84,7 @@ You are a specialist in creating robust E2E tests for web applications. You expl
83
84
  ### Waiting
84
85
  - Element appears → `wait` with `selector`
85
86
  - Text appears → `wait` with `text`
87
+ - Element/spinner/dialog disappears → `wait` with `gone` (e.g. `{ "type": "wait", "gone": ".MuiBackdrop-root" }`)
86
88
  - Fixed delay (last resort) → `wait` with `value` (ms)
87
89
 
88
90
  ### Assertions
@@ -24,9 +24,9 @@ You are a specialist in refactoring and optimizing existing E2E tests without ch
24
24
 
25
25
  - **Evaluate replacement**: Replace verbose `evaluate` actions with equivalent built-in actions (`type_react`, `click_option`, `assert_element_text`, etc.)
26
26
  - **Duplication extraction**: Identify repeated action sequences across tests and extract them into reusable modules (`$use`)
27
- - **Selector hardening**: Replace brittle selectors (nth-child, deep nesting, generated classes) with stable alternatives (`data-testid`, `id`, text-based)
27
+ - **Selector hardening**: Replace brittle selectors (nth-child, deep nesting, generated classes) with stable alternatives (`data-testid`, `id`, text-based). Applies to *interaction* selectors only — assertion selectors are treated as stable contracts (see Rules)
28
28
  - **Flaky test stabilization**: Add `wait` actions, `retries`, and `serial: true` based on historical failure data from the learning system
29
- - **Fixed delay elimination**: Replace hardcoded `wait` with ms values with proper waits on selectors or text
29
+ - **Fixed delay elimination**: Replace hardcoded `wait` with ms values with condition waits — `wait` on a `selector`/`text` to appear, or `wait` with `gone` to wait for a spinner/backdrop/dialog to disappear (e.g. `{ "type": "wait", "gone": ".MuiBackdrop-root" }`)
30
30
  - **Visual verification**: Add `expect` fields to tests that lack visual verification
31
31
  - **Serial marking**: Mark tests that share mutable state as `serial: true` to prevent race conditions
32
32
  - **Hook extraction**: Move duplicated setup/teardown actions into `beforeEach`/`beforeAll` hooks
@@ -68,9 +68,11 @@ When you find an `evaluate` action, check if it matches one of these patterns
68
68
  | `el.classList.contains(cls)` | `assert_class` with `selector` + `value` |
69
69
  | `el.hasAttribute(attr)` or `el.getAttribute(attr)` | `assert_attribute` with `selector` + `value` |
70
70
  | `document.querySelectorAll(sel).length` | `assert_count` with `selector` + `value` |
71
- | Native value setter + `dispatchEvent(new Event('input'))` | `type_react` with `selector` + `value` |
72
- | `querySelectorAll('[role="option"]')...click()` | `click_option` with `text` |
71
+ | Native value setter + `dispatchEvent(new Event('input'))` (single input) | `type_react` with `selector` + `value` (+ `blur:true` / `waitAfter` if it blurred/slept) |
72
+ | `querySelectorAll('[role="option"]')...click()` (no combobox open first) | `click_option` with `text` |
73
+ | Open combobox (focus/click input) + optional type filter + click matching option | `select_combobox` with `selector` + `text` (+ `filter`) |
73
74
  | `MuiAutocomplete-root...input.focus()` | `focus_autocomplete` with `text` |
75
+ | Find a button by text inside `[role="dialog"]`/`.MuiDialog-root` and click (often the LAST one) | `click` with `text` + `scope: "dialog"` (+ `last: true`) |
74
76
  | `querySelectorAll('button').filter(regex)...click()` | `click_regex` with `text` + optional `selector` + `value` |
75
77
  | `querySelectorAll('[class*="Chip"]')...click()` | `click_chip` with `text` |
76
78
  | `localStorage.setItem(key, val)` or `sessionStorage.setItem(...)` | `set_storage` with `value: "key=val"`, `selector: "session"` for session |
@@ -165,6 +167,7 @@ When extracting to a module, use `{{param}}` placeholders for values that vary b
165
167
  4. **Preserve test ordering** — don't reorder tests within a suite. Numeric prefix ordering is intentional.
166
168
  5. **Keep evaluates when no built-in exists** — if the evaluate does something that no built-in action covers (e.g., complex DOM manipulation, localStorage checks), leave it as-is.
167
169
  6. **Prefer selector waits over fixed delays** — replace `{ "type": "wait", "value": "3000" }` with `{ "type": "wait", "selector": ".expected-element" }` when possible. Only keep fixed delays when there's genuinely no element to wait for.
170
+ 7. **Assertion selectors are the contract** — interaction selectors may heal, but never retarget the `selector` of an `assert_*` action while hardening: that silently changes *what* the test verifies. If an assertion selector is genuinely broken, pin it to a stable `data-testid` and call it out in the summary instead of swapping it for whatever makes the test green.
168
171
 
169
172
  ## Output
170
173
 
package/bin/cli.js CHANGED
@@ -21,7 +21,8 @@
21
21
  * e2e-runner issue <url> --generate Generate test file via Claude API
22
22
  * e2e-runner issue <url> --verify Generate + run + report bug status
23
23
  * e2e-runner issue <url> --prompt Output the AI prompt (for piping)
24
- * e2e-runner init Scaffold e2e/ in the current project
24
+ * e2e-runner init Interactive wizard to scaffold e2e/
25
+ * e2e-runner init --yes Scaffold with defaults (no prompts)
25
26
  * e2e-runner --help Show help
26
27
  * e2e-runner --version Show version
27
28
  */
@@ -34,6 +35,7 @@ import { loadConfig } from '../src/config.js';
34
35
  import { startPool, stopPool, restartPool, connectToPool } from '../src/pool.js';
35
36
  import { getPoolUrls, getAggregatedPoolStatus, waitForAnyPool, selectPool } from '../src/pool-manager.js';
36
37
  import { runTestsParallel, loadTestFile, loadTestSuite, loadAllSuites, listSuites } from '../src/runner.js';
38
+ import { looksLikeBlankCapture } from '../src/actions.js';
37
39
  import { generateReport, saveReport, printReport, persistRun, printInsights } from '../src/reporter.js';
38
40
  import { startDashboard } from '../src/dashboard.js';
39
41
  import { startWatch } from '../src/watch.js';
@@ -43,6 +45,7 @@ import { verifyIssue } from '../src/verify.js';
43
45
  import { ensureProject, computeScreenshotHash, registerScreenshotHash } from '../src/db.js';
44
46
  import { log, colors as C } from '../src/logger.js';
45
47
  import { listModules } from '../src/module-resolver.js';
48
+ import { runInitWizard, renderConfig, getDefaultAnswers } from '../src/wizard.js';
46
49
  import { getLearningsSummary, getFlakySummary, getSelectorStability, getPageHealth, getApiHealth, getErrorPatterns, getTestTrends } from '../src/learner-sqlite.js';
47
50
  import { startNeo4j, stopNeo4j, getNeo4jStatus } from '../src/neo4j-pool.js';
48
51
  import {
@@ -118,6 +121,8 @@ function parseCLIConfig() {
118
121
  cliArgs.verificationStrictness = val;
119
122
  }
120
123
  }
124
+ if (getFlag('--driver')) cliArgs.cliDriverOverride = getFlag('--driver');
125
+ if (getFlag('--fallback-driver')) cliArgs.cliFallbackDriverOverride = getFlag('--fallback-driver');
121
126
  return cliArgs;
122
127
  }
123
128
 
@@ -148,6 +153,7 @@ ${C.bold}Usage:${C.reset}
148
153
  e2e-runner capture <url> --selector <sel> Wait for selector before capture
149
154
  e2e-runner capture <url> --delay <ms> Wait before capturing
150
155
  e2e-runner capture <url> --filename <name> Custom filename
156
+ e2e-runner capture <url> --force Save even if the frame is blank
151
157
 
152
158
  e2e-runner issue <url> Fetch issue and show details
153
159
  e2e-runner issue <url> --generate Generate test file via Claude API
@@ -175,7 +181,10 @@ ${C.bold}Usage:${C.reset}
175
181
  e2e-runner sync push Process sync queue (agent mode)
176
182
  e2e-runner sync pull Pull runs from hub (agent mode)
177
183
 
178
- e2e-runner init Scaffold e2e/ in the current project
184
+ e2e-runner init Interactive wizard to scaffold e2e/
185
+ e2e-runner init --yes Scaffold with defaults (CI / non-interactive)
186
+ Flags: --name, --base-url, --driver,
187
+ --pool-port, --concurrency, --no-sample
179
188
 
180
189
  ${C.bold}Options:${C.reset}
181
190
  --base-url <url> App base URL (default: http://host.docker.internal:3000)
@@ -199,6 +208,9 @@ ${C.bold}Options:${C.reset}
199
208
  --auth-login-endpoint <url> Auto-login: POST credentials to this URL to get auth token
200
209
  --auth-token-path <path> Dot-path to token in auth response (default: token)
201
210
  --verification-strictness <level> Visual verification: strict, moderate (default), lenient
211
+ --driver <name> Force pool driver for this run: browserless, cdp, lightpanda, obscura, steel
212
+ (overrides per-test "driver" field; useful for A/B benchmarks)
213
+ --fallback-driver <name> Explicit fallback if no pool with --driver is reachable (overrides per-test "fallbackDriver")
202
214
 
203
215
  ${C.bold}Watch Options:${C.reset}
204
216
  --interval <time> Run interval: 15m, 1h, 30s (required for schedule mode)
@@ -220,6 +232,21 @@ async function cmdRun() {
220
232
  const cliArgs = parseCLIConfig();
221
233
  const config = await loadConfig(cliArgs);
222
234
  config.triggeredBy = 'cli';
235
+
236
+ // Validate CLI driver overrides up-front (clearer error than waiting for first test)
237
+ if (config.cliDriverOverride || config.cliFallbackDriverOverride) {
238
+ const allowed = ['browserless', 'cdp', 'lightpanda', 'obscura', 'steel'];
239
+ for (const [flag, val] of [['--driver', config.cliDriverOverride], ['--fallback-driver', config.cliFallbackDriverOverride]]) {
240
+ if (val && !allowed.includes(val)) {
241
+ console.error(`${C.red}Invalid value for ${flag}: "${val}". Allowed: ${allowed.join(', ')}.${C.reset}`);
242
+ process.exit(1);
243
+ }
244
+ }
245
+ if (config.cliFallbackDriverOverride && !config.cliDriverOverride) {
246
+ console.error(`${C.red}--fallback-driver requires --driver.${C.reset}`);
247
+ process.exit(1);
248
+ }
249
+ }
223
250
  let tests = [];
224
251
  let hooks = {};
225
252
 
@@ -262,9 +289,32 @@ async function cmdRun() {
262
289
  process.exit(1);
263
290
  }
264
291
 
265
- // Verify pool connectivity
292
+ // Verify pool connectivity — auto-start the Docker-managed pool if none is
293
+ // reachable, so first-time users don't need a separate `pool start` step.
266
294
  log('🔌', `Checking Chrome Pool${poolUrls.length > 1 ? 's' : ''}...`);
267
- const pressure = await waitForAnyPool(poolUrls, 30000, { poolDriver: config.poolDriver, maxSessions: config.maxSessions });
295
+ const driverOpts = { poolDriver: config.poolDriver, maxSessions: config.maxSessions };
296
+ const _driver = config.poolDriver || 'auto';
297
+ const _dockerManaged = ['auto', 'browserless', 'lightpanda'].includes(_driver);
298
+ const _autoStart = config.autoStartPool !== false;
299
+ let pressure;
300
+ try {
301
+ pressure = await waitForAnyPool(poolUrls, 5000, driverOpts);
302
+ } catch {
303
+ if (_autoStart && _dockerManaged) {
304
+ log('🐳', `${C.dim}No pool detected — starting Chrome pool via Docker...${C.reset}`);
305
+ try {
306
+ startPool(config);
307
+ } catch (se) {
308
+ console.error(`${C.red}Could not auto-start the Chrome pool: ${se.message}${C.reset}`);
309
+ console.error(`${C.dim}Is Docker running? You can also start it manually: ${C.cyan}e2e-runner pool start${C.reset}`);
310
+ process.exit(1);
311
+ }
312
+ pressure = await waitForAnyPool(poolUrls, 45000, driverOpts);
313
+ } else {
314
+ console.error(`${C.red}No Chrome Pool available.${C.reset} Driver "${_driver}" is not Docker-managed — start your browser endpoint, then re-run.`);
315
+ process.exit(1);
316
+ }
317
+ }
268
318
  log('✅', `Pool ready (${pressure.running}/${pressure.maxConcurrent} sessions, queued: ${pressure.queued})`);
269
319
 
270
320
  // Wire up live progress to dashboard if running
@@ -387,10 +437,23 @@ async function cmdPool() {
387
437
  }
388
438
  }
389
439
 
390
- function cmdInit() {
440
+ async function cmdInit() {
391
441
  const cwd = process.cwd();
392
442
  const templatesDir = path.join(__dirname, '..', 'templates');
393
443
 
444
+ const skipWizard = hasFlag('--yes') || hasFlag('-y') || hasFlag('--non-interactive');
445
+ const flagOverrides = {};
446
+ if (getFlag('--name') && typeof getFlag('--name') === 'string') flagOverrides.projectName = getFlag('--name');
447
+ if (getFlag('--base-url') && typeof getFlag('--base-url') === 'string') flagOverrides.baseUrl = getFlag('--base-url');
448
+ if (getFlag('--driver') && typeof getFlag('--driver') === 'string') flagOverrides.driver = getFlag('--driver');
449
+ if (getFlag('--pool-port') && typeof getFlag('--pool-port') === 'string') flagOverrides.poolPort = parseInt(getFlag('--pool-port'), 10);
450
+ if (getFlag('--concurrency') && typeof getFlag('--concurrency') === 'string') flagOverrides.concurrency = parseInt(getFlag('--concurrency'), 10);
451
+ if (hasFlag('--no-sample')) flagOverrides.includeSampleTest = false;
452
+
453
+ const answers = skipWizard
454
+ ? { ...getDefaultAnswers(cwd), ...flagOverrides }
455
+ : await runInitWizard(cwd, flagOverrides);
456
+
394
457
  // Create directory structure
395
458
  const dirs = [
396
459
  path.join(cwd, 'e2e', 'tests'),
@@ -405,22 +468,24 @@ function cmdInit() {
405
468
  }
406
469
  }
407
470
 
408
- // Copy config template
471
+ // Write generated config
409
472
  const configDest = path.join(cwd, 'e2e.config.js');
410
473
  if (!fs.existsSync(configDest)) {
411
- fs.copyFileSync(path.join(templatesDir, 'e2e.config.js'), configDest);
474
+ fs.writeFileSync(configDest, renderConfig(answers));
412
475
  log('📄', 'Created e2e.config.js');
413
476
  } else {
414
477
  log('⏭️', 'e2e.config.js already exists, skipping');
415
478
  }
416
479
 
417
480
  // Copy sample test
418
- const testDest = path.join(cwd, 'e2e', 'tests', 'sample.json');
419
- if (!fs.existsSync(testDest)) {
420
- fs.copyFileSync(path.join(templatesDir, 'sample-test.json'), testDest);
421
- log('📄', 'Created e2e/tests/sample.json');
422
- } else {
423
- log('⏭️', 'e2e/tests/sample.json already exists, skipping');
481
+ if (answers.includeSampleTest) {
482
+ const testDest = path.join(cwd, 'e2e', 'tests', 'sample.json');
483
+ if (!fs.existsSync(testDest)) {
484
+ fs.copyFileSync(path.join(templatesDir, 'sample-test.json'), testDest);
485
+ log('📄', 'Created e2e/tests/sample.json');
486
+ } else {
487
+ log('⏭️', 'e2e/tests/sample.json already exists, skipping');
488
+ }
424
489
  }
425
490
 
426
491
  // Create .gitkeep
@@ -455,9 +520,9 @@ ${C.bold}${C.green}E2E structure created!${C.reset}
455
520
 
456
521
  ${C.bold}Next steps:${C.reset}
457
522
  1. Edit ${C.cyan}e2e.config.js${C.reset} with your app URL
458
- 2. Edit ${C.cyan}e2e/tests/sample.json${C.reset} with your tests
459
- 3. Start the pool: ${C.cyan}e2e-runner pool start${C.reset}
460
- 4. Run your tests: ${C.cyan}e2e-runner run --all${C.reset}
523
+ 2. Run your tests: ${C.cyan}e2e-runner run --all${C.reset} ${C.dim}(starts Chrome automatically)${C.reset}
524
+
525
+ ${C.dim}That's it the runner spins up the Chrome pool for you on first run.${C.reset}
461
526
  `);
462
527
  }
463
528
 
@@ -483,7 +548,7 @@ async function cmdDashboard() {
483
548
  async function cmdCapture() {
484
549
  const url = args[1];
485
550
  if (!url || url.startsWith('--')) {
486
- console.error(`${C.red}Usage: e2e-runner capture <url> [--filename <name>] [--full-page] [--selector <sel>] [--delay <ms>]${C.reset}`);
551
+ console.error(`${C.red}Usage: e2e-runner capture <url> [--filename <name>] [--full-page] [--selector <sel>] [--delay <ms>] [--force]${C.reset}`);
487
552
  process.exit(1);
488
553
  }
489
554
 
@@ -529,7 +594,16 @@ async function cmdCapture() {
529
594
 
530
595
  const screenshotPath = path.join(config.screenshotsDir, filename);
531
596
  const fullPage = hasFlag('--full-page');
532
- await page.screenshot({ path: screenshotPath, fullPage });
597
+ const captureBuf = await page.screenshot({ fullPage });
598
+
599
+ // Blank frame (uniform color — page never rendered): skip the save
600
+ // unless the user explicitly forces it.
601
+ if (!hasFlag('--force') && looksLikeBlankCapture(captureBuf, 'png')) {
602
+ log('⚠️', `${C.yellow}Capture skipped:${C.reset} page rendered a blank (uniform-color) frame — nothing saved. Use ${C.dim}--force${C.reset} to save anyway.`);
603
+ console.log('');
604
+ return;
605
+ }
606
+ fs.writeFileSync(screenshotPath, captureBuf);
533
607
 
534
608
  // Register hash in SQLite
535
609
  const cwd = process.cwd();
@@ -1138,7 +1212,7 @@ async function main() {
1138
1212
  break;
1139
1213
 
1140
1214
  case 'init':
1141
- cmdInit();
1215
+ await cmdInit();
1142
1216
  break;
1143
1217
 
1144
1218
  default:
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@matware/e2e-runner",
3
- "version": "1.3.1",
3
+ "version": "1.5.1",
4
4
  "mcpName": "io.github.fastslack/e2e-runner",
5
5
  "description": "E2E test runner using Chrome Pool (browserless/chrome) with parallel execution",
6
6
  "type": "module",
@@ -39,7 +39,7 @@
39
39
  "github-issues",
40
40
  "ai-testing"
41
41
  ],
42
- "author": "Matware",
42
+ "author": "Matias Aguirre (Matware)",
43
43
  "license": "Apache-2.0",
44
44
  "repository": {
45
45
  "type": "git",
@@ -52,7 +52,8 @@
52
52
  "puppeteer-core": "^24.0.0"
53
53
  },
54
54
  "scripts": {
55
- "build:dashboard": "node templates/build-dashboard.js"
55
+ "build:dashboard": "node templates/build-dashboard.js",
56
+ "prepublishOnly": "node templates/build-dashboard.js"
56
57
  },
57
58
  "engines": {
58
59
  "node": ">=20.0.0"
@@ -9,7 +9,7 @@ description: Create, run, and debug JSON-driven E2E browser tests with Chrome po
9
9
 
10
10
  `@matware/e2e-runner` is a JSON-driven E2E test runner. Tests are defined as JSON files with sequential browser actions — no JavaScript test code. Tests run in parallel against a Chrome pool (browserless/chrome via Docker) using Puppeteer.
11
11
 
12
- **Key capabilities:** 13 MCP tools for running tests, creating test files, capturing screenshots, analyzing network traffic, verifying GitHub/GitLab issues, and querying a learning system for stability insights.
12
+ **Key capabilities:** 17 MCP tools for running tests, creating test files, capturing screenshots, analyzing network traffic, verifying GitHub/GitLab issues, and querying a learning system for stability insights.
13
13
 
14
14
  ## Prerequisites
15
15
 
@@ -72,8 +72,9 @@ Use `e2e_create_test` to write test files. Use `e2e_create_module` for reusable
72
72
  ### Key Action Patterns
73
73
 
74
74
  - **Navigation**: `goto` (full page load), `navigate` (SPA-friendly, non-blocking)
75
- - **Interaction**: `click` (selector or text), `type`/`fill`, `select`, `press`, `hover`, `scroll`
76
- - **React/MUI**: `type_react` (controlled inputs), `click_option`, `focus_autocomplete`, `click_chip`, `click_regex`
75
+ - **Interaction**: `click` (selector or text; text mode also takes `scope:"dialog"`, `visible:true`, `last:true`), `type`/`fill`, `select`, `press`, `hover`, `scroll`
76
+ - **React/MUI**: `type_react` (controlled inputs; optional `blur`, `waitAfter`), `click_option`, `select_combobox` (open+filter+pick MUI Autocomplete/Select in one action), `focus_autocomplete`, `click_chip`, `click_regex`
77
+ - **Waiting**: prefer conditions over sleeps — `wait` takes `selector`/`text` (appear), `gone` (disappear, e.g. spinner/closing dialog), or `value` (fixed ms, last resort); `wait_network_idle`
77
78
  - **Assertions**: `assert_text` (page-wide), `assert_element_text` (scoped), `assert_url`, `assert_visible`, `assert_not_visible`, `assert_count`, `assert_attribute`, `assert_class`, `assert_input_value`, `assert_matches`
78
79
  - **Extraction**: `get_text` (non-assertion, returns element text), `screenshot`
79
80
  - **Advanced**: `evaluate` (run JS in browser), `assert_no_network_errors`, `clear_cookies`
@@ -158,6 +159,7 @@ Start/stop the web dashboard with `e2e_dashboard_start` / `e2e_dashboard_stop` f
158
159
  4. **`evaluate` is strict** — Returns starting with `FAIL:`/`ERROR:` or returning `false` will fail the test. Prefer granular assertion actions over `evaluate` with inline JS.
159
160
  5. **Serial tests** — Mark tests with `"serial": true` if they share mutable state. They run after all parallel tests.
160
161
  6. **Action retries** — Use `"retries": N` on individual actions for flaky selectors, or globally via config.
162
+ 7. **Assertion selectors are the contract** — when fixing flaky tests, heal *interaction* selectors freely, but never retarget an `assert_*` selector to make a test pass: pin assertion selectors to stable `data-testid`s.
161
163
 
162
164
  ## References
163
165
 
@@ -13,7 +13,7 @@ Complete catalog of all action types supported by @matware/e2e-runner.
13
13
 
14
14
  | Action | Fields | Description |
15
15
  |--------|--------|-------------|
16
- | `click` | `selector` OR `text` | Click by CSS selector or by visible text content. Text search covers: `button, a, [role="button"], [role="tab"], [role="menuitem"], [role="option"], [role="listitem"], div[class*="cursor"], span, li, td, th, label, p, h1-h6, dd, dt`. |
16
+ | `click` | `selector` OR `text` | Click by CSS selector or by visible text content. Text search covers: `button, a, [role="button"], [role="tab"], [role="menuitem"], [role="option"], [role="listitem"], div[class*="cursor"], span, li, td, th, label, p, h1-h6, dd, dt`. Optional text-mode refinements: `scope: "dialog"` (only match inside an open `[role="dialog"]`/`.MuiDialog-root`), `visible: true` (skip hidden/zero-size matches — implied by `scope:dialog`), `last: true` (click the LAST match instead of the first). Prefer these over hand-rolled `evaluate` button-by-text scans. |
17
17
  | `type` / `fill` | `selector`, `value` | Triple-clicks to select all, then Backspace to clear, then types with 20ms delay per character. |
18
18
  | `select` | `selector`, `value` | Select an `<option>` value in a `<select>` element. |
19
19
  | `clear` | `selector` | Triple-click + Backspace to clear an input field. |
@@ -25,9 +25,10 @@ Complete catalog of all action types supported by @matware/e2e-runner.
25
25
 
26
26
  | Action | Fields | Description |
27
27
  |--------|--------|-------------|
28
- | `type_react` | `selector`, `value` | Types into React controlled inputs using native value setter. Dispatches `input` + `change` events so React state updates. Supports `<input>` and `<textarea>`. |
28
+ | `type_react` | `selector`, `value`, `blur` (optional), `waitAfter` (optional ms) | Types into React controlled inputs using native value setter. Focuses, then dispatches `input` + `change` events so React state updates. Supports `<input>` and `<textarea>`. `blur: true` commits on blur (for fields that validate on blur); `waitAfter: "<ms>"` waits after (e.g. for debounced autocomplete). Prefer over inline `setNativeValue` evaluates. |
29
29
  | `click_regex` | `text` (regex), `selector` (optional), `value` (`"last"` optional) | Click element whose textContent matches regex (case-insensitive). Default: first match. `value: "last"` for last match. `selector` scopes the search. |
30
30
  | `click_option` | `text` | Click a `[role="option"]` element by text — for autocomplete/select dropdowns. Waits for option to appear. |
31
+ | `select_combobox` | `selector` (optional, default `input[role='combobox']`), `text` (option to pick), `filter` (optional typed text), `openWait`/`filterWait`/`waitAfter` (optional ms) | Open a MUI Autocomplete/Select, optionally type `filter` to narrow, then click the option matching `text` (case-insensitive substring). Falls back across `[role="option"]`, `.MuiAutocomplete-option`, `li.MuiMenuItem-root`. Replaces the verbose open-input + setNativeValue + scan-options `evaluate` pattern. |
31
32
  | `focus_autocomplete` | `text` (label text) | Focus an autocomplete input by label. Supports MUI `.MuiAutocomplete-root` and `[role="combobox"]`. |
32
33
  | `click_chip` | `text` | Click a chip/tag element by text. Searches `[class*="Chip"]`, `[class*="chip"]`, `[data-chip]`. |
33
34
 
@@ -46,11 +47,25 @@ Complete catalog of all action types supported by @matware/e2e-runner.
46
47
  | `click_menu_item` | `text` (menu item text), `selector` (scope, optional) | Click a menu item by text. Searches `[role="menuitem"]`, `[role="menuitemradio"]`, `[role="menuitemcheckbox"]`, `.dropdown-item`, `.menu-item`, `[class*="MenuItem"]`, `[role="menu"] > li`. Waits for element to appear. |
47
48
  | `click_in_context` | `text` (container text), `selector` (child to click) | Find the smallest container whose text includes `text`, then click the `selector` child within it. Containers: `section`, `article`, `[class*="card"]`, `li`, `tr`, `div[class]`, etc. Both fields required. |
48
49
 
50
+ ## Multi-Tab
51
+
52
+ All subsequent actions run in the active tab. The runner manages a tab registry keyed by label.
53
+
54
+ | Action | Fields | Description |
55
+ |--------|--------|-------------|
56
+ | `open_tab` | `value` (URL), `text` (label, optional) | Open a new tab and navigate (relative to `baseUrl` or absolute). Label defaults to `tab-<n>`. The new tab becomes active. |
57
+ | `switch_tab` | `value` | Switch active tab by label (exact), numeric index, or title/URL match (regex or substring). `"default"` returns to the original tab. |
58
+ | `wait_for_tab` | `text` (label, optional), `timeout` | Wait for a tab/popup opened by the app (`window.open`, `target="_blank"`) and make it active. Use right after the action that triggers the popup. |
59
+ | `assert_tab_count` | `value` | Assert number of open tabs: exact (`"2"`) or operators (`">=2"`). |
60
+ | `close_tab` | `value` (label, optional) | Close the current (or named) tab and switch back to the last remaining one. Cannot close `default` while other tabs are open. |
61
+
49
62
  ## Assertions
50
63
 
51
64
  | Action | Fields | Description |
52
65
  |--------|--------|-------------|
53
66
  | `assert_text` | `text` | Check entire page body contains text (substring match). |
67
+ | `assert_no_text` | `text` | Check text does NOT appear anywhere in the page body. Opposite of `assert_text`. |
68
+ | `assert_text_in` | `selector`, `text`, `value` (`"exact"` optional) | Check text inside a scoped container. Joins `textContent` from all matching elements. Default: case-insensitive regex; with `value: "exact"`: case-sensitive substring. |
54
69
  | `assert_element_text` | `selector`, `text`, `value` (`"exact"` optional) | Check specific element's `textContent`. Default: substring match. With `value: "exact"`: strict `trim() ===` comparison. |
55
70
  | `assert_url` | `value` | Check current URL. Path-only (`/dashboard`) compares pathname. Full URL does substring match. |
56
71
  | `assert_visible` | `selector` | Element exists and is visible (`display`, `visibility`, `opacity` checks). |
@@ -61,11 +76,14 @@ Complete catalog of all action types supported by @matware/e2e-runner.
61
76
  | `assert_input_value` | `selector`, `value` | Checks `element.value.includes(value)` on input/select/textarea. |
62
77
  | `assert_matches` | `selector`, `value` (regex) | Tests element's `textContent` against `new RegExp(value)`. |
63
78
  | `assert_no_network_errors` | — | Checks accumulated `requestfailed` events during the test. Fails with error details if any exist. |
79
+ | `assert_visual` | `value` (golden image filename), `selector` (optional element scope), `text` (max diff fraction, e.g. `"0.02"`), plus `fullPage`, `maskRegions`, `threshold` | Visual regression against a golden reference image (`goldenDir`, default `{screenshotsDir}/golden`). First run saves the golden and passes; later runs fail if more pixels differ than the max diff (default 2%) and write a diff image. `maskRegions: [{x,y,width,height}]` ignores dynamic areas (timestamps, avatars). |
64
80
 
65
81
  ### Assertion Disambiguation
66
82
 
67
83
  - **`assert_text`** → searches the **entire page body** (substring)
84
+ - **`assert_no_text`** → asserts text is **absent** from the page body (do NOT use `assert_not_visible` with `text` — it requires `selector`)
68
85
  - **`assert_element_text`** → checks a **specific element** (substring, or exact with `value: "exact"`)
86
+ - **`assert_text_in`** → checks text inside a **scoped container** (regex by default)
69
87
  - **`assert_matches`** → checks a specific element against a **regex** pattern
70
88
  - **`assert_input_value`** → reads the `.value` property (for form fields)
71
89
 
@@ -75,7 +93,7 @@ Complete catalog of all action types supported by @matware/e2e-runner.
75
93
  |--------|--------|-------------|
76
94
  | `get_text` | `selector` | Returns `{ value: textContent.trim() }`. Non-assertion — never fails. |
77
95
  | `screenshot` | `value` (filename, optional) | Captures screenshot. Filename gets timestamp suffix for uniqueness. |
78
- | `wait` | `selector` OR `text` OR `value` (ms) | Wait for selector, text on page, or fixed delay. |
96
+ | `wait` | `selector` OR `text` OR `gone` OR `value` (ms) | Prefer **conditions over fixed sleeps**: `{ selector }` waits for it to appear, `{ text }` waits for text to appear, **`{ gone: "<css>" }`** waits until a selector disappears/hides (spinner, closing dialog), `{ gone: true, selector|text }` is the explicit form, `{ value: "<ms>" }` is a fixed delay (last resort). Replacing `wait` sleeps with `gone`/`selector` makes suites faster and less flaky. |
79
97
  | `wait_network_idle` | `value` (idle ms, default 500), `timeout` (max wait ms, default 30000) | Waits for all network requests to complete. Uses Puppeteer's `page.waitForNetworkIdle()`. Useful after SPA page transitions or data loading. |
80
98
  | `evaluate` | `value` (JS code) | Run JavaScript in browser context. See **Strict Evaluate** below. |
81
99
  | `clear_cookies` | `value` (origin, optional) | Clears cookies, localStorage, sessionStorage for origin. |
@@ -107,31 +125,30 @@ Delay between retries: `actionRetryDelay` config (default 500ms).
107
125
  ### React input + autocomplete flow
108
126
  ```json
109
127
  { "type": "focus_autocomplete", "text": "Category" },
110
- { "type": "type_react", "selector": "#category-input", "value": "Electr" },
128
+ { "type": "type_react", "selector": "#category-input", "value": "Electr", "waitAfter": "400" },
111
129
  { "type": "click_option", "text": "Electronics" }
112
130
  ```
113
131
 
114
- ### Regex click (last match)
132
+ ### MUI combobox in one action (open + filter + pick)
115
133
  ```json
116
- { "type": "click_regex", "text": "add to cart", "selector": "button", "value": "last" }
134
+ { "type": "select_combobox", "selector": "[data-cy='specialty'] input", "filter": "cardio", "text": "Cardiología" }
135
+ ```
136
+
137
+ ### Condition waits instead of fixed sleeps (faster, less flaky)
138
+ ```json
139
+ { "type": "click", "text": "Guardar" },
140
+ { "type": "wait", "gone": ".MuiBackdrop-root" },
141
+ { "type": "wait", "selector": "[data-testid='saved-banner']" }
117
142
  ```
118
143
 
119
- ### Form validation assertions
144
+ ### Click a button inside an open dialog (no evaluate needed)
120
145
  ```json
121
- { "type": "assert_attribute", "selector": "input#email", "value": "type=email" },
122
- { "type": "assert_attribute", "selector": "button.submit", "value": "disabled" },
123
- { "type": "assert_class", "selector": ".nav-item:first-child", "value": "active" },
124
- { "type": "assert_input_value", "selector": "#email", "value": "user@example.com" },
125
- { "type": "assert_matches", "selector": ".phone", "value": "\\d{3}-\\d{3}-\\d{4}" },
126
- { "type": "assert_count", "selector": ".table-row", "value": ">3" }
146
+ { "type": "click", "text": "Iniciar encuentro", "scope": "dialog", "last": true }
127
147
  ```
128
148
 
129
- ### Storage operations
149
+ ### Regex click (last match)
130
150
  ```json
131
- { "type": "set_storage", "value": "authToken=eyJhbGciOiJIUzI1NiJ9..." },
132
- { "type": "assert_storage", "value": "authToken" },
133
- { "type": "set_storage", "value": "theme=dark", "selector": "session" },
134
- { "type": "assert_storage", "value": "theme=dark", "selector": "session" }
151
+ { "type": "click_regex", "text": "add to cart", "selector": "button", "value": "last" }
135
152
  ```
136
153
 
137
154
  ### Icon, menu, and contextual clicks
@@ -113,6 +113,29 @@ Module definition (in `e2e/modules/auth-login.json`):
113
113
  }
114
114
  ```
115
115
 
116
+ ### Composing modules (nested `$use` + parameter forwarding)
117
+
118
+ A module can `$use` other modules, and **forward its own params/defaults** into the
119
+ nested call's `params` block. Placeholders in a nested `params` value are resolved
120
+ against the outer module's scope before the inner module runs:
121
+
122
+ ```json
123
+ {
124
+ "$module": "login-and-open",
125
+ "params": {
126
+ "patientId": { "required": true },
127
+ "email": { "required": false, "default": "admin@test.com" }
128
+ },
129
+ "actions": [
130
+ { "$use": "auth-login", "params": { "email": "{{email}}", "password": "secret" } },
131
+ { "$use": "open-patient", "params": { "id": "{{patientId}}" } }
132
+ ]
133
+ }
134
+ ```
135
+
136
+ Cycles are detected and rejected. Action types are validated **after** all `$use`
137
+ references are expanded.
138
+
116
139
  ## Suite Naming & Ordering
117
140
 
118
141
  Files can have numeric prefixes for execution order:
@@ -193,32 +193,8 @@ The `KNOWN_ACTION_TYPES` Set in `src/actions.js` is the single source of truth.
193
193
 
194
194
  ## Screenshot Hashes
195
195
 
196
- Every screenshot captured during a run is assigned a short hash (`ss:a3f2b1c9`) the first 8 hex chars of the SHA-256 of its file path. Hashes are deterministic and computed identically on the server (Node `crypto`) and in the browser (Web Crypto API).
197
-
198
- **Flow**: screenshot saved on disk → `saveRun()` registers hash in SQLite `screenshot_hashes` table → dashboard shows `[ss:XXXXXXXX]` badge (click to copy) → user pastes hash in Claude Code → `e2e_screenshot` MCP tool looks up hash, reads file, returns the image.
199
-
200
- - Hashes are registered inside the `saveRun()` transaction (covers action, error, verification, and baseline screenshots)
201
- - The `ss:` prefix is optional when calling `e2e_screenshot` — stripped during lookup
202
- - Dashboard computes hashes client-side (Web Crypto) for the Live view (before `persistRun()` writes to DB)
203
- - Run detail API (`/api/db/runs/:id`) includes `screenshotHashes` map per test result
204
- - Dashboard endpoint `/api/screenshot-hash/:hash` serves the image by hash
205
- - Dashboard Screenshots view has a **search bar** — type a hash to find and display the screenshot
196
+ Every screenshot gets a short hash like `ss:a3f2b1c9` (deterministic, from its file path). The dashboard shows it as a click-to-copy badge; paste it (the `ss:` prefix is optional) and `e2e_screenshot` returns the image. The dashboard Screenshots view also has a hash search bar.
206
197
 
207
198
  ## Web Dashboard
208
199
 
209
- **`src/dashboard.js`** HTTP server, REST API, WebSocket broadcast, pool polling.
210
- **`templates/dashboard.html`** — SPA, dark theme, vanilla JS, safe DOM (textContent + createEl helper).
211
-
212
- **Features:**
213
- - Live test execution with WebSocket updates
214
- - Run history with inline detail expansion
215
- - Screenshots gallery with hash badges and hash search
216
- - Network request logs with clickable expandable rows (full request/response detail)
217
- - Pool status monitoring
218
- - Multi-project support via project selector
219
- - Variables tab with masked values, inline edit, add, and delete
220
-
221
- **CLI:** `e2e-runner dashboard [--port 8484]`
222
- **MCP tools:** `e2e_dashboard_start`, `e2e_dashboard_stop`
223
-
224
- Config defaults: `dashboardPort: 8484`, `maxHistoryRuns: 100`
200
+ Start with `e2e_dashboard_start` (or `e2e-runner dashboard [--port 8484]`), stop with `e2e_dashboard_stop`. Provides live execution (WebSocket), run history, screenshot gallery + hash search, expandable network logs, pool status, multi-project selector, and a variables tab.