playwright-e2e-mcp 0.4.0 → 0.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/README.md +70 -44
  2. package/package.json +1 -1
  3. package/skill/playwright-e2e-mcp/SKILL.md +2 -2
  4. package/skill/playwright-e2e-mcp/references/setup.md +1 -1
package/README.md CHANGED
@@ -1,15 +1,29 @@
1
- # playwright-e2e-mcp
1
+ <p align="center">
2
+ <img src="docs/logo.png" alt="playwright-e2e-mcp logo: a heartbeat line that ends in a green check" width="140">
3
+ </p>
4
+
5
+ <h1 align="center">playwright-e2e-mcp</h1>
6
+
7
+ <p align="center"><b>The test doctor for AI agents.</b><br>
8
+ Your Playwright test fails. Your AI finds out why, fixes it, and proves the fix works.</p>
9
+
10
+ <p align="center">
2
11
 
3
12
  [![CI](https://github.com/xsignon-secure/E2E/actions/workflows/ci.yml/badge.svg)](https://github.com/xsignon-secure/E2E/actions/workflows/ci.yml) [![release](https://img.shields.io/github/v/release/xsignon-secure/E2E)](https://github.com/xsignon-secure/E2E/releases) [![MCP Registry](https://img.shields.io/badge/MCP%20Registry-io.github.xsignon--secure%2FE2E-2563eb)](https://registry.modelcontextprotocol.io/)
4
- [![M8ven Score](https://m8ven.ai/badge/mcp/trajectiq-ai/e2e)](https://m8ven.ai/mcp/trajectiq-ai/e2e?s=readme)
5
13
 
6
14
  [![Install in Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en/install-mcp?name=playwright-e2e&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsInBsYXl3cmlnaHQtZTJlLW1jcCJdfQ==)
7
15
  [![Install in VS Code](https://img.shields.io/badge/VS_Code-Install_server-0098FF?logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect?url=vscode%3Amcp%2Finstall%3F%257B%2522name%2522%253A%2522playwright-e2e%2522%252C%2522command%2522%253A%2522npx%2522%252C%2522args%2522%253A%255B%2522-y%2522%252C%2522playwright-e2e-mcp%2522%255D%257D)
8
16
  [![Add to Claude Desktop](https://img.shields.io/badge/Claude_Desktop-.mcpb_bundle-D97757)](https://github.com/xsignon-secure/E2E/releases/latest)
9
17
 
10
- An [MCP](https://modelcontextprotocol.io) server that **diagnoses and fixes failing Playwright
11
- tests**. It is the layer that sits next to a browser-driving MCP: Playwright MCP gives an agent
12
- hands in the browser; this server gives it a test runner, a failure analyst and a verified fixer.
18
+ </p>
19
+
20
+ ## What it is
21
+
22
+ An [MCP](https://modelcontextprotocol.io) server and agent skill that gives your AI assistant
23
+ (Claude, Cursor, VS Code Copilot, Gemini, Codex) a full Playwright toolkit: a **test runner**,
24
+ a **failure detective** and a **fixer that checks its own work**.
25
+
26
+ Other tools let an AI click around a browser. This one lets it **keep your test suite green**.
13
27
 
14
28
  ```
15
29
  run-test ──▶ get-failure ──▶ suggest-fix (apply + re-run) ──▶ green
@@ -18,46 +32,59 @@ run-test ──▶ get-failure ──▶ suggest-fix (apply + re-run) ──▶
18
32
  └─ background runs, progress ──▶ analyze-history (flaky / broken / regressed)
19
33
  ```
20
34
 
21
- What it does that a browser-driving MCP or a raw `npx playwright test` does not:
35
+ ## What it does
36
+
37
+ | | You ask your AI… | It does this |
38
+ |---|---|---|
39
+ | 🏃 | "Run my tests" | Runs the suite and reports every failure with its file, line, cause and a fix hint. One root cause breaking 30 tests shows up as **one** problem. |
40
+ | 🔍 | "Why did this fail?" | Reads Playwright's trace: the page at the moment of failure, the broken request, the console error. No guessing. |
41
+ | 🛠️ | "Fix it" | Swaps in a locator proven unique on the page, **re-runs the test**, and undoes the edit if it's still red. If the button is really gone, it tells you it's an app bug instead of hiding it. |
42
+ | 🎲 | "Is this test flaky?" | Re-runs it many ways (repeated, alone, in random order) and names the exact test and seed that cause the flake. |
43
+ | 📈 | "What's been breaking lately?" | Keeps a local run history and separates flaky, newly broken and always-broken tests. |
44
+ | ♿ | "Check this page for accessibility" | Runs axe-core (WCAG 2.2 AA) and returns each problem with a ready-to-paste locator and regression test. |
45
+ | 🔐 | "Test the page after login" | Works behind sign-in with saved sessions, headers and click/fill steps. |
46
+ | 🤖 | "Do it in CI" | A [GitHub Action and `ci` command](#ci-github-action-and-ci-command) post the diagnosis, flaky verdict and fix diffs on your PR. |
47
+
48
+ **Why it stands out**
49
+
50
+ - **Fixes are proven, not guessed.** Every fix is re-run before it's kept.
51
+ - **Honest.** It won't paper over a real bug to make a test pass.
52
+ - **Lean.** In the [benchmark](bench/RESULTS.md), finding and fixing a broken locator took about
53
+ **5,300 tokens** here versus about **7,400** with Playwright MCP plus a shell.
54
+ - **Teaches as it works.** The bundled skill carries a guide to writing tests that stay green
55
+ (locators, waiting, auth, test data, mocking, flake patterns, review checklist).
56
+ - **Ready in one line.** `npx -y playwright-e2e-mcp init` sets up Claude Code, Cursor and VS Code.
57
+
58
+ <details>
59
+ <summary><b>The technical details</b></summary>
22
60
 
23
61
  - **Structured runs.** Stats plus each failure's `file:line`, kind (assertion, timeout,
24
62
  dead dev server, syntax error, crash, full disk…) and a concrete fix hint. Failures that
25
- share one cause are grouped, so 30 tests broken by one dead server read as one problem.
26
- Long suites run in the background with live progress.
63
+ share one cause are grouped. Long suites run in the background with live progress.
27
64
  - **Diagnosis from the trace.** The DOM at the moment of failure, the failed action, the
28
65
  4xx/5xx requests and console errors, read from Playwright's own trace.
29
- - **Fixes that are proven, not guessed.** `suggest-fix` matches a broken locator against the
30
- DOM at failure, swaps in a Playwright locator proven unique on the page
66
+ - **Verified fixes.** `suggest-fix` matches a broken locator against the DOM at failure, swaps
67
+ in a Playwright locator proven unique on the page
31
68
  (`getByRole('button', { name: 'Proceed to checkout' })`), re-runs the test and reverts the
32
69
  edit if it is still red. When nothing on the page plausibly matches, it says the element is
33
- missing and does not patch the test to hide a real bug.
70
+ missing and does not patch the test.
34
71
  - **Flakiness with evidence.** `diagnose-flaky` repeats failing tests in one process and
35
- gives a verdict per test; with `isolate: true` it re-runs them alone to tell a shared-state
36
- bug that only bites in parallel from a race inside the test, and with `order: true` it runs
37
- the files in random orders to catch a test that only passes when another one ran first,
38
- naming that test and the seed that reproduces it. Every run is recorded locally;
39
- `analyze-history` separates tests that flip (flaky) from ones that started failing
40
- (regressed) or always fail (broken), and groups failures that share one cause.
41
- - **In CI too.** A [GitHub Action and `ci` command](#ci-github-action-and-ci-command) post the
42
- same diagnosis, flaky verdict and fix diffs to the job summary and the PR.
72
+ gives a verdict per test; `isolate: true` re-runs them alone to tell a shared-state bug from
73
+ a race inside the test, and `order: true` runs files in random orders to catch an order
74
+ dependency, naming the test and the seed that reproduces it. `analyze-history` separates
75
+ flaky, regressed and broken tests from the local run record.
43
76
  - **Accessibility.** `audit-accessibility` runs axe-core against WCAG 2.2 AA on any page,
44
- signed in or after steps, and returns each violation with the offending elements as verified
45
- locators and a regression check to paste into the spec.
77
+ signed in or after steps, with offending elements as verified locators.
46
78
  - **Pages behind a login.** Every page tool accepts a Playwright `storageState`, extra
47
79
  headers, and click/fill steps to reach a modal or a later step.
48
- - **An agent skill that teaches the craft.** The bundled skill drives the tools and carries a
49
- guide to writing tests that stay green (locators, waiting, auth, data isolation, mocking,
50
- flake patterns, review checklist).
51
- - **Small on the wire.** In the [benchmark](bench/RESULTS.md), diagnosing and fixing one
52
- drifted locator cost about 5,300 tokens here against about 7,400 with Playwright MCP plus a
53
- shell, and the fix was applied and verified without the agent editing anything.
80
+
81
+ </details>
54
82
 
55
83
  ### What it does not do
56
84
 
57
- It does not drive an exploratory browsing session step by step, and it does not replace
58
- Playwright's own Test Agents (planner, generator, healer) inside VS Code. Use Playwright MCP
59
- for interactive browsing and this server for running, diagnosing and fixing the suite; they
60
- work side by side in the same client.
85
+ It doesn't drive an exploratory browsing session step by step, and it doesn't replace
86
+ Playwright's own Test Agents inside VS Code. Use Playwright MCP for interactive browsing and
87
+ this server for running, diagnosing and fixing the suite. They work side by side.
61
88
 
62
89
  ## Demo
63
90
 
@@ -104,7 +131,7 @@ npx -y playwright-e2e-mcp init # --client all | cursor,vscode … ·
104
131
  | **Skill only, any agent** | `npx skills add xsignon-secure/E2E` |
105
132
  | **npm (canonical, fastest)** | `npx -y playwright-e2e-mcp` |
106
133
  | **MCP Registry** (registry-aware clients discover it automatically) | `io.github.xsignon-secure/E2E` — [listing](https://registry.modelcontextprotocol.io/) |
107
- | **Any client, no npm account needed** | `npx -y github:xsignon-secure/E2E#v0.4.0` (pin a release tag) |
134
+ | **Any client, no npm account needed** | `npx -y github:xsignon-secure/E2E#v0.4.1` (pin a release tag) |
108
135
  | **Claude Desktop, zero Node setup** | double-click the [`.mcpb` extension](https://github.com/xsignon-secure/E2E/releases) |
109
136
  | **Remote-only clients (ChatGPT connectors)** | `https://playwright-e2e-mcp.vercel.app/api/mcp` |
110
137
 
@@ -475,8 +502,8 @@ builds `dist/` automatically on install). Pin a release tag: an unpinned
475
502
  `github:xsignon-secure/E2E` runs whatever is on the default branch at that moment.
476
503
 
477
504
  ```bash
478
- npx -y github:xsignon-secure/E2E#v0.4.0
479
- npm install -D github:xsignon-secure/E2E#v0.4.0 @playwright/test # or as a project dependency
505
+ npx -y github:xsignon-secure/E2E#v0.4.1
506
+ npm install -D github:xsignon-secure/E2E#v0.4.1 @playwright/test # or as a project dependency
480
507
  npx playwright install chromium
481
508
  ```
482
509
 
@@ -484,7 +511,7 @@ Or grab the packaged tarball from the repo's **GitHub Releases** page and instal
484
511
  it locally:
485
512
 
486
513
  ```bash
487
- npm install -D https://github.com/xsignon-secure/E2E/releases/download/v0.4.0/playwright-e2e-mcp-0.4.0.tgz
514
+ npm install -D https://github.com/xsignon-secure/E2E/releases/download/v0.4.1/playwright-e2e-mcp-0.4.1.tgz
488
515
  ```
489
516
 
490
517
  Listed in the **official [MCP Registry](https://registry.modelcontextprotocol.io/)** as
@@ -500,7 +527,7 @@ Listed in the **official [MCP Registry](https://registry.modelcontextprotocol.io
500
527
  "mcpServers": {
501
528
  "playwright-e2e": {
502
529
  "command": "npx",
503
- "args": ["-y", "github:xsignon-secure/E2E#v0.4.0"],
530
+ "args": ["-y", "github:xsignon-secure/E2E#v0.4.1"],
504
531
  "env": { "PW_MCP_PROJECT_ROOT": "/absolute/path/to/your/project" }
505
532
  }
506
533
  }
@@ -514,8 +541,8 @@ when the client launches it somewhere else (e.g. your home directory).
514
541
  **Codex / VS Code / Copilot CLIs:**
515
542
 
516
543
  ```bash
517
- codex mcp add playwright-e2e -- npx -y github:xsignon-secure/E2E#v0.4.0
518
- code --add-mcp '{"name":"playwright-e2e","command":"npx","args":["-y","github:xsignon-secure/E2E#v0.4.0"]}'
544
+ codex mcp add playwright-e2e -- npx -y github:xsignon-secure/E2E#v0.4.1
545
+ code --add-mcp '{"name":"playwright-e2e","command":"npx","args":["-y","github:xsignon-secure/E2E#v0.4.1"]}'
519
546
  ```
520
547
 
521
548
  Codex's defaults fight this server: the first launch clones the repo and runs
@@ -526,7 +553,7 @@ Codex's defaults fight this server: the first launch clones the repo and runs
526
553
  ```toml
527
554
  [mcp_servers.playwright-e2e]
528
555
  command = "npx"
529
- args = ["-y", "github:xsignon-secure/E2E#v0.4.0"]
556
+ args = ["-y", "github:xsignon-secure/E2E#v0.4.1"]
530
557
  startup_timeout_sec = 60
531
558
  tool_timeout_sec = 600
532
559
  ```
@@ -541,7 +568,7 @@ into `PW_MCP_PROJECT_ROOT`, so the tools point at a real project from the first
541
568
  **Claude Code:**
542
569
 
543
570
  ```bash
544
- claude mcp add playwright-e2e -- npx -y github:xsignon-secure/E2E#v0.4.0
571
+ claude mcp add playwright-e2e -- npx -y github:xsignon-secure/E2E#v0.4.1
545
572
  ```
546
573
 
547
574
  **Gemini CLI / Qwen Code:** paste the `mcpServers` block above into
@@ -676,7 +703,7 @@ steps:
676
703
  - uses: actions/setup-node@v6
677
704
  with: { node-version: 24 }
678
705
  - run: npm ci && npx playwright install --with-deps chromium
679
- - uses: xsignon-secure/E2E@v0.4.0
706
+ - uses: xsignon-secure/E2E@v0.4.1
680
707
  with:
681
708
  flaky-runs: 3 # 0 skips the flaky verdict
682
709
  suggest: 2 # suggest-fix diffs for the first 2 failures
@@ -974,8 +1001,8 @@ job requests for itself; if it is refused, the run names the setting to change.
974
1001
  ### Verifying a published release
975
1002
 
976
1003
  ```bash
977
- git checkout v0.4.0 && npm ci && npm run mcpb
978
- npm run release:verify -- v0.4.0
1004
+ git checkout v0.4.1 && npm ci && npm run mcpb
1005
+ npm run release:verify -- v0.4.1
979
1006
  ```
980
1007
 
981
1008
  The release notes promise that the tag rebuilds the published bundle byte for
@@ -1065,4 +1092,3 @@ pipeline in one shot.
1065
1092
  ## License
1066
1093
 
1067
1094
  MIT
1068
- [![M8ven Score](https://m8ven.ai/badge/mcp/trajectiq-ai-e2e-fzfqvz?v=1bc57c32547f4c84e29d89f2b9a1f82c)](https://m8ven.ai/mcp/trajectiq-ai-e2e-fzfqvz?s=readme)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "playwright-e2e-mcp",
3
- "version": "0.4.0",
3
+ "version": "0.4.1",
4
4
  "description": "MCP server and agent skill that run, diagnose and fix failing Playwright end-to-end tests: root-cause grouping, trace analysis, verified locator fixes, flaky and test-order verdicts, WCAG accessibility audits",
5
5
  "author": "xsignon-secure (https://github.com/xsignon-secure)",
6
6
  "mcpName": "io.github.xsignon-secure/E2E",
package/skill/playwright-e2e-mcp/SKILL.md CHANGED
@@ -13,7 +13,7 @@ and concrete next steps. It can also propose and apply a verified fix for a brok
13
13
  locator, inspect pages behind a login, run long suites in the background and keep
14
14
  a local history of every run.
15
15
 
16
- This skill is written for playwright-e2e-mcp 0.4.0 and works with 0.2.0 or
16
+ This skill is written for playwright-e2e-mcp 0.4.1 and works with 0.2.0 or
17
17
  newer. If `suggest-fix`, `get-run-status` or `analyze-history` is missing, the
18
18
  server is older than 0.2.0: use the steps that do not need those tools and tell
19
19
  the user to update it. `diagnose-flaky`'s per-test table and `isolate` option
@@ -179,7 +179,7 @@ pass/fail timeline.
179
179
  file it writes, then `run-test` it. Pass `write: false` to preview without
180
180
  writing.
181
181
  - **Set up CI:** when the user wants the same diagnosis on every pull request,
182
- point them to the GitHub Action (`uses: xsignon-secure/E2E@v0.4.0`) or the
182
+ point them to the GitHub Action (`uses: xsignon-secure/E2E@v0.4.1`) or the
183
183
  `npx -y playwright-e2e-mcp ci` command in `references/setup.md`. It writes the
184
184
  run, flaky verdicts and fix diffs to the job summary and one PR comment.
185
185
  - **Accessibility:** `audit-accessibility` with `url` runs axe-core against
package/skill/playwright-e2e-mcp/references/setup.md CHANGED
@@ -112,7 +112,7 @@ Exit code 0 means passed, 1 tests failed, 2 the run itself failed. On GitHub
112
112
  Actions it also writes to the job summary. As an Action:
113
113
 
114
114
  ```yaml
115
- - uses: xsignon-secure/E2E@v0.4.0
115
+ - uses: xsignon-secure/E2E@v0.4.1
116
116
  with:
117
117
  flaky-runs: 3
118
118
  suggest: 2