playwright-e2e-mcp 0.4.2 → 0.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +238 -924
- package/dist/http.d.ts +2 -1
- package/dist/http.js +2 -1
- package/dist/http.js.map +1 -1
- package/dist/server.d.ts +2 -2
- package/dist/server.js +31 -4
- package/dist/server.js.map +1 -1
- package/dist/tools/analyze-history.d.ts +12 -1
- package/dist/tools/analyze-history.js +89 -5
- package/dist/tools/analyze-history.js.map +1 -1
- package/dist/tools/audit-accessibility.d.ts +85 -0
- package/dist/tools/audit-accessibility.js +4 -1
- package/dist/tools/audit-accessibility.js.map +1 -1
- package/dist/tools/audit-performance.d.ts +333 -0
- package/dist/tools/audit-performance.js +377 -0
- package/dist/tools/audit-performance.js.map +1 -0
- package/dist/tools/compare-visual-state.d.ts +84 -0
- package/dist/tools/generate-e2e-test.d.ts +90 -0
- package/dist/tools/generate-e2e-test.js +223 -34
- package/dist/tools/generate-e2e-test.js.map +1 -1
- package/dist/tools/get-failure.js +2 -1
- package/dist/tools/get-failure.js.map +1 -1
- package/dist/tools/inspect-page.d.ts +94 -0
- package/dist/tools/inspect-page.js +42 -5
- package/dist/tools/inspect-page.js.map +1 -1
- package/dist/tools/load-report.d.ts +48 -0
- package/dist/tools/load-report.js +167 -0
- package/dist/tools/load-report.js.map +1 -0
- package/dist/tools/probe-script.d.ts +7 -0
- package/dist/tools/probe-script.js +266 -7
- package/dist/tools/probe-script.js.map +1 -1
- package/dist/tools/shared.d.ts +137 -2
- package/dist/tools/shared.js +176 -9
- package/dist/tools/shared.js.map +1 -1
- package/dist/tools/suggest-fix.d.ts +35 -1
- package/dist/tools/suggest-fix.js +197 -17
- package/dist/tools/suggest-fix.js.map +1 -1
- package/dist/tools/validate-selector.d.ts +84 -0
- package/dist/tools/validate-selector.js +7 -1
- package/dist/tools/validate-selector.js.map +1 -1
- package/dist/types/index.d.ts +28 -0
- package/dist/utils/project-detector.js +2 -2
- package/dist/utils/project-detector.js.map +1 -1
- package/dist/utils/report-parser.d.ts +28 -3
- package/dist/utils/report-parser.js +77 -11
- package/dist/utils/report-parser.js.map +1 -1
- package/dist/utils/run-history.d.ts +66 -3
- package/dist/utils/run-history.js +149 -3
- package/dist/utils/run-history.js.map +1 -1
- package/dist/utils/serverless-chromium.d.ts +32 -0
- package/dist/utils/serverless-chromium.js +64 -0
- package/dist/utils/serverless-chromium.js.map +1 -0
- package/package.json +3 -2
- package/skill/playwright-e2e-mcp/SKILL.md +71 -15
- package/skill/playwright-e2e-mcp/references/setup.md +1 -1
- package/skill/playwright-e2e-mcp/references/tools.md +67 -6
package/README.md
CHANGED
|
@@ -9,7 +9,7 @@ Your Playwright test fails. Your AI finds out why, fixes it, and proves the fix
|
|
|
9
9
|
|
|
10
10
|
<p align="center">
|
|
11
11
|
|
|
12
|
-
[](https://www.npmjs.com/package/playwright-e2e-mcp) [](https://www.npmjs.com/package/playwright-e2e-mcp) [](https://github.com/xsignon-secure/E2E/actions/workflows/ci.yml) [](https://registry.modelcontextprotocol.io/) [](LICENSE)
|
|
13
13
|
|
|
14
14
|
[](https://cursor.com/en/install-mcp?name=playwright-e2e&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsInBsYXl3cmlnaHQtZTJlLW1jcCJdfQ==)
|
|
15
15
|
[](https://insiders.vscode.dev/redirect?url=vscode%3Amcp%2Finstall%3F%257B%2522name%2522%253A%2522playwright-e2e%2522%252C%2522command%2522%253A%2522npx%2522%252C%2522args%2522%253A%255B%2522-y%2522%252C%2522playwright-e2e-mcp%2522%255D%257D)
|
|
@@ -17,22 +17,49 @@ Your Playwright test fails. Your AI finds out why, fixes it, and proves the fix
|
|
|
17
17
|
|
|
18
18
|
</p>
|
|
19
19
|
|
|
20
|
-
|
|
20
|
+
<p align="center">
|
|
21
|
+
<a href="#getting-started">Getting started</a> ·
|
|
22
|
+
<a href="#tools">Tools</a> ·
|
|
23
|
+
<a href="docs/tools.md">Tool reference</a> ·
|
|
24
|
+
<a href="#ci-github-action-and-ci-command">GitHub Action</a> ·
|
|
25
|
+
<a href="#configuration">Configuration</a> ·
|
|
26
|
+
<a href="https://github.com/xsignon-secure/E2E/releases">Changelog</a> ·
|
|
27
|
+
<a href="CONTRIBUTING.md">Contributing</a>
|
|
28
|
+
</p>
|
|
21
29
|
|
|
22
30
|
An [MCP](https://modelcontextprotocol.io) server and agent skill that gives your AI assistant
|
|
23
31
|
(Claude, Cursor, VS Code Copilot, Gemini, Codex) a full Playwright toolkit: a **test runner**,
|
|
24
|
-
a **failure detective** and a **fixer that checks its own work**.
|
|
32
|
+
a **failure detective** and a **fixer that checks its own work**. Other tools let an AI click
|
|
33
|
+
around a browser. This one lets it **keep your test suite green**.
|
|
34
|
+
|
|
35
|
+
## ❌ Without it / ✅ With it
|
|
36
|
+
|
|
37
|
+
**Without it**, your agent sees `Timeout 5000ms exceeded waiting for getByTestId('checkout')`,
|
|
38
|
+
guesses a new selector, edits the test and hopes. Or it "fixes" the test around a real bug.
|
|
25
39
|
|
|
26
|
-
|
|
40
|
+
**With it**, your agent reads the page as it was when the test failed, swaps in a locator
|
|
41
|
+
proven unique on that page, re-runs the test, and keeps the edit only if it passes. When the
|
|
42
|
+
button is really gone, it tells you it's an app bug.
|
|
27
43
|
|
|
28
44
|
```
|
|
29
|
-
run-test
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
45
|
+
run-test ──────┐
|
|
46
|
+
load-report ───┴─▶ get-failure ──▶ suggest-fix (apply + re-run) ──▶ green
|
|
47
|
+
(CI job report) │ └─ refuses when the element is really gone (app bug)
|
|
48
|
+
└─ DOM at failure, failed requests, console errors
|
|
49
|
+
every run ──▶ analyze-history (flaky / broken / slow / which commit broke it)
|
|
33
50
|
```
|
|
34
51
|
|
|
35
|
-
##
|
|
52
|
+
## Key features
|
|
53
|
+
|
|
54
|
+
- **Fixes are proven, not guessed.** Every fix is re-run before it's kept, and undone if the test is still red.
|
|
55
|
+
- **Honest.** It won't paper over a real bug to make a test pass.
|
|
56
|
+
- **Works on CI failures too.** Load a CI job's report and traces and get the same diagnosis and fixes.
|
|
57
|
+
- **Flakiness with evidence.** Names the flaky test, whether it's a race, shared state or test order, and the seed that reproduces it.
|
|
58
|
+
- **Lean.** Finding and fixing a broken locator took about **5,300 tokens** versus about **7,400**
|
|
59
|
+
with Playwright MCP plus a shell ([benchmark](#benchmark)).
|
|
60
|
+
- **Free and open source.** MIT, no account, no API key, no telemetry, no paid tier.
|
|
61
|
+
|
|
62
|
+
## What you can ask
|
|
36
63
|
|
|
37
64
|
| | You ask your AI… | It does this |
|
|
38
65
|
|---|---|---|
|
|
@@ -40,672 +67,256 @@ run-test ──▶ get-failure ──▶ suggest-fix (apply + re-run) ──▶
|
|
|
40
67
|
| 🔍 | "Why did this fail?" | Reads Playwright's trace: the page at the moment of failure, the broken request, the console error. No guessing. |
|
|
41
68
|
| 🛠️ | "Fix it" | Swaps in a locator proven unique on the page, **re-runs the test**, and undoes the edit if it's still red. If the button is really gone, it tells you it's an app bug instead of hiding it. |
|
|
42
69
|
| 🎲 | "Is this test flaky?" | Re-runs it many ways (repeated, alone, in random order) and names the exact test and seed that cause the flake. |
|
|
43
|
-
| 📈 | "What's been breaking lately?" | Keeps a local run history and separates flaky, newly broken and always-broken tests
|
|
70
|
+
| 📈 | "What's been breaking lately?" | Keeps a local run history and separates flaky, newly broken and always-broken tests; shows the slowest tests, stability by day and **the commit where each test went bad**. |
|
|
71
|
+
| ☁️ | "Why did CI fail?" | Loads the CI job's report (or merges sharded blob reports), maps its paths onto your checkout and finds the traces in the downloaded artifact, so the same diagnosis and fixes work on CI failures. |
|
|
72
|
+
| ✍️ | "Write a test for this flow" | Walks the steps on the live page first, writes a spec that replays them with assertions on where the flow landed (URL and accessibility snapshot), and runs it once to prove it passes. |
|
|
44
73
|
| ♿ | "Check this page for accessibility" | Runs axe-core (WCAG 2.2 AA) and returns each problem with a ready-to-paste locator and regression test. |
|
|
45
|
-
|
|
|
74
|
+
| ⚡ | "Why is this page slow?" | Measures Web Vitals (LCP, CLS, INP, TBT) on desktop or a throttled phone, names the slow element and the elements that jump, and hands back a budget test. |
|
|
75
|
+
| 🧪 | "Show me the error state" | Mocks API responses (500s, empty lists, slow calls) so the AI can inspect, audit or measure states that are hard to reach. |
|
|
76
|
+
| 🔐 | "Test the page after login" | Works behind sign-in with saved sessions, headers and click/fill steps, in Chromium, Firefox or WebKit, and can record or replay network traffic from a HAR file. |
|
|
46
77
|
| 🤖 | "Do it in CI" | A [GitHub Action and `ci` command](#ci-github-action-and-ci-command) post the diagnosis, flaky verdict and fix diffs on your PR. |
|
|
47
78
|
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
- **Fixes are proven, not guessed.** Every fix is re-run before it's kept.
|
|
51
|
-
- **Honest.** It won't paper over a real bug to make a test pass.
|
|
52
|
-
- **Lean.** In the [benchmark](bench/RESULTS.md), finding and fixing a broken locator took about
|
|
53
|
-
**5,300 tokens** here versus about **7,400** with Playwright MCP plus a shell.
|
|
54
|
-
- **Teaches as it works.** The bundled skill carries a guide to writing tests that stay green
|
|
55
|
-
(locators, waiting, auth, test data, mocking, flake patterns, review checklist).
|
|
56
|
-
- **Ready in one line.** `npx -y playwright-e2e-mcp init` sets up Claude Code, Cursor and VS Code.
|
|
57
|
-
|
|
58
|
-
### Free and open source, for good
|
|
59
|
-
|
|
60
|
-
- **MIT licensed.** Use it, change it, ship it in commercial work; no fees, no paid tier, no feature held back.
|
|
61
|
-
- **No account, no API key, no sign-up.** `npx -y playwright-e2e-mcp init` is the whole setup.
|
|
62
|
-
- **No telemetry.** It never phones home; your tests, traces and run history stay on your machine.
|
|
63
|
-
- **Free dependencies only:** the MCP SDK (Apache-2.0), zod (MIT), axe-core (MPL-2.0) and the Playwright your project already has (Apache-2.0).
|
|
64
|
-
- **Free to host yourself**, too: the HTTP bridge deploys to a free Vercel account (see [Hosted endpoint](#hosted-endpoint-chatgpt--remote-clients)).
|
|
65
|
-
|
|
66
|
-
<details>
|
|
67
|
-
<summary><b>The technical details</b></summary>
|
|
68
|
-
|
|
69
|
-
- **Structured runs.** Stats plus each failure's `file:line`, kind (assertion, timeout,
|
|
70
|
-
dead dev server, syntax error, crash, full disk…) and a concrete fix hint. Failures that
|
|
71
|
-
share one cause are grouped. Long suites run in the background with live progress.
|
|
72
|
-
- **Diagnosis from the trace.** The DOM at the moment of failure, the failed action, the
|
|
73
|
-
4xx/5xx requests and console errors, read from Playwright's own trace.
|
|
74
|
-
- **Verified fixes.** `suggest-fix` matches a broken locator against the DOM at failure, swaps
|
|
75
|
-
in a Playwright locator proven unique on the page
|
|
76
|
-
(`getByRole('button', { name: 'Proceed to checkout' })`), re-runs the test and reverts the
|
|
77
|
-
edit if it is still red. When nothing on the page plausibly matches, it says the element is
|
|
78
|
-
missing and does not patch the test.
|
|
79
|
-
- **Flakiness with evidence.** `diagnose-flaky` repeats failing tests in one process and
|
|
80
|
-
gives a verdict per test; `isolate: true` re-runs them alone to tell a shared-state bug from
|
|
81
|
-
a race inside the test, and `order: true` runs files in random orders to catch an order
|
|
82
|
-
dependency, naming the test and the seed that reproduces it. `analyze-history` separates
|
|
83
|
-
flaky, regressed and broken tests from the local run record.
|
|
84
|
-
- **Accessibility.** `audit-accessibility` runs axe-core against WCAG 2.2 AA on any page,
|
|
85
|
-
signed in or after steps, with offending elements as verified locators.
|
|
86
|
-
- **Pages behind a login.** Every page tool accepts a Playwright `storageState`, extra
|
|
87
|
-
headers, and click/fill steps to reach a modal or a later step.
|
|
88
|
-
|
|
89
|
-
</details>
|
|
90
|
-
|
|
91
|
-
### What it does not do
|
|
92
|
-
|
|
93
|
-
It doesn't drive an exploratory browsing session step by step, and it doesn't replace
|
|
94
|
-
Playwright's own Test Agents inside VS Code. Use Playwright MCP for interactive browsing and
|
|
95
|
-
this server for running, diagnosing and fixing the suite. They work side by side.
|
|
96
|
-
|
|
97
|
-
## Demo
|
|
98
|
-
|
|
99
|
-
**Hosted endpoint** — what an `initialize` + `tools/list` round-trip against
|
|
100
|
-
`https://playwright-e2e-mcp.vercel.app/api/mcp` returns for a client that sends the
|
|
101
|
-
bearer token (without one, only `list-tests` and `get-failure` are listed):
|
|
102
|
-
|
|
103
|
-

|
|
104
|
-
|
|
105
|
-
**A real test run** — `run-test` served over stdio by `npx -y playwright-e2e-mcp`
|
|
106
|
-
against the bundled `examples/sample-test.spec.ts` (actual output, unedited):
|
|
107
|
-
|
|
108
|
-

|
|
109
|
-
|
|
110
|
-
Images are rendered with `node scripts/gen-demo-images.mjs`: the run-test card is real
|
|
111
|
-
captured output; the endpoint card is an illustration of the authenticated listing.
|
|
112
|
-
|
|
113
|
-
> If this saves you a debugging session, a ⭐ on the repo helps other Playwright users find it.
|
|
114
|
-
|
|
115
|
-
## Install
|
|
79
|
+
## Requirements
|
|
116
80
|
|
|
117
|
-
|
|
118
|
-
|
|
81
|
+
- Node.js 20 or newer
|
|
82
|
+
- A project that uses `@playwright/test`, with a browser installed (`npx playwright install chromium`)
|
|
83
|
+
- Claude Code, Claude Desktop, Cursor, VS Code, Windsurf, Codex, Gemini CLI or any other MCP client
|
|
119
84
|
|
|
120
|
-
|
|
121
|
-
other coding agent:
|
|
85
|
+
## Getting started
|
|
122
86
|
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
or run the installer yourself from the project root. It merges the server into each
|
|
127
|
-
client's project config it finds (Claude Code, Cursor, VS Code, Gemini CLI, Codex) and
|
|
128
|
-
installs the agent skill, leaving your other servers alone:
|
|
87
|
+
**Fastest:** run this in your project root. It adds the server to every client config it finds
|
|
88
|
+
(Claude Code, Cursor, VS Code, Gemini CLI, Codex) and installs the agent skill, leaving your
|
|
89
|
+
other servers alone:
|
|
129
90
|
|
|
130
91
|
```bash
|
|
131
92
|
npx -y playwright-e2e-mcp init # --client all | cursor,vscode … · --dry-run
|
|
132
93
|
```
|
|
133
94
|
|
|
134
|
-
|
|
135
|
-
| --- | --- |
|
|
136
|
-
| **One command, any project** | `npx -y playwright-e2e-mcp init` |
|
|
137
|
-
| **Claude Code plugin (server + skill)** | `/plugin marketplace add xsignon-secure/E2E` then `/plugin install playwright-e2e-mcp@xsignon-secure` |
|
|
138
|
-
| **Gemini CLI extension (server + skill)** | `gemini extensions install https://github.com/xsignon-secure/E2E` |
|
|
139
|
-
| **Skill only, any agent** | `npx skills add xsignon-secure/E2E` |
|
|
140
|
-
| **npm (canonical, fastest)** | `npx -y playwright-e2e-mcp` |
|
|
141
|
-
| **MCP Registry** (registry-aware clients discover it automatically) | `io.github.xsignon-secure/E2E` — [listing](https://registry.modelcontextprotocol.io/) |
|
|
142
|
-
| **Any client, no npm account needed** | `npx -y github:xsignon-secure/E2E#v0.4.2` (pin a release tag) |
|
|
143
|
-
| **Claude Desktop, zero Node setup** | double-click the [`.mcpb` extension](https://github.com/xsignon-secure/E2E/releases) |
|
|
144
|
-
| **Remote-only clients (ChatGPT connectors)** | `https://playwright-e2e-mcp.vercel.app/api/mcp` |
|
|
145
|
-
|
|
146
|
-
One line for the common clients:
|
|
147
|
-
|
|
148
|
-
```bash
|
|
149
|
-
claude mcp add playwright-e2e -- npx -y playwright-e2e-mcp
|
|
150
|
-
code --add-mcp '{"name":"playwright-e2e","command":"npx","args":["-y","playwright-e2e-mcp"]}'
|
|
151
|
-
```
|
|
152
|
-
|
|
153
|
-
Only need the fix loop? `--tools=core` (or `PW_MCP_TOOLS=core`) exposes six tools and
|
|
154
|
-
halves the tool-definition context; see [Configuration](#configuration).
|
|
155
|
-
|
|
156
|
-
Details and per-client config: [Installation](#installation) ·
|
|
157
|
-
[MCP client configuration](#mcp-client-configuration).
|
|
95
|
+
**Or let your agent do it.** Paste this into any coding agent:
|
|
158
96
|
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
## Tools
|
|
162
|
-
|
|
163
|
-
| Tool | Purpose |
|
|
164
|
-
| --- | --- |
|
|
165
|
-
| `run-test` | Run Playwright tests and return stats, failures, diagnostics and hints; reports progress, and `background: true` returns a run id for long suites |
|
|
166
|
-
| `get-run-status` | Progress or final result of a background run; can wait for it or cancel it |
|
|
167
|
-
| `get-failure` | Deep analysis of one failure: stack, expected/actual, **DOM snapshot at failure (from the Playwright trace)**, failed requests, console errors, next steps |
|
|
168
|
-
| `suggest-fix` | Patch a failure: a drifted locator is replaced with one **proven unique** on the DOM at failure, changed copy gets the new expected text; `apply: true` writes it, re-runs the test and reverts if still red |
|
|
169
|
-
| `inspect-page` | Open a URL headlessly (optionally signed in, after steps) and return elements with **verified Playwright locators**, CSS selectors, visibility, text and console output |
|
|
170
|
-
| `list-tests` | List available tests (`file`, `line`, full title, projects) with filtering |
|
|
171
|
-
| `validate-selector` | Check a CSS selector, Playwright selector or **locator expression** against a live page: validity, match count, and a sturdier verified locator for each match |
|
|
172
|
-
| `generate-e2e-test` | Scaffold a Playwright test from a description using the project's **real** selectors and verified locators from the live page |
|
|
173
|
-
| `compare-visual-state` | Visual regression: screenshot before/after a change and report *what* moved and how colors shifted |
|
|
174
|
-
| `audit-accessibility` | Run axe-core (WCAG 2.2 AA by default) on a page, signed in or after steps, and return violations worst first with **verified locators** for the offending elements and a regression check |
|
|
175
|
-
| `diagnose-flaky` | Run a failing test 2–10 times **with retries disabled** and return an evidence verdict: `CONSISTENTLY FAILING`, `FLAKY` or `NOT REPRODUCING`; tells parallelism and **test-order** bugs from races |
|
|
176
|
-
| `analyze-history` | Rank flaky, regressed and broken tests from the local run history, group failures by cause, or show one test's timeline |
|
|
177
|
-
|
|
178
|
-
Two **prompts** package the main workflows for clients that show them as slash commands:
|
|
179
|
-
`fix-failing-test` (optional `test`) and `triage-flaky-tests`.
|
|
97
|
+
> Install the playwright-e2e-mcp server and skill by following
|
|
98
|
+
> https://raw.githubusercontent.com/xsignon-secure/E2E/main/llms-install.md
|
|
180
99
|
|
|
181
|
-
|
|
100
|
+
**Standard config** works in most clients:
|
|
182
101
|
|
|
183
|
-
|
|
184
|
-
|
|
102
|
+
```json
|
|
103
|
+
{
|
|
104
|
+
"mcpServers": {
|
|
105
|
+
"playwright-e2e": {
|
|
106
|
+
"command": "npx",
|
|
107
|
+
"args": ["-y", "playwright-e2e-mcp"]
|
|
108
|
+
}
|
|
109
|
+
}
|
|
110
|
+
}
|
|
111
|
+
```
|
|
185
112
|
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
| `storageState` | string | Path inside the project to a Playwright storageState JSON (cookies + localStorage), e.g. `playwright/.auth/user.json` from your setup project. Defaults to `PW_MCP_STORAGE_STATE` |
|
|
189
|
-
| `headers` | object | Extra HTTP headers for every request, e.g. `{ "Authorization": "Bearer …" }` |
|
|
190
|
-
| `actions` | array | Up to 20 steps run after load: `{ "type": "click" \| "dblclick" \| "hover" \| "fill" \| "press" \| "check" \| "uncheck" \| "select" \| "wait" \| "goto", "locator"?, "value"?, "url"?, "ms"? }`. Locators use Playwright syntax, e.g. `getByRole('button', { name: 'Open settings' })` |
|
|
191
|
-
| `viewport` | object | `{ "width": 390, "height": 844 }` (default 1280×720) |
|
|
113
|
+
The server uses its working directory as the project root. If your client starts it somewhere
|
|
114
|
+
else (such as your home directory), add `"env": { "PW_MCP_PROJECT_ROOT": "/path/to/your/project" }`.
|
|
192
115
|
|
|
193
|
-
|
|
194
|
-
|
|
116
|
+
<details>
|
|
117
|
+
<summary><b>Claude Code</b></summary>
|
|
195
118
|
|
|
196
|
-
|
|
119
|
+
Server and skill, as a plugin:
|
|
197
120
|
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
| `testFiles` | string[] | Files/directories relative to the root; `file:line` supported. Omit to run everything |
|
|
202
|
-
| `grep` | string | Only run tests whose title matches this regex |
|
|
203
|
-
| `browser` | `chromium` \| `firefox` \| `webkit` | Playwright project to run (matched against config project names) |
|
|
204
|
-
| `headed` | boolean | Visible browser window |
|
|
205
|
-
| `timeoutMs` | number | Hard wall-clock limit for the run (default `120000`); the whole process tree is killed past it and **partial results are returned** |
|
|
206
|
-
| `testTimeoutMs` | number | Per-test timeout passed to Playwright |
|
|
207
|
-
| `workers` / `retries` | number | Passed through to Playwright |
|
|
208
|
-
| `config` | string | `playwright.config` path **or 1-based index** when the project has several |
|
|
209
|
-
| `retryOnFailure` | boolean | Auto-retry failures **once** before reporting them (default `true`; ignored when `retries` is set) |
|
|
210
|
-
| `lastFailed` | boolean | Only re-run tests that failed in the previous run (Playwright `--last-failed`) — the fast fix → re-run loop |
|
|
211
|
-
| `args` | string[] | Extra Playwright flags from an allowlist (`--repeat-each=N`, `--max-failures=N`, `--update-snapshots`, `--shard=1/3`, `--trace=on`, …); values go after `=` and are checked, and flags that take a path, such as `--config` or `--output`, are rejected |
|
|
212
|
-
| `background` | boolean | Return a run id at once; poll `get-run-status` for progress and the result |
|
|
213
|
-
|
|
214
|
-
Progress: clients that send a progress token get a notification as each test finishes
|
|
215
|
-
(`3/12 tests done, 1 failed`). Every run is appended to the local history used by
|
|
216
|
-
`analyze-history`.
|
|
217
|
-
|
|
218
|
-
Flakiness handling: by default the server injects `--retries=1` (unless the config
|
|
219
|
-
already sets `retries`), so a test that passes on the retry is reported as **flaky**,
|
|
220
|
-
not failed. Traces are captured automatically (`--trace=retain-on-failure`) so
|
|
221
|
-
`get-failure` can show the DOM at the moment of failure.
|
|
222
|
-
|
|
223
|
-
Example result:
|
|
224
|
-
|
|
225
|
-
```markdown
|
|
226
|
-
## Playwright run — ❌ FAILED
|
|
227
|
-
|
|
228
|
-
**Command:** `playwright test --config playwright.config.ts tests/checkout.spec.ts --reporter=json`
|
|
229
|
-
**duration 4.2s · exit 1 · config `playwright.config.ts`**
|
|
230
|
-
|
|
231
|
-
| passed | failed | flaky | skipped | duration |
|
|
232
|
-
| ---: | ---: | ---: | ---: | ---: |
|
|
233
|
-
| 0 | 1 | 0 | 0 | 1.1s |
|
|
234
|
-
|
|
235
|
-
### ❌ 1 failing test(s)
|
|
236
|
-
|
|
237
|
-
### 1 of 1. checkout.spec.ts › pays with card
|
|
238
|
-
**File:** `checkout.spec.ts:5` | **failed · server-unreachable**
|
|
239
|
-
|
|
240
|
-
### ⚠️ SERVER_NOT_RUNNING
|
|
241
|
-
Your app (dev server) does not appear to be reachable. Start it in another terminal
|
|
242
|
-
(e.g. npm run dev / npm start), keep it running, then retry — or configure `webServer`
|
|
243
|
-
in playwright.config.* so Playwright starts it automatically.
|
|
121
|
+
```
|
|
122
|
+
/plugin marketplace add xsignon-secure/E2E
|
|
123
|
+
/plugin install playwright-e2e-mcp@xsignon-secure
|
|
244
124
|
```
|
|
245
125
|
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
| Argument | Type | Description |
|
|
249
|
-
| --- | --- | --- |
|
|
250
|
-
| `index` | number | 1-based failure index from the last run (default `1`) |
|
|
251
|
-
| `projectRoot` | string | Only used when re-reading the stored report |
|
|
252
|
-
|
|
253
|
-
Returns the message/code frame, expected vs actual, stack, failure kind with a
|
|
254
|
-
diagnosis, the test's console output, **the DOM snapshot from the Playwright trace
|
|
255
|
-
(plus the failed action, its selector, and the action log leading up to it)**,
|
|
256
|
-
**the network requests that failed** (4xx/5xx, dead endpoints, no-response — with
|
|
257
|
-
method, URL, status and resource type), **the console errors/warnings the page
|
|
258
|
-
logged before the failure**, and
|
|
259
|
-
numbered next steps (re-run this single test by `file:line`, headed/debug mode,
|
|
260
|
-
`validate-selector` when the message mentions a locator, …).
|
|
261
|
-
|
|
262
|
-
### `inspect-page`
|
|
126
|
+
Or the server only:
|
|
263
127
|
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
| `url` | string | Full http(s) URL to open (required) |
|
|
267
|
-
| `view` | `elements` \| `locators` | `locators` returns a compact map of interactive elements with verified locators (cheapest); `elements` (default) the DOM inventory |
|
|
268
|
-
| `projectRoot` | string | Project whose Playwright launches the browser |
|
|
269
|
-
| `selector` | string | Inspect matches of this selector instead of the whole DOM |
|
|
270
|
-
| `waitFor` | string | Wait for a selector (CSS or `text=…`) before inspecting |
|
|
271
|
-
| `waitUntil` | `load` \| `domcontentloaded` \| `networkidle` | Navigation wait condition |
|
|
272
|
-
| `includeHtml` | boolean | Include the rendered HTML (capped) |
|
|
273
|
-
| `maxHtmlChars` | number | HTML cap, default `20000` |
|
|
274
|
-
| `timeoutMs` | number | Overall limit, default `45000` |
|
|
275
|
-
| `storageState` / `headers` / `actions` / `viewport` | — | [Page state](#page-state-signed-in-pages-headers-and-steps) |
|
|
276
|
-
|
|
277
|
-
Each element comes with Playwright locators that were **checked on the page** to resolve to
|
|
278
|
-
exactly that element, best first (role + accessible name, test id, label, placeholder, text,
|
|
279
|
-
then CSS), plus its unique CSS selector, visibility, box and text. Console messages are
|
|
280
|
-
captured, errors first. A locator map looks like:
|
|
281
|
-
|
|
282
|
-
```markdown
|
|
283
|
-
1. `getByRole('button', { name: 'Proceed to checkout' })` — button "Proceed to checkout" · also `getByTestId('checkout-btn')`
|
|
284
|
-
2. `getByLabel('Search')` — searchbox "Search"
|
|
128
|
+
```bash
|
|
129
|
+
claude mcp add playwright-e2e -- npx -y playwright-e2e-mcp
|
|
285
130
|
```
|
|
286
131
|
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
| Argument | Type | Description |
|
|
290
|
-
| --- | --- | --- |
|
|
291
|
-
| `projectRoot` | string | Project directory |
|
|
292
|
-
| `config` | string | Config path or 1-based index |
|
|
293
|
-
| `testDir` | string | Restrict scanning to a directory (must stay inside the project) |
|
|
294
|
-
| `filter` | string | Case-insensitive substring filter on `file › title` |
|
|
295
|
-
| `limit` | number | Max tests returned, default `500` |
|
|
132
|
+
</details>
|
|
296
133
|
|
|
297
|
-
|
|
298
|
-
|
|
134
|
+
<details>
|
|
135
|
+
<summary><b>Claude Desktop</b></summary>
|
|
299
136
|
|
|
300
|
-
|
|
137
|
+
Download the `.mcpb` extension from the [latest release](https://github.com/xsignon-secure/E2E/releases/latest)
|
|
138
|
+
and double-click it. It ships its own dependencies, so no Node setup is needed. On install it
|
|
139
|
+
asks for your **project root**: choose the project folder, not your home directory.
|
|
301
140
|
|
|
302
|
-
|
|
303
|
-
| --- | --- | --- |
|
|
304
|
-
| `url` | string | Live page to test against (required) |
|
|
305
|
-
| `selector` | string | CSS, a Playwright selector (`text=`, `role=`, `>>`) or a locator expression such as `getByRole('button', { name: 'Save' }).first()` (required) |
|
|
306
|
-
| `projectRoot` | string | Project whose Playwright launches the browser |
|
|
307
|
-
| `timeoutMs` | number | Overall limit, default `45000` |
|
|
308
|
-
| `storageState` / `headers` / `actions` / `viewport` | — | [Page state](#page-state-signed-in-pages-headers-and-steps) |
|
|
141
|
+
You can also add the standard config to `claude_desktop_config.json`.
|
|
309
142
|
|
|
310
|
-
|
|
311
|
-
`getByLabel`, `getByPlaceholder`, `getByAltText`, `getByTitle`, `getByTestId`, `locator`,
|
|
312
|
-
`first`, `last`, `nth`, `filter`) and are never evaluated as code.
|
|
143
|
+
</details>
|
|
313
144
|
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
brittle (`:nth-child`, long descendant chains, XPath), `✅ VALID — 0 matches` with debugging
|
|
317
|
-
advice, or `❌ INVALID` with the parse error.
|
|
145
|
+
<details>
|
|
146
|
+
<summary><b>Cursor</b></summary>
|
|
318
147
|
|
|
319
|
-
|
|
148
|
+
Click [Install in Cursor](https://cursor.com/en/install-mcp?name=playwright-e2e&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsInBsYXl3cmlnaHQtZTJlLW1jcCJdfQ==),
|
|
149
|
+
or go to **Settings → MCP → Add new MCP server** and paste the standard config. For one
|
|
150
|
+
project only, put it in `.cursor/mcp.json`.
|
|
320
151
|
|
|
321
|
-
|
|
322
|
-
| --- | --- | --- |
|
|
323
|
-
| `description` | string | What the test should cover (required) |
|
|
324
|
-
| `pageUrl` | string | Page the test starts on (default: `baseURL` / `webServer.url` from config) |
|
|
325
|
-
| `testDir` / `file` | string | Where to write the spec (default: detected `testDir` + `generated/<slug>.spec.ts`); `file` must end in `.spec.*` or `.test.*` |
|
|
326
|
-
| `write` | boolean | Write the file to disk (default `true`) |
|
|
327
|
-
| `overwrite` | boolean | Replace an existing spec at the target path; only specs this tool generated can be replaced |
|
|
328
|
-
| `liveInspect` | boolean | Cross-check selectors against the live page (default on when a URL is known) |
|
|
329
|
-
| `projectRoot` / `config` | string | As with the other tools |
|
|
330
|
-
|
|
331
|
-
Reads the agent's recent changes (`git status`, falling back to `git diff HEAD~1`,
|
|
332
|
-
then recent mtimes), extracts the locators those files actually declare
|
|
333
|
-
(`data-testid`, `getByRole`, `aria-label`, `placeholder`, `id`, `name`, element text),
|
|
334
|
-
ranks verified-live selectors first, writes a spec built from them, and reports each
|
|
335
|
-
selector with its source `file:line`. When the source has no usable selectors, it uses the
|
|
336
|
-
verified locators from the live page that best match the description.
|
|
337
|
-
|
|
338
|
-
### `compare-visual-state`
|
|
339
|
-
|
|
340
|
-
| Argument | Type | Description |
|
|
341
|
-
| --- | --- | --- |
|
|
342
|
-
| `url` | string | Page to capture (required) |
|
|
343
|
-
| `name` | string | Baseline id, e.g. `checkout-page` (letters, digits, `. _ -`) |
|
|
344
|
-
| `action` | `compare` \| `baseline` | `compare` (default) diffs; `baseline` re-captures the reference |
|
|
345
|
-
| `selector` | string | Capture just this element |
|
|
346
|
-
| `fullPage` | boolean | Capture the full scrollable page |
|
|
347
|
-
| `tolerance` | number | Percent of pixels that may differ (default `0.1`) |
|
|
348
|
-
| `pixelThreshold` | number | Per-pixel channel delta considered different (default `60`) |
|
|
349
|
-
| `waitUntil` / `waitFor` / `timeoutMs` | — | As with `inspect-page` |
|
|
350
|
-
| `storageState` / `headers` / `actions` / `viewport` | — | [Page state](#page-state-signed-in-pages-headers-and-steps) |
|
|
351
|
-
|
|
352
|
-
The first call saves a baseline under `.pw-mcp/visual/` (add that to `.gitignore`, or
|
|
353
|
-
commit it for CI comparisons). Later calls report changed-pixel counts, **merged
|
|
354
|
-
regions** (`(x, y) 120×40 — 1,200 px`), the **average color shift** ("blue → red"),
|
|
355
|
-
and write a red-highlighted diff image for review.
|
|
356
|
-
|
|
357
|
-
### `diagnose-flaky`
|
|
358
|
-
|
|
359
|
-
| Argument | Type | Description |
|
|
360
|
-
| --- | --- | --- |
|
|
361
|
-
| `testFiles` | string[] | Tests to diagnose (`file:line` supported). Defaults to the tests that failed in the most recent run |
|
|
362
|
-
| `runs` | number | Repetitions, `2–10` (default `3`) |
|
|
363
|
-
| `isolate` | boolean | Also tell parallelism bugs from flakiness: repeat fully parallel (`workers` default `4`), then repeat the failing tests with one worker |
|
|
364
|
-
| `order` | boolean | Also run the target files in `runs` random orders with one worker (Playwright 1.64+ `--shuffle`), replay a failing order, and name the test it relies on or is broken by |
|
|
365
|
-
| `separateProcesses` | boolean | Start a fresh Playwright process per repetition instead of one `--repeat-each` process (slower; for flakes that depend on global setup or a cold start) |
|
|
366
|
-
| `browser` / `headed` / `workers` / `config` | — | As with `run-test` |
|
|
367
|
-
| `timeoutMs` | number | Hard wall-clock limit **per repetition** (default `120000`); one `--repeat-each` process gets this × `runs` |
|
|
368
|
-
| `projectRoot` | string | Project directory |
|
|
369
|
-
|
|
370
|
-
All repetitions run in **one Playwright process** with `--repeat-each` and
|
|
371
|
-
`--retries=0`, so each attempt is honest evidence and the browser and workers
|
|
372
|
-
start once instead of once per run (about 25% faster on the integration fixture,
|
|
373
|
-
more for suites with expensive setup). A small extra reporter records every
|
|
374
|
-
attempt, because Playwright's JSON report keeps only one result per test.
|
|
375
|
-
|
|
376
|
-
The verdict is **per test**: a table of passes, failures, duration spread and
|
|
377
|
-
first error for each test, plus the verdict block for the worst one:
|
|
378
|
-
|
|
379
|
-
- **❌ CONSISTENTLY FAILING** — failed every run (same error → reproducible bug,
|
|
380
|
-
different errors → still broken, just noisy). Fix it; it is not flaky.
|
|
381
|
-
- **⚠️ FLAKY** — some runs passed. Includes `N of M` counts and whether the
|
|
382
|
-
failures share one signature (real intermittent bug) or vary (timing/environment
|
|
383
|
-
instability).
|
|
384
|
-
- **✅ NOT REPRODUCING** — passed every re-run; the original failure was one-off.
|
|
385
|
-
|
|
386
|
-
With `isolate: true` a second table compares each failing test **in parallel**
|
|
387
|
-
and **alone**:
|
|
388
|
-
|
|
389
|
-
| Reading | Meaning |
|
|
390
|
-
| --- | --- |
|
|
391
|
-
| 🔀 fails only in parallel | Passes every run alone: shared state between tests (same account, DB rows, files, ports). Give each worker its own data or run the file serially |
|
|
392
|
-
| ⏱️ flaky even alone | The race is inside the test or the app: missing `await`, asserting before the UI settles |
|
|
393
|
-
| ❌ fails alone too | Not a parallelism problem; a real bug |
|
|
152
|
+
</details>
|
|
394
153
|
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
scheduled on its own. A test that fails in some orders is replayed in a failing
|
|
398
|
-
order; if it fails again it is read as **🔗 order dependent**, otherwise as
|
|
399
|
-
**flaky regardless of order**:
|
|
154
|
+
<details>
|
|
155
|
+
<summary><b>VS Code / GitHub Copilot</b></summary>
|
|
400
156
|
|
|
401
|
-
|
|
402
|
-
| `tests/cart.spec.ts:10` checkout lists the book | 2/3 | `4` | 🔗 order dependent … |
|
|
157
|
+
Click the **Install in VS Code** button above, or run:
|
|
403
158
|
|
|
404
|
-
|
|
405
|
-
-
|
|
406
|
-
- Reproduce: `npx playwright test tests/cart.spec.ts --workers=1 --fully-parallel --retries=0 --shuffle=4`
|
|
159
|
+
```bash
|
|
160
|
+
code --add-mcp '{"name":"playwright-e2e","command":"npx","args":["-y","playwright-e2e-mcp"]}'
|
|
407
161
|
```
|
|
408
162
|
|
|
409
|
-
|
|
410
|
-
Set `PW_MCP_SHUFFLE_SEEDS=4,7` on the server to replay given orders, for
|
|
411
|
-
example to check a fix.
|
|
412
|
-
|
|
413
|
-
Each repetition is recorded in the run history, and the last failing run is
|
|
414
|
-
stored, so `analyze-history` and `get-failure` work right afterwards.
|
|
415
|
-
|
|
416
|
-
### `audit-accessibility`
|
|
417
|
-
|
|
418
|
-
| Argument | Type | Description |
|
|
419
|
-
| --- | --- | --- |
|
|
420
|
-
| `url` | string | Page to audit (required) |
|
|
421
|
-
| `standard` | `wcag2a` \| `wcag2aa` \| `wcag21aa` \| `wcag22aa` \| `best-practice` \| `all` | Rule set (default `wcag22aa`: WCAG 2.0–2.2 level A and AA; `best-practice` adds axe's extra rules; `all` runs every rule) |
|
|
422
|
-
| `minImpact` | `critical` \| `serious` \| `moderate` \| `minor` | Hide violations below this impact |
|
|
423
|
-
| `selector` | string | Audit only this part of the page |
|
|
424
|
-
| `waitFor` / `waitUntil` / `timeoutMs` | — | As with `inspect-page` (`waitUntil` default `load`, `timeoutMs` default `60000`) |
|
|
425
|
-
| `storageState` / `headers` / `actions` / `viewport` | — | [Page state](#page-state-signed-in-pages-headers-and-steps) |
|
|
163
|
+
</details>
|
|
426
164
|
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
Example output:
|
|
165
|
+
<details>
|
|
166
|
+
<summary><b>Codex</b></summary>
|
|
430
167
|
|
|
168
|
+
```bash
|
|
169
|
+
codex mcp add playwright-e2e -- npx -y playwright-e2e-mcp
|
|
431
170
|
```
|
|
432
|
-
## ♿ 3 accessibility violations (2 critical, 1 serious)
|
|
433
171
|
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
- `getByTestId('go')`: Element does not have inner text that is visible to screen readers
|
|
172
|
+
A Playwright run with retries can take longer than Codex's default 60 s tool timeout. Raise
|
|
173
|
+
the timeouts in `~/.codex/config.toml`:
|
|
437
174
|
|
|
438
|
-
|
|
439
|
-
-
|
|
175
|
+
```toml
|
|
176
|
+
[mcp_servers.playwright-e2e]
|
|
177
|
+
command = "npx"
|
|
178
|
+
args = ["-y", "playwright-e2e-mcp"]
|
|
179
|
+
startup_timeout_sec = 60
|
|
180
|
+
tool_timeout_sec = 600
|
|
440
181
|
```
|
|
441
182
|
|
|
442
|
-
|
|
183
|
+
</details>
|
|
443
184
|
|
|
444
|
-
|
|
185
|
+
<details>
|
|
186
|
+
<summary><b>Gemini CLI / Qwen Code</b></summary>
|
|
445
187
|
|
|
446
|
-
|
|
447
|
-
| --- | --- | --- |
|
|
448
|
-
| `index` | number | 1-based failure index from the last run (default `1`) |
|
|
449
|
-
| `apply` | boolean | Write the fix into the spec (default `false`: return the diff only) |
|
|
450
|
-
| `verify` | boolean | After applying, re-run the test and revert the file if it still fails (default `true`) |
|
|
451
|
-
| `allowExpectationUpdate` | boolean | Allow applying a changed expected text/value (default `false`, since it can hide a real regression) |
|
|
452
|
-
| `url` | string | Heal against this live page instead of the DOM snapshot in the failure trace |
|
|
453
|
-
| `storageState` / `headers` | — | For `url`, as in [Page state](#page-state-signed-in-pages-headers-and-steps) |
|
|
454
|
-
| `config` / `timeoutMs` / `projectRoot` | — | As with `run-test` |
|
|
455
|
-
|
|
456
|
-
Two kinds of fix:
|
|
457
|
-
|
|
458
|
-
- **Locator drift.** The broken locator is read from the error, its words are matched
|
|
459
|
-
(with synonyms such as *btn → button*, *checkout → proceed to checkout*) against every
|
|
460
|
-
element in the DOM at failure, and the best match's verified-unique locator replaces it.
|
|
461
|
-
Confidence is `high` when the match is clear, `medium` otherwise.
|
|
462
|
-
- **Changed copy.** For `toHaveText`, `toContainText`, `toHaveValue`, `toHaveTitle`,
|
|
463
|
-
`toHaveURL` and `toHaveAttribute`, it proposes the value the page actually showed. It is
|
|
464
|
-
only applied with `allowExpectationUpdate: true`.
|
|
465
|
-
|
|
466
|
-
The response is a unified diff. With `apply: true` the file is written only if it has not
|
|
467
|
-
changed since the failure, the single test is re-run (same project, no retries), and the
|
|
468
|
-
edit is reverted when it is still red. If nothing on the page shares the locator's
|
|
469
|
-
distinctive words, it refuses and reports that the element is most likely **missing**.
|
|
470
|
-
|
|
471
|
-
### `get-run-status`
|
|
472
|
-
|
|
473
|
-
| Argument | Type | Description |
|
|
474
|
-
| --- | --- | --- |
|
|
475
|
-
| `runId` | string | Run id from `run-test` with `background: true` (default: the most recent run) |
|
|
476
|
-
| `waitSeconds` | number | Wait up to this long (max 55) for the run to finish |
|
|
477
|
-
| `cancel` | boolean | Stop the run and kill its browsers |
|
|
188
|
+
Server and skill, as an extension:
|
|
478
189
|
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
190
|
+
```bash
|
|
191
|
+
gemini extensions install https://github.com/xsignon-secure/E2E
|
|
192
|
+
```
|
|
482
193
|
|
|
483
|
-
|
|
194
|
+
Or paste the standard config into `.gemini/settings.json` (Qwen Code: `.qwen/settings.json`).
|
|
484
195
|
|
|
485
|
-
|
|
486
|
-
| --- | --- | --- |
|
|
487
|
-
| `view` | `flaky` \| `patterns` \| `test` | `flaky` (default) ranks unstable tests; `patterns` groups failures by kind and normalized error; `test` shows one test's timeline |
|
|
488
|
-
| `test` | string | Substring of a test's file or title (filters every view) |
|
|
489
|
-
| `lastRuns` | number | How many recent runs to analyze (default `30`) |
|
|
490
|
-
| `limit` | number | Max rows (default `15`) |
|
|
196
|
+
</details>
|
|
491
197
|
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
nothing leaves the machine; `PW_MCP_HISTORY=0` turns it off). The flaky view separates
|
|
495
|
-
tests that flip between pass and fail, tests that started failing and stayed red
|
|
496
|
-
(regressed), and tests that failed every run (broken), each with a `PFFP~` timeline.
|
|
198
|
+
<details>
|
|
199
|
+
<summary><b>Windsurf, Cline, Zed and other clients</b></summary>
|
|
497
200
|
|
|
498
|
-
|
|
201
|
+
Add the standard config to the client's MCP config file (Cline: *MCP Servers → Configure*).
|
|
202
|
+
Clients that take a command, such as Zed, run `npx` with the arguments `-y playwright-e2e-mcp`.
|
|
499
203
|
|
|
500
|
-
|
|
204
|
+
</details>
|
|
501
205
|
|
|
502
|
-
|
|
206
|
+
<details>
|
|
207
|
+
<summary><b>Freebuff / Codebuff</b></summary>
|
|
503
208
|
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
209
|
+
Drop an `mcp.json` with the standard config into your project's `.agents/` folder. This repo
|
|
210
|
+
ships one in [`.agents/mcp.json`](.agents/mcp.json), so opening the checkout attaches the
|
|
211
|
+
server workspace-wide. Freebuff asks you to trust a repository's `.agents/` on first run.
|
|
507
212
|
|
|
508
|
-
|
|
509
|
-
builds `dist/` automatically on install). Pin a release tag: an unpinned
|
|
510
|
-
`github:xsignon-secure/E2E` runs whatever is on the default branch at that moment.
|
|
213
|
+
</details>
|
|
511
214
|
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
npm install -D github:xsignon-secure/E2E#v0.4.2 @playwright/test # or as a project dependency
|
|
515
|
-
npx playwright install chromium
|
|
516
|
-
```
|
|
215
|
+
<details>
|
|
216
|
+
<summary><b>ChatGPT and other remote-only clients</b></summary>
|
|
517
217
|
|
|
518
|
-
|
|
519
|
-
|
|
218
|
+
Some clients only accept a remote HTTPS server. This repo ships a Streamable HTTP bridge you
|
|
219
|
+
can host for free on Vercel, with OAuth sign-in for claude.ai connectors. See
|
|
220
|
+
[Hosted endpoint](docs/hosted-endpoint.md). On Vercel the page tools (inspect a page, check
|
|
221
|
+
selectors, accessibility and Web Vitals audits) run on a bundled Chromium against public URLs.
|
|
222
|
+
Running and fixing your own tests needs your project, so use the local install for that.
|
|
520
223
|
|
|
521
|
-
|
|
522
|
-
npm install -D https://github.com/xsignon-secure/E2E/releases/download/v0.4.2/playwright-e2e-mcp-0.4.2.tgz
|
|
523
|
-
```
|
|
524
|
-
|
|
525
|
-
Listed in the **official [MCP Registry](https://registry.modelcontextprotocol.io/)** as
|
|
526
|
-
`io.github.xsignon-secure/E2E` — registry-aware clients discover it there, and every
|
|
527
|
-
`v*` release tag republishes the entry from CI via [`server.json`](server.json).
|
|
224
|
+
</details>
|
|
528
225
|
|
|
529
|
-
|
|
226
|
+
<details>
|
|
227
|
+
<summary><b>Other ways to install</b></summary>
|
|
530
228
|
|
|
531
|
-
|
|
229
|
+
| Route | How |
|
|
230
|
+
| --- | --- |
|
|
231
|
+
| Skill only, any agent | `npx skills add xsignon-secure/E2E` |
|
|
232
|
+
| MCP Registry | `io.github.xsignon-secure/E2E` ([listing](https://registry.modelcontextprotocol.io/)); registry-aware clients find it there |
|
|
233
|
+
| Straight from GitHub, no npm | `npx -y github:xsignon-secure/E2E#v0.5.1` (pin a release tag) |
|
|
234
|
+
| Release tarball | `npm install -D https://github.com/xsignon-secure/E2E/releases/download/v0.5.1/playwright-e2e-mcp-0.5.1.tgz` |
|
|
235
|
+
| Docker | `docker run -i --rm -v "$PWD:/work" playwright-e2e-mcp` (see [Development](docs/development.md#docker)) |
|
|
236
|
+
| Local checkout | `"command": "node", "args": ["/path/to/E2E/dist/index.js"]` |
|
|
532
237
|
|
|
533
|
-
|
|
534
|
-
{
|
|
535
|
-
"mcpServers": {
|
|
536
|
-
"playwright-e2e": {
|
|
537
|
-
"command": "npx",
|
|
538
|
-
"args": ["-y", "github:xsignon-secure/E2E#v0.4.2"],
|
|
539
|
-
"env": { "PW_MCP_PROJECT_ROOT": "/absolute/path/to/your/project" }
|
|
540
|
-
}
|
|
541
|
-
}
|
|
542
|
-
}
|
|
543
|
-
```
|
|
238
|
+
</details>
|
|
544
239
|
|
|
545
|
-
|
|
546
|
-
(Cline: *MCP Servers → Configure*). **Zed** and other clients that take a command: run `npx`
|
|
547
|
-
with the arguments `-y playwright-e2e-mcp`.
|
|
548
|
-
The server uses its working directory as the project root; set `PW_MCP_PROJECT_ROOT`
|
|
549
|
-
when the client launches it somewhere else (e.g. your home directory).
|
|
240
|
+
### Your first prompt
|
|
550
241
|
|
|
551
|
-
|
|
242
|
+
Restart your client, then ask:
|
|
552
243
|
|
|
553
|
-
```bash
|
|
554
|
-
codex mcp add playwright-e2e -- npx -y github:xsignon-secure/E2E#v0.4.2
|
|
555
|
-
code --add-mcp '{"name":"playwright-e2e","command":"npx","args":["-y","github:xsignon-secure/E2E#v0.4.2"]}'
|
|
556
244
|
```
|
|
557
|
-
|
|
558
|
-
Codex's defaults fight this server: the first launch clones the repo and runs
|
|
559
|
-
`tsc` (measured 30 s on a cold `npx` cache, against a 10 s
|
|
560
|
-
`startup_timeout_sec` default), and a Playwright run with retries beats the 60 s
|
|
561
|
-
`tool_timeout_sec` default. Raise both in `~/.codex/config.toml`:
|
|
562
|
-
|
|
563
|
-
```toml
|
|
564
|
-
[mcp_servers.playwright-e2e]
|
|
565
|
-
command = "npx"
|
|
566
|
-
args = ["-y", "github:xsignon-secure/E2E#v0.4.2"]
|
|
567
|
-
startup_timeout_sec = 60
|
|
568
|
-
tool_timeout_sec = 600
|
|
245
|
+
Run my Playwright tests and fix whatever fails.
|
|
569
246
|
```
|
|
570
247
|
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
the bundle ships its own dependencies, so no Node setup is required. On install it
|
|
574
|
-
asks you to pick your **project root** (required, no default: choose the project
|
|
575
|
-
folder, not your home directory) and wires it
|
|
576
|
-
into `PW_MCP_PROJECT_ROOT`, so the tools point at a real project from the first call.
|
|
248
|
+
Your agent should run the suite, read the failure, and propose a fix it has re-run. To check the
|
|
249
|
+
setup without your own tests, ask it to run `examples/sample-test.spec.ts` from this repo.
|
|
577
250
|
|
|
578
|
-
|
|
251
|
+
## What's new in 0.5.0
|
|
579
252
|
|
|
580
|
-
|
|
581
|
-
|
|
582
|
-
|
|
583
|
-
|
|
584
|
-
|
|
585
|
-
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
**Freebuff / Codebuff (project-scoped):** this repo ships a committed
|
|
589
|
-
[`.agents/mcp.json`](.agents/mcp.json), so opening the checkout in Freebuff
|
|
590
|
-
attaches the server workspace-wide — no global config needed. Your own projects
|
|
591
|
-
can do the same: drop an `mcp.json` with the block above into their `.agents/`
|
|
592
|
-
directory. Freebuff asks you to trust a repository's `.agents/` on first run.
|
|
253
|
+
- **Diagnose CI failures:** `load-report` reads a CI job's report (or sharded blob reports)
|
|
254
|
+
and its traces, so the same diagnosis and verified fixes work on failures from CI.
|
|
255
|
+
- **Know which commit broke a test:** `analyze-history` adds `commits`, `slow` and `trend` views.
|
|
256
|
+
- **Write tests by walking the flow:** `generate-e2e-test` with `steps` runs them on the live
|
|
257
|
+
page, writes the spec with end-state assertions, and runs it once to prove it passes.
|
|
258
|
+
- **Performance audits:** `audit-performance` measures Web Vitals and hands back a budget test.
|
|
259
|
+
- **More page control:** aria snapshots with ready `toMatchAriaSnapshot` assertions, network
|
|
260
|
+
mocks, HAR record/replay, and Chromium, Firefox or WebKit.
|
|
593
261
|
|
|
594
|
-
All
|
|
595
|
-
`idempotentHint`, `openWorldHint`), so clients can show accurate safety prompts
|
|
596
|
-
before running anything.
|
|
262
|
+
[All release notes](https://github.com/xsignon-secure/E2E/releases)
|
|
597
263
|
|
|
598
|
-
|
|
264
|
+
## Tools
|
|
599
265
|
|
|
600
|
-
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
|
|
610
|
-
|
|
266
|
+
| Tool | Purpose |
|
|
267
|
+
| --- | --- |
|
|
268
|
+
| `run-test` | Run Playwright tests and return stats, failures, diagnostics and hints; reports progress, and `background: true` returns a run id for long suites |
|
|
269
|
+
| `get-run-status` | Progress or final result of a background run; can wait for it or cancel it |
|
|
270
|
+
| `get-failure` | Deep analysis of one failure: stack, expected/actual, **DOM snapshot at failure (from the Playwright trace)**, failed requests, console errors, next steps |
|
|
271
|
+
| `load-report` | Load a run from CI: a JSON report or a folder of sharded blob reports; CI paths are mapped onto the checkout and traces found in the unpacked artifact, so `get-failure`, `suggest-fix` and `diagnose-flaky` work on it, and the run joins the history with its commit |
|
|
272
|
+
| `suggest-fix` | Patch a failure: a drifted locator is replaced with one **proven unique** on the DOM at failure, an ambiguous one (strict mode violation) gets the unique locator of the intended match, changed copy gets the new expected text; `apply: true` writes it, re-runs the test and reverts if still red |
|
|
273
|
+
| `inspect-page` | Open a URL headlessly (optionally signed in, after steps) and return elements with **verified Playwright locators**, CSS selectors, visibility, text and console output, or the **accessibility snapshot with a ready `toMatchAriaSnapshot` assertion** |
|
|
274
|
+
| `list-tests` | List available tests (`file`, `line`, full title, projects) with filtering |
|
|
275
|
+
| `validate-selector` | Check a CSS selector, Playwright selector or **locator expression** against a live page: validity, match count, and a sturdier verified locator for each match |
|
|
276
|
+
| `generate-e2e-test` | Scaffold a Playwright test from a description using the project's **real** selectors and verified locators from the live page, or **walk a flow** step by step on the live page and write a spec that is run once to prove it passes |
|
|
277
|
+
| `compare-visual-state` | Visual regression: screenshot before/after a change and report *what* moved and how colors shifted |
|
|
278
|
+
| `audit-accessibility` | Run axe-core (WCAG 2.2 AA by default) on a page, signed in or after steps, and return violations worst first with **verified locators** for the offending elements and a regression check |
|
|
279
|
+
| `audit-performance` | Measure lab **Web Vitals** (LCP, CLS, TBT, FCP, TTFB, INP during steps), optionally as a throttled phone and the median of several runs; rate them or check budgets, name the LCP and shifting elements as verified locators, list heavy and render-blocking requests, and return a budget test |
|
|
280
|
+
| `diagnose-flaky` | Run a failing test 2–10 times **with retries disabled** and return an evidence verdict: `CONSISTENTLY FAILING`, `FLAKY` or `NOT REPRODUCING`; tells parallelism and **test-order** bugs from races |
|
|
281
|
+
| `analyze-history` | Rank flaky, regressed and broken tests from the local run history, group failures by cause, list the slowest tests, show stability by day, name **the commit where each test went bad**, or show one test's timeline |
|
|
611
282
|
|
|
612
|
-
|
|
283
|
+
Two **prompts** package the main workflows for clients that show them as slash commands:
|
|
284
|
+
`fix-failing-test` and `triage-flaky-tests`.
|
|
613
285
|
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
286
|
+
Every argument and example output is in the **[tool reference](docs/tools.md)**. All tools carry
|
|
287
|
+
MCP tool annotations (`readOnlyHint`, `destructiveHint`, …), so clients can show accurate safety
|
|
288
|
+
prompts.
|
|
617
289
|
|
|
618
|
-
|
|
619
|
-
|
|
620
|
-
| **Endpoint** | `https://playwright-e2e-mcp.vercel.app/api/mcp` |
|
|
621
|
-
| **Transport** | MCP Streamable HTTP (`POST` JSON in, JSON or SSE out) |
|
|
622
|
-
| **Auth** | bearer token (`PW_MCP_HTTP_TOKEN`), or OAuth sign-in with that token as the access key (for claude.ai connectors). The deployment at this URL requires one of them; a bridge started without the variable serves only the read-only tools |
|
|
623
|
-
| **Source** | [`api/mcp.ts`](api/mcp.ts) → [`src/http.ts`](src/http.ts) |
|
|
624
|
-
|
|
625
|
-
The bridge runs the *same* `createServer()` as the stdio transport; the SDK
|
|
626
|
-
serves every request with a fresh server instance, which is what a serverless
|
|
627
|
-
function wants. `test/http-bridge.test.mjs` drives the real Node adapter over
|
|
628
|
-
`node:http` so a broken bridge fails in CI, not in ChatGPT.
|
|
629
|
-
|
|
630
|
-
**This demo deployment is token-protected.** `https://playwright-e2e-mcp.vercel.app`
|
|
631
|
-
runs on the maintainer's account and has `PW_MCP_HTTP_TOKEN` set, so a request
|
|
632
|
-
without `Authorization: Bearer <token>` (or an OAuth sign-in with it) gets
|
|
633
|
-
`401 {"error":"unauthorized"}`. Nothing is held back: the same bridge is free to
|
|
634
|
-
host yourself on a free Vercel account, with your own token:
|
|
635
|
-
|
|
636
|
-
[](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Fxsignon-secure%2FE2E&env=PW_MCP_HTTP_TOKEN&envDescription=A%20random%20secret%20of%2016%2B%20characters%3B%20clients%20send%20it%20as%20a%20bearer%20token%20or%20type%20it%20to%20sign%20in&project-name=playwright-e2e-mcp)
|
|
637
|
-
|
|
638
|
-
A serverless function has no browser, so for running and fixing your own tests
|
|
639
|
-
the free local install (`npx -y playwright-e2e-mcp init`) is the better route.
|
|
640
|
-
|
|
641
|
-
**Open vs. token-protected.** With `PW_MCP_HTTP_TOKEN` set (at least 16
|
|
642
|
-
characters; use a random value) every tool is served to clients that send
|
|
643
|
-
`Authorization: Bearer <token>`, and every other request gets `401`. Without that
|
|
644
|
-
variable anyone who reaches the URL can call the bridge, so it degrades to
|
|
645
|
-
`list-tests` and `get-failure` in restricted mode: `list-tests` scans sources
|
|
646
|
-
instead of running `playwright test --list` (which would execute the project's
|
|
647
|
-
config), callers cannot pick another `projectRoot`, and nothing spawns a process,
|
|
648
|
-
drives a browser or writes a file. Child processes started over HTTP get only an
|
|
649
|
-
allowlisted environment. `PW_MCP_ALLOWED_HOSTS` (comma separated, `*` for any)
|
|
650
|
-
limits the accepted `Host` header; without a token and without that variable,
|
|
651
|
-
only `localhost` names and the deployment's own Vercel hostnames are accepted
|
|
652
|
-
(DNS-rebinding protection).
|
|
653
|
-
|
|
654
|
-
**Add it to ChatGPT:** Settings → Connectors → turn on **Advanced → Developer
|
|
655
|
-
mode** → *Create custom connector* → paste the endpoint above. Authentication
|
|
656
|
-
**None** only works against a bridge with no token (read-only tools), so for this
|
|
657
|
-
deployment choose the connector's **API key** authentication and send
|
|
658
|
-
`Authorization: Bearer <token>` — or point the connector at your own deployment
|
|
659
|
-
with `PW_MCP_HTTP_TOKEN` unset.
|
|
660
|
-
|
|
661
|
-
**Add it to claude.ai (or Claude Desktop's connectors):** Settings → Connectors →
|
|
662
|
-
*Add custom connector* → paste the endpoint above and leave the OAuth fields
|
|
663
|
-
empty. Claude registers itself and opens a sign-in page; type the deployment's
|
|
664
|
-
access key (the value of `PW_MCP_HTTP_TOKEN`) and press **Allow access**. The
|
|
665
|
-
bridge is its own small OAuth 2.1 server (discovery, dynamic client
|
|
666
|
-
registration, authorization code with PKCE S256, refresh tokens) and keeps no
|
|
667
|
-
state: client ids, codes and tokens are signed with a key derived from
|
|
668
|
-
`PW_MCP_HTTP_TOKEN`, access tokens last an hour and refresh tokens 30 days, and
|
|
669
|
-
changing `PW_MCP_HTTP_TOKEN` signs every connected client out. The sign-in page
|
|
670
|
-
shows which site will receive access, so only type the key when that is the
|
|
671
|
-
app you are connecting. Source: [`src/oauth.ts`](src/oauth.ts); the
|
|
672
|
-
`/.well-known/oauth-*` and `/oauth/*` paths reach the function through the
|
|
673
|
-
rewrites in [`vercel.json`](vercel.json).
|
|
674
|
-
|
|
675
|
-
To self-host the open, read-only variant, deploy this repo with
|
|
676
|
-
`PW_MCP_HTTP_TOKEN` unset; nothing else changes.
|
|
677
|
-
|
|
678
|
-
Codex can also take the remote transport instead of spawning `npx`, if you'd
|
|
679
|
-
rather not ship Playwright to every machine. Point it at the token through an
|
|
680
|
-
environment variable so the secret stays out of the config file:
|
|
290
|
+
**Only need the fix loop?** `--tools=core` (or `PW_MCP_TOOLS=core`) exposes six tools and cuts the
|
|
291
|
+
tool-definition context by about three quarters. Other profiles: `triage`, `inspect`, `author`.
|
|
681
292
|
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
codex mcp add playwright-e2e-remote --url https://playwright-e2e-mcp.vercel.app/api/mcp \
|
|
685
|
-
--bearer-token-env-var PW_MCP_HTTP_TOKEN
|
|
686
|
-
```
|
|
293
|
+
<details>
|
|
294
|
+
<summary><b>Typical workflow</b></summary>
|
|
687
295
|
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
|
|
296
|
+
1. `run-test` `{ "testFiles": ["tests/checkout.spec.ts"] }`, or `{ "background": true }`
|
|
297
|
+
for a long suite and `get-run-status` `{ "waitSeconds": 30 }` until it finishes. For a
|
|
298
|
+
failure from CI, download the job's artifacts and call `load-report`
|
|
299
|
+
`{ "report": "ci-artifacts/results.json" }` instead.
|
|
300
|
+
2. `get-failure` `{ "index": 1 }` to read the code frame, expected/actual, **the DOM
|
|
301
|
+
snapshot at failure**, the failed network requests and the page's console errors.
|
|
302
|
+
3. `suggest-fix` `{ "index": 1 }` for a locator or copy change: review the diff, then
|
|
303
|
+
`{ "index": 1, "apply": true }` to write it and verify with a re-run. If it reports the
|
|
304
|
+
element is missing, fix the app, not the test.
|
|
305
|
+
4. For anything it cannot patch: `inspect-page` `{ "url": "…", "view": "locators" }`
|
|
306
|
+
(add `storageState` or `actions` to reach the right screen), then `validate-selector`
|
|
307
|
+
to prove the new locator before editing.
|
|
308
|
+
5. If a failure looks intermittent: `diagnose-flaky` `{ "runs": 3 }` for fresh evidence,
|
|
309
|
+
and `analyze-history` to see how the test behaved across earlier runs
|
|
310
|
+
(`{ "view": "commits" }` names the commit where it went bad).
|
|
311
|
+
6. After changing CSS or components: `compare-visual-state` `{ "url": "…", "name": "checkout" }`.
|
|
312
|
+
7. Re-run **only what failed** with `run-test` `{ "lastFailed": true }` until green.
|
|
695
313
|
|
|
696
|
-
|
|
697
|
-
|
|
698
|
-
|
|
699
|
-
|
|
700
|
-
-H 'content-type: application/json' \
|
|
701
|
-
-H 'accept: application/json, text/event-stream' \
|
|
702
|
-
-H "authorization: Bearer $PW_MCP_HTTP_TOKEN" \
|
|
703
|
-
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1.0"}}}'
|
|
704
|
-
```
|
|
314
|
+
To write a new test, find the locators with `inspect-page` `{ "view": "locators" }`, then call
|
|
315
|
+
`generate-e2e-test` with a `description` and the `steps` of the flow; it walks them on the live
|
|
316
|
+
page, writes the spec and runs it once. To check a page's speed or accessibility, use
|
|
317
|
+
`audit-performance` and `audit-accessibility`.
|
|
705
318
|
|
|
706
|
-
|
|
707
|
-
every push, so there is no token to manage and no CLI step — watch the
|
|
708
|
-
`Vercel` commit status for the deployment result.
|
|
319
|
+
</details>
|
|
709
320
|
|
|
710
321
|
## CI: GitHub Action and `ci` command
|
|
711
322
|
|
|
@@ -733,7 +344,7 @@ steps:
|
|
|
733
344
|
- uses: actions/setup-node@v6
|
|
734
345
|
with: { node-version: 24 }
|
|
735
346
|
- run: npm ci && npx playwright install --with-deps chromium
|
|
736
|
-
- uses: xsignon-secure/E2E@v0.
|
|
347
|
+
- uses: xsignon-secure/E2E@v0.5.1
|
|
737
348
|
with:
|
|
738
349
|
flaky-runs: 3 # 0 skips the flaky verdict
|
|
739
350
|
suggest: 2 # suggest-fix diffs for the first 2 failures
|
|
@@ -743,33 +354,25 @@ steps:
|
|
|
743
354
|
Inputs: `project-root`, `args`, `flaky-runs`, `suggest`, `comment`,
|
|
744
355
|
`fail-on-failure`, `version`, `github-token`. Outputs: `exit-code`, `report`.
|
|
745
356
|
|
|
746
|
-
## Docker
|
|
747
|
-
|
|
748
|
-
```bash
|
|
749
|
-
docker build -t playwright-e2e-mcp .
|
|
750
|
-
docker run -i --rm -v "$PWD:/work" playwright-e2e-mcp # stdio MCP server
|
|
751
|
-
docker run --rm -v "$PWD:/work" playwright-e2e-mcp ci # CI report
|
|
752
|
-
```
|
|
753
|
-
|
|
754
|
-
The image is based on `mcr.microsoft.com/playwright:v1.63.0-noble`, so its
|
|
755
|
-
browsers match Playwright 1.63. Mount your project at `/work`; its own
|
|
756
|
-
`node_modules` provide `@playwright/test`.
|
|
757
|
-
|
|
758
357
|
## Configuration
|
|
759
358
|
|
|
359
|
+
<details>
|
|
360
|
+
<summary><b>Environment variables</b></summary>
|
|
361
|
+
|
|
760
362
|
| Environment variable | Default | Purpose |
|
|
761
363
|
| --- | --- | --- |
|
|
762
364
|
| `PW_MCP_PROJECT_ROOT` | server cwd | Default project root for every tool |
|
|
763
|
-
| `PW_MCP_TOOLS` | all | Which tools to expose, as tool names and/or profiles, comma separated (same as the `--tools=` flag). Profiles: `core` (run, status, failure, fix, flaky, list), `triage` (run, status, failure, flaky, history, list), `inspect` (page, selector, accessibility, visual), `author` (list, page, selector, generate, run). All
|
|
365
|
+
| `PW_MCP_TOOLS` | all | Which tools to expose, as tool names and/or profiles, comma separated (same as the `--tools=` flag). Profiles: `core` (run, status, failure, fix, flaky, list), `triage` (run, status, failure, CI report, flaky, history, list), `inspect` (page, selector, accessibility, performance, visual), `author` (list, page, selector, generate, run). All fourteen tool definitions cost about 9k tokens of context per request; `core` costs about 2.3k |
|
|
764
366
|
| `PW_MCP_STORAGE_STATE` | — | Default `storageState` file (inside the project) for the page tools |
|
|
765
367
|
| `PW_MCP_SHUFFLE_SEEDS` | random | Comma-separated seeds for `diagnose-flaky`'s `order` pass, to replay the orders a diagnosis reported |
|
|
766
368
|
| `PW_MCP_HISTORY` | on | `0` stops recording runs to `.playwright-e2e-mcp/history.jsonl` |
|
|
767
369
|
| `PW_MCP_ALLOWED_ROOTS` | — | Extra directories a caller may pass as `projectRoot` (`:`-separated, `;` on Windows). Anything outside these and the default root is rejected |
|
|
768
|
-
| `PW_MCP_HTTP_TOKEN` | — | HTTP bridge only: bearer token (16+ characters) that unlocks all tools (see
|
|
370
|
+
| `PW_MCP_HTTP_TOKEN` | — | HTTP bridge only: bearer token (16+ characters) that unlocks all tools (see [Hosted endpoint](docs/hosted-endpoint.md)) |
|
|
769
371
|
| `PW_MCP_ALLOWED_HOSTS` | localhost + Vercel hostnames when there is no token | HTTP bridge only: comma-separated `Host` allowlist; `*` accepts any |
|
|
770
372
|
| `PW_MCP_PASSTHROUGH_ENV` | — | HTTP bridge only: comma-separated extra variables passed to test runs (e.g. `BASE_URL`) |
|
|
771
373
|
| `PW_MCP_MAX_CHILDREN` | `4` | HTTP bridge only: how many test runs and browser probes may run at once |
|
|
772
374
|
| `PW_MCP_BLOCK_PRIVATE_URLS` | off (on for the HTTP bridge) | `1` makes the URL tools refuse loopback, private-network and cloud-metadata addresses, including redirects and subresources |
|
|
375
|
+
| `PW_MCP_SERVERLESS_CHROMIUM` | on for Vercel | Launch the page tools on the bundled serverless Chromium (`@sparticuz/chromium`, Linux x64, Node 22+); `0` turns it off, `1` turns it on elsewhere |
|
|
773
376
|
| `LOG_LEVEL` / `MCP_LOG_LEVEL` | `info` | `debug` \| `info` \| `warn` \| `error` \| `silent`; `LOG_LEVEL` wins when both are set |
|
|
774
377
|
| `LOG_FORMAT` / `MCP_LOG_FORMAT` | `text` | `text`, or `json` (also `ndjson`) for one structured JSON object per line — ready for a log ingester |
|
|
775
378
|
|
|
@@ -782,24 +385,7 @@ object (`time`, `level`, `message` plus context):
|
|
|
782
385
|
{"time":"2026-10-07T08:00:00.000Z","level":"info","message":"server ready","name":"playwright-e2e-mcp"}
|
|
783
386
|
```
|
|
784
387
|
|
|
785
|
-
|
|
786
|
-
|
|
787
|
-
1. `run-test` `{ "testFiles": ["tests/checkout.spec.ts"] }`, or `{ "background": true }`
|
|
788
|
-
for a long suite and `get-run-status` `{ "waitSeconds": 30 }` until it finishes.
|
|
789
|
-
2. `get-failure` `{ "index": 1 }` to read the code frame, expected/actual, **the DOM
|
|
790
|
-
snapshot at failure**, the failed network requests and the page's console errors.
|
|
791
|
-
3. `suggest-fix` `{ "index": 1 }` for a locator or copy change: review the diff, then
|
|
792
|
-
`{ "index": 1, "apply": true }` to write it and verify with a re-run. If it reports the
|
|
793
|
-
element is missing, fix the app, not the test.
|
|
794
|
-
4. For anything it cannot patch: `inspect-page` `{ "url": "…", "view": "locators" }`
|
|
795
|
-
(add `storageState` or `actions` to reach the right screen), then `validate-selector`
|
|
796
|
-
to prove the new locator before editing.
|
|
797
|
-
5. If a failure looks intermittent: `diagnose-flaky` `{ "runs": 3 }` for fresh evidence,
|
|
798
|
-
and `analyze-history` to see how the test behaved across earlier runs.
|
|
799
|
-
6. After changing CSS or components: `compare-visual-state` `{ "url": "…", "name": "checkout" }`.
|
|
800
|
-
7. Re-run **only what failed** with `run-test` `{ "lastFailed": true }` until green.
|
|
801
|
-
|
|
802
|
-
To write a new test, start with `generate-e2e-test` `{ "description": "checkout with a saved card" }`.
|
|
388
|
+
</details>
|
|
803
389
|
|
|
804
390
|
## Benchmark
|
|
805
391
|
|
|
@@ -819,306 +405,34 @@ apply) and with Playwright MCP plus a shell (`playwright test`, `browser_navigat
|
|
|
819
405
|
Tokens are characters ÷ 4. The Playwright MCP route is a lower bound: it assumes the agent
|
|
820
406
|
picks the right locator from the snapshot on the first try, which the server does for it here.
|
|
821
407
|
|
|
822
|
-
##
|
|
823
|
-
|
|
824
|
-
| Situation | Behaviour |
|
|
825
|
-
| --- | --- |
|
|
826
|
-
| No Playwright installed | `NO_PLAYWRIGHT` error with the exact install commands for your package manager |
|
|
827
|
-
| Dev server not running | Failure classified `server-unreachable` / `SERVER_NOT_RUNNING` with a "start your dev server" hint (and `webServer` advice) |
|
|
828
|
-
| Test exceeds `timeoutMs` | Process **group** is killed (SIGINT→SIGKILL on POSIX, `taskkill /T /F` on Windows) and partial results are returned |
|
|
829
|
-
| Browser crashes | Classified `browser-crash` with retry / reinstall guidance |
|
|
830
|
-
| Windows backslashes | All paths normalized lexically (`C:\a\..\b` → `C:/b`); unit-tested on both platforms |
|
|
831
|
-
| Flaky tests | Failing tests are automatically retried once (`--retries=1`) before being reported; passes surface as **flaky** with a stability warning; `diagnose-flaky` decides flaky-vs-broken with multi-run evidence |
|
|
832
|
-
| Trace/DOM context | `--trace=retain-on-failure` is passed automatically, so `get-failure` can show the exact DOM at the moment of failure — plus the failed network requests (`*.network` logs) and console errors from the same trace |
|
|
833
|
-
| Slow re-runs after a fix | `run-test` with `lastFailed: true` re-runs only the tests that failed last time (`--last-failed`) |
|
|
834
|
-
| Several `playwright.config` files | Returns a numbered menu (`MULTIPLE_CONFIGS`); pick with `config: "2"` or a path |
|
|
835
|
-
| Syntax error in a spec | `SYNTAX_ERROR` with file:line; nothing crashes; `list-tests` falls back to a source scan |
|
|
836
|
-
| MCP client disconnects | Per-request `AbortSignal` kills the run; stdin end triggers shutdown, and every tracked child tree is force-killed (`killActiveChildren`) |
|
|
837
|
-
| Disk full | `ENOSPC` detected → `DISK_FULL` with a "free space" hint; logging never throws |
|
|
838
|
-
| Malicious paths | `../../etc/passwd`, absolute paths outside the root, a `projectRoot` outside the allowed roots, symlinks that leave the root, URLs and null bytes are rejected with `INVALID_PATH`; extra CLI args must be allowlisted Playwright flags |
|
|
839
|
-
|
|
840
|
-
## Security notes
|
|
841
|
-
|
|
842
|
-
- **No shell**: Playwright is spawned as `node <playwright/cli.js> …` with an argument
|
|
843
|
-
array — no command interpolation. Extra `args` are limited to an allowlist of
|
|
844
|
-
Playwright flags, each with a checked value (`--flag=value`), so a caller cannot
|
|
845
|
-
swap in another `--config` or `--output` or hand git an option through
|
|
846
|
-
`--only-changed`. Paths with a segment starting with `-` are rejected, so a test
|
|
847
|
-
file name cannot be read as a flag.
|
|
848
|
-
- **Path sandbox**: user paths must stay inside the project root, checked lexically
|
|
849
|
-
and again with symlinks resolved. A caller-supplied `projectRoot` must sit inside
|
|
850
|
-
`PW_MCP_PROJECT_ROOT` (or a `PW_MCP_ALLOWED_ROOTS` entry).
|
|
851
|
-
- **Generated code**: `generate-e2e-test` only writes `*.spec.*` / `*.test.*` files,
|
|
852
|
-
only overwrites specs whose header it wrote itself, never writes through a symlink,
|
|
853
|
-
and escapes every value it puts into strings or comments.
|
|
854
|
-
- **HTTP bridge**: read-only tools unless `PW_MCP_HTTP_TOKEN` is set; child processes
|
|
855
|
-
started over HTTP get only an allowlisted environment (`PATH`, `HOME`, temp dirs,
|
|
856
|
-
locale, `CI`, `PLAYWRIGHT_*`, `npm_config_*` without embedded credentials, plus
|
|
857
|
-
anything in `PW_MCP_PASSTHROUGH_ENV`), at most `PW_MCP_MAX_CHILDREN` run at once
|
|
858
|
-
(a disconnected client frees its slot immediately), and internal error messages
|
|
859
|
-
are not returned to clients. The allowlist only covers the child's own
|
|
860
|
-
environment: test code runs as the same OS user, so on Linux it could still read
|
|
861
|
-
the server's startup environment from `/proc`. That is why only token holders
|
|
862
|
-
can run project code; keep other secrets out of the bridge's environment, or run
|
|
863
|
-
it under a separate user.
|
|
864
|
-
- **SSRF guard**: on the HTTP bridge (or with `PW_MCP_BLOCK_PRIVATE_URLS=1`) the URL tools
|
|
865
|
-
refuse hosts that resolve to loopback, private, link-local/metadata or reserved
|
|
866
|
-
addresses, and route the browser through a local proxy that applies the same check
|
|
867
|
-
to every redirect hop and subresource; IPv6 forms that embed an IPv4 address
|
|
868
|
-
(mapped, NAT64, 6to4, Teredo) are blocked too, and WebRTC UDP is disabled so a
|
|
869
|
-
page cannot reach the network around the proxy. It is off for stdio by default because
|
|
870
|
-
opening `http://localhost` dev servers is what these tools are for.
|
|
871
|
-
- **Cleanup**: temp report/script files are written to the OS temp dir and removed;
|
|
872
|
-
child processes are tracked and killed on shutdown. Trace decompression has one
|
|
873
|
-
size budget per archive, and PNGs are capped at 16,384 px per side and 50 M pixels.
|
|
874
|
-
|
|
875
|
-
## Development
|
|
876
|
-
|
|
877
|
-
```
|
|
878
|
-
src/
|
|
879
|
-
├── index.ts # bin entry point (--version/--help, main-module guard)
|
|
880
|
-
├── server.ts # McpServer setup, tool registration, shutdown handling
|
|
881
|
-
├── tools/ # the twelve tools, the probe script + shared plumbing
|
|
882
|
-
├── prompts.ts # fix-failing-test, triage-flaky-tests
|
|
883
|
-
├── utils/ # playwright-runner, report-parser, project-detector, path-utils,
|
|
884
|
-
│ # logger, trace-reader (trace.zip → DOM/network/console),
|
|
885
|
-
│ # image-diff (PNG codec + pixel diff), change-analyzer,
|
|
886
|
-
│ # locator-expr, run-history, run-registry
|
|
887
|
-
└── types/ # shared interfaces and the ErrorKind taxonomy
|
|
888
|
-
```
|
|
889
|
-
|
|
890
|
-
Built on **`@modelcontextprotocol/server` v2** (the `2026-07-28` MCP spec line) with
|
|
891
|
-
Zod v4 standard schemas; every tool declares spec tool annotations.
|
|
892
|
-
|
|
893
|
-
```bash
|
|
894
|
-
npm install
|
|
895
|
-
npm run build # tsc → dist/ (zero errors)
|
|
896
|
-
npm test # build + test/run-tests.mjs (unit tests, any Node ≥20)
|
|
897
|
-
npm run e2e # build + e2e/run.mjs: live MCP ↔ Playwright integration suite
|
|
898
|
-
npm run mcpb # build + bundle dist/ into playwright-e2e-mcp-<version>.mcpb (+ .sha256)
|
|
899
|
-
npm run mcpb:smoke # ...then install that bundle in a temp dir, launch it and assert the handshake
|
|
900
|
-
npm run mcpb:sign # ...then sign it with MCPB_CERT/MCPB_KEY → <version>.signed.mcpb
|
|
901
|
-
npm run skill:zip # just the archive: dist-skill/playwright-e2e-mcp.zip
|
|
902
|
-
npm run skill:package # the uploadable folder: ../agensi-skill-<version>/ (skill copy, README, zip)
|
|
903
|
-
npm run skill:check # build, then fail if a packaged copy has drifted from skill/
|
|
904
|
-
```
|
|
905
|
-
|
|
906
|
-
Tests cover the report parser (sample Playwright JSON, trace attachments), path utils
|
|
907
|
-
(Windows and macOS paths, sandboxing), the project detector (temp-dir fixtures: config
|
|
908
|
-
discovery, multiple configs, missing install, test-file scanning), the shared tool
|
|
909
|
-
helpers, the **trace reader** (synthetic trace.zip: error, failed action, DOM snapshot,
|
|
910
|
-
`*.network` failed-request parsing, console error/warning events), the **image diff**
|
|
911
|
-
(PNG round-trip, regions, color shift, dimension changes), the **change analyzer**
|
|
912
|
-
(selector extraction, git + mtime paths), the **flaky verdict logic**
|
|
913
|
-
(failure signatures, CONSISTENTLY FAILING / FLAKY / NOT REPRODUCING / NO TESTS RAN,
|
|
914
|
-
per-test grouping of repeat attempts and the parallel-vs-alone reading), the `ci`
|
|
915
|
-
argument parser and tool profiles,
|
|
916
|
-
the **HTTP bridge** (token, restricted mode, Host allowlist), the **security
|
|
917
|
-
regressions** (sandbox escapes, argument smuggling, symlink writes, code injection,
|
|
918
|
-
SSRF ranges, env scrubbing, decode limits), the `.mcpb` manifest, the `.mcpb` zip
|
|
919
|
-
extractor, the **skill package** (byte-identical rebuilds, a CRLF checkout hashing the
|
|
920
|
-
same, refusal of junk/binary/symlinks, and drift detection between `skill/` and a
|
|
921
|
-
packaged copy), and the MCP config files.
|
|
922
|
-
|
|
923
|
-
### Integration suite (`npm run e2e`)
|
|
924
|
-
|
|
925
|
-
Unit tests prove the logic; the integration suite proves the loop. It boots the real
|
|
926
|
-
server over stdio against a live fixture app and a Playwright project under
|
|
927
|
-
`e2e/fixture/`, then drives it exactly like an MCP client and asserts ~40 behaviours
|
|
928
|
-
that only appear end-to-end:
|
|
929
|
-
|
|
930
|
-
- initialize handshake, 12 tools, 2 prompts, spec tool annotations and object input schemas,
|
|
931
|
-
- verified locators, locator validation, signed-in pages (`storageState`) and page steps,
|
|
932
|
-
- background runs with progress, run history views, and `suggest-fix` healing a drifted
|
|
933
|
-
locator and verifying it with a re-run (and refusing when the element is missing),
|
|
934
|
-
- live DOM inspection, CSS selector validation (matches, zero matches, engine
|
|
935
|
-
syntax, parse errors), dead-server detection,
|
|
936
|
-
- visual regression: baseline → unchanged compare → `blue → red` diff detection,
|
|
937
|
-
- `run-test` pass/fail/`lastFailed` stats and meta lines,
|
|
938
|
-
- `get-failure` trace diagnostics: DOM at failure, the 404 network request,
|
|
939
|
-
the `console.error` message, diagnosis and next steps,
|
|
940
|
-
- auto-retry turning a first-run failure into `PASSED (1 flaky)`,
|
|
941
|
-
- `diagnose-flaky` verdict **FLAKY** (2 of 3 runs) with retries disabled, in one
|
|
942
|
-
`--repeat-each` process, and `isolate` reading a shared-slot test as "fails only in parallel",
|
|
943
|
-
- `--tools=` profiles trimming `tools/list`, and the `ci` command's exit code, report,
|
|
944
|
-
`--output` file and GitHub job summary,
|
|
945
|
-
- `generate-e2e-test` writing its scaffold, plus error paths
|
|
946
|
-
(missing test path, unknown tool, unreachable server).
|
|
947
|
-
|
|
948
|
-
First run needs the browser once: `npx playwright install chromium`.
|
|
949
|
-
CI runs the suite on Ubuntu and Windows (see `.github/workflows/ci.yml`).
|
|
950
|
-
|
|
951
|
-
### Signing the bundle
|
|
952
|
-
|
|
953
|
-
```bash
|
|
954
|
-
MCPB_CERT=~/signing/cert.pem MCPB_KEY=~/signing/key.pem npm run mcpb:sign
|
|
955
|
-
```
|
|
956
|
-
|
|
957
|
-
Signing appends a PKCS#7 signature block to the bundle, so the signed file is
|
|
958
|
-
*the reproducible bundle plus that block*. That ordering is the point: the bytes
|
|
959
|
-
CI proved identical across platforms stay inside the signed file, and
|
|
960
|
-
`mcpb unsign` returns them exactly. `scripts/sign-mcpb.mjs` asserts that round
|
|
961
|
-
trip on every run and deletes its output if it fails, which is what keeps
|
|
962
|
-
`<version>.signed.mcpb` tied to `<version>.mcpb`.
|
|
963
|
-
|
|
964
|
-
The private key is read from a path you pass in (`--cert`/`--key` or
|
|
965
|
-
`MCPB_CERT`/`MCPB_KEY`) and never from the repository — keep it outside version
|
|
966
|
-
control, and back it up: the same key across releases is what makes the signer
|
|
967
|
-
recognisable.
|
|
968
|
-
|
|
969
|
-
What signing does and does not buy you is decided by the certificate, not by
|
|
970
|
-
this script. `mcpb verify` and `mcpb info` accept a signature only when the
|
|
971
|
-
certificate chains to something in the OS trust store, so with a self-signed
|
|
972
|
-
certificate a host still reports **Not signed** — the signature is real
|
|
973
|
-
integrity and key continuity, verifiable by the fingerprint the script prints,
|
|
974
|
-
but not third-party identity. A certificate issued by a public CA, or an
|
|
975
|
-
internal one installed on your machines, is what makes hosts report it as
|
|
976
|
-
signed.
|
|
977
|
-
|
|
978
|
-
### Assembling a release
|
|
979
|
-
|
|
980
|
-
```bash
|
|
981
|
-
npm run mcpb && npm pack # the bundle, then the npm tarball
|
|
982
|
-
npm run release:assemble -- --tag v0.2.3 # sign, checksum and write the notes
|
|
983
|
-
npm run release:publish -- --tag v0.2.3 --assets dist-release/assets \
|
|
984
|
-
--title-file dist-release/RELEASE_TITLE.txt \
|
|
985
|
-
--body-file dist-release/RELEASE_BODY.md --download dist-release/downloaded
|
|
986
|
-
npm run release:verify -- v0.2.3 --dir dist-release/downloaded
|
|
987
|
-
npm run release:publish -- --tag v0.2.3 --publish
|
|
988
|
-
```
|
|
989
|
-
|
|
990
|
-
Pushing a `v*` tag runs exactly that sequence in
|
|
991
|
-
`.github/workflows/release.yml`. The release used to be assembled by hand, and
|
|
992
|
-
the notes were written from what those commands printed: every hash in them was
|
|
993
|
-
a copy, and nothing compared the bundle hash printed there with the bundle that
|
|
994
|
-
shipped.
|
|
995
|
-
|
|
996
|
-
`scripts/assemble-release.mjs` stages the whole release into
|
|
997
|
-
`dist-release/assets/`: the bundle and its sidecar, the **signed** bundle and the
|
|
998
|
-
signing certificate when a key is configured, the npm tarball, the skill package
|
|
999
|
-
built by the same writer as `npm run skill:zip`, and `SHA256SUMS.txt` covering
|
|
1000
|
-
every file it staged. It empties that directory first, so a file left over from
|
|
1001
|
-
an earlier run cannot be hashed and published as if this run had produced it.
|
|
1002
|
-
|
|
1003
|
-
The notes are half written, half generated. `release-notes/<tag>.md` holds the
|
|
1004
|
-
prose — its H1 becomes the release title and has to name the tag — and the script
|
|
1005
|
-
appends the asset table, the reproduction recipe carrying the real bundle hash,
|
|
1006
|
-
the signer and fingerprint, and the commit list. The prose is held to the same
|
|
1007
|
-
rule: a 64-character hex string in it has to be a hash this release publishes,
|
|
1008
|
-
and writing an `### Assets` section of your own fails the assembly rather than
|
|
1009
|
-
shipping a second, staler table. See [release-notes/README.md](release-notes/README.md).
|
|
1010
|
-
|
|
1011
|
-
`scripts/publish-release.mjs` fills the release **as a draft**, uploads the
|
|
1012
|
-
assets, downloads every one of them back and compares it byte for byte, and only
|
|
1013
|
-
`--publish` makes it public — after `release:verify` has re-derived the release
|
|
1014
|
-
from the tag. Re-running is safe: a draft this workflow assembled is refreshed in
|
|
1015
|
-
place, while a release it did not assemble, or one already published, is refused
|
|
1016
|
-
by name instead of overwritten.
|
|
1017
|
-
|
|
1018
|
-
Signing is optional and its absence is stated rather than hidden: without the
|
|
1019
|
-
`MCPB_KEY` and `MCPB_CERT` repository secrets the release is assembled and
|
|
1020
|
-
published **unsigned**, with a notice in the run and a line saying so in the
|
|
1021
|
-
notes. Add both — the private key stays out of the repository and out of the logs:
|
|
1022
|
-
|
|
1023
|
-
```bash
|
|
1024
|
-
gh secret set MCPB_KEY --repo xsignon-secure/E2E < ~/signing/key.pem
|
|
1025
|
-
gh secret set MCPB_CERT --repo xsignon-secure/E2E < ~/signing/cert.pem
|
|
1026
|
-
```
|
|
1027
|
-
|
|
1028
|
-
Creating the release and uploading its assets needs `contents: write`, which the
|
|
1029
|
-
job requests for itself; if it is refused, the run names the setting to change.
|
|
408
|
+
## Demo
|
|
1030
409
|
|
|
1031
|
-
|
|
410
|
+
A real `run-test` call served over stdio by `npx -y playwright-e2e-mcp` against the bundled
|
|
411
|
+
`examples/sample-test.spec.ts` (actual output, unedited):
|
|
1032
412
|
|
|
1033
|
-
|
|
1034
|
-
git checkout v0.4.2 && npm ci && npm run mcpb
|
|
1035
|
-
npm run release:verify -- v0.4.2
|
|
1036
|
-
```
|
|
413
|
+

|
|
1037
414
|
|
|
1038
|
-
|
|
1039
|
-
byte. `scripts/verify-release.mjs` checks that promise from a checkout of the
|
|
1040
|
-
tag: the published bundle against your fresh build, every checksum the release
|
|
1041
|
-
publishes (and that nothing shippable is missing from `SHA256SUMS.txt`), a
|
|
1042
|
-
**signed** bundle against the bytes its signature wraps, the **skill package**
|
|
1043
|
-
against a rebuild from the same tag, and that the published archive still
|
|
1044
|
-
extracts with that version's manifest. The skill package is required of a tag
|
|
1045
|
-
whose tree carries the publishing workflow; older tags predate it, so there it
|
|
1046
|
-
is checked when present and skipped when not, like the signing certificate.
|
|
1047
|
-
`.github/workflows/verify-release.yml` runs it on demand for any past tag, and by
|
|
1048
|
-
itself for a release published by hand. A release assembled by
|
|
1049
|
-
`.github/workflows/release.yml` runs the same check on its own assets *before*
|
|
1050
|
-
it is published — GitHub does not raise `release: published` for a release created
|
|
1051
|
-
with the repository's own token, so the check is a step there rather than a
|
|
1052
|
-
trigger. The `.tgz` is checked for download integrity only: npm
|
|
1053
|
-
records the packer's line endings, so a tarball packed locally on another OS is
|
|
1054
|
-
legitimately different.
|
|
1055
|
-
|
|
1056
|
-
### Packaging the agent skill
|
|
415
|
+
## Free and open source
|
|
1057
416
|
|
|
1058
|
-
|
|
1059
|
-
|
|
1060
|
-
|
|
1061
|
-
|
|
1062
|
-
|
|
1063
|
-
```
|
|
417
|
+
- **MIT licensed.** Use it, change it, ship it in commercial work; no fees, no paid tier, no feature held back.
|
|
418
|
+
- **No account, no API key, no sign-up.** `npx -y playwright-e2e-mcp init` is the whole setup.
|
|
419
|
+
- **No telemetry.** It never phones home; your tests, traces and run history stay on your machine.
|
|
420
|
+
- **Free dependencies only:** the MCP SDK (Apache-2.0), zod (MIT), axe-core (MPL-2.0) and the Playwright your project already has (Apache-2.0).
|
|
421
|
+
- **Free to host yourself**, too: the HTTP bridge deploys to a free Vercel account (see [Hosted endpoint](docs/hosted-endpoint.md)).
|
|
1064
422
|
|
|
1065
|
-
|
|
1066
|
-
|
|
1067
|
-
|
|
1068
|
-
`scripts/package-skill.mjs` produces that zip, so the uploaded copy is built
|
|
1069
|
-
rather than assembled by hand, and it refuses what the marketplace rejects: a
|
|
1070
|
-
missing `SKILL.md`, a non-text file, a symlink, `Thumbs.db`.
|
|
1071
|
-
|
|
1072
|
-
`skill:package` writes the whole folder a release would otherwise assemble by
|
|
1073
|
-
hand — the skill copy, `skill/README.md` and the archive — to
|
|
1074
|
-
`../agensi-skill-<version>` (or `--out <dir>`), then verifies it with the same
|
|
1075
|
-
comparison `check` uses, so a successful exit means the folder really is the
|
|
1076
|
-
skill tree. It is idempotent: a folder that already matches is left untouched.
|
|
1077
|
-
A folder that has drifted is refused instead of overwritten, because it may
|
|
1078
|
-
already be published — bump `version` for a new package, or pass `--force` to
|
|
1079
|
-
replace that one.
|
|
1080
|
-
|
|
1081
|
-
The archive comes from the same deterministic writer as the `.mcpb`, so the
|
|
1082
|
-
same tree always hashes the same, and text payloads are LF-normalised first:
|
|
1083
|
-
a CRLF checkout builds the bytes an LF one does. CI runs `skill:check`, generates
|
|
1084
|
-
a package folder, and uploads the built zip as an artifact
|
|
1085
|
-
(`.github/workflows/ci.yml`).
|
|
1086
|
-
|
|
1087
|
-
**Every release publishes the package too.** Assembling a release builds it from
|
|
1088
|
-
the release tag and attaches it as `<name>-<version>-skill.zip`, and the
|
|
1089
|
-
`.github/workflows/release.yml` run downloads that asset back off the draft and
|
|
1090
|
-
byte-compares it with the build before anything is published. The marketplace
|
|
1091
|
-
upload is that asset, not a folder someone assembled — a deterministic archive
|
|
1092
|
-
means "the same build as the tagged release" is an exact statement, and
|
|
1093
|
-
`release:verify` checks it again against a rebuild from the tag.
|
|
1094
|
-
`.github/workflows/publish-skill-package.yml` stays for the repair path: run it
|
|
1095
|
-
by hand to attach or rebuild the package for a release that is missing it.
|
|
1096
|
-
|
|
1097
|
-
`skill:check` is the other half of that, and the part that matters when a
|
|
1098
|
-
package has already been uploaded. It builds twice (the bytes must agree),
|
|
1099
|
-
reads the archive back and compares its contents with `skill/` file by file,
|
|
1100
|
-
then compares a packaged copy — `../agensi-skill-<version>` by default, or a
|
|
1101
|
-
path you pass: the package folder, the bare `playwright-e2e-mcp/` folder, or
|
|
1102
|
-
the zip on its own. Comparisons are by content, never by archive bytes, because
|
|
1103
|
-
a zip written by a platform `zip` tool carries real timestamps that say nothing
|
|
1104
|
-
about whether the Markdown inside it is current. Any file that differs, is
|
|
1105
|
-
missing or is extra is named on stderr and the command exits 1 — the repository
|
|
1106
|
-
copy and the uploaded copy are the same document, so an edit to either one
|
|
1107
|
-
should not be invisible.
|
|
1108
|
-
|
|
1109
|
-
### Try the example
|
|
1110
|
-
|
|
1111
|
-
With Playwright installed in your project:
|
|
423
|
+
**What it does not do:** it doesn't drive an exploratory browsing session step by step, and it
|
|
424
|
+
doesn't replace Playwright's own Test Agents inside VS Code. Use Playwright MCP for interactive
|
|
425
|
+
browsing and this server for running, diagnosing and fixing the suite. They work side by side.
|
|
1112
426
|
|
|
1113
|
-
|
|
1114
|
-
npx playwright test examples/sample-test.spec.ts
|
|
1115
|
-
```
|
|
427
|
+
## More documentation
|
|
1116
428
|
|
|
1117
|
-
|
|
1118
|
-
|
|
1119
|
-
[
|
|
1120
|
-
|
|
429
|
+
- [Tool reference](docs/tools.md): every argument, example outputs, edge cases handled
|
|
430
|
+
- [Hosted endpoint](docs/hosted-endpoint.md): the HTTP bridge for ChatGPT and claude.ai connectors
|
|
431
|
+
- [Security](SECURITY.md): the security model and how to report a vulnerability
|
|
432
|
+
- [Development](docs/development.md): building, tests, Docker, signing and releasing
|
|
433
|
+
- [Contributing](CONTRIBUTING.md) · [Code of conduct](CODE_OF_CONDUCT.md)
|
|
1121
434
|
|
|
1122
435
|
## License
|
|
1123
436
|
|
|
1124
|
-
MIT
|
|
437
|
+
[MIT](LICENSE). If this saves you a debugging session, a ⭐ on the repo helps other Playwright
|
|
438
|
+
users find it.
|