playwright-e2e-mcp 0.9.0 → 0.11.0

package/README.md

@@ -64,9 +64,11 @@ red at HEAD ─▶ bisect-failure (runs it across git history, names the breakin
 - **Works on CI failures too.** Load a CI job's report and traces and get the same diagnosis and fixes.
 - **TypeScript and Python.** Works on `@playwright/test` suites and on `pytest-playwright` ones, with the same failure diagnosis and traces.
 - **Flakiness with evidence.** Names the flaky test, whether it's a race, shared state or test order, and the seed that reproduces it.
-- **Lean.** With the `core` tool set, finding and fixing a broken locator took about **5,500 tokens**
-  versus about **7,900** with Playwright MCP plus a shell. All 19 tools add about 16k tokens of
-  definitions to every request, so load only the [profiles](#configuration) you use ([benchmark](#benchmark)).
+- **Lean.** With the `core` tool set, finding and fixing a broken locator took about **3,000 tokens**
+  versus about **7,200** with Playwright MCP plus a shell, and a look at a page costs about 570 tokens
+  against 2,300 for a Playwright MCP snapshot. All 19 tools add about 11.5k tokens of definitions to
+  every request, so load only the [profiles](#configuration) you use, or start with `--tools=auto`
+  (about 2.4k) and let the agent load the rest when it needs them ([benchmark](#benchmark)).
 - **Free and open source.** MIT, no account, no API key, no telemetry. Every tool and feature is free, including the shared team history server you can host yourself.
 
 ## 🚀 What only this MCP does
@@ -190,7 +192,7 @@ or go to **Settings → MCP → Add new MCP server** and paste the standard conf
 project only, put it in `.cursor/mcp.json`.
 
 Only fixing failing tests? [Install the core tools only](https://cursor.com/en/install-mcp?name=playwright-e2e&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsInBsYXl3cmlnaHQtZTJlLW1jcCIsIi0tdG9vbHM9Y29yZSJdfQ==): the seven-tool fix loop
-(`--tools=core`), about 5k tokens of tool definitions instead of 16k.
+(`--tools=core`), about 2.3k tokens of tool definitions instead of 11.5k.
 
 </details>
 
@@ -273,8 +275,8 @@ Running and fixing your own tests needs your project, so use the local install f
 | --- | --- |
 | Skill only, any agent | `npx skills add xsignon-secure/E2E` |
 | MCP Registry | `io.github.xsignon-secure/E2E` ([listing](https://registry.modelcontextprotocol.io/)); registry-aware clients find it there |
-| Straight from GitHub, no npm | `npx -y github:xsignon-secure/E2E#v0.9.0` (pin a release tag) |
-| Release tarball | `npm install -D https://github.com/xsignon-secure/E2E/releases/download/v0.9.0/playwright-e2e-mcp-0.9.0.tgz` |
+| Straight from GitHub, no npm | `npx -y github:xsignon-secure/E2E#v0.11.0` (pin a release tag) |
+| Release tarball | `npm install -D https://github.com/xsignon-secure/E2E/releases/download/v0.11.0/playwright-e2e-mcp-0.11.0.tgz` |
 | Docker | `docker run -i --rm -v "$PWD:/work" ghcr.io/xsignon-secure/playwright-e2e-mcp` (published on each release; see [Development](docs/development.md#docker)) |
 | Local checkout | `"command": "node", "args": ["/path/to/E2E/dist/index.js"]` |
 
@@ -338,6 +340,30 @@ in the project, else `python3` / `python`.
 pip install pytest-playwright && playwright install chromium
 ```
 
+## What's new in 0.11
+
+- **Fewer tokens.** `inspect-page` now returns the compact locator map by default: on the
+  benchmark storefront a page look costs about 570 tokens instead of 3,600 (Playwright MCP's
+  snapshot of the same page is about 2,300). Repeated blocks such as product cards are shown
+  once. `view: 'elements'` still gives the full DOM inventory.
+- **Load tools on request.** `--tools=auto` (or `PW_MCP_TOOLS=auto`) starts with the core fix
+  loop plus `load-tools`, about 2.4k tokens of tool definitions instead of 11.5k; the agent
+  loads the page, audit or history tools when a task needs them.
+- **Smaller tool definitions.** Input schemas are listed without length and item limits; the
+  server still checks every call against them.
+
+## What's new in 0.10
+
+- **Asks before changing your files.** Applying a fix, writing a new test, quarantining a test or
+  replacing a visual baseline now shows you the exact change and waits for your yes, in clients
+  that support MCP questions (elicitation). No writes nothing. `PW_MCP_CONFIRM_WRITES` sets the
+  policy (`auto`, `always`, `off`).
+- **Interactive test report** in Claude, ChatGPT, Cursor and other MCP Apps clients: counts,
+  a filterable list of failures, expected vs. received, and one-click re-run or "Ask to fix".
+  Other clients show the text result as before. `PW_MCP_APPS=0` turns it off.
+- **Newer sign-in for the hosted endpoint:** OAuth clients can identify with a Client ID
+  Metadata Document (the MCP default since 2025-11-25); dynamic registration keeps working.
+
 ## What's new in 0.9
 
 - **Everything is free, including team history.** The CI history service that used to be a
@@ -423,6 +449,8 @@ prompts.
 
 **Only need the fix loop?** `--tools=core` (or `PW_MCP_TOOLS=core`) exposes seven tools and cuts the
 tool-definition context by about three quarters ([one-click core install for Cursor](https://cursor.com/en/install-mcp?name=playwright-e2e&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsInBsYXl3cmlnaHQtZTJlLW1jcCIsIi0tdG9vbHM9Y29yZSJdfQ==)). Other profiles: `triage`, `inspect`, `author`.
+Want the savings without losing tools? `--tools=auto` starts with the same seven plus `load-tools`,
+which the agent calls to turn on the page, audit or history tools when a task needs them.
 
 <details>
 <summary><b>Typical workflow</b></summary>
@@ -481,7 +509,7 @@ steps:
   - uses: actions/setup-node@v6
     with: { node-version: 24 }
   - run: npm ci && npx playwright install --with-deps chromium
-  - uses: xsignon-secure/E2E@v0.9.0
+  - uses: xsignon-secure/E2E@v0.11.0
     with:
       flaky-runs: 3      # 0 skips the flaky verdict
       suggest: 2         # suggest-fix diffs for the first 2 failures
@@ -521,10 +549,12 @@ npx -y playwright-e2e-mcp guard --base origin/main   # exit 1 on BLOCK; --strict
 | Environment variable | Default | Purpose |
 | --- | --- | --- |
 | `PW_MCP_PROJECT_ROOT` | server cwd | Default project root for every tool |
-| `PW_MCP_TOOLS` | all | Which tools to expose, as tool names and/or profiles, comma separated (same as the `--tools=` flag). Profiles: `core` (run, status, failure, fix, flaky, heal guard, list), `triage` (run, status, failure, CI report, flaky, history, test audit, bisect, quarantine, heal guard, list), `inspect` (page, selector, accessibility, performance, visual, login), `author` (list, page, selector, generate, run, login, heal guard). All nineteen tool definitions cost about 16k tokens of context per request; `core` costs about 5k (12k and 2.6k with `PW_MCP_OUTPUT_SCHEMAS=0`) |
+| `PW_MCP_TOOLS` | all | Which tools to expose, as tool names and/or profiles, comma separated (same as the `--tools=` flag). Profiles: `core` (run, status, failure, fix, flaky, heal guard, list), `triage` (run, status, failure, CI report, flaky, history, test audit, bisect, quarantine, heal guard, list), `inspect` (page, selector, accessibility, performance, visual, login), `author` (list, page, selector, generate, run, login, heal guard). `auto` starts with the core loop (or the tools listed with it, e.g. `auto,inspect`) plus a `load-tools` tool the agent calls to turn on more tools for the session; the client is told the tool list changed. `auto` needs a client that refreshes its tool list (Claude Code, Cursor, VS Code); over the hosted HTTP endpoint it lists every tool. All nineteen tool definitions cost the model about 11.5k tokens per request and `core` about 2.3k (the full `tools/list`, with output schemas most clients do not show the model, is about 16k and 4.8k) |
 | `PW_MCP_OUTPUT_SCHEMAS` | on | `0` leaves the output schemas out of the tool list to save context. Results still carry `structuredContent` (see [Structured results](docs/tools.md#structured-results)) |
 | `PW_MCP_PROTOCOL` | both | `legacy` stops offering the stateless 2026-07-28 MCP revision, for a client that trips over it. By default 2026 clients get it and 2025 clients get the old handshake |
 | `PW_MCP_NO_UPDATE_CHECK` | off | `1` stops `doctor` from asking npm whether a newer version is out (same as `doctor --offline`) |
+| `PW_MCP_CONFIRM_WRITES` | `auto` | Ask before a tool changes your files (`suggest-fix` apply, `audit-tests` fix, a new test from `generate-e2e-test`, quarantine add/remove, replacing a visual baseline). `auto` asks when the client can show a question (MCP elicitation) and writes as before when it can't; `always` refuses to write when the client can't ask; `off` never asks. The question shows the exact diff, and a yes only covers that diff |
+| `PW_MCP_APPS` | `1` | Show an interactive test report next to `run-test`, `get-run-status` and `get-failure` results in clients that support MCP Apps (Claude, ChatGPT, Cursor). The report filters failures, shows expected vs. received, and re-runs or asks to fix a test with one click. `0` turns it off; other clients show the text result as before |
 | `PW_MCP_SKILLS` | on | `0` stops serving the bundled agent skill over the MCP Skills extension (`skill://playwright-e2e-mcp/SKILL.md`) |
 | `PW_MCP_STORAGE_STATE` | — | Default `storageState` file (inside the project) for the page tools |
 | `PW_MCP_PYTHON` | project venv, then `python3` | The Python that runs a pytest-playwright suite |
@@ -538,6 +568,7 @@ npx -y playwright-e2e-mcp guard --base origin/main   # exit 1 on BLOCK; --strict
 | `PW_MCP_ALLOWED_HOSTS` | localhost + Vercel hostnames when there is no token | HTTP bridge only: comma-separated `Host` allowlist; `*` accepts any |
 | `PW_MCP_MAX_FAILED_AUTH` | `20` | HTTP bridge only: wrong tokens or access keys one client address may send in 15 minutes before it gets `429` |
 | `PW_MCP_OAUTH_REDIRECT_HOSTS` | any https host | HTTP bridge only: comma-separated hosts OAuth apps may return to, e.g. `claude.ai,chatgpt.com`; `.example.com` covers subdomains; localhost always works |
+| `PW_MCP_OAUTH_CIMD` | `1` | HTTP bridge only: accept OAuth clients that identify with a Client ID Metadata Document (an https `client_id`, the MCP default since 2025-11-25); `0` accepts only dynamically registered clients |
 | `PW_MCP_TRUST_PROXY` | off (on for Vercel) | HTTP bridge only: `1` trusts `X-Real-IP` / `X-Forwarded-For` from a reverse proxy in front, so the sign-in limit counts per client, not per proxy |
 | `PW_MCP_PASSTHROUGH_ENV` | — | HTTP bridge only: comma-separated extra variables passed to test runs (e.g. `BASE_URL`) |
 | `PW_MCP_MAX_CHILDREN` | `4` | HTTP bridge only: how many test runs and browser probes may run at once |
@@ -566,18 +597,26 @@ apply) and with Playwright MCP plus a shell (`playwright test`, `browser_navigat
 
 | Step | playwright-e2e-mcp (`core` tools) | playwright-e2e-mcp (all tools) | Playwright MCP + shell |
 | --- | ---: | ---: | ---: |
-| Tool definitions (sent with every request) | 4,826 | 16,121 | 5,225 |
-| Run the failing test | 244 | 244 | 323 |
+| Tool definitions the model reads (every request) | 2,266 | 11,480 | 4,558 |
+| Run the failing test | 257 | 257 | 323 |
 | Find the element / propose the fix | 232 | 232 | 2,286 |
 | Apply + verify (re-run) | 230 | 230 | 34 |
-| **Total for one fix** | 5,532 | 16,827 | 7,868 |
-
-Tokens are characters ÷ 4. The work itself (run, diagnose, fix, verify) takes about 700 tokens
-here versus about 2,600 with Playwright MCP, but tool definitions are sent with every request:
-with all 19 tools loaded they outweigh that saving, so set `PW_MCP_TOOLS=core` (or another
-profile) when you only fix failing tests. The Playwright MCP route is a lower bound: it assumes
-the agent picks the right locator from the snapshot on the first try, which the server does for
-it here. Run 2026-10-28 with 0.7.0 and Playwright 1.64.1; re-run with
+| **Total for one fix** | 2,985 | 12,199 | 7,201 |
+
+| Looking at the storefront page | Tokens |
+| --- | ---: |
+| `inspect-page` (default: locator map) | 571 |
+| Playwright MCP `browser_navigate` + `browser_snapshot` | 2,286 |
+
+Tokens are characters ÷ 4. Tool definitions count what the model reads: each tool's name,
+description and input schema. The full `tools/list` also carries output schemas, which clients
+keep for themselves (about 16k for all tools). The work itself (run, diagnose, fix, verify)
+takes about 700 tokens here versus about 2,600 with Playwright MCP, but tool definitions are
+sent with every request: with all 19 tools loaded they outweigh that saving, so set
+`PW_MCP_TOOLS=core` (or another profile) when you only fix failing tests, or `PW_MCP_TOOLS=auto`
+to start with the core loop (2,427 tokens) and load more tools on request. The Playwright MCP
+route is a lower bound: it assumes the agent picks the right locator from the snapshot on the
+first try, which the server does for it here. Run 2026-11-01 with Playwright 1.64.1; re-run with
 `npm run build && node bench/token-benchmark.mjs`.
 
 ## Demo

package/dist/apps/index.d.ts

@@ -0,0 +1,29 @@
+/**
+ * MCP Apps (SEP-1865, extension `io.modelcontextprotocol/ui`): an
+ * interactive test report that Claude, ChatGPT, Cursor and other MCP Apps
+ * hosts render next to the run-test, get-run-status and get-failure results.
+ *
+ * The tools say which view to show in their `_meta.ui.resourceUri`; the
+ * host reads the `ui://` resource, renders it in a sandboxed iframe and
+ * hands it the tool result. Hosts that do not support MCP Apps ignore the
+ * metadata and show the text result exactly as before, so nothing changes
+ * for them.
+ *
+ * The view makes no network requests (it declares no CSP domains) and only
+ * calls this server's own tools through the host.
+ *
+ * Set PW_MCP_APPS=0 to turn it off.
+ */
+import type { McpServer } from '@modelcontextprotocol/server';
+/** The MCP Apps resource MIME type (`text/html;profile=mcp-app`). */
+export declare const MCP_APP_MIME_TYPE = "text/html;profile=mcp-app";
+/** Older hosts read the view from this flat key (`ui/resourceUri`). */
+export declare const LEGACY_RESOURCE_URI_KEY = "ui/resourceUri";
+export declare const TEST_REPORT_URI = "ui://playwright-e2e-mcp/test-report.html";
+/** Tools whose results the report view can show. */
+export declare const REPORT_TOOLS: ReadonlySet<string>;
+export declare function appsEnabled(): boolean;
+/** The `_meta` a tool declares so MCP Apps hosts show the report, or undefined. */
+export declare function appMetaFor(toolName: string): Record<string, unknown> | undefined;
+/** Register the report view resource. A no-op when PW_MCP_APPS=0. */
+export declare function registerApps(server: McpServer): void;

package/dist/apps/index.js

@@ -0,0 +1,52 @@
+/**
+ * MCP Apps (SEP-1865, extension `io.modelcontextprotocol/ui`): an
+ * interactive test report that Claude, ChatGPT, Cursor and other MCP Apps
+ * hosts render next to the run-test, get-run-status and get-failure results.
+ *
+ * The tools say which view to show in their `_meta.ui.resourceUri`; the
+ * host reads the `ui://` resource, renders it in a sandboxed iframe and
+ * hands it the tool result. Hosts that do not support MCP Apps ignore the
+ * metadata and show the text result exactly as before, so nothing changes
+ * for them.
+ *
+ * The view makes no network requests (it declares no CSP domains) and only
+ * calls this server's own tools through the host.
+ *
+ * Set PW_MCP_APPS=0 to turn it off.
+ */
+import { TEST_REPORT_HTML } from './test-report-html.js';
+/** The MCP Apps resource MIME type (`text/html;profile=mcp-app`). */
+export const MCP_APP_MIME_TYPE = 'text/html;profile=mcp-app';
+/** Older hosts read the view from this flat key (`ui/resourceUri`). */
+export const LEGACY_RESOURCE_URI_KEY = 'ui/resourceUri';
+export const TEST_REPORT_URI = 'ui://playwright-e2e-mcp/test-report.html';
+/** Tools whose results the report view can show. */
+export const REPORT_TOOLS = new Set(['run-test', 'get-run-status', 'get-failure']);
+export function appsEnabled() {
+    return process.env.PW_MCP_APPS !== '0';
+}
+/** The `_meta` a tool declares so MCP Apps hosts show the report, or undefined. */
+export function appMetaFor(toolName) {
+    if (!appsEnabled() || !REPORT_TOOLS.has(toolName))
+        return undefined;
+    return { ui: { resourceUri: TEST_REPORT_URI }, [LEGACY_RESOURCE_URI_KEY]: TEST_REPORT_URI };
+}
+/** Rendering hints for the host: a border, and no outside origins at all. */
+const RESOURCE_UI_META = {
+    prefersBorder: true,
+    csp: { connectDomains: [], resourceDomains: [] },
+};
+/** Register the report view resource. A no-op when PW_MCP_APPS=0. */
+export function registerApps(server) {
+    if (!appsEnabled())
+        return;
+    server.registerResource('test-report', TEST_REPORT_URI, {
+        title: 'Interactive test report',
+        description: 'The view MCP Apps hosts show for run-test, get-run-status and get-failure results.',
+        mimeType: MCP_APP_MIME_TYPE,
+        _meta: { ui: RESOURCE_UI_META },
+    }, async () => ({
+        contents: [{ uri: TEST_REPORT_URI, mimeType: MCP_APP_MIME_TYPE, text: TEST_REPORT_HTML, _meta: { ui: RESOURCE_UI_META } }],
+    }));
+}
+//# sourceMappingURL=index.js.map

package/dist/apps/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/apps/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;GAeG;AAGH,OAAO,EAAE,gBAAgB,EAAE,MAAM,uBAAuB,CAAC;AAEzD,qEAAqE;AACrE,MAAM,CAAC,MAAM,iBAAiB,GAAG,2BAA2B,CAAC;AAE7D,uEAAuE;AACvE,MAAM,CAAC,MAAM,uBAAuB,GAAG,gBAAgB,CAAC;AAExD,MAAM,CAAC,MAAM,eAAe,GAAG,0CAA0C,CAAC;AAE1E,oDAAoD;AACpD,MAAM,CAAC,MAAM,YAAY,GAAwB,IAAI,GAAG,CAAC,CAAC,UAAU,EAAE,gBAAgB,EAAE,aAAa,CAAC,CAAC,CAAC;AAExG,MAAM,UAAU,WAAW;IACzB,OAAO,OAAO,CAAC,GAAG,CAAC,WAAW,KAAK,GAAG,CAAC;AACzC,CAAC;AAED,mFAAmF;AACnF,MAAM,UAAU,UAAU,CAAC,QAAgB;IACzC,IAAI,CAAC,WAAW,EAAE,IAAI,CAAC,YAAY,CAAC,GAAG,CAAC,QAAQ,CAAC;QAAE,OAAO,SAAS,CAAC;IACpE,OAAO,EAAE,EAAE,EAAE,EAAE,WAAW,EAAE,eAAe,EAAE,EAAE,CAAC,uBAAuB,CAAC,EAAE,eAAe,EAAE,CAAC;AAC9F,CAAC;AAED,6EAA6E;AAC7E,MAAM,gBAAgB,GAAG;IACvB,aAAa,EAAE,IAAI;IACnB,GAAG,EAAE,EAAE,cAAc,EAAE,EAAE,EAAE,eAAe,EAAE,EAAE,EAAE;CACjD,CAAC;AAEF,qEAAqE;AACrE,MAAM,UAAU,YAAY,CAAC,MAAiB;IAC5C,IAAI,CAAC,WAAW,EAAE;QAAE,OAAO;IAC3B,MAAM,CAAC,gBAAgB,CACrB,aAAa,EACb,eAAe,EACf;QACE,KAAK,EAAE,yBAAyB;QAChC,WAAW,EAAE,oFAAoF;QACjG,QAAQ,EAAE,iBAAiB;QAC3B,KAAK,EAAE,EAAE,EAAE,EAAE,gBAAgB,EAAE;KAChC,EACD,KAAK,IAAI,EAAE,CAAC,CAAC;QACX,QAAQ,EAAE,CAAC,EAAE,GAAG,EAAE,eAAe,EAAE,QAAQ,EAAE,iBAAiB,EAAE,IAAI,EAAE,gBAAgB,EAAE,KAAK,EAAE,EAAE,EAAE,EAAE,gBAAgB,EAAE,EAAE,CAAC;KAC3H,CAAC,CACH,CAAC;AACJ,CAAC"}

package/dist/apps/test-report-html.d.ts

@@ -0,0 +1,15 @@
+/**
+ * The interactive test report view (MCP Apps, SEP-1865).
+ *
+ * One self-contained HTML page with no external requests: the host loads it
+ * in a sandboxed iframe and talks to it with JSON-RPC over postMessage. The
+ * page speaks the small part of the MCP Apps protocol it needs directly
+ * (ui/initialize, tool-input, tool-result, tools/call, ui/message,
+ * size-changed), so it ships no framework and no bundle.
+ *
+ * Everything from the tool result is written with textContent, never as
+ * HTML: test titles and error messages come from the project under test.
+ *
+ * Kept as a TypeScript string so `tsc` ships it in dist/ with the server.
+ */
+export declare const TEST_REPORT_HTML: string;