cdp-toolkit 2.2.0 → 2.5.0

package/CHANGELOG.md

@@ -5,6 +5,39 @@ All notable changes to cdp-toolkit are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [2.5.0] - 2026-09-12
+
+### Added
+
+- **`extract_page { prompt }` — a second prompt shape, for locally served open-weight Schematron.** The tool used to send the cleaned HTML as the ONLY user message and let `response_format: json_schema strict` carry the schema; that is right for the HOSTED Schematron API, which injects the schema into the prompt server-side, and wrong for open weights served locally (e.g. an MLX `pchamart/schematron8B-mlx-8bit` lane), which were fine-tuned on a prompt with the schema INSIDE the user message and return garbage from a bare-HTML prompt even under constrained decoding. `prompt:"schematron"` sends the `inference-net/Schematron-8B` model card's own messages verbatim — a `You are a helpful assistant` system message plus a user message with the compact `JSON.stringify`'d schema, the HTML, and the `MAKE SURE ITS VALID JSON.` tail. `prompt:"html"` (the default) is byte-identical to the previous wire shape; `CDP_EXTRACT_PROMPT` sets the default and the per-call arg wins over it; anything else is rejected before any page or network work. `response_format` is still sent in BOTH modes (llguidance-constrained decoding on mlx_vlm honours it, and it is harmless elsewhere). The schema text is charged to the prompt on top of the HTML — `maxChars` caps the HTML only, and the real number comes back in `usage.promptTokens`. Five new unit tests pin both wire shapes, the env/arg precedence and the rejection (`test/extract.test.ts`).
+
+### Fixed
+
+- **`extract_page` names an output-cap truncation for what it is.** A completion the endpoint cut off at `max_tokens` (8192) used to surface as `endpoint returned invalid JSON (Unterminated string)`, which reads as a model or quoting defect and sends the operator debugging the wrong layer. The tool now reads `choices[0].finish_reason`: `length` fails with a dedicated message naming the cap, the completion-token count, and the remedy (narrow the payload with `selector`, or ask the schema for less); any other parse failure carries the `finish_reason` alongside the excerpt. Two new unit tests pin both paths (`test/extract.test.ts`, 24).
+
+## [2.4.0] - 2026-09-12
+
+**Structured extraction: pages come back as schema-conformant JSON, not prose.** A new `extract_page` tool takes the target page's HTML, cleans it in-page (scripts, styles, hidden elements and other non-content noise stripped), and sends it to an OpenAI-compatible `/chat/completions` endpoint along with your JSON Schema, returning JSON that matches the schema instead of a paragraph you have to parse. The default endpoint is **loopback llm-ferry** (`http://127.0.0.1:8090/v1`, model `schematron`), so page content stays on this machine unless the operator points `CDP_EXTRACT_BASE_URL` elsewhere; the API key is never logged and redacted from every error, and `usage` token counts (plus cost when the endpoint reports one) come back inline on every call. This is prompt-driven extraction: the extraction instructions live in the schema's property `description`s, and a schema whose properties carry no descriptions is rejected up front rather than extracting garbage. A cleaned payload over `maxChars` fails with `html_too_large` naming the actual size instead of silently truncating — shrink with `selector`/`clean:"aggressive"` or raise the cap.
+
+### Added
+
+- **`extract_page`** — cleaned page HTML → OpenAI-compatible extraction endpoint → schema-conformant JSON. Params: `schema` (required, both the contract and the prompt), `selector` (subtree scoping, pierces open shadow roots), `source` (`dom` default; `served` is a not-implemented error, not a guess), `clean` (`standard`/`aggressive`/`none`), `maxChars`, `timeoutMs`, `model`, `baseUrl`, and `savePath` (write the JSON to a file; the response carries the path and usage counts only). Errors are distinguishable by name: `html_too_large`, `upstream_error` (status + body, key redacted), `timeout`. New env knobs `CDP_EXTRACT_BASE_URL` / `CDP_EXTRACT_MODEL` / `CDP_EXTRACT_API_KEY` / `CDP_EXTRACT_TIMEOUT_MS` / `CDP_EXTRACT_MAX_CHARS`.
+- New profile group **`extraction`** (14 groups total): `CDP_TOOL_PROFILE` can now name `extraction`, and the default `full` listing grows to 49 entries (48 tools + `describe_tool`).
+- **Unit tests `test/extract.test.ts` (22)** pinning schema validation (must be a `type:"object"` JSON Schema, `additionalProperties:true` refused, the property-description rule), every argument check (`selector`/`source`/`clean`/`maxChars`/`timeoutMs`/`model`/`baseUrl`), the `served` not-implemented error, `html_too_large`, and API-key redaction.
+- New live harness **`test/extract-smoke.ts`** (`bun run extract:smoke`): a real page through the real loopback endpoint, proving the schema-conformant JSON round trip and the guard errors end to end.
+- **Firefox wiring.** `extract_page` is backend-neutral by construction (its page work runs entirely through the driver's evaluate, no CDP-only capability), so it joins the Firefox registry: 35 of the 48 tools under `CDP_BROWSER=firefox`, 36 entries in the default listing.
+
+## [2.3.0] - 2026-09-10
+
+**Drive focus-gated pages on a background tab without stealing OS window focus.** Some sites gate a button behind `document.hasFocus()`/`visibilitychange` — the canonical case is Anthropic's Claude Code OAuth page (`claude.ai/oauth/*`), whose **Authorize** button stays disabled on a background tab, forcing an agent driving `claude -p` login to keep a human clicking. Two new Chrome-only tools port the mechanism of the [`danbuhler/claude-code-auto-authorize`](https://github.com/danbuhler/claude-code-auto-authorize) extension: `Emulation.setFocusEmulationEnabled` makes the page *believe* it is focused (hasFocus→true, visibility stays `visible`, focus events fire) while the real window stays put, and `Input.dispatchMouseEvent` delivers the **trusted** click (`isTrusted:true`) that a content script's synthesized events can never produce.
+
+### Added
+
+- **`focus_emulation { enabled }`** — the raw toggle: a tab reports focused/visible with no OS focus change. Call with `enabled:false` to restore.
+- **`click_focus_gated { selector|text, timeoutMs, pollMs, keepFocus }`** — the one-call compose: enable focus emulation, poll until the button (CSS selector or exact visible-text, e.g. `"Authorize"`) exists **and** is enabled, trusted-click it, restore emulation (always, unless `keepFocus`, even on throw). Timeout error names the failure mode ("never located" vs "found but stayed disabled").
+- New capability **`emulate.focus`** (Chrome declares it; BiDi's emulation module has no focus primitive, so both tools are absent from `tools/list` under `--browser firefox`, per ADR-001).
+- New live harness **`test/focus-emulation-smoke.ts`** (`bun run focus:smoke`): a real headless Chrome serves a focus-gated Authorize fixture (disabled until `hasFocus()`), proves the gate is closed in a background tab (control), then `click_focus_gated` opens it and lands a trusted click — page log shows `gate:unlocked` + `CLICKED trusted=true` with the tab never activated. Plus unit tests `test/focus-emulation.test.ts` (16) pinning validation, capability gating, and manifest/docs/group registration.
+
 ## [2.2.0] - 2026-09-04
 
 **Several agent PROCESSES can now drive tabs of ONE user-launched Firefox at the same time.** Firefox serves exactly one WebDriver BiDi session per browser (measured on 153.0.3: a second `session.new` gets "Maximum number of active sessions", a second WebSocket to `/session/<id>` is refused at the HTTP upgrade). Previously a second cdp-toolkit process attaching to the same Firefox was refused after the session-slot wait; now the process that wins the slot fronts the one real session with a loopback BiDi multiplexer, and every other process joins it instead of waiting. **Measured** (`bun run firefox:multi:smoke`, headless Firefox 153.0.3, 4 agents × 6 rounds): before, 4 server processes attached to one Firefox — 1 of 4 completed, the other 3 refused after the session wait; after, all 4 completed concurrently (40 calls, wall 437ms vs serial 1375ms, 4 distinct processes in flight), a killed server process left 3 survivors finishing with 0 retries, and a killed daemon was replaced within the recovery path with every survivor finishing on its original tab via the stale-id hint (`firefox:coord:smoke` 11/11, `bun test` 929 pass). See [`docs/design/2026-09-04-firefox-multi-agent-session-mux.md`](docs/design/2026-09-04-firefox-multi-agent-session-mux.md).

package/README.md

@@ -1,6 +1,6 @@
 # cdp-toolkit
 
-**A lightweight, drop-in alternative to [`chrome-devtools-mcp`](https://github.com/ChromeDevTools/chrome-devtools-mcp) that won't wedge your agent.** It drives the Chrome tabs you point it at over the raw DevTools Protocol: any number of tabs, one explicitly named target per call over one direct socket, with a bounded timeout on every call, so a stuck page returns a clean error instead of hanging your agent and forcing a `/mcp` restart. Same idea, no all-target fan-out, plus tab leases so several agents can work one browser (and know when a human is using it too), plus a built-in network-mocking fake backend. **45 tools, all of them on one static, cacheable MCP listing** — or a lean 12-tool `core` listing when your client loads every schema eagerly (see [Progressive disclosure](#progressive-disclosure)). Chrome is the flagship default; a second backend drives Firefox over WebDriver BiDi behind the same tool surface (see below).
+**A lightweight, drop-in alternative to [`chrome-devtools-mcp`](https://github.com/ChromeDevTools/chrome-devtools-mcp) that won't wedge your agent.** It drives the Chrome tabs you point it at over the raw DevTools Protocol: any number of tabs, one explicitly named target per call over one direct socket, with a bounded timeout on every call, so a stuck page returns a clean error instead of hanging your agent and forcing a `/mcp` restart. Same idea, no all-target fan-out, plus tab leases so several agents can work one browser (and know when a human is using it too), plus a built-in network-mocking fake backend. **48 tools, all of them on one static, cacheable MCP listing** — or a lean 12-tool `core` listing when your client loads every schema eagerly (see [Progressive disclosure](#progressive-disclosure)). Chrome is the flagship default; a second backend drives Firefox over WebDriver BiDi behind the same tool surface (see below).
 
 > For AI-agent developers and Claude Code / Cursor users who need **the tabs they name driven reliably**, not a Puppeteer-managed browser.
 
@@ -61,7 +61,7 @@ If [`chrome-devtools-mcp`](https://github.com/ChromeDevTools/chrome-devtools-mcp
 npx -y --package cdp-toolkit cdp install     # or `cdp install` from a clone / after a global install
 ```
 
-It asks for a harness (`claude` | `codex` | `opencode`), a debug port, and a browser (`arc` | `chrome` | `firefox`), then writes an idempotent marker block into your `~/.zshrc`/`~/.bashrc`. The same flags drive it non-interactively: `--harness --browser --port --name --no-alias --yes`. The Firefox alias it writes includes `--marionette` (required for cdp-toolkit's orphaned-session auto-recovery — see [Firefox](#firefox-webdriver-bidi)). It is a CLI subcommand, **not** a 46th tool: `cdp --list` still shows 45.
+It asks for a harness (`claude` | `codex` | `opencode`), a debug port, and a browser (`arc` | `chrome` | `firefox`), then writes an idempotent marker block into your `~/.zshrc`/`~/.bashrc`. The same flags drive it non-interactively: `--harness --browser --port --name --no-alias --yes`. The Firefox alias it writes includes `--marionette` (required for cdp-toolkit's orphaned-session auto-recovery — see [Firefox](#firefox-webdriver-bidi)). It is a CLI subcommand, **not** a 49th tool: `cdp --list` still shows 48.
 
 Prefer to wire it by hand? The manual path is three steps:
 
@@ -129,7 +129,7 @@ bun run mcp:smoke   # spawn the server + a real initialize/tools-list/tools-call
 
 - **One target per call, never a broadcast.** Each call opens one WebSocket to the one target it named, with a bounded timeout on every CDP command, lazy domain enabling, and stateless element refs. Drive as many tabs as you like across calls; there is still no broadcast step that can stall on a wedged tab.
 - **Network mocking: build the UI before the backend exists.** `mock_request` arms a persistent per-target fake backend: return canned responses, force errors, or inject latency/fault rates. Mocks survive reloads and navigations until `clear_mocks`.
-- **Full chrome-devtools-mcp parity + extras.** All 29 upstream tools, plus `performance_trace` (a robust single-call trace), Lighthouse audits, heap snapshots, a cookie group that reads, writes, and deletes httpOnly cookies, real HTML5 drag-and-drop, raw scroll/mouse dispatch, download capture, permission grants, and tab-to-video screen recording. The MCP server publishes all of them on one static, cacheable `tools/list` (≈8.5k tokens, −58% vs 1.x) and serves the full per-tool prose on demand through `describe_tool`; `CDP_TOOL_PROFILE=core` trims that listing to a 12-tool everyday set (≈2.3k tokens) for clients that eagerly load every schema, and the trimmed-away tools stay callable by name. The CLI exposes all 45 regardless. They coexist with `chrome-devtools-mcp` in a separate namespace.
+- **Full chrome-devtools-mcp parity + extras.** All 29 upstream tools, plus `performance_trace` (a robust single-call trace), Lighthouse audits, heap snapshots, a cookie group that reads, writes, and deletes httpOnly cookies, real HTML5 drag-and-drop, raw scroll/mouse dispatch, download capture, permission grants, and tab-to-video screen recording. The MCP server publishes all of them on one static, cacheable `tools/list` (≈9.3k tokens, −54% vs 1.x) and serves the full per-tool prose on demand through `describe_tool`; `CDP_TOOL_PROFILE=core` trims that listing to a 12-tool everyday set (≈2.3k tokens) for clients that eagerly load every schema, and the trimmed-away tools stay callable by name. The CLI exposes all 48 regardless. They coexist with `chrome-devtools-mcp` in a separate namespace.
 - **A whole page is a file, not a wedged tab.** Chrome cannot encode a screenshot past 16384 device px on either side, and past that `Page.captureScreenshot` does not error — it never answers, and leaves the tab resized to the clip it was capturing. On a ratio-2 display that ceiling arrives at about 8192 CSS px: an ordinary long article. `take_screenshot` measures the projection before every capture and, past the cap, takes the page as vertical bands stitched losslessly into one PNG — a 140,982 CSS px page comes back as a 2780×281964 file in 18 bands, in 17 seconds, with the tab still healthy. Also per-capture `scale`, and `renderWidth`/`renderHeight` to shoot one capture at an emulated viewport and restore the tab afterwards.
 - **Many agents, one browser, no stolen tabs.** `claim_page` hands out an opaque lease token for one tab; every other tool checks it at target resolution, so an unqualified call against a leased tab is refused by name rather than silently retargeted to whatever tab a different agent is driving.
 - **Knows when a human is already using the tab you're driving.** An in-page activity beacon distinguishes a person's clicks/keys/scrolls from the toolkit's own dispatched input, so `claim_page` and `list_leases` can report `humanActiveMs` and warn on contention instead of silently fighting someone for the keyboard. See "Staleness: is a human already using this tab?" below.
@@ -222,6 +222,12 @@ await TOOLS.navigate_page({ target: "index:0", url: "https://example.com" });
 | `CDP_TIMEOUT_MS` | `15000` | Per-command timeout. |
 | `CDP_ARTIFACT_DIR` | `/tmp/cdp-toolkit` | Screenshots, traces, heap snapshots, lighthouse reports, recorder buffers. |
 | `CDP_STATE_DIR` | `/tmp/cdp-toolkit` | `select_page` selected-target file, in-flight trace state. |
+| `CDP_EXTRACT_BASE_URL` | `http://127.0.0.1:8090/v1` | `extract_page` only. Base URL of the OpenAI-compatible extraction endpoint (the server appends `/chat/completions`). The default is loopback llm-ferry, so page content never leaves the machine; overriding it sends page content wherever the operator points. |
+| `CDP_EXTRACT_MODEL` | `schematron` | `extract_page` only. Model name for the extraction endpoint call. |
+| `CDP_EXTRACT_API_KEY` | unset | `extract_page` only. Bearer token for the endpoint. Never logged and redacted from every error; the response never contains it. |
+| `CDP_EXTRACT_TIMEOUT_MS` | `90000` | `extract_page` only. Default budget (ms) for the endpoint call; per-call `timeoutMs` overrides, hard max 300000. |
+| `CDP_EXTRACT_MAX_CHARS` | `300000` | `extract_page` only. Default character cap on the cleaned HTML payload; per-call `maxChars` overrides. |
+| `CDP_EXTRACT_PROMPT` | `html` | `extract_page` only. Prompt shape: `html` sends the cleaned HTML as the only user message (the hosted Schematron API injects the schema server-side); `schematron` sends the open-weight Schematron model-card prompt with the schema inline. Per-call `prompt` overrides; any other value is a validation error. |
 | `CDP_LEASE_TTL_MS` | `900000` | How long a tab lease survives without use before another agent can reclaim it. Refreshed on every checked call. |
 | `CDP_REQUIRE_LEASE` | off | Strict mode, MCP server only (inert under the CLI regardless of value). Turns leasing from optional into mandatory: a call against an unheld tab acquires a lease instead of driving it lease-free, and `list_pages`/`list_leases` close tabs an abandoned agent left behind. See "Parallel tabs" below. |
 | `CDP_TOOL_PROFILE` | `full` | MCP server only. Startup-only filter on what `tools/list` advertises, read once and fixed for the life of the process. `full` (the default, and what unset/empty means) lists every group; `core` lists just the 12 everyday tools plus `describe_tool`; or a comma-separated group list, e.g. `core,network,console` (`core` is always included). An unknown group name is a configuration error: the server prints `CDP_TOOL_PROFILE: unknown tool group 'x'. Known: full, core, input, …` and exits 1. Tools the profile leaves out stay callable by name. See "Progressive disclosure" below. |
@@ -245,12 +251,12 @@ That shape is deliberate. The MCP **2026-07-28** revision puts lazy discovery on
 {"tool": "describe_tool", "arguments": {}}
 ```
 
-With no `name` it returns the grouped catalog of everything this server can run. The whole 45-tool surface costs 1,141 characters:
+With no `name` it returns the grouped catalog of everything this server can run. The whole 48-tool surface:
 
 ```
-cdp-toolkit 2.2.0 · browser=chrome · 45 tools available, 46 in tools/list (CDP_TOOL_PROFILE=full)
+cdp-toolkit 2.5.0 · browser=chrome · 48 tools available, 49 in tools/list (CDP_TOOL_PROFILE=full)
 [listed] core (12): list_pages, new_page, close_page, select_page, navigate_page, wait_for, take_snapshot, click, fill, type_text, evaluate_script, take_screenshot
-[listed] input (7): hover, drag, scroll, dispatch_mouse, press_key, fill_form, upload_file
+[listed] input (9): hover, drag, scroll, dispatch_mouse, press_key, fill_form, upload_file, focus_emulation, click_focus_gated
 [listed] cookies (3): list_cookies, set_cookie, delete_cookies
 [listed] network (2): list_network_requests, get_network_request
 [listed] console (2): list_console_messages, get_console_message
@@ -262,6 +268,7 @@ cdp-toolkit 2.2.0 · browser=chrome · 45 tools available, 46 in tools/list (CDP
 [listed] permissions (1): grant_permissions
 [listed] dialogs (1): handle_dialog
 [listed] downloads (1): wait_for_download
+[listed] extraction (1): extract_page
 Unlisted tools are callable by name; describe_tool {name} documents any of them.
 ```
 
@@ -271,12 +278,12 @@ Under a narrower profile the groups it leaves out read `[hidden]` instead of `[l
 
 | `CDP_TOOL_PROFILE` | entries in `tools/list` | bytes | ≈ tokens |
 |---|---|---|---|
-| unset / `full` — the default | 46 | 33,886 | ≈8,471 |
-| `core` | 13 | 9,066 | ≈2,266 |
-| `core,network,console` | 17 | 11,893 | ≈2,973 |
-| `full` under `CDP_BROWSER=firefox` | 35 | 26,533 | ≈6,633 |
+| unset / `full` — the default | 49 | 37,088 | ≈9,272 |
+| `core` | 13 | 9,077 | ≈2,269 |
+| `core,network,console` | 17 | 11,904 | ≈2,976 |
+| `full` under `CDP_BROWSER=firefox` | 36 | 28,076 | ≈7,019 |
 
-For comparison, 1.x advertised 45 tools with full prose at roughly 20,200 tokens. So the default listing is **−58% vs 1.x**, and `core` is **−89%**.
+For comparison, 1.x advertised 45 tools with full prose at roughly 20,200 tokens. So the default listing is **−54% vs 1.x**, and `core` is **−89%**.
 
 The default flipped from `core` (2.0.0) to `full` in 2.1 for three reasons: the standard puts discovery on the host, and a host that defers schemas — Claude Code lists MCP tool *names* and loads schemas on demand — pays little for a complete list; consumers that hold per-tool allowlists or per-tool interception keyed on the tool *name* for non-core tools (the console and network readers, `performance_analyze_insight`) were silently broken by a `core` default, because those tools were simply absent from `tools/list`; and a list that never changes never invalidates the host's prompt cache. Keep `CDP_TOOL_PROFILE=core` if your client eagerly loads every schema — that is where the ≈2.3k-token surface is worth the round trips.
 
@@ -284,7 +291,7 @@ Three rules worth knowing:
 
 - **Unlisted does not mean blocked.** Only discovery is filtered. `tools/call` checks backend availability, not group membership, so an agent that names an unlisted-but-available tool still executes it; only a tool the selected browser cannot run at all is refused by name.
 - **`describe_tool` works for any tool by name, unlisted ones included** — the full description and per-parameter docs behind the terse one-liner `tools/list` carries.
-- **Profiles and `describe_tool` are MCP-only.** They are MCP-server concepts: `cdp describe_tool` fails with `unknown tool 'describe_tool'`, and the CLI keeps exposing all 45 tools no matter how `CDP_TOOL_PROFILE` is set.
+- **Profiles and `describe_tool` are MCP-only.** They are MCP-server concepts: `cdp describe_tool` fails with `unknown tool 'describe_tool'`, and the CLI keeps exposing all 48 tools no matter how `CDP_TOOL_PROFILE` is set.
 
 ### MCP protocol eras
 
@@ -295,6 +302,22 @@ The server serves **both** eras off the same stdio transport, and the era is pin
 
 Measured 2026-09-01: Claude Code 2.1.258 and Codex CLI 0.147.0 both connect and both pin the **legacy** era — their binaries carry the 2026-07-28 client strings, but neither opens with `server/discover` by default, and the MCP SDK's own `Client` behaves the same unless it opts in. So serving both eras is load-bearing, not courtesy, and today the `ttlMs`/`cacheScope` hints reach only clients that ask for the modern era. The static listing pays off on either one — no mid-session tool churn, and no prompt-cache invalidation.
 
+## Structured page extraction (`extract_page`)
+
+`extract_page` turns a page into **schema-conformant JSON** instead of prose: it grabs the target page's HTML, cleans it (scripts, styles, hidden elements and other non-content noise stripped), and sends it to an OpenAI-compatible `/chat/completions` endpoint along with your JSON Schema, returning JSON that matches the schema — fields you name, not a paragraph you have to parse. A `selector` scopes extraction to one subtree (piercing open shadow roots), so you can extract the article or the table rather than the whole page.
+
+**Privacy posture: loopback by default.** The default endpoint is llm-ferry at `http://127.0.0.1:8090/v1` with model `schematron`, so **page content stays on this machine** unless the operator explicitly points `CDP_EXTRACT_BASE_URL` (or the per-call `baseUrl`) somewhere else — that override is an operator decision about where page content may travel, and the tool treats it as one. `CDP_EXTRACT_API_KEY` is never logged and is redacted from every error; `usage` token counts (and cost, when the endpoint reports one) come back inline on every call, so a bloated extraction is visible in the answer, not discovered on a bill.
+
+**The schema is the prompt.** Extraction instructions live in the schema's property `description`s — write them like prompts ("the job title, exactly as printed", "ISO date, not 'yesterday'"). A schema whose properties carry no descriptions is rejected up front with an error saying exactly what to fix, because a description-less schema extracts garbage with no hint why. Cleaned HTML over `maxChars` (default 300000) fails with `html_too_large` naming the actual size rather than silently truncating — a half page extracted as though it were whole is a wrong answer that looks right; shrink with `selector` or `clean:"aggressive"`, or raise the cap if you truly need it all.
+
+```json
+{"tool": "extract_page", "arguments": {"target": "index:0", "schema": {"type": "object", "properties": {"title": {"type": "string", "description": "the page title, exactly as printed"}}}}}
+```
+
+**Prompt shape (`prompt`, `CDP_EXTRACT_PROMPT`).** By default (`prompt:"html"`) the cleaned HTML is the *only* user message and your schema rides in `response_format` alone — which is what the **hosted** Schematron API expects, since it injects the schema into the prompt server-side. Point `baseUrl` at a **locally served open-weight Schematron** (e.g. an MLX `schematron8B`) and that breaks: the open weights were fine-tuned on a prompt with the schema *inside* the user message, and a bare-HTML prompt returns garbage even under constrained decoding. `prompt:"schematron"` sends the model card's own messages instead — a `You are a helpful assistant` system message plus a user message carrying the compact `JSON.stringify`'d schema, then the HTML, then `MAKE SURE ITS VALID JSON.` That adds roughly the stringified schema's length to the prompt (`maxChars` caps the **HTML** only; the true count still comes back in `usage.promptTokens`). `response_format` is sent in both modes. The per-call arg wins over `CDP_EXTRACT_PROMPT`, and anything but `html`/`schematron` is rejected before any page or network work.
+
+Works on both backends (Chrome and Firefox); see the `extract_page` row in [The tools](#the-tools-29-parity--19-superset--48) for the full parameter set.
+
 ## Firefox (WebDriver BiDi)
 
 cdp-toolkit ships a second backend, Firefox over [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/), behind the same tool surface. Chrome stays the default and its behavior is unchanged, opt in explicitly to reach Firefox:
@@ -390,7 +413,7 @@ This coordinates and recovers **around** Firefox's one-session limit; it does no
 - `dispatch_mouse` (needs `input.raw`): the raw move/down/up primitive is a direct `Input.dispatchMouseEvent` wrapper with no BiDi analogue.
 - `wait_for_download`, `grant_permissions` (need `browser.downloads` / `browser.permissions`): both drive Chrome's `Browser.*` domain; WebDriver BiDi has no command to redirect a download or pre-grant a permission.
 
-Everything else, including `mock_request`/`list_mocks`/`clear_mocks` (Firefox's `network.addIntercept` covers the same fake-backend use case as Chrome's `Fetch` domain), the `claim_page`/`release_page`/`list_leases` lease group, and the new `scroll` tool (Chrome dispatches `Input.dispatchMouseEvent{type:'mouseWheel'}`, Firefox uses BiDi's `wheel` input source — both live-verified), is available under both backends: 34 of the 45 tools. Under `CDP_BROWSER=firefox` the MCP server's default (`full`) listing is therefore 35 entries — those 34 plus `describe_tool` — and a narrower `CDP_TOOL_PROFILE` trims it further, the profile filter applying after this backend filter. One asymmetry to know before expecting Chrome-style concurrency: the lease group fences *tabs* on both backends, but Firefox permits only **one BiDi session per browser instance**, so multiple agent PROCESSES under Firefox share that one session rather than opening independent ones the way Chrome's unlimited CDP connections allow — coordinated by the cross-process session lease and, by default, joined through the BiDi multiplexer above, so sharing no longer means waiting a turn — see "Parallel tabs" below.
+Everything else, including `mock_request`/`list_mocks`/`clear_mocks` (Firefox's `network.addIntercept` covers the same fake-backend use case as Chrome's `Fetch` domain), the `claim_page`/`release_page`/`list_leases` lease group, the `extract_page` extraction tool (its page work runs entirely through the driver's evaluate, no CDP-only capability), and the new `scroll` tool (Chrome dispatches `Input.dispatchMouseEvent{type:'mouseWheel'}`, Firefox uses BiDi's `wheel` input source — both live-verified), is available under both backends: 35 of the 48 tools. Under `CDP_BROWSER=firefox` the MCP server's default (`full`) listing is therefore 36 entries — those 35 plus `describe_tool` — and a narrower `CDP_TOOL_PROFILE` trims it further, the profile filter applying after this backend filter. One asymmetry to know before expecting Chrome-style concurrency: the lease group fences *tabs* on both backends, but Firefox permits only **one BiDi session per browser instance**, so multiple agent PROCESSES under Firefox share that one session rather than opening independent ones the way Chrome's unlimited CDP connections allow — coordinated by the cross-process session lease and, by default, joined through the BiDi multiplexer above, so sharing no longer means waiting a turn — see "Parallel tabs" below.
 
 **Honest capability gaps, not oversold parity:**
 
@@ -403,11 +426,11 @@ Everything else, including `mock_request`/`list_mocks`/`clear_mocks` (Firefox's
 - **No real HTML5 drag-and-drop mode.** `drag`'s `mode:"html5"` requires capability `input.html5Drag` and is rejected with a clear error under Firefox; `mode:"mouse"` (the default) works on both.
 - **No per-capture screenshot `scale`; band tiling is a Chrome-only workaround Firefox has no use for.** Two *parameters* on an otherwise-universal tool, for two different reasons. `scale` (`screenshot.scale`) is a genuine protocol gap: `browsingContext.captureScreenshot` has no scale parameter at all, and the refusal points at `emulate {deviceScaleFactor}` + a scale-1 capture instead — Firefox captures are always 1x. `tile:true` (`screenshot.tile`) is **not** a Firefox gap: measured against Firefox 153.0.3, `take_screenshot --fullPage true` on a 20,000px-tall page returned one 1366×20000 PNG at scale 1 from a single BiDi `captureScreenshot(origin:"document")` call — verified complete top-to-bottom (a marker at y=0, a marker at y=20000, an unbroken ruler through the middle, no truncation). Chrome's banding exists solely to route around its 16384-device-px encode cap; Firefox has no such cap, so there is nothing for tiling to work around, and auto-tiling correctly never fires there — a Firefox `fullPage` capture is already the whole page in one shot, however long. **`renderWidth`/`renderHeight` (`screenshot.renderSize`) are available on both backends** — that one is not a gap.
 
-## The tools (29 parity + 16 superset = 45)
+## The tools (29 parity + 19 superset = 48)
 
-This table is the **full-profile view** — all 45 tools, which is what `cdp --list` and the MCP server's default listing (`CDP_TOOL_PROFILE=full`) both show. They are partitioned into 13 static profile groups: `core` (12: list_pages, new_page, close_page, select_page, navigate_page, wait_for, take_snapshot, click, fill, type_text, evaluate_script, take_screenshot), `input` (7: hover, drag, scroll, dispatch_mouse, press_key, fill_form, upload_file), `cookies` (3: list/set/delete_cookies), `network` (2: list/get_network_request), `console` (2: list/get_console_message), `mocking` (3: mock_request, list_mocks, clear_mocks), `emulation` (2: emulate, resize_page), `performance` (6: the four trace tools, take_heapsnapshot, lighthouse_audit), `recording` (2: start/stop_screen_recording), `leases` (3: claim_page, release_page, list_leases), `permissions` (1: grant_permissions), `dialogs` (1: handle_dialog), `downloads` (1: wait_for_download). `CDP_TOOL_PROFILE=core` narrows the MCP listing to the first group; any group it leaves out stays callable by name. See [Progressive disclosure](#progressive-disclosure).
+This table is the **full-profile view** — all 48 tools, which is what `cdp --list` and the MCP server's default listing (`CDP_TOOL_PROFILE=full`) both show. They are partitioned into 14 static profile groups: `core` (12: list_pages, new_page, close_page, select_page, navigate_page, wait_for, take_snapshot, click, fill, type_text, evaluate_script, take_screenshot), `input` (9: hover, drag, scroll, dispatch_mouse, press_key, fill_form, upload_file, focus_emulation, click_focus_gated), `cookies` (3: list/set/delete_cookies), `network` (2: list/get_network_request), `console` (2: list/get_console_message), `mocking` (3: mock_request, list_mocks, clear_mocks), `emulation` (2: emulate, resize_page), `performance` (6: the four trace tools, take_heapsnapshot, lighthouse_audit), `recording` (2: start/stop_screen_recording), `leases` (3: claim_page, release_page, list_leases), `permissions` (1: grant_permissions), `dialogs` (1: handle_dialog), `downloads` (1: wait_for_download), `extraction` (1: extract_page). `CDP_TOOL_PROFILE=core` narrows the MCP listing to the first group; any group it leaves out stays callable by name. See [Progressive disclosure](#progressive-disclosure).
 
-The 29 parity tools are 1:1 with `chrome-devtools-mcp`; the 16 superset tools (`performance_trace`, the `list_cookies`/`set_cookie`/`delete_cookies` cookie group, the `mock_request`/`list_mocks`/`clear_mocks` group, the `claim_page`/`release_page`/`list_leases` lease group, the `start_screen_recording`/`stop_screen_recording` screen-recording pair, `scroll`, `dispatch_mouse`, `wait_for_download`, and `grant_permissions`) are toolkit additions. Each row notes the underlying CDP method(s) and the precise parity gaps.
+The 29 parity tools are 1:1 with `chrome-devtools-mcp`; the 19 superset tools (`performance_trace`, the `list_cookies`/`set_cookie`/`delete_cookies` cookie group, the `mock_request`/`list_mocks`/`clear_mocks` group, the `claim_page`/`release_page`/`list_leases` lease group, the `start_screen_recording`/`stop_screen_recording` screen-recording pair, `scroll`, `dispatch_mouse`, `wait_for_download`, `grant_permissions`, the `focus_emulation`/`click_focus_gated` focus pair, and `extract_page`) are toolkit additions. Each row notes the underlying CDP method(s) and the precise parity gaps.
 
 | MCP name | CDP method(s) | Parity notes / gaps |
 |---|---|---|
@@ -427,6 +450,8 @@ The 29 parity tools are 1:1 with `chrome-devtools-mcp`; the 16 superset tools (`
 | `scroll` *(superset)* | Chrome: `Input.dispatchMouseEvent{type:'mouseWheel'}`; Firefox: BiDi's `wheel` input source | Anchor at `uid`/`selector`/`x`+`y`, or the viewport center if all three are omitted; an element anchor is scrolled into view first. At least one of `deltaX`/`deltaY` is required (positive `deltaY` scrolls down, positive `deltaX` scrolls right — wheel convention). Returns `{x,y,deltaX,deltaY,target}`. Works on both backends. |
 | `drag` | Chrome: `Input.dispatchMouseEvent` (press→move→release), or `mode:"html5"`: `Input.setInterceptDrags` + `Input.dispatchDragEvent` | `mode:"mouse"` (default) sends a synthetic mouse drag; Chrome does turn this into a real HTML5 drag too, but which `dragenter`/`dragover`/`drop` events reach the page depends on where the interpolated pointer path lands — at the default `steps:2`, a standard HTML5 drop zone (`preventDefault` inside `dragover`) sees zero `dragover` events and refuses the drop. `mode:"html5"` replays the page's own drag data as `dragEnter`/`dragOver`/`drop` exactly at the destination, so it works regardless of pointer path — **Chrome-only** (`input.html5Drag`), rejected with a clear error under Firefox. `steps` (default 2) sets the interpolated-move count for mouse mode. Destination is `to:{uid\|selector\|x,y}` or a new `by:{dx,dy}` offset (sliders, map panning); exactly one of `to`/`by` is required. |
 | `dispatch_mouse` *(superset)* | `Input.dispatchMouseEvent` (one event per call) | The raw primitive: dispatch exactly one `move`/`down`/`up` at absolute viewport coordinates (`x`/`y` required on every call — CDP has no notion of a "current pointer position"). Compose your own move/down/move/up sequences for anything a physical mouse can do that `click`/`drag`'s fixed sequences can't: canvas drag-painting, marquee/rubber-band selection, a custom-hit-testing widget. Takes the same `button`/`clickCount`/`modifiers` as `click`. **Chrome-only** (`input.raw`): absent from `tools/list` under `--browser firefox`, never present-and-throwing. |
+| `focus_emulation` *(superset)* | `Emulation.setFocusEmulationEnabled` | Makes the tab *believe* it is focused (`document.hasFocus()`→true, visibility stays `visible`, focus events fire) while the real OS window stays put — no focus theft. The primitive that unlocks focus-gated UI (a button a site disables unless its tab is focused, e.g. the Claude Code OAuth **Authorize** button) on a background tab. Call with `enabled:false` to restore. **Chrome-only** (`emulate.focus`): BiDi's emulation module has no focus primitive, so absent from `tools/list` under `--browser firefox`. |
+| `click_focus_gated` *(superset)* | `Emulation.setFocusEmulationEnabled` + `Input.dispatchMouseEvent` | The one-call compose: enable focus emulation, poll until the button (CSS selector or exact visible-text match, e.g. `"Authorize"`) exists **and** is enabled, deliver a **trusted** click (`isTrusted:true` — a content script's synthesized events can never satisfy a focus gate), then restore emulation (always, unless `keepFocus`, even on throw). Timeout error names the failure mode ("never located" vs "found but stayed disabled"). Same **Chrome-only** gap as `focus_emulation`. |
 | `fill` | `Input.insertText` | Atomic paste-like commit, not per-character keystrokes. |
 | `fill_form` | per field: `callFunctionOn` + `insertText` | Array of `{uid|selector,value}`; same insertText caveat. |
 | `type_text` | `Input.insertText` | Appends (does not clear first); insertText, not per-key. |
@@ -456,6 +481,7 @@ The 29 parity tools are 1:1 with `chrome-devtools-mcp`; the 16 superset tools (`
 | `list_leases` *(superset)* | lease file | Who holds what, with pid liveness and reclaimability, plus computed `idleMs`/`expiresAt` and, where the tab's beacon answers, `humanActiveMs` (absent, not null, when there's no answer). Needs no token; never returns the nonce. A lease file that could not be read or parsed is reported as an `unreadable` row instead of being skipped (with `idleMs`/`expiresAt` omitted). Under `CDP_REQUIRE_LEASE` also reaps abandoned agent tabs, reporting closures in an additive `reaped` array. |
 | `wait_for_download` *(superset)* | `Browser.setDownloadBehavior` + `Browser.downloadWillBegin`/`downloadProgress` on a standing browser-endpoint connection | Waits for a download to finish and returns it as a real file on disk: `{path,suggestedFilename,bytes,url,target}`. **Ordering rule, not optional:** call `wait_for_download{arm:true}` *before* the click that starts the download, then click, then call `wait_for_download` again to collect it — arming this late is not a preference, it's because Chrome reverts the download-behavior override the instant the arming client disconnects, and denies an unarmed download outright. **Browser-global side effect:** while armed, *every* download in the browser is redirected into the toolkit's downloads directory. **MCP-server only:** the arm lives on a connection this server process holds open; the one-shot CLI's connection dies with the process, so nothing is captured there. Chrome-only (`browser.downloads`). |
 | `grant_permissions` *(superset)* | `Browser.grantPermissions` / `Browser.resetPermissions` on the same standing connection | Grants permissions (`geolocation`, `notifications`, `clipboardReadWrite`, ...) for an origin up front, so a page doesn't show a prompt no agent can click. Keyed by **origin**, not tab — every tab on that origin is affected, including ones opened later. `reset:true` clears this server's previous grants first (or instead); note CDP's reset is not origin-scoped, so it clears every origin at once. Same MCP-server-only reasoning as `wait_for_download`: Chrome discards the grant the moment the granting connection closes. Chrome-only (`browser.permissions`). |
+| `extract_page` *(superset)* | `Runtime.evaluate` (clean the HTML in-page) → **non-CDP** `fetch` of an OpenAI-compatible `/chat/completions` endpoint | Sends the page's cleaned HTML plus your JSON Schema to the endpoint and returns **schema-conformant JSON** instead of prose. **The default endpoint is loopback** — llm-ferry at `http://127.0.0.1:8090/v1`, model `schematron` — so page content stays on this machine unless the operator points `baseUrl`/`CDP_EXTRACT_BASE_URL` elsewhere (see [Structured page extraction](#structured-page-extraction-extract_page)). The extraction *instructions* live in the schema's property `description`s; a schema whose properties carry none is rejected up front. Over `maxChars` (default 300000) it fails with `html_too_large` rather than truncating — shrink with `selector`/`clean:"aggressive"` or raise the cap. `usage` tokens (and cost, when the endpoint reports it) come back inline on every call; `savePath` keeps large payloads out of the transcript. `prompt:"schematron"` (or `CDP_EXTRACT_PROMPT`) swaps the default HTML-only user message for the open-weight Schematron model-card prompt with the schema inline — needed when `baseUrl` points at a locally served open-weight Schematron. Works on both backends. |
 
 ## Driving MV3 extensions
 
@@ -607,7 +633,7 @@ So every tab this toolkit creates is written to a creation ledger, and `list_pag
 src/
   client.ts          # CdpConnection, openPage/withPage/openBrowser, resolveTarget, timeouts
   types.ts           # Target, TargetSelector, Uid, CDP envelopes
-  index.ts           # TOOLS registry (45) + re-exported client primitives
+  index.ts           # TOOLS registry (48) + re-exported client primitives
   cli.ts             # the Bun CLI
   mcp.ts             # stdio MCP server, dual-era (serveStdio): static cacheable tools/list
   version.ts         # the VERSION constant, single source of truth with package.json

package/dist/bidi/mux-daemon.js

@@ -498,6 +498,8 @@ var init_driver = __esm(() => {
     list_mocks: ["network.intercept"],
     clear_mocks: ["network.intercept"],
     dispatch_mouse: ["input.raw"],
+    focus_emulation: ["emulate.focus"],
+    click_focus_gated: ["emulate.focus"],
     wait_for_download: ["browser.downloads"],
     grant_permissions: ["browser.permissions"]
   };