cdp-toolkit 2.1.0 → 2.3.0

package/CHANGELOG.md

@@ -5,6 +5,39 @@ All notable changes to cdp-toolkit are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [2.3.0] - 2026-09-10
+
+**Drive focus-gated pages on a background tab without stealing OS window focus.** Some sites gate a button behind `document.hasFocus()`/`visibilitychange` — the canonical case is Anthropic's Claude Code OAuth page (`claude.ai/oauth/*`), whose **Authorize** button stays disabled on a background tab, forcing an agent driving `claude -p` login to keep a human clicking. Two new Chrome-only tools port the mechanism of the [`danbuhler/claude-code-auto-authorize`](https://github.com/danbuhler/claude-code-auto-authorize) extension: `Emulation.setFocusEmulationEnabled` makes the page *believe* it is focused (hasFocus→true, visibility stays `visible`, focus events fire) while the real window stays put, and `Input.dispatchMouseEvent` delivers the **trusted** click (`isTrusted:true`) that a content script's synthesized events can never produce.
+
+### Added
+
+- **`focus_emulation { enabled }`** — the raw toggle: a tab reports focused/visible with no OS focus change. Call with `enabled:false` to restore.
+- **`click_focus_gated { selector|text, timeoutMs, pollMs, keepFocus }`** — the one-call compose: enable focus emulation, poll until the button (CSS selector or exact visible-text, e.g. `"Authorize"`) exists **and** is enabled, trusted-click it, restore emulation (always, unless `keepFocus`, even on throw). Timeout error names the failure mode ("never located" vs "found but stayed disabled").
+- New capability **`emulate.focus`** (Chrome declares it; BiDi's emulation module has no focus primitive, so both tools are absent from `tools/list` under `--browser firefox`, per ADR-001).
+- New live harness **`test/focus-emulation-smoke.ts`** (`bun run focus:smoke`): a real headless Chrome serves a focus-gated Authorize fixture (disabled until `hasFocus()`), proves the gate is closed in a background tab (control), then `click_focus_gated` opens it and lands a trusted click — page log shows `gate:unlocked` + `CLICKED trusted=true` with the tab never activated. Plus unit tests `test/focus-emulation.test.ts` (16) pinning validation, capability gating, and manifest/docs/group registration.
+
+## [2.2.0] - 2026-09-04
+
+**Several agent PROCESSES can now drive tabs of ONE user-launched Firefox at the same time.** Firefox serves exactly one WebDriver BiDi session per browser (measured on 153.0.3: a second `session.new` gets "Maximum number of active sessions", a second WebSocket to `/session/<id>` is refused at the HTTP upgrade). Previously a second cdp-toolkit process attaching to the same Firefox was refused after the session-slot wait; now the process that wins the slot fronts the one real session with a loopback BiDi multiplexer, and every other process joins it instead of waiting. **Measured** (`bun run firefox:multi:smoke`, headless Firefox 153.0.3, 4 agents × 6 rounds): before, 4 server processes attached to one Firefox — 1 of 4 completed, the other 3 refused after the session wait; after, all 4 completed concurrently (40 calls, wall 437ms vs serial 1375ms, 4 distinct processes in flight), a killed server process left 3 survivors finishing with 0 retries, and a killed daemon was replaced within the recovery path with every survivor finishing on its original tab via the stale-id hint (`firefox:coord:smoke` 11/11, `bun test` 929 pass). See [`docs/design/2026-09-04-firefox-multi-agent-session-mux.md`](docs/design/2026-09-04-firefox-multi-agent-session-mux.md).
+
+### Added
+
+- **`src/bidi/mux.ts`** — a zero-dependency loopback WebSocket server that speaks BiDi over one real upstream session: answers `session.new`/`end`/`subscribe`/`unsubscribe`/`status` locally, forwards every other command on the single upstream connection, fans events out per client.
+- **`src/bidi/mux-daemon.ts`** (built to **`dist/bidi/mux-daemon.js`**, a fourth `scripts/build.ts` entrypoint) — a detached daemon that owns the real BiDi session and hosts the mux, so the session outlives every client process. The first process to attach an endpoint spawns it; every process, including the spawner, joins it. It exits on its own after `CDP_FIREFOX_MUX_IDLE_MS` (default 15000ms) with no clients, when Firefox goes away, or on `SIGTERM`/`SIGINT`/`SIGHUP`. Diagnostics: `<CDP_ARTIFACT_DIR or /tmp/cdp-toolkit>/ff-mux-<endpoint>.log`.
+- **`CDP_FIREFOX_MUX`** (`off` | `host` | `daemon`, default `daemon` when attaching and `host` when launching), **`CDP_FIREFOX_MUX_IDLE_MS`** (default `15000`), **`CDP_FIREFOX_MUX_DAEMON`** (absolute path override of the daemon script; tests only).
+- **A stale-target-id hint.** Firefox assigns new browsing-context ids whenever its BiDi session is re-created (measured: true even after a clean `session.end`), so if the daemon dies and a new one takes over, ids issued before the handover are stale. The driver's `no-such-target` error now names this and tells the agent to re-resolve the tab with `list_pages` (`url:`/`title:` selectors) and claim it again.
+- **`src/bidi/session-lease.ts`**: `SessionSlotRecord` gains `muxEndpoint`; `acquireSessionSlot` returns `joinMux` immediately when a live holder advertises one, instead of polling out to `SessionSlotBusyError`; new `advertiseMux()` and `muxDaemonLogFile()`.
+- **New live harness `test/firefox-multi-agent-smoke.ts`** (`bun run firefox:multi:smoke`): N agent PROCESSES each claim their own tab of one shared headless Firefox, navigate, write a marker, and run several rounds of isolated reads, so a call landing in another tab fails; every call is timestamped so concurrency is measured, not assumed. Also covers a SIGKILL of one server process (survivors finish with 0 retries) and a SIGKILL of the daemon itself (survivors hit the stale-target hint, re-resolve, and finish on the same tab).
+- New unit tests `test/bidi-mux.test.ts`, `test/mux-daemon.test.ts`, plus additions to `test/session-lease.test.ts` and `test/session-coord.test.ts`.
+
+### Changed
+
+- **`test/firefox-session-coord-smoke.ts`**: scenarios 1–4 are now pinned to `CDP_FIREFOX_MUX=off` (they test the no-mux refusal/orphan-recovery semantics specifically); a new scenario 5 covers the mux default, asserting a live holder is a detached daemon, a second process joins it instead of waiting, the daemon survives its spawner exiting, and a killed daemon's slot is recovered by the existing orphan-steal + Marionette force-clear path.
+
+### Fixed
+
+- **`evaluate_script.args` now declares `items: {}`.** It was the only array property in `tools/list` without an `items` schema, and Gemini function declarations reject exactly that shape: a host that forwarded the full tool list to a Gemini model (opencode through an OpenRouter Gemini lane, 135 tools) got no response headers at all until the gateway timed the request out. Every other array in the listing already carried `items`; the live `bun src/mcp.ts` `tools/list` now shows none missing.
+
 ## [2.1.0] - 2026-09-01
 
 **Re-based on the MCP 2026-07-28 revision: `tools/list` is now one complete, deterministic, cacheable listing that never changes, and the runtime `browser_tools` activation toggle 2.0.0 shipped three days ago is removed.** The honest headline on cost: the **default** listing grows from ≈2,457 to **≈8,471 tokens** (46 entries, 33,886 bytes, measured over raw stdio against the real entry point), because a standards-conformant list must be complete and static — still **−58% against 1.x's ≈20,200**. `CDP_TOOL_PROFILE=core` keeps the lean ≈2,266-token surface (13 entries, 9,066 bytes, −89% vs 1.x) for hosts that eagerly load every schema. Tool names, parameters, and schema shapes are unchanged, now gated by a durable wire-vs-manifest deep-equal check rather than the one-off strip-descriptions comparison 1.10.0 and 2.0.0 used. No new runtime dependency.

package/README.md

@@ -228,6 +228,9 @@ await TOOLS.navigate_page({ target: "index:0", url: "https://example.com" });
 | `CDP_REAP_GRACE_MS` | `2700000` | Extra grace (on top of `CDP_LEASE_TTL_MS`) before an `expired` lease's tab is actually destroyed by reap; `dead-pid` tabs are reaped immediately regardless. See "Reap" below. |
 | `CDP_FIREFOX_MARIONETTE_PORT` | `2828` | Firefox backend only. The Marionette side-channel port used to force-clear Firefox's orphaned BiDi session during orphan-session recovery (a blind `WebDriver:DeleteSession`). Only effective when that Firefox was launched with `--marionette`. See "Firefox" below. |
 | `CDP_FIREFOX_SESSION_WAIT_MS` | `10000` | Firefox backend only. How long a second process waits for a live holder to release Firefox's one BiDi session before returning the distinguishable "held by a live process" error. See "Firefox" below. |
+| `CDP_FIREFOX_MUX` | `daemon` when attaching, `host` when launching | Firefox backend only. Which process hosts the loopback BiDi multiplexer that lets several cdp-toolkit processes drive one Firefox: `daemon` (attach-mode default) spawns/joins a detached process that owns the real session for as long as anyone needs it; `host` makes THIS process the holder (launch mode's default, since a launched Firefox and its holder already share a lifetime); `off` restores pre-mux behavior — one live session per Firefox, a second process refused after the wait. See "Firefox" below and "Parallel tabs" below. |
+| `CDP_FIREFOX_MUX_IDLE_MS` | `15000` | Firefox backend only, daemon mode. How long the mux daemon stays up with zero connected clients before it exits on its own (also counted from its own startup, so a spawner that never connects can't strand one). |
+| `CDP_FIREFOX_MUX_DAEMON` | resolved from the running module's location | Firefox backend only, tests only. Absolute path override for the daemon script the driver spawns, in place of the built-in resolution (`./mux-daemon.ts` in source, `dist/bidi/mux-daemon.js` in the published package). |
 
 ## Progressive disclosure
 
@@ -242,12 +245,12 @@ That shape is deliberate. The MCP **2026-07-28** revision puts lazy discovery on
 {"tool": "describe_tool", "arguments": {}}
 ```
 
-With no `name` it returns the grouped catalog of everything this server can run. The whole 45-tool surface costs 1,141 characters:
+With no `name` it returns the grouped catalog of everything this server can run. The whole 47-tool surface:
 
 ```
-cdp-toolkit 2.1.0 · browser=chrome · 45 tools available, 46 in tools/list (CDP_TOOL_PROFILE=full)
+cdp-toolkit 2.3.0 · browser=chrome · 47 tools available, 48 in tools/list (CDP_TOOL_PROFILE=full)
 [listed] core (12): list_pages, new_page, close_page, select_page, navigate_page, wait_for, take_snapshot, click, fill, type_text, evaluate_script, take_screenshot
-[listed] input (7): hover, drag, scroll, dispatch_mouse, press_key, fill_form, upload_file
+[listed] input (9): hover, drag, scroll, dispatch_mouse, press_key, fill_form, upload_file, focus_emulation, click_focus_gated
 [listed] cookies (3): list_cookies, set_cookie, delete_cookies
 [listed] network (2): list_network_requests, get_network_request
 [listed] console (2): list_console_messages, get_console_message
@@ -268,7 +271,7 @@ Under a narrower profile the groups it leaves out read `[hidden]` instead of `[l
 
 | `CDP_TOOL_PROFILE` | entries in `tools/list` | bytes | ≈ tokens |
 |---|---|---|---|
-| unset / `full` — the default | 46 | 33,886 | ≈8,471 |
+| unset / `full` — the default | 48 | 35,556 | ≈8,889 |
 | `core` | 13 | 9,066 | ≈2,266 |
 | `core,network,console` | 17 | 11,893 | ≈2,973 |
 | `full` under `CDP_BROWSER=firefox` | 35 | 26,533 | ≈6,633 |
@@ -328,6 +331,8 @@ cdp --connect 127.0.0.1:9223 take_snapshot          # host:port also works
 cdp --connect ws://127.0.0.1:9223/session take_snapshot   # or the full ws:// URL
 ```
 
+**A side effect of the daemon above: CLI target ids now survive across separate invocations, for as long as the daemon does.** Before it, every CLI call opened and closed its own throwaway BiDi session against the shared Firefox, and Firefox hands out fresh browsing-context ids on every new session — so a `cdp list_pages` id was already stale by the time a following `cdp <tool> --target <id>` call used it; only `url:`/`title:`/`label:` selectors survived across invocations. Measured 2026-09-04 against headless Firefox 153.0.3: three consecutive `cdp list_pages` calls on one attached Firefox returned the SAME tab id all three times in the default (daemon) mode (the 2nd and 3rd calls took ~31ms, joining the already-running daemon instead of standing up a session), and a DIFFERENT id on each of the three calls under `CDP_FIREFOX_MUX=off`. So a `list_pages` → `<tool> --target <id>` CLI pipeline now works by id — not just by `url:`/`title:`/`label:` — for as long as the daemon stays up: the idle window (`CDP_FIREFOX_MUX_IDLE_MS`, default `15000`ms after the last call) or indefinitely while any MCP server keeps a client joined to it.
+
 3. MCP client config: set the env var instead of a flag (the endpoint doesn't belong in `args`):
 
 ```json
@@ -364,7 +369,15 @@ claude mcp add cdp-toolkit -e CDP_FIREFOX_ENDPOINT=9223 -- npx -y cdp-toolkit
 - **Two genuinely live processes** are serialized rather than both wedging: the second waits up to `CDP_FIREFOX_SESSION_WAIT_MS` (default `10000`ms) for the slot to free, then returns a distinguishable, fast error naming the live holder — "Firefox's single WebDriver BiDi session on `<endpoint>` is held by a LIVE process `<label>` (pid N) … wait & retry, or point this server at a different endpoint/browser" — instead of hanging.
 - **An orphaned session from a dead holder** — the classic wedge, a client `SIGKILL`ed or crashed without `session.end`, which Firefox does not reap on its own — is now **auto-recovered**: cdp-toolkit force-clears the orphaned session over the **Marionette side channel** (a blind `WebDriver:DeleteSession` on port `2828`, `CDP_FIREFOX_MARIONETTE_PORT`) **without** killing or restarting Firefox, then retries. This works **only** if that Firefox was launched with `--marionette` (verified against FF153: `--remote-debugging-port` alone does **not** start Marionette). When Marionette is absent it degrades to a clear, actionable error ("orphaned … could not be auto-cleared. Marionette recovery needs Firefox launched with `--marionette` …; otherwise restart Firefox") rather than the old dead-end.
 
-This coordinates and recovers **around** Firefox's one-session limit; it does not remove it. Two live holders still cannot share the slot at once — one is served, the other waits or is told, by name, who holds it.
+**Several agent PROCESSES driving one Firefox at once: a live holder is now something you JOIN, not just wait on.** The slot's holder fronts the one real session with a loopback WebSocket server that itself speaks BiDi — a multiplexer (`src/bidi/mux.ts`) answering `session.new`/`end`/`subscribe`/`unsubscribe`/`status` locally and forwarding everything else on the single upstream connection, fanning events out per client. A second (third, fourth, …) cdp-toolkit process attaching to the same endpoint finds the holder's mux advertised in the slot record and dials it instead of polling the slot; from that joiner's side a joined connection is indistinguishable from a real one, and it drives its own tab exactly as if it held the session outright — measured with 4 server processes each claiming and driving its own tab concurrently on one Firefox, 0 wedges (`test/firefox-multi-agent-smoke.ts`, `bun run firefox:multi:smoke`). Three modes, `CDP_FIREFOX_MUX`:
+
+- **`daemon`** (the default when attaching): the session is owned by a small **detached daemon** process, not by any client. The first process to attach spawns it (`dist/bidi/mux-daemon.js` in the published package); every process, including the spawner, joins it the same way. This is deliberate, not incidental: with the session living inside a client process, that client exiting — even cleanly, e.g. the first agent session finishing while a second is still mid-task — would re-create the session and invalidate every other client's tab ids, leases and origin records. A client process dying is invisible to the others (measured: 3 survivors finish all rounds with 0 retries after a 4th server process is `SIGKILL`ed). The daemon exits on its own once nobody is joined to it (`CDP_FIREFOX_MUX_IDLE_MS`, default `15000`ms with zero clients, also counted from its own startup), when Firefox goes away, or on `SIGTERM`/`SIGINT`/`SIGHUP` — it never outlives the browser it fronts. It runs with no console (spawned detached, stdio ignored); its diagnostics go to `<CDP_ARTIFACT_DIR or /tmp/cdp-toolkit>/ff-mux-<endpoint>.log`, which a driver error names if a daemon it spawned never advertises within the wait window.
+- **`host`**: the mux runs inside the first process itself rather than a detached daemon. This is launch mode's default (a launched Firefox and its holder already share a lifetime, so there is nothing extra to leak) and an opt-in for attach mode if you'd rather not have a stray daemon process at all, at the cost of the invalidation risk above.
+- **`off`**: restores the pre-mux behavior exactly — no mux hosted, none joined, two live processes serialize on the slot as described above.
+
+**One caveat, and it is narrow.** Firefox regenerates every top-level browsing-context id whenever its one BiDi session is re-created — measured on 153.0.3: true even after a clean `session.end` + `session.new`, the tab survives at the same URL but `browsingContext.getTree` reports a new id. A client process dying never triggers this (that is the entire point of the daemon), but if the **daemon itself** dies and a new one takes over, ids issued before the handover go stale. The next call against a stale id fails with `no-such-target` plus a hint that the session was re-established and to re-resolve the tab with `list_pages` (`url:`/`title:` selectors) and claim it again — the tab itself, its state, and its lease are untouched, only the id changed.
+
+This coordinates and recovers **around** Firefox's one-session limit; it does not remove it. There is still exactly one real session, and `CDP_FIREFOX_MUX=off` still serializes on it exactly as before — the mux's default just means most agents never have to.
 
 **Attaching is not relaunching.** The one thing that is genuinely impossible is handing a debug port to an *already-running* Firefox process after the fact: the `--remote-debugging-port` flag only takes effect on a process's original launch, so relaunching the `firefox` binary against a running instance hands off to it and exits silently, opening no port (verified against Firefox 153.0.3). That is a real, narrow limitation of the Firefox binary itself. It is not the same claim as "Firefox cannot be attached to" — a Firefox that was launched *with* the debug port open, whether by this toolkit or by your own hand, exposes a plain BiDi endpoint that any number of fresh clients can connect to later, which is exactly what `--connect`/`CDP_FIREFOX_ENDPOINT` does.
 
@@ -377,7 +390,7 @@ This coordinates and recovers **around** Firefox's one-session limit; it does no
 - `dispatch_mouse` (needs `input.raw`): the raw move/down/up primitive is a direct `Input.dispatchMouseEvent` wrapper with no BiDi analogue.
 - `wait_for_download`, `grant_permissions` (need `browser.downloads` / `browser.permissions`): both drive Chrome's `Browser.*` domain; WebDriver BiDi has no command to redirect a download or pre-grant a permission.
 
-Everything else, including `mock_request`/`list_mocks`/`clear_mocks` (Firefox's `network.addIntercept` covers the same fake-backend use case as Chrome's `Fetch` domain), the `claim_page`/`release_page`/`list_leases` lease group, and the new `scroll` tool (Chrome dispatches `Input.dispatchMouseEvent{type:'mouseWheel'}`, Firefox uses BiDi's `wheel` input source — both live-verified), is available under both backends: 34 of the 45 tools. Under `CDP_BROWSER=firefox` the MCP server's default (`full`) listing is therefore 35 entries — those 34 plus `describe_tool` — and a narrower `CDP_TOOL_PROFILE` trims it further, the profile filter applying after this backend filter. One asymmetry to know before expecting Chrome-style concurrency: the lease group fences *tabs* on both backends, but Firefox permits only **one BiDi session per browser instance**, so multiple agents under Firefox share and serialize on that single session (coordinated by the cross-process session lease above), not the independent concurrent sessions Chrome's unlimited CDP connections allow — see "Parallel tabs" below.
+Everything else, including `mock_request`/`list_mocks`/`clear_mocks` (Firefox's `network.addIntercept` covers the same fake-backend use case as Chrome's `Fetch` domain), the `claim_page`/`release_page`/`list_leases` lease group, and the new `scroll` tool (Chrome dispatches `Input.dispatchMouseEvent{type:'mouseWheel'}`, Firefox uses BiDi's `wheel` input source — both live-verified), is available under both backends: 34 of the 45 tools. Under `CDP_BROWSER=firefox` the MCP server's default (`full`) listing is therefore 35 entries — those 34 plus `describe_tool` — and a narrower `CDP_TOOL_PROFILE` trims it further, the profile filter applying after this backend filter. One asymmetry to know before expecting Chrome-style concurrency: the lease group fences *tabs* on both backends, but Firefox permits only **one BiDi session per browser instance**, so multiple agent PROCESSES under Firefox share that one session rather than opening independent ones the way Chrome's unlimited CDP connections allow — coordinated by the cross-process session lease and, by default, joined through the BiDi multiplexer above, so sharing no longer means waiting a turn — see "Parallel tabs" below.
 
 **Honest capability gaps, not oversold parity:**
 
@@ -493,7 +506,7 @@ Two agents pointed at the same Chrome both resolve `target: active` to whatever
 
 Pass that token as `lease` on every later call against the tab. `new_page` with `claim: true` does the open-and-claim in one call. `release_page` gives it back and, by default, closes the tab if this toolkit opened it (see "Close on release" below). `list_leases` shows who holds what.
 
-**Firefox: the ceiling is one BiDi session per browser instance.** The `claim_page`/`release_page`/`list_leases` tab-lease group works under both backends, but a real asymmetry sits underneath it. Chrome accepts unlimited concurrent CDP connections, so "many agents, one browser" there means many genuinely independent sessions. Firefox permits only **one** WebDriver BiDi session per browser instance (see [Firefox](#firefox-webdriver-bidi)), so under Firefox "many agents, one browser" means many agents *sharing and serialized on that single session* — coordinated by the cross-process session lease, not running as independent sessions. Expect Chrome-style multi-session concurrency on Chrome only; on Firefox the tabs are still leasable, but the session beneath them is a single shared slot.
+**Firefox: the ceiling is one BiDi session per browser instance, but several PROCESSES can now drive it at once.** The `claim_page`/`release_page`/`list_leases` tab-lease group works under both backends, but a real asymmetry sits underneath it. Chrome accepts unlimited concurrent CDP connections, so "many agents, one browser" there means many genuinely independent sessions. Firefox permits only **one** WebDriver BiDi session per browser instance (see [Firefox](#firefox-webdriver-bidi)) — that ceiling is real and does not move — but as of the BiDi multiplexer described there, several cdp-toolkit *processes* attached to the same Firefox now share that one session by joining a mux fronting it rather than waiting their turn for it: each still claims and drives its own tab concurrently, measured at 4 distinct server processes with a call in flight at once, not the historical one-at-a-time wedge. Expect Chrome-style *within-process* concurrency (many subagents, one server, unlimited connections) on Chrome; on Firefox the tabs are still leasable per process exactly the same way, and the session underneath is one shared real session multiplexed across processes rather than independent ones — `CDP_FIREFOX_MUX=off` is the one setting that goes back to strict serialization.
 
 **Opt in, and only ever a refusal — this is the default, with `CDP_REQUIRE_LEASE` unset.** A tab nobody claimed behaves exactly as it did before. Omitting `lease` is identical to pre-1.2 behavior in every respect: no default changed, no return shape changed, no previously legal call now needs a new argument. This is why it was 1.2 and not 2.0.
 
@@ -615,6 +628,7 @@ test/
   network_mock.test.ts # pure-logic unit tests (bun test)
   mcp-smoke.ts         # MCP handshake + live tools/call round-trip (bun run mcp:smoke)
   lease-smoke.ts       # two-process live lease harness (bun run lease:smoke)
+  firefox-multi-agent-smoke.ts # N concurrent server PROCESSES on one shared Firefox (bun run firefox:multi:smoke)
 ```
 
 Each module ends with a footer comment listing the exact CDP methods it uses and its parity gaps. The full design contract lives in [`CONTRACT.md`](./CONTRACT.md).