barebrowse 0.17.0 → 0.19.1

package/CHANGELOG.md

@@ -1,5 +1,92 @@
 # Changelog
 
+## [0.19.1] - 2026-07-11
+
+### Changed
+
+- **MCP: Firefox sessions now default to `hybrid`** (was `headless`), matching the
+  Chromium default — Phase 4 gave Firefox the same headless→headed challenge
+  fallback, so there's no reason to hold it back. A failed headed relaunch (no
+  display) falls back to the headless result. Override with `BAREBROWSE_MODE`;
+  the Chromium path now honors `BAREBROWSE_MODE` too (was hardcoded `hybrid`).
+
+### Docs
+
+- **README** refocused on primitives and value — Firefox stated as a capability
+  set rather than a version-by-version log; stale counts/versions and a redundant
+  section removed.
+- **Single PRD** — the Firefox parity plan is merged into `docs/01-product/prd.md`
+  as a "Firefox Parity" section (capability matrix + goal/non-goals + forward
+  plan); the standalone `firefox-parity-plan.md` is removed.
+- Fixed stale "hybrid is chromium-only" strings in the CLI help and MCP comments.
+
+## [0.19.0] - 2026-07-11
+
+### Added
+
+- **Firefox/BiDi resilience + remaining methods (parity plan Phase 4).** The
+  Firefox engine reaches practical parity with the Chromium/CDP engine — four
+  CDP-only surfaces now work the same over WebDriver BiDi:
+  - **Hybrid mode** (`mode: 'hybrid'`). On a bot-challenge page (Cloudflare "Just
+    a moment", hCaptcha, etc.) the Firefox engine now relaunches **headed** and
+    retries, exactly like the CDP path — a real display often clears an
+    interstitial headless can't. The relaunch tears down the headless
+    Firefox+BiDi and hands a fresh connection back to the page, which rebinds to
+    it; a failed relaunch (no display) keeps the headless result. Cookies
+    injected before the challenge are **re-injected into the fresh headed
+    profile** so the retry stays authenticated. The challenge heuristic
+    (`isChallengePage`) is now shared by both engines (`challenge.js`).
+  - **`page.saveState(file)`** — persists cookies (`storage.getCookies`) +
+    localStorage (`script.evaluate`) to a `0600` JSON file, flattened to the
+    same shape the CDP path writes (was a throwing stub).
+  - **`page.waitForNavigation()`** — resolves on the next top-context
+    `browsingContext.load` event, with an SPA settle fallback (was a stub).
+  - **`page.downloads`** — real download tracking via
+    `browsingContext.downloadWillBegin`/`downloadEnd`. Downloads land in a
+    throwaway dir (Firefox `browser.download.*` prefs, parity with CDP's
+    `setDownloadBehavior`) and each record carries `{url, suggestedFilename,
+    savedPath, state}` with BiDi's `complete` normalized to CDP's `completed`
+    (was always empty).
+
+  BiDi mechanics were POC-measured against real Firefox before wiring (cookie/
+  localStorage shapes on real vs opaque origins; the `load` event sequence; that
+  Firefox emits `downloadWillBegin`/`downloadEnd` and honors the download-dir
+  prefs). The shared JS-dialog decision core was also extracted to `dialog.js`
+  (both engines — Phase-3 review finding #5). Guarded by
+  `test/unit/firefox-hybrid.test.js` (5, fake-BiDi orchestration) and Firefox
+  integration tests (saveState, waitForNavigation ×2, downloads).
+  Still Firefox-limited: `reload({ignoreCache})` (upstream BiDi gap, no local
+  fix).
+
+## [0.18.0] - 2026-07-11
+
+### Added
+
+- **Firefox/BiDi noise-reduction + dialogs (parity plan Phase 3).** Two more
+  Chromium/CDP capabilities now work the same on the Firefox engine:
+  - **Ad/tracker blocking** (`blockAds`, on by default; `blockUrls` to extend).
+    BiDi's `network.addIntercept` can't express our glob patterns — its
+    `urlPatterns` reject `*` outright and have no subdomain wildcard — so the
+    Firefox path registers a *catch-all* `beforeRequestSent` intercept and
+    matches each request URL in-process against the shared `blocklist.js`
+    (new `makeBlockMatcher` compiles the CDP globs to a predicate, single-sourced
+    across engines). Matches are failed like CDP's `ERR_BLOCKED_BY_CLIENT`; the
+    rest continue. Default is **on** (Firefox is always a launched throwaway
+    profile, never attach mode). `src/blocklist-firefox.js`.
+  - **JS dialog handling** (`alert`/`confirm`/`prompt`/`beforeunload`). The BiDi
+    session is created with `unhandledPromptBehavior:'ignore'` so Firefox no
+    longer auto-dismisses prompts before we can act; `page.dialogLog` records
+    every dialog and `page.onDialog(handler)` overrides the default (accept all
+    except `beforeunload`), mirroring the CDP surface exactly.
+
+  BiDi mechanics were POC-measured against real Firefox before wiring (the "no
+  such alert" auto-dismiss race, and that `urlPatterns` reject `*`). Guarded by
+  `test/unit/blocklist-firefox.test.js` (10) and Firefox integration tests
+  (ad-block via a hermetic CORS server so the `blockAds:false` control genuinely
+  passes through; dialog auto-accept + `dialogLog` + custom `onDialog`).
+  Remaining Firefox gaps (Phase 4+): hybrid mode, `saveState`,
+  `waitForNavigation`, downloads, `reload({ignoreCache})`.
+
 ## [0.17.0] - 2026-07-10
 
 ### Added

package/README.md

@@ -96,15 +96,15 @@ Or manually add to your config (`claude_desktop_config.json`, `.cursor/mcp.json`
 }
 ```
 
-19 tools: `browse`, `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `scroll`, `hover`, `select`, `back`, `forward`, `reload`, `drag`, `upload`, `pdf`, `screenshot`, `wait_for`, `tabs`. Plus `assess` (privacy scan) if [wearehere](https://github.com/hamr0/wearehere) is installed. Plus opt-in `eval` (`BAREBROWSE_MCP_EVAL=1`) — runs JS in the authenticated session, off by default because it can read cookies/localStorage. Session runs in hybrid mode with automatic cookie injection. Per-tool timeouts (goto/reload/wait_for 60s, back/forward 30s, interactive ops 15s, pdf/screenshot/upload 45s) with auto-retry on transient failures (idempotent only — mutating tools fail loudly to avoid double-submits).
+MCP tools: `browse`, `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `scroll`, `hover`, `select`, `back`, `forward`, `reload`, `drag`, `upload`, `pdf`, `screenshot`, `wait_for`, `tabs`. Plus `assess` (privacy scan) if [wearehere](https://github.com/hamr0/wearehere) is installed. Plus opt-in `eval` (`BAREBROWSE_MCP_EVAL=1`) — runs JS in the authenticated session, off by default because it can read cookies/localStorage. Session runs in hybrid mode with automatic cookie injection. Per-tool timeouts (goto/reload/wait_for 60s, back/forward 30s, interactive ops 15s, pdf/screenshot/upload 45s) with auto-retry on transient failures (idempotent only — mutating tools fail loudly to avoid double-submits).
 
-`browse` and `snapshot` accept `pruneMode: 'act'|'read'` (v0.9.1). `act` (default) keeps interactive elements — best for clicking/filling. `read` keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. If act-mode collapses a content-heavy page near-totally, the snapshot includes a `hint: …` line suggesting `pruneMode='read'` so the agent doesn't bail to a separate HTTP fetch.
+`browse` and `snapshot` accept `pruneMode: 'act'|'read'`. `act` (default) keeps interactive elements — best for clicking/filling. `read` keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. If act-mode collapses a content-heavy page near-totally, the snapshot includes a `hint: …` line suggesting `pruneMode='read'` so the agent doesn't bail to a separate HTTP fetch.
 
 Troubleshooting MCP setup: `npx barebrowse doctor` scans every known config location and flags scope conflicts. `npx barebrowse install --force` overwrites an existing entry pointing at a different endpoint.
 
 ### 3. Library -- for agentic automation
 
-Import barebrowse in your agent code. One-shot reads, interactive sessions, full observe-think-act loops. Works with any LLM orchestration library. Ships with a ready-made adapter for [bareagent](https://www.npmjs.com/package/bare-agent) (18 tools, auto-snapshot after every action).
+Import barebrowse in your agent code. One-shot reads, interactive sessions, full observe-think-act loops. Works with any LLM orchestration library. Ships with a ready-made adapter for [bareagent](https://www.npmjs.com/package/bare-agent) (auto-snapshot after every action).
 
 For code examples, API reference, and wiring instructions, see **[barebrowse.context.md](barebrowse.context.md)** -- the full integration guide.
 
@@ -112,10 +112,12 @@ For code examples, API reference, and wiring instructions, see **[barebrowse.con
 
 | Mode | What happens | Best for |
 |------|-------------|----------|
-| **Headless** (default) | Launches a fresh Chromium, no UI | Fast automation, scraping, reading pages |
-| **Headed** | Auto-launches a visible Chromium window | Bot-detected sites, visual debugging, CAPTCHAs |
+| **Headless** (default) | Launches a fresh browser, no UI | Fast automation, scraping, reading pages |
+| **Headed** | Auto-launches a visible browser window | Bot-detected sites, visual debugging, CAPTCHAs |
 | **Hybrid** | Tries headless first, auto-launches headed if blocked | General-purpose agent browsing |
 
+All three work on both engines (Chromium/CDP and Firefox/BiDi).
+
 ### Attach to your already-running browser
 
 Start Chromium yourself with a debug port, then drive your real logged-in session:
@@ -135,21 +137,22 @@ await page.close(); // closes only the tab barebrowse opened — your browser ke
 ### Firefox (WebDriver BiDi)
 
 CDP is deprecated in Firefox, so barebrowse drives it over the W3C WebDriver
-BiDi protocol instead — a second transport over the same `ws` dependency, no
-extra download. Same `page.*` API (the ARIA snapshot is reconstructed in-page
-since BiDi has no `getFullAXTree`):
+BiDi protocol — a second transport over the same `ws` dependency, no extra
+download. Same `page.*` API (the ARIA snapshot is reconstructed in-page since
+BiDi has no `getFullAXTree`):
 
 ```js
-const page = await connect({ engine: 'firefox' }); // headless by default
+const page = await connect({ engine: 'firefox' });
 ```
 
-From the CLI: `barebrowse open <url> --engine firefox`. From MCP: set
-`BAREBROWSE_ENGINE=firefox`. Firefox cookies (plaintext) reuse into the same
-engine. Consent auto-dismiss and headless stealth work on Firefox too (as of
-v0.16.0). Chromium (CDP) remains the default; `hybrid` mode, ad/tracker
-blocking, and the daemon's console/network capture are still Chromium-only.
+CLI: `barebrowse open <url> --engine firefox`. MCP: `BAREBROWSE_ENGINE=firefox`.
+Firefox cookies (plaintext) reuse into the same engine.
 
-No clone profile, no fresh cookies — the agent sees what you see.
+Firefox is at practical parity with Chromium: stealth, consent auto-dismiss,
+ad/tracker blocking, JS dialogs, console/network capture, hybrid fallback,
+`saveState`, `waitForNavigation`, and download tracking all work the same way.
+Chromium (CDP) stays the default; the only remaining gap is
+`reload({ignoreCache})` (upstream BiDi limitation).
 
 ### Incognito (clean session)
 
@@ -168,9 +171,9 @@ From the CLI: `barebrowse open <url> --incognito`. From MCP: set
 
 ## What it handles automatically
 
-Cookie consent walls (29 languages, with real mouse click fallback for stubborn CMPs), login walls (cookie extraction from your browsers), bot detection (ARIA node count heuristic + stealth patches + automatic headed fallback — snapshot shows `[BOT CHALLENGE DETECTED]` warning when blocked), permission prompts, SPA navigation, JS dialogs, off-screen elements, pre-filled inputs, ARIA noise, and profile locking. The agent doesn't think about any of it.
+Cookie consent walls (29 languages, with real mouse click fallback for stubborn CMPs), login walls (cookie extraction from your browsers), bot detection (challenge-phrase + node-count heuristic + stealth patches + automatic headed fallback — snapshot shows `[BOT CHALLENGE DETECTED]` warning when blocked), permission prompts, SPA navigation, JS dialogs, off-screen elements, pre-filled inputs, ARIA noise, and profile locking. The agent doesn't think about any of it.
 
-## Safe by default (v0.11.0)
+## Safe by default
 
 barebrowse hands an autonomous — and therefore prompt-injectable — agent an *authenticated* browser, so the defaults are calibrated for that threat:
 
@@ -221,8 +224,8 @@ Everything the agent can do through barebrowse:
 | **Wait for network idle** | Resolve when no pending requests for 500ms |
 | **Dialog handling** | Auto-dismiss JS alert/confirm/prompt dialogs |
 | **Save state** | Export cookies + localStorage to JSON |
-| **Inject cookies** | Extract from Firefox/Chromium and inject via CDP |
-| **Raw CDP** | Escape hatch for any Chrome DevTools Protocol command |
+| **Inject cookies** | Extract from Firefox/Chromium and inject into the session (CDP or BiDi) |
+| **Raw CDP / BiDi** | Escape hatch: `page.cdp` (Chromium) or `page.bidi` (Firefox) for any low-level command |
 
 ## Tested against
 
@@ -230,10 +233,6 @@ Everything the agent can do through barebrowse:
 
 Google, YouTube, BBC, Wikipedia, GitHub, DuckDuckGo, Hacker News, Amazon DE, The Guardian, Spiegel, Le Monde, El Pais, Corriere, NOS, Bild, Nu.nl, Booking, NYT, Stack Overflow, CNN, Reddit
 
-## Context file
-
-**[barebrowse.context.md](barebrowse.context.md)** is the full integration guide. Feed it to an AI assistant or read it yourself -- it covers the complete API, snapshot format, interaction loop, auth options, bareagent wiring, MCP setup, and gotchas. Everything you need to wire barebrowse into a project.
-
 ## How it works
 
 ```
@@ -245,12 +244,15 @@ URL -> find/launch browser (chromium.js)
     -> navigate to URL, wait for load
     -> detect + dismiss cookie consent dialogs (consent.js)
     -> get full ARIA accessibility tree (aria.js)
-    -> 9-step pruning pipeline from mcprune (prune.js)
+    -> pruning pipeline from mcprune (prune.js)
     -> dispatch real input events: click/type/scroll (interact.js)
     -> agent-ready snapshot with [ref=N] markers
 ```
 
-14 modules, ~3,000 lines, two small dependencies (`ws`, `@mozilla/readability`).
+Firefox follows the same flow over WebDriver BiDi (`bidi.js`), with the AX tree
+reconstructed in-page (`ax-snapshot.js`) and pruning/formatting shared verbatim.
+
+A small set of focused modules, two small dependencies (`ws`, `@mozilla/readability`).
 
 ## Requirements
 

package/barebrowse.context.md

@@ -95,7 +95,7 @@ const snapshot = await browse('https://example.com', {
 | `close()` | -- | void | Close page, disconnect CDP, kill browser (if headless) |
 
 **connect() options** (in addition to mode/port/consent):
-- `engine: 'chromium'|'firefox'` — Browser engine. Default `chromium` (CDP). `firefox` drives Firefox over WebDriver BiDi (CDP is deprecated there) — a separate transport with the same `page.*` API; the ARIA snapshot is reconstructed in-page since BiDi has no `getFullAXTree`. Consent auto-dismiss and stealth (headless anti-detection) work on Firefox as of v0.16.0; the CLI daemon's console/network capture and `waitForNetworkIdle` work as of v0.17.0. Still Chromium-only: `hybrid` mode and ad/tracker blocking. `reload()` ignores `ignoreCache` on Firefox (BiDi limitation).
+- `engine: 'chromium'|'firefox'` — Browser engine. Default `chromium` (CDP). `firefox` drives Firefox over WebDriver BiDi (CDP is deprecated there) — a separate transport with the same `page.*` API; the ARIA snapshot is reconstructed in-page since BiDi has no `getFullAXTree`. Consent auto-dismiss and stealth (headless anti-detection) work on Firefox as of v0.16.0; the CLI daemon's console/network capture and `waitForNetworkIdle` work as of v0.17.0; ad/tracker blocking (`blockAds`/`blockUrls`) and JS dialog handling (`dialogLog`/`onDialog`) work as of Phase 3; `hybrid` mode, `saveState`, `waitForNavigation`, and download tracking (`page.downloads`) work as of Phase 4. The only remaining Firefox gap is `reload()` ignoring `ignoreCache` (upstream BiDi limitation).
 - `port: 9222` — Attach to a Chromium already running with `--remote-debugging-port=N` instead of spawning one. The browser keeps running on `close()`. Stealth + permission denial + download capture are skipped to avoid mutating the user's running browser.
 - `proxy: 'http://...'` — HTTP/SOCKS proxy for browser
 - `viewport: '1280x720'` — Set viewport dimensions

package/cli.js

@@ -487,7 +487,7 @@ Session:
     --engine=chromium|firefox       Browser engine (default: chromium/CDP;
                                     firefox drives over WebDriver BiDi)
     --mode=headless|headed|hybrid   Browser mode (default: headless;
-                                    hybrid is chromium-only)
+                                    hybrid works on both engines)
     --port=N                        CDP port for headed mode
     --no-cookies                    Skip cookie injection
     --incognito                     Clean, unauthenticated session (skip all auth injection)

package/mcp-server.js

@@ -123,15 +123,17 @@ async function getPage() {
   if (_pageConnecting) return _pageConnecting;
   // Engine selection is a launch-time setting (the browser persists across
   // tool calls), so it's an env var, not a per-request flag. Firefox is driven
-  // over WebDriver BiDi and has no hybrid fallback — default it to headless
-  // (override with BAREBROWSE_MODE=headed).
+  // over WebDriver BiDi and, as of parity Phase 4, has the same hybrid fallback
+  // as Chromium (relaunch headed on a bot-challenge page) — so it defaults to
+  // hybrid too, matching the Chromium default (override with BAREBROWSE_MODE).
+  // A failed headed relaunch (no display) falls back to the headless result.
   // Incognito is a launch-time setting like engine/mode: the session persists
   // across tool calls, so it's an env var (BAREBROWSE_INCOGNITO=1), not a
   // per-request flag. Skips all auth injection for a clean, logged-out session.
   const incognito = process.env.BAREBROWSE_INCOGNITO === '1';
   const connectOpts = process.env.BAREBROWSE_ENGINE === 'firefox'
-    ? { engine: 'firefox', mode: process.env.BAREBROWSE_MODE || 'headless', incognito }
-    : { mode: 'hybrid', incognito };
+    ? { engine: 'firefox', mode: process.env.BAREBROWSE_MODE || 'hybrid', incognito }
+    : { mode: process.env.BAREBROWSE_MODE || 'hybrid', incognito };
   _pageConnecting = connect(connectOpts);
   try {
     _page = await _pageConnecting;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "barebrowse",
-  "version": "0.17.0",
+  "version": "0.19.1",
   "description": "Authenticated web browsing for autonomous agents via CDP. URL in, pruned ARIA snapshot out.",
   "repository": {
     "type": "git",

package/src/bidi.js

@@ -135,8 +135,14 @@ export async function createBiDi(wsUrl) {
     close() { ws.close(); },
   };
 
-  // Open the session. capabilities:{} accepts Firefox's defaults.
-  const session = await client.send('session.new', { capabilities: {} });
+  // Open the session. unhandledPromptBehavior:'ignore' stops Firefox from
+  // auto-dismissing JS dialogs (alert/confirm/prompt) before we can handle them
+  // — without it, browsingContext.handleUserPrompt loses the race and fails
+  // with "no such alert" (measured). firefox-page.js wires a userPromptOpened
+  // handler before any navigation, so no dialog is ever left hanging.
+  const session = await client.send('session.new', {
+    capabilities: { alwaysMatch: { unhandledPromptBehavior: 'ignore' } },
+  });
   client.sessionId = session.sessionId;
   client.capabilities = session.capabilities;
   return client;

package/src/blocklist-firefox.js

@@ -0,0 +1,60 @@
+/**
+ * blocklist-firefox.js — Ad/tracker blocking on the Firefox/BiDi engine.
+ *
+ * The BiDi analogue of index.js's applyBlocklist (CDP Network.setBlockedURLs).
+ * BiDi's network.addIntercept can't express our glob patterns — its urlPatterns
+ * reject '*' ("forbidden character *") and have no subdomain wildcard — so we
+ * register a *catch-all* intercept (empty urlPatterns) at the beforeRequestSent
+ * phase and decide per request in-process, matching each URL against the shared
+ * blocklist (makeBlockMatcher). Matches are failed (net error, like CDP's
+ * ERR_BLOCKED_BY_CLIENT); everything else is continued immediately.
+ *
+ * Cost vs. CDP: CDP matches browser-side with zero round-trips. Here every
+ * request pauses for one continue/fail round-trip to Node. On a typical page
+ * (tens–low-hundreds of requests over a local socket) that's negligible, and
+ * it's the only route that preserves glob parity without a second list.
+ */
+
+import { resolveBlocklistPatterns, makeBlockMatcher } from './blocklist.js';
+
+/**
+ * Install ad/tracker blocking on a BiDi session. Idempotent per session is the
+ * caller's concern; call once at connect time before the first navigation.
+ *
+ * @param {object} bidi - BiDi client (send, subscribe, on).
+ * @param {object} pageOpts
+ * @param {boolean} [pageOpts.blockAds] - false disables the default list.
+ * @param {string[]} [pageOpts.blockUrls] - extra CDP-format globs; extend the
+ *   default unless blockAds is false, in which case they're the whole list.
+ * @returns {Promise<void>}
+ */
+export async function applyFirefoxBlocklist(bidi, pageOpts = {}) {
+  const patterns = resolveBlocklistPatterns(pageOpts);
+  if (!patterns.length) return;
+
+  const isBlocked = makeBlockMatcher(patterns);
+
+  await bidi.send('network.addIntercept', {
+    phases: ['beforeRequestSent'],
+    urlPatterns: [],
+  });
+  await bidi.subscribe(['network.beforeRequestSent']);
+
+  bidi.on('network.beforeRequestSent', async (e) => {
+    // Only paused (intercepted) events need a decision; capture-only listeners
+    // see the same event with isBlocked=false and must be ignored here.
+    if (!e.isBlocked) return;
+    const id = e.request?.request;
+    if (!id) return;
+    try {
+      if (isBlocked(e.request.url)) {
+        await bidi.send('network.failRequest', { request: id });
+      } else {
+        await bidi.send('network.continueRequest', { request: id });
+      }
+    } catch {
+      // "no such request" — the request completed or was cancelled between the
+      // event and our reply (redundant nav requests, aborted fetches). Harmless.
+    }
+  });
+}

package/src/blocklist.js

@@ -200,3 +200,52 @@ export const DEFAULT_BLOCKLIST = [
   '*://*.bluekai.com/*',              // Oracle Data Cloud
   '*://*.krxd.net/*',                 // Salesforce / Krux
 ];
+
+/**
+ * Resolve the effective blocklist from a page's blockAds/blockUrls options.
+ * Single-sourced across engines (CDP applyBlocklist + BiDi applyFirefoxBlocklist)
+ * so the merge/extend rule can't drift between them:
+ *   - blockAds !== false → DEFAULT_BLOCKLIST plus any blockUrls (extend);
+ *   - blockAds === false → only blockUrls (the default list is dropped);
+ *   - neither → empty (blocking disabled).
+ *
+ * @param {object} [pageOpts]
+ * @param {boolean} [pageOpts.blockAds] - false drops the default list.
+ * @param {string[]} [pageOpts.blockUrls] - extra CDP-format globs.
+ * @returns {string[]} the patterns to block (possibly empty).
+ */
+export function resolveBlocklistPatterns(pageOpts = {}) {
+  return pageOpts.blockAds === false
+    ? (pageOpts.blockUrls || [])
+    : [...DEFAULT_BLOCKLIST, ...(pageOpts.blockUrls || [])];
+}
+
+/**
+ * Compile CDP-format glob patterns into a single URL-matching predicate.
+ *
+ * CDP blocks natively via Network.setBlockedURLs, which does the glob matching
+ * browser-side. WebDriver BiDi has no glob-capable equivalent — network.
+ * addIntercept's urlPatterns reject '*' outright ("forbidden character *") and
+ * can't express subdomain wildcards like *.doubleclick.net. So the Firefox
+ * path intercepts *all* requests and matches each URL here, in-process, against
+ * the same patterns — keeping the blocklist single-sourced across engines.
+ *
+ * Matches CDP's glob semantics: '*' = any run of characters, '?' = exactly
+ * one character, whole-URL (anchored) match; every other character is literal.
+ *
+ * @param {string[]} patterns - CDP-format globs (e.g. DEFAULT_BLOCKLIST).
+ * @returns {(url: string) => boolean} true when `url` matches any pattern.
+ */
+export function makeBlockMatcher(patterns) {
+  const regexes = patterns.map((p) => {
+    // Escape every regex metachar EXCEPT the two glob wildcards, then expand
+    // them: '*' -> '.*', '?' -> '.'. (Escaping runs first so it never touches
+    // the '.' / '*' we insert next.)
+    const escaped = p
+      .replace(/[.+^${}()|[\]\\]/g, '\\$&')
+      .replace(/\*/g, '.*')
+      .replace(/\?/g, '.');
+    return new RegExp('^' + escaped + '$');
+  });
+  return (url) => regexes.some((re) => re.test(url));
+}

package/src/challenge.js

@@ -0,0 +1,77 @@
+/**
+ * challenge.js — Bot-challenge / interstitial detection for the hybrid fallback.
+ *
+ * Extracted from index.js so both engines can share it without a circular
+ * import (index.js drives CDP; firefox-page.js drives BiDi, and both need to
+ * decide "is this a Cloudflare/hCaptcha wall we should retry headed?"). Pure —
+ * operates on the nested ARIA tree that buildTree() / ariaTree() produce, which
+ * is identical in shape across engines.
+ */
+
+/**
+ * Heuristic: does this ARIA tree look like a bot-challenge / block interstitial
+ * (Cloudflare "Just a moment", hCaptcha, Akamai) rather than real content?
+ *
+ * H9 split: STRONG_PHRASES are essentially-unambiguous challenge UI and fire
+ * regardless of page size; WEAK_PHRASES only fire when the page is ALSO tiny
+ * (so a legitimate-looking error page with "access denied" in its body doesn't
+ * trip the fallback).
+ *
+ * @param {object} tree - Nested ARIA tree (from buildTree / ariaTree)
+ * @param {number} [nodeCount] - Node count (CDP: getFullAXTree; BiDi: tree walk)
+ * @returns {boolean}
+ */
+export function isChallengePage(tree, nodeCount) {
+  if (!tree) return true; // truly empty AX tree — something went wrong fetching the page
+
+  const text = flattenTreeText(tree);
+  const lower = text.toLowerCase();
+
+  // Strong phrases — distinctive enough to identify the challenge product
+  // by name. Fire on their own regardless of node count.
+  const STRONG_PHRASES = [
+    'just a moment',                            // Cloudflare interstitial
+    'checking if the site connection is secure', // Cloudflare
+    'checking your browser',                     // Various JS challenges
+    'verify you are human',                      // hCaptcha / reCAPTCHA
+    'prove your humanity',
+    'attention required',                        // Cloudflare block page
+    'enable javascript and cookies to continue', // Cloudflare
+    'please complete the security check',        // Cloudflare/Akamai
+  ];
+  if (STRONG_PHRASES.some((p) => lower.includes(p))) return true;
+
+  // Weak phrases — show up on real challenge pages but ALSO on legitimate
+  // small error pages. Only count when the page is itself tiny (low node
+  // count or near-empty text), which is the corroborating signal that
+  // separates a real error UI from a challenge skeleton.
+  const WEAK_PHRASES = [
+    'please wait',
+    'request blocked',
+    'access denied',
+    'permission denied',
+    'unknown error',
+    'file a ticket',
+  ];
+  const tinyPage = (nodeCount !== undefined && nodeCount < 30) || text.trim().length < 50;
+  if (tinyPage && WEAK_PHRASES.some((p) => lower.includes(p))) return true;
+
+  return false;
+}
+
+/** Count every node in a nested ARIA tree (shared node-count for the tinyPage test). */
+export function countNodes(node) {
+  if (!node) return 0;
+  let n = 1;
+  for (const c of node.children || []) n += countNodes(c);
+  return n;
+}
+
+function flattenTreeText(node) {
+  if (!node) return '';
+  let text = node.name || '';
+  for (const child of node.children || []) {
+    text += ' ' + flattenTreeText(child);
+  }
+  return text;
+}

package/src/dialog.js

@@ -0,0 +1,52 @@
+/**
+ * dialog.js — Shared JS-dialog decision core for both engines.
+ *
+ * A JS dialog (alert/confirm/prompt/beforeunload) surfaces differently per
+ * protocol — CDP's `Page.javascriptDialogOpening` vs BiDi's
+ * `browsingContext.userPromptOpened` — but the *decision* (accept? what prompt
+ * text?) and the log entry are identical. This module single-sources both so
+ * the CDP path (index.js) and the BiDi path (firefox-page.js) can't drift
+ * (code-review Phase-3 finding #5).
+ */
+
+/**
+ * Build a dialogLog entry from a normalized dialog descriptor.
+ * @param {string} type - 'alert' | 'confirm' | 'prompt' | 'beforeunload'
+ * @param {string} [message]
+ * @returns {{type: string, message: string, timestamp: string}}
+ */
+export function dialogLogEntry(type, message) {
+  return { type, message: message || '', timestamp: new Date().toISOString() };
+}
+
+/**
+ * Decide how to answer a JS dialog. Default policy (both engines): accept
+ * everything except `beforeunload` (dismiss = stay on page); a `prompt`
+ * returns its default text. A caller-installed handler may override via
+ * `{ accept, promptText }`; if the handler throws we keep the defaults so the
+ * page never hangs waiting for a reply that never arrives.
+ *
+ * @param {{type: string, message?: string, defaultPrompt?: string}} dialog
+ * @param {?(function({type,message,defaultPrompt}): (object|Promise<object>))} handler
+ * @returns {Promise<{accept: boolean, promptText: string}>}
+ */
+export async function decideDialog({ type, message, defaultPrompt }, handler) {
+  let accept = type !== 'beforeunload';
+  let promptText = defaultPrompt || '';
+  if (handler) {
+    try {
+      const decision = await handler({
+        type,
+        message: message || '',
+        defaultPrompt: defaultPrompt || '',
+      });
+      if (decision && typeof decision === 'object') {
+        if (typeof decision.accept === 'boolean') accept = decision.accept;
+        if (typeof decision.promptText === 'string') promptText = decision.promptText;
+      }
+    } catch {
+      // Handler threw — keep defaults so the page doesn't hang.
+    }
+  }
+  return { accept, promptText };
+}