barebrowse 0.17.0 → 0.18.0

package/CHANGELOG.md

@@ -1,5 +1,34 @@
 # Changelog
 
+## [0.18.0] - 2026-07-11
+
+### Added
+
+- **Firefox/BiDi noise-reduction + dialogs (parity plan Phase 3).** Two more
+  Chromium/CDP capabilities now work the same on the Firefox engine:
+  - **Ad/tracker blocking** (`blockAds`, on by default; `blockUrls` to extend).
+    BiDi's `network.addIntercept` can't express our glob patterns — its
+    `urlPatterns` reject `*` outright and have no subdomain wildcard — so the
+    Firefox path registers a *catch-all* `beforeRequestSent` intercept and
+    matches each request URL in-process against the shared `blocklist.js`
+    (new `makeBlockMatcher` compiles the CDP globs to a predicate, single-sourced
+    across engines). Matches are failed like CDP's `ERR_BLOCKED_BY_CLIENT`; the
+    rest continue. Default is **on** (Firefox is always a launched throwaway
+    profile, never attach mode). `src/blocklist-firefox.js`.
+  - **JS dialog handling** (`alert`/`confirm`/`prompt`/`beforeunload`). The BiDi
+    session is created with `unhandledPromptBehavior:'ignore'` so Firefox no
+    longer auto-dismisses prompts before we can act; `page.dialogLog` records
+    every dialog and `page.onDialog(handler)` overrides the default (accept all
+    except `beforeunload`), mirroring the CDP surface exactly.
+
+  BiDi mechanics were POC-measured against real Firefox before wiring (the "no
+  such alert" auto-dismiss race, and that `urlPatterns` reject `*`). Guarded by
+  `test/unit/blocklist-firefox.test.js` (10) and Firefox integration tests
+  (ad-block via a hermetic CORS server so the `blockAds:false` control genuinely
+  passes through; dialog auto-accept + `dialogLog` + custom `onDialog`).
+  Remaining Firefox gaps (Phase 4+): hybrid mode, `saveState`,
+  `waitForNavigation`, downloads, `reload({ignoreCache})`.
+
 ## [0.17.0] - 2026-07-10
 
 ### Added

package/README.md

@@ -145,9 +145,11 @@ const page = await connect({ engine: 'firefox' }); // headless by default
 
 From the CLI: `barebrowse open <url> --engine firefox`. From MCP: set
 `BAREBROWSE_ENGINE=firefox`. Firefox cookies (plaintext) reuse into the same
-engine. Consent auto-dismiss and headless stealth work on Firefox too (as of
-v0.16.0). Chromium (CDP) remains the default; `hybrid` mode, ad/tracker
-blocking, and the daemon's console/network capture are still Chromium-only.
+engine. Consent auto-dismiss and headless stealth (v0.16.0), the daemon's
+console/network capture and `waitForNetworkIdle` (v0.17.0), and ad/tracker
+blocking plus JS dialog handling (`dialogLog`/`onDialog`, v0.18.0) all work on
+Firefox too. Chromium (CDP) remains the default; only `hybrid` mode is still
+Chromium-only.
 
 No clone profile, no fresh cookies — the agent sees what you see.
 

package/barebrowse.context.md

@@ -95,7 +95,7 @@ const snapshot = await browse('https://example.com', {
 | `close()` | -- | void | Close page, disconnect CDP, kill browser (if headless) |
 
 **connect() options** (in addition to mode/port/consent):
-- `engine: 'chromium'|'firefox'` — Browser engine. Default `chromium` (CDP). `firefox` drives Firefox over WebDriver BiDi (CDP is deprecated there) — a separate transport with the same `page.*` API; the ARIA snapshot is reconstructed in-page since BiDi has no `getFullAXTree`. Consent auto-dismiss and stealth (headless anti-detection) work on Firefox as of v0.16.0; the CLI daemon's console/network capture and `waitForNetworkIdle` work as of v0.17.0. Still Chromium-only: `hybrid` mode and ad/tracker blocking. `reload()` ignores `ignoreCache` on Firefox (BiDi limitation).
+- `engine: 'chromium'|'firefox'` — Browser engine. Default `chromium` (CDP). `firefox` drives Firefox over WebDriver BiDi (CDP is deprecated there) — a separate transport with the same `page.*` API; the ARIA snapshot is reconstructed in-page since BiDi has no `getFullAXTree`. Consent auto-dismiss and stealth (headless anti-detection) work on Firefox as of v0.16.0; the CLI daemon's console/network capture and `waitForNetworkIdle` work as of v0.17.0; ad/tracker blocking (`blockAds`/`blockUrls`) and JS dialog handling (`dialogLog`/`onDialog`) work as of Phase 3. Still Chromium-only: `hybrid` mode. `reload()` ignores `ignoreCache` on Firefox (BiDi limitation).
 - `port: 9222` — Attach to a Chromium already running with `--remote-debugging-port=N` instead of spawning one. The browser keeps running on `close()`. Stealth + permission denial + download capture are skipped to avoid mutating the user's running browser.
 - `proxy: 'http://...'` — HTTP/SOCKS proxy for browser
 - `viewport: '1280x720'` — Set viewport dimensions

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "barebrowse",
-  "version": "0.17.0",
+  "version": "0.18.0",
   "description": "Authenticated web browsing for autonomous agents via CDP. URL in, pruned ARIA snapshot out.",
   "repository": {
     "type": "git",

package/src/bidi.js

@@ -135,8 +135,14 @@ export async function createBiDi(wsUrl) {
     close() { ws.close(); },
   };
 
-  // Open the session. capabilities:{} accepts Firefox's defaults.
-  const session = await client.send('session.new', { capabilities: {} });
+  // Open the session. unhandledPromptBehavior:'ignore' stops Firefox from
+  // auto-dismissing JS dialogs (alert/confirm/prompt) before we can handle them
+  // — without it, browsingContext.handleUserPrompt loses the race and fails
+  // with "no such alert" (measured). firefox-page.js wires a userPromptOpened
+  // handler before any navigation, so no dialog is ever left hanging.
+  const session = await client.send('session.new', {
+    capabilities: { alwaysMatch: { unhandledPromptBehavior: 'ignore' } },
+  });
   client.sessionId = session.sessionId;
   client.capabilities = session.capabilities;
   return client;

package/src/blocklist-firefox.js

@@ -0,0 +1,60 @@
+/**
+ * blocklist-firefox.js — Ad/tracker blocking on the Firefox/BiDi engine.
+ *
+ * The BiDi analogue of index.js's applyBlocklist (CDP Network.setBlockedURLs).
+ * BiDi's network.addIntercept can't express our glob patterns — its urlPatterns
+ * reject '*' ("forbidden character *") and have no subdomain wildcard — so we
+ * register a *catch-all* intercept (empty urlPatterns) at the beforeRequestSent
+ * phase and decide per request in-process, matching each URL against the shared
+ * blocklist (makeBlockMatcher). Matches are failed (net error, like CDP's
+ * ERR_BLOCKED_BY_CLIENT); everything else is continued immediately.
+ *
+ * Cost vs. CDP: CDP matches browser-side with zero round-trips. Here every
+ * request pauses for one continue/fail round-trip to Node. On a typical page
+ * (tens–low-hundreds of requests over a local socket) that's negligible, and
+ * it's the only route that preserves glob parity without a second list.
+ */
+
+import { resolveBlocklistPatterns, makeBlockMatcher } from './blocklist.js';
+
+/**
+ * Install ad/tracker blocking on a BiDi session. Idempotent per session is the
+ * caller's concern; call once at connect time before the first navigation.
+ *
+ * @param {object} bidi - BiDi client (send, subscribe, on).
+ * @param {object} pageOpts
+ * @param {boolean} [pageOpts.blockAds] - false disables the default list.
+ * @param {string[]} [pageOpts.blockUrls] - extra CDP-format globs; extend the
+ *   default unless blockAds is false, in which case they're the whole list.
+ * @returns {Promise<void>}
+ */
+export async function applyFirefoxBlocklist(bidi, pageOpts = {}) {
+  const patterns = resolveBlocklistPatterns(pageOpts);
+  if (!patterns.length) return;
+
+  const isBlocked = makeBlockMatcher(patterns);
+
+  await bidi.send('network.addIntercept', {
+    phases: ['beforeRequestSent'],
+    urlPatterns: [],
+  });
+  await bidi.subscribe(['network.beforeRequestSent']);
+
+  bidi.on('network.beforeRequestSent', async (e) => {
+    // Only paused (intercepted) events need a decision; capture-only listeners
+    // see the same event with isBlocked=false and must be ignored here.
+    if (!e.isBlocked) return;
+    const id = e.request?.request;
+    if (!id) return;
+    try {
+      if (isBlocked(e.request.url)) {
+        await bidi.send('network.failRequest', { request: id });
+      } else {
+        await bidi.send('network.continueRequest', { request: id });
+      }
+    } catch {
+      // "no such request" — the request completed or was cancelled between the
+      // event and our reply (redundant nav requests, aborted fetches). Harmless.
+    }
+  });
+}

package/src/blocklist.js

@@ -200,3 +200,52 @@ export const DEFAULT_BLOCKLIST = [
   '*://*.bluekai.com/*',              // Oracle Data Cloud
   '*://*.krxd.net/*',                 // Salesforce / Krux
 ];
+
+/**
+ * Resolve the effective blocklist from a page's blockAds/blockUrls options.
+ * Single-sourced across engines (CDP applyBlocklist + BiDi applyFirefoxBlocklist)
+ * so the merge/extend rule can't drift between them:
+ *   - blockAds !== false → DEFAULT_BLOCKLIST plus any blockUrls (extend);
+ *   - blockAds === false → only blockUrls (the default list is dropped);
+ *   - neither → empty (blocking disabled).
+ *
+ * @param {object} [pageOpts]
+ * @param {boolean} [pageOpts.blockAds] - false drops the default list.
+ * @param {string[]} [pageOpts.blockUrls] - extra CDP-format globs.
+ * @returns {string[]} the patterns to block (possibly empty).
+ */
+export function resolveBlocklistPatterns(pageOpts = {}) {
+  return pageOpts.blockAds === false
+    ? (pageOpts.blockUrls || [])
+    : [...DEFAULT_BLOCKLIST, ...(pageOpts.blockUrls || [])];
+}
+
+/**
+ * Compile CDP-format glob patterns into a single URL-matching predicate.
+ *
+ * CDP blocks natively via Network.setBlockedURLs, which does the glob matching
+ * browser-side. WebDriver BiDi has no glob-capable equivalent — network.
+ * addIntercept's urlPatterns reject '*' outright ("forbidden character *") and
+ * can't express subdomain wildcards like *.doubleclick.net. So the Firefox
+ * path intercepts *all* requests and matches each URL here, in-process, against
+ * the same patterns — keeping the blocklist single-sourced across engines.
+ *
+ * Matches CDP's glob semantics: '*' = any run of characters, '?' = exactly
+ * one character, whole-URL (anchored) match; every other character is literal.
+ *
+ * @param {string[]} patterns - CDP-format globs (e.g. DEFAULT_BLOCKLIST).
+ * @returns {(url: string) => boolean} true when `url` matches any pattern.
+ */
+export function makeBlockMatcher(patterns) {
+  const regexes = patterns.map((p) => {
+    // Escape every regex metachar EXCEPT the two glob wildcards, then expand
+    // them: '*' -> '.*', '?' -> '.'. (Escaping runs first so it never touches
+    // the '.' / '*' we insert next.)
+    const escaped = p
+      .replace(/[.+^${}()|[\]\\]/g, '\\$&')
+      .replace(/\*/g, '.*')
+      .replace(/\?/g, '.');
+    return new RegExp('^' + escaped + '$');
+  });
+  return (url) => regexes.some((re) => re.test(url));
+}

package/src/firefox-page.js

@@ -170,6 +170,51 @@ export async function createFirefoxPage(bidi, opts = {}) {
     return Promise.race([p, t]).finally(() => clearTimeout(timer));
   }
 
+  // JS dialog handling (alert/confirm/prompt/beforeunload), parity with the
+  // CDP path in index.js. The BiDi session is created with
+  // unhandledPromptBehavior:'ignore' (see bidi.js) so prompts stay open until
+  // we respond; setupDialogs() must run before the first navigation or an
+  // 'ignore' prompt would hang the page. Default: accept everything except
+  // beforeunload (dismiss = stay), mirroring the CDP default. A caller can
+  // override per-dialog via page.onDialog(handler).
+  const dialogLog = [];
+  let onDialogHandler = null;
+  async function setupDialogs() {
+    await bidi.subscribe(['browsingContext.userPromptOpened']);
+    bidi.on('browsingContext.userPromptOpened', async (e) => {
+      dialogLog.push({
+        type: e.type,
+        message: e.message || '',
+        timestamp: new Date().toISOString(),
+      });
+      let accept = e.type !== 'beforeunload';
+      let userText = e.defaultValue || '';
+      if (onDialogHandler) {
+        try {
+          const decision = await onDialogHandler({
+            type: e.type,
+            message: e.message || '',
+            defaultPrompt: e.defaultValue || '',
+          });
+          if (decision && typeof decision === 'object') {
+            if (typeof decision.accept === 'boolean') accept = decision.accept;
+            if (typeof decision.promptText === 'string') userText = decision.promptText;
+          }
+        } catch {
+          // Handler threw — fall back to defaults so the prompt is still
+          // answered and the page doesn't hang on an 'ignore' dialog.
+        }
+      }
+      try {
+        await bidi.send('browsingContext.handleUserPrompt', {
+          context: e.context, accept, userText,
+        });
+      } catch {
+        // Prompt already gone (closed by page JS / navigation). Nothing to do.
+      }
+    });
+  }
+
   const page = {
     /** The BiDi escape hatch, analogous to connect()'s page.cdp. */
     get bidi() { return bidi; },
@@ -413,12 +458,25 @@ export async function createFirefoxPage(bidi, opts = {}) {
 
     // --- CDP-only surfaces, stubbed for daemon parity ---------------------
     // The daemon (src/daemon.js) dispatches these unconditionally. On the
-    // Firefox/BiDi engine download tracking and dialog capture aren't wired
-    // (Phase 3/4), so the two logs are genuinely empty; the two actions are
-    // CDP-only and fail with a clear, intentional message instead of an
-    // incidental TypeError. Documented as a known gap in CHANGELOG.
+    // Firefox/BiDi engine download tracking isn't wired (Phase 4), so downloads
+    // is genuinely empty; saveState/waitForNavigation are CDP-only and fail with
+    // a clear, intentional message instead of an incidental TypeError.
+    // (dialogLog + onDialog ARE wired — Phase 3, see setupDialogs above.)
+    // Documented as a known gap in CHANGELOG.
     get downloads() { return []; },
-    get dialogLog() { return []; },
+
+    dialogLog,
+
+    /**
+     * Install a custom JS dialog handler, mirroring connect()'s page.onDialog.
+     * Called with `{ type, message, defaultPrompt }`; may return (sync or async)
+     * `{ accept: bool, promptText: string }` to override the auto-accept
+     * default. Pass null to restore default behavior.
+     */
+    onDialog(handler) {
+      onDialogHandler = handler;
+    },
+
     async saveState() {
       throw new Error('saveState() is not supported on the Firefox/BiDi engine (CDP-only)');
     },
@@ -441,5 +499,9 @@ export async function createFirefoxPage(bidi, opts = {}) {
     await new Promise((r) => setTimeout(r, 500));
   }
 
+  // Wire the dialog handler before returning — must precede any navigation so
+  // an 'ignore' prompt (see bidi.js capability) is never left hanging.
+  await setupDialogs();
+
   return page;
 }

package/src/index.js

@@ -20,7 +20,8 @@ import { click as cdpClick, type as cdpType, scroll as cdpScroll, press as cdpPr
 import { dismissConsent } from './consent.js';
 import { applyStealth } from './stealth.js';
 import { applyFirefoxStealth } from './stealth-firefox.js';
-import { DEFAULT_BLOCKLIST } from './blocklist.js';
+import { applyFirefoxBlocklist } from './blocklist-firefox.js';
+import { resolveBlocklistPatterns } from './blocklist.js';
 import { waitForNetworkIdle } from './network-idle.js';
 import { readable as extractReadable } from './readable.js';
 import { assertNavigable, assertUploadAllowed } from './url-guard.js';
@@ -747,9 +748,12 @@ async function suppressPermissions(cdp) {
  * ref model, and AX-tree source all differ (see firefox-page.js). close() is
  * wrapped to also reap the Firefox process + temp profile.
  *
- * Chromium-only options NOT applied on this path: `blockAds`/`blockUrls` and
- * `hybrid` mode. `saveState`, `waitForNavigation`, and download/dialog capture
- * are CDP-only too (the page object stubs them — see firefox-page.js).
+ * Chromium-only options NOT applied on this path: `hybrid` mode. `saveState`,
+ * `waitForNavigation`, and download capture are CDP-only too (the page object
+ * stubs them — see firefox-page.js). `blockAds`/`blockUrls` and JS dialog
+ * handling (`dialogLog`/`onDialog`) DO apply as of Phase 3 (ad-block via a
+ * catch-all `network.addIntercept` + in-process glob match; dialogs via
+ * `browsingContext.userPromptOpened` → `handleUserPrompt`).
  * `waitForNetworkIdle` and daemon console/network capture DO apply as of Phase
  * 2 (over BiDi `network.*` and `log.entryAdded` events). `consent`
  * (auto-dismiss) and
@@ -775,6 +779,14 @@ async function connectFirefox(opts) {
   if (opts.mode !== 'headed') {
     await applyFirefoxStealth(bidi);
   }
+  // Ad/tracker blocking (parity with the CDP applyBlocklist). Firefox is always
+  // a launched, throwaway profile (never attach mode), so the default is on —
+  // unlike the CDP path, which defaults off in attach mode. Must run before the
+  // first navigation so the catch-all intercept is live when the page loads.
+  await applyFirefoxBlocklist(bidi, {
+    blockAds: opts.blockAds !== undefined ? opts.blockAds : true,
+    blockUrls: opts.blockUrls,
+  });
   const page = await createFirefoxPage(bidi, {
     pruneMode: opts.pruneMode,
     urlGuard: { allowLocalUrls: opts.allowLocalUrls, blockPrivateNetwork: opts.blockPrivateNetwork },
@@ -911,10 +923,7 @@ let blocklistWarned = false;
  * API surface.
  */
 export async function applyBlocklist(session, pageOpts) {
-  if (pageOpts.blockAds === false && !pageOpts.blockUrls) return;
-  const patterns = pageOpts.blockAds === false
-    ? (pageOpts.blockUrls || [])
-    : [...DEFAULT_BLOCKLIST, ...(pageOpts.blockUrls || [])];
+  const patterns = resolveBlocklistPatterns(pageOpts);
   if (!patterns.length) return;
   try {
     await session.send('Network.setBlockedURLs', { urls: patterns });

package/types/blocklist-firefox.d.ts

@@ -0,0 +1,15 @@
+/**
+ * Install ad/tracker blocking on a BiDi session. Idempotent per session is the
+ * caller's concern; call once at connect time before the first navigation.
+ *
+ * @param {object} bidi - BiDi client (send, subscribe, on).
+ * @param {object} pageOpts
+ * @param {boolean} [pageOpts.blockAds] - false disables the default list.
+ * @param {string[]} [pageOpts.blockUrls] - extra CDP-format globs; extend the
+ *   default unless blockAds is false, in which case they're the whole list.
+ * @returns {Promise<void>}
+ */
+export function applyFirefoxBlocklist(bidi: object, pageOpts?: {
+    blockAds?: boolean | undefined;
+    blockUrls?: string[] | undefined;
+}): Promise<void>;

package/types/blocklist.d.ts

@@ -1,3 +1,37 @@
+/**
+ * Resolve the effective blocklist from a page's blockAds/blockUrls options.
+ * Single-sourced across engines (CDP applyBlocklist + BiDi applyFirefoxBlocklist)
+ * so the merge/extend rule can't drift between them:
+ *   - blockAds !== false → DEFAULT_BLOCKLIST plus any blockUrls (extend);
+ *   - blockAds === false → only blockUrls (the default list is dropped);
+ *   - neither → empty (blocking disabled).
+ *
+ * @param {object} [pageOpts]
+ * @param {boolean} [pageOpts.blockAds] - false drops the default list.
+ * @param {string[]} [pageOpts.blockUrls] - extra CDP-format globs.
+ * @returns {string[]} the patterns to block (possibly empty).
+ */
+export function resolveBlocklistPatterns(pageOpts?: {
+    blockAds?: boolean | undefined;
+    blockUrls?: string[] | undefined;
+}): string[];
+/**
+ * Compile CDP-format glob patterns into a single URL-matching predicate.
+ *
+ * CDP blocks natively via Network.setBlockedURLs, which does the glob matching
+ * browser-side. WebDriver BiDi has no glob-capable equivalent — network.
+ * addIntercept's urlPatterns reject '*' outright ("forbidden character *") and
+ * can't express subdomain wildcards like *.doubleclick.net. So the Firefox
+ * path intercepts *all* requests and matches each URL here, in-process, against
+ * the same patterns — keeping the blocklist single-sourced across engines.
+ *
+ * Matches CDP's glob semantics: '*' = any run of characters, '?' = exactly
+ * one character, whole-URL (anchored) match; every other character is literal.
+ *
+ * @param {string[]} patterns - CDP-format globs (e.g. DEFAULT_BLOCKLIST).
+ * @returns {(url: string) => boolean} true when `url` matches any pattern.
+ */
+export function makeBlockMatcher(patterns: string[]): (url: string) => boolean;
 /**
  * blocklist.js — Ad/tracker URL patterns for CDP Network.setBlockedURLs.
  *