barebrowse 0.10.0 → 0.10.1

package/CHANGELOG.md

@@ -1,5 +1,48 @@
 # Changelog
 
+## 0.10.1
+
+### Blocklist long-tail additions + legacy-Chrome warn + switchTab attach-mode test
+
+Carry-forward items from the v0.10.0 backlog. All additive, no behavior
+change on supported Chrome.
+
+- **8 new patterns in `src/blocklist.js`** (120 → 128, still in the
+  curated 80–200 band):
+  - Mobile-measurement-on-web cluster (increasingly served from web
+    pages, not just SDKs): `*.appsflyer.com`, `*.branch.io`,
+    `*.adjust.com`.
+  - Privacy-friendly analytics that still tracks from an agent POV:
+    `static.cloudflareinsights.com` (Cloudflare Web Analytics),
+    `*.matomo.cloud` (Matomo Cloud's hosted tier).
+  - Broader Outbrain coverage: `amplify.outbrain.com`,
+    `log.outbrain.com` (in addition to the existing
+    `widgets.outbrain.com` and `*.outbrain.com/utils/*`).
+  - Broader PostHog: `*.posthog.com/static/array.js*` (the snippet
+    loader, in addition to the existing `/e/` and `/decide/` endpoints).
+- **One-time `console.warn` when `Network.setBlockedURLs` rejects.**
+  Legacy Chromium builds lacking the method previously failed silently
+  inside `applyBlocklist`; now a single warn per process surfaces the
+  reason so callers don't wonder why blocking isn't engaging. Stays
+  silent on supported Chrome (success path), stays silent when
+  `blockAds: false` opts out entirely. Module-scoped flag —
+  intentionally not per-session, since the failure mode is the
+  browser, not the session.
+- **`switchTab()` + `blockAds:true` attach-mode integration test.**
+  The v0.10.0 JSDoc claimed blocklist follows `switchTab()` in attach
+  mode but had no automated guard. New test in
+  `test/integration/blocklist.test.js` launches a real browser, opens
+  a second tab via raw CDP (bypassing barebrowse so the tab simulates
+  one the user already had open), attaches with explicit
+  `blockAds: true` + `blockUrls: [pattern]`, switches into that tab,
+  and asserts the tracker server gets zero hits and the tracker script
+  never executed. Locks in the post-switch `applyBlocklist` call site
+  that was added in v0.10.0.
+- **Tests:** 143 total (5 new). 4 new unit tests in
+  `test/unit/blocklist.test.js` (long-tail coverage drift guard +
+  3-subtest warn-once suite covering rejection, success path, and
+  opted-out paths); 1 new integration test as above.
+
 ## 0.10.0
 
 ### Ad/tracker URL blocking + canvas-noise stealth + Chromium pgid reap fix

package/barebrowse.context.md

@@ -45,7 +45,7 @@ const snapshot = await browse('https://example.com', {
   prune: true,           // apply ARIA pruning (47-95% token reduction)
   pruneMode: 'act',      // 'act' (interactive elements) | 'read' (all content)
   consent: true,         // auto-dismiss cookie consent dialogs
-  blockAds: true,        // block ~120 ad/tracker URL patterns (default on for owned browsers)
+  blockAds: true,        // block 128 ad/tracker URL patterns (default on for owned browsers)
   blockUrls: [],         // extra URL globs to block (merged with the default)
   timeout: 30000,        // navigation timeout in ms
 });
@@ -93,7 +93,7 @@ const snapshot = await browse('https://example.com', {
 - `viewport: '1280x720'` — Set viewport dimensions
 - `storageState: 'file.json'` — Load cookies/localStorage from saved state
 - `downloadPath: '/abs/dir'` — Where downloads land. Default: per-session `mkdtemp` under `/tmp/barebrowse-dl-*` that gets removed on `close()`. Caller-supplied paths are not cleaned up — caller owns the lifecycle.
-- `blockAds: true|false` — CDP-level URL blocking of ~120 common ad/tracker patterns (Google ads/analytics, FB/Amazon/MS/Adobe ad+analytics, Segment/Amplitude/Mixpanel/Heap, Hotjar/FullStory/LogRocket, Criteo/Taboola/Outbrain, the consumer-pixel cluster, AppNexus/Rubicon/PubMatic supply, marketing automation). Default `true` for launched browsers, `false` in attach mode (would affect any tab in the user's running browser). Explicit `true` in attach mode is honored and follows the session across `switchTab()`. Shrinks ARIA snapshots and speeds page loads.
+- `blockAds: true|false` — CDP-level URL blocking of 128 common ad/tracker patterns (Google ads/analytics, FB/Amazon/MS/Adobe ad+analytics, Segment/Amplitude/Mixpanel/Heap/PostHog, Hotjar/FullStory/LogRocket, Criteo/Taboola/Outbrain, the consumer-pixel cluster, AppNexus/Rubicon/PubMatic supply, marketing automation; v0.10.1 added AppsFlyer/Branch/Adjust, Cloudflare Web Analytics, Matomo Cloud). Default `true` for launched browsers, `false` in attach mode (would affect any tab in the user's running browser). Explicit `true` in attach mode is honored and follows the session across `switchTab()` (regression-tested). Shrinks ARIA snapshots and speeds page loads. On legacy Chromium lacking `Network.setBlockedURLs` a one-time `console.warn` surfaces the fallback.
 - `blockUrls: ['*://foo.com/*', ...]` — Extra glob patterns (CDP `Network.setBlockedURLs` format) to block in addition to the default. Merged with the default unless `blockAds: false`.
 
 ## Snapshot format
@@ -166,7 +166,7 @@ barebrowse can inject cookies from the user's real browser sessions, bypassing l
 | SPA navigation | `waitForNavigation()` uses loadEventFired + frameNavigated | Both |
 | Bot detection | v0.9.0 (H9): Cloudflare-strong phrases ("Just a moment", "Attention Required", "verify you are human") fire alone; generic phrases ("access denied", "unknown error") only fire on near-empty pages — no more false-positive headed-launches on legitimate 4xx/5xx pages. `botBlocked` flag set after every `goto()`. Hybrid fallback switches to headed. Snapshot shows `[BOT CHALLENGE DETECTED]` warning. | Hybrid |
 | Stealth (headless tells) | v0.9.0 (H4): `Network.setUserAgentOverride` strips "HeadlessChrome" from UA in HTTP headers AND `navigator.userAgent`; JS patches for webdriver, plugins, languages, full `chrome.runtime` enum shape, `Notification` constructor + `permission: 'default'`, `hardwareConcurrency: 8`, `deviceMemory: 8`, WebGL `UNMASKED_VENDOR_WEBGL`/`UNMASKED_RENDERER_WEBGL` spoofed to Intel. v0.10.0: canvas fingerprint noise — `toDataURL`/`getImageData` XOR a per-session `crypto.getRandomValues`-seeded mask into ~1 byte per 64-byte stride (stable within a session, different across sessions; bitmap is restored after encoding so legitimate canvas use is unaffected). | Headless |
-| Ad / tracker URL blocking | v0.10.0: CDP `Network.setBlockedURLs` with ~120 curated patterns (Google/FB/Amazon/MS/Adobe ad+analytics, the major SaaS analytics + session-replay stacks, content-rec, supply-side ad networks, marketing automation). Default on for launched browsers, off in attach mode. `opts.blockUrls` extends; `opts.blockAds: false` disables. Shrinks ARIA snapshots and speeds loads. | Launched |
+| Ad / tracker URL blocking | v0.10.0: CDP `Network.setBlockedURLs` with 128 curated patterns (Google/FB/Amazon/MS/Adobe ad+analytics, the major SaaS analytics + session-replay stacks, content-rec, supply-side ad networks, marketing automation). v0.10.1 added long-tail: AppsFlyer/Branch/Adjust, Cloudflare Web Analytics, Matomo Cloud, broader Outbrain (`amplify`/`log`) and PostHog (`/static/array.js`). Default on for launched browsers, off in attach mode. `opts.blockUrls` extends; `opts.blockAds: false` disables. Shrinks ARIA snapshots and speeds loads. v0.10.1: regression-tested across `switchTab()` in attach mode; one-time `console.warn` if Chromium lacks the CDP method. | Launched |
 | iframe / OOPIF content (Stripe, reCAPTCHA, embedded forms) | v0.9.0 (H2): `Target.setAutoAttach({flatten:true})` registers a CDP session per iframe; `ariaTree()` walks `Page.getFrameTree`, fetches each frame's AX tree on the right session, splices children under iframe placeholders via `DOM.getFrameOwner`. Refs route via `{session, backendNodeId}` so clicks dispatch in the iframe's Input domain. `--site-per-process` launch flag forces every iframe — including same-origin — into OOPIF so coords work. | Both |
 | Downloads | v0.9.0 (H7): `Browser.setDownloadBehavior({behavior:'allowAndName', downloadPath, eventsEnabled:true})` + listeners populate `page.downloads`. Files land at `savedPath` (under `--download-path` if supplied, else per-session `/tmp/barebrowse-dl-*`). | Headless + Headed (skipped in attach mode) |
 | Profile locking | Unique temp dir per headless instance | Headless |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "barebrowse",
-  "version": "0.10.0",
+  "version": "0.10.1",
   "description": "Authenticated web browsing for autonomous agents via CDP. URL in, pruned ARIA snapshot out.",
   "type": "module",
   "main": "src/index.js",

package/src/blocklist.js

@@ -99,6 +99,8 @@ export const DEFAULT_BLOCKLIST = [
   '*://trc.taboola.com/*',
   '*://widgets.outbrain.com/*',
   '*://*.outbrain.com/utils/*',
+  '*://amplify.outbrain.com/*',
+  '*://log.outbrain.com/*',
 
   // --- Tealium / Marketo / Pardot / Salesforce marketing ---
   '*://tags.tiqcdn.com/*',
@@ -152,6 +154,7 @@ export const DEFAULT_BLOCKLIST = [
   '*://heapanalytics.com/h*',
   '*://*.posthog.com/e/*',
   '*://*.posthog.com/decide/*',
+  '*://*.posthog.com/static/array.js*',
 
   // --- Marketing automation ---
   '*://track.hubspot.com/*',
@@ -170,6 +173,15 @@ export const DEFAULT_BLOCKLIST = [
   '*://sessions.bugsnag.com/*',
   '*://notify.bugsnag.com/*',
 
+  // --- Mobile-measurement (increasingly served on web too) ---
+  '*://*.appsflyer.com/*',
+  '*://*.branch.io/*',
+  '*://*.adjust.com/*',
+
+  // --- Privacy-friendly analytics (still trackers from an agent POV) ---
+  '*://static.cloudflareinsights.com/*',
+  '*://*.matomo.cloud/*',
+
   // --- Misc widely-deployed ad networks ---
   '*://*.adnxs.com/*',                // AppNexus / Xandr
   '*://*.rubiconproject.com/*',

package/src/index.js

@@ -758,12 +758,21 @@ async function attachToExistingTarget(cdp, targetId, pageOpts = {}) {
   return { session, targetId, sessionId, framesByFrameId };
 }
 
+// One-time warn flag for Network.setBlockedURLs reject. Module-scoped so the
+// warn fires once per process across every session — legacy Chrome will keep
+// rejecting and we don't want to spam.
+let blocklistWarned = false;
+
 /**
  * Apply Network.setBlockedURLs for ad/tracker blocking on a session.
  * Default list is on; pass blockAds:false to skip, blockUrls:[] to extend.
- * Silent on failure — older Chrome / unusual modes shouldn't break the page.
+ * On failure (legacy Chrome lacking the method) warns once and continues —
+ * blocking is an enhancement, not a hard requirement.
+ *
+ * Exported for unit testing of the warn-once behavior; not part of the public
+ * API surface.
  */
-async function applyBlocklist(session, pageOpts) {
+export async function applyBlocklist(session, pageOpts) {
   if (pageOpts.blockAds === false && !pageOpts.blockUrls) return;
   const patterns = pageOpts.blockAds === false
     ? (pageOpts.blockUrls || [])
@@ -771,11 +780,19 @@ async function applyBlocklist(session, pageOpts) {
   if (!patterns.length) return;
   try {
     await session.send('Network.setBlockedURLs', { urls: patterns });
-  } catch {
-    // Network.setBlockedURLs unsupported on this Chrome — skip silently.
+  } catch (err) {
+    if (!blocklistWarned) {
+      blocklistWarned = true;
+      console.warn(`barebrowse: Network.setBlockedURLs unsupported — ad/tracker blocking disabled (${err.message})`);
+    }
   }
 }
 
+/** Test-only: reset the warn-once flag. Not part of the public API. */
+export function _resetBlocklistWarning() {
+  blocklistWarned = false;
+}
+
 /**
  * Navigate to a URL and wait for the page to load.
  */