@fanboynz/network-scanner 3.1.0 → 3.2.0

package/CHANGELOG.md

@@ -2,6 +2,82 @@
 
 All notable changes to the Network Scanner (nwss.js) project.
 
+## [3.2.0] - 2026-06-04
+
+### Added
+- **`output_regex`** site option — a per-site regex whose capture group 1 (or whole match) becomes the rule body, so output can be a path-prefix rule like `||host/script/` instead of `||host^`. Collapses randomized filenames under a stable path into one rule and lets you block a folder on a host that also serves legit content; falls back to `||host^` when the regex doesn't match. Adblock-only — domain-based formats (dnsmasq/unbound/pi-hole/hosts/plain) emit the bare host. Compiled once per pattern (memoized) and validated at config load.
+- **dig resolver failover** — `digLookup` now fails over through the `--dns` resolvers on timeout / no-reply / `REFUSED` / `SERVFAIL` (up to 3 attempts, `+time=2 +tries=1` each), matching the resilience the whois retry and DNS pre-check rotation already had. With no `--dns`, the system-resolver path keeps dig's native `resolv.conf` rotation unchanged.
+
+### Changed
+- **Ghost-cursor coordinate clicks now use the same realistic press as the built-in content clicks** (`humanClick`): hover dwell + mousedown/hold/mouseup, plus hand-tremor during the hold and a mouseup drift (so mousedown ≠ mouseup coordinates) when `realistic_click` is set — replacing a 0ms `page.mouse.click`.
+- **Ghost-cursor clicks honor `interact_click_count`** (default 3, cap 20) instead of firing a single click — ad SDKs often swallow the 1st/2nd click as warmup. The bezier movement loop reserves part of `ghost_cursor_duration` for the clicks (raise the duration to fit more; the default 2000ms fits ~1 realistic click).
+- **`dig` success is judged by RCODE, not stderr** — a dig that prints a transient `communications error` warning but still returns a valid `ANSWER SECTION` is no longer discarded.
+- **dig-only configs skip the whois root-domain parse** per request (small per-request saving when no `whois`/`whois-or` is configured).
+
+### Fixed
+- **`max_redirects: 0`** now means "follow none" instead of silently becoming 10 (the `|| 10` falsy-zero bug in `nwss.js` and `lib/redirect.js`).
+- **A `REFUSED`/`SERVFAIL` dig that exhausts all resolvers returns failure** so it isn't cached — a transient resolver-side error no longer poisons a domain for the cache TTL.
+- **Ghost-cursor coordinate click no longer reports false success** — it returned `true` (and logged "Clicked") even when the click was silently skipped for lack of a page; it now returns `false` and logs the skip.
+
+### Removed
+- **`follow_redirects`** site option — documented in `--help`, the man page, the README, and example configs but never wired to any runtime behavior; removed from the docs. Use `max_redirects` instead (`0` = follow none).
+
+### Security
+- **dig argv-injection guard** — `digLookup` rejects non-hostname-shaped input before shelling out. `dig` has no `--` end-of-options marker (unlike whois) and parses `@`/`-`/`+`-leading argv tokens as options, so a crafted "domain" like `@evil-resolver` (redirects the query to an arbitrary server) or `-f /path` (reads a file as a query batch) is now rejected — out-of-charset or dash-leading values fall back to no-match.
+
+## [3.1.2] - 2026-05-30
+
+### Changed
+- **Fingerprint identity pinned to Stable Chrome 148**, not whatever Chrome-for-Testing puppeteer bundles (currently 149, ahead of Stable). The spoof must blend with the real-world population; claiming an unreleased build is itself a tell. The Chrome major + build (`CHROME_BUILD`) + GREASE brand (`CHROME_GREASE_BRAND`) are now single constants — see `lib/fingerprint.md`.
+- **UA Client Hints made fully consistent and matched to real Chrome 148** (verified field-for-field against a live desktop): brand-list order + GREASE string (`Not/A)Brand`), and the full-version build (`148.0.7778.217`) sourced from one place so JS `getHighEntropyValues` and the HTTP `Sec-CH-UA-Full-Version*` headers can't drift. Added `wow64`, `model`, `formFactors`, `uaFullVersion`, and `Sec-CH-UA-WoW64`/`-Model`/`-Form-Factors` headers; Windows `platformVersion` → `19.0.0`.
+- **`navigator.deviceMemory` and `Sec-CH-Device-Memory` both pinned to `8`** (consistent JS↔HTTP), hiding the host's real RAM; `hardwareConcurrency` reports 4–8 (hides datacenter core count).
+- **Dependencies**: puppeteer / puppeteer-core 25.1.0, lru-cache 11.5.1.
+
+### Fixed
+- **Timezone is now spoofed via CDP `emulateTimezone`** instead of JS overrides, so `Date`, `Intl`, and `getTimezoneOffset` are all consistent and DST-correct. The old JS patching left the real `Date` in the host zone — an 8-hour `Date`-vs-`Intl` contradiction and a leaked host timezone.
+- **Closed several headless tells**: Battery now reports the plugged-in default (`charging:true, level:1`); `navigator.bluetooth`, `navigator.share`/`canShare` stubs added (present in real Chrome, absent in headless); `speechSynthesis.getVoices()` returns the claimed-OS voice set (`instanceof`-correct).
+- **proxy**: a string `proxy_bypass`/`socks5_bypass` (instead of an array) no longer throws `bypass.join is not a function` in the browser-launch path.
+- **socks-relay**: a client that disconnects during the upstream-connect await is now handled, so a tunnel isn't opened for a gone client and the watchdog clears immediately.
+- **smart-cache**: the memory-check and auto-save `setInterval`s are now `unref`'d, so an error path that skips `destroy()` can no longer hang the process.
+
+### Removed
+- Dead code: `browserhealth` `testNetworkCapability` + `purgeStaleTrackers` (zero callers), and a redundant 2-voice `speechSynthesis` block superseded by the full voice set.
+
+### Added
+- **`lib/fingerprint.md`** — fingerprint spoofing coverage tables (surfaces, mitigations, gating flags) and known limitations.
+
+## [3.1.0] - 2026-05-29
+
+### Added
+- **`realistic_click`** site flag — denser mouse approach, hold tremor, and mouseup drift for sites that score click realism.
+- **`interact_click_count`** site override for popunder-discovery click volume (default content-click count also raised 2 → 3).
+- **`clear_sitedata_full_on_reload`** site flag — full storage clear between reloads; quick mode now also clears localStorage/sessionStorage.
+- **regex-tool rewritten** as a real `filterRegex` builder/tester: literal↔standard↔JSON conversion, multi-pattern + `regex_and`, and testing against real request URLs (matching mirrors the scanner exactly).
+- **Fingerprint coverage**: per-domain-seeded Battery / `navigator.connection` values, `AudioBuffer` fingerprint defeat, `PerformanceNavigationTiming` jitter, `userActivation`; UA strings bumped to Chrome 148 / Firefox 151 / Safari 19.5.
+
+### Changed
+- **`userAgent` now defaults to `"chrome"`** when a site doesn't set one — previously sites without it leaked the bundled `HeadlessChrome` UA.
+- **`Sec-CH-UA` headers and the curl content-fetch UA derive from the single UA source**, so Client Hints can't drift from `navigator.userAgent`.
+- **VPN configs force scan concurrency to 1** — the shared system routing table isn't concurrency-safe.
+- **Interaction time ceiling scales with the work envelope** (click count / `realistic_click`) instead of a flat 15s.
+
+### Fixed
+- **Per-URL timeout scales** with site timeout/delay/reload (+8s recovery grace) instead of a flat 75s that discarded partial-match recovery on multi-URL scans.
+- **Interaction hard cap is now actually enforced** (was cooperative, overshooting to 20s+ under concurrency).
+- **WireGuard** inline temp-config leaked the private key on failed connect and broke retries; temp dir is now per-PID so concurrent processes can't wipe each other's config.
+- **nettools**: fixed a dig dedup race (concurrent same-domain double lookups); whois no longer discards valid records over non-fatal stderr.
+- **Orphan resource leaks** on `Promise.race` timeout (cdp.js, clear_sitedata.js, browserhealth.js) and several un-`unref`'d `setTimeout` handles.
+- **Config keys validated at startup** with boolean-like coercion, preventing silent misconfiguration.
+
+### Security
+- **OpenVPN** `pkill`/`ping`/`curl` calls moved from shell-interpolated `execSync` to `spawnSync` arg arrays (command-injection).
+- **WireGuard/OpenVPN interface & connection names validated** against a strict charset before use in paths/commands.
+
+### Performance
+- **adblock**: O(1) exact-domain lookup for `$third-party` / `$first-party` rules.
+- Parallelized site-data clearing and window-cleanup checks.
+- Removed dead code across cdp, domain-cache, searchstring, compress, adblock-rust, and nettools.
+
 ## [3.0.3] - 2026-05-26
 
 ### Improved

package/CLAUDE.md

@@ -6,7 +6,7 @@ Puppeteer-based network scanner for analyzing web traffic, generating adblock fi
 
 - `nwss.js` — Main entry point (~5,800 lines). CLI args, URL processing, orchestration.
 - `config.json` — Default scan configuration (sites, filters, options).
-- `lib/` — 32 focused, single-purpose modules:
+- `lib/` — 33 focused, single-purpose modules:
   - `fingerprint.js` — Bot detection evasion (device/GPU/timezone spoofing)
   - `cloudflare.js` — Cloudflare challenge detection and solving
   - `browserhealth.js` — Memory management and browser lifecycle
@@ -14,6 +14,7 @@ Puppeteer-based network scanner for analyzing web traffic, generating adblock fi
   - `ghost-cursor.js` — Bezier-curve cursor pathing for human-like mouse movement
   - `smart-cache.js` — Multi-layer caching with persistence
   - `nettools.js` — WHOIS/dig integration
+  - `dns.js` — DNS pre-check resolver: multi-nameserver rotation + `--dns` override (pre-check only; not Chrome/dig)
   - `output.js` — Multi-format rule output (adblock, dnsmasq, unbound, pihole, etc.)
   - `proxy.js` — SOCKS5/HTTP proxy support
   - `socks-relay.js` — Local SOCKS proxy relay/chain helper

package/README.md

@@ -17,6 +17,7 @@ A Puppeteer-based tool for scanning websites to find third-party (or optionally
 - Subdomain handling (collapse to root or full subdomain)
 - Optionally match only first-party, third-party, or both
 - Enhanced redirect handling with JavaScript and meta refresh detection
+- Capture and drive popup/popunder chains (`capture_popups` + `interact_popups`) so domains reachable only via a clicked popup still match
 - Per-site proxy routing (SOCKS5, SOCKS4, HTTP, HTTPS) with pre-flight health checks
 
 ---
@@ -50,7 +51,6 @@ A Puppeteer-based tool for scanning websites to find third-party (or optionally
 
 | Argument                  | Description |
 |:---------------------------|:------------|
-| `--verbose`                 | Force verbose mode globally |
 | `--debug`                   | Force debug mode globally |
 | `--silent`                  | Suppress normal console logs |
 | `--titles`                  | Add `! <url>` title before each site's group |
@@ -66,7 +66,7 @@ A Puppeteer-based tool for scanning websites to find third-party (or optionally
 | `--use-puppeteer-core`      | Use `puppeteer-core` with system Chrome instead of bundled Chromium |
 | `--use-obscura`             | Connect to running Obscura CDP server (`ws://127.0.0.1:9222` or `OBSCURA_WS` env). Skips fingerprint injection — Obscura provides built-in stealth |
 | `--load-extension <path>`   | Load unpacked Chrome extension from directory (can be used multiple times) |
-| `--dns-cache`               | Persist dig/whois results to disk between runs (20hr TTL, 2000-entry cap each, `.digcache`/`.whoiscache`). Disk writes are atomic (tmp + rename); corrupt cache files are detected on load with a `[dns-cache]` warn line and reset cleanly. |
+| `--dns-cache`               | Persist dig/whois results to disk between runs (20hr TTL, 2000-entry cap each, `.digcache`/`.whoiscache`), **plus** the DNS pre-check negative cache (NXDOMAIN/ENODATA only — never resolver errors — 12h TTL, `.dnsnegcache`) so known-dead hosts aren't re-resolved next run. Disk writes are atomic (tmp + rename); corrupt cache files are detected on load with a `[dns-cache]` warn line and reset cleanly. |
 | `--no-dns-precheck`         | Disable per-URL DNS resolution check before page navigation. By default, hosts that dig/whois have already proven live (within the 20hr cache TTL) skip their c-ares pre-check via a positive-resolution index. |
 | `--block-ads=<files>`       | Block ads using EasyList format rules (comma-separated: `easylist.txt,easyprivacy.txt`) |
 | `--cdp`                     | Enable Chrome DevTools Protocol logging (now per-page if enabled) |
@@ -76,6 +76,8 @@ A Puppeteer-based tool for scanning websites to find third-party (or optionally
 | `--help`, `-h`              | Show this help menu |
 | `--version`                 | Show script version |
 | `--max-concurrent <number>` | Maximum concurrent site processing (1-50, overrides config/default) |
+| `--dns <ip[,ip,...]>` | Resolver(s) for the DNS pre-check **and** nettools' `dig` (one pins, several rotate per query; overrides `/etc/resolv.conf`). Does not affect Chrome navigation or `whois`. Useful when the system resolver is flaky and `dig`-gated domains time out |
+| `--show-dead-domains` | At end of scan, list hostnames that did not resolve / were unreachable (`NXDOMAIN`/`ENODATA` + `ERR_NAME_NOT_RESOLVED`/`ERR_ADDRESS_UNREACHABLE`). Excludes blocks/timeouts (those mean the domain is alive). For pruning dead URLs. |
 | `--cleanup-interval <number>` | Browser restart interval in URLs processed (1-1000, overrides config/default) |
 
 ### Validation Options
@@ -152,6 +154,7 @@ Example:
 | `userAgent`          | `chrome`, `chrome_mac`, `chrome_linux`, `firefox`, `firefox_mac`, `firefox_linux`, `safari` | - | User agent for page |
 | `filterRegex`        | String or Array | `.*` | Regex or list of regexes to match requests |
 | `regex_and`          | Boolean | `false` | Use AND logic for multiple filterRegex patterns - ALL patterns must match the same URL |
+| `output_regex`       | String | — | Regex applied to each matched URL to build the rule body: capture group 1 (or whole match) becomes `\|\|<capture>` instead of `\|\|host^`. E.g. `^https?:\/\/([^\/]+\/[^\/]+\/)` turns `https://host.com/script/abc.js` into `\|\|host.com/script/`. The capture must include the host. No match → falls back to `\|\|host^`. Adblock-only; domain formats (dnsmasq/pihole/hosts/plain) emit the bare host |
 | `comments`           | String or Array | - | String of comments or references |
 | `resourceTypes`      | Array | `["script", "xhr", "image", "stylesheet"]` | What resource types to monitor |
 | `reload`             | Integer | `1` | Number of times to reload page |
@@ -176,8 +179,7 @@ Example:
 
 | Field                | Values | Default | Description |
 |:---------------------|:-------|:-------:|:------------|
-| `follow_redirects`   | Boolean | `true` | Follow redirects to new domains |
-| `max_redirects`      | Integer | `10` | Maximum number of redirects to follow |
+| `max_redirects`      | Integer | `10` | Maximum number of redirects to follow (`0` = follow none) |
 | `js_redirect_timeout` | Milliseconds | `5000` | Time to wait for JavaScript redirects |
 | `detect_js_patterns` | Boolean | `true` | Analyze page source for redirect patterns |
 | `redirect_timeout_multiplier` | Number | `1.5` | Increase timeout for redirected URLs |
@@ -279,6 +281,8 @@ When a page redirects to a new domain, first-party/third-party detection is base
 | `interact_duration`  | Milliseconds | `2000` | Duration of interaction simulation |
 | `interact_scrolling` | Boolean | `true` | Enable scrolling simulation |
 | `interact_clicks`    | Boolean | `false` | Enable element clicking simulation |
+| `interact_click_count` | Integer | `3` | Number of random content-zone clicks per load (capped at 20). Default 3 = primary + 2 backups, since ad SDKs sometimes suppress the 1st/2nd click as warmup |
+| `realistic_click`    | Boolean | `false` | Higher click fidelity: denser mouse approach (15 steps), ±1px hand-tremor micro-moves during the press, and ±1.5px mouseup drift (so mousedown≠mouseup coords) — for sites that score click realism. Costs ~80–120ms/click |
 | `interact_typing`    | Boolean | `false` | Enable typing simulation |
 | `interact_intensity` | String | `"medium"` | Interaction simulation intensity: "low", "medium", "high" |
 | `cursor_mode`        | `"ghost"` | - | Use ghost-cursor Bezier mouse movements (requires `npm i ghost-cursor`) |
@@ -295,6 +299,21 @@ When a page redirects to a new domain, first-party/third-party detection is base
 | `ignore_similar_threshold` | Integer | - | Override global similarity threshold for this site |
 | `ignore_similar_ignored_domains` | Boolean | - | Override global `ignore_similar_ignored_domains` for this site |
 
+### Popup Capture Options
+
+Capture (and optionally drive) the popup/popunder windows that ad and redirect
+scripts open, so domains reachable only via a popup chain still match `filterRegex`.
+The same `filterRegex` applies to the whole chain — it must contain every pattern
+you expect along it. Popup capture only fires when the main page is actually
+clicking, so set `interact: true` **and** `interact_clicks: true` as well.
+
+| Field                | Values | Default | Description |
+|:---------------------|:-------|:-------:|:------------|
+| `capture_popups`     | Boolean | `false` | Capture popup windows opened during the scan and evaluate their landing URL + in-popup requests against `filterRegex`/`dig`/`whois` (requires `interact` + `interact_clicks` to fire user-gesture clicks) |
+| `interact_popups`    | Boolean | `false` | Mouse-click inside captured popups (3 content-zone clicks) so the chain cascades to its next redirect/ad. Requires `capture_popups`. Clicks popups up to `capture_popups_max_depth − 1` (the deepest captured popup is observed, not clicked) |
+| `capture_popups_max_depth` | Integer | `4` | Max popup-chain depth to capture (`site → p1 → p2 → p3 → destination`). Each extra level multiplies popups + time |
+| `capture_popups_window_ms` | Integer | `5000` | Per-popup capture window (ms) before the popup is auto-closed |
+
 ### VPN Options
 
 Route traffic through a VPN for specific sites. Requires `sudo` privileges. The VPN connection is established before scanning and torn down after the site completes.
@@ -596,8 +615,11 @@ node nwss.js --max-concurrent 12 --cleanup-interval 300 -o rules.txt
 {
   "url": "https://anti-bot-site.com",
   "interact": true,
+  "interact_clicks": true,
   "cursor_mode": "ghost",
-  "ghost_cursor_duration": 3000,
+  "realistic_click": true,
+  "interact_click_count": 3,
+  "ghost_cursor_duration": 5000,
   "ghost_cursor_speed": 1.2,
   "fingerprint_protection": "random",
   "filterRegex": "tracking|analytics",
@@ -610,6 +632,12 @@ Or enable globally via CLI:
 node nwss.js --ghost-cursor --debug -o rules.txt
 ```
 
+**Ghost-cursor clicks.** The cursor moves with `cursor_mode: "ghost"`, but it only *clicks* when both `interact: true` **and** `interact_clicks: true` are set (same rule as the built-in path). Click behavior:
+
+- `realistic_click: true` — each press adds hand-tremor during the hold and a mouseup drift, so `mousedown` ≠ `mouseup` coordinates (the press is routed through the same `humanClick` the built-in content clicks use).
+- `interact_click_count` — number of clicks per load (default `3`, capped at `20`). The default of 3 matters because some ad SDKs swallow the 1st/2nd click as warmup.
+- **Duration vs. clicks:** realistic clicks take ~600–700ms each, and the bezier movement loop reserves up to **half** of `ghost_cursor_duration` for them. So the default `ghost_cursor_duration: 2000` only fits **~1 click** — raise it to roughly `interact_click_count × 700 + movement` (e.g. `5000`–`8000`) to fit all of them.
+
 > **Note:** ghost-cursor is an optional dependency. Install with `npm install ghost-cursor`. If not installed, the scanner falls back to the built-in mouse simulation automatically.
 
 #### E-commerce Site Scanning

package/eslint.config.mjs

@@ -2,5 +2,17 @@ import globals from "globals";
 import { defineConfig } from "eslint/config";
 
 export default defineConfig([
-  { files: ["**/*.{js,mjs,cjs}"], languageOptions: { globals: globals.browser } },
+  {
+    files: ["**/*.{js,mjs,cjs}"],
+    // Node globals (require/module/process/Buffer/...) plus browser globals
+    // (document/window/navigator) — the latter are referenced inside
+    // page.evaluate() callbacks that eslint parses as part of the file.
+    languageOptions: { globals: { ...globals.node, ...globals.browser } },
+    // Catch undefined-variable references statically. node --check only
+    // validates syntax, so an orphaned identifier (e.g. a const that was
+    // removed while a usage remained) passes parsing but throws
+    // ReferenceError at runtime only when that branch executes. no-undef
+    // turns that whole class into a build-time failure.
+    rules: { "no-undef": "error" },
+  },
 ]);

package/lib/browserhealth.js

@@ -7,12 +7,11 @@ const { formatLogMessage, messageColors } = require('./colorize');
 const IS_PAGE_FROM_PREVIOUS_SCAN_TAG = messageColors.processing('[isPageFromPreviousScan]');
 const REALTIME_CLEANUP_TAG = messageColors.processing('[realtime_cleanup]');
 const GROUP_WINDOW_CLEANUP_TAG = messageColors.processing('[group_window_cleanup]');
-const { execSync, execFile } = require('child_process');
+const { execFile } = require('child_process');
 
 // Window cleanup delay constant
 const WINDOW_CLEANUP_DELAY_MS = 15000;
 // window_clean REALTIME
-const REALTIME_CLEANUP_BUFFER_MS = 25000; // Additional buffer time after site delay (increased for Cloudflare)
 const REALTIME_CLEANUP_THRESHOLD = 12; // Default number of pages to keep
 const REALTIME_CLEANUP_MIN_PAGES = 6; // Minimum pages before cleanup kicks in
 
@@ -380,7 +379,30 @@ async function performRealtimeWindowCleanup(browserInstance, threshold = REALTIM
 
     // Use the provided total delay (already includes appropriate buffer)
     const cleanupDelay = totalDelay;
-    
+
+    // Pre-wait short-circuit. The only pages this pass can ever close are popups
+    // (untracked) and idle pages — active main pages are protected by
+    // isPageSafeToClose. When concurrency exceeds the threshold the page count is
+    // dominated by active main pages, so without this we'd wait the full
+    // cleanupDelay and then close nothing (e.g. max_concurrent 30 vs threshold 8
+    // = a ~36s no-op on every task). If nothing is even a candidate, skip the
+    // wait. A main task that finishes during the skipped wait closes its OWN page,
+    // so realtime cleanup never needed to wait for it.
+    const hasCloseCandidate = quickPages.some(p => {
+      if (p.isClosed()) return false;
+      const usage = pageUsageTracker.get(p);
+      return !usage || !usage.isProcessing; // untracked popup, or a tracked-idle page
+    });
+    if (!hasCloseCandidate) {
+      if (forceDebug) {
+        console.log(formatLogMessage('debug', `${REALTIME_CLEANUP_TAG} ${quickPages.length} pages but all actively processing — skipping ${cleanupDelay}ms wait (nothing closeable)`));
+      }
+      result.success = true;
+      result.totalPages = quickPages.length;
+      result.reason = 'all_active';
+      return result;
+    }
+
     if (forceDebug) {
       console.log(formatLogMessage('debug', `${REALTIME_CLEANUP_TAG} Waiting ${cleanupDelay}ms before cleanup (threshold: ${threshold})`));
     }
@@ -607,16 +629,6 @@ function untrackPage(page) {
   pageUsageTracker.delete(page);
 }
 
-/**
- * No-op since the trackers were migrated to WeakMap — GC reclaims dead-page
- * entries automatically when Puppeteer drops its internal references. Kept
- * exported so the ~7 callers in nwss.js continue to compile; safe to delete
- * entirely once those callsites are scrubbed.
- */
-function purgeStaleTrackers() {
-  // intentionally empty
-}
-
 /**
  * Quick browser responsiveness test for use during page setup
  * Designed to catch browser degradation between operations
@@ -637,82 +649,6 @@ async function isQuicklyResponsive(browserInstance, timeout = 3000) {
   }
 }
 
-/**
- * Tests if browser can handle network operations (like Network.enable)
- * Creates a test page and attempts basic network setup
- * @param {import('puppeteer').Browser} browserInstance - Puppeteer browser instance
- * @param {number} timeout - Timeout in milliseconds (default: 10000)
- * @returns {Promise<object>} Network capability test result
- */
-async function testNetworkCapability(browserInstance, timeout = 10000) {
-  const result = {
-    capable: false,
-    error: null,
-    responseTime: 0
-  };
-
-  const startTime = Date.now();
-  let testPage = null;
-  // Hoisted so the catch can attach an orphan-close chain. Promise.race
-  // cannot cancel browser.newPage() — if the race times out, the underlying
-  // call may still resolve to a real Page tab nothing references. Same
-  // pattern as cdp.js (commit 0772ccd) and clear_sitedata.js (commit 780b443).
-  let testPagePromise = null;
-
-  try {
-    // Create test page
-    testPagePromise = browserInstance.newPage();
-    testPage = await raceWithTimeout(
-      testPagePromise,
-      timeout,
-      'Test page creation timeout'
-    );
-
-    // Test network operations (the critical operation that's failing)
-    await raceWithTimeout(
-      testPage.setRequestInterception(true),
-      timeout,
-      'Network.enable test timeout'
-    );
-
-    // Turn off interception. Symmetric to the enable above — Network.disable
-    // can hang for the same CDP reasons, so it needs the same watchdog.
-    await raceWithTimeout(
-      testPage.setRequestInterception(false),
-      timeout,
-      'Network.disable test timeout'
-    );
-    result.capable = true;
-    result.responseTime = Date.now() - startTime;
-
-  } catch (error) {
-    // Orphan cleanup: if testPage is null but newPage() was started, the
-    // race timed out before assignment. Close the orphan when it arrives.
-    if (!testPage && testPagePromise) {
-      testPagePromise.then(p => p.close().catch(() => {})).catch(() => {});
-    }
-    result.error = error.message;
-    result.responseTime = Date.now() - startTime;
-
-    // Classify the error type
-    if (error.message.includes('Network.enable') ||
-        error.message.includes('timed out') ||
-        error.message.includes('Protocol error')) {
-      result.error = `Network capability test failed: ${error.message}`;
-    }
-  } finally {
-    if (testPage && !testPage.isClosed()) {
-      try {
-        await testPage.close();
-      } catch (closeErr) {
-        /* ignore cleanup errors */
-      }
-    }
-  }
-
-  return result;
-}
-
 /**
  * Checks if browser instance is still responsive
  * @param {import('puppeteer').Browser} browserInstance - Puppeteer browser instance
@@ -758,8 +694,8 @@ async function checkBrowserHealth(browserInstance, timeout = 8000) {
 
     // Test 4: Create a single test page to verify both browser functionality AND network capability
     let testPage = null;
-    // Same orphan-cleanup pattern as testNetworkCapability above + cdp.js +
-    // clear_sitedata.js. Promise.race can't cancel newPage() — if the race
+    // Same orphan-cleanup pattern as cdp.js + clear_sitedata.js.
+    // Promise.race can't cancel newPage() — if the race
     // times out the underlying call may still produce a Page tab nothing
     // references → leaked tab.
     let testPagePromise = null;
@@ -1282,7 +1218,6 @@ module.exports = {
   performGroupWindowCleanup,
   performRealtimeWindowCleanup,
   trackPageForRealtime,
-  testNetworkCapability,
   isQuicklyResponsive,
   performHealthAssessment,
   monitorBrowserHealth,
@@ -1290,6 +1225,5 @@ module.exports = {
   isCriticalProtocolError,
   updatePageUsage,
   untrackPage,
-  cleanupPageBeforeReload,
-  purgeStaleTrackers
+  cleanupPageBeforeReload
 };

package/lib/dns.js

@@ -0,0 +1,238 @@
+/**
+ * DNS pre-check resolver with multi-nameserver rotation.
+ *
+ * Owns nameserver selection and robust resolution for the scan's DNS
+ * pre-check. The default global resolver leads EVERY query with the FIRST
+ * nameserver in /etc/resolv.conf, so under scan concurrency one server
+ * (typically the ISP resolver) takes the whole c-ares burst and starts
+ * answering REFUSED while the other configured servers (e.g. 8.8.8.8/8.8.4.4)
+ * sit idle. This module builds one Resolver per nameserver — each leading with
+ * a different server, the rest kept as failover order — and round-robins them
+ * per resolve attempt so the lead spreads across all servers (and across the
+ * retry). A `--dns` override pins/rotates an explicit list instead of
+ * resolv.conf.
+ *
+ * Scope: this affects the pre-check resolver only. Chrome's navigation DNS
+ * (OS resolver) and nettools' dig/whois are separate paths and unaffected.
+ */
+const net = require('node:net');
+const dnsPromises = require('node:dns/promises');
+const { getServers: getSystemDnsServers } = require('node:dns');
+const { Resolver: DnsPromiseResolver } = require('node:dns/promises');
+const { formatLogMessage } = require('./colorize');
+
+// c-ares codes that mean "resolver problem" (retry-worthy / fail-open), not
+// "the host does not exist".
+const DNS_TRANSIENT_ERRORS = new Set(['ETIMEOUT', 'ESERVFAIL', 'EREFUSED', 'ECONNREFUSED']);
+
+/**
+ * True only for a definitive "host does not exist / has no address" answer —
+ * the only case that justifies skipping a URL in the pre-check. Everything
+ * else (EREFUSED, ESERVFAIL, ETIMEOUT, ECONNREFUSED, timeout) is a resolver
+ * problem the caller should fail open on.
+ * @param {string} code
+ * @returns {boolean}
+ */
+function isNonExistenceError(code) {
+  return code === 'ENOTFOUND' || code === 'ENODATA';
+}
+
+// Accept a bare IPv4/IPv6 address, or an address with a port in the exact form
+// Resolver.setServers() understands: `ipv4:port` or `[ipv6]:port`.
+function isResolverSpec(s) {
+  if (net.isIP(s)) return true;
+  const bracketed = s.match(/^\[([0-9a-fA-F:]+)\](?::\d{1,5})?$/);
+  if (bracketed) return net.isIP(bracketed[1]) === 6;
+  const v4port = s.match(/^(\d{1,3}(?:\.\d{1,3}){3}):\d{1,5}$/);
+  if (v4port) return net.isIP(v4port[1]) === 4;
+  return false;
+}
+
+/**
+ * Parse + validate a `--dns` / config value into a clean, de-duplicated server
+ * list. Accepts a comma-separated string or an array. Each entry may be a bare
+ * IPv4/IPv6 address or an address with a port (`8.8.8.8:5353`,
+ * `[2001:db8::1]:5353`) — the form setServers() accepts. Invalid entries are
+ * warned and dropped; duplicates are collapsed so the rotation stays even.
+ * @param {string|string[]|undefined} raw
+ * @returns {string[]} validated server specs (possibly empty)
+ */
+function parseDnsServers(raw) {
+  if (!raw) return [];
+  const parts = (Array.isArray(raw) ? raw : String(raw).split(','))
+    .map(s => String(s).trim())
+    .filter(Boolean);
+  const valid = [];
+  const seen = new Set();
+  for (const p of parts) {
+    if (!isResolverSpec(p)) {
+      console.warn(`⚠ --dns: ignoring invalid server "${p}" (expected IPv4/IPv6, optionally with :port)`);
+      continue;
+    }
+    if (!seen.has(p)) { seen.add(p); valid.push(p); }
+  }
+  return valid;
+}
+
+/**
+ * Build a rotating pre-check resolver.
+ * @param {object} [opts]
+ * @param {string[]} [opts.servers] - explicit servers (from --dns). When empty,
+ *   the system resolv.conf servers are used.
+ * @param {boolean} [opts.forceDebug] - emit a debug line on the retry path.
+ * @returns {{ resolveHost: (hostname:string, timeoutMs:number)=>Promise<void>,
+ *   servers: string[], rotates: boolean, pinned: boolean }}
+ *   resolveHost resolves on success and rejects with the final error
+ *   (err.code intact) on failure.
+ */
+function createRotatingResolver(opts = {}) {
+  const forceDebug = !!opts.forceDebug;
+  const override = Array.isArray(opts.servers) && opts.servers.length > 0 ? opts.servers : null;
+
+  let systemServers = [];
+  try { systemServers = getSystemDnsServers(); } catch { systemServers = []; }
+  const servers = override || systemServers;
+
+  // Pin/rotate an explicit --dns list (even a single server — never fall back
+  // to the OS resolver in that case). For resolv.conf, only build a pool when
+  // there is more than one server to rotate; otherwise use the global API
+  // (which already reads resolv.conf).
+  const shouldPool = override ? servers.length >= 1 : servers.length > 1;
+  let pool = null;
+  if (shouldPool) {
+    pool = servers.map((_, i) => {
+      const r = new DnsPromiseResolver();
+      // setServers accepts exactly what we hold here: getServers()'s own output
+      // (system path) or net-validated specs incl. ip:port (override path).
+      // Keep the resolver's default servers if an entry is somehow rejected.
+      try { r.setServers([...servers.slice(i), ...servers.slice(0, i)]); } catch { /* keep default */ }
+      return r;
+    });
+  }
+
+  let cursor = 0;
+  // Resolver for the next attempt: rotated when a pool exists, else the global
+  // promises API. `cursor++` is a synchronous single-threaded increment, so even
+  // under heavy concurrency every caller gets a distinct slot and the lead
+  // distribution stays exactly even (no locking needed).
+  const nextResolver = () => (pool ? pool[cursor++ % pool.length] : dnsPromises);
+
+  // One resolution attempt: rotate the lead server, resolve4 first, and on
+  // no-IPv4 (ENODATA/ENOTFOUND) fall back to resolve6 so IPv6-only hosts aren't
+  // wrongly skipped. Any OTHER code propagates unchanged so the caller sees the
+  // real resolver error. A timeout is kept as a safety net — with c-ares off
+  // the libuv threadpool it should rarely fire.
+  async function attempt(hostname, timeoutMs) {
+    const resolver = nextResolver();
+    let timer;
+    try {
+      const timeoutP = new Promise((_, reject) => {
+        timer = setTimeout(() => reject(new Error('DNS timeout')), timeoutMs);
+      });
+      const chain = resolver.resolve4(hostname).catch(err => {
+        if (err && (err.code === 'ENODATA' || err.code === 'ENOTFOUND')) {
+          return resolver.resolve6(hostname);
+        }
+        throw err;
+      });
+      await Promise.race([chain, timeoutP]);
+    } finally {
+      if (timer) clearTimeout(timer);
+    }
+  }
+
+  /**
+   * Resolve a hostname, rotating the lead server per attempt and retrying once
+   * on a transient/resolver error (so the retry leads with the next server —
+   * if one REFUSES, the retry hits another).
+   */
+  async function resolveHost(hostname, timeoutMs) {
+    try {
+      await attempt(hostname, timeoutMs);
+    } catch (firstErr) {
+      const code = firstErr && firstErr.code;
+      if (DNS_TRANSIENT_ERRORS.has(code) || (firstErr && firstErr.message === 'DNS timeout')) {
+        if (forceDebug) console.log(formatLogMessage('debug', `DNS pre-check transient (${code || 'timeout'}) for ${hostname}, retrying once`));
+        await attempt(hostname, timeoutMs);
+      } else {
+        throw firstErr;
+      }
+    }
+  }
+
+  return { resolveHost, servers, rotates: !!pool, pinned: !!override };
+}
+
+/**
+ * Circuit breaker for the DNS pre-check. During a resolver-refusal storm the
+ * pre-check is worthless (every host fails open and proceeds anyway) and
+ * actively harmful (it piles ~2× the queries — with the retry — onto an
+ * already-refusing resolver). This trips when resolver errors dominate a recent
+ * window of attempts and suspends pre-checking for a cooldown so the resolver
+ * gets breathing room; sites still load (a suspended pre-check just proceeds to
+ * navigation, exactly like a single fail-open). NXDOMAIN and success count as
+ * HEALTHY (the resolver answered) — only resolver errors (EREFUSED / ESERVFAIL
+ * / ETIMEOUT / ECONNREFUSED / timeout) count against it.
+ *
+ * @param {object} [opts]
+ * @param {number} [opts.window=20]        attempts kept in the rolling window
+ * @param {number} [opts.threshold=10]     resolver-errors in the window to trip
+ * @param {number} [opts.cooldownMs=30000] how long to stay suspended once tripped
+ * @param {boolean} [opts.forceDebug]
+ * @param {function} [opts.now]            clock injection (tests); defaults to Date.now
+ * @returns {{ record:(isResolverError:boolean)=>void, isTripped:()=>boolean,
+ *   stats:()=>{tripped:boolean,errorCount:number,windowFill:number,trips:number} }}
+ */
+function createDnsCircuitBreaker(opts = {}) {
+  const windowSize = opts.window || 20;
+  const threshold = opts.threshold || 10;
+  const cooldownMs = opts.cooldownMs != null ? opts.cooldownMs : 30000;
+  const forceDebug = !!opts.forceDebug;
+  const now = opts.now || Date.now;
+
+  const recent = [];   // booleans, true = resolver error
+  let errorCount = 0;
+  let openUntil = 0;   // suspended while now() < openUntil
+  let trips = 0;
+
+  // Feed one resolve outcome. Only ever called while closed (a suspended
+  // pre-check skips the resolve, so no outcome is produced).
+  function record(isResolverError) {
+    recent.push(!!isResolverError);
+    if (isResolverError) errorCount++;
+    if (recent.length > windowSize && recent.shift()) errorCount--;
+
+    if (now() >= openUntil && errorCount >= threshold) {
+      openUntil = now() + cooldownMs;
+      trips++;
+      console.log(formatLogMessage('warn', `[dns-precheck] resolver errors ${errorCount}/${recent.length} — suspending DNS pre-check ${Math.round(cooldownMs / 1000)}s (sites still load; backing off the resolver)`));
+    }
+  }
+
+  // True while suspended. On the first call after the cooldown elapses, resume
+  // with a clean window so the storm is re-measured fresh rather than re-tripping
+  // on stale errors.
+  function isTripped() {
+    if (now() < openUntil) return true;
+    if (openUntil !== 0) {
+      openUntil = 0;
+      recent.length = 0;
+      errorCount = 0;
+      if (forceDebug) console.log(formatLogMessage('debug', '[dns-precheck] cooldown elapsed — resuming DNS pre-check'));
+    }
+    return false;
+  }
+
+  return {
+    record,
+    isTripped,
+    stats: () => ({ tripped: now() < openUntil, errorCount, windowFill: recent.length, trips }),
+  };
+}
+
+module.exports = {
+  createRotatingResolver,
+  createDnsCircuitBreaker,
+  parseDnsServers,
+  isNonExistenceError,
+};