crawlforge-mcp-server 6.7.0 → 6.8.0

package/CLAUDE.md

@@ -141,7 +141,7 @@ npm run docker:prod         # Run production container
 - **ActionExecutor**: Browser automation engine (Playwright-based)
 - **ResearchOrchestrator**: Multi-stage research with query expansion and synthesis
 - **AgentOrchestrator**: Powers the `agent` tool — NL prompt → autonomous PLAN→GATHER→ACT→DECIDE→SHAPE loop with three orchestrator-enforced hard stops (maxSteps≤10, maxUrls≤20, wall-clock) never delegated to the LLM; degraded no-LLM-key path (D2, v4.6.0)
-- **StealthBrowserManager**: Stealth mode scraping with anti-detection; Camoufox (Firefox) engine added in v4.0.0
+- **StealthBrowserManager**: Stealth mode scraping with anti-detection; Camoufox (Firefox) engine added in v4.0.0. Engine selection goes through the one resolver `resolveStealthEngine(requested)` → `{engine, fallbackWarning}`; `'auto'` (the default for `scrape.escalate_engine` and `stealth_mode.engine` since the 2026-09 stealth review Phase 2) prefers Camoufox and falls back to Chromium with a caller-visible warning when its binary is absent. `'playwright'` is the tool layer's public name for Chromium. Camoufox personas are pinned to the installed binary's Firefox major so the UA's `rv:` and `Firefox/` tokens agree — see `docs/STEALTH_REVIEW_2026-09.md`
 - **LocalizationManager**: Multi-language content and localization
 - **ChangeTracker**: Content change tracking over time
 - **SnapshotManager**: Website snapshots and version history
@@ -221,8 +221,24 @@ MAX_PAGES_PER_CRAWL=100
 RESPECT_ROBOTS_TXT=true
 ROBOTS_CACHE_TTL_MS=3600000          # how long a parsed robots.txt stays good for
 CRAWLFORGE_BLOCKED_HOSTS=            # comma-separated; extends the permanent opt-out blocklist
+CRAWLFORGE_STEALTH_ENGINE=           # which engine `auto` resolves to on THIS deployment:
+                                     # chromium|playwright pins it to Chromium; unset or anything
+                                     # else keeps the default preference for Camoufox. A caller
+                                     # naming an engine always wins. There is no correct global
+                                     # value — which engine clears a wall depends on the exit IP
+                                     # (residential favours Camoufox, the hosted datacenter IP
+                                     # favours Chromium, measured twice).
+CRAWLFORGE_STEALTH_PROXIES=          # comma-separated proxy URLs used by the escalation stage,
+                                     # stealth_mode, browser_session, scrape_with_actions and the
+                                     # deep_research fallback when the caller passes none (a proxy
+                                     # passed on the call wins). CrawlForge supplies no proxies, and
+                                     # a datacenter one does not defeat these detectors. Distinct
+                                     # from the PROXY_ROTATION_* family, which belongs to
+                                     # localization. The agent tool does not browse yet (Phase 3).
 ```
 
+Camoufox is an **optional** dependency and its transitive `language-tags@2.1.0` requires Node ≥22. npm silently drops an optional subtree that fails an engine check, so on Node 20 it is simply absent — `'auto'` then resolves to Chromium and says so in `fallbackWarning`. `engines.node` stays at `>=20.16.0` deliberately; bumping it is breaking and is a release decision.
+
 Web Bot Auth (optional, off by default — requests go out unsigned when unset):
 
 ```bash
@@ -260,7 +276,7 @@ Key mechanisms for security-conscious future sessions:
 - **endpointGuard** (`src/core/endpointGuard.js`): Hard allow-list of `{crawlforge.dev, www.crawlforge.dev, api.crawlforge.dev}` for the server's own backend calls; HTTPS required; fail-closed. Localhost only in creator mode (v3.0.18).
 - **Action allowlist** (`src/core/ActionExecutor.js`): `scrape_with_actions` accepts only 7 action types: `wait`, `click`, `type`, `press`, `scroll`, `screenshot`, `executeJavaScript`. `executeJavaScript` throws unless `ALLOW_JAVASCRIPT_EXECUTION=true` is set at deploy time (off by default).
 - **Elicitation** (`src/core/ElicitationHelper.js`): User confirmation requested for `deep_research` (>50 URLs), `batch_scrape` (sync, >25 URLs), `crawl_deep` (projected >500 pages), `extract_structured` (schema has >3 required fields, no LLM configured), and credit-low situations. Fail-open if client does not support elicitation.
-- **Browser sandboxing**: Standard pool retains OS sandbox. Stealth Chromium uses `--no-sandbox` + `--disable-web-security` (deliberate fingerprint-spoofing trade-off). Camoufox (Firefox, v4.0.0) is the alternative — see `docs/stealth-engines.md`.
+- **Browser sandboxing**: Standard pool retains OS sandbox. Stealth Chromium uses `--no-sandbox` (deliberate fingerprint-spoofing trade-off); `--disable-web-security` was removed from it in Phase 1 of the 2026-09 stealth review — a page reads it in one line — but the non-stealth render browser (`BrowserProcessor`) still carries both flags. Camoufox (Firefox, v4.0.0) is the alternative — see `docs/stealth-engines.md`.
 
 See `docs/sandboxing-and-approvals.md` for the full reference.
 

package/README.md

@@ -267,6 +267,12 @@ export CRAWLFORGE_TOOL_GROUPS="basic,search,scrape"   # unset = all tools; unkno
 export RESEARCH_STEALTH_ENGINE="auto"      # auto (default) | camoufox | chromium
 export RESEARCH_STEALTH_FALLBACK="true"    # set to "false" to disable entirely
 export RESEARCH_MAX_STEALTH_RETRIES="8"    # cap on stealth retries per research run
+
+# Optional: your own proxies for the stealth paths that have no caller to ask —
+# the scrape escalation stage, the deep_research retry, the agent, browser_session.
+# Comma-separated; a proxy passed on a call always wins. CrawlForge supplies none.
+export CRAWLFORGE_STEALTH_PROXIES="http://user:pass@gw.provider.net:8080"
+# Unrelated to the PROXY_ROTATION_* variables, which belong to the localization tool.
 ```
 
 ### MCP Spec Features
@@ -297,6 +303,18 @@ ollama pull llama3.2
 #    extract_with_llm({ url: "https://example.com", prompt: "…", model: "llama3.2" })
 ```
 
+### Stealth engines and proxies
+
+Every stealth entry point — `scrape` with `escalate: true`, `stealth_mode`, `browser_session`, `scrape_with_actions` and the `deep_research` retry — now defaults to **`auto`**: use Camoufox (Firefox anti-detect) when its binary is installed, fall back to Chromium when it is not, with the reason in the result's `warnings[]`. Every result names the engine that actually ran. Naming an engine explicitly behaves as it always has — `camoufox` fails rather than falling back, and `chromium` and `playwright` are the same engine (on `scrape`'s `escalate_engine`, spell it `playwright`).
+
+On `browser_session` and `scrape_with_actions` the engine applies only with `stealth: true` — their ordinary path is the standard Chromium pool.
+
+`auto` is not free. Measured once each on an Apple Silicon Mac on 2026-09-21 (launch plus a context and a blank page): Chromium 148 ms and 253 MB, Camoufox 946 ms and 667 MB — roughly **+0.8 s and +400 MB per stealth call**. Pin `engine: "playwright"` for high-volume work on sites that do not block.
+
+**CrawlForge supplies no proxies**, and a datacenter proxy does not fix a block: Cloudflare scores the IP's ASN and the TLS/HTTP2 handshake before any JavaScript runs, so no browser-side patch compensates for a datacenter address. Bring your own residential exit with `CRAWLFORGE_STEALTH_PROXIES` (comma-separated URLs), which the escalation stage, `stealth_mode`, `browser_session`, `scrape_with_actions` and the `deep_research` retry use when the caller passes none; a proxy passed on the call always wins. (The `agent` tool does not browse yet, so it is not in that list — it joins when Phase 3 gives it an escalation path.) With a proxy, Camoufox derives its timezone, locale and geolocation from the exit IP.
+
+Full detail, measurements and sources: [docs/stealth-engines.md](docs/stealth-engines.md).
+
 ### Stealth extraction for `deep_research` (Camoufox)
 
 `deep_research` automatically retries sources that block the normal fetch path (Reddit, Quora, forums, and Cloudflare/DataDome-protected pages return HTTP 403) through a **real fingerprinted browser**, then re-extracts from the rendered HTML. It's bounded (`RESEARCH_MAX_STEALTH_RETRIES`, default 8, plus a per-page timeout) and lazy — the browser stack only loads when a source is actually blocked.
@@ -307,7 +325,7 @@ Engine selection (`RESEARCH_STEALTH_ENGINE`):
 - **`camoufox`** — force Camoufox.
 - **`chromium`** — force the Chromium stealth engine.
 
-Headless Chromium **cannot** clear modern challenges (Cloudflare Turnstile, DataDome) — **Camoufox can**. In testing it recovered Quora and Trustpilot pages that were otherwise fully blocked. To enable it, install the optional dependency and run its one-time binary fetch:
+Camoufox gets through walls that stop headless Chromium: in a benchmark run on 2026-09-21 from a residential IP it was the only engine to clear Cloudflare Turnstile (indeed.com) and Akamai (harrods.com), both of which blocked Chromium stealth. Neither engine cleared DataDome or an interactive Turnstile, so this is a better fetch path, not a bypass. To enable it, install the optional dependency and run its one-time binary fetch:
 
 ```bash
 # Camoufox is declared as an optional dependency, so a normal install already pulls it.
@@ -320,7 +338,7 @@ npx camoufox fetch
 
 Without the Camoufox binary, `deep_research` silently falls back to Chromium stealth and then to plain fetch — no errors, just lower recovery on heavily-protected sites. Disable the whole fallback with `RESEARCH_STEALTH_FALLBACK=false`.
 
-> **Note:** Hard IP-reputation blocks (e.g. Reddit's edge `403`) resist headless stealth from any IP and require residential/mobile proxies, which CrawlForge does not provide. See [docs/stealth-engines.md](docs/stealth-engines.md) for details.
+> **Note:** Hard IP-reputation blocks (e.g. Reddit's edge `403`) resist headless stealth from any IP and require residential/mobile proxies, which CrawlForge does not provide — point the server at your own with `CRAWLFORGE_STEALTH_PROXIES`. See [docs/stealth-engines.md](docs/stealth-engines.md) for details.
 
 ### Manual Configuration
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "crawlforge-mcp-server",
-  "version": "6.7.0",
+  "version": "6.8.0",
   "mcpName": "io.github.mysleekdesigns/crawlforge-mcp-server",
   "description": "CrawlForge MCP Server - Professional Model Context Protocol server with 31 web scraping, crawling, deep-research, and autonomous-extraction tools. Returns clean Markdown and structured JSON for Claude, Cursor, and any MCP client. Defaults to local Ollama for LLM extraction (no API key needed); OpenAI/Anthropic available as opt-in. Includes a unified multi-format scrape tool, an autonomous agent, pre-built site templates, and Camoufox stealth browsing.",
   "main": "server.js",
@@ -28,6 +28,8 @@
     "docker:build": "docker build -t crawlforge .",
     "docker:dev": "docker-compose up crawlforge-dev",
     "docker:prod": "docker-compose up crawlforge-prod",
+    "bench:stealth": "node scripts/stealth-bench.mjs",
+    "bench:stealth:ci": "node scripts/stealth-bench.mjs --ci",
     "eval:selection": "node scripts/tool-selection-eval.mjs",
     "usage:report": "node scripts/tool-usage-report.mjs"
   },

package/server.js

@@ -29,7 +29,7 @@ import { GenerateLLMsTxtTool } from "./src/tools/llmstxt/generateLLMsTxt.js";
 import { ScrapeTemplateTool } from "./src/tools/templates/ScrapeTemplateTool.js"; // D3.3
 import { UnifiedScrapeTool, SCRAPE_INPUT_SHAPE } from "./src/tools/scrape/unifiedScrape.js"; // D4 D1
 import { AgentTool } from "./src/tools/agent/agent.js"; // D4 D2
-import { StealthBrowserManager } from "./src/core/StealthBrowserManager.js";
+import { StealthBrowserManager, resolveStealthEngine } from "./src/core/StealthBrowserManager.js";
 import { LocalizationManager } from "./src/core/LocalizationManager.js";
 // Stealth scrape: format conversion + the pre-fetch compliance gate (G5/G6/G7)
 import * as cheerio from "cheerio";
@@ -108,7 +108,7 @@ if (configErrors.length > 0 && config.server.nodeEnv === 'production') {
 // Create the server
 const server = new McpServer({
   name: "crawlforge",
-  version: "6.7.0",
+  version: "6.8.0",
   description: "Production-ready MCP server with 31 web scraping, crawling, and content processing tools. Features MCP Resources (crawlforge://), Prompts, Sampling fallback, Elicitation, stealth browsing, stateful browser sessions with element refs, deep research, structured extraction, embedded JavaScript state extraction, real Google SERP rank tracking, Reddit search via community archives, change tracking, local-LLM extraction via Ollama, unified multi-format scrape, and autonomous agent tool.",
   homepage: "https://www.crawlforge.dev",
   icon: "https://www.crawlforge.dev/icon.png",
@@ -249,14 +249,23 @@ const unifiedScrapeTool = new UnifiedScrapeTool({
   // the tool module never imports StealthBrowserManager (that would pull a
   // browser dependency into every unit test that loads `scrape`). Same gate
   // and same browser the stealth_mode tool drives — no new evasion, and the
-  // "playwright" → chromium engine mapping stays here with its sibling.
+  // engine name is resolved here, beside its sibling, by the one resolver
+  // every stealth entry point shares.
   escalateScrape: async ({ url, engine, respectRobots }) => {
     const warnings = await stealthComplianceGate(url, respectRobots);
+    // "auto" prefers camoufox and falls back to chromium when its binary is
+    // missing. Mapping here instead ("camoufox" or else chromium) collapsed
+    // auto to chromium and hid the fallback; the resolver says which engine it
+    // landed on and why, and the caller is told.
+    const resolved = await resolveStealthEngine(engine);
+    if (resolved.fallbackWarning) warnings.push(resolved.fallbackWarning);
     const scraped = await stealthBrowserManager.scrapeWithStealth({
       url,
-      engine: engine === 'camoufox' ? 'camoufox' : 'chromium'
+      engine: resolved.engine
     });
-    return { ...scraped, engine, warnings };
+    // The RESOLVED engine: `scrape` reports it as stealth.engine, where the
+    // requested "auto" would tell the caller nothing about what ran.
+    return { ...scraped, engine: resolved.engine, warnings };
   }
 });
 const agentTool = new AgentTool(); // D4 D2
@@ -913,7 +922,7 @@ registerToolIfEnabled("read_result", {
 
 // Tool: scrape_with_actions
 registerToolIfEnabled("scrape_with_actions", {
-  description: "Use this when you must interact with a page before scraping - login, click buttons, fill forms, scroll, or wait for dynamic content to load - for SPAs, login-gated content, or multi-step flows. Actions: snapshot, wait, click, type, press, scroll, screenshot, executeJavaScript, select (dropdowns), hover, navigate. Start a chain with {type:\"snapshot\"} to list the page's interactive elements with stable refs (@e1, @e2 ...), then target those refs in later actions instead of guessing CSS selectors; navigation invalidates refs, so snapshot again after one. Set browserOptions.stealth:true to run the chain in the stealth browser. robots.txt is respected on every navigation. Screenshots from this tool are stored as crawlforge://screenshot/{actionId} resources. Not for pages that render without interaction (scrape) and not as the first attempt on a blocked site (stealth_mode operation:\"scrape\"). Cost: 5 credits. Example: scrape_with_actions({url: \"https://app.com/dashboard\", actions: [{type:\"snapshot\"},{type:\"type\",selector:\"@e2\",text:\"user@a.com\"},{type:\"click\",selector:\"@e4\"}]})",
+  description: "Use this when you must interact with a page before scraping - login, click buttons, fill forms, scroll, or wait for dynamic content to load - for SPAs, login-gated content, or multi-step flows. Actions: snapshot, wait, click, type, press, scroll, screenshot, executeJavaScript, select (dropdowns), hover, navigate. Start a chain with {type:\"snapshot\"} to list the page's interactive elements with stable refs (@e1, @e2 ...), then target those refs in later actions instead of guessing CSS selectors; navigation invalidates refs, so snapshot again after one. Set browserOptions.stealth:true to run the chain in the stealth browser, and browserOptions.engine to pick its engine (\"auto\" by default - camoufox when it is installed, Chromium otherwise, and the result says which ran). robots.txt is respected on every navigation. Screenshots from this tool are stored as crawlforge://screenshot/{actionId} resources. Not for pages that render without interaction (scrape) and not as the first attempt on a blocked site (stealth_mode operation:\"scrape\"). Cost: 5 credits. Example: scrape_with_actions({url: \"https://app.com/dashboard\", actions: [{type:\"snapshot\"},{type:\"type\",selector:\"@e2\",text:\"user@a.com\"},{type:\"click\",selector:\"@e4\"}]})",
   annotations: { title: "Scrape with Browser Actions", readOnlyHint: true, destructiveHint: false, idempotentHint: false, openWorldHint: true },
   inputSchema: {
     url: z.string().url().describe("The URL to scrape"),
@@ -985,7 +994,8 @@ registerToolIfEnabled("scrape_with_actions", {
       viewportWidth: z.number().min(800).max(1920).default(1280),
       viewportHeight: z.number().min(600).max(1080).default(720),
       timeout: z.number().min(10000).max(120000).default(30000),
-      stealth: z.boolean().default(false).describe("Run the action chain in the stealth browser (randomized fingerprint, WebRTC/canvas spoofing) instead of the standard browser pool. Renders JavaScript; it does not solve challenges.")
+      stealth: z.boolean().default(false).describe("Run the action chain in the stealth browser (randomized fingerprint, WebRTC/canvas spoofing) instead of the standard browser pool. Renders JavaScript; it does not solve challenges."),
+      engine: z.enum(["auto", "chromium", "camoufox", "playwright"]).default("auto").describe("Stealth engine for the chain, with stealth:true. \"auto\" (default) runs camoufox when it is installed and Chromium otherwise; the result's `engine` says which ran. \"camoufox\" is Firefox-based with a higher anti-detect score; \"chromium\" (= \"playwright\") forces Chromium. Refused without stealth:true, where the browser is always Chromium.")
     }).optional().describe("Browser configuration options"),
     extractionOptions: z.object({
       selectors: z.record(z.string()).optional(),
@@ -1028,13 +1038,14 @@ registerToolIfEnabled("scrape_with_actions", {
 
 // Tool: browser_session
 registerToolIfEnabled("browser_session", {
-  description: "Use this to drive a browser across several calls, keeping the page, its cookies and its login in between. The loop is: open a session on a URL, snapshot it to list the interactive elements with stable refs (@e1, @e2 ...), act on those refs, read the content, close. Because the page stays open you can look before each step instead of committing to a whole chain up front, so a wrong selector costs one call rather than all of them. Operations: open (url, stealth, ttl, activity_ttl, viewport), snapshot, act (the same action array as scrape_with_actions), read (formats), screenshot, close, list. Navigation invalidates refs, so snapshot again after one. robots.txt is respected on every navigation, and screenshots are stored as crawlforge://screenshot/{actionId} resources. A session expires 600s after it opens or 300s after its last use, whichever comes first, so close it when you are done. Not for a page that renders without interaction (scrape), and not for an interaction you can write out in advance - that is one scrape_with_actions call for 5. Cost: 3 credits to open; read 2; snapshot, act, screenshot, close and list 1 each. Example: browser_session({operation:\"open\", url:\"https://app.com/login\"}), then browser_session({operation:\"snapshot\", session_id:\"...\"})",
+  description: "Use this to drive a browser across several calls, keeping the page, its cookies and its login in between. The loop is: open a session on a URL, snapshot it to list the interactive elements with stable refs (@e1, @e2 ...), act on those refs, read the content, close. Because the page stays open you can look before each step instead of committing to a whole chain up front, so a wrong selector costs one call rather than all of them. Operations: open (url, stealth, engine, ttl, activity_ttl, viewport), snapshot, act (the same action array as scrape_with_actions), read (formats), screenshot, close, list. Navigation invalidates refs, so snapshot again after one. robots.txt is respected on every navigation, and screenshots are stored as crawlforge://screenshot/{actionId} resources. A session expires 600s after it opens or 300s after its last use, whichever comes first, so close it when you are done. Not for a page that renders without interaction (scrape), and not for an interaction you can write out in advance - that is one scrape_with_actions call for 5. Cost: 3 credits to open; read 2; snapshot, act, screenshot, close and list 1 each. Example: browser_session({operation:\"open\", url:\"https://app.com/login\"}), then browser_session({operation:\"snapshot\", session_id:\"...\"})",
   annotations: { title: "Browser Session", readOnlyHint: false, destructiveHint: false, idempotentHint: false, openWorldHint: true },
   inputSchema: {
     operation: z.enum(["open", "snapshot", "act", "read", "screenshot", "close", "list"]).describe("open a session, observe it, act on it, read it, or close it"),
     session_id: z.string().optional().describe("The id returned by operation:\"open\". Required by every operation except open and list"),
     url: z.string().url().optional().describe("open: the URL to load the session on"),
     stealth: z.boolean().default(false).describe("open: run the session in the stealth browser"),
+    engine: z.enum(["auto", "chromium", "camoufox", "playwright"]).default("auto").describe("open: stealth engine for the session, with stealth:true. \"auto\" (default) runs camoufox when it is installed and Chromium otherwise; every operation echoes the `engine` that actually ran. \"camoufox\" is Firefox-based with a higher anti-detect score; \"chromium\" (= \"playwright\") forces Chromium. Refused without stealth:true, where the browser is always Chromium."),
     ttl: z.number().min(30).max(3600).optional().describe("open: seconds the session may live at most (default 600)"),
     activity_ttl: z.number().min(10).max(3600).optional().describe("open: seconds the session may sit idle (default 300)"),
     viewport: z.object({
@@ -1337,7 +1348,7 @@ function stealthScrapeFormats(formats, scraped) {
 
 // Tool: stealth_mode
 registerToolIfEnabled("stealth_mode", {
-  description: "Use this when a site blocks normal scraping - Cloudflare, Datadome, or other bot-detection systems. Renders in a Playwright browser with randomized fingerprints, human behavior simulation, WebRTC/canvas spoofing. operation:\"scrape\" is the one-shot path: it creates a context, navigates, returns the requested formats and tears down. The create_context -> create_page -> cleanup operations remain for multi-step work. robots.txt is respected on every navigation. Not a first choice: try scrape first and switch here after a 403/429/CAPTCHA/challenge page or an empty shell. Cost: 5 credits per browser operation; configure, enable, disable, get_stats and cleanup cost 1. Example: stealth_mode({operation:\"scrape\", url:\"https://example.com\", formats:[\"markdown\",\"links\"]})",
+  description: "Use this when a site blocks normal scraping - Cloudflare, Datadome, or other bot-detection systems. Renders in a real browser with randomized fingerprints, human behavior simulation, WebRTC/canvas spoofing - Camoufox (Firefox) when it is installed, Chromium otherwise, and the result names the one that ran. operation:\"scrape\" is the one-shot path: it creates a context, navigates, returns the requested formats and tears down. The create_context -> create_page -> cleanup operations remain for multi-step work. robots.txt is respected on every navigation. Not a first choice: try scrape first and switch here after a 403/429/CAPTCHA/challenge page or an empty shell. Cost: 5 credits per browser operation; configure, enable, disable, get_stats and cleanup cost 1. Example: stealth_mode({operation:\"scrape\", url:\"https://example.com\", formats:[\"markdown\",\"links\"]})",
   annotations: { title: "Stealth Mode", readOnlyHint: false, destructiveHint: false, idempotentHint: false, openWorldHint: true },
   inputSchema: {
     operation: z.enum(['scrape', 'configure', 'enable', 'disable', 'create_context', 'create_page', 'get_stats', 'cleanup']).default('configure').describe("Stealth operation to perform"),
@@ -1379,7 +1390,7 @@ registerToolIfEnabled("stealth_mode", {
         hardwareSpoofing: z.boolean().default(true)
       }).optional()
     }).optional().describe("Stealth browser configuration with anti-detection settings"),
-    engine: z.enum(["playwright", "camoufox"]).optional().default("playwright").describe("Browser engine: \"playwright\" (Chromium, default) or \"camoufox\" (Firefox-based, higher anti-detect score — install with npm install camoufox)"),
+    engine: z.enum(["auto", "chromium", "camoufox", "playwright"]).optional().default("auto").describe("Browser engine: \"auto\" (default — camoufox when it is installed, Chromium otherwise, and the result says which), \"camoufox\" (Firefox-based, higher anti-detect score; fails if not installed), or \"chromium\" (\"playwright\" is the same engine under its old name)"),
     contextId: z.string().optional().describe("Browser context ID for page operations"),
     urlToTest: z.string().url().optional().describe("URL to navigate to when creating a page"),
     url: z.string().url().optional().describe("URL to scrape — required for operation:\"scrape\""),
@@ -1399,12 +1410,16 @@ registerToolIfEnabled("stealth_mode", {
         // Gate first: a disallowed URL must never launch a browser.
         const warnings = await stealthComplianceGate(url, respect_robots);
 
+        // "auto" prefers camoufox and falls back to chromium when its binary
+        // is missing; "playwright" is this tool's old public name for chromium.
+        // The resolver owns both, and reports a fallback the caller can see.
+        const resolvedEngine = await resolveStealthEngine(engine);
+        if (resolvedEngine.fallbackWarning) warnings.push(resolvedEngine.fallbackWarning);
+
         const wantsScreenshot = formats.includes('screenshot');
         const scraped = await stealthBrowserManager.scrapeWithStealth({
           url,
-          // "playwright" is this tool's public name for the chromium engine
-          // (the manager and the CLI both call it chromium).
-          engine: engine === 'camoufox' ? 'camoufox' : 'chromium',
+          engine: resolvedEngine.engine,
           wait_for: wait_for || 0,
           screenshot: wantsScreenshot,
           stealthConfig
@@ -1421,6 +1436,7 @@ registerToolIfEnabled("stealth_mode", {
           url: scraped.url,
           title: scraped.title,
           status: verdict.status,
+          engine: resolvedEngine.engine,
           content: stealthScrapeFormats(formats, scraped)
         };
         if (verdict.blocked) result.blocked = verdict.blocked;
@@ -1457,10 +1473,11 @@ registerToolIfEnabled("stealth_mode", {
       case 'create_context': {
         // Forward the tool-level engine the same way operation:"scrape" does —
         // without it, create_context always ran on chromium whatever the caller
-        // asked for ("playwright" is this tool's public name for chromium).
+        // asked for. Resolved, so "auto" reaches the manager as a real engine.
+        const contextEngine = await resolveStealthEngine(engine);
         const contextData = await stealthBrowserManager.createStealthContext({
           ...(stealthConfig || {}),
-          engine: engine === 'camoufox' ? 'camoufox' : 'chromium'
+          engine: contextEngine.engine
         });
         // The full fingerprint is ~4 KB of canvas noise arrays and WebGL
         // extension lists no caller acts on. Summarise by default; verbose:true
@@ -1468,6 +1485,10 @@ registerToolIfEnabled("stealth_mode", {
         result = {
           contextId: contextData.contextId,
           created: true,
+          // Which engine the context is on, and why, when "auto" had to fall
+          // back: every page made from it inherits that engine.
+          engine: contextEngine.engine,
+          ...(contextEngine.fallbackWarning ? { warnings: [contextEngine.fallbackWarning] } : {}),
           fingerprint: verbose
             ? contextData.fingerprint
             : stealthBrowserManager.summarizeFingerprint(contextData.fingerprint)

package/src/constants/config.js

@@ -454,6 +454,28 @@ export function isStealthConfigured() {
   );
 }
 
+// The server operator's own exit IPs for every stealth path, from
+// CRAWLFORGE_STEALTH_PROXIES — a comma-separated list in proxyRotation's URL
+// format (http/https/socks4/socks5, credentials percent-encoded in the URL).
+//
+// Deliberately a function reading process.env rather than a field on `config`
+// above: a value frozen at import time cannot be changed for a single run, and
+// every consumer (the escalation stage, the deep_research fallback, the agent,
+// browser_session) resolves its proxy mid-process.
+//
+// Separate from the PROXY_ROTATION_* family under localization.proxy on
+// purpose. Those answer "which country should this page be fetched from", which
+// is the localization tool's whole job; this one answers "which address does
+// this server go out from", which is about not being the same datacentre IP a
+// wall has already seen. Overloading one on the other would make either
+// setting silently change the other's behaviour.
+export function serverStealthProxies() {
+  return (process.env.CRAWLFORGE_STEALTH_PROXIES || '')
+    .split(',')
+    .map((entry) => entry.trim())
+    .filter(Boolean);
+}
+
 // Get localization configuration
 export function getLocalizationConfig() {
   return config.localization;

package/src/core/ResearchOrchestrator.js

@@ -9,6 +9,7 @@ import { CacheManager } from './cache/CacheManager.js';
 import { Logger } from '../utils/Logger.js';
 import { LLMManager } from './llm/LLMManager.js';
 import { safeFetch, safeGoto } from '../utils/ssrfGuard.js';
+import { serverStealthProxies } from '../constants/config.js';
 import { preflightFetch, browserPreflight } from '../utils/robotsGate.js';
 import { guardFirefoxPageErrors } from '../utils/firefoxPageErrorGuard.js';
 import { noteRetryAfter } from '../utils/hostRateLimiter.js';
@@ -143,6 +144,12 @@ export class ResearchOrchestrator extends EventEmitter {
       stealthEngine = process.env.RESEARCH_STEALTH_ENGINE || 'auto',
       stealthLevel = 'medium',
       stealthTimeoutMs = 20000,
+      // Proxy list for the stealth fallback, in proxyRotation's URL format. The
+      // caller's list always wins; with none, the server-level
+      // CRAWLFORGE_STEALTH_PROXIES applies — this path launches Camoufox itself
+      // rather than through StealthBrowserManager, so it does not inherit that
+      // manager's proxy fallback and has to resolve one here.
+      stealthProxies = serverStealthProxies(),
       searchConfig = {},
       crawlConfig = {},
       extractConfig = {},
@@ -164,6 +171,7 @@ export class ResearchOrchestrator extends EventEmitter {
     this.stealthEngine = stealthEngine;
     this.stealthLevel = stealthLevel;
     this.stealthTimeoutMs = stealthTimeoutMs;
+    this.stealthProxies = Array.isArray(stealthProxies) ? stealthProxies : [];
     this._stealthManager = null;     // Chromium StealthBrowserManager (fallback engine)
     this._stealthBrowser = null;     // Camoufox browser handle (preferred engine)
     this._stealthEngineActive = null;
@@ -952,6 +960,11 @@ export class ResearchOrchestrator extends EventEmitter {
     if (!this._stealthInit) {
       this._stealthInit = (async () => {
         if (this.stealthEngine === 'camoufox' || this.stealthEngine === 'auto') {
+          // Resolved outside the try: a malformed proxy must fail loudly, not be
+          // swallowed by the 'auto' fallback into a proxyless Chromium run —
+          // "the caller believes their traffic is proxied and it is not" is the
+          // failure mode this plumbing exists to prevent.
+          const proxy = this._resolveStealthProxy();
           try {
             const { createRequire } = await import('module');
             const require = createRequire(import.meta.url);
@@ -960,15 +973,35 @@ export class ResearchOrchestrator extends EventEmitter {
             // A page whose own JavaScript throws must not take the process with
             // it — see src/utils/firefoxPageErrorGuard.js.
             guardFirefoxPageErrors();
-            this._stealthBrowser = await camoufox.Camoufox({ headless: true });
+            const launchOptions = { headless: true };
+            if (proxy) {
+              // camoufox takes the URL string and splits the credentials out
+              // itself (getProxyUrl → Playwright's {server,username,password}).
+              launchOptions.proxy = proxy.url;
+              // Same rule as StealthBrowserManager (`geoip: !!proxy`): behind a
+              // proxy, geoip derives timezone, locale and geolocation from the
+              // exit IP, which is the only thing that makes a proxied browser
+              // coherent. Without a proxy it would look up this machine's own
+              // public IP — a network call, and an external service learning
+              // our address, to confirm a location the browser is already in.
+              // No locale is passed either way here, so geoip's derived one
+              // has nothing to contradict.
+              launchOptions.geoip = true;
+            }
+            this._stealthBrowser = await camoufox.Camoufox(launchOptions);
             this._stealthEngineActive = 'camoufox';
-            this.logger.info('Stealth fallback using Camoufox (Firefox) engine');
+            this.logger.info('Stealth fallback using Camoufox (Firefox) engine', {
+              proxy: proxy ? proxy.redacted : null
+            });
             return;
           } catch (e) {
             if (this.stealthEngine === 'camoufox') throw e; // explicit request → surface
             this.logger.warn('Camoufox unavailable, falling back to Chromium stealth', { error: e.message });
           }
         }
+        // No proxy wiring on this branch: StealthBrowserManager resolves its own
+        // (resolveProxy), server-level list included, so passing one here would
+        // be a second path to the same setting.
         const { StealthBrowserManager } = await import('./StealthBrowserManager.js');
         this._stealthManager = new StealthBrowserManager();
         await this._stealthManager.launchStealthBrowser({ level: this.stealthLevel });
@@ -978,6 +1011,43 @@ export class ResearchOrchestrator extends EventEmitter {
     await this._stealthInit;
   }
 
+  /**
+   * The proxy this stealth fallback launches behind: the caller's list when they
+   * configured one, otherwise the server-level CRAWLFORGE_STEALTH_PROXIES.
+   * Returns null when neither is set, and `{ url, redacted }` otherwise — only
+   * the redacted form (scheme + host, no credentials) is ever logged.
+   *
+   * Resolved once per browser, taking the first entry: camoufox fixes its
+   * fingerprint — and with geoip its timezone, locale and geolocation — at
+   * launch, so rotating underneath it would leave it reporting the first
+   * proxy's city behind the second proxy's exit IP, which is a worse signal
+   * than not rotating at all (see StealthBrowserManager._doLaunchStealthBrowser).
+   */
+  _resolveStealthProxy() {
+    const raw = (this.stealthProxies[0] || '').trim();
+    if (!raw) return null;
+
+    // `host:port` with no scheme parses as protocol "host:" and an empty host,
+    // so the bare form gets the http:// every proxy list assumes.
+    const url = /^[a-z][a-z0-9+.-]*:\/\//i.test(raw) ? raw : `http://${raw}`;
+    let parsed;
+    try {
+      parsed = new URL(url);
+    } catch {
+      // Never echo the entry: it carries the password, and camoufox's own
+      // `new URL(proxy)` would quote the whole string in its TypeError.
+      throw new Error('Invalid stealth proxy: expected host:port or scheme://user:pass@host:port');
+    }
+    const scheme = parsed.protocol.replace(':', '').toLowerCase();
+    if (!['http', 'https', 'socks4', 'socks5'].includes(scheme)) {
+      throw new Error(`Invalid stealth proxy scheme "${scheme}": expected http, https, socks4 or socks5`);
+    }
+    if (!parsed.hostname) {
+      throw new Error('Invalid stealth proxy: no host');
+    }
+    return { url, redacted: `${parsed.protocol}//${parsed.host}` };
+  }
+
   /**
    * macOS packaging fix for camoufox-js: it expects properties.json in
    * Camoufox.app/Contents/MacOS/, but the .app bundle ships it under