crawlforge-mcp-server 6.8.0 → 6.10.0

package/CLAUDE.md

@@ -234,7 +234,12 @@ CRAWLFORGE_STEALTH_PROXIES=          # comma-separated proxy URLs used by the es
                                      # passed on the call wins). CrawlForge supplies no proxies, and
                                      # a datacenter one does not defeat these detectors. Distinct
                                      # from the PROXY_ROTATION_* family, which belongs to
-                                     # localization. The agent tool does not browse yet (Phase 3).
+                                     # localization. The agent's automatic stealth retry
+                                     # (stealth review Phase 3) reads it too.
+CRAWLFORGE_CLEARANCE_JAR=            # `off` disables the clearance jar (stealth review Phase 4):
+                                     # cf_clearance/__cf_bm/datadome cookies a stealth render earned
+                                     # are replayed to the next stealth context with the same engine,
+                                     # UA and proxy. Persisted at ~/.crawlforge/stealth-clearance.json.
 ```
 
 Camoufox is an **optional** dependency and its transitive `language-tags@2.1.0` requires Node ≥22. npm silently drops an optional subtree that fails an engine check, so on Node 20 it is simply absent — `'auto'` then resolves to Chromium and says so in `fallbackWarning`. `engines.node` stays at `>=20.16.0` deliberately; bumping it is breaking and is a release decision.

package/README.md

@@ -206,7 +206,7 @@ CrawlForge requires a CrawlForge API key — **every tool is metered and consume
 | `scrape_with_actions` | 5 | Browser automation chains |
 | `generate_llms_txt` | 5 | Generate AI interaction guidelines |
 | `stealth_mode` | 5 | Anti-detection browser management |
-| `agent` | 8 | **Autonomous research/extraction from a natural-language prompt — no URLs required.** Plans, gathers, and shapes an answer under hard safety stops (max steps/URLs/wall-clock enforced by the orchestrator, never the LLM) |
+| `agent` | 8 (+5 per stealth retry that gets the page, max 2) | **Autonomous research/extraction from a natural-language prompt — no URLs required.** Plans, gathers, and shapes an answer under hard safety stops (max steps/URLs/wall-clock enforced by the orchestrator, never the LLM). A page that walls the plain fetch is retried in the stealth browser automatically — URLs you name first — and that evidence is marked `via: "stealth"`; each retry that runs adds 5 |
 | `deep_research` | 10 | Multi-stage research with source verification |
 
 Ten tools (`scrape`, `fetch_url`, `extract_content`, `crawl_deep`, `batch_scrape`, `stealth_mode`, `scrape_with_actions`, `process_document`, `deep_research`, `extract_embedded_state`) accept `max_inline_chars` (default 40,000; env `CRAWLFORGE_MAX_INLINE_CHARS`): a result over it comes back as a `preview` plus a `result_handle` for `read_result`, with the full result kept for 1 hour under `~/.crawlforge/results/` on your own machine — nothing is uploaded.
@@ -273,6 +273,9 @@ export RESEARCH_MAX_STEALTH_RETRIES="8"    # cap on stealth retries per research
 # Comma-separated; a proxy passed on a call always wins. CrawlForge supplies none.
 export CRAWLFORGE_STEALTH_PROXIES="http://user:pass@gw.provider.net:8080"
 # Unrelated to the PROXY_ROTATION_* variables, which belong to the localization tool.
+
+# Optional: turn off the clearance jar (on by default) — see "Stealth engines and proxies"
+export CRAWLFORGE_CLEARANCE_JAR="off"
 ```
 
 ### MCP Spec Features
@@ -311,7 +314,9 @@ On `browser_session` and `scrape_with_actions` the engine applies only with `ste
 
 `auto` is not free. Measured once each on an Apple Silicon Mac on 2026-09-21 (launch plus a context and a blank page): Chromium 148 ms and 253 MB, Camoufox 946 ms and 667 MB — roughly **+0.8 s and +400 MB per stealth call**. Pin `engine: "playwright"` for high-volume work on sites that do not block.
 
-**CrawlForge supplies no proxies**, and a datacenter proxy does not fix a block: Cloudflare scores the IP's ASN and the TLS/HTTP2 handshake before any JavaScript runs, so no browser-side patch compensates for a datacenter address. Bring your own residential exit with `CRAWLFORGE_STEALTH_PROXIES` (comma-separated URLs), which the escalation stage, `stealth_mode`, `browser_session`, `scrape_with_actions` and the `deep_research` retry use when the caller passes none; a proxy passed on the call always wins. (The `agent` tool does not browse yet, so it is not in that list — it joins when Phase 3 gives it an escalation path.) With a proxy, Camoufox derives its timezone, locale and geolocation from the exit IP.
+**CrawlForge supplies no proxies**, and a datacenter proxy does not fix a block: Cloudflare scores the IP's ASN and the TLS/HTTP2 handshake before any JavaScript runs, so no browser-side patch compensates for a datacenter address. Bring your own residential exit with `CRAWLFORGE_STEALTH_PROXIES` (comma-separated URLs), which the escalation stage, `stealth_mode`, `browser_session`, `scrape_with_actions`, the `deep_research` retry and the `agent` tool's automatic stealth retry use when the caller passes none; a proxy passed on the call always wins. With a proxy, Camoufox derives its timezone, locale and geolocation from the exit IP.
+
+**A challenge solved once is not solved again.** When a stealth render gets past a Cloudflare or DataDome wall, the vendor's clearance cookies (`cf_clearance`, `__cf_bm`, `datadome` — nothing else) are kept in `~/.crawlforge/stealth-clearance.json` (mode 0600) and replayed to the next stealth context with the same engine, User-Agent and proxy, for at most the cookie's own lifetime and never more than 24 hours. A render that still meets the wall drops them. Set `CRAWLFORGE_CLEARANCE_JAR=off` to disable.
 
 Full detail, measurements and sources: [docs/stealth-engines.md](docs/stealth-engines.md).
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "crawlforge-mcp-server",
-  "version": "6.8.0",
+  "version": "6.10.0",
   "mcpName": "io.github.mysleekdesigns/crawlforge-mcp-server",
   "description": "CrawlForge MCP Server - Professional Model Context Protocol server with 31 web scraping, crawling, deep-research, and autonomous-extraction tools. Returns clean Markdown and structured JSON for Claude, Cursor, and any MCP client. Defaults to local Ollama for LLM extraction (no API key needed); OpenAI/Anthropic available as opt-in. Includes a unified multi-format scrape tool, an autonomous agent, pre-built site templates, and Camoufox stealth browsing.",
   "main": "server.js",
@@ -157,6 +157,6 @@
     "@hono/node-server": "2.0.12",
     "hono": "^4.12.4",
     "dompurify": "^3.4.0",
-    "adm-zip": "^0.6.0"
+    "adm-zip": "^0.6.1"
   }
 }

package/server.js

@@ -108,7 +108,7 @@ if (configErrors.length > 0 && config.server.nodeEnv === 'production') {
 // Create the server
 const server = new McpServer({
   name: "crawlforge",
-  version: "6.8.0",
+  version: "6.10.0",
   description: "Production-ready MCP server with 31 web scraping, crawling, and content processing tools. Features MCP Resources (crawlforge://), Prompts, Sampling fallback, Elicitation, stealth browsing, stateful browser sessions with element refs, deep research, structured extraction, embedded JavaScript state extraction, real Google SERP rank tracking, Reddit search via community archives, change tracking, local-LLM extraction via Ollama, unified multi-format scrape, and autonomous agent tool.",
   homepage: "https://www.crawlforge.dev",
   icon: "https://www.crawlforge.dev/icon.png",
@@ -243,32 +243,35 @@ const deepResearchTool = new DeepResearchTool();
 const trackChangesTool = new TrackChangesTool();
 const generateLLMsTxtTool = new GenerateLLMsTxtTool();
 const scrapeTemplateTool = new ScrapeTemplateTool(); // D3.3
+// Phase 3: the escalation stage for `scrape`'s escalate:true and the agent's
+// automatic stealth retry (stealth review Phase 3) — one function, so both
+// go through the same gate, engine resolver and server-level proxy list.
+// Injected so the tool modules never import StealthBrowserManager (that would pull a
+// browser dependency into every unit test that loads `scrape`). Same gate
+// and same browser the stealth_mode tool drives — no new evasion, and the
+// engine name is resolved here, beside its sibling, by the one resolver
+// every stealth entry point shares.
+const stealthEscalation = async ({ url, engine, respectRobots }) => {
+  const warnings = await stealthComplianceGate(url, respectRobots);
+  // "auto" prefers camoufox and falls back to chromium when its binary is
+  // missing. Mapping here instead ("camoufox" or else chromium) collapsed
+  // auto to chromium and hid the fallback; the resolver says which engine it
+  // landed on and why, and the caller is told.
+  const resolved = await resolveStealthEngine(engine);
+  if (resolved.fallbackWarning) warnings.push(resolved.fallbackWarning);
+  const scraped = await stealthBrowserManager.scrapeWithStealth({
+    url,
+    engine: resolved.engine
+  });
+  // The RESOLVED engine: `scrape` reports it as stealth.engine, where the
+  // requested "auto" would tell the caller nothing about what ran.
+  return { ...scraped, engine: resolved.engine, warnings };
+};
 const unifiedScrapeTool = new UnifiedScrapeTool({
   actionExecutor: scrapeWithActionsTool.actionExecutor, // D4 D1 (+v4.8 screenshot reuses the shared browser pool)
-  // Phase 3: the escalation stage for `scrape`'s escalate:true, injected so
-  // the tool module never imports StealthBrowserManager (that would pull a
-  // browser dependency into every unit test that loads `scrape`). Same gate
-  // and same browser the stealth_mode tool drives — no new evasion, and the
-  // engine name is resolved here, beside its sibling, by the one resolver
-  // every stealth entry point shares.
-  escalateScrape: async ({ url, engine, respectRobots }) => {
-    const warnings = await stealthComplianceGate(url, respectRobots);
-    // "auto" prefers camoufox and falls back to chromium when its binary is
-    // missing. Mapping here instead ("camoufox" or else chromium) collapsed
-    // auto to chromium and hid the fallback; the resolver says which engine it
-    // landed on and why, and the caller is told.
-    const resolved = await resolveStealthEngine(engine);
-    if (resolved.fallbackWarning) warnings.push(resolved.fallbackWarning);
-    const scraped = await stealthBrowserManager.scrapeWithStealth({
-      url,
-      engine: resolved.engine
-    });
-    // The RESOLVED engine: `scrape` reports it as stealth.engine, where the
-    // requested "auto" would tell the caller nothing about what ran.
-    return { ...scraped, engine: resolved.engine, warnings };
-  }
+  escalateScrape: stealthEscalation
 });
-const agentTool = new AgentTool(); // D4 D2
+const agentTool = new AgentTool({ escalateFetch: stealthEscalation }); // D4 D2 + stealth review Phase 3
 const stealthBrowserManager = new StealthBrowserManager();
 const localizationManager = new LocalizationManager();
 
@@ -1212,7 +1215,7 @@ registerToolIfEnabled("scrape", {
 
 // Tool: agent (D4 D2 — autonomous NL prompt → search/navigate/extract)
 registerToolIfEnabled("agent", {
-  description: "Use this when you need an autonomous agent to research, navigate, and synthesise an answer from the web - no URLs required. The agent plans search queries, fetches and filters relevant pages, and returns a prose or structured answer. model:\"pro\" uses deep multi-source research. Hard limits: maxSteps<=10, maxUrls<=20, 120s wall-clock. Confirms before pro runs. Degraded-but-useful output if no LLM keys/Ollama. Not for a URL you already have (scrape) or a question one search answers (search_web). Cost: 8 credits, scales with maxUrls. Example: agent({prompt:\"What are the top 5 MCP servers in 2025?\", maxUrls:10})",
+  description: "Use this when you need an autonomous agent to research, navigate, and synthesise an answer from the web - no URLs required. The agent plans search queries, fetches and filters relevant pages, and returns a prose or structured answer. model:\"pro\" uses deep multi-source research. Hard limits: maxSteps<=10, maxUrls<=20, 120s wall-clock. Confirms before pro runs. Degraded-but-useful output if no LLM keys/Ollama. Not for a URL you already have (scrape) or a question one search answers (search_web). Pages that block a plain fetch are retried in the stealth browser automatically (at most 2 a run, URLs you name first; evidence marked via:\"stealth\"). Cost: 18 credits at most - 8, plus 5 per stealth retry that gets the page; a retry that is blocked again is free. Example: agent({prompt:\"What are the top 5 MCP servers in 2025?\", maxUrls:10})",
   annotations: { title: "Agent (Autonomous)", readOnlyHint: true, destructiveHint: false, idempotentHint: false, openWorldHint: true },
   inputSchema: {
     prompt: z.string().min(1).max(2000).describe("Natural-language task or question"),

package/src/core/AgentOrchestrator.js

@@ -4,7 +4,8 @@
  * Design: hardcoded 3-action state machine.
  *   PLAN   — one SamplingClient call to decompose prompt into search queries
  *   GATHER — search_web (≤maxUrls results total)
- *   ACT    — fetchAndParse + relevance gate per URL
+ *   ACT    — fetchAndParse + relevance gate per URL; a walled page is retried
+ *            once in the stealth browser (capped, see tools/agent/escalation.js)
  *   DECIDE — loop or answer (step/URL/time hard stops; never LLM-trusted)
  *   SHAPE  — schema→ExtractWithLlm prose→synthesis via SamplingClient
  *
@@ -20,10 +21,21 @@
 import { fetchAndParse } from '../tools/extract/_fetchAndParse.js';
 import { SamplingClient } from './SamplingClient.js';
 import { fenceUntrusted } from '../utils/untrustedContent.js';
+import { stealthDocumentVerdict as documentVerdict } from '../utils/stealthVerdict.js';
+import { aBrowserMightPass } from '../tools/scrape/escalation.js';
+import { AGENT_MAX_ESCALATIONS } from '../tools/agent/escalation.js';
 
 const DEFAULT_WALL_CLOCK_MS = 120_000;
 const DEFAULT_MAX_STEPS = 5;
 const DEFAULT_MAX_URLS = 10;
+// A stealth retry launches a browser and waits out a challenge; one started
+// with less than this left on the wall clock would overrun the hard stop.
+const ESCALATION_MIN_REMAINING_MS = 20_000;
+
+/** A refusal (robots.txt, blocklist) is a decision, not a wall — a browser must not retry it. */
+function isRefusal(err) {
+  return err?.code === 'ROBOTS_DISALLOWED' || err?.code === 'HOST_BLOCKED';
+}
 
 // ── Helpers ───────────────────────────────────────────────────────────────────
 
@@ -168,6 +180,11 @@ export class AgentOrchestrator {
    * @param {object|null} options.mcpServer  - McpServer instance (for SamplingClient)
    * @param {object}      options.searchConfig - passed to SearchWebTool constructor
    * @param {object}      options.llmConfig    - passed to ExtractWithLlm constructor
+   * @param {Function}    [options.escalateFetch] - the stealth stage, injected from
+   *   server.js (same function `scrape`'s escalate:true uses):
+   *   ({ url, engine, respectRobots }) => { html, url, title, text, status, engine, warnings? }
+   *   Absent in unit tests and builds without a browser: ACT then keeps the
+   *   plain-fetch-or-snippet behaviour.
    */
   constructor(options = {}) {
     this._mcpServer = options.mcpServer || null;
@@ -177,6 +194,7 @@ export class AgentOrchestrator {
     this._searchTool = null;
     this._extractWithLlm = null;
     this._researchOrchestrator = null;
+    this._escalateFetch = options.escalateFetch || null;
   }
 
   /** Set MCP server (called by agent.js after construction). */
@@ -237,6 +255,9 @@ export class AgentOrchestrator {
    * @param {number}    [params.maxSteps]  - Max ACT iterations (≤10)
    * @param {number}    [params.maxUrls]   - Max URLs to fetch (≤20)
    * @param {number}    [params.wallClockMs] - Wall-clock budget in ms
+   * @param {object}    [params.usage]     - out-param: `usage.escalations` counts
+   *   stealth retries that ran (the cap), `usage.charged` those that got the
+   *   page (the bill), so the tool can bill them even if run() throws
    * @returns {Promise<object>}
    */
   async run(params) {
@@ -247,8 +268,11 @@ export class AgentOrchestrator {
       model = 'default',
       maxSteps = DEFAULT_MAX_STEPS,
       maxUrls = DEFAULT_MAX_URLS,
-      wallClockMs = DEFAULT_WALL_CLOCK_MS
+      wallClockMs = DEFAULT_WALL_CLOCK_MS,
+      usage = {}
     } = params;
+    usage.escalations = 0;
+    usage.charged = 0;
 
     const startTime = Date.now();
     const deadline = () => (Date.now() - startTime) >= wallClockMs;
@@ -365,6 +389,10 @@ export class AgentOrchestrator {
       if (!urlQueue.includes(url)) urlQueue.push(url);
       if (!priorityUrls.includes(url)) priorityUrls.push(url);
     }
+    // The URLs the caller named — seeds and sites written into the prompt.
+    // Snapshotted before GATHER adds a voted live root to priorityUrls: that
+    // root is the agent's own discovery, not the caller's.
+    const namedUrls = new Set(priorityUrls);
     const searchResults = [];
     /** url -> { title, text }: the engine's excerpt, evidence of last resort for a page that cannot be fetched. */
     const excerpts = new Map();
@@ -441,32 +469,93 @@ export class AgentOrchestrator {
     const evidence = [];
     let urlsFetched = 0;
     let step = 0;
+    const warnings = [];
 
     for (const url of urlQueue) {
       if (urlsFetched >= capUrls || step >= capSteps || deadline()) break;
       urlsFetched++;
 
+      // The plain fetch reads an error document instead of throwing on it, so
+      // a wall served as a 403 can be told apart from a 404 (the same
+      // arrangement `scrape` has).
+      let plain = null;
+      let plainError = null;
+      let verdict = null;
       try {
-        const { textContent, finalUrl } = await fetchAndParse(url, { timeoutMs: 10000 });
-        if (!isRelevant(textContent, prompt)) continue;
-        step++;
-        const sr = searchResults.find(s => s.url === url);
-        evidence.push({
-          url: finalUrl,
-          title: sr ? sr.title : '',
-          text: truncate(textContent),
-          step
-        });
-      } catch {
-        // The page is out of reach (challenge page, 403, timeout) but the
-        // search engine's excerpt of it is not: npmjs.com's "Latest version:
-        // 15.0.0" sat in the snippet while every fetch of the package page was
-        // challenged, and the run ended with no evidence at all (R17,
-        // 2026-09-04). Keep a relevant snippet as evidence, labelled as one.
-        const excerpt = excerpts.get(url);
-        if (excerpt && isRelevant(excerpt.text, prompt)) {
-          evidence.push({ url, title: excerpt.title, text: excerpt.text, snippet: true, step });
+        plain = await fetchAndParse(url, { timeoutMs: 10000, errorDocuments: true });
+        verdict = documentVerdict(
+          { url: plain.finalUrl, title: plain.$('title').first().text().trim(), text: plain.textContent, html: plain.html, status: plain.status },
+          { fetcher: 'a plain fetch', rendered: false, contentReturned: false }
+        );
+      } catch (err) {
+        plainError = err;
+      }
+
+      const ok2xx = plain && plain.status >= 200 && plain.status < 300;
+      if (ok2xx && !verdict.blocked) {
+        if (isRelevant(plain.textContent, prompt)) {
+          step++;
+          const sr = searchResults.find(s => s.url === url);
+          evidence.push({
+            url: plain.finalUrl,
+            title: sr ? sr.title : '',
+            text: truncate(plain.textContent),
+            step
+          });
+          continue;
         }
+        // A real page that is simply off-topic: nothing to retry.
+        if (verdict.success) continue;
+      }
+
+      // The page is out of reach: a wall, a 403/429, an empty shell, a
+      // timeout. A search excerpt may still carry the fact (npmjs.com's
+      // "Latest version: 15.0.0" sat in the snippet while every fetch of the
+      // package page was challenged, R17 2026-09-04).
+      const excerpt = excerpts.get(url);
+      const relevantExcerpt = excerpt && isRelevant(excerpt.text, prompt) ? excerpt : null;
+
+      // Stealth retry (review Phase 3). A URL the caller named always gets it
+      // first; a discovered one only when there is no relevant excerpt to fall
+      // back on. Never for a refusal, a 404 or a 5xx; bounded by its own cap
+      // and by the time left before the wall-clock stop.
+      const mightPass = plainError ? !isRefusal(plainError) : aBrowserMightPass(verdict, plain.status);
+      const canEscalate = this._escalateFetch &&
+        usage.escalations < Math.min(AGENT_MAX_ESCALATIONS, capUrls) &&
+        (wallClockMs - (Date.now() - startTime)) >= ESCALATION_MIN_REMAINING_MS;
+      if (mightPass && canEscalate && (namedUrls.has(url) || !relevantExcerpt)) {
+        usage.escalations++;
+        try {
+          const stealth = await this._escalateFetch({ url: plain?.finalUrl || url, engine: 'auto' });
+          if (Array.isArray(stealth.warnings)) warnings.push(...stealth.warnings);
+          const text = (stealth.text || '').trim();
+          const rendered = documentVerdict(
+            { url: stealth.url || url, title: stealth.title || '', text, html: stealth.html || '', status: stealth.status ?? null },
+            { waitedMs: stealth.gracedMs || 0, fetcher: 'the stealth browser', rendered: true, contentReturned: false }
+          );
+          // Billed only when the browser got past the wall: a retry that met
+          // the wall again, or threw, costs nothing (owner decision).
+          if (rendered.success) usage.charged++;
+          if (rendered.success && isRelevant(text, prompt)) {
+            step++;
+            const sr = searchResults.find(s => s.url === url);
+            evidence.push({
+              url: stealth.url || url,
+              title: stealth.title || (sr ? sr.title : ''),
+              text: truncate(text),
+              via: 'stealth',
+              step
+            });
+            continue;
+          }
+          warnings.push(`stealth retry of ${url} ${rendered.success ? 'returned a page unrelated to the task' : `did not get the page either${rendered.blocked?.vendor ? ` (${rendered.blocked.vendor})` : ''}`}`);
+        } catch (err) {
+          warnings.push(`stealth retry of ${url} failed: ${err.message}`);
+        }
+      }
+
+      if (relevantExcerpt) {
+        evidence.push({ url, title: relevantExcerpt.title, text: relevantExcerpt.text, snippet: true, step });
       }
     }
 
@@ -501,7 +590,10 @@ export class AgentOrchestrator {
         evidence: [],
         answer: null,
         steps: step,
-        urls_fetched: urlsFetched
+        urls_fetched: urlsFetched,
+        stealth_retries: usage.escalations,
+        stealth_retries_charged: usage.charged,
+        ...(warnings.length ? { warnings } : {})
       };
     }
 
@@ -520,11 +612,14 @@ export class AgentOrchestrator {
           answer: result.success ? result.data : null,
           structured: true,
           search_results: searchResults,
-          evidence: evidence.map(e => ({ url: e.url })),
+          evidence: evidence.map(e => (e.via ? { url: e.url, via: e.via } : { url: e.url })),
           degraded: !result.success,
           reason: result.success ? undefined : result.error,
           steps: step,
-          urls_fetched: urlsFetched
+          urls_fetched: urlsFetched,
+          stealth_retries: usage.escalations,
+        stealth_retries_charged: usage.charged,
+          ...(warnings.length ? { warnings } : {})
         };
       } catch (err) {
         // Fall through to prose synthesis
@@ -648,11 +743,14 @@ export class AgentOrchestrator {
       success: true,
       answer,
       search_results: searchResults,
-      evidence: degraded ? evidence : evidence.map(e => (e.snippet ? { url: e.url, snippet: true } : { url: e.url })),
+      evidence: degraded ? evidence : evidence.map(e => (e.snippet ? { url: e.url, snippet: true } : e.via ? { url: e.url, via: e.via } : { url: e.url })),
       degraded,
       reason: degradedReason,
       steps: step,
       urls_fetched: urlsFetched,
+      stealth_retries: usage.escalations,
+        stealth_retries_charged: usage.charged,
+      ...(warnings.length ? { warnings } : {}),
       provenance: {
         checked: !synthesisFailed,
         unverified,

package/src/core/AuthManager.js

@@ -14,6 +14,7 @@ import { logger } from '../utils/Logger.js';
 import { maskSecrets } from '../utils/secretMask.js';
 import { scrapeFormatSurcharge } from '../tools/scrape/formats.js';
 import { scrapeEscalationSurcharge } from '../tools/scrape/escalation.js';
+import { agentEscalationSurcharge } from '../tools/agent/escalation.js';
 import { searchQueryCount } from '../tools/search/batchSearch.js';
 import { redactionSurcharge, REDACT_PII_MODEL_CREDITS } from '../server/redaction.js';
 // Stamped on every usage report so support can tell which client version
@@ -686,6 +687,14 @@ class AuthManager {
       return costs.search_web * searchQueryCount(params?.queries) + redaction;
     }
 
+    // agent retries a walled page in the stealth browser automatically, at
+    // most AGENT_MAX_ESCALATIONS times a run, 5 each. The projection is that
+    // ceiling; the tool reports what actually ran (setActualCost), so a run
+    // with no retry still costs 8. The rule lives in tools/agent/escalation.js.
+    if (tool === 'agent') {
+      return costs.agent + agentEscalationSurcharge(params) + redaction;
+    }
+
     return (costs[tool] ?? 1) + redaction;
   }
 
@@ -761,7 +770,7 @@ class AuthManager {
         const agentUrls = params?.maxUrls || 10;
         const isPro = params?.model === 'pro';
         projected = Math.max(base, base + Math.ceil(agentUrls / 5) + (isPro ? 5 : 0));
-        note = `Lower-bound estimate. Scales with maxUrls (${agentUrls}).${isPro ? ' pro model adds deep-research cost.' : ''} External LLM billed separately.`;
+        note = `Lower-bound estimate. Scales with maxUrls (${agentUrls}).${isPro ? ' pro model adds deep-research cost.' : ' Includes the ceiling for automatic stealth retries of walled pages (5 each, at most 2 a run); the actual charge drops by 5 for each retry that did not run or did not get the page.'} External LLM billed separately.`;
         break;
       }
       default:

package/src/core/ClearanceJar.js

@@ -0,0 +1,229 @@
+/**
+ * ClearanceJar — a solved bot-wall challenge is not solved again for its lifetime.
+ *
+ * Stealth review Phase 4 (docs/STEALTH_REVIEW_2026-09.md). Every stealth
+ * context used to start cold, so a Cloudflare or DataDome challenge the browser
+ * had just passed was served again on the next call to the same site. The
+ * vendors hand out a clearance cookie when a challenge passes; this keeps those
+ * cookies and gives them back to the next stealth context that would present
+ * the same identity.
+ *
+ * Three rules keep it honest:
+ *
+ * - **Only clearance cookies.** The allow-list below is the vendors' own
+ *   clearance and bot-management cookies. Nothing a site sets for a login, a
+ *   cart or a preference is ever kept, so a browser_session one caller logged
+ *   into cannot leak its session into another caller's context — on the hosted
+ *   instance those are different customers.
+ * - **Keyed on who solved it.** cf_clearance is bound to the IP and the User
+ *   Agent that earned it, and a TLS stack that is not the one that earned it is
+ *   scored too. The key is the engine, the exact User-Agent and the proxy
+ *   identity (server + username, never the password), so a clearance earned
+ *   through one exit is never presented from another. A rotating proxy behind
+ *   one fixed username can still change exit underneath a key; the vendor then
+ *   challenges again, the verdict reports a block, and the host is discarded.
+ * - **Bounded.** Expiry is the cookie's own, capped at MAX_TTL_MS; the jar
+ *   holds at most MAX_KEYS identities and MAX_COOKIES_PER_KEY cookies each, and
+ *   the file is rewritten only when its contents change.
+ *
+ * Persisted at ~/.crawlforge/stealth-clearance.json (mode 0600) so an MCP
+ * server that a client restarts per session keeps its clearances.
+ * CRAWLFORGE_CLEARANCE_JAR=off disables the jar entirely.
+ */
+
+import crypto from 'crypto';
+import fs from 'fs';
+import os from 'os';
+import path from 'path';
+
+/**
+ * Cloudflare's clearance (cf_clearance) and bot-management (__cf_bm) cookies,
+ * and DataDome's (datadome). Exact names: a prefix match would sweep in
+ * whatever else a site happens to name alike.
+ */
+export const CLEARANCE_COOKIE_NAMES = new Set(['cf_clearance', '__cf_bm', 'datadome']);
+
+/** No clearance outlives this, whatever expiry the vendor set. */
+export const MAX_TTL_MS = 24 * 60 * 60 * 1000;
+/** A session cookie (no expiry) is kept this long. */
+export const SESSION_TTL_MS = 30 * 60 * 1000;
+export const MAX_KEYS = 32;
+export const MAX_COOKIES_PER_KEY = 200;
+
+const DEFAULT_FILE = path.join(os.homedir(), '.crawlforge', 'stealth-clearance.json');
+
+/** Whether a cookie domain (".example.com" or "example.com") covers `host`. */
+function domainCovers(domain, host) {
+  const d = String(domain || '').replace(/^\./, '').toLowerCase();
+  const h = String(host || '').toLowerCase();
+  return Boolean(d) && (h === d || h.endsWith(`.${d}`));
+}
+
+export class ClearanceJar {
+  /**
+   * @param {{ file?: string|null, now?: () => number }} [options]
+   *   file: where the jar persists; null keeps it in memory only.
+   */
+  constructor({ file = DEFAULT_FILE, now = Date.now } = {}) {
+    this.file = file;
+    this.now = now;
+    /** @type {Map<string, Map<string, object>>} key → (domain|path|name → cookie) */
+    this.entries = null;
+  }
+
+  /**
+   * The identity a clearance is bound to, or null when it cannot be named — an
+   * unknown User-Agent cannot be matched, so nothing is stored or reused for it.
+   *
+   * @param {{ engine?: string|null, userAgent?: string|null, proxy?: {server?: string, username?: string}|null }} identity
+   * @returns {string|null}
+   */
+  static keyFor({ engine, userAgent, proxy } = {}) {
+    if (!engine || !userAgent) return null;
+    const exit = proxy?.server ? `${proxy.server}|${proxy.username || ''}` : 'direct';
+    // Hashed so the file names no proxy host, account or persona in clear.
+    return crypto.createHash('sha256').update(`${engine}\n${userAgent}\n${exit}`).digest('hex');
+  }
+
+  /** Clearance cookies stored for `key` that have not expired, Playwright-shaped. */
+  cookiesFor(key) {
+    if (!key) return [];
+    const bucket = this._load().get(key);
+    if (!bucket) return [];
+    const nowS = this.now() / 1000;
+    const live = [...bucket.values()].filter((c) => c.expires > nowS);
+    if (live.length !== bucket.size) {
+      this._replace(key, live);
+      this._save();
+    }
+    return live;
+  }
+
+  /**
+   * Keep the clearance cookies from `cookies` (the output of
+   * BrowserContext.cookies()) under `key`, replacing older copies.
+   * @returns {number} how many clearance cookies were kept
+   */
+  store(key, cookies = []) {
+    if (!key) return 0;
+    const nowMs = this.now();
+    const kept = [];
+    for (const c of cookies) {
+      if (!c || !CLEARANCE_COOKIE_NAMES.has(c.name) || !c.value || !c.domain) continue;
+      const vendorExpiryMs = c.expires > 0 ? c.expires * 1000 : nowMs + SESSION_TTL_MS;
+      const expiresMs = Math.min(vendorExpiryMs, nowMs + MAX_TTL_MS);
+      if (expiresMs <= nowMs) continue;
+      kept.push({
+        name: c.name,
+        value: c.value,
+        domain: c.domain,
+        path: c.path || '/',
+        expires: Math.floor(expiresMs / 1000),
+        httpOnly: Boolean(c.httpOnly),
+        secure: Boolean(c.secure),
+        sameSite: c.sameSite || 'Lax'
+      });
+    }
+    if (!kept.length) return 0;
+
+    const entries = this._load();
+    const bucket = entries.get(key) || new Map();
+    let changed = false;
+    for (const c of kept) {
+      const id = `${c.domain}|${c.path}|${c.name}`;
+      const old = bucket.get(id);
+      if (old && old.value === c.value && old.expires === c.expires) continue;
+      bucket.delete(id); // re-insert at the end: insertion order is recency
+      bucket.set(id, c);
+      changed = true;
+    }
+    if (!changed) return kept.length;
+    while (bucket.size > MAX_COOKIES_PER_KEY) bucket.delete(bucket.keys().next().value);
+    entries.delete(key);
+    entries.set(key, bucket);
+    while (entries.size > MAX_KEYS) entries.delete(entries.keys().next().value);
+    this._save();
+    return kept.length;
+  }
+
+  /**
+   * Drop every clearance for `host` under `key` — called when a stealth render
+   * still met the wall, so a clearance that no longer works is not replayed.
+   * @returns {number} how many cookies were dropped
+   */
+  discard(key, host) {
+    if (!key || !host) return 0;
+    const bucket = this._load().get(key);
+    if (!bucket) return 0;
+    let dropped = 0;
+    for (const [id, c] of bucket) {
+      if (domainCovers(c.domain, host)) {
+        bucket.delete(id);
+        dropped++;
+      }
+    }
+    if (dropped) {
+      if (!bucket.size) this.entries.delete(key);
+      this._save();
+    }
+    return dropped;
+  }
+
+  _replace(key, cookies) {
+    const entries = this._load();
+    if (!cookies.length) {
+      entries.delete(key);
+      return;
+    }
+    entries.set(key, new Map(cookies.map((c) => [`${c.domain}|${c.path}|${c.name}`, c])));
+  }
+
+  _load() {
+    if (this.entries) return this.entries;
+    this.entries = new Map();
+    if (!this.file) return this.entries;
+    let raw;
+    try {
+      raw = JSON.parse(fs.readFileSync(this.file, 'utf8'));
+    } catch {
+      return this.entries; // absent or unreadable: start empty
+    }
+    const nowS = this.now() / 1000;
+    for (const [key, cookies] of Object.entries(raw?.entries || {})) {
+      if (!Array.isArray(cookies)) continue;
+      const live = cookies.filter((c) => c && CLEARANCE_COOKIE_NAMES.has(c.name) && c.expires > nowS);
+      if (live.length) this._replace(key, live.slice(-MAX_COOKIES_PER_KEY));
+    }
+    while (this.entries.size > MAX_KEYS) this.entries.delete(this.entries.keys().next().value);
+    return this.entries;
+  }
+
+  _save() {
+    if (!this.file) return;
+    const out = { version: 1, entries: {} };
+    for (const [key, bucket] of this.entries) out.entries[key] = [...bucket.values()];
+    try {
+      fs.mkdirSync(path.dirname(this.file), { recursive: true, mode: 0o700 });
+      const tmp = `${this.file}.${process.pid}.tmp`;
+      fs.writeFileSync(tmp, JSON.stringify(out), { mode: 0o600 });
+      fs.renameSync(tmp, this.file);
+    } catch {
+      // Persistence is an optimisation: an unwritable home directory costs the
+      // next process a challenge, not this call its result.
+    }
+  }
+}
+
+let shared;
+
+/**
+ * The process-wide jar, shared by every StealthBrowserManager (server.js,
+ * BrowserProcessor and ResearchOrchestrator each construct their own), or null
+ * when CRAWLFORGE_CLEARANCE_JAR=off.
+ * @returns {ClearanceJar|null}
+ */
+export function sharedClearanceJar() {
+  if (String(process.env.CRAWLFORGE_CLEARANCE_JAR || '').toLowerCase() === 'off') return null;
+  shared ??= new ClearanceJar();
+  return shared;
+}

package/src/core/StealthBrowserManager.js

@@ -18,9 +18,10 @@ import { createRequire } from 'module';
 import HumanBehaviorSimulator from '../utils/HumanBehaviorSimulator.js';
 import { BrowserContextPool } from './BrowserContextPool.js';
 import { safeGoto } from '../utils/ssrfGuard.js';
-import { looksLikeInterstitial } from '../utils/challengeDetection.js';
+import { looksLikeInterstitial, detectChallengePage } from '../utils/challengeDetection.js';
 import { guardFirefoxPageErrors } from '../utils/firefoxPageErrorGuard.js';
 import { serverStealthProxies } from '../constants/config.js';
+import { ClearanceJar, sharedClearanceJar } from './ClearanceJar.js';
 
 // Grace given to a document that rendered no title and no text (see _waitOutEmptyDocument).
 export const EMPTY_DOCUMENT_GRACE_MS = 8000;
@@ -39,6 +40,15 @@ const CAMOUFOX_PROXY = Symbol('crawlforge.camoufoxProxy');
 // browser for the same reason as the proxy: camoufox fixes it at launch.
 const CAMOUFOX_LOCALE = Symbol('crawlforge.camoufoxLocale');
 
+// The User-Agent that camoufox browser presents — the persona pinned at launch,
+// null when camoufox drew its own and we therefore do not know it. The
+// clearance jar keys on it (see ClearanceJar), and camoufox fixes it at launch.
+const CAMOUFOX_USER_AGENT = Symbol('crawlforge.camoufoxUserAgent');
+
+// How long closeContext waits to read a context's cookies before giving up on
+// keeping its clearances: a wedged browser hangs rather than errors.
+const CLEARANCE_HARVEST_TIMEOUT_MS = 2000;
+
 const StealthConfigSchema = z.object({
   level: z.enum(['basic', 'medium', 'advanced']).default('medium'),
   randomizeFingerprint: z.boolean().default(true),
@@ -175,6 +185,12 @@ export class StealthBrowserManager {
     // Chromium that nobody reports reads as a camoufox result that failed.
     this._engineFallbackWarning = null;
     this._maxContexts = parseInt(process.env.MAX_BROWSER_CONTEXTS || '10', 10);
+    // Stealth review Phase 4: clearance cookies a solved challenge earned are
+    // replayed to the next context with the same identity. One process-wide jar
+    // by default (every manager instance shares it); null turns it off — the
+    // benchmark does, since a replayed clearance would measure the jar instead
+    // of the engine.
+    this.clearanceJar = options.clearanceJar !== undefined ? options.clearanceJar : sharedClearanceJar();
     this.contexts = this._createContextPool();
     // D2.2: fingerprints Map is capped at _maxContexts to prevent unbounded growth.
     // Oldest entries are evicted when the cap is exceeded (insertion order via Map).
@@ -525,6 +541,7 @@ export class StealthBrowserManager {
       });
       browser[CAMOUFOX_PROXY] = proxy;
       browser[CAMOUFOX_LOCALE] = locale;
+      browser[CAMOUFOX_USER_AGENT] = adapter.lastUserAgent ?? null;
       this.browser = browser;
       this._launchedEngine = 'camoufox';
       return this.browser;
@@ -727,6 +744,19 @@ export class StealthBrowserManager {
       contextOptions.proxy = proxy;
     }
 
+    // The identity a clearance is bound to: engine, the User-Agent this context
+    // presents, and the exit it goes out through. Read before the camoufox
+    // block below takes the User-Agent off the context options.
+    const jarKey = this.clearanceJar
+      ? ClearanceJar.keyFor({
+          engine: this._launchedEngine,
+          userAgent: this._launchedEngine === 'camoufox'
+            ? this.browser[CAMOUFOX_USER_AGENT]
+            : contextOptions.userAgent,
+          proxy
+        })
+      : null;
+
     if (this._launchedEngine === 'camoufox') {
       // camoufox's Firefox build predates the Browser.setDefaultViewport fields
       // playwright-core 1.62 sends (screenSize, isMobile, ...) and rejects
@@ -787,11 +817,19 @@ export class StealthBrowserManager {
 
     const context = await this.browser.newContext(contextOptions);
     const contextId = this.generateContextId();
-    
+
+    // Replay clearances this identity already earned. addCookies scopes each
+    // to its own domain, so a context bound for another site sends none of
+    // them. Best effort: a context without them just meets the challenge.
+    const clearances = jarKey ? this.clearanceJar.cookiesFor(jarKey) : [];
+    if (clearances.length) {
+      await context.addCookies(clearances).catch(() => {});
+    }
+
     // Apply stealth scripts and configurations
     await this.applyAdvancedStealthConfigurations(context, validatedConfig, fingerprint);
     
-    await this.contexts.set(contextId, { context, fingerprint, config: validatedConfig });
+    await this.contexts.set(contextId, { context, fingerprint, config: validatedConfig, jarKey });
     // D2.2: enforce LRU cap on fingerprints Map
     this._setFingerprint(contextId, fingerprint);
 
@@ -2770,6 +2808,13 @@ export class StealthBrowserManager {
         ? await page.screenshot({ encoding: 'base64', fullPage: false }).catch(() => null)
         : null;
 
+      // Still a wall after everything above: whatever clearance this context
+      // carried for the site did not work, so closeContext drops it instead of
+      // keeping it for the next call.
+      if (detectChallengePage({ title, html, text })) {
+        this._markBlocked(contextId, [url, page.url()]);
+      }
+
       return { success: true, url, title, text, html, screenshot: shot, status, gracedMs, engine: engineUsed, warnings };
     } finally {
       await this.closeContext(contextId).catch(() => {});
@@ -3050,11 +3095,54 @@ export class StealthBrowserManager {
    */
   async closeContext(contextId) {
     if (this.contexts.has(contextId)) {
+      await this._harvestClearances(contextId);
       await this.contexts.dispose(contextId);
       this.fingerprints.delete(contextId);
     }
   }
 
+  /**
+   * Record that a context's render still met a bot wall on these URLs' hosts.
+   * @param {string} contextId
+   * @param {string[]} urls
+   */
+  _markBlocked(contextId, urls) {
+    const contextData = this.contexts.get(contextId);
+    if (!contextData) return;
+    contextData.blockedHosts ??= new Set();
+    for (const url of urls) {
+      try {
+        contextData.blockedHosts.add(new URL(url).hostname);
+      } catch { /* about:blank or a malformed URL names no host */ }
+    }
+  }
+
+  /**
+   * Keep the clearance cookies a context earned, and drop the ones for any host
+   * it was still blocked on. Runs before the context is closed, and never
+   * throws or hangs the close it precedes.
+   * @param {string} contextId
+   */
+  async _harvestClearances(contextId) {
+    const contextData = this.contexts.get(contextId);
+    if (!this.clearanceJar || !contextData?.jarKey) return;
+    let timer;
+    try {
+      const cookies = await Promise.race([
+        contextData.context.cookies(),
+        new Promise((resolve) => { timer = setTimeout(() => resolve([]), CLEARANCE_HARVEST_TIMEOUT_MS); })
+      ]);
+      this.clearanceJar.store(contextData.jarKey, cookies);
+      for (const host of contextData.blockedHosts ?? []) {
+        this.clearanceJar.discard(contextData.jarKey, host);
+      }
+    } catch {
+      // A closed or crashed context has nothing to keep.
+    } finally {
+      clearTimeout(timer);
+    }
+  }
+
   /**
    * D2.2: LRU-capped fingerprint setter.
    * Evicts the oldest entry when the Map exceeds _maxContexts to prevent unbounded growth.
@@ -3451,6 +3539,9 @@ export class CamoufoxAdapter extends BrowserEngine {
     // reaches only `rv:` and about half of all launches announce a Firefox
     // that is not the one running — see _pinnedFingerprint.
     const fingerprint = this._pinnedFingerprint(camoufox);
+    // The UA this launch will present, for the clearance jar's key. Only known
+    // when we pinned the persona; camoufox's own draw is not visible here.
+    this.lastUserAgent = fingerprint?.navigator?.userAgent || null;
     if (fingerprint) {
       options.fingerprint = fingerprint;
       // Switches off exactly two things: camoufox's "you passed your own

package/src/skills/agent-skills/crawlforge-deep-research/SKILL.md

@@ -31,7 +31,7 @@ skill (`scrape` / `extract_content`) instead.
 |------|------|------|
 | A ranked list of result URLs + snippets | `search_web` | 5 |
 | Reddit posts, comments, or a full thread | `reddit_search` | 5 |
-| A direct answer, agent decides what to read | `agent` | 8 (scales) |
+| A direct answer, agent decides what to read | `agent` | 8 (+5 per automatic stealth retry, max 2) |
 | A synthesized, multi-source, cited report | `deep_research` | 10+ (scales) |
 
 A common pipeline: `search_web` to find sources → `batch_scrape` (see

package/src/skills/agent-skills/crawlforge-stealth-browsing/SKILL.md

@@ -93,8 +93,8 @@ scored. Route through your own residential proxy — CrawlForge supplies none.
 
 Server-wide, set `CRAWLFORGE_STEALTH_PROXIES` (comma-separated proxy URLs): the
 `scrape` escalation stage, `stealth_mode`, `browser_session`,
-`scrape_with_actions` and the `deep_research` retry use it when no proxy is
-passed on the call. (`agent` does not browse yet, so it is not a reader.) Per call, pass
+`scrape_with_actions`, the `deep_research` retry and the `agent` tool's automatic
+stealth retry use it when no proxy is passed on the call. Per call, pass
 `stealthConfig.proxyRotation`, which always wins:
 
 ```json

package/src/skills/agent-skills/crawlforge-stealth-browsing/references/engine-selection.md

@@ -69,8 +69,8 @@ them default to `auto`.
 - `proxyRotation { enabled, proxies[], rotationInterval }` — your own proxies,
   per call. Server-wide, `CRAWLFORGE_STEALTH_PROXIES` (comma-separated URLs of
   the same form) covers the paths with no caller to ask: the `scrape` escalation
-  stage, `stealth_mode`, `browser_session`, `scrape_with_actions` and the
-  `deep_research` retry — not `agent`, which has no browsing path yet. A call's
+  stage, `stealth_mode`, `browser_session`, `scrape_with_actions`, the
+  `deep_research` retry and the `agent` tool's automatic stealth retry. A call's
   own `proxyRotation` always wins. CrawlForge supplies no proxies, and a datacenter
   proxy does not help — Cloudflare scores the ASN before it serves a challenge.
   Unrelated to the `PROXY_ROTATION_*` variables, which belong to `localization`.

package/src/tools/agent/agent.js

@@ -9,6 +9,11 @@ import { z } from 'zod';
 import { AgentOrchestrator } from '../../core/AgentOrchestrator.js';
 import { ElicitationHelper } from '../../core/ElicitationHelper.js';
 import { getToolConfig } from '../../constants/config.js';
+import { setActualCost } from '../../server/requestContext.js';
+import { AGENT_ESCALATION_CREDITS } from './escalation.js';
+
+/** The agent's published price before any stealth retry. */
+export const AGENT_BASE_CREDITS = 8;
 
 export const AgentInputSchema = z.object({
   prompt: z.string().min(1).max(2000).describe('Natural-language task or question'),
@@ -24,7 +29,10 @@ export class AgentTool {
     this._orchestrator = new AgentOrchestrator({
       mcpServer: null,
       searchConfig: getToolConfig('search_web') || {},
-      llmConfig: options.llmConfig || {}
+      llmConfig: options.llmConfig || {},
+      // The stealth stage, injected from server.js so this module never
+      // imports a browser (see AgentOrchestrator's constructor).
+      escalateFetch: options.escalateFetch || null
     });
     this._elicitation = new ElicitationHelper({});
   }
@@ -58,14 +66,26 @@ export class AgentTool {
       }
     }
 
-    return this._orchestrator.run({
-      prompt: validated.prompt,
-      urls: validated.urls,
-      schema: validated.schema,
-      model: validated.model,
-      maxSteps: validated.maxSteps,
-      maxUrls: validated.maxUrls
-    });
+    // Priced like `scrape`'s escalation: getToolCost projects the ceiling
+    // (base + every retry the cap allows), and the charge is lowered to the
+    // retries that got the page — including when run() throws. A retry that
+    // met the wall again is free.
+    const usage = { escalations: 0, charged: 0 };
+    try {
+      return await this._orchestrator.run({
+        prompt: validated.prompt,
+        urls: validated.urls,
+        schema: validated.schema,
+        model: validated.model,
+        maxSteps: validated.maxSteps,
+        maxUrls: validated.maxUrls,
+        usage
+      });
+    } finally {
+      if (validated.model !== 'pro') {
+        setActualCost(AGENT_BASE_CREDITS + AGENT_ESCALATION_CREDITS * usage.charged);
+      }
+    }
   }
 
   async destroy() {

package/src/tools/agent/escalation.js

@@ -0,0 +1,40 @@
+/**
+ * escalation.js — the one declaration of the `agent` tool's automatic stealth
+ * retry (stealth review Phase 3, decision 4: automatic, capped).
+ *
+ * When ACT's plain fetch comes back walled (challenge page, 403/429, empty
+ * shell, timeout), the agent retries the URL once through the same stealth
+ * stage `scrape`'s escalate:true uses. It is not opt-in, so it is capped per
+ * run and priced per retry that gets the page; one that meets the wall again
+ * is free.
+ *
+ * The cap and the price live here so the orchestrator, the tool wrapper and
+ * `AuthManager.getToolCost` read one declaration — the arrangement
+ * `src/tools/scrape/escalation.js` has for `scrape`. Imports nothing, so
+ * AuthManager can read it without pulling in the orchestrator.
+ */
+
+/** What one stealth retry adds to the agent's price: the stealth browser's own. */
+export const AGENT_ESCALATION_CREDITS = 5;
+
+/**
+ * Most stealth retries one run may make. A second, independent bound on top
+ * of maxSteps/maxUrls/wall-clock: each retry is a browser launch (seconds,
+ * and hundreds of MB) and 5 credits, and the projection a caller is checked
+ * against has to be a finite ceiling.
+ */
+export const AGENT_MAX_ESCALATIONS = 2;
+
+/**
+ * The most escalation can add to one call — the ceiling getToolCost projects.
+ * Reads the RAW params (getToolCost runs before validation). The pro model
+ * delegates to ResearchOrchestrator and never runs this stage, so it adds 0.
+ *
+ * @param {object|undefined} params
+ * @returns {number}
+ */
+export function agentEscalationSurcharge(params) {
+  if (params?.model === 'pro') return 0;
+  const maxUrls = Number.isFinite(params?.maxUrls) && params.maxUrls >= 1 ? Math.floor(params.maxUrls) : AGENT_MAX_ESCALATIONS;
+  return AGENT_ESCALATION_CREDITS * Math.min(AGENT_MAX_ESCALATIONS, maxUrls);
+}