crawlforge-mcp-server 4.7.1 → 4.8.0

package/src/skills/agent-skills/crawlforge-change-tracking/SKILL.md

@@ -0,0 +1,116 @@
+---
+name: crawlforge-change-tracking
+description: "Monitors web pages for changes over time with CrawlForge's track_changes tool. Use when the user wants to track changes to a page, watch a URL, monitor competitor pricing, detect when content updates, get notified of regulation or product-availability changes, or diff a page against a saved baseline. Workflow: create a baseline with operation create_baseline, then periodically compare with operation compare to get a change percentage and a diff; supports CSS-selector scoping, webhooks, and scheduled monitoring."
+metadata:
+  version: 4.8.0
+  source: crawlforge-mcp-server
+---
+
+# CrawlForge Change Tracking
+
+Detect when a web page changes over time using the `track_changes` tool. Useful
+for competitor pricing, regulation updates, product availability, and any page
+you need to watch for edits.
+
+## When to use
+
+- "Track changes to this page" / "watch this URL"
+- "Tell me when competitor pricing changes"
+- "Detect when this content updates"
+- "Diff this page against last week's version"
+- "Notify me when product availability / a regulation changes"
+
+## Core workflow
+
+`track_changes` is one tool driven by an `operation` parameter (cost: 3 credits
+per call).
+
+1. **Create a baseline** the first time:
+
+```json
+{ "tool": "track_changes", "params": { "url": "https://example.com/pricing", "operation": "create_baseline" } }
+```
+
+2. **Compare** later to get the change percentage + diff:
+
+```json
+{ "tool": "track_changes", "params": { "url": "https://example.com/pricing", "operation": "compare" } }
+```
+
+`compare` is the default operation. It returns a change percentage and a
+structured diff against the stored baseline.
+
+## Scoping to part of a page
+
+Use `trackingOptions` to ignore noise and focus on what matters:
+
+```json
+{
+  "tool": "track_changes",
+  "params": {
+    "url": "https://example.com/pricing",
+    "operation": "compare",
+    "trackingOptions": {
+      "granularity": "element",
+      "customSelectors": [".price", ".plan-name"],
+      "excludeSelectors": [".timestamp", ".ad"],
+      "ignoreWhitespace": true
+    }
+  }
+}
+```
+
+`granularity`: `page`, `section` (default), `element`, `text`. Toggle
+`trackText`, `trackStructure`, `trackLinks`, `trackImages`. Set
+`significanceThresholds` (`minor`/`moderate`/`major`) to classify change size.
+
+CLI: `crawlforge track https://example.com --selector ".price" --threshold 1`.
+
+## Scheduled monitoring & notifications
+
+Run continuous monitoring with webhooks instead of polling manually:
+
+```json
+{
+  "tool": "track_changes",
+  "params": {
+    "url": "https://example.com/pricing",
+    "operation": "create_scheduled_monitor",
+    "monitoringOptions": {
+      "enabled": true,
+      "interval": 300000,
+      "notificationThreshold": "moderate",
+      "enableWebhook": true,
+      "webhookUrl": "https://my-site.com/notify"
+    }
+  }
+}
+```
+
+CLI (runs until Ctrl+C):
+`crawlforge monitor https://example.com --interval 60 --webhook https://my-site.com/notify`.
+
+## Other operations
+
+| Operation | Purpose |
+|-----------|---------|
+| `create_baseline` | Save the first snapshot to diff against. |
+| `compare` (default) | Diff current content vs baseline → % change + diff. |
+| `monitor` | One monitoring pass. |
+| `get_history` | Retrieve past change records (`queryOptions`). |
+| `get_stats` | Summary statistics for a tracked URL. |
+| `create_scheduled_monitor` / `stop_scheduled_monitor` | Manage cron-style monitors. |
+| `get_dashboard` | Aggregate status, recent alerts, trends. |
+| `export_history` | Export change history as `json` or `csv`. |
+| `create_alert_rule` | Conditional alerts (webhook / email / slack). |
+| `generate_trend_report` | Trend analysis over time. |
+| `get_monitoring_templates` | List built-in monitoring presets. |
+
+You can also pass `content` or `html` directly to compare pre-fetched content
+against the baseline without re-fetching.
+
+## Cost note
+
+`track_changes` = 3 credits per call. A typical watch is one `create_baseline`
+plus periodic `compare` calls; scheduled monitors run server-side and notify via
+webhook/Slack so you don't pay for manual polling loops.

package/src/skills/agent-skills/crawlforge-deep-research/SKILL.md

@@ -0,0 +1,108 @@
+---
+name: crawlforge-deep-research
+description: "Runs multi-source web research and autonomous question-answering with CrawlForge's deep_research, agent, and search_web tools. Use when the user wants to research a topic, do a deep dive, compare competitors, gather facts with citations, answer a question from the web, search the web, or get a synthesized report from many sources. deep_research expands queries, fetches and dedupes sources, then synthesizes; agent autonomously plans searches and navigation from a plain-English prompt with no URLs needed; search_web returns ranked results. Caps costs via max_urls and confirms before expensive runs."
+metadata:
+  version: 4.8.0
+  source: crawlforge-mcp-server
+---
+
+# CrawlForge Deep Research
+
+Answer questions and produce reports from many web sources using the CrawlForge
+MCP server. Three tools, from lightest to heaviest: `search_web` (ranked
+results), `agent` (autonomous plan-and-answer), `deep_research` (exhaustive
+multi-source synthesis).
+
+## When to use
+
+- "Search the web for X" / "find pages about X" → `search_web`
+- "Answer this question from the web" / "what are the top 5 X" (no URLs given) → `agent`
+- "Research this topic in depth" / "compare competitors" / "give me a cited
+  report" / "gather facts with sources" → `deep_research`
+
+Do NOT use these for a single known URL — use the **crawlforge-web-scraping**
+skill (`scrape` / `extract_content`) instead.
+
+## Tool selection
+
+| Need | Tool | Cost |
+|------|------|------|
+| A ranked list of result URLs + snippets | `search_web` | 5 |
+| A direct answer, agent decides what to read | `agent` | 8 (scales) |
+| A synthesized, multi-source, cited report | `deep_research` | 10+ (scales) |
+
+A common pipeline: `search_web` to find sources → `batch_scrape` (see
+**crawlforge-batch-automation**) to read them → `deep_research` to synthesize.
+
+## search_web (cost: 5)
+
+```json
+{
+  "tool": "search_web",
+  "params": { "query": "best MCP servers 2025", "limit": 10, "time_range": "month" }
+}
+```
+
+Returns titles, URLs, snippets. Supports `lang`, `site` (domain filter),
+`file_type`, `time_range` (`day`/`week`/`month`/`year`/`all`), `expand_query`,
+`enable_ranking`, and `enable_deduplication`. CLI:
+`crawlforge search "MCP server tutorial" --limit 5`.
+
+## agent — autonomous answer (cost: 8, scales with maxUrls)
+
+No URLs required. The agent plans search queries, fetches and filters relevant
+pages, then returns a prose or structured answer.
+
+```json
+{
+  "tool": "agent",
+  "params": { "prompt": "What are the top 5 MCP servers in 2025 and who maintains them?", "maxUrls": 10 }
+}
+```
+
+- `model:"default"` runs a SamplingClient loop (no LLM keys needed).
+- `model:"pro"` runs the full ResearchOrchestrator (deeper, multi-source); it
+  asks for confirmation (elicitation) before running.
+- Pass `schema` for structured JSON output, `urls` for optional seed URLs.
+- Hard limits enforced by the orchestrator: `maxSteps` ≤ 10, `maxUrls` ≤ 20,
+  120s wall-clock. Output is degraded-but-useful if no LLM keys/Ollama exist.
+
+## deep_research — exhaustive report (cost: 10+, scales)
+
+```json
+{
+  "tool": "deep_research",
+  "params": {
+    "topic": "React vs Vue vs Angular in 2025",
+    "maxUrls": 20,
+    "researchApproach": "comparative",
+    "outputFormat": "comprehensive"
+  }
+}
+```
+
+How it works: query expansion → parallel fetch (deduped) → content extraction →
+conflict detection → synthesis. `researchApproach`: `broad`, `focused`,
+`academic`, `current_events`, `comparative`. `outputFormat`: `comprehensive`,
+`summary`, `citations_only`, `conflicts_focus`. Results are stored as a
+`crawlforge://research/{sessionId}` resource — re-read them without re-running.
+
+CLI: `crawlforge research "B2B SaaS pricing trends" --depth deep --max-urls 30`.
+
+### Cost control
+
+`deep_research` starts at 10 credits and grows with sources. **`maxUrls > 50`
+triggers a confirmation prompt (elicitation).** Cap cost with `maxUrls`
+(e.g. `maxUrls:10` ≈ 12 credits). `timeLimit` (default 120000 ms) bounds
+runtime.
+
+## Synthesis & LLM fallback
+
+The server ships with **no LLM key by design**. The fallback chain is:
+Ollama (local) → OpenAI key → Anthropic key → MCP Sampling → raw evidence.
+With no LLM configured, `deep_research` returns structured **raw evidence** for
+the calling assistant (e.g. Claude Code) to synthesize — this is expected, do
+not suggest adding API keys.
+
+See [research workflows](references/workflows.md) for pipelines, depth tiers,
+and parameter detail.

package/src/skills/agent-skills/crawlforge-deep-research/references/workflows.md

@@ -0,0 +1,76 @@
+# Deep Research — Workflows & Parameters
+
+## Pipelines
+
+### Targeted competitive research
+1. `search_web` `"competitor X pricing"` → candidate URLs.
+2. `batch_scrape` the URLs → content in parallel (see crawlforge-batch-automation).
+3. `deep_research` `"competitor X vs us"` → synthesized comparison.
+
+### Quick fact-find
+- `agent` with `model:"default"` and a tight `maxUrls` (5–8). Cheapest path to a
+  cited answer when you don't need a full report.
+
+### Exhaustive report
+- `deep_research` with `researchApproach` matched to intent and `maxUrls`
+  sized to budget.
+
+## deep_research parameters (cost: 10+, scales with maxUrls)
+
+| Param | Type | Default | Notes |
+|-------|------|---------|-------|
+| `topic` | string | — | Required, 3–500 chars. |
+| `maxDepth` | number | `5` | 1–10. |
+| `maxUrls` | number | `50` | 1–1000. **>50 triggers elicitation.** |
+| `timeLimit` | number | `120000` | 30000–300000 ms. |
+| `researchApproach` | enum | `broad` | `broad`, `focused`, `academic`, `current_events`, `comparative`. |
+| `sourceTypes` | string[] | `["any"]` | `academic`, `news`, `government`, `commercial`, `blog`, `wiki`, `any`. |
+| `credibilityThreshold` | number | `0.3` | 0–1 minimum source credibility. |
+| `enableConflictDetection` | boolean | `true` | Flag contradictory claims. |
+| `enableSourceVerification` | boolean | `true` | Score source credibility. |
+| `enableSynthesis` | boolean | `true` | Build a coherent report. |
+| `outputFormat` | enum | `comprehensive` | `comprehensive`, `summary`, `citations_only`, `conflicts_focus`. |
+| `includeRawData` | boolean | `false` | Include raw scraped data. |
+| `concurrency` | number | `5` | 1–20. |
+| `webhook` | object | — | `started`/`progress`/`completed`/`failed` callbacks. |
+
+Results stored at `crawlforge://research/{sessionId}` (list via `resources/list`).
+
+## Approximate depth tiers (CLI `--depth`)
+
+| Depth | URLs | Use case | Approx. credits |
+|-------|------|----------|-----------------|
+| basic | 5–10 | Quick overview | 10–15 |
+| standard | 15–25 | General research | 15–30 |
+| deep | 30–75 | Comprehensive analysis | 30–75+ |
+
+## agent parameters (cost: 8, scales with maxUrls)
+
+| Param | Type | Default | Notes |
+|-------|------|---------|-------|
+| `prompt` | string | — | Required, 1–2000 chars. |
+| `urls` | string[] | — | Optional seed URLs, max 20. |
+| `schema` | object | — | JSON schema for structured output. |
+| `model` | enum | `default` | `default` = SamplingClient loop; `pro` = ResearchOrchestrator (confirms first). |
+| `maxSteps` | number | `5` | Hard cap 10. |
+| `maxUrls` | number | `10` | Hard cap 20. |
+
+Orchestrator-enforced hard stops (never delegated to the LLM): maxSteps ≤ 10,
+maxUrls ≤ 20, 120s wall-clock.
+
+## search_web parameters (cost: 5)
+
+| Param | Type | Notes |
+|-------|------|-------|
+| `query` | string | Required. |
+| `limit` | number | 1–100. |
+| `offset` | number | Pagination. |
+| `lang` | string | e.g. `en`, `fr`. |
+| `time_range` | enum | `day`, `week`, `month`, `year`, `all`. |
+| `site` | string | Restrict to a domain. |
+| `file_type` | string | e.g. `pdf`, `doc`. |
+| `provider` | enum | `crawlforge` (default) or `searxng`. |
+| `expand_query` | boolean | Synonyms/stemming/etc. |
+| `enable_ranking` | boolean | BM25 + signal re-ranking. |
+| `enable_deduplication` | boolean | Drop near-duplicates. |
+| `localization` | object | `countryCode`, `language`, `timezone`, geo-targeting. |

package/src/skills/agent-skills/crawlforge-getting-started/SKILL.md

@@ -0,0 +1,89 @@
+---
+name: crawlforge-getting-started
+description: "Orientation and tool-selection guide for the CrawlForge MCP server's 26 web tools. Use when the user is getting started with CrawlForge, asks which CrawlForge tool to use, how to set up the API key, how skills or the CLI work, what a tool costs in credits, or when one tool fails and a fallback is needed. Routes requests to the right specialized skill (web scraping, deep research, stealth, structured extraction, change tracking, batch automation), and explains MCP-tools-vs-CLI, the Ollama-first LLM fallback chain, and per-tool credit costs."
+metadata:
+  version: 4.8.0
+  source: crawlforge-mcp-server
+---
+
+# CrawlForge: Getting Started
+
+CrawlForge is an MCP server with **26 tools** for web scraping, crawling,
+extraction, research, change tracking, and AI-compliance. This skill orients you
+and routes each request to the right specialized skill.
+
+## Setup
+
+1. Get an API key at https://crawlforge.dev/signup (1,000 free credits).
+2. Provide it to the server:
+
+```bash
+export CRAWLFORGE_API_KEY="your_api_key_here"
+# or, for the CLI, run: crawlforge init   (detects key, installs skills, merges MCP config)
+```
+
+Every tool is metered and requires a key — there is no free tier. User config is
+stored at `~/.crawlforge/config.json`.
+
+## Route to the right skill
+
+| The user wants to... | Use skill |
+|----------------------|-----------|
+| Scrape a page, get markdown/text/links/metadata, map or crawl a site | **crawlforge-web-scraping** |
+| Research a topic, search the web, get a cited report, autonomous Q&A | **crawlforge-deep-research** |
+| Get past 403/CAPTCHA/Cloudflare, or emulate a region/locale | **crawlforge-stealth-browsing** |
+| Extract JSON/fields, parse a PDF, summarize, analyze sentiment | **crawlforge-structured-extraction** |
+| Watch a page for changes / monitor pricing | **crawlforge-change-tracking** |
+| Scrape many URLs, run browser actions, generate llms.txt | **crawlforge-batch-automation** |
+
+## The 26 tools at a glance
+
+- **Basic (5):** fetch_url, extract_text, extract_links, extract_metadata, scrape_structured
+- **Unified (1):** scrape (multi-format single fetch)
+- **Search & research (3):** search_web, deep_research, agent
+- **Crawl (2):** crawl_deep, map_site
+- **Extract & analyze (7):** extract_content, process_document, summarize_content, analyze_content, extract_structured, extract_with_llm, list_ollama_models
+- **Batch & automation (4):** batch_scrape, get_batch_results, scrape_with_actions, generate_llms_txt
+- **Stealth & locale (2):** stealth_mode, localization
+- **Templates & tracking (2):** scrape_template, track_changes
+
+## MCP tools vs CLI
+
+- **MCP tools** — call inline within an AI assistant session (Claude Code,
+  Cursor, etc.). This is the default in chat.
+- **CLI** (`crawlforge <command>`) — for scripts, CI, and pipelines. 15 tool
+  commands + 2 skill commands. See [cli](references/cli.md).
+
+Both hit the same backend and consume the same credits.
+
+## LLM fallback chain (Ollama-first)
+
+LLM-backed tools (`extract_with_llm`, `extract_structured`, abstractive
+`summarize_content`, `deep_research`, `agent`) resolve a provider in this order:
+
+```
+Ollama (local, default, no key) → OpenAI key → Anthropic key → MCP Sampling → raw evidence
+```
+
+The server ships with **no cloud LLM key by design**. With none configured,
+research tools return raw evidence for the calling assistant to synthesize.
+Do not suggest adding API keys — local Ollama is the intended zero-cost default.
+
+## When a tool fails — fallbacks
+
+| Symptom | Try next |
+|---------|----------|
+| `scrape` / `fetch_url` returns 403, 429, CAPTCHA, or empty JS shell | `stealth_mode` (crawlforge-stealth-browsing) |
+| No template for a known site | `scrape_structured` → `extract_structured` → `extract_with_llm` |
+| LLM extraction unavailable (no Ollama/keys) | `scrape_structured` with CSS selectors |
+| Single page too slow / many pages | `batch_scrape` (async + webhook) |
+| Wrong region / currency shown | `localization` |
+| Need a big report but cost is high | lower `maxUrls` on `deep_research` |
+
+## Credits
+
+Costs range from 1 credit (fetch_url, extract_text, scrape_template,
+get_batch_results...) up to 10+ for `deep_research`. Full table:
+[credits](references/credits.md). Prefer the cheapest tool that does the job, and
+cap dynamic tools (`deep_research`, `agent`, `crawl_deep`) with their max
+parameters.

package/src/skills/agent-skills/crawlforge-getting-started/references/cli.md

@@ -0,0 +1,71 @@
+# CrawlForge CLI — Subcommand Summary
+
+The `crawlforge` CLI exposes the tools as command-line subcommands for scripts,
+CI, and pipelines.
+
+## Install & auth
+
+```bash
+npm install -g crawlforge-mcp-server      # or: npx crawlforge-mcp-server <command>
+export CRAWLFORGE_API_KEY="your_api_key_here"   # or pass --api-key per command
+crawlforge init                            # detect key, install skills, merge MCP config
+```
+
+## Global flags
+
+| Flag | Description |
+|------|-------------|
+| `--json` | Compact JSON output (pipe-friendly). |
+| `--pretty` | Pretty-printed JSON. |
+| `--quiet` | Suppress stdout (exit code only). |
+| `--api-key <key>` | Override `CRAWLFORGE_API_KEY`. |
+| `--timeout <ms>` | Request timeout (default 30000). |
+| `--version` / `--help` | Version / help. |
+
+## Tool commands (15)
+
+| Command | Maps to | Example |
+|---------|---------|---------|
+| `scrape <url>` | fetch_url / extract_content | `crawlforge scrape https://example.com --extract --format markdown` |
+| `search <query>` | search_web | `crawlforge search "MCP tutorial" --limit 5` |
+| `crawl <url>` | crawl_deep | `crawlforge crawl https://docs.example.com --depth 3 --max-pages 200` |
+| `map <url>` | map_site | `crawlforge map https://example.com --format xml` |
+| `extract <url>` | extract_structured / extract_with_llm | `crawlforge extract <url> --prompt "title, author" --model llama3.2` |
+| `track <url>` | track_changes | `crawlforge track <url> --selector ".price" --threshold 1` |
+| `analyze <url>` | analyze_content | `crawlforge analyze <url> --depth full` |
+| `research <topic>` | deep_research | `crawlforge research "AI trends 2025" --depth deep --max-urls 30` |
+| `stealth <url>` | stealth_mode | `crawlforge stealth <url> --engine camoufox --screenshot` |
+| `batch <file>` | batch_scrape | `crawlforge batch urls.txt --format markdown --concurrency 10` |
+| `actions <url>` | scrape_with_actions | `crawlforge actions <url> --script flow.json --screenshot` |
+| `localize <url>` | localization | `crawlforge localize <url> --locale fr-FR --country FR` |
+| `llmstxt <url>` | generate_llms_txt | `crawlforge llmstxt <url> --include-full` |
+| `template <id> <target>` | scrape_template | `crawlforge template github-repo https://github.com/owner/repo` |
+| `monitor <url>` | track_changes (scheduled) | `crawlforge monitor <url> --interval 60 --webhook <url>` |
+
+## Skill commands (2)
+
+| Command | Purpose |
+|---------|---------|
+| `install-skills --target <claude-code\|cursor\|vscode\|all>` | Install skill files into your AI tool. |
+| `uninstall-skills --target <...>` | Remove them. |
+
+## Output modes
+
+```bash
+crawlforge search "nodejs MCP"                       # human-readable
+crawlforge research "AI 2025" --pretty > report.json  # pretty JSON to file
+crawlforge search "nodejs" --json | jq '.results[].url'  # compact JSON to jq
+crawlforge scrape https://example.com --quiet && echo ok  # exit code only
+```
+
+## Relevant env vars
+
+| Variable | Purpose |
+|----------|---------|
+| `CRAWLFORGE_API_KEY` | API key (required for production). |
+| `OLLAMA_BASE_URL` | Ollama server (default http://localhost:11434). |
+| `OLLAMA_DEFAULT_MODEL` | Default model for `extract`. |
+| `CRAWLFORGE_STEALTH_ENGINE` | Force `playwright` or `camoufox`. |
+| `CRAWLFORGE_BROWSER_BACKEND` | `local` or `browserbase`. |
+
+Exit code 0 = success, 1 = error. `crawlforge <command> --help` for per-command help.

package/src/skills/agent-skills/crawlforge-getting-started/references/credits.md

@@ -0,0 +1,75 @@
+# CrawlForge Credit Costs
+
+Per-tool credit cost (source of truth: `AuthManager.getToolCost`). Every tool is
+metered; there is no free tier. Tools marked "scales" cost more as work grows.
+
+## 1 credit
+
+| Tool | Notes |
+|------|-------|
+| `fetch_url` | Raw HTTP fetch. |
+| `extract_text` | Tag-stripped text/markdown. |
+| `extract_links` | All links on a page. |
+| `extract_metadata` | Title / meta / OG / schema.org. |
+| `scrape_template` | Pre-built site extractor. |
+| `list_ollama_models` | List local LLMs. |
+| `get_batch_results` | Retrieve an already-paid batch. |
+
+## 2 credits
+
+| Tool | Notes |
+|------|-------|
+| `scrape` | Unified multi-format single fetch. |
+| `scrape_structured` | CSS-selector extraction. |
+| `extract_content` | Readability-cleaned article. |
+| `map_site` | URL discovery / sitemap. |
+| `process_document` | PDF / DOCX / TXT parsing. |
+| `localization` | Locale / geo emulation. |
+
+## 3 credits
+
+| Tool | Notes |
+|------|-------|
+| `track_changes` | Baseline / compare / monitor. |
+| `analyze_content` | Sentiment / entities / keywords. |
+| `extract_structured` | Schema-driven (LLM + CSS fallback). |
+| `extract_with_llm` | NL-prompt extraction. |
+
+## 4 credits
+
+| Tool | Notes |
+|------|-------|
+| `summarize_content` | Extractive / abstractive summary. |
+| `crawl_deep` | Site crawl; **scales** with page count. |
+
+## 5 credits
+
+| Tool | Notes |
+|------|-------|
+| `stealth_mode` | Anti-bot browser (screenshots add a little). |
+| `scrape_with_actions` | Browser automation then scrape. |
+| `batch_scrape` | Many URLs; projection scales with URL count. |
+| `search_web` | Web search. |
+| `generate_llms_txt` | AI-compliance file. |
+
+## 8 credits
+
+| Tool | Notes |
+|------|-------|
+| `agent` | Autonomous research; **scales** with `maxUrls` (and `model:"pro"`). |
+
+## 10+ credits
+
+| Tool | Notes |
+|------|-------|
+| `deep_research` | Base 10; **scales** with `maxUrls`. **>50 URLs prompts for confirmation.** |
+
+## Cost-control tips
+
+- Use the cheapest tool that satisfies the request (e.g. `extract_metadata` at 1
+  before `scrape` at 2).
+- Prefer one `scrape` call (2) over several single-format basic calls.
+- Cap dynamic tools: `deep_research`/`agent` via `maxUrls`, `crawl_deep` via
+  `max_pages`, `batch_scrape` via the URL list size.
+- `get_batch_results` (1) is cheap — submit a batch once, page through results.
+- Errors are charged at half the tool cost; creator mode is unlimited.

package/src/skills/agent-skills/crawlforge-stealth-browsing/SKILL.md

@@ -0,0 +1,106 @@
+---
+name: crawlforge-stealth-browsing
+description: "Bypasses bot detection and geo-restrictions with CrawlForge's stealth_mode and localization tools. Use when a site returns 403 or 429, CAPTCHAs, 'please enable JavaScript', or empty content, or is protected by Cloudflare, DataDome, or PerimeterX, or when the user needs region-specific pricing, geo-blocked content, or a specific locale, timezone, or currency. stealth_mode runs a stealth browser (playwright by default, camoufox for advanced fingerprinting) and can screenshot; localization emulates a country and language. Explains when to escalate from a normal scrape to stealth."
+metadata:
+  version: 4.8.0
+  source: crawlforge-mcp-server
+---
+
+# CrawlForge Stealth Browsing
+
+Get past bot-detection systems and geo-blocks. Use `stealth_mode` when a normal
+scrape is blocked, and `localization` when you need region-specific content,
+pricing, or locale emulation.
+
+## When to escalate to stealth_mode
+
+Escalate from a normal `scrape` / `fetch_url` (see crawlforge-web-scraping) when:
+
+- The site returns **403 or 429** on a regular fetch.
+- You get a **CAPTCHA** or a "please enable JavaScript" interstitial.
+- Content comes back **empty** or only a shell (JS-rendered SPA).
+- The site uses **Cloudflare, DataDome, PerimeterX**, or similar protection.
+
+`stealth_mode` drives a real browser with randomized fingerprints, human
+behavior simulation, and WebRTC/canvas/WebGL spoofing.
+
+## stealth_mode (cost: 5)
+
+`stealth_mode` is operation-based. Typical flow: create a context, then create a
+page that navigates to the target URL.
+
+```json
+{
+  "tool": "stealth_mode",
+  "params": {
+    "operation": "create_context",
+    "stealthConfig": { "level": "advanced", "simulateHumanBehavior": true },
+    "engine": "playwright"
+  }
+}
+```
+
+Then use the returned `contextId`:
+
+```json
+{
+  "tool": "stealth_mode",
+  "params": { "operation": "create_page", "contextId": "<id-from-create_context>", "urlToTest": "https://protected-site.com" }
+}
+```
+
+Operations: `configure`, `enable`, `disable`, `create_context`, `create_page`,
+`get_stats`, `cleanup`. `stealthConfig.level` is `basic` / `medium` (default) /
+`advanced`. Always run `cleanup` when done to release the browser.
+
+### Engine: playwright vs camoufox
+
+- `engine:"playwright"` (default) — Chromium with stealth patches. Fast, good
+  for most basic bot detection.
+- `engine:"camoufox"` — Firefox-based with native anti-detection (no patches).
+  Scores higher against DataDome / Cloudflare / PerimeterX and on CreepJS. Use
+  for heavily protected, financial, or e-commerce sites.
+
+Full decision table: [engine selection](references/engine-selection.md).
+
+### CLI
+
+```bash
+crawlforge stealth https://protected-site.com
+crawlforge stealth https://protected-site.com --engine camoufox --wait 3000 --screenshot
+```
+
+The CLI exposes a one-shot form (`--engine`, `--wait <ms>`, `--screenshot`).
+Force the engine globally with `export CRAWLFORGE_STEALTH_ENGINE=camoufox`.
+
+## localization (cost: 2)
+
+Emulate a country/language/timezone/currency for region-specific content and
+geo-blocked pages.
+
+```json
+{
+  "tool": "localization",
+  "params": { "operation": "configure_country", "countryCode": "DE", "language": "de", "currency": "EUR" }
+}
+```
+
+Operations: `configure_country`, `localize_search`, `localize_browser`,
+`generate_timezone_spoof`, `handle_geo_blocking`, `auto_detect`, `get_stats`,
+`get_supported_countries`. `countryCode` is ISO 3166-1 alpha-2; `currency` is
+ISO 4217. Supports proxy routing and GPS geolocation emulation.
+
+CLI: `crawlforge localize https://shop.example.com --locale en-GB --country GB --currency GBP`.
+
+## stealth_mode vs localization
+
+- **Blocked / bot-detected** → `stealth_mode`.
+- **Wrong region / language / currency, but not blocked** → `localization`.
+- **Geo-blocked AND bot-protected** → `localization` to set region context, then
+  `stealth_mode` for the actual fetch.
+
+## Cost note
+
+`stealth_mode` = 5 credits per call (screenshots add a small extra cost);
+`localization` = 2 credits. Try the cheaper `scrape` (2 credits) first and only
+escalate to stealth when you see the block signals above.