crawlforge-mcp-server 4.8.1 → 4.9.0

package/CLAUDE.md

@@ -60,9 +60,9 @@ These guidelines are working if: fewer unnecessary changes in diffs, fewer rewri
 
 ## Project Overview
 
-CrawlForge MCP Server - A professional MCP (Model Context Protocol) server providing 26 web scraping, crawling, and content processing tools (5 inline + 21 advanced).
+CrawlForge MCP Server - A professional MCP (Model Context Protocol) server providing 27 web scraping, crawling, and content processing tools (5 inline + 22 advanced).
 
-**Current Version:** 4.8.1
+**Current Version:** 4.9.0
 
 ## Development Commands
 
@@ -166,18 +166,21 @@ Tools are organized in subdirectories by category:
 - `crawl/` - crawlDeep, mapSite
 - `extract/` - analyzeContent, extractContent, extractStructured, extractWithLlm, listOllamaModels, processDocument, summarizeContent
 - `research/` - deepResearch
-- `search/` - searchWeb (proxied through CrawlForge.dev API)
+- `search/` - searchWeb (proxied through CrawlForge.dev API), serpRank (real Google organic rank via DataForSEO; adapter in `search/adapters/dataforseoSearch.js`)
 - `templates/` - ScrapeTemplateTool (10 pre-built site templates, v4.0.0)
 - `tracking/` - trackChanges
 - `llmstxt/` - generateLLMsTxt
 
-### Available MCP Tools (26 total)
+### Available MCP Tools (27 total)
 
 **Basic Tools (server.js inline, 5):**
 fetch_url, extract_text, extract_links, extract_metadata, scrape_structured
 
-**Advanced Tools (21):**
-search_web, crawl_deep, map_site, extract_content, process_document, summarize_content, analyze_content, extract_structured, extract_with_llm, list_ollama_models, batch_scrape, scrape_with_actions, deep_research, track_changes, generate_llms_txt, stealth_mode, localization, scrape_template, scrape, agent
+**Advanced Tools (22):**
+search_web, serp_rank, crawl_deep, map_site, extract_content, process_document, summarize_content, analyze_content, extract_structured, extract_with_llm, list_ollama_models, batch_scrape, scrape_with_actions, deep_research, track_changes, generate_llms_txt, stealth_mode, localization, scrape_template, scrape, agent
+
+**serp_rank (DataForSEO):**
+- `serp_rank` — reports where a target domain ranks in Google's REAL organic results for a keyword (the SERP position Google Custom Search / `search_web` cannot give). Backed by the DataForSEO Google Organic SERP API (Live Advanced, `POST /v3/serp/google/organic/live/advanced`, HTTP Basic auth). Credentials via `DATAFORSEO_LOGIN` / `DATAFORSEO_PASSWORD`, billed to the user's own DataForSEO account (~US$0.002/call), separate from CrawlForge credits. When unconfigured it returns `{ configured:false }` and charges **0** credits; when configured, **Cost: 5**. Never fabricates a rank.
 
 **v4.6.0 additions (Phase D):**
 - `scrape` — single fetch + one cheerio load dispatching a `formats` array (markdown/html/rawHtml/text/links/metadata/screenshot/json-schema) + `onlyMainContent`; partial-success via per-format `warnings[]`. Cost: 2.

package/README.md

@@ -3,7 +3,7 @@
 </p>
 
 <p align="center">
-  <b>26 web scraping, crawling, deep-research &amp; autonomous-extraction tools for Claude, Cursor &amp; any MCP client.</b><br>
+  <b>27 web scraping, crawling, deep-research &amp; autonomous-extraction tools for Claude, Cursor &amp; any MCP client.</b><br>
   Clean Markdown &amp; structured JSON from any site. Get started with <b>1,000 free credits</b> — no credit card required.
 </p>
 
@@ -35,7 +35,7 @@
 
 ## 🎯 Why CrawlForge?
 
-- **26 MCP-native tools** — scraping, crawling, search, deep research, an autonomous `agent`, a unified multi-format `scrape`, document processing, stealth browsing, and more, callable directly from your AI assistant.
+- **27 MCP-native tools** — scraping, crawling, search, real Google SERP rank tracking, deep research, an autonomous `agent`, a unified multi-format `scrape`, document processing, stealth browsing, and more, callable directly from your AI assistant.
 - **Generous free tier** — 1,000 credits to start instantly, no credit card. Credits never expire and roll over month-to-month.
 - **Local-LLM by default** — `extract_with_llm` runs against a local **Ollama** model out of the box: no LLM API key, no per-token cost, and your data never leaves your machine. Cloud (OpenAI/Anthropic) is opt-in.
 - **LLM-ready output** — clean Markdown, structured JSON (schema-driven), screenshots, links, and metadata from a single fetch.
@@ -47,7 +47,7 @@
 
 | | **CrawlForge MCP** | Firecrawl | Raw scraping API |
 |---|:---:|:---:|:---:|
-| Native MCP server | ✅ 26 tools | ✅ | ❌ |
+| Native MCP server | ✅ 27 tools | ✅ | ❌ |
 | Free tier | ✅ 1,000 credits, rollover | Limited | Varies |
 | Self-hosted / local LLM extraction (Ollama) | ✅ default, $0/token | ❌ | ❌ |
 | Autonomous agent (no URLs needed) | ✅ `agent` | ✅ | ❌ |
@@ -176,6 +176,7 @@ CrawlForge requires a CrawlForge API key — **every tool is metered and consume
 | `summarize_content` | 4 | Generate intelligent summaries |
 | `crawl_deep` | 4 | Deep crawl entire websites |
 | `search_web` | 5 | Search the web using Google Search API |
+| `serp_rank` | 5 | Check where a domain ranks in Google's **real organic SERP** for a keyword (the position `search_web` can't give). Powered by DataForSEO (`DATAFORSEO_LOGIN`/`DATAFORSEO_PASSWORD`, billed to your own DataForSEO account). Returns `{ configured:false }` and charges **0** credits until configured |
 | `batch_scrape` | 5 | Process multiple URLs simultaneously |
 | `scrape_with_actions` | 5 | Browser automation chains |
 | `generate_llms_txt` | 5 | Generate AI interaction guidelines |
@@ -199,7 +200,7 @@ For the full canonical capabilities reference (all tools, CLI commands, stealth
 | **Business** ($399) | 250,000 | Large scale operations |
 
 **All plans include:**
-- Access to all 26 tools
+- Access to all 27 tools
 - Credits never expire and roll over month-to-month
 - API access and webhook notifications
 

package/package.json

@@ -1,8 +1,8 @@
 {
   "name": "crawlforge-mcp-server",
-  "version": "4.8.1",
+  "version": "4.9.0",
   "mcpName": "io.github.mysleekdesigns/crawlforge-mcp-server",
-  "description": "CrawlForge MCP Server - Professional Model Context Protocol server with 26 web scraping, crawling, deep-research, and autonomous-extraction tools. Returns clean Markdown and structured JSON for Claude, Cursor, and any MCP client. Defaults to local Ollama for LLM extraction (no API key needed); OpenAI/Anthropic available as opt-in. Includes a unified multi-format scrape tool, an autonomous agent, pre-built site templates, and Camoufox stealth browsing.",
+  "description": "CrawlForge MCP Server - Professional Model Context Protocol server with 27 web scraping, crawling, deep-research, and autonomous-extraction tools. Returns clean Markdown and structured JSON for Claude, Cursor, and any MCP client. Defaults to local Ollama for LLM extraction (no API key needed); OpenAI/Anthropic available as opt-in. Includes a unified multi-format scrape tool, an autonomous agent, pre-built site templates, and Camoufox stealth browsing.",
   "main": "server.js",
   "bin": {
     "crawlforge": "src/cli/index.js",

package/server.js

@@ -9,6 +9,7 @@ import { McpServer, ResourceTemplate } from "@modelcontextprotocol/sdk/server/mc
 import { z } from "zod";
 import { logger } from "./src/utils/Logger.js";
 import { SearchWebTool } from "./src/tools/search/searchWeb.js";
+import { SerpRankTool } from "./src/tools/search/serpRank.js";
 import { CrawlDeepTool } from "./src/tools/crawl/crawlDeep.js";
 import { MapSiteTool } from "./src/tools/crawl/mapSite.js";
 import { ExtractContentTool } from "./src/tools/extract/extractContent.js";
@@ -89,8 +90,8 @@ if (configErrors.length > 0 && config.server.nodeEnv === 'production') {
 // Create the server
 const server = new McpServer({
   name: "crawlforge",
-  version: "4.8.1",
-  description: "Production-ready MCP server with 26 web scraping, crawling, and content processing tools. Features MCP Resources (crawlforge://), Prompts, Sampling fallback, Elicitation, stealth browsing, deep research, structured extraction, change tracking, local-LLM extraction via Ollama, unified multi-format scrape, and autonomous agent tool.",
+  version: "4.9.0",
+  description: "Production-ready MCP server with 27 web scraping, crawling, and content processing tools. Features MCP Resources (crawlforge://), Prompts, Sampling fallback, Elicitation, stealth browsing, deep research, structured extraction, real Google SERP rank tracking, change tracking, local-LLM extraction via Ollama, unified multi-format scrape, and autonomous agent tool.",
   homepage: "https://www.crawlforge.dev",
   icon: "https://www.crawlforge.dev/icon.png"
 });
@@ -104,11 +105,12 @@ server.prompt("getting-started", {
       role: "user",
       content: {
         type: "text",
-        text: "You have access to CrawlForge MCP with 26 web scraping tools. Key tools:\n\n" +
+        text: "You have access to CrawlForge MCP with 27 web scraping tools. Key tools:\n\n" +
           "- fetch_url: Fetch raw HTML/content from any URL\n" +
           "- extract_text: Extract clean text from a webpage\n" +
           "- extract_content: Smart content extraction with readability\n" +
           "- search_web: Search the web and get structured results\n" +
+          "- serp_rank: Check where a domain ranks in Google's real organic SERP for a keyword\n" +
           "- crawl_deep: Crawl a website following links to a specified depth\n" +
           "- map_site: Discover all pages on a website\n" +
           "- batch_scrape: Scrape multiple URLs in parallel\n" +
@@ -139,6 +141,10 @@ const withAuth = makeWithAuth({ authManager: AuthManager, logger, metrics });
 
 // Initialize tools
 const searchWebTool = new SearchWebTool(getToolConfig("search_web"));
+// serp_rank uses DataForSEO credentials (DATAFORSEO_LOGIN / DATAFORSEO_PASSWORD),
+// separate from CrawlForge billing — no getToolConfig needed. Degrades gracefully
+// when unconfigured (returns { configured: false } instead of throwing).
+const serpRankTool = new SerpRankTool();
 const crawlDeepTool = new CrawlDeepTool(getToolConfig('crawl_deep'));
 const mapSiteTool = new MapSiteTool(getToolConfig('map_site'));
 const extractContentTool = new ExtractContentTool();
@@ -363,6 +369,31 @@ server.registerTool("search_web", {
   }
 }));
 
+// Tool: serp_rank — REAL Google organic rank for a target domain (via DataForSEO)
+server.registerTool("serp_rank", {
+  description: "Use this to check where a domain ranks in Google's ORGANIC results for a keyword — real SERP position, not Custom Search order. Returns the target's organic rank, the ranking URL, and every position it holds. Example: serp_rank({keyword: \"managed wordpress hosting\", target: \"dashboardhosting.com\", location_name: \"United States\"})",
+  annotations: { title: "SERP Rank Check", readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: true },
+  inputSchema: {
+    keyword: z.string().describe("The search query to check ranking for"),
+    target: z.string().describe("Domain or URL to locate in the results (e.g. 'example.com')"),
+    location_name: z.string().optional().describe("Location, e.g. 'United States' or 'London,England,United Kingdom'"),
+    location_code: z.number().optional().describe("Numeric DataForSEO location code (overrides location_name)"),
+    language_code: z.string().optional().describe("Language code (e.g. 'en')"),
+    device: z.enum(["desktop", "mobile"]).optional().describe("Device to emulate"),
+    depth: z.number().min(10).max(200).optional().describe("How many results to scan, 10-200 (100 = 1 page of cost)")
+  }
+}, withAuth("serp_rank", async ({ keyword, target, location_name, location_code, language_code, device, depth }) => {
+  try {
+    if (!keyword || !target) {
+      return { content: [{ type: "text", text: "Both 'keyword' and 'target' are required" }], isError: true };
+    }
+    const result = await serpRankTool.execute({ keyword, target, location_name, location_code, language_code, device, depth });
+    return { content: [{ type: "text", text: JSON.stringify(result, null, 2) }] };
+  } catch (error) {
+    return { content: [{ type: "text", text: `SERP rank check failed: ${error.message}` }], isError: true };
+  }
+}));
+
 // Tool: crawl_deep
 server.registerTool("crawl_deep", {
   description: "Use this when you need to discover and optionally extract content from many pages within a site — e.g. building a knowledge base, indexing docs, or auditing all pages. Use map_site first to estimate scope, then crawl_deep for content. Example: crawl_deep({url: \"https://docs.example.com\", max_depth: 3, max_pages: 200, extract_content: true})",
@@ -1309,7 +1340,7 @@ async function runServer() {
 
   const allTools = [
     "fetch_url", "extract_text", "extract_links", "extract_metadata", "scrape_structured",
-    "search_web", "crawl_deep", "map_site",
+    "search_web", "serp_rank", "crawl_deep", "map_site",
     "extract_content", "process_document", "summarize_content", "analyze_content",
     "batch_scrape", "get_batch_results", "scrape_with_actions",
     "deep_research", "track_changes", "generate_llms_txt",
@@ -1317,7 +1348,7 @@ async function runServer() {
     "list_ollama_models", "scrape_template", // D3.3
     "scrape", "agent"  // D4
   ];
-  console.error(`Tools available (26): ${allTools.join(", ")}`);
+  console.error(`Tools available (27): ${allTools.join(", ")}`);
 
   // Start memory monitoring in development
   if (config.server.nodeEnv === "development") {

package/src/core/AuthManager.js

@@ -542,6 +542,7 @@ class AuthManager {
       scrape_with_actions: 5,
       batch_scrape: 5,
       search_web: 5,
+      serp_rank: 5,
       generate_llms_txt: 5,
 
       // 8 credits
@@ -551,6 +552,17 @@ class AuthManager {
       deep_research: 10
     };
 
+    // serp_rank calls DataForSEO, billed to the user's OWN DataForSEO account
+    // (separate from CrawlForge credits). When DataForSEO isn't configured the
+    // tool short-circuits to a no-op { configured:false } result — never charge
+    // CrawlForge credits for that. (Backend note: crawlforge-website's
+    // TOOL_CREDIT_COSTS must also carry serp_rank:5; if the backend re-derives
+    // cost server-side it cannot see this install's DATAFORSEO_* env, so the
+    // free-when-unconfigured guarantee is enforced client-side here.)
+    if (tool === 'serp_rank' && !(process.env.DATAFORSEO_LOGIN && process.env.DATAFORSEO_PASSWORD)) {
+      return 0;
+    }
+
     return costs[tool] ?? 1;
   }
 
@@ -595,6 +607,11 @@ class AuthManager {
       case 'extract_with_llm':
         note = 'External LLM API call billed by your LLM provider, separate from the credit cost.';
         break;
+      case 'serp_rank':
+        note = projected === 0
+          ? 'DataForSEO not configured — no-op, no credits charged. Set DATAFORSEO_LOGIN/PASSWORD to enable.'
+          : 'DataForSEO SERP API (~US$0.002/call) billed to your own DataForSEO account, separate from the credit cost.';
+        break;
       case 'scrape': {
         projected = base;
         note = 'Fixed cost per invocation. json format may incur external LLM cost (billed by your provider).';

package/src/server/withAuth.js

@@ -59,7 +59,16 @@ export function makeWithAuth({ authManager, logger, metrics = null }) {
         }
 
         const result = await handler(params);
-        outcome = 'success';
+
+        // Tools catch their own failures and return { isError:true } rather than
+        // throwing (the shared pattern in server.js). That is still an ERROR
+        // outcome: bill it the half-credit error rate — not full — and log/metric
+        // it as an error, honoring CLAUDE.md's "half credits on error" contract.
+        const isErrorResult = result?.isError === true;
+        outcome = isErrorResult ? 'error' : 'success';
+        const charge = creditCost === 0
+          ? 0
+          : (isErrorResult ? Math.max(1, Math.floor(creditCost * 0.5)) : creditCost);
 
         // D3.5: Surface cost transparency in all tool responses
         try {
@@ -67,7 +76,7 @@ export function makeWithAuth({ authManager, logger, metrics = null }) {
           const remainingCredits = creatorMode ? Infinity : (authManager.creditCache ? [...authManager.creditCache.values()][0] ?? null : null);
           const costMeta = {
             projected: creditCost,
-            actual: creditCost,
+            actual: creatorMode ? 0 : charge,
             remaining_credits: remainingCredits,
             projection_note: projection.note
           };
@@ -86,15 +95,21 @@ export function makeWithAuth({ authManager, logger, metrics = null }) {
           // Cost injection must never break the request path
         }
 
-        if (!creatorMode) {
-          await authManager.reportUsage(toolName, creditCost, params, 200, Date.now() - startTime);
+        // creditCost === 0 means a genuinely free call (e.g. serp_rank when
+        // DataForSEO is unconfigured — a no-op). Emit NO usage event at all so
+        // the backend has nothing to (re-)price; reporting 0 would still create
+        // a serp_rank record the backend could recompute to full cost.
+        if (!creatorMode && creditCost > 0) {
+          await authManager.reportUsage(toolName, charge, params, isErrorResult ? 500 : 200, Date.now() - startTime);
         }
 
         return result;
       } catch (error) {
         outcome = 'error';
         thrown = error;
-        if (!creatorMode) {
+        // Half-charge on error — but never charge a free (0-cost) call, and
+        // never let Math.max(1, …) floor a 0 up to 1 credit.
+        if (!creatorMode && creditCost > 0) {
           await authManager.reportUsage(
             toolName,
             Math.max(1, Math.floor(creditCost * 0.5)),

package/src/skills/agent-skills/crawlforge-getting-started/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: crawlforge-getting-started
-description: "Orientation and tool-selection guide for the CrawlForge MCP server's 26 web tools. Use when the user is getting started with CrawlForge, asks which CrawlForge tool to use, how to set up the API key, how skills or the CLI work, what a tool costs in credits, or when one tool fails and a fallback is needed. Routes requests to the right specialized skill (web scraping, deep research, stealth, structured extraction, change tracking, batch automation), and explains MCP-tools-vs-CLI, the Ollama-first LLM fallback chain, and per-tool credit costs."
+description: "Orientation and tool-selection guide for the CrawlForge MCP server's 27 web tools. Use when the user is getting started with CrawlForge, asks which CrawlForge tool to use, how to set up the API key, how skills or the CLI work, what a tool costs in credits, or when one tool fails and a fallback is needed. Routes requests to the right specialized skill (web scraping, deep research, stealth, structured extraction, change tracking, batch automation), and explains MCP-tools-vs-CLI, the Ollama-first LLM fallback chain, and per-tool credit costs."
 metadata:
   version: 4.8.0
   source: crawlforge-mcp-server
@@ -8,7 +8,7 @@ metadata:
 
 # CrawlForge: Getting Started
 
-CrawlForge is an MCP server with **26 tools** for web scraping, crawling,
+CrawlForge is an MCP server with **27 tools** for web scraping, crawling,
 extraction, research, change tracking, and AI-compliance. This skill orients you
 and routes each request to the right specialized skill.
 
@@ -36,11 +36,11 @@ stored at `~/.crawlforge/config.json`.
 | Watch a page for changes / monitor pricing | **crawlforge-change-tracking** |
 | Scrape many URLs, run browser actions, generate llms.txt | **crawlforge-batch-automation** |
 
-## The 26 tools at a glance
+## The 27 tools at a glance
 
 - **Basic (5):** fetch_url, extract_text, extract_links, extract_metadata, scrape_structured
 - **Unified (1):** scrape (multi-format single fetch)
-- **Search & research (3):** search_web, deep_research, agent
+- **Search & research (4):** search_web, serp_rank, deep_research, agent
 - **Crawl (2):** crawl_deep, map_site
 - **Extract & analyze (7):** extract_content, process_document, summarize_content, analyze_content, extract_structured, extract_with_llm, list_ollama_models
 - **Batch & automation (4):** batch_scrape, get_batch_results, scrape_with_actions, generate_llms_txt

package/src/tools/search/adapters/dataforseoSearch.js

@@ -0,0 +1,155 @@
+/**
+ * DataForSEO SERP Adapter
+ *
+ * Talks to the DataForSEO Google Organic SERP API to obtain REAL organic
+ * ranking positions (rank_group / rank_absolute) — the data that Google Custom
+ * Search (used by search_web) cannot provide. Used by the serp_rank tool.
+ *
+ * Auth: HTTP Basic (login:password) from DATAFORSEO_LOGIN / DATAFORSEO_PASSWORD.
+ * Get credentials at: https://app.dataforseo.com/api-access
+ *
+ * Endpoint (Live Advanced, synchronous — one request, one response):
+ *   POST https://api.dataforseo.com/v3/serp/google/organic/live/advanced
+ *
+ * Cost: ~US$0.002 per 100 results (depth) on Live Advanced. For high-volume
+ * scheduled tracking, DataForSEO's task-based "Standard" queue (task_post →
+ * tasks_ready → task_get) is cheaper; swap the endpoint + poll if cost matters.
+ */
+
+export class DataForSEOSearchAdapter {
+  constructor(login, password, options = {}) {
+    if (!login || !password) {
+      throw new Error('DataForSEO credentials are required (login + password).');
+    }
+
+    this.login = login;
+    this.password = password;
+    this.apiBaseUrl = options.apiBaseUrl || 'https://api.dataforseo.com';
+    // Live Advanced is synchronous and usually answers in a few seconds; cap it
+    // so a hung connection can't wedge the tool. Overridable for tests/self-host.
+    this.timeoutMs = options.timeoutMs ?? 30000;
+    // HTTP Basic auth header, computed once.
+    this.authHeader = 'Basic ' + Buffer.from(`${login}:${password}`).toString('base64');
+  }
+
+  /**
+   * Run one Google organic SERP lookup and return normalized organic results.
+   * @param {Object} params - Lookup parameters
+   * @param {string} params.keyword - The search query to rank for (required)
+   * @param {string} [params.locationName='United States'] - Human location name
+   * @param {number} [params.locationCode] - Numeric DataForSEO location code (overrides locationName)
+   * @param {string} [params.languageCode='en'] - Language code
+   * @param {('desktop'|'mobile')} [params.device='desktop'] - Device to emulate
+   * @param {number} [params.depth=100] - How many results to scan (100 = one page of cost)
+   * @returns {Promise<{items: Array<Object>, meta: Object}>} Normalized organic results + metadata
+   */
+  async search(params) {
+    const {
+      keyword,
+      locationName = 'United States',
+      locationCode,
+      languageCode = 'en',
+      device = 'desktop',
+      depth = 100,
+    } = params;
+
+    if (!keyword) {
+      throw new Error('keyword is required for a SERP lookup');
+    }
+
+    // DataForSEO accepts an ARRAY of task objects; we send exactly one.
+    const task = {
+      keyword,
+      language_code: languageCode,
+      device,
+      depth,
+    };
+    if (locationCode != null) {
+      task.location_code = locationCode;
+    } else {
+      task.location_name = locationName;
+    }
+
+    let response;
+    try {
+      response = await fetch(`${this.apiBaseUrl}/v3/serp/google/organic/live/advanced`, {
+        method: 'POST',
+        headers: {
+          'Authorization': this.authHeader,
+          'Content-Type': 'application/json',
+        },
+        body: JSON.stringify([task]),
+        signal: AbortSignal.timeout(this.timeoutMs),
+      });
+    } catch (error) {
+      // Timeout (AbortError) or network / DNS / fetch failure — surface clearly.
+      if (error.name === 'TimeoutError' || error.name === 'AbortError') {
+        throw new Error(`DataForSEO request timed out after ${this.timeoutMs}ms`);
+      }
+      throw new Error(`Network error connecting to DataForSEO: ${error.message}`);
+    }
+
+    if (!response.ok) {
+      // HTTP-level failure (auth, funds, rate limit, server error).
+      let detail = `${response.status} ${response.statusText}`;
+      if (response.status === 401) {
+        detail = 'Invalid DataForSEO credentials (check DATAFORSEO_LOGIN / DATAFORSEO_PASSWORD).';
+      } else if (response.status === 402) {
+        detail = 'DataForSEO account has insufficient funds. Top up at https://app.dataforseo.com/';
+      } else if (response.status === 429) {
+        detail = 'DataForSEO rate limit exceeded. Slow down or raise your plan limits.';
+      }
+      throw new Error(`DataForSEO request failed: ${detail}`);
+    }
+
+    const data = await response.json();
+
+    // DataForSEO returns HTTP 200 even for logical errors — 20000 means OK.
+    if (!data || data.status_code !== 20000) {
+      throw new Error(
+        `DataForSEO error ${data?.status_code ?? 'unknown'}: ${data?.status_message ?? 'no message'}`,
+      );
+    }
+
+    const task0 = Array.isArray(data.tasks) ? data.tasks[0] : undefined;
+    if (!task0 || task0.status_code !== 20000) {
+      throw new Error(
+        `DataForSEO task error ${task0?.status_code ?? 'unknown'}: ${task0?.status_message ?? 'no result returned'}`,
+      );
+    }
+
+    const result0 = Array.isArray(task0.result) ? task0.result[0] : undefined;
+    const rawItems = result0 && Array.isArray(result0.items) ? result0.items : [];
+
+    // Keep only true organic results and normalize to a small, stable shape.
+    // rank_group  = position within the organic results (the "rank" most people mean)
+    // rank_absolute = position across ALL SERP elements (ads, snippets, packs, …)
+    const items = rawItems
+      .filter((it) => it && it.type === 'organic')
+      .map((it) => ({
+        position: it.rank_group ?? null,
+        rankAbsolute: it.rank_absolute ?? null,
+        domain: (it.domain || '').toLowerCase(),
+        url: it.url || null,
+        title: it.title || null,
+        snippet: it.description || null,
+      }));
+
+    return {
+      items,
+      meta: {
+        keyword,
+        location: locationCode != null ? locationCode : locationName,
+        languageCode,
+        device,
+        depth,
+        organicCount: items.length,
+        seResultsCount: result0?.se_results_count ?? null,
+        checkUrl: result0?.check_url ?? null, // Google URL to eyeball the real SERP
+        cost: data.cost ?? task0.cost ?? null, // USD charged by DataForSEO for this call
+      },
+    };
+  }
+}
+
+export default DataForSEOSearchAdapter;

package/src/tools/search/serpRank.js

@@ -0,0 +1,119 @@
+/**
+ * serp_rank tool
+ *
+ * Reports where a target domain ranks in Google's REAL organic results for a
+ * keyword — the SERP position that Google Custom Search (search_web) cannot
+ * give you. Backed by the DataForSEO SERP adapter.
+ *
+ * Honesty rule: if DataForSEO is not configured, this returns an explained
+ * { configured: false } result — it never fabricates a rank.
+ */
+
+import { z } from 'zod';
+import { DataForSEOSearchAdapter } from './adapters/dataforseoSearch.js';
+
+const SerpRankSchema = z.object({
+  keyword: z.string().min(1),
+  target: z.string().min(1), // domain or URL to locate in the SERP
+  location_name: z.string().optional().default('United States'),
+  location_code: z.number().int().optional(),
+  language_code: z.string().optional().default('en'),
+  device: z.enum(['desktop', 'mobile']).optional().default('desktop'),
+  depth: z.number().int().min(10).max(200).optional().default(100), // DataForSEO caps depth at 200
+});
+
+/** Reduce a domain or URL to a bare, comparable host: "https://www.Example.com/x" → "example.com". */
+function toBareDomain(value) {
+  return String(value)
+    .trim()
+    .toLowerCase()
+    .replace(/^https?:\/\//, '')
+    .replace(/^www\./, '')
+    .split('/')[0]
+    .split('?')[0];
+}
+
+/** True when a SERP result's domain is the target domain (or a subdomain of it). */
+function domainMatches(resultDomain, target) {
+  const d = toBareDomain(resultDomain);
+  return d === target || d.endsWith(`.${target}`);
+}
+
+export class SerpRankTool {
+  constructor(options = {}) {
+    // DataForSEO uses its own credentials, separate from CrawlForge billing.
+    // The server can start without them — we degrade gracefully at execute().
+    this.login = options.login || process.env.DATAFORSEO_LOGIN || null;
+    this.password = options.password || process.env.DATAFORSEO_PASSWORD || null;
+    this.apiBaseUrl = options.apiBaseUrl; // optional override for tests / self-host
+    this.timeoutMs = options.timeoutMs;   // optional override for tests / self-host
+  }
+
+  async execute(params) {
+    const validated = SerpRankSchema.parse(params);
+
+    // Graceful degradation — NEVER fake a rank. If unconfigured, explain the fix.
+    if (!this.login || !this.password) {
+      return {
+        configured: false,
+        keyword: validated.keyword,
+        target: toBareDomain(validated.target),
+        note:
+          'SERP rank tracking is not configured. Set DATAFORSEO_LOGIN and ' +
+          'DATAFORSEO_PASSWORD to enable real Google organic rank lookups. ' +
+          'Get credentials at https://app.dataforseo.com/api-access',
+      };
+    }
+
+    const adapter = new DataForSEOSearchAdapter(this.login, this.password, {
+      apiBaseUrl: this.apiBaseUrl,
+      timeoutMs: this.timeoutMs,
+    });
+
+    const { items, meta } = await adapter.search({
+      keyword: validated.keyword,
+      locationName: validated.location_name,
+      locationCode: validated.location_code,
+      languageCode: validated.language_code,
+      device: validated.device,
+      depth: validated.depth,
+    });
+
+    const target = toBareDomain(validated.target);
+
+    // A domain can rank multiple times — collect every organic hit, best first.
+    const matches = items
+      .filter((it) => domainMatches(it.domain, target))
+      .sort((a, b) => (a.position ?? Infinity) - (b.position ?? Infinity))
+      .map((it) => ({
+        position: it.position,
+        rankAbsolute: it.rankAbsolute,
+        url: it.url,
+        title: it.title,
+      }));
+
+    const best = matches[0] || null;
+
+    return {
+      configured: true,
+      keyword: validated.keyword,
+      target,
+      found: matches.length > 0,
+      position: best ? best.position : null, // organic rank; null = not within top `depth`
+      rankAbsolute: best ? best.rankAbsolute : null,
+      url: best ? best.url : null,
+      title: best ? best.title : null,
+      allPositions: matches, // every place the domain ranks on this SERP
+      location: meta.location,
+      device: meta.device,
+      depthScanned: meta.depth,
+      organicResults: meta.organicCount,
+      seResultsCount: meta.seResultsCount,
+      checkUrl: meta.checkUrl, // link to view the real SERP
+      cost: meta.cost, // USD charged by DataForSEO for this lookup
+      checkedAt: new Date().toISOString(),
+    };
+  }
+}
+
+export default SerpRankTool;