@0pen1/scrapingdog-mcp 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 0pen1
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,152 @@
+# scrapingdog-mcp
+
+A [Model Context Protocol](https://modelcontextprotocol.io) server that exposes the
+[Scrapingdog](https://www.scrapingdog.com/documentation/) scraping APIs as tools for
+MCP-compatible clients (Claude Desktop, Claude Code, and others).
+
+Communicates over **stdio**. Built with TypeScript and the official MCP SDK.
+
+## What it does
+
+Nine tools, each wrapping one Scrapingdog endpoint:
+
+| Tool | Endpoint | What it returns |
+|------|----------|-----------------|
+| `scrape` | `/scrape` | Raw HTML of any URL (JS rendering, premium proxies, geotargeting, sessions, stealth, wait) |
+| `screenshot` | `/screenshot` | Page screenshot metadata (png/jpg/webp, full-page, viewport, quality) |
+| `google_search` | `/google` | Google organic results, ads, knowledge graph, SERP features (JSON or HTML) |
+| `bing_search` | `/bing/search` | Bing search results (market/geo/pagination/safe-search) |
+| `duckduckgo_search` | `/duckduckgo/search` | DuckDuckGo results (region, date filter, pagination token) |
+| `baidu_search` | `/baidu/search` | Baidu results (Chinese-language restriction, pagination) |
+| `x_profile` | `/x/profile` | X (Twitter) profile data (name, handle, follower counts, bio) |
+| `x_post` | `/x/post` | X (Twitter) post (tweet) data |
+| `datacenter_proxy` | *(forward proxy)* | Connection details + ready-to-paste curl/Python for `proxy.scrapingdog.com:8081` |
+
+Every tool accepts an optional `api_key` argument to override the configured key for that call.
+
+## Prerequisites
+
+- Node.js **≥ 18.17** (tested on 20.x).
+- A Scrapingdog API key — get one from your [Scrapingdog dashboard](https://www.scrapingdog.com/) after signing up.
+
+## Install & build
+
+```bash
+npm install
+npm run build      # outputs to dist/
+```
+
+## Provide your API key
+
+The key is resolved in this order (first wins):
+
+1. The `api_key` argument on an individual tool call.
+2. The `SCRAPINGDOG_API_KEY` environment variable.
+3. A `.env` file containing `SCRAPINGDOG_API_KEY=...` in the working directory,
+   `~/.scrapingdog.env`, or `~/.env`.
+
+Option **2** (env var) is recommended — the key never touches disk beyond your MCP
+client config, and nothing ends up in tool-call logs.
+
+## Configure your MCP client
+
+### Claude Desktop / Claude Code (`claude_desktop_config.json`)
+
+```jsonc
+{
+  "mcpServers": {
+    "scrapingdog": {
+      "command": "node",
+      "args": ["/absolute/path/to/scrcpy/dist/index.js"],
+      "env": {
+        "SCRAPINGDOG_API_KEY": "your-key-here"
+      }
+    }
+  }
+}
+```
+
+### Install from npm (no clone needed)
+
+Published as [`@0pen1/scrapingdog-mcp`](https://www.npmjs.com/package/@0pen1/scrapingdog-mcp). Run directly with `npx`:
+
+```bash
+npx @0pen1/scrapingdog-mcp
+```
+
+Add it to Claude Code in one line (the server reads `SCRAPINGDOG_API_KEY` from the env you pass):
+
+```bash
+claude mcp add scrapingdog --scope user \
+  --env SCRAPINGDOG_API_KEY=your-key-here \
+  -- npx -y @0pen1/scrapingdog-mcp
+```
+
+Or in an MCP client config (`claude_desktop_config.json` etc.):
+
+```jsonc
+{
+  "mcpServers": {
+    "scrapingdog": {
+      "command": "npx",
+      "args": ["-y", "@0pen1/scrapingdog-mcp"],
+      "env": { "SCRAPINGDOG_API_KEY": "your-key-here" }
+    }
+  }
+}
+```
+
+## Run standalone (for testing)
+
+```bash
+SCRAPINGDOG_API_KEY=your-key node dist/index.js
+```
+
+It speaks JSON-RPC over stdio, so you can pipe messages to it directly.
+
+## Example tool calls
+
+```
+scrape          { "url": "https://example.com", "dynamic": true, "country": "us" }
+google_search   { "query": "best espresso machines 2026", "results": "10", "country": "us" }
+bing_search     { "query": "site:github.com mcp server", "count": "20" }
+duckduckgo_search { "query": "rust async runtime", "df": "m" }
+baidu_search    { "query": "人工智能", "ct": 2 }
+x_profile       { "profileId": "elonmusk" }
+x_post          { "tweetId": "1655608985058267139" }
+screenshot      { "url": "https://example.com", "fullPage": true, "format": "png" }
+datacenter_proxy { "target_url": "https://httpbin.org/ip" }
+```
+
+## How responses are handled
+
+- **JSON endpoints** (search, social, etc.): the parsed JSON is returned as text so the
+  host can reason about it directly.
+- **`/scrape`**: raw HTML is returned as text.
+- **`/screenshot`**: the endpoint returns binary image bytes, which can't be carried as
+  text cleanly. The tool reports the content type and byte length, and notes how to fetch
+  the image directly. (If you need true image delivery, extend the handler to base64-encode
+  into an `image` content block.)
+- **Non-2xx responses**: returned as MCP error results (`isError: true`) with the upstream
+  status, content type, and body. Note Scrapingdog reports invalid keys as HTTP 400 with a
+  JSON "Unauthorized request" message — the body makes this clear.
+
+## Credit costs (per request)
+
+Scrape: 1 (rotating proxy) → 25 (JS render + premium). Google/Bing/DDG/Baidu search: ~5.
+Screenshot: 5. Profile/Post scrapers: 5–10. See the [pricing/credit table](https://www.scrapingdog.com/documentation/)
+for the full breakdown. Requests time out after 60s upstream.
+
+## Project layout
+
+```
+src/
+  index.ts          # entry point — creates McpServer, connects stdio transport
+  scrapingdog.ts    # API key resolution + shared HTTP request helper
+  result.ts         # response → MCP tool-result formatting (ok / error / fromResponse)
+  tools.ts          # the 9 tool definitions (schema + handler) and registration
+```
+
+## License
+
+MIT

package/dist/index.js

@@ -0,0 +1,39 @@
+#!/usr/bin/env node
+/**
+ * Scrapingdog MCP Server — entry point.
+ *
+ * A Model Context Protocol server that exposes the Scrapingdog scraping APIs
+ * as tools for MCP-compatible clients (Claude Desktop, Claude Code, etc.).
+ *
+ * Communicates over stdio. The API key is resolved from (in order):
+ *   1. the `api_key` argument on each tool call,
+ *   2. the SCRAPINGDOG_API_KEY environment variable,
+ *   3. a .env file in the cwd / home directory.
+ *
+ * Run directly:   node dist/index.js
+ * Or after build: scrapingdog-mcp
+ */
+import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
+import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
+import { registerTools } from "./tools.js";
+async function main() {
+    const server = new McpServer({
+        name: "scrapingdog-mcp",
+        version: "1.0.0",
+    }, {
+        instructions: "Scrapingdog MCP server. Tools wrap the Scrapingdog web-scraping and search APIs: " +
+            "scrape (any URL), screenshot, google_search, bing_search, duckduckgo_search, baidu_search, " +
+            "x_profile, x_post, and datacenter_proxy. " +
+            "The API key is read from SCRAPINGDOG_API_KEY, a .env file, or per-call api_key argument.",
+    });
+    registerTools(server);
+    const transport = new StdioServerTransport();
+    await server.connect(transport);
+    // Keep the process alive; the transport owns stdin/stdout. Logging to stderr
+    // so it never corrupts the stdio JSON-RPC stream on stdout.
+    process.stderr.write("[scrapingdog-mcp] server running on stdio\n");
+}
+main().catch((err) => {
+    process.stderr.write(`[scrapingdog-mcp] fatal: ${err instanceof Error ? err.stack ?? err.message : String(err)}\n`);
+    process.exit(1);
+});

package/dist/result.js

@@ -0,0 +1,43 @@
+/**
+ * Helpers for turning a Scrapingdog {@link SdResponse} into an MCP tool result.
+ *
+ * Scrapingdog endpoints return a mix of JSON, raw HTML, and binary images.
+ * The server surfaces whatever comes back so the host (the LLM) can decide
+ * what to do with it, while flagging HTTP errors explicitly.
+ */
+/** Build a success result from a Scrapingdog response. */
+export function ok(result, opts) {
+    const parts = [];
+    if (opts?.note)
+        parts.push(opts.note);
+    // Prefer JSON when we have it — easier for the host to reason about.
+    if (result.json !== null) {
+        parts.push(JSON.stringify(result.json, null, 2));
+    }
+    else {
+        parts.push(result.body);
+    }
+    return { content: [{ type: "text", text: parts.join("\n\n") }] };
+}
+/** Build an error result, surfacing the upstream status + body when present. */
+export function error(message, result) {
+    const parts = [message];
+    if (result) {
+        parts.push(`HTTP ${result.status} (${result.contentType})`);
+        parts.push(result.body || "(empty body)");
+    }
+    return {
+        content: [{ type: "text", text: parts.join("\n\n") }],
+        isError: true,
+    };
+}
+/**
+ * Decide success vs. failure by status code. Non-2xx → error result.
+ * 202 (queued) is treated as success since Scrapingdog uses it for accepted jobs.
+ */
+export function fromResponse(result, opts) {
+    if (result.status >= 200 && result.status < 300) {
+        return ok(result, opts);
+    }
+    return error(`Scrapingdog returned an error.`, result);
+}

package/dist/scrapingdog.js

@@ -0,0 +1,142 @@
+/**
+ * Scrapingdog API client + configuration.
+ *
+ * Resolves the API key with this precedence:
+ *   1. The `api_key` argument passed to an individual tool call (highest).
+ *   2. The `SCRAPINGDOG_API_KEY` environment variable.
+ *   3. A `.env` file in the working directory (SCRAPINGDOG_API_KEY=...).
+ *
+ * All requests go to https://api.scrapingdog.com. The shared request helper
+ * injects the resolved key, forwards every caller-provided parameter, and
+ * transparently returns text or JSON depending on the response content type.
+ */
+import { readFile } from "node:fs/promises";
+import { homedir } from "node:os";
+import { join } from "node:path";
+export const API_BASE = "https://api.scrapingdog.com";
+export const DEFAULT_TIMEOUT_MS = 90_000; // Scrapingdog times out at 60s; leave headroom for network.
+/**
+ * Resolve the API key. Caller-supplied key wins, then env, then .env file.
+ * Throws if no key can be found.
+ */
+export async function resolveApiKey(explicit) {
+    if (explicit && explicit.trim()) {
+        return explicit.trim();
+    }
+    const envKey = process.env.SCRAPINGDOG_API_KEY;
+    if (envKey && envKey.trim()) {
+        return envKey.trim();
+    }
+    // Fall back to a .env file in the cwd or the user's home directory.
+    const fromFile = await readDotenvKey();
+    if (fromFile) {
+        return fromFile;
+    }
+    throw new Error("No Scrapingdog API key found. Set SCRAPINGDOG_API_KEY in the environment, " +
+        "put it in a .env file (SCRAPINGDOG_API_KEY=...), or pass api_key to the tool call.");
+}
+/** Read SCRAPINGDOG_API_KEY from a .env file (cwd first, then home dir). */
+async function readDotenvKey() {
+    const candidates = [
+        join(process.cwd(), ".env"),
+        join(homedir(), ".scrapingdog.env"),
+        join(homedir(), ".env"),
+    ];
+    for (const path of candidates) {
+        try {
+            const contents = await readFile(path, "utf8");
+            const key = parseDotenv(contents);
+            if (key)
+                return key;
+        }
+        catch {
+            // File missing or unreadable — try the next candidate.
+        }
+    }
+    return undefined;
+}
+/** Minimal .env parser: finds SCRAPINGDOG_API_KEY=..., strips quotes/comments. */
+function parseDotenv(contents) {
+    for (const rawLine of contents.split("\n")) {
+        const line = rawLine.trim();
+        if (!line || line.startsWith("#"))
+            continue;
+        const eq = line.indexOf("=");
+        if (eq === -1)
+            continue;
+        const name = line.slice(0, eq).trim();
+        if (name !== "SCRAPINGDOG_API_KEY")
+            continue;
+        let value = line.slice(eq + 1).trim();
+        // Strip inline comment.
+        const hash = value.indexOf(" #");
+        if (hash !== -1)
+            value = value.slice(0, hash).trim();
+        // Strip surrounding quotes.
+        if ((value.startsWith('"') && value.endsWith('"')) ||
+            (value.startsWith("'") && value.endsWith("'"))) {
+            value = value.slice(1, -1);
+        }
+        if (value)
+            return value;
+    }
+    return undefined;
+}
+/**
+ * Send a request to the Scrapingdog API and return a normalized result.
+ *
+ * Booleans in `params` are lowercased (`true`/`false`) to match the API's
+ * expected string form. `undefined` values are dropped. Numbers are stringified.
+ */
+export async function sdRequest(opts) {
+    const apiKey = await resolveApiKey(opts.params.api_key);
+    const params = new URLSearchParams();
+    params.set("api_key", apiKey);
+    for (const [key, value] of Object.entries(opts.params)) {
+        if (key === "api_key")
+            continue; // already set, resolved form takes precedence
+        if (value === undefined || value === null)
+            continue;
+        if (typeof value === "boolean") {
+            params.set(key, value ? "true" : "false");
+        }
+        else {
+            params.set(key, String(value));
+        }
+    }
+    const url = `${API_BASE}/${opts.path.replace(/^\/+/, "")}?${params.toString()}`;
+    const controller = new AbortController();
+    const timeout = setTimeout(() => controller.abort(), opts.timeoutMs ?? DEFAULT_TIMEOUT_MS);
+    let response;
+    try {
+        response = await fetch(url, {
+            method: opts.method ?? "GET",
+            headers: opts.headers,
+            body: opts.body,
+            signal: controller.signal,
+        });
+    }
+    catch (err) {
+        clearTimeout(timeout);
+        const message = err instanceof Error ? err.message : String(err);
+        throw new Error(`Scrapingdog request failed: ${message}`);
+    }
+    clearTimeout(timeout);
+    const body = await response.text();
+    const contentType = response.headers.get("content-type") ?? "text/plain";
+    let json = null;
+    if (contentType.includes("application/json")) {
+        try {
+            json = JSON.parse(body);
+        }
+        catch {
+            // Body wasn't valid JSON even though the header said so — leave json null.
+        }
+    }
+    return {
+        status: response.status,
+        body,
+        contentType,
+        json,
+    };
+}

package/dist/tools.js

@@ -0,0 +1,415 @@
+/**
+ * Tool definitions for the Scrapingdog MCP server.
+ *
+ * Each tool wraps one Scrapingdog API endpoint documented at
+ * https://www.scrapingdog.com/documentation/. Parameters mirror the upstream
+ * query parameters; boolean flags are sent as lowercased "true"/"false".
+ *
+ * The API key is resolved per-call (see scrapingdog.ts); callers may override
+ * it by passing `api_key`.
+ */
+import { z } from "zod";
+import { sdRequest } from "./scrapingdog.js";
+import { fromResponse, error } from "./result.js";
+/** Cast the loosely-typed SDK args into the stricter request-param map. */
+function toParams(args) {
+    return args;
+}
+// ---------------------------------------------------------------------------
+// Shared schema fragments
+// ---------------------------------------------------------------------------
+/** Common optional override for the API key. */
+const apiKeyParam = {
+    api_key: z.string().optional().describe("Override the configured Scrapingdog API key for this call. If omitted, the key is read from SCRAPINGDOG_API_KEY or a local .env file."),
+};
+// ---------------------------------------------------------------------------
+// 1. Web Scraping API  —  /scrape
+// ---------------------------------------------------------------------------
+const scrapeSchema = {
+    url: z.string().url().describe("The URL of the page to scrape."),
+    dynamic: z
+        .boolean()
+        .optional()
+        .describe("Enable JavaScript rendering with a headless browser. Costs 5 credits (25 with premium proxy)."),
+    premium: z
+        .boolean()
+        .optional()
+        .describe("Use premium residential proxies instead of the rotating datacenter proxy. Costs 10 credits (25 with JS rendering)."),
+    country: z
+        .string()
+        .optional()
+        .describe("ISO country code for geotargeting (e.g. 'us', 'gb', 'in'). Sends the request from that location. Costs 10 credits."),
+    session_number: z
+        .string()
+        .optional()
+        .describe("Reuse the same proxy/IP across multiple requests by passing a stable session string."),
+    custom_headers: z
+        .boolean()
+        .optional()
+        .describe("Set to true to forward your own request headers (passed via the headers parameter)."),
+    wait: z
+        .number()
+        .int()
+        .min(0)
+        .max(35000)
+        .optional()
+        .describe("Milliseconds to wait after JS rendering before capturing HTML (0–35000). Use with dynamic=true."),
+    stealth_mode: z
+        .boolean()
+        .optional()
+        .describe("Enable stealth mode to bypass bot detection / captchas."),
+    image: z.boolean().optional().describe("Scrape image URLs found on the page."),
+    ...apiKeyParam,
+};
+async function handleScrape(args) {
+    const result = await sdRequest({ path: "scrape", params: toParams(args) });
+    return fromResponse(result, { note: `Scraped HTML from ${args.url}.` });
+}
+// ---------------------------------------------------------------------------
+// 2. Screenshot API  —  /screenshot
+// ---------------------------------------------------------------------------
+const screenshotSchema = {
+    url: z.string().url().describe("The URL of the page to screenshot."),
+    fullPage: z
+        .boolean()
+        .optional()
+        .describe("Capture the full scrollable page rather than just the visible viewport."),
+    width: z
+        .string()
+        .optional()
+        .describe("Browser viewport width in pixels (e.g. '1920')."),
+    height: z.string().optional().describe("Browser viewport height in pixels (e.g. '1080')."),
+    wait_until: z
+        .enum(["load", "domcontentloaded", "networkidle"])
+        .optional()
+        .describe("When navigation is considered complete before screenshotting. Default: domcontentloaded."),
+    format: z
+        .enum(["png", "jpg", "webp"])
+        .optional()
+        .describe("Output image format. Default: png."),
+    quality: z
+        .number()
+        .int()
+        .min(0)
+        .max(100)
+        .optional()
+        .describe("Image quality 0–100 (jpg/webp). Default: 80."),
+    ...apiKeyParam,
+};
+async function handleScreenshot(args) {
+    const result = await sdRequest({ path: "screenshot", params: toParams(args) });
+    if (result.status >= 200 && result.status < 300) {
+        // The screenshot endpoint returns binary image bytes. We can't return raw
+        // binary as text cleanly via this tool, so we describe what happened and
+        // report the content type / length. For a true image, a host would base64
+        // it; here we surface status + size and the first bytes if it's text.
+        if (result.contentType.startsWith("image/")) {
+            return {
+                content: [
+                    {
+                        type: "text",
+                        text: `Screenshot captured (${result.contentType}, ${result.body.length} bytes). The image bytes were returned by Scrapingdog but are binary; this MCP tool returns text. To save the image, call the Scrapingdog /screenshot endpoint directly and write the response body to a file.`,
+                    },
+                ],
+            };
+        }
+        return fromResponse(result, { note: `Screenshot response for ${args.url}.` });
+    }
+    return error("Scrapingdog screenshot failed.", result);
+}
+// ---------------------------------------------------------------------------
+// 3. Google Search API  —  /google
+// ---------------------------------------------------------------------------
+const googleSearchSchema = {
+    query: z.string().describe("The Google search query. Supports operators like site:, inurl:, intitle:."),
+    results: z
+        .string()
+        .optional()
+        .describe("Number of results per page."),
+    page: z
+        .string()
+        .optional()
+        .describe("Page number: 0 = first page, 1 = second, etc. Default: 0."),
+    advance_search: z
+        .boolean()
+        .optional()
+        .describe("Get advanced feature snippets. true costs 10 credits, otherwise 5."),
+    mob_search: z
+        .boolean()
+        .optional()
+        .describe("Return mobile search results. true costs 10 credits."),
+    html: z.boolean().optional().describe("Return the full raw HTML of the Google results page."),
+    domain: z
+        .string()
+        .optional()
+        .describe("Google domain for localized results, e.g. google.co.in, google.co.uk. Default: google.com."),
+    country: z.string().optional().describe("Two-letter country code, e.g. 'us', 'uk', 'fr'. Default: us."),
+    language: z.string().optional().describe("Result language code, e.g. 'en', 'es', 'fr'. Default: en."),
+    location: z
+        .string()
+        .optional()
+        .describe("Origin location (city-level recommended). Cannot be used with uule."),
+    uule: z
+        .string()
+        .optional()
+        .describe("Encoded geographic locale for tailored results. Cannot be combined with location."),
+    tbs: z
+        .string()
+        .optional()
+        .describe("Advanced filter string (time ranges, verbatim mode) copied from Google's URL."),
+    safe: z
+        .enum(["active", "off"])
+        .optional()
+        .describe("Filter adult content: 'active' to filter, 'off' to disable. Default: off."),
+    ...apiKeyParam,
+};
+async function handleGoogleSearch(args) {
+    const result = await sdRequest({ path: "google", params: toParams(args) });
+    return fromResponse(result, { note: `Google search results for "${args.query}".` });
+}
+// ---------------------------------------------------------------------------
+// 4. Bing Search API  —  /bing/search
+// ---------------------------------------------------------------------------
+const bingSearchSchema = {
+    query: z.string().describe("The Bing search query. Supports NOT, OR, site:, filetype:, near:, etc."),
+    cc: z.string().optional().describe("Two-letter ISO country code, e.g. 'us', 'de', 'gb'."),
+    mkt: z
+        .string()
+        .optional()
+        .describe("Market as <language>-<country>, e.g. 'en-US'. Mutually exclusive with cc."),
+    lat: z.string().optional().describe("GPS latitude for the search starting point."),
+    lon: z.string().optional().describe("GPS longitude for the search starting point."),
+    first: z
+        .string()
+        .optional()
+        .describe("Starting position of organic results. Default: 1. first=10 puts the 10th result first."),
+    count: z
+        .string()
+        .optional()
+        .describe("Results per page, 1–50 (actual may vary)."),
+    safeSearch: z
+        .enum(["Off", "Moderate", "Strict"])
+        .optional()
+        .describe("Adult-content filter level."),
+    filters: z
+        .string()
+        .optional()
+        .describe("Advanced filter string copied from a Bing search URL."),
+    ...apiKeyParam,
+};
+async function handleBingSearch(args) {
+    const result = await sdRequest({ path: "bing/search", params: toParams(args) });
+    return fromResponse(result, { note: `Bing search results for "${args.query}".` });
+}
+// ---------------------------------------------------------------------------
+// 5. DuckDuckGo Search API  —  /duckduckgo/search
+// ---------------------------------------------------------------------------
+const duckduckgoSearchSchema = {
+    query: z.string().describe("The DuckDuckGo search query. Supports inurl:, site:, intitle: operators."),
+    kl: z
+        .string()
+        .optional()
+        .describe("Region, e.g. 'us-en', 'uk-en', 'fr-fr'."),
+    df: z
+        .string()
+        .optional()
+        .describe("Date filter: d (past day), w (week), m (month), y (year), or from_date..to_date."),
+    next_page_token: z
+        .string()
+        .optional()
+        .describe("Token from a previous response to fetch the next page (15 results per page)."),
+    html: z.boolean().optional().describe("Return the full raw HTML of the DuckDuckGo results page."),
+    ...apiKeyParam,
+};
+async function handleDuckDuckGoSearch(args) {
+    const result = await sdRequest({ path: "duckduckgo/search", params: toParams(args) });
+    return fromResponse(result, { note: `DuckDuckGo search results for "${args.query}".` });
+}
+// ---------------------------------------------------------------------------
+// 6. Baidu Search API  —  /baidu/search
+// ---------------------------------------------------------------------------
+const baiduSearchSchema = {
+    query: z.string().describe("The Baidu search query. Supports inurl:, site:, intitle: operators."),
+    ct: z
+        .number()
+        .int()
+        .optional()
+        .describe("Language restriction: 1 = all, 2 = Simplified Chinese, 3 = Traditional Chinese."),
+    pn: z
+        .string()
+        .optional()
+        .describe("Result offset for pagination. 0 = first page, 10 = second, 20 = third, etc."),
+    rn: z
+        .string()
+        .optional()
+        .describe("Maximum number of results to return."),
+    html: z.boolean().optional().describe("Return the full raw HTML of the Baidu results page."),
+    ...apiKeyParam,
+};
+async function handleBaiduSearch(args) {
+    const result = await sdRequest({ path: "baidu/search", params: toParams(args) });
+    return fromResponse(result, { note: `Baidu search results for "${args.query}".` });
+}
+// ---------------------------------------------------------------------------
+// 7. X (Twitter) Profile Scraper  —  /x/profile
+// ---------------------------------------------------------------------------
+const xProfileSchema = {
+    profileId: z
+        .string()
+        .describe("The user ID or username of the X profile to scrape, e.g. 'elonmusk' or 'nasa'."),
+    ...apiKeyParam,
+};
+async function handleXProfile(args) {
+    const result = await sdRequest({ path: "x/profile", params: toParams(args) });
+    return fromResponse(result, { note: `X (Twitter) profile data for "${args.profileId}".` });
+}
+// ---------------------------------------------------------------------------
+// 8. X (Twitter) Post Scraper  —  /x/post
+// ---------------------------------------------------------------------------
+const xPostSchema = {
+    tweetId: z
+        .string()
+        .describe("The tweet ID, found in the post URL (e.g. for .../status/1655608985058267139 the ID is 1655608985058267139)."),
+    ...apiKeyParam,
+};
+async function handleXPost(args) {
+    const result = await sdRequest({ path: "x/post", params: toParams(args) });
+    return fromResponse(result, { note: `X (Twitter) post data for tweet ${args.tweetId}.` });
+}
+// ---------------------------------------------------------------------------
+// 9. Datacenter Proxies configuration helper
+// ---------------------------------------------------------------------------
+const datacenterProxySchema = {
+    target_url: z
+        .string()
+        .url()
+        .optional()
+        .describe("Optional target URL to test the proxy against. If provided, a ready-to-use curl command is returned."),
+    ...apiKeyParam,
+};
+async function handleDatacenterProxy(args) {
+    // The datacenter proxy is not an HTTP API — it's a forward proxy you point
+    // your own HTTP client at. This tool returns the connection details and a
+    // ready-to-paste curl command so callers can use it directly.
+    const apiKey = args.api_key;
+    // We still resolve the key so callers get a clear error if none is configured,
+    // and so we can embed it in the example.
+    let resolvedKey = "";
+    try {
+        const { resolveApiKey } = await import("./scrapingdog.js");
+        resolvedKey = await resolveApiKey(apiKey);
+    }
+    catch (err) {
+        return error(err instanceof Error ? err.message : String(err));
+    }
+    const host = "proxy.scrapingdog.com";
+    const port = "8081";
+    const username = "scrapingdog";
+    const password = resolvedKey;
+    const lines = [
+        "Scrapingdog Datacenter Proxy configuration:",
+        "",
+        `  Host:     ${host}`,
+        `  Port:     ${port}`,
+        `  Username: ${username}`,
+        `  Password: ${password}   (your Scrapingdog API key)`,
+        "",
+        "Proxy URL: http://scrapingdog:" + password + "@" + host + ":" + port,
+        "",
+        "Example (curl):",
+        `  curl -x "http://scrapingdog:${password}@${host}:${port}" -k "https://httpbin.org/ip"`,
+        "",
+        "Example (Python requests):",
+        '  proxies = {"http": "http://scrapingdog:' + password + "@" + host + ":" + port + '",',
+        '              "https": "http://scrapingdog:' + password + "@" + host + ":" + port + '"}',
+        '  requests.get("https://httpbin.org/ip", proxies=proxies, verify=False)',
+    ];
+    if (args.target_url) {
+        lines.push("", "Ready-to-run curl for your target URL:", `  curl -x "http://scrapingdog:${password}@${host}:${port}" -k "${args.target_url}"`);
+    }
+    return { content: [{ type: "text", text: lines.join("\n") }] };
+}
+const tools = [
+    {
+        name: "scrape",
+        description: "Scrape any URL via the Scrapingdog Web Scraping API (/scrape). Returns the page's HTML. " +
+            "Supports JS rendering (dynamic), premium residential proxies, geotargeting (country), " +
+            "sessions, stealth/captcha bypass, and a render wait. Costs 1–25 credits depending on options.",
+        schema: scrapeSchema,
+        handler: handleScrape,
+    },
+    {
+        name: "screenshot",
+        description: "Capture a screenshot of a URL via the Scrapingdog Screenshot API (/screenshot). " +
+            "Supports full-page capture, viewport size, wait-until, format (png/jpg/webp), and quality. " +
+            "Costs 5 credits. Note: this tool reports image metadata; binary bytes aren't returned as text.",
+        schema: screenshotSchema,
+        handler: handleScreenshot,
+    },
+    {
+        name: "google_search",
+        description: "Run a Google search via Scrapingdog (/google). Returns organic results, ads, knowledge graph, " +
+            "and other SERP features as JSON (or raw HTML with html=true). Supports localization, " +
+            "pagination, advanced filters, and mobile results. Costs 5–10 credits.",
+        schema: googleSearchSchema,
+        handler: handleGoogleSearch,
+    },
+    {
+        name: "bing_search",
+        description: "Run a Bing search via Scrapingdog (/bing/search). Returns parsed Bing results as JSON. " +
+            "Supports market/country targeting, geo coordinates, pagination, and safe-search filters.",
+        schema: bingSearchSchema,
+        handler: handleBingSearch,
+    },
+    {
+        name: "duckduckgo_search",
+        description: "Run a DuckDuckGo search via Scrapingdog (/duckduckgo/search). Returns parsed results as JSON. " +
+            "Supports region (kl), date filters (df), and pagination via next_page_token (15 per page).",
+        schema: duckduckgoSearchSchema,
+        handler: handleDuckDuckGoSearch,
+    },
+    {
+        name: "baidu_search",
+        description: "Run a Baidu search via Scrapingdog (/baidu/search). Returns parsed results as JSON. " +
+            "Supports Chinese-language restriction (ct), pagination (pn), and result count (rn).",
+        schema: baiduSearchSchema,
+        handler: handleBaiduSearch,
+    },
+    {
+        name: "x_profile",
+        description: "Scrape an X (Twitter) profile via Scrapingdog (/x/profile). Returns the profile's name, handle, " +
+            "follower counts, bio, and other public metadata as JSON. Pass a username or user ID.",
+        schema: xProfileSchema,
+        handler: handleXProfile,
+    },
+    {
+        name: "x_post",
+        description: "Scrape an X (Twitter) post (tweet) via Scrapingdog (/x/post). Pass the numeric tweet ID from the post URL.",
+        schema: xPostSchema,
+        handler: handleXPost,
+    },
+    {
+        name: "datacenter_proxy",
+        description: "Return Scrapingdog Datacenter Proxy connection details (host, port, username, password) and a " +
+            "ready-to-paste curl/Python example. This is a forward proxy (proxy.scrapingdog.com:8081), not an " +
+            "HTTP API — point your own HTTP client at it. Optionally pass target_url for a ready-to-run curl.",
+        schema: datacenterProxySchema,
+        handler: handleDatacenterProxy,
+    },
+];
+/** Register every tool on the given MCP server. */
+export function registerTools(server) {
+    for (const tool of tools) {
+        server.tool(tool.name, tool.description, tool.schema, async (args) => {
+            try {
+                return await tool.handler(args);
+            }
+            catch (err) {
+                const message = err instanceof Error ? err.message : String(err);
+                return error(`Tool '${tool.name}' failed: ${message}`);
+            }
+        });
+    }
+}
+// Re-export for tests / external use.
+export { sdRequest };

package/package.json

@@ -0,0 +1,58 @@
+{
+  "name": "@0pen1/scrapingdog-mcp",
+  "version": "0.1.0",
+  "description": "MCP server for Scrapingdog — web scraping, screenshots, and Google/Bing/DuckDuckGo/Baidu search tools.",
+  "type": "module",
+  "license": "MIT",
+  "author": "0pen1",
+  "homepage": "https://github.com/0pen1/scrapingdog-mcp#readme",
+  "repository": {
+    "type": "git",
+    "url": "git+ssh://git@github.com:0pen1/scrapingdog-mcp.git"
+  },
+  "bugs": {
+    "url": "https://github.com/0pen1/scrapingdog-mcp/issues"
+  },
+  "keywords": [
+    "mcp",
+    "mcp-server",
+    "model-context-protocol",
+    "scrapingdog",
+    "web-scraping",
+    "serp",
+    "claude",
+    "claude-code",
+    "claude-desktop"
+  ],
+  "main": "dist/index.js",
+  "bin": {
+    "scrapingdog-mcp": "dist/index.js"
+  },
+  "files": [
+    "dist",
+    "README.md",
+    "LICENSE"
+  ],
+  "scripts": {
+    "build": "tsc",
+    "start": "node dist/index.js",
+    "dev": "tsc && node dist/index.js",
+    "prepublishOnly": "npm run build",
+    "prepack": "npm run build"
+  },
+  "engines": {
+    "node": ">=18.17"
+  },
+  "dependencies": {
+    "@modelcontextprotocol/sdk": "^1.12.1",
+    "zod": "^3.25.76"
+  },
+  "devDependencies": {
+    "@types/node": "^20.11.0",
+    "typescript": "^5.4.0"
+  },
+  "publishConfig": {
+    "access": "public",
+    "registry": "https://registry.npmjs.org/"
+  }
+}