@decodo/mcp-server 1.0.2 → 1.1.0

package/README.md

@@ -1,11 +1,11 @@
 # Decodo MCP Server
 
+[![](https://dcbadge.limes.pink/api/server/https://discord.gg/Ja8dqKgvbZ)](https://discord.gg/Ja8dqKgvbZ)
+[![Install MCP Server](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en-US/install-mcp?name=Decodo&config=eyJ1cmwiOiJodHRwczovL21jcC5kZWNvZG8uY29tL21jcCIsImhlYWRlcnMiOnsiQXV0aG9yaXphdGlvbiI6IkJhc2ljIDx3ZWJfYWR2YW5jZWRfdG9rZW4%2BIn19)
 [![smithery badge](https://smithery.ai/badge/@Decodo/decodo-mcp-server)](https://smithery.ai/server/@Decodo/decodo-mcp-server)
 
 <p align="center">
-<a href="https://dashboard.decodo.com/register?page=scrapers%2Fpricing&utm_source=github.com&utm_medium=social&utm_campaign=scraper_api_github"><img src="https://github.com/user-attachments/assets/13b08523-32b0-4c85-8e99-580d7c2a9055"></a>
-
-[![](https://dcbadge.vercel.app/api/server/Ja8dqKgvbZ)](https://discord.gg/Ja8dqKgvbZ)
+<a href="https://dashboard.decodo.com/integrations?utm_source=github&utm_medium=social&utm_campaign=mcp_server"> <img src="https://github.com/user-attachments/assets/a1e52a9e-3da1-4081-b3c6-053aafb8f196"/></a>
 
 This repository provides a Model Context Protocol (MCP) server that connects LLMs and applications
 to Decodo's platform. The server facilitates integration between MCP-compatible clients and Decodo's
@@ -20,23 +20,30 @@ services, streamlining access to our tools and capabilities.
 - Simple integration. Seamless setup with popular MCP clients like Claude Desktop, Cursor, and
   Windsurf.
 
-## Quick start via Smithery (Recommended)
+## Connecting to [Decodo's MCP server](https://mcp.decodo.com/mcp)
 
-Visit the `decodo-mcp-server` page on
-[Smithery](https://smithery.ai/server/@Decodo/decodo-mcp-server), select your preferred MCP client
-and generate installation instructions.
+1. Go to [decodo.com](https://decodo.com/scraping/web) and start a Web Scraping API plan (free
+   trials available).
 
-![Smithery interface](img/smithery.png 'Smithery UI')
+2. Once your plan has started, obtain a Web Scraping API basic authentication token from the
+   [dashboard](https://dashboard.decodo.com/).
 
-## Obtain Web Scraping API credentials
+3. Open your preferred MCP client and add the following configuration:
 
-You'll need the Decodo Advanced Web Scraping API credentials, which you can get by starting a free
-trial on the [dashboard](https://dashboard.decodo.com/). Once you have a plan activated, take a note
-of your generated username and password:
+```
+{
+  "Decodo": {
+    "url": "https://mcp.decodo.com/mcp",
+    "headers": {
+      "Authorization": "Basic <basic_auth_token>"
+    }
+  }
+}
+```
 
-![Decodo dashboard](img/auth.png 'Decodo dashboard')
+## Running the MCP server locally
 
-## Running the MCP server locally (manual)
+<details>
 
 ### Prerequisites
 
@@ -76,53 +83,83 @@ this:
 
 4. Update your MCP client with the server information:
 
-### Claude Desktop
+</details>
+
+## Toolsets
 
-[Follow the guide here](https://modelcontextprotocol.io/quickstart/user) to find the setup file,
-then update `claude_desktop_config.json` to look like this:
+Tools are organized into toolsets. You can selectively enable specific toolsets by passing a
+comma-separated list via the `toolsets` query parameter:
 
 ```
-{
-  "mcpServers": {
-    "decodo-mcp": {
-      "command": "node",
-      "args": ["/Users/your.user/projects/decodo-mcp/build/index.js"],
-      "env": {
-        "SCRAPER_API_USERNAME": "your_username",
-        "SCRAPER_API_PASSWORD": "your_password"
+    "Decodo MCP Server": {
+      "url": "https://mcp.decodo.com/mcp?toolsets=web,ai",
+      "headers": {
+        "Authorization": "Basic <your_auth_token>"
       }
     }
-  }
-}
-
 ```
 
-### Cursor
+When no toolsets are specified, all tools are registered.
 
-For installation instructions, see the
-[Cursor documentation](https://docs.cursor.com/context/model-context-protocol#installing-mcp-servers).
+| Toolset        | Tools                                                                                                                                                                               |
+| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `web`          | `scrape_as_markdown`, `screenshot`                                                                                                                                                  |
+| `search`       | `google_search`, `google_ads`, `google_lens`, `google_travel_hotels`, `bing_search`                                                                                                |
+| `ecommerce`    | `amazon_search`, `amazon_product`, `amazon_pricing`, `amazon_sellers`, `amazon_bestsellers`, `walmart_search`, `walmart_product`, `target_search`, `target_product`, `tiktok_shop_search`, `tiktok_shop_product`, `tiktok_shop_url` |
+| `social_media` | `reddit_post`, `reddit_subreddit`, `reddit_user`, `tiktok_post`, `youtube_video`, `youtube_metadata`, `youtube_channel`, `youtube_subtitles`, `youtube_search`                    |
+| `ai`           | `chatgpt`, `perplexity`, `google_ai_mode`                                                                                                                                           |
 
 ## Tools
 
 The server exposes the following tools:
 
-| Tool                   | Description                                                                                | Example prompt                                                   |
-| ---------------------- | ------------------------------------------------------------------------------------------ | ---------------------------------------------------------------- |
-| `scrape_as_markdown`   | Scrapes any target URL, expects a URL to be given via prompt. Returns results in Markdown. | Scrape peacock.com from a US IP address and tell me the pricing. |
-| `google_search_parsed` | Scrapes Google Search for a given query, and returns parsed results.                       | Scrape Google Search for shoes and tell me the top position.     |
-| `amazon_search_parsed` | Scrapes Amazon Search for a given query, and returns parsed results.                       | Scrape Amazon Search for toothbrushes                            |
+| Tool                    | Description                                                                                | Example prompt                                                                          |
+| ----------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- |
+| `scrape_as_markdown`    | Scrapes any target URL, expects a URL to be given via prompt. Returns results in Markdown. | Scrape peacock.com from a US IP address and tell me the pricing.                        |
+| `screenshot`            | Captures a screenshot of any webpage and returns it as a PNG image.                        | Take a screenshot of github.com from a US IP address.                                   |
+| `google_search`         | Scrapes Google Search for a given query, and returns parsed results.                       | Scrape Google Search for shoes and tell me the top position.                            |
+| `google_ads`            | Scrapes Google Ads search results with automatic parsing.                                  | Scrape Google Ads for laptop and show me the top ads.                                   |
+| `google_lens`           | Scrapes Google Lens image search results with automatic parsing.                           | Search Google Lens for this image: https://example.com/image.jpg                        |
+| `google_ai_mode`        | Scrapes Google AI Mode (Search with AI) results with automatic parsing.                    | Ask Google AI Mode: What are the top three dog breeds?                                  |
+| `google_travel_hotels`  | Scrapes Google Travel Hotels search results.                                               | Search Google Travel Hotels for hotels in Paris.                                        |
+| `amazon_search`         | Scrapes Amazon Search for a given query, and returns parsed results.                       | Scrape Amazon Search for wireless keyboard.                                             |
+| `amazon_product`        | Scrapes Amazon Product page with automatic parsing.                                        | Scrape Amazon product B09H74FXNW and show me the details.                               |
+| `amazon_pricing`        | Scrapes Amazon Product pricing information with automatic parsing.                         | Get pricing for Amazon product B09H74FXNW.                                              |
+| `amazon_sellers`        | Scrapes Amazon Seller information with automatic parsing.                                  | Get information about Amazon seller A1R0Z7FJGTKESH.                                     |
+| `amazon_bestsellers`    | Scrapes Amazon Bestsellers list with automatic parsing.                                    | Show me Amazon bestsellers in electronics.                                              |
+| `walmart_search`        | Scrapes Walmart Search for a given query, and returns parsed results.                      | Scrape Walmart Search for camping tent.                                                 |
+| `walmart_product`       | Scrapes Walmart Product page with automatic parsing.                                       | Scrape Walmart product 15296401808.                                                     |
+| `target_search`         | Scrapes Target Search for a given query, and returns parsed results.                       | Scrape Target Search for kitchen appliances.                                            |
+| `target_product`        | Scrapes Target Product page with automatic parsing.                                        | Scrape Target product 92186007.                                                         |
+| `tiktok_post`           | Scrapes a TikTok post URL for structured data (e.g. engagement, caption, hashtags).        | Scrape this TikTok post: https://www.tiktok.com/@nba/video/7393013274725403950          |
+| `tiktok_shop_search`    | Scrapes TikTok Shop Search for a given query, and returns parsed results.                  | Scrape TikTok Shop Search for phone cases.                                              |
+| `tiktok_shop_product`   | Scrapes TikTok Shop Product page.                                                          | Scrape TikTok Shop product 1731541214379741272.                                         |
+| `tiktok_shop_url`       | Scrapes TikTok Shop page by URL.                                                           | Scrape this TikTok Shop URL: https://www.tiktok.com/shop/s?q=HEADPHONES                 |
+| `youtube_video`         | Scrapes YouTube video information.                                                         | Scrape YouTube video 6Ejga4kJUts.                                                       |
+| `youtube_metadata`      | Scrapes YouTube video metadata.                                                            | Get metadata for YouTube video dFu9aKJoqGg.                                             |
+| `youtube_channel`       | Scrapes YouTube channel videos with automatic parsing.                                     | Scrape YouTube channel @decodo_official.                                                |
+| `youtube_subtitles`     | Scrapes YouTube video subtitles.                                                           | Get subtitles for YouTube video L8zSWbQN-v8.                                            |
+| `youtube_search`        | Search YouTube videos.                                                                     | Search YouTube for "How to care for chinchillas".                                       |
+| `reddit_post`           | Scrapes a specific Reddit post for a given query, and returns parsed results.              | Scrape the following Reddit post: https://www.reddit.com/r/horseracing/comments/1nsrn3/ |
+| `reddit_subreddit`      | Scrapes a specific Reddit subreddit for a given query, and returns parsed results.         | Scrape the top 5 posts on r/Python this week.                                           |
+| `reddit_user`           | Scrapes a Reddit user profile and their posts or comments.                                 | Scrape Reddit user u/spez's profile.                                                    |
+| `bing_search`           | Scrapes Bing Search results with automatic parsing.                                        | Search Bing for laptop reviews.                                                         |
+| `chatgpt`               | Search and interact with ChatGPT for AI-powered responses and conversations.               | Ask ChatGPT to explain quantum computing in simple terms.                               |
+| `perplexity`            | Search and interact with Perplexity for AI-powered responses and conversations.            | Ask Perplexity what the latest trends in web development are.                           |
 
 ## Parameters
 
 The following parameters are inferred from user prompts:
 
-| Parameter      | Description                                                                                          |
-| -------------- | ---------------------------------------------------------------------------------------------------- |
-| `jsRender`     | Renders target URL in a headless browser.                                                            |
-| `geo`          | Sets the country from which the request will originate.                                              |
-| `locale`       | Sets the locale of the request.                                                                      |
-| `tokenLimit`   | Truncates the response content up to this limit. Useful if the context window is small.              |
-| `fullResponse` | Skips automatic truncation and returns full content. If context window is small, may throw warnings. |
+| Parameter    | Description                                                                                           |
+| ------------ | ----------------------------------------------------------------------------------------------------- |
+| `jsRender`   | Renders target URL in a headless browser.                                                             |
+| `geo`        | Sets the country from which the request will originate.                                               |
+| `locale`     | Sets the locale of the request.                                                                       |
+| `tokenLimit` | Truncates the response content up to this limit. Useful if the context window is small.               |
+| `prompt`     | Prompt to send to AI tools (`chatgpt`, `perplexity`).                                                 |
+| `search`     | Activates ChatGPT's web search functionality (`chatgpt` only).                                        |
+| `xhr`        | When true, includes XHR or fetch responses in the scrape result where supported (e.g. `tiktok_post`). |
 
 ## Examples
 

package/build/clients/scraper-api-client.js

@@ -13,35 +13,97 @@ var __rest = (this && this.__rest) || function (s, e) {
 var __importDefault = (this && this.__importDefault) || function (mod) {
     return (mod && mod.__esModule) ? mod : { "default": mod };
 };
+var _a;
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.ScraperApiClient = void 0;
 const axios_1 = __importDefault(require("axios"));
+const MAX_RETRIES = Math.max(0, parseInt((_a = process.env.MAX_RETRIES) !== null && _a !== void 0 ? _a : '2', 10) || 2);
+const RETRYABLE_STATUS_CODES = new Set([429, 502, 503, 504]);
+const RETRYABLE_NETWORK_CODES = new Set(['ECONNRESET', 'ETIMEDOUT', 'ECONNABORTED', 'ENOTFOUND']);
+const isRetryable = (error) => {
+    var _a;
+    if (error.response) {
+        return RETRYABLE_STATUS_CODES.has(error.response.status);
+    }
+    return RETRYABLE_NETWORK_CODES.has((_a = error.code) !== null && _a !== void 0 ? _a : '');
+};
+const getRetryDelay = (attempt, error) => {
+    var _a;
+    if (((_a = error.response) === null || _a === void 0 ? void 0 : _a.status) === 429) {
+        const retryAfter = error.response.headers['retry-after'];
+        if (retryAfter) {
+            const seconds = Number(retryAfter);
+            if (!isNaN(seconds)) {
+                return seconds * 1000;
+            }
+            const date = Date.parse(retryAfter);
+            if (!isNaN(date)) {
+                return Math.max(0, date - Date.now());
+            }
+        }
+    }
+    const baseMs = 1000 * Math.pow(2, attempt);
+    const jitterMs = Math.random() * 500;
+    return baseMs + jitterMs;
+};
+const sleep = (ms) => new Promise(resolve => setTimeout(resolve, ms));
 class ScraperApiClient {
-    constructor({ auth }) {
+    constructor() {
         this.transformScrapingParams = ({ scrapingParams, }) => {
-            const { jsRender } = scrapingParams, rest = __rest(scrapingParams, ["jsRender"]);
-            const transformed = Object.assign(Object.assign({}, (jsRender && { headless: 'html' })), rest);
+            const { jsRender, headless } = scrapingParams, rest = __rest(scrapingParams, ["jsRender", "headless"]);
+            const transformed = Object.assign(Object.assign({}, (headless ? { headless } : jsRender && { headless: 'html' })), rest);
             return transformed;
         };
         this.transformResponse = ({ res }) => {
             const content = res.data.results[0].content;
             return Object.assign(Object.assign({}, res), { data: content });
         };
-        this.scrape = async ({ scrapingParams }) => {
+        this.scrape = async ({ auth, scrapingParams, }) => {
+            var _a, _b, _c, _d, _e;
             const transformedParams = this.transformScrapingParams({ scrapingParams });
-            const res = await axios_1.default.request({
-                url: 'https://scraper-api.decodo.com/v2/scrape',
-                method: 'POST',
-                headers: {
-                    authorization: `Basic ${this.auth}`,
-                    'x-integration': 'mcp',
-                },
-                data: Object.assign({}, transformedParams),
-            });
-            const response = this.transformResponse({ res });
-            return response;
+            const url = process.env.DECODO_SAPI_HOST || 'https://scraper-api.decodo.com';
+            let lastError;
+            for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
+                try {
+                    const res = await axios_1.default.request({
+                        url: `${url}/v2/scrape`,
+                        method: 'POST',
+                        headers: {
+                            authorization: `Basic ${auth}`,
+                            'x-integration': 'mcp',
+                        },
+                        timeout: 180000,
+                        data: Object.assign({}, transformedParams),
+                    });
+                    return this.transformResponse({ res });
+                }
+                catch (error) {
+                    lastError = error;
+                    if (attempt < MAX_RETRIES && axios_1.default.isAxiosError(error) && isRetryable(error)) {
+                        const delayMs = getRetryDelay(attempt, error);
+                        const reason = error.response
+                            ? `status ${error.response.status}`
+                            : `network error ${error.code}`;
+                        console.error(`[scraper-api-client] Retry ${attempt + 1}/${MAX_RETRIES} after ${reason}, waiting ${Math.round(delayMs)}ms`);
+                        await sleep(delayMs);
+                        continue;
+                    }
+                    break;
+                }
+            }
+            if (axios_1.default.isAxiosError(lastError)) {
+                const status = (_a = lastError.response) === null || _a === void 0 ? void 0 : _a.status;
+                let errorMessage = (_d = (_c = (_b = lastError.response) === null || _b === void 0 ? void 0 : _b.data) === null || _c === void 0 ? void 0 : _c.message) !== null && _d !== void 0 ? _d : lastError.message;
+                if (status === 401) {
+                    errorMessage = 'Authentication failed.';
+                }
+                if (status === 429) {
+                    errorMessage = JSON.stringify((_e = lastError.response) === null || _e === void 0 ? void 0 : _e.data);
+                }
+                throw new Error(`Scraper API request failed (${status}): ${errorMessage}`);
+            }
+            throw lastError;
         };
-        this.auth = auth;
     }
 }
 exports.ScraperApiClient = ScraperApiClient;

package/build/clients/scraper-api-client.js.map

@@ -1 +1 @@
-{"version":3,"file":"scraper-api-client.js","sourceRoot":"","sources":["../../src/clients/scraper-api-client.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;AAAA,kDAA6C;AAI7C,MAAa,gBAAgB;IAG3B,YAAY,EAAE,IAAI,EAAoB;QAItC,4BAAuB,GAAG,CAAC,EACzB,cAAc,GAGf,EAAoB,EAAE;YACrB,MAAM,EAAE,QAAQ,KAAc,cAAc,EAAvB,IAAI,UAAK,cAAc,EAAtC,YAAqB,CAAiB,CAAC;YAC7C,MAAM,WAAW,mCAAQ,CAAC,QAAQ,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,CAAC,GAAK,IAAI,CAAE,CAAC;YAEvE,OAAO,WAAW,CAAC;QACrB,CAAC,CAAC;QAEF,sBAAiB,GAAG,CAAI,EAAE,GAAG,EAAqD,EAAE,EAAE;YACpF,MAAM,OAAO,GAAG,GAAG,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC;YAE5C,uCAAY,GAAG,KAAE,IAAI,EAAE,OAAO,IAAG;QACnC,CAAC,CAAC;QAEF,WAAM,GAAG,KAAK,EAAc,EAAE,cAAc,EAAyC,EAAE,EAAE;YACvF,MAAM,iBAAiB,GAAG,IAAI,CAAC,uBAAuB,CAAC,EAAE,cAAc,EAAE,CAAC,CAAC;YAE3E,MAAM,GAAG,GAAG,MAAM,eAAK,CAAC,OAAO,CAA4B;gBACzD,GAAG,EAAE,0CAA0C;gBAC/C,MAAM,EAAE,MAAM;gBACd,OAAO,EAAE;oBACP,aAAa,EAAE,SAAS,IAAI,CAAC,IAAI,EAAE;oBACnC,eAAe,EAAE,KAAK;iBACvB;gBACD,IAAI,oBACC,iBAAiB,CACrB;aACF,CAAC,CAAC;YAEH,MAAM,QAAQ,GAAG,IAAI,CAAC,iBAAiB,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC;YAEjD,OAAO,QAAQ,CAAC;QAClB,CAAC,CAAC;QAtCA,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;IACnB,CAAC;CAsCF;AA3CD,4CA2CC"}
+{"version":3,"file":"scraper-api-client.js","sourceRoot":"","sources":["../../src/clients/scraper-api-client.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;AAAA,kDAAyD;AAIzD,MAAM,WAAW,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,QAAQ,CAAC,MAAA,OAAO,CAAC,GAAG,CAAC,WAAW,mCAAI,GAAG,EAAE,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC;AACnF,MAAM,sBAAsB,GAAG,IAAI,GAAG,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC,CAAC,CAAC;AAC7D,MAAM,uBAAuB,GAAG,IAAI,GAAG,CAAC,CAAC,YAAY,EAAE,WAAW,EAAE,cAAc,EAAE,WAAW,CAAC,CAAC,CAAC;AAElG,MAAM,WAAW,GAAG,CAAC,KAAiB,EAAW,EAAE;;IACjD,IAAI,KAAK,CAAC,QAAQ,EAAE,CAAC;QACnB,OAAO,sBAAsB,CAAC,GAAG,CAAC,KAAK,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC;IAC3D,CAAC;IACD,OAAO,uBAAuB,CAAC,GAAG,CAAC,MAAA,KAAK,CAAC,IAAI,mCAAI,EAAE,CAAC,CAAC;AACvD,CAAC,CAAC;AAEF,MAAM,aAAa,GAAG,CAAC,OAAe,EAAE,KAAiB,EAAU,EAAE;;IACnE,IAAI,CAAA,MAAA,KAAK,CAAC,QAAQ,0CAAE,MAAM,MAAK,GAAG,EAAE,CAAC;QACnC,MAAM,UAAU,GAAG,KAAK,CAAC,QAAQ,CAAC,OAAO,CAAC,aAAa,CAAC,CAAC;QACzD,IAAI,UAAU,EAAE,CAAC;YACf,MAAM,OAAO,GAAG,MAAM,CAAC,UAAU,CAAC,CAAC;YACnC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,EAAE,CAAC;gBACpB,OAAO,OAAO,GAAG,IAAI,CAAC;YACxB,CAAC;YAED,MAAM,IAAI,GAAG,IAAI,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC;YACpC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC;gBACjB,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC;YACxC,CAAC;QACH,CAAC;IACH,CAAC;IAED,MAAM,MAAM,GAAG,IAAI,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,OAAO,CAAC,CAAC;IAC3C,MAAM,QAAQ,GAAG,IAAI,CAAC,MAAM,EAAE,GAAG,GAAG,CAAC;IACrC,OAAO,MAAM,GAAG,QAAQ,CAAC;AAC3B,CAAC,CAAC;AAEF,MAAM,KAAK,GAAG,CAAC,EAAU,EAAiB,EAAE,CAAC,IAAI,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC,UAAU,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC,CAAC;AAE7F,MAAa,gBAAgB;IAA7B;QACE,4BAAuB,GAAG,CAAC,EACzB,cAAc,GAGf,EAAoB,EAAE;YACrB,MAAM,EAAE,QAAQ,EAAE,QAAQ,KAAc,cAAc,EAAvB,IAAI,UAAK,cAAc,EAAhD,wBAA+B,CAAiB,CAAC;YACvD,MAAM,WAAW,mCACZ,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,CAAC,CAAC,CAAC,QAAQ,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,CAAC,GAC5D,IAAI,CACR,CAAC;YAEF,OAAO,WAAW,CAAC;QACrB,CAAC,CAAC;QAEF,sBAAiB,GAAG,CAAI,EAAE,GAAG,EAAqD,EAAE,EAAE;YACpF,MAAM,OAAO,GAAG,GAAG,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC;YAE5C,uCAAY,GAAG,KAAE,IAAI,EAAE,OAAO,IAAG;QACnC,CAAC,CAAC;QAEF,WAAM,GAAG,KAAK,EAAc,EAC1B,IAAI,EACJ,cAAc,GAIf,EAAE,EAAE;;YACH,MAAM,iBAAiB,GAAG,IAAI,CAAC,uBAAuB,CAAC,EAAE,cAAc,EAAE,CAAC,CAAC;YAE3E,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC,gBAAgB,IAAI,gCAAgC,CAAC;YAE7E,IAAI,SAAkB,CAAC;YAEvB,KAAK,IAAI,OAAO,GAAG,CAAC,EAAE,OAAO,IAAI,WAAW,EAAE,OAAO,EAAE,EAAE,CAAC;gBACxD,IAAI,CAAC;oBACH,MAAM,GAAG,GAAG,MAAM,eAAK,CAAC,OAAO,CAA4B;wBACzD,GAAG,EAAE,GAAG,GAAG,YAAY;wBACvB,MAAM,EAAE,MAAM;wBACd,OAAO,EAAE;4BACP,aAAa,EAAE,SAAS,IAAI,EAAE;4BAC9B,eAAe,EAAE,KAAK;yBACvB;wBACD,OAAO,EAAE,MAAM;wBACf,IAAI,oBACC,iBAAiB,CACrB;qBACF,CAAC,CAAC;oBAEH,OAAO,IAAI,CAAC,iBAAiB,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC;gBACzC,CAAC;gBAAC,OAAO,KAAK,EAAE,CAAC;oBACf,SAAS,GAAG,KAAK,CAAC;oBAElB,IAAI,OAAO,GAAG,WAAW,IAAI,eAAK,CAAC,YAAY,CAAC,KAAK,CAAC,IAAI,WAAW,CAAC,KAAK,CAAC,EAAE,CAAC;wBAC7E,MAAM,OAAO,GAAG,aAAa,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC;wBAC9C,MAAM,MAAM,GAAG,KAAK,CAAC,QAAQ;4BAC3B,CAAC,CAAC,UAAU,KAAK,CAAC,QAAQ,CAAC,MAAM,EAAE;4BACnC,CAAC,CAAC,iBAAiB,KAAK,CAAC,IAAI,EAAE,CAAC;wBAElC,OAAO,CAAC,KAAK,CACX,8BACE,OAAO,GAAG,CACZ,IAAI,WAAW,UAAU,MAAM,aAAa,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,IAAI,CACpE,CAAC;wBAEF,MAAM,KAAK,CAAC,OAAO,CAAC,CAAC;wBACrB,SAAS;oBACX,CAAC;oBAED,MAAM;gBACR,CAAC;YACH,CAAC;YAED,IAAI,eAAK,CAAC,YAAY,CAAC,SAAS,CAAC,EAAE,CAAC;gBAClC,MAAM,MAAM,GAAG,MAAA,SAAS,CAAC,QAAQ,0CAAE,MAAM,CAAC;gBAC1C,IAAI,YAAY,GAAG,MAAA,MAAA,MAAA,SAAS,CAAC,QAAQ,0CAAE,IAAI,0CAAE,OAAO,mCAAI,SAAS,CAAC,OAAO,CAAC;gBAE1E,IAAI,MAAM,KAAK,GAAG,EAAE,CAAC;oBACnB,YAAY,GAAG,wBAAwB,CAAC;gBAC1C,CAAC;gBACD,IAAI,MAAM,KAAK,GAAG,EAAE,CAAC;oBACnB,YAAY,GAAG,IAAI,CAAC,SAAS,CAAC,MAAA,SAAS,CAAC,QAAQ,0CAAE,IAAI,CAAC,CAAC;gBAC1D,CAAC;gBAED,MAAM,IAAI,KAAK,CAAC,+BAA+B,MAAM,MAAM,YAAY,EAAE,CAAC,CAAC;YAC7E,CAAC;YACD,MAAM,SAAS,CAAC;QAClB,CAAC,CAAC;IACJ,CAAC;CAAA;AAxFD,4CAwFC"}

package/build/constants.js

@@ -1,11 +1,45 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.SCRAPER_API_TARGETS = void 0;
+exports.SCRAPER_API_TARGETS = exports.TOOLSET = void 0;
+var TOOLSET;
+(function (TOOLSET) {
+    TOOLSET["WEB"] = "web";
+    TOOLSET["SEARCH"] = "search";
+    TOOLSET["ECOMMERCE"] = "ecommerce";
+    TOOLSET["SOCIAL_MEDIA"] = "social_media";
+    TOOLSET["AI"] = "ai";
+})(TOOLSET || (exports.TOOLSET = TOOLSET = {}));
+// todo: utils
 var SCRAPER_API_TARGETS;
 (function (SCRAPER_API_TARGETS) {
     SCRAPER_API_TARGETS["GOOGLE_SEARCH"] = "google_search";
+    SCRAPER_API_TARGETS["GOOGLE_TRAVEL_HOTELS"] = "google_travel_hotels";
+    SCRAPER_API_TARGETS["GOOGLE_ADS"] = "google_ads";
+    SCRAPER_API_TARGETS["GOOGLE_LENS"] = "google_lens";
+    SCRAPER_API_TARGETS["GOOGLE_AI_MODE"] = "google_ai_mode";
     SCRAPER_API_TARGETS["AMAZON_SEARCH"] = "amazon_search";
+    SCRAPER_API_TARGETS["AMAZON_PRODUCT"] = "amazon_product";
+    SCRAPER_API_TARGETS["AMAZON_PRICING"] = "amazon_pricing";
+    SCRAPER_API_TARGETS["AMAZON_SELLERS"] = "amazon_sellers";
+    SCRAPER_API_TARGETS["AMAZON_BESTSELLERS"] = "amazon_bestsellers";
+    SCRAPER_API_TARGETS["WALMART_SEARCH"] = "walmart_search";
+    SCRAPER_API_TARGETS["WALMART_PRODUCT"] = "walmart_product";
+    SCRAPER_API_TARGETS["TARGET_SEARCH"] = "target_search";
+    SCRAPER_API_TARGETS["TARGET_PRODUCT"] = "target_product";
+    SCRAPER_API_TARGETS["TIKTOK_POST"] = "tiktok_post";
+    SCRAPER_API_TARGETS["TIKTOK_SHOP_SEARCH"] = "tiktok_shop_search";
+    SCRAPER_API_TARGETS["TIKTOK_SHOP_PRODUCT"] = "tiktok_shop_product";
+    SCRAPER_API_TARGETS["TIKTOK_SHOP_URL"] = "tiktok";
+    SCRAPER_API_TARGETS["YOUTUBE_VIDEO"] = "youtube_video";
+    SCRAPER_API_TARGETS["YOUTUBE_METADATA"] = "youtube_metadata";
+    SCRAPER_API_TARGETS["YOUTUBE_CHANNEL"] = "youtube_channel";
+    SCRAPER_API_TARGETS["YOUTUBE_SUBTITLES"] = "youtube_subtitles";
+    SCRAPER_API_TARGETS["YOUTUBE_SEARCH"] = "youtube_search";
     SCRAPER_API_TARGETS["REDDIT_POST"] = "reddit_post";
     SCRAPER_API_TARGETS["REDDIT_SUBREDDIT"] = "reddit_subreddit";
+    SCRAPER_API_TARGETS["REDDIT_USER"] = "reddit_user";
+    SCRAPER_API_TARGETS["BING_SEARCH"] = "bing_search";
+    SCRAPER_API_TARGETS["CHATGPT"] = "chatgpt";
+    SCRAPER_API_TARGETS["PERPLEXITY"] = "perplexity";
 })(SCRAPER_API_TARGETS || (exports.SCRAPER_API_TARGETS = SCRAPER_API_TARGETS = {}));
 //# sourceMappingURL=constants.js.map

package/build/constants.js.map

@@ -1 +1 @@
-{"version":3,"file":"constants.js","sourceRoot":"","sources":["../src/constants.ts"],"names":[],"mappings":";;;AAAA,IAAY,mBAMX;AAND,WAAY,mBAAmB;IAC7B,sDAA+B,CAAA;IAC/B,sDAA+B,CAAA;IAE/B,kDAA2B,CAAA;IAC3B,4DAAqC,CAAA;AACvC,CAAC,EANW,mBAAmB,mCAAnB,mBAAmB,QAM9B"}
+{"version":3,"file":"constants.js","sourceRoot":"","sources":["../src/constants.ts"],"names":[],"mappings":";;;AAAA,IAAY,OAMX;AAND,WAAY,OAAO;IACjB,sBAAW,CAAA;IACX,4BAAiB,CAAA;IACjB,kCAAuB,CAAA;IACvB,wCAA6B,CAAA;IAC7B,oBAAS,CAAA;AACX,CAAC,EANW,OAAO,uBAAP,OAAO,QAMlB;AAED,cAAc;AACd,IAAY,mBAsCX;AAtCD,WAAY,mBAAmB;IAC7B,sDAA+B,CAAA;IAC/B,oEAA6C,CAAA;IAC7C,gDAAyB,CAAA;IACzB,kDAA2B,CAAA;IAC3B,wDAAiC,CAAA;IAEjC,sDAA+B,CAAA;IAC/B,wDAAiC,CAAA;IACjC,wDAAiC,CAAA;IACjC,wDAAiC,CAAA;IACjC,gEAAyC,CAAA;IAEzC,wDAAiC,CAAA;IACjC,0DAAmC,CAAA;IAEnC,sDAA+B,CAAA;IAC/B,wDAAiC,CAAA;IAEjC,kDAA2B,CAAA;IAC3B,gEAAyC,CAAA;IACzC,kEAA2C,CAAA;IAC3C,iDAA0B,CAAA;IAE1B,sDAA+B,CAAA;IAC/B,4DAAqC,CAAA;IACrC,0DAAmC,CAAA;IACnC,8DAAuC,CAAA;IACvC,wDAAiC,CAAA;IAEjC,kDAA2B,CAAA;IAC3B,4DAAqC,CAAA;IACrC,kDAA2B,CAAA;IAE3B,kDAA2B,CAAA;IAE3B,0CAAmB,CAAA;IACnB,gDAAyB,CAAA;AAC3B,CAAC,EAtCW,mBAAmB,mCAAnB,mBAAmB,QAsC9B"}

package/build/index.js

@@ -1,17 +1,15 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 require("dotenv/config");
-const process_1 = require("process");
 const stdio_js_1 = require("@modelcontextprotocol/sdk/server/stdio.js");
-const sapi_mcp_server_1 = require("./sapi-mcp-server");
-if (process.env.ENABLE_MCPS_LOGGER) {
-    import('mcps-logger/console');
-}
+const sapi_stdio_server_1 = require("./server/sapi-stdio-server");
+const utils_1 = require("./utils");
 const parseEnvsOrExit = () => {
     const envs = ['SCRAPER_API_USERNAME', 'SCRAPER_API_PASSWORD'];
     for (const envKey of envs) {
         if (!process.env[envKey]) {
-            (0, process_1.exit)(`env ${envKey} missing`);
+            console.error(`env ${envKey} missing`);
+            process.exit(1);
         }
     }
     return {
@@ -19,17 +17,18 @@ const parseEnvsOrExit = () => {
         sapiPassword: process.env['SCRAPER_API_PASSWORD'],
     };
 };
-async function main() {
+const main = async () => {
     const transport = new stdio_js_1.StdioServerTransport();
     // if there are no envs, some MCP clients will fail silently
     const { sapiUsername, sapiPassword } = parseEnvsOrExit();
-    const sapiMcpServer = new sapi_mcp_server_1.ScraperAPIMCPServer({
-        sapiUsername,
-        sapiPassword,
+    const auth = Buffer.from(`${sapiUsername}:${sapiPassword}`).toString('base64');
+    const sapiMcpServer = new sapi_stdio_server_1.ScraperAPIStdioServer({
+        auth,
+        toolsets: (0, utils_1.resolveToolsets)(process.env.TOOLSETS),
     });
     await sapiMcpServer.connect(transport);
     console.error('MCP Server running on stdio');
-}
+};
 main().catch(error => {
     console.error('Fatal error in main():', error);
     process.exit(1);

package/build/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";;AAAA,yBAAuB;AAEvB,qCAA+B;AAC/B,wEAAiF;AACjF,uDAAwD;AAExD,IAAI,OAAO,CAAC,GAAG,CAAC,kBAAkB,EAAE,CAAC;IACnC,MAAM,CAAC,qBAAqB,CAAC,CAAC;AAChC,CAAC;AAED,MAAM,eAAe,GAAG,GAA2B,EAAE;IACnD,MAAM,IAAI,GAAG,CAAC,sBAAsB,EAAE,sBAAsB,CAAC,CAAC;IAE9D,KAAK,MAAM,MAAM,IAAI,IAAI,EAAE,CAAC;QAC1B,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC;YACzB,IAAA,cAAI,EAAC,OAAO,MAAM,UAAU,CAAC,CAAC;QAChC,CAAC;IACH,CAAC;IAED,OAAO;QACL,YAAY,EAAE,OAAO,CAAC,GAAG,CAAC,sBAAsB,CAAW;QAC3D,YAAY,EAAE,OAAO,CAAC,GAAG,CAAC,sBAAsB,CAAW;KAC5D,CAAC;AACJ,CAAC,CAAC;AAEF,KAAK,UAAU,IAAI;IACjB,MAAM,SAAS,GAAG,IAAI,+BAAoB,EAAE,CAAC;IAE7C,4DAA4D;IAC5D,MAAM,EAAE,YAAY,EAAE,YAAY,EAAE,GAAG,eAAe,EAAE,CAAC;IAEzD,MAAM,aAAa,GAAG,IAAI,qCAAmB,CAAC;QAC5C,YAAY;QACZ,YAAY;KACb,CAAC,CAAC;IACH,MAAM,aAAa,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IAEvC,OAAO,CAAC,KAAK,CAAC,6BAA6B,CAAC,CAAC;AAC/C,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE;IACnB,OAAO,CAAC,KAAK,CAAC,wBAAwB,EAAE,KAAK,CAAC,CAAC;IAC/C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";;AAAA,yBAAuB;AAEvB,wEAAiF;AACjF,kEAAmE;AACnE,mCAA0C;AAE1C,MAAM,eAAe,GAAG,GAA2B,EAAE;IACnD,MAAM,IAAI,GAAG,CAAC,sBAAsB,EAAE,sBAAsB,CAAC,CAAC;IAE9D,KAAK,MAAM,MAAM,IAAI,IAAI,EAAE,CAAC;QAC1B,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC;YACzB,OAAO,CAAC,KAAK,CAAC,OAAO,MAAM,UAAU,CAAC,CAAC;YACvC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,CAAC;IACH,CAAC;IAED,OAAO;QACL,YAAY,EAAE,OAAO,CAAC,GAAG,CAAC,sBAAsB,CAAW;QAC3D,YAAY,EAAE,OAAO,CAAC,GAAG,CAAC,sBAAsB,CAAW;KAC5D,CAAC;AACJ,CAAC,CAAC;AAEF,MAAM,IAAI,GAAG,KAAK,IAAI,EAAE;IACtB,MAAM,SAAS,GAAG,IAAI,+BAAoB,EAAE,CAAC;IAE7C,4DAA4D;IAC5D,MAAM,EAAE,YAAY,EAAE,YAAY,EAAE,GAAG,eAAe,EAAE,CAAC;IAEzD,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,GAAG,YAAY,IAAI,YAAY,EAAE,CAAC,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC;IAE/E,MAAM,aAAa,GAAG,IAAI,yCAAqB,CAAC;QAC9C,IAAI;QACJ,QAAQ,EAAE,IAAA,uBAAe,EAAC,OAAO,CAAC,GAAG,CAAC,QAAQ,CAAC;KAChD,CAAC,CAAC;IACH,MAAM,aAAa,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IAEvC,OAAO,CAAC,KAAK,CAAC,6BAA6B,CAAC,CAAC;AAC/C,CAAC,CAAC;AAEF,IAAI,EAAE,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE;IACnB,OAAO,CAAC,KAAK,CAAC,wBAAwB,EAAE,KAAK,CAAC,CAAC;IAC/C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"}

package/build/server/sapi-base-server.js

@@ -0,0 +1,76 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.ScraperAPIBaseServer = void 0;
+const mcp_js_1 = require("@modelcontextprotocol/sdk/server/mcp.js");
+const scraper_api_client_1 = require("../clients/scraper-api-client");
+const tools_1 = require("../tools");
+class ScraperAPIBaseServer {
+    constructor({ auth, toolsets = [] }) {
+        this.auth = '';
+        this.server = new mcp_js_1.McpServer({
+            name: 'decodo',
+            version: '1.1.0',
+        });
+        this.sapiClient = new scraper_api_client_1.ScraperApiClient();
+        this.auth = auth;
+        this.registerTools({ toolsets });
+        this.registerResources();
+    }
+    connect(transport) {
+        this.server.connect(transport);
+    }
+    registerTools({ toolsets }) {
+        if (toolsets.length === 0) {
+            this.registerAllTools();
+            return;
+        }
+        for (const toolset of toolsets) {
+            const tools = ScraperAPIBaseServer.allTools.filter(tool => tool.toolset === toolset);
+            for (const tool of tools) {
+                tool.register({ server: this.server, sapiClient: this.sapiClient, auth: this.auth });
+            }
+        }
+    }
+    registerAllTools() {
+        for (const tool of ScraperAPIBaseServer.allTools) {
+            tool.register({ server: this.server, sapiClient: this.sapiClient, auth: this.auth });
+        }
+    }
+    registerResources() {
+        // todo: expose info about all targets available
+    }
+}
+exports.ScraperAPIBaseServer = ScraperAPIBaseServer;
+ScraperAPIBaseServer.allTools = [
+    new tools_1.ScrapeAsMarkdownTool(),
+    new tools_1.ScreenshotTool(),
+    new tools_1.GoogleSearchTool(),
+    new tools_1.GoogleAdsTool(),
+    new tools_1.GoogleLensTool(),
+    new tools_1.GoogleAiModeTool(),
+    new tools_1.GoogleTravelHotelsTool(),
+    new tools_1.AmazonSearchTool(),
+    new tools_1.AmazonProductTool(),
+    new tools_1.AmazonPricingTool(),
+    new tools_1.AmazonSellersTool(),
+    new tools_1.AmazonBestsellersTool(),
+    new tools_1.WalmartSearchTool(),
+    new tools_1.WalmartProductTool(),
+    new tools_1.TargetSearchTool(),
+    new tools_1.TargetProductTool(),
+    new tools_1.TiktokPostTool(),
+    new tools_1.TiktokShopSearchTool(),
+    new tools_1.TiktokShopProductTool(),
+    new tools_1.TiktokShopUrlTool(),
+    new tools_1.YoutubeMetadataTool(),
+    new tools_1.YoutubeChannelTool(),
+    new tools_1.YoutubeSubtitlesTool(),
+    new tools_1.YoutubeSearchTool(),
+    new tools_1.RedditPostTool(),
+    new tools_1.RedditSubredditTool(),
+    new tools_1.RedditUserTool(),
+    new tools_1.BingSearchTool(),
+    new tools_1.ChatGPTTool(),
+    new tools_1.PerplexityTool(),
+];
+//# sourceMappingURL=sapi-base-server.js.map

package/build/server/sapi-base-server.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"sapi-base-server.js","sourceRoot":"","sources":["../../src/server/sapi-base-server.ts"],"names":[],"mappings":";;;AAAA,oEAAoE;AAEpE,sEAAiE;AACjE,oCA+BkB;AAKlB,MAAa,oBAAoB;IAO/B,YAAY,EAAE,IAAI,EAAE,QAAQ,GAAG,EAAE,EAAyC;QAF1E,SAAI,GAAW,EAAE,CAAC;QAGhB,IAAI,CAAC,MAAM,GAAG,IAAI,kBAAS,CAAC;YAC1B,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,OAAO;SACjB,CAAC,CAAC;QACH,IAAI,CAAC,UAAU,GAAG,IAAI,qCAAgB,EAAE,CAAC;QAEzC,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QAEjB,IAAI,CAAC,aAAa,CAAC,EAAE,QAAQ,EAAE,CAAC,CAAC;QAEjC,IAAI,CAAC,iBAAiB,EAAE,CAAC;IAC3B,CAAC;IAED,OAAO,CAAC,SAA+D;QACrE,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IACjC,CAAC;IAmCD,aAAa,CAAC,EAAE,QAAQ,EAA2B;QACjD,IAAI,QAAQ,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC1B,IAAI,CAAC,gBAAgB,EAAE,CAAC;YACxB,OAAO;QACT,CAAC;QAED,KAAK,MAAM,OAAO,IAAI,QAAQ,EAAE,CAAC;YAC/B,MAAM,KAAK,GAAG,oBAAoB,CAAC,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC,OAAO,KAAK,OAAO,CAAC,CAAC;YACrF,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;gBACzB,IAAI,CAAC,QAAQ,CAAC,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,UAAU,EAAE,IAAI,CAAC,UAAU,EAAE,IAAI,EAAE,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC;YACvF,CAAC;QACH,CAAC;IACH,CAAC;IAED,gBAAgB;QACd,KAAK,MAAM,IAAI,IAAI,oBAAoB,CAAC,QAAQ,EAAE,CAAC;YACjD,IAAI,CAAC,QAAQ,CAAC,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,UAAU,EAAE,IAAI,CAAC,UAAU,EAAE,IAAI,EAAE,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC;QACvF,CAAC;IACH,CAAC;IAED,iBAAiB;QACf,gDAAgD;IAClD,CAAC;;AAhFH,oDAiFC;AAxDQ,6BAAQ,GAAW;IACxB,IAAI,4BAAoB,EAAE;IAC1B,IAAI,sBAAc,EAAE;IACpB,IAAI,wBAAgB,EAAE;IACtB,IAAI,qBAAa,EAAE;IACnB,IAAI,sBAAc,EAAE;IACpB,IAAI,wBAAgB,EAAE;IACtB,IAAI,8BAAsB,EAAE;IAC5B,IAAI,wBAAgB,EAAE;IACtB,IAAI,yBAAiB,EAAE;IACvB,IAAI,yBAAiB,EAAE;IACvB,IAAI,yBAAiB,EAAE;IACvB,IAAI,6BAAqB,EAAE;IAC3B,IAAI,yBAAiB,EAAE;IACvB,IAAI,0BAAkB,EAAE;IACxB,IAAI,wBAAgB,EAAE;IACtB,IAAI,yBAAiB,EAAE;IACvB,IAAI,sBAAc,EAAE;IACpB,IAAI,4BAAoB,EAAE;IAC1B,IAAI,6BAAqB,EAAE;IAC3B,IAAI,yBAAiB,EAAE;IACvB,IAAI,2BAAmB,EAAE;IACzB,IAAI,0BAAkB,EAAE;IACxB,IAAI,4BAAoB,EAAE;IAC1B,IAAI,yBAAiB,EAAE;IACvB,IAAI,sBAAc,EAAE;IACpB,IAAI,2BAAmB,EAAE;IACzB,IAAI,sBAAc,EAAE;IACpB,IAAI,sBAAc,EAAE;IACpB,IAAI,mBAAW,EAAE;IACjB,IAAI,sBAAc,EAAE;CACrB,AA/Bc,CA+Bb"}

package/build/server/sapi-http-server.js

@@ -0,0 +1,11 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.ScraperAPIHttpServer = void 0;
+const sapi_base_server_1 = require("./sapi-base-server");
+class ScraperAPIHttpServer extends sapi_base_server_1.ScraperAPIBaseServer {
+    constructor({ toolsets = [], auth }) {
+        super({ auth, toolsets });
+    }
+}
+exports.ScraperAPIHttpServer = ScraperAPIHttpServer;
+//# sourceMappingURL=sapi-http-server.js.map

package/build/server/sapi-http-server.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"sapi-http-server.js","sourceRoot":"","sources":["../../src/server/sapi-http-server.ts"],"names":[],"mappings":";;;AAEA,yDAA0D;AAG1D,MAAa,oBAAqB,SAAQ,uCAAoB;IAK5D,YAAY,EAAE,QAAQ,GAAG,EAAE,EAAE,IAAI,EAAyC;QACxE,KAAK,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,CAAC,CAAC;IAC5B,CAAC;CACF;AARD,oDAQC"}

package/build/server/sapi-stdio-server.js

@@ -0,0 +1,11 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.ScraperAPIStdioServer = void 0;
+const sapi_base_server_1 = require("./sapi-base-server");
+class ScraperAPIStdioServer extends sapi_base_server_1.ScraperAPIBaseServer {
+    constructor({ auth, toolsets = [] }) {
+        super({ auth, toolsets });
+    }
+}
+exports.ScraperAPIStdioServer = ScraperAPIStdioServer;
+//# sourceMappingURL=sapi-stdio-server.js.map

package/build/server/sapi-stdio-server.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"sapi-stdio-server.js","sourceRoot":"","sources":["../../src/server/sapi-stdio-server.ts"],"names":[],"mappings":";;;AAEA,yDAA0D;AAG1D,MAAa,qBAAsB,SAAQ,uCAAoB;IAK7D,YAAY,EAAE,IAAI,EAAE,QAAQ,GAAG,EAAE,EAAyC;QACxE,KAAK,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,CAAC,CAAC;IAC5B,CAAC;CACF;AARD,sDAQC"}

package/build/server.js

@@ -0,0 +1,53 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+require("dotenv/config");
+const express_1 = __importDefault(require("express"));
+const streamableHttp_js_1 = require("@modelcontextprotocol/sdk/server/streamableHttp.js");
+const sapi_http_server_1 = require("./server/sapi-http-server");
+const utils_1 = require("./utils");
+const app = (0, express_1.default)();
+app.use(express_1.default.json());
+app.get('/mcp', (_req, res) => {
+    res.status(200).send('server up, use POST /mcp to see available tools');
+});
+app.post('/mcp', async (req, res) => {
+    const auth = req.headers.authorization;
+    if (!auth) {
+        res.status(401).send('Unauthorized');
+        return;
+    }
+    const parts = auth.split(' ');
+    if (parts.length !== 2 || parts[0] !== 'Basic') {
+        res.status(401).send("Valid 'Basic' authorization required");
+        return;
+    }
+    const token = parts[1];
+    const toolsets = (0, utils_1.resolveToolsets)(req.query.toolsets);
+    const server = new sapi_http_server_1.ScraperAPIHttpServer({ toolsets, auth: token });
+    const transport = new streamableHttp_js_1.StreamableHTTPServerTransport({
+        sessionIdGenerator: undefined,
+        enableJsonResponse: true,
+    });
+    res.on('close', () => {
+        transport.close();
+    });
+    await server.connect(transport);
+    await transport.handleRequest(req, res, req.body);
+});
+app.get('/healthz', (_req, res) => {
+    res.status(200).send('ok');
+});
+const port = parseInt(process.env.PORT || '3000');
+app
+    .listen(port, () => {
+    console.log(`Demo MCP Server running on http://localhost:${port}/mcp`);
+    console.log(`API host: ${process.env.DECODO_SAPI_HOST}`);
+})
+    .on('error', (error) => {
+    console.error('Server error:', error);
+    process.exit(1);
+});
+//# sourceMappingURL=server.js.map

package/build/server.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"server.js","sourceRoot":"","sources":["../src/server.ts"],"names":[],"mappings":";;;;;AAAA,yBAAuB;AACvB,sDAA8B;AAC9B,0FAAmG;AACnG,gEAAiE;AACjE,mCAA0C;AAE1C,MAAM,GAAG,GAAG,IAAA,iBAAO,GAAE,CAAC;AAEtB,GAAG,CAAC,GAAG,CAAC,iBAAO,CAAC,IAAI,EAAE,CAAC,CAAC;AAExB,GAAG,CAAC,GAAG,CAAC,MAAM,EAAE,CAAC,IAAI,EAAE,GAAG,EAAE,EAAE;IAC5B,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,IAAI,CAAC,iDAAiD,CAAC,CAAC;AAC1E,CAAC,CAAC,CAAC;AAEH,GAAG,CAAC,IAAI,CAAC,MAAM,EAAE,KAAK,EAAE,GAAG,EAAE,GAAG,EAAE,EAAE;IAClC,MAAM,IAAI,GAAG,GAAG,CAAC,OAAO,CAAC,aAAa,CAAC;IAEvC,IAAI,CAAC,IAAI,EAAE,CAAC;QACV,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC;QACrC,OAAO;IACT,CAAC;IAED,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IAE9B,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,OAAO,EAAE,CAAC;QAC/C,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,IAAI,CAAC,sCAAsC,CAAC,CAAC;QAC7D,OAAO;IACT,CAAC;IAED,MAAM,KAAK,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;IAEvB,MAAM,QAAQ,GAAG,IAAA,uBAAe,EAAC,GAAG,CAAC,KAAK,CAAC,QAAkB,CAAC,CAAC;IAE/D,MAAM,MAAM,GAAG,IAAI,uCAAoB,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;IAEnE,MAAM,SAAS,GAAG,IAAI,iDAA6B,CAAC;QAClD,kBAAkB,EAAE,SAAS;QAC7B,kBAAkB,EAAE,IAAI;KACzB,CAAC,CAAC;IAEH,GAAG,CAAC,EAAE,CAAC,OAAO,EAAE,GAAG,EAAE;QACnB,SAAS,CAAC,KAAK,EAAE,CAAC;IACpB,CAAC,CAAC,CAAC;IAEH,MAAM,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IAEhC,MAAM,SAAS,CAAC,aAAa,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC,IAAI,CAAC,CAAC;AACpD,CAAC,CAAC,CAAC;AAEH,GAAG,CAAC,GAAG,CAAC,UAAU,EAAE,CAAC,IAAI,EAAE,GAAG,EAAE,EAAE;IAChC,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC7B,CAAC,CAAC,CAAC;AAEH,MAAM,IAAI,GAAG,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,IAAI,MAAM,CAAC,CAAC;AAElD,GAAG;KACA,MAAM,CAAC,IAAI,EAAE,GAAG,EAAE;IACjB,OAAO,CAAC,GAAG,CAAC,+CAA+C,IAAI,MAAM,CAAC,CAAC;IACvE,OAAO,CAAC,GAAG,CAAC,aAAa,OAAO,CAAC,GAAG,CAAC,gBAAgB,EAAE,CAAC,CAAC;AAC3D,CAAC,CAAC;KACD,EAAE,CAAC,OAAO,EAAE,CAAC,KAAY,EAAE,EAAE;IAC5B,OAAO,CAAC,KAAK,CAAC,eAAe,EAAE,KAAK,CAAC,CAAC;IACtC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"}