@decodo/mcp-server 1.0.0

package/README.md

@@ -0,0 +1,169 @@
+# Decodo MCP Server
+
+[![smithery badge](https://smithery.ai/badge/@Decodo/decodo-mcp-server)](https://smithery.ai/server/@Decodo/decodo-mcp-server)
+
+<p align="center">
+<a href="https://dashboard.decodo.com/register?page=scrapers%2Fpricing&utm_source=github.com&utm_medium=social&utm_campaign=scraper_api_github"><img src="https://github.com/user-attachments/assets/13b08523-32b0-4c85-8e99-580d7c2a9055"></a>
+
+[![](https://dcbadge.vercel.app/api/server/Ja8dqKgvbZ)](https://discord.gg/Ja8dqKgvbZ)
+
+This repository provides a Model Context Protocol (MCP) server that connects LLMs and applications
+to Decodo's platform. The server facilitates integration between MCP-compatible clients and Decodo's
+services, streamlining access to our tools and capabilities.
+
+## Features
+
+- Easy web data access. Simplified retrieval of information from websites and online sources.
+- Geographic flexibility. Access content regardless of regional restrictions.
+- Enhanced privacy. Browse and collect data while maintaining anonymity.
+- Reliable scraping. Advanced techniques to avoid detection and blocks.
+- Simple integration. Seamless setup with popular MCP clients like Claude Desktop, Cursor, and
+  Windsurf.
+
+## Quick start via Smithery (Recommended)
+
+Visit the `decodo-mcp-server` page on
+[Smithery](https://smithery.ai/server/@Decodo/decodo-mcp-server), select your preferred MCP client
+and generate installation instructions.
+
+![Smithery interface](img/smithery.png 'Smithery UI')
+
+## Obtain Web Scraping API credentials
+
+You'll need the Decodo Advanced Web Scraping API credentials, which you can get by starting a free
+trial on the [dashboard](https://dashboard.decodo.com/). Once you have a plan activated, take a note
+of your generated username and password:
+
+![Decodo dashboard](img/auth.png 'Decodo dashboard')
+
+## Running the MCP server locally (manual)
+
+### Prerequisites
+
+- Node.js 18.0+
+- An MCP client - popular choices are [Claude Desktop](https://claude.ai/download) and
+  [Cursor](https://www.cursor.com/)
+
+### Step-by-step guide
+
+1. Clone this repository:
+
+```
+git clone https://github.com/Decodo/decodo-mcp-server
+```
+
+2. Run the following commands in the terminal:
+
+```
+cd decodo-mcp-server
+npm install
+npm run build
+```
+
+3. Take note of your build location:
+
+```
+cd build/
+pwd
+```
+
+Adding `index.js` to the end of this directory, your build file location should look something like
+this:
+
+```
+/Users/your.user/projects/decodo-mcp/build/index.js
+```
+
+4. Update your MCP client with the server information:
+
+### Claude Desktop
+
+[Follow the guide here](https://modelcontextprotocol.io/quickstart/user) to find the setup file,
+then update `claude_desktop_config.json` to look like this:
+
+```
+{
+  "mcpServers": {
+    "decodo-mcp": {
+      "command": "node",
+      "args": ["/Users/your.user/projects/decodo-mcp/build/index.js"],
+      "env": {
+        "SCRAPER_API_USERNAME": "your_username",
+        "SCRAPER_API_PASSWORD": "your_password"
+      }
+    }
+  }
+}
+
+```
+
+### Cursor
+
+For installation instructions, see the
+[Cursor documentation](https://docs.cursor.com/context/model-context-protocol#installing-mcp-servers).
+
+## Tools
+
+The server exposes the following tools:
+
+| Tool                   | Description                                                                                | Example prompt                                                   |
+| ---------------------- | ------------------------------------------------------------------------------------------ | ---------------------------------------------------------------- |
+| `scrape_as_markdown`   | Scrapes any target URL, expects a URL to be given via prompt. Returns results in Markdown. | Scrape peacock.com from a US IP address and tell me the pricing. |
+| `google_search_parsed` | Scrapes Google Search for a given query, and returns parsed results.                       | Scrape Google Search for shoes and tell me the top position.     |
+| `amazon_search_parsed` | Scrapes Amazon Search for a given query, and returns parsed results.                       | Scrape Amazon Search for toothbrushes                            |
+
+## Parameters
+
+The following parameters are inferred from user prompts:
+
+| Parameter      | Description                                                                                          |
+| -------------- | ---------------------------------------------------------------------------------------------------- |
+| `jsRender`     | Renders target URL in a headless browser.                                                            |
+| `geo`          | Sets the country from which the request will originate.                                              |
+| `locale`       | Sets the locale of the request.                                                                      |
+| `tokenLimit`   | Truncates the response content up to this limit. Useful if the context window is small.              |
+| `fullResponse` | Skips automatic truncation and returns full content. If context window is small, may throw warnings. |
+
+## Examples
+
+### Scraping geo-restricted content
+
+Query your AI agent with the following prompt:
+
+```
+Scrape peacock.com from a German IP address and tell me the pricing.
+```
+
+This prompt will say that peacock.com is geo-restricted. To bypass the geo-restriction:
+
+```
+Scrape peacock.com from a US IP address and tell me the pricing.
+```
+
+### Limiting number of response tokens
+
+If your agent has a small context window, the content returned from scraping will be automatically
+truncated, in order to avoid context-overflow. You can increase the number of tokens returned within
+your prompt:
+
+```
+Scrape hacker news, return 50k tokens.
+```
+
+If your agent has a big context window, tell it to return `full content`:
+
+```
+Scrape hacker news, return full content.
+```
+
+## Related repositories
+
+[Web Scraping API](https://github.com/Decodo/Web-Scraping-API)
+
+[Google Maps scraper](https://github.com/Decodo/google-maps-scraper)
+
+[Amazon scraper](https://github.com/Decodo/Amazon-scraper)
+
+## License
+
+All code is released under the [MIT License](https://github.com/Decodo/Decodo/blob/master/LICENSE).

package/build/clients/scraper-api-client.js

@@ -0,0 +1,48 @@
+"use strict";
+var __rest = (this && this.__rest) || function (s, e) {
+    var t = {};
+    for (var p in s) if (Object.prototype.hasOwnProperty.call(s, p) && e.indexOf(p) < 0)
+        t[p] = s[p];
+    if (s != null && typeof Object.getOwnPropertySymbols === "function")
+        for (var i = 0, p = Object.getOwnPropertySymbols(s); i < p.length; i++) {
+            if (e.indexOf(p[i]) < 0 && Object.prototype.propertyIsEnumerable.call(s, p[i]))
+                t[p[i]] = s[p[i]];
+        }
+    return t;
+};
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.ScraperApiClient = void 0;
+const axios_1 = __importDefault(require("axios"));
+class ScraperApiClient {
+    constructor({ auth }) {
+        this.transformScrapingParams = ({ scrapingParams, }) => {
+            const { jsRender } = scrapingParams, rest = __rest(scrapingParams, ["jsRender"]);
+            const transformed = Object.assign(Object.assign({}, (jsRender && { headless: 'html' })), rest);
+            return transformed;
+        };
+        this.transformResponse = ({ res }) => {
+            const content = res.data.results[0].content;
+            return Object.assign(Object.assign({}, res), { data: content });
+        };
+        this.scrape = async ({ scrapingParams }) => {
+            const transformedParams = this.transformScrapingParams({ scrapingParams });
+            const res = await axios_1.default.request({
+                url: 'https://scraper-api.decodo.com/v2/scrape',
+                method: 'POST',
+                headers: {
+                    authorization: `Basic ${this.auth}`,
+                    'x-integration': 'mcp',
+                },
+                data: Object.assign({}, transformedParams),
+            });
+            const response = this.transformResponse({ res });
+            return response;
+        };
+        this.auth = auth;
+    }
+}
+exports.ScraperApiClient = ScraperApiClient;
+//# sourceMappingURL=scraper-api-client.js.map

package/build/clients/scraper-api-client.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"scraper-api-client.js","sourceRoot":"","sources":["../../src/clients/scraper-api-client.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;AAAA,kDAA6C;AAI7C,MAAa,gBAAgB;IAG3B,YAAY,EAAE,IAAI,EAAoB;QAItC,4BAAuB,GAAG,CAAC,EACzB,cAAc,GAGf,EAAoB,EAAE;YACrB,MAAM,EAAE,QAAQ,KAAc,cAAc,EAAvB,IAAI,UAAK,cAAc,EAAtC,YAAqB,CAAiB,CAAC;YAC7C,MAAM,WAAW,mCAAQ,CAAC,QAAQ,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,CAAC,GAAK,IAAI,CAAE,CAAC;YAEvE,OAAO,WAAW,CAAC;QACrB,CAAC,CAAC;QAEF,sBAAiB,GAAG,CAAI,EAAE,GAAG,EAAqD,EAAE,EAAE;YACpF,MAAM,OAAO,GAAG,GAAG,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC;YAE5C,uCAAY,GAAG,KAAE,IAAI,EAAE,OAAO,IAAG;QACnC,CAAC,CAAC;QAEF,WAAM,GAAG,KAAK,EAAc,EAAE,cAAc,EAAyC,EAAE,EAAE;YACvF,MAAM,iBAAiB,GAAG,IAAI,CAAC,uBAAuB,CAAC,EAAE,cAAc,EAAE,CAAC,CAAC;YAE3E,MAAM,GAAG,GAAG,MAAM,eAAK,CAAC,OAAO,CAA4B;gBACzD,GAAG,EAAE,0CAA0C;gBAC/C,MAAM,EAAE,MAAM;gBACd,OAAO,EAAE;oBACP,aAAa,EAAE,SAAS,IAAI,CAAC,IAAI,EAAE;oBACnC,eAAe,EAAE,KAAK;iBACvB;gBACD,IAAI,oBACC,iBAAiB,CACrB;aACF,CAAC,CAAC;YAEH,MAAM,QAAQ,GAAG,IAAI,CAAC,iBAAiB,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC;YAEjD,OAAO,QAAQ,CAAC;QAClB,CAAC,CAAC;QAtCA,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;IACnB,CAAC;CAsCF;AA3CD,4CA2CC"}

package/build/clients/types.js

@@ -0,0 +1,3 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+//# sourceMappingURL=types.js.map

package/build/clients/types.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"types.js","sourceRoot":"","sources":["../../src/clients/types.ts"],"names":[],"mappings":""}

package/build/constants.js

@@ -0,0 +1,11 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.SCRAPER_API_TARGETS = void 0;
+var SCRAPER_API_TARGETS;
+(function (SCRAPER_API_TARGETS) {
+    SCRAPER_API_TARGETS["GOOGLE_SEARCH"] = "google_search";
+    SCRAPER_API_TARGETS["AMAZON_SEARCH"] = "amazon_search";
+    SCRAPER_API_TARGETS["REDDIT_POST"] = "reddit_post";
+    SCRAPER_API_TARGETS["REDDIT_SUBREDDIT"] = "reddit_subreddit";
+})(SCRAPER_API_TARGETS || (exports.SCRAPER_API_TARGETS = SCRAPER_API_TARGETS = {}));
+//# sourceMappingURL=constants.js.map

package/build/constants.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"constants.js","sourceRoot":"","sources":["../src/constants.ts"],"names":[],"mappings":";;;AAAA,IAAY,mBAMX;AAND,WAAY,mBAAmB;IAC7B,sDAA+B,CAAA;IAC/B,sDAA+B,CAAA;IAE/B,kDAA2B,CAAA;IAC3B,4DAAqC,CAAA;AACvC,CAAC,EANW,mBAAmB,mCAAnB,mBAAmB,QAM9B"}

package/build/index.js

@@ -0,0 +1,37 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+require("dotenv/config");
+const process_1 = require("process");
+const stdio_js_1 = require("@modelcontextprotocol/sdk/server/stdio.js");
+const sapi_mcp_server_1 = require("./sapi-mcp-server");
+if (process.env.ENABLE_MCPS_LOGGER) {
+    import('mcps-logger/console');
+}
+const parseEnvsOrExit = () => {
+    const envs = ['SCRAPER_API_USERNAME', 'SCRAPER_API_PASSWORD'];
+    for (const envKey of envs) {
+        if (!process.env[envKey]) {
+            (0, process_1.exit)(`env ${envKey} missing`);
+        }
+    }
+    return {
+        sapiUsername: process.env['SCRAPER_API_USERNAME'],
+        sapiPassword: process.env['SCRAPER_API_PASSWORD'],
+    };
+};
+async function main() {
+    const transport = new stdio_js_1.StdioServerTransport();
+    // if there are no envs, some MCP clients will fail silently
+    const { sapiUsername, sapiPassword } = parseEnvsOrExit();
+    const sapiMcpServer = new sapi_mcp_server_1.ScraperAPIMCPServer({
+        sapiUsername,
+        sapiPassword,
+    });
+    await sapiMcpServer.connect(transport);
+    console.error('MCP Server running on stdio');
+}
+main().catch(error => {
+    console.error('Fatal error in main():', error);
+    process.exit(1);
+});
+//# sourceMappingURL=index.js.map

package/build/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";;AAAA,yBAAuB;AAEvB,qCAA+B;AAC/B,wEAAiF;AACjF,uDAAwD;AAExD,IAAI,OAAO,CAAC,GAAG,CAAC,kBAAkB,EAAE,CAAC;IACnC,MAAM,CAAC,qBAAqB,CAAC,CAAC;AAChC,CAAC;AAED,MAAM,eAAe,GAAG,GAA2B,EAAE;IACnD,MAAM,IAAI,GAAG,CAAC,sBAAsB,EAAE,sBAAsB,CAAC,CAAC;IAE9D,KAAK,MAAM,MAAM,IAAI,IAAI,EAAE,CAAC;QAC1B,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC;YACzB,IAAA,cAAI,EAAC,OAAO,MAAM,UAAU,CAAC,CAAC;QAChC,CAAC;IACH,CAAC;IAED,OAAO;QACL,YAAY,EAAE,OAAO,CAAC,GAAG,CAAC,sBAAsB,CAAW;QAC3D,YAAY,EAAE,OAAO,CAAC,GAAG,CAAC,sBAAsB,CAAW;KAC5D,CAAC;AACJ,CAAC,CAAC;AAEF,KAAK,UAAU,IAAI;IACjB,MAAM,SAAS,GAAG,IAAI,+BAAoB,EAAE,CAAC;IAE7C,4DAA4D;IAC5D,MAAM,EAAE,YAAY,EAAE,YAAY,EAAE,GAAG,eAAe,EAAE,CAAC;IAEzD,MAAM,aAAa,GAAG,IAAI,qCAAmB,CAAC;QAC5C,YAAY;QACZ,YAAY;KACb,CAAC,CAAC;IACH,MAAM,aAAa,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IAEvC,OAAO,CAAC,KAAK,CAAC,6BAA6B,CAAC,CAAC;AAC/C,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE;IACnB,OAAO,CAAC,KAAK,CAAC,wBAAwB,EAAE,KAAK,CAAC,CAAC;IAC/C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"}

package/build/sapi-mcp-server.js

@@ -0,0 +1,40 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.ScraperAPIMCPServer = void 0;
+const mcp_js_1 = require("@modelcontextprotocol/sdk/server/mcp.js");
+const scraper_api_client_1 = require("./clients/scraper-api-client");
+const tools_1 = require("./tools");
+class ScraperAPIMCPServer {
+    constructor({ sapiUsername, sapiPassword }) {
+        this.server = new mcp_js_1.McpServer({
+            name: 'decodo',
+            version: '0.1.0',
+            capabilities: {
+                logging: {},
+                resources: {},
+                tools: {},
+            },
+        });
+        const auth = Buffer.from(`${sapiUsername}:${sapiPassword}`).toString('base64');
+        this.sapiClient = new scraper_api_client_1.ScraperApiClient({ auth });
+        this.registerTools();
+        this.registerResources();
+    }
+    connect(transport) {
+        this.server.connect(transport);
+    }
+    registerTools() {
+        // scrape
+        tools_1.ScrapeAsMarkdownTool.register({ server: this.server, sapiClient: this.sapiClient });
+        // targets
+        tools_1.GoogleSearchParsedTool.register({ server: this.server, sapiClient: this.sapiClient });
+        tools_1.AmazonSearchParsedTool.register({ server: this.server, sapiClient: this.sapiClient });
+        tools_1.RedditPostTool.register({ server: this.server, sapiClient: this.sapiClient });
+        tools_1.RedditSubredditTool.register({ server: this.server, sapiClient: this.sapiClient });
+    }
+    registerResources() {
+        // todo: expose info about all targets available
+    }
+}
+exports.ScraperAPIMCPServer = ScraperAPIMCPServer;
+//# sourceMappingURL=sapi-mcp-server.js.map

package/build/sapi-mcp-server.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"sapi-mcp-server.js","sourceRoot":"","sources":["../src/sapi-mcp-server.ts"],"names":[],"mappings":";;;AAAA,oEAAoE;AAEpE,qEAAgE;AAChE,mCAMiB;AAEjB,MAAa,mBAAmB;IAK9B,YAAY,EAAE,YAAY,EAAE,YAAY,EAAkD;QACxF,IAAI,CAAC,MAAM,GAAG,IAAI,kBAAS,CAAC;YAC1B,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,OAAO;YAChB,YAAY,EAAE;gBACZ,OAAO,EAAE,EAAE;gBACX,SAAS,EAAE,EAAE;gBACb,KAAK,EAAE,EAAE;aACV;SACF,CAAC,CAAC;QAEH,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,GAAG,YAAY,IAAI,YAAY,EAAE,CAAC,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC;QAE/E,IAAI,CAAC,UAAU,GAAG,IAAI,qCAAgB,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC;QAEjD,IAAI,CAAC,aAAa,EAAE,CAAC;QACrB,IAAI,CAAC,iBAAiB,EAAE,CAAC;IAC3B,CAAC;IAED,OAAO,CAAC,SAA+B;QACrC,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IACjC,CAAC;IAED,aAAa;QACX,SAAS;QACT,4BAAoB,CAAC,QAAQ,CAAC,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,UAAU,EAAE,IAAI,CAAC,UAAU,EAAE,CAAC,CAAC;QAEpF,UAAU;QACV,8BAAsB,CAAC,QAAQ,CAAC,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,UAAU,EAAE,IAAI,CAAC,UAAU,EAAE,CAAC,CAAC;QACtF,8BAAsB,CAAC,QAAQ,CAAC,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,UAAU,EAAE,IAAI,CAAC,UAAU,EAAE,CAAC,CAAC;QACtF,sBAAc,CAAC,QAAQ,CAAC,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,UAAU,EAAE,IAAI,CAAC,UAAU,EAAE,CAAC,CAAC;QAC9E,2BAAmB,CAAC,QAAQ,CAAC,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,UAAU,EAAE,IAAI,CAAC,UAAU,EAAE,CAAC,CAAC;IACrF,CAAC;IAED,iBAAiB;QACf,gDAAgD;IAClD,CAAC;CACF;AA1CD,kDA0CC"}

package/build/tools/amazon-search-parsed-tool.js

@@ -0,0 +1,46 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+var _a;
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.AmazonSearchParsedTool = void 0;
+const zod_1 = __importDefault(require("zod"));
+const constants_1 = require("../constants");
+const utils_1 = require("../utils");
+const zod_types_1 = require("../zod/zod-types");
+class AmazonSearchParsedTool {
+}
+exports.AmazonSearchParsedTool = AmazonSearchParsedTool;
+_a = AmazonSearchParsedTool;
+AmazonSearchParsedTool.FIELDS_WITH_HIGH_CHAR_COUNT = ['suggested', 'amazons_choices', 'refinements'];
+AmazonSearchParsedTool.transformAutoParsedResponse = ({ obj }) => {
+    for (const fieldToRemove of _a.FIELDS_WITH_HIGH_CHAR_COUNT) {
+        obj = (0, utils_1.removeKeyFromNestedObject)({ obj, keyToRemove: fieldToRemove });
+    }
+    const text = JSON.stringify(obj);
+    return text;
+};
+AmazonSearchParsedTool.register = ({ server, sapiClient, }) => {
+    server.registerTool('amazon_search_parsed', {
+        description: 'Scrape Amazon Search results with automatic parsing',
+        inputSchema: {
+            query: zod_1.default.string().describe('Search query'),
+            geo: zod_types_1.zodGeo,
+            jsRender: zod_types_1.zodJsRender,
+        },
+    }, async (scrapingParams) => {
+        const params = Object.assign(Object.assign({}, scrapingParams), { target: constants_1.SCRAPER_API_TARGETS.AMAZON_SEARCH, parse: true });
+        const { data } = await sapiClient.scrape({ scrapingParams: params });
+        const text = _a.transformAutoParsedResponse({ obj: data });
+        return {
+            content: [
+                {
+                    type: 'text',
+                    text,
+                },
+            ],
+        };
+    });
+};
+//# sourceMappingURL=amazon-search-parsed-tool.js.map

package/build/tools/amazon-search-parsed-tool.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"amazon-search-parsed-tool.js","sourceRoot":"","sources":["../../src/tools/amazon-search-parsed-tool.ts"],"names":[],"mappings":";;;;;;;AAAA,8CAAoB;AAIpB,4CAAmD;AACnD,oCAAqD;AACrD,gDAAkE;AAElE,MAAa,sBAAsB;;AAAnC,wDAoDC;;AAnDQ,kDAA2B,GAAG,CAAC,WAAW,EAAE,iBAAiB,EAAE,aAAa,CAAC,AAAlD,CAAmD;AAE9E,kDAA2B,GAAG,CAAC,EAAE,GAAG,EAAmB,EAAU,EAAE;IACxE,KAAK,MAAM,aAAa,IAAI,EAAsB,CAAC,2BAA2B,EAAE,CAAC;QAC/E,GAAG,GAAG,IAAA,iCAAyB,EAAC,EAAE,GAAG,EAAE,WAAW,EAAE,aAAa,EAAE,CAAC,CAAC;IACvE,CAAC;IAED,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC;IAEjC,OAAO,IAAI,CAAC;AACd,CAAC,AARiC,CAQhC;AAEK,+BAAQ,GAAG,CAAC,EACjB,MAAM,EACN,UAAU,GAIX,EAAE,EAAE;IACH,MAAM,CAAC,YAAY,CACjB,sBAAsB,EACtB;QACE,WAAW,EAAE,qDAAqD;QAClE,WAAW,EAAE;YACX,KAAK,EAAE,aAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,CAAC,cAAc,CAAC;YAC1C,GAAG,EAAE,kBAAM;YACX,QAAQ,EAAE,uBAAW;SACtB;KACF,EACD,KAAK,EAAE,cAAiC,EAAE,EAAE;QAC1C,MAAM,MAAM,GAAG,gCACV,cAAc,KACjB,MAAM,EAAE,+BAAmB,CAAC,aAAa,EACzC,KAAK,EAAE,IAAI,GACe,CAAC;QAE7B,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,UAAU,CAAC,MAAM,CAAS,EAAE,cAAc,EAAE,MAAM,EAAE,CAAC,CAAC;QAE7E,MAAM,IAAI,GAAG,EAAI,CAAC,2BAA2B,CAAC,EAAE,GAAG,EAAE,IAAI,EAAE,CAAC,CAAC;QAE7D,OAAO;YACL,OAAO,EAAE;gBACP;oBACE,IAAI,EAAE,MAAM;oBACZ,IAAI;iBACL;aACF;SACF,CAAC;IACJ,CAAC,CACF,CAAC;AACJ,CAAC,AAtCc,CAsCb"}

package/build/tools/google-search-parsed-tool.js

@@ -0,0 +1,54 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+var _a;
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.GoogleSearchParsedTool = void 0;
+const zod_1 = __importDefault(require("zod"));
+const constants_1 = require("../constants");
+const utils_1 = require("../utils");
+const zod_types_1 = require("../zod/zod-types");
+class GoogleSearchParsedTool {
+}
+exports.GoogleSearchParsedTool = GoogleSearchParsedTool;
+_a = GoogleSearchParsedTool;
+GoogleSearchParsedTool.FIELDS_WITH_HIGH_CHAR_COUNT = [
+    'images',
+    'image_data',
+    'related_searches_urls',
+    'factoids',
+    'people_also_buy_from',
+    'what_people_are_saying',
+];
+GoogleSearchParsedTool.transformAutoParsedResponse = ({ obj }) => {
+    for (const fieldToRemove of _a.FIELDS_WITH_HIGH_CHAR_COUNT) {
+        obj = (0, utils_1.removeKeyFromNestedObject)({ obj, keyToRemove: fieldToRemove });
+    }
+    const text = JSON.stringify(obj);
+    return text;
+};
+GoogleSearchParsedTool.register = ({ server, sapiClient, }) => {
+    server.registerTool('google_search_parsed', {
+        description: 'Scrape Google Search results with automatic parsing',
+        inputSchema: {
+            query: zod_1.default.string().describe('Search query'),
+            geo: zod_types_1.zodGeo,
+            locale: zod_types_1.zodLocale,
+            jsRender: zod_types_1.zodJsRender,
+        },
+    }, async (scrapingParams) => {
+        const params = Object.assign(Object.assign({}, scrapingParams), { target: constants_1.SCRAPER_API_TARGETS.GOOGLE_SEARCH, parse: true });
+        const { data } = await sapiClient.scrape({ scrapingParams: params });
+        const text = _a.transformAutoParsedResponse({ obj: data });
+        return {
+            content: [
+                {
+                    type: 'text',
+                    text,
+                },
+            ],
+        };
+    });
+};
+//# sourceMappingURL=google-search-parsed-tool.js.map

package/build/tools/google-search-parsed-tool.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"google-search-parsed-tool.js","sourceRoot":"","sources":["../../src/tools/google-search-parsed-tool.ts"],"names":[],"mappings":";;;;;;;AAAA,8CAAoB;AAIpB,4CAAmD;AACnD,oCAAqD;AACrD,gDAAkE;AAElE,MAAa,sBAAsB;;AAAnC,wDA4DC;;AA3DQ,kDAA2B,GAAG;IACnC,QAAQ;IACR,YAAY;IACZ,uBAAuB;IACvB,UAAU;IACV,sBAAsB;IACtB,wBAAwB;CACzB,AAPiC,CAOhC;AAEK,kDAA2B,GAAG,CAAC,EAAE,GAAG,EAAmB,EAAU,EAAE;IACxE,KAAK,MAAM,aAAa,IAAI,EAAsB,CAAC,2BAA2B,EAAE,CAAC;QAC/E,GAAG,GAAG,IAAA,iCAAyB,EAAC,EAAE,GAAG,EAAE,WAAW,EAAE,aAAa,EAAE,CAAC,CAAC;IACvE,CAAC;IAED,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC;IAEjC,OAAO,IAAI,CAAC;AACd,CAAC,AARiC,CAQhC;AAEK,+BAAQ,GAAG,CAAC,EACjB,MAAM,EACN,UAAU,GAIX,EAAE,EAAE;IACH,MAAM,CAAC,YAAY,CACjB,sBAAsB,EACtB;QACE,WAAW,EAAE,qDAAqD;QAClE,WAAW,EAAE;YACX,KAAK,EAAE,aAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,CAAC,cAAc,CAAC;YAC1C,GAAG,EAAE,kBAAM;YACX,MAAM,EAAE,qBAAS;YACjB,QAAQ,EAAE,uBAAW;SACtB;KACF,EACD,KAAK,EAAE,cAAiC,EAAE,EAAE;QAC1C,MAAM,MAAM,GAAG,gCACV,cAAc,KACjB,MAAM,EAAE,+BAAmB,CAAC,aAAa,EACzC,KAAK,EAAE,IAAI,GACe,CAAC;QAE7B,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,UAAU,CAAC,MAAM,CAAS,EAAE,cAAc,EAAE,MAAM,EAAE,CAAC,CAAC;QAE7E,MAAM,IAAI,GAAG,EAAI,CAAC,2BAA2B,CAAC,EAAE,GAAG,EAAE,IAAI,EAAE,CAAC,CAAC;QAE7D,OAAO;YACL,OAAO,EAAE;gBACP;oBACE,IAAI,EAAE,MAAM;oBACZ,IAAI;iBACL;aACF;SACF,CAAC;IACJ,CAAC,CACF,CAAC;AACJ,CAAC,AAvCc,CAuCb"}

package/build/tools/index.js

@@ -0,0 +1,22 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __exportStar = (this && this.__exportStar) || function(m, exports) {
+    for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+__exportStar(require("./amazon-search-parsed-tool"), exports);
+__exportStar(require("./google-search-parsed-tool"), exports);
+__exportStar(require("./reddit-subreddit-tool"), exports);
+__exportStar(require("./reddit-post-tool"), exports);
+__exportStar(require("./scrape-as-markdown-tool"), exports);
+//# sourceMappingURL=index.js.map

package/build/tools/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/tools/index.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;AAAA,8DAA4C;AAC5C,8DAA4C;AAC5C,0DAAwC;AACxC,qDAAmC;AACnC,4DAA0C"}

package/build/tools/reddit-post-tool.js

@@ -0,0 +1,34 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+var _a;
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.RedditPostTool = void 0;
+const zod_1 = __importDefault(require("zod"));
+const constants_1 = require("../constants");
+class RedditPostTool {
+}
+exports.RedditPostTool = RedditPostTool;
+_a = RedditPostTool;
+RedditPostTool.register = ({ server, sapiClient, }) => {
+    server.registerTool('reddit_post', {
+        description: 'Scrape a specific Reddit post',
+        inputSchema: {
+            url: zod_1.default.string().describe('reddit post URL'),
+        },
+    }, async (scrapingParams) => {
+        const params = Object.assign(Object.assign({}, scrapingParams), { target: constants_1.SCRAPER_API_TARGETS.REDDIT_POST });
+        const { data } = await sapiClient.scrape({ scrapingParams: params });
+        const text = JSON.stringify(data, null, 2);
+        return {
+            content: [
+                {
+                    type: 'text',
+                    text,
+                },
+            ],
+        };
+    });
+};
+//# sourceMappingURL=reddit-post-tool.js.map

package/build/tools/reddit-post-tool.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"reddit-post-tool.js","sourceRoot":"","sources":["../../src/tools/reddit-post-tool.ts"],"names":[],"mappings":";;;;;;;AAAA,8CAAoB;AAIpB,4CAAmD;AAEnD,MAAa,cAAc;;AAA3B,wCAqCC;;AApCQ,uBAAQ,GAAG,CAAC,EACjB,MAAM,EACN,UAAU,GAIX,EAAE,EAAE;IACH,MAAM,CAAC,YAAY,CACjB,aAAa,EACb;QACE,WAAW,EAAE,+BAA+B;QAC5C,WAAW,EAAE;YACX,GAAG,EAAE,aAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,CAAC,iBAAiB,CAAC;SAC5C;KACF,EACD,KAAK,EAAE,cAAiC,EAAE,EAAE;QAC1C,MAAM,MAAM,GAAG,gCACV,cAAc,KACjB,MAAM,EAAE,+BAAmB,CAAC,WAAW,GACb,CAAC;QAE7B,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,UAAU,CAAC,MAAM,CAAS,EAAE,cAAc,EAAE,MAAM,EAAE,CAAC,CAAC;QAE7E,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;QAE3C,OAAO;YACL,OAAO,EAAE;gBACP;oBACE,IAAI,EAAE,MAAM;oBACZ,IAAI;iBACL;aACF;SACF,CAAC;IACJ,CAAC,CACF,CAAC;AACJ,CAAC,AAnCc,CAmCb"}

package/build/tools/reddit-subreddit-tool.js

@@ -0,0 +1,34 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+var _a;
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.RedditSubredditTool = void 0;
+const zod_1 = __importDefault(require("zod"));
+const constants_1 = require("../constants");
+class RedditSubredditTool {
+}
+exports.RedditSubredditTool = RedditSubredditTool;
+_a = RedditSubredditTool;
+RedditSubredditTool.register = ({ server, sapiClient, }) => {
+    server.registerTool('reddit_subreddit', {
+        description: 'Scrape Reddit subreddit results with automatic parsing',
+        inputSchema: {
+            url: zod_1.default.string().describe('URL to subreddit'),
+        },
+    }, async (scrapingParams) => {
+        const params = Object.assign(Object.assign({}, scrapingParams), { target: constants_1.SCRAPER_API_TARGETS.REDDIT_SUBREDDIT });
+        const { data } = await sapiClient.scrape({ scrapingParams: params });
+        const text = JSON.stringify(data, null, 2);
+        return {
+            content: [
+                {
+                    type: 'text',
+                    text,
+                },
+            ],
+        };
+    });
+};
+//# sourceMappingURL=reddit-subreddit-tool.js.map

package/build/tools/reddit-subreddit-tool.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"reddit-subreddit-tool.js","sourceRoot":"","sources":["../../src/tools/reddit-subreddit-tool.ts"],"names":[],"mappings":";;;;;;;AAAA,8CAAoB;AAIpB,4CAAmD;AAEnD,MAAa,mBAAmB;;AAAhC,kDAqCC;;AApCQ,4BAAQ,GAAG,CAAC,EACjB,MAAM,EACN,UAAU,GAIX,EAAE,EAAE;IACH,MAAM,CAAC,YAAY,CACjB,kBAAkB,EAClB;QACE,WAAW,EAAE,wDAAwD;QACrE,WAAW,EAAE;YACX,GAAG,EAAE,aAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,CAAC,kBAAkB,CAAC;SAC7C;KACF,EACD,KAAK,EAAE,cAAiC,EAAE,EAAE;QAC1C,MAAM,MAAM,GAAG,gCACV,cAAc,KACjB,MAAM,EAAE,+BAAmB,CAAC,gBAAgB,GAClB,CAAC;QAE7B,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,UAAU,CAAC,MAAM,CAAS,EAAE,cAAc,EAAE,MAAM,EAAE,CAAC,CAAC;QAE7E,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;QAE3C,OAAO;YACL,OAAO,EAAE;gBACP;oBACE,IAAI,EAAE,MAAM;oBACZ,IAAI;iBACL;aACF;SACF,CAAC;IACJ,CAAC,CACF,CAAC;AACJ,CAAC,AAnCc,CAmCb"}

package/build/tools/scrape-as-markdown-tool.js

@@ -0,0 +1,70 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+var _a;
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.ScrapeAsMarkdownTool = void 0;
+const zod_1 = __importDefault(require("zod"));
+const node_html_markdown_1 = require("node-html-markdown");
+const zod_types_1 = require("../zod/zod-types");
+class ScrapeAsMarkdownTool {
+}
+exports.ScrapeAsMarkdownTool = ScrapeAsMarkdownTool;
+_a = ScrapeAsMarkdownTool;
+ScrapeAsMarkdownTool.LARGE_CONTENT_SYMBOL_COUNT = 10000;
+ScrapeAsMarkdownTool.isResponseOverLimit = (content) => {
+    return content.length > _a.LARGE_CONTENT_SYMBOL_COUNT;
+};
+ScrapeAsMarkdownTool.shouldTruncateResponse = ({ content, shouldShowFullResponse, }) => {
+    if (shouldShowFullResponse) {
+        return false;
+    }
+    return _a.isResponseOverLimit(content);
+};
+ScrapeAsMarkdownTool.truncateResponse = ({ content, limit }) => {
+    return content.substring(0, limit);
+};
+ScrapeAsMarkdownTool.transformResponse = ({ html, tokenLimit, shouldShowFullResponse, }) => {
+    const markdown = node_html_markdown_1.NodeHtmlMarkdown.translate(html, {});
+    if (_a.shouldTruncateResponse({ content: markdown, shouldShowFullResponse })) {
+        const truncated = _a.truncateResponse({
+            content: markdown,
+            limit: tokenLimit || _a.LARGE_CONTENT_SYMBOL_COUNT,
+        });
+        return { markdown: truncated, isTruncated: true };
+    }
+    return { markdown, isTruncated: false };
+};
+ScrapeAsMarkdownTool.register = ({ server, sapiClient, }) => {
+    server.tool('scrape_as_markdown', 'Scrape the contents of a website and return Markdown-formatted results', {
+        url: zod_1.default.string().describe('URL to scrape'),
+        geo: zod_types_1.zodGeo,
+        locale: zod_types_1.zodLocale,
+        jsRender: zod_types_1.zodJsRender,
+        tokenLimit: zod_types_1.zodTokenLimit,
+        fullResponse: zod_types_1.zodFullResponse,
+    }, async (scrapingParams) => {
+        const { data } = await sapiClient.scrape({ scrapingParams });
+        const { markdown, isTruncated } = _a.transformResponse({
+            html: data,
+            tokenLimit: scrapingParams.tokenLimit,
+            shouldShowFullResponse: scrapingParams.fullResponse,
+        });
+        return {
+            content: [
+                {
+                    type: 'text',
+                    text: isTruncated
+                        ? `The website content is over ${_a.LARGE_CONTENT_SYMBOL_COUNT} symbols, therefore, the content has been truncated. If you wish to obtain the full response, just say "full response". Alternatively, you say a specific token limit in the prompt.`
+                        : 'Full website content retrieved.',
+                },
+                {
+                    type: 'text',
+                    text: markdown,
+                },
+            ],
+        };
+    });
+};
+//# sourceMappingURL=scrape-as-markdown-tool.js.map

package/build/tools/scrape-as-markdown-tool.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"scrape-as-markdown-tool.js","sourceRoot":"","sources":["../../src/tools/scrape-as-markdown-tool.ts"],"names":[],"mappings":";;;;;;;AAAA,8CAAoB;AAIpB,2DAAsD;AACtD,gDAAkG;AAElG,MAAa,oBAAoB;;AAAjC,oDA4FC;;AA3FQ,+CAA0B,GAAG,KAAM,AAAT,CAAU;AAEpC,wCAAmB,GAAG,CAAC,OAAe,EAAE,EAAE;IAC/C,OAAO,OAAO,CAAC,MAAM,GAAG,EAAI,CAAC,0BAA0B,CAAC;AAC1D,CAAC,AAFyB,CAExB;AAEK,2CAAsB,GAAG,CAAC,EAC/B,OAAO,EACP,sBAAsB,GAIvB,EAAE,EAAE;IACH,IAAI,sBAAsB,EAAE,CAAC;QAC3B,OAAO,KAAK,CAAC;IACf,CAAC;IAED,OAAO,EAAI,CAAC,mBAAmB,CAAC,OAAO,CAAC,CAAC;AAC3C,CAAC,AAZ4B,CAY3B;AAEK,qCAAgB,GAAG,CAAC,EAAE,OAAO,EAAE,KAAK,EAAsC,EAAE,EAAE;IACnF,OAAO,OAAO,CAAC,SAAS,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC;AACrC,CAAC,AAFsB,CAErB;AAEK,sCAAiB,GAAG,CAAC,EAC1B,IAAI,EACJ,UAAU,EACV,sBAAsB,GAKvB,EAAE,EAAE;IACH,MAAM,QAAQ,GAAG,qCAAgB,CAAC,SAAS,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;IAEtD,IAAI,EAAI,CAAC,sBAAsB,CAAC,EAAE,OAAO,EAAE,QAAQ,EAAE,sBAAsB,EAAE,CAAC,EAAE,CAAC;QAC/E,MAAM,SAAS,GAAG,EAAI,CAAC,gBAAgB,CAAC;YACtC,OAAO,EAAE,QAAQ;YACjB,KAAK,EAAE,UAAU,IAAI,EAAI,CAAC,0BAA0B;SACrD,CAAC,CAAC;QAEH,OAAO,EAAE,QAAQ,EAAE,SAAS,EAAE,WAAW,EAAE,IAAI,EAAE,CAAC;IACpD,CAAC;IAED,OAAO,EAAE,QAAQ,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;AAC1C,CAAC,AArBuB,CAqBtB;AAEK,6BAAQ,GAAG,CAAC,EACjB,MAAM,EACN,UAAU,GAIX,EAAE,EAAE;IACH,MAAM,CAAC,IAAI,CACT,oBAAoB,EACpB,wEAAwE,EACxE;QACE,GAAG,EAAE,aAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,CAAC,eAAe,CAAC;QACzC,GAAG,EAAE,kBAAM;QACX,MAAM,EAAE,qBAAS;QACjB,QAAQ,EAAE,uBAAW;QACrB,UAAU,EAAE,yBAAa;QACzB,YAAY,EAAE,2BAAe;KAC9B,EACD,KAAK,EAAE,cAAiC,EAAE,EAAE;QAC1C,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,UAAU,CAAC,MAAM,CAAC,EAAE,cAAc,EAAE,CAAC,CAAC;QAE7D,MAAM,EAAE,QAAQ,EAAE,WAAW,EAAE,GAAG,EAAI,CAAC,iBAAiB,CAAC;YACvD,IAAI,EAAE,IAAI;YACV,UAAU,EAAE,cAAc,CAAC,UAAU;YACrC,sBAAsB,EAAE,cAAc,CAAC,YAAY;SACpD,CAAC,CAAC;QAEH,OAAO;YACL,OAAO,EAAE;gBACP;oBACE,IAAI,EAAE,MAAM;oBACZ,IAAI,EAAE,WAAW;wBACf,CAAC,CAAC,+BAA+B,EAAI,CAAC,0BAA0B,sLAAsL;wBACtP,CAAC,CAAC,iCAAiC;iBACtC;gBACD;oBACE,IAAI,EAAE,MAAM;oBACZ,IAAI,EAAE,QAAQ;iBACf;aACF;SACF,CAAC;IACJ,CAAC,CACF,CAAC;AACJ,CAAC,AA3Cc,CA2Cb"}

package/build/tools/scrape-tool.js

@@ -0,0 +1,70 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+var _a;
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.ScrapeTool = void 0;
+const zod_1 = __importDefault(require("zod"));
+const node_html_markdown_1 = require("node-html-markdown");
+const zod_types_1 = require("../zod/zod-types");
+class ScrapeTool {
+}
+exports.ScrapeTool = ScrapeTool;
+_a = ScrapeTool;
+ScrapeTool.LARGE_CONTENT_SYMBOL_COUNT = 10000;
+ScrapeTool.isResponseOverLimit = (content) => {
+    return content.length > _a.LARGE_CONTENT_SYMBOL_COUNT;
+};
+ScrapeTool.shouldTruncateResponse = ({ content, shouldShowFullResponse, }) => {
+    if (shouldShowFullResponse) {
+        return false;
+    }
+    return _a.isResponseOverLimit(content);
+};
+ScrapeTool.truncateResponse = ({ content, limit }) => {
+    return content.substring(0, limit);
+};
+ScrapeTool.transformResponse = ({ html, tokenLimit, shouldShowFullResponse, }) => {
+    const markdown = node_html_markdown_1.NodeHtmlMarkdown.translate(html, {});
+    if (_a.shouldTruncateResponse({ content: markdown, shouldShowFullResponse })) {
+        const truncated = _a.truncateResponse({
+            content: markdown,
+            limit: tokenLimit || _a.LARGE_CONTENT_SYMBOL_COUNT,
+        });
+        return { markdown: truncated, isTruncated: true };
+    }
+    return { markdown, isTruncated: false };
+};
+ScrapeTool.register = ({ server, sapiClient, }) => {
+    server.tool('scrape', 'Scrape the contents of a website', {
+        url: zod_1.default.string().describe('URL to scrape'),
+        geo: zod_types_1.zodGeo,
+        locale: zod_types_1.zodLocale,
+        jsRender: zod_types_1.zodJsRender,
+        tokenLimit: zod_types_1.zodTokenLimit,
+        fullResponse: zod_types_1.zodFullResponse,
+    }, async (scrapingParams) => {
+        const { data } = await sapiClient.scrape({ scrapingParams });
+        const { markdown, isTruncated } = _a.transformResponse({
+            html: data,
+            tokenLimit: scrapingParams.tokenLimit,
+            shouldShowFullResponse: scrapingParams.fullResponse,
+        });
+        return {
+            content: [
+                {
+                    type: 'text',
+                    text: isTruncated
+                        ? `The website content is over ${_a.LARGE_CONTENT_SYMBOL_COUNT} symbols, therefore, I have truncated the content. If you wish to obtain the full response, just say "full response". Alternatively, you can tell me a specific token limit.`
+                        : 'Full website content retrieved.',
+                },
+                {
+                    type: 'text',
+                    text: markdown,
+                },
+            ],
+        };
+    });
+};
+//# sourceMappingURL=scrape-tool.js.map

package/build/tools/scrape-tool.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"scrape-tool.js","sourceRoot":"","sources":["../../src/tools/scrape-tool.ts"],"names":[],"mappings":";;;;;;;AAAA,8CAAoB;AAIpB,2DAAsD;AACtD,gDAAkG;AAElG,MAAa,UAAU;;AAAvB,gCA4FC;;AA3FQ,qCAA0B,GAAG,KAAM,AAAT,CAAU;AAEpC,8BAAmB,GAAG,CAAC,OAAe,EAAE,EAAE;IAC/C,OAAO,OAAO,CAAC,MAAM,GAAG,EAAI,CAAC,0BAA0B,CAAC;AAC1D,CAAC,AAFyB,CAExB;AAEK,iCAAsB,GAAG,CAAC,EAC/B,OAAO,EACP,sBAAsB,GAIvB,EAAE,EAAE;IACH,IAAI,sBAAsB,EAAE,CAAC;QAC3B,OAAO,KAAK,CAAC;IACf,CAAC;IAED,OAAO,EAAI,CAAC,mBAAmB,CAAC,OAAO,CAAC,CAAC;AAC3C,CAAC,AAZ4B,CAY3B;AAEK,2BAAgB,GAAG,CAAC,EAAE,OAAO,EAAE,KAAK,EAAsC,EAAE,EAAE;IACnF,OAAO,OAAO,CAAC,SAAS,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC;AACrC,CAAC,AAFsB,CAErB;AAEK,4BAAiB,GAAG,CAAC,EAC1B,IAAI,EACJ,UAAU,EACV,sBAAsB,GAKvB,EAAE,EAAE;IACH,MAAM,QAAQ,GAAG,qCAAgB,CAAC,SAAS,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;IAEtD,IAAI,EAAI,CAAC,sBAAsB,CAAC,EAAE,OAAO,EAAE,QAAQ,EAAE,sBAAsB,EAAE,CAAC,EAAE,CAAC;QAC/E,MAAM,SAAS,GAAG,EAAI,CAAC,gBAAgB,CAAC;YACtC,OAAO,EAAE,QAAQ;YACjB,KAAK,EAAE,UAAU,IAAI,EAAI,CAAC,0BAA0B;SACrD,CAAC,CAAC;QAEH,OAAO,EAAE,QAAQ,EAAE,SAAS,EAAE,WAAW,EAAE,IAAI,EAAE,CAAC;IACpD,CAAC;IAED,OAAO,EAAE,QAAQ,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;AAC1C,CAAC,AArBuB,CAqBtB;AAEK,mBAAQ,GAAG,CAAC,EACjB,MAAM,EACN,UAAU,GAIX,EAAE,EAAE;IACH,MAAM,CAAC,IAAI,CACT,QAAQ,EACR,kCAAkC,EAClC;QACE,GAAG,EAAE,aAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,CAAC,eAAe,CAAC;QACzC,GAAG,EAAE,kBAAM;QACX,MAAM,EAAE,qBAAS;QACjB,QAAQ,EAAE,uBAAW;QACrB,UAAU,EAAE,yBAAa;QACzB,YAAY,EAAE,2BAAe;KAC9B,EACD,KAAK,EAAE,cAAiC,EAAE,EAAE;QAC1C,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,UAAU,CAAC,MAAM,CAAC,EAAE,cAAc,EAAE,CAAC,CAAC;QAE7D,MAAM,EAAE,QAAQ,EAAE,WAAW,EAAE,GAAG,EAAI,CAAC,iBAAiB,CAAC;YACvD,IAAI,EAAE,IAAI;YACV,UAAU,EAAE,cAAc,CAAC,UAAU;YACrC,sBAAsB,EAAE,cAAc,CAAC,YAAY;SACpD,CAAC,CAAC;QAEH,OAAO;YACL,OAAO,EAAE;gBACP;oBACE,IAAI,EAAE,MAAM;oBACZ,IAAI,EAAE,WAAW;wBACf,CAAC,CAAC,+BAA+B,EAAI,CAAC,0BAA0B,8KAA8K;wBAC9O,CAAC,CAAC,iCAAiC;iBACtC;gBACD;oBACE,IAAI,EAAE,MAAM;oBACZ,IAAI,EAAE,QAAQ;iBACf;aACF;SACF,CAAC;IACJ,CAAC,CACF,CAAC;AACJ,CAAC,AA3Cc,CA2Cb"}

package/build/types.js

@@ -0,0 +1,3 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+//# sourceMappingURL=types.js.map

package/build/types.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"types.js","sourceRoot":"","sources":["../src/types.ts"],"names":[],"mappings":""}

package/build/utils.js

@@ -0,0 +1,21 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.removeKeyFromNestedObject = void 0;
+const removeKeyFromNestedObject = ({ obj, keyToRemove, }) => {
+    if (typeof obj !== 'object' || obj === null) {
+        return obj;
+    }
+    if (Array.isArray(obj)) {
+        return obj.map(item => (0, exports.removeKeyFromNestedObject)({ obj: item, keyToRemove }));
+    }
+    const newObj = {};
+    for (const key in obj) {
+        if (key === keyToRemove) {
+            continue;
+        }
+        newObj[key] = (0, exports.removeKeyFromNestedObject)({ obj: obj[key], keyToRemove });
+    }
+    return newObj;
+};
+exports.removeKeyFromNestedObject = removeKeyFromNestedObject;
+//# sourceMappingURL=utils.js.map

package/build/utils.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"utils.js","sourceRoot":"","sources":["../src/utils.ts"],"names":[],"mappings":";;;AAAO,MAAM,yBAAyB,GAAG,CAAC,EACxC,GAAG,EACH,WAAW,GAIZ,EAAU,EAAE;IACX,IAAI,OAAO,GAAG,KAAK,QAAQ,IAAI,GAAG,KAAK,IAAI,EAAE,CAAC;QAC5C,OAAO,GAAG,CAAC;IACb,CAAC;IAED,IAAI,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC;QACvB,OAAO,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,IAAA,iCAAyB,EAAC,EAAE,GAAG,EAAE,IAAI,EAAE,WAAW,EAAE,CAAC,CAAC,CAAC;IAChF,CAAC;IAED,MAAM,MAAM,GAAG,EAAE,CAAC;IAElB,KAAK,MAAM,GAAG,IAAI,GAAG,EAAE,CAAC;QACtB,IAAI,GAAG,KAAK,WAAW,EAAE,CAAC;YACxB,SAAS;QACX,CAAC;QAED,MAAM,CAAC,GAAG,CAAC,GAAG,IAAA,iCAAyB,EAAC,EAAE,GAAG,EAAE,GAAG,CAAC,GAAG,CAAC,EAAE,WAAW,EAAE,CAAC,CAAC;IAC1E,CAAC;IAED,OAAO,MAAM,CAAC;AAChB,CAAC,CAAC;AA1BW,QAAA,yBAAyB,6BA0BpC"}

package/build/zod/zod-types.js

@@ -0,0 +1,25 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.zodFullResponse = exports.zodTokenLimit = exports.zodJsRender = exports.zodLocale = exports.zodGeo = void 0;
+const zod_1 = __importDefault(require("zod"));
+exports.zodGeo = zod_1.default
+    .string()
+    .describe('Geolocation of the desired request, expressed as a country name')
+    .optional();
+exports.zodLocale = zod_1.default.string().describe('Locale of the desired request').optional();
+exports.zodJsRender = zod_1.default
+    .boolean()
+    .describe('Should the request be opened in a headless browser, false by default')
+    .optional();
+exports.zodTokenLimit = zod_1.default
+    .number()
+    .describe(`The number of tokens to return in the response - anything above this limit will be truncated`)
+    .optional();
+exports.zodFullResponse = zod_1.default
+    .boolean()
+    .describe(`If true, content will not be truncated`)
+    .optional();
+//# sourceMappingURL=zod-types.js.map

package/build/zod/zod-types.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"zod-types.js","sourceRoot":"","sources":["../../src/zod/zod-types.ts"],"names":[],"mappings":";;;;;;AAAA,8CAAoB;AAEP,QAAA,MAAM,GAAG,aAAC;KACpB,MAAM,EAAE;KACR,QAAQ,CAAC,iEAAiE,CAAC;KAC3E,QAAQ,EAAE,CAAC;AAED,QAAA,SAAS,GAAG,aAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,CAAC,+BAA+B,CAAC,CAAC,QAAQ,EAAE,CAAC;AAE5E,QAAA,WAAW,GAAG,aAAC;KACzB,OAAO,EAAE;KACT,QAAQ,CAAC,sEAAsE,CAAC;KAChF,QAAQ,EAAE,CAAC;AAED,QAAA,aAAa,GAAG,aAAC;KAC3B,MAAM,EAAE;KACR,QAAQ,CACP,8FAA8F,CAC/F;KACA,QAAQ,EAAE,CAAC;AAED,QAAA,eAAe,GAAG,aAAC;KAC7B,OAAO,EAAE;KACT,QAAQ,CAAC,wCAAwC,CAAC;KAClD,QAAQ,EAAE,CAAC"}

package/package.json

@@ -0,0 +1,36 @@
+{
+  "name": "@decodo/mcp-server",
+  "version": "1.0.0",
+  "description": "",
+  "bin": {
+    "decodo-mcp": "./build/index.js"
+  },
+  "main": "index.js",
+  "scripts": {
+    "build": "tsc && chmod 755 build/index.js",
+    "dev": "pnpm run build && mcp-inspector",
+    "test": "jest"
+  },
+  "keywords": [],
+  "author": "",
+  "license": "ISC",
+  "files": [
+    "build"
+  ],
+  "devDependencies": {
+    "@modelcontextprotocol/inspector": "^0.14.0",
+    "@types/jest": "^30.0.0",
+    "@types/node": "^24.0.0",
+    "jest": "^30.0.0",
+    "mcps-logger": "^1.0.0",
+    "ts-jest": "^29.4.0",
+    "typescript": "^5.8.3",
+    "zod": "^3.25.57"
+  },
+  "dependencies": {
+    "@modelcontextprotocol/sdk": "^1.12.3",
+    "axios": "^1.9.0",
+    "dotenv": "^16.5.0",
+    "node-html-markdown": "^1.3.0"
+  }
+}