scrapeunblocker-mcp 0.5.0 → 0.6.0

package/README.md

@@ -12,7 +12,8 @@ You bring your **own** API key. Nothing is shared or proxied through us.
 
 | Tool | What it does |
 |------|--------------|
-| `fetch_html` | Fetch the fully rendered HTML of a URL (optionally after running interactive browser `steps`). |
+| `fetch_html` | Fetch the fully rendered HTML of a URL. |
+| `browse_page` | Run interactive browser `steps` on a page (click, type, select, scroll, wait), then return the resulting HTML. Not read-only: it acts on the website. |
 | `list_elements` | List a page's notable elements with a ready-to-use selector for each (selector discovery). |
 | `fetch_parsed` | Fetch a page and return AI-parsed structured JSON. |
 | `google_search` | Run a Google search and return organic results as JSON. |
@@ -43,7 +44,7 @@ Each run is billed like an API request.
 
 Some pages only reveal what you need after you interact with them - accept a
 cookie banner, click a tab, type into a search box and submit, or scroll to
-trigger lazy loading. Pass an optional `steps` array to `fetch_html` and those
+trigger lazy loading. Pass a `steps` array to `browse_page` and those
 actions run in a **real browser**, in order, after the page loads; the resulting
 HTML is then returned.
 
@@ -60,8 +61,9 @@ Available actions:
 | `press_key` | `value` (`Enter`, `Tab`, `Escape`, `Backspace`, `Delete`, `Space`, `Arrow*`, `Home`, `End`, `PageUp`, `PageDown`) |
 | `scroll` | `value` (`"bottom"` or a pixel offset) |
 
-Steps are **not idempotent** - they run once per call. If a step fails,
-`fetch_html` returns which step failed, why, and the page HTML at that moment so
+Steps are **not idempotent** - they run once per call - and they act on the
+website, so `browse_page` is the one tool not marked read-only. If a step fails,
+`browse_page` returns which step failed, why, and the page HTML at that moment so
 you can fix the selector and retry.
 
 ### List elements (discover selectors first)
@@ -76,7 +78,7 @@ you can fix the selector and retry.
 ```
 
 The natural workflow is **discover, then act**: call `list_elements` to find the
-selectors you need, then pass matching `steps` to `fetch_html` to click/type/
+selectors you need, then pass matching `steps` to `browse_page` to click/type/
 select and capture the resulting HTML.
 
 ## Get an API key
@@ -140,7 +142,7 @@ SCRAPEUNBLOCKER_KEY=your_api_key_here npx -y scrapeunblocker-mcp
 
 - "Fetch the HTML of https://www.example-shop.com/product/123 and list the price."
 - "This page keeps blocking me: <url>. Use fetch_html to get it."
-- "List the elements on <url>, then use fetch_html steps to type 'laptop' into the search box, press Enter, wait for the results, and give me the HTML."
+- "List the elements on <url>, then use browse_page to type 'laptop' into the search box, press Enter, wait for the results, and give me the HTML."
 - "Search Google for 'best running shoes 2026' and give me the top 5 links."
 - "Read https://en.wikipedia.org/wiki/Web_scraping as Markdown and summarise it in five bullets."
 - "Search the web for the current population of Vilnius and cite the pages you read."

package/dist/index.js

@@ -5979,7 +5979,7 @@ ${body}`);
 }
 
 // src/index.ts
-var VERSION = "0.5.0";
+var VERSION = "0.6.0";
 var DEFAULT_BASE_URL = "https://api.scrapeunblocker.com";
 var API_KEY_HEADER = "x-scrapeunblocker-key";
 function resolveApiKey() {
@@ -6137,7 +6137,7 @@ server.registerTool(
   {
     title: "Fetch page HTML",
     annotations: { title: "Fetch page HTML", readOnlyHint: true, openWorldHint: true, destructiveHint: false },
-    description: "Fetch the fully rendered HTML of any web page through ScrapeUnblocker, bypassing anti-bot protection (Cloudflare, DataDome, PerimeterX, Akamai, Shape). Use this when a normal fetch is blocked (403/429, captcha, 'access denied') or when the page needs a real browser to render. Returns the raw HTML as text.\n\nFor pages that need interaction (accept a cookie banner, click a tab, type into a search box, scroll to trigger lazy loading) pass `steps`: an ordered list of browser actions run in a real browser AFTER the page loads, then the resulting HTML is returned. Workflow: first call the `list_elements` tool to discover the real selectors on the page, then build `steps` against them. Steps are NOT idempotent - they run once per call. If a step fails, this tool returns which step failed, why, and the page HTML at that moment so you can fix the selector and retry.",
+    description: "Fetch the fully rendered HTML of any web page through ScrapeUnblocker, bypassing anti-bot protection (Cloudflare, DataDome, PerimeterX, Akamai, Shape). Use this when a normal fetch is blocked (403/429, captcha, 'access denied') or when the page needs a real browser to render. Returns the raw HTML as text. To read or summarise a page, fetch_markdown is much smaller. To click, type or scroll before capturing, use browse_page.",
     inputSchema: {
       url: z3.string().url().describe("The absolute URL to fetch (http/https)."),
       proxy_country: z3.string().length(2).optional().describe(
@@ -6150,24 +6150,54 @@ server.registerTool(
         `The CSS selector or JS expression paired with wait_method (e.g. '#price' or 'document.readyState==="complete"').`
       ),
       method_timeout_seconds: z3.number().int().positive().optional().describe("Cap in seconds for the render-wait method."),
-      sleep_seconds: z3.number().positive().optional().describe("Extra seconds to wait after load before capturing the HTML."),
-      steps: z3.array(stepSchema).optional().describe(
-        "Optional ordered browser actions to run in a real browser after the page loads, before the HTML is captured. Use `list_elements` first to find selectors. Runs once (not idempotent)."
-      )
+      sleep_seconds: z3.number().positive().optional().describe("Extra seconds to wait after load before capturing the HTML.")
+    }
+  },
+  async (args) => {
+    try {
+      const html = await client().getPageSource(args.url, {
+        proxyCountry: args.proxy_country,
+        method: args.wait_method,
+        value: args.wait_value,
+        methodTimeout: args.method_timeout_seconds,
+        timeSleep: args.sleep_seconds
+      });
+      return { content: [{ type: "text", text: html }] };
+    } catch (err) {
+      return errorResult(args.url, err);
+    }
+  }
+);
+server.registerTool(
+  "browse_page",
+  {
+    title: "Interact with a page, then capture it",
+    annotations: {
+      title: "Interact with a page, then capture it",
+      readOnlyHint: false,
+      destructiveHint: true,
+      idempotentHint: false,
+      openWorldHint: true
+    },
+    description: "Load a web page in a real browser through ScrapeUnblocker, run an ordered list of browser actions on it (accept a cookie banner, click a tab, type into a search box, choose an option, press a key, scroll to trigger lazy loading, wait), then return the resulting HTML. Workflow: first call the `list_elements` tool to discover the real selectors on the page, then build `steps` against them. Steps act on the website and are NOT idempotent - they run once per call; never use them to buy, post, delete or change anything unless the user asked for exactly that. If a step fails, this tool returns which step failed, why, and the page HTML at that moment so you can fix the selector and retry. Billed as one request.",
+    inputSchema: {
+      url: z3.string().url().describe("The absolute URL to open (http/https)."),
+      steps: z3.array(stepSchema).min(1).describe(
+        "Ordered browser actions to run in a real browser after the page loads, before the HTML is captured. Use `list_elements` first to find selectors. Runs once (not idempotent)."
+      ),
+      proxy_country: z3.string().length(2).optional().describe(
+        "Optional ISO 3166-1 alpha-2 country code to route through, e.g. 'US', 'GB', 'DE'."
+      ),
+      wait_method: z3.enum(["css", "js"]).optional().describe(
+        "Optional render-wait before the steps run: 'css' waits for a selector, 'js' waits for a JS expression to be truthy."
+      ),
+      wait_value: z3.string().optional().describe("The CSS selector or JS expression paired with wait_method."),
+      method_timeout_seconds: z3.number().int().positive().optional().describe("Cap in seconds for the render-wait method."),
+      sleep_seconds: z3.number().positive().optional().describe("Extra seconds to wait after load.")
     }
   },
   async (args) => {
     try {
-      if (!args.steps || args.steps.length === 0) {
-        const html = await client().getPageSource(args.url, {
-          proxyCountry: args.proxy_country,
-          method: args.wait_method,
-          value: args.wait_value,
-          methodTimeout: args.method_timeout_seconds,
-          timeSleep: args.sleep_seconds
-        });
-        return { content: [{ type: "text", text: html }] };
-      }
       const res = await getPageSourceRaw({
         url: args.url,
         proxy_country: args.proxy_country,
@@ -6223,7 +6253,7 @@ server.registerTool(
   {
     title: "List page elements (selector discovery)",
     annotations: { title: "List page elements (selector discovery)", readOnlyHint: true, openWorldHint: true, destructiveHint: false },
-    description: "Load a page through ScrapeUnblocker and return a JSON list of its notable elements (links, inputs, buttons, selects, etc.) with a ready-to-use `selector` for each, plus tag, text and useful attributes (name, id, type, placeholder, aria_label, href, ...). Read-only and does not interact with the page. This is the discovery half of interactive scraping: call `list_elements` to find the selectors you need, then pass matching `steps` to `fetch_html` to click/type/select and capture the resulting HTML.",
+    description: "Load a page through ScrapeUnblocker and return a JSON list of its notable elements (links, inputs, buttons, selects, etc.) with a ready-to-use `selector` for each, plus tag, text and useful attributes (name, id, type, placeholder, aria_label, href, ...). Read-only and does not interact with the page. This is the discovery half of interactive scraping: call `list_elements` to find the selectors you need, then pass matching `steps` to `browse_page` to click/type/select and capture the resulting HTML.",
     inputSchema: {
       url: z3.string().url().describe("The absolute URL whose elements you want to list."),
       proxy_country: z3.string().length(2).optional().describe("Optional ISO country code to route through, e.g. 'US'.")

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "scrapeunblocker-mcp",
-  "version": "0.5.0",
+  "version": "0.6.0",
   "mcpName": "com.scrapeunblocker/scrapeunblocker-mcp",
   "description": "Model Context Protocol (MCP) server for ScrapeUnblocker - lets Claude fetch any web page's HTML (or AI-parsed JSON) and run 100+ site plugins that return structured JSON, through the ScrapeUnblocker anti-bot API, using your own API key.",
   "keywords": [