@spicrawl/mcp 0.1.7 → 0.1.8

package/CHANGELOG.md

@@ -3,6 +3,13 @@
 All notable changes to `@spicrawl/mcp` are documented here. This project follows
 [Semantic Versioning](https://semver.org/).
 
+## 0.1.8 — 2026-10-15
+
+Listing copy and keywords; no code changes.
+
+- `package.json`, `server.json` and the Cursor plugin manifest: a search-led description and keywords, so npm search, the MCP Registry and cursor.directory match what people look for.
+- README: a first paragraph that answers how to let Claude or Cursor scrape websites, a use cases list, an FAQ and install links for each directory.
+
 ## 0.1.7 — 2026-10-13
 
 - Every tool now declares an MCP `outputSchema` and returns matching `structuredContent` next to the same text it always returned, so a client can read fields such as `credits`, `status` and `next_cursor` without parsing text. The schemas follow the API's OpenAPI and are deliberately open: every field but an id is optional, fields the API adds later pass through (`additionalProperties` is true), and values such as a job or session `status` and an `engine` are plain strings with the known values in the description, because a client rejects a result that fails its schema. A test fails if a schema ever uses an enum, a format or a closed object. An error is still a tool error with no `structuredContent`.

package/README.md

@@ -1,14 +1,21 @@
-# Spicrawl MCP Server
+# Spicrawl MCP Server: web scraping for Claude, Cursor and AI agents
 
-The Spicrawl MCP server is a Model Context Protocol server that gives AI agents in Claude Code, Claude Desktop, Cursor, VS Code, Windsurf, Gemini CLI and Codex web scraping to Markdown, structured data extraction, screenshots, browser sessions and batch scraping as 25 native tools.
+To let Claude, Cursor or another AI agent scrape websites, connect the Spicrawl MCP server. Add the hosted endpoint `https://mcp.spicrawl.com/mcp` (or run `npx -y @spicrawl/mcp` locally) with a Spicrawl API key, and the agent can call `spicrawl_scrape` to read any URL as Markdown, text, HTML or JSON. It renders JavaScript pages, extracts structured data, scrapes up to 10,000 URLs in one batch job, takes screenshots and keeps logged-in browser sessions, all as native `spicrawl_*` tools in Claude Code, Claude Desktop, Cursor, VS Code, Windsurf, Gemini CLI and Codex. The free tier includes 1,000 credits per month.
 
 [![npm version](https://img.shields.io/npm/v/@spicrawl/mcp.svg)](https://www.npmjs.com/package/@spicrawl/mcp)
 [![npm downloads](https://img.shields.io/npm/dm/@spicrawl/mcp.svg)](https://www.npmjs.com/package/@spicrawl/mcp)
 [![license](https://img.shields.io/npm/l/@spicrawl/mcp.svg)](https://github.com/OfficialSpicrawl/mcp/blob/main/LICENSE)
+[![smithery badge](https://smithery.ai/badge/spicrawl/web-scraper)](https://smithery.ai/servers/spicrawl/web-scraper)
 [![CI](https://github.com/OfficialSpicrawl/mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/OfficialSpicrawl/mcp/actions/workflows/ci.yml)
+[![MCP Registry](https://img.shields.io/badge/MCP_Registry-io.github.OfficialSpicrawl%2Fmcp-blue)](https://registry.modelcontextprotocol.io/v0.1/servers?search=io.github.OfficialSpicrawl/mcp)
+[![Glama](https://img.shields.io/badge/Glama-MCP_connector-blue)](https://glama.ai/mcp/connectors/com.spicrawl.mcp)
+[![Dify Marketplace](https://img.shields.io/badge/Dify-Marketplace_plugin-blue)](https://marketplace.dify.ai/plugin/spicrawl/spicrawl)
+[![Cursor Directory](https://img.shields.io/badge/Cursor-Directory-blue)](https://cursor.directory/plugins/spicrawl)
 
 **[Docs](https://docs.spicrawl.com/agents/mcp)** · **[Get an API key](https://app.spicrawl.com/signup)** · **[TypeScript SDK](https://github.com/OfficialSpicrawl/sdk)** · **[CLI](https://github.com/OfficialSpicrawl/cli)** · **[Agent plugins](https://github.com/OfficialSpicrawl/agent-plugins)** · **[Changelog](https://github.com/OfficialSpicrawl/mcp/blob/main/CHANGELOG.md)**
 
+**Install from:** [npm](https://www.npmjs.com/package/@spicrawl/mcp) · [Smithery](https://smithery.ai/servers/spicrawl/web-scraper) · [Glama](https://glama.ai/mcp/connectors/com.spicrawl.mcp) · [MCP Registry](https://registry.modelcontextprotocol.io/v0.1/servers?search=io.github.OfficialSpicrawl/mcp) · [Dify plugin](https://marketplace.dify.ai/plugin/spicrawl/spicrawl) · [Cursor Directory](https://cursor.directory/plugins/spicrawl)
+
 <p>
   <a href="https://cursor.com/en/install-mcp?name=spicrawl&config=eyJ1cmwiOiJodHRwczovL21jcC5zcGljcmF3bC5jb20vbWNwIiwiaGVhZGVycyI6eyJBdXRob3JpemF0aW9uIjoiQmVhcmVyICR7ZW52OlNQSUNSQVdMX0FQSV9LRVl9In19"><img src="https://cursor.com/deeplink/mcp-install-dark.svg" alt="Add Spicrawl MCP server to Cursor" height="32" /></a>
   <a href="https://vscode.dev/redirect/mcp/install?name=spicrawl&inputs=%5B%7B%22type%22%3A%22promptString%22%2C%22id%22%3A%22spicrawl-api-key%22%2C%22description%22%3A%22Spicrawl%20API%20key%22%2C%22password%22%3Atrue%7D%5D&config=%7B%22type%22%3A%22http%22%2C%22url%22%3A%22https%3A%2F%2Fmcp.spicrawl.com%2Fmcp%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7Binput%3Aspicrawl-api-key%7D%22%7D%7D"><img src="https://img.shields.io/badge/VS_Code-Install_Spicrawl_MCP-0098FF?style=for-the-badge&logo=visualstudiocode&logoColor=white" alt="Install Spicrawl MCP server in VS Code" height="32" /></a>
@@ -35,7 +42,7 @@ The Spicrawl MCP server is a Model Context Protocol server that gives AI agents
 
 4. Ask your agent: *"Use Spicrawl to read https://example.com/pricing as markdown and list each plan with its price."* It calls `spicrawl_scrape` and gets the page back as Markdown.
 
-Both transports serve the same 25 `spicrawl_*` tools, and every call runs under your key with the same scopes, limits and credits as a direct API call.
+Both transports serve the same `spicrawl_*` tools (25 are defined; the hosted endpoint lists 24 until remote browsers launch, see [Tools](#tools)), and every call runs under your key with the same scopes, limits and credits as a direct API call.
 
 ## What it does
 
@@ -44,24 +51,33 @@ Both transports serve the same 25 `spicrawl_*` tools, and every call runs under
 - **Extract structured data** with a CSS/XPath selector map (`extract`) or from the JSON-LD, OpenGraph, Twitter Card and microdata a page publishes (`autoparse`).
 - **Drive the page** before capture with 8 typed browser actions: `click`, `fill`, `wait_for`, `wait_for_navigation`, `scroll` (including infinite scroll), `select`, `evaluate` and `screenshot`.
 - **Return screenshots and PDFs the model can see**: screenshots arrive as MCP image blocks (up to 5 MB of base64 each) and printed pages as `application/pdf` resource blocks (up to 10 MB).
-- **Batch scrape thousands of URLs** as one async job, up to 10,000 items per call, with 9 tools to submit, poll, page through results, retry, cancel and keep a job open for crawling.
+- **Batch scrape thousands of URLs** as one async job, up to 10,000 items per call, with 9 tools to submit, poll, page through results, retry, cancel and keep a job open so you can append URLs as you discover them.
 - **Keep a logged-in browser** with persistent sessions: cookies, storage and a fixed rendering tier reused across scrapes.
 - **Answer "what did that cost?"** from usage and request-log tools, and **read the Spicrawl docs** from inside the agent.
 
+## Use cases
+
+- **Scrape docs into a RAG knowledge base.** Read documentation pages as clean Markdown with `spicrawl_scrape` (`main_content_only` is on by default for Markdown), or queue up to 10,000 URLs in one `spicrawl_batch_submit` job. Spicrawl returns the Markdown; chunking and embedding are yours. See [the knowledge-base recipe](https://docs.spicrawl.com/use-cases/ai-knowledge-base).
+- **Monitor prices.** Pull a product's title, price and availability with an `extract` selector map or `autoparse`, and set `cache: false` for a fresh reading. A plain-fetch check costs 1 credit, a rendered one 3. Run it on your own schedule. See [price monitoring](https://docs.spicrawl.com/use-cases/price-monitoring).
+- **Extract job postings.** Collect listings from job boards, including infinite-scroll pages (`render: true` with a `scroll` action), as structured fields. See [job postings](https://docs.spicrawl.com/use-cases/job-postings).
+- **Read JavaScript-heavy pages.** When a plain fetch returns an empty shell, call `spicrawl_scrape` with `render: true` (3 credits) so the page builds itself first.
+
+Check each site's terms and `robots.txt` before you scrape it.
+
 ## Which tool to use
 
 - If you need one page's content or data now, use `spicrawl_scrape`.
 - If the page is blank or a skeleton without JavaScript, call `spicrawl_scrape` again with `render: true`.
 - If you know the page's structure, use `extract` with CSS selectors; if you don't, try `autoparse`.
 - If you have more than about 20 URLs, use `spicrawl_batch_submit`, then `spicrawl_batch_status` and `spicrawl_batch_results`.
-- If you discover URLs as you go (crawling), submit with `open: true`, append with `spicrawl_batch_add_items` and finish with `spicrawl_batch_close`.
+- If you discover URLs as you go, submit with `open: true`, append with `spicrawl_batch_add_items` and finish with `spicrawl_batch_close`.
 - If you must log in first, use `spicrawl_session_create` and pass its `id` as `session_id` to `spicrawl_scrape`.
 - If a call failed and you need to know why, use `spicrawl_request_get` with the request id, or `spicrawl_requests_list` with `only_errors`.
 - If you are unsure of a parameter or an error code, use `spicrawl_docs_search` before guessing.
 
 ## Tools
 
-25 tools, grouped by task. Tools reject unknown arguments, so a misspelt field fails with an error naming it instead of being silently dropped. Every tool sets a `title` and explicit `readOnlyHint`, `destructiveHint` and `openWorldHint` annotations, so a client knows which calls to confirm: `spicrawl_scrape` is not read-only (its `method` and `actions` can submit forms), and `spicrawl_batch_cancel`, `spicrawl_session_release` and `spicrawl_session_delete` are destructive. Every tool also declares an `outputSchema` and returns matching `structuredContent` beside the same text it always returned, so a client can read fields (`credits`, `status`, `next_cursor`, ...) without parsing text. The schemas list what an agent can rely on and stay open: fields the API adds later pass through, and statuses and tiers are plain strings. A tool that returns a document or a JSON array (`spicrawl_docs_read`, `spicrawl_batch_task_content`) puts it under `content` in `structuredContent`. Full argument reference: [docs.spicrawl.com/agents/mcp](https://docs.spicrawl.com/agents/mcp#tools).
+25 tools, grouped by task. The hosted endpoint lists 24 until remote browsers launch: its keyless `tools/list` and the server card leave out `spicrawl_browser_connect_url` ([Coming soon](#coming-soon)), while a session opened with your key and the stdio package (`npx -y @spicrawl/mcp`) list all 25. Tools reject unknown arguments, so a misspelt field fails with an error naming it instead of being silently dropped. Every tool sets a `title` and explicit `readOnlyHint`, `destructiveHint` and `openWorldHint` annotations, so a client knows which calls to confirm: `spicrawl_scrape` is not read-only (its `method` and `actions` can submit forms), and `spicrawl_batch_cancel`, `spicrawl_session_release` and `spicrawl_session_delete` are destructive. Every tool also declares an `outputSchema` and returns matching `structuredContent` beside the same text it always returned, so a client can read fields (`credits`, `status`, `next_cursor`, ...) without parsing text. The schemas list what an agent can rely on and stay open: fields the API adds later pass through, and statuses and tiers are plain strings. A tool that returns a document or a JSON array (`spicrawl_docs_read`, `spicrawl_batch_task_content`) puts it under `content` in `structuredContent`. Full argument reference: [docs.spicrawl.com/agents/mcp](https://docs.spicrawl.com/agents/mcp#tools).
 
 ### Scrape a website to Markdown
 
@@ -114,7 +130,7 @@ Both transports serve the same 25 `spicrawl_*` tools, and every call runs under
 
 ### Coming soon
 
-`spicrawl_browser_connect_url` will mint a single-use CDP WebSocket URL for driving a Spicrawl-hosted browser from any CDP client; remote browsers are not available yet. The schemas also accept `ai_extract`, `stealth`, `extract_preset`, `premium_proxy`, `proxy_country` and `sticky_key` ahead of launch. Agents should not send them yet. To list only what works today, set `SPICRAWL_MCP_HIDE_UNAVAILABLE=1` (see [Environment variables](https://github.com/OfficialSpicrawl/mcp#environment-variables)).
+`spicrawl_browser_connect_url` will mint a single-use CDP WebSocket URL for driving a Spicrawl-hosted browser from any CDP client; remote browsers are not available yet. It is the 25th tool and the only one the hosted keyless listing and the server card leave out (24 tools there). The schemas also accept `ai_extract`, `stealth`, `extract_preset`, `premium_proxy`, `proxy_country` and `sticky_key` ahead of launch. Agents should not send them yet. To list only what works today, set `SPICRAWL_MCP_HIDE_UNAVAILABLE=1` (see [Environment variables](https://github.com/OfficialSpicrawl/mcp#environment-variables)).
 
 ## Example prompts
 
@@ -324,7 +340,7 @@ A gateway that passes a connection parameter through as a header cannot always s
 
 In Smithery, make the API key a connection parameter, map it to the `x-api-key` header, and mark it secret. Or map it to `Authorization` and send the raw key; the server accepts that when the whole value is a Spicrawl key.
 
-The key can be left optional in the connection schema (no `required`): `initialize`, `ping` and `tools/list` on `/mcp` answer without a key, with the same tool list as the server card, so a directory can list the tools. Every tool call still needs a valid key and is refused with `401` otherwise; no session is opened and nothing is billed or read without one. Say in the parameter's description that it is needed to run any tool.
+The key can be left optional in the connection schema (no `required`): `initialize`, `ping` and `tools/list` on `/mcp` answer without a key, with the same tool list as the server card (24 tools, without `spicrawl_browser_connect_url`), so a directory can list the tools. Every tool call still needs a valid key and is refused with `401` otherwise; no session is opened and nothing is billed or read without one. Say in the parameter's description that it is needed to run any tool.
 
 If more than one is sent, the first that holds a well-formed key wins, in the order `Authorization: Bearer`, raw `Authorization`, `x-api-key`, `api-key`. A key in the query string is **not** read, because URLs end up in access logs. `/chatgpt/mcp` takes OAuth access tokens only and accepts none of these.
 
@@ -354,7 +370,7 @@ A stdio-only client runs `npx -y @spicrawl/mcp` with `SPICRAWL_API_KEY` in its e
 | `SPICRAWL_DOCS_HOST` | none | both | Legacy: an origin whose docs are at `/docs`. Read only when `SPICRAWL_DOCS_URL` is unset. |
 | `SPICRAWL_MCP_ADDR` | `127.0.0.1:8090` | HTTP | Listen address, `host:port`. |
 | `SPICRAWL_MCP_SESSION_IDLE_MS` | `1800000` (30 min) | HTTP | Idle MCP sessions are dropped after this long. |
-| `SPICRAWL_MCP_HIDE_UNAVAILABLE` | off | both | `1`, `true`, `yes` or `on` leaves out what is announced but not available yet: `spicrawl_browser_connect_url` is not registered (24 tools) and every argument marked "Coming soon" is dropped from the schemas (`ai_extract`, `stealth`, `extract_preset`, `premium_proxy`, `proxy_country`, `sticky_key`, and the session `rotate_ip`, `region_pool` and `fingerprint`). Read when the server starts a session, so restart the process after changing it. |
+| `SPICRAWL_MCP_HIDE_UNAVAILABLE` | off | both | `1`, `true`, `yes` or `on` leaves out what is announced but not available yet: `spicrawl_browser_connect_url` is not registered (24 tools instead of 25) and every argument marked "Coming soon" is dropped from the schemas (`ai_extract`, `stealth`, `extract_preset`, `premium_proxy`, `proxy_country`, `sticky_key`, and the session `rotate_ip`, `region_pool` and `fingerprint`). Read when the server starts a session, so restart the process after changing it. The keyless `tools/list` and the server card behave as if it were always on. |
 | `SPICRAWL_CHATGPT_ENABLED` | off | HTTP | `1`, `true`, `yes` or `on` turns on the [ChatGPT endpoint](#chatgpt-endpoint-oauth). Off, `/chatgpt/mcp`, its metadata and the OpenAI challenge all answer `404`. |
 | `SPICRAWL_CHATGPT_RESOURCE` | `https://mcp.spicrawl.com/chatgpt/mcp` | HTTP | The OAuth resource identifier. Tokens must be issued for exactly this `aud`; the metadata is published at `/.well-known/oauth-protected-resource` + its path. |
 | `SPICRAWL_OAUTH_ISSUER` | `https://app.spicrawl.com` | HTTP | The authorization server, listed in the metadata. Tokens are introspected at `<issuer>/api/oauth/introspect`. |
@@ -377,8 +393,8 @@ npx -y @spicrawl/mcp --http
 
 - `SPICRAWL_BASE_URL` **defaults to `http://127.0.0.1:8080`** in HTTP mode (a Spicrawl API on the same host), not to the public API. Set it to `https://api.spicrawl.com` to front the public API.
 - MCP is served at `POST`/`GET`/`DELETE /mcp`; liveness at `GET /healthz` (no auth).
-- `GET /.well-known/mcp/server-card.json` (no auth, `Cache-Control: public, max-age=3600`) is a static server card for directories such as Smithery that cannot scan `/mcp` without a key: `serverInfo`, the transport, bearer auth and the tool list. The tools are read from the same definitions as `tools/list` (as with `SPICRAWL_MCP_HIDE_UNAVAILABLE`), so it matches what an authenticated client sees.
-- Without any key, `/mcp` answers only discovery: `initialize`, `ping`, the `initialized` notification and `tools/list` (the same list as the server card; no `Mcp-Session-Id` is issued and nothing is sent upstream). Any other request, a `tools/call` included, is `401`, as is a request that sends a malformed key or names a session. A session-less `GET` is `405`.
+- `GET /.well-known/mcp/server-card.json` (no auth, `Cache-Control: public, max-age=3600`) is a static server card for directories such as Smithery that cannot scan `/mcp` without a key: `serverInfo`, the transport, bearer auth and the tool list. The tools are read from the same definitions as `tools/list`, as if `SPICRAWL_MCP_HIDE_UNAVAILABLE` were on: 24 tools, without `spicrawl_browser_connect_url`. A session opened with a key lists that tool too, unless the variable is set.
+- Without any key, `/mcp` answers only discovery: `initialize`, `ping`, the `initialized` notification and `tools/list` (the same 24-tool list as the server card; no `Mcp-Session-Id` is issued and nothing is sent upstream). Any other request, a `tools/call` included, is `401`, as is a request that sends a malformed key or names a session. A session-less `GET` is `405`.
 - Each caller sends their own key as `Authorization: Bearer spicrawl_live_...`, or, for a gateway that cannot send the `Bearer ` prefix, as a raw `Authorization: spicrawl_live_...`, `x-api-key` or `api-key` header (see "Smithery and other gateways" above; never a query parameter). The key is checked against the API before a session opens (`401` if refused, `503` with `Retry-After` if the API cannot be reached), and a session is pinned to the key that opened it.
 - The server speaks plain HTTP. Put it behind TLS before sending keys across a network.
 
@@ -422,21 +438,63 @@ Installed globally (`npm install -g @spicrawl/mcp`), the command is `spicrawl-mc
 
 ## FAQ
 
-### Is there a hosted Spicrawl MCP server?
-Yes: `https://mcp.spicrawl.com/mcp`, MCP Streamable HTTP, authenticated with `Authorization: Bearer <API key>`, so there is nothing to install.
+### Which MCP server scrapes JavaScript sites?
+The Spicrawl MCP server does. Call `spicrawl_scrape` with `render: true` (3 credits) and the page loads in a browser, runs its scripts and comes back as finished Markdown, text, HTML or JSON. Add `wait_for` with a CSS selector for content that appears late, or set `mode: "auto"` to start with a plain fetch and move up to a stronger rendering only when needed. Spicrawl picks the rendering for each request and bills the tier that served it (`fetch`, `js` or `browser`, reported as `engine` in the JSON result); there is no argument to choose it.
+
+### How do I give Claude web access?
+Connect the Spicrawl MCP server to Claude with an API key. In Claude Code:
+
+```bash
+claude mcp add --transport http spicrawl https://mcp.spicrawl.com/mcp \
+  --header "Authorization: Bearer $SPICRAWL_API_KEY"
+```
+
+Claude can then read any URL you give it by calling `spicrawl_scrape`. Spicrawl reads pages by URL; it is not a search engine. For Claude Desktop, see the install answer below.
+
+### Is there a free tier?
+Yes. During the beta every organization is on one plan of 1,000 credits per month. A plain fetch costs 1 credit, JavaScript rendering 3, and full-browser rendering (screenshots, PDFs, `actions`) 8. Failed requests cost 0, and a cache hit is billed at the same price as the fetch that stored it. When the month's credits run out, calls fail with `ERR::LIMIT::QUOTA_EXCEEDED` until it resets. See [Credits](https://docs.spicrawl.com/credits).
+
+### Does it handle bot-protected pages?
+Only with your own proxy. Pass it in `proxy` (an `http://` URL; residential proxies work best). When a site answers with a Cloudflare managed challenge, Spicrawl detects it, moves the request to full-browser rendering on the same proxy exit and returns the real page, for 8 credits. If every step is challenged you get `ERR::UPSTREAM::CHALLENGE` and pay 0. Success is likely, not guaranteed, and Spicrawl does not solve CAPTCHAs. On the default shared pool a challenged page returns that error. See [Scrape Cloudflare-protected pages](https://docs.spicrawl.com/guides/cloudflare).
+
+### How do I install it in Cursor or Claude Desktop?
+**Cursor:** click **Add to Cursor** above, or add this to `.cursor/mcp.json` and start Cursor with `SPICRAWL_API_KEY` set:
+
+```json
+{
+  "mcpServers": {
+    "spicrawl": {
+      "url": "https://mcp.spicrawl.com/mcp",
+      "headers": { "Authorization": "Bearer ${env:SPICRAWL_API_KEY}" }
+    }
+  }
+}
+```
 
-### Is Spicrawl free?
-Spicrawl bills credits per successful request, and during the beta every organization gets one plan of 1,000 credits per month. Failed requests cost 0 credits; a cache hit is billed at the same price as the fetch that stored it. See [Credits](https://docs.spicrawl.com/credits).
+**Claude Desktop** starts local (stdio) servers only, so run the package. Add this to `claude_desktop_config.json` and restart the app:
 
-### Does it render JavaScript?
-Yes. Set `render: true` on `spicrawl_scrape` to run the page in a browser, or `mode: "auto"` to let Spicrawl escalate from a plain fetch only when needed. There is no argument to choose the rendering on `spicrawl_scrape` or the batch tools: Spicrawl picks it for each request and bills the tier that served it (`fetch`, `js` or `browser`, reported as `engine` in the JSON result).
+```json
+{
+  "mcpServers": {
+    "spicrawl": {
+      "command": "npx",
+      "args": ["-y", "@spicrawl/mcp"],
+      "env": { "SPICRAWL_API_KEY": "spicrawl_live_..." }
+    }
+  }
+}
+```
 
-### Does it work with Claude Code, Cursor and VS Code?
-Yes, and with Claude Desktop, Windsurf, Gemini CLI, Codex and any client that speaks MCP Streamable HTTP with custom headers or launches stdio servers.
+Get a key at [app.spicrawl.com](https://app.spicrawl.com/signup). Steps for Claude Code, VS Code, Windsurf, Gemini CLI and Codex are under [Set up your MCP client](#set-up-your-mcp-client).
 
 ### Can it crawl a whole website?
 There is no single crawl tool. Scrape a page with `links: true` to get its links, then feed them to an open batch job (`open: true` and `spicrawl_batch_add_items`).
 
+## Other questions
+
+### Is there a hosted Spicrawl MCP server?
+Yes: `https://mcp.spicrawl.com/mcp`, MCP Streamable HTTP, authenticated with `Authorization: Bearer <API key>`, so there is nothing to install.
+
 ### Does it support OAuth?
 Not on `/mcp`: it accepts Spicrawl API keys only, so turn OAuth off for it in clients that try it on a `401`. The separate [ChatGPT endpoint](#chatgpt-endpoint-oauth) at `/chatgpt/mcp` takes OAuth access tokens only, when the operator turns it on.
 

package/package.json

@@ -1,8 +1,8 @@
 {
   "name": "@spicrawl/mcp",
-  "version": "0.1.7",
+  "version": "0.1.8",
   "mcpName": "io.github.OfficialSpicrawl/mcp",
-  "description": "The Spicrawl MCP server is a Model Context Protocol server that gives AI agents in Claude Code, Claude Desktop, Cursor, VS Code, Windsurf, Gemini CLI and Codex web scraping to Markdown, structured data extraction, screenshots, browser sessions and batch scraping as 25 native tools.",
+  "description": "MCP server for web scraping: scrape any URL to Markdown or JSON for Claude, Cursor and AI agents, with JavaScript rendering, batch scraping and data extraction",
   "license": "Apache-2.0",
   "author": "Spicrawl",
   "homepage": "https://docs.spicrawl.com/agents/mcp",
@@ -20,25 +20,20 @@
     "model-context-protocol",
     "web-scraping",
     "web-scraper",
-    "web-crawler",
-    "html-to-markdown",
+    "scraper",
+    "batch-scraping",
     "markdown",
+    "html-to-markdown",
     "data-extraction",
     "structured-data",
-    "screenshot",
     "javascript-rendering",
-    "batch-scraping",
+    "browser-automation",
     "llm",
-    "ai-agents",
     "rag",
+    "ai-agents",
     "claude",
     "claude-code",
-    "claude-desktop",
-    "cursor",
-    "vscode",
-    "codex",
-    "gemini-cli",
-    "windsurf"
+    "cursor"
   ],
   "type": "module",
   "bin": {