mcp-scraper 0.34.1 → 0.35.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -3
- package/dist/bin/api-server.cjs +3709 -2689
- package/dist/bin/api-server.cjs.map +1 -1
- package/dist/bin/api-server.js +1 -1
- package/dist/bin/mcp-scraper-cli.cjs +1 -1
- package/dist/bin/mcp-scraper-cli.cjs.map +1 -1
- package/dist/bin/mcp-scraper-cli.js +1 -1
- package/dist/bin/mcp-scraper-install.cjs +2 -2
- package/dist/bin/mcp-scraper-install.cjs.map +1 -1
- package/dist/bin/mcp-scraper-install.js +2 -2
- package/dist/bin/mcp-stdio-server.cjs +2102 -1904
- package/dist/bin/mcp-stdio-server.cjs.map +1 -1
- package/dist/bin/mcp-stdio-server.js +3 -3
- package/dist/{chunk-6OPHG76G.js → chunk-NPMW5HUS.js} +2338 -2136
- package/dist/chunk-NPMW5HUS.js.map +1 -0
- package/dist/{chunk-5PZ6N2QM.js → chunk-U44TPRST.js} +2 -2
- package/dist/chunk-U44TPRST.js.map +1 -0
- package/dist/chunk-YR4LJ6AQ.js +7 -0
- package/dist/chunk-YR4LJ6AQ.js.map +1 -0
- package/dist/{chunk-7AYRWAEK.js → chunk-YRGSEY5L.js} +2 -2
- package/dist/{chunk-7AYRWAEK.js.map → chunk-YRGSEY5L.js.map} +1 -1
- package/dist/chunk-YV2FUEBX.js +851 -0
- package/dist/chunk-YV2FUEBX.js.map +1 -0
- package/dist/{extract-bundle-GPZRLBVY.js → extract-bundle-ONWZVV55.js} +59 -8
- package/dist/extract-bundle-ONWZVV55.js.map +1 -0
- package/dist/{server-IWDHTES2.js → server-GKUTC73B.js} +224 -22
- package/dist/server-GKUTC73B.js.map +1 -0
- package/dist/{site-extract-repository-OLVWMOU2.js → site-extract-repository-L6BHWVDU.js} +2 -2
- package/docs/mcp-tool-manifest.generated.json +499 -8
- package/docs/specs/query-fanout-transport-contract-fix.md +9 -1
- package/package.json +1 -1
- package/dist/chunk-5PZ6N2QM.js.map +0 -1
- package/dist/chunk-6OPHG76G.js.map +0 -1
- package/dist/chunk-7N2KYL4U.js +0 -7
- package/dist/chunk-7N2KYL4U.js.map +0 -1
- package/dist/chunk-JWIE5NCR.js +0 -284
- package/dist/chunk-JWIE5NCR.js.map +0 -1
- package/dist/extract-bundle-GPZRLBVY.js.map +0 -1
- package/dist/server-IWDHTES2.js.map +0 -1
- /package/dist/{site-extract-repository-OLVWMOU2.js.map → site-extract-repository-L6BHWVDU.js.map} +0 -0
package/README.md
CHANGED
|
@@ -88,7 +88,7 @@ Build the branded one-click bundle:
|
|
|
88
88
|
npm run build:mcpb
|
|
89
89
|
```
|
|
90
90
|
|
|
91
|
-
The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.
|
|
91
|
+
The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.35.1`, SHA-256 `95edd500a9879b1506b622bb2fc3c2bff3bcde419b927889512c19c1aec90ae8`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, and API-key configuration field from the bundle manifest.
|
|
92
92
|
|
|
93
93
|
The MCPB install exposes every tool — web-intelligence plus all `browser_*` tools — through the one `mcp-scraper` server.
|
|
94
94
|
|
|
@@ -154,7 +154,8 @@ env = { MCP_SCRAPER_API_KEY = "sk_live_your_key" }
|
|
|
154
154
|
- `search_serp`
|
|
155
155
|
- `extract_url`
|
|
156
156
|
- `map_site_urls`
|
|
157
|
-
- `
|
|
157
|
+
- `map_wayback_snapshots` — count and inventory Wayback captures across an inclusive date range without downloading page bodies. Supports exact pages, prefixes, hosts, domains, or selected URLs; reports exact versus lower-bound counts, unique URLs/content digests, monthly coverage, missing months, and optional timestamp rows.
|
|
158
|
+
- `extract_site` — crawl a live site, batch one archived site snapshot from a Wayback replay URL, or pass a `wayback` plan for whole-site, single-page, or selected-page timelines across explicit months or a `from`/`to` range. Timeline ZIPs include month folders and a capture matrix.
|
|
158
159
|
- `youtube_harvest`
|
|
159
160
|
- `youtube_transcribe`
|
|
160
161
|
- `facebook_ad_search`
|
|
@@ -227,7 +228,7 @@ The `mcp-scraper` server (and the MCPB bundle, which runs it) exposes both secti
|
|
|
227
228
|
|
|
228
229
|
All MCP tools expose output schemas and return `structuredContent` with the IDs, URLs, CSV paths, transcripts, browser session handles, replay paths, artifacts, recipe fields, or blueprint fields needed by the next step. Browser Agent tools keep a JSON text block for older clients, but structured data is the primary contract. All tools carry MCP annotations; file-writing tools such as replay downloads and annotations state their filesystem side effects.
|
|
229
230
|
|
|
230
|
-
The canonical tool inventory is generated at `docs/mcp-tool-manifest.generated.json`. Both the `mcp-scraper` stdio server and the hosted endpoint at `https://mcpscraper.dev/mcp` expose the same
|
|
231
|
+
The canonical tool inventory is generated at `docs/mcp-tool-manifest.generated.json`. Both the `mcp-scraper` stdio server and the hosted endpoint at `https://mcpscraper.dev/mcp` expose the same 168 tools: 79 scraper, browser, workflow, billing, and connected-service tools plus 89 durable-memory tools. Release verification compares the exact local and remote tool-name sets, not only the count.
|
|
231
232
|
|
|
232
233
|
For contract parity, stdio and MCPB memory calls invoke the matching public tool on the hosted MCP Scraper `/mcp` endpoint. The hosted aggregate runtime owns MCP Scraper-specific billing, scheduling, credential, and in-process cutover policy; its internal `/memory/mcp-call` bridge is a fallback to the standalone memory service, not the public stdio execution path. Direct `mcp-memory` OAuth and stdio clients continue to use `memory.mcpscraper.dev` and must be verified as a separate dependent release surface.
|
|
233
234
|
|