mcp-scraper 0.33.7 → 0.34.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (57) hide show
  1. package/README.md +4 -4
  2. package/dist/bin/api-server.cjs +7100 -7023
  3. package/dist/bin/api-server.cjs.map +1 -1
  4. package/dist/bin/api-server.js +2 -2
  5. package/dist/bin/mcp-scraper-cli.cjs +2 -1
  6. package/dist/bin/mcp-scraper-cli.cjs.map +1 -1
  7. package/dist/bin/mcp-scraper-cli.js +3 -3
  8. package/dist/bin/mcp-scraper-install.cjs +2 -2
  9. package/dist/bin/mcp-scraper-install.cjs.map +1 -1
  10. package/dist/bin/mcp-scraper-install.js +2 -2
  11. package/dist/bin/mcp-stdio-server.cjs +43 -37
  12. package/dist/bin/mcp-stdio-server.cjs.map +1 -1
  13. package/dist/bin/mcp-stdio-server.js +6 -6
  14. package/dist/bin/paa-harvest.cjs +22 -0
  15. package/dist/bin/paa-harvest.cjs.map +1 -1
  16. package/dist/bin/paa-harvest.js +3 -3
  17. package/dist/{chunk-V36LS5YV.js → chunk-4ZB3X6BQ.js} +4 -3
  18. package/dist/chunk-4ZB3X6BQ.js.map +1 -0
  19. package/dist/{chunk-SUOHUXQS.js → chunk-5PZ6N2QM.js} +2 -2
  20. package/dist/{chunk-H6QECE2U.js → chunk-6OPHG76G.js} +30 -27
  21. package/dist/chunk-6OPHG76G.js.map +1 -0
  22. package/dist/{chunk-2EDFOQD7.js → chunk-7AYRWAEK.js} +2 -2
  23. package/dist/{chunk-2EDFOQD7.js.map → chunk-7AYRWAEK.js.map} +1 -1
  24. package/dist/chunk-7N2KYL4U.js +7 -0
  25. package/dist/chunk-7N2KYL4U.js.map +1 -0
  26. package/dist/{chunk-JBSGBSGT.js → chunk-BWXLTWF7.js} +21 -16
  27. package/dist/chunk-BWXLTWF7.js.map +1 -0
  28. package/dist/{chunk-V73MPRU6.js → chunk-CB5C3BPB.js} +17 -1
  29. package/dist/chunk-CB5C3BPB.js.map +1 -0
  30. package/dist/{chunk-XVCETVYY.js → chunk-XVVNKASZ.js} +12 -4
  31. package/dist/chunk-XVVNKASZ.js.map +1 -0
  32. package/dist/{chunk-AT2PFWHW.js → chunk-ZID3WQID.js} +2 -2
  33. package/dist/index.cjs +22 -0
  34. package/dist/index.cjs.map +1 -1
  35. package/dist/index.d.cts +3 -0
  36. package/dist/index.d.ts +3 -0
  37. package/dist/index.js +3 -3
  38. package/dist/{server-B35CQJB3.js → server-IWDHTES2.js} +185 -139
  39. package/dist/server-IWDHTES2.js.map +1 -0
  40. package/dist/{site-extract-repository-JHMVHENZ.js → site-extract-repository-OLVWMOU2.js} +3 -3
  41. package/dist/{worker-3LYNVGLF.js → worker-645BZPEK.js} +5 -5
  42. package/docs/mcp-tool-craft-lint.generated.md +171 -49
  43. package/docs/mcp-tool-manifest.generated.json +46 -11
  44. package/docs/specs/query-fanout-transport-contract-fix.md +37 -0
  45. package/package.json +1 -1
  46. package/dist/chunk-H6QECE2U.js.map +0 -1
  47. package/dist/chunk-HAMN42SP.js +0 -7
  48. package/dist/chunk-HAMN42SP.js.map +0 -1
  49. package/dist/chunk-JBSGBSGT.js.map +0 -1
  50. package/dist/chunk-V36LS5YV.js.map +0 -1
  51. package/dist/chunk-V73MPRU6.js.map +0 -1
  52. package/dist/chunk-XVCETVYY.js.map +0 -1
  53. package/dist/server-B35CQJB3.js.map +0 -1
  54. /package/dist/{chunk-SUOHUXQS.js.map → chunk-5PZ6N2QM.js.map} +0 -0
  55. /package/dist/{chunk-AT2PFWHW.js.map → chunk-ZID3WQID.js.map} +0 -0
  56. /package/dist/{site-extract-repository-JHMVHENZ.js.map → site-extract-repository-OLVWMOU2.js.map} +0 -0
  57. /package/dist/{worker-3LYNVGLF.js.map → worker-645BZPEK.js.map} +0 -0
package/README.md CHANGED
@@ -88,7 +88,7 @@ Build the branded one-click bundle:
88
88
  npm run build:mcpb
89
89
  ```
90
90
 
91
- The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.33.7`, SHA-256 `0aee7cef4fd40fe3eb1db14788937efc3bd559557e68f608e9dda11e0fa1cb90`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, and API-key configuration field from the bundle manifest.
91
+ The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.34.1`, SHA-256 `b217a7273ae7ee406003619de7f4bf95b4c8ad241c7f723244187c1bb557d947`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, and API-key configuration field from the bundle manifest.
92
92
 
93
93
  The MCPB install exposes every tool — web-intelligence plus all `browser_*` tools — through the one `mcp-scraper` server.
94
94
 
@@ -213,7 +213,7 @@ Google Search Console exposes eight bounded reads and eight gated property and s
213
213
  - `browser_replay_download` — download and save the replay MP4 under `MCP_SCRAPER_OUTPUT_DIR/browser-replays`.
214
214
  - `browser_replay_mark` — while recording, locate a DOM target and return a replay-timed annotation object.
215
215
  - `browser_replay_annotate` — download a replay MP4, render timed boxes, circles, underlines, arrows, and labels using annotation objects from `browser_replay_mark` or exact bounds from `browser_locate`, and save a new annotated MP4 under `MCP_SCRAPER_OUTPUT_DIR/browser-replays`.
216
- - `browser_capture_fanout` — capture ChatGPT/Claude AI-search fan-out from an open logged-in hosted session. Use `export=true` for durable artifacts. It writes full `fanout.json`, query/source/citation/domain/snippet CSVs, TSV, and `report.html` under `MCP_SCRAPER_OUTPUT_DIR/fanout`, returning paths relative to `MCP_SCRAPER_OUTPUT_DIR` so users can share portable paths.
216
+ - `browser_capture_fanout` — capture ChatGPT/Claude AI-search fan-out from an open logged-in hosted session. Every client receives the complete structured capture inline. Installed stdio/MCPB clients can use `export=true` for durable `fanout.json`, query/source/citation/domain/snippet CSVs, TSV, and `report.html` under `MCP_SCRAPER_OUTPUT_DIR/fanout`; hosted OAuth clients receive `exports: null` and use the inline data.
217
217
  - `browser_close`
218
218
  - `browser_list_sessions`
219
219
 
@@ -227,7 +227,7 @@ The `mcp-scraper` server (and the MCPB bundle, which runs it) exposes both secti
227
227
 
228
228
  All MCP tools expose output schemas and return `structuredContent` with the IDs, URLs, CSV paths, transcripts, browser session handles, replay paths, artifacts, recipe fields, or blueprint fields needed by the next step. Browser Agent tools keep a JSON text block for older clients, but structured data is the primary contract. All tools carry MCP annotations; file-writing tools such as replay downloads and annotations state their filesystem side effects.
229
229
 
230
- The canonical tool inventory is generated at `docs/mcp-tool-manifest.generated.json`. Both the `mcp-scraper` stdio server and the hosted endpoint at `https://mcpscraper.dev/mcp` expose the same 166 tools: 77 scraper, browser, workflow, billing, and connected-service tools plus 89 durable-memory tools. Release verification compares the exact local and remote tool-name sets, not only the count.
230
+ The canonical tool inventory is generated at `docs/mcp-tool-manifest.generated.json`. Both the `mcp-scraper` stdio server and the hosted endpoint at `https://mcpscraper.dev/mcp` expose the same 167 tools: 78 scraper, browser, workflow, billing, and connected-service tools plus 89 durable-memory tools. Release verification compares the exact local and remote tool-name sets, not only the count.
231
231
 
232
232
  For contract parity, stdio and MCPB memory calls invoke the matching public tool on the hosted MCP Scraper `/mcp` endpoint. The hosted aggregate runtime owns MCP Scraper-specific billing, scheduling, credential, and in-process cutover policy; its internal `/memory/mcp-call` bridge is a fallback to the standalone memory service, not the public stdio execution path. Direct `mcp-memory` OAuth and stdio clients continue to use `memory.mcpscraper.dev` and must be verified as a separate dependent release surface.
233
233
 
@@ -245,7 +245,7 @@ The `mcp-scraper` NPX stdio server also exposes saved reports as MCP resources:
245
245
  - `BROWSER_AGENT_PROFILE_NAME` is optional and sets the default saved hosted browser profile for `mcp-scraper` stdio sessions. Aliases: `BROWSER_SERVICE_PROFILE_NAME`, `KERNEL_BROWSER_PROFILE_NAME`, `KERNEL_PROFILE_NAME`.
246
246
  - `BROWSER_AGENT_PROFILE_SAVE_CHANGES=true` is optional hosted setup behavior. It persists cookies and storage back to the named profile when `browser_close` deletes the hosted browser session. Aliases: `BROWSER_SERVICE_PROFILE_SAVE_CHANGES`, `KERNEL_BROWSER_PROFILE_SAVE_CHANGES`, `KERNEL_PROFILE_SAVE_CHANGES`.
247
247
 
248
- Every web intelligence tool call made through `mcp-scraper` saves a full Markdown report to disk by default and returns the file path in the MCP response. The hosted `/mcp` endpoint returns reports inline only and never writes files. Browser replay downloads are saved by `browser_replay_download` under `MCP_SCRAPER_OUTPUT_DIR/browser-replays`. AI fan-out exports are saved by `browser_capture_fanout` with `export=true` under `MCP_SCRAPER_OUTPUT_DIR/fanout`, and return relative paths.
248
+ Every web intelligence tool call made through `mcp-scraper` saves a full Markdown report to disk by default and returns the file path in the MCP response. The hosted `/mcp` endpoint returns reports inline only and never writes files. Browser replay downloads are saved by `browser_replay_download` under `MCP_SCRAPER_OUTPUT_DIR/browser-replays`. AI fan-out captures are always returned inline; only installed stdio/MCPB clients write optional `export=true` files under `MCP_SCRAPER_OUTPUT_DIR/fanout`, returning relative paths. Hosted clients always receive `exports: null`.
249
249
 
250
250
  ## Updating Existing Installs
251
251