mcp-scraper 0.21.3 → 0.21.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (56) hide show
  1. package/README.md +5 -3
  2. package/dist/bin/api-server.cjs +737 -185
  3. package/dist/bin/api-server.cjs.map +1 -1
  4. package/dist/bin/api-server.js +3 -3
  5. package/dist/bin/mcp-scraper-cli.cjs +1 -1
  6. package/dist/bin/mcp-scraper-cli.cjs.map +1 -1
  7. package/dist/bin/mcp-scraper-cli.js +1 -1
  8. package/dist/bin/mcp-scraper-install.cjs +1 -1
  9. package/dist/bin/mcp-scraper-install.cjs.map +1 -1
  10. package/dist/bin/mcp-scraper-install.js +1 -1
  11. package/dist/bin/mcp-stdio-server.cjs +101 -36
  12. package/dist/bin/mcp-stdio-server.cjs.map +1 -1
  13. package/dist/bin/mcp-stdio-server.js +5 -5
  14. package/dist/bin/paa-harvest.cjs +21 -0
  15. package/dist/bin/paa-harvest.cjs.map +1 -1
  16. package/dist/bin/paa-harvest.js +3 -3
  17. package/dist/{chunk-JJ5RMEGX.js → chunk-E5J4HJBO.js} +2 -2
  18. package/dist/{chunk-H7APH5UG.js → chunk-ICT7DDHL.js} +366 -50
  19. package/dist/chunk-ICT7DDHL.js.map +1 -0
  20. package/dist/{chunk-OBNKOYOE.js → chunk-JHI373VB.js} +7 -5
  21. package/dist/chunk-JHI373VB.js.map +1 -0
  22. package/dist/{chunk-5IU5ZFVO.js → chunk-MTSBI7ZH.js} +23 -6
  23. package/dist/chunk-MTSBI7ZH.js.map +1 -0
  24. package/dist/{chunk-XDRHBGJR.js → chunk-PNBYYAEK.js} +3 -3
  25. package/dist/{chunk-7LO6KLKD.js → chunk-QO7HX67W.js} +93 -31
  26. package/dist/chunk-QO7HX67W.js.map +1 -0
  27. package/dist/{chunk-2P5VFIJM.js → chunk-SDHPQTFM.js} +2 -2
  28. package/dist/chunk-YLX3SKJN.js +7 -0
  29. package/dist/chunk-YLX3SKJN.js.map +1 -0
  30. package/dist/{db-Z35S224E.js → db-WRILVOFY.js} +4 -2
  31. package/dist/{extract-bundle-4RO3YU4Q.js → extract-bundle-NFFVMUMT.js} +2 -2
  32. package/dist/index.cjs +21 -0
  33. package/dist/index.cjs.map +1 -1
  34. package/dist/index.js +3 -3
  35. package/dist/{server-PBBRSLBP.js → server-POMKHEXJ.js} +251 -100
  36. package/dist/server-POMKHEXJ.js.map +1 -0
  37. package/dist/{site-extract-repository-FDH6ZXKT.js → site-extract-repository-2QQSL2QC.js} +4 -4
  38. package/dist/{worker-BXJQ7J4N.js → worker-RV2TYXE3.js} +9 -8
  39. package/dist/worker-RV2TYXE3.js.map +1 -0
  40. package/docs/mcp-tool-manifest.generated.json +96 -13
  41. package/docs/specs/oauth-mcp-spec.md +2 -2
  42. package/package.json +1 -1
  43. package/dist/chunk-5IU5ZFVO.js.map +0 -1
  44. package/dist/chunk-7LO6KLKD.js.map +0 -1
  45. package/dist/chunk-H7APH5UG.js.map +0 -1
  46. package/dist/chunk-OBNKOYOE.js.map +0 -1
  47. package/dist/chunk-VYUWWIVR.js +0 -7
  48. package/dist/chunk-VYUWWIVR.js.map +0 -1
  49. package/dist/server-PBBRSLBP.js.map +0 -1
  50. package/dist/worker-BXJQ7J4N.js.map +0 -1
  51. /package/dist/{chunk-JJ5RMEGX.js.map → chunk-E5J4HJBO.js.map} +0 -0
  52. /package/dist/{chunk-XDRHBGJR.js.map → chunk-PNBYYAEK.js.map} +0 -0
  53. /package/dist/{chunk-2P5VFIJM.js.map → chunk-SDHPQTFM.js.map} +0 -0
  54. /package/dist/{db-Z35S224E.js.map → db-WRILVOFY.js.map} +0 -0
  55. /package/dist/{extract-bundle-4RO3YU4Q.js.map → extract-bundle-NFFVMUMT.js.map} +0 -0
  56. /package/dist/{site-extract-repository-FDH6ZXKT.js.map → site-extract-repository-2QQSL2QC.js.map} +0 -0
package/README.md CHANGED
@@ -88,7 +88,7 @@ Build the branded one-click bundle:
88
88
  npm run build:mcpb
89
89
  ```
90
90
 
91
- The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.21.3`, SHA-256 `56e0b340ffbc3edc389a66cd58707037d181ef76cf47a52926ba8a18519fe833`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, and API-key configuration field from the bundle manifest.
91
+ The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.21.5`, SHA-256 `ab07d5d0adb37840e434dcb5931927a26274f475c8f216048d59737904d9bf5d`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, and API-key configuration field from the bundle manifest.
92
92
 
93
93
  The MCPB install exposes every tool — web-intelligence plus all `browser_*` tools — through the one `mcp-scraper` server.
94
94
 
@@ -178,11 +178,11 @@ env = { MCP_SCRAPER_API_KEY = "sk_live_your_key" }
178
178
 
179
179
  - `list_service_connections` — list this caller's tenant-owned Nango OAuth and official remote MCP connections, including exact live reads, gated actions, permanently blocked administrative tools, credential transport, and schema-discovery metadata. Connections are never shared between customers.
180
180
  - `describe_service_connection_tool` — fetch the sanitized live MCP Tool definition for one tool listed on one tenant-owned connection, including its current callability, input schema, optional output schema, safe annotations, and schema hash. Use this before constructing provider-native arguments; provider functions stay behind the generic bridges instead of becoming dozens of permanent top-level tools.
181
- - `export_connected_service_data` — fetch a bounded Gmail, Google Calendar, Zoom, or Resend time range in one MCP call. Resend's aggregate `resend_data` export walks 12 practical safe collections: sent mail, received mail, logs, contacts, broadcasts, templates, domains, segments, topics, webhooks, contact imports, and contact properties. The six core collections are also individually selectable. The server handles provider pagination, bounded detail retrieval, normalization, per-category warnings, and signed continuation. Small exports return inline; larger exports become private JSONL retained for seven days with a 15-minute signed URL. Attachments remain metadata-only and oversized records are truncated with warnings.
181
+ - `export_connected_service_data` — fetch a bounded Gmail, Google Calendar, Google Search Console, Zoom, Resend, or Meta time range in one MCP call. Search Console's `search_console_performance` dataset walks accessible properties and bounded Search Analytics pages with signed continuation. Resend's aggregate `resend_data` export walks 12 practical safe collections: sent mail, received mail, logs, contacts, broadcasts, templates, domains, segments, topics, webhooks, contact imports, and contact properties. The six core collections are also individually selectable. The server handles provider pagination, bounded detail retrieval, normalization, per-category warnings, and signed continuation. Small exports return inline; larger exports become private JSONL retained for seven days with a 15-minute signed URL. Attachments remain metadata-only and oversized records are truncated with warnings.
182
182
  - `renew_connected_data_download` — issue a fresh 15-minute signed URL for an unexpired private export artifact without pulling the provider again.
183
183
  - `read_service_connection` — run one small live read by exact allowlisted name across Nango OAuth or official remote MCP connections, including bounded Google Drive inventory, change, Doc, Sheet, and text-file tools. Do not loop it over a time range when `export_connected_service_data` supports that provider's collection.
184
184
  - `import_service_connection_to_memory` — run one exact approved read and upsert the bounded, redacted result into an existing ordinary Memory vault at a stable server-generated path. The snapshot is embedded for semantic search and explicitly marked as untrusted provider data. This is not account-wide pagination or continuous sync.
185
- - `call_service_connection_action` — run one exact mutation only after the account action switch is enabled. Google Drive, Resend, GitHub, Gmail, Calendar, Slack, Zoom, Facebook, Google Analytics, X, and YouTube actions remain live-inventory, provider-policy, tenant, and audit gated; credential, OAuth-grant, and raw editor-session administration is never exposed.
185
+ - `call_service_connection_action` — run one exact mutation only after the account action switch is enabled. Google Drive, Search Console, Resend, GitHub, Gmail, Calendar, Slack, Zoom, Facebook, Google Analytics, X, and YouTube actions remain live-inventory, provider-policy, tenant, and audit gated; credential, OAuth-grant, and raw editor-session administration is never exposed.
186
186
  - `set_scheduled_action_connections` — bind exact tenant connections and exact tool grants to one scheduled action. Deterministic connection syncs are model-free and one schedule occurrence counts as one metered event even when provider pagination spans many pages.
187
187
 
188
188
  Connecting or reconnecting a provider and calling direct connected-account reads, gated writes, supported exports, or bounded Memory imports do not consume additional Credits. Scheduled occurrences cost 75 Credits each. Agent-mode runs also add 1.5× the model provider's actual reported cost; deterministic connection syncs pay only the 75-Credit run base. Separately metered scraper, browser, and media operations keep their normal rate-card charges. Scheduling has no separate subscription or monthly run quota.
@@ -191,6 +191,8 @@ GitHub uses the same provider-neutral read/action bridges: 27 live reads cover r
191
191
 
192
192
  Google Drive Phase 1 exposes ten bounded reads plus the separately gated `create-drive-folder` and `copy-drive-file` actions through those same bridges and through per-run Mastra toolsets. It is interactive tool support, not the later deterministic bulk-export/`connection_sync` phase; large binary delivery and broader Drive mutations remain unreleased.
193
193
 
194
+ Google Search Console exposes six bounded reads and four gated property and sitemap mutations. Its deterministic `connection_sync` path backfills Search Analytics in bounded pages, persists checkpoints, and can mirror normalized property and performance records into the selected Memory vault. Scheduled agent runs can instead receive only the exact Search Console tools granted to that schedule.
195
+
194
196
  ### Browser-agent tools
195
197
 
196
198
  - `browser_open` — open a direct/no-proxy hosted browser session. Pass a saved profile name to load an authenticated session. Returns a human `watch_url`.