mcp-scraper 0.2.7 → 0.2.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. package/README.md +6 -4
  2. package/dist/bin/api-server.cjs +364 -162
  3. package/dist/bin/api-server.cjs.map +1 -1
  4. package/dist/bin/api-server.js +2 -2
  5. package/dist/bin/browser-agent-stdio-server.cjs +1 -1
  6. package/dist/bin/browser-agent-stdio-server.cjs.map +1 -1
  7. package/dist/bin/browser-agent-stdio-server.js +2 -2
  8. package/dist/bin/mcp-scraper-combined-stdio-server.cjs +68 -13
  9. package/dist/bin/mcp-scraper-combined-stdio-server.cjs.map +1 -1
  10. package/dist/bin/mcp-scraper-combined-stdio-server.js +3 -3
  11. package/dist/bin/mcp-scraper-install.cjs +2 -1
  12. package/dist/bin/mcp-scraper-install.cjs.map +1 -1
  13. package/dist/bin/mcp-scraper-install.js +2 -1
  14. package/dist/bin/mcp-scraper-install.js.map +1 -1
  15. package/dist/bin/mcp-stdio-server.cjs +68 -13
  16. package/dist/bin/mcp-stdio-server.cjs.map +1 -1
  17. package/dist/bin/mcp-stdio-server.js +2 -2
  18. package/dist/bin/paa-harvest.cjs +42 -6
  19. package/dist/bin/paa-harvest.cjs.map +1 -1
  20. package/dist/bin/paa-harvest.js +1 -1
  21. package/dist/{chunk-3TF6UT2P.js → chunk-5HMOPP76.js} +2 -2
  22. package/dist/{chunk-7SI6XIR3.js → chunk-6NEXSNSA.js} +69 -14
  23. package/dist/chunk-6NEXSNSA.js.map +1 -0
  24. package/dist/{chunk-MY3S7EX7.js → chunk-CQTAKXBN.js} +43 -7
  25. package/dist/chunk-CQTAKXBN.js.map +1 -0
  26. package/dist/chunk-I26QN7WQ.js +7 -0
  27. package/dist/chunk-I26QN7WQ.js.map +1 -0
  28. package/dist/index.cjs +42 -6
  29. package/dist/index.cjs.map +1 -1
  30. package/dist/index.js +1 -1
  31. package/dist/{server-RIKRBDOI.js → server-BTTDFPSQ.js} +238 -133
  32. package/dist/server-BTTDFPSQ.js.map +1 -0
  33. package/dist/{worker-NAKGTIF5.js → worker-5O44YBF4.js} +2 -2
  34. package/docs/mcp-tool-quality-spec.md +3 -1
  35. package/package.json +1 -1
  36. package/dist/chunk-7SI6XIR3.js.map +0 -1
  37. package/dist/chunk-IZE5UW7Y.js +0 -7
  38. package/dist/chunk-IZE5UW7Y.js.map +0 -1
  39. package/dist/chunk-MY3S7EX7.js.map +0 -1
  40. package/dist/server-RIKRBDOI.js.map +0 -1
  41. /package/dist/{chunk-3TF6UT2P.js.map → chunk-5HMOPP76.js.map} +0 -0
  42. /package/dist/{worker-NAKGTIF5.js.map → worker-5O44YBF4.js.map} +0 -0
package/README.md CHANGED
@@ -31,7 +31,7 @@ Build the branded one-click bundle:
31
31
  npm run build:mcpb
32
32
  ```
33
33
 
34
- The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.2.6`, SHA-256 `d316c1e3744d61c84b8fbbf2b155dba7da4062be52d9024d9fa7adde04c5b73c`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, and API-key configuration field from the bundle manifest.
34
+ The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.2.8`, SHA-256 `7d9d1bc0cf646a591fa5ed7e07b9bba5fa1ed598667f27cec4bd04e6197d5367`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, and API-key configuration field from the bundle manifest.
35
35
 
36
36
  The MCPB install exposes the same web-intelligence tools as `mcp-scraper` plus all `browser_*` tools from `browser-agent` through one server.
37
37
 
@@ -105,9 +105,9 @@ env = { MCP_SCRAPER_API_KEY = "sk_live_your_key" }
105
105
  - `facebook_ad_search`
106
106
  - `facebook_page_intel`
107
107
  - `facebook_ad_transcribe`
108
- - `maps_search` — search Google Maps for multiple business/profile candidates. Use for GMB/GBP prospect lists, competitors, categories, and anything needing more than the Google 3-pack. `maxResults` defaults to 10 and is capped at 50.
108
+ - `maps_search` — search Google Maps for multiple business/profile candidates. Use for GMB/GBP prospect lists, competitors, categories, and anything needing more than the Google 3-pack. In default `proxyMode: "location"`, retryable failures rotate to a new residential proxy and new browser session for up to 5 attempts. `maxResults` defaults to 10 and is capped at 50.
109
109
  - `maps_place_intel` — hydrate one known/named Google Maps business with profile details and optional reviews. Use after `maps_search` when a selected candidate needs full details.
110
- - `directory_workflow` — build city-by-city directory/prospecting datasets from Census place selection plus Google Maps searches. Use it for requests like "all cities over 100k population in Tennessee, then get 20 roofers from Maps." The saved CSV includes `source_location`, `result_position`, `business_name`, `review_stars`, `category`, `address`, `phone`, `hours_status`, `website_url`, `directions_url`, `place_url`, `cid`, `cid_decimal`, Census population, and ZIP groups. It captures Maps star ratings from list cards, not profile review counts.
110
+ - `directory_workflow` — build city-by-city directory/prospecting datasets from Census place selection plus Google Maps searches. Use it for requests like "all cities over 100k population in Tennessee, then get 20 roofers from Maps." In default `proxyMode: "location"`, each city search rotates retryable failures to a new residential proxy and new browser session for up to 5 attempts. The saved CSV includes `source_location`, `result_position`, `business_name`, `review_stars`, `category`, `address`, `phone`, `hours_status`, `website_url`, `directions_url`, `place_url`, `cid`, `cid_decimal`, Census population, and ZIP groups. It captures Maps star ratings from list cards, not profile review counts.
111
111
  - `rank_tracker_blueprint` — generate a database schema, cron/heartbeat plan, ingestion workflow, metrics list, and implementation prompt for building rank trackers. It has modes for Maps rankings via `directory_workflow`/`maps_search`, organic rankings via `search_serp`, AI Overview citation tracking, and PAA source presence tracking. This is a local planning tool and does not spend credits.
112
112
  - `credits_info`
113
113
 
@@ -133,7 +133,9 @@ env = { MCP_SCRAPER_API_KEY = "sk_live_your_key" }
133
133
 
134
134
  For accurate annotated videos, do not guess annotation times from a script. Start the replay, navigate until each target is visible and stable, call `browser_replay_mark` for each callout, then stop the replay and pass the returned annotations to `browser_replay_annotate` with the returned `source_width` and `source_height`.
135
135
 
136
- For US local SERP tools (`harvest_paa` and `search_serp`), keep `proxyMode` at the default `location` unless you are debugging. Location mode uses fresh residential proxy IDs across retries and treats CAPTCHA, proxy tunnel failure, and wrong-location evidence as retryable before returning.
136
+ For US local SERP tools (`harvest_paa` and `search_serp`), keep `proxyMode` at the default `location` unless you are debugging. Location mode uses fresh residential proxy IDs across retries. When Google shows a CAPTCHA/challenge, browser-service sessions briefly wait for automatic challenge solving first, then rotate to a new proxy/session if the challenge does not clear. Proxy tunnel failure and wrong-location evidence are also retryable before returning.
137
+
138
+ For Google Maps tools (`maps_search` and `directory_workflow`), keep `proxyMode` at the default `location` for US city/state work. Retryable Maps failures such as CAPTCHA, timeout, proxy tunnel failure, proxy unavailability, or browser-session death rotate to a new residential proxy and a new browser session for up to 5 attempts. Successful structured responses include sanitized attempt telemetry so callers can verify the proxy source, observed city/region, and attempt count without exposing full proxy or browser IDs.
137
139
 
138
140
  The MCPB bundle and `mcp-scraper-combined` expose both sections through one local MCP server. The split `mcp-scraper` entrypoint exposes only the web-intelligence tools, and the split `browser-agent` entrypoint exposes only the browser-agent tools.
139
141