mcp-scraper 0.3.1 → 0.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -1
- package/dist/bin/api-server.cjs +2050 -832
- package/dist/bin/api-server.cjs.map +1 -1
- package/dist/bin/api-server.js +2 -2
- package/dist/bin/browser-agent-stdio-server.cjs +1 -1
- package/dist/bin/browser-agent-stdio-server.cjs.map +1 -1
- package/dist/bin/browser-agent-stdio-server.js +2 -2
- package/dist/bin/mcp-scraper-cli.cjs +1 -1
- package/dist/bin/mcp-scraper-cli.cjs.map +1 -1
- package/dist/bin/mcp-scraper-cli.js +1 -1
- package/dist/bin/mcp-scraper-combined-stdio-server.cjs +351 -4
- package/dist/bin/mcp-scraper-combined-stdio-server.cjs.map +1 -1
- package/dist/bin/mcp-scraper-combined-stdio-server.js +4 -4
- package/dist/bin/mcp-scraper-install.cjs +3 -3
- package/dist/bin/mcp-scraper-install.cjs.map +1 -1
- package/dist/bin/mcp-scraper-install.js +2 -2
- package/dist/bin/mcp-stdio-server.cjs +349 -2
- package/dist/bin/mcp-stdio-server.cjs.map +1 -1
- package/dist/bin/mcp-stdio-server.js +2 -2
- package/dist/bin/paa-harvest.cjs +3 -1
- package/dist/bin/paa-harvest.cjs.map +1 -1
- package/dist/bin/paa-harvest.js +1 -1
- package/dist/{chunk-SXTXMFEQ.js → chunk-AZ5PKU4F.js} +34 -1
- package/dist/chunk-AZ5PKU4F.js.map +1 -0
- package/dist/{chunk-Q4STSM63.js → chunk-C3FGVJWH.js} +3 -3
- package/dist/chunk-C3FGVJWH.js.map +1 -0
- package/dist/{chunk-55T4SRLJ.js → chunk-MGWGZBL5.js} +350 -3
- package/dist/chunk-MGWGZBL5.js.map +1 -0
- package/dist/chunk-PVXDEREW.js +7 -0
- package/dist/chunk-PVXDEREW.js.map +1 -0
- package/dist/{chunk-LICHCMV6.js → chunk-RLBJ3QNC.js} +2 -2
- package/dist/{chunk-IPW4LFOT.js → chunk-UWSG3C5J.js} +4 -2
- package/dist/chunk-UWSG3C5J.js.map +1 -0
- package/dist/index.cjs +3 -1
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +1 -1
- package/dist/{server-AXPNL2RV.js → server-D3XHIEQN.js} +997 -253
- package/dist/server-D3XHIEQN.js.map +1 -0
- package/dist/{worker-SLQ375UG.js → worker-56IXWOQU.js} +3 -3
- package/docs/mcp-tool-craft-lint.generated.md +5 -2
- package/docs/mcp-tool-manifest.generated.json +80 -5
- package/package.json +1 -1
- package/dist/chunk-55T4SRLJ.js.map +0 -1
- package/dist/chunk-D4JDGKOV.js +0 -7
- package/dist/chunk-D4JDGKOV.js.map +0 -1
- package/dist/chunk-IPW4LFOT.js.map +0 -1
- package/dist/chunk-Q4STSM63.js.map +0 -1
- package/dist/chunk-SXTXMFEQ.js.map +0 -1
- package/dist/server-AXPNL2RV.js.map +0 -1
- /package/dist/{chunk-LICHCMV6.js.map → chunk-RLBJ3QNC.js.map} +0 -0
- /package/dist/{worker-SLQ375UG.js.map → worker-56IXWOQU.js.map} +0 -0
package/README.md
CHANGED
|
@@ -129,7 +129,7 @@ MCP_SCRAPER_API_KEY=sk_live_your_key mcp-scraper-cli agent install claude --appl
|
|
|
129
129
|
|
|
130
130
|
`browser import-chrome` copies the selected local Chrome profile into `~/.mcp-scraper/browser-profiles/<name>/user-data`, skipping cache and lock files. The managed clone can include cookies, local storage, history, session storage, and Chrome password database files. It does not upload the profile. Re-run `mcp-scraper-cli browser sync-profile <name>` after logging into new sites in normal Chrome.
|
|
131
131
|
|
|
132
|
-
Set `MCP_SCRAPER_BROWSER_MODE=local` and `MCP_SCRAPER_BROWSER_PROFILE=<name>` when you want `browser_open` to launch local Google Chrome against that managed clone. A `profile` argument passed directly to `browser_open` overrides the default for that one session.
|
|
132
|
+
Set `MCP_SCRAPER_BROWSER_MODE=local` and `MCP_SCRAPER_BROWSER_PROFILE=<name>` when you want `browser_open` to launch local Google Chrome against that managed clone. A `profile` argument passed directly to `browser_open` overrides the default for that one session. Instagram tools can use the same managed profile by passing `browserProfile=<name>` or by running the API with local browser mode configured.
|
|
133
133
|
|
|
134
134
|
Hosted browser profiles are still available. `BROWSER_AGENT_PROFILE_NAME` sets the default saved Kernel browser profile for hosted sessions. For first-time hosted setup, set `BROWSER_AGENT_PROFILE_SAVE_CHANGES=true`, open a browser with the profile, complete login through the watch URL, then call `browser_close` to persist cookies/local storage into the hosted Kernel profile.
|
|
135
135
|
|
|
@@ -198,6 +198,8 @@ env = { MCP_SCRAPER_API_KEY = "sk_live_your_key" }
|
|
|
198
198
|
- `facebook_page_intel`
|
|
199
199
|
- `facebook_ad_transcribe` — transcribe a direct Facebook ad video URL returned by `facebook_page_intel`.
|
|
200
200
|
- `facebook_video_transcribe` — transcribe an organic Facebook reel, video, watch, post, or share URL, including `fb.watch` links. The tool renders the page, extracts the best matching public Facebook CDN MP4 URL, then returns transcript text, timestamped chunks, selected quality, video metadata, and the extracted MP4 URL for follow-up download.
|
|
201
|
+
- `instagram_profile_content` — discover Instagram profile grid content links for a handle or profile URL. Returns collected post/reel/tv URLs, profile counts, type counts, shortcodes, browser mode, pagination attempts, stop reason, and limitations. Pass `browserProfile` from `browser_profile_import/sync` plus higher `maxItems`/`maxScrolls` for authenticated pagination.
|
|
202
|
+
- `instagram_media_download` — extract and download one Instagram post/reel/tv URL. Returns text/caption, image URL/downloads, selected video/audio MP4 tracks, optional muxed MP4 when `ffmpeg` is available, optional transcript, and browser mode. Pass `browserProfile` for login-gated media.
|
|
201
203
|
- `maps_search` — search Google Maps for multiple business/profile candidates. Use for GMB/GBP prospect lists, competitors, categories, and anything needing more than the Google 3-pack. In default `proxyMode: "location"`, retryable failures rotate to a new residential proxy and new browser session for up to 5 attempts. `maxResults` defaults to 10 and is capped at 50.
|
|
202
204
|
- `maps_place_intel` — hydrate one known/named Google Maps business with profile details and optional reviews. Use after `maps_search` when a selected candidate needs full details.
|
|
203
205
|
- `directory_workflow` — build city-by-city directory/prospecting datasets from Census place selection plus Google Maps searches. Use it for requests like "all cities over 100k population in Tennessee, then get 20 roofers from Maps." In default `proxyMode: "location"`, each city search rotates retryable failures to a new residential proxy and new browser session for up to 5 attempts. The saved CSV includes `source_location`, `result_position`, `business_name`, `review_stars`, `review_count`, `category`, `address`, `phone`, `hours_status`, `website_url`, `directions_url`, `place_url`, `cid`, `cid_decimal`, Census population, and ZIP groups.
|
|
@@ -260,6 +262,7 @@ The `mcp-scraper` and `mcp-scraper-combined` NPX stdio servers also expose saved
|
|
|
260
262
|
- `MCP_SCRAPER_BROWSER_PROFILE` sets the default managed local browser profile name created by `mcp-scraper-cli browser import-chrome`.
|
|
261
263
|
- `MCP_SCRAPER_BROWSER_EXECUTABLE` optionally points local browser mode at a Chrome executable. On macOS, Google Chrome is recommended so Keychain-backed Chrome state works.
|
|
262
264
|
- `MCP_SCRAPER_BROWSER_PROFILE_DIR` optionally points local browser mode at a direct Chrome-compatible user data directory instead of a managed profile manifest.
|
|
265
|
+
- Instagram tools accept `browserMode` (`auto`, `hosted`, `local`) and `browserProfile`; `browserProfile` uses the same managed local profiles created by `browser_profile_import/sync`.
|
|
263
266
|
- `BROWSER_AGENT_PROFILE_NAME` is optional and sets the default saved Kernel browser profile for hosted-mode `browser-agent` and `mcp-scraper-combined` stdio sessions. Aliases: `BROWSER_SERVICE_PROFILE_NAME`, `KERNEL_BROWSER_PROFILE_NAME`, `KERNEL_PROFILE_NAME`.
|
|
264
267
|
- `BROWSER_AGENT_PROFILE_SAVE_CHANGES=true` is optional hosted setup mode. It persists cookies and local storage back to the named profile when `browser_close` deletes the hosted browser session. Aliases: `BROWSER_SERVICE_PROFILE_SAVE_CHANGES`, `KERNEL_BROWSER_PROFILE_SAVE_CHANGES`, `KERNEL_PROFILE_SAVE_CHANGES`.
|
|
265
268
|
|