@mrscraper/mcp 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +642 -0
  3. package/dist/api.d.ts +102 -0
  4. package/dist/api.d.ts.map +1 -0
  5. package/dist/api.js +203 -0
  6. package/dist/api.js.map +1 -0
  7. package/dist/app.d.ts +17 -0
  8. package/dist/app.d.ts.map +1 -0
  9. package/dist/app.js +123 -0
  10. package/dist/app.js.map +1 -0
  11. package/dist/auth.d.ts +7 -0
  12. package/dist/auth.d.ts.map +1 -0
  13. package/dist/auth.js +64 -0
  14. package/dist/auth.js.map +1 -0
  15. package/dist/bin.d.ts +3 -0
  16. package/dist/bin.d.ts.map +1 -0
  17. package/dist/bin.js +8 -0
  18. package/dist/bin.js.map +1 -0
  19. package/dist/compliance.d.ts +4 -0
  20. package/dist/compliance.d.ts.map +1 -0
  21. package/dist/compliance.js +10 -0
  22. package/dist/compliance.js.map +1 -0
  23. package/dist/config.d.ts +28 -0
  24. package/dist/config.d.ts.map +1 -0
  25. package/dist/config.js +63 -0
  26. package/dist/config.js.map +1 -0
  27. package/dist/http.d.ts +19 -0
  28. package/dist/http.d.ts.map +1 -0
  29. package/dist/http.js +156 -0
  30. package/dist/http.js.map +1 -0
  31. package/dist/index.d.ts +11 -0
  32. package/dist/index.d.ts.map +1 -0
  33. package/dist/index.js +11 -0
  34. package/dist/index.js.map +1 -0
  35. package/dist/runtime.d.ts +8 -0
  36. package/dist/runtime.d.ts.map +1 -0
  37. package/dist/runtime.js +46 -0
  38. package/dist/runtime.js.map +1 -0
  39. package/dist/server.d.ts +9 -0
  40. package/dist/server.d.ts.map +1 -0
  41. package/dist/server.js +24 -0
  42. package/dist/server.js.map +1 -0
  43. package/dist/status.d.ts +21 -0
  44. package/dist/status.d.ts.map +1 -0
  45. package/dist/status.js +62 -0
  46. package/dist/status.js.map +1 -0
  47. package/dist/tools.d.ts +122 -0
  48. package/dist/tools.d.ts.map +1 -0
  49. package/dist/tools.js +664 -0
  50. package/dist/tools.js.map +1 -0
  51. package/package.json +77 -0
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 MrScraper
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,642 @@
1
+ # MrScraper MCP
2
+
3
+ MrScraper MCP exposes the MrScraper web-data service through the
4
+ [Model Context Protocol](https://modelcontextprotocol.io). Agents can fetch a
5
+ known page, extract structured records, search Google, inspect account usage,
6
+ rerun saved scrapers, and read stored results.
7
+
8
+ The server exposes seven web-data tools:
9
+
10
+ ```text
11
+ fetch scrape serp status rerun results result
12
+ ```
13
+
14
+ ## Deployment options
15
+
16
+ | Option | Endpoint or command | Best for |
17
+ | ----------- | --------------------------------------------- | --------------------------------------- |
18
+ | Hosted | `https://mcp.mrscraper.com/mcp` | A managed Streamable HTTP connection |
19
+ | Local HTTP | `TRANSPORT=http npx -y @mrscraper/mcp@latest` | A self-managed HTTP service |
20
+ | Local stdio | `npx -y @mrscraper/mcp@latest` | MCP clients that launch a local process |
21
+
22
+ All modes use a MrScraper API key from
23
+ [app.mrscraper.com/api-tokens](https://app.mrscraper.com/api-tokens).
24
+
25
+ ## Connect to the hosted server
26
+
27
+ Configure a Streamable HTTP MCP connection and send the API key as a bearer
28
+ token:
29
+
30
+ ```json
31
+ {
32
+ "mcpServers": {
33
+ "mrscraper": {
34
+ "type": "http",
35
+ "url": "https://mcp.mrscraper.com/mcp",
36
+ "headers": {
37
+ "Authorization": "Bearer YOUR_MRSCRAPER_API_KEY"
38
+ }
39
+ }
40
+ }
41
+ }
42
+ ```
43
+
44
+ Use the environment-variable or secret-storage feature provided by your MCP
45
+ client so the key remains outside project files.
46
+
47
+ ### Codex
48
+
49
+ ```sh
50
+ export MRSCRAPER_API_KEY="YOUR_MRSCRAPER_API_KEY"
51
+ codex mcp add mrscraper \
52
+ --url https://mcp.mrscraper.com/mcp \
53
+ --bearer-token-env-var MRSCRAPER_API_KEY
54
+ ```
55
+
56
+ Start Codex from an environment containing `MRSCRAPER_API_KEY`. Codex stores
57
+ the environment-variable name in its MCP configuration.
58
+
59
+ ### Claude Code
60
+
61
+ ```sh
62
+ claude mcp add \
63
+ --transport http \
64
+ --scope user \
65
+ --header "Authorization: Bearer YOUR_MRSCRAPER_API_KEY" \
66
+ mrscraper https://mcp.mrscraper.com/mcp
67
+ ```
68
+
69
+ Claude Code also supports project and local scopes. Choose the scope that
70
+ matches how broadly the connection should be available.
71
+
72
+ ## Copyable agent setup prompt
73
+
74
+ ```text
75
+ Connect MrScraper MCP to this agent. Detect the current MCP client and configure a Streamable HTTP server named mrscraper at https://mcp.mrscraper.com/mcp. Use a MrScraper API key from https://app.mrscraper.com/api-tokens, store it through the client's environment-variable or secret-storage mechanism, and send it as Authorization: Bearer <key>. Reload the MCP client when required, then list the tools and confirm that fetch, scrape, serp, status, rerun, results, and result are available.
76
+ ```
77
+
78
+ ## Choosing a tool
79
+
80
+ | Tool | Choose it when |
81
+ | --------- | --------------------------------------------------------------------------------- |
82
+ | `fetch` | You already have a URL and need the page response from Web Unblocker. |
83
+ | `scrape` | You already have a URL and need defined fields, records, listings, or a site map. |
84
+ | `serp` | You are starting from a Google query or Google search URL. |
85
+ | `status` | You need subscription, quota, token usage, or domain request outcomes. |
86
+ | `rerun` | You have a saved AI or manual scraper ID and want to run it again. |
87
+ | `results` | You need a paginated or filtered list of stored results. |
88
+ | `result` | You know one result ID and need its complete stored record. |
89
+
90
+ A common discovery workflow is `serp` → `fetch` or `scrape`. A saved
91
+ scraper workflow is `scrape` → `rerun` → `result`.
92
+
93
+ ## Response contract
94
+
95
+ API-backed tools return a response envelope in both MCP `structuredContent`
96
+ and a formatted JSON text block:
97
+
98
+ ```json
99
+ {
100
+ "status_code": 200,
101
+ "data": {},
102
+ "headers": {
103
+ "content-type": "application/json"
104
+ }
105
+ }
106
+ ```
107
+
108
+ | Field | Type | Meaning |
109
+ | ------------- | ----------------------------- | ------------------------------------------------------------------------------------ |
110
+ | `status_code` | number or `null` | Upstream HTTP status, or `null` when a request ends before an HTTP response arrives. |
111
+ | `data` | JSON value, string, or `null` | Parsed JSON or response text supplied by the MrScraper service. |
112
+ | `headers` | object | Response headers with credential-bearing headers filtered out. |
113
+ | `error` | string, when present | Request failure summary. |
114
+
115
+ An API failure keeps this envelope in `structuredContent` and sets the MCP
116
+ result's `isError` flag. Input-contract errors are returned as MCP tool
117
+ errors. Parsed credential metadata and generated curl credentials are
118
+ sanitized while extracted scraper data remains available to the caller.
119
+
120
+ ## Tool reference
121
+
122
+ ### `fetch`
123
+
124
+ `fetch` calls the Web Unblocker endpoint once:
125
+
126
+ ```text
127
+ GET https://api.mrscraper.com/
128
+ ```
129
+
130
+ Start with the URL alone:
131
+
132
+ ```json
133
+ {
134
+ "url": "https://example.com/products"
135
+ }
136
+ ```
137
+
138
+ Use browser rendering for JavaScript-driven content:
139
+
140
+ ```json
141
+ {
142
+ "url": "https://example.com/products",
143
+ "browser_rendering": true,
144
+ "wait_for_selector": ".product-card",
145
+ "geo_code": "ID",
146
+ "timeout": 45
147
+ }
148
+ ```
149
+
150
+ | Input | Required | Default | API mapping | Purpose |
151
+ | ------------------- | -------- | ------- | ------------------------ | ------------------------------------------------------------------------------------ |
152
+ | `url` | Yes | — | Query `url` | Target page URL. |
153
+ | `browser_rendering` | No | `false` | Query `browserRendering` | Executes page JavaScript in a browser. |
154
+ | `geo_code` | No | omitted | Query `geoCode` | Selects proxy-country routing. |
155
+ | `wait_for_selector` | No | omitted | Query `waitForSelector` | Waits for a CSS selector together with `browser_rendering: true`. |
156
+ | `home_page` | No | `false` | Query `homePage` | Visits the site root before the target URL. |
157
+ | `block_resources` | No | `false` | Query `blockResources` | Applies resource blocking during page loading. |
158
+ | `max_retries` | No | `3` | Query `maxRetries` | Sets the Web Unblocker retry limit; zero is accepted. |
159
+ | `token_cap` | No | omitted | Query `tokenCap` | Sets the retry token budget. |
160
+ | `timeout` | No | `30` | Query `timeout` | Sets the page-load deadline in seconds; transport receives an additional 30 seconds. |
161
+
162
+ The response body's value is returned in `data`, commonly as HTML.
163
+
164
+ ### `scrape`
165
+
166
+ `scrape` creates an AI scraper through:
167
+
168
+ ```text
169
+ POST https://api.app.mrscraper.com/api/v1/scrapers-ai
170
+ ```
171
+
172
+ Choose an agent based on the extraction shape:
173
+
174
+ | Agent | Designed for | Available inputs |
175
+ | --------- | ------------------------------------- | ------------------------------------------------------------------------- |
176
+ | `general` | Defined fields from a page | `prompt`, `schema_prompt`, `proxy_country` |
177
+ | `listing` | Repeated records across listing pages | `prompt`, `schema_prompt`, `proxy_country`, `max_pages` |
178
+ | `map` | URL discovery across a site | `max_depth`, `max_pages`, `limit`, `include_patterns`, `exclude_patterns` |
179
+
180
+ General extraction:
181
+
182
+ ```json
183
+ {
184
+ "url": "https://example.com/product",
185
+ "agent": "general",
186
+ "prompt": "Extract the product name, price, availability, and image URLs"
187
+ }
188
+ ```
189
+
190
+ Listing extraction:
191
+
192
+ ```json
193
+ {
194
+ "url": "https://example.com/products",
195
+ "agent": "listing",
196
+ "prompt": "Extract every product name, price, and detail URL",
197
+ "max_pages": 5
198
+ }
199
+ ```
200
+
201
+ Site map:
202
+
203
+ ```json
204
+ {
205
+ "url": "https://example.com",
206
+ "agent": "map",
207
+ "max_depth": 2,
208
+ "max_pages": 50,
209
+ "limit": 1000,
210
+ "include_patterns": "/products/",
211
+ "exclude_patterns": "/cart/"
212
+ }
213
+ ```
214
+
215
+ Best-effort schema guidance:
216
+
217
+ ```json
218
+ {
219
+ "url": "https://example.com/product",
220
+ "prompt": "Extract the product",
221
+ "schema_prompt": {
222
+ "type": "object",
223
+ "properties": {
224
+ "name": { "type": "string" },
225
+ "price": { "type": "number" }
226
+ },
227
+ "required": ["name", "price"]
228
+ }
229
+ }
230
+ ```
231
+
232
+ `schema_prompt` is appended to the natural-language instruction. Treat it as
233
+ shape guidance and validate returned data in the consuming application when
234
+ strict conformance is required.
235
+
236
+ | Input | Required | Default | Request mapping | Purpose |
237
+ | ------------------ | --------------- | --------------- | ---------------------- | ----------------------------------------------------------- |
238
+ | `url` | Yes | — | Body `url` | Target URL for every agent. |
239
+ | `prompt` | General/listing | — | Body `message` | Natural-language extraction instructions. |
240
+ | `schema_prompt` | No | omitted | Appended to `message` | Best-effort JSON Schema shape guidance for general/listing. |
241
+ | `agent` | No | `general` | Body `agent` | Selects `general`, `listing`, or `map`. |
242
+ | `proxy_country` | No | omitted | Body `proxyCountry` | Proxy country for general/listing. |
243
+ | `max_pages` | No | service default | Body `maxPages` | Page bound for listing/map. |
244
+ | `max_depth` | No | service default | Body `maxDepth` | Link-depth bound for map. |
245
+ | `limit` | No | service default | Body `limit` | URL-result bound for map. |
246
+ | `include_patterns` | No | service default | Body `includePatterns` | URL inclusion expression for map. |
247
+ | `exclude_patterns` | No | service default | Body `excludePatterns` | URL exclusion expression for map. |
248
+
249
+ General and listing send `url`, `message`, and `agent`, plus the supplied
250
+ agent inputs. Map sends `url`, `agent`, and only the supplied crawl inputs.
251
+
252
+ #### Reproduce a scrape with `rerun`
253
+
254
+ Every successful `scrape` creates a saved AI scraper configuration by default.
255
+ Its response run object contains `scraperId`. Pass that UUID to `rerun` as
256
+ `scraper_id` to apply the same saved extraction configuration to the original
257
+ URL or another URL without rebuilding the prompt and agent settings:
258
+
259
+ ```json
260
+ {
261
+ "target": "https://example.com/product-2",
262
+ "type": "ai",
263
+ "scraper_id": "scraper-uuid"
264
+ }
265
+ ```
266
+
267
+ This makes the scraper configuration reproducible, but it does not guarantee
268
+ identical extracted values when the page or model behavior changes.
269
+
270
+ `rerun` also supports dashboard-built manual workflows and asynchronous bulk
271
+ jobs across multiple target URLs. See the full [`rerun`](#rerun) section below
272
+ for the available modes and result-tracking workflow.
273
+
274
+ ### `serp`
275
+
276
+ `serp` calls the synchronous Google endpoint:
277
+
278
+ ```text
279
+ POST https://sync.scraper.mrscraper.com/api/google/serp/v2/sync
280
+ ```
281
+
282
+ Search from a query:
283
+
284
+ ```json
285
+ {
286
+ "query_or_url": "iphone 17",
287
+ "region": "id",
288
+ "language": "id",
289
+ "page": 2,
290
+ "format": "json"
291
+ }
292
+ ```
293
+
294
+ A Google search URL can supply the query, locale, and page:
295
+
296
+ ```json
297
+ {
298
+ "query_or_url": "https://www.google.com/search?q=iphone+17&gl=us&hl=en&start=20"
299
+ }
300
+ ```
301
+
302
+ The server derives `query` from `q`, `region` from `gl`, `language`
303
+ from `hl`, and a one-based page number from `start). Explicit tool inputs
304
+ take priority over URL-derived values.
305
+
306
+ | Input | Required | Default | Request mapping | Purpose |
307
+ | ---------------- | -------- | -------------------- | ---------------------- | -------------------------------------------- |
308
+ | `query_or_url` | Yes | — | Body `query` | Google query or Google search URL. |
309
+ | `region` | No | URL value or omitted | Body `region` | Result country. |
310
+ | `language` | No | URL value or omitted | Body `language` | Result language. |
311
+ | `page` | No | URL value or omitted | Body `page` | One-based result page. |
312
+ | `format` | No | `json` | Body `format` | Selects parsed JSON or result-page HTML. |
313
+ | `render_js` | No | `false` | Body `renderJs` | Waits for JavaScript-rendered SERP features. |
314
+ | `raw` | No | `false` | Body `format=html` | Compatibility alias for HTML output. |
315
+ | `client_timeout` | No | `120` | Local request deadline | Sets the upstream HTTP timeout in seconds. |
316
+
317
+ ### `status`
318
+
319
+ `status` always reads account data from:
320
+
321
+ ```text
322
+ GET https://api.app.mrscraper.com/api/v1/subscription-accounts
323
+ ```
324
+
325
+ With `domain`, it also reads request-outcome analytics from:
326
+
327
+ ```text
328
+ GET https://api.app.mrscraper.com/api/v1/analytic/statuses
329
+ ```
330
+
331
+ ```json
332
+ {
333
+ "domain": "https://www.example.com/products",
334
+ "from": "7d",
335
+ "to": "now",
336
+ "action": "fetch",
337
+ "api_token_name": "production"
338
+ }
339
+ ```
340
+
341
+ | Input | Required | Default | Purpose |
342
+ | ---------------- | -------- | ------------ | ------------------------------------------------------------------------------------------- |
343
+ | `domain` | No | omitted | Adds request-outcome analytics for a hostname or URL. |
344
+ | `from` | No | `24h` | Range start as ISO 8601, `now`, or a relative duration such as `30m`, `24h`, `7d`, or `2w`. |
345
+ | `to` | No | `now` | Range end using the same date syntax. |
346
+ | `action` | No | empty filter | Filters analytics by exact action. |
347
+ | `api_token_name` | No | empty filter | Filters analytics by API-token name. |
348
+
349
+ Successful output is a normalized account and analytics summary:
350
+
351
+ ```json
352
+ {
353
+ "kind": "mrscraper-cli-status-summary",
354
+ "source_endpoints": ["/subscription-accounts", "/analytic/statuses"],
355
+ "status_code": 200,
356
+ "data": {
357
+ "account": {
358
+ "subscription_status": "active",
359
+ "enterprise": false,
360
+ "token_usage": 250,
361
+ "token_limit": 1000,
362
+ "token_remaining": 750,
363
+ "usage_percent": 25,
364
+ "rate_limit": 10,
365
+ "rate_ttl": 60,
366
+ "auto_renew": true,
367
+ "ends_at": null,
368
+ "user": {
369
+ "name": "Ada",
370
+ "email": "ada@example.com",
371
+ "verified": true
372
+ }
373
+ },
374
+ "analytics": {
375
+ "domain": "www.example.com",
376
+ "from": "2026-08-11 00:00:00 UTC",
377
+ "to": "2026-08-18 00:00:00 UTC"
378
+ }
379
+ }
380
+ }
381
+ ```
382
+
383
+ The summary calculates `token_remaining` and `usage_percent`, normalizes a
384
+ URL to its hostname, and records the source endpoints used for the response.
385
+
386
+ ### `rerun`
387
+
388
+ `type` and `bulk` describe different parts of the request:
389
+
390
+ - Set `type` to `ai` for a saved AI scraper created by the `scrape` tool.
391
+ - Set `type` to `manual` for a saved step-based workflow created in the
392
+ MrScraper dashboard. MCP reruns that workflow but does not create it.
393
+ - Leave `bulk` as `false` for one URL. Set it to `true` to apply the same saved
394
+ configuration to a comma- or newline-separated URL list in one bulk request.
395
+
396
+ Manual reruns can be single or bulk, and bulk reruns can use either scraper
397
+ type. Bulk describes the number of targets; manual describes how the saved
398
+ scraper was built.
399
+
400
+ Bulk mode submits one asynchronous backend job; MCP does not repeat the
401
+ single-URL tool call locally. Save `data.data.bulkResultId` from the response
402
+ and pass it to `result` as `result_id` until the stored result is finished.
403
+
404
+ `rerun` routes one tool contract to four saved-scraper endpoints:
405
+
406
+ | Mode | Endpoint | ID input | Target format | Crawl controls |
407
+ | ------------- | ---------------------------------- | ------------ | ---------------------------- | ------------------- |
408
+ | Single AI | `POST /scrapers-ai-rerun` | `scraper_id` | One URL | Available |
409
+ | Bulk AI | `POST /scrapers-ai-rerun/bulk` | `id` | Comma/newline-separated URLs | Saved configuration |
410
+ | Single manual | `POST /scrapers-manual-rerun` | `scraper_id` | One URL | Saved configuration |
411
+ | Bulk manual | `POST /scrapers-manual-rerun/bulk` | `id` | Comma/newline-separated URLs | Saved configuration |
412
+
413
+ Single AI rerun:
414
+
415
+ ```json
416
+ {
417
+ "target": "https://example.com/products",
418
+ "type": "ai",
419
+ "scraper_id": "scraper-uuid",
420
+ "max_depth": 2,
421
+ "max_pages": 50,
422
+ "limit": 1000,
423
+ "include_patterns": "/products/",
424
+ "exclude_patterns": "/cart/"
425
+ }
426
+ ```
427
+
428
+ Bulk manual rerun:
429
+
430
+ ```json
431
+ {
432
+ "target": "https://example.com/a,https://example.com/b\nhttps://example.com/c",
433
+ "type": "manual",
434
+ "bulk": true,
435
+ "id": "scraper-uuid"
436
+ }
437
+ ```
438
+
439
+ | Input | Required | Default | Purpose |
440
+ | ------------------ | ----------- | ------------ | --------------------------------------------------------------- |
441
+ | `target` | Yes | — | One URL, or a comma/newline-separated URL string for bulk mode. |
442
+ | `type` | Yes | — | Selects `ai` or `manual`. |
443
+ | `bulk` | No | `false` | Selects a bulk endpoint. |
444
+ | `scraper_id` | Single mode | — | Saved scraper UUID for one URL. |
445
+ | `id` | Bulk mode | — | Saved scraper UUID for the bulk URL list. |
446
+ | `max_depth` | Single AI | `2` | Crawl depth. |
447
+ | `max_pages` | Single AI | `50` | Page bound. |
448
+ | `limit` | Single AI | `1000` | Result bound. |
449
+ | `include_patterns` | Single AI | empty string | URL inclusion expression. |
450
+ | `exclude_patterns` | Single AI | empty string | URL exclusion expression. |
451
+
452
+ Manual reruns carry a compliance acknowledgment in the MCP server
453
+ instructions. MCP clients should present that acknowledgment before executing
454
+ the manual mode.
455
+
456
+ ### `results`
457
+
458
+ `results` reads the stored result collection:
459
+
460
+ ```text
461
+ GET https://api.app.mrscraper.com/api/v1/results
462
+ ```
463
+
464
+ ```json
465
+ {
466
+ "sort_field": "updatedAt",
467
+ "sort_order": "desc",
468
+ "page_size": 25,
469
+ "page": 1,
470
+ "search": "example.com",
471
+ "date_range_column": "updatedAt",
472
+ "start_at": "2026-08-01T00:00:00Z",
473
+ "end_at": "2026-08-18T23:59:59Z"
474
+ }
475
+ ```
476
+
477
+ | Input | Required | Default | Query mapping | Purpose |
478
+ | ------------------- | -------- | ----------- | ----------------- | ------------------------------------------------------------- |
479
+ | `sort_field` | No | `updatedAt` | `sortField` | Field used by the results API for sorting. |
480
+ | `sort_order` | No | `desc` | `sortOrder` | Case-insensitive `asc` or `desc`; sent upstream in uppercase. |
481
+ | `page_size` | No | `10` | `pageSize` | Number of rows per page. |
482
+ | `page` | No | `1` | `page` | One-based page index. |
483
+ | `search` | No | omitted | `search` | Free-text result filter. |
484
+ | `date_range_column` | No | omitted | `dateRangeColumn` | Column used by the date range. |
485
+ | `start_at` | No | omitted | `startAt` | Inclusive range start. |
486
+ | `end_at` | No | omitted | `endAt` | Inclusive range end. |
487
+
488
+ ### `result`
489
+
490
+ `result` reads one stored result:
491
+
492
+ ```text
493
+ GET https://api.app.mrscraper.com/api/v1/results/{result_id}
494
+ ```
495
+
496
+ ```json
497
+ {
498
+ "result_id": "result-uuid"
499
+ }
500
+ ```
501
+
502
+ | Input | Required | Purpose |
503
+ | ----------- | -------- | ----------------------------- |
504
+ | `result_id` | Yes | Stored MrScraper result UUID. |
505
+
506
+ ## Run the server locally
507
+
508
+ Node.js 20 or newer is required.
509
+
510
+ ```sh
511
+ git clone https://github.com/mrscraper-com/mrscraper-mcp.git
512
+ cd mrscraper-mcp
513
+ npm ci
514
+ npm run build
515
+ ```
516
+
517
+ ### Streamable HTTP
518
+
519
+ ```sh
520
+ TRANSPORT=http npm start
521
+ ```
522
+
523
+ The default endpoint is `http://127.0.0.1:8000/mcp`.
524
+
525
+ ```json
526
+ {
527
+ "mcpServers": {
528
+ "mrscraper": {
529
+ "type": "http",
530
+ "url": "http://127.0.0.1:8000/mcp",
531
+ "headers": {
532
+ "Authorization": "Bearer YOUR_MRSCRAPER_API_KEY"
533
+ }
534
+ }
535
+ }
536
+ }
537
+ ```
538
+
539
+ ### stdio
540
+
541
+ ```json
542
+ {
543
+ "mcpServers": {
544
+ "mrscraper": {
545
+ "command": "npx",
546
+ "args": ["-y", "@mrscraper/mcp@latest"],
547
+ "env": {
548
+ "MRSCRAPER_API_KEY": "YOUR_MRSCRAPER_API_KEY"
549
+ }
550
+ }
551
+ }
552
+ }
553
+ ```
554
+
555
+ Stdio credential precedence is `MRSCRAPER_API_KEY`, then
556
+ `MRSCRAPER_API_TOKEN`.
557
+
558
+ ## Authentication and security
559
+
560
+ | Transport | Credential source |
561
+ | -------------------- | ----------------------------------------------------------------- |
562
+ | Hosted or local HTTP | `Authorization: Bearer <MrScraper API key>` on the MCP connection |
563
+ | Local stdio | `MRSCRAPER_API_KEY`, then `MRSCRAPER_API_TOKEN` |
564
+
565
+ HTTP bearer tokens are validated against the MrScraper account endpoint before
566
+ tool execution. Each HTTP request uses its caller's credential.
567
+
568
+ The server filters credential-bearing response headers. Parsed JSON credential
569
+ metadata and credentials embedded in generated curl commands are redacted.
570
+ Scraper run extraction values remain available in `data`.
571
+
572
+ Browser-origin requests are accepted from trusted local origins and exact
573
+ origins configured through `MRSCRAPER_ALLOWED_ORIGINS`. Service-to-service
574
+ MCP clients typically connect without an `Origin` header.
575
+
576
+ ## Docker
577
+
578
+ ```sh
579
+ docker build -f docker/Dockerfile -t mrscraper-mcp .
580
+ docker run --rm -p 8000:8000 mrscraper-mcp
581
+ ```
582
+
583
+ The image binds to `0.0.0.0`. Apply the network controls appropriate for the
584
+ deployment and keep bearer authentication enabled.
585
+
586
+ ## Environment variables
587
+
588
+ | Variable | Default | Purpose |
589
+ | -------------------------------- | --------------------------------- | ---------------------------------------------------------------- |
590
+ | `TRANSPORT` | `stdio` | Selects `stdio` or `http`. |
591
+ | `HOST` | `127.0.0.1` | HTTP bind address; the Docker image uses `0.0.0.0`. |
592
+ | `PORT` | `8000` | HTTP listen port. |
593
+ | `MRSCRAPER_API_KEY` | — | Primary stdio credential. |
594
+ | `MRSCRAPER_API_TOKEN` | — | Legacy stdio credential alias. |
595
+ | `MRSCRAPER_HTTP_AUTH` | `1` | Enables HTTP bearer verification. |
596
+ | `MRSCRAPER_ALLOWED_ORIGINS` | — | Comma-separated browser origins allowed to call the HTTP server. |
597
+ | `MRSCRAPER_API_BASE_URL` | MrScraper platform API | Platform endpoint override for development and testing. |
598
+ | `MRSCRAPER_FETCH_BASE_URL` | MrScraper Web Unblocker | Fetch endpoint override. |
599
+ | `MRSCRAPER_SYNC_BASE_URL` | MrScraper synchronous scraper API | SERP endpoint override. |
600
+ | `MRSCRAPER_LOG_HTTP_PAYLOAD` | off | Enables trusted-environment request-body diagnostics. |
601
+ | `MRSCRAPER_LOG_HTTP_PAYLOAD_MAX` | `8192` | Maximum diagnostic payload length. |
602
+
603
+ ## Troubleshooting
604
+
605
+ - **Connection error:** Confirm the server URL ends in `/mcp` and reload the
606
+ MCP client after configuration changes.
607
+ - **401 Unauthorized:** Confirm the bearer key for HTTP or the environment key
608
+ for stdio.
609
+ - **403 Forbidden Origin:** Add the exact browser origin to
610
+ `MRSCRAPER_ALLOWED_ORIGINS`.
611
+ - **Tool input error:** Compare the call with the tool's parameter table and
612
+ the selected scrape or rerun mode.
613
+ - **Upstream timeout:** Increase `fetch.timeout` or `serp.client_timeout`
614
+ when the target requires a longer request window.
615
+ - **New tools are missing:** Start a new client session and inspect
616
+ `tools/list` for all seven names.
617
+
618
+ ## Development
619
+
620
+ ```sh
621
+ npm ci
622
+ npm run format:check
623
+ npm run lint
624
+ npm test
625
+ npm run build
626
+ npm pack --dry-run
627
+ ```
628
+
629
+ Smoke-test a running HTTP server:
630
+
631
+ ```sh
632
+ npm run test:mcp -- \
633
+ --target http://127.0.0.1:8000/mcp \
634
+ --token "$MRSCRAPER_API_KEY"
635
+ ```
636
+
637
+ `npm run test:e2e` exercises all seven tools against the live MrScraper
638
+ service and creates stored scraper results.
639
+
640
+ ## License
641
+
642
+ MIT — see [LICENSE](./LICENSE).