@mrscraper/mcp 0.1.3 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +235 -409
- package/dist/api.d.ts +26 -87
- package/dist/api.d.ts.map +1 -1
- package/dist/api.js +71 -188
- package/dist/api.js.map +1 -1
- package/dist/config.d.ts +5 -12
- package/dist/config.d.ts.map +1 -1
- package/dist/config.js +6 -25
- package/dist/config.js.map +1 -1
- package/dist/http.d.ts.map +1 -1
- package/dist/http.js +5 -1
- package/dist/http.js.map +1 -1
- package/dist/index.d.ts +0 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +0 -2
- package/dist/index.js.map +1 -1
- package/dist/scopes.d.ts.map +1 -1
- package/dist/scopes.js +2 -6
- package/dist/scopes.js.map +1 -1
- package/dist/server.d.ts.map +1 -1
- package/dist/server.js +3 -16
- package/dist/server.js.map +1 -1
- package/dist/status.d.ts +0 -2
- package/dist/status.d.ts.map +1 -1
- package/dist/status.js +2 -30
- package/dist/status.js.map +1 -1
- package/dist/tools.d.ts +30 -99
- package/dist/tools.d.ts.map +1 -1
- package/dist/tools.js +184 -637
- package/dist/tools.js.map +1 -1
- package/dist/widgets/bundles.generated.d.ts.map +1 -1
- package/dist/widgets/bundles.generated.js +2 -3
- package/dist/widgets/bundles.generated.js.map +1 -1
- package/dist/widgets/index.d.ts +1 -7
- package/dist/widgets/index.d.ts.map +1 -1
- package/dist/widgets/index.js +2 -8
- package/dist/widgets/index.js.map +1 -1
- package/package.json +4 -4
- package/dist/compliance.d.ts +0 -4
- package/dist/compliance.d.ts.map +0 -1
- package/dist/compliance.js +0 -10
- package/dist/compliance.js.map +0 -1
package/README.md
CHANGED
|
@@ -1,14 +1,21 @@
|
|
|
1
|
-
# MrScraper MCP
|
|
1
|
+
# MrScraper MCP Server for Web Scraping and Google Search
|
|
2
2
|
|
|
3
|
-
MrScraper MCP
|
|
4
|
-
[Model Context Protocol](https://modelcontextprotocol.io).
|
|
5
|
-
|
|
6
|
-
|
|
3
|
+
MrScraper MCP is the official [MrScraper](https://mrscraper.com) server for the
|
|
4
|
+
[Model Context Protocol](https://modelcontextprotocol.io). It connects AI
|
|
5
|
+
assistants to page content and Google search through two SDK-compatible tools,
|
|
6
|
+
plus an account check. Retrieve HTML, Markdown, or screenshots with
|
|
7
|
+
`scraperAPI`, discover pages with `webSearch`, and check your plan and token
|
|
8
|
+
usage with `status`.
|
|
7
9
|
|
|
8
|
-
|
|
10
|
+
Use the hosted server with OAuth sign-in, or run the open-source server locally
|
|
11
|
+
with the [`@mrscraper/mcp` npm package](https://www.npmjs.com/package/@mrscraper/mcp).
|
|
12
|
+
See the [MrScraper MCP documentation](https://docs.mrscraper.com/docs/getting-started/mcp-server)
|
|
13
|
+
for setup instructions and client configurations.
|
|
14
|
+
|
|
15
|
+
The server exposes three tools:
|
|
9
16
|
|
|
10
17
|
```text
|
|
11
|
-
|
|
18
|
+
scraperAPI webSearch status
|
|
12
19
|
```
|
|
13
20
|
|
|
14
21
|
## Deployment options
|
|
@@ -54,11 +61,10 @@ the address will be refused.
|
|
|
54
61
|
|
|
55
62
|
### What you are approving
|
|
56
63
|
|
|
57
|
-
| Scope | Grants
|
|
58
|
-
| -------------- |
|
|
59
|
-
| `scrape:read` | `
|
|
60
|
-
| `
|
|
61
|
-
| `account:read` | `status`: subscription plan, quota, and request analytics |
|
|
64
|
+
| Scope | Grants |
|
|
65
|
+
| -------------- | ---------------------------- |
|
|
66
|
+
| `scrape:read` | `scraperAPI` and `webSearch` |
|
|
67
|
+
| `account:read` | `status` |
|
|
62
68
|
|
|
63
69
|
A tool called without its scope is refused with `403 insufficient_scope`, and
|
|
64
70
|
your client will offer to grant the missing permission.
|
|
@@ -118,62 +124,51 @@ matches how broadly the connection should be available.
|
|
|
118
124
|
## Copyable agent setup prompt
|
|
119
125
|
|
|
120
126
|
```text
|
|
121
|
-
Connect MrScraper MCP to this agent. Detect the current MCP client and configure a Streamable HTTP server named mrscraper at https://mcp.mrscraper.com/mcp with no credential. The server uses OAuth 2.1, so the client will open a browser to sign in. If the client cannot do OAuth, fall back to a MrScraper API key from https://app.mrscraper.com/api-tokens stored through the client's environment-variable or secret-storage mechanism and sent as Authorization: Bearer <key>. Reload the MCP client when required, then list the tools and confirm that
|
|
127
|
+
Connect MrScraper MCP to this agent. Detect the current MCP client and configure a Streamable HTTP server named mrscraper at https://mcp.mrscraper.com/mcp with no credential. The server uses OAuth 2.1, so the client will open a browser to sign in. If the client cannot do OAuth, fall back to a MrScraper API key from https://app.mrscraper.com/api-tokens stored through the client's environment-variable or secret-storage mechanism and sent as Authorization: Bearer <key>. Reload the MCP client when required, then list the tools and confirm that scraperAPI, webSearch, and status are the only available tools.
|
|
122
128
|
```
|
|
123
129
|
|
|
124
130
|
## Choosing a tool
|
|
125
131
|
|
|
126
|
-
| Tool
|
|
127
|
-
|
|
|
128
|
-
| `
|
|
129
|
-
| `
|
|
130
|
-
| `
|
|
131
|
-
| `status` | You need subscription, quota, token usage, or domain request outcomes. |
|
|
132
|
-
| `rerun` | You have a saved AI or manual scraper ID and want to run it again. |
|
|
133
|
-
| `results` | You need a paginated or filtered list of stored results. |
|
|
134
|
-
| `result` | You know one result ID and need its complete stored record. |
|
|
132
|
+
| Tool | Choose it when |
|
|
133
|
+
| ------------ | ---------------------------------------------------------------------------- |
|
|
134
|
+
| `scraperAPI` | You have a page URL and need HTML, Markdown, a screenshot, or a combination. |
|
|
135
|
+
| `webSearch` | You have a Google query or Google search URL and need search results. |
|
|
136
|
+
| `status` | You need the account's plan, token usage, or remaining tokens. |
|
|
135
137
|
|
|
136
|
-
A
|
|
137
|
-
|
|
138
|
+
A discovery workflow is `webSearch` → `scraperAPI`. Both web tools advertise
|
|
139
|
+
`readOnlyHint: true`, `openWorldHint: true`, and `destructiveHint: false`;
|
|
140
|
+
`status` is read-only and not open-world. Authentication comes from the MCP
|
|
141
|
+
connection; no tool takes an API token argument.
|
|
138
142
|
|
|
139
|
-
|
|
143
|
+
### Migration from the previous tool set
|
|
140
144
|
|
|
141
|
-
|
|
142
|
-
|
|
145
|
+
Only `scraperAPI`, `webSearch`, and `status` are registered. The other old tool
|
|
146
|
+
names are removed.
|
|
143
147
|
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
}
|
|
151
|
-
}
|
|
152
|
-
```
|
|
148
|
+
| Previous call | Replacement |
|
|
149
|
+
| -------------------------------------- | -------------------------------------------------------------------------------------------------------- |
|
|
150
|
+
| `fetch` | `scraperAPI`; uses POST, SDK options, and automatic browser fallback by default. |
|
|
151
|
+
| `serp` | `webSearch`; rename `query_or_url` to `url` and `client_timeout` to `timeout`. Set `raw=false` for JSON. |
|
|
152
|
+
| `status` | `status` for account usage; the `domain`, `from`, `to`, `action`, and `api_token_name` inputs are gone. |
|
|
153
|
+
| `scrape`, `rerun`, `results`, `result` | Removed from this MCP server. |
|
|
153
154
|
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
| `data` | JSON value, string, or `null` | Parsed JSON or response text supplied by the MrScraper service. |
|
|
158
|
-
| `headers` | object | Response headers with credential-bearing headers filtered out. |
|
|
159
|
-
| `error` | string, when present | Request failure summary. |
|
|
160
|
-
|
|
161
|
-
An API failure keeps this envelope in `structuredContent` and sets the MCP
|
|
162
|
-
result's `isError` flag. Input-contract errors are returned as MCP tool
|
|
163
|
-
errors. Parsed credential metadata and generated curl credentials are
|
|
164
|
-
sanitized while extracted scraper data remains available to the caller.
|
|
155
|
+
`super_mode` becomes `super`. `home_page` now maps to `warmStart`, and
|
|
156
|
+
`token_cap=0` is accepted. Restart the server and reconnect the MCP client after
|
|
157
|
+
updating so it refreshes `tools/list`.
|
|
165
158
|
|
|
166
159
|
## Tool reference
|
|
167
160
|
|
|
168
|
-
|
|
161
|
+
The options and defaults of `scraperAPI` and `webSearch` follow the Python SDK's
|
|
162
|
+
`scraper_api` and `web_search` methods. MCP tool names use camel case; their
|
|
163
|
+
argument names use the SDK's snake case.
|
|
169
164
|
|
|
170
|
-
`
|
|
165
|
+
### `scraperAPI`
|
|
171
166
|
|
|
172
167
|
```text
|
|
173
|
-
|
|
168
|
+
POST https://api.mrscraper.com/
|
|
174
169
|
```
|
|
175
170
|
|
|
176
|
-
Start with
|
|
171
|
+
Start with a known URL:
|
|
177
172
|
|
|
178
173
|
```json
|
|
179
174
|
{
|
|
@@ -181,424 +176,242 @@ Start with the URL alone:
|
|
|
181
176
|
}
|
|
182
177
|
```
|
|
183
178
|
|
|
184
|
-
|
|
185
|
-
inputs can return different results for the same URL:
|
|
186
|
-
|
|
187
|
-
| Browser rendering | Super Mode | Input | Loading path |
|
|
188
|
-
| ----------------- | ---------- | ----------------------------------------------------------------- | -------------------------------------------------------- |
|
|
189
|
-
| Off | Off | `{ "url": "URL" }` | Standard routing with the non-browser loader. |
|
|
190
|
-
| On | Off | `{ "url": "URL", "browser_rendering": true }` | Standard routing with browser loading and JavaScript. |
|
|
191
|
-
| Off | On | `{ "url": "URL", "super_mode": true }` | Real-device routing with the non-browser loader. |
|
|
192
|
-
| On | On | `{ "url": "URL", "browser_rendering": true, "super_mode": true }` | Real-device routing with browser loading and JavaScript. |
|
|
193
|
-
|
|
194
|
-
Start with both controls off, inspect the response, and change one axis at a
|
|
195
|
-
time when needed. Browser rendering is not guaranteed to work better: some
|
|
196
|
-
sites fail with it enabled but load without it. Try the remaining combinations
|
|
197
|
-
without repeating an identical failed request.
|
|
198
|
-
|
|
199
|
-
Use browser rendering for JavaScript-driven content:
|
|
179
|
+
Request HTML, Markdown, and a screenshot together:
|
|
200
180
|
|
|
201
181
|
```json
|
|
202
182
|
{
|
|
203
|
-
"url": "https://
|
|
204
|
-
"
|
|
205
|
-
"
|
|
183
|
+
"url": "https://example.com",
|
|
184
|
+
"html": true,
|
|
185
|
+
"markdown": true,
|
|
186
|
+
"screenshot": true,
|
|
187
|
+
"super": true,
|
|
188
|
+
"home_page": true,
|
|
206
189
|
"geo_code": "ID",
|
|
207
|
-
"timeout":
|
|
208
|
-
}
|
|
209
|
-
```
|
|
210
|
-
|
|
211
|
-
Use both controls for real-device browser loading:
|
|
212
|
-
|
|
213
|
-
```json
|
|
214
|
-
{
|
|
215
|
-
"url": "https://www.scrapethissite.com/pages/ajax-javascript/",
|
|
216
|
-
"browser_rendering": true,
|
|
217
|
-
"super_mode": true
|
|
218
|
-
}
|
|
219
|
-
```
|
|
220
|
-
|
|
221
|
-
| Input | Required | Default | API mapping | Purpose |
|
|
222
|
-
| ------------------- | -------- | ------- | ------------------------ | ------------------------------------------------------------------------------------ |
|
|
223
|
-
| `url` | Yes | - | Query `url` | Target page URL. |
|
|
224
|
-
| `browser_rendering` | No | `false` | Query `browserRendering` | Executes page JavaScript in a browser. |
|
|
225
|
-
| `super_mode` | No | `false` | Query `super` | Selects real-device routing independently of browser rendering. |
|
|
226
|
-
| `geo_code` | No | omitted | Query `geoCode` | Selects proxy-country routing. |
|
|
227
|
-
| `wait_for_selector` | No | omitted | Query `waitForSelector` | Waits for a CSS selector together with `browser_rendering: true`. |
|
|
228
|
-
| `home_page` | No | `false` | Query `homePage` | Visits the site root before the target URL. |
|
|
229
|
-
| `block_resources` | No | `false` | Query `blockResources` | Applies resource blocking during page loading. |
|
|
230
|
-
| `max_retries` | No | `3` | Query `maxRetries` | Sets the Web Unblocker retry limit; zero is accepted. |
|
|
231
|
-
| `token_cap` | No | omitted | Query `tokenCap` | Sets the retry token budget. |
|
|
232
|
-
| `timeout` | No | `30` | Query `timeout` | Sets the page-load deadline in seconds; transport receives an additional 30 seconds. |
|
|
233
|
-
|
|
234
|
-
The response body's value is returned in `data`, commonly as HTML. Toggle
|
|
235
|
-
`browser_rendering` for JavaScript needs and `super_mode` for routing needs.
|
|
236
|
-
When browser loading fails, is blocked, or returns worse content, retry the same
|
|
237
|
-
`super_mode` value with `browser_rendering: false`. Stop after a usable response
|
|
238
|
-
unless the task requires comparing modes.
|
|
239
|
-
|
|
240
|
-
### `scrape`
|
|
241
|
-
|
|
242
|
-
`scrape` creates an AI scraper through:
|
|
243
|
-
|
|
244
|
-
```text
|
|
245
|
-
POST https://api.app.mrscraper.com/api/v1/scrapers-ai
|
|
246
|
-
```
|
|
247
|
-
|
|
248
|
-
Choose an agent based on the extraction shape:
|
|
249
|
-
|
|
250
|
-
| Agent | Designed for | Available inputs |
|
|
251
|
-
| --------- | ------------------------------------- | ------------------------------------------------------------------------- |
|
|
252
|
-
| `general` | Defined fields from a page | `prompt`, `schema_prompt`, `proxy_country` |
|
|
253
|
-
| `listing` | Repeated records across listing pages | `prompt`, `schema_prompt`, `proxy_country`, `max_pages` |
|
|
254
|
-
| `map` | URL discovery across a site | `max_depth`, `max_pages`, `limit`, `include_patterns`, `exclude_patterns` |
|
|
255
|
-
|
|
256
|
-
General extraction:
|
|
257
|
-
|
|
258
|
-
```json
|
|
259
|
-
{
|
|
260
|
-
"url": "https://www.scrapethissite.com/pages/simple/",
|
|
261
|
-
"agent": "general",
|
|
262
|
-
"mode": "Super",
|
|
263
|
-
"prompt": "Extract Andorra's name, capital, population, and area"
|
|
264
|
-
}
|
|
265
|
-
```
|
|
266
|
-
|
|
267
|
-
Listing extraction:
|
|
268
|
-
|
|
269
|
-
```json
|
|
270
|
-
{
|
|
271
|
-
"url": "https://www.scrapethissite.com/pages/forms/",
|
|
272
|
-
"agent": "listing",
|
|
273
|
-
"prompt": "Extract every hockey team, year, wins, losses, and win percentage",
|
|
274
|
-
"max_pages": 5
|
|
275
|
-
}
|
|
276
|
-
```
|
|
277
|
-
|
|
278
|
-
Site map:
|
|
279
|
-
|
|
280
|
-
```json
|
|
281
|
-
{
|
|
282
|
-
"url": "https://www.scrapethissite.com/",
|
|
283
|
-
"agent": "map",
|
|
284
|
-
"max_depth": 2,
|
|
285
|
-
"max_pages": 50,
|
|
286
|
-
"limit": 1000,
|
|
287
|
-
"include_patterns": "/pages/",
|
|
288
|
-
"exclude_patterns": "/login/"
|
|
289
|
-
}
|
|
290
|
-
```
|
|
291
|
-
|
|
292
|
-
Best-effort schema guidance:
|
|
293
|
-
|
|
294
|
-
```json
|
|
295
|
-
{
|
|
296
|
-
"url": "https://www.scrapethissite.com/pages/simple/",
|
|
297
|
-
"prompt": "Extract Andorra's name and capital",
|
|
298
|
-
"schema_prompt": {
|
|
299
|
-
"type": "object",
|
|
300
|
-
"properties": {
|
|
301
|
-
"name": { "type": "string" },
|
|
302
|
-
"capital": { "type": "string" }
|
|
303
|
-
},
|
|
304
|
-
"required": ["name", "capital"]
|
|
305
|
-
}
|
|
306
|
-
}
|
|
307
|
-
```
|
|
308
|
-
|
|
309
|
-
`schema_prompt` is appended to the natural-language instruction. Treat it as
|
|
310
|
-
shape guidance and validate returned data in the consuming application when
|
|
311
|
-
strict conformance is required.
|
|
312
|
-
|
|
313
|
-
| Input | Required | Default | Request mapping | Purpose |
|
|
314
|
-
| ------------------ | --------------- | --------------- | ---------------------- | ----------------------------------------------------------- |
|
|
315
|
-
| `url` | Yes | - | Body `url` | Target URL for every agent. |
|
|
316
|
-
| `prompt` | General/listing | - | Body `message` | Natural-language extraction instructions. |
|
|
317
|
-
| `schema_prompt` | No | omitted | Appended to `message` | Best-effort JSON Schema shape guidance for general/listing. |
|
|
318
|
-
| `agent` | No | `general` | Body `agent` | Selects `general`, `listing`, or `map`. |
|
|
319
|
-
| `mode` | No | service default | Body `mode` | Selects `Cheap` or `Super` independently of the agent. |
|
|
320
|
-
| `proxy_country` | No | omitted | Body `proxyCountry` | Proxy country for general/listing. |
|
|
321
|
-
| `max_pages` | No | service default | Body `maxPages` | Page bound for listing/map. |
|
|
322
|
-
| `max_depth` | No | service default | Body `maxDepth` | Link-depth bound for map. |
|
|
323
|
-
| `limit` | No | service default | Body `limit` | URL-result bound for map. |
|
|
324
|
-
| `include_patterns` | No | service default | Body `includePatterns` | URL inclusion expression for map. |
|
|
325
|
-
| `exclude_patterns` | No | service default | Body `excludePatterns` | URL exclusion expression for map. |
|
|
326
|
-
|
|
327
|
-
General and listing send `url`, `message`, and `agent`, plus the supplied
|
|
328
|
-
agent inputs. Map sends `url`, `agent`, and only the supplied crawl inputs.
|
|
329
|
-
|
|
330
|
-
#### Reproduce a scrape with `rerun`
|
|
331
|
-
|
|
332
|
-
Every successful `scrape` creates a saved AI scraper configuration by default.
|
|
333
|
-
Its response run object contains `scraperId`. Pass that UUID to `rerun` as
|
|
334
|
-
`scraper_id` to apply the same saved extraction configuration to the original
|
|
335
|
-
URL or another URL without rebuilding the prompt and agent settings:
|
|
336
|
-
|
|
337
|
-
```json
|
|
338
|
-
{
|
|
339
|
-
"target": "https://www.scrapethissite.com/pages/forms/?page_num=2",
|
|
340
|
-
"type": "ai",
|
|
341
|
-
"scraper_id": "scraper-uuid"
|
|
190
|
+
"timeout": 120
|
|
342
191
|
}
|
|
343
192
|
```
|
|
344
193
|
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
`
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
194
|
+
Omit `browser_rendering` or use `null` to let the service try without a browser
|
|
195
|
+
and fall back to browser loading. `true` forces browser loading and `false`
|
|
196
|
+
disables it. A screenshot always forces browser loading, even when
|
|
197
|
+
`browser_rendering=false`. `super` selects real-device routing independently.
|
|
198
|
+
|
|
199
|
+
The JSON body contains `url`, `maxRetries`, and optional `tokenCap`. Page
|
|
200
|
+
options are query parameters. For API keys, the request also includes the
|
|
201
|
+
SDK-compatible `token` query parameter and `Authorization`/`x-api-token`
|
|
202
|
+
headers. OAuth JWTs are sent only through `Authorization`.
|
|
203
|
+
|
|
204
|
+
| Input | Default | API mapping / purpose |
|
|
205
|
+
| ---------------------- | -------- | --------------------------------------------------------------------------------------- |
|
|
206
|
+
| `url` | required | Body `url`; absolute HTTP(S) page URL. |
|
|
207
|
+
| `html` | `true` | Query `html`; request HTML. |
|
|
208
|
+
| `markdown` | `false` | Query `markdown`; request Markdown. |
|
|
209
|
+
| `timeout` | `120` | Query `timeout`; integer from 1 to 300 seconds. Transport allows 30 additional seconds. |
|
|
210
|
+
| `geo_code` | `"US"` | Query `geoCode`; proxy country code. |
|
|
211
|
+
| `proxy_country` | omitted | Query `proxyCountry`; compatibility country field. |
|
|
212
|
+
| `browser_rendering` | omitted | Query `browserRendering`; true/false, or omit/null for automatic fallback. |
|
|
213
|
+
| `super` | `false` | Query `super=true` when enabled. |
|
|
214
|
+
| `use_proxy` | `true` | Query `useProxy=false` when disabled. |
|
|
215
|
+
| `proxy` | omitted | Query `proxy`; custom `username:password@host:port` proxy. |
|
|
216
|
+
| `wait_for_selector` | omitted | Query `waitForSelector`; CSS selector to wait for. |
|
|
217
|
+
| `home_page` | `false` | Query `warmStart=true` when enabled. |
|
|
218
|
+
| `screenshot` | `false` | Query `screenshot=true`; also sets `browserRendering=true`. |
|
|
219
|
+
| `block_resources` | `false` | Query `blockResources=true`; block images, CSS, and fonts. |
|
|
220
|
+
| `bypass_proxy` | `false` | Query `bypassProxy=true`; proxy-bypass/resource-blocking option. |
|
|
221
|
+
| `cookie_jar` | omitted | Query `cookieJar`; cookie configuration string. |
|
|
222
|
+
| `headers` | omitted | Query `headers`; target request headers as an object or JSON string. |
|
|
223
|
+
| `max_retries` | `3` | Body `maxRetries`; nonnegative integer, 0 disables retries. |
|
|
224
|
+
| `token_cap` | omitted | Body `tokenCap`; nonnegative integer retry-token budget. |
|
|
225
|
+
| `stream` | omitted | Query `stream`; response is buffered into the normal MCP envelope. |
|
|
226
|
+
| `discard_token_secret` | `""` | Optional `x-discard-token-secret` header. |
|
|
227
|
+
|
|
228
|
+
### `webSearch`
|
|
355
229
|
|
|
356
230
|
```text
|
|
357
231
|
POST https://sync.scraper.mrscraper.com/api/google/serp/v2/sync
|
|
358
232
|
```
|
|
359
233
|
|
|
360
|
-
|
|
234
|
+
Request parsed Google results:
|
|
361
235
|
|
|
362
236
|
```json
|
|
363
237
|
{
|
|
364
|
-
"
|
|
238
|
+
"url": "MrScraper web scraping",
|
|
239
|
+
"raw": false,
|
|
365
240
|
"region": "id",
|
|
366
241
|
"language": "id",
|
|
367
|
-
"page":
|
|
242
|
+
"page": 1,
|
|
368
243
|
"format": "json"
|
|
369
244
|
}
|
|
370
245
|
```
|
|
371
246
|
|
|
372
|
-
|
|
247
|
+
`raw=true` is the Python SDK-compatible default and requests HTML regardless
|
|
248
|
+
of `format`. Set `raw=false` and `format="json"` for parsed results.
|
|
249
|
+
|
|
250
|
+
A Google search URL supplies the query, locale, and page:
|
|
373
251
|
|
|
374
252
|
```json
|
|
375
253
|
{
|
|
376
|
-
"
|
|
254
|
+
"url": "https://www.google.com/search?q=web+scraping&gl=us&hl=en&start=20",
|
|
255
|
+
"raw": false
|
|
377
256
|
}
|
|
378
257
|
```
|
|
379
258
|
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
take priority over URL-derived values.
|
|
383
|
-
|
|
384
|
-
| Input | Required | Default | Request mapping | Purpose |
|
|
385
|
-
| ---------------- | -------- | -------------------- | ---------------------- | -------------------------------------------- |
|
|
386
|
-
| `query_or_url` | Yes | - | Body `query` | Google query or Google search URL. |
|
|
387
|
-
| `region` | No | URL value or omitted | Body `region` | Result country. |
|
|
388
|
-
| `language` | No | URL value or omitted | Body `language` | Result language. |
|
|
389
|
-
| `page` | No | URL value or omitted | Body `page` | One-based result page. |
|
|
390
|
-
| `format` | No | `json` | Body `format` | Selects parsed JSON or result-page HTML. |
|
|
391
|
-
| `render_js` | No | `false` | Body `renderJs` | Waits for JavaScript-rendered SERP features. |
|
|
392
|
-
| `raw` | No | `false` | Body `format=html` | Compatibility alias for HTML output. |
|
|
393
|
-
| `client_timeout` | No | `120` | Local request deadline | Sets the upstream HTTP timeout in seconds. |
|
|
259
|
+
This derives `query="web scraping"`, `region="us"`, `language="en"`, and
|
|
260
|
+
`page=3`. Explicit options override URL-derived values.
|
|
394
261
|
|
|
395
|
-
|
|
262
|
+
| Input | Default | Request mapping / purpose |
|
|
263
|
+
| ----------- | -------------------- | ----------------------------------------------------------- |
|
|
264
|
+
| `url` | required | Search text or Google URL; normalized to body `query`. |
|
|
265
|
+
| `raw` | `true` | Forces body `format="html"`; false allows `format`. |
|
|
266
|
+
| `format` | `"json"` | Body `format`, either `json` or `html`, when raw is false. |
|
|
267
|
+
| `region` | URL value or omitted | Body `region`; overrides `gl`. |
|
|
268
|
+
| `language` | URL value or omitted | Body `language`; overrides `hl`. |
|
|
269
|
+
| `page` | URL value or omitted | Body `page`; positive one-based integer, overrides `start`. |
|
|
270
|
+
| `render_js` | `false` | Body `renderJs`; render dynamic search features. |
|
|
271
|
+
| `timeout` | `600` | Local HTTP deadline in seconds; omitted from the body. |
|
|
396
272
|
|
|
397
|
-
`status`
|
|
273
|
+
### `status`
|
|
398
274
|
|
|
399
275
|
```text
|
|
400
276
|
GET https://api.app.mrscraper.com/api/v1/subscription-accounts
|
|
401
277
|
```
|
|
402
278
|
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
```json
|
|
410
|
-
{
|
|
411
|
-
"domain": "https://www.scrapethissite.com/pages/",
|
|
412
|
-
"from": "7d",
|
|
413
|
-
"to": "now",
|
|
414
|
-
"action": "fetch",
|
|
415
|
-
"api_token_name": "production"
|
|
416
|
-
}
|
|
417
|
-
```
|
|
418
|
-
|
|
419
|
-
| Input | Required | Default | Purpose |
|
|
420
|
-
| ---------------- | -------- | ------------ | ------------------------------------------------------------------------------------------- |
|
|
421
|
-
| `domain` | No | omitted | Adds request-outcome analytics for a hostname or URL. |
|
|
422
|
-
| `from` | No | `24h` | Range start as ISO 8601, `now`, or a relative duration such as `30m`, `24h`, `7d`, or `2w`. |
|
|
423
|
-
| `to` | No | `now` | Range end using the same date syntax. |
|
|
424
|
-
| `action` | No | empty filter | Filters analytics by exact action. |
|
|
425
|
-
| `api_token_name` | No | empty filter | Filters analytics by API-token name. |
|
|
426
|
-
|
|
427
|
-
Successful output is a normalized account and analytics summary:
|
|
279
|
+
`status` takes no input. It returns the connected account's identity (name,
|
|
280
|
+
email, and verification state), subscription status, token limit, token usage,
|
|
281
|
+
remaining tokens, usage percentage, rate limit, auto-renewal, and renewal date.
|
|
282
|
+
It copies only those fields, so the API key and Stripe identifiers in the raw
|
|
283
|
+
account record never reach the client.
|
|
428
284
|
|
|
429
285
|
```json
|
|
430
286
|
{
|
|
431
287
|
"kind": "mrscraper-cli-status-summary",
|
|
432
|
-
"source_endpoints": ["/subscription-accounts"
|
|
288
|
+
"source_endpoints": ["/subscription-accounts"],
|
|
433
289
|
"status_code": 200,
|
|
434
290
|
"data": {
|
|
435
291
|
"account": {
|
|
436
292
|
"subscription_status": "active",
|
|
437
293
|
"enterprise": false,
|
|
438
|
-
"token_usage":
|
|
439
|
-
"token_limit":
|
|
440
|
-
"token_remaining":
|
|
441
|
-
"usage_percent":
|
|
442
|
-
"rate_limit":
|
|
443
|
-
"rate_ttl":
|
|
294
|
+
"token_usage": 61240,
|
|
295
|
+
"token_limit": 100000,
|
|
296
|
+
"token_remaining": 38760,
|
|
297
|
+
"usage_percent": 61.24,
|
|
298
|
+
"rate_limit": 0,
|
|
299
|
+
"rate_ttl": 0,
|
|
444
300
|
"auto_renew": true,
|
|
445
|
-
"ends_at":
|
|
301
|
+
"ends_at": "2026-12-01T00:00:00.000Z",
|
|
446
302
|
"user": {
|
|
447
303
|
"name": "Ada",
|
|
448
|
-
"email": "ada@example.
|
|
304
|
+
"email": "ada@example.com",
|
|
449
305
|
"verified": true
|
|
450
306
|
}
|
|
451
|
-
},
|
|
452
|
-
"analytics": {
|
|
453
|
-
"domain": "www.scrapethissite.com",
|
|
454
|
-
"from": "2026-08-11 00:00:00 UTC",
|
|
455
|
-
"to": "2026-08-18 00:00:00 UTC"
|
|
456
307
|
}
|
|
457
308
|
}
|
|
458
309
|
}
|
|
459
310
|
```
|
|
460
311
|
|
|
461
|
-
|
|
462
|
-
URL to its hostname, and records the source endpoints used for the response.
|
|
312
|
+
An OAuth connection needs the `account:read` scope for `status`.
|
|
463
313
|
|
|
464
|
-
###
|
|
314
|
+
### Response envelope
|
|
465
315
|
|
|
466
|
-
`
|
|
316
|
+
`scraperAPI` and `webSearch` return `status_code`, `data`, and `headers` in
|
|
317
|
+
`structuredContent` and as JSON text. JSON bodies are parsed; other response
|
|
318
|
+
bodies remain text. `status` returns the summary shown above.
|
|
467
319
|
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
- Leave `bulk` as `false` for one URL. Set it to `true` to apply the same saved
|
|
472
|
-
configuration to a comma- or newline-separated URL list in one bulk request.
|
|
320
|
+
A `scraperAPI` response carries the requested representations in `data`;
|
|
321
|
+
`screenshot` is a base64-encoded JPEG and `x-status-code` is the target page's
|
|
322
|
+
status:
|
|
473
323
|
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
|
|
324
|
+
```json
|
|
325
|
+
{
|
|
326
|
+
"status_code": 200,
|
|
327
|
+
"data": {
|
|
328
|
+
"html": "<!DOCTYPE html>...",
|
|
329
|
+
"markdown": null,
|
|
330
|
+
"screenshot": null,
|
|
331
|
+
"token_usage": 3,
|
|
332
|
+
"runtime": 18.02,
|
|
333
|
+
"retry_count": 0,
|
|
334
|
+
"error": null
|
|
335
|
+
},
|
|
336
|
+
"headers": {
|
|
337
|
+
"content-type": "application/json",
|
|
338
|
+
"x-status-code": "200",
|
|
339
|
+
"token_usage": "3"
|
|
340
|
+
}
|
|
341
|
+
}
|
|
342
|
+
```
|
|
477
343
|
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
and pass it to `result` as `result_id` until the stored result is finished.
|
|
344
|
+
A parsed `webSearch` response (`raw=false`) has organic results in
|
|
345
|
+
`data.data.organic`.
|
|
481
346
|
|
|
482
|
-
`
|
|
347
|
+
Failures include `error` and set MCP `isError=true`. Network failures have
|
|
348
|
+
`status_code=null`. Credential-bearing response headers and JSON metadata
|
|
349
|
+
are filtered; network errors redact signed query credentials.
|
|
483
350
|
|
|
484
|
-
|
|
485
|
-
| ------------- | ---------------------------------- | ------------ | ---------------------------- | ------------------- |
|
|
486
|
-
| Single AI | `POST /scrapers-ai-rerun` | `scraper_id` | One URL | Available |
|
|
487
|
-
| Bulk AI | `POST /scrapers-ai-rerun/bulk` | `id` | Comma/newline-separated URLs | Saved configuration |
|
|
488
|
-
| Single manual | `POST /scrapers-manual-rerun` | `scraper_id` | One URL | Saved configuration |
|
|
489
|
-
| Bulk manual | `POST /scrapers-manual-rerun/bulk` | `id` | Comma/newline-separated URLs | Saved configuration |
|
|
351
|
+
## Manual testing: one tool at a time
|
|
490
352
|
|
|
491
|
-
|
|
353
|
+
[`scripts/test-tools.ts`](scripts/test-tools.ts) starts the compiled local MCP
|
|
354
|
+
server over stdio, verifies that exactly the three tools are exposed, calls one
|
|
355
|
+
tool, prints its response, and exits. No separate server terminal is needed.
|
|
356
|
+
The runner loads `.env`, or reads `MRSCRAPER_API_KEY` / `MRSCRAPER_API_TOKEN`
|
|
357
|
+
from your shell. Each tool call uses the live MrScraper service.
|
|
492
358
|
|
|
493
|
-
|
|
494
|
-
{
|
|
495
|
-
"target": "https://www.scrapethissite.com/pages/forms/",
|
|
496
|
-
"type": "ai",
|
|
497
|
-
"scraper_id": "scraper-uuid",
|
|
498
|
-
"max_depth": 2,
|
|
499
|
-
"max_pages": 50,
|
|
500
|
-
"limit": 1000,
|
|
501
|
-
"include_patterns": "/pages/forms/",
|
|
502
|
-
"exclude_patterns": "/login/",
|
|
503
|
-
"proxy_country": "ID",
|
|
504
|
-
"max_retry": 4,
|
|
505
|
-
"timeout": 120
|
|
506
|
-
}
|
|
507
|
-
```
|
|
359
|
+
Run these commands individually from the repository directory:
|
|
508
360
|
|
|
509
|
-
|
|
361
|
+
1. Install dependencies:
|
|
510
362
|
|
|
511
|
-
```
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
"type": "manual",
|
|
515
|
-
"bulk": true,
|
|
516
|
-
"id": "scraper-uuid"
|
|
517
|
-
}
|
|
518
|
-
```
|
|
363
|
+
```sh
|
|
364
|
+
npm ci
|
|
365
|
+
```
|
|
519
366
|
|
|
520
|
-
|
|
521
|
-
| ------------------ | ----------- | ------- | -------------------------------------------------------------------- |
|
|
522
|
-
| `target` | Yes | - | One URL, or a comma/newline-separated URL string for bulk mode. |
|
|
523
|
-
| `type` | Yes | - | Selects `ai` or `manual`. |
|
|
524
|
-
| `bulk` | No | `false` | Selects a bulk endpoint. |
|
|
525
|
-
| `scraper_id` | Single mode | - | Saved scraper UUID for one URL. |
|
|
526
|
-
| `id` | Bulk mode | - | Saved scraper UUID for the bulk URL list. |
|
|
527
|
-
| `max_depth` | Single AI | omitted | Crawl depth; omission preserves the saved scraper/backend default. |
|
|
528
|
-
| `max_pages` | Single AI | omitted | Page bound; omission preserves the saved scraper/backend default. |
|
|
529
|
-
| `limit` | Single AI | omitted | Result bound; omission preserves the saved scraper/backend default. |
|
|
530
|
-
| `include_patterns` | Single AI | omitted | URL inclusion expression; omission preserves saved/backend defaults. |
|
|
531
|
-
| `exclude_patterns` | Single AI | omitted | URL exclusion expression; omission preserves saved/backend defaults. |
|
|
532
|
-
| `proxy_country` | Single AI | omitted | Proxy country code. |
|
|
533
|
-
| `max_retry` | Single AI | omitted | Retry limit; zero is accepted. |
|
|
534
|
-
| `timeout` | Single AI | omitted | Timeout in seconds, used by listing reruns. |
|
|
535
|
-
|
|
536
|
-
The MCP server sends single-AI controls only when supplied. Manual and bulk
|
|
537
|
-
reruns reject them.
|
|
538
|
-
|
|
539
|
-
Manual reruns carry a compliance acknowledgment in the MCP server
|
|
540
|
-
instructions. MCP clients should present that acknowledgment before executing
|
|
541
|
-
the manual mode.
|
|
542
|
-
|
|
543
|
-
### `results`
|
|
544
|
-
|
|
545
|
-
`results` reads the stored result collection:
|
|
367
|
+
2. Build the current server:
|
|
546
368
|
|
|
547
|
-
```
|
|
548
|
-
|
|
549
|
-
```
|
|
369
|
+
```sh
|
|
370
|
+
npm run build
|
|
371
|
+
```
|
|
550
372
|
|
|
551
|
-
|
|
552
|
-
{
|
|
553
|
-
"sort_field": "updatedAt",
|
|
554
|
-
"sort_order": "desc",
|
|
555
|
-
"page_size": 25,
|
|
556
|
-
"page": 1,
|
|
557
|
-
"search": "scrapethissite.com",
|
|
558
|
-
"scraper_id": "scraper-uuid",
|
|
559
|
-
"status": "Finished",
|
|
560
|
-
"type": "Rerun-AI",
|
|
561
|
-
"url": "https://www.scrapethissite.com/pages/forms/",
|
|
562
|
-
"date_range_column": "updatedAt",
|
|
563
|
-
"start_at": "2026-08-01T00:00:00Z",
|
|
564
|
-
"end_at": "2026-08-18T23:59:59Z"
|
|
565
|
-
}
|
|
566
|
-
```
|
|
373
|
+
3. Set your API key:
|
|
567
374
|
|
|
568
|
-
|
|
569
|
-
|
|
570
|
-
|
|
571
|
-
| `sort_order` | No | `desc` | `sortOrder` | Case-insensitive `asc` or `desc`; sent upstream in uppercase. |
|
|
572
|
-
| `page_size` | No | `10` | `pageSize` | Number of rows per page. |
|
|
573
|
-
| `page` | No | `1` | `page` | One-based page index. |
|
|
574
|
-
| `search` | No | omitted | `search` | Free-text result filter. |
|
|
575
|
-
| `scraper_id` | No | omitted | `filters[scraperId]` | Exact saved scraper UUID filter. |
|
|
576
|
-
| `status` | No | omitted | `filters[status]` | Exact Draft, Finished, Running, Failed, or Cancelled filter. |
|
|
577
|
-
| `type` | No | omitted | `filters[type]` | Exact result type filter, such as AI or Rerun-AI. |
|
|
578
|
-
| `url` | No | omitted | `filters[url]` | Exact stored target URL filter. |
|
|
579
|
-
| `date_range_column` | No | omitted | `dateRangeColumn` | Column used by the date range. |
|
|
580
|
-
| `start_at` | No | omitted | `startAt` | Inclusive range start. |
|
|
581
|
-
| `end_at` | No | omitted | `endAt` | Inclusive range end. |
|
|
582
|
-
|
|
583
|
-
### `result`
|
|
584
|
-
|
|
585
|
-
`result` reads one stored result:
|
|
375
|
+
```sh
|
|
376
|
+
export MRSCRAPER_API_KEY="YOUR_MRSCRAPER_API_KEY"
|
|
377
|
+
```
|
|
586
378
|
|
|
587
|
-
|
|
588
|
-
|
|
379
|
+
4. Test `scraperAPI`:
|
|
380
|
+
|
|
381
|
+
```sh
|
|
382
|
+
npm run test:manual -- scraperAPI --url "https://www.scrapethissite.com/pages/simple/"
|
|
383
|
+
```
|
|
384
|
+
|
|
385
|
+
5. Test `webSearch`:
|
|
386
|
+
|
|
387
|
+
```sh
|
|
388
|
+
npm run test:manual -- webSearch --query "MrScraper web scraping"
|
|
389
|
+
```
|
|
390
|
+
|
|
391
|
+
6. Test `status`:
|
|
392
|
+
|
|
393
|
+
```sh
|
|
394
|
+
npm run test:manual -- status
|
|
395
|
+
```
|
|
396
|
+
|
|
397
|
+
The webSearch sample explicitly requests `raw=false`, `format="json"`, US
|
|
398
|
+
results, and English. Each command prints `Tools: scraperAPI, webSearch, status`
|
|
399
|
+
and `PASS <tool>` on success. Exit codes are 0 for success, 1 for a tool or
|
|
400
|
+
connection failure, and 2 for invalid command arguments.
|
|
401
|
+
|
|
402
|
+
Pass a complete JSON object to test any supported options:
|
|
403
|
+
|
|
404
|
+
```sh
|
|
405
|
+
npm run test:manual -- scraperAPI --args '{"url":"https://example.com","markdown":true,"screenshot":true,"super":true}'
|
|
589
406
|
```
|
|
590
407
|
|
|
591
|
-
```
|
|
592
|
-
{
|
|
593
|
-
"result_id": "result-uuid",
|
|
594
|
-
"include_html": false
|
|
595
|
-
}
|
|
408
|
+
```sh
|
|
409
|
+
npm run test:manual -- webSearch --args '{"url":"web scraping","raw":false,"region":"id","language":"id","page":1,"render_js":true}'
|
|
596
410
|
```
|
|
597
411
|
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
|
|
601
|
-
| `include_html` | No | `true` | Includes stored HTML; set false for smaller polling or data-only output. |
|
|
412
|
+
To test an already running HTTP MCP server, append
|
|
413
|
+
`--target http://127.0.0.1:8000/mcp`. Rebuild after source changes when testing
|
|
414
|
+
local stdio. `npm run test:manual -- --help` lists runner options.
|
|
602
415
|
|
|
603
416
|
## Run the server locally
|
|
604
417
|
|
|
@@ -675,7 +488,7 @@ server environment credentials are never used on the HTTP transport.
|
|
|
675
488
|
|
|
676
489
|
The server filters credential-bearing response headers. Parsed JSON credential
|
|
677
490
|
metadata and credentials embedded in generated curl commands are redacted.
|
|
678
|
-
|
|
491
|
+
Page content remains available in `data`.
|
|
679
492
|
|
|
680
493
|
Browser-origin requests are accepted from trusted local origins, from
|
|
681
494
|
`claude.ai` and `chatgpt.com`, and from exact origins configured through
|
|
@@ -685,10 +498,10 @@ checks, since any client must be able to read them before it has credentials.
|
|
|
685
498
|
|
|
686
499
|
## Interactive widgets
|
|
687
500
|
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
the JSON
|
|
501
|
+
`webSearch` carries a self-contained Google results widget for parsed JSON
|
|
502
|
+
results. HTML responses show a prompt to request parsed results. `status`
|
|
503
|
+
carries an account widget with the plan, token usage, and a usage meter. Hosts
|
|
504
|
+
without MCP Apps support display the normal JSON tool response.
|
|
692
505
|
|
|
693
506
|
Each widget is a self-contained document with its script and styles inlined,
|
|
694
507
|
because strict host CSP blocks anything external. Each one declares an empty
|
|
@@ -714,7 +527,7 @@ deployment and keep bearer authentication enabled.
|
|
|
714
527
|
| `HOST` | `127.0.0.1` | HTTP bind address; the Docker image uses `0.0.0.0`. |
|
|
715
528
|
| `PORT` | `8000` | HTTP listen port. |
|
|
716
529
|
| `MRSCRAPER_API_KEY` | - | Primary stdio credential. |
|
|
717
|
-
| `MRSCRAPER_API_TOKEN` | - |
|
|
530
|
+
| `MRSCRAPER_API_TOKEN` | - | Alternative stdio credential (also used by the Python SDK). |
|
|
718
531
|
| `MRSCRAPER_HTTP_AUTH` | `1` | Enables HTTP bearer verification. |
|
|
719
532
|
| `MRSCRAPER_OAUTH` | `1` | Accepts OAuth 2.1 access tokens alongside API keys. |
|
|
720
533
|
| `MRSCRAPER_MCP_PUBLIC_URL` | `https://mcp.mrscraper.com` | Public origin; `<origin>/mcp` is the required token audience. |
|
|
@@ -722,8 +535,8 @@ deployment and keep bearer authentication enabled.
|
|
|
722
535
|
| `MRSCRAPER_OAUTH_JWKS_URL` | `<issuer>/.well-known/jwks.json` | Key set used to verify access tokens. |
|
|
723
536
|
| `MRSCRAPER_ALLOWED_ORIGINS` | - | Comma-separated browser origins allowed to call the HTTP server. |
|
|
724
537
|
| `MRSCRAPER_API_BASE_URL` | MrScraper platform API | Platform endpoint override for development and testing. |
|
|
725
|
-
| `MRSCRAPER_FETCH_BASE_URL` | MrScraper Web Unblocker |
|
|
726
|
-
| `MRSCRAPER_SYNC_BASE_URL` | MrScraper synchronous scraper API |
|
|
538
|
+
| `MRSCRAPER_FETCH_BASE_URL` | MrScraper Web Unblocker | Scraper API endpoint override. |
|
|
539
|
+
| `MRSCRAPER_SYNC_BASE_URL` | MrScraper synchronous scraper API | Web search endpoint override. |
|
|
727
540
|
| `MRSCRAPER_LOG_HTTP_PAYLOAD` | off | Enables trusted-environment request-body diagnostics. |
|
|
728
541
|
| `MRSCRAPER_LOG_HTTP_PAYLOAD_MAX` | `8192` | Maximum diagnostic payload length. |
|
|
729
542
|
|
|
@@ -741,11 +554,11 @@ deployment and keep bearer authentication enabled.
|
|
|
741
554
|
- **403 Forbidden Origin:** Add the exact browser origin to
|
|
742
555
|
`MRSCRAPER_ALLOWED_ORIGINS`.
|
|
743
556
|
- **Tool input error:** Compare the call with the tool's parameter table and
|
|
744
|
-
the
|
|
745
|
-
- **Upstream timeout:** Increase `
|
|
557
|
+
the documented SDK-compatible defaults.
|
|
558
|
+
- **Upstream timeout:** Increase `scraperAPI.timeout` (up to 300 seconds) or `webSearch.timeout`
|
|
746
559
|
when the target requires a longer request window.
|
|
747
560
|
- **New tools are missing:** Start a new client session and inspect
|
|
748
|
-
`tools/list` for
|
|
561
|
+
`tools/list` for exactly `scraperAPI` and `webSearch`.
|
|
749
562
|
|
|
750
563
|
## Development
|
|
751
564
|
|
|
@@ -766,8 +579,21 @@ npm run test:mcp -- \
|
|
|
766
579
|
--token "$MRSCRAPER_API_KEY"
|
|
767
580
|
```
|
|
768
581
|
|
|
769
|
-
`npm run test:e2e`
|
|
770
|
-
service
|
|
582
|
+
`npm run test:e2e` runs both new tools sequentially against the live MrScraper
|
|
583
|
+
service using the manual runner. Use `-- --url URL --query QUERY` to customize
|
|
584
|
+
the targets, or `-- --target http://127.0.0.1:8000/mcp` to use HTTP.
|
|
585
|
+
|
|
586
|
+
## MCP Registry
|
|
587
|
+
|
|
588
|
+
The hosted server's MCP Registry manifest is defined in
|
|
589
|
+
[`server.json`](server.json). It describes the Streamable HTTP endpoint and
|
|
590
|
+
links to this repository and the setup documentation.
|
|
591
|
+
|
|
592
|
+
See the [official registry publishing guide](https://modelcontextprotocol.io/registry/github-actions)
|
|
593
|
+
for GitHub Actions authentication and publication. Registry versions are
|
|
594
|
+
immutable, so use a new version in `server.json` when publishing changed
|
|
595
|
+
metadata. Publishing this manifest registers the hosted server; it does not
|
|
596
|
+
publish a new npm package.
|
|
771
597
|
|
|
772
598
|
## License
|
|
773
599
|
|