@spicrawl/mcp 0.1.4 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -0
- package/README.md +71 -12
- package/dist/card.d.ts +63 -0
- package/dist/card.js +50 -0
- package/dist/card.js.map +1 -0
- package/dist/http.d.ts +5 -2
- package/dist/http.js +36 -9
- package/dist/http.js.map +1 -1
- package/dist/httputil.d.ts +7 -0
- package/dist/httputil.js +11 -0
- package/dist/httputil.js.map +1 -1
- package/dist/tools/batch.js +0 -9
- package/dist/tools/batch.js.map +1 -1
- package/dist/tools/browser.js +8 -18
- package/dist/tools/browser.js.map +1 -1
- package/dist/tools/scrape.js +5 -10
- package/dist/tools/scrape.js.map +1 -1
- package/dist/tools/sessions.js +9 -8
- package/dist/tools/sessions.js.map +1 -1
- package/package.json +8 -3
package/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,27 @@
|
|
|
3
3
|
All notable changes to `@spicrawl/mcp` are documented here. This project follows
|
|
4
4
|
[Semantic Versioning](https://semver.org/).
|
|
5
5
|
|
|
6
|
+
## 0.1.6 — 2026-10-13
|
|
7
|
+
|
|
8
|
+
- `/mcp` now accepts the API key in the forms a gateway such as Smithery can send, because Smithery maps a connection parameter to a header but sends the raw value with no `Bearer ` prefix, and clients often cannot name the source `authorization`. In order: `Authorization: Bearer <key>` (unchanged), a raw `Authorization: <key>` when the whole value is a Spicrawl key, `x-api-key: <key>`, then `api-key: <key>`. The first form that holds a well-formed key wins.
|
|
9
|
+
- A key is read from headers only, never from a query parameter, because URLs end up in access logs. A malformed value in any form is still `401`.
|
|
10
|
+
- The `401` message mentions `x-api-key` as an alternative; its `WWW-Authenticate` challenge is unchanged. The session pinning, the key check at `initialize` and the server card are unchanged.
|
|
11
|
+
- `/chatgpt/mcp` is unchanged: it takes OAuth access tokens only and ignores these headers.
|
|
12
|
+
- README: Smithery configuration guidance (header `x-api-key`, or `Authorization` with the raw key).
|
|
13
|
+
|
|
14
|
+
## 0.1.5 — 2026-10-12
|
|
15
|
+
|
|
16
|
+
- A static MCP server card at `GET /.well-known/mcp/server-card.json` on the HTTP server, so directories such as Smithery can list the server without a key (`/mcp` answers 401). It carries `serverInfo`, the streamable-http transport, bearer auth and the tool list, generated from the real tool definitions. The tool definitions name no execution engine, so the card is not filtered: it is the `tools/list` of `/mcp` with `SPICRAWL_MCP_HIDE_UNAVAILABLE` on, including the session `engine` tiers (`fetch`, `js`, `browser`). A test fails if the card or any published file ever names one. Public and cacheable for an hour. `/mcp` is unchanged.
|
|
17
|
+
|
|
18
|
+
Internal engine names no longer appear in the tool definitions, the README or the npm package. Customers see the public rendering tiers (`fetch`, `js`, `browser`) and nothing else.
|
|
19
|
+
|
|
20
|
+
- **Breaking:** `engine` and its display-mode flag are no longer arguments of `spicrawl_scrape`, and `engine` is no longer an argument of `spicrawl_batch_submit` and `spicrawl_batch_add_items`. The API ignored them (it answers with an `ENGINE_IGNORED` warning), because it chooses the rendering itself. A call that still sends one is refused with an "unknown argument" error; drop it.
|
|
21
|
+
- **Breaking:** `engine` on `spicrawl_session_create` and `spicrawl_session_list` now takes the public tiers `fetch`, `js` or `browser`. On create it is optional and fixed for the session's life; leave it out and one is chosen for you.
|
|
22
|
+
- **Breaking:** `spicrawl_browser_connect_url` takes `engine` of `browser` (default) or `js`, and no longer takes the display-mode flag: the API chooses it. Its result no longer carries ready-made connect snippets for particular client libraries; use the returned `url` as the browser WebSocket endpoint of any CDP client. The tool is still coming soon.
|
|
23
|
+
- Descriptions no longer name an internal engine or a browser library. `format: "pdf"` is described as printed in a full browser, routed there for you (it needs no `render` or `engine`).
|
|
24
|
+
- README, the `package.json` keywords and the examples follow.
|
|
25
|
+
- A new test fails if the whole `tools/list` output (names, titles, descriptions, schemas) of `/mcp`, with and without `SPICRAWL_MCP_HIDE_UNAVAILABLE`, or of `/chatgpt/mcp`, names an internal engine or browser tooling.
|
|
26
|
+
|
|
6
27
|
## 0.1.4 — 2026-10-05
|
|
7
28
|
|
|
8
29
|
The repository moved to https://github.com/OfficialSpicrawl/mcp, and `mcpName` is now `io.github.OfficialSpicrawl/mcp`. No code or tool changes.
|
package/README.md
CHANGED
|
@@ -40,12 +40,12 @@ Both transports serve the same 25 `spicrawl_*` tools, and every call runs under
|
|
|
40
40
|
## What it does
|
|
41
41
|
|
|
42
42
|
- **Scrape any URL to Markdown**, text, HTML, a JSON envelope or a printed PDF with one tool, `spicrawl_scrape`. Markdown is the default and strips the page to its main content.
|
|
43
|
-
- **Render JavaScript** with `render: true` (
|
|
43
|
+
- **Render JavaScript** with `render: true` (3 credits). A plain fetch costs 1 credit, screenshots, PDFs and `actions` use a full browser (8 credits), and `mode: "auto"` escalates from fetch to browser and bills only the rung that worked ([price table](https://docs.spicrawl.com/credits)). You never pick the rendering yourself; Spicrawl chooses the cheapest that can do what the request asks.
|
|
44
44
|
- **Extract structured data** with a CSS/XPath selector map (`extract`) or from the JSON-LD, OpenGraph, Twitter Card and microdata a page publishes (`autoparse`).
|
|
45
45
|
- **Drive the page** before capture with 8 typed browser actions: `click`, `fill`, `wait_for`, `wait_for_navigation`, `scroll` (including infinite scroll), `select`, `evaluate` and `screenshot`.
|
|
46
46
|
- **Return screenshots and PDFs the model can see**: screenshots arrive as MCP image blocks (up to 5 MB of base64 each) and printed pages as `application/pdf` resource blocks (up to 10 MB).
|
|
47
47
|
- **Batch scrape thousands of URLs** as one async job, up to 10,000 items per call, with 9 tools to submit, poll, page through results, retry, cancel and keep a job open for crawling.
|
|
48
|
-
- **Keep a logged-in browser** with persistent sessions: cookies, storage and a
|
|
48
|
+
- **Keep a logged-in browser** with persistent sessions: cookies, storage and a fixed rendering tier reused across scrapes.
|
|
49
49
|
- **Answer "what did that cost?"** from usage and request-log tools, and **read the Spicrawl docs** from inside the agent.
|
|
50
50
|
|
|
51
51
|
## Which tool to use
|
|
@@ -67,7 +67,7 @@ Both transports serve the same 25 `spicrawl_*` tools, and every call runs under
|
|
|
67
67
|
|
|
68
68
|
| Tool | What it does | Key parameters |
|
|
69
69
|
|---|---|---|
|
|
70
|
-
| `spicrawl_scrape` | Retrieve one URL as `markdown` (default), `text`, `html`, `json` or `pdf`, with optional rendering, extraction, links, screenshots and browser actions. | `url`, `format`, `render`, `
|
|
70
|
+
| `spicrawl_scrape` | Retrieve one URL as `markdown` (default), `text`, `html`, `json` or `pdf`, with optional rendering, extraction, links, screenshots and browser actions. | `url`, `format`, `render`, `mode`, `main_content_only`, `include_tags`, `exclude_tags`, `extract`, `autoparse`, `links`, `screenshot`, `actions`, `wait_for`, `session_id`, `proxy`, `cache`, `max_cost` |
|
|
71
71
|
|
|
72
72
|
### Batch scrape many URLs
|
|
73
73
|
|
|
@@ -87,9 +87,9 @@ Both transports serve the same 25 `spicrawl_*` tools, and every call runs under
|
|
|
87
87
|
|
|
88
88
|
| Tool | What it does | Key parameters |
|
|
89
89
|
|---|---|---|
|
|
90
|
-
| `spicrawl_session_create` | Create a persistent browser identity (cookies, storage,
|
|
90
|
+
| `spicrawl_session_create` | Create a persistent browser identity (cookies, storage, a fixed rendering tier). | `engine`, `ttl_seconds`, `session_context` |
|
|
91
91
|
| `spicrawl_session_list` | List sessions, newest first. | `status`, `engine`, `limit`, `cursor` |
|
|
92
|
-
| `spicrawl_session_get` | One session's status,
|
|
92
|
+
| `spicrawl_session_get` | One session's status, rendering tier, exit, usage and expiry. | `session_id` |
|
|
93
93
|
| `spicrawl_session_context` | Dump a live session's cookies and storage (contains secrets). | `session_id` |
|
|
94
94
|
| `spicrawl_session_release` | End a session; its record stays as `released`. | `session_id` |
|
|
95
95
|
| `spicrawl_session_delete` | Permanently delete a session and its state. | `session_id` |
|
|
@@ -101,7 +101,7 @@ Both transports serve the same 25 `spicrawl_*` tools, and every call runs under
|
|
|
101
101
|
| `spicrawl_usage` | Billing-grade usage over a date window (`read` scope). | `from`, `to`, `group_by`, `metrics` |
|
|
102
102
|
| `spicrawl_usage_summary` | Current-period totals (`read` scope). | none |
|
|
103
103
|
| `spicrawl_usage_reconciliation` | One day's drift between the request log and the billed rollup (`read` scope). | `day` |
|
|
104
|
-
| `spicrawl_requests_list` | Recent API calls with status, error code,
|
|
104
|
+
| `spicrawl_requests_list` | Recent API calls with status, error code, rendering tier, latency and credits. | `only_errors`, `status`, `limit`, `all_projects` |
|
|
105
105
|
| `spicrawl_request_get` | The full log record of one call. | `id` |
|
|
106
106
|
|
|
107
107
|
### Docs
|
|
@@ -114,7 +114,7 @@ Both transports serve the same 25 `spicrawl_*` tools, and every call runs under
|
|
|
114
114
|
|
|
115
115
|
### Coming soon
|
|
116
116
|
|
|
117
|
-
`spicrawl_browser_connect_url` will mint a single-use CDP WebSocket URL for driving a Spicrawl-hosted browser from
|
|
117
|
+
`spicrawl_browser_connect_url` will mint a single-use CDP WebSocket URL for driving a Spicrawl-hosted browser from any CDP client; remote browsers are not available yet. The schemas also accept `ai_extract`, `stealth`, `extract_preset`, `premium_proxy`, `proxy_country` and `sticky_key` ahead of launch. Agents should not send them yet. To list only what works today, set `SPICRAWL_MCP_HIDE_UNAVAILABLE=1` (see [Environment variables](https://github.com/OfficialSpicrawl/mcp#environment-variables)).
|
|
118
118
|
|
|
119
119
|
## Example prompts
|
|
120
120
|
|
|
@@ -311,6 +311,23 @@ More: [docs.spicrawl.com/agents/codex](https://docs.spicrawl.com/agents/codex).
|
|
|
311
311
|
|
|
312
312
|
</details>
|
|
313
313
|
|
|
314
|
+
<details>
|
|
315
|
+
<summary><b>Smithery and other gateways</b></summary>
|
|
316
|
+
|
|
317
|
+
A gateway that passes a connection parameter through as a header cannot always send `Authorization: Bearer <key>`: Smithery maps a parameter to a header but sends the raw value, with no `Bearer ` prefix, and a client often cannot name its source `authorization`. `/mcp` takes the key in any of these headers:
|
|
318
|
+
|
|
319
|
+
| Header | Value |
|
|
320
|
+
|---|---|
|
|
321
|
+
| `x-api-key` | `spicrawl_live_...` (recommended for a gateway) |
|
|
322
|
+
| `api-key` | `spicrawl_live_...` |
|
|
323
|
+
| `Authorization` | `spicrawl_live_...` (the raw key, no `Bearer `), or `Bearer spicrawl_live_...` |
|
|
324
|
+
|
|
325
|
+
In Smithery, make the API key a connection parameter, map it to the `x-api-key` header, and mark it secret. Or map it to `Authorization` and send the raw key; the server accepts that when the whole value is a Spicrawl key.
|
|
326
|
+
|
|
327
|
+
If more than one is sent, the first that holds a well-formed key wins, in the order `Authorization: Bearer`, raw `Authorization`, `x-api-key`, `api-key`. A key in the query string is **not** read, because URLs end up in access logs. `/chatgpt/mcp` takes OAuth access tokens only and accepts none of these.
|
|
328
|
+
|
|
329
|
+
</details>
|
|
330
|
+
|
|
314
331
|
<details>
|
|
315
332
|
<summary><b>Any other MCP client</b></summary>
|
|
316
333
|
|
|
@@ -318,7 +335,7 @@ More: [docs.spicrawl.com/agents/codex](https://docs.spicrawl.com/agents/codex).
|
|
|
318
335
|
|---|---|
|
|
319
336
|
| Transport | Streamable HTTP |
|
|
320
337
|
| URL | `https://mcp.spicrawl.com/mcp` |
|
|
321
|
-
| Header | `Authorization: Bearer <your key>` |
|
|
338
|
+
| Header | `Authorization: Bearer <your key>` (or `x-api-key: <your key>`) |
|
|
322
339
|
|
|
323
340
|
A stdio-only client runs `npx -y @spicrawl/mcp` with `SPICRAWL_API_KEY` in its environment. The server accepts API keys only, not OAuth. Setup pages for more clients (OpenCode, Zed, JetBrains, Cline, Goose and others): [docs.spicrawl.com/agents/mcp](https://docs.spicrawl.com/agents/mcp).
|
|
324
341
|
|
|
@@ -335,7 +352,7 @@ A stdio-only client runs `npx -y @spicrawl/mcp` with `SPICRAWL_API_KEY` in its e
|
|
|
335
352
|
| `SPICRAWL_DOCS_HOST` | none | both | Legacy: an origin whose docs are at `/docs`. Read only when `SPICRAWL_DOCS_URL` is unset. |
|
|
336
353
|
| `SPICRAWL_MCP_ADDR` | `127.0.0.1:8090` | HTTP | Listen address, `host:port`. |
|
|
337
354
|
| `SPICRAWL_MCP_SESSION_IDLE_MS` | `1800000` (30 min) | HTTP | Idle MCP sessions are dropped after this long. |
|
|
338
|
-
| `SPICRAWL_MCP_HIDE_UNAVAILABLE` | off | both | `1`, `true`, `yes` or `on` leaves out what is announced but not available yet: `spicrawl_browser_connect_url` is not registered (24 tools) and every argument marked "Coming soon" is dropped from the schemas (`ai_extract`, `stealth`, `extract_preset`, `premium_proxy`, `proxy_country`, `sticky_key`, and the session `rotate_ip`, `region_pool` and `fingerprint`).
|
|
355
|
+
| `SPICRAWL_MCP_HIDE_UNAVAILABLE` | off | both | `1`, `true`, `yes` or `on` leaves out what is announced but not available yet: `spicrawl_browser_connect_url` is not registered (24 tools) and every argument marked "Coming soon" is dropped from the schemas (`ai_extract`, `stealth`, `extract_preset`, `premium_proxy`, `proxy_country`, `sticky_key`, and the session `rotate_ip`, `region_pool` and `fingerprint`). Read when the server starts a session, so restart the process after changing it. |
|
|
339
356
|
| `SPICRAWL_CHATGPT_ENABLED` | off | HTTP | `1`, `true`, `yes` or `on` turns on the [ChatGPT endpoint](#chatgpt-endpoint-oauth). Off, `/chatgpt/mcp`, its metadata and the OpenAI challenge all answer `404`. |
|
|
340
357
|
| `SPICRAWL_CHATGPT_RESOURCE` | `https://mcp.spicrawl.com/chatgpt/mcp` | HTTP | The OAuth resource identifier. Tokens must be issued for exactly this `aud`; the metadata is published at `/.well-known/oauth-protected-resource` + its path. |
|
|
341
358
|
| `SPICRAWL_OAUTH_ISSUER` | `https://app.spicrawl.com` | HTTP | The authorization server, listed in the metadata. Tokens are introspected at `<issuer>/api/oauth/introspect`. |
|
|
@@ -358,7 +375,8 @@ npx -y @spicrawl/mcp --http
|
|
|
358
375
|
|
|
359
376
|
- `SPICRAWL_BASE_URL` **defaults to `http://127.0.0.1:8080`** in HTTP mode (a Spicrawl API on the same host), not to the public API. Set it to `https://api.spicrawl.com` to front the public API.
|
|
360
377
|
- MCP is served at `POST`/`GET`/`DELETE /mcp`; liveness at `GET /healthz` (no auth).
|
|
361
|
-
-
|
|
378
|
+
- `GET /.well-known/mcp/server-card.json` (no auth, `Cache-Control: public, max-age=3600`) is a static server card for directories such as Smithery that cannot scan `/mcp` without a key: `serverInfo`, the transport, bearer auth and the tool list. The tools are read from the same definitions as `tools/list` (as with `SPICRAWL_MCP_HIDE_UNAVAILABLE`), so it matches what an authenticated client sees.
|
|
379
|
+
- Each caller sends their own key as `Authorization: Bearer spicrawl_live_...`, or, for a gateway that cannot send the `Bearer ` prefix, as a raw `Authorization: spicrawl_live_...`, `x-api-key` or `api-key` header (see "Smithery and other gateways" above; never a query parameter). The key is checked against the API before a session opens (`401` if refused, `503` with `Retry-After` if the API cannot be reached), and a session is pinned to the key that opened it.
|
|
362
380
|
- The server speaks plain HTTP. Put it behind TLS before sending keys across a network.
|
|
363
381
|
|
|
364
382
|
### ChatGPT endpoint (OAuth)
|
|
@@ -408,7 +426,7 @@ Yes: `https://mcp.spicrawl.com/mcp`, MCP Streamable HTTP, authenticated with `Au
|
|
|
408
426
|
Spicrawl bills credits per successful request, and during the beta every organization gets one plan of 1,000 credits per month. Failed requests cost 0 credits; a cache hit is billed at the same price as the fetch that stored it. See [Credits](https://docs.spicrawl.com/credits).
|
|
409
427
|
|
|
410
428
|
### Does it render JavaScript?
|
|
411
|
-
Yes. Set `render: true` on `spicrawl_scrape` to run the page in a browser, or `mode: "auto"` to let Spicrawl escalate from a plain fetch only when needed.
|
|
429
|
+
Yes. Set `render: true` on `spicrawl_scrape` to run the page in a browser, or `mode: "auto"` to let Spicrawl escalate from a plain fetch only when needed. There is no argument to choose the rendering on `spicrawl_scrape` or the batch tools: Spicrawl picks it for each request and bills the tier that served it (`fetch`, `js` or `browser`, reported as `engine` in the JSON result).
|
|
412
430
|
|
|
413
431
|
### Does it work with Claude Code, Cursor and VS Code?
|
|
414
432
|
Yes, and with Claude Desktop, Windsurf, Gemini CLI, Codex and any client that speaks MCP Streamable HTTP with custom headers or launches stdio servers.
|
|
@@ -426,7 +444,7 @@ Use the MCP server when an AI agent should call Spicrawl as tools. Use [`@spicra
|
|
|
426
444
|
|
|
427
445
|
| Symptom | Fix |
|
|
428
446
|
|---|---|
|
|
429
|
-
| HTTP `401` or "Unauthorized: send your Spicrawl API key" | The header is missing, or the key is unknown, revoked or expired. Check that the variable is set in the environment the client started from. |
|
|
447
|
+
| HTTP `401` or "Unauthorized: send your Spicrawl API key" | The header is missing, or the key is malformed, unknown, revoked or expired. Check that the variable is set in the environment the client started from. A gateway that cannot send `Bearer ` can use an `x-api-key` header; a key in the URL is never read. |
|
|
430
448
|
| The header contains a literal `${SPICRAWL_API_KEY}` | The client did not expand it. Use `${VAR}` in Claude Code, `${env:VAR}` in Cursor, `${input:id}` in VS Code, `bearer_token_env_var` in Codex. |
|
|
431
449
|
| HTTP `503` with `Retry-After` on connect | The server could not reach the API to check your key. Retry; your key is fine. |
|
|
432
450
|
| `ERR::AUTH::INSUFFICIENT_SCOPE` from a `spicrawl_usage*` tool | Grant the key the `read` scope in the dashboard. |
|
|
@@ -451,6 +469,47 @@ Use the MCP server when an AI agent should call Spicrawl as tools. Use [`@spicra
|
|
|
451
469
|
- Issues: [github.com/OfficialSpicrawl/mcp/issues](https://github.com/OfficialSpicrawl/mcp/issues)
|
|
452
470
|
- Security: [SECURITY.md](https://github.com/OfficialSpicrawl/mcp/blob/main/SECURITY.md). Never commit an API key; revoke a leaked one at [app.spicrawl.com](https://app.spicrawl.com).
|
|
453
471
|
|
|
472
|
+
## Publishing to the MCP Registry
|
|
473
|
+
|
|
474
|
+
[`server.json`](./server.json) lists both the npm/stdio package and the hosted
|
|
475
|
+
Streamable HTTP endpoint under `io.github.OfficialSpicrawl/mcp`. API keys are
|
|
476
|
+
requested from the person installing the server; no key belongs in the manifest.
|
|
477
|
+
|
|
478
|
+
The existing `release` workflow publishes to the MCP Registry **after** its npm
|
|
479
|
+
job succeeds, on pushes to `main` or manual runs from `main`. Registry authentication
|
|
480
|
+
uses GitHub OIDC (`id-token: write`), so it needs no new secret, device login or DNS
|
|
481
|
+
record. The npm release keeps using the existing `NPM_TOKEN` secret.
|
|
482
|
+
|
|
483
|
+
For a release, run `npm version --no-git-tag-version X.Y.Z`, update the changelog,
|
|
484
|
+
and submit the changes to `main`. The npm version hook synchronizes both versions
|
|
485
|
+
in `server.json`. If you edit `package.json` by hand, run `npm run registry:sync`.
|
|
486
|
+
CI checks the manifest against its official `$schema` with Ajv and rejects name
|
|
487
|
+
or version drift before npm publication. To run that check locally:
|
|
488
|
+
|
|
489
|
+
```bash
|
|
490
|
+
npm ci
|
|
491
|
+
npm run registry:validate
|
|
492
|
+
# Offline: pass a downloaded copy of the exact server.json $schema URL.
|
|
493
|
+
npm run registry:validate -- /path/to/server.schema.json
|
|
494
|
+
```
|
|
495
|
+
|
|
496
|
+
The registry job checks that the published npm package has the matching `mcpName`
|
|
497
|
+
before publishing. An existing registry version is skipped; lookup failures stop
|
|
498
|
+
the job. To retry a registry failure or list the already-published current npm
|
|
499
|
+
version, run the `release` workflow on `main` again. Use a new version for metadata
|
|
500
|
+
changes to an existing listing. The publisher binary is pinned and its download
|
|
501
|
+
is checked against the release checksums; update its version in `release.yml`
|
|
502
|
+
when upgrading the CLI.
|
|
503
|
+
|
|
504
|
+
Verify publication at
|
|
505
|
+
<https://registry.modelcontextprotocol.io/v0.1/servers?search=io.github.OfficialSpicrawl/mcp>.
|
|
506
|
+
The workflow must be merged into `OfficialSpicrawl/mcp` before OIDC can publish
|
|
507
|
+
that namespace. Fork pull requests validate and test, but never run the registry job.
|
|
508
|
+
|
|
509
|
+
Official references: [publishing quickstart](https://github.com/modelcontextprotocol/registry/blob/main/docs/modelcontextprotocol-io/quickstart.mdx),
|
|
510
|
+
[GitHub Actions](https://github.com/modelcontextprotocol/registry/blob/main/docs/modelcontextprotocol-io/github-actions.mdx),
|
|
511
|
+
and [remote servers](https://github.com/modelcontextprotocol/registry/blob/main/docs/modelcontextprotocol-io/remote-servers.mdx).
|
|
512
|
+
|
|
454
513
|
## License
|
|
455
514
|
|
|
456
515
|
[Apache-2.0](https://github.com/OfficialSpicrawl/mcp/blob/main/LICENSE)
|
package/dist/card.d.ts
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The static MCP server card, served unauthenticated at
|
|
3
|
+
* GET /.well-known/mcp/server-card.json (src/http.ts). Directories such as
|
|
4
|
+
* Smithery cannot scan /mcp (it answers 401 without a key) and read this instead.
|
|
5
|
+
*
|
|
6
|
+
* The tool list is not written by hand: it is what a real session's tools/list
|
|
7
|
+
* answers, taken from the same buildServer() the endpoint uses, so the card
|
|
8
|
+
* cannot drift from the tools. Tool definitions name no internal engine, so
|
|
9
|
+
* nothing is filtered here; test/annotations.test.mjs and test/card.test.mjs
|
|
10
|
+
* fail the build if one ever does.
|
|
11
|
+
*/
|
|
12
|
+
export declare const HOSTED_MCP_URL = "https://mcp.spicrawl.com/mcp";
|
|
13
|
+
export declare function buildServerCard(): Promise<{
|
|
14
|
+
serverInfo: {
|
|
15
|
+
name: string;
|
|
16
|
+
title: string;
|
|
17
|
+
version: string;
|
|
18
|
+
description: string;
|
|
19
|
+
websiteUrl: string;
|
|
20
|
+
};
|
|
21
|
+
transport: {
|
|
22
|
+
type: string;
|
|
23
|
+
endpoint: string;
|
|
24
|
+
};
|
|
25
|
+
authentication: {
|
|
26
|
+
required: boolean;
|
|
27
|
+
schemes: string[];
|
|
28
|
+
};
|
|
29
|
+
tools: {
|
|
30
|
+
inputSchema: {
|
|
31
|
+
[x: string]: unknown;
|
|
32
|
+
type: "object";
|
|
33
|
+
properties?: Record<string, object> | undefined;
|
|
34
|
+
required?: string[] | undefined;
|
|
35
|
+
};
|
|
36
|
+
name: string;
|
|
37
|
+
description?: string | undefined;
|
|
38
|
+
outputSchema?: {
|
|
39
|
+
[x: string]: unknown;
|
|
40
|
+
type: "object";
|
|
41
|
+
properties?: Record<string, object> | undefined;
|
|
42
|
+
required?: string[] | undefined;
|
|
43
|
+
} | undefined;
|
|
44
|
+
annotations?: {
|
|
45
|
+
title?: string | undefined;
|
|
46
|
+
readOnlyHint?: boolean | undefined;
|
|
47
|
+
destructiveHint?: boolean | undefined;
|
|
48
|
+
idempotentHint?: boolean | undefined;
|
|
49
|
+
openWorldHint?: boolean | undefined;
|
|
50
|
+
} | undefined;
|
|
51
|
+
execution?: {
|
|
52
|
+
taskSupport?: "optional" | "required" | "forbidden" | undefined;
|
|
53
|
+
} | undefined;
|
|
54
|
+
_meta?: Record<string, unknown> | undefined;
|
|
55
|
+
icons?: {
|
|
56
|
+
src: string;
|
|
57
|
+
mimeType?: string | undefined;
|
|
58
|
+
sizes?: string[] | undefined;
|
|
59
|
+
theme?: "light" | "dark" | undefined;
|
|
60
|
+
}[] | undefined;
|
|
61
|
+
title?: string | undefined;
|
|
62
|
+
}[];
|
|
63
|
+
}>;
|
package/dist/card.js
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The static MCP server card, served unauthenticated at
|
|
3
|
+
* GET /.well-known/mcp/server-card.json (src/http.ts). Directories such as
|
|
4
|
+
* Smithery cannot scan /mcp (it answers 401 without a key) and read this instead.
|
|
5
|
+
*
|
|
6
|
+
* The tool list is not written by hand: it is what a real session's tools/list
|
|
7
|
+
* answers, taken from the same buildServer() the endpoint uses, so the card
|
|
8
|
+
* cannot drift from the tools. Tool definitions name no internal engine, so
|
|
9
|
+
* nothing is filtered here; test/annotations.test.mjs and test/card.test.mjs
|
|
10
|
+
* fail the build if one ever does.
|
|
11
|
+
*/
|
|
12
|
+
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
|
|
13
|
+
import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
|
|
14
|
+
import { ILayerClient, PACKAGE_VERSION } from "./client.js";
|
|
15
|
+
import { buildServer } from "./server.js";
|
|
16
|
+
// ponytail: the hosted product's URL; a self-hosted card still names it. Add an env override if anyone self-hosts and registers.
|
|
17
|
+
export const HOSTED_MCP_URL = "https://mcp.spicrawl.com/mcp";
|
|
18
|
+
const DOCS_URL = "https://docs.spicrawl.com/agents/mcp";
|
|
19
|
+
/** What a session's tools/list says: the real definitions, from an in-memory client. */
|
|
20
|
+
async function listTools() {
|
|
21
|
+
// The key and URLs are never used: tools/list calls nothing upstream.
|
|
22
|
+
const server = buildServer(new ILayerClient({ apiKey: "spicrawl_live_card", baseURL: HOSTED_MCP_URL, publicBaseURL: HOSTED_MCP_URL }), {
|
|
23
|
+
hideUnavailable: true, // list what works today, like a customer who set SPICRAWL_MCP_HIDE_UNAVAILABLE
|
|
24
|
+
});
|
|
25
|
+
const client = new Client({ name: "server-card", version: PACKAGE_VERSION });
|
|
26
|
+
const [clientSide, serverSide] = InMemoryTransport.createLinkedPair();
|
|
27
|
+
await Promise.all([server.connect(serverSide), client.connect(clientSide)]);
|
|
28
|
+
try {
|
|
29
|
+
return (await client.listTools()).tools;
|
|
30
|
+
}
|
|
31
|
+
finally {
|
|
32
|
+
await client.close().catch(() => { });
|
|
33
|
+
await server.close().catch(() => { });
|
|
34
|
+
}
|
|
35
|
+
}
|
|
36
|
+
export async function buildServerCard() {
|
|
37
|
+
return {
|
|
38
|
+
serverInfo: {
|
|
39
|
+
name: "Spicrawl",
|
|
40
|
+
title: "Spicrawl",
|
|
41
|
+
version: PACKAGE_VERSION,
|
|
42
|
+
description: "Web scraping to Markdown and structured data, screenshots, batch jobs and browser sessions for AI agents.",
|
|
43
|
+
websiteUrl: DOCS_URL,
|
|
44
|
+
},
|
|
45
|
+
transport: { type: "streamable-http", endpoint: HOSTED_MCP_URL },
|
|
46
|
+
authentication: { required: true, schemes: ["bearer"] },
|
|
47
|
+
tools: await listTools(),
|
|
48
|
+
};
|
|
49
|
+
}
|
|
50
|
+
//# sourceMappingURL=card.js.map
|
package/dist/card.js.map
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"card.js","sourceRoot":"","sources":["../src/card.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AAEH,OAAO,EAAE,MAAM,EAAE,MAAM,2CAA2C,CAAC;AACnE,OAAO,EAAE,iBAAiB,EAAE,MAAM,uCAAuC,CAAC;AAC1E,OAAO,EAAE,YAAY,EAAE,eAAe,EAAE,MAAM,aAAa,CAAC;AAC5D,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAE1C,iIAAiI;AACjI,MAAM,CAAC,MAAM,cAAc,GAAG,8BAA8B,CAAC;AAC7D,MAAM,QAAQ,GAAG,sCAAsC,CAAC;AAExD,wFAAwF;AACxF,KAAK,UAAU,SAAS;IACtB,sEAAsE;IACtE,MAAM,MAAM,GAAG,WAAW,CAAC,IAAI,YAAY,CAAC,EAAE,MAAM,EAAE,oBAAoB,EAAE,OAAO,EAAE,cAAc,EAAE,aAAa,EAAE,cAAc,EAAE,CAAC,EAAE;QACrI,eAAe,EAAE,IAAI,EAAE,+EAA+E;KACvG,CAAC,CAAC;IACH,MAAM,MAAM,GAAG,IAAI,MAAM,CAAC,EAAE,IAAI,EAAE,aAAa,EAAE,OAAO,EAAE,eAAe,EAAE,CAAC,CAAC;IAC7E,MAAM,CAAC,UAAU,EAAE,UAAU,CAAC,GAAG,iBAAiB,CAAC,gBAAgB,EAAE,CAAC;IACtE,MAAM,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,UAAU,CAAC,EAAE,MAAM,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC;IAC5E,IAAI,CAAC;QACH,OAAO,CAAC,MAAM,MAAM,CAAC,SAAS,EAAE,CAAC,CAAC,KAAK,CAAC;IAC1C,CAAC;YAAS,CAAC;QACT,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;QACrC,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;IACvC,CAAC;AACH,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,eAAe;IACnC,OAAO;QACL,UAAU,EAAE;YACV,IAAI,EAAE,UAAU;YAChB,KAAK,EAAE,UAAU;YACjB,OAAO,EAAE,eAAe;YACxB,WAAW,EAAE,2GAA2G;YACxH,UAAU,EAAE,QAAQ;SACrB;QACD,SAAS,EAAE,EAAE,IAAI,EAAE,iBAAiB,EAAE,QAAQ,EAAE,cAAc,EAAE;QAChE,cAAc,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC,QAAQ,CAAC,EAAE;QACvD,KAAK,EAAE,MAAM,SAAS,EAAE;KACzB,CAAC;AACJ,CAAC"}
|
package/dist/http.d.ts
CHANGED
|
@@ -3,11 +3,14 @@
|
|
|
3
3
|
* spicrawl-mcp-http — the Spicrawl MCP server over Streamable HTTP, so a user can
|
|
4
4
|
* connect an MCP client to a hosted URL instead of installing anything.
|
|
5
5
|
*
|
|
6
|
-
* POST/GET/DELETE /mcp MCP Streamable HTTP endpoint (
|
|
6
|
+
* POST/GET/DELETE /mcp MCP Streamable HTTP endpoint (API key auth)
|
|
7
7
|
* GET /healthz liveness, no auth
|
|
8
|
+
* GET /.well-known/mcp/server-card.json static server card for directories (src/card.ts), no auth
|
|
8
9
|
* /chatgpt/mcp and its OAuth metadata: see src/chatgpt/http.ts (off by default)
|
|
9
10
|
*
|
|
10
|
-
* Auth is the caller's own Spicrawl API key
|
|
11
|
+
* Auth is the caller's own Spicrawl API key, in `Authorization: Bearer spicrawl_live_...`
|
|
12
|
+
* or, for gateways that cannot send a Bearer prefix, a raw `Authorization: spicrawl_live_...`,
|
|
13
|
+
* `x-api-key` or `api-key` (see apiKeyFromRequest; never a query parameter);
|
|
11
14
|
* every tool call runs upstream under that key. One MCP server per session, and
|
|
12
15
|
* a session is pinned to the key that created it: a request presenting another
|
|
13
16
|
* key with that session id is refused (403), so no caller can run under someone
|
package/dist/http.js
CHANGED
|
@@ -3,11 +3,14 @@
|
|
|
3
3
|
* spicrawl-mcp-http — the Spicrawl MCP server over Streamable HTTP, so a user can
|
|
4
4
|
* connect an MCP client to a hosted URL instead of installing anything.
|
|
5
5
|
*
|
|
6
|
-
* POST/GET/DELETE /mcp MCP Streamable HTTP endpoint (
|
|
6
|
+
* POST/GET/DELETE /mcp MCP Streamable HTTP endpoint (API key auth)
|
|
7
7
|
* GET /healthz liveness, no auth
|
|
8
|
+
* GET /.well-known/mcp/server-card.json static server card for directories (src/card.ts), no auth
|
|
8
9
|
* /chatgpt/mcp and its OAuth metadata: see src/chatgpt/http.ts (off by default)
|
|
9
10
|
*
|
|
10
|
-
* Auth is the caller's own Spicrawl API key
|
|
11
|
+
* Auth is the caller's own Spicrawl API key, in `Authorization: Bearer spicrawl_live_...`
|
|
12
|
+
* or, for gateways that cannot send a Bearer prefix, a raw `Authorization: spicrawl_live_...`,
|
|
13
|
+
* `x-api-key` or `api-key` (see apiKeyFromRequest; never a query parameter);
|
|
11
14
|
* every tool call runs upstream under that key. One MCP server per session, and
|
|
12
15
|
* a session is pinned to the key that created it: a request presenting another
|
|
13
16
|
* key with that session id is refused (403), so no caller can run under someone
|
|
@@ -37,9 +40,10 @@ import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/
|
|
|
37
40
|
import { isInitializeRequest } from "@modelcontextprotocol/sdk/types.js";
|
|
38
41
|
import { ILayerClient, USER_AGENT } from "./client.js";
|
|
39
42
|
import { buildServer, isTruthy } from "./server.js";
|
|
40
|
-
import {
|
|
43
|
+
import { apiKeyFromRequest, readJSON, rpcError } from "./httputil.js";
|
|
41
44
|
import { chatgptConfigFromEnv } from "./chatgpt/oauth.js";
|
|
42
45
|
import { chatgptRoutes } from "./chatgpt/http.js";
|
|
46
|
+
import { buildServerCard } from "./card.js";
|
|
43
47
|
const ADDR = process.env.SPICRAWL_MCP_ADDR || "127.0.0.1:8090";
|
|
44
48
|
const BASE_URL = process.env.SPICRAWL_BASE_URL || "http://127.0.0.1:8080";
|
|
45
49
|
const PUBLIC_BASE_URL = process.env.SPICRAWL_PUBLIC_BASE_URL || BASE_URL;
|
|
@@ -48,10 +52,6 @@ const KEY_CHECK_TIMEOUT_MS = 5_000;
|
|
|
48
52
|
const sessions = new Map();
|
|
49
53
|
const log = (...a) => console.error(new Date().toISOString(), ...a);
|
|
50
54
|
const hashKey = (k) => createHash("sha256").update(k).digest();
|
|
51
|
-
function bearer(req) {
|
|
52
|
-
const t = bearerToken(req);
|
|
53
|
-
return t && KEY_RE.test(t) ? t : null;
|
|
54
|
-
}
|
|
55
55
|
// The key's shape is checked locally, but only the API knows whether it exists.
|
|
56
56
|
// Without asking, a revoked key opens a session and fails only on its first tool
|
|
57
57
|
// call, which clients surface as a broken tool rather than a bad credential.
|
|
@@ -105,9 +105,9 @@ async function closeSession(id, why) {
|
|
|
105
105
|
await s.server.close().catch(() => { });
|
|
106
106
|
}
|
|
107
107
|
async function handleMcp(req, res) {
|
|
108
|
-
const key =
|
|
108
|
+
const key = apiKeyFromRequest(req);
|
|
109
109
|
if (!key) {
|
|
110
|
-
return rpcError(res, 401, -32001, "Unauthorized: send your Spicrawl API key as 'Authorization: Bearer spicrawl_live_...'", { "WWW-Authenticate": 'Bearer realm="spicrawl-mcp"' });
|
|
110
|
+
return rpcError(res, 401, -32001, "Unauthorized: send your Spicrawl API key as 'Authorization: Bearer spicrawl_live_...' (or in an 'x-api-key' header)", { "WWW-Authenticate": 'Bearer realm="spicrawl-mcp"' });
|
|
111
111
|
}
|
|
112
112
|
const keyHash = hashKey(key);
|
|
113
113
|
let body;
|
|
@@ -181,8 +181,35 @@ if (chatgptConfig.enabled) {
|
|
|
181
181
|
log("chatgpt: SPICRAWL_OAUTH_INTROSPECT_SECRET is not set; every token will be answered 503");
|
|
182
182
|
}
|
|
183
183
|
const chatgpt = chatgptRoutes(chatgptConfig, { baseURL: BASE_URL, publicBaseURL: PUBLIC_BASE_URL, log });
|
|
184
|
+
// Built on first request and kept: it only changes with a new release. A failed
|
|
185
|
+
// build is not kept, so the next request tries again.
|
|
186
|
+
let serverCard;
|
|
187
|
+
async function handleCard(req, res) {
|
|
188
|
+
try {
|
|
189
|
+
serverCard ??= buildServerCard().then((c) => JSON.stringify(c));
|
|
190
|
+
const body = await serverCard;
|
|
191
|
+
res.writeHead(200, {
|
|
192
|
+
"Content-Type": "application/json; charset=utf-8",
|
|
193
|
+
"Cache-Control": "public, max-age=3600",
|
|
194
|
+
"Access-Control-Allow-Origin": "*",
|
|
195
|
+
});
|
|
196
|
+
res.end(req.method === "HEAD" ? undefined : body);
|
|
197
|
+
}
|
|
198
|
+
catch (err) {
|
|
199
|
+
serverCard = undefined;
|
|
200
|
+
log("server card failed:", err instanceof Error ? err.message : String(err));
|
|
201
|
+
rpcError(res, 500, -32603, "Internal error");
|
|
202
|
+
}
|
|
203
|
+
}
|
|
184
204
|
const http = createServer((req, res) => {
|
|
185
205
|
const path = (req.url || "/").split("?")[0];
|
|
206
|
+
if (path === "/.well-known/mcp/server-card.json") {
|
|
207
|
+
if (req.method === "GET" || req.method === "HEAD")
|
|
208
|
+
void handleCard(req, res);
|
|
209
|
+
else
|
|
210
|
+
rpcError(res, 405, -32000, "Method not allowed", { Allow: "GET, HEAD" });
|
|
211
|
+
return;
|
|
212
|
+
}
|
|
186
213
|
if (path === "/healthz" && (req.method === "GET" || req.method === "HEAD")) {
|
|
187
214
|
res.writeHead(200, { "Content-Type": "application/json" });
|
|
188
215
|
res.end(req.method === "HEAD" ? undefined : JSON.stringify({ status: "ok", sessions: sessions.size }));
|
package/dist/http.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"http.js","sourceRoot":"","sources":["../src/http.ts"],"names":[],"mappings":";AACA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA+BG;AAEH,OAAO,EAAE,UAAU,EAAE,UAAU,EAAE,eAAe,EAAE,MAAM,aAAa,CAAC;AACtE,OAAO,EAAE,YAAY,EAA6C,MAAM,WAAW,CAAC;AACpF,OAAO,EAAE,6BAA6B,EAAE,MAAM,oDAAoD,CAAC;AAEnG,OAAO,EAAE,mBAAmB,EAAE,MAAM,oCAAoC,CAAC;AACzE,OAAO,EAAE,YAAY,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AACvD,OAAO,EAAE,WAAW,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AACpD,OAAO,EAAE,MAAM,EAAE,WAAW,EAAE,QAAQ,EAAE,QAAQ,EAAE,MAAM,eAAe,CAAC;AACxE,OAAO,EAAE,oBAAoB,EAAsB,MAAM,oBAAoB,CAAC;AAC9E,OAAO,EAAE,aAAa,EAAE,MAAM,mBAAmB,CAAC;AAElD,MAAM,IAAI,GAAG,OAAO,CAAC,GAAG,CAAC,iBAAiB,IAAI,gBAAgB,CAAC;AAC/D,MAAM,QAAQ,GAAG,OAAO,CAAC,GAAG,CAAC,iBAAiB,IAAI,uBAAuB,CAAC;AAC1E,MAAM,eAAe,GAAG,OAAO,CAAC,GAAG,CAAC,wBAAwB,IAAI,QAAQ,CAAC;AACzE,MAAM,OAAO,GAAG,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,4BAA4B,CAAC,IAAI,EAAE,GAAG,EAAE,GAAG,IAAI,CAAC;AACnF,MAAM,oBAAoB,GAAG,KAAK,CAAC;AASnC,MAAM,QAAQ,GAAG,IAAI,GAAG,EAAmB,CAAC;AAE5C,MAAM,GAAG,GAAG,CAAC,GAAG,CAAY,EAAE,EAAE,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE,EAAE,GAAG,CAAC,CAAC,CAAC;AAC/E,MAAM,OAAO,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,UAAU,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC;AAEvE,SAAS,MAAM,CAAC,GAAoB;IAClC,MAAM,CAAC,GAAG,WAAW,CAAC,GAAG,CAAC,CAAC;IAC3B,OAAO,CAAC,IAAI,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;AACxC,CAAC;AAID,gFAAgF;AAChF,iFAAiF;AACjF,6EAA6E;AAC7E,+EAA+E;AAC/E,8DAA8D;AAC9D,KAAK,UAAU,QAAQ,CAAC,GAAW;IACjC,IAAI,GAAa,CAAC;IAClB,IAAI,CAAC;QACH,GAAG,GAAG,MAAM,KAAK,CAAC,GAAG,QAAQ,sBAAsB,EAAE;YACnD,OAAO,EAAE,EAAE,aAAa,EAAE,UAAU,GAAG,EAAE,EAAE,MAAM,EAAE,kBAAkB,EAAE,YAAY,EAAE,UAAU,EAAE;YACjG,MAAM,EAAE,WAAW,CAAC,OAAO,CAAC,oBAAoB,CAAC;SAClD,CAAC,CAAC;IACL,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,GAAG,CAAC,6BAA6B,EAAE,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC;QACrF,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,wFAAwF,EAAE,CAAC;IACvI,CAAC;IACD,MAAM,IAAI,GAAG,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,EAAE,CAAC,CAAC;IAC9C,IAAI,GAAG,CAAC,MAAM,KAAK,GAAG,EAAE,CAAC;QACvB,IAAI,IAAI,GAAG,wBAAwB,CAAC;QACpC,IAAI,MAAM,GAAG,4CAA4C,CAAC;QAC1D,IAAI,CAAC;YACH,MAAM,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAA0D,CAAC;YACpF,IAAI,OAAO,CAAC,CAAC,IAAI,KAAK,QAAQ,IAAI,CAAC,CAAC,IAAI;gBAAE,IAAI,GAAG,CAAC,CAAC,IAAI,CAAC;YACxD,IAAI,OAAO,CAAC,CAAC,MAAM,KAAK,QAAQ,IAAI,CAAC,CAAC,MAAM;gBAAE,MAAM,GAAG,CAAC,CAAC,MAAM,CAAC;iBAC3D,IAAI,OAAO,CAAC,CAAC,KAAK,KAAK,QAAQ,IAAI,CAAC,CAAC,KAAK;gBAAE,MAAM,GAAG,CAAC,CAAC,KAAK,CAAC;QACpE,CAAC;QAAC,MAAM,CAAC;YACP,yDAAyD;QAC3D,CAAC;QACD,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,iBAAiB,IAAI,KAAK,MAAM,EAAE,EAAE,CAAC;IACjF,CAAC;IACD,yEAAyE;IACzE,6CAA6C;IAC7C,IAAI,GAAG,CAAC,MAAM,IAAI,GAAG,EAAE,CAAC;QACtB,GAAG,CAAC,gCAAgC,GAAG,CAAC,MAAM,EAAE,CAAC,CAAC;QAClD,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,uDAAuD,GAAG,CAAC,MAAM,yCAAyC,EAAE,CAAC;IACzJ,CAAC;IACD,OAAO,EAAE,EAAE,EAAE,IAAI,EAAE,CAAC;AACtB,CAAC;AAED,KAAK,UAAU,YAAY,CAAC,EAAU,EAAE,GAAW;IACjD,MAAM,CAAC,GAAG,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;IAC3B,IAAI,CAAC,CAAC;QAAE,OAAO;IACf,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;IACpB,GAAG,CAAC,WAAW,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,YAAY,GAAG,MAAM,QAAQ,CAAC,IAAI,OAAO,CAAC,CAAC;IACxE,MAAM,CAAC,CAAC,SAAS,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;IAC1C,MAAM,CAAC,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;AACzC,CAAC;AAED,KAAK,UAAU,SAAS,CAAC,GAAoB,EAAE,GAAmB;IAChE,MAAM,GAAG,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IACxB,IAAI,CAAC,GAAG,EAAE,CAAC;QACT,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAC9B,uFAAuF,EACvF,EAAE,kBAAkB,EAAE,6BAA6B,EAAE,CAAC,CAAC;IAC3D,CAAC;IACD,MAAM,OAAO,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC;IAE7B,IAAI,IAAa,CAAC;IAClB,IAAI,GAAG,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;QAC1B,IAAI,CAAC;YACH,IAAI,GAAG,MAAM,QAAQ,CAAC,GAAG,CAAC,CAAC;QAC7B,CAAC;QAAC,MAAM,CAAC;YACP,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,4CAA4C,CAAC,CAAC;QAClF,CAAC;IACH,CAAC;IAED,MAAM,GAAG,GAAG,GAAG,CAAC,OAAO,CAAC,gBAAgB,CAAC,CAAC;IAC1C,IAAI,OAAO,GAAG,KAAK,QAAQ,IAAI,GAAG,EAAE,CAAC;QACnC,MAAM,CAAC,GAAG,QAAQ,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;QAC5B,IAAI,CAAC,CAAC;YAAE,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,6CAA6C,CAAC,CAAC;QACzF,IAAI,CAAC,eAAe,CAAC,CAAC,CAAC,OAAO,EAAE,OAAO,CAAC,EAAE,CAAC;YACzC,GAAG,CAAC,WAAW,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,4CAA4C,CAAC,CAAC;YAC5E,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,wDAAwD,CAAC,CAAC;QAC9F,CAAC;QACD,CAAC,CAAC,QAAQ,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;QACxB,MAAM,CAAC,CAAC,SAAS,CAAC,aAAa,CAAC,GAAG,EAAE,GAAG,EAAE,IAAI,CAAC,CAAC;QAChD,OAAO;IACT,CAAC;IAED,IAAI,GAAG,CAAC,MAAM,KAAK,MAAM,IAAI,CAAC,mBAAmB,CAAC,IAAI,CAAC,EAAE,CAAC;QACxD,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,sEAAsE,CAAC,CAAC;IAC5G,CAAC;IAED,0EAA0E;IAC1E,yEAAyE;IACzE,MAAM,KAAK,GAAG,MAAM,QAAQ,CAAC,GAAG,CAAC,CAAC;IAClC,IAAI,CAAC,KAAK,CAAC,EAAE,EAAE,CAAC;QACd,OAAO,KAAK,CAAC,MAAM,KAAK,GAAG;YACzB,CAAC,CAAC,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,KAAK,CAAC,OAAO,EAAE,EAAE,kBAAkB,EAAE,6BAA6B,EAAE,CAAC;YAClG,CAAC,CAAC,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,KAAK,CAAC,OAAO,EAAE,EAAE,aAAa,EAAE,GAAG,EAAE,CAAC,CAAC;IACxE,CAAC;IAED,MAAM,MAAM,GAAG,WAAW,CAAC,IAAI,YAAY,CAAC,EAAE,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,QAAQ,EAAE,aAAa,EAAE,eAAe,EAAE,CAAC,CAAC,CAAC;IACjH,MAAM,SAAS,GAAkC,IAAI,6BAA6B,CAAC;QACjF,kBAAkB,EAAE,GAAG,EAAE,CAAC,UAAU,EAAE;QACtC,oBAAoB,EAAE,CAAC,EAAE,EAAE,EAAE;YAC3B,QAAQ,CAAC,GAAG,CAAC,EAAE,EAAE,EAAE,SAAS,EAAE,MAAM,EAAE,OAAO,EAAE,QAAQ,EAAE,IAAI,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;YACvE,GAAG,CAAC,WAAW,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,YAAY,QAAQ,CAAC,IAAI,OAAO,CAAC,CAAC;QACjE,CAAC;KACF,CAAC,CAAC;IACH,SAAS,CAAC,OAAO,GAAG,GAAG,EAAE;QACvB,IAAI,SAAS,CAAC,SAAS;YAAE,KAAK,YAAY,CAAC,SAAS,CAAC,SAAS,EAAE,QAAQ,CAAC,CAAC;IAC5E,CAAC,CAAC;IACF,MAAM,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IAChC,MAAM,SAAS,CAAC,aAAa,CAAC,GAAG,EAAE,GAAG,EAAE,IAAI,CAAC,CAAC;AAChD,CAAC;AAED,4EAA4E;AAC5E,2EAA2E;AAC3E,uCAAuC;AACvC,IAAI,aAA4B,CAAC;AACjC,IAAI,CAAC;IACH,aAAa,GAAG,oBAAoB,EAAE,CAAC;AACzC,CAAC;AAAC,OAAO,GAAG,EAAE,CAAC;IACb,MAAM,GAAG,GAAG,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;IAC7D,IAAI,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,wBAAwB,CAAC,EAAE,CAAC;QACnD,GAAG,CAAC,mCAAmC,GAAG,EAAE,CAAC,CAAC;QAC9C,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACnB,CAAC;IACD,GAAG,CAAC,yDAAyD,GAAG,EAAE,CAAC,CAAC;IACpE,aAAa,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE,QAAQ,EAAE,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,gBAAgB,EAAE,EAAE,EAAE,OAAO,EAAE,KAAK,EAAE,aAAa,EAAE,EAAE,EAAE,CAAC;AACxH,CAAC;AACD,IAAI,aAAa,CAAC,OAAO,EAAE,CAAC;IAC1B,GAAG,CAAC,yCAAyC,aAAa,CAAC,QAAQ,YAAY,aAAa,CAAC,MAAM,EAAE;QACnG,GAAG,aAAa,CAAC,OAAO,CAAC,CAAC,CAAC,6CAA6C,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;IACnF,IAAI,CAAC,aAAa,CAAC,gBAAgB;QAAE,GAAG,CAAC,wFAAwF,CAAC,CAAC;AACrI,CAAC;AACD,MAAM,OAAO,GAAG,aAAa,CAAC,aAAa,EAAE,EAAE,OAAO,EAAE,QAAQ,EAAE,aAAa,EAAE,eAAe,EAAE,GAAG,EAAE,CAAC,CAAC;AAEzG,MAAM,IAAI,GAAG,YAAY,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,EAAE;IACrC,MAAM,IAAI,GAAG,CAAC,GAAG,CAAC,GAAG,IAAI,GAAG,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC;IAC5C,IAAI,IAAI,KAAK,UAAU,IAAI,CAAC,GAAG,CAAC,MAAM,KAAK,KAAK,IAAI,GAAG,CAAC,MAAM,KAAK,MAAM,CAAC,EAAE,CAAC;QAC3E,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAC;QAC3D,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,MAAM,KAAK,MAAM,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,MAAM,EAAE,IAAI,EAAE,QAAQ,EAAE,QAAQ,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;QACvG,OAAO;IACT,CAAC;IACD,IAAI,OAAO,CAAC,GAAG,EAAE,GAAG,EAAE,IAAI,CAAC;QAAE,OAAO;IACpC,IAAI,IAAI,KAAK,MAAM;QAAE,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,qCAAqC,CAAC,CAAC;IAC9F,IAAI,CAAC,CAAC,KAAK,EAAE,MAAM,EAAE,QAAQ,CAAC,CAAC,QAAQ,CAAC,GAAG,CAAC,MAAM,IAAI,EAAE,CAAC,EAAE,CAAC;QAC1D,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,oBAAoB,EAAE,EAAE,KAAK,EAAE,mBAAmB,EAAE,CAAC,CAAC;IAC1F,CAAC;IACD,SAAS,CAAC,GAAG,EAAE,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,GAAG,EAAE,EAAE;QAChC,GAAG,CAAC,iBAAiB,EAAE,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC;QACzE,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,gBAAgB,CAAC,CAAC;IAC/C,CAAC,CAAC,CAAC;AACL,CAAC,CAAC,CAAC;AAEH,4EAA4E;AAC5E,uCAAuC;AACvC,WAAW,CAAC,GAAG,EAAE;IACf,MAAM,MAAM,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,OAAO,CAAC;IACpC,KAAK,MAAM,CAAC,EAAE,EAAE,CAAC,CAAC,IAAI,QAAQ;QAAE,IAAI,CAAC,CAAC,QAAQ,GAAG,MAAM;YAAE,KAAK,YAAY,CAAC,EAAE,EAAE,MAAM,CAAC,CAAC;AACzF,CAAC,EAAE,MAAM,CAAC,CAAC,KAAK,EAAE,CAAC;AAEnB,MAAM,CAAC,GAAG,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC;AAChC,MAAM,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,WAAW,CAAC;AAC5E,MAAM,IAAI,GAAG,MAAM,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;AACvD,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,IAAI,IAAI,GAAG,KAAK,EAAE,CAAC;IACzD,GAAG,CAAC,8BAA8B,IAAI,oBAAoB,CAAC,CAAC;IAC5D,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;AACnB,CAAC;AAED,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE,IAAI,EAAE,GAAG,EAAE,CAAC,GAAG,CAAC,yCAAyC,IAAI,IAAI,IAAI,WAAW,QAAQ,EAAE,CAAC,CAAC,CAAC;AAC/G,IAAI,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC,GAAG,EAAE,EAAE;IACvB,GAAG,CAAC,2BAA2B,EAAE,GAAG,CAAC,OAAO,CAAC,CAAC;IAC9C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC;AAEH,MAAM,QAAQ,GAAG,GAAG,EAAE;IACpB,GAAG,CAAC,eAAe,CAAC,CAAC;IACrB,IAAI,CAAC,KAAK,EAAE,CAAC;IACb,KAAK,MAAM,EAAE,IAAI,CAAC,GAAG,QAAQ,CAAC,IAAI,EAAE,CAAC;QAAE,KAAK,YAAY,CAAC,EAAE,EAAE,UAAU,CAAC,CAAC;IACzE,UAAU,CAAC,GAAG,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,CAAC,CAAC,KAAK,EAAE,CAAC;AAClD,CAAC,CAAC;AACF,OAAO,CAAC,EAAE,CAAC,SAAS,EAAE,QAAQ,CAAC,CAAC;AAChC,OAAO,CAAC,EAAE,CAAC,QAAQ,EAAE,QAAQ,CAAC,CAAC"}
|
|
1
|
+
{"version":3,"file":"http.js","sourceRoot":"","sources":["../src/http.ts"],"names":[],"mappings":";AACA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAkCG;AAEH,OAAO,EAAE,UAAU,EAAE,UAAU,EAAE,eAAe,EAAE,MAAM,aAAa,CAAC;AACtE,OAAO,EAAE,YAAY,EAA6C,MAAM,WAAW,CAAC;AACpF,OAAO,EAAE,6BAA6B,EAAE,MAAM,oDAAoD,CAAC;AAEnG,OAAO,EAAE,mBAAmB,EAAE,MAAM,oCAAoC,CAAC;AACzE,OAAO,EAAE,YAAY,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AACvD,OAAO,EAAE,WAAW,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AACpD,OAAO,EAAE,iBAAiB,EAAE,QAAQ,EAAE,QAAQ,EAAE,MAAM,eAAe,CAAC;AACtE,OAAO,EAAE,oBAAoB,EAAsB,MAAM,oBAAoB,CAAC;AAC9E,OAAO,EAAE,aAAa,EAAE,MAAM,mBAAmB,CAAC;AAClD,OAAO,EAAE,eAAe,EAAE,MAAM,WAAW,CAAC;AAE5C,MAAM,IAAI,GAAG,OAAO,CAAC,GAAG,CAAC,iBAAiB,IAAI,gBAAgB,CAAC;AAC/D,MAAM,QAAQ,GAAG,OAAO,CAAC,GAAG,CAAC,iBAAiB,IAAI,uBAAuB,CAAC;AAC1E,MAAM,eAAe,GAAG,OAAO,CAAC,GAAG,CAAC,wBAAwB,IAAI,QAAQ,CAAC;AACzE,MAAM,OAAO,GAAG,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,4BAA4B,CAAC,IAAI,EAAE,GAAG,EAAE,GAAG,IAAI,CAAC;AACnF,MAAM,oBAAoB,GAAG,KAAK,CAAC;AASnC,MAAM,QAAQ,GAAG,IAAI,GAAG,EAAmB,CAAC;AAE5C,MAAM,GAAG,GAAG,CAAC,GAAG,CAAY,EAAE,EAAE,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE,EAAE,GAAG,CAAC,CAAC,CAAC;AAC/E,MAAM,OAAO,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,UAAU,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC;AAIvE,gFAAgF;AAChF,iFAAiF;AACjF,6EAA6E;AAC7E,+EAA+E;AAC/E,8DAA8D;AAC9D,KAAK,UAAU,QAAQ,CAAC,GAAW;IACjC,IAAI,GAAa,CAAC;IAClB,IAAI,CAAC;QACH,GAAG,GAAG,MAAM,KAAK,CAAC,GAAG,QAAQ,sBAAsB,EAAE;YACnD,OAAO,EAAE,EAAE,aAAa,EAAE,UAAU,GAAG,EAAE,EAAE,MAAM,EAAE,kBAAkB,EAAE,YAAY,EAAE,UAAU,EAAE;YACjG,MAAM,EAAE,WAAW,CAAC,OAAO,CAAC,oBAAoB,CAAC;SAClD,CAAC,CAAC;IACL,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,GAAG,CAAC,6BAA6B,EAAE,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC;QACrF,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,wFAAwF,EAAE,CAAC;IACvI,CAAC;IACD,MAAM,IAAI,GAAG,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,EAAE,CAAC,CAAC;IAC9C,IAAI,GAAG,CAAC,MAAM,KAAK,GAAG,EAAE,CAAC;QACvB,IAAI,IAAI,GAAG,wBAAwB,CAAC;QACpC,IAAI,MAAM,GAAG,4CAA4C,CAAC;QAC1D,IAAI,CAAC;YACH,MAAM,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAA0D,CAAC;YACpF,IAAI,OAAO,CAAC,CAAC,IAAI,KAAK,QAAQ,IAAI,CAAC,CAAC,IAAI;gBAAE,IAAI,GAAG,CAAC,CAAC,IAAI,CAAC;YACxD,IAAI,OAAO,CAAC,CAAC,MAAM,KAAK,QAAQ,IAAI,CAAC,CAAC,MAAM;gBAAE,MAAM,GAAG,CAAC,CAAC,MAAM,CAAC;iBAC3D,IAAI,OAAO,CAAC,CAAC,KAAK,KAAK,QAAQ,IAAI,CAAC,CAAC,KAAK;gBAAE,MAAM,GAAG,CAAC,CAAC,KAAK,CAAC;QACpE,CAAC;QAAC,MAAM,CAAC;YACP,yDAAyD;QAC3D,CAAC;QACD,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,iBAAiB,IAAI,KAAK,MAAM,EAAE,EAAE,CAAC;IACjF,CAAC;IACD,yEAAyE;IACzE,6CAA6C;IAC7C,IAAI,GAAG,CAAC,MAAM,IAAI,GAAG,EAAE,CAAC;QACtB,GAAG,CAAC,gCAAgC,GAAG,CAAC,MAAM,EAAE,CAAC,CAAC;QAClD,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,uDAAuD,GAAG,CAAC,MAAM,yCAAyC,EAAE,CAAC;IACzJ,CAAC;IACD,OAAO,EAAE,EAAE,EAAE,IAAI,EAAE,CAAC;AACtB,CAAC;AAED,KAAK,UAAU,YAAY,CAAC,EAAU,EAAE,GAAW;IACjD,MAAM,CAAC,GAAG,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;IAC3B,IAAI,CAAC,CAAC;QAAE,OAAO;IACf,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;IACpB,GAAG,CAAC,WAAW,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,YAAY,GAAG,MAAM,QAAQ,CAAC,IAAI,OAAO,CAAC,CAAC;IACxE,MAAM,CAAC,CAAC,SAAS,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;IAC1C,MAAM,CAAC,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;AACzC,CAAC;AAED,KAAK,UAAU,SAAS,CAAC,GAAoB,EAAE,GAAmB;IAChE,MAAM,GAAG,GAAG,iBAAiB,CAAC,GAAG,CAAC,CAAC;IACnC,IAAI,CAAC,GAAG,EAAE,CAAC;QACT,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAC9B,qHAAqH,EACrH,EAAE,kBAAkB,EAAE,6BAA6B,EAAE,CAAC,CAAC;IAC3D,CAAC;IACD,MAAM,OAAO,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC;IAE7B,IAAI,IAAa,CAAC;IAClB,IAAI,GAAG,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;QAC1B,IAAI,CAAC;YACH,IAAI,GAAG,MAAM,QAAQ,CAAC,GAAG,CAAC,CAAC;QAC7B,CAAC;QAAC,MAAM,CAAC;YACP,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,4CAA4C,CAAC,CAAC;QAClF,CAAC;IACH,CAAC;IAED,MAAM,GAAG,GAAG,GAAG,CAAC,OAAO,CAAC,gBAAgB,CAAC,CAAC;IAC1C,IAAI,OAAO,GAAG,KAAK,QAAQ,IAAI,GAAG,EAAE,CAAC;QACnC,MAAM,CAAC,GAAG,QAAQ,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;QAC5B,IAAI,CAAC,CAAC;YAAE,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,6CAA6C,CAAC,CAAC;QACzF,IAAI,CAAC,eAAe,CAAC,CAAC,CAAC,OAAO,EAAE,OAAO,CAAC,EAAE,CAAC;YACzC,GAAG,CAAC,WAAW,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,4CAA4C,CAAC,CAAC;YAC5E,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,wDAAwD,CAAC,CAAC;QAC9F,CAAC;QACD,CAAC,CAAC,QAAQ,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;QACxB,MAAM,CAAC,CAAC,SAAS,CAAC,aAAa,CAAC,GAAG,EAAE,GAAG,EAAE,IAAI,CAAC,CAAC;QAChD,OAAO;IACT,CAAC;IAED,IAAI,GAAG,CAAC,MAAM,KAAK,MAAM,IAAI,CAAC,mBAAmB,CAAC,IAAI,CAAC,EAAE,CAAC;QACxD,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,sEAAsE,CAAC,CAAC;IAC5G,CAAC;IAED,0EAA0E;IAC1E,yEAAyE;IACzE,MAAM,KAAK,GAAG,MAAM,QAAQ,CAAC,GAAG,CAAC,CAAC;IAClC,IAAI,CAAC,KAAK,CAAC,EAAE,EAAE,CAAC;QACd,OAAO,KAAK,CAAC,MAAM,KAAK,GAAG;YACzB,CAAC,CAAC,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,KAAK,CAAC,OAAO,EAAE,EAAE,kBAAkB,EAAE,6BAA6B,EAAE,CAAC;YAClG,CAAC,CAAC,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,KAAK,CAAC,OAAO,EAAE,EAAE,aAAa,EAAE,GAAG,EAAE,CAAC,CAAC;IACxE,CAAC;IAED,MAAM,MAAM,GAAG,WAAW,CAAC,IAAI,YAAY,CAAC,EAAE,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,QAAQ,EAAE,aAAa,EAAE,eAAe,EAAE,CAAC,CAAC,CAAC;IACjH,MAAM,SAAS,GAAkC,IAAI,6BAA6B,CAAC;QACjF,kBAAkB,EAAE,GAAG,EAAE,CAAC,UAAU,EAAE;QACtC,oBAAoB,EAAE,CAAC,EAAE,EAAE,EAAE;YAC3B,QAAQ,CAAC,GAAG,CAAC,EAAE,EAAE,EAAE,SAAS,EAAE,MAAM,EAAE,OAAO,EAAE,QAAQ,EAAE,IAAI,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;YACvE,GAAG,CAAC,WAAW,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,YAAY,QAAQ,CAAC,IAAI,OAAO,CAAC,CAAC;QACjE,CAAC;KACF,CAAC,CAAC;IACH,SAAS,CAAC,OAAO,GAAG,GAAG,EAAE;QACvB,IAAI,SAAS,CAAC,SAAS;YAAE,KAAK,YAAY,CAAC,SAAS,CAAC,SAAS,EAAE,QAAQ,CAAC,CAAC;IAC5E,CAAC,CAAC;IACF,MAAM,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IAChC,MAAM,SAAS,CAAC,aAAa,CAAC,GAAG,EAAE,GAAG,EAAE,IAAI,CAAC,CAAC;AAChD,CAAC;AAED,4EAA4E;AAC5E,2EAA2E;AAC3E,uCAAuC;AACvC,IAAI,aAA4B,CAAC;AACjC,IAAI,CAAC;IACH,aAAa,GAAG,oBAAoB,EAAE,CAAC;AACzC,CAAC;AAAC,OAAO,GAAG,EAAE,CAAC;IACb,MAAM,GAAG,GAAG,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;IAC7D,IAAI,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,wBAAwB,CAAC,EAAE,CAAC;QACnD,GAAG,CAAC,mCAAmC,GAAG,EAAE,CAAC,CAAC;QAC9C,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACnB,CAAC;IACD,GAAG,CAAC,yDAAyD,GAAG,EAAE,CAAC,CAAC;IACpE,aAAa,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE,QAAQ,EAAE,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,gBAAgB,EAAE,EAAE,EAAE,OAAO,EAAE,KAAK,EAAE,aAAa,EAAE,EAAE,EAAE,CAAC;AACxH,CAAC;AACD,IAAI,aAAa,CAAC,OAAO,EAAE,CAAC;IAC1B,GAAG,CAAC,yCAAyC,aAAa,CAAC,QAAQ,YAAY,aAAa,CAAC,MAAM,EAAE;QACnG,GAAG,aAAa,CAAC,OAAO,CAAC,CAAC,CAAC,6CAA6C,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;IACnF,IAAI,CAAC,aAAa,CAAC,gBAAgB;QAAE,GAAG,CAAC,wFAAwF,CAAC,CAAC;AACrI,CAAC;AACD,MAAM,OAAO,GAAG,aAAa,CAAC,aAAa,EAAE,EAAE,OAAO,EAAE,QAAQ,EAAE,aAAa,EAAE,eAAe,EAAE,GAAG,EAAE,CAAC,CAAC;AAEzG,gFAAgF;AAChF,sDAAsD;AACtD,IAAI,UAAuC,CAAC;AAE5C,KAAK,UAAU,UAAU,CAAC,GAAoB,EAAE,GAAmB;IACjE,IAAI,CAAC;QACH,UAAU,KAAK,eAAe,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC;QAChE,MAAM,IAAI,GAAG,MAAM,UAAU,CAAC;QAC9B,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE;YACjB,cAAc,EAAE,iCAAiC;YACjD,eAAe,EAAE,sBAAsB;YACvC,6BAA6B,EAAE,GAAG;SACnC,CAAC,CAAC;QACH,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,MAAM,KAAK,MAAM,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;IACpD,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,UAAU,GAAG,SAAS,CAAC;QACvB,GAAG,CAAC,qBAAqB,EAAE,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC;QAC7E,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,gBAAgB,CAAC,CAAC;IAC/C,CAAC;AACH,CAAC;AAED,MAAM,IAAI,GAAG,YAAY,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,EAAE;IACrC,MAAM,IAAI,GAAG,CAAC,GAAG,CAAC,GAAG,IAAI,GAAG,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC;IAC5C,IAAI,IAAI,KAAK,mCAAmC,EAAE,CAAC;QACjD,IAAI,GAAG,CAAC,MAAM,KAAK,KAAK,IAAI,GAAG,CAAC,MAAM,KAAK,MAAM;YAAE,KAAK,UAAU,CAAC,GAAG,EAAE,GAAG,CAAC,CAAC;;YACxE,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,oBAAoB,EAAE,EAAE,KAAK,EAAE,WAAW,EAAE,CAAC,CAAC;QAC9E,OAAO;IACT,CAAC;IACD,IAAI,IAAI,KAAK,UAAU,IAAI,CAAC,GAAG,CAAC,MAAM,KAAK,KAAK,IAAI,GAAG,CAAC,MAAM,KAAK,MAAM,CAAC,EAAE,CAAC;QAC3E,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAC;QAC3D,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,MAAM,KAAK,MAAM,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,MAAM,EAAE,IAAI,EAAE,QAAQ,EAAE,QAAQ,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;QACvG,OAAO;IACT,CAAC;IACD,IAAI,OAAO,CAAC,GAAG,EAAE,GAAG,EAAE,IAAI,CAAC;QAAE,OAAO;IACpC,IAAI,IAAI,KAAK,MAAM;QAAE,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,qCAAqC,CAAC,CAAC;IAC9F,IAAI,CAAC,CAAC,KAAK,EAAE,MAAM,EAAE,QAAQ,CAAC,CAAC,QAAQ,CAAC,GAAG,CAAC,MAAM,IAAI,EAAE,CAAC,EAAE,CAAC;QAC1D,OAAO,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,oBAAoB,EAAE,EAAE,KAAK,EAAE,mBAAmB,EAAE,CAAC,CAAC;IAC1F,CAAC;IACD,SAAS,CAAC,GAAG,EAAE,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,GAAG,EAAE,EAAE;QAChC,GAAG,CAAC,iBAAiB,EAAE,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC;QACzE,QAAQ,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,KAAK,EAAE,gBAAgB,CAAC,CAAC;IAC/C,CAAC,CAAC,CAAC;AACL,CAAC,CAAC,CAAC;AAEH,4EAA4E;AAC5E,uCAAuC;AACvC,WAAW,CAAC,GAAG,EAAE;IACf,MAAM,MAAM,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,OAAO,CAAC;IACpC,KAAK,MAAM,CAAC,EAAE,EAAE,CAAC,CAAC,IAAI,QAAQ;QAAE,IAAI,CAAC,CAAC,QAAQ,GAAG,MAAM;YAAE,KAAK,YAAY,CAAC,EAAE,EAAE,MAAM,CAAC,CAAC;AACzF,CAAC,EAAE,MAAM,CAAC,CAAC,KAAK,EAAE,CAAC;AAEnB,MAAM,CAAC,GAAG,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC;AAChC,MAAM,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,WAAW,CAAC;AAC5E,MAAM,IAAI,GAAG,MAAM,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;AACvD,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,IAAI,IAAI,GAAG,KAAK,EAAE,CAAC;IACzD,GAAG,CAAC,8BAA8B,IAAI,oBAAoB,CAAC,CAAC;IAC5D,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;AACnB,CAAC;AAED,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE,IAAI,EAAE,GAAG,EAAE,CAAC,GAAG,CAAC,yCAAyC,IAAI,IAAI,IAAI,WAAW,QAAQ,EAAE,CAAC,CAAC,CAAC;AAC/G,IAAI,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC,GAAG,EAAE,EAAE;IACvB,GAAG,CAAC,2BAA2B,EAAE,GAAG,CAAC,OAAO,CAAC,CAAC;IAC9C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC;AAEH,MAAM,QAAQ,GAAG,GAAG,EAAE;IACpB,GAAG,CAAC,eAAe,CAAC,CAAC;IACrB,IAAI,CAAC,KAAK,EAAE,CAAC;IACb,KAAK,MAAM,EAAE,IAAI,CAAC,GAAG,QAAQ,CAAC,IAAI,EAAE,CAAC;QAAE,KAAK,YAAY,CAAC,EAAE,EAAE,UAAU,CAAC,CAAC;IACzE,UAAU,CAAC,GAAG,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,CAAC,CAAC,KAAK,EAAE,CAAC;AAClD,CAAC,CAAC;AACF,OAAO,CAAC,EAAE,CAAC,SAAS,EAAE,QAAQ,CAAC,CAAC;AAChC,OAAO,CAAC,EAAE,CAAC,QAAQ,EAAE,QAAQ,CAAC,CAAC"}
|
package/dist/httputil.d.ts
CHANGED
|
@@ -9,3 +9,10 @@ export declare function readJSON(req: IncomingMessage): Promise<unknown>;
|
|
|
9
9
|
export declare function bearerToken(req: IncomingMessage): string | null;
|
|
10
10
|
/** Spicrawl API keys (`spicrawl_live_…`, `spicrawl_test_…`). */
|
|
11
11
|
export declare const KEY_RE: RegExp;
|
|
12
|
+
/**
|
|
13
|
+
* The API key a caller of /mcp sent, or null. Tried in order, the first that has a
|
|
14
|
+
* key's shape wins: `Authorization: Bearer <key>`, `Authorization: <key>` (the raw
|
|
15
|
+
* value a gateway such as Smithery sends, with no scheme), `x-api-key`, `api-key`.
|
|
16
|
+
* Headers only: a key in a query string would land in access logs.
|
|
17
|
+
*/
|
|
18
|
+
export declare function apiKeyFromRequest(req: IncomingMessage): string | null;
|
package/dist/httputil.js
CHANGED
|
@@ -30,4 +30,15 @@ export function bearerToken(req) {
|
|
|
30
30
|
}
|
|
31
31
|
/** Spicrawl API keys (`spicrawl_live_…`, `spicrawl_test_…`). */
|
|
32
32
|
export const KEY_RE = /^spicrawl_(live|test)_[A-Za-z0-9_-]{8,}$/;
|
|
33
|
+
/**
|
|
34
|
+
* The API key a caller of /mcp sent, or null. Tried in order, the first that has a
|
|
35
|
+
* key's shape wins: `Authorization: Bearer <key>`, `Authorization: <key>` (the raw
|
|
36
|
+
* value a gateway such as Smithery sends, with no scheme), `x-api-key`, `api-key`.
|
|
37
|
+
* Headers only: a key in a query string would land in access logs.
|
|
38
|
+
*/
|
|
39
|
+
export function apiKeyFromRequest(req) {
|
|
40
|
+
const h = req.headers;
|
|
41
|
+
const candidates = [bearerToken(req), h.authorization?.trim(), h["x-api-key"], h["api-key"]];
|
|
42
|
+
return candidates.find((c) => typeof c === "string" && KEY_RE.test(c)) ?? null;
|
|
43
|
+
}
|
|
33
44
|
//# sourceMappingURL=httputil.js.map
|
package/dist/httputil.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"httputil.js","sourceRoot":"","sources":["../src/httputil.ts"],"names":[],"mappings":"AAEA,2DAA2D;AAC3D,MAAM,CAAC,MAAM,QAAQ,GAAG,CAAC,GAAG,IAAI,GAAG,IAAI,CAAC;AAExC,oFAAoF;AACpF,MAAM,UAAU,QAAQ,CAAC,GAAmB,EAAE,MAAc,EAAE,IAAY,EAAE,OAAe,EAAE,QAAgC,EAAE;IAC7H,IAAI,GAAG,CAAC,WAAW;QAAE,OAAO;IAC5B,GAAG,CAAC,SAAS,CAAC,MAAM,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,GAAG,KAAK,EAAE,CAAC,CAAC;IACxE,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,OAAO,EAAE,EAAE,EAAE,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;AAClF,CAAC;AAED,0FAA0F;AAC1F,MAAM,CAAC,KAAK,UAAU,QAAQ,CAAC,GAAoB;IACjD,MAAM,MAAM,GAAa,EAAE,CAAC;IAC5B,IAAI,IAAI,GAAG,CAAC,CAAC;IACb,IAAI,KAAK,EAAE,MAAM,CAAC,IAAI,GAAG,EAAE,CAAC;QAC1B,IAAI,IAAK,CAAY,CAAC,MAAM,CAAC;QAC7B,IAAI,IAAI,GAAG,QAAQ;YAAE,MAAM,IAAI,KAAK,CAAC,gBAAgB,CAAC,CAAC;QACvD,MAAM,CAAC,IAAI,CAAC,CAAW,CAAC,CAAC;IAC3B,CAAC;IACD,MAAM,GAAG,GAAG,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC;IACnD,OAAO,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;AAC3C,CAAC;AAED,oGAAoG;AACpG,MAAM,UAAU,WAAW,CAAC,GAAoB;IAC9C,MAAM,CAAC,GAAG,GAAG,CAAC,OAAO,CAAC,aAAa,CAAC;IACpC,IAAI,CAAC,CAAC;QAAE,OAAO,IAAI,CAAC;IACpB,MAAM,CAAC,GAAG,sBAAsB,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IACzC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;AACzB,CAAC;AAED,gEAAgE;AAChE,MAAM,CAAC,MAAM,MAAM,GAAG,0CAA0C,CAAC"}
|
|
1
|
+
{"version":3,"file":"httputil.js","sourceRoot":"","sources":["../src/httputil.ts"],"names":[],"mappings":"AAEA,2DAA2D;AAC3D,MAAM,CAAC,MAAM,QAAQ,GAAG,CAAC,GAAG,IAAI,GAAG,IAAI,CAAC;AAExC,oFAAoF;AACpF,MAAM,UAAU,QAAQ,CAAC,GAAmB,EAAE,MAAc,EAAE,IAAY,EAAE,OAAe,EAAE,QAAgC,EAAE;IAC7H,IAAI,GAAG,CAAC,WAAW;QAAE,OAAO;IAC5B,GAAG,CAAC,SAAS,CAAC,MAAM,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,GAAG,KAAK,EAAE,CAAC,CAAC;IACxE,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,OAAO,EAAE,EAAE,EAAE,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;AAClF,CAAC;AAED,0FAA0F;AAC1F,MAAM,CAAC,KAAK,UAAU,QAAQ,CAAC,GAAoB;IACjD,MAAM,MAAM,GAAa,EAAE,CAAC;IAC5B,IAAI,IAAI,GAAG,CAAC,CAAC;IACb,IAAI,KAAK,EAAE,MAAM,CAAC,IAAI,GAAG,EAAE,CAAC;QAC1B,IAAI,IAAK,CAAY,CAAC,MAAM,CAAC;QAC7B,IAAI,IAAI,GAAG,QAAQ;YAAE,MAAM,IAAI,KAAK,CAAC,gBAAgB,CAAC,CAAC;QACvD,MAAM,CAAC,IAAI,CAAC,CAAW,CAAC,CAAC;IAC3B,CAAC;IACD,MAAM,GAAG,GAAG,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC;IACnD,OAAO,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;AAC3C,CAAC;AAED,oGAAoG;AACpG,MAAM,UAAU,WAAW,CAAC,GAAoB;IAC9C,MAAM,CAAC,GAAG,GAAG,CAAC,OAAO,CAAC,aAAa,CAAC;IACpC,IAAI,CAAC,CAAC;QAAE,OAAO,IAAI,CAAC;IACpB,MAAM,CAAC,GAAG,sBAAsB,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IACzC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;AACzB,CAAC;AAED,gEAAgE;AAChE,MAAM,CAAC,MAAM,MAAM,GAAG,0CAA0C,CAAC;AAEjE;;;;;GAKG;AACH,MAAM,UAAU,iBAAiB,CAAC,GAAoB;IACpD,MAAM,CAAC,GAAG,GAAG,CAAC,OAAO,CAAC;IACtB,MAAM,UAAU,GAAG,CAAC,WAAW,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,aAAa,EAAE,IAAI,EAAE,EAAE,CAAC,CAAC,WAAW,CAAC,EAAE,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC;IAC7F,OAAO,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC,EAAe,EAAE,CAAC,OAAO,CAAC,KAAK,QAAQ,IAAI,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC;AAC9F,CAAC"}
|
package/dist/tools/batch.js
CHANGED
|
@@ -46,13 +46,6 @@ const jobId = z.string().min(1).describe("The batch job id returned by spicrawl_
|
|
|
46
46
|
// every item of the job (an item in `items` may override any of them).
|
|
47
47
|
const sharedParams = {
|
|
48
48
|
render: z.boolean().optional().describe("Render every URL with a browser (`js_render`)."),
|
|
49
|
-
// Batch can pin only these three: its worker picks the render engine from
|
|
50
|
-
// capability flags and has no way to name chromium, so the API refuses a
|
|
51
|
-
// chromium pin (use `render: true`, or spicrawl_scrape to pin chromium).
|
|
52
|
-
engine: z
|
|
53
|
-
.enum(["fetch", "obscura", "camoufox"])
|
|
54
|
-
.optional()
|
|
55
|
-
.describe("Pin the scrape engine for every item: `fetch` or `obscura` (must be allowed by the plan and served by the deployment); `camoufox` is coming soon, do not pin it yet. `chromium` cannot be pinned in a batch — use `render: true`, or spicrawl_scrape."),
|
|
56
49
|
format: z
|
|
57
50
|
.enum(["markdown", "text", "html", "json"])
|
|
58
51
|
.optional()
|
|
@@ -82,8 +75,6 @@ function sharedBody(input) {
|
|
|
82
75
|
const b = {};
|
|
83
76
|
if (input.render !== undefined)
|
|
84
77
|
b.js_render = input.render;
|
|
85
|
-
if (input.engine !== undefined)
|
|
86
|
-
b.engine = input.engine;
|
|
87
78
|
if (input.format !== undefined)
|
|
88
79
|
b.response_format = input.format;
|
|
89
80
|
if (input.main_content_only !== undefined)
|
package/dist/tools/batch.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"batch.js","sourceRoot":"","sources":["../../src/tools/batch.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAExB,OAAO,EAAE,WAAW,EAAE,aAAa,EAAE,GAAG,EAAsB,MAAM,aAAa,CAAC;AAElF,MAAM,CAAC,MAAM,OAAO,GAAG,CAAC,EAAU,EAAE,EAAE,CAAC,aAAa,kBAAkB,CAAC,EAAE,CAAC,EAAE,CAAC;AAE7E,oEAAoE;AACpE,SAAS,SAAS,CAAC,IAAY,EAAE,CAAwD;IACvF,MAAM,MAAM,GAAG,IAAI,eAAe,EAAE,CAAC;IACrC,KAAK,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC;QAAE,IAAI,CAAC,KAAK,SAAS;YAAE,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC;IACtF,MAAM,CAAC,GAAG,MAAM,CAAC,QAAQ,EAAE,CAAC;IAC5B,OAAO,CAAC,CAAC,CAAC,CAAC,GAAG,IAAI,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;AACnC,CAAC;AAED;;;GAGG;AACH,MAAM,CAAC,KAAK,UAAU,YAAY,CAChC,MAAoB,EACpB,MAAc,EACd,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAwD;IAE/E,MAAM,GAAG,GAAG,MAAM,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,SAAS,CAAC,GAAG,OAAO,CAAC,MAAM,CAAC,UAAU,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC;IAC5G,yFAAyF;IACzF,uDAAuD;IACvD,IAAI,OAAkB,CAAC;IACvB,IAAI,GAAG,IAAI,OAAO,GAAG,KAAK,QAAQ,IAAI,OAAQ,GAA6B,CAAC,OAAO,KAAK,QAAQ,EAAE,CAAC;QACjG,OAAO,GAAI,GAA2B,CAAC,OAAO;aAC3C,KAAK,CAAC,IAAI,CAAC;aACX,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,KAAK,EAAE,CAAC;aAC9B,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAY,CAAC,CAAC;IAC1C,CAAC;SAAM,IAAI,GAAG,IAAI,OAAO,GAAG,KAAK,QAAQ,IAAI,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACzE,OAAO,GAAG,CAAC,GAAG,CAAC,CAAC;IAClB,CAAC;SAAM,CAAC;QACN,OAAO,GAAG,EAAE,CAAC;IACf,CAAC;IACD,MAAM,GAAG,GAA4B,EAAE,OAAO,EAAE,CAAC;IACjD,oFAAoF;IACpF,kFAAkF;IAClF,MAAM,QAAQ,GAAG,KAAK,IAAI,GAAG,CAAC;IAC9B,MAAM,IAAI,GAAG,OAAO,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,CAAiC,CAAC;IACzE,IAAI,OAAO,CAAC,MAAM,KAAK,QAAQ,IAAI,IAAI,IAAI,OAAO,IAAI,CAAC,GAAG,KAAK,QAAQ,EAAE,CAAC;QACxE,GAAG,CAAC,WAAW,GAAG,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC;IAClE,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,MAAM,KAAK,GAAG,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,4EAA4E,CAAC,CAAC;AAEvH,gFAAgF;AAChF,uEAAuE;AACvE,MAAM,YAAY,GAAG;IACnB,MAAM,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,gDAAgD,CAAC;IACzF,0EAA0E;IAC1E,yEAAyE;IACzE,yEAAyE;IACzE,MAAM,EAAE,CAAC;SACN,IAAI,CAAC,CAAC,OAAO,EAAE,SAAS,EAAE,UAAU,CAAC,CAAC;SACtC,QAAQ,EAAE;SACV,QAAQ,CACP,uPAAuP,CACxP;IACH,MAAM,EAAE,CAAC;SACN,IAAI,CAAC,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;SAC1C,QAAQ,EAAE;SACV,QAAQ,CACP,4KAA4K,CAC7K;IACH,iBAAiB,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,sCAAsC,CAAC;IAC1F,aAAa,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,WAAW,GAAG,mDAAmD,CAAC;IACjH,aAAa,EAAE,CAAC;SACb,MAAM,EAAE;SACR,MAAM,CAAC,CAAC,CAAC;SACT,QAAQ,EAAE;SACV,QAAQ,CAAC,WAAW,GAAG,wDAAwD,CAAC;IACnF,IAAI,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,qDAAqD,CAAC;IACxG,QAAQ,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,+DAA+D,CAAC;IACzG,eAAe,EAAE,CAAC;SACf,KAAK,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC;SACjB,QAAQ,EAAE;SACV,QAAQ,CAAC,+EAA+E,CAAC;IAC5F,cAAc,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,4CAA4C,CAAC;IACtG,QAAQ,EAAE,CAAC;SACR,MAAM,EAAE;SACR,GAAG,EAAE;SACL,GAAG,CAAC,CAAC,CAAC;SACN,QAAQ,EAAE;SACV,QAAQ,CAAC,0EAA0E,CAAC;CACxF,CAAC;AAgBF,SAAS,UAAU,CAAC,KAAkB;IACpC,MAAM,CAAC,GAA4B,EAAE,CAAC;IACtC,IAAI,KAAK,CAAC,MAAM,KAAK,SAAS;QAAE,CAAC,CAAC,SAAS,GAAG,KAAK,CAAC,MAAM,CAAC;IAC3D,IAAI,KAAK,CAAC,MAAM,KAAK,SAAS;QAAE,CAAC,CAAC,MAAM,GAAG,KAAK,CAAC,MAAM,CAAC;IACxD,IAAI,KAAK,CAAC,MAAM,KAAK,SAAS;QAAE,CAAC,CAAC,eAAe,GAAG,KAAK,CAAC,MAAM,CAAC;IACjE,IAAI,KAAK,CAAC,iBAAiB,KAAK,SAAS;QAAE,CAAC,CAAC,iBAAiB,GAAG,KAAK,CAAC,iBAAiB,CAAC;IACzF,IAAI,KAAK,CAAC,aAAa,KAAK,SAAS;QAAE,CAAC,CAAC,aAAa,GAAG,KAAK,CAAC,aAAa,CAAC;IAC7E,IAAI,KAAK,CAAC,aAAa,KAAK,SAAS;QAAE,CAAC,CAAC,aAAa,GAAG,KAAK,CAAC,aAAa,CAAC;IAC7E,IAAI,KAAK,CAAC,IAAI,KAAK,SAAS;QAAE,CAAC,CAAC,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC;IAClD,IAAI,KAAK,CAAC,QAAQ,KAAK,SAAS;QAAE,CAAC,CAAC,QAAQ,GAAG,KAAK,CAAC,QAAQ,CAAC;IAC9D,IAAI,KAAK,CAAC,eAAe,KAAK,SAAS;QAAE,CAAC,CAAC,eAAe,GAAG,KAAK,CAAC,eAAe,CAAC;IACnF,IAAI,KAAK,CAAC,cAAc,KAAK,SAAS;QAAE,CAAC,CAAC,cAAc,GAAG,KAAK,CAAC,cAAc,CAAC;IAChF,IAAI,KAAK,CAAC,QAAQ,KAAK,SAAS;QAAE,CAAC,CAAC,QAAQ,GAAG,KAAK,CAAC,QAAQ,CAAC;IAC9D,OAAO,CAAC,CAAC;AACX,CAAC;AAED,MAAM,UAAU,GAAG;IACjB,IAAI,EAAE,CAAC;SACJ,KAAK,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC;SACvB,QAAQ,EAAE;SACV,QAAQ,CAAC,8GAA8G,CAAC;IAC3H,KAAK,EAAE,CAAC;SACL,KAAK,CACJ,CAAC;SACE,MAAM,CAAC;QACN,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,QAAQ,CAAC,uBAAuB,CAAC;QACvD,WAAW,EAAE,CAAC;aACX,MAAM,EAAE;aACR,QAAQ,EAAE;aACV,QAAQ,CAAC,oEAAoE,CAAC;KAClF,CAAC;SACD,WAAW,EAAE,CACjB;SACA,QAAQ,EAAE;SACV,QAAQ,CACP,qGAAqG;QACnG,0IAA0I,CAC7I;CACJ,CAAC;AAEF,MAAM,CAAC,MAAM,kBAAkB,GAAkB,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE,eAAe,EAAE,EAAE,EAAE;IACvF,MAAM,MAAM,GAAG,aAAa,CAAC,YAAY,EAAE,eAAe,CAAC,CAAC;IAE5D,4EAA4E;IAC5E,yCAAyC;IACzC,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,uBAAuB,EACvB;QACE,KAAK,EAAE,2BAA2B;QAClC,WAAW,EACT,qGAAqG;YACrG,4GAA4G;YAC5G,mHAAmH;YACnH,qHAAqH;YACrH,kHAAkH;YAClH,iDAAiD;QACnD,WAAW,EAAE;YACX,GAAG,UAAU;YACb,GAAG,MAAM;YACT,IAAI,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,4BAA4B,CAAC;YAClE,IAAI,EAAE,CAAC;iBACJ,OAAO,EAAE;iBACT,QAAQ,EAAE;iBACV,QAAQ,CAAC,yHAAyH,CAAC;YACtI,WAAW,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,uFAAuF,CAAC;YACjJ,QAAQ,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,6DAA6D,CAAC;YAC7G,YAAY,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,qCAAqC,CAAC;YACxG,iBAAiB,EAAE,CAAC;iBACjB,MAAM,EAAE;iBACR,GAAG,EAAE;iBACL,GAAG,CAAC,CAAC,CAAC;iBACN,QAAQ,EAAE;iBACV,QAAQ,CAAC,iDAAiD,CAAC;YAC9D,aAAa,EAAE,CAAC;iBACb,MAAM,EAAE;iBACR,GAAG,EAAE;iBACL,GAAG,CAAC,CAAC,CAAC;iBACN,QAAQ,EAAE;iBACV,QAAQ,CAAC,iFAAiF,CAAC;YAC9F,mBAAmB,EAAE,CAAC;iBACnB,MAAM,EAAE;iBACR,QAAQ,EAAE;iBACV,QAAQ,CAAC,4FAA4F,CAAC;SAC1G;QACD,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,KAAK,EAAE,aAAa,EAAE,IAAI,EAAE;KACzG,EACD,KAAK,EAAE,KAAK,EAAE,EAAE;QACd,MAAM,IAAI,GAA4B,UAAU,CAAC,KAAK,CAAC,CAAC;QACxD,sEAAsE;QACtE,uEAAuE;QACvE,IAAI,IAAI,CAAC,eAAe,KAAK,SAAS;YAAE,IAAI,CAAC,eAAe,GAAG,UAAU,CAAC;QAC1E,IAAI,KAAK,CAAC,IAAI;YAAE,IAAI,CAAC,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC;QACvC,IAAI,KAAK,CAAC,KAAK;YAAE,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC;QAC1C,IAAI,KAAK,CAAC,IAAI,KAAK,SAAS;YAAE,IAAI,CAAC,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC;QACrD,IAAI,KAAK,CAAC,IAAI;YAAE,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjC,IAAI,KAAK,CAAC,WAAW,KAAK,SAAS;YAAE,IAAI,CAAC,WAAW,GAAG,KAAK,CAAC,WAAW,CAAC;QAC1E,IAAI,KAAK,CAAC,QAAQ,KAAK,SAAS;YAAE,IAAI,CAAC,QAAQ,GAAG,KAAK,CAAC,QAAQ,CAAC;QACjE,IAAI,KAAK,CAAC,YAAY,KAAK,SAAS;YAAE,IAAI,CAAC,YAAY,GAAG,KAAK,CAAC,YAAY,CAAC;QAC7E,IAAI,KAAK,CAAC,iBAAiB,KAAK,SAAS;YAAE,IAAI,CAAC,iBAAiB,GAAG,KAAK,CAAC,iBAAiB,CAAC;QAC5F,IAAI,KAAK,CAAC,aAAa,KAAK,SAAS;YAAE,IAAI,CAAC,aAAa,GAAG,KAAK,CAAC,aAAa,CAAC;QAChF,IAAI,KAAK,CAAC,mBAAmB,KAAK,SAAS;YAAE,IAAI,CAAC,mBAAmB,GAAG,KAAK,CAAC,mBAAmB,CAAC;QAClG,OAAO,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,WAAW,EAAE,IAAI,CAAC,CAAC,CAAC;IAC9D,CAAC,CACF,CAAC;IAEF,4EAA4E;IAC5E,sCAAsC;IACtC,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,qBAAqB,EACrB;QACE,KAAK,EAAE,iBAAiB;QACxB,WAAW,EACT,oFAAoF;YACpF,oGAAoG;QACtG,WAAW,EAAE;YACX,MAAM,EAAE,CAAC;iBACN,IAAI,CAAC,CAAC,QAAQ,EAAE,SAAS,EAAE,QAAQ,EAAE,YAAY,EAAE,WAAW,EAAE,QAAQ,EAAE,WAAW,CAAC,CAAC;iBACvF,QAAQ,EAAE;iBACV,QAAQ,CAAC,2BAA2B,CAAC;YACxC,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,kCAAkC,CAAC;YAC/F,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,+CAA+C,CAAC;SACxF;QACD,WAAW,EAAE,EAAE,YAAY,EAAE,IAAI,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,EAAE,EAAE,CAClC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,SAAS,CAAC,WAAW,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,CACtF,CAAC;IAEF,4EAA4E;IAC5E,6CAA6C;IAC7C,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,uBAAuB,EACvB;QACE,KAAK,EAAE,4BAA4B;QACnC,WAAW,EACT,mHAAmH;YACnH,8HAA8H;QAChI,WAAW,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE;QAC9B,WAAW,EAAE,EAAE,YAAY,EAAE,IAAI,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,CACxE,CAAC;IAEF,4EAA4E;IAC5E,mEAAmE;IACnE,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,wBAAwB,EACxB;QACE,KAAK,EAAE,4BAA4B;QACnC,WAAW,EACT,6GAA6G;YAC7G,6JAA6J;YAC7J,6FAA6F;YAC7F,6DAA6D;QAC/D,WAAW,EAAE;YACX,MAAM,EAAE,KAAK;YACb,MAAM,EAAE,CAAC;iBACN,IAAI,CAAC,CAAC,WAAW,EAAE,QAAQ,EAAE,WAAW,EAAE,SAAS,CAAC,CAAC;iBACrD,QAAQ,EAAE;iBACV,QAAQ,CAAC,4CAA4C,CAAC;YACzD,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,oCAAoC,CAAC;YAClG,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,+CAA+C,CAAC;SACxF;QACD,WAAW,EAAE,EAAE,YAAY,EAAE,IAAI,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,YAAY,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,CAAC,CAAC,CAChH,CAAC;IAEF,4EAA4E;IAC5E,uEAAuE;IACvE,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,6BAA6B,EAC7B;QACE,KAAK,EAAE,+BAA+B;QACtC,WAAW,EACT,uGAAuG;YACvG,wGAAwG;YACxG,6GAA6G;YAC7G,uFAAuF;QACzF,WAAW,EAAE;YACX,MAAM,EAAE,KAAK;YACb,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,yDAAyD,CAAC;SACjG;QACD,WAAW,EAAE,EAAE,YAAY,EAAE,IAAI,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,GAAG,EAAE,EAAE,EAAE,CACxB,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC,UAAU,kBAAkB,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,UAAU,CAAC,CAAC,CAC1G,CAAC;IAEF,4EAA4E;IAC5E,qDAAqD;IACrD,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,uBAAuB,EACvB;QACE,KAAK,EAAE,oBAAoB;QAC3B,WAAW,EACT,0GAA0G;YAC1G,qGAAqG;YACrG,yGAAyG;QAC3G,WAAW,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE;QAC9B,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,IAAI,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC,SAAS,CAAC,CAAC,CACrF,CAAC;IAEF,4EAA4E;IAC5E,mDAAmD;IACnD,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,sBAAsB,EACtB;QACE,KAAK,EAAE,kCAAkC;QACzC,WAAW,EACT,2GAA2G;YAC3G,oFAAoF;QACtF,WAAW,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE;QAC9B,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,KAAK,EAAE,aAAa,EAAE,IAAI,EAAE;KACzG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CACpF,CAAC;IAEF,4EAA4E;IAC5E,uDAAuD;IACvD,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,0BAA0B,EAC1B;QACE,KAAK,EAAE,+BAA+B;QACtC,WAAW,EACT,2FAA2F;YAC3F,8HAA8H;YAC9H,uFAAuF;YACvF,mDAAmD;QACrD,WAAW,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE,GAAG,UAAU,EAAE,GAAG,MAAM,EAAE;QACxD,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,KAAK,EAAE,aAAa,EAAE,IAAI,EAAE;KACzG,EACD,KAAK,EAAE,KAAK,EAAE,EAAE;QACd,MAAM,IAAI,GAA4B,UAAU,CAAC,KAAK,CAAC,CAAC;QACxD,sEAAsE;QACtE,uEAAuE;QACvE,IAAI,IAAI,CAAC,eAAe,KAAK,SAAS;YAAE,IAAI,CAAC,eAAe,GAAG,UAAU,CAAC;QAC1E,IAAI,KAAK,CAAC,IAAI;YAAE,IAAI,CAAC,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC;QACvC,IAAI,KAAK,CAAC,KAAK;YAAE,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC;QAC1C,OAAO,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,OAAO,CAAC,KAAK,CAAC,MAAM,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC,CAAC;IACnF,CAAC,CACF,CAAC;IAEF,4EAA4E;IAC5E,mDAAmD;IACnD,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,sBAAsB,EACtB;QACE,KAAK,EAAE,yBAAyB;QAChC,WAAW,EACT,uFAAuF;YACvF,qEAAqE;QACvE,WAAW,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE;QAC9B,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACzG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CACpF,CAAC;AACJ,CAAC,CAAC"}
|
|
1
|
+
{"version":3,"file":"batch.js","sourceRoot":"","sources":["../../src/tools/batch.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAExB,OAAO,EAAE,WAAW,EAAE,aAAa,EAAE,GAAG,EAAsB,MAAM,aAAa,CAAC;AAElF,MAAM,CAAC,MAAM,OAAO,GAAG,CAAC,EAAU,EAAE,EAAE,CAAC,aAAa,kBAAkB,CAAC,EAAE,CAAC,EAAE,CAAC;AAE7E,oEAAoE;AACpE,SAAS,SAAS,CAAC,IAAY,EAAE,CAAwD;IACvF,MAAM,MAAM,GAAG,IAAI,eAAe,EAAE,CAAC;IACrC,KAAK,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC;QAAE,IAAI,CAAC,KAAK,SAAS;YAAE,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC;IACtF,MAAM,CAAC,GAAG,MAAM,CAAC,QAAQ,EAAE,CAAC;IAC5B,OAAO,CAAC,CAAC,CAAC,CAAC,GAAG,IAAI,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;AACnC,CAAC;AAED;;;GAGG;AACH,MAAM,CAAC,KAAK,UAAU,YAAY,CAChC,MAAoB,EACpB,MAAc,EACd,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAwD;IAE/E,MAAM,GAAG,GAAG,MAAM,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,SAAS,CAAC,GAAG,OAAO,CAAC,MAAM,CAAC,UAAU,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC;IAC5G,yFAAyF;IACzF,uDAAuD;IACvD,IAAI,OAAkB,CAAC;IACvB,IAAI,GAAG,IAAI,OAAO,GAAG,KAAK,QAAQ,IAAI,OAAQ,GAA6B,CAAC,OAAO,KAAK,QAAQ,EAAE,CAAC;QACjG,OAAO,GAAI,GAA2B,CAAC,OAAO;aAC3C,KAAK,CAAC,IAAI,CAAC;aACX,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,KAAK,EAAE,CAAC;aAC9B,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAY,CAAC,CAAC;IAC1C,CAAC;SAAM,IAAI,GAAG,IAAI,OAAO,GAAG,KAAK,QAAQ,IAAI,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACzE,OAAO,GAAG,CAAC,GAAG,CAAC,CAAC;IAClB,CAAC;SAAM,CAAC;QACN,OAAO,GAAG,EAAE,CAAC;IACf,CAAC;IACD,MAAM,GAAG,GAA4B,EAAE,OAAO,EAAE,CAAC;IACjD,oFAAoF;IACpF,kFAAkF;IAClF,MAAM,QAAQ,GAAG,KAAK,IAAI,GAAG,CAAC;IAC9B,MAAM,IAAI,GAAG,OAAO,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,CAAiC,CAAC;IACzE,IAAI,OAAO,CAAC,MAAM,KAAK,QAAQ,IAAI,IAAI,IAAI,OAAO,IAAI,CAAC,GAAG,KAAK,QAAQ,EAAE,CAAC;QACxE,GAAG,CAAC,WAAW,GAAG,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC;IAClE,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,MAAM,KAAK,GAAG,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,4EAA4E,CAAC,CAAC;AAEvH,gFAAgF;AAChF,uEAAuE;AACvE,MAAM,YAAY,GAAG;IACnB,MAAM,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,gDAAgD,CAAC;IACzF,MAAM,EAAE,CAAC;SACN,IAAI,CAAC,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;SAC1C,QAAQ,EAAE;SACV,QAAQ,CACP,4KAA4K,CAC7K;IACH,iBAAiB,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,sCAAsC,CAAC;IAC1F,aAAa,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,WAAW,GAAG,mDAAmD,CAAC;IACjH,aAAa,EAAE,CAAC;SACb,MAAM,EAAE;SACR,MAAM,CAAC,CAAC,CAAC;SACT,QAAQ,EAAE;SACV,QAAQ,CAAC,WAAW,GAAG,wDAAwD,CAAC;IACnF,IAAI,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,qDAAqD,CAAC;IACxG,QAAQ,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,+DAA+D,CAAC;IACzG,eAAe,EAAE,CAAC;SACf,KAAK,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC;SACjB,QAAQ,EAAE;SACV,QAAQ,CAAC,+EAA+E,CAAC;IAC5F,cAAc,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,4CAA4C,CAAC;IACtG,QAAQ,EAAE,CAAC;SACR,MAAM,EAAE;SACR,GAAG,EAAE;SACL,GAAG,CAAC,CAAC,CAAC;SACN,QAAQ,EAAE;SACV,QAAQ,CAAC,0EAA0E,CAAC;CACxF,CAAC;AAeF,SAAS,UAAU,CAAC,KAAkB;IACpC,MAAM,CAAC,GAA4B,EAAE,CAAC;IACtC,IAAI,KAAK,CAAC,MAAM,KAAK,SAAS;QAAE,CAAC,CAAC,SAAS,GAAG,KAAK,CAAC,MAAM,CAAC;IAC3D,IAAI,KAAK,CAAC,MAAM,KAAK,SAAS;QAAE,CAAC,CAAC,eAAe,GAAG,KAAK,CAAC,MAAM,CAAC;IACjE,IAAI,KAAK,CAAC,iBAAiB,KAAK,SAAS;QAAE,CAAC,CAAC,iBAAiB,GAAG,KAAK,CAAC,iBAAiB,CAAC;IACzF,IAAI,KAAK,CAAC,aAAa,KAAK,SAAS;QAAE,CAAC,CAAC,aAAa,GAAG,KAAK,CAAC,aAAa,CAAC;IAC7E,IAAI,KAAK,CAAC,aAAa,KAAK,SAAS;QAAE,CAAC,CAAC,aAAa,GAAG,KAAK,CAAC,aAAa,CAAC;IAC7E,IAAI,KAAK,CAAC,IAAI,KAAK,SAAS;QAAE,CAAC,CAAC,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC;IAClD,IAAI,KAAK,CAAC,QAAQ,KAAK,SAAS;QAAE,CAAC,CAAC,QAAQ,GAAG,KAAK,CAAC,QAAQ,CAAC;IAC9D,IAAI,KAAK,CAAC,eAAe,KAAK,SAAS;QAAE,CAAC,CAAC,eAAe,GAAG,KAAK,CAAC,eAAe,CAAC;IACnF,IAAI,KAAK,CAAC,cAAc,KAAK,SAAS;QAAE,CAAC,CAAC,cAAc,GAAG,KAAK,CAAC,cAAc,CAAC;IAChF,IAAI,KAAK,CAAC,QAAQ,KAAK,SAAS;QAAE,CAAC,CAAC,QAAQ,GAAG,KAAK,CAAC,QAAQ,CAAC;IAC9D,OAAO,CAAC,CAAC;AACX,CAAC;AAED,MAAM,UAAU,GAAG;IACjB,IAAI,EAAE,CAAC;SACJ,KAAK,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC;SACvB,QAAQ,EAAE;SACV,QAAQ,CAAC,8GAA8G,CAAC;IAC3H,KAAK,EAAE,CAAC;SACL,KAAK,CACJ,CAAC;SACE,MAAM,CAAC;QACN,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,QAAQ,CAAC,uBAAuB,CAAC;QACvD,WAAW,EAAE,CAAC;aACX,MAAM,EAAE;aACR,QAAQ,EAAE;aACV,QAAQ,CAAC,oEAAoE,CAAC;KAClF,CAAC;SACD,WAAW,EAAE,CACjB;SACA,QAAQ,EAAE;SACV,QAAQ,CACP,qGAAqG;QACnG,0IAA0I,CAC7I;CACJ,CAAC;AAEF,MAAM,CAAC,MAAM,kBAAkB,GAAkB,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE,eAAe,EAAE,EAAE,EAAE;IACvF,MAAM,MAAM,GAAG,aAAa,CAAC,YAAY,EAAE,eAAe,CAAC,CAAC;IAE5D,4EAA4E;IAC5E,yCAAyC;IACzC,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,uBAAuB,EACvB;QACE,KAAK,EAAE,2BAA2B;QAClC,WAAW,EACT,qGAAqG;YACrG,4GAA4G;YAC5G,mHAAmH;YACnH,qHAAqH;YACrH,kHAAkH;YAClH,iDAAiD;QACnD,WAAW,EAAE;YACX,GAAG,UAAU;YACb,GAAG,MAAM;YACT,IAAI,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,4BAA4B,CAAC;YAClE,IAAI,EAAE,CAAC;iBACJ,OAAO,EAAE;iBACT,QAAQ,EAAE;iBACV,QAAQ,CAAC,yHAAyH,CAAC;YACtI,WAAW,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,uFAAuF,CAAC;YACjJ,QAAQ,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,6DAA6D,CAAC;YAC7G,YAAY,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,qCAAqC,CAAC;YACxG,iBAAiB,EAAE,CAAC;iBACjB,MAAM,EAAE;iBACR,GAAG,EAAE;iBACL,GAAG,CAAC,CAAC,CAAC;iBACN,QAAQ,EAAE;iBACV,QAAQ,CAAC,iDAAiD,CAAC;YAC9D,aAAa,EAAE,CAAC;iBACb,MAAM,EAAE;iBACR,GAAG,EAAE;iBACL,GAAG,CAAC,CAAC,CAAC;iBACN,QAAQ,EAAE;iBACV,QAAQ,CAAC,iFAAiF,CAAC;YAC9F,mBAAmB,EAAE,CAAC;iBACnB,MAAM,EAAE;iBACR,QAAQ,EAAE;iBACV,QAAQ,CAAC,4FAA4F,CAAC;SAC1G;QACD,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,KAAK,EAAE,aAAa,EAAE,IAAI,EAAE;KACzG,EACD,KAAK,EAAE,KAAK,EAAE,EAAE;QACd,MAAM,IAAI,GAA4B,UAAU,CAAC,KAAK,CAAC,CAAC;QACxD,sEAAsE;QACtE,uEAAuE;QACvE,IAAI,IAAI,CAAC,eAAe,KAAK,SAAS;YAAE,IAAI,CAAC,eAAe,GAAG,UAAU,CAAC;QAC1E,IAAI,KAAK,CAAC,IAAI;YAAE,IAAI,CAAC,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC;QACvC,IAAI,KAAK,CAAC,KAAK;YAAE,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC;QAC1C,IAAI,KAAK,CAAC,IAAI,KAAK,SAAS;YAAE,IAAI,CAAC,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC;QACrD,IAAI,KAAK,CAAC,IAAI;YAAE,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjC,IAAI,KAAK,CAAC,WAAW,KAAK,SAAS;YAAE,IAAI,CAAC,WAAW,GAAG,KAAK,CAAC,WAAW,CAAC;QAC1E,IAAI,KAAK,CAAC,QAAQ,KAAK,SAAS;YAAE,IAAI,CAAC,QAAQ,GAAG,KAAK,CAAC,QAAQ,CAAC;QACjE,IAAI,KAAK,CAAC,YAAY,KAAK,SAAS;YAAE,IAAI,CAAC,YAAY,GAAG,KAAK,CAAC,YAAY,CAAC;QAC7E,IAAI,KAAK,CAAC,iBAAiB,KAAK,SAAS;YAAE,IAAI,CAAC,iBAAiB,GAAG,KAAK,CAAC,iBAAiB,CAAC;QAC5F,IAAI,KAAK,CAAC,aAAa,KAAK,SAAS;YAAE,IAAI,CAAC,aAAa,GAAG,KAAK,CAAC,aAAa,CAAC;QAChF,IAAI,KAAK,CAAC,mBAAmB,KAAK,SAAS;YAAE,IAAI,CAAC,mBAAmB,GAAG,KAAK,CAAC,mBAAmB,CAAC;QAClG,OAAO,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,WAAW,EAAE,IAAI,CAAC,CAAC,CAAC;IAC9D,CAAC,CACF,CAAC;IAEF,4EAA4E;IAC5E,sCAAsC;IACtC,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,qBAAqB,EACrB;QACE,KAAK,EAAE,iBAAiB;QACxB,WAAW,EACT,oFAAoF;YACpF,oGAAoG;QACtG,WAAW,EAAE;YACX,MAAM,EAAE,CAAC;iBACN,IAAI,CAAC,CAAC,QAAQ,EAAE,SAAS,EAAE,QAAQ,EAAE,YAAY,EAAE,WAAW,EAAE,QAAQ,EAAE,WAAW,CAAC,CAAC;iBACvF,QAAQ,EAAE;iBACV,QAAQ,CAAC,2BAA2B,CAAC;YACxC,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,kCAAkC,CAAC;YAC/F,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,+CAA+C,CAAC;SACxF;QACD,WAAW,EAAE,EAAE,YAAY,EAAE,IAAI,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,EAAE,EAAE,CAClC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,SAAS,CAAC,WAAW,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,CACtF,CAAC;IAEF,4EAA4E;IAC5E,6CAA6C;IAC7C,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,uBAAuB,EACvB;QACE,KAAK,EAAE,4BAA4B;QACnC,WAAW,EACT,mHAAmH;YACnH,8HAA8H;QAChI,WAAW,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE;QAC9B,WAAW,EAAE,EAAE,YAAY,EAAE,IAAI,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,CACxE,CAAC;IAEF,4EAA4E;IAC5E,mEAAmE;IACnE,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,wBAAwB,EACxB;QACE,KAAK,EAAE,4BAA4B;QACnC,WAAW,EACT,6GAA6G;YAC7G,6JAA6J;YAC7J,6FAA6F;YAC7F,6DAA6D;QAC/D,WAAW,EAAE;YACX,MAAM,EAAE,KAAK;YACb,MAAM,EAAE,CAAC;iBACN,IAAI,CAAC,CAAC,WAAW,EAAE,QAAQ,EAAE,WAAW,EAAE,SAAS,CAAC,CAAC;iBACrD,QAAQ,EAAE;iBACV,QAAQ,CAAC,4CAA4C,CAAC;YACzD,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,oCAAoC,CAAC;YAClG,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,+CAA+C,CAAC;SACxF;QACD,WAAW,EAAE,EAAE,YAAY,EAAE,IAAI,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,YAAY,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,CAAC,CAAC,CAChH,CAAC;IAEF,4EAA4E;IAC5E,uEAAuE;IACvE,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,6BAA6B,EAC7B;QACE,KAAK,EAAE,+BAA+B;QACtC,WAAW,EACT,uGAAuG;YACvG,wGAAwG;YACxG,6GAA6G;YAC7G,uFAAuF;QACzF,WAAW,EAAE;YACX,MAAM,EAAE,KAAK;YACb,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,yDAAyD,CAAC;SACjG;QACD,WAAW,EAAE,EAAE,YAAY,EAAE,IAAI,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,GAAG,EAAE,EAAE,EAAE,CACxB,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC,UAAU,kBAAkB,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,UAAU,CAAC,CAAC,CAC1G,CAAC;IAEF,4EAA4E;IAC5E,qDAAqD;IACrD,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,uBAAuB,EACvB;QACE,KAAK,EAAE,oBAAoB;QAC3B,WAAW,EACT,0GAA0G;YAC1G,qGAAqG;YACrG,yGAAyG;QAC3G,WAAW,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE;QAC9B,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,IAAI,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC,SAAS,CAAC,CAAC,CACrF,CAAC;IAEF,4EAA4E;IAC5E,mDAAmD;IACnD,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,sBAAsB,EACtB;QACE,KAAK,EAAE,kCAAkC;QACzC,WAAW,EACT,2GAA2G;YAC3G,oFAAoF;QACtF,WAAW,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE;QAC9B,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,KAAK,EAAE,aAAa,EAAE,IAAI,EAAE;KACzG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CACpF,CAAC;IAEF,4EAA4E;IAC5E,uDAAuD;IACvD,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,0BAA0B,EAC1B;QACE,KAAK,EAAE,+BAA+B;QACtC,WAAW,EACT,2FAA2F;YAC3F,8HAA8H;YAC9H,uFAAuF;YACvF,mDAAmD;QACrD,WAAW,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE,GAAG,UAAU,EAAE,GAAG,MAAM,EAAE;QACxD,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,KAAK,EAAE,aAAa,EAAE,IAAI,EAAE;KACzG,EACD,KAAK,EAAE,KAAK,EAAE,EAAE;QACd,MAAM,IAAI,GAA4B,UAAU,CAAC,KAAK,CAAC,CAAC;QACxD,sEAAsE;QACtE,uEAAuE;QACvE,IAAI,IAAI,CAAC,eAAe,KAAK,SAAS;YAAE,IAAI,CAAC,eAAe,GAAG,UAAU,CAAC;QAC1E,IAAI,KAAK,CAAC,IAAI;YAAE,IAAI,CAAC,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC;QACvC,IAAI,KAAK,CAAC,KAAK;YAAE,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC;QAC1C,OAAO,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,OAAO,CAAC,KAAK,CAAC,MAAM,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC,CAAC;IACnF,CAAC,CACF,CAAC;IAEF,4EAA4E;IAC5E,mDAAmD;IACnD,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,sBAAsB,EACtB;QACE,KAAK,EAAE,yBAAyB;QAChC,WAAW,EACT,uFAAuF;YACvF,qEAAqE;QACvE,WAAW,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE;QAC9B,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACzG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CACpF,CAAC;AACJ,CAAC,CAAC"}
|
package/dist/tools/browser.js
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { z } from "zod";
|
|
2
2
|
import { run } from "./common.js";
|
|
3
3
|
// Browser (CDP): GET /v1/browser is a WebSocket; MCP cannot carry it, so this
|
|
4
|
-
// module mints a single-use token URL for the caller's own
|
|
4
|
+
// module mints a single-use token URL for the caller's own CDP client.
|
|
5
5
|
// Remote browsers are coming soon and minting always fails today, so the tool is
|
|
6
6
|
// not registered when the server hides what is unavailable (SPICRAWL_MCP_HIDE_UNAVAILABLE).
|
|
7
7
|
export const registerBrowserTools = (server, client, { hideUnavailable }) => {
|
|
@@ -11,19 +11,19 @@ export const registerBrowserTools = (server, client, { hideUnavailable }) => {
|
|
|
11
11
|
title: "Browser (CDP) connect URL",
|
|
12
12
|
description: "Coming soon: remote browsers are not available yet, so minting fails; do not rely on this tool. For scripted page " +
|
|
13
13
|
"interaction today use `actions` on spicrawl_scrape. " +
|
|
14
|
-
"Mints a single-use connect URL for Spicrawl's remote browser (GET /v1/browser,
|
|
15
|
-
"
|
|
14
|
+
"Mints a single-use connect URL for Spicrawl's remote browser (GET /v1/browser, a DevTools Protocol (CDP) WebSocket) so YOUR code can drive it " +
|
|
15
|
+
"with any CDP client, using the URL as the browser WebSocket endpoint. It calls POST /v1/browser/token and returns a " +
|
|
16
16
|
"ws:// or wss:// URL carrying a short-lived token, never the API key: the URL works ONCE, within 60 seconds, and only with the options " +
|
|
17
17
|
"given here (editing them in the URL is refused), so call this right before connecting and again for every new session. " +
|
|
18
|
-
"Prefer sending the key as an Authorization header when the client supports headers on the upgrade " +
|
|
19
|
-
"
|
|
18
|
+
"Prefer sending the key as an Authorization header when the client supports headers on the WebSocket upgrade: " +
|
|
19
|
+
"then connect to the plain /v1/browser URL " +
|
|
20
20
|
"with 'Authorization: Bearer <key>' and no token is needed. The endpoint exists only when the deployment was installed with --with-cdp, " +
|
|
21
21
|
"and the key must carry the `browser` scope; otherwise minting fails (404 or 403). Parameter errors are reported when minting.",
|
|
22
22
|
inputSchema: {
|
|
23
23
|
engine: z
|
|
24
|
-
.enum(["
|
|
24
|
+
.enum(["browser", "js"])
|
|
25
25
|
.optional()
|
|
26
|
-
.describe("
|
|
26
|
+
.describe("Rendering tier of the remote browser: `browser` (full browser, full protocol; the default) or `js` (lighter JavaScript rendering with a smaller protocol surface). A tier your plan does not include is refused."),
|
|
27
27
|
proxy_country: z
|
|
28
28
|
.string()
|
|
29
29
|
.regex(/^[a-z]{2}$/)
|
|
@@ -45,10 +45,6 @@ export const registerBrowserTools = (server, client, { hideUnavailable }) => {
|
|
|
45
45
|
.max(900)
|
|
46
46
|
.optional()
|
|
47
47
|
.describe("Session lifetime in seconds, 60-900. Default 180."),
|
|
48
|
-
headless: z
|
|
49
|
-
.boolean()
|
|
50
|
-
.optional()
|
|
51
|
-
.describe("false = headful browser with a display (desktop-sized screen). Omit for the deployment default."),
|
|
52
48
|
proxy: z
|
|
53
49
|
.string()
|
|
54
50
|
.optional()
|
|
@@ -60,9 +56,6 @@ export const registerBrowserTools = (server, client, { hideUnavailable }) => {
|
|
|
60
56
|
if (args.proxy_country && args.proxy_region && args.proxy_region.toLowerCase() !== "global") {
|
|
61
57
|
throw new Error("Send either proxy_country or proxy_region, not both: they choose the same thing.");
|
|
62
58
|
}
|
|
63
|
-
if (args.engine === "obscura" && args.headless === false) {
|
|
64
|
-
throw new Error("engine=obscura cannot run headful (headless=false). Use engine=chromium or drop headless.");
|
|
65
|
-
}
|
|
66
59
|
const body = {};
|
|
67
60
|
if (args.engine)
|
|
68
61
|
body.engine = args.engine;
|
|
@@ -74,8 +67,6 @@ export const registerBrowserTools = (server, client, { hideUnavailable }) => {
|
|
|
74
67
|
body.sticky_key = args.sticky_key;
|
|
75
68
|
if (args.session_ttl !== undefined)
|
|
76
69
|
body.session_ttl = args.session_ttl;
|
|
77
|
-
if (args.headless !== undefined)
|
|
78
|
-
body.headless = args.headless;
|
|
79
70
|
if (args.proxy)
|
|
80
71
|
body.proxy = args.proxy;
|
|
81
72
|
const minted = (await client.request("POST", "/v1/browser/token", body));
|
|
@@ -90,11 +81,10 @@ export const registerBrowserTools = (server, client, { hideUnavailable }) => {
|
|
|
90
81
|
const ws = url.toString();
|
|
91
82
|
return {
|
|
92
83
|
url: ws,
|
|
93
|
-
puppeteer: `const browser = await puppeteer.connect({ browserWSEndpoint: ${JSON.stringify(ws)} });`,
|
|
94
|
-
playwright: `const browser = await chromium.connectOverCDP(${JSON.stringify(ws)});`,
|
|
95
84
|
expires_at: minted.expires_at,
|
|
96
85
|
single_use: true,
|
|
97
86
|
notes: [
|
|
87
|
+
"Use `url` as the browser WebSocket endpoint of any CDP client.",
|
|
98
88
|
"The URL carries a single-use token, not your API key. It expires 60 s after minting and works for one connection; mint a new one per session.",
|
|
99
89
|
"If your client can set headers on the WebSocket upgrade, you can instead connect to /v1/browser with 'Authorization: Bearer <key>'.",
|
|
100
90
|
"Requires a deployment installed with --with-cdp and a key with the `browser` scope.",
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"browser.js","sourceRoot":"","sources":["../../src/tools/browser.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAAE,GAAG,EAAsB,MAAM,aAAa,CAAC;AAEtD,8EAA8E;AAC9E,
|
|
1
|
+
{"version":3,"file":"browser.js","sourceRoot":"","sources":["../../src/tools/browser.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAAE,GAAG,EAAsB,MAAM,aAAa,CAAC;AAEtD,8EAA8E;AAC9E,uEAAuE;AACvE,iFAAiF;AACjF,4FAA4F;AAC5F,MAAM,CAAC,MAAM,oBAAoB,GAAkB,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE,eAAe,EAAE,EAAE,EAAE;IACzF,IAAI,eAAe;QAAE,OAAO;IAC5B,MAAM,CAAC,YAAY,CACjB,8BAA8B,EAC9B;QACE,KAAK,EAAE,2BAA2B;QAClC,WAAW,EACT,oHAAoH;YACpH,sDAAsD;YACtD,gJAAgJ;YAChJ,sHAAsH;YACtH,wIAAwI;YACxI,yHAAyH;YACzH,+GAA+G;YAC/G,4CAA4C;YAC5C,yIAAyI;YACzI,+HAA+H;QACjI,WAAW,EAAE;YACX,MAAM,EAAE,CAAC;iBACN,IAAI,CAAC,CAAC,SAAS,EAAE,IAAI,CAAC,CAAC;iBACvB,QAAQ,EAAE;iBACV,QAAQ,CAAC,kNAAkN,CAAC;YAC/N,aAAa,EAAE,CAAC;iBACb,MAAM,EAAE;iBACR,KAAK,CAAC,YAAY,CAAC;iBACnB,QAAQ,EAAE;iBACV,QAAQ,CAAC,6FAA6F,CAAC;YAC1G,YAAY,EAAE,CAAC;iBACZ,MAAM,EAAE;iBACR,QAAQ,EAAE;iBACV,QAAQ,CAAC,qGAAqG,CAAC;YAClH,UAAU,EAAE,CAAC;iBACV,MAAM,EAAE;iBACR,KAAK,CAAC,sBAAsB,CAAC;iBAC7B,QAAQ,EAAE;iBACV,QAAQ,CAAC,gHAAgH,CAAC;YAC7H,WAAW,EAAE,CAAC;iBACX,MAAM,EAAE;iBACR,GAAG,EAAE;iBACL,GAAG,CAAC,EAAE,CAAC;iBACP,GAAG,CAAC,GAAG,CAAC;iBACR,QAAQ,EAAE;iBACV,QAAQ,CAAC,mDAAmD,CAAC;YAChE,KAAK,EAAE,CAAC;iBACL,MAAM,EAAE;iBACR,QAAQ,EAAE;iBACV,QAAQ,CAAC,2KAA2K,CAAC;SACzL;QACD,6EAA6E;QAC7E,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,KAAK,EAAE,aAAa,EAAE,IAAI,EAAE;KACzG,EACD,KAAK,EAAE,IAAI,EAAE,EAAE,CACb,GAAG,CAAC,KAAK,IAAI,EAAE;QACb,IAAI,IAAI,CAAC,aAAa,IAAI,IAAI,CAAC,YAAY,IAAI,IAAI,CAAC,YAAY,CAAC,WAAW,EAAE,KAAK,QAAQ,EAAE,CAAC;YAC5F,MAAM,IAAI,KAAK,CAAC,kFAAkF,CAAC,CAAC;QACtG,CAAC;QACD,MAAM,IAAI,GAA4B,EAAE,CAAC;QACzC,IAAI,IAAI,CAAC,MAAM;YAAE,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC;QAC3C,IAAI,IAAI,CAAC,aAAa;YAAE,IAAI,CAAC,aAAa,GAAG,IAAI,CAAC,aAAa,CAAC;QAChE,IAAI,IAAI,CAAC,YAAY;YAAE,IAAI,CAAC,YAAY,GAAG,IAAI,CAAC,YAAY,CAAC;QAC7D,IAAI,IAAI,CAAC,UAAU;YAAE,IAAI,CAAC,UAAU,GAAG,IAAI,CAAC,UAAU,CAAC;QACvD,IAAI,IAAI,CAAC,WAAW,KAAK,SAAS;YAAE,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC,WAAW,CAAC;QACxE,IAAI,IAAI,CAAC,KAAK;YAAE,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC;QACxC,MAAM,MAAM,GAAG,CAAC,MAAM,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,mBAAmB,EAAE,IAAI,CAAC,CAGtE,CAAC;QACF,IAAI,CAAC,MAAM,EAAE,IAAI,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,UAAU,CAAC,cAAc,CAAC,EAAE,CAAC;YAC7D,MAAM,IAAI,KAAK,CAAC,8CAA8C,CAAC,CAAC;QAClE,CAAC;QACD,sEAAsE;QACtE,yEAAyE;QACzE,gFAAgF;QAChF,MAAM,GAAG,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,aAAa,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,GAAG,MAAM,CAAC,IAAI,CAAC,CAAC;QAC5E,GAAG,CAAC,QAAQ,GAAG,GAAG,CAAC,QAAQ,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,KAAK,CAAC;QAC1D,MAAM,EAAE,GAAG,GAAG,CAAC,QAAQ,EAAE,CAAC;QAC1B,OAAO;YACL,GAAG,EAAE,EAAE;YACP,UAAU,EAAE,MAAM,CAAC,UAAU;YAC7B,UAAU,EAAE,IAAI;YAChB,KAAK,EAAE;gBACL,gEAAgE;gBAChE,+IAA+I;gBAC/I,qIAAqI;gBACrI,qFAAqF;gBACrF,kFAAkF;aACnF;SACF,CAAC;IACJ,CAAC,CAAC,CACL,CAAC;AACJ,CAAC,CAAC"}
|
package/dist/tools/scrape.js
CHANGED
|
@@ -17,7 +17,7 @@ export const registerScrapeTools = (server, client, { hideUnavailable }) => {
|
|
|
17
17
|
.enum(["markdown", "text", "html", "json", "pdf"])
|
|
18
18
|
.default("markdown")
|
|
19
19
|
.describe("Output format. 'markdown' (default) is best for feeding to an LLM; 'html' returns the raw document; 'json' returns a structured envelope; " +
|
|
20
|
-
"'pdf' prints the page in a browser and returns the file as a resource content block
|
|
20
|
+
"'pdf' prints the page in a full browser (routed there for you) and returns the file as a resource content block."),
|
|
21
21
|
main_content_only: z
|
|
22
22
|
.boolean()
|
|
23
23
|
.optional()
|
|
@@ -82,13 +82,8 @@ export const registerScrapeTools = (server, client, { hideUnavailable }) => {
|
|
|
82
82
|
.optional()
|
|
83
83
|
.describe("Milliseconds to wait after load before capturing (render only)."),
|
|
84
84
|
// --- The rest of POST /v1/scrape, under the API's own names (passed through).
|
|
85
|
-
engine: z
|
|
86
|
-
.enum(["fetch", "obscura", "chromium", "camoufox"])
|
|
87
|
-
.optional()
|
|
88
|
-
.describe("Pin the execution engine instead of letting the router choose. A pinned engine the deployment does not run is refused (ERR::ENGINE::UNAVAILABLE), never substituted. `camoufox` is coming soon: do not pin it yet."),
|
|
89
85
|
mode: z.enum(["auto"]).optional().describe("Routing mode. `auto` lets the platform escalate fetch -> browser as needed, billing only the rung that worked."),
|
|
90
|
-
stealth: z.boolean().optional().describe(COMING_SOON + "Stealth mode: render in
|
|
91
|
-
headless: z.boolean().optional().describe("false runs Chromium on a real display (1920x1080 screen). chromium engine only."),
|
|
86
|
+
stealth: z.boolean().optional().describe(COMING_SOON + "Stealth mode: render in a full browser hardened against bot detection. Default false."),
|
|
92
87
|
method: z
|
|
93
88
|
.string()
|
|
94
89
|
.optional()
|
|
@@ -143,7 +138,7 @@ export const registerScrapeTools = (server, client, { hideUnavailable }) => {
|
|
|
143
138
|
const SCRAPE_ARG_NAMES = { js_render: "render", response_format: "format" };
|
|
144
139
|
/** Arguments passed to the API unchanged: the tool uses the REST name. */
|
|
145
140
|
const PASSTHROUGH = [
|
|
146
|
-
"
|
|
141
|
+
"mode", "stealth", "method", "custom_headers", "session_id", "sticky_key",
|
|
147
142
|
"proxy_verify", "wait_for", "wait_for_timeout", "block_resources", "screenshot_fullpage",
|
|
148
143
|
"screenshot_selector", "screenshot_format", "screenshot_quality", "autoparse", "extract_preset",
|
|
149
144
|
"network_capture", "parse_pdf", "max_cost", "original_status", "allowed_status_codes",
|
|
@@ -163,9 +158,9 @@ export const registerScrapeTools = (server, client, { hideUnavailable }) => {
|
|
|
163
158
|
"can change state on the target site: set them only when the user asks for that. " +
|
|
164
159
|
"For markdown, text and HTML it returns `{ content }`. With `format: \"json\"`, or when " +
|
|
165
160
|
`\`extract\`, ${aiExtractKey}\`autoparse\`, \`links\`, \`screenshot\` or \`network_capture\` is set, it returns the JSON envelope: ` +
|
|
166
|
-
"`content`, the site's `status`, `credits`, `engine`, `warnings` and `data` for extraction. " +
|
|
161
|
+
"`content`, the site's `status`, `credits`, `engine` (the rendering tier: `fetch`, `js` or `browser`), `warnings` and `data` for extraction. " +
|
|
167
162
|
"Screenshots come back as image content blocks; each `screenshots` entry in the JSON keeps its metadata and says which block holds it. " +
|
|
168
|
-
"A PDF comes back as a resource content block, with its size
|
|
163
|
+
"A PDF comes back as a resource content block, with its size, the rendering tier and the credits in the JSON.",
|
|
169
164
|
inputSchema: availableOnly(ScrapeInput, hideUnavailable),
|
|
170
165
|
// Not read-only, whatever the default call does: `method` accepts POST/PUT/PATCH/DELETE
|
|
171
166
|
// and `actions` can click, fill and run scripts on the target page, so a call can submit
|
package/dist/tools/scrape.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"scrape.js","sourceRoot":"","sources":["../../src/tools/scrape.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAAE,WAAW,EAAE,aAAa,EAAE,GAAG,EAAsB,MAAM,aAAa,CAAC;AAClF,OAAO,EAAE,iBAAiB,EAAE,MAAM,kBAAkB,CAAC;AACrD,OAAO,EAAE,SAAS,EAAE,MAAM,UAAU,CAAC;AACrC,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAE3D,MAAM,CAAC,MAAM,mBAAmB,GAAkB,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE,eAAe,EAAE,EAAE,EAAE;IAC1F,8EAA8E;IAC9E,0DAA0D;IAC1D,8EAA8E;IAE9E,MAAM,WAAW,GAAG;QAClB,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,QAAQ,CAAC,+BAA+B,CAAC;QAC/D,MAAM,EAAE,CAAC;aACN,OAAO,EAAE;aACT,OAAO,CAAC,KAAK,CAAC;aACd,QAAQ,CACP,mKAAmK,CACpK;QACH,MAAM,EAAE,CAAC;aACN,IAAI,CAAC,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,KAAK,CAAC,CAAC;aACjD,OAAO,CAAC,UAAU,CAAC;aACnB,QAAQ,CACP,4IAA4I;YAC1I,
|
|
1
|
+
{"version":3,"file":"scrape.js","sourceRoot":"","sources":["../../src/tools/scrape.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAAE,WAAW,EAAE,aAAa,EAAE,GAAG,EAAsB,MAAM,aAAa,CAAC;AAClF,OAAO,EAAE,iBAAiB,EAAE,MAAM,kBAAkB,CAAC;AACrD,OAAO,EAAE,SAAS,EAAE,MAAM,UAAU,CAAC;AACrC,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAE3D,MAAM,CAAC,MAAM,mBAAmB,GAAkB,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE,eAAe,EAAE,EAAE,EAAE;IAC1F,8EAA8E;IAC9E,0DAA0D;IAC1D,8EAA8E;IAE9E,MAAM,WAAW,GAAG;QAClB,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,QAAQ,CAAC,+BAA+B,CAAC;QAC/D,MAAM,EAAE,CAAC;aACN,OAAO,EAAE;aACT,OAAO,CAAC,KAAK,CAAC;aACd,QAAQ,CACP,mKAAmK,CACpK;QACH,MAAM,EAAE,CAAC;aACN,IAAI,CAAC,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,KAAK,CAAC,CAAC;aACjD,OAAO,CAAC,UAAU,CAAC;aACnB,QAAQ,CACP,4IAA4I;YAC1I,kHAAkH,CACrH;QACH,iBAAiB,EAAE,CAAC;aACjB,OAAO,EAAE;aACT,QAAQ,EAAE;aACV,QAAQ,CACP,0FAA0F,CAC3F;QACH,YAAY,EAAE,CAAC;aACZ,KAAK,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC;aACjB,QAAQ,EAAE;aACV,QAAQ,CAAC,qDAAqD,CAAC;QAClE,YAAY,EAAE,CAAC;aACZ,KAAK,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC;aACjB,QAAQ,EAAE;aACV,QAAQ,CAAC,yDAAyD,CAAC;QACtE,KAAK,EAAE,CAAC;aACL,OAAO,EAAE;aACT,OAAO,CAAC,KAAK,CAAC;aACd,QAAQ,CAAC,6EAA6E,CAAC;QAC1F,OAAO,EAAE,CAAC;aACP,MAAM,CAAC,CAAC,CAAC,GAAG,EAAE,CAAC;aACf,QAAQ,EAAE;aACV,QAAQ,CACP,wLAAwL,CACzL;QACH,UAAU,EAAE,CAAC;aACV,MAAM,CAAC;YACN,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,+CAA+C,CAAC;YACvF,MAAM,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,wCAAwC,CAAC;SACxF,CAAC;aACD,MAAM,EAAE,CAAC,mEAAmE;aAC5E,QAAQ,EAAE;aACV,QAAQ,CACP,WAAW;YACT,iLAAiL,CACpL;QACH,UAAU,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,QAAQ,CAAC,yCAAyC,CAAC;QAC1F,WAAW,EAAE,CAAC;aACX,OAAO,EAAE;aACT,QAAQ,EAAE;aACV,QAAQ,CACP,+JAA+J,CAChK;QACH,aAAa,EAAE,CAAC;aACb,OAAO,EAAE;aACT,OAAO,CAAC,KAAK,CAAC;aACd,QAAQ,CAAC,WAAW,GAAG,+GAA+G,CAAC;QAC1I,aAAa,EAAE,CAAC;aACb,MAAM,EAAE;aACR,MAAM,CAAC,CAAC,CAAC;aACT,QAAQ,EAAE;aACV,QAAQ,CAAC,WAAW,GAAG,gFAAgF,CAAC;QAC3G,KAAK,EAAE,CAAC;aACL,MAAM,EAAE;aACR,QAAQ,EAAE;aACV,QAAQ,CAAC,qFAAqF,CAAC;QAClG,KAAK,EAAE,CAAC;aACL,OAAO,EAAE;aACT,QAAQ,EAAE;aACV,QAAQ,CAAC,kMAAkM,CAAC;QAC/M,SAAS,EAAE,CAAC;aACT,MAAM,EAAE;aACR,GAAG,EAAE;aACL,GAAG,CAAC,CAAC,CAAC;aACN,QAAQ,EAAE;aACV,QAAQ,CAAC,sIAAsI,CAAC;QACnJ,IAAI,EAAE,CAAC;aACJ,MAAM,EAAE;aACR,GAAG,EAAE;aACL,GAAG,CAAC,CAAC,CAAC;aACN,QAAQ,EAAE;aACV,QAAQ,CAAC,iEAAiE,CAAC;QAC9E,+EAA+E;QAC/E,IAAI,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,gHAAgH,CAAC;QAC5J,OAAO,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,WAAW,GAAG,uFAAuF,CAAC;QAC/I,MAAM,EAAE,CAAC;aACN,MAAM,EAAE;aACR,QAAQ,EAAE;aACV,QAAQ,CACP,mLAAmL,CACpL;QACH,cAAc,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,8EAA8E,CAAC;QACxI,UAAU,EAAE,CAAC;aACV,MAAM,EAAE;aACR,QAAQ,EAAE;aACV,QAAQ,CAAC,yHAAyH,CAAC;QACtI,UAAU,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,WAAW,GAAG,0FAA0F,CAAC;QACpJ,YAAY,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,8CAA8C,CAAC;QAC7F,QAAQ,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,0DAA0D,CAAC;QACpG,gBAAgB,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,0CAA0C,CAAC;QACzG,eAAe,EAAE,CAAC;aACf,KAAK,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC;aACjB,QAAQ,EAAE;aACV,QAAQ,CAAC,sHAAsH,CAAC;QACnI,OAAO,EAAE,aAAa,CAAC,QAAQ,EAAE;QACjC,mBAAmB,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,gDAAgD,CAAC;QACtG,mBAAmB,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,sDAAsD,CAAC;QAC3G,iBAAiB,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,0BAA0B,CAAC;QAC7E,kBAAkB,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,2CAA2C,CAAC;QACrH,SAAS,EAAE,CAAC;aACT,OAAO,EAAE;aACT,QAAQ,EAAE;aACV,QAAQ,CAAC,6IAA6I,CAAC;QAC1J,cAAc,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,WAAW,GAAG,kHAAkH,CAAC;QAChL,eAAe,EAAE,CAAC;aACf,MAAM,CAAC;YACN,IAAI,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,oDAAoD,CAAC;YACnG,cAAc,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,qDAAqD,CAAC;YAC9G,cAAc,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,0BAA0B,CAAC;YAC3E,cAAc,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,wBAAwB,CAAC;YAC9E,aAAa,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,4BAA4B,CAAC;SAClF,CAAC;aACD,MAAM,EAAE,CAAC,mEAAmE;aAC5E,QAAQ,EAAE;aACV,QAAQ,CAAC,2GAA2G,CAAC;QACxH,SAAS,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,0EAA0E,CAAC;QACtH,QAAQ,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,iFAAiF,CAAC;QACxI,eAAe,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,6EAA6E,CAAC;QAC/H,oBAAoB,EAAE,CAAC;aACpB,KAAK,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC;aACvB,QAAQ,EAAE;aACV,QAAQ,CAAC,mGAAmG,CAAC;KACjH,CAAC;IAEF;;;;OAIG;IACH,MAAM,gBAAgB,GAA2B,EAAE,SAAS,EAAE,QAAQ,EAAE,eAAe,EAAE,QAAQ,EAAE,CAAC;IAEpG,0EAA0E;IAC1E,MAAM,WAAW,GAAG;QAClB,MAAM,EAAE,SAAS,EAAE,QAAQ,EAAE,gBAAgB,EAAE,YAAY,EAAE,YAAY;QACzE,cAAc,EAAE,UAAU,EAAE,kBAAkB,EAAE,iBAAiB,EAAE,qBAAqB;QACxF,qBAAqB,EAAE,mBAAmB,EAAE,oBAAoB,EAAE,WAAW,EAAE,gBAAgB;QAC/F,iBAAiB,EAAE,WAAW,EAAE,UAAU,EAAE,iBAAiB,EAAE,sBAAsB;KAC7E,CAAC;IAEX,4FAA4F;IAC5F,MAAM,aAAa,GAAG,eAAe,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,mDAAmD,CAAC;IACjG,MAAM,YAAY,GAAG,eAAe,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,gBAAgB,CAAC;IAE7D,MAAM,CAAC,YAAY,CACjB,iBAAiB,EACjB;QACE,KAAK,EAAE,cAAc;QACrB,WAAW,EACT,mHAAmH;YACnH,qFAAqF;YACrF,uFAAuF,aAAa,IAAI;YACxG,uGAAuG;YACvG,iEAAiE;YACjE,0KAA0K;YAC1K,uJAAuJ;YACvJ,kFAAkF;YAClF,yFAAyF;YACzF,gBAAgB,YAAY,wGAAwG;YACpI,8IAA8I;YAC9I,wIAAwI;YACxI,8GAA8G;QAChH,WAAW,EAAE,aAAa,CAAC,WAAW,EAAE,eAAe,CAAC;QACxD,wFAAwF;QACxF,yFAAyF;QACzF,oFAAoF;QACpF,qEAAqE;QACrE,WAAW,EAAE;YACX,YAAY,EAAE,KAAK;YACnB,eAAe,EAAE,KAAK;YACtB,cAAc,EAAE,KAAK;YACrB,aAAa,EAAE,IAAI;SACpB;KACF,EACD,KAAK,EAAE,KAAK,EAAE,EAAE;QACd,MAAM,MAAM,GAA4B,EAAE,GAAG,EAAE,KAAK,CAAC,GAAG,EAAE,eAAe,EAAE,KAAK,CAAC,MAAM,EAAE,CAAC;QAC1F,IAAI,KAAK,CAAC,MAAM;YAAE,MAAM,CAAC,SAAS,GAAG,IAAI,CAAC;QAC1C,IAAI,KAAK,CAAC,iBAAiB,KAAK,SAAS;YAAE,MAAM,CAAC,iBAAiB,GAAG,KAAK,CAAC,iBAAiB,CAAC;QAC9F,IAAI,KAAK,CAAC,YAAY,EAAE,MAAM;YAAE,MAAM,CAAC,YAAY,GAAG,KAAK,CAAC,YAAY,CAAC;QACzE,IAAI,KAAK,CAAC,YAAY,EAAE,MAAM;YAAE,MAAM,CAAC,YAAY,GAAG,KAAK,CAAC,YAAY,CAAC;QACzE,IAAI,KAAK,CAAC,KAAK;YAAE,MAAM,CAAC,KAAK,GAAG,IAAI,CAAC;QACrC,IAAI,KAAK,CAAC,OAAO;YAAE,MAAM,CAAC,OAAO,GAAG,KAAK,CAAC,OAAO,CAAC;QAClD,IAAI,KAAK,CAAC,UAAU;YAAE,MAAM,CAAC,UAAU,GAAG,KAAK,CAAC,UAAU,CAAC;QAC3D,IAAI,KAAK,CAAC,UAAU;YAAE,MAAM,CAAC,UAAU,GAAG,IAAI,CAAC;QAC/C,IAAI,KAAK,CAAC,WAAW,KAAK,SAAS;YAAE,MAAM,CAAC,WAAW,GAAG,KAAK,CAAC,WAAW,CAAC;QAC5E,IAAI,KAAK,CAAC,aAAa;YAAE,MAAM,CAAC,aAAa,GAAG,IAAI,CAAC;QACrD,IAAI,KAAK,CAAC,aAAa;YAAE,MAAM,CAAC,aAAa,GAAG,KAAK,CAAC,aAAa,CAAC;QACpE,IAAI,KAAK,CAAC,KAAK;YAAE,MAAM,CAAC,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC;QAC5C,IAAI,KAAK,CAAC,KAAK,KAAK,SAAS;YAAE,MAAM,CAAC,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC;QAC1D,IAAI,KAAK,CAAC,SAAS,KAAK,SAAS;YAAE,MAAM,CAAC,SAAS,GAAG,KAAK,CAAC,SAAS,CAAC;QACtE,IAAI,KAAK,CAAC,IAAI,KAAK,SAAS;YAAE,MAAM,CAAC,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC;QACvD,IAAI,KAAK,CAAC,OAAO,EAAE,MAAM;YAAE,MAAM,CAAC,OAAO,GAAG,YAAY,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;QACxE,KAAK,MAAM,CAAC,IAAI,WAAW,EAAE,CAAC;YAC5B,MAAM,CAAC,GAAI,KAAiC,CAAC,CAAC,CAAC,CAAC;YAChD,IAAI,CAAC,KAAK,SAAS;gBAAE,MAAM,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC;QACrC,CAAC;QACD,6EAA6E;QAC7E,MAAM,MAAM,GAAG,CAAC,IAAa,EAAE,EAAE;YAC/B,MAAM,KAAK,GAAG,iBAAiB,CAAC,IAAI,CAAC,CAAC;YACtC,MAAM,GAAG,GAAG,SAAS,CAAC,KAAK,CAAC,IAAI,EAAE,KAAK,CAAC,GAAG,EAAE,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;YAClE,OAAO,EAAE,IAAI,EAAE,GAAG,CAAC,IAAI,EAAE,MAAM,EAAE,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,GAAG,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC;QACtE,CAAC,CAAC;QACF,OAAO,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,EAAE,MAAM,EAAE,gBAAgB,CAAC,CAAC;IACpE,CAAC,CACF,CAAC;AACF,CAAC,CAAC"}
|
package/dist/tools/sessions.js
CHANGED
|
@@ -14,7 +14,8 @@ const force = z
|
|
|
14
14
|
.boolean()
|
|
15
15
|
.optional()
|
|
16
16
|
.describe("Break an active lease (a scrape currently using the session). Only set when you know it is safe.");
|
|
17
|
-
|
|
17
|
+
// The rendering tiers a session can be bound to (the API's public names).
|
|
18
|
+
const engines = ["fetch", "js", "browser"];
|
|
18
19
|
// Sessions: /v1/sessions.
|
|
19
20
|
export const registerSessionsTools = (server, client, { hideUnavailable }) => {
|
|
20
21
|
// -------------------------------------------------------------------------
|
|
@@ -22,17 +23,17 @@ export const registerSessionsTools = (server, client, { hideUnavailable }) => {
|
|
|
22
23
|
// -------------------------------------------------------------------------
|
|
23
24
|
server.registerTool("spicrawl_session_create", {
|
|
24
25
|
title: "Create a browsing session",
|
|
25
|
-
description: "Create a session: a persistent browser identity (cookie jar, localStorage,
|
|
26
|
+
description: "Create a session: a persistent browser identity (cookie jar, localStorage, a fixed rendering tier and, by default, a " +
|
|
26
27
|
"sticky exit IP) that survives across scrapes. Use it for multi-step flows — log in, then fetch pages behind " +
|
|
27
28
|
"the login — by passing the returned `id` as `session_id` to every `spicrawl_scrape` call that should share the " +
|
|
28
|
-
"state. The
|
|
29
|
-
"
|
|
29
|
+
"state. The rendering tier is fixed for the session's lifetime and every scrape on the session runs on it. Leave " +
|
|
30
|
+
"`engine` out and one is chosen for you. Release the session with `spicrawl_session_release` when done. " +
|
|
30
31
|
"Returns the session object (id, engine, status, sticky_key, proxy, expires_at).",
|
|
31
32
|
inputSchema: availableOnly({
|
|
32
33
|
engine: z
|
|
33
34
|
.enum(engines)
|
|
34
35
|
.optional()
|
|
35
|
-
.describe("
|
|
36
|
+
.describe("Rendering tier fixed for the session's lifetime. Optional: leave it out and one is chosen for you. `fetch` is plain HTTP; `js` is JavaScript rendering and refuses `actions`, `block_resources`, screenshots and PDFs; `browser` is a full browser."),
|
|
36
37
|
ttl_seconds: z
|
|
37
38
|
.number()
|
|
38
39
|
.int()
|
|
@@ -57,7 +58,7 @@ export const registerSessionsTools = (server, client, { hideUnavailable }) => {
|
|
|
57
58
|
fingerprint: z
|
|
58
59
|
.record(z.unknown())
|
|
59
60
|
.optional()
|
|
60
|
-
.describe(COMING_SOON + "
|
|
61
|
+
.describe(COMING_SOON + "Browser fingerprint pin, stored verbatim. Needs a full-browser session (`engine: \"browser\"`)."),
|
|
61
62
|
domain_scores: z
|
|
62
63
|
.record(z.object({
|
|
63
64
|
score: z.number(),
|
|
@@ -90,7 +91,7 @@ export const registerSessionsTools = (server, client, { hideUnavailable }) => {
|
|
|
90
91
|
"`next_cursor` back as `cursor` for the next page. Never includes the stored credentials.",
|
|
91
92
|
inputSchema: {
|
|
92
93
|
status: z.enum(["active", "released", "expired"]).optional().describe("Only sessions in this state."),
|
|
93
|
-
engine: z.enum(engines).optional().describe("Only sessions
|
|
94
|
+
engine: z.enum(engines).optional().describe("Only sessions on this rendering tier."),
|
|
94
95
|
limit: z.number().int().min(1).max(200).optional().describe("Page size (default 50, max 200)."),
|
|
95
96
|
cursor: z.string().optional().describe("`next_cursor` from a previous page, verbatim."),
|
|
96
97
|
},
|
|
@@ -101,7 +102,7 @@ export const registerSessionsTools = (server, client, { hideUnavailable }) => {
|
|
|
101
102
|
// -------------------------------------------------------------------------
|
|
102
103
|
server.registerTool("spicrawl_session_get", {
|
|
103
104
|
title: "Get a session",
|
|
104
|
-
description: "Return one session's metadata: status, engine, proxy/exit, usage count, expiry, lease holder, and a summary " +
|
|
105
|
+
description: "Return one session's metadata: status, `engine` (its rendering tier), proxy/exit, usage count, expiry, lease holder, and a summary " +
|
|
105
106
|
"of its stored context. Never returns the credentials themselves (use spicrawl_session_context for that).",
|
|
106
107
|
inputSchema: { session_id: sessionId },
|
|
107
108
|
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"sessions.js","sourceRoot":"","sources":["../../src/tools/sessions.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAAE,WAAW,EAAE,aAAa,EAAE,GAAG,EAAsB,MAAM,aAAa,CAAC;AAElF,MAAM,WAAW,GAAG,CAAC,EAAU,EAAE,EAAE,CAAC,gBAAgB,kBAAkB,CAAC,EAAE,CAAC,EAAE,CAAC;AAE7E,SAAS,SAAS,CAAC,IAAY,EAAE,CAAwD;IACvF,MAAM,MAAM,GAAG,IAAI,eAAe,EAAE,CAAC;IACrC,KAAK,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC;QAAE,IAAI,CAAC,KAAK,SAAS;YAAE,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC;IACtF,MAAM,CAAC,GAAG,MAAM,CAAC,QAAQ,EAAE,CAAC;IAC5B,OAAO,CAAC,CAAC,CAAC,CAAC,GAAG,IAAI,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;AACnC,CAAC;AAED,MAAM,SAAS,GAAG,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,8EAA8E,CAAC,CAAC;AAC7H,MAAM,KAAK,GAAG,CAAC;KACZ,OAAO,EAAE;KACT,QAAQ,EAAE;KACV,QAAQ,CAAC,kGAAkG,CAAC,CAAC;AAEhH,MAAM,OAAO,GAAG,CAAC,OAAO,EAAE,
|
|
1
|
+
{"version":3,"file":"sessions.js","sourceRoot":"","sources":["../../src/tools/sessions.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAAE,WAAW,EAAE,aAAa,EAAE,GAAG,EAAsB,MAAM,aAAa,CAAC;AAElF,MAAM,WAAW,GAAG,CAAC,EAAU,EAAE,EAAE,CAAC,gBAAgB,kBAAkB,CAAC,EAAE,CAAC,EAAE,CAAC;AAE7E,SAAS,SAAS,CAAC,IAAY,EAAE,CAAwD;IACvF,MAAM,MAAM,GAAG,IAAI,eAAe,EAAE,CAAC;IACrC,KAAK,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC;QAAE,IAAI,CAAC,KAAK,SAAS;YAAE,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC;IACtF,MAAM,CAAC,GAAG,MAAM,CAAC,QAAQ,EAAE,CAAC;IAC5B,OAAO,CAAC,CAAC,CAAC,CAAC,GAAG,IAAI,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;AACnC,CAAC;AAED,MAAM,SAAS,GAAG,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,8EAA8E,CAAC,CAAC;AAC7H,MAAM,KAAK,GAAG,CAAC;KACZ,OAAO,EAAE;KACT,QAAQ,EAAE;KACV,QAAQ,CAAC,kGAAkG,CAAC,CAAC;AAEhH,0EAA0E;AAC1E,MAAM,OAAO,GAAG,CAAC,OAAO,EAAE,IAAI,EAAE,SAAS,CAAU,CAAC;AAEpD,0BAA0B;AAC1B,MAAM,CAAC,MAAM,qBAAqB,GAAkB,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE,eAAe,EAAE,EAAE,EAAE;IAC1F,4EAA4E;IAC5E,8CAA8C;IAC9C,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,yBAAyB,EACzB;QACE,KAAK,EAAE,2BAA2B;QAClC,WAAW,EACT,uHAAuH;YACvH,8GAA8G;YAC9G,iHAAiH;YACjH,kHAAkH;YAClH,yGAAyG;YACzG,iFAAiF;QACnF,WAAW,EAAE,aAAa,CAAC;YACzB,MAAM,EAAE,CAAC;iBACN,IAAI,CAAC,OAAO,CAAC;iBACb,QAAQ,EAAE;iBACV,QAAQ,CAAC,qPAAqP,CAAC;YAClQ,WAAW,EAAE,CAAC;iBACX,MAAM,EAAE;iBACR,GAAG,EAAE;iBACL,GAAG,CAAC,EAAE,CAAC;iBACP,QAAQ,EAAE;iBACV,QAAQ,CAAC,uGAAuG,CAAC;YACpH,UAAU,EAAE,CAAC;iBACV,MAAM,EAAE;iBACR,QAAQ,EAAE;iBACV,QAAQ,CAAC,WAAW,GAAG,gHAAgH,CAAC;YAC3I,SAAS,EAAE,CAAC;iBACT,OAAO,EAAE;iBACT,QAAQ,EAAE;iBACV,QAAQ,CAAC,WAAW,GAAG,6GAA6G,CAAC;YACxI,WAAW,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,WAAW,GAAG,4CAA4C,CAAC;YACvG,aAAa,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,WAAW,GAAG,qEAAqE,CAAC;YACnI,aAAa,EAAE,CAAC;iBACb,MAAM,EAAE;iBACR,MAAM,CAAC,CAAC,CAAC;iBACT,QAAQ,EAAE;iBACV,QAAQ,CAAC,WAAW,GAAG,wDAAwD,CAAC;YACnF,WAAW,EAAE,CAAC;iBACX,MAAM,CAAC,CAAC,CAAC,OAAO,EAAE,CAAC;iBACnB,QAAQ,EAAE;iBACV,QAAQ,CAAC,WAAW,GAAG,iGAAiG,CAAC;YAC5H,aAAa,EAAE,CAAC;iBACb,MAAM,CACL,CAAC,CAAC,MAAM,CAAC;gBACP,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE;gBACjB,IAAI,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE;gBACtB,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE;gBACrB,YAAY,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE;aAC/B,CAAC,CAAC,WAAW,EAAE,CACjB;iBACA,QAAQ,EAAE;iBACV,QAAQ,CAAC,qFAAqF,CAAC;YAClG,eAAe,EAAE,CAAC;iBACf,MAAM,CAAC,CAAC,CAAC,OAAO,EAAE,CAAC;iBACnB,QAAQ,EAAE;iBACV,QAAQ,CACP,oIAAoI,CACrI;SACJ,EAAE,eAAe,CAAC;QACnB,0FAA0F;QAC1F,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,KAAK,EAAE,aAAa,EAAE,KAAK,EAAE;KAC1G,EACD,KAAK,EAAE,KAAK,EAAE,EAAE;QACd,MAAM,IAAI,GAA4B,EAAE,CAAC;QACzC,KAAK,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,KAAK,CAAC;YAAE,IAAI,CAAC,KAAK,SAAS;gBAAE,IAAI,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC;QAC7E,OAAO,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,cAAc,EAAE,IAAI,CAAC,CAAC,CAAC;IACjE,CAAC,CACF,CAAC;IAEF,4EAA4E;IAC5E,2CAA2C;IAC3C,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,uBAAuB,EACvB;QACE,KAAK,EAAE,eAAe;QACtB,WAAW,EACT,0GAA0G;YAC1G,0FAA0F;QAC5F,WAAW,EAAE;YACX,MAAM,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,QAAQ,EAAE,UAAU,EAAE,SAAS,CAAC,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,8BAA8B,CAAC;YACrG,MAAM,EAAE,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,uCAAuC,CAAC;YACpF,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,kCAAkC,CAAC;YAC/F,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,+CAA+C,CAAC;SACxF;QACD,WAAW,EAAE,EAAE,YAAY,EAAE,IAAI,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,MAAM,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,EAAE,EAAE,CAC1C,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,SAAS,CAAC,cAAc,EAAE,EAAE,MAAM,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,CACjG,CAAC;IAEF,4EAA4E;IAC5E,+CAA+C;IAC/C,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,sBAAsB,EACtB;QACE,KAAK,EAAE,eAAe;QACtB,WAAW,EACT,qIAAqI;YACrI,0GAA0G;QAC5G,WAAW,EAAE,EAAE,UAAU,EAAE,SAAS,EAAE;QACtC,WAAW,EAAE,EAAE,YAAY,EAAE,IAAI,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,UAAU,EAAE,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,WAAW,CAAC,UAAU,CAAC,CAAC,CAAC,CACpF,CAAC;IAEF,4EAA4E;IAC5E,2DAA2D;IAC3D,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,0BAA0B,EAC1B;QACE,KAAK,EAAE,iCAAiC;QACxC,WAAW,EACT,0GAA0G;YAC1G,gHAAgH;YAChH,4GAA4G;QAC9G,WAAW,EAAE,EAAE,UAAU,EAAE,SAAS,EAAE;QACtC,WAAW,EAAE,EAAE,YAAY,EAAE,IAAI,EAAE,eAAe,EAAE,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,UAAU,EAAE,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,GAAG,WAAW,CAAC,UAAU,CAAC,UAAU,CAAC,CAAC,CACjG,CAAC;IAEF,4EAA4E;IAC5E,4DAA4D;IAC5D,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,0BAA0B,EAC1B;QACE,KAAK,EAAE,mBAAmB;QAC1B,WAAW,EACT,8GAA8G;YAC9G,wIAAwI;YACxI,sFAAsF;YACtF,sGAAsG;QACxG,WAAW,EAAE,EAAE,UAAU,EAAE,SAAS,EAAE,KAAK,EAAE;QAC7C,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,IAAI,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,UAAU,EAAE,KAAK,EAAE,EAAE,EAAE,CAC9B,GAAG,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,SAAS,CAAC,GAAG,WAAW,CAAC,UAAU,CAAC,UAAU,EAAE,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,CAChG,CAAC;IAEF,4EAA4E;IAC5E,qDAAqD;IACrD,4EAA4E;IAC5E,MAAM,CAAC,YAAY,CACjB,yBAAyB,EACzB;QACE,KAAK,EAAE,kBAAkB;QACzB,WAAW,EACT,kHAAkH;YAClH,gHAAgH;YAChH,0DAA0D;QAC5D,WAAW,EAAE,EAAE,UAAU,EAAE,SAAS,EAAE,KAAK,EAAE;QAC7C,WAAW,EAAE,EAAE,YAAY,EAAE,KAAK,EAAE,eAAe,EAAE,IAAI,EAAE,cAAc,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE;KACxG,EACD,KAAK,EAAE,EAAE,UAAU,EAAE,KAAK,EAAE,EAAE,EAAE,CAC9B,GAAG,CAAC,KAAK,IAAI,EAAE;QACb,MAAM,MAAM,CAAC,OAAO,CAAC,QAAQ,EAAE,SAAS,CAAC,WAAW,CAAC,UAAU,CAAC,EAAE,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC;QAC9E,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,CAAC;IACvC,CAAC,CAAC,CACL,CAAC;AACJ,CAAC,CAAC"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@spicrawl/mcp",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.6",
|
|
4
4
|
"mcpName": "io.github.OfficialSpicrawl/mcp",
|
|
5
5
|
"description": "The Spicrawl MCP server is a Model Context Protocol server that gives AI agents in Claude Code, Claude Desktop, Cursor, VS Code, Windsurf, Gemini CLI and Codex web scraping to Markdown, structured data extraction, screenshots, browser sessions and batch scraping as 25 native tools.",
|
|
6
6
|
"license": "Apache-2.0",
|
|
@@ -26,7 +26,7 @@
|
|
|
26
26
|
"data-extraction",
|
|
27
27
|
"structured-data",
|
|
28
28
|
"screenshot",
|
|
29
|
-
"
|
|
29
|
+
"javascript-rendering",
|
|
30
30
|
"batch-scraping",
|
|
31
31
|
"llm",
|
|
32
32
|
"ai-agents",
|
|
@@ -64,7 +64,10 @@
|
|
|
64
64
|
"dev": "tsc --watch",
|
|
65
65
|
"typecheck": "tsc --noEmit",
|
|
66
66
|
"test": "npm run build && node --test test/*.test.mjs",
|
|
67
|
-
"prepublishOnly": "npm run typecheck && npm test"
|
|
67
|
+
"prepublishOnly": "npm run typecheck && npm test",
|
|
68
|
+
"registry:validate": "node scripts/registry.mjs validate",
|
|
69
|
+
"registry:sync": "node scripts/registry.mjs sync",
|
|
70
|
+
"version": "npm run registry:sync"
|
|
68
71
|
},
|
|
69
72
|
"dependencies": {
|
|
70
73
|
"@modelcontextprotocol/sdk": "^1.12.0",
|
|
@@ -72,6 +75,8 @@
|
|
|
72
75
|
},
|
|
73
76
|
"devDependencies": {
|
|
74
77
|
"@types/node": "^22.10.5",
|
|
78
|
+
"ajv": "8.20.0",
|
|
79
|
+
"ajv-formats": "3.0.1",
|
|
75
80
|
"typescript": "^5.7.3"
|
|
76
81
|
}
|
|
77
82
|
}
|