crw-mcp 0.16.0 → 0.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/init.js +1 -1
- package/package.json +5 -5
- package/skills/SKILL.md +17 -4
package/bin/init.js
CHANGED
|
@@ -124,7 +124,7 @@ Without flags, auto-detects installed agents and installs to all of them.
|
|
|
124
124
|
} else {
|
|
125
125
|
console.log("\nCloud mode (fastcrw.com):");
|
|
126
126
|
console.log(" 500 free one-time credits, managed infra — https://fastcrw.com");
|
|
127
|
-
console.log(" export CRW_API_KEY=
|
|
127
|
+
console.log(" export CRW_API_KEY=crw_live_xxx");
|
|
128
128
|
console.log(" export CRW_API_URL=https://api.fastcrw.com");
|
|
129
129
|
console.log(" Terms of Service: https://fastcrw.com/terms");
|
|
130
130
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "crw-mcp",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.17.0",
|
|
4
4
|
"description": "MCP server for CRW web scraper — scrape, crawl, map, search, and PDF-parse tools for AI agents",
|
|
5
5
|
"license": "AGPL-3.0",
|
|
6
6
|
"homepage": "https://github.com/us/crw",
|
|
@@ -30,9 +30,9 @@
|
|
|
30
30
|
"skills/SKILL.md"
|
|
31
31
|
],
|
|
32
32
|
"optionalDependencies": {
|
|
33
|
-
"crw-mcp-darwin-x64": "0.
|
|
34
|
-
"crw-mcp-darwin-arm64": "0.
|
|
35
|
-
"crw-mcp-linux-x64": "0.
|
|
36
|
-
"crw-mcp-linux-arm64": "0.
|
|
33
|
+
"crw-mcp-darwin-x64": "0.17.0",
|
|
34
|
+
"crw-mcp-darwin-arm64": "0.17.0",
|
|
35
|
+
"crw-mcp-linux-x64": "0.17.0",
|
|
36
|
+
"crw-mcp-linux-arm64": "0.17.0"
|
|
37
37
|
}
|
|
38
38
|
}
|
package/skills/SKILL.md
CHANGED
|
@@ -32,7 +32,7 @@ This installs the CRW skill and MCP server to all detected AI agents (Claude Cod
|
|
|
32
32
|
## Authentication
|
|
33
33
|
|
|
34
34
|
- **Embedded mode** (default): No key needed — the MCP server runs a self-contained scraper in ~6 MB RAM. No server required.
|
|
35
|
-
- **Cloud mode** (fastcrw.com): Set `CRW_API_KEY=crw_live_...` and `CRW_API_URL=https://fastcrw.com
|
|
35
|
+
- **Cloud mode** (fastcrw.com): Set `CRW_API_KEY=crw_live_...` and `CRW_API_URL=https://api.fastcrw.com`. Get a free key at https://fastcrw.com with 500 one-time lifetime credits (never resets, not monthly).
|
|
36
36
|
|
|
37
37
|
## MCP Tools
|
|
38
38
|
|
|
@@ -76,7 +76,9 @@ Parameters:
|
|
|
76
76
|
- `id` (required) — The crawl job ID from `crw_crawl`
|
|
77
77
|
- `maxLength` — Truncate each page's content fields to this many chars. `0` = unbounded. Default: ~15 000
|
|
78
78
|
|
|
79
|
-
Returns: `{ "status": "
|
|
79
|
+
Returns: `{ "status": "scraping|completed|failed", "data": [...] }`
|
|
80
|
+
|
|
81
|
+
> **Browser Automation:** Full interactive browser control (JavaScript rendering, click, fill, etc.) requires the separate **crw-browse** MCP server binary (`command: crw-browse`). It exposes its own tools (`goto`, `tree`, and others) and is not part of this MCP server. Do not call `crw_browse` here — it is not a tool in crw-mcp and will return a JSON-RPC -32602 "Unknown tool" error.
|
|
80
82
|
|
|
81
83
|
### crw_search
|
|
82
84
|
|
|
@@ -86,7 +88,6 @@ Parameters:
|
|
|
86
88
|
- `query` (required) — The search query
|
|
87
89
|
- `limit` — Maximum number of results to return. Default: `5`
|
|
88
90
|
- `lang` — Language code for results (e.g. `"en"`, `"tr"`)
|
|
89
|
-
- `country` — Country code for results (e.g. `"us"`, `"tr"`)
|
|
90
91
|
- `tbs` — Time filter: `qdr:h|qdr:d|qdr:w|qdr:m|qdr:y` (past hour/day/week/month/year)
|
|
91
92
|
- `sources` — If set, group results by source: `web`, `news`, `images`
|
|
92
93
|
- `categories` — Bias toward a category (e.g. `"pdf"`, `"github"`, `"research"`, or a native SearXNG category)
|
|
@@ -137,6 +138,18 @@ crw_check_crawl_status(id="...") → poll until completed
|
|
|
137
138
|
crw_search(query="your search query", limit=5)
|
|
138
139
|
```
|
|
139
140
|
|
|
141
|
+
**Search from the CLI (one-shot LLM-ready output):**
|
|
142
|
+
|
|
143
|
+
When the `crw` binary is available, prefer the native field projection
|
|
144
|
+
over piping through `jq` — it's one call instead of two:
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
crw search "renewable energy 2024" --json --fields title,url,snippet --limit 3
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
Available fields: `title`, `url`, `description`, `snippet`, `position`,
|
|
151
|
+
`score`, `category`. `--json` is shorthand for `--format json`.
|
|
152
|
+
|
|
140
153
|
## Common Edge Cases
|
|
141
154
|
|
|
142
155
|
- **JavaScript-heavy sites**: Set `renderJs: true` if the page is blank or returns a loading skeleton
|
|
@@ -146,7 +159,7 @@ crw_search(query="your search query", limit=5)
|
|
|
146
159
|
|
|
147
160
|
## Links
|
|
148
161
|
|
|
149
|
-
- Cloud API: https://fastcrw.com — 500 free credits
|
|
162
|
+
- Cloud API: https://fastcrw.com — 500 one-time lifetime free credits (never resets, not monthly)
|
|
150
163
|
- Docs: https://docs.fastcrw.com
|
|
151
164
|
- GitHub: https://github.com/us/crw
|
|
152
165
|
- Firecrawl-compatible: same REST endpoints at `/v1/scrape`, `/v1/crawl`, `/v1/map`, `/v1/search`
|