@imenam/simple-scraper 1.0.3 → 1.0.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +36 -0
- package/dist/index.js +1 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -29,6 +29,17 @@ npm install -g @imenam/simple-scraper
|
|
|
29
29
|
simple-scraper
|
|
30
30
|
```
|
|
31
31
|
|
|
32
|
+
## Environment Variables
|
|
33
|
+
|
|
34
|
+
| Variable | Required | Default | Description |
|
|
35
|
+
|----------|----------|---------|-------------|
|
|
36
|
+
| `PUPPETEER_HEADLESS` | No | `true` | Run Chromium in headless mode. Set to `false` to display the browser window. |
|
|
37
|
+
| `PUPPETEER_TIMEOUT` | No | `30000` | Default timeout in milliseconds for page navigation and waits. |
|
|
38
|
+
| `COOKIES_DIR` | No | — | Absolute path to a folder containing Netscape-format `.txt` cookie files. All files are loaded and merged automatically before each request. |
|
|
39
|
+
| `PROXY_URL` | No | — | Base URL of the [MCP HTTP Gateway](https://www.npmjs.com/package/@imenam/mcp-http-gateway). Required to enable the GUI. |
|
|
40
|
+
| `PROXY_APP_PATH` | No | `/simple-scraper-mcp` | URL path under which the GUI is registered on the proxy. |
|
|
41
|
+
| `PROXY_APP_NAME` | No | `Simple Scraper MCP` | Display name shown in the proxy's app list. |
|
|
42
|
+
|
|
32
43
|
## Configuration
|
|
33
44
|
|
|
34
45
|
Copy `.env.example` to `.env` and configure the variables:
|
|
@@ -84,6 +95,31 @@ To load cookies automatically, add `COOKIES_DIR` pointing to a folder containing
|
|
|
84
95
|
}
|
|
85
96
|
```
|
|
86
97
|
|
|
98
|
+
## Usage with Cursor
|
|
99
|
+
|
|
100
|
+
In Cursor, MCP servers are configured in `.cursor/mcp.json`. You can pass environment variables directly in the config. Full example with all available options:
|
|
101
|
+
|
|
102
|
+
```json
|
|
103
|
+
{
|
|
104
|
+
"mcpServers": {
|
|
105
|
+
"simple-scraper": {
|
|
106
|
+
"command": "npx",
|
|
107
|
+
"args": ["-y", "@imenam/simple-scraper"],
|
|
108
|
+
"env": {
|
|
109
|
+
"PUPPETEER_HEADLESS": "true",
|
|
110
|
+
"PUPPETEER_TIMEOUT": "30000",
|
|
111
|
+
"COOKIES_DIR": "/path/to/your/cookies",
|
|
112
|
+
"PROXY_URL": "http://localhost:4500",
|
|
113
|
+
"PROXY_APP_PATH": "/simple-scraper",
|
|
114
|
+
"PROXY_APP_NAME": "Simple Scraper"
|
|
115
|
+
}
|
|
116
|
+
}
|
|
117
|
+
}
|
|
118
|
+
}
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
> **Note:** The `-y` flag in `args` avoids the interactive confirmation prompt when using `npx`.
|
|
122
|
+
|
|
87
123
|
## MCP Tools
|
|
88
124
|
|
|
89
125
|
### `scrape_page`
|
package/dist/index.js
CHANGED