imperium-crawl 1.1.0 → 1.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -173,17 +173,22 @@ Returns fresh structured data with all detected fields. Skills are saved in `~/.
173
173
 
174
174
  ## API Discovery Workflow
175
175
 
176
- This is the workflow that no other MCP server supports:
176
+ This is the workflow that no other MCP server supports. Real results from actual testing:
177
177
 
178
178
  ```
179
- 1. discover_apis({ url: "https://github.com/anthropics" })
180
- Finds 5 hidden JSON API endpoints (refs, tree-commit-info, overview-files...)
181
-
182
- 2. query_api({ url: "https://github.com/.../refs", method: "GET" })
179
+ 1. discover_apis({ url: "https://weather.com" })
180
+ Found 11 hidden API endpoints:
181
+ • Main weather API (api.weather.com) with exposed API key
182
+ mParticle analytics endpoints
183
+ • Taboola content recommendation API
184
+ • OneTrust consent management API
185
+ • DAA/AdChoices opt-out endpoints
186
+
187
+ 2. query_api({ url: "https://api.weather.com/v3/...", method: "GET" })
183
188
  → Direct API call, bypasses DOM entirely — 10x faster, structured JSON response
184
189
 
185
190
  3. monitor_websocket({ url: "https://binance.com/en/trade/BTC_USDT", duration_seconds: 10 })
186
- → Captures 38 real-time WebSocket messages from 3 connections
191
+ → Captures real-time WebSocket messages financial tickers, live data feeds
187
192
  ```
188
193
 
189
194
  Turn any website into an API. No documentation needed.
@@ -193,13 +198,43 @@ Turn any website into an API. No documentation needed.
193
198
  ## Resilience
194
199
 
195
200
  - **Exponential backoff with full jitter** — AWS-recommended retry pattern, no thundering herd
196
- - **Per-domain circuit breaker** — 5 consecutive failures opens the circuit for 60s, then half-open probing
201
+ - **Per-domain circuit breaker** — 5 consecutive failures opens the circuit for 60s, then half-open probing with automatic recovery
197
202
  - **URL normalization** — 11-step pipeline removes tracking params (utm_*, fbclid, gclid), sorts query params, normalizes encoding
198
203
  - **Concurrency limiting** — per-domain request throttling via p-queue
204
+ - **Input validation** — all 16 tool schemas enforce strict bounds (URL length, query size, concurrency limits, body size)
205
+ - **HTTP transport hardening** — rate limiting (100 req/min), 1MB body limit, 5min request timeout
206
+ - **Proxy support** — single proxy (`PROXY_URL`) or rotating pool (`PROXY_URLS`) with http/https/socks4/socks5 support
207
+ - **Browser pool** — keyed by proxy URL, auto-eviction, configurable pool size
208
+ - **Graceful shutdown** — 10s timeout on browser cleanup to prevent hung processes
199
209
  - **robots.txt** — respected by default (configurable)
200
210
 
201
211
  ---
202
212
 
213
+ ## Real-World Test Results
214
+
215
+ Every tool tested against production websites with real anti-bot defenses:
216
+
217
+ | Tool | Test Target | Result |
218
+ |------|------------|--------|
219
+ | **scrape** | bbc.com/news | Full markdown extraction, stealth level 3 auto-escalation |
220
+ | **crawl** | blog.cloudflare.com | 213K chars crawled across multiple pages with depth control |
221
+ | **map** | bbc.com | Full URL discovery via sitemap + page link extraction |
222
+ | **extract** | imdb.com | Structured CSS selector extraction with `@attr` support |
223
+ | **readability** | Medium article | Clean article extraction — title, author, content, date |
224
+ | **screenshot** | producthunt.com | Captured Cloudflare Turnstile challenge page (auto-solves with 2Captcha key) |
225
+ | **search** | "test" query | Web results via Brave API |
226
+ | **news_search** | "AI" query | News results with freshness ranking |
227
+ | **image_search** | "landscape" query | Image results with thumbnails and source URLs |
228
+ | **video_search** | "tutorial" query | Video results across platforms |
229
+ | **discover_apis** | weather.com | Found 11 hidden APIs including main weather API with exposed key |
230
+ | **query_api** | jsonplaceholder API | Direct JSON API calls with stealth headers |
231
+ | **monitor_websocket** | example.com | WebSocket connection monitoring and message capture |
232
+ | **create_skill** | Any page | Auto-detects repeating patterns, generates CSS selectors |
233
+ | **run_skill** | Saved skill | Fresh structured data from saved extraction config |
234
+ | **list_skills** | — | Lists all saved skills with configurations |
235
+
236
+ ---
237
+
203
238
  ## Environment Variables
204
239
 
205
240
  | Variable | Required | Description |
@@ -208,6 +243,10 @@ Turn any website into an API. No documentation needed.
208
243
  | `TWOCAPTCHA_API_KEY` | No | 2Captcha API key (enables auto CAPTCHA solving) |
209
244
  | `TRANSPORT` | No | `stdio` (default) or `http` |
210
245
  | `PORT` | No | HTTP port (default: 3000) |
246
+ | `PROXY_URL` | No | Single proxy URL (http/https/socks4/socks5) |
247
+ | `PROXY_URLS` | No | Comma-separated proxy URLs for rotation |
248
+ | `BROWSER_POOL_SIZE` | No | Max pooled browser instances (default: 3) |
249
+ | `CHROME_PROFILE_PATH` | No | Chrome user data dir for authenticated sessions |
211
250
  | `RESPECT_ROBOTS` | No | Respect robots.txt (default: `true`) |
212
251
 
213
252
  ---
@@ -219,7 +258,7 @@ git clone https://github.com/ceoimperiumprojects/imperium-crawl
219
258
  cd imperium-crawl
220
259
  npm install
221
260
  npm run build
222
- npm test # 120 tests
261
+ npm test # 285 tests
223
262
  npm start
224
263
  ```
225
264
 
@@ -1,5 +1,5 @@
1
1
  export declare const PACKAGE_NAME = "imperium-crawl";
2
- export declare const PACKAGE_VERSION = "1.1.0";
2
+ export declare const PACKAGE_VERSION = "1.1.2";
3
3
  export declare const DEFAULT_TIMEOUT_MS = 30000;
4
4
  export declare const DEFAULT_MAX_PAGES = 10;
5
5
  export declare const DEFAULT_MAX_DEPTH = 2;
package/dist/constants.js CHANGED
@@ -1,5 +1,5 @@
1
1
  export const PACKAGE_NAME = "imperium-crawl";
2
- export const PACKAGE_VERSION = "1.1.0";
2
+ export const PACKAGE_VERSION = "1.1.2";
3
3
  export const DEFAULT_TIMEOUT_MS = 30_000;
4
4
  export const DEFAULT_MAX_PAGES = 10;
5
5
  export const DEFAULT_MAX_DEPTH = 2;
@@ -3,7 +3,7 @@ export declare const name = "extract";
3
3
  export declare const description = "Extract structured data from a web page using CSS selectors. Returns JSON with the extracted fields.";
4
4
  export declare const schema: z.ZodObject<{
5
5
  url: z.ZodString;
6
- selectors: z.ZodEffects<z.ZodRecord<z.ZodString, z.ZodString>, Record<string, string>, Record<string, string>>;
6
+ selectors: z.ZodEffects<z.ZodEffects<z.ZodRecord<z.ZodString, z.ZodString>, Record<string, string>, Record<string, string>>, Record<string, string>, unknown>;
7
7
  items_selector: z.ZodOptional<z.ZodString>;
8
8
  proxy: z.ZodOptional<z.ZodString>;
9
9
  chrome_profile: z.ZodOptional<z.ZodString>;
@@ -15,9 +15,9 @@ export declare const schema: z.ZodObject<{
15
15
  items_selector?: string | undefined;
16
16
  }, {
17
17
  url: string;
18
- selectors: Record<string, string>;
19
18
  proxy?: string | undefined;
20
19
  chrome_profile?: string | undefined;
20
+ selectors?: unknown;
21
21
  items_selector?: string | undefined;
22
22
  }>;
23
23
  export type ExtractInput = z.infer<typeof schema>;
@@ -1 +1 @@
1
- {"version":3,"file":"extract.d.ts","sourceRoot":"","sources":["../../src/tools/extract.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAOxB,eAAO,MAAM,IAAI,YAAY,CAAC;AAE9B,eAAO,MAAM,WAAW,yGACgF,CAAC;AAEzG,eAAO,MAAM,MAAM;;;;;;;;;;;;;;;;;;EAejB,CAAC;AAEH,MAAM,MAAM,YAAY,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,MAAM,CAAC,CAAC;AAclD,wBAAsB,OAAO,CAAC,KAAK,EAAE,YAAY;;;;;GAgDhD"}
1
+ {"version":3,"file":"extract.d.ts","sourceRoot":"","sources":["../../src/tools/extract.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAOxB,eAAO,MAAM,IAAI,YAAY,CAAC;AAE9B,eAAO,MAAM,WAAW,yGACgF,CAAC;AAEzG,eAAO,MAAM,MAAM;;;;;;;;;;;;;;;;;;EAiBjB,CAAC;AAEH,MAAM,MAAM,YAAY,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,MAAM,CAAC,CAAC;AAclD,wBAAsB,OAAO,CAAC,KAAK,EAAE,YAAY;;;;;GAgDhD"}
@@ -7,12 +7,11 @@ export const name = "extract";
7
7
  export const description = "Extract structured data from a web page using CSS selectors. Returns JSON with the extracted fields.";
8
8
  export const schema = z.object({
9
9
  url: z.string().max(MAX_URL_LENGTH).describe("The URL to extract data from"),
10
- selectors: z
10
+ selectors: z.preprocess((val) => (typeof val === "string" ? JSON.parse(val) : val), z
11
11
  .record(z.string().max(MAX_SELECTOR_LENGTH))
12
12
  .refine((obj) => Object.keys(obj).length <= MAX_SELECTOR_KEYS, {
13
13
  message: `Too many selectors (max ${MAX_SELECTOR_KEYS})`,
14
- })
15
- .describe("Map of field names to CSS selectors. Use @attr suffix to extract attributes (e.g. 'a @href')"),
14
+ })).describe("Map of field names to CSS selectors. Use @attr suffix to extract attributes (e.g. 'a @href')"),
16
15
  items_selector: z
17
16
  .string()
18
17
  .max(MAX_SELECTOR_LENGTH)
@@ -1 +1 @@
1
- {"version":3,"file":"extract.js","sourceRoot":"","sources":["../../src/tools/extract.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAAE,SAAS,EAAE,MAAM,qBAAqB,CAAC;AAChD,OAAO,EAAE,YAAY,EAAE,MAAM,iBAAiB,CAAC;AAC/C,OAAO,EAAE,cAAc,EAAE,mBAAmB,EAAE,iBAAiB,EAAE,MAAM,iBAAiB,CAAC;AACzF,OAAO,KAAK,OAAO,MAAM,SAAS,CAAC;AAGnC,MAAM,CAAC,MAAM,IAAI,GAAG,SAAS,CAAC;AAE9B,MAAM,CAAC,MAAM,WAAW,GACtB,sGAAsG,CAAC;AAEzG,MAAM,CAAC,MAAM,MAAM,GAAG,CAAC,CAAC,MAAM,CAAC;IAC7B,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,cAAc,CAAC,CAAC,QAAQ,CAAC,8BAA8B,CAAC;IAC5E,SAAS,EAAE,CAAC;SACT,MAAM,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,mBAAmB,CAAC,CAAC;SAC3C,MAAM,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,MAAM,IAAI,iBAAiB,EAAE;QAC7D,OAAO,EAAE,2BAA2B,iBAAiB,GAAG;KACzD,CAAC;SACD,QAAQ,CAAC,8FAA8F,CAAC;IAC3G,cAAc,EAAE,CAAC;SACd,MAAM,EAAE;SACR,GAAG,CAAC,mBAAmB,CAAC;SACxB,QAAQ,EAAE;SACV,QAAQ,CAAC,yEAAyE,CAAC;IACtF,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,cAAc,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,oEAAoE,CAAC;IAC/H,cAAc,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,+HAA+H,CAAC;CAC1L,CAAC,CAAC;AAIH,SAAS,YAAY,CAAC,CAAqB,EAAE,EAA4B,EAAE,WAAmB;IAC5F,MAAM,KAAK,GAAG,WAAW,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IACtC,MAAM,QAAQ,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IACjC,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC;IAE9B,MAAM,MAAM,GAAG,QAAQ,CAAC,CAAC,CAAC,EAAE,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;IACjD,IAAI,IAAI,EAAE,CAAC;QACT,OAAO,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC;IACjC,CAAC;IACD,OAAO,MAAM,CAAC,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC;AAC9B,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,OAAO,CAAC,KAAmB;IAC/C,MAAM,GAAG,GAAG,YAAY,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IACpC,MAAM,MAAM,GAAG,MAAM,SAAS,CAAC,GAAG,EAAE,EAAE,KAAK,EAAE,KAAK,CAAC,KAAK,EAAE,aAAa,EAAE,KAAK,CAAC,cAAc,EAAE,CAAC,CAAC;IACjG,MAAM,CAAC,GAAG,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;IAEpC,IAAI,KAAK,CAAC,cAAc,EAAE,CAAC;QACzB,wBAAwB;QACxB,MAAM,KAAK,GAA6B,EAAE,CAAC;QAC3C,CAAC,CAAC,KAAK,CAAC,cAAc,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,EAAE,EAAE;YACrC,MAAM,IAAI,GAA2B,EAAE,CAAC;YACxC,KAAK,MAAM,CAAC,KAAK,EAAE,QAAQ,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,KAAK,CAAC,SAAS,CAAC,EAAE,CAAC;gBAChE,IAAI,CAAC,KAAK,CAAC,GAAG,YAAY,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,CAAC,EAAE,QAAQ,CAAC,CAAC;YACjD,CAAC;YACD,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACnB,CAAC,CAAC,CAAC;QAEH,OAAO;YACL,OAAO,EAAE;gBACP;oBACE,IAAI,EAAE,MAAe;oBACrB,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,EAAE,GAAG,EAAE,MAAM,CAAC,GAAG,EAAE,WAAW,EAAE,KAAK,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,CAAC,CAAC;iBACrF;aACF;SACF,CAAC;IACJ,CAAC;IAED,wBAAwB;IACxB,MAAM,IAAI,GAA2B,EAAE,CAAC;IACxC,KAAK,MAAM,CAAC,KAAK,EAAE,WAAW,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,KAAK,CAAC,SAAS,CAAC,EAAE,CAAC;QACnE,MAAM,KAAK,GAAG,WAAW,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACtC,MAAM,QAAQ,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACjC,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC;QAE9B,IAAI,IAAI,EAAE,CAAC;YACT,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,QAAQ,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC;QAC7C,CAAC;aAAM,CAAC;YACN,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,QAAQ,CAAC,CAAC,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC;QAC1C,CAAC;IACH,CAAC;IAED,OAAO;QACL,OAAO,EAAE;YACP;gBACE,IAAI,EAAE,MAAe;gBACrB,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,EAAE,GAAG,EAAE,MAAM,CAAC,GAAG,EAAE,IAAI,EAAE,EAAE,IAAI,EAAE,CAAC,CAAC;aACzD;SACF;KACF,CAAC;AACJ,CAAC"}
1
+ {"version":3,"file":"extract.js","sourceRoot":"","sources":["../../src/tools/extract.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAAE,SAAS,EAAE,MAAM,qBAAqB,CAAC;AAChD,OAAO,EAAE,YAAY,EAAE,MAAM,iBAAiB,CAAC;AAC/C,OAAO,EAAE,cAAc,EAAE,mBAAmB,EAAE,iBAAiB,EAAE,MAAM,iBAAiB,CAAC;AACzF,OAAO,KAAK,OAAO,MAAM,SAAS,CAAC;AAGnC,MAAM,CAAC,MAAM,IAAI,GAAG,SAAS,CAAC;AAE9B,MAAM,CAAC,MAAM,WAAW,GACtB,sGAAsG,CAAC;AAEzG,MAAM,CAAC,MAAM,MAAM,GAAG,CAAC,CAAC,MAAM,CAAC;IAC7B,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,cAAc,CAAC,CAAC,QAAQ,CAAC,8BAA8B,CAAC;IAC5E,SAAS,EAAE,CAAC,CAAC,UAAU,CACrB,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC,OAAO,GAAG,KAAK,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,EAC1D,CAAC;SACE,MAAM,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,mBAAmB,CAAC,CAAC;SAC3C,MAAM,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,MAAM,IAAI,iBAAiB,EAAE;QAC7D,OAAO,EAAE,2BAA2B,iBAAiB,GAAG;KACzD,CAAC,CACL,CAAC,QAAQ,CAAC,8FAA8F,CAAC;IAC1G,cAAc,EAAE,CAAC;SACd,MAAM,EAAE;SACR,GAAG,CAAC,mBAAmB,CAAC;SACxB,QAAQ,EAAE;SACV,QAAQ,CAAC,yEAAyE,CAAC;IACtF,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,cAAc,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,oEAAoE,CAAC;IAC/H,cAAc,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,QAAQ,EAAE,CAAC,QAAQ,CAAC,+HAA+H,CAAC;CAC1L,CAAC,CAAC;AAIH,SAAS,YAAY,CAAC,CAAqB,EAAE,EAA4B,EAAE,WAAmB;IAC5F,MAAM,KAAK,GAAG,WAAW,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IACtC,MAAM,QAAQ,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IACjC,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC;IAE9B,MAAM,MAAM,GAAG,QAAQ,CAAC,CAAC,CAAC,EAAE,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;IACjD,IAAI,IAAI,EAAE,CAAC;QACT,OAAO,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC;IACjC,CAAC;IACD,OAAO,MAAM,CAAC,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC;AAC9B,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,OAAO,CAAC,KAAmB;IAC/C,MAAM,GAAG,GAAG,YAAY,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IACpC,MAAM,MAAM,GAAG,MAAM,SAAS,CAAC,GAAG,EAAE,EAAE,KAAK,EAAE,KAAK,CAAC,KAAK,EAAE,aAAa,EAAE,KAAK,CAAC,cAAc,EAAE,CAAC,CAAC;IACjG,MAAM,CAAC,GAAG,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;IAEpC,IAAI,KAAK,CAAC,cAAc,EAAE,CAAC;QACzB,wBAAwB;QACxB,MAAM,KAAK,GAA6B,EAAE,CAAC;QAC3C,CAAC,CAAC,KAAK,CAAC,cAAc,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,EAAE,EAAE;YACrC,MAAM,IAAI,GAA2B,EAAE,CAAC;YACxC,KAAK,MAAM,CAAC,KAAK,EAAE,QAAQ,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,KAAK,CAAC,SAAS,CAAC,EAAE,CAAC;gBAChE,IAAI,CAAC,KAAK,CAAC,GAAG,YAAY,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,CAAC,EAAE,QAAQ,CAAC,CAAC;YACjD,CAAC;YACD,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACnB,CAAC,CAAC,CAAC;QAEH,OAAO;YACL,OAAO,EAAE;gBACP;oBACE,IAAI,EAAE,MAAe;oBACrB,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,EAAE,GAAG,EAAE,MAAM,CAAC,GAAG,EAAE,WAAW,EAAE,KAAK,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,CAAC,CAAC;iBACrF;aACF;SACF,CAAC;IACJ,CAAC;IAED,wBAAwB;IACxB,MAAM,IAAI,GAA2B,EAAE,CAAC;IACxC,KAAK,MAAM,CAAC,KAAK,EAAE,WAAW,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,KAAK,CAAC,SAAS,CAAC,EAAE,CAAC;QACnE,MAAM,KAAK,GAAG,WAAW,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACtC,MAAM,QAAQ,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACjC,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC;QAE9B,IAAI,IAAI,EAAE,CAAC;YACT,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,QAAQ,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC;QAC7C,CAAC;aAAM,CAAC;YACN,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,QAAQ,CAAC,CAAC,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC;QAC1C,CAAC;IACH,CAAC;IAED,OAAO;QACL,OAAO,EAAE;YACP;gBACE,IAAI,EAAE,MAAe;gBACrB,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,EAAE,GAAG,EAAE,MAAM,CAAC,GAAG,EAAE,IAAI,EAAE,EAAE,IAAI,EAAE,CAAC,CAAC;aACzD;SACF;KACF,CAAC;AACJ,CAAC"}
@@ -1 +1 @@
1
- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/tools/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAoBxB,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,WAAW,EAAE,MAAM,CAAC;IACpB,MAAM,EAAE,CAAC,CAAC,OAAO,CAAC;IAElB,OAAO,EAAE,CAAC,KAAK,EAAE,GAAG,KAAK,OAAO,CAAC;QAAE,OAAO,EAAE,KAAK,CAAC;YAAE,IAAI,EAAE,MAAM,CAAC;YAAC,IAAI,CAAC,EAAE,MAAM,CAAC;YAAC,IAAI,CAAC,EAAE,MAAM,CAAC;YAAC,QAAQ,CAAC,EAAE,MAAM,CAAA;SAAE,CAAC,CAAA;KAAE,CAAC,CAAC;CACzH;AAED,eAAO,MAAM,QAAQ,EAAE,cAAc,EAuBpC,CAAC"}
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/tools/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAmBxB,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,WAAW,EAAE,MAAM,CAAC;IACpB,MAAM,EAAE,CAAC,CAAC,OAAO,CAAC;IAElB,OAAO,EAAE,CAAC,KAAK,EAAE,GAAG,KAAK,OAAO,CAAC;QAAE,OAAO,EAAE,KAAK,CAAC;YAAE,IAAI,EAAE,MAAM,CAAC;YAAC,IAAI,CAAC,EAAE,MAAM,CAAC;YAAC,IAAI,CAAC,EAAE,MAAM,CAAC;YAAC,QAAQ,CAAC,EAAE,MAAM,CAAA;SAAE,CAAC,CAAA;KAAE,CAAC,CAAC;CACzH;AAED,eAAO,MAAM,QAAQ,EAAE,cAAc,EAqBpC,CAAC"}
@@ -14,7 +14,6 @@ import * as listSkills from "./list-skills.js";
14
14
  import * as discoverApis from "./discover-apis.js";
15
15
  import * as queryApi from "./query-api.js";
16
16
  import * as monitorWebsocket from "./monitor-websocket.js";
17
- import * as visualBuilder from "./visual-builder.js";
18
17
  export const allTools = [
19
18
  // Scraping tools (no API key needed)
20
19
  scrape,
@@ -36,7 +35,5 @@ export const allTools = [
36
35
  discoverApis,
37
36
  queryApi,
38
37
  monitorWebsocket,
39
- // Visual builder (interactive, opens headed browser)
40
- visualBuilder,
41
38
  ];
42
39
  //# sourceMappingURL=index.js.map
@@ -1 +1 @@
1
- {"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/tools/index.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,MAAM,MAAM,aAAa,CAAC;AACtC,OAAO,KAAK,KAAK,MAAM,YAAY,CAAC;AACpC,OAAO,KAAK,GAAG,MAAM,UAAU,CAAC;AAChC,OAAO,KAAK,OAAO,MAAM,cAAc,CAAC;AACxC,OAAO,KAAK,WAAW,MAAM,kBAAkB,CAAC;AAChD,OAAO,KAAK,UAAU,MAAM,iBAAiB,CAAC;AAC9C,OAAO,KAAK,MAAM,MAAM,aAAa,CAAC;AACtC,OAAO,KAAK,UAAU,MAAM,kBAAkB,CAAC;AAC/C,OAAO,KAAK,WAAW,MAAM,mBAAmB,CAAC;AACjD,OAAO,KAAK,WAAW,MAAM,mBAAmB,CAAC;AACjD,OAAO,KAAK,WAAW,MAAM,mBAAmB,CAAC;AACjD,OAAO,KAAK,QAAQ,MAAM,gBAAgB,CAAC;AAC3C,OAAO,KAAK,UAAU,MAAM,kBAAkB,CAAC;AAC/C,OAAO,KAAK,YAAY,MAAM,oBAAoB,CAAC;AACnD,OAAO,KAAK,QAAQ,MAAM,gBAAgB,CAAC;AAC3C,OAAO,KAAK,gBAAgB,MAAM,wBAAwB,CAAC;AAC3D,OAAO,KAAK,aAAa,MAAM,qBAAqB,CAAC;AAUrD,MAAM,CAAC,MAAM,QAAQ,GAAqB;IACxC,qCAAqC;IACrC,MAAwB;IACxB,KAAuB;IACvB,GAAqB;IACrB,OAAyB;IACzB,WAA6B;IAC7B,UAA4B;IAC5B,sCAAsC;IACtC,MAAwB;IACxB,UAA4B;IAC5B,WAA6B;IAC7B,WAA6B;IAC7B,eAAe;IACf,WAA6B;IAC7B,QAA0B;IAC1B,UAA4B;IAC5B,sDAAsD;IACtD,YAA8B;IAC9B,QAA0B;IAC1B,gBAAkC;IAClC,qDAAqD;IACrD,aAA+B;CAChC,CAAC"}
1
+ {"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/tools/index.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,MAAM,MAAM,aAAa,CAAC;AACtC,OAAO,KAAK,KAAK,MAAM,YAAY,CAAC;AACpC,OAAO,KAAK,GAAG,MAAM,UAAU,CAAC;AAChC,OAAO,KAAK,OAAO,MAAM,cAAc,CAAC;AACxC,OAAO,KAAK,WAAW,MAAM,kBAAkB,CAAC;AAChD,OAAO,KAAK,UAAU,MAAM,iBAAiB,CAAC;AAC9C,OAAO,KAAK,MAAM,MAAM,aAAa,CAAC;AACtC,OAAO,KAAK,UAAU,MAAM,kBAAkB,CAAC;AAC/C,OAAO,KAAK,WAAW,MAAM,mBAAmB,CAAC;AACjD,OAAO,KAAK,WAAW,MAAM,mBAAmB,CAAC;AACjD,OAAO,KAAK,WAAW,MAAM,mBAAmB,CAAC;AACjD,OAAO,KAAK,QAAQ,MAAM,gBAAgB,CAAC;AAC3C,OAAO,KAAK,UAAU,MAAM,kBAAkB,CAAC;AAC/C,OAAO,KAAK,YAAY,MAAM,oBAAoB,CAAC;AACnD,OAAO,KAAK,QAAQ,MAAM,gBAAgB,CAAC;AAC3C,OAAO,KAAK,gBAAgB,MAAM,wBAAwB,CAAC;AAU3D,MAAM,CAAC,MAAM,QAAQ,GAAqB;IACxC,qCAAqC;IACrC,MAAwB;IACxB,KAAuB;IACvB,GAAqB;IACrB,OAAyB;IACzB,WAA6B;IAC7B,UAA4B;IAC5B,sCAAsC;IACtC,MAAwB;IACxB,UAA4B;IAC5B,WAA6B;IAC7B,WAA6B;IAC7B,eAAe;IACf,WAA6B;IAC7B,QAA0B;IAC1B,UAA4B;IAC5B,sDAAsD;IACtD,YAA8B;IAC9B,QAA0B;IAC1B,gBAAkC;CACnC,CAAC"}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "imperium-crawl",
3
- "version": "1.1.0",
3
+ "version": "1.1.2",
4
4
  "description": "Open-source MCP server with Firecrawl-like scraping, crawling, search, and custom skills",
5
5
  "type": "module",
6
6
  "bin": {