imperium-crawl 1.1.8 → 1.1.9

package/README.md

@@ -79,6 +79,30 @@ npm i rebrowser-playwright
 npx playwright install chromium
 ```
 
+### AI Agent Guide (SKILL.md)
+
+imperium-crawl ships with [`SKILL.md`](./SKILL.md) — a structured guide that teaches AI agents (Claude, GPT, etc.) how to use all 16 tools effectively. It includes 6 proven workflows, decision trees, error recovery strategies, and advanced patterns like manual skill refinement.
+
+**Without SKILL.md**, agents can call tools but won't know which tool to try first, when to fallback, or how to chain tools together optimally.
+
+**With SKILL.md**, agents follow battle-tested workflows — readability → scrape → extract fallback chains, auto-detect → manual refinement for skills, search → select → deep-scrape for research, and more.
+
+**Two ways to connect SKILL.md to any agent:**
+
+| Method | Setup | Works with |
+|--------|-------|-----------|
+| **MCP + SKILL.md** | Add imperium-crawl as MCP server + SKILL.md in agent context | Claude Code, Cursor, Windsurf, any MCP client |
+| **CLI + SKILL.md** | `npm i -g imperium-crawl` + SKILL.md in agent context | **Any agent with bash access** — OpenClaw, ChatGPT, GPT agents, custom agents, anything |
+
+The CLI approach is universal — any agent that can run shell commands can use all 16 tools. No MCP required.
+
+| AI Agent | How to add SKILL.md |
+|----------|-------------------|
+| **Claude Code** | Copy `SKILL.md` to your project root — Claude Code reads it automatically |
+| **Cursor / Windsurf** | Add `SKILL.md` to project rules or include in system prompt |
+| **OpenClaw / custom agents** | Include SKILL.md content in your system prompt or context window |
+| **ChatGPT / GPT agents** | Paste SKILL.md content into custom instructions |
+
 ---
 
 ## CLI Mode
@@ -332,27 +356,22 @@ Every tool tested against production websites with real anti-bot defenses:
 
 | Tool | Target | Result |
 |------|--------|--------|
-| 🕷️ **extract** | Amazon (AirPods Pro 2) | Product title, 45,297 reviews, brand extracted |
-| 🔓 **discover_apis** | Spotify | **8 hidden APIs** — access token exposed, client ID, dealer servers, analytics |
-| 🕷️ **extract** | Stack Overflow | **15 top questions** — #1 with 27,520 votes |
-| 📡 **monitor_websocket** | Binance BTC/USDT | **3 WebSocket connections, 23 live messages** — real-time price $69,390 |
-| 🔓 **discover_apis** | Airbnb Paris | **34 hidden APIs** — DataDome anti-bot, Google Maps key exposed, internal search/polygon/viewport APIs |
-| 🕷️ **extract** | Hacker News | **30 front-page posts** — titles + URLs extracted |
-| 🔓 **discover_apis** | Netflix | **5 APIs** — OneTrust consent, geolocation (detected country: Serbia 🇷🇸) |
 | 📄 **scrape** | BBC News | Full markdown content, stealth level 3 auto-escalation |
 | 🕸️ **crawl** | Cloudflare Blog | **213K characters** crawled with depth control |
 | 🗺️ **map** | BBC | Full URL discovery via sitemap + page link extraction |
+| 🕷️ **extract** | Amazon (AirPods Pro 2) | Product title, 45,297 reviews, brand extracted |
 | 📖 **readability** | Medium article | Clean extraction — title, author, content, publish date |
 | 📸 **screenshot** | ProductHunt | Captured Cloudflare Turnstile challenge page |
-| 🔓 **discover_apis** | weather.com | **11 hidden APIs** — main weather API with exposed key |
-| ⚡ **query_api** | jsonplaceholder | Direct JSON API call with stealth headers |
 | 🔍 **search** | Brave Web Search | Web results with snippets and URLs |
 | 📰 **news_search** | Brave News Search | News results with freshness ranking |
 | 🖼️ **image_search** | Brave Image Search | Images with thumbnails and source URLs |
 | 🎬 **video_search** | Brave Video Search | Video results across platforms |
-| 🛠️ **create_skill** | Any page | Auto-detects repeating patterns, generates CSS selectors |
+| 🛠️ **create_skill** | Hacker News | Auto-detected 30 repeating stories with CSS selectors |
 | ▶️ **run_skill** | Saved skill | Fresh structured data from saved extraction config |
 | 📋 **list_skills** | — | Lists all saved skills with configurations |
+| 🔓 **discover_apis** | Airbnb Paris | **34 hidden APIs** — DataDome anti-bot, Google Maps key, internal APIs |
+| ⚡ **query_api** | jsonplaceholder | Direct JSON API call with stealth headers |
+| 📡 **monitor_websocket** | Binance BTC/USDT | **3 WebSocket connections, 23 live messages** — BTC price live |
 
 > 🏆 **16/16 tools working. 58 hidden APIs discovered. Live crypto feed captured. Zero API keys needed for scraping.**
 

package/dist/constants.d.ts

@@ -1,5 +1,5 @@
 export declare const PACKAGE_NAME = "imperium-crawl";
-export declare const PACKAGE_VERSION = "1.1.8";
+export declare const PACKAGE_VERSION = "1.1.9";
 export declare const DEFAULT_TIMEOUT_MS = 30000;
 export declare const DEFAULT_MAX_PAGES = 10;
 export declare const DEFAULT_MAX_DEPTH = 2;

package/dist/constants.js

@@ -1,5 +1,5 @@
 export const PACKAGE_NAME = "imperium-crawl";
-export const PACKAGE_VERSION = "1.1.8";
+export const PACKAGE_VERSION = "1.1.9";
 export const DEFAULT_TIMEOUT_MS = 30_000;
 export const DEFAULT_MAX_PAGES = 10;
 export const DEFAULT_MAX_DEPTH = 2;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "imperium-crawl",
-  "version": "1.1.8",
+  "version": "1.1.9",
   "description": "Open-source MCP server with Firecrawl-like scraping, crawling, search, and custom skills",
   "type": "module",
   "bin": {