webpeel 0.6.1 → 0.7.1

package/README.md

@@ -1,236 +1,125 @@
-# WebPeel
+<p align="center">
+  <a href="https://webpeel.dev">
+    <img src=".github/banner.svg" alt="WebPeel — Web fetching for AI agents" width="100%">
+  </a>
+</p>
+
+<p align="center">
+  <a href="https://www.npmjs.com/package/webpeel"><img src="https://img.shields.io/npm/v/webpeel.svg" alt="npm version"></a>
+  <a href="https://pypi.org/project/webpeel/"><img src="https://img.shields.io/pypi/v/webpeel.svg" alt="PyPI version"></a>
+  <a href="https://www.npmjs.com/package/webpeel"><img src="https://img.shields.io/npm/dm/webpeel.svg" alt="npm downloads"></a>
+  <a href="https://github.com/webpeel/webpeel/stargazers"><img src="https://img.shields.io/github/stars/webpeel/webpeel.svg" alt="GitHub stars"></a>
+  <a href="https://github.com/webpeel/webpeel/actions/workflows/ci.yml"><img src="https://github.com/webpeel/webpeel/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
+  <a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/TypeScript-5.6-blue.svg" alt="TypeScript"></a>
+  <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="MIT License"></a>
+</p>
+
+<p align="center">
+  <b>Turn any web page into AI-ready markdown. Smart escalation. Stealth mode. Free to start.</b>
+</p>
+
+<p align="center">
+  <a href="https://webpeel.dev">Website</a> ·
+  <a href="https://webpeel.dev/docs">Docs</a> ·
+  <a href="https://webpeel.dev/playground">Playground</a> ·
+  <a href="https://app.webpeel.dev">Dashboard</a> ·
+  <a href="https://github.com/webpeel/webpeel/discussions">Discussions</a>
+</p>
 
-[![npm version](https://img.shields.io/npm/v/webpeel.svg)](https://www.npmjs.com/package/webpeel)
-[![npm downloads](https://img.shields.io/npm/dm/webpeel.svg)](https://www.npmjs.com/package/webpeel)
-[![GitHub stars](https://img.shields.io/github/stars/JakeLiuMe/webpeel.svg)](https://github.com/JakeLiuMe/webpeel/stargazers)
-[![CI](https://github.com/JakeLiuMe/webpeel/actions/workflows/ci.yml/badge.svg)](https://github.com/JakeLiuMe/webpeel/actions/workflows/ci.yml)
-[![TypeScript](https://img.shields.io/badge/TypeScript-5.6-blue.svg)](https://www.typescriptlang.org/)
-[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
+---
 
-Turn any web page into clean markdown. **Smart escalation. Stealth mode. Crawl mode. Free to start.**
+## Quick Start
 
 ```bash
+# Zero install — just run it
 npx webpeel https://news.ycombinator.com
 ```
 
-**Output:**
-```markdown
-# Hacker News
-
-**New** | **Past** | **Comments** | **Ask** | **Show** | **Jobs** | **Submit**
-
-## Top Stories
-
-1. **Show HN: WebPeel – Turn any webpage into AI-ready markdown**
-   [https://github.com/JakeLiuMe/webpeel](https://github.com/JakeLiuMe/webpeel)
-   142 points by jakeliu 2 hours ago | 31 comments
-
-2. **The End of the API Era**
-   ...
-```
-
----
-
-## Why WebPeel?
-
-|  | **WebPeel** | Firecrawl | Jina Reader | MCP Fetch |
-|---|:---:|:---:|:---:|:---:|
-| **Free tier** | ✅ 125/week | 500 one-time | ❌ Cloud only | ✅ Unlimited |
-| **Smart escalation** | ✅ HTTP→Browser→Stealth | Manual mode | ❌ No | ❌ No |
-| **Stealth mode** | ✅ All plans | ✅ Yes | ⚠️ Limited | ❌ No |
-| **Crawl + Map** | ✅ All plans | ✅ Yes | ❌ No | ❌ No |
-| **AI Extraction** | ✅ BYOK (any LLM) | ✅ Built-in | ❌ No | ❌ No |
-| **Branding** | ✅ Design system | ✅ Yes | ❌ No | ❌ No |
-| **Change Tracking** | ✅ Local snapshots | ✅ Server-side | ❌ No | ❌ No |
-| **Python SDK** | ✅ Zero deps | ✅ httpx/pydantic | ❌ No | ❌ No |
-| **LangChain** | ✅ Official | ✅ Official | ❌ No | ❌ No |
-| **MCP Server** | ✅ Built-in (6 tools) | ✅ Separate repo | ❌ No | ✅ Yes |
-| **Token Budget** | ✅ `--max-tokens` | ❌ No | ❌ No | ❌ No |
-| **Zero config** | ✅ `npx webpeel` | ❌ API key required | ❌ API key required | ✅ Yes |
-| **Pricing** | $0 local / $9-$29 | $16-$333/mo | $10/mo+ | Free |
-| **License** | MIT | AGPL-3.0 | Proprietary | MIT |
-
-**WebPeel gives you Firecrawl's power with a generous free tier and MIT license.**
-
-### Usage Model
-
-WebPeel uses a **weekly usage budget** for all users (CLI and API):
-
-- **First 25 fetches**: No account needed — try it instantly
-- **Free tier**: 125 fetches/week (resets every Monday)
-- **Pro tier**: 1,250 fetches/week ($9/mo)
-- **Max tier**: 6,250 fetches/week ($29/mo)
-
-**Credit costs**: Basic fetch = 1 credit, Stealth mode = 5 credits, Search = 1 credit, Crawl = 1 credit/page
-
-**Open source**: The CLI is MIT licensed — you can self-host if needed. But the hosted API requires authentication after 25 fetches.
-
-### Highlights
-
-1. **🎭 Stealth Mode** — Bypass bot detection with playwright-extra stealth plugin. Works on sites that block regular scrapers.
-2. **🕷️ Crawl Mode** — Follow links and extract entire sites. Respects robots.txt and rate limits automatically.
-3. **💰 Generous Free Tier** — 125 free fetches every week. First 25 work instantly with no signup. Basic fetch + JS rendering included free.
-
----
-
-## Quick Start
-
-### CLI (Zero Install)
-
 ```bash
-# First 25 fetches work instantly, no signup
-npx webpeel https://example.com
-
-# After 25 fetches, sign up for free (125/week)
-webpeel login
-
-# Check your usage
-webpeel usage
-
 # Stealth mode (bypass bot detection)
 npx webpeel https://protected-site.com --stealth
 
-# Page actions: click, scroll, type before extraction (v0.4.0)
-npx webpeel https://example.com --action "click:.cookie-accept" --action "wait:2000" --action "scroll:bottom"
-
-# Structured data extraction with CSS selectors (v0.4.0)
-npx webpeel https://example.com --extract '{"title": "h1", "price": ".price", "description": ".desc"}'
-
-# Token budget: truncate output to max tokens (v0.4.0)
-npx webpeel https://example.com --max-tokens 2000
-
-# Map discovery: find all URLs on a domain via sitemap & crawling (v0.4.0)
-npx webpeel map https://example.com --max-urls 5000
-
-# Extract branding/design system from a page (v0.5.0)
-npx webpeel brand https://example.com
+# Crawl a website
+npx webpeel crawl https://example.com --max-pages 20
 
-# Track content changes over time (v0.5.0)
-npx webpeel track https://example.com
+# Search the web
+npx webpeel search "best AI frameworks 2026"
 
-# Crawl a website (follow links, respect robots.txt)
-npx webpeel crawl https://example.com --max-pages 20 --max-depth 2
-
-# Sitemap-first crawl with content deduplication (v0.4.0)
-npx webpeel crawl https://example.com --sitemap-first --max-pages 100
-
-# JSON output with metadata
-npx webpeel https://example.com --json
+# Autonomous agent (BYOK LLM)
+npx webpeel agent "Find the founders of Stripe" --llm-key sk-...
+```
 
-# Cache results locally (avoid repeat fetches)
-npx webpeel https://example.com --cache 5m
+First 25 fetches work instantly, no signup. After that, [sign up free](https://app.webpeel.dev/signup) for 125/week.
 
-# Extract just the links from a page
-npx webpeel https://example.com --links
+## Why WebPeel?
 
-# Extract just the metadata (title, description, author)
-npx webpeel https://example.com --meta
+| Feature | **WebPeel** | Firecrawl | Jina Reader | MCP Fetch |
+|---------|:-----------:|:---------:|:-----------:|:---------:|
+| **Free tier** | ✅ 125/wk recurring | 500 one-time | ❌ Cloud only | ✅ Unlimited |
+| **Smart escalation** | ✅ HTTP→Browser→Stealth | Manual | ❌ | ❌ |
+| **Stealth mode** | ✅ All plans | ✅ | ⚠️ Limited | ❌ |
+| **Firecrawl-compatible** | ✅ Drop-in replacement | ✅ Native | ❌ | ❌ |
+| **Self-hosting** | ✅ Docker compose | ⚠️ Complex | ❌ | N/A |
+| **Autonomous agent** | ✅ BYOK any LLM | ⚠️ Locked | ❌ | ❌ |
+| **MCP tools** | ✅ 9 tools | 3 | 0 | 1 |
+| **License** | ✅ MIT | AGPL-3.0 | Proprietary | MIT |
+| **Pricing** | **Free / $9 / $29** | $0 / $16 / $83 | Custom | Free |
 
-# Batch fetch from file or stdin
-cat urls.txt | npx webpeel batch
+## Install
 
-# Force browser rendering (for JS-heavy sites)
-npx webpeel https://x.com/elonmusk --render
+```bash
+# Node.js
+npm install webpeel        # or: pnpm add webpeel
 
-# Wait for dynamic content
-npx webpeel https://example.com --render --wait 3000
+# Python
+pip install webpeel
 
-# View your config and cache stats
-webpeel config
+# Global CLI
+npm install -g webpeel
 ```
 
-### Library (TypeScript)
+## Usage
 
-```bash
-npm install webpeel
-```
+### Node.js
 
 ```typescript
 import { peel } from 'webpeel';
 
-// Simple usage
 const result = await peel('https://example.com');
 console.log(result.content);    // Clean markdown
 console.log(result.metadata);   // { title, description, author, ... }
 console.log(result.tokens);     // Estimated token count
 
-// Branding extraction (v0.5.0)
-const brand = await peel('https://stripe.com', { branding: true, render: true });
-console.log(brand.branding);    // { colors, fonts, typography, cssVariables, ... }
-
-// Change tracking (v0.5.0)
-const tracked = await peel('https://example.com/pricing', { changeTracking: true });
-console.log(tracked.changeTracking);  // { changeStatus: 'new' | 'same' | 'changed', diff: ... }
-
-// AI extraction with your own LLM key (v0.5.0)
-const extracted = await peel('https://example.com', {
-  extract: { prompt: 'Extract the pricing plans', llmApiKey: 'sk-...' },
+// With options
+const advanced = await peel('https://example.com', {
+  render: true,           // Browser for JS-heavy sites
+  stealth: true,          // Anti-bot stealth mode
+  maxTokens: 4000,        // Limit output
+  includeTags: ['main'],  // Filter HTML tags
 });
-console.log(extracted.extracted);
 ```
 
-### Python SDK (v0.5.0)
-
-```bash
-pip install webpeel
-```
+### Python
 
 ```python
 from webpeel import WebPeel
 
-client = WebPeel()  # Free tier, no API key needed
+client = WebPeel()  # Free tier, no key needed
 
-# Scrape
 result = client.scrape("https://example.com")
 print(result.content)  # Clean markdown
 
-# Search
 results = client.search("python web scraping")
-
-# Crawl (async job)
 job = client.crawl("https://docs.example.com", limit=100)
-status = client.get_job(job.id)
-```
-
-Zero dependencies. Pure Python 3.8+ stdlib. [Full docs →](python-sdk/README.md)
-
-### MCP Server (Claude Desktop, Cursor, VS Code, Windsurf)
-
-WebPeel provides six MCP tools: `webpeel_fetch`, `webpeel_search`, `webpeel_crawl`, `webpeel_map`, `webpeel_extract`, and `webpeel_batch`.
-
-#### Claude Desktop
-
-Add to `~/Library/Application Support/Claude/claude_desktop_config.json`:
-
-```json
-{
-  "mcpServers": {
-    "webpeel": {
-      "command": "npx",
-      "args": ["-y", "webpeel", "mcp"]
-    }
-  }
-}
 ```
 
-#### Cursor
+Zero dependencies. Pure Python 3.8+. [Full SDK docs →](python-sdk/README.md)
 
-Add to Cursor Settings → MCP Servers:
-
-```json
-{
-  "mcpServers": {
-    "webpeel": {
-      "command": "npx",
-      "args": ["-y", "webpeel", "mcp"]
-    }
-  }
-}
-```
+### MCP Server
 
-#### VS Code (with Cline or other MCP clients)
+9 tools for Claude Desktop, Cursor, VS Code, and Windsurf:
 
-Create or edit `~/.vscode/mcp.json`:
+`webpeel_fetch` · `webpeel_search` · `webpeel_crawl` · `webpeel_map` · `webpeel_extract` · `webpeel_batch` · `webpeel_agent` · `webpeel_summarize` · `webpeel_brand`
 
 ```json
 {
@@ -243,378 +132,129 @@ Create or edit `~/.vscode/mcp.json`:
 }
 ```
 
-Or install with one click:
-
 [![Install in Claude Desktop](https://img.shields.io/badge/Install-Claude%20Desktop-5B3FFF?style=for-the-badge&logo=anthropic)](https://mcp.so/install/webpeel?for=claude)
 [![Install in VS Code](https://img.shields.io/badge/Install-VS%20Code-007ACC?style=for-the-badge&logo=visualstudiocode)](https://mcp.so/install/webpeel?for=vscode)
 
-#### Windsurf
-
-Add to `~/.codeium/windsurf/mcp_config.json`:
+> **Where to add this config:** Claude Desktop → `~/Library/Application Support/Claude/claude_desktop_config.json` · Cursor → Settings → MCP Servers · VS Code → `~/.vscode/mcp.json` · Windsurf → `~/.codeium/windsurf/mcp_config.json`
 
-```json
-{
-  "mcpServers": {
-    "webpeel": {
-      "command": "npx",
-      "args": ["-y", "webpeel", "mcp"]
-    }
-  }
-}
-```
-
----
-
-## Use with Claude Code
-
-One command to add WebPeel to Claude Code:
+### Docker (Self-Hosted)
 
 ```bash
-claude mcp add webpeel -- npx -y webpeel mcp
-```
-
-Or add to your project's `.mcp.json` for team sharing:
-
-```json
-{
-  "mcpServers": {
-    "webpeel": {
-      "command": "npx",
-      "args": ["-y", "webpeel", "mcp"]
-    }
-  }
-}
-```
-
-This gives Claude Code access to:
-- **webpeel_fetch** — Fetch any URL as clean markdown (with stealth mode, actions, extraction & token budget)
-- **webpeel_search** — Search the web via DuckDuckGo
-- **webpeel_batch** — Fetch multiple URLs concurrently
-- **webpeel_crawl** — Crawl websites following links (with sitemap-first & deduplication)
-- **webpeel_map** — Discover all URLs on a domain via sitemap.xml & link crawling
-- **webpeel_extract** — Extract structured data using CSS selectors or JSON schema
-
----
-
-## How It Works: Smart Escalation
-
-WebPeel tries the fastest method first, then escalates only when needed:
-
+git clone https://github.com/webpeel/webpeel.git
+cd webpeel && docker compose up
 ```
-┌─────────────────────────────────────────────────────────────┐
-│                    Smart Escalation                          │
-└─────────────────────────────────────────────────────────────┘
-
-Simple HTTP Fetch     →     Browser Rendering    →     Stealth Mode
-    ~200ms                      ~2 seconds               ~5 seconds
-       │                            │                       │
-       ├─ User-Agent headers        ├─ Full JS execution   ├─ Anti-detect
-       ├─ Cheerio parsing           ├─ Wait for content    ├─ Fingerprint mask
-       ├─ Fast & cheap              ├─ Screenshots         ├─ Cloudflare bypass
-       │                            │                       │
-       ▼                            ▼                       ▼
-   Works for 80%              Works for 15%            Works for 5%
-   of websites                (JS-heavy sites)         (bot-protected)
-```
-
-**Why this matters:**
-- **Speed**: Don't waste 2 seconds rendering when 200ms will do
-- **Cost**: Headless browsers burn CPU and memory
-- **Reliability**: Auto-retry with browser if simple fetch fails
-
-WebPeel automatically detects blocked requests (403, 503, Cloudflare challenges) and retries with browser mode. You get the best of both worlds.
 
----
+Full API at `http://localhost:3000`. MIT licensed — no restrictions.
 
-## API Reference
+## Features
 
-### `peel(url, options?)`
+### 🎯 Smart Escalation
 
-Fetch and extract content from a URL.
+Automatically uses the fastest method, escalates only when needed:
 
-```typescript
-interface PeelOptions {
-  render?: boolean;        // Force browser mode (default: false)
-  wait?: number;           // Wait time after page load in ms (default: 0)
-  format?: 'markdown' | 'text' | 'html';  // Output format (default: 'markdown')
-  timeout?: number;        // Request timeout in ms (default: 30000)
-  userAgent?: string;      // Custom user agent
-}
-
-interface PeelResult {
-  url: string;             // Final URL (after redirects)
-  title: string;           // Page title
-  content: string;         // Page content in requested format
-  metadata: {              // Extracted metadata
-    description?: string;
-    author?: string;
-    published?: string;    // ISO 8601 date
-    image?: string;        // Open Graph image
-    canonical?: string;
-  };
-  links: string[];         // All links on page (absolute URLs)
-  tokens: number;          // Estimated token count
-  method: 'simple' | 'browser';  // Method used
-  elapsed: number;         // Time taken (ms)
-}
 ```
-
-### Error Types
-
-```typescript
-import { TimeoutError, BlockedError, NetworkError } from 'webpeel';
-
-try {
-  const result = await peel('https://example.com');
-} catch (error) {
-  if (error instanceof TimeoutError) {
-    // Request timed out
-  } else if (error instanceof BlockedError) {
-    // Site blocked the request (403, Cloudflare, etc.)
-  } else if (error instanceof NetworkError) {
-    // Network/DNS error
-  }
-}
+HTTP Fetch (200ms)  →  Browser Rendering (2s)  →  Stealth Mode (5s)
+     80% of sites          15% of sites             5% of sites
 ```
 
-### `cleanup()`
-
-Clean up browser resources. Call this when you're done using WebPeel in your application:
+### 🎭 Stealth Mode
 
-```typescript
-import { peel, cleanup } from 'webpeel';
-
-// ... use peel() ...
+Bypass Cloudflare and bot detection. Masks browser fingerprints, navigator properties, WebGL vendor.
 
-await cleanup();  // Close browser instances
+```bash
+npx webpeel https://protected-site.com --stealth
 ```
 
----
+### 🕷️ Crawl & Map
 
-## Hosted API
-
-Live at `https://api.webpeel.dev` — authentication required after first 25 fetches.
+Crawl websites with link following, sitemap discovery, robots.txt compliance, and deduplication.
 
 ```bash
-# Register and get your API key
-curl -X POST https://api.webpeel.dev/v1/auth/register \
-  -H "Content-Type: application/json" \
-  -d '{"email":"you@example.com","password":"your-password"}'
-
-# Fetch a page
-curl "https://api.webpeel.dev/v1/fetch?url=https://example.com" \
-  -H "Authorization: Bearer wp_live_your_api_key"
+npx webpeel crawl https://docs.example.com --max-pages 100
+npx webpeel map https://example.com --max-urls 5000
 ```
 
-### Pricing — Weekly Reset Model
-
-Usage resets every **Monday at 00:00 UTC**, just like Claude Code.
-
-| Plan | Price | Weekly Fetches | Burst Limit | All Features | Extra Usage |
-|------|------:|---------------:|:-----------:|:------------:|:-----------:|
-| **Free** | $0 | 125/wk (~500/mo) | 25/hr | ✅ | ❌ |
-| **Pro** | $9/mo | 1,250/wk (~5K/mo) | 100/hr | ✅ | ✅ |
-| **Max** | $29/mo | 6,250/wk (~25K/mo) | 500/hr | ✅ | ✅ |
-
-**Three layers of usage control:**
-1. **Burst limit** — Per-hour cap (25/hr free, 100/hr Pro, 500/hr Max) prevents hammering
-2. **Weekly limit** — Main usage gate, resets every Monday
-3. **Extra usage** — When you hit your weekly limit, keep fetching at pay-as-you-go rates
-
-**Extra usage rates (Pro/Max only):**
-| Fetch Type | Cost |
-|-----------|------|
-| Basic (HTTP) | $0.002 |
-| Stealth (browser) | $0.01 |
-| Search | $0.001 |
-
-### Why WebPeel Beats Firecrawl
-
-| Feature | WebPeel Free | WebPeel Pro | Firecrawl Hobby |
-|---------|:-------------:|:-----------:|:---------------:|
-| **Price** | $0 | $9/mo | $16/mo |
-| **Weekly Fetches** | 125/wk | 1,250/wk | ~750/wk |
-| **Rollover** | ❌ | ✅ 1 week | ❌ Expire monthly |
-| **Soft Limits** | ✅ Degrades | ✅ Never locked out | ❌ Hard cut-off |
-| **Extra Usage** | ❌ | ✅ Pay-as-you-go | ❌ Upgrade only |
-| **Self-Host** | ✅ MIT | ✅ MIT | ❌ AGPL |
-
-**Key differentiators:**
-- **Like Claude Code** — Generous free tier (125/week), pay when you need more
-- **Weekly resets** — Your usage refreshes every Monday, not once a month
-- **Soft limits on every tier** — At 100%, we degrade gracefully instead of blocking you
-- **Extra usage** — Pro/Max users can toggle on pay-as-you-go with spending caps (no surprise bills)
-- **First 25 free** — Try it instantly, no signup required
-- **Open source** — MIT licensed, self-host if you want full control
-
-See pricing at [webpeel.dev](https://webpeel.dev/#pricing)
-
----
-
-## Examples
+### 🤖 Autonomous Agent (BYOK)
 
-### Extract blog post metadata
+Give it a prompt, it researches the web using your own LLM key.
 
-```typescript
-const result = await peel('https://example.com/blog/post');
-
-console.log(result.metadata);
-// {
-//   title: "How We Built WebPeel",
-//   description: "A deep dive into smart escalation...",
-//   author: "Jake Liu",
-//   published: "2026-02-12T18:00:00Z",
-//   image: "https://example.com/og-image.png"
-// }
+```bash
+npx webpeel agent "Compare pricing of Notion vs Coda" --llm-key sk-...
 ```
 
-### Get all links from a page
+### 📊 More Features
 
-```typescript
-const result = await peel('https://news.ycombinator.com');
-
-console.log(result.links.slice(0, 5));
-// [
-//   "https://news.ycombinator.com/newest",
-//   "https://news.ycombinator.com/submit",
-//   "https://github.com/example/repo",
-//   ...
-// ]
-```
+| Feature | CLI | Node.js | Python | API |
+|---------|:---:|:-------:|:------:|:---:|
+| Structured extraction | ✅ | ✅ | ✅ | ✅ |
+| Screenshots | ✅ | ✅ | — | ✅ |
+| Branding extraction | ✅ | ✅ | — | — |
+| Change tracking | ✅ | ✅ | — | — |
+| Token budget | ✅ | ✅ | ✅ | ✅ |
+| Tag filtering | ✅ | ✅ | ✅ | ✅ |
+| Image extraction | ✅ | ✅ | — | ✅ |
+| AI summarization | ✅ | ✅ | — | ✅ |
+| Batch processing | — | ✅ | — | ✅ |
+| PDF extraction | ✅ | ✅ | — | — |
 
-### Force browser rendering for JavaScript-heavy sites
-
-```typescript
-// Twitter/X requires JavaScript
-const result = await peel('https://x.com/elonmusk', {
-  render: true,
-  wait: 2000,  // Wait for tweets to load
-});
+## Integrations
 
-console.log(result.content);  // Rendered tweet content
-```
+Works with **LangChain**, **LlamaIndex**, **CrewAI**, **Dify**, and **n8n**. [Integration docs →](https://webpeel.dev/docs)
 
-### Token counting for LLM usage
+## Hosted API
 
-```typescript
-const result = await peel('https://example.com/long-article');
+Live at [`api.webpeel.dev`](https://api.webpeel.dev) — Firecrawl-compatible endpoints.
 
-console.log(`Content is ~${result.tokens} tokens`);
-// Content is ~3,247 tokens
+```bash
+# Fetch a page (free, no auth needed for first 25)
+curl "https://api.webpeel.dev/v1/fetch?url=https://example.com"
 
-if (result.tokens > 4000) {
-  console.log('Too long for GPT-3.5 context window');
-}
+# With API key
+curl "https://api.webpeel.dev/v1/fetch?url=https://example.com" \
+  -H "Authorization: Bearer wp_..."
 ```
 
----
+### Pricing
 
-## Use Cases
+| Plan | Price | Weekly Fetches | Burst | Extra Usage |
+|------|------:|---------------:|:-----:|:-----------:|
+| **Free** | $0 | 125/wk | 25/hr | — |
+| **Pro** | $9/mo | 1,250/wk | 100/hr | ✅ from $0.001 |
+| **Max** | $29/mo | 6,250/wk | 500/hr | ✅ from $0.001 |
 
-- **AI Agents**: Feed web content to Claude, GPT, or local LLMs
-- **Research**: Bulk extract articles, docs, or social media
-- **Monitoring**: Track content changes on websites
-- **Archiving**: Save web pages as clean markdown
-- **Data Pipelines**: Extract structured data from web sources
-
----
+Extra credit costs: fetch $0.002, search $0.001, stealth $0.01. Resets every Monday. All features on all plans. [Compare with Firecrawl →](https://webpeel.dev/migrate-from-firecrawl)
 
 ## Development
 
 ```bash
-# Clone the repo
-git clone https://github.com/JakeLiuMe/webpeel.git
+git clone https://github.com/webpeel/webpeel.git
 cd webpeel
-
-# Install dependencies
-npm install
-
-# Build
-npm run build
-
-# Run tests
+npm install && npm run build
 npm test
-
-# Watch mode (auto-rebuild)
-npm run dev
-
-# Test the CLI locally
-node dist/cli.js https://example.com
-
-# Test the MCP server
-npm run mcp
 ```
 
 See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
 
----
-
-## Roadmap
-
-- [x] CLI with smart escalation
-- [x] TypeScript library
-- [x] MCP server for Claude/Cursor/VS Code
-- [x] Hosted API with authentication and usage tracking
-- [x] Rate limiting and caching
-- [x] Batch processing API (`batch <file>`)
-- [x] Screenshot capture (`--screenshot`)
-- [x] CSS selector filtering (`--selector`, `--exclude`)
-- [x] DuckDuckGo search (`search <query>`)
-- [x] Custom headers and cookies
-- [x] Weekly reset usage model with extra usage
-- [x] Stealth mode (playwright-extra + anti-detect)
-- [x] Crawl mode (follow links, respect robots.txt)
-- [x] PDF extraction (v0.4.0)
-- [x] Structured data extraction with CSS selectors and JSON schema (v0.4.0)
-- [x] Page actions: click, scroll, type, fill, select, press, hover (v0.4.0)
-- [x] Map/sitemap discovery for full site URL mapping (v0.4.0)
-- [x] Token budget for output truncation (v0.4.0)
-- [x] Advanced crawl: sitemap-first, BFS/DFS, content deduplication (v0.4.0)
-- [ ] Webhook notifications for monitoring
-
-Vote on features and roadmap at [GitHub Discussions](https://github.com/JakeLiuMe/webpeel/discussions).
-
----
-
-## FAQ
-
-**Q: How is this different from Firecrawl?**  
-A: WebPeel has a more generous free tier (125/week vs Firecrawl's 500 one-time credits) and uses weekly resets like Claude Code. We also have smart escalation to avoid burning resources on simple pages.
+## Links
 
-**Q: Can I self-host the API server?**  
-A: Yes! Run `npm run serve` to start the API server. See [docs/self-hosting.md](docs/self-hosting.md) (coming soon).
+[Documentation](https://webpeel.dev/docs) · [Playground](https://webpeel.dev/playground) · [API Reference](https://webpeel.dev/docs/api-reference) · [npm](https://www.npmjs.com/package/webpeel) · [PyPI](https://pypi.org/project/webpeel/) · [Migration Guide](https://webpeel.dev/migrate-from-firecrawl) · [Blog](https://webpeel.dev/blog) · [Discussions](https://github.com/webpeel/webpeel/discussions)
 
-**Q: Does this violate websites' Terms of Service?**  
-A: WebPeel is a tool — how you use it is up to you. Always check a site's ToS before fetching at scale. We recommend respecting `robots.txt` in your own workflows.
+## Star History
 
-**Q: What about Cloudflare and bot protection?**  
-A: WebPeel handles most Cloudflare challenges automatically via stealth mode (available on all plans). For heavily protected sites, stealth mode uses browser fingerprint randomization to bypass detection.
-
-**Q: Can I use this in production?**  
-A: Yes! The hosted API at `https://api.webpeel.dev` is production-ready with authentication, rate limiting, and usage tracking.
-
----
-
-## Credits
-
-Built with:
-- [Playwright](https://playwright.dev/) — Headless browser automation
-- [Cheerio](https://cheerio.js.org/) — Fast HTML parsing
-- [Turndown](https://github.com/mixmark-io/turndown) — HTML to Markdown conversion
-- [Commander](https://github.com/tj/commander.js) — CLI framework
-
----
-
-## Contributing
-
-Contributions are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
-
----
+<a href="https://star-history.com/#webpeel/webpeel&Date">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=webpeel/webpeel&type=Date&theme=dark" />
+    <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=webpeel/webpeel&type=Date" />
+    <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=webpeel/webpeel&type=Date" width="600" />
+  </picture>
+</a>
 
 ## License
 
-MIT © [Jake Liu](https://github.com/JakeLiuMe)
+MIT © [WebPeel](https://github.com/webpeel)
 
 ---
 
-**Like WebPeel?** [⭐ Star us on GitHub](https://github.com/JakeLiuMe/webpeel) — it helps others discover the project!
+<p align="center">
+  <b>Like WebPeel?</b> <a href="https://github.com/webpeel/webpeel">⭐ Star us on GitHub</a> — it helps others discover the project!
+</p>