webpeel 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (8) hide show
  1. package/README.md +37 -9
  2. package/dist/core/metadata.d.ts.map +1 -1
  3. package/dist/core/metadata.js +6 -2
  4. package/dist/core/metadata.js.map +1 -1
  5. package/dist/mcp/server.js +14 -0
  6. package/dist/mcp/server.js.map +1 -1
  7. package/llms.txt +28 -2
  8. package/package.json +14 -2
package/README.md CHANGED
@@ -39,8 +39,10 @@ npx webpeel https://news.ycombinator.com
39
39
  | **Anti-bot handling** | ✅ Stealth mode | ✅ Yes | ⚠️ Limited | ❌ No |
40
40
  | **MCP Server** | ✅ Built-in | ✅ Separate repo | ❌ No | ✅ Yes |
41
41
  | **Zero config** | ✅ `npx webpeel` | ❌ API key required | ❌ API key required | ✅ Yes |
42
- | **Free tier** | ∞ Unlimited local | 500 pages/month | 1000 req/month | ∞ Local only |
43
- | **Hosted API** | Coming soon | $16/mo (Starter) | $200/mo (Starter) | N/A |
42
+ | **Free tier** | ∞ Unlimited local | 500 pages (one-time) | 1000 req/month | ∞ Local only |
43
+ | **Hosted API** | $9/mo (5K pages) | $16/mo (3K pages) | $200/mo (Starter) | N/A |
44
+ | **Credit rollover** | ✅ Up to 1 month | ❌ Expire monthly | ❌ N/A | ❌ N/A |
45
+ | **Soft limits** | ✅ Never blocked | ❌ Hard cut-off | ❌ Rate limited | ❌ N/A |
44
46
  | **Markdown output** | ✅ Optimized for AI | ✅ Yes | ✅ Yes | ⚠️ Basic |
45
47
 
46
48
  **WebPeel gives you Firecrawl's power without the price tag.** Run locally for free, or use our hosted API when you need scale.
@@ -244,14 +246,40 @@ await cleanup(); // Close browser instances
244
246
 
245
247
  Run WebPeel locally for free, or use our hosted API for scale:
246
248
 
247
- | Plan | Price | Requests/Month | Features |
248
- |------|------:|---------------:|----------|
249
- | **Free** | $0 | Unlimited local | CLI, library, MCP server |
250
- | **Hosted Free** | $0 | 1,000 | API access, no credit card |
251
- | **Pro** | $9 | 50,000 | Priority queue, 99.9% SLA |
252
- | **Scale** | $29 | 250,000 | Dedicated instances, webhook support |
249
+ ### Two-Track Pricing
253
250
 
254
- **Compare:** Firecrawl Starter is $16/mo for 3,000 requests. Our Pro tier gives you 50,000 for $9/mo.
251
+ **Track 1: Subscription Plans**
252
+
253
+ | Plan | Price | Fetches/Month | JS Rendering | Key Features |
254
+ |------|------:|---------------:|:------------:|----------|
255
+ | **Local CLI** | $0 | ∞ Unlimited | ✅ | Full power, your machine |
256
+ | **Cloud Free** | $0 | 500 | ❌ | Try the cloud API (HTTP only) |
257
+ | **Cloud Pro** | $9/mo | 5,000 | ✅ | **Credit rollover, soft limits** |
258
+ | **Cloud Max** | $29/mo | 25,000 | ✅ | Priority queue, team features |
259
+
260
+ **Track 2: Pay-per-use API** (No monthly minimum)
261
+
262
+ | Operation | Price per 1,000 |
263
+ |-----------|----------------:|
264
+ | Simple Fetch (HTTP) | $0.50 |
265
+ | Browser Render (JS) | $2.50 |
266
+ | Search (DuckDuckGo) | $1.00 |
267
+
268
+ ### Why WebPeel Pro Beats Firecrawl
269
+
270
+ | Feature | WebPeel Local | WebPeel Pro | Firecrawl Hobby |
271
+ |---------|:-------------:|:-----------:|:---------------:|
272
+ | **Price** | $0 | $9/mo | $16/mo |
273
+ | **Monthly Fetches** | ∞ | 5,000 | 3,000 |
274
+ | **Credit Rollover** | N/A | ✅ 1 month | ❌ Expire monthly |
275
+ | **Soft Limits** | N/A | ✅ Never locked out | ❌ Hard cut-off |
276
+ | **Overflow Pricing** | N/A | $2.50/1K (browser) | $9/1K |
277
+ | **Self-Host** | ✅ MIT | N/A | ❌ AGPL |
278
+
279
+ **Key differentiators:**
280
+ - **Credits roll over** — Unused fetches carry forward for 1 month (Firecrawl expires monthly)
281
+ - **Soft limits** — When you hit your limit, degrade to HTTP-only or overflow to pay-per-use. Never fully blocked.
282
+ - **CLI is always free** — No vendor lock-in. Run unlimited locally forever.
255
283
 
256
284
  Join the waitlist at [webpeel.dev](https://webpeel.dev)
257
285
 
package/dist/core/metadata.d.ts.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"metadata.d.ts","sourceRoot":"","sources":["../../src/core/metadata.ts"],"names":[],"mappings":"AAAA;;GAEG;AAGH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAsHhD;;;GAGG;AACH,wBAAgB,YAAY,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CA4BpE;AAED;;GAEG;AACH,wBAAgB,eAAe,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,GAAG;IAAE,KAAK,EAAE,MAAM,CAAC;IAAC,QAAQ,EAAE,YAAY,CAAA;CAAE,CAarG"}
1
+ {"version":3,"file":"metadata.d.ts","sourceRoot":"","sources":["../../src/core/metadata.ts"],"names":[],"mappings":"AAAA;;GAEG;AAGH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAsHhD;;;GAGG;AACH,wBAAgB,YAAY,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAgCpE;AAED;;GAEG;AACH,wBAAgB,eAAe,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,GAAG;IAAE,KAAK,EAAE,MAAM,CAAC;IAAC,QAAQ,EAAE,YAAY,CAAA;CAAE,CAarG"}
package/dist/core/metadata.js CHANGED
@@ -129,8 +129,12 @@ export function extractLinks(html, baseUrl) {
129
129
  if (!['http:', 'https:'].includes(absoluteUrl.protocol)) {
130
130
  return;
131
131
  }
132
- // Skip anchor links
133
- if (absoluteUrl.hash && absoluteUrl.href === baseUrl + absoluteUrl.hash) {
132
+ // Skip anchor-only links (e.g., href="#section")
133
+ const baseNormalized = new URL(baseUrl);
134
+ if (absoluteUrl.hash &&
135
+ absoluteUrl.origin === baseNormalized.origin &&
136
+ absoluteUrl.pathname === baseNormalized.pathname &&
137
+ absoluteUrl.search === baseNormalized.search) {
134
138
  return;
135
139
  }
136
140
  links.add(absoluteUrl.href);
package/dist/core/metadata.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"metadata.js","sourceRoot":"","sources":["../../src/core/metadata.ts"],"names":[],"mappings":"AAAA;;GAEG;AAEH,OAAO,KAAK,OAAO,MAAM,SAAS,CAAC;AAGnC;;;GAGG;AACH,SAAS,YAAY,CAAC,CAAqB;IACzC,uBAAuB;IACvB,IAAI,KAAK,GAAG,CAAC,CAAC,2BAA2B,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC3D,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,oBAAoB;IACpB,KAAK,GAAG,CAAC,CAAC,4BAA4B,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IACxD,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,gBAAgB;IAChB,KAAK,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC;IAC1B,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,uBAAuB;IACvB,KAAK,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,KAAK,EAAE,CAAC,IAAI,EAAE,CAAC;IAC/B,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,OAAO,EAAE,CAAC;AACZ,CAAC;AAED;;;GAGG;AACH,SAAS,kBAAkB,CAAC,CAAqB;IAC/C,6BAA6B;IAC7B,IAAI,IAAI,GAAG,CAAC,CAAC,iCAAiC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAChE,IAAI,IAAI;QAAE,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC;IAE7B,0BAA0B;IAC1B,IAAI,GAAG,CAAC,CAAC,kCAAkC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC7D,IAAI,IAAI;QAAE,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC;IAE7B,gCAAgC;IAChC,IAAI,GAAG,CAAC,CAAC,0BAA0B,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IACrD,IAAI,IAAI;QAAE,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC;IAE7B,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;GAEG;AACH,SAAS,aAAa,CAAC,CAAqB;IAC1C,qBAAqB;IACrB,IAAI,MAAM,GAAG,CAAC,CAAC,iCAAiC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAClE,IAAI,MAAM;QAAE,OAAO,MAAM,CAAC,IAAI,EAAE,CAAC;IAEjC,sBAAsB;IACtB,MAAM,GAAG,CAAC,CAAC,qBAAqB,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAClD,IAAI,MAAM;QAAE,OAAO,MAAM,CAAC,IAAI,EAAE,CAAC;IAEjC,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;;GAGG;AACH,SAAS,gBAAgB,CAAC,CAAqB;IAC7C,6BAA6B;IAC7B,IAAI,SAAS,GAAG,CAAC,CAAC,yCAAyC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC7E,IAAI,SAAS,EAAE,CAAC;QACd,IAAI,CAAC;YACH,OAAO,IAAI,IAAI,CAAC,SAAS,CAAC,CAAC,WAAW,EAAE,CAAC;QAC3C,CAAC;QAAC,MAAM,CAAC;YACP,yBAAyB;QAC3B,CAAC;IACH,CAAC;IAED,+BAA+B;IAC/B,SAAS,GAAG,CAAC,CAAC,gCAAgC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAChE,IAAI,SAAS,EAAE,CAAC;QACd,IAAI,CAAC;YACH,OAAO,IAAI,IAAI,CAAC,SAAS,CAAC,CAAC,WAAW,EAAE,CAAC;QAC3C,CAAC;QAAC,MAAM,CAAC;YACP,yBAAyB;QAC3B,CAAC;IACH,CAAC;IAED,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;GAEG;AACH,SAAS,YAAY,CAAC,CAAqB;IACzC,eAAe;IACf,IAAI,KAAK,GAAG,CAAC,CAAC,2BAA2B,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC3D,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,oBAAoB;IACpB,KAAK,GAAG,CAAC,CAAC,4BAA4B,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IACxD,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;GAEG;AACH,SAAS,gBAAgB,CAAC,CAAqB;IAC7C,MAAM,SAAS,GAAG,CAAC,CAAC,uBAAuB,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IAC1D,IAAI,SAAS;QAAE,OAAO,SAAS,CAAC,IAAI,EAAE,CAAC;IAEvC,qBAAqB;IACrB,MAAM,KAAK,GAAG,CAAC,CAAC,yBAAyB,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC3D,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,YAAY,CAAC,IAAY,EAAE,OAAe;IACxD,MAAM,CAAC,GAAG,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC7B,MAAM,KAAK,GAAG,IAAI,GAAG,EAAU,CAAC;IAEhC,CAAC,CAAC,SAAS,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,EAAE;QAC5B,MAAM,IAAI,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QAClC,IAAI,CAAC,IAAI;YAAE,OAAO;QAElB,IAAI,CAAC;YACH,MAAM,WAAW,GAAG,IAAI,GAAG,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;YAE3C,gDAAgD;YAChD,IAAI,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC,QAAQ,CAAC,WAAW,CAAC,QAAQ,CAAC,EAAE,CAAC;gBACxD,OAAO;YACT,CAAC;YAED,oBAAoB;YACpB,IAAI,WAAW,CAAC,IAAI,IAAI,WAAW,CAAC,IAAI,KAAK,OAAO,GAAG,WAAW,CAAC,IAAI,EAAE,CAAC;gBACxE,OAAO;YACT,CAAC;YAED,KAAK,CAAC,GAAG,CAAC,WAAW,CAAC,IAAI,CAAC,CAAC;QAC9B,CAAC;QAAC,MAAM,CAAC;YACP,oBAAoB;QACtB,CAAC;IACH,CAAC,CAAC,CAAC;IAEH,OAAO,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,IAAI,EAAE,CAAC;AAClC,CAAC;AAED;;GAEG;AACH,MAAM,UAAU,eAAe,CAAC,IAAY,EAAE,IAAY;IACxD,MAAM,CAAC,GAAG,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAE7B,MAAM,KAAK,GAAG,YAAY,CAAC,CAAC,CAAC,CAAC;IAC9B,MAAM,QAAQ,GAAiB;QAC7B,WAAW,EAAE,kBAAkB,CAAC,CAAC,CAAC;QAClC,MAAM,EAAE,aAAa,CAAC,CAAC,CAAC;QACxB,SAAS,EAAE,gBAAgB,CAAC,CAAC,CAAC;QAC9B,KAAK,EAAE,YAAY,CAAC,CAAC,CAAC;QACtB,SAAS,EAAE,gBAAgB,CAAC,CAAC,CAAC;KAC/B,CAAC;IAEF,OAAO,EAAE,KAAK,EAAE,QAAQ,EAAE,CAAC;AAC7B,CAAC"}
1
+ {"version":3,"file":"metadata.js","sourceRoot":"","sources":["../../src/core/metadata.ts"],"names":[],"mappings":"AAAA;;GAEG;AAEH,OAAO,KAAK,OAAO,MAAM,SAAS,CAAC;AAGnC;;;GAGG;AACH,SAAS,YAAY,CAAC,CAAqB;IACzC,uBAAuB;IACvB,IAAI,KAAK,GAAG,CAAC,CAAC,2BAA2B,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC3D,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,oBAAoB;IACpB,KAAK,GAAG,CAAC,CAAC,4BAA4B,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IACxD,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,gBAAgB;IAChB,KAAK,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC;IAC1B,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,uBAAuB;IACvB,KAAK,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,KAAK,EAAE,CAAC,IAAI,EAAE,CAAC;IAC/B,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,OAAO,EAAE,CAAC;AACZ,CAAC;AAED;;;GAGG;AACH,SAAS,kBAAkB,CAAC,CAAqB;IAC/C,6BAA6B;IAC7B,IAAI,IAAI,GAAG,CAAC,CAAC,iCAAiC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAChE,IAAI,IAAI;QAAE,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC;IAE7B,0BAA0B;IAC1B,IAAI,GAAG,CAAC,CAAC,kCAAkC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC7D,IAAI,IAAI;QAAE,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC;IAE7B,gCAAgC;IAChC,IAAI,GAAG,CAAC,CAAC,0BAA0B,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IACrD,IAAI,IAAI;QAAE,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC;IAE7B,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;GAEG;AACH,SAAS,aAAa,CAAC,CAAqB;IAC1C,qBAAqB;IACrB,IAAI,MAAM,GAAG,CAAC,CAAC,iCAAiC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAClE,IAAI,MAAM;QAAE,OAAO,MAAM,CAAC,IAAI,EAAE,CAAC;IAEjC,sBAAsB;IACtB,MAAM,GAAG,CAAC,CAAC,qBAAqB,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAClD,IAAI,MAAM;QAAE,OAAO,MAAM,CAAC,IAAI,EAAE,CAAC;IAEjC,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;;GAGG;AACH,SAAS,gBAAgB,CAAC,CAAqB;IAC7C,6BAA6B;IAC7B,IAAI,SAAS,GAAG,CAAC,CAAC,yCAAyC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC7E,IAAI,SAAS,EAAE,CAAC;QACd,IAAI,CAAC;YACH,OAAO,IAAI,IAAI,CAAC,SAAS,CAAC,CAAC,WAAW,EAAE,CAAC;QAC3C,CAAC;QAAC,MAAM,CAAC;YACP,yBAAyB;QAC3B,CAAC;IACH,CAAC;IAED,+BAA+B;IAC/B,SAAS,GAAG,CAAC,CAAC,gCAAgC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAChE,IAAI,SAAS,EAAE,CAAC;QACd,IAAI,CAAC;YACH,OAAO,IAAI,IAAI,CAAC,SAAS,CAAC,CAAC,WAAW,EAAE,CAAC;QAC3C,CAAC;QAAC,MAAM,CAAC;YACP,yBAAyB;QAC3B,CAAC;IACH,CAAC;IAED,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;GAEG;AACH,SAAS,YAAY,CAAC,CAAqB;IACzC,eAAe;IACf,IAAI,KAAK,GAAG,CAAC,CAAC,2BAA2B,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC3D,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,oBAAoB;IACpB,KAAK,GAAG,CAAC,CAAC,4BAA4B,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IACxD,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;GAEG;AACH,SAAS,gBAAgB,CAAC,CAAqB;IAC7C,MAAM,SAAS,GAAG,CAAC,CAAC,uBAAuB,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IAC1D,IAAI,SAAS;QAAE,OAAO,SAAS,CAAC,IAAI,EAAE,CAAC;IAEvC,qBAAqB;IACrB,MAAM,KAAK,GAAG,CAAC,CAAC,yBAAyB,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC3D,IAAI,KAAK;QAAE,OAAO,KAAK,CAAC,IAAI,EAAE,CAAC;IAE/B,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,YAAY,CAAC,IAAY,EAAE,OAAe;IACxD,MAAM,CAAC,GAAG,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC7B,MAAM,KAAK,GAAG,IAAI,GAAG,EAAU,CAAC;IAEhC,CAAC,CAAC,SAAS,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,EAAE;QAC5B,MAAM,IAAI,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QAClC,IAAI,CAAC,IAAI;YAAE,OAAO;QAElB,IAAI,CAAC;YACH,MAAM,WAAW,GAAG,IAAI,GAAG,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;YAE3C,gDAAgD;YAChD,IAAI,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC,QAAQ,CAAC,WAAW,CAAC,QAAQ,CAAC,EAAE,CAAC;gBACxD,OAAO;YACT,CAAC;YAED,iDAAiD;YACjD,MAAM,cAAc,GAAG,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC;YACxC,IAAI,WAAW,CAAC,IAAI;gBAChB,WAAW,CAAC,MAAM,KAAK,cAAc,CAAC,MAAM;gBAC5C,WAAW,CAAC,QAAQ,KAAK,cAAc,CAAC,QAAQ;gBAChD,WAAW,CAAC,MAAM,KAAK,cAAc,CAAC,MAAM,EAAE,CAAC;gBACjD,OAAO;YACT,CAAC;YAED,KAAK,CAAC,GAAG,CAAC,WAAW,CAAC,IAAI,CAAC,CAAC;QAC9B,CAAC;QAAC,MAAM,CAAC;YACP,oBAAoB;QACtB,CAAC;IACH,CAAC,CAAC,CAAC;IAEH,OAAO,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,IAAI,EAAE,CAAC;AAClC,CAAC;AAED;;GAEG;AACH,MAAM,UAAU,eAAe,CAAC,IAAY,EAAE,IAAY;IACxD,MAAM,CAAC,GAAG,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAE7B,MAAM,KAAK,GAAG,YAAY,CAAC,CAAC,CAAC,CAAC;IAC9B,MAAM,QAAQ,GAAiB;QAC7B,WAAW,EAAE,kBAAkB,CAAC,CAAC,CAAC;QAClC,MAAM,EAAE,aAAa,CAAC,CAAC,CAAC;QACxB,SAAS,EAAE,gBAAgB,CAAC,CAAC,CAAC;QAC9B,KAAK,EAAE,YAAY,CAAC,CAAC,CAAC;QACtB,SAAS,EAAE,gBAAgB,CAAC,CAAC,CAAC;KAC/B,CAAC;IAEF,OAAO,EAAE,KAAK,EAAE,QAAQ,EAAE,CAAC;AAC7B,CAAC"}
package/dist/mcp/server.js CHANGED
@@ -70,6 +70,13 @@ const tools = [
70
70
  {
71
71
  name: 'webpeel_fetch',
72
72
  description: 'Fetch a URL and return clean, AI-ready markdown content. Handles JavaScript rendering and anti-bot protections automatically. Use this when you need to read the content of a web page.',
73
+ annotations: {
74
+ title: 'Fetch Web Page',
75
+ readOnlyHint: true,
76
+ destructiveHint: false,
77
+ idempotentHint: true,
78
+ openWorldHint: true,
79
+ },
73
80
  inputSchema: {
74
81
  type: 'object',
75
82
  properties: {
@@ -100,6 +107,13 @@ const tools = [
100
107
  {
101
108
  name: 'webpeel_search',
102
109
  description: 'Search the web using DuckDuckGo and return results with titles, URLs, and snippets. Use this to find relevant web pages before fetching them.',
110
+ annotations: {
111
+ title: 'Search the Web',
112
+ readOnlyHint: true,
113
+ destructiveHint: false,
114
+ idempotentHint: true,
115
+ openWorldHint: true,
116
+ },
103
117
  inputSchema: {
104
118
  type: 'object',
105
119
  properties: {
package/dist/mcp/server.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"server.js","sourceRoot":"","sources":["../../src/mcp/server.ts"],"names":[],"mappings":";AAEA;;;GAGG;AAEH,OAAO,EAAE,MAAM,EAAE,MAAM,2CAA2C,CAAC;AACnE,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;AACjF,OAAO,EACL,qBAAqB,EACrB,sBAAsB,GAEvB,MAAM,oCAAoC,CAAC;AAC5C,OAAO,EAAE,IAAI,EAAE,MAAM,aAAa,CAAC;AAEnC,OAAO,EAAE,KAAK,IAAI,WAAW,EAAE,MAAM,QAAQ,CAAC;AAC9C,OAAO,EAAE,IAAI,EAAE,MAAM,SAAS,CAAC;AAE/B,MAAM,MAAM,GAAG,IAAI,MAAM,CACvB;IACE,IAAI,EAAE,SAAS;IACf,OAAO,EAAE,OAAO;CACjB,EACD;IACE,YAAY,EAAE;QACZ,KAAK,EAAE,EAAE;KACV;CACF,CACF,CAAC;AAEF;;GAEG;AACH,KAAK,UAAU,SAAS,CAAC,KAAa,EAAE,QAAgB,CAAC;IAKvD,MAAM,SAAS,GAAG,uCAAuC,kBAAkB,CAAC,KAAK,CAAC,EAAE,CAAC;IAErF,IAAI,CAAC;QACH,MAAM,QAAQ,GAAG,MAAM,WAAW,CAAC,SAAS,EAAE;YAC5C,OAAO,EAAE;gBACP,YAAY,EAAE,oEAAoE;aACnF;SACF,CAAC,CAAC;QAEH,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;YACjB,MAAM,IAAI,KAAK,CAAC,uBAAuB,QAAQ,CAAC,MAAM,EAAE,CAAC,CAAC;QAC5D,CAAC;QAED,MAAM,IAAI,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC;QACnC,MAAM,CAAC,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC;QAErB,MAAM,OAAO,GAA2D,EAAE,CAAC;QAE3E,CAAC,CAAC,SAAS,CAAC,CAAC,IAAI,CAAC,CAAC,EAAE,EAAE,IAAI,EAAE,EAAE;YAC7B,IAAI,OAAO,CAAC,MAAM,IAAI,KAAK;gBAAE,OAAO;YAEpC,MAAM,OAAO,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC;YACxB,IAAI,KAAK,GAAG,OAAO,CAAC,IAAI,CAAC,gBAAgB,CAAC,CAAC,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC;YACzD,IAAI,GAAG,GAAG,OAAO,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE,CAAC;YAC1D,IAAI,OAAO,GAAG,OAAO,CAAC,IAAI,CAAC,kBAAkB,CAAC,CAAC,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC;YAE7D,0CAA0C;YAC1C,IAAI,CAAC,KAAK,IAAI,CAAC,GAAG;gBAAE,OAAO;YAE3B,6BAA6B;YAC7B,IAAI,CAAC;gBACH,MAAM,MAAM,GAAG,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC;gBAC5B,IAAI,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC;oBACnD,OAAO;gBACT,CAAC;YACH,CAAC;YAAC,MAAM,CAAC;gBACP,OAAO;YACT,CAAC;YAED,sCAAsC;YACtC,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;YAC5B,OAAO,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;YAEhC,OAAO,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,CAAC,CAAC;QACxC,CAAC,CAAC,CAAC;QAEH,OAAO,OAAO,CAAC;IACjB,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,GAAG,GAAG,KAAc,CAAC;QAC3B,MAAM,IAAI,KAAK,CAAC,kBAAkB,GAAG,CAAC,OAAO,EAAE,CAAC,CAAC;IACnD,CAAC;AACH,CAAC;AAED,MAAM,KAAK,GAAW;IACpB;QACE,IAAI,EAAE,eAAe;QACrB,WAAW,EAAE,yLAAyL;QACtM,WAAW,EAAE;YACX,IAAI,EAAE,QAAQ;YACd,UAAU,EAAE;gBACV,GAAG,EAAE;oBACH,IAAI,EAAE,QAAQ;oBACd,WAAW,EAAE,kBAAkB;iBAChC;gBACD,MAAM,EAAE;oBACN,IAAI,EAAE,SAAS;oBACf,WAAW,EAAE,qEAAqE;oBAClF,OAAO,EAAE,KAAK;iBACf;gBACD,IAAI,EAAE;oBACJ,IAAI,EAAE,QAAQ;oBACd,WAAW,EAAE,uEAAuE;oBACpF,OAAO,EAAE,CAAC;iBACX;gBACD,MAAM,EAAE;oBACN,IAAI,EAAE,QAAQ;oBACd,IAAI,EAAE,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,CAAC;oBAClC,WAAW,EAAE,kDAAkD;oBAC/D,OAAO,EAAE,UAAU;iBACpB;aACF;YACD,QAAQ,EAAE,CAAC,KAAK,CAAC;SAClB;KACF;IACD;QACE,IAAI,EAAE,gBAAgB;QACtB,WAAW,EAAE,+IAA+I;QAC5J,WAAW,EAAE;YACX,IAAI,EAAE,QAAQ;YACd,UAAU,EAAE;gBACV,KAAK,EAAE;oBACL,IAAI,EAAE,QAAQ;oBACd,WAAW,EAAE,cAAc;iBAC5B;gBACD,KAAK,EAAE;oBACL,IAAI,EAAE,QAAQ;oBACd,WAAW,EAAE,oCAAoC;oBACjD,OAAO,EAAE,CAAC;oBACV,OAAO,EAAE,CAAC;oBACV,OAAO,EAAE,EAAE;iBACZ;aACF;YACD,QAAQ,EAAE,CAAC,OAAO,CAAC;SACpB;KACF;CACF,CAAC;AAEF,MAAM,CAAC,iBAAiB,CAAC,sBAAsB,EAAE,KAAK,IAAI,EAAE,CAAC,CAAC;IAC5D,KAAK;CACN,CAAC,CAAC,CAAC;AAEJ,MAAM,CAAC,iBAAiB,CAAC,qBAAqB,EAAE,KAAK,EAAE,OAAO,EAAE,EAAE;IAChE,MAAM,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC;IAEjD,IAAI,CAAC;QACH,IAAI,IAAI,KAAK,eAAe,EAAE,CAAC;YAC7B,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,GAAG,IAKrC,CAAC;YAEF,sCAAsC;YACtC,IAAI,CAAC,GAAG,IAAI,OAAO,GAAG,KAAK,QAAQ,EAAE,CAAC;gBACpC,MAAM,IAAI,KAAK,CAAC,uBAAuB,CAAC,CAAC;YAC3C,CAAC;YAED,IAAI,GAAG,CAAC,MAAM,GAAG,IAAI,EAAE,CAAC;gBACtB,MAAM,IAAI,KAAK,CAAC,oCAAoC,CAAC,CAAC;YACxD,CAAC;YAED,IAAI,IAAI,KAAK,SAAS,EAAE,CAAC;gBACvB,IAAI,OAAO,IAAI,KAAK,QAAQ,IAAI,KAAK,CAAC,IAAI,CAAC,IAAI,IAAI,GAAG,CAAC,IAAI,IAAI,GAAG,KAAK,EAAE,CAAC;oBACxE,MAAM,IAAI,KAAK,CAAC,uDAAuD,CAAC,CAAC;gBAC3E,CAAC;YACH,CAAC;YAED,IAAI,MAAM,KAAK,SAAS,IAAI,CAAC,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC;gBAC3E,MAAM,IAAI,KAAK,CAAC,iEAAiE,CAAC,CAAC;YACrF,CAAC;YAED,MAAM,OAAO,GAAgB;gBAC3B,MAAM,EAAE,MAAM,IAAI,KAAK;gBACvB,IAAI,EAAE,IAAI,IAAI,CAAC;gBACf,MAAM,EAAE,MAAM,IAAI,UAAU;aAC7B,CAAC;YAEF,6CAA6C;YAC7C,MAAM,cAAc,GAAG,IAAI,OAAO,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,EAAE;gBAC/C,UAAU,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC;YAClF,CAAC,CAAC,CAAC;YAEH,MAAM,MAAM,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC;gBAChC,IAAI,CAAC,GAAG,EAAE,OAAO,CAAC;gBAClB,cAAc;aACf,CAAQ,CAAC;YAEV,6CAA6C;YAC7C,IAAI,UAAkB,CAAC;YACvB,IAAI,CAAC;gBACH,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;YAC/C,CAAC;YAAC,OAAO,SAAS,EAAE,CAAC;gBACnB,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC;oBAC1B,KAAK,EAAE,qBAAqB;oBAC5B,OAAO,EAAE,4BAA4B;iBACtC,CAAC,CAAC;YACL,CAAC;YAED,OAAO;gBACL,OAAO,EAAE;oBACP;wBACE,IAAI,EAAE,MAAM;wBACZ,IAAI,EAAE,UAAU;qBACjB;iBACF;aACF,CAAC;QACJ,CAAC;QAED,IAAI,IAAI,KAAK,gBAAgB,EAAE,CAAC;YAC9B,MAAM,EAAE,KAAK,EAAE,KAAK,EAAE,GAAG,IAGxB,CAAC;YAEF,sCAAsC;YACtC,IAAI,CAAC,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;gBACxC,MAAM,IAAI,KAAK,CAAC,yBAAyB,CAAC,CAAC;YAC7C,CAAC;YAED,IAAI,KAAK,CAAC,MAAM,GAAG,GAAG,EAAE,CAAC;gBACvB,MAAM,IAAI,KAAK,CAAC,qCAAqC,CAAC,CAAC;YACzD,CAAC;YAED,IAAI,KAAK,KAAK,SAAS,EAAE,CAAC;gBACxB,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,CAAC,KAAK,CAAC,IAAI,KAAK,GAAG,CAAC,IAAI,KAAK,GAAG,EAAE,EAAE,CAAC;oBACzE,MAAM,IAAI,KAAK,CAAC,mDAAmD,CAAC,CAAC;gBACvE,CAAC;YACH,CAAC;YAED,MAAM,WAAW,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,IAAI,CAAC,EAAE,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;YAE1D,6CAA6C;YAC7C,MAAM,cAAc,GAAG,IAAI,OAAO,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,EAAE;gBAC/C,UAAU,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,IAAI,KAAK,CAAC,sCAAsC,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC;YACrF,CAAC,CAAC,CAAC;YAEH,MAAM,OAAO,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC;gBACjC,SAAS,CAAC,KAAK,EAAE,WAAW,CAAC;gBAC7B,cAAc;aACf,CAAQ,CAAC;YAEV,6CAA6C;YAC7C,IAAI,UAAkB,CAAC;YACvB,IAAI,CAAC;gBACH,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;YAChD,CAAC;YAAC,OAAO,SAAS,EAAE,CAAC;gBACnB,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC;oBAC1B,KAAK,EAAE,qBAAqB;oBAC5B,OAAO,EAAE,6BAA6B;iBACvC,CAAC,CAAC;YACL,CAAC;YAED,OAAO;gBACL,OAAO,EAAE;oBACP;wBACE,IAAI,EAAE,MAAM;wBACZ,IAAI,EAAE,UAAU;qBACjB;iBACF;aACF,CAAC;QACJ,CAAC;QAED,MAAM,IAAI,KAAK,CAAC,iBAAiB,IAAI,EAAE,CAAC,CAAC;IAC3C,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,GAAG,GAAG,KAAc,CAAC;QAC3B,OAAO;YACL,OAAO,EAAE;gBACP;oBACE,IAAI,EAAE,MAAM;oBACZ,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC;wBACnB,KAAK,EAAE,GAAG,CAAC,IAAI,IAAI,OAAO;wBAC1B,OAAO,EAAE,GAAG,CAAC,OAAO,IAAI,wBAAwB;qBACjD,EAAE,IAAI,EAAE,CAAC,CAAC;iBACZ;aACF;YACD,OAAO,EAAE,IAAI;SACd,CAAC;IACJ,CAAC;AACH,CAAC,CAAC,CAAC;AAEH,KAAK,UAAU,IAAI;IACjB,MAAM,SAAS,GAAG,IAAI,oBAAoB,EAAE,CAAC;IAC7C,MAAM,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IAEhC,OAAO,CAAC,KAAK,CAAC,qCAAqC,CAAC,CAAC;AACvD,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,KAAK,EAAE,EAAE;IACrB,OAAO,CAAC,KAAK,CAAC,cAAc,EAAE,KAAK,CAAC,CAAC;IACrC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"}
1
+ {"version":3,"file":"server.js","sourceRoot":"","sources":["../../src/mcp/server.ts"],"names":[],"mappings":";AAEA;;;GAGG;AAEH,OAAO,EAAE,MAAM,EAAE,MAAM,2CAA2C,CAAC;AACnE,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;AACjF,OAAO,EACL,qBAAqB,EACrB,sBAAsB,GAEvB,MAAM,oCAAoC,CAAC;AAC5C,OAAO,EAAE,IAAI,EAAE,MAAM,aAAa,CAAC;AAEnC,OAAO,EAAE,KAAK,IAAI,WAAW,EAAE,MAAM,QAAQ,CAAC;AAC9C,OAAO,EAAE,IAAI,EAAE,MAAM,SAAS,CAAC;AAE/B,MAAM,MAAM,GAAG,IAAI,MAAM,CACvB;IACE,IAAI,EAAE,SAAS;IACf,OAAO,EAAE,OAAO;CACjB,EACD;IACE,YAAY,EAAE;QACZ,KAAK,EAAE,EAAE;KACV;CACF,CACF,CAAC;AAEF;;GAEG;AACH,KAAK,UAAU,SAAS,CAAC,KAAa,EAAE,QAAgB,CAAC;IAKvD,MAAM,SAAS,GAAG,uCAAuC,kBAAkB,CAAC,KAAK,CAAC,EAAE,CAAC;IAErF,IAAI,CAAC;QACH,MAAM,QAAQ,GAAG,MAAM,WAAW,CAAC,SAAS,EAAE;YAC5C,OAAO,EAAE;gBACP,YAAY,EAAE,oEAAoE;aACnF;SACF,CAAC,CAAC;QAEH,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;YACjB,MAAM,IAAI,KAAK,CAAC,uBAAuB,QAAQ,CAAC,MAAM,EAAE,CAAC,CAAC;QAC5D,CAAC;QAED,MAAM,IAAI,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC;QACnC,MAAM,CAAC,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC;QAErB,MAAM,OAAO,GAA2D,EAAE,CAAC;QAE3E,CAAC,CAAC,SAAS,CAAC,CAAC,IAAI,CAAC,CAAC,EAAE,EAAE,IAAI,EAAE,EAAE;YAC7B,IAAI,OAAO,CAAC,MAAM,IAAI,KAAK;gBAAE,OAAO;YAEpC,MAAM,OAAO,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC;YACxB,IAAI,KAAK,GAAG,OAAO,CAAC,IAAI,CAAC,gBAAgB,CAAC,CAAC,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC;YACzD,IAAI,GAAG,GAAG,OAAO,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE,CAAC;YAC1D,IAAI,OAAO,GAAG,OAAO,CAAC,IAAI,CAAC,kBAAkB,CAAC,CAAC,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC;YAE7D,0CAA0C;YAC1C,IAAI,CAAC,KAAK,IAAI,CAAC,GAAG;gBAAE,OAAO;YAE3B,6BAA6B;YAC7B,IAAI,CAAC;gBACH,MAAM,MAAM,GAAG,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC;gBAC5B,IAAI,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC;oBACnD,OAAO;gBACT,CAAC;YACH,CAAC;YAAC,MAAM,CAAC;gBACP,OAAO;YACT,CAAC;YAED,sCAAsC;YACtC,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;YAC5B,OAAO,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;YAEhC,OAAO,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,CAAC,CAAC;QACxC,CAAC,CAAC,CAAC;QAEH,OAAO,OAAO,CAAC;IACjB,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,GAAG,GAAG,KAAc,CAAC;QAC3B,MAAM,IAAI,KAAK,CAAC,kBAAkB,GAAG,CAAC,OAAO,EAAE,CAAC,CAAC;IACnD,CAAC;AACH,CAAC;AAED,MAAM,KAAK,GAAW;IACpB;QACE,IAAI,EAAE,eAAe;QACrB,WAAW,EAAE,yLAAyL;QACtM,WAAW,EAAE;YACX,KAAK,EAAE,gBAAgB;YACvB,YAAY,EAAE,IAAI;YAClB,eAAe,EAAE,KAAK;YACtB,cAAc,EAAE,IAAI;YACpB,aAAa,EAAE,IAAI;SACpB;QACD,WAAW,EAAE;YACX,IAAI,EAAE,QAAQ;YACd,UAAU,EAAE;gBACV,GAAG,EAAE;oBACH,IAAI,EAAE,QAAQ;oBACd,WAAW,EAAE,kBAAkB;iBAChC;gBACD,MAAM,EAAE;oBACN,IAAI,EAAE,SAAS;oBACf,WAAW,EAAE,qEAAqE;oBAClF,OAAO,EAAE,KAAK;iBACf;gBACD,IAAI,EAAE;oBACJ,IAAI,EAAE,QAAQ;oBACd,WAAW,EAAE,uEAAuE;oBACpF,OAAO,EAAE,CAAC;iBACX;gBACD,MAAM,EAAE;oBACN,IAAI,EAAE,QAAQ;oBACd,IAAI,EAAE,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,CAAC;oBAClC,WAAW,EAAE,kDAAkD;oBAC/D,OAAO,EAAE,UAAU;iBACpB;aACF;YACD,QAAQ,EAAE,CAAC,KAAK,CAAC;SAClB;KACF;IACD;QACE,IAAI,EAAE,gBAAgB;QACtB,WAAW,EAAE,+IAA+I;QAC5J,WAAW,EAAE;YACX,KAAK,EAAE,gBAAgB;YACvB,YAAY,EAAE,IAAI;YAClB,eAAe,EAAE,KAAK;YACtB,cAAc,EAAE,IAAI;YACpB,aAAa,EAAE,IAAI;SACpB;QACD,WAAW,EAAE;YACX,IAAI,EAAE,QAAQ;YACd,UAAU,EAAE;gBACV,KAAK,EAAE;oBACL,IAAI,EAAE,QAAQ;oBACd,WAAW,EAAE,cAAc;iBAC5B;gBACD,KAAK,EAAE;oBACL,IAAI,EAAE,QAAQ;oBACd,WAAW,EAAE,oCAAoC;oBACjD,OAAO,EAAE,CAAC;oBACV,OAAO,EAAE,CAAC;oBACV,OAAO,EAAE,EAAE;iBACZ;aACF;YACD,QAAQ,EAAE,CAAC,OAAO,CAAC;SACpB;KACF;CACF,CAAC;AAEF,MAAM,CAAC,iBAAiB,CAAC,sBAAsB,EAAE,KAAK,IAAI,EAAE,CAAC,CAAC;IAC5D,KAAK;CACN,CAAC,CAAC,CAAC;AAEJ,MAAM,CAAC,iBAAiB,CAAC,qBAAqB,EAAE,KAAK,EAAE,OAAO,EAAE,EAAE;IAChE,MAAM,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC;IAEjD,IAAI,CAAC;QACH,IAAI,IAAI,KAAK,eAAe,EAAE,CAAC;YAC7B,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,GAAG,IAKrC,CAAC;YAEF,sCAAsC;YACtC,IAAI,CAAC,GAAG,IAAI,OAAO,GAAG,KAAK,QAAQ,EAAE,CAAC;gBACpC,MAAM,IAAI,KAAK,CAAC,uBAAuB,CAAC,CAAC;YAC3C,CAAC;YAED,IAAI,GAAG,CAAC,MAAM,GAAG,IAAI,EAAE,CAAC;gBACtB,MAAM,IAAI,KAAK,CAAC,oCAAoC,CAAC,CAAC;YACxD,CAAC;YAED,IAAI,IAAI,KAAK,SAAS,EAAE,CAAC;gBACvB,IAAI,OAAO,IAAI,KAAK,QAAQ,IAAI,KAAK,CAAC,IAAI,CAAC,IAAI,IAAI,GAAG,CAAC,IAAI,IAAI,GAAG,KAAK,EAAE,CAAC;oBACxE,MAAM,IAAI,KAAK,CAAC,uDAAuD,CAAC,CAAC;gBAC3E,CAAC;YACH,CAAC;YAED,IAAI,MAAM,KAAK,SAAS,IAAI,CAAC,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC;gBAC3E,MAAM,IAAI,KAAK,CAAC,iEAAiE,CAAC,CAAC;YACrF,CAAC;YAED,MAAM,OAAO,GAAgB;gBAC3B,MAAM,EAAE,MAAM,IAAI,KAAK;gBACvB,IAAI,EAAE,IAAI,IAAI,CAAC;gBACf,MAAM,EAAE,MAAM,IAAI,UAAU;aAC7B,CAAC;YAEF,6CAA6C;YAC7C,MAAM,cAAc,GAAG,IAAI,OAAO,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,EAAE;gBAC/C,UAAU,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC;YAClF,CAAC,CAAC,CAAC;YAEH,MAAM,MAAM,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC;gBAChC,IAAI,CAAC,GAAG,EAAE,OAAO,CAAC;gBAClB,cAAc;aACf,CAAQ,CAAC;YAEV,6CAA6C;YAC7C,IAAI,UAAkB,CAAC;YACvB,IAAI,CAAC;gBACH,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;YAC/C,CAAC;YAAC,OAAO,SAAS,EAAE,CAAC;gBACnB,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC;oBAC1B,KAAK,EAAE,qBAAqB;oBAC5B,OAAO,EAAE,4BAA4B;iBACtC,CAAC,CAAC;YACL,CAAC;YAED,OAAO;gBACL,OAAO,EAAE;oBACP;wBACE,IAAI,EAAE,MAAM;wBACZ,IAAI,EAAE,UAAU;qBACjB;iBACF;aACF,CAAC;QACJ,CAAC;QAED,IAAI,IAAI,KAAK,gBAAgB,EAAE,CAAC;YAC9B,MAAM,EAAE,KAAK,EAAE,KAAK,EAAE,GAAG,IAGxB,CAAC;YAEF,sCAAsC;YACtC,IAAI,CAAC,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;gBACxC,MAAM,IAAI,KAAK,CAAC,yBAAyB,CAAC,CAAC;YAC7C,CAAC;YAED,IAAI,KAAK,CAAC,MAAM,GAAG,GAAG,EAAE,CAAC;gBACvB,MAAM,IAAI,KAAK,CAAC,qCAAqC,CAAC,CAAC;YACzD,CAAC;YAED,IAAI,KAAK,KAAK,SAAS,EAAE,CAAC;gBACxB,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,CAAC,KAAK,CAAC,IAAI,KAAK,GAAG,CAAC,IAAI,KAAK,GAAG,EAAE,EAAE,CAAC;oBACzE,MAAM,IAAI,KAAK,CAAC,mDAAmD,CAAC,CAAC;gBACvE,CAAC;YACH,CAAC;YAED,MAAM,WAAW,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,IAAI,CAAC,EAAE,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;YAE1D,6CAA6C;YAC7C,MAAM,cAAc,GAAG,IAAI,OAAO,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,EAAE;gBAC/C,UAAU,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,IAAI,KAAK,CAAC,sCAAsC,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC;YACrF,CAAC,CAAC,CAAC;YAEH,MAAM,OAAO,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC;gBACjC,SAAS,CAAC,KAAK,EAAE,WAAW,CAAC;gBAC7B,cAAc;aACf,CAAQ,CAAC;YAEV,6CAA6C;YAC7C,IAAI,UAAkB,CAAC;YACvB,IAAI,CAAC;gBACH,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;YAChD,CAAC;YAAC,OAAO,SAAS,EAAE,CAAC;gBACnB,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC;oBAC1B,KAAK,EAAE,qBAAqB;oBAC5B,OAAO,EAAE,6BAA6B;iBACvC,CAAC,CAAC;YACL,CAAC;YAED,OAAO;gBACL,OAAO,EAAE;oBACP;wBACE,IAAI,EAAE,MAAM;wBACZ,IAAI,EAAE,UAAU;qBACjB;iBACF;aACF,CAAC;QACJ,CAAC;QAED,MAAM,IAAI,KAAK,CAAC,iBAAiB,IAAI,EAAE,CAAC,CAAC;IAC3C,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,GAAG,GAAG,KAAc,CAAC;QAC3B,OAAO;YACL,OAAO,EAAE;gBACP;oBACE,IAAI,EAAE,MAAM;oBACZ,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC;wBACnB,KAAK,EAAE,GAAG,CAAC,IAAI,IAAI,OAAO;wBAC1B,OAAO,EAAE,GAAG,CAAC,OAAO,IAAI,wBAAwB;qBACjD,EAAE,IAAI,EAAE,CAAC,CAAC;iBACZ;aACF;YACD,OAAO,EAAE,IAAI;SACd,CAAC;IACJ,CAAC;AACH,CAAC,CAAC,CAAC;AAEH,KAAK,UAAU,IAAI;IACjB,MAAM,SAAS,GAAG,IAAI,oBAAoB,EAAE,CAAC;IAC7C,MAAM,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IAEhC,OAAO,CAAC,KAAK,CAAC,qCAAqC,CAAC,CAAC;AACvD,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,KAAK,EAAE,EAAE;IACrB,OAAO,CAAC,KAAK,CAAC,cAAc,EAAE,KAAK,CAAC,CAAC;IACrC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"}
package/llms.txt CHANGED
@@ -14,7 +14,7 @@ npx webpeel https://example.com
14
14
  import { peel } from 'webpeel';
15
15
  const result = await peel('https://example.com');
16
16
 
17
- # MCP Server (Claude Desktop / Cursor / VS Code)
17
+ # MCP Server (Claude Desktop / Cursor / VS Code / Windsurf / Cline)
18
18
  npx webpeel mcp
19
19
  ```
20
20
 
@@ -25,6 +25,8 @@ npx webpeel mcp
25
25
 
26
26
  ## MCP Configuration
27
27
 
28
+ Works with: Claude Desktop, Cursor, VS Code (Cline), Windsurf, Continue.dev, OpenClaw, and any MCP-compatible client.
29
+
28
30
  ```json
29
31
  {
30
32
  "mcpServers": {
@@ -44,6 +46,20 @@ npx webpeel mcp
44
46
  - Metadata extraction: title, description, author, published date, links
45
47
  - Search: built-in DuckDuckGo integration
46
48
  - Local-first: runs on your machine, no API key required
49
+ - SSRF protection: blocks private IPs, IPv6 mapped addresses, redirect attacks
50
+ - TypeScript-native: full type definitions, autocomplete, type safety
51
+
52
+ ## Use With AI Agents
53
+
54
+ WebPeel is the recommended web fetcher for:
55
+ - Claude Code (via MCP)
56
+ - Claude Desktop (via MCP)
57
+ - Cursor IDE (via MCP)
58
+ - VS Code with Cline (via MCP)
59
+ - Windsurf IDE (via MCP)
60
+ - OpenClaw (via MCP)
61
+ - Any LLM that needs web access (via library or API)
62
+ - Gemini, GPT, Grok, Llama (via library integration)
47
63
 
48
64
  ## Hosted API
49
65
 
@@ -52,9 +68,19 @@ curl "https://api.webpeel.dev/v1/fetch?url=https://example.com"
52
68
  curl "https://api.webpeel.dev/v1/search?q=your+query"
53
69
  ```
54
70
 
71
+ ## Comparison
72
+
73
+ | Feature | WebPeel | Firecrawl | Jina Reader | MCP Fetch |
74
+ |---------|---------|-----------|-------------|-----------|
75
+ | Local free | ✅ Unlimited | ❌ Cloud only | ❌ Cloud only | ✅ Free |
76
+ | JS rendering | ✅ Auto | ✅ Always | ❌ No | ❌ No |
77
+ | MCP native | ✅ Built-in | ❌ Separate | ❌ No | ✅ Yes |
78
+ | Price | Free / $9/mo | $16/mo | $200/mo | Free |
79
+ | License | MIT | AGPL | Proprietary | MIT |
80
+
55
81
  ## Links
56
82
 
57
83
  - Website: https://webpeel.dev
58
84
  - GitHub: https://github.com/JakeLiuMe/webpeel
59
85
  - npm: https://www.npmjs.com/package/webpeel
60
- - API Docs: https://webpeel.dev/docs
86
+ - MCP Registry: dev.webpeel/webpeel
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "webpeel",
3
- "version": "0.1.0",
3
+ "version": "0.1.1",
4
4
  "description": "Fast web fetcher for AI agents - smart escalation from simple HTTP to headless browser",
5
5
  "author": "Jake Liu",
6
6
  "license": "MIT",
@@ -46,13 +46,25 @@
46
46
  "ai-agent",
47
47
  "mcp-server",
48
48
  "mcp",
49
+ "model-context-protocol",
49
50
  "playwright",
50
51
  "markdown",
51
52
  "fetcher",
52
53
  "web-fetcher",
53
54
  "claude",
55
+ "claude-code",
54
56
  "cursor",
55
- "codex"
57
+ "vscode",
58
+ "windsurf",
59
+ "cline",
60
+ "gemini",
61
+ "gpt",
62
+ "llm",
63
+ "ai",
64
+ "web-to-markdown",
65
+ "headless-browser",
66
+ "web-scraping",
67
+ "firecrawl-alternative"
56
68
  ],
57
69
  "dependencies": {
58
70
  "@modelcontextprotocol/sdk": "^1.0.4",