spectrawl 0.3.16 → 0.3.18

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/README.md +18 -48
  2. package/package.json +1 -1
  3. package/src/config.js +1 -1
package/README.md CHANGED
@@ -1,50 +1,20 @@
1
1
  # Spectrawl
2
2
 
3
- The unified web layer for AI agents. Search, browse, authenticate, and act on platforms — one tool, self-hosted, free.
3
+ The unified web layer for AI agents. Search, browse, authenticate, and act on platforms — one package, self-hosted.
4
4
 
5
- **5,000 free searches/month** with Google-quality results via Gemini Grounded Search. Better answers than Tavily. Self-hosted.
5
+ **5,000 free searches/month** via Gemini Grounded Search. Full page scraping, stealth browsing, 24 platform adapters.
6
6
 
7
7
  ## What It Does
8
8
 
9
- AI agents need to interact with the web. That means searching, browsing pages, logging into platforms, and posting content. Today you duct-tape together Playwright + Tavily + cookie managers + platform-specific scripts. Spectrawl replaces all of that.
9
+ AI agents need to interact with the web — searching, browsing pages, logging into platforms, posting content. Today you wire together Playwright + a search API + cookie managers + platform-specific scripts. Spectrawl is one package that does all of it.
10
10
 
11
11
  ```
12
12
  npm install spectrawl
13
13
  ```
14
14
 
15
- ## Real Output
15
+ ## How It Works
16
16
 
17
- Here's actual output from Spectrawl vs Tavily on the same query:
18
-
19
- **Query:** `"best open source AI agent frameworks 2025"`
20
-
21
- ### Spectrawl (free)
22
- ```
23
- Time: 16.8s | Sources: 19
24
-
25
- Answer: The leading open-source AI agent frameworks for 2025 include AutoGen,
26
- CrewAI, LangChain, LangGraph, and Semantic Kernel [1, 2, 3]. AutoGen is
27
- recognized for enabling complex multi-agent conversations, while CrewAI
28
- focuses on orchestrating collaborative AI agents [1, 2]. LangChain and its
29
- component LangGraph provide robust tools for building sophisticated agent
30
- workflows and state management [1, 2, 3]. Semantic Kernel, developed by
31
- Microsoft, integrates large language models with conventional programming
32
- languages [1, 2, 3].
33
-
34
- Other prominent frameworks include LlamaIndex, Haystack, BabyAGI, AgentGPT,
35
- SuperAGI, MetaGPT, and Open Interpreter [1, 2].
36
- ```
37
- **12 frameworks named, inline citations, 19 sources**
38
-
39
- ### Tavily ($0.01/query)
40
- ```
41
- Time: 2s | Sources: 10
42
-
43
- Answer: In 2025, LangGraph and Microsoft's AutoGen + Semantic Kernel are
44
- top open-source AI agent frameworks, favored for their robust orchestration
45
- and enterprise security features.
46
- ```
47
- **3 frameworks named, no citations, 10 sources**
17
+ Spectrawl searches via Gemini Grounded Search (Google-quality results), scrapes the top pages for full content, and returns everything to your agent. Your agent's LLM reads the actual sources and forms its own answer — no pre-chewed summaries.
48
18
 
49
19
  ## Quick Start
50
20
 
@@ -74,22 +44,22 @@ const basic = await web.search('query')
74
44
 
75
45
  > **Why no summary by default?** Your agent already has an LLM. If we summarize AND your agent summarizes, you're paying two LLMs for one answer. We return rich sources — your agent does the rest.
76
46
 
77
- ## vs Tavily
47
+ ## Spectrawl vs Tavily
48
+
49
+ Different tools for different needs.
78
50
 
79
51
  | | Tavily | Spectrawl |
80
52
  |---|---|---|
81
- | Speed | ~2s ✅ | ~7-17s |
82
- | Answer quality | Generic (3 items) | **Detailed** (12+ items) ✅ |
83
- | Inline citations | ❌ | **[1] [2] [3]** ✅ |
84
- | Results per query | 10 | **12-19** ✅ |
85
- | Cost | $0.01/query | **Free** (5K/mo) ✅ |
86
- | Self-hosted | No | **Yes** ✅ |
87
- | Source ranking | No | **Domain trust scoring** ✅ |
88
- | Stealth scraping | No | **Yes** ✅ |
89
- | Auth + posting | No | **24 adapters** ✅ |
90
- | Cached repeats | No | **<1ms** ✅ |
91
-
92
- Spectrawl wins on answer quality, result volume, features, and cost. Tavily wins on speed.
53
+ | Speed | ~2s | ~6-10s |
54
+ | Free tier | 1,000/month | 5,000/month |
55
+ | Returns | Snippets + AI answer | Full page content + snippets |
56
+ | Self-hosted | No | Yes |
57
+ | Stealth browsing | No | Yes (Camoufox + Playwright) |
58
+ | Platform posting | No | 24 adapters |
59
+ | Auth management | No | Cookie store + auto-refresh |
60
+ | Cached repeats | No | <1ms |
61
+
62
+ **Tavily** is fast and simple — great for agents that need quick answers. **Spectrawl** returns richer data and does more (browse, auth, post) — but it's slower. Choose based on your use case.
93
63
 
94
64
  ## Search
95
65
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "spectrawl",
3
- "version": "0.3.16",
3
+ "version": "0.3.18",
4
4
  "description": "The unified web layer for AI agents. Search (6 engines), stealth browse (Camoufox + Playwright), auth (cookies, multi-account), act (24 adapters, 30+ platforms), proxy rotation. Self-hosted, free.",
5
5
  "main": "src/index.js",
6
6
  "types": "index.d.ts",
package/src/config.js CHANGED
@@ -4,7 +4,7 @@ const path = require('path')
4
4
  const DEFAULTS = {
5
5
  port: 3900,
6
6
  search: {
7
- cascade: ['gemini-grounded', 'tavily', 'brave', 'ddg'],
7
+ cascade: ['gemini-grounded', 'tavily', 'brave'],
8
8
  scrapeTop: 5,
9
9
  searxng: { url: 'http://localhost:8888' },
10
10
  llm: null // { provider, model, apiKey }