@framers/agentos-ext-web-search 1.1.0 → 1.2.0

package/README.md

@@ -1,218 +1,61 @@
 # Web Search Extension for AgentOS
 
-Enable powerful web search capabilities for your AgentOS agents.
+Professional web search, research aggregation, and fact-checking for AgentOS agents.
 
-## Features
-
-- 🔍 **Web Search** - Search the web with multiple providers
-- 📚 **Research Aggregator** - Compile research from multiple searches
-- ✅ **Fact Checker** - Verify claims across sources
-- 🌍 **Multi-provider Support** - Serper, SerpAPI, Brave, DuckDuckGo
+Supports multiple providers (Serper.dev, SerpAPI, Brave) with a DuckDuckGo fallback when no API keys are configured.
 
 ## Installation
 
 ```bash
-npm install @framers/agentos-ext-search
+npm install @framers/agentos-ext-web-search
 ```
 
 ## Quick Start
 
-```typescript
-import { AgentOS } from '@framers/agentos';
-import searchExtension from '@framers/agentos-ext-search';
-
-const agentos = new AgentOS();
-
-await agentos.initialize({
-  // ... other config
-  extensionManifest: {
-    packs: [
-      {
-        factory: () => searchExtension({
-          manifestEntry: {} as any,
-          source: { sourceName: '@framers/agentos-ext-search' },
-          options: {
-            search: {
-              provider: 'serper',
-              apiKey: process.env.SERPER_API_KEY
-            }
-          }
-        })
-      }
-    ]
-  }
-});
-```
-
-## Configuration
-
-### Environment Variables
-
-```bash
-# Option 1: Serper (2,500 free queries)
-SERPER_API_KEY=your_serper_api_key
-
-# Option 2: SerpAPI (100 free/month)
-SERPAPI_API_KEY=your_serpapi_key
-
-# Option 3: Brave (2,000 free/month)
-BRAVE_SEARCH_API_KEY=your_brave_api_key
-
-# Optional: Override provider
-SEARCH_PROVIDER=serper  # or serpapi, brave, duckduckgo
-```
+```ts
+import { ExtensionManager } from '@framers/agentos';
+import { createExtensionPack } from '@framers/agentos-ext-web-search';
 
-### Programmatic Configuration
+const extensionManager = new ExtensionManager();
 
-```typescript
-{
-  search: {
-    provider: 'serper',  // 'serper' | 'serpapi' | 'brave' | 'duckduckgo'
-    apiKey: 'your-api-key',
-    rateLimit: 5  // requests per second
-  }
-}
+extensionManager.register(createExtensionPack({
+  options: {
+    // Any one of these is enough; Serper/SerpAPI/Brave are tried in order.
+    serperApiKey: process.env.SERPER_API_KEY,
+    serpApiKey: process.env.SERPAPI_API_KEY,
+    braveApiKey: process.env.BRAVE_API_KEY,
+  },
+  logger: console,
+}));
 ```
 
-## Available Tools
+## Environment Variables
 
-### 1. Web Search
+- `SERPER_API_KEY` (optional)
+- `SERPAPI_API_KEY` (optional)
+- `BRAVE_API_KEY` (optional)
 
-Search the web for information.
+If none are set, the extension falls back to DuckDuckGo.
 
-**Input:**
-```typescript
-{
-  query: string;           // Search query
-  numResults?: number;     // Results to return (1-10, default: 5)
-  searchType?: string;     // 'web' | 'news' | 'images' | 'videos'
-  timeRange?: string;      // 'any' | 'day' | 'week' | 'month' | 'year'
-  region?: string;         // Country code: 'us', 'uk', 'fr', etc.
-}
-```
+## Tools
 
-**Example:**
-```typescript
-const result = await tool.execute({
-  query: "latest AI developments",
-  numResults: 5,
-  timeRange: "week"
-});
-```
+### `web_search`
 
-### 2. Research Aggregator
+Search the web.
 
-Perform comprehensive research on a topic.
-
-**Input:**
-```typescript
-{
-  topic: string;                // Research topic
-  searchQueries?: string[];     // Specific queries to run
-  sources?: string[];           // 'web' | 'academic' | 'news'
-  maxResultsPerQuery?: number;  // Results per query (default: 3)
-}
-```
-
-### 3. Fact Checker
-
-Verify claims across multiple sources.
-
-**Input:**
-```typescript
-{
-  claim: string;         // Claim to verify
-  context?: string;      // Additional context
-  sources?: string[];    // Specific sources to check
-}
-```
+### `research_aggregate`
 
-## Provider Setup
+Run multiple searches and aggregate results.
 
-### Serper.dev (Recommended)
-1. Sign up: https://serper.dev/signup
-2. Get API key from dashboard
-3. 2,500 free queries
+### `fact_check`
 
-### SerpAPI
-1. Sign up: https://serpapi.com/users/sign_up
-2. Copy API key from account page
-3. 100 free searches/month
+Cross-check a claim across sources.
 
-### Brave Search
-1. Register: https://brave.com/search/api/
-2. Get API key
-3. 2,000 free queries/month
+## Notes
 
-### DuckDuckGo (Fallback)
-- No API key required
-- Limited functionality
-- Rate limited
-
-## Usage with Agents
-
-Once installed, agents can use search tools:
-
-```typescript
-// Agent conversation
-User: "Search for information about quantum computing"
-Agent: [Uses webSearch tool] Here's what I found about quantum computing...
-
-User: "Research the latest renewable energy developments"
-Agent: [Uses researchAggregator tool] I've compiled research from multiple sources...
-
-User: "Fact check: Is coffee the second most traded commodity?"
-Agent: [Uses factCheck tool] After checking multiple sources...
-```
-
-## Error Handling
-
-The extension handles various error scenarios:
-
-```typescript
-// Missing API key
-{
-  success: false,
-  error: "Search provider not configured",
-  details: {
-    message: "Please configure a search API key",
-    providers: [/* signup links */]
-  }
-}
-
-// Rate limiting
-// Automatically enforced based on provider limits
-
-// Network errors
-{
-  success: false,
-  error: "Search failed: Network error",
-  details: { provider: "serper", timestamp: "..." }
-}
-```
-
-## Development
-
-```bash
-# Install dependencies
-npm install
-
-# Build
-npm run build
-
-# Run tests
-npm test
-
-# Watch mode
-npm run dev
-```
+- Tools are read-only (no side effects).
+- Provider selection is automatic based on which keys are configured.
 
 ## License
 
 MIT © Frame.dev
-
-## Support
-
-- Issues: https://github.com/framersai/agentos-extensions/issues
-- Documentation: https://agentos.sh/docs/extensions
-- Discord: https://discord.gg/agentos

package/dist/index.d.ts

@@ -4,13 +4,13 @@
  * Provides web search capabilities through multiple providers with automatic fallback.
  *
  * @module @framers/agentos-ext-web-search
- * @version 1.0.0
+ * @version 1.1.0
  * @license MIT
  */
-import { ExtensionContext, ExtensionPack } from '@framers/agentos';
-import { WebSearchTool } from './tools/webSearch';
-import { ResearchAggregatorTool } from './tools/researchAggregator';
-import { FactCheckTool } from './tools/factCheck';
+import type { ExtensionPackContext, ExtensionPack } from '@framers/agentos';
+import { WebSearchTool } from './tools/webSearch.js';
+import { ResearchAggregatorTool } from './tools/researchAggregator.js';
+import { FactCheckTool } from './tools/factCheck.js';
 /**
  * Extension configuration options
  */
@@ -21,6 +21,12 @@ export interface WebSearchExtensionOptions {
     serpApiKey?: string;
     /** Brave Search API key */
     braveApiKey?: string;
+    /** Tavily API key — AI-optimized search for RAG, research, citations */
+    tavilyApiKey?: string;
+    /** Firecrawl API key — web scraping + search */
+    firecrawlApiKey?: string;
+    /** SearXNG instance URL (e.g. http://searxng:8080 or http://localhost:8888) */
+    searxngUrl?: string;
     /** Default maximum results for searches */
     defaultMaxResults?: number;
     /** Rate limiting configuration */
@@ -28,13 +34,15 @@ export interface WebSearchExtensionOptions {
         maxRequests: number;
         windowMs: number;
     };
+    /** Default multi-provider parallel search for all search tools */
+    defaultMultiSearch?: boolean;
     /** Extension priority in the stack */
     priority?: number;
 }
 /**
  * Creates the web search extension pack
  *
- * @param {ExtensionContext} context - The extension context
+ * @param {ExtensionPackContext} context - The extension context
  * @returns {ExtensionPack} The configured extension pack
  *
  * @example
@@ -45,14 +53,13 @@ export interface WebSearchExtensionOptions {
  *   options: {
  *     serperApiKey: process.env.SERPER_API_KEY,
  *     defaultMaxResults: 10
- *   },
- *   logger: console
+ *   }
  * });
  * ```
  */
-export declare function createExtensionPack(context: ExtensionContext): ExtensionPack;
+export declare function createExtensionPack(context: ExtensionPackContext): ExtensionPack;
 export { WebSearchTool, ResearchAggregatorTool, FactCheckTool };
-export { SearchProviderService, SearchResult, ProviderResponse } from './services/searchProvider';
-export type { SearchProviderConfig } from './services/searchProvider';
+export { SearchProviderService, SearchResult, ProviderResponse } from './services/searchProvider.js';
+export type { SearchProviderConfig, MultiSearchResult, MultiSearchResponse } from './services/searchProvider.js';
 export default createExtensionPack;
 //# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,OAAO,EAAE,gBAAgB,EAAE,aAAa,EAAE,MAAM,kBAAkB,CAAC;AACnE,OAAO,EAAE,aAAa,EAAE,MAAM,mBAAmB,CAAC;AAClD,OAAO,EAAE,sBAAsB,EAAE,MAAM,4BAA4B,CAAC;AACpE,OAAO,EAAE,aAAa,EAAE,MAAM,mBAAmB,CAAC;AAGlD;;GAEG;AACH,MAAM,WAAW,yBAAyB;IACxC,yBAAyB;IACzB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,sBAAsB;IACtB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,2BAA2B;IAC3B,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,2CAA2C;IAC3C,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,kCAAkC;IAClC,SAAS,CAAC,EAAE;QACV,WAAW,EAAE,MAAM,CAAC;QACpB,QAAQ,EAAE,MAAM,CAAC;KAClB,CAAC;IACF,sCAAsC;IACtC,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED;;;;;;;;;;;;;;;;;;GAkBG;AACH,wBAAgB,mBAAmB,CAAC,OAAO,EAAE,gBAAgB,GAAG,aAAa,CA0D5E;AAGD,OAAO,EAAE,aAAa,EAAE,sBAAsB,EAAE,aAAa,EAAE,CAAC;AAChE,OAAO,EAAE,qBAAqB,EAAE,YAAY,EAAE,gBAAgB,EAAE,MAAM,2BAA2B,CAAC;AAClG,YAAY,EAAE,oBAAoB,EAAE,MAAM,2BAA2B,CAAC;AAGtE,eAAe,mBAAmB,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AACA;;;;;;;;GAQG;AAEH,OAAO,KAAK,EAAE,oBAAoB,EAAE,aAAa,EAA6B,MAAM,kBAAkB,CAAC;AACvG,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAC;AACrD,OAAO,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AACvE,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAC;AAGrD;;GAEG;AACH,MAAM,WAAW,yBAAyB;IACxC,yBAAyB;IACzB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,sBAAsB;IACtB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,2BAA2B;IAC3B,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,wEAAwE;IACxE,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,gDAAgD;IAChD,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,+EAA+E;IAC/E,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,2CAA2C;IAC3C,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,kCAAkC;IAClC,SAAS,CAAC,EAAE;QACV,WAAW,EAAE,MAAM,CAAC;QACpB,QAAQ,EAAE,MAAM,CAAC;KAClB,CAAC;IACF,kEAAkE;IAClE,kBAAkB,CAAC,EAAE,OAAO,CAAC;IAC7B,sCAAsC;IACtC,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED;;;;;;;;;;;;;;;;;GAiBG;AACH,wBAAgB,mBAAmB,CAAC,OAAO,EAAE,oBAAoB,GAAG,aAAa,CA+EhF;AAGD,OAAO,EAAE,aAAa,EAAE,sBAAsB,EAAE,aAAa,EAAE,CAAC;AAChE,OAAO,EAAE,qBAAqB,EAAE,YAAY,EAAE,gBAAgB,EAAE,MAAM,8BAA8B,CAAC;AACrG,YAAY,EAAE,oBAAoB,EAAE,iBAAiB,EAAE,mBAAmB,EAAE,MAAM,8BAA8B,CAAC;AAGjH,eAAe,mBAAmB,CAAC"}

package/dist/index.js

@@ -1,27 +1,21 @@
-"use strict";
+// @ts-nocheck
 /**
  * AgentOS Web Search Extension
  *
  * Provides web search capabilities through multiple providers with automatic fallback.
  *
  * @module @framers/agentos-ext-web-search
- * @version 1.0.0
+ * @version 1.1.0
  * @license MIT
  */
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.SearchProviderService = exports.FactCheckTool = exports.ResearchAggregatorTool = exports.WebSearchTool = void 0;
-exports.createExtensionPack = createExtensionPack;
-const webSearch_1 = require("./tools/webSearch");
-Object.defineProperty(exports, "WebSearchTool", { enumerable: true, get: function () { return webSearch_1.WebSearchTool; } });
-const researchAggregator_1 = require("./tools/researchAggregator");
-Object.defineProperty(exports, "ResearchAggregatorTool", { enumerable: true, get: function () { return researchAggregator_1.ResearchAggregatorTool; } });
-const factCheck_1 = require("./tools/factCheck");
-Object.defineProperty(exports, "FactCheckTool", { enumerable: true, get: function () { return factCheck_1.FactCheckTool; } });
-const searchProvider_1 = require("./services/searchProvider");
+import { WebSearchTool } from './tools/webSearch.js';
+import { ResearchAggregatorTool } from './tools/researchAggregator.js';
+import { FactCheckTool } from './tools/factCheck.js';
+import { SearchProviderService } from './services/searchProvider.js';
 /**
  * Creates the web search extension pack
  *
- * @param {ExtensionContext} context - The extension context
+ * @param {ExtensionPackContext} context - The extension context
  * @returns {ExtensionPack} The configured extension pack
  *
  * @example
@@ -32,69 +26,82 @@ const searchProvider_1 = require("./services/searchProvider");
  *   options: {
  *     serperApiKey: process.env.SERPER_API_KEY,
  *     defaultMaxResults: 10
- *   },
- *   logger: console
+ *   }
  * });
  * ```
  */
-function createExtensionPack(context) {
-    const options = context.options || {};
+export function createExtensionPack(context) {
+    const options = (context.options ?? {});
+    const serperApiKey = options.serperApiKey || process.env.SERPER_API_KEY;
+    const serpApiKey = options.serpApiKey || process.env.SERPAPI_API_KEY;
+    const braveApiKey = options.braveApiKey || process.env.BRAVE_API_KEY;
+    const tavilyApiKey = options.tavilyApiKey || process.env.TAVILY_API_KEY;
+    const firecrawlApiKey = options.firecrawlApiKey || process.env.FIRECRAWL_API_KEY;
+    const searxngUrl = options.searxngUrl || process.env.SEARXNG_URL;
     // Initialize search service with configuration
-    const searchService = new searchProvider_1.SearchProviderService({
-        serperApiKey: options.serperApiKey,
-        serpApiKey: options.serpApiKey,
-        braveApiKey: options.braveApiKey,
+    const searchService = new SearchProviderService({
+        serperApiKey,
+        serpApiKey,
+        braveApiKey,
+        tavilyApiKey,
+        firecrawlApiKey,
+        searxngUrl,
         rateLimit: options.rateLimit
     });
+    const defaultMultiSearch = options.defaultMultiSearch ?? false;
     // Create tool instances
-    const webSearchTool = new webSearch_1.WebSearchTool(searchService);
-    const researchAggregator = new researchAggregator_1.ResearchAggregatorTool(searchService);
-    const factCheckTool = new factCheck_1.FactCheckTool(searchService);
+    const webSearchTool = new WebSearchTool(searchService, defaultMultiSearch);
+    const researchAggregator = new ResearchAggregatorTool(searchService, defaultMultiSearch);
+    const factCheckTool = new FactCheckTool(searchService, defaultMultiSearch);
     return {
         name: '@framers/agentos-ext-web-search',
-        version: '1.0.0',
+        version: '1.1.0',
         descriptors: [
             {
-                id: 'webSearch',
+                id: webSearchTool.name,
                 kind: 'tool',
                 priority: options.priority || 50,
-                payload: webSearchTool
+                payload: webSearchTool,
+                requiredSecrets: [
+                    { id: 'serper.apiKey', optional: true },
+                    { id: 'serpapi.apiKey', optional: true },
+                    { id: 'brave.apiKey', optional: true },
+                ],
             },
             {
-                id: 'researchAggregator',
+                id: researchAggregator.name,
                 kind: 'tool',
                 priority: options.priority || 50,
-                payload: researchAggregator
+                payload: researchAggregator,
+                requiredSecrets: [
+                    { id: 'serper.apiKey', optional: true },
+                    { id: 'serpapi.apiKey', optional: true },
+                    { id: 'brave.apiKey', optional: true },
+                ],
             },
             {
-                id: 'factCheck',
+                id: factCheckTool.name,
                 kind: 'tool',
                 priority: options.priority || 50,
-                payload: factCheckTool
+                payload: factCheckTool,
+                requiredSecrets: [
+                    { id: 'serper.apiKey', optional: true },
+                    { id: 'serpapi.apiKey', optional: true },
+                    { id: 'brave.apiKey', optional: true },
+                ],
             }
         ],
-        /**
-         * Called when extension is activated
-         */
-        onActivate: async () => {
-            if (context.onActivate) {
-                await context.onActivate();
-            }
-            context.logger?.info('Web Search Extension activated');
+        onActivate: async (lc) => {
+            lc.logger?.info('Web Search Extension activated');
         },
-        /**
-         * Called when extension is deactivated
-         */
-        onDeactivate: async () => {
-            if (context.onDeactivate) {
-                await context.onDeactivate();
-            }
-            context.logger?.info('Web Search Extension deactivated');
+        onDeactivate: async (lc) => {
+            lc.logger?.info('Web Search Extension deactivated');
         }
     };
 }
-var searchProvider_2 = require("./services/searchProvider");
-Object.defineProperty(exports, "SearchProviderService", { enumerable: true, get: function () { return searchProvider_2.SearchProviderService; } });
+// Export types for consumers
+export { WebSearchTool, ResearchAggregatorTool, FactCheckTool };
+export { SearchProviderService } from './services/searchProvider.js';
 // Default export for convenience
-exports.default = createExtensionPack;
+export default createExtensionPack;
 //# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAAA;;;;;;;;GAQG;;;AAgDH,kDA0DC;AAvGD,iDAAkD;AA0GzC,8FA1GA,yBAAa,OA0GA;AAzGtB,mEAAoE;AAyG5C,uGAzGf,2CAAsB,OAyGe;AAxG9C,iDAAkD;AAwGF,8FAxGvC,yBAAa,OAwGuC;AAvG7D,8DAAkE;AAuBlE;;;;;;;;;;;;;;;;;;GAkBG;AACH,SAAgB,mBAAmB,CAAC,OAAyB;IAC3D,MAAM,OAAO,GAAG,OAAO,CAAC,OAAoC,IAAI,EAAE,CAAC;IAEnE,+CAA+C;IAC/C,MAAM,aAAa,GAAG,IAAI,sCAAqB,CAAC;QAC9C,YAAY,EAAE,OAAO,CAAC,YAAY;QAClC,UAAU,EAAE,OAAO,CAAC,UAAU;QAC9B,WAAW,EAAE,OAAO,CAAC,WAAW;QAChC,SAAS,EAAE,OAAO,CAAC,SAAS;KAC7B,CAAC,CAAC;IAEH,wBAAwB;IACxB,MAAM,aAAa,GAAG,IAAI,yBAAa,CAAC,aAAa,CAAC,CAAC;IACvD,MAAM,kBAAkB,GAAG,IAAI,2CAAsB,CAAC,aAAa,CAAC,CAAC;IACrE,MAAM,aAAa,GAAG,IAAI,yBAAa,CAAC,aAAa,CAAC,CAAC;IAEvD,OAAO;QACL,IAAI,EAAE,iCAAiC;QACvC,OAAO,EAAE,OAAO;QAChB,WAAW,EAAE;YACX;gBACE,EAAE,EAAE,WAAW;gBACf,IAAI,EAAE,MAAM;gBACZ,QAAQ,EAAE,OAAO,CAAC,QAAQ,IAAI,EAAE;gBAChC,OAAO,EAAE,aAAa;aACvB;YACD;gBACE,EAAE,EAAE,oBAAoB;gBACxB,IAAI,EAAE,MAAM;gBACZ,QAAQ,EAAE,OAAO,CAAC,QAAQ,IAAI,EAAE;gBAChC,OAAO,EAAE,kBAAkB;aAC5B;YACD;gBACE,EAAE,EAAE,WAAW;gBACf,IAAI,EAAE,MAAM;gBACZ,QAAQ,EAAE,OAAO,CAAC,QAAQ,IAAI,EAAE;gBAChC,OAAO,EAAE,aAAa;aACvB;SACF;QACD;;WAEG;QACH,UAAU,EAAE,KAAK,IAAI,EAAE;YACrB,IAAI,OAAO,CAAC,UAAU,EAAE,CAAC;gBACvB,MAAM,OAAO,CAAC,UAAU,EAAE,CAAC;YAC7B,CAAC;YACD,OAAO,CAAC,MAAM,EAAE,IAAI,CAAC,gCAAgC,CAAC,CAAC;QACzD,CAAC;QACD;;WAEG;QACH,YAAY,EAAE,KAAK,IAAI,EAAE;YACvB,IAAI,OAAO,CAAC,YAAY,EAAE,CAAC;gBACzB,MAAM,OAAO,CAAC,YAAY,EAAE,CAAC;YAC/B,CAAC;YACD,OAAO,CAAC,MAAM,EAAE,IAAI,CAAC,kCAAkC,CAAC,CAAC;QAC3D,CAAC;KACF,CAAC;AACJ,CAAC;AAID,4DAAkG;AAAzF,uHAAA,qBAAqB,OAAA;AAG9B,iCAAiC;AACjC,kBAAe,mBAAmB,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,cAAc;AACd;;;;;;;;GAQG;AAGH,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAC;AACrD,OAAO,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AACvE,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAC;AACrD,OAAO,EAAE,qBAAqB,EAAE,MAAM,8BAA8B,CAAC;AA+BrE;;;;;;;;;;;;;;;;;GAiBG;AACH,MAAM,UAAU,mBAAmB,CAAC,OAA6B;IAC/D,MAAM,OAAO,GAAG,CAAC,OAAO,CAAC,OAAO,IAAI,EAAE,CAA8B,CAAC;IAErE,MAAM,YAAY,GAChB,OAAO,CAAC,YAAY,IAAI,OAAO,CAAC,GAAG,CAAC,cAAc,CAAC;IACrD,MAAM,UAAU,GACd,OAAO,CAAC,UAAU,IAAI,OAAO,CAAC,GAAG,CAAC,eAAe,CAAC;IACpD,MAAM,WAAW,GACf,OAAO,CAAC,WAAW,IAAI,OAAO,CAAC,GAAG,CAAC,aAAa,CAAC;IACnD,MAAM,YAAY,GAChB,OAAO,CAAC,YAAY,IAAI,OAAO,CAAC,GAAG,CAAC,cAAc,CAAC;IACrD,MAAM,eAAe,GACnB,OAAO,CAAC,eAAe,IAAI,OAAO,CAAC,GAAG,CAAC,iBAAiB,CAAC;IAC3D,MAAM,UAAU,GACd,OAAO,CAAC,UAAU,IAAI,OAAO,CAAC,GAAG,CAAC,WAAW,CAAC;IAEhD,+CAA+C;IAC/C,MAAM,aAAa,GAAG,IAAI,qBAAqB,CAAC;QAC9C,YAAY;QACZ,UAAU;QACV,WAAW;QACX,YAAY;QACZ,eAAe;QACf,UAAU;QACV,SAAS,EAAE,OAAO,CAAC,SAAS;KAC7B,CAAC,CAAC;IAEH,MAAM,kBAAkB,GAAG,OAAO,CAAC,kBAAkB,IAAI,KAAK,CAAC;IAE/D,wBAAwB;IACxB,MAAM,aAAa,GAAG,IAAI,aAAa,CAAC,aAAa,EAAE,kBAAkB,CAAC,CAAC;IAC3E,MAAM,kBAAkB,GAAG,IAAI,sBAAsB,CAAC,aAAa,EAAE,kBAAkB,CAAC,CAAC;IACzF,MAAM,aAAa,GAAG,IAAI,aAAa,CAAC,aAAa,EAAE,kBAAkB,CAAC,CAAC;IAE3E,OAAO;QACL,IAAI,EAAE,iCAAiC;QACvC,OAAO,EAAE,OAAO;QAChB,WAAW,EAAE;YACX;gBACE,EAAE,EAAE,aAAa,CAAC,IAAI;gBACtB,IAAI,EAAE,MAAM;gBACZ,QAAQ,EAAE,OAAO,CAAC,QAAQ,IAAI,EAAE;gBAChC,OAAO,EAAE,aAAa;gBACtB,eAAe,EAAE;oBACf,EAAE,EAAE,EAAE,eAAe,EAAE,QAAQ,EAAE,IAAI,EAAE;oBACvC,EAAE,EAAE,EAAE,gBAAgB,EAAE,QAAQ,EAAE,IAAI,EAAE;oBACxC,EAAE,EAAE,EAAE,cAAc,EAAE,QAAQ,EAAE,IAAI,EAAE;iBACvC;aACF;YACD;gBACE,EAAE,EAAE,kBAAkB,CAAC,IAAI;gBAC3B,IAAI,EAAE,MAAM;gBACZ,QAAQ,EAAE,OAAO,CAAC,QAAQ,IAAI,EAAE;gBAChC,OAAO,EAAE,kBAAkB;gBAC3B,eAAe,EAAE;oBACf,EAAE,EAAE,EAAE,eAAe,EAAE,QAAQ,EAAE,IAAI,EAAE;oBACvC,EAAE,EAAE,EAAE,gBAAgB,EAAE,QAAQ,EAAE,IAAI,EAAE;oBACxC,EAAE,EAAE,EAAE,cAAc,EAAE,QAAQ,EAAE,IAAI,EAAE;iBACvC;aACF;YACD;gBACE,EAAE,EAAE,aAAa,CAAC,IAAI;gBACtB,IAAI,EAAE,MAAM;gBACZ,QAAQ,EAAE,OAAO,CAAC,QAAQ,IAAI,EAAE;gBAChC,OAAO,EAAE,aAAa;gBACtB,eAAe,EAAE;oBACf,EAAE,EAAE,EAAE,eAAe,EAAE,QAAQ,EAAE,IAAI,EAAE;oBACvC,EAAE,EAAE,EAAE,gBAAgB,EAAE,QAAQ,EAAE,IAAI,EAAE;oBACxC,EAAE,EAAE,EAAE,cAAc,EAAE,QAAQ,EAAE,IAAI,EAAE;iBACvC;aACF;SACF;QACD,UAAU,EAAE,KAAK,EAAE,EAA6B,EAAE,EAAE;YAClD,EAAE,CAAC,MAAM,EAAE,IAAI,CAAC,gCAAgC,CAAC,CAAC;QACpD,CAAC;QACD,YAAY,EAAE,KAAK,EAAE,EAA6B,EAAE,EAAE;YACpD,EAAE,CAAC,MAAM,EAAE,IAAI,CAAC,kCAAkC,CAAC,CAAC;QACtD,CAAC;KACF,CAAC;AACJ,CAAC;AAED,6BAA6B;AAC7B,OAAO,EAAE,aAAa,EAAE,sBAAsB,EAAE,aAAa,EAAE,CAAC;AAChE,OAAO,EAAE,qBAAqB,EAAkC,MAAM,8BAA8B,CAAC;AAGrG,iCAAiC;AACjC,eAAe,mBAAmB,CAAC"}

package/dist/services/searchProvider.d.ts

@@ -5,6 +5,9 @@ export interface SearchProviderConfig {
     serperApiKey?: string;
     serpApiKey?: string;
     braveApiKey?: string;
+    searxngUrl?: string;
+    tavilyApiKey?: string;
+    firecrawlApiKey?: string;
     maxRetries?: number;
     rateLimit?: {
         maxRequests: number;
@@ -33,6 +36,35 @@ export interface ProviderResponse {
         fallback?: boolean;
     };
 }
+/**
+ * A search result enriched with cross-provider agreement data.
+ */
+export interface MultiSearchResult extends SearchResult {
+    /** Which providers returned this URL */
+    providers: string[];
+    /** Number of providers that returned this URL */
+    agreementCount: number;
+    /** Confidence score (0-100) based on cross-provider agreement + position */
+    confidenceScore: number;
+    /** The position this result appeared at in each provider's results */
+    providerPositions: Record<string, number>;
+}
+/**
+ * Response from a multi-provider parallel search aggregation.
+ */
+export interface MultiSearchResponse {
+    results: MultiSearchResult[];
+    metadata: {
+        query: string;
+        timestamp: string;
+        totalResponseTime: number;
+        providersQueried: string[];
+        providersSucceeded: string[];
+        providersFailed: string[];
+        totalRawResults: number;
+        deduplicatedCount: number;
+    };
+}
 /**
  * Service for managing multiple search providers with fallback support
  *
@@ -71,6 +103,7 @@ export declare class SearchProviderService {
     search(query: string, options?: {
         maxResults?: number;
         provider?: string;
+        category?: string;
     }): Promise<ProviderResponse>;
     /**
      * Searches using a specific provider
@@ -86,10 +119,9 @@ export declare class SearchProviderService {
     /**
      * Gets list of available providers based on configured API keys
      *
-     * @private
      * @returns {string[]} Array of available provider names
      */
-    private getAvailableProviders;
+    getAvailableProviders(): string[];
     /**
      * Checks if a provider is within rate limits
      *
@@ -136,7 +168,16 @@ export declare class SearchProviderService {
      */
     private searchDuckDuckGo;
     /**
-     * Search using DuckDuckGo instant answer API
+     * Search using DuckDuckGo HTML search page.
+     *
+     * The Instant Answer API (`api.duckduckgo.com/?format=json`) only returns
+     * Wikipedia-style abstracts and related topics — it does NOT return actual
+     * web search results.  For most factual queries (follower counts, prices,
+     * current stats) it returns nothing useful.
+     *
+     * This method POSTs to the HTML search endpoint instead, which returns real
+     * organic search results.  If the HTML scrape fails, it falls back to the
+     * Instant Answer API as a last resort.
      *
      * @private
      * @param {string} query - Search query
@@ -144,6 +185,51 @@ export declare class SearchProviderService {
      * @returns {Promise<SearchResult[]>} Search results
      */
     private searchDuckDuckGoAPI;
+    /**
+     * Scrapes real search results from DuckDuckGo's HTML search page.
+     *
+     * @private
+     */
+    private scrapeDuckDuckGoHTML;
+    /**
+     * DuckDuckGo Instant Answer API (limited — only returns Wikipedia-style
+     * abstracts and related topics, NOT real search results).
+     *
+     * @private
+     */
+    private searchDuckDuckGoInstantAnswer;
+    /**
+     * Search using a self-hosted SearXNG metasearch instance.
+     *
+     * @private
+     * @param {string} query - Search query
+     * @param {number} maxResults - Maximum results
+     * @param {Object} [options] - SearXNG-specific options
+     * @param {string} [options.categories] - SearXNG categories (e.g. 'general', 'news', 'images')
+     * @returns {Promise<SearchResult[]>} Search results
+     */
+    private searchSearXNG;
+    /**
+     * Normalizes a URL for deduplication — strips www, tracking params, trailing slash.
+     */
+    static normalizeUrl(rawUrl: string): string;
+    /**
+     * Search using the Tavily API (AI-optimized search).
+     * @see https://docs.tavily.com/docs/tavily-api/rest-api
+     */
+    private searchTavily;
+    /**
+     * Search using the Firecrawl /search endpoint.
+     * @see https://docs.firecrawl.dev/api-reference/endpoint/search
+     */
+    private searchFirecrawl;
+    /**
+     * Searches ALL available providers in parallel, then merges, deduplicates,
+     * and reranks results by cross-provider agreement.
+     */
+    multiSearch(query: string, options?: {
+        maxResults?: number;
+    }): Promise<MultiSearchResponse>;
     /**
      * Gets recommended search providers with signup information
      *

package/dist/services/searchProvider.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"searchProvider.d.ts","sourceRoot":"","sources":["../../src/services/searchProvider.ts"],"names":[],"mappings":"AAAA;;GAEG;AACH,MAAM,WAAW,oBAAoB;IACnC,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,SAAS,CAAC,EAAE;QACV,WAAW,EAAE,MAAM,CAAC;QACpB,QAAQ,EAAE,MAAM,CAAC;KAClB,CAAC;CACH;AAED;;GAEG;AACH,MAAM,WAAW,YAAY;IAC3B,KAAK,EAAE,MAAM,CAAC;IACd,GAAG,EAAE,MAAM,CAAC;IACZ,OAAO,EAAE,MAAM,CAAC;IAChB,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED;;GAEG;AACH,MAAM,WAAW,gBAAgB;IAC/B,QAAQ,EAAE,MAAM,CAAC;IACjB,OAAO,EAAE,YAAY,EAAE,CAAC;IACxB,QAAQ,EAAE;QACR,KAAK,EAAE,MAAM,CAAC;QACd,SAAS,EAAE,MAAM,CAAC;QAClB,YAAY,CAAC,EAAE,MAAM,CAAC;QACtB,QAAQ,CAAC,EAAE,OAAO,CAAC;KACpB,CAAC;CACH;AAED;;;;;;;;;;;;;;GAcG;AACH,qBAAa,qBAAqB;IAChC,OAAO,CAAC,MAAM,CAAuB;IACrC,OAAO,CAAC,cAAc,CAAoD;IAE1E;;;;OAIG;gBACS,MAAM,EAAE,oBAAoB;IAYxC;;;;;;;;;;OAUG;IACG,MAAM,CACV,KAAK,EAAE,MAAM,EACb,OAAO,GAAE;QACP,UAAU,CAAC,EAAE,MAAM,CAAC;QACpB,QAAQ,CAAC,EAAE,MAAM,CAAA;KACb,GACL,OAAO,CAAC,gBAAgB,CAAC;IA2B5B;;;;;;;;;OASG;YACW,kBAAkB;IAmChC;;;;;OAKG;IACH,OAAO,CAAC,qBAAqB;IAU7B;;;;;;OAMG;YACW,cAAc;IAoB5B;;;;;;;OAOG;YACW,YAAY;IAqB1B;;;;;;;OAOG;YACW,aAAa;IAmB3B;;;;;;;OAOG;YACW,WAAW;IAuBzB;;;;;;;;OAQG;YACW,gBAAgB;IAgC9B;;;;;;;OAOG;YACW,mBAAmB;IAuCjC;;;;;OAKG;IACH,MAAM,CAAC,uBAAuB;;;;;;CA4B/B"}
+{"version":3,"file":"searchProvider.d.ts","sourceRoot":"","sources":["../../src/services/searchProvider.ts"],"names":[],"mappings":"AACA;;GAEG;AACH,MAAM,WAAW,oBAAoB;IACnC,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,SAAS,CAAC,EAAE;QACV,WAAW,EAAE,MAAM,CAAC;QACpB,QAAQ,EAAE,MAAM,CAAC;KAClB,CAAC;CACH;AAED;;GAEG;AACH,MAAM,WAAW,YAAY;IAC3B,KAAK,EAAE,MAAM,CAAC;IACd,GAAG,EAAE,MAAM,CAAC;IACZ,OAAO,EAAE,MAAM,CAAC;IAChB,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED;;GAEG;AACH,MAAM,WAAW,gBAAgB;IAC/B,QAAQ,EAAE,MAAM,CAAC;IACjB,OAAO,EAAE,YAAY,EAAE,CAAC;IACxB,QAAQ,EAAE;QACR,KAAK,EAAE,MAAM,CAAC;QACd,SAAS,EAAE,MAAM,CAAC;QAClB,YAAY,CAAC,EAAE,MAAM,CAAC;QACtB,QAAQ,CAAC,EAAE,OAAO,CAAC;KACpB,CAAC;CACH;AAED;;GAEG;AACH,MAAM,WAAW,iBAAkB,SAAQ,YAAY;IACrD,wCAAwC;IACxC,SAAS,EAAE,MAAM,EAAE,CAAC;IACpB,iDAAiD;IACjD,cAAc,EAAE,MAAM,CAAC;IACvB,4EAA4E;IAC5E,eAAe,EAAE,MAAM,CAAC;IACxB,sEAAsE;IACtE,iBAAiB,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;CAC3C;AAED;;GAEG;AACH,MAAM,WAAW,mBAAmB;IAClC,OAAO,EAAE,iBAAiB,EAAE,CAAC;IAC7B,QAAQ,EAAE;QACR,KAAK,EAAE,MAAM,CAAC;QACd,SAAS,EAAE,MAAM,CAAC;QAClB,iBAAiB,EAAE,MAAM,CAAC;QAC1B,gBAAgB,EAAE,MAAM,EAAE,CAAC;QAC3B,kBAAkB,EAAE,MAAM,EAAE,CAAC;QAC7B,eAAe,EAAE,MAAM,EAAE,CAAC;QAC1B,eAAe,EAAE,MAAM,CAAC;QACxB,iBAAiB,EAAE,MAAM,CAAC;KAC3B,CAAC;CACH;AAED;;;;;;;;;;;;;;GAcG;AACH,qBAAa,qBAAqB;IAChC,OAAO,CAAC,MAAM,CAAuB;IACrC,OAAO,CAAC,cAAc,CAAoD;IAE1E;;;;OAIG;gBACS,MAAM,EAAE,oBAAoB;IAiBxC;;;;;;;;;;OAUG;IACG,MAAM,CACV,KAAK,EAAE,MAAM,EACb,OAAO,GAAE;QACP,UAAU,CAAC,EAAE,MAAM,CAAC;QACpB,QAAQ,CAAC,EAAE,MAAM,CAAC;QAClB,QAAQ,CAAC,EAAE,MAAM,CAAC;KACd,GACL,OAAO,CAAC,gBAAgB,CAAC;IA2B5B;;;;;;;;;OASG;YACW,kBAAkB;IA6ChC;;;;OAIG;IACI,qBAAqB,IAAI,MAAM,EAAE;IAaxC;;;;;;OAMG;YACW,cAAc;IAoB5B;;;;;;;OAOG;YACW,YAAY;IAqB1B;;;;;;;OAOG;YACW,aAAa;IAmB3B;;;;;;;OAOG;YACW,WAAW;IAuBzB;;;;;;;;OAQG;YACW,gBAAgB;IAgC9B;;;;;;;;;;;;;;;;OAgBG;YACW,mBAAmB;IAYjC;;;;OAIG;YACW,oBAAoB;IAqDlC;;;;;OAKG;YACW,6BAA6B;IAoC3C;;;;;;;;;OASG;YACW,aAAa;IAyB3B;;OAEG;IACH,MAAM,CAAC,YAAY,CAAC,MAAM,EAAE,MAAM,GAAG,MAAM;IAiB3C;;;OAGG;YACW,YAAY;IAuC1B;;;OAGG;YACW,eAAe;IAuC7B;;;OAGG;IACG,WAAW,CACf,KAAK,EAAE,MAAM,EACb,OAAO,GAAE;QAAE,UAAU,CAAC,EAAE,MAAM,CAAA;KAAO,GACpC,OAAO,CAAC,mBAAmB,CAAC;IAkG/B;;;;;OAKG;IACH,MAAM,CAAC,uBAAuB;;;;;;CAkC/B"}