@oevortex/ddg_search 1.1.6 → 1.1.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,10 +1,26 @@
1
1
  # Changelog
2
2
 
3
3
  All notable changes to this project will be documented in this file.
4
+ ## [1.1.8] - 2025-12-03
5
+ ### Added
6
+ - Added new `getRandomUserAgent` function to rotate user agents
7
+ - Added new `src/utils/user_agents.js` file containing list of user agents
8
+ - switch to use `user_agents.js` file for user agent rotation
9
+ - Removed stream from iaskTool.js & search_iask.js
10
+ - Added new `monica-search` tool for AI-powered search using Monica AI
11
+
12
+ ### Changed
13
+ - Updated `src/index.ts` to use IAsk tool instead of Felo tool
14
+ - Updated `package.json` description, keywords, and dependencies (`turndown`, `ws`)
15
+ - Updated `README.md` to reference IAsk AI and document new tool parameters
16
+ - Removed old Felo tool files (`feloTool.js`, `search_felo.js`)
4
17
 
5
- ## [1.1.5] - 2025-11-30
18
+ ## [1.1.7] - 2025-11-30
6
19
  ### Changed
7
20
  - Replaced Felo AI tool with IAsk AI tool for advanced AI-powered search
21
+ - Added new dependencies: `turndown` for HTML to Markdown conversion, `ws` for WebSocket support
22
+ - Updated README to reflect changes and new tool usage
23
+ - Added new modes: 'short', 'detailed' in web search tool
8
24
  - Added `src/utils/search_iask.js` implementing IAsk API client
9
25
  - Added `src/tools/iaskTool.js` tool definition and handler
10
26
  - Updated `src/index.ts` to use IAsk tool instead of Felo
package/README.md CHANGED
@@ -1,12 +1,22 @@
1
1
  <div align="center">
2
- <img src="https://img.shields.io/npm/v/@oevortex/ddg_search.svg" alt="npm version" />
3
- <img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License: Apache 2.0" />
4
- <img src="https://img.shields.io/badge/YouTube-%40OEvortex-red.svg" alt="YouTube Channel" />
5
- <h1>DuckDuckGo & IAsk AI Search MCP 🔍🧠</h1>
6
- <p>A blazing-fast, privacy-friendly Model Context Protocol (MCP) server for web search and AI-powered responses using DuckDuckGo and IAsk AI.</p>
2
+ <a href="https://www.npmjs.com/package/@oevortex/ddg_search">
3
+ <img src="https://img.shields.io/npm/v/@oevortex/ddg_search.svg" alt="npm version" />
4
+ </a>
5
+ <a href="https://github.com/OEvortex/ddg_search/blob/main/LICENSE">
6
+ <img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License: Apache 2.0" />
7
+ </a>
8
+ <a href="https://youtube.com/@OEvortex">
9
+ <img src="https://img.shields.io/badge/YouTube-%40OEvortex-red.svg" alt="YouTube Channel" />
10
+ </a>
11
+ <h1>DuckDuckGo, IAsk AI & Monica Search MCP <span style="font-size:2.2rem;">🔍🧠</span></h1>
12
+ <p style="font-size:1.15rem; max-width:600px; margin:0 auto;">
13
+ <strong>Lightning-fast, privacy-first Model Context Protocol (MCP) server for web search and AI-powered answers.<br>
14
+ Powered by DuckDuckGo, IAsk AI and Monica.</strong>
15
+ </p>
7
16
  <a href="https://glama.ai/mcp/servers/@OEvortex/ddg_search">
8
17
  <img width="380" height="200" src="https://glama.ai/mcp/servers/@OEvortex/ddg_search/badge" alt="DuckDuckGo Search MCP server" />
9
18
  </a>
19
+ <br>
10
20
  <a href="https://youtube.com/@OEvortex"><strong>Subscribe for updates & tutorials</strong></a>
11
21
  </div>
12
22
 
@@ -20,13 +30,14 @@
20
30
  ## ✨ Features
21
31
 
22
32
  <div style="display: flex; flex-wrap: wrap; gap: 1.5em; margin-bottom: 1.5em;"> <div><b>🌐 Web search</b> using DuckDuckGo HTML</div>
23
- <div><b>🧠 AI search</b> using IAsk AI</div>
33
+ <div><b>🧠 AI search</b> using IAsk AI & Monica</div>
24
34
  <div><b>⚡ Performance optimized</b> with caching</div>
25
35
  <div><b>🛡️ Security features</b> including rate limiting and rotating user agents</div>
26
36
  <div><b>🔌 MCP-compliant</b> server implementation</div>
27
37
  <div><b>🆓 No API keys required</b> - works out of the box</div>
28
38
  </div>
29
39
 
40
+
30
41
  > [!IMPORTANT]
31
42
  > Unlike many search tools, this package performs actual web scraping rather than using limited APIs, giving you more comprehensive results.
32
43
 
@@ -42,6 +53,7 @@ npx -y @oevortex/ddg_search@latest
42
53
  ```
43
54
  </div>
44
55
 
56
+
45
57
  > [!TIP]
46
58
  > This will download and run the latest version of the MCP server directly without installation – perfect for quick use with AI assistants.
47
59
 
@@ -188,10 +200,17 @@ Or if installed globally:
188
200
  <li><b>query</b> (string, required): The search query or question</li>
189
201
  <li><b>mode</b> (string, optional, default: "question"): Search mode - "question", "academic", "forums", "wiki", or "thinking"</li>
190
202
  <li><b>detailLevel</b> (string, optional): Response detail level - "concise", "detailed", or "comprehensive"</li>
191
- <li><b>stream</b> (boolean, optional, default: false): Whether to stream the response</li>
192
203
  </ul>
193
204
  <i>Example: Search IAsk AI for "Explain quantum computing in simple terms"</i>
194
205
  </div>
206
+ <div style="margin-bottom: 1.5em;">
207
+ <b>🤖 Monica AI Search Tool</b><br/>
208
+ <code>monica-search</code><br/>
209
+ <ul>
210
+ <li><b>query</b> (string, required): The search query or question</li>
211
+ </ul>
212
+ <i>Example: Search Monica AI for "Latest advancements in AI"</i>
213
+ </div>
195
214
  </div>
196
215
 
197
216
  ---
@@ -206,8 +225,11 @@ src/
206
225
  tools/ # Tool definitions and handlers
207
226
  searchTool.js
208
227
  iaskTool.js
228
+ monicaTool.js
209
229
  utils/
210
230
  search.js # Search and URL utilities
231
+ user_agents.js
232
+ search_monica.js
211
233
  search_iask.js # IAsk AI search utilities
212
234
  package.json
213
235
  README.md
package/bin/cli.js CHANGED
@@ -13,13 +13,14 @@ async function startServer() {
13
13
  // Dynamically import the modules
14
14
  const { searchToolDefinition, searchToolHandler } = await import(`${modulePath}/tools/searchTool.js`);
15
15
  const { iaskToolDefinition, iaskToolHandler } = await import(`${modulePath}/tools/iaskTool.js`);
16
+ const { monicaToolDefinition, monicaToolHandler } = await import(`${modulePath}/tools/monicaTool.js`);
16
17
 
17
18
  // Create the MCP server
18
19
  const server = new Server({
19
20
  id: 'ddg-search-mcp',
20
- name: 'DuckDuckGo & IAsk AI Search MCP',
21
- description: 'A Model Context Protocol server for web search using DuckDuckGo and IAsk AI',
22
- version: '1.1.4'
21
+ name: 'DuckDuckGo, IAsk AI & Monica Search MCP',
22
+ description: 'A Model Context Protocol server for web search using DuckDuckGo, IAsk AI and Monica',
23
+ version: '1.1.8'
23
24
  }, {
24
25
  capabilities: {
25
26
  tools: {
@@ -31,7 +32,8 @@ async function startServer() {
31
32
  // Global variable to track available tools
32
33
  let availableTools = [
33
34
  searchToolDefinition,
34
- iaskToolDefinition
35
+ iaskToolDefinition,
36
+ monicaToolDefinition
35
37
  ];
36
38
 
37
39
  // Define available tools
@@ -54,7 +56,7 @@ async function startServer() {
54
56
  const { name, arguments: args } = request.params;
55
57
 
56
58
  // Validate tool name
57
- const validTools = ['web-search', 'iask-search'];
59
+ const validTools = ['web-search', 'iask-search', 'monica-search'];
58
60
  if (!validTools.includes(name)) {
59
61
  throw new Error(`Unknown tool: ${name}`);
60
62
  }
@@ -67,6 +69,9 @@ async function startServer() {
67
69
  case 'iask-search':
68
70
  return await iaskToolHandler(args);
69
71
 
72
+ case 'monica-search':
73
+ return await monicaToolHandler(args);
74
+
70
75
  default:
71
76
  throw new Error(`Tool not found: ${name}`);
72
77
  }
@@ -96,7 +101,7 @@ async function startServer() {
96
101
  // Start the server with stdio transport
97
102
  const transport = new StdioServerTransport();
98
103
  await server.connect(transport);
99
- console.error('DuckDuckGo & IAsk AI Search MCP server started and listening on stdio');
104
+ console.error('DuckDuckGo, IAsk AI & Monica Search MCP server started and listening on stdio');
100
105
  } catch (error) {
101
106
  console.error('Failed to start server:', error);
102
107
  process.exit(1);
@@ -110,7 +115,7 @@ const versionFlag = args.includes('--version') || args.includes('-v');
110
115
 
111
116
  if (helpFlag) {
112
117
  console.log(`
113
- DuckDuckGo & IAsk AI Search MCP - A Model Context Protocol server for web search
118
+ DuckDuckGo, IAsk AI & Monica Search MCP - A Model Context Protocol server for web search
114
119
 
115
120
  Usage:
116
121
  npx -y @oevortex/ddg_search@latest [options]
@@ -122,6 +127,7 @@ Options:
122
127
  This MCP server provides the following tools:
123
128
  - web-search: Search the web using DuckDuckGo
124
129
  - iask-search: Search using IAsk AI for AI-generated responses
130
+ - monica-search: Search using Monica AI for AI-generated responses
125
131
 
126
132
  Created by @OEvortex
127
133
  Subscribe to youtube.com/@OEvortex for more tools and tutorials!
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@oevortex/ddg_search",
3
- "version": "1.1.6",
3
+ "version": "1.1.8",
4
4
  "description": "A Model Context Protocol server for web search using DuckDuckGo and IAsk AI",
5
5
  "main": "src/index.js",
6
6
  "module": "src/index.ts",
package/src/index.js CHANGED
@@ -4,6 +4,7 @@ import { CallToolRequestSchema, ListToolsRequestSchema } from '@modelcontextprot
4
4
  // Import tool definitions and handlers
5
5
  import { searchToolDefinition, searchToolHandler } from './tools/searchTool.js';
6
6
  import { iaskToolDefinition, iaskToolHandler } from './tools/iaskTool.js';
7
+ import { monicaToolDefinition, monicaToolHandler } from './tools/monicaTool.js';
7
8
 
8
9
  // Required: Export default createServer function for Smithery
9
10
  export default function createServer({ config } = {}) {
@@ -12,7 +13,8 @@ export default function createServer({ config } = {}) {
12
13
  // Global variable to track available tools
13
14
  const availableTools = [
14
15
  searchToolDefinition,
15
- iaskToolDefinition
16
+ iaskToolDefinition,
17
+ monicaToolDefinition
16
18
  ];
17
19
 
18
20
  console.log('Available tools:', availableTools.map(t => t.name));
@@ -51,6 +53,9 @@ export default function createServer({ config } = {}) {
51
53
  case 'iask-search':
52
54
  return await iaskToolHandler(args);
53
55
 
56
+ case 'monica-search':
57
+ return await monicaToolHandler(args);
58
+
54
59
  default:
55
60
  throw new Error(`Tool not found: ${name}`);
56
61
  }
package/src/index.ts CHANGED
@@ -4,6 +4,7 @@ import { CallToolRequestSchema, ListToolsRequestSchema } from '@modelcontextprot
4
4
  // Import tool definitions and handlers
5
5
  import { searchToolDefinition, searchToolHandler } from './tools/searchTool.js';
6
6
  import { iaskToolDefinition, iaskToolHandler } from './tools/iaskTool.js';
7
+ import { monicaToolDefinition, monicaToolHandler } from './tools/monicaTool.js';
7
8
 
8
9
  // Required: Export default createServer function for Smithery
9
10
  export default function createServer({ config }: { config?: any } = {}) {
@@ -12,7 +13,8 @@ export default function createServer({ config }: { config?: any } = {}) {
12
13
  // Global variable to track available tools
13
14
  const availableTools = [
14
15
  searchToolDefinition,
15
- iaskToolDefinition
16
+ iaskToolDefinition,
17
+ monicaToolDefinition
16
18
  ];
17
19
 
18
20
  console.log('Available tools:', availableTools.map(t => t.name));
@@ -51,6 +53,9 @@ export default function createServer({ config }: { config?: any } = {}) {
51
53
  case 'iask-search':
52
54
  return await iaskToolHandler(args);
53
55
 
56
+ case 'monica-search':
57
+ return await monicaToolHandler(args);
58
+
54
59
  default:
55
60
  throw new Error(`Tool not found: ${name}`);
56
61
  }
@@ -24,11 +24,6 @@ export const iaskToolDefinition = {
24
24
  type: 'string',
25
25
  description: 'Level of detail in the response. Options: "concise" (brief), "detailed" (moderate), "comprehensive" (extensive). Default is null (standard response).',
26
26
  enum: VALID_DETAIL_LEVELS
27
- },
28
- stream: {
29
- type: 'boolean',
30
- description: 'Enable streaming mode to receive incremental results. Default is false.',
31
- default: false
32
27
  }
33
28
  },
34
29
  required: ['query']
@@ -47,45 +42,23 @@ export const iaskToolDefinition = {
47
42
  export async function iaskToolHandler(params) {
48
43
  const {
49
44
  query,
50
- mode = 'question',
51
- detailLevel = null,
52
- stream = false
45
+ mode = 'thinking',
46
+ detailLevel = null
53
47
  } = params;
54
48
 
55
- console.log(`Searching IAsk AI for: "${query}" (mode: ${mode}, detailLevel: ${detailLevel || 'default'}, stream: ${stream})`);
49
+ console.log(`Searching IAsk AI for: "${query}" (mode: ${mode}, detailLevel: ${detailLevel || 'default'})`);
56
50
 
57
51
  try {
58
- if (stream) {
59
- // For streaming responses, collect them and return
60
- let fullResponse = '';
61
- const chunks = [];
62
-
63
- for await (const chunk of await searchIAsk(query, true, false, mode, detailLevel)) {
64
- chunks.push(chunk);
65
- fullResponse += chunk;
66
- }
67
-
68
- return {
69
- content: [
70
- {
71
- type: 'text',
72
- text: fullResponse || 'No results found.'
73
- }
74
- ]
75
- };
76
- } else {
77
- // For non-streaming responses
78
- const response = await searchIAsk(query, false, false, mode, detailLevel);
79
-
80
- return {
81
- content: [
82
- {
83
- type: 'text',
84
- text: response || 'No results found.'
85
- }
86
- ]
87
- };
88
- }
52
+ const response = await searchIAsk(query, mode, detailLevel);
53
+
54
+ return {
55
+ content: [
56
+ {
57
+ type: 'text',
58
+ text: response || 'No results found.'
59
+ }
60
+ ]
61
+ };
89
62
  } catch (error) {
90
63
  console.error(`Error in IAsk search: ${error.message}`);
91
64
  return {
@@ -0,0 +1,58 @@
1
+ import { searchMonica } from '../utils/search_monica.js';
2
+
3
+ /**
4
+ * Monica AI search tool definition
5
+ */
6
+ export const monicaToolDefinition = {
7
+ name: 'monica-search',
8
+ title: 'Monica AI Search',
9
+ description: 'AI-powered search using Monica AI. Returns AI-generated responses based on web content.',
10
+ inputSchema: {
11
+ type: 'object',
12
+ properties: {
13
+ query: {
14
+ type: 'string',
15
+ description: 'The search query or question.'
16
+ }
17
+ },
18
+ required: ['query']
19
+ },
20
+ annotations: {
21
+ readOnlyHint: true,
22
+ openWorldHint: false
23
+ }
24
+ };
25
+
26
+ /**
27
+ * Monica AI search tool handler
28
+ * @param {Object} params - The tool parameters
29
+ * @returns {Promise<Object>} - The tool result
30
+ */
31
+ export async function monicaToolHandler(params) {
32
+ const { query } = params;
33
+
34
+ console.log(`Searching Monica AI for: "${query}"`);
35
+
36
+ try {
37
+ const result = await searchMonica(query);
38
+ return {
39
+ content: [
40
+ {
41
+ type: 'text',
42
+ text: result || 'No results found.'
43
+ }
44
+ ]
45
+ };
46
+ } catch (error) {
47
+ console.error(`Error in Monica search: ${error.message}`);
48
+ return {
49
+ isError: true,
50
+ content: [
51
+ {
52
+ type: 'text',
53
+ text: `Error searching Monica: ${error.message}`
54
+ }
55
+ ]
56
+ };
57
+ }
58
+ }
@@ -6,26 +6,26 @@ import { searchDuckDuckGo } from '../utils/search.js';
6
6
  export const searchToolDefinition = {
7
7
  name: 'web-search',
8
8
  title: 'Web Search',
9
- description: 'Search the web using DuckDuckGo and return comprehensive results with titles, URLs, and snippets',
9
+ description: 'Perform a web search using DuckDuckGo and receive detailed results including titles, URLs, and summaries.',
10
10
  inputSchema: {
11
11
  type: 'object',
12
12
  properties: {
13
13
  query: {
14
14
  type: 'string',
15
- description: 'The search query to find relevant web pages'
16
- },
17
- page: {
18
- type: 'integer',
19
- description: 'Page number for pagination (default: 1)',
20
- default: 1,
21
- minimum: 1
15
+ description: 'Enter your search query to find the most relevant web pages.'
22
16
  },
23
17
  numResults: {
24
18
  type: 'integer',
25
- description: 'Number of results to return per page (default: 10, max: 20)',
26
- default: 10,
19
+ description: 'Specify how many results to display (default: 3, maximum: 20).',
20
+ default: 3,
27
21
  minimum: 1,
28
22
  maximum: 20
23
+ },
24
+ mode: {
25
+ type: 'string',
26
+ description: "Choose 'short' for basic results (no Description) or 'detailed' for full results (includes Description).",
27
+ enum: ['short', 'detailed'],
28
+ default: 'short'
29
29
  }
30
30
  },
31
31
  required: ['query']
@@ -38,22 +38,17 @@ export const searchToolDefinition = {
38
38
  * @returns {Promise<Object>} - The tool result
39
39
  */
40
40
  export async function searchToolHandler(params) {
41
- const { query, page = 1, numResults = 10 } = params;
42
- console.log(`Searching for: ${query} (page ${page}, ${numResults} results)`);
43
-
44
- const results = await searchDuckDuckGo(query, page, numResults);
41
+ const { query, numResults = 3, mode = 'short' } = params;
42
+ console.log(`Searching for: ${query} (${numResults} results, mode: ${mode})`);
43
+
44
+ const results = await searchDuckDuckGo(query, numResults, mode);
45
45
  console.log(`Found ${results.length} results`);
46
-
47
- // Format the results for display
48
- const formattedResults = results.map((result, index) =>
49
- `${index + 1}. [${result.title}](${result.url})\n ${result.snippet}`
50
- ).join('\n\n');
51
-
46
+
52
47
  return {
53
48
  content: [
54
49
  {
55
50
  type: 'text',
56
- text: formattedResults || 'No results found.'
51
+ text: JSON.stringify(results)
57
52
  }
58
53
  ]
59
54
  };
@@ -1,20 +1,11 @@
1
1
  import axios from 'axios';
2
2
  import * as cheerio from 'cheerio';
3
3
  import https from 'https';
4
+ import { getRandomUserAgent } from './user_agents.js';
4
5
 
5
6
  // Constants
6
- const RESULTS_PER_PAGE = 10;
7
7
  const MAX_CACHE_PAGES = 5;
8
8
 
9
- // Rotating User Agents
10
- const USER_AGENTS = [
11
- 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
12
- 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) Edge/120.0.0.0',
13
- 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2.1 Safari/605.1.15',
14
- 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:122.0) Gecko/20100101 Firefox/122.0',
15
- 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'
16
- ];
17
-
18
9
  // Cache results to avoid repeated requests
19
10
  const resultsCache = new Map();
20
11
  const CACHE_DURATION = 5 * 60 * 1000; // 5 minutes
@@ -29,21 +20,12 @@ const httpsAgent = new https.Agent({
29
20
  });
30
21
 
31
22
  /**
32
- * Get a random user agent from the list
33
- * @returns {string} A random user agent string
34
- */
35
- function getRandomUserAgent() {
36
- return USER_AGENTS[Math.floor(Math.random() * USER_AGENTS.length)];
37
- }
38
-
39
- /**
40
- * Generate a cache key for a search query and page
23
+ * Generate a cache key for a search query
41
24
  * @param {string} query - The search query
42
- * @param {number} page - The page number
43
25
  * @returns {string} The cache key
44
26
  */
45
- function getCacheKey(query, page) {
46
- return `${query}-${page}`;
27
+ function getCacheKey(query) {
28
+ return `${query}`;
47
29
  }
48
30
 
49
31
  /**
@@ -125,23 +107,20 @@ function getFaviconUrl(url) {
125
107
  }
126
108
  }
127
109
 
110
+
128
111
  /**
129
112
  * Scrapes search results from DuckDuckGo HTML
130
113
  * @param {string} query - The search query
131
- * @param {number} page - The page number (default: 1)
132
114
  * @param {number} numResults - Number of results to return (default: 10)
133
115
  * @returns {Promise<Array>} - Array of search results
134
116
  */
135
- async function searchDuckDuckGo(query, page = 1, numResults = 10) {
117
+ async function searchDuckDuckGo(query, numResults = 10, mode = 'short') {
136
118
  try {
137
119
  // Clear old cache entries
138
120
  clearOldCache();
139
121
 
140
- // Calculate start index for pagination
141
- const startIndex = (page - 1) * RESULTS_PER_PAGE;
142
-
143
122
  // Check cache first
144
- const cacheKey = getCacheKey(query, page);
123
+ const cacheKey = getCacheKey(query);
145
124
  const cachedResults = resultsCache.get(cacheKey);
146
125
 
147
126
  if (cachedResults && Date.now() - cachedResults.timestamp < CACHE_DURATION) {
@@ -153,7 +132,7 @@ async function searchDuckDuckGo(query, page = 1, numResults = 10) {
153
132
 
154
133
  // Fetch results
155
134
  const response = await axios.get(
156
- `https://duckduckgo.com/html/?q=${encodeURIComponent(query)}&s=${startIndex}`,
135
+ `https://duckduckgo.com/html/?q=${encodeURIComponent(query)}`,
157
136
  {
158
137
  headers: {
159
138
  'User-Agent': userAgent
@@ -172,6 +151,7 @@ async function searchDuckDuckGo(query, page = 1, numResults = 10) {
172
151
  const $ = cheerio.load(html);
173
152
 
174
153
  const results = [];
154
+ const jinaFetchPromises = [];
175
155
  $('.result').each((i, result) => {
176
156
  const $result = $(result);
177
157
  const titleEl = $result.find('.result__title a');
@@ -185,24 +165,69 @@ async function searchDuckDuckGo(query, page = 1, numResults = 10) {
185
165
 
186
166
  const directLink = extractDirectUrl(rawLink || '');
187
167
  const favicon = getFaviconUrl(directLink);
168
+ const jinaUrl = getJinaAiUrl(directLink);
188
169
 
189
170
  if (title && directLink) {
190
- results.push({
191
- title,
192
- url: directLink,
193
- snippet: description || '',
194
- favicon: favicon,
195
- displayUrl: displayUrl || ''
196
- });
171
+ if (mode === 'detailed') {
172
+ jinaFetchPromises.push(
173
+ axios.get(jinaUrl, {
174
+ headers: {
175
+ 'User-Agent': getRandomUserAgent()
176
+ },
177
+ httpsAgent: httpsAgent,
178
+ timeout: 10000
179
+ })
180
+ .then(jinaRes => {
181
+ let jinaContent = '';
182
+ if (jinaRes.status === 200 && typeof jinaRes.data === 'string') {
183
+ const $jina = cheerio.load(jinaRes.data);
184
+ jinaContent = $jina('body').text()
185
+ }
186
+ return {
187
+ title,
188
+ url: directLink,
189
+ snippet: description || '',
190
+ favicon: favicon,
191
+ displayUrl: displayUrl || '',
192
+ Description: jinaContent
193
+ };
194
+ })
195
+ .catch(() => {
196
+ return {
197
+ title,
198
+ url: directLink,
199
+ snippet: description || '',
200
+ favicon: favicon,
201
+ displayUrl: displayUrl || '',
202
+ Description: ''
203
+ };
204
+ })
205
+ );
206
+ } else {
207
+ // short mode: omit Description
208
+ jinaFetchPromises.push(
209
+ Promise.resolve({
210
+ title,
211
+ url: directLink,
212
+ snippet: description || '',
213
+ favicon: favicon,
214
+ displayUrl: displayUrl || ''
215
+ })
216
+ );
217
+ }
197
218
  }
198
219
  });
199
220
 
200
- // Get paginated results
201
- const paginatedResults = results.slice(0, numResults);
221
+ // Wait for all Jina AI fetches to complete
222
+ const jinaResults = await Promise.all(jinaFetchPromises);
223
+ results.push(...jinaResults);
224
+
225
+ // Get limited results
226
+ const limitedResults = results.slice(0, numResults);
202
227
 
203
228
  // Cache the results
204
229
  resultsCache.set(cacheKey, {
205
- results: paginatedResults,
230
+ results: limitedResults,
206
231
  timestamp: Date.now()
207
232
  });
208
233
 
@@ -212,7 +237,7 @@ async function searchDuckDuckGo(query, page = 1, numResults = 10) {
212
237
  resultsCache.delete(oldestKey);
213
238
  }
214
239
 
215
- return paginatedResults;
240
+ return limitedResults;
216
241
  } catch (error) {
217
242
  console.error('Error searching DuckDuckGo:', error.message);
218
243
  throw error;
@@ -225,3 +250,19 @@ export {
225
250
  extractDirectUrl,
226
251
  getFaviconUrl
227
252
  };
253
+
254
+ /**
255
+ * Generate a Jina AI URL for a given website URL
256
+ * @param {string} url - The website URL
257
+ * @returns {string} The Jina AI URL
258
+ */
259
+ function getJinaAiUrl(url) {
260
+ try {
261
+ const urlObj = new URL(url);
262
+ return `https://r.jina.ai/${urlObj.href}`;
263
+ } catch {
264
+ return '';
265
+ }
266
+ }
267
+
268
+ export { getJinaAiUrl };
@@ -4,6 +4,7 @@ import * as cheerio from 'cheerio';
4
4
  import TurndownService from 'turndown';
5
5
  import * as tough from 'tough-cookie';
6
6
  import { wrapper } from 'axios-cookiejar-support';
7
+ import { getRandomUserAgent } from './user_agents.js';
7
8
 
8
9
  const { CookieJar } = tough;
9
10
 
@@ -109,13 +110,11 @@ function formatHtml(htmlContent) {
109
110
  /**
110
111
  * Search using IAsk AI via WebSocket (Phoenix LiveView)
111
112
  * @param {string} prompt - The search query or prompt
112
- * @param {boolean} stream - If true, returns async generator for streaming
113
- * @param {boolean} raw - If true, returns raw response (not used currently)
114
113
  * @param {string} mode - Search mode: 'question', 'academic', 'forums', 'wiki', 'thinking'
115
114
  * @param {string|null} detailLevel - Detail level: 'concise', 'detailed', 'comprehensive'
116
- * @returns {Promise<string|AsyncGenerator<string>>} The search results
115
+ * @returns {Promise<string>} The search results
117
116
  */
118
- async function searchIAsk(prompt, stream = false, raw = false, mode = 'question', detailLevel = null) {
117
+ async function searchIAsk(prompt, mode = 'thinking', detailLevel = null) {
119
118
  // Validate mode
120
119
  if (!VALID_MODES.includes(mode)) {
121
120
  throw new Error(`Invalid mode: ${mode}. Valid modes are: ${VALID_MODES.join(', ')}`);
@@ -133,11 +132,7 @@ async function searchIAsk(prompt, stream = false, raw = false, mode = 'question'
133
132
  const cachedResults = resultsCache.get(cacheKey);
134
133
 
135
134
  if (cachedResults && Date.now() - cachedResults.timestamp < CACHE_DURATION) {
136
- const result = cachedResults.results;
137
- if (stream) {
138
- return (async function*() { yield result; })();
139
- }
140
- return result;
135
+ return cachedResults.results;
141
136
  }
142
137
 
143
138
  // Build URL parameters
@@ -155,7 +150,7 @@ async function searchIAsk(prompt, stream = false, raw = false, mode = 'question'
155
150
  params: Object.fromEntries(params),
156
151
  timeout: DEFAULT_TIMEOUT,
157
152
  headers: {
158
- 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'
153
+ 'User-Agent': getRandomUserAgent()
159
154
  }
160
155
  });
161
156
 
@@ -188,13 +183,12 @@ async function searchIAsk(prompt, stream = false, raw = false, mode = 'question'
188
183
  const ws = new WebSocket(wsUrl, {
189
184
  headers: {
190
185
  'Cookie': cookieString,
191
- 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
186
+ 'User-Agent': getRandomUserAgent(),
192
187
  'Origin': 'https://iask.ai'
193
188
  }
194
189
  });
195
190
 
196
191
  let buffer = '';
197
- const chunks = [];
198
192
  let timeoutId;
199
193
 
200
194
  ws.on('open', () => {
@@ -237,7 +231,6 @@ async function searchIAsk(prompt, stream = false, raw = false, mode = 'question'
237
231
  }
238
232
 
239
233
  buffer += formatted;
240
- chunks.push(formatted);
241
234
  }
242
235
  } else {
243
236
  throw new Error('No diff.e');
@@ -253,7 +246,6 @@ async function searchIAsk(prompt, stream = false, raw = false, mode = 'question'
253
246
  formatted = cache;
254
247
  }
255
248
  buffer += formatted;
256
- chunks.push(formatted);
257
249
  // Close after cache find
258
250
  ws.close();
259
251
  return;
@@ -276,15 +268,7 @@ async function searchIAsk(prompt, stream = false, raw = false, mode = 'question'
276
268
  });
277
269
  }
278
270
 
279
- if (stream) {
280
- resolve((async function*() {
281
- for (const chunk of chunks) {
282
- yield chunk;
283
- }
284
- })());
285
- } else {
286
- resolve(buffer || 'No results found.');
287
- }
271
+ resolve(buffer || 'No results found.');
288
272
  });
289
273
 
290
274
  ws.on('error', (err) => {
@@ -0,0 +1,131 @@
1
+ import axios from 'axios';
2
+ import { randomUUID } from 'crypto';
3
+ import { getRandomUserAgent } from './user_agents.js';
4
+
5
+ class MonicaClient {
6
+ constructor(timeout = 60000) {
7
+ this.apiEndpoint = "https://monica.so/api/search_v1/search";
8
+ this.timeout = timeout;
9
+ this.clientId = randomUUID();
10
+ this.sessionId = "";
11
+
12
+ this.headers = {
13
+ "accept": "*/*",
14
+ "accept-encoding": "gzip, deflate, br, zstd",
15
+ "accept-language": "en-US,en;q=0.9",
16
+ "content-type": "application/json",
17
+ "dnt": "1",
18
+ "origin": "https://monica.so",
19
+ "referer": "https://monica.so/answers",
20
+ "sec-ch-ua": '"Microsoft Edge";v="135", "Not-A.Brand";v="8", "Chromium";v="135"',
21
+ "sec-ch-ua-mobile": "?0",
22
+ "sec-ch-ua-platform": '"Windows"',
23
+ "sec-fetch-dest": "empty",
24
+ "sec-fetch-mode": "cors",
25
+ "sec-fetch-site": "same-origin",
26
+ "sec-gpc": "1",
27
+ "user-agent": getRandomUserAgent(),
28
+ "x-client-id": this.clientId,
29
+ "x-client-locale": "en",
30
+ "x-client-type": "web",
31
+ "x-client-version": "5.4.3",
32
+ "x-from-channel": "NA",
33
+ "x-product-name": "Monica-Search",
34
+ "x-time-zone": "Asia/Calcutta;-330"
35
+ };
36
+
37
+ // Axios instance
38
+ this.client = axios.create({
39
+ headers: this.headers,
40
+ timeout: this.timeout,
41
+ withCredentials: true
42
+ });
43
+ }
44
+
45
+ formatResponse(text) {
46
+ // Clean up markdown formatting
47
+ let cleanedText = text.replace(/\*\*/g, '');
48
+
49
+ // Remove any empty lines
50
+ cleanedText = cleanedText.replace(/\n\s*\n/g, '\n\n');
51
+
52
+ // Remove any trailing whitespace
53
+ return cleanedText.trim();
54
+ }
55
+
56
+ async search(prompt) {
57
+ const taskId = randomUUID();
58
+ const payload = {
59
+ "pro": false,
60
+ "query": prompt,
61
+ "round": 1,
62
+ "session_id": this.sessionId,
63
+ "language": "auto",
64
+ "task_id": taskId
65
+ };
66
+
67
+ const cookies = {
68
+ "monica_home_theme": "auto"
69
+ };
70
+
71
+ // Convert cookies object to string
72
+ const cookieString = Object.entries(cookies).map(([k, v]) => `${k}=${v}`).join('; ');
73
+
74
+ try {
75
+ const response = await this.client.post(this.apiEndpoint, payload, {
76
+ headers: {
77
+ ...this.headers,
78
+ 'Cookie': cookieString
79
+ },
80
+ responseType: 'stream'
81
+ });
82
+
83
+ let fullText = '';
84
+
85
+ return new Promise((resolve, reject) => {
86
+ response.data.on('data', (chunk) => {
87
+ const lines = chunk.toString().split('\n');
88
+ for (const line of lines) {
89
+ if (line.startsWith('data: ')) {
90
+ try {
91
+ const jsonStr = line.substring(6);
92
+ const data = JSON.parse(jsonStr);
93
+
94
+ if (data.session_id) {
95
+ this.sessionId = data.session_id;
96
+ }
97
+
98
+ if (data.text) {
99
+ fullText += data.text;
100
+ }
101
+ } catch (e) {
102
+ // Ignore parse errors
103
+ }
104
+ }
105
+ }
106
+ });
107
+
108
+ response.data.on('end', () => {
109
+ resolve(this.formatResponse(fullText));
110
+ });
111
+
112
+ response.data.on('error', (err) => {
113
+ reject(err);
114
+ });
115
+ });
116
+
117
+ } catch (error) {
118
+ throw new Error(`Monica API request failed: ${error.message}`);
119
+ }
120
+ }
121
+ }
122
+
123
+ /**
124
+ * Search using Monica AI
125
+ * @param {string} query - The search query
126
+ * @returns {Promise<string>} The search results
127
+ */
128
+ export async function searchMonica(query) {
129
+ const client = new MonicaClient();
130
+ return await client.search(query);
131
+ }
@@ -0,0 +1,27 @@
1
+ /**
2
+ * List of user agents for rotation
3
+ */
4
+ const USER_AGENTS = [
5
+ 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
6
+ 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) Edge/120.0.0.0',
7
+ 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2.1 Safari/605.1.15',
8
+ 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:122.0) Gecko/20100101 Firefox/122.0',
9
+ 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
10
+ 'Mozilla/5.0 (iPhone; CPU iPhone OS 17_2_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1',
11
+ 'Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Mobile Safari/537.36',
12
+ 'Mozilla/5.0 (iPad; CPU OS 17_2_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/120.0.6099.119 Mobile/15E148 Safari/604.1',
13
+ 'Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko',
14
+ 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
15
+ 'Mozilla/5.0 (Linux; Android 13; SM-G998B) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Mobile Safari/537.36',
16
+ 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 OPR/105.0.0.0',
17
+ 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Vivaldi/6.4.3160.42',
18
+ 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/119.0',
19
+ ];
20
+
21
+ /**
22
+ * Get a random user agent from the list
23
+ * @returns {string} A random user agent string
24
+ */
25
+ export function getRandomUserAgent() {
26
+ return USER_AGENTS[Math.floor(Math.random() * USER_AGENTS.length)];
27
+ }