ai-web-search 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 AI Web Search
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,382 @@
1
+ # AI Web Search
2
+
3
+ 🔍 为AI助手提供联网搜索能力的轻量级工具
4
+
5
+ ## 功能特点
6
+
7
+ - **开箱即用**:从GitHub克隆后 `npm install` 即可使用,无需复杂配置
8
+ - **免费搜索**:使用 DuckDuckGo 搜索引擎,无需API密钥
9
+ - **智能回退**:DuckDuckGo不可用时自动切换到百度搜索,确保可用性
10
+ - **兼容性强**:提供标准HTTP API,可与任何AI软件集成
11
+ - **轻量快速**:启动速度快,资源占用少
12
+ - **隐私保护**:不记录搜索历史,保护用户隐私
13
+
14
+ ## 安装
15
+
16
+ ### 方式1:从GitHub安装(推荐)
17
+
18
+ ```bash
19
+ # 克隆仓库
20
+ git clone https://github.com/your-username/ai-web-search.git
21
+
22
+ # 进入项目目录
23
+ cd ai-web-search
24
+
25
+ # 安装依赖
26
+ npm install
27
+
28
+ # 创建全局命令(可选)
29
+ npm link
30
+ ```
31
+
32
+ ### 方式2:直接从GitHub安装
33
+
34
+ ```bash
35
+ # 直接从GitHub安装
36
+ npm install git+https://github.com/your-username/ai-web-search.git
37
+
38
+ # 或者指定分支
39
+ npm install git+https://github.com/your-username/ai-web-search.git#main
40
+ ```
41
+
42
+ ### 方式3:下载ZIP文件
43
+
44
+ 1. 从GitHub下载ZIP文件
45
+ 2. 解压到本地目录
46
+ 3. 进入目录运行 `npm install`
47
+
48
+ ## 快速开始
49
+
50
+ ### 1. 启动服务器
51
+
52
+ ```bash
53
+ # 如果已创建全局命令
54
+ ai-web-search
55
+
56
+ # 或者直接运行
57
+ node src/index.js
58
+
59
+ # 或者使用npm脚本
60
+ npm start
61
+ ```
62
+
63
+ 服务器默认运行在 `http://localhost:3000`
64
+
65
+ ### 2. 测试搜索
66
+
67
+ ```bash
68
+ # 使用curl测试
69
+ curl "http://localhost:3000/search?query=人工智能最新发展"
70
+
71
+ # 或者在浏览器中访问
72
+ http://localhost:3000/search?query=人工智能最新发展
73
+ ```
74
+
75
+ ### 3. 集成到AI软件
76
+
77
+ 在你的AI软件中添加以下工具配置:
78
+
79
+ ```json
80
+ {
81
+ "name": "web_search",
82
+ "endpoint": "http://localhost:3000/search",
83
+ "method": "GET",
84
+ "parameters": {
85
+ "query": "string",
86
+ "limit": "integer",
87
+ "type": "string"
88
+ }
89
+ }
90
+ ```
91
+
92
+ ## 配置选项
93
+
94
+ ### 命令行参数
95
+
96
+ ```bash
97
+ ai-web-search [选项]
98
+
99
+ 选项:
100
+ -p, --port <端口> 服务器端口(默认:3000)
101
+ -H, --host <主机> 服务器主机(默认:localhost)
102
+ -d, --dev 开发模式运行
103
+ -s, --search-function <路径> 自定义搜索引擎模块路径
104
+ -h, --help 显示帮助信息
105
+ -v, --version 显示版本号
106
+ ```
107
+
108
+ ### 环境变量
109
+
110
+ 创建 `.env` 文件:
111
+
112
+ ```env
113
+ PORT=3000
114
+ HOST=localhost
115
+ RATE_LIMIT_MAX=100
116
+ SEARCH_FUNCTION_PATH=./examples/custom-search.js
117
+ ```
118
+
119
+ ### 自定义搜索引擎
120
+
121
+ 如果默认的DuckDuckGo搜索引擎无法使用(如网络限制),你可以提供自定义搜索引擎实现。
122
+
123
+ #### 创建自定义搜索引擎模块
124
+
125
+ ```javascript
126
+ // custom-search.js
127
+ async function customSearch(query, options = {}) {
128
+ const { limit = 5, timeRange = 'month', type = 'web' } = options;
129
+
130
+ // 实现你的搜索逻辑
131
+ // 可以是百度、搜狗、Bing等搜索引擎的API调用
132
+ // 或者爬虫实现
133
+
134
+ return {
135
+ results: [
136
+ {
137
+ title: '搜索结果标题',
138
+ url: 'https://example.com',
139
+ description: '搜索结果描述',
140
+ source: 'custom'
141
+ }
142
+ ]
143
+ };
144
+ }
145
+
146
+ module.exports = customSearch;
147
+ ```
148
+
149
+ #### 使用自定义搜索引擎
150
+
151
+ ```bash
152
+ # 命令行参数
153
+ ai-web-search -s ./custom-search.js
154
+
155
+ # 环境变量
156
+ SEARCH_FUNCTION_PATH=./custom-search.js ai-web-search
157
+
158
+ # .env 文件
159
+ SEARCH_FUNCTION_PATH=./custom-search.js
160
+ ```
161
+
162
+ 详细的自定义搜索引擎示例请参考 `examples/custom-search.js` 文件。
163
+
164
+ ## API 文档
165
+
166
+ ### 搜索接口
167
+
168
+ ```
169
+ GET /search
170
+ ```
171
+
172
+ **参数:**
173
+
174
+ | 参数 | 类型 | 必需 | 默认值 | 描述 |
175
+ |------|------|------|--------|------|
176
+ | query | string | 是 | - | 搜索查询关键词 |
177
+ | limit | integer | 否 | 5 | 返回结果数量(1-20) |
178
+ | type | string | 否 | web | 搜索类型:web(网页)或 news(新闻) |
179
+
180
+ **响应示例:**
181
+
182
+ ```json
183
+ {
184
+ "query": "人工智能",
185
+ "results": [
186
+ {
187
+ "title": "人工智能发展现状",
188
+ "url": "https://example.com/ai-development",
189
+ "snippet": "人工智能正在快速发展...",
190
+ "source": "duckduckgo"
191
+ }
192
+ ],
193
+ "totalResults": 10,
194
+ "timestamp": "2026-06-08T10:30:00.000Z"
195
+ }
196
+ ```
197
+
198
+ ### 工具定义接口
199
+
200
+ ```
201
+ GET /tool-definition
202
+ ```
203
+
204
+ 返回兼容 OpenAI 工具格式的 JSON 定义:
205
+
206
+ ```json
207
+ {
208
+ "type": "function",
209
+ "function": {
210
+ "name": "web_search",
211
+ "description": "Search the internet to get latest information",
212
+ "parameters": {
213
+ "type": "object",
214
+ "properties": {
215
+ "query": {
216
+ "type": "string",
217
+ "description": "The search query to find information"
218
+ },
219
+ "limit": {
220
+ "type": "integer",
221
+ "description": "Number of results to return (default: 5, max: 20)",
222
+ "default": 5
223
+ },
224
+ "type": {
225
+ "type": "string",
226
+ "enum": ["web", "news"],
227
+ "description": "Type of search",
228
+ "default": "web"
229
+ }
230
+ },
231
+ "required": ["query"]
232
+ }
233
+ }
234
+ }
235
+ ```
236
+
237
+ ### 搜索建议接口
238
+
239
+ ```
240
+ GET /suggest?query=部分关键词
241
+ ```
242
+
243
+ ### 健康检查接口
244
+
245
+ ```
246
+ GET /health
247
+ ```
248
+
249
+ ## 使用示例
250
+
251
+ ### Python AI 助手集成
252
+
253
+ ```python
254
+ import requests
255
+
256
+ def web_search(query, limit=5):
257
+ response = requests.get(
258
+ "http://localhost:3000/search",
259
+ params={"query": query, "limit": limit}
260
+ )
261
+ return response.json()
262
+
263
+ # 使用示例
264
+ results = web_search("最新的AI技术")
265
+ print(results)
266
+ ```
267
+
268
+ ### JavaScript AI 助手集成
269
+
270
+ ```javascript
271
+ async function webSearch(query, limit = 5) {
272
+ const response = await fetch(
273
+ `http://localhost:3000/search?query=${encodeURIComponent(query)}&limit=${limit}`
274
+ );
275
+ return await response.json();
276
+ }
277
+
278
+ // 使用示例
279
+ const results = await webSearch("机器学习最新进展");
280
+ console.log(results);
281
+ ```
282
+
283
+ ## 故障排除
284
+
285
+ ### 1. 端口被占用
286
+
287
+ ```bash
288
+ # 使用其他端口
289
+ ai-web-search -p 8080
290
+ ```
291
+
292
+ ### 2. 搜索无结果
293
+
294
+ - 检查网络连接
295
+ - 尝试不同的搜索关键词
296
+ - 查看服务器日志
297
+
298
+ ### 3. 连接被拒绝
299
+
300
+ - 确认服务器正在运行
301
+ - 检查防火墙设置
302
+ - 验证端口是否正确
303
+
304
+ ### 4. Git克隆失败
305
+
306
+ ```bash
307
+ # 如果git clone失败,检查网络连接
308
+ git config --global http.proxy # 设置代理(如果需要)
309
+
310
+ # 或者使用SSH方式(需要配置SSH密钥)
311
+ git clone git@github.com:your-username/ai-web-search.git
312
+
313
+ # 如果证书问题,可以临时跳过SSL验证(不推荐)
314
+ git -c http.sslVerify=false clone https://github.com/your-username/ai-web-search.git
315
+ ```
316
+
317
+ ### 5. npm install失败
318
+
319
+ ```bash
320
+ # 清除npm缓存
321
+ npm cache clean --force
322
+
323
+ # 使用淘宝镜像(国内加速)
324
+ npm install --registry=https://registry.npmmirror.com
325
+
326
+ # 检查Node.js版本
327
+ node --version # 需要 >= 14.0.0
328
+ ```
329
+
330
+ ## 开发
331
+
332
+ ### 本地开发
333
+
334
+ ```bash
335
+ # 克隆项目
336
+ git clone https://github.com/your-username/ai-web-search.git
337
+ cd ai-web-search
338
+
339
+ # 安装依赖
340
+ npm install
341
+
342
+ # 开发模式运行
343
+ npm run dev
344
+
345
+ # 运行测试
346
+ npm test
347
+ ```
348
+
349
+ ### 项目结构
350
+
351
+ ```
352
+ ai-web-search/
353
+ ├── bin/
354
+ │ └── cli.js # 命令行入口
355
+ ├── src/
356
+ │ ├── index.js # 主入口
357
+ │ ├── server.js # HTTP服务器
358
+ │ └── search.js # 搜索引擎模块
359
+ ├── examples/
360
+ │ ├── custom-search.js # 自定义搜索引擎示例
361
+ │ └── baidu-search.js # 百度搜索引擎实现
362
+ ├── test/
363
+ │ ├── test.js # 测试文件
364
+ │ └── test-custom.js # 自定义搜索测试
365
+ ├── package.json
366
+ ├── README.md
367
+ ├── .gitignore
368
+ └── .env.example
369
+ ```
370
+
371
+ ## 许可证
372
+
373
+ MIT License
374
+
375
+ ## 贡献
376
+
377
+ 欢迎提交 Issue 和 Pull Request!
378
+
379
+ ## 相关项目
380
+
381
+ - [duck-duck-scrape](https://github.com/Snazzah/duck-duck-scrape) - DuckDuckGo 搜索库
382
+ - [Express](https://expressjs.com/) - Web 框架
package/bin/cli.js ADDED
@@ -0,0 +1,115 @@
1
+ #!/usr/bin/env node
2
+
3
+ const { createServer } = require('../src/server');
4
+ const dotenv = require('dotenv');
5
+
6
+ // Load environment variables
7
+ dotenv.config();
8
+
9
+ // Parse command line arguments
10
+ const args = process.argv.slice(2);
11
+ const options = {};
12
+
13
+ for (let i = 0; i < args.length; i++) {
14
+ switch (args[i]) {
15
+ case '-p':
16
+ case '--port':
17
+ options.port = parseInt(args[++i]) || 3000;
18
+ break;
19
+ case '-H':
20
+ case '--host':
21
+ options.host = args[++i] || 'localhost';
22
+ break;
23
+ case '-d':
24
+ case '--dev':
25
+ options.dev = true;
26
+ break;
27
+ case '-s':
28
+ case '--search-function':
29
+ options.searchFn = args[++i];
30
+ break;
31
+ case '-h':
32
+ case '--help':
33
+ showHelp();
34
+ process.exit(0);
35
+ break;
36
+ case '-v':
37
+ case '--version':
38
+ showVersion();
39
+ process.exit(0);
40
+ break;
41
+ }
42
+ }
43
+
44
+ // Configuration
45
+ const config = {
46
+ port: options.port || process.env.PORT || 3000,
47
+ host: options.host || process.env.HOST || 'localhost',
48
+ dev: options.dev || process.argv.includes('--dev'),
49
+ searchFnPath: options.searchFn || process.env.SEARCH_FUNCTION_PATH || null,
50
+ rateLimit: {
51
+ windowMs: 15 * 60 * 1000,
52
+ max: parseInt(process.env.RATE_LIMIT_MAX) || 100
53
+ }
54
+ };
55
+
56
+ function showHelp() {
57
+ console.log(`
58
+ 🔍 AI Web Search - Enable AI assistants to search the internet
59
+
60
+ Usage:
61
+ ai-web-search [options]
62
+
63
+ Options:
64
+ -p, --port <port> Port to run server on (default: 3000)
65
+ -H, --host <host> Host to bind server to (default: localhost)
66
+ -d, --dev Run in development mode
67
+ -s, --search-function <path> Path to custom search function module
68
+ -h, --help Show this help message
69
+ -v, --version Show version number
70
+
71
+ Environment Variables:
72
+ PORT Server port (default: 3000)
73
+ HOST Server host (default: localhost)
74
+ RATE_LIMIT_MAX Max requests per 15 minutes (default: 100)
75
+ SEARCH_FUNCTION_PATH Path to custom search function module
76
+
77
+ Examples:
78
+ ai-web-search # Start with default settings
79
+ ai-web-search -p 8080 # Start on port 8080
80
+ ai-web-search -H 0.0.0.0 # Bind to all interfaces
81
+ ai-web-search --dev # Start in development mode
82
+ `);
83
+ }
84
+
85
+ function showVersion() {
86
+ const pkg = require('../package.json');
87
+ console.log(pkg.version);
88
+ }
89
+
90
+ // Start server
91
+ const server = createServer(config);
92
+
93
+ server.listen(config.port, config.host, () => {
94
+ console.log(`🔍 AI Web Search server running at http://${config.host}:${config.port}`);
95
+ console.log(`📡 Search endpoint: http://${config.host}:${config.port}/search`);
96
+ console.log(`🔧 Environment: ${config.dev ? 'development' : 'production'}`);
97
+ console.log(`⚡ Press Ctrl+C to stop`);
98
+ });
99
+
100
+ // Handle graceful shutdown
101
+ process.on('SIGINT', () => {
102
+ console.log('\n🛑 Shutting down server...');
103
+ server.close(() => {
104
+ console.log('✅ Server stopped');
105
+ process.exit(0);
106
+ });
107
+ });
108
+
109
+ process.on('SIGTERM', () => {
110
+ console.log('\n🛑 Received SIGTERM, shutting down...');
111
+ server.close(() => {
112
+ console.log('✅ Server stopped');
113
+ process.exit(0);
114
+ });
115
+ });
@@ -0,0 +1,116 @@
1
+ /**
2
+ * Baidu Search Engine for ai-web-search
3
+ *
4
+ * 注意:此实现为爬虫方式,可能违反百度服务条款。
5
+ * 仅用于学习和测试,请勿用于商业用途。
6
+ *
7
+ * @param {string} query - Search query
8
+ * @param {Object} options - Search options
9
+ * @returns {Promise<Object>} Search results
10
+ */
11
+ const axios = require('axios');
12
+ const cheerio = require('cheerio');
13
+
14
+ async function baiduSearch(query, options = {}) {
15
+ const { limit = 5, type = 'web' } = options;
16
+
17
+ console.log(`🔍 Baidu search: "${query}" (limit: ${limit}, type: ${type})`);
18
+
19
+ try {
20
+ // 设置请求头,模拟浏览器
21
+ const headers = {
22
+ 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
23
+ 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
24
+ 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8',
25
+ 'Accept-Encoding': 'gzip, deflate',
26
+ 'Connection': 'keep-alive',
27
+ 'Upgrade-Insecure-Requests': '1'
28
+ };
29
+
30
+ // 请求百度搜索
31
+ const response = await axios.get('https://www.baidu.com/s', {
32
+ params: {
33
+ wd: query,
34
+ rn: limit,
35
+ ie: 'utf-8'
36
+ },
37
+ headers,
38
+ timeout: 10000
39
+ });
40
+
41
+ // 解析HTML
42
+ const $ = cheerio.load(response.data);
43
+ const results = [];
44
+
45
+ // 百度搜索结果通常在 class 为 'result' 或 'c-container' 的div中
46
+ $('.result, .c-container').each((i, element) => {
47
+ if (results.length >= limit) return false;
48
+
49
+ const titleElement = $(element).find('h3').first();
50
+ const title = titleElement.text().trim();
51
+ const url = titleElement.find('a').attr('href');
52
+
53
+ // 获取摘要
54
+ const abstractElement = $(element).find('.c-abstract, .content-right_8Zs40');
55
+ const snippet = abstractElement.text().trim();
56
+
57
+ // 有时候百度会用不同的结构
58
+ const alternativeSnippet = $(element).find('.c-span-last').text().trim();
59
+
60
+ if (title && url) {
61
+ results.push({
62
+ title: title.replace(/\s+/g, ' '),
63
+ url: url,
64
+ snippet: snippet || alternativeSnippet || '暂无摘要',
65
+ source: 'baidu'
66
+ });
67
+ }
68
+ });
69
+
70
+ // 如果没有找到结果,尝试其他选择器
71
+ if (results.length === 0) {
72
+ // 尝试更通用的选择器
73
+ $('div[id^="content_left"] > div').each((i, element) => {
74
+ if (results.length >= limit) return false;
75
+
76
+ const titleElement = $(element).find('h3').first();
77
+ const title = titleElement.text().trim();
78
+ const url = titleElement.find('a').attr('href');
79
+
80
+ const snippet = $(element).find('span.content-right_8Zs40, .c-abstract').text().trim();
81
+
82
+ if (title && url && title !== '') {
83
+ results.push({
84
+ title: title.replace(/\s+/g, ' '),
85
+ url: url,
86
+ snippet: snippet || '暂无摘要',
87
+ source: 'baidu'
88
+ });
89
+ }
90
+ });
91
+ }
92
+
93
+ console.log(` Found ${results.length} results from Baidu`);
94
+
95
+ return {
96
+ results: results.slice(0, limit)
97
+ };
98
+
99
+ } catch (error) {
100
+ console.error('Baidu search error:', error.message);
101
+
102
+ // 返回一个友好的错误信息,而不是抛出异常
103
+ return {
104
+ results: [
105
+ {
106
+ title: '百度搜索暂时不可用',
107
+ url: 'https://www.baidu.com',
108
+ snippet: `搜索出错: ${error.message}。请稍后重试或检查网络连接。`,
109
+ source: 'baidu-error'
110
+ }
111
+ ]
112
+ };
113
+ }
114
+ }
115
+
116
+ module.exports = baiduSearch;
@@ -0,0 +1,64 @@
1
+ /**
2
+ * Example custom search function for ai-web-search
3
+ *
4
+ * This function can be used as a template for implementing custom search engines.
5
+ * The function receives a query and options, and returns search results in the standard format.
6
+ *
7
+ * Usage:
8
+ * ai-web-search -s ./examples/custom-search.js
9
+ * or
10
+ * SEARCH_FUNCTION_PATH=./examples/custom-search.js ai-web-search
11
+ *
12
+ * @param {string} query - Search query
13
+ * @param {Object} options - Search options
14
+ * @param {number} options.limit - Number of results to return
15
+ * @param {string} options.timeRange - Time range for search
16
+ * @param {string} options.type - Search type: 'web' or 'news'
17
+ * @returns {Promise<Object>} Search results
18
+ */
19
+ async function customSearch(query, options = {}) {
20
+ const { limit = 5, timeRange = 'month', type = 'web' } = options;
21
+
22
+ console.log(`🔍 Custom search: "${query}" (limit: ${limit}, type: ${type})`);
23
+
24
+ // Example: Using a mock search engine
25
+ // In real implementation, you would call your custom search API here
26
+
27
+ // Simulate API delay
28
+ await new Promise(resolve => setTimeout(resolve, 100));
29
+
30
+ // Return mock results
31
+ const results = [
32
+ {
33
+ title: `Result 1 for "${query}"`,
34
+ url: `https://example.com/result1?q=${encodeURIComponent(query)}`,
35
+ description: `This is the first search result for "${query}". It contains relevant information about your query.`,
36
+ source: 'custom-search'
37
+ },
38
+ {
39
+ title: `Result 2 for "${query}"`,
40
+ url: `https://example.com/result2?q=${encodeURIComponent(query)}`,
41
+ description: `Another search result for "${query}". This provides additional information and perspectives.`,
42
+ source: 'custom-search'
43
+ },
44
+ {
45
+ title: `Result 3 for "${query}"`,
46
+ url: `https://example.com/result3?q=${encodeURIComponent(query)}`,
47
+ description: `Third result for "${query}". More details and related information.`,
48
+ source: 'custom-search'
49
+ }
50
+ ].slice(0, limit);
51
+
52
+ return {
53
+ results: results.map(result => ({
54
+ title: result.title,
55
+ url: result.url,
56
+ description: result.description,
57
+ snippet: result.description, // alias for compatibility
58
+ source: result.source
59
+ }))
60
+ };
61
+ }
62
+
63
+ // Export the search function
64
+ module.exports = customSearch;
package/package.json ADDED
@@ -0,0 +1,51 @@
1
+ {
2
+ "name": "ai-web-search",
3
+ "version": "1.0.0",
4
+ "description": "A lightweight tool that enables AI assistants to search the internet via local HTTP server",
5
+ "main": "src/index.js",
6
+ "type": "commonjs",
7
+ "scripts": {
8
+ "start": "node src/index.js",
9
+ "dev": "node src/index.js --dev",
10
+ "test": "node test/test.js",
11
+ "lint": "eslint src/"
12
+ },
13
+ "keywords": [
14
+ "ai",
15
+ "search",
16
+ "duckduckgo",
17
+ "web-search",
18
+ "openai",
19
+ "tool",
20
+ "api"
21
+ ],
22
+ "author": "",
23
+ "license": "MIT",
24
+ "dependencies": {
25
+ "duck-duck-scrape": "^2.2.7",
26
+ "express": "^4.18.2",
27
+ "cors": "^2.8.5",
28
+ "dotenv": "^16.3.1",
29
+ "express-rate-limit": "^7.1.5",
30
+ "axios": "^1.6.7",
31
+ "cheerio": "^1.0.0-rc.12"
32
+ },
33
+ "devDependencies": {
34
+ "eslint": "^8.56.0",
35
+ "jest": "^29.7.0"
36
+ },
37
+ "engines": {
38
+ "node": ">=14.0.0"
39
+ },
40
+ "files": [
41
+ "src/",
42
+ "bin/",
43
+ "examples/",
44
+ "test/",
45
+ "README.md",
46
+ "LICENSE"
47
+ ],
48
+ "bin": {
49
+ "ai-web-search": "./bin/cli.js"
50
+ }
51
+ }
package/src/index.js ADDED
@@ -0,0 +1,46 @@
1
+ #!/usr/bin/env node
2
+
3
+ const dotenv = require('dotenv');
4
+ const { createServer } = require('./server');
5
+
6
+ // Load environment variables
7
+ dotenv.config();
8
+
9
+ // Configuration
10
+ const config = {
11
+ port: process.env.PORT || 3000,
12
+ host: process.env.HOST || 'localhost',
13
+ dev: process.argv.includes('--dev'),
14
+ searchFnPath: process.env.SEARCH_FUNCTION_PATH || null,
15
+ rateLimit: {
16
+ windowMs: 15 * 60 * 1000, // 15 minutes
17
+ max: parseInt(process.env.RATE_LIMIT_MAX) || 100 // limit each IP to 100 requests per windowMs
18
+ }
19
+ };
20
+
21
+ // Start server
22
+ const server = createServer(config);
23
+
24
+ server.listen(config.port, config.host, () => {
25
+ console.log(`🔍 AI Web Search server running at http://${config.host}:${config.port}`);
26
+ console.log(`📡 Search endpoint: http://${config.host}:${config.port}/search`);
27
+ console.log(`🔧 Environment: ${config.dev ? 'development' : 'production'}`);
28
+ console.log(`⚡ Press Ctrl+C to stop`);
29
+ });
30
+
31
+ // Handle graceful shutdown
32
+ process.on('SIGINT', () => {
33
+ console.log('\n🛑 Shutting down server...');
34
+ server.close(() => {
35
+ console.log('✅ Server stopped');
36
+ process.exit(0);
37
+ });
38
+ });
39
+
40
+ process.on('SIGTERM', () => {
41
+ console.log('\n🛑 Received SIGTERM, shutting down...');
42
+ server.close(() => {
43
+ console.log('✅ Server stopped');
44
+ process.exit(0);
45
+ });
46
+ });
package/src/search.js ADDED
@@ -0,0 +1,224 @@
1
+ const DuckDuckGo = require('duck-duck-scrape');
2
+ let baiduSearchFn = null;
3
+ try {
4
+ baiduSearchFn = require('../examples/baidu-search');
5
+ } catch (e) {
6
+ // Baidu search module not available, ignore
7
+ }
8
+
9
+ class SearchEngine {
10
+ constructor(options = {}) {
11
+ this.options = {
12
+ safeSearch: options.safeSearch || DuckDuckGo.SafeSearchType.MODERATE,
13
+ timeout: options.timeout || 10000,
14
+ searchFn: options.searchFn || null,
15
+ ...options
16
+ };
17
+ }
18
+
19
+ /**
20
+ * Perform web search
21
+ * @param {string} query - Search query
22
+ * @param {Object} options - Search options
23
+ * @returns {Promise<Array>} Search results
24
+ */
25
+ async webSearch(query, options = {}) {
26
+ const { limit = 5, timeRange = 'month' } = options;
27
+
28
+ try {
29
+ // Use custom search function if provided
30
+ if (this.options.searchFn) {
31
+ const searchResults = await this.options.searchFn(query, {
32
+ limit,
33
+ timeRange,
34
+ type: 'web',
35
+ ...options
36
+ });
37
+
38
+ // Format results according to our standard
39
+ const results = (searchResults.results || []).slice(0, limit).map(result => ({
40
+ title: result.title || '',
41
+ url: result.url || '',
42
+ snippet: result.description || result.snippet || '',
43
+ source: result.source || 'custom'
44
+ }));
45
+
46
+ return {
47
+ query,
48
+ results,
49
+ totalResults: results.length,
50
+ timestamp: new Date().toISOString()
51
+ };
52
+ }
53
+
54
+ // Default: Use DuckDuckGo
55
+ const searchResults = await DuckDuckGo.search(query, {
56
+ safeSearch: this.options.safeSearch,
57
+ timeRange,
58
+ ...this.options
59
+ });
60
+
61
+ // Limit and format results
62
+ const results = searchResults.results.slice(0, limit).map(result => ({
63
+ title: result.title,
64
+ url: result.url,
65
+ snippet: result.description,
66
+ source: result.source || 'duckduckgo'
67
+ }));
68
+
69
+ return {
70
+ query,
71
+ results,
72
+ totalResults: searchResults.results.length,
73
+ timestamp: new Date().toISOString()
74
+ };
75
+ } catch (error) {
76
+ console.log(`⚠️ DuckDuckGo search failed: ${error.message}`);
77
+
78
+ // Try Baidu as fallback
79
+ if (baiduSearchFn) {
80
+ console.log('🔄 Trying Baidu as fallback...');
81
+ try {
82
+ const searchResults = await baiduSearchFn(query, {
83
+ limit,
84
+ timeRange,
85
+ type: 'web',
86
+ ...options
87
+ });
88
+
89
+ const results = (searchResults.results || []).slice(0, limit).map(result => ({
90
+ title: result.title || '',
91
+ url: result.url || '',
92
+ snippet: result.description || result.snippet || '',
93
+ source: result.source || 'baidu-fallback'
94
+ }));
95
+
96
+ return {
97
+ query,
98
+ results,
99
+ totalResults: results.length,
100
+ timestamp: new Date().toISOString(),
101
+ note: 'Used Baidu as fallback due to DuckDuckGo failure'
102
+ };
103
+ } catch (fallbackError) {
104
+ console.log(`⚠️ Baidu fallback also failed: ${fallbackError.message}`);
105
+ }
106
+ }
107
+
108
+ throw new Error(`Search failed: ${error.message}`);
109
+ }
110
+ }
111
+
112
+ /**
113
+ * Perform news search
114
+ * @param {string} query - Search query
115
+ * @param {Object} options - Search options
116
+ * @returns {Promise<Array>} News results
117
+ */
118
+ async newsSearch(query, options = {}) {
119
+ const { limit = 5, timeRange = 'week' } = options;
120
+
121
+ try {
122
+ // Use custom search function if provided
123
+ if (this.options.searchFn) {
124
+ const searchResults = await this.options.searchFn(query, {
125
+ limit,
126
+ timeRange,
127
+ type: 'news',
128
+ ...options
129
+ });
130
+
131
+ // Format results according to our standard
132
+ const results = (searchResults.results || []).slice(0, limit).map(result => ({
133
+ title: result.title || '',
134
+ url: result.url || '',
135
+ snippet: result.description || result.snippet || '',
136
+ source: result.source || 'custom',
137
+ date: result.publishedAt || result.date || new Date().toISOString()
138
+ }));
139
+
140
+ return {
141
+ query,
142
+ results,
143
+ totalResults: results.length,
144
+ timestamp: new Date().toISOString()
145
+ };
146
+ }
147
+
148
+ // Default: Use DuckDuckGo
149
+ const searchResults = await DuckDuckGo.search(query, {
150
+ safeSearch: this.options.safeSearch,
151
+ news: true,
152
+ timeRange,
153
+ ...this.options
154
+ });
155
+
156
+ const results = searchResults.results.slice(0, limit).map(result => ({
157
+ title: result.title,
158
+ url: result.url,
159
+ snippet: result.description,
160
+ source: result.source,
161
+ date: result.publishedAt || new Date().toISOString()
162
+ }));
163
+
164
+ return {
165
+ query,
166
+ results,
167
+ totalResults: searchResults.results.length,
168
+ timestamp: new Date().toISOString()
169
+ };
170
+ } catch (error) {
171
+ console.log(`⚠️ DuckDuckGo news search failed: ${error.message}`);
172
+
173
+ // Try Baidu as fallback for news
174
+ if (baiduSearchFn) {
175
+ console.log('🔄 Trying Baidu news as fallback...');
176
+ try {
177
+ const searchResults = await baiduSearchFn(query, {
178
+ limit,
179
+ timeRange,
180
+ type: 'news',
181
+ ...options
182
+ });
183
+
184
+ const results = (searchResults.results || []).slice(0, limit).map(result => ({
185
+ title: result.title || '',
186
+ url: result.url || '',
187
+ snippet: result.description || result.snippet || '',
188
+ source: result.source || 'baidu-fallback',
189
+ date: result.publishedAt || result.date || new Date().toISOString()
190
+ }));
191
+
192
+ return {
193
+ query,
194
+ results,
195
+ totalResults: results.length,
196
+ timestamp: new Date().toISOString(),
197
+ note: 'Used Baidu as fallback due to DuckDuckGo failure'
198
+ };
199
+ } catch (fallbackError) {
200
+ console.log(`⚠️ Baidu news fallback also failed: ${fallbackError.message}`);
201
+ }
202
+ }
203
+
204
+ throw new Error(`News search failed: ${error.message}`);
205
+ }
206
+ }
207
+
208
+ /**
209
+ * Get search suggestions (placeholder - not implemented in duck-duck-scrape)
210
+ * @param {string} query - Partial query
211
+ * @returns {Promise<Array>} Search suggestions
212
+ */
213
+ async getSuggestions(query) {
214
+ // duck-duck-scrape doesn't provide getSuggestions method
215
+ // Return empty suggestions for now
216
+ return {
217
+ query,
218
+ suggestions: [],
219
+ timestamp: new Date().toISOString()
220
+ };
221
+ }
222
+ }
223
+
224
+ module.exports = SearchEngine;
package/src/server.js ADDED
@@ -0,0 +1,156 @@
1
+ const express = require('express');
2
+ const cors = require('cors');
3
+ const rateLimit = require('express-rate-limit');
4
+ const SearchEngine = require('./search');
5
+
6
+ function createServer(config) {
7
+ const app = express();
8
+ // Load custom search function if provided
9
+ let searchFn = null;
10
+ if (config.searchFnPath) {
11
+ try {
12
+ searchFn = require(config.searchFnPath);
13
+ console.log(`🔌 Loaded custom search function from ${config.searchFnPath}`);
14
+ } catch (error) {
15
+ console.error(`❌ Failed to load custom search function from ${config.searchFnPath}:`, error.message);
16
+ }
17
+ }
18
+
19
+ const searchEngine = new SearchEngine({ searchFn });
20
+
21
+ // Middleware
22
+ app.use(cors());
23
+ app.use(express.json());
24
+ app.use(express.urlencoded({ extended: true }));
25
+
26
+ // Rate limiting
27
+ const limiter = rateLimit({
28
+ windowMs: config.rateLimit.windowMs,
29
+ max: config.rateLimit.max,
30
+ message: {
31
+ error: 'Too many requests',
32
+ message: 'Please try again later'
33
+ }
34
+ });
35
+ app.use('/search', limiter);
36
+
37
+ // Health check endpoint
38
+ app.get('/health', (req, res) => {
39
+ res.json({
40
+ status: 'healthy',
41
+ timestamp: new Date().toISOString(),
42
+ uptime: process.uptime()
43
+ });
44
+ });
45
+
46
+ // Tool definition endpoint (for AI software integration)
47
+ app.get('/tool-definition', (req, res) => {
48
+ res.json({
49
+ type: 'function',
50
+ function: {
51
+ name: 'web_search',
52
+ description: 'Search the internet to get latest information. Useful for finding current events, facts, news, or any information that might have changed recently.',
53
+ parameters: {
54
+ type: 'object',
55
+ properties: {
56
+ query: {
57
+ type: 'string',
58
+ description: 'The search query to find information'
59
+ },
60
+ limit: {
61
+ type: 'integer',
62
+ description: 'Number of results to return (default: 5, max: 20)',
63
+ default: 5,
64
+ minimum: 1,
65
+ maximum: 20
66
+ },
67
+ type: {
68
+ type: 'string',
69
+ enum: ['web', 'news'],
70
+ description: 'Type of search: "web" for general web search, "news" for news articles',
71
+ default: 'web'
72
+ }
73
+ },
74
+ required: ['query']
75
+ }
76
+ }
77
+ });
78
+ });
79
+
80
+ // Main search endpoint
81
+ app.get('/search', async (req, res) => {
82
+ try {
83
+ const { query, limit = 5, type = 'web' } = req.query;
84
+
85
+ // Validate query
86
+ if (!query || typeof query !== 'string' || query.trim().length === 0) {
87
+ return res.status(400).json({
88
+ error: 'Invalid query',
89
+ message: 'Query parameter is required and must be a non-empty string'
90
+ });
91
+ }
92
+
93
+ // Validate limit
94
+ const searchLimit = Math.min(Math.max(parseInt(limit) || 5, 1), 20);
95
+
96
+ let results;
97
+ if (type === 'news') {
98
+ results = await searchEngine.newsSearch(query, { limit: searchLimit });
99
+ } else {
100
+ results = await searchEngine.webSearch(query, { limit: searchLimit });
101
+ }
102
+
103
+ res.json(results);
104
+ } catch (error) {
105
+ console.error('Search error:', error);
106
+ res.status(500).json({
107
+ error: 'Search failed',
108
+ message: error.message
109
+ });
110
+ }
111
+ });
112
+
113
+ // Search suggestions endpoint
114
+ app.get('/suggest', async (req, res) => {
115
+ try {
116
+ const { query } = req.query;
117
+
118
+ if (!query || typeof query !== 'string') {
119
+ return res.status(400).json({
120
+ error: 'Invalid query',
121
+ message: 'Query parameter is required'
122
+ });
123
+ }
124
+
125
+ const suggestions = await searchEngine.getSuggestions(query);
126
+ res.json(suggestions);
127
+ } catch (error) {
128
+ console.error('Suggestion error:', error);
129
+ res.status(500).json({
130
+ error: 'Failed to get suggestions',
131
+ message: error.message
132
+ });
133
+ }
134
+ });
135
+
136
+ // 404 handler
137
+ app.use((req, res) => {
138
+ res.status(404).json({
139
+ error: 'Not found',
140
+ message: `Endpoint ${req.path} does not exist`
141
+ });
142
+ });
143
+
144
+ // Error handler
145
+ app.use((err, req, res, next) => {
146
+ console.error('Server error:', err);
147
+ res.status(500).json({
148
+ error: 'Internal server error',
149
+ message: config.dev ? err.message : 'Something went wrong'
150
+ });
151
+ });
152
+
153
+ return app;
154
+ }
155
+
156
+ module.exports = { createServer };
package/test/test.js ADDED
@@ -0,0 +1,65 @@
1
+ #!/usr/bin/env node
2
+
3
+ const SearchEngine = require('../src/search');
4
+
5
+ async function runTests() {
6
+ console.log('🧪 Running AI Web Search Tests\n');
7
+
8
+ const searchEngine = new SearchEngine();
9
+
10
+ // Test 1: Basic web search
11
+ console.log('1. Testing web search...');
12
+ try {
13
+ const webResults = await searchEngine.webSearch('人工智能最新发展', { limit: 3 });
14
+ console.log('✅ Web search successful');
15
+ console.log(` Found ${webResults.results.length} results for "${webResults.query}"`);
16
+ if (webResults.results.length > 0) {
17
+ console.log(` First result: ${webResults.results[0].title}`);
18
+ console.log(` URL: ${webResults.results[0].url}`);
19
+ }
20
+ console.log('');
21
+ } catch (error) {
22
+ console.log('❌ Web search failed:', error.message);
23
+ console.log('');
24
+ }
25
+
26
+ // Test 2: News search
27
+ console.log('2. Testing news search...');
28
+ try {
29
+ const newsResults = await searchEngine.newsSearch('科技新闻', { limit: 3 });
30
+ console.log('✅ News search successful');
31
+ console.log(` Found ${newsResults.results.length} news results`);
32
+ if (newsResults.results.length > 0) {
33
+ console.log(` Latest news: ${newsResults.results[0].title}`);
34
+ }
35
+ console.log('');
36
+ } catch (error) {
37
+ console.log('❌ News search failed:', error.message);
38
+ console.log('');
39
+ }
40
+
41
+ // Test 3: Search suggestions
42
+ console.log('3. Testing search suggestions...');
43
+ try {
44
+ const suggestions = await searchEngine.getSuggestions('机器学');
45
+ console.log('✅ Suggestions successful');
46
+ console.log(` Got ${suggestions.suggestions.length} suggestions`);
47
+ if (suggestions.suggestions.length > 0) {
48
+ console.log(` First suggestion: ${suggestions.suggestions[0]}`);
49
+ }
50
+ console.log('');
51
+ } catch (error) {
52
+ console.log('❌ Suggestions failed:', error.message);
53
+ console.log('');
54
+ }
55
+
56
+ console.log('🎉 All tests completed!');
57
+ console.log('\nNote: Some tests may fail due to network issues or DuckDuckGo rate limiting.');
58
+ console.log('If tests fail, try again in a few minutes.');
59
+ }
60
+
61
+ // Run tests
62
+ runTests().catch(error => {
63
+ console.error('Test runner failed:', error);
64
+ process.exit(1);
65
+ });