@epoch-agent/plugin-web 3.1.2 → 4.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +45 -3
- package/dist/index.d.ts +8 -0
- package/dist/index.js +110 -62
- package/package.json +3 -3
package/README.md
CHANGED
|
@@ -7,8 +7,8 @@
|
|
|
7
7
|
web 搜索(native / tavily / brave / searxng / keyless 五个后端)
|
|
8
8
|
- ❌ **不做**:不执行 JS、不渲染页面、不带浏览器;**不抓搜索引擎的 HTML 结果页**
|
|
9
9
|
(违反 ToS、随时会碎、要处理验证码 —— 要搜就走后端,不走爬虫)
|
|
10
|
-
-
|
|
11
|
-
HTML 转换是手写的(无第三方库)
|
|
10
|
+
- **依赖**:`@epoch-agent/protocol` + `@epoch-agent/infra`,两个都是 `dependencies`
|
|
11
|
+
(本包**没有 `peerDependencies`**),HTML 转换是手写的(无第三方库)
|
|
12
12
|
|
|
13
13
|
## 工具参数
|
|
14
14
|
|
|
@@ -50,7 +50,29 @@
|
|
|
50
50
|
- **`keyless`**(`search/keyless.ts`)打 Exa / Parallel 的匿名 MCP 端点,
|
|
51
51
|
带游标轮转摊平两家免费额度。**查询词会发给第三方**,所以它排在最后、且能整档关掉。
|
|
52
52
|
|
|
53
|
-
|
|
53
|
+
两个开关**缺省开**,只有显式写 `search.native: false` / `search.keyless: false` 才关
|
|
54
|
+
(实现是 `=== false ? 关 : 开`,**不是「真值才开」**)。
|
|
55
|
+
|
|
56
|
+
#### 为什么是这两家(2026-09-15 实探,不是照抄)
|
|
57
|
+
|
|
58
|
+
四家里只有 epoch 原来会「什么都没有就把工具拿掉」,所以选型时把同级项目的做法
|
|
59
|
+
和这几家的实际表现一起量了一遍:
|
|
60
|
+
|
|
61
|
+
| 结论 | 证据 |
|
|
62
|
+
| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
|
|
63
|
+
| Exa / Parallel 公开 MCP 端点**匿名可用** | 无 Authorization,`tools/call` 返回 200 |
|
|
64
|
+
| **直连可达**,不需要代理 | 绕开本机代理重测:两域名 200 / 0.7s,结果一致 |
|
|
65
|
+
| Exa 中文覆盖**不差**、但**质量差** | 中文查 8/8 是中文结果,其中 3 条品牌软文、2 条豆丁网 |
|
|
66
|
+
| Parallel 有语义漂移 | 英文查 10 条里 3 条是 Curtain **Wall**(建筑幕墙) |
|
|
67
|
+
| **DeepSeek 服务端搜索更好**(所以 native 排第一) | `POST /anthropic/v1/messages` + `web_search_20250305`:10 条全中文、**0 条软文**,还捞到了 Exa / Parallel 都没有的平台维度数据 |
|
|
68
|
+
|
|
69
|
+
横向对照:codex 用 OpenAI Responses 的服务端 `web_search`、gemini-cli 用 Gemini
|
|
70
|
+
grounding(两家都是 `native` 那一档),opencode 打 Exa / Parallel 公开端点(就是
|
|
71
|
+
`keyless` 这一档),hermes-agent 做了最全的探测级联 + keyless ring。
|
|
72
|
+
|
|
73
|
+
> ⚠️ **native 排在 tavily 前面是有代价的决定**:只设了 `TAVILY_API_KEY` 却没写
|
|
74
|
+
> `search.provider` 的人,从这一版起会改走 native,那把 key 变成静默失效的配置。
|
|
75
|
+
> 要钉住一家就显式写 `search.provider`。
|
|
54
76
|
|
|
55
77
|
> SearXNG **默认没开** JSON 输出,实例的 `settings.yml` 里要有
|
|
56
78
|
> `search: { formats: [html, json] }`。没开时它返回 403,我们把这句话写进了错误里 ——
|
|
@@ -83,6 +105,26 @@
|
|
|
83
105
|
|
|
84
106
|
其余限制:响应上限 5 MB,Cloudflare 挑战会重试一次。
|
|
85
107
|
|
|
108
|
+
## 回执语言:给模型读的一律英文
|
|
109
|
+
|
|
110
|
+
`web_fetch` / `web_search` 的**工具回执**——失败说明、被拦截的原因、provider 报错、
|
|
111
|
+
「一条都没搜到」这类说明——**全部是英文**,判据同
|
|
112
|
+
[core 的 `code-mode/notice.ts`](../../core/src/code-mode/notice.ts) 文件头(2026-11-05
|
|
113
|
+
拍的口径:回执的读者是模型,跟工具 description 同一档,不跟界面语言走)。
|
|
114
|
+
所以这几处 `t()` 调用显式传了 `'en'`,`check:i18n` 那边也不要指望它们跟着 `zh` 变。
|
|
115
|
+
|
|
116
|
+
反过来,这些**不**在这一档里:
|
|
117
|
+
|
|
118
|
+
- **抓回来的网页正文 / 搜索结果的标题、摘要**——那是别人的数据,原样回给模型
|
|
119
|
+
- **`unavailableReason()` 与 `selectSearchProvider()` 的 `detail`**——启动诊断,
|
|
120
|
+
用户读的,照旧跟着界面语言走
|
|
121
|
+
- **`native` 档的 `buildPrompt()`**——发给对端模型的指令,用户读的正是它的回答,
|
|
122
|
+
界面切成英文却用中文问它,答案多半也是中文
|
|
123
|
+
|
|
124
|
+
拦截原因那一格有个额外要求:**URL 和判定原因都得留着**。`url-safety` 只给原因
|
|
125
|
+
(带主机名 / 解析到的地址),被拒的那个完整 URL 由 `web-fetch` 补在前面 ——
|
|
126
|
+
少了 URL 模型不知道是第一跳还是第五跳被拒,少了原因它只能盲目换个地址重试。
|
|
127
|
+
|
|
86
128
|
## 三个 annotation 不只是元数据
|
|
87
129
|
|
|
88
130
|
```ts
|
package/dist/index.d.ts
CHANGED
|
@@ -238,6 +238,10 @@ declare function createBraveProvider(opts?: BraveOptions): SearchProvider;
|
|
|
238
238
|
* 一份实现就够。`openai`(Responses API)和 `google`(grounding)的传输层
|
|
239
239
|
* 完全不同,是**另外两份实现**,不在这一版里;能力表留着扩展位,
|
|
240
240
|
* 表里没有的 provider 会如实报原因然后落到下一档,不是报错。
|
|
241
|
+
*
|
|
242
|
+
* ⚠️ **这一档只有一处跟界面语言走:`buildPrompt()`** —— 那是发给对端模型的指令,
|
|
243
|
+
* 用户读的正是它的回答。而 `SearchError` / `ignored` 那两句进的是**工具回执**,
|
|
244
|
+
* 一律英文(`t(..., 'en')`,2026-11-05 的口径)。
|
|
241
245
|
*/
|
|
242
246
|
|
|
243
247
|
interface NativeOptions {
|
|
@@ -283,6 +287,10 @@ declare function createNativeProvider(opts?: NativeOptions): SearchProvider;
|
|
|
283
287
|
* - Parallel 还收一个可选的 `model_name` 分析字段 —— **故意不传**。
|
|
284
288
|
*
|
|
285
289
|
* 即便如此,查询词本身仍然会发给第三方。要整档关掉:`search.keyless: false`。
|
|
290
|
+
*
|
|
291
|
+
* ⚠️ **`SearchError` / `ignored` 那两句是进工具回执的,一律英文**
|
|
292
|
+
* (`t(..., 'en')`,2026-11-05 的口径);而 `unavailableReason()` 那句是启动诊断、
|
|
293
|
+
* 给用户看的,照旧跟界面语言走。
|
|
286
294
|
*/
|
|
287
295
|
|
|
288
296
|
/** 给用例用:把游标拨回 0,否则用例之间会互相影响 */
|
package/dist/index.js
CHANGED
|
@@ -110,24 +110,30 @@ function checkUrlSafety(url) {
|
|
|
110
110
|
try {
|
|
111
111
|
parsed = new URL(url);
|
|
112
112
|
} catch {
|
|
113
|
-
return { safe: false, reason: "
|
|
113
|
+
return { safe: false, reason: "not a valid URL (it could not be parsed)" };
|
|
114
114
|
}
|
|
115
115
|
if (!ALLOWED_PROTOCOLS.includes(parsed.protocol)) {
|
|
116
|
-
return {
|
|
116
|
+
return {
|
|
117
|
+
safe: false,
|
|
118
|
+
reason: `disallowed protocol: ${parsed.protocol} (only http and https are allowed)`
|
|
119
|
+
};
|
|
117
120
|
}
|
|
118
121
|
if (parsed.username || parsed.password) {
|
|
119
|
-
return { safe: false, reason: "URL
|
|
122
|
+
return { safe: false, reason: "URL must not carry a username or password" };
|
|
120
123
|
}
|
|
121
124
|
const host = parsed.hostname.toLowerCase().replace(/^\[|\]$/g, "").replace(/\.$/, "");
|
|
122
|
-
if (!host) return { safe: false, reason: "
|
|
125
|
+
if (!host) return { safe: false, reason: "missing hostname" };
|
|
123
126
|
if (BLOCKED_HOSTNAMES.has(host)) {
|
|
124
|
-
return {
|
|
127
|
+
return {
|
|
128
|
+
safe: false,
|
|
129
|
+
reason: `blocked host (loopback or cloud metadata endpoint): ${host}`
|
|
130
|
+
};
|
|
125
131
|
}
|
|
126
132
|
if (BLOCKED_SUFFIXES.some((s) => host.endsWith(s))) {
|
|
127
|
-
return { safe: false, reason:
|
|
133
|
+
return { safe: false, reason: `blocked internal domain suffix: ${host}` };
|
|
128
134
|
}
|
|
129
135
|
if (isIP(host) !== 0 && isBlockedAddress(host)) {
|
|
130
|
-
return { safe: false, reason:
|
|
136
|
+
return { safe: false, reason: `blocked private or reserved address: ${host}` };
|
|
131
137
|
}
|
|
132
138
|
return { safe: true };
|
|
133
139
|
}
|
|
@@ -142,7 +148,7 @@ async function checkUrlSafetyAsync(url) {
|
|
|
142
148
|
if (isBlockedAddress(r.address)) {
|
|
143
149
|
return {
|
|
144
150
|
safe: false,
|
|
145
|
-
reason: `${host}
|
|
151
|
+
reason: `${host} resolves to the private or reserved address ${r.address} \u2014 refusing to connect`
|
|
146
152
|
};
|
|
147
153
|
}
|
|
148
154
|
}
|
|
@@ -154,6 +160,8 @@ async function checkUrlSafetyAsync(url) {
|
|
|
154
160
|
// src/tools/web-fetch.ts
|
|
155
161
|
var MAX_SIZE = 5 * 1024 * 1024;
|
|
156
162
|
var MAX_REDIRECTS = 5;
|
|
163
|
+
var TOO_LARGE = "Response too large (exceeds the 5MB limit) \u2014 fetch a smaller resource instead";
|
|
164
|
+
var TIMEOUT_HINT = "Request timed out \u2014 retry with a larger timeout (up to 120s) or fetch a smaller page";
|
|
157
165
|
var BROWSER_UA = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36";
|
|
158
166
|
function acceptFor(format2) {
|
|
159
167
|
if (format2 === "markdown") return "text/markdown;q=1.0, text/html;q=0.9, text/plain;q=0.8";
|
|
@@ -165,7 +173,7 @@ async function fetchWithSafeRedirects(startUrl, init) {
|
|
|
165
173
|
for (let hop = 0; hop <= MAX_REDIRECTS; hop++) {
|
|
166
174
|
const safety = await checkUrlSafetyAsync(current);
|
|
167
175
|
if (!safety.safe) {
|
|
168
|
-
const where = hop === 0 ?
|
|
176
|
+
const where = hop === 0 ? `Refused to fetch ${current}` : `Refused redirect hop ${hop} (${current})`;
|
|
169
177
|
return { error: `${where}: ${safety.reason}` };
|
|
170
178
|
}
|
|
171
179
|
const response = await fetch(current, { ...init, redirect: "manual" });
|
|
@@ -177,18 +185,21 @@ async function fetchWithSafeRedirects(startUrl, init) {
|
|
|
177
185
|
try {
|
|
178
186
|
next = new URL(location, current).toString();
|
|
179
187
|
} catch {
|
|
180
|
-
return { error:
|
|
188
|
+
return { error: `Redirect target could not be parsed: ${location}` };
|
|
181
189
|
}
|
|
182
190
|
current = next;
|
|
183
191
|
continue;
|
|
184
192
|
}
|
|
185
193
|
return { response, finalUrl: current };
|
|
186
194
|
}
|
|
187
|
-
return {
|
|
195
|
+
return {
|
|
196
|
+
error: `Gave up after more than ${MAX_REDIRECTS} redirects (possible redirect loop)`
|
|
197
|
+
};
|
|
188
198
|
}
|
|
189
199
|
async function execWebFetch(args, _ctx) {
|
|
190
200
|
const url = String(args["url"] || "").trim();
|
|
191
|
-
if (!url)
|
|
201
|
+
if (!url)
|
|
202
|
+
return fail(5001, "URL is empty \u2014 pass a full page URL that starts with http:// or https://");
|
|
192
203
|
const format2 = args["format"] || "markdown";
|
|
193
204
|
const timeout = Math.min(args["timeout"] || 30, 120) * 1e3;
|
|
194
205
|
const controller = new AbortController();
|
|
@@ -214,10 +225,9 @@ async function execWebFetch(args, _ctx) {
|
|
|
214
225
|
if (!response.ok) return fail(5003, `HTTP ${response.status}: ${response.statusText}`);
|
|
215
226
|
const contentType = response.headers.get("content-type") || "";
|
|
216
227
|
const contentLen = response.headers.get("content-length");
|
|
217
|
-
if (contentLen && Number(contentLen) > MAX_SIZE)
|
|
218
|
-
return fail(5004, "\u54CD\u5E94\u8FC7\u5927 (exceeds 5MB limit)");
|
|
228
|
+
if (contentLen && Number(contentLen) > MAX_SIZE) return fail(5004, TOO_LARGE);
|
|
219
229
|
const body = await readCapped(response, MAX_SIZE);
|
|
220
|
-
if (body === null) return fail(5004,
|
|
230
|
+
if (body === null) return fail(5004, TOO_LARGE);
|
|
221
231
|
const raw = new TextDecoder().decode(body);
|
|
222
232
|
const isHtml = contentType.includes("text/html") || raw.includes("<html") || raw.includes("<HTML");
|
|
223
233
|
let output;
|
|
@@ -227,7 +237,7 @@ async function execWebFetch(args, _ctx) {
|
|
|
227
237
|
output = raw.slice(0, 5e4);
|
|
228
238
|
}
|
|
229
239
|
const title = extractTitle(raw, isHtml);
|
|
230
|
-
const prefix = finalUrl !== url ? `>
|
|
240
|
+
const prefix = finalUrl !== url ? `> Final URL: ${finalUrl}
|
|
231
241
|
|
|
232
242
|
` : "";
|
|
233
243
|
return {
|
|
@@ -239,9 +249,9 @@ ${output}` : output),
|
|
|
239
249
|
};
|
|
240
250
|
} catch (err) {
|
|
241
251
|
if (err instanceof Error && err.name === "AbortError") {
|
|
242
|
-
return fail(5005,
|
|
252
|
+
return fail(5005, TIMEOUT_HINT);
|
|
243
253
|
}
|
|
244
|
-
return fail(5006,
|
|
254
|
+
return fail(5006, `Network error: ${err instanceof Error ? err.message : String(err)}`);
|
|
245
255
|
} finally {
|
|
246
256
|
clearTimeout(timer);
|
|
247
257
|
}
|
|
@@ -296,18 +306,20 @@ async function searchFetch(url, init, who) {
|
|
|
296
306
|
response = await fetch(url, init);
|
|
297
307
|
} catch (err) {
|
|
298
308
|
if (err instanceof Error && err.name === "AbortError") {
|
|
299
|
-
throw new SearchError(`${who}
|
|
309
|
+
throw new SearchError(`${who} search timed out`);
|
|
300
310
|
}
|
|
301
|
-
throw new SearchError(
|
|
311
|
+
throw new SearchError(
|
|
312
|
+
`${who} is unreachable: ${err instanceof Error ? err.message : String(err)}`
|
|
313
|
+
);
|
|
302
314
|
}
|
|
303
315
|
if (!response.ok) {
|
|
304
|
-
const hint = response.status === 401 || response.status === 403 ? "
|
|
305
|
-
throw new SearchError(`${who}
|
|
316
|
+
const hint = response.status === 401 || response.status === 403 ? " (invalid or unauthorized API key)" : response.status === 429 ? " (quota exhausted or rate limited \u2014 retry later or switch provider)" : "";
|
|
317
|
+
throw new SearchError(`${who} returned HTTP ${response.status}${hint}`, response.status);
|
|
306
318
|
}
|
|
307
319
|
try {
|
|
308
320
|
return await response.json();
|
|
309
321
|
} catch {
|
|
310
|
-
throw new SearchError(`${who}
|
|
322
|
+
throw new SearchError(`${who} returned a response that is not valid JSON`);
|
|
311
323
|
}
|
|
312
324
|
}
|
|
313
325
|
function str(obj, key) {
|
|
@@ -347,10 +359,14 @@ function format(results, ignored, summary) {
|
|
|
347
359
|
${snippet}` : `${i + 1}. ${r.title}${date}
|
|
348
360
|
${r.url}`;
|
|
349
361
|
});
|
|
350
|
-
const head = `${results.length} \
|
|
351
|
-
const lead = summary ? [
|
|
362
|
+
const head = `${results.length} result${results.length === 1 ? "" : "s"} (titles, URLs and snippets only \u2014 call web_fetch on a specific URL to read the page)`;
|
|
363
|
+
const lead = summary ? [
|
|
364
|
+
"",
|
|
365
|
+
"Backend summary (synthesized from the sources below \u2014 cite a concrete URL when you quote it):",
|
|
366
|
+
summary
|
|
367
|
+
] : [];
|
|
352
368
|
const tail = ignored.length > 0 ? [`
|
|
353
|
-
|
|
369
|
+
Note: ${ignored.join("; ")}`] : [];
|
|
354
370
|
return [head, ...lead, "", ...lines, ...tail].join("\n");
|
|
355
371
|
}
|
|
356
372
|
function stringArray(value) {
|
|
@@ -366,7 +382,7 @@ function createWebSearchExec(deps) {
|
|
|
366
382
|
return {
|
|
367
383
|
success: false,
|
|
368
384
|
output: "",
|
|
369
|
-
error: { code: 5101, message: "
|
|
385
|
+
error: { code: 5101, message: "Query must not be empty" },
|
|
370
386
|
duration: 0
|
|
371
387
|
};
|
|
372
388
|
}
|
|
@@ -387,7 +403,11 @@ function createWebSearchExec(deps) {
|
|
|
387
403
|
if (cached) {
|
|
388
404
|
return {
|
|
389
405
|
success: true,
|
|
390
|
-
output: format(
|
|
406
|
+
output: format(
|
|
407
|
+
cached.results,
|
|
408
|
+
["served from this session cache \u2014 no new network request was made"],
|
|
409
|
+
cached.summary
|
|
410
|
+
),
|
|
391
411
|
duration: Date.now() - started
|
|
392
412
|
};
|
|
393
413
|
}
|
|
@@ -402,7 +422,7 @@ function createWebSearchExec(deps) {
|
|
|
402
422
|
if (outcome.results.length === 0) {
|
|
403
423
|
return {
|
|
404
424
|
success: true,
|
|
405
|
-
output:
|
|
425
|
+
output: `No results from ${deps.provider.name}. Try different wording, or loosen the filters (domains / recencyDays).`,
|
|
406
426
|
duration: Date.now() - started
|
|
407
427
|
};
|
|
408
428
|
}
|
|
@@ -412,14 +432,14 @@ function createWebSearchExec(deps) {
|
|
|
412
432
|
duration: Date.now() - started
|
|
413
433
|
};
|
|
414
434
|
} catch (err) {
|
|
415
|
-
const message = err instanceof SearchError ? err.message :
|
|
435
|
+
const message = err instanceof SearchError ? err.message : `Search failed: ${err instanceof Error ? err.message : String(err)}`;
|
|
416
436
|
return {
|
|
417
437
|
success: false,
|
|
418
438
|
output: "",
|
|
419
439
|
error: {
|
|
420
440
|
code: 5102,
|
|
421
441
|
message,
|
|
422
|
-
suggestion: "\
|
|
442
|
+
suggestion: "This is not fatal \u2014 carry on with other work, or retry later."
|
|
423
443
|
},
|
|
424
444
|
duration: Date.now() - started
|
|
425
445
|
};
|
|
@@ -474,7 +494,9 @@ function createBraveProvider(opts = {}) {
|
|
|
474
494
|
}
|
|
475
495
|
const ignored = [];
|
|
476
496
|
if (options.recencyDays !== void 0 && !freshness) {
|
|
477
|
-
ignored.push(
|
|
497
|
+
ignored.push(
|
|
498
|
+
'recencyDays (brave freshness only reaches "past year"; larger day counts were ignored)'
|
|
499
|
+
);
|
|
478
500
|
}
|
|
479
501
|
return { results, ...ignored.length > 0 ? { ignored } : {} };
|
|
480
502
|
}
|
|
@@ -553,25 +575,29 @@ async function mcpCall(init) {
|
|
|
553
575
|
});
|
|
554
576
|
} catch (err) {
|
|
555
577
|
if (err instanceof Error && err.name === "AbortError") {
|
|
556
|
-
throw new SearchError(t2("web_search_setup.err_timeout", { who: init.who }));
|
|
578
|
+
throw new SearchError(t2("web_search_setup.err_timeout", { who: init.who }, "en"));
|
|
557
579
|
}
|
|
558
580
|
throw new SearchError(
|
|
559
|
-
t2(
|
|
560
|
-
|
|
561
|
-
|
|
562
|
-
|
|
581
|
+
t2(
|
|
582
|
+
"web_search_setup.err_unreachable",
|
|
583
|
+
{
|
|
584
|
+
who: init.who,
|
|
585
|
+
detail: err instanceof Error ? err.message : String(err)
|
|
586
|
+
},
|
|
587
|
+
"en"
|
|
588
|
+
)
|
|
563
589
|
);
|
|
564
590
|
}
|
|
565
591
|
if (!response.ok) {
|
|
566
|
-
const hint = response.status === 429 ? t2("web_search_setup.err_rate_limited_hint") : "";
|
|
592
|
+
const hint = response.status === 429 ? t2("web_search_setup.err_rate_limited_hint", void 0, "en") : "";
|
|
567
593
|
throw new SearchError(
|
|
568
|
-
t2("web_search_setup.err_http", { who: init.who, status: response.status, hint }),
|
|
594
|
+
t2("web_search_setup.err_http", { who: init.who, status: response.status, hint }, "en"),
|
|
569
595
|
response.status
|
|
570
596
|
);
|
|
571
597
|
}
|
|
572
598
|
const text = parseMcpResponse(await response.text());
|
|
573
599
|
if (text === void 0)
|
|
574
|
-
throw new SearchError(t2("web_search_setup.err_no_content", { who: init.who }));
|
|
600
|
+
throw new SearchError(t2("web_search_setup.err_no_content", { who: init.who }, "en"));
|
|
575
601
|
return text;
|
|
576
602
|
}
|
|
577
603
|
// src/search/keyless.ts
|
|
@@ -630,7 +656,7 @@ function parseParallel(text, max) {
|
|
|
630
656
|
try {
|
|
631
657
|
data = JSON.parse(text);
|
|
632
658
|
} catch {
|
|
633
|
-
throw new SearchError(t3("web_search_setup.err_bad_json", { who: "parallel" }));
|
|
659
|
+
throw new SearchError(t3("web_search_setup.err_bad_json", { who: "parallel" }, "en"));
|
|
634
660
|
}
|
|
635
661
|
if (!Array.isArray(data.results)) return [];
|
|
636
662
|
const out = [];
|
|
@@ -699,9 +725,13 @@ function createKeylessProvider(opts = {}) {
|
|
|
699
725
|
}
|
|
700
726
|
}
|
|
701
727
|
throw new SearchError(
|
|
702
|
-
t3(
|
|
703
|
-
|
|
704
|
-
|
|
728
|
+
t3(
|
|
729
|
+
"web_search_setup.keyless_all_failed",
|
|
730
|
+
{
|
|
731
|
+
tried: failures.join(t3("web_search_setup.tried_sep", void 0, "en"))
|
|
732
|
+
},
|
|
733
|
+
"en"
|
|
734
|
+
)
|
|
705
735
|
);
|
|
706
736
|
}
|
|
707
737
|
};
|
|
@@ -709,9 +739,11 @@ function createKeylessProvider(opts = {}) {
|
|
|
709
739
|
function ignoredOf(options) {
|
|
710
740
|
const ignored = [];
|
|
711
741
|
if (options.recencyDays !== void 0)
|
|
712
|
-
ignored.push(t3("web_search_setup.ignored_recency_keyless"));
|
|
713
|
-
if (options.domains?.length)
|
|
714
|
-
|
|
742
|
+
ignored.push(t3("web_search_setup.ignored_recency_keyless", void 0, "en"));
|
|
743
|
+
if (options.domains?.length)
|
|
744
|
+
ignored.push(t3("web_search_setup.ignored_domains_keyless", void 0, "en"));
|
|
745
|
+
if (options.excludeDomains?.length)
|
|
746
|
+
ignored.push(t3("web_search_setup.ignored_exclude_keyless", void 0, "en"));
|
|
715
747
|
return ignored;
|
|
716
748
|
}
|
|
717
749
|
// src/search/native.ts
|
|
@@ -795,11 +827,11 @@ function createNativeProvider(opts = {}) {
|
|
|
795
827
|
const results = collectResults(blocks, options.maxResults);
|
|
796
828
|
if (results.length === 0) {
|
|
797
829
|
throw new SearchError(
|
|
798
|
-
t4("web_search_setup.native_no_search", { provider: opts.providerType })
|
|
830
|
+
t4("web_search_setup.native_no_search", { provider: opts.providerType }, "en")
|
|
799
831
|
);
|
|
800
832
|
}
|
|
801
833
|
const summary = collectSummary(blocks);
|
|
802
|
-
const ignored = options.recencyDays === void 0 ? [] : [t4("web_search_setup.ignored_recency_native")];
|
|
834
|
+
const ignored = options.recencyDays === void 0 ? [] : [t4("web_search_setup.ignored_recency_native", void 0, "en")];
|
|
803
835
|
return {
|
|
804
836
|
results,
|
|
805
837
|
...summary ? { summary } : {},
|
|
@@ -844,7 +876,7 @@ function createSearxngProvider(opts) {
|
|
|
844
876
|
).catch((err) => {
|
|
845
877
|
if (err instanceof SearchError && err.status === 403) {
|
|
846
878
|
throw new SearchError(
|
|
847
|
-
"searxng
|
|
879
|
+
"searxng returned 403 \u2014 most likely the instance has JSON output disabled (settings.yml needs json under search.formats)",
|
|
848
880
|
403
|
|
849
881
|
);
|
|
850
882
|
}
|
|
@@ -864,7 +896,9 @@ function createSearxngProvider(opts) {
|
|
|
864
896
|
}
|
|
865
897
|
const ignored = [];
|
|
866
898
|
if (options.recencyDays !== void 0 && !range) {
|
|
867
|
-
ignored.push(
|
|
899
|
+
ignored.push(
|
|
900
|
+
"recencyDays (searxng only supports day/week/month/year buckets; larger day counts were ignored)"
|
|
901
|
+
);
|
|
868
902
|
}
|
|
869
903
|
return { results, ...ignored.length > 0 ? { ignored } : {} };
|
|
870
904
|
}
|
|
@@ -991,17 +1025,21 @@ function selectSearchProvider(config, native) {
|
|
|
991
1025
|
var webPlugin = {
|
|
992
1026
|
name: "@epoch-agent/plugin-web",
|
|
993
1027
|
version: "0.0.0",
|
|
994
|
-
description: "Web
|
|
1028
|
+
description: "Web fetching and searching \u2014 URL safety checks, HTML to Markdown, SSRF protection",
|
|
995
1029
|
tools: [
|
|
996
1030
|
{
|
|
997
1031
|
name: "web_fetch",
|
|
998
|
-
description: "
|
|
1032
|
+
description: "Fetch a web page and return its content. HTML is converted to Markdown and script/style are stripped. Fails on private and loopback addresses (SSRF protection).",
|
|
999
1033
|
parameters: {
|
|
1000
1034
|
type: "object",
|
|
1001
1035
|
properties: {
|
|
1002
|
-
url: { type: "string", description: "
|
|
1003
|
-
format: {
|
|
1004
|
-
|
|
1036
|
+
url: { type: "string", description: "Page URL. Must start with http:// or https://." },
|
|
1037
|
+
format: {
|
|
1038
|
+
type: "string",
|
|
1039
|
+
enum: ["markdown", "text", "html"],
|
|
1040
|
+
description: "What to return (default markdown)."
|
|
1041
|
+
},
|
|
1042
|
+
timeout: { type: "number", description: "Timeout in seconds (default 30, max 120)." }
|
|
1005
1043
|
},
|
|
1006
1044
|
required: ["url"]
|
|
1007
1045
|
},
|
|
@@ -1015,19 +1053,29 @@ var webPlugin = {
|
|
|
1015
1053
|
function createWebSearchTool(provider, opts = {}) {
|
|
1016
1054
|
return {
|
|
1017
1055
|
name: "web_search",
|
|
1018
|
-
description:
|
|
1056
|
+
description: `Search the web with ${provider.name}. **Returns titles, URLs and snippets only \u2014 not page contents.** Read the snippets, decide which result is worth opening, then call web_fetch on that URL.`,
|
|
1019
1057
|
parameters: {
|
|
1020
1058
|
type: "object",
|
|
1021
1059
|
properties: {
|
|
1022
|
-
query: { type: "string", description: "
|
|
1023
|
-
maxResults: {
|
|
1024
|
-
|
|
1060
|
+
query: { type: "string", description: "Search query." },
|
|
1061
|
+
maxResults: {
|
|
1062
|
+
type: "number",
|
|
1063
|
+
description: "How many results to return (default 5, max 20)."
|
|
1064
|
+
},
|
|
1065
|
+
recencyDays: {
|
|
1066
|
+
type: "number",
|
|
1067
|
+
description: "Only results published within the last N days."
|
|
1068
|
+
},
|
|
1025
1069
|
domains: {
|
|
1026
1070
|
type: "array",
|
|
1027
1071
|
items: { type: "string" },
|
|
1028
|
-
description: '
|
|
1072
|
+
description: 'Only search within these domains, e.g. ["nodejs.org"].'
|
|
1029
1073
|
},
|
|
1030
|
-
excludeDomains: {
|
|
1074
|
+
excludeDomains: {
|
|
1075
|
+
type: "array",
|
|
1076
|
+
items: { type: "string" },
|
|
1077
|
+
description: "Exclude these domains."
|
|
1078
|
+
}
|
|
1031
1079
|
},
|
|
1032
1080
|
required: ["query"]
|
|
1033
1081
|
},
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@epoch-agent/plugin-web",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "4.0.0",
|
|
4
4
|
"private": false,
|
|
5
5
|
"description": "Web 抓取插件 — URL 安全校验 + HTML→Markdown + SSRF 防护",
|
|
6
6
|
"repository": {
|
|
@@ -22,8 +22,8 @@
|
|
|
22
22
|
"dist"
|
|
23
23
|
],
|
|
24
24
|
"dependencies": {
|
|
25
|
-
"@epoch-agent/infra": "
|
|
26
|
-
"@epoch-agent/protocol": "
|
|
25
|
+
"@epoch-agent/infra": "4.0.0",
|
|
26
|
+
"@epoch-agent/protocol": "4.0.0"
|
|
27
27
|
},
|
|
28
28
|
"devDependencies": {
|
|
29
29
|
"@types/node": "^22.0.0",
|