dsh-free-search 0.7.5 → 0.8.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -34,17 +34,19 @@ dsh 默认的搜索 provider 依赖 DeepSeek 官方 API key(`DEEPSEEK_API_KEY`
34
34
  - **弹出式切换命令** —— 聊天框输入 `/free-search-engine`,弹出引擎选择窗口,点选即切换(等效设置页 + 保存)
35
35
  - **引擎测试** —— `free_search_test` 工具让 agent 一键测试所有引擎;设置页也有"测试引擎"按钮(直测当前引擎,不走回退链,付费引擎无 key 会明确报错)
36
36
  - **全局引擎开关与回退优先级** —— 设置页可勾选/取消引擎(取消后全局生效:普通搜索 / Auto / `advanced_search` / `multi_search` / 引擎测试都不再用它),并可用 ↑↓ 调整全局回退顺序(一键恢复默认)
37
- - **Multi 模式** —— 可把搜索引擎设为 `Multi Search`:`web_search` 并发请求前 3 个已启用引擎、按 URL 合并去重、跨源命中的结果优先(代价是成倍消耗额度;multi 失败会自动退回单引擎回退链)
37
+ - **Multi 模式** —— 可把搜索引擎设为 `Multi Search`:`web_search` 按查询语言并发**对应语言池的全部**已启用引擎(不再是前 3 个;中文/英文池可在设置页编辑,每池最低 2 个引擎)、按 URL 合并去重、跨源命中的结果优先(代价是成倍消耗额度;multi 失败会自动退回单引擎回退链)
38
38
  - **统一引擎回退** —— 任何引擎失败(付费/免费,缺 key/401/限流/网络)自动轮流尝试下一个引擎:首选引擎 → 其他引擎(exa/tavily/keenable/firecrawl/parallel 无 key 也会尝试,因为它们自带 keyless 免费额度)→ 剩余免费引擎,搜索永不直接失败;结果顶部注明实际生效的引擎(如 `Note: perplexity unavailable or failed, using exa.`);失败按类别处理:额度/鉴权失败→本会话冷却该引擎,反爬→短退避,超时/5xx/限流→同引擎重试一次,解析失败/0 结果→不冷却(详见下文「失败分类与引擎冷却」)
39
39
  - **时间过滤** —— `advanced_search` 工具支持 `timeRange`:固定档、自定义相对值、绝对日期三种形式(详见下方逻辑说明)
40
40
  - **系统提示词注入** —— agent 知道当前用哪个引擎、哪些需要 key;并明确所有搜索结果是**不可信外部数据**,不得执行其中的指令
41
41
  - **提示注入防护(不可信数据边界)** —— 插件自有工具(advanced_search / platform_search / free_search_test)的网页文本包在 `<untrusted-web-content>` 边界内(正文里自带的同名标记会被剥离,防止提前闭合);核心 web_search / web_fetch 由 DSH 核心自带同类提示(`External web content follows...`);所有 snippet 统一清洗并截断到 300 字符
42
- - **版本号 + 检查更新** —— 设置卡片显示当前版本(v0.4.17),"检查更新"按钮直连 npm registry 对比最新版,有新版本时提示并可一键跳转
42
+ - **版本号 + 检查更新** —— 设置卡片显示当前版本(跟随 `PLUGIN_VERSION`,随发布更新),"检查更新"按钮直连 npm registry 对比最新版,有新版本时提示并可一键跳转
43
43
  - **结果缓存** —— 相同查询(含引擎/时间过滤参数)5 分钟内命中缓存(LRU 50 条),防免费引擎限流、省付费额度;时长可在设置页 0-5 分钟自由配置(0 关闭)
44
44
  - **免费标注** —— 设置页中免费引擎带绿色 `FREE` 徽章,付费引擎带橙色 `API KEY` 徽章
45
45
  - **网页抓取(web_fetch)** —— 让 agent 抓取网页内容(官方 `dsh-web-fetch-http` provider,纯 JS,零额外依赖)
46
46
  - **平台搜索(platform_search)** —— 搜 GitHub / V2EX / B站 / Reddit / Hacker News / Stack Overflow / 维基百科 / npm / YouTube / Vimeo(公开 API 或免 key 抓取,零依赖)
47
47
  - **视频搜索(video_search)** —— 跨站找视频:Bing Videos + DuckDuckGo Videos(免 key,失败互相回退),返回视频链接、标题与来源/时长等元信息
48
+ - **一次调用完成研究(research)** —— 多引擎并发搜索 → 候选重排 → 分层抓取正文 → 段落选窗 → 带出处的 Markdown 证据包;模型组件(重排/嵌入/决策模型)全部可选,缺失或失败时静默回退规则路径
49
+ - **检索正文缓存(cache_search)** —— 全文检索 `research` 抓过的正文(`node:sqlite` 持久缓存,存储 7 天;`node:sqlite` 不可用时自动降级为 no-op,免 flag 需 ≥ 22.13,22.5–22.12 需 `--experimental-sqlite`)
48
50
  - **干净集成** —— 实现官方 `WebSearchProvider` seam 接口,与官方插件共存
49
51
 
50
52
  如果这个插件帮到了你,欢迎给仓库点个 ⭐([GitHub](https://github.com/DDDMUC/dsh-free-search))——星标是开发者继续维护的最大动力,感谢支持!
@@ -70,9 +72,12 @@ dsh 默认的搜索 provider 依赖 DeepSeek 官方 API key(`DEEPSEEK_API_KEY`
70
72
  | `deepseek-official` | DeepSeek 官方 | 付费 | 需 `DEEPSEEK_API_KEY` |
71
73
  | `you` | You.com | 付费 | 需 `YOUCOM_API_KEY`(you.com/platform/api-keys,注册即送免费额度) |
72
74
  | `baidu` | 百度千帆 AI 搜索 | 额度 | 需 `BAIDU_API_KEY`(千帆 AI 搜索,每日赠 50 次,超额按量后付费);中文全网,支持时间过滤 |
75
+ | `kimi` | Kimi(Moonshot)联网搜索 | 付费 | 需 `MOONSHOT_API_KEY`(basic 约 ¥0.01/次),返回带正文 chunks 的中文结果;**无免费额度**,适合显式选用或作质量升级池,不建议放进默认并发池 |
73
76
  | `kimi` | Kimi(Moonshot)联网搜索 | 付费 | 需 `MOONSHOT_API_KEY`(basic 约 ¥0.01/次),返回带正文 chunks 的中文结果 |
74
77
  | `aliyun` | 阿里云百炼 EnhancedSearch | 付费 | 需 `DASHSCOPE_API_KEY`(MCP search_pro,约 ¥0.03/次,新用户 200 次免费包);中文全网带来源 hostname |
75
78
  | `doubao` | 火山引擎联网搜索(豆包搜索) | 免费额度 | 需 `DOUBAO_SEARCH_API_KEY`(联网搜索控制台开通,**每月 500 次免费**,超出按量付费);中文时效内容强,支持时间/站点过滤,返回千字级摘要 |
79
+ | `zhihu_global` | 知乎全网搜索(开放平台 MCP) | 免费额度 | 需 `ZHIHU_API_KEY`;**每日 5,000 次试用额度**(试用结束需另行开通);支持时间过滤(`publish_time`);无 key 时自动跳过 |
80
+ | `zhihu_site` | 知乎站内搜索(开放平台 MCP) | 免费额度 | 需 `ZHIHU_API_KEY`;**每日 5,000 次试用额度**(与 `zhihu_global` 共享);不支持时间过滤 |
76
81
  | `zhihu_global` | 知乎全网搜索(开放平台 MCP) | 付费 | 需 `ZHIHU_API_KEY`;支持时间过滤(`publish_time`);无 key 时自动跳过 |
77
82
  | `zhihu_site` | 知乎站内搜索(开放平台 MCP) | 付费 | 需 `ZHIHU_API_KEY`;不支持时间过滤 |
78
83
  | `openai` | OpenAI 模型内置搜索 | 付费 | 需 `OPENAI_API_KEY`;走 Responses API 的 `web_search` 工具,返回带引用的回答。**按次计费,仅在显式选中时使用**(不参与自动回退/auto);默认 `gpt-6-luna`,模型与端点可配(`openaiModel` / `openaiBaseUrl`) |
@@ -93,6 +98,20 @@ dsh 默认的搜索 provider 依赖 DeepSeek 官方 API key(`DEEPSEEK_API_KEY`
93
98
  - 豆包搜索(火山联网搜索):<https://console.volcengine.com/search-infinity/web-search>
94
99
  - You.com:<https://you.com/platform/api-keys>
95
100
 
101
+ #### 引擎额度分层
102
+
103
+ 「免费 / 付费」二分不足以决定**谁该进默认池**——真正的约束是额度模型:有没有免费量、有多少、用超了怎么算。按这个口径分五档(配置并发池时按这档选,能避免整池并发时额度翻倍):
104
+
105
+ | 档位 | 引擎 | 超出后 |
106
+ |---|---|---|
107
+ | ① 无限免 key | `ddg` / `ddg-lite` / `bing` / `searxng` / `anysearch` | — |
108
+ | ② 匿名额度(不配 key 也能跑) | `exa` / `tavily` / `keenable` / `firecrawl` / `parallel` | 配 key 提额 |
109
+ | ③ 免费额度(需 key) | `zhihu_global` / `zhihu_site`(每日 5,000 次试用)、`doubao`(每月 500 次)、`baidu`(每日 50 次)、`aliyun`(新用户 200 次包) | 按量后付费 |
110
+ | ④ 付费升级(需 key 才有产出) | `serpbase`(注册送 100 次)/ `serply`(30 天 2,500 次)/ `you` / `perplexity` / `deepseek-official` / `kimi` | 按量 / 按次 |
111
+ | ⑤ 按次计费(模型内置,显式-only) | `openai` / `gemini` / `claude` | 按 token + 每次搜索 |
112
+
113
+ 选默认池的经验:**① 打底 + ③ 打主力**;② 可以进但要知道它在烧匿名额度;④ 只在你确实需要该源时显式选用;⑤ 永不入池。完整的逐引擎额度事实与默认池提案见 [docs/engine-quota-tiers.md](./docs/engine-quota-tiers.md)。
114
+
96
115
  #### 为什么免费引擎不需要 key?
97
116
 
98
117
  - **AnySearch**:其 `v1/search` REST 接口提供匿名的公共搜索额度,无需注册或 API key。额度有限流(适合日常搜索),但作为免费引擎之一,与其他免费引擎互相回退,体验稳定。
@@ -173,7 +192,7 @@ Error: configured web provider "ddg" is not registered
173
192
  - **推荐**:付费引擎 key 建议写入 harness 凭据中心 `~/.dsh/.credentials.yaml`(如 `DEEPSEEK_API_KEY: sk-...`,与官方 LLM provider 一致,一处管理所有 key)。插件读取优先级:凭据中心 > 设置页 > 环境变量,设置页填的 key 仅作为遗留兼容。
174
193
  - **Test engine**:直测当前引擎可用性(不走回退链,付费引擎无 key 会明确报错)
175
194
  - **Use Bing default**:把当前搜索引擎切回稳定的免费 Bing;`Discard` 只撤销尚未保存的编辑
176
- - **Platform search**:勾选启用 GitHub / V2EX / Bilibili / YouTube / Vimeo 平台搜索(`platform_search` 工具按此过滤)
195
+ - **Platform search**:勾选启用 GitHub / V2EX / Bilibili / Reddit / Hacker News / Stack Overflow / 维基百科 / npm / YouTube / Vimeo 平台搜索(`platform_search` 工具按此过滤)
177
196
  - **EN / 中文**:切换界面语言(默认中文)
178
197
 
179
198
  <table align="center" style="border: none; border-collapse: collapse;">
@@ -203,11 +222,12 @@ Error: configured web provider "ddg" is not registered
203
222
 
204
223
  #### 引擎开关、回退优先级与 Multi 模式
205
224
 
206
- 设置页有三个新块(都会保存进条目 config):
225
+ 设置页有这些块(都会保存进条目 config):
207
226
 
208
227
  - **全局启用的搜索引擎**:取消勾选后,该引擎会从普通 `web_search` 回退链、Auto 智能路由、`advanced_search`、`multi_search` 和引擎测试中**全局排除**;至少要保留一个引擎。
209
228
  - **全局回退优先级**:用 ↑↓ 调整先后顺序。首选引擎仍先尝试;Auto 保留"语言/时间"路由规则,但自定义后同一分组内及后续回退按此顺序。被禁用的引擎保留在列表里(标记「已禁用」)但不执行;「恢复默认顺序」一键还原。
210
- - **搜索引擎下拉里的 `Multi Search`**:选中后 `web_search` 会并发查询路由/优先级前 3 个已启用引擎,URL 去重合并、跨源命中的结果排前面。注意并发会成倍消耗额度;Multi 失败时会自动退回普通单引擎回退链并在结果里注明。
229
+ - **中文查询池(zhPool)/ 英文查询池(enPool)**:Auto 路由的起始组,也是 Multi 模式的并发池。按查询语言自动二选一(检测到中日韩文字走中文池,其余走英文池);两池都可用 ↑↓ 排序、下拉添加引擎、「−」移除(**每池最低 2 个引擎,无上限**)。默认中文池 `bing / baidu / aliyun / anysearch`,默认英文池 `bing / exa / tavily`;禁用/冷却/缺 key 的引擎运行时自动跳过。
230
+ - **搜索引擎下拉里的 `Multi Search`**:选中后 `web_search` 会并发查询**查询语言对应池的全部**已启用引擎(不再是前 3 个),URL 去重合并、跨源命中的结果排前面。注意整池并发会成倍消耗额度(池里含按额度计费引擎时尤其注意);Multi 失败时会自动退回普通单引擎回退链并在结果里注明。
211
231
 
212
232
  #### 失败分类与引擎冷却(Failure-aware fallback)
213
233
 
@@ -233,7 +253,7 @@ DSH 0.1.7-rc.1 起,配置跟随 profile 的插件条目保存:设置页与 `
233
253
 
234
254
  ```yaml
235
255
  # profiles/<profile>/cordis.patch.yml 中该条目的 config:
236
- provider: bing # ddg / ddg-lite / bing / searxng / anysearch / exa / tavily / keenable / firecrawl / parallel / perplexity / serpbase / serply / deepseek-official / you / baidu / kimi / aliyun / doubao / openai / gemini / claude / auto / multi
256
+ provider: bing # ddg / ddg-lite / bing / searxng / anysearch / exa / tavily / keenable / firecrawl / parallel / perplexity / serpbase / serply / deepseek-official / you / baidu / kimi / aliyun / doubao / zhihu_global / zhihu_site / openai / gemini / claude / auto / multi
237
257
  fallbackOn: [quota, auth, bot-wall, transient, invalid-response, unknown] # 允许触发回退的失败类别;[] = 任何失败都立即中止(默认全部)
238
258
  lang: zh # 设置页界面语言(zh / en)
239
259
  bingMarket: zh-CN # Bing 市场
@@ -297,7 +317,7 @@ Search engine test:
297
317
 
298
318
  当需要对重要问题做**多源交叉验证**、避免单一引擎偏差或单源死锁时,可以让 agent 调用 `multi_search` 工具:
299
319
 
300
- - **并发请求**:默认基于当前查询类型并发请求前 3 个优选引擎(或显式传入 `engines` 列表),各引擎独立解析 API Key 与容错(缺 key 引擎自动跳过,不阻断其他引擎)。
320
+ - **并发请求**:默认按查询语言并发**对应语言池的全部**引擎(不再是前 3 个;或显式传入 `engines` 列表),各引擎独立解析 API Key 与容错(缺 key 引擎自动跳过,不阻断其他引擎)。
301
321
  - **去重与合并**:按规范化 URL 去除结尾斜杠并合并结果,多引擎共同命中的条目优先置顶排在最前,并在结果附带 `seenIn` 命中来源清单(如 `[seen in: bing, exa]`)。
302
322
  - **不可信边界与清洗**:严格遵守 `<untrusted-web-content>` 数据边界,正文 snippet 统一清洗。
303
323
  - ⚠️ 注:多源并发会消耗更多 API 配额,建议在需要多角度核验时按需使用。
@@ -340,6 +360,35 @@ Search engine test:
340
360
 
341
361
  默认两个源都试、失败互相回退,返回 `url / title / snippet`(来源站点、时长等)。**免 key 抓取,对方改版可能失效**;要按站点搜(YouTube / Vimeo / B站)请用 `platform_search`。
342
362
 
363
+ #### 一次调用完成研究(research)
364
+
365
+ `research` 让 agent 一次调用拿到带出处的证据包:多引擎并发搜索 → 候选重排 → 分层抓取正文 → 段落选窗 → Markdown 简报。适合"需要多源核验的问题",可替代 agent 自己串联 `web_search` + `web_fetch`。
366
+
367
+ - **参数**:`question`(问题原文)、`depth`(抓取正文的篇数,1–5,默认 3;自动取整并夹到该区间)。
368
+ - **输出**:`sources` 检索结果、`documents` 抓到的正文(每条含 `kind`、`text_origin`、`tier`、`published`、`fetched_at`、`engine_hits` 等字段)、`notes` 执行注记、`estimatedTokens` 粗估 token。
369
+ - **共享预算**:全阶段共用 90 秒截止(`DEADLINE_MS`)与 12 次搜索调用上限(`SEARCH_REQUESTS_MAX`);到点取消在途请求、排队任务不再启动,并返回已拿到的部分结果,在 `notes` 里记 `deadline_hit` 或 `budget_exhausted`。
370
+ - **降级**:重排、嵌入、决策模型任一失败即静默回退规则路径,不影响返回结果;抓取失败时退化为只用检索摘要(`snippets only`)。
371
+ - **不可信边界**:返回的简报包在 `<untrusted-web-content>` 内。
372
+
373
+ 无 key 时的降级行为(**基本路径仍可用,不会因缺 key 中断**):
374
+
375
+ | 未配置的 key | 影响 |
376
+ |---|---|
377
+ | `SILICONFLOW_RERANK_API_KEY` | 跳过候选重排与段落模型打分,改用轻筛排序;抓取前语义簇标记也跳过。`notes` 记录降级路径 |
378
+ | `DECISION_MODEL_API_KEY` / `DECISION_MODEL_URL` | 决策模型整体关闭(`decide()` 直接返回 null),意图判定、充分性闸门、改写挑选全部走规则版。两者需**同时**配置才生效 |
379
+ | `EXA_API_KEY` / `FIRECRAWL_API_KEY` / `CRAWL4AI_API_KEY` | 分层抓取自动跳过对应层,其余层照常 |
380
+
381
+ > 决策模型端点默认**关闭**(`DECISION_MODEL_URL` 默认为空串):不配置就不会外呼第三方端点。需要时自行指定 `DECISION_MODEL_URL`。
382
+
383
+ #### 检索正文缓存(cache_search)
384
+
385
+ `cache_search` 全文检索 `research` 抓过的正文,用来"回头看之前查到什么",不必重新抓取。
386
+
387
+ - **参数**:`query`(检索词)、`limit`(返回条数上限,默认 8,最大 20)。
388
+ - **返回**:`url` / `title` / `excerpt` / `fetched_at`,同样包在 `<untrusted-web-content>` 内。
389
+ - **存储位置与清理**:默认写入 `~/.cache/dsh-free-search/research.db`,可用环境变量 `RESEARCH_CACHE_DIR` 换成别的目录(指向备份盘、或想随项目走时用)。**清理方式**:删除该目录即可,没有后台任务残留、没有注册表项。缓存 7 天后过期,查询时惰性清理,过期行不占检索结果。
390
+ - **`node:sqlite` 不可用时降级**:无法加载 `node:sqlite` 时缓存整体禁用(no-op)——`research` 正常运行,`cache_search` 返回空数组,插件不会因此报错。`node:sqlite` 自 v22.13.0 / v23.4.0 起免 flag;22.5–22.12 需以 `--experimental-sqlite` 启动,否则同样走 no-op。
391
+
343
392
  ### 本地引擎切换工具(tools/)
344
393
 
345
394
  `tools/` 目录附带了一个本地切换小工具(零依赖):
@@ -351,6 +400,21 @@ Search engine test:
351
400
 
352
401
  切换后重启 `dsh web` 生效。
353
402
 
403
+ #### 离线自检断言(tools/)
404
+
405
+ 仓库自带 6 个零依赖断言脚本,**不需要 API key、不联网、不需要测试框架**,Node ≥ 20 即可跑:
406
+
407
+ | 脚本 | 覆盖 | 运行 |
408
+ |---|---|---|
409
+ | `assert-engines.mjs` | 引擎清单 / 免费引擎口径 / 语言分池路由 / 池配置字段 / 知乎 URL 剥 utm | `node tools/assert-engines.mjs`(19 项) |
410
+ | `assert-research-degradation.mjs` | research 无 key 全降级、`node:sqlite` no-op、90s 预算隔离、缓存目录可配置、`budget.exhausted` 契约名 | `node tools/assert-research-degradation.mjs`(19 项) |
411
+ | `assert-fetch-chain.mjs` | `fs-quality-fetch` 四层链序、任一层命中即停、exa 失败静默降级 firecrawl、全层失败聚合报错 | `node tools/assert-fetch-chain.mjs`(8 项) |
412
+ | `assert-vertical.mjs` | `domain_search` 37 引擎注册表、每引擎 desc、未知引擎报错、不依赖 research | `node tools/assert-vertical.mjs`(8 项) |
413
+ | `assert-free-search-test-schema.mjs` | `free_search_test` 输出 schema 与真实产出一致(含 auto/multi 虚拟模式) | `node tools/assert-free-search-test-schema.mjs`(8 项) |
414
+ | `assert-single-fetch-layer.mjs` | 抓取层全库只有一份实现、依赖单向、替身键不串层 | `node tools/assert-single-fetch-layer.mjs`(22 项) |
415
+
416
+ 任何一个失败会以非 0 退出码结束并列出失败项;把某个旧版本文件路径作为参数传进去,可以看到修复前必红(用来证明断言不是空转)。改动发布相关逻辑后建议先跑一遍再发版。
417
+
354
418
  > 配置卡片挂在左侧「插件」页的 `plugins.row.config` 行配置插槽(dsh 自带),配置读写走插件自建 bridge,**不依赖 dsh-web-ui**,插件可独立使用。
355
419
 
356
420
  ### 代理说明(国内用户)
@@ -401,17 +465,19 @@ This plugin provides multiple free search engines with automatic fallback, compl
401
465
  - **Popup Switch Command** — Type `/free-search-engine` in the chat: a picker opens with all engines; click one to switch (equivalent to the settings page + save)
402
466
  - **Engine Testing** — `free_search_test` for the agent to check all engines in one call; the settings UI also has a "Test engine" button that tests the selected engine directly (no fallback chain; paid engines without a key report an explicit error)
403
467
  - **Global engine enable/disable & fallback priority** — uncheck engines you never want (applies everywhere: web_search fallback, Auto routing, advanced_search, multi_search, engine tests) and reorder the global fallback chain with ↑↓ (one-click reset)
404
- - **Multi mode** — pick `Multi Search` as the engine: web_search queries the top 3 enabled engines concurrently, merges/deduplicates URLs and prioritizes cross-source hits (costs more quota; a failed multi run automatically falls back to the single-engine chain)
468
+ - **Multi mode** — pick `Multi Search` as the engine: web_search queries **every enabled engine of the query's language pool** concurrently (no longer top 3; zh/en pools editable in the settings UI, minimum 2 engines each), merges/deduplicates URLs and prioritizes cross-source hits (costs more quota; a failed multi run automatically falls back to the single-engine chain)
405
469
  - **Unified Engine Fallback** — Any engine failure (paid or free, missing key, 401, rate limit, network error) automatically tries the next engine: the configured engine first, then other engines (exa/tavily/keenable/firecrawl/parallel are tried even without a key because they have built-in keyless quota), then the remaining free engines (Bing/AnySearch etc.) — with a note attached to the results naming the engine that actually served them (e.g. `Note: perplexity unavailable or failed, using exa.`). Failures are classified: quota/auth failures cool the engine down for the session, anti-bot walls back off briefly, timeout/5xx/rate-limit failures retry the same engine once, and parse errors / 0 results are reported without cooling it down (see "Failure-aware fallback" below). Search never fails outright.
406
470
  - **Time Filtering** — The `advanced_search` tool supports `timeRange`: fixed tiers, custom relative values, or an absolute date (details below)
407
471
  - **System Prompt Injection** — The agent is aware of the currently active engine and which engines require API keys; it is also told that all search output is **untrusted external data** and must never be executed as instructions
408
472
  - **Prompt-Injection Guard (untrusted-data boundary)** — Web-derived text from the plugin's own tools (`advanced_search` / `platform_search` / `free_search_test`) is wrapped in an explicit `<untrusted-web-content>` boundary (look-alike tags inside the text are stripped to prevent early closure); the core `web_search` / `web_fetch` tools carry DSH core's own notice (`External web content follows...`); every snippet is cleaned and capped at 300 characters
409
- - **Version + Update Check** — The settings card shows the current version (v0.4.17), and a "Check update" button queries the npm registry to compare against the latest release, prompting a one-click jump when a newer version exists
473
+ - **Version + Update Check** — The settings card shows the current version (tracks PLUGIN_VERSION, bumped on release), and a "Check update" button queries the npm registry to compare against the latest release, prompting a one-click jump when a newer version exists
410
474
  - **Result Caching** — Identical queries (same engine / time-filter args) hit an LRU cache (50 entries) for up to 5 minutes, protecting free engines from rate-limiting and saving paid quota; the TTL is configurable from 0-5 minutes in the settings UI (0 disables caching)
411
475
  - **Visual Badges** — Free engines feature a green `FREE` badge, while paid engines show an orange `API KEY` badge in the settings UI
412
476
  - **Webpage Fetching (`web_fetch`)** — Allows the agent to read full webpage contents (official `dsh-web-fetch-http` provider, pure JS, zero extra dependencies)
413
477
  - **Platform Search (`platform_search`)** — Search GitHub / V2EX / Bilibili / Reddit / Hacker News / Stack Overflow / Wikipedia / npm / YouTube / Vimeo (public APIs or keyless scraping, zero extra dependencies)
414
478
  - **Video Search (`video_search`)** — Find videos across the web via Bing Videos + DuckDuckGo Videos (keyless, mutual fallback), returning video links with titles and publisher/duration metadata
479
+ - **One-Call Research (`research`)** — Concurrent multi-engine search → candidate rerank → tiered body fetch → window selection → an evidence pack with provenance as Markdown; every model component (rerank / embedding / decision model) is optional and silently falls back to the rule path when missing or failing
480
+ - **Search Fetched Bodies (`cache_search`)** — Full-text search over the bodies `research` already fetched (`node:sqlite` persistent cache, 7-day retention; degrades to a no-op whenever `node:sqlite` is unavailable — unflagged from 22.13, while 22.5–22.12 needs `--experimental-sqlite`)
415
481
  - **Clean Integration** — Implements the official `WebSearchProvider` seam interface, coexisting seamlessly with official plugins
416
482
 
417
483
  If this plugin has been helpful, a ⭐ on [GitHub](https://github.com/DDDMUC/dsh-free-search) would mean a lot — it's the biggest motivation for the developer to keep maintaining it. Thank you!
@@ -437,9 +503,12 @@ If this plugin has been helpful, a ⭐ on [GitHub](https://github.com/DDDMUC/dsh
437
503
  | `deepseek-official` | DeepSeek Official | Paid | Requires `DEEPSEEK_API_KEY` |
438
504
  | `you` | You.com | Paid | Requires `YOUCOM_API_KEY` (you.com/platform/api-keys, free tier on signup) |
439
505
  | `baidu` | Baidu Qianfan AI Search | Quota | Requires `BAIDU_API_KEY` (Qianfan AI Search, 50 free calls/day then pay-as-you-go); Chinese web-wide, supports time filtering |
506
+ | `kimi` | Kimi (Moonshot) Web Search | Paid | Requires `MOONSHOT_API_KEY` (basic ~¥0.01/call); returns Chinese results with body chunks; **no free quota**, so prefer explicit selection or the escalation pool over the default concurrent pool |
440
507
  | `kimi` | Kimi (Moonshot) Web Search | Paid | Requires `MOONSHOT_API_KEY` (basic ~¥0.01/call); returns Chinese results with body chunks |
441
508
  | `aliyun` | Aliyun Bailian EnhancedSearch | Paid | Requires `DASHSCOPE_API_KEY` (MCP search_pro, ~¥0.03/call, 200 free calls for new users); Chinese web-wide with source hostnames |
442
509
  | `doubao` | Volcano Web Search (Doubao) | Free quota | Requires `DOUBAO_SEARCH_API_KEY` (open it in the Web Search console; **500 free queries/month**, pay-as-you-go beyond); strong for fresh Chinese content, supports time/site filters, returns long summaries |
510
+ | `zhihu_global` | Zhihu web-wide search (open-platform MCP) | Free quota | Requires `ZHIHU_API_KEY`; **5,000 calls/day trial quota** (paid access after the trial); supports the time filter (`publish_time`); skipped automatically when no key is set |
511
+ | `zhihu_site` | Zhihu site search (open-platform MCP) | Free quota | Requires `ZHIHU_API_KEY`; **5,000 calls/day trial quota** (shared with `zhihu_global`); time filtering not offered |
443
512
  | `zhihu_global` | Zhihu web-wide search (open-platform MCP) | Paid | Requires `ZHIHU_API_KEY`; supports the time filter (`publish_time`); skipped automatically when no key is set |
444
513
  | `zhihu_site` | Zhihu site search (open-platform MCP) | Paid | Requires `ZHIHU_API_KEY`; time filtering not verified, so it is not offered |
445
514
  | `openai` | OpenAI built-in web search | Paid | Requires `OPENAI_API_KEY`; uses the Responses API `web_search` tool and returns a cited answer. **Billed per search, explicit selection only** (never picked by the fallback chain or Auto); default `gpt-6-luna`, model/endpoint configurable (`openaiModel` / `openaiBaseUrl`) |
@@ -460,8 +529,23 @@ If this plugin has been helpful, a ⭐ on [GitHub](https://github.com/DDDMUC/dsh
460
529
  - Doubao / Volcano Web Search: <https://console.volcengine.com/search-infinity/web-search>
461
530
  - You.com: <https://you.com/platform/api-keys>
462
531
 
532
+ #### Engine quota tiers
463
533
  #### Why are some engines free?
464
534
 
535
+ The free/paid binary is not what decides **who belongs in a default pool** — the real constraint is the quota model: is there a free allowance, how much, and what happens past it. Five tiers (pick concurrent pools from these, so a full-pool run does not silently multiply quota):
536
+
537
+ | Tier | Engines | Past the allowance |
538
+ |---|---|---|
539
+ | ① Unlimited, no key | `ddg` / `ddg-lite` / `bing` / `searxng` / `anysearch` | — |
540
+ | ② Anonymous quota (runs without a key) | `exa` / `tavily` / `keenable` / `firecrawl` / `parallel` | key raises the quota |
541
+ | ③ Free quota (key required) | `zhihu_global` / `zhihu_site` (5,000/day trial), `doubao` (500/month), `baidu` (50/day), `aliyun` (200-call starter pack) | pay-as-you-go |
542
+ | ④ Paid upgrade (needs a key to return anything) | `serpbase` (100 free on signup) / `serply` (2,500 for 30 days) / `you` / `perplexity` / `deepseek-official` / `kimi` | per call / per query |
543
+ | ⑤ Per-call billing (model built-in, explicit-only) | `openai` / `gemini` / `claude` | per token + per search |
544
+
545
+ Rule of thumb for a default pool: **① as the floor + ③ as the workhorse**; ② is allowed but remember it burns anonymous quota; ④ only when you actually want that source; ⑤ never enters a pool. Per-engine facts and a default-pool proposal live in [docs/engine-quota-tiers.md](./docs/engine-quota-tiers.md).
546
+
547
+ #### Why do some engines not need a key?
548
+
465
549
  - **AnySearch**: its `v1/search` REST endpoint provides anonymous public search quota without registration or an API key. Quota is rate-limited (fine for daily queries), but as one of the free engines with mutual fallback it stays reliable.
466
550
  - **Exa**: its public MCP endpoint (`mcp.exa.ai/mcp`) supports anonymous requests, so it works without a key; configuring `EXA_API_KEY` grants a higher usage quota.
467
551
  - **Tavily**: offers keyless anonymous quota via the `x-tavily-access-mode: keyless` header — it works without a key; configuring `TAVILY_API_KEY` switches to the account tier for higher quota and more stable results.
@@ -570,11 +654,12 @@ The command only changes the preferred engine; search still goes through `web_se
570
654
 
571
655
  #### Engine Switches, Fallback Priority and Multi Mode
572
656
 
573
- The settings page has three new blocks (all saved into the entry config):
657
+ The settings page has these blocks (all saved into the entry config):
574
658
 
575
659
  - **Globally enabled engines**: unchecking an engine excludes it everywhere — the web_search fallback chain, Auto routing, `advanced_search`, `multi_search` and engine tests. At least one engine must remain enabled.
576
660
  - **Global fallback priority**: use ↑↓ to reorder. The preferred engine is still tried first; Auto keeps its language/time routing, while a customized order governs engines inside a route group and the remaining fallback chain. Disabled engines stay in the list (marked "disabled") but are skipped; "Reset default order" restores the defaults.
577
- - **`Multi Search` in the engine dropdown**: web_search then queries the top 3 enabled routed/prioritized engines concurrently, merges/deduplicates URLs and prioritizes cross-source hits. This multiplies quota usage; if a multi run fails it automatically falls back to the single-engine chain and says so in the result note.
661
+ - **Chinese query pool (zhPool) / English query pool (enPool)**: the route head for Auto and the concurrent pool for Multi. Picked automatically by query language (CJK scripts go to the zh pool, everything else to en). Both pools support ↑↓ reordering, adding engines from a dropdown and removing with "−" (**minimum 2 engines per pool, no upper limit**). Defaults: zh `bing / baidu / aliyun / anysearch`, en `bing / exa / tavily`; disabled/cooling/keyless engines are skipped at runtime.
662
+ - **`Multi Search` in the engine dropdown**: web_search then queries **every enabled engine of the query's language pool** concurrently (no longer top 3), merges/deduplicates URLs and prioritizes cross-source hits. Running a whole pool multiplies quota usage (mind engines billed per credit); if a multi run fails it automatically falls back to the single-engine chain and says so in the result note.
578
663
 
579
664
  #### Failure-aware fallback
580
665
 
@@ -600,7 +685,7 @@ The old `free-search:` section of `~/.dsh/settings.yaml` is **not** imported by
600
685
 
601
686
  ```yaml
602
687
  # config of that entry in profiles/<profile>/cordis.patch.yml:
603
- provider: bing # ddg / ddg-lite / bing / searxng / anysearch / exa / tavily / keenable / firecrawl / parallel / perplexity / serpbase / serply / deepseek-official / you / baidu / kimi / aliyun / doubao / openai / gemini / claude / auto / multi
688
+ provider: bing # ddg / ddg-lite / bing / searxng / anysearch / exa / tavily / keenable / firecrawl / parallel / perplexity / serpbase / serply / deepseek-official / you / baidu / kimi / aliyun / doubao / zhihu_global / zhihu_site / openai / gemini / claude / auto / multi
604
689
  fallbackOn: [quota, auth, bot-wall, transient, invalid-response, unknown] # failure classes allowed to trigger fallback; [] = abort on any failure (default: all)
605
690
  lang: zh # settings UI language (zh / en)
606
691
  bingMarket: zh-CN # Bing market
@@ -664,7 +749,7 @@ Example: *"Find DSH news from the last 3 days"* → agent calls `advanced_search
664
749
 
665
750
  When you need **cross-source verification** to prevent biases or fail-safes from a single engine, the agent can call `multi_search`:
666
751
 
667
- - **Concurrent Execution**: Defaults to querying the top 3 recommended engines for the query type (or an explicit list passed via `engines`). Each engine independently resolves API keys and handles errors (missing keys are skipped without failing the batch).
752
+ - **Concurrent Execution**: Defaults to querying **every engine of the query's language pool** (no longer top 3; or an explicit list passed via `engines`). Each engine independently resolves API keys and handles errors (missing keys are skipped without failing the batch).
668
753
  - **Deduplication & Merge**: Normalizes URLs and ranks items by the number of engines that found them (`seenIn` counts), with cross-hit results placed at the top.
669
754
  - **Untrusted Boundary**: Enforces the `<untrusted-web-content>` safety wrapper and trims snippets.
670
755
  - ⚠️ Note: Running multiple engines in parallel consumes more search quota; use on demand when high source diversity is needed.
@@ -707,6 +792,35 @@ Ask the agent to find videos: *"find some videos about X"*, *"any tutorials for
707
792
 
708
793
  Both sources are tried by default with mutual fallback, returning `url / title / snippet` (publisher, duration, etc.). **Keyless scraping — may break if the site changes its markup**; to search a specific site (YouTube / Vimeo / Bilibili) use `platform_search`.
709
794
 
795
+ #### One-Call Research (`research`)
796
+
797
+ `research` gives the agent an evidence pack with provenance in a single call: concurrent multi-engine search → candidate rerank → tiered body fetch → window selection → Markdown brief. Use it when a question needs multi-source verification, instead of chaining `web_search` + `web_fetch` yourself.
798
+
799
+ - **Arguments**: `question` (the question itself) and `depth` (how many bodies to fetch in full, 1–5, default 3; rounded and clamped to that range).
800
+ - **Output**: `sources` (search hits), `documents` (fetched bodies, each with `kind`, `text_origin`, `tier`, `published`, `fetched_at`, `engine_hits` and more), `notes` (execution notes) and `estimatedTokens`.
801
+ - **Shared budget**: every stage shares a 90-second deadline (`DEADLINE_MS`) and a cap of 12 search calls (`SEARCH_REQUESTS_MAX`). When time is up, in-flight requests are cancelled, queued work does not start, and the partial results collected so far are returned with `deadline_hit` or `budget_exhausted` recorded in `notes`.
802
+ - **Degradation**: if rerank, embedding or the decision model fails, the chain silently falls back to the rule path and keeps going; when fetching fails it degrades to search snippets only (`snippets only`).
803
+ - **Untrusted boundary**: the returned brief is wrapped in `<untrusted-web-content>`.
804
+
805
+ Behavior without keys (**the basic path stays usable and never breaks over a missing key**):
806
+
807
+ | Key not configured | Effect |
808
+ |---|---|
809
+ | `SILICONFLOW_RERANK_API_KEY` | Candidate rerank and window model scoring are skipped in favor of light filtering; the pre-fetch semantic cluster tagging is skipped too. The fallback path is recorded in `notes` |
810
+ | `DECISION_MODEL_API_KEY` / `DECISION_MODEL_URL` | The decision model is fully off (`decide()` returns null immediately): intent judging, the sufficiency gate and rewrite picking all use the rule path. Both must be set for it to run |
811
+ | `EXA_API_KEY` / `FIRECRAWL_API_KEY` / `CRAWL4AI_API_KEY` | The corresponding tier of the fetch chain is skipped; the other tiers still run |
812
+
813
+ > The decision-model endpoint is **off by default** (`DECISION_MODEL_URL` defaults to an empty string): no third-party endpoint is called unless you configure one.
814
+
815
+ #### Search Fetched Bodies (`cache_search`)
816
+
817
+ `cache_search` full-text searches the bodies `research` already fetched, so you can look back at earlier findings without re-fetching.
818
+
819
+ - **Arguments**: `query` (text to look for) and `limit` (max entries, default 8, up to 20).
820
+ - **Returns**: `url` / `title` / `excerpt` / `fetched_at`, also wrapped in `<untrusted-web-content>`.
821
+ - **Where it lives, and how to clear it**: defaults to `~/.cache/dsh-free-search/research.db`; set `RESEARCH_CACHE_DIR` to relocate it (a backup drive, or wherever you want it to travel with). **To clear it**: delete that directory — there is no background job and nothing left in the registry. Entries expire after 7 days and are pruned lazily on query, so stale rows never reach your results.
822
+ - **`node:sqlite` unavailable**: when `node:sqlite` cannot be loaded, the cache is disabled entirely (no-op) — `research` still runs, `cache_search` returns an empty array, and the plugin never errors out over it. `node:sqlite` is unflagged from v22.13.0 / v23.4.0; 22.5–22.12 needs `--experimental-sqlite`, otherwise it takes the same no-op path.
823
+
710
824
  ### Local Engine Switcher (`tools/`)
711
825
 
712
826
  The `tools/` directory includes a lightweight, zero-dependency switcher:
@@ -718,6 +832,21 @@ The `tools/` directory includes a lightweight, zero-dependency switcher:
718
832
 
719
833
  Restart `dsh web` after switching to apply changes.
720
834
 
835
+ #### Offline self-check assertions (`tools/`)
836
+
837
+ Six zero-dependency assertion scripts ship with the repo: **no API keys, no network, no test framework**, Node ≥ 20:
838
+
839
+ | Script | Covers | Run |
840
+ |---|---|---|
841
+ | `assert-engines.mjs` | engine registry / free-engine rule / language-pool routing / pool config fields / Zhihu URL utm stripping | `node tools/assert-engines.mjs` (19) |
842
+ | `assert-research-degradation.mjs` | research degrades fully without keys, `node:sqlite` no-op, 90s budget isolation, configurable cache dir, `budget.exhausted` contract names | `node tools/assert-research-degradation.mjs` (19) |
843
+ | `assert-fetch-chain.mjs` | `fs-quality-fetch` tier ordering, stop-on-first-success, silent firecrawl degradation, aggregated all-tiers failure | `node tools/assert-fetch-chain.mjs` (8) |
844
+ | `assert-vertical.mjs` | `domain_search` registry of 37 engines, per-engine descriptions, unknown-engine errors, no research dependency | `node tools/assert-vertical.mjs` (8) |
845
+ | `assert-free-search-test-schema.mjs` | `free_search_test` output schema matches real output (incl. auto/multi virtual modes) | `node tools/assert-free-search-test-schema.mjs` (8) |
846
+ | `assert-single-fetch-layer.mjs` | exactly one fetch implementation repo-wide, one-way deps, seams owned by the right layer | `node tools/assert-single-fetch-layer.mjs` (22) |
847
+
848
+ Any failure exits non-zero with the failing items listed. Passing the path of an older file shows the pre-fix failures (proving the assertions are not vacuous). Worth running before a release.
849
+
721
850
  > The settings card mounts into the official `plugins.row.config` component-row slot on the sidebar Plugins page (built into DSH), and configuration reads/writes go through the plugin's own bridge. **No `dsh-web-ui` dependency — the plugin can be used standalone.**
722
851
 
723
852
  ### Proxy Note (for Users in Mainland China)
package/lib/cache.js ADDED
@@ -0,0 +1,49 @@
1
+ // research 文档持久缓存(node:sqlite,零依赖)。Node <22.5 无 node:sqlite 时整体降级为 no-op,插件照常工作。
2
+ // ponytail: cache_search 用 LIKE 而非 FTS5 MATCH——个人规模(百级行)LIKE 即时返回,且无 FTS5 查询语法转义坑;行数过万再升 FTS5 虚表。
3
+
4
+ import { createRequire } from "node:module";
5
+ import { mkdirSync } from "node:fs";
6
+ import { join } from "node:path";
7
+ import { homedir } from "node:os";
8
+
9
+ let db = null;
10
+ try {
11
+ const req = createRequire(import.meta.url);
12
+ const { DatabaseSync } = req("node:sqlite");
13
+ // 缓存位置可配置:RESEARCH_CACHE_DIR 优先(放哪、是否随备份走由用户决定),
14
+ // 默认 ~/.cache/dsh-free-search/research.db;删掉该目录即清空缓存,无后台任务残留。
15
+ const dir = process.env.RESEARCH_CACHE_DIR || join(homedir(), ".cache", "dsh-free-search");
16
+ mkdirSync(dir, { recursive: true });
17
+ db = new DatabaseSync(join(dir, "research.db"));
18
+ db.exec("CREATE TABLE IF NOT EXISTS pages(url TEXT PRIMARY KEY, title TEXT NOT NULL DEFAULT '', content TEXT NOT NULL, fetched_at INTEGER NOT NULL)");
19
+ } catch { /* Node <22.5:缓存禁用 */ }
20
+
21
+ /** 新鲜则返回 {title, content, fetched_at},否则 null。 */
22
+ export function getFresh(url, maxAgeMs) {
23
+ if (!db) return null;
24
+ const r = db.prepare("SELECT title, content, fetched_at FROM pages WHERE url = ?").get(url);
25
+ if (!r || Date.now() - r.fetched_at > maxAgeMs) return null;
26
+ return r;
27
+ }
28
+
29
+ export function put(url, title, content) {
30
+ if (!db) return;
31
+ db.prepare("DELETE FROM pages WHERE url = ?").run(url);
32
+ db.prepare("INSERT INTO pages(url, title, content, fetched_at) VALUES (?, ?, ?, ?)").run(url, title || "", content, Date.now());
33
+ }
34
+
35
+ /** 全文 LIKE 检索已缓存页面(7 天内),返回 [{url, title, excerpt, fetched_at}]。 */
36
+ const STALE_MS = 7 * 24 * 3600 * 1000; // 与 research.js 的 DOC_CACHE_MAX_AGE_MS 对齐
37
+ export function search(query, limit = 8) {
38
+ if (!db) return [];
39
+ db.prepare("DELETE FROM pages WHERE fetched_at < ?").run(Date.now() - STALE_MS); // ponytail: 查询时惰性 GC,无后台任务
40
+ const like = `%${String(query).replace(/[%_]/g, " ")}%`; // ponytail: 通配符转空格而非 ESCAPE 子句,查询词含 %/_ 属罕见
41
+ const rows = db
42
+ .prepare("SELECT url, title, content, fetched_at FROM pages WHERE (title LIKE ? OR content LIKE ?) AND fetched_at > ? ORDER BY fetched_at DESC LIMIT ?")
43
+ .all(like, like, Date.now() - STALE_MS, limit);
44
+ return rows.map((r) => {
45
+ const i = r.content.toLowerCase().indexOf(String(query).toLowerCase());
46
+ const start = Math.max(0, i - 40);
47
+ return { url: r.url, title: r.title, excerpt: (start > 0 ? "…" : "") + r.content.slice(start, start + 160).replace(/\s+/g, " "), fetched_at: new Date(r.fetched_at).toISOString() };
48
+ });
49
+ }