dsh-free-search 0.9.1 → 0.9.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +21 -15
- package/lib/client.js +6 -4
- package/lib/index.js +124 -33
- package/lib/multi_policy.js +271 -270
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -29,7 +29,7 @@ dsh 默认的搜索 provider 依赖 DeepSeek 官方 API key(`DEEPSEEK_API_KEY`
|
|
|
29
29
|
### 特性
|
|
30
30
|
|
|
31
31
|
- **零成本** —— 多个免费引擎,无需 key、无需注册
|
|
32
|
-
- **多引擎可选**:DuckDuckGo(html/lite)、Bing、SearXNG(元搜索,支持自定义实例)、AnySearch、Exa、Tavily、Keenable、Firecrawl、Parallel、Perplexity、SerpBase、Serply、DeepSeek 官方、You.com、百度千帆、Kimi、阿里云百炼、火山联网搜索(豆包搜索)、知乎全网/站内(`zhihu_global`/`zhihu_site`,经知乎开放平台 MCP)、OpenAI / Gemini / Claude 模型内置搜索
|
|
32
|
+
- **多引擎可选**:DuckDuckGo(html/lite)、Bing、SearXNG(元搜索,支持自定义实例)、AnySearch、Exa、Tavily、Keenable、Firecrawl、Parallel、Perplexity、SerpBase、Serply、DeepSeek 官方、DeepSeek 原生搜索 API、You.com、百度千帆、Kimi、阿里云百炼、火山联网搜索(豆包搜索)、知乎全网/站内(`zhihu_global`/`zhihu_site`,经知乎开放平台 MCP)、OpenAI / Gemini / Claude 模型内置搜索
|
|
33
33
|
- **网页设置页** —— 引擎切换 + API key 配置(UI 中 key 脱敏显示"已配置")+ 中英文切换;入口在左侧「插件」页的组件行配置(`plugins.row.config`,DSH 0.1.7-rc.1+)
|
|
34
34
|
- **弹出式切换命令** —— 聊天框输入 `/free-search-engine`,弹出引擎选择窗口,点选即切换(等效设置页 + 保存)
|
|
35
35
|
- **引擎测试** —— `free_search_test` 工具让 agent 一键测试所有引擎;设置页也有"测试引擎"按钮(直测当前引擎,不走回退链,付费引擎无 key 会明确报错)
|
|
@@ -71,7 +71,8 @@ dsh 默认的搜索 provider 依赖 DeepSeek 官方 API key(`DEEPSEEK_API_KEY`
|
|
|
71
71
|
| `perplexity` | Perplexity | 付费 | 需 `PERPLEXITY_API_KEY` |
|
|
72
72
|
| `serpbase` | SerpBase | 付费 | 需 `SERPBASE_API_KEY`(serpbase.dev,注册送 100 次免费额度) |
|
|
73
73
|
| `serply` | Serply | 付费 | 需 `SERPLY_API_KEY`(serply.io,新账户 30 天内 2,500 次免费额度);Google 网页结果,按设置页语言本地化 |
|
|
74
|
-
| `deepseek-official` | DeepSeek 官方 | 付费 | 需 `DEEPSEEK_API_KEY` |
|
|
74
|
+
| `deepseek-official` | DeepSeek 官方 | 付费 | 需 `DEEPSEEK_API_KEY`;走 Anthropic 兼容的 `web_search` 服务端工具,每次搜索是一整个模型轮次(慢、耗 token) |
|
|
75
|
+
| `deepseek-tools` | DeepSeek 原生搜索 API | 付费 | 需 `DEEPSEEK_API_KEY`(与上一档共用);直接 `POST /v1/tools/search`,无模型参与,返回结构化 `results[]`(`title`/`url`/`summary`/`date`),同 query 实测 ~0.3s vs 上一档 ~5-6s |
|
|
75
76
|
| `you` | You.com | 付费 | 需 `YOUCOM_API_KEY`(you.com/platform/api-keys,注册即送免费额度) |
|
|
76
77
|
| `baidu` | 百度千帆 AI 搜索 | 额度 | 需 `BAIDU_API_KEY`(千帆 AI 搜索,每日赠 50 次,超额按量后付费);中文全网,支持时间过滤 |
|
|
77
78
|
| `kimi` | Kimi(Moonshot)联网搜索 | 付费 | 需 `MOONSHOT_API_KEY`(basic 约 ¥0.01/次),返回带正文 chunks 的中文结果;**无免费额度**,适合显式选用或作质量升级池,不建议放进默认并发池 |
|
|
@@ -109,7 +110,7 @@ dsh 默认的搜索 provider 依赖 DeepSeek 官方 API key(`DEEPSEEK_API_KEY`
|
|
|
109
110
|
| ① 无限免 key | `ddg` / `ddg-lite` / `bing` / `searxng` / `anysearch` | — |
|
|
110
111
|
| ② 匿名额度(不配 key 也能跑) | `exa` / `tavily` / `keenable` / `firecrawl` / `parallel` | 配 key 提额 |
|
|
111
112
|
| ③ 免费额度(需 key) | `zhihu_global` / `zhihu_site`(每日 5,000 次试用)、`doubao`(每月 500 次)、`baidu`(每日 50 次)、`aliyun`(新用户 200 次包) | 按量后付费 |
|
|
112
|
-
| ④ 付费升级(需 key 才有产出) | `serpbase`(注册送 100 次)/ `serply`(30 天 2,500 次)/ `you` / `perplexity` / `deepseek-official` / `kimi` | 按量 / 按次 |
|
|
113
|
+
| ④ 付费升级(需 key 才有产出) | `serpbase`(注册送 100 次)/ `serply`(30 天 2,500 次)/ `you` / `perplexity` / `deepseek-official` / `deepseek-tools` / `kimi` | 按量 / 按次 |
|
|
113
114
|
| ⑤ 按次计费(模型内置,显式-only) | `openai` / `gemini` / `claude` | 按 token + 每次搜索 |
|
|
114
115
|
|
|
115
116
|
选默认池的经验:**① 打底 + ③ 打主力**;② 可以进但要知道它在烧匿名额度;④ 只在你确实需要该源时显式选用;⑤ 永不入池。完整的逐引擎额度事实与默认池提案见 [docs/engine-quota-tiers.md](./docs/engine-quota-tiers.md)。
|
|
@@ -267,7 +268,7 @@ DSH 0.1.7-rc.1 起,配置跟随 profile 的插件条目保存:设置页与 `
|
|
|
267
268
|
|
|
268
269
|
```yaml
|
|
269
270
|
# profiles/<profile>/cordis.patch.yml 中该条目的 config:
|
|
270
|
-
provider: bing # ddg / ddg-lite / bing / searxng / anysearch / exa / tavily / keenable / firecrawl / parallel / perplexity / serpbase / serply / deepseek-official / you / baidu / kimi / aliyun / doubao / zhihu_global / zhihu_site / openai / gemini / claude / auto / multi
|
|
271
|
+
provider: bing # ddg / ddg-lite / bing / searxng / anysearch / exa / tavily / keenable / firecrawl / parallel / perplexity / serpbase / serply / deepseek-official / deepseek-tools / you / baidu / kimi / aliyun / doubao / zhihu_global / zhihu_site / openai / gemini / claude / auto / multi
|
|
271
272
|
fallbackOn: [quota, auth, bot-wall, transient, invalid-response, unknown] # 允许触发回退的失败类别;[] = 任何失败都立即中止(默认全部)
|
|
272
273
|
lang: zh # 设置页界面语言(zh / en)
|
|
273
274
|
bingMarket: zh-CN # Bing 市场
|
|
@@ -326,10 +327,11 @@ Search engine test:
|
|
|
326
327
|
| SearXNG | `time_range` | ⚠️ 近似 | 同上 |
|
|
327
328
|
| DuckDuckGo / Lite | `df` | ⚠️ 近似 | 同上 |
|
|
328
329
|
| Bing / AnySearch | — | ❌ 忽略 | 无对应参数 |
|
|
330
|
+
| DeepSeek 原生搜索(`deepseek-tools`) | query 语法 `after:` | ✅ 精确 | 端点无独立时间字段,改为把 `after:YYYY-MM-DD` 拼进 query;相对天数按 N 天前折算 |
|
|
329
331
|
|
|
330
332
|
**"最近似档位"映射规则**:`≤2 天 → day`,`≤14 天 → week`,`≤90 天 → month`,否则 `year`。例如 `3d` 在 Tavily 上按 `day` 处理,`2mo` 按 `month` 处理。
|
|
331
333
|
|
|
332
|
-
**引擎链优先级**:当带 timeRange 搜索时,支持时间过滤的引擎(tavily / exa / keenable / firecrawl / parallel / searxng / ddg / ddg-lite)会排到引擎链前面,确保过滤真正生效——即使首选引擎是 bing(不支持过滤),也会先尝试支持过滤的引擎。
|
|
334
|
+
**引擎链优先级**:当带 timeRange 搜索时,支持时间过滤的引擎(tavily / exa / keenable / firecrawl / parallel / searxng / ddg / ddg-lite / deepseek-tools)会排到引擎链前面,确保过滤真正生效——即使首选引擎是 bing(不支持过滤),也会先尝试支持过滤的引擎。
|
|
333
335
|
|
|
334
336
|
示例对话:*"帮我搜最近 3 天关于 DSH 的新闻"* → agent 调用 `advanced_search`,`timeRange: "3d"`。
|
|
335
337
|
|
|
@@ -434,13 +436,13 @@ Search engine test:
|
|
|
434
436
|
|
|
435
437
|
#### 离线自检断言(tools/)
|
|
436
438
|
|
|
437
|
-
仓库自带 **
|
|
439
|
+
仓库自带 **16 个**零依赖断言脚本(合计 **495 项**),**不需要 API key、不联网、不需要测试框架**,Node ≥ 20 即可跑:
|
|
438
440
|
|
|
439
441
|
| 脚本 | 覆盖 | 运行(项数) |
|
|
440
442
|
|---|---|---|
|
|
441
|
-
| `assert-engines.mjs` | 引擎清单 / 免费引擎口径 / 语言分池路由 / 池配置字段 / 知乎 URL 剥 utm | `node tools/assert-engines.mjs`(
|
|
443
|
+
| `assert-engines.mjs` | 引擎清单 / 免费引擎口径 / 语言分池路由 / 池配置字段 / 知乎 URL 剥 utm / `deepseek-tools` 接线与 snippetCap 放宽 | `node tools/assert-engines.mjs`(41) |
|
|
442
444
|
| `assert-blocked-domains.mjs` | 域名黑名单:解析、匹配、收口,且前后端清洗同口径 | `node tools/assert-blocked-domains.mjs`(69) |
|
|
443
|
-
| `assert-multi-policy.mjs` | multi 执行策略:并发上限是天花板 / 目标成功源非硬下限 / 凭据资格在上限之前分流 / **Fast·Balanced·Deep 档位** | `node tools/assert-multi-policy.mjs`(
|
|
445
|
+
| `assert-multi-policy.mjs` | multi 执行策略:并发上限是天花板 / 目标成功源非硬下限 / 凭据资格在上限之前分流 / **Fast·Balanced·Deep 档位** | `node tools/assert-multi-policy.mjs`(146) |
|
|
444
446
|
| `assert-result-ranking.mjs` | 合并排序:**eTLD+1 同域判定** / 域名轮转 / `seenIn` 降为末位 / 时效仅带时间过滤时生效 | `node tools/assert-result-ranking.mjs`(31) |
|
|
445
447
|
| `assert-url-norm.mjs` | URL 规范化:只剔已审计的 tracking 参数,保留可能区分资源的参数 | `node tools/assert-url-norm.mjs`(72) |
|
|
446
448
|
| `assert-prompt-contract.mjs` | 系统提示词契约:**检索≠抓取**、不可信边界、模型内置搜索只在显式选中时运行 | `node tools/assert-prompt-contract.mjs`(9) |
|
|
@@ -453,6 +455,7 @@ Search engine test:
|
|
|
453
455
|
| `assert-weather-contract.mjs` | `weather` 工具输出 schema 契约(缺字段裁剪、英文输出、空 geocode 抛错) | `node tools/assert-weather-contract.mjs`(11) |
|
|
454
456
|
| `assert-killtree-win32.mjs` | Windows 子进程树终止(#74):不再依赖 `taskkill` | `node tools/assert-killtree-win32.mjs`(8) |
|
|
455
457
|
| `assert-killtree-portable.mjs` | 跨平台 `killTree` 便携版契约 | `node tools/assert-killtree-portable.mjs`(5) |
|
|
458
|
+
| `assert-parallel-mcp.mjs` | Parallel MCP 引擎:25 s 超时/调用方取消覆盖**响应体读取**、共享 SSE 解析器(通知/多行帧)、6 类错误路径、REST 精确时间过滤 | `node tools/assert-parallel-mcp.mjs`(15) |
|
|
456
459
|
|
|
457
460
|
任何一个失败会以非 0 退出码结束并列出失败项;把某个旧版本文件路径作为参数传进去,可以看到修复前必红(用来证明断言不是空转)。改动发布相关逻辑后建议先跑一遍再发版。
|
|
458
461
|
|
|
@@ -543,7 +546,8 @@ If this plugin has been helpful, a ⭐ on [GitHub](https://github.com/DDDMUC/dsh
|
|
|
543
546
|
| `perplexity` | Perplexity | Paid | Requires `PERPLEXITY_API_KEY` |
|
|
544
547
|
| `serpbase` | SerpBase | Paid | Requires `SERPBASE_API_KEY` (serpbase.dev, 100 free queries on signup) |
|
|
545
548
|
| `serply` | Serply | Paid | Requires `SERPLY_API_KEY` (serply.io, 2,500 free credits for the first 30 days); Google web results localized to the settings language |
|
|
546
|
-
| `deepseek-official` | DeepSeek Official | Paid | Requires `DEEPSEEK_API_KEY` |
|
|
549
|
+
| `deepseek-official` | DeepSeek Official | Paid | Requires `DEEPSEEK_API_KEY`; uses the Anthropic-compatible `web_search` server tool, so every search is a full model turn (slow, token-billed) |
|
|
550
|
+
| `deepseek-tools` | DeepSeek Native Search API | Paid | Requires `DEEPSEEK_API_KEY` (shared with the row above); calls `POST /v1/tools/search` directly, no model involved, returns structured `results[]` (`title`/`url`/`summary`/`date`) — measured ~0.3s vs ~5-6s for the row above on the same query |
|
|
547
551
|
| `you` | You.com | Paid | Requires `YOUCOM_API_KEY` (you.com/platform/api-keys, free tier on signup) |
|
|
548
552
|
| `baidu` | Baidu Qianfan AI Search | Quota | Requires `BAIDU_API_KEY` (Qianfan AI Search, 50 free calls/day then pay-as-you-go); Chinese web-wide, supports time filtering |
|
|
549
553
|
| `kimi` | Kimi (Moonshot) Web Search | Paid | Requires `MOONSHOT_API_KEY` (basic ~¥0.01/call); returns Chinese results with body chunks; **no free quota**, so prefer explicit selection or the escalation pool over the default concurrent pool |
|
|
@@ -582,7 +586,7 @@ The free/paid binary is not what decides **who belongs in a default pool** — t
|
|
|
582
586
|
| ① Unlimited, no key | `ddg` / `ddg-lite` / `bing` / `searxng` / `anysearch` | — |
|
|
583
587
|
| ② Anonymous quota (runs without a key) | `exa` / `tavily` / `keenable` / `firecrawl` / `parallel` | key raises the quota |
|
|
584
588
|
| ③ Free quota (key required) | `zhihu_global` / `zhihu_site` (5,000/day trial), `doubao` (500/month), `baidu` (50/day), `aliyun` (200-call starter pack) | pay-as-you-go |
|
|
585
|
-
| ④ Paid upgrade (needs a key to return anything) | `serpbase` (100 free on signup) / `serply` (2,500 for 30 days) / `you` / `perplexity` / `deepseek-official` / `kimi` | per call / per query |
|
|
589
|
+
| ④ Paid upgrade (needs a key to return anything) | `serpbase` (100 free on signup) / `serply` (2,500 for 30 days) / `you` / `perplexity` / `deepseek-official` / `deepseek-tools` / `kimi` | per call / per query |
|
|
586
590
|
| ⑤ Per-call billing (model built-in, explicit-only) | `openai` / `gemini` / `claude` | per token + per search |
|
|
587
591
|
|
|
588
592
|
Rule of thumb for a default pool: **① as the floor + ③ as the workhorse**; ② is allowed but remember it burns anonymous quota; ④ only when you actually want that source; ⑤ never enters a pool. Per-engine facts and a default-pool proposal live in [docs/engine-quota-tiers.md](./docs/engine-quota-tiers.md).
|
|
@@ -740,7 +744,7 @@ The old `free-search:` section of `~/.dsh/settings.yaml` is **not** imported by
|
|
|
740
744
|
|
|
741
745
|
```yaml
|
|
742
746
|
# config of that entry in profiles/<profile>/cordis.patch.yml:
|
|
743
|
-
provider: bing # ddg / ddg-lite / bing / searxng / anysearch / exa / tavily / keenable / firecrawl / parallel / perplexity / serpbase / serply / deepseek-official / you / baidu / kimi / aliyun / doubao / zhihu_global / zhihu_site / openai / gemini / claude / auto / multi
|
|
747
|
+
provider: bing # ddg / ddg-lite / bing / searxng / anysearch / exa / tavily / keenable / firecrawl / parallel / perplexity / serpbase / serply / deepseek-official / deepseek-tools / you / baidu / kimi / aliyun / doubao / zhihu_global / zhihu_site / openai / gemini / claude / auto / multi
|
|
744
748
|
fallbackOn: [quota, auth, bot-wall, transient, invalid-response, unknown] # failure classes allowed to trigger fallback; [] = abort on any failure (default: all)
|
|
745
749
|
lang: zh # settings UI language (zh / en)
|
|
746
750
|
bingMarket: zh-CN # Bing market
|
|
@@ -799,10 +803,11 @@ Ask the agent for *"news from the last week"*, *"releases this month"*, *"update
|
|
|
799
803
|
| SearXNG | `time_range` | ⚠️ approximate | same as above |
|
|
800
804
|
| DuckDuckGo / Lite | `df` | ⚠️ approximate | same as above |
|
|
801
805
|
| Bing / AnySearch | — | ❌ ignored | no corresponding parameter |
|
|
806
|
+
| DeepSeek Native Search (`deepseek-tools`) | `after:` query syntax | ✅ precise | the endpoint has no dedicated date field, so `after:YYYY-MM-DD` is appended to the query; relative day counts are converted to an N-days-ago date |
|
|
802
807
|
|
|
803
808
|
**Nearest-tier mapping rule**: `≤2 days → day`, `≤14 days → week`, `≤90 days → month`, otherwise `year`. For example, `3d` becomes `day` on Tavily, and `2mo` becomes `month`.
|
|
804
809
|
|
|
805
|
-
**Engine-chain priority**: when a `timeRange` is present, engines that support time filtering (tavily / exa / keenable / firecrawl / parallel / searxng / ddg / ddg-lite) are moved to the front of the fallback chain, so the filter actually takes effect — even if the preferred engine is bing (which does not support filtering), a filtering-capable engine is tried first.
|
|
810
|
+
**Engine-chain priority**: when a `timeRange` is present, engines that support time filtering (tavily / exa / keenable / firecrawl / parallel / searxng / ddg / ddg-lite / deepseek-tools) are moved to the front of the fallback chain, so the filter actually takes effect — even if the preferred engine is bing (which does not support filtering), a filtering-capable engine is tried first.
|
|
806
811
|
|
|
807
812
|
Example: *"Find DSH news from the last 3 days"* → agent calls `advanced_search` with `timeRange: "3d"`.
|
|
808
813
|
|
|
@@ -907,13 +912,13 @@ Restart `dsh web` after switching to apply changes.
|
|
|
907
912
|
|
|
908
913
|
#### Offline self-check assertions (`tools/`)
|
|
909
914
|
|
|
910
|
-
**
|
|
915
|
+
**Sixteen** zero-dependency assertion scripts ship with the repo (**495 assertions** total): **no API keys, no network, no test framework**, Node ≥ 20:
|
|
911
916
|
|
|
912
917
|
| Script | Covers | Run (count) |
|
|
913
918
|
|---|---|---|
|
|
914
|
-
| `assert-engines.mjs` | engine registry / free-engine rule / language-pool routing / pool config fields / Zhihu URL utm stripping | `node tools/assert-engines.mjs` (
|
|
919
|
+
| `assert-engines.mjs` | engine registry / free-engine rule / language-pool routing / pool config fields / Zhihu URL utm stripping / `deepseek-tools` wiring and snippetCap widening | `node tools/assert-engines.mjs` (41) |
|
|
915
920
|
| `assert-blocked-domains.mjs` | domain blocklist: parsing, matching, choke points, and byte-identical front/back-end sanitizing | `node tools/assert-blocked-domains.mjs` (69) |
|
|
916
|
-
| `assert-multi-policy.mjs` | multi execution policy: ceiling semantics / target-is-not-a-hard-minimum / credential eligibility before the ceiling / **Fast·Balanced·Deep presets** | `node tools/assert-multi-policy.mjs` (
|
|
921
|
+
| `assert-multi-policy.mjs` | multi execution policy: ceiling semantics / target-is-not-a-hard-minimum / credential eligibility before the ceiling / **Fast·Balanced·Deep presets** | `node tools/assert-multi-policy.mjs` (146) |
|
|
917
922
|
| `assert-result-ranking.mjs` | merged ranking: **eTLD+1 same-site detection** / domain round-robin / `seenIn` demoted / freshness only under a time filter | `node tools/assert-result-ranking.mjs` (31) |
|
|
918
923
|
| `assert-url-norm.mjs` | URL normalization: strips only audited tracking params, keeps params that may identify different resources | `node tools/assert-url-norm.mjs` (72) |
|
|
919
924
|
| `assert-prompt-contract.mjs` | system-prompt contract: **search ≠ fetch**, untrusted-data boundary, model engines only when explicitly selected | `node tools/assert-prompt-contract.mjs` (9) |
|
|
@@ -926,6 +931,7 @@ Restart `dsh web` after switching to apply changes.
|
|
|
926
931
|
| `assert-weather-contract.mjs` | `weather` output schema contract (field pruning, English output, empty geocode throws) | `node tools/assert-weather-contract.mjs` (11) |
|
|
927
932
|
| `assert-killtree-win32.mjs` | Windows process-tree termination (#74): no longer relies on `taskkill` | `node tools/assert-killtree-win32.mjs` (8) |
|
|
928
933
|
| `assert-killtree-portable.mjs` | portable `killTree` contract | `node tools/assert-killtree-portable.mjs` (5) |
|
|
934
|
+
| `assert-parallel-mcp.mjs` | Parallel MCP engine: the 25 s timeout / caller abort covers the **body read**, shared SSE parser (notifications, multiline frames), six error paths, REST precise time filtering | `node tools/assert-parallel-mcp.mjs` (15) |
|
|
929
935
|
|
|
930
936
|
Any failure exits non-zero with the failing items listed. Passing the path of an older file shows the pre-fix failures (proving the assertions are not vacuous). Worth running before a release.
|
|
931
937
|
|
package/lib/client.js
CHANGED
|
@@ -223,7 +223,7 @@ window.__ModuleLoader__.load({
|
|
|
223
223
|
multiMinSourcesHint: "希望至少几个引擎成功返回。不够时会自动用池外引擎补跑,达标即停。注意这是「尽量达到」而非硬性要求:补跑也达不到时仍会返回已有结果,只在备注里说明,不会让搜索失败。",
|
|
224
224
|
multiPreset: "执行档位",
|
|
225
225
|
multiPresetCustom: "自定义",
|
|
226
|
-
multiPresetHint: (now) => `一键把上面两个数字设成常用组合(Fast = 1 并发 / 1 目标,Balanced = 3 / 1,Deep = 6 / 2);手改数字会自动回到「自定义」。当前:${now}。`,
|
|
226
|
+
multiPresetHint: (now) => `一键把上面两个数字设成常用组合(Fast = 1 并发 / 1 目标,Balanced = 3 / 1,Deep = 6 / 2);手改数字会自动回到「自定义」。当前:${now}。`,
|
|
227
227
|
blockedDomainsLabel: "域名黑名单(blockedDomains)",
|
|
228
228
|
blockedDomainsHint: "一行一个域名,命中的站点及其所有子域都会从搜索结果里剔除(csdn.net 覆盖 blog.csdn.net)。留空=不过滤。可以直接粘贴完整网址,scheme、路径和端口会被自动去掉。",
|
|
229
229
|
blockedDomainsWarn: (corrected, dupes, dropped) => `已忽略 ${dupes} 个重复项、${dropped} 行无法识别;${corrected} 行已自动修正为裸域名。规则对下一次搜索生效,旧结果缓存会同步失效。`,
|
|
@@ -388,7 +388,7 @@ window.__ModuleLoader__.load({
|
|
|
388
388
|
};
|
|
389
389
|
const tt = (lang) => I18N[lang === "en" ? "en" : "zh"];
|
|
390
390
|
// 当前插件版本(与 lib/index.js 的 PLUGIN_VERSION 保持一致;跟随上游 v0.7.5)
|
|
391
|
-
const PLUGIN_VERSION = "0.9.
|
|
391
|
+
const PLUGIN_VERSION = "0.9.2";
|
|
392
392
|
const ENGINES = [
|
|
393
393
|
{ id: "auto", label: "Auto · 智能路由", badge: "AUTO", link: null },
|
|
394
394
|
{ id: "multi", label: "Multi Search · 多引擎", badge: "MULTI", link: null },
|
|
@@ -406,6 +406,7 @@ window.__ModuleLoader__.load({
|
|
|
406
406
|
{ id: "serpbase", label: "SerpBase · Google", badge: "API KEY", link: "https://serpbase.dev" },
|
|
407
407
|
{ id: "serply", label: "Serply · Google", badge: "API KEY", link: "https://serply.io" },
|
|
408
408
|
{ id: "deepseek-official", label: "DeepSeek Official", badge: "API KEY", link: "https://platform.deepseek.com/api_keys" },
|
|
409
|
+
{ id: "deepseek-tools", label: "DeepSeek Tools API", badge: "API KEY", link: "https://api-docs.deepseek.com/zh-cn/guides/search" },
|
|
409
410
|
{ id: "you", label: "You.com", badge: "API KEY", link: "https://you.com/platform/api-keys" },
|
|
410
411
|
{ id: "baidu", label: "Baidu · 百度千帆", badge: "API KEY", link: "https://console.bce.baidu.com/qianfan" },
|
|
411
412
|
{ id: "kimi", label: "Kimi · Moonshot", badge: "API KEY", link: "https://platform.moonshot.cn" },
|
|
@@ -423,7 +424,7 @@ window.__ModuleLoader__.load({
|
|
|
423
424
|
// 与 lib/index.js 的 EXPLICIT_ONLY_ENGINES 同源:按次计费、只在显式选中时运行,因此不提供入池候选。
|
|
424
425
|
const EXPLICIT_ONLY_ENGINES = ["openai", "gemini", "claude"];
|
|
425
426
|
const MODEL_ENGINES = ENGINES.filter((engine) => EXPLICIT_ONLY_ENGINES.includes(engine.id));
|
|
426
|
-
const DEFAULT_FALLBACK_ORDER = ["exa", "tavily", "keenable", "firecrawl", "parallel", "perplexity", "serpbase", "serply", "deepseek-official", "you", "baidu", "kimi", "aliyun", "doubao", "tinyfish", "jina", "bing", "anysearch", "ddg", "ddg-lite", "searxng", "zhihu_global", "zhihu_site"];
|
|
427
|
+
const DEFAULT_FALLBACK_ORDER = ["exa", "tavily", "keenable", "firecrawl", "parallel", "perplexity", "serpbase", "serply", "deepseek-tools", "deepseek-official", "you", "baidu", "kimi", "aliyun", "doubao", "tinyfish", "jina", "bing", "anysearch", "ddg", "ddg-lite", "searxng", "zhihu_global", "zhihu_site"];
|
|
427
428
|
const normalizeFallbackOrder = (value) => {
|
|
428
429
|
const configured = Array.isArray(value) ? value.filter((id) => ENGINE_IDS.includes(id)) : [];
|
|
429
430
|
return [...new Set([...configured, ...DEFAULT_FALLBACK_ORDER, ...ENGINE_IDS.filter((id) => !EXPLICIT_ONLY_ENGINES.includes(id))])].filter((id) => !EXPLICIT_ONLY_ENGINES.includes(id));
|
|
@@ -671,6 +672,7 @@ window.__ModuleLoader__.load({
|
|
|
671
672
|
parallel: [parallelKey, setParallelKey],
|
|
672
673
|
perplexity: [perplexityKey, setPerplexityKey],
|
|
673
674
|
"deepseek-official": [deepseekKey, setDeepseekKey],
|
|
675
|
+
"deepseek-tools": [deepseekKey, setDeepseekKey],
|
|
674
676
|
serpbase: [serpbaseKey, setSerpbaseKey],
|
|
675
677
|
serply: [serplyKey, setSerplyKey],
|
|
676
678
|
you: [youcomKey, setYoucomKey],
|
|
@@ -688,7 +690,7 @@ window.__ModuleLoader__.load({
|
|
|
688
690
|
claude: [claudeKey, setClaudeKey],
|
|
689
691
|
};
|
|
690
692
|
// #68 密钥状态可视化:引擎 id → 配置中心键名(少数引擎不同名),三态:已配置/可选/缺失
|
|
691
|
-
const CONFIGURED_KEY_ID = { "deepseek-official": "deepseek", you: "youcom", zhihu_global: "zhihu", zhihu_site: "zhihu" };
|
|
693
|
+
const CONFIGURED_KEY_ID = { "deepseek-official": "deepseek", "deepseek-tools": "deepseek", you: "youcom", zhihu_global: "zhihu", zhihu_site: "zhihu" };
|
|
692
694
|
const NO_KEY_ENGINES = ["ddg", "ddg-lite", "bing"];
|
|
693
695
|
const OPTIONAL_KEY_ENGINES = ["anysearch", "exa", "tavily", "keenable", "firecrawl", "parallel", "searxng"];
|
|
694
696
|
const hasConfiguredKey = (id) => {
|
package/lib/index.js
CHANGED
|
@@ -5,7 +5,7 @@ import fs from "node:fs";
|
|
|
5
5
|
import os from "node:os";
|
|
6
6
|
import path from "node:path";
|
|
7
7
|
import { exec } from "node:child_process";
|
|
8
|
-
import { mcpCall, mcpContentText } from "./mcp_adapter.js";
|
|
8
|
+
import { mcpCall, mcpContentText, parseMcpResponseText } from "./mcp_adapter.js";
|
|
9
9
|
import { registerResearchTool } from "./research.js";
|
|
10
10
|
import { registerDomainSearchTool } from "./vertical.js";
|
|
11
11
|
import { createFetchProvider } from "./fetch_provider.js";
|
|
@@ -90,7 +90,7 @@ const FREE_SEARCH_NS = "web-search-free";
|
|
|
90
90
|
const BRIDGE_PREFIX = "/api/dsh-free-search-settings";
|
|
91
91
|
// 意见 A:zhihu_global / zhihu_site 需要 ZHIHU_API_KEY,不属于免 key 引擎(不得进入 FREE_ENGINES)
|
|
92
92
|
const FREE_ENGINES = ["ddg", "ddg-lite", "bing", "searxng", "anysearch"];
|
|
93
|
-
const ALL_ENGINES = ["ddg", "ddg-lite", "bing", "searxng", "anysearch", "exa", "tavily", "keenable", "firecrawl", "parallel", "perplexity", "serpbase", "serply", "deepseek-official", "you", "baidu", "kimi", "aliyun", "doubao", "tinyfish", "jina", "zhihu_global", "zhihu_site", "openai", "gemini", "claude"];
|
|
93
|
+
const ALL_ENGINES = ["ddg", "ddg-lite", "bing", "searxng", "anysearch", "exa", "tavily", "keenable", "firecrawl", "parallel", "perplexity", "serpbase", "serply", "deepseek-official", "deepseek-tools", "you", "baidu", "kimi", "aliyun", "doubao", "tinyfish", "jina", "zhihu_global", "zhihu_site", "openai", "gemini", "claude"];
|
|
94
94
|
|
|
95
95
|
// 模型内置搜索(按次计费):只在你显式选中该引擎时使用;自动回退链与 auto 路由都不会兜底到它们。
|
|
96
96
|
const EXPLICIT_ONLY_ENGINES = ["openai", "gemini", "claude"];
|
|
@@ -166,12 +166,13 @@ const FAILURE_CLASS_LABELS = {
|
|
|
166
166
|
|
|
167
167
|
|
|
168
168
|
// 支持 time_range 过滤的引擎(zhihu_global:global_search 的 filter 支持 publish_time>=unix秒,已实测;zhihu_site 未验证,不加)
|
|
169
|
-
|
|
169
|
+
// deepseek-tools:原生搜索端点没有独立时间字段,但 query 语法支持 after:/before:,由 deepseekToolsQuery 注入。
|
|
170
|
+
const TIME_ENGINES = ["tavily", "exa", "keenable", "firecrawl", "parallel", "searxng", "ddg", "ddg-lite", "baidu", "doubao", "tinyfish", "deepseek-tools", "zhihu_global"];
|
|
170
171
|
|
|
171
172
|
// 虚拟搜索模式:不是单个引擎,而是 provider.search 内部的路由/并发策略
|
|
172
173
|
const SEARCH_MODES = ["auto", "multi"];
|
|
173
174
|
|
|
174
|
-
const DEFAULT_FALLBACK_ORDER = ["exa", "tavily", "keenable", "firecrawl", "parallel", "perplexity", "serpbase", "serply", "deepseek-official", "you", "baidu", "kimi", "aliyun", "doubao", "tinyfish", "jina", "bing", "anysearch", "ddg", "ddg-lite", "searxng", "zhihu_global", "zhihu_site"];
|
|
175
|
+
const DEFAULT_FALLBACK_ORDER = ["exa", "tavily", "keenable", "firecrawl", "parallel", "perplexity", "serpbase", "serply", "deepseek-tools", "deepseek-official", "you", "baidu", "kimi", "aliyun", "doubao", "tinyfish", "jina", "bing", "anysearch", "ddg", "ddg-lite", "searxng", "zhihu_global", "zhihu_site"];
|
|
175
176
|
|
|
176
177
|
function normalizeDisabledEngines(value) {
|
|
177
178
|
if (!Array.isArray(value)) return [];
|
|
@@ -244,7 +245,7 @@ function routeEngines(query, { timeRange, disabledEngines = [], fallbackOrder =
|
|
|
244
245
|
}
|
|
245
246
|
|
|
246
247
|
// 当前插件版本(发布时与 package.json 同步;跟随上游 v0.7.5,不在 PR 里 bump)
|
|
247
|
-
const PLUGIN_VERSION = "0.9.
|
|
248
|
+
const PLUGIN_VERSION = "0.9.2";
|
|
248
249
|
// 检查更新的 npm registry 元数据地址(dsh-free-search 是 npmjs 上的公开包)
|
|
249
250
|
const NPM_REGISTRY_URL = "https://registry.npmjs.org/dsh-free-search/latest";
|
|
250
251
|
const PLUGIN_NPM_URL = "https://www.npmjs.com/package/dsh-free-search";
|
|
@@ -395,8 +396,10 @@ function cleanSnippet(text, max = 300) {
|
|
|
395
396
|
}
|
|
396
397
|
|
|
397
398
|
// 各引擎的摘要展示上限:豆包搜索返回的是千字级 Summary(实测 1300-1800 字),放宽到 2000;
|
|
398
|
-
//
|
|
399
|
-
|
|
399
|
+
// deepseek-tools 的 summary 官方文档写 600~800 字(实测 530),按 300 截断会把摘要拦腰砍断,
|
|
400
|
+
// 放宽到 2000 与豆包同理;其余引擎维持 300。
|
|
401
|
+
// 该上限同时用于 provider 成功后的清洗与 runEngineTest(喂 multi_search)。
|
|
402
|
+
const SNIPPET_MAX_BY_ENGINE = { doubao: 2000, "deepseek-tools": 2000 };
|
|
400
403
|
function snippetCap(engine) {
|
|
401
404
|
return SNIPPET_MAX_BY_ENGINE[engine] ?? 300;
|
|
402
405
|
}
|
|
@@ -1612,15 +1615,20 @@ async function searchParallelMCP(query, maxResults, timeRange, signal) {
|
|
|
1612
1615
|
const timer = setTimeout(() => controller.abort(), 25000);
|
|
1613
1616
|
const onAbort = () => controller.abort();
|
|
1614
1617
|
signal?.addEventListener("abort", onAbort);
|
|
1615
|
-
|
|
1618
|
+
if (signal?.aborted) controller.abort();
|
|
1619
|
+
let text;
|
|
1616
1620
|
try {
|
|
1617
1621
|
const after = timeRange
|
|
1618
1622
|
? timeRange.after ?? (timeRange.days !== undefined ? isoDaysAgo(timeRange.days).slice(0, 10) : undefined)
|
|
1619
1623
|
: undefined;
|
|
1620
1624
|
const objective = after ? `${query} (prefer results published after ${after})` : query;
|
|
1621
|
-
response = await fetch(PARALLEL_MCP_URL, {
|
|
1625
|
+
const response = await fetch(PARALLEL_MCP_URL, {
|
|
1622
1626
|
method: "POST",
|
|
1623
|
-
headers: {
|
|
1627
|
+
headers: {
|
|
1628
|
+
"content-type": "application/json",
|
|
1629
|
+
accept: "application/json, text/event-stream",
|
|
1630
|
+
"user-agent": "dsh-free-search (+https://github.com/DDDMUC/dsh-free-search)",
|
|
1631
|
+
},
|
|
1624
1632
|
body: JSON.stringify({
|
|
1625
1633
|
jsonrpc: "2.0",
|
|
1626
1634
|
id: Date.now(),
|
|
@@ -1636,6 +1644,9 @@ async function searchParallelMCP(query, maxResults, timeRange, signal) {
|
|
|
1636
1644
|
}),
|
|
1637
1645
|
signal: controller.signal,
|
|
1638
1646
|
});
|
|
1647
|
+
if (!response.ok) throw new Error(`Parallel MCP error (HTTP ${response.status})`);
|
|
1648
|
+
// Keep the timeout and caller cancellation active while the body is streaming.
|
|
1649
|
+
text = await response.text();
|
|
1639
1650
|
} catch (error) {
|
|
1640
1651
|
if (signal?.aborted) throw error;
|
|
1641
1652
|
throw new Error(`Parallel MCP request failed: ${error?.message ?? String(error)}`);
|
|
@@ -1643,33 +1654,25 @@ async function searchParallelMCP(query, maxResults, timeRange, signal) {
|
|
|
1643
1654
|
clearTimeout(timer);
|
|
1644
1655
|
signal?.removeEventListener("abort", onAbort);
|
|
1645
1656
|
}
|
|
1646
|
-
|
|
1647
|
-
const
|
|
1648
|
-
|
|
1649
|
-
|
|
1650
|
-
|
|
1651
|
-
|
|
1652
|
-
} catch {
|
|
1653
|
-
for (const line of text.split("\n")) {
|
|
1654
|
-
if (line.startsWith("data: ")) {
|
|
1655
|
-
try {
|
|
1656
|
-
json = JSON.parse(line.slice(6));
|
|
1657
|
-
break;
|
|
1658
|
-
} catch {}
|
|
1659
|
-
}
|
|
1660
|
-
}
|
|
1661
|
-
}
|
|
1662
|
-
if (!json || json.error) throw new Error(`Parallel MCP error: ${json?.error?.message ?? "no data"}`);
|
|
1663
|
-
const blocks = (json.result?.content ?? []).filter((b) => b.type === "text").map((b) => b.text);
|
|
1657
|
+
// The shared parser skips SSE notifications and joins multiline data frames.
|
|
1658
|
+
const json = parseMcpResponseText(text);
|
|
1659
|
+
if (!json) throw new Error("Parallel MCP parse failed: invalid JSON or SSE");
|
|
1660
|
+
if (json.error) throw new Error(`Parallel MCP error: ${json.error.message ?? "unknown error"}`);
|
|
1661
|
+
if (!Array.isArray(json.result?.content)) throw new Error("Parallel MCP malformed content");
|
|
1662
|
+
const blocks = json.result.content.filter((b) => b?.type === "text" && typeof b.text === "string").map((b) => b.text);
|
|
1664
1663
|
if (json.result?.isError) throw new Error(`Parallel MCP error: ${blocks.join(" ").slice(0, 200) || "tool call failed"}`);
|
|
1665
1664
|
let payload = null;
|
|
1666
1665
|
for (const block of blocks) {
|
|
1667
1666
|
try {
|
|
1668
|
-
|
|
1669
|
-
|
|
1667
|
+
const parsed = JSON.parse(block);
|
|
1668
|
+
if (Array.isArray(parsed?.results)) {
|
|
1669
|
+
payload = parsed;
|
|
1670
|
+
break;
|
|
1671
|
+
}
|
|
1670
1672
|
} catch {}
|
|
1671
1673
|
}
|
|
1672
|
-
|
|
1674
|
+
if (!payload) throw new Error("Parallel MCP malformed search results");
|
|
1675
|
+
const sources = payload.results
|
|
1673
1676
|
.filter((r) => r.url)
|
|
1674
1677
|
.map((r) => {
|
|
1675
1678
|
const excerpt = (r.excerpts ?? []).find((e) => String(e).trim().length > 0);
|
|
@@ -1900,6 +1903,79 @@ async function searchDeepSeekOfficial(query, maxResults, apiKey, signal) {
|
|
|
1900
1903
|
return { sources: uniqueSources(sources, maxResults ?? 10), truncated: false };
|
|
1901
1904
|
}
|
|
1902
1905
|
|
|
1906
|
+
// DeepSeek 官方原生搜索 API(POST /v1/tools/search)
|
|
1907
|
+
// key = DEEPSEEK_API_KEY(与 deepseek-official 共用,接口不同)
|
|
1908
|
+
//
|
|
1909
|
+
// 与 deepseek-official 的区别:那一档走 Anthropic Messages +
|
|
1910
|
+
// `web_search_20250305` 服务端工具,每次搜索是一个完整模型轮次(慢、耗 token、
|
|
1911
|
+
// 结果要从 web_search_tool_result 块里再解析);这一档直接调原生搜索端点,
|
|
1912
|
+
// 无模型参与,返回结构化 results[]。实测同 query 下 ~0.3s vs ~5-6s。
|
|
1913
|
+
//
|
|
1914
|
+
// 计费按「请求次数」而非条数(count 不影响单价),所以这里超额取数不增加成本。
|
|
1915
|
+
const DEEPSEEK_TOOLS_SEARCH_URL = "https://api.deepseek.com/v1/tools/search";
|
|
1916
|
+
|
|
1917
|
+
// 原生搜索接口没有独立的时间字段,时间过滤写在 query 语法里(after:/before:)。
|
|
1918
|
+
// 支持 `after:2025`、`after:2025-01-01`;这里把统一的 timeRange 映射成 after:。
|
|
1919
|
+
function deepseekToolsQuery(query, timeRange) {
|
|
1920
|
+
const parsed = parseTimeRange(timeRange);
|
|
1921
|
+
if (!parsed) return query;
|
|
1922
|
+
if (parsed.after) return `${query} after:${parsed.after}`;
|
|
1923
|
+
if (parsed.days) {
|
|
1924
|
+
const since = new Date(Date.now() - parsed.days * 86400000).toISOString().slice(0, 10);
|
|
1925
|
+
return `${query} after:${since}`;
|
|
1926
|
+
}
|
|
1927
|
+
return query;
|
|
1928
|
+
}
|
|
1929
|
+
|
|
1930
|
+
async function searchDeepSeekTools(query, maxResults, apiKey, timeRange, signal) {
|
|
1931
|
+
if (!apiKey) throw new Error("DeepSeek search requires DEEPSEEK_API_KEY");
|
|
1932
|
+
const limit = Math.min(Math.max(maxResults ?? 5, 1), 50);
|
|
1933
|
+
// 超额取数:按请求计费,且缺 url 的行会被丢弃,多取一些补位成本为零
|
|
1934
|
+
const count = Math.min(50, Math.max(limit, Math.ceil(limit * 1.5)));
|
|
1935
|
+
|
|
1936
|
+
const response = await fetch(DEEPSEEK_TOOLS_SEARCH_URL, {
|
|
1937
|
+
method: "POST",
|
|
1938
|
+
redirect: "error",
|
|
1939
|
+
headers: {
|
|
1940
|
+
authorization: `Bearer ${apiKey}`,
|
|
1941
|
+
"content-type": "application/json",
|
|
1942
|
+
accept: "application/json",
|
|
1943
|
+
"user-agent": "deepseek-harness/free-search",
|
|
1944
|
+
},
|
|
1945
|
+
body: JSON.stringify({ query: deepseekToolsQuery(query, timeRange), count }),
|
|
1946
|
+
signal: AbortSignal.any([...(signal !== undefined ? [signal] : []), AbortSignal.timeout(20000)]),
|
|
1947
|
+
});
|
|
1948
|
+
if (!response.ok) {
|
|
1949
|
+
const detail = await response.text().catch(() => "");
|
|
1950
|
+
if (response.status === 401) {
|
|
1951
|
+
throw new Error("DeepSeek API key is invalid (HTTP 401) - update it in Settings > Plugins > Free Search");
|
|
1952
|
+
}
|
|
1953
|
+
// 402 余额不足是终态:不要当成可重试的暂时故障
|
|
1954
|
+
if (response.status === 402) {
|
|
1955
|
+
throw new Error("DeepSeek account balance is insufficient (HTTP 402) - top up at https://platform.deepseek.com");
|
|
1956
|
+
}
|
|
1957
|
+
throw new Error(`DeepSeek API error (HTTP ${response.status}): ${detail.slice(0, 200)}`);
|
|
1958
|
+
}
|
|
1959
|
+
|
|
1960
|
+
const data = await response.json();
|
|
1961
|
+
const rows = Array.isArray(data?.results) ? data.results : [];
|
|
1962
|
+
const sources = rows
|
|
1963
|
+
.filter((r) => r && typeof r.url === "string" && r.url.length > 0)
|
|
1964
|
+
.map((r) => {
|
|
1965
|
+
// summary(600~800 字,面向模型)优先于 snippet(100~200 字,面向渲染)
|
|
1966
|
+
const raw = typeof r.summary === "string" && r.summary.trim() ? r.summary : r.snippet;
|
|
1967
|
+
const snippet = typeof raw === "string" ? raw.trim() : "";
|
|
1968
|
+
return {
|
|
1969
|
+
url: r.url,
|
|
1970
|
+
...(r.title ? { title: String(r.title) } : {}),
|
|
1971
|
+
...(snippet ? { snippet: snippet.slice(0, 600) } : {}),
|
|
1972
|
+
// date 是 UTC RFC3339,只保留日期部分与其它引擎的 publishedAt 对齐
|
|
1973
|
+
...(r.date ? { publishedAt: String(r.date).slice(0, 10) } : {}),
|
|
1974
|
+
};
|
|
1975
|
+
});
|
|
1976
|
+
return { sources: uniqueSources(sources, maxResults ?? 10), truncated: false };
|
|
1977
|
+
}
|
|
1978
|
+
|
|
1903
1979
|
// SerpBase: 必须配置 SERPBASE_API_KEY(无 key 跳过,同 perplexity)
|
|
1904
1980
|
// Google SERP API:POST + X-API-Key 头,返回 organic[](title/link/snippet/published_at)。
|
|
1905
1981
|
// 注意:SerpBase 始终返回 HTTP 200,业务状态放在 JSON 的 status 字段(0=成功,1001=key 无效/缺失,1000=请求非法)。
|
|
@@ -3602,6 +3678,15 @@ function apply(ctx, config) {
|
|
|
3602
3678
|
continue; // 无 key 跳过
|
|
3603
3679
|
}
|
|
3604
3680
|
result = await searchDeepSeekOfficial(request.query, request.maxResults, key, effSignal);
|
|
3681
|
+
} else if (engine === "deepseek-tools") {
|
|
3682
|
+
const key = await resolveApiKey("DEEPSEEK_API_KEY", "deepseekApiKey");
|
|
3683
|
+
if (!key) {
|
|
3684
|
+
lastError = new Error("DeepSeek requires DEEPSEEK_API_KEY");
|
|
3685
|
+
if (engine === preferred) preferredFailure = "DEEPSEEK_API_KEY is not configured";
|
|
3686
|
+
logger.warn(`free-search: engine "${engine}" skipped (no key), trying next engine`);
|
|
3687
|
+
continue; // 无 key 跳过
|
|
3688
|
+
}
|
|
3689
|
+
result = await searchDeepSeekTools(request.query, request.maxResults, key, request.timeRange, effSignal);
|
|
3605
3690
|
} else if (engine === "serpbase") {
|
|
3606
3691
|
// serpbase:必须配置 SERPBASE_API_KEY(无 key 跳过,同 perplexity)
|
|
3607
3692
|
const key = await resolveApiKey("SERPBASE_API_KEY", "serpbaseApiKey");
|
|
@@ -3958,6 +4043,11 @@ function apply(ctx, config) {
|
|
|
3958
4043
|
if (!key) return { ok: false, error: "DEEPSEEK_API_KEY not configured" };
|
|
3959
4044
|
return await searchDeepSeekOfficial(q, n, key);
|
|
3960
4045
|
}
|
|
4046
|
+
case "deepseek-tools": {
|
|
4047
|
+
const key = await resolveApiKey("DEEPSEEK_API_KEY", "deepseekApiKey");
|
|
4048
|
+
if (!key) return { ok: false, error: "DEEPSEEK_API_KEY not configured" };
|
|
4049
|
+
return await searchDeepSeekTools(q, n, key, tr);
|
|
4050
|
+
}
|
|
3961
4051
|
case "serpbase": {
|
|
3962
4052
|
const key = await resolveApiKey("SERPBASE_API_KEY", "serpbaseApiKey");
|
|
3963
4053
|
if (!key) return { ok: false, error: "SERPBASE_API_KEY not configured" };
|
|
@@ -4324,7 +4414,7 @@ function apply(ctx, config) {
|
|
|
4324
4414
|
parameters: {
|
|
4325
4415
|
engines: {
|
|
4326
4416
|
type: "array",
|
|
4327
|
-
description: "Which engines to test (default: all enabled engines). Options: ddg, ddg-lite, bing, searxng, anysearch, exa, tavily, keenable, firecrawl, parallel, perplexity, serpbase, serply, deepseek-official, you, baidu, kimi, aliyun, doubao, tinyfish, jina, zhihu_global, zhihu_site, openai, gemini, claude; zhihu_global (web-wide, supports time filter) and zhihu_site (Zhihu-only) need ZHIHU_API_KEY. The virtual modes auto and multi are also accepted and run the real routing/concurrent path. openai/gemini/claude are model-based and bill per search, so they are only used when explicitly named.",
|
|
4417
|
+
description: "Which engines to test (default: all enabled engines). Options: ddg, ddg-lite, bing, searxng, anysearch, exa, tavily, keenable, firecrawl, parallel, perplexity, serpbase, serply, deepseek-official, deepseek-tools, you, baidu, kimi, aliyun, doubao, tinyfish, jina, zhihu_global, zhihu_site, openai, gemini, claude; zhihu_global (web-wide, supports time filter) and zhihu_site (Zhihu-only) need ZHIHU_API_KEY. The virtual modes auto and multi are also accepted and run the real routing/concurrent path. openai/gemini/claude are model-based and bill per search, so they are only used when explicitly named.",
|
|
4328
4418
|
items: { type: "string" },
|
|
4329
4419
|
},
|
|
4330
4420
|
query: {
|
|
@@ -4702,7 +4792,7 @@ function apply(ctx, config) {
|
|
|
4702
4792
|
},
|
|
4703
4793
|
engine: {
|
|
4704
4794
|
type: "string",
|
|
4705
|
-
description: "Optional search mode/engine: multi (parallel full language-pool merge), auto, ddg, ddg-lite, bing, searxng, anysearch, exa, tavily, keenable, firecrawl, parallel, perplexity, serpbase, serply, deepseek-official, you, baidu, kimi, aliyun, doubao, tinyfish, jina, zhihu_global, zhihu_site, openai, gemini, claude (model-based; explicit selection only).",
|
|
4795
|
+
description: "Optional search mode/engine: multi (parallel full language-pool merge), auto, ddg, ddg-lite, bing, searxng, anysearch, exa, tavily, keenable, firecrawl, parallel, perplexity, serpbase, serply, deepseek-official (Anthropic-compatible web_search tool), deepseek-tools (DeepSeek native POST /v1/tools/search), you, baidu, kimi, aliyun, doubao, tinyfish, jina, zhihu_global, zhihu_site, openai, gemini, claude (model-based; explicit selection only).",
|
|
4706
4796
|
},
|
|
4707
4797
|
},
|
|
4708
4798
|
output: {
|
|
@@ -4950,6 +5040,7 @@ export {
|
|
|
4950
5040
|
searchBing,
|
|
4951
5041
|
searchBilibili,
|
|
4952
5042
|
searchDeepSeekOfficial,
|
|
5043
|
+
searchDeepSeekTools,
|
|
4953
5044
|
searchDdgHtml,
|
|
4954
5045
|
searchDdgLite,
|
|
4955
5046
|
searchExa,
|
package/lib/multi_policy.js
CHANGED
|
@@ -1,270 +1,271 @@
|
|
|
1
|
-
// #69 Phase 1(第一批):multi 模式的执行策略——纯函数,便于离线断言。
|
|
2
|
-
//
|
|
3
|
-
// 为什么单独成文件:这几个判断(并发上限怎么取、目标成功数怎么补跑)跑在
|
|
4
|
-
// runMultiSearch 里,而那是 apply() 内的闭包、无法导出单测。抽成纯函数后
|
|
5
|
-
// 既可以直接断言,也符合 reporter「三个预设共用同一条执行管线」的方向
|
|
6
|
-
// (将来加 Fast/Balanced/Deep 时,这里就是预设唯一的作用点)。
|
|
7
|
-
//
|
|
8
|
-
// 两条约束来自 reporter #69,都有断言钉住:
|
|
9
|
-
// §1 并发上限是「天花板」不是「必须跑满」——候选不够就少跑,不得凑数。
|
|
10
|
-
// §3 目标成功数(minSources)是「尽量达到」而非硬下限——达不到时返回已有
|
|
11
|
-
// 结果并附提示,绝不丢弃有效证据、绝不因此让整次搜索失败。
|
|
12
|
-
// 另有兼容硬约束:默认值必须让现有行为完全不变,升级对用户透明。
|
|
13
|
-
|
|
14
|
-
/** 0 / 非数字 / 负数 一律视为「不限制」,保证未配置时行为与今天一致。 */
|
|
15
|
-
export function resolveMaxEngines(value, poolSize) {
|
|
16
|
-
const n = Number(value);
|
|
17
|
-
if (!Number.isFinite(n) || n <= 0) return poolSize;
|
|
18
|
-
return Math.min(Math.floor(n), poolSize);
|
|
19
|
-
}
|
|
20
|
-
|
|
21
|
-
/**
|
|
22
|
-
* 从候选池里挑出本次实际执行的引擎。
|
|
23
|
-
* 只做「取前 N」,保持池内顺序——池内顺序就是用户在 zhPool/enPool 里配的优先级,
|
|
24
|
-
* 未经用户许可不重排(轮换是后续批次的事)。
|
|
25
|
-
*/
|
|
26
|
-
export function selectEngines(engines, configuredMax) {
|
|
27
|
-
const list = Array.isArray(engines) ? engines.filter((e) => typeof e === "string" && e) : [];
|
|
28
|
-
if (list.length === 0) return [];
|
|
29
|
-
return list.slice(0, resolveMaxEngines(configuredMax, list.length));
|
|
30
|
-
}
|
|
31
|
-
|
|
32
|
-
/** 目标成功源数:至少 1(0/负数/非数字都归 1,因为「0 个成功源」没有意义)。 */
|
|
33
|
-
export function resolveMinSources(value, poolSize) {
|
|
34
|
-
const n = Number(value);
|
|
35
|
-
if (!Number.isFinite(n) || n <= 0) return 1;
|
|
36
|
-
return Math.max(1, Math.floor(n));
|
|
37
|
-
}
|
|
38
|
-
|
|
39
|
-
/**
|
|
40
|
-
* 主轮结束后,若成功引擎数未达目标,从链路里挑「还没试过」的候选补跑。
|
|
41
|
-
* 已达标时返回空数组——调用方据此一次额外请求都不发(省钱、省限流)。
|
|
42
|
-
*/
|
|
43
|
-
export function planTopUp({ attempted = [], succeeded = 0, minSources = 1, chain = [] } = {}) {
|
|
44
|
-
if (succeeded >= minSources) return [];
|
|
45
|
-
const tried = new Set(Array.isArray(attempted) ? attempted : []);
|
|
46
|
-
const candidates = Array.isArray(chain) ? chain : [];
|
|
47
|
-
return candidates.filter((engine) => engine && !tried.has(engine));
|
|
48
|
-
}
|
|
49
|
-
|
|
50
|
-
/** 未达标时的说明文字;达标返回 null(不制造噪音)。 */
|
|
51
|
-
export function describeTarget({ minSources = 1, achieved = 0 } = {}) {
|
|
52
|
-
if (achieved >= minSources) return null;
|
|
53
|
-
return `sourcesTarget: ${minSources}, achieved: ${achieved}`;
|
|
54
|
-
}
|
|
55
|
-
|
|
56
|
-
/**
|
|
57
|
-
* 执行档位(#69 §1)。**只复用现有两个旋钮**:并发上限 + 目标成功源数。
|
|
58
|
-
* 不新增后端配置——UI 点档位就是把这两个数字写进去,手改数字即回到「自定义」。
|
|
59
|
-
* 这样零迁移、零缓存键影响,且在 planner 统一之前不改变任何现有语义。
|
|
60
|
-
*/
|
|
61
|
-
export const MULTI_PRESETS = {
|
|
62
|
-
fast: { maxEngines: 1, minSources: 1 },
|
|
63
|
-
balanced: { maxEngines: 3, minSources: 1 },
|
|
64
|
-
deep: { maxEngines: 6, minSources: 2 },
|
|
65
|
-
};
|
|
66
|
-
|
|
67
|
-
/** 由当前 (上限, 目标源数) 反推档位名;不匹配任何档位时返回 "custom"。 */
|
|
68
|
-
export function presetForSettings(maxEngines, minSources) {
|
|
69
|
-
const max = Number(maxEngines);
|
|
70
|
-
const min = Number(minSources);
|
|
71
|
-
const normMax = Number.isFinite(max) && max > 0 ? Math.floor(max) : 0;
|
|
72
|
-
const normMin = Number.isFinite(min) && min > 0 ? Math.floor(min) : 1;
|
|
73
|
-
for (const [id, p] of Object.entries(MULTI_PRESETS)) {
|
|
74
|
-
if (p.maxEngines === normMax && p.minSources === normMin) return id;
|
|
75
|
-
}
|
|
76
|
-
return "custom";
|
|
77
|
-
}
|
|
78
|
-
|
|
79
|
-
/**
|
|
80
|
-
* 常见「两段式」公共后缀(co.uk / com.cn / co.jp …)。不追求完整 PSL——
|
|
81
|
-
* 只覆盖公开搜索里真会撞到的那批;未命中时退化为「最后两段」。
|
|
82
|
-
*/
|
|
83
|
-
const MULTI_PART_SUFFIXES = new Set([
|
|
84
|
-
// 中国 / 港台
|
|
85
|
-
"com.cn", "net.cn", "org.cn", "gov.cn", "edu.cn", "ac.cn",
|
|
86
|
-
"com.hk", "org.hk", "net.hk", "edu.hk", "gov.hk",
|
|
87
|
-
"com.tw", "org.tw", "net.tw", "edu.tw", "gov.tw",
|
|
88
|
-
// 日韩
|
|
89
|
-
"co.jp", "or.jp", "ne.jp", "ac.jp", "go.jp",
|
|
90
|
-
"co.kr", "or.kr", "ne.kr", "go.kr", "re.kr",
|
|
91
|
-
// 英国 / 英联邦 / 大洋洲
|
|
92
|
-
"co.uk", "org.uk", "ac.uk", "gov.uk", "me.uk", "net.uk", "sch.uk",
|
|
93
|
-
"com.au", "net.au", "org.au", "edu.au", "gov.au",
|
|
94
|
-
"co.nz", "net.nz", "org.nz", "govt.nz", "ac.nz",
|
|
95
|
-
"co.za", "org.za", "net.za", "gov.za",
|
|
96
|
-
"co.in", "net.in", "org.in", "gov.in", "ac.in",
|
|
97
|
-
"com.sg", "com.my", "com.ph", "com.pk", "com.vn", "co.th", "co.id",
|
|
98
|
-
// 欧洲 / 拉美 / 中东 / 非洲
|
|
99
|
-
"com.br", "com.mx", "com.ar", "com.tr", "com.ua", "com.pl", "com.ru",
|
|
100
|
-
"co.il", "com.sa", "com.eg", "com.ng", "co.ke",
|
|
101
|
-
]);
|
|
102
|
-
|
|
103
|
-
/**
|
|
104
|
-
* 取 URL 的**可注册域**(eTLD+1)用于「同域」判定。
|
|
105
|
-
*
|
|
106
|
-
* 为什么不用主机名:blog.example.com 与 download.example.com 是同一站点的两个子域,
|
|
107
|
-
* 拿主机名分组会把它们当成两个「独立来源」,与 #69 §4 的域名多样性意图正好相反。
|
|
108
|
-
* (这正是 0.8.10 里自认的 known limitation。)
|
|
109
|
-
*
|
|
110
|
-
* IP / 含端口的 IPv6 / localhost / 单标签主机 / 非法 URL 一律原样小写返回,不误判。
|
|
111
|
-
*/
|
|
112
|
-
export function domainOf(url) {
|
|
113
|
-
const raw = String(url ?? "").trim();
|
|
114
|
-
if (!raw) return "";
|
|
115
|
-
let host;
|
|
116
|
-
try {
|
|
117
|
-
host = new URL(raw).hostname.toLowerCase();
|
|
118
|
-
} catch {
|
|
119
|
-
return raw.toLowerCase();
|
|
120
|
-
}
|
|
121
|
-
host = host.replace(/\.+$/, "").replace(/^www\./, "");
|
|
122
|
-
if (!host) return "";
|
|
123
|
-
if (!host.includes(".") || host.includes(":") || /^\d{1,3}(\.\d{1,3}){3}$/.test(host)) return host;
|
|
124
|
-
const parts = host.split(".");
|
|
125
|
-
if (parts.length <= 2) return host;
|
|
126
|
-
const last2 = parts.slice(-2).join(".");
|
|
127
|
-
return MULTI_PART_SUFFIXES.has(last2) ? parts.slice(-3).join(".") : last2;
|
|
128
|
-
}
|
|
129
|
-
|
|
130
|
-
function publishedTs(value) {
|
|
131
|
-
const ts = Date.parse(String(value ?? ""));
|
|
132
|
-
return Number.isFinite(ts) ? ts : 0;
|
|
133
|
-
}
|
|
134
|
-
|
|
135
|
-
/**
|
|
136
|
-
* 合并后的排序(#69 §4)。
|
|
137
|
-
*
|
|
138
|
-
* reporter 的要求:排序信号用「真实可得且有意义」的那些——原始结果顺序、域名多样性、
|
|
139
|
-
* 需要时的时效性;而 `seenIn` **只能当溯源/佐证元数据,不能当主排序键**,理由是
|
|
140
|
-
* "Six matching search results do not necessarily constitute six independent sources"
|
|
141
|
-
* (多个引擎可能共享索引与排序偏见,所以同域堆叠并不等于可信)。
|
|
142
|
-
*
|
|
143
|
-
* 做法(确定性、可离线断言):
|
|
144
|
-
* 1. 按域名分组;
|
|
145
|
-
* 2. 组内次序:时效性生效时新的优先 → 原始顺序 → 佐证数(仅末位 tiebreak);
|
|
146
|
-
* 3. 组的次序:按各域「最佳条目」的上述次序;
|
|
147
|
-
* 4. 轮转交错输出:第一轮每域各出一条,再第二轮…… → 前 N 条尽量来自不同域。
|
|
148
|
-
*
|
|
149
|
-
* 刻意只重排、不剔除:多样性是排序信号,不是过滤条件(过滤由 blockedDomains 负责)。
|
|
150
|
-
*
|
|
151
|
-
* 同域判定走 domainOf() 的 eTLD+1:blog.example.com 与 download.example.com 归为同一域。
|
|
152
|
-
* 局限:后缀表是精简版(非完整 PSL),冷门多段后缀仍退化为「最后两段」。
|
|
153
|
-
*
|
|
154
|
-
* @param {{domain?: string, order?: number, corroboration?: number, publishedAt?: string}[]} items
|
|
155
|
-
* @param {{appliesFreshness?: boolean}} [options] appliesFreshness 仅在本次搜索带时间过滤时为 true
|
|
156
|
-
*/
|
|
157
|
-
export function rankMergedResults(items, { appliesFreshness = false } = {}) {
|
|
158
|
-
const list = Array.isArray(items) ? items.slice() : [];
|
|
159
|
-
if (list.length <= 1) return list;
|
|
160
|
-
|
|
161
|
-
const withinKey = (x) => [
|
|
162
|
-
appliesFreshness ? -publishedTs(x?.publishedAt) : 0,
|
|
163
|
-
Number(x?.order) || 0,
|
|
164
|
-
-(Number(x?.corroboration) || 0),
|
|
165
|
-
];
|
|
166
|
-
const cmp = (a, b) => {
|
|
167
|
-
const ka = withinKey(a);
|
|
168
|
-
const kb = withinKey(b);
|
|
169
|
-
for (let i = 0; i < ka.length; i++) {
|
|
170
|
-
if (ka[i] !== kb[i]) return ka[i] - kb[i];
|
|
171
|
-
}
|
|
172
|
-
return 0;
|
|
173
|
-
};
|
|
174
|
-
|
|
175
|
-
const byDomain = new Map();
|
|
176
|
-
for (const item of list) {
|
|
177
|
-
const d = String(item?.domain ?? "");
|
|
178
|
-
if (!byDomain.has(d)) byDomain.set(d, []);
|
|
179
|
-
byDomain.get(d).push(item);
|
|
180
|
-
}
|
|
181
|
-
|
|
182
|
-
const groups = [...byDomain.values()];
|
|
183
|
-
for (const group of groups) group.sort(cmp);
|
|
184
|
-
groups.sort((a, b) => cmp(a[0], b[0]));
|
|
185
|
-
|
|
186
|
-
const out = [];
|
|
187
|
-
const longest = groups.reduce((max, g) => Math.max(max, g.length), 0);
|
|
188
|
-
for (let round = 0; round < longest; round++) {
|
|
189
|
-
for (const group of groups) {
|
|
190
|
-
if (round < group.length) out.push(group[round]);
|
|
191
|
-
}
|
|
192
|
-
}
|
|
193
|
-
return out;
|
|
194
|
-
}
|
|
195
|
-
|
|
196
|
-
/**
|
|
197
|
-
* 引擎 → 必需凭据描述符(`{ env, settings }`,即 resolveApiKey 的两个参数)。
|
|
198
|
-
* 数组表示「任一命中即算配了」,与 index.js 里 resolveDoubaoKey / resolveGeminiKey
|
|
199
|
-
* 的回退逻辑逐条对应。
|
|
200
|
-
*
|
|
201
|
-
* 为什么需要它:#69 §2 要求引擎资格必须考虑 "Missing credentials"。原状的资格过滤
|
|
202
|
-
* 只看了 disabled / 冷却 / zhihu 预探测,缺 key 的引擎要到 runEngineTest 才发现 ——
|
|
203
|
-
* 而那时它已经占掉了并发上限的名额。本机默认池 [bing, baidu, aliyun, anysearch] 里
|
|
204
|
-
* baidu/aliyun 都没 key,上限设 3 时取走的正是这三个不可用的,唯一能跑的 anysearch
|
|
205
|
-
* 被切掉,整次 multi 直接失败。
|
|
206
|
-
*
|
|
207
|
-
* 收录纪律(有交叉校验断言钉住):**只收 index.js 里 runEngineTest 显式
|
|
208
|
-
* `if (!key) return { error: "<ENV> not configured" }` 的引擎**,与代码自身对
|
|
209
|
-
* 「缺 key」的声明严格一致。
|
|
210
|
-
* - 免 key 引擎 ddg/ddg-lite/bing/searxng 不收;
|
|
211
|
-
* - 有免 key 兜底的 anysearch(key 可空)/ exa、parallel(回落 MCP)不收;
|
|
212
|
-
* - tavily/keenable/firecrawl 分支没有 not configured 检查(缺 key 时会带着空串去请求,
|
|
213
|
-
* 以鉴权失败收场),暂不收——那属于另一个行为变更,需单独决定。
|
|
214
|
-
* settings 字段必须带上:只在环境变量里找 key 会漏掉「仅填在设置页」的用户。
|
|
215
|
-
*/
|
|
216
|
-
export const REQUIRED_CREDENTIALS = {
|
|
217
|
-
perplexity: [{ env: "PERPLEXITY_API_KEY", settings: "perplexityApiKey" }],
|
|
218
|
-
"deepseek-official": [{ env: "DEEPSEEK_API_KEY", settings: "deepseekApiKey" }],
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
{ env: "
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
{ env: "
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
}
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
}
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
*
|
|
251
|
-
*
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
*
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
const
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
}
|
|
1
|
+
// #69 Phase 1(第一批):multi 模式的执行策略——纯函数,便于离线断言。
|
|
2
|
+
//
|
|
3
|
+
// 为什么单独成文件:这几个判断(并发上限怎么取、目标成功数怎么补跑)跑在
|
|
4
|
+
// runMultiSearch 里,而那是 apply() 内的闭包、无法导出单测。抽成纯函数后
|
|
5
|
+
// 既可以直接断言,也符合 reporter「三个预设共用同一条执行管线」的方向
|
|
6
|
+
// (将来加 Fast/Balanced/Deep 时,这里就是预设唯一的作用点)。
|
|
7
|
+
//
|
|
8
|
+
// 两条约束来自 reporter #69,都有断言钉住:
|
|
9
|
+
// §1 并发上限是「天花板」不是「必须跑满」——候选不够就少跑,不得凑数。
|
|
10
|
+
// §3 目标成功数(minSources)是「尽量达到」而非硬下限——达不到时返回已有
|
|
11
|
+
// 结果并附提示,绝不丢弃有效证据、绝不因此让整次搜索失败。
|
|
12
|
+
// 另有兼容硬约束:默认值必须让现有行为完全不变,升级对用户透明。
|
|
13
|
+
|
|
14
|
+
/** 0 / 非数字 / 负数 一律视为「不限制」,保证未配置时行为与今天一致。 */
|
|
15
|
+
export function resolveMaxEngines(value, poolSize) {
|
|
16
|
+
const n = Number(value);
|
|
17
|
+
if (!Number.isFinite(n) || n <= 0) return poolSize;
|
|
18
|
+
return Math.min(Math.floor(n), poolSize);
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
/**
|
|
22
|
+
* 从候选池里挑出本次实际执行的引擎。
|
|
23
|
+
* 只做「取前 N」,保持池内顺序——池内顺序就是用户在 zhPool/enPool 里配的优先级,
|
|
24
|
+
* 未经用户许可不重排(轮换是后续批次的事)。
|
|
25
|
+
*/
|
|
26
|
+
export function selectEngines(engines, configuredMax) {
|
|
27
|
+
const list = Array.isArray(engines) ? engines.filter((e) => typeof e === "string" && e) : [];
|
|
28
|
+
if (list.length === 0) return [];
|
|
29
|
+
return list.slice(0, resolveMaxEngines(configuredMax, list.length));
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
/** 目标成功源数:至少 1(0/负数/非数字都归 1,因为「0 个成功源」没有意义)。 */
|
|
33
|
+
export function resolveMinSources(value, poolSize) {
|
|
34
|
+
const n = Number(value);
|
|
35
|
+
if (!Number.isFinite(n) || n <= 0) return 1;
|
|
36
|
+
return Math.max(1, Math.floor(n));
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
/**
|
|
40
|
+
* 主轮结束后,若成功引擎数未达目标,从链路里挑「还没试过」的候选补跑。
|
|
41
|
+
* 已达标时返回空数组——调用方据此一次额外请求都不发(省钱、省限流)。
|
|
42
|
+
*/
|
|
43
|
+
export function planTopUp({ attempted = [], succeeded = 0, minSources = 1, chain = [] } = {}) {
|
|
44
|
+
if (succeeded >= minSources) return [];
|
|
45
|
+
const tried = new Set(Array.isArray(attempted) ? attempted : []);
|
|
46
|
+
const candidates = Array.isArray(chain) ? chain : [];
|
|
47
|
+
return candidates.filter((engine) => engine && !tried.has(engine));
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
/** 未达标时的说明文字;达标返回 null(不制造噪音)。 */
|
|
51
|
+
export function describeTarget({ minSources = 1, achieved = 0 } = {}) {
|
|
52
|
+
if (achieved >= minSources) return null;
|
|
53
|
+
return `sourcesTarget: ${minSources}, achieved: ${achieved}`;
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
/**
|
|
57
|
+
* 执行档位(#69 §1)。**只复用现有两个旋钮**:并发上限 + 目标成功源数。
|
|
58
|
+
* 不新增后端配置——UI 点档位就是把这两个数字写进去,手改数字即回到「自定义」。
|
|
59
|
+
* 这样零迁移、零缓存键影响,且在 planner 统一之前不改变任何现有语义。
|
|
60
|
+
*/
|
|
61
|
+
export const MULTI_PRESETS = {
|
|
62
|
+
fast: { maxEngines: 1, minSources: 1 },
|
|
63
|
+
balanced: { maxEngines: 3, minSources: 1 },
|
|
64
|
+
deep: { maxEngines: 6, minSources: 2 },
|
|
65
|
+
};
|
|
66
|
+
|
|
67
|
+
/** 由当前 (上限, 目标源数) 反推档位名;不匹配任何档位时返回 "custom"。 */
|
|
68
|
+
export function presetForSettings(maxEngines, minSources) {
|
|
69
|
+
const max = Number(maxEngines);
|
|
70
|
+
const min = Number(minSources);
|
|
71
|
+
const normMax = Number.isFinite(max) && max > 0 ? Math.floor(max) : 0;
|
|
72
|
+
const normMin = Number.isFinite(min) && min > 0 ? Math.floor(min) : 1;
|
|
73
|
+
for (const [id, p] of Object.entries(MULTI_PRESETS)) {
|
|
74
|
+
if (p.maxEngines === normMax && p.minSources === normMin) return id;
|
|
75
|
+
}
|
|
76
|
+
return "custom";
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
/**
|
|
80
|
+
* 常见「两段式」公共后缀(co.uk / com.cn / co.jp …)。不追求完整 PSL——
|
|
81
|
+
* 只覆盖公开搜索里真会撞到的那批;未命中时退化为「最后两段」。
|
|
82
|
+
*/
|
|
83
|
+
const MULTI_PART_SUFFIXES = new Set([
|
|
84
|
+
// 中国 / 港台
|
|
85
|
+
"com.cn", "net.cn", "org.cn", "gov.cn", "edu.cn", "ac.cn",
|
|
86
|
+
"com.hk", "org.hk", "net.hk", "edu.hk", "gov.hk",
|
|
87
|
+
"com.tw", "org.tw", "net.tw", "edu.tw", "gov.tw",
|
|
88
|
+
// 日韩
|
|
89
|
+
"co.jp", "or.jp", "ne.jp", "ac.jp", "go.jp",
|
|
90
|
+
"co.kr", "or.kr", "ne.kr", "go.kr", "re.kr",
|
|
91
|
+
// 英国 / 英联邦 / 大洋洲
|
|
92
|
+
"co.uk", "org.uk", "ac.uk", "gov.uk", "me.uk", "net.uk", "sch.uk",
|
|
93
|
+
"com.au", "net.au", "org.au", "edu.au", "gov.au",
|
|
94
|
+
"co.nz", "net.nz", "org.nz", "govt.nz", "ac.nz",
|
|
95
|
+
"co.za", "org.za", "net.za", "gov.za",
|
|
96
|
+
"co.in", "net.in", "org.in", "gov.in", "ac.in",
|
|
97
|
+
"com.sg", "com.my", "com.ph", "com.pk", "com.vn", "co.th", "co.id",
|
|
98
|
+
// 欧洲 / 拉美 / 中东 / 非洲
|
|
99
|
+
"com.br", "com.mx", "com.ar", "com.tr", "com.ua", "com.pl", "com.ru",
|
|
100
|
+
"co.il", "com.sa", "com.eg", "com.ng", "co.ke",
|
|
101
|
+
]);
|
|
102
|
+
|
|
103
|
+
/**
|
|
104
|
+
* 取 URL 的**可注册域**(eTLD+1)用于「同域」判定。
|
|
105
|
+
*
|
|
106
|
+
* 为什么不用主机名:blog.example.com 与 download.example.com 是同一站点的两个子域,
|
|
107
|
+
* 拿主机名分组会把它们当成两个「独立来源」,与 #69 §4 的域名多样性意图正好相反。
|
|
108
|
+
* (这正是 0.8.10 里自认的 known limitation。)
|
|
109
|
+
*
|
|
110
|
+
* IP / 含端口的 IPv6 / localhost / 单标签主机 / 非法 URL 一律原样小写返回,不误判。
|
|
111
|
+
*/
|
|
112
|
+
export function domainOf(url) {
|
|
113
|
+
const raw = String(url ?? "").trim();
|
|
114
|
+
if (!raw) return "";
|
|
115
|
+
let host;
|
|
116
|
+
try {
|
|
117
|
+
host = new URL(raw).hostname.toLowerCase();
|
|
118
|
+
} catch {
|
|
119
|
+
return raw.toLowerCase();
|
|
120
|
+
}
|
|
121
|
+
host = host.replace(/\.+$/, "").replace(/^www\./, "");
|
|
122
|
+
if (!host) return "";
|
|
123
|
+
if (!host.includes(".") || host.includes(":") || /^\d{1,3}(\.\d{1,3}){3}$/.test(host)) return host;
|
|
124
|
+
const parts = host.split(".");
|
|
125
|
+
if (parts.length <= 2) return host;
|
|
126
|
+
const last2 = parts.slice(-2).join(".");
|
|
127
|
+
return MULTI_PART_SUFFIXES.has(last2) ? parts.slice(-3).join(".") : last2;
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
function publishedTs(value) {
|
|
131
|
+
const ts = Date.parse(String(value ?? ""));
|
|
132
|
+
return Number.isFinite(ts) ? ts : 0;
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
/**
|
|
136
|
+
* 合并后的排序(#69 §4)。
|
|
137
|
+
*
|
|
138
|
+
* reporter 的要求:排序信号用「真实可得且有意义」的那些——原始结果顺序、域名多样性、
|
|
139
|
+
* 需要时的时效性;而 `seenIn` **只能当溯源/佐证元数据,不能当主排序键**,理由是
|
|
140
|
+
* "Six matching search results do not necessarily constitute six independent sources"
|
|
141
|
+
* (多个引擎可能共享索引与排序偏见,所以同域堆叠并不等于可信)。
|
|
142
|
+
*
|
|
143
|
+
* 做法(确定性、可离线断言):
|
|
144
|
+
* 1. 按域名分组;
|
|
145
|
+
* 2. 组内次序:时效性生效时新的优先 → 原始顺序 → 佐证数(仅末位 tiebreak);
|
|
146
|
+
* 3. 组的次序:按各域「最佳条目」的上述次序;
|
|
147
|
+
* 4. 轮转交错输出:第一轮每域各出一条,再第二轮…… → 前 N 条尽量来自不同域。
|
|
148
|
+
*
|
|
149
|
+
* 刻意只重排、不剔除:多样性是排序信号,不是过滤条件(过滤由 blockedDomains 负责)。
|
|
150
|
+
*
|
|
151
|
+
* 同域判定走 domainOf() 的 eTLD+1:blog.example.com 与 download.example.com 归为同一域。
|
|
152
|
+
* 局限:后缀表是精简版(非完整 PSL),冷门多段后缀仍退化为「最后两段」。
|
|
153
|
+
*
|
|
154
|
+
* @param {{domain?: string, order?: number, corroboration?: number, publishedAt?: string}[]} items
|
|
155
|
+
* @param {{appliesFreshness?: boolean}} [options] appliesFreshness 仅在本次搜索带时间过滤时为 true
|
|
156
|
+
*/
|
|
157
|
+
export function rankMergedResults(items, { appliesFreshness = false } = {}) {
|
|
158
|
+
const list = Array.isArray(items) ? items.slice() : [];
|
|
159
|
+
if (list.length <= 1) return list;
|
|
160
|
+
|
|
161
|
+
const withinKey = (x) => [
|
|
162
|
+
appliesFreshness ? -publishedTs(x?.publishedAt) : 0,
|
|
163
|
+
Number(x?.order) || 0,
|
|
164
|
+
-(Number(x?.corroboration) || 0),
|
|
165
|
+
];
|
|
166
|
+
const cmp = (a, b) => {
|
|
167
|
+
const ka = withinKey(a);
|
|
168
|
+
const kb = withinKey(b);
|
|
169
|
+
for (let i = 0; i < ka.length; i++) {
|
|
170
|
+
if (ka[i] !== kb[i]) return ka[i] - kb[i];
|
|
171
|
+
}
|
|
172
|
+
return 0;
|
|
173
|
+
};
|
|
174
|
+
|
|
175
|
+
const byDomain = new Map();
|
|
176
|
+
for (const item of list) {
|
|
177
|
+
const d = String(item?.domain ?? "");
|
|
178
|
+
if (!byDomain.has(d)) byDomain.set(d, []);
|
|
179
|
+
byDomain.get(d).push(item);
|
|
180
|
+
}
|
|
181
|
+
|
|
182
|
+
const groups = [...byDomain.values()];
|
|
183
|
+
for (const group of groups) group.sort(cmp);
|
|
184
|
+
groups.sort((a, b) => cmp(a[0], b[0]));
|
|
185
|
+
|
|
186
|
+
const out = [];
|
|
187
|
+
const longest = groups.reduce((max, g) => Math.max(max, g.length), 0);
|
|
188
|
+
for (let round = 0; round < longest; round++) {
|
|
189
|
+
for (const group of groups) {
|
|
190
|
+
if (round < group.length) out.push(group[round]);
|
|
191
|
+
}
|
|
192
|
+
}
|
|
193
|
+
return out;
|
|
194
|
+
}
|
|
195
|
+
|
|
196
|
+
/**
|
|
197
|
+
* 引擎 → 必需凭据描述符(`{ env, settings }`,即 resolveApiKey 的两个参数)。
|
|
198
|
+
* 数组表示「任一命中即算配了」,与 index.js 里 resolveDoubaoKey / resolveGeminiKey
|
|
199
|
+
* 的回退逻辑逐条对应。
|
|
200
|
+
*
|
|
201
|
+
* 为什么需要它:#69 §2 要求引擎资格必须考虑 "Missing credentials"。原状的资格过滤
|
|
202
|
+
* 只看了 disabled / 冷却 / zhihu 预探测,缺 key 的引擎要到 runEngineTest 才发现 ——
|
|
203
|
+
* 而那时它已经占掉了并发上限的名额。本机默认池 [bing, baidu, aliyun, anysearch] 里
|
|
204
|
+
* baidu/aliyun 都没 key,上限设 3 时取走的正是这三个不可用的,唯一能跑的 anysearch
|
|
205
|
+
* 被切掉,整次 multi 直接失败。
|
|
206
|
+
*
|
|
207
|
+
* 收录纪律(有交叉校验断言钉住):**只收 index.js 里 runEngineTest 显式
|
|
208
|
+
* `if (!key) return { error: "<ENV> not configured" }` 的引擎**,与代码自身对
|
|
209
|
+
* 「缺 key」的声明严格一致。
|
|
210
|
+
* - 免 key 引擎 ddg/ddg-lite/bing/searxng 不收;
|
|
211
|
+
* - 有免 key 兜底的 anysearch(key 可空)/ exa、parallel(回落 MCP)不收;
|
|
212
|
+
* - tavily/keenable/firecrawl 分支没有 not configured 检查(缺 key 时会带着空串去请求,
|
|
213
|
+
* 以鉴权失败收场),暂不收——那属于另一个行为变更,需单独决定。
|
|
214
|
+
* settings 字段必须带上:只在环境变量里找 key 会漏掉「仅填在设置页」的用户。
|
|
215
|
+
*/
|
|
216
|
+
export const REQUIRED_CREDENTIALS = {
|
|
217
|
+
perplexity: [{ env: "PERPLEXITY_API_KEY", settings: "perplexityApiKey" }],
|
|
218
|
+
"deepseek-official": [{ env: "DEEPSEEK_API_KEY", settings: "deepseekApiKey" }],
|
|
219
|
+
"deepseek-tools": [{ env: "DEEPSEEK_API_KEY", settings: "deepseekApiKey" }],
|
|
220
|
+
serpbase: [{ env: "SERPBASE_API_KEY", settings: "serpbaseApiKey" }],
|
|
221
|
+
serply: [{ env: "SERPLY_API_KEY", settings: "serplyApiKey" }],
|
|
222
|
+
tinyfish: [{ env: "TINYFISH_API_KEY", settings: "tinyfishApiKey" }],
|
|
223
|
+
jina: [{ env: "JINA_API_KEY", settings: "jinaApiKey" }],
|
|
224
|
+
baidu: [{ env: "BAIDU_API_KEY", settings: "baiduApiKey" }],
|
|
225
|
+
kimi: [{ env: "MOONSHOT_API_KEY", settings: "kimiApiKey" }],
|
|
226
|
+
aliyun: [{ env: "DASHSCOPE_API_KEY", settings: "aliyunApiKey" }],
|
|
227
|
+
doubao: [
|
|
228
|
+
{ env: "DOUBAO_SEARCH_API_KEY", settings: "doubaoApiKey" },
|
|
229
|
+
{ env: "WEB_SEARCH_API_KEY", settings: null },
|
|
230
|
+
],
|
|
231
|
+
zhihu_global: [{ env: "ZHIHU_API_KEY", settings: "zhihuApiKey" }],
|
|
232
|
+
zhihu_site: [{ env: "ZHIHU_API_KEY", settings: "zhihuApiKey" }],
|
|
233
|
+
openai: [{ env: "OPENAI_API_KEY", settings: "openaiApiKey" }],
|
|
234
|
+
gemini: [
|
|
235
|
+
{ env: "GEMINI_API_KEY", settings: "geminiApiKey" },
|
|
236
|
+
{ env: "GOOGLE_API_KEY", settings: null },
|
|
237
|
+
],
|
|
238
|
+
claude: [{ env: "ANTHROPIC_API_KEY", settings: "claudeApiKey" }],
|
|
239
|
+
you: [{ env: "YOUCOM_API_KEY", settings: "youcomApiKey" }],
|
|
240
|
+
};
|
|
241
|
+
|
|
242
|
+
/** 该引擎需要哪些凭据描述符;不需要凭据(免 key / 有兜底)时返回 null。 */
|
|
243
|
+
export function credentialRequirement(engine) {
|
|
244
|
+
const req = REQUIRED_CREDENTIALS[engine];
|
|
245
|
+
if (!req) return null;
|
|
246
|
+
return req.map((d) => ({ ...d }));
|
|
247
|
+
}
|
|
248
|
+
|
|
249
|
+
/**
|
|
250
|
+
* 引擎凭据是否齐备。`isAvailable(engine)` 由调用方提供(服务端负责真正解析凭据),
|
|
251
|
+
* 这里只做判定与分流,保持纯函数、可离线断言。
|
|
252
|
+
* 未知引擎一律算合格——宁放过,不误杀未来的新引擎。
|
|
253
|
+
*/
|
|
254
|
+
export function isCredentialEligible(engine, isAvailable) {
|
|
255
|
+
if (!credentialRequirement(engine)) return true;
|
|
256
|
+
return Boolean(isAvailable && isAvailable(engine));
|
|
257
|
+
}
|
|
258
|
+
|
|
259
|
+
/**
|
|
260
|
+
* 按凭据把候选池分成「合格」与「缺凭据」两份,顺序均保持原样。
|
|
261
|
+
* 调用方应把 missing 记进 skipped 备注,让用户仍然看得见某引擎为什么没跑。
|
|
262
|
+
*/
|
|
263
|
+
export function partitionByCredentials(engines, isAvailable) {
|
|
264
|
+
const eligible = [];
|
|
265
|
+
const missing = [];
|
|
266
|
+
for (const engine of Array.isArray(engines) ? engines : []) {
|
|
267
|
+
if (isCredentialEligible(engine, isAvailable)) eligible.push(engine);
|
|
268
|
+
else missing.push(engine);
|
|
269
|
+
}
|
|
270
|
+
return { eligible, missing };
|
|
271
|
+
}
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-free-search",
|
|
3
|
-
"version": "0.9.
|
|
4
|
-
"description": "Free web search for DeepSeek Harness:
|
|
3
|
+
"version": "0.9.2",
|
|
4
|
+
"description": "Free web search for DeepSeek Harness: 27 engines (Bing/DuckDuckGo/AnySearch/SearXNG/Exa/Tavily/Keenable/Firecrawl/Parallel keyless; Zhihu global/site via MCP and Perplexity/SerpBase/DeepSeek (official + native tools/search)/You.com/Baidu/Kimi/Aliyun/Doubao with key; OpenAI/Gemini/Claude model built-in search, explicit-only) + auto smart routing + global engine enable/disable & fallback priority + editable zh/en language pools + multi mode (parallel full language-pool merge) + multi_search cross-source tool + failure-aware fallback (quota/auth cooldown, transient retry, fallbackOn) + time filtering + platform search + domain blocklist (blockedDomains) + web_fetch, with web settings UI.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "lib/index.js",
|
|
7
7
|
"exports": {
|