@240xu/dsh-websearch 2.7.3 → 2.8.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -115,6 +115,36 @@ pnpm install
115
115
 
116
116
  环境变量兜底:未在 credentials 配置时回退读同名环境变量;`DSH_UNIFIED_SEARCH_BACKENDS` 可逗号分隔强制指定启用集合。
117
117
 
118
+ ## v2.8.1 SettingsForms 适配 | What's new in v2.8.1
119
+
120
+ - **0.2.0 设置面板表单**:Config 标记 `meta.volatile`——0.2.0 SettingsForms 从插件 Config 自动生成表单,但 volatileForm 只保留 volatile 字段(此前 websearch 表单为空);现整棵 Config 成为 0.2.0 表单。裁决与考证见 dsh-plugin-hub/docs/reviews/cross-settings-forms.md。
121
+ - 删除 0.2.0 线上与可见 UI 自相矛盾的 "no installSection" console.warn(0.1.5 路径不变)。
122
+ - 补齐全部裸字段(×ApiKeyEnv/×BaseURL/×Model)的 description——0.2.0 表单渲染唯一用户可见文案来自 meta.description。
123
+
124
+ ## v2.8.0 查全查多 | What's new in v2.8.0
125
+
126
+ 面向「查的多一点、查的全一点」的检索深度升级,全部 feature-gated、默认关:
127
+
128
+ ### 1. multiQuery 多查询 RRF 融合(`multiQueryEnabled`,默认关)
129
+
130
+ - 门控启发式(刻意从严):查询长度 > 60 字符,或含 `vs / and / 比较 / 对比 / 和` 分隔词,才触发;简单查询单扇出,行为与旧版完全一致(arXiv:2404.01037:盲目多查询会劣化)。
131
+ - 触发后派生 ≤3 个子查询(原查询永远参与融合且保持全权重),各自走完整 11 后端扇出,再用 RRF(k=60)融合:出现在多个子查询结果里的条目被提升,单列表独占条目按位次衰减;URL 去重、首见元数据保留。
132
+ - 遥测行合并各变体并截断至 12 条,避免淹没结果。
133
+
134
+ ### 2. deepCoverage 深度覆盖(`deepCoverage`,默认关)
135
+
136
+ - 每后端请求条数上调至 `ceil(maxResults × 1.5)`,全局去重后仍按 maxResults 截断——牺牲带宽换覆盖面。
137
+ - 后端能力适配(feature-detect,不破坏旧参数):Tavily 自动切 `search_depth: advanced`(2 credits,多 snippet/URL);SearXNG 类目扩为 `general,news,it`(Search API `categories` 逗号列表)。
138
+ - Exa:`inferExaCategory` 从查询推断 data category(github / research paper / news,仅高置信注入)。
139
+
140
+ ### 3. 运维可观测(P2)
141
+
142
+ - `GET /api/websearch/history` 响应新增 `backends` 字段:每后端最近真实成败/延迟/熔断状态(`{id, ok, ms, at, failCount, cooled}`,无错误串无凭证),兜底冗余度可观测。
143
+
144
+ ### 4. per-backend 超时上限(P2)
145
+
146
+ - `ddg` / `searxng` 专属超时上限 5s(`min(5s, backendTimeoutMs)`,全局默认仍 30s、配置更短则以配置为准)——不可达后端不再拖住整个扇出尾。实测见 docs/real-call-2.8.0.md。
147
+
118
148
  ## v2.7.3 兼容修复 | What's new in v2.7.3
119
149
 
120
150
  - **DSH 0.1.7 客户端兼容(P0)**:settingsScope 客户端服务在 0.1.7 被移除;现改为 feature-detect——0.1.5(或装 dsh-settings-scope-shim)行为不变,0.1.7 无 shim 时设置卡片优雅停用(console.warn 指引),客户端不再静默全死。
@@ -216,7 +246,9 @@ v2.7.0 缓存/熔断/历史三项的依据与刻意取舍(含查证过的反
216
246
  1. **熔断器(Circuit Breaker)**:出处 Michael T. Nygard《Release It!》(Pragmatic Bookshelf, 2007) 与 Microsoft Azure Architecture Center「Circuit Breaker pattern」https://learn.microsoft.com/en-us/azure/architecture/patterns/circuit-breaker 。经典状态机是 Closed → Open → Half-Open(半开态放行少量探测请求)。**本插件刻意省略半开态**:冷却到期即视为 Closed(下一次真实搜索就是探测),失败则立即重新开窗——因为这里的"请求"是一次多后端扇出中的单个后端调用,探测请求必然真实发生且失败成本只是一次软降级,无需额外探测计数器;同时保留 fail-open 兜底(全部冷却时放行),比经典模式更保守。
217
247
  2. **缓存失效(TTL-only vs 事件驱动)**:Cloudflare「Retention vs Freshness (TTL)」https://developers.cloudflare.com/cache/concepts/retention-vs-freshness/ 与「Revalidation」https://developers.cloudflare.com/cache/concepts/revalidation/ 。搜索结果没有可靠的原站变更信号可订阅(网页索引持续变化),事件驱动失效无锚点;TTL-only + 把 TTL 做成设置项(默认 900s)+ key 纳入后端组合,是"可接受的最终一致性"。取舍:900s 内索引更新不可见——对搜索场景可接受,且用户可调低。
218
248
  3. **原子写(tmp + renameSync)**:POSIX 上 rename(2) 原子替换(Node 文档:newPath 已存在时覆盖);Windows 上 libuv 将 fs.rename 映射为 MoveFileExW(MOVEFILE_REPLACE_EXISTING)(libuv#283 https://github.com/joyent/libuv/issues/283 ,LWN 讨论 https://lwn.net/Articles/682988/ 指出 MS 文档不承诺该调用的原子性)。**结论:POSIX 双端原子;Windows 实践上等价但官方文档不背书**——因此本插件在写入端额外做了兜底:读端永远校验 JSON 完整性、坏条目按 miss 处理并删除,即使 Windows 上出现半写状态也不会返回坏数据。
219
- 4. **搜索历史隐私**:Article 29 Working Party 意见书 WP148 https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2008/wp148_en.pdf 认定查询日志(query + 时间 + 来源标识)属个人数据;EFF 六条自保建议 https://w2.eff.org/Privacy/search/searchtips.pdf 的第一条是"别在搜索词里放 PII"——说明 query 天然可能携带 PII(人名、账号、地址)。本插件的取舍:历史只存 {query, time, resultCount, backendsOk/Total}(无 IP、无 cookie、无结果 URL)、环形 50 条自动滚动即短保留、本机存储不出网、提供 historyEnabled 开关与 POST clear 端点。**未做 query 脱敏**:脱敏会破坏"查看最近搜了什么"的核心用途,且本机单用户场景下该数据本就在 DSH 会话日志里存在;若未来历史跨设备同步,脱敏/加密是上线前置条件。
249
+ 4b. **查全查多(v2.8.0)**:Tavily Search API(`search_depth: advanced` 2 credits、`max_results` 0-20、`include_answer`、topic=news 时间加权)https://docs.tavily.com/api-reference/endpoint/search ;Exa Search API(type 分类检索、category 枚举、livecrawl;注意 2026 版 coding-agent 指南声明 neural 为 legacy 术语、且告诫勿臆造 category 值——本插件只注入 OpenAPI 枚举内的高置信值)https://exa.ai/docs/reference/search-api-guide-for-coding-agents 与 https://github.com/exa-labs/openapi-spec/blob/master/exa-openapi-spec.yaml ;SearXNG Search API(`categories` 逗号多类目、`language`)https://docs.searxng.org/dev/search_api.html ;Brave Search API(`freshness`、`result_filter`,本轮仅调研未接线)https://api-dashboard.search.brave.com/app/documentation 。多查询融合:RRF 出自 Cormack et al. 2009,RAG-Fusion 实现 https://github.com/Raudaschl/rag-fusion ;HyDE https://arxiv.org/abs/2212.10496 ;**门控依据** arXiv:2404.01037(盲目多查询劣化)——因此 multiQuery 默认关、启发式从严、原查询全权重参与。
250
+ 4c. **per-backend 超时**:不可达主机的 TCP 连接失败不受应用层超时保护,只能靠 abort 计时器兜底;ddg(HTML 抓取)与 SearXNG(公共实例无 SLA)是拖尾主力,5s 上限取「P50 成功延迟的一个数量级」。
251
+ 4d. **搜索历史隐私**:Article 29 Working Party 意见书 WP148 https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2008/wp148_en.pdf 认定查询日志(query + 时间 + 来源标识)属个人数据;EFF 六条自保建议 https://w2.eff.org/Privacy/search/searchtips.pdf 的第一条是"别在搜索词里放 PII"——说明 query 天然可能携带 PII(人名、账号、地址)。本插件的取舍:历史只存 {query, time, resultCount, backendsOk/Total}(无 IP、无 cookie、无结果 URL)、环形 50 条自动滚动即短保留、本机存储不出网、提供 historyEnabled 开关与 POST clear 端点。**未做 query 脱敏**:脱敏会破坏"查看最近搜了什么"的核心用途,且本机单用户场景下该数据本就在 DSH 会话日志里存在;若未来历史跨设备同步,脱敏/加密是上线前置条件。
220
252
 
221
253
  ## License | 许可
222
254
 
package/cordis.patch.yml CHANGED
@@ -1,3 +1,13 @@
1
+ # Bundle patch layer:
2
+ # 1. insert the plugin row (registers the unified search provider)
3
+ # 2. wire the built-in web_search tool to the unified provider (the plugin's
4
+ # whole reason to exist — previously a manual cordis.patch.yml edit, now
5
+ # automatic; revert = remove this row or disable the plugin, the base
6
+ # default (deepseek-official) is one yaml line away).
1
7
  - insert:
2
8
  - id: dsh-websearch
3
9
  name: '@240xu/dsh-websearch'
10
+ - id: web
11
+ name: '@deepseek-ai/dsh-web'
12
+ config:
13
+ searchProvider: unified
@@ -13,6 +13,22 @@ import { WebError } from "@deepseek-ai/dsh-web";
13
13
  const EXA_ENDPOINT = "https://mcp.exa.ai/mcp";
14
14
  const EXA_TOOL = "web_search_exa";
15
15
 
16
+ /**
17
+ * Infer an Exa data category from the query (v2.8.0). Only high-confidence
18
+ * patterns map — Exa rejects unknown categories with a 400, so this returns
19
+ * undefined unless a signal is explicit. Valid values per the Exa OpenAPI
20
+ * enum: github / "research paper" / news / pdf / company / people /
21
+ * "personal site" / "financial report".
22
+ */
23
+ export function inferExaCategory(query) {
24
+ const q = typeof query === "string" ? query.toLowerCase() : "";
25
+ if (!q) return undefined;
26
+ if (/\bgithub\b|\brepo\b|repository|开源项目|源码/.test(q)) return "github";
27
+ if (/\barxiv\b|\bpaper\b|论文|研究文献|preprint/.test(q)) return "research paper";
28
+ if (/\bnews\b|新闻|最新消息|动态/.test(q)) return "news";
29
+ return undefined;
30
+ }
31
+
16
32
  export const exaBackend = {
17
33
  id: "exa",
18
34
  requiresCredential: false,
@@ -26,6 +42,10 @@ export const exaBackend = {
26
42
  query: request.query,
27
43
  numResults: request.maxResults ?? 8,
28
44
  };
45
+ // v2.8.0: category inference (feature-detect — only sent when confident;
46
+ // if the endpoint rejects it the fan-out demotes softly).
47
+ const category = inferExaCategory(request.query);
48
+ if (category) args.category = category;
29
49
  hooks.recordRequest?.({
30
50
  endpoint: EXA_ENDPOINT,
31
51
  toolName: EXA_TOOL,
@@ -33,7 +33,9 @@ export const searxngBackend = {
33
33
  url.pathname = url.pathname.replace(/\/+$/,"") + "/search";
34
34
  url.searchParams.set("q", request.query);
35
35
  url.searchParams.set("format", "json");
36
- url.searchParams.set("categories", "general");
36
+ // deepCoverage widens the category sweep (docs.searxng.org Search API:
37
+ // categories is a comma-separated list) — breadth over one category.
38
+ url.searchParams.set("categories", request.deepCoverage === true ? "general,news,it" : "general");
37
39
  // Language: settings/request filter wins, else historical "en" default.
38
40
  const filters = normalizeFilters(request.filters);
39
41
  url.searchParams.set("language", filters.lang ?? "en");
@@ -31,7 +31,12 @@ export const tavilyBackend = {
31
31
 
32
32
  const baseURL = backendOpts.baseURL ?? TAVILY_ENDPOINT;
33
33
  const maxResults = Math.min(request.maxResults ?? 8, 20);
34
- const searchDepth = backendOpts.searchDepth ?? "basic";
34
+ // deepCoverage (provider sets request.deepCoverage) upgrades the depth:
35
+ // advanced = highest relevance, multiple snippets per URL (2 API credits).
36
+ // An explicit operator setting keeps precedence unless it was left basic.
37
+ const searchDepth = request.deepCoverage === true && backendOpts.searchDepth !== "advanced"
38
+ ? "advanced"
39
+ : (backendOpts.searchDepth ?? "basic");
35
40
 
36
41
  const body = {
37
42
  query: request.query,
package/lib/cache.js CHANGED
@@ -33,13 +33,14 @@ export const CACHE_DEFAULTS = {
33
33
  * mix must never serve a stale result produced by a different combination
34
34
  * (arch-review P1-1). Pure function; exported for tests and host reuse.
35
35
  */
36
- export function cacheKeyFor({ query, filters, maxResults, backends }) {
36
+ export function cacheKeyFor({ query, filters, maxResults, backends, mode }) {
37
37
  const backendSet = Array.isArray(backends) ? [...backends].sort().join(",") : "";
38
38
  const canonical = JSON.stringify({
39
39
  q: typeof query === "string" ? query : "",
40
40
  f: filters ?? {},
41
41
  n: Number(maxResults) || 0,
42
42
  b: backendSet,
43
+ m: typeof mode === "string" ? mode : "",
43
44
  });
44
45
  return createHash("sha256").update(canonical).digest("hex").slice(0, 32);
45
46
  }
package/lib/history.js CHANGED
@@ -115,13 +115,37 @@ export function isTrustedApiRequest(req) {
115
115
  }
116
116
  }
117
117
 
118
+ /**
119
+ * Project the module-level backend health map into a small observability
120
+ * payload (v2.8.0 P2): per-backend last real success/failure, latency and
121
+ * breaker state — no error strings, no credentials, safe for the GUI.
122
+ */
123
+ export function toBackendObservability(healthMap, nowMs = Date.now()) {
124
+ const out = [];
125
+ if (healthMap instanceof Map) {
126
+ for (const [id, e] of healthMap) {
127
+ out.push({
128
+ id,
129
+ ok: e?.ok === true,
130
+ ms: Number.isFinite(e?.ms) ? e.ms : null,
131
+ at: Number.isFinite(e?.at) ? e.at : null,
132
+ failCount: Number.isFinite(e?.failCount) ? e.failCount : 0,
133
+ cooled: Number.isFinite(e?.cooledUntil) && e.cooledUntil > nowMs,
134
+ });
135
+ }
136
+ }
137
+ return out;
138
+ }
139
+
118
140
  /**
119
141
  * Register the read-only history routes on the (late-mounted) webServer.
120
- * GET /api/websearch/history -> { ok, entries }
142
+ * GET /api/websearch/history -> { ok, entries, backends }
143
+ * backends: per-backend last real outcome/latency/breaker state (v2.8.0),
144
+ * so fallback redundancy is observable without a separate endpoint.
121
145
  * POST /api/websearch/history/clear -> { ok, cleared: true }
122
146
  * Both routes sit behind isTrustedApiRequest (loopback Host + same-site).
123
147
  */
124
- export function registerHistoryRoutes(ws, { getStore }) {
148
+ export function registerHistoryRoutes(ws, { getStore, getBackendHealth }) {
125
149
  if (!ws || typeof ws.register !== "function") return;
126
150
  const send = (res, code, body) => {
127
151
  res.writeHead(code, { "content-type": "application/json" });
@@ -144,7 +168,11 @@ export function registerHistoryRoutes(ws, { getStore }) {
144
168
  return;
145
169
  }
146
170
  const store = getStore();
147
- send(res, 200, { ok: true, entries: store ? store.list() : [] });
171
+ send(res, 200, {
172
+ ok: true,
173
+ entries: store ? store.list() : [],
174
+ backends: toBackendObservability(typeof getBackendHealth === "function" ? getBackendHealth() : getBackendHealth),
175
+ });
148
176
  }),
149
177
  });
150
178
  ws.register({
package/lib/index.js CHANGED
@@ -169,6 +169,12 @@ export const Config = z.object({
169
169
  historyEnabled: z.boolean().default(true)
170
170
  .description("记录搜索历史【历史】:只追加,最多 50 条;可在设置 → 搜索 中查看与清空"),
171
171
 
172
+ // v2.8.0 查全查多
173
+ multiQueryEnabled: z.boolean().default(false)
174
+ .description("多查询融合【查全】:复杂查询(长度>60 或含 vs/and/比较/对比/和)自动派生 2-3 个子查询,各扇出后用 RRF(k=60) 融合;默认关"),
175
+ deepCoverage: z.boolean().default(false)
176
+ .description("深度覆盖【查全】:每后端请求条数上调 50% 再全局去重截断,牺牲带宽换覆盖面;Tavily 自动切 advanced 深度,SearXNG 扩展多类目;默认关"),
177
+
172
178
  // Enable toggles (one per backend)
173
179
  enableExa: z.boolean().default(true).description(DEFAULT_LABELS.exa),
174
180
  enableParallel: z.boolean().default(true).description(DEFAULT_LABELS.parallel),
@@ -183,30 +189,48 @@ export const Config = z.object({
183
189
  enableOpenai: z.boolean().default(false).description(DEFAULT_LABELS.openai),
184
190
 
185
191
  // Credential refs (env-var names; secrets live in the credentials service)
186
- braveApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.brave),
187
- tavilyApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.tavily),
188
- serperApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.serper),
189
- mojeekApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.mojeek),
190
- deepseekApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.deepseek),
191
- anthropicApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.anthropic),
192
- openaiApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.openai),
192
+ braveApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.brave)
193
+ .description("Brave API Key 的凭证引用(填环境变量名,实际密钥存凭证库)"),
194
+ tavilyApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.tavily)
195
+ .description("Tavily API Key 的凭证引用(填环境变量名,实际密钥存凭证库)"),
196
+ serperApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.serper)
197
+ .description("Serper API Key 的凭证引用(填环境变量名,实际密钥存凭证库)"),
198
+ mojeekApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.mojeek)
199
+ .description("Mojeek API Key 的凭证引用(填环境变量名,实际密钥存凭证库)"),
200
+ deepseekApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.deepseek)
201
+ .description("Deepseek API Key 的凭证引用(填环境变量名,实际密钥存凭证库)"),
202
+ anthropicApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.anthropic)
203
+ .description("Anthropic API Key 的凭证引用(填环境变量名,实际密钥存凭证库)"),
204
+ openaiApiKeyEnv: z.string().role("credential-ref").default(DEFAULT_KEY_ENV.openai)
205
+ .description("Openai API Key 的凭证引用(填环境变量名,实际密钥存凭证库)"),
193
206
  searxngApiKeyEnv: z.string().role("credential-ref").default("SEARXNG_API_KEY")
194
- .description("SearXNG 私有实例的 API Key(公共实例留空)"),
207
+ .description("SearXNG 私有实例的 API Key 凭证引用(公共实例留空)"),
195
208
 
196
209
  // Base URLs (defaults from built-in registry)
197
- braveBaseURL: z.string().default(DEFAULT_BASE_URLS.brave),
198
- tavilyBaseURL: z.string().default(DEFAULT_BASE_URLS.tavily),
199
- serperBaseURL: z.string().default(DEFAULT_BASE_URLS.serper),
200
- mojeekBaseURL: z.string().default(DEFAULT_BASE_URLS.mojeek),
201
- searxngBaseURL: z.string().default(DEFAULT_BASE_URLS.searxng),
202
- deepseekBaseURL: z.string().default(DEFAULT_BASE_URLS.deepseek),
203
- anthropicBaseURL: z.string().default(DEFAULT_BASE_URLS.anthropic),
204
- openaiBaseURL: z.string().default(DEFAULT_BASE_URLS.openai),
210
+ braveBaseURL: z.string().default(DEFAULT_BASE_URLS.brave)
211
+ .description("Brave API 端点(默认官方;自托管/代理实例改此项)"),
212
+ tavilyBaseURL: z.string().default(DEFAULT_BASE_URLS.tavily)
213
+ .description("Tavily API 端点(默认官方;自托管/代理实例改此项)"),
214
+ serperBaseURL: z.string().default(DEFAULT_BASE_URLS.serper)
215
+ .description("Serper.dev API 端点(默认官方;自托管/代理实例改此项)"),
216
+ mojeekBaseURL: z.string().default(DEFAULT_BASE_URLS.mojeek)
217
+ .description("Mojeek API 端点(默认官方;自托管/代理实例改此项)"),
218
+ searxngBaseURL: z.string().default(DEFAULT_BASE_URLS.searxng)
219
+ .description("SearXNG API 端点(默认官方;自托管/代理实例改此项)"),
220
+ deepseekBaseURL: z.string().default(DEFAULT_BASE_URLS.deepseek)
221
+ .description("DeepSeek API 端点(默认官方;自托管/代理实例改此项)"),
222
+ anthropicBaseURL: z.string().default(DEFAULT_BASE_URLS.anthropic)
223
+ .description("Anthropic API 端点(默认官方;自托管/代理实例改此项)"),
224
+ openaiBaseURL: z.string().default(DEFAULT_BASE_URLS.openai)
225
+ .description("OpenAI API 端点(默认官方;自托管/代理实例改此项)"),
205
226
 
206
227
  // Models
207
- deepseekModel: z.string().default(DEFAULT_MODELS.deepseek),
208
- anthropicModel: z.string().default(DEFAULT_MODELS.anthropic),
209
- openaiModel: z.string().default(DEFAULT_MODELS.openai),
228
+ deepseekModel: z.string().default(DEFAULT_MODELS.deepseek)
229
+ .description("DeepSeek 后端使用的模型名"),
230
+ anthropicModel: z.string().default(DEFAULT_MODELS.anthropic)
231
+ .description("Anthropic 后端使用的模型名"),
232
+ openaiModel: z.string().default(DEFAULT_MODELS.openai)
233
+ .description("OpenAI 后端使用的模型名"),
210
234
  tavilySearchDepth: z.union(["basic", "advanced"]).default("basic")
211
235
  .description("Tavily 搜索深度"),
212
236
 
@@ -215,6 +239,15 @@ export const Config = z.object({
215
239
  .description("显式启用后端列表(优先于上方开关;留空则用开关)"),
216
240
  });
217
241
 
242
+ // 0.2.0 SettingsForms adapter (cross-settings-forms.md ruling): the 0.2.0
243
+ // settings service auto-generates its form from entry.fiber.runtime.Config —
244
+ // but volatileForm (RT dsh-settings lib/index.js:122-138) keeps ONLY fields
245
+ // with meta.volatile; a Config without any yields an empty form. Marking the
246
+ // top object makes the whole Config the form. Direct meta assignment (not
247
+ // .volatile(), which returns a NEW schema and throws on re-wrap) is
248
+ // equivalent and idempotent on both the 0.1.5 and 0.2.0 schemastery lines.
249
+ Config.meta = { ...Config.meta, volatile: true };
250
+
218
251
  /**
219
252
  * Per-field user-language labels (fe-ui W1 remainder): the settings renderer
220
253
  * falls back to camelCase keys when no label is present, so attach one to
@@ -326,6 +359,9 @@ export function resolveOptions(ctx, config) {
326
359
  ? Math.max(0, Math.floor(Number(config?.breakerCooldownMs) || 0)) // 0 = disabled
327
360
  : BREAKER_DEFAULTS.cooldownMs,
328
361
  historyEnabled: config?.historyEnabled !== false,
362
+ // v2.8.0 coverage / multiQuery
363
+ multiQueryEnabled: config?.multiQueryEnabled === true,
364
+ deepCoverage: config?.deepCoverage === true,
329
365
  filters: {
330
366
  recency: typeof config?.recency === "string" && config.recency !== "any" ? config.recency : undefined,
331
367
  lang: language.length > 0 ? language : undefined,
@@ -359,6 +395,7 @@ export function apply(ctx, config) {
359
395
  ctx.inject(["settings"], (settingsCtx) => {
360
396
  const settings = settingsCtx.settings;
361
397
  if (typeof settings?.installSection === "function") {
398
+ // 0.1.5 SettingsProvider line: explicit section install (unchanged).
362
399
  settings.installSection(ctx, NAMESPACE, Config, config, {
363
400
  setSource: (source) => {
364
401
  current = source;
@@ -367,15 +404,10 @@ export function apply(ctx, config) {
367
404
  });
368
405
  return;
369
406
  }
370
- // DSH 0.1.7+: the settings service is SettingsForms and has no
371
- // installSection. The Config schema still resolves through cordis config
372
- // (this file reads `current()`), so search keeps working; surface the gap
373
- // once instead of silently swallowing it (compat-audit P1).
374
- console.warn(
375
- "[dsh-websearch] settings service has no installSection (DSH >= 0.1.7 SettingsForms line); " +
376
- "the Settings panel section is not installed. Configure the unified-search namespace via cordis config " +
377
- "or the client settings card; migration to the SettingsForms API is tracked for a future release.",
378
- );
407
+ // 0.2.0 SettingsForms line: no installSection; the settings form is
408
+ // auto-generated from the volatile-marked Config above — no action and no
409
+ // warning here (the form IS the section; a warning would contradict the
410
+ // visible UI). Config values still resolve via cordis config.
379
411
  });
380
412
 
381
413
  // v2.7.0 stores: disk cache + append-only search history, both under
@@ -423,6 +455,10 @@ export function apply(ctx, config) {
423
455
  getBackendHealth: getLatestBackendHealth,
424
456
  });
425
457
  // v2.7.0 search history: read-only GET + explicit clear POST.
426
- registerHistoryRoutes(webCtx.webServer, { getStore: () => historyStore });
458
+ // v2.8.0: GET also carries per-backend last-outcome observability.
459
+ registerHistoryRoutes(webCtx.webServer, {
460
+ getStore: () => historyStore,
461
+ getBackendHealth: getLatestBackendHealth,
462
+ });
427
463
  });
428
464
  }
@@ -0,0 +1,48 @@
1
+ // multiquery.js — v2.8.0 query-variant derivation (pure functions, zero deps).
2
+ //
3
+ // Gated multi-query: arXiv:2404.01037 (Wang et al.) shows BLIND multi-query
4
+ // can degrade retrieval — so variants are only derived for queries that look
5
+ // genuinely multi-faceted, and the original query always participates in the
6
+ // fusion with full weight. Splitting is purely mechanical (at separator
7
+ // words/punctuation); entity text is never rewritten.
8
+ //
9
+ // Separators per the v2.8 spec: "vs", "and", 比较, 对比, 和 — plus sentence
10
+ // punctuation for the long-query case.
11
+
12
+ const SEPARATOR_SPLIT_RE = /\s+(?:vs\.?|and)\s+|\s*(?:比较|对比|和)\s*|\s*[、;;。!?!?|]+\s*/i;
13
+ const COMPLEX_MARK_RE = /(?:\bvs\.?\b|\band\b|比较|对比|和)/i;
14
+
15
+ /** Sub-query hard cap (original + 2 variants max). */
16
+ export const MAX_SUB_QUERIES = 3;
17
+
18
+ /**
19
+ * Heuristic gate: does this query look multi-faceted enough to justify
20
+ * multi-query fan-out? Long queries (>60 chars) or queries containing a
21
+ * separator word qualify. Returns false for anything else — blind
22
+ * multi-query degrades (arXiv:2404.01037), so the default is NO.
23
+ */
24
+ export function isComplexQuery(query) {
25
+ if (typeof query !== "string") return false;
26
+ const q = query.trim();
27
+ if (q.length === 0) return false;
28
+ if (q.length > 60) return true;
29
+ return COMPLEX_MARK_RE.test(q);
30
+ }
31
+
32
+ /**
33
+ * Derive 2-3 sub-query variants from a complex query: the original query
34
+ * (always first, full weight in fusion) plus up to two mechanical splits at
35
+ * separator boundaries. Never rewrites entity text; returns [query] when no
36
+ * usable split exists.
37
+ */
38
+ export function deriveSubQueries(query) {
39
+ const base = typeof query === "string" ? query.trim() : "";
40
+ if (base.length === 0) return [];
41
+ const parts = base
42
+ .split(SEPARATOR_SPLIT_RE)
43
+ .map((p) => p.trim())
44
+ .filter((p) => p.length >= 2);
45
+ if (parts.length < 2) return [base];
46
+ const variants = [base, parts[0], parts.slice(1).join(" ")];
47
+ return [...new Set(variants.map((v) => v.trim()).filter(Boolean))].slice(0, MAX_SUB_QUERIES);
48
+ }
package/lib/provider.js CHANGED
@@ -25,9 +25,27 @@ import { normalizeFilters } from "./util/filters.js";
25
25
  import { shapeQuery, resultHeader, TELEMETRY_TAG } from "./prompting.js";
26
26
  import { cacheKeyFor } from "./cache.js";
27
27
  import { foldBreaker, isCooled, cooledAnnotation } from "./breaker.js";
28
+ import { rrfFuse } from "./rrf.js";
29
+ import { isComplexQuery, deriveSubQueries } from "./multiquery.js";
28
30
 
29
31
  const PROVIDER_ID = "unified";
30
32
 
33
+ // Per-backend timeout ceilings (v2.8.0, P2): ddg/scrapes and self-hosted
34
+ // SearXNG instances are the slowest to fail when unreachable — a 30s global
35
+ // default lets one dead backend stall the whole fan-out tail. The effective
36
+ // timeout is min(override, backendTimeoutMs), so an operator's shorter global
37
+ // timeout always wins.
38
+ const BACKEND_TIMEOUT_OVERRIDES_MS = {
39
+ ddg: 5000,
40
+ searxng: 5000,
41
+ };
42
+
43
+ /** Effective per-backend timeout = min(override ?? ∞, global). Exported for tests. */
44
+ export function effectiveBackendTimeoutMs(id, backendTimeoutMs) {
45
+ const override = BACKEND_TIMEOUT_OVERRIDES_MS[id];
46
+ return Number.isFinite(override) ? Math.min(override, backendTimeoutMs) : backendTimeoutMs;
47
+ }
48
+
31
49
  // Module-level per-backend health telemetry (id -> entry), updated on every
32
50
  // backend completion during a real search. Purely passive: nothing writes here
33
51
  // outside search(), so the read-only /api/unified-search/health route can
@@ -177,14 +195,25 @@ export function createUnifiedSearchProvider({ ctx, resolveOptions, backends, cac
177
195
  const concurrency = opts.concurrency ?? 6;
178
196
  const backendTimeoutMs = opts.backendTimeoutMs ?? 30000;
179
197
 
180
- // v2.7.0 disk cache: keyed on (shaped query + filters + maxResults).
181
- // A hit short-circuits the fan-out entirely; sources are byte-identical
182
- // to the first search and content carries a "cache hit, age Ns" note.
183
- // Cache failures never break the search (cache API never throws).
198
+ // v2.8.0 coverage / multiQuery gates (both default OFF; arXiv:2404.01037
199
+ // warns blind multi-query degrades, so the gate is deliberately strict).
200
+ const coverage = opts.deepCoverage === true;
201
+ // Per-backend widening: fetch more per backend, dedupe down afterwards.
202
+ const backendMaxResults = coverage ? Math.ceil(maxResults * 1.5) : maxResults;
203
+ const multi = opts.multiQueryEnabled === true && isComplexQuery(effReq.query);
204
+ const variants = multi ? deriveSubQueries(effReq.query) : [effReq.query];
205
+ // Cache-key mode marker: different coverage/multiQuery settings must not
206
+ // serve each other's cached results.
207
+ const cacheMode = (multi ? "multi" : "single") + (coverage ? "+coverage" : "");
208
+
209
+ // v2.7.0 disk cache: keyed on (shaped query + filters + maxResults +
210
+ // backends + mode). A hit short-circuits the fan-out entirely; sources
211
+ // are byte-identical to the first search and content carries a
212
+ // "cache hit, age Ns" note. Cache failures never break the search.
184
213
  const cacheEnabled = opts.cacheEnabled === true && cache;
185
214
  let cacheKey = null;
186
215
  if (cacheEnabled) {
187
- cacheKey = cacheKeyFor({ query: effReq.query, filters, maxResults, backends: opts.enabledBackends });
216
+ cacheKey = cacheKeyFor({ query: effReq.query, filters, maxResults, backends: opts.enabledBackends, mode: cacheMode });
188
217
  const hit = cache.get(cacheKey);
189
218
  if (hit && Array.isArray(hit.value?.sources) && hit.value.sources.length > 0) {
190
219
  // Defense in depth: the fresh path drops non-http(s) schemes before
@@ -219,7 +248,7 @@ export function createUnifiedSearchProvider({ ctx, resolveOptions, backends, cac
219
248
  // backend inside its 60s cooldown window is skipped (annotated in
220
249
  // telemetry as "id ⏸cooled"). Fail-open: if EVERY candidate is cooled,
221
250
  // run them all anyway — the breaker must never make search worse than
222
- // no breaker.
251
+ // no breaker. The pick is shared by every query variant.
223
252
  const breakerCfg = opts.breakerEnabled === false ? null : {
224
253
  threshold: opts.breakerThreshold,
225
254
  cooldownMs: opts.breakerCooldownMs,
@@ -255,98 +284,109 @@ export function createUnifiedSearchProvider({ ctx, resolveOptions, backends, cac
255
284
  );
256
285
  }
257
286
 
258
- // Settings-level default filters, overridden by per-request filters
259
- // (normalizeFilters drops unknown/invalid values so typos degrade to
260
- // "no filter" instead of breaking the search).
261
-
262
- const telemetry = [];
263
- // Concurrency limiter
264
- const limit = createLimiter(concurrency);
265
-
266
- // Run one backend under the per-backend timeout. The internal timer's
267
- // controller is merged into the caller's signal; when the INTERNAL timer
268
- // fired (caller did not abort), a backend abort-shaped failure is
269
- // reclassified as WEB_PROVIDER_ERROR so timeouts surface as real
270
- // failures instead of being silently demoted like user cancellations.
271
- const runBackend = ({ id, backend, beOpts, hooks }) => {
272
- const startedAtMs = Date.now();
273
- // Trust boundary: baseURL is operator-configurable and API keys are sent to it.
274
- // Enforce http(s); plain http only for private hosts (self-hosted SearXNG).
275
- // Reject embedded credentials. Failure is a per-backend soft error.
276
- if (beOpts.baseURL !== undefined) {
277
- let bu = null;
278
- try { bu = new URL(beOpts.baseURL); } catch { bu = null; }
279
- const privHost = bu ? /^(localhost|127(\.\d{1,3}){3}|10(\.\d{1,3}){3}|192\.168(\.\d{1,3}){3}|172\.(1[6-9]|2\d|3[01])(\.\d{1,3}){3})$/.test(bu.hostname) : false;
280
- const schemeOk = bu && (bu.protocol === "https:" || (bu.protocol === "http:" && privHost));
281
- if (!schemeOk || bu.username || bu.password) {
282
- throw new WebError(`backend "${id}" has an invalid baseURL: https required (plain http only for private hosts), credentials-in-URL rejected`, "WEB_PROVIDER_ERROR");
287
+ // Run ONE query variant through the full fan-out: breaker-aware backend
288
+ // execution under the per-variant limiter, then PHASE 1 collect. The
289
+ // internal abort/timeout semantics are unchanged from v2.7.
290
+ const fanoutVariant = async (variantReq) => {
291
+ const telemetry = [];
292
+ const limit = createLimiter(concurrency);
293
+ const runBackend = ({ id, backend, beOpts, hooks }) => {
294
+ const startedAtMs = Date.now();
295
+ // Trust boundary: baseURL is operator-configurable and API keys are sent to it.
296
+ // Enforce http(s); plain http only for private hosts (self-hosted SearXNG).
297
+ // Reject embedded credentials. Failure is a per-backend soft error.
298
+ if (beOpts.baseURL !== undefined) {
299
+ let bu = null;
300
+ try { bu = new URL(beOpts.baseURL); } catch { bu = null; }
301
+ const privHost = bu ? /^(localhost|127(\.\d{1,3}){3}|10(\.\d{1,3}){3}|192\.168(\.\d{1,3}){3}|172\.(1[6-9]|2\d|3[01])(\.\d{1,3}){3})$/.test(bu.hostname) : false;
302
+ const schemeOk = bu && (bu.protocol === "https:" || (bu.protocol === "http:" && privHost));
303
+ if (!schemeOk || bu.username || bu.password) {
304
+ throw new WebError(`backend "${id}" has an invalid baseURL: https required (plain http only for private hosts), credentials-in-URL rejected`, "WEB_PROVIDER_ERROR");
305
+ }
283
306
  }
284
- }
285
- const controller = new AbortController();
286
- const mergedSignal = signal
287
- ? AbortSignal.any([signal, controller.signal])
288
- : controller.signal;
289
- const timeoutId = setTimeout(() => controller.abort(), backendTimeoutMs);
290
- return backend
291
- .search(effReq, mergedSignal, beOpts, hooks)
292
- .then(
293
- (value) => {
294
- telemetry.push({ id, ok: true, ms: Date.now() - startedAtMs, count: value?.sources?.length ?? 0 });
295
- recordBackendHealth(id, { ok: true, at: Date.now(), ms: Date.now() - startedAtMs, count: value?.sources?.length ?? 0 }, breakerCfg);
296
- return value;
297
- },
298
- (err) => {
299
- const msg = String(err instanceof Error ? err.message : err).slice(0, 80);
300
- telemetry.push({ id, ok: false, ms: Date.now() - startedAtMs, error: msg });
301
- recordBackendHealth(id, { ok: false, at: Date.now(), ms: Date.now() - startedAtMs, error: msg }, breakerCfg);
307
+ const controller = new AbortController();
308
+ const mergedSignal = signal
309
+ ? AbortSignal.any([signal, controller.signal])
310
+ : controller.signal;
311
+ const effectiveTimeoutMs = effectiveBackendTimeoutMs(id, backendTimeoutMs);
312
+ const timeoutId = setTimeout(() => controller.abort(), effectiveTimeoutMs);
313
+ return backend
314
+ .search(variantReq, mergedSignal, beOpts, hooks)
315
+ .then(
316
+ (value) => {
317
+ telemetry.push({ id, ok: true, ms: Date.now() - startedAtMs, count: value?.sources?.length ?? 0 });
318
+ recordBackendHealth(id, { ok: true, at: Date.now(), ms: Date.now() - startedAtMs, count: value?.sources?.length ?? 0 }, breakerCfg);
319
+ return value;
320
+ },
321
+ (err) => {
322
+ const msg = String(err instanceof Error ? err.message : err).slice(0, 80);
323
+ telemetry.push({ id, ok: false, ms: Date.now() - startedAtMs, error: msg });
324
+ recordBackendHealth(id, { ok: false, at: Date.now(), ms: Date.now() - startedAtMs, error: msg }, breakerCfg);
325
+ throw err;
326
+ },
327
+ )
328
+ .catch((err) => {
329
+ if (controller.signal.aborted && !(signal && signal.aborted)) {
330
+ throw new WebError(
331
+ `backend "${id}" timed out after ${effectiveTimeoutMs}ms`,
332
+ "WEB_PROVIDER_ERROR",
333
+ { cause: err instanceof Error ? err : undefined },
334
+ );
335
+ }
302
336
  throw err;
303
- },
304
- )
305
- .catch((err) => {
306
- if (controller.signal.aborted && !(signal && signal.aborted)) {
307
- throw new WebError(
308
- `backend "${id}" timed out after ${backendTimeoutMs}ms`,
309
- "WEB_PROVIDER_ERROR",
310
- { cause: err instanceof Error ? err : undefined },
311
- );
312
- }
313
- throw err;
314
- })
315
- .finally(() => clearTimeout(timeoutId));
316
- };
317
-
318
- const results = await Promise.allSettled(
319
- eligible.map(({ id, backend, beOpts }) => {
320
- const hooks = {
321
- recordRequest: (req) => recordBackendRequest(ctx, id, req),
322
- recordOutcome: (out) => recordBackendOutcome(ctx, id, out),
323
- };
324
- return limit(() => runBackend({ id, backend, beOpts, hooks }));
325
- }),
326
- );
327
-
328
- throwIfSearchAborted(signal);
337
+ })
338
+ .finally(() => clearTimeout(timeoutId));
339
+ };
329
340
 
330
- // PHASE 1 collect: fold the settled fan-out into raw sources (fan-out
331
- // priority order), first backend content, and per-backend failures.
332
- const { collected, mergedContent, hardAborted, hardError, failures } =
333
- collectFanoutResults(results, eligible, signal);
341
+ const results = await Promise.allSettled(
342
+ eligible.map(({ id, backend, beOpts }) => {
343
+ const hooks = {
344
+ recordRequest: (req) => recordBackendRequest(ctx, id, req),
345
+ recordOutcome: (out) => recordBackendOutcome(ctx, id, out),
346
+ };
347
+ return limit(() => runBackend({ id, backend, beOpts, hooks }));
348
+ }),
349
+ );
350
+ throwIfSearchAborted(signal);
351
+ const fold = collectFanoutResults(results, eligible, signal);
352
+ return { ...fold, telemetry };
353
+ };
334
354
 
335
- // Caller-cancellation check (hard abort fires before we yield results).
336
- if (signal?.aborted === true || hardAborted) {
337
- throw searchAborted(signal);
355
+ // Fan out per variant (sequential — the limiter already parallelizes
356
+ // backends within a variant), dedupe/rank each list, then fuse.
357
+ const perVariant = [];
358
+ const allTelemetry = [];
359
+ const allFailures = [];
360
+ let hardError = null;
361
+ for (const variantQuery of variants) {
362
+ const variantReq = {
363
+ ...effReq,
364
+ query: variantQuery,
365
+ maxResults: backendMaxResults,
366
+ ...(coverage ? { deepCoverage: true } : {}),
367
+ };
368
+ const fold = await fanoutVariant(variantReq);
369
+ allTelemetry.push(...fold.telemetry);
370
+ allFailures.push(...fold.failures);
371
+ if (fold.hardError !== null && hardError === null) hardError = fold.hardError;
372
+ // Caller-cancellation check (hard abort fires before we yield results).
373
+ if (signal?.aborted === true || fold.hardAborted) {
374
+ throw searchAborted(signal);
375
+ }
376
+ // PHASE 2 dedupe / PHASE 3 rerank (per variant list, pre-fusion).
377
+ const strategy = opts.dedupStrategy === "url+title" ? "url+title" : "url";
378
+ const deduped = dedupeSources(fold.collected, strategy);
379
+ const ordered = opts.rerank === true ? rerankSources(deduped, variantQuery) : deduped;
380
+ perVariant.push({ query: variantQuery, ordered, mergedContent: fold.mergedContent });
338
381
  }
339
382
 
340
- // PHASE 2 dedupe / PHASE 3 rerank: "url" preserves the historical
341
- // URL-key behavior; "url+title"
342
- // additionally collapses same-story syndicated copies across backends.
343
- const strategy = opts.dedupStrategy === "url+title" ? "url+title" : "url";
344
- const deduped = dedupeSources(collected, strategy);
345
- // Optional deterministic relevance rerank (stable ties keep fan-out order).
346
- const ordered =
347
- opts.rerank === true
348
- ? rerankSources(deduped, typeof request?.query === "string" ? request.query : "")
349
- : deduped;
383
+ // PHASE 4 fuse: multi-query mode fuses the per-variant ranked lists with
384
+ // RRF (k=60); single-query mode keeps the historical behavior (the one
385
+ // list, unchanged ordering).
386
+ const orderedLists = perVariant.filter((v) => v.ordered.length > 0).map((v) => v.ordered);
387
+ const ordered = multi
388
+ ? rrfFuse(orderedLists).fused
389
+ : (orderedLists[0] ?? []);
350
390
 
351
391
  // If we got ANY sources, return them even if some backends failed.
352
392
  if (ordered.length > 0) {
@@ -355,7 +395,7 @@ export function createUnifiedSearchProvider({ ctx, resolveOptions, backends, cac
355
395
  // 置于结果块最前,要求逐条引用来源 URL、优先官方/一手与较新来源;
356
396
  // 语言按查询文本自动选择(含 CJK 用中文头)。可用
357
397
  // opts.resultHeader === false 关闭。
358
- let outContent = mergedContent;
398
+ let outContent = perVariant.find((v) => v.mergedContent !== undefined)?.mergedContent;
359
399
  if (opts.resultHeader !== false && (sources.length > 0 || outContent !== undefined)) {
360
400
  const header = resultHeader(typeof request?.query === "string" && /[\u4e00-\u9fff]/.test(request.query) ? "zh" : "en");
361
401
  outContent = outContent === undefined ? header : header + "\n\n" + outContent;
@@ -369,18 +409,22 @@ export function createUnifiedSearchProvider({ ctx, resolveOptions, backends, cac
369
409
  query: request?.query,
370
410
  time: Date.now(),
371
411
  resultCount: sources.length,
372
- backendsOk: telemetry.filter((t) => t.ok).length,
373
- backendsTotal: telemetry.length,
412
+ backendsOk: allTelemetry.filter((t) => t.ok).length,
413
+ backendsTotal: allTelemetry.length,
374
414
  });
375
- if (opts.resultTelemetry !== false && (telemetry.length > 0 || cooled.size > 0)) {
415
+ if (opts.resultTelemetry !== false && (allTelemetry.length > 0 || cooled.size > 0)) {
376
416
  const cooledParts = [];
377
417
  for (const [cid, centry] of cooled) {
378
418
  const ann = cooledAnnotation(centry, Date.now()) || "⏸cooled";
379
419
  cooledParts.push(cid + " " + ann);
380
420
  }
381
- const parts = telemetry
421
+ // Multi-query fans every variant through every backend; cap the line
422
+ // so the telemetry note cannot dwarf the results themselves.
423
+ const parts = allTelemetry
424
+ .slice(0, 12)
382
425
  .map((t) => t.id + (t.ok ? " ✓" : " ✗") + t.ms + "ms" + (t.ok ? "/" + t.count : (t.error ? " (" + String(t.error).slice(0, 40) + ")" : "")))
383
426
  .concat(cooledParts);
427
+ if (allTelemetry.length > 12) parts.push(`… +${allTelemetry.length - 12}`);
384
428
  const line = "[websearch backends]" + TELEMETRY_TAG + " " + parts.join(" · ");
385
429
  outContent = outContent === undefined ? line : outContent + "\n\n" + line;
386
430
  }
@@ -393,17 +437,17 @@ export function createUnifiedSearchProvider({ ctx, resolveOptions, backends, cac
393
437
 
394
438
  // All enabled backends failed - synthesize one meaningful error that
395
439
  // names each backend's own reason so the caller (model or user) can act.
396
- const detail = failures.slice(0, 4)
440
+ const detail = allFailures.slice(0, 4)
397
441
  .map((f) => f.id + ": " + String(f.err instanceof Error ? f.err.message : f.err).slice(0, 120))
398
442
  .join("; ");
399
443
  const allKeyMissing =
400
- failures.length > 0 &&
401
- failures.every((f) => f.err instanceof WebError && (f.err.code === "WEB_PROVIDER_UNAVAILABLE" || f.err.code === "WEB_PROVIDER_CREDENTIAL_MISSING"));
444
+ allFailures.length > 0 &&
445
+ allFailures.every((f) => f.err instanceof WebError && (f.err.code === "WEB_PROVIDER_UNAVAILABLE" || f.err.code === "WEB_PROVIDER_CREDENTIAL_MISSING"));
402
446
  const hint = allKeyMissing
403
447
  ? " (every enabled backend is missing its API key - add keys in Settings > Web Search, or enable a keyless backend)"
404
448
  : "";
405
449
  throw new WebError(
406
- `unified search: all enabled backends failed (${failures.length}) - ${detail}` + hint,
450
+ `unified search: all enabled backends failed (${allFailures.length}) - ${detail}` + hint,
407
451
  "WEB_PROVIDER_ERROR",
408
452
  { cause: hardError ?? undefined },
409
453
  );
package/lib/rrf.js ADDED
@@ -0,0 +1,56 @@
1
+ // rrf.js — v2.8.0 Reciprocal Rank Fusion (pure functions, zero deps).
2
+ //
3
+ // RRF merges multiple ranked result lists into one: each list contributes
4
+ // score 1/(k + rank) to every item it contains (k=60 is the canonical value
5
+ // from Cormack et al. 2009, popularized by RAG-Fusion
6
+ // https://github.com/Raudaschl/rag-fusion). Items are deduped by URL; the
7
+ // first-seen entry wins for metadata, scores accumulate across lists.
8
+ //
9
+ // Why RRF over naive concatenation: it needs no score calibration across
10
+ // backends/queries (only ranks), and demotes one-off hits while promoting
11
+ // items that appear in several lists.
12
+
13
+ /** Canonical RRF constant. */
14
+ export const RRF_K = 60;
15
+
16
+ /** Stable key for dedupe (URL string; missing/odd URLs fall back to title). */
17
+ function itemKey(item) {
18
+ if (item && typeof item.url === "string" && item.url.length > 0) return item.url;
19
+ if (item && typeof item.title === "string") return `t:${item.title}`;
20
+ return "";
21
+ }
22
+
23
+ /**
24
+ * Fuse ranked lists of search sources with Reciprocal Rank Fusion.
25
+ *
26
+ * @param {Array<Array<object>>} lists each list is already ranked best-first
27
+ * @param {{k?: number}} [opts] k constant (default 60)
28
+ * @returns {{ fused: object[], scores: Map<string, number> }}
29
+ * fused: sources sorted by accumulated RRF score desc (ties keep first
30
+ * appearance order); sources without a usable key are dropped.
31
+ */
32
+ export function rrfFuse(lists, { k = RRF_K } = {}) {
33
+ const kk = Number(k) > 0 ? Number(k) : RRF_K;
34
+ const scores = new Map(); // key -> accumulated score
35
+ const firstSeen = new Map(); // key -> source object (first metadata wins)
36
+ const order = []; // keys in first-appearance order (stable ties)
37
+
38
+ for (const list of Array.isArray(lists) ? lists : []) {
39
+ if (!Array.isArray(list)) continue;
40
+ for (let rank = 0; rank < list.length; rank++) {
41
+ const item = list[rank];
42
+ const key = itemKey(item);
43
+ if (!key) continue;
44
+ scores.set(key, (scores.get(key) ?? 0) + 1 / (kk + rank));
45
+ if (!firstSeen.has(key)) {
46
+ firstSeen.set(key, item);
47
+ order.push(key);
48
+ }
49
+ }
50
+ }
51
+
52
+ const fused = [...order]
53
+ .sort((a, b) => (scores.get(b) ?? 0) - (scores.get(a) ?? 0))
54
+ .map((key) => firstSeen.get(key));
55
+ return { fused, scores };
56
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@240xu/dsh-websearch",
3
- "version": "2.7.3",
3
+ "version": "2.8.1",
4
4
  "description": "Unified web search provider for DeepSeek Harness (ctx.web): 11 backends — Exa + Parallel + DDG + SearXNG (keyless), DeepSeek + Anthropic + OpenAI + Brave + Tavily + Serper + Mojeek (key-gated via Settings credentials). Registers provider id \"unified\" for zero-config web_search. 统一网页搜索聚合:11 后端并发扇出、去重合并,零依赖。See CHANGELOG.md for version history.",
5
5
  "type": "module",
6
6
  "main": "lib/index.js",