bookmark-research 0.5.0-beta.11 → 0.5.0-beta.12

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -9,7 +9,7 @@
9
9
  "source": {
10
10
  "source": "npm",
11
11
  "package": "bookmark-research",
12
- "version": "0.5.0-beta.11",
12
+ "version": "0.5.0-beta.12",
13
13
  "registry": "https://registry.npmjs.org"
14
14
  },
15
15
  "policy": {
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "bookmark-research",
3
- "version": "0.5.0-beta.11",
3
+ "version": "0.5.0-beta.12",
4
4
  "description": "Check bookmark URLs or Canvas packages, compare sources with host-led research, and retain verifiable reports.",
5
5
  "author": {
6
6
  "name": "kwenxu"
package/README.md CHANGED
@@ -25,13 +25,13 @@ Works in **Codex, Claude Code, Pi and DSH (DeepSeek Harness)**.
25
25
 
26
26
  ## Install
27
27
 
28
- **Current beta (`0.5.0-beta.11`)**, interactive on macOS, Linux and Windows:
28
+ **Current beta (`0.5.0-beta.12`)**, interactive on macOS, Linux and Windows:
29
29
 
30
30
  ```sh
31
- npx bookmark-research@0.5.0-beta.11 install --source npm:bookmark-research@0.5.0-beta.11
31
+ npx bookmark-research@0.5.0-beta.12 install --source npm:bookmark-research@0.5.0-beta.12
32
32
  ```
33
33
 
34
- This pins both the installer and the content. The `@latest` examples below follow the default channel, which remains `0.5.0-beta.3`; `@beta` currently points to `0.5.0-beta.11`. Changing only the npx selector does not change the installer's default content source.
34
+ This pins both the installer and the content. The `@latest` examples below follow the default channel, which remains `0.5.0-beta.3`; `@beta` currently points to `0.5.0-beta.12`. Changing only the npx selector does not change the installer's default content source.
35
35
 
36
36
  **Default-channel interactive installation:**
37
37
 
package/README.zh.md CHANGED
@@ -25,13 +25,13 @@
25
25
 
26
26
  ## 安装
27
27
 
28
- **当前测试版(`0.5.0-beta.11`)**,支持 macOS、Linux、Windows 交互安装:
28
+ **当前测试版(`0.5.0-beta.12`)**,支持 macOS、Linux、Windows 交互安装:
29
29
 
30
30
  ```sh
31
- npx bookmark-research@0.5.0-beta.11 install --source npm:bookmark-research@0.5.0-beta.11
31
+ npx bookmark-research@0.5.0-beta.12 install --source npm:bookmark-research@0.5.0-beta.12
32
32
  ```
33
33
 
34
- 这条命令同时固定安装器和插件内容。下面的 `@latest` 示例使用默认渠道,目前仍为 `0.5.0-beta.3`;`@beta` 当前指向 `0.5.0-beta.11`。只改 npx 的版本选择,不会改变安装器默认下载的内容来源。
34
+ 这条命令同时固定安装器和插件内容。下面的 `@latest` 示例使用默认渠道,目前仍为 `0.5.0-beta.3`;`@beta` 当前指向 `0.5.0-beta.12`。只改 npx 的版本选择,不会改变安装器默认下载的内容来源。
35
35
 
36
36
  **默认渠道的交互安装:**
37
37
 
@@ -121,13 +121,13 @@ This version has no embeddings, vector database or background page monitoring. R
121
121
 
122
122
  These modes follow [OpenAI's web search guide](https://developers.openai.com/api/docs/guides/tools-web-search). Having a URL does not determine whether a model reasons. `research_route` recommends an entrypoint from reported current tools and commands without launching it. Professional APIs have separate authentication and lifecycle contracts; anonymous Search MCP access does not authorize a Task API.
123
123
 
124
- One MCP entry point aggregates several backend services, and one Skill executes all three workflows. Reading defaults to Exa followed by concurrent Parallel + Jina Reader fallback for unresolved URLs; Tavily extraction can be added to `fetch.providers`. Search starts with Exa + Parallel, then calls `search.fallback_providers` (Tavily + keyed Jina) concurrently only for queries with no usable URL. Missing fallback credentials are skipped; successful results stay intact. Set the search fallback list to `[]` to disable it. An existing saved search provider list without a fallback field retains its original service scope.
124
+ One MCP entry point aggregates several backend services, and one Skill executes all three workflows. Reading starts with Exa under `fetch.strategy` (staggered by default: the next provider joins once the current one stalls past `fetch.grace_seconds`; hedged and sequential are the alternatives; abandoned providers are listed in `abandoned_providers`), then concurrent Parallel + Jina Reader fallback for unresolved URLs; Tavily extraction can be added to `fetch.providers`. `fetch.reuse_seconds` can return text saved within that window as `reused_sources` without a new request, and `search.cache_seconds` serves a repeated identical search from a short-lived cache. Search starts with Exa + Parallel under `search.strategy` (hedged by default, with staggered, sequential and wait_all alternatives) and the `search.grace_seconds` / `search.deadline_seconds` bounds; dropped or uncalled providers are listed in `routing`. Then `search.fallback_providers` (Tavily + keyed Jina) run concurrently only for queries with no usable URL. Missing fallback credentials are skipped; successful results stay intact. Set the search fallback list to `[]` to disable it. An existing saved search provider list without a fallback field retains its original service scope.
125
125
 
126
126
  The host model leads both research modes. Codex native subagents, Claude ordinary subagents or enabled teams, Pi extensions, and configured DSH subagents/workflows supply collaboration; see [host capabilities](harness-compatibility.md). The grouped package scripts are optional for batch research. Exa `agent_run`, complete Parallel Task MCP tools and enabled OpenAI/Parallel APIs can handle scoped external work after host assessment. A saved API preference orders these optional services; an explicit provider selects it exclusively. On confirmed failure, use cumulative `failed_routes` to continue the same investigation, preserving running/unknown/cancelled jobs. API execution follows prepare → start → status → result → import and requires separate credentials. Scrapling, Firecrawl and MarkItDown remain researched projects. A public GitHub README fetch does not inspect the whole repository.
127
127
 
128
128
  Read the [deep research guide](../skills/bookmark-research/references/deep-research.md) for inputs and CLI examples. `research_start` freezes the complete indexed packages selected by `source_ids`; a subset requires explicit `scope_mode:"subset"`. Paginate `research_inventory` to the end. `bookmark_refs` identify focus items without silently narrowing whole-package scope.
129
129
 
130
- `research_search` and `research_fetch` reserve retrieval attempts; repeating the same `operation_id` returns its recorded outcome. The default fetch budget accounts for the selected input size: 207 URLs require at least 26 full batches and receive 38 calls, within an 80-call limit. Explicit budgets are retained; insufficient capacity is reported without removing URLs. These controls do not measure or limit invisible host/model calls.
130
+ `research_search` and `research_fetch` reserve retrieval attempts; repeating the same `operation_id` returns its recorded outcome. The default fetch budget accounts for the selected input size: 207 URLs require at least 13 full batches and receive 25 calls, within an 80-call limit. Explicit budgets are retained; insufficient capacity is reported without removing URLs. These controls do not measure or limit invisible host/model calls.
131
131
 
132
132
  `research_source` reads saved text. Record content acceptance, quoted claims and each original URL's substantive judgment. A rejected source or retracted claim reopens affected questions. `research_coverage` reports gaps; `research_finish` refuses completion while original-source reviews, questions, conflicts or active/unknown external runs remain unresolved. Blocked and excluded sources remain visible in the original denominator. An `incomplete` report preserves the remaining work.
133
133
 
@@ -218,7 +218,7 @@ The [shared Skill](../skills/bookmark-research/SKILL.md) guides the agent's work
218
218
  | **Tavily** | Built-in search fallback and selectable extraction; `TAVILY_API_KEY` uses Bearer, otherwise explicit keyless mode |
219
219
  | **GitHub** | Use the host's existing GitHub MCP or `gh` for repository-specific tasks. No GitHub MCP is bundled; public pages can also be read through Exa / Parallel. |
220
220
 
221
- All four retrieval providers run through the plugin's own MCP or CLI; additional host MCP registrations are unnecessary. Search starts with Exa + Parallel and uses configured fallbacks for unresolved queries; reading falls back to Parallel + Jina after Exa. Anonymous/keyless access and quotas depend on each service's policy; optional credentials use `EXA_API_KEY`, `PARALLEL_API_KEY`, `TAVILY_API_KEY` or `JINA_API_KEY` in the launching environment. Local queries work offline.
221
+ All four retrieval providers run through the plugin's own MCP or CLI; additional host MCP registrations are unnecessary. Search starts with Exa + Parallel under a selectable strategy (hedged by default) with a grace window and total deadline, and uses configured fallbacks for unresolved queries; reading falls back to Parallel + Jina after Exa. Anonymous/keyless access and quotas depend on each service's policy; optional credentials use `EXA_API_KEY`, `PARALLEL_API_KEY`, `TAVILY_API_KEY` or `JINA_API_KEY` in the launching environment. Local queries work offline.
222
222
 
223
223
  `search_providers` describes configuration without network access; `probe:true` discovers remote MCP tools, while Jina's HTTP mapping remains explicitly `not_probed`. Discovery does not prove successful retrieval. Each MCP process reuses remote sessions and a bounded tool catalog. Calls are not automatically replayed after errors. See [MCP architecture and observed service behavior](mcp-aggregation-0.2.0.md) for the initial implementation and checks.
224
224
 
package/docs/details.md CHANGED
@@ -26,7 +26,7 @@
26
26
  | 场景 | 基础流程 | 检索与委派支持 |
27
27
  | --- | --- | --- |
28
28
  | URL 直读 | 当前模型读取已知页面并回答 | Exa 未读到的 URL 交给 Parallel + Jina Reader 并发读取;可配置 Tavily 提取。 |
29
- | Agentic Web Search | 当前宿主模型先规划 → 搜索/读文 → 判断证据 → 针对缺口补查 | Exa + Parallel 提供搜索结果;无有效网址的查询再交给 Tavily + 有 key 的 Jina。普通问题无需完整研究会话。 |
29
+ | Agentic Web Search | 当前宿主模型先规划 → 搜索/读文 → 判断证据 → 针对缺口补查 | Exa + Parallel 按 `search.strategy` 提供搜索结果(默认并发对冲);无有效网址的查询再交给 Tavily + 有 key 的 Jina。普通问题无需完整研究会话。 |
30
30
  | Deep Research | 在宿主研究循环上增加持续调查、按需协作、独立核验、覆盖检查与报告 | 优先利用宿主现有子代理/工作流;外部 Research MCP/API 可承担宿主判断后的子问题或用户明确指定的任务。 |
31
31
 
32
32
  四个基座共用这套方法:Codex 原生子代理、Claude 普通子代理或已启用团队、Pi 扩展、已配置的 DSH 子代理/工作流;具体差异与官方依据见 [宿主兼容说明](harness-compatibility.md)。整包脚本用于分组研究。保存的 API 偏好只排列外部服务,显式 provider 才直接指定服务;明确路线失败可继续换路,运行中、结果未知或已取消任务不自动重建。
@@ -63,7 +63,7 @@ JSON/`.canvas` 是源数据,SQLite 是可重建的本地查询索引,无
63
63
 
64
64
  ## 搜索、配置与语言
65
65
 
66
- `search.providers` 为第一轮,`search.fallback_providers` 为无结果时的备用列表;设为空数组可关闭搜索回退。读取单独配置 `fetch.provider` 和 `fetch.providers`。调用时明确指定 provider/providers 则只用指定服务。已有配置若只设置过搜索 providers,会保留原来的服务范围;新研究冻结当时的选择,旧记录重放不会增加请求。瀑布流按轮补缺,同轮服务并发;已发出的请求仍需等待结束或超时。普通网页归档开启,深度研究始终保存任务证据。
66
+ `search.providers` 为第一轮,`search.fallback_providers` 为无结果时的备用列表;设为空数组可关闭搜索回退。第一轮策略由 `search.strategy` 选择(默认 hedged 全并发对冲;staggered 错峰、sequential 顺序、wait_all 等全部),按 `search.grace_seconds`(默认 2)与 `search.deadline_seconds`(默认 10)约束:超限的 provider 被放弃或不再调用,分别记入 `routing.abandoned_providers` 与 `routing.not_attempted_providers`,被放弃的请求可能仍消耗额度;研究任务保持等待全部。读取单独配置 `fetch.provider` 和 `fetch.providers`,策略由 `fetch.strategy` 选择(默认 staggered:当前提供商停滞超过 `fetch.grace_seconds` 才并行下一家;hedged 全并发、sequential 逐家;被放弃者记入 `abandoned_providers`)。`fetch.reuse_seconds` 可在窗口内直接复用已存正文(返回 `reused_sources`,不联网、不占抓取预算),`search.cache_seconds` 让重复的相同搜索走短期缓存(批次标记 `cached:true`)。调用时明确指定 provider/providers 则只用指定服务。已有配置若只设置过搜索 providers,会保留原来的服务范围;新研究冻结当时的选择,旧记录重放不会增加请求。瀑布流按轮补缺,同轮服务并发;已发出的请求仍需等待结束或超时。普通网页归档开启,深度研究始终保存任务证据。
67
67
 
68
68
  可以读取公开 GitHub 仓库页面、README、文件和 issues;读取 README 不等于遍历整个代码仓库。代码审查按任务选择具体文件或宿主已有 GitHub 工具,插件未额外安装 GitHub MCP。
69
69
 
@@ -21,7 +21,7 @@
21
21
 
22
22
  候选版本在 `release/` 分支同步三个版本清单并运行 CI。该分支的 registry 集成测试使用当前已发布渠道版本,源码测试、原生安装测试和 npm 打包检查仍验证候选内容;主分支的 registry 测试始终验证 marketplace 固定版本。发布通过 CI 的 npm 包并核验 registry 后,再将候选分支合入主分支,避免默认安装清单提前指向未发布版本。
23
23
 
24
- 测试版使用 npm 的 `beta` 标签;若创建 GitHub 版本说明,则标记为预发布。不会自动移动 npm 的 `latest` 标签。安装测试版时同时固定安装器与内容来源,例如 `npx bookmark-research@0.5.0-beta.11 install --source npm:bookmark-research@0.5.0-beta.11`。仅将 npx 改为 `@beta` 不会改变默认内容来源 `latest`;已固定的旧版安装也不会自动换成测试版。
24
+ 测试版使用 npm 的 `beta` 标签;若创建 GitHub 版本说明,则标记为预发布。不会自动移动 npm 的 `latest` 标签。安装测试版时同时固定安装器与内容来源,例如 `npx bookmark-research@0.5.0-beta.12 install --source npm:bookmark-research@0.5.0-beta.12`。仅将 npx 改为 `@beta` 不会改变默认内容来源 `latest`;已固定的旧版安装也不会自动换成测试版。
25
25
 
26
26
  ## 官方资料与项目实例
27
27
 
@@ -0,0 +1,65 @@
1
+ # 插件后续改进计划(2026-10-15)
2
+
3
+ > 依据:2026-10-15 的 LLM Wiki 调研任务(`r-eb7778ec8c954b9a`,已完成,报告与证据在该任务目录)+当日实测(Exa 免 Key 通道两次 fail-closed、wiki lint 对 4 个存量页面报出 3 个时效复核类警告(口径见 §0)、wiki work 显示该调研任务 16 条 claim 待编译)。
4
+ > 调研结论三条:形态流行(代理记忆基础设施与 LLM 编译个人 wiki 两支);主流失败模式是维护衰减、时效失真、评测混杂与厂商数字不可比;来源级问责与"把维护移入系统"被同行评审(SciTrue)与行业观察支持。
5
+ > 已完成项见 `search-hedge-plan-2026-10-14.md` §9.1(读取对冲与回合压缩)与 §9.2(缓存复用、并发配置、targets 一致性、宿主提示同步);本文件只列未完成项与新增项,按优先级逐个讨论后执行。
6
+
7
+ ## 0. 当前基线(2026-10-15 收尾)
8
+
9
+ - 代码:第 1、2、4–7 项已实现并验证(全量 659 项通过、3 项跳过);第 3 项(档位与通道)停放,需 REST + Key,免 Key MCP 通道不暴露档位。
10
+ - 提交:本轮全部改动已入库,含本目录的 `search-hedge-plan-2026-10-14.md`。
11
+ - 激活:本机安装 bundle 仍为旧版;需重装 bundle、重启会话后新功能生效,并核对 receipt 版本与运行代码一致。
12
+ - Wiki 体系:4 个存量页面(`rag-search-and-mcp-20260912` rev2、`rag-evaluation-and-wiki-20260912`、`personal-knowledge-evidence-and-wiki-20261009`、`code-retrieval-evidence-20261009`);lint 状态 ok,其中 3 条复核类警告(`newer_source_snapshot`×1、`source_input_changed`×2;lint 同时报出 16 条 `source_capture_expired`、2 条 `linked_page_needs_review`、1 条 `link_has_newer_revision`,均不阻塞);本调研任务 16 条 eligible claim 待编译。
13
+
14
+ ## 1. 收尾与发布(P0)
15
+
16
+ 1.1 重装 bundle 并重启会话;核对运行代码为新版(工具 schema 应出现 `reuse_seconds`、`cache_seconds` 与 `server` 节)。
17
+ 1.2 会话内冒烟:`research_fetch` 带 `reuse_seconds` 复用已存正文;`search.cache_seconds` 生效;`config set --server-workers` 重启后生效。
18
+ 1.3 发版 0.5.0-beta.12(安装器已支持无缝覆盖升级)。
19
+ 1.4 激活后第一次真实使用:编译本调研任务的 16 条 claim 成 1–2 个 Wiki 页面,并处理现存 3 条复核类 lint 警告(与 2.1 联动;lint 另报的 16 条 capture 过期与 3 条页间链接类不阻塞)。
20
+
21
+ ## 2. 时效治理闭环(P1,调研报告直指)
22
+
23
+ 背景:调研结论指出主流知识库的头号死因是维护衰减与时效失真(94% 内容月内未动;检索不辨时效)。本插件差异化恰在此处,但闭环只做了一半:lint 能发现"被引 URL 有新快照 / 输入变化",复核仍是粗粒度,只说来源和页面,不说哪几条 claim 受影响、引文是否仍然成立。
24
+
25
+ 2.1 **引文级复核辅助**(只读)。wiki lint 与 wiki refresh 输出扩展:对每条受影响 claim 列出 claim id,并检查其引文在新快照中的逐字存在性(仍在 / 已变 / 消失)+变更摘录。积木:`search_archive` 逐字检索与 source_versions 快照。边界:只回答"引文是否仍逐字存在",不做语义等价判断,修订仍走人工审阅。
26
+ 2.2 **批量刷新**。`wiki refresh` 现在一次一页(每页最多 16 个被引 URL);增加"所有过期页面"批量入口,配合 cron 或 watch 例行化,把维护真正移入系统。
27
+ 2.3 (小)纯问题型任务的覆盖率表格补 claims 计数与来源审阅计数,替代 0/0 n/a(本次调研实测)。
28
+ 验收:对现存 3 条复核类警告输出引文级差异报告;批量刷新覆盖 2 页以上。
29
+
30
+ ## 3. Exa fail-closed:已归因并完成抓取侧修复(P2 → 2026-10-16)
31
+
32
+ 背景(2026-10-16 补证):2026-10-15 的 8 URL 批量抓取"全部 `unrecognized_or_not_returned`"经归档原文复现,**不是限流**。该响应(`r-748272ccc9fc4749` 证据目录 `20261015T032831001944Z-a124bc85cbb5/response.json`,已固化为测试 fixture `tests/fixtures/exa-fetch-8url-fence-desync-20261015.json`)含全部 8 页正文(63,410 字符)、无 error 信封、无 Error fetching 页脚;失败原因是本地解析器拒收:第 2 页 markdown 围栏未闭合,遮罩把第 4、5 页真实页头盖住(偏移 32510–37279),8 条记录只识别 5 条后整批放弃,瀑布流转 Parallel 全部成功。全库 102 份 capture 中 7 份同签名(4 份围栏未闭合、3 份页头被遮罩),另有 20 份含真实提供商错误页(如 `CRAWL_HTTP_403`),"真错误"走的是另一通道。搜索侧"6 批全失败"回执为 2 批 `result_format` + 4 批熔断跳过 + 1 批传输瞬断;当日原始响应未留存,限流与形状变化无法事后判定(同查询 2026-10-16 实测 5/5 正常,原始响应为单文本块 `Title:/URL:` 记录格式)。
33
+
34
+ 3.1 抓取侧修复(已完成):`_boundary_text` 不再对未闭合围栏整批放弃;保守规则细化为安全阀——被遮罩页头的 URL 集合恰好等于缺失的请求 URL 集合(无重复、各出现一次)时按记录恢复,否则整批保持 raw;新增状态拆分 `no_content_returned`(真无正文)与 `content_present_unparsed`(有正文、记录级解析失败),熔断只对前者计数,本地解析缺口不再消耗提供商熔断预算。残余风险:被遮罩示例恰好逐一引用缺失 URL 时会被误收;条件已收紧到精确集合相等,且原始响应始终归档可复核。
35
+ 3.2 搜索侧保留未识别原始响应(已完成):`result_format` 失败的批次在回执中携带 `unparsed_excerpt`(≤400 字符、空白折叠);"限流形状映射 + 一次失败即冷却"(原 3.2)待拿到真实样本再定,冷却逻辑暂不动。
36
+ 3.3 (可选,未做)提供商健康跨进程持久化:冷却状态写入数据目录,CLI 单次进程也能先跳过坏提供商;可与 §5-7 任务型排序合并。
37
+ 验收:fixture 8/8 恢复(已达成);下一次真实 fail-closed 事件在回执中可读到原始响应开头(机制已就位,待实测)。
38
+
39
+ ## 4. Wiki 编译辅助(P3)
40
+
41
+ 4.1 `wiki work` 增加只读草稿大纲:按 question 或 topic 分组 eligible claims(附引文首句),供宿主作者快速成页;不落盘、不改审阅门槛。
42
+ 验收:对 `r-eb7778ec8c954b9a` 输出 2–3 个页面大纲。
43
+
44
+ ## 5. 扩展项(按需,多数需 Key,沿用 §5 清单)
45
+
46
+ - web_map 与 tavily_crawl(需免费 Tavily key)、Context7 文档通道(零 key 可跑)、免费池加入顺序(§5-1/2/5)。
47
+ - 交叉印证标记(§5-3):provider_count 提升为显式多源命中信号;边界不变,印证的是链接重叠,不是内容真值。
48
+ - 档位与通道(第 3 项):Exa REST instant/fast、Parallel turbo;需 Key。
49
+ - 维持否决:Parallel 摘录默认与 allow_live_fetch 纯缓存模式(理由见 §9.2);定位不扩到向量库、知识图谱、多模态产出(调研结论:属范围取舍,非缺口)。
50
+
51
+ ## 6. 风险与边界
52
+
53
+ - 新功能默认关闭或只读(沿用 §9.2 风格:`cache_seconds` 与 `reuse_seconds` 默认 0);replay 与预算语义不得回退。
54
+ - 引文级检查与草稿大纲都不替用户做语义判断;审阅门槛与不可变修订保持。
55
+ - 每次发版前跑全量测试与真实网络冒烟;激活后核对 receipt 与运行代码。
56
+
57
+ ## 7. 实施记录(2026-10-16)
58
+
59
+ - **抓取侧修复**(`src/archive.py`):`_boundary_text` 不再对未闭合围栏整批放弃(遮罩只影响边界识别,正文仍按原始文本切片);`_text_rows` 保守规则细化为安全阀——被遮罩页头的 URL 集合恰好等于缺失的请求 URL 集合(无重复、各一次)时按记录恢复,否则整批保持 raw;`_select` 增加 `empty_status` 参数,新增 `_carries_content`/`_empty_status`;`save`/`compact_fetch` 传递空状态。`compact_fetch` 同时收紧背书检查:归档存在时只服务与 manifest sha256 一致的正文(manifest 未记录的正文一律拒绝),旧解析器写的 capture 不会经新解析器"复活"——该收紧由全量测试暴露(`test_legacy_mixed_source_capture_is_rejected_on_read_citation_and_existing_archive_search`)。
60
+ - **熔断与状态**(`src/web_search.py`):`_fetch_coverage` 传递空状态;抓取熔断只对 `no_content_returned` 计数,`content_present_unparsed` 不再消耗提供商熔断预算;waterfall quality 排除表扩展两个新状态。
61
+ - **搜索侧留存**(`src/web_search.py`):`result_format` 失败批次在回执中携带 `unparsed_excerpt`(≤400 字符、空白折叠);搜索冷却逻辑未动(待真实样本再定)。
62
+ - **测试**:新增 fixture `tests/fixtures/exa-fetch-8url-fence-desync-20261015.json`(2026-10-15 真实归档响应,63,410 字符);新增/改写 8 项:fixture 8/8 恢复(archive 与 coverage 两层)、未闭合围栏恢复缺失请求页、隐藏未请求页保持 raw、空信封/未解析区分、未解析内容不触发熔断、搜索回执 excerpt、截断围栏恢复且不串页、截断围栏隐藏未请求页保持 raw。全量 666 项通过、3 项跳过、128.5 秒。
63
+ - **冒烟(真实网络)**:`fetch-web https://example.com` 正常(archive saved);同查询 `search-web` 5/5 URL 正常。
64
+ - **文档修正**:§0/§1.4/§2 的"3 个警告"改为"3 条复核类警告"并列明 lint 其余计数;`search-hedge-plan-2026-10-14.md` 追加 §9.3 修正记录。
65
+ - **状态**:改动在工作树,未提交;§3 仅剩 3.3(跨进程健康持久化)可选未做。
@@ -38,7 +38,7 @@ Its structured result contains `status:ready/blocked`, `research_id`, `total`, `
38
38
 
39
39
  ## Reader children
40
40
 
41
- Read exactly `{assigned_inventory_ids}`. Recover original URLs and bookmark context with research_inventory filtered by those IDs. Inspect existing reviews/evidence first. Fetch needed URLs in batches of up to 8 where applicable, respecting initial_fetch_plan and the existing budget. Read relevant saved-text pages through research_source. Search only for identified evidence gaps.
41
+ Read exactly `{assigned_inventory_ids}`. Recover original URLs and bookmark context with research_inventory filtered by those IDs. Inspect existing reviews/evidence first. Fetch needed URLs in batches of up to 16 where applicable, respecting initial_fetch_plan and the existing budget, and pass `reuse_seconds` (for example 3600) so a URL an earlier reader already saved returns its existing text. Read relevant saved-text pages through research_source. Search only for identified evidence gaps.
42
42
 
43
43
  Operation IDs begin with `{run_key}-r{round}-g{group_index}`. Review page identity, relevance, date/version and completeness with source_review. Record source-backed claims with exact quotations and question IDs.
44
44
 
@@ -95,7 +95,7 @@ Check saved progress and live host state after timeout or interruption. Pending/
95
95
  Codex follows the same scope, language and evidence rules using native tools; it does not run the shared JavaScript workflow. The canonical delegation file explicitly requests native agents for multi-group research. Inherit the current model, reasoning, permissions and tools unless the user chooses otherwise.
96
96
 
97
97
  1. Read research_status, explicitly resume incomplete research and never reopen completed/cancelled research. Read every unfiltered inventory page. The optional `python3 hosts/codex/prepare.py --research-id RID` helper prepares exhaustive groups and starts no agents. Preserve every `u-` ID and bookmark instance; package source_id, inventory ID and `sN` evidence ID differ.
98
- 2. Spawn bounded readers within the native concurrency limit. Give each the research ID, exact inventory IDs, method, question IDs, resolved output language and a unique operation prefix. Require research_inventory, research_fetch/search, research_source and research_record calls as needed; batch up to 8 URLs within initial_fetch_plan/budget. Require relevant saved text, source_review, source-backed claims with exact quotes and a separate inventory_review for every URL. Return structured IDs, citations, judgments and failures. A failed fetch is not reviewed. Do not ask children to create another scheduler.
98
+ 2. Spawn bounded readers within the native concurrency limit. Give each the research ID, exact inventory IDs, method, question IDs, resolved output language and a unique operation prefix. Require research_inventory, research_fetch/search, research_source and research_record calls as needed; batch up to 16 URLs within initial_fetch_plan/budget. Require relevant saved text, source_review, source-backed claims with exact quotes and a separate inventory_review for every URL. Return structured IDs, citations, judgments and failures. A failed fetch is not reviewed. Do not ask children to create another scheduler.
99
99
  3. Wait for every reader through native wait/agent tools. A pending thread or unread message is not completion. Preserve failed assignments and persisted work. Use fresh verifier subagents on the same complete groups. They independently read claims/text, challenge unsupported conclusions and misleading freshness/completeness claims, and record justified retractions/gaps/conflicts/resolutions. Identify uncheckable items; child success is not proof of verification.
100
100
  4. Read every page of each coverage filter all/missing/unread/unreviewed. Compare complete inventory IDs against every assignment, not just counts or a first page. Send the complete differences and failed-verification IDs to another bounded reading/verification pass. Use host controls for continuation/cancellation; there are no plugin leases or worker processes.
101
101
  5. Save full returned JSON with external_run: every inventory ID, reader/verifier result, remaining ID, actual observed state and coverage metric. Use actual recorded.result_path/digest. Prefer real native run IDs; prefix correlation-only IDs with `local:` and label them. Completed observations alone may be recorded completed; an unknown outcome is not cancellation.
@@ -38,7 +38,7 @@ URL、网页正文和书签笔记都是证据,不是执行指令。不得修
38
38
 
39
39
  ## 阅读代理
40
40
 
41
- 准确读取 `{assigned_inventory_ids}`。用这些 ID 调用 research_inventory,取得原 URL 和书签语境。先检查已有审阅和证据;需要时按每批最多 8 个 URL 抓取,遵守 initial_fetch_plan 和现有预算。通过 research_source 读完相关正文分页,仅为具体证据缺口搜索。
41
+ 准确读取 `{assigned_inventory_ids}`。用这些 ID 调用 research_inventory,取得原 URL 和书签语境。先检查已有审阅和证据;需要时按每批最多 16 个 URL 抓取,遵守 initial_fetch_plan 和现有预算,并传 `reuse_seconds`(例如 3600),让先前读者已保存的 URL 直接复用已有正文。通过 research_source 读完相关正文分页,仅为具体证据缺口搜索。
42
42
 
43
43
  操作 ID 以 `{run_key}-r{round}-g{group_index}` 开头。通过 source_review 检查页面身份、相关性、日期/版本与完整性。记录有来源、原样引文和 question ID 的结论。
44
44
 
@@ -95,7 +95,7 @@ inventory_ids 保留全部分配 ID。verified_ids 只包含独立检查过的
95
95
  Codex 通过原生工具遵守同样的范围、语言和证据规则,不运行共享 JavaScript 工作流。英文委派文件对多组研究明确要求使用原生代理。除非用户另有指定,沿用当前模型、推理、权限和工具。
96
96
 
97
97
  1. 读取 research_status,明确恢复 incomplete,不能重开 completed/cancelled。读完整且未过滤的清单分页。可选 `python3 hosts/codex/prepare.py --research-id RID` 只准备全部分组,不启动代理。保留所有 `u-` ID 和书签实例;包 source_id、清单 ID、`sN` 证据 ID 不同。
98
- 2. 在原生并发上限内派有明确范围的读者。传入 research ID、完整分配 ID、方法、question IDs、确定的输出语言及唯一操作前缀。按需用 research_inventory、research_fetch/search、research_source、research_record;适用时每批最多 8 URL,遵守 initial_fetch_plan/预算。要求阅读相关已存正文、保存 source_review、精确引文的结论及每个 URL 独立的 inventory_review,返回结构化 ID、引文、判断和失败。失败抓取不能记已审阅,不要求子代理另建调度器。
98
+ 2. 在原生并发上限内派有明确范围的读者。传入 research ID、完整分配 ID、方法、question IDs、确定的输出语言及唯一操作前缀。按需用 research_inventory、research_fetch/search、research_source、research_record;适用时每批最多 16 URL,遵守 initial_fetch_plan/预算。要求阅读相关已存正文、保存 source_review、精确引文的结论及每个 URL 独立的 inventory_review,返回结构化 ID、引文、判断和失败。失败抓取不能记已审阅,不要求子代理另建调度器。
99
99
  3. 通过原生等待/代理工具等所有读者结束。pending 线程或未读消息不代表完成,保留失败分组与已存工作。派新核验代理处理相同完整分组,亲自读取结论/正文,质疑不受支持的结论和错误的实时/完整性说法,按实际证据记录撤回、缺口、矛盾及解决。标明无法检查的项,子代理成功不证明核验成功。
100
100
  4. 分别读完 all/missing/unread/unreviewed 的全部覆盖分页,将完整清单 ID 与每份分配比较,不能只看计数或第一页。把完整差集和核验失败 ID 交给下一轮有界阅读与独立核验。继续/取消由宿主控制,没有插件租约或 worker 进程。
101
101
  5. 用 external_run 保存完整返回 JSON,包含全部清单 ID、读者/核验结果、剩余 ID、实际观察状态与覆盖指标。采用真实 recorded.result_path 和摘要。优先保存原生 run ID;只有关联号时使用 `local:` 前缀并说明。只有已结束的观察能记录 completed,未知结果不等于取消。
@@ -0,0 +1,31 @@
1
+ # Bookmark Research 0.5.0 beta 12
2
+
3
+ **Beta / 测试版.** Published through the npm `beta` channel; the `latest` tag remains unchanged.
4
+
5
+ ## Install
6
+
7
+ Pin both the installer and the plugin content:
8
+
9
+ ```sh
10
+ npx bookmark-research@0.5.0-beta.12 install --source npm:bookmark-research@0.5.0-beta.12
11
+ ```
12
+
13
+ Choose Codex, Claude Code, Pi or DSH in the installer. For an existing DSH profile:
14
+
15
+ ```sh
16
+ dsh plugin --profile <name> add bookmark-research@0.5.0-beta.12
17
+ ```
18
+
19
+ ## Changes
20
+
21
+ - **Hedged Fetching & Round-Trip Compression**: Introduced request hedging across search providers, racing fetch requests, compressing round trips, and prioritizing reuse of previously verified local evidence.
22
+ - **Fence Desync Page Recovery**: Added intelligent markdown code fence desync recovery in `archive.py`. When provider markdown truncates fences but contains valid requested page headers, the parser safely extracts and isolates each page without corrupting boundaries or dropping content.
23
+ - **Provider Circuit Breaker Refinements**: Refined fetch breaker logic in `web_search.py` to differentiate between empty provider responses and unparsed content layouts, preventing spurious provider cooldowns when valid data is received.
24
+ - **Bounded Diagnostic Excerpts**: Captured bounded raw excerpts for unreadable or malformed search provider results to streamline debugging without bloating logs.
25
+ - **End-to-End Reliability**: Full test suite updated to 666 passing tests covering fence desync recovery, provider hedging, and multi-host installations.
26
+
27
+ ## Verification
28
+
29
+ Full unit test suite (666 tests) passing across Linux, macOS and Windows environments.
30
+
31
+ Install from [npm](https://www.npmjs.com/package/bookmark-research/v/0.5.0-beta.12). GitHub provides source and release notes; npm is the shared release package for all four hosts.
@@ -0,0 +1,158 @@
1
+ # 搜索对冲与总期限实施计划(2026-10-14)
2
+
3
+ > 本轮只实施一件事:把 `search_web` 主提供商阶段的"等待全部"改为"对冲加总期限"。
4
+ > 不新增 MCP 工具、不新增提供商、不改 fetch 路径。其余待办见文末"后续待办",供下次讨论后再做。
5
+
6
+ ## 1. 背景与本机实测
7
+
8
+ 2026-10-14 在本机安装版上的实测(单次采样,含约 60ms 进程启动,受网络条件影响):
9
+
10
+ | 组合 | 总耗时 |
11
+ | --- | ---: |
12
+ | 仅 Exa | 6.36 s;复测 5.22 s |
13
+ | 仅 Parallel | 2.00 s |
14
+ | 两者并行 | 5.61 s |
15
+
16
+ 结论:跨提供商并行已生效(5.61 秒约等于慢者而非相加),但 `web_search.search()` 等待全部主提供商,最慢的提供商决定总延迟,Parallel 2 秒得到的结果被扣住 3 秒多。参考项目 aether-search 的两通道同样"等全部",其竞速仅用于 Grok 多端点且默认关闭;smartsearch 无竞速。文献(Google "The Tail at Scale" 的对冲请求)表明先到先得加取消输家可在约 2% 额外请求量下大幅降低尾部延迟。
17
+
18
+ 在线探测(2026-10-14):Exa 与 Parallel 的 hosted MCP 搜索工具均不暴露速度档(instant/fast/turbo 只在 REST API),因此对冲是当前 MCP 通道上唯一可用的延迟杠杆。
19
+
20
+ ## 2. 本轮范围
21
+
22
+ 做:
23
+
24
+ - `SearchProviders.search()` 主提供商阶段:先到先得 + 宽限窗口 + 总期限 + 放弃披露。
25
+ - 新设置 `search.grace_seconds`(默认 2,null 关闭宽限)与 `search.deadline_seconds`(默认 10,null 不设总期限)。
26
+ - 研究路径(`research_search`)显式 `hedge=False`,保持等待全部与预算语义不变。
27
+
28
+ 不做(后续再议):不新增 MCP 工具(web_map 等)、不新增提供商、不改 fetch 路径、不动 fallback 阶段。
29
+
30
+ ## 3. 设计
31
+
32
+ 主提供商阶段语义:
33
+
34
+ 1. 全部主提供商工作线程同时启动(现状不变)。
35
+ 2. 任一提供商完成后:若所有查询均已解析,最多再等 `grace_seconds`,窗口内完成的提供商结果照常并入 RRF;窗口结束仍未完成的提供商被放弃。
36
+ 3. 若仍有查询未解析,继续等待其余提供商,直到全部完成或到达总期限。
37
+ 4. 总期限到达时返回已到结果,未完成的提供商放弃。
38
+ 5. 输出在 `routing` 增加 `hedge`(生效的 grace/deadline 值)与 `abandoned_providers`(provider 加 reason:grace_expired/deadline)。
39
+ 6. fallback 阶段(仅在存在未解析查询时触发)保持等待全部,不设期限。
40
+
41
+ 边界与如实声明:
42
+
43
+ - 线程无法真正取消:被放弃的请求可能继续执行到超时,并照常消耗提供商额度;其 usage 不计入本次返回(在 routing 与 retry_policy 中说明)。
44
+ - 对冲使结果集依赖时序;研究路径用 `hedge=False` 保持完整性优先。
45
+ - grace 与 deadline 接受整数或小数秒,范围 0 到 60;grace=0 表示解析完成即返回。
46
+
47
+ 实现清单:
48
+
49
+ - `src/settings.py`:defaults 与校验增加两个字段。
50
+ - `src/web_search.py`:新增 `_search_batches_hedged`;`search()` 增加仅关键字参数 `hedge`(None 用设置,False 强制等待全部);routing 输出。
51
+ - `src/research.py`(`research_search` 调用处):加 `hedge=False`。
52
+ - `src/mcp_server.py`:SETTINGS_SCHEMA 的 search 增加两字段;`search_web` 描述补一句。
53
+ - `skills/bookmark-research/references/settings-and-archive.md`:补充两个字段说明。
54
+ - `tests/test_web_search.py`:新增对冲行为测试(宽限内合并、超窗放弃、未解析继续等待、期限放弃、null 与 hedge=False 保持等待全部)。
55
+
56
+ ## 4. 验收标准
57
+
58
+ - 全量测试通过(现基线 633 项、3 跳过)加新增测试。
59
+ - 固定题集本机复测:慢提供商场景总耗时从约 5.6 s 降到约 4.0 s(grace=2);两家都快的场景不增加延迟(宽限窗口内合并)。
60
+ - `research_search` 行为与预算语义不变;`abandoned_providers` 与 usage 口径如实。
61
+
62
+ ## 5. 后续待办(防遗忘)
63
+
64
+ 以下每一项都在 2026-10-14 会话中核实过事实与来源,下次讨论后按此清单逐项决定是否实施。
65
+
66
+ 1. **web_map 站点地图 / tavily_crawl**。插件现有 Tavily 连接已暴露 `tavily_map` 与 `tavily_crawl`(2026-10-14 在线探测);aether 的 web_map 就是调 Tavily `/map`(`aether-search-main/src/aether_search/fetcher.py:69`,参数 url/instructions/max_depth/max_breadth/limit)。keyless 模式不支持 map/crawl,需免费 Tavily key(1,000 credits/月,无需信用卡;map 每 10 页 1 credit,带 instructions 2 credits)。改动 = 白名单加工具 + 适配器加参数 + 工具入口。
67
+ 2. **Context7 文档通道**。smartsearch 的 `providers/context7.py`:两段式(先搜库再取该库代码与文档片段),公开库免费、key 可选。接入模式同 Jina(http 传输),零 key 可先跑。
68
+ 3. **交叉印证标记**。aether `verify.py` 两级核验(证据集交叉印证 + 存活探测);插件 RRF 输出已有 `provider_count`,可提升为显式"多源命中"信号,探活做成可选。保留"印证的是链接重叠,不是内容真值"的边界。
69
+ 4. **Parallel fetch 修正**。适配层现发 `full_content: true`(`src/provider_adapters.py:126`),Parallel 官方工具描述警告其显著增加体积与延迟;其 `web_fetch` 还有 `allow_live_fetch=false` 纯缓存模式未被使用。计划:默认摘录 + 显式全文 + 缓存选项。
70
+ 5. **免费池与提供商加入顺序**(aether `docs/BACKENDS.md`,2026-07-27 一手核实):Tavily 1,000/月无卡(建议第一个拿,解锁 map)> You.com $100 约 20,000 次无卡 > Parallel 5,000/月 > Linkup 4,000 一次性 > Serper 2,500(Google 透传)> Exa $10/月加注册 $20 > Firecrawl 约 1,000/月 > Brave $5/月(需绑卡)> Perplexity Sonar 无免费层。国内层候选:智谱(0.01 到 0.05 元/次;smartsearch 有 `zhipu.py` 与 `zhipu_mcp.py` 两种现成实现)、博查(REST,`api.bochaai.com/v1/web-search`,带语义重排 API)。加入方式沿用现有可选机制(`search.providers` 与 `search.fallback_providers` 选择启用,单次可用 `providers` 参数限制);建议新提供商默认不进入默认列表(opt-in),由用户显式启用,避免升级后悄悄增加调用量与额度消耗。Grok/xAI 备注(2026-10-15):它是答案引擎而非结果列表 API;aether 以 Grok 为主通道、smartsearch 以 `xai-responses` 为默认链首,两者都产出带引用的模型答案。若接入本插件,位置应是外部研究服务槽(opt-in,与 OpenAI/Parallel 并列),不是 `search.providers`;独有价值是 X 搜索;无免费层(Live Search 约 $0.025/来源,另加 token 费)。
71
+ 6. **子代理编排开箱化**。子代理调用插件 MCP 工具已实测通过(23 次调用、约 12.6 分钟、16 万 token 完成画布查询),但有摩擦:`search_bookmarks` 与 `search_web` 的 targets 形状不一致(字符串数组 vs 对象)、多词 target 是 AND 字面匹配须拆短词、只查元数据字段(正文走 `search_archive`)。bundle 内已有 `workflows/bookmark-research.js`,待启用与验证。
72
+ 7. **任务型提供商排序**。openbenchmarks(2026-09/10):无通吃赢家。Exa fast 569ms/99.3%(事实查询)、Perplexity 77.3%(开发任务)、Parallel basic 93.3%/1.8s、Parallel turbo 333ms/64.7%。可据此做默认顺序的启发式或按任务预设。
73
+ 8. **发布注意**。本机安装回执(`~/.local/share/bookmark-research/installations/claude-6e6c0896f096aa68/receipt.json`)仍为 `0.5.0-beta.10`,bundle 的 src 与仓库逐字节一致(beta11 只改安装器脚本)。本次改动完成后需重装 bundle、重启会话,并核对 receipt 版本与运行代码一致。
74
+ 9. **Exa 免 Key 通道 fail-closed 复发调查**。2026-10-15 两次观测(q2 检索批次、8 URL 批量抓取)中 Exa 返回无法识别的结果形状(`unrecognized_or_not_returned`),fallback 到 Parallel 后全部成功;同日早些时候 Exa 正常。需查明是匿名配额限流还是响应形状变化,决定调整识别逻辑或默认顺序(可与第 5 项免费池排序一起做)。
75
+
76
+ ## 6. 实施记录
77
+
78
+ 2026-10-14 至 10-15 完成(工作树改动,尚未提交):
79
+
80
+ - 改动文件:`src/settings.py`(defaults 与校验:`grace_seconds` 默认 2、`deadline_seconds` 默认 10,接受 null 与小数秒)、`src/web_search.py`(新增 `_search_batches_hedged`;`search()` 增加仅关键字参数 `hedge`;routing 增加 `hedge` 与 `abandoned_providers`;retry_policy 补充放弃说明)、`src/research.py`(research_search 调用加 `hedge=False`)、`src/mcp_server.py`(SETTINGS_SCHEMA 与 search_web 描述)、`skills/bookmark-research/references/settings-and-archive.md`(中英两版)、`tests/test_web_search.py`(新增 6 项对冲测试),以及 `tests/test_settings.py`、`tests/test_research.py`、`scripts/verify_fixture.py` 的假引擎签名适配。
81
+ - 测试:定向 8 个模块 150 项通过;全量 `python3 -B -m unittest discover -s tests` 639 项通过、3 项跳过、125.8 秒(基线 633 项加新增 6 项)。
82
+ - 本机联网复测(2026-10-15 凌晨,仓库新代码):同一条查询 7.56 秒,exa 与 parallel 均完成且合并,`abandoned_providers` 为空,routing 正确输出 `hedge` 字段。本次两家都慢且完成时间接近,未触发放弃;对冲收益出现在提供商延迟分化的场景:按 2026-10-14 白天实测的分化(Parallel 2.0 秒、Exa 5.2 秒),grace=2 时应在约 4.0 秒返回并放弃 Exa,从约 5.6 秒降到约 4.0 秒;调低 grace 可换更快。
83
+ - 待激活:本会话运行的 MCP 仍是旧 bundle 代码;需按第 5 节第 8 项刷新安装并重启会话后生效。
84
+
85
+ ## 7. 追加:搜索策略可选(2026-10-15)
86
+
87
+ 按用户要求把"怎么搜"做成配置项,也为后续增加提供商做准备(提供商越多,策略间的额度差异越大)。新增 `search.strategy`:
88
+
89
+ | 值 | 行为 | 额度 | 延迟 |
90
+ | --- | --- | --- | --- |
91
+ | `hedged`(默认) | 全部主提供商并发,先到先得加宽限窗口 | 每家都调用 | 最快 |
92
+ | `staggered` | 先发第一家,宽限时间内未全部解析再发下一家;已解析则后面的不调用 | 未调用的省下 | 居中 |
93
+ | `sequential` | 按顺序逐家调用,只把未解析查询交给下一家 | 最省 | 等于所用到的最慢一家 |
94
+ | `wait_all` | 等全部提供商返回(改动前的行为) | 每家都调用 | 最慢 |
95
+
96
+ - 配置入口:设置文件、MCP `update_settings`、CLI `config set --search-strategy`。
97
+ - `grace_seconds` 与 `deadline_seconds` 只对 hedged 与 staggered 生效;两者均为 null 时自动等价 wait_all;研究路径固定 wait_all 不受影响。
98
+ - 输出新增 `routing.strategy` 与 `routing.not_attempted_providers`。
99
+ - 测试与文档随本轮同步更新(英文与中文设置参考、details 两版)。验证:全量 645 项通过、3 项跳过、127.7 秒;CLI `config set --search-strategy` 与非法值拒绝在临时配置上实测通过。
100
+
101
+ ## 8. 研究路径实测:画布调研包完整生命周期(2026-10-15)
102
+
103
+ 用仓库新代码(`python3 -B src/cli.py`,非本会话旧 bundle)在画布调研包上跑了一遍完整研究生命周期,任务 `r-748272ccc9fc4749`:source `research` 以 whole 模式冻结 207 个 URL(223 个实例),预算 6 搜索/6 抓取/3 轮,问题 q1(搜索 API 延迟与质量对比)与 q2(Exa/Parallel 模式与延迟档位)。
104
+
105
+ - 流程(全部实测通过):start → search 第 1 轮(4 个 provider 批次,约 7 秒,http_requests 10)→ fetch 2 次操作 3 个 URL(exa 提取 12000/10769/8434 字符)→ source 全文读取 → record 两批 12 条(3 source_review、6 claim、1 inventory_review、2 answer)→ coverage → finish(incomplete)。
106
+ - 引文为已存正文的逐字子串;q1 结论:AA 搜索指数 80/75/74/73 对基线 33,Time per Task(模型+实测搜索时间)Exa auto 33.1s、Parallel basic 27.7s、Parallel advanced 42.7s,单次快不等于每任务总时间短;q2 结论:Exa 六档搜索类型(约 250ms 到 12-40s),Parallel 四模式(turbo/fast/basic/advanced,AA 口径)。
107
+ - 覆盖率:207 个冻结 URL 中 substantive review 1 个(0.48%,画布保存的 Exa 指南页,并完成 inventory_review),206 个未读;问题完成 2/2;finish 正确判为 incomplete,blocker 为“原始输入仍需实质审阅”。产物:任务目录下 `report.md`、`sources.json`、`coverage-*.json`。
108
+ - 观察:研究路径 routing 为 `strategy: wait_all`(hedge=False)符合设计,同日 search-web 对冲实测 5.17s(两家完成、无放弃);Exa 对 q2 一次 fail-closed 格式错误,同查询复测成功(4.91s),但当日稍后一次 8 URL 批量抓取中 Exa 对 8 个 URL 全部 fail-closed(`unrecognized_or_not_returned`),fallback 到 Parallel 全部成功,已按复发问题记入第 5 节第 9 项;AA 页在冻结清单中为 zh 变体,本次抓取的 en 页由搜索发现,未计入 inventory_review。
109
+ - 激活状态不变:本会话 MCP 仍为旧 bundle;研究路径新代码经 CLI 验证端到端可用。
110
+
111
+ ### 8.1 宿主与插件耗时归因(2026-10-15 实测)
112
+
113
+ 44 分 29 秒的任务墙钟拆开:插件侧联网操作合计 15.2 秒(检索轮 7.0s、两次抓取 4.3s/4.0s),占 0.6%;其余 99.4% 在宿主侧(模型回合与逐步操作),最大空档是检索完成到发起抓取之间的约 13 分钟、读文拼记录之间的约 14.5 分钟。瓶颈是宿主回合数乘每回合模型延迟,不是插件网络。
114
+
115
+ 对照实验(同任务 resume 后,resume 流程实测通过):一次 `research_fetch` 调用批量抓 8 个画布 URL,插件侧 23.3 秒、宿主 1 个回合、每页约 2.9 秒(1 URL 一次 4.0s、2 URL 一次 4.3s、8 URL 一次 23.3s);8 页全部经 fallback(Parallel)提取入库。批量把宿主回合数从 8 降到 1。
116
+
117
+ 可用杠杆(按性价比):1) 每次抓取 8 个 URL、每回合并行发多个工具调用(服务端支持 4 路并发、16 排队);2) 子代理/工作流并行审阅(第 5 节第 6 项);3) 控制工具输出体积,超大结果会拖慢其后每个宿主回合。
118
+
119
+ 批量抓取后的覆盖率口径(同任务实测):8 页全部入库(sources 19 条,含 8 条 Exa 失败记录),但实质指标不变(usable_text 1/207、substantive_review 1/207):抓取只把 `missing` 从 206 降到 200,`unread`/`unreviewed` 仍为 206——审阅才是实质覆盖,完成门槛因此仍未通过。8 个 URL 中 6 个直接对上冻结清单(accounted 1→7);两个 docs.parallel.ai 链接因清单中带 `_gl` 跟踪参数未直接匹配,审阅时按原 URL 处理。任务保持 active,新增问题 q3 待答;预算 fetch 4/6(失败的一次调用也计 1 次),余 2 次。
120
+
121
+ ## 9. 插件侧强化候选清单(2026-10-15,待讨论)
122
+
123
+ 背景:实测确认研究任务墙钟 99.4% 在宿主侧(§8.1)。以下为插件可自行强化、不依赖宿主的候选项,按慢宿主假设下的性价比排序;第 1、2 项已实施(见 9.1),第 4–7 项已实施(见 9.2,2026-10-15),第 3 项因免 Key MCP 通道不暴露档位而停放。
124
+
125
+ 1. **读取对冲与短熔断**。读取当前是"主提供商完成或失败后再并发其余"(今日 8 URL 实测:先整段 Exa 失败阶段,后 Parallel 成功,总 23.3s)。搜索侧已有 hedged/staggered/grace/deadline,可复用到 fetch:超宽限即并行启动 fallback(staggered 省额度、hedged 最快,与搜索一致);连续失败的提供商短时降级(熔断),避免每批先等它失败。与第 5 节第 9 项(Exa fail-closed 复发)联动。
126
+ 2. **回合压缩**。抓取已支持 8 URL/调用(实测 1 回合 8 页),可将上限提到 16 并让 Skill 默认满批。`research_source` 现在一次读 1 个来源,读 8 页需 8 个宿主回合;改为一次最多 8 个来源(每页限长 + next_offset)可把读取回合压 8 倍。Skill 明示每回合并行多个工具调用(服务端 4 并发、16 排队)。
127
+ 3. **档位与通道(需 Key)**。Exa REST instant/fast、Parallel turbo(第 5 节第 5/7 项);免 Key MCP 通道不暴露档位。单次延迟 10 倍杠杆,等 Key 决策。
128
+ 4. **缓存复用**。重复读取同一 URL 目前重新联网并追加快照;可加"新鲜度内直接返回已存正文"选项(source_freshness 已存在);搜索加短 TTL 缓存,服务子代理并行场景。
129
+ 5. **子代理/工作流开箱化**(第 5 节第 6 项)。慢宿主下唯一能并行上下文的路径;bundle 工作流待启用验证。
130
+ 6. **并发上限可配置**。4 并发/16 排队现为保护性硬限;配置化后配合 Key 提高吞吐,默认保持保守。
131
+ 7. **精简与一致性**。工具面本轮已精简,本地查询 68-107ms 非瓶颈;优先统一 targets 形状(第 5 节第 6 项)与继续限制工具输出体积。
132
+
133
+ 优先级:慢宿主下先做 1、2(复用现有机制、不依赖 Key);4、5 次之;3 等 Key;6 最后。宿主若变快,顺序变为 1、3、4 优先(网络时间重回主导),2、5 仍为低成本收益。
134
+
135
+ ### 9.1 实施记录:第 1、2 项(2026-10-15,工作树改动)
136
+
137
+ - **读取策略 `fetch.strategy`**:`staggered`(默认)在当前提供商停滞超过 `fetch.grace_seconds`(默认 25)后把未解决 URL 并行交给下一家,正常完成则等同 sequential;`hedged` 全部并发先到先得;`sequential` 为改动前行为;`fetch.deadline_seconds` 默认 null。放弃发生在全部 URL 已解决或到达期限时,被放弃者记入 `abandoned_providers`,其请求可能仍在后台消耗额度;放弃不阻塞返回,单次尝试且无放弃时保持旧输出形状,有放弃时保留瀑布形状(per-URL provider 归属与披露不丢)。
138
+ - **短熔断**:提供商连续两次整批无可用正文(读取 `unrecognized_or_not_returned`、搜索 result_format)即冷却 300 秒,期间快速跳过并记 `provider_stalled` 与 `skipped_due_to_cooldown`;任何可用结果清零计数;进程内生效。与第 5 节第 9 项(Exa 免 Key 复发)联动:复发时后续批次不再等它失败。
139
+ - **回合压缩**:抓取批量上限 8→16(`web_search.fetch`、`research.FETCH_BATCH_SIZE`、`fetch_web`/`research_fetch`/`wiki_refresh` schema、`read_capture`);`research_source` 支持 `source_ids`(一次最多 8 个,CLI `--source-id` 可重复);默认抓取预算公式改为 `min(80,max(12,ceil(URL数/16)+12))`(207 URL → 下限 13、默认 25)。
140
+ - **入口**:设置文件、MCP `update_settings`(SETTINGS_SCHEMA 同步)、CLI `config set --fetch-strategy`(非法值拒绝)。
141
+ - **验证**:全量 650 项通过、3 项跳过(新增:staggered 停滞重叠、hedged 先到先得与放弃披露、短熔断、批量读来源、fetch 策略默认与校验;同步更新:抓取边界 16、预算公式 13/25、wiki refresh 上限 17、641→1281 输入)。CLI 冒烟(真实网络):批量读来源一次返回 2 个(count=2)、`config set --fetch-strategy hedged` 生效、`fetch-web` 两个 URL 3.9s 单提供商形状正常。
142
+ - **未实施**:第 3 项(等 Key)、第 4–7 项(已于同日实施,见 9.2)。激活仍待第 5 节第 8 项(重装 bundle + 重启会话)。
143
+
144
+ ### 9.2 实施记录:第 4–7 项(2026-10-15,工作树改动)
145
+
146
+ - **第 4 项 缓存复用(两层,默认都关闭)**:
147
+ - 搜索缓存 `search.cache_seconds`(0–3600,默认 0):进程内按(提供商,查询,限额)缓存短时结果;命中批次标记 `cached:true`、保留原 `retrieved_at`、`usage` 记零、不调用提供商。研究预算诚实扣减:主阶段与 fallback 阶段都先问 `SearchProviders.cached_search_pairs()`(只统计距到期 5 秒以上的条目,避免预留后缓存过期导致超预算执行),命中项不占 `search_calls`。缓存只在 MCP 长驻进程内有效;CLI 每次命令是新进程,天然没有跨命令搜索缓存。
148
+ - 读取复用 `fetch.reuse_seconds`(0–86400,默认 0;`research_fetch` 可用同名参数覆盖):窗口内已有 extracted 正文的 URL 直接返回 `reused_sources`(原 `sN`、原检索时间,追加 `reused:true`,不追加新快照、不写新证据文件、不占 `fetch_calls`);全部复用时 `attempts` 为空、`per_url` 标 `provider:"reused"`、`status:"ok"`,预算已耗尽也能执行。复用基于任务持久状态,跨进程有效。`reuse_seconds=0` 不写入 operation parameters,旧回执重放参数不变。
149
+ - **被否决的做法(记录理由)**:不把 Parallel `full_content` 默认改为摘录——摘录是面向 objective 的 LLM 过滤文本,会破坏"引文必须是已存正文的逐字子串"的证据链;不启用 `allow_live_fetch=false` 纯缓存模式——静默返回陈旧正文与 `source_freshness` 的实时对比语义冲突。
150
+ - **第 5 项 子代理/工作流提示同步**:`hosts/shared/research-flow.js` 与 `hosts/codex/delegate.md` 的每批 8 → 16,并加"传 `reuse_seconds`(如 3600)让先前读者已保存的 URL 直接复用";`docs/prompt-reference.md`/`.en.md` 同步。工作流生效仍待第 5 节第 8 项(重装 bundle + 重启会话),生效后建议用一次有界工作流验证。
151
+ - **第 6 项 并发上限可配置**:新 `server` 设置节 `network_workers`(1–16,默认 4)与 `max_pending_calls`(1–64,默认 16,且不得小于 workers);`serve()` 新增 `workers`/`max_pending` 显式参数,CLI `serve` 启动时读取(配置不可读则回退默认并在 stderr 提示);CLI `config set --server-workers/--server-max-pending`,MCP `update_settings` 同步;重启后生效,默认值保持保守。
152
+ - **第 7 项 精简与一致性**:`search_web` 的 `targets` 现在接受纯字符串(schema `type:["string","object"]`,等价 `{target:s, query:s}`),与 `search_bookmarks` 的字符串 targets 不再割裂;`cli.md`、`research-workflow.md`(中英)与工具描述同步;多词 AND 字面匹配与"正文走 `search_archive`"维持既有文档说明;输出体积由 16 URL/8 来源批上限与分页承担。
153
+ - **验证(2026-10-15)**:全量 659 项通过、3 项跳过、135.2 秒(9.1 基线 650 加新增 9 项:搜索缓存命中与默认关闭、`cached_search_pairs`、字符串 targets、设置校验、读取复用单条/混合/预算、显式 serve 并发参数)。实测(仓库代码、真实网络):同一查询两次 5.53s→0.01s,第二次 `cached:[true,true]`、`tool_calls=0`、结果 5+5 条、`retrieved_at` 保留、预算扣减计数 2;`research_fetch` 第二次 `reused_sources=[s1]`(原检索时间)、`usage` 全零、`attempts=[]`、`per_url.provider="reused"`;新配置项写入与非法组合拒绝(`server.max_pending_calls must be at least server.network_workers`)实测通过。
154
+ - **未实施**:第 3 项(免 Key MCP 通道不暴露档位,需 REST + Key;用户确认不等待)。激活仍待第 5 节第 8 项(重装 bundle + 重启会话)。
155
+
156
+ ### 9.3 修正(2026-10-16):熔断读取状态与 fail-closed 归因
157
+
158
+ §9.1 的短熔断在抓取侧原读取 `unrecognized_or_not_returned`。该状态已拆分(`no_content_returned` / `content_present_unparsed`,另见 `plugin-improvement-plan-2026-10-15.md` §3):8 URL 批次"全部 fail-closed"经归档原文复现为本地解析拒收(第 2 页围栏未闭合,遮罩盖住后续真实页头),非提供商问题;熔断改为只对 `no_content_returned` 计数,本地解析缺口不再消耗提供商熔断预算。搜索侧 `result_format` 失败批次现于回执保留 `unparsed_excerpt`(≤400 字符)。
@@ -5,7 +5,7 @@ Use the native subagent tools visible in the current Codex session. This file ex
5
5
  Resolve the output language from the user's explicit requirement, otherwise their actual task/conversation, with English as the fallback. Preserve it in the research brief and pass it to every reader, verifier and follow-up assignment. Use it for authored answers, report prose, review notes and Wiki titles/sections. Keep original quotations, URLs, IDs, code and technical field names unchanged; put translations outside quotations. This English instruction file does not select the answer language. The bilingual reading reference is in `docs/prompt-reference.en.md` and `docs/prompt-reference.md`.
6
6
 
7
7
  1. Read `research_status`. Resume an `incomplete` research explicitly with `research_record` kind `resume`; never reopen completed/cancelled research. Read every page of `research_inventory` without narrowing the frozen scope. The optional `python3 hosts/codex/prepare.py --research-id RID` helper calls the same shared MCP tools and emits exhaustive groups; it starts no agents. Keep every `u-` inventory ID and its original bookmark instances. A local package `source_id`, a `u-` inventory ID and an `sN` extracted source ID are different identifiers.
8
- 2. Spawn one bounded reader per group within the native concurrency limit. Give each reader the research ID, exact assigned inventory IDs, method, relevant question IDs and a unique operation prefix. Readers call `research_inventory`, `research_fetch`/`research_search`, `research_source`, and `research_record`. Batch up to 8 needed URLs per fetch where applicable, respecting the existing `initial_fetch_plan` and budget. Require full relevant saved text, an explicit `source_review`, source-backed claims with exact quotes and a separate `inventory_review` for every assigned URL. Submit ready records through `research_record.entries` in batches of up to 50 with stable `batch_id` values under the reader's unique prefix. Place reviews before their claims; use returned claim IDs in subsequent inventory/answer batches. An invalid entry rejects the whole batch; identical retries reuse the batch ID. Return structured IDs, citations, judgments and failures. Failed fetches cannot become reviewed entries. Do not ask a child to create another scheduler.
8
+ 2. Spawn one bounded reader per group within the native concurrency limit. Give each reader the research ID, exact assigned inventory IDs, method, relevant question IDs and a unique operation prefix. Readers call `research_inventory`, `research_fetch`/`research_search`, `research_source`, and `research_record`. Batch up to 16 needed URLs per fetch where applicable, respecting the existing `initial_fetch_plan` and budget; pass `reuse_seconds` (for example 3600) so a URL an earlier reader already saved returns its existing text instead of being fetched again. Require full relevant saved text, an explicit `source_review`, source-backed claims with exact quotes and a separate `inventory_review` for every assigned URL. Submit ready records through `research_record.entries` in batches of up to 50 with stable `batch_id` values under the reader's unique prefix. Place reviews before their claims; use returned claim IDs in subsequent inventory/answer batches. An invalid entry rejects the whole batch; identical retries reuse the batch ID. Return structured IDs, citations, judgments and failures. Failed fetches cannot become reviewed entries. Do not ask a child to create another scheduler.
9
9
  3. Wait for every reader through the native wait/agent controls. A pending thread or unread message is not completion. Preserve failed assignments and already-persisted progress. Then use separate, fresh verifier subagents for the same complete ID groups. Verifiers independently read claims and saved evidence, challenge unsupported conclusions and misleading freshness/completeness claims, and use retraction/gap/conflict/resolution records where justified. They must identify uncheckable items; a successful child turn is not proof of successful verification.
10
10
  4. Read `research_coverage` with `all`, `missing`, `unread` and `unreviewed`, following `next_offset` separately for every filter. Compare the complete inventory with every returned assignment, not just the first page or a count. Send the full difference plus failed verification IDs to another bounded reading and independent verification pass when needed. Use the host's current controls for child continuation or cancellation; no plugin lease or worker process exists.
11
11
  5. Save complete returned JSON with `research_record` kind `external_run`: include all inventory IDs, reader/verifier results, remaining IDs, actual observed states, and coverage metrics. Read the saved location from `recorded.result_path` and its digest from `recorded.result_sha256`. Use actual native run IDs when available. If only a caller correlation key is available, prefix it with `local:` and say it is not a native host run ID. Record only completed observations as completed; an unknown outcome is not cancellation.
@@ -107,7 +107,9 @@ for (let round = 0; round <= gapRounds && remaining.length; round += 1) {
107
107
  "Read exactly these inventory IDs: " + JSON.stringify(ids) + ".\n" +
108
108
  "Call research_inventory with these inventory_ids to recover original URLs and bookmark context. " +
109
109
  "Inspect existing reviews and evidence first. For needed URLs use research_fetch in " +
110
- "batches of up to 8 URLs where applicable, respecting the existing initial_fetch_plan and budget. " +
110
+ "batches of up to 16 URLs where applicable, respecting the existing initial_fetch_plan and budget; " +
111
+ "pass reuse_seconds (for example 3600) so a URL an earlier reader already saved returns its existing " +
112
+ "text instead of being fetched again. " +
111
113
  "Read saved text with " +
112
114
  "research_source through relevant next_offset pages. Search only for specific evidence gaps. " +
113
115
  "Operation IDs must begin with " + JSON.stringify(input.run_key + "-r" + round + "-g" + index) + ". " +
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "bookmark-research",
3
- "version": "0.5.0-beta.11",
3
+ "version": "0.5.0-beta.12",
4
4
  "description": "Check bookmark URLs or Canvas packages, compare sources with host-led research, and retain verifiable reports.",
5
5
  "license": "GPL-3.0-only",
6
6
  "repository": {
@@ -50,7 +50,7 @@ class FixtureProvider:
50
50
  def __init__(self, fixture):
51
51
  self.fixture = fixture
52
52
 
53
- def search(self, targets, providers, limit_per_target):
53
+ def search(self, targets, providers, limit_per_target, hedge=False):
54
54
  batches = [{"provider": provider, "target": target["target"], "query": target["query"],
55
55
  "status": "ok", "retrieved_at": "2026-09-10T00:00:00Z", "results": [
56
56
  {"url": url, "title": self.fixture["pages"][url].get("title", "Unavailable fixture"),
@@ -55,9 +55,10 @@ python3 <root>/src/cli.py search-archive 'exact phrase or terms' --url example.c
55
55
  python3 <root>/src/cli.py config show
56
56
  python3 <root>/src/cli.py config set --search-provider exa --archive true
57
57
  python3 <root>/src/cli.py config set --archive-dir /absolute/path/to/knowledge
58
+ python3 <root>/src/cli.py config set --search-cache-seconds 120 --fetch-reuse-seconds 3600
58
59
  ```
59
60
 
60
- `config show` / `config set` correspond to MCP `get_settings` / `update_settings`. `config set --input /path/to/changes.json` (or `-` for stdin) merges partial settings. CLI flags override matching fields from that invocation's JSON. `fetch-web` archives by default; `--archive` / `--no-archive` affects one call. Set future defaults with `config set --archive true|false`. See [settings and archives](settings-and-archive.md).
61
+ `config show` / `config set` correspond to MCP `get_settings` / `update_settings`. `config set --input /path/to/changes.json` (or `-` for stdin) merges partial settings. CLI flags override matching fields from that invocation's JSON. `fetch-web` archives by default; `--archive` / `--no-archive` affects one call. Set future defaults with `config set --archive true|false`. `--search-cache-seconds` and `--fetch-reuse-seconds` set the search-cache TTL and the saved-text reuse window (0 disables, the default); `--server-workers` and `--server-max-pending` set stdio server concurrency and apply after restart. See [settings and archives](settings-and-archive.md).
61
62
 
62
63
  `setup` guides preferences, hidden key entry and selected service checks. `--non-interactive --input FILE` is the agent path; `--skip-checks` prevents network checks. `readiness` uses saved cached/always/manual policy; `--refresh` forces checks, `--offline` prevents network, and `--test-retrieval` opts into sample search/read quota. Repeat `--provider` to restrict retrieval checks, `--service openai|parallel` to select a professional service, and `--tool` for actually observed host tools. `--host` accepts `codex`, `claude_code`, `pi`, `dsh`, `unknown`. Use readiness for unclear access, diagnostics or external service selection, reusing current results; no professional job starts. See the settings reference for status meanings and private credential storage.
63
64
 
@@ -79,7 +80,7 @@ Use `search-web --input /path/to/query.json` for stable target labels and repeat
79
80
  }
80
81
  ```
81
82
 
82
- Limits are 12 target/query pairs and 20 returned URLs per target. `fetch-web` accepts up to 8 URLs per call. Providers run concurrently; each provider's requests run sequentially and reuse its tool catalog within the process.
83
+ Limits are 12 target/query pairs and 20 returned URLs per target. `fetch-web` accepts up to 16 URLs per call. Providers run concurrently; each provider's requests run sequentially and reuse its tool catalog within the process.
83
84
 
84
85
  To merge **actual results** from other host MCPs, normalize them to this shape and call `merge-results /path/to/batches.json`. Do not invent URLs, ranks or provider names from snippets.
85
86
 
@@ -117,6 +118,7 @@ python3 <root>/src/cli.py research coverage <research-id> --filter unreviewed --
117
118
  python3 <root>/src/cli.py research search <research-id> --input /path/to/search-step.json
118
119
  python3 <root>/src/cli.py research fetch <research-id> --input /path/to/fetch-step.json
119
120
  python3 <root>/src/cli.py research source <research-id> s1 --offset 0 --limit 12000
121
+ python3 <root>/src/cli.py research source <research-id> --source-id s1 --source-id s2 --limit 8000
120
122
  python3 <root>/src/cli.py research import-evidence <research-id> --input /path/to/actual-evidence.json
121
123
  python3 <root>/src/cli.py research record <research-id> --input /path/to/entry.json
122
124
  python3 <root>/src/cli.py research finish <research-id> --summary 'Verified conclusions' --status completed
@@ -26,7 +26,7 @@ Create answerable questions, scope, language and retrieval budgets with `researc
26
26
  ## Evidence cycle
27
27
 
28
28
  1. Fetch known URLs with `research_fetch`; discover missing sources with `research_search`, mapping queries to existing `question_id` values. Honor provider restrictions and inspect unresolved attempts.
29
- 2. Fetch returns metadata. Read saved text through `research_source`, continuing pagination for relevant passages. Check page identity, date/version and applicability; reject login shells or unrelated pages. An extract may be partial. Save `source_review` before claims.
29
+ 2. Fetch returns metadata. Read saved text through `research_source` (one source, or up to eight `source_ids` per call), continuing pagination for relevant passages. Check page identity, date/version and applicability; reject login shells or unrelated pages. An extract may be partial. Save `source_review` before claims.
30
30
  3. Record claims with exact saved quotations and actual evidence IDs. Quote/hash checks establish presence, not semantic support. Mark inferences with `inference:true`; preserve conditions and counterevidence in the final answer. Multiple providers returning one page are not independent sources.
31
31
  4. For original input, record `inventory_review` against accepted text from the matching original URL. A replacement page or external report may answer a question without satisfying original-URL coverage. Record specific blocks or justified exclusions when necessary.
32
32
  5. Record supported `answer` entries using that question's claim IDs. Check `research_coverage` when choosing follow-up work or finishing; fetch the relevant differences rather than rereading every unchanged record. Resolve conflicts or retain gaps.
@@ -67,12 +67,12 @@ Budgets bound plugin retrieval attempts, not money or model tokens:
67
67
  | Counter | Unit | Default / maximum |
68
68
  | --- | --- | --- |
69
69
  | `search_calls` | A deduplicated provider + question + query attempt | 16 / 120 |
70
- | `fetch_calls` | Provider attempt: Exa/Parallel/Tavily batch up to 8 URLs; Jina counts each URL | 12 without input; otherwise initial capacity plus 12 follow-ups, capped at 80 |
70
+ | `fetch_calls` | Provider attempt: Exa/Parallel/Tavily batch up to 16 URLs; Jina counts each URL | 12 without input; otherwise initial capacity plus 12 follow-ups, capped at 80 |
71
71
  | `rounds` | One `research_search` batch | 8 / 40 |
72
72
 
73
73
  Search accepts at most 12 queries and returns at most 20 URLs per question. Two queries across two providers cost four search calls and one round. A three-URL Exa → Parallel + Jina fallback costs 1 + 1 + 3 fetch calls.
74
74
 
75
- Without explicit `max_fetch_calls`, the default is `min(80,max(12,ceil(URL_count/8)+12))`; replace the batch term with `URL_count` when Jina is first. `initial_fetch_plan` reports capacity and shortfall. Preserve explicit budgets and full scope even when capacity is insufficient. Saved-source reads, records and reports use no retrieval allowance; budgets may be zero. Host/provider usage outside these calls must be recorded separately.
75
+ Without explicit `max_fetch_calls`, the default is `min(80,max(12,ceil(URL_count/16)+12))`; replace the batch term with `URL_count` when Jina is first. `initial_fetch_plan` reports capacity and shortfall. Preserve explicit budgets and full scope even when capacity is insufficient. Saved-source reads, records and reports use no retrieval allowance; budgets may be zero. Host/provider usage outside these calls must be recorded separately. Batch `research_fetch` up to 16 URLs and `research_source` up to eight `source_ids` per call: fewer round trips matter most when host turns are slow.
76
76
 
77
77
  Give each network step a stable `operation_id`. Identical parameters replay saved results; changed parameters fail. Reservations persist through auth/network failures and unknown outcomes. Do not enlarge budgets or resubmit solely to inspect a result.
78
78
 
@@ -32,7 +32,7 @@ Public queries contain only necessary public names, URLs and conditions, not ent
32
32
 
33
33
  ## Merging and verification
34
34
 
35
- `search_web` calls Exa + Parallel concurrently by default. Supply `targets:[{target,query}]`; `target` is a stable label and may have multiple queries. Merge, deduplicate, rank and limit results separately per target. Returned `batches` and result `sources` retain provider, query, URL, rank and retrieval time. Ranking scores control display order only.
35
+ `search_web` calls Exa + Parallel concurrently by default. Supply `targets:[{target,query}]`; `target` is a stable label and may have multiple queries, and a plain query string is shorthand for `{target: s, query: s}`. Merge, deduplicate, rank and limit results separately per target. Returned `batches` and result `sources` retain provider, query, URL, rank and retrieval time. Ranking scores control display order only.
36
36
 
37
37
  For each query without a usable HTTP(S) result from the primary group, call `search.fallback_providers` concurrently (defaults: Tavily + keyed Jina). Queries already answered by either primary provider are not repeated. Explicit per-call `providers` disables automatic fallback. Missing fallback keys appear in `routing.skipped_providers` without a network request. Inspect `unresolved_queries` and budget-limited `remaining_attempts`; each fallback provider/target/query reserves one attempt before access. New research sessions freeze their fallback list; explicit session providers and older sessions do not silently acquire extra providers.
38
38