bookmark-research 0.5.0-beta.1

package/docs/user-guide.en.md

@@ -0,0 +1,74 @@
+# Using Bookmark Research
+
+**English** · [中文](user-guide.md)
+
+Use one plugin in either language. Provide your package and explain the question, scope, and result you need. The host model reads the Skill and calls the tools; no fixed trigger phrase is required.
+
+## First use
+
+After installing, start a new host session so it loads the current Skill and tools. Provide your own Bookmark Canvas directory, ZIP, or single-card JSON:
+
+> Use Bookmark Research to read the package at "/absolute/path/to/my-package". First list its cards, folders, and bookmark counts offline.
+
+Local queries require no API key. The plugin starts without personal bookmarks or a seed database. Later requests can refer to the registered source; a new export of the same canvas should explicitly reuse that source.
+
+## Choose the depth and scope
+
+| Task | Example request | What happens |
+| --- | --- | --- |
+| Local query | “Which card contains this bookmark?” | Searches local metadata and canvas relationships. |
+| Quick check | “Read the official page I bookmarked and check whether it supports MCP.” | Reads the known URL, or searches to find a source, then answers with citations. |
+| Agentic search | “Compare the search tools in these cards, including their authentication and research capabilities.” | Uses card context to plan questions, searches and reads pages, and follows evidence gaps. |
+| Deep research | “Research every URL in this package, verify the rankings, compare it with our plugin, and save a report and Wiki.” | Preserves the full source inventory, organizes sustained investigation, reviews evidence and coverage, and saves the requested deliverables. |
+
+Quick checking does not switch the host to a non-reasoning model. Depth and scope are separate: a deep study may cover one explicitly selected card; a whole-package request keeps every original URL and every duplicate bookmark's context. Pagination, batching, and subagent count do not reduce that scope. Search snippets and external reports do not count as reading the original pages.
+
+Deep research can run through the host's reasoning, subagents, or existing workflows. Optional OpenAI Deep Research and Parallel Task APIs require separate configuration and credentials. Search access alone does not enable those services. An unavailable explicitly requested service is reported rather than silently substituted.
+
+## Snapshots and live directories
+
+Ordinary exports default to `snapshot`. The plugin saves the necessary source files and index, so the downloaded export can later be moved or deleted. Directories inside Git repositories default to `live`; existing registrations retain their selected mode. For a persistent full mirror, say:
+
+> Register this directory as a live source. It is the complete synchronized canvas directory.
+
+The plugin checks registered live directories while its MCP process runs and before queries. A standalone CLI command exits when finished; use `python3 src/cli.py watch` for continuous foreground monitoring. Git synchronization itself belongs to your existing sync tool.
+
+Partial exports preserve omitted files. A complete mirror can remove missing files; a supplied complete card can remove items even in partial mode. Single cards are always partial. Both modes save reusable versions. Updated input prompts research/Wiki review; it does not automatically fetch pages or rewrite conclusions.
+
+## Settings
+
+Ask “Show Bookmark Research settings” or run `python3 src/cli.py config show`. Set a lasting preference with a request such as “Use only Exa for future searches”; say “for this request” for a temporary override.
+
+| Setting | Default | Example |
+| --- | --- | --- |
+| Web search | Exa + Parallel; 5 hits per target per search call | “Use only Exa for future searches.” |
+| Page reading | Exa; requested length 12,000 characters | “Use Parallel to read this page.” |
+| Ordinary page archives | Enabled | “Do not archive this page read.” |
+| Research depth | `auto` | “Use deep research for this task.” |
+| Answer language | `auto`, follows the question | “Use Chinese for future reports.” |
+| Service readiness | Cached for 15 minutes; refresh on key/config changes | “Check availability before every question.” |
+| Professional research API | Disabled | “Show what is needed to enable OpenAI research.” |
+
+The search result limit is not a limit on package scope. Requested text length does not guarantee a complete page. Deep research always preserves its task evidence, independently of the ordinary page archive preference.
+
+Run `python3 <plugin-root>/src/cli.py setup` for guided preferences and hidden key entry. Keys come from the host environment first, then a private `credentials.json` next to settings: `EXA_API_KEY`, `PARALLEL_API_KEY`, `TAVILY_API_KEY`, `JINA_API_KEY`, and, for professional OpenAI research, `OPENAI_API_KEY`. Access and quotas depend on each provider. Host-led research needs no additional professional-service key.
+
+Before a new web question, the Skill checks selected services and observed host tools with `research_readiness`. Cached/always/manual modes control frequency; local queries stay offline. Optional native MCP connection/login steps are shown by setup, with actual OAuth handled in the host. Keys, visible tools and successfully tested operations have distinct status; checks never create a paid research job. See [settings and readiness](../skills/bookmark-research/references/settings-and-archive.md).
+
+Default settings live in `~/.config/bookmark-research/settings.json`; data lives in `~/.local/share/bookmark-research/`. XDG settings and the plugin's data/config path overrides are supported. SQLite stores the local query index; source versions, webpage archives, research records, and Wiki revisions live alongside it. Multiple hosts can share these locations. Keep user data outside the plugin and original canvas package.
+
+## Languages and evidence
+
+Answers, progress updates, report text and Wiki titles/sections follow the current explicit request, then saved `research.response_language`; `auto` follows the task and conversation. For example:
+
+> Research these Chinese and English sources, but write the report in English. Preserve original quotations and add translations separately.
+
+The Skill's instruction language does not determine output language. Installer `--lang auto|en|zh` selects help and onboarding; the wizard separately asks which answer language to save. With `auto`, installation checks `LC_ALL`, then `LC_MESSAGES`, then `LANG`; Chinese locales select Chinese, others select English. `BOOKMARK_RESEARCH_INSTALL_LANG` carries that choice from the shell bootstrap to Python.
+
+The execution Skill and all 11 method references are in English, with complete Chinese reading copies. The [instruction index](instructions.en.md) also links the bilingual host prompt reference. The host preserves the selected output language in research briefs and delegated tasks; scripted workflows accept `output_language`. If the task and conversation select no language, English is the fallback.
+
+Original URLs, identifiers, evidence text and quotations remain unchanged. Tool/JSON fields, technical diagnostics and fixed report/Wiki metadata labels remain in English; the host supplies the authored prose in your requested language. Historical research records retain their original language.
+
+If an older session exposes only some MCP tools, the host can use the bundled CLI for missing operations. A new session loads the updated Skill and tool catalog. Selecting a route or importing a package alone does not start background research.
+
+See the [shared Skill](../skills/bookmark-research/SKILL.md), [settings reference](../skills/bookmark-research/references/settings-and-archive.md), and [Wiki/evaluation guide](wiki-quality.md) for implementation details.

package/docs/user-guide.md

@@ -0,0 +1,74 @@
+# 使用 Bookmark Research
+
+[English](user-guide.en.md) · **中文**
+
+中英文用户使用同一个插件。提供数据包，并说明问题、范围和交付要求，宿主模型会读取 Skill、调用工具执行；不需要固定触发词。
+
+## 首次使用
+
+安装后新建宿主会话，以加载当前 Skill 和工具。提供自己的 Bookmark Canvas 目录、ZIP 或单卡 JSON：
+
+> 用 Bookmark Research 读取 "/absolute/path/to/my-package"，先离线列出卡片、文件夹和书签数量。
+
+本地查询无需 API Key。插件不附带个人书签或预建数据库。之后可沿用已登记来源；同一画布的新导出应明确说明沿用原来源。
+
+## 选择研究深度与范围
+
+| 任务 | 提问示例 | 实际行为 |
+| --- | --- | --- |
+| 本地查询 | “这个书签在哪张卡片里？” | 查询本地元数据和画布关系。 |
+| 快速查证 | “读一下我收藏的官方页面，确认它是否支持 MCP。” | 读取已有 URL，必要时搜索发现来源，再带引用答复。 |
+| 主动搜索 | “比较这些卡片中的搜索工具，包括认证方式与研究能力。” | 根据卡片语境拆问题，搜索、读文，并沿证据缺口继续补查。 |
+| 深度研究 | “全量研究这个包，核验排行榜，与我们的插件比较，保存报告和 Wiki。” | 保留完整来源清单，组织持续调查，审阅证据、核验覆盖，保存交付物。 |
+
+快速查证不会把宿主切换成非推理模型。深度与范围分别决定：深度研究可以只覆盖明确指定的卡片；要求研究整包时，保留每个原始 URL 和重复书签各自的语境。分页、分批和子代理数量不能缩小范围。搜索摘要和外部报告不能代替读取原始网页。
+
+深度研究可以使用宿主的推理、子代理或已有工作流。可选的 OpenAI Deep Research／Parallel Task API 需要单独启用并配置凭据。搜索可用不代表专业研究服务已启用；明确指定的服务不可用时应说明，不静默替换。
+
+## 快照与持续目录
+
+普通导出默认使用 `snapshot`，接入后保存必要原文件和索引，下载包移走或删除后仍可使用。Git 仓库中的目录默认使用 `live`；已登记来源沿用原模式。长期完整镜像可这样说明：
+
+> 把这个目录登记为持续来源，它是完整的画布同步目录。
+
+MCP 进程运行期间及查询前会检查 live 目录。一次性 CLI 命令执行后退出；要持续监控，运行前台 `python3 src/cli.py watch`。Git 同步仍由原有同步工具执行。
+
+局部导出保留此次未提供的文件；完整镜像可删除缺失文件。已提供完整栏目内的 item 删除在局部模式下也会同步，单卡固定使用局部模式。两种模式都会保存可恢复版本。输入变化会提示研究／Wiki 复核，不会自动抓网页或改写结论。
+
+## 配置
+
+说“显示 Bookmark Research 的配置”，或运行 `python3 src/cli.py config show`。要修改长期偏好，可以说“以后只用 Exa 搜索”；仅临时覆盖时说明“这次”。
+
+| 项目 | 默认值 | 修改示例 |
+| --- | --- | --- |
+| 网页搜索 | Exa + Parallel；每次搜索每目标 5 条命中 | “以后只用 Exa 搜索。” |
+| 正文读取 | Exa；请求长度 12,000 字符 | “这次用 Parallel 读取。” |
+| 普通网页归档 | 开启 | “这次读取不要归档。” |
+| 研究深度 | `auto` | “这次用深度研究。” |
+| 答复语言 | `auto`，跟随提问 | “以后的报告默认用中文。” |
+| 服务检查 | 缓存 15 分钟，配置／密钥变化后刷新 | “每次新问题前都检查服务可用性。” |
+| 专业研究 API | 关闭 | “显示启用 OpenAI 研究需要什么配置。” |
+
+搜索结果条数不限制数据包范围；长度参数不保证完整正文。深度研究始终保留任务证据，不受普通网页归档开关影响。
+
+可运行 `python3 <插件根目录>/src/cli.py setup` 引导设置偏好并隐藏输入密钥。密钥优先从宿主环境读取，其次读取配置旁的私有 `credentials.json`：`EXA_API_KEY`、`PARALLEL_API_KEY`、`TAVILY_API_KEY`、`JINA_API_KEY`；OpenAI 专业研究使用 `OPENAI_API_KEY`。访问和额度取决于服务方。宿主自身研究无需另配专业研究服务密钥。
+
+每个联网研究新问题前，Skill 用 `research_readiness` 检查所选服务和已观察到的宿主工具；cached／always／manual 控制频率，本地查询保持离线。setup 展示可选原生 MCP 的接入／授权步骤，实际 OAuth 由宿主管理。已填密钥、工具可见与实际操作成功分别报告，检查不会创建付费研究任务。详见[配置与就绪检查](../skills/bookmark-research/references/zh/settings-and-archive.md)。
+
+默认配置位于 `~/.config/bookmark-research/settings.json`，数据位于 `~/.local/share/bookmark-research/`，支持 XDG 及插件的数据／配置路径覆盖。SQLite 保存本地查询索引，旁边保存来源快照、网页归档、研究档案与 Wiki 修订。多个宿主可以共享这些位置，用户数据应位于插件和原始画布包之外。
+
+## 语言与证据
+
+答复、进展、报告正文和 Wiki 标题／章节优先采用本次明确要求，其次是保存的 `research.response_language`；`auto` 跟随实际任务和对话。例如：
+
+> 调查这些中英文资料，用中文写报告，保留原文引用，译文单独列出。
+
+Skill 指令本身的语言不决定输出语言。安装器 `--lang auto|en|zh` 控制帮助与提示；向导另外询问并保存答复语言。`auto` 依次读取 `LC_ALL`、`LC_MESSAGES`、`LANG`；中文 locale 使用中文，其余或未设置时使用英文。`BOOKMARK_RESEARCH_INSTALL_LANG` 用于 Shell 向 Python 安装器传递界面语言。
+
+执行 Skill 和 11 篇方法参考已使用英文，配完整中文阅读对照，[指令索引](instructions.md)也包含双语宿主提示说明。宿主将输出语言保存在研究 brief 并传给子代理；脚本工作流支持 `output_language`。任务和对话都没有选择语言时默认英文。
+
+原始 URL、标识符、正文证据和引用保留原样。工具／JSON 字段、技术诊断及报告／Wiki 的固定元数据标签仍使用英文，宿主编写的正文使用指定语言。历史研究记录保留原语言。
+
+旧会话只暴露部分 MCP 工具时，宿主可用同包 CLI 执行缺少的操作，新建会话后加载新版 Skill 和工具列表。仅选择路由或接入数据包不会启动后台研究。
+
+实现细节见[中文 Skill 说明](../skills/bookmark-research/references/zh/skill-guide.md)、[配置参考](../skills/bookmark-research/references/zh/settings-and-archive.md)和[Wiki／评测指南](wiki-quality.zh.md)；实际加载的是[英文执行 Skill](../skills/bookmark-research/SKILL.md)。

package/docs/validation-0.2.0.md

@@ -0,0 +1,126 @@
+# 0.2.0 验证记录与测试包
+
+核验日期：2026-09-10。这里区分本地行为测试、真实数据包验证、服务连通性和客户端加载；某一层成功不替代其他层的验证。
+
+## 需求与证据
+
+| 需求 | 实现与可核对证据 |
+| --- | --- |
+| 保持独立插件、版本 0.2.0 | `.codex-plugin/plugin.json`；导出器从同一 manifest 读取版本；原项目只作资料参考 |
+| 聚合 MCP | `src/provider_adapters.py`、`web_search.py`、`remote_mcp.py`；真实工具 schema fixture 与四个聚合相关测试模块 |
+| 三层检索路由 | `skills/bookmark-research/SKILL.md` 与 `references/deep-research.md`；官方出处与设计取舍见 [调研报告](research-0.2.0.md) |
+| 可继续的深度研究 | `src/research.py` 与 `tests/test_research.py`：规划、预算、检索、引用、冲突、修订、恢复与报告 |
+| 保留书签语境 | `bookmark_refs` → `context.json` → source 的书签实例关联；同 URL 在不同栏目保持独立身份 |
+| 有依据的结论 | 来源身份审阅、原句匹配、哈希校验、被拒绝来源拦截、撤回结论后重开问题 |
+| 学习 Qwen 安装结构 | [安装设计](install-design-0.2.0.md)：原生 CLI、install/update/verify、来源选择、固定版本与配置分离 |
+| 可下载和验证的包 | `scripts/build_zip.py` 生成普通 ZIP 与 `--include-tests` 验证包；确定性文件清单与 SHA-256 |
+| 详细查看调研 URL | [v0.2.0 历史来源清单](https://github.com/Browser-bookmark-hub/Bookmark-Research/blob/v0.2.0/docs/research-sources-0.2.0.json)：逐项读取、失败、私人链接处理和来源质量状态；新版 ZIP 不附带实例清单 |
+
+## 离线测试
+
+v0.2.0 发布快照测试：Python 3.9 / Codex CLI 0.153.4 环境中 **183 项全部通过，0 跳过**（6.010 秒）。Plugin validator、Skill quick validator、Python 编译检查与 `git diff --check` 通过。最终合成包验证完成两轮研究，输入 5 文件哈希未变。
+
+发布后的 `main` 补充一条命令安装和首次使用引导；2026-09-10 的完整测试为 **193 项全部通过，0 跳过**（15.876 秒）。新增检查覆盖本地 Git 仓库的管道安装、临时目录清理后继续使用、固定 ref、已有 personal 安装识别，以及实际配置读取、默认配置无需文件、已有设置保留、配置损坏提示和 stdout JSON 与 stderr 引导分离。
+
+在源码根目录或解压后的测试包根目录运行：
+
+```sh
+python3 -B -m unittest discover -s tests -v
+python3 scripts/verify_fixture.py --output /tmp/bookmark-research-validation
+```
+
+测试包包含虚构域名、合成书签和录制的公开工具 schema，不包含个人书签库、API Key 或预建索引。默认不向网页服务发请求；系统有 Codex CLI 时，安装测试使用临时独立 profile 实际执行原生命令。没有 Codex 时相关客户端测试会 skip，不能把 skip 当作已验证安装。
+
+`verify_fixture.py` 使用固定离线响应，验证如下可观察行为：
+
+- 独立读取 JSON 计数并与 SQLite 对照；永久副本不增加总书签数。
+- 同 URL 的不同实例、栏目、组、子文件夹范围均保留。
+- 导入前后源文件 SHA-256 相同；第二次同步未变更。
+- 完成两轮研究，保留缺口、实际引用、失败页面与研究来源映射。
+- 关闭并重开运行时后，相同操作编号复用结果，不再次执行检索。
+- 交付独立的 `context.json`、`report.md`、`sources.json` 与原响应／正文快照。
+
+测试返回 `network_calls:0`；研究计数中的 4 search calls / 3 fetch calls 是合成适配器调用，不能当作真实服务质量或费用测试。
+
+单元测试还覆盖预算被并发调用消耗、请求中断后保留预留、没有正文／引用原句不存在／正文被修改、来源被拒绝、结论撤回、冲突解决依据失效、未回答问题无法标记完成，以及 CLI 和 MCP 的输入边界。
+
+独立审查发现并修复了回执已写但状态未提交时的来源恢复、超过 2 MiB 的研究状态返回、MCP 错误标记、撤回结论被脚注渲染隐藏等问题。回归用例包括 45 条 claim／540 段长引用的完整分页与完成响应；审查者另用实际 Markdown 渲染器确认历史结论和跳转链接可见。
+
+另一位代理按 Skill 自主探索合成画布，没有读取预设验证流程或答案。它找到 Investigation 组的 4 个书签及后续连线，发现 scenario 缺少 update 页后诚实交付 incomplete。补齐合成资料后，评估又暴露 incomplete 报告缺少恢复入口；现通过 `research_record kind=resume` 在同一任务继续。原报告、状态和来源快照保留，补读后完成报告，fetch 预算从 2/4 增至 3/4，原 unknown_outcome 不退款。初次失败和修复后结果保存在本机独立验收目录，没有把首次运行改写成全程成功。
+
+恢复另经独立故障注入复核：连续两次恢复的 6 份快照逐字节一致，原报告的 9 个相对链接可解析；快照写入或状态保存失败时原产物保持完整，再次恢复不重复网络调用、不增加预算。
+
+## 用户提供的数据包
+
+实际在提供的示例画布上运行了：
+
+```sh
+python3 scripts/verify_fixture.py --output /tmp/bookmark-research-0.2.0-fixture-check \
+  --package '/absolute/path/to/书签画布-20260906'
+```
+
+| 指标 | 实际结果 |
+| --- | --- |
+| 书签实例 | 638 |
+| 不同原始 URL | 557 |
+| 文件夹 | 122 |
+| 栏目／画布节点／连线 | 5／7／3 |
+| 已检查源文件 | 7，含包内指南 |
+| 源文件变化 | 无；逐文件 SHA-256 一致 |
+| 第二次同步 | `changed_files:0` |
+
+计数使用原 JSON 的独立遍历与导入后的 SQLite 交叉核对。这里没有联网抓取这 557 个 URL。完整个人路径与逐文件哈希保存在本地验证输出，公开测试包只收录合成数据。
+
+另一个 AI 升级调研包的 223 个书签实例／207 个去重 URL，其研究覆盖与无法读取项单独列在 [v0.2.0 历史来源清单](https://github.com/Browser-bookmark-hub/Bookmark-Research/blob/v0.2.0/docs/research-sources-0.2.0.json)，不把这些实例数混作公司或独立证据数。
+
+## 真实服务验证
+
+运行命令需要显式打开网络验证：
+
+```sh
+python3 tests/live_provider_smoke.py --run-live --output /tmp/bookmark-provider-smoke.json
+```
+
+一次最多三次搜索与三次提取，测试进程清除服务密钥并使用临时设置。观察结果见 [provider-smoke-0.2.0.json](provider-smoke-0.2.0.json)：三家工具目录与提取成功；Parallel、Tavily 搜索成功；首次 Exa 搜索连接失败，另一次明确发起的 Exa 搜索成功。原始集合保留 `passed:false`，成功复验单列，未把自动重试或持续可用性当成已验证。
+
+这些检查未执行供应商托管 Deep Research、OAuth 或付费研究任务。Exa Agent、Parallel Task MCP、Tavily Research 的独立接口与认证依据见 [MCP 接入说明](mcp-aggregation-0.2.0.md)。本插件第三层是宿主模型使用可恢复研究工具执行的流程。
+
+## 真实研究流程
+
+使用工作区 0.2.0 的 `ResearchSessions`，由宿主模型实际完成“核验三层搜索与原生深度研究服务的边界”。任务 `r-e7090e9833194564` 保存在本机独立研究目录；网络步骤在不同进程中继续，读取已存来源、审核正文、记录原句后综合报告。
+
+| 观察项 | 实际结果 |
+| --- | --- |
+| 问题／搜索轮次 | 2／2 |
+| 已预留检索调用 | 4 search calls、3 fetch calls |
+| 实际来源快照 | 4，来自 3 个不同官方 URL |
+| 主张与修订 | 10 条，其中 1 条拆分后撤回；历史原句保留 |
+| 冲突 | 1 项，同 URL 摘录写 basic／fast；结论明确保留模式未知 |
+| 最终状态 | completed，报告、来源清单、操作回执与原文均存在 |
+
+这次不是脚本预置回答：第二轮根据第一轮定义补读接口，发现不同快照的模式差异后额外读取并记录冲突。Exa 和 Parallel 的真实 search/fetch 调用均成功。完成标准是回答两道接口边界问题；不能据此声称已经运行供应商原生研究任务、核实服务器默认模式或通过通用研究质量基准。
+
+实际摘录的 Search MCP 模式证据：Parallel 快照 SHA-256 `fbff308d1ca7bf7cdaa1fffc001dcb27cf2faa63a52423f2d6a1aa202d4bf925` 写 `Anonymous free-tier searches run in [fast mode] by default`（原 Markdown 带链接）；Exa 快照 SHA-256 `cc9ab5f5e83ca5e7186ce1701f8567e2bae9626c2737990a3fa56c13ec0f49e2` 写 `Search runs in basic mode`（原 Markdown 带代码标记）。保存的逐字原句和限制见该任务 `report.md`。
+
+调研来源清单另经复核：207 个原 URL 指纹、223 个实例映射、191 份正文／响应文件哈希和 16 处短引用全部匹配；原调研包 9 个文件哈希未变。覆盖不足和身份错配仍按原记录保留。
+
+## 安装、更新与包完整性
+
+安装测试使用 Codex CLI 0.153.4 和 Python 3.9：中文、空格、引号与字面 `$()` 路径；重复安装；同版本内容更新；版本更新；已存在索引和用户设置保留；来源冲突；安装缓存中的 doctor 与 MCP 初始化／工具发现。
+
+普通发行 ZIP 与测试包使用同一版本源码构建，并作为 [v0.2.0 Release](https://github.com/Browser-bookmark-hub/Bookmark-Research/releases/tag/v0.2.0) 附件分发。Release 的 `SHA256SUMS` 校验整个 ZIP，各包的 `MANIFEST.sha256` 校验内部文件。复现构建：
+
+```sh
+python3 scripts/build_zip.py --output dist/bookmark-research-0.2.0.zip
+python3 scripts/build_zip.py --include-tests --output dist/bookmark-research-test-pack-0.2.0.zip
+```
+
+输出路径应不存在；需要重新构建时用新路径或先自行移动旧产物。解压后核对 `MANIFEST.sha256`，再运行验证：
+
+```sh
+shasum -a 256 -c MANIFEST.sha256
+python3 src/cli.py doctor
+python3 scripts/install.py verify
+```
+
+最后一条要求已在该客户端安装插件。测试包还可运行 `verify_fixture.py`；普通插件包不含测试资料。导出测试覆盖 Codex、Claude Code、Pi、DSH 与 Agent Plugins 五种目录结构，以及从任意工作目录启动。Claude Code、Pi、DSH 的客户端内实际加载仍未验证，不能仅凭导出成功宣称原生客户端已工作。

package/docs/validation-0.4.0.md

@@ -0,0 +1,154 @@
+# Bookmark Research 0.4.0 validation
+
+Version 0.4.0 combines whole-package research, host workflows, optional professional research, Wiki/evaluation, source lifecycle management and English/Chinese instructions. Earlier unpublished 0.3.0 work is included in this version; original validation measurements retain their observed versions and limits. A passing code test, a recognized plugin manifest and a successful model-driven workflow are separate observations.
+
+Manual export folders can disappear, and a later export can have a different filename or location. Stable source identity is separate from input paths, snapshots are recoverable, and explicitly persistent directories can be monitored. Directory, ZIP and single-card input use the existing Bookmark Canvas export protocol.
+
+## Implemented behavior
+
+- Directory, wrapped/unwrapped ZIP, and permanent/temporary section JSON input. Unknown fields, exact raw protocol bytes, bookmark instances and relationships are retained. Ambiguous multi-package archives and loose cards mixed into a package fail visibly instead of reducing the imported scope.
+- `snapshot` and `live` source modes, persistent path aliases, immutable content versions and `source_history`. The current exported protocol has no reliable global canvas ID; moved exports of the same canvas reuse an explicit `source_id`, and distinct canvases are not merged by titles or URLs.
+- Separate `partial` and `complete` deletion rules. Single cards are partial inputs. Full mirrors remove absent files while preserving stable item identity through renames. Empty directories, Git write locks and invalid/inconsistent files retain the last valid index.
+- A process-scoped Python polling monitor, separate SQLite connections, consistent query reads, debounced updates, a longer deletion grace period, visible pending/error states and reconnect/query checks. Monitoring stops when the MCP process stops; standalone CLI users can run `watch`.
+- Full managed snapshots after partial imports, including recorded relationships needed to recover an older layout after a card-only rename. Recovery validates hashes. Legacy JSON recovered only from SQLite is explicitly labeled as lacking original whitespace.
+- Research status compares frozen scope with current indexed input. Wiki reads/lint flag changed bookmark input for review. No automatic webpage fetch, LLM research, evidence rewrite or Wiki publication is performed.
+- Installation discovery probes use an isolated empty data directory, so verifying an installation cannot start monitoring the user's sources. First-use instructions describe the supported input forms and continuous-directory mode.
+
+## Task-driven user simulations (2026-09-14)
+
+Three delegated agents first followed the Skill on real tasks: a single GitHub bookmark check, an Exa/Parallel topic comparison within the supplied Canvas, and a resumable comparison of three ordinary bookmarks. The baseline was frozen at `baf1611`. Each case used isolated data, recorded CLI calls and actual retrieved text. For the revised Skill, the agents rotated to different cases without reading those cases' baseline answers. A fresh fourth agent could not be created because of the host thread limit. These are observed usability cases, not a controlled quality or speed benchmark.
+
+| User need | Baseline friction | Revised behavior observed |
+| --- | --- | --- |
+| Check one known URL | Main Skill plus four references and CLI help before reading; repeated provider text filled the response. | Main Skill alone was sufficient. Two fetches covered the requested facts: the default extract stopped before authentication details, so the agent requested a larger extract. No import, research session or Wiki. |
+| Compare a Canvas topic | One invalid CLI spelling; ten CLI calls, two fetch operations; seven related bookmarks retained. | Ten valid CLI calls, three fetch operations; all seven original URLs attempted, six had usable text. Official-document follow-up answered the topic while retaining the Smithery navigation-only gap. Fourteen substantive table claims passed independent semantic review. No whole-package investigation or Wiki. |
+| Research three ordinary bookmarks | Native inventory was empty; the agent maintained a separate manual list and created an unrequested Wiki. | `research_start.urls` froze all three originals without a Canvas import. Original-source review was 3/3; 17 substantive claims passed independent review of the draft and evidence. The saved report remained `incomplete` with 3/4 questions answered and Reader scanned-PDF OCR evidence still missing. A resume input was supplied; no Wiki was created. |
+
+The main English Skill decreased from **2,163 to 1,254 whitespace-delimited words**. References load by task; professional research remains optional support after the host's first pass. Wiki creation requires a request to retain knowledge there. Ordinary URL lists preserve duplicate positions and unsupported/local entries, and use the same frozen-scope completion checks as Canvas input. During the deep case, an inventory review linked to a supplementary-only claim was atomically rejected, then corrected to cite the original source.
+
+Default `fetch_web` / `fetch-web` now returns one selected text per URL, retaining every attempt's status and archive path. Full provider envelopes remain available through archives or `raw:true` / `--raw`. Reformatting the *same* three saved live responses with identical JSON serialization produced:
+
+| Response | Original JSON characters | Compact characters | Reduction |
+| --- | ---: | ---: | ---: |
+| Single GitHub README | 97,643 | 29,313 | 70.0% |
+| Canvas first fetch | 233,623 | 61,814 | 73.5% |
+| Canvas follow-up fetch | 387,546 | 102,871 | 73.5% |
+
+All eight selected texts matched their archived text and SHA-256 hashes. This measures response length, not latency, token use or research accuracy. Live provider availability and extracted contents differed between runs, and follow-up calls sometimes increased.
+
+Independent baseline review caught unsupported wording introduced by the final summary despite passing quotation/hash checks: treating page output as proof of PDF completeness, omitting same-page counterevidence, and presenting one OCR option as required. The revised instructions require review of the final synthesized answer as well as claims, with corrections before completion. Semantic reliability still depends on the readers and reviewers; the runtime enforces record consistency and scope, not factual truth.
+
+The local Codex source was backed up, re-exported and reinstalled at base version 0.4.0 with an installation-only cachebuster. All 61 source/cache files matched. A fresh stdio process exposed **35 tools**, accepted `research_start.urls` and `fetch_web.raw`, retained two distinct URLs/three instances with duplicate positions, and rejected unread input marked `completed`. The offline check saved an `incomplete` report without creating an index or making network calls. A new conversation loads the updated Skill and catalog.
+
+The full supplied Canvas was also rechecked offline: 207 distinct URLs, 223 instances, all three inventory pages, and all nine original files unchanged. This does not establish review of all 207 webpages. No professional research job, PDF conversion benchmark or new four-host end-to-end acceptance was performed. Raw case artifacts remain outside the repository and distribution.
+
+## Whole-package scope
+
+The supplied package was read offline through the same index and research inventory APIs. Independent counts and the frozen inventory agree: **223 bookmark instances, 207 distinct original URL strings, 22 folders, 4 sections, 5 canvas nodes and 4 group memberships**. This package has no directed edges; the synthetic fixture separately exercises directed edges and copy cards.
+
+All three inventory pages were consumed and compared with the complete indexed inventory. Every original instance remains mapped to its URL and context. All nine files in the original directory had identical SHA-256 hashes before and after validation. Personal source files, paths and inventory contents are not included in the distribution.
+
+For 207 URLs, the default fetch plan reports a 26-call lower bound at eight URLs per call and a 38-call configured budget. Explicit user budgets remain unchanged. A budget below the first-pass capacity reports the shortfall without removing sources. This verifies data handling and capacity accounting; it does not mean all 207 webpages were fetched or reviewed during this offline check.
+
+Both professional-service payloads were also prepared offline from this actual frozen input. Each retains all 207 inventory IDs: 206 HTTP(S) URLs are supplied as retrievable sources and one `file://` input remains explicitly local-only. Nothing was submitted. A regression checks 207 URLs/223 instances, an explicit 150-ID subproblem and oversized private context without reducing source scope or uploading notes.
+
+## Automated verification
+
+The latest integration tree passed **376 tests** in 39.178 seconds. New regressions cover ordinary URL inventories, duplicate instances, invalid/mixed inputs, pagination, partial-completion rejection, compact-response fidelity and MCP/CLI entry points. Plugin/Skill validators, local document-path checks and the installed-runtime check passed. The earlier consolidated checkpoint passed 319 tests in 34.511 seconds, including 63 JavaScript host workflow contract scenarios. The plugin manifest, local MCP server identity and remote MCP client identity all use 0.4.0; exporters derive their package versions from the manifest.
+
+The source-lifecycle checkpoint passed **313 tests** on Python 3.9.6 / SQLite 3.43.2 with FTS5. This includes 32 source-lifecycle cases and 20 installer cases. The source-lifecycle module was also run independently after the final consistency adjustment, and the installer module was run independently to verify it does not depend on test discovery order.
+
+```sh
+python3 -B -m unittest discover -s tests -v
+python3 -B -m unittest discover -s tests -p 'test_source_manager.py'
+python3 -B -m unittest discover -s tests -p 'test_install.py'
+```
+
+Cases cover persistent reopen, changed paths, duplicate source ambiguity, single-card merge, raw-byte recovery, renamed layout bindings, corrupt snapshots, ZIP traversal/symlinks/duplicates/multiple roots/size bounds, full-mirror deletion, partial retention, temporary directory loss, Git locks, changes during import, idle background updates, MCP reconnect, frozen research scope, Wiki review flags, and CLI/MCP behavior. The host workflow, research service and coverage regressions remain included.
+
+Skill/plugin validation and `git diff --check` passed. These checks establish implementation behavior and packaging validity, not semantic research quality.
+
+The tests cover full inventories, explicit subsets, repeated URLs, original context, concurrent evidence writes, legacy state, pagination, rejected evidence, partial review, external reports, unknown service outcomes and completion gates. Reviewing only part of the input cannot complete a whole-package investigation. Accounted items, accepted text, substantive review and answered questions have separate denominators.
+
+MCP/CLI integration tests establish a complete research session, cross interface boundaries when reading and importing, reject partial completion, and publish/search/lint a Wiki page from accepted evidence. They also check routing, settings and evaluation parity. The evaluation fixture is synthetic; its scores make no provider-quality claim. The earlier host/research checkpoint passed 280 tests, including 19 service lifecycle tests and 19 installer tests; its host wrapper exercised 36 JavaScript scenarios. These are checkpoint measurements, not the final combined test count.
+
+```sh
+python3 -B -m unittest discover -s tests -v
+python3 -B scripts/verify_fixture.py --output /absolute/external/new-directory
+python3 -B scripts/verify_quality.py
+python3 -B -m unittest discover -s tests -p 'test_host_workflows.py'
+```
+
+Append `--package /absolute/path/to/package` to the fixture verifier for read-only validation of a real package. Use a new output directory outside source packages and plugin caches. The fixture deliberately retains blocked sources and produces an incomplete report, proving that complete accounting is not complete research.
+
+## English and Chinese instructions
+
+The execution Skill and all 11 method references are maintained in English, with 12 complete Chinese reading copies. There is one discoverable Skill. Starter prompts and onboarding are bilingual; Codex delegation and shared Claude/Pi/DSH workflows carry the requested output language. The [instruction index](instructions.en.md) and [prompt reference](prompt-reference.en.md) provide both reading languages.
+
+The bilingual update passed 56 relevant Python tests, including 63 JavaScript workflow contract scenarios. Tests cover language propagation, unchanged scope and status fields, all five export formats, one Skill entry point, and installer/bootstrap behavior. Local links and ZIP integrity manifests were also checked. Installer `--lang` selects onboarding language only; it does not persist a research-language preference.
+
+An independent agent followed the English Skill with an offline synthetic package: 12 successful CLI commands retained all seven bookmark instances and six unique URLs across four section cards. A copy anchor did not duplicate its underlying bookmarks. The brief, gaps and answer were in English; all five input files were unchanged. With zero retrieval budgets, no fetched pages or claims, the saved research correctly remained incomplete. This is an instruction-use check, not a live webpage or professional-service evaluation.
+
+The bilingual local update was reinstalled and enabled at base version 0.4.0. All shipped files matched the staged export apart from the installation cachebuster. Its actual stdio runtime exposed 35 MCP tools, with SQLite FTS5 available; isolated discovery created no database. Technical diagnostics, original evidence and historical validation records retain their original language.
+
+## Real input and installed runtime
+
+Read-only verification against real user exports produced:
+
+| Input | Bookmark instances | Distinct original URLs | Protocol files |
+| --- | ---: | ---: | ---: |
+| Research directory export | 223 | 207 | 5 |
+| Sample ZIP export | 638 | 557 | 6 |
+
+Changing the research export's directory and reimporting with the same source ID produced zero changed files and zero inserted/updated items. Importing one of its cards preserved all 223 instances. Removing the disposable export copies still allowed queries. Restoring the managed snapshot into a new database reproduced the same counts and input fingerprint. A modification to a separate live copy was indexed without a query. Hashes confirmed that the original directory files and ZIP were unchanged.
+
+The existing personal Codex plugin was updated to 0.4.0 after preserving its previous source directory and taking a consistent SQLite backup. The installed shared runtime matched the repository bytes. Its actual stdio entry exposed **35 tools**, accepted the new source parameters, imported the complete 223-instance package, queried its saved snapshot and listed its history. Runtime/FTS5 checks passed, with no extra output on the MCP stream.
+
+The two previously registered manual exports were migrated to snapshot mode. Their source IDs, item primary keys, item revisions, counts and indexed input fingerprints remained unchanged; neither migration inserted, updated or deleted any bookmark item. A new Codex thread loads the updated Skill and tool catalog.
+
+## Host and provider boundaries
+
+The [host validation evidence](host-validation.json) retains the actual client versions, tool counts and failures observed at each earlier checkpoint.
+
+| Route | Established evidence | Separate acceptance still required |
+| --- | --- | --- |
+| Codex | Native delegation recipe; preparation script reads all 207 IDs through real stdio MCP; delegated Skill acceptance on a fresh synthetic package produced a correctly incomplete report and two local-evidence Wiki pages | Live webpage research with multiple reader groups and independent child verification |
+| Claude Code | 2.1.247 strict plugin validation; live launch loaded the workflow command and connected all 34 MCP tools | Existing configured model endpoint returned 429 / Service Unavailable before any child ran; workflow execution and recovery remain unverified |
+| Pi | Project registrar, stdio bridge and isolated extension scenarios | Pi and the two required extensions are not installed in the validation environment |
+| DSH | Native call-object generator and isolated workflow scenarios | A configured DSH profile, engine, service and tool are not installed in the validation environment |
+| OpenAI / Parallel research APIs | Thin clients and lifecycle/contract tests, separate credentials, secondary report import | No research API credentials were configured; no paid task was submitted |
+
+At the earlier prerelease checkpoint, recorded as 0.3.0 in the original evidence, the personal Codex installation was updated with Codex CLI 0.153.4 after a byte-verified backup. The installed cache exposed all 34 tools over its actual stdio launch command; the CLI FTS5 check and read-only routing/service-description calls passed. All 36 non-manifest export files, including the three Codex host assets, matched the installed bytes. The installation-only cachebuster is separate from the repository version. A new Codex thread is required to load the updated Skill and tools.
+
+Workflow scenarios cover full pagination, independent verifier failures, gap-only follow-up, partial/fatal errors and false completion. The host remains responsible for checking authoritative saved state and artifacts after a workflow returns. These scenarios do not simulate a provider-quality benchmark or prove unavailable clients were run.
+
+An independently delegated Codex agent followed the revised Skill with only the original synthetic canvas input, without reading expected answers or changing code. It preserved seven instances and six URLs, read five local JSON/canvas files, saved six metadata claims, wrote two linked Wiki pages and checked saved state, coverage and hashes. All six URLs use the reserved `.test` domain, so they remained explicitly blocked: accounting was 6/6, original-page text and substantive review were both 0/6, and the investigation remained incomplete. No network or professional research API was called. An additional verifier could not start because the host thread limit was reached; the agent disclosed its own second review rather than claiming independent child verification. The acceptance exposed a report-label issue for local documents, now corrected to show their evidence and content kinds without changing coverage.
+
+The live Claude attempt used an isolated two-URL/three-instance synthetic package with pre-imported text and zero retrieval budget. The first isolated launch did not load user authentication; the second used the existing user model configuration with hooks disabled for that invocation. Both started zero subagents and reported zero usage cost. No global client settings changed. Tool discovery succeeded, but the research run did not; the successful manifest test is not presented as live execution acceptance.
+
+Actual search observations in this implementation session: Exa returned usable search results; Tavily reported `monthly_cap_reached_bonus_eligible` in a structured error envelope despite MCP `isError:false`; Parallel had an SSL EOF transport failure. Error handling now classifies these as provider failures instead of empty successful search. No bonus signup or payment was attempted, and these observations are not permanent claims about service availability.
+
+## Wiki and quality claims
+
+Wiki verification checks accepted, active source claims, immutable revisions, optimistic updates, provenance, changed hashes, crosslinks, Chinese body search and stale-evidence exclusion. Lint reports semantic support as unscored. A semantic review declaration is recorded, not mechanically proven.
+
+The quality verifier uses six explicitly synthetic outputs to exercise answer F1, citation annotations, report rubric scores, four coverage metrics, cost/latency and repeat variance. Unknown labels and measurements remain null. It does not establish quality improvements over ordinary host research or professional services. Real comparisons require actual matched tasks, recorded conditions and independent judgments; see [Wiki and evaluation](wiki-quality.md).
+
+## Limits and distribution
+
+The monitor uses periodic file checks, not native OS file events or a system daemon. Defaults are a 1-second poll, 2-second debounce and 5-second deletion grace; these are scheduling settings, not guaranteed update latency. It does not perform Git authentication or synchronization. Stable-file checks and Git locks cannot make an external exporter atomic. A complete mirror must have valid, agreeing references before its update can be committed.
+
+Snapshots are retained without automatic garbage collection. Local index freshness does not establish that remote webpages are current. Professional API access and full model-driven runs in all four hosts have not been newly validated by this version; the host and provider boundaries above retain those observations.
+
+```sh
+python3 scripts/build_zip.py --output dist/bookmark-research-0.4.0.zip
+python3 scripts/build_zip.py --include-tests --output dist/bookmark-research-test-pack-0.4.0.zip
+```
+
+Each distribution has an internal `MANIFEST.sha256`. User exports, source snapshots, databases, research artifacts, credentials and local validation directories are excluded. GitHub publication is separate from the completed local installation and validation.
+
+Each ZIP has a `MANIFEST.sha256`. Exports carry the shared runtime, Skill and the host-specific adapter; they exclude source packages, user settings, databases and credentials. Git installation remains independent of GitHub Releases. Host loading and optional service authentication follow their real interfaces, not the archive format.
+
+The historical 0.2.0 source-audit JSON contains original bookmark instance mappings, so it is excluded from both 0.4.0 ZIPs. Historical method documents link to that release's repository snapshot instead. The production package contains no synthetic test fixtures; the explicitly requested developer test pack adds only curated tests and synthetic fixtures.
+
+Installer checks require the Codex delegation recipe, preparation script and shared stdio bridge. Local installation and updates compare these assets with the selected source; standalone verification also rejects missing or symlinked host assets. Native Codex tests verify that changed assets reach the updated cache. These are installation integrity checks, not model-driven workflow acceptance.

package/docs/wiki-quality.md

@@ -0,0 +1,105 @@
+# Reviewed Wiki and annotated research evaluation
+
+**English** · [中文](wiki-quality.zh.md)
+
+The host writes and reviews knowledge pages. The plugin stores their evidence and revisions, checks their integrity, and searches the organized prose. Evaluation consumes explicitly annotated outputs; it does not call a model or infer semantic support from matching words.
+
+## Wiki API
+
+`WikiStore(directory=None, settings=None, research_sessions=None)` uses an explicit directory first, then `settings.wiki.directory` (default: the data directory's `wiki/`). Plugin directories and canvas package directories are rejected. Research evidence remains in its research session, outside the original input package.
+
+```python
+from wiki import WikiStore
+
+wiki = WikiStore(research_sessions=sessions)
+page = {
+    "title": "Reusable sessions",
+    "kind": "topic",  # or entity
+    "sections": [{
+        "heading": "Supported behavior",
+        "text": "The host's reviewed synthesis, including relevant limits.",
+        "claims": [{"research_id": research_id, "claim_id": "c1"}],
+    }],
+    "links": [],
+    "review": {
+        "method": "model",  # or human
+        "reviewer": "The actual model/version or human reviewer",
+        "note": "Explain the scope and support checks actually performed.",
+    },
+}
+written = wiki.write("reusable-sessions", page, "Organize reviewed findings", expected_revision=0)
+current = wiki.get("reusable-sessions")
+wiki.write("reusable-sessions", revised_page, "Explain the new limitation", expected_revision=current["revision"])
+```
+
+Every section requires explicit research claim references. A reference is accepted only when its claim is active, every cited source is `accepted`, the quote occurs in its saved body, and the body hash matches. The Wiki snapshot retains the research ID, claim ID, source ID, original-inventory IDs, bookmark instance references, exact quotes, reviews, retrieval metadata and artifact hashes. Up to 24 distinct claims may support one page; oversized topics must be split.
+
+The review declaration addresses whether the prose is supported, uses the correct entities and scope, preserves uncertainty, and acknowledges contrary evidence. The program records this declaration and its rubric. It cannot verify that the named human or model performed the judgment, and a successful write is not an independent semantic endorsement.
+
+An imported external report stays secondary evidence. Its provenance and quoted report text are retained; its bibliography is not expanded into pages supposedly read or original sources supposedly covered.
+
+### Revisions, links and inspection
+
+Each update writes an immutable JSON revision and a rendered Markdown revision under `revisions/<page_id>/`, then atomically updates `index.json`. The index is the publication point. A failed publication can leave unreferenced files, while the preceding revision stays readable. `expected_revision=0` creates a page; an update must name the revision the caller read. After an interrupted write, inspect the current revision before retrying.
+
+Cross-links have the form `{"page_id": "existing-page", "relation": "The host's explanation"}`. The target must already exist. Rendered links point to the target revision present when the page was written, preserving the historical meaning. A later target update produces a lint warning; the host can explicitly revise the relationship. Prior page revisions remain available with `get(page_id, revision=N)`.
+
+| Method | Result |
+| --- | --- |
+| `write(page_id, page, change_note, expected_revision=0)` | Revision metadata and absolute artifact paths |
+| `get(page_id, revision=None)` | Authored page, evidence snapshot, update record and current provenance validation |
+| `list(offset=0, limit=20)` | Paginated current page metadata |
+| `lint(page_id=None)` | Source review, active-claim, citation, hash, link and history issues |
+| `search(query, offset=0, limit=10)` | Matching current Wiki prose, with stale pages excluded |
+
+Lint detects later rejection, withdrawal, source/claim changes and altered or missing artifacts. Historical text is preserved and identified as stale. Lint explicitly returns `semantic_support: "not_scored"`.
+
+Search checks titles, section headings and authored section text. It uses case-folded literal terms, including Chinese substrings, and simple occurrence ordering. Saving a provider extract alone does not make that extract searchable as organized Wiki knowledge. This is not embedding retrieval or semantic reranking; neither is selected without actual question-answering evaluation.
+
+## Quality evaluation input
+
+`evaluation.evaluate(suite, runs, judgments=None)` is a pure computation over supplied data. The complete runnable input is [research-quality.json](https://github.com/Browser-bookmark-hub/Bookmark-Research/blob/main/tests/fixtures/research-quality.json), also included at `tests/fixtures/research-quality.json` in the source checkout and verification pack.
+
+- A **suite** identifies tasks, their original source/question IDs, gold answers and a versioned rubric. `gold_answers: null` means no answer-set benchmark is available. The rubric defines answer identity matching, citation support and report dimensions on a 0–4 scale.
+- A **run** records its task, system, mode, repeat, final report, atomic answers/claims/citations, coverage IDs and measurements. It also records model, tools, budget, input snapshot/version, time window and harness version. Citations are unique claim/source pairs.
+- A **judgment** identifies the rubric version and the actual human or model evaluator. It supplies predicted-answer → gold-ID matches, supported/unsupported/uncertain citation labels with reasons, and report rubric scores with reasons. Fixture judgments must identify themselves as synthetic.
+
+Answer matching is an annotation task. Evaluators must read the actual prediction, establish identity and scope, and use `gold_id: null` for an incorrect entity. The scorer does not guess aliases or use regex matches as a semantic judge. Multiple predictions matched to one gold entity count once for set precision/recall; unmatched predictions count as false positives. Duplicate matches are also reported.
+
+Each citation judgment concerns the complete claim and the actual source context, including identity, dates and scope. `supported` means the text supports the claim. `unsupported` and `uncertain` both remain in the full accuracy denominator. Exact-quote matching alone is not a support judgment.
+
+### Metrics and missing information
+
+| Output | Meaning and limit |
+| --- | --- |
+| Answer precision, recall, F1 | Uses gold data and complete answer-match annotations; otherwise null |
+| Semantic citation accuracy | Supported / all citations when all are judged; partial-label statistics are shown separately |
+| Semantic claim support | Claims with a supported citation / all claims, once all citations are judged |
+| Citation presence | Claims with any citation / all claims; this is a structural measurement |
+| Report quality | Mean supplied dimension scores normalized from 0–4, only when every rubric dimension is scored |
+| Accounted/text/reviewed coverage | Separate rates against the original source set; failed sources remain in the denominator |
+| Question coverage | Recorded answered question IDs / original question IDs; this does not judge answer truth |
+| Cost and latency | Supplied USD/seconds with observed, reported or synthetic measurement basis; unknown values stay null |
+| Repeat variation | Count, mean, sample variance, sample standard deviation and range under the same declared conditions |
+
+Coverage must satisfy `reviewed ⊆ text ⊆ accounted ⊆ original scope`. These reported IDs still need the research evidence audit: an input label alone does not prove that a source was read. External tool and model usage unavailable to the plugin must remain unknown.
+
+Groups compare the same task. Differences in model, tools, budget, input version, time window, harness, evaluator or measurement basis are reported. Differing conditions form separate aggregates. Unknown budgets prevent a controlled-comparison declaration. A single observation has no variance estimate; missing costs are not averaged as zero. Failed and incomplete outputs remain in the supplied run set.
+
+The output pins suite, run and judgment hashes, retains evaluator/rubric identity and includes null-score diagnostics. It does not produce a provider ranking, a causal claim, a RACE/FACT score or a public leaderboard score. Real claims about improved research quality require real task runs and independently defensible annotations.
+
+## Reproduce the offline demonstration
+
+These commands need a source checkout or verification pack. Runtime exports contain the guides, but not the test fixtures or verification scripts.
+
+```sh
+python3 -B scripts/verify_quality.py
+python3 -B -m unittest discover -s tests -p 'test_wiki.py'
+python3 -B -m unittest discover -s tests -p 'test_evaluation.py'
+```
+
+The verification script writes `inputs.json`, `evaluation.json` and a readable `evaluation.md` under a new external data-directory run. `--output /absolute/external/directory` selects another artifact directory. `--fixture /path/input.json` accepts a wrapper containing `suite`, `runs`, `judgments` and optional known fixture expectations.
+
+The included example contains four invented source records, two gold entities and six hand-authored outputs labeled as direct, workflow and professional modes. One source is unavailable, some outputs miss or invent an answer, and one citation has uncertain scope. Costs and durations are synthetic values; one cost is deliberately unknown. These cases exercise denominators, semantic-label completeness, repeat variance and missing measurements. **They do not measure those three execution modes or any real host/model/provider.**
+
+Wiki tests additionally verify immutable history, failed index publication, cross-links, Chinese prose search, source rejection/withdrawal, secondary-report provenance and source/package boundaries. Live host research, real professional-service comparisons and independent semantic judging remain separate validation work.