bookmark-research 0.5.0-beta.7 → 0.5.0-beta.8

package/.agents/plugins/marketplace.json

@@ -9,7 +9,7 @@
       "source": {
         "source": "npm",
         "package": "bookmark-research",
-        "version": "0.5.0-beta.7",
+        "version": "0.5.0-beta.8",
         "registry": "https://registry.npmjs.org"
       },
       "policy": {

package/.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "bookmark-research",
-  "version": "0.5.0-beta.7",
+  "version": "0.5.0-beta.8",
   "description": "Check bookmark URLs or Canvas packages, compare sources with host-led research, and retain verifiable reports.",
   "author": {
     "name": "kwenxu"

package/README.md

@@ -22,13 +22,13 @@ Works in **Codex, Claude Code, Pi and DSH (DeepSeek Harness)**.
 
 ## Install
 
-**Current beta (`0.5.0-beta.7`)**, interactive on macOS, Linux and Windows:
+**Current beta (`0.5.0-beta.8`)**, interactive on macOS, Linux and Windows:
 
 ```sh
-npx bookmark-research@0.5.0-beta.7 install --source npm:bookmark-research@0.5.0-beta.7
+npx bookmark-research@0.5.0-beta.8 install --source npm:bookmark-research@0.5.0-beta.8
 ```
 
-This pins both the installer and the content. The `@latest` examples below follow the default channel, which remains `0.5.0-beta.3`; `@beta` currently points to `0.5.0-beta.7`. Changing only the npx selector does not change the installer's default content source.
+This pins both the installer and the content. The `@latest` examples below follow the default channel, which remains `0.5.0-beta.3`; `@beta` currently points to `0.5.0-beta.8`. Changing only the npx selector does not change the installer's default content source.
 
 **Default-channel interactive installation:**
 

package/README.zh.md

@@ -22,13 +22,13 @@
 
 ## 安装
 
-**当前测试版（`0.5.0-beta.7`）**，支持 macOS、Linux、Windows 交互安装：
+**当前测试版（`0.5.0-beta.8`）**，支持 macOS、Linux、Windows 交互安装：
 
 ```sh
-npx bookmark-research@0.5.0-beta.7 install --source npm:bookmark-research@0.5.0-beta.7
+npx bookmark-research@0.5.0-beta.8 install --source npm:bookmark-research@0.5.0-beta.8
 ```
 
-这条命令同时固定安装器和插件内容。下面的 `@latest` 示例使用默认渠道，目前仍为 `0.5.0-beta.3`；`@beta` 当前指向 `0.5.0-beta.7`。只改 npx 的版本选择，不会改变安装器默认下载的内容来源。
+这条命令同时固定安装器和插件内容。下面的 `@latest` 示例使用默认渠道，目前仍为 `0.5.0-beta.3`；`@beta` 当前指向 `0.5.0-beta.8`。只改 npx 的版本选择，不会改变安装器默认下载的内容来源。
 
 **默认渠道的交互安装：**
 

package/docs/diff-review-2026-10-09.md

@@ -0,0 +1,36 @@
+# Diff 审查与修正记录（2026-10-09）
+
+本轮审查从 `a4b7630` 之后的工作树 diff 出发，通过 CodeGraph 定位实现与调用链，交叉检查 DSH 分发、检索、研究并发和 Wiki 证据生命周期。原始改动已单独保存为 `264d910`；其发布文件与 npm `0.5.0-beta.7` 的 120 个文件逐字节一致。
+
+DSH 修复的方向正确：发布包具有入口与 `dsh.bundle.patch` 声明，路径从安装位置解析，ESM 范围限制在 DSH 目录，环境变量按明确列表转发。用户已确认当前 DSH 跑通，旧评估报告中的失效状态属于历史记录。本轮没有重新安装或修改用户的 DSH profile。
+
+原始 diff 的 561 项测试通过，但交叉审查发现了并发和兼容边界，特别是来源数据保护与证据复核。以下问题均有隔离复现，并已在后续源码提交中修正。
+
+| 优先级 | 原始 diff 的问题 | 本轮修正 |
+| --- | --- | --- |
+| P1 | 研究取消或生成 incomplete 报告后，慢请求仍提交结果，导致最终报告与状态文件不一致；取消后还可能追加 fallback 请求。 | 提交与 fallback 预算预留都重新检查研究为 active、操作为 pending。已关闭或确认中断的操作拒绝迟到结果。 |
+| P1 | merge 在事务外判断版本相同，同步可在检查后写入独有内容，再被 merge 删除。两个未版本化的 legacy 来源也会因 `None == None` 被误合并。 | 在写事务内重新读取并校验双方有效内容版本；legacy 来源须先同步。 |
+| P1 | remove/merge 提交数据库后删除原目录，可能删掉另一连接刚以相同 source ID 重新导入的快照。 | 提交前将旧快照改名为唯一隔离目录，提交后只清理该目录；异常回滚恢复旧版本且不覆盖并发新版本。 |
+| P1 | 已发布证据的响应制品被修改后，`wiki acknowledge` 可重新计算 hash，把 `invalid_evidence` 清成 current。 | 持研究锁校验上一修订的证据与链接；存在 error 时拒绝复核，保留原校验基线。 |
+| P2 | 并发导入最终分配的 source ID 与提前写出的 manifest ID 不一致。 | 在提交锁内分配最终 ID 后生成 manifest，回放保留该 ID。 |
+| P2 | 删除引用来源后，freshness 为 unknown，但 Wiki 仍显示 current。 | 不可解析的来源产生 `source_input_unavailable`，页面进入 needs_review。 |
+| P2 | 旧版 DSH 包缺少新增的嵌套 `package.json`，无法进入承诺的旧包导出兼容分支。 | 仅在文件缺失时生成 ESM manifest；符号链接和错误文件类型仍拒绝。 |
+| P2 | Codex、Claude、Agent Plugins 导出没有根 `package.json`，握手版本退成 `0+unknown`。 | 读取各分发格式自己的 manifest；存在但损坏的优先 manifest 不被其他文件掩盖。 |
+| P2 | 含 `%`、`_`、反斜杠的画布线索在 SQL 命中后又被丢弃。 | SQL 保留转义，结果归属使用原始查询文本。 |
+| P2 | doctor 泄漏探测文件描述符、可能删除预存同名文件，并把普通文件阻断的目录路径报为可写。 | 使用自动关闭的随机临时文件；遇到非目录路径组件立即报告不可写。 |
+
+DSH 的重装提示也已改为明确指定 profile 和版本的原生命令，避免默认 `latest` 指向旧发布。Wiki 复核记录保存本次提交的说明。
+
+修正快照清理后，第二轮交叉复核发现了“目录改名后、数据库提交前强制终止进程”的恢复窗口。已补启动恢复：在写事务内，根据数据库仍引用的版本及 manifest/hash 校验恢复目录，保留已有新版本，不复活已删除来源。真实子进程强制退出及后续 partial 导入均纳入验证。
+
+验证结果：
+
+- 最终本地全量测试：**587 项，0 失败，3 项按条件跳过**，Python 3.9.6 / SQLite 3.43.2。跳过项为原生 Pi/DSH 安装、显式启用的 Codex/npm 集成和全宿主 npm 下载测试；不将其计作本轮宿主安装通过。
+- 独立下载真实 npm beta.5，使用当前导出器生成 DSH 包，删除源副本并搬到含中文、空格和引号的路径后，Skill 和实际 MCP 调用通过。
+- 当前五种导出同样在删除源副本、搬迁后完成真实 MCP initialize，全部报告 beta.7；DSH 解释器、参数和环境转发契约通过。
+- 用户画布样例验收：638 个书签实例、557 个唯一 URL、122 个文件夹、5 个栏目；完整库存分页保留全部 ID，二次同步识别未修改内容，原文件 hash 未变。测试未调用网络研究服务。
+- 合成研究验收正确阻止未读完来源时宣称研究完成。已发布 npm 包的 CLI help、doctor、status 冒烟通过。
+
+实现仍有明确边界：取消后已经返回的响应可能留下未引用制品，但不能再进入已结束研究的状态或报告。启动恢复只处理数据库仍引用且校验通过的快照；未引用的 `.removed-*` 残留保守保留，不自动回收。普通异常回滚、并发重新导入、回滚后出现新版本，以及提交前/后强制退出均有回归覆盖。
+
+这些审查修正属于 `264d910` 之后的源码提交，**不包含在已经发布的 npm beta.7 中**。本轮只提交并推送 Git，没有重新发布 npm，也没有移动 dist-tag。复现所用本机报告、索引和私人样例未纳入提交。

package/docs/distribution.md

@@ -19,7 +19,9 @@
 
 发布顺序：修改源码版本并完成 CI，发布已经验证的 npm 包，确认 registry 可取得该版本，再更新 GitHub marketplace 的固定版本。未发布源码不能通过 npm 入口安装；需要先用源码入口测试。
 
-测试版使用 npm 的 `beta` 标签；若创建 GitHub 版本说明，则标记为预发布。不会自动移动 npm 的 `latest` 标签。安装测试版时同时固定安装器与内容来源，例如 `npx bookmark-research@0.5.0-beta.7 install --source npm:bookmark-research@0.5.0-beta.7`。仅将 npx 改为 `@beta` 不会改变默认内容来源 `latest`；已固定的旧版安装也不会自动换成测试版。
+候选版本在 `release/` 分支同步三个版本清单并运行 CI。该分支的 registry 集成测试使用当前已发布渠道版本，源码测试、原生安装测试和 npm 打包检查仍验证候选内容；主分支的 registry 测试始终验证 marketplace 固定版本。发布通过 CI 的 npm 包并核验 registry 后，再将候选分支合入主分支，避免默认安装清单提前指向未发布版本。
+
+测试版使用 npm 的 `beta` 标签；若创建 GitHub 版本说明，则标记为预发布。不会自动移动 npm 的 `latest` 标签。安装测试版时同时固定安装器与内容来源，例如 `npx bookmark-research@0.5.0-beta.8 install --source npm:bookmark-research@0.5.0-beta.8`。仅将 npx 改为 `@beta` 不会改变默认内容来源 `latest`；已固定的旧版安装也不会自动换成测试版。
 
 ## 官方资料与项目实例
 

package/docs/release-notes-0.5.0-beta.8.md

@@ -0,0 +1,34 @@
+# Bookmark Research 0.5.0 beta 8
+
+**Beta / 测试版.** Published through the npm `beta` channel; the `latest` tag remains unchanged.
+
+## Install
+
+Pin both the installer and the plugin content:
+
+```sh
+npx bookmark-research@0.5.0-beta.8 install --source npm:bookmark-research@0.5.0-beta.8
+```
+
+Choose Codex, Claude Code, Pi or DSH in the installer. For an existing DSH profile, the released layer can also be installed directly:
+
+```sh
+dsh plugin --profile <name> add bookmark-research@0.5.0-beta.8
+```
+
+## Changes
+
+- DSH exports support legacy npm packages that lack the nested ESM manifest. Repair instructions retain the selected profile and release version. The Desktop prerequisite test now resolves actual executable shims on Windows.
+- Relocated Codex, Claude Code and Agent Plugins exports report their own release version during the real MCP handshake.
+- Cancelled or incomplete research tasks reject late provider results and further fallback reservations. Concurrent evidence imports retain the final source IDs in their manifests.
+- Source merging checks matching, nonempty synchronized versions inside the write transaction. Source removal preserves concurrent reimports; startup recovers hash-verified snapshots left by an interrupted removal without reviving committed deletions.
+- Wiki acknowledgements cannot accept invalid evidence. Unavailable sources require review, and a new acknowledgement retains its review note.
+- Canvas searches preserve literal `%`, `_` and backslash characters. Storage diagnostics close probe handles, preserve existing files and report paths blocked by files.
+
+## Verification
+
+Regression coverage includes research cancellation, source merge/removal races, process-exit recovery, Wiki evidence integrity, exported release identity and DSH installation prerequisites. CI runs the Python suite and npm command smoke checks on Linux, macOS and Windows, plus isolated native Codex installation, migration and update tests.
+
+Release branches test registry integration against the currently published package while validating the candidate's source and packed runtime. After publication, the main branch verifies the newly pinned npm version. Personal canvas packages and local evaluation reports are excluded from the published package.
+
+Install from [npm](https://www.npmjs.com/package/bookmark-research/v/0.5.0-beta.8). GitHub provides source and release notes; npm is the shared release package for all four hosts.

package/hosts/dsh/plugin.js

@@ -70,8 +70,9 @@ export function apply(ctx) {
   const missing = [cli, fileURLToPath(skillUrl)].filter((path) => !existsSync(path));
   if (missing.length > 0) {
     process.stderr.write("bookmark-research: installation is incomplete; missing " + missing.join(", ") + ".\n"
-      + "Reinstall the plugin (for example: npx --yes --package=bookmark-research@latest bookmark-research install"
-      + " --host dsh) or remove the stale registration that points at a deleted install directory.\n");
+      + "Reinstall the intended version in the affected profile: dsh plugin --profile <profile> add"
+      + " bookmark-research@<version> (use the Desktop app's CLI for profile desktop), or remove the stale"
+      + " registration that points at a deleted install directory.\n");
     return;
   }
   const env = Object.fromEntries(FORWARDED_ENV

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bookmark-research",
-  "version": "0.5.0-beta.7",
+  "version": "0.5.0-beta.8",
   "description": "Check bookmark URLs or Canvas packages, compare sources with host-led research, and retain verifiable reports.",
   "license": "GPL-3.0-only",
   "repository": {

package/scripts/host_assets.py

@@ -13,13 +13,15 @@ HOST_SOURCE_FILES = (
 )
 
 
-def read_asset(root, relative):
+def read_asset(root, relative, default=None):
     """Read a curated asset without following source symlinks."""
     current = Path(root)
     for part in Path(relative).parts:
         current = current / part
         if current.is_symlink():
             raise ValueError("Source symlinks are not exported: " + relative)
+    if default is not None and not current.exists():
+        return default
     if not current.is_file():
         raise ValueError("Required host asset is missing: " + relative)
     return current.read_bytes()
@@ -62,7 +64,10 @@ def export_host_assets(source, stage, host):
                 files["hosts/dsh/plugin.js"] = read_asset(source, "hosts/dsh/plugin.js")
                 # Scope ESM to the entry module so a package root may stay CommonJS
                 # (the npm package keeps a CommonJS bin and the pi field).
-                files["hosts/dsh/package.json"] = read_asset(source, "hosts/dsh/package.json")
+                # Older releases predate the nested manifest. Supply the scope
+                # when absent, while still rejecting links or invalid file types.
+                files["hosts/dsh/package.json"] = read_asset(
+                    source, "hosts/dsh/package.json", default=b'{"type":"module"}\n')
                 files["workflows/bookmark-research/meta.json"] = (
                     json.dumps(definition["meta"], ensure_ascii=False, indent=2) + "\n").encode("utf-8")
                 relative = "hosts/dsh/workflow-call.py"

package/src/bookmark_index.py

@@ -890,8 +890,8 @@ class BookmarkIndex:
         matches = []
         for row in found:
             haystack = ((row.get("label") or "") + "\n" + (row.get("text") or "")).casefold()
-            for target, pattern in zip(targets, patterns):
-                needle = pattern.strip("%").casefold()
+            for target in targets:
+                needle = target.casefold()
                 if needle and needle in haystack:
                     matches.append({"target": target, **row})
         return matches

package/src/cli.py

@@ -6,6 +6,7 @@ import json
 import os
 import sqlite3
 import sys
+import tempfile
 from pathlib import Path
 
 from bookmark_index import BookmarkIndex
@@ -28,19 +29,17 @@ def _writable_directory(path):
     """Report whether the nearest existing ancestor of ``path`` accepts a new file."""
     candidate = Path(path).expanduser()
     for parent in (candidate, *candidate.parents):
-        if parent.is_dir():
-            probe = parent / (".bookmark-research-write-probe-%d" % os.getpid())
-            try:
-                descriptor = os.open(str(probe), os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o600)
-            except OSError as error:
-                return {"path": str(candidate), "checked": str(parent), "writable": False,
-                        "error": error.strerror or str(error)}
-            finally:
-                try:
-                    probe.unlink()
-                except OSError:
+        try:
+            if parent.is_dir():
+                with tempfile.TemporaryFile(prefix=".bookmark-research-write-probe-", dir=parent):
                     pass
-            return {"path": str(candidate), "checked": str(parent), "writable": True, "error": None}
+                return {"path": str(candidate), "checked": str(parent), "writable": True, "error": None}
+            if parent.exists() or parent.is_symlink():
+                return {"path": str(candidate), "checked": str(parent), "writable": False,
+                        "error": "Existing path component is not a directory"}
+        except OSError as error:
+            return {"path": str(candidate), "checked": str(parent), "writable": False,
+                    "error": error.strerror or str(error)}
     return {"path": str(candidate), "checked": None, "writable": False, "error": "no existing parent directory"}
 
 

package/src/mcp_server.py

@@ -131,8 +131,8 @@ TOOL_DESCRIPTIONS = {
     "research_readiness": "Check retrieval and optional research service readiness before each new web research question. Respects cached/always/manual preferences; refresh forces checks and offline prevents network. Reports missing credentials, catalog reachability and the exact scope of verified auth separately. observed_tools establish host tool visibility, not OAuth login. test_retrieval additionally uses provider quota for a sample search/read; no professional research job is started.",
     "sync_package": "Import a directory, ZIP or single section JSON, retaining a managed snapshot outside the original. Reuse source_id for a moved or partial export of the same canvas; path aliases are remembered. New exports default to snapshot; Git directories default to live monitoring. Existing sources retain their mode. completeness=partial preserves absent cards; complete reconciles a full mirror. Single cards must be partial. Source files are never modified.",
     "source_history": "List saved source versions and their complete managed snapshot paths, including data retained from partial imports. Reimport a snapshot path to recover its data and recorded relationships. This does not refresh sources or change existing research inventories.",
-    "source_remove": "Delete an imported source, its indexed rows and its managed snapshots. Without confirm=true this only reports what would be removed. Research tasks that froze the source keep their saved inventory but stop resolving its items, and Wiki pages citing those claims become stale; prefer source_merge for an identical duplicate.",
-    "source_merge": "Drop a duplicate source whose current content version equals into_source_id, moving its path aliases to the survivor so a later import of the same folder updates it. Fails when the two hold different content. Without confirm=true this only reports the merge.",
+    "source_remove": "Delete an imported source, its indexed rows and its managed snapshots. Without confirm=true this only reports what would be removed. Research tasks that froze the source keep their saved inventory but stop resolving its items, and Wiki pages citing those claims require review; prefer source_merge for an identical duplicate.",
+    "source_merge": "Drop a duplicate source whose current content version equals into_source_id, moving its path aliases to the survivor so a later import of the same folder updates it. Requires matching synchronized content versions; synchronize legacy sources first. Without confirm=true this only reports the merge.",
     "search_bookmarks": "Search bookmark metadata with literal title/URL/note/tag/folder-path matching and exact SQL scopes. Every whitespace-separated term must occur; three-character and longer terms use the trigram index while shorter ones use LIKE substring matching. Targets have independent totals and pages. Report instances separately from unique_urls: a copy anchor returns another card's shared tree, so per-section totals must not be summed. limit=0 or count_only returns counts without rows; item_types=[\"folder\"] enumerates folders, including the synthetic untitled tree container marked synthetic. Canvas text cards, group labels and edge labels matching a target are reported separately in canvas_matches and never inflate total. Refresh checks live directories by default; snapshots remain usable after the original export disappears. Inspect source.state for pending changes or errors. refresh=false uses the last synchronized index, not an arbitrary historical version.",
     "get_context": "Read section headers, bookmark metadata, folder ancestry, geometric group membership and directed canvas edges. Copy anchors share their primary tree. Refresh is enabled by default.",
     "index_status": "Read source modes, original input availability, saved snapshots, pending files, last check/error, counts and local monitor status. Does not refresh or fetch webpages. unchecked means a live directory has not been checked recently.",
@@ -316,8 +316,8 @@ TOOL_DESCRIPTIONS.update({
     "research_service_attach": "Attach an independently known provider run ID without starting remote work. Can recover a missing create response or reference a run started through an existing host integration. Status remains unverified until observed.",
     "research_service_import": "Import a saved completed provider report into a research session as unreviewed external-report evidence. Original URL coverage is unchanged; read and assess cited originals separately.",
     "wiki_write": "Publish an authored topic/entity page outside source packages, with reviewed active claims, source provenance, relationships and an immutable revision. Supply the current expected_revision when updating. Pass reviewed_input_version to record that the page was checked against that already-synchronized input version, which closes source_input_changed for cited tasks at that version. Semantic support requires the named human/model review.",
-    "wiki_acknowledge": "Re-publish a Wiki page unchanged to record that its cited sources were reviewed against their current input version, closing source_input_changed without editing content. Writes an immutable revision and preserves history; the note records who reviewed what. No network access and no semantic scoring.",
-    "wiki_get": "Read a Wiki page or an immutable earlier revision with source/claim links resolved at current research locations. validation.status=needs_review signals changed bookmark input after the cited research; this does not automatically revise the page or refetch webpages.",
+    "wiki_acknowledge": "Re-publish a Wiki page unchanged to record that its cited sources were reviewed against their current input version, closing source_input_changed without editing content. Writes an immutable revision and preserves history; the note records who reviewed what. Invalid evidence or links must be repaired first, and unavailable bookmark input still requires review. No network access and no semantic scoring.",
+    "wiki_get": "Read a Wiki page or an immutable earlier revision with source/claim links resolved at current research locations. validation.status=needs_review signals changed or unavailable bookmark input after the cited research; this does not automatically revise the page or refetch webpages.",
     "wiki_list": "List paginated Wiki pages and current revision metadata without network access.",
     "wiki_search": "Find saved knowledge when the task needs prior research absent from the conversation. Return matching Wiki text, revision dates and validation; needs_review flags changed bookmark input. Invalidated or retracted evidence is excluded. Literal full-text retrieval, not vector search.",
     "wiki_lint": "Check Wiki evidence hashes, active claims, source review states and cross-page links. Reports semantic review declarations separately from mechanical integrity checks.",

package/src/release.py

@@ -1,13 +1,10 @@
-"""Single release-version source, read from the installed package manifest.
+"""Release identity from the installed distribution's own manifest.
 
-The plugin is distributed as an npm package, so ``package.json`` travels with
-the code at the plugin root. Reading it here keeps the MCP handshake, remote
-client identity and HTTP user agents on the same version as the published
-manifest instead of drifting across hardcoded copies.
-
-When the manifest is missing (an exotic layout, or a single file copied out of
-the package) the version is reported as unknown rather than guessed, so a
-mismatch stays visible instead of being silently papered over.
+The npm, Pi and DSH packages carry ``package.json``. Codex, Claude and Agent
+Plugins exports retain the release version in their native plugin manifest.
+Read that identity for MCP and HTTP clients instead of hardcoding a version.
+An absent or invalid manifest reports an unknown version; an invalid preferred
+manifest never falls through to a different manifest that might hide damage.
 """
 
 import json
@@ -32,7 +29,12 @@ def read_manifest_version(path):
 @lru_cache(maxsize=1)
 def release_version():
     """Return this installation's manifest version."""
-    return read_manifest_version(Path(__file__).resolve().parents[1] / "package.json")
+    root = Path(__file__).resolve().parents[1]
+    for relative in ("package.json", ".codex-plugin/plugin.json", ".claude-plugin/plugin.json", "plugin.json"):
+        path = root / relative
+        if path.exists() or path.is_symlink():
+            return read_manifest_version(path)
+    return UNKNOWN_VERSION
 
 
 def user_agent(product="bookmark-research"):

package/src/research.py

@@ -774,6 +774,14 @@ class ResearchSessions:
                     references.append({key: bookmark[key] for key in ("source_id", "section_id", "item_id")})
         return [entry["id"] for entry in entries], references
 
+    @classmethod
+    def _pending_operation(cls, state, operation_id):
+        cls._active(state)
+        operation = cls._find(state["operations"], operation_id, "operation")
+        if operation["status"] != "pending":
+            raise ValueError("Research operation is no longer pending; its late result was not committed")
+        return operation
+
     def _operation(self, research_id, operation_id, kind, parameters, cost, execute):
         operation_id = self._identifier(operation_id, "operation_id")
         path = self._path(research_id)
@@ -823,14 +831,17 @@ class ResearchSessions:
         # writes made by another process while the request was in flight survive.
         with self._task_lock(path):
             _, state = self._load(research_id)
-            stored = next((row for row in state["operations"] if row["id"] == operation_id), None)
-            if stored is None:
-                raise ValueError("Research operation disappeared before it was committed")
+            # Cancellation and explicit interruption take effect immediately.
+            # A late response must not change a terminal report or revive its step.
+            stored = self._pending_operation(state, operation_id)
             new_sources = payload.pop("sources", [])
             committed = []
             for source in new_sources:
                 source = copy.deepcopy(source)
                 source["id"] = "s%s" % (len(state["sources"]) + 1)
+                if kind == "import_evidence":
+                    self._write(self._artifact(path, source["manifest_file"]),
+                                {"schema_version": 1, "kind": "host_evidence_import", "source": source})
                 state["sources"].append(source)
                 committed.append(source)
             stored["status"] = status
@@ -875,15 +886,14 @@ class ResearchSessions:
                 # latest state under the lock instead of trusting the snapshot.
                 with self._task_lock(path):
                     _, latest = self._load(research_id, path)
+                    stored = self._pending_operation(latest, operation_id)
                     remaining = latest["budget"]["max_search_calls"] - latest["usage"]["search_calls"]
                     chosen = jobs[:max(0, remaining)]
                     if chosen:
                         latest["usage"]["search_calls"] += len(chosen)
-                        stored = next((row for row in latest["operations"] if row["id"] == operation_id), None)
-                        if stored is not None:
-                            stored["reserved"]["search_calls"] = stored["reserved"].get("search_calls", 0) + len(chosen)
-                            stored["fallback_attempts"] = [{"provider": name, "target": target, "query": query}
-                                                           for name, target, query in chosen]
+                        stored["reserved"]["search_calls"] = stored["reserved"].get("search_calls", 0) + len(chosen)
+                        stored["fallback_attempts"] = [{"provider": name, "target": target, "query": query}
+                                                       for name, target, query in chosen]
                         self._save(path, latest)
                 return chosen
 
@@ -961,6 +971,7 @@ class ResearchSessions:
                 # so a concurrent writer cannot lose its usage or sources.
                 with self._task_lock(path):
                     _, latest = self._load(research_id, path)
+                    stored = self._pending_operation(latest, operation_id)
                     remaining = latest["budget"]["max_fetch_calls"] - latest["usage"]["fetch_calls"]
                     selected, reserved = [], 0
                     for name in candidates:
@@ -971,10 +982,8 @@ class ResearchSessions:
                             remaining -= cost
                     if selected:
                         latest["usage"]["fetch_calls"] += reserved
-                        stored = next((row for row in latest["operations"] if row["id"] == operation_id), None)
-                        if stored is not None:
-                            stored["reserved"]["fetch_calls"] = stored["reserved"].get("fetch_calls", 0) + reserved
-                            stored["fallback_attempts"] = [{"provider": name, "urls": pending} for name in selected]
+                        stored["reserved"]["fetch_calls"] = stored["reserved"].get("fetch_calls", 0) + reserved
+                        stored["fallback_attempts"] = [{"provider": name, "urls": pending} for name in selected]
                         self._save(path, latest)
                 return selected
 
@@ -1060,7 +1069,7 @@ class ResearchSessions:
             identifiers, refs = self._source_links(current, url)
             if supplied_ids is not None:
                 identifiers = supplied_ids
-            source = {"id": "s%s" % (len(current["sources"]) + 1), "question_id": question_id,
+            source = {"question_id": question_id,
                       "operation_id": operation_id, "url": url, "canonical_url": canonical,
                       "provider": provenance.get("provider", "host_import"),
                       "retrieved_at": provenance.get("retrieved_at"), "imported_at": imported_at,
@@ -1075,8 +1084,8 @@ class ResearchSessions:
                       "inventory_ids": [] if kind == "external_report" else identifiers,
                       "referenced_inventory_ids": (supplied_ids or []) if kind == "external_report" else [],
                       "bookmark_refs": [] if kind == "external_report" else refs}
-            self._write(manifest_file, {"schema_version": 1, "kind": "host_evidence_import", "source": source})
-            current["sources"].append(source)
+            # The commit assigns the source id and writes its manifest together.
+            # The request snapshot cannot predict ids allocated by other writers.
             return {"status": "ok", "sources": [source], "next_action":
                     "Read and review the saved text. An imported report is one secondary source and does not mark its referenced original pages as read."}
 

package/src/source_manager.py

@@ -3,11 +3,13 @@
 import hashlib
 import json
 import os
+import re
 import shutil
 import sqlite3
 import tempfile
 import time
 import uuid
+from contextlib import contextmanager
 from datetime import datetime, timezone
 from pathlib import Path
 
@@ -36,6 +38,60 @@ class SourceManager:
         with index.transaction():
             for statement in SCHEMA:
                 self.connection.execute(statement)
+            self._recover_retired_snapshots()
+
+    def _recover_retired_snapshots(self):
+        """Recover a removal interrupted before commit, using surviving DB references."""
+        if self.storage is None or self.storage.is_symlink() or not self.storage.is_dir():
+            return
+        retired = {}
+        for directory in self.storage.iterdir():
+            match = re.fullmatch(r"\.removed-([0-9a-f]{64})-[0-9a-f]{32}", directory.name)
+            if match and not directory.is_symlink() and directory.is_dir():
+                retired.setdefault(match[1], []).append(directory)
+        if not retired:
+            return
+        for row in self.connection.execute("SELECT source_id FROM sources").fetchall():
+            source_id = row[0]
+            try:
+                canonical = self._snapshot_directory(source_id)
+            except ValueError:
+                continue
+            candidates = retired.get(canonical.name, [])
+            if not candidates:
+                continue
+            references = list(self.connection.execute(
+                "SELECT version_id,snapshot_path FROM source_snapshots WHERE source_id=?", (source_id,)))
+            try:
+                current = self._record(source_id)
+            except (ValueError, TypeError):
+                current = None
+            if isinstance(current, dict):
+                references.append((current.get("version_id"), current.get("snapshot_path")))
+            for version_id, snapshot_path in references:
+                if not isinstance(version_id, str) or not re.fullmatch(r"v-[0-9a-f]{64}", version_id):
+                    continue
+                destination = canonical / version_id
+                if (not isinstance(snapshot_path, str) or Path(snapshot_path) != destination / "package"
+                        or destination.exists() or destination.is_symlink()):
+                    continue
+                for directory in candidates:
+                    saved = directory / version_id
+                    if saved.is_symlink() or not saved.is_dir():
+                        continue
+                    try:
+                        if any(entry.is_symlink() for entry in saved.rglob("*")):
+                            continue
+                        manifest = self._imported_snapshot(read_input(saved / "package"))
+                    except (OSError, ValueError, TypeError, KeyError):
+                        continue
+                    if not manifest or manifest.get("source_id") != source_id or manifest.get("version_id") != version_id:
+                        continue
+                    canonical.mkdir(exist_ok=True)
+                    saved.rename(destination)
+                    break
+        # Unreferenced versions and empty retirement directories stay untouched.
+        # A committed removal must never be undone just because its cleanup died.
 
     def _now(self):
         return datetime.fromtimestamp(self.clock(), timezone.utc).isoformat()
@@ -296,36 +352,66 @@ class SourceManager:
             raise ValueError("Managed snapshot directory is not safe to change")
         return directory
 
+    @contextmanager
+    def _source_removal(self, source_id):
+        """Retire snapshots before commit; clean only this removal's directory."""
+        if self.connection.in_transaction:
+            raise ValueError("Source remove/merge must run outside another index transaction")
+        snapshot = retired = None
+        cleanup = {"snapshot_removed": False}
+        try:
+            with self.index.transaction():
+                yield cleanup
+                snapshot = self._snapshot_directory(source_id)
+                if snapshot is not None and snapshot.is_dir():
+                    retired = snapshot.with_name(".removed-" + snapshot.name + "-" + uuid.uuid4().hex)
+                    snapshot.rename(retired)
+        except BaseException:
+            if retired is not None and retired.is_dir():
+                # A failed commit may have released SQLite's lock. Reacquire it
+                # and preserve any new snapshot published since that rollback.
+                with self.index.transaction():
+                    self._snapshot_directory(source_id)
+                    if not snapshot.exists():
+                        retired.rename(snapshot)
+                    else:
+                        # Version directories are immutable. Restore missing old
+                        # versions without replacing a concurrent import's files.
+                        for version in retired.iterdir():
+                            destination = snapshot / version.name
+                            if not destination.exists():
+                                version.rename(destination)
+                        shutil.rmtree(retired)
+            raise
+        if retired is not None:
+            shutil.rmtree(retired)
+            cleanup["snapshot_removed"] = True
+
     def remove(self, source_id, confirm=False):
         """Delete a source, its indexed rows and its managed snapshots.
 
         Research inventories that froze this source keep their saved input but
         can no longer resolve it, so a caller must confirm explicitly.
         """
-        record = self._record(source_id)
-        if record is None:
-            raise ValueError("Unknown source_id: " + str(source_id))
-        counts = self._source_counts(source_id)
-        snapshot = self._snapshot_directory(source_id)
-        aliases = [row[0] for row in self.connection.execute(
-            "SELECT input_path FROM source_aliases WHERE source_id=? ORDER BY input_path", (source_id,))]
-        if not confirm:
-            return {"source_id": source_id, "status": "confirmation_required", "would_remove": counts,
-                    "aliases": aliases, "snapshot_directory": str(snapshot) if snapshot else None,
-                    "impact": "Research tasks that froze this source keep their saved inventory but stop resolving "
-                              "its items; Wiki pages citing those claims become stale. Re-run with confirm=true "
-                              "(CLI: --confirm) to delete."}
-        with self.index.transaction():
+        transaction = self._source_removal(source_id) if confirm else self.index.read_snapshot()
+        with transaction as cleanup:
+            self._record(source_id)
+            counts = self._source_counts(source_id)
+            snapshot = self._snapshot_directory(source_id)
+            aliases = [row[0] for row in self.connection.execute(
+                "SELECT input_path FROM source_aliases WHERE source_id=? ORDER BY input_path", (source_id,))]
+            if not confirm:
+                return {"source_id": source_id, "status": "confirmation_required", "would_remove": counts,
+                        "aliases": aliases, "snapshot_directory": str(snapshot) if snapshot else None,
+                        "impact": "Research tasks that froze this source keep their saved inventory but stop resolving "
+                                  "its items; Wiki pages citing those claims require review. Re-run with confirm=true "
+                                  "(CLI: --confirm) to delete."}
             for table in ("items", "sections", "files", "nodes", "edges", "memberships",
                           "source_snapshots", "source_aliases", "source_records"):
                 self.connection.execute("DELETE FROM %s WHERE source_id=?" % table, (source_id,))
             self.connection.execute("DELETE FROM sources WHERE source_id=?", (source_id,))
-        removed_snapshot = False
-        if snapshot is not None and snapshot.is_dir() and not snapshot.is_symlink():
-            shutil.rmtree(snapshot)
-            removed_snapshot = True
         return {"source_id": source_id, "status": "removed", "removed": counts,
-                "aliases": aliases, "snapshot_removed": removed_snapshot}
+                "aliases": aliases, **cleanup}
 
     def merge(self, source_id, into_source_id, confirm=False):
         """Drop a duplicate source whose content version equals the target's.
@@ -335,23 +421,23 @@ class SourceManager:
         """
         if source_id == into_source_id:
             raise ValueError("source_id and into_source_id must differ")
-        record, target = self._record(source_id), self._record(into_source_id)
-        if record is None:
-            raise ValueError("Unknown source_id: " + str(source_id))
-        if target is None:
-            raise ValueError("Unknown into_source_id: " + str(into_source_id))
-        if record.get("version_id") != target.get("version_id"):
-            raise ValueError("Sources hold different content versions; keep one explicitly with source remove "
-                             "instead of merging")
-        aliases = [row[0] for row in self.connection.execute(
-            "SELECT input_path FROM source_aliases WHERE source_id=? ORDER BY input_path", (source_id,))]
-        if not confirm:
-            return {"source_id": source_id, "into_source_id": into_source_id, "status": "confirmation_required",
-                    "version_id": record.get("version_id"), "moved_aliases": aliases,
-                    "impact": "The duplicate is deleted and its paths re-point at into_source_id; saved research "
-                              "inventories that froze the duplicate stop resolving. Re-run with confirm=true "
-                              "(CLI: --confirm) to merge."}
-        with self.index.transaction():
+        transaction = self._source_removal(source_id) if confirm else self.index.read_snapshot()
+        with transaction as cleanup:
+            record, target = self._record(source_id), self._record(into_source_id)
+            if any(not isinstance(row.get("version_id"), str)
+                   or not re.fullmatch(r"v-[0-9a-f]{64}", row["version_id"]) for row in (record, target)):
+                raise ValueError("Sources need synchronized content versions before merging; sync legacy sources first")
+            if record["version_id"] != target["version_id"]:
+                raise ValueError("Sources hold different content versions; keep one explicitly with source remove "
+                                 "instead of merging")
+            aliases = [row[0] for row in self.connection.execute(
+                "SELECT input_path FROM source_aliases WHERE source_id=? ORDER BY input_path", (source_id,))]
+            if not confirm:
+                return {"source_id": source_id, "into_source_id": into_source_id, "status": "confirmation_required",
+                        "version_id": record["version_id"], "moved_aliases": aliases,
+                        "impact": "The duplicate is deleted and its paths re-point at into_source_id; saved research "
+                                  "inventories that froze the duplicate stop resolving. Re-run with confirm=true "
+                                  "(CLI: --confirm) to merge."}
             for alias in aliases:
                 self.connection.execute("INSERT OR IGNORE INTO source_aliases VALUES(?,?)", (into_source_id, alias))
             self.connection.execute("INSERT OR IGNORE INTO source_aliases VALUES(?,?)",
@@ -360,14 +446,9 @@ class SourceManager:
                           "source_snapshots", "source_aliases", "source_records"):
                 self.connection.execute("DELETE FROM %s WHERE source_id=?" % table, (source_id,))
             self.connection.execute("DELETE FROM sources WHERE source_id=?", (source_id,))
-        snapshot = self._snapshot_directory(source_id)
-        removed_snapshot = False
-        if snapshot is not None and snapshot.is_dir() and not snapshot.is_symlink():
-            shutil.rmtree(snapshot)
-            removed_snapshot = True
         return {"source_id": source_id, "into_source_id": into_source_id, "status": "merged",
                 "version_id": record.get("version_id"), "moved_aliases": aliases,
-                "snapshot_removed": removed_snapshot}
+                **cleanup}
 
     def _observation(self, source_id, record, state, **fields):
         record.update(state=state, checked_at=self._now(), **fields)

package/src/wiki.py

@@ -206,7 +206,11 @@ class WikiStore:
             if key not in cache:
                 cache[key] = self.research.source_freshness(state=cache[research_id][1])
             freshness = cache[key]
-            if freshness["requires_review"]:
+            if freshness.get("state") == "unknown":
+                issues.append({"severity": "warning", "code": "source_input_unavailable", "research_id": research_id,
+                    "message": "The cited bookmark input cannot be checked; restore or synchronize its source before reviewing this page.",
+                    "source_freshness": freshness})
+            elif freshness["requires_review"]:
                 # A recorded review of this exact input version closes the
                 # warning. A later change produces a new version and reopens it.
                 acknowledged = next((row for row in self._acknowledged(record)
@@ -312,6 +316,11 @@ class WikiStore:
             if expected_revision != (previous["revision"] if previous else 0):
                 raise ValueError("Wiki revision changed; read the current page before updating")
             prior_record = self._revision(previous) if previous else None
+            if acknowledge_all and prior_record is not None:
+                # A freshness acknowledgement must preserve the previously
+                # published evidence baseline, even if it changed after get().
+                if any(issue["severity"] == "error" for issue in self._issues(prior_record, index, cache)):
+                    raise ValueError("Wiki acknowledgement cannot accept invalid evidence or links; repair them before reviewing input changes")
             acknowledged = self._acknowledgements(references, cache, reviewed_input_version, acknowledge_all,
                                                   page["review"]["note"], prior_record)
             if prior_record is None and len(index["pages"]) >= 5000:
@@ -361,6 +370,7 @@ class WikiStore:
         note = self._text(note, "Acknowledge note")
         current = self.get(page_id)
         revision = current["revision"] if expected_revision is None else expected_revision
+        current["page"]["review"]["note"] = note
         result = self.write(page_id, current["page"], "Reviewed against the current input: " + note,
                             expected_revision=revision, acknowledge_all=True)
         result["acknowledged_note"] = note
@@ -424,7 +434,8 @@ class WikiStore:
     @staticmethod
     def _validation(issues):
         return {"status": "stale" if any(item["severity"] == "error" for item in issues) else
-                          "needs_review" if any(item["code"] == "source_input_changed" for item in issues) else "current",
+                          "needs_review" if any(item["code"] in ("source_input_changed", "source_input_unavailable")
+                                                for item in issues) else "current",
                 "issues": issues, "semantic_review": "caller_declared; not evaluated by lint"}
 
     def get(self, page_id, revision=None):