@yolk_vat-y/dsh-project-memory 0.2.0 → 0.3.0

package/CHANGELOG.md

@@ -1,11 +1,24 @@
 # Changelog
 
+## 0.2.1 (unreleased)
+
+### 存储：无锁同步事务重构
+- 删除全部锁机制（`withStoreLock`、`dirLocks`、Promise 链锁）：写入统一走同步事务 `store.commit(fn)`，fn 成功后才原子落盘；单进程内天然串行，`remember`/`forget` 不再被 watch 重索引排队阻塞
+- 新增幂等更新 `store.applyFileUpdate(rel, { expectedHash, ... })`：CAS 校验统一 null 处理；`deleted` 删除跳过 hash 对比；type/size 完整透传不猜测
+- `watch`/`index_repo`/`index_doc`/`lazy` 全部改为「事务外计算 → 单次 commit 提交」：LLM 摘要、符号扫描等重活不持任何锁
+- watch 修正：`seen.add` 前置保护全部遍历文件；dump 文件标记 `deleted` 更新 snapshot 但不索引；snapshot 只更新成功处理的文件，CAS 失败回滚下轮重试；索引失败删除 snapshot 自动重试
+- watch 修复回归：snapshot 改用首轮采集的签名落定（而非 commit 后重新 stat），文件在计算窗口内被修改时下一轮能重新检出并重索引，恢复自愈语义；顺带去掉代码文件重复 push
+- `watch_repo` 的 session watchlist 镜像同步去锁
+- 测试 157/157 全绿
+
 ## 0.2.0 (2026-08-27)
 
 ### CJK 检索增强
-- 链接侧：非拉丁符号名改用 CJK 边界正则 `(?<![CJK])名(?![CJK])`，解决 `用户服务` 误链 `用户服务管理器`；英文分支零改动
+- 链接侧：非拉丁符号名改用 CJK 后边界正则 `名(?![CJK])`，去掉前边界，解决 `调用用户服务` 漏链 `用户服务`；混合名（含字母数字）尾部同时挡 CJK 与字母数字，阻断 `用户服务V2管理器`/`用户服务V22` 误链
 - 查询侧：BM25 增加精确短语乘法加分 —— 3+ 字 CJK 短语命中 `title`/`keywords` 时 `score *= 1.5` 并重排，自适应不压过高相关结果
 - 查询侧：同义词表（`数据库连接池` ↔ `连接池` ↔ `DB pool`），查询展开后再走短语加分
+- 健壮性：`buildBm25.score(undefined)` 不再抛错
+- 测试：新增 CJK 链接边界、短语加分、同义词展开、空值保护回归测试
 
 ### 经验笔记 supersede 阈值收紧
 - 双向重叠判定：`overlap / query_tokens ≥ 0.7` 且 `overlap / item_tokens ≥ 0.7`（原单向 0.6），减少短问题误吞长笔记

package/README.md

@@ -15,8 +15,8 @@ Persistent project memory for [DeepSeek Harness](https://github.com/deepseek-ai/
 - **Automatic refresh** — a background poll (`watch_repo`) detects new or changed files by content hash and re-indexes only those.
 - **Read-time indexing** — files are indexed the moment the model actually reads them (`fs/observed`), so the index is a byproduct of normal work, not a separate upfront scan. Files that are never read are never indexed. The project root is detected by markers (`.git`, `package.json`, …), a README plus source directories, or the file's own directory as a last resort.
 - **Doc ↔ code cross-linking** — when a document mentions a symbol, the match is recorded as a `reference`; querying a symbol also surfaces the documents that describe it.
-- **BM25 retrieval** — ranked search over documents, symbols, and experience notes, with optional LLM query expansion to handle vocabulary mismatch.
-- **Experience notes** — problems → solutions; similar problems supersede instead of duplicating, and notes are returned only when a search matches. The note store is bounded: capacity scales with project size (clamped to 100–2000), and the oldest notes are pruned when the limit is exceeded.
+- **BM25 retrieval** — ranked search over documents, symbols, and experience notes, with optional LLM query expansion to handle vocabulary mismatch. **CJK-optimized**: precise phrase boost (3+ char phrases ×1.5 score on title/keywords match), synonym table (e.g. 数据库连接池 ↔ 连接池 ↔ DB pool), and CJK-aware word boundaries for doc↔symbol linking.
+- **Experience notes** — problems → solutions; similar problems supersede instead of duplicating, and notes are returned only when a search matches. The note store is bounded: capacity scales with project size (clamped to 100–2000), and the oldest notes are pruned when the limit is exceeded. **Supersede tightened to bidirectional 0.7 overlap** (was 0.6); **experience `problem` field now participates in CJK phrase boost** for long-tail query recall.
 - **Minimal dependencies** — pure JavaScript; the only runtime dependency is `pdfjs-dist` (PDF text extraction), no native builds required.
 
 ## How it works
@@ -34,11 +34,10 @@ The store is per-project and follows the codebase: changed files are re-extracte
 
 ## Installation
 
-Tested against dsh **0.1.0-rc.7 through 0.1.1-rc.2**. Newer rc lines are expected to work — the plugin only touches stable surfaces (`defineTool`, `llm.stream`, `Schema`) — but they are not individually verified. The host provides `@deepseek-ai/cordis` ^4.0.1 and `@deepseek-ai/schemastery` ^3.18.1, declared via peerDependencies.
+Tested against dsh **0.1.0-rc.7 through 0.1.2-alpha.1**. The plugin relies exclusively on stable public APIs (`defineTool`, `llm.stream`, `Schema`) declared via peerDependencies, ensuring compatibility with future rc releases without changes.
 
 ```bash
-cd dsh-project-memory
-dsh plugin --profile web add . -w
+cd dsh-project-memory && dsh plugin --profile web add . -w
 ```
 
 The `-w` (workspace-root) flag is required: the profile directory is a pnpm workspace root, and pnpm rejects `add` there without it. From any other directory, the path form works the same: `dsh plugin --profile web add /path/to/dsh-project-memory -w`.
@@ -52,7 +51,7 @@ dsh plugin --profile web add @yolk_vat-y/dsh-project-memory -w
 A prebuilt tarball is published with each release, installable without a build step:
 
 ```bash
-dsh plugin --profile web add /path/to/dsh-project-memory-0.1.6.tgz
+dsh plugin --profile web add /path/to/dsh-project-memory-0.2.0.tgz
 ```
 
 Each indexed project has its own store at `<root>/.dsh-project-memory/`. Add it to `.gitignore` if it should not be committed.
@@ -99,6 +98,7 @@ These are deliberate scope choices.
 - **Absolute source paths** — entries cite absolute paths; moving a project invalidates citations until the next re-index.
 - **`forget` by query is eager** — keyword deletion matches at ≥0.5 token overlap and may remove several notes at once; prefer deleting by id for precision.
 - **Cross-language recall depends on index time** — with `llmQueryExpansion` off, a Chinese-only query reaches English content through bilingual keywords captured when docs are indexed, plus doc↔symbol links; queries stay LLM-free. Stores indexed before v0.1.1 gain bilingual keywords as files change, or immediately via `index_repo` with `reindex: true`.
+- **CJK retrieval** — phrase boost and synonym expansion are purely query-side; they do not increase index size or LLM usage. Link boundaries use CJK-aware regex only; English symbols keep the original word-boundary behavior. The experience supersede threshold (0.7 bidirectional) is a conservative default; adjust via config if false positives/negatives appear in practice.
 
 ## Configuration
 

package/README.zh-CN.md

@@ -15,8 +15,8 @@
 - **自动刷新** — `watch_repo` 后台轮询，按内容哈希识别新增或变更文件，仅重抽这些文件。
 - **读到即索引** — 文件在模型**实际读取的瞬间**被索引（监听 `fs/observed`），索引是正常工作的副产品，而非额外的一次全量扫描。从未读过的文件不会被索引。项目根通过标记（`.git`、`package.json` 等）、README 加源码目录、或兜底到文件所在目录逐级识别。
 - **文档 ↔ 代码交叉链接** — 文档提及某符号时记录为 `reference`；查询符号时同时带出描述该符号的文档。
-- **BM25 检索** — 对文档、符号与经验笔记进行排序召回，可选 LLM 查询扩展以应对表述不一致。
-- **经验笔记** — 记录问题 → 方案；相似问题覆盖而非重复；笔记仅在检索命中时返回。笔记数量有界：容量随项目规模伸缩（钳制在 100–2000），超限时淘汰最旧的笔记。
+- **BM25 检索** — 对文档、符号与经验笔记进行排序召回，可选 LLM 查询扩展以应对表述不一致。**CJK 增强**：精确短语乘法加分（3+ 字短语在标题/关键词命中 ×1.5）、同义词表（如 数据库连接池 ↔ 连接池 ↔ DB pool）、CJK 感知的文档↔符号链接边界。
+- **经验笔记** — 记录问题 → 方案；相似问题覆盖而非重复；笔记仅在检索命中时返回。笔记数量有界：容量随项目规模伸缩（钳制在 100–2000），超限时淘汰最旧的笔记。**覆盖阈值收紧为双向 0.7 重叠**（原 0.6）；**经验 `problem` 字段现参与 CJK 短语加分**，提升长尾问句召回。
 - **依赖极简** — 纯 JavaScript；唯一运行时依赖是 `pdfjs-dist`（PDF 文本提取），无需原生构建。
 
 ## 工作原理
@@ -34,11 +34,10 @@
 
 ## 安装
 
-实测覆盖 dsh **0.1.0-rc.7 → 0.1.1-rc.2**。更高的 rc 版本预期可用——插件只使用稳定接口（`defineTool`、`llm.stream`、`Schema`）——但未经逐一验证。宿主需提供 `@deepseek-ai/cordis` ^4.0.1 与 `@deepseek-ai/schemastery` ^3.18.1，已通过 peerDependencies 声明。
+实测覆盖 dsh **0.1.0-rc.7 至 0.1.2-alpha.1**。插件仅依赖通过 peerDependencies 声明的稳定公共 API（`defineTool`、`llm.stream`、`Schema`），保证与后续 rc 版本无需改动即兼容。
 
 ```bash
-cd dsh-project-memory
-dsh plugin --profile web add . -w
+cd dsh-project-memory && dsh plugin --profile web add . -w
 ```
 
 `-w`（workspace-root）标志是必需的：profile 目录是 pnpm 工作区根目录，不带该标志 pnpm 会拒绝 add。其他目录下同样可用路径形式：`dsh plugin --profile web add /path/to/dsh-project-memory -w`。
@@ -52,7 +51,7 @@ dsh plugin --profile web add @yolk_vat-y/dsh-project-memory -w
 每个版本会附带预构建 tarball，无需构建步骤即可安装：
 
 ```bash
-dsh plugin --profile web add /path/to/dsh-project-memory-0.1.6.tgz
+dsh plugin --profile web add /path/to/dsh-project-memory-0.2.0.tgz
 ```
 
 每个被索引的项目在 `<root>/.dsh-project-memory/` 下有独立存储。如无需入库，可加入 `.gitignore`。
@@ -87,18 +86,19 @@ v0.2.0 之前创建的库（单文件 `entries.json` / `index.json`）在首次
 - **增量** — 按文件内容哈希，仅重新抽取变更文件。
 - **交叉链接** — 索引后将文档摘要与符号名匹配，命中符号以 `references` 挂载到文档条目，由 `query_memory` 带出。
 - **查询扩展** — `llmQueryExpansion` 开启时，`query_memory` 让 `ctx.llm` 将查询改写为多个变体（同义词、中英、符号名猜测），再跨变体合并 BM25 分数；关闭时查询完全不碰 LLM。跨语种召回（中文问题命中英文内容）改由索引时承担：文档 keywords 要求同时覆盖文档语言与英文，doc↔symbol 链接也会从中文命中带出英文符号名。
-- **一致性** — 事实层跟随代码库（哈希重抽 / 删除即移除）；经验层仅检索，配合覆盖与 `forget` 机制。每个记忆目录的写入按进程内互斥锁串行化；请避免多个 dsh 实例同时写同一项目存储。
+- **一致性** — 事实层跟随代码库（哈希重抽 / 删除即移除）；经验层仅检索，配合覆盖与 `forget` 机制。每个记忆目录的写入走同步事务 `store.commit(fn)`：fn 内完成校验与变更、成功后才原子落盘，单进程内天然串行；请避免多个 dsh 实例同时写同一项目存储。
 
 ## 设计取舍
 
 以下是刻意的范围选择。
 
-- **进程内锁** — 存储写入按记忆目录在一个 dsh 进程内串行化；两个 dsh 实例共享同一项目存储时后写覆盖先写。跨进程锁需要常驻守护进程，违背纯 JS 插件、无后台服务的定位，故明确不支持多实例共写。
-- **watch 轮询持锁** — watcher 重索引变更文档（LLM 摘要）期间，`remember`/`forget` 会排队等待。轮询（mtime + 内容哈希）而非 `fs.watch` 事件驱动，是为了跨平台行为一致；重叠轮询靠同一把锁串行：安全，但大改动时可能堆积。间隔可用 `watchInterval` 调整。
+- **无锁同步事务** — 不采用锁：所有写入（index / watch / remember / forget / watch_repo）统一走同步事务 `store.commit(fn)`，fn 成功后才一次落盘；JS 单线程事件循环保证事务间不交错，`remember`/`forget` 不会被 watch 重索引阻塞排队。两个 dsh 实例共享同一项目存储时后写覆盖先写；跨进程协调需要常驻守护进程，违背纯 JS 插件、无后台服务的定位，故明确不支持多实例共写。
+- **watch 计算与提交分离** — watcher 在事务外完成 mtime/哈希/符号扫描/LLM 摘要等重活，再以单次 `commit` 原子应用全部变更；`applyFileUpdate` 用 CAS 校验（统一 null 处理、删除跳过对比）防并发修改，失败回滚 snapshot 下轮重试，索引失败删除 snapshot 自动重试。轮询（mtime + 内容哈希）而非 `fs.watch` 事件驱动，是为了跨平台行为一致。间隔可用 `watchInterval` 调整。
 - **损坏隔离重建** — 存储 JSON 损坏时该文件回落为空并在下次写入时重建；坏文件会改名备份为 `*.corrupt` 并输出错误日志，但该文件内的数据无法恢复。自动修复半写文件需要预写日志或嵌入式数据库，代价与收益不成比例——而隔离一个坏文件的成本几乎为零。
 - **绝对路径引用** — 条目引用绝对路径；项目搬家后引用失效，重建索引即恢复。
 - **`forget` 按关键词删除偏激进** — 关键词删除按 ≥0.5 token 重叠匹配，可能一次删掉多条；追求精确请用 id 删除。
 - **跨语种召回依赖索引时** — `llmQueryExpansion` 关闭时，纯中文查询靠索引时捕获的双语 keywords 和 doc↔symbol 链接触达英文内容，查询侧保持零 LLM 调用。v0.1.1 之前建立的索引随文件变更逐步获得双语关键词，或用 `index_repo` 的 `reindex: true` 立即重建。
+- **CJK 检索** — 短语加分与同义词展开完全在查询侧，不增加索引体积、不额外消耗 LLM token。链接边界仅对 CJK 使用正则边界，英文符号保持原有词边界行为。经验层 supersede 阈值（双向 0.7）为保守默认；若实测出现误覆盖/误漏报，可通过配置调整。
 
 ## 配置
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@yolk_vat-y/dsh-project-memory",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "description": "Persistent project memory for dsh agents: index docs (PDF/Markdown/text) and code symbols into a searchable per-workspace store, recall them with cited sources, and keep experience entries (problems -> solutions) searchable on demand.",
   "type": "module",
   "main": "src/index.js",
@@ -51,14 +51,22 @@
     "@deepseek-ai/dsh-llm": ">=0.0.1-rc.1 <0.1.0 || >=0.1.0-rc.1 <0.2.0-0 || >=0.1.1-rc.1 <0.2.0-0"
   },
   "peerDependenciesMeta": {
-    "@deepseek-ai/cordis": { "optional": true },
-    "@deepseek-ai/dsh-tools": { "optional": true },
-    "@deepseek-ai/schemastery": { "optional": true },
-    "@deepseek-ai/dsh-llm": { "optional": true }
+    "@deepseek-ai/cordis": {
+      "optional": true
+    },
+    "@deepseek-ai/dsh-tools": {
+      "optional": true
+    },
+    "@deepseek-ai/schemastery": {
+      "optional": true
+    },
+    "@deepseek-ai/dsh-llm": {
+      "optional": true
+    }
   },
   "dsh": {
     "bundle": {
       "patch": "./cordis.patch.yml"
     }
   }
-}
+}

package/src/lazy.js

@@ -5,7 +5,7 @@ import { isSupportedCode, isSupportedDoc, memoryRootFor, relativePath, sha256OfF
 import { buildDocEntries } from './doc-pipeline.js'
 import { scanSymbols } from './symbols.js'
 import { linkEntries } from './link.js'
-import { ProjectMemoryStore, withStoreLock } from './store.js'
+import { ProjectMemoryStore } from './store.js'
 
 const STRONG_MARKERS = ['.git', '.hg', '.svn']
 
@@ -82,50 +82,55 @@ export async function indexFile(ctx, config, filePath, watchManager = null) {
   if (!root) return false
 
   const memoryDir = memoryRootFor(root, config.memoryDir)
-  return withStoreLock(memoryDir, async () => {
-    const store = new ProjectMemoryStore(memoryDir).load()
-    const rel = storeKey(relativePath(root, filePath))
-    const existing = store.fileRecord(rel)
-    let hash
-    let size
-    try {
-      if (isSupportedCode(ext) && config.maxFileSizeMb && statSync(filePath).size > config.maxFileSizeMb * 1024 * 1024) {
-        return false
-      }
-      ;({ hash, size } = await sha256OfFile(filePath))
-    } catch {
+  const store = new ProjectMemoryStore(memoryDir).load()
+  const rel = storeKey(relativePath(root, filePath))
+  const existing = store.fileRecord(rel)
+  let hash
+  let size
+  try {
+    if (isSupportedCode(ext) && config.maxFileSizeMb && statSync(filePath).size > config.maxFileSizeMb * 1024 * 1024) {
       return false
     }
-    if (existing && existing.sha256 === hash) return false
+    ;({ hash, size } = await sha256OfFile(filePath))
+  } catch {
+    return false
+  }
+  if (existing && existing.sha256 === hash) return false
 
-    if (watchManager) {
-      watchManager.addRoot(root)
-      store.addWatch(root)
-    }
+  if (watchManager) {
+    watchManager.addRoot(root)
+    store.addWatch(root)
+  }
 
-    let entries
-    if (isSupportedCode(ext)) {
-      entries = scanSymbols(filePath, readFileSync(filePath, 'utf8'))
-      store.markFile(rel, { sha256: hash, size, type: 'code', indexedAt: new Date().toISOString() })
-    } else {
-      entries = await buildDocEntries(ctx.llm, filePath, {
-        chunkChars: config.chunkChars,
-        maxChunks: config.maxChunksPerFile,
-        maxFileSizeMb: config.maxFileSizeMb,
-        maxPdfPages: config.maxPdfPages,
-      })
-      if (entries === null) {
-        store.removeFile(rel)
-        store.save()
+  let entries
+  if (isSupportedCode(ext)) {
+    entries = scanSymbols(filePath, readFileSync(filePath, 'utf8'))
+    return store.commit((s) => {
+      s.markFile(rel, { sha256: hash, size, type: 'code', indexedAt: new Date().toISOString() })
+      s.setEntries(rel, entries)
+      linkEntries(s)
+      return true
+    })
+  } else {
+    entries = await buildDocEntries(ctx.llm, filePath, {
+      chunkChars: config.chunkChars,
+      maxChunks: config.maxChunksPerFile,
+      maxFileSizeMb: config.maxFileSizeMb,
+      maxPdfPages: config.maxPdfPages,
+    })
+    if (entries === null) {
+      return store.commit((s) => {
+        s.removeFile(rel)
         return false
-      }
-      store.markFile(rel, { sha256: hash, size, type: 'doc', indexedAt: new Date().toISOString() })
+      })
     }
-    store.setEntries(rel, entries)
-    linkEntries(store)
-    store.save()
-    return true
-  })
+    return store.commit((s) => {
+      s.markFile(rel, { sha256: hash, size, type: 'doc', indexedAt: new Date().toISOString() })
+      s.setEntries(rel, entries)
+      linkEntries(s)
+      return true
+    })
+  }
 }
 
 export function codeFirst(paths) {

package/src/link.js

@@ -7,7 +7,8 @@ function buildMatcher(name) {
     return { lower, re: new RegExp(`(?<![a-z0-9_$])${lower}(?![a-z0-9_$])`) }
   }
   const escaped = lower.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
-  return { lower, re: new RegExp(`(?<!${CJK_CHAR.source})${escaped}(?!${CJK_CHAR.source})`) }
+  // 尾部统一挡 CJK + 字母数字下划线，防止纯 CJK 名误链混合后缀、混合名误链更长后缀
+  return { lower, re: new RegExp(`${escaped}(?!${CJK_CHAR.source})(?![a-z0-9_$])`) }
 }
 
 export function linkEntries(store) {

package/src/parsers/pdfjs-parser.js

@@ -1,5 +1,24 @@
 import { readFile } from 'node:fs/promises'
-import { getDocument, GlobalWorkerOptions, OPS } from 'pdfjs-dist/legacy/build/pdf.mjs'
+
+let pdfjsPromise
+let configuredWorkerSrc
+
+function loadPdfjs() {
+  if (!pdfjsPromise) {
+    pdfjsPromise = import('pdfjs-dist/legacy/build/pdf.mjs')
+      .then((mod) => {
+        if (configuredWorkerSrc) {
+          mod.GlobalWorkerOptions.workerSrc = configuredWorkerSrc
+        }
+        return mod
+      })
+      .catch((err) => {
+        pdfjsPromise = undefined
+        throw err
+      })
+  }
+  return pdfjsPromise
+}
 
 const PDFJS_OPTIONS = {
   useSystemFonts: true,
@@ -36,6 +55,7 @@ function extractPageText(items) {
 
 export async function parsePdf(filePath, { pages = null, maxPages = 1000, backend = 'pdfjs', collectLayoutStats = false } = {}) {
   const data = new Uint8Array(await readFile(filePath))
+  const { getDocument } = await loadPdfjs()
   const loadingTask = getDocument({ data, ...PDFJS_OPTIONS })
   const doc = await loadingTask.promise
 
@@ -55,6 +75,7 @@ export async function parsePdf(filePath, { pages = null, maxPages = 1000, backen
       pageList.push({ page: n, text: extractPageText(content.items) })
 
       if (collectLayoutStats && sampledPages < 5) {
+        const { OPS } = await loadPdfjs()
         const opList = await page.getOperatorList()
         for (const fn of opList.fnArray) {
           if (fn === OPS.paintImageXObject || fn === OPS.paintInlineImageXObject) imageCount++
@@ -86,6 +107,7 @@ export async function parsePdf(filePath, { pages = null, maxPages = 1000, backen
 
 export async function parsePdfInfo(filePath, maxPages = 1000) {
   const data = new Uint8Array(await readFile(filePath))
+  const { getDocument } = await loadPdfjs()
   const loadingTask = getDocument({ data, ...PDFJS_OPTIONS })
   const doc = await loadingTask.promise
 
@@ -115,5 +137,10 @@ export async function parsePdfInfo(filePath, maxPages = 1000) {
 }
 
 export function configurePdfjsWorker(workerSrc) {
-  GlobalWorkerOptions.workerSrc = workerSrc
+  configuredWorkerSrc = workerSrc
+  if (pdfjsPromise) {
+    pdfjsPromise.then((mod) => {
+      mod.GlobalWorkerOptions.workerSrc = workerSrc
+    })
+  }
 }

package/src/store.js

@@ -10,27 +10,8 @@ const EXPERIENCE_FILE = 'experience.json'
 const WATCH_FILE = 'watch.json'
 const SHARDS_DIR = 'shards'
 
-const dirLocks = new Map()
 const storeCache = new Map()
 
-export async function withStoreLock(memoryDir, fn) {
-  const key = path.resolve(memoryDir)
-  const prev = dirLocks.get(key) || Promise.resolve()
-  let release
-  const cur = new Promise((resolve) => {
-    release = resolve
-  })
-  const chain = prev.then(() => cur)
-  dirLocks.set(key, chain)
-  await prev
-  try {
-    return await fn()
-  } finally {
-    release()
-    if (dirLocks.get(key) === chain) dirLocks.delete(key)
-  }
-}
-
 function loadJson(filePath, fallback) {
   let raw
   try {
@@ -339,6 +320,31 @@ export class ProjectMemoryStore {
       experience: this.experience.length,
     }
   }
+
+  commit(fn) {
+    const result = fn(this)
+    this.save()
+    return result
+  }
+
+  applyFileUpdate(relPath, { expectedHash, hash, entries, meta, deleted, type, size }) {
+    const cur = this.fileRecord(relPath)
+    if (!deleted && (cur?.sha256 ?? null) !== (expectedHash ?? null)) {
+      return { skipped: true }
+    }
+    if (deleted) {
+      this.removeFile(relPath)
+      return { ok: true }
+    }
+    this.markFile(relPath, {
+      sha256: hash,
+      size,
+      type,
+      indexedAt: new Date().toISOString(),
+    })
+    this.setEntries(relPath, entries)
+    return { ok: true }
+  }
 }
 
 function existsSafe(p) {

package/src/tools/forget.js

@@ -1,6 +1,6 @@
 import { defineTool } from '@deepseek-ai/dsh-tools'
 import { memoryRootFor, resolveIndexRoot } from '../util/fs.js'
-import { ProjectMemoryStore, withStoreLock } from '../store.js'
+import { ProjectMemoryStore } from '../store.js'
 
 export function forgetTool(config) {
   return defineTool({
@@ -25,10 +25,9 @@ export function forgetTool(config) {
     async execute(args, exec) {
       const root = resolveIndexRoot(exec, args.root)
       const memoryDir = memoryRootFor(root, config.memoryDir)
-      return withStoreLock(memoryDir, () => {
-        const store = new ProjectMemoryStore(memoryDir).load()
-        const removed = store.removeExperience(args.id_or_query)
-        store.save()
+      const store = new ProjectMemoryStore(memoryDir).load()
+      return store.commit((s) => {
+        const removed = s.removeExperience(args.id_or_query)
         return removed > 0 ? `Removed ${removed} experience note(s).` : 'No matching experience note found.'
       })
     },

package/src/tools/index-doc.js

@@ -3,7 +3,7 @@ import path from 'node:path'
 import { assertReadableFile, memoryRootFor, sha256OfFile, storeKey } from '../util/fs.js'
 import { buildDocEntries } from '../doc-pipeline.js'
 import { linkEntries } from '../link.js'
-import { ProjectMemoryStore, withStoreLock } from '../store.js'
+import { ProjectMemoryStore } from '../store.js'
 import { findProjectRoot } from '../lazy.js'
 
 export function indexDocTool(ctx, config) {
@@ -33,32 +33,32 @@ export function indexDocTool(ctx, config) {
       const root = path.resolve(args.root && args.root.trim() ? args.root : findProjectRoot(filePath))
       const memoryDir = memoryRootFor(root, config.memoryDir)
 
-      return withStoreLock(memoryDir, async () => {
-        const store = new ProjectMemoryStore(memoryDir).load()
+      const store = new ProjectMemoryStore(memoryDir).load()
 
-        const rel = storeKey(path.relative(root, filePath).split(path.sep).join('/'))
-        const { hash, size } = await sha256OfFile(filePath)
-        const existing = store.fileRecord(rel)
-        if (existing && existing.sha256 === hash) {
-          return `Skipped (unchanged): ${rel}\nAlready indexed with ${(store.entries[rel] || []).length} entry/entries.`
-        }
+      const rel = storeKey(path.relative(root, filePath).split(path.sep).join('/'))
+      const { hash, size } = await sha256OfFile(filePath)
+      const existing = store.fileRecord(rel)
+      if (existing && existing.sha256 === hash) {
+        return `Skipped (unchanged): ${rel}\nAlready indexed with ${(store.entries[rel] || []).length} entry/entries.`
+      }
 
-        const entries = await buildDocEntries(ctx.llm, filePath, {
-          chunkChars: config.chunkChars,
-          maxChunks: config.maxChunksPerFile,
-          maxFileSizeMb: config.maxFileSizeMb,
-          maxPdfPages: config.maxPdfPages,
-        })
-        if (entries === null) {
-          store.removeFile(rel)
-          store.save()
+      const entries = await buildDocEntries(ctx.llm, filePath, {
+        chunkChars: config.chunkChars,
+        maxChunks: config.maxChunksPerFile,
+        maxFileSizeMb: config.maxFileSizeMb,
+        maxPdfPages: config.maxPdfPages,
+      })
+      if (entries === null) {
+        return store.commit((s) => {
+          s.removeFile(rel)
           return `Skipped: ${rel} looks like a reflection dump, not a document.`
-        }
-        store.setEntries(rel, entries)
-        store.markFile(rel, { sha256: hash, size, type: 'doc', indexedAt: new Date().toISOString() })
-        linkEntries(store)
-        store.save()
+        })
+      }
 
+      return store.commit((s) => {
+        s.setEntries(rel, entries)
+        s.markFile(rel, { sha256: hash, size, type: 'doc', indexedAt: new Date().toISOString() })
+        linkEntries(s)
         const preview = entries
           .map((e) => `  - ${e.title} @ ${rel}:${e.sourceLine}`)
           .join('\n')

package/src/tools/index-repo.js

@@ -5,90 +5,101 @@ import { isSupportedCode, isSupportedDoc, looksLikeDump, memoryRootFor, relative
 import { buildDocEntries } from '../doc-pipeline.js'
 import { scanSymbols } from '../symbols.js'
 import { linkEntries } from '../link.js'
-import { ProjectMemoryStore, withStoreLock } from '../store.js'
+import { ProjectMemoryStore } from '../store.js'
 
 export async function indexRepository(ctx, config, root, { reindex = false } = {}) {
-  return withStoreLock(memoryRootFor(root, config.memoryDir), async () => {
-    const store = new ProjectMemoryStore(memoryRootFor(root, config.memoryDir)).load()
+  const memoryDir = memoryRootFor(root, config.memoryDir)
+  const store = new ProjectMemoryStore(memoryDir).load()
 
-    const files = walkDir(root)
-    const seen = new Set()
-    let indexed = 0
-    let updated = 0
-    let skipped = 0
-    let removed = 0
-    const failures = []
+  const files = walkDir(root)
+  const seen = new Set()
+  let indexed = 0
+  let updated = 0
+  let skipped = 0
+  let removed = 0
+  const failures = []
 
-    for (const filePath of files) {
-      const rel = storeKey(relativePath(root, filePath))
-      seen.add(rel)
-      const ext = path.extname(filePath).toLowerCase()
-      if (!isSupportedDoc(ext) && !isSupportedCode(ext)) continue
+  // First pass: collect all file info and compute hashes/entries (async work outside commit)
+  const fileUpdates = []
 
-      try {
-        const size = statSync(filePath).size
-        if (isSupportedCode(ext) && config.maxFileSizeMb && size > config.maxFileSizeMb * 1024 * 1024) {
-          store.removeFile(rel)
+  for (const filePath of files) {
+    const rel = storeKey(relativePath(root, filePath))
+    seen.add(rel)
+    const ext = path.extname(filePath).toLowerCase()
+    if (!isSupportedDoc(ext) && !isSupportedCode(ext)) continue
+
+    try {
+      const size = statSync(filePath).size
+        const existing = store.fileRecord(rel)
+      if (isSupportedCode(ext) && config.maxFileSizeMb && size > config.maxFileSizeMb * 1024 * 1024) {
+        fileUpdates.push({ rel, deleted: true })
+        skipped++
+        continue
+      }
+
+      // existing declared above before hash
+      const { hash } = await sha256OfFile(filePath)
+      if (!reindex && existing && existing.sha256 === hash) {
+        skipped++
+        continue
+      }
+
+      let entries
+      if (isSupportedCode(ext)) {
+        const content = readFileSync(filePath, 'utf8')
+        entries = scanSymbols(filePath, content)
+        fileUpdates.push({ rel, expectedHash: existing?.sha256, hash, size, entries, type: 'code' })
+        updated++
+      } else {
+        const content = readFileSync(filePath, 'utf8')
+        if (looksLikeDump(content)) {
+          fileUpdates.push({ rel, deleted: true })
           skipped++
           continue
         }
-
-        const existing = store.fileRecord(rel)
-        const { hash } = await sha256OfFile(filePath)
-        if (!reindex && existing && existing.sha256 === hash) {
+        entries = await buildDocEntries(ctx.llm, filePath, {
+          chunkChars: config.chunkChars,
+          maxChunks: config.maxChunksPerFile,
+          maxFileSizeMb: config.maxFileSizeMb,
+          maxPdfPages: config.maxPdfPages,
+        })
+        if (entries === null) {
+          fileUpdates.push({ rel, deleted: true })
           skipped++
           continue
         }
+        fileUpdates.push({ rel, expectedHash: existing?.sha256, hash, size, entries, type: 'doc' })
+        indexed++
+      }
+    } catch (err) {
+      failures.push(rel)
+    }
+  }
 
-        let entries
-        if (isSupportedCode(ext)) {
-          const content = readFileSync(filePath, 'utf8')
-          entries = scanSymbols(filePath, content)
-          store.markFile(rel, { sha256: hash, size, type: 'code', indexedAt: new Date().toISOString() })
-          updated++
-        } else {
-          const content = readFileSync(filePath, 'utf8')
-          if (looksLikeDump(content)) {
-            store.removeFile(rel)
-            skipped++
-            continue
-          }
-          entries = await buildDocEntries(ctx.llm, filePath, {
-            chunkChars: config.chunkChars,
-            maxChunks: config.maxChunksPerFile,
-            maxFileSizeMb: config.maxFileSizeMb,
-            maxPdfPages: config.maxPdfPages,
-          })
-          if (entries === null) {
-            store.removeFile(rel)
-            skipped++
-            continue
-          }
-          store.markFile(rel, { sha256: hash, size, type: 'doc', indexedAt: new Date().toISOString() })
-          indexed++
-        }
-        store.setEntries(rel, entries)
-      } catch (err) {
-        store.removeFile(rel)
-        failures.push(rel)
+  // Second pass: single commit with all updates
+  return store.commit((s) => {
+    for (const update of fileUpdates) {
+      const result = s.applyFileUpdate(update.rel, update)
+      if (result.skipped) {
+        // CAS failed - file was modified concurrently, skip
+        continue
       }
     }
 
-    for (const rel of Object.keys(store.files)) {
+    // Remove files not seen
+    for (const rel of Object.keys(s.files)) {
       if (!seen.has(rel)) {
-        store.removeFile(rel)
+        s.removeFile(rel)
         removed++
       }
     }
 
-    const links = linkEntries(store)
-    store.save()
-    const stats = store.stats()
+    linkEntries(s)
+    const stats = s.stats()
     let report =
       `Indexed project: ${root}\n` +
       `docs indexed: ${indexed}, code symbols updated: ${updated}, unchanged skipped: ${skipped}, removed: ${removed}\n` +
-      `memory store: ${stats.files} files, ${stats.entries} entries, ${stats.experience} experience notes` +
-      (links ? `, ${links} doc<->symbol links` : '')
+      `memory store: ${stats.files} files, ${stats.entries} entries, ${stats.experience} experience notes`
     if (failures.length) {
       report += `\nfailed to index ${failures.length} file(s): ${failures.join(', ')}`
     }

package/src/tools/remember.js

@@ -1,6 +1,6 @@
 import { defineTool } from '@deepseek-ai/dsh-tools'
 import { memoryRootFor, resolveIndexRoot } from '../util/fs.js'
-import { ProjectMemoryStore, withStoreLock } from '../store.js'
+import { ProjectMemoryStore } from '../store.js'
 
 export function rememberTool(config) {
   return defineTool({
@@ -36,14 +36,13 @@ export function rememberTool(config) {
     async execute(args, exec) {
       const root = resolveIndexRoot(exec, args.root)
       const memoryDir = memoryRootFor(root, config.memoryDir)
-      return withStoreLock(memoryDir, () => {
-        const store = new ProjectMemoryStore(memoryDir).load()
-        const result = store.addExperience({
+      const store = new ProjectMemoryStore(memoryDir).load()
+      return store.commit((s) => {
+        const result = s.addExperience({
           problem: args.problem,
           solution: args.solution,
           sourceFile: args.source_file,
         })
-        store.save()
         return result.superseded
           ? `Updated existing experience note (${result.id}).`
           : `Saved experience note (${result.id}).`

package/src/tools/watch-repo.js

@@ -1,7 +1,7 @@
 import { defineTool } from '@deepseek-ai/dsh-tools'
 import path from 'node:path'
 import { memoryRootFor, resolveIndexRoot } from '../util/fs.js'
-import { ProjectMemoryStore, withStoreLock } from '../store.js'
+import { ProjectMemoryStore } from '../store.js'
 
 export function watchRepoTool(watchManager, config) {
   return defineTool({
@@ -30,31 +30,27 @@ export function watchRepoTool(watchManager, config) {
       const memoryDir = memoryRootFor(root, config.memoryDir)
       const sessionRoot = resolveIndexRoot(exec)
       const sessionMemoryDir = memoryRootFor(sessionRoot, config.memoryDir)
-      return withStoreLock(memoryDir, async () => {
-        const store = new ProjectMemoryStore(memoryDir).load()
-        const mirrorSessionWatchlist = async (present) => {
-          if (path.resolve(sessionMemoryDir) === path.resolve(memoryDir)) return
-          await withStoreLock(sessionMemoryDir, () => {
-            const sessionStore = new ProjectMemoryStore(sessionMemoryDir).load()
-            const removed = sessionStore.removeWatch(root)
-            const added = present ? sessionStore.addWatch(root) : false
-            if (removed || added) sessionStore.save()
-          })
-        }
-        if (args.watch === false) {
-          watchManager.removeRoot(root)
-          store.removeWatch(root)
-          store.save()
-          await mirrorSessionWatchlist(false)
-          return `Stopped watching: ${root}`
-        }
-        store.addWatch(root)
-        store.save()
-        await mirrorSessionWatchlist(true)
-        watchManager.addRoot(root)
-        watchManager.start(config.watchInterval * 1000)
-        return `Watching ${root} (interval ${config.watchInterval}s). Docs/code changes will be re-indexed silently.`
-      })
+      const store = new ProjectMemoryStore(memoryDir).load()
+
+      const mirrorSessionWatchlist = async (present) => {
+        if (path.resolve(sessionMemoryDir) === path.resolve(memoryDir)) return
+        const sessionStore = new ProjectMemoryStore(sessionMemoryDir).load()
+        const removed = sessionStore.removeWatch(root)
+        const added = present ? sessionStore.addWatch(root) : false
+        if (removed || added) sessionStore.save()
+      }
+
+      if (args.watch === false) {
+        watchManager.removeRoot(root)
+        store.commit((s) => s.removeWatch(root))
+        await mirrorSessionWatchlist(false)
+        return `Stopped watching: ${root}`
+      }
+      store.commit((s) => s.addWatch(root))
+      await mirrorSessionWatchlist(true)
+      watchManager.addRoot(root)
+      watchManager.start(config.watchInterval * 1000)
+      return `Watching ${root} (interval ${config.watchInterval}s). Docs/code changes will be re-indexed silently.`
     },
   })
 }

package/src/util/search.js

@@ -26,6 +26,7 @@ function extractCjkPhrases(text) {
 }
 
 function expandQuery(query) {
+  if (!query) return { original: '', expanded: [], cjkPhrases: [] }
   const lower = query.toLowerCase()
   const expanded = new Set([lower])
   for (const [key, vals] of SYNONYMS) {
@@ -73,15 +74,16 @@ function avgDocLen(docs) {
   return docs.reduce((sum, d) => sum + d.length, 0) / docs.length
 }
 
-export function buildBm25(docs, getFieldText) {
+export function buildBm25(docs, getFieldText, getPhraseFields) {
   const documents = docs.map((doc) => {
     const text = getFieldText(doc)
     const terms = tokenizeRaw(text)
     const tf = {}
     for (const t of terms) tf[t] = (tf[t] || 0) + 1
-    const title = (doc.title || '').toLowerCase()
-    const keywords = (doc.keywords || []).join(' ').toLowerCase()
-    return { doc, length: terms.length, tf, title, keywords }
+    const rawFields = getPhraseFields ? getPhraseFields(doc) : [(doc.title || ''), (doc.keywords || []).join(' ')]
+    const phraseFields = Array.isArray(rawFields) ? rawFields : [rawFields]
+    const phraseText = phraseFields.filter(Boolean).join(' ').toLowerCase()
+    return { doc, length: terms.length, tf, phraseText }
   })
   const df = {}
   for (const d of documents) {
@@ -112,7 +114,7 @@ export function buildBm25(docs, getFieldText) {
           }
           for (const phrase of cjkPhrases) {
             const lowerPhrase = phrase.toLowerCase()
-            if (d.title.includes(lowerPhrase) || d.keywords.includes(lowerPhrase)) {
+            if (d.phraseText.includes(lowerPhrase)) {
               score *= 1.5
             }
           }
@@ -167,8 +169,10 @@ export function rankExperience(items, queryOrQueries, limit = 5) {
 }
 
 export function rankExperienceScored(items, queryOrQueries, limit = 5) {
-  const bm25 = buildBm25(items, (item) =>
-    `${item.problem} ${item.problem} ${item.problem} ${item.solution} ${item.sourceFile || ''}`,
+  const bm25 = buildBm25(
+    items,
+    (item) => `${item.problem} ${item.problem} ${item.problem} ${item.solution} ${item.sourceFile || ''}`,
+    (item) => [item.problem],
   )
   const queries = Array.isArray(queryOrQueries) ? queryOrQueries : [queryOrQueries]
   const merged = new Map()

package/src/watch.js

@@ -4,7 +4,7 @@ import { isSupportedCode, isSupportedDoc, memoryRootFor, relativePath, sha256OfF
 import { buildDocEntries } from './doc-pipeline.js'
 import { scanSymbols } from './symbols.js'
 import { linkEntries } from './link.js'
-import { ProjectMemoryStore, withStoreLock } from './store.js'
+import { ProjectMemoryStore } from './store.js'
 
 export class WatchManager {
   constructor(ctx, config) {
@@ -62,75 +62,95 @@ export class WatchManager {
   }
 
   async pollRoot(root, state) {
-    const memoryDir = memoryRootFor(root, this.config.memoryDir)
-    await withStoreLock(memoryDir, async () => {
-      state.store = new ProjectMemoryStore(memoryDir).load()
-      const files = walkDir(root)
-      const seen = new Set()
-      let changed = 0
-
-      for (const filePath of files) {
-        const rel = storeKey(relativePath(root, filePath))
-        seen.add(rel)
-        const ext = path.extname(filePath).toLowerCase()
-        if (!isSupportedDoc(ext) && !isSupportedCode(ext)) continue
-
-        let stats
-        try {
-          stats = statSync(filePath)
-        } catch {
-          continue
-        }
-        const sig = `${stats.mtimeMs}:${stats.size}`
-        if (state.snapshot[rel] === sig) continue
-        state.snapshot[rel] = sig
+    const files = walkDir(root)
+    const seen = new Set()
+    let changed = 0
 
-        if (isSupportedCode(ext) && this.config.maxFileSizeMb && stats.size > this.config.maxFileSizeMb * 1024 * 1024) {
-          continue
-        }
+    // First pass: collect all file info and compute hashes/entries (async work outside commit)
+    const fileUpdates = []
+
+    for (const filePath of files) {
+      const rel = storeKey(relativePath(root, filePath))
+      seen.add(rel)
+      const ext = path.extname(filePath).toLowerCase()
+      if (!isSupportedDoc(ext) && !isSupportedCode(ext)) continue
+
+      let stats
+      try {
+        stats = statSync(filePath)
+      } catch {
+        continue
+      }
+      const sig = `${stats.mtimeMs}:${stats.size}`
+      if (state.snapshot[rel] === sig) continue
+
+      if (isSupportedCode(ext) && this.config.maxFileSizeMb && stats.size > this.config.maxFileSizeMb * 1024 * 1024) {
+        continue
+      }
+
+      const { hash } = await sha256OfFile(filePath)
+      const existing = state.store.fileRecord(rel)
+      if (existing && existing.sha256 === hash) continue
 
-        const { hash } = await sha256OfFile(filePath)
-        const existing = state.store.fileRecord(rel)
-        if (existing && existing.sha256 === hash) continue
-
-        try {
-          let entries
-          if (isSupportedCode(ext)) {
-            entries = scanSymbols(filePath, readFileSync(filePath, 'utf8'))
-            state.store.markFile(rel, { sha256: hash, size: stats.size, type: 'code', indexedAt: new Date().toISOString() })
-          } else {
-            entries = await buildDocEntries(this.ctx.llm, filePath, {
-              chunkChars: this.config.chunkChars,
-              maxChunks: this.config.maxChunksPerFile,
-              maxFileSizeMb: this.config.maxFileSizeMb,
-              maxPdfPages: this.config.maxPdfPages,
-            })
-            if (entries === null) {
-              state.store.removeFile(rel)
-              changed++
-              continue
-            }
-            state.store.markFile(rel, { sha256: hash, size: stats.size, type: 'doc', indexedAt: new Date().toISOString() })
+      try {
+        let entries
+        if (isSupportedCode(ext)) {
+          entries = scanSymbols(filePath, readFileSync(filePath, 'utf8'))
+        } else {
+          entries = await buildDocEntries(this.ctx.llm, filePath, {
+            chunkChars: this.config.chunkChars,
+            maxChunks: this.config.maxChunksPerFile,
+            maxFileSizeMb: this.config.maxFileSizeMb,
+            maxPdfPages: this.config.maxPdfPages,
+          })
+          if (entries === null) {
+            // Dump file - update snapshot so we don't re-hash next poll, but don't index
+            fileUpdates.push({ rel, expectedHash: state.store.fileRecord(rel)?.sha256, deleted: true, _sig: sig })
+            changed++
+            continue
           }
-          state.store.setEntries(rel, entries)
-          changed++
-        } catch (err) {
-          delete state.snapshot[rel]
-          console.error(`[dsh-project-memory] re-index failed for ${rel}: ${err.message}`)
+        }
+        fileUpdates.push({ rel, expectedHash: state.store.fileRecord(rel)?.sha256, hash, size: stats.size, entries, type: isSupportedCode(ext) ? 'code' : 'doc', _sig: sig })
+        changed++
+      } catch (err) {
+        // Index failed - rollback snapshot so next poll retries
+        delete state.snapshot[rel]
+        console.error(`[dsh-project-memory] re-index failed for ${rel}: ${err.message}`)
+        continue
+      }
+    }
+
+    // Single commit with all updates
+    state.store.commit((s) => {
+      for (const update of fileUpdates) {
+        const result = s.applyFileUpdate(update.rel, update)
+        if (result.skipped) {
+          // CAS failed - file was modified concurrently, rollback snapshot to retry next poll
+          delete state.snapshot[update.rel]
+          // Mark this update to skip snapshot update after commit
+          update._skipSnapshot = true
         }
       }
 
-      for (const rel of Object.keys(state.store.files)) {
+      // Remove deleted files
+      for (const rel of Object.keys(s.files)) {
         if (!seen.has(rel)) {
-          state.store.removeFile(rel)
-          changed++
+          s.removeFile(rel)
         }
       }
 
       if (changed) {
-        linkEntries(state.store)
-        state.store.save()
+        linkEntries(s)
       }
     })
+
+    // Update snapshot for successfully processed files only, using the
+    // first-pass signature: if the file changed during the compute window
+    // (hash / scanSymbols / LLM summary), the next poll's signature differs
+    // and the file is re-indexed — restoring the pre-refactor self-healing.
+    for (const update of fileUpdates) {
+      if (update._skipSnapshot) continue
+      state.snapshot[update.rel] = update._sig
+    }
   }
 }