@yolk_vat-y/dsh-project-memory 0.3.0 → 0.3.1

package/CHANGELOG.md

@@ -1,6 +1,6 @@
 # Changelog
 
-## 0.2.1 (unreleased)
+## 0.3.0 (2026-08-29)
 
 ### 存储：无锁同步事务重构
 - 删除全部锁机制（`withStoreLock`、`dirLocks`、Promise 链锁）：写入统一走同步事务 `store.commit(fn)`，fn 成功后才原子落盘；单进程内天然串行，`remember`/`forget` 不再被 watch 重索引排队阻塞

package/README.md

@@ -2,7 +2,7 @@
 
 [English](README.md) | [简体中文](README.zh-CN.md)
 
-[![ci](https://github.com/00080000/dsh-project-memory/actions/workflows/ci.yml/badge.svg)](https://github.com/00080000/dsh-project-memory/actions/workflows/ci.yml) [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) [![npm](https://img.shields.io/npm/v/@yolk_vat-y/dsh-project-memory)](https://www.npmjs.com/package/@yolk_vat-y/dsh-project-memory) [![Listed on dsh-plugin.org](https://dsh-plugin.org/badges/listed.svg)](https://dsh-plugin.org/plugins/00080000/dsh-project-memory)
+[![ci](https://github.com/00080000/dsh-project-memory/actions/workflows/ci.yml/badge.svg)](https://github.com/00080000/dsh-project-memory/actions/workflows/ci.yml) [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) [![npm](https://img.shields.io/npm/v/@yolk_vat-y/dsh-project-memory)](https://www.npmjs.com/package/@yolk_vat-y/dsh-project-memory) [![Listed on dsh-plugin.org](https://dsh-plugin.org/badges/listed.svg)](https://dsh-plugin.org/plugins/00080000/dsh-project-memory) [![Awesome](https://awesome-dsh-plugin.com/badge.svg)](https://awesome-dsh-plugin.com)
 
 Persistent project memory for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)(dsh) agents. Indexes documents (PDF / Markdown / txt) and code symbols into a per-workspace store, refreshes them automatically, and recalls them with source citations — documents are cross-linked to the code symbols they reference.
 
@@ -18,6 +18,7 @@ Persistent project memory for [DeepSeek Harness](https://github.com/deepseek-ai/
 - **BM25 retrieval** — ranked search over documents, symbols, and experience notes, with optional LLM query expansion to handle vocabulary mismatch. **CJK-optimized**: precise phrase boost (3+ char phrases ×1.5 score on title/keywords match), synonym table (e.g. 数据库连接池 ↔ 连接池 ↔ DB pool), and CJK-aware word boundaries for doc↔symbol linking.
 - **Experience notes** — problems → solutions; similar problems supersede instead of duplicating, and notes are returned only when a search matches. The note store is bounded: capacity scales with project size (clamped to 100–2000), and the oldest notes are pruned when the limit is exceeded. **Supersede tightened to bidirectional 0.7 overlap** (was 0.6); **experience `problem` field now participates in CJK phrase boost** for long-tail query recall.
 - **Minimal dependencies** — pure JavaScript; the only runtime dependency is `pdfjs-dist` (PDF text extraction), no native builds required.
+- **Negligible overhead** — pure in-process operation; cold start <100 ms (5k files), typical project query median 2–3 ms (p99 < 7 ms); bottleneck is LLM summarization and PDF parsing, not the plugin.
 
 ## How it works
 
@@ -51,7 +52,7 @@ dsh plugin --profile web add @yolk_vat-y/dsh-project-memory -w
 A prebuilt tarball is published with each release, installable without a build step:
 
 ```bash
-dsh plugin --profile web add /path/to/dsh-project-memory-0.2.0.tgz
+dsh plugin --profile web add /path/to/dsh-project-memory-0.3.0.tgz
 ```
 
 Each indexed project has its own store at `<root>/.dsh-project-memory/`. Add it to `.gitignore` if it should not be committed.
@@ -95,7 +96,6 @@ These are deliberate scope choices.
 - **In-process locking** — store writes are serialized per memory directory within one dsh process; two dsh instances sharing a project store is last-writer-wins. A cross-process lock would need a resident daemon, which conflicts with the pure-JS, no-background-service positioning, so multi-instance writes are explicitly unsupported.
 - **Watch poll holds the lock** — while the watcher re-indexes changed docs (LLM summarization), `remember`/`forget` queue behind it. Polling (mtime + content hash) instead of `fs.watch` events keeps behavior consistent across platforms; overlapping polls serialize on the same lock: safe, but they can pile up on very large diffs. Tune via `watchInterval`.
 - **Corrupt files are quarantined** — a store JSON that fails to parse falls back to empty for that file and is rebuilt on the next write; the broken file is renamed to `*.corrupt` with an error logged, but its data cannot be recovered. Auto-repairing partial writes would need a write-ahead journal or an embedded database — out of proportion when quarantining one bad file costs nothing.
-- **Absolute source paths** — entries cite absolute paths; moving a project invalidates citations until the next re-index.
 - **`forget` by query is eager** — keyword deletion matches at ≥0.5 token overlap and may remove several notes at once; prefer deleting by id for precision.
 - **Cross-language recall depends on index time** — with `llmQueryExpansion` off, a Chinese-only query reaches English content through bilingual keywords captured when docs are indexed, plus doc↔symbol links; queries stay LLM-free. Stores indexed before v0.1.1 gain bilingual keywords as files change, or immediately via `index_repo` with `reindex: true`.
 - **CJK retrieval** — phrase boost and synonym expansion are purely query-side; they do not increase index size or LLM usage. Link boundaries use CJK-aware regex only; English symbols keep the original word-boundary behavior. The experience supersede threshold (0.7 bidirectional) is a conservative default; adjust via config if false positives/negatives appear in practice.

package/README.zh-CN.md

@@ -2,7 +2,7 @@
 
 [English](README.md) | [简体中文](README.zh-CN.md)
 
-[![ci](https://github.com/00080000/dsh-project-memory/actions/workflows/ci.yml/badge.svg)](https://github.com/00080000/dsh-project-memory/actions/workflows/ci.yml) [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) [![npm](https://img.shields.io/npm/v/@yolk_vat-y/dsh-project-memory)](https://www.npmjs.com/package/@yolk_vat-y/dsh-project-memory) [![Listed on dsh-plugin.org](https://dsh-plugin.org/badges/listed.svg)](https://dsh-plugin.org/plugins/00080000/dsh-project-memory)
+[![ci](https://github.com/00080000/dsh-project-memory/actions/workflows/ci.yml/badge.svg)](https://github.com/00080000/dsh-project-memory/actions/workflows/ci.yml) [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) [![npm](https://img.shields.io/npm/v/@yolk_vat-y/dsh-project-memory)](https://www.npmjs.com/package/@yolk_vat-y/dsh-project-memory) [![Listed on dsh-plugin.org](https://dsh-plugin.org/badges/listed.svg)](https://dsh-plugin.org/plugins/00080000/dsh-project-memory) [![Awesome](https://awesome-dsh-plugin.com/badge.svg)](https://awesome-dsh-plugin.com)
 
 为 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)（dsh）agent 提供持久化的项目记忆。将文档（PDF / Markdown / txt）与代码符号索引进每个工作区独立的存储库，自动维护更新，召回时附源文件引用——文档自动交叉链接到其提及的代码符号。
 
@@ -18,6 +18,7 @@
 - **BM25 检索** — 对文档、符号与经验笔记进行排序召回，可选 LLM 查询扩展以应对表述不一致。**CJK 增强**：精确短语乘法加分（3+ 字短语在标题/关键词命中 ×1.5）、同义词表（如 数据库连接池 ↔ 连接池 ↔ DB pool）、CJK 感知的文档↔符号链接边界。
 - **经验笔记** — 记录问题 → 方案；相似问题覆盖而非重复；笔记仅在检索命中时返回。笔记数量有界：容量随项目规模伸缩（钳制在 100–2000），超限时淘汰最旧的笔记。**覆盖阈值收紧为双向 0.7 重叠**（原 0.6）；**经验 `problem` 字段现参与 CJK 短语加分**，提升长尾问句召回。
 - **依赖极简** — 纯 JavaScript；唯一运行时依赖是 `pdfjs-dist`（PDF 文本提取），无需原生构建。
+- **开销可忽略** — 纯进程内操作；冷启动 <100 ms（5k 文件），典型项目查询中位数 2–3 ms（p99 < 7 ms）；瓶颈在 LLM 摘要与 PDF 解析，插件本身不阻塞。
 
 ## 工作原理
 
@@ -51,7 +52,7 @@ dsh plugin --profile web add @yolk_vat-y/dsh-project-memory -w
 每个版本会附带预构建 tarball，无需构建步骤即可安装：
 
 ```bash
-dsh plugin --profile web add /path/to/dsh-project-memory-0.2.0.tgz
+dsh plugin --profile web add /path/to/dsh-project-memory-0.3.0.tgz
 ```
 
 每个被索引的项目在 `<root>/.dsh-project-memory/` 下有独立存储。如无需入库，可加入 `.gitignore`。
@@ -95,7 +96,6 @@ v0.2.0 之前创建的库（单文件 `entries.json` / `index.json`）在首次
 - **无锁同步事务** — 不采用锁：所有写入（index / watch / remember / forget / watch_repo）统一走同步事务 `store.commit(fn)`，fn 成功后才一次落盘；JS 单线程事件循环保证事务间不交错，`remember`/`forget` 不会被 watch 重索引阻塞排队。两个 dsh 实例共享同一项目存储时后写覆盖先写；跨进程协调需要常驻守护进程，违背纯 JS 插件、无后台服务的定位，故明确不支持多实例共写。
 - **watch 计算与提交分离** — watcher 在事务外完成 mtime/哈希/符号扫描/LLM 摘要等重活，再以单次 `commit` 原子应用全部变更；`applyFileUpdate` 用 CAS 校验（统一 null 处理、删除跳过对比）防并发修改，失败回滚 snapshot 下轮重试，索引失败删除 snapshot 自动重试。轮询（mtime + 内容哈希）而非 `fs.watch` 事件驱动，是为了跨平台行为一致。间隔可用 `watchInterval` 调整。
 - **损坏隔离重建** — 存储 JSON 损坏时该文件回落为空并在下次写入时重建；坏文件会改名备份为 `*.corrupt` 并输出错误日志，但该文件内的数据无法恢复。自动修复半写文件需要预写日志或嵌入式数据库，代价与收益不成比例——而隔离一个坏文件的成本几乎为零。
-- **绝对路径引用** — 条目引用绝对路径；项目搬家后引用失效，重建索引即恢复。
 - **`forget` 按关键词删除偏激进** — 关键词删除按 ≥0.5 token 重叠匹配，可能一次删掉多条；追求精确请用 id 删除。
 - **跨语种召回依赖索引时** — `llmQueryExpansion` 关闭时，纯中文查询靠索引时捕获的双语 keywords 和 doc↔symbol 链接触达英文内容，查询侧保持零 LLM 调用。v0.1.1 之前建立的索引随文件变更逐步获得双语关键词，或用 `index_repo` 的 `reindex: true` 立即重建。
 - **CJK 检索** — 短语加分与同义词展开完全在查询侧，不增加索引体积、不额外消耗 LLM token。链接边界仅对 CJK 使用正则边界，英文符号保持原有词边界行为。经验层 supersede 阈值（双向 0.7）为保守默认；若实测出现误覆盖/误漏报，可通过配置调整。

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@yolk_vat-y/dsh-project-memory",
-  "version": "0.3.0",
+  "version": "0.3.1",
   "description": "Persistent project memory for dsh agents: index docs (PDF/Markdown/text) and code symbols into a searchable per-workspace store, recall them with cited sources, and keep experience entries (problems -> solutions) searchable on demand.",
   "type": "module",
   "main": "src/index.js",

package/src/doc-pipeline.js

@@ -22,10 +22,12 @@ export async function extractTextFromFile(filePath, { maxFileSizeMb = 50, maxPdf
 
 const DOC_CONCURRENCY = 4
 
-export async function buildDocEntries(llm, filePath, { chunkChars = 3000, maxChunks = 40, maxFileSizeMb = 50, maxPdfPages = 1000 } = {}) {
-  const text = await extractTextFromFile(filePath, { maxFileSizeMb, maxPdfPages })
+export async function buildDocEntries(llm, a, b, c) {
+  // Backward compatible: old signature (llm, filePath, opts) or new (llm, relPath, filePath, opts)
+  const [relPath, filePath, opts] = c === undefined ? [a, a, b] : [a, b, c]
+  const text = await extractTextFromFile(filePath, opts)
   if (looksLikeDump(text)) return null
-  const chunks = chunkText(text, chunkChars, maxChunks)
+  const chunks = chunkText(text, opts.chunkChars, opts.maxChunks)
   const metas = new Array(chunks.length)
   let cursor = 0
   await Promise.all(
@@ -39,8 +41,8 @@ export async function buildDocEntries(llm, filePath, { chunkChars = 3000, maxChu
     ),
   )
   return metas.map((meta, i) => ({
-    id: `${relativeId(filePath)}#${i}`,
-    sourcePath: filePath,
+    id: `${relativeId(relPath)}#${i}`,
+    sourcePath: relPath,
     sourceLine: chunks[i].line,
     type: 'doc',
     title: meta.title,

package/src/lazy.js

@@ -104,7 +104,7 @@ export async function indexFile(ctx, config, filePath, watchManager = null) {
 
   let entries
   if (isSupportedCode(ext)) {
-    entries = scanSymbols(filePath, readFileSync(filePath, 'utf8'))
+    entries = scanSymbols(rel, filePath, readFileSync(filePath, 'utf8'))
     return store.commit((s) => {
       s.markFile(rel, { sha256: hash, size, type: 'code', indexedAt: new Date().toISOString() })
       s.setEntries(rel, entries)
@@ -112,7 +112,7 @@ export async function indexFile(ctx, config, filePath, watchManager = null) {
       return true
     })
   } else {
-    entries = await buildDocEntries(ctx.llm, filePath, {
+    entries = await buildDocEntries(ctx.llm, rel, filePath, {
       chunkChars: config.chunkChars,
       maxChunks: config.maxChunksPerFile,
       maxFileSizeMb: config.maxFileSizeMb,

package/src/symbols.js

@@ -120,7 +120,7 @@ const JS_NON_METHOD = new Set([
   'new', 'delete', 'typeof', 'instanceof', 'void', 'await', 'yield', 'with',
 ])
 
-function scanJsLike(masked, filePath, rawLines) {
+function scanJsLike(masked, relPath, rawLines) {
   const symbols = []
   let prevOpensBlock = false
   for (let i = 0; i < masked.length; i++) {
@@ -148,13 +148,13 @@ function scanJsLike(masked, filePath, rawLines) {
         if (/[{};]/.test(tail)) break
       }
     }
-    if (matched) symbols.push(buildSymbol(matched, filePath, rawLines[i], i + 1))
+    if (matched) symbols.push(buildSymbol(matched, relPath, rawLines[i], i + 1))
     prevOpensBlock = masked[i].trim().endsWith('{')
   }
   return symbols
 }
 
-function scanPython(masked, filePath, rawLines) {
+function scanPython(masked, relPath, rawLines) {
   const symbols = []
   let depth = 0
   for (let i = 0; i < masked.length; i++) {
@@ -169,7 +169,7 @@ function scanPython(masked, filePath, rawLines) {
     const cls = fn ? null : text.match(/^class\s+(\w+)\s*[(:]/)
     if (fn) matched = { name: fn[1], kind: 'function' }
     else if (cls) matched = { name: cls[1], kind: 'class' }
-    if (matched) symbols.push(buildSymbol(matched, filePath, rawLines[i], i + 1))
+    if (matched) symbols.push(buildSymbol(matched, relPath, rawLines[i], i + 1))
     depth += delta
   }
   return symbols
@@ -208,24 +208,26 @@ function matchShell(line) {
   return m ? { name: m[1], kind: 'function' } : null
 }
 
-function buildSymbol(matched, filePath, rawLine, lineNo) {
+function buildSymbol(matched, relPath, rawLine, lineNo) {
   return {
-    id: `${String(filePath).replace(/[\\/:\s]/g, '_')}#${lineNo}`,
-    sourcePath: filePath,
+    id: `${String(relPath).replace(/[\\/:\s]/g, '_')}#${lineNo}`,
+    sourcePath: relPath,
     sourceLine: lineNo,
     type: 'symbol',
     title: `${matched.name} (${matched.kind})`,
-    summary: `${matched.kind} "${matched.name}" declared at ${filePath}:${lineNo}`,
+    summary: `${matched.kind} "${matched.name}" declared at ${relPath}:${lineNo}`,
     keywords: [matched.name, matched.kind],
     text: String(rawLine).trim().slice(0, 200),
   }
 }
 
-export function scanSymbols(filePath, content) {
-  const ext = filePath.slice(filePath.lastIndexOf('.'))
+export function scanSymbols(a, b, c) {
+  // Backward compatible: old signature (filePath, content) or new (relPath, filePath, content)
+  const [relPath, filePath, content] = c === undefined ? [a, a, b] : [a, b, c]
+  const ext = relPath.slice(relPath.lastIndexOf('.'))
   const lines = content.split(/\r?\n/)
-  if (JS_LIKE.has(ext)) return scanJsLike(maskTokens(lines, JS_MASKER), filePath, lines)
-  if (PYTHON.has(ext)) return scanPython(maskTokens(lines, PY_MASKER), filePath, lines)
+  if (JS_LIKE.has(ext)) return scanJsLike(maskTokens(lines, JS_MASKER), relPath, lines)
+  if (PYTHON.has(ext)) return scanPython(maskTokens(lines, PY_MASKER), relPath, lines)
 
   let masker = null
   if (GO.has(ext)) masker = GO_MASKER
@@ -247,7 +249,7 @@ export function scanSymbols(filePath, content) {
     else if (/^(?:def|func|fn|function)\s+(\w+)/.test(line)) {
       matched = { name: line.match(/^(?:def|func|fn|function)\s+(\w+)/)[1], kind: 'function' }
     }
-    if (matched) symbols.push(buildSymbol(matched, filePath, raw, i + 1))
+    if (matched) symbols.push(buildSymbol(matched, relPath, raw, i + 1))
   }
   return symbols
 }

package/src/tools/index-doc.js

@@ -42,7 +42,7 @@ export function indexDocTool(ctx, config) {
         return `Skipped (unchanged): ${rel}\nAlready indexed with ${(store.entries[rel] || []).length} entry/entries.`
       }
 
-      const entries = await buildDocEntries(ctx.llm, filePath, {
+      const entries = await buildDocEntries(ctx.llm, rel, filePath, {
         chunkChars: config.chunkChars,
         maxChunks: config.maxChunksPerFile,
         maxFileSizeMb: config.maxFileSizeMb,

package/src/tools/index-repo.js

@@ -47,7 +47,7 @@ export async function indexRepository(ctx, config, root, { reindex = false } = {
       let entries
       if (isSupportedCode(ext)) {
         const content = readFileSync(filePath, 'utf8')
-        entries = scanSymbols(filePath, content)
+        entries = scanSymbols(rel, filePath, content)
         fileUpdates.push({ rel, expectedHash: existing?.sha256, hash, size, entries, type: 'code' })
         updated++
       } else {
@@ -57,7 +57,7 @@ export async function indexRepository(ctx, config, root, { reindex = false } = {
           skipped++
           continue
         }
-        entries = await buildDocEntries(ctx.llm, filePath, {
+        entries = await buildDocEntries(ctx.llm, rel, filePath, {
           chunkChars: config.chunkChars,
           maxChunks: config.maxChunksPerFile,
           maxFileSizeMb: config.maxFileSizeMb,

package/src/tools/query-memory.js

@@ -1,10 +1,15 @@
 import { defineTool } from '@deepseek-ai/dsh-tools'
+import path from 'node:path'
 import { memoryRootFor, resolveIndexRoot } from '../util/fs.js'
 import { ProjectMemoryStore, storeOverview } from '../store.js'
 import { expandQuery } from '../llm.js'
 import { rankEntriesMergedScored, rankExperienceScored } from '../util/search.js'
 import { truncate } from '../util/text.js'
 
+function toAbs(root, rel) {
+  return path.isAbsolute(rel) ? rel : path.join(root, rel)
+}
+
 export function queryMemoryTool(ctx, config) {
   return defineTool({
     name: 'query_memory',
@@ -59,13 +64,13 @@ export function queryMemoryTool(ctx, config) {
           const top = scored[0].score || 1
           lines.push(`## Memory (${type === 'all' ? 'docs + symbols' : type})`)
           for (const { entry: e, score } of scored) {
-            const source = e.sourceLine ? `${e.sourcePath}:${e.sourceLine}` : e.sourcePath
+            const absSource = e.sourceLine ? `${toAbs(root, e.sourcePath)}:${e.sourceLine}` : toAbs(root, e.sourcePath)
             const rel = Math.round((score / top) * 100)
-            lines.push(`### ${e.title} (score: ${rel})\n- source: ${source}\n- ${e.summary}`)
+            lines.push(`### ${e.title} (score: ${rel})\n- source: ${absSource}\n- ${e.summary}`)
             if (e.type === 'doc' && Array.isArray(e.linkedSymbols) && e.linkedSymbols.length) {
               const refs = e.linkedSymbols.slice(0, 5).map((id) => {
                 const s = symbolById.get(id)
-                return s ? `${s.title} @ ${s.sourcePath}:${s.sourceLine}` : id
+                return s ? `${s.title} @ ${toAbs(root, s.sourcePath)}:${s.sourceLine}` : id
               })
               lines.push(`- references: ${refs.join('; ')}`)
             }
@@ -78,7 +83,7 @@ export function queryMemoryTool(ctx, config) {
           const expTop = scoredExp[0].score || 1
           lines.push(`## Experience (past problems -> solutions)`)
           for (const { item: e, score } of scoredExp) {
-            const source = e.sourceFile ? ` (source: ${e.sourceFile})` : ''
+            const source = e.sourceFile ? ` (source: ${toAbs(root, e.sourceFile)})` : ''
             lines.push(
               `### Problem: ${e.problem} (score: ${Math.round((score / expTop) * 100)}, id: ${e.id})\n- solution: ${e.solution}${source}\n- updated: ${e.updatedAt}`,
             )

package/src/watch.js

@@ -95,9 +95,9 @@ export class WatchManager {
       try {
         let entries
         if (isSupportedCode(ext)) {
-          entries = scanSymbols(filePath, readFileSync(filePath, 'utf8'))
+          entries = scanSymbols(rel, filePath, readFileSync(filePath, 'utf8'))
         } else {
-          entries = await buildDocEntries(this.ctx.llm, filePath, {
+          entries = await buildDocEntries(this.ctx.llm, rel, filePath, {
             chunkChars: this.config.chunkChars,
             maxChunks: this.config.maxChunksPerFile,
             maxFileSizeMb: this.config.maxFileSizeMb,