@yolk_vat-y/dsh-project-memory 0.3.0 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +33 -1
- package/README.md +22 -16
- package/README.zh-CN.md +21 -15
- package/package.json +11 -6
- package/src/doc-pipeline.js +14 -5
- package/src/enhancer.js +444 -0
- package/src/index.js +8 -0
- package/src/lazy.js +5 -3
- package/src/llm.js +12 -5
- package/src/symbols.js +241 -30
- package/src/tools/index-doc.js +1 -1
- package/src/tools/index-repo.js +16 -3
- package/src/tools/query-memory.js +9 -4
- package/src/util/fs.js +1 -1
- package/src/watch.js +10 -6
package/CHANGELOG.md
CHANGED
|
@@ -1,6 +1,38 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
-
## 0.2
|
|
3
|
+
## 0.3.2 (2026-08-31)
|
|
4
|
+
|
|
5
|
+
### TS Compiler API 增强器 (Phase 2 L2/L3)
|
|
6
|
+
- **L2 语义增强层**:用户项目安装 `typescript`(`npm i -D typescript`)后,插件自动激活,利用 TS Compiler API 推导返回类型、实例化泛型、提取接口与类型别名、丰富箭头函数签名
|
|
7
|
+
- **L3 磁盘缓存层**:增强结果按文件内容 SHA256 哈希缓存至 `.dsh-project-memory/type-cache/<hash>.json`,冷启动毫秒级复用,跨会话持久化
|
|
8
|
+
- **三级优先级队列**:P0 ACTIVE(`fs/observed` 读文件瞬间)> P1 RECENT(`watch` 变更后)> P2 BATCH(`index_repo` 批量)> P3 BACKLOG(启动补全历史),`setImmediate` 每任务后让出事件循环
|
|
9
|
+
- **零配置、零感知**:装 TS 重启 dsh 即可;无 TS 或 `enableTypeScript: false` 时优雅回退 L1 正则;TS 6.x/7.x 检测到警告并回退 L1
|
|
10
|
+
- **解析策略**:单文件 `createProgram`(快 10x),`createRequire(import.meta.url)` 兼容 ESM,解析优先级:配置 `tsPath` → 项目 cwd 向上 `node_modules/typescript` → 插件自身 `node_modules`
|
|
11
|
+
- **新增配置**:`tsPath`(可选指定 TS 路径)、`enableTypeScript`(默认 true,设 false 彻底禁用)
|
|
12
|
+
- **扩展名支持**:新增 `.mts` `.cts`
|
|
13
|
+
- **文档同步**:README.md / README.zh-CN.md 新增功能介绍与配置表
|
|
14
|
+
- **依赖升级**:cordis 4.0.2、schemastery 3.18.2、dsh-tools/llm 0.1.2-alpha.2
|
|
15
|
+
|
|
16
|
+
## Unreleased
|
|
17
|
+
|
|
18
|
+
### 符号层:只存身份牌,不存行为
|
|
19
|
+
- `src/symbols.js` `buildSymbol`:删除 `summary` 废话字段;`text` 不再截断,保存完整声明行(含签名);返回一行身份牌 `fn(a: A, b: B): R — file.ts:42`;删除冗余 `sig` 字段
|
|
20
|
+
|
|
21
|
+
### 文档层:答案级摘要 + 自报盲区
|
|
22
|
+
- `src/llm.js` `extractDocEntry`:新增 `blindSpots` 字段(自报盲区,如 `// 未覆盖:部署细节、性能基准、v0.2 前 API`);Prompt 要求 LLM 返回 `blindSpots`;摘要通过 `summarizeText()` 截断至 300 字符;`blindSpots` 追加在摘要末尾 `// 未覆盖:...`
|
|
23
|
+
- `src/doc-pipeline.js` `buildDocEntries`:新增 `hash` 字段(SHA256 内容哈希,用于更新检测);新增 `blindSpots` 字段存入分片
|
|
24
|
+
|
|
25
|
+
## 0.3.1 (2026-08-30)
|
|
26
|
+
|
|
27
|
+
### 存储:相对路径存储
|
|
28
|
+
- Entry 存相对路径(如 `src/main.js`),查询时按项目根解析绝对路径;项目搬家仅在根目录变更时需重新索引,兼容旧绝对路径 entry
|
|
29
|
+
- 所有写入路径(`index_doc` / `index_repo` / `watch` / `lazy`)统一传相对路径给构建函数
|
|
30
|
+
- `scanSymbols` / `buildDocEntries` 兼容旧签名(绝对路径 entry 仍可读)
|
|
31
|
+
|
|
32
|
+
### 设计取舍同步
|
|
33
|
+
- 移除「绝对路径引用」项,该限制已由相对路径存储方案解决
|
|
34
|
+
|
|
35
|
+
## 0.3.0 (2026-08-29)
|
|
4
36
|
|
|
5
37
|
### 存储:无锁同步事务重构
|
|
6
38
|
- 删除全部锁机制(`withStoreLock`、`dirLocks`、Promise 链锁):写入统一走同步事务 `store.commit(fn)`,fn 成功后才原子落盘;单进程内天然串行,`remember`/`forget` 不再被 watch 重索引排队阻塞
|
package/README.md
CHANGED
|
@@ -2,33 +2,36 @@
|
|
|
2
2
|
|
|
3
3
|
[English](README.md) | [简体中文](README.zh-CN.md)
|
|
4
4
|
|
|
5
|
-
[](https://github.com/00080000/dsh-project-memory/actions/workflows/ci.yml) [](LICENSE) [](https://www.npmjs.com/package/@yolk_vat-y/dsh-project-memory) [](https://dsh-plugin.org/plugins/00080000/dsh-project-memory)
|
|
5
|
+
[](https://github.com/00080000/dsh-project-memory/actions/workflows/ci.yml) [](LICENSE) [](https://www.npmjs.com/package/@yolk_vat-y/dsh-project-memory) [](https://dsh-plugin.org/plugins/00080000/dsh-project-memory) [](https://awesome-dsh-plugin.com)
|
|
6
6
|
|
|
7
|
-
Persistent project memory for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)(dsh) agents.
|
|
7
|
+
Persistent project memory for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)(dsh) agents. **Memorizes** documents (PDF / Markdown / txt) and code symbols into a per-workspace store, refreshes them automatically, and **recalls** them with source citations — documents are cross-linked to the code symbols they reference.
|
|
8
8
|
|
|
9
|
-
> The plugin keeps a compact project
|
|
9
|
+
> The plugin keeps a compact project **memory** on disk, with every entry pointing to a concrete file and line — the agent can reorient quickly instead of re-reading the whole project.
|
|
10
10
|
|
|
11
11
|
## Features
|
|
12
12
|
|
|
13
|
-
- **Document
|
|
14
|
-
- **Code symbol
|
|
15
|
-
- **
|
|
16
|
-
- **
|
|
13
|
+
- **Document memorization** — PDF, Markdown, and plain text files are chunked and summarized by the LLM; each entry carries a `path:line` citation back to the source.
|
|
14
|
+
- **Code symbol memory** — function, class, and method names with full type signatures (generics, parameters, return types, overloads) are extracted by a dependency-free source scanner (string/comment masking, multi-line signature joining, indentation-aware Python, class-method context), without LLM token usage.
|
|
15
|
+
- **Optional TypeScript semantic enhancement** — when `typescript` is installed in the user project (`npm i -D typescript`), the plugin automatically activates a second layer (L2) that uses the TS Compiler API to infer return types, resolve generics, extract interfaces and type aliases, and enrich arrow functions — all asynchronously in a priority queue (P0 on `fs/observed`, P1 on `watch`, P2 on `index_repo`). Results are cached on disk keyed by file content hash for instant cold-start reuse. Zero config: just install TS and restart dsh. Fully optional; if TS is absent or disabled via `enableTypeScript: false`, the plugin falls back to L1 regex-only extraction.
|
|
16
|
+
- **Automatic refresh** — a background poll (`watch_repo`) detects new or changed files by content hash and re-memorizes only those.
|
|
17
|
+
- **Read-time memorization** — files are memorized the moment the model actually reads them (`fs/observed`), so the memory is a byproduct of normal work, not a separate upfront scan. Files that are never read are never indexed. The project root is detected by markers (`.git`, `package.json`, …), a README plus source directories, or the file's own directory as a last resort.
|
|
17
18
|
- **Doc ↔ code cross-linking** — when a document mentions a symbol, the match is recorded as a `reference`; querying a symbol also surfaces the documents that describe it.
|
|
18
|
-
- **BM25
|
|
19
|
+
- **BM25 memory recall** — ranked search over documents, symbols, and experience notes, with optional LLM query expansion to handle vocabulary mismatch. **CJK-optimized**: precise phrase boost (3+ char phrases ×1.5 score on title/keywords match), synonym table (e.g. 数据库连接池 ↔ 连接池 ↔ DB pool), and CJK-aware word boundaries for doc↔symbol linking.
|
|
19
20
|
- **Experience notes** — problems → solutions; similar problems supersede instead of duplicating, and notes are returned only when a search matches. The note store is bounded: capacity scales with project size (clamped to 100–2000), and the oldest notes are pruned when the limit is exceeded. **Supersede tightened to bidirectional 0.7 overlap** (was 0.6); **experience `problem` field now participates in CJK phrase boost** for long-tail query recall.
|
|
21
|
+
- **Lock-free sync transactions** — all writes (index / watch / remember / forget / watch_repo) go through synchronous transactions `store.commit(fn)`; fn succeeds then atomic write; JS single-threaded event loop guarantees no interleaving; `remember`/`forget` never blocked by watch re-indexing.
|
|
20
22
|
- **Minimal dependencies** — pure JavaScript; the only runtime dependency is `pdfjs-dist` (PDF text extraction), no native builds required.
|
|
23
|
+
- **Negligible overhead** — pure in-process operation; cold start <100 ms (5k files), typical project query median 2–3 ms (p99 < 7 ms); bottleneck is LLM summarization and PDF parsing, not the plugin.
|
|
21
24
|
|
|
22
25
|
## How it works
|
|
23
26
|
|
|
24
27
|
The design follows four principles:
|
|
25
28
|
|
|
26
29
|
- **Volatility** — context is ephemeral; it is lost when a session is compacted.
|
|
27
|
-
- **Persistence** — the
|
|
28
|
-
- **Compactness** — only summaries are stored; the
|
|
29
|
-
- **Verifiability** —
|
|
30
|
+
- **Persistence** — the **memory** is stored on disk and survives compaction and new sessions.
|
|
31
|
+
- **Compactness** — only summaries are stored; the **memory** runs around 0.5% the size of the source it covers (8.8 MB of source → 49 KB of index in the example project), so **recall** replaces re-reading the full file.
|
|
32
|
+
- **Verifiability** — **recalls** carry a `path:line` citation where applicable, so the agent can confirm details against the source.
|
|
30
33
|
|
|
31
|
-
Building the
|
|
34
|
+
Building the **memory** does not require an upfront scan: files are memorized as the model reads them, so the **memory** grows to cover exactly what has been worked with. Re-reading a file that has not changed is a no-op (content hash), so the **memory** stays fresh with minimal ongoing overhead.
|
|
32
35
|
|
|
33
36
|
The store is per-project and follows the codebase: changed files are re-extracted by content hash, deleted files are removed. Experience notes are retrieval-only, so accumulation does not affect context.
|
|
34
37
|
|
|
@@ -51,7 +54,7 @@ dsh plugin --profile web add @yolk_vat-y/dsh-project-memory -w
|
|
|
51
54
|
A prebuilt tarball is published with each release, installable without a build step:
|
|
52
55
|
|
|
53
56
|
```bash
|
|
54
|
-
dsh plugin --profile web add /path/to/dsh-project-memory
|
|
57
|
+
dsh plugin --profile web add /path/to/dsh-project-memory.tgz
|
|
55
58
|
```
|
|
56
59
|
|
|
57
60
|
Each indexed project has its own store at `<root>/.dsh-project-memory/`. Add it to `.gitignore` if it should not be committed.
|
|
@@ -92,10 +95,9 @@ Stores created before v0.2.0 (single `entries.json` / `index.json`) migrate auto
|
|
|
92
95
|
|
|
93
96
|
These are deliberate scope choices.
|
|
94
97
|
|
|
95
|
-
- **
|
|
96
|
-
- **Watch
|
|
98
|
+
- **Lock-free sync transactions** — all writes (index / watch / remember / forget / watch_repo) go through synchronous transactions `store.commit(fn)`; fn succeeds then atomic write; JS single-threaded event loop guarantees no interleaving; `remember`/`forget` never blocked by watch re-indexing. Multi-instance concurrent writes to the same project store are naturally idempotent via CAS + atomic commits. We don't fear multi-process DSH because DSH is built on Cordis, and Cordis's single-process architecture is foundational — changing it would be a breaking change for the entire ecosystem.
|
|
99
|
+
- **Watch compute and commit separated** — watcher does heavy work (mtime/hash/symbol scan/LLM summary) outside transaction, then single `commit` applies atomically; `applyFileUpdate` uses CAS verification (unified null handling, delete skips hash compare) to prevent concurrent modification; failures rollback snapshot for next retry; indexing failures delete snapshot for auto-retry. Polling (mtime + content hash) instead of `fs.watch` events keeps behavior consistent across platforms.
|
|
97
100
|
- **Corrupt files are quarantined** — a store JSON that fails to parse falls back to empty for that file and is rebuilt on the next write; the broken file is renamed to `*.corrupt` with an error logged, but its data cannot be recovered. Auto-repairing partial writes would need a write-ahead journal or an embedded database — out of proportion when quarantining one bad file costs nothing.
|
|
98
|
-
- **Absolute source paths** — entries cite absolute paths; moving a project invalidates citations until the next re-index.
|
|
99
101
|
- **`forget` by query is eager** — keyword deletion matches at ≥0.5 token overlap and may remove several notes at once; prefer deleting by id for precision.
|
|
100
102
|
- **Cross-language recall depends on index time** — with `llmQueryExpansion` off, a Chinese-only query reaches English content through bilingual keywords captured when docs are indexed, plus doc↔symbol links; queries stay LLM-free. Stores indexed before v0.1.1 gain bilingual keywords as files change, or immediately via `index_repo` with `reindex: true`.
|
|
101
103
|
- **CJK retrieval** — phrase boost and synonym expansion are purely query-side; they do not increase index size or LLM usage. Link boundaries use CJK-aware regex only; English symbols keep the original word-boundary behavior. The experience supersede threshold (0.7 bidirectional) is a conservative default; adjust via config if false positives/negatives appear in practice.
|
|
@@ -116,6 +118,8 @@ These are deliberate scope choices.
|
|
|
116
118
|
| `autoIndexOnFirstUse` | false | full scan of the current working directory on plugin load (opt-in) |
|
|
117
119
|
| `watch` | true | enable the background refresh |
|
|
118
120
|
| `watchInterval` | 15 | poll interval (seconds) |
|
|
121
|
+
| `tsPath` | (auto) | optional absolute path to a specific `typescript` install; if omitted, resolves from project cwd → plugin node_modules |
|
|
122
|
+
| `enableTypeScript` | true | set `false` to disable L2 TS enhancement entirely (L1 regex only) |
|
|
119
123
|
|
|
120
124
|
### Toggling features
|
|
121
125
|
|
|
@@ -131,6 +135,8 @@ Settings live in the plugin's config object. To change them, add an override ent
|
|
|
131
135
|
llmQueryExpansion: false # off: do not spend tokens on LLM query expansion (default)
|
|
132
136
|
watch: true # on: background refresh for watched roots (default)
|
|
133
137
|
watchInterval: 15 # poll interval in seconds
|
|
138
|
+
enableTypeScript: true # on: L2 TS enhancement when TS is installed (default)
|
|
139
|
+
# tsPath: /custom/path/to/typescript # optional: force specific TS install
|
|
134
140
|
```
|
|
135
141
|
|
|
136
142
|
Only list the keys you want to change; the rest fall back to the plugin defaults. Verify the result with `dsh --profile web --dump-config`.
|
package/README.zh-CN.md
CHANGED
|
@@ -2,33 +2,36 @@
|
|
|
2
2
|
|
|
3
3
|
[English](README.md) | [简体中文](README.zh-CN.md)
|
|
4
4
|
|
|
5
|
-
[](https://github.com/00080000/dsh-project-memory/actions/workflows/ci.yml) [](LICENSE) [](https://www.npmjs.com/package/@yolk_vat-y/dsh-project-memory) [](https://dsh-plugin.org/plugins/00080000/dsh-project-memory)
|
|
5
|
+
[](https://github.com/00080000/dsh-project-memory/actions/workflows/ci.yml) [](LICENSE) [](https://www.npmjs.com/package/@yolk_vat-y/dsh-project-memory) [](https://dsh-plugin.org/plugins/00080000/dsh-project-memory) [](https://awesome-dsh-plugin.com)
|
|
6
6
|
|
|
7
|
-
为 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)(dsh)agent
|
|
7
|
+
为 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)(dsh)agent 提供持久化的项目记忆。**记住**文档(PDF / Markdown / txt)与代码符号,写入每个工作区独立的存储库,自动维护更新,**回忆**时附源文件引用——文档自动交叉链接到其提及的代码符号。
|
|
8
8
|
|
|
9
|
-
>
|
|
9
|
+
> 插件在磁盘上维护一份精简的项目**记忆**,每条记录指向具体的文件与行号;agent 需要快速了解项目时先查**记忆**,无需重读整个项目。
|
|
10
10
|
|
|
11
11
|
## 特性
|
|
12
12
|
|
|
13
|
-
-
|
|
14
|
-
-
|
|
15
|
-
-
|
|
16
|
-
-
|
|
13
|
+
- **文档记忆** — PDF、Markdown、纯文本按块切分并由 LLM 生成摘要,每条记忆携带 `路径:行号` 引用回源文件。
|
|
14
|
+
- **代码符号记忆** — 通过零依赖的源码扫描器提取函数、类与方法名及完整类型签名(泛型、参数类型、返回类型、重载签名),包含字符串/注释掩码、多行签名续行、Python 缩进感知、类方法上下文,不使用 LLM token。
|
|
15
|
+
- **可选 TypeScript 语义增强** — 当用户项目安装了 `typescript`(`npm i -D typescript`),插件自动激活第二层(L2),利用 TS Compiler API 推导返回类型、实例化泛型、提取接口与类型别名、丰富箭头函数签名 —— 全部在优先级队列中异步后台处理(P0:`fs/observed` 读文件瞬间、P1:`watch` 变更后、P2:`index_repo` 批量索引)。结果按文件内容哈希缓存到磁盘,冷启动毫秒级复用。零配置:装 TS 再重启 dsh 即可。完全可选;若无 TS 或设置 `enableTypeScript: false`,回退至 L1 正则提取。
|
|
16
|
+
- **自动刷新** — `watch_repo` 后台轮询,按内容哈希识别新增或变更文件,仅重记这些文件。
|
|
17
|
+
- **读到即记忆** — 文件在模型**实际读取的瞬间**被记忆(监听 `fs/observed`),记忆是正常工作的副产品,而非额外的一次全量扫描。从未读过的文件不会被记忆。项目根通过标记(`.git`、`package.json` 等)、README 加源码目录、或兜底到文件所在目录逐级识别。
|
|
17
18
|
- **文档 ↔ 代码交叉链接** — 文档提及某符号时记录为 `reference`;查询符号时同时带出描述该符号的文档。
|
|
18
|
-
- **BM25
|
|
19
|
+
- **BM25 记忆召回** — 对文档、符号与经验笔记进行排序召回,可选 LLM 查询扩展以应对表述不一致。**CJK 增强**:精确短语乘法加分(3+ 字短语在标题/关键词命中 ×1.5)、同义词表(如 数据库连接池 ↔ 连接池 ↔ DB pool)、CJK 感知的文档↔符号链接边界。
|
|
19
20
|
- **经验笔记** — 记录问题 → 方案;相似问题覆盖而非重复;笔记仅在检索命中时返回。笔记数量有界:容量随项目规模伸缩(钳制在 100–2000),超限时淘汰最旧的笔记。**覆盖阈值收紧为双向 0.7 重叠**(原 0.6);**经验 `problem` 字段现参与 CJK 短语加分**,提升长尾问句召回。
|
|
21
|
+
- **无锁同步事务** — 不采用锁:所有写入(index / watch / remember / forget / watch_repo)统一走同步事务 `store.commit(fn)`,fn 成功后才一次落盘;JS 单线程事件循环保证事务间不交错,`remember`/`forget` 不会被 watch 重索引阻塞排队。多实例并发写入同一项目存储时,得益于 CAS 幂等更新与原子提交,自然具备幂等性,无数据损坏风险。
|
|
20
22
|
- **依赖极简** — 纯 JavaScript;唯一运行时依赖是 `pdfjs-dist`(PDF 文本提取),无需原生构建。
|
|
23
|
+
- **开销可忽略** — 纯进程内操作;冷启动 <100 ms(5k 文件),典型项目查询中位数 2–3 ms(p99 < 7 ms);瓶颈在 LLM 摘要与 PDF 解析,插件本身不阻塞。
|
|
21
24
|
|
|
22
25
|
## 工作原理
|
|
23
26
|
|
|
24
27
|
设计遵循四个原则:
|
|
25
28
|
|
|
26
29
|
- **易失性** — 上下文是临时的,会话压缩即丢失。
|
|
27
|
-
- **持久性** —
|
|
28
|
-
- **紧凑性** —
|
|
29
|
-
- **可核验性** —
|
|
30
|
+
- **持久性** — **记忆**存于磁盘,跨压缩与会话保留。
|
|
31
|
+
- **紧凑性** — 仅存摘要;**记忆**规模约为其覆盖源码的 0.5%(示例项目中 8.8 MB 源码 → 49 KB 索引),**召回**替代了通读整个文件。
|
|
32
|
+
- **可核验性** — **召回**在适用时携带 `路径:行号` 引用,agent 可对照源文件核实。
|
|
30
33
|
|
|
31
|
-
|
|
34
|
+
构建**记忆**无需预先全量扫描:文件在模型读取时被记忆,**记忆**恰好覆盖实际处理过的内容。未变更的文件重读是空操作(内容哈希),因此**记忆**的持续维护开销很低。
|
|
32
35
|
|
|
33
36
|
存储按项目独立存放,并跟随代码库变化:文件变更按内容哈希重新抽取,文件删除则同步移除。经验层仅检索,累积不影响上下文。
|
|
34
37
|
|
|
@@ -51,7 +54,7 @@ dsh plugin --profile web add @yolk_vat-y/dsh-project-memory -w
|
|
|
51
54
|
每个版本会附带预构建 tarball,无需构建步骤即可安装:
|
|
52
55
|
|
|
53
56
|
```bash
|
|
54
|
-
dsh plugin --profile web add /path/to/dsh-project-memory
|
|
57
|
+
dsh plugin --profile web add /path/to/dsh-project-memory.tgz
|
|
55
58
|
```
|
|
56
59
|
|
|
57
60
|
每个被索引的项目在 `<root>/.dsh-project-memory/` 下有独立存储。如无需入库,可加入 `.gitignore`。
|
|
@@ -92,10 +95,9 @@ v0.2.0 之前创建的库(单文件 `entries.json` / `index.json`)在首次
|
|
|
92
95
|
|
|
93
96
|
以下是刻意的范围选择。
|
|
94
97
|
|
|
95
|
-
- **无锁同步事务** — 不采用锁:所有写入(index / watch / remember / forget / watch_repo)统一走同步事务 `store.commit(fn)`,fn 成功后才一次落盘;JS 单线程事件循环保证事务间不交错,`remember`/`forget` 不会被 watch
|
|
98
|
+
- **无锁同步事务** — 不采用锁:所有写入(index / watch / remember / forget / watch_repo)统一走同步事务 `store.commit(fn)`,fn 成功后才一次落盘;JS 单线程事件循环保证事务间不交错,`remember`/`forget` 不会被 watch 重索引阻塞排队。多实例并发写入同一项目存储时,得益于 CAS 幂等更新与原子提交,自然具备幂等性,无数据损坏风险。我们不担心 dsh 变多进程,因为 dsh 基于 Cordis,而 Cordis 的单进程架构是基础——改变它将是整个生态的破坏性变更。
|
|
96
99
|
- **watch 计算与提交分离** — watcher 在事务外完成 mtime/哈希/符号扫描/LLM 摘要等重活,再以单次 `commit` 原子应用全部变更;`applyFileUpdate` 用 CAS 校验(统一 null 处理、删除跳过对比)防并发修改,失败回滚 snapshot 下轮重试,索引失败删除 snapshot 自动重试。轮询(mtime + 内容哈希)而非 `fs.watch` 事件驱动,是为了跨平台行为一致。间隔可用 `watchInterval` 调整。
|
|
97
100
|
- **损坏隔离重建** — 存储 JSON 损坏时该文件回落为空并在下次写入时重建;坏文件会改名备份为 `*.corrupt` 并输出错误日志,但该文件内的数据无法恢复。自动修复半写文件需要预写日志或嵌入式数据库,代价与收益不成比例——而隔离一个坏文件的成本几乎为零。
|
|
98
|
-
- **绝对路径引用** — 条目引用绝对路径;项目搬家后引用失效,重建索引即恢复。
|
|
99
101
|
- **`forget` 按关键词删除偏激进** — 关键词删除按 ≥0.5 token 重叠匹配,可能一次删掉多条;追求精确请用 id 删除。
|
|
100
102
|
- **跨语种召回依赖索引时** — `llmQueryExpansion` 关闭时,纯中文查询靠索引时捕获的双语 keywords 和 doc↔symbol 链接触达英文内容,查询侧保持零 LLM 调用。v0.1.1 之前建立的索引随文件变更逐步获得双语关键词,或用 `index_repo` 的 `reindex: true` 立即重建。
|
|
101
103
|
- **CJK 检索** — 短语加分与同义词展开完全在查询侧,不增加索引体积、不额外消耗 LLM token。链接边界仅对 CJK 使用正则边界,英文符号保持原有词边界行为。经验层 supersede 阈值(双向 0.7)为保守默认;若实测出现误覆盖/误漏报,可通过配置调整。
|
|
@@ -116,6 +118,8 @@ v0.2.0 之前创建的库(单文件 `entries.json` / `index.json`)在首次
|
|
|
116
118
|
| `autoIndexOnFirstUse` | false | 插件加载时对当前工作目录做全量扫描(可选) |
|
|
117
119
|
| `watch` | true | 启用后台刷新 |
|
|
118
120
|
| `watchInterval` | 15 | 轮询间隔(秒) |
|
|
121
|
+
| `tsPath` | (自动) | 可选:强制指定特定 `typescript` 安装路径;省略时按项目 cwd → 插件 node_modules 向上解析 |
|
|
122
|
+
| `enableTypeScript` | true | 设为 `false` 彻底禁用 L2 TS 增强(仅保留 L1 正则) |
|
|
119
123
|
|
|
120
124
|
### 功能开关
|
|
121
125
|
|
|
@@ -131,6 +135,8 @@ v0.2.0 之前创建的库(单文件 `entries.json` / `index.json`)在首次
|
|
|
131
135
|
llmQueryExpansion: false # 关闭:不用 LLM 扩展查询,节省 token(默认)
|
|
132
136
|
watch: true # 开启:被监听根目录后台保持新鲜(默认)
|
|
133
137
|
watchInterval: 15 # 轮询间隔(秒)
|
|
138
|
+
enableTypeScript: true # 开启:装了 TS 时启用 L2 语义增强(默认)
|
|
139
|
+
# tsPath: /custom/path/to/typescript # 可选:强制指定 TS 安装路径
|
|
134
140
|
```
|
|
135
141
|
|
|
136
142
|
只需列出要改的键,其余键回落到插件默认值。用 `dsh --profile web --dump-config` 验证生效。
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@yolk_vat-y/dsh-project-memory",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.2",
|
|
4
4
|
"description": "Persistent project memory for dsh agents: index docs (PDF/Markdown/text) and code symbols into a searchable per-workspace store, recall them with cited sources, and keep experience entries (problems -> solutions) searchable on demand.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "src/index.js",
|
|
@@ -39,16 +39,18 @@
|
|
|
39
39
|
"pdfjs-dist": "^4.10.38"
|
|
40
40
|
},
|
|
41
41
|
"devDependencies": {
|
|
42
|
-
"@deepseek-ai/cordis": "4.0.
|
|
43
|
-
"@deepseek-ai/dsh-
|
|
44
|
-
"@deepseek-ai/
|
|
45
|
-
"@deepseek-ai/
|
|
42
|
+
"@deepseek-ai/cordis": "^4.0.2",
|
|
43
|
+
"@deepseek-ai/dsh-llm": "^0.1.2-alpha.2",
|
|
44
|
+
"@deepseek-ai/dsh-tools": "^0.1.2-alpha.2",
|
|
45
|
+
"@deepseek-ai/schemastery": "^3.18.2",
|
|
46
|
+
"typescript": "^5.6.3"
|
|
46
47
|
},
|
|
47
48
|
"peerDependencies": {
|
|
48
49
|
"@deepseek-ai/cordis": "^4.0.1",
|
|
50
|
+
"@deepseek-ai/dsh-llm": ">=0.0.1-rc.1 <0.1.0 || >=0.1.0-rc.1 <0.2.0-0 || >=0.1.1-rc.1 <0.2.0-0",
|
|
49
51
|
"@deepseek-ai/dsh-tools": ">=0.0.1-rc.1 <0.1.0 || >=0.1.0-rc.1 <0.2.0-0 || >=0.1.1-rc.1 <0.2.0-0",
|
|
50
52
|
"@deepseek-ai/schemastery": "^3.18.1",
|
|
51
|
-
"
|
|
53
|
+
"typescript": "^5.0.0"
|
|
52
54
|
},
|
|
53
55
|
"peerDependenciesMeta": {
|
|
54
56
|
"@deepseek-ai/cordis": {
|
|
@@ -62,6 +64,9 @@
|
|
|
62
64
|
},
|
|
63
65
|
"@deepseek-ai/dsh-llm": {
|
|
64
66
|
"optional": true
|
|
67
|
+
},
|
|
68
|
+
"typescript": {
|
|
69
|
+
"optional": true
|
|
65
70
|
}
|
|
66
71
|
},
|
|
67
72
|
"dsh": {
|
package/src/doc-pipeline.js
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { createHash } from 'node:crypto'
|
|
1
2
|
import path from 'node:path'
|
|
2
3
|
import { stat } from 'node:fs/promises'
|
|
3
4
|
import { looksLikeDump, readTextFile } from './util/fs.js'
|
|
@@ -22,10 +23,16 @@ export async function extractTextFromFile(filePath, { maxFileSizeMb = 50, maxPdf
|
|
|
22
23
|
|
|
23
24
|
const DOC_CONCURRENCY = 4
|
|
24
25
|
|
|
25
|
-
export async function buildDocEntries(llm,
|
|
26
|
-
|
|
26
|
+
export async function buildDocEntries(llm, a, b, c) {
|
|
27
|
+
// Backward compatible: old signature (llm, filePath, opts) or new (llm, relPath, filePath, opts)
|
|
28
|
+
const [relPath, filePath, opts] = c === undefined ? [a, a, b] : [a, b, c]
|
|
29
|
+
const text = await extractTextFromFile(filePath, opts)
|
|
27
30
|
if (looksLikeDump(text)) return null
|
|
28
|
-
|
|
31
|
+
|
|
32
|
+
// Compute content hash for update detection
|
|
33
|
+
const hash = createHash('sha256').update(text).digest('hex').slice(0, 16)
|
|
34
|
+
|
|
35
|
+
const chunks = chunkText(text, opts.chunkChars, opts.maxChunks)
|
|
29
36
|
const metas = new Array(chunks.length)
|
|
30
37
|
let cursor = 0
|
|
31
38
|
await Promise.all(
|
|
@@ -39,13 +46,15 @@ export async function buildDocEntries(llm, filePath, { chunkChars = 3000, maxChu
|
|
|
39
46
|
),
|
|
40
47
|
)
|
|
41
48
|
return metas.map((meta, i) => ({
|
|
42
|
-
id: `${relativeId(
|
|
43
|
-
sourcePath:
|
|
49
|
+
id: `${relativeId(relPath)}#${i}`,
|
|
50
|
+
sourcePath: relPath,
|
|
44
51
|
sourceLine: chunks[i].line,
|
|
45
52
|
type: 'doc',
|
|
46
53
|
title: meta.title,
|
|
47
54
|
summary: meta.summary,
|
|
55
|
+
blindSpots: meta.blindSpots || '',
|
|
48
56
|
keywords: meta.keywords,
|
|
57
|
+
hash,
|
|
49
58
|
}))
|
|
50
59
|
}
|
|
51
60
|
|