@morningljn/mnemo 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/config.d.ts +2 -0
- package/dist/config.js +28 -0
- package/dist/config.js.map +1 -0
- package/dist/dream-engine.d.ts +17 -0
- package/dist/dream-engine.js +144 -0
- package/dist/dream-engine.js.map +1 -0
- package/dist/dream.d.ts +2 -0
- package/dist/dream.js +20 -0
- package/dist/dream.js.map +1 -0
- package/dist/init.js +4 -24
- package/dist/init.js.map +1 -1
- package/dist/llm-client.d.ts +10 -0
- package/dist/llm-client.js +55 -0
- package/dist/llm-client.js.map +1 -0
- package/dist/resources.d.ts +22 -8
- package/dist/resources.js +66 -20
- package/dist/resources.js.map +1 -1
- package/dist/retriever.js +12 -5
- package/dist/retriever.js.map +1 -1
- package/dist/schema.d.ts +1 -1
- package/dist/schema.js +2 -2
- package/dist/server.js +40 -6
- package/dist/server.js.map +1 -1
- package/dist/store.d.ts +23 -1
- package/dist/store.js +169 -4
- package/dist/store.js.map +1 -1
- package/dist/types.d.ts +41 -1
- package/docs/superpowers/plans/2026-05-16-llm-dream.md +973 -0
- package/docs/superpowers/plans/2026-05-16-memory-dreaming.md +626 -0
- package/openspec/changes/archive/2026-05-16-memory-dreaming/.openspec.yaml +2 -0
- package/openspec/changes/archive/2026-05-16-memory-dreaming/design.md +71 -0
- package/openspec/changes/archive/2026-05-16-memory-dreaming/proposal.md +32 -0
- package/openspec/changes/archive/2026-05-16-memory-dreaming/specs/compact-search/spec.md +16 -0
- package/openspec/changes/archive/2026-05-16-memory-dreaming/specs/dream-cycle/spec.md +38 -0
- package/openspec/changes/archive/2026-05-16-memory-dreaming/tasks.md +27 -0
- package/openspec/changes/llm-dream/.openspec.yaml +2 -0
- package/openspec/changes/llm-dream/design.md +84 -0
- package/openspec/changes/llm-dream/proposal.md +36 -0
- package/openspec/changes/llm-dream/specs/dream-cycle/spec.md +42 -0
- package/openspec/changes/llm-dream/specs/llm-client/spec.md +57 -0
- package/openspec/changes/llm-dream/specs/llm-dream-engine/spec.md +72 -0
- package/openspec/changes/llm-dream/tasks.md +32 -0
- package/openspec/specs/compact-search/spec.md +16 -0
- package/openspec/specs/dream-cycle/spec.md +38 -0
- package/package.json +3 -2
- package/src/config.ts +29 -0
- package/src/dream-engine.ts +162 -0
- package/src/dream.ts +20 -0
- package/src/init.ts +4 -24
- package/src/llm-client.ts +59 -0
- package/src/resources.ts +77 -21
- package/src/retriever.ts +9 -5
- package/src/schema.ts +2 -2
- package/src/server.ts +46 -7
- package/src/store.ts +198 -5
- package/src/types.ts +41 -1
- package/tests/dream-engine.test.ts +163 -0
- package/tests/llm-client.test.ts +105 -0
- package/tests/resource.test.ts +25 -23
- package/tests/store.test.ts +130 -2
|
@@ -0,0 +1,626 @@
|
|
|
1
|
+
# Memory Dreaming 实现计划
|
|
2
|
+
|
|
3
|
+
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
4
|
+
|
|
5
|
+
**Goal:** 为 mnemo-mcp 添加后台记忆整理(dream)和搜索结果精简功能,保持数据库精炼、减少 token 消耗。
|
|
6
|
+
|
|
7
|
+
**Architecture:** 在 `MemoryStore` 中新增 `runDream()` 方法,编排合并去重、摘要压缩、分类修正三阶段。新增 `CompactResult` 类型精简搜索返回字段。新增 CLI 入口 `src/dream.ts`。
|
|
8
|
+
|
|
9
|
+
**Tech Stack:** TypeScript, better-sqlite3, Vitest
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## File Structure
|
|
14
|
+
|
|
15
|
+
| File | Responsibility |
|
|
16
|
+
|------|---------------|
|
|
17
|
+
| `src/types.ts` | 新增 `DreamReport` 和 `CompactFactResult` 类型 |
|
|
18
|
+
| `src/store.ts` | 新增 `runDream()`、`mergeOverlappingFacts()`、`compressLongFacts()`、`reclassifyFacts()`、`backupDatabase()` |
|
|
19
|
+
| `src/retriever.ts` | 搜索结果格式化为 `CompactFactResult` |
|
|
20
|
+
| `src/server.ts` | 新增 `dream` action,所有搜索 action 使用精简格式 |
|
|
21
|
+
| `src/dream.ts` | CLI 入口,执行 dream 并输出 report |
|
|
22
|
+
| `package.json` | 新增 `mnemo-dream` bin |
|
|
23
|
+
| `tests/store.test.ts` | dream 相关单元测试 |
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
### Task 1: 新增类型定义
|
|
28
|
+
|
|
29
|
+
**Files:**
|
|
30
|
+
- Modify: `src/types.ts`
|
|
31
|
+
|
|
32
|
+
- [ ] **Step 1: 在 types.ts 末尾添加 DreamReport 和 CompactFactResult 类型**
|
|
33
|
+
|
|
34
|
+
```typescript
|
|
35
|
+
/** Dream 整理报告 */
|
|
36
|
+
export interface DreamReport {
|
|
37
|
+
merged: number
|
|
38
|
+
compressed: number
|
|
39
|
+
reclassified: number
|
|
40
|
+
deleted: number
|
|
41
|
+
mergeDetails: Array<{ kept: number; removed: number; similarity: number }>
|
|
42
|
+
health: {
|
|
43
|
+
total: number
|
|
44
|
+
avg_trust: number
|
|
45
|
+
avg_length: number
|
|
46
|
+
coverage: Record<FactCategory, number>
|
|
47
|
+
}
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
/** 精简搜索结果 */
|
|
51
|
+
export interface CompactFactResult {
|
|
52
|
+
factId: number
|
|
53
|
+
display: string
|
|
54
|
+
category: FactCategory
|
|
55
|
+
trustScore: number
|
|
56
|
+
score: number
|
|
57
|
+
}
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
- [ ] **Step 2: 更新 FactStoreArgs 的 action 联合类型**
|
|
61
|
+
|
|
62
|
+
在 `src/types.ts` 的 `FactStoreArgs.action` 字段中,把 `'learn' | 'audit'` 改为 `'learn' | 'audit' | 'dream'`:
|
|
63
|
+
|
|
64
|
+
```typescript
|
|
65
|
+
action: 'add' | 'search' | 'probe' | 'related' | 'reason' | 'contradict' | 'update' | 'remove' | 'list' | 'learn' | 'audit' | 'dream'
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
- [ ] **Step 3: 运行构建确认类型无报错**
|
|
69
|
+
|
|
70
|
+
Run: `npm run build`
|
|
71
|
+
Expected: 编译成功,无报错
|
|
72
|
+
|
|
73
|
+
- [ ] **Step 4: Commit**
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
git add src/types.ts
|
|
77
|
+
git commit -m "feat(types): add DreamReport and CompactFactResult types"
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
---
|
|
81
|
+
|
|
82
|
+
### Task 2: Store 层 dream 方法 — 备份与压缩
|
|
83
|
+
|
|
84
|
+
**Files:**
|
|
85
|
+
- Modify: `src/store.ts`
|
|
86
|
+
|
|
87
|
+
- [ ] **Step 1: 写失败测试 — backupDatabase**
|
|
88
|
+
|
|
89
|
+
在 `tests/store.test.ts` 的最后一个 `describe` 块之后添加:
|
|
90
|
+
|
|
91
|
+
```typescript
|
|
92
|
+
describe('dream - backup', () => {
|
|
93
|
+
it('creates backup before dream', () => {
|
|
94
|
+
const id = store.addFact('test fact for backup', 'general')
|
|
95
|
+
const result = store.backupDatabase()
|
|
96
|
+
expect(result).toBeTruthy()
|
|
97
|
+
expect(result).toContain('dream-')
|
|
98
|
+
expect(result).toContain('.db')
|
|
99
|
+
})
|
|
100
|
+
})
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Run: `npx vitest run tests/store.test.ts`
|
|
104
|
+
Expected: FAIL — `backupDatabase` 不存在
|
|
105
|
+
|
|
106
|
+
- [ ] **Step 2: 实现 backupDatabase**
|
|
107
|
+
|
|
108
|
+
在 `src/store.ts` 的 `runAudit()` 方法之后、`get connection()` 之前添加:
|
|
109
|
+
|
|
110
|
+
```typescript
|
|
111
|
+
/** Dream 前备份数据库 */
|
|
112
|
+
backupDatabase(): string {
|
|
113
|
+
const { mkdirSync, copyFileSync } = require('node:fs')
|
|
114
|
+
const { join, dirname } = require('node:path')
|
|
115
|
+
const { homedir } = require('node:os')
|
|
116
|
+
const backupDir = join(homedir(), '.mnemo', 'backup')
|
|
117
|
+
mkdirSync(backupDir, { recursive: true })
|
|
118
|
+
const timestamp = new Date().toISOString().replace(/[:.]/g, '-').slice(0, 19)
|
|
119
|
+
const backupPath = join(backupDir, `dream-${timestamp}.db`)
|
|
120
|
+
copyFileSync(this.db.name, backupPath)
|
|
121
|
+
return backupPath
|
|
122
|
+
}
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
注意:需要在文件顶部添加 `import { copyFileSync, mkdirSync } from 'node:fs'`(如果还没有的话)。实际上 `mkdirSync` 已在顶部导入了,只需确认 `copyFileSync` 也已导入。检查 store.ts 顶部的 import,如果没有 `copyFileSync`,添加到现有 `mkdirSync` 的 import 行中。
|
|
126
|
+
|
|
127
|
+
- [ ] **Step 3: 运行测试确认通过**
|
|
128
|
+
|
|
129
|
+
Run: `npx vitest run tests/store.test.ts`
|
|
130
|
+
Expected: PASS
|
|
131
|
+
|
|
132
|
+
- [ ] **Step 4: 写失败测试 — compressLongFacts**
|
|
133
|
+
|
|
134
|
+
```typescript
|
|
135
|
+
describe('dream - compress', () => {
|
|
136
|
+
it('generates summary for long facts without summary', () => {
|
|
137
|
+
const longContent = '用户偏好使用 TypeScript 开发前端项目。偏好 React 框架进行组件化开发。' + '额外补充说明'.repeat(50)
|
|
138
|
+
store.addFact(longContent, 'coding_style')
|
|
139
|
+
const result = store.compressLongFacts()
|
|
140
|
+
expect(result).toBeGreaterThanOrEqual(1)
|
|
141
|
+
const row = store.connection.prepare('SELECT summary FROM facts WHERE content = ?').get(longContent) as any
|
|
142
|
+
expect(row.summary).toBeTruthy()
|
|
143
|
+
expect(row.summary.length).toBeLessThanOrEqual(150)
|
|
144
|
+
expect(row.summary).toContain('TypeScript')
|
|
145
|
+
})
|
|
146
|
+
|
|
147
|
+
it('skips facts with existing summary', () => {
|
|
148
|
+
const longContent = 'x'.repeat(300)
|
|
149
|
+
const id = store.addFact(longContent, 'general')
|
|
150
|
+
store.connection.prepare('UPDATE facts SET summary = ? WHERE fact_id = ?').run('existing summary', id)
|
|
151
|
+
const result = store.compressLongFacts()
|
|
152
|
+
const row = store.connection.prepare('SELECT summary FROM facts WHERE fact_id = ?').get(id) as any
|
|
153
|
+
expect(row.summary).toBe('existing summary')
|
|
154
|
+
})
|
|
155
|
+
|
|
156
|
+
it('skips short facts', () => {
|
|
157
|
+
store.addFact('short fact', 'general')
|
|
158
|
+
const result = store.compressLongFacts()
|
|
159
|
+
expect(result).toBe(0)
|
|
160
|
+
})
|
|
161
|
+
})
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
Run: `npx vitest run tests/store.test.ts`
|
|
165
|
+
Expected: FAIL — `compressLongFacts` 不存在
|
|
166
|
+
|
|
167
|
+
- [ ] **Step 5: 实现 compressLongFacts**
|
|
168
|
+
|
|
169
|
+
在 `store.ts` 的 `backupDatabase()` 之后添加:
|
|
170
|
+
|
|
171
|
+
```typescript
|
|
172
|
+
/** 压缩长 fact:content > 200 字且无 summary 的,自动提取前 2 句作为 summary */
|
|
173
|
+
compressLongFacts(): number {
|
|
174
|
+
const rows = this.db.prepare(
|
|
175
|
+
'SELECT fact_id, content FROM facts WHERE length(content) > 200 AND (summary IS NULL OR summary = "")'
|
|
176
|
+
).all() as Array<{ fact_id: number; content: string }>
|
|
177
|
+
|
|
178
|
+
let compressed = 0
|
|
179
|
+
for (const row of rows) {
|
|
180
|
+
const summary = this.extractSummary(row.content)
|
|
181
|
+
if (summary) {
|
|
182
|
+
this.db.prepare('UPDATE facts SET summary = ? WHERE fact_id = ?').run(summary, row.fact_id)
|
|
183
|
+
compressed++
|
|
184
|
+
}
|
|
185
|
+
}
|
|
186
|
+
return compressed
|
|
187
|
+
}
|
|
188
|
+
|
|
189
|
+
/** 从 content 提取前 2 个完整句子(总长 ≤ 150 字) */
|
|
190
|
+
private extractSummary(content: string): string | null {
|
|
191
|
+
const sentences = content.split(/[。\n.]/).map(s => s.trim()).filter(s => s.length > 0)
|
|
192
|
+
if (sentences.length === 0) return null
|
|
193
|
+
let summary = sentences[0]
|
|
194
|
+
if (sentences.length > 1 && summary.length + sentences[1].length <= 148) {
|
|
195
|
+
summary += '。' + sentences[1]
|
|
196
|
+
}
|
|
197
|
+
return summary.length <= 150 ? summary : summary.slice(0, 147) + '...'
|
|
198
|
+
}
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
- [ ] **Step 6: 运行测试确认通过**
|
|
202
|
+
|
|
203
|
+
Run: `npx vitest run tests/store.test.ts`
|
|
204
|
+
Expected: PASS
|
|
205
|
+
|
|
206
|
+
- [ ] **Step 7: Commit**
|
|
207
|
+
|
|
208
|
+
```bash
|
|
209
|
+
git add src/store.ts tests/store.test.ts
|
|
210
|
+
git commit -m "feat(store): add backupDatabase and compressLongFacts for dream cycle"
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
---
|
|
214
|
+
|
|
215
|
+
### Task 3: Store 层 dream 方法 — 合并与分类修正
|
|
216
|
+
|
|
217
|
+
**Files:**
|
|
218
|
+
- Modify: `src/store.ts`
|
|
219
|
+
- Modify: `tests/store.test.ts`
|
|
220
|
+
|
|
221
|
+
- [ ] **Step 1: 写失败测试 — mergeOverlappingFacts**
|
|
222
|
+
|
|
223
|
+
```typescript
|
|
224
|
+
describe('dream - merge', () => {
|
|
225
|
+
it('merges overlapping facts in same category', () => {
|
|
226
|
+
store.addFact('用户偏好使用 TypeScript 编写前端代码', 'coding_style')
|
|
227
|
+
store.addFact('用户偏好使用 TypeScript 编写后端代码', 'coding_style')
|
|
228
|
+
const result = store.mergeOverlappingFacts()
|
|
229
|
+
expect(result.merged).toBeGreaterThanOrEqual(1)
|
|
230
|
+
expect(result.details.length).toBeGreaterThanOrEqual(1)
|
|
231
|
+
})
|
|
232
|
+
|
|
233
|
+
it('protects high frequency facts from deletion', () => {
|
|
234
|
+
const id1 = store.addFact('用户偏好使用 TypeScript 编写前端代码', 'coding_style')
|
|
235
|
+
const id2 = store.addFact('用户偏好使用 TypeScript 编写前端代码扩展', 'coding_style')
|
|
236
|
+
store.connection.prepare('UPDATE facts SET retrieval_count = 200 WHERE fact_id = ?').run(id1)
|
|
237
|
+
const result = store.mergeOverlappingFacts()
|
|
238
|
+
const kept = store.connection.prepare('SELECT fact_id FROM facts WHERE fact_id = ?').get(id1) as any
|
|
239
|
+
expect(kept).toBeTruthy()
|
|
240
|
+
})
|
|
241
|
+
|
|
242
|
+
it('does not merge facts across categories', () => {
|
|
243
|
+
store.addFact('用户偏好使用 TypeScript 编写前端代码', 'coding_style')
|
|
244
|
+
store.addFact('用户偏好使用 TypeScript 编写前端代码', 'general')
|
|
245
|
+
const result = store.mergeOverlappingFacts()
|
|
246
|
+
expect(result.merged).toBe(0)
|
|
247
|
+
})
|
|
248
|
+
})
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
Run: `npx vitest run tests/store.test.ts`
|
|
252
|
+
Expected: FAIL — `mergeOverlappingFacts` 不存在
|
|
253
|
+
|
|
254
|
+
- [ ] **Step 2: 实现 mergeOverlappingFacts**
|
|
255
|
+
|
|
256
|
+
在 `store.ts` 的 `compressLongFacts()` 之后添加:
|
|
257
|
+
|
|
258
|
+
```typescript
|
|
259
|
+
/** 合并同 category 内 Jaccard > 0.6 的重叠 fact */
|
|
260
|
+
mergeOverlappingFacts(): { merged: number; details: Array<{ kept: number; removed: number; similarity: number }> } {
|
|
261
|
+
const categories: FactCategory[] = ['identity', 'coding_style', 'tool_pref', 'workflow', 'general']
|
|
262
|
+
let merged = 0
|
|
263
|
+
const details: Array<{ kept: number; removed: number; similarity: number }> = []
|
|
264
|
+
|
|
265
|
+
for (const cat of categories) {
|
|
266
|
+
const rows = this.db.prepare(
|
|
267
|
+
'SELECT fact_id, content, retrieval_count FROM facts WHERE category = ? ORDER BY trust_score DESC'
|
|
268
|
+
).all(cat) as Array<{ fact_id: number; content: string; retrieval_count: number }>
|
|
269
|
+
|
|
270
|
+
const removed = new Set<number>()
|
|
271
|
+
|
|
272
|
+
for (let i = 0; i < rows.length; i++) {
|
|
273
|
+
if (removed.has(rows[i].fact_id)) continue
|
|
274
|
+
const tokensA = this.tokenizeForDedup(rows[i].content)
|
|
275
|
+
|
|
276
|
+
for (let j = i + 1; j < rows.length; j++) {
|
|
277
|
+
if (removed.has(rows[j].fact_id)) continue
|
|
278
|
+
const tokensB = this.tokenizeForDedup(rows[j].content)
|
|
279
|
+
const sim = this.jaccardSimilarity(tokensA, tokensB)
|
|
280
|
+
|
|
281
|
+
if (sim > 0.6) {
|
|
282
|
+
// 高频保护:retrieval_count > 100 的不能被删除
|
|
283
|
+
const aHighFreq = rows[i].retrieval_count > 100
|
|
284
|
+
const bHighFreq = rows[j].retrieval_count > 100
|
|
285
|
+
|
|
286
|
+
if (aHighFreq && bHighFreq) continue
|
|
287
|
+
|
|
288
|
+
let keptId: number, removedId: number
|
|
289
|
+
if (bHighFreq) {
|
|
290
|
+
keptId = rows[j].fact_id
|
|
291
|
+
removedId = rows[i].fact_id
|
|
292
|
+
} else {
|
|
293
|
+
keptId = rows[i].fact_id
|
|
294
|
+
removedId = rows[j].fact_id
|
|
295
|
+
}
|
|
296
|
+
|
|
297
|
+
this.removeFact(removedId)
|
|
298
|
+
removed.add(removedId)
|
|
299
|
+
details.push({ kept: keptId, removed: removedId, similarity: Math.round(sim * 100) / 100 })
|
|
300
|
+
merged++
|
|
301
|
+
}
|
|
302
|
+
}
|
|
303
|
+
}
|
|
304
|
+
}
|
|
305
|
+
return { merged, details }
|
|
306
|
+
}
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
- [ ] **Step 3: 运行测试确认通过**
|
|
310
|
+
|
|
311
|
+
Run: `npx vitest run tests/store.test.ts`
|
|
312
|
+
Expected: PASS
|
|
313
|
+
|
|
314
|
+
- [ ] **Step 4: 写失败测试 — reclassifyFacts**
|
|
315
|
+
|
|
316
|
+
```typescript
|
|
317
|
+
describe('dream - reclassify', () => {
|
|
318
|
+
it('moves miscategorized facts by keywords', () => {
|
|
319
|
+
const id = store.addFact('编码规范:文件不超过 500 行', 'identity')
|
|
320
|
+
const result = store.reclassifyFacts()
|
|
321
|
+
expect(result).toBeGreaterThanOrEqual(1)
|
|
322
|
+
const row = store.connection.prepare('SELECT category FROM facts WHERE fact_id = ?').get(id) as any
|
|
323
|
+
expect(row.category).toBe('coding_style')
|
|
324
|
+
})
|
|
325
|
+
|
|
326
|
+
it('skips correctly categorized facts', () => {
|
|
327
|
+
store.addFact('用户偏好使用 VS Code 编辑器', 'tool_pref')
|
|
328
|
+
const result = store.reclassifyFacts()
|
|
329
|
+
expect(result).toBe(0)
|
|
330
|
+
})
|
|
331
|
+
})
|
|
332
|
+
```
|
|
333
|
+
|
|
334
|
+
Run: `npx vitest run tests/store.test.ts`
|
|
335
|
+
Expected: FAIL — `reclassifyFacts` 不存在
|
|
336
|
+
|
|
337
|
+
- [ ] **Step 5: 实现 reclassifyFacts**
|
|
338
|
+
|
|
339
|
+
在 `store.ts` 的 `mergeOverlappingFacts()` 之后添加:
|
|
340
|
+
|
|
341
|
+
```typescript
|
|
342
|
+
/** 分类修正:按关键词规则表将误分类的 fact 挪到正确 category */
|
|
343
|
+
reclassifyFacts(): number {
|
|
344
|
+
const rules: Array<{ keywords: string[]; target: FactCategory }> = [
|
|
345
|
+
{ keywords: ['角色设定', '暖暖', '身份', '编程女朋友', '暖宝宝'], target: 'identity' },
|
|
346
|
+
{ keywords: ['编码规范', '代码风格', 'pytest', '文件不超过', '方法不超过'], target: 'coding_style' },
|
|
347
|
+
{ keywords: ['工作流', 'OpenSpec', 'writing-plans', 'subagent'], target: 'workflow' },
|
|
348
|
+
{ keywords: ['偏好', 'VS Code', '编辑器', 'IDE', '快捷键'], target: 'tool_pref' },
|
|
349
|
+
]
|
|
350
|
+
|
|
351
|
+
const rows = this.db.prepare(
|
|
352
|
+
'SELECT fact_id, content, category FROM facts'
|
|
353
|
+
).all() as Array<{ fact_id: number; content: string; category: string }>
|
|
354
|
+
|
|
355
|
+
let reclassified = 0
|
|
356
|
+
for (const row of rows) {
|
|
357
|
+
for (const rule of rules) {
|
|
358
|
+
if (rule.target === row.category) continue
|
|
359
|
+
if (rule.keywords.some(kw => row.content.includes(kw))) {
|
|
360
|
+
this.db.prepare('UPDATE facts SET category = ? WHERE fact_id = ?').run(rule.target, row.fact_id)
|
|
361
|
+
reclassified++
|
|
362
|
+
break
|
|
363
|
+
}
|
|
364
|
+
}
|
|
365
|
+
}
|
|
366
|
+
return reclassified
|
|
367
|
+
}
|
|
368
|
+
```
|
|
369
|
+
|
|
370
|
+
- [ ] **Step 6: 运行测试确认通过**
|
|
371
|
+
|
|
372
|
+
Run: `npx vitest run tests/store.test.ts`
|
|
373
|
+
Expected: PASS
|
|
374
|
+
|
|
375
|
+
- [ ] **Step 7: 写端到端测试 — runDream**
|
|
376
|
+
|
|
377
|
+
```typescript
|
|
378
|
+
describe('dream - runDream', () => {
|
|
379
|
+
it('runs full dream cycle and returns report', () => {
|
|
380
|
+
// 长文 fact(触发压缩)
|
|
381
|
+
store.addFact('用户偏好使用 TypeScript 开发前端。使用 React 框架。' + 'x'.repeat(250), 'coding_style')
|
|
382
|
+
// 重叠 fact(触发合并)
|
|
383
|
+
store.addFact('用户偏好使用 TypeScript 开发前端代码', 'coding_style')
|
|
384
|
+
// 分类错误 fact(触发重分类)
|
|
385
|
+
store.addFact('编码规范:文件不超过 500 行', 'identity')
|
|
386
|
+
|
|
387
|
+
const report = store.runDream({ skipBackup: true })
|
|
388
|
+
expect(report.compressed).toBeGreaterThanOrEqual(0)
|
|
389
|
+
expect(report.merged).toBeGreaterThanOrEqual(0)
|
|
390
|
+
expect(report.reclassified).toBeGreaterThanOrEqual(0)
|
|
391
|
+
expect(report.health.total).toBeGreaterThanOrEqual(1)
|
|
392
|
+
expect(report.health.coverage).toBeTruthy()
|
|
393
|
+
})
|
|
394
|
+
})
|
|
395
|
+
```
|
|
396
|
+
|
|
397
|
+
Run: `npx vitest run tests/store.test.ts`
|
|
398
|
+
Expected: FAIL — `runDream` 不存在
|
|
399
|
+
|
|
400
|
+
- [ ] **Step 8: 实现 runDream**
|
|
401
|
+
|
|
402
|
+
在 `store.ts` 的 `reclassifyFacts()` 之后添加:
|
|
403
|
+
|
|
404
|
+
```typescript
|
|
405
|
+
/** 执行完整 dream cycle:备份 → 压缩 → 合并 → 重分类 → 报告 */
|
|
406
|
+
runDream(options?: { skipBackup?: boolean }): DreamReport {
|
|
407
|
+
// 备份
|
|
408
|
+
if (!options?.skipBackup) {
|
|
409
|
+
this.backupDatabase()
|
|
410
|
+
}
|
|
411
|
+
|
|
412
|
+
// 阶段 1:压缩长文
|
|
413
|
+
const compressed = this.compressLongFacts()
|
|
414
|
+
|
|
415
|
+
// 阶段 2:合并重叠
|
|
416
|
+
const mergeResult = this.mergeOverlappingFacts()
|
|
417
|
+
|
|
418
|
+
// 阶段 3:分类修正
|
|
419
|
+
const reclassified = this.reclassifyFacts()
|
|
420
|
+
|
|
421
|
+
// 健康统计
|
|
422
|
+
const stats = this.db.prepare(`
|
|
423
|
+
SELECT COUNT(*) as total,
|
|
424
|
+
AVG(trust_score) as avg_trust,
|
|
425
|
+
AVG(length(content)) as avg_length
|
|
426
|
+
FROM facts
|
|
427
|
+
`).get() as { total: number; avg_trust: number; avg_length: number }
|
|
428
|
+
|
|
429
|
+
const categories: FactCategory[] = ['identity', 'coding_style', 'tool_pref', 'workflow', 'general']
|
|
430
|
+
const coverage: Record<string, number> = {}
|
|
431
|
+
for (const cat of categories) {
|
|
432
|
+
const row = this.db.prepare('SELECT COUNT(*) as c FROM facts WHERE category = ?').get(cat) as { c: number }
|
|
433
|
+
coverage[cat] = row.c
|
|
434
|
+
}
|
|
435
|
+
|
|
436
|
+
return {
|
|
437
|
+
merged: mergeResult.merged,
|
|
438
|
+
compressed,
|
|
439
|
+
reclassified,
|
|
440
|
+
deleted: mergeResult.merged,
|
|
441
|
+
mergeDetails: mergeResult.details,
|
|
442
|
+
health: {
|
|
443
|
+
total: stats.total,
|
|
444
|
+
avg_trust: Math.round((stats.avg_trust ?? 0) * 100) / 100,
|
|
445
|
+
avg_length: Math.round(stats.avg_length ?? 0),
|
|
446
|
+
coverage: coverage as Record<FactCategory, number>,
|
|
447
|
+
},
|
|
448
|
+
}
|
|
449
|
+
}
|
|
450
|
+
```
|
|
451
|
+
|
|
452
|
+
需要在 `store.ts` 顶部添加 `import type { DreamReport } from './types.js'`(如果还没有的话)。
|
|
453
|
+
|
|
454
|
+
- [ ] **Step 9: 运行全部测试**
|
|
455
|
+
|
|
456
|
+
Run: `npx vitest run`
|
|
457
|
+
Expected: ALL PASS
|
|
458
|
+
|
|
459
|
+
- [ ] **Step 10: Commit**
|
|
460
|
+
|
|
461
|
+
```bash
|
|
462
|
+
git add src/store.ts tests/store.test.ts
|
|
463
|
+
git commit -m "feat(store): add mergeOverlappingFacts, reclassifyFacts, runDream for dream cycle"
|
|
464
|
+
```
|
|
465
|
+
|
|
466
|
+
---
|
|
467
|
+
|
|
468
|
+
### Task 4: Server 层 — dream action + 精简搜索格式
|
|
469
|
+
|
|
470
|
+
**Files:**
|
|
471
|
+
- Modify: `src/server.ts`
|
|
472
|
+
- Modify: `src/retriever.ts`
|
|
473
|
+
|
|
474
|
+
- [ ] **Step 1: 在 server.ts 添加 dream case**
|
|
475
|
+
|
|
476
|
+
在 `server.ts` 的 `case 'audit'` 块之后、`case 'list'` 之前添加:
|
|
477
|
+
|
|
478
|
+
```typescript
|
|
479
|
+
case 'dream': {
|
|
480
|
+
const report = store.runDream()
|
|
481
|
+
retriever.getCache().clear()
|
|
482
|
+
resourceManager.invalidate()
|
|
483
|
+
return { content: [{ type: 'text' as const, text: JSON.stringify(report) }] }
|
|
484
|
+
}
|
|
485
|
+
```
|
|
486
|
+
|
|
487
|
+
同时在 `factStoreSchema` 的 `action` 枚举中添加 `'dream'`。
|
|
488
|
+
|
|
489
|
+
- [ ] **Step 2: 更新 factStoreSchema 的 action 枚举**
|
|
490
|
+
|
|
491
|
+
在 `src/server.ts` 中,把 `action: z.enum([... 'learn', 'audit'])` 改为 `action: z.enum([... 'learn', 'audit', 'dream'])`。
|
|
492
|
+
|
|
493
|
+
- [ ] **Step 3: 在 server.ts 添加 toCompactResult 辅助函数**
|
|
494
|
+
|
|
495
|
+
在 server.ts 的 `resolveCategory` 函数之后添加:
|
|
496
|
+
|
|
497
|
+
```typescript
|
|
498
|
+
function toCompactResult(f: ScoredFact): CompactFactResult {
|
|
499
|
+
return {
|
|
500
|
+
factId: f.factId,
|
|
501
|
+
display: f.summary ?? (f.content.length > 100 ? f.content.slice(0, 100) + '...' : f.content),
|
|
502
|
+
category: f.category,
|
|
503
|
+
trustScore: Math.round(f.trustScore * 100) / 100,
|
|
504
|
+
score: Math.round(f.score * 1000) / 1000,
|
|
505
|
+
}
|
|
506
|
+
}
|
|
507
|
+
```
|
|
508
|
+
|
|
509
|
+
需要在 `server.ts` 顶部添加 `import type { CompactFactResult } from './types.js'`。
|
|
510
|
+
|
|
511
|
+
- [ ] **Step 4: 更新 search/probe/related/reason 响应使用精简格式**
|
|
512
|
+
|
|
513
|
+
把 `case 'search'` 中的:
|
|
514
|
+
```typescript
|
|
515
|
+
return { content: [{ type: 'text' as const, text: JSON.stringify({ results, count: results.length }) }] }
|
|
516
|
+
```
|
|
517
|
+
改为:
|
|
518
|
+
```typescript
|
|
519
|
+
const compact = results.map(toCompactResult)
|
|
520
|
+
return { content: [{ type: 'text' as const, text: JSON.stringify({ results: compact, count: compact.length }) }] }
|
|
521
|
+
```
|
|
522
|
+
|
|
523
|
+
对 `probe`、`related`、`reason` 做同样的改动。
|
|
524
|
+
|
|
525
|
+
- [ ] **Step 5: 构建并运行测试**
|
|
526
|
+
|
|
527
|
+
Run: `npm run build && npx vitest run`
|
|
528
|
+
Expected: ALL PASS
|
|
529
|
+
|
|
530
|
+
- [ ] **Step 6: Commit**
|
|
531
|
+
|
|
532
|
+
```bash
|
|
533
|
+
git add src/server.ts
|
|
534
|
+
git commit -m "feat(server): add dream action, compact search result format"
|
|
535
|
+
```
|
|
536
|
+
|
|
537
|
+
---
|
|
538
|
+
|
|
539
|
+
### Task 5: CLI dream 命令
|
|
540
|
+
|
|
541
|
+
**Files:**
|
|
542
|
+
- Create: `src/dream.ts`
|
|
543
|
+
- Modify: `package.json`
|
|
544
|
+
|
|
545
|
+
- [ ] **Step 1: 创建 src/dream.ts**
|
|
546
|
+
|
|
547
|
+
```typescript
|
|
548
|
+
#!/usr/bin/env node
|
|
549
|
+
|
|
550
|
+
import { MemoryStore } from './store.js'
|
|
551
|
+
import { join } from 'node:path'
|
|
552
|
+
import { homedir } from 'node:os'
|
|
553
|
+
|
|
554
|
+
const dbPath = join(homedir(), '.mnemo', 'facts.db')
|
|
555
|
+
const store = new MemoryStore(dbPath)
|
|
556
|
+
|
|
557
|
+
try {
|
|
558
|
+
console.log('[mnemo dream] 开始整理记忆库...\n')
|
|
559
|
+
const report = store.runDream()
|
|
560
|
+
console.log(JSON.stringify(report, null, 2))
|
|
561
|
+
console.log(`\n[mnemo dream] 完成: merged=${report.merged} compressed=${report.compressed} reclassified=${report.reclassified} deleted=${report.deleted}`)
|
|
562
|
+
} catch (err) {
|
|
563
|
+
console.error('[mnemo dream] error:', err)
|
|
564
|
+
process.exit(1)
|
|
565
|
+
} finally {
|
|
566
|
+
store.close()
|
|
567
|
+
}
|
|
568
|
+
```
|
|
569
|
+
|
|
570
|
+
- [ ] **Step 2: 在 package.json 添加 bin 入口**
|
|
571
|
+
|
|
572
|
+
在 `package.json` 的 `bin` 字段中添加 `"mnemo-dream": "dist/dream.js"`:
|
|
573
|
+
|
|
574
|
+
```json
|
|
575
|
+
"bin": {
|
|
576
|
+
"mnemo": "dist/server.js",
|
|
577
|
+
"mnemo-init": "dist/init.js",
|
|
578
|
+
"mnemo-dream": "dist/dream.js"
|
|
579
|
+
}
|
|
580
|
+
```
|
|
581
|
+
|
|
582
|
+
- [ ] **Step 3: 构建并验证**
|
|
583
|
+
|
|
584
|
+
Run: `npm run build && node dist/dream.js`
|
|
585
|
+
Expected: 输出 dream report JSON
|
|
586
|
+
|
|
587
|
+
- [ ] **Step 4: Commit**
|
|
588
|
+
|
|
589
|
+
```bash
|
|
590
|
+
git add src/dream.ts package.json
|
|
591
|
+
git commit -m "feat(cli): add mnemo dream CLI command"
|
|
592
|
+
```
|
|
593
|
+
|
|
594
|
+
---
|
|
595
|
+
|
|
596
|
+
### Task 6: 最终验证
|
|
597
|
+
|
|
598
|
+
- [ ] **Step 1: 运行全部测试**
|
|
599
|
+
|
|
600
|
+
Run: `npx vitest run`
|
|
601
|
+
Expected: ALL PASS
|
|
602
|
+
|
|
603
|
+
- [ ] **Step 2: 构建并验证 MCP 协议**
|
|
604
|
+
|
|
605
|
+
Run: `npm run build`
|
|
606
|
+
|
|
607
|
+
验证 dream action:
|
|
608
|
+
```bash
|
|
609
|
+
printf '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}\n{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"fact_store","arguments":{"action":"dream"}}}\n' | node dist/server.js 2>/dev/null
|
|
610
|
+
```
|
|
611
|
+
Expected: 返回 dream report JSON
|
|
612
|
+
|
|
613
|
+
验证精简搜索格式:
|
|
614
|
+
```bash
|
|
615
|
+
printf '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}\n{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"fact_store","arguments":{"action":"search","query":"编码规范"}}}\n' | node dist/server.js 2>/dev/null
|
|
616
|
+
```
|
|
617
|
+
Expected: 返回包含 `display` 字段(而非完整 `content`)的精简结果
|
|
618
|
+
|
|
619
|
+
- [ ] **Step 3: 版本号更新并最终 Commit**
|
|
620
|
+
|
|
621
|
+
Run:
|
|
622
|
+
```bash
|
|
623
|
+
npm version patch --no-git-tag-version
|
|
624
|
+
git add -A
|
|
625
|
+
git commit -m "feat: memory dreaming — auto merge, compress, reclassify + compact search results"
|
|
626
|
+
```
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
## Context
|
|
2
|
+
|
|
3
|
+
mnemo-mcp 是一个基于 SQLite + FTS5 的结构化事实记忆系统。长期使用后,fact 库会出现以下问题:
|
|
4
|
+
- 多条 fact 内容重叠(如 5 条 Python 编码规范偏好,内容有交叉)
|
|
5
|
+
- 单条 fact 过长(超 500 字的"万能条",占 token、降低检索精度)
|
|
6
|
+
- 分类错误(identity 类里混了 workflow 内容)
|
|
7
|
+
- trust 评分不准确(`runLearning` 只看 feedback rate,忽略高频检索信号)
|
|
8
|
+
|
|
9
|
+
当前 `runLearning()` 仅调 trust 分,`runAudit()` 只读不改。没有内容层面的整理能力。
|
|
10
|
+
|
|
11
|
+
## Goals / Non-Goals
|
|
12
|
+
|
|
13
|
+
**Goals:**
|
|
14
|
+
- 定期自动整理 fact 库:合并重叠、压缩长文、修正分类
|
|
15
|
+
- 整理后搜索结果更精准、token 消耗更少
|
|
16
|
+
- 支持 CLI 命令 `mnemo dream` 手动触发,也支持 cron 定时
|
|
17
|
+
- 整理操作安全:合并前确认、可回滚
|
|
18
|
+
|
|
19
|
+
**Non-Goals:**
|
|
20
|
+
- 不做 AI 生成 summary(用规则提取关键句,不调 LLM)
|
|
21
|
+
- 不做跨项目记忆迁移
|
|
22
|
+
- 不做可视化 dashboard
|
|
23
|
+
- 不改变 MCP Resource 注入方式
|
|
24
|
+
|
|
25
|
+
## Decisions
|
|
26
|
+
|
|
27
|
+
### 1. 合并策略:Jaccard 相似度 > 0.6 且同 category
|
|
28
|
+
|
|
29
|
+
用现有的 `tokenizeForDedup()` + `jaccardSimilarity()` 计算两两相似度。同 category 内 Jaccard > 0.6 的 fact 对,合并为一条(保留 content 更长或 trust 更高的,删除另一条)。
|
|
30
|
+
|
|
31
|
+
**不跨 category 合并**:不同 category 的 fact 即使内容相似也保留(可能是不同语境)。
|
|
32
|
+
|
|
33
|
+
### 2. 长文压缩:规则提取关键句
|
|
34
|
+
|
|
35
|
+
content > 200 字且无 summary 的 fact,自动提取前 2 个完整句子作为 summary。不调 LLM,纯规则:
|
|
36
|
+
- 按中文句号(。)、英文句号(.)、换行符分割
|
|
37
|
+
- 取前 2 个句子(总长 ≤ 150 字)
|
|
38
|
+
|
|
39
|
+
### 3. 分类修正:关键词规则表
|
|
40
|
+
|
|
41
|
+
硬编码规则表,dream 时扫描并修正:
|
|
42
|
+
- 包含"角色设定/暖暖/身份" → identity
|
|
43
|
+
- 包含"编码规范/代码风格/pytest" → coding_style
|
|
44
|
+
- 包含"工作流/OpenSpec/工作流" → workflow
|
|
45
|
+
- 包含"偏好/VS Code/编辑器" → tool_pref
|
|
46
|
+
|
|
47
|
+
### 4. Dream report 输出格式
|
|
48
|
+
|
|
49
|
+
```
|
|
50
|
+
{
|
|
51
|
+
merged: 3,
|
|
52
|
+
compressed: 5,
|
|
53
|
+
reclassified: 2,
|
|
54
|
+
deleted: 4,
|
|
55
|
+
health: { total: 65, avg_trust: 0.62, avg_length: 180, coverage: { identity: 8, coding_style: 12, ... } }
|
|
56
|
+
}
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
### 5. 搜索结果精简
|
|
60
|
+
|
|
61
|
+
`search` 返回时:
|
|
62
|
+
- 有 summary → 返回 summary
|
|
63
|
+
- 无 summary → 返回 content 前 100 字 + "..."
|
|
64
|
+
- 始终返回 factId、category、trustScore、score
|
|
65
|
+
- 不返回完整 content(减少 token)
|
|
66
|
+
|
|
67
|
+
## Risks / Trade-offs
|
|
68
|
+
|
|
69
|
+
- **[误合并]** → Jaccard > 0.6 可能误判相似内容。缓解:合并前 log 记录,dream report 列出所有合并对
|
|
70
|
+
- **[summary 质量低]** → 规则提取的前 2 句可能不是最关键的。缓解:用户可手动 update summary
|
|
71
|
+
- **[不可逆删除]** → 合并后删除冗余 fact 是不可逆的。缓解:dream 前自动备份数据库
|