@morningljn/mnemo 0.1.4 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,932 @@
1
+ # Memory Self-Learning Implementation Plan
2
+
3
+ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4
+
5
+ **Goal:** Improve mnemo-mcp retrieval accuracy by reverting v3 dynamic weights, adding length penalty for super-long facts, summary field for data quality, retrieval log for self-learning, and learn/audit actions for automatic trust adjustment.
6
+
7
+ **Architecture:** Revert retriever to static 0.5/0.5 FTS/Jaccard weights. Add length penalty based on matchText (summary or content). New retrieval_log table auto-records every search. New learn action adjusts trust based on retrieval/helpful stats. New audit action reports data quality. Summary field lets long facts provide short matching text.
8
+
9
+ **Tech Stack:** TypeScript, Node.js, better-sqlite3, @modelcontextprotocol/sdk, zod/v4, vitest
10
+
11
+ ---
12
+
13
+ ## File Structure
14
+
15
+ | File | Responsibility |
16
+ |------|---------------|
17
+ | `src/schema.ts` | **MODIFY** — Add retrieval_log table, FTS5 with summary column, update triggers |
18
+ | `src/store.ts` | **MODIFY** — Add summary/last_retrieved_at columns, logRetrieval, pruneRetrievalLog, runLearning, runAudit |
19
+ | `src/retriever.ts` | **MODIFY** — Revert dynamic weights, remove gate, add length penalty, summary matching |
20
+ | `src/server.ts` | **MODIFY** — Add learn/audit handlers, summary support, length warning, auto-learn on start |
21
+ | `src/types.ts` | **MODIFY** — Add summary/last_retrieved_at to Fact, learn/audit action types |
22
+ | `tests/store.test.ts` | **MODIFY** — New tests for retrieval_log, learning, audit |
23
+ | `tests/retriever.test.ts` | **MODIFY** — New tests for length penalty, summary, static weights |
24
+
25
+ ---
26
+
27
+ ## Task 1: Schema Migration — retrieval_log + summary + last_retrieved_at + FTS5 rebuild
28
+
29
+ **Files:**
30
+ - Modify: `src/schema.ts`
31
+ - Modify: `src/store.ts` (migrateSchema)
32
+ - Test: `tests/store.test.ts`
33
+
34
+ - [ ] **Step 1: Write failing test for new columns**
35
+
36
+ Append to `tests/store.test.ts`:
37
+
38
+ ```typescript
39
+ describe('schema migration', () => {
40
+ it('has summary column on facts table', () => {
41
+ const cols = store.connection.pragma('table_info(facts)') as Array<{ name: string }>
42
+ const colNames = cols.map(c => c.name)
43
+ expect(colNames).toContain('summary')
44
+ expect(colNames).toContain('last_retrieved_at')
45
+ })
46
+
47
+ it('has retrieval_log table', () => {
48
+ const tables = store.connection.prepare(
49
+ "SELECT name FROM sqlite_master WHERE type='table' AND name='retrieval_log'"
50
+ ).get()
51
+ expect(tables).toBeTruthy()
52
+ })
53
+
54
+ it('summary defaults to null', () => {
55
+ const id = store.addFact('test summary fact', 'general')
56
+ const row = store.connection.prepare('SELECT summary FROM facts WHERE fact_id = ?').get(id) as { summary: string | null }
57
+ expect(row.summary).toBeNull()
58
+ })
59
+ })
60
+ ```
61
+
62
+ - [ ] **Step 2: Run test to verify it fails**
63
+
64
+ Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/store.test.ts -t "schema migration"`
65
+ Expected: FAIL — summary column doesn't exist yet
66
+
67
+ - [ ] **Step 3: Update `src/schema.ts` — add retrieval_log table, rebuild FTS5 with summary**
68
+
69
+ Replace entire file content:
70
+
71
+ ```typescript
72
+ export const SCHEMA = `
73
+ -- 事实表
74
+ CREATE TABLE IF NOT EXISTS facts (
75
+ fact_id INTEGER PRIMARY KEY AUTOINCREMENT,
76
+ content TEXT NOT NULL UNIQUE,
77
+ category TEXT DEFAULT 'general',
78
+ tags TEXT DEFAULT '',
79
+ keywords TEXT DEFAULT '[]',
80
+ trust_score REAL DEFAULT 0.5,
81
+ retrieval_count INTEGER DEFAULT 0,
82
+ helpful_count INTEGER DEFAULT 0,
83
+ created_at TEXT DEFAULT (datetime('now', 'localtime')),
84
+ updated_at TEXT DEFAULT (datetime('now', 'localtime'))
85
+ );
86
+
87
+ -- 实体表
88
+ CREATE TABLE IF NOT EXISTS entities (
89
+ entity_id INTEGER PRIMARY KEY AUTOINCREMENT,
90
+ name TEXT NOT NULL,
91
+ entity_type TEXT DEFAULT 'unknown',
92
+ aliases TEXT DEFAULT '',
93
+ created_at TEXT DEFAULT (datetime('now', 'localtime'))
94
+ );
95
+
96
+ -- 事实-实体关联表
97
+ CREATE TABLE IF NOT EXISTS fact_entities (
98
+ fact_id INTEGER NOT NULL REFERENCES facts(fact_id) ON DELETE CASCADE,
99
+ entity_id INTEGER NOT NULL REFERENCES entities(entity_id) ON DELETE CASCADE,
100
+ PRIMARY KEY (fact_id, entity_id)
101
+ );
102
+
103
+ -- 检索日志表
104
+ CREATE TABLE IF NOT EXISTS retrieval_log (
105
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
106
+ query TEXT NOT NULL,
107
+ results TEXT NOT NULL,
108
+ timestamp TEXT DEFAULT (datetime('now', 'localtime'))
109
+ );
110
+
111
+ -- 索引
112
+ CREATE INDEX IF NOT EXISTS idx_facts_trust ON facts(trust_score DESC);
113
+ CREATE INDEX IF NOT EXISTS idx_facts_category ON facts(category);
114
+ CREATE INDEX IF NOT EXISTS idx_entities_name ON entities(name);
115
+ CREATE INDEX IF NOT EXISTS idx_fact_entities_entity ON fact_entities(entity_id);
116
+ CREATE INDEX IF NOT EXISTS idx_retrieval_log_ts ON retrieval_log(timestamp);
117
+
118
+ -- FTS5 全文索引(含 summary 列)
119
+ CREATE VIRTUAL TABLE IF NOT EXISTS facts_fts
120
+ USING fts5(content, tags, summary, content=facts, content_rowid=fact_id);
121
+
122
+ -- FTS5 同步触发器:插入
123
+ CREATE TRIGGER IF NOT EXISTS facts_ai AFTER INSERT ON facts BEGIN
124
+ INSERT INTO facts_fts(rowid, content, tags, summary)
125
+ VALUES (new.fact_id, new.content, new.tags, COALESCE(new.summary, ''));
126
+ END;
127
+
128
+ -- FTS5 同步触发器:删除
129
+ CREATE TRIGGER IF NOT EXISTS facts_ad AFTER DELETE ON facts BEGIN
130
+ INSERT INTO facts_fts(facts_fts, rowid, content, tags, summary)
131
+ VALUES ('delete', old.fact_id, old.content, old.tags, COALESCE(old.summary, ''));
132
+ END;
133
+
134
+ -- FTS5 同步触发器:更新
135
+ CREATE TRIGGER IF NOT EXISTS facts_au AFTER UPDATE ON facts BEGIN
136
+ INSERT INTO facts_fts(facts_fts, rowid, content, tags, summary)
137
+ VALUES ('delete', old.fact_id, old.content, old.tags, COALESCE(old.summary, ''));
138
+ INSERT INTO facts_fts(rowid, content, tags, summary)
139
+ VALUES (new.fact_id, new.content, new.tags, COALESCE(new.summary, ''));
140
+ END;
141
+ `
142
+ ```
143
+
144
+ - [ ] **Step 4: Update `src/store.ts` migrateSchema — add columns + rebuild FTS5**
145
+
146
+ Replace the `migrateSchema()` method in `src/store.ts` (lines 112-117):
147
+
148
+ ```typescript
149
+ /** 增量迁移:添加新列/新表(已存在则跳过) */
150
+ private migrateSchema(): void {
151
+ const addColumn = (sql: string) => {
152
+ try { this.db.exec(sql) } catch { /* 列已存在 */ }
153
+ }
154
+
155
+ addColumn("ALTER TABLE facts ADD COLUMN keywords TEXT DEFAULT '[]'")
156
+ addColumn('ALTER TABLE facts ADD COLUMN summary TEXT DEFAULT NULL')
157
+ addColumn('ALTER TABLE facts ADD COLUMN last_retrieved_at TEXT DEFAULT NULL')
158
+
159
+ // 重建 FTS5 以包含 summary 列(仅在 facts_fts 无 summary 列时执行)
160
+ try {
161
+ this.db.prepare("SELECT summary FROM facts_fts LIMIT 0").get()
162
+ } catch {
163
+ // facts_fts 没有 summary 列,需要重建
164
+ this.db.exec('DROP TABLE IF EXISTS facts_fts')
165
+ this.db.exec('DROP TRIGGER IF EXISTS facts_ai')
166
+ this.db.exec('DROP TRIGGER IF EXISTS facts_ad')
167
+ this.db.exec('DROP TRIGGER IF EXISTS facts_au')
168
+ // 重建会由 SCHEMA 中的 CREATE IF NOT EXISTS 处理
169
+ this.db.exec(`
170
+ CREATE VIRTUAL TABLE IF NOT EXISTS facts_fts
171
+ USING fts5(content, tags, summary, content=facts, content_rowid=fact_id)
172
+ `)
173
+ // 重建触发器
174
+ this.db.exec(`CREATE TRIGGER IF NOT EXISTS facts_ai AFTER INSERT ON facts BEGIN
175
+ INSERT INTO facts_fts(rowid, content, tags, summary)
176
+ VALUES (new.fact_id, new.content, new.tags, COALESCE(new.summary, ''));
177
+ END`)
178
+ this.db.exec(`CREATE TRIGGER IF NOT EXISTS facts_ad AFTER DELETE ON facts BEGIN
179
+ INSERT INTO facts_fts(facts_fts, rowid, content, tags, summary)
180
+ VALUES ('delete', old.fact_id, old.content, old.tags, COALESCE(old.summary, ''));
181
+ END`)
182
+ this.db.exec(`CREATE TRIGGER IF NOT EXISTS facts_au AFTER UPDATE ON facts BEGIN
183
+ INSERT INTO facts_fts(facts_fts, rowid, content, tags, summary)
184
+ VALUES ('delete', old.fact_id, old.content, old.tags, COALESCE(old.summary, ''));
185
+ INSERT INTO facts_fts(rowid, content, tags, summary)
186
+ VALUES (new.fact_id, new.content, new.tags, COALESCE(new.summary, ''));
187
+ END`)
188
+ // 重新填充 FTS5
189
+ this.db.exec(`INSERT INTO facts_fts(rowid, content, tags, summary)
190
+ SELECT fact_id, content, tags, COALESCE(summary, '') FROM facts`)
191
+ }
192
+ }
193
+ ```
194
+
195
+ - [ ] **Step 5: Update `src/types.ts` — add summary and last_retrieved_at to Fact**
196
+
197
+ In `src/types.ts`, update the `Fact` interface (lines 5-16):
198
+
199
+ ```typescript
200
+ /** 存储的事实记录 */
201
+ export interface Fact {
202
+ factId: number
203
+ content: string
204
+ summary: string | null
205
+ category: FactCategory
206
+ tags: string
207
+ keywords: string
208
+ trustScore: number
209
+ retrievalCount: number
210
+ helpfulCount: number
211
+ createdAt: string
212
+ updatedAt: string
213
+ lastRetrievedAt: string | null
214
+ }
215
+ ```
216
+
217
+ Update `FactStoreArgs` action union to include new actions (line 56):
218
+
219
+ ```typescript
220
+ export interface FactStoreArgs {
221
+ action: 'add' | 'search' | 'probe' | 'related' | 'reason' | 'contradict' | 'update' | 'remove' | 'list' | 'learn' | 'audit'
222
+ content?: string | string[]
223
+ summary?: string
224
+ query?: string
225
+ entity?: string
226
+ entities?: string[]
227
+ fact_id?: number | number[]
228
+ category?: string
229
+ tags?: string
230
+ trust_delta?: number
231
+ min_trust?: number
232
+ limit?: number
233
+ }
234
+ ```
235
+
236
+ - [ ] **Step 6: Update `src/store.ts` rowToFact — add new fields**
237
+
238
+ Replace `rowToFact` method (lines 791-803):
239
+
240
+ ```typescript
241
+ private rowToFact(row: FactRow): Fact {
242
+ return {
243
+ factId: row.fact_id,
244
+ content: row.content,
245
+ summary: (row as any).summary ?? null,
246
+ category: row.category as FactCategory,
247
+ tags: row.tags,
248
+ keywords: row.keywords,
249
+ trustScore: row.trust_score,
250
+ retrievalCount: row.retrieval_count,
251
+ helpfulCount: row.helpful_count,
252
+ createdAt: row.created_at,
253
+ updatedAt: row.updated_at,
254
+ lastRetrievedAt: (row as any).last_retrieved_at ?? null,
255
+ }
256
+ }
257
+ ```
258
+
259
+ Update all SQL SELECT statements in store.ts that query facts to include `summary` and `last_retrieved_at`. For `listFacts` (line 363):
260
+
261
+ ```sql
262
+ SELECT fact_id, content, summary, category, tags, keywords, trust_score,
263
+ retrieval_count, helpful_count, created_at, updated_at, last_retrieved_at
264
+ FROM facts
265
+ ```
266
+
267
+ Apply the same pattern to `getFactsByEntity`, `getFactsByEntities`, `updateFact`, and `recordFeedback` queries.
268
+
269
+ - [ ] **Step 7: Run schema migration tests**
270
+
271
+ Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/store.test.ts -t "schema migration"`
272
+ Expected: PASS
273
+
274
+ - [ ] **Step 8: Run full build**
275
+
276
+ Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npm run build`
277
+ Expected: BUILD OK
278
+
279
+ - [ ] **Step 9: Commit**
280
+
281
+ ```bash
282
+ cd /Users/ljn/Documents/demo/ocean/mnemo-mcp
283
+ git add src/schema.ts src/store.ts src/types.ts tests/store.test.ts
284
+ git commit -m "feat(store): add retrieval_log table, summary/last_retrieved_at columns, FTS5 rebuild"
285
+ ```
286
+
287
+ ---
288
+
289
+ ## Task 2: Store — logRetrieval + pruneRetrievalLog + runLearning + runAudit
290
+
291
+ **Files:**
292
+ - Modify: `src/store.ts`
293
+ - Test: `tests/store.test.ts`
294
+
295
+ - [ ] **Step 1: Write failing tests**
296
+
297
+ Append to `tests/store.test.ts`:
298
+
299
+ ```typescript
300
+ describe('logRetrieval', () => {
301
+ it('writes retrieval log with results', () => {
302
+ const id = store.addFact('test fact for logging', 'general')
303
+ store.logRetrieval('test query', [{ id, score: 0.8 }])
304
+ const rows = store.connection.prepare('SELECT * FROM retrieval_log').all() as Array<any>
305
+ expect(rows.length).toBe(1)
306
+ expect(rows[0].query).toBe('test query')
307
+ expect(JSON.parse(rows[0].results)).toEqual([{ id, score: 0.8 }])
308
+ })
309
+
310
+ it('updates last_retrieved_at for returned facts', () => {
311
+ const id = store.addFact('test fact for timestamp', 'general')
312
+ store.logRetrieval('query', [{ id, score: 0.5 }])
313
+ const row = store.connection.prepare('SELECT last_retrieved_at FROM facts WHERE fact_id = ?').get(id) as any
314
+ expect(row.last_retrieved_at).not.toBeNull()
315
+ })
316
+
317
+ it('prunes log to max entries', () => {
318
+ for (let i = 0; i < 12; i++) {
319
+ store.logRetrieval(`query ${i}`, [])
320
+ }
321
+ store.pruneRetrievalLog(10)
322
+ const count = (store.connection.prepare('SELECT COUNT(*) as c FROM retrieval_log').get() as any).c
323
+ expect(count).toBe(10)
324
+ })
325
+ })
326
+
327
+ describe('runLearning', () => {
328
+ it('demotes high retrieval low helpful facts', () => {
329
+ const id = store.addFact('demote me', 'general')
330
+ // Simulate 100 retrievals, 2 helpful
331
+ store.connection.prepare('UPDATE facts SET retrieval_count = 100, helpful_count = 2, trust_score = 1.0 WHERE fact_id = ?').run(id)
332
+ const result = store.runLearning()
333
+ const row = store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any
334
+ expect(row.trust_score).toBeLessThan(1.0)
335
+ expect(result.demoted).toBeGreaterThanOrEqual(1)
336
+ })
337
+
338
+ it('promotes high helpful rate facts', () => {
339
+ const id = store.addFact('promote me', 'general')
340
+ store.connection.prepare('UPDATE facts SET retrieval_count = 50, helpful_count = 20, trust_score = 0.5 WHERE fact_id = ?').run(id)
341
+ store.runLearning()
342
+ const row = store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any
343
+ expect(row.trust_score).toBeGreaterThan(0.5)
344
+ })
345
+
346
+ it('does not adjust facts with low retrieval count', () => {
347
+ const id = store.addFact('new fact', 'general')
348
+ store.connection.prepare('UPDATE facts SET retrieval_count = 5, helpful_count = 0, trust_score = 0.8 WHERE fact_id = ?').run(id)
349
+ store.runLearning()
350
+ const row = store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any
351
+ expect(row.trust_score).toBe(0.8)
352
+ })
353
+
354
+ it('ages facts not retrieved for 60 days', () => {
355
+ const id = store.addFact('old fact', 'general')
356
+ store.connection.prepare("UPDATE facts SET last_retrieved_at = datetime('now', '-61 days'), trust_score = 0.8 WHERE fact_id = ?").run(id)
357
+ store.runLearning()
358
+ const row = store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any
359
+ expect(row.trust_score).toBeLessThan(0.8)
360
+ })
361
+
362
+ it('protects new facts with null last_retrieved_at from aging', () => {
363
+ const id = store.addFact('brand new fact', 'general')
364
+ store.connection.prepare('UPDATE facts SET trust_score = 0.5, last_retrieved_at = NULL WHERE fact_id = ?').run(id)
365
+ store.runLearning()
366
+ const row = store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any
367
+ expect(row.trust_score).toBe(0.5)
368
+ })
369
+
370
+ it('returns long_facts report', () => {
371
+ const id = store.addFact('x'.repeat(600), 'general')
372
+ const result = store.runLearning()
373
+ expect(result.long_facts.length).toBeGreaterThanOrEqual(1)
374
+ expect(result.long_facts.some((f: any) => f.id === id)).toBe(true)
375
+ })
376
+ })
377
+
378
+ describe('runAudit', () => {
379
+ it('returns quality report without modifying data', () => {
380
+ const id = store.addFact('a'.repeat(600), 'general')
381
+ store.connection.prepare('UPDATE facts SET retrieval_count = 100, helpful_count = 1 WHERE fact_id = ?').run(id)
382
+ const before = (store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any).trust_score
383
+ const report = store.runAudit()
384
+ const after = (store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any).trust_score
385
+ expect(before).toBe(after) // audit does not modify
386
+ expect(report.total_facts).toBeGreaterThanOrEqual(1)
387
+ expect(report.long_without_summary.length).toBeGreaterThanOrEqual(1)
388
+ })
389
+ })
390
+ ```
391
+
392
+ - [ ] **Step 2: Run test to verify it fails**
393
+
394
+ Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/store.test.ts -t "logRetrieval|runLearning|runAudit"`
395
+ Expected: FAIL — methods don't exist yet
396
+
397
+ - [ ] **Step 3: Add methods to `src/store.ts`**
398
+
399
+ Add these methods to the `MemoryStore` class, after the `getTotalCount()` method (after line 636):
400
+
401
+ ```typescript
402
+ /** 记录检索日志并更新 last_retrieved_at */
403
+ logRetrieval(query: string, results: Array<{ id: number; score: number }>): void {
404
+ const resultsJson = JSON.stringify(results)
405
+ this.db.prepare(
406
+ "INSERT INTO retrieval_log (query, results) VALUES (?, ?)"
407
+ ).run(query, resultsJson)
408
+
409
+ // 更新返回 fact 的 last_retrieved_at
410
+ if (results.length > 0) {
411
+ const ids = results.map(r => r.id)
412
+ const placeholders = ids.map(() => '?').join(',')
413
+ this.db.prepare(
414
+ `UPDATE facts SET last_retrieved_at = datetime('now', 'localtime') WHERE fact_id IN (${placeholders})`
415
+ ).run(...ids)
416
+ }
417
+
418
+ // 自动清理日志
419
+ this.pruneRetrievalLog(5000)
420
+ }
421
+
422
+ /** 清理检索日志,保留最近 maxEntries 条 */
423
+ pruneRetrievalLog(maxEntries = 5000): void {
424
+ this.db.prepare(
425
+ `DELETE FROM retrieval_log WHERE id NOT IN (
426
+ SELECT id FROM retrieval_log ORDER BY id DESC LIMIT ?
427
+ )`
428
+ ).run(maxEntries)
429
+ }
430
+
431
+ /** 自学习:基于检索统计自动调整 trust_score */
432
+ runLearning(): {
433
+ promoted: number
434
+ demoted: number
435
+ aged: number
436
+ unchanged: number
437
+ long_facts: Array<{ id: number; content_length: number; penalty: number; has_summary: boolean }>
438
+ } {
439
+ const rows = this.db.prepare(
440
+ 'SELECT fact_id, content, summary, retrieval_count, helpful_count, trust_score, last_retrieved_at FROM facts'
441
+ ).all() as Array<{
442
+ fact_id: number; content: string; summary: string | null;
443
+ retrieval_count: number; helpful_count: number; trust_score: number; last_retrieved_at: string | null
444
+ }>
445
+
446
+ let promoted = 0
447
+ let demoted = 0
448
+ let aged = 0
449
+ let unchanged = 0
450
+ const longFacts: Array<{ id: number; content_length: number; penalty: number; has_summary: boolean }> = []
451
+
452
+ const now = Date.now()
453
+
454
+ for (const row of rows) {
455
+ let changed = false
456
+ const rate = row.retrieval_count > 0 ? row.helpful_count / row.retrieval_count : 0
457
+
458
+ // Rate-based adjustment (需要 30+ 次检索)
459
+ if (row.retrieval_count > 30) {
460
+ if (rate < 0.05) {
461
+ const newTrust = clampTrust(row.trust_score * 0.9)
462
+ this.db.prepare('UPDATE facts SET trust_score = ? WHERE fact_id = ?').run(newTrust, row.fact_id)
463
+ demoted++
464
+ changed = true
465
+ } else if (rate > 0.3) {
466
+ const newTrust = clampTrust(row.trust_score + 0.05)
467
+ this.db.prepare('UPDATE facts SET trust_score = ? WHERE fact_id = ?').run(newTrust, row.fact_id)
468
+ promoted++
469
+ changed = true
470
+ }
471
+ }
472
+
473
+ // Aging (60 天未检索)
474
+ if (row.last_retrieved_at) {
475
+ const lastRetrieved = new Date(row.last_retrieved_at + 'Z').getTime()
476
+ const daysSinceRetrieval = (now - lastRetrieved) / 86_400_000
477
+ if (daysSinceRetrieval > 60) {
478
+ const currentTrust = this.db.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(row.fact_id) as any
479
+ const newTrust = clampTrust(currentTrust.trust_score * 0.95)
480
+ this.db.prepare('UPDATE facts SET trust_score = ? WHERE fact_id = ?').run(newTrust, row.fact_id)
481
+ aged++
482
+ changed = true
483
+ }
484
+ }
485
+ // last_retrieved_at 为 NULL = 新 fact,不老化
486
+
487
+ if (!changed) unchanged++
488
+
489
+ // Long facts report (content > 300 字无 summary)
490
+ const matchLength = row.summary ? row.summary.length : row.content.length
491
+ if (matchLength > 300) {
492
+ longFacts.push({
493
+ id: row.fact_id,
494
+ content_length: row.content.length,
495
+ penalty: Math.min(1.0, 300 / matchLength),
496
+ has_summary: !!row.summary,
497
+ })
498
+ }
499
+ }
500
+
501
+ return { promoted, demoted, aged, unchanged, long_facts: longFacts }
502
+ }
503
+
504
+ /** 数据质量审计(只读,不修改数据) */
505
+ runAudit(): {
506
+ total_facts: number
507
+ long_without_summary: Array<{ id: number; content_length: number }>
508
+ low_helpful_rate: Array<{ id: number; rate: number; retrieval_count: number }>
509
+ aging_candidates: Array<{ id: number; last_retrieved_at: string | null }>
510
+ } {
511
+ const rows = this.db.prepare(
512
+ 'SELECT fact_id, content, summary, retrieval_count, helpful_count, last_retrieved_at FROM facts'
513
+ ).all() as Array<{
514
+ fact_id: number; content: string; summary: string | null;
515
+ retrieval_count: number; helpful_count: number; last_retrieved_at: string | null
516
+ }>
517
+
518
+ const longWithoutSummary: Array<{ id: number; content_length: number }> = []
519
+ const lowHelpfulRate: Array<{ id: number; rate: number; retrieval_count: number }> = []
520
+ const agingCandidates: Array<{ id: number; last_retrieved_at: string | null }> = []
521
+
522
+ const now = Date.now()
523
+
524
+ for (const row of rows) {
525
+ // 超 500 字无 summary
526
+ if (row.content.length > 500 && !row.summary) {
527
+ longWithoutSummary.push({ id: row.fact_id, content_length: row.content.length })
528
+ }
529
+
530
+ // 低 helpful 率(>30 次检索,rate < 5%)
531
+ if (row.retrieval_count > 30) {
532
+ const rate = row.helpful_count / row.retrieval_count
533
+ if (rate < 0.05) {
534
+ lowHelpfulRate.push({ id: row.fact_id, rate: Math.round(rate * 1000) / 1000, retrieval_count: row.retrieval_count })
535
+ }
536
+ }
537
+
538
+ // 老化候选(>60 天未检索)
539
+ if (row.last_retrieved_at) {
540
+ const lastRetrieved = new Date(row.last_retrieved_at + 'Z').getTime()
541
+ const daysSince = (now - lastRetrieved) / 86_400_000
542
+ if (daysSince > 60) {
543
+ agingCandidates.push({ id: row.fact_id, last_retrieved_at: row.last_retrieved_at })
544
+ }
545
+ }
546
+ }
547
+
548
+ return {
549
+ total_facts: rows.length,
550
+ long_without_summary: longWithoutSummary,
551
+ low_helpful_rate: lowHelpfulRate,
552
+ aging_candidates: agingCandidates,
553
+ }
554
+ }
555
+ ```
556
+
557
+ - [ ] **Step 4: Run tests**
558
+
559
+ Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/store.test.ts -t "logRetrieval|runLearning|runAudit"`
560
+ Expected: ALL PASS
561
+
562
+ - [ ] **Step 5: Commit**
563
+
564
+ ```bash
565
+ cd /Users/ljn/Documents/demo/ocean/mnemo-mcp
566
+ git add src/store.ts tests/store.test.ts
567
+ git commit -m "feat(store): add logRetrieval, pruneRetrievalLog, runLearning, runAudit"
568
+ ```
569
+
570
+ ---
571
+
572
+ ## Task 3: Retriever — revert dynamic weights + length penalty + summary matching
573
+
574
+ **Files:**
575
+ - Modify: `src/retriever.ts`
576
+ - Test: `tests/retriever.test.ts`
577
+
578
+ - [ ] **Step 1: Write failing tests**
579
+
580
+ Append to `tests/retriever.test.ts`:
581
+
582
+ ```typescript
583
+ describe('static weights (no dynamic)', () => {
584
+ it('uses same weights for short and long queries', () => {
585
+ store.addFact('用户偏好 VS Code 编辑器', 'tool_pref')
586
+ const shortResults = retriever.search('VS Code')
587
+ const longResults = retriever.search('为什么 VS Code 编辑器总是报错说找不到模块')
588
+ // Both should return the same fact — static weights don't change by query length
589
+ expect(shortResults.some(r => r.content.includes('VS Code'))).toBe(true)
590
+ expect(longResults.some(r => r.content.includes('VS Code'))).toBe(true)
591
+ })
592
+ })
593
+
594
+ describe('length penalty', () => {
595
+ it('penalizes long facts without summary', () => {
596
+ const longContent = '用户偏好 ' + '详细说明'.repeat(200) // ~800 chars
597
+ store.addFact(longContent, 'tool_pref')
598
+ store.addFact('用户偏好 VS Code', 'tool_pref')
599
+ const results = retriever.search('用户偏好')
600
+ // Short fact should rank higher
601
+ if (results.length >= 2) {
602
+ const shortFact = results.find(r => r.content === '用户偏好 VS Code')
603
+ const longFact = results.find(r => r.content.length > 500)
604
+ if (shortFact && longFact) {
605
+ expect(shortFact.score).toBeGreaterThan(longFact.score)
606
+ }
607
+ }
608
+ })
609
+
610
+ it('does not penalize long facts with short summary', () => {
611
+ const longContent = '详细内容' + '补充说明'.repeat(200)
612
+ // Add via SQL to set summary
613
+ const id = store.addFact(longContent, 'general')
614
+ store.connection.prepare('UPDATE facts SET summary = ? WHERE fact_id = ?').run('用户偏好', id)
615
+ store.addFact('完全无关的内容', 'general')
616
+ const results = retriever.search('用户偏好')
617
+ const summaryFact = results.find(r => r.factId === id)
618
+ expect(summaryFact).toBeTruthy()
619
+ })
620
+ })
621
+
622
+ describe('no relevance gate', () => {
623
+ it('returns results even with low scores', () => {
624
+ store.addFact('完全不相关关于天气', 'general')
625
+ const results = retriever.search('天气')
626
+ // Should not be filtered out by a 0.15 threshold
627
+ expect(results.length).toBeGreaterThanOrEqual(1)
628
+ })
629
+ })
630
+ ```
631
+
632
+ - [ ] **Step 2: Run test to verify it fails**
633
+
634
+ Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/retriever.test.ts -t "static weights|length penalty|no relevance gate"`
635
+ Expected: Some tests FAIL (dynamic weights still active, no length penalty)
636
+
637
+ - [ ] **Step 3: Modify `src/retriever.ts` search() method**
638
+
639
+ Replace lines 116-175 (the scoring and filtering section) in `search()`:
640
+
641
+ ```typescript
642
+ // Stage 2-4: Jaccard 重排序 + 信任评分 + 时间衰减
643
+ // 静态权重(回退 v3 动态权重)
644
+ const queryTokens = this.tokenize(searchQuery)
645
+
646
+ const scored: ScoredFact[] = []
647
+
648
+ for (const fact of candidates) {
649
+ // summary 优先用于匹配
650
+ const matchText = fact.summary ?? fact.content
651
+ const matchTokens = this.tokenize(matchText)
652
+ const tagTokens = this.tokenize(fact.tags)
653
+ const allTokens = new Set([...matchTokens, ...tagTokens])
654
+
655
+ const jaccard = this.jaccardSimilarity(queryTokens, allTokens)
656
+ const qInF = this.containmentScore(queryTokens, allTokens)
657
+ const similarity = 0.3 * jaccard + 0.7 * qInF
658
+ const ftsScore = fact.ftsRank
659
+
660
+ // 静态权重 0.5/0.5
661
+ const relevance = 0.5 * ftsScore + 0.5 * similarity
662
+ let score = relevance * fact.trustScore
663
+
664
+ // 时间衰减
665
+ if (this.halfLifeDays > 0) {
666
+ score *= this.temporalDecay(fact.updatedAt || fact.createdAt)
667
+ }
668
+
669
+ // Length penalty:基于 matchText 长度
670
+ score *= Math.min(1.0, 300 / matchText.length)
671
+
672
+ scored.push({ ...fact, score })
673
+ }
674
+
675
+ scored.sort((a, b) => b.score - a.score)
676
+
677
+ // 取 limit 条(不再做 relevance gate 和 content dedup)
678
+ const results = scored.slice(0, limit)
679
+
680
+ // 检索追踪:递增 retrieval_count + top3 信任刷新
681
+ if (results.length > 0) {
682
+ this.trackRetrieval(results)
683
+ }
684
+ ```
685
+
686
+ Note: The `trackRetrieval` call at the end stays the same. Remove the `RELEVANCE_THRESHOLD` constant and the content dedup loop entirely.
687
+
688
+ - [ ] **Step 4: Update `store.ts` `logRetrieval` call — add retrieval logging**
689
+
690
+ After the `trackRetrieval(results)` call in `search()`, add:
691
+
692
+ ```typescript
693
+ // 记录检索日志
694
+ this.store.logRetrieval(searchQuery, results.map(r => ({ id: r.factId, score: Math.round(r.score * 1000) / 1000 })))
695
+ ```
696
+
697
+ But wrap it so it doesn't log when cache hits (the return before this point). The `logRetrieval` should only be called on cache miss. Place it right before the cache set at line ~182.
698
+
699
+ - [ ] **Step 5: Update ftsCandidates to search summary field**
700
+
701
+ In `ftsCandidates()` method, the SQL query (line ~483) joins `facts_fts` which now includes the `summary` column. FTS5 will automatically match against all indexed columns (content, tags, summary). No SQL change needed — the FTS5 virtual table now includes summary.
702
+
703
+ However, update the `ftsCandidates` return to include `summary` from the joined row. Add to the mapping (around line 506):
704
+
705
+ ```typescript
706
+ summary: String(row.summary ?? ''),
707
+ ```
708
+
709
+ - [ ] **Step 6: Run tests**
710
+
711
+ Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/retriever.test.ts`
712
+ Expected: ALL PASS (including new tests)
713
+
714
+ - [ ] **Step 7: Commit**
715
+
716
+ ```bash
717
+ cd /Users/ljn/Documents/demo/ocean/mnemo-mcp
718
+ git add src/retriever.ts tests/retriever.test.ts
719
+ git commit -m "feat(retriever): revert dynamic weights, add length penalty, summary matching, remove relevance gate"
720
+ ```
721
+
722
+ ---
723
+
724
+ ## Task 4: Server — learn/audit handlers + summary support + length warning + auto-learn
725
+
726
+ **Files:**
727
+ - Modify: `src/server.ts`
728
+
729
+ - [ ] **Step 1: Update factStoreSchema — add learn/audit actions and summary param**
730
+
731
+ Replace `factStoreSchema` (lines 29-41):
732
+
733
+ ```typescript
734
+ const factStoreSchema = {
735
+ action: z.enum(['add', 'search', 'probe', 'related', 'reason', 'contradict', 'update', 'remove', 'list', 'learn', 'audit']),
736
+ content: z.union([z.string(), z.array(z.string())]).optional().describe("事实内容('add' 必需,支持批量)"),
737
+ summary: z.string().optional().describe('超长事实的摘要(检索用 summary 匹配)'),
738
+ query: z.string().optional().describe("搜索查询('search' 必需)"),
739
+ entity: z.string().optional().describe("实体名('probe'/'related' 使用)"),
740
+ entities: z.array(z.string()).optional().describe("实体列表('reason' 使用)"),
741
+ fact_id: z.union([z.number(), z.array(z.number())]).optional().describe("事实 ID('update'/'remove' 使用,支持批量)"),
742
+ category: z.enum(['identity', 'coding_style', 'tool_pref', 'workflow', 'general']).optional(),
743
+ tags: z.string().optional().describe('逗号分隔标签'),
744
+ trust_delta: z.number().optional().describe("'update' 的信任调整值"),
745
+ min_trust: z.number().optional().describe('最低信任过滤(默认 0.3)'),
746
+ limit: z.number().optional().describe('最大结果数(默认 10)'),
747
+ }
748
+ ```
749
+
750
+ - [ ] **Step 2: Update add handler — summary support + 500-char warning**
751
+
752
+ Replace the `case 'add':` block (lines 82-112). After the existing `for` loop but before cache clear, add summary handling. In the `for` loop, when adding a new fact, also save summary:
753
+
754
+ After `store.addFact(content, category, a.tags ?? '')` for new facts, add:
755
+ ```typescript
756
+ if (a.summary) {
757
+ store.connection.prepare('UPDATE facts SET summary = ? WHERE fact_id = ?').run(a.summary, factId)
758
+ }
759
+ ```
760
+
761
+ And before the results push for both similar and new cases, add the 500-char warning:
762
+ ```typescript
763
+ if (content.length > 500 && !a.summary) {
764
+ warnings = [...(warnings ?? []), 'content 超过 500 字,建议提供 summary 或拆分为多条 fact']
765
+ }
766
+ ```
767
+
768
+ Also trigger FTS reindex after summary update by deleting and reinserting into facts_fts.
769
+
770
+ - [ ] **Step 3: Update update handler — summary support**
771
+
772
+ In `case 'update':`, after `store.updateFact(...)`, add summary update:
773
+ ```typescript
774
+ if (a.summary !== undefined) {
775
+ store.connection.prepare('UPDATE facts SET summary = ? WHERE fact_id = ?').run(a.summary, a.fact_id as number)
776
+ // Trigger FTS reindex
777
+ store.connection.prepare(
778
+ "INSERT INTO facts_fts(facts_fts, rowid, content, tags, summary) VALUES ('delete', ?, '', '', '')"
779
+ ).run(a.fact_id as number)
780
+ const row = store.connection.prepare('SELECT content, tags, summary FROM facts WHERE fact_id = ?').get(a.fact_id as number) as any
781
+ store.connection.prepare(
782
+ 'INSERT INTO facts_fts(rowid, content, tags, summary) VALUES (?, ?, ?, ?)'
783
+ ).run(a.fact_id as number, row.content, row.tags, row.summary ?? '')
784
+ }
785
+ ```
786
+
787
+ Actually, the FTS update trigger should handle this automatically if we UPDATE the facts table. But the trigger uses `COALESCE(new.summary, '')` which is correct. Let's use `updateFact` to set summary by extending the updateFact method or just doing a direct UPDATE + relying on the trigger.
788
+
789
+ The simplest approach: update summary via direct SQL after `updateFact`:
790
+ ```typescript
791
+ if (a.summary !== undefined) {
792
+ store.connection.prepare('UPDATE facts SET summary = ? WHERE fact_id = ?').run(a.summary, a.fact_id as number)
793
+ }
794
+ ```
795
+ The `facts_au` trigger will automatically reindex FTS5.
796
+
797
+ - [ ] **Step 4: Add learn and audit cases**
798
+
799
+ Add before `case 'list':`:
800
+
801
+ ```typescript
802
+ case 'learn': {
803
+ const result = store.runLearning()
804
+ return { content: [{ type: 'text' as const, text: JSON.stringify(result) }] }
805
+ }
806
+
807
+ case 'audit': {
808
+ const report = store.runAudit()
809
+ return { content: [{ type: 'text' as const, text: JSON.stringify(report) }] }
810
+ }
811
+ ```
812
+
813
+ - [ ] **Step 5: Add startup auto-learn with nextTick**
814
+
815
+ Replace lines 62-63 (startup maintenance):
816
+
817
+ ```typescript
818
+ // Startup maintenance
819
+ store.decayTrustScores()
820
+ store.auditContradictions()
821
+
822
+ // Auto-learn on startup (non-blocking)
823
+ process.nextTick(() => {
824
+ try {
825
+ const result = store.runLearning()
826
+ if (result.demoted > 0 || result.aged > 0 || result.long_facts.length > 0) {
827
+ console.error(`[mnemo:auto-learn] promoted=${result.promoted} demoted=${result.demoted} aged=${result.aged} long_facts=${result.long_facts.length}`)
828
+ }
829
+ } catch (err) {
830
+ console.error('[mnemo:auto-learn] error:', err)
831
+ }
832
+ })
833
+ ```
834
+
835
+ - [ ] **Step 6: Run build + all tests**
836
+
837
+ Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npm run build && npx vitest run`
838
+ Expected: BUILD OK, ALL TESTS PASS
839
+
840
+ - [ ] **Step 7: Commit**
841
+
842
+ ```bash
843
+ cd /Users/ljn/Documents/demo/ocean/mnemo-mcp
844
+ git add src/server.ts
845
+ git commit -m "feat(server): add learn/audit actions, summary support, length warning, auto-learn on startup"
846
+ ```
847
+
848
+ ---
849
+
850
+ ## Task 5: End-to-end verification
851
+
852
+ **Files:** None (manual verification)
853
+
854
+ - [ ] **Step 1: Build and test**
855
+
856
+ Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npm run build && npx vitest run`
857
+ Expected: ALL PASS
858
+
859
+ - [ ] **Step 2: Test against real database**
860
+
861
+ ```bash
862
+ cd /Users/ljn/Documents/demo/ocean/mnemo-mcp
863
+ echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}
864
+ {"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"fact_store","arguments":{"action":"search","query":"你是谁"}}}' | node dist/server.js 2>&1 | tail -5
865
+ ```
866
+
867
+ - [ ] **Step 3: Test audit action**
868
+
869
+ ```bash
870
+ echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}
871
+ {"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"fact_store","arguments":{"action":"audit"}}}' | node dist/server.js 2>&1 | tail -5
872
+ ```
873
+
874
+ - [ ] **Step 4: Bump version**
875
+
876
+ ```bash
877
+ cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npm version minor --no-git-tag-version
878
+ ```
879
+
880
+ - [ ] **Step 5: Final commit**
881
+
882
+ ```bash
883
+ cd /Users/ljn/Documents/demo/ocean/mnemo-mcp
884
+ git add .
885
+ git commit -m "chore: prepare v0.2.0 — memory self-learning release"
886
+ ```
887
+
888
+ ---
889
+
890
+ ## Self-Review
891
+
892
+ ### Spec Coverage
893
+
894
+ | Spec Requirement | Task |
895
+ |-----------------|------|
896
+ | retrieval_log 表 + [{id, score}] 格式 | Task 1 (schema) + Task 2 (logRetrieval) |
897
+ | logRetrieval 更新 last_retrieved_at | Task 2 |
898
+ | pruneRetrievalLog 5000 条上限 | Task 2 |
899
+ | learn action: rate-based trust adjustment | Task 2 (runLearning) + Task 4 (handler) |
900
+ | learn action: aging based on last_retrieved_at | Task 2 |
901
+ | learn action: new fact protection (NULL last_retrieved_at) | Task 2 |
902
+ | learn returns long_facts report | Task 2 |
903
+ | audit action: quality report, no data modification | Task 2 (runAudit) + Task 4 (handler) |
904
+ | length penalty: matchText-based (summary or content) | Task 3 |
905
+ | length penalty: 300-char threshold | Task 3 |
906
+ | static FTS/Jaccard 0.5/0.5 weights | Task 3 |
907
+ | summary field on facts table | Task 1 |
908
+ | summary indexed by FTS5 | Task 1 (schema + triggers) |
909
+ | summary matching in retriever | Task 3 |
910
+ | 500-char write warning | Task 4 |
911
+ | add/update support summary param | Task 4 |
912
+ | server startup auto-learn with nextTick | Task 4 |
913
+ | keep refineQuery | No change needed (already in code) |
914
+ | remove relevance gate | Task 3 |
915
+ | remove content dedup | Task 3 |
916
+ | backward compatible migration | Task 1 |
917
+
918
+ ### Placeholder Scan
919
+
920
+ - [x] No "TBD", "TODO", "implement later"
921
+ - [x] No vague "add error handling" without code
922
+ - [x] All file paths are exact
923
+ - [x] All code blocks contain complete implementations
924
+
925
+ ### Type Consistency
926
+
927
+ - [x] `Fact.summary` is `string | null` in types.ts, store.ts rowToFact, retriever.ts
928
+ - [x] `Fact.lastRetrievedAt` is `string | null` in types.ts and store.ts
929
+ - [x] `FactStoreArgs.action` includes `'learn' | 'audit'` in types.ts and server.ts zod schema
930
+ - [x] `FactStoreArgs.summary` is `string` optional in types.ts and server.ts zod schema
931
+ - [x] `runLearning()` return type matches `{promoted, demoted, aged, unchanged, long_facts}` in store.ts and server.ts handler
932
+ - [x] `runAudit()` return type matches `{total_facts, long_without_summary, low_helpful_rate, aging_candidates}` in store.ts and server.ts handler