@morningljn/mnemo 0.1.3 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +43 -14
- package/dist/init.js +16 -8
- package/dist/init.js.map +1 -1
- package/dist/refine.d.ts +14 -0
- package/dist/refine.js +115 -0
- package/dist/refine.js.map +1 -0
- package/dist/resources.d.ts +27 -0
- package/dist/resources.js +56 -0
- package/dist/resources.js.map +1 -0
- package/dist/retriever.d.ts +3 -1
- package/dist/retriever.js +42 -42
- package/dist/retriever.js.map +1 -1
- package/dist/schema.d.ts +1 -1
- package/dist/schema.js +21 -10
- package/dist/schema.js.map +1 -1
- package/dist/server.js +41 -1
- package/dist/server.js.map +1 -1
- package/dist/store.d.ts +37 -0
- package/dist/store.js +166 -9
- package/dist/store.js.map +1 -1
- package/dist/types.d.ts +4 -1
- package/docs/superpowers/plans/2026-05-15-mnemo-mcp.md +1154 -0
- package/docs/superpowers/plans/2026-05-16-memory-self-learning.md +932 -0
- package/docs/superpowers/plans/2026-05-16-mnemo-query-cache.md +613 -0
- package/docs/superpowers/plans/2026-05-16-retrieval-and-injection-optimization.md +770 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/.openspec.yaml +2 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/design.md +83 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/proposal.md +32 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/specs/fact-retrieval/spec.md +75 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/specs/fact-store/spec.md +83 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/specs/mcp-server/spec.md +34 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/specs/security/spec.md +37 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/tasks.md +44 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/.openspec.yaml +2 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/design.md +96 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/proposal.md +29 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/specs/batch-operations/spec.md +42 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/specs/perf-metrics/spec.md +55 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/specs/query-cache/spec.md +65 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/tasks.md +45 -0
- package/openspec/changes/memory-self-learning/.openspec.yaml +2 -0
- package/openspec/changes/memory-self-learning/design.md +174 -0
- package/openspec/changes/memory-self-learning/proposal.md +35 -0
- package/openspec/changes/memory-self-learning/specs/fact-retrieval/spec.md +35 -0
- package/openspec/changes/memory-self-learning/specs/fact-summary/spec.md +45 -0
- package/openspec/changes/memory-self-learning/specs/length-penalty/spec.md +27 -0
- package/openspec/changes/memory-self-learning/specs/retrieval-log/spec.md +41 -0
- package/openspec/changes/memory-self-learning/specs/self-learning/spec.md +68 -0
- package/openspec/changes/memory-self-learning/tasks.md +56 -0
- package/openspec/changes/retrieval-and-injection-optimization/.openspec.yaml +2 -0
- package/openspec/changes/retrieval-and-injection-optimization/design.md +117 -0
- package/openspec/changes/retrieval-and-injection-optimization/proposal.md +30 -0
- package/openspec/changes/retrieval-and-injection-optimization/specs/adaptive-scoring/spec.md +43 -0
- package/openspec/changes/retrieval-and-injection-optimization/specs/injection-protocol/spec.md +48 -0
- package/openspec/changes/retrieval-and-injection-optimization/specs/mcp-resources/spec.md +39 -0
- package/openspec/changes/retrieval-and-injection-optimization/specs/query-refinement/spec.md +39 -0
- package/openspec/changes/retrieval-and-injection-optimization/tasks.md +33 -0
- package/openspec/config.yaml +20 -0
- package/package.json +1 -1
- package/src/init.ts +17 -9
- package/src/refine.ts +127 -0
- package/src/resources.ts +78 -0
- package/src/retriever.ts +46 -44
- package/src/schema.ts +21 -10
- package/src/server.ts +44 -1
- package/src/store.ts +215 -9
- package/src/types.ts +4 -1
- package/tests/refine.test.ts +52 -0
- package/tests/resource.test.ts +62 -0
- package/tests/retriever.test.ts +53 -0
- package/tests/store.test.ts +112 -0
|
@@ -0,0 +1,932 @@
|
|
|
1
|
+
# Memory Self-Learning Implementation Plan
|
|
2
|
+
|
|
3
|
+
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
4
|
+
|
|
5
|
+
**Goal:** Improve mnemo-mcp retrieval accuracy by reverting v3 dynamic weights, adding length penalty for super-long facts, summary field for data quality, retrieval log for self-learning, and learn/audit actions for automatic trust adjustment.
|
|
6
|
+
|
|
7
|
+
**Architecture:** Revert retriever to static 0.5/0.5 FTS/Jaccard weights. Add length penalty based on matchText (summary or content). New retrieval_log table auto-records every search. New learn action adjusts trust based on retrieval/helpful stats. New audit action reports data quality. Summary field lets long facts provide short matching text.
|
|
8
|
+
|
|
9
|
+
**Tech Stack:** TypeScript, Node.js, better-sqlite3, @modelcontextprotocol/sdk, zod/v4, vitest
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## File Structure
|
|
14
|
+
|
|
15
|
+
| File | Responsibility |
|
|
16
|
+
|------|---------------|
|
|
17
|
+
| `src/schema.ts` | **MODIFY** — Add retrieval_log table, FTS5 with summary column, update triggers |
|
|
18
|
+
| `src/store.ts` | **MODIFY** — Add summary/last_retrieved_at columns, logRetrieval, pruneRetrievalLog, runLearning, runAudit |
|
|
19
|
+
| `src/retriever.ts` | **MODIFY** — Revert dynamic weights, remove gate, add length penalty, summary matching |
|
|
20
|
+
| `src/server.ts` | **MODIFY** — Add learn/audit handlers, summary support, length warning, auto-learn on start |
|
|
21
|
+
| `src/types.ts` | **MODIFY** — Add summary/last_retrieved_at to Fact, learn/audit action types |
|
|
22
|
+
| `tests/store.test.ts` | **MODIFY** — New tests for retrieval_log, learning, audit |
|
|
23
|
+
| `tests/retriever.test.ts` | **MODIFY** — New tests for length penalty, summary, static weights |
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## Task 1: Schema Migration — retrieval_log + summary + last_retrieved_at + FTS5 rebuild
|
|
28
|
+
|
|
29
|
+
**Files:**
|
|
30
|
+
- Modify: `src/schema.ts`
|
|
31
|
+
- Modify: `src/store.ts` (migrateSchema)
|
|
32
|
+
- Test: `tests/store.test.ts`
|
|
33
|
+
|
|
34
|
+
- [ ] **Step 1: Write failing test for new columns**
|
|
35
|
+
|
|
36
|
+
Append to `tests/store.test.ts`:
|
|
37
|
+
|
|
38
|
+
```typescript
|
|
39
|
+
describe('schema migration', () => {
|
|
40
|
+
it('has summary column on facts table', () => {
|
|
41
|
+
const cols = store.connection.pragma('table_info(facts)') as Array<{ name: string }>
|
|
42
|
+
const colNames = cols.map(c => c.name)
|
|
43
|
+
expect(colNames).toContain('summary')
|
|
44
|
+
expect(colNames).toContain('last_retrieved_at')
|
|
45
|
+
})
|
|
46
|
+
|
|
47
|
+
it('has retrieval_log table', () => {
|
|
48
|
+
const tables = store.connection.prepare(
|
|
49
|
+
"SELECT name FROM sqlite_master WHERE type='table' AND name='retrieval_log'"
|
|
50
|
+
).get()
|
|
51
|
+
expect(tables).toBeTruthy()
|
|
52
|
+
})
|
|
53
|
+
|
|
54
|
+
it('summary defaults to null', () => {
|
|
55
|
+
const id = store.addFact('test summary fact', 'general')
|
|
56
|
+
const row = store.connection.prepare('SELECT summary FROM facts WHERE fact_id = ?').get(id) as { summary: string | null }
|
|
57
|
+
expect(row.summary).toBeNull()
|
|
58
|
+
})
|
|
59
|
+
})
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
- [ ] **Step 2: Run test to verify it fails**
|
|
63
|
+
|
|
64
|
+
Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/store.test.ts -t "schema migration"`
|
|
65
|
+
Expected: FAIL — summary column doesn't exist yet
|
|
66
|
+
|
|
67
|
+
- [ ] **Step 3: Update `src/schema.ts` — add retrieval_log table, rebuild FTS5 with summary**
|
|
68
|
+
|
|
69
|
+
Replace entire file content:
|
|
70
|
+
|
|
71
|
+
```typescript
|
|
72
|
+
export const SCHEMA = `
|
|
73
|
+
-- 事实表
|
|
74
|
+
CREATE TABLE IF NOT EXISTS facts (
|
|
75
|
+
fact_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
76
|
+
content TEXT NOT NULL UNIQUE,
|
|
77
|
+
category TEXT DEFAULT 'general',
|
|
78
|
+
tags TEXT DEFAULT '',
|
|
79
|
+
keywords TEXT DEFAULT '[]',
|
|
80
|
+
trust_score REAL DEFAULT 0.5,
|
|
81
|
+
retrieval_count INTEGER DEFAULT 0,
|
|
82
|
+
helpful_count INTEGER DEFAULT 0,
|
|
83
|
+
created_at TEXT DEFAULT (datetime('now', 'localtime')),
|
|
84
|
+
updated_at TEXT DEFAULT (datetime('now', 'localtime'))
|
|
85
|
+
);
|
|
86
|
+
|
|
87
|
+
-- 实体表
|
|
88
|
+
CREATE TABLE IF NOT EXISTS entities (
|
|
89
|
+
entity_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
90
|
+
name TEXT NOT NULL,
|
|
91
|
+
entity_type TEXT DEFAULT 'unknown',
|
|
92
|
+
aliases TEXT DEFAULT '',
|
|
93
|
+
created_at TEXT DEFAULT (datetime('now', 'localtime'))
|
|
94
|
+
);
|
|
95
|
+
|
|
96
|
+
-- 事实-实体关联表
|
|
97
|
+
CREATE TABLE IF NOT EXISTS fact_entities (
|
|
98
|
+
fact_id INTEGER NOT NULL REFERENCES facts(fact_id) ON DELETE CASCADE,
|
|
99
|
+
entity_id INTEGER NOT NULL REFERENCES entities(entity_id) ON DELETE CASCADE,
|
|
100
|
+
PRIMARY KEY (fact_id, entity_id)
|
|
101
|
+
);
|
|
102
|
+
|
|
103
|
+
-- 检索日志表
|
|
104
|
+
CREATE TABLE IF NOT EXISTS retrieval_log (
|
|
105
|
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
106
|
+
query TEXT NOT NULL,
|
|
107
|
+
results TEXT NOT NULL,
|
|
108
|
+
timestamp TEXT DEFAULT (datetime('now', 'localtime'))
|
|
109
|
+
);
|
|
110
|
+
|
|
111
|
+
-- 索引
|
|
112
|
+
CREATE INDEX IF NOT EXISTS idx_facts_trust ON facts(trust_score DESC);
|
|
113
|
+
CREATE INDEX IF NOT EXISTS idx_facts_category ON facts(category);
|
|
114
|
+
CREATE INDEX IF NOT EXISTS idx_entities_name ON entities(name);
|
|
115
|
+
CREATE INDEX IF NOT EXISTS idx_fact_entities_entity ON fact_entities(entity_id);
|
|
116
|
+
CREATE INDEX IF NOT EXISTS idx_retrieval_log_ts ON retrieval_log(timestamp);
|
|
117
|
+
|
|
118
|
+
-- FTS5 全文索引(含 summary 列)
|
|
119
|
+
CREATE VIRTUAL TABLE IF NOT EXISTS facts_fts
|
|
120
|
+
USING fts5(content, tags, summary, content=facts, content_rowid=fact_id);
|
|
121
|
+
|
|
122
|
+
-- FTS5 同步触发器:插入
|
|
123
|
+
CREATE TRIGGER IF NOT EXISTS facts_ai AFTER INSERT ON facts BEGIN
|
|
124
|
+
INSERT INTO facts_fts(rowid, content, tags, summary)
|
|
125
|
+
VALUES (new.fact_id, new.content, new.tags, COALESCE(new.summary, ''));
|
|
126
|
+
END;
|
|
127
|
+
|
|
128
|
+
-- FTS5 同步触发器:删除
|
|
129
|
+
CREATE TRIGGER IF NOT EXISTS facts_ad AFTER DELETE ON facts BEGIN
|
|
130
|
+
INSERT INTO facts_fts(facts_fts, rowid, content, tags, summary)
|
|
131
|
+
VALUES ('delete', old.fact_id, old.content, old.tags, COALESCE(old.summary, ''));
|
|
132
|
+
END;
|
|
133
|
+
|
|
134
|
+
-- FTS5 同步触发器:更新
|
|
135
|
+
CREATE TRIGGER IF NOT EXISTS facts_au AFTER UPDATE ON facts BEGIN
|
|
136
|
+
INSERT INTO facts_fts(facts_fts, rowid, content, tags, summary)
|
|
137
|
+
VALUES ('delete', old.fact_id, old.content, old.tags, COALESCE(old.summary, ''));
|
|
138
|
+
INSERT INTO facts_fts(rowid, content, tags, summary)
|
|
139
|
+
VALUES (new.fact_id, new.content, new.tags, COALESCE(new.summary, ''));
|
|
140
|
+
END;
|
|
141
|
+
`
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
- [ ] **Step 4: Update `src/store.ts` migrateSchema — add columns + rebuild FTS5**
|
|
145
|
+
|
|
146
|
+
Replace the `migrateSchema()` method in `src/store.ts` (lines 112-117):
|
|
147
|
+
|
|
148
|
+
```typescript
|
|
149
|
+
/** 增量迁移:添加新列/新表(已存在则跳过) */
|
|
150
|
+
private migrateSchema(): void {
|
|
151
|
+
const addColumn = (sql: string) => {
|
|
152
|
+
try { this.db.exec(sql) } catch { /* 列已存在 */ }
|
|
153
|
+
}
|
|
154
|
+
|
|
155
|
+
addColumn("ALTER TABLE facts ADD COLUMN keywords TEXT DEFAULT '[]'")
|
|
156
|
+
addColumn('ALTER TABLE facts ADD COLUMN summary TEXT DEFAULT NULL')
|
|
157
|
+
addColumn('ALTER TABLE facts ADD COLUMN last_retrieved_at TEXT DEFAULT NULL')
|
|
158
|
+
|
|
159
|
+
// 重建 FTS5 以包含 summary 列(仅在 facts_fts 无 summary 列时执行)
|
|
160
|
+
try {
|
|
161
|
+
this.db.prepare("SELECT summary FROM facts_fts LIMIT 0").get()
|
|
162
|
+
} catch {
|
|
163
|
+
// facts_fts 没有 summary 列,需要重建
|
|
164
|
+
this.db.exec('DROP TABLE IF EXISTS facts_fts')
|
|
165
|
+
this.db.exec('DROP TRIGGER IF EXISTS facts_ai')
|
|
166
|
+
this.db.exec('DROP TRIGGER IF EXISTS facts_ad')
|
|
167
|
+
this.db.exec('DROP TRIGGER IF EXISTS facts_au')
|
|
168
|
+
// 重建会由 SCHEMA 中的 CREATE IF NOT EXISTS 处理
|
|
169
|
+
this.db.exec(`
|
|
170
|
+
CREATE VIRTUAL TABLE IF NOT EXISTS facts_fts
|
|
171
|
+
USING fts5(content, tags, summary, content=facts, content_rowid=fact_id)
|
|
172
|
+
`)
|
|
173
|
+
// 重建触发器
|
|
174
|
+
this.db.exec(`CREATE TRIGGER IF NOT EXISTS facts_ai AFTER INSERT ON facts BEGIN
|
|
175
|
+
INSERT INTO facts_fts(rowid, content, tags, summary)
|
|
176
|
+
VALUES (new.fact_id, new.content, new.tags, COALESCE(new.summary, ''));
|
|
177
|
+
END`)
|
|
178
|
+
this.db.exec(`CREATE TRIGGER IF NOT EXISTS facts_ad AFTER DELETE ON facts BEGIN
|
|
179
|
+
INSERT INTO facts_fts(facts_fts, rowid, content, tags, summary)
|
|
180
|
+
VALUES ('delete', old.fact_id, old.content, old.tags, COALESCE(old.summary, ''));
|
|
181
|
+
END`)
|
|
182
|
+
this.db.exec(`CREATE TRIGGER IF NOT EXISTS facts_au AFTER UPDATE ON facts BEGIN
|
|
183
|
+
INSERT INTO facts_fts(facts_fts, rowid, content, tags, summary)
|
|
184
|
+
VALUES ('delete', old.fact_id, old.content, old.tags, COALESCE(old.summary, ''));
|
|
185
|
+
INSERT INTO facts_fts(rowid, content, tags, summary)
|
|
186
|
+
VALUES (new.fact_id, new.content, new.tags, COALESCE(new.summary, ''));
|
|
187
|
+
END`)
|
|
188
|
+
// 重新填充 FTS5
|
|
189
|
+
this.db.exec(`INSERT INTO facts_fts(rowid, content, tags, summary)
|
|
190
|
+
SELECT fact_id, content, tags, COALESCE(summary, '') FROM facts`)
|
|
191
|
+
}
|
|
192
|
+
}
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
- [ ] **Step 5: Update `src/types.ts` — add summary and last_retrieved_at to Fact**
|
|
196
|
+
|
|
197
|
+
In `src/types.ts`, update the `Fact` interface (lines 5-16):
|
|
198
|
+
|
|
199
|
+
```typescript
|
|
200
|
+
/** 存储的事实记录 */
|
|
201
|
+
export interface Fact {
|
|
202
|
+
factId: number
|
|
203
|
+
content: string
|
|
204
|
+
summary: string | null
|
|
205
|
+
category: FactCategory
|
|
206
|
+
tags: string
|
|
207
|
+
keywords: string
|
|
208
|
+
trustScore: number
|
|
209
|
+
retrievalCount: number
|
|
210
|
+
helpfulCount: number
|
|
211
|
+
createdAt: string
|
|
212
|
+
updatedAt: string
|
|
213
|
+
lastRetrievedAt: string | null
|
|
214
|
+
}
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
Update `FactStoreArgs` action union to include new actions (line 56):
|
|
218
|
+
|
|
219
|
+
```typescript
|
|
220
|
+
export interface FactStoreArgs {
|
|
221
|
+
action: 'add' | 'search' | 'probe' | 'related' | 'reason' | 'contradict' | 'update' | 'remove' | 'list' | 'learn' | 'audit'
|
|
222
|
+
content?: string | string[]
|
|
223
|
+
summary?: string
|
|
224
|
+
query?: string
|
|
225
|
+
entity?: string
|
|
226
|
+
entities?: string[]
|
|
227
|
+
fact_id?: number | number[]
|
|
228
|
+
category?: string
|
|
229
|
+
tags?: string
|
|
230
|
+
trust_delta?: number
|
|
231
|
+
min_trust?: number
|
|
232
|
+
limit?: number
|
|
233
|
+
}
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
- [ ] **Step 6: Update `src/store.ts` rowToFact — add new fields**
|
|
237
|
+
|
|
238
|
+
Replace `rowToFact` method (lines 791-803):
|
|
239
|
+
|
|
240
|
+
```typescript
|
|
241
|
+
private rowToFact(row: FactRow): Fact {
|
|
242
|
+
return {
|
|
243
|
+
factId: row.fact_id,
|
|
244
|
+
content: row.content,
|
|
245
|
+
summary: (row as any).summary ?? null,
|
|
246
|
+
category: row.category as FactCategory,
|
|
247
|
+
tags: row.tags,
|
|
248
|
+
keywords: row.keywords,
|
|
249
|
+
trustScore: row.trust_score,
|
|
250
|
+
retrievalCount: row.retrieval_count,
|
|
251
|
+
helpfulCount: row.helpful_count,
|
|
252
|
+
createdAt: row.created_at,
|
|
253
|
+
updatedAt: row.updated_at,
|
|
254
|
+
lastRetrievedAt: (row as any).last_retrieved_at ?? null,
|
|
255
|
+
}
|
|
256
|
+
}
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
Update all SQL SELECT statements in store.ts that query facts to include `summary` and `last_retrieved_at`. For `listFacts` (line 363):
|
|
260
|
+
|
|
261
|
+
```sql
|
|
262
|
+
SELECT fact_id, content, summary, category, tags, keywords, trust_score,
|
|
263
|
+
retrieval_count, helpful_count, created_at, updated_at, last_retrieved_at
|
|
264
|
+
FROM facts
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
Apply the same pattern to `getFactsByEntity`, `getFactsByEntities`, `updateFact`, and `recordFeedback` queries.
|
|
268
|
+
|
|
269
|
+
- [ ] **Step 7: Run schema migration tests**
|
|
270
|
+
|
|
271
|
+
Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/store.test.ts -t "schema migration"`
|
|
272
|
+
Expected: PASS
|
|
273
|
+
|
|
274
|
+
- [ ] **Step 8: Run full build**
|
|
275
|
+
|
|
276
|
+
Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npm run build`
|
|
277
|
+
Expected: BUILD OK
|
|
278
|
+
|
|
279
|
+
- [ ] **Step 9: Commit**
|
|
280
|
+
|
|
281
|
+
```bash
|
|
282
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp
|
|
283
|
+
git add src/schema.ts src/store.ts src/types.ts tests/store.test.ts
|
|
284
|
+
git commit -m "feat(store): add retrieval_log table, summary/last_retrieved_at columns, FTS5 rebuild"
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
---
|
|
288
|
+
|
|
289
|
+
## Task 2: Store — logRetrieval + pruneRetrievalLog + runLearning + runAudit
|
|
290
|
+
|
|
291
|
+
**Files:**
|
|
292
|
+
- Modify: `src/store.ts`
|
|
293
|
+
- Test: `tests/store.test.ts`
|
|
294
|
+
|
|
295
|
+
- [ ] **Step 1: Write failing tests**
|
|
296
|
+
|
|
297
|
+
Append to `tests/store.test.ts`:
|
|
298
|
+
|
|
299
|
+
```typescript
|
|
300
|
+
describe('logRetrieval', () => {
|
|
301
|
+
it('writes retrieval log with results', () => {
|
|
302
|
+
const id = store.addFact('test fact for logging', 'general')
|
|
303
|
+
store.logRetrieval('test query', [{ id, score: 0.8 }])
|
|
304
|
+
const rows = store.connection.prepare('SELECT * FROM retrieval_log').all() as Array<any>
|
|
305
|
+
expect(rows.length).toBe(1)
|
|
306
|
+
expect(rows[0].query).toBe('test query')
|
|
307
|
+
expect(JSON.parse(rows[0].results)).toEqual([{ id, score: 0.8 }])
|
|
308
|
+
})
|
|
309
|
+
|
|
310
|
+
it('updates last_retrieved_at for returned facts', () => {
|
|
311
|
+
const id = store.addFact('test fact for timestamp', 'general')
|
|
312
|
+
store.logRetrieval('query', [{ id, score: 0.5 }])
|
|
313
|
+
const row = store.connection.prepare('SELECT last_retrieved_at FROM facts WHERE fact_id = ?').get(id) as any
|
|
314
|
+
expect(row.last_retrieved_at).not.toBeNull()
|
|
315
|
+
})
|
|
316
|
+
|
|
317
|
+
it('prunes log to max entries', () => {
|
|
318
|
+
for (let i = 0; i < 12; i++) {
|
|
319
|
+
store.logRetrieval(`query ${i}`, [])
|
|
320
|
+
}
|
|
321
|
+
store.pruneRetrievalLog(10)
|
|
322
|
+
const count = (store.connection.prepare('SELECT COUNT(*) as c FROM retrieval_log').get() as any).c
|
|
323
|
+
expect(count).toBe(10)
|
|
324
|
+
})
|
|
325
|
+
})
|
|
326
|
+
|
|
327
|
+
describe('runLearning', () => {
|
|
328
|
+
it('demotes high retrieval low helpful facts', () => {
|
|
329
|
+
const id = store.addFact('demote me', 'general')
|
|
330
|
+
// Simulate 100 retrievals, 2 helpful
|
|
331
|
+
store.connection.prepare('UPDATE facts SET retrieval_count = 100, helpful_count = 2, trust_score = 1.0 WHERE fact_id = ?').run(id)
|
|
332
|
+
const result = store.runLearning()
|
|
333
|
+
const row = store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any
|
|
334
|
+
expect(row.trust_score).toBeLessThan(1.0)
|
|
335
|
+
expect(result.demoted).toBeGreaterThanOrEqual(1)
|
|
336
|
+
})
|
|
337
|
+
|
|
338
|
+
it('promotes high helpful rate facts', () => {
|
|
339
|
+
const id = store.addFact('promote me', 'general')
|
|
340
|
+
store.connection.prepare('UPDATE facts SET retrieval_count = 50, helpful_count = 20, trust_score = 0.5 WHERE fact_id = ?').run(id)
|
|
341
|
+
store.runLearning()
|
|
342
|
+
const row = store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any
|
|
343
|
+
expect(row.trust_score).toBeGreaterThan(0.5)
|
|
344
|
+
})
|
|
345
|
+
|
|
346
|
+
it('does not adjust facts with low retrieval count', () => {
|
|
347
|
+
const id = store.addFact('new fact', 'general')
|
|
348
|
+
store.connection.prepare('UPDATE facts SET retrieval_count = 5, helpful_count = 0, trust_score = 0.8 WHERE fact_id = ?').run(id)
|
|
349
|
+
store.runLearning()
|
|
350
|
+
const row = store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any
|
|
351
|
+
expect(row.trust_score).toBe(0.8)
|
|
352
|
+
})
|
|
353
|
+
|
|
354
|
+
it('ages facts not retrieved for 60 days', () => {
|
|
355
|
+
const id = store.addFact('old fact', 'general')
|
|
356
|
+
store.connection.prepare("UPDATE facts SET last_retrieved_at = datetime('now', '-61 days'), trust_score = 0.8 WHERE fact_id = ?").run(id)
|
|
357
|
+
store.runLearning()
|
|
358
|
+
const row = store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any
|
|
359
|
+
expect(row.trust_score).toBeLessThan(0.8)
|
|
360
|
+
})
|
|
361
|
+
|
|
362
|
+
it('protects new facts with null last_retrieved_at from aging', () => {
|
|
363
|
+
const id = store.addFact('brand new fact', 'general')
|
|
364
|
+
store.connection.prepare('UPDATE facts SET trust_score = 0.5, last_retrieved_at = NULL WHERE fact_id = ?').run(id)
|
|
365
|
+
store.runLearning()
|
|
366
|
+
const row = store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any
|
|
367
|
+
expect(row.trust_score).toBe(0.5)
|
|
368
|
+
})
|
|
369
|
+
|
|
370
|
+
it('returns long_facts report', () => {
|
|
371
|
+
const id = store.addFact('x'.repeat(600), 'general')
|
|
372
|
+
const result = store.runLearning()
|
|
373
|
+
expect(result.long_facts.length).toBeGreaterThanOrEqual(1)
|
|
374
|
+
expect(result.long_facts.some((f: any) => f.id === id)).toBe(true)
|
|
375
|
+
})
|
|
376
|
+
})
|
|
377
|
+
|
|
378
|
+
describe('runAudit', () => {
|
|
379
|
+
it('returns quality report without modifying data', () => {
|
|
380
|
+
const id = store.addFact('a'.repeat(600), 'general')
|
|
381
|
+
store.connection.prepare('UPDATE facts SET retrieval_count = 100, helpful_count = 1 WHERE fact_id = ?').run(id)
|
|
382
|
+
const before = (store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any).trust_score
|
|
383
|
+
const report = store.runAudit()
|
|
384
|
+
const after = (store.connection.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(id) as any).trust_score
|
|
385
|
+
expect(before).toBe(after) // audit does not modify
|
|
386
|
+
expect(report.total_facts).toBeGreaterThanOrEqual(1)
|
|
387
|
+
expect(report.long_without_summary.length).toBeGreaterThanOrEqual(1)
|
|
388
|
+
})
|
|
389
|
+
})
|
|
390
|
+
```
|
|
391
|
+
|
|
392
|
+
- [ ] **Step 2: Run test to verify it fails**
|
|
393
|
+
|
|
394
|
+
Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/store.test.ts -t "logRetrieval|runLearning|runAudit"`
|
|
395
|
+
Expected: FAIL — methods don't exist yet
|
|
396
|
+
|
|
397
|
+
- [ ] **Step 3: Add methods to `src/store.ts`**
|
|
398
|
+
|
|
399
|
+
Add these methods to the `MemoryStore` class, after the `getTotalCount()` method (after line 636):
|
|
400
|
+
|
|
401
|
+
```typescript
|
|
402
|
+
/** 记录检索日志并更新 last_retrieved_at */
|
|
403
|
+
logRetrieval(query: string, results: Array<{ id: number; score: number }>): void {
|
|
404
|
+
const resultsJson = JSON.stringify(results)
|
|
405
|
+
this.db.prepare(
|
|
406
|
+
"INSERT INTO retrieval_log (query, results) VALUES (?, ?)"
|
|
407
|
+
).run(query, resultsJson)
|
|
408
|
+
|
|
409
|
+
// 更新返回 fact 的 last_retrieved_at
|
|
410
|
+
if (results.length > 0) {
|
|
411
|
+
const ids = results.map(r => r.id)
|
|
412
|
+
const placeholders = ids.map(() => '?').join(',')
|
|
413
|
+
this.db.prepare(
|
|
414
|
+
`UPDATE facts SET last_retrieved_at = datetime('now', 'localtime') WHERE fact_id IN (${placeholders})`
|
|
415
|
+
).run(...ids)
|
|
416
|
+
}
|
|
417
|
+
|
|
418
|
+
// 自动清理日志
|
|
419
|
+
this.pruneRetrievalLog(5000)
|
|
420
|
+
}
|
|
421
|
+
|
|
422
|
+
/** 清理检索日志,保留最近 maxEntries 条 */
|
|
423
|
+
pruneRetrievalLog(maxEntries = 5000): void {
|
|
424
|
+
this.db.prepare(
|
|
425
|
+
`DELETE FROM retrieval_log WHERE id NOT IN (
|
|
426
|
+
SELECT id FROM retrieval_log ORDER BY id DESC LIMIT ?
|
|
427
|
+
)`
|
|
428
|
+
).run(maxEntries)
|
|
429
|
+
}
|
|
430
|
+
|
|
431
|
+
/** 自学习:基于检索统计自动调整 trust_score */
|
|
432
|
+
runLearning(): {
|
|
433
|
+
promoted: number
|
|
434
|
+
demoted: number
|
|
435
|
+
aged: number
|
|
436
|
+
unchanged: number
|
|
437
|
+
long_facts: Array<{ id: number; content_length: number; penalty: number; has_summary: boolean }>
|
|
438
|
+
} {
|
|
439
|
+
const rows = this.db.prepare(
|
|
440
|
+
'SELECT fact_id, content, summary, retrieval_count, helpful_count, trust_score, last_retrieved_at FROM facts'
|
|
441
|
+
).all() as Array<{
|
|
442
|
+
fact_id: number; content: string; summary: string | null;
|
|
443
|
+
retrieval_count: number; helpful_count: number; trust_score: number; last_retrieved_at: string | null
|
|
444
|
+
}>
|
|
445
|
+
|
|
446
|
+
let promoted = 0
|
|
447
|
+
let demoted = 0
|
|
448
|
+
let aged = 0
|
|
449
|
+
let unchanged = 0
|
|
450
|
+
const longFacts: Array<{ id: number; content_length: number; penalty: number; has_summary: boolean }> = []
|
|
451
|
+
|
|
452
|
+
const now = Date.now()
|
|
453
|
+
|
|
454
|
+
for (const row of rows) {
|
|
455
|
+
let changed = false
|
|
456
|
+
const rate = row.retrieval_count > 0 ? row.helpful_count / row.retrieval_count : 0
|
|
457
|
+
|
|
458
|
+
// Rate-based adjustment (需要 30+ 次检索)
|
|
459
|
+
if (row.retrieval_count > 30) {
|
|
460
|
+
if (rate < 0.05) {
|
|
461
|
+
const newTrust = clampTrust(row.trust_score * 0.9)
|
|
462
|
+
this.db.prepare('UPDATE facts SET trust_score = ? WHERE fact_id = ?').run(newTrust, row.fact_id)
|
|
463
|
+
demoted++
|
|
464
|
+
changed = true
|
|
465
|
+
} else if (rate > 0.3) {
|
|
466
|
+
const newTrust = clampTrust(row.trust_score + 0.05)
|
|
467
|
+
this.db.prepare('UPDATE facts SET trust_score = ? WHERE fact_id = ?').run(newTrust, row.fact_id)
|
|
468
|
+
promoted++
|
|
469
|
+
changed = true
|
|
470
|
+
}
|
|
471
|
+
}
|
|
472
|
+
|
|
473
|
+
// Aging (60 天未检索)
|
|
474
|
+
if (row.last_retrieved_at) {
|
|
475
|
+
const lastRetrieved = new Date(row.last_retrieved_at + 'Z').getTime()
|
|
476
|
+
const daysSinceRetrieval = (now - lastRetrieved) / 86_400_000
|
|
477
|
+
if (daysSinceRetrieval > 60) {
|
|
478
|
+
const currentTrust = this.db.prepare('SELECT trust_score FROM facts WHERE fact_id = ?').get(row.fact_id) as any
|
|
479
|
+
const newTrust = clampTrust(currentTrust.trust_score * 0.95)
|
|
480
|
+
this.db.prepare('UPDATE facts SET trust_score = ? WHERE fact_id = ?').run(newTrust, row.fact_id)
|
|
481
|
+
aged++
|
|
482
|
+
changed = true
|
|
483
|
+
}
|
|
484
|
+
}
|
|
485
|
+
// last_retrieved_at 为 NULL = 新 fact,不老化
|
|
486
|
+
|
|
487
|
+
if (!changed) unchanged++
|
|
488
|
+
|
|
489
|
+
// Long facts report (content > 300 字无 summary)
|
|
490
|
+
const matchLength = row.summary ? row.summary.length : row.content.length
|
|
491
|
+
if (matchLength > 300) {
|
|
492
|
+
longFacts.push({
|
|
493
|
+
id: row.fact_id,
|
|
494
|
+
content_length: row.content.length,
|
|
495
|
+
penalty: Math.min(1.0, 300 / matchLength),
|
|
496
|
+
has_summary: !!row.summary,
|
|
497
|
+
})
|
|
498
|
+
}
|
|
499
|
+
}
|
|
500
|
+
|
|
501
|
+
return { promoted, demoted, aged, unchanged, long_facts: longFacts }
|
|
502
|
+
}
|
|
503
|
+
|
|
504
|
+
/** 数据质量审计(只读,不修改数据) */
|
|
505
|
+
runAudit(): {
|
|
506
|
+
total_facts: number
|
|
507
|
+
long_without_summary: Array<{ id: number; content_length: number }>
|
|
508
|
+
low_helpful_rate: Array<{ id: number; rate: number; retrieval_count: number }>
|
|
509
|
+
aging_candidates: Array<{ id: number; last_retrieved_at: string | null }>
|
|
510
|
+
} {
|
|
511
|
+
const rows = this.db.prepare(
|
|
512
|
+
'SELECT fact_id, content, summary, retrieval_count, helpful_count, last_retrieved_at FROM facts'
|
|
513
|
+
).all() as Array<{
|
|
514
|
+
fact_id: number; content: string; summary: string | null;
|
|
515
|
+
retrieval_count: number; helpful_count: number; last_retrieved_at: string | null
|
|
516
|
+
}>
|
|
517
|
+
|
|
518
|
+
const longWithoutSummary: Array<{ id: number; content_length: number }> = []
|
|
519
|
+
const lowHelpfulRate: Array<{ id: number; rate: number; retrieval_count: number }> = []
|
|
520
|
+
const agingCandidates: Array<{ id: number; last_retrieved_at: string | null }> = []
|
|
521
|
+
|
|
522
|
+
const now = Date.now()
|
|
523
|
+
|
|
524
|
+
for (const row of rows) {
|
|
525
|
+
// 超 500 字无 summary
|
|
526
|
+
if (row.content.length > 500 && !row.summary) {
|
|
527
|
+
longWithoutSummary.push({ id: row.fact_id, content_length: row.content.length })
|
|
528
|
+
}
|
|
529
|
+
|
|
530
|
+
// 低 helpful 率(>30 次检索,rate < 5%)
|
|
531
|
+
if (row.retrieval_count > 30) {
|
|
532
|
+
const rate = row.helpful_count / row.retrieval_count
|
|
533
|
+
if (rate < 0.05) {
|
|
534
|
+
lowHelpfulRate.push({ id: row.fact_id, rate: Math.round(rate * 1000) / 1000, retrieval_count: row.retrieval_count })
|
|
535
|
+
}
|
|
536
|
+
}
|
|
537
|
+
|
|
538
|
+
// 老化候选(>60 天未检索)
|
|
539
|
+
if (row.last_retrieved_at) {
|
|
540
|
+
const lastRetrieved = new Date(row.last_retrieved_at + 'Z').getTime()
|
|
541
|
+
const daysSince = (now - lastRetrieved) / 86_400_000
|
|
542
|
+
if (daysSince > 60) {
|
|
543
|
+
agingCandidates.push({ id: row.fact_id, last_retrieved_at: row.last_retrieved_at })
|
|
544
|
+
}
|
|
545
|
+
}
|
|
546
|
+
}
|
|
547
|
+
|
|
548
|
+
return {
|
|
549
|
+
total_facts: rows.length,
|
|
550
|
+
long_without_summary: longWithoutSummary,
|
|
551
|
+
low_helpful_rate: lowHelpfulRate,
|
|
552
|
+
aging_candidates: agingCandidates,
|
|
553
|
+
}
|
|
554
|
+
}
|
|
555
|
+
```
|
|
556
|
+
|
|
557
|
+
- [ ] **Step 4: Run tests**
|
|
558
|
+
|
|
559
|
+
Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/store.test.ts -t "logRetrieval|runLearning|runAudit"`
|
|
560
|
+
Expected: ALL PASS
|
|
561
|
+
|
|
562
|
+
- [ ] **Step 5: Commit**
|
|
563
|
+
|
|
564
|
+
```bash
|
|
565
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp
|
|
566
|
+
git add src/store.ts tests/store.test.ts
|
|
567
|
+
git commit -m "feat(store): add logRetrieval, pruneRetrievalLog, runLearning, runAudit"
|
|
568
|
+
```
|
|
569
|
+
|
|
570
|
+
---
|
|
571
|
+
|
|
572
|
+
## Task 3: Retriever — revert dynamic weights + length penalty + summary matching
|
|
573
|
+
|
|
574
|
+
**Files:**
|
|
575
|
+
- Modify: `src/retriever.ts`
|
|
576
|
+
- Test: `tests/retriever.test.ts`
|
|
577
|
+
|
|
578
|
+
- [ ] **Step 1: Write failing tests**
|
|
579
|
+
|
|
580
|
+
Append to `tests/retriever.test.ts`:
|
|
581
|
+
|
|
582
|
+
```typescript
|
|
583
|
+
describe('static weights (no dynamic)', () => {
|
|
584
|
+
it('uses same weights for short and long queries', () => {
|
|
585
|
+
store.addFact('用户偏好 VS Code 编辑器', 'tool_pref')
|
|
586
|
+
const shortResults = retriever.search('VS Code')
|
|
587
|
+
const longResults = retriever.search('为什么 VS Code 编辑器总是报错说找不到模块')
|
|
588
|
+
// Both should return the same fact — static weights don't change by query length
|
|
589
|
+
expect(shortResults.some(r => r.content.includes('VS Code'))).toBe(true)
|
|
590
|
+
expect(longResults.some(r => r.content.includes('VS Code'))).toBe(true)
|
|
591
|
+
})
|
|
592
|
+
})
|
|
593
|
+
|
|
594
|
+
describe('length penalty', () => {
|
|
595
|
+
it('penalizes long facts without summary', () => {
|
|
596
|
+
const longContent = '用户偏好 ' + '详细说明'.repeat(200) // ~800 chars
|
|
597
|
+
store.addFact(longContent, 'tool_pref')
|
|
598
|
+
store.addFact('用户偏好 VS Code', 'tool_pref')
|
|
599
|
+
const results = retriever.search('用户偏好')
|
|
600
|
+
// Short fact should rank higher
|
|
601
|
+
if (results.length >= 2) {
|
|
602
|
+
const shortFact = results.find(r => r.content === '用户偏好 VS Code')
|
|
603
|
+
const longFact = results.find(r => r.content.length > 500)
|
|
604
|
+
if (shortFact && longFact) {
|
|
605
|
+
expect(shortFact.score).toBeGreaterThan(longFact.score)
|
|
606
|
+
}
|
|
607
|
+
}
|
|
608
|
+
})
|
|
609
|
+
|
|
610
|
+
it('does not penalize long facts with short summary', () => {
|
|
611
|
+
const longContent = '详细内容' + '补充说明'.repeat(200)
|
|
612
|
+
// Add via SQL to set summary
|
|
613
|
+
const id = store.addFact(longContent, 'general')
|
|
614
|
+
store.connection.prepare('UPDATE facts SET summary = ? WHERE fact_id = ?').run('用户偏好', id)
|
|
615
|
+
store.addFact('完全无关的内容', 'general')
|
|
616
|
+
const results = retriever.search('用户偏好')
|
|
617
|
+
const summaryFact = results.find(r => r.factId === id)
|
|
618
|
+
expect(summaryFact).toBeTruthy()
|
|
619
|
+
})
|
|
620
|
+
})
|
|
621
|
+
|
|
622
|
+
describe('no relevance gate', () => {
|
|
623
|
+
it('returns results even with low scores', () => {
|
|
624
|
+
store.addFact('完全不相关关于天气', 'general')
|
|
625
|
+
const results = retriever.search('天气')
|
|
626
|
+
// Should not be filtered out by a 0.15 threshold
|
|
627
|
+
expect(results.length).toBeGreaterThanOrEqual(1)
|
|
628
|
+
})
|
|
629
|
+
})
|
|
630
|
+
```
|
|
631
|
+
|
|
632
|
+
- [ ] **Step 2: Run test to verify it fails**
|
|
633
|
+
|
|
634
|
+
Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/retriever.test.ts -t "static weights|length penalty|no relevance gate"`
|
|
635
|
+
Expected: Some tests FAIL (dynamic weights still active, no length penalty)
|
|
636
|
+
|
|
637
|
+
- [ ] **Step 3: Modify `src/retriever.ts` search() method**
|
|
638
|
+
|
|
639
|
+
Replace lines 116-175 (the scoring and filtering section) in `search()`:
|
|
640
|
+
|
|
641
|
+
```typescript
|
|
642
|
+
// Stage 2-4: Jaccard 重排序 + 信任评分 + 时间衰减
|
|
643
|
+
// 静态权重(回退 v3 动态权重)
|
|
644
|
+
const queryTokens = this.tokenize(searchQuery)
|
|
645
|
+
|
|
646
|
+
const scored: ScoredFact[] = []
|
|
647
|
+
|
|
648
|
+
for (const fact of candidates) {
|
|
649
|
+
// summary 优先用于匹配
|
|
650
|
+
const matchText = fact.summary ?? fact.content
|
|
651
|
+
const matchTokens = this.tokenize(matchText)
|
|
652
|
+
const tagTokens = this.tokenize(fact.tags)
|
|
653
|
+
const allTokens = new Set([...matchTokens, ...tagTokens])
|
|
654
|
+
|
|
655
|
+
const jaccard = this.jaccardSimilarity(queryTokens, allTokens)
|
|
656
|
+
const qInF = this.containmentScore(queryTokens, allTokens)
|
|
657
|
+
const similarity = 0.3 * jaccard + 0.7 * qInF
|
|
658
|
+
const ftsScore = fact.ftsRank
|
|
659
|
+
|
|
660
|
+
// 静态权重 0.5/0.5
|
|
661
|
+
const relevance = 0.5 * ftsScore + 0.5 * similarity
|
|
662
|
+
let score = relevance * fact.trustScore
|
|
663
|
+
|
|
664
|
+
// 时间衰减
|
|
665
|
+
if (this.halfLifeDays > 0) {
|
|
666
|
+
score *= this.temporalDecay(fact.updatedAt || fact.createdAt)
|
|
667
|
+
}
|
|
668
|
+
|
|
669
|
+
// Length penalty:基于 matchText 长度
|
|
670
|
+
score *= Math.min(1.0, 300 / matchText.length)
|
|
671
|
+
|
|
672
|
+
scored.push({ ...fact, score })
|
|
673
|
+
}
|
|
674
|
+
|
|
675
|
+
scored.sort((a, b) => b.score - a.score)
|
|
676
|
+
|
|
677
|
+
// 取 limit 条(不再做 relevance gate 和 content dedup)
|
|
678
|
+
const results = scored.slice(0, limit)
|
|
679
|
+
|
|
680
|
+
// 检索追踪:递增 retrieval_count + top3 信任刷新
|
|
681
|
+
if (results.length > 0) {
|
|
682
|
+
this.trackRetrieval(results)
|
|
683
|
+
}
|
|
684
|
+
```
|
|
685
|
+
|
|
686
|
+
Note: The `trackRetrieval` call at the end stays the same. Remove the `RELEVANCE_THRESHOLD` constant and the content dedup loop entirely.
|
|
687
|
+
|
|
688
|
+
- [ ] **Step 4: Update `store.ts` `logRetrieval` call — add retrieval logging**
|
|
689
|
+
|
|
690
|
+
After the `trackRetrieval(results)` call in `search()`, add:
|
|
691
|
+
|
|
692
|
+
```typescript
|
|
693
|
+
// 记录检索日志
|
|
694
|
+
this.store.logRetrieval(searchQuery, results.map(r => ({ id: r.factId, score: Math.round(r.score * 1000) / 1000 })))
|
|
695
|
+
```
|
|
696
|
+
|
|
697
|
+
But wrap it so it doesn't log when cache hits (the return before this point). The `logRetrieval` should only be called on cache miss. Place it right before the cache set at line ~182.
|
|
698
|
+
|
|
699
|
+
- [ ] **Step 5: Update ftsCandidates to search summary field**
|
|
700
|
+
|
|
701
|
+
In `ftsCandidates()` method, the SQL query (line ~483) joins `facts_fts` which now includes the `summary` column. FTS5 will automatically match against all indexed columns (content, tags, summary). No SQL change needed — the FTS5 virtual table now includes summary.
|
|
702
|
+
|
|
703
|
+
However, update the `ftsCandidates` return to include `summary` from the joined row. Add to the mapping (around line 506):
|
|
704
|
+
|
|
705
|
+
```typescript
|
|
706
|
+
summary: String(row.summary ?? ''),
|
|
707
|
+
```
|
|
708
|
+
|
|
709
|
+
- [ ] **Step 6: Run tests**
|
|
710
|
+
|
|
711
|
+
Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/retriever.test.ts`
|
|
712
|
+
Expected: ALL PASS (including new tests)
|
|
713
|
+
|
|
714
|
+
- [ ] **Step 7: Commit**
|
|
715
|
+
|
|
716
|
+
```bash
|
|
717
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp
|
|
718
|
+
git add src/retriever.ts tests/retriever.test.ts
|
|
719
|
+
git commit -m "feat(retriever): revert dynamic weights, add length penalty, summary matching, remove relevance gate"
|
|
720
|
+
```
|
|
721
|
+
|
|
722
|
+
---
|
|
723
|
+
|
|
724
|
+
## Task 4: Server — learn/audit handlers + summary support + length warning + auto-learn
|
|
725
|
+
|
|
726
|
+
**Files:**
|
|
727
|
+
- Modify: `src/server.ts`
|
|
728
|
+
|
|
729
|
+
- [ ] **Step 1: Update factStoreSchema — add learn/audit actions and summary param**
|
|
730
|
+
|
|
731
|
+
Replace `factStoreSchema` (lines 29-41):
|
|
732
|
+
|
|
733
|
+
```typescript
|
|
734
|
+
const factStoreSchema = {
|
|
735
|
+
action: z.enum(['add', 'search', 'probe', 'related', 'reason', 'contradict', 'update', 'remove', 'list', 'learn', 'audit']),
|
|
736
|
+
content: z.union([z.string(), z.array(z.string())]).optional().describe("事实内容('add' 必需,支持批量)"),
|
|
737
|
+
summary: z.string().optional().describe('超长事实的摘要(检索用 summary 匹配)'),
|
|
738
|
+
query: z.string().optional().describe("搜索查询('search' 必需)"),
|
|
739
|
+
entity: z.string().optional().describe("实体名('probe'/'related' 使用)"),
|
|
740
|
+
entities: z.array(z.string()).optional().describe("实体列表('reason' 使用)"),
|
|
741
|
+
fact_id: z.union([z.number(), z.array(z.number())]).optional().describe("事实 ID('update'/'remove' 使用,支持批量)"),
|
|
742
|
+
category: z.enum(['identity', 'coding_style', 'tool_pref', 'workflow', 'general']).optional(),
|
|
743
|
+
tags: z.string().optional().describe('逗号分隔标签'),
|
|
744
|
+
trust_delta: z.number().optional().describe("'update' 的信任调整值"),
|
|
745
|
+
min_trust: z.number().optional().describe('最低信任过滤(默认 0.3)'),
|
|
746
|
+
limit: z.number().optional().describe('最大结果数(默认 10)'),
|
|
747
|
+
}
|
|
748
|
+
```
|
|
749
|
+
|
|
750
|
+
- [ ] **Step 2: Update add handler — summary support + 500-char warning**
|
|
751
|
+
|
|
752
|
+
Replace the `case 'add':` block (lines 82-112). After the existing `for` loop but before cache clear, add summary handling. In the `for` loop, when adding a new fact, also save summary:
|
|
753
|
+
|
|
754
|
+
After `store.addFact(content, category, a.tags ?? '')` for new facts, add:
|
|
755
|
+
```typescript
|
|
756
|
+
if (a.summary) {
|
|
757
|
+
store.connection.prepare('UPDATE facts SET summary = ? WHERE fact_id = ?').run(a.summary, factId)
|
|
758
|
+
}
|
|
759
|
+
```
|
|
760
|
+
|
|
761
|
+
And before the results push for both similar and new cases, add the 500-char warning:
|
|
762
|
+
```typescript
|
|
763
|
+
if (content.length > 500 && !a.summary) {
|
|
764
|
+
warnings = [...(warnings ?? []), 'content 超过 500 字,建议提供 summary 或拆分为多条 fact']
|
|
765
|
+
}
|
|
766
|
+
```
|
|
767
|
+
|
|
768
|
+
Also trigger FTS reindex after summary update by deleting and reinserting into facts_fts.
|
|
769
|
+
|
|
770
|
+
- [ ] **Step 3: Update update handler — summary support**
|
|
771
|
+
|
|
772
|
+
In `case 'update':`, after `store.updateFact(...)`, add summary update:
|
|
773
|
+
```typescript
|
|
774
|
+
if (a.summary !== undefined) {
|
|
775
|
+
store.connection.prepare('UPDATE facts SET summary = ? WHERE fact_id = ?').run(a.summary, a.fact_id as number)
|
|
776
|
+
// Trigger FTS reindex
|
|
777
|
+
store.connection.prepare(
|
|
778
|
+
"INSERT INTO facts_fts(facts_fts, rowid, content, tags, summary) VALUES ('delete', ?, '', '', '')"
|
|
779
|
+
).run(a.fact_id as number)
|
|
780
|
+
const row = store.connection.prepare('SELECT content, tags, summary FROM facts WHERE fact_id = ?').get(a.fact_id as number) as any
|
|
781
|
+
store.connection.prepare(
|
|
782
|
+
'INSERT INTO facts_fts(rowid, content, tags, summary) VALUES (?, ?, ?, ?)'
|
|
783
|
+
).run(a.fact_id as number, row.content, row.tags, row.summary ?? '')
|
|
784
|
+
}
|
|
785
|
+
```
|
|
786
|
+
|
|
787
|
+
Actually, the FTS update trigger should handle this automatically if we UPDATE the facts table. But the trigger uses `COALESCE(new.summary, '')` which is correct. Let's use `updateFact` to set summary by extending the updateFact method or just doing a direct UPDATE + relying on the trigger.
|
|
788
|
+
|
|
789
|
+
The simplest approach: update summary via direct SQL after `updateFact`:
|
|
790
|
+
```typescript
|
|
791
|
+
if (a.summary !== undefined) {
|
|
792
|
+
store.connection.prepare('UPDATE facts SET summary = ? WHERE fact_id = ?').run(a.summary, a.fact_id as number)
|
|
793
|
+
}
|
|
794
|
+
```
|
|
795
|
+
The `facts_au` trigger will automatically reindex FTS5.
|
|
796
|
+
|
|
797
|
+
- [ ] **Step 4: Add learn and audit cases**
|
|
798
|
+
|
|
799
|
+
Add before `case 'list':`:
|
|
800
|
+
|
|
801
|
+
```typescript
|
|
802
|
+
case 'learn': {
|
|
803
|
+
const result = store.runLearning()
|
|
804
|
+
return { content: [{ type: 'text' as const, text: JSON.stringify(result) }] }
|
|
805
|
+
}
|
|
806
|
+
|
|
807
|
+
case 'audit': {
|
|
808
|
+
const report = store.runAudit()
|
|
809
|
+
return { content: [{ type: 'text' as const, text: JSON.stringify(report) }] }
|
|
810
|
+
}
|
|
811
|
+
```
|
|
812
|
+
|
|
813
|
+
- [ ] **Step 5: Add startup auto-learn with nextTick**
|
|
814
|
+
|
|
815
|
+
Replace lines 62-63 (startup maintenance):
|
|
816
|
+
|
|
817
|
+
```typescript
|
|
818
|
+
// Startup maintenance
|
|
819
|
+
store.decayTrustScores()
|
|
820
|
+
store.auditContradictions()
|
|
821
|
+
|
|
822
|
+
// Auto-learn on startup (non-blocking)
|
|
823
|
+
process.nextTick(() => {
|
|
824
|
+
try {
|
|
825
|
+
const result = store.runLearning()
|
|
826
|
+
if (result.demoted > 0 || result.aged > 0 || result.long_facts.length > 0) {
|
|
827
|
+
console.error(`[mnemo:auto-learn] promoted=${result.promoted} demoted=${result.demoted} aged=${result.aged} long_facts=${result.long_facts.length}`)
|
|
828
|
+
}
|
|
829
|
+
} catch (err) {
|
|
830
|
+
console.error('[mnemo:auto-learn] error:', err)
|
|
831
|
+
}
|
|
832
|
+
})
|
|
833
|
+
```
|
|
834
|
+
|
|
835
|
+
- [ ] **Step 6: Run build + all tests**
|
|
836
|
+
|
|
837
|
+
Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npm run build && npx vitest run`
|
|
838
|
+
Expected: BUILD OK, ALL TESTS PASS
|
|
839
|
+
|
|
840
|
+
- [ ] **Step 7: Commit**
|
|
841
|
+
|
|
842
|
+
```bash
|
|
843
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp
|
|
844
|
+
git add src/server.ts
|
|
845
|
+
git commit -m "feat(server): add learn/audit actions, summary support, length warning, auto-learn on startup"
|
|
846
|
+
```
|
|
847
|
+
|
|
848
|
+
---
|
|
849
|
+
|
|
850
|
+
## Task 5: End-to-end verification
|
|
851
|
+
|
|
852
|
+
**Files:** None (manual verification)
|
|
853
|
+
|
|
854
|
+
- [ ] **Step 1: Build and test**
|
|
855
|
+
|
|
856
|
+
Run: `cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npm run build && npx vitest run`
|
|
857
|
+
Expected: ALL PASS
|
|
858
|
+
|
|
859
|
+
- [ ] **Step 2: Test against real database**
|
|
860
|
+
|
|
861
|
+
```bash
|
|
862
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp
|
|
863
|
+
echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}
|
|
864
|
+
{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"fact_store","arguments":{"action":"search","query":"你是谁"}}}' | node dist/server.js 2>&1 | tail -5
|
|
865
|
+
```
|
|
866
|
+
|
|
867
|
+
- [ ] **Step 3: Test audit action**
|
|
868
|
+
|
|
869
|
+
```bash
|
|
870
|
+
echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}
|
|
871
|
+
{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"fact_store","arguments":{"action":"audit"}}}' | node dist/server.js 2>&1 | tail -5
|
|
872
|
+
```
|
|
873
|
+
|
|
874
|
+
- [ ] **Step 4: Bump version**
|
|
875
|
+
|
|
876
|
+
```bash
|
|
877
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npm version minor --no-git-tag-version
|
|
878
|
+
```
|
|
879
|
+
|
|
880
|
+
- [ ] **Step 5: Final commit**
|
|
881
|
+
|
|
882
|
+
```bash
|
|
883
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp
|
|
884
|
+
git add .
|
|
885
|
+
git commit -m "chore: prepare v0.2.0 — memory self-learning release"
|
|
886
|
+
```
|
|
887
|
+
|
|
888
|
+
---
|
|
889
|
+
|
|
890
|
+
## Self-Review
|
|
891
|
+
|
|
892
|
+
### Spec Coverage
|
|
893
|
+
|
|
894
|
+
| Spec Requirement | Task |
|
|
895
|
+
|-----------------|------|
|
|
896
|
+
| retrieval_log 表 + [{id, score}] 格式 | Task 1 (schema) + Task 2 (logRetrieval) |
|
|
897
|
+
| logRetrieval 更新 last_retrieved_at | Task 2 |
|
|
898
|
+
| pruneRetrievalLog 5000 条上限 | Task 2 |
|
|
899
|
+
| learn action: rate-based trust adjustment | Task 2 (runLearning) + Task 4 (handler) |
|
|
900
|
+
| learn action: aging based on last_retrieved_at | Task 2 |
|
|
901
|
+
| learn action: new fact protection (NULL last_retrieved_at) | Task 2 |
|
|
902
|
+
| learn returns long_facts report | Task 2 |
|
|
903
|
+
| audit action: quality report, no data modification | Task 2 (runAudit) + Task 4 (handler) |
|
|
904
|
+
| length penalty: matchText-based (summary or content) | Task 3 |
|
|
905
|
+
| length penalty: 300-char threshold | Task 3 |
|
|
906
|
+
| static FTS/Jaccard 0.5/0.5 weights | Task 3 |
|
|
907
|
+
| summary field on facts table | Task 1 |
|
|
908
|
+
| summary indexed by FTS5 | Task 1 (schema + triggers) |
|
|
909
|
+
| summary matching in retriever | Task 3 |
|
|
910
|
+
| 500-char write warning | Task 4 |
|
|
911
|
+
| add/update support summary param | Task 4 |
|
|
912
|
+
| server startup auto-learn with nextTick | Task 4 |
|
|
913
|
+
| keep refineQuery | No change needed (already in code) |
|
|
914
|
+
| remove relevance gate | Task 3 |
|
|
915
|
+
| remove content dedup | Task 3 |
|
|
916
|
+
| backward compatible migration | Task 1 |
|
|
917
|
+
|
|
918
|
+
### Placeholder Scan
|
|
919
|
+
|
|
920
|
+
- [x] No "TBD", "TODO", "implement later"
|
|
921
|
+
- [x] No vague "add error handling" without code
|
|
922
|
+
- [x] All file paths are exact
|
|
923
|
+
- [x] All code blocks contain complete implementations
|
|
924
|
+
|
|
925
|
+
### Type Consistency
|
|
926
|
+
|
|
927
|
+
- [x] `Fact.summary` is `string | null` in types.ts, store.ts rowToFact, retriever.ts
|
|
928
|
+
- [x] `Fact.lastRetrievedAt` is `string | null` in types.ts and store.ts
|
|
929
|
+
- [x] `FactStoreArgs.action` includes `'learn' | 'audit'` in types.ts and server.ts zod schema
|
|
930
|
+
- [x] `FactStoreArgs.summary` is `string` optional in types.ts and server.ts zod schema
|
|
931
|
+
- [x] `runLearning()` return type matches `{promoted, demoted, aged, unchanged, long_facts}` in store.ts and server.ts handler
|
|
932
|
+
- [x] `runAudit()` return type matches `{total_facts, long_without_summary, low_helpful_rate, aging_candidates}` in store.ts and server.ts handler
|