@morningljn/mnemo 0.1.2 → 0.1.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +44 -15
- package/README_zh.md +1 -1
- package/dist/cache.d.ts +23 -0
- package/dist/cache.js +44 -0
- package/dist/cache.js.map +1 -0
- package/dist/init.js +16 -8
- package/dist/init.js.map +1 -1
- package/dist/metrics.d.ts +31 -0
- package/dist/metrics.js +57 -0
- package/dist/metrics.js.map +1 -0
- package/dist/refine.d.ts +14 -0
- package/dist/refine.js +115 -0
- package/dist/refine.js.map +1 -0
- package/dist/resources.d.ts +27 -0
- package/dist/resources.js +56 -0
- package/dist/resources.js.map +1 -0
- package/dist/retriever.d.ts +14 -2
- package/dist/retriever.js +126 -36
- package/dist/retriever.js.map +1 -1
- package/dist/server.js +40 -16
- package/dist/server.js.map +1 -1
- package/dist/types.d.ts +2 -2
- package/docs/superpowers/plans/2026-05-15-mnemo-mcp.md +1154 -0
- package/docs/superpowers/plans/2026-05-16-mnemo-query-cache.md +613 -0
- package/docs/superpowers/plans/2026-05-16-retrieval-and-injection-optimization.md +770 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/.openspec.yaml +2 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/design.md +83 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/proposal.md +32 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/specs/fact-retrieval/spec.md +75 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/specs/fact-store/spec.md +83 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/specs/mcp-server/spec.md +34 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/specs/security/spec.md +37 -0
- package/openspec/changes/archive/2026-05-15-mnemo-mcp/tasks.md +44 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/.openspec.yaml +2 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/design.md +96 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/proposal.md +29 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/specs/batch-operations/spec.md +42 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/specs/perf-metrics/spec.md +55 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/specs/query-cache/spec.md +65 -0
- package/openspec/changes/archive/2026-05-16-mnemo-query-cache/tasks.md +45 -0
- package/openspec/changes/retrieval-and-injection-optimization/.openspec.yaml +2 -0
- package/openspec/changes/retrieval-and-injection-optimization/design.md +117 -0
- package/openspec/changes/retrieval-and-injection-optimization/proposal.md +30 -0
- package/openspec/changes/retrieval-and-injection-optimization/specs/adaptive-scoring/spec.md +43 -0
- package/openspec/changes/retrieval-and-injection-optimization/specs/injection-protocol/spec.md +48 -0
- package/openspec/changes/retrieval-and-injection-optimization/specs/mcp-resources/spec.md +39 -0
- package/openspec/changes/retrieval-and-injection-optimization/specs/query-refinement/spec.md +39 -0
- package/openspec/changes/retrieval-and-injection-optimization/tasks.md +33 -0
- package/openspec/config.yaml +20 -0
- package/package.json +1 -1
- package/src/cache.ts +65 -0
- package/src/init.ts +17 -9
- package/src/metrics.ts +81 -0
- package/src/refine.ts +127 -0
- package/src/resources.ts +78 -0
- package/src/retriever.ts +141 -34
- package/src/server.ts +42 -17
- package/src/types.ts +2 -2
- package/tests/refine.test.ts +52 -0
- package/tests/resource.test.ts +62 -0
|
@@ -0,0 +1,770 @@
|
|
|
1
|
+
# Retrieval and Injection Optimization Implementation Plan
|
|
2
|
+
|
|
3
|
+
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
4
|
+
|
|
5
|
+
**Goal:** Improve mnemo-mcp retrieval accuracy and reduce injection overhead by adding MCP Resources for session warmup, query refinement, adaptive scoring, and a new injection protocol.
|
|
6
|
+
|
|
7
|
+
**Architecture:** Add a `ResourceManager` to serve per-category memory summaries via MCP Resource protocol, a `refineQuery()` pure function to strip noise tokens from user messages, dynamic FTS/Jaccard weighting based on query length, content-based deduplication, and a relevance score gate. Update CLAUDE.md rules to shift from "search every message" to "Resource warmup + on-demand search".
|
|
8
|
+
|
|
9
|
+
**Tech Stack:** TypeScript, Node.js, better-sqlite3, @modelcontextprotocol/sdk v1.29, zod v4, vitest
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## File Structure
|
|
14
|
+
|
|
15
|
+
| File | Responsibility |
|
|
16
|
+
|------|---------------|
|
|
17
|
+
| `src/refine.ts` | **NEW** — `refineQuery()` pure function: token filtering, action word removal, entity extraction |
|
|
18
|
+
| `src/resources.ts` | **NEW** — `ResourceManager` class: per-category Resource registration, caching, cache invalidation |
|
|
19
|
+
| `src/retriever.ts` | **MODIFY** — Dynamic scoring weights, relevance gate, content-based deduplication, integrate refineQuery |
|
|
20
|
+
| `src/server.ts` | **MODIFY** — Wire ResourceManager, pass refineQuery into search flow |
|
|
21
|
+
| `src/types.ts` | **MODIFY** — Add `RefineResult` type |
|
|
22
|
+
| `tests/refine.test.ts` | **NEW** — Query refinement unit tests |
|
|
23
|
+
| `tests/resource.test.ts` | **NEW** — Resource caching and invalidation tests |
|
|
24
|
+
| `tests/retriever.test.ts` | **MODIFY** — Dynamic weight, relevance gate, dedup tests |
|
|
25
|
+
| `CLAUDE.md` (user-side) | **UPDATE** — New injection protocol rules (provided in docs) |
|
|
26
|
+
|
|
27
|
+
---
|
|
28
|
+
|
|
29
|
+
## Task 1: Query Refinement Module
|
|
30
|
+
|
|
31
|
+
**Files:**
|
|
32
|
+
- Create: `src/refine.ts`
|
|
33
|
+
- Test: `tests/refine.test.ts`
|
|
34
|
+
|
|
35
|
+
- [ ] **Step 1: Create `src/refine.ts`**
|
|
36
|
+
|
|
37
|
+
```typescript
|
|
38
|
+
/**
|
|
39
|
+
* Query refinement: strip noise tokens from user messages before memory search.
|
|
40
|
+
* Pure function — no side effects, no DB access.
|
|
41
|
+
*/
|
|
42
|
+
|
|
43
|
+
import type { FactCategory } from './types.js'
|
|
44
|
+
|
|
45
|
+
// Action words / helper phrases to strip (Chinese)
|
|
46
|
+
const ACTION_WORDS = new Set([
|
|
47
|
+
'帮我', '看看', '看一下', '做一下', '帮我看看', '能不能', '为什么', '怎么',
|
|
48
|
+
'是什么', '如何', '请', '麻烦', '可以', '能不能', '能不能帮我', '给我',
|
|
49
|
+
'给我看看', '给我做', '给我写', '给我查', '给我找', '给我说', '给我讲',
|
|
50
|
+
'告诉我', '跟我说', '跟我讲', '给我解释', '给我说明', '给我介绍',
|
|
51
|
+
'运行', '执行', '启动', '停止', '创建', '删除', '修改', '更新', '查看',
|
|
52
|
+
'检查', '测试', '提交', '推送', '拉取', '合并', '切换', '重置',
|
|
53
|
+
])
|
|
54
|
+
|
|
55
|
+
// Reuse existing stop words from retriever
|
|
56
|
+
const CN_STOP_WORDS = new Set([
|
|
57
|
+
'的', '了', '是', '在', '有', '和', '就', '不', '人', '都',
|
|
58
|
+
'一', '个', '上', '也', '很', '到', '说', '要', '去', '你',
|
|
59
|
+
'会', '着', '没', '看', '好', '自', '这', '他', '她', '它',
|
|
60
|
+
'那', '些', '用', '对', '下', '为', '从', '被', '把', '能',
|
|
61
|
+
'可', '以', '所', '而', '又', '与', '但', '或', '等', '中',
|
|
62
|
+
'大', '小', '多', '少', '其', '之', '做', '让', '给', '已',
|
|
63
|
+
'还', '来', '地', '得', '过', '时', '里', '后', '前', '当',
|
|
64
|
+
])
|
|
65
|
+
|
|
66
|
+
export interface RefineResult {
|
|
67
|
+
query: string | null
|
|
68
|
+
tokens: string[]
|
|
69
|
+
entityTokens: string[]
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
/**
|
|
73
|
+
* Refine a raw user message into memory-searchable keywords.
|
|
74
|
+
* Returns null if the message is a pure operation command with no memory relevance.
|
|
75
|
+
*/
|
|
76
|
+
export function refineQuery(raw: string): RefineResult | null {
|
|
77
|
+
const trimmed = raw.trim()
|
|
78
|
+
if (!trimmed) return null
|
|
79
|
+
|
|
80
|
+
// Extract high-signal tokens first: quoted content, book titles, capitalized phrases
|
|
81
|
+
const entityTokens: string[] = []
|
|
82
|
+
|
|
83
|
+
// Chinese quotes: 「深色主题」 or "深色主题" or '深色主题'
|
|
84
|
+
for (const m of trimmed.matchAll(/[「""'']([^「""'']{2,20})[」""'']?/g)) {
|
|
85
|
+
entityTokens.push(m[1])
|
|
86
|
+
}
|
|
87
|
+
// Book titles: 《记忆系统》
|
|
88
|
+
for (const m of trimmed.matchAll(/《([^》]+)》/g)) {
|
|
89
|
+
entityTokens.push(m[1])
|
|
90
|
+
}
|
|
91
|
+
// Capitalized English phrases: "TypeScript", "Visual Studio Code"
|
|
92
|
+
for (const m of trimmed.matchAll(/\b([A-Z][a-zA-Z]*(?:\s+[A-Z][a-zA-Z]*)+)\b/g)) {
|
|
93
|
+
entityTokens.push(m[1])
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
// Tokenize: split by spaces and Chinese character boundaries
|
|
97
|
+
const tokens: string[] = []
|
|
98
|
+
const parts = trimmed.split(/\s+/)
|
|
99
|
+
for (const part of parts) {
|
|
100
|
+
// English words
|
|
101
|
+
for (const word of part.match(/[a-zA-Z0-9_\-.]+/g) ?? []) {
|
|
102
|
+
if (word.length >= 2) tokens.push(word)
|
|
103
|
+
}
|
|
104
|
+
// Chinese characters (individual chars and 2-grams)
|
|
105
|
+
const cnChars = part.match(/[\u4e00-\u9fff]/g) ?? []
|
|
106
|
+
for (const c of cnChars) {
|
|
107
|
+
if (!CN_STOP_WORDS.has(c)) tokens.push(c)
|
|
108
|
+
}
|
|
109
|
+
// Chinese 2-grams for better matching
|
|
110
|
+
for (let i = 0; i < cnChars.length - 1; i++) {
|
|
111
|
+
const bigram = cnChars[i] + cnChars[i + 1]
|
|
112
|
+
tokens.push(bigram)
|
|
113
|
+
}
|
|
114
|
+
}
|
|
115
|
+
|
|
116
|
+
// Filter action words and stop words
|
|
117
|
+
const filtered = tokens.filter(t => {
|
|
118
|
+
if (ACTION_WORDS.has(t)) return false
|
|
119
|
+
if (CN_STOP_WORDS.has(t)) return false
|
|
120
|
+
if (t.length < 2) return false
|
|
121
|
+
return true
|
|
122
|
+
})
|
|
123
|
+
|
|
124
|
+
// Deduplicate while preserving order
|
|
125
|
+
const seen = new Set<string>()
|
|
126
|
+
const deduped: string[] = []
|
|
127
|
+
for (const t of filtered) {
|
|
128
|
+
if (!seen.has(t)) {
|
|
129
|
+
seen.add(t)
|
|
130
|
+
deduped.push(t)
|
|
131
|
+
}
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
// If nothing left after filtering, check if we have entity tokens
|
|
135
|
+
if (deduped.length === 0 && entityTokens.length === 0) {
|
|
136
|
+
return null
|
|
137
|
+
}
|
|
138
|
+
|
|
139
|
+
// Combine: entity tokens first (higher signal), then deduped tokens
|
|
140
|
+
const allTokens = [...entityTokens, ...deduped.filter(t => !entityTokens.includes(t))]
|
|
141
|
+
const query = allTokens.join(' ')
|
|
142
|
+
|
|
143
|
+
return { query, tokens: deduped, entityTokens }
|
|
144
|
+
}
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
- [ ] **Step 2: Create `tests/refine.test.ts`**
|
|
148
|
+
|
|
149
|
+
```typescript
|
|
150
|
+
import { describe, it, expect } from 'vitest'
|
|
151
|
+
import { refineQuery } from '../src/refine.js'
|
|
152
|
+
|
|
153
|
+
describe('refineQuery', () => {
|
|
154
|
+
it('filters action words from Chinese query', () => {
|
|
155
|
+
const result = refineQuery('帮我用 TypeScript 重构 auth 模块')
|
|
156
|
+
expect(result).not.toBeNull()
|
|
157
|
+
expect(result!.query).toContain('TypeScript')
|
|
158
|
+
expect(result!.query).toContain('auth')
|
|
159
|
+
expect(result!.query).not.toContain('帮我')
|
|
160
|
+
expect(result!.query).not.toContain('重构')
|
|
161
|
+
})
|
|
162
|
+
|
|
163
|
+
it('returns null for pure operation commands', () => {
|
|
164
|
+
expect(refineQuery('运行测试')).toBeNull()
|
|
165
|
+
expect(refineQuery('git status')).toBeNull()
|
|
166
|
+
expect(refineQuery('创建文件')).toBeNull()
|
|
167
|
+
})
|
|
168
|
+
|
|
169
|
+
it('extracts quoted Chinese entities', () => {
|
|
170
|
+
const result = refineQuery('我喜欢「深色主题」')
|
|
171
|
+
expect(result).not.toBeNull()
|
|
172
|
+
expect(result!.entityTokens).toContain('深色主题')
|
|
173
|
+
expect(result!.query).toContain('深色主题')
|
|
174
|
+
})
|
|
175
|
+
|
|
176
|
+
it('extracts book title entities', () => {
|
|
177
|
+
const result = refineQuery('读了《设计模式》这本书')
|
|
178
|
+
expect(result).not.toBeNull()
|
|
179
|
+
expect(result!.entityTokens).toContain('设计模式')
|
|
180
|
+
})
|
|
181
|
+
|
|
182
|
+
it('extracts capitalized English phrases', () => {
|
|
183
|
+
const result = refineQuery('使用 Visual Studio Code 编辑器')
|
|
184
|
+
expect(result).not.toBeNull()
|
|
185
|
+
expect(result!.entityTokens).toContain('Visual Studio Code')
|
|
186
|
+
})
|
|
187
|
+
|
|
188
|
+
it('returns null for empty string', () => {
|
|
189
|
+
expect(refineQuery('')).toBeNull()
|
|
190
|
+
expect(refineQuery(' ')).toBeNull()
|
|
191
|
+
})
|
|
192
|
+
|
|
193
|
+
it('preserves meaningful Chinese tokens', () => {
|
|
194
|
+
const result = refineQuery('用户偏好深色主题')
|
|
195
|
+
expect(result).not.toBeNull()
|
|
196
|
+
expect(result!.query).toContain('用户')
|
|
197
|
+
expect(result!.query).toContain('偏好')
|
|
198
|
+
expect(result!.query).toContain('深色')
|
|
199
|
+
expect(result!.query).toContain('主题')
|
|
200
|
+
})
|
|
201
|
+
})
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
- [ ] **Step 3: Run tests**
|
|
205
|
+
|
|
206
|
+
```bash
|
|
207
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/refine.test.ts
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
Expected: 7 tests PASS
|
|
211
|
+
|
|
212
|
+
- [ ] **Step 4: Commit**
|
|
213
|
+
|
|
214
|
+
```bash
|
|
215
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && git add src/refine.ts tests/refine.test.ts && git commit -m "feat(refine): add query refinement module with action word filtering"
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
---
|
|
219
|
+
|
|
220
|
+
## Task 2: Resource Manager Module
|
|
221
|
+
|
|
222
|
+
**Files:**
|
|
223
|
+
- Create: `src/resources.ts`
|
|
224
|
+
- Test: `tests/resource.test.ts`
|
|
225
|
+
|
|
226
|
+
- [ ] **Step 1: Create `src/resources.ts`**
|
|
227
|
+
|
|
228
|
+
```typescript
|
|
229
|
+
/**
|
|
230
|
+
* MCP Resource manager for mnemo-mcp.
|
|
231
|
+
* Exposes per-category memory summaries as MCP Resources for session warmup injection.
|
|
232
|
+
*/
|
|
233
|
+
|
|
234
|
+
import type { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'
|
|
235
|
+
import { ResourceTemplate } from '@modelcontextprotocol/sdk/server/mcp.js'
|
|
236
|
+
import type { MemoryStore } from './store.js'
|
|
237
|
+
import type { FactCategory } from './types.js'
|
|
238
|
+
|
|
239
|
+
const CATEGORIES: FactCategory[] = ['identity', 'coding_style', 'tool_pref', 'workflow', 'general']
|
|
240
|
+
const RESOURCE_LIMIT = 10
|
|
241
|
+
|
|
242
|
+
export interface ResourceFact {
|
|
243
|
+
fact_id: number
|
|
244
|
+
content: string
|
|
245
|
+
trust_score: number
|
|
246
|
+
}
|
|
247
|
+
|
|
248
|
+
export class ResourceManager {
|
|
249
|
+
private cache = new Map<FactCategory, ResourceFact[]>()
|
|
250
|
+
|
|
251
|
+
constructor(
|
|
252
|
+
private server: McpServer,
|
|
253
|
+
private store: MemoryStore,
|
|
254
|
+
) {}
|
|
255
|
+
|
|
256
|
+
/** Register all category resources with the MCP server */
|
|
257
|
+
registerResources(): void {
|
|
258
|
+
for (const category of CATEGORIES) {
|
|
259
|
+
const template = new ResourceTemplate(
|
|
260
|
+
`mnemo://global/{category}`,
|
|
261
|
+
{ list: undefined },
|
|
262
|
+
)
|
|
263
|
+
this.server.registerResource(
|
|
264
|
+
`mnemo-global-${category}`,
|
|
265
|
+
template,
|
|
266
|
+
{
|
|
267
|
+
description: `${category} category global facts (top ${RESOURCE_LIMIT} by trust)`,
|
|
268
|
+
mimeType: 'application/json',
|
|
269
|
+
},
|
|
270
|
+
(uri, variables) => this.readResource(uri, variables),
|
|
271
|
+
)
|
|
272
|
+
}
|
|
273
|
+
}
|
|
274
|
+
|
|
275
|
+
/** Read handler for resource template */
|
|
276
|
+
private readResource(uri: URL, variables: Record<string, string | string[]>): { contents: Array<{ uri: string; mimeType: string; text: string }> } {
|
|
277
|
+
const category = (Array.isArray(variables.category) ? variables.category[0] : variables.category) as FactCategory
|
|
278
|
+
if (!CATEGORIES.includes(category)) {
|
|
279
|
+
return { contents: [{ uri: uri.toString(), mimeType: 'application/json', text: '[]' }] }
|
|
280
|
+
}
|
|
281
|
+
|
|
282
|
+
const facts = this.getFacts(category)
|
|
283
|
+
return {
|
|
284
|
+
contents: [{
|
|
285
|
+
uri: uri.toString(),
|
|
286
|
+
mimeType: 'application/json',
|
|
287
|
+
text: JSON.stringify(facts, null, 2),
|
|
288
|
+
}],
|
|
289
|
+
}
|
|
290
|
+
}
|
|
291
|
+
|
|
292
|
+
/** Get facts for a category — with caching */
|
|
293
|
+
getFacts(category: FactCategory): ResourceFact[] {
|
|
294
|
+
const cached = this.cache.get(category)
|
|
295
|
+
if (cached) return cached
|
|
296
|
+
|
|
297
|
+
const facts = this.store.listFacts(category, 0.0, RESOURCE_LIMIT).map(f => ({
|
|
298
|
+
fact_id: f.factId,
|
|
299
|
+
content: f.content,
|
|
300
|
+
trust_score: f.trustScore,
|
|
301
|
+
}))
|
|
302
|
+
|
|
303
|
+
this.cache.set(category, facts)
|
|
304
|
+
return facts
|
|
305
|
+
}
|
|
306
|
+
|
|
307
|
+
/** Invalidate all caches — call after any write operation */
|
|
308
|
+
invalidate(): void {
|
|
309
|
+
this.cache.clear()
|
|
310
|
+
}
|
|
311
|
+
|
|
312
|
+
/** Get cache size for debugging */
|
|
313
|
+
cacheSize(): number {
|
|
314
|
+
return this.cache.size
|
|
315
|
+
}
|
|
316
|
+
}
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
- [ ] **Step 2: Create `tests/resource.test.ts`**
|
|
320
|
+
|
|
321
|
+
```typescript
|
|
322
|
+
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
|
|
323
|
+
import { MemoryStore } from '../src/store.js'
|
|
324
|
+
import { ResourceManager } from '../src/resources.js'
|
|
325
|
+
import { mkdtempSync, rmSync } from 'node:fs'
|
|
326
|
+
import { join } from 'node:path'
|
|
327
|
+
import { tmpdir } from 'node:os'
|
|
328
|
+
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'
|
|
329
|
+
|
|
330
|
+
let store: MemoryStore
|
|
331
|
+
let server: McpServer
|
|
332
|
+
let manager: ResourceManager
|
|
333
|
+
let tmpDir: string
|
|
334
|
+
|
|
335
|
+
beforeEach(() => {
|
|
336
|
+
tmpDir = mkdtempSync(join(tmpdir(), 'mnemo-test-'))
|
|
337
|
+
store = new MemoryStore(join(tmpDir, 'test.db'))
|
|
338
|
+
server = new McpServer({ name: 'test', version: '0.1.0' })
|
|
339
|
+
manager = new ResourceManager(server, store)
|
|
340
|
+
})
|
|
341
|
+
|
|
342
|
+
afterEach(() => {
|
|
343
|
+
store.close()
|
|
344
|
+
rmSync(tmpDir, { recursive: true, force: true })
|
|
345
|
+
})
|
|
346
|
+
|
|
347
|
+
describe('ResourceManager', () => {
|
|
348
|
+
it('returns empty array for empty category', () => {
|
|
349
|
+
const facts = manager.getFacts('identity')
|
|
350
|
+
expect(facts).toEqual([])
|
|
351
|
+
})
|
|
352
|
+
|
|
353
|
+
it('returns facts ordered by trust score', () => {
|
|
354
|
+
store.addFact('用户偏好深色主题', 'tool_pref')
|
|
355
|
+
store.addFact('用户喜欢 VS Code', 'tool_pref')
|
|
356
|
+
const facts = manager.getFacts('tool_pref')
|
|
357
|
+
expect(facts.length).toBe(2)
|
|
358
|
+
expect(facts[0].content).toBe('用户偏好深色主题')
|
|
359
|
+
})
|
|
360
|
+
|
|
361
|
+
it('caches results', () => {
|
|
362
|
+
store.addFact('测试事实', 'general')
|
|
363
|
+
manager.getFacts('general')
|
|
364
|
+
expect(manager.cacheSize()).toBe(1)
|
|
365
|
+
// Second call should hit cache
|
|
366
|
+
const facts2 = manager.getFacts('general')
|
|
367
|
+
expect(facts2.length).toBe(1)
|
|
368
|
+
})
|
|
369
|
+
|
|
370
|
+
it('invalidates cache on write', () => {
|
|
371
|
+
store.addFact('测试事实', 'general')
|
|
372
|
+
manager.getFacts('general')
|
|
373
|
+
expect(manager.cacheSize()).toBe(1)
|
|
374
|
+
manager.invalidate()
|
|
375
|
+
expect(manager.cacheSize()).toBe(0)
|
|
376
|
+
})
|
|
377
|
+
|
|
378
|
+
it('limits to top 10 facts', () => {
|
|
379
|
+
for (let i = 0; i < 15; i++) {
|
|
380
|
+
store.addFact(`事实 ${i}`, 'general')
|
|
381
|
+
}
|
|
382
|
+
const facts = manager.getFacts('general')
|
|
383
|
+
expect(facts.length).toBe(10)
|
|
384
|
+
})
|
|
385
|
+
})
|
|
386
|
+
```
|
|
387
|
+
|
|
388
|
+
- [ ] **Step 3: Run tests**
|
|
389
|
+
|
|
390
|
+
```bash
|
|
391
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/resource.test.ts
|
|
392
|
+
```
|
|
393
|
+
|
|
394
|
+
Expected: 5 tests PASS
|
|
395
|
+
|
|
396
|
+
- [ ] **Step 4: Commit**
|
|
397
|
+
|
|
398
|
+
```bash
|
|
399
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && git add src/resources.ts tests/resource.test.ts && git commit -m "feat(resources): add MCP Resource manager for per-category memory warmup"
|
|
400
|
+
```
|
|
401
|
+
|
|
402
|
+
---
|
|
403
|
+
|
|
404
|
+
## Task 3: Update Retriever with Dynamic Scoring, Relevance Gate, and Deduplication
|
|
405
|
+
|
|
406
|
+
**Files:**
|
|
407
|
+
- Modify: `src/retriever.ts`
|
|
408
|
+
- Modify: `src/types.ts`
|
|
409
|
+
- Test: `tests/retriever.test.ts`
|
|
410
|
+
|
|
411
|
+
- [ ] **Step 1: Update `src/types.ts` — add `RefineResult`**
|
|
412
|
+
|
|
413
|
+
Add to `src/types.ts` after `SecurityScanResult`:
|
|
414
|
+
|
|
415
|
+
```typescript
|
|
416
|
+
/** 查询提炼结果 */
|
|
417
|
+
export interface RefineResult {
|
|
418
|
+
query: string | null
|
|
419
|
+
tokens: string[]
|
|
420
|
+
entityTokens: string[]
|
|
421
|
+
}
|
|
422
|
+
```
|
|
423
|
+
|
|
424
|
+
- [ ] **Step 2: Modify `src/retriever.ts` — integrate refineQuery, dynamic weights, relevance gate, dedup**
|
|
425
|
+
|
|
426
|
+
Replace the `search()` method (lines 68-171) with:
|
|
427
|
+
|
|
428
|
+
```typescript
|
|
429
|
+
/** 主搜索:FTS5 → LIKE → 字符交叉 → 分类推断 → Jaccard → 信任评分 → 时间衰减 */
|
|
430
|
+
search(query: string, options?: SearchOptions & { skipRefine?: boolean }): ScoredFact[] {
|
|
431
|
+
const startTime = performance.now()
|
|
432
|
+
const minTrust = options?.minTrust ?? 0.3
|
|
433
|
+
const limit = options?.limit ?? 10
|
|
434
|
+
const category = options?.category
|
|
435
|
+
|
|
436
|
+
// Query refinement (unless explicitly skipped)
|
|
437
|
+
let searchQuery = query
|
|
438
|
+
if (!options?.skipRefine) {
|
|
439
|
+
const refined = refineQuery(query)
|
|
440
|
+
if (refined?.query) {
|
|
441
|
+
searchQuery = refined.query
|
|
442
|
+
}
|
|
443
|
+
}
|
|
444
|
+
|
|
445
|
+
// Cache check using refined query
|
|
446
|
+
const cacheKey = this.cache.makeKey({ action: 'search', query: searchQuery, category, minTrust, limit })
|
|
447
|
+
const cached = this.cache.get(cacheKey)
|
|
448
|
+
if (cached) {
|
|
449
|
+
this.metrics.record({ action: 'search', durationMs: performance.now() - startTime, resultCount: cached.length, cacheHit: true })
|
|
450
|
+
return cached
|
|
451
|
+
}
|
|
452
|
+
|
|
453
|
+
// Query bilingual expansion
|
|
454
|
+
const expandedQuery = this.expandQueryBilingually(searchQuery)
|
|
455
|
+
|
|
456
|
+
// Stage 1: FTS5 candidates with fallback chain
|
|
457
|
+
let candidates = this.ftsCandidates(expandedQuery, category, minTrust, limit * 3)
|
|
458
|
+
if (candidates.length === 0) candidates = this.likeFallback(expandedQuery, category, minTrust, limit * 3)
|
|
459
|
+
if (candidates.length === 0) candidates = this.charOverlapFallback(expandedQuery, category, minTrust, limit * 3)
|
|
460
|
+
if (candidates.length === 0) {
|
|
461
|
+
if (!category) {
|
|
462
|
+
const inferred = this.categoryInferFallback(searchQuery, minTrust, limit)
|
|
463
|
+
if (inferred.length > 0) return inferred
|
|
464
|
+
}
|
|
465
|
+
if (this.isPersonalQuery(searchQuery)) {
|
|
466
|
+
return this.trustFallback(category, minTrust, limit)
|
|
467
|
+
}
|
|
468
|
+
return []
|
|
469
|
+
}
|
|
470
|
+
|
|
471
|
+
// Dynamic weighting based on query token count
|
|
472
|
+
const queryTokens = this.tokenize(searchQuery)
|
|
473
|
+
const tokenCount = queryTokens.size
|
|
474
|
+
const ftsWeight = tokenCount <= 3 ? 0.7 : 0.3
|
|
475
|
+
const jaccardWeight = tokenCount <= 3 ? 0.3 : 0.7
|
|
476
|
+
|
|
477
|
+
// Stage 2-4: Score candidates
|
|
478
|
+
const scored: ScoredFact[] = []
|
|
479
|
+
for (const fact of candidates) {
|
|
480
|
+
const contentTokens = this.tokenize(fact.content)
|
|
481
|
+
const tagTokens = this.tokenize(fact.tags)
|
|
482
|
+
const allTokens = new Set([...contentTokens, ...tagTokens])
|
|
483
|
+
|
|
484
|
+
const jaccard = this.jaccardSimilarity(queryTokens, allTokens)
|
|
485
|
+
const qInF = this.containmentScore(queryTokens, allTokens)
|
|
486
|
+
const similarity = 0.3 * jaccard + 0.7 * qInF
|
|
487
|
+
const ftsScore = fact.ftsRank
|
|
488
|
+
|
|
489
|
+
// Dynamic relevance
|
|
490
|
+
const relevance = ftsWeight * ftsScore + jaccardWeight * similarity
|
|
491
|
+
let score = relevance * fact.trustScore
|
|
492
|
+
|
|
493
|
+
if (this.halfLifeDays > 0) {
|
|
494
|
+
score *= this.temporalDecay(fact.updatedAt || fact.createdAt)
|
|
495
|
+
}
|
|
496
|
+
|
|
497
|
+
scored.push({ ...fact, score })
|
|
498
|
+
}
|
|
499
|
+
|
|
500
|
+
scored.sort((a, b) => b.score - a.score)
|
|
501
|
+
|
|
502
|
+
// Relevance gate: filter out low-relevance results
|
|
503
|
+
const RELEVANCE_THRESHOLD = 0.15
|
|
504
|
+
const gated = scored.filter(s => s.score >= RELEVANCE_THRESHOLD)
|
|
505
|
+
const resultsToDedup = gated.length > 0 ? gated : scored
|
|
506
|
+
|
|
507
|
+
// Content-based deduplication (Jaccard > 0.7) instead of category-per-top1
|
|
508
|
+
const deduped: ScoredFact[] = []
|
|
509
|
+
for (const candidate of resultsToDedup) {
|
|
510
|
+
let isDuplicate = false
|
|
511
|
+
const candidateTokens = this.tokenize(candidate.content)
|
|
512
|
+
for (const kept of deduped) {
|
|
513
|
+
const keptTokens = this.tokenize(kept.content)
|
|
514
|
+
if (this.jaccardSimilarity(candidateTokens, keptTokens) > 0.7) {
|
|
515
|
+
isDuplicate = true
|
|
516
|
+
break
|
|
517
|
+
}
|
|
518
|
+
}
|
|
519
|
+
if (!isDuplicate) {
|
|
520
|
+
deduped.push(candidate)
|
|
521
|
+
if (deduped.length >= limit) break
|
|
522
|
+
}
|
|
523
|
+
}
|
|
524
|
+
|
|
525
|
+
const results = deduped
|
|
526
|
+
|
|
527
|
+
// Track retrieval
|
|
528
|
+
if (results.length > 0) {
|
|
529
|
+
this.trackRetrieval(results)
|
|
530
|
+
}
|
|
531
|
+
|
|
532
|
+
this.cache.set(cacheKey, results)
|
|
533
|
+
this.metrics.record({ action: 'search', durationMs: performance.now() - startTime, resultCount: results.length, cacheHit: false, retrievalPath: 'FTS5' })
|
|
534
|
+
return results
|
|
535
|
+
}
|
|
536
|
+
```
|
|
537
|
+
|
|
538
|
+
Add import at top of `src/retriever.ts`:
|
|
539
|
+
|
|
540
|
+
```typescript
|
|
541
|
+
import { refineQuery } from './refine.js'
|
|
542
|
+
```
|
|
543
|
+
|
|
544
|
+
- [ ] **Step 3: Update `tests/retriever.test.ts` — add new test cases**
|
|
545
|
+
|
|
546
|
+
Append to existing test file:
|
|
547
|
+
|
|
548
|
+
```typescript
|
|
549
|
+
describe('dynamic scoring', () => {
|
|
550
|
+
it('uses high FTS weight for short queries', () => {
|
|
551
|
+
store.addFact('用户偏好深色主题', 'tool_pref')
|
|
552
|
+
const results = retriever.search('深色主题')
|
|
553
|
+
expect(results.length).toBeGreaterThan(0)
|
|
554
|
+
})
|
|
555
|
+
|
|
556
|
+
it('uses high Jaccard weight for long queries', () => {
|
|
557
|
+
store.addFact('用户偏好使用 TypeScript 开发后端 API', 'coding_style')
|
|
558
|
+
const results = retriever.search('为什么 TypeScript 编译报错找不到模块')
|
|
559
|
+
// Should still return something due to Jaccard overlap on "TypeScript"
|
|
560
|
+
expect(results.some(r => r.content.includes('TypeScript'))).toBe(true)
|
|
561
|
+
})
|
|
562
|
+
})
|
|
563
|
+
|
|
564
|
+
describe('relevance gate', () => {
|
|
565
|
+
it('filters out low relevance results', () => {
|
|
566
|
+
store.addFact('完全不相关的内容关于天气和食物', 'general')
|
|
567
|
+
store.addFact('用户偏好深色主题', 'tool_pref')
|
|
568
|
+
const results = retriever.search('深色主题')
|
|
569
|
+
expect(results.every(r => r.content.includes('深色') || r.content.includes('主题'))).toBe(true)
|
|
570
|
+
})
|
|
571
|
+
})
|
|
572
|
+
|
|
573
|
+
describe('content deduplication', () => {
|
|
574
|
+
it('allows multiple general facts if content differs', () => {
|
|
575
|
+
store.addFact('用户喜欢蓝色', 'general')
|
|
576
|
+
store.addFact('用户偏好深色主题', 'general')
|
|
577
|
+
store.addFact('用户使用 VS Code', 'general')
|
|
578
|
+
const results = retriever.search('用户偏好')
|
|
579
|
+
expect(results.filter(r => r.category === 'general').length).toBeGreaterThan(1)
|
|
580
|
+
})
|
|
581
|
+
|
|
582
|
+
it('deduplicates highly similar facts', () => {
|
|
583
|
+
store.addFact('用户偏好深色主题', 'tool_pref')
|
|
584
|
+
store.addFact('用户偏好深色主题和蓝色', 'tool_pref')
|
|
585
|
+
const results = retriever.search('深色主题')
|
|
586
|
+
// Both are similar but not identical; Jaccard should be high
|
|
587
|
+
const similarCount = results.filter(r => r.content.includes('深色主题')).length
|
|
588
|
+
expect(similarCount).toBeLessThanOrEqual(2)
|
|
589
|
+
})
|
|
590
|
+
})
|
|
591
|
+
```
|
|
592
|
+
|
|
593
|
+
- [ ] **Step 4: Run all retriever tests**
|
|
594
|
+
|
|
595
|
+
```bash
|
|
596
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npx vitest run tests/retriever.test.ts
|
|
597
|
+
```
|
|
598
|
+
|
|
599
|
+
Expected: All tests PASS (existing + new)
|
|
600
|
+
|
|
601
|
+
- [ ] **Step 5: Commit**
|
|
602
|
+
|
|
603
|
+
```bash
|
|
604
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && git add src/retriever.ts src/types.ts tests/retriever.test.ts && git commit -m "feat(retriever): dynamic scoring, relevance gate, content dedup, query refinement"
|
|
605
|
+
```
|
|
606
|
+
|
|
607
|
+
---
|
|
608
|
+
|
|
609
|
+
## Task 4: Wire Resources and Refinement into Server
|
|
610
|
+
|
|
611
|
+
**Files:**
|
|
612
|
+
- Modify: `src/server.ts`
|
|
613
|
+
|
|
614
|
+
- [ ] **Step 1: Update `src/server.ts` — import and wire ResourceManager**
|
|
615
|
+
|
|
616
|
+
Add imports:
|
|
617
|
+
|
|
618
|
+
```typescript
|
|
619
|
+
import { ResourceManager } from './resources.js'
|
|
620
|
+
```
|
|
621
|
+
|
|
622
|
+
After retriever initialization, add:
|
|
623
|
+
|
|
624
|
+
```typescript
|
|
625
|
+
const resourceManager = new ResourceManager(server, store)
|
|
626
|
+
resourceManager.registerResources()
|
|
627
|
+
```
|
|
628
|
+
|
|
629
|
+
In write operations (add/update/remove cases), after `retriever.getCache().clear()`, add:
|
|
630
|
+
|
|
631
|
+
```typescript
|
|
632
|
+
resourceManager.invalidate()
|
|
633
|
+
```
|
|
634
|
+
|
|
635
|
+
For the `search` case, integrate refineQuery:
|
|
636
|
+
|
|
637
|
+
```typescript
|
|
638
|
+
case 'search': {
|
|
639
|
+
if (!a.query) return { content: [{ type: 'text' as const, text: JSON.stringify({ error: 'Missing required argument: query' }) }] }
|
|
640
|
+
// Skip refinement for explicit tool calls (user knows what they're searching)
|
|
641
|
+
const results = retriever.search(a.query, { category: a.category ? category : undefined, minTrust: a.min_trust ?? minTrust, limit: a.limit ?? 10, skipRefine: true })
|
|
642
|
+
return { content: [{ type: 'text' as const, text: JSON.stringify({ results, count: results.length }) }] }
|
|
643
|
+
}
|
|
644
|
+
```
|
|
645
|
+
|
|
646
|
+
- [ ] **Step 2: Verify build**
|
|
647
|
+
|
|
648
|
+
```bash
|
|
649
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npm run build
|
|
650
|
+
```
|
|
651
|
+
|
|
652
|
+
Expected: BUILD OK (no TypeScript errors)
|
|
653
|
+
|
|
654
|
+
- [ ] **Step 3: Run all tests**
|
|
655
|
+
|
|
656
|
+
```bash
|
|
657
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && npm run test
|
|
658
|
+
```
|
|
659
|
+
|
|
660
|
+
Expected: All tests PASS
|
|
661
|
+
|
|
662
|
+
- [ ] **Step 4: Commit**
|
|
663
|
+
|
|
664
|
+
```bash
|
|
665
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && git add src/server.ts && git commit -m "feat(server): wire ResourceManager and query refinement into MCP server"
|
|
666
|
+
```
|
|
667
|
+
|
|
668
|
+
---
|
|
669
|
+
|
|
670
|
+
## Task 5: Update CLAUDE.md Rules and Documentation
|
|
671
|
+
|
|
672
|
+
**Files:**
|
|
673
|
+
- Modify: `README.md` (in project)
|
|
674
|
+
- Provide: CLAUDE.md rule template (for user to copy)
|
|
675
|
+
|
|
676
|
+
- [ ] **Step 1: Update `README.md` — add Resource and injection protocol sections**
|
|
677
|
+
|
|
678
|
+
Add after the existing Tools section:
|
|
679
|
+
|
|
680
|
+
```markdown
|
|
681
|
+
## MCP Resources
|
|
682
|
+
|
|
683
|
+
mnemo-mcp exposes 5 global category resources for session warmup injection:
|
|
684
|
+
|
|
685
|
+
| Resource URI | Description |
|
|
686
|
+
|-------------|-------------|
|
|
687
|
+
| `mnemo://global/identity` | Identity facts (top 10 by trust) |
|
|
688
|
+
| `mnemo://global/coding_style` | Coding style preferences |
|
|
689
|
+
| `mnemo://global/tool_pref` | Tool preferences |
|
|
690
|
+
| `mnemo://global/workflow` | Workflow preferences |
|
|
691
|
+
| `mnemo://global/general` | General facts |
|
|
692
|
+
|
|
693
|
+
Clients (Claude Code / Codex) automatically fetch these resources at session start,
|
|
694
|
+
injecting memory into system context without any tool calls.
|
|
695
|
+
|
|
696
|
+
## Injection Protocol
|
|
697
|
+
|
|
698
|
+
### For Claude Code Users
|
|
699
|
+
|
|
700
|
+
Update your `CLAUDE.md` rules to use the new protocol:
|
|
701
|
+
|
|
702
|
+
```markdown
|
|
703
|
+
# 记忆系统使用规则
|
|
704
|
+
|
|
705
|
+
你有 mnemo 记忆工具(fact_store / fact_feedback),必须按以下规则使用:
|
|
706
|
+
|
|
707
|
+
## 规则 1:会话预热(自动)
|
|
708
|
+
会话启动时,mnemo-mcp 的 MCP Resource 会自动注入全局记忆到 system context。
|
|
709
|
+
你不需要主动调用 fact_store(search) 来获取高频记忆。
|
|
710
|
+
|
|
711
|
+
## 规则 2:按需补充查询
|
|
712
|
+
仅在以下情况调用 fact_store(action="search"):
|
|
713
|
+
- 用户消息涉及个人偏好/习惯/工具选择且预热中未覆盖
|
|
714
|
+
- 用户明确查询记忆("我之前说过什么""按我的习惯")
|
|
715
|
+
- 技术选型时需要确认用户偏好
|
|
716
|
+
|
|
717
|
+
不触发查询的情况:
|
|
718
|
+
- 纯操作指令("运行测试""git commit")
|
|
719
|
+
- 通用技术问题("Promise 怎么用")
|
|
720
|
+
- 代码审查/解释请求
|
|
721
|
+
|
|
722
|
+
## 规则 3:写入记忆
|
|
723
|
+
用户说"记住""记下来"时,调用 fact_store(action="add", content="...", category="...")。
|
|
724
|
+
- 先 search 检查是否已有相似事实,有则 update
|
|
725
|
+
- category:identity / coding_style / tool_pref / workflow / general
|
|
726
|
+
|
|
727
|
+
## 规则 4:反馈强化
|
|
728
|
+
成功使用某条记忆时,调用 fact_feedback(action="helpful", fact_id=...)。
|
|
729
|
+
```
|
|
730
|
+
```
|
|
731
|
+
|
|
732
|
+
- [ ] **Step 2: Commit docs**
|
|
733
|
+
|
|
734
|
+
```bash
|
|
735
|
+
cd /Users/ljn/Documents/demo/ocean/mnemo-mcp && git add README.md && git commit -m "docs: add MCP Resource and injection protocol documentation"
|
|
736
|
+
```
|
|
737
|
+
|
|
738
|
+
---
|
|
739
|
+
|
|
740
|
+
## Self-Review
|
|
741
|
+
|
|
742
|
+
### Spec Coverage
|
|
743
|
+
|
|
744
|
+
| Spec Requirement | Task |
|
|
745
|
+
|-----------------|------|
|
|
746
|
+
| MCP Resource 暴露 5 个 category | Task 2 |
|
|
747
|
+
| Resource 缓存 + 写操作失效 | Task 2 |
|
|
748
|
+
| 查询提炼过滤动作词/虚词 | Task 1 |
|
|
749
|
+
| 纯操作指令返回 null | Task 1 |
|
|
750
|
+
| 引号/书名号实体提取 | Task 1 |
|
|
751
|
+
| 动态 FTS/Jaccard 权重 | Task 3 |
|
|
752
|
+
| 相关性评分门控 (0.15) | Task 3 |
|
|
753
|
+
| 内容相似度去重 (Jaccard > 0.7) | Task 3 |
|
|
754
|
+
| 会话预热注入协议 | Task 4 (Resource wiring) + Task 5 (docs) |
|
|
755
|
+
| 按需补充查询规则 | Task 5 (CLAUDE.md template) |
|
|
756
|
+
|
|
757
|
+
### Placeholder Scan
|
|
758
|
+
|
|
759
|
+
- [x] No "TBD", "TODO", "implement later"
|
|
760
|
+
- [x] No vague "add error handling" without code
|
|
761
|
+
- [x] No "write tests for the above" without test code
|
|
762
|
+
- [x] All file paths are exact
|
|
763
|
+
- [x] All code blocks contain complete implementations
|
|
764
|
+
|
|
765
|
+
### Type Consistency
|
|
766
|
+
|
|
767
|
+
- [x] `refineQuery()` returns `RefineResult | null` consistently
|
|
768
|
+
- [x] `ResourceManager.getFacts()` returns `ResourceFact[]`
|
|
769
|
+
- [x] `search()` accepts `skipRefine?: boolean` in options
|
|
770
|
+
- [x] Dynamic weights use `ftsWeight`/`jaccardWeight` variables (same names as constructor params)
|