recallnest 1.4.1 → 2.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (233) hide show
  1. package/.env +4 -0
  2. package/CLAUDE.md +7 -3
  3. package/README.md +202 -130
  4. package/README_CN.md +263 -190
  5. package/bin/recallnest.mjs +62 -0
  6. package/config.json +55 -0
  7. package/daily-recall.sh +23 -0
  8. package/integrations/setup-all.sh +31 -0
  9. package/logs/daily-recall.error.log +0 -0
  10. package/logs/daily-recall.log +30 -0
  11. package/logs/ingest-2026-04-11.log +110 -0
  12. package/logs/ingest-2026-04-12.log +90 -0
  13. package/logs/launchagent-stderr.log +0 -0
  14. package/logs/launchagent-stdout.log +0 -0
  15. package/package.json +10 -3
  16. package/src/access-tracker.ts +3 -2
  17. package/src/activity-counter.ts +109 -0
  18. package/src/adaptive-retrieval.ts +6 -5
  19. package/src/admission-control.ts +163 -0
  20. package/src/anchor-generator.ts +80 -0
  21. package/src/api-server.ts +10 -0
  22. package/src/audit-log.ts +122 -0
  23. package/src/auto-gc.ts +98 -29
  24. package/src/auto-recall.ts +33 -4
  25. package/src/capture-engine.ts +372 -23
  26. package/src/cascade-forget.ts +3 -2
  27. package/src/cli.ts +107 -5
  28. package/src/compat.ts +22 -0
  29. package/src/confidence-tracker.ts +210 -47
  30. package/src/conflict-store.ts +12 -3
  31. package/src/consolidation-engine.ts +296 -8
  32. package/src/context-collapse-renderer.ts +222 -0
  33. package/src/context-composer-stable-selection.ts +5 -1
  34. package/src/context-composer.ts +239 -4
  35. package/src/context-reconstructor.ts +404 -0
  36. package/src/conversation-importer.ts +441 -0
  37. package/src/data-checkup.ts +232 -0
  38. package/src/decay-engine.ts +84 -0
  39. package/src/distill-lock.ts +122 -0
  40. package/src/doctor.ts +8 -10
  41. package/src/dream-pipeline.ts +227 -0
  42. package/src/emotion-detector.ts +90 -0
  43. package/src/eval.ts +3 -2
  44. package/src/forget-engine.ts +269 -0
  45. package/src/frequency-tracker.ts +175 -0
  46. package/src/graph-export.ts +644 -0
  47. package/src/ingest.ts +36 -6
  48. package/src/interference-detector.ts +187 -0
  49. package/src/kg-extractor.ts +15 -24
  50. package/src/kg-store.ts +2 -1
  51. package/src/language-hook.ts +67 -0
  52. package/src/llm-client.ts +289 -6
  53. package/src/mcp-server.ts +707 -80
  54. package/src/memory-assets.ts +22 -5
  55. package/src/memory-evolution.ts +365 -0
  56. package/src/memory-lint.ts +429 -0
  57. package/src/memory-output.ts +69 -0
  58. package/src/memory-schema.ts +74 -2
  59. package/src/narrative-schema.ts +79 -0
  60. package/src/narrative-tagger.ts +164 -0
  61. package/src/pii-detector.ts +98 -0
  62. package/src/post-retrieval-filter.ts +76 -0
  63. package/src/prediction-engine.ts +253 -0
  64. package/src/preference-slots.ts +65 -2
  65. package/src/prospective-memory.ts +215 -0
  66. package/src/query-expander.ts +67 -0
  67. package/src/recall-governor.ts +183 -0
  68. package/src/retention-policy.ts +90 -0
  69. package/src/retriever.ts +828 -55
  70. package/src/rif.ts +34 -8
  71. package/src/runtime-config.ts +23 -8
  72. package/src/scope-inventory.ts +4 -3
  73. package/src/scope-policy.ts +4 -0
  74. package/src/session-distiller.ts +307 -0
  75. package/src/session-output.ts +40 -0
  76. package/src/session-schema.ts +37 -0
  77. package/src/session-store.ts +7 -2
  78. package/src/skill-engine.ts +208 -0
  79. package/src/skill-promotion.ts +293 -0
  80. package/src/skill-schema.ts +39 -0
  81. package/src/store.ts +96 -10
  82. package/src/topic-tag.ts +66 -0
  83. package/src/tracker.ts +3 -1
  84. package/src/ui-server.ts +80 -8
  85. package/src/workflow-observation-scope-review.ts +2 -1
  86. package/src/workflow-observation-store.ts +2 -1
  87. package/src/write-verifier.ts +126 -0
  88. package/.claude-plugin/CLAUDE.md +0 -29
  89. package/.claude-plugin/marketplace.json +0 -18
  90. package/.github/workflows/ci.yml +0 -28
  91. package/.mcp.json +0 -9
  92. package/CHANGELOG.md +0 -51
  93. package/DEMO.md +0 -94
  94. package/EVAL.md +0 -89
  95. package/FAILURES.md +0 -75
  96. package/OPERATIONS.md +0 -59
  97. package/ROADMAP.md +0 -198
  98. package/assets/recallnest-workbench.png +0 -0
  99. package/assets/ui/app.js +0 -807
  100. package/assets/ui/index.html +0 -154
  101. package/assets/ui/styles.css +0 -605
  102. package/assets/wechat-qr.jpg +0 -0
  103. package/daily-reflections/2026-03-05.md +0 -15
  104. package/daily-reflections/2026-03-06.md +0 -15
  105. package/daily-reflections/2026-03-07.md +0 -7
  106. package/daily-reflections/2026-03-08.md +0 -34
  107. package/daily-reflections/2026-03-09.md +0 -35
  108. package/daily-reflections/2026-03-10.md +0 -36
  109. package/daily-reflections/2026-03-11.md +0 -35
  110. package/daily-reflections/2026-03-12.md +0 -36
  111. package/daily-reflections/2026-03-13.md +0 -35
  112. package/daily-reflections/2026-03-14.md +0 -35
  113. package/daily-reflections/2026-03-15.md +0 -35
  114. package/daily-reflections/2026-03-16.md +0 -41
  115. package/daily-reflections/2026-03-17.md +0 -10
  116. package/daily-reflections/2026-03-18.md +0 -10
  117. package/daily-reflections/2026-03-19.md +0 -10
  118. package/daily-reflections/2026-03-20.md +0 -10
  119. package/daily-reflections/2026-03-21.md +0 -10
  120. package/daily-reflections/2026-03-22.md +0 -10
  121. package/daily-reflections/2026-03-23.md +0 -10
  122. package/daily-reflections/2026-03-24.md +0 -10
  123. package/daily-reflections/2026-03-25.md +0 -32
  124. package/daily-reflections/2026-03-26.md +0 -37
  125. package/daily-reflections/2026-03-27.md +0 -35
  126. package/daily-reflections/2026-03-28.md +0 -35
  127. package/daily-reflections/2026-03-29.md +0 -35
  128. package/daily-reflections/2026-03-30.md +0 -36
  129. package/daily-reflections/2026-03-31.md +0 -35
  130. package/daily-reflections/2026-04-01.md +0 -35
  131. package/docs/api-reference.md +0 -1291
  132. package/docs/architecture.md +0 -168
  133. package/docs/memory-boundary-contract.md +0 -226
  134. package/docs/memory-categories.md +0 -119
  135. package/docs/pipeline-audit-ultramemory.md +0 -177
  136. package/docs/plans/2026-03-08-query-expansion.md +0 -270
  137. package/docs/plans/2026-03-16-continuity-layer.md +0 -196
  138. package/docs/plans/mcp-tiering-plan.md +0 -126
  139. package/docs/research/recall-quality-audit-2026-03-22.md +0 -153
  140. package/docs/self-evolution.md +0 -198
  141. package/docs/superpowers/plans/2026-03-19-ingest-pipeline-upgrade.md +0 -782
  142. package/docs/superpowers/specs/2026-03-19-ingest-pipeline-upgrade-design.md +0 -77
  143. package/docs/superpowers/specs/2026-03-23-pull-claude-desktop-design.md +0 -95
  144. package/eval/continuity/results-2026-03-16.md +0 -117
  145. package/eval/continuity/results-2026-03-19.md +0 -453
  146. package/eval/continuity/runbook.md +0 -110
  147. package/eval/continuity/seed-cases.md +0 -54
  148. package/eval/reports/2026-03-06-abstract-query-baseline.md +0 -96
  149. package/eval/reports/2026-03-06-aws-ssh-baseline.md +0 -82
  150. package/eval/reports/2026-03-06-baseline.md +0 -82
  151. package/eval/reports/query-expansion.md +0 -138
  152. package/lm +0 -19
  153. package/plan-v1.2-distributable.md +0 -105
  154. package/scripts/archive-distilled-junk.ts +0 -116
  155. package/scripts/backfill-l0l1l2.ts +0 -98
  156. package/scripts/blind-scan.ts +0 -588
  157. package/scripts/cleanup-step1-exact-dups.ts +0 -117
  158. package/scripts/cleanup-step2-template-dups.ts +0 -113
  159. package/scripts/cleanup-step3-stale-content.ts +0 -149
  160. package/scripts/cleanup-step4-orphan-scopes.ts +0 -131
  161. package/scripts/delete-distilled-junk.ts +0 -88
  162. package/scripts/distill-facts.ts +0 -305
  163. package/scripts/health-check.ts +0 -343
  164. package/scripts/incremental-ingest.sh +0 -71
  165. package/scripts/mlp-health-check.ts +0 -382
  166. package/scripts/pull-claude-desktop.ts +0 -515
  167. package/scripts/re-ingest-cc.sh +0 -66
  168. package/scripts/rebalance-memory-health.ts +0 -181
  169. package/scripts/run-desktop-ingest.sh +0 -11
  170. package/scripts/start-server.sh +0 -12
  171. package/scripts/viz-dashboard.html +0 -423
  172. package/scripts/viz-export.ts +0 -703
  173. package/src/__tests__/access-tracker-novelty.test.ts +0 -228
  174. package/src/__tests__/auto-consolidation.test.ts +0 -168
  175. package/src/__tests__/auto-recall.test.ts +0 -131
  176. package/src/__tests__/batch-dedup.test.ts +0 -65
  177. package/src/__tests__/capture-engine.test.ts +0 -1142
  178. package/src/__tests__/capture-heuristic.test.ts +0 -171
  179. package/src/__tests__/cascade-forget.test.ts +0 -210
  180. package/src/__tests__/category-min-scores.test.ts +0 -33
  181. package/src/__tests__/confidence-tracker.test.ts +0 -196
  182. package/src/__tests__/conflict-advisor.test.ts +0 -206
  183. package/src/__tests__/conflict-escalation.test.ts +0 -149
  184. package/src/__tests__/conflict-lifecycle.test.ts +0 -83
  185. package/src/__tests__/conflict-output.test.ts +0 -85
  186. package/src/__tests__/conflict-store.test.ts +0 -374
  187. package/src/__tests__/consolidation-engine.test.ts +0 -181
  188. package/src/__tests__/context-composer-pins.test.ts +0 -64
  189. package/src/__tests__/context-composer-task-selection.test.ts +0 -24
  190. package/src/__tests__/context-renderer.test.ts +0 -91
  191. package/src/__tests__/continuity-eval.test.ts +0 -275
  192. package/src/__tests__/continuity-schema.test.ts +0 -97
  193. package/src/__tests__/core-recall-eval.test.ts +0 -235
  194. package/src/__tests__/core-summary.test.ts +0 -109
  195. package/src/__tests__/dedup-actions.test.ts +0 -58
  196. package/src/__tests__/deterministic-id.test.ts +0 -39
  197. package/src/__tests__/doctor.test.ts +0 -303
  198. package/src/__tests__/drill-down.test.ts +0 -108
  199. package/src/__tests__/embedder.test.ts +0 -122
  200. package/src/__tests__/entity-resolver.test.ts +0 -58
  201. package/src/__tests__/event-segmenter.test.ts +0 -153
  202. package/src/__tests__/hotness-blend.test.ts +0 -76
  203. package/src/__tests__/ingest.test.ts +0 -360
  204. package/src/__tests__/kg-extractor.test.ts +0 -184
  205. package/src/__tests__/kg-store.test.ts +0 -198
  206. package/src/__tests__/llm-consolidation.test.ts +0 -226
  207. package/src/__tests__/memory-boundaries.test.ts +0 -237
  208. package/src/__tests__/memory-health-rebalance.test.ts +0 -144
  209. package/src/__tests__/memory-output.test.ts +0 -215
  210. package/src/__tests__/multi-vector.test.ts +0 -242
  211. package/src/__tests__/ppr-traversal.test.ts +0 -256
  212. package/src/__tests__/preference-guard-smoke.test.ts +0 -211
  213. package/src/__tests__/preference-matcher.test.ts +0 -230
  214. package/src/__tests__/preference-slots.test.ts +0 -106
  215. package/src/__tests__/prospective-memory.test.ts +0 -283
  216. package/src/__tests__/query-entity-detector.test.ts +0 -171
  217. package/src/__tests__/query-expander.test.ts +0 -34
  218. package/src/__tests__/recall-mode.test.ts +0 -64
  219. package/src/__tests__/result-synthesizer.test.ts +0 -198
  220. package/src/__tests__/resume-context.test.ts +0 -4743
  221. package/src/__tests__/retrieval-trace.test.ts +0 -213
  222. package/src/__tests__/retriever-boundary-weight.test.ts +0 -136
  223. package/src/__tests__/retroactive-boost.test.ts +0 -223
  224. package/src/__tests__/scope-inventory.test.ts +0 -200
  225. package/src/__tests__/scope-policy.test.ts +0 -69
  226. package/src/__tests__/session-checkpoint.test.ts +0 -259
  227. package/src/__tests__/session-gc.test.ts +0 -215
  228. package/src/__tests__/store-scope-audit.test.ts +0 -81
  229. package/src/__tests__/temporal-parser.test.ts +0 -241
  230. package/src/__tests__/term-registry.test.ts +0 -121
  231. package/src/__tests__/tool-output-compressor.test.ts +0 -133
  232. package/src/__tests__/version-manager.test.ts +0 -213
  233. package/src/__tests__/workflow-observation.test.ts +0 -400
package/.env ADDED
@@ -0,0 +1,4 @@
1
+ JINA_API_KEY=jina_a953576fa7b746019e57d1370122fb8aNQLVOjq95WHeDBKlBr3JeCj0K5SL
2
+ QWEN_API_KEY=sk-8b8f215de882422a991653d5fc3cc275
3
+ RECALLNEST_EMOTION_SCORING=true
4
+ RECALLNEST_CONSTRUCTIVE_RETRIEVAL=true
package/CLAUDE.md CHANGED
@@ -24,9 +24,9 @@
24
24
 
25
25
  ## 4. Git Push 规则(重要!)
26
26
 
27
- - **所有 push 只推 origin**(`trihippo/recallnest`)
27
+ - **所有 push 只推 origin**(`AliceLJY/recallnest`)
28
28
  - **绝对不要 push 到 upstream**(`CortexReach/memory-lancedb-pro`)—— 那是上游公开仓库,推了等于暴露私有改造
29
- - **绝对不要 push 到 public**(`AliceLJY/recallnest`)—— 除非用户明确要求
29
+ - `trihippo/recallnest` 已停止维护,不再推送
30
30
  - 默认 `git push` 即可(默认推 origin)
31
31
  - 需要给上游提 PR 时,走 fork + PR 流程,不直接 push
32
32
 
@@ -34,9 +34,13 @@
34
34
 
35
35
  - `RECALLNEST_MULTI_VECTOR=true` — 多向量 L0/L1/L2 检索
36
36
  - `RECALLNEST_KG_MODE=true` — KG 三元组提取 + 图遍历
37
+ - `RECALLNEST_EMOTION_SCORING=true` — Emotion detection + salience-weighted Weibull decay + arousal boost + retrieval scoring
38
+ - `RECALLNEST_CONSTRUCTIVE_RETRIEVAL=true` — Multi-source candidate expansion + source-map grounded reconstruction (resume default, search opt-in)
39
+ - `RECALLNEST_NARRATIVE_MODE=true` — Autobiographical narrative metadata layer (life-period / general-event / specific-event)
40
+ - `RECALLNEST_PREDICTIVE_MEMORY=true` — Heuristic-predicted prospective reminders (zero LLM, behavioral signals)
37
41
 
38
42
  ## 6. 测试基线
39
43
 
40
44
  - 改完代码必须跑 `bun test`,全量通过才能 commit
41
- - 当前基线:643 tests / 0 fail
45
+ - 当前基线:1428 tests / 0 fail
42
46
  - 新增功能必须配套测试,基线只能涨不能降
package/README.md CHANGED
@@ -12,7 +12,8 @@ A local-first memory system backed by LanceDB that turns scattered conversation
12
12
  [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
13
13
  [![Bun](https://img.shields.io/badge/Runtime-Bun-f9f1e1?logo=bun)](https://bun.sh)
14
14
  [![LanceDB](https://img.shields.io/badge/LanceDB-Vector+FTS-orange)](https://lancedb.com)
15
- [![MCP](https://img.shields.io/badge/MCP-29_tools-blue)](https://modelcontextprotocol.io)
15
+ [![MCP](https://img.shields.io/badge/MCP-41_tools-blue)](https://modelcontextprotocol.io)
16
+ [![Tests](https://img.shields.io/badge/Tests-1428_pass-brightgreen)](https://github.com/AliceLJY/recallnest)
16
17
  [![CC Plugin](https://img.shields.io/badge/Claude_Code-Plugin-blueviolet)](https://github.com/AliceLJY/recallnest)
17
18
 
18
19
  **English** | [简体中文](README_CN.md) | [Roadmap](ROADMAP.md)
@@ -23,52 +24,22 @@ A local-first memory system backed by LanceDB that turns scattered conversation
23
24
 
24
25
  ## Why RecallNest?
25
26
 
26
- Most coding agents forget everything when you open a new window. Worse — your history is scattered across three different terminals with no shared memory.
27
+ Coding agents forget everything between windows. Your context — project configs, debugging decisions, entity mappings — is scattered across Claude Code, Codex, and Gemini CLI with no shared memory.
27
28
 
28
- ### Without RecallNest — every window starts from zero:
29
+ RecallNest solves this: **a single LanceDB-backed memory layer that all three terminals read and write**. Context stored in one window is auto-recalled in another. Sessions checkpoint on exit and resume on start. Memory decays, evolves, and self-organizes — not just raw log storage.
29
30
 
30
- > **You (Claude Code):** "The Docker config lives at `/opt/app/config.json`, use port 4318."
31
- >
32
- > *(switch to Codex)*
33
- >
34
- > **You:** "The config is at... wait, let me find it again." 😤
35
- >
36
- > *(next day, new Claude Code window)*
37
- >
38
- > **You:** "We already fixed this exact bug last week! The solution was..."
39
- >
40
- > **Agent:** "I don't have context about previous sessions." 🤷
31
+ ### Benchmark: LongMemEval (ICLR 2025)
41
32
 
42
- ### With RecallNest — context carries over:
33
+ Evaluated on 500 questions across 6 memory abilities ([methodology](https://arxiv.org/abs/2407.15168)):
43
34
 
44
- > **You (Claude Code):** "The Docker config lives at `/opt/app/config.json`, use port 4318."
45
- >
46
- > *(switch to Codex — same memory layer)*
47
- >
48
- > **Agent:** *(auto-recalls project entities)* "Using config at `/opt/app/config.json`, port 4318." ✅
49
- >
50
- > *(next day, new window)*
51
- >
52
- > **Agent:** *(resume_context fires)* "Continuing from yesterday — the Docker port conflict was resolved by..." ✅
35
+ | | RecallNest | Vector-only baseline | Delta |
36
+ |---|---|---|---|
37
+ | Overall Accuracy | **29.6%** | 24.2% | **+5.4pp** |
38
+ | User Facts | **64.3%** | 52.9% | +11.4pp |
39
+ | Knowledge Update | **43.6%** | 42.3% | +1.3pp |
40
+ | Abstention Rate | **55.6%** | 67.8% | **-12.2pp** |
53
41
 
54
- That's the difference: **one memory shared across terminals**, with context that survives window boundaries.
55
-
56
- ### What you get
57
-
58
- | | Capability |
59
- |---|---|
60
- | **CC Plugin** | Install in Claude Code with one command — no manual config |
61
- | **Shared Index** | One LanceDB store for Claude Code, Codex, and Gemini CLI |
62
- | **Dual Interface** | MCP (stdio) for CLI tools + HTTP API for custom agents |
63
- | **One-Click Setup** | Integration scripts install MCP access and continuity rules |
64
- | **Hybrid Retrieval** | Vector + BM25 + reranking + Weibull decay + tier promotion |
65
- | **Session Continuity** | `checkpoint_session` + `resume_context` for cross-window recovery |
66
- | **Workflow Observation** | Dedicated append-only workflow health records, outside regular memory |
67
- | **Structured Assets** | Pins, briefs, and distilled summaries — not just raw logs |
68
- | **Smart Promotion** | Evidence → durable memory with conflict guards and merge resolution |
69
- | **6 Categories** | profile, preferences, entities, events, cases, patterns |
70
- | **4 Retrieval Profiles** | default, writing, debug, fact-check — tuned for different tasks |
71
- | **Multi-Source Ingest** | Import existing transcripts from all three terminals |
42
+ Wins or ties in **all 6 categories**, with no regression. The hybrid retrieval pipeline (BM25 + vector + recency + RIF dedup) surfaces 12.2% more relevant context than vector-only search.
72
43
 
73
44
  ---
74
45
 
@@ -140,6 +111,117 @@ bun run src/cli.ts doctor
140
111
 
141
112
  ---
142
113
 
114
+ ## Web UI
115
+
116
+ <p align="center">
117
+ <img src="assets/dashboard.png" alt="RecallNest Dashboard" width="800" />
118
+ <br><em>Dashboard — total count, category distribution, health score, and growth trends at a glance.</em>
119
+ </p>
120
+
121
+ <p align="center">
122
+ <img src="assets/screenshots/ui-full.png" alt="RecallNest Search Workbench" width="800" />
123
+ <br><em>Search Workbench — hybrid search with topic tag filtering, 4 retrieval profiles, Skills browser, and asset management.</em>
124
+ </p>
125
+
126
+ <p align="center">
127
+ <img src="assets/knowledge-graph.png" alt="RecallNest Knowledge Graph" width="800" />
128
+ <br><em>Knowledge Graph — interactive force-directed visualization with semantic bridges revealing cross-domain connections.</em>
129
+ </p>
130
+
131
+ ```bash
132
+ bun run src/ui-server.ts
133
+ # → http://localhost:4317
134
+ ```
135
+
136
+ ---
137
+
138
+ ## Core Capabilities
139
+
140
+ ### Access & Setup
141
+
142
+ | Capability | Description |
143
+ |---|---|
144
+ | **CC Plugin** | Install in Claude Code with one command — no manual config |
145
+ | **Shared Index** | One LanceDB store for Claude Code, Codex, and Gemini CLI |
146
+ | **Dual Interface** | MCP (stdio) for CLI tools + HTTP API for custom agents |
147
+ | **One-Click Setup** | Integration scripts install MCP access and continuity rules |
148
+
149
+ ### Recall & Continuity
150
+
151
+ | Capability | Description |
152
+ |---|---|
153
+ | **Hybrid Retrieval** | 6-channel: vector + BM25 + L0/L1/L2 multi-vector + KG graph (PPR) |
154
+ | **4 Retrieval Profiles** | default, writing, debug, fact-check — tuned for different tasks |
155
+ | **Session Continuity** | `checkpoint_session` + `resume_context` (full/light/summary modes) with repo-state guard |
156
+ | **Session Distiller** | 3-layer conversation compression: microcompact → LLM summary → knowledge extraction |
157
+ | **Conversation Import** | Import from Claude Code, Claude.ai, ChatGPT, Slack, and plaintext |
158
+ | **Topic Tags** | Intra-scope topic partitioning — auto-detected, filterable in search |
159
+
160
+ ### Memory Lifecycle & Governance
161
+
162
+ | Capability | Description |
163
+ |---|---|
164
+ | **Memory Evolution** | Supersede chains, decay scoring, LLM importance, consolidation, archival |
165
+ | **Smart Promotion** | Evidence → durable memory with conflict guards, merge resolution, and audit trail |
166
+ | **Privacy Tiers** | 4-tier (`ephemeral` / `private` / `durable` / `shared`) with cascade forgetting |
167
+ | **Admission Control** | Write-time gating: noise filter, importance floor, dedup, rate limiting |
168
+ | **Memory Lint** | Contradiction, duplicate, stale, and orphan detection with health score |
169
+ | **Offline Consolidation** | `dream` command: clustering, merging, pruning of accumulated memories |
170
+
171
+ ### Reasoning & Structure
172
+
173
+ | Capability | Description |
174
+ |---|---|
175
+ | **Knowledge Graph** | Entity relation graph with PPR algorithm for multi-hop questions |
176
+ | **Constructive Retrieval** | Multi-source candidate expansion + grounded context reconstruction |
177
+ | **Narrative Architecture** | 3-layer autobiographical metadata (life-period → general-event → specific-event) |
178
+ | **Skill Memory** | Store, retrieve, and promote executable skills from recurring patterns |
179
+ | **Predictive Reminders** | Behavioral-signal prediction engine surfaces "you might need this" suggestions |
180
+ | **6 Categories** | profile, preferences, entities, events, cases, patterns — with category-aware merge strategies |
181
+
182
+ ### Visibility & Operations
183
+
184
+ | Capability | Description |
185
+ |---|---|
186
+ | **Dashboard** | Web UI with stats, category distribution, growth trends, and health |
187
+ | **Workflow Observation** | Dedicated append-only workflow health records, outside regular memory |
188
+ | **Structured Assets** | Pins, briefs, and distilled summaries — not just raw logs |
189
+ | **Data Checkup** | Data quality health checks on the memory store |
190
+ | **Export Graph** | Export interactive HTML knowledge graph visualization |
191
+ | **Batch Operations** | Store up to 20 memories in a single call with dedup |
192
+
193
+ ---
194
+
195
+ ## New in v2.1: Philosophy-Informed Memory
196
+
197
+ v2.0 built the operational memory platform; v2.1 added philosophy-informed memory behavior.
198
+
199
+ Five upgrades derived from 9 research dimensions in philosophy of memory, each mapped to concrete engineering:
200
+
201
+ - **Emotion-Aware Decay** *(Affective Memory Theory)* — Memories with strong emotional content decay 20-30% slower. Keyword-based emotion detection computes `salience` (mnemonic significance), which feeds into the Weibull half-life formula and a rebalanced 4-factor evolution score. Zero LLM cost.
202
+
203
+ - **Memory Ethics Layer** *(Right to Be Forgotten / GDPR Art. 17)* — Four privacy tiers (`ephemeral` / `private` / `durable` / `shared`). Cascade forgetting engine that propagates deletion through KG triples, evolution chains, pin assets, and briefs. Full audit trail. `forget_memory` MCP tool for agent-driven deletion.
204
+
205
+ - **Autobiographical Narrative** *(Narrative Identity Theory / Conway's 3-layer model)* — Memories are tagged with `lifePeriod → generalEvent → specificEvent` hierarchy, orthogonal to existing 6 categories. Retrieval pulls narrative siblings. Context rendering groups by life period. Rule-based tagger with EN+CN support.
206
+
207
+ - **Constructive Retrieval** *(Simulation Theory / Michaelian)* — Instead of returning raw stored text, RecallNest now reconstructs context from an expanded candidate set: KG neighbors + evolution chains + cluster members + narrative siblings. Source-map grounded coverage replaces lexical overlap. Contradictions are detected and flagged.
208
+
209
+ - **Predictive Prospective Memory** *(Mental Time Travel / Tulving)* — Heuristic prediction engine that surfaces "you might need this" reminders from behavioral signals: stale checkpoint open loops, corrected workflow observations, high-frequency dormant memories, and uncovered query topics. Zero LLM cost. Auto-expire in 7 days if unaccepted.
210
+
211
+ ---
212
+
213
+ ## New in v2.2: Retrieval Quality Hardening
214
+
215
+ v2.1 added philosophy-informed behavior; v2.2 closes the last three engine-layer gaps identified by a frontier research scan (ACC, PI-LLM, TSM).
216
+
217
+ - **Memory Confidence Meta-tags** *(ACC / Dual-Process UQ)* — Each memory now carries structured `ConfidenceMetadata` (score, reliability tier: `direct` / `inferred` / `hearsay`). Auto-assigned from source on write (`manual` = 0.9, `agent` = 0.7, `conversation_import` = 0.5). Retrieval scores are weighted by confidence. `resume_context` tags low-confidence items with `[低置信]`.
218
+
219
+ - **Interference Detection + Active Forgetting Gate** *(PI-LLM / SleepGate)* — Semantic cluster detection identifies groups of near-duplicate memories competing for retrieval. Enhanced RIF keeps only top-K (default 3) per cluster; extras are demoted 50% instead of removed. Write-time pre-warning: when a scope accumulates ≥5 high-similarity active memories, the weakest is flagged `pending_review`. `data_checkup` reports interference density.
220
+
221
+ - **Temporal Validity Windows** *(TSM / TiMem / Zep)* — `store_memory` accepts `validUntil` (expiration) and `eventTime` (when the event actually happened). `search_memory` supports `validAt` (point-in-time query) and `includeExpired` (demote 80% instead of hide). Auto-GC applies 2× decay acceleration to expired memories.
222
+
223
+ ---
224
+
143
225
  ## Architecture
144
226
 
145
227
  ```
@@ -157,7 +239,7 @@ bun run src/cli.ts doctor
157
239
  │ Integration Layer │
158
240
  │ ┌─────────────────────┐ ┌────────────────────────────┐ │
159
241
  │ │ MCP Server │ │ HTTP API Server │ │
160
- │ │ 29 tools │ │ 19 endpoints │ │
242
+ │ │ 41 tools │ │ 21 endpoints │ │
161
243
  │ └─────────┬───────────┘ └──────────┬─────────────────┘ │
162
244
  └────────────┼─────────────────────────┼───────────────────┘
163
245
  └──────────┬──────────────┘
@@ -187,11 +269,19 @@ bun run src/cli.ts doctor
187
269
  └──────────────────────────────────────────────────────────┘
188
270
  ```
189
271
 
272
+ ### Internal Design
273
+
274
+ - **L0 / L1 / L2 Dynamic Folding** — every memory stores 3 granularity layers (one-liner / bullet summary / full content); retrieval dynamically selects which layer to return based on relevance score and token budget
275
+ - **Weibull Decay + Emotion Modulation** — memories decay along a parametric Weibull curve; importance scores modulate the half-life, and emotional salience extends it further (up to 30%)
276
+ - **Vector Pre-filter + LLM Dedup** — 90% of dedup decisions use cheap cosine similarity (>= 0.92); only borderline cases invoke LLM judgment, keeping costs low without sacrificing accuracy
277
+ - **Category-Aware Merge Strategies** — `profile` and `preferences` use merge-on-conflict (latest wins); `events` and `cases` use append-only (history preserved)
278
+ - **Display Score vs Elimination Score** — dual-track retrieval: tier floor prevents core memories from ever dropping out, while decay boost lets fresh memories surface temporarily without permanently displacing stable ones
279
+
190
280
  > Full architecture deep-dive: [`docs/architecture.md`](docs/architecture.md)
191
281
 
192
282
  ---
193
283
 
194
- ## Integrations
284
+ ## Interfaces
195
285
 
196
286
  RecallNest serves two interfaces:
197
287
 
@@ -210,77 +300,8 @@ Examples live in [`integrations/examples/`](integrations/examples/):
210
300
 
211
301
  ---
212
302
 
213
- ## Core Features
214
-
215
- ### Hybrid Retrieval
216
-
217
- ```
218
- Query → Embedding ──┐
219
- ├── Hybrid Fusion → Rerank → Weibull Decay → Filter → Top-K
220
- Query → BM25 FTS ──┘
221
- ```
222
-
223
- - **Vector search** — semantic similarity via LanceDB ANN
224
- - **BM25 full-text search** — exact keyword matching via LanceDB FTS
225
- - **Hybrid fusion** — vector + BM25 combined scoring
226
- - **Reranking** — Jina cross-encoder reranking
227
- - **Decay + tiering** — Weibull freshness model with Core / Working / Peripheral tiers
228
-
229
- ### Session Continuity
230
-
231
- The killer feature for multi-window workflows:
232
-
233
- - **`checkpoint_session`** — snapshot current work state (decisions, open loops, next actions)
234
- - **Repo-state guard** — saved checkpoints strip `git status` / modified-file text so volatile repo state does not contaminate later handoffs
235
- - **`resume_context`** — compose startup context from checkpoints + durable memory + pins
236
- - **Managed rules** — integration scripts install continuity rules so `resume_context` fires automatically
237
-
238
- ### Workflow Observation
239
-
240
- RecallNest now keeps workflow observations in a dedicated append-only store instead of stuffing them into the regular memory index:
241
-
242
- - **`workflow_observe`** — record whether `resume_context`, `checkpoint_session`, or another workflow primitive succeeded, failed, was corrected, or was missed
243
- - **`workflow_health`** — aggregate 7d / 30d health for one workflow or show a degraded-workflow dashboard
244
- - **`workflow_evidence`** — package recent issue observations, top signals, and suggested next actions for debugging
245
-
246
- These records live under `data/workflow-observations`, not in the 6 memory categories, and they are never composed into `resume_context` as stable recall.
247
- Managed MCP / HTTP continuity calls now append observations automatically for `resume_context` and `checkpoint_session`, while repo-state sanitization is recorded as a `corrected` checkpoint observation instead of polluting durable memory.
248
-
249
- ### Memory Promotion & Conflict Resolution
250
-
251
- Raw transcripts don't silently become long-term memory:
252
-
253
- - **Evidence → durable** — explicit `promote_memory` with `canonicalKey` and provenance
254
- - **Conflict guards** — canonical-key collisions surface as conflict candidates
255
- - **Resolution** — keep existing, accept new, or merge — with advice and cluster views
256
- - **Audit + escalation** — `conflicts audit --export` for operational review
257
-
258
- ### Retrieval Profiles
259
-
260
- | Profile | Best for | Bias |
261
- |---------|----------|------|
262
- | `default` | Everyday recall | Balanced |
263
- | `writing` | Drafting and idea mining | Broader semantic, older material OK |
264
- | `debug` | Errors, commands, fixes | Keyword-heavy, recency-biased |
265
- | `fact-check` | Evidence lookup | Tighter cutoff, exact-match bias |
266
-
267
- ### Memory Categories
268
-
269
- | Category | Description | Strategy |
270
- |----------|-------------|----------|
271
- | `profile` | User identity and background | Merge |
272
- | `preferences` | Habits, style, dislikes | Merge |
273
- | `entities` | Projects, tools, people | Merge |
274
- | `events` | Things that happened | Append |
275
- | `cases` | Problem → solution pairs | Append |
276
- | `patterns` | Reusable workflows | Merge |
277
-
278
- Details: [`docs/memory-categories.md`](docs/memory-categories.md)
279
-
280
- ---
281
-
282
303
  <details>
283
- <summary><strong>MCP Tools (29 tools)</strong></summary>
304
+ <summary><strong>MCP Tools (41 tools)</strong></summary>
284
305
 
285
306
  | Tool | Description |
286
307
  |------|-------------|
@@ -313,11 +334,23 @@ Details: [`docs/memory-categories.md`](docs/memory-categories.md)
313
334
  | `auto_capture` | Heuristically extract and store memory signals from text (zero LLM calls) |
314
335
  | `set_reminder` | Set a prospective memory reminder to surface in a future session |
315
336
  | `consolidate_memories` | Cluster near-duplicate memories and merge them (dry-run by default) |
337
+ | `store_skill` | Store an executable skill with trigger conditions and verification |
338
+ | `retrieve_skill` | Retrieve matching executable skills by semantic similarity |
339
+ | `scan_skill_promotions` | Scan cases/patterns for promotion candidates to skills |
340
+ | `list_tools` | Discover available tools by tier (core/advanced/full) |
341
+ | `batch_store` | Store up to 20 memories in a single call with dedup |
342
+ | `distill_session` | Distill a conversation into structured knowledge via 3-layer pipeline |
343
+ | `import_conversations` | Import conversations from Claude Code, ChatGPT, Slack, and more |
344
+ | `data_checkup` | Run data quality health checks on the memory store |
345
+ | `dream` | Run offline memory consolidation (clustering, merging, pruning) |
346
+ | `memory_lint` | Run memory quality checks: contradictions, duplicates, stale entries, orphans |
347
+ | `forget_memory` | Cascade-delete a memory with KG cleanup, pin archival, and audit trail |
348
+ | `export_graph` | Export memories as an interactive HTML knowledge graph |
316
349
 
317
350
  </details>
318
351
 
319
352
  <details>
320
- <summary><strong>HTTP API (19 endpoints)</strong></summary>
353
+ <summary><strong>HTTP API (21 endpoints)</strong></summary>
321
354
 
322
355
  Base URL: `http://localhost:4318`
323
356
 
@@ -341,6 +374,7 @@ Base URL: `http://localhost:4318`
341
374
  | `/v1/resume` | POST | Compose startup context for a fresh window |
342
375
  | `/v1/search` | POST | Advanced search with full metadata |
343
376
  | `/v1/stats` | GET | Memory statistics |
377
+ | `/v1/lint` | GET | Memory quality lint report |
344
378
  | `/v1/health` | GET | Health check |
345
379
 
346
380
  Full documentation: [`docs/api-reference.md`](docs/api-reference.md)
@@ -374,6 +408,12 @@ bun run src/cli.ts conflicts resolve af70545a --keep-existing
374
408
  bun run src/cli.ts conflicts resolve af70545a --merge
375
409
  bun run src/cli.ts conflicts resolve --all --keep-existing --status open
376
410
 
411
+ # Memory health & visualization
412
+ bun run src/cli.ts lint # memory quality report
413
+ bun run src/cli.ts lint --scope project:myapp # lint a specific scope
414
+ bun run src/cli.ts graph --open # export & open knowledge graph
415
+ bun run src/cli.ts graph --max-nodes 50 # smaller graph
416
+
377
417
  # Ingestion & diagnostics
378
418
  bun run src/cli.ts ingest --source all
379
419
  bun run src/cli.ts doctor
@@ -381,17 +421,36 @@ bun run src/cli.ts doctor
381
421
 
382
422
  </details>
383
423
 
384
- <details>
385
- <summary><strong>Web UI (debugging)</strong></summary>
424
+ ---
425
+
426
+ ## Multilingual Support
427
+
428
+ RecallNest works out of the box with English. For multilingual memory (Chinese, Japanese, Thai, and 20+ more), install [babel-memory](https://github.com/AliceLJY/babel-memory) with the language packs you need:
386
429
 
387
430
  ```bash
388
- bun run src/ui-server.ts
389
- # → http://localhost:4317
431
+ # Chinese
432
+ npm install babel-memory jieba-wasm
433
+
434
+ # Japanese
435
+ npm install babel-memory @sglkc/kuromoji
436
+
437
+ # Thai
438
+ npm install babel-memory wordcut
439
+
440
+ # European languages (German, French, Spanish, Russian, etc.)
441
+ npm install babel-memory snowball-stemmers
442
+
443
+ # Multiple languages at once
444
+ npm install babel-memory jieba-wasm @sglkc/kuromoji snowball-stemmers
390
445
  ```
391
446
 
392
- The Web UI is for debugging and exploration, not the primary production interface.
447
+ RecallNest auto-detects babel-memory at startup — no configuration needed. Without babel-memory, RecallNest still works perfectly with standard BM25 text search.
393
448
 
394
- </details>
449
+ ---
450
+
451
+ ## Project Status & Roadmap
452
+
453
+ RecallNest is actively maintained. All major architecture phases are complete — see the full [Roadmap](ROADMAP.md) for current priorities and future plans.
395
454
 
396
455
  ---
397
456
 
@@ -406,21 +465,34 @@ RecallNest started as a fork of [memory-lancedb-pro](https://github.com/CortexRe
406
465
 
407
466
  | Source | Contribution |
408
467
  |--------|-------------|
409
- | [claude-memory-pro](https://github.com/CortexReach/claude-memory-pro) by [@win4r](https://github.com/win4r) | Retrieval core ideas and implementation base |
468
+ | [memory-lancedb-pro](https://github.com/CortexReach/memory-lancedb-pro) by [@win4r](https://github.com/win4r) | Fork base — hybrid retrieval, decay modeling, and memory architecture |
410
469
  | Claude Code | Foundation and early project scaffolding |
411
470
  | OpenAI Codex | Productization and MCP expansion |
412
471
 
413
472
  Special thanks to Qin Chao ([@win4r](https://github.com/win4r)) and the [CortexReach](https://github.com/CortexReach) team for the foundational work.
414
473
 
415
- ## Star History
474
+ <details>
475
+ <summary><strong>Ecosystem</strong></summary>
476
+
477
+ Part of the **小试AI** open-source AI workflow:
478
+
479
+ | Project | Description |
480
+ |---------|-------------|
481
+ | [babel-memory](https://github.com/AliceLJY/babel-memory) | Multilingual preprocessing for BM25 — 27+ languages, zero deps |
482
+ | [content-alchemy](https://github.com/AliceLJY/content-alchemy) | 5-stage AI writing pipeline |
483
+ | [content-publisher](https://github.com/AliceLJY/content-publisher) | Image generation + layout + WeChat publishing |
484
+ | [wechat-ai-bridge](https://github.com/AliceLJY/wechat-ai-bridge) | Run Claude Code / Codex / Gemini in WeChat with session management |
485
+ | [telegram-ai-bridge](https://github.com/AliceLJY/telegram-ai-bridge) | Telegram bots for Claude, Codex, and Gemini |
486
+ | [telegram-cli-bridge](https://github.com/AliceLJY/telegram-cli-bridge) | Telegram CLI bridge for Gemini CLI |
487
+ | [openclaw-tunnel](https://github.com/AliceLJY/openclaw-tunnel) | Docker ↔ host CLI bridge (/cc /codex /gemini) |
488
+ | [openclaw-config](https://github.com/AliceLJY/openclaw-config) | OpenClaw bots configuration and memory backup |
489
+ | [digital-clone-skill](https://github.com/AliceLJY/digital-clone-skill) | Build digital clones from corpus data |
490
+ | [claude-code-studio](https://github.com/AliceLJY/claude-code-studio) | Multi-session collaboration platform for Claude Code |
491
+ | [cc-genius](https://github.com/AliceLJY/cc-genius) | Web-based Claude chat client (PWA) — self-hosted, iPad-ready |
492
+ | [agent-nexus](https://github.com/AliceLJY/agent-nexus) | One-command installer for memory + remote control |
493
+ | [cc-cabin](https://github.com/AliceLJY/cc-cabin) | Complete Claude Code workflow scaffold |
416
494
 
417
- <a href="https://star-history.com/#AliceLJY/recallnest&Date">
418
- <picture>
419
- <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=AliceLJY/recallnest&type=Date&theme=dark&transparent=true" />
420
- <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=AliceLJY/recallnest&type=Date&transparent=true" />
421
- <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=AliceLJY/recallnest&type=Date&transparent=true" />
422
- </picture>
423
- </a>
495
+ </details>
424
496
 
425
497
  ## License
426
498