recallnest 2.2.0 → 2.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (249) hide show
  1. package/.env +4 -0
  2. package/CLAUDE.md +1 -1
  3. package/README.md +30 -5
  4. package/README_CN.md +30 -5
  5. package/config.json +55 -0
  6. package/connectors/examples/email-adapter.ts +73 -0
  7. package/connectors/examples/logs-adapter.ts +103 -0
  8. package/connectors/examples/rss-adapter.ts +95 -0
  9. package/logs/daily-recall.error.log +0 -0
  10. package/logs/daily-recall.log +60 -0
  11. package/logs/ingest-2026-04-11.log +110 -0
  12. package/logs/ingest-2026-04-12.log +90 -0
  13. package/logs/ingest-2026-04-13.log +87 -0
  14. package/logs/launchagent-stderr.log +0 -0
  15. package/logs/launchagent-stdout.log +0 -0
  16. package/package.json +1 -1
  17. package/src/cli.ts +30 -0
  18. package/src/connector-types.ts +57 -0
  19. package/src/conversation-importer.ts +25 -1
  20. package/src/data-checkup.ts +31 -1
  21. package/src/doctor.ts +29 -0
  22. package/src/ingest.ts +203 -0
  23. package/src/mcp-server.ts +2 -2
  24. package/src/obsidian-connector.ts +282 -0
  25. package/src/session-schema.ts +2 -2
  26. package/src/source-heartbeat.ts +108 -0
  27. package/.claude-plugin/CLAUDE.md +0 -29
  28. package/.claude-plugin/marketplace.json +0 -18
  29. package/.dockerignore +0 -11
  30. package/.github/workflows/ci.yml +0 -28
  31. package/.mcp.json +0 -9
  32. package/CHANGELOG.md +0 -51
  33. package/DEMO.md +0 -94
  34. package/Dockerfile +0 -20
  35. package/EVAL.md +0 -89
  36. package/FAILURES.md +0 -75
  37. package/OPERATIONS.md +0 -59
  38. package/ROADMAP.md +0 -245
  39. package/SKILL.md +0 -250
  40. package/assets/dashboard.png +0 -0
  41. package/assets/knowledge-graph.png +0 -0
  42. package/assets/recallnest-workbench.png +0 -0
  43. package/assets/screenshots/ui-full.png +0 -0
  44. package/assets/screenshots/ui-overview.png +0 -0
  45. package/assets/screenshots/ui-search.png +0 -0
  46. package/assets/ui/app.js +0 -1031
  47. package/assets/ui/index.html +0 -224
  48. package/assets/ui/styles.css +0 -781
  49. package/assets/wechat-qr.jpg +0 -0
  50. package/bun.lock +0 -337
  51. package/daily-reflections/2026-03-05.md +0 -15
  52. package/daily-reflections/2026-03-06.md +0 -15
  53. package/daily-reflections/2026-03-07.md +0 -7
  54. package/daily-reflections/2026-03-08.md +0 -34
  55. package/daily-reflections/2026-03-09.md +0 -35
  56. package/daily-reflections/2026-03-10.md +0 -36
  57. package/daily-reflections/2026-03-11.md +0 -35
  58. package/daily-reflections/2026-03-12.md +0 -36
  59. package/daily-reflections/2026-03-13.md +0 -35
  60. package/daily-reflections/2026-03-14.md +0 -35
  61. package/daily-reflections/2026-03-15.md +0 -35
  62. package/daily-reflections/2026-03-16.md +0 -41
  63. package/daily-reflections/2026-03-17.md +0 -10
  64. package/daily-reflections/2026-03-18.md +0 -10
  65. package/daily-reflections/2026-03-19.md +0 -10
  66. package/daily-reflections/2026-03-20.md +0 -10
  67. package/daily-reflections/2026-03-21.md +0 -10
  68. package/daily-reflections/2026-03-22.md +0 -10
  69. package/daily-reflections/2026-03-23.md +0 -10
  70. package/daily-reflections/2026-03-24.md +0 -10
  71. package/daily-reflections/2026-03-25.md +0 -32
  72. package/daily-reflections/2026-03-26.md +0 -37
  73. package/daily-reflections/2026-03-27.md +0 -35
  74. package/daily-reflections/2026-03-28.md +0 -35
  75. package/daily-reflections/2026-03-29.md +0 -35
  76. package/daily-reflections/2026-03-30.md +0 -36
  77. package/daily-reflections/2026-03-31.md +0 -35
  78. package/daily-reflections/2026-04-01.md +0 -35
  79. package/daily-reflections/2026-04-02.md +0 -35
  80. package/daily-reflections/2026-04-03.md +0 -37
  81. package/daily-reflections/2026-04-04.md +0 -37
  82. package/daily-reflections/2026-04-05.md +0 -35
  83. package/daily-reflections/2026-04-06.md +0 -36
  84. package/daily-reflections/2026-04-07.md +0 -35
  85. package/daily-reflections/2026-04-08.md +0 -35
  86. package/daily-reflections/2026-04-12.md +0 -35
  87. package/data/alias-map.example.json +0 -22
  88. package/docs/api-reference.md +0 -1291
  89. package/docs/architecture.md +0 -168
  90. package/docs/memory-boundary-contract.md +0 -226
  91. package/docs/memory-categories.md +0 -119
  92. package/docs/pipeline-audit-ultramemory.md +0 -177
  93. package/docs/plans/2026-03-08-query-expansion.md +0 -270
  94. package/docs/plans/2026-03-16-continuity-layer.md +0 -196
  95. package/docs/plans/2026-04-01-recall-boost.md +0 -100
  96. package/docs/plans/mcp-tiering-plan.md +0 -126
  97. package/docs/research/recall-quality-audit-2026-03-22.md +0 -153
  98. package/docs/self-evolution.md +0 -198
  99. package/docs/superpowers/plans/2026-03-19-ingest-pipeline-upgrade.md +0 -782
  100. package/docs/superpowers/plans/2026-04-11-constructive-retrieval-emotion.md +0 -1195
  101. package/docs/superpowers/specs/2026-03-19-ingest-pipeline-upgrade-design.md +0 -77
  102. package/docs/superpowers/specs/2026-03-23-pull-claude-desktop-design.md +0 -95
  103. package/docs/superpowers/specs/2026-04-11-constructive-retrieval-emotion-design.md +0 -398
  104. package/eval/continuity/results-2026-03-16.md +0 -117
  105. package/eval/continuity/results-2026-03-19.md +0 -453
  106. package/eval/continuity/runbook.md +0 -110
  107. package/eval/continuity/seed-cases.md +0 -54
  108. package/eval/reports/2026-03-06-abstract-query-baseline.md +0 -96
  109. package/eval/reports/2026-03-06-aws-ssh-baseline.md +0 -82
  110. package/eval/reports/2026-03-06-baseline.md +0 -82
  111. package/eval/reports/query-expansion.md +0 -138
  112. package/glama.json +0 -6
  113. package/lm +0 -19
  114. package/plan-philosophy-upgrades.md +0 -363
  115. package/plan-product-upgrades.md +0 -155
  116. package/plan-v1.2-distributable.md +0 -105
  117. package/plan.md +0 -140
  118. package/research-memory-evolution.md +0 -440
  119. package/research-rl-agent-memory.md +0 -330
  120. package/scripts/archive-distilled-junk.ts +0 -116
  121. package/scripts/backfill-anchors.ts +0 -130
  122. package/scripts/backfill-l0l1l2.ts +0 -98
  123. package/scripts/blind-scan.ts +0 -588
  124. package/scripts/cleanup-step1-exact-dups.ts +0 -117
  125. package/scripts/cleanup-step2-template-dups.ts +0 -113
  126. package/scripts/cleanup-step3-stale-content.ts +0 -149
  127. package/scripts/cleanup-step4-orphan-scopes.ts +0 -131
  128. package/scripts/delete-distilled-junk.ts +0 -88
  129. package/scripts/diagnose-vector-search.ts +0 -132
  130. package/scripts/distill-facts.ts +0 -305
  131. package/scripts/health-check.ts +0 -343
  132. package/scripts/incremental-ingest.sh +0 -71
  133. package/scripts/migrate-language.ts +0 -75
  134. package/scripts/mlp-health-check.ts +0 -382
  135. package/scripts/pull-claude-desktop.ts +0 -515
  136. package/scripts/re-ingest-cc.sh +0 -66
  137. package/scripts/rebalance-memory-health.ts +0 -181
  138. package/scripts/run-desktop-ingest.sh +0 -11
  139. package/scripts/start-server.sh +0 -12
  140. package/scripts/verify-emotion-decay.ts +0 -148
  141. package/scripts/viz-dashboard.html +0 -423
  142. package/scripts/viz-export.ts +0 -703
  143. package/src/__tests__/access-tracker-novelty.test.ts +0 -228
  144. package/src/__tests__/activity-counter.test.ts +0 -96
  145. package/src/__tests__/admission-control.test.ts +0 -206
  146. package/src/__tests__/anchor-generator.test.ts +0 -67
  147. package/src/__tests__/auto-consolidation.test.ts +0 -168
  148. package/src/__tests__/auto-recall.test.ts +0 -131
  149. package/src/__tests__/babel-integration.test.ts +0 -139
  150. package/src/__tests__/batch-dedup.test.ts +0 -65
  151. package/src/__tests__/capture-engine.test.ts +0 -1144
  152. package/src/__tests__/capture-heuristic.test.ts +0 -171
  153. package/src/__tests__/cascade-forget.test.ts +0 -210
  154. package/src/__tests__/category-min-scores.test.ts +0 -33
  155. package/src/__tests__/cc8-cc1-essential-context.test.ts +0 -298
  156. package/src/__tests__/circuit-breaker.test.ts +0 -99
  157. package/src/__tests__/cluster-consolidation.test.ts +0 -712
  158. package/src/__tests__/confidence-metadata.test.ts +0 -125
  159. package/src/__tests__/confidence-tracker.test.ts +0 -249
  160. package/src/__tests__/conflict-advisor.test.ts +0 -206
  161. package/src/__tests__/conflict-escalation.test.ts +0 -149
  162. package/src/__tests__/conflict-lifecycle.test.ts +0 -83
  163. package/src/__tests__/conflict-output.test.ts +0 -85
  164. package/src/__tests__/conflict-store.test.ts +0 -374
  165. package/src/__tests__/consolidation-engine.test.ts +0 -181
  166. package/src/__tests__/constructive-retrieval.test.ts +0 -509
  167. package/src/__tests__/context-collapse-renderer.test.ts +0 -214
  168. package/src/__tests__/context-composer-pins.test.ts +0 -64
  169. package/src/__tests__/context-composer-task-selection.test.ts +0 -24
  170. package/src/__tests__/context-reconstructor.test.ts +0 -65
  171. package/src/__tests__/context-renderer.test.ts +0 -91
  172. package/src/__tests__/continuity-eval.test.ts +0 -275
  173. package/src/__tests__/continuity-schema.test.ts +0 -97
  174. package/src/__tests__/conversation-importer.test.ts +0 -692
  175. package/src/__tests__/core-recall-eval.test.ts +0 -235
  176. package/src/__tests__/core-summary.test.ts +0 -109
  177. package/src/__tests__/data-checkup.test.ts +0 -225
  178. package/src/__tests__/decay-exemption.test.ts +0 -84
  179. package/src/__tests__/dedup-actions.test.ts +0 -58
  180. package/src/__tests__/dedup-guard.test.ts +0 -39
  181. package/src/__tests__/deterministic-id.test.ts +0 -39
  182. package/src/__tests__/distill-lock.test.ts +0 -176
  183. package/src/__tests__/doctor.test.ts +0 -303
  184. package/src/__tests__/dream-pipeline.test.ts +0 -232
  185. package/src/__tests__/drill-down.test.ts +0 -108
  186. package/src/__tests__/embedder.test.ts +0 -122
  187. package/src/__tests__/emotion-decay.test.ts +0 -46
  188. package/src/__tests__/emotion-detector.test.ts +0 -56
  189. package/src/__tests__/emotion-phase1-integration.test.ts +0 -153
  190. package/src/__tests__/emotion-retrieval.test.ts +0 -56
  191. package/src/__tests__/entity-resolver.test.ts +0 -58
  192. package/src/__tests__/ethics-layer.test.ts +0 -476
  193. package/src/__tests__/event-segmenter.test.ts +0 -153
  194. package/src/__tests__/feature-flags-integration.test.ts +0 -54
  195. package/src/__tests__/frequency-tracker.test.ts +0 -113
  196. package/src/__tests__/graph-export.test.ts +0 -489
  197. package/src/__tests__/hotness-blend.test.ts +0 -76
  198. package/src/__tests__/ingest.test.ts +0 -360
  199. package/src/__tests__/interference-detector.test.ts +0 -203
  200. package/src/__tests__/kg-extractor.test.ts +0 -184
  201. package/src/__tests__/kg-store.test.ts +0 -198
  202. package/src/__tests__/llm-consolidation.test.ts +0 -226
  203. package/src/__tests__/mcp-efficiency.test.ts +0 -288
  204. package/src/__tests__/memory-boundaries.test.ts +0 -237
  205. package/src/__tests__/memory-evolution-b2-b3-e1.test.ts +0 -243
  206. package/src/__tests__/memory-evolution.test.ts +0 -271
  207. package/src/__tests__/memory-health-rebalance.test.ts +0 -144
  208. package/src/__tests__/memory-lint.test.ts +0 -271
  209. package/src/__tests__/memory-output.test.ts +0 -215
  210. package/src/__tests__/multi-vector.test.ts +0 -242
  211. package/src/__tests__/narrative-architecture.test.ts +0 -456
  212. package/src/__tests__/pending-review.test.ts +0 -77
  213. package/src/__tests__/post-retrieval-filter.test.ts +0 -178
  214. package/src/__tests__/ppr-traversal.test.ts +0 -256
  215. package/src/__tests__/prediction-engine.test.ts +0 -349
  216. package/src/__tests__/predictive-prospective-memory.test.ts +0 -416
  217. package/src/__tests__/preference-guard-smoke.test.ts +0 -211
  218. package/src/__tests__/preference-matcher.test.ts +0 -230
  219. package/src/__tests__/preference-slots.test.ts +0 -205
  220. package/src/__tests__/prospective-memory.test.ts +0 -283
  221. package/src/__tests__/query-entity-detector.test.ts +0 -171
  222. package/src/__tests__/query-expander-alias.test.ts +0 -89
  223. package/src/__tests__/query-expander.test.ts +0 -34
  224. package/src/__tests__/recall-governor.test.ts +0 -227
  225. package/src/__tests__/recall-mode.test.ts +0 -64
  226. package/src/__tests__/result-synthesizer.test.ts +0 -198
  227. package/src/__tests__/resume-context.test.ts +0 -4898
  228. package/src/__tests__/retrieval-trace.test.ts +0 -213
  229. package/src/__tests__/retriever-boundary-weight.test.ts +0 -136
  230. package/src/__tests__/retroactive-boost.test.ts +0 -223
  231. package/src/__tests__/salience-scoring.test.ts +0 -106
  232. package/src/__tests__/scope-inventory.test.ts +0 -200
  233. package/src/__tests__/scope-policy.test.ts +0 -69
  234. package/src/__tests__/session-checkpoint.test.ts +0 -259
  235. package/src/__tests__/session-distiller.test.ts +0 -611
  236. package/src/__tests__/session-gc.test.ts +0 -215
  237. package/src/__tests__/skill-engine.test.ts +0 -546
  238. package/src/__tests__/skill-promotion.test.ts +0 -349
  239. package/src/__tests__/store-scope-audit.test.ts +0 -81
  240. package/src/__tests__/supersede-chain.test.ts +0 -154
  241. package/src/__tests__/temporal-parser.test.ts +0 -241
  242. package/src/__tests__/temporal-validity.test.ts +0 -160
  243. package/src/__tests__/term-registry.test.ts +0 -121
  244. package/src/__tests__/tool-output-compressor.test.ts +0 -133
  245. package/src/__tests__/topic-tag.test.ts +0 -113
  246. package/src/__tests__/trusted-memory.test.ts +0 -828
  247. package/src/__tests__/version-manager.test.ts +0 -213
  248. package/src/__tests__/workflow-observation.test.ts +0 -400
  249. package/src/__tests__/write-verifier.test.ts +0 -122
@@ -1,77 +0,0 @@
1
- # RecallNest Ingest Pipeline Upgrade: Conversation Fragments → Knowledge Extraction
2
-
3
- **Date**: 2026-03-19
4
- **Status**: Approved
5
- **Scope**: Route B (bulk distillation) + Route A (pipeline default change)
6
-
7
- ## Problem
8
-
9
- 29,984 total records; 23,453 (78.2%) are `fact` category — raw conversation slices (`[助手]`/`[用户]` prefixed) with no knowledge extraction. This is the root cause of 96.7% dead memory rate.
10
-
11
- ## Decisions
12
-
13
- | Decision | Choice | Rationale |
14
- |----------|--------|-----------|
15
- | Disposition | New + Archive (B) | Preserve originals for traceability; ~5K new records is not bloat |
16
- | Extraction model | Reuse `smartExtractBatch()` (C) | Already has 6-category prompt + scoring; no new wheel |
17
- | Batch strategy | Dry-run then batch+checkpoint (C→B) | Verify quality first, then fault-tolerant batch |
18
-
19
- ## Route B: Bulk Distillation
20
-
21
- ### New file: `scripts/distill-facts.ts`
22
-
23
- **Modes**:
24
- - `--dry-run`: Process 1 session (~30-50 facts), print results for human review
25
- - Default: Process all sessions with checkpoint
26
-
27
- **Pipeline per batch (20 texts)**:
28
- 1. `smartExtractBatch(texts)` → `SmartExtraction[]` (category, importance, l0, l1)
29
- 2. `embedBatchPassage(extractedTexts)` → vectors
30
- 3. Dedup against existing non-fact records (cosine > 0.80 → skip)
31
- 4. Store new records: `boundary.layer = "durable"`, `boundary.authority = "distillation"`
32
- 5. Archive originals: metadata += `{ archived: true, archivedAt, distilledTo: [ids] }`
33
- 6. Write checkpoint: `data/distill-progress.json` (scope → done/skip)
34
-
35
- **Expected output**: 23K facts → ~3K-5K structured knowledge (5:1 to 8:1 compression)
36
-
37
- ### Archive marking
38
-
39
- ```json
40
- {
41
- "archived": true,
42
- "archivedAt": "2026-03-19T10:00:00Z",
43
- "distilledTo": ["uuid-1", "uuid-2"],
44
- "originalCategory": "fact"
45
- }
46
- ```
47
-
48
- ## Route A: Pipeline Default Change
49
-
50
- ### Changes to `src/ingest.ts`
51
-
52
- 1. `smartExtractBatch()` becomes **default** (was opt-in via `--llm`)
53
- 2. New `--no-llm` flag for fallback mode
54
- 3. LLM failure → don't store raw slice; queue to `data/pending-extraction.json`
55
- 4. Next `lm ingest` auto-retries pending queue
56
-
57
- **Core principle**: Better to skip than to store garbage.
58
-
59
- ### Changes to `src/retriever.ts`
60
-
61
- - Default filter: exclude `archived = true` from search results
62
- - New `includeArchived` parameter for traceability queries
63
-
64
- ## Cost Estimate
65
-
66
- ~1,173 LLM calls (20 texts/batch), ~4.1M input + ~1.2M output tokens.
67
- GPT-4o-mini: ~$1.3 | Gemini Flash: ~$0.7 | Claude Haiku: ~$8
68
-
69
- ## Files to create/modify
70
-
71
- | File | Action |
72
- |------|--------|
73
- | `scripts/distill-facts.ts` | Create — bulk distillation script |
74
- | `src/ingest.ts` | Modify — flip smartExtract default |
75
- | `src/retriever.ts` | Modify — archived filter |
76
- | `data/distill-progress.json` | Auto-created at runtime |
77
- | `data/pending-extraction.json` | Auto-created at runtime |
@@ -1,95 +0,0 @@
1
- # Pull Claude Desktop Conversations into RecallNest
2
-
3
- Date: 2026-03-23
4
-
5
- ## Goal
6
-
7
- One script to pull all historical Claude Desktop / claude.ai conversations and convert them into RecallNest's ingest format, so `lm ingest` can process them through the existing pipeline (noise filter, dedup, LLM extraction, 6-category classification).
8
-
9
- ## Script
10
-
11
- `~/recallnest/scripts/pull-claude-desktop.ts` (Bun runtime)
12
-
13
- ## Authentication
14
-
15
- - `CLAUDE_SESSION_KEY` env var (from browser DevTools → Application → Cookies → claude.ai → `sessionKey`)
16
- - Also accept `--session-key` CLI flag
17
-
18
- ## Data Flow
19
-
20
- ```
21
- sessionKey → GET /api/organizations → orgId
22
- → GET /api/organizations/{orgId}/chat_conversations (paginated, 50/page)
23
- → for each conversation:
24
- GET .../chat_conversations/{id}?tree=True&rendering_mode=messages&render_all_tools=true
25
- → parse branches → extract latest branch message chain
26
- → convert to CC transcript .jsonl
27
- → write to data/desktop-import/{conversationId}.jsonl
28
- ```
29
-
30
- ## Output Format
31
-
32
- Each conversation becomes one `.jsonl` file with entries matching CC transcript schema:
33
-
34
- ```jsonl
35
- {"type":"user","message":{"role":"user","content":"text"},"sessionId":"conv-uuid","timestamp":"2025-01-15T10:00:00Z"}
36
- {"type":"assistant","message":{"role":"assistant","content":"reply"},"sessionId":"conv-uuid","timestamp":"2025-01-15T10:00:01Z"}
37
- ```
38
-
39
- ## Output Directory
40
-
41
- ```
42
- data/desktop-import/
43
- .sync-state.json
44
- {conversation-id}.jsonl
45
- ...
46
- ```
47
-
48
- ## Sync State (Resumable)
49
-
50
- `.sync-state.json` tracks:
51
-
52
- ```json
53
- {
54
- "lastSyncedAt": "ISO8601",
55
- "conversations": {
56
- "conv-id": {
57
- "messageCount": 42,
58
- "updatedAt": "ISO8601",
59
- "pulledAt": "ISO8601"
60
- }
61
- }
62
- }
63
- ```
64
-
65
- Re-run skips conversations whose `updatedAt` hasn't changed since last pull.
66
-
67
- ## Branch Handling
68
-
69
- Claude web conversations support branching (edit/retry). The script extracts only the latest branch by walking `parent_message_uuid` chain from the most recent message backward.
70
-
71
- ## Rate Limiting
72
-
73
- - 200ms delay between conversation detail requests
74
- - On 429 or 5xx: exponential backoff starting at 1s, max 3 retries
75
- - On auth failure (401/403): abort immediately with clear message
76
-
77
- ## Error Handling
78
-
79
- - Single conversation failure does not abort the batch
80
- - Failed conversations logged and reported in final summary
81
- - Summary output: total conversations, pulled, skipped (already synced), failed
82
-
83
- ## Ingest Integration
84
-
85
- After pulling, user runs:
86
-
87
- ```bash
88
- lm ingest --source cc
89
- ```
90
-
91
- Need to verify that ingest scans `data/desktop-import/` or that the path is configurable. If not, the script will symlink or copy files to the CC transcript directory.
92
-
93
- ## Dependencies
94
-
95
- - Only Bun built-ins (fetch, fs, path) — no new npm packages
@@ -1,398 +0,0 @@
1
- # Constructive Retrieval + Emotional Valence — Design Spec
2
-
3
- > Date: 2026-04-11
4
- > Status: Approved
5
- > Scope: RecallNest v1.5 upgrade
6
- > Philosophy basis: Simulation Theory (Michaelian) + Affective Memory (cognitive science)
7
- > Risk strategy: Triple fallback — every new path degrades gracefully to current behavior
8
-
9
- ---
10
-
11
- ## 1. Problem Statement
12
-
13
- RecallNest's retrieval pipeline (21 stages) returns **raw stored text**. This is a "causal theory" model — faithfully reproducing what was stored. But human memory doesn't work this way: we reconstruct memories in context, adapting them to our current needs.
14
-
15
- Additionally, the decay engine lacks an emotional dimension. Cognitive science shows emotionally charged memories decay slower and are retrieved preferentially in matching emotional contexts. RecallNest currently treats all memories as emotionally neutral.
16
-
17
- ## 2. Goals
18
-
19
- 1. **Constructive Retrieval**: When context benefits from it, synthesize a coherent narrative from top-N retrieved memories, adapted to the current query context
20
- 2. **Emotional Valence**: Tag memories with emotional metadata at storage time, use it to modulate decay rates and retrieval scoring
21
- 3. **Zero regression**: All new behavior behind feature flags, all paths fallback to current behavior
22
-
23
- ## 3. Non-Goals
24
-
25
- - Replacing the existing retrieval pipeline (it stays as-is)
26
- - Complex emotion models (no 6-axis emotion wheels; just valence + arousal)
27
- - Multi-agent shared memory (future phase)
28
- - Autobiographical narrative architecture (future phase)
29
-
30
- ---
31
-
32
- ## 4. Constructive Retrieval
33
-
34
- ### 4.1 Architecture
35
-
36
- New component: `context-reconstructor.ts` (~200-300 LOC)
37
-
38
- Position in pipeline:
39
- ```
40
- Existing 21-stage pipeline → top-N RetrievalResult[]
41
- ↓
42
- Reconstruction Gate
43
- (should reconstruct?)
44
- ↓ yes ↓ no
45
- Context Reconstructor Return raw
46
- ↓
47
- Grounding Validator
48
- ↓
49
- Return { reconstructed, sources, confidence, raw }
50
- ```
51
-
52
- ### 4.2 Reconstruction Gate
53
-
54
- Conditions for reconstruction (ALL must be true):
55
- - Feature flag `RECALLNEST_CONSTRUCTIVE_RETRIEVAL=true`
56
- - Caller opt-in: `reconstruct: true` in retrieval context
57
- - Retrieved results count ≥ 3 (too few = not enough material)
58
- - LLM client available and responding
59
-
60
- If any condition fails → return raw results immediately. No error, no log noise.
61
-
62
- ### 4.3 Context Reconstructor
63
-
64
- **Input:**
65
- ```ts
66
- interface ReconstructionInput {
67
- query: string; // current user query / resume context
68
- results: RetrievalResult[]; // top-N from pipeline (max 10)
69
- mode: "resume" | "search"; // affects prompt style
70
- maxTokens?: number; // output budget (default 500)
71
- }
72
- ```
73
-
74
- **LLM Prompt Strategy:**
75
- ```
76
- System: You are a memory reconstruction engine. Given a set of stored memories
77
- and a current context, synthesize a coherent, contextually relevant summary.
78
-
79
- Rules:
80
- 1. Every factual claim MUST cite its source as [src:MEMORY_ID]
81
- 2. Do NOT invent facts not present in any source memory
82
- 3. If memories contradict each other, present both views with [conflict] tag
83
- 4. Prioritize information most relevant to the current query context
84
- 5. Keep output concise — this is context, not an essay
85
-
86
- Current context: {query}
87
-
88
- Source memories:
89
- {for each result: [ID: {id}] {text} (importance: {importance}, stored: {date})}
90
-
91
- Synthesize a contextually relevant reconstruction:
92
- ```
93
-
94
- **Output:**
95
- ```ts
96
- interface ReconstructionOutput {
97
- reconstructed: string; // synthesized text with [src:ID] citations
98
- sources: string[]; // list of cited memory IDs
99
- confidence: number; // [0, 1] grounding confidence
100
- fallbackReason?: string; // if degraded, why
101
- raw: RetrievalResult[]; // always included
102
- }
103
- ```
104
-
105
- ### 4.4 Grounding Validator
106
-
107
- Three-layer validation, executed sequentially:
108
-
109
- **Layer 1: ID Verification**
110
- ```ts
111
- for (const citedId of extractCitedIds(reconstructed)) {
112
- if (!topNResults.some(r => r.id === citedId)) {
113
- reconstructed = removeSentencesWithId(reconstructed, citedId);
114
- confidence -= 0.2;
115
- }
116
- }
117
- ```
118
- - Extract all `[src:MEMORY_ID]` references from LLM output
119
- - Verify each ID exists in the top-N result set
120
- - Remove sentences citing non-existent IDs (hallucination)
121
- - Penalize confidence per phantom citation
122
-
123
- **Layer 2: Coverage Check**
124
- ```ts
125
- const coverage = computeSemanticOverlap(reconstructed, topNResults);
126
- if (coverage < 0.6) {
127
- return { reconstructed: null, fallbackReason: "low_grounding", raw };
128
- }
129
- ```
130
- - Compute semantic overlap between reconstructed text and source memories
131
- - Method: chunk reconstructed text into sentences, embed each, find max cosine similarity to any source memory
132
- - Coverage = percentage of sentences with similarity > 0.7
133
- - Below 60% → full degradation to raw results
134
-
135
- **Layer 3: Timeout Guard**
136
- ```ts
137
- const result = await Promise.race([
138
- reconstruct(input),
139
- timeout(3000).then(() => ({ fallbackReason: "timeout" }))
140
- ]);
141
- ```
142
- - 3-second hard timeout on entire reconstruction pipeline
143
- - Timeout → return raw results
144
-
145
- ### 4.5 Integration Points
146
-
147
- **resume_context (context-composer.ts):**
148
- - After assembling stable context + patterns + cases
149
- - Call reconstructor with mode="resume", query=task description from checkpoint
150
- - If reconstruction succeeds, use it as the primary context narrative
151
- - Always keep raw results in response for transparency
152
- - Default: reconstruction ON when feature flag is ON
153
-
154
- **search_memory (mcp-server.ts):**
155
- - New optional parameter: `reconstruct?: boolean` (default false)
156
- - When true, pass through to retriever with reconstruct flag
157
- - Return both reconstructed and raw in tool response
158
-
159
- **brief_memory, memory_lint, list_pins, etc.:**
160
- - Never reconstruct. These tools need raw data.
161
-
162
- ### 4.6 Observability
163
-
164
- - Trace stage: `reconstruction` added to TraceCollector
165
- - Logged fields: `{ attempted: boolean, succeeded: boolean, confidence: number, fallbackReason?: string, latencyMs: number, sourceCount: number }`
166
- - No memory content logged (privacy)
167
-
168
- ---
169
-
170
- ## 5. Emotional Valence
171
-
172
- ### 5.1 Schema Extension
173
-
174
- Add optional `emotion` field to memory metadata:
175
-
176
- ```ts
177
- interface EmotionMetadata {
178
- valence: number; // [-1, 1] negative ← → positive
179
- arousal: number; // [0, 1] calm ← → excited
180
- label?: string; // human-readable: "frustration" | "excitement" | "neutral" | etc.
181
- }
182
- ```
183
-
184
- Stored in `metadata` JSON string (no schema migration needed):
185
- ```json
186
- {
187
- "tier": "working",
188
- "emotion": { "valence": -0.7, "arousal": 0.8, "label": "frustration" }
189
- }
190
- ```
191
-
192
- ### 5.2 Detection at Store Time
193
-
194
- **Two-tier detection (store.ts / noise-filter.ts):**
195
-
196
- **Tier 1: Heuristic (zero LLM cost, always runs)**
197
- ```ts
198
- const NEGATIVE_SIGNALS = ["失败", "痛苦", "困扰", "bug", "broken", "frustrat", "hate", "wrong", "error"];
199
- const POSITIVE_SIGNALS = ["突破", "搞定", "成功", "solved", "great", "love", "perfect", "works"];
200
- const HIGH_AROUSAL = ["!", "紧急", "urgent", "critical", "immediately", "ASAP"];
201
-
202
- function heuristicEmotion(text: string): EmotionMetadata {
203
- const negCount = NEGATIVE_SIGNALS.filter(s => text.toLowerCase().includes(s)).length;
204
- const posCount = POSITIVE_SIGNALS.filter(s => text.toLowerCase().includes(s)).length;
205
- const arousalCount = HIGH_AROUSAL.filter(s => text.includes(s)).length;
206
-
207
- const valence = clamp((posCount - negCount) * 0.3, -1, 1);
208
- const arousal = clamp(arousalCount * 0.3, 0, 1);
209
- const label = valence > 0.3 ? "positive" : valence < -0.3 ? "negative" : "neutral";
210
-
211
- return { valence, arousal, label };
212
- }
213
- ```
214
-
215
- **Tier 2: LLM (optional, piggybacked on importance evaluation)**
216
- - When LLM is called to evaluate importance (existing flow), add to prompt:
217
- ```
218
- Also rate the emotional tone:
219
- - valence: [-1, 1] (negative to positive)
220
- - arousal: [0, 1] (calm to excited)
221
- - label: one word (e.g., "frustration", "excitement", "neutral")
222
- ```
223
- - Merged into existing LLM call → zero additional API cost
224
- - LLM result overrides heuristic result when available
225
-
226
- **Backward compatibility:** Existing memories without emotion field → treated as `{ valence: 0, arousal: 0, label: "neutral" }` everywhere.
227
-
228
- ### 5.3 Decay Integration (decay-engine.ts)
229
-
230
- **Modified half-life calculation:**
231
- ```ts
232
- function adjustedHalfLife(baseHalfLife: number, emotion?: EmotionMetadata): number {
233
- if (!emotion) return baseHalfLife;
234
- const emotionalIntensity = Math.abs(emotion.valence);
235
- // Emotional memories last up to 30% longer
236
- const boost = 1 + 0.3 * emotionalIntensity;
237
- return baseHalfLife * boost;
238
- }
239
- ```
240
-
241
- - Strong negative memory (valence = -0.9): half-life × 1.27
242
- - Strong positive memory (valence = 0.8): half-life × 1.24
243
- - Neutral memory (valence = 0.1): half-life × 1.03 (negligible)
244
- - No emotion field: no change
245
-
246
- **Arousal modulation (secondary):**
247
- ```ts
248
- // High arousal slightly accelerates initial strength but doesn't change half-life
249
- // This models the "flashbulb memory" effect — vivid but not necessarily longer-lasting
250
- const initialBoost = 1 + 0.1 * (emotion?.arousal ?? 0);
251
- ```
252
-
253
- ### 5.4 Retrieval Integration (retriever.ts)
254
-
255
- **New scoring stage: `applyEmotionWeight` (after `applyImportanceWeight`)**
256
-
257
- ```ts
258
- function applyEmotionWeight(
259
- results: ScoredResult[],
260
- queryEmotion: EmotionMetadata | null
261
- ): ScoredResult[] {
262
- if (!queryEmotion || Math.abs(queryEmotion.valence) < 0.2) {
263
- return results; // neutral query → no-op
264
- }
265
-
266
- return results.map(r => {
267
- const memEmotion = parseEmotion(r.metadata);
268
- if (!memEmotion) return r; // no emotion data → unchanged
269
-
270
- // Emotional alignment boost: same-valence memories score higher
271
- const alignment = 1 - Math.abs(queryEmotion.valence - memEmotion.valence) / 2;
272
- const boost = 1.0 + 0.15 * alignment;
273
-
274
- return { ...r, score: r.score * boost };
275
- });
276
- }
277
- ```
278
-
279
- - Query has emotional tone + memory has matching emotion → up to 15% score boost
280
- - Query is neutral → stage is no-op (zero overhead)
281
- - Memory has no emotion data → unchanged
282
-
283
- **Query emotion detection:**
284
- - Reuse heuristic detector on query text
285
- - Lightweight, no LLM needed at query time
286
-
287
- ### 5.5 Feature Flag
288
-
289
- `RECALLNEST_EMOTION_SCORING=true` — controls:
290
- - Emotion detection at store time
291
- - Emotion-adjusted decay calculation
292
- - Emotion scoring stage in retriever
293
-
294
- When OFF: all emotion code paths are no-op. Existing behavior preserved exactly.
295
-
296
- ---
297
-
298
- ## 6. File Change Map
299
-
300
- | File | Change Type | Description |
301
- |------|-------------|-------------|
302
- | `src/context-reconstructor.ts` | **NEW** | Reconstruction engine + grounding validator |
303
- | `src/retriever.ts` | Modify | Add reconstruction gate at output + emotion scoring stage |
304
- | `src/decay-engine.ts` | Modify | Add emotion-adjusted half-life |
305
- | `src/memory-schema.ts` | Modify | Add EmotionMetadata type definition |
306
- | `src/store.ts` | Modify | Call emotion detector at store time |
307
- | `src/context-composer.ts` | Modify | Call reconstructor in resume flow |
308
- | `src/mcp-server.ts` | Modify | Add `reconstruct` param to search_memory |
309
- | `src/config.ts` | Modify | Add feature flags |
310
- | `src/__tests__/context-reconstructor.test.ts` | **NEW** | Unit tests for reconstruction + grounding |
311
- | `src/__tests__/emotion-scoring.test.ts` | **NEW** | Unit tests for emotion detection + decay + retrieval |
312
-
313
- **Estimated LOC:**
314
- - New code: ~400-500 lines
315
- - Modified code: ~100-150 lines across existing files
316
- - Tests: ~300-400 lines
317
-
318
- ---
319
-
320
- ## 7. Test Strategy
321
-
322
- ### Unit Tests
323
- - Reconstruction gate: all 4 conditions tested (flag off, no opt-in, too few results, LLM down)
324
- - Grounding validator: phantom ID removal, coverage threshold, timeout
325
- - Emotion heuristic: positive/negative/neutral/mixed texts
326
- - Decay adjustment: verify half-life calculations
327
- - Retrieval emotion stage: neutral query no-op, matching boost, missing data handling
328
-
329
- ### Integration Tests
330
- - End-to-end: store memories → search with reconstruct=true → verify output format
331
- - End-to-end: store emotional memory → verify decay score after N days vs neutral memory
332
- - resume_context with reconstruction → verify grounding and fallback
333
-
334
- ### Regression Tests
335
- - All feature flags OFF → existing test suite must pass unchanged (zero regression)
336
- - Memories without emotion field → same scores as before
337
-
338
- ---
339
-
340
- ## 8. Rollout Plan
341
-
342
- **Phase 1: Emotion Valence (low risk, no LLM dependency)**
343
- 1. Add schema types
344
- 2. Implement heuristic detector
345
- 3. Integrate into store path
346
- 4. Add decay adjustment
347
- 5. Add retrieval scoring stage
348
- 6. Tests
349
- 7. Feature flag ON for testing
350
-
351
- **Phase 2: Constructive Retrieval (medium risk, LLM dependent)**
352
- 1. Implement context-reconstructor.ts
353
- 2. Implement grounding validator
354
- 3. Integrate into retriever output
355
- 4. Integrate into context-composer
356
- 5. Add search_memory parameter
357
- 6. Tests
358
- 7. Feature flag ON for testing
359
-
360
- **Phase 3: LLM Emotion Detection (optional enhancement)**
361
- 1. Piggyback on importance evaluation prompt
362
- 2. Override heuristic when available
363
- 3. A/B test heuristic vs LLM accuracy
364
-
365
- ---
366
-
367
- ## 9. Success Criteria
368
-
369
- - [ ] Feature flags OFF → all existing tests pass (baseline per CLAUDE.md)
370
- - [ ] Emotion-tagged memories decay 20-30% slower than neutral (verified in test)
371
- - [ ] Reconstruction produces grounded output (coverage ≥ 0.6) in ≥ 80% of cases
372
- - [ ] Reconstruction fallback triggers correctly when LLM is unavailable
373
- - [ ] resume_context with reconstruction provides more contextually relevant summaries (qualitative)
374
- - [ ] No additional LLM API calls for emotion detection (piggybacked on existing calls)
375
- - [ ] Reconstruction latency < 3s in 95% of cases
376
-
377
- ---
378
-
379
- ## 10. Open Questions (resolved)
380
-
381
- | Question | Decision |
382
- |----------|----------|
383
- | How aggressive should reconstruction be? | **Moderate (B)**: LLM rewrite from top-N, not fragment synthesis |
384
- | What emotion model? | **Valence-Arousal**: simple, proven, low overhead |
385
- | When to reconstruct by default? | **resume_context only**; search_memory opt-in |
386
- | Grounding threshold? | **0.6 coverage**; tunable via config |
387
- | Timeout? | **3 seconds**; hard cutoff |
388
-
389
- ---
390
-
391
- ## Sources
392
-
393
- - Michaelian, K. — Simulation Theory of Memory
394
- - Ebbinghaus Forgetting Curve + Weibull extensions
395
- - Cognitive science consensus on emotional memory persistence
396
- - [Stanford Encyclopedia of Philosophy: Memory](https://plato.stanford.edu/entries/memory/)
397
- - [Memory in the Age of AI Agents](https://arxiv.org/abs/2512.13564)
398
- - [SuperLocalMemory V3.3: Biologically-Inspired Forgetting](https://arxiv.org/html/2604.04514)
@@ -1,117 +0,0 @@
1
- # RecallNest Continuity Eval
2
-
3
- - Generated: 2026-03-16T03:35:14.079Z
4
- - Cases: 5
5
- - Passed: 5/5
6
- - Average score: 94.0%
7
-
8
- | Case | Profile | Score | Pass | Stable | Patterns | Cases | Checkpoint |
9
- |------|---------|-------|------|--------|----------|-------|------------|
10
- | writing_style_continuity | writing | 100% | yes | 3 | 1 | 0 | no |
11
- | visual_style_continuity | default | 100% | yes | 3 | 0 | 0 | no |
12
- | project_context_continuity | default | 100% | yes | 1 | 2 | 1 | no |
13
- | workflow_pattern_gap_check | default | 70% | yes | 3 | 3 | 1 | no |
14
- | recallnest_checkpoint_continuity | default | 100% | yes | 3 | 1 | 1 | yes |
15
-
16
- ## Case Notes
17
-
18
- ### writing_style_continuity
19
- - Task: 继续写我的AI小木屋的文章
20
- - Score: 100%
21
- - Pass: yes
22
- - Stable items: 3
23
- - Pattern items: 1
24
- - Case items: 0
25
- - Checkpoint present: no
26
- - Stable preview: Task focus: 小木屋的文章 | Pinned: 用户稳定偏好口语化、不端着、可以自嘲但不说教。表达应像朋友聊天,避免居高临下、用力过猛或碎嘴。 | Pinned: 用户常用视觉风格是手绘涂鸦风加高对比撞色;在内容包装和配图生成时,应优先沿用这一审美方向,除非任务目标明确要求其他风格。
27
- - Pattern preview: Workflow pattern: Promote recurring continuity workflow Use when: When the same startup or handoff workflow repeats a...
28
- - Case preview: -
29
- - Checkpoint summary: -
30
- - Matched stable any: 口语化, 不端着
31
- - Matched stable all: -
32
- - Matched patterns any: -
33
- - Matched cases any: -
34
- - Matched checkpoint any: -
35
- - Forbidden matches: -
36
- - Notes: Fresh windows should carry over stable writing-style preferences.
37
-
38
- ### visual_style_continuity
39
- - Task: 给文章做封面和配图
40
- - Score: 100%
41
- - Pass: yes
42
- - Stable items: 3
43
- - Pattern items: 0
44
- - Case items: 0
45
- - Checkpoint present: no
46
- - Stable preview: Task focus: 给文章做封面和配图 | Pinned: 用户常用视觉风格是手绘涂鸦风加高对比撞色;在内容包装和配图生成时,应优先沿用这一审美方向,除非任务目标明确要求其他风格。 | Pinned: 用户稳定偏好口语化、不端着、可以自嘲但不说教。表达应像朋友聊天,避免居高临下、用力过猛或碎嘴。
47
- - Pattern preview: -
48
- - Case preview: -
49
- - Checkpoint summary: -
50
- - Matched stable any: 手绘涂鸦, 高对比撞色
51
- - Matched stable all: -
52
- - Matched patterns any: -
53
- - Matched cases any: -
54
- - Matched checkpoint any: -
55
- - Forbidden matches: -
56
- - Notes: Visual direction should survive without the user restating it.
57
-
58
- ### project_context_continuity
59
- - Task: 一起继续排查 RecallNest 的连续性问题
60
- - Score: 100%
61
- - Pass: yes
62
- - Stable items: 1
63
- - Pattern items: 2
64
- - Case items: 1
65
- - Checkpoint present: no
66
- - Stable preview: Task focus: recallnest
67
- - Pattern preview: Workflow pattern: Cross-window continuity handoff Use when: When opening a fresh terminal window for the same project... | Workflow pattern: Promote recurring continuity workflow Use when: When the same startup or handoff workflow repeats a...
68
- - Case preview: 我先看真实召回密度,确认问题是不是出在 `scope: recallnest` 太窄。如果当前库里几乎没有这个 scope 的 durable memory,就该让 `resume_context` 更积极地退回到项目名驱动的 glo...
69
- - Checkpoint summary: -
70
- - Matched stable any: RecallNest
71
- - Matched stable all: -
72
- - Matched patterns any: -
73
- - Matched cases any: -
74
- - Matched checkpoint any: -
75
- - Forbidden matches: -
76
- - Notes: Fresh windows should at least recover the active project context.
77
-
78
- ### workflow_pattern_gap_check
79
- - Task: 开个新窗口继续做同一个终端项目
80
- - Score: 70%
81
- - Pass: yes
82
- - Stable items: 3
83
- - Pattern items: 3
84
- - Case items: 1
85
- - Checkpoint present: no
86
- - Stable preview: Task focus: 开个新窗口继续做同一个终端项目 | Pinned: 用户稳定偏好口语化、不端着、可以自嘲但不说教。表达应像朋友聊天,避免居高临下、用力过猛或碎嘴。 | Pinned: 用户偏好先本地测通再上 Bot,避免烧配额;代码改动完成后要求 commit + push,避免本地与 GitHub 版本漂移。
87
- - Pattern preview: Workflow pattern: Cross-window continuity handoff Use when: When opening a fresh terminal window for the same project... | Workflow pattern: Promote recurring continuity workflow Use when: When the same startup or handoff workflow repeats a... | Workflow pattern: Checkpoint before switching windows Use when: When leaving an active coding window that will likely...
88
- - Case preview: Case: Three-terminal continuity trigger validation Problem: Claude Code, Codex, and Gemini CLI were configured with M...
89
- - Checkpoint summary: -
90
- - Matched stable any: -
91
- - Matched stable all: -
92
- - Matched patterns any: resume_context, checkpoint
93
- - Matched cases any: -
94
- - Matched checkpoint any: -
95
- - Forbidden matches: -
96
- - Notes: This is a stretch case. It should start passing as more workflow patterns are promoted into durable memory.
97
-
98
- ### recallnest_checkpoint_continuity
99
- - Task: 继续 recallnest
100
- - Score: 100%
101
- - Pass: yes
102
- - Stable items: 3
103
- - Pattern items: 1
104
- - Case items: 1
105
- - Checkpoint present: yes
106
- - Stable preview: Checkpoint focus: RecallNest continuity layer — 状态交接给下个窗口 | Checkpoint summary: RecallNest Phase 3/4 并行推进中。核心三件套 store_memory/checkpoint_session/resume_context 已上线三端 MCP,但 resum... | Checkpoint decision: Phase 3/4 并行推进,continuity layer 是当前核心方向
107
- - Pattern preview: Workflow pattern: Promote recurring continuity workflow Use when: When the same startup or handoff workflow repeats a...
108
- - Case preview: Case: RecallNest sparse startup context cleanup Problem: resume_context was returning noisy transcript fragments and ...
109
- - Checkpoint summary: RecallNest Phase 3/4 并行推进中。核心三件套 store_memory/checkpoint_session/resume_context 已上线三端 MCP,但 resume_context compose 质量是最大短板——实测只返回 3 条 stable context、0 patter...
110
- - Matched stable any: -
111
- - Matched stable all: -
112
- - Matched patterns any: -
113
- - Matched cases any: -
114
- - Matched checkpoint any: Phase 3, resume_context, store_memory
115
- - Forbidden matches: -
116
- - Notes: When a project checkpoint exists, resume_context should prefer checkpoint-led continuity over noisy transcript fragments.
117
-