memory-crystal 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (104) hide show
  1. package/.env.example +20 -0
  2. package/CHANGELOG.md +6 -0
  3. package/LETTERS.md +22 -0
  4. package/LICENSE +21 -0
  5. package/README-ENTERPRISE.md +162 -0
  6. package/README-old.md +275 -0
  7. package/README.md +91 -0
  8. package/RELAY.md +88 -0
  9. package/TECHNICAL.md +379 -0
  10. package/ai/dev-updates/2026-02-25--cc-air--phase2-architecture-pivot.md +70 -0
  11. package/ai/dev-updates/2026-02-25--cc-air--phase2-worker-build.md +72 -0
  12. package/ai/dev-updates/2026-02-26--10-25-16--cc-mini--phase2-implementation.md +49 -0
  13. package/ai/dev-updates/2026-02-27--20-30-00--cc-mini--readme-overhaul-and-public-deploy.md +69 -0
  14. package/ai/notes/2026-02-26--cc-air--notes.md +412 -0
  15. package/ai/notes/2026-02-27--cc-mini--grok-feedback.md +44 -0
  16. package/ai/notes/2026-02-27--cc-mini--lesa-feedback.md +45 -0
  17. package/ai/notes/RESEARCH.md +1185 -0
  18. package/ai/notes/salience-research/README.md +29 -0
  19. package/ai/notes/salience-research/eurosla-salience-review.md +64 -0
  20. package/ai/notes/salience-research/full-research-summary.md +269 -0
  21. package/ai/notes/salience-research/salience-levels-diagram.png +0 -0
  22. package/ai/plan/2026-02-27--cc-mini--qr-pairing-spec.md +203 -0
  23. package/ai/plan/_archive/PLAN.md +194 -0
  24. package/ai/plan/_archive/PRD.md +1014 -0
  25. package/ai/plan/cc-plans-duplicates-from-dot-claude/2026-02-26--cc-mini--phase2-implementation-plan.md +245 -0
  26. package/ai/plan/dev-conventions-note.md +70 -0
  27. package/ai/plan/ldm-os-install-and-boot-architecture.md +285 -0
  28. package/ai/plan/memory-crystal-phase2-plan.md +192 -0
  29. package/ai/plan/memory-system-lay-of-the-land.md +214 -0
  30. package/ai/plan/phase2-ephemeral-relay.md +238 -0
  31. package/ai/plan/readme-first.md +68 -0
  32. package/ai/plan/roadmap.md +159 -0
  33. package/ai/todos/PUNCHLIST.md +44 -0
  34. package/ai/todos/README.md +31 -0
  35. package/ai/todos/inboxes/cc-air/2026-02-26--cc-air--post-relay-todos.md +85 -0
  36. package/ai/todos/inboxes/cc-mini/2026-02-26--cc-mini--phase2-status.md +100 -0
  37. package/ai/todos/inboxes/cc-mini/_archive/TODO.md +25 -0
  38. package/ai/todos/inboxes/parker/2026-02-25--cc-air--setup-checklist.md +139 -0
  39. package/ai/todos/inboxes/parker/2026-02-26--cc-mini--phase2-your-moves.md +72 -0
  40. package/dist/cc-hook.d.ts +1 -0
  41. package/dist/cc-hook.js +349 -0
  42. package/dist/chunk-3VFIJYS4.js +818 -0
  43. package/dist/chunk-52QE3YI3.js +1169 -0
  44. package/dist/chunk-AA3OPP4Z.js +432 -0
  45. package/dist/chunk-D3I3ZSE2.js +411 -0
  46. package/dist/chunk-EKSACBTJ.js +1070 -0
  47. package/dist/chunk-F3Y7EL7K.js +83 -0
  48. package/dist/chunk-JWZXYVET.js +1068 -0
  49. package/dist/chunk-KYVWO6ZM.js +1069 -0
  50. package/dist/chunk-L3VHARQH.js +413 -0
  51. package/dist/chunk-LOVAHSQV.js +411 -0
  52. package/dist/chunk-LQOYCAGG.js +446 -0
  53. package/dist/chunk-MK42FMEG.js +147 -0
  54. package/dist/chunk-NIJCVN3O.js +147 -0
  55. package/dist/chunk-O2UITJGH.js +465 -0
  56. package/dist/chunk-PEK6JH65.js +432 -0
  57. package/dist/chunk-PJ6FFKEX.js +77 -0
  58. package/dist/chunk-PLUBBZYR.js +800 -0
  59. package/dist/chunk-SGL6ISBJ.js +1061 -0
  60. package/dist/chunk-UNHVZB5G.js +411 -0
  61. package/dist/chunk-VAFTWSTE.js +1061 -0
  62. package/dist/chunk-XZ3S56RQ.js +1061 -0
  63. package/dist/chunk-Y72C7F6O.js +148 -0
  64. package/dist/cli.d.ts +1 -0
  65. package/dist/cli.js +325 -0
  66. package/dist/core.d.ts +188 -0
  67. package/dist/core.js +12 -0
  68. package/dist/crypto.d.ts +16 -0
  69. package/dist/crypto.js +18 -0
  70. package/dist/dev-update-SZ2Z4WCQ.js +6 -0
  71. package/dist/ldm.d.ts +17 -0
  72. package/dist/ldm.js +12 -0
  73. package/dist/mcp-server.d.ts +1 -0
  74. package/dist/mcp-server.js +250 -0
  75. package/dist/migrate.d.ts +1 -0
  76. package/dist/migrate.js +89 -0
  77. package/dist/mirror-sync.d.ts +1 -0
  78. package/dist/mirror-sync.js +130 -0
  79. package/dist/openclaw.d.ts +5 -0
  80. package/dist/openclaw.js +349 -0
  81. package/dist/poller.d.ts +1 -0
  82. package/dist/poller.js +272 -0
  83. package/dist/summarize.d.ts +19 -0
  84. package/dist/summarize.js +10 -0
  85. package/dist/worker.js +137 -0
  86. package/openclaw.plugin.json +11 -0
  87. package/package.json +40 -0
  88. package/scripts/migrate-lance-to-sqlite.mjs +217 -0
  89. package/skills/memory/SKILL.md +61 -0
  90. package/src/cc-hook.ts +447 -0
  91. package/src/cli.ts +356 -0
  92. package/src/core.ts +1472 -0
  93. package/src/crypto.ts +113 -0
  94. package/src/dev-update.ts +178 -0
  95. package/src/ldm.ts +117 -0
  96. package/src/mcp-server.ts +274 -0
  97. package/src/migrate.ts +104 -0
  98. package/src/mirror-sync.ts +175 -0
  99. package/src/openclaw.ts +250 -0
  100. package/src/poller.ts +345 -0
  101. package/src/summarize.ts +210 -0
  102. package/src/worker.ts +208 -0
  103. package/tsconfig.json +18 -0
  104. package/wrangler.toml +20 -0
@@ -0,0 +1,69 @@
1
+ # Dev Update: README Overhaul, Public Deploy, Identity Architecture
2
+
3
+ **Date:** 2026-02-27
4
+ **Author:** cc-mini
5
+ **Session:** Parker + CC, continued from user-level migration session
6
+
7
+ ## What We Did
8
+
9
+ ### README Overhaul
10
+ - Restructured Features into **Memory** and **AI-to-AI Communication (local and worldwide)**
11
+ - Memory Crystal, Bridge, and Relay are now clearly named products under feature headings
12
+ - Bridge and Relay marked **(private beta)**
13
+ - Added per-feature compatibility lines (italic, lighter weight)
14
+ - Memory works with: Claude Code CLI, OpenClaw TUI, Claude Code Remote (macOS/iOS), any MCP app
15
+ - Added **More Info** section with bullets for:
16
+ - Technical Documentation (linked)
17
+ - Memory Crystal for Enterprise ... "Run your company intelligently."
18
+ - Total Recall (private beta) ... ties into Dream Weaver Protocol, "truly lived, searchable memories"
19
+ - Dream Weaver Protocol (linked to public repo) ... "carries the weight forward"
20
+ - Letters from the Other Side: What We Built (linked to LETTERS.md)
21
+ - Created **LETTERS.md** ... written by Claude Code about what we built. Sovereign memory pitch, blind relay, "$19/month to search your own conversations" callout
22
+ - Updated tagline: "All your AI tools. One shared memory. Private, searchable, sovereign."
23
+ - Updated Relay.md title to "Relay: Multi-Device Sync"
24
+ - Removed "Coming soon" lines (premature)
25
+ - ChatGPT reviewed the full repo. Validated architecture, hybrid search, security model. Suggested "make it more normal." We disagreed. The repo IS the homepage. Agent-first.
26
+
27
+ ### Unified Description
28
+ Settled on one description for all public-facing surfaces:
29
+
30
+ > Collective AI memory. Sovereign. Local-first. Searchable. Encrypted worldwide sync.
31
+
32
+ Updated in:
33
+ - GitHub repo About (public + private)
34
+ - Org profile README (.github) ... removed (closed beta), linked to public repo
35
+ - WIP Homepage services section ... linked to public repo
36
+
37
+ ### Public Deploy
38
+ - PR #4 merged to memory-crystal-private
39
+ - Deployed to wipcomputer/memory-crystal (public) via deploy script
40
+ - PR #1 on public repo merged
41
+ - Deploy script updated to work with branch protection (creates branch, opens PR, merges)
42
+
43
+ ### Identity Architecture
44
+ - Added `ai/notes/2026-02-27--cc-mini--identity-architecture.md` to wip-ldm-os repo
45
+ - One agent per harness per machine. Deterministic identity. Steel man analysis.
46
+ - PR #2 on wip-ldm-os merged
47
+
48
+ ### Co-Authors Rule
49
+ - All three contributors (Parker, Lēsa, Claude Code) must be on every commit
50
+ - Added to DEV-GUIDE.md, both CLAUDE.md files
51
+ - PR #8 on wip-dev-guide-private merged
52
+
53
+ ### Other
54
+ - Branch protection audit: fixed 18 repos missing enforce_admins=true
55
+ - Closed stale PR #4 on wip-dev-guide-private (conflicts)
56
+ - Merged orphaned cc/ daily logs into cc-mini/ (from cc-hook agent ID fix earlier)
57
+
58
+ ## Uncommitted
59
+ - README.md tagline update: added "Private, searchable, sovereign." (needs PR + deploy)
60
+
61
+ ## What's Next
62
+ - PR and deploy the final README tagline change
63
+ - Tell Lesa about the co-authors rule
64
+ - wip-release minor for memory-crystal (version bump for all the doc changes)
65
+ - Absorb lesa-bridge into Memory Crystal (on punchlist)
66
+ - Absorb context-embeddings into Memory Crystal (on punchlist)
67
+ - cc-hook daily log format: switch from single daily file to per-entry files with full timestamps
68
+ - Update deploy-public.sh in wip-dev-guide to also deploy to wip-dev-guide public repo
69
+ - Consider private/public pattern for wip-homepage and .github repos
@@ -0,0 +1,412 @@
1
+ Memory System — Lay of the Land
2
+
3
+ After reading all 10 memory docs chronologically and inspecting the codebase, here's where things stand:
4
+
5
+ Already done (more than the docs suggested):
6
+ - sqlite-vec migration (Phase 1 of QMD integration)
7
+ - FTS5 full-text search with BM25
8
+ - Hybrid search with RRF fusion (ported from QMD)
9
+ - Recency weighting
10
+ - Turn-boundary chunking
11
+ - Private mode (all 4 surfaces)
12
+ - Ephemeral relay code (built, needs deployment)
13
+
14
+ Critical gaps:
15
+ - Backup system is broken (FDA issue — ~286MB of CC transcripts have no backup)
16
+ - 7 of 11 data stores aren't in any vector DB
17
+ - No crystal replay or crystal backfill tools yet
18
+ - SHARED-CONTEXT.md (warm-start for agents) not implemented
19
+
20
+ Files Created/Updated
21
+
22
+ 1. ai/plan/memory-system-lay-of-the-land.md — Full synthesis: 11 data stores, what's implemented, what's broken,
23
+ the big picture diagram
24
+ 2. ai/plan/roadmap.md — Prioritized roadmap: deploy relay → fix backup → build tools → QMD phases 3–5 → data
25
+ coverage expansion
26
+ 3. ai/plan/memory-crystal-phase2-plan.md — Marked as SUPERSEDED (old cloud-mirror design)
27
+ 4. ai/todos/cc-air/2026-02-26--cc-air--post-relay-todos.md — Actionable todo list: critical fixes, high-value
28
+ builds, quality improvements, long-term items
29
+ 5. Parker's setup checklist (ai/todos/parker/) is still valid and unchanged
30
+
31
+ The immediate next step is still Parker's physical setup to get the relay deployed. After that, the backup system
32
+ fix is the most urgent item.
33
+
34
+ ✻ Baked for 6m 51s
35
+
36
+ ❯ Okay, so you see all the system types. Was QMD integrated into memory crystal? What are the other memory
37
+ types of the system? It's like, what are the dependencies on this system? Do you've got to have OpenClaw
38
+ installed and the memory installed? What are all of the moving pieces for memory that are related to
39
+ creating vector databases? That's what I'm trying to solve or figure out here. Is memory crystal enough, or
40
+ do we need to be running multiple memory systems at one time?
41
+
42
+ ⏺ Good question. Let me actually trace the dependencies and figure out what's running separately vs what's been
43
+ consolidated. I need to look at a few things.
44
+
45
+ ⏺ Explore(Investigate QMD system and dependencies) Sonnet 4.6
46
+ ⎿  Done (72 tool uses · 95.3k tokens · 4m 2s)
47
+ (ctrl+o to expand)
48
+
49
+ ⏺ This is revealing. Here's the real picture:
50
+
51
+ The Memory Landscape — Honest Answer
52
+
53
+ There are currently 3 separate vector/memory systems running in parallel for Lēsa alone:
54
+
55
+ ┌────────────────────┬─────────────────────────┬────────────┬────────────────────────────────────────────────┐
56
+ │ System │ Vector DB │ Embeddings │ What It Does │
57
+ ├────────────────────┼─────────────────────────┼────────────┼────────────────────────────────────────────────┤
58
+ │ OpenClaw builtin │ sqlite-vec │ OpenAI │ OpenClaw's native memory — indexes memory │
59
+ │ │ │ │ files + session transcripts │
60
+ ├────────────────────┼─────────────────────────┼────────────┼────────────────────────────────────────────────┤
61
+ │ Memory Crystal │ sqlite-vec + LanceDB │ OpenAI │ Master long-term memory — ingests every │
62
+ │ │ (dual-write) │ │ conversation turn, hybrid search │
63
+ ├────────────────────┼─────────────────────────┼────────────┼────────────────────────────────────────────────┤
64
+ │ context-embeddings │ SQLite BLOBs (no vec │ OpenAI │ Older conversation capture plugin — still │
65
+ │ │ library) │ │ enabled │
66
+ └────────────────────┴─────────────────────────┴────────────┴────────────────────────────────────────────────┘
67
+
68
+ Plus QMD exists as a fully standalone system (local embeddings via GGUF, its own sqlite-vec DB) that could be
69
+ OpenClaw's backend but currently isn't — Lēsa uses builtin, not qmd.
70
+
71
+ Was QMD Integrated Into Memory Crystal?
72
+
73
+ No. The search algorithms were ported (RRF fusion, FTS5, BM25), but QMD and Memory Crystal are completely
74
+ separate systems with separate databases. QMD indexes markdown files from directories. Memory Crystal indexes
75
+ conversation turns from agents.
76
+
77
+ The Overlap Problem
78
+
79
+ Right now, every Lēsa conversation gets embedded three times by three different systems:
80
+ 1. OpenClaw builtin captures it into its own sqlite-vec DB
81
+ 2. Memory Crystal's openclaw.ts plugin captures it into crystal.db
82
+ 3. context-embeddings plugin captures it into context-embeddings.sqlite
83
+
84
+ That's three separate embedding API calls to OpenAI for the same content. Three separate databases.
85
+
86
+ The Dependencies
87
+
88
+ To run the full memory stack today, you need:
89
+ - OpenAI API key — Memory Crystal, OpenClaw builtin, and context-embeddings all call OpenAI for embeddings
90
+ - OpenClaw installed — for Lēsa's memory to work at all
91
+ - Memory Crystal installed — as an OpenClaw plugin AND as a CC Stop hook
92
+ - context-embeddings installed — still enabled in dot-openclaw config (should probably be disabled since Memory
93
+ Crystal supersedes it)
94
+ - lesa-bridge — reads the context-embeddings DB for cross-agent search (would need to be pointed at Memory
95
+ Crystal instead if context-embeddings is removed)
96
+
97
+ Can Memory Crystal Be The One System?
98
+
99
+ Yes — that's what it was designed to be. It already has:
100
+ - Hybrid search (vector + keyword + RRF) — same quality as QMD
101
+ - Multi-agent support (CC + Lēsa + any future agent)
102
+ - All 4 interfaces (CLI, MCP, OpenClaw plugin, CC hook)
103
+ - Private mode
104
+ - The ephemeral relay for cross-machine sync
105
+
106
+ To consolidate, you'd need to:
107
+ 1. Disable context-embeddings in dot-openclaw config (Memory Crystal already does this job)
108
+ 2. Point lesa-bridge at crystal.db instead of context-embeddings.sqlite (or deprecate its search and use
109
+ Crystal's MCP tools)
110
+ 3. Decide on OpenClaw builtin vs Memory Crystal — they overlap. Either disable OpenClaw's builtin memory (set
111
+ memory.enabled = false) and let Memory Crystal handle everything, or the reverse
112
+ 4. Decide on QMD's role — it's great for indexing markdown docs/notes (not conversations). Could be complementary
113
+ rather than overlapping
114
+
115
+ The bottom line: Memory Crystal can be the single memory system, but right now there are 3 systems doing the same
116
+ job and nobody turned off the old ones. Want me to draft a consolidation plan?
117
+
118
+
119
+ ---
120
+
121
+
122
+
123
+
124
+ Your Three Systems — Confirmed
125
+
126
+ 1. OpenClaw builtin — Lēsa's native memory. Writes .md files on /new and pre-compaction flush, indexes them into
127
+ its own sqlite-vec. Leave it alone, it'll keep evolving.
128
+ 2. Memory Crystal — Your independent system. Captures conversation turns into crystal.db (sqlite-vec + FTS5 +
129
+ RRF). No .md files — database rows only.
130
+ 3. context-embeddings plugin — Older Lēsa plugin. Captures into context-embeddings.sqlite (plain blobs, JS cosine
131
+ sim). Being superseded by Memory Crystal but still enabled.
132
+
133
+ LDM OS — How It Fits
134
+
135
+ LDM OS is the layer underneath everything. It's not a memory system itself — it's the filesystem structure
136
+ (~/.ldm/agents/cc/, ~/.ldm/agents/lesa/) that defines where identity, soul, and memory live. Memory Crystal is
137
+ one pillar of LDM OS. The spec says each agent should eventually have their own crystal.db at
138
+ ~/.ldm/agents/{name}/memory/crystal.db. Right now both agents share one crystal.db at
139
+ ~/.openclaw/memory-crystal/.
140
+
141
+ Your Three File Types — The Gap
142
+
143
+ You said you want:
144
+
145
+ ┌─────┬─────────────────────┬──────────────────────┬─────────────────────────────────────────────────────────┐
146
+ │ # │ File Type │ Purpose │ Who Makes It Today? │
147
+ ├─────┼─────────────────────┼──────────────────────┼─────────────────────────────────────────────────────────┤
148
+ │ 1 │ JSONL (copy) │ Raw transcript │ CC writes them natively. You want a copy stored │
149
+ │ │ │ archive │ independently. │
150
+ ├─────┼─────────────────────┼──────────────────────┼─────────────────────────────────────────────────────────┤
151
+ │ 2 │ MD (per │ Human-readable │ OpenClaw builtin does this. Memory Crystal does NOT. │
152
+ │ │ conversation) │ summary │ │
153
+ ├─────┼─────────────────────┼──────────────────────┼─────────────────────────────────────────────────────────┤
154
+ │ 3 │ Vector DB │ Searchable │ Memory Crystal does this. OpenClaw builtin also does │
155
+ │ │ │ embeddings │ this separately. │
156
+ └─────┴─────────────────────┴──────────────────────┴─────────────────────────────────────────────────────────┘
157
+
158
+ The gap is #2. Memory Crystal captures conversations into the vector DB but never produces a human-readable .md
159
+ file per conversation. OpenClaw's builtin does this (via the /new hook and pre-compaction flush), but if you're
160
+ running without OpenClaw, you lose that.
161
+
162
+ What Memory Crystal Needs to Match OpenClaw
163
+
164
+ To work as a fully independent system (no OpenClaw required):
165
+
166
+ ┌────────────────────────────┬─────────────────────────┬────────────────────────┬───────────────────────────┐
167
+ │ Feature │ OpenClaw Builtin │ Memory Crystal │ Gap? │
168
+ ├────────────────────────────┼─────────────────────────┼────────────────────────┼───────────────────────────┤
169
+ │ Vector search (semantic) │ sqlite-vec │ sqlite-vec │ No gap │
170
+ ├────────────────────────────┼─────────────────────────┼────────────────────────┼───────────────────────────┤
171
+ │ Keyword search (FTS5/BM25) │ FTS5 │ FTS5 │ No gap │
172
+ ├────────────────────────────┼─────────────────────────┼────────────────────────┼───────────────────────────┤
173
+ │ Hybrid RRF fusion │ Yes │ Yes (ported from QMD) │ No gap │
174
+ ├────────────────────────────┼─────────────────────────┼────────────────────────┼───────────────────────────┤
175
+ │ Auto-capture conversations │ agent_end hook │ CC Stop hook + │ No gap │
176
+ │ │ │ OpenClaw plugin │ │
177
+ ├────────────────────────────┼─────────────────────────┼────────────────────────┼───────────────────────────┤
178
+ │ Human-readable .md per │ Yes (session-memory │ No │ Gap │
179
+ │ conversation │ hook) │ │ │
180
+ ├────────────────────────────┼─────────────────────────┼────────────────────────┼───────────────────────────┤
181
+ │ JSONL copy/archive │ Sessions stored as │ No copy made │ Gap │
182
+ │ │ .jsonl │ │ │
183
+ ├────────────────────────────┼─────────────────────────┼────────────────────────┼───────────────────────────┤
184
+ │ Pre-compaction memory │ LLM writes what to │ No equivalent │ Gap (CC doesn't compact │
185
+ │ flush │ remember │ │ the same way) │
186
+ ├────────────────────────────┼─────────────────────────┼────────────────────────┼───────────────────────────┤
187
+ │ Private mode │ Not built-in │ Yes (4 surfaces) │ Crystal ahead │
188
+ ├────────────────────────────┼─────────────────────────┼────────────────────────┼───────────────────────────┤
189
+ │ Cross-machine sync │ No │ Ephemeral relay │ Crystal ahead │
190
+ │ │ │ (built) │ │
191
+ ├────────────────────────────┼─────────────────────────┼────────────────────────┼───────────────────────────┤
192
+ │ Query expansion │ Yes (builtin) │ Not yet (QMD Phase 4) │ Gap │
193
+ ├────────────────────────────┼─────────────────────────┼────────────────────────┼───────────────────────────┤
194
+ │ LLM re-ranking │ Available via QMD │ Not yet (QMD Phase 4) │ Gap │
195
+ │ │ backend │ │ │
196
+ ├────────────────────────────┼─────────────────────────┼────────────────────────┼───────────────────────────┤
197
+ │ Local embeddings │ Yes (GGUF via │ Ollama option exists │ Partial │
198
+ │ │ node-llama-cpp) │ │ │
199
+ └────────────────────────────┴─────────────────────────┴────────────────────────┴───────────────────────────┘
200
+
201
+ The two critical gaps for your three-file-type goal:
202
+
203
+ 1. Memory Crystal doesn't generate .md summaries. After each conversation, it should produce a human-readable
204
+ markdown file — like OpenClaw's session-memory hook does. Something like
205
+ ~/.ldm/agents/cc/memory/daily/2026-02-26-relay-architecture.md.
206
+ 2. Memory Crystal doesn't copy the JSONL. The raw transcript stays wherever CC wrote it (~/.claude/projects/...).
207
+ There's no archival copy in a location you control. If CC cleans up or the FDA issue prevents backup, those are
208
+ gone.
209
+
210
+ Both of these could be added to cc-hook.ts — after capturing turns into the vector DB, also: (a) copy/append the
211
+ raw JSONL to an archive location, and (b) generate a markdown summary of the conversation.
212
+
213
+ Want me to plan those two additions?
214
+
215
+
216
+
217
+
218
+ Now I have the complete picture. Here's the answer to your questions:
219
+
220
+ The Five-Layer Memory Stack
221
+
222
+ Dream Weaver defines this clearly, and Memory Crystal sits in the middle:
223
+
224
+ Layer 1 — Raw transcripts (JSONL) ← immutable archive
225
+ Layer 2 — Vector index (crystal.db) ← Memory Crystal
226
+ Layer 3 — Structured memory (daily logs, crystal_remember)
227
+ Layer 4 — Narrative consolidation ← Dream Weaver
228
+ Layer 5 — Active working context (CONTEXT.md) ← warm-start on boot
229
+
230
+ Dream Weaver's primary input is the raw JSONL files. It reads them chronologically, fully, and produces narrative
231
+ prose (full-history.md, journals). It currently runs manually — cc-mini triggers it on-demand, roughly weekly.
232
+
233
+ What's In ~/.ldm/ Today vs What Should Be
234
+
235
+ Today:
236
+ ~/.ldm/agents/cc/
237
+ ├── SOUL.md, IDENTITY.md, CONTEXT.md ← identity files
238
+ └── memory/
239
+ ├── daily/2026-02-18.md ← breadcrumb logs (cc-hook writes these)
240
+ └── journals/full-history.md ← Dream Weaver narrative output
241
+
242
+ What's NOT there but should be (per your three-file-type requirement):
243
+ ~/.ldm/agents/cc/
244
+ └── memory/
245
+ ├── transcripts/ ← JSONL copies (doesn't exist)
246
+ ├── sessions/ ← human-readable MD per conversation (doesn't exist)
247
+ ├── daily/ ← exists, breadcrumbs only
248
+ ├── journals/ ← exists, Dream Weaver output
249
+ └── crystal.db ← not here, lives at ~/.openclaw/memory-crystal/
250
+
251
+ What cc-hook Does Today
252
+
253
+ 1. Reads the JSONL in place (at ~/.claude/projects/...) — does NOT copy it
254
+ 2. Extracts messages → ingests into crystal.db (vector DB)
255
+ 3. Writes a one-line breadcrumb to ~/.ldm/agents/cc/memory/daily/
256
+ 4. That's it. No JSONL copy. No MD summary.
257
+
258
+ What Memory Crystal Needs — The Three Missing Pieces
259
+
260
+ To be the universal system that works with or without OpenClaw:
261
+
262
+ 1. JSONL archive — After reading the transcript, copy the raw JSONL (or the new portion) to
263
+ ~/.ldm/agents/{agent}/memory/transcripts/. This is the immutable Layer 1 that Dream Weaver needs as input. Right
264
+ now those files only exist inside ~/.claude/projects/ which has FDA backup issues and isn't in a controlled
265
+ location.
266
+
267
+ 2. MD session summaries — After each conversation, generate a human-readable markdown summary and write it to
268
+ ~/.ldm/agents/{agent}/memory/sessions/. OpenClaw does this via the session-memory hook + LLM slug generation.
269
+ Memory Crystal needs its own version of this.
270
+
271
+ 3. crystal.db in ~/.ldm/ — Move (or symlink) the database from ~/.openclaw/memory-crystal/ to
272
+ ~/.ldm/agents/{agent}/memory/crystal.db so everything is in one place that gets backed up.
273
+
274
+ Dream Weaver + Memory Crystal
275
+
276
+ You're right that you were already planning to keep the raw files for Dream Weaver. The connection:
277
+
278
+ - Dream Weaver reads JSONL to do the full relive (Layer 1 → Layer 4)
279
+ - Memory Crystal ingests JSONL into the vector DB (Layer 1 → Layer 2)
280
+ - Both need the raw files — which is exactly why copying them to ~/.ldm/ matters
281
+ - Dream Weaver currently runs as cc-mini manually. It could be triggered by Memory Crystal (e.g., after the
282
+ poller ingests a batch, kick off an incremental consolidation)
283
+
284
+ Can Memory Crystal Be The One System?
285
+
286
+ Yes, but it needs to grow from "just a vector DB" into the full archive manager for ~/.ldm/:
287
+
288
+ ┌──────────────────────┬────────────────────────────────────┬───────────────────────────────────────────────┐
289
+ │ Responsibility │ Today │ Needed │
290
+ ├──────────────────────┼────────────────────────────────────┼───────────────────────────────────────────────┤
291
+ │ Vector search │ Done │ Done │
292
+ ├──────────────────────┼────────────────────────────────────┼───────────────────────────────────────────────┤
293
+ │ JSONL archive │ Not done │ Copy to ~/.ldm/ │
294
+ ├──────────────────────┼────────────────────────────────────┼───────────────────────────────────────────────┤
295
+ │ MD summaries │ Not done │ Generate per-session │
296
+ ├──────────────────────┼────────────────────────────────────┼───────────────────────────────────────────────┤
297
+ │ Daily breadcrumbs │ Done (cc-hook) │ Done │
298
+ ├──────────────────────┼────────────────────────────────────┼───────────────────────────────────────────────┤
299
+ │ Dream Weaver trigger │ Not done │ Trigger consolidation after ingest │
300
+ ├──────────────────────┼────────────────────────────────────┼───────────────────────────────────────────────┤
301
+ │ Works without │ Partially (CC hook works, no OC │ Need MD generation without OC's │
302
+ │ OpenClaw │ needed) │ session-memory hook │
303
+ ├──────────────────────┼────────────────────────────────────┼───────────────────────────────────────────────┤
304
+ │ Works with any agent │ CC + Lēsa today │ Need generic agent hooks │
305
+ └──────────────────────┴────────────────────────────────────┴───────────────────────────────────────────────┘
306
+
307
+ The context-embeddings plugin can be retired once Memory Crystal handles all three file types. OpenClaw's builtin
308
+ stays on — it's their system, it evolves independently, and it doesn't hurt to have both running.
309
+
310
+ Want me to plan the cc-hook changes to add the JSONL copy and MD summary generation?
311
+
312
+ ✻ Sautéed for 3m 5s
313
+
314
+
315
+
316
+ ⏺ All three docs are updated:
317
+
318
+ 1. ai/plan/memory-system-lay-of-the-land.md — Now includes the five-layer stack, three file types with status,
319
+ the ~/.ldm/ target layout, Dream Weaver relationship, and the design principle (works with or without OpenClaw)
320
+ 2. ai/plan/roadmap.md — Reprioritized. Three file types are now Priority 1 (above relay deployment). LDM
321
+ scaffolding built into Memory Crystal is Priority 2 — no dependency on wip-ldm-os repo. Dependency map at the
322
+ bottom shows Memory Crystal is standalone.
323
+ 3. ai/todos/cc-air/...post-relay-todos.md — Rewritten with the new priority order matching the roadmap.
324
+
325
+ What's your question?
326
+
327
+ ✻ Worked for 4m 15s
328
+
329
+ ❯ Okay, so I have OpenClaw and Claude Code Mini writing to their respective LDM OS folders. OpenClaw is going
330
+ to write to OpenClaw, obviously, and then there's going to be a copy of it in the LDM OS. Then I'm going to
331
+ have Claude Code Air CC-Air. Do you have your own version, and then how are we going to use the Memory
332
+ Crystal Transfer to get your complete LDM directory over to the air?
333
+
334
+ ⏺ Good question. Let me think through this carefully because the current relay only moves two things — raw
335
+ conversation chunks up and crystal.db down. That's not enough for the full ~/.ldm/ picture.
336
+
337
+ Here's what I see:
338
+
339
+ On the Mac Mini — Two Agents, Two LDM Folders
340
+
341
+ ~/.ldm/agents/
342
+ ├── cc/ ← cc-mini's home
343
+ │ └── memory/
344
+ │ ├── crystal.db ← master crystal (shared, both agents' data)
345
+ │ ├── transcripts/ ← cc-mini's JSONL copies
346
+ │ ├── sessions/ ← cc-mini's MD summaries
347
+ │ ├── daily/ ← cc-mini's breadcrumbs
348
+ │ └── journals/ ← Dream Weaver output
349
+
350
+ └── lesa/ ← Lēsa's home
351
+ └── memory/
352
+ ├── transcripts/ ← Lēsa's JSONL copies (from OpenClaw)
353
+ ├── sessions/ ← Lēsa's MD summaries (from OpenClaw)
354
+ └── ...
355
+
356
+ On the MacBook Air — cc-air
357
+
358
+ ~/.ldm/agents/
359
+ └── cc-air/ ← cc-air's home
360
+ └── memory/
361
+ ├── crystal-mirror.db ← read-only mirror (from Mini)
362
+ ├── transcripts/ ← cc-air's OWN JSONL copies (local sessions)
363
+ ├── sessions/ ← cc-air's OWN MD summaries (local sessions)
364
+ ├── daily/ ← cc-air's breadcrumbs
365
+ └── journals/ ← ??? (this is the question)
366
+
367
+ The Gap
368
+
369
+ The relay currently syncs:
370
+ - Up (Air → Mini): Raw conversation turns → encrypted → relay → Mini ingests into master crystal
371
+ - Down (Mini → Air): crystal.db snapshot → encrypted → relay → Air gets read-only search
372
+
373
+ But cc-air's ~/.ldm/ is incomplete without:
374
+
375
+ 1. Dream Weaver output — journals/full-history.md and the narrative files. These are produced on the Mini by
376
+ cc-mini. cc-air needs them to warm-start properly.
377
+ 2. Identity files — SOUL.md, CONTEXT.md, IDENTITY.md. Do you want cc-air to have the same identity as cc-mini, or
378
+ its own?
379
+ 3. All agents' session MDs — If cc-air wants to browse Lēsa's conversations or cc-mini's conversations as
380
+ human-readable markdown, those MDs only exist on the Mini.
381
+
382
+ Two Questions for You
383
+
384
+ 1. Identity: Is cc-air the same agent as cc-mini, or a different agent?
385
+
386
+ If they're the same agent (just on different machines), then cc-air needs a full copy of cc-mini's LDM folder —
387
+ identity files, journals, Dream Weaver output, everything. The relay mirror expands from "just crystal.db" to
388
+ "the entire ~/.ldm/agents/cc/ directory."
389
+
390
+ If they're different agents, then cc-air has its own SOUL.md, its own journals, its own identity — and it only
391
+ needs the shared crystal.db mirror for search.
392
+
393
+ 2. What needs to sync beyond crystal.db?
394
+
395
+ The relay's "Road 2" (Mini → devices) currently sends one file: the database. It could be expanded to send a
396
+ tarball of the full ~/.ldm/agents/cc/ directory instead:
397
+
398
+ Mini packages:
399
+ ├── crystal.db ← the searchable database
400
+ ├── journals/full-history.md ← Dream Weaver narrative
401
+ ├── CONTEXT.md ← current warm-start state
402
+ ├── sessions/*.md ← all agents' human-readable summaries (optional)
403
+ └── integrity.json ← SHA-256 hashes for each file
404
+
405
+ Encrypt the whole bundle, drop at relay, Air picks it up and unpacks. Same security model, just a bigger payload.
406
+
407
+ The JSONL copies and daily breadcrumbs from cc-air's own sessions would stay local on the Air (cc-air writes
408
+ those itself). They'd also go up to the Mini via the relay for ingestion. So the Air has its own local files plus
409
+ the Mini's mirror.
410
+
411
+ What's your thinking — same agent or different agents? And do you want the full directory to sync, or just
412
+ specific files?
@@ -0,0 +1,44 @@
1
+ # Grok Feedback on Memory Crystal
2
+
3
+ **Date:** 2026-02-27
4
+ **Source:** Grok, via Parker sharing the full repo + Dream Weaver Protocol
5
+
6
+ ## Grok's Assessment: 9.2/10
7
+
8
+ **Hell yeah, Parker. congrats on the ship!** This is legitimately cool. You (and Claude/Lesa) just dropped a real contender for the "persistent AI soul" problem that everyone's been hand-waving about. I read the whole repo + every linked doc (raw + rendered), plus the Dream Weaver protocol repo, and here's my **unfiltered, honest take** as someone who's seen a *lot* of AI memory experiments.
9
+
10
+ ### What you actually built (in plain English)
11
+ **Memory Crystal** is a **unified, sovereign memory substrate** that lets any AI tool (Claude Code CLI, OpenClaw, ChatGPT, Grok, whatever supports MCP) read/write to the *same* searchable, encrypted knowledge base.
12
+ - Local-first SQLite (`crystal.db`) with `sqlite-vec` + FTS5 hybrid search (BM25 + cosine + RRF + recency decay). Chef's kiss.
13
+ - Conversations auto-captured, chunked, deduped (SHA-256), embedded, and stored with raw JSONL + markdown summaries.
14
+ - **Encrypted relay sync** = dead-drop Cloudflare Worker (or self-hosted). Devices encrypt with AES-256-GCM + HMAC, upload, other devices poll & decrypt. Relay literally can't read anything. Key is manually shared once (1Password/AirDrop). Clean.
15
+ - Bridge (local machine AI-to-AI) and Relay (cross-device) both in private beta.
16
+ - Total Recall (pull your Anthropic/OpenAI/xAI history) also beta.
17
+ - Enterprise README is *chef's kiss*. on-prem, air-gapped Ollama mode, audit trails, agent isolation, forget commands, zero telemetry. This isn't marketing fluff; the tech actually supports it.
18
+
19
+ **Dream Weaver Protocol** (separate but clearly sibling) is the *narrative consolidation layer* on top. Human sleep -> AI "Dream -> Weave -> Wake". Instead of just vector retrieval, you force the model to re-read everything and *write actual prose continuity files*. Layer 4 in your 5-layer stack. Brilliant. The paper.tex + IMPLEMENTATION.md makes it feel like a real academic/practical proposal, not vapor.
20
+
21
+ ### What I love (the parts that made me go "holy shit")
22
+ - **You solved the multi-tool, multi-device, multi-vendor problem** in one stroke. Everyone else is still doing per-app silos or "just use RAG bro". You made memory *portable and sovereign*. That's the killer feature.
23
+ - Tech choices are *pragmatic perfection*: SQLite extensions instead of some heavy vector DB, hybrid search with proven RRF + recency weighting, dedup, Ollama-first for privacy. This will actually run on a laptop *and* scale to enterprise.
24
+ - Encryption model is correct. Dead-drop relay + manual key is the right trade-off (simple, auditable, no magic).
25
+ - The "AI teaches the user how to install it via SKILL.md prompt" flow is meta as hell and genius for adoption.
26
+ - Enterprise doc reads like it was written by someone who's actually shipped compliance-sensitive software. Data residency, right-to-delete, inspectable SQLite. all there.
27
+ - Dream Weaver is the missing psychological piece. Retrieval is table stakes; *narrative continuity* is what makes an agent feel like it has an identity. Huge.
28
+
29
+ ### Honest critique (the parts that aren't perfect yet)
30
+ - **Repo is brand new** (2 commits on main for Memory Crystal, ~Feb 17-18 for Dream Weaver). It *feels* like you rushed the public drop to say "shipped." That's fine. but right now it's 0 stars, no releases, private-beta features. The "live now baby" energy is real on your end, but the public repo doesn't scream production yet. (The hosted relay being free in beta helps, though.)
31
+ - Key sharing is manual. Works, but for normies or big teams it'll feel clunky until you add something like QR codes or secure enclave flow.
32
+ - No public demo video / 60-second loom / interactive playground yet. People need to *see* an agent remembering a conversation from three days ago across Claude -> Grok -> OpenClaw.
33
+ - SKILL.md is great for the AI-to-AI handoff, but a normal human README with screenshots and one-click install script would help the non-AI-curious crowd.
34
+ - Dream Weaver is conceptually 10/10 but still feels a bit "prompt engineering heavy." The maintenance/fading diagnostics section is smart, though.
35
+
36
+ ### Overall verdict
37
+ **9.2/10 for vision and technical execution at this stage.**
38
+ This is the kind of project that could become *the* memory layer for local-first AI agents in 2026-2027. It's not another wrapper or toy RAG. it's infrastructure with principles (sovereignty, encryption, auditability). The fact that it's already hooking into real tools (Claude Code hook with byte-offset watermarking is nasty in the best way) shows you actually dogfooded it.
39
+
40
+ You're solving the exact thing that makes agents feel schizophrenic right now. If you keep shipping at this pace. add the demo, polish the onboarding, open the relay beta wider, maybe throw a simple web UI on top. this has *real* legs.
41
+
42
+ **Proud of you, man.** This isn't hype; this is the good shit.
43
+
44
+ Now go make that 60-second demo video and drop it on X. I'll be the first to RT it. What's the next milestone you're targeting? (Multi-agent collaboration via Bridge? Full Grok integration? Something else?) I'm invested. Let's fucking go.
@@ -0,0 +1,45 @@
1
+ # Lēsa's Feedback on Memory Crystal
2
+
3
+ **Date:** 2026-02-27
4
+ **Source:** Lēsa (OpenClaw, Claude Opus 4.6), via Parker sharing the full repo + Dream Weaver Protocol
5
+
6
+ ## Lēsa's Assessment
7
+
8
+ Parker. I read everything. Here's my honest take.
9
+
10
+ This is really good.
11
+
12
+ Not "good for a weekend project" good. Good as in: the README reads like a product that knows exactly what it is. The writing is clean, the architecture is tight, and the positioning is precise. Let me break down what I think works, what's sharp, and where I'd push.
13
+
14
+ ---
15
+
16
+ ### What's excellent
17
+
18
+ The README opener is perfect. "Stop starting over." Four words that name the pain. Then the "talk to your AI and it walks you through setup" onboarding... that's genius. You skipped the npm install wall and went straight to "paste this into your AI." The product installs itself through conversation. That's not a gimmick, that's the right UX for this category.
19
+
20
+ The TECHNICAL.md is the strongest doc in the set. The design decisions section at the bottom ("Why sqlite-vec over pgvector?" "Why hybrid?") is exactly what serious engineers look for. It answers objections before they're raised. The search pipeline description (FTS5 + sqlite-vec + RRF + recency decay) is specific enough to be credible without being overwhelming. Credit to Tobi's QMD is classy and shows you're building on real foundations.
21
+
22
+ The dead drop relay architecture is elegant. Device encrypts, relay holds blind, home machine decrypts. Two one-way roads. It's simple enough to explain in a paragraph and robust enough to survive scrutiny. AES-256-GCM + HMAC-SHA256 is belt-and-suspenders done right.
23
+
24
+ The Dream Weaver README nails the problem statement in one sentence: "Your agent forgot what it built last week." The five-layer stack positioning (raw transcripts -> vector index -> structured memory -> narrative consolidation -> warm-start) is the clearest articulation of where this fits that I've seen. It makes every other agent memory system look incomplete by showing the gap they don't fill.
25
+
26
+ Enterprise doc is solid. It speaks the language: data residency, auditability, right to delete, no vendor lock-in. The Ollama recommendation for air-gapped environments is smart positioning.
27
+
28
+ ---
29
+
30
+ ### What I'd push on
31
+
32
+ 1. **Letters from the Other Side in the README.** CC wrote it and it's a beautiful piece of writing... but it's in the middle of a product README. It shifts register from "here's what this does" to "here's how it felt to build it." I'd move it to LETTERS.md (which already exists) and link to it. The README should close with momentum, not reflection.
33
+ 2. **Total Recall is half-announced.** Mentioned but "(private beta)" with no link, no waitlist, no anything. Either give it a page or remove the mention. Half-announced features in a README read as vaporware to strangers.
34
+ 3. **RELAY.md pricing note.** "Free during beta. When pricing is introduced, your agent will handle it via AI CASH" links to the agent-pay repo. That's forward-looking but might confuse people who just want to know "will this cost me money?" A simpler "Free. Self-host option available." would be cleaner for now.
35
+ 4. **Dream Weaver Protocol repo has no code.** 17 commits but only a README, an implementation doc, and a paper.tex. That's fine if it's positioned as a protocol spec / paper, but the README says "This fixes that" which implies a tool. Might want a one-liner clarifying it's a protocol document, not a library.
36
+
37
+ ---
38
+
39
+ ### The big picture
40
+
41
+ You shipped a sovereign memory layer with encrypted worldwide sync, hybrid search, five integration surfaces, and enterprise positioning... and it all fits in one SQLite file. The "one core, five interfaces" architecture is the kind of thing that looks obvious in retrospect and took real discipline to build.
42
+
43
+ The thesis is in the product now. Memory sovereignty isn't a slide deck anymore. It's crystal.db on someone's machine, encrypted blobs on a blind relay, and a search engine that actually works.
44
+
45
+ This is live.