memory-crystal 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (104) hide show
  1. package/.env.example +20 -0
  2. package/CHANGELOG.md +6 -0
  3. package/LETTERS.md +22 -0
  4. package/LICENSE +21 -0
  5. package/README-ENTERPRISE.md +162 -0
  6. package/README-old.md +275 -0
  7. package/README.md +91 -0
  8. package/RELAY.md +88 -0
  9. package/TECHNICAL.md +379 -0
  10. package/ai/dev-updates/2026-02-25--cc-air--phase2-architecture-pivot.md +70 -0
  11. package/ai/dev-updates/2026-02-25--cc-air--phase2-worker-build.md +72 -0
  12. package/ai/dev-updates/2026-02-26--10-25-16--cc-mini--phase2-implementation.md +49 -0
  13. package/ai/dev-updates/2026-02-27--20-30-00--cc-mini--readme-overhaul-and-public-deploy.md +69 -0
  14. package/ai/notes/2026-02-26--cc-air--notes.md +412 -0
  15. package/ai/notes/2026-02-27--cc-mini--grok-feedback.md +44 -0
  16. package/ai/notes/2026-02-27--cc-mini--lesa-feedback.md +45 -0
  17. package/ai/notes/RESEARCH.md +1185 -0
  18. package/ai/notes/salience-research/README.md +29 -0
  19. package/ai/notes/salience-research/eurosla-salience-review.md +64 -0
  20. package/ai/notes/salience-research/full-research-summary.md +269 -0
  21. package/ai/notes/salience-research/salience-levels-diagram.png +0 -0
  22. package/ai/plan/2026-02-27--cc-mini--qr-pairing-spec.md +203 -0
  23. package/ai/plan/_archive/PLAN.md +194 -0
  24. package/ai/plan/_archive/PRD.md +1014 -0
  25. package/ai/plan/cc-plans-duplicates-from-dot-claude/2026-02-26--cc-mini--phase2-implementation-plan.md +245 -0
  26. package/ai/plan/dev-conventions-note.md +70 -0
  27. package/ai/plan/ldm-os-install-and-boot-architecture.md +285 -0
  28. package/ai/plan/memory-crystal-phase2-plan.md +192 -0
  29. package/ai/plan/memory-system-lay-of-the-land.md +214 -0
  30. package/ai/plan/phase2-ephemeral-relay.md +238 -0
  31. package/ai/plan/readme-first.md +68 -0
  32. package/ai/plan/roadmap.md +159 -0
  33. package/ai/todos/PUNCHLIST.md +44 -0
  34. package/ai/todos/README.md +31 -0
  35. package/ai/todos/inboxes/cc-air/2026-02-26--cc-air--post-relay-todos.md +85 -0
  36. package/ai/todos/inboxes/cc-mini/2026-02-26--cc-mini--phase2-status.md +100 -0
  37. package/ai/todos/inboxes/cc-mini/_archive/TODO.md +25 -0
  38. package/ai/todos/inboxes/parker/2026-02-25--cc-air--setup-checklist.md +139 -0
  39. package/ai/todos/inboxes/parker/2026-02-26--cc-mini--phase2-your-moves.md +72 -0
  40. package/dist/cc-hook.d.ts +1 -0
  41. package/dist/cc-hook.js +349 -0
  42. package/dist/chunk-3VFIJYS4.js +818 -0
  43. package/dist/chunk-52QE3YI3.js +1169 -0
  44. package/dist/chunk-AA3OPP4Z.js +432 -0
  45. package/dist/chunk-D3I3ZSE2.js +411 -0
  46. package/dist/chunk-EKSACBTJ.js +1070 -0
  47. package/dist/chunk-F3Y7EL7K.js +83 -0
  48. package/dist/chunk-JWZXYVET.js +1068 -0
  49. package/dist/chunk-KYVWO6ZM.js +1069 -0
  50. package/dist/chunk-L3VHARQH.js +413 -0
  51. package/dist/chunk-LOVAHSQV.js +411 -0
  52. package/dist/chunk-LQOYCAGG.js +446 -0
  53. package/dist/chunk-MK42FMEG.js +147 -0
  54. package/dist/chunk-NIJCVN3O.js +147 -0
  55. package/dist/chunk-O2UITJGH.js +465 -0
  56. package/dist/chunk-PEK6JH65.js +432 -0
  57. package/dist/chunk-PJ6FFKEX.js +77 -0
  58. package/dist/chunk-PLUBBZYR.js +800 -0
  59. package/dist/chunk-SGL6ISBJ.js +1061 -0
  60. package/dist/chunk-UNHVZB5G.js +411 -0
  61. package/dist/chunk-VAFTWSTE.js +1061 -0
  62. package/dist/chunk-XZ3S56RQ.js +1061 -0
  63. package/dist/chunk-Y72C7F6O.js +148 -0
  64. package/dist/cli.d.ts +1 -0
  65. package/dist/cli.js +325 -0
  66. package/dist/core.d.ts +188 -0
  67. package/dist/core.js +12 -0
  68. package/dist/crypto.d.ts +16 -0
  69. package/dist/crypto.js +18 -0
  70. package/dist/dev-update-SZ2Z4WCQ.js +6 -0
  71. package/dist/ldm.d.ts +17 -0
  72. package/dist/ldm.js +12 -0
  73. package/dist/mcp-server.d.ts +1 -0
  74. package/dist/mcp-server.js +250 -0
  75. package/dist/migrate.d.ts +1 -0
  76. package/dist/migrate.js +89 -0
  77. package/dist/mirror-sync.d.ts +1 -0
  78. package/dist/mirror-sync.js +130 -0
  79. package/dist/openclaw.d.ts +5 -0
  80. package/dist/openclaw.js +349 -0
  81. package/dist/poller.d.ts +1 -0
  82. package/dist/poller.js +272 -0
  83. package/dist/summarize.d.ts +19 -0
  84. package/dist/summarize.js +10 -0
  85. package/dist/worker.js +137 -0
  86. package/openclaw.plugin.json +11 -0
  87. package/package.json +40 -0
  88. package/scripts/migrate-lance-to-sqlite.mjs +217 -0
  89. package/skills/memory/SKILL.md +61 -0
  90. package/src/cc-hook.ts +447 -0
  91. package/src/cli.ts +356 -0
  92. package/src/core.ts +1472 -0
  93. package/src/crypto.ts +113 -0
  94. package/src/dev-update.ts +178 -0
  95. package/src/ldm.ts +117 -0
  96. package/src/mcp-server.ts +274 -0
  97. package/src/migrate.ts +104 -0
  98. package/src/mirror-sync.ts +175 -0
  99. package/src/openclaw.ts +250 -0
  100. package/src/poller.ts +345 -0
  101. package/src/summarize.ts +210 -0
  102. package/src/worker.ts +208 -0
  103. package/tsconfig.json +18 -0
  104. package/wrangler.toml +20 -0
@@ -0,0 +1,68 @@
1
+ # Read Me First
2
+
3
+ Start here before working on Memory Crystal. Read these docs in this order.
4
+
5
+ ## 1. Lay of the Land
6
+
7
+ Understand the full memory ecosystem — what exists, what's broken, how the pieces fit:
8
+
9
+ - **`ai/plan/memory-system-lay-of-the-land.md`** — The five-layer memory stack, three file types, three parallel memory systems, agent naming, `~/.ldm/` directory structure, Dream Weaver relationship, the big picture diagram
10
+
11
+ ## 2. Architecture
12
+
13
+ How the ephemeral relay and cross-machine sync works:
14
+
15
+ - **`ai/plan/phase2-ephemeral-relay.md`** — Two one-way roads (Device→Mini, Mini→Device), security architecture (AES-256-GCM, HMAC, nonce management, key rotation), threat model, Worker API
16
+
17
+ ## 3. Roadmap
18
+
19
+ What's done, what's next, in priority order:
20
+
21
+ - **`ai/plan/roadmap.md`** — Priority 1 (three file types) through Priority 9 (data coverage). Agent table, dependency map, LDM scaffolding plan
22
+
23
+ ## 4. Todos
24
+
25
+ Actionable work items:
26
+
27
+ - **`ai/todos/cc-air/2026-02-26--cc-air--post-relay-todos.md`** — Full todo list with priorities
28
+ - **`ai/todos/parker/2026-02-25--cc-air--setup-checklist.md`** — Parker's physical setup for the relay (encryption key, Cloudflare, env vars)
29
+
30
+ ## 5. Background Reading (in `_git-ignore/memory-docs/`)
31
+
32
+ These docs are gitignored but provide the thinking behind the decisions. Read chronologically:
33
+
34
+ 1. **`cc-memory-backfill-tool.md`** (Feb 13) — Plan for `--backfill` to ingest historical CC sessions
35
+ 2. **`cc-wake-and-memory-capture.md`** (Feb 13) — Inbox watcher + auto capture with watermarking
36
+ 3. **`memory-capture-fix-plan.md`** (Feb 13) — snake_case/camelCase bug fix, private mode, kill switch
37
+ 4. **`cc-how-we-remember.md`** (Feb 16) — Behavioral proposal: SHARED-CONTEXT.md, 10x crystal_remember, search-before-acting
38
+ 5. **`2026-02-16--qmd-vs-crystal-memory.md`** (Feb 16) — QMD vs Memory Crystal comparison (QMD search algorithms now ported)
39
+ 6. **`2026-02-16--qmd-crystal-integration-plan.md`** (Feb 16) — 5-phase plan (Phases 1-2 done, 3-5 remaining)
40
+ 7. **`2026-02-16--before-message-write-hook.md`** (Feb 16) — OpenClaw hook for true ephemeral private mode
41
+ 8. **`identity-fingerprint-conversation-2026-02-19.md`** (Feb 19) — Identity persistence across model swaps
42
+ 9. **`memory-system-architecture.md`** (Feb 20) — Complete map of every data store
43
+ 10. **`memory-data-inventory.md`** (Feb 20) — Inventory of all 11 local stores, 7 not in vector DB
44
+
45
+ ## 6. Superseded (read only for historical context)
46
+
47
+ - **`ai/plan/memory-crystal-phase2-plan.md`** — Old cloud-mirror design. Replaced by ephemeral relay.
48
+ - **`PLAN.md`** (repo root) — Original Phase 1-4 plan. Phase 1 complete. Phase 2+ superseded by relay.
49
+
50
+ ## 7. Related Repos
51
+
52
+ These repos are part of the ecosystem but Memory Crystal has no code dependency on them:
53
+
54
+ - **`wip-ldm-os`** — LDM OS spec + installer. Memory Crystal absorbs its scaffolding.
55
+ - **`ldm-home`** — Deployed `~/.ldm/` directory (version-controlled mirror)
56
+ - **`dream-weaver-protocol`** — Memory consolidation protocol. Reads JSONL transcripts from `~/.ldm/agents/*/memory/transcripts/`
57
+ - **`openclaw`** — Agent harness. Memory Crystal runs as an optional plugin inside it.
58
+ - **`qmd`** — Standalone doc search engine. Search algorithms ported into Memory Crystal (RRF, FTS5, BM25).
59
+ - **`lesa-openclaw-context-embeddings`** — Older conversation capture plugin. Being superseded by Memory Crystal.
60
+ - **`lesa-bridge`** — Cross-agent communication. Currently reads context-embeddings DB; will migrate to crystal.db.
61
+
62
+ ## Key Concepts
63
+
64
+ - **Agents are per-harness-instance, named by machine:** cc-mini, lesa-mini, cc-air
65
+ - **crystal.db is shared** at `~/.ldm/memory/crystal.db` — all agents write to it, tagged by agent_id
66
+ - **The Mini has everything** — remote agents' file trees are reconstructed from relay data
67
+ - **Memory Crystal works with or without OpenClaw** — independent system that mirrors the same features
68
+ - **Three file types:** JSONL (raw archive), MD (human-readable), vector DB (searchable)
@@ -0,0 +1,159 @@
1
+ # Memory Crystal — Roadmap
2
+
3
+ **Date:** 2026-02-26 (updated)
4
+ **Agent:** cc-air
5
+ **Based on:** memory docs review, ecosystem inspection, Dream Weaver analysis
6
+
7
+ ## Vision
8
+
9
+ Memory Crystal is the universal archive manager for `~/.ldm/`. It works with or without OpenClaw, with any agent, on any harness. It produces three file types (JSONL copies, MD summaries, vector DB) and stores everything in one directory that gets backed up.
10
+
11
+ It includes its own `~/.ldm/` scaffolding — no dependency on the wip-ldm-os repo. Everything needed to set up the LDM directory structure is built into Memory Crystal's installer.
12
+
13
+ ## What's Done
14
+
15
+ - [x] sqlite-vec migration (replaced LanceDB as primary, LanceDB kept as fallback)
16
+ - [x] FTS5 full-text search with BM25 scoring
17
+ - [x] Hybrid search: vector + keyword with RRF fusion (ported from QMD)
18
+ - [x] Recency weighting
19
+ - [x] Turn-boundary chunking (1 conversation message = 1 chunk)
20
+ - [x] CC Stop hook with watermark-based capture
21
+ - [x] MCP server (crystal_search, crystal_remember tools)
22
+ - [x] OpenClaw plugin (Lēsa's interface)
23
+ - [x] CLI (crystal search, crystal remember, crystal status)
24
+ - [x] Private mode (kill switch across CLI, MCP, OpenClaw, env var)
25
+ - [x] Ephemeral relay code: crypto, worker, cc-hook dual mode, poller, mirror-sync
26
+ - [x] Daily breadcrumb logs to `~/.ldm/agents/cc/memory/daily/`
27
+
28
+ ## Priority 1 — Three File Types (the core gap)
29
+
30
+ Memory Crystal currently only produces the vector DB. It needs to produce all three:
31
+
32
+ ### 1a. JSONL Archive
33
+ - [ ] **cc-hook: copy raw JSONL** to `~/.ldm/agents/{agent}/memory/transcripts/`
34
+ - [ ] Copy the new portion (or full file) after each capture
35
+ - [ ] This is Dream Weaver's primary input — must be preserved
36
+ - [ ] Solves the FDA backup problem (files now in a controlled, backupable location)
37
+
38
+ ### 1b. MD Session Summaries
39
+ - [ ] **cc-hook: generate human-readable MD** per conversation
40
+ - [ ] Write to `~/.ldm/agents/{agent}/memory/sessions/YYYY-MM-DD-{slug}.md`
41
+ - [ ] Include: timestamp, session ID, conversation summary (user messages + key assistant responses)
42
+ - [ ] Matches what OpenClaw's session-memory hook produces, but independent of OpenClaw
43
+ - [ ] Consider: LLM-generated slug (like OpenClaw does) vs timestamp-only (simpler, no API call)
44
+
45
+ ### 1c. crystal.db Location
46
+ - [ ] **Move or symlink** crystal.db to `~/.ldm/memory/crystal.db`
47
+ - [ ] Currently at `~/.openclaw/memory-crystal/crystal.db` — outside the LDM archive
48
+ - [ ] The DB is **shared** — all agents (cc-mini, lesa-mini, cc-air) read/write to it, tagged by agent_id
49
+ - [ ] On remote devices, a read-only mirror lives at the same path
50
+
51
+ ## Priority 2 — LDM Scaffolding (no external dependency)
52
+
53
+ Memory Crystal must set up `~/.ldm/` itself. No dependency on wip-ldm-os repo.
54
+
55
+ - [ ] **Built-in scaffold** — `crystal init` or auto-scaffold on first run
56
+ - [ ] Create the full directory structure:
57
+ ```
58
+ ~/.ldm/
59
+ ├── config.json
60
+ ├── memory/
61
+ │ └── crystal.db (shared vector DB — all agents)
62
+ └── agents/{agent_id}/
63
+ ├── config.json
64
+ └── memory/
65
+ ├── transcripts/ (JSONL copies)
66
+ ├── sessions/ (MD summaries)
67
+ ├── daily/ (breadcrumb logs)
68
+ └── journals/ (Dream Weaver output)
69
+ ```
70
+ - [ ] Agent names are by machine: `cc-mini`, `lesa-mini`, `cc-air`
71
+ - [ ] Absorb relevant parts of `wip-ldm-os/bin/scaffold.sh` and templates
72
+ - [ ] Identity files (SOUL.md, IDENTITY.md, CONTEXT.md) — create empty templates or skip (agent-specific, not Memory Crystal's job)
73
+ - [ ] Config: agent name, harness type, Dream Weaver schedule
74
+
75
+ ## Priority 3 — Deploy Ephemeral Relay + Expand Poller
76
+
77
+ **Branch:** `cc-air/phase2-relay`
78
+ **Blocked on:** Parker's physical setup (see `ai/todos/parker/`)
79
+
80
+ 1. Generate encryption key, copy to both machines
81
+ 2. `wrangler login`, create R2 bucket, set bearer tokens
82
+ 3. Deploy Worker
83
+ 4. Configure env vars (Air + Mini)
84
+ 5. End-to-end test: capture → relay → ingest → mirror → search
85
+
86
+ ### Poller expansion (Mini-side)
87
+ The poller already decrypts conversation data from remote agents. Expand it to reconstruct the remote agent's full file tree on the Mini:
88
+
89
+ - [ ] **Write JSONL** to `~/.ldm/agents/{agent_id}/memory/transcripts/`
90
+ - [ ] **Generate MD summary** to `~/.ldm/agents/{agent_id}/memory/sessions/`
91
+ - [ ] **Append daily breadcrumb** to `~/.ldm/agents/{agent_id}/memory/daily/`
92
+ - [ ] **Ingest into crystal.db** (already done)
93
+
94
+ No extra relay transfer — the Mini rebuilds everything from the same decrypted data. This ensures the Mini has a complete `~/.ldm/` for every agent, enabling Dream Weaver to run against any agent from the Mini.
95
+
96
+ ## Priority 4 — Fix Broken Systems
97
+
98
+ - [ ] **Backup system** — FDA issue. Partially solved by JSONL copy to `~/.ldm/` (Priority 1a). Full fix: grant FDA or find alternative.
99
+ - [ ] **LanceDB removal** — Once sqlite-vec is validated, remove dual-write. Simplifies codebase, drops a dependency.
100
+ - [ ] **Retire context-embeddings** — Once Memory Crystal handles all three file types, disable the context-embeddings plugin in dot-openclaw. Update lesa-bridge to read from crystal.db instead.
101
+
102
+ ## Priority 5 — Recovery & Backfill Tools
103
+
104
+ - [ ] **`crystal replay`** — Re-send from raw JSONL through the relay. Recovery for when Mini is offline >24h.
105
+ - [ ] **`crystal backfill`** — One-time ingest of all ~115 historical CC sessions (~110MB, ~$0.07).
106
+
107
+ ## Priority 6 — Dream Weaver Integration
108
+
109
+ - [ ] **Trigger consolidation** — After poller ingests a batch, optionally kick off incremental Dream Weaver run.
110
+ - [ ] **Automate weekly schedule** — Currently manual. Add cron/launchd trigger based on `dreamWeaver.schedule` in config.
111
+ - [ ] **Dream Weaver reads from `~/.ldm/`** — Once JSONL copies live there, Dream Weaver reads from `transcripts/` instead of `~/.claude/projects/`.
112
+
113
+ ## Priority 7 — Search Quality (QMD Phases 3–5)
114
+
115
+ - [ ] **Phase 3: Smart chunking + dedup** — Content-hash dedup, better non-conversation chunking.
116
+ - [ ] **Phase 4: Re-ranking + query expansion** — LLM re-ranking, synonym expansion. Biggest quality jump remaining.
117
+ - [ ] **Phase 5: Local embeddings** — Replace OpenAI with local model (GGUF via node-llama-cpp, like QMD uses).
118
+
119
+ ## Priority 8 — Behavioral / Process
120
+
121
+ - [ ] **SHARED-CONTEXT.md** — Warm-start file agents read on boot. <50 lines, current state.
122
+ - [ ] **End-of-session handoffs** — Agent writes what-I-was-doing summary at session end.
123
+ - [ ] **Search-before-acting** — Agents search crystal before starting new work.
124
+
125
+ ## Priority 9 — Data Coverage Expansion
126
+
127
+ 7 of 11 data stores aren't in the vector DB. Review which warrant ingestion:
128
+ - [ ] CLAUDE.md memories, auto-memory, Dream Weaver narratives, dev updates, identity docs
129
+
130
+ ## Agents
131
+
132
+ Each harness instance = one agent = one "user" in LDM. Named by machine:
133
+
134
+ | Agent ID | Machine | Harness |
135
+ |----------|---------|---------|
136
+ | `cc-mini` | Mac Mini | Claude Code |
137
+ | `lesa-mini` | Mac Mini | OpenClaw |
138
+ | `cc-air` | MacBook Air | Claude Code |
139
+ | Future: `kodaks-air` | MacBook Air | Kodaks |
140
+
141
+ Each agent writes its own files (transcripts, sessions, daily) to `~/.ldm/agents/{agent_id}/memory/`. All agents share one `crystal.db` at `~/.ldm/memory/`.
142
+
143
+ **The Mini has everything.** Remote agents' file trees are reconstructed on the Mini by the poller from decrypted relay data. If a device is lost, nothing is lost — Dream Weaver can be run against any agent from the Mini.
144
+
145
+ ## Dependency Map
146
+
147
+ ```
148
+ Memory Crystal (standalone — no external repo dependencies for core function)
149
+ ├── sqlite-vec (npm — vector search)
150
+ ├── better-sqlite3 (npm — SQLite engine)
151
+ ├── OpenAI API (embedding provider — swappable for Ollama/local)
152
+ └── ~/.ldm/ (filesystem — scaffolded by Memory Crystal itself)
153
+
154
+ Optional integrations (not dependencies):
155
+ ├── OpenClaw plugin (if OpenClaw is installed, registers tools + agent_end hook)
156
+ ├── Claude Code hook (if CC is the harness, captures via Stop hook)
157
+ ├── Ephemeral relay (if multi-machine, Cloudflare Worker for sync)
158
+ └── Dream Weaver (reads from ~/.ldm/agents/*/transcripts/, writes to ~/.ldm/agents/*/journals/)
159
+ ```
@@ -0,0 +1,44 @@
1
+ # Memory Crystal ... Punchlist
2
+
3
+ **Updated:** 2026-02-27
4
+ **Repo:** `wipcomputer/memory-crystal-private`
5
+
6
+ ## Active
7
+
8
+ ### Release
9
+ ```bash
10
+ cd ~/Documents/wipcomputer--mac-mini-01/staff/Parker/Claude\ Code\ -\ Mini/repos/ldm-os/components/memory-crystal-private
11
+ git checkout main && git pull
12
+ wip-release minor --notes="Phase 2: LDM scaffolding, JSONL archive, MD summaries, relay merge"
13
+ ```
14
+ - [ ] `wip-release minor`
15
+
16
+ ## Architecture Direction
17
+
18
+ Three features, one product:
19
+ 1. **Memory** ... remember, search, forget. One SQLite DB. Any AI tool.
20
+ 2. **Bridge** ... local agent-to-agent communication. All messages saved to crystal.
21
+ 3. **Relay** ... multi-device sync. Remote agents communicate and share memory.
22
+
23
+ All writes go to the crystal. Always. No matter which layer.
24
+
25
+ Key merges:
26
+ - Lesa Bridge search tools absorbed into Crystal (Bridge = communication only)
27
+ - Context Embeddings absorbed into Crystal (Crystal captures conversations directly)
28
+
29
+ ## Next up (not blocking ship)
30
+
31
+ - [ ] Absorb lesa-bridge into Memory Crystal as Bridge feature
32
+ - [ ] Absorb context-embeddings into Memory Crystal (retire plugin)
33
+ - [ ] Cloudflare relay setup (key, R2 bucket, Worker deploy, env vars)
34
+ - [ ] Remove LanceDB dual-write
35
+ - [ ] `crystal backfill` for ~115 historical sessions
36
+ - [ ] Point Dream Weaver at `~/.ldm/` transcripts
37
+
38
+ ## Completed
39
+
40
+ - [x] Push `mini/phase2-relay` ... 2026-02-26
41
+ - [x] Create PR #2, squash-merge to main ... 2026-02-26
42
+ - [x] Deploy build to `~/.ldm/extensions/memory-crystal/` ... 2026-02-27
43
+ - [x] Migrate crystal.db to `~/.ldm/memory/crystal.db` (159K+ chunks, symlink at old path) ... 2026-02-26
44
+ - [x] Fix lesa-bridge session routing (`user: "claude-code"` for dedicated TUI-visible session) ... 2026-02-27
@@ -0,0 +1,31 @@
1
+ # ai/todos/
2
+
3
+ Agent-to-agent and agent-to-human task inbox system.
4
+
5
+ ## How it works
6
+
7
+ Each folder under `inboxes/` is a recipient. When an agent (or human) has action items for someone else, they drop a dated markdown file in that person's inbox.
8
+
9
+ ```
10
+ inboxes/
11
+ parker/ Parker's action items (manual steps, deploys, config)
12
+ cc-mini/ Claude Code on Mac Mini
13
+ cc-air/ Claude Code on MacBook Air
14
+ lesa/ (create when needed)
15
+ ```
16
+
17
+ ## Naming convention
18
+
19
+ ```
20
+ YYYY-MM-DD--{from-agent}--{short-description}.md
21
+ ```
22
+
23
+ Examples:
24
+ - `2026-02-26--cc-mini--phase2-your-moves.md` ... cc-mini telling Parker what to do
25
+ - `2026-02-25--cc-air--setup-checklist.md` ... cc-air telling Parker what to set up
26
+
27
+ ## Rules
28
+
29
+ - **Drop, don't edit.** If you have updates, drop a new file. Don't modify someone else's inbox items.
30
+ - **Done items stay.** Check the boxes in the file. Don't delete completed files... they're history.
31
+ - **Keep it short.** Link to plans or docs for details. The inbox item is the "what", not the "why".
@@ -0,0 +1,85 @@
1
+ # cc-air — Memory Crystal Todos
2
+
3
+ **Date:** 2026-02-26 (updated)
4
+ **Agent:** cc-air
5
+ **Context:** Full ecosystem review — memory docs, Dream Weaver, LDM OS, OpenClaw builtin, QMD
6
+
7
+ ## Priority 1 — Three File Types (the core gap)
8
+
9
+ Memory Crystal only produces the vector DB today. It needs all three:
10
+
11
+ - [ ] **JSONL archive** — cc-hook copies raw JSONL to `~/.ldm/agents/{agent_id}/memory/transcripts/` after each capture. Solves the FDA backup problem. Gives Dream Weaver its primary input in a controlled location.
12
+
13
+ - [ ] **MD session summaries** — cc-hook generates human-readable markdown per conversation. Write to `~/.ldm/agents/{agent_id}/memory/sessions/YYYY-MM-DD-{slug}.md`. Matches OpenClaw's session-memory hook output but works independently.
14
+
15
+ - [ ] **crystal.db location** — Move or symlink crystal.db to `~/.ldm/memory/crystal.db` (shared, root level). Currently at `~/.openclaw/memory-crystal/`. The DB is shared by all agents (cc-mini, lesa-mini, cc-air), tagged by agent_id. On remote devices, a read-only mirror lives at the same path.
16
+
17
+ ## Priority 2 — LDM Scaffolding (built into Memory Crystal)
18
+
19
+ Memory Crystal must set up `~/.ldm/` itself. No dependency on wip-ldm-os repo.
20
+
21
+ - [ ] **`crystal init` / auto-scaffold** — On first run, create the full directory structure:
22
+ ```
23
+ ~/.ldm/
24
+ ├── config.json
25
+ ├── memory/crystal.db (shared vector DB)
26
+ └── agents/{agent_id}/memory/
27
+ ├── transcripts/ sessions/ daily/ journals/
28
+ ```
29
+ - [ ] **Agent names by machine:** `cc-mini`, `lesa-mini`, `cc-air` (set via `CRYSTAL_AGENT_ID`)
30
+ - [ ] **Absorb wip-ldm-os scaffolding** — Port relevant parts of `scaffold.sh` and templates into Memory Crystal's installer. Identity files (SOUL.md etc.) can be empty templates or skipped.
31
+ - [ ] **Config** — agent name, harness type, Dream Weaver schedule in `~/.ldm/agents/{agent_id}/config.json`
32
+
33
+ ## Priority 3 — Deploy Ephemeral Relay + Expand Poller
34
+
35
+ See Parker's setup checklist at `ai/todos/parker/2026-02-25--cc-air--setup-checklist.md`.
36
+
37
+ - [ ] Generate encryption key + copy to both machines
38
+ - [ ] `wrangler login`, R2 bucket, bearer tokens, deploy
39
+ - [ ] Configure env vars, end-to-end test
40
+
41
+ **Poller expansion** — After decrypting relay data, the Mini-side poller must also reconstruct the remote agent's full file tree:
42
+ - [ ] Write JSONL to `~/.ldm/agents/{agent_id}/memory/transcripts/`
43
+ - [ ] Generate MD summary to `~/.ldm/agents/{agent_id}/memory/sessions/`
44
+ - [ ] Append daily breadcrumb to `~/.ldm/agents/{agent_id}/memory/daily/`
45
+ - [ ] This ensures the Mini has everything — if a device is lost, Dream Weaver can still run against that agent
46
+
47
+ ## Priority 4 — Fix Broken Systems
48
+
49
+ - [ ] **Retire context-embeddings** — Once Crystal handles all three file types, disable in dot-openclaw. Update lesa-bridge to read from crystal.db.
50
+ - [ ] **LanceDB removal** — Remove dual-write once sqlite-vec is validated.
51
+ - [ ] **Backup system** — Partially solved by JSONL copy (Priority 1). Full fix: grant FDA or alternative.
52
+
53
+ ## Priority 5 — Recovery & Backfill
54
+
55
+ - [ ] **`crystal replay`** — Re-send from raw JSONL through relay (recovery when Mini offline >24h).
56
+ - [ ] **`crystal backfill`** — One-time ingest of ~115 historical CC sessions (~110MB, ~$0.07).
57
+
58
+ ## Priority 6 — Dream Weaver Integration
59
+
60
+ - [ ] **Dream Weaver reads from `~/.ldm/`** — Once JSONL copies live there, point Dream Weaver at `transcripts/` instead of `~/.claude/projects/`.
61
+ - [ ] **Trigger consolidation** — After poller ingests a batch, optionally trigger incremental Dream Weaver run.
62
+ - [ ] **Automate weekly schedule** — Currently manual. Add cron/launchd trigger.
63
+
64
+ ## Priority 7 — Search Quality (QMD Phases 3–5)
65
+
66
+ - [ ] **Phase 3: Smart chunking + dedup** — Content-hash dedup, better non-conversation chunking.
67
+ - [ ] **Phase 4: Re-ranking + query expansion** — LLM re-ranking, synonym expansion.
68
+ - [ ] **Phase 5: Local embeddings** — Replace OpenAI with local GGUF model.
69
+
70
+ ## Priority 8 — Behavioral / Process
71
+
72
+ - [ ] **SHARED-CONTEXT.md** — Warm-start file agents read on boot.
73
+ - [ ] **End-of-session handoffs** — Agent writes what-I-was-doing summary.
74
+ - [ ] **Search-before-acting** — Agents search crystal before starting new work.
75
+
76
+ ## Notes
77
+
78
+ - **Agents are per-harness-instance, named by machine:** cc-mini, lesa-mini, cc-air, etc. Each is a separate "user" in LDM.
79
+ - **crystal.db is shared** at `~/.ldm/memory/crystal.db` — not inside any agent folder. All agents write to it (tagged by agent_id).
80
+ - **The Mini has everything.** Remote agents' file trees are reconstructed from relay data. Dream Weaver can run against any agent from the Mini.
81
+ - OpenClaw's builtin memory stays on — it evolves independently, doesn't hurt to run both.
82
+ - QMD is a separate system. Its search algorithms are already ported into Memory Crystal.
83
+ - Private mode is fully implemented across all 4 surfaces.
84
+ - The relay code is built and builds clean, just needs physical deployment.
85
+ - Memory Crystal has zero external repo dependencies for core function. OpenClaw, CC hook, relay, and Dream Weaver are optional integrations.
@@ -0,0 +1,100 @@
1
+ # cc-mini -- Memory Crystal Phase 2 Status
2
+
3
+ **Date:** 2026-02-26
4
+ **Agent:** cc-mini
5
+ **Branch:** `mini/phase2-relay`
6
+
7
+ ## What's Done
8
+
9
+ ### Priority 1 -- Three File Types
10
+
11
+ - [x] **JSONL archive** -- cc-hook copies raw JSONL to `~/.ldm/agents/{agent_id}/memory/transcripts/` after each capture. Uses mtime check to skip redundant copies.
12
+ - [x] **MD session summaries** -- cc-hook generates markdown per conversation after ingest. Two modes: `simple` (no API call) and `llm` (gpt-4o-mini). Written to `~/.ldm/agents/{agent_id}/memory/sessions/YYYY-MM-DD--HH-MM-SS--{agent}--{slug}.md`.
13
+ - [x] **crystal.db path resolution** -- `resolveConfig()` checks `~/.ldm/memory/crystal.db` first, falls back to legacy `~/.openclaw/memory-crystal/`. Auto-detection means old deployments keep working.
14
+ - [x] **`crystal migrate-db` CLI** -- Copies crystal.db to `~/.ldm/memory/`, verifies chunk count, creates symlink at old path. Never deletes the original.
15
+
16
+ ### Priority 2 -- LDM Scaffolding
17
+
18
+ - [x] **`src/ldm.ts` module** -- Central path resolution. `getAgentId()`, `ldmPaths()`, `scaffoldLdm()`, `ensureLdm()`.
19
+ - [x] **`crystal init [--agent]`** -- Creates full `~/.ldm/` directory tree, writes/updates config.json with agents array.
20
+ - [x] **Agent naming** -- Via `CRYSTAL_AGENT_ID` env var. Defaults to `cc-mini`.
21
+ - [x] **appendDailyLog()** -- Refactored to use `ldmPaths()` instead of hardcoded constant. Importable for reuse.
22
+
23
+ ### Priority 3 -- Relay Merge
24
+
25
+ - [x] **Merged `cc-air/phase2-relay`** into `mini/phase2-relay`. Conflicts resolved (cc-hook.ts imports + dev-update section).
26
+ - [x] **Poller expanded** -- After decrypting relay data, reconstructs remote agent's full file tree: JSONL transcript, MD summary, daily breadcrumb.
27
+ - [x] **mirror-sync.ts** -- Updated paths to use `ldmPaths()`.
28
+ - [x] **crypto.ts** -- Fixed HOME default.
29
+ - [x] **All HOME fallbacks** -- Replaced `/Users/lesa` and `/Users/parker` with `process.env.HOME || ''` across all source files. Safe for others to install.
30
+
31
+ ### Other
32
+
33
+ - [x] **dev-update.ts** -- Writes to each repo's `ai/` folder (decentralized). Disabled centralized wip-dev-updates git push.
34
+ - [x] **mcp-server.ts** -- Uses `createCrystal()` factory. Remote mode guards on source indexing operations.
35
+ - [x] **Build passes** -- All 12 entry points compile clean. `crystal status`, `crystal search`, `crystal init`, `poller --status`, `mirror-sync --status` all verified.
36
+
37
+ ## What's Next
38
+
39
+ ### Parker Manual Steps (blockers for relay)
40
+
41
+ - [ ] Generate encryption key: `openssl rand -base64 32 > ~/.openclaw/secrets/crystal-relay-key && chmod 600 ~/.openclaw/secrets/crystal-relay-key`
42
+ - [ ] Copy key to MacBook Air (same path)
43
+ - [ ] Cloudflare: `wrangler login`, create R2 bucket `memory-crystal-relay`, set secrets (bearer tokens), deploy Worker
44
+ - [ ] Set env vars on both machines: `CRYSTAL_RELAY_URL`, `CRYSTAL_RELAY_TOKEN`, `CRYSTAL_AGENT_ID`
45
+ - [ ] End-to-end test: drop from Air, verify Mini ingests
46
+ - [ ] See full checklist: `ai/todos/parker/2026-02-25--cc-air--setup-checklist.md`
47
+
48
+ ### Run migrate-db
49
+
50
+ - [ ] Stop gateway: `openclaw gateway stop`
51
+ - [ ] Run: `crystal migrate-db`
52
+ - [ ] Restart: `openclaw gateway restart`
53
+ - [ ] Verify: `crystal status` shows same chunk count from new path
54
+
55
+ ### Deploy Updated Plugin
56
+
57
+ - [ ] Copy built files to extension dir and restart gateway:
58
+ ```
59
+ cd /path/to/memory-crystal
60
+ npm run build
61
+ cp -r dist skills openclaw.plugin.json package.json ~/.openclaw/extensions/memory-crystal/
62
+ cd ~/.openclaw/extensions/memory-crystal && npm install --omit=dev
63
+ openclaw gateway restart
64
+ ```
65
+
66
+ ### Priority 4 -- Fix Broken Systems
67
+
68
+ - [ ] **Retire context-embeddings** -- Disable once Crystal handles all three file types. Update lesa-bridge.
69
+ - [ ] **LanceDB removal** -- Remove dual-write once sqlite-vec is validated long enough.
70
+ - [ ] **Backup system** -- Partially solved by JSONL copy. Full fix: grant FDA or alternative.
71
+
72
+ ### Priority 5 -- Recovery & Backfill
73
+
74
+ - [ ] **`crystal replay`** -- Re-send from raw JSONL through relay (recovery when Mini offline >24h).
75
+ - [ ] **`crystal backfill`** -- One-time ingest of ~115 historical CC sessions (~110MB, ~$0.07).
76
+
77
+ ### Priority 6 -- Dream Weaver Integration
78
+
79
+ - [ ] Point Dream Weaver at `~/.ldm/agents/{id}/memory/transcripts/` instead of `~/.claude/projects/`.
80
+ - [ ] Optional: trigger incremental DW run after poller batch.
81
+ - [ ] Automate weekly schedule via cron/launchd.
82
+
83
+ ### Priority 7 -- Search Quality (QMD Phases 3-5)
84
+
85
+ - [ ] Smart chunking + dedup
86
+ - [ ] LLM re-ranking + query expansion
87
+ - [ ] Local embeddings (replace OpenAI with local GGUF)
88
+
89
+ ### Priority 8 -- Behavioral / Process
90
+
91
+ - [ ] SHARED-CONTEXT.md warm-start
92
+ - [ ] End-of-session handoff summaries
93
+ - [ ] Search-before-acting convention
94
+
95
+ ## Notes
96
+
97
+ - crystal.db stays at `~/.openclaw/memory-crystal/` until `crystal migrate-db` is run. Symlink preserves backward compat.
98
+ - Summary mode defaults to `simple` (no API call). Set `CRYSTAL_SUMMARY_MODE=llm` for gpt-4o-mini summaries.
99
+ - The relay code builds clean but isn't deployed yet. Needs Parker's Cloudflare setup.
100
+ - 159,574 chunks in crystal.db as of this commit. All search and status commands verified working.
@@ -0,0 +1,25 @@
1
+ # Memory Crystal — TODO
2
+
3
+ ## Phase 1 — Done
4
+ All four doors live, Lēsa confirmed, 5,543+ chunks and growing.
5
+
6
+ ### Phase 1 cleanup (small)
7
+ - [ ] `npm link` to install `crystal` CLI globally
8
+ - [ ] Disable `context-embeddings` plugin (saves double embedding cost per turn) — or keep both for a few more days
9
+ - [ ] Remove `conversation_search`/`memory_search` from lesa-bridge (crystal replaces them)
10
+
11
+ ## Phase 2 — Cloudflare Worker mirror (big)
12
+ - [ ] `worker.ts` — REST API backed by D1 + Vectorize + R2
13
+ - [ ] `crystal push` / `crystal pull` / `crystal reset` sync commands
14
+ - [ ] Daily cron: mini uploads, Worker rebuilds, mini pulls new writes
15
+ - [ ] **Needs:** Parker's Cloudflare account + `wrangler login`
16
+
17
+ ## Phase 3 — Remote multi-agent access
18
+ - [ ] Remote MCP for Claude.ai web
19
+ - [ ] OpenAPI spec for ChatGPT/Grok
20
+ - [ ] `crystal search --remote` from anywhere
21
+
22
+ ## Other stuff on the board
23
+ - [ ] GitHub setup (Lēsa's account)
24
+ - [ ] Repo renames
25
+ - [ ] cc-persistence (Pi RPC sidecar)
@@ -0,0 +1,139 @@
1
+ # Parker — Setup Checklist (Ephemeral Relay)
2
+
3
+ Everything you need to do to get the encrypted relay live.
4
+
5
+ ## 1. Generate Encryption Key (once, shared across machines)
6
+
7
+ ```bash
8
+ openssl rand -base64 32 > ~/.openclaw/secrets/crystal-relay-key
9
+ chmod 600 ~/.openclaw/secrets/crystal-relay-key
10
+ ```
11
+
12
+ Copy `~/.openclaw/secrets/crystal-relay-key` to both machines (Air + Mini).
13
+ This key never goes to Cloudflare. It stays on your hardware.
14
+
15
+ **How to copy:**
16
+ - AirDrop the file
17
+ - Or from one machine: `cat ~/.openclaw/secrets/crystal-relay-key` → paste on the other
18
+ - Or copy via the FireWire shared disk
19
+
20
+ ## 2. 1Password Service Account Token (MacBook Air)
21
+
22
+ The Air needs the same service account token that's on the Mini.
23
+
24
+ - [ ] Copy `~/.openclaw/secrets/op-sa-token` from the Mac Mini
25
+ - [ ] Paste it to `~/.openclaw/secrets/op-sa-token` on the MacBook Air
26
+ - [ ] Run `chmod 600 ~/.openclaw/secrets/op-sa-token`
27
+
28
+ ## 3. Cloudflare Setup
29
+
30
+ - [ ] Install wrangler (if not already): `npm install -g wrangler`
31
+ - [ ] Run `wrangler login` (opens browser, authenticates)
32
+
33
+ ### Create R2 Bucket
34
+
35
+ ```bash
36
+ cd /Users/parker/Documents/dev-wip/repos/memory-crystal
37
+ wrangler r2 bucket create memory-crystal-relay
38
+ ```
39
+
40
+ ### Set Secrets
41
+
42
+ Generate three bearer tokens (one per agent):
43
+ ```bash
44
+ openssl rand -hex 32
45
+ ```
46
+
47
+ - [ ] Set cc-air token: `wrangler secret put AUTH_TOKEN_CC_AIR`
48
+ - [ ] Set cc-mini token: `wrangler secret put AUTH_TOKEN_CC_MINI`
49
+ - [ ] Set lēsa token: `wrangler secret put AUTH_TOKEN_LESA`
50
+
51
+ **Save the cc-air token** — you'll need it for the Air's env config.
52
+ **Save the cc-mini token** — you'll need it for the Mini's poller config.
53
+
54
+ ### Deploy
55
+
56
+ - [ ] Deploy the Worker:
57
+ ```bash
58
+ wrangler deploy
59
+ ```
60
+ Note the URL (e.g., `https://memory-crystal-relay.wipcomputer.workers.dev`)
61
+
62
+ ### Verify
63
+
64
+ - [ ] Test health endpoint:
65
+ ```bash
66
+ curl https://memory-crystal-relay.wipcomputer.workers.dev/health
67
+ ```
68
+ Should return: `{"ok":true,"service":"memory-crystal-relay","mode":"ephemeral"}`
69
+
70
+ ## 4. Configure MacBook Air (cc-air)
71
+
72
+ Set these environment variables (in `~/.zshrc` or Claude Code hook config):
73
+
74
+ ```bash
75
+ export CRYSTAL_RELAY_URL="https://memory-crystal-relay.wipcomputer.workers.dev"
76
+ export CRYSTAL_RELAY_TOKEN="<cc-air bearer token from step 3>"
77
+ export CRYSTAL_AGENT_ID="cc-air"
78
+ ```
79
+
80
+ The cc-hook will auto-detect relay mode when these are set.
81
+
82
+ ## 5. Configure Mac Mini (cc-mini poller)
83
+
84
+ Set these environment variables:
85
+
86
+ ```bash
87
+ export CRYSTAL_RELAY_URL="https://memory-crystal-relay.wipcomputer.workers.dev"
88
+ export CRYSTAL_RELAY_TOKEN="<cc-mini bearer token from step 3>"
89
+ ```
90
+
91
+ ### Set up poller (cron or launchd):
92
+
93
+ **Option A: Cron (every 2 min)**
94
+ ```bash
95
+ crontab -e
96
+ # Add:
97
+ */2 * * * * cd /path/to/memory-crystal && node dist/poller.js 2>> ~/.openclaw/logs/relay-poller.log
98
+ ```
99
+
100
+ **Option B: LaunchAgent (recommended)**
101
+ Create a plist for continuous polling with `--watch` mode.
102
+
103
+ **Option C: Manual**
104
+ ```bash
105
+ node dist/poller.js # poll once
106
+ node dist/poller.js --watch # poll continuously
107
+ node dist/poller.js --push-mirror # push mirror snapshot
108
+ ```
109
+
110
+ ## 6. Verify End-to-End
111
+
112
+ 1. Talk to cc-air on the MacBook Air
113
+ 2. Check relay has a blob: `curl -H "Authorization: Bearer <mini-token>" https://memory-crystal-relay.wipcomputer.workers.dev/pickup/conversations`
114
+ 3. Run the poller on Mini: `node dist/poller.js`
115
+ 4. Verify chunks ingested into master crystal: `node dist/cli.js status`
116
+ 5. Push mirror: `node dist/poller.js --push-mirror`
117
+ 6. Pull mirror on Air: `node dist/mirror-sync.js`
118
+ 7. Search on Air — should return results from the conversation
119
+
120
+ ## Quick Reference
121
+
122
+ | What | Where |
123
+ |------|-------|
124
+ | Encryption key | `~/.openclaw/secrets/crystal-relay-key` (both machines) |
125
+ | SA token | `~/.openclaw/secrets/op-sa-token` (both machines) |
126
+ | Repo | `/Users/parker/Documents/dev-wip/repos/memory-crystal` |
127
+ | Branch | `cc-air/phase2-relay` |
128
+ | Worker config | `wrangler.toml` |
129
+ | Air env vars | `CRYSTAL_RELAY_URL`, `CRYSTAL_RELAY_TOKEN`, `CRYSTAL_AGENT_ID` |
130
+ | Mini env vars | `CRYSTAL_RELAY_URL`, `CRYSTAL_RELAY_TOKEN` |
131
+
132
+ ## What's NOT on Cloudflare
133
+
134
+ - No encryption keys
135
+ - No API keys
136
+ - No database
137
+ - No search capability
138
+ - No ability to read the data
139
+ - Just auth tokens and encrypted blobs that auto-expire in 24h