memory-crystal 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +20 -0
- package/CHANGELOG.md +6 -0
- package/LETTERS.md +22 -0
- package/LICENSE +21 -0
- package/README-ENTERPRISE.md +162 -0
- package/README-old.md +275 -0
- package/README.md +91 -0
- package/RELAY.md +88 -0
- package/TECHNICAL.md +379 -0
- package/ai/dev-updates/2026-02-25--cc-air--phase2-architecture-pivot.md +70 -0
- package/ai/dev-updates/2026-02-25--cc-air--phase2-worker-build.md +72 -0
- package/ai/dev-updates/2026-02-26--10-25-16--cc-mini--phase2-implementation.md +49 -0
- package/ai/dev-updates/2026-02-27--20-30-00--cc-mini--readme-overhaul-and-public-deploy.md +69 -0
- package/ai/notes/2026-02-26--cc-air--notes.md +412 -0
- package/ai/notes/2026-02-27--cc-mini--grok-feedback.md +44 -0
- package/ai/notes/2026-02-27--cc-mini--lesa-feedback.md +45 -0
- package/ai/notes/RESEARCH.md +1185 -0
- package/ai/notes/salience-research/README.md +29 -0
- package/ai/notes/salience-research/eurosla-salience-review.md +64 -0
- package/ai/notes/salience-research/full-research-summary.md +269 -0
- package/ai/notes/salience-research/salience-levels-diagram.png +0 -0
- package/ai/plan/2026-02-27--cc-mini--qr-pairing-spec.md +203 -0
- package/ai/plan/_archive/PLAN.md +194 -0
- package/ai/plan/_archive/PRD.md +1014 -0
- package/ai/plan/cc-plans-duplicates-from-dot-claude/2026-02-26--cc-mini--phase2-implementation-plan.md +245 -0
- package/ai/plan/dev-conventions-note.md +70 -0
- package/ai/plan/ldm-os-install-and-boot-architecture.md +285 -0
- package/ai/plan/memory-crystal-phase2-plan.md +192 -0
- package/ai/plan/memory-system-lay-of-the-land.md +214 -0
- package/ai/plan/phase2-ephemeral-relay.md +238 -0
- package/ai/plan/readme-first.md +68 -0
- package/ai/plan/roadmap.md +159 -0
- package/ai/todos/PUNCHLIST.md +44 -0
- package/ai/todos/README.md +31 -0
- package/ai/todos/inboxes/cc-air/2026-02-26--cc-air--post-relay-todos.md +85 -0
- package/ai/todos/inboxes/cc-mini/2026-02-26--cc-mini--phase2-status.md +100 -0
- package/ai/todos/inboxes/cc-mini/_archive/TODO.md +25 -0
- package/ai/todos/inboxes/parker/2026-02-25--cc-air--setup-checklist.md +139 -0
- package/ai/todos/inboxes/parker/2026-02-26--cc-mini--phase2-your-moves.md +72 -0
- package/dist/cc-hook.d.ts +1 -0
- package/dist/cc-hook.js +349 -0
- package/dist/chunk-3VFIJYS4.js +818 -0
- package/dist/chunk-52QE3YI3.js +1169 -0
- package/dist/chunk-AA3OPP4Z.js +432 -0
- package/dist/chunk-D3I3ZSE2.js +411 -0
- package/dist/chunk-EKSACBTJ.js +1070 -0
- package/dist/chunk-F3Y7EL7K.js +83 -0
- package/dist/chunk-JWZXYVET.js +1068 -0
- package/dist/chunk-KYVWO6ZM.js +1069 -0
- package/dist/chunk-L3VHARQH.js +413 -0
- package/dist/chunk-LOVAHSQV.js +411 -0
- package/dist/chunk-LQOYCAGG.js +446 -0
- package/dist/chunk-MK42FMEG.js +147 -0
- package/dist/chunk-NIJCVN3O.js +147 -0
- package/dist/chunk-O2UITJGH.js +465 -0
- package/dist/chunk-PEK6JH65.js +432 -0
- package/dist/chunk-PJ6FFKEX.js +77 -0
- package/dist/chunk-PLUBBZYR.js +800 -0
- package/dist/chunk-SGL6ISBJ.js +1061 -0
- package/dist/chunk-UNHVZB5G.js +411 -0
- package/dist/chunk-VAFTWSTE.js +1061 -0
- package/dist/chunk-XZ3S56RQ.js +1061 -0
- package/dist/chunk-Y72C7F6O.js +148 -0
- package/dist/cli.d.ts +1 -0
- package/dist/cli.js +325 -0
- package/dist/core.d.ts +188 -0
- package/dist/core.js +12 -0
- package/dist/crypto.d.ts +16 -0
- package/dist/crypto.js +18 -0
- package/dist/dev-update-SZ2Z4WCQ.js +6 -0
- package/dist/ldm.d.ts +17 -0
- package/dist/ldm.js +12 -0
- package/dist/mcp-server.d.ts +1 -0
- package/dist/mcp-server.js +250 -0
- package/dist/migrate.d.ts +1 -0
- package/dist/migrate.js +89 -0
- package/dist/mirror-sync.d.ts +1 -0
- package/dist/mirror-sync.js +130 -0
- package/dist/openclaw.d.ts +5 -0
- package/dist/openclaw.js +349 -0
- package/dist/poller.d.ts +1 -0
- package/dist/poller.js +272 -0
- package/dist/summarize.d.ts +19 -0
- package/dist/summarize.js +10 -0
- package/dist/worker.js +137 -0
- package/openclaw.plugin.json +11 -0
- package/package.json +40 -0
- package/scripts/migrate-lance-to-sqlite.mjs +217 -0
- package/skills/memory/SKILL.md +61 -0
- package/src/cc-hook.ts +447 -0
- package/src/cli.ts +356 -0
- package/src/core.ts +1472 -0
- package/src/crypto.ts +113 -0
- package/src/dev-update.ts +178 -0
- package/src/ldm.ts +117 -0
- package/src/mcp-server.ts +274 -0
- package/src/migrate.ts +104 -0
- package/src/mirror-sync.ts +175 -0
- package/src/openclaw.ts +250 -0
- package/src/poller.ts +345 -0
- package/src/summarize.ts +210 -0
- package/src/worker.ts +208 -0
- package/tsconfig.json +18 -0
- package/wrangler.toml +20 -0
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# Read Me First
|
|
2
|
+
|
|
3
|
+
Start here before working on Memory Crystal. Read these docs in this order.
|
|
4
|
+
|
|
5
|
+
## 1. Lay of the Land
|
|
6
|
+
|
|
7
|
+
Understand the full memory ecosystem — what exists, what's broken, how the pieces fit:
|
|
8
|
+
|
|
9
|
+
- **`ai/plan/memory-system-lay-of-the-land.md`** — The five-layer memory stack, three file types, three parallel memory systems, agent naming, `~/.ldm/` directory structure, Dream Weaver relationship, the big picture diagram
|
|
10
|
+
|
|
11
|
+
## 2. Architecture
|
|
12
|
+
|
|
13
|
+
How the ephemeral relay and cross-machine sync works:
|
|
14
|
+
|
|
15
|
+
- **`ai/plan/phase2-ephemeral-relay.md`** — Two one-way roads (Device→Mini, Mini→Device), security architecture (AES-256-GCM, HMAC, nonce management, key rotation), threat model, Worker API
|
|
16
|
+
|
|
17
|
+
## 3. Roadmap
|
|
18
|
+
|
|
19
|
+
What's done, what's next, in priority order:
|
|
20
|
+
|
|
21
|
+
- **`ai/plan/roadmap.md`** — Priority 1 (three file types) through Priority 9 (data coverage). Agent table, dependency map, LDM scaffolding plan
|
|
22
|
+
|
|
23
|
+
## 4. Todos
|
|
24
|
+
|
|
25
|
+
Actionable work items:
|
|
26
|
+
|
|
27
|
+
- **`ai/todos/cc-air/2026-02-26--cc-air--post-relay-todos.md`** — Full todo list with priorities
|
|
28
|
+
- **`ai/todos/parker/2026-02-25--cc-air--setup-checklist.md`** — Parker's physical setup for the relay (encryption key, Cloudflare, env vars)
|
|
29
|
+
|
|
30
|
+
## 5. Background Reading (in `_git-ignore/memory-docs/`)
|
|
31
|
+
|
|
32
|
+
These docs are gitignored but provide the thinking behind the decisions. Read chronologically:
|
|
33
|
+
|
|
34
|
+
1. **`cc-memory-backfill-tool.md`** (Feb 13) — Plan for `--backfill` to ingest historical CC sessions
|
|
35
|
+
2. **`cc-wake-and-memory-capture.md`** (Feb 13) — Inbox watcher + auto capture with watermarking
|
|
36
|
+
3. **`memory-capture-fix-plan.md`** (Feb 13) — snake_case/camelCase bug fix, private mode, kill switch
|
|
37
|
+
4. **`cc-how-we-remember.md`** (Feb 16) — Behavioral proposal: SHARED-CONTEXT.md, 10x crystal_remember, search-before-acting
|
|
38
|
+
5. **`2026-02-16--qmd-vs-crystal-memory.md`** (Feb 16) — QMD vs Memory Crystal comparison (QMD search algorithms now ported)
|
|
39
|
+
6. **`2026-02-16--qmd-crystal-integration-plan.md`** (Feb 16) — 5-phase plan (Phases 1-2 done, 3-5 remaining)
|
|
40
|
+
7. **`2026-02-16--before-message-write-hook.md`** (Feb 16) — OpenClaw hook for true ephemeral private mode
|
|
41
|
+
8. **`identity-fingerprint-conversation-2026-02-19.md`** (Feb 19) — Identity persistence across model swaps
|
|
42
|
+
9. **`memory-system-architecture.md`** (Feb 20) — Complete map of every data store
|
|
43
|
+
10. **`memory-data-inventory.md`** (Feb 20) — Inventory of all 11 local stores, 7 not in vector DB
|
|
44
|
+
|
|
45
|
+
## 6. Superseded (read only for historical context)
|
|
46
|
+
|
|
47
|
+
- **`ai/plan/memory-crystal-phase2-plan.md`** — Old cloud-mirror design. Replaced by ephemeral relay.
|
|
48
|
+
- **`PLAN.md`** (repo root) — Original Phase 1-4 plan. Phase 1 complete. Phase 2+ superseded by relay.
|
|
49
|
+
|
|
50
|
+
## 7. Related Repos
|
|
51
|
+
|
|
52
|
+
These repos are part of the ecosystem but Memory Crystal has no code dependency on them:
|
|
53
|
+
|
|
54
|
+
- **`wip-ldm-os`** — LDM OS spec + installer. Memory Crystal absorbs its scaffolding.
|
|
55
|
+
- **`ldm-home`** — Deployed `~/.ldm/` directory (version-controlled mirror)
|
|
56
|
+
- **`dream-weaver-protocol`** — Memory consolidation protocol. Reads JSONL transcripts from `~/.ldm/agents/*/memory/transcripts/`
|
|
57
|
+
- **`openclaw`** — Agent harness. Memory Crystal runs as an optional plugin inside it.
|
|
58
|
+
- **`qmd`** — Standalone doc search engine. Search algorithms ported into Memory Crystal (RRF, FTS5, BM25).
|
|
59
|
+
- **`lesa-openclaw-context-embeddings`** — Older conversation capture plugin. Being superseded by Memory Crystal.
|
|
60
|
+
- **`lesa-bridge`** — Cross-agent communication. Currently reads context-embeddings DB; will migrate to crystal.db.
|
|
61
|
+
|
|
62
|
+
## Key Concepts
|
|
63
|
+
|
|
64
|
+
- **Agents are per-harness-instance, named by machine:** cc-mini, lesa-mini, cc-air
|
|
65
|
+
- **crystal.db is shared** at `~/.ldm/memory/crystal.db` — all agents write to it, tagged by agent_id
|
|
66
|
+
- **The Mini has everything** — remote agents' file trees are reconstructed from relay data
|
|
67
|
+
- **Memory Crystal works with or without OpenClaw** — independent system that mirrors the same features
|
|
68
|
+
- **Three file types:** JSONL (raw archive), MD (human-readable), vector DB (searchable)
|
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
# Memory Crystal — Roadmap
|
|
2
|
+
|
|
3
|
+
**Date:** 2026-02-26 (updated)
|
|
4
|
+
**Agent:** cc-air
|
|
5
|
+
**Based on:** memory docs review, ecosystem inspection, Dream Weaver analysis
|
|
6
|
+
|
|
7
|
+
## Vision
|
|
8
|
+
|
|
9
|
+
Memory Crystal is the universal archive manager for `~/.ldm/`. It works with or without OpenClaw, with any agent, on any harness. It produces three file types (JSONL copies, MD summaries, vector DB) and stores everything in one directory that gets backed up.
|
|
10
|
+
|
|
11
|
+
It includes its own `~/.ldm/` scaffolding — no dependency on the wip-ldm-os repo. Everything needed to set up the LDM directory structure is built into Memory Crystal's installer.
|
|
12
|
+
|
|
13
|
+
## What's Done
|
|
14
|
+
|
|
15
|
+
- [x] sqlite-vec migration (replaced LanceDB as primary, LanceDB kept as fallback)
|
|
16
|
+
- [x] FTS5 full-text search with BM25 scoring
|
|
17
|
+
- [x] Hybrid search: vector + keyword with RRF fusion (ported from QMD)
|
|
18
|
+
- [x] Recency weighting
|
|
19
|
+
- [x] Turn-boundary chunking (1 conversation message = 1 chunk)
|
|
20
|
+
- [x] CC Stop hook with watermark-based capture
|
|
21
|
+
- [x] MCP server (crystal_search, crystal_remember tools)
|
|
22
|
+
- [x] OpenClaw plugin (Lēsa's interface)
|
|
23
|
+
- [x] CLI (crystal search, crystal remember, crystal status)
|
|
24
|
+
- [x] Private mode (kill switch across CLI, MCP, OpenClaw, env var)
|
|
25
|
+
- [x] Ephemeral relay code: crypto, worker, cc-hook dual mode, poller, mirror-sync
|
|
26
|
+
- [x] Daily breadcrumb logs to `~/.ldm/agents/cc/memory/daily/`
|
|
27
|
+
|
|
28
|
+
## Priority 1 — Three File Types (the core gap)
|
|
29
|
+
|
|
30
|
+
Memory Crystal currently only produces the vector DB. It needs to produce all three:
|
|
31
|
+
|
|
32
|
+
### 1a. JSONL Archive
|
|
33
|
+
- [ ] **cc-hook: copy raw JSONL** to `~/.ldm/agents/{agent}/memory/transcripts/`
|
|
34
|
+
- [ ] Copy the new portion (or full file) after each capture
|
|
35
|
+
- [ ] This is Dream Weaver's primary input — must be preserved
|
|
36
|
+
- [ ] Solves the FDA backup problem (files now in a controlled, backupable location)
|
|
37
|
+
|
|
38
|
+
### 1b. MD Session Summaries
|
|
39
|
+
- [ ] **cc-hook: generate human-readable MD** per conversation
|
|
40
|
+
- [ ] Write to `~/.ldm/agents/{agent}/memory/sessions/YYYY-MM-DD-{slug}.md`
|
|
41
|
+
- [ ] Include: timestamp, session ID, conversation summary (user messages + key assistant responses)
|
|
42
|
+
- [ ] Matches what OpenClaw's session-memory hook produces, but independent of OpenClaw
|
|
43
|
+
- [ ] Consider: LLM-generated slug (like OpenClaw does) vs timestamp-only (simpler, no API call)
|
|
44
|
+
|
|
45
|
+
### 1c. crystal.db Location
|
|
46
|
+
- [ ] **Move or symlink** crystal.db to `~/.ldm/memory/crystal.db`
|
|
47
|
+
- [ ] Currently at `~/.openclaw/memory-crystal/crystal.db` — outside the LDM archive
|
|
48
|
+
- [ ] The DB is **shared** — all agents (cc-mini, lesa-mini, cc-air) read/write to it, tagged by agent_id
|
|
49
|
+
- [ ] On remote devices, a read-only mirror lives at the same path
|
|
50
|
+
|
|
51
|
+
## Priority 2 — LDM Scaffolding (no external dependency)
|
|
52
|
+
|
|
53
|
+
Memory Crystal must set up `~/.ldm/` itself. No dependency on wip-ldm-os repo.
|
|
54
|
+
|
|
55
|
+
- [ ] **Built-in scaffold** — `crystal init` or auto-scaffold on first run
|
|
56
|
+
- [ ] Create the full directory structure:
|
|
57
|
+
```
|
|
58
|
+
~/.ldm/
|
|
59
|
+
├── config.json
|
|
60
|
+
├── memory/
|
|
61
|
+
│ └── crystal.db (shared vector DB — all agents)
|
|
62
|
+
└── agents/{agent_id}/
|
|
63
|
+
├── config.json
|
|
64
|
+
└── memory/
|
|
65
|
+
├── transcripts/ (JSONL copies)
|
|
66
|
+
├── sessions/ (MD summaries)
|
|
67
|
+
├── daily/ (breadcrumb logs)
|
|
68
|
+
└── journals/ (Dream Weaver output)
|
|
69
|
+
```
|
|
70
|
+
- [ ] Agent names are by machine: `cc-mini`, `lesa-mini`, `cc-air`
|
|
71
|
+
- [ ] Absorb relevant parts of `wip-ldm-os/bin/scaffold.sh` and templates
|
|
72
|
+
- [ ] Identity files (SOUL.md, IDENTITY.md, CONTEXT.md) — create empty templates or skip (agent-specific, not Memory Crystal's job)
|
|
73
|
+
- [ ] Config: agent name, harness type, Dream Weaver schedule
|
|
74
|
+
|
|
75
|
+
## Priority 3 — Deploy Ephemeral Relay + Expand Poller
|
|
76
|
+
|
|
77
|
+
**Branch:** `cc-air/phase2-relay`
|
|
78
|
+
**Blocked on:** Parker's physical setup (see `ai/todos/parker/`)
|
|
79
|
+
|
|
80
|
+
1. Generate encryption key, copy to both machines
|
|
81
|
+
2. `wrangler login`, create R2 bucket, set bearer tokens
|
|
82
|
+
3. Deploy Worker
|
|
83
|
+
4. Configure env vars (Air + Mini)
|
|
84
|
+
5. End-to-end test: capture → relay → ingest → mirror → search
|
|
85
|
+
|
|
86
|
+
### Poller expansion (Mini-side)
|
|
87
|
+
The poller already decrypts conversation data from remote agents. Expand it to reconstruct the remote agent's full file tree on the Mini:
|
|
88
|
+
|
|
89
|
+
- [ ] **Write JSONL** to `~/.ldm/agents/{agent_id}/memory/transcripts/`
|
|
90
|
+
- [ ] **Generate MD summary** to `~/.ldm/agents/{agent_id}/memory/sessions/`
|
|
91
|
+
- [ ] **Append daily breadcrumb** to `~/.ldm/agents/{agent_id}/memory/daily/`
|
|
92
|
+
- [ ] **Ingest into crystal.db** (already done)
|
|
93
|
+
|
|
94
|
+
No extra relay transfer — the Mini rebuilds everything from the same decrypted data. This ensures the Mini has a complete `~/.ldm/` for every agent, enabling Dream Weaver to run against any agent from the Mini.
|
|
95
|
+
|
|
96
|
+
## Priority 4 — Fix Broken Systems
|
|
97
|
+
|
|
98
|
+
- [ ] **Backup system** — FDA issue. Partially solved by JSONL copy to `~/.ldm/` (Priority 1a). Full fix: grant FDA or find alternative.
|
|
99
|
+
- [ ] **LanceDB removal** — Once sqlite-vec is validated, remove dual-write. Simplifies codebase, drops a dependency.
|
|
100
|
+
- [ ] **Retire context-embeddings** — Once Memory Crystal handles all three file types, disable the context-embeddings plugin in dot-openclaw. Update lesa-bridge to read from crystal.db instead.
|
|
101
|
+
|
|
102
|
+
## Priority 5 — Recovery & Backfill Tools
|
|
103
|
+
|
|
104
|
+
- [ ] **`crystal replay`** — Re-send from raw JSONL through the relay. Recovery for when Mini is offline >24h.
|
|
105
|
+
- [ ] **`crystal backfill`** — One-time ingest of all ~115 historical CC sessions (~110MB, ~$0.07).
|
|
106
|
+
|
|
107
|
+
## Priority 6 — Dream Weaver Integration
|
|
108
|
+
|
|
109
|
+
- [ ] **Trigger consolidation** — After poller ingests a batch, optionally kick off incremental Dream Weaver run.
|
|
110
|
+
- [ ] **Automate weekly schedule** — Currently manual. Add cron/launchd trigger based on `dreamWeaver.schedule` in config.
|
|
111
|
+
- [ ] **Dream Weaver reads from `~/.ldm/`** — Once JSONL copies live there, Dream Weaver reads from `transcripts/` instead of `~/.claude/projects/`.
|
|
112
|
+
|
|
113
|
+
## Priority 7 — Search Quality (QMD Phases 3–5)
|
|
114
|
+
|
|
115
|
+
- [ ] **Phase 3: Smart chunking + dedup** — Content-hash dedup, better non-conversation chunking.
|
|
116
|
+
- [ ] **Phase 4: Re-ranking + query expansion** — LLM re-ranking, synonym expansion. Biggest quality jump remaining.
|
|
117
|
+
- [ ] **Phase 5: Local embeddings** — Replace OpenAI with local model (GGUF via node-llama-cpp, like QMD uses).
|
|
118
|
+
|
|
119
|
+
## Priority 8 — Behavioral / Process
|
|
120
|
+
|
|
121
|
+
- [ ] **SHARED-CONTEXT.md** — Warm-start file agents read on boot. <50 lines, current state.
|
|
122
|
+
- [ ] **End-of-session handoffs** — Agent writes what-I-was-doing summary at session end.
|
|
123
|
+
- [ ] **Search-before-acting** — Agents search crystal before starting new work.
|
|
124
|
+
|
|
125
|
+
## Priority 9 — Data Coverage Expansion
|
|
126
|
+
|
|
127
|
+
7 of 11 data stores aren't in the vector DB. Review which warrant ingestion:
|
|
128
|
+
- [ ] CLAUDE.md memories, auto-memory, Dream Weaver narratives, dev updates, identity docs
|
|
129
|
+
|
|
130
|
+
## Agents
|
|
131
|
+
|
|
132
|
+
Each harness instance = one agent = one "user" in LDM. Named by machine:
|
|
133
|
+
|
|
134
|
+
| Agent ID | Machine | Harness |
|
|
135
|
+
|----------|---------|---------|
|
|
136
|
+
| `cc-mini` | Mac Mini | Claude Code |
|
|
137
|
+
| `lesa-mini` | Mac Mini | OpenClaw |
|
|
138
|
+
| `cc-air` | MacBook Air | Claude Code |
|
|
139
|
+
| Future: `kodaks-air` | MacBook Air | Kodaks |
|
|
140
|
+
|
|
141
|
+
Each agent writes its own files (transcripts, sessions, daily) to `~/.ldm/agents/{agent_id}/memory/`. All agents share one `crystal.db` at `~/.ldm/memory/`.
|
|
142
|
+
|
|
143
|
+
**The Mini has everything.** Remote agents' file trees are reconstructed on the Mini by the poller from decrypted relay data. If a device is lost, nothing is lost — Dream Weaver can be run against any agent from the Mini.
|
|
144
|
+
|
|
145
|
+
## Dependency Map
|
|
146
|
+
|
|
147
|
+
```
|
|
148
|
+
Memory Crystal (standalone — no external repo dependencies for core function)
|
|
149
|
+
├── sqlite-vec (npm — vector search)
|
|
150
|
+
├── better-sqlite3 (npm — SQLite engine)
|
|
151
|
+
├── OpenAI API (embedding provider — swappable for Ollama/local)
|
|
152
|
+
└── ~/.ldm/ (filesystem — scaffolded by Memory Crystal itself)
|
|
153
|
+
|
|
154
|
+
Optional integrations (not dependencies):
|
|
155
|
+
├── OpenClaw plugin (if OpenClaw is installed, registers tools + agent_end hook)
|
|
156
|
+
├── Claude Code hook (if CC is the harness, captures via Stop hook)
|
|
157
|
+
├── Ephemeral relay (if multi-machine, Cloudflare Worker for sync)
|
|
158
|
+
└── Dream Weaver (reads from ~/.ldm/agents/*/transcripts/, writes to ~/.ldm/agents/*/journals/)
|
|
159
|
+
```
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# Memory Crystal ... Punchlist
|
|
2
|
+
|
|
3
|
+
**Updated:** 2026-02-27
|
|
4
|
+
**Repo:** `wipcomputer/memory-crystal-private`
|
|
5
|
+
|
|
6
|
+
## Active
|
|
7
|
+
|
|
8
|
+
### Release
|
|
9
|
+
```bash
|
|
10
|
+
cd ~/Documents/wipcomputer--mac-mini-01/staff/Parker/Claude\ Code\ -\ Mini/repos/ldm-os/components/memory-crystal-private
|
|
11
|
+
git checkout main && git pull
|
|
12
|
+
wip-release minor --notes="Phase 2: LDM scaffolding, JSONL archive, MD summaries, relay merge"
|
|
13
|
+
```
|
|
14
|
+
- [ ] `wip-release minor`
|
|
15
|
+
|
|
16
|
+
## Architecture Direction
|
|
17
|
+
|
|
18
|
+
Three features, one product:
|
|
19
|
+
1. **Memory** ... remember, search, forget. One SQLite DB. Any AI tool.
|
|
20
|
+
2. **Bridge** ... local agent-to-agent communication. All messages saved to crystal.
|
|
21
|
+
3. **Relay** ... multi-device sync. Remote agents communicate and share memory.
|
|
22
|
+
|
|
23
|
+
All writes go to the crystal. Always. No matter which layer.
|
|
24
|
+
|
|
25
|
+
Key merges:
|
|
26
|
+
- Lesa Bridge search tools absorbed into Crystal (Bridge = communication only)
|
|
27
|
+
- Context Embeddings absorbed into Crystal (Crystal captures conversations directly)
|
|
28
|
+
|
|
29
|
+
## Next up (not blocking ship)
|
|
30
|
+
|
|
31
|
+
- [ ] Absorb lesa-bridge into Memory Crystal as Bridge feature
|
|
32
|
+
- [ ] Absorb context-embeddings into Memory Crystal (retire plugin)
|
|
33
|
+
- [ ] Cloudflare relay setup (key, R2 bucket, Worker deploy, env vars)
|
|
34
|
+
- [ ] Remove LanceDB dual-write
|
|
35
|
+
- [ ] `crystal backfill` for ~115 historical sessions
|
|
36
|
+
- [ ] Point Dream Weaver at `~/.ldm/` transcripts
|
|
37
|
+
|
|
38
|
+
## Completed
|
|
39
|
+
|
|
40
|
+
- [x] Push `mini/phase2-relay` ... 2026-02-26
|
|
41
|
+
- [x] Create PR #2, squash-merge to main ... 2026-02-26
|
|
42
|
+
- [x] Deploy build to `~/.ldm/extensions/memory-crystal/` ... 2026-02-27
|
|
43
|
+
- [x] Migrate crystal.db to `~/.ldm/memory/crystal.db` (159K+ chunks, symlink at old path) ... 2026-02-26
|
|
44
|
+
- [x] Fix lesa-bridge session routing (`user: "claude-code"` for dedicated TUI-visible session) ... 2026-02-27
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# ai/todos/
|
|
2
|
+
|
|
3
|
+
Agent-to-agent and agent-to-human task inbox system.
|
|
4
|
+
|
|
5
|
+
## How it works
|
|
6
|
+
|
|
7
|
+
Each folder under `inboxes/` is a recipient. When an agent (or human) has action items for someone else, they drop a dated markdown file in that person's inbox.
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
inboxes/
|
|
11
|
+
parker/ Parker's action items (manual steps, deploys, config)
|
|
12
|
+
cc-mini/ Claude Code on Mac Mini
|
|
13
|
+
cc-air/ Claude Code on MacBook Air
|
|
14
|
+
lesa/ (create when needed)
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
## Naming convention
|
|
18
|
+
|
|
19
|
+
```
|
|
20
|
+
YYYY-MM-DD--{from-agent}--{short-description}.md
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
Examples:
|
|
24
|
+
- `2026-02-26--cc-mini--phase2-your-moves.md` ... cc-mini telling Parker what to do
|
|
25
|
+
- `2026-02-25--cc-air--setup-checklist.md` ... cc-air telling Parker what to set up
|
|
26
|
+
|
|
27
|
+
## Rules
|
|
28
|
+
|
|
29
|
+
- **Drop, don't edit.** If you have updates, drop a new file. Don't modify someone else's inbox items.
|
|
30
|
+
- **Done items stay.** Check the boxes in the file. Don't delete completed files... they're history.
|
|
31
|
+
- **Keep it short.** Link to plans or docs for details. The inbox item is the "what", not the "why".
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# cc-air — Memory Crystal Todos
|
|
2
|
+
|
|
3
|
+
**Date:** 2026-02-26 (updated)
|
|
4
|
+
**Agent:** cc-air
|
|
5
|
+
**Context:** Full ecosystem review — memory docs, Dream Weaver, LDM OS, OpenClaw builtin, QMD
|
|
6
|
+
|
|
7
|
+
## Priority 1 — Three File Types (the core gap)
|
|
8
|
+
|
|
9
|
+
Memory Crystal only produces the vector DB today. It needs all three:
|
|
10
|
+
|
|
11
|
+
- [ ] **JSONL archive** — cc-hook copies raw JSONL to `~/.ldm/agents/{agent_id}/memory/transcripts/` after each capture. Solves the FDA backup problem. Gives Dream Weaver its primary input in a controlled location.
|
|
12
|
+
|
|
13
|
+
- [ ] **MD session summaries** — cc-hook generates human-readable markdown per conversation. Write to `~/.ldm/agents/{agent_id}/memory/sessions/YYYY-MM-DD-{slug}.md`. Matches OpenClaw's session-memory hook output but works independently.
|
|
14
|
+
|
|
15
|
+
- [ ] **crystal.db location** — Move or symlink crystal.db to `~/.ldm/memory/crystal.db` (shared, root level). Currently at `~/.openclaw/memory-crystal/`. The DB is shared by all agents (cc-mini, lesa-mini, cc-air), tagged by agent_id. On remote devices, a read-only mirror lives at the same path.
|
|
16
|
+
|
|
17
|
+
## Priority 2 — LDM Scaffolding (built into Memory Crystal)
|
|
18
|
+
|
|
19
|
+
Memory Crystal must set up `~/.ldm/` itself. No dependency on wip-ldm-os repo.
|
|
20
|
+
|
|
21
|
+
- [ ] **`crystal init` / auto-scaffold** — On first run, create the full directory structure:
|
|
22
|
+
```
|
|
23
|
+
~/.ldm/
|
|
24
|
+
├── config.json
|
|
25
|
+
├── memory/crystal.db (shared vector DB)
|
|
26
|
+
└── agents/{agent_id}/memory/
|
|
27
|
+
├── transcripts/ sessions/ daily/ journals/
|
|
28
|
+
```
|
|
29
|
+
- [ ] **Agent names by machine:** `cc-mini`, `lesa-mini`, `cc-air` (set via `CRYSTAL_AGENT_ID`)
|
|
30
|
+
- [ ] **Absorb wip-ldm-os scaffolding** — Port relevant parts of `scaffold.sh` and templates into Memory Crystal's installer. Identity files (SOUL.md etc.) can be empty templates or skipped.
|
|
31
|
+
- [ ] **Config** — agent name, harness type, Dream Weaver schedule in `~/.ldm/agents/{agent_id}/config.json`
|
|
32
|
+
|
|
33
|
+
## Priority 3 — Deploy Ephemeral Relay + Expand Poller
|
|
34
|
+
|
|
35
|
+
See Parker's setup checklist at `ai/todos/parker/2026-02-25--cc-air--setup-checklist.md`.
|
|
36
|
+
|
|
37
|
+
- [ ] Generate encryption key + copy to both machines
|
|
38
|
+
- [ ] `wrangler login`, R2 bucket, bearer tokens, deploy
|
|
39
|
+
- [ ] Configure env vars, end-to-end test
|
|
40
|
+
|
|
41
|
+
**Poller expansion** — After decrypting relay data, the Mini-side poller must also reconstruct the remote agent's full file tree:
|
|
42
|
+
- [ ] Write JSONL to `~/.ldm/agents/{agent_id}/memory/transcripts/`
|
|
43
|
+
- [ ] Generate MD summary to `~/.ldm/agents/{agent_id}/memory/sessions/`
|
|
44
|
+
- [ ] Append daily breadcrumb to `~/.ldm/agents/{agent_id}/memory/daily/`
|
|
45
|
+
- [ ] This ensures the Mini has everything — if a device is lost, Dream Weaver can still run against that agent
|
|
46
|
+
|
|
47
|
+
## Priority 4 — Fix Broken Systems
|
|
48
|
+
|
|
49
|
+
- [ ] **Retire context-embeddings** — Once Crystal handles all three file types, disable in dot-openclaw. Update lesa-bridge to read from crystal.db.
|
|
50
|
+
- [ ] **LanceDB removal** — Remove dual-write once sqlite-vec is validated.
|
|
51
|
+
- [ ] **Backup system** — Partially solved by JSONL copy (Priority 1). Full fix: grant FDA or alternative.
|
|
52
|
+
|
|
53
|
+
## Priority 5 — Recovery & Backfill
|
|
54
|
+
|
|
55
|
+
- [ ] **`crystal replay`** — Re-send from raw JSONL through relay (recovery when Mini offline >24h).
|
|
56
|
+
- [ ] **`crystal backfill`** — One-time ingest of ~115 historical CC sessions (~110MB, ~$0.07).
|
|
57
|
+
|
|
58
|
+
## Priority 6 — Dream Weaver Integration
|
|
59
|
+
|
|
60
|
+
- [ ] **Dream Weaver reads from `~/.ldm/`** — Once JSONL copies live there, point Dream Weaver at `transcripts/` instead of `~/.claude/projects/`.
|
|
61
|
+
- [ ] **Trigger consolidation** — After poller ingests a batch, optionally trigger incremental Dream Weaver run.
|
|
62
|
+
- [ ] **Automate weekly schedule** — Currently manual. Add cron/launchd trigger.
|
|
63
|
+
|
|
64
|
+
## Priority 7 — Search Quality (QMD Phases 3–5)
|
|
65
|
+
|
|
66
|
+
- [ ] **Phase 3: Smart chunking + dedup** — Content-hash dedup, better non-conversation chunking.
|
|
67
|
+
- [ ] **Phase 4: Re-ranking + query expansion** — LLM re-ranking, synonym expansion.
|
|
68
|
+
- [ ] **Phase 5: Local embeddings** — Replace OpenAI with local GGUF model.
|
|
69
|
+
|
|
70
|
+
## Priority 8 — Behavioral / Process
|
|
71
|
+
|
|
72
|
+
- [ ] **SHARED-CONTEXT.md** — Warm-start file agents read on boot.
|
|
73
|
+
- [ ] **End-of-session handoffs** — Agent writes what-I-was-doing summary.
|
|
74
|
+
- [ ] **Search-before-acting** — Agents search crystal before starting new work.
|
|
75
|
+
|
|
76
|
+
## Notes
|
|
77
|
+
|
|
78
|
+
- **Agents are per-harness-instance, named by machine:** cc-mini, lesa-mini, cc-air, etc. Each is a separate "user" in LDM.
|
|
79
|
+
- **crystal.db is shared** at `~/.ldm/memory/crystal.db` — not inside any agent folder. All agents write to it (tagged by agent_id).
|
|
80
|
+
- **The Mini has everything.** Remote agents' file trees are reconstructed from relay data. Dream Weaver can run against any agent from the Mini.
|
|
81
|
+
- OpenClaw's builtin memory stays on — it evolves independently, doesn't hurt to run both.
|
|
82
|
+
- QMD is a separate system. Its search algorithms are already ported into Memory Crystal.
|
|
83
|
+
- Private mode is fully implemented across all 4 surfaces.
|
|
84
|
+
- The relay code is built and builds clean, just needs physical deployment.
|
|
85
|
+
- Memory Crystal has zero external repo dependencies for core function. OpenClaw, CC hook, relay, and Dream Weaver are optional integrations.
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
# cc-mini -- Memory Crystal Phase 2 Status
|
|
2
|
+
|
|
3
|
+
**Date:** 2026-02-26
|
|
4
|
+
**Agent:** cc-mini
|
|
5
|
+
**Branch:** `mini/phase2-relay`
|
|
6
|
+
|
|
7
|
+
## What's Done
|
|
8
|
+
|
|
9
|
+
### Priority 1 -- Three File Types
|
|
10
|
+
|
|
11
|
+
- [x] **JSONL archive** -- cc-hook copies raw JSONL to `~/.ldm/agents/{agent_id}/memory/transcripts/` after each capture. Uses mtime check to skip redundant copies.
|
|
12
|
+
- [x] **MD session summaries** -- cc-hook generates markdown per conversation after ingest. Two modes: `simple` (no API call) and `llm` (gpt-4o-mini). Written to `~/.ldm/agents/{agent_id}/memory/sessions/YYYY-MM-DD--HH-MM-SS--{agent}--{slug}.md`.
|
|
13
|
+
- [x] **crystal.db path resolution** -- `resolveConfig()` checks `~/.ldm/memory/crystal.db` first, falls back to legacy `~/.openclaw/memory-crystal/`. Auto-detection means old deployments keep working.
|
|
14
|
+
- [x] **`crystal migrate-db` CLI** -- Copies crystal.db to `~/.ldm/memory/`, verifies chunk count, creates symlink at old path. Never deletes the original.
|
|
15
|
+
|
|
16
|
+
### Priority 2 -- LDM Scaffolding
|
|
17
|
+
|
|
18
|
+
- [x] **`src/ldm.ts` module** -- Central path resolution. `getAgentId()`, `ldmPaths()`, `scaffoldLdm()`, `ensureLdm()`.
|
|
19
|
+
- [x] **`crystal init [--agent]`** -- Creates full `~/.ldm/` directory tree, writes/updates config.json with agents array.
|
|
20
|
+
- [x] **Agent naming** -- Via `CRYSTAL_AGENT_ID` env var. Defaults to `cc-mini`.
|
|
21
|
+
- [x] **appendDailyLog()** -- Refactored to use `ldmPaths()` instead of hardcoded constant. Importable for reuse.
|
|
22
|
+
|
|
23
|
+
### Priority 3 -- Relay Merge
|
|
24
|
+
|
|
25
|
+
- [x] **Merged `cc-air/phase2-relay`** into `mini/phase2-relay`. Conflicts resolved (cc-hook.ts imports + dev-update section).
|
|
26
|
+
- [x] **Poller expanded** -- After decrypting relay data, reconstructs remote agent's full file tree: JSONL transcript, MD summary, daily breadcrumb.
|
|
27
|
+
- [x] **mirror-sync.ts** -- Updated paths to use `ldmPaths()`.
|
|
28
|
+
- [x] **crypto.ts** -- Fixed HOME default.
|
|
29
|
+
- [x] **All HOME fallbacks** -- Replaced `/Users/lesa` and `/Users/parker` with `process.env.HOME || ''` across all source files. Safe for others to install.
|
|
30
|
+
|
|
31
|
+
### Other
|
|
32
|
+
|
|
33
|
+
- [x] **dev-update.ts** -- Writes to each repo's `ai/` folder (decentralized). Disabled centralized wip-dev-updates git push.
|
|
34
|
+
- [x] **mcp-server.ts** -- Uses `createCrystal()` factory. Remote mode guards on source indexing operations.
|
|
35
|
+
- [x] **Build passes** -- All 12 entry points compile clean. `crystal status`, `crystal search`, `crystal init`, `poller --status`, `mirror-sync --status` all verified.
|
|
36
|
+
|
|
37
|
+
## What's Next
|
|
38
|
+
|
|
39
|
+
### Parker Manual Steps (blockers for relay)
|
|
40
|
+
|
|
41
|
+
- [ ] Generate encryption key: `openssl rand -base64 32 > ~/.openclaw/secrets/crystal-relay-key && chmod 600 ~/.openclaw/secrets/crystal-relay-key`
|
|
42
|
+
- [ ] Copy key to MacBook Air (same path)
|
|
43
|
+
- [ ] Cloudflare: `wrangler login`, create R2 bucket `memory-crystal-relay`, set secrets (bearer tokens), deploy Worker
|
|
44
|
+
- [ ] Set env vars on both machines: `CRYSTAL_RELAY_URL`, `CRYSTAL_RELAY_TOKEN`, `CRYSTAL_AGENT_ID`
|
|
45
|
+
- [ ] End-to-end test: drop from Air, verify Mini ingests
|
|
46
|
+
- [ ] See full checklist: `ai/todos/parker/2026-02-25--cc-air--setup-checklist.md`
|
|
47
|
+
|
|
48
|
+
### Run migrate-db
|
|
49
|
+
|
|
50
|
+
- [ ] Stop gateway: `openclaw gateway stop`
|
|
51
|
+
- [ ] Run: `crystal migrate-db`
|
|
52
|
+
- [ ] Restart: `openclaw gateway restart`
|
|
53
|
+
- [ ] Verify: `crystal status` shows same chunk count from new path
|
|
54
|
+
|
|
55
|
+
### Deploy Updated Plugin
|
|
56
|
+
|
|
57
|
+
- [ ] Copy built files to extension dir and restart gateway:
|
|
58
|
+
```
|
|
59
|
+
cd /path/to/memory-crystal
|
|
60
|
+
npm run build
|
|
61
|
+
cp -r dist skills openclaw.plugin.json package.json ~/.openclaw/extensions/memory-crystal/
|
|
62
|
+
cd ~/.openclaw/extensions/memory-crystal && npm install --omit=dev
|
|
63
|
+
openclaw gateway restart
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
### Priority 4 -- Fix Broken Systems
|
|
67
|
+
|
|
68
|
+
- [ ] **Retire context-embeddings** -- Disable once Crystal handles all three file types. Update lesa-bridge.
|
|
69
|
+
- [ ] **LanceDB removal** -- Remove dual-write once sqlite-vec is validated long enough.
|
|
70
|
+
- [ ] **Backup system** -- Partially solved by JSONL copy. Full fix: grant FDA or alternative.
|
|
71
|
+
|
|
72
|
+
### Priority 5 -- Recovery & Backfill
|
|
73
|
+
|
|
74
|
+
- [ ] **`crystal replay`** -- Re-send from raw JSONL through relay (recovery when Mini offline >24h).
|
|
75
|
+
- [ ] **`crystal backfill`** -- One-time ingest of ~115 historical CC sessions (~110MB, ~$0.07).
|
|
76
|
+
|
|
77
|
+
### Priority 6 -- Dream Weaver Integration
|
|
78
|
+
|
|
79
|
+
- [ ] Point Dream Weaver at `~/.ldm/agents/{id}/memory/transcripts/` instead of `~/.claude/projects/`.
|
|
80
|
+
- [ ] Optional: trigger incremental DW run after poller batch.
|
|
81
|
+
- [ ] Automate weekly schedule via cron/launchd.
|
|
82
|
+
|
|
83
|
+
### Priority 7 -- Search Quality (QMD Phases 3-5)
|
|
84
|
+
|
|
85
|
+
- [ ] Smart chunking + dedup
|
|
86
|
+
- [ ] LLM re-ranking + query expansion
|
|
87
|
+
- [ ] Local embeddings (replace OpenAI with local GGUF)
|
|
88
|
+
|
|
89
|
+
### Priority 8 -- Behavioral / Process
|
|
90
|
+
|
|
91
|
+
- [ ] SHARED-CONTEXT.md warm-start
|
|
92
|
+
- [ ] End-of-session handoff summaries
|
|
93
|
+
- [ ] Search-before-acting convention
|
|
94
|
+
|
|
95
|
+
## Notes
|
|
96
|
+
|
|
97
|
+
- crystal.db stays at `~/.openclaw/memory-crystal/` until `crystal migrate-db` is run. Symlink preserves backward compat.
|
|
98
|
+
- Summary mode defaults to `simple` (no API call). Set `CRYSTAL_SUMMARY_MODE=llm` for gpt-4o-mini summaries.
|
|
99
|
+
- The relay code builds clean but isn't deployed yet. Needs Parker's Cloudflare setup.
|
|
100
|
+
- 159,574 chunks in crystal.db as of this commit. All search and status commands verified working.
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# Memory Crystal — TODO
|
|
2
|
+
|
|
3
|
+
## Phase 1 — Done
|
|
4
|
+
All four doors live, Lēsa confirmed, 5,543+ chunks and growing.
|
|
5
|
+
|
|
6
|
+
### Phase 1 cleanup (small)
|
|
7
|
+
- [ ] `npm link` to install `crystal` CLI globally
|
|
8
|
+
- [ ] Disable `context-embeddings` plugin (saves double embedding cost per turn) — or keep both for a few more days
|
|
9
|
+
- [ ] Remove `conversation_search`/`memory_search` from lesa-bridge (crystal replaces them)
|
|
10
|
+
|
|
11
|
+
## Phase 2 — Cloudflare Worker mirror (big)
|
|
12
|
+
- [ ] `worker.ts` — REST API backed by D1 + Vectorize + R2
|
|
13
|
+
- [ ] `crystal push` / `crystal pull` / `crystal reset` sync commands
|
|
14
|
+
- [ ] Daily cron: mini uploads, Worker rebuilds, mini pulls new writes
|
|
15
|
+
- [ ] **Needs:** Parker's Cloudflare account + `wrangler login`
|
|
16
|
+
|
|
17
|
+
## Phase 3 — Remote multi-agent access
|
|
18
|
+
- [ ] Remote MCP for Claude.ai web
|
|
19
|
+
- [ ] OpenAPI spec for ChatGPT/Grok
|
|
20
|
+
- [ ] `crystal search --remote` from anywhere
|
|
21
|
+
|
|
22
|
+
## Other stuff on the board
|
|
23
|
+
- [ ] GitHub setup (Lēsa's account)
|
|
24
|
+
- [ ] Repo renames
|
|
25
|
+
- [ ] cc-persistence (Pi RPC sidecar)
|
|
@@ -0,0 +1,139 @@
|
|
|
1
|
+
# Parker — Setup Checklist (Ephemeral Relay)
|
|
2
|
+
|
|
3
|
+
Everything you need to do to get the encrypted relay live.
|
|
4
|
+
|
|
5
|
+
## 1. Generate Encryption Key (once, shared across machines)
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
openssl rand -base64 32 > ~/.openclaw/secrets/crystal-relay-key
|
|
9
|
+
chmod 600 ~/.openclaw/secrets/crystal-relay-key
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
Copy `~/.openclaw/secrets/crystal-relay-key` to both machines (Air + Mini).
|
|
13
|
+
This key never goes to Cloudflare. It stays on your hardware.
|
|
14
|
+
|
|
15
|
+
**How to copy:**
|
|
16
|
+
- AirDrop the file
|
|
17
|
+
- Or from one machine: `cat ~/.openclaw/secrets/crystal-relay-key` → paste on the other
|
|
18
|
+
- Or copy via the FireWire shared disk
|
|
19
|
+
|
|
20
|
+
## 2. 1Password Service Account Token (MacBook Air)
|
|
21
|
+
|
|
22
|
+
The Air needs the same service account token that's on the Mini.
|
|
23
|
+
|
|
24
|
+
- [ ] Copy `~/.openclaw/secrets/op-sa-token` from the Mac Mini
|
|
25
|
+
- [ ] Paste it to `~/.openclaw/secrets/op-sa-token` on the MacBook Air
|
|
26
|
+
- [ ] Run `chmod 600 ~/.openclaw/secrets/op-sa-token`
|
|
27
|
+
|
|
28
|
+
## 3. Cloudflare Setup
|
|
29
|
+
|
|
30
|
+
- [ ] Install wrangler (if not already): `npm install -g wrangler`
|
|
31
|
+
- [ ] Run `wrangler login` (opens browser, authenticates)
|
|
32
|
+
|
|
33
|
+
### Create R2 Bucket
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
cd /Users/parker/Documents/dev-wip/repos/memory-crystal
|
|
37
|
+
wrangler r2 bucket create memory-crystal-relay
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
### Set Secrets
|
|
41
|
+
|
|
42
|
+
Generate three bearer tokens (one per agent):
|
|
43
|
+
```bash
|
|
44
|
+
openssl rand -hex 32
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
- [ ] Set cc-air token: `wrangler secret put AUTH_TOKEN_CC_AIR`
|
|
48
|
+
- [ ] Set cc-mini token: `wrangler secret put AUTH_TOKEN_CC_MINI`
|
|
49
|
+
- [ ] Set lēsa token: `wrangler secret put AUTH_TOKEN_LESA`
|
|
50
|
+
|
|
51
|
+
**Save the cc-air token** — you'll need it for the Air's env config.
|
|
52
|
+
**Save the cc-mini token** — you'll need it for the Mini's poller config.
|
|
53
|
+
|
|
54
|
+
### Deploy
|
|
55
|
+
|
|
56
|
+
- [ ] Deploy the Worker:
|
|
57
|
+
```bash
|
|
58
|
+
wrangler deploy
|
|
59
|
+
```
|
|
60
|
+
Note the URL (e.g., `https://memory-crystal-relay.wipcomputer.workers.dev`)
|
|
61
|
+
|
|
62
|
+
### Verify
|
|
63
|
+
|
|
64
|
+
- [ ] Test health endpoint:
|
|
65
|
+
```bash
|
|
66
|
+
curl https://memory-crystal-relay.wipcomputer.workers.dev/health
|
|
67
|
+
```
|
|
68
|
+
Should return: `{"ok":true,"service":"memory-crystal-relay","mode":"ephemeral"}`
|
|
69
|
+
|
|
70
|
+
## 4. Configure MacBook Air (cc-air)
|
|
71
|
+
|
|
72
|
+
Set these environment variables (in `~/.zshrc` or Claude Code hook config):
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
export CRYSTAL_RELAY_URL="https://memory-crystal-relay.wipcomputer.workers.dev"
|
|
76
|
+
export CRYSTAL_RELAY_TOKEN="<cc-air bearer token from step 3>"
|
|
77
|
+
export CRYSTAL_AGENT_ID="cc-air"
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
The cc-hook will auto-detect relay mode when these are set.
|
|
81
|
+
|
|
82
|
+
## 5. Configure Mac Mini (cc-mini poller)
|
|
83
|
+
|
|
84
|
+
Set these environment variables:
|
|
85
|
+
|
|
86
|
+
```bash
|
|
87
|
+
export CRYSTAL_RELAY_URL="https://memory-crystal-relay.wipcomputer.workers.dev"
|
|
88
|
+
export CRYSTAL_RELAY_TOKEN="<cc-mini bearer token from step 3>"
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
### Set up poller (cron or launchd):
|
|
92
|
+
|
|
93
|
+
**Option A: Cron (every 2 min)**
|
|
94
|
+
```bash
|
|
95
|
+
crontab -e
|
|
96
|
+
# Add:
|
|
97
|
+
*/2 * * * * cd /path/to/memory-crystal && node dist/poller.js 2>> ~/.openclaw/logs/relay-poller.log
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
**Option B: LaunchAgent (recommended)**
|
|
101
|
+
Create a plist for continuous polling with `--watch` mode.
|
|
102
|
+
|
|
103
|
+
**Option C: Manual**
|
|
104
|
+
```bash
|
|
105
|
+
node dist/poller.js # poll once
|
|
106
|
+
node dist/poller.js --watch # poll continuously
|
|
107
|
+
node dist/poller.js --push-mirror # push mirror snapshot
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
## 6. Verify End-to-End
|
|
111
|
+
|
|
112
|
+
1. Talk to cc-air on the MacBook Air
|
|
113
|
+
2. Check relay has a blob: `curl -H "Authorization: Bearer <mini-token>" https://memory-crystal-relay.wipcomputer.workers.dev/pickup/conversations`
|
|
114
|
+
3. Run the poller on Mini: `node dist/poller.js`
|
|
115
|
+
4. Verify chunks ingested into master crystal: `node dist/cli.js status`
|
|
116
|
+
5. Push mirror: `node dist/poller.js --push-mirror`
|
|
117
|
+
6. Pull mirror on Air: `node dist/mirror-sync.js`
|
|
118
|
+
7. Search on Air — should return results from the conversation
|
|
119
|
+
|
|
120
|
+
## Quick Reference
|
|
121
|
+
|
|
122
|
+
| What | Where |
|
|
123
|
+
|------|-------|
|
|
124
|
+
| Encryption key | `~/.openclaw/secrets/crystal-relay-key` (both machines) |
|
|
125
|
+
| SA token | `~/.openclaw/secrets/op-sa-token` (both machines) |
|
|
126
|
+
| Repo | `/Users/parker/Documents/dev-wip/repos/memory-crystal` |
|
|
127
|
+
| Branch | `cc-air/phase2-relay` |
|
|
128
|
+
| Worker config | `wrangler.toml` |
|
|
129
|
+
| Air env vars | `CRYSTAL_RELAY_URL`, `CRYSTAL_RELAY_TOKEN`, `CRYSTAL_AGENT_ID` |
|
|
130
|
+
| Mini env vars | `CRYSTAL_RELAY_URL`, `CRYSTAL_RELAY_TOKEN` |
|
|
131
|
+
|
|
132
|
+
## What's NOT on Cloudflare
|
|
133
|
+
|
|
134
|
+
- No encryption keys
|
|
135
|
+
- No API keys
|
|
136
|
+
- No database
|
|
137
|
+
- No search capability
|
|
138
|
+
- No ability to read the data
|
|
139
|
+
- Just auth tokens and encrypted blobs that auto-expire in 24h
|