@wipcomputer/memory-crystal 0.7.22 → 0.7.24

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/CHANGELOG.md +127 -0
  2. package/README.md +28 -4
  3. package/_trash/RELEASE-NOTES-v0-7-23.md +48 -0
  4. package/package.json +1 -1
package/CHANGELOG.md CHANGED
@@ -19,6 +19,133 @@
19
19
 
20
20
 
21
21
 
22
+ ## 0.7.24 (2026-03-15)
23
+
24
+ # Dev Update: Search Quality v2 + MLX Local LLM
25
+
26
+ **Date:** 2026-03-15
27
+ **Author:** CC-Mini
28
+ **Session:** memory-crstal01 (continued from Mar 13-14)
29
+
30
+ ---
31
+
32
+ ## Summary
33
+
34
+ Six search quality features from QMD v2.0 analysis, plus MLX local LLM infrastructure for Apple Silicon. All coded, tested, merged. Not yet deployed.
35
+
36
+ ## What Shipped
37
+
38
+ ### Search Quality (PR #75)
39
+
40
+ 1. **Intent parameter.** Disambiguates queries without adding search terms. `crystal search "security" --intent "1Password"` steers toward 1Password results. Flows through expansion prompt (guides LLM variations), disables strong-signal bypass, prepended to rerank query. Available via CLI `--intent`, MCP `intent`.
41
+
42
+ 2. **candidateLimit.** Tunable rerank pool size. `crystal search "query" --candidates 60`. Default stays 40. More candidates = better recall, slower reranking. Available via CLI `--candidates`, MCP `candidate_limit`.
43
+
44
+ 3. **Explain mode.** Per-result scoring breakdown showing FTS score, vector score, RRF rank, reranker score, recency weight, and final blended score. `crystal search "query" --explain`. Available via CLI `--explain`, MCP `explain`.
45
+
46
+ 4. **Persistent LLM cache.** `llm_cache` table in crystal.db. Expansion and reranking results cached with 7-day TTL. Content-addressable reranking (keyed by query + sorted passage hashes). Same query = instant on repeat searches. Configurable TTL via `CRYSTAL_CACHE_TTL_DAYS`.
47
+
48
+ 5. **Structured search API.** `crystal.structuredSearch(queries)` accepts pre-expanded StructuredQuery[] (lex, vec, hyde). Skips LLM expansion entirely. Agents construct their own queries when they know what they want. RRF fusion with first list weighted 2x.
49
+
50
+ ### MLX Local LLM (PR #76)
51
+
52
+ 6. **MLX auto-install.** New `src/mlx-setup.ts` with full setup flow:
53
+ - `detectPlatform()` ... Apple Silicon / Intel Mac / Linux / other
54
+ - `installMlxLm()` ... uv > pip3 > pip3 --user fallback chain
55
+ - `createLaunchAgent()` ... always-on MLX server via LaunchAgent
56
+ - `verifyServer()` ... 30s warmup wait for model loading
57
+ - `setupMlx()` ... full flow: detect, install, configure, start, verify
58
+
59
+ 7. **Crystal MLX CLI.** `crystal mlx setup/status/stop` subcommands.
60
+
61
+ 8. **Doctor check #13.** MLX health check with three states: not installed, installed but not running, running. Suggests fix for each.
62
+
63
+ 9. **Installer integration.** `crystal init` detects Apple Silicon and suggests `crystal mlx setup` when MLX is not installed.
64
+
65
+ 10. **Port 18791.** LDM service ports: 18789 (OpenClaw), 18790 (Crystal Core), 18791 (MLX LLM).
66
+
67
+ 11. **Model: Qwen 2.5 3B Instruct 4-bit.** `mlx-community/Qwen2.5-3B-Instruct-4bit`. ~1.5 GB, fast on M-series, good at instruction following for query expansion and relevance scoring.
68
+
69
+ ### Also
70
+
71
+ - QMD v2.0 analysis written (`ai/product/notes/2026-03-15--cc-mini--qmd-v2.0-analysis.md`)
72
+ - Search quality plan written (`ai/product/plans-prds/current/2026-03-15--cc-mini--search-quality-qmd-v2-port.md`)
73
+ - MLX plan moved from upcoming to current
74
+ - Stashed roadmap + readme-first updates recovered and committed (PR #74)
75
+ - README footer: QMD credit restored, CLA + dual license confirmed on both repos
76
+
77
+ ## Files Changed
78
+
79
+ | File | Change |
80
+ |------|--------|
81
+ | `src/search-pipeline.ts` | Intent support, candidateLimit param, explain traces, DeepSearchResult type |
82
+ | `src/llm.ts` | Intent in expansion prompt, persistent DB cache (expansion + reranking), setLLMCacheDb() |
83
+ | `src/core.ts` | llm_cache table schema, deepSearch options, structuredSearch() method, StructuredQuery type |
84
+ | `src/mcp-server.ts` | intent, candidate_limit, explain params on crystal_search, LLM cache DB wiring |
85
+ | `src/cli.ts` | --intent, --candidates, --explain flags, crystal mlx subcommand |
86
+ | `src/mlx-setup.ts` | **NEW** ... full MLX setup, doctor check, state management |
87
+ | `src/doctor.ts` | MLX health check (#13) |
88
+ | `src/installer.ts` | MLX detection in crystal init flow |
89
+
90
+ ## What This Enables
91
+
92
+ - **Free deep search.** MLX replaces OpenAI API calls for expansion + reranking. Zero cost per search.
93
+ - **Faster repeated searches.** Persistent cache means the LLM call happens once per unique query.
94
+ - **Smarter agent queries.** Structured search lets agents skip expansion when they know what they want.
95
+ - **Debuggable search.** Explain mode shows exactly why each result ranked where it did.
96
+ - **Offline search quality.** MLX works without internet. API fallback when MLX is down.
97
+
98
+ ## 0.7.23 (2026-03-15)
99
+
100
+ # Release Notes: Memory Crystal v0.7.23
101
+
102
+ **Date:** 2026-03-15
103
+
104
+ ## Search Quality v2 + MLX Local LLM
105
+
106
+ This release adds six search quality features ported from the QMD v2.0 analysis, plus the complete MLX local LLM infrastructure for Apple Silicon. Deep search is now disambiguatable, cacheable, debuggable, and can run entirely offline on Apple Silicon.
107
+
108
+ ### Intent parameter
109
+
110
+ Disambiguates queries without adding search terms. `crystal search "security" --intent "1Password"` steers results toward 1Password-related security instead of repo permissions or agent secrets. Intent flows through the expansion prompt (guides LLM variations), disables strong-signal bypass (keyword match might not be what the caller wants), and is prepended to the rerank query. Available via CLI `--intent` and MCP `intent`.
111
+
112
+ ### Persistent LLM cache
113
+
114
+ Expansion and reranking results are now cached in crystal.db (`llm_cache` table) with a 7-day TTL. Same query = instant on repeat searches. Reranking cache is content-addressable (keyed by query + sorted passage hashes), so identical content from different sessions shares cached scores. Configurable via `CRYSTAL_CACHE_TTL_DAYS` env var.
115
+
116
+ ### Explain mode
117
+
118
+ Per-result scoring breakdown showing FTS score, vector score, RRF rank, reranker score, recency weight, and final blended score. `crystal search "query" --explain`. Available via CLI `--explain` and MCP `explain`. Makes search quality transparent and debuggable.
119
+
120
+ ### candidateLimit
121
+
122
+ Tunable rerank pool size. `crystal search "query" --candidates 60`. Default stays 40. More candidates = better recall, slower reranking. Available via CLI `--candidates` and MCP `candidate_limit`.
123
+
124
+ ### Structured search API
125
+
126
+ `crystal.structuredSearch(queries)` accepts pre-expanded StructuredQuery[] with typed sub-queries (lex, vec, hyde). Skips LLM expansion entirely. Agents construct their own queries when they already know what they want. RRF fusion with first list weighted 2x.
127
+
128
+ ### MLX local LLM (Phase 3)
129
+
130
+ Complete auto-install infrastructure for running a local LLM on Apple Silicon:
131
+
132
+ - `crystal mlx setup` detects Apple Silicon, installs mlx-lm (uv > pip3 > pip3 --user), creates LaunchAgent for always-on server
133
+ - Model: `mlx-community/Qwen2.5-3B-Instruct-4bit` (~1.5 GB, fast on M-series)
134
+ - Port 18791 (18789 OpenClaw, 18790 Crystal Core, 18791 MLX)
135
+ - `crystal mlx status` and `crystal mlx stop` for server management
136
+ - `crystal doctor` check #13: MLX health (not installed / down / running)
137
+ - `crystal init` detects Apple Silicon and suggests MLX setup
138
+ - State file at `~/.ldm/state/mlx-server.json`
139
+
140
+ ### Also in this release
141
+
142
+ - QMD v2.0 analysis documented (`ai/product/notes/`)
143
+ - Search quality plan written (`ai/product/plans-prds/current/`)
144
+ - MLX plan moved from upcoming to current
145
+ - Stashed roadmap + readme-first updates recovered (PR #74)
146
+
147
+ Closes #57, #63, #64.
148
+
22
149
  ## 0.7.22 (2026-03-14)
23
150
 
24
151
  Remove standalone role
package/README.md CHANGED
@@ -115,13 +115,37 @@ Run `ldm install` to see other components you can add.
115
115
 
116
116
  ## License
117
117
 
118
+ Dual-license model designed to keep tools free while preventing commercial resellers.
119
+
118
120
  ```
119
- src/core.ts, cli.ts, mcp-server.ts, skills/ MIT (use anywhere, no restrictions)
120
- src/worker.ts, src/worker-mcp.ts AGPL (relay + cloud server)
121
+ MIT All CLI tools, MCP servers, skills, and hooks (use anywhere, no restrictions).
122
+ AGPLv3 Commercial redistribution, marketplace listings, or bundling into paid services.
121
123
  ```
122
124
 
123
- AGPL for personal use is free.
125
+ AGPLv3 for personal use is free. Commercial licenses available.
126
+
127
+ ### Can I use this?
128
+
129
+ **Yes, freely:**
130
+ - Use any tool locally or on your own servers
131
+ - Modify the code for your own projects
132
+ - Include in your internal CI/CD pipelines
133
+ - Fork it and send us feedback via PRs (we'd love that)
134
+
135
+ **Need a commercial license:**
136
+ - Bundle into a product you sell
137
+ - List on a marketplace (Claude Marketplace, OAI GPT/Apps, Clawhub.ai, VS Code, etc.)
138
+ - Offer as part of a hosted/SaaS platform
139
+ - Redistribute commercially
124
140
 
125
- Built by Parker Todd Brooks, Lēsa (OpenClaw, Claude Opus 4.6), Claude Code CLI (Claude Opus 4.6).
141
+ Using these tools to build your own software is fine. Reselling the tools themselves is what requires a commercial license.
142
+
143
+ By submitting a PR, you agree to the [Contributor License Agreement](CLA.md).
144
+
145
+ ---
146
+
147
+ Built by Parker Todd Brooks, Lēsa (OpenClaw, Claude Opus 4.6), Claude Code (Claude Opus 4.6), GPT 5.x, Grok 4.20).
126
148
 
127
149
  Search architecture inspired by [QMD](https://github.com/tobi/qmd) by Tobi Lutke (MIT, 2024-2026).
150
+
151
+ *WIP.computer. Learning Dreaming Machines.*
package/_trash/RELEASE-NOTES-v0-7-23.md ADDED
@@ -0,0 +1,48 @@
1
+ # Release Notes: Memory Crystal v0.7.23
2
+
3
+ **Date:** 2026-03-15
4
+
5
+ ## Search Quality v2 + MLX Local LLM
6
+
7
+ This release adds six search quality features ported from the QMD v2.0 analysis, plus the complete MLX local LLM infrastructure for Apple Silicon. Deep search is now disambiguatable, cacheable, debuggable, and can run entirely offline on Apple Silicon.
8
+
9
+ ### Intent parameter
10
+
11
+ Disambiguates queries without adding search terms. `crystal search "security" --intent "1Password"` steers results toward 1Password-related security instead of repo permissions or agent secrets. Intent flows through the expansion prompt (guides LLM variations), disables strong-signal bypass (keyword match might not be what the caller wants), and is prepended to the rerank query. Available via CLI `--intent` and MCP `intent`.
12
+
13
+ ### Persistent LLM cache
14
+
15
+ Expansion and reranking results are now cached in crystal.db (`llm_cache` table) with a 7-day TTL. Same query = instant on repeat searches. Reranking cache is content-addressable (keyed by query + sorted passage hashes), so identical content from different sessions shares cached scores. Configurable via `CRYSTAL_CACHE_TTL_DAYS` env var.
16
+
17
+ ### Explain mode
18
+
19
+ Per-result scoring breakdown showing FTS score, vector score, RRF rank, reranker score, recency weight, and final blended score. `crystal search "query" --explain`. Available via CLI `--explain` and MCP `explain`. Makes search quality transparent and debuggable.
20
+
21
+ ### candidateLimit
22
+
23
+ Tunable rerank pool size. `crystal search "query" --candidates 60`. Default stays 40. More candidates = better recall, slower reranking. Available via CLI `--candidates` and MCP `candidate_limit`.
24
+
25
+ ### Structured search API
26
+
27
+ `crystal.structuredSearch(queries)` accepts pre-expanded StructuredQuery[] with typed sub-queries (lex, vec, hyde). Skips LLM expansion entirely. Agents construct their own queries when they already know what they want. RRF fusion with first list weighted 2x.
28
+
29
+ ### MLX local LLM (Phase 3)
30
+
31
+ Complete auto-install infrastructure for running a local LLM on Apple Silicon:
32
+
33
+ - `crystal mlx setup` detects Apple Silicon, installs mlx-lm (uv > pip3 > pip3 --user), creates LaunchAgent for always-on server
34
+ - Model: `mlx-community/Qwen2.5-3B-Instruct-4bit` (~1.5 GB, fast on M-series)
35
+ - Port 18791 (18789 OpenClaw, 18790 Crystal Core, 18791 MLX)
36
+ - `crystal mlx status` and `crystal mlx stop` for server management
37
+ - `crystal doctor` check #13: MLX health (not installed / down / running)
38
+ - `crystal init` detects Apple Silicon and suggests MLX setup
39
+ - State file at `~/.ldm/state/mlx-server.json`
40
+
41
+ ### Also in this release
42
+
43
+ - QMD v2.0 analysis documented (`ai/product/notes/`)
44
+ - Search quality plan written (`ai/product/plans-prds/current/`)
45
+ - MLX plan moved from upcoming to current
46
+ - Stashed roadmap + readme-first updates recovered (PR #74)
47
+
48
+ Closes #57, #63, #64.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@wipcomputer/memory-crystal",
3
- "version": "0.7.22",
3
+ "version": "0.7.24",
4
4
  "description": "Sovereign memory system — local-first with ephemeral encrypted relay. Your memory, your machine, your rules.",
5
5
  "type": "module",
6
6
  "main": "dist/core.js",