superlocalmemory 3.4.42 → 3.4.44
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +102 -0
- package/README.md +41 -0
- package/package.json +1 -1
- package/pyproject.toml +43 -38
- package/scripts/install.ps1 +19 -10
- package/scripts/install.sh +15 -21
- package/scripts/postinstall.js +9 -77
- package/src/superlocalmemory/__init__.py +1 -1
- package/src/superlocalmemory/cli/commands.py +57 -27
- package/src/superlocalmemory/core/embedding_worker.py +9 -8
- package/src/superlocalmemory/core/engine_wiring.py +10 -29
- package/src/superlocalmemory/hooks/before_web_hook.py +128 -0
- package/src/superlocalmemory/hooks/claude_code_hooks.py +57 -15
- package/src/superlocalmemory/hooks/hook_handlers.py +27 -15
- package/src/superlocalmemory/hooks/topic_shift_hook.py +272 -0
- package/src/superlocalmemory/server/unified_daemon.py +36 -3
package/CHANGELOG.md
CHANGED
|
@@ -9,6 +9,108 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
9
9
|
|
|
10
10
|
---
|
|
11
11
|
|
|
12
|
+
## [3.4.43] - 2026-05-12
|
|
13
|
+
|
|
14
|
+
Smart-hook architecture release. Replaces the time-based 15-minute recall
|
|
15
|
+
reminder with event-based detection that only fires when there's a real
|
|
16
|
+
signal to recall against. Adds a pre-web-search recall hook so SLM's local
|
|
17
|
+
memories are always surfaced before paying for external research.
|
|
18
|
+
|
|
19
|
+
Both additions are perf-budgeted, fail-open, and idempotent. They activate
|
|
20
|
+
on the next `slm hooks install` (or `slm init`); existing installations
|
|
21
|
+
keep working unchanged until upgraded.
|
|
22
|
+
|
|
23
|
+
### Added
|
|
24
|
+
- **`slm hook topic_shift`** — UserPromptSubmit handler that keeps a 5-prompt
|
|
25
|
+
sliding window of content-word lists per session and emits a single-line
|
|
26
|
+
recall reminder ONLY when the current prompt's content-word set has zero
|
|
27
|
+
overlap with EVERY recent prompt (the strictest defensible signal for a
|
|
28
|
+
genuine topic pivot). Per-prompt max-overlap algorithm; not jaccard-vs-union
|
|
29
|
+
which over-fires on natural conversational drift. Stdlib-only, latency
|
|
30
|
+
<10ms p99. State file at `/tmp/slm-topicstate-{sha256(session_id)[:16]}.json`,
|
|
31
|
+
auto-purged after 24h. Observability log at `~/.superlocalmemory/logs/
|
|
32
|
+
topic-shift.log` (TSV: timestamp, session_hash, current_words_count,
|
|
33
|
+
window_depth, max_overlap, fired, prompt_preview). Disable with
|
|
34
|
+
`SLM_TOPIC_SHIFT_LOG=0`. Module: `superlocalmemory/hooks/topic_shift_hook.py`.
|
|
35
|
+
- **`slm hook before_web`** — PreToolUse handler wired on
|
|
36
|
+
`matcher="WebSearch|WebFetch"`. Extracts the search query / URL / prompt
|
|
37
|
+
from Claude Code stdin, runs `slm recall <query> --limit 5`, injects
|
|
38
|
+
results as a `<system-reminder>` with the standard untrusted-boundary
|
|
39
|
+
markers so Claude reads local memory BEFORE the web call fires. Cost:
|
|
40
|
+
~500-800ms warm per fire, but only on web tool calls (5-20x per typical
|
|
41
|
+
session). Fail-open on SLM-down / timeout / empty results. Module:
|
|
42
|
+
`superlocalmemory/hooks/before_web_hook.py`.
|
|
43
|
+
- **`HOOKS_VERSION = "3.4.43"`** — bumped so `slm hooks status` flags
|
|
44
|
+
pre-3.4.43 wirings as outdated. Run `slm hooks install` to upgrade
|
|
45
|
+
to the new wiring.
|
|
46
|
+
|
|
47
|
+
### Changed
|
|
48
|
+
- **`_hook_checkpoint` periodic nag REMOVED.** The 15-minute "[SLM] 15+ min
|
|
49
|
+
since last context refresh" and 30-minute "[SLM] Call
|
|
50
|
+
mcp__superlocalmemory__get_learned_patterns" reminders previously emitted
|
|
51
|
+
by `slm hook checkpoint` are gone. Time-based reminders were noisy on
|
|
52
|
+
focused sessions and blind to quick topic pivots within a window. The
|
|
53
|
+
event-based topic_shift hook is the replacement; on-demand
|
|
54
|
+
`get_learned_patterns` MCP calls cover the learning side.
|
|
55
|
+
`_hook_checkpoint`'s real value — auto-observe on file-change events —
|
|
56
|
+
is unchanged. The `_RECALL_INTERVAL` and `_LEARN_INTERVAL` constants
|
|
57
|
+
are retained for backward import compatibility.
|
|
58
|
+
|
|
59
|
+
### Fixed
|
|
60
|
+
- **`slm mode <X>` CLI no longer clobbers embedding / retrieval / evolution /
|
|
61
|
+
forgetting / math settings.** Before this release the CLI handler called
|
|
62
|
+
`SLMConfig.for_mode(...)` passing only `llm_*` kwargs — silently
|
|
63
|
+
re-deriving every other field from mode defaults. A user with a tuned
|
|
64
|
+
cross-encoder (`cross-encoder/ms-marco-MiniLM-L-12-v2`) or a custom
|
|
65
|
+
embedding endpoint would lose their settings on every `slm mode b`.
|
|
66
|
+
The v3.4.34 `mode_change=True` guard only protected the `mode` field
|
|
67
|
+
itself; surrounding fields were lost. v3.4.43 reworks `cmd_mode` to
|
|
68
|
+
mutate only `config.mode` and save — preserving all other config
|
|
69
|
+
byte-for-byte. Mode-appropriate LLM defaults are populated ONLY when
|
|
70
|
+
the user has no provider set (so the daemon can still come up on a
|
|
71
|
+
fresh install). Tests: `tests/test_mode_switch_preservation.py` (7 new
|
|
72
|
+
regression tests covering A↔B, B↔A, anchor preservation, JSON path,
|
|
73
|
+
no-write-on-read, and the "Embedding model changed" warning that
|
|
74
|
+
used to fire on every benign mode switch).
|
|
75
|
+
- **Default `PreToolUse` entry added on `slm hooks install`**. Previously
|
|
76
|
+
PreToolUse was empty unless `include_gate=True`. Now it contains one
|
|
77
|
+
entry (`before_web` on `WebSearch|WebFetch`) by default; gating users
|
|
78
|
+
get that PLUS the firewall entry. Existing settings are merged
|
|
79
|
+
idempotently — `_is_slm_hook_entry` recognises the new wiring so
|
|
80
|
+
`slm hooks remove` cleans it up properly.
|
|
81
|
+
|
|
82
|
+
### Security
|
|
83
|
+
- **CVE-2025-69872 closed (diskcache pickle deserialization RCE).** `diskcache`
|
|
84
|
+
was declared in `pyproject.toml` but never imported anywhere in `src/` or
|
|
85
|
+
`tests/` — a phantom dependency. Removed entirely. The `slm doctor`
|
|
86
|
+
performance-deps check no longer references it. Zero behavior change for
|
|
87
|
+
users; lower attack surface; smaller install.
|
|
88
|
+
- **CVE-2026-1839 (transformers Trainer torch.load RCE) — UNREACHABLE in SLM,
|
|
89
|
+
upstream-pinned.** The vulnerable method `Trainer._load_rng_state` is in
|
|
90
|
+
training code paths. SLM is inference-only (uses `sentence-transformers`
|
|
91
|
+
with ONNX backend; never instantiates `Trainer`). pip-audit flags the dep
|
|
92
|
+
version because the vulnerable bytes are installed, but the code path is
|
|
93
|
+
never executed by SLM. We CANNOT pin `transformers>=5.0.0` (the upstream
|
|
94
|
+
fix) yet because `optimum-onnx 0.1.0` (the latest upstream release as of
|
|
95
|
+
v3.4.43) caps `transformers<4.58.0` — and `embedding_worker.py` requires
|
|
96
|
+
the ONNX backend. Will tighten the pin when optimum-onnx ships a
|
|
97
|
+
transformers-5.x-compatible build. Tracking issue: see project changelog
|
|
98
|
+
for v3.4.44+. Sentence-transformers minimum bumped to `>=5.2.0` to lock
|
|
99
|
+
out 5.0.0-5.1.2 (which capped transformers `<5.0.0` even more strictly)
|
|
100
|
+
and give the resolver maximum headroom for when the upstream pin lifts.
|
|
101
|
+
|
|
102
|
+
### Migration
|
|
103
|
+
- Existing v3.4.42 users: run `slm hooks install` (or `slm init`) once
|
|
104
|
+
after upgrading to pull in the new UserPromptSubmit and PreToolUse
|
|
105
|
+
entries. `slm hooks status` will flag the version mismatch.
|
|
106
|
+
- The settings.json merge is idempotent; running install twice is safe.
|
|
107
|
+
- Topic-shift detection works immediately on first new session — no DB
|
|
108
|
+
or state migration required.
|
|
109
|
+
- `pip install -U superlocalmemory` will pull `transformers>=5.0.0` and
|
|
110
|
+
drop the unused `diskcache` dep automatically.
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
12
114
|
## [3.4.42] - 2026-05-11
|
|
13
115
|
|
|
14
116
|
Operational reliability release. Three latent bugs in the daemon /
|
package/README.md
CHANGED
|
@@ -234,6 +234,47 @@ All `--json` responses follow a consistent envelope with `success`, `command`, `
|
|
|
234
234
|
|
|
235
235
|
---
|
|
236
236
|
|
|
237
|
+
## Smart-hook architecture (v3.4.43)
|
|
238
|
+
|
|
239
|
+
SLM ships a small set of Claude Code hooks that fire memory operations only
|
|
240
|
+
when there's a real signal — not on a timer, not on every keystroke. The
|
|
241
|
+
hooks are perf-budgeted (<10ms p99 for the hot path) and fail-open (any
|
|
242
|
+
crash → silent exit, never blocks your prompt). Install them with one
|
|
243
|
+
command:
|
|
244
|
+
|
|
245
|
+
```bash
|
|
246
|
+
slm hooks install # wires hooks into ~/.claude/settings.json
|
|
247
|
+
slm hooks status # shows what's installed
|
|
248
|
+
slm hooks remove # cleans up, preserves non-SLM hooks
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
| Hook | Event | When it fires | Why |
|
|
252
|
+
|---|---|---|---|
|
|
253
|
+
| `slm hook start` | SessionStart | Once at session boot | Injects core memory + recent context + learned patterns. ~80ms. |
|
|
254
|
+
| `slm hook user_prompt_rehash` | UserPromptSubmit | Every prompt | Detects re-queries within 60s (negative signal that prior recall didn't satisfy). <10ms hot path. |
|
|
255
|
+
| **`slm hook topic_shift`** *(new in 3.4.43)* | UserPromptSubmit | When current prompt shares zero content words with every prompt in a 5-turn sliding window | Surfaces a one-line "consider recall" hint on real topic pivots. Replaces the time-based 15-min nag — event-based, not timer-based. <10ms. |
|
|
256
|
+
| **`slm hook before_web`** *(new in 3.4.43)* | PreToolUse on `WebSearch\|WebFetch` | Every web search/fetch | Runs `slm recall <query> --limit 5` and injects local memories as a system-reminder BEFORE the web call. Cost: ~500-800ms per fire, fires 5-20× per session. |
|
|
257
|
+
| `slm hook checkpoint` | PostToolUse on `Write\|Edit` | Every file write/edit | Auto-observes file changes into SLM. No periodic nag (removed in v3.4.43). |
|
|
258
|
+
| `slm hook post_tool_outcome` | PostToolUse (all tools) | Every tool call | Tracks which recalled facts got used (learning signal). |
|
|
259
|
+
| `slm hook stop` | Stop | Session end | Saves rich session summary with git context. |
|
|
260
|
+
|
|
261
|
+
**What "smart" means here:** the hooks don't interrupt you on a schedule.
|
|
262
|
+
They watch for specific events that indicate memory work would add value —
|
|
263
|
+
a topic pivot, a web call about to fire, a re-asked question, a file edit.
|
|
264
|
+
Otherwise they stay out of your way.
|
|
265
|
+
|
|
266
|
+
**Observability for the new hooks:**
|
|
267
|
+
`topic_shift` writes one TSV line per decision to
|
|
268
|
+
`~/.superlocalmemory/logs/topic-shift.log`
|
|
269
|
+
(`timestamp | session_hash | current_words_count | window_depth | max_overlap |
|
|
270
|
+
fired | prompt_preview`). Disable with `SLM_TOPIC_SHIFT_LOG=0`.
|
|
271
|
+
|
|
272
|
+
**Upgrading from v3.4.42 or older:** Run `slm hooks install` once after
|
|
273
|
+
upgrade to pull in the new wiring. `slm hooks status` will flag the
|
|
274
|
+
version mismatch. Merge is idempotent — safe to run twice.
|
|
275
|
+
|
|
276
|
+
---
|
|
277
|
+
|
|
237
278
|
## Three Operating Modes
|
|
238
279
|
|
|
239
280
|
| Mode | What | Cloud? | EU AI Act | Best For |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "superlocalmemory",
|
|
3
|
-
"version": "3.4.
|
|
3
|
+
"version": "3.4.44",
|
|
4
4
|
"description": "Information-geometric agent memory with mathematical guarantees. 4-channel retrieval, Fisher-Rao similarity, zero-LLM mode, EU AI Act compliant. Works with Claude, Cursor, Windsurf, and 17+ AI tools.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ai-memory",
|
package/pyproject.toml
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
[project]
|
|
2
2
|
name = "superlocalmemory"
|
|
3
|
-
version = "3.4.
|
|
3
|
+
version = "3.4.44"
|
|
4
4
|
description = "Information-geometric agent memory with mathematical guarantees"
|
|
5
5
|
readme = "README.md"
|
|
6
6
|
license = {text = "AGPL-3.0-or-later"}
|
|
@@ -29,37 +29,42 @@ classifiers = [
|
|
|
29
29
|
]
|
|
30
30
|
|
|
31
31
|
dependencies = [
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
"
|
|
37
|
-
"
|
|
38
|
-
"
|
|
39
|
-
"
|
|
40
|
-
"
|
|
41
|
-
"
|
|
42
|
-
"
|
|
43
|
-
"
|
|
44
|
-
"
|
|
45
|
-
"
|
|
46
|
-
"
|
|
47
|
-
|
|
48
|
-
"
|
|
49
|
-
"
|
|
50
|
-
"
|
|
51
|
-
"
|
|
52
|
-
|
|
53
|
-
"
|
|
54
|
-
"
|
|
55
|
-
|
|
56
|
-
"portalocker
|
|
57
|
-
#
|
|
58
|
-
#
|
|
59
|
-
#
|
|
60
|
-
"sentence-transformers[onnx]
|
|
61
|
-
"
|
|
62
|
-
"
|
|
32
|
+
# All versions hard-pinned to the verified-good combination.
|
|
33
|
+
# Mixing versions outside these pins triggers per-batch memory
|
|
34
|
+
# blow-up in the embedding worker on Apple Silicon and breaks
|
|
35
|
+
# recall/remember latency targets. Update only after benchmarking.
|
|
36
|
+
"httpx==0.28.1",
|
|
37
|
+
"numpy==2.4.4",
|
|
38
|
+
"scipy==1.17.1",
|
|
39
|
+
"networkx==3.6.1",
|
|
40
|
+
"mcp==1.27.1",
|
|
41
|
+
"python-dateutil==2.9.0.post0",
|
|
42
|
+
"rank-bm25==0.2.2",
|
|
43
|
+
"vadersentiment==3.3.2",
|
|
44
|
+
"einops==0.8.2",
|
|
45
|
+
"fastapi[all]==0.136.1",
|
|
46
|
+
"uvicorn==0.46.0",
|
|
47
|
+
"websockets==16.0",
|
|
48
|
+
"lightgbm==4.6.0",
|
|
49
|
+
"orjson==3.11.9",
|
|
50
|
+
"tree-sitter==0.25.2",
|
|
51
|
+
"tree-sitter-language-pack==0.13.0",
|
|
52
|
+
"rustworkx==0.17.1",
|
|
53
|
+
"watchdog==5.0.3",
|
|
54
|
+
"psutil==7.2.2",
|
|
55
|
+
"structlog==25.5.0",
|
|
56
|
+
"portalocker==3.2.0",
|
|
57
|
+
# Semantic search + cross-encoder reranker. Embedding stack is
|
|
58
|
+
# extremely sensitive to version drift on Apple Silicon — newer
|
|
59
|
+
# versions allocate dramatically more per-batch memory.
|
|
60
|
+
"sentence-transformers[onnx]==5.3.0",
|
|
61
|
+
"onnxruntime==1.24.4",
|
|
62
|
+
"transformers==4.57.6",
|
|
63
|
+
"huggingface_hub==0.36.2",
|
|
64
|
+
"torch==2.11.0",
|
|
65
|
+
"scikit-learn==1.8.0",
|
|
66
|
+
# Vector KNN extension for the semantic channel.
|
|
67
|
+
"sqlite-vec==0.1.9",
|
|
63
68
|
]
|
|
64
69
|
|
|
65
70
|
[project.optional-dependencies]
|
|
@@ -67,12 +72,13 @@ dependencies = [
|
|
|
67
72
|
# moved into core in v3.4.18. ``pip install superlocalmemory[search]`` still
|
|
68
73
|
# works but installs nothing extra.
|
|
69
74
|
search = [
|
|
70
|
-
|
|
71
|
-
"
|
|
72
|
-
"
|
|
73
|
-
"
|
|
75
|
+
# Same hard pin as core deps — see comment above.
|
|
76
|
+
"sentence-transformers[onnx]==5.3.0",
|
|
77
|
+
"einops==0.8.2",
|
|
78
|
+
"torch==2.11.0",
|
|
79
|
+
"scikit-learn==1.8.0",
|
|
74
80
|
"geoopt>=0.5.0",
|
|
75
|
-
"onnxruntime
|
|
81
|
+
"onnxruntime==1.24.4",
|
|
76
82
|
]
|
|
77
83
|
ui = [
|
|
78
84
|
"fastapi[all]>=0.135.1",
|
|
@@ -83,7 +89,6 @@ learning = [
|
|
|
83
89
|
"lightgbm>=4.0.0",
|
|
84
90
|
]
|
|
85
91
|
performance = [
|
|
86
|
-
"diskcache>=5.6.0",
|
|
87
92
|
"orjson>=3.9.0",
|
|
88
93
|
]
|
|
89
94
|
ingestion = [
|
package/scripts/install.ps1
CHANGED
|
@@ -233,22 +233,31 @@ print('Database ready')
|
|
|
233
233
|
Write-Host "WARNING: setup_validator.py not found, skipping database init" -ForegroundColor Yellow
|
|
234
234
|
}
|
|
235
235
|
|
|
236
|
-
# Install
|
|
236
|
+
# Install SuperLocalMemory and all dependencies via pyproject.toml (single source of truth)
|
|
237
237
|
Write-Host ""
|
|
238
|
-
Write-Host "Installing
|
|
239
|
-
Write-Host "INFO:
|
|
238
|
+
Write-Host "Installing SuperLocalMemory and all dependencies..."
|
|
239
|
+
Write-Host "INFO: Versions are pinned in pyproject.toml -- same versions for every install path" -ForegroundColor Yellow
|
|
240
|
+
|
|
241
|
+
# Find pyproject.toml (parent of scripts/ or scripts/ itself)
|
|
242
|
+
$ParentDir = Split-Path -Parent $REPO_DIR
|
|
243
|
+
if (Test-Path (Join-Path $ParentDir "pyproject.toml")) {
|
|
244
|
+
$ProjRoot = $ParentDir
|
|
245
|
+
} elseif (Test-Path (Join-Path $REPO_DIR "pyproject.toml")) {
|
|
246
|
+
$ProjRoot = $REPO_DIR
|
|
247
|
+
} else {
|
|
248
|
+
$ProjRoot = $null
|
|
249
|
+
}
|
|
240
250
|
|
|
241
|
-
|
|
242
|
-
if (Test-Path $coreRequirements) {
|
|
251
|
+
if ($ProjRoot) {
|
|
243
252
|
try {
|
|
244
|
-
& python -m pip install -q -
|
|
245
|
-
Write-Host "OK
|
|
253
|
+
& python -m pip install -q -e $ProjRoot 2>$null
|
|
254
|
+
Write-Host "OK SuperLocalMemory and all dependencies installed (pinned versions)" -ForegroundColor Green
|
|
246
255
|
} catch {
|
|
247
|
-
Write-Host "WARNING:
|
|
248
|
-
Write-Host " Install manually: python -m pip install -
|
|
256
|
+
Write-Host "WARNING: Dependency installation failed." -ForegroundColor Yellow
|
|
257
|
+
Write-Host " Install manually: python -m pip install -e $ProjRoot" -ForegroundColor Yellow
|
|
249
258
|
}
|
|
250
259
|
} else {
|
|
251
|
-
Write-Host "WARNING:
|
|
260
|
+
Write-Host "WARNING: pyproject.toml not found, cannot install dependencies" -ForegroundColor Yellow
|
|
252
261
|
}
|
|
253
262
|
|
|
254
263
|
# Initialize knowledge graph and pattern learning
|
package/scripts/install.sh
CHANGED
|
@@ -358,8 +358,8 @@ except Exception as e:
|
|
|
358
358
|
|
|
359
359
|
# Install core dependencies (required for graph & dashboard)
|
|
360
360
|
echo ""
|
|
361
|
-
echo "Installing
|
|
362
|
-
echo "⏳
|
|
361
|
+
echo "Installing SuperLocalMemory and all dependencies..."
|
|
362
|
+
echo "⏳ Versions are pinned in pyproject.toml — same versions for every install path"
|
|
363
363
|
|
|
364
364
|
# Detect pip installation method
|
|
365
365
|
if pip3 install --help | grep -q "break-system-packages"; then
|
|
@@ -368,31 +368,25 @@ else
|
|
|
368
368
|
PIP_FLAGS=""
|
|
369
369
|
fi
|
|
370
370
|
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
fi
|
|
371
|
+
# Find the repo root (parent of scripts/)
|
|
372
|
+
PKG_ROOT="$(cd "${REPO_DIR}/.." && pwd)"
|
|
373
|
+
if [ -f "${PKG_ROOT}/pyproject.toml" ]; then
|
|
374
|
+
PROJ_ROOT="${PKG_ROOT}"
|
|
375
|
+
elif [ -f "${REPO_DIR}/pyproject.toml" ]; then
|
|
376
|
+
PROJ_ROOT="${REPO_DIR}"
|
|
378
377
|
else
|
|
379
|
-
|
|
378
|
+
PROJ_ROOT=""
|
|
380
379
|
fi
|
|
381
380
|
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
echo "
|
|
385
|
-
echo " Enables intelligent pattern learning and personalized recall"
|
|
386
|
-
|
|
387
|
-
if [ -f "${REPO_DIR}/requirements-learning.txt" ]; then
|
|
388
|
-
if pip3 install $PIP_FLAGS -q -r "${REPO_DIR}/requirements-learning.txt" 2>/dev/null; then
|
|
389
|
-
echo "✓ Learning dependencies installed (personalized ranking enabled)"
|
|
381
|
+
if [ -n "${PROJ_ROOT}" ]; then
|
|
382
|
+
if pip3 install $PIP_FLAGS -q -e "${PROJ_ROOT}"; then
|
|
383
|
+
echo "✓ SuperLocalMemory and all dependencies installed (pinned versions)"
|
|
390
384
|
else
|
|
391
|
-
echo "
|
|
392
|
-
echo "
|
|
385
|
+
echo "⚠️ Dependency installation failed."
|
|
386
|
+
echo " Install manually: pip3 install -e ${PROJ_ROOT}"
|
|
393
387
|
fi
|
|
394
388
|
else
|
|
395
|
-
echo "
|
|
389
|
+
echo "⚠️ pyproject.toml not found, cannot install dependencies"
|
|
396
390
|
fi
|
|
397
391
|
|
|
398
392
|
# Initialize knowledge graph and pattern learning
|
package/scripts/postinstall.js
CHANGED
|
@@ -97,83 +97,15 @@ function pipInstall(packages, label) {
|
|
|
97
97
|
return false;
|
|
98
98
|
}
|
|
99
99
|
|
|
100
|
-
//
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
console.log('✓ Core dependencies installed (math, search, NLP)');
|
|
110
|
-
} else {
|
|
111
|
-
console.log('⚠ Core dependency installation failed.');
|
|
112
|
-
console.log(' Run manually: pip install ' + coreDeps.join(' '));
|
|
113
|
-
}
|
|
114
|
-
|
|
115
|
-
// Search + ONNX reranking (V3.3.2 — enables 6-channel retrieval + cross-encoder)
|
|
116
|
-
const searchDeps = [
|
|
117
|
-
'sentence-transformers[onnx]>=4.0.0',
|
|
118
|
-
'einops>=0.7.0', 'geoopt>=0.5.0',
|
|
119
|
-
'onnxruntime>=1.17.0',
|
|
120
|
-
];
|
|
121
|
-
|
|
122
|
-
console.log('\nInstalling semantic search + ONNX reranking engine...');
|
|
123
|
-
console.log(' (sentence-transformers 4+, ONNX Runtime, Fisher-Rao geometry)');
|
|
124
|
-
if (pipInstall(searchDeps, 'search')) {
|
|
125
|
-
console.log('✓ Search engine installed (sentence-transformers + ONNX + Fisher-Rao)');
|
|
126
|
-
console.log(' Cross-encoder reranking enabled for ALL modes (+30pp quality)');
|
|
127
|
-
console.log('');
|
|
128
|
-
console.log(' Models auto-download on first use:');
|
|
129
|
-
console.log(' - Embedding: nomic-ai/nomic-embed-text-v1.5 (~500MB)');
|
|
130
|
-
console.log(' - Reranker: cross-encoder/ms-marco-MiniLM-L-6-v2 (~90MB)');
|
|
131
|
-
console.log(' To pre-download now, run: slm warmup');
|
|
132
|
-
} else {
|
|
133
|
-
console.log('⚠ Search engine installation failed (BM25 keyword search still works).');
|
|
134
|
-
console.log(' For full 6-channel retrieval + reranking, run:');
|
|
135
|
-
console.log(' pip install "sentence-transformers[onnx]>=4.0.0" einops geoopt onnxruntime');
|
|
136
|
-
}
|
|
137
|
-
|
|
138
|
-
// Dashboard dependencies (IMPORTANT — enables web dashboard + MCP server)
|
|
139
|
-
const dashboardDeps = ['fastapi[all]>=0.135.1', 'uvicorn>=0.42.0', 'websockets>=16.0'];
|
|
140
|
-
console.log('\nInstalling dashboard & server dependencies...');
|
|
141
|
-
if (pipInstall(dashboardDeps, 'dashboard')) {
|
|
142
|
-
console.log('✓ Dashboard & MCP server dependencies installed (fastapi + uvicorn)');
|
|
143
|
-
} else {
|
|
144
|
-
console.log('⚠ Dashboard installation failed.');
|
|
145
|
-
console.log(' Run manually: pip install \'fastapi[all]\' uvicorn websockets');
|
|
146
|
-
}
|
|
147
|
-
|
|
148
|
-
// Learning dependencies (enables adaptive retrieval after 200+ signals)
|
|
149
|
-
const learningDeps = ['lightgbm>=4.0.0'];
|
|
150
|
-
console.log('\nInstalling learning engine...');
|
|
151
|
-
if (pipInstall(learningDeps, 'learning')) {
|
|
152
|
-
console.log('✓ Learning engine installed (lightgbm — adaptive ranking)');
|
|
153
|
-
} else {
|
|
154
|
-
console.log('⚠ Learning installation failed (retrieval still works without it).');
|
|
155
|
-
console.log(' Run manually: pip install lightgbm');
|
|
156
|
-
}
|
|
157
|
-
|
|
158
|
-
// Performance dependencies (optional — improves caching and JSON speed)
|
|
159
|
-
const perfDeps = ['diskcache>=5.6.0', 'orjson>=3.9.0'];
|
|
160
|
-
console.log('\nInstalling performance optimizations...');
|
|
161
|
-
if (pipInstall(perfDeps, 'performance')) {
|
|
162
|
-
console.log('✓ Performance optimizations installed (diskcache + orjson)');
|
|
163
|
-
} else {
|
|
164
|
-
console.log('⚠ Performance deps skipped (system works fine without them).');
|
|
165
|
-
}
|
|
166
|
-
|
|
167
|
-
// V3.4.3: Unified Brain dependencies (health monitor, structured logging, file watching)
|
|
168
|
-
const brainDeps = ['psutil>=5.9.0', 'structlog>=24.0.0', 'watchdog>=4.0.0'];
|
|
169
|
-
console.log('\nInstalling Unified Brain dependencies (health monitor, file watcher)...');
|
|
170
|
-
if (pipInstall(brainDeps, 'brain')) {
|
|
171
|
-
console.log('✓ Unified Brain deps installed (psutil + structlog + watchdog)');
|
|
172
|
-
console.log(' Health monitoring, structured logging, and file watching enabled');
|
|
173
|
-
} else {
|
|
174
|
-
console.log('⚠ Unified Brain deps partially installed (health monitoring may be limited).');
|
|
175
|
-
console.log(' Run manually: pip install psutil structlog watchdog');
|
|
176
|
-
}
|
|
100
|
+
// Install the superlocalmemory package and all its pinned dependencies
|
|
101
|
+
// in one shot. pyproject.toml is the single source of truth for versions,
|
|
102
|
+
// so users via npm get exactly the same dep set as users via pip.
|
|
103
|
+
console.log('\nInstalling SuperLocalMemory and all dependencies...');
|
|
104
|
+
console.log(' (Single pip install — versions pinned in pyproject.toml)');
|
|
105
|
+
console.log(' This may take 1-3 minutes (downloads ~500MB of models on first use).');
|
|
106
|
+
console.log('');
|
|
107
|
+
console.log(' Includes: numpy, scipy, fastapi, sentence-transformers, onnxruntime,');
|
|
108
|
+
console.log(' torch, transformers, sqlite-vec, lightgbm, mcp, and more.');
|
|
177
109
|
|
|
178
110
|
// --- Step 3b: Install the superlocalmemory package itself ---
|
|
179
111
|
// This ensures `python -m superlocalmemory.cli.main` always resolves the
|
|
@@ -629,24 +629,53 @@ def cmd_setup(args: Namespace) -> None:
|
|
|
629
629
|
|
|
630
630
|
|
|
631
631
|
def cmd_mode(args: Namespace) -> None:
|
|
632
|
-
"""Get or set the operating mode.
|
|
632
|
+
"""Get or set the operating mode.
|
|
633
|
+
|
|
634
|
+
v3.4.43 behavior change: switching modes via this CLI now PRESERVES the
|
|
635
|
+
user's existing embedding, retrieval, evolution, forgetting, and math
|
|
636
|
+
settings. Previously the CLI called ``SLMConfig.for_mode(...)`` which
|
|
637
|
+
re-derived every field from mode defaults — silently clobbering user
|
|
638
|
+
customizations (e.g. a tuned cross-encoder model, a custom embedding
|
|
639
|
+
endpoint, or custom forgetting half-lives). The v3.4.34 ``mode_change=True``
|
|
640
|
+
guard only protected the ``mode`` field itself; everything else was lost.
|
|
641
|
+
|
|
642
|
+
New rules:
|
|
643
|
+
- Only ``config.mode`` changes.
|
|
644
|
+
- If the user has NO LLM provider configured AND is switching to a mode
|
|
645
|
+
that typically needs one (B or C), mode-appropriate LLM defaults are
|
|
646
|
+
populated to avoid the daemon coming up dead. Existing LLM config
|
|
647
|
+
is preserved as-is.
|
|
648
|
+
- Embedding / retrieval / evolution / forgetting / math: untouched.
|
|
649
|
+
"""
|
|
633
650
|
from superlocalmemory.core.config import SLMConfig
|
|
634
651
|
from superlocalmemory.storage.models import Mode
|
|
635
652
|
|
|
636
653
|
config = SLMConfig.load()
|
|
637
654
|
|
|
655
|
+
def _apply_mode_change(new_value: str) -> tuple[SLMConfig, bool]:
|
|
656
|
+
"""Mutate-in-place mode switch. Returns (updated_config, llm_was_set).
|
|
657
|
+
|
|
658
|
+
Only changes ``config.mode``. If the user has no LLM provider
|
|
659
|
+
configured AND is moving to Mode B or C, populates the mode's
|
|
660
|
+
default LLM block so the daemon has something to talk to.
|
|
661
|
+
Everything else (embedding, retrieval, evolution, forgetting,
|
|
662
|
+
math, profile) is preserved byte-for-byte.
|
|
663
|
+
"""
|
|
664
|
+
new_mode = Mode(new_value)
|
|
665
|
+
llm_was_set = False
|
|
666
|
+
if new_mode != Mode.A and not config.llm.provider:
|
|
667
|
+
defaults = SLMConfig.for_mode(new_mode)
|
|
668
|
+
config.llm = defaults.llm
|
|
669
|
+
llm_was_set = True
|
|
670
|
+
config.mode = new_mode
|
|
671
|
+
config.save(mode_change=True)
|
|
672
|
+
return config, llm_was_set
|
|
673
|
+
|
|
638
674
|
if getattr(args, 'json', False):
|
|
639
675
|
from superlocalmemory.cli.json_output import json_print
|
|
640
676
|
if args.value:
|
|
641
677
|
old_mode = config.mode.value.upper()
|
|
642
|
-
updated =
|
|
643
|
-
Mode(args.value),
|
|
644
|
-
llm_provider=config.llm.provider,
|
|
645
|
-
llm_model=config.llm.model,
|
|
646
|
-
llm_api_key=config.llm.api_key,
|
|
647
|
-
llm_api_base=config.llm.api_base,
|
|
648
|
-
)
|
|
649
|
-
updated.save(mode_change=True)
|
|
678
|
+
updated, _ = _apply_mode_change(args.value)
|
|
650
679
|
json_print("mode", data={
|
|
651
680
|
"previous_mode": old_mode, "current_mode": args.value.upper(),
|
|
652
681
|
}, next_actions=[
|
|
@@ -661,20 +690,18 @@ def cmd_mode(args: Namespace) -> None:
|
|
|
661
690
|
return
|
|
662
691
|
|
|
663
692
|
if args.value:
|
|
664
|
-
updated =
|
|
665
|
-
Mode(args.value),
|
|
666
|
-
llm_provider=config.llm.provider,
|
|
667
|
-
llm_model=config.llm.model,
|
|
668
|
-
llm_api_key=config.llm.api_key,
|
|
669
|
-
llm_api_base=config.llm.api_base,
|
|
670
|
-
)
|
|
671
|
-
updated.save(mode_change=True)
|
|
693
|
+
updated, llm_was_set = _apply_mode_change(args.value)
|
|
672
694
|
print(f"Mode set to: {args.value.upper()}")
|
|
673
695
|
|
|
674
|
-
#
|
|
675
|
-
|
|
676
|
-
|
|
677
|
-
|
|
696
|
+
# v3.4.43: embedding/retrieval are now preserved, so the old
|
|
697
|
+
# "Embedding model changed. Re-indexing will run on next recall."
|
|
698
|
+
# warning no longer fires from a CLI mode switch — that was the
|
|
699
|
+
# symptom of the bug. The warning is retained ONLY as an
|
|
700
|
+
# informational note when LLM defaults were freshly populated.
|
|
701
|
+
if llm_was_set:
|
|
702
|
+
print(f" ℹ LLM provider populated from mode defaults: "
|
|
703
|
+
f"{updated.llm.provider}/{updated.llm.model}. "
|
|
704
|
+
f"Run `slm provider set` to customize.")
|
|
678
705
|
|
|
679
706
|
# V3.3.4: Warn if Mode C lacks cloud API key
|
|
680
707
|
if args.value == "c" and not updated.llm.api_key:
|
|
@@ -1422,19 +1449,22 @@ def cmd_doctor(args: Namespace) -> None:
|
|
|
1422
1449
|
"brew install libomp && pip install --force-reinstall lightgbm")
|
|
1423
1450
|
|
|
1424
1451
|
# 6. Performance deps
|
|
1452
|
+
# v3.4.43: diskcache removed from this check — it was a phantom dependency
|
|
1453
|
+
# (declared in pyproject.toml but never imported anywhere in src/ or tests/).
|
|
1454
|
+
# Dropping it closes CVE-2025-69872 (pickle deserialization RCE) without any
|
|
1455
|
+
# behavior change. orjson remains a real performance dep.
|
|
1425
1456
|
perf_ok = []
|
|
1426
|
-
for mod in ["
|
|
1457
|
+
for mod in ["orjson"]:
|
|
1427
1458
|
try:
|
|
1428
1459
|
__import__(mod)
|
|
1429
1460
|
perf_ok.append(mod)
|
|
1430
1461
|
except ImportError:
|
|
1431
1462
|
pass
|
|
1432
|
-
if
|
|
1433
|
-
_check("Performance deps", "PASS", "
|
|
1463
|
+
if perf_ok:
|
|
1464
|
+
_check("Performance deps", "PASS", "orjson")
|
|
1434
1465
|
else:
|
|
1435
|
-
|
|
1436
|
-
|
|
1437
|
-
"pip install diskcache orjson")
|
|
1466
|
+
_check("Performance deps", "WARN", "Missing: orjson",
|
|
1467
|
+
"pip install orjson")
|
|
1438
1468
|
|
|
1439
1469
|
# 7. Embedding worker functional test — skipped under --quick.
|
|
1440
1470
|
if quick:
|
|
@@ -53,24 +53,25 @@ def _start_parent_watchdog() -> None:
|
|
|
53
53
|
|
|
54
54
|
|
|
55
55
|
def _load_embedding_model(name: str) -> tuple:
|
|
56
|
-
"""Load embedding model. ONNX first
|
|
57
|
-
|
|
58
|
-
V3.3.17: PyTorch SentenceTransformer on ARM64 Mac leaks memory —
|
|
59
|
-
grows from 300MB to 17GB after ~200 encode calls. ONNX Runtime
|
|
60
|
-
has no such issue. Same approach as CrossEncoder ONNX migration.
|
|
56
|
+
"""Load embedding model. ONNX CPU-only first, PyTorch fallback.
|
|
61
57
|
|
|
62
58
|
Returns (model, backend_name) or (None, "").
|
|
63
59
|
"""
|
|
64
60
|
from sentence_transformers import SentenceTransformer
|
|
65
61
|
|
|
66
|
-
#
|
|
62
|
+
# ONNX with explicit CPU provider — avoids CoreML EP memory overhead.
|
|
67
63
|
try:
|
|
68
|
-
m = SentenceTransformer(
|
|
64
|
+
m = SentenceTransformer(
|
|
65
|
+
name,
|
|
66
|
+
backend="onnx",
|
|
67
|
+
trust_remote_code=True,
|
|
68
|
+
model_kwargs={"provider": "CPUExecutionProvider"},
|
|
69
|
+
)
|
|
69
70
|
return m, "onnx"
|
|
70
71
|
except Exception:
|
|
71
72
|
pass
|
|
72
73
|
|
|
73
|
-
#
|
|
74
|
+
# PyTorch CPU fallback.
|
|
74
75
|
try:
|
|
75
76
|
import torch
|
|
76
77
|
with torch.inference_mode():
|