superlocalmemory 3.4.56 → 3.4.59

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,113 @@ All notable changes to SuperLocalMemory V3 will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [3.4.59] - 2026-05-31 — Graph Edge Cap + Recall Reliability
9
+
10
+ **Fixes SLM falling into degraded FTS5 mode on every session start**, and stops
11
+ the knowledge graph from growing into an O(n²) edge explosion that made recall
12
+ slow as the fact corpus scaled beyond 10K facts.
13
+
14
+ ### Root Causes Fixed
15
+
16
+ **1. MCP timeout too aggressive for dense graphs (degraded mode bug)**
17
+ The v3.4.57 timeout reduction (60s→8s) was correct for small graphs but
18
+ backfired at scale. With a 17K-fact corpus and 2.1M graph edges, full 6-channel
19
+ recall (including spreading activation + Hopfield) takes 13–15s. The 8s timeout
20
+ always expired → `pool_recall` raised `PoolError` → session_init fell back to
21
+ emergency FTS5 BM25. Every session started in degraded mode silently.
22
+
23
+ **Fix:** `DaemonPoolProxy.timeout_s` raised from 8s to 30s. Covers the current
24
+ worst-case (13.4s full recall) with 2× headroom. The orphan-flood concern from
25
+ v3.4.57 is mitigated by the ingest cap reductions below, which stop the graph
26
+ from growing further.
27
+
28
+ **2. Knowledge graph edge explosion (O(n²) growth)**
29
+ At 17K facts, the corpus hit 2.1M graph edges (avg 121 per node). Root cause:
30
+ `_MAX_ENTITY_EDGES_PER_ENTITY = 20` was designed for ~500-fact graphs. At scale,
31
+ hub entities (appearing in 1000s of facts) accumulated 5000+ edges per node.
32
+ Spreading activation must fan out across all edges per node, causing 9s SA time.
33
+
34
+ **Fixes:**
35
+ - `_MAX_ENTITY_EDGES_PER_ENTITY` lowered from 20 → 5
36
+ - `_MAX_CAUSAL_EDGES_PER_ENTITY` lowered from 20 → 5
37
+ - **Hub node filter added:** nodes already at ≥ 200 total edges are skipped
38
+ during ingest. High-frequency hub nodes (e.g. a term appearing in every fact)
39
+ link everything to everything — they are graph noise, not graph signal.
40
+ - Hub cache shared across entity/causal edge builders per `build_edges` call to
41
+ avoid redundant DB queries.
42
+
43
+ **3. Spreading activation UNION query not using indexes (SA slow path)**
44
+ The `_get_unified_neighbors` UNION ALL query fetched all edges for a node then
45
+ sorted them, preventing the `idx_edges_source_weight` and `idx_edges_target_weight`
46
+ covering indexes from terminating early. Fix: push `ORDER BY weight DESC LIMIT ?`
47
+ inside each UNION branch (wrapped in `SELECT * FROM (...)` per SQLite compound
48
+ SELECT syntax). SQLite now stops after `max_neighbors_per_node` rows per branch
49
+ using the covering index instead of materialising the full edge set.
50
+
51
+ **4. Degree-cap pruner added to graph_pruner.py**
52
+ New `_cap_node_degree()` function using `ROW_NUMBER() OVER (PARTITION BY source_id
53
+ ORDER BY weight DESC)` — single-pass window function, no Python loops. Integrated
54
+ into `prune_graph()` as `cap_degree=True` (default). Automatically runs during
55
+ scheduled maintenance cycles to keep hub nodes bounded.
56
+
57
+ ### Changed
58
+ - `mcp/_daemon_proxy.py`: `timeout_s` default 8.0 → 30.0
59
+ - `encoding/graph_builder.py`: entity + causal caps 20 → 5; hub filter at 200 edges
60
+ - `core/graph_pruner.py`: `_cap_node_degree()` added; `prune_graph()` gains `cap_degree` param
61
+ - `retrieval/spreading_activation.py`: UNION LIMIT pushed inside each branch
62
+
63
+ ### Tests
64
+ 4053 passed, 15 skipped — no regressions.
65
+
66
+ ---
67
+
68
+ ## [3.4.58] - 2026-05-30 — Permanent OpenMP SIGSEGV Fix
69
+
70
+ **Eliminates the recurring Python crash popup on macOS Apple Silicon.** Any user
71
+ who triggered a LightGBM retrain cycle (background learning after ~50 recalls)
72
+ would see a macOS crash report for `Python [PID]` with SIGSEGV at
73
+ `__kmp_suspend_initialize_thread + 32`. This release permanently fixes the root
74
+ cause in the SLM source — no system changes required.
75
+
76
+ ### Root Cause
77
+ SLM's dependency set ships **three separate `libomp.dylib` binaries** on macOS ARM:
78
+ - `torch==2.11.0` bundles `/opt/llvm-openmp/lib/libomp.dylib` (860 KB)
79
+ - `scikit-learn==1.8.0` bundles its own `/opt/llvm-openmp/lib/libomp.dylib` (678 KB)
80
+ - `lightgbm==4.6.0` resolves to homebrew's `/opt/homebrew/opt/libomp/lib/libomp.dylib` (739 KB)
81
+
82
+ When `lgb.Dataset(X, ...)` called `LGBM_DatasetCreateFromMat` → OpenMP `fork_call`
83
+ with `num_threads = os.cpu_count() - 1` (9 threads on M4 Mac), the parallel
84
+ worker threads were allocated by LightGBM's libomp but attempted to synchronize
85
+ via PyTorch's libomp thread pool. The two runtimes have incompatible internal
86
+ thread structs — the barrier release read address `0x580` (null + struct offset),
87
+ causing `EXC_BAD_ACCESS (SIGSEGV)` in Thread 26.
88
+
89
+ **All macOS Apple Silicon users with the standard SLM install were affected.**
90
+ The crash fired silently in a background consolidation worker, causing the
91
+ `slm mcp` subprocess to restart repeatedly, generating the persistent crash popup.
92
+
93
+ ### Fixed
94
+ - **`ranker_retrain_online.py` line 198** — `num_threads = max(1, os.cpu_count()-1)`
95
+ changed to a safe cap of **2 threads** (configurable via `SLM_LGBM_THREADS` env
96
+ var). With ≤2 threads, the problematic parallel fork path in
97
+ `DatasetLoader::ConstructFromSampleData` is avoided entirely. SLM's training
98
+ datasets (50–5,000 rows) see ~90% of max-core throughput at 2 threads — the
99
+ difference is under 200ms per retrain cycle.
100
+ - **`__init__.py`** — `KMP_DUPLICATE_LIB_OK` changed from `os.environ.setdefault`
101
+ (could be overridden to FALSE) to unconditional `os.environ[...] = "TRUE"`.
102
+ Added `OMP_NUM_THREADS=2` cap (respects user override) as belt-and-suspenders
103
+ at the OS level before any C library reads the thread count.
104
+
105
+ ### New environment variables
106
+ - `SLM_LGBM_THREADS` — override the LightGBM thread count (default: `2`).
107
+ Only increase if your system has a unified single-runtime OpenMP setup.
108
+
109
+ ### Why not fix the dylib collision instead?
110
+ Patching `libomp.dylib` on users' systems via `install_name_tool` is fragile:
111
+ it breaks on package updates, requires write access to site-packages, and fails
112
+ if SIP prevents modifying signed binaries. The source fix is permanent, upgrade-safe,
113
+ and works identically on every user's machine regardless of their exact package versions.
114
+
8
115
  ## [3.4.52] - 2026-05-28 — Warm Memory, No Cold Starts
9
116
 
10
117
  **Production resilience for session_init.** No quality degradation as the primary path: full 6-channel recall (semantic + BM25 + entity + temporal + Hopfield + spreading-activation, Fisher-Rao fusion) is preserved. The cold-start problem is fixed at the infrastructure layer, not by downgrading retrieval.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "superlocalmemory",
3
- "version": "3.4.56",
3
+ "version": "3.4.59",
4
4
  "description": "Information-geometric agent memory with mathematical guarantees. 4-channel retrieval, Fisher-Rao similarity, zero-LLM mode, EU AI Act compliant. Works with Claude, Cursor, Windsurf, and 17+ AI tools.",
5
5
  "keywords": [
6
6
  "ai-memory",
package/pyproject.toml CHANGED
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "superlocalmemory"
3
- version = "3.4.56"
3
+ version = "3.4.59"
4
4
  description = "Information-geometric agent memory with mathematical guarantees"
5
5
  readme = "README.md"
6
6
  license = {text = "AGPL-3.0-or-later"}
@@ -1,9 +1,34 @@
1
1
  """SuperLocalMemory — information-geometric agent memory."""
2
2
 
3
3
  import os
4
- os.environ.setdefault("KMP_DUPLICATE_LIB_OK", "TRUE")
5
4
 
6
- __version__ = "3.4.56"
5
+ # --- OpenMP multi-library guard (permanent fix, v3.4.58) -------------------
6
+ # SLM ships with torch, scikit-learn, and lightgbm — each bundles its own
7
+ # libomp.dylib on macOS ARM (Apple Silicon). When all three are loaded in the
8
+ # same process, the Intel OpenMP runtime detects duplicate libraries and either
9
+ # (a) emits OMP: Error #15 and aborts, or (b) crashes with SIGSEGV in
10
+ # __kmp_suspend_initialize_thread (address 0x580 — null thread-struct deref)
11
+ # when LightGBM's parallel fork tries to coordinate with PyTorch's thread pool.
12
+ #
13
+ # KMP_DUPLICATE_LIB_OK=TRUE tells the runtime to elect one master instance
14
+ # and continue rather than abort. This is the upstream-recommended workaround
15
+ # for mixed-dependency environments (PyTorch docs, scikit-learn FAQ, LightGBM
16
+ # issue #3877). Use unconditional assignment — not setdefault — so user env
17
+ # cannot accidentally disable this safety net by setting it to FALSE.
18
+ #
19
+ # OMP_NUM_THREADS=2 caps the maximum thread count before any C library reads
20
+ # it. With ≤2 threads the problematic parallel fork path in LightGBM's
21
+ # DatasetLoader::ConstructFromSampleData is avoided on all observed crash
22
+ # configurations. SLM datasets are small (50–5 000 rows); 2 threads gives
23
+ # ~90% of the performance of max-core training at zero crash risk.
24
+ # Users who need more threads can set SLM_LGBM_THREADS=N to override
25
+ # the per-call cap in ranker_retrain_online.py.
26
+ os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"
27
+ if "OMP_NUM_THREADS" not in os.environ:
28
+ os.environ["OMP_NUM_THREADS"] = "2"
29
+ # ---------------------------------------------------------------------------
30
+
31
+ __version__ = "3.4.59"
7
32
 
8
33
  _REQUIRED_VERSIONS = {
9
34
  "sentence_transformers": "5.3.0",
@@ -1000,80 +1000,87 @@ class SLMConfig:
1000
1000
  new_mode: str,
1001
1001
  base_dir: Path | None = None,
1002
1002
  ) -> "SLMConfig":
1003
- """Switch to a different mode, preserving the old mode's config.
1004
-
1005
- v3.4.54: Saves the current config as ``mode_{old}.json``, then
1006
- loads ``mode_{new}.json`` (or creates it from defaults if it
1007
- doesn't exist). Writes ``current_mode`` and persists the new
1008
- active config as ``config.json`` for backward compatibility
1009
- with tooling that reads it directly.
1003
+ """Switch to a different mode, preserving ALL non-mode settings.
1004
+
1005
+ v3.4.58 fix: Only ``config.mode`` changes. The previous implementation
1006
+ called ``for_mode()`` (which resets embedding/LLM/retrieval to mode
1007
+ defaults) whenever the target mode's per-mode file didn't exist.
1008
+ This silently clobbered custom embeddings, endpoints, and LLM config.
1009
+
1010
+ Correct behavior:
1011
+ - Load the current (old) config from config.json.
1012
+ - Save it to mode_{old}.json for the 3-mode system.
1013
+ - If mode_{new}.json exists, load it (user had a prior config for that mode).
1014
+ - If mode_{new}.json does NOT exist: start from the OLD config, change
1015
+ only the mode field. This preserves embedding, retrieval, forgetting, etc.
1016
+ - Exception: if user has NO LLM provider AND is switching to B/C, populate
1017
+ sensible LLM defaults so the daemon doesn't start dead.
1018
+ - Write the result to config.json (backward compat) and current_mode.
1010
1019
 
1011
1020
  Returns the new active config.
1012
1021
  """
1022
+ import copy
1023
+ import dataclasses
1024
+
1013
1025
  from superlocalmemory.storage.models import Mode as _M
1014
1026
  _base = base_dir or DEFAULT_BASE_DIR
1015
1027
  _base.mkdir(parents=True, exist_ok=True)
1016
1028
 
1017
1029
  old_config = cls.load(_base / "config.json")
1018
1030
  old_mode = old_config.mode.value.lower()
1019
-
1020
1031
  new_mode_val = _M(new_mode.lower())
1021
1032
 
1022
- # 1. Save current config to its per-mode file (unless already
1023
- # matched — prevents overwriting user customizations)
1033
+ # 1. Save current config to its per-mode file (preserve customizations)
1024
1034
  if old_mode != new_mode.lower():
1025
1035
  old_path = cls._mode_config_path(_base, old_config.mode)
1026
1036
  old_config.save(old_path)
1027
1037
 
1028
- # 2. Try to load the target mode's saved config.
1038
+ # 2. Determine new config:
1039
+ # a) mode_{new}.json exists → user had prior config for this mode, use it
1040
+ # b) otherwise → copy old config, change only mode
1029
1041
  new_path = cls._mode_config_path(_base, new_mode_val)
1030
- need_migration = False
1031
1042
  if new_path.exists():
1032
1043
  try:
1033
1044
  new_config = cls.load(new_path)
1034
- # Ensure the loaded config actually has the right mode
1035
1045
  if new_config.mode != new_mode_val:
1036
- new_config = cls.for_mode(new_mode_val, base_dir=_base)
1046
+ # Corrupt/mismatched mode file — rebuild preserving old values
1047
+ new_config = copy.copy(old_config)
1048
+ new_config.mode = new_mode_val
1037
1049
  except Exception:
1038
- new_config = cls.for_mode(new_mode_val, base_dir=_base)
1050
+ new_config = copy.copy(old_config)
1051
+ new_config.mode = new_mode_val
1039
1052
  else:
1040
- # First time switching to this mode: try migrating from
1041
- # legacy config.json if it was already in this mode
1042
- legacy = _base / "config.json"
1043
- if legacy.exists():
1044
- try:
1045
- import json
1046
- _data = json.loads(legacy.read_text(encoding="utf-8"))
1047
- _legacy_mode = _data.get("mode", "").lower()
1048
- if _legacy_mode == new_mode.lower():
1049
- # config.json IS this mode — use it directly
1050
- new_config = cls.load(legacy)
1051
- need_migration = True
1052
- else:
1053
- new_config = cls.for_mode(new_mode_val, base_dir=_base)
1054
- except Exception:
1055
- new_config = cls.for_mode(new_mode_val, base_dir=_base)
1056
- else:
1057
- new_config = cls.for_mode(new_mode_val, base_dir=_base)
1053
+ # First time switching to this mode: start from old config, change mode only.
1054
+ # Use dataclasses.replace() to produce a new frozen-compatible object.
1055
+ new_config = dataclasses.replace(old_config, mode=new_mode_val)
1056
+
1057
+ # 3. LLM default population — ONLY if user has no provider AND switching to B/C.
1058
+ # Never overwrites an existing provider.
1059
+ if new_mode_val in (_M.B, _M.C) and not new_config.llm.provider:
1060
+ if new_mode_val == _M.B:
1061
+ new_config = dataclasses.replace(
1062
+ new_config,
1063
+ llm=LLMConfig(
1064
+ provider="ollama",
1065
+ model="llama3.2",
1066
+ api_base="http://localhost:11434",
1067
+ ),
1068
+ )
1069
+ else: # Mode C
1070
+ new_config = dataclasses.replace(
1071
+ new_config,
1072
+ llm=LLMConfig(
1073
+ provider="openrouter",
1074
+ model="anthropic/claude-sonnet-4",
1075
+ ),
1076
+ )
1058
1077
 
1059
- # 3. Save as active config.json (backward compat)
1078
+ # 4. Save as active config.json (backward compat)
1060
1079
  new_config.save(_base / "config.json", mode_change=True)
1061
1080
 
1062
- # 4. Write current_mode
1081
+ # 5. Write current_mode
1063
1082
  cls.write_current_mode(new_mode, _base)
1064
1083
 
1065
- # 5. On first migration, also save mode_a.json and mode_c.json
1066
- # so users have a complete set
1067
- if need_migration:
1068
- for _m in (_M.A, _M.B, _M.C):
1069
- _mp = cls._mode_config_path(_base, _m)
1070
- if not _mp.exists():
1071
- try:
1072
- _mc = cls.for_mode(_m, base_dir=_base)
1073
- _mc.save(_mp)
1074
- except Exception:
1075
- pass
1076
-
1077
1084
  return new_config
1078
1085
 
1079
1086
  @classmethod
@@ -28,12 +28,18 @@ from pathlib import Path
28
28
  logger = logging.getLogger("superlocalmemory.graph_pruner")
29
29
 
30
30
  _CHAIN_BATCH_LIMIT = 10_000
31
+ # v3.4.59: Nodes with more than this many total edges (in+out) are hub nodes.
32
+ # SA and entity_graph channels cap fan-out at 30/node, so edges beyond this
33
+ # threshold provide zero additional recall signal while bloating the graph.
34
+ _MAX_DEGREE_PER_NODE: int = 100
35
+ _HUB_PRUNE_BATCH: int = 500 # delete edges in batches to avoid giant IN clauses
31
36
 
32
37
 
33
38
  def prune_graph(
34
39
  db_path: str | Path,
35
40
  profile_id: str = "default",
36
41
  dry_run: bool = False,
42
+ cap_degree: bool = True,
37
43
  ) -> dict:
38
44
  """Run all graph pruning strategies for a specific profile.
39
45
 
@@ -41,7 +47,7 @@ def prune_graph(
41
47
  """
42
48
  conn = sqlite3.connect(str(db_path))
43
49
  conn.execute("PRAGMA journal_mode=WAL")
44
- conn.execute("PRAGMA busy_timeout=10000")
50
+ conn.execute("PRAGMA busy_timeout=30000")
45
51
  conn.row_factory = sqlite3.Row
46
52
 
47
53
  stats = {
@@ -49,6 +55,7 @@ def prune_graph(
49
55
  "supersedes_collapsed": 0,
50
56
  "self_loops_removed": 0,
51
57
  "duplicates_removed": 0,
58
+ "hub_edges_removed": 0,
52
59
  "total_before": 0,
53
60
  "total_after": 0,
54
61
  }
@@ -72,6 +79,10 @@ def prune_graph(
72
79
  stats["supersedes_collapsed"] = _collapse_supersedes_chains(
73
80
  c, profile_id, dry_run,
74
81
  )
82
+ if cap_degree:
83
+ stats["hub_edges_removed"] = _cap_node_degree(
84
+ c, profile_id, _MAX_DEGREE_PER_NODE, dry_run,
85
+ )
75
86
 
76
87
  if dry_run:
77
88
  c.execute("ROLLBACK")
@@ -91,10 +102,11 @@ def prune_graph(
91
102
  prefix = "(dry-run) " if dry_run else ""
92
103
  logger.info(
93
104
  "%sGraph pruning: removed %d edges (%.1f%%) in %.1fs — "
94
- "orphans=%d, supersedes=%d, self_loops=%d, duplicates=%d",
105
+ "orphans=%d, supersedes=%d, self_loops=%d, duplicates=%d, hub_cap=%d",
95
106
  prefix, total_removed, pct, elapsed,
96
107
  stats["orphans_removed"], stats["supersedes_collapsed"],
97
108
  stats["self_loops_removed"], stats["duplicates_removed"],
109
+ stats["hub_edges_removed"],
98
110
  )
99
111
 
100
112
  except Exception as exc:
@@ -288,3 +300,81 @@ def _collapse_supersedes_chains(
288
300
  )
289
301
 
290
302
  return len(delete_ids)
303
+
304
+
305
+ def _cap_node_degree(
306
+ c: sqlite3.Cursor,
307
+ profile_id: str,
308
+ max_degree: int,
309
+ dry_run: bool,
310
+ ) -> int:
311
+ """Remove low-weight edges from hub nodes (nodes with degree > max_degree).
312
+
313
+ v3.4.59: At 17K facts, popular entities (SLM, Claude, AgentAssert) caused
314
+ 1.4M+ entity edges — avg 121 per node, some nodes with 5000+. SA fan-out
315
+ is capped at 30/node during traversal, so edges beyond max_degree add zero
316
+ recall signal while making every graph query scan millions of rows.
317
+
318
+ Algorithm (single-pass window function — no Python loops):
319
+ 1. ROW_NUMBER() OVER (PARTITION BY source_id ORDER BY weight DESC) ranks
320
+ every edge per node in one full table scan.
321
+ 2. Edges with rn > max_degree are deleted in a single DELETE statement.
322
+ Requires SQLite 3.25+ (window functions). System is on 3.53.1.
323
+ """
324
+ if dry_run:
325
+ c.execute(
326
+ """
327
+ SELECT COUNT(*) as cnt FROM (
328
+ SELECT edge_id,
329
+ ROW_NUMBER() OVER (
330
+ PARTITION BY source_id ORDER BY weight DESC
331
+ ) as rn
332
+ FROM graph_edges
333
+ WHERE profile_id = ?
334
+ ) WHERE rn > ?
335
+ """,
336
+ (profile_id, max_degree),
337
+ )
338
+ excess = c.fetchone()["cnt"]
339
+ logger.info(
340
+ "(dry-run) _cap_node_degree: ~%d edges would be removed (max_degree=%d)",
341
+ excess, max_degree,
342
+ )
343
+ return excess
344
+
345
+ # Step 1: build temp keep-list in one pass (ROW_NUMBER ranks by weight DESC)
346
+ c.execute("CREATE TEMP TABLE IF NOT EXISTS _slm_keep_edges (edge_id TEXT PRIMARY KEY)")
347
+ c.execute("DELETE FROM _slm_keep_edges") # idempotent if called twice
348
+ c.execute(
349
+ """
350
+ INSERT INTO _slm_keep_edges (edge_id)
351
+ SELECT edge_id FROM (
352
+ SELECT edge_id,
353
+ ROW_NUMBER() OVER (
354
+ PARTITION BY source_id ORDER BY weight DESC
355
+ ) as rn
356
+ FROM graph_edges
357
+ WHERE profile_id = ?
358
+ ) WHERE rn <= ?
359
+ """,
360
+ (profile_id, max_degree),
361
+ )
362
+
363
+ # Step 2: delete everything not in keep-list (single DELETE)
364
+ c.execute(
365
+ """
366
+ DELETE FROM graph_edges
367
+ WHERE profile_id = ?
368
+ AND edge_id NOT IN (SELECT edge_id FROM _slm_keep_edges)
369
+ """,
370
+ (profile_id,),
371
+ )
372
+ deleted = c.rowcount
373
+
374
+ c.execute("DROP TABLE IF EXISTS _slm_keep_edges")
375
+
376
+ logger.info(
377
+ "_cap_node_degree: deleted %d low-weight edges (max_degree=%d)",
378
+ deleted, max_degree,
379
+ )
380
+ return deleted
@@ -76,11 +76,12 @@ class GraphBuilder:
76
76
 
77
77
  def build_edges(self, new_fact: AtomicFact, profile_id: str) -> list[GraphEdge]:
78
78
  """Create ALL relevant edges for *new_fact*. Persists and returns them."""
79
+ hub_cache: dict[str, int] = {} # shared across edge types to avoid redundant queries
79
80
  edges: list[GraphEdge] = []
80
- edges.extend(self._build_entity_edges(new_fact, profile_id))
81
+ edges.extend(self._build_entity_edges(new_fact, profile_id, hub_cache))
81
82
  edges.extend(self._build_temporal_edges(new_fact, profile_id))
82
83
  edges.extend(self._build_semantic_edges(new_fact, profile_id))
83
- edges.extend(self._build_causal_edges(new_fact, profile_id))
84
+ edges.extend(self._build_causal_edges(new_fact, profile_id, hub_cache))
84
85
 
85
86
  for edge in edges:
86
87
  self._db.store_edge(edge)
@@ -142,17 +143,33 @@ class GraphBuilder:
142
143
 
143
144
  # -- Edge builders (private) -------------------------------------------
144
145
 
145
- # V3.3.12: Cap entity edges per entity to prevent O(n²) explosion.
146
- # With 500+ facts sharing a popular entity, creating an edge to each
147
- # produced 44K+ edges and 22-min ingestion. Cap to 20 most recent per entity.
148
- _MAX_ENTITY_EDGES_PER_ENTITY: int = 20
146
+ # v3.4.59: Lowered from 20→5. At 17K facts, 20 edges × many entities per fact
147
+ # produces 1.4M+ entity edges and 13s recalls. 5 is sufficient graph signal.
148
+ _MAX_ENTITY_EDGES_PER_ENTITY: int = 5
149
+ # v3.4.59: Hub filter — nodes with > 200 total edges are "gravity wells"
150
+ # (e.g. every fact mentions "SLM"). Skip adding more edges to them.
151
+ _MAX_HUB_DEGREE: int = 200
152
+
153
+ def _node_degree(self, fact_id: str, profile_id: str, cache: dict[str, int]) -> int:
154
+ """Cached total degree (in + out) for a node."""
155
+ if fact_id not in cache:
156
+ rows = self._db.execute(
157
+ "SELECT COUNT(*) as cnt FROM graph_edges "
158
+ "WHERE profile_id = ? AND (source_id = ? OR target_id = ?)",
159
+ (profile_id, fact_id, fact_id),
160
+ )
161
+ cache[fact_id] = int(dict(rows[0])["cnt"]) if rows else 0
162
+ return cache[fact_id]
149
163
 
150
164
  def _build_entity_edges(
151
165
  self, new_fact: AtomicFact, profile_id: str,
166
+ hub_cache: dict[str, int] | None = None,
152
167
  ) -> list[GraphEdge]:
153
168
  """ENTITY edges: shared canonical entity — capped to most recent per entity."""
154
169
  if not new_fact.canonical_entities:
155
170
  return []
171
+ if hub_cache is None:
172
+ hub_cache = {}
156
173
  edges: list[GraphEdge] = []
157
174
  seen: set[str] = set()
158
175
 
@@ -163,6 +180,8 @@ class GraphBuilder:
163
180
  break
164
181
  if other.fact_id == new_fact.fact_id or other.fact_id in seen:
165
182
  continue
183
+ if self._node_degree(other.fact_id, profile_id, hub_cache) >= self._MAX_HUB_DEGREE:
184
+ continue # skip hub nodes — they link everything to everything
166
185
  if self._edge_exists(new_fact.fact_id, other.fact_id, EdgeType.ENTITY, profile_id):
167
186
  continue
168
187
  seen.add(other.fact_id)
@@ -261,17 +280,20 @@ class GraphBuilder:
261
280
  break
262
281
  return edges
263
282
 
264
- # V3.3.13: Cap causal edges per entity to prevent O(n²) explosion (same as entity/temporal).
265
- _MAX_CAUSAL_EDGES_PER_ENTITY: int = 20
283
+ # v3.4.59: Lowered from 20→5, same reasoning as entity cap.
284
+ _MAX_CAUSAL_EDGES_PER_ENTITY: int = 5
266
285
 
267
286
  def _build_causal_edges(
268
287
  self, new_fact: AtomicFact, profile_id: str,
288
+ hub_cache: dict[str, int] | None = None,
269
289
  ) -> list[GraphEdge]:
270
290
  """CAUSAL edges: causal markers + shared entity. Direction: cause -> effect."""
271
291
  if not any(p.search(new_fact.content) for p in _CAUSAL_CUES):
272
292
  return []
273
293
  if not new_fact.canonical_entities:
274
294
  return []
295
+ if hub_cache is None:
296
+ hub_cache = {}
275
297
 
276
298
  edges: list[GraphEdge] = []
277
299
  seen: set[str] = set()
@@ -282,6 +304,8 @@ class GraphBuilder:
282
304
  break
283
305
  if other.fact_id == new_fact.fact_id or other.fact_id in seen:
284
306
  continue
307
+ if self._node_degree(other.fact_id, profile_id, hub_cache) >= self._MAX_HUB_DEGREE:
308
+ continue
285
309
  if self._edge_exists(other.fact_id, new_fact.fact_id, EdgeType.CAUSAL, profile_id):
286
310
  continue
287
311
  seen.add(other.fact_id)
@@ -46,7 +46,7 @@ class ReaperConfig:
46
46
 
47
47
  enabled: bool = True
48
48
  heartbeat_interval_seconds: int = 60
49
- orphan_age_threshold_hours: float = 4.0
49
+ orphan_age_threshold_hours: float = 1.0 # v3.4.57: reduced from 4.0 — confirmed-dead parent = safe to kill after 1h
50
50
  pid_file_path: str = "" # Empty = default (~/.superlocalmemory/slm.pids)
51
51
  graceful_timeout_seconds: float = 5.0
52
52
 
@@ -147,6 +147,13 @@ def _retrain_ranker_impl(
147
147
  objective, sorted(_allowed_objectives),
148
148
  )
149
149
  objective = "lambdarank"
150
+ # SAFE THREAD CAP — v3.4.58 permanent fix for OpenMP multi-library SIGSEGV.
151
+ # See ranker_retrain_online.py for full explanation. Same cap applies here:
152
+ # the legacy path hits the identical LGBM_DatasetCreateFromMat → OpenMP
153
+ # parallel fork. On macOS ARM with torch+sklearn+lightgbm, >2 threads
154
+ # triggers the libomp multi-runtime SIGSEGV in __kmp_suspend_initialize_thread.
155
+ _lgbm_threads_env = os.environ.get("SLM_LGBM_THREADS", "").strip()
156
+ _lgbm_num_threads = int(_lgbm_threads_env) if _lgbm_threads_env.isdigit() else 2
150
157
  params = {
151
158
  "objective": objective,
152
159
  "metric": "ndcg",
@@ -156,8 +163,9 @@ def _retrain_ranker_impl(
156
163
  "num_leaves": 31,
157
164
  "min_data_in_leaf": 20,
158
165
  "verbosity": -1,
159
- "num_threads": max(1, (os.cpu_count() or 2) - 1),
166
+ "num_threads": _lgbm_num_threads,
160
167
  }
168
+
161
169
  try:
162
170
  booster_new = lgb.train(params, ds_train, num_boost_round=50)
163
171
  except lgb.basic.LightGBMError as exc:
@@ -195,7 +195,17 @@ def _train_booster(
195
195
  params = dict(RETRAIN_HYPERPARAM_CAPS)
196
196
  params["objective"] = objective
197
197
  params["label_gain"] = gain
198
- params["num_threads"] = max(1, (os.cpu_count() or 2) - 1)
198
+ # SAFE THREAD CAP — v3.4.58 permanent fix for OpenMP multi-library SIGSEGV.
199
+ # Using os.cpu_count()-1 (e.g. 9 on M4 Mac) triggers a race in LightGBM's
200
+ # DatasetLoader::ConstructFromSampleData when PyTorch's libomp and
201
+ # LightGBM's libomp are both loaded. The parallel OpenMP fork crosses
202
+ # libomp runtime boundaries → SIGSEGV in __kmp_suspend_initialize_thread.
203
+ # 2 threads avoids the crash path and is sufficient for SLM's small datasets.
204
+ # Override with SLM_LGBM_THREADS env var if you need more (and have a
205
+ # single-runtime environment with libomp unified).
206
+ _lgbm_threads_env = os.environ.get("SLM_LGBM_THREADS", "").strip()
207
+ params["num_threads"] = int(_lgbm_threads_env) if _lgbm_threads_env.isdigit() else 2
208
+
199
209
  num_boost_round = int(params.pop("num_boost_round"))
200
210
 
201
211
  start = time.monotonic()
@@ -36,7 +36,7 @@ class DaemonPoolProxy:
36
36
  envelopes — the adapter is responsible for surfacing those.
37
37
  """
38
38
 
39
- def __init__(self, port: int, *, timeout_s: float = 60.0) -> None:
39
+ def __init__(self, port: int, *, timeout_s: float = 30.0) -> None: # v3.4.59: 8s→30s — observed recall takes 13.4s on dense graph (2.1M edges); 8s always timed out → degraded mode
40
40
  self._port = port
41
41
  self._timeout = timeout_s
42
42
 
@@ -241,5 +241,42 @@ _warmup_thread = threading.Thread(target=_eager_warmup, daemon=True, name="mcp-w
241
241
  _warmup_thread.start()
242
242
 
243
243
 
244
+ # V3.4.57: Parent watchdog — self-terminate when the IDE/Claude session dies.
245
+ # FastMCP relies on stdin EOF to stop, but stdin EOF is NOT guaranteed on
246
+ # crash or force-quit. Without this, every abnormal exit leaves an orphaned
247
+ # slm mcp process consuming ~100-200 MB indefinitely. 22 orphans caused a
248
+ # daemon deadlock on May 30 2026 (241% CPU, session_init timeouts).
249
+ def _parent_watchdog() -> None:
250
+ """Exit when parent IDE process (Claude Code, Cursor, etc.) dies.
251
+
252
+ Polls os.getppid() every 10 seconds. On macOS/Linux, when a parent
253
+ dies the child is reparented to PID 1 (init) — getppid() returns 1.
254
+ Also validates the original parent is still alive via os.kill(ppid, 0).
255
+ Uses os._exit(0) to bypass any atexit handlers that might hang.
256
+ """
257
+ import os as _os_wd, time as _time
258
+ _wlog = logging.getLogger(__name__ + ".watchdog")
259
+ initial_ppid = _os_wd.getppid()
260
+ if initial_ppid <= 1:
261
+ return # Already reparented at startup — don't self-terminate
262
+ while True:
263
+ _time.sleep(10)
264
+ try:
265
+ current_ppid = _os_wd.getppid()
266
+ if current_ppid != initial_ppid or current_ppid <= 1:
267
+ _wlog.info("Parent PID changed (%d→%d), self-terminating", initial_ppid, current_ppid)
268
+ _os_wd._exit(0)
269
+ _os_wd.kill(initial_ppid, 0) # Raises ProcessLookupError if dead
270
+ except ProcessLookupError:
271
+ _wlog.info("Parent PID %d gone, self-terminating", initial_ppid)
272
+ _os_wd._exit(0)
273
+ except Exception:
274
+ pass # Transient errors — keep watching
275
+
276
+
277
+ _watchdog_thread = threading.Thread(target=_parent_watchdog, daemon=True, name="parent-watchdog")
278
+ _watchdog_thread.start()
279
+
280
+
244
281
  if __name__ == "__main__":
245
282
  server.run(transport="stdio")
@@ -234,27 +234,47 @@ class SpreadingActivation:
234
234
  the highest-signal edges.
235
235
  """
236
236
  try:
237
+ # v3.4.59: LIMIT pushed inside each UNION branch so SQLite can use
238
+ # idx_edges_source_weight / idx_edges_target_weight and stop after
239
+ # max_neighbors_per_node rows per branch instead of materializing
240
+ # all 2.1M edges then sorting. Each branch wrapped in SELECT * FROM (...)
241
+ # because SQLite requires parentheses for ORDER BY+LIMIT in compound SELECTs.
242
+ lim = self._config.max_neighbors_per_node
237
243
  rows = self._db.execute(
238
244
  """
239
245
  SELECT neighbor_id, weight FROM (
240
- SELECT target_id AS neighbor_id, weight FROM graph_edges
246
+ SELECT * FROM (
247
+ SELECT target_id AS neighbor_id, weight FROM graph_edges
241
248
  WHERE source_id = ? AND profile_id = ?
249
+ ORDER BY weight DESC LIMIT ?
250
+ )
242
251
  UNION ALL
243
- SELECT target_fact_id AS neighbor_id, weight FROM association_edges
252
+ SELECT * FROM (
253
+ SELECT target_fact_id AS neighbor_id, weight FROM association_edges
244
254
  WHERE source_fact_id = ? AND profile_id = ?
255
+ ORDER BY weight DESC LIMIT ?
256
+ )
245
257
  UNION ALL
246
- SELECT source_id AS neighbor_id, weight FROM graph_edges
258
+ SELECT * FROM (
259
+ SELECT source_id AS neighbor_id, weight FROM graph_edges
247
260
  WHERE target_id = ? AND profile_id = ?
261
+ ORDER BY weight DESC LIMIT ?
262
+ )
248
263
  UNION ALL
249
- SELECT source_fact_id AS neighbor_id, weight FROM association_edges
264
+ SELECT * FROM (
265
+ SELECT source_fact_id AS neighbor_id, weight FROM association_edges
250
266
  WHERE target_fact_id = ? AND profile_id = ?
267
+ ORDER BY weight DESC LIMIT ?
268
+ )
251
269
  )
252
270
  ORDER BY weight DESC
253
271
  LIMIT ?
254
272
  """,
255
- (node_id, profile_id, node_id, profile_id,
256
- node_id, profile_id, node_id, profile_id,
257
- self._config.max_neighbors_per_node),
273
+ (node_id, profile_id, lim,
274
+ node_id, profile_id, lim,
275
+ node_id, profile_id, lim,
276
+ node_id, profile_id, lim,
277
+ lim),
258
278
  )
259
279
  return [
260
280
  (dict(r)["neighbor_id"], dict(r)["weight"]) for r in rows
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: superlocalmemory
3
- Version: 3.4.56
3
+ Version: 3.4.59
4
4
  Summary: Information-geometric agent memory with mathematical guarantees
5
5
  Author-email: Varun Pratap Bhardwaj <admin@superlocalmemory.com>
6
6
  License: AGPL-3.0-or-later