superlocalmemory 3.6.5 → 3.6.7

package/CHANGELOG.md

@@ -5,6 +5,70 @@ All notable changes to SuperLocalMemory V3 will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.6.7] - 2026-06-10 — MCP Streamable-HTTP Transport (Embedded)
+
+### Added
+- **Embedded MCP HTTP Transport:** The FastMCP server is now mounted directly inside the unified daemon at `/mcp`. All MCP clients (Claude Code, subagents, desktop, Hermes) now share a single daemon process instead of spawning separate `slm mcp` subprocesses per connection, eliminating process overhead and orphaned process risks.
+- **Graceful Fallback:** If the HTTP mount fails, the daemon continues to operate normally, preserving stdio transport compatibility.
+
+### Changed
+- **Thread Suppression in Daemon Context:** Three background threads that are safe in a standalone `slm mcp` subprocess but harmful inside the daemon are now suppressed when `SLM_MCP_EMBEDDED=1`:
+  - `mcp-warmup` thread (prevents duplicate LIGHT engine creation)
+  - `parent-watchdog` thread (prevents accidental daemon termination via `os._exit(0)`)
+  - `stdin-eof-monitor` thread (irrelevant inside the daemon process)
+- **Session Manager Idempotency:** Added defensive reset of `_session_manager` during app creation to guarantee safe re-initialization if the app factory is called multiple times in the same process.
+
+### Fixed
+- **Lifespan Ordering:** Ensured the MCP Streamable-HTTP session manager lifecycle is correctly wrapped in an `AsyncExitStack` within the daemon's lifespan, preventing "Task group is not initialized" errors on `/mcp` requests.
+- **Route Prefix Stripping:** Explicitly set `streamable_http_path="/"` on the FastMCP instance so that FastAPI's mount prefix stripping correctly routes requests to the sub-app's root endpoint.
+
+## [3.6.6] - 2026-06-10 — Recall precision & memory hygiene
+
+Memory means providing the best data, not the most data. v3.6.6 makes recall
+output disciplined and the write path quality-gated, with full parity across
+MCP, CLI, the daemon HTTP route, the in-process adapter, and the WorkerPool
+fallback (identical output regardless of surface or mode A/B).
+
+### Added
+
+- **Evidence floor (recall):** results must earn retrieval evidence — semantic
+  cosine ≥ 0.60, or BM25 / entity-graph / temporal signal, or a pinned fact.
+  Associative-only channels (spreading-activation, hopfield) no longer fabricate
+  matches. A query with no confident match returns an empty result set plus
+  `no_confident_match: true` instead of filler. Config:
+  `retrieval.evidence_floor_enabled`, `retrieval.min_semantic_evidence`.
+  Kill-switch: `SLM_RECALL_NO_FLOOR=1`.
+- **Recall output budget:** per-fact content clamp (default 2,400 chars,
+  head+tail preserved) and per-response budget (default 12,000 chars; remaining
+  results returned as stubs). New optional `full=true` parameter bypasses
+  clamping; clamped results carry `truncated: true`. Config:
+  `retrieval.recall_per_fact_max_chars`, `retrieval.recall_total_max_chars`.
+- **source_content discipline:** `source_content` defaults to a ≤280-char
+  preview (`include_source=true` restores full); internal prompt-template
+  content is never returned.
+- **Ingest gate (remember):** content over 24,000 chars is stored as a
+  head+tail-clamped fact while the full original is preserved in the source
+  memory record; content over 1MB is rejected. Prompt-template text can no
+  longer be stored as a memory. Config: `store.max_verbatim_chars`,
+  `store.max_ingest_bytes`. Kill-switch: `SLM_INGEST_NO_GATE=1`.
+- **Core-block hygiene:** core memory blocks now deduplicate lines at compile
+  time, drop low-quality/template source facts, enforce a per-block char cap,
+  and recompile on the daily maintenance schedule.
+
+### Compatibility
+
+- All MCP/HTTP/CLI signatures unchanged; new parameters and response fields are
+  additive. No database schema migrations. Every new behavior has a config
+  field and an environment kill-switch.
+
+## [3.6.5] - 2026-06-09 — Dependency-check hardening
+
+### Fixed
+
+- Dependency version check no longer eager-imports `torch` into every process
+  (caused a Python 3.14 test segfault and Apple-Silicon memory blow-up); now
+  uses `importlib.metadata.version()`.
+
 ## [3.6.4] - 2026-06-09 — Memory-integrity & reliability hardening
 
 ### Fixed

package/README.md

@@ -309,6 +309,32 @@ slm status
 
 ### MCP Integration (Claude, Cursor, Windsurf, VS Code, etc.)
 
+SLM supports **two MCP transports** — use whichever fits your tool. Both expose the same 33 tools and 7 resources.
+
+#### Option A — HTTP transport (v3.6.7+, recommended)
+
+One shared process handles every client. RAM is flat regardless of how many IDE windows, subagents, or concurrent sessions connect. Requires the SLM daemon to be running (`slm start`).
+
+```json
+{
+  "mcpServers": {
+    "superlocalmemory": {
+      "type": "http",
+      "url": "http://127.0.0.1:8765/mcp/"
+    }
+  }
+}
+```
+
+> **Claude Code** also accepts:
+> ```bash
+> claude mcp add --transport http superlocalmemory http://127.0.0.1:8765/mcp/
+> ```
+
+#### Option B — stdio transport (universal, works everywhere)
+
+Spawns one `slm mcp` subprocess per client connection (~90–110 MB each). Works with every MCP-compatible tool including those that do not yet support HTTP transport. No daemon required.
+
 ```json
 {
   "mcpServers": {
@@ -320,7 +346,36 @@ slm status
 }
 ```
 
-33 MCP tools by default (+42 optional behind `SLM_MCP_ALL_TOOLS=1`) + 7 resources. Works with any MCP-compatible client — we ship templated configs for Claude Code, Cursor, Windsurf, VS Code Copilot, Continue, Cody, ChatGPT Desktop, Gemini CLI, JetBrains, Zed, and Antigravity (15 IDE configs in `ide/configs/`). **V3.3: Adaptive lifecycle, smart compression, and pattern learning.**
+#### Option C — `mcp-remote` bridge (for stdio-only tools that want HTTP)
+
+Some CLIs (e.g. Grok CLI) only speak stdio but you still want the RAM benefit of HTTP. The [`@modelcontextprotocol/client-cli`](https://www.npmjs.com/package/@modelcontextprotocol/client-cli) package bridges them:
+
+```bash
+npm install -g @modelcontextprotocol/client-cli
+```
+
+```json
+{
+  "mcpServers": {
+    "superlocalmemory": {
+      "command": "mcp-remote",
+      "args": ["http://127.0.0.1:8765/mcp/", "--allow-http", "--transport", "http-only"]
+    }
+  }
+}
+```
+
+#### When to use which
+
+| Situation | Use |
+|-----------|-----|
+| Claude Code / Claude Desktop (v3.6.7+) | **HTTP** — zero new processes per session |
+| Cursor, Windsurf, Gemini CLI, Antigravity | **HTTP** — native support |
+| Grok CLI, tools that only support stdio | **`mcp-remote` bridge** |
+| Offline / daemon-free usage | **stdio** |
+| Any tool, any version | **stdio** always works as fallback |
+
+See [`docs/ide-setup.md`](docs/ide-setup.md) for per-IDE configs. 33 MCP tools by default (+42 optional behind `SLM_MCP_ALL_TOOLS=1`) + 7 resources. Works with any MCP-compatible client — we ship templated configs for Claude Code, Cursor, Windsurf, VS Code Copilot, Continue, Cody, ChatGPT Desktop, Gemini CLI, JetBrains, Zed, and Antigravity (15 IDE configs in `ide/configs/`).
 
 ### Dual Interface: MCP + CLI
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "superlocalmemory",
-  "version": "3.6.5",
+  "version": "3.6.7",
   "description": "Information-geometric agent memory with mathematical guarantees. 4-channel retrieval, Fisher-Rao similarity, zero-LLM mode, EU AI Act compliant. Works with Claude, Cursor, Windsurf, and 17+ AI tools.",
   "keywords": [
     "ai-memory",

package/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "superlocalmemory"
-version = "3.6.5"
+version = "3.6.7"
 description = "Information-geometric agent memory with mathematical guarantees"
 readme = "README.md"
 license = {text = "AGPL-3.0-or-later"}

package/src/superlocalmemory/__init__.py

@@ -28,7 +28,7 @@ if "OMP_NUM_THREADS" not in os.environ:
     os.environ["OMP_NUM_THREADS"] = "2"
 # ---------------------------------------------------------------------------
 
-__version__ = "3.6.5"
+__version__ = "3.6.7"
 
 _REQUIRED_VERSIONS = {
     "sentence_transformers": "5.3.0",

package/src/superlocalmemory/cli/commands.py

@@ -1001,7 +1001,9 @@ def cmd_recall(args: Namespace) -> None:
                     ])
                     return
                 if not result["results"]:
-                    print("No matching memories found.")
+                    print("No confident match."
+                          if result.get("no_confident_match")
+                          else "No matching memories found.")
                     return
                 # Text output
                 print(f"SpreadingActivation.search completed via daemon ({result.get('retrieval_time_ms', 0):.0f}ms)")
@@ -1030,20 +1032,38 @@ def cmd_recall(args: Namespace) -> None:
             sys.exit(1)
         raise
 
+    # v3.6.6: route the direct-fallback path through the SAME shared
+    # serializer the daemon uses, so CLI-without-daemon output is identical
+    # to CLI/MCP-with-daemon (budget + source discipline + no_confident_match).
+    from superlocalmemory.server.recall_serializer import serialize_recall_response
+    _rc = getattr(config, "retrieval", None)
+    _ser, _no_match = serialize_recall_response(
+        response,
+        limit=args.limit,
+        per_fact_max=getattr(_rc, "recall_per_fact_max_chars", 2400),
+        total_max=getattr(_rc, "recall_total_max_chars", 12000),
+        full=getattr(args, "full", False),
+    )
+
     if use_json:
         from superlocalmemory.cli.json_output import json_print
         items = []
-        for r in response.results:
+        for d in _ser:
             item = {
-                "fact_id": r.fact.fact_id, "content": r.fact.content,
-                "score": round(r.score, 3),
+                "fact_id": d["fact_id"], "content": d["content"],
+                "score": round(d["score"], 3),
             }
-            if hasattr(r, "channel_scores") and r.channel_scores:
-                item["channel_scores"] = {k: round(v, 3) for k, v in r.channel_scores.items()}
+            if d.get("channel_scores"):
+                item["channel_scores"] = {k: round(v, 3) for k, v in d["channel_scores"].items()}
+            if d.get("truncated"):
+                item["truncated"] = True
+            if d.get("stub"):
+                item["stub"] = True
             items.append(item)
         json_print("recall", data={
             "results": items, "count": len(items),
             "query_type": getattr(response, "query_type", "unknown"),
+            "no_confident_match": _no_match,
         }, next_actions=[
             {"command": "slm list --json", "description": "List recent memories"},
         ])
@@ -1055,11 +1075,11 @@ def cmd_recall(args: Namespace) -> None:
     except Exception:
         pass
 
-    if not response.results:
-        print("No memories found.")
+    if not _ser:
+        print("No confident match." if _no_match else "No memories found.")
         return
-    for i, r in enumerate(response.results, 1):
-        print(f"  {i}. [{r.score:.2f}] {r.fact.content[:120]}")
+    for i, d in enumerate(_ser, 1):
+        print(f"  {i}. [{d['score']:.2f}] {d['content']}")
 
 
 def _cli_record_signals(config, query, results):

package/src/superlocalmemory/core/block_hygiene.py

@@ -0,0 +1,147 @@
+# Copyright (c) 2026 Varun Pratap Bhardwaj / Qualixar
+# Licensed under AGPL-3.0-or-later - see LICENSE file
+# Part of SuperLocalMemory V3 — Core Block Hygiene (v3.6.6)
+
+"""Core Memory Block hygiene helpers (v3.6.6 F-5).
+
+Three pure functions used by the block compiler:
+  - dedupe_block_content: normalized-line dedup within a block
+  - filter_low_quality_block_facts: drop is_low_quality facts
+  - compile_block_content: full compile pipeline (filter + dedup + cap)
+
+Also exports: _recompile_core_blocks — the hook called by MaintenanceScheduler.
+
+These are pure functions (no I/O). All I/O lives in the scheduler / consolidation
+engine that calls them.
+"""
+
+from __future__ import annotations
+
+import logging
+
+logger = logging.getLogger(__name__)
+
+_BLOCK_SEPARATOR = "\n---\n"
+_PLACEHOLDER = "No data available."
+
+
+# ---------------------------------------------------------------------------
+# dedupe_block_content
+# ---------------------------------------------------------------------------
+
+def dedupe_block_content(lines: list[str]) -> list[str]:
+    """Remove duplicate and empty lines from a block content line-list.
+
+    Normalization: lowercased, whitespace-collapsed.
+    Order is preserved; only the FIRST occurrence is kept.
+
+    Returns a new list — never mutates input.
+    """
+    seen: set[str] = set()
+    result: list[str] = []
+    for line in lines:
+        stripped = line.strip()
+        if not stripped:
+            continue
+        key = " ".join(stripped.lower().split())
+        if key in seen:
+            continue
+        seen.add(key)
+        result.append(line)
+    return result
+
+
+# ---------------------------------------------------------------------------
+# filter_low_quality_block_facts
+# ---------------------------------------------------------------------------
+
+def filter_low_quality_block_facts(facts: list[dict]) -> list[dict]:
+    """Filter fact dicts whose content is low-quality or prompt-template.
+
+    Delegates to injection.is_low_quality and injection.is_prompt_template.
+    Returns a new list — never mutates input.
+    """
+    try:
+        from superlocalmemory.core.injection import is_low_quality, is_prompt_template
+    except Exception:
+        return list(facts)
+
+    result: list[dict] = []
+    for f in facts:
+        content = f.get("content", "") or ""
+        if is_low_quality(content):
+            continue
+        if is_prompt_template(content):
+            continue
+        result.append(f)
+    return result
+
+
+# ---------------------------------------------------------------------------
+# compile_block_content
+# ---------------------------------------------------------------------------
+
+def compile_block_content(
+    facts: list[dict],
+    max_chars: int = 2000,
+) -> str:
+    """Compile facts into block content with hygiene and char cap.
+
+    Pipeline:
+      1. filter_low_quality_block_facts
+      2. Extract content lines from each fact (split by newline / separator)
+      3. dedupe_block_content across all lines
+      4. Join with separator, truncate to max_chars
+
+    Returns a string ≤ max_chars. Returns empty string if all facts filtered.
+    """
+    clean_facts = filter_low_quality_block_facts(facts)
+    if not clean_facts:
+        return ""
+
+    all_lines: list[str] = []
+    for f in clean_facts:
+        content = (f.get("content") or "").strip()
+        if not content:
+            continue
+        # Split by separator or newline to treat each line independently
+        for line in content.replace(_BLOCK_SEPARATOR, "\n").split("\n"):
+            all_lines.append(line)
+
+    deduped = dedupe_block_content(all_lines)
+    if not deduped:
+        return ""
+
+    joined = _BLOCK_SEPARATOR.join(deduped)
+    return joined[:max_chars]
+
+
+# ---------------------------------------------------------------------------
+# _recompile_core_blocks — scheduler hook (F-5 daily recompile)
+# ---------------------------------------------------------------------------
+
+def _recompile_core_blocks(
+    db,
+    config,
+    profile_id: str,
+) -> dict:
+    """Recompile core memory blocks with hygiene applied.
+
+    Called by MaintenanceScheduler._run() on the daily cycle.
+    Delegates to ConsolidationEngine.compile_core_blocks_mode_a() with
+    the hygiene improvements applied at the _facts_to_content step.
+
+    Returns a dict with stats: {blocks_compiled, profile_id}.
+    """
+    try:
+        from superlocalmemory.core.consolidation_engine import ConsolidationEngine
+        engine = ConsolidationEngine(db=db, config=config.consolidation)
+        result = engine.compile_core_blocks_mode_a(profile_id)
+        logger.info(
+            "Daily core-block recompile: profile=%s blocks=%s",
+            profile_id, result.get("blocks_compiled", 0),
+        )
+        return {**result, "profile_id": profile_id}
+    except Exception as exc:
+        logger.warning("Core-block recompile failed: %s", exc)
+        return {"blocks_compiled": 0, "profile_id": profile_id, "error": str(exc)}

package/src/superlocalmemory/core/config.py

@@ -197,6 +197,17 @@ class RetrievalConfig:
     # Used by s19_runner for ablation experiments. Empty = all channels active.
     disabled_channels: list[str] = field(default_factory=list)
 
+    # v3.6.6: Evidence floor — gate on per-channel scores, not fused/RRF score.
+    # Nonsense queries earn 0.0 on every primary channel; real matches earn
+    # semantic >= 0.85 or bm25 > 0. The discriminator is earned channel evidence.
+    # Env kill-switch: SLM_RECALL_NO_FLOOR=1 disables without release.
+    evidence_floor_enabled: bool = True
+    min_semantic_evidence: float = 0.60   # Minimum cosine similarity to keep a result
+
+    # v3.6.6: Recall output budget — protect consuming agents from 585KB responses.
+    recall_per_fact_max_chars: int = 2400  # ~600 tokens; head 70% + tail 30%
+    recall_total_max_chars: int = 12000    # ~3K tokens; stubs beyond this
+
 
 # ---------------------------------------------------------------------------
 # Math Config
@@ -233,6 +244,30 @@ class MathConfig:
     # Rate-Distortion (production only, disabled for benchmarks)
 
 
+# ---------------------------------------------------------------------------
+# Store Config (v3.6.6)
+# ---------------------------------------------------------------------------
+
+@dataclass(frozen=True)
+class StoreConfig:
+    """Configuration for the remember/store write path (v3.6.6).
+
+    Ingest gate: protect the DB from oversized facts and prompt-template
+    pollution. Defaults ON; env kill-switch SLM_INGEST_NO_GATE=1.
+    """
+
+    # Max chars for the FACT content stored in atomic_facts.content.
+    # Content above this is clamped to head 70% + tail 30% + truncation marker.
+    # The FULL original is preserved in the memories table row. Set high (24K
+    # ≈ 6K tokens) so only pathological pastes are touched; normal dense
+    # session-handoff memories (6-15K chars) are stored 100% intact.
+    max_verbatim_chars: int = 24000
+
+    # Hard upper bound in bytes. Content above this is rejected outright.
+    # Nobody's "memory" is a megabyte (MCP: success=False, HTTP: 413).
+    max_ingest_bytes: int = 1_048_576  # 1 MB
+
+
 # ---------------------------------------------------------------------------
 # Context Injection (v3.4.65)
 # ---------------------------------------------------------------------------
@@ -658,6 +693,7 @@ class SLMConfig:
         default_factory=ParameterizationConfig,
     )
     injection: InjectionConfig = field(default_factory=InjectionConfig)
+    store: StoreConfig = field(default_factory=StoreConfig)
     # v3.5.0: scaling backends — "sqlite" / "cozo" / "auto" / "lancedb" / "sqlite-vec" / "auto".
     graph_backend: str = "auto"       # "auto" = cozo if pycozo installed, else sqlite
     vector_backend: str = "auto"      # "auto" = lancedb if installed, else sqlite-vec

package/src/superlocalmemory/core/consolidation_engine.py

@@ -808,22 +808,25 @@ class ConsolidationEngine:
     def _facts_to_content(
         self, facts: list[dict], char_limit: int,
     ) -> str:
-        """Join fact contents with separators, capped at char_limit."""
-        parts = [f.get("content", "") for f in facts if f.get("content")]
-        joined = "\n---\n".join(parts)
-        return joined[:char_limit] if joined else "No data available."
+        """Compile fact contents into a block with hygiene (v3.6.6 F-5).
+
+        Filters low-quality/template facts, dedupes lines WITHIN the block
+        (fixes the "same fixture ×5" core-block bug), caps at char_limit.
+        """
+        from superlocalmemory.core.block_hygiene import compile_block_content
+        compiled = compile_block_content(facts, max_chars=char_limit)
+        return compiled if compiled else "No data available."
 
     def _rows_to_content(
         self, rows: list | None, char_limit: int,
     ) -> str:
-        """Convert DB rows to content string."""
+        """Convert DB rows to a hygienic block content string (v3.6.6 F-5)."""
         if not rows:
             return "No data available."
-        parts = [
-            dict(r).get("content", "") for r in rows if dict(r).get("content")
-        ]
-        joined = "\n---\n".join(parts)
-        return joined[:char_limit] if joined else "No data available."
+        from superlocalmemory.core.block_hygiene import compile_block_content
+        facts = [dict(r) for r in rows]
+        compiled = compile_block_content(facts, max_chars=char_limit)
+        return compiled if compiled else "No data available."
 
     def _compile_behavioral_block(
         self, profile_id: str, char_limit: int,

package/src/superlocalmemory/core/engine.py

@@ -430,6 +430,25 @@ class MemoryEngine:
                 return []
         except Exception:
             pass
+        # v3.6.6 ingest gate: reject 1MB monsters + prompt-template pollution;
+        # clamp the searchable FACT copy (head+tail) while the memories row keeps
+        # the FULL original. Embedding/BM25 use the clamped copy so a 167KB paste
+        # never produces a garbage vector. Env kill-switch: SLM_INGEST_NO_GATE=1.
+        fact_text = content
+        try:
+            from superlocalmemory.core.ingest_gate import apply_ingest_gate
+            _sc = getattr(self._config, "store", None)
+            gate = apply_ingest_gate(
+                content,
+                max_verbatim_chars=getattr(_sc, "max_verbatim_chars", 24000),
+                max_ingest_bytes=getattr(_sc, "max_ingest_bytes", 1_048_576),
+            )
+            if gate.rejected:
+                logger.debug("store_fast ingest gate rejected: %s", gate.rejection_reason)
+                return []
+            fact_text = gate.fact_content
+        except ImportError:
+            pass  # gate module missing → store verbatim (never block a write)
         now = datetime.now(timezone.utc).isoformat()
         record = MemoryRecord(
             profile_id=self._profile_id, content=content,
@@ -440,8 +459,8 @@ class MemoryEngine:
         # the entity_graph channel has something to work with before enrichment.
         ents = sorted(
             {m.group(1) for m in _re.finditer(
-                r"\b([A-Z][a-z]+(?:\s[A-Z][a-z]+){0,3})\b", content)}
-            | {m.group(1) for m in _re.finditer(r"\b([A-Z]{2,})\b", content)}
+                r"\b([A-Z][a-z]+(?:\s[A-Z][a-z]+){0,3})\b", fact_text)}
+            | {m.group(1) for m in _re.finditer(r"\b([A-Z]{2,})\b", fact_text)}
         )
         # v3.5.5: compute the embedding SYNCHRONOUSLY. A single warm embed is
         # ~22ms (the 30-180s of full store() was LLM fact-extraction + graph,
@@ -452,14 +471,14 @@ class MemoryEngine:
         emb = None
         fmean = fvar = None
         try:
-            emb = self._embedder.embed(content) if self._embedder else None
+            emb = self._embedder.embed(fact_text) if self._embedder else None
             if emb:
                 fmean, fvar = self._embedder.compute_fisher_params(emb)
         except Exception:
             emb = None
         fact = AtomicFact(
             fact_id=_uuid.uuid4().hex[:16], memory_id=record.memory_id,
-            profile_id=self._profile_id, content=content,
+            profile_id=self._profile_id, content=fact_text,
             fact_type=FactType.EPISODIC, entities=ents,
             observation_date=now[:10], confidence=0.7, importance=0.5,
             embedding=emb, fisher_mean=fmean, fisher_variance=fvar,
@@ -477,7 +496,7 @@ class MemoryEngine:
         try:
             bm25 = getattr(self._retrieval_engine, "_bm25", None)
             if bm25:
-                bm25.add(fact.fact_id, content, self._profile_id)
+                bm25.add(fact.fact_id, fact_text, self._profile_id)
         except Exception:
             pass
         return [fact.fact_id]