superlocalmemory 3.6.9 → 3.6.11

package/CHANGELOG.md

@@ -5,20 +5,122 @@ All notable changes to SuperLocalMemory V3 will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [3.6.9] - 2026-06-11
+## [3.6.11] - 2026-06-14 — Optimize Everywhere: three surfaces (proxy · MCP tools · skill)
 
-### Fixed
-- **BUG-A: Health monitor no longer kills the embedding worker under memory pressure.** The watchdog now spares the embedding worker (load-bearing for recall quality) and prefers the reranker instead. The RSS budget defaults to 40% of physical RAM (floor 2500 MB) rather than a hardcoded 2500 MB, preventing thrash on machines with ≥8 GB. `SLM_RSS_BUDGET_MB` env and a new `"health"` config section give full operator control. `HealthConfig` dataclass + `SLMConfig.load()` parsing added so config.json `"health"` keys now actually take effect.
-- **#34: Mesh tools no longer block the daemon event loop.** All 8 async mesh tools wrapped `_ensure_registered` and `_mesh_request` in `asyncio.to_thread` so the blocking loopback HTTP calls leave the event loop (v3.6.7 in-process transport made this a self-deadlock). MCP lifespan hardened so a tool-level exception cannot propagate to uvicorn's shutdown handler. `heartbeat_active` and `registered` now return live state instead of hardcoded literals.
-- **#35: `session_init` now returns a `session_id`.** Clients pass it to `remember()` and `close_session()`. `store_fast` lifts `session_id` from metadata onto the `MemoryRecord` row so facts are correctly attributed. `close_session` queries the DB for the most recent session instead of chasing a phantom `_last_session_id` that was never assigned — `summary_events_created` now returns real counts.
-- **#36-1: HTTP MCP reachable from LAN.** New `SLM_MCP_ALLOWED_HOSTS` (opt-in, default localhost-only) overrides MCP's DNS-rebinding protection. Accepts comma-separated `host:port*` patterns or `*` to disable protection entirely on a trusted LAN.
-- **#36-2: `slm mcp` and the HTTP daemon no longer race for port 8765.** `ensure_daemon` now checks TCP connectivity in addition to PID file + HTTP health, detecting systemd-started daemons mid-startup before attempting a second bind. `SLM_DAEMON_PORT` is now fully wired end-to-end (previously only read in `commands.py` URL building, not in the actual uvicorn bind or `_start_daemon_subprocess`).
-- **#32: Docs correctly state Python 3.11+ requirement** (was "3.10 or later"). Added Ubuntu 22.04 deadsnakes install instructions and a new `docs/install-linux.md` guide.
+Cache + compress across **every setup** — proxy, MCP tools, or skill. Five new MCP tools land directly inside `slm mcp` (no proxy, full 1M context window preserved). A new `slm-optimize` skill makes compression and routed-result caching zero-config for Claude Code users. Overclaim in prior docs fixed; three-surfaces table added.
+
+### Added — Surface B: MCP Optimize Tools (proxy-free)
+
+Five new tools registered in `slm mcp` from v3.6.11+:
+
+- **`slm_compress`** — compress text or tool output. `mode=normalize` is lossless whitespace normalization; `mode=auto`/`aggressive` delegates to the existing CompressRouter engine. Lossy + reversible stores the original in CCR and returns a `ccr_id`. Content >1 MB is stored unreversed. All fail-open.
+- **`slm_retrieve`** — recover the exact original bytes from a `ccr_id` UUID4 returned by `slm_compress`. Validates UUID format before lookup.
+- **`slm_cache_set`** — cache any string result the agent routes through SLM (file reads, bash output, search results, sub-model calls). Key is SHA-256 namespaced `mcpkv:{agent_id}:{key}` — no cross-agent collisions. TTL default 24 h.
+- **`slm_cache_get`** — retrieve a cached result by key. Returns `hit:True/False`. Increments in-module `_kv_hits`/`_kv_misses` counters for stats.
+- **`slm_optimize_stats`** — session compression + cache statistics. Proxy counters read from `CacheDB.metrics_load()` (daemon-persisted, accurate across restarts); KV counters are in-module for this MCP process session.
+
+All five are **fail-open**: any internal error returns `{ok: False, note: <error>}` with the original content unchanged — never raises to the agent.
+
+### Added — Surface C: slm-optimize Skill (zero-config)
+
+- **`skills/slm-optimize/SKILL.md`** and **`ide/skills/slm-optimize/SKILL.md`** — agent-behavior instruction file. Eight behavioral rules: compress CLAUDE.md at session start, compress large tool outputs via `slm_compress`, KV-cache repeated file reads and bash/search results, recover originals with `slm_retrieve`, never-compress list (secrets, JSON, code/Edit/Write outputs), emit stats on request, fail-open on `ok:False`. Compatible with Claude Code, Cursor, Antigravity, Codex, and any IDE that supports MCP tool calls.
+- **`skills/slm-optimize/README.md`** — human install guide: prerequisites, install steps, activation options, feature table, and explicit "what it does NOT do" section.
+
+### Changed — `CacheDB` (new public method)
+
+- **`CacheDB.get_value(cache_key, tenant_id)`** added to `optimize/storage/db.py` — pure `SELECT` lookup with no `hit_count` side-effect (unlike `get()`). Used by `slm_cache_get` to avoid inflating proxy cache counters when agents read KV entries. Fail-open: returns `None` on any SQLite/decryption/zlib error.
+
+### Changed — MCP server registration
+
+- Five optimize tool names added to `_ESSENTIAL_TOOLS` frozenset in `mcp/server.py` (v3.6.11 Surface B). Without this the `_FilteredServer` wrapper silently drops tools whose names are not in the set.
+- `register_optimize_tools(_target)` call added after other `register_*` calls.
+
+### Fixed — Overclaim in README / docs
+
+- Replaced "Save up to 90% on every LLM API call" headline with accurate three-surfaces framing: proxy (full-turn caching on metered API), MCP tools (routed-result caching + compression), skill (auto-applied compression). Hard constraint documented: primary Claude conversation turn cannot be cached without a proxy — never implied otherwise.
+
+## [3.6.10] - 2026-06-14 — Optimize correctness (cache + lossless compression) · MCP per-agent identity · runtime toggles · benchmark + shadow-capture · Issue #38
+
+This release makes the **Optimize** subsystem (the HTTP proxy that caches and compresses LLM API calls) correct, observable, and **independently controllable at runtime**, adds **per-agent identity** to the HTTP MCP transport, ships a **benchmark + shadow-capture** harness that proves the cache and compression behaviour, and fixes GitHub issue #38. Cache and compression remain **default-OFF** and are now separately toggleable from the dashboard.
+
+### Added — Independent runtime control of cache vs compression
+
+- **Cache and compression are separate switches**, both live at runtime from the dashboard — enable caching only, compression only, both, or neither, without restarting the daemon. The config watchdog rebuilds the proxy hook chain in place on change.
+
+### Added — MCP per-agent identity over HTTP (`/mcp/{agent_id}`)
+
+- **Per-agent attribution without per-agent processes.** The HTTP MCP endpoint accepts an agent-id path segment — `http://127.0.0.1:8765/mcp/claude`, `/mcp/hermes`, `/mcp/gemini`, etc. The daemon extracts it (root_path-aware ASGI wrapper) into a per-request `ContextVar`, so `remember`, `recall`, `observe`, `delete_memory`, `update_memory`, `session_init`, and event emission all tag the correct agent. Many agents share one daemon instead of one `slm mcp` stdio process each. Bare `/mcp/` is unchanged → default `mcp_client` (backward compatible). Precedence: URL path → `SLM_AGENT_ID` env (stdio) → `mcp_client`. URL agent-ids are sanitized (charset-restricted, 64-char cap). See `docs/distributed-deployment.md`.
+
+### Added — Benchmark + shadow-capture (`benchmarks/optimize/`)
+
+- **Correctness benchmark** drives the shipped cache + compression code: exact-cache replay is 100% hit + byte-identical, exact false-hit rate is **0** (the wrong-answer guard), semantic-tier wiring honours its threshold, and safe-mode compression is **lossless** for JSON/code/prose (code forwarded unchanged). Enforced in CI by `tests/optimize/test_benchmark.py`.
+- **Shadow-capture mode** (`SLM_OPTIMIZE_CAPTURE=1`): pure-passthrough proxy records real `{request, response, model, tokens, content_type}` exchanges to `~/.superlocalmemory/optimize_capture.jsonl` (0600, `O_NOFOLLOW`, git-ignored, 1 MB/side cap, secrets never recorded) for replay into the benchmark. No cache/compress while capturing.
+
+### Changed — Compression rebuilt on a research-backed layered design
+
+- **Removed the homegrown extractive JSON/code compressors** (they truncated JSON strings, capped arrays, and stubbed code bodies — lossy and unsafe). Safe mode is now **lossless** (whitespace + compact-JSON normalization only; code untouched). Aggressive mode adds **LLMLingua-2** (`microsoft/llmlingua-2-xlm-roberta-large-meetingbank`) for **prose only** — never code, numbers, structured data, instructions, or the current turn. Dead `compress_json`/`compress_code` toggles removed.
+
+### Fixed — Cache correctness & observability
+
+- **Semantic cache tier is now wired** into the proxy `get/check/set` path (was dead code); default-off, conservative 0.98 return threshold, benchmarked false-hit guard. See `docs/optimize-config.md` for threshold-tuning guidance.
+- **Token accounting corrected** — input tokens (the biggest save) and real output tokens are now counted on cache hits instead of 0/estimates.
+- **Cache survives machine-id changes** — the AES-GCM key/salt is persisted (0600), fixing the total-miss after VM clone / container / OS reinstall.
+- **UI opt-in is real** — a `proxy_enabled` toggle now actually mounts the proxy.
+
+### Fixed — Issue #38 (frontend)
+
+- **Brain page no longer stuck on "Couldn't load Brain"** on a healthy backend — pane activation fires `fetchBrain` and reads the install token from the same source as the working endpoints.
+- **"Test Connection" no longer 401s on an empty API key** in no-auth (Mode B) — when the key is empty no `Authorization` header is sent.
+
+### Security (v3.6.10 audit, Stage 8/9)
+
+- Capture file: `O_NOFOLLOW` + unconditional `0600` (symlink-append + TOCTOU closed); in-memory stream accumulator bounded (CWE-400); synchronous capture writes offloaded off the event loop; `extract_usage` uses an explicit provider allowlist (no silent mis-parse). URL agent-ids sanitized at the single extraction chokepoint.
+
+## [3.6.9] - 2026-06-11 — Full 7-layer recall quality · event-loop safety · health watchdog
+
+This release fixes seven GitHub issues, one critical production incident, five post-implementation audit bugs, and two event-loop deadlocks introduced in v3.6.7. All 7 retrieval layers (semantic, BM25, temporal, spreading activation, Hopfield, entity graph, cross-encoder reranker) now operate at full designed quality with no capability tradeoffs.
+
+### Fixed — Production incident
+
+- **BUG-A (CRITICAL): Health monitor RSS watchdog no longer kills the embedding worker.** On machines with <16 GB RAM the hardcoded 2500 MB budget caused the watchdog to kill the ~1 GB ONNX embedding worker during recall bursts. Semantic channel silently scored 0.0, recall fell back to keyword-only FTS5/BM25 ("DEGRADED MODE"), and the v3.6.8 health monitor reported false-healthy because it checked worker liveness rather than semantic signal quality. Fixed by: (1) defaulting the budget to 40% of physical RAM (floor 2500 MB) with `SLM_RSS_BUDGET_MB` env override; (2) protecting the embedder from the kill list — the reranker is targeted first as it can be recreated without losing recall quality; (3) adding `HealthConfig` dataclass and `SLMConfig.load()` parsing so `config.json` `"health"` keys now actually take effect.
+
+### Fixed — GitHub issues
+
+- **#34: Mesh tools no longer deadlock the daemon.** All 8 async mesh MCP tools called blocking `urllib.urlopen` loopback HTTP against the daemon's own `/mcp` endpoint introduced in v3.6.7. From inside the event loop this self-deadlocked uvicorn's single-thread executor, causing a graceful-shutdown timeout that killed the daemon. All 8 call sites wrapped in `asyncio.to_thread`. MCP lifespan hardened so a tool-level exception cannot propagate to uvicorn's shutdown handler. `heartbeat_active` and `registered` return live state instead of hardcoded literals.
+- **#35: `session_init` now returns a `session_id`.** Three spots fixed: (1) `tools_active.py` — `session_init` now passes `session_id` through to callers; (2) `engine.py` `store_fast` — lifts `session_id` from metadata onto the `MemoryRecord` row so facts are correctly attributed to sessions; (3) `close_session` — queries the DB for the most recent session instead of chasing a phantom `_last_session_id` that was never assigned, and `summary_events_created` now returns real counts.
+- **#36-1: HTTP MCP now reachable from LAN hosts.** The new `SLM_MCP_ALLOWED_HOSTS` env (opt-in, default localhost-only) overrides FastMCP's DNS-rebinding protection that rejected non-`127.0.0.1` `Host` headers on LAN deployments. Accepts comma-separated `host:port*` patterns or `*`. Security default is unchanged — the endpoint stays localhost-only without explicit opt-in.
+- **#36-2: `slm mcp` and the HTTP daemon no longer race for port 8765.** `ensure_daemon` now checks TCP connectivity in addition to PID file + HTTP health, detecting systemd-started daemons mid-startup before attempting a second bind. `SLM_DAEMON_PORT` is now fully wired end-to-end (previously only read in `commands.py` URL construction, ignored in the actual uvicorn bind and `_start_daemon_subprocess`). Wrapped in `try/except ValueError` so a malformed env value fails with a clear message rather than a silent zero-port bind.
+- **#33 + #37: Environment variable reference table published.** All ~90 `SLM_*` variables documented in `docs/distributed-deployment.md` with type, default, scope, and example. Closes both issues.
+- **#32: Python version requirement corrected.** Docs previously stated "3.10 or later" — the codebase enforces 3.11+. Updated `getting-started.md`, added Ubuntu 22.04 deadsnakes install path and a new `docs/install-linux.md`.
+
+### Fixed — Event-loop blocking (post-v3.6.7 audit)
+
+The v3.6.7 in-process HTTP MCP transport changed the execution context of all MCP tools from stdio subprocess threads to async event-loop coroutines. Three core tools made blocking HTTP or I/O calls that safe in a subprocess become event-loop deadlocks in the new context.
+
+- **`session_init` no longer blocks the event loop.** `pool_recall` called `DaemonPoolProxy.recall` which uses blocking `urllib.urlopen` internally. Wrapped in `asyncio.to_thread`.
+- **`observe` no longer blocks the event loop.** `auto.capture()` is a synchronous function that makes an internal HTTP call. Wrapped in `asyncio.to_thread`.
+- **`remember` no longer blocks the event loop.** Both `is_daemon_running()` (file-system check + HTTP probe) and `daemon_request()` (blocking HTTP POST) wrapped in `asyncio.to_thread`.
+
+### Fixed — Post-implementation audit (5 bugs)
+
+- **`close_session` crash eliminated.** `tools_active.py` `close_session` called `engine._db._get_conn()`, a private method removed in a prior refactor. Changed to the public `engine._db.execute()` API.
+- **`TransportSecuritySettings` import no longer fails on older MCP SDK versions.** The import was at module level in `unified_daemon.py`, causing an `ImportError` on MCP SDK < 1.27. Moved inside the `if _mcp_allowed:` conditional so it only loads when the HTTP transport is actually being activated.
+- **`SLM_DAEMON_PORT` no longer silently becomes 0 on invalid input.** `int(os.environ.get("SLM_DAEMON_PORT", "") or 8765)` evaluated to `int("")` → `ValueError` → port 0. Wrapped in `try/except ValueError` with a fallback to 8765 and a log warning.
+- **Health monitor cmdline truncation increased.** `cmdline[:80]` silently dropped long Python command lines (common with virtualenv paths), producing false "not SLM" negatives. Increased to `cmdline[:200]`.
+- **Daemon `_DEFAULT_PORT` no longer crashes on malformed env.** Same `ValueError` guard added to the module-level `_DEFAULT_PORT` assignment in `cli/daemon.py`.
+
+### Changed — Recall performance (zero quality tradeoff)
+
+- **SpreadingActivation: 418ms → 36ms (12×).** Neighbor lookups for each graph node are now cached across propagation iterations. The neighbor list is static within a single recall — re-querying SQL per iteration (up to ~120 queries per propagation) was pure waste. The cache is local to the call, so correctness is unchanged.
+- **`fast=True` deprecated.** `fast=True` was added in v3.4.40 when SpreadingActivation took 418ms. With SA now completing in ~36ms, `fast=True` (which disables SA) is actually *slower* than `fast=False` and reduces recall quality by dropping a full retrieval channel. `MemoryEngine.recall()` now logs a `WARNING` and silently treats `fast=True` as `False`. The parameter is retained for API backward compatibility and will be removed in v3.7.x. All built-in callers already passed `fast=False`.
+- Hopfield `prefilter_candidates` and entity-graph scoring candidates retain their original designed values (1000 and 100 respectively) — no quality tradeoffs were made in pursuit of latency targets.
 
 ### Added
+
 - `HealthConfig` dataclass in `core/config.py` and `"health"` section parsing in `SLMConfig.load()`.
-- `docs/distributed-deployment.md` — complete guide for LXC/container/multi-machine setups, including a full ~90-entry `SLM_*` environment variable reference table (closes #33 + #37).
-- `docs/install-linux.md` — Ubuntu 22.04 / Debian install guide with venv, pipx, and pyenv options, plus a systemd unit template.
+- `SLM_RSS_BUDGET_MB` environment variable for operator control of the health watchdog kill threshold.
+- `docs/distributed-deployment.md` — complete guide for LXC/container/multi-machine setups with full `SLM_*` environment variable reference (closes #33 + #37).
+- `docs/install-linux.md` — Ubuntu 22.04 / Debian install guide with venv, pipx, and pyenv paths plus a systemd unit template.
 
 ## [3.6.8] - 2026-06-11 — Runtime recall-health monitor (self-healing recall)
 

package/README.md

@@ -2,18 +2,18 @@
   <img src="https://superlocalmemory.com/assets/logo-mark.png" alt="SuperLocalMemory" width="200"/>
 </p>
 
-<h1 align="center">SuperLocalMemory V3.6</h1>
-<p align="center"><strong>Save up to 90% on every LLM API call. Cache. Compress. Remember.</strong><br/><em>The only local-first memory system that SKIPS repeat calls (100% saved), SHRINKS prompts 60-95%, and REMEMBERS everything — locally, for free. For Claude Code, Cursor, Windsurf, and any AI client.</em></p>
-<p align="center"><code>v3.6.0 "Optimize"</code> — <strong>Cache & Compress & Align. Save up to 90% on every LLM API call — locally.</strong> One command: <code>slm wrap claude</code><br>Also includes v3.5 Scale-Ready: 6-channel recall &lt;1s, CozoDB + LanceDB, Core Memory Block. Your database auto-migrates.</p>
+<h1 align="center">SuperLocalMemory V3.6.11</h1>
+<p align="center"><strong>Cache. Compress. Remember. Three surfaces — proxy, MCP tools, or skill. Every setup covered.</strong><br/><em>The only local-first layer that pairs persistent memory with compression + caching across every Claude plan. Full 1M window preserved in MCP and skill mode.</em></p>
+<p align="center"><code>v3.6.11 "Optimize Everywhere"</code> — <strong>Compress + cache on any plan, three ways in.</strong><br/>Proxy (full-turn cache): <code>slm wrap claude</code> &nbsp;·&nbsp; MCP (proxy-free): add <code>slm_compress</code> to your MCP config &nbsp;·&nbsp; Skill (zero-config): <code>~/.claude/skills/slm-optimize/</code></p>
 <p align="center"><strong>Backed by 3 published research papers</strong> (arXiv preprints + Zenodo-archived) · <a href="https://arxiv.org/abs/2603.02240">arXiv:2603.02240</a> · <a href="https://arxiv.org/abs/2603.14588">arXiv:2603.14588</a> · <a href="https://arxiv.org/abs/2604.04514">arXiv:2604.04514</a></p>
 
 <p align="center">
-  <code>Saves up to 90% on LLM API costs</code> &nbsp;·&nbsp; <code>+10.6pp vs Mem0 zero-LLM</code> &nbsp;·&nbsp; <code>85% Open-Domain (best zero-LLM score)</code> &nbsp;·&nbsp; <code>EU AI Act Ready</code>
+  <code>Proxy · MCP tools · Skill — three surfaces</code> &nbsp;·&nbsp; <code>+10.6pp vs Mem0 zero-LLM</code> &nbsp;·&nbsp; <code>85% Open-Domain (best zero-LLM score)</code> &nbsp;·&nbsp; <code>EU AI Act Ready</code>
 </p>
 
 <p align="center">
   <a href="https://arxiv.org/abs/2603.14588"><img src="https://img.shields.io/badge/arXiv-2603.14588-b31b1b?style=for-the-badge&logo=arxiv&logoColor=white" alt="arXiv Paper"/></a>
-  <a href="https://img.shields.io/badge/Saves_90%25_on_LLM_Costs-22c55e?style=for-the-badge"><img src="https://img.shields.io/badge/Saves_90%25_on_LLM_Costs-22c55e?style=for-the-badge" alt="Saves 90% on LLM Costs"/></a>
+  <a href="#three-surfaces-proxy--mcp-tools--skill"><img src="https://img.shields.io/badge/Proxy_|_MCP_|_Skill-22c55e?style=for-the-badge" alt="Three Surfaces: Proxy, MCP Tools, Skill"/></a>
   <a href="https://pypi.org/project/superlocalmemory/"><img src="https://img.shields.io/pypi/v/superlocalmemory?style=for-the-badge&logo=pypi&logoColor=white" alt="PyPI"/></a>
   <a href="https://www.npmjs.com/package/superlocalmemory"><img src="https://img.shields.io/npm/v/superlocalmemory?style=for-the-badge&logo=npm&logoColor=white" alt="npm"/></a>
   <a href="https://www.gnu.org/licenses/agpl-3.0"><img src="https://img.shields.io/badge/License-AGPL_v3-blue.svg?style=for-the-badge" alt="AGPL v3"/></a>
@@ -33,18 +33,24 @@
 <details>
 <summary><strong>What's New in V3.6 — Optimize: SKIP, SHRINK, DISCOUNT, REMEMBER</strong> (click to expand)</summary>
 
-> V3.6 is the only local-first layer that SKIPS repeat LLM calls (cache: 100% saved), SHRINKS prompts 60-95% (compress: extractive + LLMLingua-2), and DISCOUNTS prefix costs (align: native KV-cache) — and remembers everything — in one install. **Your first cache hit pays for the install time. Hours of coding on repeat, minimal API cost.**
+> V3.6 is the only local-first layer that SKIPS repeat LLM calls (cache: 100% on a hit), SHRINKS tool outputs and injected context (compress: lossless-by-default, opt-in LLMLingua-2), and DISCOUNTS prefix costs (align: native KV-cache) — and remembers everything — in one install.
+>
+> **v3.6.11 "Optimize Everywhere":** Three surfaces. **Proxy** (Surface A) — full-turn cache + compress on transport; needs `ANTHROPIC_BASE_URL`, shrinks the context window. **MCP tools** (Surface B) — `slm_compress`, `slm_retrieve`, `slm_cache_set`, `slm_cache_get`, `slm_optimize_stats`; no proxy, no window shrink, works on any Claude subscription. **Skill** (Surface C) — `slm-optimize` installs in `~/.claude/skills/`; zero-config auto-compress for large tool outputs and CLAUDE.md. No proxy, full 1M window. [See Three Surfaces →](#three-surfaces-proxy--mcp-tools--skill)
+>
+> **v3.6.10:** cache and compression are now **independent runtime switches** (cache-only, compress-only, both, or neither — toggle live from the dashboard, no restart). Compression was rebuilt to be **lossless by default** (the old string/array/code truncation is gone); aggressive mode adds LLMLingua-2 for **prose only** — never code, numbers, structured data, or the current turn.
 
 ### The Three Levers
 
 | Lever | Mechanism | Saving | Off by default? |
 |-------|-----------|:------:|:---------------:|
-| **Cache** | Skip repeat calls — exact-match SQLite lookup, vCache-gated semantic (opt-in) | **100% on a hit** (input + output) | Cache ON, Semantic OFF |
-| **Compress** | Shrink prompts — extractive JSON/code (lossless) + LLMLingua-2 prose (opt-in) | **60–95% on a miss** (input only) | Safe mode ON, Aggressive OFF |
+| **Cache** | Skip repeat calls — exact-match SQLite lookup (zero false hits), vCache-gated semantic (opt-in) | **100% on a hit** (input + output) | Cache ON, Semantic OFF |
+| **Compress** | Shrink prompts — **safe = lossless** normalization; **aggressive = LLMLingua-2 prose only** (opt-in) | Safe: small + lossless · Aggressive: large on prose | Safe mode, Aggressive OFF |
 | **Align** | Stabilize prefix — maximize provider prefix-cache discounts | **Lossless extra** | ON when compression is ON |
 
 **Memory** (v3.5's existing engine) runs in parallel — it shapes *what is in* the prompt (relevant facts); Optimize decides *whether and how* it is sent.
 
+> **Independent at runtime:** enable caching only, compression only, both, or neither — from the dashboard Optimize tab, applied live (no restart). Each AI client can also get its own memory identity over HTTP MCP via `http://127.0.0.1:8765/mcp/{agent_id}`.
+
 ### Quick Start
 
 ```bash
@@ -60,7 +66,7 @@ slm wrap claude
 |:--------|:-------------|
 | `slm optimize status\|on\|off\|savings` | Master Optimize control + savings report (USD/INR/tokens) |
 | `slm cache status\|clear\|invalidate\|ttl\|semantic` | Cache sub-control — exact + semantic tiers |
-| `slm compress status\|mode\|code\|prose\|ccr\|align` | Compression control — per-channel toggles |
+| `slm compress status\|mode\|prose` | Compression control — safe (lossless) / aggressive (LLMLingua-2 prose) |
 | `slm proxy [--port] [--provider]` | Start the interception proxy (port 8765) |
 | `slm wrap <agent>` | Proxy-activate an agent — one command to start saving |
 | `slm help-optimize [topic]` | Full developer reference + per-agent setup recipes |
@@ -116,6 +122,57 @@ Full docs:
 
 ---
 
+## Three Surfaces: Proxy · MCP Tools · Skill
+
+v3.6.11 delivers one engine across **three ways in** — choose the surface that fits your setup:
+
+| Surface | How you use it | Requires proxy? | Window effect | Cache scope |
+|---------|---------------|:---------------:|:-------------:|-------------|
+| **A — Proxy** | `slm wrap claude` or `ANTHROPIC_BASE_URL=http://127.0.0.1:8765` | **Yes** | Shrinks (proxy intercepts full context) | Full-turn cache — every Claude call |
+| **B — MCP tools** | Add 5 tools to MCP config; call `slm_compress`, `slm_cache_set/get` | **No** | **Preserved** (full 1M) | Results you explicitly route through SLM |
+| **C — Skill** | Copy `skills/slm-optimize/SKILL.md` → `~/.claude/skills/` | **No** | **Preserved** (full 1M) | Auto-applied by the agent per skill rules |
+
+**How to choose:**
+- On a **metered API** (pay-per-token) and want to cache every call → **Proxy (A)**
+- On a **Pro/Max/Team subscription** or any plan where you can't or won't run a proxy → **MCP tools (B)** or **Skill (C)**
+- Want zero configuration → **Skill (C)**: install once, auto-compresses CLAUDE.md and large outputs
+- Want agent-controlled caching of repeated file reads and tool outputs → **MCP tools (B)**
+
+**The hard constraint:** The primary Claude conversation turn cannot be cached without a proxy — the MCP/skill path caches results you explicitly route through SLM (tool outputs, file reads, sub-model calls).
+
+### MCP Tools Setup (Surface B)
+
+Add to your `claude_desktop_config.json` or IDE MCP config alongside your existing SLM entry:
+
+```json
+{
+  "mcpServers": {
+    "superlocalmemory": {
+      "command": "slm",
+      "args": ["mcp"]
+    }
+  }
+}
+```
+
+The 5 optimize tools (`slm_compress`, `slm_retrieve`, `slm_cache_set`, `slm_cache_get`, `slm_optimize_stats`) are included automatically from v3.6.11+. Verify with `slm_optimize_stats()`.
+
+### Skill Setup (Surface C)
+
+```bash
+mkdir -p ~/.claude/skills/slm-optimize
+cp $(pip show superlocalmemory | grep Location | awk '{print $2}')/superlocalmemory/skills/slm-optimize/SKILL.md \
+   ~/.claude/skills/slm-optimize/SKILL.md
+```
+
+Then reference in your `CLAUDE.md`:
+```markdown
+## Context Management
+Use the `slm-optimize` skill to compress large outputs and cache repeated reads.
+```
+
+---
+
 <details>
 <summary><strong>What's New in V3.3 — The Living Brain Evolves</strong> (click to expand)</summary>
 
@@ -292,6 +349,7 @@ No manual commands. No data loss. Zero downtime.
 
 | Version | Codename | Key Features |
 |---|---|---|
+| **v3.6.11** | Optimize Everywhere | **Three surfaces** — Proxy (A: full-turn cache), MCP tools (B: `slm_compress`/`slm_retrieve`/`slm_cache_set`/`slm_cache_get`/`slm_optimize_stats` — proxy-free, 1M window), Skill (C: `slm-optimize` zero-config). `CacheDB.get_value()` (pure KV lookup). 23 new tests. Links: [Three Surfaces →](#three-surfaces-proxy--mcp-tools--skill) · [docs/optimize-overview.md](docs/optimize-overview.md) |
 | **v3.6.0** | Optimize | **Cache** (skip repeat calls, 100% on hit) · **Compress** (shrink prompts 60-95%) · **Align** (KV-cache stabilization) · `slm optimize\|cache\|compress\|proxy\|wrap` CLI · Live savings dashboard (USD/INR/tokens) · Hot-reload config · Safe defaults · Links: [docs/optimize-overview.md](docs/optimize-overview.md) · [V3.6 Wiki](https://github.com/qualixar/superlocalmemory/wiki/V3.6-Overview) |
 | **v3.5.0** | Scale-Ready + Context Injection v2 | CozoDB/LanceDB migration, 6-channel recall <1s, Core Memory Block, BM25→FTS5, context injection v2, score normalization |
 | **v3.4.5** | Scale-Ready (foundation) | Tiered storage (active/warm/cold), graph pruning, BackendOrchestrator scaffolding, CozoDB + LanceDB init + migration code (read path wired in v3.5.0) |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "superlocalmemory",
-  "version": "3.6.9",
+  "version": "3.6.11",
   "description": "Information-geometric agent memory with mathematical guarantees. 4-channel retrieval, Fisher-Rao similarity, zero-LLM mode, EU AI Act compliant. Works with Claude, Cursor, Windsurf, and 17+ AI tools.",
   "keywords": [
     "ai-memory",

package/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "superlocalmemory"
-version = "3.6.9"
+version = "3.6.11"
 description = "Information-geometric agent memory with mathematical guarantees"
 readme = "README.md"
 license = {text = "AGPL-3.0-or-later"}
@@ -69,6 +69,11 @@ dependencies = [
     "scikit-learn==1.8.0",
     # Vector KNN extension for the semantic channel.
     "sqlite-vec==0.1.9",
+    # v3.6.10: LLMLingua-2 prose compression (aggressive mode, opt-in at runtime).
+    # Hard dependency so `pip install superlocalmemory` ships compression-ready;
+    # the ~560MB model downloads on first setup/warmup (fail-open). Verified the
+    # pin does NOT disturb the transformers/torch/numpy pins above.
+    "llmlingua==0.2.2",
 ]
 
 [project.optional-dependencies]

package/skills/slm-optimize/README.md

@@ -0,0 +1,55 @@
+# slm-optimize — Context Optimizer Skill
+
+Automatically compresses large tool outputs and caches repeated reads, reducing
+context window usage without a proxy and without losing the full 1M window.
+
+## Prerequisites
+
+- SuperLocalMemory v3.6.11+
+- `slm mcp` daemon running
+- `slm_compress` visible in `tools/list` (verify with your IDE's MCP tool inspector)
+
+## Install (Claude Code)
+
+```bash
+cp skills/slm-optimize/SKILL.md ~/.claude/skills/slm-optimize/SKILL.md
+```
+
+## Activate
+
+**Option A — On demand:**
+Invoke via the `Skill` tool: `Skill("slm-optimize")`
+
+**Option B — Auto-activate:**
+Add to your project or global `CLAUDE.md`:
+```markdown
+## Context Management
+Use the `slm-optimize` skill to compress large outputs and cache repeated reads.
+Invoke at session start if context > 50k tokens.
+```
+
+## Verify
+
+After activation, run `slm_optimize_stats()` from Claude Code. If it returns `ok:True`,
+all 5 optimize tools are reachable and the skill is working.
+
+## What it does
+
+| Feature | How |
+|---|---|
+| Compresses large tool outputs | `slm_compress` → compressed text + optional `ccr_id` |
+| Caches repeated file reads | `slm_cache_set` / `slm_cache_get` keyed by file path |
+| Caches repeated bash/search | Same KV tools, keyed by command |
+| Recovers exact originals | `slm_retrieve(ccr_id)` when byte-identical content needed |
+| Session stats | `slm_optimize_stats()` |
+
+## What it does NOT do
+
+- **Full-turn caching**: impossible without a proxy (`ANTHROPIC_BASE_URL`).
+  Use Surface A (proxy mode) for that.
+- **Guarantee savings**: results depend on content type and daemon compress config.
+
+## For Cursor, Antigravity, Codex
+
+Same SKILL.md works — all IDEs that support MCP tool calls can use the 5 tools.
+The skill itself is pure Markdown with no IDE-specific code.

package/skills/slm-optimize/SKILL.md

@@ -0,0 +1,139 @@
+---
+name: slm-optimize
+description: >
+  Context optimizer for Claude Code + SLM. Compresses large outputs via
+  slm_compress (reversible CCR), caches repeated reads via slm_cache_set/get,
+  recovers originals via slm_retrieve. Works on any Claude plan, no proxy,
+  full context window preserved.
+  Requires SuperLocalMemory v3.6.11+ with optimize MCP tools enabled.
+version: "3.6.11"
+license: AGPL-3.0-or-later
+triggers:
+  - "slm-optimize"
+  - "compress context"
+  - "optimize context"
+  - "reduce tokens"
+---
+
+# slm-optimize: Context Optimizer for Claude Code
+
+Reduces context window usage through compression and caching.
+No proxy required. Full 1M window preserved.
+
+**What this skill does NOT do:**
+- Cache the primary Claude conversation turn (impossible without a proxy).
+- Guarantee any specific savings percentage — results depend on content type and daemon config.
+
+---
+
+## When This Skill Is Active
+
+Apply these rules automatically throughout the session once this skill is loaded.
+
+---
+
+## Decision Rules
+
+### RULE 1 — Session start: compress CLAUDE.md if large
+
+At the start of any session where this skill is loaded:
+1. Read `CLAUDE.md` (or project CLAUDE.md).
+2. If `len(content) > 3000` characters: call `slm_compress(content=<text>, mode="normalize", reversible=False)`.
+3. Replace the in-context CLAUDE.md block with the compressed version.
+4. **Do not modify the actual file** — compress only the in-context representation.
+
+### RULE 2 — Large tool output: compress before processing
+
+After any tool call that returns > 2000 characters:
+1. Call `slm_compress(content=<output>, mode="auto", reversible=True)`.
+2. Work from `compressed` in the response.
+3. If `lossy=True`, store the returned `ccr_id` — call `slm_retrieve` only if you need the exact original for verification, diffing, or code parsing.
+
+**Never compress** (see RULE 6 for the complete exclusion list).
+
+### RULE 3 — Repeated file reads: KV cache
+
+Before calling `Read(file_path)` for a file you have already read this session:
+1. `slm_cache_get(key=f"file:{file_path}")`.
+2. If `hit=True` → use `value` directly. Skip the `Read` call.
+3. If `hit=False` → call `Read`, then `slm_cache_set(key=f"file:{file_path}", value=<content>, ttl_seconds=1800)`.
+
+If you edit a file via Edit/Write, immediately update or invalidate the cache entry.
+
+### RULE 4 — Repeated bash/search results: KV cache
+
+For any Bash or WebSearch call whose result you expect to reuse:
+1. `slm_cache_set(key=f"bash:{command[:64]}", value=<output>, ttl_seconds=600)`.
+2. On the next identical call: `slm_cache_get` first.
+
+Use `ttl_seconds=3600` for web search results (more expensive to re-run).
+
+### RULE 5 — Retrieve original via slm_retrieve
+
+If you compressed content with `lossy=True` and need the exact original:
+1. Call `slm_retrieve(ccr_id=<ccr_id from prior compress response>)`.
+2. Check `size_bytes` before retrieving if context is nearly full.
+3. Do not retrieve unless you actually need byte-identical content.
+
+### RULE 6 — What NOT to compress or cache
+
+- **Code being sent to Edit/Write**: never compress — the editor needs exact bytes.
+- **JSON or structured data you will parse**: compression can alter whitespace and break parsers.
+- **Secrets, API keys, credentials, tokens**: never store via `slm_cache_set`.
+- **CCR IDs**: never cache via `slm_cache_set` — treat as ephemeral session values.
+- **Very short outputs (< 500 chars)**: compression overhead exceeds the gain.
+
+### RULE 7 — Stats and awareness
+
+If the user asks "how much context have you saved?" or "what is the optimization status?":
+1. Call `slm_optimize_stats()`.
+2. Report `cache_kv_hits`, `compress_runs`, and `tokens_saved_compress` clearly.
+3. Note that proxy stats are daemon-persisted; KV stats are this session only.
+
+### RULE 8 — Fail-open behavior
+
+If any `slm_compress`, `slm_retrieve`, `slm_cache_set`, `slm_cache_get`, or
+`slm_optimize_stats` call returns `ok:False`:
+- Continue with the original (uncompressed/uncached) content.
+- Do not retry. Do not surface the error unless the user asks about optimization.
+- Optimization is always advisory — it must never block any primary task.
+
+---
+
+## Examples
+
+### Compress a large file read
+
+```
+# Instead of working with 8000-char file output directly:
+result = slm_compress(content=<file_content>, mode="auto", reversible=True)
+# Work from result["compressed"]; keep result["ccr_id"] if lossy=True
+```
+
+### Cache a repeated grep result
+
+```
+cached = slm_cache_get(key="bash:grep -rn MyClass src/")
+if not cached["hit"]:
+    output = Bash("grep -rn MyClass src/")
+    slm_cache_set(key="bash:grep -rn MyClass src/", value=output, ttl_seconds=600)
+```
+
+### Recover original when needed
+
+```
+original = slm_retrieve(ccr_id=<ccr_id>)
+# original["content"] is byte-identical to what was compressed
+```
+
+---
+
+## Stats and Troubleshooting
+
+Run `slm_optimize_stats()` to see current session counters.
+
+If tools return `ok:False`, the daemon may be down or the optimize module may not be
+configured. Continue with normal tool calls — this skill degrades gracefully.
+
+Verify tools are available: check that `slm_compress` appears in your MCP `tools/list`.
+Requires SuperLocalMemory v3.6.11+ with `slm mcp` running.

package/src/superlocalmemory/__init__.py

@@ -32,7 +32,7 @@ if "OMP_NUM_THREADS" not in os.environ:
     os.environ["OMP_NUM_THREADS"] = "2"
 # ---------------------------------------------------------------------------
 
-__version__ = "3.6.9"
+__version__ = "3.6.10"
 
 _REQUIRED_VERSIONS = {
     "sentence_transformers": "5.3.0",