superlocalmemory 2.4.2 → 2.5.1

package/CHANGELOG.md

@@ -16,6 +16,68 @@ SuperLocalMemory V2 - Intelligent local memory system for AI coding assistants.
 
 ---
 
+## [2.5.1] - 2026-02-13
+
+**Release Type:** Framework Integration Release — "Plugged Into the Ecosystem"
+**Backward Compatible:** Yes (additive packages only, no core changes)
+
+### The Big Picture
+SuperLocalMemory is now a first-class memory backend for LangChain and LlamaIndex — the two largest AI/LLM frameworks. Two pip-installable packages, zero cloud dependencies.
+
+### Added
+- **LangChain Integration** (`langchain-superlocalmemory`): Persistent chat message history backed by local SQLite. Works with `RunnableWithMessageHistory` and LCEL chains. `pip install langchain-superlocalmemory`
+- **LlamaIndex Integration** (`llama-index-storage-chat-store-superlocalmemory`): Full `BaseChatStore` implementation. Works with `ChatMemoryBuffer` and `SimpleChatEngine`. `pip install llama-index-storage-chat-store-superlocalmemory`
+- **Example scripts** for both frameworks in `examples/` directory — runnable without API keys
+- **Session isolation**: Framework memories are tagged separately and never appear in normal `slm recall`
+
+---
+
+## [2.5.0] - 2026-02-12
+
+**Release Type:** Major Feature Release — "Your AI Memory Has a Heartbeat"
+**Backward Compatible:** Yes (additive tables, columns, and modules only)
+
+### The Big Picture
+SuperLocalMemory transforms from passive storage to **active coordination layer**. Every memory operation now triggers real-time events. Agents are tracked. Trust is scored. The dashboard is live.
+
+### Added
+- **DbConnectionManager** (`src/db_connection_manager.py`): SQLite WAL mode, 5s busy timeout, thread-local read pool, serialized write queue. Fixes "database is locked" errors when multiple agents write simultaneously
+- **Event Bus** (`src/event_bus.py`): Real-time event broadcasting — every memory write/update/delete fires events to SSE, WebSocket, and Webhook subscribers. Tiered retention (hot 48h, warm 14d, cold 30d)
+- **Subscription Manager** (`src/subscription_manager.py`): Durable + ephemeral event subscriptions with filters (event type, importance, protocol)
+- **Webhook Dispatcher** (`src/webhook_dispatcher.py`): Background HTTP POST delivery with exponential backoff retry (3 attempts)
+- **Agent Registry** (`src/agent_registry.py`): Tracks which AI agents connect — protocol, write/recall counters, last seen, metadata
+- **Provenance Tracker** (`src/provenance_tracker.py`): Tracks who created each memory — created_by, source_protocol, trust_score, provenance_chain
+- **Trust Scorer** (`src/trust_scorer.py`): Bayesian trust signal collection — positive (cross-agent recall, high importance), negative (quick delete, burst write). Silent collection in v2.5, enforcement in v2.6
+- **SSE endpoint** (`/events/stream`): Real-time Server-Sent Events with cross-process DB polling, Last-Event-ID replay, event type filtering
+- **Dashboard: Live Events tab**: Real-time event stream with color-coded types, filter dropdown, stats counters
+- **Dashboard: Agents tab**: Connected agents table with protocol badges, trust scores, write/recall counters, trust overview
+- **4 new database tables**: `memory_events`, `subscriptions`, `agent_registry`, `trust_signals`
+- **4 new columns on `memories`**: `created_by`, `source_protocol`, `trust_score`, `provenance_chain`
+- **28 API endpoints** across 8 modular route files
+- **Architecture document**: `docs/ARCHITECTURE-V2.5.md`
+
+### Changed
+- **`memory_store_v2.py`**: Replaced 15 direct `sqlite3.connect()` calls with `DbConnectionManager` (read pool + write queue). Added EventBus, Provenance, Trust integration. Fully backward compatible — falls back to direct SQLite if new modules unavailable
+- **`mcp_server.py`**: Replaced 9 per-call `MemoryStoreV2(DB_PATH)` instantiations with shared singleton. Added agent registration and provenance tracking for MCP calls
+- **`ui_server.py`**: Refactored from 2008-line monolith to 194-line app shell + 9 route modules in `routes/` directory using FastAPI APIRouter
+- **`ui/app.js`**: Split from 1588-line monolith to 13 modular files in `ui/js/` directory (largest: 230 lines)
+
+### Fixed
+- **"Database is locked" errors**: WAL mode + write serialization eliminates concurrent access failures
+- **Fresh database crashes**: `/api/stats`, `/api/graph`, `/api/clusters` now return graceful empty responses when `graph_nodes`/`graph_edges`/`sessions` tables don't exist yet
+- **Per-call overhead**: MCP tool handlers no longer rebuild TF-IDF vectorizer on every call
+
+### Testing
+- 63 pytest tests (unit, concurrency, regression, security, edge cases, Docker/new-user scenarios)
+- 27 end-to-end tests (all v2.5 components + 50 concurrent writes + reads during writes)
+- 15 fresh-database edge case tests
+- 17 backward compatibility tests against live v2.4.2 database
+- All CLI commands and skill scripts verified
+- Profile isolation verified across 2 profiles
+- Playwright dashboard testing (all 8 tabs, SSE stream, profile switching)
+
+---
+
 ## [2.4.2] - 2026-02-11
 
 **Release Type:** Bug Fix Release

package/README.md

@@ -1,8 +1,9 @@
 <p align="center">
-  <img src="https://img.shields.io/badge/🧠_SuperLocalMemory-V2-blueviolet?style=for-the-badge" alt="SuperLocalMemory V2"/>
+  <img src="https://varun369.github.io/SuperLocalMemoryV2/assets/branding/icon-512.png" alt="SuperLocalMemory V2" width="200"/>
 </p>
 
-<h1 align="center">Your AI Finally Remembers You</h1>
+<h1 align="center">SuperLocalMemory V2</h1>
+<p align="center"><strong>Your AI Finally Remembers You</strong></p>
 
 <p align="center">
   <strong>⚡ Created & Architected by <a href="https://github.com/varun369">Varun Pratap Bhardwaj</a> ⚡</strong><br/>
@@ -39,6 +40,63 @@
 
 ---
 
+## NEW: v2.5 — "Your AI Memory Has a Heartbeat"
+
+> **SuperLocalMemory is no longer passive storage — it's a real-time coordination layer.**
+
+| What's New | Why It Matters |
+|------------|----------------|
+| **Real-Time Event Stream** | See every memory operation live in the dashboard — no refresh needed. SSE-powered, cross-process. |
+| **No More "Database Locked"** | WAL mode + serialized write queue. 50 concurrent agents writing? Zero errors. |
+| **Agent Tracking** | Know exactly which AI tool wrote what. Claude, Cursor, Windsurf, CLI — all tracked automatically. |
+| **Trust Scoring** | Bayesian trust signals detect spam, quick-deletes, and cross-agent validation. Silent in v2.5, enforced in v2.6. |
+| **Memory Provenance** | Every memory records who created it, via which protocol, with full derivation lineage. |
+| **Production-Grade Code** | 28 API endpoints across 8 modular route files. 13 modular JS files. 63 pytest tests. |
+
+**Upgrade:** `npm install -g superlocalmemory@latest`
+
+**Dashboard:** `python3 ~/.claude-memory/ui_server.py` then open `http://localhost:8765`
+
+[Interactive Architecture Diagram](https://varun369.github.io/SuperLocalMemoryV2/architecture.html) | [Architecture Doc](docs/ARCHITECTURE-V2.5.md) | [Full Changelog](CHANGELOG.md)
+
+---
+
+## NEW: Framework Integrations (v2.5.1)
+
+Use SuperLocalMemory as a memory backend in your LangChain and LlamaIndex applications — 100% local, zero cloud.
+
+### LangChain
+
+```bash
+pip install langchain-superlocalmemory
+```
+
+```python
+from langchain_superlocalmemory import SuperLocalMemoryChatMessageHistory
+from langchain_core.runnables.history import RunnableWithMessageHistory
+
+history = SuperLocalMemoryChatMessageHistory(session_id="my-session")
+# Messages persist across sessions, stored locally in ~/.claude-memory/memory.db
+```
+
+### LlamaIndex
+
+```bash
+pip install llama-index-storage-chat-store-superlocalmemory
+```
+
+```python
+from llama_index.storage.chat_store.superlocalmemory import SuperLocalMemoryChatStore
+from llama_index.core.memory import ChatMemoryBuffer
+
+chat_store = SuperLocalMemoryChatStore()
+memory = ChatMemoryBuffer.from_defaults(chat_store=chat_store, chat_store_key="user-1")
+```
+
+[LangChain Guide](https://github.com/varun369/SuperLocalMemoryV2/wiki/LangChain-Integration) | [LlamaIndex Guide](https://github.com/varun369/SuperLocalMemoryV2/wiki/LlamaIndex-Integration)
+
+---
+
 ## Install in One Command
 
 ```bash
@@ -424,6 +482,8 @@ Not another simple key-value store. SuperLocalMemory implements **cutting-edge m
 
 ### Multi-Layer Memory Architecture
 
+**[View Interactive Architecture Diagram](https://varun369.github.io/SuperLocalMemoryV2/architecture.html)** — Click any layer for details, research references, and file paths.
+
 ```
 ┌─────────────────────────────────────────────────────────────┐
 │  Layer 9: VISUALIZATION (NEW v2.2.0)                        │

package/docs/ARCHITECTURE-V2.5.md

@@ -0,0 +1,190 @@
+# SuperLocalMemory V2.5 Architecture — "Your AI Memory Has a Heartbeat"
+
+**Version:** 2.5.0 | **Date:** February 12, 2026 | **Author:** Varun Pratap Bhardwaj
+
+---
+
+## What Changed in v2.5
+
+SuperLocalMemory transforms from **passive storage** (filing cabinet) to **active coordination layer** (nervous system). Every memory write, update, delete, or recall now triggers real-time events visible across all connected tools.
+
+### Before v2.5 (Passive)
+```
+Claude writes memory → saved to SQLite → done (silent)
+Cursor reads memory → returned → done (silent)
+```
+
+### After v2.5 (Active)
+```
+Claude writes memory → saved to SQLite → Event Bus fires
+                                           ├── SSE → Dashboard shows it live
+                                           ├── Agent registered in registry
+                                           ├── Trust signal recorded
+                                           ├── Provenance tracked
+                                           └── Webhook dispatched (if configured)
+```
+
+---
+
+## System Architecture
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    ACCESS LAYER                              │
+│  MCP Server │ CLI │ REST API │ Skills │ Python Import │ A2A  │
+└──────────────────────┬──────────────────────────────────────┘
+                       │
+┌──────────────────────┴──────────────────────────────────────┐
+│                MEMORY STORE V2                               │
+│  ┌─────────────────────────────────────────────────────────┐    │
+│  │  DbConnectionManager (Singleton)                     │    │
+│  │  ├── WAL Mode (concurrent reads + serialized writes) │    │
+│  │  ├── Busy Timeout (5s wait, not fail)                │    │
+│  │  ├── Thread-Local Read Pool                          │    │
+│  │  ├── Serialized Write Queue (threading.Queue)        │    │
+│  │  └── Post-Write Hooks → Event Bus                    │    │
+│  └─────────────────────────────────────────────────────┘    │
+│                                                              │
+│  On every write:                                             │
+│  ├── EventBus.emit() → memory_events table + SSE + WS + WH  │
+│  ├── ProvenanceTracker → created_by, source_protocol columns │
+│  ├── AgentRegistry → agent tracking + write/recall counters  │
+│  └── TrustScorer → signal collection (silent, no enforcement)│
+└──────────────────────────────────────────────────────────────┘
+                       │
+┌──────────────────────┴──────────────────────────────────────┐
+│                    STORAGE LAYER                             │
+│  SQLite (single file: ~/.claude-memory/memory.db)            │
+│                                                              │
+│  Tables:                                                     │
+│  ├── memories (+ provenance columns: created_by,             │
+│  │            source_protocol, trust_score, provenance_chain) │
+│  ├── memory_events (event log with tiered retention)         │
+│  ├── subscriptions (durable + ephemeral event subscriptions) │
+│  ├── agent_registry (connected agents + statistics)          │
+│  ├── trust_signals (trust signal audit trail)                │
+│  ├── graph_nodes, graph_edges, graph_clusters (knowledge graph)│
+│  ├── identity_patterns (learned preferences)                 │
+│  ├── sessions, creator_metadata                              │
+│  └── memories_fts (FTS5 full-text search index)              │
+└──────────────────────────────────────────────────────────────┘
+                       │
+┌──────────────────────┴──────────────────────────────────────┐
+│                    DASHBOARD LAYER                            │
+│  FastAPI UI Server (modular routes)                          │
+│  ├── 8 route modules in routes/ directory                    │
+│  ├── 28 API endpoints                                        │
+│  ├── SSE /events/stream (real-time, cross-process)           │
+│  ├── WebSocket /ws/updates                                   │
+│  ├── 13 modular JS files in ui/js/                           │
+│  └── 8 dashboard tabs (Graph, Memories, Clusters, Patterns,  │
+│       Timeline, Live Events, Agents, Settings)               │
+└──────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## New Components (v2.5)
+
+| Component | File | Purpose |
+|-----------|------|---------|
+| **DbConnectionManager** | `src/db_connection_manager.py` | WAL mode, busy timeout, read pool, write queue, post-write hooks |
+| **EventBus** | `src/event_bus.py` | Event emission, persistence, in-memory buffer, tiered retention |
+| **SubscriptionManager** | `src/subscription_manager.py` | Durable + ephemeral event subscriptions with filters |
+| **WebhookDispatcher** | `src/webhook_dispatcher.py` | Background HTTP POST delivery with retry logic |
+| **AgentRegistry** | `src/agent_registry.py` | Agent tracking, write/recall counters, protocol detection |
+| **ProvenanceTracker** | `src/provenance_tracker.py` | Memory origin tracking (who, how, trust, lineage) |
+| **TrustScorer** | `src/trust_scorer.py` | Bayesian trust scoring with decay, burst detection |
+
+## New Database Tables (v2.5)
+
+| Table | Columns | Purpose |
+|-------|---------|---------|
+| `memory_events` | id, event_type, memory_id, source_agent, source_protocol, payload, importance, tier, created_at | Event log with tiered retention |
+| `subscriptions` | id, subscriber_id, channel, filter, webhook_url, durable, last_event_id | Event subscriptions |
+| `agent_registry` | id, agent_id, agent_name, protocol, first_seen, last_seen, memories_written, memories_recalled, trust_score, metadata | Agent tracking |
+| `trust_signals` | id, agent_id, signal_type, delta, old_score, new_score, context, created_at | Trust audit trail |
+
+## New Columns on `memories` Table (v2.5)
+
+| Column | Type | Default | Purpose |
+|--------|------|---------|---------|
+| `created_by` | TEXT | 'user' | Agent ID that created this memory |
+| `source_protocol` | TEXT | 'cli' | Protocol used (mcp, cli, rest, python, a2a) |
+| `trust_score` | REAL | 1.0 | Trust score at creation time |
+| `provenance_chain` | TEXT | '[]' | JSON derivation lineage |
+
+---
+
+## Event Types
+
+| Event | Trigger |
+|-------|---------|
+| `memory.created` | New memory written |
+| `memory.updated` | Existing memory modified |
+| `memory.deleted` | Memory removed |
+| `memory.recalled` | Memory retrieved by an agent |
+| `graph.updated` | Knowledge graph rebuilt |
+| `pattern.learned` | New pattern detected |
+| `agent.connected` | New agent connects |
+| `agent.disconnected` | Agent disconnects |
+
+## Event Retention Tiers
+
+| Tier | Window | Kept |
+|------|--------|------|
+| Hot | 0-48h | All events |
+| Warm | 2-14d | Importance >= 5 only |
+| Cold | 14-30d | Daily aggregates |
+| Archive | 30d+ | Pruned (stats in pattern_learner) |
+
+---
+
+## Trust Scoring (Silent Collection — v2.5)
+
+All agents start at trust 1.0. Signals adjust the score with a Bayesian decay factor that stabilizes over time.
+
+| Signal | Delta | When |
+|--------|-------|------|
+| `high_importance_write` | +0.015 | Memory with importance >= 7 |
+| `memory_recalled_by_others` | +0.02 | Another agent finds this memory useful |
+| `quick_delete` | -0.03 | Memory deleted within 1 hour |
+| `high_volume_burst` | -0.02 | >20 writes in 5 minutes |
+
+**v2.5:** Silent collection only. **v2.6:** Trust-weighted recall ranking. **v3.0:** Active enforcement.
+
+---
+
+## API Endpoints (28 total)
+
+| Route | Method | Module | Purpose |
+|-------|--------|--------|---------|
+| `/` | GET | ui_server.py | Dashboard |
+| `/health` | GET | ui_server.py | Health check |
+| `/api/memories` | GET | routes/memories.py | List memories |
+| `/api/graph` | GET | routes/memories.py | Knowledge graph data |
+| `/api/search` | POST | routes/memories.py | Semantic search |
+| `/api/clusters` | GET | routes/memories.py | Cluster info |
+| `/api/clusters/{id}` | GET | routes/memories.py | Cluster detail |
+| `/api/stats` | GET | routes/stats.py | System statistics |
+| `/api/timeline` | GET | routes/stats.py | Timeline aggregation |
+| `/api/patterns` | GET | routes/stats.py | Learned patterns |
+| `/api/profiles` | GET | routes/profiles.py | List profiles |
+| `/api/profiles/{n}/switch` | POST | routes/profiles.py | Switch profile |
+| `/api/profiles/create` | POST | routes/profiles.py | Create profile |
+| `/api/profiles/{n}` | DELETE | routes/profiles.py | Delete profile |
+| `/api/export` | GET | routes/data_io.py | Export memories |
+| `/api/import` | POST | routes/data_io.py | Import memories |
+| `/api/backup/*` | GET/POST | routes/backup.py | Backup management |
+| `/events/stream` | GET | routes/events.py | SSE real-time stream |
+| `/api/events` | GET | routes/events.py | Event polling |
+| `/api/events/stats` | GET | routes/events.py | Event statistics |
+| `/api/agents` | GET | routes/agents.py | Agent list |
+| `/api/agents/stats` | GET | routes/agents.py | Agent statistics |
+| `/api/trust/stats` | GET | routes/agents.py | Trust overview |
+| `/api/trust/signals/{id}` | GET | routes/agents.py | Agent trust signals |
+| `/ws/updates` | WS | routes/ws.py | WebSocket |
+
+---
+
+*SuperLocalMemory V2.5 — Created by Varun Pratap Bhardwaj. MIT License.*