superlocalmemory 2.8.5 → 3.0.0

package/docs/api-reference.md

@@ -0,0 +1,284 @@
+# API Reference
+> SuperLocalMemory V3 Documentation
+> https://superlocalmemory.com | Part of Qualixar
+
+Use SuperLocalMemory programmatically from Python. This is for developers building integrations, custom workflows, or framework adapters.
+
+---
+
+## Installation
+
+```bash
+pip install superlocalmemory
+```
+
+Or, if you installed via npm, the Python package is available at:
+
+```python
+import sys
+sys.path.insert(0, "/path/to/superlocalmemory/python")
+```
+
+## Quick Start
+
+```python
+from superlocalmemory.core.engine import MemoryEngine
+from superlocalmemory.core.config import SLMConfig
+
+# Initialize with default config (Mode A, default profile)
+config = SLMConfig()
+engine = MemoryEngine(config)
+
+# Store a memory
+engine.store("The API uses OAuth 2.0 with PKCE for mobile clients")
+
+# Recall memories
+results = engine.recall("authentication method for mobile")
+for result in results:
+    print(f"[{result.score:.2f}] {result.content}")
+
+# Clean up
+engine.close()
+```
+
+## Configuration
+
+### SLMConfig
+
+```python
+from superlocalmemory.core.config import SLMConfig
+
+# Default configuration
+config = SLMConfig()
+
+# Custom configuration
+config = SLMConfig(
+    mode="a",                      # "a", "b", or "c"
+    profile="work",                # Active profile name
+    data_dir="~/.superlocalmemory",  # Database location
+    embedding_model="all-MiniLM-L6-v2",
+    max_results=10,
+)
+```
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `mode` | str | `"a"` | Operating mode |
+| `profile` | str | `"default"` | Active profile |
+| `data_dir` | str | `"~/.superlocalmemory"` | Data directory |
+| `embedding_model` | str | `"all-MiniLM-L6-v2"` | Sentence transformer model |
+| `max_results` | int | `10` | Default max results for recall |
+
+## Core Operations
+
+### Store
+
+```python
+# Basic store
+memory_id = engine.store("React 19 uses the new compiler by default")
+
+# Store with tags
+memory_id = engine.store(
+    "Deploy to staging before 2pm on Fridays",
+    tags=["process", "deployment"]
+)
+
+# Store with metadata
+memory_id = engine.store(
+    "Maria approved the new database schema",
+    metadata={"source": "slack", "channel": "#backend"}
+)
+```
+
+**Returns:** `int` — the ID of the stored memory.
+
+### Recall
+
+```python
+# Basic recall
+results = engine.recall("database schema approval")
+
+# With limit
+results = engine.recall("deployment schedule", limit=5)
+```
+
+**Returns:** `list[RecallResult]`
+
+Each `RecallResult` has:
+
+| Attribute | Type | Description |
+|-----------|------|-------------|
+| `id` | int | Memory ID |
+| `content` | str | Memory text |
+| `score` | float | Relevance score (0.0 to 1.0) |
+| `timestamp` | datetime | When the memory was stored |
+| `tags` | list[str] | Tags attached to the memory |
+| `metadata` | dict | Additional metadata |
+
+### Recall with Trace
+
+```python
+# Get channel-by-channel breakdown
+results = engine.recall_trace("who approved the schema")
+
+for result in results:
+    print(f"[{result.score:.2f}] {result.content}")
+    print(f"  Semantic: {result.trace.semantic:.2f}")
+    print(f"  BM25:     {result.trace.bm25:.2f}")
+    print(f"  Entity:   {result.trace.entity:.2f}")
+    print(f"  Temporal:  {result.trace.temporal:.2f}")
+```
+
+### Delete
+
+```python
+# Delete by ID
+engine.delete(memory_id=42)
+
+# Delete by query (deletes best match)
+engine.delete(query="old staging credentials")
+```
+
+### List Recent
+
+```python
+memories = engine.list_recent(limit=20)
+for m in memories:
+    print(f"[{m.id}] {m.content[:80]}...")
+```
+
+## Profile Management
+
+```python
+# List profiles
+profiles = engine.list_profiles()
+
+# Switch profile
+engine.switch_profile("client-acme")
+
+# Create profile
+engine.create_profile("new-project")
+
+# Delete profile
+engine.delete_profile("old-project")
+```
+
+## Health and Diagnostics
+
+```python
+# System status
+status = engine.status()
+print(f"Mode: {status.mode}")
+print(f"Profile: {status.profile}")
+print(f"Memories: {status.memory_count}")
+
+# Health check
+health = engine.health()
+print(f"Fisher-Rao: {health.fisher}")
+print(f"Sheaf: {health.sheaf}")
+print(f"Langevin: {health.langevin}")
+
+# Consistency check
+contradictions = engine.consistency_check()
+for c in contradictions:
+    print(f"Conflict: '{c.memory_a}' vs '{c.memory_b}'")
+```
+
+## Framework Integrations
+
+### LangChain
+
+```python
+from superlocalmemory.integrations.langchain import SLMMemory
+
+memory = SLMMemory(mode="a", profile="langchain-app")
+
+# Use as LangChain memory
+from langchain.chains import ConversationChain
+from langchain.llms import OpenAI
+
+chain = ConversationChain(
+    llm=OpenAI(),
+    memory=memory,
+)
+```
+
+### LlamaIndex
+
+```python
+from superlocalmemory.integrations.llamaindex import SLMMemoryStore
+
+store = SLMMemoryStore(mode="a", profile="llamaindex-app")
+
+# Use as a LlamaIndex document store
+```
+
+### Direct MCP Integration
+
+If you are building an MCP-compatible tool and want to add SLM as a backend:
+
+```python
+from superlocalmemory.mcp import create_mcp_server
+
+server = create_mcp_server(
+    mode="a",
+    profile="my-tool",
+)
+
+# The server exposes all 24 tools and 6 resources
+server.run()
+```
+
+## Context Manager
+
+For scripts that need clean resource management:
+
+```python
+from superlocalmemory.core.engine import MemoryEngine
+from superlocalmemory.core.config import SLMConfig
+
+with MemoryEngine(SLMConfig()) as engine:
+    engine.store("Ephemeral context for this script run")
+    results = engine.recall("context")
+    # engine.close() called automatically
+```
+
+## Error Handling
+
+```python
+from superlocalmemory.core.exceptions import (
+    SLMError,            # Base exception
+    ProfileNotFoundError,
+    MemoryNotFoundError,
+    ConfigError,
+    DatabaseError,
+)
+
+try:
+    engine.switch_profile("nonexistent")
+except ProfileNotFoundError:
+    print("Profile does not exist. Create it first.")
+
+try:
+    engine.delete(memory_id=99999)
+except MemoryNotFoundError:
+    print("Memory not found.")
+```
+
+## Thread Safety
+
+The `MemoryEngine` uses SQLite WAL mode and is safe for concurrent reads from multiple threads. Writes are serialized at the database level.
+
+For multi-threaded applications, create one engine instance and share it:
+
+```python
+# Safe: one engine, multiple threads reading
+engine = MemoryEngine(SLMConfig())
+
+# Each thread can call engine.recall() concurrently
+# Writes (engine.store()) are automatically serialized
+```
+
+---
+
+*SuperLocalMemory V3 — Copyright 2026 Varun Pratap Bhardwaj. MIT License. Part of Qualixar.*

package/docs/architecture.md

@@ -0,0 +1,149 @@
+# Architecture
+> SuperLocalMemory V3 Documentation
+> https://superlocalmemory.com | Part of Qualixar
+
+A high-level overview of how SuperLocalMemory V3 stores, organizes, and retrieves your memories.
+
+---
+
+## Design Principles
+
+1. **Local-first.** Your data stays on your machine by default. Cloud is opt-in.
+2. **Zero-configuration.** Install and forget. Memory capture and recall happen automatically.
+3. **Multi-channel retrieval.** No single search method is best for every query. V3 combines four channels and picks the best results.
+4. **Mathematically grounded.** Retrieval quality, consistency, and lifecycle management are backed by information geometry rather than heuristics.
+
+## System Overview
+
+```
+Your IDE (Claude, Cursor, VS Code, ...)
+       |
+       | MCP Protocol
+       v
++------------------+
+| MCP Server       |  24 tools, 6 resources
++------------------+
+       |
+       v
++------------------+
+| Memory Engine    |  Ingestion + Retrieval + Lifecycle
++------------------+
+       |
+       v
++------------------+
+| SQLite Database  |  ~/.superlocalmemory/memory.db
++------------------+
+```
+
+## How Memories Are Stored (Ingestion)
+
+When you store a memory (or one is auto-captured), it passes through an 11-step pipeline:
+
+| Step | What it does |
+|------|-------------|
+| 1. Metadata extraction | Timestamps, session context, source identification |
+| 2. Entity extraction | People, projects, technologies, organizations mentioned |
+| 3. Fact extraction | Discrete facts pulled from the text |
+| 4. Emotion detection | Sentiment and emotional tone (frustration, confidence, etc.) |
+| 5. Belief extraction | Opinions, preferences, and stated beliefs |
+| 6. Entity resolution | Links mentions to canonical entities (e.g., "React" and "ReactJS" become one) |
+| 7. Graph wiring | Connects entities and facts into a relationship graph |
+| 8. Foresight tagging | Predicts what future queries this memory might answer |
+| 9. Observation building | Creates structured observations for pattern learning |
+| 10. Entropy gating | Filters out redundant or low-value information |
+| 11. Storage | Writes to all indexes: semantic vectors, BM25 tokens, graph edges, temporal events |
+
+In Mode A, all steps run locally without any LLM calls. In Modes B and C, the LLM enhances entity extraction and fact decomposition.
+
+## How Memories Are Retrieved (Recall)
+
+Every recall query runs through four independent retrieval channels, then fuses the results:
+
+### The Four Channels
+
+| Channel | How it works | Best for |
+|---------|-------------|----------|
+| **Semantic** | Vector similarity using sentence embeddings, enhanced by Fisher-Rao geometry | "Queries that mean the same thing but use different words" |
+| **BM25** | Classic keyword matching with term frequency scoring | "Queries with specific names, codes, or exact terms" |
+| **Entity Graph** | Traverses the relationship graph via spreading activation | "Who works with Maria?" or "What connects service A to service B?" |
+| **Temporal** | Matches based on time references and event ordering | "What did we decide last Friday?" or "Changes since the sprint started" |
+
+### Fusion and Ranking
+
+1. Each channel returns its top candidates with scores
+2. **Reciprocal Rank Fusion** (RRF) combines the four ranked lists into one
+3. In Mode C, a **cross-encoder** reranks the top results for final precision
+4. The top results are returned to your AI assistant
+
+This multi-channel approach means V3 finds memories that any single search method would miss. A keyword search alone might miss paraphrased content. A vector search alone might miss exact names. The combination catches both.
+
+## Three Operating Modes
+
+| Mode | Retrieval | LLM Usage | Data Location |
+|------|-----------|-----------|---------------|
+| **A: Zero-Cloud** | 4-channel + math scoring | None | 100% local |
+| **B: Local LLM** | 4-channel + local LLM reranking | Ollama (local) | 100% local |
+| **C: Cloud LLM** | 4-channel + cross-encoder + agentic retrieval | Cloud provider | Queries sent to cloud |
+
+Mode A is the default. It delivers strong recall quality using mathematical scoring without any network calls.
+
+## Mathematical Foundations
+
+V3 uses three mathematical layers. These are not academic additions — they solve specific practical problems.
+
+### Fisher-Rao Similarity
+
+**Problem:** Standard vector similarity treats all memories equally, regardless of how much evidence backs them.
+
+**Solution:** Fisher-Rao geometry accounts for the statistical confidence of each memory's embedding. A memory accessed and confirmed many times gets a tighter confidence region. A memory stored once and never validated has wider uncertainty. Similarity scoring respects this difference.
+
+**Effect:** Frequently validated memories rank higher. Uncertain memories are flagged for verification.
+
+### Sheaf Consistency
+
+**Problem:** Over time, you store contradictory memories. "We use PostgreSQL" and later "We migrated to MySQL." Simple retrieval returns both without flagging the conflict.
+
+**Solution:** Sheaf cohomology detects when memories attached to the same entity or topic contradict each other. When a contradiction is found, the system marks the older memory as superseded and surfaces the newer one.
+
+**Effect:** Recall returns consistent information. Contradictions are flagged for your review.
+
+### Langevin Lifecycle
+
+**Problem:** Memory databases grow endlessly. Old memories dilute retrieval quality.
+
+**Solution:** Langevin dynamics models each memory's lifecycle — from Active (frequently accessed) through Warm, Cold, and eventually Archived. The transition is not based on simple time rules but on a self-organizing dynamic that balances recency, access frequency, and information value.
+
+**Effect:** Active memories stay prominent. Stale memories fade gracefully. Storage stays efficient.
+
+## EU AI Act Compliance
+
+Mode A satisfies data sovereignty requirements under the EU AI Act by design:
+
+- **No cloud dependency.** All memory operations run locally. No data leaves your machine.
+- **Right to erasure.** `slm forget` deletes data locally. No cloud logs to purge.
+- **Transparency.** The retrieval pipeline is auditable. No black-box LLM decisions in Mode A.
+- **Risk classification.** A local retrieval system with no AI decision-making qualifies as minimal risk.
+
+Mode C sends queries to a cloud LLM provider. In that mode, the cloud provider's compliance posture applies to those queries.
+
+## Database
+
+All data is stored in a single SQLite database:
+
+```
+~/.superlocalmemory/memory.db
+```
+
+The database uses WAL (Write-Ahead Logging) mode for safe concurrent access from multiple IDE connections.
+
+Key table groups:
+
+- **Core:** memories, sessions, profiles
+- **Knowledge:** semantic_facts, kg_nodes, memory_edges, canonical_entities
+- **Retrieval indexes:** bm25_tokens, memory_metadata, temporal_events
+- **Math layers:** fisher_state, sheaf_sections, langevin_state
+- **Compliance:** trust_scores, provenance, audit_trail, retention_policies
+
+---
+
+*SuperLocalMemory V3 — Copyright 2026 Varun Pratap Bhardwaj. MIT License. Part of Qualixar.*

package/docs/auto-memory.md

@@ -0,0 +1,150 @@
+# Auto-Memory
+> SuperLocalMemory V3 Documentation
+> https://superlocalmemory.com | Part of Qualixar
+
+SuperLocalMemory captures and recalls context automatically. Install it once, then forget about it — your AI assistant gets smarter over time without any manual effort.
+
+---
+
+## How Auto-Capture Works
+
+When you work with an AI assistant that has SLM connected, certain types of information are automatically stored as memories:
+
+| What gets captured | Example |
+|-------------------|---------|
+| **Decisions** | "Let's use WebSocket instead of SSE" |
+| **Bug fixes** | "The crash was caused by a null pointer in the auth middleware" |
+| **Architecture choices** | "We're going with a microservices approach for the payment system" |
+| **Preferences** | "I prefer functional components over class components" |
+| **Project context** | "The staging server is at 10.0.1.50, port 8080" |
+| **People and roles** | "Sarah is the lead on the mobile team" |
+| **Corrections** | "Actually, the deadline is March 20, not March 15" |
+
+### What does NOT get captured
+
+- Raw code blocks (too noisy, changes too fast)
+- Casual conversation ("thanks", "sounds good")
+- Repeated information already in memory
+- Content filtered by the entropy gate (redundant or low-value)
+
+### How the system decides what to capture
+
+The ingestion pipeline scores each candidate memory on:
+
+1. **Information value** — Does this add new knowledge not already stored?
+2. **Specificity** — Is this a concrete fact or a vague statement?
+3. **Reusability** — Is this likely to be useful in a future session?
+
+Memories that score below the threshold are discarded. This keeps your database focused and retrieval quality high.
+
+## How Auto-Recall Works
+
+Before your AI assistant responds to a question, SLM automatically searches for relevant memories and injects them as context.
+
+### The flow
+
+```
+You ask a question
+    |
+    v
+SLM runs a recall query using your question as the search input
+    |
+    v
+Relevant memories are injected into the AI's context window
+    |
+    v
+The AI responds with awareness of your past decisions, preferences, and project context
+```
+
+### What this looks like in practice
+
+**Without SLM:**
+> You: "What database should I use for the new service?"
+> AI: Generic advice about PostgreSQL vs MySQL vs MongoDB...
+
+**With SLM:**
+> You: "What database should I use for the new service?"
+> AI: "Based on your previous decision to standardize on PostgreSQL 16 (stored March 5), and your preference for managed services on AWS (stored February 20), I'd recommend Amazon RDS for PostgreSQL. Your auth service and payment service already use PostgreSQL, so this keeps the stack consistent."
+
+The AI did not "remember" this on its own. SLM injected the relevant memories before the AI generated its response.
+
+## Configuration
+
+### Toggle auto-capture and auto-recall
+
+In `~/.superlocalmemory/config.json`:
+
+```json
+{
+  "auto_capture": true,
+  "auto_recall": true
+}
+```
+
+Set either to `false` to disable. When disabled, you can still use `slm remember` and `slm recall` manually.
+
+### Adjust recall sensitivity
+
+```json
+{
+  "max_recall_results": 10,
+  "recall_threshold": 0.3
+}
+```
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| `max_recall_results` | `10` | Maximum memories injected per query |
+| `recall_threshold` | `0.3` | Minimum relevance score (0.0 to 1.0). Lower = more memories, possibly less relevant. Higher = fewer but more precise. |
+
+### Adjust capture sensitivity
+
+```json
+{
+  "capture_threshold": 0.5
+}
+```
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| `capture_threshold` | `0.5` | Minimum information value to auto-capture. Lower = capture more. Higher = capture only high-value statements. |
+
+## Manual Override
+
+You always have full control:
+
+```bash
+# Explicitly store something
+slm remember "The API rate limit on production is 500 req/min, staging is 100 req/min"
+
+# Explicitly recall
+slm recall "rate limits"
+
+# Delete a memory
+slm forget --id 42
+```
+
+Manual operations work regardless of auto-capture/auto-recall settings.
+
+## Learning Over Time
+
+SLM's adaptive learning system observes which memories are recalled frequently, which are marked helpful or outdated, and adjusts its behavior:
+
+- **Frequently helpful memories** get higher ranking in future recalls
+- **Memories marked "outdated"** are deprioritized or flagged for review
+- **Usage patterns** inform what types of information to prioritize for capture
+
+You can see what the system has learned:
+
+```bash
+slm patterns            # Show learned patterns
+slm patterns correct 5  # Correct pattern #5 if it's wrong
+```
+
+## Privacy
+
+Auto-capture and auto-recall happen entirely within SLM on your machine. In Mode A, no data leaves your device at any point. In Mode C, recall queries are sent to your configured LLM provider, but the memories themselves remain local.
+
+---
+
+*SuperLocalMemory V3 — Copyright 2026 Varun Pratap Bhardwaj. MIT License. Part of Qualixar.*