superlocalmemory 3.4.8 → 3.4.10

package/README.md

@@ -378,6 +378,15 @@ Auto-capture hooks: `slm hooks install` + `slm observe` + `slm session-context`.
 - Auto-learned soft prompts injected into agent context
 - Behavioral pattern detection and outcome tracking
 
+### Skill Evolution (NEW in v3.4.10)
+- **Per-skill performance tracking** — automatically tracks which skills succeed and which fail, across sessions
+- **Execution trace analysis** — mines tool usage patterns around skill invocations to determine effectiveness
+- **Skill entities in Entity Explorer** — each skill becomes a browsable entity with performance facts and evolution history
+- **Dedicated Skill Evolution dashboard tab** — overview cards, performance assertions, skill correlations
+- **Behavioral assertions for skill routing** — soft prompts recommend high-performing skills in future sessions
+- **ECC integration** — enhanced observation support with [Everything Claude Code](https://github.com/affaan-m/everything-claude-code) via `slm ingest --source ecc`
+- **IDE compatibility:** Skill tracking currently works with Claude Code. The `/api/v3/tool-event` endpoint accepts events from any IDE client — adapters for other IDEs in future releases.
+
 ### Trust & Security
 - Bayesian Beta-distribution trust scoring (per-agent, per-fact)
 - Trust gates (block low-trust agents from writing/deleting)
@@ -508,6 +517,11 @@ Copyright (c) 2026 Varun Pratap Bhardwaj / Qualixar.
 
 Part of [Qualixar](https://qualixar.com) · Author: [Varun Pratap Bhardwaj](https://varunpratap.com)
 
+### Acknowledgments
+
+- **[Everything Claude Code (ECC)](https://github.com/affaan-m/everything-claude-code)** — SLM's skill observation patterns were inspired by ECC's continuous learning architecture. SLM supports direct ingestion of ECC observations via `slm ingest --source ecc`, giving ECC users richer skill performance tracking. We recommend ECC for Claude Code users who want the deepest learning experience alongside SLM.
+- **[HKUDS/OpenSpace](https://github.com/HKUDS/OpenSpace)** — The skill evolution research in SLM draws from the EvoSkills co-evolutionary verification concepts (arXiv:2604.01687). We adopted their 3-trigger evolution system and anti-loop guard patterns.
+
 ---
 
 <p align="center">

package/docs/cloud-backup.md

@@ -0,0 +1,174 @@
+# Cloud Backup — Google Drive & GitHub
+
+SuperLocalMemory v3.4.10+ can automatically back up your memory databases to **Google Drive** and **GitHub**. All credentials are stored in your OS keychain (macOS Keychain, Windows Credential Locker, or Linux Secret Service) — never in plaintext.
+
+## GitHub Backup (Recommended)
+
+GitHub backup works out of the box. No additional setup needed beyond a Personal Access Token.
+
+### Setup (2 minutes)
+
+1. Open the SLM Dashboard: `http://localhost:8765`
+2. Click the **account widget** in the sidebar (bottom), then click the **GitHub icon**
+3. You'll see the "Connect GitHub" form:
+
+   - **Personal Access Token**: Click [Create one here](https://github.com/settings/tokens/new?scopes=repo&description=SLM+Backup) — this opens GitHub with the `repo` scope pre-selected. Click "Generate token" and copy it.
+   - **Repository Name**: Default is `slm-backup`. Change if you want.
+
+4. Click **Connect**
+
+That's it. SLM will:
+- Verify your token
+- Create a **private** repository (always private — your data is never public)
+- Initialize it with a README
+- Show your GitHub avatar and username in the sidebar
+
+### How It Works
+
+- Each backup creates a **GitHub Release** with your database files as assets
+- ALL databases are included: memory.db, learning.db, audit_chain.db, code_graph.db, pending.db
+- Only the last **5 releases** are kept — older ones are automatically deleted to prevent storage bloat
+- Backups run in the background — the dashboard never freezes
+
+### Restoring from GitHub
+
+1. Go to your `slm-backup` repo on GitHub
+2. Click **Releases** in the sidebar
+3. Download the `.db` files from the latest release
+4. Copy them to `~/.superlocalmemory/`
+5. Run `slm restart`
+
+---
+
+## Google Drive Backup
+
+Google Drive backup requires a one-time OAuth client setup through Google Cloud Console. This is a Google requirement for any application that accesses Drive on behalf of users.
+
+### Why Is This Needed?
+
+Google requires every application to register an "OAuth client" before it can access your Drive. This is a security measure — it ensures you know exactly which application is accessing your data. For GitHub, a simple Personal Access Token is enough, but Google's security model is stricter.
+
+### Setup (5 minutes)
+
+#### Step 1: Create a Google Cloud Project
+
+1. Go to [Google Cloud Console](https://console.cloud.google.com/)
+2. Click the project dropdown (top bar) → **New Project**
+3. Name it anything (e.g., `slm-backup`) → **Create**
+4. Select the new project from the dropdown
+
+#### Step 2: Enable APIs
+
+1. Go to **APIs & Services** → **Library**
+2. Search for and enable:
+   - **Google Drive API**
+   - **People API** (for showing your email/name)
+
+#### Step 3: Configure OAuth Consent Screen
+
+1. Go to **APIs & Services** → **OAuth consent screen**
+2. Select **External** → **Create**
+3. Fill in:
+   - **App name**: `SuperLocalMemory` (or anything)
+   - **User support email**: your Gmail
+   - **Developer contact email**: your Gmail
+4. Click **Save and Continue** through the remaining steps
+5. Go to **Test users** → **Add users** → add your Gmail address
+
+#### Step 4: Create OAuth Client
+
+1. Go to **APIs & Services** → **Credentials**
+2. Click **Create Credentials** → **OAuth client ID**
+3. Application type: **Web application**
+4. Name: `SLM Dashboard` (or anything)
+5. Under **Authorized redirect URIs**, add:
+   ```
+   http://localhost:8765/api/backup/oauth/google/callback
+   ```
+6. Click **Create**
+7. Copy the **Client ID** and **Client Secret** (you'll need both)
+
+#### Step 5: Connect in SLM
+
+1. Open the SLM Dashboard: `http://localhost:8765`
+2. Click the **Google icon** in the sidebar account widget
+3. Paste your **Client ID** and **Client Secret**
+4. Click **Save & Connect Google Drive**
+5. Google's login page opens — sign in and click **Allow**
+6. You'll see "Google Drive Connected!" — close the popup
+
+### How It Works
+
+- Backups are uploaded to a `SLM-Backup` folder in your Google Drive
+- Files are **replaced in-place** (no duplicates, no storage bloat)
+- ALL databases are backed up, not just memory.db
+- Your OAuth credentials are stored in your OS keychain
+- Backups run in the background
+
+### Restoring from Google Drive
+
+1. Open Google Drive → `SLM-Backup` folder
+2. Download all `.db` files
+3. Copy them to `~/.superlocalmemory/`
+4. Run `slm restart`
+
+---
+
+## Sync & Schedule
+
+### Manual Sync
+
+Click **Sync Now** (cloud upload icon) in the sidebar account widget, or go to **Settings** → **Cloud Backup** → **Sync Now**.
+
+### Auto-Backup
+
+SLM automatically creates local backups on a schedule (default: weekly). When cloud destinations are connected, backups are also pushed to the cloud after each auto-backup.
+
+Configure the schedule in **Settings** → **Backup Configuration**:
+- **Interval**: Daily or Weekly
+- **Max backups**: How many local backups to keep (default: 10)
+
+### Export
+
+Click the **download icon** in the sidebar to export a compressed `.gz` backup file you can store anywhere.
+
+---
+
+## What Gets Backed Up
+
+| Database | Contents | Typical Size |
+|---|---|---|
+| `memory.db` | Facts, entities, graph edges, embeddings, sessions | 50 MB — 2 GB |
+| `learning.db` | Learning signals, behavioral patterns, ranker data | 0.5 — 5 MB |
+| `audit_chain.db` | Audit trail, compliance provenance | 0.5 — 2 MB |
+| `code_graph.db` | Code knowledge graph (if used) | 0.1 — 10 MB |
+| `pending.db` | Pending operations queue | 0.1 — 1 MB |
+
+All databases are backed up using SQLite's `sqlite3.backup()` API, which creates a consistent, atomic snapshot even while the daemon is running.
+
+---
+
+## Security
+
+- **GitHub repos are always private** — hardcoded, cannot be changed
+- **Credentials stored in OS keychain** — macOS Keychain, Windows Credential Locker, or Linux Secret Service
+- **Fallback**: On systems without a keychain (headless Linux), credentials are stored in `~/.superlocalmemory/.credentials.json` with `chmod 0600` (owner-only)
+- **Google OAuth tokens** are refresh tokens — they can be revoked from your [Google Account Security page](https://myaccount.google.com/permissions)
+- **GitHub PATs** can be revoked from [GitHub Settings → Tokens](https://github.com/settings/tokens)
+
+---
+
+## Troubleshooting
+
+### "Sync failed" in the sidebar
+Check the destination status in **Settings** → **Cloud Backup**. Common causes:
+- GitHub: PAT expired or revoked → reconnect with a new token
+- Google: OAuth token expired → click "Connect Google Drive" again to re-authorize
+
+### Google Drive shows "Connection Failed"
+- Make sure you added yourself as a **test user** in the OAuth consent screen
+- Verify the redirect URI matches exactly: `http://localhost:8765/api/backup/oauth/google/callback`
+- Check that the SLM daemon is running on port 8765
+
+### Dashboard freezes during sync
+This was fixed in v3.4.10 — syncs now run in a background thread. If you're on an older version, update: `pip install -U superlocalmemory`

package/docs/skill-evolution.md

@@ -0,0 +1,189 @@
+# Skill Evolution — SuperLocalMemory
+
+> Track, analyze, and evolve your AI agent skills automatically.
+
+---
+
+## What Is Skill Evolution?
+
+Skill Evolution turns SuperLocalMemory from a passive memory system into an active learning engine that tracks how your skills perform and helps them improve over time.
+
+**The problem:** AI agent skills (SKILL.md files, slash commands, agent definitions) are static. A skill installed today works the same way 6 months from now — even if it failed 50 times, even if a better approach was discovered.
+
+**The solution:** SLM observes every skill invocation, builds execution traces, computes performance metrics, and surfaces insights so you (and eventually the system itself) can evolve skills based on real data.
+
+---
+
+## How It Works
+
+```
+Your session
+  │
+  │ SLM hook captures every tool call (enriched: input, output, session, project)
+  ▼
+tool_events table (rich execution data)
+  │
+  │ SkillPerformanceMiner runs during consolidation
+  ▼
+Per-skill metrics + behavioral assertions + skill entities
+  │
+  │ Next session's soft prompts include skill routing recommendations
+  ▼
+Smarter skill selection, session by session
+```
+
+### Three Data Sources
+
+| Source | What It Captures | Cost |
+|--------|-----------------|------|
+| **SLM Hook** (primary) | Every tool call with input/output (500 chars), session ID, project path. Secret scrubbing built-in. | Zero — runs locally, no LLM calls |
+| **ECC Integration** (optional) | Rich observations from [Everything Claude Code](https://github.com/affaan-m/everything-claude-code) via `slm ingest --source ecc` | Zero — reads existing ECC data |
+| **Consolidation Pipeline** | Mines tool_events for patterns, creates assertions, updates skill entities | Zero — statistical analysis only |
+
+### What Gets Tracked Per Skill
+
+| Metric | Description |
+|--------|-------------|
+| **Invocation count** | How many times the skill was used |
+| **Effective score** | Approximate success rate based on execution trace analysis |
+| **Session count** | How many sessions used this skill |
+| **Skill correlations** | Which skills are frequently used together |
+
+### Outcome Heuristic
+
+SLM uses a conservative, approximate heuristic to determine if a skill invocation was effective:
+
+| Signal | Type | What It Means |
+|--------|------|---------------|
+| Productive tools follow (Edit, Write, Bash success) | Positive | Skill likely helped |
+| Same skill re-invoked within 5 minutes | Negative | Likely retry = failure |
+| Bash errors in next 3 tool events | Negative | Something went wrong |
+| Session continues 10+ events | Weak positive | User stayed engaged |
+
+These signals are labeled as **approximate** everywhere. They inform soft prompt routing but do not trigger automatic changes without human review.
+
+---
+
+## Dashboard: Skill Evolution Tab
+
+The dedicated **Skill Evolution** tab in the SLM dashboard shows:
+
+- **Overview cards** — Total skill events, unique skills, performance assertions, skill correlations
+- **Skill performance cards** — Per-skill effective score, invocation count, confidence level
+- **Skill correlations** — Which skills work well together
+- **Compatibility notice** — Current IDE support status
+
+Access: Open `http://localhost:8765` and navigate to the Skill Evolution tab in the sidebar.
+
+---
+
+## IDE Compatibility
+
+| IDE | Status | How |
+|-----|--------|-----|
+| **Claude Code** | Supported | SLM hook auto-registered via `slm init` |
+| **Cursor** | Planned | Adapter needed |
+| **Windsurf** | Planned | Adapter needed |
+| **VS Code Copilot** | Planned | Extension events adapter needed |
+| **JetBrains** | Planned | Adapter needed |
+| **Any IDE** | API available | POST to `/api/v3/tool-event` directly |
+
+The backend (API, miner, database, dashboard) is fully IDE-agnostic. Any client that POSTs tool events to the `/api/v3/tool-event` endpoint gets full benefit. The hook that ships with SLM is currently optimized for Claude Code.
+
+### API Endpoint
+
+```bash
+POST http://localhost:8765/api/v3/tool-event
+Content-Type: application/json
+
+{
+  "tool_name": "Skill",
+  "event_type": "complete",
+  "input_summary": "{\"skill\": \"my-skill-name\", \"args\": \"...\"}",
+  "output_summary": "{\"success\": true}",
+  "session_id": "your-session-id",
+  "project_path": "/path/to/project"
+}
+```
+
+All fields except `tool_name` are optional. Existing integrations that send only `tool_name` + `event_type` continue to work.
+
+---
+
+## ECC Integration
+
+[Everything Claude Code (ECC)](https://github.com/affaan-m/everything-claude-code) is a popular plugin for Claude Code that provides continuous learning, instinct-based pattern detection, and a rich observation pipeline.
+
+SLM's skill observation patterns were inspired by ECC's architecture. If you have ECC installed, you can enrich SLM's skill tracking with ECC's deeper observations:
+
+```bash
+# One-time import of existing ECC observations
+slm ingest --source ecc
+
+# Preview without writing (dry run)
+slm ingest --source ecc --dry-run
+```
+
+This reads ECC's observation files from `~/.claude/homunculus/projects/*/observations.jsonl` and imports them into SLM's `tool_events` table with full input/output preservation.
+
+**ECC is not required.** SLM is fully self-sufficient — its own hook captures all the data needed for skill tracking. ECC integration is an optional enhancement for users who want both systems working together.
+
+---
+
+## Configuration
+
+### Enable/Disable Skill Tracking
+
+Skill tracking is enabled by default when the SLM hook is registered. To check:
+
+```bash
+slm status  # Shows hook registration status
+```
+
+### Consolidation
+
+Skill performance mining runs as Step 10 in the consolidation pipeline. It processes only new events since the last run (incremental). To trigger manually:
+
+```bash
+slm consolidate --cognitive
+```
+
+### Minimum Thresholds
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| MIN_INVOCATIONS | 5 | Minimum uses before creating assertions |
+| MIN_CONFIDENCE | 0.5 | Minimum confidence for soft prompt injection |
+| TRACE_WINDOW | 10 | Tool events to analyze after each Skill call |
+| RETRY_WINDOW | 300s | Same Skill within this window = potential retry |
+
+These are conservative by design — we'd rather miss a pattern than hallucinate one.
+
+---
+
+## Research Foundations
+
+SLM's skill evolution system draws from:
+
+- **[EvoSkills](https://arxiv.org/abs/2604.01687)** (HKUDS, 2026) — Co-evolutionary verification with information isolation. +30pp improvement from blind verification.
+- **[OpenSpace](https://github.com/HKUDS/OpenSpace)** (HKUDS, MIT) — 3-trigger evolution system (post-analysis + tool degradation + metric monitor). Anti-loop guards. Version DAG model.
+- **[SkillsBench](https://arxiv.org/abs/2602.12670)** (2026) — 86-task benchmark showing self-generated skills provide zero benefit without verification. Focused 2-3 module skills outperform exhaustive docs.
+- **[SoK: Agent Skills](https://arxiv.org/abs/2602.12430)** (2026) — Four-axis taxonomy. Skills and MCP are orthogonal layers.
+
+---
+
+## What's Next
+
+- **C2: Arsenal Evolution Engine** — Automatic skill mutation with 3-trigger system, LLM confirmation gate, and blind verification. Evolved skills stored in `~/.claude/skills/evolved/`.
+- **C3: SLM Productization** — New MCP tools (`evolve_skill`, `skill_health`, `skill_lineage`), new fact types, full dashboard evolution history.
+- **IDE Adapters** — Cursor, Windsurf, VS Code Copilot, JetBrains support.
+
+---
+
+## Links
+
+- [SLM GitHub](https://github.com/qualixar/superlocalmemory)
+- [Qualixar](https://qualixar.com)
+- [Everything Claude Code](https://github.com/affaan-m/everything-claude-code)
+- [OpenSpace](https://github.com/HKUDS/OpenSpace)
+- [EvoSkills Paper](https://arxiv.org/abs/2604.01687)

package/ide/hooks/tool-event-hook.sh

@@ -1,11 +1,14 @@
 #!/usr/bin/env bash
-# SuperLocalMemory V3.4.7 — Tool Event Learning Hook
+# SuperLocalMemory V3.4.10 — Enriched Tool Event Learning Hook
 # Copyright (c) 2026 Varun Pratap Bhardwaj
 # Licensed under AGPL-3.0-or-later
 #
-# PostToolUse hook that logs tool events to SLM for behavioral learning.
-# Fires after every tool call. Lightweight — no LLM calls, just HTTP POST.
-# All data stays 100% local.
+# PostToolUse hook that logs RICH tool events to SLM for behavioral learning.
+# Captures: tool_name, input_summary (500 chars), output_summary (500 chars),
+#           session_id, project_path. Scrubs secrets before storing.
+#
+# Modeled after ECC observe.sh patterns — SLM is self-sufficient for observation.
+# An end user with only SLM (no ECC) gets full learning quality.
 #
 # Installation: slm init (auto-registers) or manually add to settings.json:
 #   { "type": "PostToolUse", "matcher": "*",
@@ -17,8 +20,26 @@ set -euo pipefail
 # Read stdin (Claude Code passes JSON with tool_name, input, output)
 INPUT=$(cat)
 
-# Extract tool name and event info
-TOOL_NAME=$(echo "$INPUT" | python3 -c "
+# Exit if no input
+if [ -z "$INPUT" ]; then
+  exit 0
+fi
+
+# Extract tool name (lightweight — bash-only fallback if python unavailable)
+TOOL_NAME=""
+if command -v python3 >/dev/null 2>&1; then
+  PYTHON_CMD="python3"
+elif command -v python >/dev/null 2>&1; then
+  PYTHON_CMD="python"
+else
+  # No python — fallback to basic extraction
+  TOOL_NAME=$(echo "$INPUT" | grep -o '"tool_name"[[:space:]]*:[[:space:]]*"[^"]*"' | head -1 | sed 's/.*"tool_name"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/' 2>/dev/null || echo "unknown")
+  PYTHON_CMD=""
+fi
+
+# Skip logging for our own tools (avoid recursion)
+if [ -n "$PYTHON_CMD" ]; then
+  TOOL_NAME=$(echo "$INPUT" | "$PYTHON_CMD" -c "
 import sys, json
 try:
     d = json.load(sys.stdin)
@@ -26,10 +47,11 @@ try:
 except:
     print('unknown')
 " 2>/dev/null || echo "unknown")
+fi
 
-# Skip logging for our own tools (avoid recursion)
 case "$TOOL_NAME" in
-    log_tool_event|get_assertions|reinforce_assertion|contradict_assertion)
+    log_tool_event|get_assertions|reinforce_assertion|contradict_assertion|\
+    mcp__superlocalmemory__*|session_init|recall|remember|observe|mesh_*)
         exit 0
         ;;
 esac
@@ -39,8 +61,76 @@ PORT=8765
 PORT_FILE="$HOME/.superlocalmemory/daemon.port"
 [ -f "$PORT_FILE" ] && PORT=$(cat "$PORT_FILE" 2>/dev/null || echo 8765)
 
-# Log event to SLM daemon (fire and forget, 2s timeout)
-curl -s -m 2 -X POST "http://127.0.0.1:${PORT}/api/v3/tool-event" \
+# If no python, send basic event (backward compatible)
+if [ -z "$PYTHON_CMD" ]; then
+  curl -s -m 2 -X POST "http://127.0.0.1:${PORT}/api/v3/tool-event" \
+      -H "Content-Type: application/json" \
+      -d "{\"tool_name\": \"${TOOL_NAME}\", \"event_type\": \"complete\"}" \
+      >/dev/null 2>&1 || true
+  exit 0
+fi
+
+# Parse full JSON and build enriched payload (with secret scrubbing)
+ENRICHED=$(echo "$INPUT" | "$PYTHON_CMD" -c '
+import json, sys, os, re
+
+# Secret scrubbing pattern (adopted from ECC observe.sh)
+_SECRET_RE = re.compile(
+    r"(?i)(api[_-]?key|token|secret|password|authorization|credentials?|auth)"
+    r"""([\"'"'"'"'"'"'\s:=]+)"""
+    r"([A-Za-z]+\s+)?"
+    r"([A-Za-z0-9_\-/.+=]{8,})"
+)
+
+def scrub(val, max_len=500):
+    """Truncate and scrub secrets from a value."""
+    if val is None:
+        return ""
+    if isinstance(val, dict):
+        s = json.dumps(val, default=str)
+    elif isinstance(val, list):
+        s = json.dumps(val, default=str)
+    else:
+        s = str(val)
+    s = s[:max_len]
+    return _SECRET_RE.sub(
+        lambda m: m.group(1) + m.group(2) + (m.group(3) or "") + "[REDACTED]", s
+    )
+
+try:
+    data = json.load(sys.stdin)
+
+    tool_name = data.get("tool_name", data.get("tool", "unknown"))
+
+    # Extract input — Claude Code uses tool_input or input
+    raw_input = data.get("tool_input", data.get("input", ""))
+    input_summary = scrub(raw_input, 500)
+
+    # Extract output — Claude Code uses tool_response, tool_output, or output
+    raw_output = data.get("tool_response")
+    if raw_output is None:
+        raw_output = data.get("tool_output", data.get("output", ""))
+    output_summary = scrub(raw_output, 500)
+
+    session_id = data.get("session_id", "")
+    project_path = data.get("cwd", "")
+
+    payload = {
+        "tool_name": tool_name,
+        "event_type": "complete",
+        "input_summary": input_summary,
+        "output_summary": output_summary,
+        "session_id": session_id,
+        "project_path": project_path,
+    }
+    print(json.dumps(payload))
+except Exception:
+    # Fallback — send basic event
+    print(json.dumps({"tool_name": "unknown", "event_type": "complete"}))
+' 2>/dev/null || echo '{"tool_name":"unknown","event_type":"complete"}')
+
+# Send enriched event to SLM daemon (fire and forget, 2s timeout)
+echo "$ENRICHED" | curl -s -m 2 -X POST "http://127.0.0.1:${PORT}/api/v3/tool-event" \
     -H "Content-Type: application/json" \
-    -d "{\"tool_name\": \"${TOOL_NAME}\", \"event_type\": \"complete\"}" \
+    -d @- \
     >/dev/null 2>&1 || true

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "superlocalmemory",
-  "version": "3.4.8",
+  "version": "3.4.10",
   "description": "Information-geometric agent memory with mathematical guarantees. 4-channel retrieval, Fisher-Rao similarity, zero-LLM mode, EU AI Act compliant. Works with Claude, Cursor, Windsurf, and 17+ AI tools.",
   "keywords": [
     "ai-memory",

package/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "superlocalmemory"
-version = "3.4.8"
+version = "3.4.10"
 description = "Information-geometric agent memory with mathematical guarantees"
 readme = "README.md"
 license = {text = "AGPL-3.0-or-later"}