squish-memory 1.0.2 → 1.2.0

package/README.md

@@ -1,368 +1,273 @@
-# Squish - Universal Two-Tier Memory for AI Agents
+# Squish - Memory Runtime for AI Agents
 
 [![npm version](https://img.shields.io/npm/v/squish-memory)](https://www.npmjs.com/package/squish-memory)
 [![npm downloads](https://img.shields.io/npm/dm/squish-memory)](https://www.npmjs.com/package/squish-memory)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 [![TypeScript](https://img.shields.io/badge/TypeScript-5.7-blue.svg)](https://www.typescriptlang.org/)
 
-**Squish gives any AI agent persistent, intelligent memory through a two-tier architecture.** Without memory, agents forget everything between sessions. With Squish, they learn, adapt, and get smarter over time - regardless of which agent framework you use.
+**Your agent forgets. Squish fixes that.** It auto-captures useful context, derives durable beliefs like decisions and constraints, and restores that context through CLI, MCP, and a local web UI.
+
+> Squish does not have a crypto token, has no token launch planned, and nobody is authorized to launch one on behalf of the project.
 
 ```bash
-npm install squish-memory
+bun add squish-memory
 ```
 
-**Works out-of-the-box.** No database setup required - local TF-IDF embeddings work instantly. Upgrade to vector search with OpenAI/Ollama when ready.
-
-## Why Agents Need Memory
+## Why Squish
 
 | Without Squish | With Squish |
 |----------------|-------------|
-| Forgets after every session | Remembers across sessions |
+| Forgets everything after session | Remembers across sessions |
 | Repeats the same mistakes | Learns from past decisions |
-| No context awareness | Builds project understanding |
-| Can't track preferences | Adapts to user style |
+| No project awareness | Builds understanding over time |
+| Can't track preferences | Adapts to your style |
 
-## How It Works
+## What It Does
 
-Squish uses a two-tier memory architecture for optimal performance:
-- **Short-term Memory (QMD)**: Lightning-fast file-based search using QMD (BM25 + vector). Instant recall, optimized for recent/active context.
-- **Long-term Memory (Database)**: SQLite (local) or PostgreSQL (team) for durable, searchable storage of important memories.
+Squish is a forward-only memory runtime for agents:
 
-```
-Agent Action -----> [Squish Memory Layer]
-                            |
-                            v
-                     ┌──────────────┐
-                     │  Trigger     │ <-- "remember this", "important"
-                     │  Detection   │
-                     └──────────────┘
-                            |
-                            v
-                     ┌──────────────┐
-                     │  Write Gate  │ <-- Validate, sanitize, score
-                     └──────────────┘
-                            |
-            ┌───────────────┴───────────────┐
-            v                               v
-     ┌──────────────┐              ┌──────────────┐
-     │ Short-term   │              │ Long-term    │
-     │ QMD (Files)  │              │ SQLite/PG    │
-     │ Fast recall  │              │ Persistent   │
-     └──────────────┘              └──────────────┘
-            |                               |
-            v                               v
-     ┌──────────────────────────────────────────┐
-     │  Hybrid Retrieval: QMD + Vector Ranking  │
-     └──────────────────────────────────────────┘
-                            |
-                            v
-                     Agent Context
-```
-
-## Key Features
+- **Auto-capture** stores durable signal without relying on the model to remember to save it.
+- **Belief derivation** turns memories into decisions, constraints, and preferences that can change future behavior.
+- **Context restore** gives a restarted agent the relevant state instead of a cold start.
+- **Inspection surfaces** let you see what was stored, what belief was derived, and why it was injected back.
 
-### Memory Intelligence
-- **Trigger Detection**: Auto-detects "remember", "important", corrections
-- **Contradiction Resolution**: Auto-updates when facts change
-- **Temporal Facts**: Handles time-bound information ("until January")
-- **Confidence Scoring**: Knows how reliable each memory is
+Under the hood, Squish uses a hybrid memory pipeline for signal quality, persistence, and wake-up continuity:
 
-### Retrieval Quality
-- **Hybrid Search**: Vector + keyword (BM25) with fusion
-- **Multi-factor Ranking**: Semantic, recency, importance, confidence
-- **Telemetry**: Tracks which memories are actually useful
+```
+User Action ──► Signal Distillation ──► Write Gate ──► Session Working Set
+                                                          │
+                                                          ├─ Durable Distilled Memory
+                                                          ├─ Raw Fallback Snapshot
+                                                          └─ SQLite/Postgres + Hybrid Retrieval
+```
 
-### Agent Safety
-- **Write Gate**: Validates content before storage
-- **Secret Detection**: Auto-redacts API keys, passwords
-- **Graceful Degradation**: Works even when database fails
+- **Signal distillation**: Squish suppresses noisy output, keeps session-only context local, and only promotes durable signal.
+- **Session working set**: Active files, recent commands, failures, hypotheses, active places, and small graph cues are compacted for the next wake-up.
+- **Places**: Durable memories are routed into spatial buckets like `WIP`, `Sandbox`, `Board`, and `Ref` for segmented retrieval.
+- **Graph enrichment**: Durable memories strengthen entity and relationship structure used by retrieval scoring.
+- **Durable memory**: Stable facts, corrections, decisions, and fixes are stored for long-term retrieval.
+- **Raw fallback**: Nuance-sensitive output can keep an internal raw artifact for inspection without polluting normal context.
 
 ## Quick Start
 
-### Simple Example
-```bash
-# Store a memory
-squish remember "User prefers TypeScript"
-
-# Search memories
-squish search "preferences"
-
-# Check health
-squish health
-```
+### Install with add-mcp (Recommended)
+One command installs Squish into Claude Code, OpenCode, Cursor, VS Code, Codex, and other MCP-capable clients:
 
-### Universal Plugin Installer (Recommended)
 ```bash
-# Install for your AI assistant(s)
-npx squish-memory install-plugin --client=claude-code  # Claude Code
-npx squish-memory install-plugin --client=openclaw    # OpenClaw
-npx squish-memory install-plugin --client=opencode    # OpenCode
-npx squish-memory install-plugin --client=all         # All supported clients
-
-# Verify installation
-npx squish-memory install-plugin --client=claude-code --verify
+npx add-mcp squish-memory
 ```
 
-### For Claude Code (Plugin) - Legacy Method
-```bash
-# Install from marketplace
-/plugin marketplace add https://github.com/michielhdoteth/squish.git
-/plugin install squish@michielhdoteth-squish
-```
+Or install the package directly:
 
-### For OpenClaw (npm) - Legacy Method
 ```bash
-npm install -g squish-memory
+bun add squish-memory
 ```
 
-Add to your OpenClaw MCP config - done.
+New installs should work on first run with the current schema. If you are upgrading an older local install, use `squish doctor --migrate` to repair it forward.
 
-### Universal CLI
-```bash
-# Works with any agent framework
-squish remember "User prefers TypeScript"
-squish search "preferences"
-squish health
-```
+Most memory behavior is automatic once Squish is installed. The CLI remains available for explicit saves, inspection, diagnostics, and one-command demos:
 
-### Universal API
 ```bash
-# Start the universal HTTP server
-bun run universal:server
-
-# Add memory via HTTP (stored in both QMD index and SQLite)
-curl -X POST http://localhost:3000/api/memories \
-  -H "Content-Type: application/json" \
-  -d '{"content": "User prefers TypeScript", "type": "preference", "container": "my-project"}'
-
-# Search memories via HTTP (uses QMD for fast hybrid search)
-curl "http://localhost:3000/api/memories/search?q=TypeScript"
-```
-
-**That's it.** One install, persistent memory for any AI agent.
-
-## MCP Tools for Agents
+# Zero-touch demo: show current project context and derived beliefs
+squish context --json
 
-| Tool | What It Does |
-|------|--------------|
-| `remember` | Store a memory |
-| `search` | Find relevant memories |
-| `recall` | Get specific memory by ID |
-| `core_memory` | Always-visible context (persona, user info) |
-| `context` | Get project-relevant memories |
-| `observe` | Record patterns from tool usage |
+# Explicit save when you want to pin something intentionally
+squish remember "We chose PostgreSQL for team mode" --type decision
 
-## Execution Model
+# Inspect why a memory exists and which beliefs it supports
+squish inspect <memory-id> --json
 
-- **Universal First**: Works with any AI agent via MCP, CLI, or HTTP API
-- **Transport Agnostic**: MCP (stdio/SSE), CLI, or HTTP/WebSocket - choose your preference
-- **Storage Flexible**: SQLite for local, PostgreSQL for team deployments
-
-## Universal Plugin Architecture
-
-Squish now provides a **universal plugin system** that works as a plugin across all major AI assistant frameworks through a single manifest-driven installer.
-
-### How It Works
-1. Single `plugin-manifest.json` defines the plugin for all clients
-2. `npx squish-memory install-plugin --client=<target>` handles installation
-3. Auto-installs dependencies (mcporter, qmd) with pinned versions
-4. Generates client-specific configurations automatically
-5. Provides unified verification and troubleshooting
-
-### Supported Clients
-| Client | Installation Method | Status |
-|--------|-------------------|---------|
-| Claude Code | Plugin hooks (.claude-plugin/) | ✅ Stable |
-| OpenClaw | Memory slot via MCP bridge | ✅ Stable |
-| OpenCode | MCP server config | ✅ Stable |
-| Codex | MCP server config | ✅ Stable |
-| Cursor | MCP server config | ✅ Beta |
-| VS Code | MCP server config | ✅ Beta |
-| Windsurf | MCP server config | ✅ Beta |
-
-### Quick Installation Examples
-```bash
-# Install for Claude Code (recommended for Claude users)
-npx squish-memory install-plugin --client=claude-code --verify
-
-# Install for OpenClaw (recommended for OpenClaw users)  
-npx squish-memory install-plugin --client=openclaw --verify
-
-# Install for all supported clients
-npx squish-memory install-plugin --client=all --verify
-
-# Install for multiple specific clients
-npx squish-memory install-plugin --client=claude-code,openclaw,opencode --verify
+# Repair an older install forward if local schema drifted
+squish doctor --json --migrate
 ```
 
-### Benefits
-- **One manifest to rule them all**: Single source of truth
-- **Zero manual configuration**: Automatic dependency installation
-- **Version pinned dependencies**: Stable, reproducible builds
-- **Unified verification**: One command to check all clients
-- **Backward compatible**: Existing integrations still work
-
-## Universal API
-
-Squish now provides a universal HTTP API that works with any AI agent:
-
-```typescript
-// Add memory via HTTP
-POST /api/memories
-{
-  "content": "User prefers TypeScript",
-  "type": "preference",
-  "container": "my-project",
-  "tags": ["preferences", "coding-style"]
-}
-
-// Search memories via HTTP
-GET /api/memories/search?query=TypeScript&limit=10
-```
-
-**Universal Benefits:**
-- Works with any AI agent (Claude, OpenAI, Anthropic, custom)
-- HTTP RESTful API + WebSocket for real-time sync
-- PostgreSQL + pgvector for scalable memory
-- Docker-ready for easy deployment
-
-### Docker Deployment
+Or use the other shipped surfaces directly:
 
 ```bash
-# Quick start with Docker Compose
-docker-compose -f docker-compose.universal.yml up
+# MCP health check / manual startup surface
+squish-mcp --health
 
-# Or deploy to cloud
-docker build -t squish-universal .
-docker run -p 3000:3000 squish-universal
+# Local web UI
+squish run web
 ```
 
-## Open-Core Model
+## Features
 
-- **OSS Core (MIT)**: local mode, self-hosted workflows, MCP/CLI tooling
-- **Commercial Remote**: managed remote control plane, enterprise ops, support
-- **Universal API**: HTTP REST + WebSocket for any AI agent
-- **Sponsor development**: https://github.com/sponsors/michielhdoteth
+### Memory Intelligence
+- Auto-detects "remember this", "important", corrections
+- Distills noisy tool output before durable writes
+- Splits events into discarded, session-only, durable, and durable-with-raw-fallback paths
+- Handles contradictions when facts change
+- Temporal facts with expiration ("until January")
+- Confidence scoring for each memory
+- **Memory Runtime**: Hot/cold memory lifecycle with automatic decay (hot=active, cold=archived)
+- **Graph-boosted retrieval**: associations between memories boost relevance
+- **Belief System**: Derived semantic layer - decisions, preferences, constraints extracted from memories
+- **Persistent Hot Cache**: Karpathy-style wiki layer that survives restarts (not just session)
+- **Scheduler Resilience**: Jobs catch up after machine sleep/wake - no missed maintenance
+
+### Retrieval Quality
+- Session wake-up uses compacted working-set context before broad recall
+- Place context remains attached to retrieved memories and can shape context selection
+- Hybrid search: semantic + keyword (BM25) with Reciprocal Rank Fusion
+- Multi-factor ranking: relevance, recency, importance, graph-boost
+- LLM-powered context extraction with Ollama (local)
+- **Graph associations**: memories linked by coactivation boost search results
+
+### Security & Encryption
+- **Client-side encryption**: AES-256-GCM encryption for sensitive memories
+- **Passphrase management**: Via `SQUISH_ENCRYPTION_PASSPHRASE` env var (not exposed via MCP)
+- Encryption passphrase configured in `.env` file in data directory
+
+### Universal Compatibility
+- **CLI**: `squish remember`, `squish recall`, `squish inspect`, `squish context`, `squish stats`, `squish doctor`
+- **MCP Server**: Works with Claude Code, OpenCode, Cursor, VS Code, OpenClaw
+- **Web UI**: Inspect memories, projects, and recent observations locally
+- **SQLite**: Local, zero-config
+- **PostgreSQL**: Team mode with Supabase/pgvector
+- **QMD Integration**: Native .md file search via @tobilu/qmd npm package
+
+### Current MCP Tools (12 tools)
+- `squish_timeline` - 3-layer progressive disclosure
+- `squish_remember` - Store memory or learning (auto-detects type)
+- `squish_recall` - Recall memories by query or retrieve memory by ID
+- `squish_forget` - Delete memory by ID or bulk delete
+- `squish_link` - Manage memory associations
+- `squish_context` - Get project context
+- `squish_health` - Check system health
+- `squish_stats` - Get memory statistics
+- `squish_inspect` - Inspect memory retention
+- `squish_pin` - Pin/unpin memory
+- `squish_recent` - Get recent memories
+- `squish_stale` - Show stale memories
+
+## Benchmark Results
+
+Real tests using academic benchmarks with both configurations:
+
+### LoCoMo (1540 questions)
+| Configuration | Score | Notes |
+|---------------|-------|-------|
+| **Without LLM** | **65.19%** | Vector-only |
+| With LLM | 67.34% | Vector + LLM extraction |
+| Mem0 | 66.88% | Requires LLM |
+| TrueMemory | 91.5% | Uses LLM |
+
+**Squish achieves competitive scores WITHOUT requiring LLM tokens.**
+
+#### LoCoMo Breakdown (No LLM)
+| Category | Score |
+|----------|-------|
+| Single-hop | 85.14% |
+| Multi-hop | 47.78% |
+| Temporal | 95.24% |
+| Open-domain | 98.00% |
+| Common-sense | 1.79% |
+
+### LongMemEval (100 questions)
+| Configuration | Score | Breakdown |
+|---------------|-------|-----------|
+| **Without LLM** | **67.00%** | Temporal: 85%, Multi-session: 40% |
+| With LLM | 67.00% | Temporal: 85%, Multi-session: 40% |
+
+Note: LLM extraction helps during storage but scoring uses retrieved context keyword matching.
+
+### Performance
+| Metric | Result |
+|--------|--------|
+| Embedding Latency | 1-5ms |
+| API Latency | 1-20ms |
+| Max Throughput | 943 ops/sec |
+| Package Size | **283 KB** |
+
+### Local Runtime Characteristics
+
+| Characteristic | Result |
+|----------------|--------|
+| Default Cost | $0 local runtime |
+| Local-first | Yes |
+| Setup | 1 command |
+| API keys | Not required for the default path |
+| Session continuity | Built in |
+
+## Supported Clients
+
+| Client | Status |
+|--------|--------|
+| Claude Code | Stable |
+| OpenCode | Stable |
+| OpenClaw | Stable |
+| Cursor | Beta |
+| VS Code | Beta |
+| Windsurf | Beta |
 
 ## Configuration
 
-### Environment Variables
+**Zero config required** - works out of the box with local embeddings.
 
-**Required (local mode - default):**
-- None! Works out-of-the-box with local TF-IDF embeddings
+For customization:
 
-**Universal API:**
 ```bash
-# For universal HTTP API mode
-DATABASE_URL=postgresql://user:pass@host/db  # Required for universal mode
-REDIS_URL=redis://localhost:6379             # Optional for caching
-PORT=3000                                   # API server port
-```
-
-**Configuration File (config/settings.json):**
-```json
-{
-  "embeddings": {
-    "provider": "local",
-    "models": {
-      "openai": { "model": "text-embedding-3-small" },
-      "google": { "model": "gemini-embedding-001" },
-      "ollama": { "model": "nomic-embed-text:v1.5" }
-    }
-  }
-}
-```
-
-**Environment Variables (override settings.json):**
-```bash
-SQUISH_DATA_DIR=./.squish          # Custom data directory
-SQUISH_EMBEDDINGS_PROVIDER=local   # local, openai, ollama, google, none, auto
-
-# Model selection (optional, uses defaults if not set)
-SQUISH_OPENAI_EMBEDDING_MODEL=text-embedding-3-small
-SQUISH_GOOGLE_EMBEDDING_MODEL=gemini-embedding-001
-SQUISH_OLLAMA_EMBEDDING_MODEL=nomic-embed-text:v1.5
-
-# API credentials (for cloud providers)
-SQUISH_OPENAI_API_KEY=sk-...
+# Environment variables
+SQUISH_DATA_DIR=./.squish
+SQUISH_EMBEDDINGS_PROVIDER=ollama  # openai, ollama, google, local
 SQUISH_OLLAMA_URL=http://localhost:11434
-GOOGLE_CLOUD_PROJECT=your-project
-GOOGLE_CLOUD_API_KEY=your-key
 
-# Embedding performance & reliability
-SQUISH_EMBEDDINGS_TIMEOUT_MS=30000
-SQUISH_EMBEDDINGS_MAX_RETRIES=3
-SQUISH_EMBEDDINGS_RETRY_DELAY_MS=1000
-
-# Core memory size (default: 16KB total, 4KB per section)
-SQUISH_CORE_MEMORY_TOTAL_BYTES=16384
-SQUISH_CORE_MEMORY_SECTION_BYTES=4096
-
-# For team mode
+# Team mode
 DATABASE_URL=postgresql://user:pass@host/db
 ```
 
 ## Architecture
 
-### Two-Tier Memory System
-Squish employs a two-tier architecture for optimal performance and reliability:
-- **Short-term (QMD)**: File-based search using QMD (BM25 + vectors). Ultra-fast for recent context and active memories.
-- **Long-term (Database)**: SQLite (local) or PostgreSQL (team). Durable ACID-compliant storage for important memories that need persistence.
+### Two-Tier Memory
+- **QMD (Files)**: BM25 + vectors for fast recall
+- **SQLite/PostgreSQL**: ACID-compliant persistent storage
 
-### Universal Interfaces
-- **MCP Server**: Native integration for Claude Code, OpenClaw, and any MCP-compatible agent
-- **HTTP REST API**: Universal JSON API works with any AI agent capable of HTTP requests
-- **WebSocket**: Real-time memory sync and notifications for collaborative agents
-- **CLI**: Standalone command-line tool for shell-based agents and debugging
+### Runtime Pipeline
+- **Signal engine**: classifies captured events as discard, session-only, durable-distilled, or durable-with-raw-fallback
+- **Session working set**: persists active working context, active places, and small graph cues between sessions
+- **Places**: spatially segment durable memory for retrieval and wake-up continuity
+- **Graph**: incrementally enriches durable memories so graph boost applies to cleaner signal
+- **Inspection path**: lets you inspect why a memory was retained and whether a raw fallback artifact exists
 
-### Memory Organization
-- **Core Memory (configurable, default 16KB total)**: Always-visible sections for persona, user info, project context, and working notes. Each section limited to 4KB by default. Token estimation helps track LLM context usage.
-- **Context Paging**: Agent-controlled retrieval with token budgeting (8KB default)
-- **Background Jobs**: Automatic memory maintenance including decay, deduplication, and consolidation
+### Interfaces
+- **packages/mcp**: Native agent integration via Model Context Protocol
+- **HTTP**: Streamable HTTP server
+- **CLI**: Shell and scripts
 
 ### Memory Lifecycle
-- **Sectors**: episodic, semantic, procedural, autobiographical, working memory
-- **Tiers**: hot (recently accessed), warm (accessible), cold (archived but searchable)
-- **Status**: active, merged, superseded, expired (with automatic handling)
-
-### Deployment Flexibility
-- **Local SQLite**: Zero-configuration, perfect for individual agents and edge deployment
-- **PostgreSQL**: Horizontal scaling for teams and enterprise deployments
-- **Docker**: Single-command deployment with docker-compose.universal.yml
-- **Cloud**: Ready for AWS/GCP/Azure with standard PostgreSQL compatibility
+- **Sectors**: episodic, semantic, procedural, autobiographical
+- **Tiers**: hot (recent), warm (accessible), cold (archived)
+- **Status**: active, merged, superseded, expired
 
 ## Development
 
 ```bash
-# Install dependencies
 bun install
-
-# Build
 bun run build
-
-# Test
 bun test
-
-# Verify MCP
 bun run verify:mcp
 ```
 
 ## Troubleshooting
 
-### Database Issues
-- **SQLite corrupted**: Delete `.squish/squish.db` and restart
-- **PostgreSQL connection**: Verify DATABASE_URL format
+```bash
+# Repair an older local install forward
+squish doctor --migrate
+
+# Zero-touch runtime check
+squish context --json
 
-### MCP Issues
-- **Hooks not working**: Run `bun run build` first
-- **API prompts**: Set `SQUISH_EMBEDDINGS_PROVIDER=local`
+# MCP health check
+squish-mcp --health
+```
 
 ## License
 
-MIT for OSS core. See `LICENSE` for details.
+MIT License. See [LICENSE](LICENSE).
 
 ## Links
 
-- GitHub: https://github.com/michielhdoteth/squish
-- Issues: https://github.com/michielhdoteth/squish/issues
-- Sponsors: https://github.com/sponsors/michielhdoteth
+- [Documentation](https://github.com/michielhdoteth/squish)
+- [Benchmarks](docs/BENCHMARK.md)
+- [Issues](https://github.com/michielhdoteth/squish/issues)