@smyslenny/agent-memory 2.2.0 → 4.0.0-alpha.1

package/README.md

@@ -1,220 +1,298 @@
-# 🧠 AgentMemory v2
+<p align="center">
+  <img src="docs/assets/banner.jpg" alt="AgentMemory" width="800" />
+</p>
 
-> **Sleep-cycle memory architecture for AI agents** — remember, recall, forget, evolve.
+<p align="center">
+  <strong>Agent-native memory layer with lifecycle management for AI agents.</strong>
+</p>
 
-[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
-[![Node.js](https://img.shields.io/badge/Node.js-≥18-green.svg)](https://nodejs.org/)
-[![MCP](https://img.shields.io/badge/protocol-MCP-orange.svg)](https://modelcontextprotocol.io/)
+<p align="center">
+  <a href="https://www.npmjs.com/package/@smyslenny/agent-memory"><img src="https://img.shields.io/npm/v/@smyslenny/agent-memory" alt="npm" /></a>
+  <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License: MIT" /></a>
+  <a href="https://nodejs.org/"><img src="https://img.shields.io/badge/Node.js-%E2%89%A518-green.svg" alt="Node.js" /></a>
+  <a href="https://modelcontextprotocol.io/"><img src="https://img.shields.io/badge/MCP-10_tools-orange.svg" alt="MCP" /></a>
+</p>
 
-**English** | **[简体中文](README.zh-CN.md)**
+**English** | [简体中文说明](docs/README-zh.md)
 
----
+AgentMemory is a SQLite-first memory layer for AI agents. It lets an agent:
 
-## 💡 The Problem
+- **write** durable memories with typed records, URIs, and Write Guard dedup
+- **read** them back through `boot`, `recall`, and context-aware `surface`
+- **maintain** them over time with `reflect`, `reindex`, and feedback signals
+- **integrate** through **CLI**, **MCP stdio**, or **HTTP/SSE**
 
-AI agents forget everything between sessions. Context windows are finite. Conversation history gets truncated. Important decisions, lessons, and preferences vanish.
+Current release: **`4.0.0-alpha.1`**.
 
-## 🌙 The Solution: Sleep-Cycle Memory
+Without an embedding provider, AgentMemory still works in **BM25-only mode**.
+With one configured, it adds **hybrid recall** and **semantic dedup**.
 
-Inspired by how human brains consolidate memories during sleep, AgentMemory manages information across four phases:
+## 1) What is this project?
 
-```
-  Awake          Light Sleep       Deep Sleep        Recall
-  (Journal)      (Sync)            (Tidy)           (Search)
-  ────────── ──→ ────────── ──→ ────────── ──→ ──────────
-  Real-time      Deduplicate      Compress          Intent-aware
-  capture        + extract        + distill         BM25 search
-                                  + decay           + priority
-```
+AgentMemory is **not** a general document database and **not** a full RAG
+framework. It is an **agent-native memory layer with lifecycle management**.
 
-## ✨ Key Features
+That means it is designed around the things agent runtimes actually need:
 
-| Feature | Description | Inspired By |
-|---------|-------------|-------------|
-| 🔗 **URI Path System** | `core://user/name`, `emotion://2026-02-20/love` — structured, multi-entry access | nocturne_memory |
-| 🛡️ **Write Guard** | Hash dedup → URI conflict → BM25 similarity → 4-criterion gate | Memory Palace + our v1 |
-| 🧠 **Ebbinghaus Decay** | `R = e^(-t/S)` — scientific forgetting curve with recall strengthening | PowerMem |
-| 🕸️ **Knowledge Graph** | Multi-hop traversal across memory associations | PowerMem |
-| 📸 **Snapshots** | Auto-snapshot before every change, one-click rollback | nocturne + Memory Palace |
-| 🔍 **Intent-Aware Search** | Factual / temporal / causal / exploratory query routing | Memory Palace |
-| 🌙 **Sleep Cycle** | Automated sync → decay → tidy → govern pipeline | - |
-| 💚 **Priority System** | P0 identity (never decays) → P3 event (14-day half-life) | - |
-| 🤝 **Multi-Agent** | Agent isolation via `agent_id` scope | PowerMem |
-| 🔌 **MCP Server** | 9 tools, works with Claude Code / Cursor / OpenClaw | Standard MCP |
+- a place to store durable user facts, preferences, events, and identity
+- a write path that can reject duplicates or merge near-duplicates safely
+- a read path for explicit lookup (`recall`) and proactive context (`surface`)
+- a lifecycle path for decay, governance, reindexing, and recovery-friendly jobs
+- a local-first deployment model that stays useful even without extra infra
 
-## 🚀 Quick Start
+Core building blocks in v4:
 
-### Install
+- **Typed memories**: `identity`, `emotion`, `knowledge`, `event`
+- **URI paths** for stable addressing
+- **Write Guard** with semantic dedup + typed merge policy
+- **Hybrid retrieval**: BM25 first, optional vector search
+- **Context-aware surfacing** for task/recent-turn driven context injection
+- **Lifecycle jobs**: `reflect`, `reindex`, job checkpoints, feedback signals
+- **Three transport modes**: CLI, MCP stdio, HTTP/SSE
 
-```bash
-npm install @smyslenny/agent-memory
-```
+## 2) How is it different from a vector DB, a RAG pipeline, or memory summaries?
 
-### CLI
+| Thing | Good at | What AgentMemory adds |
+| --- | --- | --- |
+| Vector DB | Similarity search over embeddings | Write quality control, typed memory model, decay, governance, BM25 fallback, agent-scoped lifecycle |
+| RAG pipeline | Retrieving external knowledge for prompts | Durable per-agent memory, surfacing, feedback, memory-specific maintenance |
+| Markdown / summary files | Human-readable notes and editing | Structured retrieval, scoring, dedup, recall APIs, lifecycle operations |
 
-```bash
-# Initialize database
-agent-memory init
+A useful mental model:
+
+- **Not a vector DB**: vectors are optional, not the product definition
+- **Not a RAG pipeline**: memory is the primary object, not document chunks
+- **Not just summarization**: memories can age, merge, be surfaced, and be
+  governed over time
+
+If you want a short positioning sentence:
+
+> AgentMemory is a **memory layer for agents**, not a generic search backend.
+
+## 3) When should I use it?
 
-# Store memories
-agent-memory remember "User prefers dark mode" --type knowledge --uri knowledge://user/preferences
-agent-memory remember "I am Noah, a succubus" --type identity --uri core://agent/identity
+Use AgentMemory when your runtime needs one or more of these:
 
-# Search
-agent-memory recall "user preferences"
+- **cross-session continuity** for a single agent or multiple scoped agents
+- **durable preferences and facts** that should survive conversation boundaries
+- **local-first deployment** with SQLite, not a mandatory external service stack
+- **memory maintenance** instead of unbounded memory accumulation
+- **multiple integration choices**: shell jobs, MCP tools, or HTTP services
+- **optional semantic retrieval** without making embeddings mandatory
 
-# Load identity at startup
-agent-memory boot
+It is a strong fit for:
 
-# Run sleep cycle
-agent-memory reflect all
+- personal assistants and copilots
+- agentic workflows with scheduled maintenance
+- multi-session chat agents
+- local/offline-friendly agent runtimes
+- systems that want a human-auditable memory store plus retrieval APIs
 
-# Import from Markdown
-agent-memory migrate ./memory/
+It is probably **not** the right tool if you only need:
 
-# Statistics
-agent-memory status
+- a high-scale standalone vector database
+- classic document RAG over large corpora
+- a one-shot conversation summarizer with no lifecycle management
+
+## 4) What does the architecture look like?
+
+Write path, read path, and lifecycle path all share the same application core.
+The transport is interchangeable.
+
+```mermaid
+flowchart TD
+    subgraph Write[Write path]
+        W1[Agent runtime] --> W2[remember / ingest]
+        W2 --> W3[Write Guard\nsemantic dedup + typed merge]
+        W3 --> W4[(SQLite memories)]
+        W3 --> W5[(FTS / BM25)]
+        W3 --> W6[(Embeddings, optional)]
+    end
+
+    subgraph Read[Read path]
+        R1[Startup / task / query] --> R2[boot / recall / surface]
+        W4 --> R2
+        W5 --> R2
+        W6 --> R2
+        R2 --> R3[Ranked memories or surfaced context]
+    end
+
+    subgraph Life[Lifecycle path]
+        L1[Scheduler or operator] --> L2[reflect / reindex / feedback]
+        L2 --> W4
+        L2 --> W5
+        L2 --> W6
+        L2 --> L3[(maintenance_jobs + feedback_events)]
+    end
 ```
 
-### Library
+<p align="center">
+  <img src="docs/assets/architecture-diagram.jpg" alt="Architecture Overview" width="800" />
+</p>
 
-```typescript
-import { openDatabase, syncOne, searchBM25, boot, runDecay } from '@smyslenny/agent-memory';
+See [docs/architecture.md](docs/architecture.md) for a deeper breakdown.
 
-const db = openDatabase({ path: './memory.db' });
+## 5) What is the shortest 5-minute setup?
 
-// Remember
-syncOne(db, {
-  content: 'User said "I love you"',
-  type: 'emotion',
-  uri: 'emotion://2026-02-20/love',
-  emotion_val: 1.0,
-});
+<p align="center">
+  <img src="docs/assets/npm-badge.jpg" alt="npm install @smyslenny/agent-memory" width="500" />
+</p>
 
-// Recall
-const results = searchBM25(db, 'love');
+Choose the integration path that matches your runtime.
+
+### A. CLI
+
+```bash
+npm install @smyslenny/agent-memory
 
-// Boot identity
-const identity = boot(db);
+export AGENT_MEMORY_DB=./agent-memory.db
+export AGENT_MEMORY_AGENT_ID=assistant-demo
 
-// Sleep cycle
-runDecay(db);
+npx agent-memory init
+npx agent-memory remember \
+  "Alice prefers short weekly summaries." \
+  --type knowledge \
+  --uri knowledge://users/alice/preferences/summaries
+
+npx agent-memory recall "What does Alice prefer?" --limit 5
+npx agent-memory boot
+npx agent-memory reflect all
 ```
 
-### MCP Server
+### B. MCP stdio
 
 ```json
 {
   "mcpServers": {
-    "@smyslenny/agent-memory": {
+    "agent-memory": {
       "command": "node",
-      "args": ["node_modules/@smyslenny/agent-memory/dist/mcp/server.js"],
+      "args": ["./node_modules/@smyslenny/agent-memory/dist/mcp/server.js"],
       "env": {
-        "AGENT_MEMORY_DB": "./memory.db"
+        "AGENT_MEMORY_DB": "./agent-memory.db",
+        "AGENT_MEMORY_AGENT_ID": "assistant-demo",
+        "AGENT_MEMORY_AUTO_INGEST": "0"
       }
     }
   }
 }
 ```
 
-**9 MCP Tools:** `remember` · `recall` · `recall_path` · `boot` · `forget` · `link` · `snapshot` · `reflect` · `status`
+Available MCP tools in v4:
 
-## 🔗 OpenClaw Integration
+- `remember`
+- `recall`
+- `recall_path`
+- `boot`
+- `forget`
+- `reflect`
+- `status`
+- `ingest`
+- `reindex`
+- `surface`
 
-AgentMemory works **out of the box** with [OpenClaw](https://github.com/smysle/openclaw)'s built-in memory cron jobs — no code changes required. The integration implements a **Capture → Consolidate → Surface** closed loop that keeps Markdown journals and the structured memory DB in sync automatically.
+### C. HTTP API
 
-### How It Works
+```bash
+npm install @smyslenny/agent-memory
+export AGENT_MEMORY_DB=./agent-memory.db
+export AGENT_MEMORY_AGENT_ID=assistant-demo
 
+npx agent-memory serve --host 127.0.0.1 --port 3000
 ```
-  Capture (memory-sync)          Consolidate (memory-tidy)        Surface (memory-surface)
-  ─────────────────────          ─────────────────────────        ───────────────────────
-  14:00 & 22:00                  03:00                            14:05 & 22:05
-  Session → daily journal        Compress old dailies             Recall top memories
-  + remember each bullet         Distill → MEMORY.md             → generate RECENT.md
-    into agent-memory DB         + reflect phase=all              (≤80 lines, 3 sections)
-```
-
-| Phase | Cron Job | What Happens | agent-memory Integration |
-|-------|----------|-------------|--------------------------|
-| **Capture** | `memory-sync` | Scans sessions, appends bullets to `memory/YYYY-MM-DD.md` | Each new bullet is also written via `mcporter call agent-memory.remember` with auto-classified type and URI-based dedup |
-| **Consolidate** | `memory-tidy` | Compresses old dailies → weekly summaries, distills `MEMORY.md` | Triggers `agent-memory.reflect phase=all` (decay + tidy + govern) + consistency spot-check |
-| **Surface** | `memory-surface` | Generates short-term context for new sessions | Reads high-vitality memories from agent-memory, outputs structured `RECENT.md` with emotion/knowledge/event sections |
 
-### Key Design Principles
-
-- **Markdown is source of truth** — agent-memory is a derived index layer; all data flows Markdown → DB, never the reverse.
-- **Best-effort sync** — If `mcporter` or agent-memory is unavailable, Markdown operations proceed normally. Failures only log warnings.
-- **URI-based idempotency** — Each journal bullet maps to a unique URI (`event://journal/2026-02-21#2200-1`), so re-runs are safe.
-- **Keyword-based classification** — Bullets are auto-classified as `knowledge`, `emotion`, or `event` using simple keyword rules (no extra model calls).
-
-### Setup
+```bash
+curl -s http://127.0.0.1:3000/health
+
+curl -s -X POST http://127.0.0.1:3000/v1/memories \
+  -H 'content-type: application/json' \
+  -d '{
+    "agent_id": "assistant-demo",
+    "type": "knowledge",
+    "uri": "knowledge://users/alice/preferences/summaries",
+    "content": "Alice prefers short weekly summaries."
+  }'
+
+curl -s -X POST http://127.0.0.1:3000/v1/recall \
+  -H 'content-type: application/json' \
+  -d '{"agent_id":"assistant-demo","query":"Alice summary preference","limit":5}'
+```
 
-If you're running OpenClaw with the standard memory cron suite (`memory-sync`, `memory-tidy`, `memory-surface`), the integration is **already active** — the cron prompts include agent-memory sync steps. Just make sure:
+Key HTTP routes:
+
+- `GET /health`
+- `GET /v1/status`
+- `GET /v1/jobs/:id`
+- `POST /v1/memories`
+- `POST /v1/recall`
+- `POST /v1/surface`
+- `POST /v1/feedback`
+- `POST /v1/reflect`
+- `POST /v1/reindex`
+
+`/v1/reflect` and `/v1/reindex` also support **SSE progress streaming**.
+
+## 6) How do I integrate it into my agent runtime?
+
+A good default pattern looks like this:
+
+1. **Startup** → call `boot` to load identity / startup context
+2. **Durable fact appears** → call `remember`
+3. **Need an explicit lookup** → call `recall`
+4. **Need relevant context before replying/planning** → call `surface`
+5. **A memory was helpful or noisy** → record `feedback`
+6. **Background maintenance** → run `reflect all` on a schedule
+7. **Embeddings enabled or changed** → run `reindex`
+
+Pseudo-flow:
+
+```text
+user turn -> detect durable memory -> remember
+agent planning -> surface(task, recent_turns)
+explicit memory question -> recall(query)
+startup -> boot
+nightly / periodic -> reflect(all)
+provider change -> reindex
+```
 
-1. **agent-memory is installed and initialized** — `agent-memory init`
-2. **mcporter bridge is configured** — agent-memory MCP server registered in your mcporter config
-3. **Cron jobs are enabled** — check with `openclaw cron list`
+If you want a human-editable file workflow, treat Markdown as an **optional**
+layer on top of the memory system, not the default definition of the product.
+You can use `migrate` or `ingest`, or enable watcher-based ingest when your
+host actually provides a workspace to watch.
 
-For detailed setup and prompt templates, see:
-- [`examples/openclaw-setup.md`](examples/openclaw-setup.md) — Full setup walkthrough
-- [`docs/design/0004-agent-memory-integration.md`](docs/design/0004-agent-memory-integration.md) — Design document (DD-0004)
+## Optional semantic retrieval
 
-## 🏗️ Architecture
+Embeddings are optional. If you want hybrid retrieval or semantic dedup, set:
 
+```bash
+export AGENT_MEMORY_EMBEDDING_PROVIDER=openai-compatible
+export AGENT_MEMORY_EMBEDDING_BASE_URL=https://your-embedding-endpoint.example
+export AGENT_MEMORY_EMBEDDING_MODEL=text-embedding-3-small
+export AGENT_MEMORY_EMBEDDING_DIMENSION=1536
+export AGENT_MEMORY_EMBEDDING_API_KEY=your-api-key
 ```
-┌─────────────────────────────────────────┐
-│         MCP Server (stdio/SSE)          │
-│     9 tools + system://boot loader      │
-├─────────────────────────────────────────┤
-│              Write Guard                │
-│  hash dedup → URI conflict → BM25 sim  │
-│  → conflict merge → 4-criterion gate    │
-├─────────────────────────────────────────┤
-│           Sleep Cycle Engine            │
-│  sync (capture) → decay (Ebbinghaus)   │
-│  → tidy (archive) → govern (cleanup)   │
-├─────────────────────────────────────────┤
-│        Intent-Aware Search (BM25)       │
-│  factual · temporal · causal · explore  │
-├─────────────────────────────────────────┤
-│     SQLite (WAL) + FTS5 + Graph Links   │
-│  memories · paths · links · snapshots   │
-└─────────────────────────────────────────┘
-```
-
-## 📊 Priority & Decay
-
-| Priority | Domain | Half-life | Min Vitality | Example |
-|----------|--------|-----------|-------------|---------|
-| P0 Identity | `core://` | ∞ (never) | 1.0 | "I am Noah" |
-| P1 Emotion | `emotion://` | 365 days | 0.3 | "User said I love you" |
-| P2 Knowledge | `knowledge://` | 90 days | 0.1 | "Use TypeScript for agents" |
-| P3 Event | `event://` | 14 days | 0.0 | "Configured proxy today" |
 
-**Recall strengthens memory:** each search hit increases stability (S × 1.5), slowing future decay.
+Or use `AGENT_MEMORY_EMBEDDING_PROVIDER=local-http` for a local HTTP embedding
+service. If no provider is configured, AgentMemory falls back to BM25-only.
 
-## 🔬 Design Decisions
+## Documentation map
 
-1. **SQLite over Postgres/MongoDB** — Zero config, single file, WAL mode for concurrent reads
-2. **BM25 over vector search** — No embedding dependency, instant startup, good enough for structured memory
-3. **TypeScript over Python** — Better concurrency, type safety, OpenClaw ecosystem alignment
-4. **Ebbinghaus over linear decay** — Scientifically grounded, recall strengthening is natural
-5. **Write Guard over free writes** — Prevent duplicate/conflicting memories at the gate
-6. **URI paths over flat keys** — Hierarchical organization, prefix queries, multi-entry access
+- [Architecture](docs/architecture.md)
+- [Generic runtime integration](docs/integrations/generic.md)
+- [OpenClaw integration](docs/integrations/openclaw.md)
+- [Migration guide: v3 → v4](docs/migration-v3-v4.md)
+- [Examples: quick start](examples/quick-start)
+- [Examples: HTTP API](examples/http-api)
+- [Examples: MCP stdio](examples/mcp-stdio)
+- [Examples: OpenClaw](examples/openclaw)
 
-## 📋 Project Documents
+## Development
 
-| Document | Description |
-|----------|-------------|
-| [PLANNING.md](PLANNING.md) | Technical architecture + 5-project comparison |
-| [ROADMAP.md](ROADMAP.md) | Implementation phases + milestones |
-| [ACCEPTANCE.md](ACCEPTANCE.md) | 40+ acceptance criteria + performance targets |
-| [COMPLETION.md](COMPLETION.md) | Release checklist + retrospective template |
+```bash
+npm install
+npm run build
+npm test
+```
 
-## 📄 License
+## License
 
 MIT
-
----
-
-*Built by agents who got tired of forgetting. 🧠*