@smyslenny/agent-memory 3.1.0 → 4.1.0-alpha.1

package/CHANGELOG.md

@@ -1,12 +1,127 @@
 # Changelog
 
+## 4.0.0-alpha.1 (2026-03-09)
+
+### 🚀 Repositioning
+
+AgentMemory v4 is now documented and packaged as an **agent-native memory layer
+with lifecycle management**.
+
+What changed at the product level:
+
+- README is now **English-first** and generic-runtime-first
+- OpenClaw is still supported, but now documented as an **optional host example**
+- `memory/*.md + MEMORY.md` is treated as an **optional workflow**, not the
+  product definition
+- CLI, MCP stdio, and HTTP/SSE are all first-class integration paths
+
+### ✨ Added in Phase 1 — optional vector retrieval layer
+
+- Added **optional embedding provider support** for hybrid retrieval:
+  - `openai-compatible`
+  - `local-http`
+- Added **hybrid recall** with BM25 + vector fusion
+- Added embedding-aware storage and reindex support:
+  - `provider_id`
+  - `content_hash`
+  - `status`
+- Added **`reindex`** support for backfill / rebuild workflows
+- Added provider configuration via environment variables:
+  - `AGENT_MEMORY_EMBEDDING_PROVIDER`
+  - `AGENT_MEMORY_EMBEDDING_BASE_URL`
+  - `AGENT_MEMORY_EMBEDDING_MODEL`
+  - `AGENT_MEMORY_EMBEDDING_DIMENSION`
+  - `AGENT_MEMORY_EMBEDDING_API_KEY`
+- Kept **BM25-only mode** as a supported fallback when no provider is configured
+
+### ✨ Added in Phase 2 — semantic dedup + lifecycle reliability
+
+- Upgraded **Write Guard** from simple duplicate checks to semantic dedup flow
+- Added **typed merge policy** so similar memories can be merged more safely
+- Added **maintenance job tracking** for lifecycle operations
+- Added checkpoint-aware **reflect orchestrator** for:
+  - `decay`
+  - `tidy`
+  - `govern`
+- Improved lifecycle observability and recovery-friendliness for interrupted
+  maintenance runs
+
+### ✨ Added in Phase 3 — HTTP/SSE API + better surface
+
+- Added long-lived **HTTP API** transport
+- Added **SSE progress streaming** for long-running jobs
+- Added HTTP routes for:
+  - `POST /v1/memories`
+  - `POST /v1/recall`
+  - `POST /v1/surface`
+  - `POST /v1/feedback`
+  - `POST /v1/reflect`
+  - `POST /v1/reindex`
+  - `GET /v1/status`
+  - `GET /v1/jobs/:id`
+  - `GET /health`
+- Added CLI server mode:
+  - `agent-memory serve`
+- Upgraded **surface** into a more **context-aware** API using:
+  - `task`
+  - `query`
+  - `recent_turns`
+  - `intent`
+  - type filters
+  - feedback priors
+- Added **feedback events** so runtimes can record whether `recall` / `surface`
+  results were actually useful
+
+### 🧰 Tooling / interface changes
+
+- MCP toolset is now **10 tools**:
+  - `remember`
+  - `recall`
+  - `recall_path`
+  - `boot`
+  - `forget`
+  - `reflect`
+  - `status`
+  - `ingest`
+  - `reindex`
+  - `surface`
+- CLI now includes:
+  - `db:migrate`
+  - `reindex`
+  - `serve`
+
+### 📚 Documentation and examples overhaul
+
+- Rewrote `README.md` for OSS evaluation and generic runtime adoption
+- Removed the split `README.md` / `README.en.md` homepage model
+- Added dedicated docs:
+  - `docs/architecture.md`
+  - `docs/integrations/generic.md`
+  - `docs/integrations/openclaw.md`
+  - `docs/migration-v3-v4.md`
+- Reorganized examples into:
+  - `examples/quick-start/`
+  - `examples/http-api/`
+  - `examples/mcp-stdio/`
+  - `examples/openclaw/`
+
+### ✅ Compatibility notes
+
+- Existing CLI and MCP usage remains available
+- HTTP/SSE is **additive**, not a replacement
+- Existing SQLite deployments can upgrade incrementally
+- Full embeddings are **optional** and can be enabled later with `reindex`
+
+---
+
 ## 3.0.1 (2026-02-24)
 
 ### 🛠️ OpenClaw P0 fixes
 
 - **Fixed memory-sync session path mismatch** in cron prompt:
   - removed hardcoded `~/.openclaw/agents/main/sessions/*.jsonl`
-  - switched to dynamic discovery with `noah` + env-derived agent path + `main` fallback
+  - switched to dynamic discovery with `noah` + env-derived agent path + `main`
+    fallback
 - **Aligned memory-tidy prompt** with the same session path health check strategy
 - **Added memory-sync health output contract**:
   - `session_scan_glob`
@@ -39,7 +154,8 @@
 ### 📚 Documentation realigned to v3 reality
 
 - Rewrote `README.md` and `README.en.md` to match actual v3 capabilities
-- Removed stale v2-era claims (embedding/reranker/link/snapshot/hybrid stack narrative)
+- Removed stale v2-era claims (embedding/reranker/link/snapshot/hybrid stack
+  narrative)
 - Added explicit auto-ingest watcher behavior and env vars
 
 ---
@@ -51,11 +167,13 @@
 - Repositioned agent-memory as a structured companion to memory-core
 - Removed redundant v2 capabilities at API/tooling level
 - MCP toolset finalized at 9 tools:
-  - `remember`, `recall`, `recall_path`, `boot`, `forget`, `reflect`, `status`, `ingest`, `surface`
+  - `remember`, `recall`, `recall_path`, `boot`, `forget`, `reflect`, `status`,
+    `ingest`, `surface`
 - Added narrative warm-boot and human-readable reflect report
 
 ---
 
 ## 2.x (legacy)
 
-v2.x included embedding/reranker/link/snapshot-era behavior. See git history and design docs for full details.
+v2.x included embedding/reranker/link/snapshot-era behavior. See git history and
+design docs for full details.

package/README.md

@@ -1,88 +1,183 @@
-# 🧠 AgentMemory v3
+<p align="center">
+  <img src="docs/assets/banner.jpg" alt="AgentMemory" width="800" />
+</p>
 
-> 面向 AI Agent 的结构化长期记忆层：可写入、可检索、可衰减、可自动摄取。
+<p align="center">
+  <strong>Agent-native memory layer with lifecycle management for AI agents.</strong>
+</p>
 
-[![npm](https://img.shields.io/npm/v/@smyslenny/agent-memory)](https://www.npmjs.com/package/@smyslenny/agent-memory)
-[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
-[![Node.js](https://img.shields.io/badge/Node.js-≥18-green.svg)](https://nodejs.org/)
-[![MCP](https://img.shields.io/badge/MCP-9_tools-orange.svg)](https://modelcontextprotocol.io/)
+<p align="center">
+  <a href="https://www.npmjs.com/package/@smyslenny/agent-memory"><img src="https://img.shields.io/npm/v/@smyslenny/agent-memory" alt="npm" /></a>
+  <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License: MIT" /></a>
+  <a href="https://nodejs.org/"><img src="https://img.shields.io/badge/Node.js-%E2%89%A518-green.svg" alt="Node.js" /></a>
+  <a href="https://modelcontextprotocol.io/"><img src="https://img.shields.io/badge/MCP-10_tools-orange.svg" alt="MCP" /></a>
+</p>
 
-**简体中文** | **[English](README.en.md)**
+**English** | [简体中文说明](docs/README-zh.md)
 
----
+AgentMemory is a SQLite-first memory layer for AI agents. It lets an agent:
 
-## 项目定位（v3）
+- **write** durable memories with typed records, URIs, and Write Guard dedup
+- **read** them back through `boot`, `recall`, and context-aware `surface`
+- **maintain** them over time with `reflect`, `reindex`, and feedback signals
+- **integrate** through **CLI**, **MCP stdio**, or **HTTP/SSE**
 
-AgentMemory 在 v3 中明确定位为 **OpenClaw memory-core 的结构化补充层**，而不是第二套全栈检索系统：
+Current release: **`4.0.0-alpha.1`**.
 
-- Markdown（`memory/*.md` + `MEMORY.md`）是可读可编辑的事实源
-- agent-memory 是派生索引层，负责结构化记忆生命周期
+Without an embedding provider, AgentMemory still works in **BM25-only mode**.
+With one configured, it adds **hybrid recall** and **semantic dedup**.
 
-核心能力：
+## 1) What is this project?
 
-- **类型化记忆**：`identity / emotion / knowledge / event`
-- **URI 路径寻址**：`core://`、`emotion://`、`knowledge://`、`event://`
-- **Write Guard**：写入前做去重与冲突门控
-- **BM25 检索**：带 priority × vitality 加权
-- **睡眠周期**：`reflect` 触发 decay / tidy / govern
-- **ingest 自动摄取**：从 markdown 提取并入库
-- **surface 只读浮现**：无副作用地补充上下文
-- **warm boot / reflect 报告**：人类可读输出
-- **多 Agent 隔离**：同库不同 agent_id 互不污染
+AgentMemory is **not** a general document database and **not** a full RAG
+framework. It is an **agent-native memory layer with lifecycle management**.
 
----
+That means it is designed around the things agent runtimes actually need:
 
-## 快速开始
+- a place to store durable user facts, preferences, events, and identity
+- a write path that can reject duplicates or merge near-duplicates safely
+- a read path for explicit lookup (`recall`) and proactive context (`surface`)
+- a lifecycle path for decay, governance, reindexing, and recovery-friendly jobs
+- a local-first deployment model that stays useful even without extra infra
 
-### 安装
+Core building blocks in v4:
 
-```bash
-npm install -g @smyslenny/agent-memory
-```
+- **Typed memories**: `identity`, `emotion`, `knowledge`, `event`
+- **URI paths** for stable addressing
+- **Write Guard** with semantic dedup + typed merge policy
+- **Hybrid retrieval**: BM25 first, optional vector search
+- **Context-aware surfacing** for task/recent-turn driven context injection
+- **Lifecycle jobs**: `reflect`, `reindex`, job checkpoints, feedback signals
+- **Three transport modes**: CLI, MCP stdio, HTTP/SSE
 
-### CLI 示例
+## 2) How is it different from a vector DB, a RAG pipeline, or memory summaries?
 
-```bash
-# 初始化数据库
-agent-memory init
+| Thing | Good at | What AgentMemory adds |
+| --- | --- | --- |
+| Vector DB | Similarity search over embeddings | Write quality control, typed memory model, decay, governance, BM25 fallback, agent-scoped lifecycle |
+| RAG pipeline | Retrieving external knowledge for prompts | Durable per-agent memory, surfacing, feedback, memory-specific maintenance |
+| Markdown / summary files | Human-readable notes and editing | Structured retrieval, scoring, dedup, recall APIs, lifecycle operations |
+
+A useful mental model:
+
+- **Not a vector DB**: vectors are optional, not the product definition
+- **Not a RAG pipeline**: memory is the primary object, not document chunks
+- **Not just summarization**: memories can age, merge, be surfaced, and be
+  governed over time
+
+If you want a short positioning sentence:
+
+> AgentMemory is a **memory layer for agents**, not a generic search backend.
+
+## 3) When should I use it?
+
+Use AgentMemory when your runtime needs one or more of these:
+
+- **cross-session continuity** for a single agent or multiple scoped agents
+- **durable preferences and facts** that should survive conversation boundaries
+- **local-first deployment** with SQLite, not a mandatory external service stack
+- **memory maintenance** instead of unbounded memory accumulation
+- **multiple integration choices**: shell jobs, MCP tools, or HTTP services
+- **optional semantic retrieval** without making embeddings mandatory
 
-# 写入记忆
-agent-memory remember "用户偏好深色模式" --type knowledge --uri knowledge://preferences/theme
+It is a strong fit for:
 
-# 检索
-agent-memory recall "用户偏好" --limit 5
+- personal assistants and copilots
+- agentic workflows with scheduled maintenance
+- multi-session chat agents
+- local/offline-friendly agent runtimes
+- systems that want a human-auditable memory store plus retrieval APIs
 
-# 启动时加载（叙事格式）
-agent-memory boot
+It is probably **not** the right tool if you only need:
 
-# 触发睡眠周期
-agent-memory reflect all
+- a high-scale standalone vector database
+- classic document RAG over large corpora
+- a one-shot conversation summarizer with no lifecycle management
+
+## 4) What does the architecture look like?
+
+Write path, read path, and lifecycle path all share the same application core.
+The transport is interchangeable.
+
+```mermaid
+flowchart TD
+    subgraph Write[Write path]
+        W1[Agent runtime] --> W2[remember / ingest]
+        W2 --> W3[Write Guard\nsemantic dedup + typed merge]
+        W3 --> W4[(SQLite memories)]
+        W3 --> W5[(FTS / BM25)]
+        W3 --> W6[(Embeddings, optional)]
+    end
+
+    subgraph Read[Read path]
+        R1[Startup / task / query] --> R2[boot / recall / surface]
+        W4 --> R2
+        W5 --> R2
+        W6 --> R2
+        R2 --> R3[Ranked memories or surfaced context]
+    end
+
+    subgraph Life[Lifecycle path]
+        L1[Scheduler or operator] --> L2[reflect / reindex / feedback]
+        L2 --> W4
+        L2 --> W5
+        L2 --> W6
+        L2 --> L3[(maintenance_jobs + feedback_events)]
+    end
 ```
 
----
+<p align="center">
+  <img src="docs/assets/architecture-diagram.jpg" alt="Architecture Overview" width="800" />
+</p>
 
-## MCP Server
+See [docs/architecture.md](docs/architecture.md) for a deeper breakdown.
 
-### 配置示例
+## 5) What is the shortest 5-minute setup?
+
+<p align="center">
+  <img src="docs/assets/npm-badge.jpg" alt="npm install @smyslenny/agent-memory" width="500" />
+</p>
+
+Choose the integration path that matches your runtime.
+
+### A. CLI
+
+```bash
+npm install @smyslenny/agent-memory
+
+export AGENT_MEMORY_DB=./agent-memory.db
+export AGENT_MEMORY_AGENT_ID=assistant-demo
+
+npx agent-memory init
+npx agent-memory remember \
+  "Alice prefers short weekly summaries." \
+  --type knowledge \
+  --uri knowledge://users/alice/preferences/summaries
+
+npx agent-memory recall "What does Alice prefer?" --limit 5
+npx agent-memory boot
+npx agent-memory reflect all
+```
+
+### B. MCP stdio
 
 ```json
 {
   "mcpServers": {
     "agent-memory": {
       "command": "node",
-      "args": ["node_modules/@smyslenny/agent-memory/dist/mcp/server.js"],
+      "args": ["./node_modules/@smyslenny/agent-memory/dist/mcp/server.js"],
       "env": {
         "AGENT_MEMORY_DB": "./agent-memory.db",
-        "AGENT_MEMORY_AGENT_ID": "noah",
-        "AGENT_MEMORY_AUTO_INGEST": "1",
-        "AGENT_MEMORY_WORKSPACE": "/home/user/.openclaw/workspace"
+        "AGENT_MEMORY_AGENT_ID": "assistant-demo",
+        "AGENT_MEMORY_AUTO_INGEST": "0"
       }
     }
   }
 }
 ```
 
-### MCP 工具（9个）
+Available MCP tools in v4:
 
 - `remember`
 - `recall`
@@ -92,62 +187,112 @@ agent-memory reflect all
 - `reflect`
 - `status`
 - `ingest`
+- `reindex`
 - `surface`
 
-> v3 已移除 `link` / `snapshot` 工具。
+### C. HTTP API
 
----
-
-## Auto-Ingest（文件变更自动入库）
-
-MCP server 启动后会默认开启 watcher（`fs.watch`）：
-
-- `~/.openclaw/workspace/memory/*.md`
-- `~/.openclaw/workspace/MEMORY.md`
-
-当文件变化时自动执行 ingest（复用 Write Guard，幂等/去重）。
+```bash
+npm install @smyslenny/agent-memory
+export AGENT_MEMORY_DB=./agent-memory.db
+export AGENT_MEMORY_AGENT_ID=assistant-demo
 
-环境变量：
+npx agent-memory serve --host 127.0.0.1 --port 3000
+```
 
-- `AGENT_MEMORY_AUTO_INGEST`
-  - `1`（默认）：开启
-  - `0`：关闭
-- `AGENT_MEMORY_WORKSPACE`
-  - 默认：`$HOME/.openclaw/workspace`
+```bash
+curl -s http://127.0.0.1:3000/health
+
+curl -s -X POST http://127.0.0.1:3000/v1/memories \
+  -H 'content-type: application/json' \
+  -d '{
+    "agent_id": "assistant-demo",
+    "type": "knowledge",
+    "uri": "knowledge://users/alice/preferences/summaries",
+    "content": "Alice prefers short weekly summaries."
+  }'
+
+curl -s -X POST http://127.0.0.1:3000/v1/recall \
+  -H 'content-type: application/json' \
+  -d '{"agent_id":"assistant-demo","query":"Alice summary preference","limit":5}'
+```
 
----
+Key HTTP routes:
+
+- `GET /health`
+- `GET /v1/status`
+- `GET /v1/jobs/:id`
+- `POST /v1/memories`
+- `POST /v1/recall`
+- `POST /v1/surface`
+- `POST /v1/feedback`
+- `POST /v1/reflect`
+- `POST /v1/reindex`
+
+`/v1/reflect` and `/v1/reindex` also support **SSE progress streaming**.
+
+## 6) How do I integrate it into my agent runtime?
+
+A good default pattern looks like this:
+
+1. **Startup** → call `boot` to load identity / startup context
+2. **Durable fact appears** → call `remember`
+3. **Need an explicit lookup** → call `recall`
+4. **Need relevant context before replying/planning** → call `surface`
+5. **A memory was helpful or noisy** → record `feedback`
+6. **Background maintenance** → run `reflect all` on a schedule
+7. **Embeddings enabled or changed** → run `reindex`
+
+Pseudo-flow:
+
+```text
+user turn -> detect durable memory -> remember
+agent planning -> surface(task, recent_turns)
+explicit memory question -> recall(query)
+startup -> boot
+nightly / periodic -> reflect(all)
+provider change -> reindex
+```
 
-## OpenClaw 集成建议（方案A）
+If you want a human-editable file workflow, treat Markdown as an **optional**
+layer on top of the memory system, not the default definition of the product.
+You can use `migrate` or `ingest`, or enable watcher-based ingest when your
+host actually provides a workspace to watch.
 
-推荐三段 cron：
+## Optional semantic retrieval
 
-1. `memory-sync`（14:00 / 22:00）
-   - 动态发现 session JSONL
-   - 增量写入 `memory/YYYY-MM-DD.md`
-   - best-effort 同步到 `agent-memory.remember`
-   - 输出健康指标（扫描路径、会话文件数、提取数、写库数）
+Embeddings are optional. If you want hybrid retrieval or semantic dedup, set:
 
-2. `memory-tidy`（03:00）
-   - 压缩/蒸馏 markdown
-   - 调用 `agent-memory.reflect phase=all`
+```bash
+export AGENT_MEMORY_EMBEDDING_PROVIDER=openai-compatible
+export AGENT_MEMORY_EMBEDDING_BASE_URL=https://your-embedding-endpoint.example
+export AGENT_MEMORY_EMBEDDING_MODEL=text-embedding-3-small
+export AGENT_MEMORY_EMBEDDING_DIMENSION=1536
+export AGENT_MEMORY_EMBEDDING_API_KEY=your-api-key
+```
 
-3. `memory-surface`（14:05 / 22:05）
-   - 生成 `RECENT.md`
+Or use `AGENT_MEMORY_EMBEDDING_PROVIDER=local-http` for a local HTTP embedding
+service. If no provider is configured, AgentMemory falls back to BM25-only.
 
-设计原则：**Markdown 是真相源，agent-memory 是派生索引层。**
+## Documentation map
 
----
+- [Architecture](docs/architecture.md)
+- [Generic runtime integration](docs/integrations/generic.md)
+- [OpenClaw integration](docs/integrations/openclaw.md)
+- [Migration guide: v3 → v4](docs/migration-v3-v4.md)
+- [Examples: quick start](examples/quick-start)
+- [Examples: HTTP API](examples/http-api)
+- [Examples: MCP stdio](examples/mcp-stdio)
+- [Examples: OpenClaw](examples/openclaw)
 
-## 开发
+## Development
 
 ```bash
 npm install
-npm test
 npm run build
+npm test
 ```
 
----
-
 ## License
 
 MIT