@elizaos/plugin-memory 1.0.5 → 1.1.0

package/README.md

@@ -1,34 +1,22 @@
 # @elizaos/plugin-memory
 
-Advanced memory management plugin for ElizaOS that provides intelligent conversation summarization and persistent long-term memory storage.
+Advanced cognitive memory system for ElizaOS agents with persistent memory, intelligent fact extraction, and hybrid retrieval.
 
-## Features
+> 📚 **Research-Driven**: Built on analysis of 40+ academic papers. See [`refactor.md`](./refactor.md) and [`EVALUATION.md`](./EVALUATION.md) for details.
 
-### 🔄 Short-term Memory (Conversation Summarization)
+---
 
-- **Automatic Summarization**: Compresses long conversations when they exceed configurable thresholds
-- **Context Preservation**: Maintains conversation flow while dramatically reducing token usage
-- **Recent Message Retention**: Keeps the most recent messages for immediate context
-- **Topic Extraction**: Identifies and tracks main topics discussed in each session
+## Overview
 
-### 🧠 Long-term Memory (Persistent Facts)
+Transforms stateless LLM agents into persistent entities with human-like memory capabilities:
 
-- **Intelligent Extraction**: Automatically learns facts about users from conversations
-- **Categorized Storage**: Organizes information into 9 semantic categories
-- **Confidence Scoring**: Tracks reliability of stored information
-- **Cross-session Persistence**: Remembers user preferences and context across all interactions
+- **Three-Tier Memory**: Episodic (events), Semantic (facts), Procedural (skills)
+- **Smart Consolidation**: Automatically extracts persistent facts from conversations
+- **Hybrid Retrieval**: Vector + BM25 + Graph search with exponential decay
+- **Contextual Embeddings**: Self-contained memories that improve retrieval accuracy
+- **Contradiction Detection**: Automatically handles conflicting information
 
-### 📊 Memory Categories
-
-1. **Identity**: User's name, role, profession (e.g., "I'm a data scientist")
-2. **Expertise**: Domain knowledge, skills, familiarity with topics
-3. **Projects**: Ongoing work, past interactions, recurring topics
-4. **Preferences**: Communication style, format preferences, verbosity
-5. **Data Sources**: Frequently used files, databases, APIs
-6. **Goals**: Broader intentions and objectives
-7. **Constraints**: User-defined rules or limitations
-8. **Definitions**: Custom terms, acronyms, glossaries
-9. **Behavioral Patterns**: Interaction styles and tendencies
+---
 
 ## Installation
 
@@ -36,276 +24,412 @@ Advanced memory management plugin for ElizaOS that provides intelligent conversa
 bun add @elizaos/plugin-memory
 ```
 
-## Usage
-
-### Basic Setup
+Add to your character file:
 
-```typescript
-import { memoryPlugin } from '@elizaos/plugin-memory';
-
-const agent = new Agent({
-  name: 'MyAgent',
-  plugins: [
-    memoryPlugin,
-    // ... other plugins
-  ],
-});
+```json
+{
+  "plugins": ["@elizaos/plugin-memory"]
+}
 ```
 
-### Configuration
+The plugin auto-creates database tables on first load and handles everything automatically.
+
+---
+
+## Configuration
+
+### Quick Reference
+
+| Setting                          | Default | Description                             |
+| -------------------------------- | ------- | --------------------------------------- |
+| `MEMORY_CONSOLIDATION_THRESHOLD` | `12`    | Messages before fact extraction         |
+| `MEMORY_MIN_CONFIDENCE`          | `0.7`   | Minimum confidence to store facts (0-1) |
+| `MEMORY_ENABLE_VECTOR_SEARCH`    | `true`  | Enable semantic search                  |
+| `MEMORY_ENABLE_BM25`             | `true`  | Enable keyword search                   |
+| `MEMORY_RETRIEVAL_LIMIT`         | `5`     | Max memories per query                  |
+| `MEMORY_TOKEN_BUDGET`            | `1000`  | Token budget for memory context         |
+| `MEMORY_SUMMARY_ENABLED`         | `true`  | Enable conversation summarization       |
+| `MEMORY_MESSAGES_PER_SUMMARY`    | `7`     | Messages per summary                    |
+| `MEMORY_SUMMARIES_PER_LEVEL`     | `5`     | Summaries before next level             |
+| `MEMORY_SUMMARY_MAX_DEPTH`       | `3`     | Maximum summary hierarchy depth         |
+| `MEMORY_SUMMARY_TOKEN_BUDGET`    | `500`   | Token budget for summaries              |
+| `CONTEXT_OVERLAP_USER_MESSAGES`  | `2`     | User messages overlap after summary     |
+
+### Configuration Methods
+
+Settings are loaded via `runtime.getSetting()` with priority:
+
+1. **Character `settings`** (highest priority)
+2. **Environment variables** (`.env` file)
+3. **Plugin defaults** (fallback)
 
-Configure the plugin via environment variables in your `.env` file:
+#### Character File Example
+
+```json
+{
+  "name": "MyAgent",
+  "plugins": ["@elizaos/plugin-memory"],
+  "settings": {
+    "MEMORY_CONSOLIDATION_THRESHOLD": "15",
+    "MEMORY_MIN_CONFIDENCE": "0.8",
+    "MEMORY_RETRIEVAL_LIMIT": "10"
+  }
+}
+```
+
+#### `.env` File Example
 
 ```env
-# Short-term Memory Settings
-MEMORY_SUMMARIZATION_THRESHOLD=16    # Messages before summarization starts (default: 16)
-MEMORY_SUMMARIZATION_INTERVAL=10     # Update summary every N messages (default: 10)
-MEMORY_RETAIN_RECENT=10             # Recent messages to keep (default: 10)
-MEMORY_MAX_NEW_MESSAGES=20          # Max new messages in summary update (default: 20)
-
-# Long-term Memory Settings
-MEMORY_LONG_TERM_ENABLED=true       # Enable long-term extraction (default: true)
-MEMORY_EXTRACTION_THRESHOLD=20      # Min messages before extraction starts (default: 20)
-MEMORY_EXTRACTION_INTERVAL=5        # Run extraction every N messages (default: 5)
-MEMORY_CONFIDENCE_THRESHOLD=0.7     # Minimum confidence to store (default: 0.7)
+# Core Settings
+MEMORY_CONSOLIDATION_THRESHOLD=12
+MEMORY_MIN_CONFIDENCE=0.7
+
+# Retrieval Settings
+MEMORY_ENABLE_VECTOR_SEARCH=true
+MEMORY_ENABLE_BM25=true
+MEMORY_RETRIEVAL_LIMIT=5
+MEMORY_TOKEN_BUDGET=1000
+
+# Summarization Settings
+MEMORY_SUMMARY_ENABLED=true
+MEMORY_MESSAGES_PER_SUMMARY=7
+MEMORY_SUMMARIES_PER_LEVEL=5
+MEMORY_SUMMARY_MAX_DEPTH=3
+MEMORY_SUMMARY_TOKEN_BUDGET=500
+CONTEXT_OVERLAP_USER_MESSAGES=2
+```
+
+---
+
+## Configuration Details
+
+### Core Memory Settings
+
+#### `MEMORY_CONSOLIDATION_THRESHOLD` (default: `12`)
+
+Number of messages to buffer before triggering memory consolidation.
+
+- **Lower** (5-8): Extract facts frequently, more LLM calls, better for fast-paced chats
+- **Higher** (15-30): Let conversations develop, fewer LLM calls, better for long-form discussions
+
+#### `MEMORY_MIN_CONFIDENCE` (default: `0.7`)
+
+Minimum confidence score (0-1) to store extracted facts.
+
+- **Higher** (0.8-0.9): Only store high-certainty facts, good for critical applications
+- **Lower** (0.5-0.6): Capture more nuanced information, risk storing noise
+
+### Retrieval Settings
+
+#### `MEMORY_ENABLE_VECTOR_SEARCH` (default: `true`)
+
+Enable semantic similarity search using embeddings. Recommended to keep enabled.
+
+#### `MEMORY_ENABLE_BM25` (default: `true`)
+
+Enable keyword-based search with stemming. Works best combined with vector search.
+
+#### `MEMORY_RETRIEVAL_LIMIT` (default: `5`)
+
+Maximum number of memories to retrieve per query.
+
+- **Lower** (3): Faster responses, less context
+- **Higher** (10-20): More context, uses more tokens
+
+#### `MEMORY_TOKEN_BUDGET` (default: `1000`)
+
+Maximum tokens for memory context in prompts. Automatically trims to fit budget.
+
+### Summarization Settings
+
+#### `MEMORY_SUMMARY_ENABLED` (default: `true`)
+
+Enable hierarchical conversation summarization. Reduces token usage for long conversations.
+
+#### `MEMORY_MESSAGES_PER_SUMMARY` (default: `7`)
+
+How many messages to include in each Level 1 summary.
+
+- **Lower** (5): More granular summaries
+- **Higher** (15-20): Broader summaries
+
+#### `MEMORY_SUMMARIES_PER_LEVEL` (default: `5`)
+
+How many summaries to accumulate before creating the next level.
+
+- Example: 5 Level-1 summaries → 1 Level-2 summary
+
+#### `MEMORY_SUMMARY_MAX_DEPTH` (default: `3`)
+
+Maximum depth of the summary hierarchy.
+
+- Level 1: Direct message summaries
+- Level 2: Summaries of summaries
+- Level 3+: Higher-level abstractions
+
+#### `MEMORY_SUMMARY_TOKEN_BUDGET` (default: `500`)
+
+Maximum tokens for conversation summaries in context.
+
+#### `CONTEXT_OVERLAP_USER_MESSAGES` (default: `2`)
+
+Number of user messages to show as overlap after summarization for conversation continuity.
+
+---
+
+## Configuration Presets
+
+### High-Frequency Chatbot
+
+```json
+{
+  "MEMORY_CONSOLIDATION_THRESHOLD": "8",
+  "MEMORY_MIN_CONFIDENCE": "0.8",
+  "MEMORY_RETRIEVAL_LIMIT": "3",
+  "MEMORY_MESSAGES_PER_SUMMARY": "5"
+}
 ```
 
-### Manual Memory Storage
+**Why**: Fast extraction, high-quality facts only, minimal context for speed.
 
-Users can explicitly ask the agent to remember information:
+### Long-Form Conversations (Therapy, Coaching, Research)
 
+```json
+{
+  "MEMORY_CONSOLIDATION_THRESHOLD": "20",
+  "MEMORY_MIN_CONFIDENCE": "0.6",
+  "MEMORY_RETRIEVAL_LIMIT": "10",
+  "MEMORY_TOKEN_BUDGET": "2000"
+}
 ```
-User: "Remember that I prefer TypeScript over JavaScript"
-Agent: I've made a note of that in my Preferences memory: "User prefers TypeScript over JavaScript"
 
-User: "Keep in mind I'm working on a startup project"
-Agent: I've made a note of that in my Projects memory: "User is working on a startup project"
+**Why**: Natural conversation flow, capture nuanced details, rich context.
+
+### Critical Applications (Medical, Legal, Financial)
 
-User: "Don't forget I use Python 3.11"
-Agent: I've made a note of that in my Data Sources memory: "User uses Python 3.11"
+```json
+{
+  "MEMORY_MIN_CONFIDENCE": "0.9",
+  "MEMORY_ENABLE_BM25": "true",
+  "MEMORY_ENABLE_VECTOR_SEARCH": "true"
+}
 ```
 
-### Accessing the Memory Service
+**Why**: Only store very certain facts, use both keyword and semantic matching.
 
-```typescript
-import { MemoryService } from '@elizaos/plugin-memory';
+---
 
-// Get the service from runtime
-const memoryService = runtime.getService('memory') as MemoryService;
+## Basic Usage
 
-// Store a long-term memory manually
-await memoryService.storeLongTermMemory({
-  agentId: runtime.agentId,
-  entityId: userId,
-  category: LongTermMemoryCategory.PREFERENCES,
-  content: 'User prefers concise responses',
-  confidence: 0.9,
-  source: 'manual',
-});
+The plugin works automatically once installed. No code changes required.
+
+### How It Works
 
-// Retrieve memories
-const memories = await memoryService.getLongTermMemories(userId);
+1. **Conversation**: Agent buffers messages
+2. **Consolidation**: After threshold (default 12), LLM extracts persistent facts
+3. **Storage**: Facts stored with confidence ≥ 0.7 (configurable)
+4. **Retrieval**: Hybrid search (vector + BM25) with decay scoring
+5. **Context**: Relevant memories added to prompts automatically
 
-// Get session summaries
-const summaries = await memoryService.getSessionSummaries(roomId);
+### Example Flow
+
+```
+User: "I'm a software engineer working on React. I prefer TypeScript."
+→ [12 messages later, consolidation triggers]
+→ Extracted: "User is a software engineer" (confidence: 0.95)
+→ Extracted: "User works with React" (confidence: 0.92)
+→ Extracted: "User prefers TypeScript" (confidence: 0.95)
+
+User: "What's the best state management for my project?"
+→ Retrieves: React + TypeScript preferences
+→ Agent: "For React with TypeScript, I'd recommend Zustand or..."
 ```
 
-## Database Setup
+---
 
-The plugin uses ElizaOS's dynamic migration system. Database tables are automatically created when the plugin is loaded. The plugin defines three tables:
+## Programmatic Access
 
-- **`long_term_memories`**: Stores persistent facts about users
-- **`session_summaries`**: Stores conversation summaries
-- **`memory_access_logs`**: Optional usage tracking for analytics
+```typescript
+import { MemoryService, MemoryType, DecayFunction } from '@elizaos/plugin-memory';
 
-No manual migration is required - the schema is handled automatically by the runtime.
+// Get service
+const memoryService = runtime.getService<MemoryService>('memory');
 
-## Architecture
+// Store memory manually
+await memoryService.storeLongTermMemory({
+  agentId: runtime.agentId,
+  entityId: userId,
+  roomId: roomId,
+  type: MemoryType.SEMANTIC,
+  content: 'User is allergic to peanuts',
+  embeddingContext: '[Health fact]: User is allergic to peanuts',
+  confidence: 1.0,
+  decayRate: 0.0, // Never decays
+  decayFunction: DecayFunction.NONE,
+  source: { authorId: userId },
+  metadata: { category: 'health', critical: true },
+});
 
-### Components
+// Search memories
+const memories = await memoryService.searchLongTermMemories({
+  entityId: userId,
+  query: 'What foods should I avoid?',
+  type: MemoryType.SEMANTIC,
+  limit: 5,
+  minConfidence: 0.7,
+});
 
-#### Services
+// Get all memories by type
+const facts = await memoryService.getLongTermMemories(userId, MemoryType.SEMANTIC, 20);
+```
 
-- **MemoryService**: Core service managing all memory operations
-  - Tracks message counts for summarization triggers
-  - Stores and retrieves long-term memories
-  - Manages session summaries
-  - Provides formatted memory context
+---
 
-#### Evaluators
+## Troubleshooting
 
-- **summarizationEvaluator**: Runs after conversations reach threshold
-  - Generates comprehensive summaries using LLM
-  - Extracts topics and key points
-  - Archives old messages while preserving summaries
-- **longTermExtractionEvaluator**: Periodically analyzes conversations
-  - Identifies facts worth remembering long-term
-  - Categorizes information semantically
-  - Assigns confidence scores
-  - Stores high-confidence memories
+### No memories being extracted
 
-#### Providers
+**Check**:
 
-- **longTermMemoryProvider**: Injects persistent user facts into context
-  - Runs early (position: 50) to establish user context
-  - Formats memories by category
-  - Provides "What I Know About You" context
-- **shortTermMemoryProvider**: Provides conversation summaries
-  - Runs before recentMessages (position: 95)
-  - Includes recent session summaries
-  - Shows topics and message counts
+1. Has threshold been reached? (default: 12 messages)
+2. Is LLM provider configured?
+3. Check logs: `grep "consolidation" agent.log`
 
-#### Actions
+**Debug**:
 
-- **rememberAction**: Handles explicit memory requests
-  - Triggers on keywords like "remember", "keep in mind", etc.
-  - Uses LLM to extract what to remember
-  - Categorizes and stores with confirmation
+```typescript
+const memoryService = runtime.getService<MemoryService>('memory');
+console.log('Config:', memoryService.getConfig());
+```
 
-## How It Works
+### Agent doesn't remember facts
 
-### Short-term Memory Flow
+**Check**:
 
-1. **Tracking**: MemoryService tracks message count per room
-2. **Trigger**: When count reaches threshold (default: 50), summarizationEvaluator activates
-3. **Summarization**: LLM generates comprehensive summary of conversation
-4. **Archival**: Older messages deleted, summary stored, recent messages retained
-5. **Context Injection**: shortTermMemoryProvider injects summaries in future conversations
+1. Database: `SELECT * FROM long_term_memories WHERE isActive = true;`
+2. Confidence: `WHERE confidence >= 0.7`
+3. Decay scores (old memories may have decayed)
 
-### Long-term Memory Flow
+**Debug**:
 
-1. **Warm-up Period**: Extraction waits until 20+ messages (configurable) to ensure meaningful patterns
-2. **Monitoring**: longTermExtractionEvaluator runs periodically (every 5 messages after threshold)
-3. **Analysis**: LLM analyzes conversation for **persistent, important** facts worth remembering
-4. **Strict Filtering**: Only extracts facts that show repetition or strong explicit statements
-5. **Storage**: High-confidence facts stored in long_term_memories table
-6. **Retrieval**: longTermMemoryProvider injects relevant facts in all future conversations
+```typescript
+const memories = await memoryService.getLongTermMemories(userId, undefined, 50, true);
+console.log('Total:', memories.length, 'Active:', memories.filter((m) => m.isActive).length);
+```
 
-**Extraction Criteria**: The evaluator uses strict criteria to avoid storing trivial information:
-- ✅ Extracts: Repeated patterns (3+ occurrences), explicit personal info, stated preferences
-- ❌ Skips: One-time requests, casual interactions, testing behavior, temporary context
+### High token usage
 
-### Manual Memory Flow
+**Solutions**:
 
-1. **Detection**: User says "remember that..." or similar trigger phrase
-2. **Validation**: rememberAction validates the request
-3. **Extraction**: LLM extracts what to remember and categorizes it
-4. **Storage**: Fact stored with 'manual' source and high confidence
-5. **Confirmation**: Agent confirms what was stored
+- Lower `MEMORY_CONSOLIDATION_THRESHOLD` (consolidate more often)
+- Reduce `MEMORY_RETRIEVAL_LIMIT` (fewer memories per query)
+- Increase `MEMORY_MIN_CONFIDENCE` (only high-quality facts)
+- Reduce `MEMORY_TOKEN_BUDGET` and `MEMORY_SUMMARY_TOKEN_BUDGET`
 
-## Performance Optimization
+### Transient requests stored as facts
 
-### Context Reduction
+**Solutions**:
 
-- Without plugin: 1000 messages = ~200,000 tokens
-- With plugin: 1000 messages = ~20 summaries + 10 recent = ~25,000 tokens
-- **Savings**: ~85% reduction in context size
+- Increase `MEMORY_MIN_CONFIDENCE` to 0.8+
+- Use better LLM model (GPT-4, Claude)
+- Customize extraction prompts in `prompts/consolidation.ts`
 
-### Token Efficiency
+---
 
-- Summaries are 1/10th the size of original conversations
-- Long-term memories provide rich context in minimal tokens
-- Recent messages still available for immediate context
+## Database Schema
 
-### Database Optimization
+Tables auto-created on first load via ElizaOS dynamic migrations:
 
-- Indexed queries for fast retrieval
-- Separate tables for different memory types
-- Optional vector search for semantic similarity (requires pgvector)
+### `long_term_memories`
 
-## Best Practices
+Stores episodic, semantic, and procedural memories with:
 
-### For Users
+- Embeddings (pgvector)
+- Confidence scores
+- Decay rates
+- Access tracking
+- Soft deletes
 
-- Use explicit commands: "Remember that...", "Keep in mind...", "Don't forget..."
-- Provide clear, factual information for better storage
-- Verify important memories were stored correctly
+### `conversation_summaries`
 
-### For Developers
+Hierarchical conversation summaries with:
 
-- Adjust thresholds based on your use case
-- Monitor summarization quality with test conversations
-- Use confidence thresholds to filter low-quality extractions
-- Consider enabling vector search for large-scale deployments
+- Multi-level compression
+- Time ranges
+- Token estimates
+- Access tracking
 
-### Configuration Tips
+---
 
-**Short-term Memory:**
-- **High-frequency chatbots**: Lower summarization threshold (10-15 messages)
-- **Long-form conversations**: Higher threshold (20-30 messages)
-- **Adjust retention**: Keep more recent messages for immediate context
+## Architecture Highlights
 
-**Long-term Memory:**
-- **Quick extraction**: Lower extraction threshold to 10-15 messages (may reduce quality)
-- **Quality-focused**: Keep threshold at 20+ messages for better pattern recognition
-- **Critical applications**: Raise confidence threshold to 0.8-0.9
-- **Exploratory use**: Keep default 0.7 confidence threshold
-- **Frequent updates**: Lower extraction interval to run every 3-5 messages
-- **Conservative updates**: Raise interval to 10+ messages
+### Memory Types
 
-## Advanced Features
+- **Episodic**: Time-anchored events ("User asked about X on Tuesday")
+- **Semantic**: Timeless facts ("User is allergic to peanuts")
+- **Procedural**: Skills and patterns ("User prefers git rebase")
 
-### Vector Search (Optional)
+### Hybrid Retrieval
 
-Enable semantic search for memories by:
+Combines:
 
-1. Installing pgvector extension
-2. Setting `MEMORY_VECTOR_SEARCH_ENABLED=true`
-3. Generating embeddings for memories
+- **Vector Search**: Semantic similarity (pgvector HNSW index)
+- **BM25**: Keyword matching with stemming
+- **Decay Scoring**: `Confidence × e^(-λ×t) × (1 + log(accessCount))`
 
-### Memory Analytics
+### Contextual Embeddings
 
-Use the `memory_access_logs` table to:
+Enriches content before embedding:
 
-- Track which memories are most frequently accessed
-- Identify useful vs. unused memories
-- Optimize extraction strategies
+- Raw: "It was terrible"
+- Enriched: "[Movie preference]: The movie Inception was terrible"
 
-### Custom Categories
+### Contradiction Detection
 
-Extend `LongTermMemoryCategory` enum for domain-specific categories:
+- Detects conflicting facts via LLM
+- Soft deletes old memories (marks inactive)
+- Maintains provenance chain via `supersedesId`
 
-```typescript
-export enum CustomMemoryCategory {
-  ...LongTermMemoryCategory,
-  MEDICAL_HISTORY = 'medical_history',
-  FINANCIAL_DATA = 'financial_data',
-}
-```
+---
 
 ## Testing
 
-Run the test suite:
-
 ```bash
 cd packages/plugin-memory
 bun test
 ```
 
-## Troubleshooting
+Tests cover:
 
-### Summaries not generating
+- XML parsing
+- Memory consolidation
+- Decay algorithms
+- Contradiction detection
+- Hybrid retrieval
 
-- Check that message threshold is reached
-- Verify MemoryService is registered
-- Check LLM provider is configured
+---
 
-### Long-term memories not stored
+## Research & References
 
-- Verify `MEMORY_LONG_TERM_ENABLED=true`
-- Check confidence threshold isn't too high
-- Ensure facts are being extracted (check logs)
+Based on comprehensive research documented in [`refactor.md`](./refactor.md):
 
-### High token usage
+- **Cognitive Science**: Ebbinghaus forgetting curve, memory consolidation theory
+- **AI/RAG Systems**: MemGPT, GraphRAG, Contextual Retrieval (Anthropic), HyDE
+- **Agent Architecture**: A-MEM, AgentCore (AWS), MLE-Benchmark
 
-- Lower summarization threshold
-- Reduce number of retained recent messages
-- Limit number of long-term memories retrieved
+Full analysis: [`EVALUATION.md`](./EVALUATION.md)
+
+---
 
 ## License
 
 MIT
 
-## Contributing
+---
+
+## Support
 
-Contributions welcome! Please see the main ElizaOS contributing guide.
+- **Research**: [`refactor.md`](./refactor.md)
+- **Evaluation**: [`EVALUATION.md`](./EVALUATION.md)
+- **Issues**: [GitHub Issues](https://github.com/elizaOS/eliza/issues)
+- **Discord**: [ElizaOS Community](https://discord.gg/elizaos)