@rlabs-inc/memory 0.3.5 → 0.3.7

package/README.md

@@ -55,25 +55,76 @@ Uses `all-MiniLM-L6-v2` for 384-dimensional embeddings. Memories are retrieved b
 Sub-microsecond vector search via fsdb
 ```
 
-### 10-Dimensional Scoring
-Memories are scored across multiple dimensions:
-
-| Dimension | Weight | Description |
-|-----------|--------|-------------|
-| Vector similarity | 10% | Semantic match to your message |
-| Trigger phrases | 10% | Activation patterns set by curator |
-| Tag matching | 5% | Keyword overlap |
-| Question types | 5% | "How", "why", "what" alignment |
-| Importance | 20% | Curator's assessment |
-| Temporal | 10% | Persistent vs session vs temporary |
-| Context | 10% | Technical, personal, debugging... |
-| Confidence | 10% | Curator's certainty |
-| Emotion | 10% | Joy, frustration, discovery... |
-| Problem-solution | 5% | Bug fix patterns |
+### Activation Signal Retrieval Algorithm
+
+The retrieval system uses an activation signal approach. Philosophy: **quality over quantity, silence over noise**.
+
+A memory is relevant if **multiple signals agree** it should activate. Not coincidence - intentionally crafted metadata matching intentional queries.
+
+**Phase 0 - Pre-Filter**: Exclude inactive, superseded, or wrong-scope memories
+
+**Phase 1 - Activation Signals** (6 binary signals, need ≥2 to proceed)
+
+| Signal | Description |
+|--------|-------------|
+| Trigger | Trigger phrase matched (≥50% word match) |
+| Tags | 2+ semantic tags found in message |
+| Domain | Domain word found in message |
+| Feature | Feature word found in message |
+| Content | 3+ significant content words overlap |
+| Vector | Semantic similarity ≥ 40% |
+
+**Phase 2 - Importance Ranking** (among relevant memories)
+
+| Bonus | Amount | Condition |
+|-------|--------|-----------|
+| Base | 0-1 | `importance_weight` from curator |
+| Signal boost | +0.2 / +0.1 | 4+ or 3 signals fired |
+| Awaiting | +0.15 / +0.1 | `awaiting_implementation` / `awaiting_decision` |
+| Temporal | +0.1 / +0.05 | `eternal` / `long_term` temporal class |
+| Context match | +0.1 | User intent matches memory type |
+| Problem/solution | +0.1 | User has problem words + memory is pair |
+| Confidence penalty | -0.1 | `confidence_score` < 0.5 |
+
+**Selection**: Sort by signal count (DESC) → importance score (DESC). Max 2 global memories (tech prioritized), project memories fill remaining slots.
+
+### Global vs Project Memories
+
+Memories are stored in two scopes:
+
+- **Global**: Personal memories, philosophy, preferences, cross-project breakthroughs - shared across ALL projects
+- **Project**: Technical details, debugging insights, project-specific decisions - isolated per project
+
+Global memories are retrieved alongside project memories, with a maximum of 2 globals per retrieval (technical types prioritized).
 
 ### Smart Curation
 At session end (or before context compaction), the same Claude instance reviews the conversation and extracts memories. No API key needed—uses Claude Code's `--resume` flag.
 
+### Memory Manager Agent
+
+After curation, an autonomous manager agent organizes the memory store:
+
+- **Supersedes** outdated memories when new information replaces old
+- **Resolves** unresolved/todo memories when solutions emerge
+- **Links** related memories via relationship fields
+- **Updates** personal primer with new personal context
+
+The manager runs in a sandboxed environment with access only to memory storage directories.
+
+### Personal Memories Control
+
+Personal memory extraction can be disabled for shared or professional environments:
+
+```bash
+# Via environment variable
+MEMORY_PERSONAL_ENABLED=0 memory serve
+
+# Or in configuration
+personalMemoriesEnabled: false
+```
+
+When disabled, the curator skips personal/relationship context extraction.
+
 ### Session Primer
 First message of each session receives temporal context:
 
@@ -128,9 +179,14 @@ Compact visual representation for efficient parsing:
 │  ┌─────────────┐  ┌──────────────┐  ┌───────────────┐  │
 │  │   Engine    │  │  Embeddings  │  │   Curator     │  │
 │  │  (context)  │  │  (MiniLM)    │  │ (CLI resume)  │  │
-│  └──────┬──────┘  └──────────────┘  └───────────────┘  │
-│         │                                               │
-│         ▼                                               │
+│  └──────┬──────┘  └──────────────┘  └───────┬───────┘  │
+│         │                                    │          │
+│         │                                    ▼          │
+│         │                           ┌───────────────┐  │
+│         │                           │   Manager     │  │
+│         │                           │ (CLI sandbox) │  │
+│         │                           └───────┬───────┘  │
+│         ▼                                   │          │
 │  ┌─────────────────────────────────────────────────┐   │
 │  │                    fsdb                          │   │
 │  │         (markdown files + parallel arrays)       │   │
@@ -139,9 +195,12 @@ Compact visual representation for efficient parsing:
                            │
                            ▼
               ~/.local/share/memory/
-                  ├── memories/     # Curated memories as .md
-                  ├── sessions/     # Session metadata
-                  └── summaries/    # Session summaries
+                  ├── global/
+                  │   └── memories/   # Personal, philosophy (shared)
+                  └── {project-id}/
+                      ├── memories/   # Project-specific memories
+                      ├── sessions/   # Session metadata
+                      └── summaries/  # Session summaries
 ```
 
 ## Storage Format
@@ -150,17 +209,33 @@ Memories are stored as human-readable markdown with YAML frontmatter:
 
 ```markdown
 ---
+# Core fields (v1)
 importance_weight: 0.9
+confidence_score: 0.85
 context_type: technical
 temporal_relevance: persistent
-semantic_tags:
-  - embeddings
-  - vectors
-  - memory-system
-trigger_phrases:
-  - working with embeddings
-  - vector search
-embedding: [0.023, -0.041, 0.087, ...]  # 384 dimensions
+semantic_tags: [embeddings, vectors, memory-system]
+trigger_phrases: [working with embeddings, vector search]
+question_types: [how, what]
+knowledge_domain: architecture
+emotional_resonance: discovery
+
+# Lifecycle fields (v2)
+schema_version: 2
+status: active                    # active, pending, superseded, deprecated, archived
+scope: project                    # global or project
+temporal_class: long_term         # eternal, long_term, medium_term, short_term, ephemeral
+fade_rate: 0.02                   # decay per session (0 = no decay)
+domain: embeddings
+feature: vector-search
+
+# Relationships
+related_to: [memory-xyz, memory-abc]
+supersedes: memory-old-id
+superseded_by: null
+
+# Embedding (384 dimensions)
+embedding: [0.023, -0.041, 0.087, ...]
 ---
 
 Embeddings are 384-dimensional vectors generated by all-MiniLM-L6-v2.
@@ -179,15 +254,20 @@ Benefits:
 memory serve              # Start memory server (default port 8765)
 memory serve --port 9000  # Custom port
 memory serve --verbose    # Detailed logging
+memory serve --quiet      # Minimal output
 
 memory install            # Set up Claude Code hooks
 memory install --force    # Overwrite existing hooks
+memory install --gemini   # Install for Gemini CLI instead
 
 memory doctor             # Health check
 memory doctor --verbose   # Detailed diagnostics
 
 memory stats              # Show memory statistics
 memory stats --project x  # Project-specific stats
+
+memory migrate            # Upgrade memories to latest schema (v1 → v2)
+memory migrate --dry-run  # Preview changes without applying
 ```
 
 ## Environment Variables
@@ -197,6 +277,13 @@ MEMORY_PORT=8765              # Server port
 MEMORY_HOST=localhost         # Server host
 MEMORY_STORAGE_MODE=central   # 'central' or 'local'
 MEMORY_API_URL=http://localhost:8765  # For hooks
+
+# Feature toggles
+MEMORY_MANAGER_ENABLED=1      # Enable/disable memory manager agent (default: 1)
+MEMORY_PERSONAL_ENABLED=1     # Enable/disable personal memory extraction (default: 1)
+
+# Optional: for SDK curation mode (alternative to CLI mode)
+ANTHROPIC_API_KEY=sk-...      # Uses SDK instead of CLI for curation
 ```
 
 ## How It Works
@@ -206,21 +293,30 @@ When you start Claude Code, the `SessionStart` hook injects a primer with:
 - Time since last session
 - Previous session summary
 - Project status
+- Personal primer (relationship context, injected every session)
 - Current datetime for temporal awareness
 
 ### 2. Every Message
 The `UserPromptSubmit` hook:
 1. Embeds your message (~5ms)
-2. Searches stored memories using 10-dimensional scoring
-3. Filters through gatekeeper (relevance > 5%, total > 30%)
-4. Injects top matches into your message context
+2. Searches both global and project memories
+3. Applies two-phase scoring with dual gatekeepers
+4. Injects top matches (max 5 by default, max 2 global)
 
 ### 3. Session End
 The `PreCompact` or `SessionEnd` hook triggers curation:
 1. Resumes the same Claude session via CLI
 2. Claude reviews the conversation
-3. Extracts important memories with rich metadata
+3. Extracts important memories with rich metadata (v2 lifecycle fields)
 4. Stores as markdown files with embeddings
+5. Determines scope: global (personal/philosophy) vs project (technical)
+
+### 4. Memory Management (Async)
+After curation completes, the manager agent:
+1. Scans for outdated memories to supersede
+2. Resolves unresolved/todo items when solutions appear
+3. Links related memories together
+4. Updates the personal primer with new relationship context
 
 ## Requirements