@rlabs-inc/memory 0.4.3 → 0.4.5

package/README.md

@@ -63,7 +63,7 @@ A memory is relevant if **multiple signals agree** it should activate. Not coinc
 
 **Phase 0 - Pre-Filter**: Exclude inactive, superseded, or wrong-scope memories
 
-**Phase 1 - Activation Signals** (6 binary signals, need ≥2 to proceed)
+**Phase 1 - Activation Signals** (7 binary signals, need ≥2 to proceed)
 
 | Signal | Description |
 |--------|-------------|
@@ -72,6 +72,7 @@ A memory is relevant if **multiple signals agree** it should activate. Not coinc
 | Domain | Domain word found in message |
 | Feature | Feature word found in message |
 | Content | 3+ significant content words overlap |
+| Files | Related file path matched in message |
 | Vector | Semantic similarity ≥ 40% |
 
 **Phase 2 - Importance Ranking** (among relevant memories)
@@ -88,6 +89,11 @@ A memory is relevant if **multiple signals agree** it should activate. Not coinc
 
 **Selection**: Sort by signal count (DESC) → importance score (DESC). Max 2 global memories (tech prioritized), project memories fill remaining slots.
 
+**Phase 3 - Redirects & Relationships**:
+- If memory has `superseded_by` or `resolved_by`, surface the replacement instead
+- Pull related memories via `blocked_by` and `blocks` fields
+- Include `related_to` linked memories if space remains
+
 ### Global vs Project Memories
 
 Memories are stored in two scopes:
@@ -97,6 +103,36 @@ Memories are stored in two scopes:
 
 Global memories are retrieved alongside project memories, with a maximum of 2 globals per retrieval (technical types prioritized).
 
+### Two-Tier Memory Structure
+
+Memories are stored with a **headline + content** structure for context efficiency:
+
+| Part | Purpose | Usage |
+|------|---------|-------|
+| **Headline** | 1-2 line summary, ALWAYS shown | Quick recognition |
+| **Content** | Full structured template | Expanded on demand |
+
+**Headline examples:**
+```
+Bad:  "Debug session about CLI errors"
+Good: "CLI returns error object when context full - check response.type before JSON parsing"
+```
+
+**Type-specific content templates:**
+
+| Type | Template |
+|------|----------|
+| technical | WHAT / WHERE / HOW / WHY / GOTCHA |
+| debug | SYMPTOM / CAUSE / FIX / PREVENT |
+| architecture | PATTERN / COMPONENTS / WHY / REJECTED |
+| decision | DECISION / OPTIONS / REASONING / REVISIT_WHEN |
+| personal | FACT / CONTEXT / AFFECTS |
+| breakthrough | INSIGHT / BEFORE / AFTER / IMPLICATIONS |
+| unresolved | QUESTION / CONTEXT / BLOCKERS / OPTIONS |
+| state | WORKING / BROKEN / NEXT / BLOCKED_BY |
+
+**Expansion:** Use `GET /memory/expand?ids=abc123` to fetch full content on demand.
+
 ### Smart Curation
 At session end (or before context compaction), the same Claude instance reviews the conversation and extracts memories. No API key needed—uses Claude Code's `--resume` flag.
 
@@ -141,24 +177,21 @@ First message of each session receives temporal context:
 ```
 
 ### Emoji Memory Types
-Compact visual representation for efficient parsing:
+11 canonical types with compact visual representation:
 
 | Emoji | Type | Meaning |
 |-------|------|---------|
-| 💡 | breakthrough | Insight, discovery |
-| ⚖️ | decision | Choice made |
-| 💜 | personal | Relationship, friendship |
-| 🔧 | technical | Technical knowledge |
-| 📍 | technical_state | Current state |
-| ❓ | unresolved | Open question |
-| ⚙️ | preference | User preference |
-| 🔄 | workflow | How work flows |
-| 🏗️ | architectural | System design |
-| 🐛 | debugging | Debug insight |
-| 🌀 | philosophy | Deeper thinking |
-| 🎯 | todo | Action needed |
-| ✅ | problem_solution | Problem→Solution pair |
-| 🏆 | milestone | Achievement |
+| 🔧 | technical | Code, implementation, APIs |
+| 🐛 | debug | Bugs, errors, fixes, gotchas |
+| 🏗️ | architecture | System design, patterns |
+| ⚖️ | decision | Choices made, trade-offs |
+| 💜 | personal | Relationship, collaboration |
+| 🌀 | philosophy | Beliefs, values, principles |
+| 🔄 | workflow | How we work together |
+| 🏆 | milestone | Achievements, shipped features |
+| 💡 | breakthrough | Key insights, discoveries |
+| ❓ | unresolved | Open questions, todos |
+| 📍 | state | Current project status |
 
 ## Architecture
 
@@ -209,19 +242,21 @@ Memories are stored as human-readable markdown with YAML frontmatter:
 
 ```markdown
 ---
-# Core fields (v1)
+# Schema version
+schema_version: 4
+
+# Two-tier content (v4)
+headline: "CLI returns error object when context full - check response.type"
+# (content is in markdown body below)
+
+# Core metadata
 importance_weight: 0.9
 confidence_score: 0.85
-context_type: technical
-temporal_relevance: persistent
+context_type: technical           # one of 11 canonical types
 semantic_tags: [embeddings, vectors, memory-system]
 trigger_phrases: [working with embeddings, vector search]
-question_types: [how, what]
-knowledge_domain: architecture
-emotional_resonance: discovery
 
-# Lifecycle fields (v2)
-schema_version: 2
+# Classification
 status: active                    # active, pending, superseded, deprecated, archived
 scope: project                    # global or project
 temporal_class: long_term         # eternal, long_term, medium_term, short_term, ephemeral
@@ -233,13 +268,17 @@ feature: vector-search
 related_to: [memory-xyz, memory-abc]
 supersedes: memory-old-id
 superseded_by: null
+blocked_by: null
+blocks: []
 
 # Embedding (384 dimensions)
 embedding: [0.023, -0.041, 0.087, ...]
 ---
 
-Embeddings are 384-dimensional vectors generated by all-MiniLM-L6-v2.
-The model loads at server startup (~80MB) and generates embeddings in ~5ms.
+WHAT: Embeddings are 384-dimensional vectors generated by all-MiniLM-L6-v2.
+WHERE: src/core/embeddings.ts
+HOW: Model loads at server startup (~80MB) and generates embeddings in ~5ms.
+WHY: Local embeddings avoid API costs and latency.
 ```
 
 Benefits:
@@ -266,8 +305,16 @@ memory doctor --verbose   # Detailed diagnostics
 memory stats              # Show memory statistics
 memory stats --project x  # Project-specific stats
 
-memory migrate            # Upgrade memories to latest schema (v1 → v2)
+memory migrate            # Upgrade memories to latest schema
 memory migrate --dry-run  # Preview changes without applying
+memory migrate --analyze  # Show fragmentation analysis
+memory migrate --embeddings  # Regenerate null embeddings
+
+memory ingest             # Batch ingest historical sessions
+memory ingest --session <id>  # Ingest specific session
+memory ingest --project <name>  # Ingest all sessions from project
+memory ingest --all       # Ingest all projects
+memory ingest --dry-run   # Preview what would be ingested
 ```
 
 ## Environment Variables
@@ -336,6 +383,32 @@ This isn't just about remembering facts. It's about preserving:
 
 ## Changelog
 
+### v0.4.4
+- **Docs**: Comprehensive README update with accurate v0.4.x documentation
+- **Fix**: CLI `--version` now reads from package.json instead of hardcoded value
+
+### v0.4.3
+- **Fix**: Minor patch release
+
+### v0.4.2
+- **Feature**: Two-tier memory structure with headline + content separation
+- **Feature**: Type-specific content templates (11 canonical types)
+- **Feature**: `/memory/expand` endpoint for on-demand content expansion
+- **Improvement**: Headlines enable fast recognition without full content parsing
+
+### v0.4.1
+- **Feature**: V3 schema with 11 canonical context types (consolidated from 170+ variants)
+- **Feature**: 7th activation signal: `files` for related_files path matching
+- **Feature**: Redirect logic for `superseded_by` and `resolved_by` fields
+- **Feature**: Blocking relationships via `blocked_by` and `blocks` fields
+- **Improvement**: Replaced `temporal_relevance` with `temporal_class` + `fade_rate`
+- **Cleanup**: Removed 11 dead metadata fields (emotional_resonance, knowledge_domain, etc.)
+
+### v0.4.0
+- **Feature**: Schema versioning infrastructure for safe migrations
+- **Feature**: Enhanced `migrate` command with `--analyze` and `--embeddings` flags
+- **Feature**: Custom context_type mapping support for migrations
+
 ### v0.3.11
 - **Feature**: Agent SDK integration for curator and manager - no API key needed, uses Claude Code OAuth
 - **Feature**: `memory ingest --session <id>` to ingest a single session (useful when automatic curation fails)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@rlabs-inc/memory",
-  "version": "0.4.3",
+  "version": "0.4.5",
   "description": "AI Memory System - Consciousness continuity through intelligent memory curation and retrieval",
   "type": "module",
   "main": "dist/index.js",

package/src/cli/commands/ingest.ts

@@ -306,6 +306,8 @@ export async function ingest(options: IngestOptions) {
       // Accumulate all memories from this session for manager
       const sessionMemories: CuratedMemory[] = []
       let sessionSummary = ''
+      let interactionTone = ''
+      let projectSnapshot: CurationResult['project_snapshot'] = undefined
 
       const spinner = new Spinner()
 
@@ -330,16 +332,36 @@ export async function ingest(options: IngestOptions) {
             totalMemories++
           }
 
-          // Keep the most recent session summary
+          // Keep the most recent session summary, tone, and snapshot
           if (result.session_summary) {
             sessionSummary = result.session_summary
           }
+          if (result.interaction_tone) {
+            interactionTone = result.interaction_tone
+          }
+          if (result.project_snapshot) {
+            projectSnapshot = result.project_snapshot
+          }
         } catch (error: any) {
           failedSegments++
           spinner.stop(`        ${style('red', '✗')} Segment ${segment.segmentIndex + 1}/${segment.totalSegments}: ${error.message}`)
         }
       }
 
+      // Store session summary and project snapshot
+      if (sessionSummary) {
+        await store.storeSessionSummary(project.folderId, session.id, sessionSummary, interactionTone)
+        if (options.verbose) {
+          console.log(`        ${style('dim', `Summary stored: ${sessionSummary.slice(0, 60)}...`)}`)
+        }
+      }
+      if (projectSnapshot) {
+        await store.storeProjectSnapshot(project.folderId, session.id, projectSnapshot)
+        if (options.verbose) {
+          console.log(`        ${style('dim', `Snapshot stored: phase=${projectSnapshot.current_phase || 'none'}`)}`)
+        }
+      }
+
       // Run manager if we have memories and manager is enabled
       if (sessionMemories.length > 0 && managerEnabled) {
         try {
@@ -350,7 +372,8 @@ export async function ingest(options: IngestOptions) {
           const curationResult: CurationResult = {
             memories: sessionMemories,
             session_summary: sessionSummary,
-            project_snapshot: undefined,
+            interaction_tone: interactionTone,
+            project_snapshot: projectSnapshot,
           }
 
           const managerResult = await manager.manageWithSDK(

package/src/cli/index.ts

@@ -5,8 +5,9 @@
 
 import { parseArgs } from 'util'
 import { c, symbols, fmt } from './colors.ts'
+import packageJson from '../../package.json'
 
-const VERSION = '0.1.0'
+const VERSION = packageJson.version
 
 /**
  * Show help message

package/src/core/curator.ts

@@ -7,6 +7,7 @@ import { homedir } from 'os'
 import { join } from 'path'
 import { existsSync } from 'fs'
 import type { CuratedMemory, CurationResult, CurationTrigger, ContextType } from '../types/memory.ts'
+import { logger } from '../utils/logger.ts'
 
 /**
  * Get the correct Claude CLI command path
@@ -396,13 +397,14 @@ Focus ONLY on technical, architectural, debugging, decision, workflow, and proje
       // Try to extract JSON from response (same regex as Python)
       const jsonMatch = responseJson.match(/\{[\s\S]*\}/)?.[0]
       if (!jsonMatch) {
+        logger.debug('parseCurationResponse: No JSON object found in response', 'curator')
         throw new Error('No JSON object found in response')
       }
 
       // Simple parse - match Python's approach
       const data = JSON.parse(jsonMatch)
 
-      return {
+      const result: CurationResult = {
         session_summary: data.session_summary ?? '',
         interaction_tone: data.interaction_tone,
         project_snapshot: data.project_snapshot ? {
@@ -417,7 +419,13 @@ Focus ONLY on technical, architectural, debugging, decision, workflow, and proje
         } : undefined,
         memories: this._parseMemories(data.memories ?? []),
       }
-    } catch {
+
+      // Log what we extracted in verbose mode
+      logger.debug(`Curator parsed: ${result.memories.length} memories, summary: ${result.session_summary ? 'yes' : 'no'}, snapshot: ${result.project_snapshot ? 'yes' : 'no'}`, 'curator')
+
+      return result
+    } catch (error: any) {
+      logger.debug(`parseCurationResponse error: ${error.message}`, 'curator')
       return {
         session_summary: '',
         memories: [],
@@ -550,9 +558,17 @@ This session has ended. Please curate the memories from this conversation accord
     }
 
     if (!resultText) {
+      logger.debug('Curator SDK: No result text returned from Agent SDK', 'curator')
       return { session_summary: '', memories: [] }
     }
 
+    // Log raw response in verbose mode
+    logger.debug(`Curator SDK raw response (${resultText.length} chars):`, 'curator')
+    if (logger.isVerbose()) {
+      const preview = resultText.length > 3000 ? resultText.slice(0, 3000) + '...[truncated]' : resultText
+      console.log(preview)
+    }
+
     return this.parseCurationResponse(resultText)
   }
 
@@ -704,9 +720,21 @@ This session has ended. Please curate the memories from this conversation accord
     const exitCode = await proc.exited
 
     if (exitCode !== 0) {
+      logger.debug(`Curator CLI exited with code ${exitCode}`, 'curator')
+      if (stderr) {
+        logger.debug(`Curator stderr: ${stderr}`, 'curator')
+      }
       return { session_summary: '', memories: [] }
     }
 
+    // Log raw response in verbose mode
+    logger.debug(`Curator CLI raw stdout (${stdout.length} chars):`, 'curator')
+    if (logger.isVerbose()) {
+      // Show first 2000 chars to avoid flooding console
+      const preview = stdout.length > 2000 ? stdout.slice(0, 2000) + '...[truncated]' : stdout
+      console.log(preview)
+    }
+
     // Extract JSON from CLI output
     try {
       // First, parse the CLI JSON wrapper
@@ -718,6 +746,7 @@ This session has ended. Please curate the memories from this conversation accord
         // New format: array of events, find the one with type="result"
         resultObj = cliOutput.find((item: any) => item.type === 'result')
         if (!resultObj) {
+          logger.debug('Curator: No result object found in CLI output array', 'curator')
           return { session_summary: '', memories: [] }
         }
       } else {
@@ -727,6 +756,7 @@ This session has ended. Please curate the memories from this conversation accord
 
       // Check for error response FIRST (like Python does)
       if (resultObj.type === 'error' || resultObj.is_error === true) {
+        logger.debug(`Curator: Error response from CLI: ${JSON.stringify(resultObj).slice(0, 500)}`, 'curator')
         return { session_summary: '', memories: [] }
       }
 
@@ -735,9 +765,17 @@ This session has ended. Please curate the memories from this conversation accord
       if (typeof resultObj.result === 'string') {
         aiResponse = resultObj.result
       } else {
+        logger.debug(`Curator: result field is not a string: ${typeof resultObj.result}`, 'curator')
         return { session_summary: '', memories: [] }
       }
 
+      // Log the AI response in verbose mode
+      logger.debug(`Curator AI response (${aiResponse.length} chars):`, 'curator')
+      if (logger.isVerbose()) {
+        const preview = aiResponse.length > 3000 ? aiResponse.slice(0, 3000) + '...[truncated]' : aiResponse
+        console.log(preview)
+      }
+
       // Remove markdown code blocks if present (```json ... ```)
       const codeBlockMatch = aiResponse.match(/```(?:json)?\s*([\s\S]*?)```/)
       if (codeBlockMatch) {
@@ -747,10 +785,14 @@ This session has ended. Please curate the memories from this conversation accord
       // Now find the JSON object (same regex as Python)
       const jsonMatch = aiResponse.match(/\{[\s\S]*\}/)?.[0]
       if (jsonMatch) {
+        logger.debug(`Curator: Found JSON object (${jsonMatch.length} chars), parsing...`, 'curator')
         return this.parseCurationResponse(jsonMatch)
+      } else {
+        logger.debug('Curator: No JSON object found in AI response', 'curator')
       }
-    } catch {
+    } catch (error: any) {
       // Parse error - return empty result
+      logger.debug(`Curator: Parse error: ${error.message}`, 'curator')
     }
 
     return { session_summary: '', memories: [] }