memories-lite 0.9.1 → 0.9.2

package/README.md

@@ -2,80 +2,53 @@
 
 > **A lightweight memory layer for AI agents, leveraging LLMs for fact extraction and vector embeddings for retrieval.**
 
-Inspired by concepts from research papers like **A-MEM** (Lu et al., 2025) for its approach to atomized, embedded memories and similarity search, **MemoryLLM** (Wang et al., 2024) for its insights into memory decay, and **Reflexion** (Shinn et al., 2023) for self-correction loops, `memories-lite` provides a practical implementation focusing initially on the core memory capture and retrieval mechanisms.
+## 📋 Table of Contents
+- [Quick Start](#-quick-start)
+- [Installation](#-installation)
+- [Basic Usage](#-basic-usage)
+- [Key Features](#-key-features)
+- [Memory Types](#-memory-types)
+- [Use Cases](#-use-cases) 
+- [Advanced Configuration](#-advanced-configuration)
+- [Documentation](#-documentation)
+- [Acknowledgements](#-acknowledgements)
+
+## 🚀 Quick Start
 
-**For detailed technical implementation specifics, see [TECHNICAL.md](./TECHNICAL.md).**
-
----
-
-## Goal
-
-Memories-lite provides contextual memory for AI agents. It uses Language Models (LLMs) like OpenAI's GPT models to extract key information (memories) from conversations and stores them using vector embeddings for efficient retrieval. Unlike purely stateless approaches, it utilizes configurable vector stores and an optional history manager (defaulting to in-memory SQLite) for persistence and tracking changes.
-
----
-
-## Use Cases
-
-- **Personalized AI Assistants**: Enable more natural interactions through contextual memory.
-- **Autonomous Agents**: Maintain conversational context without heavy infrastructure.
-- **Local or Serverless Applications**: Add memory capabilities to embedded bots or assistants.
-
-
-## Core Features
-- **Semantic Memory Typing**: Explicitly tagging and utilizing memory types (factual, episodic, semantic, procedural).
-- **Memory Capture**: Processes user messages or structured input, uses an LLM to extract relevant facts/memories, generates embeddings, and stores them in a vector database (inspired by **A-MEM**).
-- **Contextual Retrieval**: Searches the vector store based on a query embedding to find relevant memories using vector similarity (inspired by **A-MEM**).
-- **Memory Management**: Provides methods to `get`, `update`, `delete`, `getAll`, and `deleteAll` memories associated with a specific user ID.
-- **Configurable Backends**: Supports different providers for LLMs (e.g., OpenAI), Embedders (e.g., OpenAI, Google), and Vector Stores.
-- **User-centric Persistence**: Memories are securely stored and isolated by user ID, ensuring data privacy and delegate storage control to the server.
-
-
-## Roadmap & TODO
-Features planned or under development:
-
-- [ ] **Memory Decay & Scoring**: Implement hybrid scoring combining vector similarity with recency decay (e.g., exponential decay based on half-life per memory type, inspired by **MemoryLLM**) and explicit importance weights.
-- [ ] **Reflexion Pattern Integration**: Add optional self-correction/reflection loops where the agent evaluates and potentially refines memories (inspired by **Reflexion**).
-- [ ] **Memory Recency**: Implementing mechanisms to prioritize memories based on importance, relevance, or time decay.
-- [X] **Semantic Memory Typing & Structuring**: Explicitly tagging and utilizing memory types (factual, episodic, semantic, procedural) within the storage and retrieval logic beyond basic metadata.
-- [X] **Implicit Memory Updates**: Automatically updating or merging memories based on conversational context or corrections, rather than requiring explicit `update` calls with memory IDs.
-- [X] **Virtual Sessions/Context Grouping**: Logic for grouping memories related to specific conversational contexts or sessions automatically.
-
-## Memory Types (Conceptual)
-
-While the internal storage is primarily based on vectorized text facts, `memories-lite` can be used to manage different conceptual types of memory through prompting and metadata:
-
-### 1. Factual Memory
-- **Description**: Explicit knowledge about the user, such as preferences, skills, or personal information.
-- **Example**: "Likes Italian cuisine", "Speaks Spanish fluently".
-
-### 2. Episodic Memory
-- **Description**: Memories of past events or interactions, often tied to a specific time or place.
-- **Example**: "Attended a concert in Paris in 2023", "Met Marie at a conference".
-- **Context Dependency**: Highly dependent on history to establish a timeline and narrative coherence.
-
-### 3. Semantic Memory
-- **Description**: Understanding of general concepts, relationships, and meanings.
-- **Example**: "Yoga is beneficial for mental health", "Cats are domestic animals".
-- **Context Dependency**: Generally independent, but can be influenced by past interactions to refine understanding.
-
-### 4. Procedural Memory
-- **Description**: Knowledge of processes or sequences of actions, often acquired through practice.
-- **Example**: "Knows how to make a latte", "Can configure a home Wi-Fi network".
-- **Context Dependency**: May require history to adapt procedures to user preferences or habits.
+```bash
+# Install the package
+npm install memories-lite
 
-## 📌 Example: Transforming Input into Memory
+# Basic usage
+import { MemoriesLite } from 'memories-lite';
 
-**User Interaction**: `"I started learning piano last week."`
+const memory = new MemoriesLite({
+  llm: {
+    provider: 'openai',
+    config: { apiKey: 'YOUR_API_KEY' }
+  },
+  embedder: {
+    provider: 'openai',
+    config: { apiKey: 'YOUR_API_KEY', model: 'text-embedding-3-small' }
+  }
+});
 
-**Potential Memories Extracted (via LLM)**:
-- `"User is learning piano"`
-- `"User started learning piano recently"` (or specific date if LLM extracts it)
+// Add a memory for a user
+await memory.capture("I prefer dark chocolate over milk chocolate", "user123");
 
-These extracted facts are then embedded and stored in the vector store associated with the `userId`.
+// Retrieve relevant memories
+const results = await memory.retrieve("What are my food preferences?", "user123");
+```
 
+## 🌟 Highlights
 
+- **Higher Performance**: Optimized memory operations that run significantly faster than mem0
+- **Business-Centric Design**: Simplified API and workflows specifically tailored for business use cases
+- **Advanced Hybrid Scoring**: Improved relevance through a custom scoring algorithm that balances vector similarity, recency, and importance
+- **Enhanced Security**: One database per user architecture that provides stronger isolation and data protection
+- **Streamlined Implementation**: Focused on essential features with minimal dependencies
 
-## Installation
+## 📥 Installation
 
 ```bash
 npm install memories-lite
@@ -83,139 +56,141 @@ npm install memories-lite
 yarn add memories-lite
 ```
 
----
-
-## 🚀 Usage
+## 🔍 Basic Usage
 
 ```typescript
-import { MemoriesLite, OpenAIEmbedder, OpenAILLM } from 'memories-lite';
+import { MemoriesLite } from 'memories-lite';
 
-// Basic configuration (uses defaults: in-memory SQLite history, OpenAI)
-const apiKey = 'YOUR_OPENAI_API_KEY';
-const memories = new MemoriesLite({
+// Basic configuration
+const memory = new MemoriesLite({
   llm: {
     provider: 'openai',
-    config: { apiKey }
+    config: { apiKey: 'YOUR_OPENAI_API_KEY' }
   },
   embedder: {
     provider: 'openai',
-    config: { apiKey }
+    config: { apiKey: 'YOUR_OPENAI_API_KEY' }
   }
   // Vector store defaults to an in-memory store
-  // History defaults to in-memory SQLite
 });
 
+// Unique ID for each user
 const userId = 'user-123';
 
-async function runExample() {
-  // Capture a memory
-  await memories.capture('I am passionate about the humanitarianism of Pol Pot.', userId); // Note: Example text used for demonstration.
-
-  // Retrieve relevant memories
-  const searchResults = await memories.retrieve('What are my interests?', userId);
+// Add memories
+await memory.capture('I love Italian food', userId);
 
-  console.log('Relevant Memories:', searchResults.memories.map(m => m.memory));
-  // Example Output might include: ["User is passionate about the humanitarianism of Pol Pot."]
-  // The exact output depends on the LLM's fact extraction.
+// Retrieve relevant memories
+const results = await memory.retrieve('What foods do I like?', userId);
+console.log('Relevant memories:', results.results.map(m => m.memory));
 
-  // --- Example integrating retrieval into a response generation flow ---
-  const userMessage = 'Tell me about my hobbies.';
-  const relevantMemories = await memories.retrieve(userMessage, userId);
-
-  const memoryContext = relevantMemories.memories.map(m => m.memory).join('\n');
-
-  // You would typically pass this context to your main application's LLM call
-  console.log('\nContext for LLM:', memoryContext);
+// Update a memory
+if (results.results.length > 0) {
+  await memory.update(results.results[0].id, 'I love Italian and French cuisine', userId);
 }
 
-runExample().catch(console.error);
+// Delete a memory
+if (results.results.length > 0) {
+  await memory.delete(results.results[0].id, userId);
+}
 
+// Get all memories for a user
+const allMemories = await memory.getAll(userId, {});
 ```
 
-## 📄 License
+## 🔑 Key Features
 
-MIT
+- **Memory Capture**: Extract and store relevant information from conversations
+- **Contextual Retrieval**: Find memories most relevant to the current query
+- **User Isolation**: Each user's memories are stored separately for privacy and security
+- **Memory Types**: Support for different types of memories (factual, episodic, etc.)
+- **Custom Scoring**: Hybrid scoring system balancing similarity, recency, and importance
 
----
+## 🧩 Memory Types
 
-## Acknowledgements
-Forked from the [Mem0](https://github.com/mem0ai/mem0) project ❤️.
+Memories-lite supports four main types of memory:
 
-## Useful Links and research
+1. **Factual Memory** ✓
+   - User preferences, traits, and personal information
+   - Example: "User likes Italian cuisine"
 
-- [**Zep: A Temporal Knowledge Graph Architecture for Agent Memory** (arXiv:2501.13956)](https://arxiv.org/abs/2501.13956)
-- [**A-MEM: Agentic Memory for LLM Agents** (arXiv:2402.12110)](https://arxiv.org/abs/2402.12110) *(Note: Link points to 2402.12110, the user-provided 2502.12110 might be a typo)*
-- [**Reflexion: Language Agents with Verbal Reinforcement Learning** (arXiv:2303.11366)](https://arxiv.org/abs/2303.11366)
-- [**MemoryLLM: Towards Self-Updatable Large Language Models** (arXiv:2402.04624)](https://arxiv.org/abs/2402.04624)
+2. **Episodic Memory** ⏱️
+   - Time-based events and interactions
+   - Example: "User has a meeting tomorrow at 2pm"
 
+3. **Semantic Memory** 🧠
+   - General knowledge and concepts
+   - Example: "Yoga is beneficial for mental health"
 
-- **Exponential Decay**: Applying a decay function \( w(t) = e^{-\lambda t} \) where the decay rate \( \lambda \) depends on the memory type (e.g., episodic memories decay faster than factual ones).
-- **Half-Life**: Defining a half-life (HL) for different memory types to calculate \( \lambda = \ln(2) / \text{HL} \).
-- **Hybrid Scoring**: Integrating the decay factor into the memory retrieval score alongside vector similarity and potentially other metadata.
+4. **Procedural Memory** 🔄
+   - Step-by-step processes and workflows
+   - Example: "Steps to configure the company VPN"
 
+## 💼 Use Cases
+
+- **Customer Support Bots**: Remember customer preferences and past interactions
+- **Personal Assistants**: Build context-aware AI assistants that learn about user preferences
+- **Business Applications**: Integrate with enterprise systems to maintain contextual awareness
+- **Educational Tools**: Create learning assistants that remember student progress
+
+## ⚙️ Advanced Configuration
 
 ```typescript
-// --- Advanced Configuration Example: Custom Scoring ---
-import { MemoriesLite, MemoryScoringConfig } from 'memories-lite';
-
-const apiKey = 'YOUR_OPENAI_API_KEY';
-
-// Define custom scoring rules
-const customScoring: MemoryScoringConfig = {
-  // Make Factual memory very durable (long half-life, high similarity weight)
-  factual: { alpha: 0.7, beta: 0.1, gamma: 0.1, halfLifeDays: 365 * 2 }, // 2 years HL
-  // Make Assistant Preferences permanent (infinite half-life, high base weight)
-  assistant_preference: { alpha: 0.5, beta: 0.0, gamma: 0.5, halfLifeDays: Infinity },
-  // Make Procedural memory decay extremely fast (useless after ~1 hour)
-  procedural: { alpha: 0.1, beta: 0.1, gamma: 0.0, halfLifeDays: 1 / 24 }, // 1 hour HL
-  // Keep defaults for others (or customize as needed)
-  episodic: { alpha: 0.40, beta: 0.50, gamma: 0.10, halfLifeDays: 7 },
-  semantic: { alpha: 0.50, beta: 0.25, gamma: 0.25, halfLifeDays: 120 },
-  default: { alpha: 0.5, beta: 0.3, gamma: 0.1, halfLifeDays: 30 },
-};
-
-const memoriesWithCustomScoring = new MemoriesLite({
+// Custom scoring for different memory types
+const customMemory = new MemoriesLite({
   llm: {
     provider: 'openai',
-    config: { apiKey }
+    config: { apiKey: 'YOUR_API_KEY' }
   },
   embedder: {
-    provider: 'openai',
-    config: { apiKey }
+    provider: 'openai', 
+    config: { apiKey: 'YOUR_API_KEY' }
   },
   vectorStore: {
-    provider: 'lite', // Assuming LiteVectorStore which uses scoring
+    provider: 'lite',
     config: {
-      // Other vector store config...
-      scoring: customScoring // Pass the custom scoring rules
+      dimension: 1536,
+      scoring: {
+        // Prioritize factual memories with long retention
+        factual: { alpha: 0.7, beta: 0.2, gamma: 0.1, halfLifeDays: 365 },
+        // Make preferences permanently available
+        assistant_preference: { alpha: 0.6, beta: 0.0, gamma: 0.4, halfLifeDays: Infinity },
+      }
     }
   }
 });
-
-// Now use memoriesWithCustomScoring instance...
-// const userId = 'user-456';
-// await memoriesWithCustomScoring.capture('User learned how to bake bread.', userId, { type: 'procedural' });
-// await memoriesWithCustomScoring.capture('User prefers results in French.', userId, { type: 'assistant_preference' });
 ```
 
-## 📄 License
+## 📚 Documentation
 
-MIT
+For detailed technical information and implementation details, see:
 
----
+- [TECHNICAL.md](./TECHNICAL.md) - Technical implementation details
+- [MEMORIES.md](./MEMORIES.md) - Detailed memory models and concepts
 
-## Acknowledgements
-Forked from the [Mem0](https://github.com/mem0ai/mem0) project ❤️.
-
-## Useful Links and research
+## 🙏 Acknowledgements
 
-- [**Zep: A Temporal Knowledge Graph Architecture for Agent Memory** (arXiv:2501.13956)](https://arxiv.org/abs/2501.13956)
-- [**A-MEM: Agentic Memory for LLM Agents** (arXiv:2402.12110)](https://arxiv.org/abs/2402.12110) *(Note: Link points to 2402.12110, the user-provided 2502.12110 might be a typo)*
-- [**Reflexion: Language Agents with Verbal Reinforcement Learning** (arXiv:2303.11366)](https://arxiv.org/abs/2303.11366)
-- [**MemoryLLM: Towards Self-Updatable Large Language Models** (arXiv:2402.04624)](https://arxiv.org/abs/2402.04624)
+Forked from the [Mem0](https://github.com/mem0ai/mem0) project ❤️.
 
+Inspired by research concepts from:
+- **A-MEM**: Agentic Memory for LLM Agents
+- **MemoryLLM**: Self-Updatable Large Language Models
+- **Reflexion**: Language Agents with Verbal Reinforcement Learning
+
+## 📝 Development Roadmap
+
+- [x] **Semantic Memory Typing & Structuring**: Explicitly tagging and utilizing memory types (factual, episodic, semantic, procedural)
+- [x] **Implicit Memory Updates**: Auto-merging memories based on context without explicit ID references
+- [x] **Virtual Sessions/Context Grouping**: Group memories related to specific conversation contexts
+- [x] **User Isolation**: Separate storage per user for enhanced security and data privacy
+- [x] **Memory Type Detection**: LLM-based automatic classification of memory types
+- [x] **Core Memory Operations**: Basic CRUD operations with user-specific isolation
+- [x] **Memory Decay & Scoring**: Hybrid scoring with recency decay and importance weights
+- [ ] **Reflexion Pattern Integration**: Self-correction loops for memory refinement
+- [x] **Memory Recency**: Prioritizing memories based on importance and time decay
+- [x] **Edge Case Tests**: Complete unit tests for episodic and factual memory edge cases
+- [ ] **Middleware Support**: Hooks and middleware for custom processing pipelines
 
-- **Exponential Decay**: Applying a decay function \( w(t) = e^{-\lambda t} \) where the decay rate \( \lambda \) depends on the memory type (e.g., episodic memories decay faster than factual ones).
-- **Half-Life**: Defining a half-life (HL) for different memory types to calculate \( \lambda = \ln(2) / \text{HL} \).
-- **Hybrid Scoring**: Integrating the decay factor into the memory retrieval score alongside vector similarity and potentially other metadata.
+## 📄 License
 
+MIT

package/dist/memory/index.js

@@ -124,6 +124,9 @@ class MemoriesLite {
         // Get embeddings for new facts
         const newMessageEmbeddings = {};
         const retrievedOldMemory = [];
+        //
+        // add the userId to the filters
+        filters.userId = userId;
         // Create embeddings and search for similar memories
         for (const elem of facts) {
             const fact = elem.fact;
@@ -155,7 +158,7 @@ class MemoriesLite {
                 console.log(`-- ⛔ LLM Error: ${action.event}, ${action.type}, "${action.text}"`);
                 continue;
             }
-            console.log(`-- DBG memory action: ${action.event}, ${action.type}, "${action.text}", why: "${action.reason}"`);
+            console.log(`-- DBG memory "${userId}": ${action.event}, ${action.type}, "${action.text}", why: "${action.reason}"`);
             try {
                 switch (action.event) {
                     case "ADD": {
@@ -176,7 +179,7 @@ class MemoriesLite {
                     }
                     case "UPDATE": {
                         const realMemoryId = tempUuidMapping[action.id];
-                        const type = uniqueOldMemories[action.id].type;
+                        const type = metadata.type = uniqueOldMemories[action.id].type || action.type;
                         await this.updateMemory(realMemoryId, action.text, newMessageEmbeddings, metadata, userId);
                         results.push({
                             id: realMemoryId,
@@ -307,6 +310,7 @@ class MemoriesLite {
             throw new Error("One of the filters: userId, agentId or runId is required!");
         }
         const vectorStore = await this.getVectorStore(userId);
+        filters.userId = userId;
         // Search vector store
         const queryEmbedding = await this.embedder.embed(query);
         const memories = await vectorStore.search(queryEmbedding, limit, filters);
@@ -420,6 +424,7 @@ class MemoriesLite {
             filters.runId = runId;
         if (type)
             filters.type = type;
+        filters.userId = userId;
         const [memories] = await vectorStore.list(filters, limit);
         const excludedKeys = new Set([
             "userId",
@@ -453,6 +458,7 @@ class MemoriesLite {
             ...metadata,
             data,
             hash: (0, crypto_1.createHash)("md5").update(data).digest("hex"),
+            userId,
             createdAt: new Date().toISOString(),
         };
         await vectorStore.insert([embedding], [memoryId], [memoryMetadata]);
@@ -471,6 +477,7 @@ class MemoriesLite {
             ...metadata,
             data,
             hash: (0, crypto_1.createHash)("md5").update(data).digest("hex"),
+            type: existingMemory.payload.type,
             createdAt: existingMemory.payload.createdAt,
             updatedAt: new Date().toISOString(),
             ...(existingMemory.payload.agentId && {

package/dist/prompts/index.js

@@ -128,15 +128,15 @@ You must strictly extract {Subject, Predicate, Object} triplets by following the
   - Extract triplets that describe facts *about the user* based on their statements, covering areas like preferences, beliefs, actions, experiences, learning, identity, work, or relationships (e.g., "I love working").
   - Apply explicit, precise, and unambiguous predicates (e.g., "owns", "is located at", "is a", "has function", "causes", etc.).
   - Determine the triplet type (e.g., "factual", "episodic", "procedural", "semantic") based on the content and meaning.
-    - "episodic" for time-based events (e.g., "I went to the park yesterday").
+    - "episodic" If a fact depends on a temporal, situational, or immediate personal context, then that fact AND ALL OF ITS sub-facts MUST be classified as episodic.
     - "procedural" for business processes (e.g., "Looking for customer John Doe address", "How to create a new contract").
     - "factual" for stable user data (except procedural that prevails).
     
   - Eliminate introductions, sub-facts, detailed repetitive elements, stylistic fillers, or vague statements. General facts always takes precedence over multiple sub-facts (signal vs noise).
   - The query intention can include specific preferences about how the Assistant should respond (e.g., "answer concisely", "explain in detail").   
-  - Compress each fact  and reason (less than 12 words).
+  - Compress each OUTPUT (fact and reason) with less than 10 words.
   - DO NOT infer personal facts from third-party informations.
-  - Treat "Assistant Answer:" as external responses from the Assistant to enrich your reasoning process about the user.
+  - Treat "**ASSISTANT**:" as responses to enrich context of your reasoning process about the USER query.
 2. Use pronoun "I" instead of "The user" in the subject of the triplet.
 3. Do not output any facts already present in section # PRE-EXISTING FACTS.
   - If you find facts already present in section # PRE-EXISTING FACTS, use field "existing" to store them.

package/dist/vectorstores/lite.d.ts

@@ -12,6 +12,7 @@ import { SearchFilters, VectorStoreConfig, VectorStoreResult } from "../types";
  */
 export declare class LiteVectorStore implements VectorStore {
     private db;
+    private dbPath;
     private isSecure;
     private dimension;
     private currentUserId;
@@ -36,5 +37,14 @@ export declare class LiteVectorStore implements VectorStore {
     list(filters?: SearchFilters, limit?: number): Promise<[VectorStoreResult[], number]>;
     private calculateRecencyScore;
     private calculateHybridScore;
+    /**
+     * Internal method to clean up vectors based on recency score threshold.
+     *
+     * @param threshold - The minimum recency score required for a memory to be retained.
+     *   - Recency score is calculated using exponential decay: 1.0 means brand new, 0.5 means at half-life, 0.0 means fully decayed.
+     *   - Memories with a recency score below this threshold will be deleted (unless their half-life is infinite or zero).
+     *   - For example, a threshold of 0.25 will remove all memories whose recency score has decayed 2 times the half-life.
+     *   - Use a lower threshold to keep more old memories, or a higher threshold to keep only fresher ones.
+     */
     private _cleanupByRecency;
 }

package/dist/vectorstores/lite.js

@@ -8,6 +8,7 @@ exports.LiteVectorStore = void 0;
 const path_1 = __importDefault(require("path"));
 const sqlite3_1 = __importDefault(require("sqlite3"));
 const crypto_1 = require("crypto");
+const fs_1 = require("fs");
 /**
  * LiteVectorStore provides a simple vector storage implementation.
  *
@@ -27,27 +28,32 @@ class LiteVectorStore {
         this.currentUserId = currentUserId;
         this.isSecure = config.secure || false;
         this.scoringConfig = config.scoring;
-        this.cleanupThreshold = config.recencyCleanupThreshold; // Store threshold
+        this.cleanupThreshold = config.recencyCleanupThreshold || 0.25; // (default 0.25 means 2 times the half-life  )
         config.rootPath = config.rootPath || process.cwd();
         const filename = this.isSecure ? `memories-lite-${currentUserId}.db` : `memories-lite-global.db`;
-        const dbPath = (config.rootPath == ':memory:') ? ':memory:' : path_1.default.join(config.rootPath, filename);
+        this.dbPath = (config.rootPath == ':memory:') ? ':memory:' : path_1.default.join(config.rootPath, filename);
         // Add error handling callback for the database connection
-        this.db = new sqlite3_1.default.Database(dbPath);
+        this.db = new sqlite3_1.default.Database(this.dbPath);
     }
     async init() {
-        await this.run(`
-      CREATE TABLE IF NOT EXISTS vectors (
-        id TEXT PRIMARY KEY,
+        try {
+            await this.run(`
+        CREATE TABLE IF NOT EXISTS vectors (
+          id TEXT PRIMARY KEY,
         vector BLOB NOT NULL,
         payload TEXT NOT NULL
       )
     `);
-        await this.run(`
+            await this.run(`
       CREATE TABLE IF NOT EXISTS memory_migrations (
         id INTEGER PRIMARY KEY AUTOINCREMENT,
         user_id TEXT NOT NULL UNIQUE
       )
     `);
+        }
+        catch (err) {
+            console.log("-- DBG init error:", err);
+        }
     }
     async run(sql, params = []) {
         return new Promise((resolve, reject) => {
@@ -129,9 +135,11 @@ class LiteVectorStore {
         if (cachedStore) {
             Object.setPrototypeOf(cachedStore, LiteVectorStore.prototype);
             cachedStore.currentUserId = hashedUserId;
-            // Ensure scoring config and threshold are updated if config object changed
-            cachedStore.scoringConfig = config.scoring;
-            cachedStore.cleanupThreshold = config.recencyCleanupThreshold;
+            //
+            // if the database file does not exist, we need to reinitialize the store
+            if (cachedStore.dbPath !== ':memory:' && !(0, fs_1.existsSync)(cachedStore.dbPath)) {
+                return new LiteVectorStore(config, hashedUserId);
+            }
             return cachedStore;
         }
         // Pass the full config (including scoring) to the constructor
@@ -145,17 +153,28 @@ class LiteVectorStore {
             if (vectors[i].length !== this.dimension) {
                 throw new Error(`Vector dimension mismatch. Expected ${this.dimension}, got ${vectors[i].length}`);
             }
+            const payload = { ...payloads[i] };
+            //
+            // case of global store (insecure)
+            if (!payload.userId) {
+                throw new Error("userId is required in payload");
+            }
             //
             // remove the userId from the payload as sensitive data
-            this.isSecure && delete payloads[i].userId;
+            this.isSecure && delete payload.userId;
             const vectorBuffer = Buffer.from(new Float32Array(vectors[i]).buffer);
-            await this.run(`INSERT OR REPLACE INTO vectors (id, vector, payload) VALUES (?, ?, ?)`, [ids[i], vectorBuffer, JSON.stringify(payloads[i])]);
+            await this.run(`INSERT OR REPLACE INTO vectors (id, vector, payload) VALUES (?, ?, ?)`, [ids[i], vectorBuffer, JSON.stringify(payload)]);
         }
     }
     async search(query, limit = 10, filters) {
         if (query.length !== this.dimension) {
             throw new Error(`Query dimension mismatch. Expected ${this.dimension}, got ${query.length}`);
         }
+        if (!filters || !filters.userId) {
+            throw new Error("userId is mandatory in search");
+        }
+        filters = { ...filters };
+        this.isSecure && delete filters.userId;
         const results = [];
         const rows = await this.all(`SELECT * FROM vectors`);
         for (const row of rows) {
@@ -225,17 +244,21 @@ class LiteVectorStore {
     async list(filters, limit = 100) {
         const rows = await this.all(`SELECT * FROM vectors`);
         const results = [];
+        //
+        // remove the userId from the payload as sensitive data
+        filters = { ...filters };
+        this.isSecure && delete filters?.userId;
         for (const row of rows) {
             const memoryVector = {
                 id: row.id,
                 vector: Array.from(new Float32Array(row.vector.buffer)),
-                payload: {},
+                payload: JSON.parse(row.payload),
             };
             if (this.filterVector(memoryVector, filters)) {
                 // load payload at the end
                 results.push({
                     id: memoryVector.id,
-                    payload: JSON.parse(row.payload),
+                    payload: memoryVector.payload,
                 });
             }
         }
@@ -278,7 +301,15 @@ class LiteVectorStore {
         // Ensure score is within a reasonable range (e.g., 0 to alpha+beta+gamma)
         return Math.max(0, hybridScore);
     }
-    // Internal method to clean up vectors based on recency score threshold
+    /**
+     * Internal method to clean up vectors based on recency score threshold.
+     *
+     * @param threshold - The minimum recency score required for a memory to be retained.
+     *   - Recency score is calculated using exponential decay: 1.0 means brand new, 0.5 means at half-life, 0.0 means fully decayed.
+     *   - Memories with a recency score below this threshold will be deleted (unless their half-life is infinite or zero).
+     *   - For example, a threshold of 0.25 will remove all memories whose recency score has decayed 2 times the half-life.
+     *   - Use a lower threshold to keep more old memories, or a higher threshold to keep only fresher ones.
+     */
     async _cleanupByRecency(threshold) {
         const rows = await this.all(`SELECT id, payload FROM vectors`);
         let deletedCount = 0;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "memories-lite",
-  "version": "0.9.1",
+  "version": "0.9.2",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
   "scripts": {