cozo-memory 1.2.6 → 1.2.9

package/README.md

@@ -3,8 +3,12 @@
 [![npm](https://img.shields.io/npm/v/cozo-memory)](https://www.npmjs.com/package/cozo-memory)
 [![Node](https://img.shields.io/node/v/cozo-memory)](https://nodejs.org)
 [![License](https://img.shields.io/badge/license-Apache%202.0-blue)](LICENSE)
+[![MCP Badge](https://lobehub.com/badge/mcp/tobs-code-cozo-memory)](https://lobehub.com/mcp/tobs-code-cozo-memory)
 
-**Local-first memory for Claude & AI agents with hybrid search, Graph-RAG, and time-travel – all in a single binary, no cloud, no Docker.**
+> **Why Cozo Memory?**  
+> LLMs have short-term memory limits. Standard RAG retrieves documents but can't connect facts across time. Cozo Memory gives your AI agent **persistent, structured memory** – it remembers past conversations, infers relationships, detects contradictions, and explores its knowledge graph – fully on your machine, with **optional local LLM integration via Ollama** for intelligent actions (cleanup, reflection, summarization, agentic routing).
+
+**Local-first memory for Claude & AI agents with hybrid search, Graph-RAG, and time-travel – runs entirely on your machine. Optional [Ollama](https://ollama.ai) integration enables LLM-powered actions (cleanup, reflect, summarize, agentic retrieval).**
 
 ## Table of Contents
 
@@ -51,7 +55,7 @@ Now add the server to your MCP client (e.g. Claude Desktop) – see [Integration
 
 ⏳ **Temporal Conflict Resolution** - Automatic detection and resolution of contradictory observations with semantic analysis and audit preservation
 
-🏠 **100% Local** - Embeddings via ONNX/Transformers; no external services, no cloud, complete data ownership
+🏠 **100% Local** - Embeddings via ONNX/Transformers; data stays on your machine. Some advanced features (cleanup, reflect, summarize, agentic search) require an optional [Ollama](https://ollama.ai) service for local LLM inference — but the core search, CRUD, and graph operations work **without any LLM**.
 
 🧠 **Multi-Hop Reasoning** - Logic-aware graph traversal with vector pivots for deep relational reasoning
 
@@ -89,9 +93,34 @@ The core advantage is **Intelligence and Traceability**: By combining an **Agent
 - **RAM: 1.7 GB minimum** (for default bge-m3 model)
   - Model download: ~600 MB
   - Runtime memory: ~1.1 GB
-  - For lower-spec machines, see [Embedding Model Options](#embedding-model-options) below
+  - ⚡ **Too heavy?** Use `EMBEDDING_MODEL=Xenova/all-MiniLM-L6-v2` – only **~400 MB RAM** needed (see [Embedding Model Options](#embedding-model-options))
 - CozoDB native dependency is installed via `cozo-node`
 
+### Optional: Ollama for LLM-powered actions
+
+Some advanced actions use a local LLM via [Ollama](https://ollama.ai) for intelligent
+processing. **The core server works without Ollama** (CRUD, search, graph operations),
+but the following actions require it:
+
+| Action | Purpose |
+|--------|---------|
+| `cleanup` | LLM-backed observation consolidation |
+| `reflect` | Generate insights, detect contradictions |
+| `summarize_communities` | LLM-generated community summaries |
+| `compact` | Session / entity compaction with LLM summarization |
+| `agentic_search` | Query intent classification for auto-routing |
+
+**Setup (if you need these features):**
+```bash
+# 1. Install Ollama from https://ollama.ai
+# 2. Pull a model (e.g. small + fast for dev):
+ollama pull demyagent-4b-i1:Q6_K
+# 3. Ollama runs automatically on http://localhost:11434
+```
+
+If Ollama is not running, the affected actions gracefully fall back to non-LLM behavior
+(where possible) or return a clear error message.
+
 ### Via npm (Easiest)
 
 ```bash
@@ -337,10 +366,10 @@ The interface is reduced to **5 consolidated tools**:
 
 | Tool | Purpose | Key Actions |
 |------|---------|-------------|
-| `mutate_memory` | Write operations | create_entity, update_entity, delete_entity, add_observation, create_relation, transactions, sessions, tasks |
-| `query_memory` | Read operations | search, advancedSearch, context, graph_rag, graph_walking, agentic_search, adaptive_retrieval |
+| `mutate_memory` | Write operations | create_entity, update_entity, delete_entity, add_observation, create_relation, transactions, sessions, tasks, update_observation, batch_delete, manage_tags, batch |
+| `query_memory` | Read operations | search, advancedSearch, context, graph_rag, graph_walking, agentic_search, adaptive_retrieval, list_entities, get_entity_detail, get_session_context, list_sessions |
 | `analyze_graph` | Graph analysis | explore, communities, pagerank, betweenness, hits, shortest_path, semantic_walk |
-| `manage_system` | Maintenance | health, metrics, export, import, cleanup, defrag, reflect, snapshots |
+| `manage_system` | Maintenance | health, metrics, stats, export, import, cleanup, defrag, reflect, snapshots |
 | `edit_user_profile` | User preferences | Edit global user profile with preferences and work style |
 
 > **See [docs/API.md](docs/API.md) for complete API reference with all parameters and examples**
@@ -354,10 +383,12 @@ The interface is reduced to **5 consolidated tools**:
 - This is normal and only happens once
 - Subsequent starts are fast (< 2 seconds)
 
-**Cleanup/Reflect Requires Ollama**
-- If using `cleanup` or `reflect` actions, an Ollama service must be running locally
+**LLM-powered actions require Ollama**
+- The following actions use a local LLM for intelligent processing: `cleanup`, `reflect`, `summarize_communities`, `compact`, `agentic_search`
 - Install Ollama from https://ollama.ai
 - Pull the desired model: `ollama pull demyagent-4b-i1:Q6_K` (or your preferred model)
+- Without Ollama, these actions fall back to non-LLM behavior or return a clear error
+- **Core features (CRUD, search, graph, infer) work without any LLM**
 
 **Windows-Specific**
 - Embeddings are processed on CPU for maximum compatibility
@@ -403,30 +434,6 @@ npm run benchmark    # Runs performance tests
 npm run eval         # Runs evaluation suite
 ```
 
-## Roadmap
-
-### Near-Term (v1.x)
-
-- **GPU Acceleration** - CUDA support for embedding generation (10-50x faster)
-- **Streaming Ingestion** - Real-time data ingestion from logs, APIs, webhooks
-- **Advanced Chunking** - Semantic chunking for `ingest_file` (paragraph-aware splitting)
-- **Query Optimization** - Automatic query plan optimization for complex graph traversals
-- **Additional Export Formats** - Notion, Roam Research, Logseq compatibility
-
-### Mid-Term (v2.x)
-
-- **Multi-Modal Embeddings** - Support for images, audio, code
-- **Distributed Memory** - Sharding and replication for large-scale deployments
-- **Advanced Inference** - Neural-symbolic reasoning, causal inference
-- **Real-Time Sync** - WebSocket-based real-time updates
-- **Web UI** - Browser-based management interface
-
-### Long-Term (v3.x)
-
-- **Federated Learning** - Privacy-preserving collaborative learning
-- **Quantum-Inspired Algorithms** - Advanced graph algorithms
-- **Multi-Agent Coordination** - Shared memory across multiple agents
-
 ## Contributing
 
 Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

package/dist/db-service.test.js

@@ -0,0 +1,313 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+const db_service_1 = require("./db-service");
+describe("DatabaseService", () => {
+    let db;
+    beforeEach(async () => {
+        db = new db_service_1.DatabaseService(":memory:", "sqlite");
+        await db.initialize();
+    });
+    // ── Entity CRUD ──────────────────────────────────────────────
+    describe("Entity CRUD", () => {
+        const sampleEntity = {
+            id: "e1",
+            name: "Test Entity",
+            type: "Test",
+            embedding: [0.1, 0.2, 0.3],
+            name_embedding: [0.4, 0.5, 0.6],
+            metadata: { key: "value" },
+            created_at: 1000,
+        };
+        it("should create and get an entity", async () => {
+            await db.createEntity(sampleEntity);
+            const result = await db.getEntity("e1");
+            expect(result).not.toBeNull();
+            expect(result.name).toBe("Test Entity");
+            expect(result.type).toBe("Test");
+            expect(result.embedding).toEqual([0.1, 0.2, 0.3]);
+            expect(result.metadata).toEqual({ key: "value" });
+        });
+        it("should return null for non-existent entity", async () => {
+            const result = await db.getEntity("non_existent");
+            expect(result).toBeNull();
+        });
+        it("should update an entity partially", async () => {
+            await db.createEntity(sampleEntity);
+            await db.updateEntity("e1", { name: "Updated Entity", metadata: { new: "meta" } });
+            const result = await db.getEntity("e1");
+            expect(result).not.toBeNull();
+            expect(result.name).toBe("Updated Entity");
+            // metadata should be merged
+            expect(result.metadata).toEqual({ key: "value", new: "meta" });
+        });
+        it("should update embedding fields", async () => {
+            await db.createEntity(sampleEntity);
+            const newEmbedding = [0.9, 0.8, 0.7];
+            await db.updateEntity("e1", { embedding: newEmbedding });
+            const result = await db.getEntity("e1");
+            expect(result.embedding).toEqual(newEmbedding);
+            // name_embedding should remain unchanged
+            expect(result.name_embedding).toEqual([0.4, 0.5, 0.6]);
+        });
+        it("should not throw when updating non-existent entity", async () => {
+            await expect(db.updateEntity("ghost", { name: "X" })).resolves.not.toThrow();
+        });
+        it("should delete an entity and its observations", async () => {
+            await db.createEntity(sampleEntity);
+            await db.addObservation({
+                id: "obs1",
+                entity_id: "e1",
+                text: "test",
+                embedding: [],
+                metadata: {},
+                created_at: 1001,
+            });
+            await db.deleteEntity("e1");
+            const entity = await db.getEntity("e1");
+            expect(entity).toBeNull();
+            // Observation should also be deleted
+            const obs = await db.getObservationsForEntity("e1");
+            expect(obs).toHaveLength(0);
+        });
+    });
+    // ── Observations ─────────────────────────────────────────────
+    describe("Observation CRUD", () => {
+        beforeEach(async () => {
+            await db.createEntity({
+                id: "e2",
+                name: "Entity With Obs",
+                type: "Test",
+                embedding: [],
+                name_embedding: [],
+                metadata: {},
+                created_at: 2000,
+            });
+        });
+        it("should add and retrieve observations for an entity", async () => {
+            await db.addObservation({
+                id: "obs1",
+                entity_id: "e2",
+                text: "First observation",
+                embedding: [1.0, 2.0],
+                metadata: { source: "test" },
+                created_at: 2001,
+            });
+            await db.addObservation({
+                id: "obs2",
+                entity_id: "e2",
+                text: "Second observation",
+                embedding: [3.0, 4.0],
+                metadata: {},
+                created_at: 2002,
+            });
+            const obs = await db.getObservationsForEntity("e2");
+            expect(obs).toHaveLength(2);
+            expect(obs[0].text).toBe("First observation");
+            expect(obs[1].text).toBe("Second observation");
+        });
+        it("should return empty array for entity with no observations", async () => {
+            const obs = await db.getObservationsForEntity("e2");
+            expect(obs).toHaveLength(0);
+        });
+        it("should return empty array for non-existent entity", async () => {
+            const obs = await db.getObservationsForEntity("ghost");
+            expect(obs).toHaveLength(0);
+        });
+    });
+    // ── Relationships ────────────────────────────────────────────
+    describe("Relationship CRUD", () => {
+        beforeEach(async () => {
+            for (const id of ["a", "b", "c"]) {
+                await db.createEntity({
+                    id,
+                    name: `Entity ${id}`,
+                    type: "Test",
+                    embedding: [],
+                    name_embedding: [],
+                    metadata: {},
+                    created_at: 3000,
+                });
+            }
+        });
+        it("should create and list all relations", async () => {
+            await db.createRelation({
+                from_id: "a", to_id: "b", relation_type: "knows",
+                strength: 0.8, metadata: {}, created_at: 3001,
+            });
+            await db.createRelation({
+                from_id: "b", to_id: "c", relation_type: "likes",
+                strength: 1.0, metadata: { since: "2024" }, created_at: 3002,
+            });
+            const all = await db.getRelations();
+            expect(all).toHaveLength(2);
+        });
+        it("should filter relations by from_id", async () => {
+            await db.createRelation({
+                from_id: "a", to_id: "b", relation_type: "knows",
+                strength: 0.5, metadata: {}, created_at: 3001,
+            });
+            await db.createRelation({
+                from_id: "a", to_id: "c", relation_type: "knows",
+                strength: 0.3, metadata: {}, created_at: 3002,
+            });
+            await db.createRelation({
+                from_id: "b", to_id: "c", relation_type: "likes",
+                strength: 0.9, metadata: {}, created_at: 3003,
+            });
+            const fromA = await db.getRelations("a");
+            expect(fromA).toHaveLength(2);
+            const fromB = await db.getRelations("b");
+            expect(fromB).toHaveLength(1);
+            expect(fromB[0].to_id).toBe("c");
+        });
+        it("should filter relations by to_id", async () => {
+            await db.createRelation({
+                from_id: "a", to_id: "c", relation_type: "knows",
+                strength: 0.5, metadata: {}, created_at: 3001,
+            });
+            await db.createRelation({
+                from_id: "b", to_id: "c", relation_type: "likes",
+                strength: 0.9, metadata: {}, created_at: 3002,
+            });
+            const toC = await db.getRelations(undefined, "c");
+            expect(toC).toHaveLength(2);
+        });
+        it("should delete entity and cascade its relations", async () => {
+            await db.createRelation({
+                from_id: "a", to_id: "b", relation_type: "knows",
+                strength: 0.5, metadata: {}, created_at: 3001,
+            });
+            await db.deleteEntity("a");
+            const all = await db.getRelations();
+            expect(all).toHaveLength(0);
+        });
+    });
+    // ── Vector Search ────────────────────────────────────────────
+    describe("Vector Search", () => {
+        beforeEach(async () => {
+            const entities = [
+                { id: "vec1", name: "Cat", embedding: [1.0, 0.0, 0.0] },
+                { id: "vec2", name: "Dog", embedding: [0.0, 1.0, 0.0] },
+                { id: "vec3", name: "Fish", embedding: [0.0, 0.0, 1.0] },
+            ];
+            for (const e of entities) {
+                await db.createEntity({
+                    id: e.id, name: e.name, type: "Animal",
+                    embedding: e.embedding, name_embedding: e.embedding,
+                    metadata: {}, created_at: 4000,
+                });
+            }
+        });
+        it("should find closest entity by cosine similarity", async () => {
+            // Query vector closest to [1, 0, 0] → "Cat"
+            const results = await db.vectorSearchEntity([0.9, 0.1, 0.0], 1);
+            expect(results).toHaveLength(1);
+            expect(results[0][0]).toBe("vec1"); // id
+            expect(results[0][4]).toBeGreaterThan(0.9); // score
+        });
+        it("should return correct limit", async () => {
+            const results = await db.vectorSearchEntity([0.5, 0.5, 0.5], 2);
+            expect(results).toHaveLength(2);
+        });
+        it("should handle empty query vector gracefully (returns zero-score results)", async () => {
+            const results = await db.vectorSearchEntity([], 10);
+            // Empty vector produces cosine=0 for all entities, so all are returned with score 0
+            expect(results).toHaveLength(3);
+            expect(results[0][4]).toBe(0);
+        });
+    });
+    // ── Full-Text Search ─────────────────────────────────────────
+    describe("Full-Text Search", () => {
+        beforeEach(async () => {
+            await db.createEntity({
+                id: "fts1", name: "Alice Wonderland", type: "Person",
+                embedding: [], name_embedding: [], metadata: {}, created_at: 5000,
+            });
+            await db.createEntity({
+                id: "fts2", name: "Bob The Builder", type: "Person",
+                embedding: [], name_embedding: [], metadata: {}, created_at: 5001,
+            });
+            await db.addObservation({
+                id: "fobs1", entity_id: "fts1",
+                text: "Alice lives in a wonderland of dreams",
+                embedding: [], metadata: {}, created_at: 5002,
+            });
+        });
+        it("should find entity by name substring", async () => {
+            const results = await db.fullTextSearchEntity("alice");
+            expect(results).toHaveLength(1);
+            expect(results[0][0]).toBe("fts1");
+        });
+        it("should be case-insensitive", async () => {
+            const results = await db.fullTextSearchEntity("BOB");
+            expect(results).toHaveLength(1);
+            expect(results[0][0]).toBe("fts2");
+        });
+        it("should find observation by text substring", async () => {
+            const results = await db.fullTextSearchObservation("wonderland");
+            expect(results).toHaveLength(1);
+            expect(results[0][1]).toBe("fts1");
+        });
+        it("should return empty array for no match", async () => {
+            const results = await db.fullTextSearchEntity("nobody");
+            expect(results).toHaveLength(0);
+        });
+    });
+    // ── Export & Stats ───────────────────────────────────────────
+    describe("Export & Stats", () => {
+        it("should export empty database", async () => {
+            const exported = await db.exportRelations();
+            expect(exported).toHaveProperty("entity");
+            expect(exported).toHaveProperty("observation");
+            expect(exported).toHaveProperty("relationship");
+            expect(exported.entity).toHaveLength(0);
+            expect(exported.observation).toHaveLength(0);
+            expect(exported.relationship).toHaveLength(0);
+        });
+        it("should export correct counts", async () => {
+            await db.createEntity({
+                id: "exp1", name: "Export Test", type: "Test",
+                embedding: [], name_embedding: [], metadata: {}, created_at: 6000,
+            });
+            await db.addObservation({
+                id: "exp_obs1", entity_id: "exp1", text: "Obs",
+                embedding: [], metadata: {}, created_at: 6001,
+            });
+            await db.createRelation({
+                from_id: "exp1", to_id: "exp1", relation_type: "self",
+                strength: 1.0, metadata: {}, created_at: 6002,
+            });
+            const exported = await db.exportRelations();
+            expect(exported.entity).toHaveLength(1);
+            expect(exported.observation).toHaveLength(1);
+            expect(exported.relationship).toHaveLength(1);
+        });
+        it("should return correct stats", async () => {
+            await db.createEntity({
+                id: "stat1", name: "Stats", type: "T",
+                embedding: [], name_embedding: [], metadata: {}, created_at: 7000,
+            });
+            const stats = await db.getStats();
+            expect(stats.entities).toBe(1);
+            expect(stats.observations).toBe(0);
+            expect(stats.relationships).toBe(0);
+        });
+    });
+    // ── Lifecycle ────────────────────────────────────────────────
+    describe("Lifecycle", () => {
+        it("should initialize without error", async () => {
+            await expect(db.initialize()).resolves.not.toThrow();
+        });
+        it("should close without error", async () => {
+            await expect(db.close()).resolves.not.toThrow();
+        });
+        it("should backup and restore without error", async () => {
+            await expect(db.backup("/tmp/test_backup.cozo")).resolves.not.toThrow();
+            await expect(db.restore("/tmp/test_backup.cozo")).resolves.not.toThrow();
+        });
+        it("should run a query without error", async () => {
+            const result = await db.runQuery("SELECT 1");
+            expect(result).toEqual({ rows: [] });
+        });
+    });
+});

package/dist/export-import-service.js

@@ -7,8 +7,10 @@ exports.ExportImportService = void 0;
 const archiver_1 = __importDefault(require("archiver"));
 class ExportImportService {
     dbService;
-    constructor(dbService) {
+    embeddingDim;
+    constructor(dbService, embeddingDim = 1024) {
         this.dbService = dbService;
+        this.embeddingDim = embeddingDim;
     }
     /**
      * Export memory to various formats
@@ -418,7 +420,7 @@ class ExportImportService {
     }
     async createEntityWithId(id, name, type, metadata) {
         const now = Date.now() * 1000;
-        const zeroVec = new Array(1024).fill(0);
+        const zeroVec = new Array(this.embeddingDim).fill(0);
         // Escape strings properly for CozoDB
         const escapedName = name.replace(/"/g, '\\"');
         const escapedType = type.replace(/"/g, '\\"');
@@ -442,14 +444,16 @@ class ExportImportService {
     }
     async createObservationWithId(id, entityId, text, metadata) {
         const now = Date.now() * 1000;
-        const zeroVec = new Array(1024).fill(0);
+        const zeroVec = new Array(this.embeddingDim).fill(0);
         const escapedText = text.replace(/"/g, '\\"').replace(/\n/g, '\\n');
         await this.dbService.run(`
-      ?[id, entity_id, text, embedding, metadata, created_at] <- [[$id, $entity_id, $text, $embedding, $metadata, [${now}, true]]]
-      :insert observation {id, entity_id, text, embedding, metadata, created_at}
+      ?[id, entity_id, session_id, task_id, text, embedding, metadata, created_at] <- [[$id, $entity_id, $session_id, $task_id, $text, $embedding, $metadata, [${now}, true]]]
+      :insert observation {id, entity_id, session_id, task_id, text, embedding, metadata, created_at}
     `, {
             id,
             entity_id: entityId,
+            session_id: "",
+            task_id: "",
             text: escapedText,
             embedding: zeroVec,
             metadata: metadata || {}