cozo-memory 1.0.7 → 1.0.8

package/README.md

@@ -59,7 +59,9 @@ Now you can add the server to your MCP client (e.g. Claude Desktop).
 
 🔍 **Hybrid Search (since v0.7)** - Combines semantic search (HNSW), full-text search (FTS), and graph signals via Reciprocal Rank Fusion (RRF)
 
-🕸️ **Graph-RAG & Graph-Walking (since v1.7)** - Advanced retrieval combining vector seeds with recursive graph traversals using optimized Datalog algorithms
+🕸️ **Graph-RAG & Graph-Walking (v1.7/v2.0)** - Hierarchical retrieval with community detection and summarization; recursive traversals using optimized Datalog algorithms
+
+🧠 **Agentic Retrieval Layer (v2.0)** - Auto-routing engine that analyzes query intent via local LLM to select the optimal search strategy (Vector, Graph, or Community)
 
 🎯 **Multi-Vector Support (since v1.7)** - Dual embeddings per entity: content-embedding for context, name-embedding for identification
 
@@ -71,7 +73,9 @@ Now you can add the server to your MCP client (e.g. Claude Desktop).
 
 📊 **Graph Algorithms (since v1.3/v1.6)** - PageRank, Betweenness Centrality, HITS, Community Detection, Shortest Path
 
-🧹 **Janitor Service** - LLM-backed automatic cleanup with hierarchical summarization
+🏗️ **Hierarchical GraphRAG (v2.0)** - Automatic generation of thematic "Community Summaries" using local LLMs to enable global "Big Picture" reasoning
+
+🧹 **Janitor Service** - LLM-backed automatic cleanup with hierarchical summarization and observation pruning
 
 👤 **User Preference Profiling** - Persistent user preferences with automatic 50% search boost
 
@@ -136,13 +140,14 @@ This server fills the gap in between ("Sweet Spot"): A **local, database-backed
 | Feature | **CozoDB Memory (This Project)** | **Official Reference (`@modelcontextprotocol/server-memory`)** | **mcp-memory-service (Community)** | **Database Adapters (Qdrant/Neo4j)** |
 | :--- | :--- | :--- | :--- | :--- |
 | **Backend** | **CozoDB** (Graph + Vector + Relational) | JSON file (`memory.jsonl`) | SQLite / Cloudflare | Specialized DB (only Vector or Graph) |
-| **Search Logic** | **Hybrid (RRF)**: Vector + Keyword + Graph | Keyword only / Exact Graph Match | Vector + Keyword | Mostly only one dimension |
+| **Search Logic** | **Agentic (Auto-Route)**: Hybrid + Graph + Summaries | Keyword only / Exact Graph Match | Vector + Keyword | Mostly only one dimension |
 | **Inference** | **Yes**: Built-in engine for implicit knowledge | No | No ("Dreaming" is consolidation) | No (Retrieval only) |
+| **Community** | **Yes**: Hierarchical Community Summaries | No | No | Only clustering (no summary) |
 | **Time-Travel** | **Yes**: Queries at any point in time (`Validity`) | No (current state only) | History available, no native DB feature | No |
 | **Maintenance** | **Janitor**: LLM-backed cleanup | Manual | Automatic consolidation | Mostly manual |
 | **Deployment** | **Local** (Node.js + Embedded DB) | Local (Docker/NPX) | Local or Cloud | Often requires external DB server |
 
-The core advantage is **Retrieval Quality and Traceability**: By combining graph algorithms (PageRank, Community Detection) and vector indices (HNSW), context can be provided much more precisely than through pure similarity search.
+The core advantage is **Intelligence and Traceability**: By combining an **Agentic Retrieval Layer** with **Hierarchical GraphRAG**, the system can answer both specific factual questions and broad thematic queries with much higher accuracy than pure vector stores.
 
 ## Performance & Benchmarks
 
@@ -173,6 +178,22 @@ This tool (`src/benchmark.ts`) performs the following tests:
 3.  **Search Performance**: Latency measurement for Hybrid Search vs. Raw Vector Search.
 4.  **RRF Overhead**: Determination of additional computation time for fusion logic.
 
+### Running Evaluation Suite (RAG Quality)
+
+To evaluate the quality and recall of different retrieval strategies (Search vs. Graph-RAG vs. Graph-Walking), use the evaluation suite:
+
+```bash
+npm run eval
+ ```
+ 
+This tool compares strategies using a synthetic dataset and measures **Recall@K**, **MRR**, and **Latency**.
+
+| Method | Recall@10 | Avg Latency | Best For |
+| :--- | :--- | :--- | :--- |
+| **Graph-RAG** | **1.00** | **~32 ms** | Deep relational reasoning |
+| **Graph-Walking** | 1.00 | ~50 ms | Associative path exploration |
+| **Hybrid Search** | 1.00 | ~89 ms | Broad factual retrieval |
+
 ## Architecture
 
 ```mermaid
@@ -488,9 +509,9 @@ The interface is reduced to **4 consolidated tools**. The concrete operation is
 | Tool | Purpose | Key Actions |
 |------|---------|-------------|
 | `mutate_memory` | Write operations | create_entity, update_entity, delete_entity, add_observation, create_relation, run_transaction, add_inference_rule, ingest_file |
-| `query_memory` | Read operations | search, advancedSearch, context, entity_details, history, graph_rag, graph_walking |
+| `query_memory` | Read operations | search, advancedSearch, context, entity_details, history, graph_rag, graph_walking, agentic_search |
 | `analyze_graph` | Graph analysis | explore, communities, pagerank, betweenness, hits, shortest_path, bridge_discovery, semantic_walk, infer_relations |
-| `manage_system` | Maintenance | health, metrics, export_memory, import_memory, snapshot_create, snapshot_list, snapshot_diff, cleanup, reflect, clear_memory |
+| `manage_system` | Maintenance | health, metrics, export_memory, import_memory, snapshot_create, snapshot_list, snapshot_diff, cleanup, reflect, summarize_communities, clear_memory |
 
 ### mutate_memory (Write)
 
@@ -594,6 +615,7 @@ Actions:
 - `history`: `{ entity_id }`
 - `graph_rag`: `{ query, max_depth?, limit?, filters? }` Graph-based reasoning. Finds vector seeds (with inline filtering) first and then expands transitive relationships. Uses recursive Datalog for efficient BFS expansion.
 - `graph_walking`: `{ query, start_entity_id?, max_depth?, limit? }` (v1.7) Recursive semantic graph search. Starts at vector seeds or a specific entity and follows relationships to other semantically relevant entities. Ideal for deeper path exploration.
+- `agentic_search`: `{ query, limit? }` **(New v2.0)**: **Auto-Routing Search**. Uses a local LLM (Ollama) to analyze query intent and automatically routes it to the most appropriate strategy (`vector_search`, `graph_walk`, or `community_summary`).
 - `get_relation_evolution`: `{ from_id, to_id?, since?, until? }` (in `analyze_graph`) Shows temporal development of relationships including time range filter and diff summary.
 
 Important Details:
@@ -687,7 +709,8 @@ Actions:
 - `snapshot_list`: `{}`
 - `snapshot_diff`: `{ snapshot_id_a, snapshot_id_b }`
 - `cleanup`: `{ confirm, older_than_days?, max_observations?, min_entity_degree?, model? }`
-- `reflect`: `{ entity_id?, model? }` Analyzes memory for contradictions and new insights.
+- `summarize_communities`: `{ model?, min_community_size? }` **(New v2.0)**: Triggers the **Hierarchical GraphRAG** pipeline. Recomputes communities, generates thematic summaries via LLM, and stores them as `CommunitySummary` entities.
+- `reflect`: `{ entity_id?, mode?, model? }` Analyzes memory for contradictions and new insights. Supports `summary` (default) and `discovery` (autonomous link refinement) modes.
 - `clear_memory`: `{ confirm }`
 
 Janitor Cleanup Details:
@@ -711,9 +734,14 @@ Entity: Project X
    with a team of 5 developers, currently deployed to staging environment."
 ```
 
-Reflection Service Details:
-- `reflect` analyzes observations of an entity (or top 5 active entities) to find contradictions, patterns, or temporal developments.
-- Results are persisted as new observations with metadata field `{ "kind": "reflection" }` and are retrievable via `context`.
+### Self-Improving Memory Loop (`reflect`)
+The `reflect` service analyzes observations of an entity (or top 5 active entities) to find contradictions, patterns, or temporal developments.
+- **Modes**:
+  - `summary` (default): Generates a textual "Reflective Insight" observation.
+  - `discovery`: Autonomously finds potential relationships using the Inference Engine and validates them via LLM. 
+    - High-confidence links (>0.8) are created automatically.
+    - Medium-confidence links (>0.5) are returned as suggestions.
+- Results are persisted as new observations (for `summary`) or relationships (for `discovery`) with metadata field `{ "kind": "reflection" }` or `{ "source": "reflection" }`.
 - Text is stored with prefix `Reflective Insight: `.
 
 Defaults: `older_than_days=30`, `max_observations=20`, `min_entity_degree=2`, `model="demyagent-4b-i1:Q6_K"`.

package/dist/eval-suite.js

@@ -0,0 +1,136 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+const index_1 = require("./index");
+const perf_hooks_1 = require("perf_hooks");
+const path_1 = __importDefault(require("path"));
+const fs_1 = __importDefault(require("fs"));
+const EVAL_DB_PATH = path_1.default.join(process.cwd(), "eval_db");
+const EVAL_DATASET = [
+    {
+        query: "Where does Alice live?",
+        expectedEntityNames: ["Alice"],
+        type: "factual"
+    },
+    {
+        query: "Which technology does CozoDB use?",
+        expectedEntityNames: ["Datalog"],
+        type: "factual"
+    },
+    {
+        query: "Which technology is used by the project Alice works on?",
+        expectedEntityNames: ["Datalog"],
+        type: "multi-hop"
+    },
+    {
+        query: "What language is used by the project Bob works on?",
+        expectedEntityNames: ["Datalog"],
+        type: "multi-hop"
+    },
+    {
+        query: "Who is Bob's colleague?",
+        expectedEntityNames: ["Alice"],
+        type: "relational"
+    },
+    {
+        query: "What is the background of Bob's coworkers?",
+        expectedEntityNames: ["Alice"],
+        type: "relational"
+    }
+];
+async function setupEvalData(server) {
+    console.log("• Setting up Evaluation Knowledge Graph...");
+    // Create Entities
+    const alice = await server.createEntity({ name: "Alice", type: "Person", metadata: { location: "Berlin" } });
+    const bob = await server.createEntity({ name: "Bob", type: "Person", metadata: { role: "Developer" } });
+    const cozodb = await server.createEntity({ name: "CozoDB", type: "Project", metadata: { category: "Database" } });
+    const datalog = await server.createEntity({ name: "Datalog", type: "Technology", metadata: { type: "Logic Language" } });
+    const aid = alice.id;
+    const bid = bob.id;
+    const cid = cozodb.id;
+    const did = datalog.id;
+    // Add Observations
+    await server.addObservation({ entity_id: aid, text: "Alice lives in Berlin and works as a Senior Engineer." });
+    await server.addObservation({ entity_id: aid, text: "Alice is a leading expert in CozoDB and recursive queries." });
+    await server.addObservation({ entity_id: cid, text: "CozoDB is a powerful graph-vector relational database." });
+    await server.addObservation({ entity_id: cid, text: "CozoDB uses Datalog as its primary query language." });
+    await server.addObservation({ entity_id: bid, text: "Bob is a software developer focused on backend systems." });
+    await server.addObservation({ entity_id: did, text: "Datalog is a declarative logic programming language used for deductive databases." });
+    // Create Relations
+    await server.createRelation({ from_id: aid, to_id: cid, relation_type: "works_on", strength: 1.0 });
+    await server.createRelation({ from_id: bid, to_id: cid, relation_type: "works_on", strength: 1.0 });
+    await server.createRelation({ from_id: bid, to_id: aid, relation_type: "colleague_of", strength: 0.9 });
+    await server.createRelation({ from_id: cid, to_id: did, relation_type: "uses_tech", strength: 1.0 });
+    console.log("  -> Knowledge Graph populated.");
+}
+function calculateRecall(results, expected, k) {
+    const topK = results.slice(0, k);
+    const found = expected.filter(name => topK.some(r => (r.name || "").toLowerCase() === name.toLowerCase()));
+    return found.length / expected.length;
+}
+function calculateMRR(results, expected) {
+    for (let i = 0; i < results.length; i++) {
+        const r = results[i];
+        if (expected.some(name => (r.name || "").toLowerCase() === name.toLowerCase())) {
+            return 1 / (i + 1);
+        }
+    }
+    return 0;
+}
+async function runEvaluation() {
+    console.log("==================================================");
+    console.log("🚀 Cozo Memory Evaluation Suite");
+    console.log("==================================================");
+    if (fs_1.default.existsSync(EVAL_DB_PATH + ".db")) {
+        fs_1.default.unlinkSync(EVAL_DB_PATH + ".db");
+    }
+    const server = new index_1.MemoryServer(EVAL_DB_PATH);
+    await server.embeddingService.embed("warmup");
+    await setupEvalData(server);
+    const methods = [
+        { name: "Hybrid Search", func: (q) => server.hybridSearch.search({ query: q, limit: 10 }) },
+        { name: "Graph-RAG", func: (q) => server.hybridSearch.graphRag({ query: q, limit: 10, graphConstraints: { maxDepth: 2 } }) },
+        { name: "Graph-Walking", func: (q) => server.graph_walking({ query: q, limit: 10, max_depth: 3 }) }
+    ];
+    const summary = [];
+    for (const method of methods) {
+        console.log(`\n• Evaluating ${method.name}...`);
+        let totalRecall3 = 0;
+        let totalRecall10 = 0;
+        let totalMRR = 0;
+        let totalLatency = 0;
+        for (const task of EVAL_DATASET) {
+            const t0 = perf_hooks_1.performance.now();
+            const results = await method.func(task.query);
+            const t1 = perf_hooks_1.performance.now();
+            const r3 = calculateRecall(results, task.expectedEntityNames, 3);
+            const r10 = calculateRecall(results, task.expectedEntityNames, 10);
+            const mrr = calculateMRR(results, task.expectedEntityNames);
+            totalRecall3 += r3;
+            totalRecall10 += r10;
+            totalMRR += mrr;
+            totalLatency += (t1 - t0);
+        }
+        const n = EVAL_DATASET.length;
+        summary.push({
+            Method: method.name,
+            "Recall@3": (totalRecall3 / n).toFixed(3),
+            "Recall@10": (totalRecall10 / n).toFixed(3),
+            MRR: (totalMRR / n).toFixed(3),
+            "Avg Latency": (totalLatency / n).toFixed(2) + "ms"
+        });
+    }
+    console.log("\n==================================================");
+    console.log("Final Evaluation Results:");
+    console.table(summary);
+    console.log("==================================================");
+    // Cleanup
+    // @ts-ignore
+    server.db.close();
+    if (fs_1.default.existsSync(EVAL_DB_PATH + ".db")) {
+        fs_1.default.unlinkSync(EVAL_DB_PATH + ".db");
+    }
+}
+runEvaluation().catch(console.error);

package/dist/hybrid-search.js

@@ -338,5 +338,104 @@ class HybridSearch {
             return this.search(options);
         }
     }
+    async agenticRetrieve(options) {
+        console.error("[HybridSearch] Starting agenticRetrieve with query:", options.query);
+        const { query, routingModel = "demyagent-4b-i1:Q6_K" } = options;
+        const systemPrompt = `You are a Routing Agent for an advanced Memory/RAG system.
+Your job is to analyze the user's query and decide which search strategy is the most appropriate.
+Available strategies:
+- "vector_search": Best for finding specific facts or general semantic queries (e.g., "What ORM is used?", "How does feature X work?", "Welches System nutzt B?").
+- "graph_walk": Best for queries about relations, links, or paths between entities (e.g., "Who works with Bob?", "What depends on Project A?", "Wer arbeitet mit ReactJS?").
+- "community_summary": Best for high-level, overarching questions about the big picture, status, or general themes of entire clusters (e.g., "What is the general tech stack?", "Give me an overview of the legacy project", "Wie ist der generelle Status aller Frontend-Projekte?").
+- "hybrid": Best for complex queries that require both factual retrieval and deep relational context.
+
+You MUST respond with a JSON object in this format: {"strategy": "one_of_the_above"}.
+No markdown, no explanation. Just the JSON.`;
+        let strategy = "vector_search"; // Fallback
+        try {
+            const ollamaMod = await import("ollama");
+            const ollamaClient = ollamaMod?.default ?? ollamaMod;
+            const response = await ollamaClient.chat({
+                model: routingModel,
+                messages: [
+                    { role: "system", content: systemPrompt },
+                    { role: "user", content: query },
+                ],
+                format: "json",
+            });
+            let responseText = response?.message?.content?.trim?.() ?? "";
+            // Clean markdown formatting if Ollama returned it
+            if (responseText.startsWith("```json")) {
+                responseText = responseText.replace(/```json/g, "").replace(/```/g, "").trim();
+            }
+            else if (responseText.startsWith("```")) {
+                responseText = responseText.replace(/```/g, "").trim();
+            }
+            console.error(`[AgenticRetrieve] DEBUG: Raw LLM Output: ${responseText}`);
+            try {
+                const parsed = JSON.parse(responseText);
+                let selected = "";
+                if (Array.isArray(parsed) && parsed.length > 0) {
+                    selected = parsed[0];
+                }
+                else if (typeof parsed === "object" && parsed.strategy) {
+                    selected = parsed.strategy;
+                }
+                else if (typeof parsed === "string") {
+                    selected = parsed;
+                }
+                if (selected) {
+                    const s = selected.toLowerCase();
+                    if (["vector_search", "graph_walk", "community_summary", "hybrid"].includes(s)) {
+                        strategy = s;
+                    }
+                }
+            }
+            catch (e) {
+                console.warn(`[AgenticRetrieve] Failed to parse JSON from LLM: ${responseText}`);
+            }
+        }
+        catch (e) {
+            console.warn(`[AgenticRetrieve] Ollama routing failed: ${e.message}. Using fallback.`);
+        }
+        console.error(`[AgenticRetrieve] Selected Strategy: ${strategy}`);
+        let results = [];
+        // Map strategy to existing search methods
+        switch (strategy) {
+            case "graph_walk":
+            case "hybrid":
+                // Use our advanced Datalog graph-rag for deep relational context
+                results = await this.graphRag(options);
+                break;
+            case "community_summary":
+                // Prioritize CommunitySummary entities
+                results = await this.advancedSearch({
+                    ...options,
+                    filters: {
+                        ...options.filters,
+                        entityTypes: ["CommunitySummary"]
+                    }
+                });
+                // If no community summaries found, fallback to standard search
+                if (results.length === 0) {
+                    console.error(`[AgenticRetrieve] No Community Summaries found, falling back to hybrid search.`);
+                    results = await this.advancedSearch(options);
+                }
+                break;
+            case "vector_search":
+            default:
+                // Standard vector search (which is advancedSearch in v1.7)
+                results = await this.advancedSearch(options);
+                break;
+        }
+        // Annotate results with the routing decision for testing/transparency
+        return results.map(r => ({
+            ...r,
+            metadata: {
+                ...(r.metadata || {}),
+                agentic_routing: strategy
+            }
+        }));
+    }
 }
 exports.HybridSearch = HybridSearch;