@zabaca/lattice 0.3.4 → 1.0.0

package/README.md

@@ -24,30 +24,31 @@ That's it. Two commands to build a knowledge base.
 | Feature | Lattice | Other GraphRAG Tools |
 |---------|---------|---------------------|
 | **LLM for extraction** | Your Claude Code subscription | Separate API key + costs |
-| **Setup time** | 5 minutes | 30+ minutes |
-| **Containers** | 1 (FalkorDB) | 2-3 (DB + vector + graph) |
+| **Setup time** | 2 minutes | 30+ minutes |
+| **Database** | Embedded DuckDB (zero config) | Docker containers required |
+| **External dependencies** | None | 2-3 (DB + vector + graph) |
 | **API keys needed** | 1 (Voyage AI for embeddings) | 2-3 (LLM + embedding + rerank) |
 | **Workflow** | `/research` → `/graph-sync` | Custom scripts |
 
 ---
 
-## Quick Start (5 Minutes)
+## Quick Start (2 Minutes)
 
 ### What You Need
 
 - **Claude Code** (you probably already have it)
-- **Docker** (for FalkorDB)
 - **Voyage AI API key** ([get one here](https://www.voyageai.com/) - embeddings only, ~$0.01/1M tokens)
 
-### 1. Install & Start
+### 1. Install
 
 ```bash
-bun add -g @zabaca/lattice                    # Install CLI
-docker run -d -p 6379:6379 falkordb/falkordb  # Start database
-export VOYAGE_API_KEY=your-key-here           # Set API key
-lattice init --global                         # Install Claude Code commands
+bun add -g @zabaca/lattice          # Install CLI
+export VOYAGE_API_KEY=your-key-here  # Set API key
+lattice init --global                # Install Claude Code commands
 ```
 
+That's it. No Docker. No containers. DuckDB is embedded.
+
 ### 2. Start Researching
 
 ```bash
@@ -67,7 +68,7 @@ The `/research` command will:
 The `/graph-sync` command will:
 - Detect all new/changed documents
 - Extract entities using Claude Code (your subscription)
-- Sync to FalkorDB for semantic search
+- Sync to DuckDB for semantic search
 
 ---
 
@@ -152,6 +153,24 @@ Semantic search across the knowledge graph.
 
 ```bash
 lattice search "query"          # Search all entity types
+lattice search "query" -l Tool  # Filter by label
+```
+
+### `lattice sql`
+
+Execute raw SQL queries against DuckDB.
+
+```bash
+lattice sql "SELECT * FROM nodes LIMIT 10"
+lattice sql "SELECT label, COUNT(*) FROM nodes GROUP BY label"
+```
+
+### `lattice rels`
+
+Show relationships for a node.
+
+```bash
+lattice rels "TypeScript"       # Show all relationships for an entity
 ```
 
 ### `lattice validate`
@@ -182,15 +201,24 @@ lattice ontology                # Show entity types and relationship types
 | Variable | Description | Default |
 |----------|-------------|---------|
 | `VOYAGE_API_KEY` | Voyage AI API key for embeddings | *required* |
-| `FALKORDB_HOST` | FalkorDB server hostname | `localhost` |
-| `FALKORDB_PORT` | FalkorDB server port | `6379` |
+| `DUCKDB_PATH` | Path to DuckDB database file | `./.lattice.duckdb` |
+| `EMBEDDING_DIMENSIONS` | Embedding vector dimensions | `512` |
+
+### Database Location
+
+Lattice stores its knowledge graph in a single `.lattice.duckdb` file in your docs directory. This file contains:
+- All extracted entities (nodes)
+- Relationships between entities
+- Vector embeddings for semantic search
+
+You can back up, copy, or version control this file like any other.
 
 <details>
 <summary><b>How It Works (Technical Details)</b></summary>
 
 ### Entity Extraction
 
-When you run `/graph-sync`, Claude Code extracts entities from your documents and writes them to YAML frontmatter. The Lattice CLI then syncs this to FalkorDB.
+When you run `/graph-sync`, Claude Code extracts entities from your documents and writes them to YAML frontmatter. The Lattice CLI then syncs this to DuckDB.
 
 ```yaml
 ---
@@ -208,57 +236,35 @@ relationships:
 
 You don't need to write this manually — Claude Code handles it automatically.
 
-</details>
-
----
-
-## Infrastructure
-
-<details>
-<summary><b>Docker Compose (Alternative Setup)</b></summary>
-
-If you prefer Docker Compose over a single `docker run` command:
-
-```yaml
-version: '3.8'
-
-services:
-  falkordb:
-    image: falkordb/falkordb:latest
-    ports:
-      - "6379:6379"
-    volumes:
-      - falkordb-data:/data
-    restart: unless-stopped
-
-volumes:
-  falkordb-data:
+### Database Schema
+
+Lattice uses two main tables:
+
+```sql
+-- Nodes (entities)
+CREATE TABLE nodes (
+    label VARCHAR NOT NULL,      -- Entity type: Document, Technology, etc.
+    name VARCHAR NOT NULL,       -- Unique identifier
+    properties JSON,             -- Additional metadata
+    embedding FLOAT[512],        -- Vector for semantic search
+    PRIMARY KEY(label, name)
+);
+
+-- Relationships
+CREATE TABLE relationships (
+    source_label VARCHAR NOT NULL,
+    source_name VARCHAR NOT NULL,
+    relation_type VARCHAR NOT NULL,
+    target_label VARCHAR NOT NULL,
+    target_name VARCHAR NOT NULL,
+    properties JSON,
+    PRIMARY KEY(source_label, source_name, relation_type, target_label, target_name)
+);
 ```
 
-```bash
-docker-compose up -d
-```
+### Vector Search
 
-</details>
-
-<details>
-<summary><b>Kubernetes (k3s)</b></summary>
-
-For production deployments, use the provided k3s manifests:
-
-```bash
-kubectl apply -f infra/k3s/namespace.yaml
-kubectl apply -f infra/k3s/pv.yaml
-kubectl apply -f infra/k3s/pvc.yaml
-kubectl apply -f infra/k3s/deployment.yaml
-kubectl apply -f infra/k3s/service.yaml
-
-# Optional: NodePort for external access
-kubectl apply -f infra/k3s/nodeport-service.yaml
-
-# Optional: Ingress
-kubectl apply -f infra/k3s/ingress.yaml
-```
+Lattice uses DuckDB's VSS extension for HNSW-based vector similarity search with cosine distance.
 
 </details>
 
@@ -273,7 +279,6 @@ kubectl apply -f infra/k3s/ingress.yaml
 
 - Node.js >= 18.0.0
 - Bun (recommended) or npm
-- Docker (for FalkorDB)
 
 ### Setup
 
@@ -282,7 +287,6 @@ git clone https://github.com/Zabaca/lattice.git
 cd lattice
 bun install
 cp .env.example .env
-docker-compose -f infra/docker-compose.yaml up -d
 ```
 
 ### Running Locally
@@ -329,4 +333,4 @@ MIT License - see [LICENSE](LICENSE) for details.
 
 ---
 
-Built with [FalkorDB](https://www.falkordb.com/), [Voyage AI](https://www.voyageai.com/), and [Claude Code](https://claude.ai/code)
+Built with [DuckDB](https://duckdb.org/), [Voyage AI](https://www.voyageai.com/), and [Claude Code](https://claude.ai/code)

package/dist/main.js

@@ -140,10 +140,9 @@ import { glob } from "glob";
 
 // src/schemas/config.schemas.ts
 import { z } from "zod";
-var FalkorDBConfigSchema = z.object({
-  host: z.string().min(1).default("localhost"),
-  port: z.coerce.number().int().positive().default(6379),
-  graphName: z.string().min(1).default("research_knowledge")
+var DuckDBConfigSchema = z.object({
+  dbPath: z.string().optional(),
+  embeddingDimensions: z.coerce.number().int().positive().default(512)
 });
 var EmbeddingConfigSchema = z.object({
   provider: z.enum(["openai", "voyage", "nomic", "mock"]).default("voyage"),
@@ -790,82 +789,124 @@ EmbeddingService = __legacyDecorateClassTS([
 ], EmbeddingService);
 
 // src/graph/graph.service.ts
+import { DuckDBInstance } from "@duckdb/node-api";
 import { Injectable as Injectable6, Logger as Logger3 } from "@nestjs/common";
 import { ConfigService as ConfigService2 } from "@nestjs/config";
-import Redis from "ioredis";
 class GraphService {
   configService;
   logger = new Logger3(GraphService.name);
-  redis = null;
-  config;
+  instance = null;
+  connection = null;
+  dbPath;
   connecting = null;
+  vectorIndexes = new Set;
+  embeddingDimensions;
   constructor(configService) {
     this.configService = configService;
-    this.config = FalkorDBConfigSchema.parse({
-      host: this.configService.get("FALKORDB_HOST"),
-      port: this.configService.get("FALKORDB_PORT"),
-      graphName: this.configService.get("GRAPH_NAME")
-    });
+    this.dbPath = this.configService.get("DUCKDB_PATH") || "./.lattice.duckdb";
+    this.embeddingDimensions = this.configService.get("EMBEDDING_DIMENSIONS") || 512;
   }
   async onModuleDestroy() {
     await this.disconnect();
   }
   async ensureConnected() {
-    if (this.redis) {
-      return this.redis;
+    if (this.connection) {
+      return this.connection;
     }
     if (this.connecting) {
       await this.connecting;
-      return this.redis;
+      if (!this.connection) {
+        throw new Error("Connection failed to establish");
+      }
+      return this.connection;
     }
     this.connecting = this.connect();
     await this.connecting;
     this.connecting = null;
-    return this.redis;
+    if (!this.connection) {
+      throw new Error("Connection failed to establish");
+    }
+    return this.connection;
   }
   async connect() {
     try {
-      this.redis = new Redis({
-        host: this.config.host,
-        port: this.config.port,
-        maxRetriesPerRequest: 3,
-        retryStrategy: (times) => {
-          if (times > 3) {
-            return null;
-          }
-          const delay = Math.min(times * 50, 2000);
-          return delay;
-        },
-        lazyConnect: true
-      });
-      this.redis.on("error", (err) => {
-        this.logger.debug(`Redis connection error: ${err.message}`);
+      this.instance = await DuckDBInstance.create(this.dbPath, {
+        allow_unsigned_extensions: "true"
       });
-      await this.redis.ping();
-      this.logger.log(`Connected to FalkorDB at ${this.config.host}:${this.config.port}`);
+      this.connection = await this.instance.connect();
+      await this.connection.run("INSTALL vss; LOAD vss;");
+      await this.connection.run("SET hnsw_enable_experimental_persistence = true;");
+      try {
+        await this.connection.run("SET custom_extension_repository = 'http://duckpgq.s3.eu-north-1.amazonaws.com';");
+        await this.connection.run("FORCE INSTALL 'duckpgq';");
+        await this.connection.run("LOAD 'duckpgq';");
+        this.logger.log("DuckPGQ extension loaded successfully");
+      } catch (e) {
+        this.logger.warn(`DuckPGQ extension not available: ${e instanceof Error ? e.message : String(e)}`);
+      }
+      await this.initializeSchema();
+      this.logger.log(`Connected to DuckDB at ${this.dbPath}`);
     } catch (error) {
-      this.redis = null;
-      this.logger.error(`Failed to connect to FalkorDB: ${error instanceof Error ? error.message : String(error)}`);
+      this.connection = null;
+      this.instance = null;
+      this.logger.error(`Failed to connect to DuckDB: ${error instanceof Error ? error.message : String(error)}`);
       throw error;
     }
   }
   async disconnect() {
-    if (this.redis) {
-      await this.redis.quit();
-      this.logger.log("Disconnected from FalkorDB");
-    }
-  }
-  async query(cypher, _params) {
+    if (this.connection) {
+      this.connection.closeSync();
+      this.connection = null;
+      this.logger.log("Disconnected from DuckDB");
+    }
+    if (this.instance) {
+      this.instance = null;
+    }
+  }
+  async initializeSchema() {
+    if (!this.connection) {
+      throw new Error("Cannot initialize schema: not connected");
+    }
+    const conn = this.connection;
+    await conn.run(`
+			CREATE TABLE IF NOT EXISTS nodes (
+				label VARCHAR NOT NULL,
+				name VARCHAR NOT NULL,
+				properties JSON,
+				embedding FLOAT[${this.embeddingDimensions}],
+				created_at TIMESTAMP DEFAULT NOW(),
+				updated_at TIMESTAMP DEFAULT NOW(),
+				PRIMARY KEY(label, name)
+			)
+		`);
+    await conn.run(`
+			CREATE TABLE IF NOT EXISTS relationships (
+				source_label VARCHAR NOT NULL,
+				source_name VARCHAR NOT NULL,
+				relation_type VARCHAR NOT NULL,
+				target_label VARCHAR NOT NULL,
+				target_name VARCHAR NOT NULL,
+				properties JSON,
+				created_at TIMESTAMP DEFAULT NOW(),
+				PRIMARY KEY(source_label, source_name, relation_type, target_label, target_name)
+			)
+		`);
+    await conn.run("CREATE INDEX IF NOT EXISTS idx_nodes_label ON nodes(label)");
+    await conn.run("CREATE INDEX IF NOT EXISTS idx_nodes_label_name ON nodes(label, name)");
+    await conn.run("CREATE INDEX IF NOT EXISTS idx_rels_source ON relationships(source_label, source_name)");
+    await conn.run("CREATE INDEX IF NOT EXISTS idx_rels_target ON relationships(target_label, target_name)");
+  }
+  async query(sql, _params) {
     try {
-      const redis = await this.ensureConnected();
-      const result = await redis.call("GRAPH.QUERY", this.config.graphName, cypher);
-      const resultArray = Array.isArray(result) ? result : [];
+      const conn = await this.ensureConnected();
+      const reader = await conn.runAndReadAll(sql);
+      const rows = reader.getRows();
       return {
-        resultSet: Array.isArray(resultArray[1]) ? resultArray[1] : [],
-        stats: this.parseStats(result)
+        resultSet: rows,
+        stats: undefined
       };
     } catch (error) {
-      this.logger.error(`Cypher query failed: ${error instanceof Error ? error.message : String(error)}`);
+      this.logger.error(`Query failed: ${error instanceof Error ? error.message : String(error)}`);
       throw error;
     }
   }
@@ -875,18 +916,15 @@ class GraphService {
       if (!name) {
         throw new Error("Node must have a 'name' property");
       }
-      const escapedName = this.escapeCypher(String(name));
-      const escapedLabel = this.escapeCypher(label);
-      const propAssignments = Object.entries({
-        name,
-        ...otherProps
-      }).map(([key, value]) => {
-        const escapedKey = this.escapeCypher(key);
-        const escapedValue = this.escapeCypherValue(value);
-        return `n.\`${escapedKey}\` = ${escapedValue}`;
-      }).join(", ");
-      const cypher = `MERGE (n:\`${escapedLabel}\` { name: '${escapedName}' }) ` + `SET ${propAssignments}`;
-      await this.query(cypher);
+      const conn = await this.ensureConnected();
+      const propsJson = JSON.stringify(otherProps);
+      await conn.run(`
+				INSERT INTO nodes (label, name, properties)
+				VALUES ('${this.escape(String(label))}', '${this.escape(String(name))}', '${this.escape(propsJson)}')
+				ON CONFLICT (label, name) DO UPDATE SET
+					properties = EXCLUDED.properties,
+					updated_at = NOW()
+			`);
     } catch (error) {
       this.logger.error(`Failed to upsert node: ${error instanceof Error ? error.message : String(error)}`);
       throw error;
@@ -894,21 +932,31 @@ class GraphService {
   }
   async upsertRelationship(sourceLabel, sourceName, relation, targetLabel, targetName, properties) {
     try {
-      const escapedSourceLabel = this.escapeCypher(sourceLabel);
-      const escapedSourceName = this.escapeCypher(sourceName);
-      const escapedRelation = this.escapeCypher(relation);
-      const escapedTargetLabel = this.escapeCypher(targetLabel);
-      const escapedTargetName = this.escapeCypher(targetName);
-      let relPropAssignments = "";
-      if (properties && Object.keys(properties).length > 0) {
-        relPropAssignments = ` SET ` + Object.entries(properties).map(([key, value]) => {
-          const escapedKey = this.escapeCypher(key);
-          const escapedValue = this.escapeCypherValue(value);
-          return `r.\`${escapedKey}\` = ${escapedValue}`;
-        }).join(", ");
-      }
-      const cypher = `MERGE (source:\`${escapedSourceLabel}\` { name: '${escapedSourceName}' }) ` + `MERGE (target:\`${escapedTargetLabel}\` { name: '${escapedTargetName}' }) ` + `MERGE (source)-[r:\`${escapedRelation}\`]->(target)` + relPropAssignments;
-      await this.query(cypher);
+      const conn = await this.ensureConnected();
+      await conn.run(`
+				INSERT INTO nodes (label, name, properties)
+				VALUES ('${this.escape(sourceLabel)}', '${this.escape(sourceName)}', '{}')
+				ON CONFLICT (label, name) DO NOTHING
+			`);
+      await conn.run(`
+				INSERT INTO nodes (label, name, properties)
+				VALUES ('${this.escape(targetLabel)}', '${this.escape(targetName)}', '{}')
+				ON CONFLICT (label, name) DO NOTHING
+			`);
+      const propsJson = properties ? JSON.stringify(properties) : "{}";
+      await conn.run(`
+				INSERT INTO relationships (source_label, source_name, relation_type, target_label, target_name, properties)
+				VALUES (
+					'${this.escape(sourceLabel)}',
+					'${this.escape(sourceName)}',
+					'${this.escape(relation)}',
+					'${this.escape(targetLabel)}',
+					'${this.escape(targetName)}',
+					'${this.escape(propsJson)}'
+				)
+				ON CONFLICT (source_label, source_name, relation_type, target_label, target_name) DO UPDATE SET
+					properties = EXCLUDED.properties
+			`);
     } catch (error) {
       this.logger.error(`Failed to upsert relationship: ${error instanceof Error ? error.message : String(error)}`);
       throw error;
@@ -916,10 +964,16 @@ class GraphService {
   }
   async deleteNode(label, name) {
     try {
-      const escapedLabel = this.escapeCypher(label);
-      const escapedName = this.escapeCypher(name);
-      const cypher = `MATCH (n:\`${escapedLabel}\` { name: '${escapedName}' }) ` + `DETACH DELETE n`;
-      await this.query(cypher);
+      const conn = await this.ensureConnected();
+      await conn.run(`
+				DELETE FROM relationships
+				WHERE (source_label = '${this.escape(label)}' AND source_name = '${this.escape(name)}')
+				   OR (target_label = '${this.escape(label)}' AND target_name = '${this.escape(name)}')
+			`);
+      await conn.run(`
+				DELETE FROM nodes
+				WHERE label = '${this.escape(label)}' AND name = '${this.escape(name)}'
+			`);
     } catch (error) {
       this.logger.error(`Failed to delete node: ${error instanceof Error ? error.message : String(error)}`);
       throw error;
@@ -927,9 +981,11 @@ class GraphService {
   }
   async deleteDocumentRelationships(documentPath) {
     try {
-      const escapedPath = this.escapeCypher(documentPath);
-      const cypher = `MATCH ()-[r { documentPath: '${escapedPath}' }]-() DELETE r`;
-      await this.query(cypher);
+      const conn = await this.ensureConnected();
+      await conn.run(`
+				DELETE FROM relationships
+				WHERE properties->>'documentPath' = '${this.escape(documentPath)}'
+			`);
     } catch (error) {
       this.logger.error(`Failed to delete document relationships: ${error instanceof Error ? error.message : String(error)}`);
       throw error;
@@ -937,11 +993,18 @@ class GraphService {
   }
   async findNodesByLabel(label, limit) {
     try {
-      const escapedLabel = this.escapeCypher(label);
+      const conn = await this.ensureConnected();
       const limitClause = limit ? ` LIMIT ${limit}` : "";
-      const cypher = `MATCH (n:\`${escapedLabel}\`) RETURN n${limitClause}`;
-      const result = await this.query(cypher);
-      return (result.resultSet || []).map((row) => row[0]);
+      const reader = await conn.runAndReadAll(`
+				SELECT name, properties
+				FROM nodes
+				WHERE label = '${this.escape(label)}'${limitClause}
+			`);
+      return reader.getRows().map((row) => {
+        const [name, properties] = row;
+        const props = properties ? JSON.parse(properties) : {};
+        return { name, ...props };
+      });
     } catch (error) {
       this.logger.error(`Failed to find nodes by label: ${error instanceof Error ? error.message : String(error)}`);
       return [];
@@ -949,10 +1012,17 @@ class GraphService {
   }
   async findRelationships(nodeName) {
     try {
-      const escapedName = this.escapeCypher(nodeName);
-      const cypher = `MATCH (n { name: '${escapedName}' })-[r]-(m) ` + `RETURN type(r), m.name`;
-      const result = await this.query(cypher);
-      return result.resultSet || [];
+      const conn = await this.ensureConnected();
+      const reader = await conn.runAndReadAll(`
+				SELECT relation_type, target_name, source_name
+				FROM relationships
+				WHERE source_name = '${this.escape(nodeName)}'
+				   OR target_name = '${this.escape(nodeName)}'
+			`);
+      return reader.getRows().map((row) => {
+        const [relType, targetName, sourceName] = row;
+        return [relType, sourceName === nodeName ? targetName : sourceName];
+      });
     } catch (error) {
       this.logger.error(`Failed to find relationships: ${error instanceof Error ? error.message : String(error)}`);
       return [];
@@ -960,27 +1030,39 @@ class GraphService {
   }
   async createVectorIndex(label, property, dimensions) {
     try {
-      const escapedLabel = this.escapeCypher(label);
-      const escapedProperty = this.escapeCypher(property);
-      const cypher = `CREATE VECTOR INDEX FOR (n:\`${escapedLabel}\`) ON (n.\`${escapedProperty}\`) OPTIONS { dimension: ${dimensions}, similarityFunction: 'cosine' }`;
-      await this.query(cypher);
+      const indexKey = `${label}_${property}`;
+      if (this.vectorIndexes.has(indexKey)) {
+        return;
+      }
+      const conn = await this.ensureConnected();
+      try {
+        await conn.run(`
+					CREATE INDEX idx_embedding_${this.escape(label)}
+					ON nodes USING HNSW (embedding)
+					WITH (metric = 'cosine')
+				`);
+      } catch {
+        this.logger.debug(`Vector index on ${label}.${property} already exists`);
+      }
+      this.vectorIndexes.add(indexKey);
       this.logger.log(`Created vector index on ${label}.${property} with ${dimensions} dimensions`);
     } catch (error) {
       const errorMessage = error instanceof Error ? error.message : String(error);
-      if (!errorMessage.includes("already indexed")) {
+      if (!errorMessage.includes("already exists")) {
         this.logger.error(`Failed to create vector index: ${errorMessage}`);
         throw error;
       }
-      this.logger.debug(`Vector index on ${label}.${property} already exists`);
     }
   }
   async updateNodeEmbedding(label, name, embedding) {
     try {
-      const escapedLabel = this.escapeCypher(label);
-      const escapedName = this.escapeCypher(name);
+      const conn = await this.ensureConnected();
       const vectorStr = `[${embedding.join(", ")}]`;
-      const cypher = `MATCH (n:\`${escapedLabel}\` { name: '${escapedName}' }) ` + `SET n.embedding = vecf32(${vectorStr})`;
-      await this.query(cypher);
+      await conn.run(`
+				UPDATE nodes
+				SET embedding = ${vectorStr}::FLOAT[${this.embeddingDimensions}]
+				WHERE label = '${this.escape(label)}' AND name = '${this.escape(name)}'
+			`);
     } catch (error) {
       this.logger.error(`Failed to update node embedding: ${error instanceof Error ? error.message : String(error)}`);
       throw error;
@@ -988,111 +1070,66 @@ class GraphService {
   }
   async vectorSearch(label, queryVector, k = 10) {
     try {
-      const escapedLabel = this.escapeCypher(label);
+      const conn = await this.ensureConnected();
       const vectorStr = `[${queryVector.join(", ")}]`;
-      const cypher = `CALL db.idx.vector.queryNodes('${escapedLabel}', 'embedding', ${k}, vecf32(${vectorStr})) ` + `YIELD node, score ` + `RETURN node.name AS name, node.title AS title, (2 - score) / 2 AS similarity ` + `ORDER BY similarity DESC`;
-      const result = await this.query(cypher);
-      return (result.resultSet || []).map((row) => ({
-        name: row[0],
-        title: row[1],
-        score: row[2]
-      }));
+      const reader = await conn.runAndReadAll(`
+				SELECT
+					name,
+					properties->>'title' as title,
+					array_cosine_similarity(embedding, ${vectorStr}::FLOAT[${this.embeddingDimensions}]) as similarity
+				FROM nodes
+				WHERE label = '${this.escape(label)}'
+				  AND embedding IS NOT NULL
+				ORDER BY similarity DESC
+				LIMIT ${k}
+			`);
+      return reader.getRows().map((row) => {
+        const [name, title, similarity] = row;
+        return {
+          name,
+          title: title || undefined,
+          score: similarity
+        };
+      });
     } catch (error) {
       this.logger.error(`Vector search failed: ${error instanceof Error ? error.message : String(error)}`);
       throw error;
     }
   }
   async vectorSearchAll(queryVector, k = 10) {
-    const allLabels = [
-      "Document",
-      "Concept",
-      "Process",
-      "Tool",
-      "Technology",
-      "Organization",
-      "Topic",
-      "Person"
-    ];
     const allResults = [];
-    for (const label of allLabels) {
-      try {
-        const escapedLabel = this.escapeCypher(label);
-        const vectorStr = `[${queryVector.join(", ")}]`;
-        const cypher = `CALL db.idx.vector.queryNodes('${escapedLabel}', 'embedding', ${k}, vecf32(${vectorStr})) ` + `YIELD node, score ` + `RETURN node.name AS name, node.title AS title, node.description AS description, (2 - score) / 2 AS similarity ` + `ORDER BY similarity DESC`;
-        const result = await this.query(cypher);
-        const labelResults = (result.resultSet || []).map((row) => ({
-          name: row[0],
+    const conn = await this.ensureConnected();
+    const vectorStr = `[${queryVector.join(", ")}]`;
+    try {
+      const reader = await conn.runAndReadAll(`
+				SELECT
+					name,
+					label,
+					properties->>'title' as title,
+					properties->>'description' as description,
+					array_cosine_similarity(embedding, ${vectorStr}::FLOAT[${this.embeddingDimensions}]) as similarity
+				FROM nodes
+				WHERE embedding IS NOT NULL
+				ORDER BY similarity DESC
+				LIMIT ${k}
+			`);
+      for (const row of reader.getRows()) {
+        const [name, label, title, description, similarity] = row;
+        allResults.push({
+          name,
           label,
-          title: row[1],
-          description: row[2],
-          score: row[3]
-        }));
-        allResults.push(...labelResults);
-      } catch (error) {
-        this.logger.debug(`Vector search for ${label} skipped: ${error instanceof Error ? error.message : String(error)}`);
+          title: title || undefined,
+          description: description || undefined,
+          score: similarity
+        });
       }
+    } catch (error) {
+      this.logger.debug(`Vector search failed: ${error instanceof Error ? error.message : String(error)}`);
     }
     return allResults.sort((a, b) => b.score - a.score).slice(0, k);
   }
-  escapeCypher(value) {
-    return value.replace(/\\/g, "\\\\").replace(/'/g, "\\'").replace(/"/g, "\\\"");
-  }
-  escapeCypherValue(value) {
-    if (value === null || value === undefined) {
-      return "null";
-    }
-    if (typeof value === "string") {
-      const escaped = this.escapeCypher(value);
-      return `'${escaped}'`;
-    }
-    if (typeof value === "number" || typeof value === "boolean") {
-      return String(value);
-    }
-    if (Array.isArray(value)) {
-      return `[${value.map((v) => this.escapeCypherValue(v)).join(", ")}]`;
-    }
-    if (typeof value === "object") {
-      const pairs = Object.entries(value).map(([k, v]) => `${k}: ${this.escapeCypherValue(v)}`).join(", ");
-      return `{${pairs}}`;
-    }
-    return String(value);
-  }
-  parseStats(result) {
-    if (!Array.isArray(result) || result.length < 3) {
-      return;
-    }
-    const statsStr = result[2];
-    if (!statsStr || typeof statsStr !== "string") {
-      return;
-    }
-    const stats = {
-      nodesCreated: 0,
-      nodesDeleted: 0,
-      relationshipsCreated: 0,
-      relationshipsDeleted: 0,
-      propertiesSet: 0
-    };
-    const nodeCreatedMatch = statsStr.match(/Nodes created: (\d+)/);
-    if (nodeCreatedMatch) {
-      stats.nodesCreated = parseInt(nodeCreatedMatch[1], 10);
-    }
-    const nodeDeletedMatch = statsStr.match(/Nodes deleted: (\d+)/);
-    if (nodeDeletedMatch) {
-      stats.nodesDeleted = parseInt(nodeDeletedMatch[1], 10);
-    }
-    const relCreatedMatch = statsStr.match(/Relationships created: (\d+)/);
-    if (relCreatedMatch) {
-      stats.relationshipsCreated = parseInt(relCreatedMatch[1], 10);
-    }
-    const relDeletedMatch = statsStr.match(/Relationships deleted: (\d+)/);
-    if (relDeletedMatch) {
-      stats.relationshipsDeleted = parseInt(relDeletedMatch[1], 10);
-    }
-    const propSetMatch = statsStr.match(/Properties set: (\d+)/);
-    if (propSetMatch) {
-      stats.propertiesSet = parseInt(propSetMatch[1], 10);
-    }
-    return stats;
+  escape(value) {
+    return value.replace(/'/g, "''");
   }
 }
 GraphService = __legacyDecorateClassTS([
@@ -1218,49 +1255,19 @@ class RelsCommand extends CommandRunner3 {
   async run(inputs) {
     const name = inputs[0];
     try {
-      const escapedName = name.replace(/'/g, "\\'");
-      const cypher = `MATCH (a { name: '${escapedName}' })-[r]-(b) RETURN a, r, b`;
-      const result = await this.graphService.query(cypher);
-      const results = result.resultSet || [];
+      const relationships = await this.graphService.findRelationships(name);
       console.log(`
 === Relationships for "${name}" ===
 `);
-      if (results.length === 0) {
+      if (relationships.length === 0) {
         console.log(`No relationships found.
 `);
         process.exit(0);
       }
-      const incoming = [];
-      const outgoing = [];
-      for (const row of results) {
-        const [source, rel, target] = row;
-        const sourceObj = Object.fromEntries(source);
-        const targetObj = Object.fromEntries(target);
-        const relObj = Object.fromEntries(rel);
-        const sourceProps = Object.fromEntries(sourceObj.properties || []);
-        const targetProps = Object.fromEntries(targetObj.properties || []);
-        const sourceName = sourceProps.name || "unknown";
-        const targetName = targetProps.name || "unknown";
-        const relType = relObj.type || "UNKNOWN";
-        if (sourceName === name) {
-          outgoing.push(`  -[${relType}]-> ${targetName}`);
-        } else {
-          incoming.push(`  <-[${relType}]- ${sourceName}`);
-        }
-      }
-      if (outgoing.length > 0) {
-        console.log("Outgoing:");
-        for (const r of outgoing) {
-          console.log(r);
-        }
-      }
-      if (incoming.length > 0) {
-        if (outgoing.length > 0)
-          console.log();
-        console.log("Incoming:");
-        for (const r of incoming) {
-          console.log(r);
-        }
+      console.log("Relationships:");
+      for (const rel of relationships) {
+        const [relType, targetName] = rel;
+        console.log(`  -[${relType}]-> ${targetName}`);
       }
       console.log();
       process.exit(0);
@@ -1282,7 +1289,7 @@ RelsCommand = __legacyDecorateClassTS([
   ])
 ], RelsCommand);
 
-class CypherCommand extends CommandRunner3 {
+class SqlCommand extends CommandRunner3 {
   graphService;
   constructor(graphService) {
     super();
@@ -1293,9 +1300,10 @@ class CypherCommand extends CommandRunner3 {
     try {
       const result = await this.graphService.query(query);
       console.log(`
-=== Cypher Query Results ===
+=== SQL Query Results ===
 `);
-      console.log(JSON.stringify(result, null, 2));
+      const replacer = (_key, value) => typeof value === "bigint" ? Number(value) : value;
+      console.log(JSON.stringify(result, replacer, 2));
       console.log();
       process.exit(0);
     } catch (error) {
@@ -1304,17 +1312,17 @@ class CypherCommand extends CommandRunner3 {
     }
   }
 }
-CypherCommand = __legacyDecorateClassTS([
+SqlCommand = __legacyDecorateClassTS([
   Injectable7(),
   Command3({
-    name: "cypher",
+    name: "sql",
     arguments: "<query>",
-    description: "Execute raw Cypher query"
+    description: "Execute raw SQL query against DuckDB"
   }),
   __legacyMetadataTS("design:paramtypes", [
     typeof GraphService === "undefined" ? Object : GraphService
   ])
-], CypherCommand);
+], SqlCommand);
 // src/commands/status.command.ts
 import { Injectable as Injectable12 } from "@nestjs/common";
 import { Command as Command4, CommandRunner as CommandRunner4, Option as Option3 } from "nest-commander";
@@ -1432,6 +1440,7 @@ ManifestService = __legacyDecorateClassTS([
 
 // src/sync/sync.service.ts
 import { Injectable as Injectable11, Logger as Logger5 } from "@nestjs/common";
+
 // src/pure/embedding-text.ts
 function composeDocumentEmbeddingText(doc) {
   const parts = [];
@@ -1645,10 +1654,13 @@ class CascadeService {
   }
   async findAffectedByRename(entityName, _newName) {
     try {
-      const escapedName = this.escapeForCypher(entityName);
+      const escapedName = this.escapeForSql(entityName);
       const query = `
-				MATCH (e {name: '${escapedName}'})-[:APPEARS_IN]->(d:Document)
-				RETURN d.name, d.title
+				SELECT DISTINCT n.name, n.properties->>'title' as title
+				FROM nodes n
+				INNER JOIN relationships r ON r.target_label = n.label AND r.target_name = n.name
+				WHERE r.source_name = '${escapedName}'
+				  AND n.label = 'Document'
 			`.trim();
       const result = await this.graph.query(query);
       return (result.resultSet || []).map((row) => ({
@@ -1665,10 +1677,13 @@ class CascadeService {
   }
   async findAffectedByDeletion(entityName) {
     try {
-      const escapedName = this.escapeForCypher(entityName);
+      const escapedName = this.escapeForSql(entityName);
       const query = `
-				MATCH (e {name: '${escapedName}'})-[:APPEARS_IN]->(d:Document)
-				RETURN d.name, d.title
+				SELECT DISTINCT n.name, n.properties->>'title' as title
+				FROM nodes n
+				INNER JOIN relationships r ON r.target_label = n.label AND r.target_name = n.name
+				WHERE r.source_name = '${escapedName}'
+				  AND n.label = 'Document'
 			`.trim();
       const result = await this.graph.query(query);
       return (result.resultSet || []).map((row) => ({
@@ -1685,10 +1700,13 @@ class CascadeService {
   }
   async findAffectedByTypeChange(entityName, oldType, newType) {
     try {
-      const escapedName = this.escapeForCypher(entityName);
+      const escapedName = this.escapeForSql(entityName);
       const query = `
-				MATCH (e {name: '${escapedName}'})-[:APPEARS_IN]->(d:Document)
-				RETURN d.name, d.title
+				SELECT DISTINCT n.name, n.properties->>'title' as title
+				FROM nodes n
+				INNER JOIN relationships r ON r.target_label = n.label AND r.target_name = n.name
+				WHERE r.source_name = '${escapedName}'
+				  AND n.label = 'Document'
 			`.trim();
       const result = await this.graph.query(query);
       return (result.resultSet || []).map((row) => ({
@@ -1705,10 +1723,13 @@ class CascadeService {
   }
   async findAffectedByRelationshipChange(entityName) {
     try {
-      const escapedName = this.escapeForCypher(entityName);
+      const escapedName = this.escapeForSql(entityName);
       const query = `
-				MATCH (e {name: '${escapedName}'})-[r]->(d:Document)
-				RETURN d.name, d.title, type(r) as relType
+				SELECT DISTINCT n.name, n.properties->>'title' as title, r.relation_type
+				FROM nodes n
+				INNER JOIN relationships r ON r.target_label = n.label AND r.target_name = n.name
+				WHERE r.source_name = '${escapedName}'
+				  AND n.label = 'Document'
 			`.trim();
       const result = await this.graph.query(query);
       return (result.resultSet || []).map((row) => ({
@@ -1725,10 +1746,13 @@ class CascadeService {
   }
   async findAffectedByDocumentDeletion(documentPath) {
     try {
-      const escapedPath = this.escapeForCypher(documentPath);
+      const escapedPath = this.escapeForSql(documentPath);
       const query = `
-				MATCH (d:Document)-[r]->(deleted:Document {name: '${escapedPath}'})
-				RETURN d.name, type(r) as relType
+				SELECT DISTINCT n.name, r.relation_type
+				FROM nodes n
+				INNER JOIN relationships r ON r.source_label = n.label AND r.source_name = n.name
+				WHERE r.target_name = '${escapedPath}'
+				  AND n.label = 'Document'
 			`.trim();
       const result = await this.graph.query(query);
       return (result.resultSet || []).map((row) => ({
@@ -1815,8 +1839,8 @@ class CascadeService {
         return action;
     }
   }
-  escapeForCypher(value) {
-    return value.replace(/\\/g, "\\\\").replace(/'/g, "\\'").replace(/"/g, "\\\"");
+  escapeForSql(value) {
+    return value.replace(/'/g, "''");
   }
 }
 CascadeService = __legacyDecorateClassTS([
@@ -2840,9 +2864,9 @@ class QueryService {
   constructor(graphService) {
     this.graphService = graphService;
   }
-  async query(cypher) {
-    this.logger.debug(`Executing query: ${cypher}`);
-    return await this.graphService.query(cypher);
+  async query(sql) {
+    this.logger.debug(`Executing query: ${sql}`);
+    return await this.graphService.query(sql);
   }
 }
 QueryService = __legacyDecorateClassTS([
@@ -2908,7 +2932,7 @@ AppModule = __legacyDecorateClassTS([
       StatusCommand,
       SearchCommand,
       RelsCommand,
-      CypherCommand,
+      SqlCommand,
       ValidateCommand,
       OntologyCommand,
       InitCommand

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@zabaca/lattice",
-	"version": "0.3.4",
+	"version": "1.0.0",
 	"description": "Human-initiated, AI-powered knowledge graph for markdown documentation",
 	"type": "module",
 	"bin": {
@@ -26,7 +26,7 @@
 	},
 	"keywords": [
 		"knowledge-graph",
-		"falkordb",
+		"duckdb",
 		"documentation",
 		"markdown",
 		"ai",
@@ -43,13 +43,13 @@
 		"url": "https://github.com/Zabaca/lattice.git"
 	},
 	"dependencies": {
+		"@duckdb/node-api": "1.3.1-alpha.23",
 		"@nestjs/common": "^10.0.0",
 		"@nestjs/config": "^3.0.0",
 		"@nestjs/core": "^10.0.0",
 		"chalk": "^5.3.0",
 		"glob": "^10.3.0",
 		"gray-matter": "^4.0.3",
-		"ioredis": "^5.3.0",
 		"nest-commander": "^3.12.0",
 		"reflect-metadata": "^0.2.0",
 		"rxjs": "^7.8.0",
@@ -59,7 +59,6 @@
 	},
 	"devDependencies": {
 		"@biomejs/biome": "^2.2.6",
-		"@tkoehlerlg/bun-mock-extended": "^1.0.0",
 		"@types/bun": "latest",
 		"typescript": "^5.0.0"
 	}