@vibe-agent-toolkit/rag-lancedb 0.1.0-rc.7

package/README.md

@@ -0,0 +1,122 @@
+# @vibe-agent-toolkit/rag-lancedb
+
+LanceDB implementation of RAG interfaces for vibe-agent-toolkit.
+
+## Overview
+
+This package provides a complete RAG (Retrieval-Augmented Generation) implementation using LanceDB as the vector database. It implements both `RAGQueryProvider` (read-only) and `RAGAdminProvider` (read/write) interfaces.
+
+**Features:**
+- Local file-based vector database (no server required)
+- Fast vector search with LanceDB
+- Resource-level change detection (content hashing)
+- Readonly and admin modes
+- Persistent storage with Apache Arrow
+
+## Installation
+
+```bash
+bun add @vibe-agent-toolkit/rag-lancedb
+```
+
+## Quick Start
+
+### Building a RAG Database
+
+```typescript
+import { LanceDBRAGProvider } from '@vibe-agent-toolkit/rag-lancedb';
+import { TransformersEmbeddingProvider } from '@vibe-agent-toolkit/rag';
+import { ResourceRegistry } from '@vibe-agent-toolkit/resources';
+
+// 1. Scan resources
+const registry = new ResourceRegistry();
+await registry.crawl({ baseDir: './docs' });
+
+// 2. Create admin provider
+const admin = await LanceDBRAGProvider.create({
+  dbPath: './rag-db',
+  embeddingProvider: new TransformersEmbeddingProvider(),
+});
+
+// 3. Index resources
+const result = await admin.indexResources(registry.getAllResources());
+console.log(`Indexed ${result.chunksCreated} chunks`);
+
+await admin.close();
+```
+
+### Querying a RAG Database
+
+```typescript
+import { LanceDBRAGProvider } from '@vibe-agent-toolkit/rag-lancedb';
+
+// Open in readonly mode (for agents)
+const rag = await LanceDBRAGProvider.create({
+  dbPath: './rag-db',
+  readonly: true,
+});
+
+// Query
+const result = await rag.query({
+  text: 'How do I validate schemas?',
+  limit: 5,
+  filters: {
+    tags: ['validation'],
+  },
+});
+
+// Use results
+for (const chunk of result.chunks) {
+  console.log(`[${chunk.headingPath}] ${chunk.content}`);
+}
+```
+
+## Architecture
+
+- **Database**: LanceDB (Apache Arrow format)
+- **Storage**: Local file-based (`.lance` directory)
+- **Change Detection**: Content hashing (SHA-256)
+- **Updates**: Resource-level (delete old chunks, re-embed new)
+
+## Known Issues
+
+### Bun + vectordb@0.4.x Arrow Buffer Detachment
+
+**Issue**: When running under Bun runtime with `vectordb@0.4.20`, Apache Arrow buffers can become detached after table modifications (add/delete operations), causing "Buffer is already detached" errors on subsequent queries. This primarily affects rapid multi-resource indexing operations in short-lived test scenarios.
+
+**Technical Details**:
+- **Root Cause**: Apache Arrow uses zero-copy memory buffers for performance. When LanceDB performs table modifications, these buffers can be garbage collected or invalidated before subsequent queries can access them. Bun's garbage collector appears to be more aggressive than Node.js in reclaiming these buffers.
+- **Trigger**: Indexing multiple resources in rapid succession, then querying the database in the same connection lifecycle
+- **Error**: `TypeError: Buffer is already detached` when calling `.execute()` on query results
+
+**Workarounds Implemented**:
+1. **Connection Recreation**: Before each query/stats operation, we recreate the database connection entirely (`connect()` + `openTable()`)
+2. **Immediate Materialization**: Results are materialized using `JSON.parse(JSON.stringify())` immediately after query execution to copy data out of Arrow buffers before they're invalidated
+3. **Data Extraction Before Modifications**: When checking for existing resources during indexing, we extract all needed data from Arrow buffers before performing any table modifications
+
+**Production Impact**: **Minimal**
+- Production usage typically involves long-lived provider connections where this issue doesn't manifest
+- Single-resource operations work reliably
+- Query operations (the most common in production) are stable after the connection recreation workaround
+
+**Test Impact**: **Significant**
+- 18 integration and system tests are skipped due to this issue
+- Tests that index multiple resources in rapid succession fail consistently
+- Single-resource tests pass reliably
+
+**Resolution Options**:
+1. **Upgrade LanceDB**: Wait for newer versions of `vectordb` that may have fixed this issue (current: ^0.4.0)
+2. **Use Node.js**: Run tests with Node.js instead of Bun (the issue is Bun-specific)
+3. **Accept Limitation**: Document and skip affected tests, as production usage is unaffected
+
+**Why Not Use `structuredClone()`?**
+The standard `structuredClone()` function doesn't work with Arrow buffers because they contain native memory references that can't be cloned. `JSON.parse(JSON.stringify())` forces full serialization/deserialization, creating new JavaScript objects completely independent of Arrow's memory management.
+
+**References**:
+- Issue affects: `packages/rag-lancedb/src/lancedb-rag-provider.ts` (see comments at lines 107, 138, 198, 220, 282, 289)
+- Skipped tests: `packages/rag-lancedb/test/integration/indexing.integration.test.ts`
+- Skipped system tests: `packages/rag-lancedb/test/system/lancedb-rag-provider.system.test.ts`
+
+## License
+
+MIT

package/dist/index.d.ts

@@ -0,0 +1,8 @@
+/**
+ * @vibe-agent-toolkit/rag-lancedb
+ *
+ * LanceDB implementation of RAG interfaces for vibe-agent-toolkit.
+ */
+export { LanceDBRAGProvider, type LanceDBConfig } from './lancedb-rag-provider.js';
+export { chunkToLanceRow, lanceRowToChunk, type LanceDBRow } from './schema.js';
+//# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,EAAE,kBAAkB,EAAE,KAAK,aAAa,EAAE,MAAM,2BAA2B,CAAC;AACnF,OAAO,EAAE,eAAe,EAAE,eAAe,EAAE,KAAK,UAAU,EAAE,MAAM,aAAa,CAAC"}

package/dist/index.js

@@ -0,0 +1,8 @@
+/**
+ * @vibe-agent-toolkit/rag-lancedb
+ *
+ * LanceDB implementation of RAG interfaces for vibe-agent-toolkit.
+ */
+export { LanceDBRAGProvider } from './lancedb-rag-provider.js';
+export { chunkToLanceRow, lanceRowToChunk } from './schema.js';
+//# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,EAAE,kBAAkB,EAAsB,MAAM,2BAA2B,CAAC;AACnF,OAAO,EAAE,eAAe,EAAE,eAAe,EAAmB,MAAM,aAAa,CAAC"}

package/dist/lancedb-rag-provider.d.ts

@@ -0,0 +1,87 @@
+/**
+ * LanceDB RAG Provider
+ *
+ * Implements both RAGQueryProvider and RAGAdminProvider using LanceDB.
+ */
+import type { EmbeddingProvider, IndexResult, RAGAdminProvider, RAGQuery, RAGResult, RAGStats } from '@vibe-agent-toolkit/rag';
+import { type ResourceMetadata } from '@vibe-agent-toolkit/resources';
+/**
+ * Configuration for LanceDBRAGProvider
+ */
+export interface LanceDBConfig {
+    /** Path to LanceDB database directory */
+    dbPath: string;
+    /** Readonly mode (query only) */
+    readonly?: boolean;
+    /** Embedding provider (default: TransformersEmbeddingProvider) */
+    embeddingProvider?: EmbeddingProvider;
+    /** Target chunk size in tokens (default: 512) */
+    targetChunkSize?: number;
+    /** Padding factor for token estimation (default: 0.9) */
+    paddingFactor?: number;
+}
+/**
+ * LanceDBRAGProvider
+ *
+ * Complete RAG implementation using LanceDB for vector storage.
+ */
+export declare class LanceDBRAGProvider implements RAGAdminProvider {
+    private readonly config;
+    private connection;
+    private table;
+    private readonly tokenCounter;
+    private constructor();
+    /**
+     * Create and initialize LanceDBRAGProvider
+     *
+     * @param config - Configuration
+     * @returns Initialized provider
+     */
+    static create(config: LanceDBConfig): Promise<LanceDBRAGProvider>;
+    /**
+     * Initialize database connection and table
+     */
+    private initialize;
+    /**
+     * Query the RAG database
+     */
+    query(query: RAGQuery): Promise<RAGResult>;
+    /**
+     * Build WHERE clause from filters
+     */
+    private buildWhereClause;
+    /**
+     * Get database statistics
+     */
+    getStats(): Promise<RAGStats>;
+    /**
+     * Index resources into the RAG database
+     */
+    indexResources(resources: ResourceMetadata[]): Promise<IndexResult>;
+    /**
+     * Index a single resource
+     */
+    private indexResource;
+    /**
+     * Update a specific resource
+     */
+    updateResource(_resourceId: string): Promise<void>;
+    /**
+     * Delete a specific resource and all its chunks
+     *
+     * @param resourceId - ID of resource to delete
+     */
+    deleteResource(resourceId: string): Promise<void>;
+    /**
+     * Clear the entire database
+     *
+     * Deletes all data and removes the database directory.
+     * This is a destructive operation that cannot be undone.
+     */
+    clear(): Promise<void>;
+    /**
+     * Close database connection
+     */
+    close(): Promise<void>;
+}
+//# sourceMappingURL=lancedb-rag-provider.d.ts.map

package/dist/lancedb-rag-provider.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"lancedb-rag-provider.d.ts","sourceRoot":"","sources":["../src/lancedb-rag-provider.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAIH,OAAO,KAAK,EACV,iBAAiB,EACjB,WAAW,EACX,gBAAgB,EAEhB,QAAQ,EACR,SAAS,EACT,QAAQ,EACT,MAAM,yBAAyB,CAAC;AAQjC,OAAO,EAAiB,KAAK,gBAAgB,EAAE,MAAM,+BAA+B,CAAC;AAMrF;;GAEG;AACH,MAAM,WAAW,aAAa;IAC5B,yCAAyC;IACzC,MAAM,EAAE,MAAM,CAAC;IAEf,iCAAiC;IACjC,QAAQ,CAAC,EAAE,OAAO,CAAC;IAEnB,kEAAkE;IAClE,iBAAiB,CAAC,EAAE,iBAAiB,CAAC;IAEtC,iDAAiD;IACjD,eAAe,CAAC,EAAE,MAAM,CAAC;IAEzB,yDAAyD;IACzD,aAAa,CAAC,EAAE,MAAM,CAAC;CACxB;AAID;;;;GAIG;AACH,qBAAa,kBAAmB,YAAW,gBAAgB;IACzD,OAAO,CAAC,QAAQ,CAAC,MAAM,CAA0B;IACjD,OAAO,CAAC,UAAU,CAA2B;IAC7C,OAAO,CAAC,KAAK,CAAkC;IAC/C,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAiC;IAE9D,OAAO;IAUP;;;;;OAKG;WACU,MAAM,CAAC,MAAM,EAAE,aAAa,GAAG,OAAO,CAAC,kBAAkB,CAAC;IAMvE;;OAEG;YACW,UAAU;IAcxB;;OAEG;IACG,KAAK,CAAC,KAAK,EAAE,QAAQ,GAAG,OAAO,CAAC,SAAS,CAAC;IAqDhD;;OAEG;IACH,OAAO,CAAC,gBAAgB;IAgCxB;;OAEG;IACG,QAAQ,IAAI,OAAO,CAAC,QAAQ,CAAC;IAqCnC;;OAEG;IACG,cAAc,CAAC,SAAS,EAAE,gBAAgB,EAAE,GAAG,OAAO,CAAC,WAAW,CAAC;IA+BzE;;OAEG;YACW,aAAa;IAqG3B;;OAEG;IACG,cAAc,CAAC,WAAW,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAIxD;;;;OAIG;IACG,cAAc,CAAC,UAAU,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAavD;;;;;OAKG;IACG,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC;IAe5B;;OAEG;IACG,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC;CAK7B"}

package/dist/lancedb-rag-provider.js

@@ -0,0 +1,319 @@
+/**
+ * LanceDB RAG Provider
+ *
+ * Implements both RAGQueryProvider and RAGAdminProvider using LanceDB.
+ */
+import fs from 'node:fs';
+import { ApproximateTokenCounter, chunkResource, enrichChunks, generateContentHash, TransformersEmbeddingProvider, } from '@vibe-agent-toolkit/rag';
+import { parseMarkdown } from '@vibe-agent-toolkit/resources';
+import { connect } from 'vectordb';
+import { chunkToLanceRow, lanceRowToChunk } from './schema.js';
+const TABLE_NAME = 'rag_chunks';
+/**
+ * LanceDBRAGProvider
+ *
+ * Complete RAG implementation using LanceDB for vector storage.
+ */
+export class LanceDBRAGProvider {
+    config;
+    connection = null;
+    table = null;
+    tokenCounter = new ApproximateTokenCounter();
+    constructor(config) {
+        this.config = {
+            readonly: false,
+            embeddingProvider: new TransformersEmbeddingProvider(),
+            targetChunkSize: 512,
+            paddingFactor: 0.9,
+            ...config,
+        };
+    }
+    /**
+     * Create and initialize LanceDBRAGProvider
+     *
+     * @param config - Configuration
+     * @returns Initialized provider
+     */
+    static async create(config) {
+        const provider = new LanceDBRAGProvider(config);
+        await provider.initialize();
+        return provider;
+    }
+    /**
+     * Initialize database connection and table
+     */
+    async initialize() {
+        this.connection = await connect(this.config.dbPath);
+        const tableNames = await this.connection.tableNames();
+        if (tableNames.includes(TABLE_NAME)) {
+            this.table = await this.connection.openTable(TABLE_NAME);
+        }
+        else {
+            // Table doesn't exist
+            // In admin mode: will be created on first insert
+            // In readonly mode: operations that require table will fail gracefully
+            this.table = null;
+        }
+    }
+    /**
+     * Query the RAG database
+     */
+    async query(query) {
+        // Workaround for vectordb@0.4.20 + Bun Arrow buffer lifecycle bug
+        // After table modifications, we need to recreate the connection entirely
+        this.connection = await connect(this.config.dbPath);
+        const tableNames = await this.connection.tableNames();
+        if (!tableNames.includes(TABLE_NAME)) {
+            throw new Error('No data indexed yet');
+        }
+        this.table = (await this.connection.openTable(TABLE_NAME));
+        if (!this.table) {
+            throw new Error('No data indexed yet');
+        }
+        const startTime = Date.now();
+        // Embed query text
+        const queryEmbedding = await this.config.embeddingProvider.embed(query.text);
+        // Perform vector search
+        let search = this.table.search(queryEmbedding).limit(query.limit ?? 10);
+        // Apply filters if provided
+        if (query.filters) {
+            const whereClause = this.buildWhereClause(query.filters);
+            if (whereClause) {
+                search = search.where(whereClause);
+            }
+        }
+        const results = await search.execute();
+        // Convert results to plain objects immediately to avoid Arrow buffer issues
+        // eslint-disable-next-line unicorn/prefer-structured-clone -- JSON.parse/stringify is intentional workaround for Arrow buffer lifecycle bug
+        const materializedResults = JSON.parse(JSON.stringify(results));
+        // Convert results to RAGChunks
+        const chunks = materializedResults.map((row) => lanceRowToChunk(row));
+        const searchDurationMs = Date.now() - startTime;
+        return {
+            chunks,
+            stats: {
+                totalMatches: chunks.length,
+                searchDurationMs,
+                embedding: {
+                    model: this.config.embeddingProvider.model,
+                },
+            },
+        };
+    }
+    /**
+     * Build WHERE clause from filters
+     */
+    buildWhereClause(filters) {
+        if (!filters) {
+            return null;
+        }
+        const conditions = [];
+        if (filters.resourceId !== undefined) {
+            const ids = Array.isArray(filters.resourceId) ? filters.resourceId : [filters.resourceId];
+            // Handle empty array case - should match nothing
+            if (ids.length === 0) {
+                conditions.push('1 = 0'); // Always false condition
+            }
+            else {
+                const idList = ids.map((id) => `'${id}'`).join(', ');
+                // Use backticks for column names
+                conditions.push(`\`resourceId\` IN (${idList})`);
+            }
+        }
+        if (filters.type) {
+            conditions.push(`type = '${filters.type}'`);
+        }
+        if (filters.headingPath) {
+            // Use backticks for column names
+            conditions.push(`\`headingPath\` = '${filters.headingPath}'`);
+        }
+        return conditions.length > 0 ? conditions.join(' AND ') : null;
+    }
+    /**
+     * Get database statistics
+     */
+    async getStats() {
+        // Workaround for vectordb@0.4.20 + Bun Arrow buffer lifecycle bug
+        // After table modifications, we need to recreate the connection entirely
+        this.connection = await connect(this.config.dbPath);
+        const tableNames = await this.connection.tableNames();
+        if (tableNames.includes(TABLE_NAME)) {
+            this.table = (await this.connection.openTable(TABLE_NAME));
+        }
+        if (!this.table) {
+            return {
+                totalChunks: 0,
+                totalResources: 0,
+                dbSizeBytes: 0,
+                embeddingModel: this.config.embeddingProvider.model,
+                lastIndexed: new Date(0),
+            };
+        }
+        const count = await this.table.countRows();
+        // Get unique resource count (use a condition that matches all rows)
+        const allRows = await this.table.filter('1 = 1').execute();
+        // Materialize immediately to avoid Arrow buffer issues
+        // eslint-disable-next-line unicorn/prefer-structured-clone -- JSON.parse/stringify is intentional workaround for Arrow buffer lifecycle bug
+        const rows = JSON.parse(JSON.stringify(allRows));
+        const uniqueResources = new Set(rows.map((r) => r.resourceId)).size;
+        return {
+            totalChunks: count,
+            totalResources: uniqueResources,
+            dbSizeBytes: 0, // LanceDB doesn't provide size info easily
+            embeddingModel: this.config.embeddingProvider.model,
+            lastIndexed: new Date(), // Would need to track this separately
+        };
+    }
+    /**
+     * Index resources into the RAG database
+     */
+    async indexResources(resources) {
+        if (this.config.readonly) {
+            throw new Error('Cannot index in readonly mode');
+        }
+        const startTime = Date.now();
+        const result = {
+            resourcesIndexed: 0,
+            resourcesSkipped: 0,
+            resourcesUpdated: 0,
+            chunksCreated: 0,
+            chunksDeleted: 0,
+            durationMs: 0,
+            errors: [],
+        };
+        for (const resource of resources) {
+            try {
+                await this.indexResource(resource, result);
+            }
+            catch (error) {
+                result.errors?.push({
+                    resourceId: resource.id,
+                    error: error instanceof Error ? error.message : String(error),
+                });
+            }
+        }
+        result.durationMs = Date.now() - startTime;
+        return result;
+    }
+    /**
+     * Index a single resource
+     */
+    async indexResource(resource, result) {
+        // Read file content using parseMarkdown
+        const parseResult = await parseMarkdown(resource.filePath);
+        // Generate content hash for change detection
+        const resourceContentHash = generateContentHash(parseResult.content);
+        // Check if resource already exists with same content hash
+        // Extract all needed data BEFORE any table modifications to avoid Arrow buffer issues
+        let shouldSkip = false;
+        let deleteCount = 0;
+        let shouldUpdate = false;
+        if (this.table) {
+            const existingRows = await this.table.filter(`\`resourceId\` = '${resource.id}'`).execute();
+            // Materialize immediately to avoid Arrow buffer issues
+            // eslint-disable-next-line unicorn/prefer-structured-clone -- JSON.parse/stringify is intentional workaround for Arrow buffer lifecycle bug
+            const existing = JSON.parse(JSON.stringify(existingRows));
+            // Extract data immediately while Arrow buffers are valid
+            if (existing.length > 0) {
+                const existingHash = existing[0]?.resourceContentHash;
+                deleteCount = existing.length;
+                if (existingHash === resourceContentHash) {
+                    shouldSkip = true;
+                }
+                else {
+                    shouldUpdate = true;
+                }
+            }
+        }
+        // Handle skip case
+        if (shouldSkip) {
+            result.resourcesSkipped++;
+            return;
+        }
+        // Delete old chunks if updating
+        if (shouldUpdate) {
+            await this.deleteResource(resource.id);
+            result.chunksDeleted += deleteCount;
+            result.resourcesUpdated++;
+        }
+        // Chunk the resource
+        const chunkingResult = chunkResource({
+            ...resource,
+            content: parseResult.content,
+            frontmatter: {},
+        }, {
+            targetChunkSize: this.config.targetChunkSize,
+            modelTokenLimit: 8191,
+            paddingFactor: this.config.paddingFactor,
+            tokenCounter: this.tokenCounter,
+        });
+        // Embed chunks
+        const embeddings = await this.config.embeddingProvider.embedBatch(chunkingResult.chunks.map((c) => c.content));
+        // Enrich chunks with full metadata
+        const chunkableResource = {
+            ...resource,
+            content: parseResult.content,
+            frontmatter: {},
+        };
+        const ragChunks = enrichChunks(chunkingResult.chunks, chunkableResource, embeddings, this.config.embeddingProvider.model);
+        // Convert to LanceDB rows
+        const rows = ragChunks.map((chunk) => chunkToLanceRow(chunk, resourceContentHash));
+        // Insert into LanceDB
+        if (!this.table && this.connection) {
+            this.table = (await this.connection.createTable(TABLE_NAME, rows));
+        }
+        else if (this.table) {
+            await this.table.add(rows);
+        }
+        result.resourcesIndexed++;
+        result.chunksCreated += rows.length;
+    }
+    /**
+     * Update a specific resource
+     */
+    async updateResource(_resourceId) {
+        throw new Error('Not implemented - use indexResources() instead');
+    }
+    /**
+     * Delete a specific resource and all its chunks
+     *
+     * @param resourceId - ID of resource to delete
+     */
+    async deleteResource(resourceId) {
+        if (this.config.readonly) {
+            throw new Error('Cannot delete in readonly mode');
+        }
+        if (!this.table) {
+            return;
+        }
+        // Delete chunks (use backticks for column names)
+        await this.table.delete(`\`resourceId\` = '${resourceId}'`);
+    }
+    /**
+     * Clear the entire database
+     *
+     * Deletes all data and removes the database directory.
+     * This is a destructive operation that cannot be undone.
+     */
+    async clear() {
+        if (this.config.readonly) {
+            throw new Error('Cannot clear in readonly mode');
+        }
+        // Close connection first
+        await this.close();
+        // Delete entire database directory
+        // eslint-disable-next-line security/detect-non-literal-fs-filename -- dbPath comes from validated config
+        if (fs.existsSync(this.config.dbPath)) {
+            fs.rmSync(this.config.dbPath, { recursive: true, force: true });
+        }
+    }
+    /**
+     * Close database connection
+     */
+    async close() {
+        // LanceDB connections are automatically managed
+        this.connection = null;
+        this.table = null;
+    }
+}
+//# sourceMappingURL=lancedb-rag-provider.js.map

package/dist/lancedb-rag-provider.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"lancedb-rag-provider.js","sourceRoot":"","sources":["../src/lancedb-rag-provider.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,EAAE,MAAM,SAAS,CAAC;AAWzB,OAAO,EACL,uBAAuB,EACvB,aAAa,EACb,YAAY,EACZ,mBAAmB,EACnB,6BAA6B,GAC9B,MAAM,yBAAyB,CAAC;AACjC,OAAO,EAAE,aAAa,EAAyB,MAAM,+BAA+B,CAAC;AACrF,OAAO,EAAE,OAAO,EAAE,MAAM,UAAU,CAAC;AAGnC,OAAO,EAAE,eAAe,EAAE,eAAe,EAAmB,MAAM,aAAa,CAAC;AAsBhF,MAAM,UAAU,GAAG,YAAY,CAAC;AAEhC;;;;GAIG;AACH,MAAM,OAAO,kBAAkB;IACZ,MAAM,CAA0B;IACzC,UAAU,GAAsB,IAAI,CAAC;IACrC,KAAK,GAA6B,IAAI,CAAC;IAC9B,YAAY,GAAG,IAAI,uBAAuB,EAAE,CAAC;IAE9D,YAAoB,MAAqB;QACvC,IAAI,CAAC,MAAM,GAAG;YACZ,QAAQ,EAAE,KAAK;YACf,iBAAiB,EAAE,IAAI,6BAA6B,EAAE;YACtD,eAAe,EAAE,GAAG;YACpB,aAAa,EAAE,GAAG;YAClB,GAAG,MAAM;SACV,CAAC;IACJ,CAAC;IAED;;;;;OAKG;IACH,MAAM,CAAC,KAAK,CAAC,MAAM,CAAC,MAAqB;QACvC,MAAM,QAAQ,GAAG,IAAI,kBAAkB,CAAC,MAAM,CAAC,CAAC;QAChD,MAAM,QAAQ,CAAC,UAAU,EAAE,CAAC;QAC5B,OAAO,QAAQ,CAAC;IAClB,CAAC;IAED;;OAEG;IACK,KAAK,CAAC,UAAU;QACtB,IAAI,CAAC,UAAU,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;QAEpD,MAAM,UAAU,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,UAAU,EAAE,CAAC;QACtD,IAAI,UAAU,CAAC,QAAQ,CAAC,UAAU,CAAC,EAAE,CAAC;YACpC,IAAI,CAAC,KAAK,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,UAAU,CAAC,CAAC;QAC3D,CAAC;aAAM,CAAC;YACN,sBAAsB;YACtB,iDAAiD;YACjD,uEAAuE;YACvE,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC;QACpB,CAAC;IACH,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,KAAK,CAAC,KAAe;QACzB,kEAAkE;QAClE,yEAAyE;QACzE,IAAI,CAAC,UAAU,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;QACpD,MAAM,UAAU,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,UAAU,EAAE,CAAC;QACtD,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,UAAU,CAAC,EAAE,CAAC;YACrC,MAAM,IAAI,KAAK,CAAC,qBAAqB,CAAC,CAAC;QACzC,CAAC;QACD,IAAI,CAAC,KAAK,GAAG,CAAC,MAAM,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,UAAU,CAAC,CAAiC,CAAC;QAE3F,IAAI,CAAC,IAAI,CAAC,KAAK,EAAE,CAAC;YAChB,MAAM,IAAI,KAAK,CAAC,qBAAqB,CAAC,CAAC;QACzC,CAAC;QAED,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;QAE7B,mBAAmB;QACnB,MAAM,cAAc,GAAG,MAAM,IAAI,CAAC,MAAM,CAAC,iBAAiB,CAAC,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QAE7E,wBAAwB;QACxB,IAAI,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,cAAuC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC;QAEjG,4BAA4B;QAC5B,IAAI,KAAK,CAAC,OAAO,EAAE,CAAC;YAClB,MAAM,WAAW,GAAG,IAAI,CAAC,gBAAgB,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;YACzD,IAAI,WAAW,EAAE,CAAC;gBAChB,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,WAAW,CAAC,CAAC;YACrC,CAAC;QACH,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,MAAM,CAAC,OAAO,EAAE,CAAC;QAEvC,4EAA4E;QAC5E,4IAA4I;QAC5I,MAAM,mBAAmB,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,CAAiB,CAAC;QAEhF,+BAA+B;QAC/B,MAAM,MAAM,GAAG,mBAAmB,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,eAAe,CAAC,GAAG,CAAC,CAAC,CAAC;QAEtE,MAAM,gBAAgB,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,SAAS,CAAC;QAEhD,OAAO;YACL,MAAM;YACN,KAAK,EAAE;gBACL,YAAY,EAAE,MAAM,CAAC,MAAM;gBAC3B,gBAAgB;gBAChB,SAAS,EAAE;oBACT,KAAK,EAAE,IAAI,CAAC,MAAM,CAAC,iBAAiB,CAAC,KAAK;iBAC3C;aACF;SACF,CAAC;IACJ,CAAC;IAED;;OAEG;IACK,gBAAgB,CAAC,OAA4B;QACnD,IAAI,CAAC,OAAO,EAAE,CAAC;YACb,OAAO,IAAI,CAAC;QACd,CAAC;QAED,MAAM,UAAU,GAAa,EAAE,CAAC;QAEhC,IAAI,OAAO,CAAC,UAAU,KAAK,SAAS,EAAE,CAAC;YACrC,MAAM,GAAG,GAAG,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC;YAE1F,iDAAiD;YACjD,IAAI,GAAG,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBACrB,UAAU,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,yBAAyB;YACrD,CAAC;iBAAM,CAAC;gBACN,MAAM,MAAM,GAAG,GAAG,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,IAAI,EAAE,GAAG,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;gBACrD,iCAAiC;gBACjC,UAAU,CAAC,IAAI,CAAC,sBAAsB,MAAM,GAAG,CAAC,CAAC;YACnD,CAAC;QACH,CAAC;QAED,IAAI,OAAO,CAAC,IAAI,EAAE,CAAC;YACjB,UAAU,CAAC,IAAI,CAAC,WAAW,OAAO,CAAC,IAAI,GAAG,CAAC,CAAC;QAC9C,CAAC;QAED,IAAI,OAAO,CAAC,WAAW,EAAE,CAAC;YACxB,iCAAiC;YACjC,UAAU,CAAC,IAAI,CAAC,sBAAsB,OAAO,CAAC,WAAW,GAAG,CAAC,CAAC;QAChE,CAAC;QAED,OAAO,UAAU,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;IACjE,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,QAAQ;QACZ,kEAAkE;QAClE,yEAAyE;QACzE,IAAI,CAAC,UAAU,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;QACpD,MAAM,UAAU,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,UAAU,EAAE,CAAC;QACtD,IAAI,UAAU,CAAC,QAAQ,CAAC,UAAU,CAAC,EAAE,CAAC;YACpC,IAAI,CAAC,KAAK,GAAG,CAAC,MAAM,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,UAAU,CAAC,CAAiC,CAAC;QAC7F,CAAC;QAED,IAAI,CAAC,IAAI,CAAC,KAAK,EAAE,CAAC;YAChB,OAAO;gBACL,WAAW,EAAE,CAAC;gBACd,cAAc,EAAE,CAAC;gBACjB,WAAW,EAAE,CAAC;gBACd,cAAc,EAAE,IAAI,CAAC,MAAM,CAAC,iBAAiB,CAAC,KAAK;gBACnD,WAAW,EAAE,IAAI,IAAI,CAAC,CAAC,CAAC;aACzB,CAAC;QACJ,CAAC;QAED,MAAM,KAAK,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,SAAS,EAAE,CAAC;QAE3C,oEAAoE;QACpE,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,OAAO,EAAE,CAAC;QAC3D,uDAAuD;QACvD,4IAA4I;QAC5I,MAAM,IAAI,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,CAAiB,CAAC;QACjE,MAAM,eAAe,GAAG,IAAI,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC;QAEpE,OAAO;YACL,WAAW,EAAE,KAAK;YAClB,cAAc,EAAE,eAAe;YAC/B,WAAW,EAAE,CAAC,EAAE,2CAA2C;YAC3D,cAAc,EAAE,IAAI,CAAC,MAAM,CAAC,iBAAiB,CAAC,KAAK;YACnD,WAAW,EAAE,IAAI,IAAI,EAAE,EAAE,sCAAsC;SAChE,CAAC;IACJ,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,cAAc,CAAC,SAA6B;QAChD,IAAI,IAAI,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC;YACzB,MAAM,IAAI,KAAK,CAAC,+BAA+B,CAAC,CAAC;QACnD,CAAC;QAED,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;QAC7B,MAAM,MAAM,GAAgB;YAC1B,gBAAgB,EAAE,CAAC;YACnB,gBAAgB,EAAE,CAAC;YACnB,gBAAgB,EAAE,CAAC;YACnB,aAAa,EAAE,CAAC;YAChB,aAAa,EAAE,CAAC;YAChB,UAAU,EAAE,CAAC;YACb,MAAM,EAAE,EAAE;SACX,CAAC;QAEF,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;YACjC,IAAI,CAAC;gBACH,MAAM,IAAI,CAAC,aAAa,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAC;YAC7C,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,MAAM,CAAC,MAAM,EAAE,IAAI,CAAC;oBAClB,UAAU,EAAE,QAAQ,CAAC,EAAE;oBACvB,KAAK,EAAE,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC;iBAC9D,CAAC,CAAC;YACL,CAAC;QACH,CAAC;QAED,MAAM,CAAC,UAAU,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,SAAS,CAAC;QAC3C,OAAO,MAAM,CAAC;IAChB,CAAC;IAED;;OAEG;IACK,KAAK,CAAC,aAAa,CACzB,QAA0B,EAC1B,MAAmB;QAEnB,wCAAwC;QACxC,MAAM,WAAW,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC;QAE3D,6CAA6C;QAC7C,MAAM,mBAAmB,GAAG,mBAAmB,CAAC,WAAW,CAAC,OAAO,CAAC,CAAC;QAErE,0DAA0D;QAC1D,sFAAsF;QACtF,IAAI,UAAU,GAAG,KAAK,CAAC;QACvB,IAAI,WAAW,GAAG,CAAC,CAAC;QACpB,IAAI,YAAY,GAAG,KAAK,CAAC;QAEzB,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;YACf,MAAM,YAAY,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,qBAAqB,QAAQ,CAAC,EAAE,GAAG,CAAC,CAAC,OAAO,EAAE,CAAC;YAC5F,uDAAuD;YACvD,4IAA4I;YAC5I,MAAM,QAAQ,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,YAAY,CAAC,CAAiB,CAAC;YAE1E,yDAAyD;YACzD,IAAI,QAAQ,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;gBACxB,MAAM,YAAY,GAAG,QAAQ,CAAC,CAAC,CAAC,EAAE,mBAAmB,CAAC;gBACtD,WAAW,GAAG,QAAQ,CAAC,MAAM,CAAC;gBAE9B,IAAI,YAAY,KAAK,mBAAmB,EAAE,CAAC;oBACzC,UAAU,GAAG,IAAI,CAAC;gBACpB,CAAC;qBAAM,CAAC;oBACN,YAAY,GAAG,IAAI,CAAC;gBACtB,CAAC;YACH,CAAC;QACH,CAAC;QAED,mBAAmB;QACnB,IAAI,UAAU,EAAE,CAAC;YACf,MAAM,CAAC,gBAAgB,EAAE,CAAC;YAC1B,OAAO;QACT,CAAC;QAED,gCAAgC;QAChC,IAAI,YAAY,EAAE,CAAC;YACjB,MAAM,IAAI,CAAC,cAAc,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC;YACvC,MAAM,CAAC,aAAa,IAAI,WAAW,CAAC;YACpC,MAAM,CAAC,gBAAgB,EAAE,CAAC;QAC5B,CAAC;QAED,qBAAqB;QACrB,MAAM,cAAc,GAAG,aAAa,CAClC;YACE,GAAG,QAAQ;YACX,OAAO,EAAE,WAAW,CAAC,OAAO;YAC5B,WAAW,EAAE,EAAE;SAChB,EACD;YACE,eAAe,EAAE,IAAI,CAAC,MAAM,CAAC,eAAe;YAC5C,eAAe,EAAE,IAAI;YACrB,aAAa,EAAE,IAAI,CAAC,MAAM,CAAC,aAAa;YACxC,YAAY,EAAE,IAAI,CAAC,YAAY;SAChC,CACF,CAAC;QAEF,eAAe;QACf,MAAM,UAAU,GAAG,MAAM,IAAI,CAAC,MAAM,CAAC,iBAAiB,CAAC,UAAU,CAC/D,cAAc,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,CAC5C,CAAC;QAEF,mCAAmC;QACnC,MAAM,iBAAiB,GAAG;YACxB,GAAG,QAAQ;YACX,OAAO,EAAE,WAAW,CAAC,OAAO;YAC5B,WAAW,EAAE,EAAE;SAChB,CAAC;QAEF,MAAM,SAAS,GAAG,YAAY,CAC5B,cAAc,CAAC,MAAM,EACrB,iBAAiB,EACjB,UAAU,EACV,IAAI,CAAC,MAAM,CAAC,iBAAiB,CAAC,KAAK,CACpC,CAAC;QAEF,0BAA0B;QAC1B,MAAM,IAAI,GAAG,SAAS,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CACnC,eAAe,CAAC,KAAiB,EAAE,mBAAmB,CAAC,CACxD,CAAC;QAEF,sBAAsB;QACtB,IAAI,CAAC,IAAI,CAAC,KAAK,IAAI,IAAI,CAAC,UAAU,EAAE,CAAC;YACnC,IAAI,CAAC,KAAK,GAAG,CAAC,MAAM,IAAI,CAAC,UAAU,CAAC,WAAW,CAC7C,UAAU,EACV,IAAI,CACL,CAAiC,CAAC;QACrC,CAAC;aAAM,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;YACtB,MAAM,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QAC7B,CAAC;QAED,MAAM,CAAC,gBAAgB,EAAE,CAAC;QAC1B,MAAM,CAAC,aAAa,IAAI,IAAI,CAAC,MAAM,CAAC;IACtC,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,cAAc,CAAC,WAAmB;QACtC,MAAM,IAAI,KAAK,CAAC,gDAAgD,CAAC,CAAC;IACpE,CAAC;IAED;;;;OAIG;IACH,KAAK,CAAC,cAAc,CAAC,UAAkB;QACrC,IAAI,IAAI,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC;YACzB,MAAM,IAAI,KAAK,CAAC,gCAAgC,CAAC,CAAC;QACpD,CAAC;QAED,IAAI,CAAC,IAAI,CAAC,KAAK,EAAE,CAAC;YAChB,OAAO;QACT,CAAC;QAED,iDAAiD;QACjD,MAAM,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,qBAAqB,UAAU,GAAG,CAAC,CAAC;IAC9D,CAAC;IAED;;;;;OAKG;IACH,KAAK,CAAC,KAAK;QACT,IAAI,IAAI,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC;YACzB,MAAM,IAAI,KAAK,CAAC,+BAA+B,CAAC,CAAC;QACnD,CAAC;QAED,yBAAyB;QACzB,MAAM,IAAI,CAAC,KAAK,EAAE,CAAC;QAEnB,mCAAmC;QACnC,yGAAyG;QACzG,IAAI,EAAE,CAAC,UAAU,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,EAAE,CAAC;YACtC,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC;QAClE,CAAC;IACH,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,KAAK;QACT,gDAAgD;QAChD,IAAI,CAAC,UAAU,GAAG,IAAI,CAAC;QACvB,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC;IACpB,CAAC;CACF"}

package/dist/schema.d.ts

@@ -0,0 +1,52 @@
+/**
+ * LanceDB schema mapping
+ *
+ * Converts between RAGChunk and LanceDB row format.
+ */
+import type { RAGChunk } from '@vibe-agent-toolkit/rag';
+/**
+ * LanceDB row format
+ *
+ * LanceDB stores data in Apache Arrow format with specific type requirements.
+ * Must have index signature for LanceDB compatibility.
+ *
+ * Note: LanceDB/Arrow cannot infer array types if all values are null,
+ * so we use empty arrays instead of null for array fields.
+ */
+export interface LanceDBRow extends Record<string, unknown> {
+    vector: number[];
+    chunkId: string;
+    resourceId: string;
+    content: string;
+    contentHash: string;
+    resourceContentHash: string;
+    tokenCount: number;
+    headingPath: string;
+    headingLevel: number | null;
+    startLine: number | null;
+    endLine: number | null;
+    filePath: string;
+    tags: string;
+    type: string;
+    title: string;
+    embeddingModel: string;
+    embeddedAt: number;
+    previousChunkId: string;
+    nextChunkId: string;
+}
+/**
+ * Convert RAGChunk to LanceDB row format
+ *
+ * @param chunk - RAGChunk to convert
+ * @param resourceContentHash - Hash of the full resource content (for change detection)
+ * @returns LanceDB row
+ */
+export declare function chunkToLanceRow(chunk: RAGChunk, resourceContentHash: string): LanceDBRow;
+/**
+ * Convert LanceDB row to RAGChunk
+ *
+ * @param row - LanceDB row
+ * @returns RAGChunk
+ */
+export declare function lanceRowToChunk(row: LanceDBRow): RAGChunk;
+//# sourceMappingURL=schema.d.ts.map

package/dist/schema.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"schema.d.ts","sourceRoot":"","sources":["../src/schema.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,yBAAyB,CAAC;AAExD;;;;;;;;GAQG;AACH,MAAM,WAAW,UAAW,SAAQ,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC;IAEzD,MAAM,EAAE,MAAM,EAAE,CAAC;IAGjB,OAAO,EAAE,MAAM,CAAC;IAChB,UAAU,EAAE,MAAM,CAAC;IACnB,OAAO,EAAE,MAAM,CAAC;IAChB,WAAW,EAAE,MAAM,CAAC;IACpB,mBAAmB,EAAE,MAAM,CAAC;IAC5B,UAAU,EAAE,MAAM,CAAC;IAGnB,WAAW,EAAE,MAAM,CAAC;IACpB,YAAY,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IACzB,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IAGvB,QAAQ,EAAE,MAAM,CAAC;IACjB,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,EAAE,MAAM,CAAC;IAGd,cAAc,EAAE,MAAM,CAAC;IACvB,UAAU,EAAE,MAAM,CAAC;IAGnB,eAAe,EAAE,MAAM,CAAC;IACxB,WAAW,EAAE,MAAM,CAAC;CACrB;AAED;;;;;;GAMG;AACH,wBAAgB,eAAe,CAAC,KAAK,EAAE,QAAQ,EAAE,mBAAmB,EAAE,MAAM,GAAG,UAAU,CAsBxF;AAED;;;;;GAKG;AACH,wBAAgB,eAAe,CAAC,GAAG,EAAE,UAAU,GAAG,QAAQ,CA2BzD"}

package/dist/schema.js

@@ -0,0 +1,76 @@
+/**
+ * LanceDB schema mapping
+ *
+ * Converts between RAGChunk and LanceDB row format.
+ */
+/**
+ * Convert RAGChunk to LanceDB row format
+ *
+ * @param chunk - RAGChunk to convert
+ * @param resourceContentHash - Hash of the full resource content (for change detection)
+ * @returns LanceDB row
+ */
+export function chunkToLanceRow(chunk, resourceContentHash) {
+    return {
+        vector: chunk.embedding,
+        chunkId: chunk.chunkId,
+        resourceId: chunk.resourceId,
+        content: chunk.content,
+        contentHash: chunk.contentHash,
+        resourceContentHash,
+        tokenCount: chunk.tokenCount,
+        headingPath: chunk.headingPath ?? '', // Empty string for Arrow
+        headingLevel: chunk.headingLevel ?? null,
+        startLine: chunk.startLine ?? null,
+        endLine: chunk.endLine ?? null,
+        filePath: chunk.filePath,
+        tags: chunk.tags?.join(',') ?? '', // Convert array to comma-separated string
+        type: chunk.type ?? '', // Empty string for Arrow
+        title: chunk.title ?? '', // Empty string for Arrow
+        embeddingModel: chunk.embeddingModel,
+        embeddedAt: chunk.embeddedAt.getTime(),
+        previousChunkId: chunk.previousChunkId ?? '', // Empty string for Arrow
+        nextChunkId: chunk.nextChunkId ?? '', // Empty string for Arrow
+    };
+}
+/**
+ * Convert LanceDB row to RAGChunk
+ *
+ * @param row - LanceDB row
+ * @returns RAGChunk
+ */
+export function lanceRowToChunk(row) {
+    const chunk = {
+        chunkId: row.chunkId,
+        resourceId: row.resourceId,
+        content: row.content,
+        contentHash: row.contentHash,
+        tokenCount: row.tokenCount,
+        filePath: row.filePath,
+        embedding: row.vector,
+        embeddingModel: row.embeddingModel,
+        embeddedAt: new Date(row.embeddedAt),
+    };
+    // Only add optional properties if they exist (non-empty strings)
+    if (row.headingPath && row.headingPath.length > 0)
+        chunk.headingPath = row.headingPath;
+    if (row.headingLevel !== null)
+        chunk.headingLevel = row.headingLevel;
+    if (row.startLine !== null)
+        chunk.startLine = row.startLine;
+    if (row.endLine !== null)
+        chunk.endLine = row.endLine;
+    if (row.tags && row.tags.length > 0) {
+        chunk.tags = row.tags.split(','); // Convert comma-separated string back to array
+    }
+    if (row.type && row.type.length > 0)
+        chunk.type = row.type;
+    if (row.title && row.title.length > 0)
+        chunk.title = row.title;
+    if (row.previousChunkId && row.previousChunkId.length > 0)
+        chunk.previousChunkId = row.previousChunkId;
+    if (row.nextChunkId && row.nextChunkId.length > 0)
+        chunk.nextChunkId = row.nextChunkId;
+    return chunk;
+}
+//# sourceMappingURL=schema.js.map

package/dist/schema.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"schema.js","sourceRoot":"","sources":["../src/schema.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AA8CH;;;;;;GAMG;AACH,MAAM,UAAU,eAAe,CAAC,KAAe,EAAE,mBAA2B;IAC1E,OAAO;QACL,MAAM,EAAE,KAAK,CAAC,SAAS;QACvB,OAAO,EAAE,KAAK,CAAC,OAAO;QACtB,UAAU,EAAE,KAAK,CAAC,UAAU;QAC5B,OAAO,EAAE,KAAK,CAAC,OAAO;QACtB,WAAW,EAAE,KAAK,CAAC,WAAW;QAC9B,mBAAmB;QACnB,UAAU,EAAE,KAAK,CAAC,UAAU;QAC5B,WAAW,EAAE,KAAK,CAAC,WAAW,IAAI,EAAE,EAAE,yBAAyB;QAC/D,YAAY,EAAE,KAAK,CAAC,YAAY,IAAI,IAAI;QACxC,SAAS,EAAE,KAAK,CAAC,SAAS,IAAI,IAAI;QAClC,OAAO,EAAE,KAAK,CAAC,OAAO,IAAI,IAAI;QAC9B,QAAQ,EAAE,KAAK,CAAC,QAAQ;QACxB,IAAI,EAAE,KAAK,CAAC,IAAI,EAAE,IAAI,CAAC,GAAG,CAAC,IAAI,EAAE,EAAE,0CAA0C;QAC7E,IAAI,EAAE,KAAK,CAAC,IAAI,IAAI,EAAE,EAAE,yBAAyB;QACjD,KAAK,EAAE,KAAK,CAAC,KAAK,IAAI,EAAE,EAAE,yBAAyB;QACnD,cAAc,EAAE,KAAK,CAAC,cAAc;QACpC,UAAU,EAAE,KAAK,CAAC,UAAU,CAAC,OAAO,EAAE;QACtC,eAAe,EAAE,KAAK,CAAC,eAAe,IAAI,EAAE,EAAE,yBAAyB;QACvE,WAAW,EAAE,KAAK,CAAC,WAAW,IAAI,EAAE,EAAE,yBAAyB;KAChE,CAAC;AACJ,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,eAAe,CAAC,GAAe;IAC7C,MAAM,KAAK,GAAa;QACtB,OAAO,EAAE,GAAG,CAAC,OAAO;QACpB,UAAU,EAAE,GAAG,CAAC,UAAU;QAC1B,OAAO,EAAE,GAAG,CAAC,OAAO;QACpB,WAAW,EAAE,GAAG,CAAC,WAAW;QAC5B,UAAU,EAAE,GAAG,CAAC,UAAU;QAC1B,QAAQ,EAAE,GAAG,CAAC,QAAQ;QACtB,SAAS,EAAE,GAAG,CAAC,MAAM;QACrB,cAAc,EAAE,GAAG,CAAC,cAAc;QAClC,UAAU,EAAE,IAAI,IAAI,CAAC,GAAG,CAAC,UAAU,CAAC;KACrC,CAAC;IAEF,iEAAiE;IACjE,IAAI,GAAG,CAAC,WAAW,IAAI,GAAG,CAAC,WAAW,CAAC,MAAM,GAAG,CAAC;QAAE,KAAK,CAAC,WAAW,GAAG,GAAG,CAAC,WAAW,CAAC;IACvF,IAAI,GAAG,CAAC,YAAY,KAAK,IAAI;QAAE,KAAK,CAAC,YAAY,GAAG,GAAG,CAAC,YAAY,CAAC;IACrE,IAAI,GAAG,CAAC,SAAS,KAAK,IAAI;QAAE,KAAK,CAAC,SAAS,GAAG,GAAG,CAAC,SAAS,CAAC;IAC5D,IAAI,GAAG,CAAC,OAAO,KAAK,IAAI;QAAE,KAAK,CAAC,OAAO,GAAG,GAAG,CAAC,OAAO,CAAC;IACtD,IAAI,GAAG,CAAC,IAAI,IAAI,GAAG,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACpC,KAAK,CAAC,IAAI,GAAG,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,+CAA+C;IACnF,CAAC;IACD,IAAI,GAAG,CAAC,IAAI,IAAI,GAAG,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC;QAAE,KAAK,CAAC,IAAI,GAAG,GAAG,CAAC,IAAI,CAAC;IAC3D,IAAI,GAAG,CAAC,KAAK,IAAI,GAAG,CAAC,KAAK,CAAC,MAAM,GAAG,CAAC;QAAE,KAAK,CAAC,KAAK,GAAG,GAAG,CAAC,KAAK,CAAC;IAC/D,IAAI,GAAG,CAAC,eAAe,IAAI,GAAG,CAAC,eAAe,CAAC,MAAM,GAAG,CAAC;QAAE,KAAK,CAAC,eAAe,GAAG,GAAG,CAAC,eAAe,CAAC;IACvG,IAAI,GAAG,CAAC,WAAW,IAAI,GAAG,CAAC,WAAW,CAAC,MAAM,GAAG,CAAC;QAAE,KAAK,CAAC,WAAW,GAAG,GAAG,CAAC,WAAW,CAAC;IAEvF,OAAO,KAAK,CAAC;AACf,CAAC"}

package/package.json

@@ -0,0 +1,50 @@
+{
+  "name": "@vibe-agent-toolkit/rag-lancedb",
+  "version": "0.1.0-rc.7",
+  "type": "module",
+  "description": "LanceDB implementation of RAG interfaces for vibe-agent-toolkit",
+  "keywords": [
+    "rag",
+    "vector-search",
+    "lancedb",
+    "embeddings"
+  ],
+  "author": "Jeff Dutton",
+  "license": "MIT",
+  "main": "./dist/index.js",
+  "types": "./dist/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.js"
+    }
+  },
+  "files": [
+    "dist",
+    "README.md"
+  ],
+  "scripts": {
+    "build": "tsc",
+    "clean": "rimraf dist",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "test:integration": "vitest run --config vitest.integration.config.ts",
+    "test:system": "vitest run --config vitest.system.config.ts",
+    "typecheck": "tsc --noEmit"
+  },
+  "dependencies": {
+    "@vibe-agent-toolkit/rag": "workspace:*",
+    "@vibe-agent-toolkit/resources": "workspace:*",
+    "@vibe-agent-toolkit/utils": "workspace:*",
+    "vectordb": "^0.4.0"
+  },
+  "devDependencies": {
+    "@types/node": "^25.0.0",
+    "typescript": "^5.9.0",
+    "vitest": "^3.2.0"
+  },
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/jdutton/vibe-agent-toolkit.git"
+  }
+}