@mastra/duckdb 0.0.0

package/README.md

@@ -0,0 +1,253 @@
+# @mastra/duckdb
+
+DuckDB vector store implementation for Mastra, providing high-performance embedded vector similarity search with HNSW indexing. No external server required - runs entirely in-process.
+
+## Installation
+
+```bash
+npm install @mastra/duckdb
+```
+
+## Usage
+
+### Vector Store
+
+```typescript
+import { DuckDBVector } from '@mastra/duckdb';
+
+// Create a vector store with in-memory database
+const vectorStore = new DuckDBVector({
+  id: 'my-vector-store',
+  path: ':memory:', // or './vectors.duckdb' for persistence
+});
+
+// Create a new index with vector support
+await vectorStore.createIndex({
+  indexName: 'my_vectors',
+  dimension: 1536,
+  metric: 'cosine',
+});
+
+// Add vectors with metadata
+const ids = await vectorStore.upsert({
+  indexName: 'my_vectors',
+  vectors: [
+    [0.1, 0.2, 0.3],
+    [0.3, 0.4, 0.5],
+  ], // truncated - use actual 1536-dim vectors
+  metadata: [
+    { text: 'doc1', category: 'A' },
+    { text: 'doc2', category: 'B' },
+  ],
+});
+
+// Query similar vectors
+const results = await vectorStore.query({
+  indexName: 'my_vectors',
+  queryVector: [0.1, 0.2, 0.3], // truncated - use actual 1536-dim vector
+  topK: 10,
+  filter: { category: 'A' },
+  includeVector: false,
+});
+
+// Clean up
+await vectorStore.close();
+```
+
+### With RAG Pipeline
+
+```typescript
+import { Mastra } from '@mastra/core';
+import { DuckDBVector } from '@mastra/duckdb';
+
+const vectorStore = new DuckDBVector({
+  id: 'rag-store',
+  path: './rag-vectors.duckdb',
+});
+
+// Use with Mastra's RAG system
+const mastra = new Mastra({
+  vectors: {
+    ragStore: vectorStore,
+  },
+});
+```
+
+## Configuration
+
+### Constructor Options
+
+| Option       | Type                                      | Default      | Description                                               |
+| ------------ | ----------------------------------------- | ------------ | --------------------------------------------------------- |
+| `id`         | `string`                                  | required     | Unique identifier for the vector store instance           |
+| `path`       | `string`                                  | `':memory:'` | Database file path. Use `:memory:` for in-memory database |
+| `dimensions` | `number`                                  | `1536`       | Default dimension for vector embeddings                   |
+| `metric`     | `'cosine' \| 'euclidean' \| 'dotproduct'` | `'cosine'`   | Default distance metric for similarity search             |
+
+### Example Configurations
+
+```typescript
+// In-memory (fast, non-persistent)
+const memoryStore = new DuckDBVector({
+  id: 'memory-store',
+  path: ':memory:',
+});
+
+// File-based (persistent)
+const fileStore = new DuckDBVector({
+  id: 'file-store',
+  path: './data/vectors.duckdb',
+  dimensions: 768,
+  metric: 'cosine',
+});
+
+// With euclidean distance
+const euclideanStore = new DuckDBVector({
+  id: 'euclidean-store',
+  path: ':memory:',
+  metric: 'euclidean',
+});
+```
+
+## Features
+
+### Vector Store Features
+
+- Embedded database with no external server required
+- HNSW indexing for fast approximate nearest neighbor search
+- Vector similarity search with cosine, euclidean, and dot product metrics
+- Advanced metadata filtering with MongoDB-like query syntax
+- Automatic UUID generation for vectors
+- File-based persistence or in-memory operation
+- Table management (create, list, describe, delete)
+
+### Key Benefits
+
+- **Zero infrastructure** - No database server to manage
+- **High performance** - HNSW indexing with configurable parameters
+- **SQL interface** - Familiar query language for metadata filtering
+- **Parquet support** - Native import/export capabilities
+- **Low memory footprint** - Efficient resource usage
+
+## Supported Filter Operators
+
+The following filter operators are supported for metadata queries:
+
+- Comparison: `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`
+- Logical: `$and`, `$or`, `$not`, `$nor`
+- Array: `$in`, `$nin`
+- Element: `$exists`
+- Text: `$contains`
+
+### Filter Examples
+
+```typescript
+// Simple equality
+const results = await vectorStore.query({
+  indexName: 'docs',
+  queryVector: [0.1, 0.2, 0.3], // truncated - use actual embedding vector
+  filter: { category: 'technology' },
+});
+
+// Comparison operators
+const results = await vectorStore.query({
+  indexName: 'docs',
+  queryVector: [0.1, 0.2, 0.3], // truncated - use actual embedding vector
+  filter: { price: { $gt: 100, $lte: 500 } },
+});
+
+// Logical operators
+const results = await vectorStore.query({
+  indexName: 'docs',
+  queryVector: [0.1, 0.2, 0.3], // truncated - use actual embedding vector
+  filter: {
+    $and: [{ category: 'electronics' }, { $or: [{ brand: 'Apple' }, { brand: 'Samsung' }] }],
+  },
+});
+
+// Array operators
+const results = await vectorStore.query({
+  indexName: 'docs',
+  queryVector: [0.1, 0.2, 0.3], // truncated - use actual embedding vector
+  filter: { tags: { $in: ['featured', 'sale'] } },
+});
+
+// Nested field access
+const results = await vectorStore.query({
+  indexName: 'docs',
+  queryVector: [0.1, 0.2, 0.3], // truncated - use actual embedding vector
+  filter: { 'user.profile.tier': 'premium' },
+});
+```
+
+## Vector Store Methods
+
+### Index Management
+
+- `createIndex({ indexName, dimension, metric? })`: Create a new table with vector support and optional HNSW index
+- `listIndexes()`: List all vector-enabled tables
+- `describeIndex({ indexName })`: Get table statistics (dimension, count, metric)
+- `deleteIndex({ indexName })`: Delete a table and its data
+
+### Vector Operations
+
+- `upsert({ indexName, vectors, metadata?, ids? })`: Add or update vectors
+- `query({ indexName, queryVector, topK?, filter?, includeVector? })`: Search for similar vectors
+- `updateVector({ indexName, id?, filter?, update })`: Update a vector by ID or metadata filter
+- `deleteVector({ indexName, id })`: Delete a single vector by ID
+- `deleteVectors({ indexName, ids?, filter? })`: Delete multiple vectors by IDs or metadata filter
+
+### Connection Management
+
+- `close()`: Close the database connection
+
+## Distance Metrics
+
+| Metric       | Description       | Score Range         | Best For                            |
+| ------------ | ----------------- | ------------------- | ----------------------------------- |
+| `cosine`     | Cosine similarity | 0-1 (1 = identical) | Text embeddings, normalized vectors |
+| `euclidean`  | L2 distance       | 0-∞ (0 = identical) | Image embeddings, spatial data      |
+| `dotproduct` | Inner product     | -∞ to ∞             | When magnitude matters              |
+
+## Use Cases
+
+### Embedded Semantic Search
+
+Build offline-capable AI applications with semantic search that runs entirely in-process without external dependencies.
+
+```typescript
+const vectorStore = new DuckDBVector({
+  id: 'semantic-search',
+  path: './search.duckdb',
+});
+```
+
+### Local RAG Pipelines
+
+Process sensitive documents locally without sending data to cloud vector databases.
+
+```typescript
+const vectorStore = new DuckDBVector({
+  id: 'local-rag',
+  path: './private-docs.duckdb',
+  dimensions: 1536, // OpenAI embeddings
+});
+```
+
+### Development and Testing
+
+Rapidly prototype vector search features with zero infrastructure setup.
+
+```typescript
+const vectorStore = new DuckDBVector({
+  id: 'dev-store',
+  path: ':memory:', // Fast in-memory for testing
+});
+```
+
+## Related Links
+
+- [DuckDB Documentation](https://duckdb.org/docs/)
+- [DuckDB VSS Extension](https://duckdb.org/docs/extensions/vss)
+- [Mastra Documentation](https://mastra.ai/docs)
+- [GitHub Issue #8140](https://github.com/mastra-ai/mastra/issues/8140)