native-vector-store 0.2.0 → 0.3.0

package/README.md

@@ -2,6 +2,17 @@
 
 High-performance vector store with SIMD optimization for MCP servers and local RAG applications.
 
+## Design Philosophy
+
+This vector store is designed for **immutable, one-time loading** scenarios common in modern cloud deployments:
+
+- **📚 Load Once, Query Many**: Documents are loaded at startup and remain immutable during serving
+- **🚀 Optimized for Cold Starts**: Perfect for serverless functions and containerized deployments
+- **📁 File-Based Organization**: Leverages filesystem for natural document organization and versioning
+- **🎯 Focused API**: Does one thing exceptionally well - fast similarity search over focused corpora (sweet spot: <100k documents)
+
+This design eliminates complex state management, ensures consistent performance, and aligns perfectly with cloud-native deployment patterns where domain-specific knowledge bases are the norm.
+
 ## Features
 
 - **🚀 High Performance**: C++ implementation with OpenMP SIMD optimization
@@ -17,7 +28,7 @@ High-performance vector store with SIMD optimization for MCP servers and local R
 - **Load Time**: <1 second for 100,000 documents (achieved: ~560ms)
 - **Search Latency**: <10ms for top-k similarity search (achieved: 1-2ms)
 - **Memory Efficiency**: Minimal fragmentation via arena allocation
-- **Scalability**: Designed for <1M embeddings
+- **Scalability**: Designed for focused corpora (<100k documents optimal, <1M maximum)
 - **Throughput**: 178k+ documents per second with parallel loading
 
 ## Installation
@@ -76,6 +87,164 @@ const results = store.search(queryEmbedding, 5); // Top 5 results
 console.log(results[0]); // { score: 0.95, id: 'doc-1', text: '...', metadata_json: '...' }
 ```
 
+## Usage Patterns
+
+### Serverless Deployment (AWS Lambda, Vercel)
+
+```javascript
+// Initialize once during cold start
+let store;
+
+async function initializeStore() {
+  if (!store) {
+    store = new VectorStore(1536);
+    store.loadDir('./knowledge-base'); // Loads and finalizes
+  }
+  return store;
+}
+
+// Handler reuses the store across invocations
+export async function handler(event) {
+  const store = await initializeStore();
+  const embedding = new Float32Array(event.embedding);
+  return store.search(embedding, 10);
+}
+```
+
+### Local MCP Server
+
+```javascript
+const { VectorStore } = require('native-vector-store');
+
+// Load different knowledge domains at startup
+const stores = {
+  products: new VectorStore(1536),
+  support: new VectorStore(1536),
+  general: new VectorStore(1536)
+};
+
+stores.products.loadDir('./knowledge/products');
+stores.support.loadDir('./knowledge/support');
+stores.general.loadDir('./knowledge/general');
+
+// Route searches to appropriate domain
+server.on('search', (query) => {
+  const store = stores[query.domain] || stores.general;
+  const results = store.search(query.embedding, 5);
+  return results.filter(r => r.score > 0.7);
+});
+```
+
+### CLI Tool with Persistent Context
+
+```javascript
+#!/usr/bin/env node
+const { VectorStore } = require('native-vector-store');
+
+// Load knowledge base once
+const store = new VectorStore(1536);
+store.loadDir(process.env.KNOWLEDGE_PATH || './docs');
+
+// Interactive REPL with fast responses
+const repl = require('repl');
+const r = repl.start('> ');
+r.context.search = (embedding, k = 5) => store.search(embedding, k);
+```
+
+### File Organization Best Practices
+
+Structure your documents by category for separate vector stores:
+
+```
+knowledge-base/
+├── products/          # Product documentation
+│   ├── api-reference.json
+│   └── user-guide.json
+├── support/           # Support articles
+│   ├── faq.json
+│   └── troubleshooting.json
+└── context/           # Context-specific docs
+    ├── company-info.json
+    └── policies.json
+```
+
+Load each category into its own VectorStore:
+
+```javascript
+// Create separate stores for different domains
+const productStore = new VectorStore(1536);
+const supportStore = new VectorStore(1536);
+const contextStore = new VectorStore(1536);
+
+// Load each category independently
+productStore.loadDir('./knowledge-base/products');
+supportStore.loadDir('./knowledge-base/support');
+contextStore.loadDir('./knowledge-base/context');
+
+// Search specific domains
+const productResults = productStore.search(queryEmbedding, 5);
+const supportResults = supportStore.search(queryEmbedding, 5);
+```
+
+Each JSON file contains self-contained documents with embeddings:
+
+```json
+{
+  "id": "unique-id",
+  "text": "Document content...",
+  "metadata": {
+    "embedding": [0.1, 0.2, ...],
+    "category": "product",
+    "lastUpdated": "2024-01-01"
+  }
+}
+```
+
+### Deployment Strategies
+
+#### Blue-Green Deployment
+
+```javascript
+// Load new version without downtime
+const newStore = new VectorStore(1536);
+newStore.loadDir('./knowledge-base-v2');
+
+// Atomic switch
+app.locals.store = newStore;
+```
+
+#### Versioned Directories
+
+```
+deployments/
+├── v1.0.0/
+│   └── documents/
+├── v1.1.0/
+│   └── documents/
+└── current -> v1.1.0  # Symlink to active version
+```
+
+#### Watch for Updates (Development)
+
+```javascript
+const fs = require('fs');
+
+function reloadStore() {
+  const newStore = new VectorStore(1536);
+  newStore.loadDir('./documents');
+  global.store = newStore;
+  console.log(`Reloaded ${newStore.size()} documents`);
+}
+
+// Initial load
+reloadStore();
+
+// Watch for changes in development
+if (process.env.NODE_ENV === 'development') {
+  fs.watch('./documents', { recursive: true }, reloadStore);
+}
+```
+
 ## MCP Server Integration
 
 Perfect for building local RAG capabilities in MCP servers:
@@ -148,6 +317,60 @@ Check if the store has been finalized and is ready for searching.
 ##### `size(): number`
 Get the number of documents in the store.
 
+## Performance
+
+### Why It's Fast
+
+The native-vector-store achieves exceptional performance through:
+
+1. **Producer-Consumer Loading**: Parallel file I/O and JSON parsing achieve 178k+ documents/second
+2. **SIMD Optimizations**: OpenMP vectorization for dot product calculations
+3. **Arena Allocation**: Contiguous memory layout with 64MB chunks for cache efficiency
+4. **Zero-Copy Design**: String views and pre-allocated buffers minimize allocations
+5. **Two-Phase Architecture**: Loading phase allows concurrent writes, serving phase optimizes for reads
+
+### Benchmarks
+
+Performance on typical hardware (M1 MacBook Pro):
+
+| Operation | Documents | Time | Throughput |
+|-----------|-----------|------|------------|
+| Loading (from disk) | 100,000 | ~560ms | 178k docs/sec |
+| Search (k=10) | 10,000 corpus | 1-2ms | 500-1000 queries/sec |
+| Search (k=100) | 100,000 corpus | 8-12ms | 80-125 queries/sec |
+| Normalization | 100,000 | <100ms | 1M+ docs/sec |
+
+### Performance Tips
+
+1. **Optimal File Organization**: 
+   - Keep 1000-10000 documents per JSON file for best I/O performance
+   - Use arrays of documents in each file rather than one file per document
+
+2. **Memory Considerations**:
+   - Each document requires: `embedding_size * 4 bytes + metadata_size + text_size`
+   - 100k documents with 1536-dim embeddings ≈ 600MB embeddings + metadata
+
+3. **Search Performance**:
+   - Scales linearly with corpus size and k value
+   - Use smaller k values (5-20) for interactive applications
+   - Pre-normalize query embeddings if making multiple searches
+
+4. **Corpus Size Optimization**:
+   - Sweet spot: <100k documents for optimal load/search balance
+   - Beyond 100k: Consider if your use case truly needs all documents
+   - Focus on curated, domain-specific content rather than exhaustive datasets
+
+### Comparison with Alternatives
+
+| Feature | native-vector-store | Faiss | ChromaDB | Pinecone |
+|---------|-------------------|--------|----------|----------|
+| Load 100k docs | <1s | 2-5s | 30-60s | N/A (API) |
+| Search latency | 1-2ms | 0.5-1ms | 50-200ms | 50-300ms |
+| Memory efficiency | High | Medium | Low | N/A |
+| Dependencies | Minimal | Heavy | Heavy | None |
+| Deployment | Simple | Complex | Complex | SaaS |
+| Sweet spot | <100k docs | Any size | Any size | Any size |
+
 ## Building from Source
 
 ```bash
@@ -188,21 +411,21 @@ npm run example
 
 ### MCP Servers
 Ideal for building local RAG (Retrieval-Augmented Generation) capabilities:
-- Fast document loading from knowledge bases
+- Fast document loading from focused knowledge bases
 - Low-latency similarity search for context retrieval
-- Memory-efficient storage for large document collections
+- Memory-efficient storage for domain-specific corpora
 
 ### Knowledge Management
 Perfect for personal knowledge management systems:
-- Index personal documents and notes
-- Fast semantic search across content
+- Index personal documents and notes (typically <10k documents)
+- Fast semantic search across focused content
 - Offline operation without external dependencies
 
 ### Research Applications
-Suitable for academic and research projects:
-- Literature review and citation analysis
-- Semantic clustering of research papers
-- Cross-reference discovery in document collections
+Suitable for academic and research projects with focused datasets:
+- Literature review within specific domains
+- Semantic clustering of curated paper collections
+- Cross-reference discovery in specialized corpora
 
 ## Contributing
 

package/binding.gyp

@@ -2,11 +2,12 @@
   "targets": [
     {
       "target_name": "vector_store",
-      "sources": ["src/binding.cc", "src/vector_store.cpp", "src/vector_store_loader.cpp", "src/vector_store_loader_mmap.cpp", "src/vector_store_loader_adaptive.cpp", "deps/simdjson.cpp"],
+      "sources": ["src/binding.cc", "src/vector_store.cpp", "src/vector_store_loader.cpp", "src/vector_store_loader_mmap.cpp", "src/vector_store_loader_adaptive.cpp", "deps/simdjson/simdjson.cpp"],
       "include_dirs": [
         "<!@(node -p \"require('node-addon-api').include\")",
         "src",
-        "deps"
+        "deps/simdjson",
+        "deps/atomic_queue"
       ],
       "dependencies": ["<!(node -p \"require('node-addon-api').gyp\")"],
       "cflags_cc": [

package/docs/PERFORMANCE_CASE_STUDY.md

@@ -0,0 +1,130 @@
+# Performance Case Study: Adaptive File Loading in Native Vector Store
+
+## Executive Summary
+
+We achieved significant performance improvements in our native vector store by implementing an adaptive file loading strategy that automatically selects the optimal I/O method based on file size. This resulted in up to **3x faster loading times** for typical workloads while maintaining simplicity for users.
+
+## Background
+
+Our vector store loads JSON documents containing embeddings from disk. Initial implementation used standard file I/O with buffering, which performed well but had room for improvement, especially when dealing with directories containing many files of varying sizes.
+
+## The Challenge
+
+We discovered that optimal file loading strategies vary dramatically based on file characteristics:
+- **Large files (>5MB)**: Sequential reads with pre-allocated buffers perform best
+- **Small files (<5MB)**: Memory-mapped I/O significantly reduces overhead
+
+## Implementation Journey
+
+### Phase 1: Baseline Optimization
+First, we optimized the standard loader:
+- Pre-allocated reusable buffers (1MB initial, grows as needed)
+- Used `filesystem::file_size()` to avoid redundant syscalls
+- Implemented producer-consumer pattern with lock-free queues
+
+**Result**: ~10-15% improvement over naive implementation
+
+### Phase 2: Memory-Mapped I/O
+We added memory-mapped file support:
+- Zero-copy access to file data
+- Cross-platform support (mmap on POSIX, MapViewOfFile on Windows)
+- Eliminated buffer allocation overhead
+
+**Result**: Mixed results - faster for small files, slower for large files
+
+### Phase 3: Adaptive Strategy
+The key insight was that no single approach works best for all cases:
+
+```cpp
+// Adaptive loader chooses the best method per file
+constexpr size_t SIZE_THRESHOLD = 5 * 1024 * 1024; // 5MB
+
+for (const auto& file_info : file_infos) {
+    if (file_info.size < SIZE_THRESHOLD) {
+        // Use memory mapping for small files
+        load_with_mmap(file_info);
+    } else {
+        // Use standard I/O for large files
+        load_with_standard_io(file_info);
+    }
+}
+```
+
+## Benchmark Results
+
+### Test Dataset 1: Large Files (2 files, ~340MB total)
+| Method | Load Time | Relative Performance |
+|--------|-----------|---------------------|
+| Standard Loader | 731ms | 1.0x (baseline) |
+| Memory-Mapped | 1070ms | 0.68x (slower) |
+| **Adaptive** | **735ms** | **0.99x** |
+
+### Test Dataset 2: Partitioned Files (66 files, ~340MB total)
+| Method | Load Time | Relative Performance |
+|--------|-----------|---------------------|
+| Standard Loader | 415ms | 1.0x (baseline) |
+| Memory-Mapped | 283ms | 1.47x (faster) |
+| **Adaptive** | **278ms** | **1.49x (faster)** |
+
+### Test Dataset 3: Small Files (465 files, ~45MB total)
+| Method | Load Time | Relative Performance |
+|--------|-----------|---------------------|
+| Standard Loader | 146ms | 1.0x (baseline) |
+| Memory-Mapped | 51ms | 2.86x (faster) |
+| **Adaptive** | **49ms** | **2.98x (faster)** |
+
+## Key Findings
+
+1. **File size matters more than total data volume**
+   - Memory mapping excels with many small files
+   - Standard I/O wins for few large files
+
+2. **The 5MB threshold is optimal**
+   - Below 5MB: Memory mapping eliminates per-file overhead
+   - Above 5MB: SimdJSON's padding requirement negates mmap benefits
+
+3. **Adaptive loading provides consistent best performance**
+   - Automatically selects optimal strategy
+   - No configuration required
+   - Negligible decision overhead (<1μs per file)
+
+## Technical Details
+
+### Why Memory Mapping Helps Small Files
+- Eliminates buffer allocation (saves ~1-2ms per file)
+- OS handles caching and prefetching
+- Reduces memory copies
+
+### Why Standard I/O Helps Large Files
+- SimdJSON requires padded strings, forcing a copy anyway
+- Sequential reads are highly optimized by OS
+- Single large allocation is more efficient than mmap overhead
+
+### Thread Safety Considerations
+- Both strategies use the same producer-consumer pattern
+- Lock-free atomic queues for work distribution
+- No overlapping OpenMP regions (prevents TSAN warnings)
+
+## Usage
+
+The adaptive loader is now the default:
+
+```javascript
+const store = new VectorStore(1536);
+store.loadDir('./documents'); // Automatically uses adaptive strategy
+```
+
+For specific use cases, individual strategies remain available:
+```javascript
+store.loadDirMMap('./documents');     // Force memory mapping
+store.loadDirAdaptive('./documents'); // Explicit adaptive
+```
+
+## Conclusion
+
+By implementing an adaptive loading strategy, we achieved:
+- **Up to 3x faster loading** for typical workloads
+- **Zero configuration** - it just works
+- **Consistent performance** across diverse file distributions
+
+The lesson: Sometimes the best optimization is knowing when to use which technique. Our adaptive loader makes this decision automatically, giving users optimal performance without complexity.

package/docs/PREBUILDS.md

@@ -0,0 +1,69 @@
+# Prebuilt Binaries
+
+This package includes prebuilt binaries for common platforms to make installation easier. Users don't need build tools or system dependencies when installing from npm if their platform is supported.
+
+## Supported Platforms
+
+Prebuilds are automatically created for:
+- **Linux**: x64, arm64
+- **macOS**: x64, arm64 (including Apple Silicon)
+- **Windows**: x64
+
+## How It Works
+
+1. When users run `npm install native-vector-store`, the install script uses `node-gyp-build`
+2. `node-gyp-build` checks if a prebuild exists for the current platform
+3. If found, it uses the prebuild (fast, no compilation needed)
+4. If not found, it falls back to building from source
+
+## Building Prebuilds
+
+### Locally (for current platform)
+```bash
+npm run prebuildify
+```
+
+### For all platforms (using GitHub Actions)
+1. Push a tag starting with 'v' (e.g., v0.1.0)
+2. GitHub Actions will automatically build for all platforms
+3. Prebuilds will be attached to the GitHub release
+
+### Manual trigger
+You can also manually trigger the prebuild workflow from the Actions tab on GitHub.
+
+## Including Prebuilds in npm Package
+
+The prebuilds are automatically included when you run `npm publish`. The directory structure is:
+```
+native-vector-store/
+├── prebuilds/
+│   ├── linux-x64/
+│   ├── linux-arm64/
+│   ├── darwin-x64/
+│   ├── darwin-arm64/
+│   └── win32-x64/
+└── ... other files
+```
+
+## Fallback Behavior
+
+If a prebuild isn't available, users will need:
+- C++17 compatible compiler
+- simdjson library
+- OpenMP support
+- Python and build tools
+
+## Testing Prebuilds
+
+After building prebuilds:
+```bash
+# Test that it loads correctly
+node -e "console.log(require('.'))"
+```
+
+## Troubleshooting
+
+If prebuilds aren't working:
+1. Check that `node-gyp-build` is in dependencies (not devDependencies)
+2. Ensure prebuilds/ directory is not in .npmignore
+3. Verify the binary names match node-gyp-build expectations

package/docs/VectorStore.html

@@ -0,0 +1,180 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="utf-8">
+    <title>JSDoc: Class: VectorStore</title>
+
+    <script src="scripts/prettify/prettify.js"> </script>
+    <script src="scripts/prettify/lang-css.js"> </script>
+    <!--[if lt IE 9]>
+      <script src="//html5shiv.googlecode.com/svn/trunk/html5.js"></script>
+    <![endif]-->
+    <link type="text/css" rel="stylesheet" href="styles/prettify-tomorrow.css">
+    <link type="text/css" rel="stylesheet" href="styles/jsdoc-default.css">
+</head>
+
+<body>
+
+<div id="main">
+
+    <h1 class="page-title">Class: VectorStore</h1>
+
+    
+
+
+
+
+<section>
+
+<header>
+    
+        <h2><span class="attribs"><span class="type-signature"></span></span>VectorStore<span class="signature">()</span><span class="type-signature"></span></h2>
+        
+    
+</header>
+
+<article>
+    <div class="container-overview">
+    
+        
+
+    
+
+    
+    <h4 class="name" id="VectorStore"><span class="type-signature"></span>new VectorStore<span class="signature">()</span><span class="type-signature"></span></h4>
+    
+
+    
+
+
+
+<div class="description">
+    <p>High-performance vector store with SIMD optimization for similarity search.
+Designed for immutable, one-time loading scenarios with fast searches over focused corpora.</p>
+</div>
+
+
+
+
+
+
+
+
+
+
+
+
+
+<dl class="details">
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+</dl>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+    <h5>Examples</h5>
+    
+    <pre class="prettyprint"><code>// Basic usage
+const store = new VectorStore(1536);
+store.loadDir('./documents');
+const results = store.search(queryEmbedding, 10);</code></pre>
+
+    <pre class="prettyprint"><code>// Multiple domain-specific stores
+const productStore = new VectorStore(1536);
+const supportStore = new VectorStore(1536);
+productStore.loadDir('./knowledge/products');
+supportStore.loadDir('./knowledge/support');</code></pre>
+
+
+
+    
+    </div>
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+</article>
+
+</section>
+
+
+
+
+</div>
+
+<nav>
+    <h2><a href="index.html">Home</a></h2><h3>Classes</h3><ul><li><a href="VectorStore.html">VectorStore</a></li><li><a href="VectorStoreWrapper.html">VectorStoreWrapper</a></li></ul><h3><a href="global.html">Global</a></h3>
+</nav>
+
+<br class="clear">
+
+<footer>
+    Documentation generated by <a href="https://github.com/jsdoc/jsdoc">JSDoc 4.0.4</a>
+</footer>
+
+<script> prettyPrint(); </script>
+<script src="scripts/linenumber.js"> </script>
+</body>
+</html>