native-vector-store 0.1.0 → 0.3.0

package/docs/PERFORMANCE_CASE_STUDY.md

@@ -0,0 +1,130 @@
+# Performance Case Study: Adaptive File Loading in Native Vector Store
+
+## Executive Summary
+
+We achieved significant performance improvements in our native vector store by implementing an adaptive file loading strategy that automatically selects the optimal I/O method based on file size. This resulted in up to **3x faster loading times** for typical workloads while maintaining simplicity for users.
+
+## Background
+
+Our vector store loads JSON documents containing embeddings from disk. Initial implementation used standard file I/O with buffering, which performed well but had room for improvement, especially when dealing with directories containing many files of varying sizes.
+
+## The Challenge
+
+We discovered that optimal file loading strategies vary dramatically based on file characteristics:
+- **Large files (>5MB)**: Sequential reads with pre-allocated buffers perform best
+- **Small files (<5MB)**: Memory-mapped I/O significantly reduces overhead
+
+## Implementation Journey
+
+### Phase 1: Baseline Optimization
+First, we optimized the standard loader:
+- Pre-allocated reusable buffers (1MB initial, grows as needed)
+- Used `filesystem::file_size()` to avoid redundant syscalls
+- Implemented producer-consumer pattern with lock-free queues
+
+**Result**: ~10-15% improvement over naive implementation
+
+### Phase 2: Memory-Mapped I/O
+We added memory-mapped file support:
+- Zero-copy access to file data
+- Cross-platform support (mmap on POSIX, MapViewOfFile on Windows)
+- Eliminated buffer allocation overhead
+
+**Result**: Mixed results - faster for small files, slower for large files
+
+### Phase 3: Adaptive Strategy
+The key insight was that no single approach works best for all cases:
+
+```cpp
+// Adaptive loader chooses the best method per file
+constexpr size_t SIZE_THRESHOLD = 5 * 1024 * 1024; // 5MB
+
+for (const auto& file_info : file_infos) {
+    if (file_info.size < SIZE_THRESHOLD) {
+        // Use memory mapping for small files
+        load_with_mmap(file_info);
+    } else {
+        // Use standard I/O for large files
+        load_with_standard_io(file_info);
+    }
+}
+```
+
+## Benchmark Results
+
+### Test Dataset 1: Large Files (2 files, ~340MB total)
+| Method | Load Time | Relative Performance |
+|--------|-----------|---------------------|
+| Standard Loader | 731ms | 1.0x (baseline) |
+| Memory-Mapped | 1070ms | 0.68x (slower) |
+| **Adaptive** | **735ms** | **0.99x** |
+
+### Test Dataset 2: Partitioned Files (66 files, ~340MB total)
+| Method | Load Time | Relative Performance |
+|--------|-----------|---------------------|
+| Standard Loader | 415ms | 1.0x (baseline) |
+| Memory-Mapped | 283ms | 1.47x (faster) |
+| **Adaptive** | **278ms** | **1.49x (faster)** |
+
+### Test Dataset 3: Small Files (465 files, ~45MB total)
+| Method | Load Time | Relative Performance |
+|--------|-----------|---------------------|
+| Standard Loader | 146ms | 1.0x (baseline) |
+| Memory-Mapped | 51ms | 2.86x (faster) |
+| **Adaptive** | **49ms** | **2.98x (faster)** |
+
+## Key Findings
+
+1. **File size matters more than total data volume**
+   - Memory mapping excels with many small files
+   - Standard I/O wins for few large files
+
+2. **The 5MB threshold is optimal**
+   - Below 5MB: Memory mapping eliminates per-file overhead
+   - Above 5MB: SimdJSON's padding requirement negates mmap benefits
+
+3. **Adaptive loading provides consistent best performance**
+   - Automatically selects optimal strategy
+   - No configuration required
+   - Negligible decision overhead (<1μs per file)
+
+## Technical Details
+
+### Why Memory Mapping Helps Small Files
+- Eliminates buffer allocation (saves ~1-2ms per file)
+- OS handles caching and prefetching
+- Reduces memory copies
+
+### Why Standard I/O Helps Large Files
+- SimdJSON requires padded strings, forcing a copy anyway
+- Sequential reads are highly optimized by OS
+- Single large allocation is more efficient than mmap overhead
+
+### Thread Safety Considerations
+- Both strategies use the same producer-consumer pattern
+- Lock-free atomic queues for work distribution
+- No overlapping OpenMP regions (prevents TSAN warnings)
+
+## Usage
+
+The adaptive loader is now the default:
+
+```javascript
+const store = new VectorStore(1536);
+store.loadDir('./documents'); // Automatically uses adaptive strategy
+```
+
+For specific use cases, individual strategies remain available:
+```javascript
+store.loadDirMMap('./documents');     // Force memory mapping
+store.loadDirAdaptive('./documents'); // Explicit adaptive
+```
+
+## Conclusion
+
+By implementing an adaptive loading strategy, we achieved:
+- **Up to 3x faster loading** for typical workloads
+- **Zero configuration** - it just works
+- **Consistent performance** across diverse file distributions
+
+The lesson: Sometimes the best optimization is knowing when to use which technique. Our adaptive loader makes this decision automatically, giving users optimal performance without complexity.

package/docs/PREBUILDS.md

@@ -0,0 +1,69 @@
+# Prebuilt Binaries
+
+This package includes prebuilt binaries for common platforms to make installation easier. Users don't need build tools or system dependencies when installing from npm if their platform is supported.
+
+## Supported Platforms
+
+Prebuilds are automatically created for:
+- **Linux**: x64, arm64
+- **macOS**: x64, arm64 (including Apple Silicon)
+- **Windows**: x64
+
+## How It Works
+
+1. When users run `npm install native-vector-store`, the install script uses `node-gyp-build`
+2. `node-gyp-build` checks if a prebuild exists for the current platform
+3. If found, it uses the prebuild (fast, no compilation needed)
+4. If not found, it falls back to building from source
+
+## Building Prebuilds
+
+### Locally (for current platform)
+```bash
+npm run prebuildify
+```
+
+### For all platforms (using GitHub Actions)
+1. Push a tag starting with 'v' (e.g., v0.1.0)
+2. GitHub Actions will automatically build for all platforms
+3. Prebuilds will be attached to the GitHub release
+
+### Manual trigger
+You can also manually trigger the prebuild workflow from the Actions tab on GitHub.
+
+## Including Prebuilds in npm Package
+
+The prebuilds are automatically included when you run `npm publish`. The directory structure is:
+```
+native-vector-store/
+├── prebuilds/
+│   ├── linux-x64/
+│   ├── linux-arm64/
+│   ├── darwin-x64/
+│   ├── darwin-arm64/
+│   └── win32-x64/
+└── ... other files
+```
+
+## Fallback Behavior
+
+If a prebuild isn't available, users will need:
+- C++17 compatible compiler
+- simdjson library
+- OpenMP support
+- Python and build tools
+
+## Testing Prebuilds
+
+After building prebuilds:
+```bash
+# Test that it loads correctly
+node -e "console.log(require('.'))"
+```
+
+## Troubleshooting
+
+If prebuilds aren't working:
+1. Check that `node-gyp-build` is in dependencies (not devDependencies)
+2. Ensure prebuilds/ directory is not in .npmignore
+3. Verify the binary names match node-gyp-build expectations

package/docs/VectorStore.html

@@ -0,0 +1,180 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="utf-8">
+    <title>JSDoc: Class: VectorStore</title>
+
+    <script src="scripts/prettify/prettify.js"> </script>
+    <script src="scripts/prettify/lang-css.js"> </script>
+    <!--[if lt IE 9]>
+      <script src="//html5shiv.googlecode.com/svn/trunk/html5.js"></script>
+    <![endif]-->
+    <link type="text/css" rel="stylesheet" href="styles/prettify-tomorrow.css">
+    <link type="text/css" rel="stylesheet" href="styles/jsdoc-default.css">
+</head>
+
+<body>
+
+<div id="main">
+
+    <h1 class="page-title">Class: VectorStore</h1>
+
+    
+
+
+
+
+<section>
+
+<header>
+    
+        <h2><span class="attribs"><span class="type-signature"></span></span>VectorStore<span class="signature">()</span><span class="type-signature"></span></h2>
+        
+    
+</header>
+
+<article>
+    <div class="container-overview">
+    
+        
+
+    
+
+    
+    <h4 class="name" id="VectorStore"><span class="type-signature"></span>new VectorStore<span class="signature">()</span><span class="type-signature"></span></h4>
+    
+
+    
+
+
+
+<div class="description">
+    <p>High-performance vector store with SIMD optimization for similarity search.
+Designed for immutable, one-time loading scenarios with fast searches over focused corpora.</p>
+</div>
+
+
+
+
+
+
+
+
+
+
+
+
+
+<dl class="details">
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+</dl>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+    <h5>Examples</h5>
+    
+    <pre class="prettyprint"><code>// Basic usage
+const store = new VectorStore(1536);
+store.loadDir('./documents');
+const results = store.search(queryEmbedding, 10);</code></pre>
+
+    <pre class="prettyprint"><code>// Multiple domain-specific stores
+const productStore = new VectorStore(1536);
+const supportStore = new VectorStore(1536);
+productStore.loadDir('./knowledge/products');
+supportStore.loadDir('./knowledge/support');</code></pre>
+
+
+
+    
+    </div>
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+
+    
+</article>
+
+</section>
+
+
+
+
+</div>
+
+<nav>
+    <h2><a href="index.html">Home</a></h2><h3>Classes</h3><ul><li><a href="VectorStore.html">VectorStore</a></li><li><a href="VectorStoreWrapper.html">VectorStoreWrapper</a></li></ul><h3><a href="global.html">Global</a></h3>
+</nav>
+
+<br class="clear">
+
+<footer>
+    Documentation generated by <a href="https://github.com/jsdoc/jsdoc">JSDoc 4.0.4</a>
+</footer>
+
+<script> prettyPrint(); </script>
+<script src="scripts/linenumber.js"> </script>
+</body>
+</html>