@faiss-node/native 0.1.4

package/.env.example

@@ -0,0 +1,5 @@
+# npm Publishing Token
+# Copy this file to .env and add your npm token
+# DO NOT commit .env to git
+
+NPM_TOKEN=your_npm_token_here

package/API.md

@@ -0,0 +1,416 @@
+# API Reference
+
+Complete API documentation for `@faiss-node/native`.
+
+## Table of Contents
+
+- [FaissIndex Class](#faissindex-class)
+- [Index Types](#index-types)
+- [Methods](#methods)
+- [Types](#types)
+- [Examples](#examples)
+
+## FaissIndex Class
+
+The main class for creating and managing FAISS indexes.
+
+### Constructor
+
+```typescript
+new FaissIndex(config: FaissIndexConfig): FaissIndex
+```
+
+Creates a new FAISS index with the specified configuration.
+
+**Parameters:**
+
+- `config.type` (string, required): Index type
+  - `'FLAT_L2'` - Exact search, brute force
+  - `'IVF_FLAT'` - Fast approximate search with clustering
+  - `'HNSW'` - State-of-the-art approximate search
+- `config.dims` (number, required): Vector dimensions (must be positive integer)
+- `config.nlist` (number, optional): Number of clusters for IVF_FLAT (default: 100)
+- `config.nprobe` (number, optional): Clusters to search for IVF_FLAT (default: 10)
+- `config.M` (number, optional): Connections per node for HNSW (default: 16)
+- `config.efConstruction` (number, optional): HNSW construction parameter (default: 200)
+- `config.efSearch` (number, optional): HNSW search parameter (default: 50)
+
+**Example:**
+
+```javascript
+const index = new FaissIndex({ type: 'HNSW', dims: 768 });
+```
+
+## Index Types
+
+### FLAT_L2 (IndexFlatL2)
+
+Exact search using brute-force L2 distance calculation.
+
+**Best for:**
+- Small datasets (< 10k vectors)
+- When 100% recall is required
+- Prototyping and testing
+
+**Performance:**
+- Search: O(n) - linear scan
+- Memory: 4 × dims × n bytes
+- Accuracy: 100% recall
+
+**Example:**
+
+```javascript
+const index = new FaissIndex({ type: 'FLAT_L2', dims: 128 });
+```
+
+### IVF_FLAT (IndexIVFFlat)
+
+Fast approximate search using inverted file index with flat vectors.
+
+**Best for:**
+- Medium datasets (10k - 1M vectors)
+- When 90-95% recall is acceptable
+- Production systems with medium-sized datasets
+
+**Performance:**
+- Search: O(nprobe × n/nlist) - much faster than FLAT
+- Memory: Similar to FLAT + cluster overhead
+- Accuracy: ~90-95% recall (configurable)
+
+**Requirements:**
+- Must call `train()` before adding vectors
+- Training typically requires 10k-100k sample vectors
+
+**Example:**
+
+```javascript
+const index = new FaissIndex({ 
+  type: 'IVF_FLAT', 
+  dims: 768,
+  nlist: 100,    // Number of clusters
+  nprobe: 10     // Clusters to search
+});
+
+// Must train before adding vectors
+await index.train(trainingVectors);
+await index.add(dataVectors);
+```
+
+### HNSW (IndexHNSW)
+
+Hierarchical Navigable Small World graph - state-of-the-art approximate search.
+
+**Best for:**
+- Large datasets (> 100k vectors)
+- Best speed/accuracy tradeoff
+- Production systems requiring high performance
+
+**Performance:**
+- Search: O(log n) - logarithmic search
+- Memory: ~1.5-2× more than FLAT
+- Accuracy: ~95-99% recall (configurable)
+
+**No training required** - ready to use immediately.
+
+**Example:**
+
+```javascript
+const index = new FaissIndex({ 
+  type: 'HNSW', 
+  dims: 1536,
+  M: 16,              // Connections per node
+  efConstruction: 200, // Construction parameter
+  efSearch: 50        // Search parameter
+});
+```
+
+## Methods
+
+### add(vectors: Float32Array): Promise<void>
+
+Add vectors to the index. Can add a single vector or a batch of vectors.
+
+**Parameters:**
+- `vectors` (Float32Array): Vector data. For batch operations, vectors must be concatenated: `[v1[0..d-1], v2[0..d-1], ...]`
+
+**Example:**
+
+```javascript
+// Single vector
+await index.add(new Float32Array([1, 2, 3, 4]));
+
+// Batch of 4 vectors (each 4 dimensions)
+await index.add(new Float32Array([
+  1, 0, 0, 0,  // Vector 1
+  0, 1, 0, 0,  // Vector 2
+  0, 0, 1, 0,  // Vector 3
+  0, 0, 0, 1   // Vector 4
+]));
+```
+
+**Throws:**
+- `Error` if index is disposed
+- `Error` if vector dimensions don't match index dimensions
+- `Error` if IVF_FLAT index is not trained
+
+### search(query: Float32Array, k: number): Promise<SearchResults>
+
+Search for k nearest neighbors.
+
+**Parameters:**
+- `query` (Float32Array): Query vector (must match index dimensions)
+- `k` (number): Number of nearest neighbors to return
+
+**Returns:**
+- `Promise<SearchResults>`: Object containing:
+  - `distances` (Float32Array): L2 distances to nearest neighbors
+  - `labels` (Int32Array): Indices of nearest neighbors
+
+**Example:**
+
+```javascript
+const query = new Float32Array([1, 0, 0, 0]);
+const results = await index.search(query, 5);
+
+console.log('Nearest neighbors:', results.labels);
+console.log('Distances:', results.distances);
+```
+
+**Throws:**
+- `Error` if index is disposed
+- `Error` if query dimensions don't match
+- `Error` if k is invalid
+
+### searchBatch(queries: Float32Array, k: number): Promise<SearchResults>
+
+Perform batch search for multiple queries efficiently.
+
+**Parameters:**
+- `queries` (Float32Array): Query vectors concatenated: `[q1[0..d-1], q2[0..d-1], ...]`
+- `k` (number): Number of nearest neighbors per query
+
+**Returns:**
+- `Promise<SearchResults>`: Object containing:
+  - `distances` (Float32Array): Shape `[nq * k]` - distances for all queries
+  - `labels` (Int32Array): Shape `[nq * k]` - labels for all queries
+
+**Example:**
+
+```javascript
+// 3 queries of 4 dimensions each
+const queries = new Float32Array([
+  1, 0, 0, 0,  // Query 1
+  0, 1, 0, 0,  // Query 2
+  0, 0, 1, 0   // Query 3
+]);
+const results = await index.searchBatch(queries, 5);
+
+// results.distances[0..4] = distances for query 1
+// results.distances[5..9] = distances for query 2
+// results.distances[10..14] = distances for query 3
+```
+
+### train(vectors: Float32Array): Promise<void>
+
+Train an IVF_FLAT index. Required before adding vectors.
+
+**Parameters:**
+- `vectors` (Float32Array): Training vectors (typically 10k-100k vectors)
+
+**Example:**
+
+```javascript
+const trainingVectors = new Float32Array(/* 10k vectors */);
+await ivfIndex.train(trainingVectors);
+await ivfIndex.add(dataVectors);  // Now you can add vectors
+```
+
+**Throws:**
+- `Error` if index is not IVF_FLAT
+- `Error` if index is disposed
+
+### setNprobe(nprobe: number): Promise<void>
+
+Set the number of clusters to search for IVF_FLAT indexes.
+
+**Parameters:**
+- `nprobe` (number): Number of clusters to search (higher = more accurate, slower)
+
+**Example:**
+
+```javascript
+await ivfIndex.setNprobe(20);  // Search more clusters
+```
+
+**Throws:**
+- `Error` if index is not IVF_FLAT
+- `Error` if index is disposed
+
+### getStats(): IndexStats
+
+Get index statistics.
+
+**Returns:**
+- `IndexStats`: Object containing:
+  - `ntotal` (number): Total vectors in index
+  - `dims` (number): Vector dimensions
+  - `isTrained` (boolean): Whether index is trained (IVF only)
+  - `type` (string): Index type
+
+**Example:**
+
+```javascript
+const stats = index.getStats();
+console.log(`Index has ${stats.ntotal} vectors of ${stats.dims} dimensions`);
+```
+
+### save(filename: string): Promise<void>
+
+Save index to disk.
+
+**Parameters:**
+- `filename` (string): File path to save index
+
+**Example:**
+
+```javascript
+await index.save('./my-index.faiss');
+```
+
+**Throws:**
+- `Error` if index is disposed
+- `Error` if file cannot be written
+
+### static load(filename: string): Promise<FaissIndex>
+
+Load index from disk.
+
+**Parameters:**
+- `filename` (string): File path to load index from
+
+**Returns:**
+- `Promise<FaissIndex>`: Loaded index instance
+
+**Example:**
+
+```javascript
+const index = await FaissIndex.load('./my-index.faiss');
+```
+
+**Throws:**
+- `Error` if file cannot be read
+- `Error` if file is not a valid FAISS index
+
+### toBuffer(): Promise<Buffer>
+
+Serialize index to a Node.js Buffer.
+
+**Returns:**
+- `Promise<Buffer>`: Serialized index data
+
+**Example:**
+
+```javascript
+const buffer = await index.toBuffer();
+// Store in database, send over network, etc.
+```
+
+**Throws:**
+- `Error` if index is disposed
+
+### static fromBuffer(buffer: Buffer): Promise<FaissIndex>
+
+Deserialize index from Buffer.
+
+**Parameters:**
+- `buffer` (Buffer): Serialized index data
+
+**Returns:**
+- `Promise<FaissIndex>`: Deserialized index instance
+
+**Example:**
+
+```javascript
+const index = await FaissIndex.fromBuffer(buffer);
+```
+
+**Throws:**
+- `Error` if buffer is not valid FAISS index data
+
+### mergeFrom(otherIndex: FaissIndex): Promise<void>
+
+Merge vectors from another index into this index.
+
+**Parameters:**
+- `otherIndex` (FaissIndex): Index to merge from
+
+**Example:**
+
+```javascript
+const index1 = new FaissIndex({ type: 'FLAT_L2', dims: 128 });
+const index2 = new FaissIndex({ type: 'FLAT_L2', dims: 128 });
+
+await index1.add(vectors1);
+await index2.add(vectors2);
+
+await index1.mergeFrom(index2);  // index1 now contains vectors from both
+// Note: index2 is now empty (FAISS behavior)
+```
+
+**Throws:**
+- `Error` if index is disposed
+- `Error` if otherIndex is disposed
+- `Error` if dimensions don't match
+
+### dispose(): void
+
+Explicitly dispose of the index and free resources.
+
+**Example:**
+
+```javascript
+index.dispose();
+// Index is now unusable - all operations will throw errors
+```
+
+**Note:** Disposal is automatic on garbage collection, but explicit disposal is recommended for immediate resource cleanup.
+
+## Types
+
+### FaissIndexConfig
+
+```typescript
+interface FaissIndexConfig {
+  type: 'FLAT_L2' | 'IVF_FLAT' | 'HNSW';
+  dims: number;
+  nlist?: number;
+  nprobe?: number;
+  M?: number;
+  efConstruction?: number;
+  efSearch?: number;
+}
+```
+
+### SearchResults
+
+```typescript
+interface SearchResults {
+  distances: Float32Array;
+  labels: Int32Array;
+}
+```
+
+### IndexStats
+
+```typescript
+interface IndexStats {
+  ntotal: number;
+  dims: number;
+  isTrained: boolean;
+  type: string;
+}
+```
+
+## Examples
+
+See the [README.md](./README.md#examples) for complete usage examples.

package/DOCUMENTATION.md

@@ -0,0 +1,203 @@
+# Documentation
+
+This project uses multiple documentation tools to provide comprehensive documentation for both JavaScript/TypeScript API and C++ native code.
+
+## Documentation Structure
+
+- **README.md** - Main project documentation and quick start guide
+- **API.md** - Complete API reference (manually maintained)
+- **docs/api/** - TypeDoc-generated JavaScript/TypeScript API documentation
+- **docs/html/** - Doxygen-generated C++ native code documentation
+
+## Documentation Tools
+
+### 1. TypeDoc (JavaScript/TypeScript API)
+
+**Purpose:** Generate API documentation from TypeScript definitions and JSDoc comments.
+
+**Configuration:** `typedoc.json`
+
+**Generate:**
+```bash
+npm run docs:js
+```
+
+**Output:** `docs/api/` (HTML documentation)
+
+**Features:**
+- Extracts documentation from `types.d.ts` and JSDoc comments
+- Type-safe documentation with full type information
+- Interactive search and navigation
+- Automatically deployed to GitHub Pages
+
+### 2. Doxygen (C++ Native Code)
+
+**Purpose:** Generate documentation for C++ native bindings.
+
+**Configuration:** `Doxyfile`
+
+**Generate:**
+```bash
+npm run docs:cpp
+```
+
+**Output:** `docs/html/` (HTML documentation)
+
+**Features:**
+- Documents C++ classes, functions, and structures
+- Source code browsing
+- Call graphs and dependency diagrams
+- Automatically deployed to GitHub Pages
+
+### 3. Generate All Documentation
+
+```bash
+npm run docs  # Generates both TypeDoc and Doxygen docs
+```
+
+## Automated Documentation
+
+### GitHub Actions (Fully Automatic)
+
+Documentation is **automatically generated and deployed** when you push to `main`:
+
+- ✅ **Auto-generates** on every push to `main`
+- ✅ **Auto-deploys** to GitHub Pages
+- ✅ **Runs on PRs** to verify docs generate correctly
+- ✅ **Monitors** source files, config files, and README
+
+**No manual steps required!** Just push your code and documentation updates automatically.
+
+### Local Development (Watch Mode)
+
+For local development with auto-regeneration:
+
+```bash
+# Watch mode (TypeDoc only - regenerates on file changes)
+npm run docs:watch
+
+# Or use the auto-docs script (both TypeDoc and Doxygen)
+./scripts/auto-docs.sh
+```
+
+The auto-docs script watches for changes and regenerates both docs automatically.
+
+### Pre-Publish Hook
+
+Documentation is automatically generated before publishing to npm:
+
+```bash
+npm publish  # Automatically runs 'npm run docs' first
+```
+
+## Viewing Documentation
+
+### Local Development
+
+**TypeDoc (JavaScript API):**
+```bash
+npm run docs:serve
+# Visit http://localhost:8000
+```
+
+**Doxygen (C++ Code):**
+```bash
+npm run docs:serve:cpp
+# Visit http://localhost:8001
+```
+
+### Online
+
+- **JavaScript API Docs:** [GitHub Pages - API](https://anupammaurya6767.github.io/faiss-node-native/api/)
+- **C++ Native Docs:** [GitHub Pages - Native](https://anupammaurya6767.github.io/faiss-node-native/)
+
+## Writing Documentation
+
+### JavaScript/TypeScript
+
+Add JSDoc comments to your code:
+
+```javascript
+/**
+ * Add vectors to the index
+ * @param {Float32Array} vectors - Single vector or batch of vectors
+ * @returns {Promise<void>}
+ * @example
+ * await index.add(new Float32Array([1, 2, 3, 4]));
+ */
+async add(vectors) {
+  // ...
+}
+```
+
+TypeDoc will automatically extract:
+- Parameter types from TypeScript definitions
+- Return types
+- JSDoc comments
+- Examples
+
+### C++
+
+Add Doxygen comments:
+
+```cpp
+/**
+ * Add vectors to the index
+ * @param vectors Pointer to float array (n * dims elements)
+ * @param n Number of vectors
+ * @throws std::runtime_error if index is disposed
+ */
+void Add(const float* vectors, size_t n);
+```
+
+## Automated Deployment
+
+Both documentation types are automatically deployed to GitHub Pages:
+
+- **TypeDoc:** Deployed via `.github/workflows/docs-typedoc.yml`
+- **Doxygen:** Deployed via `.github/workflows/docs.yml`
+
+Documentation is updated automatically when:
+- Code is pushed to `main` branch
+- Source files are modified
+- Documentation configuration files are updated
+
+## Best Practices
+
+1. **Keep JSDoc comments up-to-date** - They're the source of truth for API docs
+2. **Use TypeScript types** - TypeDoc extracts types automatically
+3. **Add examples** - Use `@example` tags in JSDoc
+4. **Document parameters** - Use `@param` for all parameters
+5. **Document return values** - Use `@returns` for return types
+6. **Document errors** - Use `@throws` for exceptions
+
+## Troubleshooting
+
+### TypeDoc not generating
+
+```bash
+# Install dependencies
+npm install
+
+# Check TypeDoc version
+npx typedoc --version
+
+# Generate with verbose output
+npx typedoc --verbose
+```
+
+### Doxygen not generating
+
+```bash
+# Check Doxygen installation
+doxygen --version
+
+# Generate with verbose output
+doxygen Doxyfile 2>&1 | head -20
+```
+
+### Documentation not deploying
+
+- Check GitHub Actions workflow runs
+- Verify GitHub Pages is enabled in repository settings
+- Check for errors in workflow logs

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Anupam Maurya
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/QUICK_PUBLISH.md

@@ -0,0 +1,48 @@
+# Quick Publishing Guide
+
+## 🚀 Publish Your First Package (3 Steps)
+
+### Step 1: Test First (Dry Run)
+```bash
+npm run publish:dry-run
+```
+This shows what will be published WITHOUT actually publishing.
+
+### Step 2: Verify Everything Looks Good
+Check the output - should show:
+- ✅ Package name: `@faiss-node/native`
+- ✅ Version: `0.1.0`
+- ✅ Author: Your name
+- ✅ Description: Looks good
+- ✅ Files: Only necessary files (no test/, .github/, etc.)
+
+### Step 3: Publish!
+```bash
+npm run publish:local
+```
+
+That's it! Your package will be live on npm in a few minutes.
+
+## 📍 After Publishing
+
+1. **Check your package**: https://www.npmjs.com/package/@faiss-node/native
+2. **Test installation**: 
+   ```bash
+   cd /tmp && mkdir test && cd test
+   npm init -y
+   npm install @faiss-node/native
+   node -e "const { FaissIndex } = require('@faiss-node/native'); console.log('✅ Works!');"
+   ```
+
+## ⚠️ Important Notes
+
+- **You can't unpublish easily** (only within 72 hours if no one installed it)
+- **Version numbers are permanent** - can't republish same version
+- **Always test with `publish:dry-run` first**
+
+## 🎉 You're Done!
+
+Your package is now live! Share it:
+- Twitter/X: "Just published @faiss-node/native - high-performance FAISS bindings for Node.js 🚀"
+- GitHub: Update your profile README
+- Reddit: r/node, r/javascript