hnswlib-wasm-node 0.1.0

package/README.md

@@ -0,0 +1,370 @@
+# hnswlib-wasm-node
+
+Persistence layer for [hnswlib-wasm](https://www.npmjs.com/package/hnswlib-wasm) with JSON and binary format support for Node.js. This package enables saving and loading HNSW vector indexes to disk, making it easy to persist your vector search indexes across application restarts.
+
+## Features
+
+- ✅ **Dual Package Support**: Works with both ESM (`import`) and CommonJS (`require()`)
+- ✅ **Dual Format Support**: Save/load indexes in both JSON (human-readable) and binary (compact) formats
+- ✅ **Node.js Optimized**: Automatic environment setup for Node.js
+- ✅ **Type Safe**: Full JSDoc documentation
+- ✅ **Error Handling**: Comprehensive validation and error messages
+- ✅ **Optional Logging**: Configurable logger interface
+- ✅ **Production Ready**: Buffer bounds checking, input validation, edge case handling
+
+## Installation
+
+```bash
+npm install hnswlib-wasm-node
+```
+
+**Peer Dependency**: This package requires `hnswlib-wasm` to be installed:
+
+```bash
+npm install hnswlib-wasm
+```
+
+## Quick Start
+
+### ESM (Recommended)
+
+```javascript
+import { loadHnswlib, saveIndexToFile, loadIndexFromFile } from 'hnswlib-wasm-node';
+
+// Load the library (environment setup is automatic)
+const hnswlib = await loadHnswlib();
+
+// Create an index
+const index = new hnswlib.HierarchicalNSW('l2', 128, '');
+index.initIndex(1000, 16, 200, 100);
+
+// Add vectors
+const vectors = [
+  [1.0, 2.0, 3.0, ...], // 128-dimensional vector
+  [4.0, 5.0, 6.0, ...],
+  // ... more vectors
+];
+
+vectors.forEach((vector, i) => {
+  index.addPoint(vector, i, false);
+});
+
+// Save to disk (JSON format)
+const metadata = {
+  spaceName: 'l2',
+  maxElements: 1000,
+  m: 16,
+  efConstruction: 200,
+  randomSeed: 100
+};
+
+await saveIndexToFile(index, 'my-index.json', metadata);
+
+// Later, load from disk
+const { index: loadedIndex, metadata: loadedMetadata } = await loadIndexFromFile(hnswlib, 'my-index.json');
+
+// Use the loaded index
+const results = loadedIndex.searchKnn([1.0, 2.0, 3.0, ...], 5, undefined);
+```
+
+### CommonJS
+
+```javascript
+const { loadHnswlib, saveIndexToFile, loadIndexFromFile } = require('hnswlib-wasm-node');
+
+// Load the library (environment setup is automatic)
+const hnswlib = await loadHnswlib();
+
+// ... rest of the code is the same as ESM example ...
+```
+
+## API Reference
+
+### `loadHnswlib()`
+
+Loads the hnswlib-wasm library with automatic Node.js environment setup.
+
+```javascript
+import { loadHnswlib } from 'hnswlib-wasm-node';
+
+const hnswlib = await loadHnswlib();
+// Returns the loaded hnswlib module
+```
+
+**Returns**: `Promise<Object>` - The loaded hnswlib module
+
+**Throws**: `Error` if library fails to load
+
+---
+
+### `saveIndexToFile(index, filename, metadata?)`
+
+Saves an HNSW index to disk in JSON or binary format (determined by file extension).
+
+```javascript
+import { saveIndexToFile } from 'hnswlib-wasm-node';
+
+await saveIndexToFile(index, 'index.json', {
+  spaceName: 'l2',        // 'l2', 'ip', or 'cosine'
+  maxElements: 1000,      // Maximum number of elements
+  m: 16,                  // Number of bi-directional links
+  efConstruction: 200,    // Construction time/accuracy trade-off
+  randomSeed: 100         // Random seed
+});
+```
+
+**Parameters**:
+- `index` (Object): The hnswlib index object
+- `filename` (string): Output filename (`.json` for JSON, `.bin` or `.dat` for binary)
+- `metadata` (Object, optional): Index metadata
+  - `spaceName` (string): 'l2', 'ip', or 'cosine'
+  - `maxElements` (number): Maximum number of elements
+  - `m` (number): Number of bi-directional links (default: 16)
+  - `efConstruction` (number): Construction parameter (default: 200)
+  - `randomSeed` (number): Random seed (default: 100)
+
+**Returns**: `Promise<void>`
+
+**Throws**: `Error` if save operation fails
+
+---
+
+### `loadIndexFromFile(hnswlib, filename)`
+
+Loads an HNSW index from disk (JSON or binary format).
+
+```javascript
+import { loadIndexFromFile } from 'hnswlib-wasm-node';
+
+const { index, metadata } = await loadIndexFromFile(hnswlib, 'index.json');
+```
+
+**Parameters**:
+- `hnswlib` (Object): The loaded hnswlib module (from `loadHnswlib()`)
+- `filename` (string): Input filename (`.json` for JSON, `.bin` or `.dat` for binary)
+
+**Returns**: `Promise<{index: Object, metadata: Object}>`
+- `index`: The recreated index object
+- `metadata`: Index metadata (spaceName, numDimensions, maxElements, m, efConstruction, randomSeed)
+
+**Throws**: `Error` if load operation fails
+
+---
+
+### `setLogger(logger)`
+
+Configure custom logger for the library (optional). By default, the library is silent (no console output).
+
+```javascript
+import { setLogger } from 'hnswlib-wasm-node';
+
+// Enable logging
+setLogger({
+  log: console.log,      // Info messages
+  error: console.error   // Error messages (accepts message and error object)
+});
+
+// Disable all logging
+setLogger({
+  log: () => {},
+  error: () => {}
+});
+```
+
+**Parameters**:
+- `logger` (Object): Logger object with `log` and `error` methods
+
+---
+
+## Format Comparison
+
+### JSON Format (`.json`)
+
+**Pros**:
+- Human-readable and editable
+- Easy to debug
+- Portable across systems
+
+**Cons**:
+- Larger file size
+- Slower to parse
+
+**Example**:
+```json
+{
+  "version": 1,
+  "spaceName": "l2",
+  "numDimensions": 128,
+  "maxElements": 1000,
+  "m": 16,
+  "efConstruction": 200,
+  "randomSeed": 100,
+  "numVectors": 3,
+  "vectors": [
+    {
+      "label": 0,
+      "point": [1.0, 2.0, 3.0, ...]
+    },
+    ...
+  ]
+}
+```
+
+### Binary Format (`.bin` or `.dat`)
+
+**Pros**:
+- Compact file size (~83% smaller than JSON)
+- Fast read/write operations
+- Efficient for large datasets
+
+**Cons**:
+- Not human-readable
+- Requires a reader to inspect
+
+**File Structure**:
+- Header (40 bytes): version, spaceName, dimensions, parameters, vector count, reserved space (14 bytes for future use)
+- Vector records: label (4 bytes) + point (numDimensions × 4 bytes)
+
+---
+
+## Complete Examples
+
+### Example 1: Basic Usage
+
+```javascript
+import { loadHnswlib, saveIndexToFile, loadIndexFromFile } from 'hnswlib-wasm-node';
+
+async function main() {
+  // Load library
+  const hnswlib = await loadHnswlib();
+  
+  // Create index
+  const index = new hnswlib.HierarchicalNSW('l2', 3, '');
+  index.initIndex(10, 16, 200, 100);
+  
+  // Add vectors
+  const vectors = [
+    [1, 0, 0],
+    [0, 1, 0],
+    [0, 0, 1]
+  ];
+  
+  vectors.forEach((vector, i) => {
+    index.addPoint(vector, i, false);
+  });
+  
+  // Save
+  await saveIndexToFile(index, 'vectors.json', {
+    spaceName: 'l2',
+    maxElements: 10,
+    m: 16,
+    efConstruction: 200,
+    randomSeed: 100
+  });
+  
+  // Load
+  const { index: loadedIndex } = await loadIndexFromFile(hnswlib, 'vectors.json');
+  
+  // Search
+  const results = loadedIndex.searchKnn([0.9, 0.1, 0], 2, undefined);
+  console.log(results);
+}
+
+main().catch(console.error);
+```
+
+### Example 2: Using Binary Format
+
+```javascript
+import { loadHnswlib, saveIndexToFile, loadIndexFromFile } from 'hnswlib-wasm-node';
+
+async function main() {
+  const hnswlib = await loadHnswlib();
+  const index = new hnswlib.HierarchicalNSW('cosine', 128, '');
+  index.initIndex(10000, 32, 200, 100);
+  
+  // ... add vectors ...
+  
+  // Save as binary (more efficient for large datasets)
+  await saveIndexToFile(index, 'large-index.bin', {
+    spaceName: 'cosine',
+    maxElements: 10000,
+    m: 32,
+    efConstruction: 200,
+    randomSeed: 100
+  });
+  
+  // Load binary file
+  const { index: loadedIndex } = await loadIndexFromFile(hnswlib, 'large-index.bin');
+}
+```
+
+### Example 3: Error Handling
+
+```javascript
+import { loadHnswlib, saveIndexToFile, loadIndexFromFile } from 'hnswlib-wasm-node';
+
+async function main() {
+  const hnswlib = await loadHnswlib();
+  
+  try {
+    // This will throw if file doesn't exist
+    const { index } = await loadIndexFromFile(hnswlib, 'nonexistent.json');
+  } catch (error) {
+    console.error('Load failed:', error.message);
+    // Error: File not found: nonexistent.json
+  }
+  
+  try {
+    // This will throw if index is empty
+    const emptyIndex = new hnswlib.HierarchicalNSW('l2', 3, '');
+    emptyIndex.initIndex(10, 16, 200, 100);
+    await saveIndexToFile(emptyIndex, 'empty.json');
+  } catch (error) {
+    console.error('Save failed:', error.message);
+    // Error: Cannot save empty index (no vectors added)
+  }
+}
+```
+
+### Example 4: Custom Logging
+
+```javascript
+import { loadHnswlib, saveIndexToFile, setLogger } from 'hnswlib-wasm-node';
+
+// Enable verbose logging
+setLogger({
+  log: (message) => console.log(`[INFO] ${message}`),
+  error: (message, error) => console.error(`[ERROR] ${message}`, error)
+});
+
+const hnswlib = await loadHnswlib();
+const index = new hnswlib.HierarchicalNSW('l2', 128, '');
+// ... create and save index ...
+// Will output: [INFO] Index saved to index.json (JSON format, 100 vectors)
+```
+
+---
+
+## Requirements
+
+- **Node.js**: >= 18.0.0
+- **Peer Dependency**: `hnswlib-wasm` ^0.8.2
+
+## License
+
+MIT
+
+## Related Projects
+
+- [hnswlib-wasm](https://www.npmjs.com/package/hnswlib-wasm) - WebAssembly bindings for HNSWlib
+- [hnswlib-node](https://www.npmjs.com/package/hnswlib-node) - Native Node.js bindings for HNSWlib
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.
+
+## Support
+
+For issues and questions, please open an issue on the GitHub repository.
+

package/dist/index.cjs

@@ -0,0 +1,357 @@
+var __create = Object.create;
+var __defProp = Object.defineProperty;
+var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
+var __getOwnPropNames = Object.getOwnPropertyNames;
+var __getProtoOf = Object.getPrototypeOf;
+var __hasOwnProp = Object.prototype.hasOwnProperty;
+var __export = (target, all) => {
+  for (var name in all)
+    __defProp(target, name, { get: all[name], enumerable: true });
+};
+var __copyProps = (to, from, except, desc) => {
+  if (from && typeof from === "object" || typeof from === "function") {
+    for (let key of __getOwnPropNames(from))
+      if (!__hasOwnProp.call(to, key) && key !== except)
+        __defProp(to, key, { get: () => from[key], enumerable: !(desc = __getOwnPropDesc(from, key)) || desc.enumerable });
+  }
+  return to;
+};
+var __toESM = (mod, isNodeMode, target) => (target = mod != null ? __create(__getProtoOf(mod)) : {}, __copyProps(
+  // If the importer is in node compatibility mode or this is not an ESM
+  // file that has been converted to a CommonJS file using a Babel-
+  // compatible transform (i.e. "__esModule" has not been set), then set
+  // "default" to the CommonJS "module.exports" for node compatibility.
+  isNodeMode || !mod || !mod.__esModule ? __defProp(target, "default", { value: mod, enumerable: true }) : target,
+  mod
+));
+var __toCommonJS = (mod) => __copyProps(__defProp({}, "__esModule", { value: true }), mod);
+
+// index.js
+var index_exports = {};
+__export(index_exports, {
+  loadHnswlib: () => loadHnswlib,
+  loadIndexFromFile: () => loadIndexFromFile,
+  saveIndexToFile: () => saveIndexToFile,
+  setLogger: () => setLogger
+});
+module.exports = __toCommonJS(index_exports);
+var import_promises = __toESM(require("fs/promises"), 1);
+var defaultLogger = {
+  log: () => {
+  },
+  error: console.error
+};
+var logger = defaultLogger;
+function setLogger(customLogger) {
+  logger = { ...defaultLogger, ...customLogger };
+}
+function setupNodeEnvironment() {
+  if (typeof indexedDB === "undefined") {
+    global.indexedDB = {
+      open: () => ({
+        onerror: null,
+        onsuccess: null,
+        onupgradeneeded: null,
+        result: {
+          createObjectStore: () => ({}),
+          objectStoreNames: { contains: () => false }
+        }
+      })
+    };
+  }
+  if (typeof window === "undefined") {
+    global.window = {};
+  }
+}
+async function loadHnswlib() {
+  try {
+    setupNodeEnvironment();
+    const { loadHnswlib: loadHnswlibOriginal } = await import("hnswlib-wasm/dist/hnswlib.js");
+    return await loadHnswlibOriginal();
+  } catch (error) {
+    logger.error("Failed to load hnswlib:", error);
+    throw new Error(`Failed to load hnswlib: ${error.message}`);
+  }
+}
+function extractVectorsFromIndex(index) {
+  try {
+    const usedLabels = index.getUsedLabels();
+    const vectors = [];
+    for (const label of usedLabels) {
+      const point = index.getPoint(label);
+      const vector = Array.isArray(point) ? point : Array.from(point);
+      vectors.push({ label, point: vector });
+    }
+    return vectors;
+  } catch (error) {
+    throw new Error(`Failed to extract vectors from index: ${error.message}`);
+  }
+}
+function validateIndex(index) {
+  if (!index) {
+    throw new Error("Index parameter is required");
+  }
+  if (typeof index.getNumDimensions !== "function" || typeof index.getCurrentCount !== "function" || typeof index.getUsedLabels !== "function" || typeof index.getPoint !== "function") {
+    throw new Error("Invalid index object: missing required methods");
+  }
+}
+function validateFilename(filename) {
+  if (!filename || typeof filename !== "string") {
+    throw new Error("Filename must be a non-empty string");
+  }
+  if (filename.trim().length === 0) {
+    throw new Error("Filename cannot be empty");
+  }
+}
+async function saveIndexToFile(index, filename, metadata = {}) {
+  validateIndex(index);
+  validateFilename(filename);
+  try {
+    const numDimensions = index.getNumDimensions();
+    const numVectors = index.getCurrentCount();
+    if (numVectors === 0) {
+      throw new Error("Cannot save empty index (no vectors added)");
+    }
+    const vectors = extractVectorsFromIndex(index);
+    const isBinary = filename.endsWith(".bin") || filename.endsWith(".dat");
+    if (isBinary) {
+      await saveIndexBinary(index, filename, metadata, numDimensions, vectors);
+    } else {
+      await saveIndexJSON(index, filename, metadata, numDimensions, vectors);
+    }
+  } catch (error) {
+    logger.error(`Failed to save index to ${filename}:`, error);
+    throw error;
+  }
+}
+async function saveIndexJSON(index, filename, metadata, numDimensions, vectors) {
+  const data = {
+    version: 1,
+    spaceName: metadata.spaceName || "l2",
+    numDimensions,
+    maxElements: metadata.maxElements || index.getMaxElements(),
+    m: metadata.m || 16,
+    efConstruction: metadata.efConstruction || 200,
+    randomSeed: metadata.randomSeed || 100,
+    numVectors: vectors.length,
+    vectors
+  };
+  try {
+    await import_promises.default.writeFile(filename, JSON.stringify(data, null, 2), "utf8");
+    logger.log(`Index saved to ${filename} (JSON format, ${vectors.length} vectors)`);
+  } catch (error) {
+    throw new Error(`Failed to write JSON file: ${error.message}`);
+  }
+}
+async function saveIndexBinary(index, filename, metadata, numDimensions, vectors) {
+  const spaceNameMap = { "l2": 0, "ip": 1, "cosine": 2 };
+  const spaceNameCode = spaceNameMap[metadata.spaceName || "l2"] || 0;
+  const headerSize = 40;
+  const reservedSize = 14;
+  const vectorSize = 4 + numDimensions * 4;
+  const bufferSize = headerSize + vectors.length * vectorSize;
+  try {
+    const buffer = Buffer.allocUnsafe(bufferSize);
+    let offset = 0;
+    buffer.writeUInt8(1, offset);
+    offset += 1;
+    buffer.writeUInt8(spaceNameCode, offset);
+    offset += 1;
+    buffer.writeUInt32LE(numDimensions, offset);
+    offset += 4;
+    buffer.writeUInt32LE(metadata.maxElements || index.getMaxElements(), offset);
+    offset += 4;
+    buffer.writeUInt32LE(metadata.m || 16, offset);
+    offset += 4;
+    buffer.writeUInt32LE(metadata.efConstruction || 200, offset);
+    offset += 4;
+    buffer.writeUInt32LE(metadata.randomSeed || 100, offset);
+    offset += 4;
+    buffer.writeUInt32LE(vectors.length, offset);
+    offset += 4;
+    buffer.fill(0, offset, offset + reservedSize);
+    offset += reservedSize;
+    for (const { label, point } of vectors) {
+      if (point.length !== numDimensions) {
+        throw new Error(`Vector dimension mismatch: expected ${numDimensions}, got ${point.length}`);
+      }
+      buffer.writeUInt32LE(label, offset);
+      offset += 4;
+      for (const value of point) {
+        if (typeof value !== "number" || !isFinite(value)) {
+          throw new Error(`Invalid vector value: ${value}`);
+        }
+        buffer.writeFloatLE(value, offset);
+        offset += 4;
+      }
+    }
+    await import_promises.default.writeFile(filename, buffer);
+    logger.log(`Index saved to ${filename} (binary format, ${vectors.length} vectors, ${bufferSize} bytes)`);
+  } catch (error) {
+    throw new Error(`Failed to write binary file: ${error.message}`);
+  }
+}
+async function loadIndexFromFile(hnswlib, filename) {
+  validateFilename(filename);
+  if (!hnswlib || typeof hnswlib.HierarchicalNSW !== "function") {
+    throw new Error("Invalid hnswlib module: HierarchicalNSW constructor not found");
+  }
+  try {
+    await import_promises.default.access(filename);
+  } catch (error) {
+    throw new Error(`File not found: ${filename}`);
+  }
+  const isBinary = filename.endsWith(".bin") || filename.endsWith(".dat");
+  try {
+    if (isBinary) {
+      return await loadIndexBinary(hnswlib, filename);
+    } else {
+      return await loadIndexJSON(hnswlib, filename);
+    }
+  } catch (error) {
+    logger.error(`Failed to load index from ${filename}:`, error);
+    throw error;
+  }
+}
+async function loadIndexJSON(hnswlib, filename) {
+  let data;
+  try {
+    const content = await import_promises.default.readFile(filename, "utf8");
+    data = JSON.parse(content);
+  } catch (error) {
+    if (error instanceof SyntaxError) {
+      throw new Error(`Invalid JSON file: ${error.message}`);
+    }
+    throw new Error(`Failed to read file: ${error.message}`);
+  }
+  if (!data.vectors || !Array.isArray(data.vectors)) {
+    throw new Error("Invalid index file format: missing or invalid vectors array");
+  }
+  if (typeof data.numDimensions !== "number" || data.numDimensions <= 0) {
+    throw new Error("Invalid index file format: invalid numDimensions");
+  }
+  if (!data.spaceName || !["l2", "ip", "cosine"].includes(data.spaceName)) {
+    throw new Error(`Invalid spaceName: ${data.spaceName}. Must be 'l2', 'ip', or 'cosine'`);
+  }
+  try {
+    const index = new hnswlib.HierarchicalNSW(data.spaceName, data.numDimensions, "");
+    index.initIndex(
+      data.maxElements || 100,
+      data.m || 16,
+      data.efConstruction || 200,
+      data.randomSeed || 100
+    );
+    for (const { label, point } of data.vectors) {
+      if (typeof label !== "number") {
+        throw new Error(`Invalid label: ${label}. Must be a number`);
+      }
+      if (!Array.isArray(point) || point.length !== data.numDimensions) {
+        throw new Error(`Invalid vector: expected array of length ${data.numDimensions}`);
+      }
+      index.addPoint(point, label, false);
+    }
+    logger.log(`Index loaded from ${filename} (JSON format, ${data.vectors.length} vectors)`);
+    return {
+      index,
+      metadata: {
+        spaceName: data.spaceName,
+        numDimensions: data.numDimensions,
+        maxElements: data.maxElements || 100,
+        m: data.m || 16,
+        efConstruction: data.efConstruction || 200,
+        randomSeed: data.randomSeed || 100
+      }
+    };
+  } catch (error) {
+    throw new Error(`Failed to recreate index: ${error.message}`);
+  }
+}
+async function loadIndexBinary(hnswlib, filename) {
+  let buffer;
+  try {
+    buffer = await import_promises.default.readFile(filename);
+  } catch (error) {
+    throw new Error(`Failed to read file: ${error.message}`);
+  }
+  const minSize = 40;
+  if (buffer.length < minSize) {
+    throw new Error(`File too small to be a valid index file (${buffer.length} bytes, minimum ${minSize} bytes)`);
+  }
+  let offset = 0;
+  const version = buffer.readUInt8(offset);
+  offset += 1;
+  if (version !== 1) {
+    throw new Error(`Unsupported binary format version: ${version}. Expected version 1`);
+  }
+  if (offset >= buffer.length) throw new Error("Unexpected end of file while reading header");
+  const spaceNameCode = buffer.readUInt8(offset);
+  offset += 1;
+  const spaceNameMap = ["l2", "ip", "cosine"];
+  const spaceName = spaceNameMap[spaceNameCode] || "l2";
+  if (offset + 24 > buffer.length) throw new Error("Unexpected end of file while reading header");
+  const numDimensions = buffer.readUInt32LE(offset);
+  offset += 4;
+  const maxElements = buffer.readUInt32LE(offset);
+  offset += 4;
+  const m = buffer.readUInt32LE(offset);
+  offset += 4;
+  const efConstruction = buffer.readUInt32LE(offset);
+  offset += 4;
+  const randomSeed = buffer.readUInt32LE(offset);
+  offset += 4;
+  const numVectors = buffer.readUInt32LE(offset);
+  offset += 4;
+  offset += 14;
+  if (numDimensions <= 0 || numDimensions > 1e5) {
+    throw new Error(`Invalid numDimensions: ${numDimensions}`);
+  }
+  if (numVectors < 0 || numVectors > 1e8) {
+    throw new Error(`Invalid numVectors: ${numVectors}`);
+  }
+  const vectorSize = 4 + numDimensions * 4;
+  const expectedSize = 40 + numVectors * vectorSize;
+  if (buffer.length < expectedSize) {
+    throw new Error(`File size mismatch: expected ${expectedSize} bytes, got ${buffer.length} bytes`);
+  }
+  try {
+    const index = new hnswlib.HierarchicalNSW(spaceName, numDimensions, "");
+    index.initIndex(maxElements, m, efConstruction, randomSeed);
+    for (let i = 0; i < numVectors; i++) {
+      if (offset + 4 > buffer.length) {
+        throw new Error(`Unexpected end of file while reading vector ${i}`);
+      }
+      const label = buffer.readUInt32LE(offset);
+      offset += 4;
+      const point = [];
+      for (let j = 0; j < numDimensions; j++) {
+        if (offset + 4 > buffer.length) {
+          throw new Error(`Unexpected end of file while reading vector ${i}, dimension ${j}`);
+        }
+        point.push(buffer.readFloatLE(offset));
+        offset += 4;
+      }
+      index.addPoint(point, label, false);
+    }
+    logger.log(`Index loaded from ${filename} (binary format, ${numVectors} vectors)`);
+    return {
+      index,
+      metadata: {
+        spaceName,
+        numDimensions,
+        maxElements,
+        m,
+        efConstruction,
+        randomSeed
+      }
+    };
+  } catch (error) {
+    throw new Error(`Failed to recreate index: ${error.message}`);
+  }
+}
+// Annotate the CommonJS export names for ESM import in node:
+0 && (module.exports = {
+  loadHnswlib,
+  loadIndexFromFile,
+  saveIndexToFile,
+  setLogger
+});

package/dist/index.js

@@ -0,0 +1,320 @@
+// index.js
+import fs from "fs/promises";
+var defaultLogger = {
+  log: () => {
+  },
+  error: console.error
+};
+var logger = defaultLogger;
+function setLogger(customLogger) {
+  logger = { ...defaultLogger, ...customLogger };
+}
+function setupNodeEnvironment() {
+  if (typeof indexedDB === "undefined") {
+    global.indexedDB = {
+      open: () => ({
+        onerror: null,
+        onsuccess: null,
+        onupgradeneeded: null,
+        result: {
+          createObjectStore: () => ({}),
+          objectStoreNames: { contains: () => false }
+        }
+      })
+    };
+  }
+  if (typeof window === "undefined") {
+    global.window = {};
+  }
+}
+async function loadHnswlib() {
+  try {
+    setupNodeEnvironment();
+    const { loadHnswlib: loadHnswlibOriginal } = await import("hnswlib-wasm/dist/hnswlib.js");
+    return await loadHnswlibOriginal();
+  } catch (error) {
+    logger.error("Failed to load hnswlib:", error);
+    throw new Error(`Failed to load hnswlib: ${error.message}`);
+  }
+}
+function extractVectorsFromIndex(index) {
+  try {
+    const usedLabels = index.getUsedLabels();
+    const vectors = [];
+    for (const label of usedLabels) {
+      const point = index.getPoint(label);
+      const vector = Array.isArray(point) ? point : Array.from(point);
+      vectors.push({ label, point: vector });
+    }
+    return vectors;
+  } catch (error) {
+    throw new Error(`Failed to extract vectors from index: ${error.message}`);
+  }
+}
+function validateIndex(index) {
+  if (!index) {
+    throw new Error("Index parameter is required");
+  }
+  if (typeof index.getNumDimensions !== "function" || typeof index.getCurrentCount !== "function" || typeof index.getUsedLabels !== "function" || typeof index.getPoint !== "function") {
+    throw new Error("Invalid index object: missing required methods");
+  }
+}
+function validateFilename(filename) {
+  if (!filename || typeof filename !== "string") {
+    throw new Error("Filename must be a non-empty string");
+  }
+  if (filename.trim().length === 0) {
+    throw new Error("Filename cannot be empty");
+  }
+}
+async function saveIndexToFile(index, filename, metadata = {}) {
+  validateIndex(index);
+  validateFilename(filename);
+  try {
+    const numDimensions = index.getNumDimensions();
+    const numVectors = index.getCurrentCount();
+    if (numVectors === 0) {
+      throw new Error("Cannot save empty index (no vectors added)");
+    }
+    const vectors = extractVectorsFromIndex(index);
+    const isBinary = filename.endsWith(".bin") || filename.endsWith(".dat");
+    if (isBinary) {
+      await saveIndexBinary(index, filename, metadata, numDimensions, vectors);
+    } else {
+      await saveIndexJSON(index, filename, metadata, numDimensions, vectors);
+    }
+  } catch (error) {
+    logger.error(`Failed to save index to ${filename}:`, error);
+    throw error;
+  }
+}
+async function saveIndexJSON(index, filename, metadata, numDimensions, vectors) {
+  const data = {
+    version: 1,
+    spaceName: metadata.spaceName || "l2",
+    numDimensions,
+    maxElements: metadata.maxElements || index.getMaxElements(),
+    m: metadata.m || 16,
+    efConstruction: metadata.efConstruction || 200,
+    randomSeed: metadata.randomSeed || 100,
+    numVectors: vectors.length,
+    vectors
+  };
+  try {
+    await fs.writeFile(filename, JSON.stringify(data, null, 2), "utf8");
+    logger.log(`Index saved to ${filename} (JSON format, ${vectors.length} vectors)`);
+  } catch (error) {
+    throw new Error(`Failed to write JSON file: ${error.message}`);
+  }
+}
+async function saveIndexBinary(index, filename, metadata, numDimensions, vectors) {
+  const spaceNameMap = { "l2": 0, "ip": 1, "cosine": 2 };
+  const spaceNameCode = spaceNameMap[metadata.spaceName || "l2"] || 0;
+  const headerSize = 40;
+  const reservedSize = 14;
+  const vectorSize = 4 + numDimensions * 4;
+  const bufferSize = headerSize + vectors.length * vectorSize;
+  try {
+    const buffer = Buffer.allocUnsafe(bufferSize);
+    let offset = 0;
+    buffer.writeUInt8(1, offset);
+    offset += 1;
+    buffer.writeUInt8(spaceNameCode, offset);
+    offset += 1;
+    buffer.writeUInt32LE(numDimensions, offset);
+    offset += 4;
+    buffer.writeUInt32LE(metadata.maxElements || index.getMaxElements(), offset);
+    offset += 4;
+    buffer.writeUInt32LE(metadata.m || 16, offset);
+    offset += 4;
+    buffer.writeUInt32LE(metadata.efConstruction || 200, offset);
+    offset += 4;
+    buffer.writeUInt32LE(metadata.randomSeed || 100, offset);
+    offset += 4;
+    buffer.writeUInt32LE(vectors.length, offset);
+    offset += 4;
+    buffer.fill(0, offset, offset + reservedSize);
+    offset += reservedSize;
+    for (const { label, point } of vectors) {
+      if (point.length !== numDimensions) {
+        throw new Error(`Vector dimension mismatch: expected ${numDimensions}, got ${point.length}`);
+      }
+      buffer.writeUInt32LE(label, offset);
+      offset += 4;
+      for (const value of point) {
+        if (typeof value !== "number" || !isFinite(value)) {
+          throw new Error(`Invalid vector value: ${value}`);
+        }
+        buffer.writeFloatLE(value, offset);
+        offset += 4;
+      }
+    }
+    await fs.writeFile(filename, buffer);
+    logger.log(`Index saved to ${filename} (binary format, ${vectors.length} vectors, ${bufferSize} bytes)`);
+  } catch (error) {
+    throw new Error(`Failed to write binary file: ${error.message}`);
+  }
+}
+async function loadIndexFromFile(hnswlib, filename) {
+  validateFilename(filename);
+  if (!hnswlib || typeof hnswlib.HierarchicalNSW !== "function") {
+    throw new Error("Invalid hnswlib module: HierarchicalNSW constructor not found");
+  }
+  try {
+    await fs.access(filename);
+  } catch (error) {
+    throw new Error(`File not found: ${filename}`);
+  }
+  const isBinary = filename.endsWith(".bin") || filename.endsWith(".dat");
+  try {
+    if (isBinary) {
+      return await loadIndexBinary(hnswlib, filename);
+    } else {
+      return await loadIndexJSON(hnswlib, filename);
+    }
+  } catch (error) {
+    logger.error(`Failed to load index from ${filename}:`, error);
+    throw error;
+  }
+}
+async function loadIndexJSON(hnswlib, filename) {
+  let data;
+  try {
+    const content = await fs.readFile(filename, "utf8");
+    data = JSON.parse(content);
+  } catch (error) {
+    if (error instanceof SyntaxError) {
+      throw new Error(`Invalid JSON file: ${error.message}`);
+    }
+    throw new Error(`Failed to read file: ${error.message}`);
+  }
+  if (!data.vectors || !Array.isArray(data.vectors)) {
+    throw new Error("Invalid index file format: missing or invalid vectors array");
+  }
+  if (typeof data.numDimensions !== "number" || data.numDimensions <= 0) {
+    throw new Error("Invalid index file format: invalid numDimensions");
+  }
+  if (!data.spaceName || !["l2", "ip", "cosine"].includes(data.spaceName)) {
+    throw new Error(`Invalid spaceName: ${data.spaceName}. Must be 'l2', 'ip', or 'cosine'`);
+  }
+  try {
+    const index = new hnswlib.HierarchicalNSW(data.spaceName, data.numDimensions, "");
+    index.initIndex(
+      data.maxElements || 100,
+      data.m || 16,
+      data.efConstruction || 200,
+      data.randomSeed || 100
+    );
+    for (const { label, point } of data.vectors) {
+      if (typeof label !== "number") {
+        throw new Error(`Invalid label: ${label}. Must be a number`);
+      }
+      if (!Array.isArray(point) || point.length !== data.numDimensions) {
+        throw new Error(`Invalid vector: expected array of length ${data.numDimensions}`);
+      }
+      index.addPoint(point, label, false);
+    }
+    logger.log(`Index loaded from ${filename} (JSON format, ${data.vectors.length} vectors)`);
+    return {
+      index,
+      metadata: {
+        spaceName: data.spaceName,
+        numDimensions: data.numDimensions,
+        maxElements: data.maxElements || 100,
+        m: data.m || 16,
+        efConstruction: data.efConstruction || 200,
+        randomSeed: data.randomSeed || 100
+      }
+    };
+  } catch (error) {
+    throw new Error(`Failed to recreate index: ${error.message}`);
+  }
+}
+async function loadIndexBinary(hnswlib, filename) {
+  let buffer;
+  try {
+    buffer = await fs.readFile(filename);
+  } catch (error) {
+    throw new Error(`Failed to read file: ${error.message}`);
+  }
+  const minSize = 40;
+  if (buffer.length < minSize) {
+    throw new Error(`File too small to be a valid index file (${buffer.length} bytes, minimum ${minSize} bytes)`);
+  }
+  let offset = 0;
+  const version = buffer.readUInt8(offset);
+  offset += 1;
+  if (version !== 1) {
+    throw new Error(`Unsupported binary format version: ${version}. Expected version 1`);
+  }
+  if (offset >= buffer.length) throw new Error("Unexpected end of file while reading header");
+  const spaceNameCode = buffer.readUInt8(offset);
+  offset += 1;
+  const spaceNameMap = ["l2", "ip", "cosine"];
+  const spaceName = spaceNameMap[spaceNameCode] || "l2";
+  if (offset + 24 > buffer.length) throw new Error("Unexpected end of file while reading header");
+  const numDimensions = buffer.readUInt32LE(offset);
+  offset += 4;
+  const maxElements = buffer.readUInt32LE(offset);
+  offset += 4;
+  const m = buffer.readUInt32LE(offset);
+  offset += 4;
+  const efConstruction = buffer.readUInt32LE(offset);
+  offset += 4;
+  const randomSeed = buffer.readUInt32LE(offset);
+  offset += 4;
+  const numVectors = buffer.readUInt32LE(offset);
+  offset += 4;
+  offset += 14;
+  if (numDimensions <= 0 || numDimensions > 1e5) {
+    throw new Error(`Invalid numDimensions: ${numDimensions}`);
+  }
+  if (numVectors < 0 || numVectors > 1e8) {
+    throw new Error(`Invalid numVectors: ${numVectors}`);
+  }
+  const vectorSize = 4 + numDimensions * 4;
+  const expectedSize = 40 + numVectors * vectorSize;
+  if (buffer.length < expectedSize) {
+    throw new Error(`File size mismatch: expected ${expectedSize} bytes, got ${buffer.length} bytes`);
+  }
+  try {
+    const index = new hnswlib.HierarchicalNSW(spaceName, numDimensions, "");
+    index.initIndex(maxElements, m, efConstruction, randomSeed);
+    for (let i = 0; i < numVectors; i++) {
+      if (offset + 4 > buffer.length) {
+        throw new Error(`Unexpected end of file while reading vector ${i}`);
+      }
+      const label = buffer.readUInt32LE(offset);
+      offset += 4;
+      const point = [];
+      for (let j = 0; j < numDimensions; j++) {
+        if (offset + 4 > buffer.length) {
+          throw new Error(`Unexpected end of file while reading vector ${i}, dimension ${j}`);
+        }
+        point.push(buffer.readFloatLE(offset));
+        offset += 4;
+      }
+      index.addPoint(point, label, false);
+    }
+    logger.log(`Index loaded from ${filename} (binary format, ${numVectors} vectors)`);
+    return {
+      index,
+      metadata: {
+        spaceName,
+        numDimensions,
+        maxElements,
+        m,
+        efConstruction,
+        randomSeed
+      }
+    };
+  } catch (error) {
+    throw new Error(`Failed to recreate index: ${error.message}`);
+  }
+}
+export {
+  loadHnswlib,
+  loadIndexFromFile,
+  saveIndexToFile,
+  setLogger
+};

package/package.json

@@ -0,0 +1,51 @@
+{
+  "name": "hnswlib-wasm-node",
+  "version": "0.1.0",
+  "description": "Persistence layer for hnswlib-wasm with JSON and binary format support for Node.js",
+  "main": "./dist/index.cjs",
+  "type": "module",
+  "exports": {
+    ".": {
+      "import": "./dist/index.js",
+      "require": "./dist/index.cjs"
+    }
+  },
+  "files": [
+    "dist"
+  ],
+  "keywords": [
+    "hnswlib",
+    "hnswlib-wasm",
+    "vector-search",
+    "similarity-search",
+    "persistence",
+    "index",
+    "nearest-neighbor",
+    "nodejs"
+  ],
+  "author": "",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": ""
+  },
+  "bugs": {
+    "url": ""
+  },
+  "homepage": "",
+  "peerDependencies": {
+    "hnswlib-wasm": "^0.8.2"
+  },
+  "engines": {
+    "node": ">=18.0.0"
+  },
+  "scripts": {
+    "build": "tsup",
+    "prepublishOnly": "npm run build",
+    "test": "echo \"Error: no test specified\" && exit 1"
+  },
+  "devDependencies": {
+    "tsup": "^8.5.1",
+    "typescript": "^5.9.3"
+  }
+}