eigen-db 4.3.0 → 5.0.0

package/CHANGELOG.md

@@ -1,3 +1,11 @@
+# v5.0.0
+
+Changed: replaced `topK` with `limit` and `order` parameters in `query()` method
+
+# v4.4.0
+
+Added: `entries()`, `keys()`, `values()`, `delete()`, `has()` methods, and `dimensions` property
+
 # v4.3.0
 
 Added: option to choose between in memory or OPFS storage backends

package/README.md

@@ -1,6 +1,6 @@
 # Eigen DB
 
-High-performance vector database for the web.
+High-performance vector database for the web, powered by Web Assembly.
 
 `eigen-db` stores and queries embedding vectors in-browser, using:
 
@@ -16,7 +16,7 @@ npm install eigen-db
 
 ## Guide: Set up and query
 
-### 1) Open a database
+### Open a database
 
 ```ts
 import { DB } from "eigen-db";
@@ -39,7 +39,7 @@ const db = await DB.open({
 });
 ```
 
-### 2) Insert vectors
+### Insert vectors
 
 ```ts
 db.set("doc:1", embedding1);
@@ -56,13 +56,40 @@ Notes:
 - Each vector must be a `number[]` (or `Float32Array`) with exactly `dimensions` elements.
 - Duplicate keys use last-write-wins semantics.
 
-### 3) Query nearest vectors
+### Look up, check, and remove vectors
+
+```ts
+db.get("doc:1"); // number[] | undefined
+db.has("doc:1"); // true
+db.delete("doc:1"); // true (removed), false (not found)
+db.dimensions; // configured vector dimensions
+db.size; // number of entries
+```
+
+### Iterate over the database
+
+```ts
+// Iterate over all keys
+for (const key of db.keys()) {
+  console.log(key);
+}
+
+// Iterate over all [key, value] pairs
+for (const [key, vector] of db.entries()) {
+  console.log(key, vector);
+}
+
+// Spread into an array (uses Symbol.iterator, same as entries())
+const all = [...db];
+```
+
+### Query nearest vectors
 
 ```ts
 const queryVector = embeddingQuery;
 
 // Returns a plain array of { key, similarity } sorted by descending similarity
-const results = db.query(queryVector, { topK: 10 });
+const results = db.query(queryVector, { limit: 10 });
 
 for (const { key, similarity } of results) {
   console.log(key, similarity);
@@ -72,7 +99,7 @@ for (const { key, similarity } of results) {
 For lazy iteration (useful for pagination or early stopping):
 
 ```ts
-const results = db.query(queryVector, { topK: 100, iterable: true });
+const results = db.query(queryVector, { limit: 100, iterable: true });
 
 // Iterate and break early — keys are resolved on demand
 for (const { key, similarity } of results) {
@@ -84,17 +111,27 @@ for (const { key, similarity } of results) {
 const all = [...results];
 ```
 
-Use `minSimilarity` to automatically cut off results below a threshold:
+Use `minSimilarity` and `maxSimilarity` to filter results by a similarity range:
 
 ```ts
 // Only return results with similarity ≥ 0.7 (inclusive)
 const results = db.query(queryVector, { minSimilarity: 0.7 });
 
-// Works with iterable mode too — iteration stops early at the threshold
-const results = db.query(queryVector, { minSimilarity: 0.7, iterable: true });
+// Only return results with similarity ≤ 0.5 (inclusive)
+const results = db.query(queryVector, { maxSimilarity: 0.5 });
+
+// Combine both for a range
+const results = db.query(queryVector, { minSimilarity: 0.3, maxSimilarity: 0.8 });
+```
+
+Use `order: "ascend"` to get the least similar results first (bottom-K):
+
+```ts
+// Least similar results first
+const bottomK = db.query(queryVector, { order: "ascend", limit: 10 });
 ```
 
-### 4) Persist and lifecycle
+### Persist and lifecycle
 
 ```ts
 await db.flush(); // persist current state
@@ -107,7 +144,7 @@ To delete all vectors and storage:
 await db.clear();
 ```
 
-### 5) Export and import
+### Export and import
 
 Export the entire database as a streaming binary file:
 
@@ -150,11 +187,11 @@ Similarity is the dot product of the query and stored vectors.
 
 **When to normalize:**
 
-| Scenario | Normalize? | Notes |
-| --- | --- | --- |
+| Scenario                                   | Normalize?       | Notes                                                                       |
+| ------------------------------------------ | ---------------- | --------------------------------------------------------------------------- |
 | Using embeddings from OpenAI, Cohere, etc. | `true` (default) | Embeddings may not be unit-length; normalization ensures cosine similarity. |
-| Vectors are already unit-length | Either | Setting `false` avoids redundant work. |
-| You need raw dot-product semantics | `false` | Similarity will be the raw dot product; range depends on vector magnitudes. |
+| Vectors are already unit-length            | Either           | Setting `false` avoids redundant work.                                      |
+| You need raw dot-product semantics         | `false`          | Similarity will be the raw dot product; range depends on vector magnitudes. |
 
 ## Full API Reference
 
@@ -183,6 +220,7 @@ Opens (or creates) a database instance and loads persisted data.
 #### Properties
 
 - `size: number` — current number of key-vector pairs
+- `dimensions: number` — number of dimensions per vector
 
 #### Methods
 
@@ -191,10 +229,20 @@ Opens (or creates) a database instance and loads persisted data.
   - Throws on dimension mismatch.
 - `get(key: string): number[] | undefined`
   - Returns a copy of the stored vector.
+- `has(key: string): boolean`
+  - Returns `true` if the key exists, `false` otherwise. O(1) lookup.
+- `delete(key: string): boolean`
+  - Removes the entry for the given key. Returns `true` if the key existed, `false` otherwise.
 - `setMany(entries: [string, VectorInput][]): void`
   - Batch insert/update.
 - `getMany(keys: string[]): (number[] | undefined)[]`
   - Batch lookup.
+- `keys(): IterableIterator<string>`
+  - Returns an iterable of all keys.
+- `entries(): IterableIterator<[string, number[]]>`
+  - Returns an iterable of `[key, value]` pairs. Values are plain number array copies.
+- `[Symbol.iterator](): IterableIterator<[string, number[]]>`
+  - Same as `entries()`. Enables `[...db]` and `for...of` iteration.
 - `query(value: VectorInput, options?: QueryOptions): ResultItem[]`
   - Returns results sorted by descending similarity as a plain array.
   - Throws on dimension mismatch.
@@ -265,8 +313,10 @@ interface SetOptions {
 
 ```ts
 interface QueryOptions {
-  topK?: number; // default: Infinity (all results)
+  limit?: number; // default: Infinity (all results)
+  order?: "ascend" | "descend"; // default: "descend" (most similar first)
   minSimilarity?: number; // inclusive lower bound on similarity; results below this are excluded
+  maxSimilarity?: number; // inclusive upper bound on similarity; results above this are excluded
   normalize?: boolean;
   iterable?: boolean; // when true, returns Iterable<ResultItem> instead of ResultItem[]
 }
@@ -311,12 +361,12 @@ Thrown when memory growth would exceed WASM 32-bit memory limits for the configu
 
 WASM SIMD vs pure JavaScript performance on 1536-dimensional vectors (OpenAI embedding size), measured with `vitest bench` (Node.js):
 
-| Operation | JS (ops/s) | WASM SIMD (ops/s) | Speedup |
-| --- | --- | --- | --- |
-| normalize (1536 dims) | 223,117 | 2,226,734 | **~10×** |
-| searchAll (100 vectors × 1536 dims) | 3,429 | 77,130 | **~22×** |
-| searchAll (1,000 vectors × 1536 dims) | 344 | 8,009 | **~23×** |
-| searchAll (10,000 vectors × 1536 dims) | 34 | 398 | **~12×** |
+| Operation                              | JS (ops/s) | WASM SIMD (ops/s) | Speedup  |
+| -------------------------------------- | ---------- | ----------------- | -------- |
+| normalize (1536 dims)                  | 223,117    | 2,226,734         | **~10×** |
+| searchAll (100 vectors × 1536 dims)    | 3,429      | 77,130            | **~22×** |
+| searchAll (1,000 vectors × 1536 dims)  | 344        | 8,009             | **~23×** |
+| searchAll (10,000 vectors × 1536 dims) | 34         | 398               | **~12×** |
 
 The WASM SIMD layer uses 2-vector outer loop unrolling (halving query memory reads) and 4× inner loop unrolling with multiple independent accumulators.
 
@@ -338,7 +388,8 @@ npm run dev
 ## Practical notes
 
 - Similarity is the dot product of query and stored vectors; with normalization enabled (default), this behaves like cosine similarity (1 = identical, -1 = opposite).
-- `topK` defaults to `Infinity`, returning all stored vectors sorted by similarity. Use `minSimilarity` to limit results by proximity.
+- `limit` defaults to `Infinity`, returning all stored vectors sorted by similarity. Use `minSimilarity` and `maxSimilarity` to filter results by proximity range.
+- `order` defaults to `"descend"` (most similar first). Use `"ascend"` to get least similar first.
 - Querying an empty database returns an empty array (`[]`).
 - `flush()` writes deduplicated state, and reopen preserves key-to-slot mapping.
 

package/dist/compute.d.ts

@@ -0,0 +1,20 @@
+/**
+ * Pure JavaScript compute functions for vector operations.
+ * These serve as the reference implementation and fallback when WASM SIMD is unavailable.
+ */
+/**
+ * Normalizes a vector in-place to unit length.
+ * After normalization, cosine similarity reduces to a simple dot product.
+ */
+export declare function normalize(vec: Float32Array): void;
+/**
+ * Computes dot products of query against all vectors in the database.
+ * Writes scores to the output array.
+ *
+ * @param query - Normalized query vector (length = dimensions)
+ * @param db - Contiguous flat array of normalized vectors (length = dbSize * dimensions)
+ * @param scores - Output array for dot product scores (length = dbSize)
+ * @param dbSize - Number of vectors in the database
+ * @param dimensions - Dimensionality of each vector
+ */
+export declare function searchAll(query: Float32Array, db: Float32Array, scores: Float32Array, dbSize: number, dimensions: number): void;