@zukhruf/embeddings 0.3.47 → 0.3.48

package/README.md

@@ -47,6 +47,23 @@ collection.nearest('a', { k: 10, withinDays: 7 }); // [{ id: 'b', similarity:
 collection.similarity('a', 'c');
 ```
 
+### Search a vector table of your own
+
+Another program can write the vectors into its own table. A `VectorSearch` searches that table with the same rules as a collection. The table or view must have the columns `id` (text), `vector` and `published_at` (milliseconds since the Unix epoch). Each `vector` must be a unit vector, as a BLOB of float32 values in the byte order of the platform ([ADR 0005](./docs/adr/0005-a-vector-search-reads-a-vector-table-of-unit-vectors.md)).
+
+```ts
+import { DatabaseSync } from 'node:sqlite';
+
+import { VectorSearch } from '@zukhruf/embeddings';
+
+using database = new DatabaseSync('listings.sqlite');
+const search = new VectorSearch({ database, table: 'listing_vectors' });
+
+search.nearest('listing-42', { k: 10, withinDays: 30 });
+```
+
+To search a table with other column names, make a view that renames the columns, and give the name of the view.
+
 ## API
 
 - `new Collection({ database, model })`. The caller opens and closes `database`. The collection makes its table `embedding_document` in it. `model` is an AI SDK embedding model object.
@@ -54,10 +71,12 @@ collection.similarity('a', 'c');
 - `nearest(id, { k, withinDays })` gives at most `k` neighbours of the document, with the highest similarity first. A neighbour has a publish date at most `withinDays` days before or after the publish date of the document. Equal similarities sort by id.
 - `neighbourGraph({ from, to, k, withinDays })` gives an iterator of `[id, neighbours]` entries, one for each document published from `from` to `to`, in the order of the publish dates and then of the ids. The neighbours of each entry are equal to the result of `nearest(id, { k, withinDays })`. A document outside the range has no entry, but it can be a neighbour of a document in the range. The call reads and decodes each vector one time ([ADR 0004](./docs/adr/0004-a-neighbour-graph-decodes-each-vector-once.md)). Use it when you need the neighbours of many documents.
 - `similarity(a, b)` gives the cosine similarity of two stored documents.
+- `new VectorSearch({ database, table })`. The caller opens and closes `database`. `table` is the name of a table or a view in it, with the columns `id`, `vector` and `published_at`. The vector search only reads it. Each call reads the table again, so the table can change between two calls.
+- `VectorSearch` gives `nearest`, `neighbourGraph` and `similarity`, with the same rules and results as these calls of a collection. A collection gives these calls to a `VectorSearch` over its table `embedding_document`.
 
 ## Errors
 
-- `UnknownDocumentError`: `nearest` or `similarity` got an id that the collection does not store.
+- `UnknownDocumentError`: `nearest` or `similarity` got an id that the vector table does not contain.
 - `DimensionMismatchError`: the model gave a vector with a dimension that is not the dimension of the stored vectors. The upsert stores none of its documents. To change to a model with another dimension, upsert all documents in one call.
 
 ## Limits
@@ -65,6 +84,7 @@ collection.similarity('a', 'c');
 - One database holds one collection.
 - The collection stores the vector of the text, not the text. The caller chooses the text, for example a title and the first sentences.
 - A document has one vector. A long text gives one mean vector, and its details become a small part of it.
+- A vector search requires unit vectors. It does not examine the length or the dimension of a vector.
 - The search reads each document in the window ([ADR 0001](./docs/adr/0001-search-compares-each-document-in-the-window.md)). A window of many thousand documents is slow. A neighbour graph decodes each vector once, but it still calculates a similarity for each pair of documents in a window.
 - After a model change, upsert all documents again ([ADR 0002](./docs/adr/0002-the-hash-includes-the-model.md)).
 
@@ -74,4 +94,5 @@ collection.similarity('a', 'c');
 - [ADR 0002: The hash includes the model](./docs/adr/0002-the-hash-includes-the-model.md)
 - [ADR 0003: All vectors of a collection have one dimension](./docs/adr/0003-all-vectors-of-a-collection-have-one-dimension.md)
 - [ADR 0004: A neighbour graph decodes each vector once](./docs/adr/0004-a-neighbour-graph-decodes-each-vector-once.md)
+- [ADR 0005: A vector search reads a vector table of unit vectors](./docs/adr/0005-a-vector-search-reads-a-vector-table-of-unit-vectors.md)
 - [Code copied from @deepagents/retrieval](./docs/copied-from-deepagents-retrieval.md)

package/dist/collection.d.ts

@@ -1,14 +1,11 @@
 import type { DatabaseSync } from 'node:sqlite';
 import { type EmbeddingModel } from 'ai';
+import { type Neighbour } from './vector-search.ts';
 export type Document = {
     readonly id: string;
     readonly text: string;
     readonly publishedAt: Date;
 };
-export type Neighbour = {
-    readonly id: string;
-    readonly similarity: number;
-};
 export type CollectionOptions = {
     readonly database: DatabaseSync;
     readonly model: Exclude<EmbeddingModel, string>;

package/dist/collection.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"collection.d.ts","sourceRoot":"","sources":["../src/collection.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAEhD,OAAO,EAAE,KAAK,cAAc,EAAa,MAAM,IAAI,CAAC;AAWpD,MAAM,MAAM,QAAQ,GAAG;IACrB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IAEpB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,WAAW,EAAE,IAAI,CAAC;CAC5B,CAAC;AAEF,MAAM,MAAM,SAAS,GAAG;IACtB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;CAC7B,CAAC;AAEF,MAAM,MAAM,iBAAiB,GAAG;IAE9B,QAAQ,CAAC,QAAQ,EAAE,YAAY,CAAC;IAGhC,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAC,cAAc,EAAE,MAAM,CAAC,CAAC;CACjD,CAAC;AAKF,qBAAa,UAAU;;gBAIT,OAAO,EAAE,iBAAiB;IAQhC,MAAM,CAAC,SAAS,EAAE,SAAS,QAAQ,EAAE,GAAG,OAAO,CAAC,IAAI,CAAC;IAsD3D,OAAO,CACL,EAAE,EAAE,MAAM,EACV,OAAO,EAAE;QAAE,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;KAAE,GAC3D,SAAS,EAAE;IAoBd,cAAc,CAAC,OAAO,EAAE;QACtB,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC;QACpB,QAAQ,CAAC,EAAE,EAAE,IAAI,CAAC;QAClB,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC;QACnB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;KAC7B,GAAG,cAAc,CAAC,CAAC,MAAM,EAAE,SAAS,EAAE,CAAC,CAAC;IAgCzC,UAAU,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,MAAM,GAAG,MAAM;CAqDzC"}
+{"version":3,"file":"collection.d.ts","sourceRoot":"","sources":["../src/collection.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAEhD,OAAO,EAAE,KAAK,cAAc,EAAa,MAAM,IAAI,CAAC;AAMpD,OAAO,EAAE,KAAK,SAAS,EAAgB,MAAM,oBAAoB,CAAC;AAGlE,MAAM,MAAM,QAAQ,GAAG;IACrB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IAEpB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,WAAW,EAAE,IAAI,CAAC;CAC5B,CAAC;AAEF,MAAM,MAAM,iBAAiB,GAAG;IAE9B,QAAQ,CAAC,QAAQ,EAAE,YAAY,CAAC;IAGhC,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAC,cAAc,EAAE,MAAM,CAAC,CAAC;CACjD,CAAC;AAIF,qBAAa,UAAU;;gBAKT,OAAO,EAAE,iBAAiB;IAYhC,MAAM,CAAC,SAAS,EAAE,SAAS,QAAQ,EAAE,GAAG,OAAO,CAAC,IAAI,CAAC;IAoD3D,OAAO,CACL,EAAE,EAAE,MAAM,EACV,OAAO,EAAE;QAAE,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;KAAE,GAC3D,SAAS,EAAE;IAKd,cAAc,CAAC,OAAO,EAAE;QACtB,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC;QACpB,QAAQ,CAAC,EAAE,EAAE,IAAI,CAAC;QAClB,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC;QACnB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;KAC7B,GAAG,cAAc,CAAC,CAAC,MAAM,EAAE,SAAS,EAAE,CAAC,CAAC;IAKzC,UAAU,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,MAAM,GAAG,MAAM;CAwCzC"}

package/dist/collection.js

@@ -3,19 +3,22 @@ import { embedMany } from 'ai';
 import { transaction } from '@zukhruf/sqlite-transaction';
 import { DimensionMismatchError } from "./dimension-mismatch-error.js";
 import { collectionSchema } from "./schema.js";
-import { UnknownDocumentError } from "./unknown-document-error.js";
+import { VectorSearch } from "./vector-search.js";
 import { UnitVector } from "./vector.js";
-const DAY_MS = 86_400_000;
 // The embeddings of documents, one vector for each document, in a SQLite
-// database. A search compares the document with each document published
-// inside the window, with no index (docs/adr/0001).
+// database. The collection searches its table through a VectorSearch.
 export class Collection {
     #database;
     #model;
+    #search;
     constructor(options) {
         this.#database = options.database;
         this.#model = options.model;
         collectionSchema.apply(this.#database);
+        this.#search = new VectorSearch({
+            database: this.#database,
+            table: 'embedding_document',
+        });
     }
     // Embeds the documents whose text or model changed since their last upsert,
     // and stores the publish date of each document.
@@ -42,50 +45,17 @@ export class Collection {
             this.#checkDimension(changed.map(({ document }) => document.id), vectors);
         }, { mode: 'immediate' });
     }
-    // The k documents most similar to the document, by cosine similarity, from
-    // those published at most withinDays before or after it. The document
-    // itself is not one of them. Equal similarities sort by id.
+    // The neighbours of a document (docs/adr/0005).
     nearest(id, options) {
-        const { vector, publishedAt } = this.#read(id);
-        const window = options.withinDays * DAY_MS;
-        const candidates = this.#database
-            .prepare('SELECT "id", "vector" FROM "embedding_document" WHERE "published_at" BETWEEN ? AND ? AND "id" != ?')
-            .all(publishedAt - window, publishedAt + window, id)
-            .map((row) => ({
-            id: String(row.id),
-            vector: UnitVector.fromBlob(blob(row.vector)),
-        }));
-        return ranked(vector, candidates, options.k);
+        return this.#search.nearest(id, options);
     }
-    // The k nearest neighbours of each document published from `from` to `to`.
-    // Each entry is what nearest gives for the document with the same k and
-    // withinDays. The call reads and decodes each vector once
-    // (docs/adr/0004). The entries come in the order of the publish dates, then
-    // of the ids.
+    // The neighbours of each document published from `from` to `to`.
     neighbourGraph(options) {
-        const from = options.from.getTime();
-        const to = options.to.getTime();
-        const window = options.withinDays * DAY_MS;
-        const documents = this.#database
-            .prepare('SELECT "id", "published_at", "vector" FROM "embedding_document" WHERE "published_at" BETWEEN ? AND ? ORDER BY "published_at", "id"')
-            .all(from - window, to + window)
-            .map((row) => ({
-            id: String(row.id),
-            publishedAt: Number(row.published_at),
-            vector: UnitVector.fromBlob(blob(row.vector)),
-        }));
-        return documents
-            .values()
-            .filter(({ publishedAt }) => publishedAt >= from && publishedAt <= to)
-            .map((document) => [
-            document.id,
-            ranked(document.vector, documents.filter((other) => other.id !== document.id &&
-                Math.abs(other.publishedAt - document.publishedAt) <= window), options.k),
-        ]);
+        return this.#search.neighbourGraph(options);
     }
     // The cosine similarity of two stored documents.
     similarity(a, b) {
-        return this.#read(a).vector.cosine(this.#read(b).vector);
+        return this.#search.similarity(a, b);
     }
     // No texts make no call to the model.
     async #embed(texts) {
@@ -103,17 +73,6 @@ export class Collection {
             .update(JSON.stringify([this.#model.provider, this.#model.modelId, text]))
             .digest('hex');
     }
-    #read(id) {
-        const row = this.#database
-            .prepare('SELECT "vector", "published_at" FROM "embedding_document" WHERE "id" = ?')
-            .get(id);
-        if (row === undefined)
-            throw new UnknownDocumentError(id);
-        return {
-            vector: UnitVector.fromBlob(blob(row.vector)),
-            publishedAt: Number(row.published_at),
-        };
-    }
     // Before the upsert, each stored vector has one dimension. The new vectors
     // must have the dimension of a vector that the upsert did not replace, and
     // of each other.
@@ -132,28 +91,4 @@ export class Collection {
                 throw new DimensionMismatchError(expected, vector.dimension);
     }
 }
-// The k candidates most similar to the vector. Equal similarities sort by id.
-function ranked(vector, candidates, k) {
-    return candidates
-        .map((candidate) => ({
-        id: candidate.id,
-        similarity: vector.cosine(candidate.vector),
-    }))
-        .toSorted((left, right) => right.similarity - left.similarity || compareIds(left.id, right.id))
-        .slice(0, k);
-}
-// Compares by code unit, not by locale, so the order is the same on every
-// machine.
-function compareIds(left, right) {
-    if (left < right)
-        return -1;
-    if (left > right)
-        return 1;
-    return 0;
-}
-function blob(value) {
-    if (!(value instanceof Uint8Array))
-        throw new TypeError('A stored vector must be a BLOB.');
-    return value;
-}
 //# sourceMappingURL=collection.js.map

package/dist/collection.js.map

@@ -1 +1 @@
-{"version":3,"file":"collection.js","sourceRoot":"","sources":["../src/collection.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAGzC,OAAO,EAAuB,SAAS,EAAE,MAAM,IAAI,CAAC;AAEpD,OAAO,EAAE,WAAW,EAAE,MAAM,6BAA6B,CAAC;AAE1D,OAAO,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AACvE,OAAO,EAAE,gBAAgB,EAAE,MAAM,aAAa,CAAC;AAC/C,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAEzC,MAAM,MAAM,GAAG,UAAU,CAAC;AAsB1B,yEAAyE;AACzE,wEAAwE;AACxE,oDAAoD;AACpD,MAAM,OAAO,UAAU;IACZ,SAAS,CAAe;IACxB,MAAM,CAAkC;IAEjD,YAAY,OAA0B;QACpC,IAAI,CAAC,SAAS,GAAG,OAAO,CAAC,QAAQ,CAAC;QAClC,IAAI,CAAC,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC;QAC5B,gBAAgB,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IACzC,CAAC;IAED,4EAA4E;IAC5E,gDAAgD;IAChD,KAAK,CAAC,MAAM,CAAC,SAA8B;QACzC,MAAM,MAAM,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACnC,wDAAwD,CACzD,CAAC;QACF,MAAM,MAAM,GAAG,SAAS,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC;YAC1C,QAAQ;YACR,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC;SAChC,CAAC,CAAC,CAAC;QACJ,MAAM,EAAE,OAAO,GAAG,EAAE,EAAE,SAAS,GAAG,EAAE,EAAE,GAAG,MAAM,CAAC,OAAO,CACrD,MAAM,EACN,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE,EAAE,CACrB,MAAM,CAAC,GAAG,CAAC,QAAQ,CAAC,EAAE,CAAC,EAAE,IAAI,KAAK,IAAI,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,SAAS,CACnE,CAAC;QACF,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,MAAM,CAC/B,OAAO,CAAC,GAAG,CAAC,CAAC,EAAE,QAAQ,EAAE,EAAE,EAAE,CAAC,QAAQ,CAAC,IAAI,CAAC,CAC7C,CAAC;QAEF,WAAW,CACT,IAAI,CAAC,SAAS,EACd,GAAG,EAAE;YACH,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CAClC;4IACkI,CACnI,CAAC;YACF,KAAK,MAAM,CAAC,KAAK,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC,IAAI,OAAO,CAAC,OAAO,EAAE,EAAE,CAAC;gBAC5D,MAAM,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC;gBAC9B,IAAI,MAAM,KAAK,SAAS;oBACtB,MAAM,IAAI,KAAK,CACb,kBAAkB,OAAO,CAAC,MAAM,mBAAmB,OAAO,CAAC,MAAM,SAAS,CAC3E,CAAC;gBACJ,KAAK,CAAC,GAAG,CACP,QAAQ,CAAC,EAAE,EACX,IAAI,EACJ,QAAQ,CAAC,WAAW,CAAC,OAAO,EAAE,EAC9B,MAAM,CAAC,MAAM,EAAE,CAChB,CAAC;YACJ,CAAC;YACD,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACjC,mEAAmE,CACpE,CAAC;YACF,KAAK,MAAM,EAAE,QAAQ,EAAE,IAAI,SAAS;gBAClC,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,WAAW,CAAC,OAAO,EAAE,EAAE,QAAQ,CAAC,EAAE,CAAC,CAAC;YACxD,IAAI,CAAC,eAAe,CAClB,OAAO,CAAC,GAAG,CAAC,CAAC,EAAE,QAAQ,EAAE,EAAE,EAAE,CAAC,QAAQ,CAAC,EAAE,CAAC,EAC1C,OAAO,CACR,CAAC;QACJ,CAAC,EACD,EAAE,IAAI,EAAE,WAAW,EAAE,CACtB,CAAC;IACJ,CAAC;IAED,2EAA2E;IAC3E,sEAAsE;IACtE,4DAA4D;IAC5D,OAAO,CACL,EAAU,EACV,OAA4D;QAE5D,MAAM,EAAE,MAAM,EAAE,WAAW,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;QAC/C,MAAM,MAAM,GAAG,OAAO,CAAC,UAAU,GAAG,MAAM,CAAC;QAC3C,MAAM,UAAU,GAAG,IAAI,CAAC,SAAS;aAC9B,OAAO,CACN,oGAAoG,CACrG;aACA,GAAG,CAAC,WAAW,GAAG,MAAM,EAAE,WAAW,GAAG,MAAM,EAAE,EAAE,CAAC;aACnD,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;YACb,EAAE,EAAE,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;YAClB,MAAM,EAAE,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;SAC9C,CAAC,CAAC,CAAC;QACN,OAAO,MAAM,CAAC,MAAM,EAAE,UAAU,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC;IAC/C,CAAC;IAED,2EAA2E;IAC3E,wEAAwE;IACxE,0DAA0D;IAC1D,4EAA4E;IAC5E,cAAc;IACd,cAAc,CAAC,OAKd;QACC,MAAM,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,OAAO,EAAE,CAAC;QACpC,MAAM,EAAE,GAAG,OAAO,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC;QAChC,MAAM,MAAM,GAAG,OAAO,CAAC,UAAU,GAAG,MAAM,CAAC;QAC3C,MAAM,SAAS,GAAG,IAAI,CAAC,SAAS;aAC7B,OAAO,CACN,oIAAoI,CACrI;aACA,GAAG,CAAC,IAAI,GAAG,MAAM,EAAE,EAAE,GAAG,MAAM,CAAC;aAC/B,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;YACb,EAAE,EAAE,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;YAClB,WAAW,EAAE,MAAM,CAAC,GAAG,CAAC,YAAY,CAAC;YACrC,MAAM,EAAE,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;SAC9C,CAAC,CAAC,CAAC;QACN,OAAO,SAAS;aACb,MAAM,EAAE;aACR,MAAM,CAAC,CAAC,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC,WAAW,IAAI,IAAI,IAAI,WAAW,IAAI,EAAE,CAAC;aACrE,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC;YACjB,QAAQ,CAAC,EAAE;YACX,MAAM,CACJ,QAAQ,CAAC,MAAM,EACf,SAAS,CAAC,MAAM,CACd,CAAC,KAAK,EAAE,EAAE,CACR,KAAK,CAAC,EAAE,KAAK,QAAQ,CAAC,EAAE;gBACxB,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,WAAW,GAAG,QAAQ,CAAC,WAAW,CAAC,IAAI,MAAM,CAC/D,EACD,OAAO,CAAC,CAAC,CACV;SACF,CAAC,CAAC;IACP,CAAC;IAED,iDAAiD;IACjD,UAAU,CAAC,CAAS,EAAE,CAAS;QAC7B,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;IAC3D,CAAC;IAED,sCAAsC;IACtC,KAAK,CAAC,MAAM,CAAC,KAAe;QAC1B,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,EAAE,CAAC;QAClC,MAAM,EAAE,UAAU,EAAE,GAAG,MAAM,SAAS,CAAC;YACrC,KAAK,EAAE,IAAI,CAAC,MAAM;YAClB,MAAM,EAAE,KAAK;SACd,CAAC,CAAC;QACH,OAAO,UAAU,CAAC,GAAG,CAAC,CAAC,SAAS,EAAE,EAAE,CAAC,UAAU,CAAC,SAAS,CAAC,SAAS,CAAC,CAAC,CAAC;IACxE,CAAC;IAED,uEAAuE;IACvE,KAAK,CAAC,IAAY;QAChB,OAAO,UAAU,CAAC,QAAQ,CAAC;aACxB,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,QAAQ,EAAE,IAAI,CAAC,MAAM,CAAC,OAAO,EAAE,IAAI,CAAC,CAAC,CAAC;aACzE,MAAM,CAAC,KAAK,CAAC,CAAC;IACnB,CAAC;IAED,KAAK,CAAC,EAAU;QACd,MAAM,GAAG,GAAG,IAAI,CAAC,SAAS;aACvB,OAAO,CACN,0EAA0E,CAC3E;aACA,GAAG,CAAC,EAAE,CAAC,CAAC;QACX,IAAI,GAAG,KAAK,SAAS;YAAE,MAAM,IAAI,oBAAoB,CAAC,EAAE,CAAC,CAAC;QAC1D,OAAO;YACL,MAAM,EAAE,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;YAC7C,WAAW,EAAE,MAAM,CAAC,GAAG,CAAC,YAAY,CAAC;SACtC,CAAC;IACJ,CAAC;IAED,2EAA2E;IAC3E,2EAA2E;IAC3E,iBAAiB;IACjB,eAAe,CAAC,QAA2B,EAAE,OAA8B;QACzE,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,CAAC;QACxB,IAAI,KAAK,KAAK,SAAS;YAAE,OAAO;QAChC,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS;aACxB,OAAO,CACN,2HAA2H,CAC5H;aACA,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC,CAAC;QACjC,MAAM,QAAQ,GACZ,IAAI,KAAK,SAAS;YAChB,CAAC,CAAC,KAAK,CAAC,SAAS;YACjB,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,YAAY,CAAC,iBAAiB,CAAC;QAC1D,KAAK,MAAM,MAAM,IAAI,OAAO;YAC1B,IAAI,MAAM,CAAC,SAAS,KAAK,QAAQ;gBAC/B,MAAM,IAAI,sBAAsB,CAAC,QAAQ,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;IACnE,CAAC;CACF;AAED,8EAA8E;AAC9E,SAAS,MAAM,CACb,MAAkB,EAClB,UAA2E,EAC3E,CAAS;IAET,OAAO,UAAU;SACd,GAAG,CAAC,CAAC,SAAS,EAAE,EAAE,CAAC,CAAC;QACnB,EAAE,EAAE,SAAS,CAAC,EAAE;QAChB,UAAU,EAAE,MAAM,CAAC,MAAM,CAAC,SAAS,CAAC,MAAM,CAAC;KAC5C,CAAC,CAAC;SACF,QAAQ,CACP,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE,CACd,KAAK,CAAC,UAAU,GAAG,IAAI,CAAC,UAAU,IAAI,UAAU,CAAC,IAAI,CAAC,EAAE,EAAE,KAAK,CAAC,EAAE,CAAC,CACtE;SACA,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;AACjB,CAAC;AAED,0EAA0E;AAC1E,WAAW;AACX,SAAS,UAAU,CAAC,IAAY,EAAE,KAAa;IAC7C,IAAI,IAAI,GAAG,KAAK;QAAE,OAAO,CAAC,CAAC,CAAC;IAC5B,IAAI,IAAI,GAAG,KAAK;QAAE,OAAO,CAAC,CAAC;IAC3B,OAAO,CAAC,CAAC;AACX,CAAC;AAED,SAAS,IAAI,CAAC,KAAc;IAC1B,IAAI,CAAC,CAAC,KAAK,YAAY,UAAU,CAAC;QAChC,MAAM,IAAI,SAAS,CAAC,iCAAiC,CAAC,CAAC;IACzD,OAAO,KAAK,CAAC;AACf,CAAC"}
+{"version":3,"file":"collection.js","sourceRoot":"","sources":["../src/collection.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAGzC,OAAO,EAAuB,SAAS,EAAE,MAAM,IAAI,CAAC;AAEpD,OAAO,EAAE,WAAW,EAAE,MAAM,6BAA6B,CAAC;AAE1D,OAAO,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AACvE,OAAO,EAAE,gBAAgB,EAAE,MAAM,aAAa,CAAC;AAC/C,OAAO,EAAkB,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAClE,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAiBzC,yEAAyE;AACzE,sEAAsE;AACtE,MAAM,OAAO,UAAU;IACZ,SAAS,CAAe;IACxB,MAAM,CAAkC;IACxC,OAAO,CAAe;IAE/B,YAAY,OAA0B;QACpC,IAAI,CAAC,SAAS,GAAG,OAAO,CAAC,QAAQ,CAAC;QAClC,IAAI,CAAC,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC;QAC5B,gBAAgB,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;QACvC,IAAI,CAAC,OAAO,GAAG,IAAI,YAAY,CAAC;YAC9B,QAAQ,EAAE,IAAI,CAAC,SAAS;YACxB,KAAK,EAAE,oBAAoB;SAC5B,CAAC,CAAC;IACL,CAAC;IAED,4EAA4E;IAC5E,gDAAgD;IAChD,KAAK,CAAC,MAAM,CAAC,SAA8B;QACzC,MAAM,MAAM,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACnC,wDAAwD,CACzD,CAAC;QACF,MAAM,MAAM,GAAG,SAAS,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC;YAC1C,QAAQ;YACR,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC;SAChC,CAAC,CAAC,CAAC;QACJ,MAAM,EAAE,OAAO,GAAG,EAAE,EAAE,SAAS,GAAG,EAAE,EAAE,GAAG,MAAM,CAAC,OAAO,CACrD,MAAM,EACN,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE,EAAE,CACrB,MAAM,CAAC,GAAG,CAAC,QAAQ,CAAC,EAAE,CAAC,EAAE,IAAI,KAAK,IAAI,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,SAAS,CACnE,CAAC;QACF,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,MAAM,CAC/B,OAAO,CAAC,GAAG,CAAC,CAAC,EAAE,QAAQ,EAAE,EAAE,EAAE,CAAC,QAAQ,CAAC,IAAI,CAAC,CAC7C,CAAC;QAEF,WAAW,CACT,IAAI,CAAC,SAAS,EACd,GAAG,EAAE;YACH,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CAClC;4IACkI,CACnI,CAAC;YACF,KAAK,MAAM,CAAC,KAAK,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC,IAAI,OAAO,CAAC,OAAO,EAAE,EAAE,CAAC;gBAC5D,MAAM,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC;gBAC9B,IAAI,MAAM,KAAK,SAAS;oBACtB,MAAM,IAAI,KAAK,CACb,kBAAkB,OAAO,CAAC,MAAM,mBAAmB,OAAO,CAAC,MAAM,SAAS,CAC3E,CAAC;gBACJ,KAAK,CAAC,GAAG,CACP,QAAQ,CAAC,EAAE,EACX,IAAI,EACJ,QAAQ,CAAC,WAAW,CAAC,OAAO,EAAE,EAC9B,MAAM,CAAC,MAAM,EAAE,CAChB,CAAC;YACJ,CAAC;YACD,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACjC,mEAAmE,CACpE,CAAC;YACF,KAAK,MAAM,EAAE,QAAQ,EAAE,IAAI,SAAS;gBAClC,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,WAAW,CAAC,OAAO,EAAE,EAAE,QAAQ,CAAC,EAAE,CAAC,CAAC;YACxD,IAAI,CAAC,eAAe,CAClB,OAAO,CAAC,GAAG,CAAC,CAAC,EAAE,QAAQ,EAAE,EAAE,EAAE,CAAC,QAAQ,CAAC,EAAE,CAAC,EAC1C,OAAO,CACR,CAAC;QACJ,CAAC,EACD,EAAE,IAAI,EAAE,WAAW,EAAE,CACtB,CAAC;IACJ,CAAC;IAED,gDAAgD;IAChD,OAAO,CACL,EAAU,EACV,OAA4D;QAE5D,OAAO,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,EAAE,OAAO,CAAC,CAAC;IAC3C,CAAC;IAED,iEAAiE;IACjE,cAAc,CAAC,OAKd;QACC,OAAO,IAAI,CAAC,OAAO,CAAC,cAAc,CAAC,OAAO,CAAC,CAAC;IAC9C,CAAC;IAED,iDAAiD;IACjD,UAAU,CAAC,CAAS,EAAE,CAAS;QAC7B,OAAO,IAAI,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IACvC,CAAC;IAED,sCAAsC;IACtC,KAAK,CAAC,MAAM,CAAC,KAAe;QAC1B,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,EAAE,CAAC;QAClC,MAAM,EAAE,UAAU,EAAE,GAAG,MAAM,SAAS,CAAC;YACrC,KAAK,EAAE,IAAI,CAAC,MAAM;YAClB,MAAM,EAAE,KAAK;SACd,CAAC,CAAC;QACH,OAAO,UAAU,CAAC,GAAG,CAAC,CAAC,SAAS,EAAE,EAAE,CAAC,UAAU,CAAC,SAAS,CAAC,SAAS,CAAC,CAAC,CAAC;IACxE,CAAC;IAED,uEAAuE;IACvE,KAAK,CAAC,IAAY;QAChB,OAAO,UAAU,CAAC,QAAQ,CAAC;aACxB,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,QAAQ,EAAE,IAAI,CAAC,MAAM,CAAC,OAAO,EAAE,IAAI,CAAC,CAAC,CAAC;aACzE,MAAM,CAAC,KAAK,CAAC,CAAC;IACnB,CAAC;IAED,2EAA2E;IAC3E,2EAA2E;IAC3E,iBAAiB;IACjB,eAAe,CAAC,QAA2B,EAAE,OAA8B;QACzE,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,CAAC;QACxB,IAAI,KAAK,KAAK,SAAS;YAAE,OAAO;QAChC,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS;aACxB,OAAO,CACN,2HAA2H,CAC5H;aACA,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC,CAAC;QACjC,MAAM,QAAQ,GACZ,IAAI,KAAK,SAAS;YAChB,CAAC,CAAC,KAAK,CAAC,SAAS;YACjB,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,YAAY,CAAC,iBAAiB,CAAC;QAC1D,KAAK,MAAM,MAAM,IAAI,OAAO;YAC1B,IAAI,MAAM,CAAC,SAAS,KAAK,QAAQ;gBAC/B,MAAM,IAAI,sBAAsB,CAAC,QAAQ,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;IACnE,CAAC;CACF"}

package/dist/index.d.ts

@@ -1,4 +1,5 @@
-export { Collection, type CollectionOptions, type Document, type Neighbour, } from './collection.ts';
+export { Collection, type CollectionOptions, type Document, } from './collection.ts';
 export { DimensionMismatchError } from './dimension-mismatch-error.ts';
 export { UnknownDocumentError } from './unknown-document-error.ts';
+export { type Neighbour, VectorSearch, type VectorSearchOptions, } from './vector-search.ts';
 //# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,UAAU,EACV,KAAK,iBAAiB,EACtB,KAAK,QAAQ,EACb,KAAK,SAAS,GACf,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AACvE,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,UAAU,EACV,KAAK,iBAAiB,EACtB,KAAK,QAAQ,GACd,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AACvE,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,EACL,KAAK,SAAS,EACd,YAAY,EACZ,KAAK,mBAAmB,GACzB,MAAM,oBAAoB,CAAC"}

package/dist/index.js

@@ -1,4 +1,5 @@
 export { Collection, } from "./collection.js";
 export { DimensionMismatchError } from "./dimension-mismatch-error.js";
 export { UnknownDocumentError } from "./unknown-document-error.js";
+export { VectorSearch, } from "./vector-search.js";
 //# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,UAAU,GAIX,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AACvE,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,UAAU,GAGX,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AACvE,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,EAEL,YAAY,GAEb,MAAM,oBAAoB,CAAC"}

package/dist/unknown-document-error.js

@@ -1,8 +1,8 @@
-// The collection stores no document with this id.
+// The vector table has no document with this id.
 export class UnknownDocumentError extends Error {
     id;
     constructor(id) {
-        super(`The collection stores no document with the id ${id}.`);
+        super(`The vector table has no document with the id ${id}.`);
         this.name = 'UnknownDocumentError';
         this.id = id;
     }

package/dist/unknown-document-error.js.map

@@ -1 +1 @@
-{"version":3,"file":"unknown-document-error.js","sourceRoot":"","sources":["../src/unknown-document-error.ts"],"names":[],"mappings":"AAAA,kDAAkD;AAClD,MAAM,OAAO,oBAAqB,SAAQ,KAAK;IACpC,EAAE,CAAS;IAEpB,YAAY,EAAU;QACpB,KAAK,CAAC,iDAAiD,EAAE,GAAG,CAAC,CAAC;QAC9D,IAAI,CAAC,IAAI,GAAG,sBAAsB,CAAC;QACnC,IAAI,CAAC,EAAE,GAAG,EAAE,CAAC;IACf,CAAC;CACF"}
+{"version":3,"file":"unknown-document-error.js","sourceRoot":"","sources":["../src/unknown-document-error.ts"],"names":[],"mappings":"AAAA,iDAAiD;AACjD,MAAM,OAAO,oBAAqB,SAAQ,KAAK;IACpC,EAAE,CAAS;IAEpB,YAAY,EAAU;QACpB,KAAK,CAAC,gDAAgD,EAAE,GAAG,CAAC,CAAC;QAC7D,IAAI,CAAC,IAAI,GAAG,sBAAsB,CAAC;QACnC,IAAI,CAAC,EAAE,GAAG,EAAE,CAAC;IACf,CAAC;CACF"}

package/dist/vector-search.d.ts

@@ -0,0 +1,25 @@
+import type { DatabaseSync } from 'node:sqlite';
+export type Neighbour = {
+    readonly id: string;
+    readonly similarity: number;
+};
+export type VectorSearchOptions = {
+    readonly database: DatabaseSync;
+    readonly table: string;
+};
+export declare class VectorSearch {
+    #private;
+    constructor(options: VectorSearchOptions);
+    nearest(id: string, options: {
+        readonly k: number;
+        readonly withinDays: number;
+    }): Neighbour[];
+    neighbourGraph(options: {
+        readonly from: Date;
+        readonly to: Date;
+        readonly k: number;
+        readonly withinDays: number;
+    }): IteratorObject<[string, Neighbour[]]>;
+    similarity(a: string, b: string): number;
+}
+//# sourceMappingURL=vector-search.d.ts.map

package/dist/vector-search.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"vector-search.d.ts","sourceRoot":"","sources":["../src/vector-search.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAOhD,MAAM,MAAM,SAAS,GAAG;IACtB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;CAC7B,CAAC;AAEF,MAAM,MAAM,mBAAmB,GAAG;IAEhC,QAAQ,CAAC,QAAQ,EAAE,YAAY,CAAC;IAIhC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;CACxB,CAAC;AAMF,qBAAa,YAAY;;gBAIX,OAAO,EAAE,mBAAmB;IAQxC,OAAO,CACL,EAAE,EAAE,MAAM,EACV,OAAO,EAAE;QAAE,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;KAAE,GAC3D,SAAS,EAAE;IAoBd,cAAc,CAAC,OAAO,EAAE;QACtB,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC;QACpB,QAAQ,CAAC,EAAE,EAAE,IAAI,CAAC;QAClB,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC;QACnB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;KAC7B,GAAG,cAAc,CAAC,CAAC,MAAM,EAAE,SAAS,EAAE,CAAC,CAAC;IAgCzC,UAAU,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,MAAM,GAAG,MAAM;CAgBzC"}

package/dist/vector-search.js

@@ -0,0 +1,96 @@
+import { UnknownDocumentError } from "./unknown-document-error.js";
+import { UnitVector } from "./vector.js";
+const DAY_MS = 86_400_000;
+// The neighbours of the documents in a vector table. A search compares the
+// document with each document published inside the window, with no index
+// (docs/adr/0001). Each call prepares its statements, so the table can come
+// after the search.
+export class VectorSearch {
+    #database;
+    #table;
+    constructor(options) {
+        this.#database = options.database;
+        this.#table = `"${options.table.replaceAll('"', '""')}"`;
+    }
+    // The k documents most similar to the document, by cosine similarity, from
+    // those published at most withinDays before or after it. The document
+    // itself is not one of them. Equal similarities sort by id.
+    nearest(id, options) {
+        const { vector, publishedAt } = this.#read(id);
+        const window = options.withinDays * DAY_MS;
+        const candidates = this.#database
+            .prepare(`SELECT "id", "vector" FROM ${this.#table} WHERE "published_at" BETWEEN ? AND ? AND "id" != ?`)
+            .all(publishedAt - window, publishedAt + window, id)
+            .map((row) => ({
+            id: String(row.id),
+            vector: UnitVector.fromBlob(blob(row.vector)),
+        }));
+        return ranked(vector, candidates, options.k);
+    }
+    // The k nearest neighbours of each document published from `from` to `to`.
+    // Each entry is what nearest gives for the document with the same k and
+    // withinDays. The call reads and decodes each vector once
+    // (docs/adr/0004). The entries come in the order of the publish dates, then
+    // of the ids.
+    neighbourGraph(options) {
+        const from = options.from.getTime();
+        const to = options.to.getTime();
+        const window = options.withinDays * DAY_MS;
+        const documents = this.#database
+            .prepare(`SELECT "id", "published_at", "vector" FROM ${this.#table} WHERE "published_at" BETWEEN ? AND ? ORDER BY "published_at", "id"`)
+            .all(from - window, to + window)
+            .map((row) => ({
+            id: String(row.id),
+            publishedAt: Number(row.published_at),
+            vector: UnitVector.fromBlob(blob(row.vector)),
+        }));
+        return documents
+            .values()
+            .filter(({ publishedAt }) => publishedAt >= from && publishedAt <= to)
+            .map((document) => [
+            document.id,
+            ranked(document.vector, documents.filter((other) => other.id !== document.id &&
+                Math.abs(other.publishedAt - document.publishedAt) <= window), options.k),
+        ]);
+    }
+    // The cosine similarity of two documents.
+    similarity(a, b) {
+        return this.#read(a).vector.cosine(this.#read(b).vector);
+    }
+    #read(id) {
+        const row = this.#database
+            .prepare(`SELECT "vector", "published_at" FROM ${this.#table} WHERE "id" = ?`)
+            .get(id);
+        if (row === undefined)
+            throw new UnknownDocumentError(id);
+        return {
+            vector: UnitVector.fromBlob(blob(row.vector)),
+            publishedAt: Number(row.published_at),
+        };
+    }
+}
+// The k candidates most similar to the vector. Equal similarities sort by id.
+function ranked(vector, candidates, k) {
+    return candidates
+        .map((candidate) => ({
+        id: candidate.id,
+        similarity: vector.cosine(candidate.vector),
+    }))
+        .toSorted((left, right) => right.similarity - left.similarity || compareIds(left.id, right.id))
+        .slice(0, k);
+}
+// Compares by code unit, not by locale, so the order is the same on every
+// machine.
+function compareIds(left, right) {
+    if (left < right)
+        return -1;
+    if (left > right)
+        return 1;
+    return 0;
+}
+function blob(value) {
+    if (!(value instanceof Uint8Array))
+        throw new TypeError('A stored vector must be a BLOB.');
+    return value;
+}
+//# sourceMappingURL=vector-search.js.map

package/dist/vector-search.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"vector-search.js","sourceRoot":"","sources":["../src/vector-search.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAEzC,MAAM,MAAM,GAAG,UAAU,CAAC;AAgB1B,2EAA2E;AAC3E,yEAAyE;AACzE,4EAA4E;AAC5E,oBAAoB;AACpB,MAAM,OAAO,YAAY;IACd,SAAS,CAAe;IACxB,MAAM,CAAS;IAExB,YAAY,OAA4B;QACtC,IAAI,CAAC,SAAS,GAAG,OAAO,CAAC,QAAQ,CAAC;QAClC,IAAI,CAAC,MAAM,GAAG,IAAI,OAAO,CAAC,KAAK,CAAC,UAAU,CAAC,GAAG,EAAE,IAAI,CAAC,GAAG,CAAC;IAC3D,CAAC;IAED,2EAA2E;IAC3E,sEAAsE;IACtE,4DAA4D;IAC5D,OAAO,CACL,EAAU,EACV,OAA4D;QAE5D,MAAM,EAAE,MAAM,EAAE,WAAW,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;QAC/C,MAAM,MAAM,GAAG,OAAO,CAAC,UAAU,GAAG,MAAM,CAAC;QAC3C,MAAM,UAAU,GAAG,IAAI,CAAC,SAAS;aAC9B,OAAO,CACN,8BAA8B,IAAI,CAAC,MAAM,qDAAqD,CAC/F;aACA,GAAG,CAAC,WAAW,GAAG,MAAM,EAAE,WAAW,GAAG,MAAM,EAAE,EAAE,CAAC;aACnD,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;YACb,EAAE,EAAE,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;YAClB,MAAM,EAAE,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;SAC9C,CAAC,CAAC,CAAC;QACN,OAAO,MAAM,CAAC,MAAM,EAAE,UAAU,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC;IAC/C,CAAC;IAED,2EAA2E;IAC3E,wEAAwE;IACxE,0DAA0D;IAC1D,4EAA4E;IAC5E,cAAc;IACd,cAAc,CAAC,OAKd;QACC,MAAM,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,OAAO,EAAE,CAAC;QACpC,MAAM,EAAE,GAAG,OAAO,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC;QAChC,MAAM,MAAM,GAAG,OAAO,CAAC,UAAU,GAAG,MAAM,CAAC;QAC3C,MAAM,SAAS,GAAG,IAAI,CAAC,SAAS;aAC7B,OAAO,CACN,8CAA8C,IAAI,CAAC,MAAM,qEAAqE,CAC/H;aACA,GAAG,CAAC,IAAI,GAAG,MAAM,EAAE,EAAE,GAAG,MAAM,CAAC;aAC/B,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;YACb,EAAE,EAAE,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;YAClB,WAAW,EAAE,MAAM,CAAC,GAAG,CAAC,YAAY,CAAC;YACrC,MAAM,EAAE,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;SAC9C,CAAC,CAAC,CAAC;QACN,OAAO,SAAS;aACb,MAAM,EAAE;aACR,MAAM,CAAC,CAAC,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC,WAAW,IAAI,IAAI,IAAI,WAAW,IAAI,EAAE,CAAC;aACrE,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC;YACjB,QAAQ,CAAC,EAAE;YACX,MAAM,CACJ,QAAQ,CAAC,MAAM,EACf,SAAS,CAAC,MAAM,CACd,CAAC,KAAK,EAAE,EAAE,CACR,KAAK,CAAC,EAAE,KAAK,QAAQ,CAAC,EAAE;gBACxB,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,WAAW,GAAG,QAAQ,CAAC,WAAW,CAAC,IAAI,MAAM,CAC/D,EACD,OAAO,CAAC,CAAC,CACV;SACF,CAAC,CAAC;IACP,CAAC;IAED,0CAA0C;IAC1C,UAAU,CAAC,CAAS,EAAE,CAAS;QAC7B,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;IAC3D,CAAC;IAED,KAAK,CAAC,EAAU;QACd,MAAM,GAAG,GAAG,IAAI,CAAC,SAAS;aACvB,OAAO,CACN,wCAAwC,IAAI,CAAC,MAAM,iBAAiB,CACrE;aACA,GAAG,CAAC,EAAE,CAAC,CAAC;QACX,IAAI,GAAG,KAAK,SAAS;YAAE,MAAM,IAAI,oBAAoB,CAAC,EAAE,CAAC,CAAC;QAC1D,OAAO;YACL,MAAM,EAAE,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;YAC7C,WAAW,EAAE,MAAM,CAAC,GAAG,CAAC,YAAY,CAAC;SACtC,CAAC;IACJ,CAAC;CACF;AAED,8EAA8E;AAC9E,SAAS,MAAM,CACb,MAAkB,EAClB,UAA2E,EAC3E,CAAS;IAET,OAAO,UAAU;SACd,GAAG,CAAC,CAAC,SAAS,EAAE,EAAE,CAAC,CAAC;QACnB,EAAE,EAAE,SAAS,CAAC,EAAE;QAChB,UAAU,EAAE,MAAM,CAAC,MAAM,CAAC,SAAS,CAAC,MAAM,CAAC;KAC5C,CAAC,CAAC;SACF,QAAQ,CACP,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE,CACd,KAAK,CAAC,UAAU,GAAG,IAAI,CAAC,UAAU,IAAI,UAAU,CAAC,IAAI,CAAC,EAAE,EAAE,KAAK,CAAC,EAAE,CAAC,CACtE;SACA,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;AACjB,CAAC;AAED,0EAA0E;AAC1E,WAAW;AACX,SAAS,UAAU,CAAC,IAAY,EAAE,KAAa;IAC7C,IAAI,IAAI,GAAG,KAAK;QAAE,OAAO,CAAC,CAAC,CAAC;IAC5B,IAAI,IAAI,GAAG,KAAK;QAAE,OAAO,CAAC,CAAC;IAC3B,OAAO,CAAC,CAAC;AACX,CAAC;AAED,SAAS,IAAI,CAAC,KAAc;IAC1B,IAAI,CAAC,CAAC,KAAK,YAAY,UAAU,CAAC;QAChC,MAAM,IAAI,SAAS,CAAC,iCAAiC,CAAC,CAAC;IACzD,OAAO,KAAK,CAAC;AACf,CAAC"}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@zukhruf/embeddings",
-  "version": "0.3.47",
+  "version": "0.3.48",
   "description": "A collection of document embeddings in SQLite. It embeds only new or changed text through an AI SDK embedding model, and finds the nearest documents of a document by exact cosine similarity inside a publish-date window.",
   "license": "MIT",
   "repository": {
@@ -41,11 +41,11 @@
     "access": "public"
   },
   "dependencies": {
-    "@zukhruf/sqlite-drift": "^0.3.47",
-    "@zukhruf/sqlite-transaction": "^0.3.47",
+    "@zukhruf/sqlite-drift": "^0.3.48",
+    "@zukhruf/sqlite-transaction": "^0.3.48",
     "ai": "^7.0.154"
   },
   "devDependencies": {
-    "@zukhruf/testing": "^0.3.47"
+    "@zukhruf/testing": "^0.3.48"
   }
 }