@zukhruf/embeddings 0.3.53 → 0.3.54
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +24 -4
- package/dist/collection.d.ts +7 -0
- package/dist/collection.d.ts.map +1 -1
- package/dist/collection.js +32 -1
- package/dist/collection.js.map +1 -1
- package/dist/dimension-mismatch-error.js +3 -3
- package/dist/dimension-mismatch-error.js.map +1 -1
- package/dist/vector-search.d.ts +4 -0
- package/dist/vector-search.d.ts.map +1 -1
- package/dist/vector-search.js +24 -6
- package/dist/vector-search.js.map +1 -1
- package/dist/vector.d.ts +2 -0
- package/dist/vector.d.ts.map +1 -1
- package/dist/vector.js +10 -0
- package/dist/vector.js.map +1 -1
- package/package.json +5 -5
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @zukhruf/embeddings
|
|
2
2
|
|
|
3
|
-
A program that compares texts by meaning needs a vector for each text, and the model call that makes the vector is slow. This package keeps the vectors of documents in a SQLite database. An upsert calls the model only for new or changed text. A search gives the nearest documents of a document, by exact cosine similarity, from the documents published near it. This kind of package is also known as a vector store.
|
|
3
|
+
A program that compares texts by meaning needs a vector for each text, and the model call that makes the vector is slow. This package keeps the vectors of documents in a SQLite database. An upsert calls the model only for new or changed text. A search gives the nearest documents of a document, by exact cosine similarity, from the documents published near it. A query gives the similarity of a vector of another text, for example a search sentence, to each document of a date range. This kind of package is also known as a vector store.
|
|
4
4
|
|
|
5
5
|
The words in these documents have one meaning each. See the glossary in [CONTEXT.md](./CONTEXT.md).
|
|
6
6
|
|
|
@@ -16,6 +16,7 @@ npm install @zukhruf/embeddings ai
|
|
|
16
16
|
import { DatabaseSync } from 'node:sqlite';
|
|
17
17
|
|
|
18
18
|
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
|
|
19
|
+
import { embed } from 'ai';
|
|
19
20
|
|
|
20
21
|
import { Collection } from '@zukhruf/embeddings';
|
|
21
22
|
|
|
@@ -45,6 +46,20 @@ await collection.upsert([
|
|
|
45
46
|
|
|
46
47
|
collection.nearest('a', { k: 10, withinDays: 7 }); // [{ id: 'b', similarity: … }]
|
|
47
48
|
collection.similarity('a', 'c');
|
|
49
|
+
|
|
50
|
+
// Compare a text that is not a document with the documents of October.
|
|
51
|
+
const { embedding } = await embed({
|
|
52
|
+
model: local.embeddingModel('embeddinggemma'),
|
|
53
|
+
value: 'task: search result | query: ports closed by weather',
|
|
54
|
+
});
|
|
55
|
+
collection.query(embedding, {
|
|
56
|
+
from: new Date('2026-10-01'),
|
|
57
|
+
to: new Date('2026-10-31'),
|
|
58
|
+
}); // [{ id: 'a', similarity: … }, { id: 'b', similarity: … }]
|
|
59
|
+
|
|
60
|
+
collection.fetch(['a']); // Map { 'a' => Float32Array [ … ] }
|
|
61
|
+
collection.delete(['c']);
|
|
62
|
+
collection.count(); // 2
|
|
48
63
|
```
|
|
49
64
|
|
|
50
65
|
### Search a vector table of your own
|
|
@@ -71,13 +86,17 @@ To search a table with other column names, make a view that renames the columns,
|
|
|
71
86
|
- `nearest(id, { k, withinDays })` gives at most `k` neighbours of the document, with the highest similarity first. A neighbour has a publish date at most `withinDays` days before or after the publish date of the document. Equal similarities sort by id.
|
|
72
87
|
- `neighbourGraph({ from, to, k, withinDays })` gives an iterator of `[id, neighbours]` entries, one for each document published from `from` to `to`, in the order of the publish dates and then of the ids. The neighbours of each entry are equal to the result of `nearest(id, { k, withinDays })`. A document outside the range has no entry, but it can be a neighbour of a document in the range. The call reads and decodes each vector one time ([ADR 0004](./docs/adr/0004-a-neighbour-graph-decodes-each-vector-once.md)). Use it when you need the neighbours of many documents.
|
|
73
88
|
- `similarity(a, b)` gives the cosine similarity of two stored documents.
|
|
89
|
+
- `query(vector, { from, to })` gives the similarity of a vector of the caller to each document published from `from` to `to`, as `{ id, similarity }` entries. The highest similarity is first, and equal similarities sort by id. The call makes the vector a unit vector first. The vector must come from the model of the stored vectors ([ADR 0006](./docs/adr/0006-query-fetch-delete-and-count-follow-the-vector-stores.md)).
|
|
90
|
+
- `fetch(ids)` gives a `Map` from each id to a copy of its stored unit vector, as a `Float32Array`, in the order of the ids.
|
|
91
|
+
- `delete(ids)` removes the documents. An id that the collection does not store is not an error, so you can delete the same ids again.
|
|
92
|
+
- `count()` gives the number of stored documents.
|
|
74
93
|
- `new VectorSearch({ database, table })`. The caller opens and closes `database`. `table` is the name of a table or a view in it, with the columns `id`, `vector` and `published_at`. The vector search only reads it. Each call reads the table again, so the table can change between two calls.
|
|
75
|
-
- `VectorSearch` gives `nearest`, `neighbourGraph` and `
|
|
94
|
+
- `VectorSearch` gives `nearest`, `neighbourGraph`, `similarity` and `query`, with the same rules and results as these calls of a collection. A collection gives these calls to a `VectorSearch` over its table `embedding_document`.
|
|
76
95
|
|
|
77
96
|
## Errors
|
|
78
97
|
|
|
79
|
-
- `UnknownDocumentError`: `nearest` or `
|
|
80
|
-
- `DimensionMismatchError`:
|
|
98
|
+
- `UnknownDocumentError`: `nearest`, `similarity` or `fetch` got an id that the vector table does not contain. `fetch` gives no vector then.
|
|
99
|
+
- `DimensionMismatchError`: a vector has a dimension that is not the dimension of the stored vectors. In an upsert, the model gave the vector, and the upsert stores none of its documents. To change to a model with another dimension, upsert all documents in one call. In a query, the caller gave the vector. Give a vector of the model of the stored vectors.
|
|
81
100
|
|
|
82
101
|
## Limits
|
|
83
102
|
|
|
@@ -95,4 +114,5 @@ To search a table with other column names, make a view that renames the columns,
|
|
|
95
114
|
- [ADR 0003: All vectors of a collection have one dimension](./docs/adr/0003-all-vectors-of-a-collection-have-one-dimension.md)
|
|
96
115
|
- [ADR 0004: A neighbour graph decodes each vector once](./docs/adr/0004-a-neighbour-graph-decodes-each-vector-once.md)
|
|
97
116
|
- [ADR 0005: A vector search reads a vector table of unit vectors](./docs/adr/0005-a-vector-search-reads-a-vector-table-of-unit-vectors.md)
|
|
117
|
+
- [ADR 0006: Query, fetch, delete and count follow the vector stores](./docs/adr/0006-query-fetch-delete-and-count-follow-the-vector-stores.md)
|
|
98
118
|
- [Code copied from @deepagents/retrieval](./docs/copied-from-deepagents-retrieval.md)
|
package/dist/collection.d.ts
CHANGED
|
@@ -25,5 +25,12 @@ export declare class Collection {
|
|
|
25
25
|
readonly withinDays: number;
|
|
26
26
|
}): IteratorObject<[string, Neighbour[]]>;
|
|
27
27
|
similarity(a: string, b: string): number;
|
|
28
|
+
query(vector: readonly number[], options: {
|
|
29
|
+
readonly from: Date;
|
|
30
|
+
readonly to: Date;
|
|
31
|
+
}): Neighbour[];
|
|
32
|
+
fetch(ids: readonly string[]): Map<string, Float32Array>;
|
|
33
|
+
delete(ids: readonly string[]): void;
|
|
34
|
+
count(): number;
|
|
28
35
|
}
|
|
29
36
|
//# sourceMappingURL=collection.d.ts.map
|
package/dist/collection.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"collection.d.ts","sourceRoot":"","sources":["../src/collection.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAEhD,OAAO,EAAE,KAAK,cAAc,EAAa,MAAM,IAAI,CAAC;
|
|
1
|
+
{"version":3,"file":"collection.d.ts","sourceRoot":"","sources":["../src/collection.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAEhD,OAAO,EAAE,KAAK,cAAc,EAAa,MAAM,IAAI,CAAC;AAOpD,OAAO,EAAE,KAAK,SAAS,EAAgB,MAAM,oBAAoB,CAAC;AAGlE,MAAM,MAAM,QAAQ,GAAG;IACrB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IAEpB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,WAAW,EAAE,IAAI,CAAC;CAC5B,CAAC;AAEF,MAAM,MAAM,iBAAiB,GAAG;IAE9B,QAAQ,CAAC,QAAQ,EAAE,YAAY,CAAC;IAGhC,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAC,cAAc,EAAE,MAAM,CAAC,CAAC;CACjD,CAAC;AAIF,qBAAa,UAAU;;gBAKT,OAAO,EAAE,iBAAiB;IAYhC,MAAM,CAAC,SAAS,EAAE,SAAS,QAAQ,EAAE,GAAG,OAAO,CAAC,IAAI,CAAC;IAoD3D,OAAO,CACL,EAAE,EAAE,MAAM,EACV,OAAO,EAAE;QAAE,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;KAAE,GAC3D,SAAS,EAAE;IAKd,cAAc,CAAC,OAAO,EAAE;QACtB,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC;QACpB,QAAQ,CAAC,EAAE,EAAE,IAAI,CAAC;QAClB,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC;QACnB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;KAC7B,GAAG,cAAc,CAAC,CAAC,MAAM,EAAE,SAAS,EAAE,CAAC,CAAC;IAKzC,UAAU,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,MAAM,GAAG,MAAM;IAMxC,KAAK,CACH,MAAM,EAAE,SAAS,MAAM,EAAE,EACzB,OAAO,EAAE;QAAE,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC;QAAC,QAAQ,CAAC,EAAE,EAAE,IAAI,CAAA;KAAE,GAClD,SAAS,EAAE;IAMd,KAAK,CAAC,GAAG,EAAE,SAAS,MAAM,EAAE,GAAG,GAAG,CAAC,MAAM,EAAE,YAAY,CAAC;IAexD,MAAM,CAAC,GAAG,EAAE,SAAS,MAAM,EAAE,GAAG,IAAI;IASpC,KAAK,IAAI,MAAM;CA2ChB"}
|
package/dist/collection.js
CHANGED
|
@@ -3,8 +3,9 @@ import { embedMany } from 'ai';
|
|
|
3
3
|
import { transaction } from '@zukhruf/sqlite-transaction';
|
|
4
4
|
import { DimensionMismatchError } from "./dimension-mismatch-error.js";
|
|
5
5
|
import { collectionSchema } from "./schema.js";
|
|
6
|
+
import { UnknownDocumentError } from "./unknown-document-error.js";
|
|
6
7
|
import { VectorSearch } from "./vector-search.js";
|
|
7
|
-
import { UnitVector } from "./vector.js";
|
|
8
|
+
import { UnitVector, blob } from "./vector.js";
|
|
8
9
|
// The embeddings of documents, one vector for each document, in a SQLite
|
|
9
10
|
// database. The collection searches its table through a VectorSearch.
|
|
10
11
|
export class Collection {
|
|
@@ -57,6 +58,36 @@ export class Collection {
|
|
|
57
58
|
similarity(a, b) {
|
|
58
59
|
return this.#search.similarity(a, b);
|
|
59
60
|
}
|
|
61
|
+
// The similarity of a vector of the caller to each document published from
|
|
62
|
+
// `from` to `to`, the highest first (docs/adr/0006).
|
|
63
|
+
query(vector, options) {
|
|
64
|
+
return this.#search.query(vector, options);
|
|
65
|
+
}
|
|
66
|
+
// The stored unit vector of each id, in the order of the ids. One unknown id
|
|
67
|
+
// fails the call (docs/adr/0006).
|
|
68
|
+
fetch(ids) {
|
|
69
|
+
const read = this.#database.prepare('SELECT "vector" FROM "embedding_document" WHERE "id" = ?');
|
|
70
|
+
return new Map(ids.map((id) => {
|
|
71
|
+
const row = read.get(id);
|
|
72
|
+
if (row === undefined)
|
|
73
|
+
throw new UnknownDocumentError(id);
|
|
74
|
+
return [id, UnitVector.fromBlob(blob(row.vector)).toFloat32Array()];
|
|
75
|
+
}));
|
|
76
|
+
}
|
|
77
|
+
// Removes the documents. An id that the collection does not store is not an
|
|
78
|
+
// error, so a second delete of the same ids does nothing (docs/adr/0006).
|
|
79
|
+
delete(ids) {
|
|
80
|
+
this.#database
|
|
81
|
+
.prepare('DELETE FROM "embedding_document" WHERE "id" IN (SELECT "value" FROM json_each(?))')
|
|
82
|
+
.run(JSON.stringify(ids));
|
|
83
|
+
}
|
|
84
|
+
// The number of stored documents.
|
|
85
|
+
count() {
|
|
86
|
+
const row = this.#database
|
|
87
|
+
.prepare('SELECT count(*) AS "count" FROM "embedding_document"')
|
|
88
|
+
.get();
|
|
89
|
+
return Number(row?.count);
|
|
90
|
+
}
|
|
60
91
|
// No texts make no call to the model.
|
|
61
92
|
async #embed(texts) {
|
|
62
93
|
if (texts.length === 0)
|
package/dist/collection.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"collection.js","sourceRoot":"","sources":["../src/collection.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAGzC,OAAO,EAAuB,SAAS,EAAE,MAAM,IAAI,CAAC;AAEpD,OAAO,EAAE,WAAW,EAAE,MAAM,6BAA6B,CAAC;AAE1D,OAAO,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AACvE,OAAO,EAAE,gBAAgB,EAAE,MAAM,aAAa,CAAC;AAC/C,OAAO,EAAkB,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAClE,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;
|
|
1
|
+
{"version":3,"file":"collection.js","sourceRoot":"","sources":["../src/collection.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAGzC,OAAO,EAAuB,SAAS,EAAE,MAAM,IAAI,CAAC;AAEpD,OAAO,EAAE,WAAW,EAAE,MAAM,6BAA6B,CAAC;AAE1D,OAAO,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AACvE,OAAO,EAAE,gBAAgB,EAAE,MAAM,aAAa,CAAC;AAC/C,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,EAAkB,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAClE,OAAO,EAAE,UAAU,EAAE,IAAI,EAAE,MAAM,aAAa,CAAC;AAiB/C,yEAAyE;AACzE,sEAAsE;AACtE,MAAM,OAAO,UAAU;IACZ,SAAS,CAAe;IACxB,MAAM,CAAkC;IACxC,OAAO,CAAe;IAE/B,YAAY,OAA0B;QACpC,IAAI,CAAC,SAAS,GAAG,OAAO,CAAC,QAAQ,CAAC;QAClC,IAAI,CAAC,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC;QAC5B,gBAAgB,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;QACvC,IAAI,CAAC,OAAO,GAAG,IAAI,YAAY,CAAC;YAC9B,QAAQ,EAAE,IAAI,CAAC,SAAS;YACxB,KAAK,EAAE,oBAAoB;SAC5B,CAAC,CAAC;IACL,CAAC;IAED,4EAA4E;IAC5E,gDAAgD;IAChD,KAAK,CAAC,MAAM,CAAC,SAA8B;QACzC,MAAM,MAAM,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACnC,wDAAwD,CACzD,CAAC;QACF,MAAM,MAAM,GAAG,SAAS,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC;YAC1C,QAAQ;YACR,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC;SAChC,CAAC,CAAC,CAAC;QACJ,MAAM,EAAE,OAAO,GAAG,EAAE,EAAE,SAAS,GAAG,EAAE,EAAE,GAAG,MAAM,CAAC,OAAO,CACrD,MAAM,EACN,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE,EAAE,CACrB,MAAM,CAAC,GAAG,CAAC,QAAQ,CAAC,EAAE,CAAC,EAAE,IAAI,KAAK,IAAI,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,SAAS,CACnE,CAAC;QACF,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,MAAM,CAC/B,OAAO,CAAC,GAAG,CAAC,CAAC,EAAE,QAAQ,EAAE,EAAE,EAAE,CAAC,QAAQ,CAAC,IAAI,CAAC,CAC7C,CAAC;QAEF,WAAW,CACT,IAAI,CAAC,SAAS,EACd,GAAG,EAAE;YACH,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CAClC;4IACkI,CACnI,CAAC;YACF,KAAK,MAAM,CAAC,KAAK,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC,IAAI,OAAO,CAAC,OAAO,EAAE,EAAE,CAAC;gBAC5D,MAAM,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC;gBAC9B,IAAI,MAAM,KAAK,SAAS;oBACtB,MAAM,IAAI,KAAK,CACb,kBAAkB,OAAO,CAAC,MAAM,mBAAmB,OAAO,CAAC,MAAM,SAAS,CAC3E,CAAC;gBACJ,KAAK,CAAC,GAAG,CACP,QAAQ,CAAC,EAAE,EACX,IAAI,EACJ,QAAQ,CAAC,WAAW,CAAC,OAAO,EAAE,EAC9B,MAAM,CAAC,MAAM,EAAE,CAChB,CAAC;YACJ,CAAC;YACD,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACjC,mEAAmE,CACpE,CAAC;YACF,KAAK,MAAM,EAAE,QAAQ,EAAE,IAAI,SAAS;gBAClC,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,WAAW,CAAC,OAAO,EAAE,EAAE,QAAQ,CAAC,EAAE,CAAC,CAAC;YACxD,IAAI,CAAC,eAAe,CAClB,OAAO,CAAC,GAAG,CAAC,CAAC,EAAE,QAAQ,EAAE,EAAE,EAAE,CAAC,QAAQ,CAAC,EAAE,CAAC,EAC1C,OAAO,CACR,CAAC;QACJ,CAAC,EACD,EAAE,IAAI,EAAE,WAAW,EAAE,CACtB,CAAC;IACJ,CAAC;IAED,gDAAgD;IAChD,OAAO,CACL,EAAU,EACV,OAA4D;QAE5D,OAAO,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,EAAE,OAAO,CAAC,CAAC;IAC3C,CAAC;IAED,iEAAiE;IACjE,cAAc,CAAC,OAKd;QACC,OAAO,IAAI,CAAC,OAAO,CAAC,cAAc,CAAC,OAAO,CAAC,CAAC;IAC9C,CAAC;IAED,iDAAiD;IACjD,UAAU,CAAC,CAAS,EAAE,CAAS;QAC7B,OAAO,IAAI,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IACvC,CAAC;IAED,2EAA2E;IAC3E,qDAAqD;IACrD,KAAK,CACH,MAAyB,EACzB,OAAmD;QAEnD,OAAO,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IAC7C,CAAC;IAED,6EAA6E;IAC7E,kCAAkC;IAClC,KAAK,CAAC,GAAsB;QAC1B,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACjC,0DAA0D,CAC3D,CAAC;QACF,OAAO,IAAI,GAAG,CACZ,GAAG,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE;YACb,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;YACzB,IAAI,GAAG,KAAK,SAAS;gBAAE,MAAM,IAAI,oBAAoB,CAAC,EAAE,CAAC,CAAC;YAC1D,OAAO,CAAC,EAAE,EAAE,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,cAAc,EAAE,CAAC,CAAC;QACtE,CAAC,CAAC,CACH,CAAC;IACJ,CAAC;IAED,4EAA4E;IAC5E,0EAA0E;IAC1E,MAAM,CAAC,GAAsB;QAC3B,IAAI,CAAC,SAAS;aACX,OAAO,CACN,mFAAmF,CACpF;aACA,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC;IAC9B,CAAC;IAED,kCAAkC;IAClC,KAAK;QACH,MAAM,GAAG,GAAG,IAAI,CAAC,SAAS;aACvB,OAAO,CAAC,sDAAsD,CAAC;aAC/D,GAAG,EAAE,CAAC;QACT,OAAO,MAAM,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC;IAC5B,CAAC;IAED,sCAAsC;IACtC,KAAK,CAAC,MAAM,CAAC,KAAe;QAC1B,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,EAAE,CAAC;QAClC,MAAM,EAAE,UAAU,EAAE,GAAG,MAAM,SAAS,CAAC;YACrC,KAAK,EAAE,IAAI,CAAC,MAAM;YAClB,MAAM,EAAE,KAAK;SACd,CAAC,CAAC;QACH,OAAO,UAAU,CAAC,GAAG,CAAC,CAAC,SAAS,EAAE,EAAE,CAAC,UAAU,CAAC,SAAS,CAAC,SAAS,CAAC,CAAC,CAAC;IACxE,CAAC;IAED,uEAAuE;IACvE,KAAK,CAAC,IAAY;QAChB,OAAO,UAAU,CAAC,QAAQ,CAAC;aACxB,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,QAAQ,EAAE,IAAI,CAAC,MAAM,CAAC,OAAO,EAAE,IAAI,CAAC,CAAC,CAAC;aACzE,MAAM,CAAC,KAAK,CAAC,CAAC;IACnB,CAAC;IAED,2EAA2E;IAC3E,2EAA2E;IAC3E,iBAAiB;IACjB,eAAe,CAAC,QAA2B,EAAE,OAA8B;QACzE,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,CAAC;QACxB,IAAI,KAAK,KAAK,SAAS;YAAE,OAAO;QAChC,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS;aACxB,OAAO,CACN,2HAA2H,CAC5H;aACA,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC,CAAC;QACjC,MAAM,QAAQ,GACZ,IAAI,KAAK,SAAS;YAChB,CAAC,CAAC,KAAK,CAAC,SAAS;YACjB,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,YAAY,CAAC,iBAAiB,CAAC;QAC1D,KAAK,MAAM,MAAM,IAAI,OAAO;YAC1B,IAAI,MAAM,CAAC,SAAS,KAAK,QAAQ;gBAC/B,MAAM,IAAI,sBAAsB,CAAC,QAAQ,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;IACnE,CAAC;CACF"}
|
|
@@ -1,10 +1,10 @@
|
|
|
1
|
-
//
|
|
2
|
-
//
|
|
1
|
+
// A vector whose dimension is not the dimension of the stored vectors: a
|
|
2
|
+
// vector of the model in an upsert, or a vector of the caller in a query.
|
|
3
3
|
export class DimensionMismatchError extends Error {
|
|
4
4
|
expected;
|
|
5
5
|
actual;
|
|
6
6
|
constructor(expected, actual) {
|
|
7
|
-
super(`The
|
|
7
|
+
super(`The stored vectors have dimension ${expected}, and the new vector has dimension ${actual}. To change the model, upsert every document again with the new model, in one call. To query, give a vector of the model of the stored vectors.`);
|
|
8
8
|
this.name = 'DimensionMismatchError';
|
|
9
9
|
this.expected = expected;
|
|
10
10
|
this.actual = actual;
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"dimension-mismatch-error.js","sourceRoot":"","sources":["../src/dimension-mismatch-error.ts"],"names":[],"mappings":"AAAA,
|
|
1
|
+
{"version":3,"file":"dimension-mismatch-error.js","sourceRoot":"","sources":["../src/dimension-mismatch-error.ts"],"names":[],"mappings":"AAAA,yEAAyE;AACzE,0EAA0E;AAC1E,MAAM,OAAO,sBAAuB,SAAQ,KAAK;IACtC,QAAQ,CAAS;IACjB,MAAM,CAAS;IAExB,YAAY,QAAgB,EAAE,MAAc;QAC1C,KAAK,CACH,qCAAqC,QAAQ,sCAAsC,MAAM,iJAAiJ,CAC3O,CAAC;QACF,IAAI,CAAC,IAAI,GAAG,wBAAwB,CAAC;QACrC,IAAI,CAAC,QAAQ,GAAG,QAAQ,CAAC;QACzB,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;IACvB,CAAC;CACF"}
|
package/dist/vector-search.d.ts
CHANGED
|
@@ -20,6 +20,10 @@ export declare class VectorSearch {
|
|
|
20
20
|
readonly k: number;
|
|
21
21
|
readonly withinDays: number;
|
|
22
22
|
}): IteratorObject<[string, Neighbour[]]>;
|
|
23
|
+
query(vector: readonly number[], options: {
|
|
24
|
+
readonly from: Date;
|
|
25
|
+
readonly to: Date;
|
|
26
|
+
}): Neighbour[];
|
|
23
27
|
similarity(a: string, b: string): number;
|
|
24
28
|
}
|
|
25
29
|
//# sourceMappingURL=vector-search.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"vector-search.d.ts","sourceRoot":"","sources":["../src/vector-search.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;
|
|
1
|
+
{"version":3,"file":"vector-search.d.ts","sourceRoot":"","sources":["../src/vector-search.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAQhD,MAAM,MAAM,SAAS,GAAG;IACtB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;CAC7B,CAAC;AAEF,MAAM,MAAM,mBAAmB,GAAG;IAEhC,QAAQ,CAAC,QAAQ,EAAE,YAAY,CAAC;IAIhC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;CACxB,CAAC;AAMF,qBAAa,YAAY;;gBAIX,OAAO,EAAE,mBAAmB;IAQxC,OAAO,CACL,EAAE,EAAE,MAAM,EACV,OAAO,EAAE;QAAE,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;KAAE,GAC3D,SAAS,EAAE;IAoBd,cAAc,CAAC,OAAO,EAAE;QACtB,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC;QACpB,QAAQ,CAAC,EAAE,EAAE,IAAI,CAAC;QAClB,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC;QACnB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;KAC7B,GAAG,cAAc,CAAC,CAAC,MAAM,EAAE,SAAS,EAAE,CAAC,CAAC;IAkCzC,KAAK,CACH,MAAM,EAAE,SAAS,MAAM,EAAE,EACzB,OAAO,EAAE;QAAE,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC;QAAC,QAAQ,CAAC,EAAE,EAAE,IAAI,CAAA;KAAE,GAClD,SAAS,EAAE;IAsBd,UAAU,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,MAAM,GAAG,MAAM;CAgBzC"}
|
package/dist/vector-search.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
|
+
import { DimensionMismatchError } from "./dimension-mismatch-error.js";
|
|
1
2
|
import { UnknownDocumentError } from "./unknown-document-error.js";
|
|
2
|
-
import { UnitVector } from "./vector.js";
|
|
3
|
+
import { UnitVector, blob } from "./vector.js";
|
|
3
4
|
const DAY_MS = 86_400_000;
|
|
4
5
|
// The neighbours of the documents in a vector table. A search compares the
|
|
5
6
|
// document with each document published inside the window, with no index
|
|
@@ -53,6 +54,28 @@ export class VectorSearch {
|
|
|
53
54
|
Math.abs(other.publishedAt - document.publishedAt) <= window), options.k),
|
|
54
55
|
]);
|
|
55
56
|
}
|
|
57
|
+
// The similarity of a vector from outside the table to each document
|
|
58
|
+
// published from `from` to `to`, the highest first. Equal similarities sort
|
|
59
|
+
// by id (docs/adr/0006).
|
|
60
|
+
query(vector, options) {
|
|
61
|
+
const query = UnitVector.normalize(vector);
|
|
62
|
+
const stored = this.#database
|
|
63
|
+
.prepare(`SELECT length("vector") AS "bytes" FROM ${this.#table} LIMIT 1`)
|
|
64
|
+
.get();
|
|
65
|
+
if (stored === undefined)
|
|
66
|
+
return [];
|
|
67
|
+
const dimension = Number(stored.bytes) / Float32Array.BYTES_PER_ELEMENT;
|
|
68
|
+
if (dimension !== query.dimension)
|
|
69
|
+
throw new DimensionMismatchError(dimension, query.dimension);
|
|
70
|
+
const documents = this.#database
|
|
71
|
+
.prepare(`SELECT "id", "vector" FROM ${this.#table} WHERE "published_at" BETWEEN ? AND ?`)
|
|
72
|
+
.all(options.from.getTime(), options.to.getTime())
|
|
73
|
+
.map((row) => ({
|
|
74
|
+
id: String(row.id),
|
|
75
|
+
vector: UnitVector.fromBlob(blob(row.vector)),
|
|
76
|
+
}));
|
|
77
|
+
return ranked(query, documents, documents.length);
|
|
78
|
+
}
|
|
56
79
|
// The cosine similarity of two documents.
|
|
57
80
|
similarity(a, b) {
|
|
58
81
|
return this.#read(a).vector.cosine(this.#read(b).vector);
|
|
@@ -88,9 +111,4 @@ function compareIds(left, right) {
|
|
|
88
111
|
return 1;
|
|
89
112
|
return 0;
|
|
90
113
|
}
|
|
91
|
-
function blob(value) {
|
|
92
|
-
if (!(value instanceof Uint8Array))
|
|
93
|
-
throw new TypeError('A stored vector must be a BLOB.');
|
|
94
|
-
return value;
|
|
95
|
-
}
|
|
96
114
|
//# sourceMappingURL=vector-search.js.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"vector-search.js","sourceRoot":"","sources":["../src/vector-search.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;
|
|
1
|
+
{"version":3,"file":"vector-search.js","sourceRoot":"","sources":["../src/vector-search.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,sBAAsB,EAAE,MAAM,+BAA+B,CAAC;AACvE,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,EAAE,UAAU,EAAE,IAAI,EAAE,MAAM,aAAa,CAAC;AAE/C,MAAM,MAAM,GAAG,UAAU,CAAC;AAgB1B,2EAA2E;AAC3E,yEAAyE;AACzE,4EAA4E;AAC5E,oBAAoB;AACpB,MAAM,OAAO,YAAY;IACd,SAAS,CAAe;IACxB,MAAM,CAAS;IAExB,YAAY,OAA4B;QACtC,IAAI,CAAC,SAAS,GAAG,OAAO,CAAC,QAAQ,CAAC;QAClC,IAAI,CAAC,MAAM,GAAG,IAAI,OAAO,CAAC,KAAK,CAAC,UAAU,CAAC,GAAG,EAAE,IAAI,CAAC,GAAG,CAAC;IAC3D,CAAC;IAED,2EAA2E;IAC3E,sEAAsE;IACtE,4DAA4D;IAC5D,OAAO,CACL,EAAU,EACV,OAA4D;QAE5D,MAAM,EAAE,MAAM,EAAE,WAAW,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;QAC/C,MAAM,MAAM,GAAG,OAAO,CAAC,UAAU,GAAG,MAAM,CAAC;QAC3C,MAAM,UAAU,GAAG,IAAI,CAAC,SAAS;aAC9B,OAAO,CACN,8BAA8B,IAAI,CAAC,MAAM,qDAAqD,CAC/F;aACA,GAAG,CAAC,WAAW,GAAG,MAAM,EAAE,WAAW,GAAG,MAAM,EAAE,EAAE,CAAC;aACnD,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;YACb,EAAE,EAAE,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;YAClB,MAAM,EAAE,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;SAC9C,CAAC,CAAC,CAAC;QACN,OAAO,MAAM,CAAC,MAAM,EAAE,UAAU,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC;IAC/C,CAAC;IAED,2EAA2E;IAC3E,wEAAwE;IACxE,0DAA0D;IAC1D,4EAA4E;IAC5E,cAAc;IACd,cAAc,CAAC,OAKd;QACC,MAAM,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,OAAO,EAAE,CAAC;QACpC,MAAM,EAAE,GAAG,OAAO,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC;QAChC,MAAM,MAAM,GAAG,OAAO,CAAC,UAAU,GAAG,MAAM,CAAC;QAC3C,MAAM,SAAS,GAAG,IAAI,CAAC,SAAS;aAC7B,OAAO,CACN,8CAA8C,IAAI,CAAC,MAAM,qEAAqE,CAC/H;aACA,GAAG,CAAC,IAAI,GAAG,MAAM,EAAE,EAAE,GAAG,MAAM,CAAC;aAC/B,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;YACb,EAAE,EAAE,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;YAClB,WAAW,EAAE,MAAM,CAAC,GAAG,CAAC,YAAY,CAAC;YACrC,MAAM,EAAE,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;SAC9C,CAAC,CAAC,CAAC;QACN,OAAO,SAAS;aACb,MAAM,EAAE;aACR,MAAM,CAAC,CAAC,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC,WAAW,IAAI,IAAI,IAAI,WAAW,IAAI,EAAE,CAAC;aACrE,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC;YACjB,QAAQ,CAAC,EAAE;YACX,MAAM,CACJ,QAAQ,CAAC,MAAM,EACf,SAAS,CAAC,MAAM,CACd,CAAC,KAAK,EAAE,EAAE,CACR,KAAK,CAAC,EAAE,KAAK,QAAQ,CAAC,EAAE;gBACxB,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,WAAW,GAAG,QAAQ,CAAC,WAAW,CAAC,IAAI,MAAM,CAC/D,EACD,OAAO,CAAC,CAAC,CACV;SACF,CAAC,CAAC;IACP,CAAC;IAED,qEAAqE;IACrE,4EAA4E;IAC5E,yBAAyB;IACzB,KAAK,CACH,MAAyB,EACzB,OAAmD;QAEnD,MAAM,KAAK,GAAG,UAAU,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC;QAC3C,MAAM,MAAM,GAAG,IAAI,CAAC,SAAS;aAC1B,OAAO,CAAC,2CAA2C,IAAI,CAAC,MAAM,UAAU,CAAC;aACzE,GAAG,EAAE,CAAC;QACT,IAAI,MAAM,KAAK,SAAS;YAAE,OAAO,EAAE,CAAC;QACpC,MAAM,SAAS,GAAG,MAAM,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,YAAY,CAAC,iBAAiB,CAAC;QACxE,IAAI,SAAS,KAAK,KAAK,CAAC,SAAS;YAC/B,MAAM,IAAI,sBAAsB,CAAC,SAAS,EAAE,KAAK,CAAC,SAAS,CAAC,CAAC;QAC/D,MAAM,SAAS,GAAG,IAAI,CAAC,SAAS;aAC7B,OAAO,CACN,8BAA8B,IAAI,CAAC,MAAM,uCAAuC,CACjF;aACA,GAAG,CAAC,OAAO,CAAC,IAAI,CAAC,OAAO,EAAE,EAAE,OAAO,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC;aACjD,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;YACb,EAAE,EAAE,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;YAClB,MAAM,EAAE,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;SAC9C,CAAC,CAAC,CAAC;QACN,OAAO,MAAM,CAAC,KAAK,EAAE,SAAS,EAAE,SAAS,CAAC,MAAM,CAAC,CAAC;IACpD,CAAC;IAED,0CAA0C;IAC1C,UAAU,CAAC,CAAS,EAAE,CAAS;QAC7B,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;IAC3D,CAAC;IAED,KAAK,CAAC,EAAU;QACd,MAAM,GAAG,GAAG,IAAI,CAAC,SAAS;aACvB,OAAO,CACN,wCAAwC,IAAI,CAAC,MAAM,iBAAiB,CACrE;aACA,GAAG,CAAC,EAAE,CAAC,CAAC;QACX,IAAI,GAAG,KAAK,SAAS;YAAE,MAAM,IAAI,oBAAoB,CAAC,EAAE,CAAC,CAAC;QAC1D,OAAO;YACL,MAAM,EAAE,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;YAC7C,WAAW,EAAE,MAAM,CAAC,GAAG,CAAC,YAAY,CAAC;SACtC,CAAC;IACJ,CAAC;CACF;AAED,8EAA8E;AAC9E,SAAS,MAAM,CACb,MAAkB,EAClB,UAA2E,EAC3E,CAAS;IAET,OAAO,UAAU;SACd,GAAG,CAAC,CAAC,SAAS,EAAE,EAAE,CAAC,CAAC;QACnB,EAAE,EAAE,SAAS,CAAC,EAAE;QAChB,UAAU,EAAE,MAAM,CAAC,MAAM,CAAC,SAAS,CAAC,MAAM,CAAC;KAC5C,CAAC,CAAC;SACF,QAAQ,CACP,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE,CACd,KAAK,CAAC,UAAU,GAAG,IAAI,CAAC,UAAU,IAAI,UAAU,CAAC,IAAI,CAAC,EAAE,EAAE,KAAK,CAAC,EAAE,CAAC,CACtE;SACA,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;AACjB,CAAC;AAED,0EAA0E;AAC1E,WAAW;AACX,SAAS,UAAU,CAAC,IAAY,EAAE,KAAa;IAC7C,IAAI,IAAI,GAAG,KAAK;QAAE,OAAO,CAAC,CAAC,CAAC;IAC5B,IAAI,IAAI,GAAG,KAAK;QAAE,OAAO,CAAC,CAAC;IAC3B,OAAO,CAAC,CAAC;AACX,CAAC"}
|
package/dist/vector.d.ts
CHANGED
|
@@ -4,7 +4,9 @@ export declare class UnitVector {
|
|
|
4
4
|
static normalize(values: readonly number[]): UnitVector;
|
|
5
5
|
static fromBlob(blob: Uint8Array): UnitVector;
|
|
6
6
|
get dimension(): number;
|
|
7
|
+
toFloat32Array(): Float32Array;
|
|
7
8
|
toBlob(): Uint8Array;
|
|
8
9
|
cosine(other: UnitVector): number;
|
|
9
10
|
}
|
|
11
|
+
export declare function blob(value: unknown): Uint8Array;
|
|
10
12
|
//# sourceMappingURL=vector.d.ts.map
|
package/dist/vector.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"vector.d.ts","sourceRoot":"","sources":["../src/vector.ts"],"names":[],"mappings":"AAEA,qBAAa,UAAU;;IAGrB,OAAO;IAKP,MAAM,CAAC,SAAS,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,GAAG,UAAU;IAavD,MAAM,CAAC,QAAQ,CAAC,IAAI,EAAE,UAAU,GAAG,UAAU;IAI7C,IAAI,SAAS,IAAI,MAAM,CAEtB;
|
|
1
|
+
{"version":3,"file":"vector.d.ts","sourceRoot":"","sources":["../src/vector.ts"],"names":[],"mappings":"AAEA,qBAAa,UAAU;;IAGrB,OAAO;IAKP,MAAM,CAAC,SAAS,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,GAAG,UAAU;IAavD,MAAM,CAAC,QAAQ,CAAC,IAAI,EAAE,UAAU,GAAG,UAAU;IAI7C,IAAI,SAAS,IAAI,MAAM,CAEtB;IAGD,cAAc,IAAI,YAAY;IAI9B,MAAM,IAAI,UAAU;IAWpB,MAAM,CAAC,KAAK,EAAE,UAAU,GAAG,MAAM;CAQlC;AAGD,wBAAgB,IAAI,CAAC,KAAK,EAAE,OAAO,GAAG,UAAU,CAI/C"}
|
package/dist/vector.js
CHANGED
|
@@ -20,6 +20,10 @@ export class UnitVector {
|
|
|
20
20
|
get dimension() {
|
|
21
21
|
return this.#values.length;
|
|
22
22
|
}
|
|
23
|
+
// A copy of the values, so the caller cannot change the vector.
|
|
24
|
+
toFloat32Array() {
|
|
25
|
+
return this.#values.slice();
|
|
26
|
+
}
|
|
23
27
|
toBlob() {
|
|
24
28
|
return new Uint8Array(this.#values.buffer, this.#values.byteOffset, this.#values.byteLength);
|
|
25
29
|
}
|
|
@@ -35,4 +39,10 @@ export class UnitVector {
|
|
|
35
39
|
return sum;
|
|
36
40
|
}
|
|
37
41
|
}
|
|
42
|
+
// The value of a vector column, which SQLite gives as a Uint8Array.
|
|
43
|
+
export function blob(value) {
|
|
44
|
+
if (!(value instanceof Uint8Array))
|
|
45
|
+
throw new TypeError('A stored vector must be a BLOB.');
|
|
46
|
+
return value;
|
|
47
|
+
}
|
|
38
48
|
//# sourceMappingURL=vector.js.map
|
package/dist/vector.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"vector.js","sourceRoot":"","sources":["../src/vector.ts"],"names":[],"mappings":"AAAA,8EAA8E;AAC9E,qBAAqB;AACrB,MAAM,OAAO,UAAU;IACZ,OAAO,CAAe;IAE/B,YAAoB,MAAoB;QACtC,IAAI,CAAC,OAAO,GAAG,MAAM,CAAC;IACxB,CAAC;IAED,kDAAkD;IAClD,MAAM,CAAC,SAAS,CAAC,MAAyB;QACxC,MAAM,MAAM,GAAG,IAAI,CAAC,IAAI,CACtB,MAAM,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,KAAK,EAAE,EAAE,CAAC,GAAG,GAAG,KAAK,GAAG,KAAK,EAAE,CAAC,CAAC,CACtD,CAAC;QACF,IAAI,MAAM,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC;YAC1C,MAAM,IAAI,UAAU,CAClB,0EAA0E,MAAM,GAAG,CACpF,CAAC;QACJ,OAAO,IAAI,UAAU,CAAC,YAAY,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,GAAG,MAAM,CAAC,CAAC,CAAC;IAC9E,CAAC;IAED,0EAA0E;IAC1E,kCAAkC;IAClC,MAAM,CAAC,QAAQ,CAAC,IAAgB;QAC9B,OAAO,IAAI,UAAU,CAAC,IAAI,YAAY,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;IACxE,CAAC;IAED,IAAI,SAAS;QACX,OAAO,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC;IAC7B,CAAC;IAED,MAAM;QACJ,OAAO,IAAI,UAAU,CACnB,IAAI,CAAC,OAAO,CAAC,MAAM,EACnB,IAAI,CAAC,OAAO,CAAC,UAAU,EACvB,IAAI,CAAC,OAAO,CAAC,UAAU,CACxB,CAAC;IACJ,CAAC;IAED,qEAAqE;IACrE,2EAA2E;IAC3E,gBAAgB;IAChB,MAAM,CAAC,KAAiB;QACtB,MAAM,MAAM,GAAG,IAAI,CAAC,OAAO,CAAC;QAC5B,MAAM,MAAM,GAAG,KAAK,CAAC,OAAO,CAAC;QAC7B,IAAI,GAAG,GAAG,CAAC,CAAC;QACZ,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,KAAK,GAAG,MAAM,CAAC,MAAM,EAAE,KAAK,EAAE;YAChD,GAAG,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC;QACrD,OAAO,GAAG,CAAC;IACb,CAAC;CACF"}
|
|
1
|
+
{"version":3,"file":"vector.js","sourceRoot":"","sources":["../src/vector.ts"],"names":[],"mappings":"AAAA,8EAA8E;AAC9E,qBAAqB;AACrB,MAAM,OAAO,UAAU;IACZ,OAAO,CAAe;IAE/B,YAAoB,MAAoB;QACtC,IAAI,CAAC,OAAO,GAAG,MAAM,CAAC;IACxB,CAAC;IAED,kDAAkD;IAClD,MAAM,CAAC,SAAS,CAAC,MAAyB;QACxC,MAAM,MAAM,GAAG,IAAI,CAAC,IAAI,CACtB,MAAM,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,KAAK,EAAE,EAAE,CAAC,GAAG,GAAG,KAAK,GAAG,KAAK,EAAE,CAAC,CAAC,CACtD,CAAC;QACF,IAAI,MAAM,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC;YAC1C,MAAM,IAAI,UAAU,CAClB,0EAA0E,MAAM,GAAG,CACpF,CAAC;QACJ,OAAO,IAAI,UAAU,CAAC,YAAY,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,GAAG,MAAM,CAAC,CAAC,CAAC;IAC9E,CAAC;IAED,0EAA0E;IAC1E,kCAAkC;IAClC,MAAM,CAAC,QAAQ,CAAC,IAAgB;QAC9B,OAAO,IAAI,UAAU,CAAC,IAAI,YAAY,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;IACxE,CAAC;IAED,IAAI,SAAS;QACX,OAAO,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC;IAC7B,CAAC;IAED,gEAAgE;IAChE,cAAc;QACZ,OAAO,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC;IAC9B,CAAC;IAED,MAAM;QACJ,OAAO,IAAI,UAAU,CACnB,IAAI,CAAC,OAAO,CAAC,MAAM,EACnB,IAAI,CAAC,OAAO,CAAC,UAAU,EACvB,IAAI,CAAC,OAAO,CAAC,UAAU,CACxB,CAAC;IACJ,CAAC;IAED,qEAAqE;IACrE,2EAA2E;IAC3E,gBAAgB;IAChB,MAAM,CAAC,KAAiB;QACtB,MAAM,MAAM,GAAG,IAAI,CAAC,OAAO,CAAC;QAC5B,MAAM,MAAM,GAAG,KAAK,CAAC,OAAO,CAAC;QAC7B,IAAI,GAAG,GAAG,CAAC,CAAC;QACZ,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,KAAK,GAAG,MAAM,CAAC,MAAM,EAAE,KAAK,EAAE;YAChD,GAAG,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC;QACrD,OAAO,GAAG,CAAC;IACb,CAAC;CACF;AAED,oEAAoE;AACpE,MAAM,UAAU,IAAI,CAAC,KAAc;IACjC,IAAI,CAAC,CAAC,KAAK,YAAY,UAAU,CAAC;QAChC,MAAM,IAAI,SAAS,CAAC,iCAAiC,CAAC,CAAC;IACzD,OAAO,KAAK,CAAC;AACf,CAAC"}
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@zukhruf/embeddings",
|
|
3
|
-
"version": "0.3.
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "0.3.54",
|
|
4
|
+
"description": "Document embeddings in SQLite. Embeds only new or changed text through an AI SDK model. Finds the nearest documents of a document by exact cosine similarity in a publish-date window, and the similarity of a query vector to each document of a date range.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
7
7
|
"type": "git",
|
|
@@ -41,11 +41,11 @@
|
|
|
41
41
|
"access": "public"
|
|
42
42
|
},
|
|
43
43
|
"dependencies": {
|
|
44
|
-
"@zukhruf/sqlite-drift": "^0.3.
|
|
45
|
-
"@zukhruf/sqlite-transaction": "^0.3.
|
|
44
|
+
"@zukhruf/sqlite-drift": "^0.3.54",
|
|
45
|
+
"@zukhruf/sqlite-transaction": "^0.3.54",
|
|
46
46
|
"ai": "^7.0.154"
|
|
47
47
|
},
|
|
48
48
|
"devDependencies": {
|
|
49
|
-
"@zukhruf/testing": "^0.3.
|
|
49
|
+
"@zukhruf/testing": "^0.3.54"
|
|
50
50
|
}
|
|
51
51
|
}
|