montycat 1.1.4 β 1.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +125 -24
- package/dist/classes/generic.d.ts +87 -3
- package/dist/classes/generic.js +115 -15
- package/dist/core/engine.d.ts +119 -0
- package/dist/core/engine.js +174 -0
- package/dist/core/inmemory.d.ts +16 -5
- package/dist/core/inmemory.js +16 -10
- package/dist/core/persistent.d.ts +27 -45
- package/dist/core/persistent.js +29 -62
- package/dist/functions/storeGenericFunctions.js +17 -4
- package/package.json +45 -8
package/README.md
CHANGED
|
@@ -1,12 +1,33 @@
|
|
|
1
|
-
# π Montycat Node.js
|
|
1
|
+
# π Montycat for Node.js & TypeScript β The AI-Native NoSQL Database with Semantic Search for RAG & Agents
|
|
2
|
+
|
|
3
|
+
### Abolish the two-database stack.
|
|
4
|
+
|
|
5
|
+
The official Node.js & TypeScript SDK for [Montycat](https://montygovernance.com) β a self-hosted **NoSQL + vector database** with AI **semantic search** forged into the core, built for **RAG and AI-agent memory**. One Rust engine, not a sprawl of services. **Your hardware. Your data. Your meaning.**
|
|
2
6
|
|
|
3
7
|
[](https://www.npmjs.com/package/montycat)
|
|
4
8
|
[](https://www.npmjs.com/package/montycat)
|
|
5
|
-
[](https://hub.docker.com/r/montygovernance/montycat)
|
|
6
10
|
[](https://nodejs.org/)
|
|
7
|
-
[](https://www.typescriptlang.org/)
|
|
12
|
+
[](https://github.com/MontyGovernance/montycat_node/blob/master/LICENSE)
|
|
13
|
+
|
|
14
|
+
```typescript
|
|
15
|
+
// Search your data by MEANING β no external APIs, no separate vector database.
|
|
16
|
+
// (already ON by default in the montycat-semantic server edition)
|
|
17
|
+
const hits = await Sales.semanticSearchGetValues({ query: 'something to listen to music without wires', limitOutput: { start: 0, stop: 5 } });
|
|
18
|
+
// β [{ key, score, value: { name: 'Wireless Headphones' } }, ...] (matched by meaning, not keywords)
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
> ### π§© All-in-one. AI-native. **Zero external dependencies.**
|
|
22
|
+
> The vector-embedding engine runs **inside** the database β **no** separate vector DB, **no** embedding API, **no** API keys, **no** sidecar service. One engine, one binary, your hardware.
|
|
23
|
+
|
|
24
|
+
## What is Montycat?
|
|
25
|
+
|
|
26
|
+
For a generation we were told the price of intelligence was two systems: a database for your records, and a separate vector store β with its per-query bill β for their meaning. Montycat rejects that tax. It is a **self-hosted NoSQL + vector database**: one Rust-powered engine with semantic search built in, so **RAG, AI-agent memory, and vector search** live where your data already lives. No cloud lock-in. No ops headache. Decentralized by nature, ultra-fast, and natively async.
|
|
8
27
|
|
|
9
|
-
|
|
28
|
+
Think of it as an **open-source, self-hosted alternative to Pinecone, Weaviate, Chroma, Qdrant, and Redis** β a **vector database _and_ a NoSQL store in a single engine**, so your records and their embeddings live together instead of in two systems you have to keep in sync.
|
|
29
|
+
|
|
30
|
+
This client gives JavaScript and TypeScript developers structured, decentralized, secure data through a clean async API that feels native to the language β not another bloated driver. Montycat is not an incremental improvement on the databases you know. It is a break with them.
|
|
10
31
|
|
|
11
32
|
## π§ The Montycat Philosophy
|
|
12
33
|
|
|
@@ -27,6 +48,7 @@ Montycat Node.js Client is the official JavaScript and TypeScript SDK for Montyc
|
|
|
27
48
|
- π‘ `Real-Time Subscriptions` - Subscribe to keyspace or key changes with live updates.
|
|
28
49
|
- π `TLS Security` - Encrypted communication and authenticated connections.
|
|
29
50
|
- 𧬠`Schema Support` - Optional schema layer for validation, structure, and type safety.
|
|
51
|
+
- π§ `AI Semantic & Vector Search` - Rank stored items by meaning with on-device embeddings. kNN vector search for **RAG, AI agents & LLM apps** β no external API, no separate vector database.
|
|
30
52
|
- π§± `Zero Dependencies on ORMs` - No extra abstractions. Just pure, beautiful logic.
|
|
31
53
|
- π§ `Easy Integration` - Works with Express, Fastify, Next.js, Deno, and any Node.js runtime.
|
|
32
54
|
|
|
@@ -45,22 +67,37 @@ Montycat Node.js Client is the official JavaScript and TypeScript SDK for Montyc
|
|
|
45
67
|
|
|
46
68
|
## π The Future Is Structured
|
|
47
69
|
|
|
48
|
-
Other databases
|
|
49
|
-
`Montycat` was built
|
|
50
|
-
No adapters
|
|
70
|
+
Other databases were written in C, C++, or Java β for a world that no longer exists.
|
|
71
|
+
`Montycat` was built in Rust and made to speak JavaScript and TypeScript fluently.
|
|
72
|
+
No adapters. No legacy baggage. No permission asked. Just pure async data flow.
|
|
73
|
+
|
|
74
|
+
The old stack β a database here, a vector store there, an embedding API somewhere in the cloud, and a tangle of sync jobs holding it together β is not an architecture. It is a compromise. Montycat refuses the compromise: **one engine, your hardware, your data, your meaning.**
|
|
75
|
+
|
|
76
|
+
> **When you use Montycat, you are not querying a database. You are interacting with a living data mesh that speaks your language.**
|
|
77
|
+
|
|
78
|
+
## π Example Use Cases
|
|
51
79
|
|
|
52
|
-
|
|
80
|
+
- **RAG pipelines & semantic retrieval** for LLM-powered apps
|
|
81
|
+
- **AI agent / chatbot long-term memory** that survives restarts
|
|
82
|
+
- **Semantic product search & recommendations** β match intent, not keywords
|
|
83
|
+
- Real-time dashboards, analytics, and live collaborative apps
|
|
84
|
+
- Microservice data stores and event-driven systems
|
|
85
|
+
- Data products in a decentralized Mesh architecture
|
|
53
86
|
|
|
54
|
-
|
|
55
|
-
| ------------------------------ | ------------------------ |
|
|
56
|
-
| Insert (10k records) | 40k/sec |
|
|
57
|
-
| Lookup (single key) | 80k/sec |
|
|
58
|
-
| Lookup (filtered query) | 30k/sec |
|
|
59
|
-
| Real-Time subscription updates | < 2ms latency per update |
|
|
87
|
+
## π Get the Engine (30 seconds)
|
|
60
88
|
|
|
61
|
-
|
|
89
|
+
The client talks to a Montycat server. Fastest way β Docker, with AI semantic search built in:
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
docker run -d --name montycat \
|
|
93
|
+
-p 21210:21210 -p 21211:21211 \
|
|
94
|
+
-e MONTYCAT_SUPEROWNER="admin" \
|
|
95
|
+
-e MONTYCAT_PASSWORD="change-me" \
|
|
96
|
+
-v montycat_data:/app/.montycat \
|
|
97
|
+
montygovernance/montycat:semantic
|
|
98
|
+
```
|
|
62
99
|
|
|
63
|
-
|
|
100
|
+
Prefer the lean edition without the embedding engine? Use the `latest` tag. Prebuilt packages (apt, macOS, Windows) at **https://montygovernance.com**.
|
|
64
101
|
|
|
65
102
|
## Installation
|
|
66
103
|
|
|
@@ -74,6 +111,10 @@ or with Yarn:
|
|
|
74
111
|
yarn add montycat
|
|
75
112
|
```
|
|
76
113
|
|
|
114
|
+
> **ESM-only, Node.js 18+.** This package ships native ES modules β use `import`, not
|
|
115
|
+
> `require()`. Works out of the box with TypeScript, Deno, Bun, and any modern Node.js
|
|
116
|
+
> project (`"type": "module"` or `.mjs`).
|
|
117
|
+
|
|
77
118
|
## Quick Start
|
|
78
119
|
|
|
79
120
|
### TypeScript Example
|
|
@@ -151,7 +192,7 @@ const newSale = new SalesSchema({
|
|
|
151
192
|
|
|
152
193
|
const newOrder = new ProductionSchema({
|
|
153
194
|
workOrder: 'WO 000012',
|
|
154
|
-
|
|
195
|
+
customer: 'ACME Corp',
|
|
155
196
|
}).serialize();
|
|
156
197
|
|
|
157
198
|
const res3 = await Sales.insertValue({ value: newSale });
|
|
@@ -167,12 +208,12 @@ const res5 = await Sales.lookupValuesWhere({
|
|
|
167
208
|
});
|
|
168
209
|
|
|
169
210
|
const res6 = await Production.lookupValuesWhere({
|
|
170
|
-
searchCriteria: {
|
|
211
|
+
searchCriteria: { customer: 'ACME Corp' },
|
|
171
212
|
schema: ProductionSchema,
|
|
172
213
|
keyIncluded: true
|
|
173
214
|
});
|
|
174
215
|
|
|
175
|
-
console.log('Lookup results:',
|
|
216
|
+
console.log('Lookup results:', res5, res6);
|
|
176
217
|
|
|
177
218
|
```
|
|
178
219
|
|
|
@@ -233,7 +274,7 @@ const newSale = new SalesSchema({
|
|
|
233
274
|
|
|
234
275
|
const newOrder = new ProductionSchema({
|
|
235
276
|
workOrder: 'WO 000012',
|
|
236
|
-
|
|
277
|
+
customer: 'ACME Corp',
|
|
237
278
|
}).serialize();
|
|
238
279
|
|
|
239
280
|
const res3 = await Sales.insertValue({ value: newSale });
|
|
@@ -249,11 +290,71 @@ const res5 = await Sales.lookupValuesWhere({
|
|
|
249
290
|
});
|
|
250
291
|
|
|
251
292
|
const res6 = await Production.lookupValuesWhere({
|
|
252
|
-
searchCriteria: {
|
|
253
|
-
schema: ProductionSchema
|
|
293
|
+
searchCriteria: { customer: 'ACME Corp' },
|
|
294
|
+
schema: ProductionSchema,
|
|
254
295
|
keyIncluded: true
|
|
255
296
|
});
|
|
256
297
|
|
|
257
|
-
console.log('Lookup results:',
|
|
298
|
+
console.log('Lookup results:', res5, res6);
|
|
299
|
+
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
## π§ AI-Native Semantic Search β Vector Search Built Into Your Database
|
|
303
|
+
|
|
304
|
+
**Stop bolting a separate vector database onto your stack.** Montycat ranks your data by
|
|
305
|
+
*meaning*, not keywords β an embedded, on-device vector-embedding engine turns every write
|
|
306
|
+
into a searchable vector automatically. It's the retrieval layer for **RAG pipelines, AI
|
|
307
|
+
agents, semantic search, recommendation engines, and LLM-powered apps** β with **zero
|
|
308
|
+
external APIs, zero API keys, and zero extra infrastructure.**
|
|
309
|
+
|
|
310
|
+
- π **Semantic / vector search** β kNN similarity over on-device embeddings, not brittle keyword matches.
|
|
311
|
+
- π€ **Built for AI** β RAG, semantic retrieval, AI agents, recommendations, dedup, clustering.
|
|
312
|
+
- π **Private & free** β embeddings never leave your machine. No OpenAI/Cohere bill, no data egress.
|
|
313
|
+
- β‘ **One system, not two** β your data *and* its vectors live in the same database. No sync jobs, no drift, no second service to run.
|
|
314
|
+
- π **Zero setup** β no index tuning, no pipeline: `enableSemanticSearch()` and you're ranking by meaning.
|
|
315
|
+
|
|
316
|
+
> **β οΈ Requires the semantic edition of the server β nothing to compile.** Semantic
|
|
317
|
+
> search runs an embedded ONNX vector-embedding engine that ships only in the
|
|
318
|
+
> **`montycat-semantic`** edition; the default lean `montycat` server does not include it.
|
|
319
|
+
> Get it the way that suits you β pull the **Docker image**
|
|
320
|
+
> (`montygovernance/montycat:semantic`), download the prebuilt **package**, or install
|
|
321
|
+
> `montycat-semantic` from the **apt repository**. The Node.js client API is identical
|
|
322
|
+
> either way; just point it at a semantic-edition server (semantic search is enabled by
|
|
323
|
+
> default there, using the `bge-small` model).
|
|
324
|
+
|
|
325
|
+
The switch is DB-wide and already on in the semantic edition; every keyspace is embedded
|
|
326
|
+
in the background as data is written (the embedding model is downloaded on demand).
|
|
327
|
+
|
|
328
|
+
```typescript
|
|
329
|
+
// Semantic search is ON by default in the montycat-semantic edition β just search.
|
|
330
|
+
// Rank stored items by meaning β two flavors:
|
|
331
|
+
// getValues β each hit is { key, score, value }
|
|
332
|
+
// getKeys β each hit is { key, score } (lighter; fetch a page later with getBulk)
|
|
333
|
+
const hits = await Sales.semanticSearchGetValues({ query: 'something to listen to music without wires', limitOutput: { start: 0, stop: 5 } });
|
|
334
|
+
const keys = await Sales.semanticSearchGetKeys({ query: 'something to listen to music without wires', limitOutput: { start: 0, stop: 5 } });
|
|
335
|
+
|
|
336
|
+
// Optionally drop weak matches by cosine similarity (range [-1, 1]).
|
|
337
|
+
const strong = await Sales.semanticSearchGetKeys({ query: 'something to listen to music without wires', limitOutput: { start: 0, stop: 5 }, minScore: 0.35 });
|
|
338
|
+
|
|
339
|
+
// Control the DB-wide switch (optional β it's already on):
|
|
340
|
+
// switch the embedding model: 'minilm' | 'bge-small' (default) | 'bge-base' | 'e5-small'
|
|
341
|
+
await engine.enableSemanticSearch({ model: 'bge-base' });
|
|
342
|
+
|
|
343
|
+
// turn it off (vectors are kept so re-enabling resumes instantly;
|
|
344
|
+
// pass { dropVectors: true } to also clear stored vectors)
|
|
345
|
+
await engine.disableSemanticSearch();
|
|
346
|
+
```
|
|
347
|
+
|
|
348
|
+
## π Links
|
|
349
|
+
|
|
350
|
+
- π **Website & Docs** β https://montygovernance.com
|
|
351
|
+
- π¦ **npm** β https://www.npmjs.com/package/montycat
|
|
352
|
+
- π³ **Docker Hub** β https://hub.docker.com/r/montygovernance/montycat
|
|
353
|
+
- π» **Source** β https://github.com/MontyGovernance/montycat_node
|
|
354
|
+
|
|
355
|
+
## β FAQ
|
|
258
356
|
|
|
259
|
-
|
|
357
|
+
- **Is Montycat a vector database or a NoSQL database?** Both β one engine. Store records and query them by *meaning* (vector / semantic search) or by key/schema, without running two systems.
|
|
358
|
+
- **Do I need OpenAI or an embedding API?** No. Embeddings run on-device in the `montycat-semantic` server. No API keys, no per-query bill, no data egress.
|
|
359
|
+
- **Is it a Pinecone / Weaviate / Chroma / Qdrant alternative?** Yes β self-hosted and open-source, with a NoSQL store built in.
|
|
360
|
+
- **TypeScript support?** First-class β the package ships its own type definitions. Works with Node.js, Deno, Bun, Express, Fastify, and Next.js.
|
|
@@ -52,9 +52,10 @@ declare class GenericKV {
|
|
|
52
52
|
* @param customKey - The custom key to delete.
|
|
53
53
|
* @return A promise that resolves with the result of the deletion.
|
|
54
54
|
* */
|
|
55
|
-
static deleteKey({ key, customKey }?: {
|
|
55
|
+
static deleteKey({ key, customKey, waitForIndex }?: {
|
|
56
56
|
key?: string;
|
|
57
57
|
customKey?: string | null;
|
|
58
|
+
waitForIndex?: boolean | null;
|
|
58
59
|
}): Promise<any>;
|
|
59
60
|
/**
|
|
60
61
|
* Deletes a bulk of keys from the keyspace.
|
|
@@ -62,9 +63,10 @@ declare class GenericKV {
|
|
|
62
63
|
* @param bulkCustomKeys - An array of custom keys to delete.
|
|
63
64
|
* @return A promise that resolves with the result of the deletion.
|
|
64
65
|
* */
|
|
65
|
-
static deleteBulk({ bulkKeys, bulkCustomKeys }?: {
|
|
66
|
+
static deleteBulk({ bulkKeys, bulkCustomKeys, waitForIndex }?: {
|
|
66
67
|
bulkKeys?: string[];
|
|
67
68
|
bulkCustomKeys?: string[];
|
|
69
|
+
waitForIndex?: boolean | null;
|
|
68
70
|
}): Promise<any>;
|
|
69
71
|
/**
|
|
70
72
|
* Gets bulk values from the keyspace.
|
|
@@ -97,13 +99,14 @@ declare class GenericKV {
|
|
|
97
99
|
* @param bulkCustomKeysValues - An object where custom keys are the keys to update and values are the new values.
|
|
98
100
|
* @return A promise that resolves with the result of the update.
|
|
99
101
|
*/
|
|
100
|
-
static updateBulk({ bulkKeysValues, bulkCustomKeysValues }?: {
|
|
102
|
+
static updateBulk({ bulkKeysValues, bulkCustomKeysValues, waitForIndex }?: {
|
|
101
103
|
bulkKeysValues?: {
|
|
102
104
|
[key: string]: any;
|
|
103
105
|
};
|
|
104
106
|
bulkCustomKeysValues?: {
|
|
105
107
|
[key: string]: any;
|
|
106
108
|
};
|
|
109
|
+
waitForIndex?: boolean | null;
|
|
107
110
|
}): Promise<any>;
|
|
108
111
|
/**
|
|
109
112
|
* Looks up keys based on search criteria.
|
|
@@ -145,6 +148,74 @@ declare class GenericKV {
|
|
|
145
148
|
keyIncluded?: boolean;
|
|
146
149
|
pointersMetadata?: boolean;
|
|
147
150
|
}): Promise<any>;
|
|
151
|
+
/**
|
|
152
|
+
* Shared core for `semanticSearchGetKeys` / `semanticSearchGetValues`.
|
|
153
|
+
* The server command is the same either way (`semantic_search`); the two
|
|
154
|
+
* public methods differ only in which value-inclusion flags they pass,
|
|
155
|
+
* so the wire call lives here once.
|
|
156
|
+
*/
|
|
157
|
+
private static semanticSearchCore;
|
|
158
|
+
/**
|
|
159
|
+
* Semantic (vector similarity) search returning ranked keys only.
|
|
160
|
+
*
|
|
161
|
+
* Ranks stored items by how close their embeddings are to the embedding of
|
|
162
|
+
* `query` and returns just the matched key and score for each hit β the
|
|
163
|
+
* lightweight variant when you only need identity + ranking (e.g. to then
|
|
164
|
+
* `getBulk` a page, or to test membership). Use `semanticSearchGetValues`
|
|
165
|
+
* when you want the values inline.
|
|
166
|
+
*
|
|
167
|
+
* Semantic search must be enabled first (see `Engine.enableSemanticSearch`).
|
|
168
|
+
* The keyspace is embedded in the background as items are written, so
|
|
169
|
+
* results reflect whatever has been embedded so far.
|
|
170
|
+
*
|
|
171
|
+
* @param query - The natural-language query text to embed and search for.
|
|
172
|
+
* @param limitOutput - Start/stop over the ranked hits; `{start: 0, stop: 0}`
|
|
173
|
+
* (the default) lets the server apply its default top-k (10).
|
|
174
|
+
* @param minScore - Drop hits whose cosine similarity (in [-1, 1]) is below
|
|
175
|
+
* this value. Default null (no score filter).
|
|
176
|
+
* @return A promise resolving with ranked hits, each `{key, score}`.
|
|
177
|
+
*/
|
|
178
|
+
static semanticSearchGetKeys({ query, limitOutput, minScore }: {
|
|
179
|
+
query: string;
|
|
180
|
+
limitOutput?: {
|
|
181
|
+
start: number;
|
|
182
|
+
stop: number;
|
|
183
|
+
};
|
|
184
|
+
minScore?: number | null;
|
|
185
|
+
}): Promise<any>;
|
|
186
|
+
/**
|
|
187
|
+
* Semantic (vector similarity) search returning ranked hits with their values.
|
|
188
|
+
*
|
|
189
|
+
* Ranks stored items by how close their embeddings are to the embedding of
|
|
190
|
+
* `query` and returns the value inline with each hit β the key is always
|
|
191
|
+
* included so every value is tagged with its key. Use
|
|
192
|
+
* `semanticSearchGetKeys` when you only need keys + scores.
|
|
193
|
+
*
|
|
194
|
+
* Semantic search must be enabled first (see `Engine.enableSemanticSearch`).
|
|
195
|
+
* The keyspace is embedded in the background as items are written, so
|
|
196
|
+
* results reflect whatever has been embedded so far.
|
|
197
|
+
*
|
|
198
|
+
* @param query - The natural-language query text to embed and search for.
|
|
199
|
+
* @param limitOutput - Start/stop over the ranked hits; `{start: 0, stop: 0}`
|
|
200
|
+
* (the default) lets the server apply its default top-k (10).
|
|
201
|
+
* @param minScore - Drop hits whose cosine similarity (in [-1, 1]) is below
|
|
202
|
+
* this value. Default null (no score filter).
|
|
203
|
+
* @param withPointers - Whether to include pointers (foreign values) in each
|
|
204
|
+
* returned value.
|
|
205
|
+
* @param pointersMetadata - Whether to include pointer metadata in each
|
|
206
|
+
* returned value.
|
|
207
|
+
* @return A promise resolving with ranked hits, each `{key, score, value}`.
|
|
208
|
+
*/
|
|
209
|
+
static semanticSearchGetValues({ query, limitOutput, minScore, withPointers, pointersMetadata }: {
|
|
210
|
+
query: string;
|
|
211
|
+
limitOutput?: {
|
|
212
|
+
start: number;
|
|
213
|
+
stop: number;
|
|
214
|
+
};
|
|
215
|
+
minScore?: number | null;
|
|
216
|
+
withPointers?: boolean;
|
|
217
|
+
pointersMetadata?: boolean;
|
|
218
|
+
}): Promise<any>;
|
|
148
219
|
/**
|
|
149
220
|
* Gets the length of the keyspace.
|
|
150
221
|
* @returns A promise that resolves with the length of the keyspace.
|
|
@@ -173,5 +244,18 @@ declare class GenericKV {
|
|
|
173
244
|
* @returns The current instance of GenericKV.
|
|
174
245
|
*/
|
|
175
246
|
showStoreProperties(): this;
|
|
247
|
+
/**
|
|
248
|
+
* Subscribes to changes in the keyspace.
|
|
249
|
+
* Inherited by InMemory and Persistent - `this.persistent` resolves correctly per subclass.
|
|
250
|
+
* @param callback - A callback function to handle incoming data.
|
|
251
|
+
* @param key - A specific key to subscribe to.
|
|
252
|
+
* @param customKey - A custom key to subscribe to (will be hashed with XXH32).
|
|
253
|
+
* @returns A promise that resolves with a subscription handle ({ stop() }).
|
|
254
|
+
*/
|
|
255
|
+
static subscribe({ callback, key, customKey }: {
|
|
256
|
+
callback?: (data: any) => void;
|
|
257
|
+
key?: string;
|
|
258
|
+
customKey?: string;
|
|
259
|
+
}): Promise<any>;
|
|
176
260
|
}
|
|
177
261
|
export default GenericKV;
|
package/dist/classes/generic.js
CHANGED
|
@@ -34,15 +34,15 @@ class GenericKV {
|
|
|
34
34
|
* @return A promise that resolves with the retrieved value.
|
|
35
35
|
*/
|
|
36
36
|
static async getValue({ key = "", customKey = null, withPointers = false, keyIncluded = false, pointersMetadata = false } = {}) {
|
|
37
|
-
if (pointersMetadata && withPointers) {
|
|
38
|
-
throw new Error("You select both pointers value and pointers metadata. Choose one");
|
|
39
|
-
}
|
|
40
37
|
try {
|
|
38
|
+
if (key && customKey) {
|
|
39
|
+
throw new Error("Provide either 'key' or 'customKey', not both.");
|
|
40
|
+
}
|
|
41
|
+
if (!key && !customKey) {
|
|
42
|
+
throw new Error("Provide either 'key' or 'customKey'.");
|
|
43
|
+
}
|
|
41
44
|
if (customKey)
|
|
42
45
|
key = convertCustomKey(customKey);
|
|
43
|
-
if (!key) {
|
|
44
|
-
throw new Error("No key provided");
|
|
45
|
-
}
|
|
46
46
|
this.command = "get_value";
|
|
47
47
|
const query = convertToBinaryQuery(this, { key, withPointers, keyIncluded, pointersMetadata });
|
|
48
48
|
return runQuery(this, query);
|
|
@@ -59,11 +59,14 @@ class GenericKV {
|
|
|
59
59
|
* */
|
|
60
60
|
static async listAllDependingKeys({ key = "", customKey = null } = {}) {
|
|
61
61
|
try {
|
|
62
|
+
if (key && customKey) {
|
|
63
|
+
throw new Error("Provide either 'key' or 'customKey', not both.");
|
|
64
|
+
}
|
|
65
|
+
if (!key && !customKey) {
|
|
66
|
+
throw new Error("Provide either 'key' or 'customKey'.");
|
|
67
|
+
}
|
|
62
68
|
if (customKey)
|
|
63
69
|
key = convertCustomKey(customKey);
|
|
64
|
-
if (!key) {
|
|
65
|
-
throw new Error("No key provided");
|
|
66
|
-
}
|
|
67
70
|
this.command = "list_all_depending_keys";
|
|
68
71
|
const query = convertToBinaryQuery(this, { key });
|
|
69
72
|
return runQuery(this, query);
|
|
@@ -83,15 +86,18 @@ class GenericKV {
|
|
|
83
86
|
* @param customKey - The custom key to delete.
|
|
84
87
|
* @return A promise that resolves with the result of the deletion.
|
|
85
88
|
* */
|
|
86
|
-
static async deleteKey({ key = "", customKey = null } = {}) {
|
|
89
|
+
static async deleteKey({ key = "", customKey = null, waitForIndex = null } = {}) {
|
|
87
90
|
try {
|
|
91
|
+
if (key && customKey) {
|
|
92
|
+
throw new Error("Provide either 'key' or 'customKey', not both.");
|
|
93
|
+
}
|
|
88
94
|
if (customKey)
|
|
89
95
|
key = convertCustomKey(customKey);
|
|
90
96
|
if (!key) {
|
|
91
97
|
throw new Error("No key provided");
|
|
92
98
|
}
|
|
93
99
|
this.command = "delete_key";
|
|
94
|
-
const query = convertToBinaryQuery(this, { key });
|
|
100
|
+
const query = convertToBinaryQuery(this, { key, waitForIndex });
|
|
95
101
|
return runQuery(this, query);
|
|
96
102
|
}
|
|
97
103
|
catch (err) {
|
|
@@ -104,7 +110,7 @@ class GenericKV {
|
|
|
104
110
|
* @param bulkCustomKeys - An array of custom keys to delete.
|
|
105
111
|
* @return A promise that resolves with the result of the deletion.
|
|
106
112
|
* */
|
|
107
|
-
static async deleteBulk({ bulkKeys = [], bulkCustomKeys = [] } = {}) {
|
|
113
|
+
static async deleteBulk({ bulkKeys = [], bulkCustomKeys = [], waitForIndex = null } = {}) {
|
|
108
114
|
try {
|
|
109
115
|
if (bulkCustomKeys.length)
|
|
110
116
|
bulkKeys = bulkKeys.concat(convertCustomKeys(bulkCustomKeys));
|
|
@@ -112,7 +118,7 @@ class GenericKV {
|
|
|
112
118
|
throw new Error("No keys provided");
|
|
113
119
|
}
|
|
114
120
|
this.command = "delete_bulk";
|
|
115
|
-
const query = convertToBinaryQuery(this, { bulkKeys });
|
|
121
|
+
const query = convertToBinaryQuery(this, { bulkKeys, waitForIndex });
|
|
116
122
|
return runQuery(this, query);
|
|
117
123
|
}
|
|
118
124
|
catch (err) {
|
|
@@ -156,7 +162,7 @@ class GenericKV {
|
|
|
156
162
|
* @param bulkCustomKeysValues - An object where custom keys are the keys to update and values are the new values.
|
|
157
163
|
* @return A promise that resolves with the result of the update.
|
|
158
164
|
*/
|
|
159
|
-
static async updateBulk({ bulkKeysValues = {}, bulkCustomKeysValues = {} } = {}) {
|
|
165
|
+
static async updateBulk({ bulkKeysValues = {}, bulkCustomKeysValues = {}, waitForIndex = null } = {}) {
|
|
160
166
|
try {
|
|
161
167
|
if (Object.keys(bulkCustomKeysValues).length) {
|
|
162
168
|
bulkKeysValues = { ...bulkKeysValues, ...convertCustomKeysValues(bulkCustomKeysValues) };
|
|
@@ -165,7 +171,7 @@ class GenericKV {
|
|
|
165
171
|
throw new Error("No keys provided");
|
|
166
172
|
}
|
|
167
173
|
this.command = "update_bulk";
|
|
168
|
-
const query = convertToBinaryQuery(this, { bulkKeysValues });
|
|
174
|
+
const query = convertToBinaryQuery(this, { bulkKeysValues, waitForIndex });
|
|
169
175
|
return runQuery(this, query);
|
|
170
176
|
}
|
|
171
177
|
catch (err) {
|
|
@@ -209,6 +215,76 @@ class GenericKV {
|
|
|
209
215
|
throw err;
|
|
210
216
|
}
|
|
211
217
|
}
|
|
218
|
+
/**
|
|
219
|
+
* Shared core for `semanticSearchGetKeys` / `semanticSearchGetValues`.
|
|
220
|
+
* The server command is the same either way (`semantic_search`); the two
|
|
221
|
+
* public methods differ only in which value-inclusion flags they pass,
|
|
222
|
+
* so the wire call lives here once.
|
|
223
|
+
*/
|
|
224
|
+
static async semanticSearchCore(query, limitOutput, minScore, withPointers, keyIncluded, pointersMetadata) {
|
|
225
|
+
if (!query || !query.trim()) {
|
|
226
|
+
throw new Error("No query text provided for semantic search.");
|
|
227
|
+
}
|
|
228
|
+
this.command = "semantic_search";
|
|
229
|
+
const binaryQuery = convertToBinaryQuery(this, {
|
|
230
|
+
semanticQuery: query,
|
|
231
|
+
limitOutput,
|
|
232
|
+
minScore,
|
|
233
|
+
withPointers,
|
|
234
|
+
keyIncluded,
|
|
235
|
+
pointersMetadata,
|
|
236
|
+
});
|
|
237
|
+
return runQuery(this, binaryQuery);
|
|
238
|
+
}
|
|
239
|
+
/**
|
|
240
|
+
* Semantic (vector similarity) search returning ranked keys only.
|
|
241
|
+
*
|
|
242
|
+
* Ranks stored items by how close their embeddings are to the embedding of
|
|
243
|
+
* `query` and returns just the matched key and score for each hit β the
|
|
244
|
+
* lightweight variant when you only need identity + ranking (e.g. to then
|
|
245
|
+
* `getBulk` a page, or to test membership). Use `semanticSearchGetValues`
|
|
246
|
+
* when you want the values inline.
|
|
247
|
+
*
|
|
248
|
+
* Semantic search must be enabled first (see `Engine.enableSemanticSearch`).
|
|
249
|
+
* The keyspace is embedded in the background as items are written, so
|
|
250
|
+
* results reflect whatever has been embedded so far.
|
|
251
|
+
*
|
|
252
|
+
* @param query - The natural-language query text to embed and search for.
|
|
253
|
+
* @param limitOutput - Start/stop over the ranked hits; `{start: 0, stop: 0}`
|
|
254
|
+
* (the default) lets the server apply its default top-k (10).
|
|
255
|
+
* @param minScore - Drop hits whose cosine similarity (in [-1, 1]) is below
|
|
256
|
+
* this value. Default null (no score filter).
|
|
257
|
+
* @return A promise resolving with ranked hits, each `{key, score}`.
|
|
258
|
+
*/
|
|
259
|
+
static async semanticSearchGetKeys({ query, limitOutput = { start: 0, stop: 0 }, minScore = null }) {
|
|
260
|
+
return this.semanticSearchCore(query, limitOutput, minScore, false, false, false);
|
|
261
|
+
}
|
|
262
|
+
/**
|
|
263
|
+
* Semantic (vector similarity) search returning ranked hits with their values.
|
|
264
|
+
*
|
|
265
|
+
* Ranks stored items by how close their embeddings are to the embedding of
|
|
266
|
+
* `query` and returns the value inline with each hit β the key is always
|
|
267
|
+
* included so every value is tagged with its key. Use
|
|
268
|
+
* `semanticSearchGetKeys` when you only need keys + scores.
|
|
269
|
+
*
|
|
270
|
+
* Semantic search must be enabled first (see `Engine.enableSemanticSearch`).
|
|
271
|
+
* The keyspace is embedded in the background as items are written, so
|
|
272
|
+
* results reflect whatever has been embedded so far.
|
|
273
|
+
*
|
|
274
|
+
* @param query - The natural-language query text to embed and search for.
|
|
275
|
+
* @param limitOutput - Start/stop over the ranked hits; `{start: 0, stop: 0}`
|
|
276
|
+
* (the default) lets the server apply its default top-k (10).
|
|
277
|
+
* @param minScore - Drop hits whose cosine similarity (in [-1, 1]) is below
|
|
278
|
+
* this value. Default null (no score filter).
|
|
279
|
+
* @param withPointers - Whether to include pointers (foreign values) in each
|
|
280
|
+
* returned value.
|
|
281
|
+
* @param pointersMetadata - Whether to include pointer metadata in each
|
|
282
|
+
* returned value.
|
|
283
|
+
* @return A promise resolving with ranked hits, each `{key, score, value}`.
|
|
284
|
+
*/
|
|
285
|
+
static async semanticSearchGetValues({ query, limitOutput = { start: 0, stop: 0 }, minScore = null, withPointers = false, pointersMetadata = false }) {
|
|
286
|
+
return this.semanticSearchCore(query, limitOutput, minScore, withPointers, true, pointersMetadata);
|
|
287
|
+
}
|
|
212
288
|
/**
|
|
213
289
|
* Gets the length of the keyspace.
|
|
214
290
|
* @returns A promise that resolves with the length of the keyspace.
|
|
@@ -301,5 +377,29 @@ class GenericKV {
|
|
|
301
377
|
showStoreProperties() {
|
|
302
378
|
return this;
|
|
303
379
|
}
|
|
380
|
+
/**
|
|
381
|
+
* Subscribes to changes in the keyspace.
|
|
382
|
+
* Inherited by InMemory and Persistent - `this.persistent` resolves correctly per subclass.
|
|
383
|
+
* @param callback - A callback function to handle incoming data.
|
|
384
|
+
* @param key - A specific key to subscribe to.
|
|
385
|
+
* @param customKey - A custom key to subscribe to (will be hashed with XXH32).
|
|
386
|
+
* @returns A promise that resolves with a subscription handle ({ stop() }).
|
|
387
|
+
*/
|
|
388
|
+
static async subscribe({ callback, key, customKey }) {
|
|
389
|
+
if (key && customKey) {
|
|
390
|
+
throw new Error("Provide either 'key' or 'customKey', not both.");
|
|
391
|
+
}
|
|
392
|
+
const effectiveKey = customKey ? convertCustomKey(customKey) : (key || null);
|
|
393
|
+
const queryObj = {
|
|
394
|
+
subscribe: true,
|
|
395
|
+
store: this.store,
|
|
396
|
+
keyspace: this.keyspace,
|
|
397
|
+
username: this.username,
|
|
398
|
+
password: this.password,
|
|
399
|
+
persistent: this.persistent,
|
|
400
|
+
key: effectiveKey,
|
|
401
|
+
};
|
|
402
|
+
return await runQuery(this, JSON.stringify(queryObj), callback, true);
|
|
403
|
+
}
|
|
304
404
|
}
|
|
305
405
|
export default GenericKV;
|