@marianmeres/kv 4.0.0 → 4.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +108 -28
- package/API.md +123 -26
- package/README.md +113 -8
- package/dist/adapter/postgres.js +16 -6
- package/dist/adapter/sqlite.d.ts +23 -7
- package/dist/adapter/sqlite.js +228 -107
- package/dist/kv.d.ts +9 -1
- package/dist/kv.js +8 -1
- package/dist/utils/driver.d.ts +2 -1
- package/dist/utils/driver.js +7 -1
- package/dist/utils/sqlite-driver.d.ts +37 -0
- package/dist/utils/sqlite-driver.js +67 -0
- package/package.json +9 -8
package/README.md
CHANGED
|
@@ -4,6 +4,7 @@
|
|
|
4
4
|
[](https://opensource.org/licenses/MIT)
|
|
5
5
|
|
|
6
6
|
Key-value storage abstraction layer with support for multiple backend adapters:
|
|
7
|
+
|
|
7
8
|
- redis
|
|
8
9
|
- postgresql
|
|
9
10
|
- sqlite (`node:sqlite`)
|
|
@@ -22,8 +23,11 @@ Versions up to 3.1.1 are also on JSR. From 4.0 the package is published to npm o
|
|
|
22
23
|
The `redis`, `postgres` and `sqlite` adapters take a client you create yourself: a
|
|
23
24
|
`redis` client; a `pg.Pool`, a `pg.Client` or a
|
|
24
25
|
[`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit) `Driver`; or a
|
|
25
|
-
`node:sqlite` `DatabaseSync`, which ships with Node
|
|
26
|
-
uses the postgres adapter has it already.
|
|
26
|
+
`node:sqlite` `DatabaseSync`, which ships with Node, or a sqlkit `Driver` over SQLite. kv
|
|
27
|
+
does not install `pg`: an app that uses the postgres adapter has it already.
|
|
28
|
+
|
|
29
|
+
Requires Node 24.20 or later on the 24 line, or 26.6 or later; not 25. That is what
|
|
30
|
+
`@marianmeres/sqlkit` 0.3 requires, which kv installs since 4.1.0.
|
|
27
31
|
|
|
28
32
|
## Usage
|
|
29
33
|
|
|
@@ -139,22 +143,40 @@ const db = createPgDriver(new pg.Pool({ connectionString: process.env.DATABASE_U
|
|
|
139
143
|
const client = createKVClient("my-app-namespace:", "postgres", { db });
|
|
140
144
|
```
|
|
141
145
|
|
|
146
|
+
A `Driver` over [PGlite](https://pglite.dev), Postgres compiled to WebAssembly, is a
|
|
147
|
+
tested target too: a database inside your process, with no server, for tests or a
|
|
148
|
+
small app.
|
|
149
|
+
|
|
150
|
+
```typescript
|
|
151
|
+
import { PGlite } from "@electric-sql/pglite";
|
|
152
|
+
import { createPgliteDriver } from "@marianmeres/sqlkit/pglite";
|
|
153
|
+
|
|
154
|
+
const db = createPgliteDriver(await PGlite.create()); // in memory; pass a path to persist
|
|
155
|
+
const client = createKVClient("my-app-namespace:", "postgres", { db });
|
|
156
|
+
```
|
|
157
|
+
|
|
142
158
|
- **The adapter never closes what it was given.** `destroy()` leaves the pool, the client
|
|
143
159
|
or the driver open; ending it is yours. Several adapters can share one.
|
|
144
160
|
- **`transaction()` runs on one connection.** On a `pg.Pool` it pins one for the whole
|
|
145
161
|
unit. On one `pg.Client` the calls take turns: another call waits for the transaction
|
|
146
162
|
to finish.
|
|
147
163
|
- **A call made inside a transaction you opened on the same `Driver` joins it**, and
|
|
148
|
-
commits or rolls back with it.
|
|
164
|
+
commits or rolls back with it. A `transaction()` made there is a savepoint of yours:
|
|
165
|
+
when it fails it is undone alone, and your transaction goes on.
|
|
149
166
|
- **A statement the database refused rejects with sqlkit's `SqlError`**: it carries
|
|
150
167
|
Postgres's `code` and message, with node-postgres's error as `cause`. kv's own
|
|
151
|
-
`TypeError` and `RangeError` are unchanged.
|
|
168
|
+
`TypeError` and `RangeError` are unchanged. To test for it with `instanceof`, your own
|
|
169
|
+
`@marianmeres/sqlkit` has to be 0.3.0 or newer.
|
|
152
170
|
- Creates a table (default: `__kv`) in your database.
|
|
153
171
|
- `tableName` must match `^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)?$` (word chars plus an optional `schema.` prefix).
|
|
154
172
|
- Supports optional TTL cleanup via `ttlCleanupIntervalSec` option.
|
|
155
173
|
|
|
156
174
|
### SQLite
|
|
157
175
|
|
|
176
|
+
`db` takes an open `node:sqlite` `DatabaseSync`, or a
|
|
177
|
+
[`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit) `Driver` of the
|
|
178
|
+
`sqlite` dialect:
|
|
179
|
+
|
|
158
180
|
```typescript
|
|
159
181
|
import { DatabaseSync } from "node:sqlite";
|
|
160
182
|
|
|
@@ -163,17 +185,31 @@ const client = createKVClient("my-app-namespace:", "sqlite", {
|
|
|
163
185
|
});
|
|
164
186
|
```
|
|
165
187
|
|
|
166
|
-
|
|
167
|
-
|
|
188
|
+
An app that opened its database with sqlkit passes the driver it shares:
|
|
189
|
+
|
|
190
|
+
```typescript
|
|
191
|
+
import { openSqlite } from "@marianmeres/sqlkit/sqlite";
|
|
192
|
+
|
|
193
|
+
const db = openSqlite("kv.db"); // a busy timeout, and the WAL journal for a file
|
|
194
|
+
const client = createKVClient("my-app-namespace:", "sqlite", { db });
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
- **The adapter never closes what it was given.** `destroy()` leaves the handle or the
|
|
198
|
+
driver open; it is yours, and can be shared with your own tables or other adapters.
|
|
168
199
|
- Creates a table (default: `__kv`). `tableName` must match `^[A-Za-z_][A-Za-z0-9_]*$` — no
|
|
169
200
|
`schema.` prefix.
|
|
170
201
|
- `node:sqlite` is synchronous: each call blocks the event loop while it runs.
|
|
171
202
|
- Multi-statement operations (`getSet`, `incr`/`decr`, `setMultiple`, `clear`,
|
|
172
203
|
`transaction`) run in `BEGIN IMMEDIATE`. Called inside a transaction you opened on the
|
|
173
|
-
same handle
|
|
204
|
+
same handle or through the same `Driver`, they use a savepoint instead: one that fails
|
|
205
|
+
is undone alone, and your `COMMIT`/`ROLLBACK` decides the rest.
|
|
206
|
+
- **On a `Driver`, calls take their turn.** One made inside a transaction you opened on
|
|
207
|
+
the driver joins it; one made from elsewhere waits for it to end.
|
|
208
|
+
- **A statement SQLite refused rejects with sqlkit's `SqlError`**, with node:sqlite's
|
|
209
|
+
error as `cause`. kv's own `TypeError` and `RangeError` are unchanged.
|
|
174
210
|
- The adapter does not touch connection settings. For several processes on one file,
|
|
175
211
|
open it with a busy timeout (`new DatabaseSync(path, { timeout: 5000 })`) and consider
|
|
176
|
-
`PRAGMA journal_mode = WAL
|
|
212
|
+
`PRAGMA journal_mode = WAL`; `openSqlite()` does both.
|
|
177
213
|
- `cas()` compares the stored JSON text, so object key order matters (as with Memory and
|
|
178
214
|
Redis; PostgreSQL compares `jsonb`, which ignores key order).
|
|
179
215
|
- Supports optional TTL cleanup via `ttlCleanupIntervalSec` option.
|
|
@@ -183,6 +219,73 @@ const client = createKVClient("my-app-namespace:", "sqlite", {
|
|
|
183
219
|
- Data is not persisted (in-memory only).
|
|
184
220
|
- Supports optional TTL cleanup via `ttlCleanupIntervalSec` option.
|
|
185
221
|
|
|
222
|
+
## Changes in 4.2.0
|
|
223
|
+
|
|
224
|
+
The sqlite adapter now runs on `@marianmeres/sqlkit`, as the postgres adapter has since
|
|
225
|
+
4.0.0, and takes a sqlkit `Driver` as well as a `DatabaseSync`. A caller who passes a
|
|
226
|
+
`DatabaseSync` sees the same answers, with the edges below. The first five notes are
|
|
227
|
+
about the sqlite adapter, the last about the PostgreSQL one. kv stays on the 4.x line,
|
|
228
|
+
so the second note ships in a minor although 4.0.0 called the same change on PostgreSQL
|
|
229
|
+
breaking. Each note says first what you have to do, if anything.
|
|
230
|
+
|
|
231
|
+
- **The sqlite adapter's `db` also takes a sqlkit `Driver` of the `sqlite` dialect.**
|
|
232
|
+
Nothing to do; pass one if your app opened its database with `openSqlite()`, or wants
|
|
233
|
+
kv and its own code on one driver. It needs a sync tier and `savepoint()`, so it comes
|
|
234
|
+
from sqlkit 0.3.0 or newer; the node:sqlite driver has both. Calls take their turn on
|
|
235
|
+
it, and one made inside a transaction you opened on it joins that transaction (see
|
|
236
|
+
[API.md](API.md#adaptersqliteoptions)).
|
|
237
|
+
- **A statement SQLite refused rejects with sqlkit's `SqlError`.** If your code read
|
|
238
|
+
node:sqlite's `code` or `errcode` on an error from kv, read them on its `cause`, or
|
|
239
|
+
test `instanceof` and read `kind`. A locked database, for one, is a `BusyError`
|
|
240
|
+
(`kind` `"busy"`, `retryable`) with the same message, where it was node:sqlite's own
|
|
241
|
+
`Error` (`code` `"ERR_SQLITE_ERROR"`, `errcode` `5`). `instanceof SqlError` holds when
|
|
242
|
+
your own `@marianmeres/sqlkit` is 0.3.0 or newer. kv's own `TypeError` and
|
|
243
|
+
`RangeError` are unchanged.
|
|
244
|
+
- **`options.db` is typed `SqliteDb`, what the option takes: a `DatabaseSync` or a
|
|
245
|
+
`Driver`.** If you read `client.options.db` back as a `DatabaseSync`, narrow it. It is
|
|
246
|
+
still the object you passed.
|
|
247
|
+
- **`initialize()` and `destroy(true)` do their work after an `await`.** Await them; they
|
|
248
|
+
always returned a promise. On a `DatabaseSync`, the first of them imports sqlkit's
|
|
249
|
+
SQLite driver, and the table is created, or dropped, when the promise settles, no
|
|
250
|
+
longer before the call returns. Every other operation on a `DatabaseSync` still runs
|
|
251
|
+
inside the call.
|
|
252
|
+
- **A `db` that is neither a `DatabaseSync` nor a `Driver` is refused at construction,
|
|
253
|
+
with a `TypeError`.** Nothing to do: such a `db` never worked. So is a `Driver` of
|
|
254
|
+
another dialect, or one without a sync tier or `savepoint()`.
|
|
255
|
+
- **On PostgreSQL, `incr`/`decr` on a value that is not a number no longer aborts a
|
|
256
|
+
transaction you opened on the same `Driver`.** Nothing to do. It still rejects with
|
|
257
|
+
`TypeError("KV value is not a number")`, but the statement no longer fails in the
|
|
258
|
+
database, so your transaction goes on and can commit. Before, your commit rejected
|
|
259
|
+
with a `TransactionAbortedError` and everything in the transaction was lost.
|
|
260
|
+
|
|
261
|
+
## Changes in 4.1.0
|
|
262
|
+
|
|
263
|
+
No signature changes. kv now installs `@marianmeres/sqlkit` 0.3, where 4.0.0 installed
|
|
264
|
+
0.1, and two things follow from that. The last two notes are about the PostgreSQL
|
|
265
|
+
adapter. Each note says first what you have to do, if anything.
|
|
266
|
+
|
|
267
|
+
- **Node 24.20 or later on the 24 line, or 26.6 or later; not 25.** Update Node if yours
|
|
268
|
+
is older. It is what sqlkit requires since 0.2, and kv's `engines` now say the same;
|
|
269
|
+
4.0.0 declared Node >= 24.12. An install on an older Node warns, and fails under
|
|
270
|
+
`engine-strict`.
|
|
271
|
+
- **An app that depends on `@marianmeres/sqlkit` itself moves it to `^0.3.0` together
|
|
272
|
+
with kv.** Nothing to do if you never import sqlkit. Two things need it:
|
|
273
|
+
- A `Driver` you pass as `db` has to come from sqlkit 0.3. The type has a new method,
|
|
274
|
+
`savepoint`, so TypeScript refuses a driver an older sqlkit built.
|
|
275
|
+
- `instanceof SqlError` against a sqlkit of 0.2 or older is `false` for an error from
|
|
276
|
+
the driver kv builds over your pool or client. From 0.3.0 on, two copies of sqlkit
|
|
277
|
+
know each other's errors, so the test holds even when yours is not the copy kv
|
|
278
|
+
installs.
|
|
279
|
+
- **`transaction()` inside a transaction you opened on the same `Driver` is undone alone
|
|
280
|
+
when it fails.** Nothing to do. It runs as a savepoint of your transaction, which goes
|
|
281
|
+
on either way; your commit or rollback still decides what stays. In 4.0.0 the
|
|
282
|
+
operations before the failure stayed in your transaction, and a statement Postgres
|
|
283
|
+
refused aborted it. Outside a transaction of yours nothing changes. A `Driver` without
|
|
284
|
+
`savepoint()` (a sqlkit older than 0.3.0) is refused at construction, with a
|
|
285
|
+
`TypeError`.
|
|
286
|
+
- **A sqlkit `Driver` over PGlite is a tested `db` for the postgres adapter.** Nothing to
|
|
287
|
+
do. 4.0.0 said it was expected to work; now every test of the suite runs on it.
|
|
288
|
+
|
|
186
289
|
## Breaking Changes (4.0.0)
|
|
187
290
|
|
|
188
291
|
No signature changed, and a `pg.Pool` or `pg.Client` you pass today still works. What
|
|
@@ -262,6 +365,7 @@ from you. Each note says first what you have to do, if anything.
|
|
|
262
365
|
Shipped as a minor release, although the first two items are breaking:
|
|
263
366
|
|
|
264
367
|
**Breaking:**
|
|
368
|
+
|
|
265
369
|
- **`ttl(key)` now returns a discriminated `TtlResult`** — `{ state: "missing" | "no-ttl" | "expires", at? }` — instead of `Date | null | false`. Migration:
|
|
266
370
|
|
|
267
371
|
```ts
|
|
@@ -282,6 +386,7 @@ Shipped as a minor release, although the first two items are breaking:
|
|
|
282
386
|
- **Key validation is on by default.** Non-empty strings, no `\0`, total `namespace + key` length ≤ 512. Set `validateKeys: false` on the adapter options to opt out. Previous behavior silently accepted empty/overlong keys and let the backend error cryptically.
|
|
283
387
|
|
|
284
388
|
**Additive:**
|
|
389
|
+
|
|
285
390
|
- New methods `setIfAbsent`, `incr`, `decr`, `getSet`, `cas`, `keysIter` on every adapter.
|
|
286
391
|
- `setMultiple` accepts `{ key, value, ttl? }[]` entries in addition to the legacy `[key, value][]` tuples. Per-pair `ttl` overrides batch `options.ttl`.
|
|
287
392
|
- Deno KV: new `strictDeleteResult: true` option pre-checks existence so `delete()` returns the real did-it-exist flag (default stays `true` for BC).
|
package/dist/adapter/postgres.js
CHANGED
|
@@ -190,8 +190,10 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
190
190
|
const expiresAt = ttl ? new Date(Date.now() + ttl * 1000) : null;
|
|
191
191
|
// - New key OR expired key: value = delta, expires_at = $ttl
|
|
192
192
|
// - Live key: value = old + delta, preserve existing expires_at
|
|
193
|
-
//
|
|
194
|
-
//
|
|
193
|
+
// - Live key that is not a JSON number: the WHERE of DO UPDATE skips it,
|
|
194
|
+
// so the statement updates nothing and returns no row. It raises nothing,
|
|
195
|
+
// so a transaction the host opened on the same Driver is not aborted.
|
|
196
|
+
// The 22P02 catch is a backstop: a delta PostgreSQL cannot cast as numeric.
|
|
195
197
|
try {
|
|
196
198
|
const { rows } = await this.#db.query(`INSERT INTO ${tableName} (key, value, expires_at, updated_at)
|
|
197
199
|
VALUES ($1, to_jsonb($2::numeric), $3, NOW())
|
|
@@ -207,7 +209,11 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
207
209
|
ELSE ${tableName}.expires_at
|
|
208
210
|
END,
|
|
209
211
|
updated_at = NOW()
|
|
212
|
+
WHERE (${tableName}.expires_at IS NOT NULL AND ${tableName}.expires_at <= NOW())
|
|
213
|
+
OR jsonb_typeof(${tableName}.value) = 'number'
|
|
210
214
|
RETURNING value`, [nsKey, delta, expiresAt]);
|
|
215
|
+
if (!rows.length)
|
|
216
|
+
throw new TypeError("KV value is not a number");
|
|
211
217
|
return Number(rows[0].value);
|
|
212
218
|
}
|
|
213
219
|
catch (err) {
|
|
@@ -427,10 +433,14 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
427
433
|
/** @inheritdoc */
|
|
428
434
|
async transaction(operations) {
|
|
429
435
|
this._assertInitialized(); // `async` keeps this a rejection
|
|
430
|
-
//
|
|
431
|
-
//
|
|
432
|
-
//
|
|
433
|
-
|
|
436
|
+
// `savepoint()`, not `transaction()`: outside a transaction the two are the same
|
|
437
|
+
// call, and the driver pins one connection (or, on one client, holds every other
|
|
438
|
+
// caller back), checks the COMMIT, and rolls back when the callback rejects.
|
|
439
|
+
// Inside a transaction the host opened on the same driver, the unit is a
|
|
440
|
+
// savepoint of it: undone alone when it fails, while the host's transaction goes
|
|
441
|
+
// on and decides the rest. A joined `transaction()` would leave the operations
|
|
442
|
+
// before the failure in the host's transaction.
|
|
443
|
+
return this.#db.savepoint(async (tx) => {
|
|
434
444
|
const results = [];
|
|
435
445
|
for (const op of operations) {
|
|
436
446
|
switch (op.type) {
|
package/dist/adapter/sqlite.d.ts
CHANGED
|
@@ -1,19 +1,22 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* @module
|
|
3
3
|
*
|
|
4
|
-
* SQLite key-value storage adapter implementation (`node:sqlite
|
|
4
|
+
* SQLite key-value storage adapter implementation (`node:sqlite`, through a sqlkit
|
|
5
|
+
* driver: kv's own over a handle, or the host's).
|
|
5
6
|
*/
|
|
6
|
-
import type
|
|
7
|
+
import { type SqliteDb } from "../utils/sqlite-driver.ts";
|
|
7
8
|
import { AdapterAbstract, type AdapterAbstractOptions, type Operation, type SetMultipleEntry, type SetOptions, type TtlResult } from "./abstract.ts";
|
|
8
9
|
/**
|
|
9
10
|
* Configuration options for the SQLite KV adapter.
|
|
10
11
|
*/
|
|
11
12
|
export interface AdapterSqliteOptions extends AdapterAbstractOptions {
|
|
12
13
|
/**
|
|
13
|
-
*
|
|
14
|
-
*
|
|
14
|
+
* The database: an open `node:sqlite` handle, e.g. `new DatabaseSync("kv.db")`, or a
|
|
15
|
+
* sqlkit `Driver` of the `sqlite` dialect with a sync tier, e.g. `openSqlite("kv.db")`.
|
|
16
|
+
* The caller owns it: `destroy()` never closes it, so one can be shared across
|
|
17
|
+
* adapters.
|
|
15
18
|
*/
|
|
16
|
-
db:
|
|
19
|
+
db: SqliteDb;
|
|
17
20
|
/**
|
|
18
21
|
* Name of the table to use for storing key-value pairs.
|
|
19
22
|
* The table will be created automatically if it doesn't exist.
|
|
@@ -36,11 +39,20 @@ export interface AdapterSqliteOptions extends AdapterAbstractOptions {
|
|
|
36
39
|
* and index on initialization.
|
|
37
40
|
*
|
|
38
41
|
* @remarks
|
|
42
|
+
* - Over a `DatabaseSync`, every statement goes through kv's own sqlkit driver over the
|
|
43
|
+
* handle, on its synchronous tier: one driver per handle, imported on first use. A
|
|
44
|
+
* statement runs inside the call, and nothing else on the handle interleaves.
|
|
45
|
+
* - Over a sqlkit `Driver` the host passed, a statement takes its turn on the driver's
|
|
46
|
+
* async tier: it waits for a transaction the host has open elsewhere, and joins one
|
|
47
|
+
* it was issued inside.
|
|
39
48
|
* - `node:sqlite` is synchronous, so every operation blocks the event loop for its
|
|
40
|
-
* duration (typically microseconds)
|
|
49
|
+
* duration (typically microseconds).
|
|
41
50
|
* - Multi-statement operations run in `BEGIN IMMEDIATE`, so they stay atomic against
|
|
42
51
|
* other connections and processes on the same file. Inside a transaction the caller
|
|
43
|
-
* already opened on the same handle, they use a
|
|
52
|
+
* already opened (on the same handle, or through the same `Driver`), they use a
|
|
53
|
+
* savepoint instead.
|
|
54
|
+
* - A statement SQLite refused rejects with sqlkit's `SqlError`, node:sqlite's error as
|
|
55
|
+
* its `cause`.
|
|
44
56
|
*
|
|
45
57
|
* @example
|
|
46
58
|
* ```typescript
|
|
@@ -51,6 +63,10 @@ export interface AdapterSqliteOptions extends AdapterAbstractOptions {
|
|
|
51
63
|
* });
|
|
52
64
|
* await client.initialize();
|
|
53
65
|
* await client.set("session:abc", { userId: 123 }, { ttl: 3600 });
|
|
66
|
+
*
|
|
67
|
+
* // or on a sqlkit Driver
|
|
68
|
+
* import { openSqlite } from "@marianmeres/sqlkit/sqlite";
|
|
69
|
+
* const onDriver = createKVClient("myapp:", "sqlite", { db: openSqlite("kv.db") });
|
|
54
70
|
* ```
|
|
55
71
|
*/
|
|
56
72
|
export declare class AdapterSqlite extends AdapterAbstract {
|