@marianmeres/kv 3.1.1 → 4.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,10 +1,10 @@
1
1
  # @marianmeres/kv
2
2
 
3
3
  [![NPM version](https://img.shields.io/npm/v/@marianmeres/kv.svg)](https://www.npmjs.com/package/@marianmeres/kv)
4
- [![JSR version](https://jsr.io/badges/@marianmeres/kv)](https://jsr.io/@marianmeres/kv)
5
4
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
6
5
 
7
6
  Key-value storage abstraction layer with support for multiple backend adapters:
7
+
8
8
  - redis
9
9
  - postgresql
10
10
  - sqlite (`node:sqlite`)
@@ -16,11 +16,18 @@ The API is inspired by the Redis API.
16
16
 
17
17
  ```sh
18
18
  npm install @marianmeres/kv # or pnpm add
19
- deno add jsr:@marianmeres/kv # Deno
20
19
  ```
21
20
 
22
- The `redis`, `postgres` and `sqlite` adapters take a client you create yourself (`redis`,
23
- `pg`, or a `node:sqlite` `DatabaseSync`, which ships with Node). Requires Node >= 24.12.
21
+ Versions up to 3.1.1 are also on JSR. From 4.0 the package is published to npm only.
22
+
23
+ The `redis`, `postgres` and `sqlite` adapters take a client you create yourself: a
24
+ `redis` client; a `pg.Pool`, a `pg.Client` or a
25
+ [`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit) `Driver`; or a
26
+ `node:sqlite` `DatabaseSync`, which ships with Node, or a sqlkit `Driver` over SQLite. kv
27
+ does not install `pg`: an app that uses the postgres adapter has it already.
28
+
29
+ Requires Node 24.20 or later on the 24 line, or 26.6 or later; not 25. That is what
30
+ `@marianmeres/sqlkit` 0.3 requires, which kv installs since 4.1.0.
24
31
 
25
32
  ## Usage
26
33
 
@@ -98,20 +105,20 @@ type TtlResult =
98
105
 
99
106
  ## Adapter Feature Matrix
100
107
 
101
- | Feature | Memory | Redis | PostgreSQL | SQLite |
102
- | --------------------------------------------------------- | :-----------------: | :--------------------: | :--------------: | :-----------------------: |
103
- | Persistent | ✗ | ✓ | ✓ | ✓ (file database) |
104
- | TTL on `set()` | ✓ | ✓ | ✓ | ✓ |
105
- | `ttl()` → `expires`/`no-ttl`/`missing` | ✓ | ✓ | ✓ | ✓ |
106
- | `expire()` | ✓ | ✓ | ✓ | ✓ |
107
- | Atomic `setIfAbsent` / `incr` / `decr` / `cas` / `getSet` | ✓ | ✓ (Lua) | ✓ (UPSERT) | ✓ (UPSERT / IMMEDIATE tx) |
108
- | Sorted `keys()` | ✓ | ✓ | ✓ | ✓ |
109
- | Streaming `keysIter()` | ✓ | ✓ (SCAN) | ✓ (cursor) | ✓ (keyset paging) |
110
- | Per-pair TTL in `setMultiple` | ✓ | ✓ | ✓ | ✓ |
111
- | Atomic `transaction()` | ✓ (single-threaded) | ✓ (MULTI) | ✓ (BEGIN/COMMIT) | ✓ (BEGIN IMMEDIATE) |
112
- | `delete()` returns real existed-flag | ✓ | ✓ | ✓ | ✓ |
113
- | Background TTL cleanup | optional | native (Redis does it) | optional | optional |
114
- | `keys()` / `clear()` | ✓ | ✓ (non-cluster only) | ✓ | ✓ |
108
+ | Feature | Memory | Redis | PostgreSQL | SQLite |
109
+ | --------------------------------------------------------- | :-----------------: | :--------------------: | :---------------: | :-----------------------: |
110
+ | Persistent | ✗ | ✓ | ✓ | ✓ (file database) |
111
+ | TTL on `set()` | ✓ | ✓ | ✓ | ✓ |
112
+ | `ttl()` → `expires`/`no-ttl`/`missing` | ✓ | ✓ | ✓ | ✓ |
113
+ | `expire()` | ✓ | ✓ | ✓ | ✓ |
114
+ | Atomic `setIfAbsent` / `incr` / `decr` / `cas` / `getSet` | ✓ | ✓ (Lua) | ✓ (UPSERT) | ✓ (UPSERT / IMMEDIATE tx) |
115
+ | Sorted `keys()` | ✓ | ✓ | ✓ | ✓ |
116
+ | Streaming `keysIter()` | ✓ | ✓ (SCAN) | ✓ (keyset paging) | ✓ (keyset paging) |
117
+ | Per-pair TTL in `setMultiple` | ✓ | ✓ | ✓ | ✓ |
118
+ | Atomic `transaction()` | ✓ (single-threaded) | ✓ (MULTI) | ✓ (BEGIN/COMMIT) | ✓ (BEGIN IMMEDIATE) |
119
+ | `delete()` returns real existed-flag | ✓ | ✓ | ✓ | ✓ |
120
+ | Background TTL cleanup | optional | native (Redis does it) | optional | optional |
121
+ | `keys()` / `clear()` | ✓ | ✓ (non-cluster only) | ✓ | ✓ |
115
122
 
116
123
  ## Adapter-Specific Limitations
117
124
 
@@ -123,13 +130,53 @@ type TtlResult =
123
130
 
124
131
  ### PostgreSQL
125
132
 
133
+ `db` takes a `pg.Pool`, a connected `pg.Client`, or a `Driver` from
134
+ [`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit) of the `pg`
135
+ dialect. An app that already builds one driver for everything that talks SQL passes it
136
+ here too:
137
+
138
+ ```typescript
139
+ import { createPgDriver } from "@marianmeres/sqlkit/pg";
140
+ import pg from "pg";
141
+
142
+ const db = createPgDriver(new pg.Pool({ connectionString: process.env.DATABASE_URL }));
143
+ const client = createKVClient("my-app-namespace:", "postgres", { db });
144
+ ```
145
+
146
+ A `Driver` over [PGlite](https://pglite.dev), Postgres compiled to WebAssembly, is a
147
+ tested target too: a database inside your process, with no server, for tests or a
148
+ small app.
149
+
150
+ ```typescript
151
+ import { PGlite } from "@electric-sql/pglite";
152
+ import { createPgliteDriver } from "@marianmeres/sqlkit/pglite";
153
+
154
+ const db = createPgliteDriver(await PGlite.create()); // in memory; pass a path to persist
155
+ const client = createKVClient("my-app-namespace:", "postgres", { db });
156
+ ```
157
+
158
+ - **The adapter never closes what it was given.** `destroy()` leaves the pool, the client
159
+ or the driver open; ending it is yours. Several adapters can share one.
160
+ - **`transaction()` runs on one connection.** On a `pg.Pool` it pins one for the whole
161
+ unit. On one `pg.Client` the calls take turns: another call waits for the transaction
162
+ to finish.
163
+ - **A call made inside a transaction you opened on the same `Driver` joins it**, and
164
+ commits or rolls back with it. A `transaction()` made there is a savepoint of yours:
165
+ when it fails it is undone alone, and your transaction goes on.
166
+ - **A statement the database refused rejects with sqlkit's `SqlError`**: it carries
167
+ Postgres's `code` and message, with node-postgres's error as `cause`. kv's own
168
+ `TypeError` and `RangeError` are unchanged. To test for it with `instanceof`, your own
169
+ `@marianmeres/sqlkit` has to be 0.3.0 or newer.
126
170
  - Creates a table (default: `__kv`) in your database.
127
171
  - `tableName` must match `^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)?$` (word chars plus an optional `schema.` prefix).
128
- - `transaction()` pins a single connection from the Pool for the whole BEGIN/…/COMMIT block — safe to call against a `pg.Pool`.
129
172
  - Supports optional TTL cleanup via `ttlCleanupIntervalSec` option.
130
173
 
131
174
  ### SQLite
132
175
 
176
+ `db` takes an open `node:sqlite` `DatabaseSync`, or a
177
+ [`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit) `Driver` of the
178
+ `sqlite` dialect:
179
+
133
180
  ```typescript
134
181
  import { DatabaseSync } from "node:sqlite";
135
182
 
@@ -138,17 +185,31 @@ const client = createKVClient("my-app-namespace:", "sqlite", {
138
185
  });
139
186
  ```
140
187
 
141
- - Takes an open `node:sqlite` `DatabaseSync` and never closes it — the handle is yours,
142
- and can be shared with your own tables or other adapters.
188
+ An app that opened its database with sqlkit passes the driver it shares:
189
+
190
+ ```typescript
191
+ import { openSqlite } from "@marianmeres/sqlkit/sqlite";
192
+
193
+ const db = openSqlite("kv.db"); // a busy timeout, and the WAL journal for a file
194
+ const client = createKVClient("my-app-namespace:", "sqlite", { db });
195
+ ```
196
+
197
+ - **The adapter never closes what it was given.** `destroy()` leaves the handle or the
198
+ driver open; it is yours, and can be shared with your own tables or other adapters.
143
199
  - Creates a table (default: `__kv`). `tableName` must match `^[A-Za-z_][A-Za-z0-9_]*$` — no
144
200
  `schema.` prefix.
145
201
  - `node:sqlite` is synchronous: each call blocks the event loop while it runs.
146
202
  - Multi-statement operations (`getSet`, `incr`/`decr`, `setMultiple`, `clear`,
147
203
  `transaction`) run in `BEGIN IMMEDIATE`. Called inside a transaction you opened on the
148
- same handle, they use a savepoint instead, so your `COMMIT`/`ROLLBACK` decides.
204
+ same handle or through the same `Driver`, they use a savepoint instead: one that fails
205
+ is undone alone, and your `COMMIT`/`ROLLBACK` decides the rest.
206
+ - **On a `Driver`, calls take their turn.** One made inside a transaction you opened on
207
+ the driver joins it; one made from elsewhere waits for it to end.
208
+ - **A statement SQLite refused rejects with sqlkit's `SqlError`**, with node:sqlite's
209
+ error as `cause`. kv's own `TypeError` and `RangeError` are unchanged.
149
210
  - The adapter does not touch connection settings. For several processes on one file,
150
211
  open it with a busy timeout (`new DatabaseSync(path, { timeout: 5000 })`) and consider
151
- `PRAGMA journal_mode = WAL`.
212
+ `PRAGMA journal_mode = WAL`; `openSqlite()` does both.
152
213
  - `cas()` compares the stored JSON text, so object key order matters (as with Memory and
153
214
  Redis; PostgreSQL compares `jsonb`, which ignores key order).
154
215
  - Supports optional TTL cleanup via `ttlCleanupIntervalSec` option.
@@ -158,6 +219,136 @@ const client = createKVClient("my-app-namespace:", "sqlite", {
158
219
  - Data is not persisted (in-memory only).
159
220
  - Supports optional TTL cleanup via `ttlCleanupIntervalSec` option.
160
221
 
222
+ ## Changes in 4.2.0
223
+
224
+ The sqlite adapter now runs on `@marianmeres/sqlkit`, as the postgres adapter has since
225
+ 4.0.0, and takes a sqlkit `Driver` as well as a `DatabaseSync`. A caller who passes a
226
+ `DatabaseSync` sees the same answers, with the edges below. The first five notes are
227
+ about the sqlite adapter, the last about the PostgreSQL one. kv stays on the 4.x line,
228
+ so the second note ships in a minor although 4.0.0 called the same change on PostgreSQL
229
+ breaking. Each note says first what you have to do, if anything.
230
+
231
+ - **The sqlite adapter's `db` also takes a sqlkit `Driver` of the `sqlite` dialect.**
232
+ Nothing to do; pass one if your app opened its database with `openSqlite()`, or wants
233
+ kv and its own code on one driver. It needs a sync tier and `savepoint()`, so it comes
234
+ from sqlkit 0.3.0 or newer; the node:sqlite driver has both. Calls take their turn on
235
+ it, and one made inside a transaction you opened on it joins that transaction (see
236
+ [API.md](API.md#adaptersqliteoptions)).
237
+ - **A statement SQLite refused rejects with sqlkit's `SqlError`.** If your code read
238
+ node:sqlite's `code` or `errcode` on an error from kv, read them on its `cause`, or
239
+ test `instanceof` and read `kind`. A locked database, for one, is a `BusyError`
240
+ (`kind` `"busy"`, `retryable`) with the same message, where it was node:sqlite's own
241
+ `Error` (`code` `"ERR_SQLITE_ERROR"`, `errcode` `5`). `instanceof SqlError` holds when
242
+ your own `@marianmeres/sqlkit` is 0.3.0 or newer. kv's own `TypeError` and
243
+ `RangeError` are unchanged.
244
+ - **`options.db` is typed `SqliteDb`, what the option takes: a `DatabaseSync` or a
245
+ `Driver`.** If you read `client.options.db` back as a `DatabaseSync`, narrow it. It is
246
+ still the object you passed.
247
+ - **`initialize()` and `destroy(true)` do their work after an `await`.** Await them; they
248
+ always returned a promise. On a `DatabaseSync`, the first of them imports sqlkit's
249
+ SQLite driver, and the table is created, or dropped, when the promise settles, no
250
+ longer before the call returns. Every other operation on a `DatabaseSync` still runs
251
+ inside the call.
252
+ - **A `db` that is neither a `DatabaseSync` nor a `Driver` is refused at construction,
253
+ with a `TypeError`.** Nothing to do: such a `db` never worked. So is a `Driver` of
254
+ another dialect, or one without a sync tier or `savepoint()`.
255
+ - **On PostgreSQL, `incr`/`decr` on a value that is not a number no longer aborts a
256
+ transaction you opened on the same `Driver`.** Nothing to do. It still rejects with
257
+ `TypeError("KV value is not a number")`, but the statement no longer fails in the
258
+ database, so your transaction goes on and can commit. Before, your commit rejected
259
+ with a `TransactionAbortedError` and everything in the transaction was lost.
260
+
261
+ ## Changes in 4.1.0
262
+
263
+ No signature changes. kv now installs `@marianmeres/sqlkit` 0.3, where 4.0.0 installed
264
+ 0.1, and two things follow from that. The last two notes are about the PostgreSQL
265
+ adapter. Each note says first what you have to do, if anything.
266
+
267
+ - **Node 24.20 or later on the 24 line, or 26.6 or later; not 25.** Update Node if yours
268
+ is older. It is what sqlkit requires since 0.2, and kv's `engines` now say the same;
269
+ 4.0.0 declared Node >= 24.12. An install on an older Node warns, and fails under
270
+ `engine-strict`.
271
+ - **An app that depends on `@marianmeres/sqlkit` itself moves it to `^0.3.0` together
272
+ with kv.** Nothing to do if you never import sqlkit. Two things need it:
273
+ - A `Driver` you pass as `db` has to come from sqlkit 0.3. The type has a new method,
274
+ `savepoint`, so TypeScript refuses a driver an older sqlkit built.
275
+ - `instanceof SqlError` against a sqlkit of 0.2 or older is `false` for an error from
276
+ the driver kv builds over your pool or client. From 0.3.0 on, two copies of sqlkit
277
+ know each other's errors, so the test holds even when yours is not the copy kv
278
+ installs.
279
+ - **`transaction()` inside a transaction you opened on the same `Driver` is undone alone
280
+ when it fails.** Nothing to do. It runs as a savepoint of your transaction, which goes
281
+ on either way; your commit or rollback still decides what stays. In 4.0.0 the
282
+ operations before the failure stayed in your transaction, and a statement Postgres
283
+ refused aborted it. Outside a transaction of yours nothing changes. A `Driver` without
284
+ `savepoint()` (a sqlkit older than 0.3.0) is refused at construction, with a
285
+ `TypeError`.
286
+ - **A sqlkit `Driver` over PGlite is a tested `db` for the postgres adapter.** Nothing to
287
+ do. 4.0.0 said it was expected to work; now every test of the suite runs on it.
288
+
289
+ ## Breaking Changes (4.0.0)
290
+
291
+ No signature changed, and a `pg.Pool` or `pg.Client` you pass today still works. What
292
+ changed is what gets installed and how the PostgreSQL adapter behaves at its edges. The
293
+ memory and Redis adapters do not change: if they are all you use, the bump needs nothing
294
+ from you. Each note says first what you have to do, if anything.
295
+
296
+ **What to install**
297
+
298
+ - **`pg` and `@types/pg` are no longer installed with kv; `@marianmeres/sqlkit` is.** If
299
+ you relied on kv to bring `pg` in, add it to your own `dependencies`. The postgres
300
+ adapter never loaded `pg` itself: it takes the pool or client you built, so an app that
301
+ uses it depends on `pg` already.
302
+
303
+ **What the postgres adapter's `db` takes**
304
+
305
+ - **`db` also takes a sqlkit `Driver` of the `pg` dialect.** Nothing to do; pass one if you
306
+ want your app and kv on one driver. The option's type, `PostgresDb`, is declared by
307
+ shape, and a `pg.Pool` and a `pg.Client` fit it as they are. A driver of another
308
+ dialect is refused at construction, with a `TypeError`.
309
+ - **A `db` that is neither a pool, a client nor a driver is refused at construction, with
310
+ a `TypeError`.** Nothing to do: such a `db` never worked. It used to fail at
311
+ `initialize()`.
312
+ - **The sqlite adapter refuses a sqlkit `Driver` at construction, with a `TypeError`.**
313
+ Pass a `node:sqlite` `DatabaseSync`, as before. A driver never worked there: it used to
314
+ fail at the first operation. Nothing else about the sqlite adapter changes.
315
+
316
+ **Errors**
317
+
318
+ - **A statement Postgres refused rejects with sqlkit's `SqlError`.** If your code tests
319
+ `instanceof` against node-postgres's error class, test for `SqlError` (exported by
320
+ `@marianmeres/sqlkit`) instead. `code` and the message are Postgres's, as before, and
321
+ node-postgres's error is the `cause`. kv's own `TypeError` and `RangeError` are
322
+ unchanged.
323
+ - **The adapter listens for the pool's or client's `error` event.** Nothing to do. An
324
+ idle connection the server dropped is reported through `process.emitWarning` when you
325
+ have no listener of your own. It used to end the process.
326
+
327
+ **One connection, and transactions**
328
+
329
+ - **On one `pg.Client`, calls take turns.** Nothing to do. A call issued while
330
+ `transaction()` runs waits for it to finish. It used to run inside that transaction and
331
+ could be rolled back with it.
332
+ - **A call issued inside a transaction you opened on the same `Driver` joins that
333
+ transaction.** Nothing to do, unless kv's writes must outlive your rollback: then make
334
+ them outside your transaction. Joining is what sharing a driver means.
335
+ - **`transaction()` checks its `COMMIT`.** Nothing to do. A `COMMIT` that did not commit
336
+ rejects, and a connection whose `ROLLBACK` failed is destroyed, not returned to the
337
+ pool.
338
+
339
+ **`keysIter()`**
340
+
341
+ - **On PostgreSQL, `keysIter()` pages by key, 500 at a time, in place of a server-side
342
+ cursor.** If you need a list consistent to one moment, use `keys()`. No connection is
343
+ held between yields, so a write made between two of them no longer waits for the walk
344
+ to end. The walk is no longer one snapshot: a key added or removed during it may or may
345
+ not be seen, as on Redis and SQLite.
346
+
347
+ **JSR**
348
+
349
+ - **The package is no longer published to JSR.** Install from npm. 3.1.1 is the last
350
+ version on JSR and stays there.
351
+
161
352
  ## Breaking Changes (3.0.0)
162
353
 
163
354
  - **The Deno KV adapter is removed.** `"deno-kv"` is no longer an adapter type, and
@@ -174,6 +365,7 @@ const client = createKVClient("my-app-namespace:", "sqlite", {
174
365
  Shipped as a minor release, although the first two items are breaking:
175
366
 
176
367
  **Breaking:**
368
+
177
369
  - **`ttl(key)` now returns a discriminated `TtlResult`** — `{ state: "missing" | "no-ttl" | "expires", at? }` — instead of `Date | null | false`. Migration:
178
370
 
179
371
  ```ts
@@ -194,6 +386,7 @@ Shipped as a minor release, although the first two items are breaking:
194
386
  - **Key validation is on by default.** Non-empty strings, no `\0`, total `namespace + key` length ≤ 512. Set `validateKeys: false` on the adapter options to opt out. Previous behavior silently accepted empty/overlong keys and let the backend error cryptically.
195
387
 
196
388
  **Additive:**
389
+
197
390
  - New methods `setIfAbsent`, `incr`, `decr`, `getSet`, `cas`, `keysIter` on every adapter.
198
391
  - `setMultiple` accepts `{ key, value, ttl? }[]` entries in addition to the legacy `[key, value][]` tuples. Per-pair `ttl` overrides batch `options.ttl`.
199
392
  - Deno KV: new `strictDeleteResult: true` option pre-checks existence so `delete()` returns the real did-it-exist flag (default stays `true` for BC).
@@ -245,15 +438,15 @@ Redis (SQLite runs in memory, nothing to set up). Copy `.env.example` to `.env`
245
438
  in the connection details. Redis keys in the target database are deleted before each
246
439
  test, so point it at a scratch database (the default is `/13`).
247
440
 
248
- | Command | Does |
249
- | ----------------------------------- | -------------------------------------------------- |
250
- | `pnpm install` | installs dependencies and dev tooling |
251
- | `pnpm test` / `pnpm run test:watch` | `node --test` over `tests/**/*.test.ts` |
252
- | `pnpm run typecheck` | `tsc --noEmit` over `src` and `tests` |
253
- | `pnpm run lint` / `pnpm run format` | `biome check .` read-only / with `--write` |
254
- | `pnpm run ci` | typecheck + lint + test, what CI runs |
255
- | `pnpm run build` | `tsc -p tsconfig.build.json` → `dist/` |
256
- | `pnpm run rp` / `pnpm run rpm` | patch / minor release, then publish to npm and JSR |
441
+ | Command | Does |
442
+ | ----------------------------------- | ------------------------------------------ |
443
+ | `pnpm install` | installs dependencies and dev tooling |
444
+ | `pnpm test` / `pnpm run test:watch` | `node --test` over `tests/**/*.test.ts` |
445
+ | `pnpm run typecheck` | `tsc --noEmit` over `src` and `tests` |
446
+ | `pnpm run lint` / `pnpm run format` | `biome check .` read-only / with `--write` |
447
+ | `pnpm run ci` | typecheck + lint + test, what CI runs |
448
+ | `pnpm run build` | `tsc -p tsconfig.build.json` → `dist/` |
449
+ | `pnpm run rp` / `pnpm run rpm` | patch / minor release, then publish to npm |
257
450
 
258
451
  ## License
259
452
 
@@ -3,14 +3,17 @@
3
3
  *
4
4
  * PostgreSQL key-value storage adapter implementation.
5
5
  */
6
- import type pg from "pg";
6
+ import { type PostgresDb } from "../utils/driver.ts";
7
7
  import { AdapterAbstract, type AdapterAbstractOptions, type Operation, type SetMultipleEntry, type SetOptions, type TtlResult } from "./abstract.ts";
8
8
  /**
9
9
  * Configuration options for the PostgreSQL KV adapter.
10
10
  */
11
11
  export interface AdapterPostgresOptions extends AdapterAbstractOptions {
12
- /** PostgreSQL connection instance - either `pg.Pool` or `pg.Client`. */
13
- db: pg.Pool | pg.Client;
12
+ /**
13
+ * The database: a node-postgres `pg.Pool`, a connected `pg.Client`, or a sqlkit
14
+ * `Driver` of the `pg` dialect. The caller owns it: the adapter never closes it.
15
+ */
16
+ db: PostgresDb;
14
17
  /**
15
18
  * Name of the table to use for storing key-value pairs.
16
19
  * The table will be created automatically if it doesn't exist.