@marianmeres/kv 3.1.1 → 4.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/AGENTS.md CHANGED
@@ -5,12 +5,10 @@ Machine-friendly reference for AI agents working with this codebase.
5
5
  ## Quick Reference
6
6
 
7
7
  - **Stack**: TypeScript (erasable-only), Node LTS (`.node-version`), pnpm, `node:test`, Biome.
8
- - **Install**: `pnpm install` | **Test**: `pnpm test` (needs Postgres + Redis; SQLite runs in memory) |
8
+ - **Install**: `pnpm install` | **Test**: `pnpm test` (needs Postgres + Redis; SQLite runs in memory;
9
+ test files run one at a time) |
9
10
  **Check all**: `pnpm run ci` | **Build**: `pnpm run build`
10
- - **Published**: npm (`dist/`, built by `tsc`) and JSR (`src/` via `jsr.json`, for Deno consumers)
11
- - **JSR imports**: `jsr.json` maps `@marianmeres/clog` to `jsr:` so Deno apps get one copy;
12
- keep its range equal to `package.json`'s. `pg` and `redis` resolve from `package.json` as
13
- `npm:`. Only `deno publish` reads `jsr.json`: `test:deno` still runs against npm clog.
11
+ - **Published**: npm (`dist/`, built by `tsc`)
14
12
 
15
13
  ## Purpose
16
14
 
@@ -20,7 +18,7 @@ Key-value storage abstraction layer providing a unified Redis-inspired API acros
20
18
 
21
19
  ```
22
20
  src/
23
- ├── index.ts — The only entry point (package.json "exports", jsr.json "exports")
21
+ ├── index.ts — The only entry point (package.json "exports")
24
22
  ├── kv.ts — Factory function and all exports
25
23
  ├── adapter/
26
24
  │ ├── abstract.ts — Base class + shared types
@@ -29,16 +27,21 @@ src/
29
27
  │ ├── postgres.ts — PostgreSQL adapter
30
28
  │ └── sqlite.ts — SQLite adapter (`node:sqlite`)
31
29
  └── utils/
30
+ ├── driver.ts — `toPgDriver()`: the postgres adapter's `db` as a sqlkit `Driver`
32
31
  └── sleep.ts — Sleep utility function
33
32
 
34
33
  tests/
35
34
  ├── kv.test.ts — Main test file (node:test); every test runs against all adapters
35
+ ├── driver.test.ts — The postgres adapter on a pool, a client and a sqlkit Driver
36
36
  ├── _tests-runner.ts — Test harness: builds one client per adapter per test
37
37
  ├── _redis.ts — Redis test helper
38
38
  ├── _pg.ts — PostgreSQL test helper
39
39
  ├── _sqlite.ts — SQLite test helper (fresh `:memory:` database per test)
40
40
  └── _no-destroy-child.ts — child process: an adapter left running must not hold it open
41
41
 
42
+ docs/
43
+ └── backlog.md — Open work, ranked, not scheduled (the open copy)
44
+
42
45
  dist/ — `tsc -p tsconfig.build.json` output; gitignored; the npm payload
43
46
  ```
44
47
 
@@ -46,8 +49,8 @@ dist/ — `tsc -p tsconfig.build.json` output; gitignored; the
46
49
 
47
50
  1. **Runtime policy** — Node at the keyboard and in production, npm as the registry. No
48
51
  `Deno.*`, `Bun.*`, `bun:*`, `jsr:` or `npm:` specifiers. Web standards first; `node:*`
49
- where they don't reach. Nothing is built, tested or documented for Bun or Deno; JSR is
50
- published only while Deno consumers exist (`publish:jsr` runs `test:deno` first).
52
+ where they don't reach. Nothing is built, tested or documented for Bun or Deno.
53
+ Published to npm only; 3.1.1 was the last version on JSR.
51
54
  2. **Erasable TypeScript** — no `enum`, `namespace`, constructor parameter properties;
52
55
  `import type` for types; explicit `.ts` import extensions (rewritten to `.js` on emit).
53
56
  Node strips types and never checks them: `pnpm run typecheck` is the only type gate.
@@ -64,32 +67,47 @@ dist/ — `tsc -p tsconfig.build.json` output; gitignored; the
64
67
  8. Pattern matching: Redis-style (`*` = any chars, `?` = single char). All other characters — including `.`, `(`, `[`, `%`, `_` — are matched **literally**.
65
68
  9. TTL resolution: `options.ttl ?? defaultTtl`; `0` or negative disables expiration for that call (explicitly overrides `defaultTtl`).
66
69
  10. Shared helpers live in `abstract.ts` — use `_globToRegex()`, `_resolveTtl()`, `_normalizePairs()`; do not re-implement per-adapter. The SQL adapters keep their own pattern escaping next to the SQL that uses it: Postgres `#likePattern` (LIKE, `ESCAPE '\'`), SQLite `#globPattern` (GLOB — case-sensitive, unlike SQLite's LIKE; `[` → `[[]`). Both escape the namespace as a literal prefix.
67
- 11. Postgres `transaction()` must pin a single connection via `pool.connect()`; every nested query uses that client, not `this.options.db`.
68
- 12. Redis adapter owns the connection iff `initialize()` opened it (`#weOpened` flag). The Postgres and SQLite adapters never close the `db` they were given.
70
+ 11. Postgres `transaction()` is `this.#db.transaction()`: the driver pins the connection
71
+ (or, on one `pg.Client`, holds every other caller back), checks the `COMMIT` and rolls
72
+ back. Nothing in the adapter sends `BEGIN`, `COMMIT` or `ROLLBACK`. Pass the callback's
73
+ `tx` to the `…On` helpers.
74
+ 12. Redis adapter owns the connection iff `initialize()` opened it (`#weOpened` flag). The Postgres and SQLite adapters never close the `db` they were given; the Postgres adapter never calls `close()` on a driver, its own or the host's (it would end the host's pool or client).
69
75
  13. Key validation lives in `_withNs()` — enforced when `options.validateKeys !== false`. Reject empty strings, keys with `\0`, and total `namespace + key` length > 512.
70
76
  14. `ttl()` returns `TtlResult = { state: "missing" | "no-ttl" | "expires"; at? }` — discriminated union, not `Date | null | false`.
71
77
  15. `incr`/`decr`/`getSet`/`cas`/`setIfAbsent` must be atomic: Redis = Lua or NX; Postgres = UPSERT/UPDATE with RETURNING; SQLite = a single UPSERT/UPDATE (`setIfAbsent`, `cas`) or a read-modify-write inside `#atomic()` (`getSet`, `incr`/`decr`); Memory = plain read-modify-write (single-threaded).
72
- 16. Non-numeric `incr`/`decr` target throws `TypeError("KV value is not a number")`. In Postgres, detect via error code `22P02`; SQLite checks `typeof` in JS, like Memory.
73
- 17. `keysIter` must clean up on early consumer break — PG releases its server-side cursor in the generator's `finally`. SQLite pages by keyset (`key > last`), so nothing stays open between yields.
78
+ 16. Non-numeric `incr`/`decr` target throws `TypeError("KV value is not a number")`. In Postgres, detect via the `22P02` in the sqlkit `SqlError`'s `code`; SQLite checks `typeof` in JS, like Memory. No other database error is caught, wrapped or translated: the caller gets sqlkit's `SqlError`.
79
+ 17. `keysIter` in both SQL adapters pages by keyset, 500 keys a page, so nothing stays open between yields and an early consumer break leaves nothing to release. It is not one snapshot. SQLite pages with `key > last`. Postgres pages in byte order on the `text_pattern_ops` prefix index (`key ~>~ $2`, `ORDER BY key USING ~<~`); a plain `key > $2 ORDER BY key` would walk the primary key across every namespace. Postgres `keys()` has no SQL `ORDER BY` for the same reason: it sorts in JS.
74
80
  18. SQLite: `src/` imports `node:sqlite` **type-only** — the caller passes the `DatabaseSync`. A runtime import would make every consumer of the package load `node:sqlite`.
75
81
  19. SQLite multi-statement operations go through `#atomic()`: `BEGIN IMMEDIATE`, or a savepoint when `db.isTransaction` (the caller's transaction on the same handle). The callback must stay synchronous — an `await` inside would let other work join the transaction.
76
82
  20. SQLite stores `value` as JSON text and `expires_at` as epoch ms (INTEGER), compared against a bound `Date.now()` — never SQLite's own clock. Read counts/integers through `Number()` (a handle opened with `readBigInts` returns bigints).
77
83
  21. TTL cleanup timers are `unref()`ed (`unref?.()`: browser and worker timers are plain numbers), so a client nobody destroyed cannot hold the process open; `destroy()` still clears them. Postgres's tick awaits its query, so it re-checks `_initialized` before re-arming, and `destroy()` clears `_initialized` before its first `await`. SQLite stops the cleanup (one error log) when it finds `db.isOpen` false.
84
+ 22. Postgres talks to the database only through `src/utils/driver.ts`: `toPgDriver(db)`,
85
+ called once in the constructor, turns a `pg.Pool` or `pg.Client` into a sqlkit
86
+ `Driver` (one per object, in a `WeakMap`: `createPgDriver` adds an `error` listener on
87
+ every call, and several adapters on one pool is normal), and uses a host's `Driver` as
88
+ it is. A `Driver` of another dialect is a `TypeError`. `options.db` stays what the host
89
+ passed; every statement goes through `this.#db`, with its text and `$n` parameters.
90
+ Nothing in `src/` imports `pg` (it is a dev dependency); `@marianmeres/sqlkit/pg` is
91
+ imported statically, so the package root loads `node:async_hooks`. A kv call issued
92
+ inside a transaction the host opened on the same `Driver` joins it, by design.
93
+ 23. The SQLite adapter takes only a `DatabaseSync`: its constructor refuses a sqlkit
94
+ `Driver` with a `TypeError`.
78
95
 
79
96
  ## Before Making Changes
80
97
 
81
98
  - [ ] Check existing patterns in adapter implementations
82
- - [ ] Add or extend a test in `tests/kv.test.ts`, then run `pnpm run ci`
99
+ - [ ] Add or extend a test in `tests/kv.test.ts` (or in `tests/driver.test.ts` for what only
100
+ the postgres adapter's `db` decides), then run `pnpm run ci`
83
101
  - [ ] Verify adapter limitations are documented (README feature matrix, API.md)
84
102
 
85
103
  ## Supported Adapters
86
104
 
87
- | Adapter | Type String | Required Options | Limitations |
88
- | ---------- | ------------ | --------------------- | --------------------------------------------------------------------------- |
89
- | Memory | `"memory"` | None | Not persisted |
90
- | Redis | `"redis"` | `db` (Redis client) | Namespace required; `keys()`/`clear()` unavailable in cluster mode |
91
- | PostgreSQL | `"postgres"` | `db` (pg.Pool/Client) | `tableName` limited to `[A-Za-z0-9_]` + optional `schema.` prefix |
92
- | SQLite | `"sqlite"` | `db` (`DatabaseSync`) | `tableName` limited to `[A-Za-z0-9_]`, no schema prefix; synchronous driver |
105
+ | Adapter | Type String | Required Options | Limitations |
106
+ | ---------- | ------------ | -------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
107
+ | Memory | `"memory"` | None | Not persisted |
108
+ | Redis | `"redis"` | `db` (Redis client) | Namespace required; `keys()`/`clear()` unavailable in cluster mode |
109
+ | PostgreSQL | `"postgres"` | `db` (`pg.Pool`, `pg.Client` or a sqlkit `Driver`) | `tableName` limited to `[A-Za-z0-9_]` + optional `schema.` prefix |
110
+ | SQLite | `"sqlite"` | `db` (`DatabaseSync`) | `tableName` limited to `[A-Za-z0-9_]`, no schema prefix; synchronous driver; a sqlkit `Driver` is refused |
93
111
 
94
112
  ## Core API
95
113
 
@@ -156,6 +174,9 @@ interface Operation {
156
174
 
157
175
  type SetMultipleEntry = [string, any] | { key: string; value: any; ttl?: number };
158
176
 
177
+ // the postgres adapter's `db`; sqlkit's structural types, so `src/` never imports `pg`
178
+ type PostgresDb = PgPoolLike | PgStandaloneClientLike | Driver;
179
+
159
180
  type TtlResult =
160
181
  | { state: "missing" }
161
182
  | { state: "no-ttl" }
@@ -165,22 +186,23 @@ type TtlResult =
165
186
  ## Commands
166
187
 
167
188
  ```bash
168
- pnpm test # node --test over tests/**/*.test.ts, loads .env if present
189
+ pnpm test # node --test over tests/**/*.test.ts, one file at a time
190
+ # (--test-concurrency=1: they share `__kv` and one Redis DB), loads .env
169
191
  pnpm run test:watch
170
192
  pnpm run typecheck # tsc --noEmit
171
193
  pnpm run lint # biome check . (read-only); `pnpm run format` writes
172
194
  pnpm run ci # typecheck + lint + test — what CI runs
173
195
  pnpm run build # tsc -p tsconfig.build.json → dist/
174
- pnpm run test:deno # the same suite on Deno; only gates publish:jsr
175
196
  ```
176
197
 
177
- Releasing (`pnpm run rp` / `rpm`, or `pnpm run release major` + `pnpm run publish:all`) is a
178
- human action.
198
+ Releasing (`pnpm run rp` / `rpm`, or `pnpm run release major` + `pnpm publish`) is a human
199
+ action.
179
200
 
180
201
  ## Test Environment Variables
181
202
 
182
- Every test runs against all four adapters, so tests need a live Postgres and Redis (SQLite
183
- uses a fresh `:memory:` database per test and needs nothing). Put
203
+ Every test in `tests/kv.test.ts` runs against all four adapters, so tests need a live
204
+ Postgres and Redis (SQLite uses a fresh `:memory:` database per test and needs nothing).
205
+ `tests/driver.test.ts` needs Postgres only. Put
184
206
  these in `.env` (see `.env.example`); CI provides them via service containers
185
207
  (`.github/workflows/ci.yml`). The harness deletes **every key** in the Redis DB before each
186
208
  test — use a scratch DB.
@@ -209,3 +231,4 @@ To add a new adapter:
209
231
 
210
232
  - [API Reference](./API.md) — Complete API documentation
211
233
  - [README](./README.md) — Quick start, feature matrix, breaking changes, development
234
+ - [Backlog](./docs/backlog.md) — Open work after 4.0.0, with each row's trigger
package/API.md CHANGED
@@ -43,6 +43,7 @@ function createKVClient<T extends keyof KnownTypes>(
43
43
  - `TypeError` - If namespace doesn't end with `:` (and is not empty)
44
44
  - `TypeError` - If an unsupported adapter type is specified
45
45
  - `Error` - If required options (like `db`) are missing for the adapter
46
+ - `TypeError` - If `db` is not of a kind the adapter takes (see its options below)
46
47
 
47
48
  **Example:**
48
49
 
@@ -64,6 +65,9 @@ import pg from "pg";
64
65
  const pool = new pg.Pool({ connectionString: "..." });
65
66
  const postgres = createKVClient("myapp:", "postgres", { db: pool });
66
67
  await postgres.initialize();
68
+ // ...or on the sqlkit Driver your app already shares
69
+ import { createPgDriver } from "@marianmeres/sqlkit/pg";
70
+ const shared = createKVClient("myapp:", "postgres", { db: createPgDriver(pool) });
67
71
 
68
72
  // SQLite storage (node:sqlite ships with Node)
69
73
  import { DatabaseSync } from "node:sqlite";
@@ -456,11 +460,12 @@ Iterates matching keys without materializing the full list. Prefer this over `ke
456
460
  keysIter(pattern: string): AsyncIterable<string>
457
461
  ```
458
462
 
459
- Ordering is not guaranteed across adapters — use `keys()` if you need sorted output.
463
+ Ordering is not guaranteed across adapters — use `keys()` if you need sorted output. The
464
+ walk is not one snapshot: a key added or removed while it runs may or may not be seen.
460
465
 
461
466
  **Per-adapter behavior:**
462
467
  - Redis: wraps `SCAN` (non-blocking).
463
- - PostgreSQL: uses a server-side cursor (`DECLARE CURSOR`) inside a short transaction. Early break releases the cursor and commits.
468
+ - PostgreSQL: keyset pages of 500 keys, in byte order on the prefix index `initialize()` creates. Nothing stays open between pages, so an early break has nothing to release and a write between two yields does not wait for the walk.
464
469
  - SQLite: keyset pages of 500 keys (`key > last ORDER BY key`). Nothing stays open between pages, so an early break has nothing to release.
465
470
  - Memory: yields from the internal `Map`.
466
471
 
@@ -468,7 +473,7 @@ Ordering is not guaranteed across adapters — use `keys()` if you need sorted o
468
473
 
469
474
  ```typescript
470
475
  for await (const k of client.keysIter("user:*")) {
471
- if (someCondition(k)) break; // safe — cursor is released
476
+ if (someCondition(k)) break; // safe — nothing is left open
472
477
  }
473
478
  ```
474
479
 
@@ -585,12 +590,12 @@ transaction(operations: Operation[]): Promise<any[]>
585
590
 
586
591
  Atomicity per adapter:
587
592
 
588
- | Adapter | Atomic | Notes |
589
- | ---------- | :----: | -------------------------------------------------------------------- |
590
- | Memory | ✓ | Single-threaded by nature |
591
- | Redis | ✓ | Uses `MULTI`/`EXEC` |
592
- | PostgreSQL | ✓ | Pins a single connection for `BEGIN`/…/`COMMIT`, even on a `pg.Pool` |
593
- | SQLite | ✓ | `BEGIN IMMEDIATE`; a savepoint inside a caller-opened transaction |
593
+ | Adapter | Atomic | Notes |
594
+ | ---------- | :----: | ------------------------------------------------------------------------------------ |
595
+ | Memory | ✓ | Single-threaded by nature |
596
+ | Redis | ✓ | Uses `MULTI`/`EXEC` |
597
+ | PostgreSQL | ✓ | One connection for the whole unit: pinned on a `pg.Pool`, in turn on one `pg.Client` |
598
+ | SQLite | ✓ | `BEGIN IMMEDIATE`; a savepoint inside a caller-opened transaction |
594
599
 
595
600
  Per-operation `options.ttl` is honored by **all** adapters (Redis, PostgreSQL, SQLite, Memory).
596
601
 
@@ -708,6 +713,27 @@ interface Operation {
708
713
 
709
714
  ---
710
715
 
716
+ ### `PostgresDb`
717
+
718
+ What the postgres adapter's `db` option takes.
719
+
720
+ ```typescript
721
+ import type { Driver } from "@marianmeres/sqlkit";
722
+ import type { PgPoolLike, PgStandaloneClientLike } from "@marianmeres/sqlkit/pg";
723
+
724
+ type PostgresDb = PgPoolLike | PgStandaloneClientLike | Driver;
725
+ ```
726
+
727
+ - `PgPoolLike` — a node-postgres `pg.Pool`.
728
+ - `PgStandaloneClientLike` — a connected node-postgres `pg.Client`.
729
+ - `Driver` — a sqlkit driver of the `pg` dialect, e.g. `createPgDriver(pool)` from
730
+ `@marianmeres/sqlkit/pg`.
731
+
732
+ The first two are declared by shape, so kv imports nothing from `pg`, and a `pg.Pool` and
733
+ a `pg.Client` fit them as they are. See [`AdapterPostgresOptions`](#adapterpostgresoptions).
734
+
735
+ ---
736
+
711
737
  ## Adapter-Specific Options
712
738
 
713
739
  ### `AdapterMemoryOptions`
@@ -757,8 +783,11 @@ interface AdapterRedisOptions extends AdapterAbstractOptions {
757
783
 
758
784
  ```typescript
759
785
  interface AdapterPostgresOptions extends AdapterAbstractOptions {
760
- /** PostgreSQL connection instance - either `pg.Pool` or `pg.Client`. */
761
- db: pg.Pool | pg.Client;
786
+ /**
787
+ * The database: a node-postgres `pg.Pool`, a connected `pg.Client`, or a sqlkit
788
+ * `Driver` of the `pg` dialect. The caller owns it: the adapter never closes it.
789
+ */
790
+ db: PostgresDb;
762
791
  /**
763
792
  * Name of the table to use for storing key-value pairs.
764
793
  * The table will be created automatically if it doesn't exist.
@@ -777,8 +806,45 @@ interface AdapterPostgresOptions extends AdapterAbstractOptions {
777
806
  }
778
807
  ```
779
808
 
780
- **Transactions on a Pool:**
781
- `transaction()` checks out a single client from the pool (via `pool.connect()`) for the entire `BEGIN`/`COMMIT`/`ROLLBACK` block, so atomicity holds even when multiple transactions run concurrently against the same `pg.Pool`. On a `pg.Client`, no checkout happens.
809
+ `db` is a [`PostgresDb`](#postgresdb): pass a `pg.Pool` or a connected `pg.Client` as it
810
+ is, or a [`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit)
811
+ `Driver` your app built. The constructor refuses, with a `TypeError`, a `Driver` of
812
+ another dialect and anything that is neither a pool, a client nor a driver.
813
+
814
+ **Connection ownership:** the adapter never closes what it was given — not the pool, not
815
+ the client, not the driver — so `destroy()` leaves it open and several adapters can share
816
+ one.
817
+
818
+ **Transactions and connections:** every statement runs on a sqlkit `Driver`. The adapter
819
+ builds one over a `pg.Pool` or a `pg.Client`, one per pool or client object however many
820
+ adapters share it, and uses a `Driver` you pass as it is. `transaction()` runs on one
821
+ connection for the whole unit, and a `COMMIT` that did not commit rejects:
822
+
823
+ - **On a `pg.Pool`**, it pins one connection for its length; other calls run on other
824
+ connections at once. A connection whose `ROLLBACK` failed is destroyed, not returned to
825
+ the pool.
826
+ - **On one `pg.Client`**, calls take turns: one issued while a transaction runs waits for
827
+ it to finish, and never becomes part of it.
828
+ - **On a `Driver` you pass**, the same holds for the pool or client under it. In addition,
829
+ a kv call issued inside a transaction you opened on that same driver
830
+ (`driver.transaction(async () => { await kv.set(…) })`) joins your transaction and
831
+ commits or rolls back with it. Make the call outside your transaction if it must
832
+ outlive your rollback.
833
+
834
+ The adapter listens for the pool's or client's `error` event, so an idle connection the
835
+ server dropped does not end the process: it is reported through `process.emitWarning`
836
+ when you have no listener of your own.
837
+
838
+ A `Driver` over PGlite (`createPgliteDriver` from `@marianmeres/sqlkit/pglite`) is of the
839
+ `pg` dialect and is expected to work; it is not part of the test suite yet.
840
+
841
+ **Errors:** a statement the database refused rejects with sqlkit's `SqlError` (exported
842
+ by `@marianmeres/sqlkit`), unwrapped and untranslated. Its `code` is Postgres's SQLSTATE
843
+ (e.g. `22P05`), its message is Postgres's, and its `cause` is the driver library's own
844
+ error (node-postgres's, on a pool or a client). The one database error the adapter reads
845
+ is a non-numeric `incr`/`decr` target (`22P02`), which it turns into
846
+ `TypeError("KV value is not a number")`, as every adapter does. kv's own `TypeError` and
847
+ `RangeError`, for an invalid key or namespace, are unchanged.
782
848
 
783
849
  **Table Schema:**
784
850
  - `key` (VARCHAR 512) - Primary key
@@ -814,6 +880,9 @@ interface AdapterSqliteOptions extends AdapterAbstractOptions {
814
880
  }
815
881
  ```
816
882
 
883
+ A sqlkit `Driver` (e.g. from `openSqlite()`) is refused at construction with a
884
+ `TypeError`: the adapter takes the `DatabaseSync` itself.
885
+
817
886
  **Connection ownership:** the adapter never opens, closes or reconfigures the handle — it
818
887
  sets no pragmas. Call `destroy()` before you close the handle: if the TTL cleanup finds it
819
888
  closed, it logs one error and stops. For several processes on one file, open it with a busy
package/README.md CHANGED
@@ -1,7 +1,6 @@
1
1
  # @marianmeres/kv
2
2
 
3
3
  [![NPM version](https://img.shields.io/npm/v/@marianmeres/kv.svg)](https://www.npmjs.com/package/@marianmeres/kv)
4
- [![JSR version](https://jsr.io/badges/@marianmeres/kv)](https://jsr.io/@marianmeres/kv)
5
4
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
6
5
 
7
6
  Key-value storage abstraction layer with support for multiple backend adapters:
@@ -16,11 +15,15 @@ The API is inspired by the Redis API.
16
15
 
17
16
  ```sh
18
17
  npm install @marianmeres/kv # or pnpm add
19
- deno add jsr:@marianmeres/kv # Deno
20
18
  ```
21
19
 
22
- The `redis`, `postgres` and `sqlite` adapters take a client you create yourself (`redis`,
23
- `pg`, or a `node:sqlite` `DatabaseSync`, which ships with Node). Requires Node >= 24.12.
20
+ Versions up to 3.1.1 are also on JSR. From 4.0 the package is published to npm only.
21
+
22
+ The `redis`, `postgres` and `sqlite` adapters take a client you create yourself: a
23
+ `redis` client; a `pg.Pool`, a `pg.Client` or a
24
+ [`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit) `Driver`; or a
25
+ `node:sqlite` `DatabaseSync`, which ships with Node. kv does not install `pg`: an app that
26
+ uses the postgres adapter has it already. Requires Node >= 24.12.
24
27
 
25
28
  ## Usage
26
29
 
@@ -98,20 +101,20 @@ type TtlResult =
98
101
 
99
102
  ## Adapter Feature Matrix
100
103
 
101
- | Feature | Memory | Redis | PostgreSQL | SQLite |
102
- | --------------------------------------------------------- | :-----------------: | :--------------------: | :--------------: | :-----------------------: |
103
- | Persistent | ✗ | ✓ | ✓ | ✓ (file database) |
104
- | TTL on `set()` | ✓ | ✓ | ✓ | ✓ |
105
- | `ttl()` → `expires`/`no-ttl`/`missing` | ✓ | ✓ | ✓ | ✓ |
106
- | `expire()` | ✓ | ✓ | ✓ | ✓ |
107
- | Atomic `setIfAbsent` / `incr` / `decr` / `cas` / `getSet` | ✓ | ✓ (Lua) | ✓ (UPSERT) | ✓ (UPSERT / IMMEDIATE tx) |
108
- | Sorted `keys()` | ✓ | ✓ | ✓ | ✓ |
109
- | Streaming `keysIter()` | ✓ | ✓ (SCAN) | ✓ (cursor) | ✓ (keyset paging) |
110
- | Per-pair TTL in `setMultiple` | ✓ | ✓ | ✓ | ✓ |
111
- | Atomic `transaction()` | ✓ (single-threaded) | ✓ (MULTI) | ✓ (BEGIN/COMMIT) | ✓ (BEGIN IMMEDIATE) |
112
- | `delete()` returns real existed-flag | ✓ | ✓ | ✓ | ✓ |
113
- | Background TTL cleanup | optional | native (Redis does it) | optional | optional |
114
- | `keys()` / `clear()` | ✓ | ✓ (non-cluster only) | ✓ | ✓ |
104
+ | Feature | Memory | Redis | PostgreSQL | SQLite |
105
+ | --------------------------------------------------------- | :-----------------: | :--------------------: | :---------------: | :-----------------------: |
106
+ | Persistent | ✗ | ✓ | ✓ | ✓ (file database) |
107
+ | TTL on `set()` | ✓ | ✓ | ✓ | ✓ |
108
+ | `ttl()` → `expires`/`no-ttl`/`missing` | ✓ | ✓ | ✓ | ✓ |
109
+ | `expire()` | ✓ | ✓ | ✓ | ✓ |
110
+ | Atomic `setIfAbsent` / `incr` / `decr` / `cas` / `getSet` | ✓ | ✓ (Lua) | ✓ (UPSERT) | ✓ (UPSERT / IMMEDIATE tx) |
111
+ | Sorted `keys()` | ✓ | ✓ | ✓ | ✓ |
112
+ | Streaming `keysIter()` | ✓ | ✓ (SCAN) | ✓ (keyset paging) | ✓ (keyset paging) |
113
+ | Per-pair TTL in `setMultiple` | ✓ | ✓ | ✓ | ✓ |
114
+ | Atomic `transaction()` | ✓ (single-threaded) | ✓ (MULTI) | ✓ (BEGIN/COMMIT) | ✓ (BEGIN IMMEDIATE) |
115
+ | `delete()` returns real existed-flag | ✓ | ✓ | ✓ | ✓ |
116
+ | Background TTL cleanup | optional | native (Redis does it) | optional | optional |
117
+ | `keys()` / `clear()` | ✓ | ✓ (non-cluster only) | ✓ | ✓ |
115
118
 
116
119
  ## Adapter-Specific Limitations
117
120
 
@@ -123,9 +126,31 @@ type TtlResult =
123
126
 
124
127
  ### PostgreSQL
125
128
 
129
+ `db` takes a `pg.Pool`, a connected `pg.Client`, or a `Driver` from
130
+ [`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit) of the `pg`
131
+ dialect. An app that already builds one driver for everything that talks SQL passes it
132
+ here too:
133
+
134
+ ```typescript
135
+ import { createPgDriver } from "@marianmeres/sqlkit/pg";
136
+ import pg from "pg";
137
+
138
+ const db = createPgDriver(new pg.Pool({ connectionString: process.env.DATABASE_URL }));
139
+ const client = createKVClient("my-app-namespace:", "postgres", { db });
140
+ ```
141
+
142
+ - **The adapter never closes what it was given.** `destroy()` leaves the pool, the client
143
+ or the driver open; ending it is yours. Several adapters can share one.
144
+ - **`transaction()` runs on one connection.** On a `pg.Pool` it pins one for the whole
145
+ unit. On one `pg.Client` the calls take turns: another call waits for the transaction
146
+ to finish.
147
+ - **A call made inside a transaction you opened on the same `Driver` joins it**, and
148
+ commits or rolls back with it.
149
+ - **A statement the database refused rejects with sqlkit's `SqlError`**: it carries
150
+ Postgres's `code` and message, with node-postgres's error as `cause`. kv's own
151
+ `TypeError` and `RangeError` are unchanged.
126
152
  - Creates a table (default: `__kv`) in your database.
127
153
  - `tableName` must match `^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)?$` (word chars plus an optional `schema.` prefix).
128
- - `transaction()` pins a single connection from the Pool for the whole BEGIN/…/COMMIT block — safe to call against a `pg.Pool`.
129
154
  - Supports optional TTL cleanup via `ttlCleanupIntervalSec` option.
130
155
 
131
156
  ### SQLite
@@ -158,6 +183,69 @@ const client = createKVClient("my-app-namespace:", "sqlite", {
158
183
  - Data is not persisted (in-memory only).
159
184
  - Supports optional TTL cleanup via `ttlCleanupIntervalSec` option.
160
185
 
186
+ ## Breaking Changes (4.0.0)
187
+
188
+ No signature changed, and a `pg.Pool` or `pg.Client` you pass today still works. What
189
+ changed is what gets installed and how the PostgreSQL adapter behaves at its edges. The
190
+ memory and Redis adapters do not change: if they are all you use, the bump needs nothing
191
+ from you. Each note says first what you have to do, if anything.
192
+
193
+ **What to install**
194
+
195
+ - **`pg` and `@types/pg` are no longer installed with kv; `@marianmeres/sqlkit` is.** If
196
+ you relied on kv to bring `pg` in, add it to your own `dependencies`. The postgres
197
+ adapter never loaded `pg` itself: it takes the pool or client you built, so an app that
198
+ uses it depends on `pg` already.
199
+
200
+ **What the postgres adapter's `db` takes**
201
+
202
+ - **`db` also takes a sqlkit `Driver` of the `pg` dialect.** Nothing to do; pass one if you
203
+ want your app and kv on one driver. The option's type, `PostgresDb`, is declared by
204
+ shape, and a `pg.Pool` and a `pg.Client` fit it as they are. A driver of another
205
+ dialect is refused at construction, with a `TypeError`.
206
+ - **A `db` that is neither a pool, a client nor a driver is refused at construction, with
207
+ a `TypeError`.** Nothing to do: such a `db` never worked. It used to fail at
208
+ `initialize()`.
209
+ - **The sqlite adapter refuses a sqlkit `Driver` at construction, with a `TypeError`.**
210
+ Pass a `node:sqlite` `DatabaseSync`, as before. A driver never worked there: it used to
211
+ fail at the first operation. Nothing else about the sqlite adapter changes.
212
+
213
+ **Errors**
214
+
215
+ - **A statement Postgres refused rejects with sqlkit's `SqlError`.** If your code tests
216
+ `instanceof` against node-postgres's error class, test for `SqlError` (exported by
217
+ `@marianmeres/sqlkit`) instead. `code` and the message are Postgres's, as before, and
218
+ node-postgres's error is the `cause`. kv's own `TypeError` and `RangeError` are
219
+ unchanged.
220
+ - **The adapter listens for the pool's or client's `error` event.** Nothing to do. An
221
+ idle connection the server dropped is reported through `process.emitWarning` when you
222
+ have no listener of your own. It used to end the process.
223
+
224
+ **One connection, and transactions**
225
+
226
+ - **On one `pg.Client`, calls take turns.** Nothing to do. A call issued while
227
+ `transaction()` runs waits for it to finish. It used to run inside that transaction and
228
+ could be rolled back with it.
229
+ - **A call issued inside a transaction you opened on the same `Driver` joins that
230
+ transaction.** Nothing to do, unless kv's writes must outlive your rollback: then make
231
+ them outside your transaction. Joining is what sharing a driver means.
232
+ - **`transaction()` checks its `COMMIT`.** Nothing to do. A `COMMIT` that did not commit
233
+ rejects, and a connection whose `ROLLBACK` failed is destroyed, not returned to the
234
+ pool.
235
+
236
+ **`keysIter()`**
237
+
238
+ - **On PostgreSQL, `keysIter()` pages by key, 500 at a time, in place of a server-side
239
+ cursor.** If you need a list consistent to one moment, use `keys()`. No connection is
240
+ held between yields, so a write made between two of them no longer waits for the walk
241
+ to end. The walk is no longer one snapshot: a key added or removed during it may or may
242
+ not be seen, as on Redis and SQLite.
243
+
244
+ **JSR**
245
+
246
+ - **The package is no longer published to JSR.** Install from npm. 3.1.1 is the last
247
+ version on JSR and stays there.
248
+
161
249
  ## Breaking Changes (3.0.0)
162
250
 
163
251
  - **The Deno KV adapter is removed.** `"deno-kv"` is no longer an adapter type, and
@@ -245,15 +333,15 @@ Redis (SQLite runs in memory, nothing to set up). Copy `.env.example` to `.env`
245
333
  in the connection details. Redis keys in the target database are deleted before each
246
334
  test, so point it at a scratch database (the default is `/13`).
247
335
 
248
- | Command | Does |
249
- | ----------------------------------- | -------------------------------------------------- |
250
- | `pnpm install` | installs dependencies and dev tooling |
251
- | `pnpm test` / `pnpm run test:watch` | `node --test` over `tests/**/*.test.ts` |
252
- | `pnpm run typecheck` | `tsc --noEmit` over `src` and `tests` |
253
- | `pnpm run lint` / `pnpm run format` | `biome check .` read-only / with `--write` |
254
- | `pnpm run ci` | typecheck + lint + test, what CI runs |
255
- | `pnpm run build` | `tsc -p tsconfig.build.json` → `dist/` |
256
- | `pnpm run rp` / `pnpm run rpm` | patch / minor release, then publish to npm and JSR |
336
+ | Command | Does |
337
+ | ----------------------------------- | ------------------------------------------ |
338
+ | `pnpm install` | installs dependencies and dev tooling |
339
+ | `pnpm test` / `pnpm run test:watch` | `node --test` over `tests/**/*.test.ts` |
340
+ | `pnpm run typecheck` | `tsc --noEmit` over `src` and `tests` |
341
+ | `pnpm run lint` / `pnpm run format` | `biome check .` read-only / with `--write` |
342
+ | `pnpm run ci` | typecheck + lint + test, what CI runs |
343
+ | `pnpm run build` | `tsc -p tsconfig.build.json` → `dist/` |
344
+ | `pnpm run rp` / `pnpm run rpm` | patch / minor release, then publish to npm |
257
345
 
258
346
  ## License
259
347
 
@@ -3,14 +3,17 @@
3
3
  *
4
4
  * PostgreSQL key-value storage adapter implementation.
5
5
  */
6
- import type pg from "pg";
6
+ import { type PostgresDb } from "../utils/driver.ts";
7
7
  import { AdapterAbstract, type AdapterAbstractOptions, type Operation, type SetMultipleEntry, type SetOptions, type TtlResult } from "./abstract.ts";
8
8
  /**
9
9
  * Configuration options for the PostgreSQL KV adapter.
10
10
  */
11
11
  export interface AdapterPostgresOptions extends AdapterAbstractOptions {
12
- /** PostgreSQL connection instance - either `pg.Pool` or `pg.Client`. */
13
- db: pg.Pool | pg.Client;
12
+ /**
13
+ * The database: a node-postgres `pg.Pool`, a connected `pg.Client`, or a sqlkit
14
+ * `Driver` of the `pg` dialect. The caller owns it: the adapter never closes it.
15
+ */
16
+ db: PostgresDb;
14
17
  /**
15
18
  * Name of the table to use for storing key-value pairs.
16
19
  * The table will be created automatically if it doesn't exist.
@@ -4,7 +4,10 @@
4
4
  * PostgreSQL key-value storage adapter implementation.
5
5
  */
6
6
  import { createClog } from "@marianmeres/clog";
7
+ import { toPgDriver } from "../utils/driver.js";
7
8
  import { AdapterAbstract, } from "./abstract.js";
9
+ /** Rows fetched per page by {@link AdapterPostgres.keysIter}. */
10
+ const KEYS_ITER_PAGE = 500;
8
11
  /**
9
12
  * PostgreSQL key-value storage adapter.
10
13
  *
@@ -22,6 +25,8 @@ export class AdapterPostgres extends AdapterAbstract {
22
25
  validateKeys: true,
23
26
  };
24
27
  #cleanupTimer;
28
+ /** What every statement runs on: built from `options.db` by {@link toPgDriver}. */
29
+ #db;
25
30
  constructor(namespace = "", options = {}) {
26
31
  super(namespace);
27
32
  this.options = Object.freeze({ ...this.options, ...(options || {}) });
@@ -33,6 +38,8 @@ export class AdapterPostgres extends AdapterAbstract {
33
38
  if (!/^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)?$/.test(this.options.tableName)) {
34
39
  throw new Error(`Invalid tableName "${this.options.tableName}". Only word characters and a single "schema." prefix are allowed.`);
35
40
  }
41
+ // last, so a constructor that throws leaves no driver (and no listener) behind
42
+ this.#db = toPgDriver(this.options.db);
36
43
  }
37
44
  /** Safe slug derived from tableName for use inside other identifiers (e.g. index names). */
38
45
  get #safeSlug() {
@@ -40,8 +47,8 @@ export class AdapterPostgres extends AdapterAbstract {
40
47
  }
41
48
  /** @inheritdoc */
42
49
  async initialize() {
43
- const { db, tableName } = this.options;
44
- await db.query(`
50
+ const { tableName } = this.options;
51
+ await this.#db.query(`
45
52
  CREATE TABLE IF NOT EXISTS ${tableName} (
46
53
  key VARCHAR(512) PRIMARY KEY,
47
54
  value JSONB NOT NULL,
@@ -50,13 +57,13 @@ export class AdapterPostgres extends AdapterAbstract {
50
57
  updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
51
58
  )
52
59
  `);
53
- await db.query(`
60
+ await this.#db.query(`
54
61
  CREATE INDEX IF NOT EXISTS idx_${this.#safeSlug}_expires_at
55
62
  ON ${tableName} (expires_at)
56
63
  WHERE expires_at IS NOT NULL
57
64
  `);
58
65
  // Index for efficient prefix lookups with LIKE 'prefix%' queries
59
- await db.query(`
66
+ await this.#db.query(`
60
67
  CREATE INDEX IF NOT EXISTS idx_${this.#safeSlug}_key_prefix
61
68
  ON ${tableName} (key text_pattern_ops)
62
69
  `);
@@ -71,17 +78,17 @@ export class AdapterPostgres extends AdapterAbstract {
71
78
  this.#cleanupTimer = undefined;
72
79
  this._initialized = false;
73
80
  if (hard) {
74
- const { db, tableName } = this.options;
75
- await db.query(`DROP TABLE IF EXISTS ${tableName}`);
81
+ const { tableName } = this.options;
82
+ await this.#db.query(`DROP TABLE IF EXISTS ${tableName}`);
76
83
  }
77
84
  }
78
85
  async #maybeTTLCleanup() {
79
86
  clearTimeout(this.#cleanupTimer); // safety
80
87
  if (this.options.ttlCleanupIntervalSec) {
81
- const { db, tableName, logger } = this.options;
88
+ const { tableName, logger } = this.options;
82
89
  // do the cleanup now
83
90
  try {
84
- await db.query(`
91
+ await this.#db.query(`
85
92
  DELETE FROM ${tableName}
86
93
  WHERE expires_at IS NOT NULL AND expires_at <= NOW()`);
87
94
  }
@@ -118,7 +125,7 @@ export class AdapterPostgres extends AdapterAbstract {
118
125
  }
119
126
  /** @inheritdoc */
120
127
  set(key, value, options = {}) {
121
- return this.#setOn(this.options.db, key, value, options);
128
+ return this.#setOn(this.#db, key, value, options);
122
129
  }
123
130
  async #setOn(exec, key, value, options = {}) {
124
131
  this._assertInitialized();
@@ -138,12 +145,12 @@ export class AdapterPostgres extends AdapterAbstract {
138
145
  async setIfAbsent(key, value, options = {}) {
139
146
  this._assertInitialized();
140
147
  const nsKey = this._withNs(key);
141
- const { db, tableName } = this.options;
148
+ const { tableName } = this.options;
142
149
  const ttl = this._resolveTtl(options);
143
150
  const expiresAt = ttl ? new Date(Date.now() + ttl * 1000) : null;
144
151
  // INSERT on new keys; UPDATE only when the existing row is expired.
145
152
  // (Live keys stay untouched — the hallmark of setIfAbsent.)
146
- const { rowCount } = await db.query(`INSERT INTO ${tableName} (key, value, expires_at, updated_at)
153
+ const { rowCount } = await this.#db.query(`INSERT INTO ${tableName} (key, value, expires_at, updated_at)
147
154
  VALUES ($1, $2, $3, NOW())
148
155
  ON CONFLICT (key) DO UPDATE SET
149
156
  value = EXCLUDED.value,
@@ -156,11 +163,11 @@ export class AdapterPostgres extends AdapterAbstract {
156
163
  async getSet(key, value, options = {}) {
157
164
  this._assertInitialized();
158
165
  const nsKey = this._withNs(key);
159
- const { db, tableName } = this.options;
166
+ const { tableName } = this.options;
160
167
  const ttl = this._resolveTtl(options);
161
168
  const expiresAt = ttl ? new Date(Date.now() + ttl * 1000) : null;
162
169
  // CTE captures pre-write state; UPSERT writes; RETURNING surfaces the prev.
163
- const { rows } = await db.query(`WITH prev AS (
170
+ const { rows } = await this.#db.query(`WITH prev AS (
164
171
  SELECT value FROM ${tableName}
165
172
  WHERE key = $1 AND (expires_at IS NULL OR expires_at > NOW())
166
173
  ), upserted AS (
@@ -178,7 +185,7 @@ export class AdapterPostgres extends AdapterAbstract {
178
185
  async #incrBy(key, delta, options = {}) {
179
186
  this._assertInitialized();
180
187
  const nsKey = this._withNs(key);
181
- const { db, tableName } = this.options;
188
+ const { tableName } = this.options;
182
189
  const ttl = this._resolveTtl(options);
183
190
  const expiresAt = ttl ? new Date(Date.now() + ttl * 1000) : null;
184
191
  // - New key OR expired key: value = delta, expires_at = $ttl
@@ -186,7 +193,7 @@ export class AdapterPostgres extends AdapterAbstract {
186
193
  // The nested cast `value::text::numeric` throws 22P02 when the stored
187
194
  // value is not a JSON number — we translate that to TypeError.
188
195
  try {
189
- const { rows } = await db.query(`INSERT INTO ${tableName} (key, value, expires_at, updated_at)
196
+ const { rows } = await this.#db.query(`INSERT INTO ${tableName} (key, value, expires_at, updated_at)
190
197
  VALUES ($1, to_jsonb($2::numeric), $3, NOW())
191
198
  ON CONFLICT (key) DO UPDATE SET
192
199
  value = CASE
@@ -223,7 +230,7 @@ export class AdapterPostgres extends AdapterAbstract {
223
230
  async cas(key, expected, next, options = {}) {
224
231
  this._assertInitialized();
225
232
  const nsKey = this._withNs(key);
226
- const { db, tableName } = this.options;
233
+ const { tableName } = this.options;
227
234
  const hasTtl = options.ttl !== undefined;
228
235
  const ttl = hasTtl ? this._resolveTtl(options) : undefined;
229
236
  const expiresAt = ttl ? new Date(Date.now() + ttl * 1000) : null;
@@ -243,12 +250,12 @@ export class AdapterPostgres extends AdapterAbstract {
243
250
  const params = [nsKey, nextJson, expectedJson];
244
251
  if (hasTtl)
245
252
  params.push(expiresAt);
246
- const { rowCount } = await db.query(sql, params);
253
+ const { rowCount } = await this.#db.query(sql, params);
247
254
  return (rowCount ?? 0) > 0;
248
255
  }
249
256
  /** @inheritdoc */
250
257
  get(key) {
251
- return this.#getOn(this.options.db, key);
258
+ return this.#getOn(this.#db, key);
252
259
  }
253
260
  async #getOn(exec, key) {
254
261
  this._assertInitialized();
@@ -263,7 +270,7 @@ export class AdapterPostgres extends AdapterAbstract {
263
270
  }
264
271
  /** @inheritdoc */
265
272
  delete(key) {
266
- return this.#deleteOn(this.options.db, key);
273
+ return this.#deleteOn(this.#db, key);
267
274
  }
268
275
  async #deleteOn(exec, key) {
269
276
  this._assertInitialized();
@@ -278,81 +285,58 @@ export class AdapterPostgres extends AdapterAbstract {
278
285
  async exists(key) {
279
286
  this._assertInitialized();
280
287
  key = this._withNs(key);
281
- const { db, tableName } = this.options;
282
- const { rows } = await db.query(`SELECT 1 FROM ${tableName}
288
+ const { tableName } = this.options;
289
+ const { rows } = await this.#db.query(`SELECT 1 FROM ${tableName}
283
290
  WHERE key = $1 AND (expires_at IS NULL OR expires_at > NOW())`, [key]);
284
291
  return rows.length > 0;
285
292
  }
286
293
  /** @inheritdoc */
287
294
  async keys(pattern) {
288
295
  this._assertInitialized();
289
- const { db, tableName } = this.options;
290
- const { rows } = await db.query(`SELECT key FROM ${tableName}
296
+ const { tableName } = this.options;
297
+ const { rows } = await this.#db.query(`SELECT key FROM ${tableName}
291
298
  WHERE key LIKE $1${this.#likeEscape}
292
- AND (expires_at IS NULL OR expires_at > NOW())
293
- ORDER BY key`, [this.#withNsLike(pattern)]);
299
+ AND (expires_at IS NULL OR expires_at > NOW())`, [this.#withNsLike(pattern)]);
294
300
  return rows.map((row) => this._withoutNs(row.key)).toSorted();
295
301
  }
296
302
  /** @inheritdoc */
297
303
  async *keysIter(pattern) {
298
304
  this._assertInitialized();
299
305
  const { tableName } = this.options;
300
- const { client, release } = await this.#acquireClient();
301
- // Use a per-call cursor name so multiple iterators on the same PG
302
- // client can't collide. `toString(36)` only emits [0-9a-z].
303
- const cursorName = `kv_iter_${Math.random().toString(36).slice(2)}`;
304
- let inTxn = false;
305
- try {
306
- await client.query("BEGIN");
307
- inTxn = true;
308
- await client.query(`DECLARE ${cursorName} NO SCROLL CURSOR FOR
309
- SELECT key FROM ${tableName}
310
- WHERE key LIKE $1${this.#likeEscape}
311
- AND (expires_at IS NULL OR expires_at > NOW())
312
- ORDER BY key`, [this.#withNsLike(pattern)]);
313
- while (true) {
314
- const { rows } = await client.query(`FETCH 500 FROM ${cursorName}`);
315
- if (rows.length === 0)
316
- break;
317
- for (const row of rows) {
318
- yield this._withoutNs(row.key);
319
- }
306
+ const like = this.#withNsLike(pattern);
307
+ // Keyset pages in byte order, which is the order of the text_pattern_ops index
308
+ // initialize() creates: each page is one range scan of it. Nothing stays open
309
+ // between yields, so an early break leaves nothing to release.
310
+ let after;
311
+ while (true) {
312
+ const { rows } = after === undefined
313
+ ? await this.#db.query(`SELECT key FROM ${tableName}
314
+ WHERE key LIKE $1${this.#likeEscape}
315
+ AND (expires_at IS NULL OR expires_at > NOW())
316
+ ORDER BY key USING ~<~ LIMIT ${KEYS_ITER_PAGE}`, [like])
317
+ : await this.#db.query(`SELECT key FROM ${tableName}
318
+ WHERE key LIKE $1${this.#likeEscape} AND key ~>~ $2
319
+ AND (expires_at IS NULL OR expires_at > NOW())
320
+ ORDER BY key USING ~<~ LIMIT ${KEYS_ITER_PAGE}`, [like, after]);
321
+ for (const row of rows) {
322
+ yield this._withoutNs(row.key);
320
323
  }
321
- }
322
- finally {
323
- if (inTxn) {
324
- try {
325
- await client.query(`CLOSE ${cursorName}`);
326
- }
327
- catch {
328
- // cursor may already be gone if the txn aborted
329
- }
330
- try {
331
- await client.query("COMMIT");
332
- }
333
- catch {
334
- try {
335
- await client.query("ROLLBACK");
336
- }
337
- catch {
338
- // ignore — caller will see the original error
339
- }
340
- }
341
- }
342
- release();
324
+ if (rows.length < KEYS_ITER_PAGE)
325
+ return;
326
+ after = rows[rows.length - 1].key;
343
327
  }
344
328
  }
345
329
  /** @inheritdoc */
346
330
  async clear(pattern) {
347
331
  this._assertInitialized();
348
- const { db, tableName } = this.options;
332
+ const { tableName } = this.options;
349
333
  const likePattern = this.#withNsLike(pattern);
350
334
  // Batch large deletions to prevent long-running transactions
351
335
  const batchSize = 10000;
352
336
  let totalDeleted = 0;
353
337
  let deleted;
354
338
  do {
355
- const { rowCount } = await db.query(`DELETE FROM ${tableName} WHERE key IN (
339
+ const { rowCount } = await this.#db.query(`DELETE FROM ${tableName} WHERE key IN (
356
340
  SELECT key FROM ${tableName} WHERE key LIKE $1${this.#likeEscape} LIMIT $2
357
341
  )`, [likePattern, batchSize]);
358
342
  deleted = rowCount ?? 0;
@@ -366,7 +350,7 @@ export class AdapterPostgres extends AdapterAbstract {
366
350
  if (entries.length === 0)
367
351
  return [];
368
352
  const normalized = this._normalizePairs(entries);
369
- const { db, tableName } = this.options;
353
+ const { tableName } = this.options;
370
354
  const placeholders = [];
371
355
  const params = [];
372
356
  let idx = 1;
@@ -384,7 +368,7 @@ export class AdapterPostgres extends AdapterAbstract {
384
368
  expires_at = EXCLUDED.expires_at,
385
369
  updated_at = NOW()
386
370
  `;
387
- await db.query(sql, params);
371
+ await this.#db.query(sql, params);
388
372
  return normalized.map(() => true);
389
373
  }
390
374
  /** @inheritdoc */
@@ -393,14 +377,14 @@ export class AdapterPostgres extends AdapterAbstract {
393
377
  if (keys.length === 0)
394
378
  return {};
395
379
  const placeholders = keys.map((_, i) => `$${i + 1}`).join(", ");
396
- const { db, tableName } = this.options;
380
+ const { tableName } = this.options;
397
381
  const sql = `
398
382
  SELECT key, value
399
383
  FROM ${tableName}
400
384
  WHERE key IN (${placeholders}) AND (expires_at IS NULL OR expires_at > NOW())
401
385
  `;
402
386
  const params = keys.map((k) => this._withNs(k));
403
- const { rows } = await db.query(sql, params);
387
+ const { rows } = await this.#db.query(sql, params);
404
388
  const found = Object.create(null);
405
389
  for (const row of rows) {
406
390
  found[this._withoutNs(row.key)] = row.value;
@@ -416,9 +400,9 @@ export class AdapterPostgres extends AdapterAbstract {
416
400
  async expire(key, ttl) {
417
401
  this._assertInitialized();
418
402
  key = this._withNs(key);
419
- const { db, tableName } = this.options;
403
+ const { tableName } = this.options;
420
404
  const expiresAt = new Date(Date.now() + ttl * 1000);
421
- const { rowCount } = await db.query(`UPDATE ${tableName}
405
+ const { rowCount } = await this.#db.query(`UPDATE ${tableName}
422
406
  SET expires_at = $2, updated_at = NOW()
423
407
  WHERE key = $1 AND (expires_at IS NULL OR expires_at > NOW())`, [key, expiresAt]);
424
408
  return rowCount > 0;
@@ -427,8 +411,8 @@ export class AdapterPostgres extends AdapterAbstract {
427
411
  async ttl(key) {
428
412
  this._assertInitialized();
429
413
  key = this._withNs(key);
430
- const { db, tableName } = this.options;
431
- const { rows } = await db.query(`SELECT expires_at FROM ${tableName} WHERE key = $1`, [key]);
414
+ const { tableName } = this.options;
415
+ const { rows } = await this.#db.query(`SELECT expires_at FROM ${tableName} WHERE key = $1`, [key]);
432
416
  if (rows.length === 0)
433
417
  return { state: "missing" };
434
418
  const expiresAt = rows[0].expires_at;
@@ -440,60 +424,34 @@ export class AdapterPostgres extends AdapterAbstract {
440
424
  return { state: "missing" };
441
425
  return { state: "expires", at };
442
426
  }
443
- /**
444
- * Acquire a single pinned connection for a transaction.
445
- */
446
- async #acquireClient() {
447
- const { db } = this.options;
448
- if (typeof db.totalCount === "number" && typeof db.connect === "function") {
449
- const client = await db.connect();
450
- return { client, release: () => client.release() };
451
- }
452
- return { client: db, release: () => { } };
453
- }
454
427
  /** @inheritdoc */
455
428
  async transaction(operations) {
456
- this._assertInitialized();
457
- const { logger } = this.options;
458
- const { client, release } = await this.#acquireClient();
459
- const results = [];
460
- try {
461
- await client.query("BEGIN");
462
- try {
463
- for (const op of operations) {
464
- switch (op.type) {
465
- case "set":
466
- results.push(await this.#setOn(client, op.key, op.value, op.options || {}));
467
- break;
468
- case "get":
469
- results.push(await this.#getOn(client, op.key));
470
- break;
471
- case "delete":
472
- results.push(await this.#deleteOn(client, op.key));
473
- break;
474
- }
475
- }
476
- await client.query("COMMIT");
477
- }
478
- catch (err) {
479
- try {
480
- await client.query("ROLLBACK");
429
+ this._assertInitialized(); // `async` keeps this a rejection
430
+ // The driver pins one connection (or, on one client, holds every other caller
431
+ // back), checks the COMMIT, and rolls back when the callback rejects. Called from
432
+ // inside a transaction the host opened on the same driver, this one joins it.
433
+ return this.#db.transaction(async (tx) => {
434
+ const results = [];
435
+ for (const op of operations) {
436
+ switch (op.type) {
437
+ case "set":
438
+ results.push(await this.#setOn(tx, op.key, op.value, op.options || {}));
439
+ break;
440
+ case "get":
441
+ results.push(await this.#getOn(tx, op.key));
442
+ break;
443
+ case "delete":
444
+ results.push(await this.#deleteOn(tx, op.key));
445
+ break;
481
446
  }
482
- catch (rollbackErr) {
483
- logger?.error?.(`Rollback failed: ${rollbackErr}`);
484
- }
485
- throw err;
486
447
  }
487
- }
488
- finally {
489
- release();
490
- }
491
- return results;
448
+ return results;
449
+ });
492
450
  }
493
451
  async __debug_dump() {
494
452
  this._assertInitialized();
495
- const { db, tableName } = this.options;
496
- const { rows } = await db.query(`SELECT key, value, expires_at FROM ${tableName} WHERE key LIKE $1${this.#likeEscape}`, [this.#withNsLike("*")]);
453
+ const { tableName } = this.options;
454
+ const { rows } = await this.#db.query(`SELECT key, value, expires_at FROM ${tableName} WHERE key LIKE $1${this.#likeEscape}`, [this.#withNsLike("*")]);
497
455
  return rows.reduce((m, row) => {
498
456
  m[this._withoutNs(row.key)] = {
499
457
  value: row.value,
@@ -4,6 +4,7 @@
4
4
  * SQLite key-value storage adapter implementation (`node:sqlite`).
5
5
  */
6
6
  import { createClog } from "@marianmeres/clog";
7
+ import { isDriver } from "../utils/driver.js";
7
8
  import { AdapterAbstract, } from "./abstract.js";
8
9
  /** Rows fetched per page by {@link AdapterSqlite.keysIter}. */
9
10
  const KEYS_ITER_PAGE = 500;
@@ -53,6 +54,9 @@ export class AdapterSqlite extends AdapterAbstract {
53
54
  if (!this.options.db) {
54
55
  throw new Error("Missing sqlite database instance");
55
56
  }
57
+ if (isDriver(this.options.db)) {
58
+ throw new TypeError("The sqlite adapter takes a node:sqlite DatabaseSync; a sqlkit Driver is not supported yet");
59
+ }
56
60
  // tableName is spliced into SQL verbatim, so guard against surprises.
57
61
  if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(this.options.tableName)) {
58
62
  throw new Error(`Invalid tableName "${this.options.tableName}". Only word characters are allowed.`);
package/dist/kv.d.ts CHANGED
@@ -16,6 +16,7 @@ export { AdapterMemory, type AdapterMemoryOptions } from "./adapter/memory.ts";
16
16
  export { AdapterPostgres, type AdapterPostgresOptions } from "./adapter/postgres.ts";
17
17
  export { AdapterRedis, type AdapterRedisOptions } from "./adapter/redis.ts";
18
18
  export { AdapterSqlite, type AdapterSqliteOptions } from "./adapter/sqlite.ts";
19
+ export type { PostgresDb } from "./utils/driver.ts";
19
20
  export { sleep } from "./utils/sleep.ts";
20
21
  /**
21
22
  * Type mapping for all supported KV adapter types.
@@ -52,7 +53,8 @@ interface KnownTypes {
52
53
  * @param type - The storage backend to use:
53
54
  * - `"memory"` - In-memory storage (default, not persisted)
54
55
  * - `"redis"` - Redis server (requires `db` option with Redis client)
55
- * - `"postgres"` - PostgreSQL database (requires `db` option with pg client/pool)
56
+ * - `"postgres"` - PostgreSQL database (requires `db` option: a `pg.Pool`, a
57
+ * connected `pg.Client`, or a sqlkit `Driver` of the `pg` dialect)
56
58
  * - `"sqlite"` - SQLite database (requires `db` option with a `node:sqlite` `DatabaseSync`)
57
59
  * @param options - Adapter-specific configuration options.
58
60
  *
package/dist/kv.js CHANGED
@@ -29,7 +29,8 @@ export { sleep } from "./utils/sleep.js";
29
29
  * @param type - The storage backend to use:
30
30
  * - `"memory"` - In-memory storage (default, not persisted)
31
31
  * - `"redis"` - Redis server (requires `db` option with Redis client)
32
- * - `"postgres"` - PostgreSQL database (requires `db` option with pg client/pool)
32
+ * - `"postgres"` - PostgreSQL database (requires `db` option: a `pg.Pool`, a
33
+ * connected `pg.Client`, or a sqlkit `Driver` of the `pg` dialect)
33
34
  * - `"sqlite"` - SQLite database (requires `db` option with a `node:sqlite` `DatabaseSync`)
34
35
  * @param options - Adapter-specific configuration options.
35
36
  *
@@ -0,0 +1,25 @@
1
+ /**
2
+ * @module
3
+ *
4
+ * The one seam between the postgres adapter and the database: whatever the host passed
5
+ * as `db` becomes a sqlkit `Driver`, and every statement the adapter sends goes through it.
6
+ */
7
+ import type { Driver } from "@marianmeres/sqlkit";
8
+ import { type PgPoolLike, type PgStandaloneClientLike } from "@marianmeres/sqlkit/pg";
9
+ /**
10
+ * What the postgres adapter's `db` option takes: a node-postgres `pg.Pool`, a
11
+ * connected `pg.Client`, or a sqlkit `Driver` of the `pg` dialect. The first two are
12
+ * declared by shape, so nothing here imports `pg`.
13
+ */
14
+ export type PostgresDb = PgPoolLike | PgStandaloneClientLike | Driver;
15
+ /** Whether `db` is a sqlkit `Driver` (by shape). */
16
+ export declare function isDriver(db: unknown): db is Driver;
17
+ /** A `pg.Pool` has `connect()` and counts its clients; a `pg.Client` does not. */
18
+ export declare function isPool(db: unknown): db is PgPoolLike;
19
+ /**
20
+ * The `Driver` the postgres adapter runs on. A host's `Driver` is used as it is; a
21
+ * pool or a client gets one driver per object. The adapter never closes any of them.
22
+ *
23
+ * @throws {TypeError} For a `Driver` of another dialect.
24
+ */
25
+ export declare function toPgDriver(db: PostgresDb): Driver;
@@ -0,0 +1,48 @@
1
+ /**
2
+ * @module
3
+ *
4
+ * The one seam between the postgres adapter and the database: whatever the host passed
5
+ * as `db` becomes a sqlkit `Driver`, and every statement the adapter sends goes through it.
6
+ */
7
+ import { createPgClientDriver, createPgDriver, } from "@marianmeres/sqlkit/pg";
8
+ /** One driver per pool or client object, however many adapters share it. */
9
+ const drivers = new WeakMap();
10
+ /** Whether `db` is a sqlkit `Driver` (by shape). */
11
+ export function isDriver(db) {
12
+ const d = db;
13
+ return (typeof d === "object" &&
14
+ d !== null &&
15
+ typeof d.dialect === "string" &&
16
+ typeof d.query === "function" &&
17
+ typeof d.exec === "function" &&
18
+ typeof d.transaction === "function" &&
19
+ typeof d.capabilities === "object" &&
20
+ d.capabilities !== null);
21
+ }
22
+ /** A `pg.Pool` has `connect()` and counts its clients; a `pg.Client` does not. */
23
+ export function isPool(db) {
24
+ return (typeof db === "object" &&
25
+ db !== null &&
26
+ typeof db.connect === "function" &&
27
+ typeof db.totalCount === "number");
28
+ }
29
+ /**
30
+ * The `Driver` the postgres adapter runs on. A host's `Driver` is used as it is; a
31
+ * pool or a client gets one driver per object. The adapter never closes any of them.
32
+ *
33
+ * @throws {TypeError} For a `Driver` of another dialect.
34
+ */
35
+ export function toPgDriver(db) {
36
+ if (isDriver(db)) {
37
+ if (db.dialect !== "pg") {
38
+ throw new TypeError(`The postgres adapter runs on PostgreSQL: got a "${db.dialect}" driver (${db.adapter})`);
39
+ }
40
+ return db;
41
+ }
42
+ let driver = drivers.get(db);
43
+ if (driver === undefined) {
44
+ driver = isPool(db) ? createPgDriver(db) : createPgClientDriver(db);
45
+ drivers.set(db, driver);
46
+ }
47
+ return driver;
48
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@marianmeres/kv",
3
- "version": "3.1.1",
3
+ "version": "4.0.0",
4
4
  "description": "Key-value storage abstraction with a unified, Redis-inspired API over memory, Redis, PostgreSQL and SQLite backends.",
5
5
  "license": "MIT",
6
6
  "author": "Marian Meres",
@@ -44,13 +44,14 @@
44
44
  ],
45
45
  "dependencies": {
46
46
  "@marianmeres/clog": "^3.23.0",
47
- "@types/pg": "^8.23.1",
48
- "pg": "^8.23.1",
47
+ "@marianmeres/sqlkit": "^0.1.1",
49
48
  "redis": "^6.3.1"
50
49
  },
51
50
  "devDependencies": {
52
51
  "@biomejs/biome": "^2.6.4",
53
52
  "@types/node": "^26.6.6",
53
+ "@types/pg": "^8.23.1",
54
+ "pg": "^8.23.1",
54
55
  "typescript": "^7.0.2"
55
56
  },
56
57
  "scripts": {
@@ -59,15 +60,11 @@
59
60
  "typecheck": "tsc --noEmit",
60
61
  "lint": "biome check .",
61
62
  "format": "biome check --write .",
62
- "test": "node --test --env-file-if-exists=.env \"tests/**/*.test.ts\"",
63
- "test:watch": "node --test --watch --env-file-if-exists=.env \"tests/**/*.test.ts\"",
64
- "test:deno": "deno test -A --no-check --env-file tests/",
63
+ "test": "node --test --test-concurrency=1 --env-file-if-exists=.env \"tests/**/*.test.ts\"",
64
+ "test:watch": "node --test --test-concurrency=1 --watch --env-file-if-exists=.env \"tests/**/*.test.ts\"",
65
65
  "ci": "pnpm run typecheck && pnpm run lint && pnpm test",
66
- "jsr:check": "deno publish --dry-run --allow-dirty --no-check --set-version \"$npm_package_version\"",
67
66
  "release": "npx --yes @marianmeres/release@^2.2.0",
68
- "publish:jsr": "pnpm run test:deno && deno publish --no-check --set-version \"$npm_package_version\"",
69
- "publish:all": "pnpm publish && pnpm run publish:jsr",
70
- "rp": "pnpm run release -y && pnpm run publish:all",
71
- "rpm": "pnpm run release minor -y && pnpm run publish:all"
67
+ "rp": "pnpm run release -y && pnpm publish",
68
+ "rpm": "pnpm run release minor -y && pnpm publish"
72
69
  }
73
70
  }