@marianmeres/kv 3.1.0 → 4.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +51 -23
- package/API.md +92 -16
- package/README.md +116 -28
- package/dist/adapter/memory.d.ts +1 -0
- package/dist/adapter/memory.js +3 -0
- package/dist/adapter/postgres.d.ts +7 -3
- package/dist/adapter/postgres.js +91 -126
- package/dist/adapter/sqlite.d.ts +1 -0
- package/dist/adapter/sqlite.js +14 -1
- package/dist/kv.d.ts +3 -1
- package/dist/kv.js +2 -1
- package/dist/utils/driver.d.ts +25 -0
- package/dist/utils/driver.js +48 -0
- package/package.json +8 -10
package/AGENTS.md
CHANGED
|
@@ -5,9 +5,10 @@ Machine-friendly reference for AI agents working with this codebase.
|
|
|
5
5
|
## Quick Reference
|
|
6
6
|
|
|
7
7
|
- **Stack**: TypeScript (erasable-only), Node LTS (`.node-version`), pnpm, `node:test`, Biome.
|
|
8
|
-
- **Install**: `pnpm install` | **Test**: `pnpm test` (needs Postgres + Redis; SQLite runs in memory
|
|
8
|
+
- **Install**: `pnpm install` | **Test**: `pnpm test` (needs Postgres + Redis; SQLite runs in memory;
|
|
9
|
+
test files run one at a time) |
|
|
9
10
|
**Check all**: `pnpm run ci` | **Build**: `pnpm run build`
|
|
10
|
-
- **Published**: npm (`dist/`, built by `tsc`)
|
|
11
|
+
- **Published**: npm (`dist/`, built by `tsc`)
|
|
11
12
|
|
|
12
13
|
## Purpose
|
|
13
14
|
|
|
@@ -17,7 +18,7 @@ Key-value storage abstraction layer providing a unified Redis-inspired API acros
|
|
|
17
18
|
|
|
18
19
|
```
|
|
19
20
|
src/
|
|
20
|
-
├── index.ts — The only entry point (package.json "exports"
|
|
21
|
+
├── index.ts — The only entry point (package.json "exports")
|
|
21
22
|
├── kv.ts — Factory function and all exports
|
|
22
23
|
├── adapter/
|
|
23
24
|
│ ├── abstract.ts — Base class + shared types
|
|
@@ -26,14 +27,20 @@ src/
|
|
|
26
27
|
│ ├── postgres.ts — PostgreSQL adapter
|
|
27
28
|
│ └── sqlite.ts — SQLite adapter (`node:sqlite`)
|
|
28
29
|
└── utils/
|
|
30
|
+
├── driver.ts — `toPgDriver()`: the postgres adapter's `db` as a sqlkit `Driver`
|
|
29
31
|
└── sleep.ts — Sleep utility function
|
|
30
32
|
|
|
31
33
|
tests/
|
|
32
34
|
├── kv.test.ts — Main test file (node:test); every test runs against all adapters
|
|
35
|
+
├── driver.test.ts — The postgres adapter on a pool, a client and a sqlkit Driver
|
|
33
36
|
├── _tests-runner.ts — Test harness: builds one client per adapter per test
|
|
34
37
|
├── _redis.ts — Redis test helper
|
|
35
38
|
├── _pg.ts — PostgreSQL test helper
|
|
36
|
-
|
|
39
|
+
├── _sqlite.ts — SQLite test helper (fresh `:memory:` database per test)
|
|
40
|
+
└── _no-destroy-child.ts — child process: an adapter left running must not hold it open
|
|
41
|
+
|
|
42
|
+
docs/
|
|
43
|
+
└── backlog.md — Open work, ranked, not scheduled (the open copy)
|
|
37
44
|
|
|
38
45
|
dist/ — `tsc -p tsconfig.build.json` output; gitignored; the npm payload
|
|
39
46
|
```
|
|
@@ -42,8 +49,8 @@ dist/ — `tsc -p tsconfig.build.json` output; gitignored; the
|
|
|
42
49
|
|
|
43
50
|
1. **Runtime policy** — Node at the keyboard and in production, npm as the registry. No
|
|
44
51
|
`Deno.*`, `Bun.*`, `bun:*`, `jsr:` or `npm:` specifiers. Web standards first; `node:*`
|
|
45
|
-
where they don't reach. Nothing is built, tested or documented for Bun or Deno
|
|
46
|
-
|
|
52
|
+
where they don't reach. Nothing is built, tested or documented for Bun or Deno.
|
|
53
|
+
Published to npm only; 3.1.1 was the last version on JSR.
|
|
47
54
|
2. **Erasable TypeScript** — no `enum`, `namespace`, constructor parameter properties;
|
|
48
55
|
`import type` for types; explicit `.ts` import extensions (rewritten to `.js` on emit).
|
|
49
56
|
Node strips types and never checks them: `pnpm run typecheck` is the only type gate.
|
|
@@ -60,31 +67,47 @@ dist/ — `tsc -p tsconfig.build.json` output; gitignored; the
|
|
|
60
67
|
8. Pattern matching: Redis-style (`*` = any chars, `?` = single char). All other characters — including `.`, `(`, `[`, `%`, `_` — are matched **literally**.
|
|
61
68
|
9. TTL resolution: `options.ttl ?? defaultTtl`; `0` or negative disables expiration for that call (explicitly overrides `defaultTtl`).
|
|
62
69
|
10. Shared helpers live in `abstract.ts` — use `_globToRegex()`, `_resolveTtl()`, `_normalizePairs()`; do not re-implement per-adapter. The SQL adapters keep their own pattern escaping next to the SQL that uses it: Postgres `#likePattern` (LIKE, `ESCAPE '\'`), SQLite `#globPattern` (GLOB — case-sensitive, unlike SQLite's LIKE; `[` → `[[]`). Both escape the namespace as a literal prefix.
|
|
63
|
-
11. Postgres `transaction()`
|
|
64
|
-
|
|
70
|
+
11. Postgres `transaction()` is `this.#db.transaction()`: the driver pins the connection
|
|
71
|
+
(or, on one `pg.Client`, holds every other caller back), checks the `COMMIT` and rolls
|
|
72
|
+
back. Nothing in the adapter sends `BEGIN`, `COMMIT` or `ROLLBACK`. Pass the callback's
|
|
73
|
+
`tx` to the `…On` helpers.
|
|
74
|
+
12. Redis adapter owns the connection iff `initialize()` opened it (`#weOpened` flag). The Postgres and SQLite adapters never close the `db` they were given; the Postgres adapter never calls `close()` on a driver, its own or the host's (it would end the host's pool or client).
|
|
65
75
|
13. Key validation lives in `_withNs()` — enforced when `options.validateKeys !== false`. Reject empty strings, keys with `\0`, and total `namespace + key` length > 512.
|
|
66
76
|
14. `ttl()` returns `TtlResult = { state: "missing" | "no-ttl" | "expires"; at? }` — discriminated union, not `Date | null | false`.
|
|
67
77
|
15. `incr`/`decr`/`getSet`/`cas`/`setIfAbsent` must be atomic: Redis = Lua or NX; Postgres = UPSERT/UPDATE with RETURNING; SQLite = a single UPSERT/UPDATE (`setIfAbsent`, `cas`) or a read-modify-write inside `#atomic()` (`getSet`, `incr`/`decr`); Memory = plain read-modify-write (single-threaded).
|
|
68
|
-
16. Non-numeric `incr`/`decr` target throws `TypeError("KV value is not a number")`. In Postgres, detect via
|
|
69
|
-
17. `keysIter`
|
|
78
|
+
16. Non-numeric `incr`/`decr` target throws `TypeError("KV value is not a number")`. In Postgres, detect via the `22P02` in the sqlkit `SqlError`'s `code`; SQLite checks `typeof` in JS, like Memory. No other database error is caught, wrapped or translated: the caller gets sqlkit's `SqlError`.
|
|
79
|
+
17. `keysIter` in both SQL adapters pages by keyset, 500 keys a page, so nothing stays open between yields and an early consumer break leaves nothing to release. It is not one snapshot. SQLite pages with `key > last`. Postgres pages in byte order on the `text_pattern_ops` prefix index (`key ~>~ $2`, `ORDER BY key USING ~<~`); a plain `key > $2 ORDER BY key` would walk the primary key across every namespace. Postgres `keys()` has no SQL `ORDER BY` for the same reason: it sorts in JS.
|
|
70
80
|
18. SQLite: `src/` imports `node:sqlite` **type-only** — the caller passes the `DatabaseSync`. A runtime import would make every consumer of the package load `node:sqlite`.
|
|
71
81
|
19. SQLite multi-statement operations go through `#atomic()`: `BEGIN IMMEDIATE`, or a savepoint when `db.isTransaction` (the caller's transaction on the same handle). The callback must stay synchronous — an `await` inside would let other work join the transaction.
|
|
72
82
|
20. SQLite stores `value` as JSON text and `expires_at` as epoch ms (INTEGER), compared against a bound `Date.now()` — never SQLite's own clock. Read counts/integers through `Number()` (a handle opened with `readBigInts` returns bigints).
|
|
83
|
+
21. TTL cleanup timers are `unref()`ed (`unref?.()`: browser and worker timers are plain numbers), so a client nobody destroyed cannot hold the process open; `destroy()` still clears them. Postgres's tick awaits its query, so it re-checks `_initialized` before re-arming, and `destroy()` clears `_initialized` before its first `await`. SQLite stops the cleanup (one error log) when it finds `db.isOpen` false.
|
|
84
|
+
22. Postgres talks to the database only through `src/utils/driver.ts`: `toPgDriver(db)`,
|
|
85
|
+
called once in the constructor, turns a `pg.Pool` or `pg.Client` into a sqlkit
|
|
86
|
+
`Driver` (one per object, in a `WeakMap`: `createPgDriver` adds an `error` listener on
|
|
87
|
+
every call, and several adapters on one pool is normal), and uses a host's `Driver` as
|
|
88
|
+
it is. A `Driver` of another dialect is a `TypeError`. `options.db` stays what the host
|
|
89
|
+
passed; every statement goes through `this.#db`, with its text and `$n` parameters.
|
|
90
|
+
Nothing in `src/` imports `pg` (it is a dev dependency); `@marianmeres/sqlkit/pg` is
|
|
91
|
+
imported statically, so the package root loads `node:async_hooks`. A kv call issued
|
|
92
|
+
inside a transaction the host opened on the same `Driver` joins it, by design.
|
|
93
|
+
23. The SQLite adapter takes only a `DatabaseSync`: its constructor refuses a sqlkit
|
|
94
|
+
`Driver` with a `TypeError`.
|
|
73
95
|
|
|
74
96
|
## Before Making Changes
|
|
75
97
|
|
|
76
98
|
- [ ] Check existing patterns in adapter implementations
|
|
77
|
-
- [ ] Add or extend a test in `tests/kv.test.ts
|
|
99
|
+
- [ ] Add or extend a test in `tests/kv.test.ts` (or in `tests/driver.test.ts` for what only
|
|
100
|
+
the postgres adapter's `db` decides), then run `pnpm run ci`
|
|
78
101
|
- [ ] Verify adapter limitations are documented (README feature matrix, API.md)
|
|
79
102
|
|
|
80
103
|
## Supported Adapters
|
|
81
104
|
|
|
82
|
-
| Adapter | Type String | Required Options
|
|
83
|
-
| ---------- | ------------ |
|
|
84
|
-
| Memory | `"memory"` | None
|
|
85
|
-
| Redis | `"redis"` | `db` (Redis client)
|
|
86
|
-
| PostgreSQL | `"postgres"` | `db` (pg.Pool
|
|
87
|
-
| SQLite | `"sqlite"` | `db` (`DatabaseSync`)
|
|
105
|
+
| Adapter | Type String | Required Options | Limitations |
|
|
106
|
+
| ---------- | ------------ | -------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
|
|
107
|
+
| Memory | `"memory"` | None | Not persisted |
|
|
108
|
+
| Redis | `"redis"` | `db` (Redis client) | Namespace required; `keys()`/`clear()` unavailable in cluster mode |
|
|
109
|
+
| PostgreSQL | `"postgres"` | `db` (`pg.Pool`, `pg.Client` or a sqlkit `Driver`) | `tableName` limited to `[A-Za-z0-9_]` + optional `schema.` prefix |
|
|
110
|
+
| SQLite | `"sqlite"` | `db` (`DatabaseSync`) | `tableName` limited to `[A-Za-z0-9_]`, no schema prefix; synchronous driver; a sqlkit `Driver` is refused |
|
|
88
111
|
|
|
89
112
|
## Core API
|
|
90
113
|
|
|
@@ -151,6 +174,9 @@ interface Operation {
|
|
|
151
174
|
|
|
152
175
|
type SetMultipleEntry = [string, any] | { key: string; value: any; ttl?: number };
|
|
153
176
|
|
|
177
|
+
// the postgres adapter's `db`; sqlkit's structural types, so `src/` never imports `pg`
|
|
178
|
+
type PostgresDb = PgPoolLike | PgStandaloneClientLike | Driver;
|
|
179
|
+
|
|
154
180
|
type TtlResult =
|
|
155
181
|
| { state: "missing" }
|
|
156
182
|
| { state: "no-ttl" }
|
|
@@ -160,22 +186,23 @@ type TtlResult =
|
|
|
160
186
|
## Commands
|
|
161
187
|
|
|
162
188
|
```bash
|
|
163
|
-
pnpm test # node --test over tests/**/*.test.ts,
|
|
189
|
+
pnpm test # node --test over tests/**/*.test.ts, one file at a time
|
|
190
|
+
# (--test-concurrency=1: they share `__kv` and one Redis DB), loads .env
|
|
164
191
|
pnpm run test:watch
|
|
165
192
|
pnpm run typecheck # tsc --noEmit
|
|
166
193
|
pnpm run lint # biome check . (read-only); `pnpm run format` writes
|
|
167
194
|
pnpm run ci # typecheck + lint + test — what CI runs
|
|
168
195
|
pnpm run build # tsc -p tsconfig.build.json → dist/
|
|
169
|
-
pnpm run test:deno # the same suite on Deno; only gates publish:jsr
|
|
170
196
|
```
|
|
171
197
|
|
|
172
|
-
Releasing (`pnpm run rp` / `rpm`, or `pnpm run release major` + `pnpm
|
|
173
|
-
|
|
198
|
+
Releasing (`pnpm run rp` / `rpm`, or `pnpm run release major` + `pnpm publish`) is a human
|
|
199
|
+
action.
|
|
174
200
|
|
|
175
201
|
## Test Environment Variables
|
|
176
202
|
|
|
177
|
-
Every test runs against all four adapters, so tests need a live
|
|
178
|
-
uses a fresh `:memory:` database per test and needs nothing).
|
|
203
|
+
Every test in `tests/kv.test.ts` runs against all four adapters, so tests need a live
|
|
204
|
+
Postgres and Redis (SQLite uses a fresh `:memory:` database per test and needs nothing).
|
|
205
|
+
`tests/driver.test.ts` needs Postgres only. Put
|
|
179
206
|
these in `.env` (see `.env.example`); CI provides them via service containers
|
|
180
207
|
(`.github/workflows/ci.yml`). The harness deletes **every key** in the Redis DB before each
|
|
181
208
|
test — use a scratch DB.
|
|
@@ -204,3 +231,4 @@ To add a new adapter:
|
|
|
204
231
|
|
|
205
232
|
- [API Reference](./API.md) — Complete API documentation
|
|
206
233
|
- [README](./README.md) — Quick start, feature matrix, breaking changes, development
|
|
234
|
+
- [Backlog](./docs/backlog.md) — Open work after 4.0.0, with each row's trigger
|
package/API.md
CHANGED
|
@@ -43,6 +43,7 @@ function createKVClient<T extends keyof KnownTypes>(
|
|
|
43
43
|
- `TypeError` - If namespace doesn't end with `:` (and is not empty)
|
|
44
44
|
- `TypeError` - If an unsupported adapter type is specified
|
|
45
45
|
- `Error` - If required options (like `db`) are missing for the adapter
|
|
46
|
+
- `TypeError` - If `db` is not of a kind the adapter takes (see its options below)
|
|
46
47
|
|
|
47
48
|
**Example:**
|
|
48
49
|
|
|
@@ -64,6 +65,9 @@ import pg from "pg";
|
|
|
64
65
|
const pool = new pg.Pool({ connectionString: "..." });
|
|
65
66
|
const postgres = createKVClient("myapp:", "postgres", { db: pool });
|
|
66
67
|
await postgres.initialize();
|
|
68
|
+
// ...or on the sqlkit Driver your app already shares
|
|
69
|
+
import { createPgDriver } from "@marianmeres/sqlkit/pg";
|
|
70
|
+
const shared = createKVClient("myapp:", "postgres", { db: createPgDriver(pool) });
|
|
67
71
|
|
|
68
72
|
// SQLite storage (node:sqlite ships with Node)
|
|
69
73
|
import { DatabaseSync } from "node:sqlite";
|
|
@@ -107,6 +111,9 @@ destroy(hard?: boolean): Promise<void>
|
|
|
107
111
|
| --------- | --------- | ------- | --------------------------------------------------------------------------------- |
|
|
108
112
|
| `hard` | `boolean` | `false` | If `true`, performs a hard cleanup (e.g., drops tables in PostgreSQL and SQLite). |
|
|
109
113
|
|
|
114
|
+
Stops the TTL cleanup timer (`ttlCleanupIntervalSec`). That timer never keeps the process
|
|
115
|
+
alive on its own, so a short-lived script that skips `destroy()` still exits.
|
|
116
|
+
|
|
110
117
|
---
|
|
111
118
|
|
|
112
119
|
#### `info()`
|
|
@@ -453,11 +460,12 @@ Iterates matching keys without materializing the full list. Prefer this over `ke
|
|
|
453
460
|
keysIter(pattern: string): AsyncIterable<string>
|
|
454
461
|
```
|
|
455
462
|
|
|
456
|
-
Ordering is not guaranteed across adapters — use `keys()` if you need sorted output.
|
|
463
|
+
Ordering is not guaranteed across adapters — use `keys()` if you need sorted output. The
|
|
464
|
+
walk is not one snapshot: a key added or removed while it runs may or may not be seen.
|
|
457
465
|
|
|
458
466
|
**Per-adapter behavior:**
|
|
459
467
|
- Redis: wraps `SCAN` (non-blocking).
|
|
460
|
-
- PostgreSQL:
|
|
468
|
+
- PostgreSQL: keyset pages of 500 keys, in byte order on the prefix index `initialize()` creates. Nothing stays open between pages, so an early break has nothing to release and a write between two yields does not wait for the walk.
|
|
461
469
|
- SQLite: keyset pages of 500 keys (`key > last ORDER BY key`). Nothing stays open between pages, so an early break has nothing to release.
|
|
462
470
|
- Memory: yields from the internal `Map`.
|
|
463
471
|
|
|
@@ -465,7 +473,7 @@ Ordering is not guaranteed across adapters — use `keys()` if you need sorted o
|
|
|
465
473
|
|
|
466
474
|
```typescript
|
|
467
475
|
for await (const k of client.keysIter("user:*")) {
|
|
468
|
-
if (someCondition(k)) break; // safe —
|
|
476
|
+
if (someCondition(k)) break; // safe — nothing is left open
|
|
469
477
|
}
|
|
470
478
|
```
|
|
471
479
|
|
|
@@ -582,12 +590,12 @@ transaction(operations: Operation[]): Promise<any[]>
|
|
|
582
590
|
|
|
583
591
|
Atomicity per adapter:
|
|
584
592
|
|
|
585
|
-
| Adapter | Atomic | Notes
|
|
586
|
-
| ---------- | :----: |
|
|
587
|
-
| Memory | ✓ | Single-threaded by nature
|
|
588
|
-
| Redis | ✓ | Uses `MULTI`/`EXEC`
|
|
589
|
-
| PostgreSQL | ✓ |
|
|
590
|
-
| SQLite | ✓ | `BEGIN IMMEDIATE`; a savepoint inside a caller-opened transaction
|
|
593
|
+
| Adapter | Atomic | Notes |
|
|
594
|
+
| ---------- | :----: | ------------------------------------------------------------------------------------ |
|
|
595
|
+
| Memory | ✓ | Single-threaded by nature |
|
|
596
|
+
| Redis | ✓ | Uses `MULTI`/`EXEC` |
|
|
597
|
+
| PostgreSQL | ✓ | One connection for the whole unit: pinned on a `pg.Pool`, in turn on one `pg.Client` |
|
|
598
|
+
| SQLite | ✓ | `BEGIN IMMEDIATE`; a savepoint inside a caller-opened transaction |
|
|
591
599
|
|
|
592
600
|
Per-operation `options.ttl` is honored by **all** adapters (Redis, PostgreSQL, SQLite, Memory).
|
|
593
601
|
|
|
@@ -705,6 +713,27 @@ interface Operation {
|
|
|
705
713
|
|
|
706
714
|
---
|
|
707
715
|
|
|
716
|
+
### `PostgresDb`
|
|
717
|
+
|
|
718
|
+
What the postgres adapter's `db` option takes.
|
|
719
|
+
|
|
720
|
+
```typescript
|
|
721
|
+
import type { Driver } from "@marianmeres/sqlkit";
|
|
722
|
+
import type { PgPoolLike, PgStandaloneClientLike } from "@marianmeres/sqlkit/pg";
|
|
723
|
+
|
|
724
|
+
type PostgresDb = PgPoolLike | PgStandaloneClientLike | Driver;
|
|
725
|
+
```
|
|
726
|
+
|
|
727
|
+
- `PgPoolLike` — a node-postgres `pg.Pool`.
|
|
728
|
+
- `PgStandaloneClientLike` — a connected node-postgres `pg.Client`.
|
|
729
|
+
- `Driver` — a sqlkit driver of the `pg` dialect, e.g. `createPgDriver(pool)` from
|
|
730
|
+
`@marianmeres/sqlkit/pg`.
|
|
731
|
+
|
|
732
|
+
The first two are declared by shape, so kv imports nothing from `pg`, and a `pg.Pool` and
|
|
733
|
+
a `pg.Client` fit them as they are. See [`AdapterPostgresOptions`](#adapterpostgresoptions).
|
|
734
|
+
|
|
735
|
+
---
|
|
736
|
+
|
|
708
737
|
## Adapter-Specific Options
|
|
709
738
|
|
|
710
739
|
### `AdapterMemoryOptions`
|
|
@@ -715,6 +744,7 @@ interface AdapterMemoryOptions extends AdapterAbstractOptions {
|
|
|
715
744
|
* Interval in seconds for automatic cleanup of expired keys.
|
|
716
745
|
* Set to 0 to disable automatic cleanup (expired keys will still be
|
|
717
746
|
* lazily removed on access).
|
|
747
|
+
* The timer never keeps the process alive; `destroy()` stops it.
|
|
718
748
|
*/
|
|
719
749
|
ttlCleanupIntervalSec: number;
|
|
720
750
|
}
|
|
@@ -753,8 +783,11 @@ interface AdapterRedisOptions extends AdapterAbstractOptions {
|
|
|
753
783
|
|
|
754
784
|
```typescript
|
|
755
785
|
interface AdapterPostgresOptions extends AdapterAbstractOptions {
|
|
756
|
-
/**
|
|
757
|
-
|
|
786
|
+
/**
|
|
787
|
+
* The database: a node-postgres `pg.Pool`, a connected `pg.Client`, or a sqlkit
|
|
788
|
+
* `Driver` of the `pg` dialect. The caller owns it: the adapter never closes it.
|
|
789
|
+
*/
|
|
790
|
+
db: PostgresDb;
|
|
758
791
|
/**
|
|
759
792
|
* Name of the table to use for storing key-value pairs.
|
|
760
793
|
* The table will be created automatically if it doesn't exist.
|
|
@@ -767,13 +800,51 @@ interface AdapterPostgresOptions extends AdapterAbstractOptions {
|
|
|
767
800
|
/**
|
|
768
801
|
* Interval in seconds for automatic cleanup of expired keys.
|
|
769
802
|
* Set to 0 to disable automatic cleanup.
|
|
803
|
+
* The timer never keeps the process alive; `destroy()` stops it.
|
|
770
804
|
*/
|
|
771
805
|
ttlCleanupIntervalSec: number;
|
|
772
806
|
}
|
|
773
807
|
```
|
|
774
808
|
|
|
775
|
-
|
|
776
|
-
|
|
809
|
+
`db` is a [`PostgresDb`](#postgresdb): pass a `pg.Pool` or a connected `pg.Client` as it
|
|
810
|
+
is, or a [`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit)
|
|
811
|
+
`Driver` your app built. The constructor refuses, with a `TypeError`, a `Driver` of
|
|
812
|
+
another dialect and anything that is neither a pool, a client nor a driver.
|
|
813
|
+
|
|
814
|
+
**Connection ownership:** the adapter never closes what it was given — not the pool, not
|
|
815
|
+
the client, not the driver — so `destroy()` leaves it open and several adapters can share
|
|
816
|
+
one.
|
|
817
|
+
|
|
818
|
+
**Transactions and connections:** every statement runs on a sqlkit `Driver`. The adapter
|
|
819
|
+
builds one over a `pg.Pool` or a `pg.Client`, one per pool or client object however many
|
|
820
|
+
adapters share it, and uses a `Driver` you pass as it is. `transaction()` runs on one
|
|
821
|
+
connection for the whole unit, and a `COMMIT` that did not commit rejects:
|
|
822
|
+
|
|
823
|
+
- **On a `pg.Pool`**, it pins one connection for its length; other calls run on other
|
|
824
|
+
connections at once. A connection whose `ROLLBACK` failed is destroyed, not returned to
|
|
825
|
+
the pool.
|
|
826
|
+
- **On one `pg.Client`**, calls take turns: one issued while a transaction runs waits for
|
|
827
|
+
it to finish, and never becomes part of it.
|
|
828
|
+
- **On a `Driver` you pass**, the same holds for the pool or client under it. In addition,
|
|
829
|
+
a kv call issued inside a transaction you opened on that same driver
|
|
830
|
+
(`driver.transaction(async () => { await kv.set(…) })`) joins your transaction and
|
|
831
|
+
commits or rolls back with it. Make the call outside your transaction if it must
|
|
832
|
+
outlive your rollback.
|
|
833
|
+
|
|
834
|
+
The adapter listens for the pool's or client's `error` event, so an idle connection the
|
|
835
|
+
server dropped does not end the process: it is reported through `process.emitWarning`
|
|
836
|
+
when you have no listener of your own.
|
|
837
|
+
|
|
838
|
+
A `Driver` over PGlite (`createPgliteDriver` from `@marianmeres/sqlkit/pglite`) is of the
|
|
839
|
+
`pg` dialect and is expected to work; it is not part of the test suite yet.
|
|
840
|
+
|
|
841
|
+
**Errors:** a statement the database refused rejects with sqlkit's `SqlError` (exported
|
|
842
|
+
by `@marianmeres/sqlkit`), unwrapped and untranslated. Its `code` is Postgres's SQLSTATE
|
|
843
|
+
(e.g. `22P05`), its message is Postgres's, and its `cause` is the driver library's own
|
|
844
|
+
error (node-postgres's, on a pool or a client). The one database error the adapter reads
|
|
845
|
+
is a non-numeric `incr`/`decr` target (`22P02`), which it turns into
|
|
846
|
+
`TypeError("KV value is not a number")`, as every adapter does. kv's own `TypeError` and
|
|
847
|
+
`RangeError`, for an invalid key or namespace, are unchanged.
|
|
777
848
|
|
|
778
849
|
**Table Schema:**
|
|
779
850
|
- `key` (VARCHAR 512) - Primary key
|
|
@@ -803,15 +874,20 @@ interface AdapterSqliteOptions extends AdapterAbstractOptions {
|
|
|
803
874
|
/**
|
|
804
875
|
* Interval in seconds for automatic cleanup of expired keys.
|
|
805
876
|
* Set to 0 to disable automatic cleanup.
|
|
877
|
+
* The timer never keeps the process alive; `destroy()` stops it.
|
|
806
878
|
*/
|
|
807
879
|
ttlCleanupIntervalSec: number;
|
|
808
880
|
}
|
|
809
881
|
```
|
|
810
882
|
|
|
883
|
+
A sqlkit `Driver` (e.g. from `openSqlite()`) is refused at construction with a
|
|
884
|
+
`TypeError`: the adapter takes the `DatabaseSync` itself.
|
|
885
|
+
|
|
811
886
|
**Connection ownership:** the adapter never opens, closes or reconfigures the handle — it
|
|
812
|
-
sets no pragmas.
|
|
813
|
-
|
|
814
|
-
|
|
887
|
+
sets no pragmas. Call `destroy()` before you close the handle: if the TTL cleanup finds it
|
|
888
|
+
closed, it logs one error and stops. For several processes on one file, open it with a busy
|
|
889
|
+
timeout (`new DatabaseSync(path, { timeout: 5000 })`) so a locked write waits instead of
|
|
890
|
+
failing, and consider `PRAGMA journal_mode = WAL`.
|
|
815
891
|
|
|
816
892
|
**Atomicity:** `setIfAbsent`, `cas`, `expire` and `set` are single statements.
|
|
817
893
|
`getSet`, `incr`/`decr`, `setMultiple`, `clear` and `transaction` run in `BEGIN IMMEDIATE`,
|
package/README.md
CHANGED
|
@@ -1,7 +1,6 @@
|
|
|
1
1
|
# @marianmeres/kv
|
|
2
2
|
|
|
3
3
|
[](https://www.npmjs.com/package/@marianmeres/kv)
|
|
4
|
-
[](https://jsr.io/@marianmeres/kv)
|
|
5
4
|
[](https://opensource.org/licenses/MIT)
|
|
6
5
|
|
|
7
6
|
Key-value storage abstraction layer with support for multiple backend adapters:
|
|
@@ -16,11 +15,15 @@ The API is inspired by the Redis API.
|
|
|
16
15
|
|
|
17
16
|
```sh
|
|
18
17
|
npm install @marianmeres/kv # or pnpm add
|
|
19
|
-
deno add jsr:@marianmeres/kv # Deno
|
|
20
18
|
```
|
|
21
19
|
|
|
22
|
-
|
|
23
|
-
|
|
20
|
+
Versions up to 3.1.1 are also on JSR. From 4.0 the package is published to npm only.
|
|
21
|
+
|
|
22
|
+
The `redis`, `postgres` and `sqlite` adapters take a client you create yourself: a
|
|
23
|
+
`redis` client; a `pg.Pool`, a `pg.Client` or a
|
|
24
|
+
[`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit) `Driver`; or a
|
|
25
|
+
`node:sqlite` `DatabaseSync`, which ships with Node. kv does not install `pg`: an app that
|
|
26
|
+
uses the postgres adapter has it already. Requires Node >= 24.12.
|
|
24
27
|
|
|
25
28
|
## Usage
|
|
26
29
|
|
|
@@ -98,20 +101,20 @@ type TtlResult =
|
|
|
98
101
|
|
|
99
102
|
## Adapter Feature Matrix
|
|
100
103
|
|
|
101
|
-
| Feature | Memory | Redis | PostgreSQL
|
|
102
|
-
| --------------------------------------------------------- | :-----------------: | :--------------------: |
|
|
103
|
-
| Persistent | ✗ | ✓ |
|
|
104
|
-
| TTL on `set()` | ✓ | ✓ |
|
|
105
|
-
| `ttl()` → `expires`/`no-ttl`/`missing` | ✓ | ✓ |
|
|
106
|
-
| `expire()` | ✓ | ✓ |
|
|
107
|
-
| Atomic `setIfAbsent` / `incr` / `decr` / `cas` / `getSet` | ✓ | ✓ (Lua) | ✓ (UPSERT)
|
|
108
|
-
| Sorted `keys()` | ✓ | ✓ |
|
|
109
|
-
| Streaming `keysIter()` | ✓ | ✓ (SCAN) |
|
|
110
|
-
| Per-pair TTL in `setMultiple` | ✓ | ✓ |
|
|
111
|
-
| Atomic `transaction()` | ✓ (single-threaded) | ✓ (MULTI) | ✓ (BEGIN/COMMIT)
|
|
112
|
-
| `delete()` returns real existed-flag | ✓ | ✓ |
|
|
113
|
-
| Background TTL cleanup | optional | native (Redis does it) | optional
|
|
114
|
-
| `keys()` / `clear()` | ✓ | ✓ (non-cluster only) |
|
|
104
|
+
| Feature | Memory | Redis | PostgreSQL | SQLite |
|
|
105
|
+
| --------------------------------------------------------- | :-----------------: | :--------------------: | :---------------: | :-----------------------: |
|
|
106
|
+
| Persistent | ✗ | ✓ | ✓ | ✓ (file database) |
|
|
107
|
+
| TTL on `set()` | ✓ | ✓ | ✓ | ✓ |
|
|
108
|
+
| `ttl()` → `expires`/`no-ttl`/`missing` | ✓ | ✓ | ✓ | ✓ |
|
|
109
|
+
| `expire()` | ✓ | ✓ | ✓ | ✓ |
|
|
110
|
+
| Atomic `setIfAbsent` / `incr` / `decr` / `cas` / `getSet` | ✓ | ✓ (Lua) | ✓ (UPSERT) | ✓ (UPSERT / IMMEDIATE tx) |
|
|
111
|
+
| Sorted `keys()` | ✓ | ✓ | ✓ | ✓ |
|
|
112
|
+
| Streaming `keysIter()` | ✓ | ✓ (SCAN) | ✓ (keyset paging) | ✓ (keyset paging) |
|
|
113
|
+
| Per-pair TTL in `setMultiple` | ✓ | ✓ | ✓ | ✓ |
|
|
114
|
+
| Atomic `transaction()` | ✓ (single-threaded) | ✓ (MULTI) | ✓ (BEGIN/COMMIT) | ✓ (BEGIN IMMEDIATE) |
|
|
115
|
+
| `delete()` returns real existed-flag | ✓ | ✓ | ✓ | ✓ |
|
|
116
|
+
| Background TTL cleanup | optional | native (Redis does it) | optional | optional |
|
|
117
|
+
| `keys()` / `clear()` | ✓ | ✓ (non-cluster only) | ✓ | ✓ |
|
|
115
118
|
|
|
116
119
|
## Adapter-Specific Limitations
|
|
117
120
|
|
|
@@ -123,9 +126,31 @@ type TtlResult =
|
|
|
123
126
|
|
|
124
127
|
### PostgreSQL
|
|
125
128
|
|
|
129
|
+
`db` takes a `pg.Pool`, a connected `pg.Client`, or a `Driver` from
|
|
130
|
+
[`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit) of the `pg`
|
|
131
|
+
dialect. An app that already builds one driver for everything that talks SQL passes it
|
|
132
|
+
here too:
|
|
133
|
+
|
|
134
|
+
```typescript
|
|
135
|
+
import { createPgDriver } from "@marianmeres/sqlkit/pg";
|
|
136
|
+
import pg from "pg";
|
|
137
|
+
|
|
138
|
+
const db = createPgDriver(new pg.Pool({ connectionString: process.env.DATABASE_URL }));
|
|
139
|
+
const client = createKVClient("my-app-namespace:", "postgres", { db });
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
- **The adapter never closes what it was given.** `destroy()` leaves the pool, the client
|
|
143
|
+
or the driver open; ending it is yours. Several adapters can share one.
|
|
144
|
+
- **`transaction()` runs on one connection.** On a `pg.Pool` it pins one for the whole
|
|
145
|
+
unit. On one `pg.Client` the calls take turns: another call waits for the transaction
|
|
146
|
+
to finish.
|
|
147
|
+
- **A call made inside a transaction you opened on the same `Driver` joins it**, and
|
|
148
|
+
commits or rolls back with it.
|
|
149
|
+
- **A statement the database refused rejects with sqlkit's `SqlError`**: it carries
|
|
150
|
+
Postgres's `code` and message, with node-postgres's error as `cause`. kv's own
|
|
151
|
+
`TypeError` and `RangeError` are unchanged.
|
|
126
152
|
- Creates a table (default: `__kv`) in your database.
|
|
127
153
|
- `tableName` must match `^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)?$` (word chars plus an optional `schema.` prefix).
|
|
128
|
-
- `transaction()` pins a single connection from the Pool for the whole BEGIN/…/COMMIT block — safe to call against a `pg.Pool`.
|
|
129
154
|
- Supports optional TTL cleanup via `ttlCleanupIntervalSec` option.
|
|
130
155
|
|
|
131
156
|
### SQLite
|
|
@@ -158,6 +183,69 @@ const client = createKVClient("my-app-namespace:", "sqlite", {
|
|
|
158
183
|
- Data is not persisted (in-memory only).
|
|
159
184
|
- Supports optional TTL cleanup via `ttlCleanupIntervalSec` option.
|
|
160
185
|
|
|
186
|
+
## Breaking Changes (4.0.0)
|
|
187
|
+
|
|
188
|
+
No signature changed, and a `pg.Pool` or `pg.Client` you pass today still works. What
|
|
189
|
+
changed is what gets installed and how the PostgreSQL adapter behaves at its edges. The
|
|
190
|
+
memory and Redis adapters do not change: if they are all you use, the bump needs nothing
|
|
191
|
+
from you. Each note says first what you have to do, if anything.
|
|
192
|
+
|
|
193
|
+
**What to install**
|
|
194
|
+
|
|
195
|
+
- **`pg` and `@types/pg` are no longer installed with kv; `@marianmeres/sqlkit` is.** If
|
|
196
|
+
you relied on kv to bring `pg` in, add it to your own `dependencies`. The postgres
|
|
197
|
+
adapter never loaded `pg` itself: it takes the pool or client you built, so an app that
|
|
198
|
+
uses it depends on `pg` already.
|
|
199
|
+
|
|
200
|
+
**What the postgres adapter's `db` takes**
|
|
201
|
+
|
|
202
|
+
- **`db` also takes a sqlkit `Driver` of the `pg` dialect.** Nothing to do; pass one if you
|
|
203
|
+
want your app and kv on one driver. The option's type, `PostgresDb`, is declared by
|
|
204
|
+
shape, and a `pg.Pool` and a `pg.Client` fit it as they are. A driver of another
|
|
205
|
+
dialect is refused at construction, with a `TypeError`.
|
|
206
|
+
- **A `db` that is neither a pool, a client nor a driver is refused at construction, with
|
|
207
|
+
a `TypeError`.** Nothing to do: such a `db` never worked. It used to fail at
|
|
208
|
+
`initialize()`.
|
|
209
|
+
- **The sqlite adapter refuses a sqlkit `Driver` at construction, with a `TypeError`.**
|
|
210
|
+
Pass a `node:sqlite` `DatabaseSync`, as before. A driver never worked there: it used to
|
|
211
|
+
fail at the first operation. Nothing else about the sqlite adapter changes.
|
|
212
|
+
|
|
213
|
+
**Errors**
|
|
214
|
+
|
|
215
|
+
- **A statement Postgres refused rejects with sqlkit's `SqlError`.** If your code tests
|
|
216
|
+
`instanceof` against node-postgres's error class, test for `SqlError` (exported by
|
|
217
|
+
`@marianmeres/sqlkit`) instead. `code` and the message are Postgres's, as before, and
|
|
218
|
+
node-postgres's error is the `cause`. kv's own `TypeError` and `RangeError` are
|
|
219
|
+
unchanged.
|
|
220
|
+
- **The adapter listens for the pool's or client's `error` event.** Nothing to do. An
|
|
221
|
+
idle connection the server dropped is reported through `process.emitWarning` when you
|
|
222
|
+
have no listener of your own. It used to end the process.
|
|
223
|
+
|
|
224
|
+
**One connection, and transactions**
|
|
225
|
+
|
|
226
|
+
- **On one `pg.Client`, calls take turns.** Nothing to do. A call issued while
|
|
227
|
+
`transaction()` runs waits for it to finish. It used to run inside that transaction and
|
|
228
|
+
could be rolled back with it.
|
|
229
|
+
- **A call issued inside a transaction you opened on the same `Driver` joins that
|
|
230
|
+
transaction.** Nothing to do, unless kv's writes must outlive your rollback: then make
|
|
231
|
+
them outside your transaction. Joining is what sharing a driver means.
|
|
232
|
+
- **`transaction()` checks its `COMMIT`.** Nothing to do. A `COMMIT` that did not commit
|
|
233
|
+
rejects, and a connection whose `ROLLBACK` failed is destroyed, not returned to the
|
|
234
|
+
pool.
|
|
235
|
+
|
|
236
|
+
**`keysIter()`**
|
|
237
|
+
|
|
238
|
+
- **On PostgreSQL, `keysIter()` pages by key, 500 at a time, in place of a server-side
|
|
239
|
+
cursor.** If you need a list consistent to one moment, use `keys()`. No connection is
|
|
240
|
+
held between yields, so a write made between two of them no longer waits for the walk
|
|
241
|
+
to end. The walk is no longer one snapshot: a key added or removed during it may or may
|
|
242
|
+
not be seen, as on Redis and SQLite.
|
|
243
|
+
|
|
244
|
+
**JSR**
|
|
245
|
+
|
|
246
|
+
- **The package is no longer published to JSR.** Install from npm. 3.1.1 is the last
|
|
247
|
+
version on JSR and stays there.
|
|
248
|
+
|
|
161
249
|
## Breaking Changes (3.0.0)
|
|
162
250
|
|
|
163
251
|
- **The Deno KV adapter is removed.** `"deno-kv"` is no longer an adapter type, and
|
|
@@ -245,15 +333,15 @@ Redis (SQLite runs in memory, nothing to set up). Copy `.env.example` to `.env`
|
|
|
245
333
|
in the connection details. Redis keys in the target database are deleted before each
|
|
246
334
|
test, so point it at a scratch database (the default is `/13`).
|
|
247
335
|
|
|
248
|
-
| Command | Does
|
|
249
|
-
| ----------------------------------- |
|
|
250
|
-
| `pnpm install` | installs dependencies and dev tooling
|
|
251
|
-
| `pnpm test` / `pnpm run test:watch` | `node --test` over `tests/**/*.test.ts`
|
|
252
|
-
| `pnpm run typecheck` | `tsc --noEmit` over `src` and `tests`
|
|
253
|
-
| `pnpm run lint` / `pnpm run format` | `biome check .` read-only / with `--write`
|
|
254
|
-
| `pnpm run ci` | typecheck + lint + test, what CI runs
|
|
255
|
-
| `pnpm run build` | `tsc -p tsconfig.build.json` → `dist/`
|
|
256
|
-
| `pnpm run rp` / `pnpm run rpm` | patch / minor release, then publish to npm
|
|
336
|
+
| Command | Does |
|
|
337
|
+
| ----------------------------------- | ------------------------------------------ |
|
|
338
|
+
| `pnpm install` | installs dependencies and dev tooling |
|
|
339
|
+
| `pnpm test` / `pnpm run test:watch` | `node --test` over `tests/**/*.test.ts` |
|
|
340
|
+
| `pnpm run typecheck` | `tsc --noEmit` over `src` and `tests` |
|
|
341
|
+
| `pnpm run lint` / `pnpm run format` | `biome check .` read-only / with `--write` |
|
|
342
|
+
| `pnpm run ci` | typecheck + lint + test, what CI runs |
|
|
343
|
+
| `pnpm run build` | `tsc -p tsconfig.build.json` → `dist/` |
|
|
344
|
+
| `pnpm run rp` / `pnpm run rpm` | patch / minor release, then publish to npm |
|
|
257
345
|
|
|
258
346
|
## License
|
|
259
347
|
|
package/dist/adapter/memory.d.ts
CHANGED
|
@@ -12,6 +12,7 @@ export interface AdapterMemoryOptions extends AdapterAbstractOptions {
|
|
|
12
12
|
* Interval in seconds for automatic cleanup of expired keys.
|
|
13
13
|
* Set to 0 to disable automatic cleanup (expired keys will still be
|
|
14
14
|
* lazily removed on access).
|
|
15
|
+
* The timer never keeps the process alive; `destroy()` stops it.
|
|
15
16
|
*/
|
|
16
17
|
ttlCleanupIntervalSec: number;
|
|
17
18
|
}
|
package/dist/adapter/memory.js
CHANGED
|
@@ -67,6 +67,9 @@ export class AdapterMemory extends AdapterAbstract {
|
|
|
67
67
|
}
|
|
68
68
|
// schedule next...
|
|
69
69
|
this.#cleanupTimer = setTimeout(this.#maybeTTLCleanup.bind(this), this.options.ttlCleanupIntervalSec * 1000);
|
|
70
|
+
// housekeeping must not hold the process open; destroy() still clears it
|
|
71
|
+
// (`?.`: browser and worker timers are plain numbers)
|
|
72
|
+
this.#cleanupTimer.unref?.();
|
|
70
73
|
}
|
|
71
74
|
}
|
|
72
75
|
#isExpired(key) {
|
|
@@ -3,14 +3,17 @@
|
|
|
3
3
|
*
|
|
4
4
|
* PostgreSQL key-value storage adapter implementation.
|
|
5
5
|
*/
|
|
6
|
-
import type
|
|
6
|
+
import { type PostgresDb } from "../utils/driver.ts";
|
|
7
7
|
import { AdapterAbstract, type AdapterAbstractOptions, type Operation, type SetMultipleEntry, type SetOptions, type TtlResult } from "./abstract.ts";
|
|
8
8
|
/**
|
|
9
9
|
* Configuration options for the PostgreSQL KV adapter.
|
|
10
10
|
*/
|
|
11
11
|
export interface AdapterPostgresOptions extends AdapterAbstractOptions {
|
|
12
|
-
/**
|
|
13
|
-
|
|
12
|
+
/**
|
|
13
|
+
* The database: a node-postgres `pg.Pool`, a connected `pg.Client`, or a sqlkit
|
|
14
|
+
* `Driver` of the `pg` dialect. The caller owns it: the adapter never closes it.
|
|
15
|
+
*/
|
|
16
|
+
db: PostgresDb;
|
|
14
17
|
/**
|
|
15
18
|
* Name of the table to use for storing key-value pairs.
|
|
16
19
|
* The table will be created automatically if it doesn't exist.
|
|
@@ -22,6 +25,7 @@ export interface AdapterPostgresOptions extends AdapterAbstractOptions {
|
|
|
22
25
|
/**
|
|
23
26
|
* Interval in seconds for automatic cleanup of expired keys.
|
|
24
27
|
* Set to 0 to disable automatic cleanup.
|
|
28
|
+
* The timer never keeps the process alive; `destroy()` stops it.
|
|
25
29
|
*/
|
|
26
30
|
ttlCleanupIntervalSec: number;
|
|
27
31
|
}
|
package/dist/adapter/postgres.js
CHANGED
|
@@ -4,7 +4,10 @@
|
|
|
4
4
|
* PostgreSQL key-value storage adapter implementation.
|
|
5
5
|
*/
|
|
6
6
|
import { createClog } from "@marianmeres/clog";
|
|
7
|
+
import { toPgDriver } from "../utils/driver.js";
|
|
7
8
|
import { AdapterAbstract, } from "./abstract.js";
|
|
9
|
+
/** Rows fetched per page by {@link AdapterPostgres.keysIter}. */
|
|
10
|
+
const KEYS_ITER_PAGE = 500;
|
|
8
11
|
/**
|
|
9
12
|
* PostgreSQL key-value storage adapter.
|
|
10
13
|
*
|
|
@@ -22,6 +25,8 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
22
25
|
validateKeys: true,
|
|
23
26
|
};
|
|
24
27
|
#cleanupTimer;
|
|
28
|
+
/** What every statement runs on: built from `options.db` by {@link toPgDriver}. */
|
|
29
|
+
#db;
|
|
25
30
|
constructor(namespace = "", options = {}) {
|
|
26
31
|
super(namespace);
|
|
27
32
|
this.options = Object.freeze({ ...this.options, ...(options || {}) });
|
|
@@ -33,6 +38,8 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
33
38
|
if (!/^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)?$/.test(this.options.tableName)) {
|
|
34
39
|
throw new Error(`Invalid tableName "${this.options.tableName}". Only word characters and a single "schema." prefix are allowed.`);
|
|
35
40
|
}
|
|
41
|
+
// last, so a constructor that throws leaves no driver (and no listener) behind
|
|
42
|
+
this.#db = toPgDriver(this.options.db);
|
|
36
43
|
}
|
|
37
44
|
/** Safe slug derived from tableName for use inside other identifiers (e.g. index names). */
|
|
38
45
|
get #safeSlug() {
|
|
@@ -40,8 +47,8 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
40
47
|
}
|
|
41
48
|
/** @inheritdoc */
|
|
42
49
|
async initialize() {
|
|
43
|
-
const {
|
|
44
|
-
await db.query(`
|
|
50
|
+
const { tableName } = this.options;
|
|
51
|
+
await this.#db.query(`
|
|
45
52
|
CREATE TABLE IF NOT EXISTS ${tableName} (
|
|
46
53
|
key VARCHAR(512) PRIMARY KEY,
|
|
47
54
|
value JSONB NOT NULL,
|
|
@@ -50,13 +57,13 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
50
57
|
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
|
|
51
58
|
)
|
|
52
59
|
`);
|
|
53
|
-
await db.query(`
|
|
60
|
+
await this.#db.query(`
|
|
54
61
|
CREATE INDEX IF NOT EXISTS idx_${this.#safeSlug}_expires_at
|
|
55
62
|
ON ${tableName} (expires_at)
|
|
56
63
|
WHERE expires_at IS NOT NULL
|
|
57
64
|
`);
|
|
58
65
|
// Index for efficient prefix lookups with LIKE 'prefix%' queries
|
|
59
|
-
await db.query(`
|
|
66
|
+
await this.#db.query(`
|
|
60
67
|
CREATE INDEX IF NOT EXISTS idx_${this.#safeSlug}_key_prefix
|
|
61
68
|
ON ${tableName} (key text_pattern_ops)
|
|
62
69
|
`);
|
|
@@ -65,30 +72,37 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
65
72
|
}
|
|
66
73
|
/** @inheritdoc */
|
|
67
74
|
async destroy(hard) {
|
|
68
|
-
// stop the timer first so a pending tick cannot fire during DROP
|
|
75
|
+
// stop the timer first so a pending tick cannot fire during DROP, and mark the
|
|
76
|
+
// adapter down before any await so an in-flight tick does not re-arm it
|
|
69
77
|
clearTimeout(this.#cleanupTimer);
|
|
70
78
|
this.#cleanupTimer = undefined;
|
|
79
|
+
this._initialized = false;
|
|
71
80
|
if (hard) {
|
|
72
|
-
const {
|
|
73
|
-
await db.query(`DROP TABLE IF EXISTS ${tableName}`);
|
|
81
|
+
const { tableName } = this.options;
|
|
82
|
+
await this.#db.query(`DROP TABLE IF EXISTS ${tableName}`);
|
|
74
83
|
}
|
|
75
|
-
this._initialized = false;
|
|
76
84
|
}
|
|
77
85
|
async #maybeTTLCleanup() {
|
|
78
86
|
clearTimeout(this.#cleanupTimer); // safety
|
|
79
87
|
if (this.options.ttlCleanupIntervalSec) {
|
|
80
|
-
const {
|
|
88
|
+
const { tableName, logger } = this.options;
|
|
81
89
|
// do the cleanup now
|
|
82
90
|
try {
|
|
83
|
-
await db.query(`
|
|
91
|
+
await this.#db.query(`
|
|
84
92
|
DELETE FROM ${tableName}
|
|
85
93
|
WHERE expires_at IS NOT NULL AND expires_at <= NOW()`);
|
|
86
94
|
}
|
|
87
95
|
catch (err) {
|
|
88
96
|
logger?.error?.(`TTL cleanup error: ${err}`);
|
|
89
97
|
}
|
|
98
|
+
// destroy() ran while the query was in flight
|
|
99
|
+
if (!this._initialized)
|
|
100
|
+
return;
|
|
90
101
|
// schedule next...
|
|
91
102
|
this.#cleanupTimer = setTimeout(this.#maybeTTLCleanup.bind(this), this.options.ttlCleanupIntervalSec * 1000);
|
|
103
|
+
// housekeeping must not hold the process open; destroy() still clears it
|
|
104
|
+
// (`?.`: browser and worker timers are plain numbers)
|
|
105
|
+
this.#cleanupTimer.unref?.();
|
|
92
106
|
}
|
|
93
107
|
}
|
|
94
108
|
/**
|
|
@@ -111,7 +125,7 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
111
125
|
}
|
|
112
126
|
/** @inheritdoc */
|
|
113
127
|
set(key, value, options = {}) {
|
|
114
|
-
return this.#setOn(this
|
|
128
|
+
return this.#setOn(this.#db, key, value, options);
|
|
115
129
|
}
|
|
116
130
|
async #setOn(exec, key, value, options = {}) {
|
|
117
131
|
this._assertInitialized();
|
|
@@ -131,12 +145,12 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
131
145
|
async setIfAbsent(key, value, options = {}) {
|
|
132
146
|
this._assertInitialized();
|
|
133
147
|
const nsKey = this._withNs(key);
|
|
134
|
-
const {
|
|
148
|
+
const { tableName } = this.options;
|
|
135
149
|
const ttl = this._resolveTtl(options);
|
|
136
150
|
const expiresAt = ttl ? new Date(Date.now() + ttl * 1000) : null;
|
|
137
151
|
// INSERT on new keys; UPDATE only when the existing row is expired.
|
|
138
152
|
// (Live keys stay untouched — the hallmark of setIfAbsent.)
|
|
139
|
-
const { rowCount } = await db.query(`INSERT INTO ${tableName} (key, value, expires_at, updated_at)
|
|
153
|
+
const { rowCount } = await this.#db.query(`INSERT INTO ${tableName} (key, value, expires_at, updated_at)
|
|
140
154
|
VALUES ($1, $2, $3, NOW())
|
|
141
155
|
ON CONFLICT (key) DO UPDATE SET
|
|
142
156
|
value = EXCLUDED.value,
|
|
@@ -149,11 +163,11 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
149
163
|
async getSet(key, value, options = {}) {
|
|
150
164
|
this._assertInitialized();
|
|
151
165
|
const nsKey = this._withNs(key);
|
|
152
|
-
const {
|
|
166
|
+
const { tableName } = this.options;
|
|
153
167
|
const ttl = this._resolveTtl(options);
|
|
154
168
|
const expiresAt = ttl ? new Date(Date.now() + ttl * 1000) : null;
|
|
155
169
|
// CTE captures pre-write state; UPSERT writes; RETURNING surfaces the prev.
|
|
156
|
-
const { rows } = await db.query(`WITH prev AS (
|
|
170
|
+
const { rows } = await this.#db.query(`WITH prev AS (
|
|
157
171
|
SELECT value FROM ${tableName}
|
|
158
172
|
WHERE key = $1 AND (expires_at IS NULL OR expires_at > NOW())
|
|
159
173
|
), upserted AS (
|
|
@@ -171,7 +185,7 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
171
185
|
async #incrBy(key, delta, options = {}) {
|
|
172
186
|
this._assertInitialized();
|
|
173
187
|
const nsKey = this._withNs(key);
|
|
174
|
-
const {
|
|
188
|
+
const { tableName } = this.options;
|
|
175
189
|
const ttl = this._resolveTtl(options);
|
|
176
190
|
const expiresAt = ttl ? new Date(Date.now() + ttl * 1000) : null;
|
|
177
191
|
// - New key OR expired key: value = delta, expires_at = $ttl
|
|
@@ -179,7 +193,7 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
179
193
|
// The nested cast `value::text::numeric` throws 22P02 when the stored
|
|
180
194
|
// value is not a JSON number — we translate that to TypeError.
|
|
181
195
|
try {
|
|
182
|
-
const { rows } = await db.query(`INSERT INTO ${tableName} (key, value, expires_at, updated_at)
|
|
196
|
+
const { rows } = await this.#db.query(`INSERT INTO ${tableName} (key, value, expires_at, updated_at)
|
|
183
197
|
VALUES ($1, to_jsonb($2::numeric), $3, NOW())
|
|
184
198
|
ON CONFLICT (key) DO UPDATE SET
|
|
185
199
|
value = CASE
|
|
@@ -216,7 +230,7 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
216
230
|
async cas(key, expected, next, options = {}) {
|
|
217
231
|
this._assertInitialized();
|
|
218
232
|
const nsKey = this._withNs(key);
|
|
219
|
-
const {
|
|
233
|
+
const { tableName } = this.options;
|
|
220
234
|
const hasTtl = options.ttl !== undefined;
|
|
221
235
|
const ttl = hasTtl ? this._resolveTtl(options) : undefined;
|
|
222
236
|
const expiresAt = ttl ? new Date(Date.now() + ttl * 1000) : null;
|
|
@@ -236,12 +250,12 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
236
250
|
const params = [nsKey, nextJson, expectedJson];
|
|
237
251
|
if (hasTtl)
|
|
238
252
|
params.push(expiresAt);
|
|
239
|
-
const { rowCount } = await db.query(sql, params);
|
|
253
|
+
const { rowCount } = await this.#db.query(sql, params);
|
|
240
254
|
return (rowCount ?? 0) > 0;
|
|
241
255
|
}
|
|
242
256
|
/** @inheritdoc */
|
|
243
257
|
get(key) {
|
|
244
|
-
return this.#getOn(this
|
|
258
|
+
return this.#getOn(this.#db, key);
|
|
245
259
|
}
|
|
246
260
|
async #getOn(exec, key) {
|
|
247
261
|
this._assertInitialized();
|
|
@@ -256,7 +270,7 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
256
270
|
}
|
|
257
271
|
/** @inheritdoc */
|
|
258
272
|
delete(key) {
|
|
259
|
-
return this.#deleteOn(this
|
|
273
|
+
return this.#deleteOn(this.#db, key);
|
|
260
274
|
}
|
|
261
275
|
async #deleteOn(exec, key) {
|
|
262
276
|
this._assertInitialized();
|
|
@@ -271,81 +285,58 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
271
285
|
async exists(key) {
|
|
272
286
|
this._assertInitialized();
|
|
273
287
|
key = this._withNs(key);
|
|
274
|
-
const {
|
|
275
|
-
const { rows } = await db.query(`SELECT 1 FROM ${tableName}
|
|
288
|
+
const { tableName } = this.options;
|
|
289
|
+
const { rows } = await this.#db.query(`SELECT 1 FROM ${tableName}
|
|
276
290
|
WHERE key = $1 AND (expires_at IS NULL OR expires_at > NOW())`, [key]);
|
|
277
291
|
return rows.length > 0;
|
|
278
292
|
}
|
|
279
293
|
/** @inheritdoc */
|
|
280
294
|
async keys(pattern) {
|
|
281
295
|
this._assertInitialized();
|
|
282
|
-
const {
|
|
283
|
-
const { rows } = await db.query(`SELECT key FROM ${tableName}
|
|
296
|
+
const { tableName } = this.options;
|
|
297
|
+
const { rows } = await this.#db.query(`SELECT key FROM ${tableName}
|
|
284
298
|
WHERE key LIKE $1${this.#likeEscape}
|
|
285
|
-
AND (expires_at IS NULL OR expires_at > NOW())
|
|
286
|
-
ORDER BY key`, [this.#withNsLike(pattern)]);
|
|
299
|
+
AND (expires_at IS NULL OR expires_at > NOW())`, [this.#withNsLike(pattern)]);
|
|
287
300
|
return rows.map((row) => this._withoutNs(row.key)).toSorted();
|
|
288
301
|
}
|
|
289
302
|
/** @inheritdoc */
|
|
290
303
|
async *keysIter(pattern) {
|
|
291
304
|
this._assertInitialized();
|
|
292
305
|
const { tableName } = this.options;
|
|
293
|
-
const
|
|
294
|
-
//
|
|
295
|
-
//
|
|
296
|
-
|
|
297
|
-
let
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
for (const row of rows) {
|
|
311
|
-
yield this._withoutNs(row.key);
|
|
312
|
-
}
|
|
306
|
+
const like = this.#withNsLike(pattern);
|
|
307
|
+
// Keyset pages in byte order, which is the order of the text_pattern_ops index
|
|
308
|
+
// initialize() creates: each page is one range scan of it. Nothing stays open
|
|
309
|
+
// between yields, so an early break leaves nothing to release.
|
|
310
|
+
let after;
|
|
311
|
+
while (true) {
|
|
312
|
+
const { rows } = after === undefined
|
|
313
|
+
? await this.#db.query(`SELECT key FROM ${tableName}
|
|
314
|
+
WHERE key LIKE $1${this.#likeEscape}
|
|
315
|
+
AND (expires_at IS NULL OR expires_at > NOW())
|
|
316
|
+
ORDER BY key USING ~<~ LIMIT ${KEYS_ITER_PAGE}`, [like])
|
|
317
|
+
: await this.#db.query(`SELECT key FROM ${tableName}
|
|
318
|
+
WHERE key LIKE $1${this.#likeEscape} AND key ~>~ $2
|
|
319
|
+
AND (expires_at IS NULL OR expires_at > NOW())
|
|
320
|
+
ORDER BY key USING ~<~ LIMIT ${KEYS_ITER_PAGE}`, [like, after]);
|
|
321
|
+
for (const row of rows) {
|
|
322
|
+
yield this._withoutNs(row.key);
|
|
313
323
|
}
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
try {
|
|
318
|
-
await client.query(`CLOSE ${cursorName}`);
|
|
319
|
-
}
|
|
320
|
-
catch {
|
|
321
|
-
// cursor may already be gone if the txn aborted
|
|
322
|
-
}
|
|
323
|
-
try {
|
|
324
|
-
await client.query("COMMIT");
|
|
325
|
-
}
|
|
326
|
-
catch {
|
|
327
|
-
try {
|
|
328
|
-
await client.query("ROLLBACK");
|
|
329
|
-
}
|
|
330
|
-
catch {
|
|
331
|
-
// ignore — caller will see the original error
|
|
332
|
-
}
|
|
333
|
-
}
|
|
334
|
-
}
|
|
335
|
-
release();
|
|
324
|
+
if (rows.length < KEYS_ITER_PAGE)
|
|
325
|
+
return;
|
|
326
|
+
after = rows[rows.length - 1].key;
|
|
336
327
|
}
|
|
337
328
|
}
|
|
338
329
|
/** @inheritdoc */
|
|
339
330
|
async clear(pattern) {
|
|
340
331
|
this._assertInitialized();
|
|
341
|
-
const {
|
|
332
|
+
const { tableName } = this.options;
|
|
342
333
|
const likePattern = this.#withNsLike(pattern);
|
|
343
334
|
// Batch large deletions to prevent long-running transactions
|
|
344
335
|
const batchSize = 10000;
|
|
345
336
|
let totalDeleted = 0;
|
|
346
337
|
let deleted;
|
|
347
338
|
do {
|
|
348
|
-
const { rowCount } = await db.query(`DELETE FROM ${tableName} WHERE key IN (
|
|
339
|
+
const { rowCount } = await this.#db.query(`DELETE FROM ${tableName} WHERE key IN (
|
|
349
340
|
SELECT key FROM ${tableName} WHERE key LIKE $1${this.#likeEscape} LIMIT $2
|
|
350
341
|
)`, [likePattern, batchSize]);
|
|
351
342
|
deleted = rowCount ?? 0;
|
|
@@ -359,7 +350,7 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
359
350
|
if (entries.length === 0)
|
|
360
351
|
return [];
|
|
361
352
|
const normalized = this._normalizePairs(entries);
|
|
362
|
-
const {
|
|
353
|
+
const { tableName } = this.options;
|
|
363
354
|
const placeholders = [];
|
|
364
355
|
const params = [];
|
|
365
356
|
let idx = 1;
|
|
@@ -377,7 +368,7 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
377
368
|
expires_at = EXCLUDED.expires_at,
|
|
378
369
|
updated_at = NOW()
|
|
379
370
|
`;
|
|
380
|
-
await db.query(sql, params);
|
|
371
|
+
await this.#db.query(sql, params);
|
|
381
372
|
return normalized.map(() => true);
|
|
382
373
|
}
|
|
383
374
|
/** @inheritdoc */
|
|
@@ -386,14 +377,14 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
386
377
|
if (keys.length === 0)
|
|
387
378
|
return {};
|
|
388
379
|
const placeholders = keys.map((_, i) => `$${i + 1}`).join(", ");
|
|
389
|
-
const {
|
|
380
|
+
const { tableName } = this.options;
|
|
390
381
|
const sql = `
|
|
391
382
|
SELECT key, value
|
|
392
383
|
FROM ${tableName}
|
|
393
384
|
WHERE key IN (${placeholders}) AND (expires_at IS NULL OR expires_at > NOW())
|
|
394
385
|
`;
|
|
395
386
|
const params = keys.map((k) => this._withNs(k));
|
|
396
|
-
const { rows } = await db.query(sql, params);
|
|
387
|
+
const { rows } = await this.#db.query(sql, params);
|
|
397
388
|
const found = Object.create(null);
|
|
398
389
|
for (const row of rows) {
|
|
399
390
|
found[this._withoutNs(row.key)] = row.value;
|
|
@@ -409,9 +400,9 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
409
400
|
async expire(key, ttl) {
|
|
410
401
|
this._assertInitialized();
|
|
411
402
|
key = this._withNs(key);
|
|
412
|
-
const {
|
|
403
|
+
const { tableName } = this.options;
|
|
413
404
|
const expiresAt = new Date(Date.now() + ttl * 1000);
|
|
414
|
-
const { rowCount } = await db.query(`UPDATE ${tableName}
|
|
405
|
+
const { rowCount } = await this.#db.query(`UPDATE ${tableName}
|
|
415
406
|
SET expires_at = $2, updated_at = NOW()
|
|
416
407
|
WHERE key = $1 AND (expires_at IS NULL OR expires_at > NOW())`, [key, expiresAt]);
|
|
417
408
|
return rowCount > 0;
|
|
@@ -420,8 +411,8 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
420
411
|
async ttl(key) {
|
|
421
412
|
this._assertInitialized();
|
|
422
413
|
key = this._withNs(key);
|
|
423
|
-
const {
|
|
424
|
-
const { rows } = await db.query(`SELECT expires_at FROM ${tableName} WHERE key = $1`, [key]);
|
|
414
|
+
const { tableName } = this.options;
|
|
415
|
+
const { rows } = await this.#db.query(`SELECT expires_at FROM ${tableName} WHERE key = $1`, [key]);
|
|
425
416
|
if (rows.length === 0)
|
|
426
417
|
return { state: "missing" };
|
|
427
418
|
const expiresAt = rows[0].expires_at;
|
|
@@ -433,60 +424,34 @@ export class AdapterPostgres extends AdapterAbstract {
|
|
|
433
424
|
return { state: "missing" };
|
|
434
425
|
return { state: "expires", at };
|
|
435
426
|
}
|
|
436
|
-
/**
|
|
437
|
-
* Acquire a single pinned connection for a transaction.
|
|
438
|
-
*/
|
|
439
|
-
async #acquireClient() {
|
|
440
|
-
const { db } = this.options;
|
|
441
|
-
if (typeof db.totalCount === "number" && typeof db.connect === "function") {
|
|
442
|
-
const client = await db.connect();
|
|
443
|
-
return { client, release: () => client.release() };
|
|
444
|
-
}
|
|
445
|
-
return { client: db, release: () => { } };
|
|
446
|
-
}
|
|
447
427
|
/** @inheritdoc */
|
|
448
428
|
async transaction(operations) {
|
|
449
|
-
this._assertInitialized();
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
break;
|
|
467
|
-
}
|
|
429
|
+
this._assertInitialized(); // `async` keeps this a rejection
|
|
430
|
+
// The driver pins one connection (or, on one client, holds every other caller
|
|
431
|
+
// back), checks the COMMIT, and rolls back when the callback rejects. Called from
|
|
432
|
+
// inside a transaction the host opened on the same driver, this one joins it.
|
|
433
|
+
return this.#db.transaction(async (tx) => {
|
|
434
|
+
const results = [];
|
|
435
|
+
for (const op of operations) {
|
|
436
|
+
switch (op.type) {
|
|
437
|
+
case "set":
|
|
438
|
+
results.push(await this.#setOn(tx, op.key, op.value, op.options || {}));
|
|
439
|
+
break;
|
|
440
|
+
case "get":
|
|
441
|
+
results.push(await this.#getOn(tx, op.key));
|
|
442
|
+
break;
|
|
443
|
+
case "delete":
|
|
444
|
+
results.push(await this.#deleteOn(tx, op.key));
|
|
445
|
+
break;
|
|
468
446
|
}
|
|
469
|
-
await client.query("COMMIT");
|
|
470
447
|
}
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
await client.query("ROLLBACK");
|
|
474
|
-
}
|
|
475
|
-
catch (rollbackErr) {
|
|
476
|
-
logger?.error?.(`Rollback failed: ${rollbackErr}`);
|
|
477
|
-
}
|
|
478
|
-
throw err;
|
|
479
|
-
}
|
|
480
|
-
}
|
|
481
|
-
finally {
|
|
482
|
-
release();
|
|
483
|
-
}
|
|
484
|
-
return results;
|
|
448
|
+
return results;
|
|
449
|
+
});
|
|
485
450
|
}
|
|
486
451
|
async __debug_dump() {
|
|
487
452
|
this._assertInitialized();
|
|
488
|
-
const {
|
|
489
|
-
const { rows } = await db.query(`SELECT key, value, expires_at FROM ${tableName} WHERE key LIKE $1${this.#likeEscape}`, [this.#withNsLike("*")]);
|
|
453
|
+
const { tableName } = this.options;
|
|
454
|
+
const { rows } = await this.#db.query(`SELECT key, value, expires_at FROM ${tableName} WHERE key LIKE $1${this.#likeEscape}`, [this.#withNsLike("*")]);
|
|
490
455
|
return rows.reduce((m, row) => {
|
|
491
456
|
m[this._withoutNs(row.key)] = {
|
|
492
457
|
value: row.value,
|
package/dist/adapter/sqlite.d.ts
CHANGED
|
@@ -24,6 +24,7 @@ export interface AdapterSqliteOptions extends AdapterAbstractOptions {
|
|
|
24
24
|
/**
|
|
25
25
|
* Interval in seconds for automatic cleanup of expired keys.
|
|
26
26
|
* Set to 0 to disable automatic cleanup.
|
|
27
|
+
* The timer never keeps the process alive; `destroy()` stops it.
|
|
27
28
|
*/
|
|
28
29
|
ttlCleanupIntervalSec: number;
|
|
29
30
|
}
|
package/dist/adapter/sqlite.js
CHANGED
|
@@ -4,6 +4,7 @@
|
|
|
4
4
|
* SQLite key-value storage adapter implementation (`node:sqlite`).
|
|
5
5
|
*/
|
|
6
6
|
import { createClog } from "@marianmeres/clog";
|
|
7
|
+
import { isDriver } from "../utils/driver.js";
|
|
7
8
|
import { AdapterAbstract, } from "./abstract.js";
|
|
8
9
|
/** Rows fetched per page by {@link AdapterSqlite.keysIter}. */
|
|
9
10
|
const KEYS_ITER_PAGE = 500;
|
|
@@ -53,6 +54,9 @@ export class AdapterSqlite extends AdapterAbstract {
|
|
|
53
54
|
if (!this.options.db) {
|
|
54
55
|
throw new Error("Missing sqlite database instance");
|
|
55
56
|
}
|
|
57
|
+
if (isDriver(this.options.db)) {
|
|
58
|
+
throw new TypeError("The sqlite adapter takes a node:sqlite DatabaseSync; a sqlkit Driver is not supported yet");
|
|
59
|
+
}
|
|
56
60
|
// tableName is spliced into SQL verbatim, so guard against surprises.
|
|
57
61
|
if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(this.options.tableName)) {
|
|
58
62
|
throw new Error(`Invalid tableName "${this.options.tableName}". Only word characters are allowed.`);
|
|
@@ -134,7 +138,13 @@ export class AdapterSqlite extends AdapterAbstract {
|
|
|
134
138
|
#maybeTTLCleanup() {
|
|
135
139
|
clearTimeout(this.#cleanupTimer); // safety
|
|
136
140
|
if (this.options.ttlCleanupIntervalSec) {
|
|
137
|
-
const { tableName, logger } = this.options;
|
|
141
|
+
const { db, tableName, logger } = this.options;
|
|
142
|
+
// The caller closed the handle without destroy(). Every later tick would fail
|
|
143
|
+
// the same way, so say it once and stop.
|
|
144
|
+
if (!db.isOpen) {
|
|
145
|
+
logger?.error?.("TTL cleanup stopped: the database handle was closed before destroy()");
|
|
146
|
+
return;
|
|
147
|
+
}
|
|
138
148
|
// do the cleanup now
|
|
139
149
|
try {
|
|
140
150
|
this.#stmt(`DELETE FROM ${tableName}
|
|
@@ -145,6 +155,9 @@ export class AdapterSqlite extends AdapterAbstract {
|
|
|
145
155
|
}
|
|
146
156
|
// schedule next...
|
|
147
157
|
this.#cleanupTimer = setTimeout(this.#maybeTTLCleanup.bind(this), this.options.ttlCleanupIntervalSec * 1000);
|
|
158
|
+
// housekeeping must not hold the process open; destroy() still clears it
|
|
159
|
+
// (`?.`: browser and worker timers are plain numbers)
|
|
160
|
+
this.#cleanupTimer.unref?.();
|
|
148
161
|
}
|
|
149
162
|
}
|
|
150
163
|
/**
|
package/dist/kv.d.ts
CHANGED
|
@@ -16,6 +16,7 @@ export { AdapterMemory, type AdapterMemoryOptions } from "./adapter/memory.ts";
|
|
|
16
16
|
export { AdapterPostgres, type AdapterPostgresOptions } from "./adapter/postgres.ts";
|
|
17
17
|
export { AdapterRedis, type AdapterRedisOptions } from "./adapter/redis.ts";
|
|
18
18
|
export { AdapterSqlite, type AdapterSqliteOptions } from "./adapter/sqlite.ts";
|
|
19
|
+
export type { PostgresDb } from "./utils/driver.ts";
|
|
19
20
|
export { sleep } from "./utils/sleep.ts";
|
|
20
21
|
/**
|
|
21
22
|
* Type mapping for all supported KV adapter types.
|
|
@@ -52,7 +53,8 @@ interface KnownTypes {
|
|
|
52
53
|
* @param type - The storage backend to use:
|
|
53
54
|
* - `"memory"` - In-memory storage (default, not persisted)
|
|
54
55
|
* - `"redis"` - Redis server (requires `db` option with Redis client)
|
|
55
|
-
* - `"postgres"` - PostgreSQL database (requires `db` option
|
|
56
|
+
* - `"postgres"` - PostgreSQL database (requires `db` option: a `pg.Pool`, a
|
|
57
|
+
* connected `pg.Client`, or a sqlkit `Driver` of the `pg` dialect)
|
|
56
58
|
* - `"sqlite"` - SQLite database (requires `db` option with a `node:sqlite` `DatabaseSync`)
|
|
57
59
|
* @param options - Adapter-specific configuration options.
|
|
58
60
|
*
|
package/dist/kv.js
CHANGED
|
@@ -29,7 +29,8 @@ export { sleep } from "./utils/sleep.js";
|
|
|
29
29
|
* @param type - The storage backend to use:
|
|
30
30
|
* - `"memory"` - In-memory storage (default, not persisted)
|
|
31
31
|
* - `"redis"` - Redis server (requires `db` option with Redis client)
|
|
32
|
-
* - `"postgres"` - PostgreSQL database (requires `db` option
|
|
32
|
+
* - `"postgres"` - PostgreSQL database (requires `db` option: a `pg.Pool`, a
|
|
33
|
+
* connected `pg.Client`, or a sqlkit `Driver` of the `pg` dialect)
|
|
33
34
|
* - `"sqlite"` - SQLite database (requires `db` option with a `node:sqlite` `DatabaseSync`)
|
|
34
35
|
* @param options - Adapter-specific configuration options.
|
|
35
36
|
*
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @module
|
|
3
|
+
*
|
|
4
|
+
* The one seam between the postgres adapter and the database: whatever the host passed
|
|
5
|
+
* as `db` becomes a sqlkit `Driver`, and every statement the adapter sends goes through it.
|
|
6
|
+
*/
|
|
7
|
+
import type { Driver } from "@marianmeres/sqlkit";
|
|
8
|
+
import { type PgPoolLike, type PgStandaloneClientLike } from "@marianmeres/sqlkit/pg";
|
|
9
|
+
/**
|
|
10
|
+
* What the postgres adapter's `db` option takes: a node-postgres `pg.Pool`, a
|
|
11
|
+
* connected `pg.Client`, or a sqlkit `Driver` of the `pg` dialect. The first two are
|
|
12
|
+
* declared by shape, so nothing here imports `pg`.
|
|
13
|
+
*/
|
|
14
|
+
export type PostgresDb = PgPoolLike | PgStandaloneClientLike | Driver;
|
|
15
|
+
/** Whether `db` is a sqlkit `Driver` (by shape). */
|
|
16
|
+
export declare function isDriver(db: unknown): db is Driver;
|
|
17
|
+
/** A `pg.Pool` has `connect()` and counts its clients; a `pg.Client` does not. */
|
|
18
|
+
export declare function isPool(db: unknown): db is PgPoolLike;
|
|
19
|
+
/**
|
|
20
|
+
* The `Driver` the postgres adapter runs on. A host's `Driver` is used as it is; a
|
|
21
|
+
* pool or a client gets one driver per object. The adapter never closes any of them.
|
|
22
|
+
*
|
|
23
|
+
* @throws {TypeError} For a `Driver` of another dialect.
|
|
24
|
+
*/
|
|
25
|
+
export declare function toPgDriver(db: PostgresDb): Driver;
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @module
|
|
3
|
+
*
|
|
4
|
+
* The one seam between the postgres adapter and the database: whatever the host passed
|
|
5
|
+
* as `db` becomes a sqlkit `Driver`, and every statement the adapter sends goes through it.
|
|
6
|
+
*/
|
|
7
|
+
import { createPgClientDriver, createPgDriver, } from "@marianmeres/sqlkit/pg";
|
|
8
|
+
/** One driver per pool or client object, however many adapters share it. */
|
|
9
|
+
const drivers = new WeakMap();
|
|
10
|
+
/** Whether `db` is a sqlkit `Driver` (by shape). */
|
|
11
|
+
export function isDriver(db) {
|
|
12
|
+
const d = db;
|
|
13
|
+
return (typeof d === "object" &&
|
|
14
|
+
d !== null &&
|
|
15
|
+
typeof d.dialect === "string" &&
|
|
16
|
+
typeof d.query === "function" &&
|
|
17
|
+
typeof d.exec === "function" &&
|
|
18
|
+
typeof d.transaction === "function" &&
|
|
19
|
+
typeof d.capabilities === "object" &&
|
|
20
|
+
d.capabilities !== null);
|
|
21
|
+
}
|
|
22
|
+
/** A `pg.Pool` has `connect()` and counts its clients; a `pg.Client` does not. */
|
|
23
|
+
export function isPool(db) {
|
|
24
|
+
return (typeof db === "object" &&
|
|
25
|
+
db !== null &&
|
|
26
|
+
typeof db.connect === "function" &&
|
|
27
|
+
typeof db.totalCount === "number");
|
|
28
|
+
}
|
|
29
|
+
/**
|
|
30
|
+
* The `Driver` the postgres adapter runs on. A host's `Driver` is used as it is; a
|
|
31
|
+
* pool or a client gets one driver per object. The adapter never closes any of them.
|
|
32
|
+
*
|
|
33
|
+
* @throws {TypeError} For a `Driver` of another dialect.
|
|
34
|
+
*/
|
|
35
|
+
export function toPgDriver(db) {
|
|
36
|
+
if (isDriver(db)) {
|
|
37
|
+
if (db.dialect !== "pg") {
|
|
38
|
+
throw new TypeError(`The postgres adapter runs on PostgreSQL: got a "${db.dialect}" driver (${db.adapter})`);
|
|
39
|
+
}
|
|
40
|
+
return db;
|
|
41
|
+
}
|
|
42
|
+
let driver = drivers.get(db);
|
|
43
|
+
if (driver === undefined) {
|
|
44
|
+
driver = isPool(db) ? createPgDriver(db) : createPgClientDriver(db);
|
|
45
|
+
drivers.set(db, driver);
|
|
46
|
+
}
|
|
47
|
+
return driver;
|
|
48
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@marianmeres/kv",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "4.0.0",
|
|
4
4
|
"description": "Key-value storage abstraction with a unified, Redis-inspired API over memory, Redis, PostgreSQL and SQLite backends.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "Marian Meres",
|
|
@@ -44,13 +44,14 @@
|
|
|
44
44
|
],
|
|
45
45
|
"dependencies": {
|
|
46
46
|
"@marianmeres/clog": "^3.23.0",
|
|
47
|
-
"@
|
|
48
|
-
"pg": "^8.23.1",
|
|
47
|
+
"@marianmeres/sqlkit": "^0.1.1",
|
|
49
48
|
"redis": "^6.3.1"
|
|
50
49
|
},
|
|
51
50
|
"devDependencies": {
|
|
52
51
|
"@biomejs/biome": "^2.6.4",
|
|
53
52
|
"@types/node": "^26.6.6",
|
|
53
|
+
"@types/pg": "^8.23.1",
|
|
54
|
+
"pg": "^8.23.1",
|
|
54
55
|
"typescript": "^7.0.2"
|
|
55
56
|
},
|
|
56
57
|
"scripts": {
|
|
@@ -59,14 +60,11 @@
|
|
|
59
60
|
"typecheck": "tsc --noEmit",
|
|
60
61
|
"lint": "biome check .",
|
|
61
62
|
"format": "biome check --write .",
|
|
62
|
-
"test": "node --test --env-file-if-exists=.env \"tests/**/*.test.ts\"",
|
|
63
|
-
"test:watch": "node --test --watch --env-file-if-exists=.env \"tests/**/*.test.ts\"",
|
|
64
|
-
"test:deno": "deno test -A --no-check --env-file tests/",
|
|
63
|
+
"test": "node --test --test-concurrency=1 --env-file-if-exists=.env \"tests/**/*.test.ts\"",
|
|
64
|
+
"test:watch": "node --test --test-concurrency=1 --watch --env-file-if-exists=.env \"tests/**/*.test.ts\"",
|
|
65
65
|
"ci": "pnpm run typecheck && pnpm run lint && pnpm test",
|
|
66
66
|
"release": "npx --yes @marianmeres/release@^2.2.0",
|
|
67
|
-
"
|
|
68
|
-
"
|
|
69
|
-
"rp": "pnpm run release -y && pnpm run publish:all",
|
|
70
|
-
"rpm": "pnpm run release minor -y && pnpm run publish:all"
|
|
67
|
+
"rp": "pnpm run release -y && pnpm publish",
|
|
68
|
+
"rpm": "pnpm run release minor -y && pnpm publish"
|
|
71
69
|
}
|
|
72
70
|
}
|