@marianmeres/kv 4.0.0 → 4.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +108 -28
- package/API.md +123 -26
- package/README.md +113 -8
- package/dist/adapter/postgres.js +16 -6
- package/dist/adapter/sqlite.d.ts +23 -7
- package/dist/adapter/sqlite.js +228 -107
- package/dist/kv.d.ts +9 -1
- package/dist/kv.js +8 -1
- package/dist/utils/driver.d.ts +2 -1
- package/dist/utils/driver.js +7 -1
- package/dist/utils/sqlite-driver.d.ts +37 -0
- package/dist/utils/sqlite-driver.js +67 -0
- package/package.json +9 -8
package/AGENTS.md
CHANGED
|
@@ -5,7 +5,7 @@ Machine-friendly reference for AI agents working with this codebase.
|
|
|
5
5
|
## Quick Reference
|
|
6
6
|
|
|
7
7
|
- **Stack**: TypeScript (erasable-only), Node LTS (`.node-version`), pnpm, `node:test`, Biome.
|
|
8
|
-
- **Install**: `pnpm install` | **Test**: `pnpm test` (needs Postgres + Redis; SQLite
|
|
8
|
+
- **Install**: `pnpm install` | **Test**: `pnpm test` (needs Postgres + Redis; SQLite and PGlite run in memory;
|
|
9
9
|
test files run one at a time) |
|
|
10
10
|
**Check all**: `pnpm run ci` | **Build**: `pnpm run build`
|
|
11
11
|
- **Published**: npm (`dist/`, built by `tsc`)
|
|
@@ -25,17 +25,26 @@ src/
|
|
|
25
25
|
│ ├── memory.ts — In-memory adapter
|
|
26
26
|
│ ├── redis.ts — Redis adapter
|
|
27
27
|
│ ├── postgres.ts — PostgreSQL adapter
|
|
28
|
-
│ └── sqlite.ts — SQLite adapter (`node:sqlite
|
|
28
|
+
│ └── sqlite.ts — SQLite adapter (`node:sqlite`, through a sqlkit driver)
|
|
29
29
|
└── utils/
|
|
30
|
-
├── driver.ts — `toPgDriver()`: the postgres adapter's `db` as a sqlkit `Driver
|
|
30
|
+
├── driver.ts — `toPgDriver()`: the postgres adapter's `db` as a sqlkit `Driver`;
|
|
31
|
+
│ `isDriver()`, which the sqlite adapter shares
|
|
32
|
+
├── sqlite-driver.ts — `ownSqliteDriver()`: the sqlite adapter's handle as kv's own
|
|
33
|
+
│ sqlkit `Driver`, its module imported lazily; `SqliteDb`, and
|
|
34
|
+
│ `assertSqliteDriver()` for a host's `Driver`
|
|
31
35
|
└── sleep.ts — Sleep utility function
|
|
32
36
|
|
|
33
37
|
tests/
|
|
34
|
-
├── kv.test.ts — Main test file (node:test); every test runs against all adapters
|
|
38
|
+
├── kv.test.ts — Main test file (node:test); every test runs against all adapters,
|
|
39
|
+
│ the postgres adapter on PGlite and the sqlite adapter on a
|
|
40
|
+
│ sqlkit Driver too
|
|
35
41
|
├── driver.test.ts — The postgres adapter on a pool, a client and a sqlkit Driver
|
|
42
|
+
│ (over node-postgres and over PGlite), and the sqlite adapter
|
|
43
|
+
│ on a sqlkit Driver the host built
|
|
36
44
|
├── _tests-runner.ts — Test harness: builds one client per adapter per test
|
|
37
45
|
├── _redis.ts — Redis test helper
|
|
38
46
|
├── _pg.ts — PostgreSQL test helper
|
|
47
|
+
├── _pglite.ts — PGlite test helper (one in-memory PGlite per test file, as a sqlkit Driver)
|
|
39
48
|
├── _sqlite.ts — SQLite test helper (fresh `:memory:` database per test)
|
|
40
49
|
└── _no-destroy-child.ts — child process: an adapter left running must not hold it open
|
|
41
50
|
|
|
@@ -50,7 +59,9 @@ dist/ — `tsc -p tsconfig.build.json` output; gitignored; the
|
|
|
50
59
|
1. **Runtime policy** — Node at the keyboard and in production, npm as the registry. No
|
|
51
60
|
`Deno.*`, `Bun.*`, `bun:*`, `jsr:` or `npm:` specifiers. Web standards first; `node:*`
|
|
52
61
|
where they don't reach. Nothing is built, tested or documented for Bun or Deno.
|
|
53
|
-
Published to npm only; 3.1.1 was the last version on JSR.
|
|
62
|
+
Published to npm only; 3.1.1 was the last version on JSR. `engines.node` is
|
|
63
|
+
`@marianmeres/sqlkit`'s (`^24.20.0 || >=26.6.0`): kv installs it, so kv's floor is
|
|
64
|
+
never below it (convention 24).
|
|
54
65
|
2. **Erasable TypeScript** — no `enum`, `namespace`, constructor parameter properties;
|
|
55
66
|
`import type` for types; explicit `.ts` import extensions (rewritten to `.js` on emit).
|
|
56
67
|
Node strips types and never checks them: `pnpm run typecheck` is the only type gate.
|
|
@@ -67,47 +78,107 @@ dist/ — `tsc -p tsconfig.build.json` output; gitignored; the
|
|
|
67
78
|
8. Pattern matching: Redis-style (`*` = any chars, `?` = single char). All other characters — including `.`, `(`, `[`, `%`, `_` — are matched **literally**.
|
|
68
79
|
9. TTL resolution: `options.ttl ?? defaultTtl`; `0` or negative disables expiration for that call (explicitly overrides `defaultTtl`).
|
|
69
80
|
10. Shared helpers live in `abstract.ts` — use `_globToRegex()`, `_resolveTtl()`, `_normalizePairs()`; do not re-implement per-adapter. The SQL adapters keep their own pattern escaping next to the SQL that uses it: Postgres `#likePattern` (LIKE, `ESCAPE '\'`), SQLite `#globPattern` (GLOB — case-sensitive, unlike SQLite's LIKE; `[` → `[[]`). Both escape the namespace as a literal prefix.
|
|
70
|
-
11. Postgres `transaction()` is `this.#db.
|
|
71
|
-
(or, on one `pg.Client`, holds every
|
|
72
|
-
back
|
|
73
|
-
|
|
74
|
-
|
|
81
|
+
11. Postgres `transaction()` is `this.#db.savepoint()`. Outside a transaction that is the
|
|
82
|
+
driver's `transaction()`: it pins the connection (or, on one `pg.Client`, holds every
|
|
83
|
+
other caller back), checks the `COMMIT` and rolls back. Inside a transaction the host
|
|
84
|
+
opened on the same `Driver` it is a savepoint of it, so a failed unit is undone alone
|
|
85
|
+
and the host's transaction goes on, as on SQLite (convention 19). Nothing in the
|
|
86
|
+
adapter sends `BEGIN`, `COMMIT`, `ROLLBACK` or `SAVEPOINT`. Pass the callback's `tx`
|
|
87
|
+
to the `…On` helpers.
|
|
88
|
+
12. Redis adapter owns the connection iff `initialize()` opened it (`#weOpened` flag). The Postgres and SQLite adapters never close the `db` they were given; the Postgres adapter never calls `close()` on a driver, its own or the host's (it would end the host's pool or client). The SQLite adapter never calls `close()` on a driver either, its own or the host's: sqlkit's `createSqliteDriver(db).close()` closes the host's handle.
|
|
75
89
|
13. Key validation lives in `_withNs()` — enforced when `options.validateKeys !== false`. Reject empty strings, keys with `\0`, and total `namespace + key` length > 512.
|
|
76
90
|
14. `ttl()` returns `TtlResult = { state: "missing" | "no-ttl" | "expires"; at? }` — discriminated union, not `Date | null | false`.
|
|
77
91
|
15. `incr`/`decr`/`getSet`/`cas`/`setIfAbsent` must be atomic: Redis = Lua or NX; Postgres = UPSERT/UPDATE with RETURNING; SQLite = a single UPSERT/UPDATE (`setIfAbsent`, `cas`) or a read-modify-write inside `#atomic()` (`getSet`, `incr`/`decr`); Memory = plain read-modify-write (single-threaded).
|
|
78
|
-
16. Non-numeric `incr`/`decr` target throws `TypeError("KV value is not a number")`. In Postgres,
|
|
92
|
+
16. Non-numeric `incr`/`decr` target throws `TypeError("KV value is not a number")`. In Postgres, the upsert's `DO UPDATE` has a `WHERE` that skips a live value that is not a JSON number, so the statement returns no row and raises nothing (a transaction the host opened on the same `Driver` is not aborted); no row is the `TypeError`, and the `22P02` catch on the sqlkit `SqlError`'s `code` is a backstop for a delta PostgreSQL cannot cast. SQLite checks `typeof` in JS, like Memory. Neither SQL adapter catches, wraps or translates any other database error: the caller gets sqlkit's `SqlError` (on SQLite with node:sqlite's error as its `cause`).
|
|
79
93
|
17. `keysIter` in both SQL adapters pages by keyset, 500 keys a page, so nothing stays open between yields and an early consumer break leaves nothing to release. It is not one snapshot. SQLite pages with `key > last`. Postgres pages in byte order on the `text_pattern_ops` prefix index (`key ~>~ $2`, `ORDER BY key USING ~<~`); a plain `key > $2 ORDER BY key` would walk the primary key across every namespace. Postgres `keys()` has no SQL `ORDER BY` for the same reason: it sorts in JS.
|
|
80
|
-
18. SQLite
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
94
|
+
18. SQLite talks to the database through `src/utils/sqlite-driver.ts`: `ownSqliteDriver(db)`
|
|
95
|
+
builds kv's own sqlkit driver over the host's `DatabaseSync` (`createSqliteDriver`, one
|
|
96
|
+
per handle in a `WeakMap`, keeping 128 compiled statements: without the number every
|
|
97
|
+
call compiles). `@marianmeres/sqlkit/sqlite` loads `node:sqlite`, so it is imported
|
|
98
|
+
there with `import()`, when `initialize()` or `destroy(true)` first opens the driver
|
|
99
|
+
over a handle (on a host's `Driver` nothing is imported). Nothing in `src/` imports it
|
|
100
|
+
statically, and `node:sqlite` only as types: a static import would make every
|
|
101
|
+
consumer of the package load `node:sqlite` (a test checks the package root loads
|
|
102
|
+
none). Every statement goes through `#q()`, and the DDL through `#exec()`: on kv's own
|
|
103
|
+
driver on its sync tier, so a statement runs inside the call; on a host's `Driver` on
|
|
104
|
+
its async tier, so it waits its turn and joins the host's transaction when it was
|
|
105
|
+
issued inside one. Statement texts open with their keyword (`INSERT`,
|
|
106
|
+
`SELECT`, `UPDATE`, `DELETE`): the driver keeps only a query or a data change, and an
|
|
107
|
+
`exec` (the DDL) or a pragma empties what it kept, so neither inside an operation.
|
|
108
|
+
19. SQLite multi-statement operations go through `#atomic(fn)`, `fn` a synchronous
|
|
109
|
+
function over the `SyncDriver` it is given: the driver's `sync.savepoint(fn)`, which
|
|
110
|
+
outside a transaction is `BEGIN IMMEDIATE` … `COMMIT`. When the caller has a `BEGIN`
|
|
111
|
+
open on the handle (`db.isTransaction`), the driver does not know that transaction and
|
|
112
|
+
SQLite refuses a second one, so `SAVEPOINT`, `ROLLBACK TO` and `RELEASE` go to the
|
|
113
|
+
handle and the unit's statements through the driver: the one place the adapter still
|
|
114
|
+
talks to the handle, besides `isOpen`. Neither `fn` nor `#atomic` may `await`: a unit
|
|
115
|
+
is over when the call returns, and tests pin it. The statements a unit shares with
|
|
116
|
+
`set`/`get`/`delete` are built once (`#setStatement`, `#getStatement`,
|
|
117
|
+
`#deleteStatement`), and the key is validated there, so an invalid key throws inside
|
|
118
|
+
the unit and undoes it. On a host's `Driver` the same `fn` runs in
|
|
119
|
+
`driver.savepoint(async () => fn(driver.sync))`: sqlkit's rule S3 puts its sync calls
|
|
120
|
+
inside that savepoint, which is a transaction of its own outside one, and a savepoint
|
|
121
|
+
of the host's transaction when the call was issued inside it (a failed unit is undone
|
|
122
|
+
alone, the host's transaction goes on). The unit waits its turn, then runs without a
|
|
123
|
+
break. There `driver.sync` is never used outside that callback: it is a `TypeError`
|
|
124
|
+
whenever the host has a transaction open elsewhere (S4).
|
|
125
|
+
20. SQLite stores `value` as JSON text and `expires_at` as epoch ms (INTEGER), compared against a bound `Date.now()` — never SQLite's own clock. Read integer columns through `Number()` (a handle opened with `readBigInts` returns bigints); the driver's `rowCount` is already a number.
|
|
126
|
+
21. TTL cleanup timers are `unref()`ed (`unref?.()`: browser and worker timers are plain numbers), so a client nobody destroyed cannot hold the process open; `destroy()` still clears them. Postgres's tick awaits its query, so it re-checks `_initialized` before re-arming, and `destroy()` clears `_initialized` before its first `await`. SQLite over a handle stops the cleanup (one error log) when it finds `db.isOpen` false. On a host's `Driver` its tick is Postgres's: it awaits the delete, logs a failure, re-checks `_initialized` and re-arms; a closed driver is told from another failure only by its message, so it is not told apart.
|
|
84
127
|
22. Postgres talks to the database only through `src/utils/driver.ts`: `toPgDriver(db)`,
|
|
85
128
|
called once in the constructor, turns a `pg.Pool` or `pg.Client` into a sqlkit
|
|
86
129
|
`Driver` (one per object, in a `WeakMap`: `createPgDriver` adds an `error` listener on
|
|
87
130
|
every call, and several adapters on one pool is normal), and uses a host's `Driver` as
|
|
88
|
-
it is. A `Driver` of another dialect is a `TypeError
|
|
131
|
+
it is. A `Driver` of another dialect is a `TypeError`, and so is one without
|
|
132
|
+
`savepoint()` (built by a sqlkit older than 0.3.0). `options.db` stays what the host
|
|
89
133
|
passed; every statement goes through `this.#db`, with its text and `$n` parameters.
|
|
90
134
|
Nothing in `src/` imports `pg` (it is a dev dependency); `@marianmeres/sqlkit/pg` is
|
|
91
135
|
imported statically, so the package root loads `node:async_hooks`. A kv call issued
|
|
92
136
|
inside a transaction the host opened on the same `Driver` joins it, by design.
|
|
93
|
-
23. The SQLite adapter
|
|
94
|
-
`
|
|
137
|
+
23. The SQLite adapter's `db` is a `SqliteDb`: a `DatabaseSync`, or a sqlkit `Driver` of
|
|
138
|
+
the `sqlite` dialect with a sync tier and `savepoint()` (node:sqlite's, from
|
|
139
|
+
`openSqlite()` or `createSqliteDriver()`; by its shape Durable Objects' too, which no
|
|
140
|
+
test covers). One flag set in the constructor, `#onHostDriver`, picks the path; it is
|
|
141
|
+
read in `#open`, `#q`, `#exec`, `#atomic` and the cleanup, and nowhere else. A
|
|
142
|
+
`Driver` of another dialect, one without a sync tier or without `savepoint()`
|
|
143
|
+
(`assertSqliteDriver`), and a `db` that is neither (`isDatabaseSync`) are a
|
|
144
|
+
`TypeError` at construction. `options.db` stays what the host passed. A `BEGIN` sent
|
|
145
|
+
to the handle behind a host's `Driver` makes a unit fail (`SqlError`, `cannot start a
|
|
146
|
+
transaction within a transaction`): documented, not handled, since kv has no handle
|
|
147
|
+
to look at. kv's methods are async, so they cannot be called from inside the host's
|
|
148
|
+
sync transaction (S5).
|
|
149
|
+
24. A sqlkit minor is a kv minor. sqlkit is on `0.x`, so each minor is a caret range of
|
|
150
|
+
its own: a consumer whose own sqlkit is on another minor gets two copies. Since 0.3.0
|
|
151
|
+
two copies know each other's `SqlError` (a mark `instanceof` reads; a copy of 0.2.0
|
|
152
|
+
or older has none, so its `instanceof SqlError` misses the errors of the driver kv
|
|
153
|
+
built). A minor can still change the `Driver` type both SQL adapters take, as 0.3.0
|
|
154
|
+
did with `savepoint`. Moving kv's range across a sqlkit minor therefore takes,
|
|
155
|
+
together: `engines.node` to sqlkit's, the README's "Requires Node" and the `^0.x.0`
|
|
156
|
+
in its notes and in `API.md`, and a "Changes in" section in the README telling
|
|
157
|
+
consumers to move their own sqlkit.
|
|
158
|
+
25. PGlite is a tested target of the postgres adapter, not an adapter of its own.
|
|
159
|
+
`tests/_tests-runner.ts` builds a fifth client, `pglite`: the postgres adapter on a
|
|
160
|
+
sqlkit `Driver` over the test file's one in-memory PGlite (`tests/_pglite.ts`). Its
|
|
161
|
+
`info().type` is `"postgres"`, so a test that needs a client's adapter type reads
|
|
162
|
+
`client.info().type`, never the key in `clients`. `@electric-sql/pglite` is a dev
|
|
163
|
+
dependency; nothing in `src/` imports it. A sixth client, `sqliteDriver`, is the
|
|
164
|
+
sqlite adapter on a sqlkit `Driver` from `openSqlite()`, a fresh in-memory database
|
|
165
|
+
per test closed in the `finally`; its `info().type` is `"sqlite"`.
|
|
95
166
|
|
|
96
167
|
## Before Making Changes
|
|
97
168
|
|
|
98
169
|
- [ ] Check existing patterns in adapter implementations
|
|
99
170
|
- [ ] Add or extend a test in `tests/kv.test.ts` (or in `tests/driver.test.ts` for what only
|
|
100
|
-
|
|
171
|
+
an SQL adapter's `db` decides), then run `pnpm run ci`
|
|
101
172
|
- [ ] Verify adapter limitations are documented (README feature matrix, API.md)
|
|
102
173
|
|
|
103
174
|
## Supported Adapters
|
|
104
175
|
|
|
105
|
-
| Adapter | Type String | Required Options
|
|
106
|
-
| ---------- | ------------ |
|
|
107
|
-
| Memory | `"memory"` | None
|
|
108
|
-
| Redis | `"redis"` | `db` (Redis client)
|
|
109
|
-
| PostgreSQL | `"postgres"` | `db` (`pg.Pool`, `pg.Client` or a sqlkit `Driver
|
|
110
|
-
| SQLite | `"sqlite"` | `db` (`DatabaseSync`)
|
|
176
|
+
| Adapter | Type String | Required Options | Limitations |
|
|
177
|
+
| ---------- | ------------ | ------------------------------------------------------------------------------------ | --------------------------------------------------------------------------- |
|
|
178
|
+
| Memory | `"memory"` | None | Not persisted |
|
|
179
|
+
| Redis | `"redis"` | `db` (Redis client) | Namespace required; `keys()`/`clear()` unavailable in cluster mode |
|
|
180
|
+
| PostgreSQL | `"postgres"` | `db` (`pg.Pool`, `pg.Client` or a sqlkit `Driver`, PGlite's included) | `tableName` limited to `[A-Za-z0-9_]` + optional `schema.` prefix |
|
|
181
|
+
| SQLite | `"sqlite"` | `db` (`DatabaseSync`, or a sqlkit `Driver` of the `sqlite` dialect with a sync tier) | `tableName` limited to `[A-Za-z0-9_]`, no schema prefix; synchronous engine |
|
|
111
182
|
|
|
112
183
|
## Core API
|
|
113
184
|
|
|
@@ -177,6 +248,9 @@ type SetMultipleEntry = [string, any] | { key: string; value: any; ttl?: number
|
|
|
177
248
|
// the postgres adapter's `db`; sqlkit's structural types, so `src/` never imports `pg`
|
|
178
249
|
type PostgresDb = PgPoolLike | PgStandaloneClientLike | Driver;
|
|
179
250
|
|
|
251
|
+
// the sqlite adapter's `db`; `DatabaseSync` is imported as a type only
|
|
252
|
+
type SqliteDb = DatabaseSync | Driver;
|
|
253
|
+
|
|
180
254
|
type TtlResult =
|
|
181
255
|
| { state: "missing" }
|
|
182
256
|
| { state: "no-ttl" }
|
|
@@ -195,12 +269,17 @@ pnpm run ci # typecheck + lint + test — what CI runs
|
|
|
195
269
|
pnpm run build # tsc -p tsconfig.build.json → dist/
|
|
196
270
|
```
|
|
197
271
|
|
|
198
|
-
Releasing
|
|
199
|
-
|
|
272
|
+
Releasing is a human action: `pnpm run rp` / `rpm` (patch / minor), or
|
|
273
|
+
`pnpm run release major --before-push "pnpm publish"`. `release` runs `pnpm run ci` before
|
|
274
|
+
it writes anything (`--before-bump`), and the publish runs between the tag and the push
|
|
275
|
+
(`--before-push`): a failing gate changes nothing, and a failed publish leaves no tag on
|
|
276
|
+
the remote.
|
|
200
277
|
|
|
201
278
|
## Test Environment Variables
|
|
202
279
|
|
|
203
|
-
Every test in `tests/kv.test.ts` runs against all four adapters,
|
|
280
|
+
Every test in `tests/kv.test.ts` runs against all four adapters, against the postgres
|
|
281
|
+
adapter on PGlite (in memory, in the test process: it needs nothing) and against the
|
|
282
|
+
sqlite adapter on a sqlkit `Driver` (in memory too), so tests need a live
|
|
204
283
|
Postgres and Redis (SQLite uses a fresh `:memory:` database per test and needs nothing).
|
|
205
284
|
`tests/driver.test.ts` needs Postgres only. Put
|
|
206
285
|
these in `.env` (see `.env.example`); CI provides them via service containers
|
|
@@ -219,6 +298,7 @@ TEST_REDIS_URL=redis://localhost:6379/13
|
|
|
219
298
|
## Extending
|
|
220
299
|
|
|
221
300
|
To add a new adapter:
|
|
301
|
+
|
|
222
302
|
1. Create `src/adapter/<name>.ts`
|
|
223
303
|
2. Extend `AdapterAbstract`; call `super(namespace)` and re-merge `options` (convention 5)
|
|
224
304
|
3. Implement all abstract/interface methods
|
package/API.md
CHANGED
|
@@ -27,6 +27,7 @@ function createKVClient<T extends keyof KnownTypes>(
|
|
|
27
27
|
```
|
|
28
28
|
|
|
29
29
|
**Type Parameters:**
|
|
30
|
+
|
|
30
31
|
- `T` - The adapter type (inferred from the `type` parameter)
|
|
31
32
|
|
|
32
33
|
**Parameters:**
|
|
@@ -40,6 +41,7 @@ function createKVClient<T extends keyof KnownTypes>(
|
|
|
40
41
|
**Returns:** A configured adapter instance. Call `initialize()` before use.
|
|
41
42
|
|
|
42
43
|
**Throws:**
|
|
44
|
+
|
|
43
45
|
- `TypeError` - If namespace doesn't end with `:` (and is not empty)
|
|
44
46
|
- `TypeError` - If an unsupported adapter type is specified
|
|
45
47
|
- `Error` - If required options (like `db`) are missing for the adapter
|
|
@@ -73,6 +75,9 @@ const shared = createKVClient("myapp:", "postgres", { db: createPgDriver(pool) }
|
|
|
73
75
|
import { DatabaseSync } from "node:sqlite";
|
|
74
76
|
const sqlite = createKVClient("myapp:", "sqlite", { db: new DatabaseSync("kv.db") });
|
|
75
77
|
await sqlite.initialize();
|
|
78
|
+
// ...or on a sqlkit Driver over SQLite
|
|
79
|
+
import { openSqlite } from "@marianmeres/sqlkit/sqlite";
|
|
80
|
+
const onDriver = createKVClient("myapp:", "sqlite", { db: openSqlite("kv.db") });
|
|
76
81
|
```
|
|
77
82
|
|
|
78
83
|
---
|
|
@@ -93,6 +98,9 @@ initialize(): Promise<void>
|
|
|
93
98
|
|
|
94
99
|
Must be called before any other operations. For some adapters (like memory), this is a no-op, but should always be called for consistency.
|
|
95
100
|
|
|
101
|
+
Await it: the SQL adapters create their table after an `await`. On the sqlite adapter
|
|
102
|
+
over a `DatabaseSync`, the first call imports sqlkit's SQLite driver.
|
|
103
|
+
|
|
96
104
|
**Throws:** `Error` - If connection fails.
|
|
97
105
|
|
|
98
106
|
---
|
|
@@ -224,7 +232,7 @@ decr(key: string, by?: number, options?: Partial<SetOptions>): Promise<number>
|
|
|
224
232
|
|
|
225
233
|
**Returns:** the new value after the operation.
|
|
226
234
|
|
|
227
|
-
**Errors:** throws `TypeError("KV value is not a number")` when the stored value is not a JSON number.
|
|
235
|
+
**Errors:** throws `TypeError("KV value is not a number")` when the stored value is not a JSON number. On the SQL adapters that fails nothing in the database: a transaction you opened on the same `Driver` goes on.
|
|
228
236
|
|
|
229
237
|
**Example:**
|
|
230
238
|
|
|
@@ -376,6 +384,7 @@ type TtlResult =
|
|
|
376
384
|
| `key` | `string` | The key to check. |
|
|
377
385
|
|
|
378
386
|
**Returns:** A `TtlResult`. Switch on `.state`:
|
|
387
|
+
|
|
379
388
|
- `"missing"` — key doesn't exist (or has expired)
|
|
380
389
|
- `"no-ttl"` — key exists with no expiration
|
|
381
390
|
- `"expires"` — key exists and will expire at `.at`
|
|
@@ -426,6 +435,7 @@ keys(pattern: string): Promise<string[]>
|
|
|
426
435
|
```
|
|
427
436
|
|
|
428
437
|
Uses Redis-style wildcard patterns:
|
|
438
|
+
|
|
429
439
|
- `*` matches any number of characters
|
|
430
440
|
- `?` matches exactly one character
|
|
431
441
|
- **Every other character is a literal match**, including regex metacharacters (`.`, `(`, `[`, `+`, `^`, `$`, `|`) and SQL LIKE metacharacters (`%`, `_`, `\`).
|
|
@@ -464,6 +474,7 @@ Ordering is not guaranteed across adapters — use `keys()` if you need sorted o
|
|
|
464
474
|
walk is not one snapshot: a key added or removed while it runs may or may not be seen.
|
|
465
475
|
|
|
466
476
|
**Per-adapter behavior:**
|
|
477
|
+
|
|
467
478
|
- Redis: wraps `SCAN` (non-blocking).
|
|
468
479
|
- PostgreSQL: keyset pages of 500 keys, in byte order on the prefix index `initialize()` creates. Nothing stays open between pages, so an early break has nothing to release and a write between two yields does not wait for the walk.
|
|
469
480
|
- SQLite: keyset pages of 500 keys (`key > last ORDER BY key`). Nothing stays open between pages, so an early break has nothing to release.
|
|
@@ -590,12 +601,12 @@ transaction(operations: Operation[]): Promise<any[]>
|
|
|
590
601
|
|
|
591
602
|
Atomicity per adapter:
|
|
592
603
|
|
|
593
|
-
| Adapter | Atomic | Notes
|
|
594
|
-
| ---------- | :----: |
|
|
595
|
-
| Memory | ✓ | Single-threaded by nature
|
|
596
|
-
| Redis | ✓ | Uses `MULTI`/`EXEC`
|
|
597
|
-
| PostgreSQL | ✓ | One connection for the whole unit: pinned on a `pg.Pool`, in turn on one `pg.Client` |
|
|
598
|
-
| SQLite | ✓ | `BEGIN IMMEDIATE`; a savepoint inside a
|
|
604
|
+
| Adapter | Atomic | Notes |
|
|
605
|
+
| ---------- | :----: | -------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
606
|
+
| Memory | ✓ | Single-threaded by nature |
|
|
607
|
+
| Redis | ✓ | Uses `MULTI`/`EXEC` |
|
|
608
|
+
| PostgreSQL | ✓ | One connection for the whole unit: pinned on a `pg.Pool`, in turn on one `pg.Client`; a savepoint inside a transaction opened on the same `Driver` |
|
|
609
|
+
| SQLite | ✓ | `BEGIN IMMEDIATE`; a savepoint inside a transaction you opened on the same handle or `Driver` |
|
|
599
610
|
|
|
600
611
|
Per-operation `options.ttl` is honored by **all** adapters (Redis, PostgreSQL, SQLite, Memory).
|
|
601
612
|
|
|
@@ -734,6 +745,28 @@ a `pg.Client` fit them as they are. See [`AdapterPostgresOptions`](#adapterpostg
|
|
|
734
745
|
|
|
735
746
|
---
|
|
736
747
|
|
|
748
|
+
### `SqliteDb`
|
|
749
|
+
|
|
750
|
+
What the sqlite adapter's `db` option takes.
|
|
751
|
+
|
|
752
|
+
```typescript
|
|
753
|
+
import type { DatabaseSync } from "node:sqlite";
|
|
754
|
+
import type { Driver } from "@marianmeres/sqlkit";
|
|
755
|
+
|
|
756
|
+
type SqliteDb = DatabaseSync | Driver;
|
|
757
|
+
```
|
|
758
|
+
|
|
759
|
+
- `DatabaseSync` — an open `node:sqlite` handle, e.g. `new DatabaseSync("kv.db")`.
|
|
760
|
+
- `Driver` — a sqlkit driver of the `sqlite` dialect with a sync tier and `savepoint()`,
|
|
761
|
+
e.g. `openSqlite("kv.db")` or `createSqliteDriver(handle)` from
|
|
762
|
+
`@marianmeres/sqlkit/sqlite`.
|
|
763
|
+
|
|
764
|
+
kv imports `DatabaseSync` as a type only: the package root loads no `node:sqlite`, and the
|
|
765
|
+
adapter imports sqlkit's SQLite driver only when `db` is a `DatabaseSync`. See
|
|
766
|
+
[`AdapterSqliteOptions`](#adaptersqliteoptions).
|
|
767
|
+
|
|
768
|
+
---
|
|
769
|
+
|
|
737
770
|
## Adapter-Specific Options
|
|
738
771
|
|
|
739
772
|
### `AdapterMemoryOptions`
|
|
@@ -829,24 +862,40 @@ connection for the whole unit, and a `COMMIT` that did not commit rejects:
|
|
|
829
862
|
a kv call issued inside a transaction you opened on that same driver
|
|
830
863
|
(`driver.transaction(async () => { await kv.set(…) })`) joins your transaction and
|
|
831
864
|
commits or rolls back with it. Make the call outside your transaction if it must
|
|
832
|
-
outlive your rollback.
|
|
865
|
+
outlive your rollback. A kv `transaction()` issued there is a savepoint of your
|
|
866
|
+
transaction: when it fails, what it did is undone alone, your transaction is not
|
|
867
|
+
aborted, and it goes on. The `Driver` has to come from sqlkit 0.3.0 or newer, which
|
|
868
|
+
has `savepoint()`; an older one is refused at construction, with a `TypeError`.
|
|
833
869
|
|
|
834
870
|
The adapter listens for the pool's or client's `error` event, so an idle connection the
|
|
835
871
|
server dropped does not end the process: it is reported through `process.emitWarning`
|
|
836
872
|
when you have no listener of your own.
|
|
837
873
|
|
|
838
874
|
A `Driver` over PGlite (`createPgliteDriver` from `@marianmeres/sqlkit/pglite`) is of the
|
|
839
|
-
`pg` dialect and is
|
|
875
|
+
`pg` dialect and is a tested target: every test of the suite runs on it, as on a
|
|
876
|
+
`pg.Pool`. PGlite is Postgres compiled to WebAssembly, one connection inside your
|
|
877
|
+
process, so calls take turns there as on one `pg.Client`. For that your app depends on
|
|
878
|
+
`@electric-sql/pglite` and on `@marianmeres/sqlkit` itself (`^0.3.0`).
|
|
840
879
|
|
|
841
880
|
**Errors:** a statement the database refused rejects with sqlkit's `SqlError` (exported
|
|
842
881
|
by `@marianmeres/sqlkit`), unwrapped and untranslated. Its `code` is Postgres's SQLSTATE
|
|
843
882
|
(e.g. `22P05`), its message is Postgres's, and its `cause` is the driver library's own
|
|
844
|
-
error (node-postgres's, on a pool or a client).
|
|
845
|
-
|
|
846
|
-
`TypeError("KV value is not a number")`, as every adapter does
|
|
847
|
-
|
|
883
|
+
error (node-postgres's, on a pool or a client). An `incr`/`decr` on a stored value that is
|
|
884
|
+
not a number is no database error: the statement leaves the value as it is and returns no
|
|
885
|
+
row, the adapter answers `TypeError("KV value is not a number")`, as every adapter does,
|
|
886
|
+
and a transaction you opened on the same `Driver` goes on. The one database error the
|
|
887
|
+
adapter reads is `22P02` from that statement, a backstop it answers with the same
|
|
888
|
+
`TypeError`. kv's own `TypeError` and `RangeError`, for an invalid key or namespace, are
|
|
889
|
+
unchanged.
|
|
890
|
+
|
|
891
|
+
`instanceof SqlError` holds when your app's own `@marianmeres/sqlkit` is 0.3.0 or newer
|
|
892
|
+
(kv installs `^0.3.1`). From 0.3.0 a `SqlError` is recognized across copies of the
|
|
893
|
+
package, so yours need not be the copy kv loads. A copy of 0.2 or older does not know an
|
|
894
|
+
error from a driver the adapter built over your pool or client, and the test is `false`.
|
|
895
|
+
`kind`, `code` and the message read the same on any copy.
|
|
848
896
|
|
|
849
897
|
**Table Schema:**
|
|
898
|
+
|
|
850
899
|
- `key` (VARCHAR 512) - Primary key
|
|
851
900
|
- `value` (JSONB) - Stored value
|
|
852
901
|
- `expires_at` (TIMESTAMP WITH TIME ZONE) - Expiration time
|
|
@@ -860,10 +909,12 @@ is a non-numeric `incr`/`decr` target (`22P02`), which it turns into
|
|
|
860
909
|
```typescript
|
|
861
910
|
interface AdapterSqliteOptions extends AdapterAbstractOptions {
|
|
862
911
|
/**
|
|
863
|
-
*
|
|
864
|
-
*
|
|
912
|
+
* The database: an open `node:sqlite` handle, e.g. `new DatabaseSync("kv.db")`, or a
|
|
913
|
+
* sqlkit `Driver` of the `sqlite` dialect with a sync tier, e.g. `openSqlite("kv.db")`.
|
|
914
|
+
* The caller owns it: `destroy()` never closes it, so one can be shared across
|
|
915
|
+
* adapters.
|
|
865
916
|
*/
|
|
866
|
-
db:
|
|
917
|
+
db: SqliteDb;
|
|
867
918
|
/**
|
|
868
919
|
* Name of the table to use for storing key-value pairs.
|
|
869
920
|
* The table will be created automatically if it doesn't exist.
|
|
@@ -880,30 +931,76 @@ interface AdapterSqliteOptions extends AdapterAbstractOptions {
|
|
|
880
931
|
}
|
|
881
932
|
```
|
|
882
933
|
|
|
883
|
-
|
|
884
|
-
`
|
|
885
|
-
|
|
886
|
-
|
|
887
|
-
|
|
888
|
-
|
|
934
|
+
`db` is a [`SqliteDb`](#sqlitedb): an open `DatabaseSync` as it is, or a
|
|
935
|
+
[`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit) `Driver` your
|
|
936
|
+
app built. The constructor refuses, with a `TypeError`, anything that is neither, a
|
|
937
|
+
`Driver` of another dialect, and a `Driver` without a sync tier or `savepoint()`.
|
|
938
|
+
`options.db` stays what you passed.
|
|
939
|
+
|
|
940
|
+
**On a `DatabaseSync`**, every statement runs on a sqlkit driver the adapter builds over
|
|
941
|
+
the handle, one per handle however many adapters share it, on its synchronous tier: a
|
|
942
|
+
statement has run when the call returns, and so has a multi-statement operation, so
|
|
943
|
+
nothing else on the handle comes between its statements. The first `initialize()` or
|
|
944
|
+
`destroy(true)` imports `@marianmeres/sqlkit/sqlite` for it.
|
|
945
|
+
|
|
946
|
+
**On a `Driver` you pass**, it has to be of the `sqlite` dialect, with a sync tier and
|
|
947
|
+
`savepoint()`, so from sqlkit 0.3.0 or newer: `openSqlite()` and `createSqliteDriver()`
|
|
948
|
+
from `@marianmeres/sqlkit/sqlite` build one. What holds there:
|
|
949
|
+
|
|
950
|
+
- **Calls take their turn.** One issued while a transaction you opened on the driver is
|
|
951
|
+
open, from outside it, waits for it to end and never becomes part of it.
|
|
952
|
+
- **A call issued inside a transaction you opened on the driver joins it**
|
|
953
|
+
(`driver.transaction(async () => { await kv.set(…) })`), and commits or rolls back with
|
|
954
|
+
it. Make the call outside your transaction if it must outlive your rollback. A
|
|
955
|
+
multi-statement operation issued there is a savepoint of your transaction: when it
|
|
956
|
+
fails, it is undone alone, and your transaction goes on.
|
|
957
|
+
- **kv cannot be called from inside a synchronous transaction of yours**
|
|
958
|
+
(`driver.sync.transaction(…)`): its methods are async, and the call rejects with a
|
|
959
|
+
`TypeError`.
|
|
960
|
+
- **A `BEGIN` you send to the handle behind the driver is not a transaction the driver
|
|
961
|
+
knows.** A single statement runs inside it, and a multi-statement operation rejects
|
|
962
|
+
with a `SqlError` (`cannot start a transaction within a transaction`). Open your
|
|
963
|
+
transactions through the driver.
|
|
964
|
+
|
|
965
|
+
sqlkit's driver for a Durable Object's SQLite storage has the same shape, and is expected
|
|
966
|
+
to work; it is not part of the test suite.
|
|
967
|
+
|
|
968
|
+
**Connection ownership:** the adapter never opens, closes or reconfigures the handle or
|
|
969
|
+
the driver — it sets no pragmas, and never calls the driver's `close()`, which would close
|
|
970
|
+
the handle under it. Call `destroy()` before you close either: on a `DatabaseSync`, a TTL
|
|
971
|
+
cleanup that finds the handle closed logs one error and stops; on a `Driver`, it logs
|
|
972
|
+
each failed tick until `destroy()`. For several processes on one file, open it with a busy
|
|
889
973
|
timeout (`new DatabaseSync(path, { timeout: 5000 })`) so a locked write waits instead of
|
|
890
|
-
failing, and consider `PRAGMA journal_mode = WAL`.
|
|
974
|
+
failing, and consider `PRAGMA journal_mode = WAL`. `openSqlite()` sets a busy timeout,
|
|
975
|
+
and the WAL journal for a file.
|
|
891
976
|
|
|
892
977
|
**Atomicity:** `setIfAbsent`, `cas`, `expire` and `set` are single statements.
|
|
893
978
|
`getSet`, `incr`/`decr`, `setMultiple`, `clear` and `transaction` run in `BEGIN IMMEDIATE`,
|
|
894
979
|
which takes the write lock up front, so they stay atomic against other connections and
|
|
895
|
-
processes. Called inside a transaction you opened on the same handle (`db.isTransaction`)
|
|
896
|
-
they use a savepoint instead: their own failure rolls back
|
|
897
|
-
`COMMIT`/`ROLLBACK` decides the rest.
|
|
980
|
+
processes. Called inside a transaction you opened on the same handle (`db.isTransaction`)
|
|
981
|
+
or through the same `Driver`, they use a savepoint instead: their own failure rolls back
|
|
982
|
+
only themselves, and your `COMMIT`/`ROLLBACK` decides the rest.
|
|
898
983
|
|
|
899
984
|
**Synchronous driver:** `node:sqlite` blocks the event loop for the duration of each call.
|
|
900
985
|
Every method still returns a Promise, like the other adapters.
|
|
901
986
|
|
|
987
|
+
**Errors:** a statement SQLite refused rejects with sqlkit's `SqlError` (exported by
|
|
988
|
+
`@marianmeres/sqlkit`), unwrapped and untranslated, as on PostgreSQL. Its class and
|
|
989
|
+
`kind` say what happened: a locked database, for one, is a `BusyError` (`kind` `"busy"`,
|
|
990
|
+
`retryable`). Its `code` is SQLite's result code (`"5"` for busy), its message is
|
|
991
|
+
SQLite's, and its `cause` is node:sqlite's own error, which carries node:sqlite's `code`
|
|
992
|
+
and `errcode`. The adapter catches no database error. A non-numeric `incr`/`decr` target
|
|
993
|
+
is checked in JavaScript and rejects with `TypeError("KV value is not a number")`, as on
|
|
994
|
+
every adapter. kv's own `TypeError` and `RangeError`, for an invalid key or namespace, are
|
|
995
|
+
unchanged. `instanceof SqlError` holds when your app's own `@marianmeres/sqlkit` is 0.3.0
|
|
996
|
+
or newer, as on PostgreSQL.
|
|
997
|
+
|
|
902
998
|
**Pattern matching** uses SQLite `GLOB`, which is case-sensitive and uses the primary-key
|
|
903
999
|
index for a literal prefix. `[` is escaped, so `keys()` / `clear()` follow the same
|
|
904
1000
|
literal-except-`*`/`?` rules as every adapter.
|
|
905
1001
|
|
|
906
1002
|
**Table Schema:**
|
|
1003
|
+
|
|
907
1004
|
- `key` (TEXT) - Primary key
|
|
908
1005
|
- `value` (TEXT) - Stored value as JSON text
|
|
909
1006
|
- `expires_at` (INTEGER) - Expiration time as epoch milliseconds; `NULL` = no expiration
|