@n8io/baseline-postgres 5.2.0 → 5.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +59 -34
- package/dist/index.d.ts +5 -2
- package/dist/index.js +22 -3
- package/package.json +4 -4
package/README.md
CHANGED
|
@@ -1,56 +1,74 @@
|
|
|
1
1
|
# @n8io/baseline-postgres
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**One pool, transactions that follow your Result, and migrations that run once, however many processes start.** Plus prefixed ids stored as `uuid`, keyset paging, advisory locks, and a helper that forces the interleaving a race test needs.
|
|
4
4
|
|
|
5
5
|
```sh
|
|
6
6
|
pnpm add @n8io/baseline-postgres
|
|
7
7
|
```
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
<p align="center"><img src="https://raw.githubusercontent.com/n8io/baseline/main/docs/readme/postgres-transaction.svg" width="880" alt="postgresTransactionRun: a run passes BEGIN and the work; when the work returns ok it commits, and when it returns err or throws it rolls back."></p>
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
11
|
+
## Guarantees
|
|
12
|
+
|
|
13
|
+
| You get | How |
|
|
14
|
+
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
15
|
+
| **A pool that does not wait forever** | `postgresPoolCreate` defaults: 10 connections, a 3 s connect timeout, a query rejects after 15 s, and an idle connection is probed after 10 s. |
|
|
16
|
+
| **Transactions that follow your Result** | `postgresTransactionRun` commits on `ok`, rolls back on `err` or a throw, and Postgres ends it after 5 s. |
|
|
17
|
+
| **No query outside its transaction** | The client it hands your work throws on any use once the transaction has ended. |
|
|
18
|
+
| **Each migration applied once** | `postgresMigrate` runs under an advisory lock, waits up to 60 s for it, and refuses a connection pooler before it changes anything. |
|
|
19
|
+
| **Prefixed ids in a `uuid` column** | `postgresIdColumnCreate("foo")` stores 16 bytes and reads back `Id<"foo">`. |
|
|
20
|
+
| **Unique violations as data** | `postgresUniqueViolationIs` and `postgresUniqueViolationRead` find one anywhere in the `cause` chain. |
|
|
21
|
+
| **Stable keyset paging** | `postgresKeysetCreate` pages on `(sort column, id)` for text, numbers, integers and dates; the sort column must be `NOT NULL` (see Exports for dates). |
|
|
22
|
+
| **A lock with no row to lock** | `postgresAdvisoryLockTake` takes a transaction-scoped lock on a string. |
|
|
23
|
+
| **Race tests that prove the lock** | `postgresTestInterleave` pauses one transaction mid-flight and reports whether the second waited. |
|
|
21
24
|
|
|
22
25
|
## Example
|
|
23
26
|
|
|
24
27
|
```ts
|
|
25
|
-
import {
|
|
26
|
-
postgresIdColumnCreate,
|
|
27
|
-
postgresPoolCreate,
|
|
28
|
-
postgresTransactionRun,
|
|
29
|
-
} from "@n8io/baseline-postgres";
|
|
30
|
-
import { err, ok } from "@n8io/baseline-result";
|
|
31
|
-
import { drizzle } from "drizzle-orm/node-postgres";
|
|
32
|
-
import { pgTable, text } from "drizzle-orm/pg-core";
|
|
33
|
-
|
|
34
28
|
export const fooTable = pgTable("foo", {
|
|
35
29
|
id: postgresIdColumnCreate("foo")().primaryKey(),
|
|
36
30
|
name: text().notNull(),
|
|
37
31
|
});
|
|
32
|
+
```
|
|
38
33
|
|
|
34
|
+
```ts
|
|
39
35
|
const pgPool = postgresPoolCreate({ databaseUrl: ENV.DATABASE_URL });
|
|
40
36
|
|
|
41
37
|
const result = await postgresTransactionRun({
|
|
42
38
|
pgPool,
|
|
43
39
|
work: async (pgClient) => {
|
|
44
|
-
const
|
|
40
|
+
const [foo] = await drizzle({ client: pgClient })
|
|
45
41
|
.insert(fooTable)
|
|
46
|
-
.values(foo)
|
|
42
|
+
.values({ id: idCreate("foo"), name })
|
|
47
43
|
.returning();
|
|
48
|
-
return
|
|
44
|
+
return foo ? ok(foo) : err({ type: "FooNotCreated" as const });
|
|
49
45
|
},
|
|
50
46
|
});
|
|
51
47
|
```
|
|
52
48
|
|
|
53
|
-
|
|
49
|
+
Test a race with `postgresTestInterleave` from `@n8io/baseline-postgres/testing` (see Race tests below). The Package's own tests need Docker: `pnpm test` starts a Postgres container and, for the migrate tests, a connection pooler.
|
|
50
|
+
|
|
51
|
+
## Reference
|
|
52
|
+
|
|
53
|
+
<details><summary><b>Exports</b></summary>
|
|
54
|
+
|
|
55
|
+
- `postgresPoolCreate({ databaseUrl, max?, connectTimeoutMs?, queryTimeoutMs?, vendor? })`: the process's one pool (10 connections, 3 s connect timeout). A query that takes longer than `queryTimeoutMs` (default 15 s, `null` for no limit) rejects with `Query read timeout`, so a server that stops answering cannot hold a request or a pool slot forever. The pool closes that connection; inside `postgresTransactionRun` it is closed once the rollback has been tried, which can take one more `queryTimeoutMs`. The timer runs in this process and sends no cancel: a server that is slow, not gone, may still finish the query, a write included, so treat the rejection as "unknown", not "did not happen". The limit is per query, inside a transaction too. An idle connection is probed after 10 s (TCP keep-alive). `vendor` takes pg's own pool settings, such as `ssl` or `idleTimeoutMillis`, and passes them through. The named options win: `vendor` throws on the keys they set, on `host`, `port`, `database`, `user` and `password` (pg lets the URL win, so put them in `databaseUrl`), and on `ssl` when the URL carries an `ssl*` parameter. It is outside the contract: nothing in it is tested here, and moving off pg deletes it.
|
|
56
|
+
- `PostgresPool`, `PostgresClient`: the pool and a checked-out connection. `await using` calls the pool's `end()` when its block ends. Name these in your own signatures, not `pg`'s types. They are pg's own pool and client, because Drizzle's `drizzle({ client })` needs the real ones.
|
|
57
|
+
- `postgresTransactionRun({ pgPool, work, timeoutMs? })`: runs `work(pgClient)` in one transaction; COMMIT on `ok`, ROLLBACK on `err` or a throw. It ends after `timeoutMs` (default 5 s).
|
|
58
|
+
- `postgresMigrate({ databaseUrl, migrationsFolder, lockWaitMs?, vendor? })`: applies pending Drizzle migrations under an advisory lock, so concurrent starts apply each one once. A run waits up to `lockWaitMs` (default 60 s) for the lock, then fails with an error that names it. The lock belongs to the session, so `databaseUrl` must reach Postgres itself: through a connection pooler such as PgBouncer the run refuses before it changes anything. Queries here have no time limit. `vendor` takes pg's settings for its one connection, under the same rules as the pool's.
|
|
59
|
+
- Types for the options and results above: `PostgresPoolOptions`, `PostgresPoolVendorOptions`, `PostgresMigrateVendorOptions`, `PostgresKeysetKind`, `PostgresKeysetOptions`, `PostgresUniqueViolation`, `PostgresExecutor`; from `/testing`, `PostgresTestInterleaveOptions` and `PostgresTestInterleavePool`.
|
|
60
|
+
- `@n8io/baseline-postgres/testing`: `postgresTestInterleave` for race tests (below). Import it from tests only.
|
|
61
|
+
- `postgresIdColumnCreate(prefix)`: a Drizzle `uuid` column that reads and writes `Id<prefix>`.
|
|
62
|
+
- `postgresUniqueViolationIs(error, constraint)`: true when a unique violation on that constraint is anywhere in the `cause` chain.
|
|
63
|
+
- `postgresUniqueViolationRead(error)`: the first unique violation in the `cause` chain as `{ constraint, columns, values }`, or `undefined`. `columns` and `values` come from Postgres's detail (`Key (league_id, name)=(7, Eagles)`), so a value is text as Postgres printed it. Postgres quotes nothing there, so when a key of two or more columns holds a value with `", "` in it, both lists are empty rather than misaligned; an expression key such as `lower(name)` comes back as Postgres printed it, and one with `", "` inside may not split cleanly. Read `constraint` when you need certainty.
|
|
64
|
+
- `postgresKeysetCreate({ after?, descending, idColumn, kind, sortColumn })`: keyset paging on `(sortColumn, idColumn)`. Returns `{ where, orderBy }` for a Drizzle `select`; `where` is `undefined` on the first page. `after` is the last row of the previous page, `{ id, value }`, with `id` written through `idColumn` (a prefixed id works). `kind` says how the sort compares: `text` by byte order (`COLLATE "C"`, the same in every locale, so give the column a `COLLATE "C"` index); `number` as the column's own type, so its index is used (for `integer`, `real` and `double precision` columns; a `numeric` or `bigint` column reads as a string or a bigint and is not supported); `integer` the same, for an `integer` or `serial` column only (any other column throws), taking only a whole number from -2147483648 to 2147483647; `date` as an instant, with `value` an ISO string with an offset (`toISOString()`). The sort column must be `NOT NULL`, because a NULL never compares, and a `date` column must hold no more than milliseconds (`timestamp({ precision: 3, withTimezone: true })`, or written only from a JS `Date`): a cursor made from a `Date` drops the microseconds a `now()` default stores, and pages would repeat or skip rows. A `value` of the wrong type for `kind`, a date that is not an ISO instant, or a number an `integer` or `smallint` sort column cannot hold (a fraction, or past its range), throws a `TypeError`: decode and check a cursor before you pass it.
|
|
65
|
+
- `postgresAdvisoryLockTake(db, key)`: waits for and takes a transaction-scoped advisory lock on the string `key`; Postgres drops it at COMMIT or ROLLBACK. Use it to serialize work that has no row to lock, such as counting a League's Teams before inserting one. `db` is anything with Drizzle's `execute` (`PostgresExecutor`). Call it inside `postgresTransactionRun`: outside a transaction the lock ends with the statement.
|
|
66
|
+
|
|
67
|
+
The example above imports `postgresIdColumnCreate`, `postgresPoolCreate` and `postgresTransactionRun` from this Package, `idCreate` from `@n8io/baseline-id`, `err` and `ok` from `@n8io/baseline-result`, `drizzle` from `drizzle-orm/node-postgres`, and `pgTable` and `text` from `drizzle-orm/pg-core`.
|
|
68
|
+
|
|
69
|
+
</details>
|
|
70
|
+
|
|
71
|
+
<details><summary><b>Keyset paging</b></summary>
|
|
54
72
|
|
|
55
73
|
```ts
|
|
56
74
|
import { postgresKeysetCreate } from "@n8io/baseline-postgres";
|
|
@@ -76,7 +94,9 @@ const nextCursor =
|
|
|
76
94
|
: null;
|
|
77
95
|
```
|
|
78
96
|
|
|
79
|
-
|
|
97
|
+
</details>
|
|
98
|
+
|
|
99
|
+
<details><summary><b>Advisory locks</b></summary>
|
|
80
100
|
|
|
81
101
|
```ts
|
|
82
102
|
import { postgresAdvisoryLockTake } from "@n8io/baseline-postgres";
|
|
@@ -89,7 +109,9 @@ const [counted] = await db
|
|
|
89
109
|
.where(eq(teamTable.leagueId, leagueId));
|
|
90
110
|
```
|
|
91
111
|
|
|
92
|
-
|
|
112
|
+
</details>
|
|
113
|
+
|
|
114
|
+
<details><summary><b>Race tests</b></summary>
|
|
93
115
|
|
|
94
116
|
`@n8io/baseline-postgres/testing` exports `postgresTestInterleave({ first, second, pgPool, timeoutMs? })`, for tests only. `first(pause)` runs until it awaits `pause()`, typically right after it read and locked what it will write. Then `second()` starts, and `first` resumes once `second` waits on a lock (`pg_stat_activity`) or has finished. It returns `{ first, second, secondWaited }`; `secondWaited` is false when no lock made `second` wait. It fails when `first` never pauses, or when `second` neither waits nor finishes within `timeoutMs` (default 5 s). A connection that already waited on a lock before `second` started does not count; any other one in the same database does, so keep other work off that database while it runs. It needs three connections from `pgPool` at once: `first`'s, `second`'s, and one to watch `pg_stat_activity`.
|
|
95
117
|
|
|
@@ -126,15 +148,18 @@ expect(result.secondWaited).toBe(true);
|
|
|
126
148
|
|
|
127
149
|
A race test must fail with the lock removed. Before you trust one, take the lock out (`FOR UPDATE`, the advisory lock), see the test go red for the right reason, and put it back. A plain `Promise.all` of two writes usually passes either way, because one finishes before the other starts.
|
|
128
150
|
|
|
129
|
-
|
|
151
|
+
</details>
|
|
130
152
|
|
|
131
|
-
|
|
132
|
-
- Env, by the template's convention: `DATABASE_URL` (`postgres://` or `postgresql://`), `DATABASE_POOL_MAX` for `max`, and `DATABASE_MIGRATE_URL` for `postgresMigrate` where `DATABASE_URL` goes through a pooler. Transactions use `transaction_timeout`, so Postgres 17 or newer.
|
|
133
|
-
|
|
134
|
-
## Sizing the pool
|
|
153
|
+
<details><summary><b>Sizing the pool</b></summary>
|
|
135
154
|
|
|
136
155
|
Every process opens up to `max` connections, and Postgres refuses new ones past its `max_connections`. Keep the sum over every process (`max` times the number of processes, counting the extra ones a rollout starts) under that limit, with a few left for migrate and for people. Behind a pooler, keep it under the pooler's limit instead.
|
|
137
156
|
|
|
138
|
-
|
|
157
|
+
</details>
|
|
158
|
+
|
|
159
|
+
<details><summary><b>Env</b></summary>
|
|
160
|
+
|
|
161
|
+
By the Template's convention: `DATABASE_URL` (`postgres://` or `postgresql://`), `DATABASE_POOL_MAX` for `max`, and `DATABASE_MIGRATE_URL` for `postgresMigrate` where `DATABASE_URL` goes through a pooler. Transactions use `transaction_timeout`, so Postgres 17 or newer.
|
|
162
|
+
|
|
163
|
+
</details>
|
|
139
164
|
|
|
140
|
-
`
|
|
165
|
+
**Needs** peers `pg`, `drizzle-orm` (`1.0.0-rc.5`, pinned until 1.0 is out) and, for types, `@types/pg`; `DATABASE_URL` by the Template's convention; Postgres 17 or newer. Runs on [pg](https://node-postgres.com) and [Drizzle](https://orm.drizzle.team).
|
package/dist/index.d.ts
CHANGED
|
@@ -14,8 +14,11 @@ type PostgresIdColumnBuilder<P extends string> = PgCustomColumnBuilder<{
|
|
|
14
14
|
driverParam: string;
|
|
15
15
|
}>;
|
|
16
16
|
export declare const postgresIdColumnCreate: <P extends string>(prefix: P) => ((name?: string) => PostgresIdColumnBuilder<P>);
|
|
17
|
-
/**
|
|
18
|
-
|
|
17
|
+
/**
|
|
18
|
+
* How the sort column compares: `text` by byte order (`COLLATE "C"`), `date` as an instant,
|
|
19
|
+
* `integer` and `number` as numbers (`integer` takes only a whole number).
|
|
20
|
+
*/
|
|
21
|
+
type PostgresKeysetKind = "date" | "integer" | "number" | "text";
|
|
19
22
|
type PostgresKeysetOptions = {
|
|
20
23
|
/** The last row of the previous page; none for the first page. */
|
|
21
24
|
after?: {
|
package/dist/index.js
CHANGED
|
@@ -29,10 +29,25 @@ const isoInstantIs = (value) => {
|
|
|
29
29
|
const [year = 0, month = 0, day = 0] = value.slice(0, 10).split("-").map(Number);
|
|
30
30
|
return new Date(Date.UTC(year, month - 1, day)).getUTCDate() === day;
|
|
31
31
|
};
|
|
32
|
-
const
|
|
33
|
-
|
|
32
|
+
const INT4 = [-(2 ** 31), 2 ** 31 - 1];
|
|
33
|
+
const INT2 = [-(2 ** 15), 2 ** 15 - 1];
|
|
34
|
+
const integerRanges = {
|
|
35
|
+
integer: INT4,
|
|
36
|
+
serial: INT4,
|
|
37
|
+
smallint: INT2,
|
|
38
|
+
smallserial: INT2
|
|
39
|
+
};
|
|
40
|
+
const integerKindAssert = (sortColumn) => {
|
|
41
|
+
const type = sortColumn.getSQLType();
|
|
42
|
+
if (integerRanges[type] !== INT4) throw new TypeError(`An integer keyset needs an integer or serial column, got ${type}`);
|
|
43
|
+
};
|
|
44
|
+
const afterValueAssert = ({ kind, sortColumn, value }) => {
|
|
45
|
+
const numeric = kind === "integer" || kind === "number";
|
|
46
|
+
const range = numeric ? integerRanges[sortColumn.getSQLType()] : void 0;
|
|
47
|
+
if (!(numeric ? typeof value === "number" && (range ? Number.isInteger(value) && value >= range[0] && value <= range[1] : Number.isFinite(value)) : typeof value === "string" && (kind === "text" || isoInstantIs(value)))) throw new TypeError(`${kind === "integer" ? "An" : "A"} ${kind} keyset needs ${range ? `an integer from ${range[0]} to ${range[1]}` : kind === "number" ? "a finite number" : kind === "date" ? "an ISO instant such as 2026-01-01T00:00:00.000Z" : "a string"}, got ${JSON.stringify(value)}`);
|
|
34
48
|
};
|
|
35
49
|
const postgresKeysetCreate = ({ after, descending, idColumn, kind, sortColumn }) => {
|
|
50
|
+
if (kind === "integer") integerKindAssert(sortColumn);
|
|
36
51
|
const sortBy = kind === "text" ? sql`${sortColumn} COLLATE "C"` : sql`${sortColumn}`;
|
|
37
52
|
const direction = descending ? sql`DESC` : sql`ASC`;
|
|
38
53
|
const orderBy = [sql`${sortBy} ${direction}`, sql`${idColumn} ${direction}`];
|
|
@@ -40,7 +55,11 @@ const postgresKeysetCreate = ({ after, descending, idColumn, kind, sortColumn })
|
|
|
40
55
|
orderBy,
|
|
41
56
|
where: void 0
|
|
42
57
|
};
|
|
43
|
-
afterValueAssert(
|
|
58
|
+
afterValueAssert({
|
|
59
|
+
kind,
|
|
60
|
+
sortColumn,
|
|
61
|
+
value: after.value
|
|
62
|
+
});
|
|
44
63
|
const value = kind === "date" ? sql`${after.value}::timestamptz` : sql`${after.value}`;
|
|
45
64
|
return {
|
|
46
65
|
orderBy,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@n8io/baseline-postgres",
|
|
3
|
-
"version": "5.
|
|
3
|
+
"version": "5.3.1",
|
|
4
4
|
"description": "Postgres pool, Result-aware transactions, migrations and prefixed-id columns on pg and Drizzle.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -31,9 +31,9 @@
|
|
|
31
31
|
"registry": "https://registry.npmjs.org"
|
|
32
32
|
},
|
|
33
33
|
"dependencies": {
|
|
34
|
-
"@n8io/baseline-id": "^3.0.
|
|
35
|
-
"@n8io/baseline-logger": "^3.1.
|
|
36
|
-
"@n8io/baseline-result": "^1.1.
|
|
34
|
+
"@n8io/baseline-id": "^3.0.1",
|
|
35
|
+
"@n8io/baseline-logger": "^3.1.1",
|
|
36
|
+
"@n8io/baseline-result": "^1.1.2"
|
|
37
37
|
},
|
|
38
38
|
"peerDependencies": {
|
|
39
39
|
"@types/pg": "^8",
|