@n8io/baseline-postgres 5.0.0 → 5.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +89 -5
- package/dist/index.d.ts +42 -3
- package/dist/index.js +76 -9
- package/dist/testing.d.ts +21 -0
- package/dist/testing.js +57 -0
- package/package.json +5 -1
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @n8io/baseline-postgres
|
|
2
2
|
|
|
3
|
-
Postgres on [pg](https://node-postgres.com) and [Drizzle](https://orm.drizzle.team): one pool, Result-aware transactions, migrations under a lock,
|
|
3
|
+
Postgres on [pg](https://node-postgres.com) and [Drizzle](https://orm.drizzle.team): one pool, Result-aware transactions, migrations under a lock, columns that store prefixed ids as `uuid`, keyset paging, advisory locks, and a helper that forces the interleaving a race test needs.
|
|
4
4
|
|
|
5
5
|
```sh
|
|
6
6
|
pnpm add @n8io/baseline-postgres
|
|
@@ -8,12 +8,16 @@ pnpm add @n8io/baseline-postgres
|
|
|
8
8
|
|
|
9
9
|
## Exports
|
|
10
10
|
|
|
11
|
-
- `postgresPoolCreate({ databaseUrl, max?, connectTimeoutMs?, vendor? })`: the process's one pool (10 connections, 3 s connect timeout). `vendor` takes pg's own pool settings, such as `ssl` or `idleTimeoutMillis`, and passes them through. The named options win: `vendor` throws on the keys they set, on `host`, `port`, `database`, `user` and `password` (pg lets the URL win, so put them in `databaseUrl`), and on `ssl` when the URL carries an `ssl*` parameter. It is outside the contract: nothing in it is tested here, and moving off pg deletes it.
|
|
11
|
+
- `postgresPoolCreate({ databaseUrl, max?, connectTimeoutMs?, queryTimeoutMs?, vendor? })`: the process's one pool (10 connections, 3 s connect timeout). A query that takes longer than `queryTimeoutMs` (default 15 s, `null` for no limit) rejects with `Query read timeout`, so a server that stops answering cannot hold a request or a pool slot forever. The pool closes that connection; inside `postgresTransactionRun` it is closed once the rollback has been tried, which can take one more `queryTimeoutMs`. The timer runs in this process and sends no cancel: a server that is slow, not gone, may still finish the query, a write included, so treat the rejection as "unknown", not "did not happen". The limit is per query, inside a transaction too. An idle connection is probed after 10 s (TCP keep-alive). `vendor` takes pg's own pool settings, such as `ssl` or `idleTimeoutMillis`, and passes them through. The named options win: `vendor` throws on the keys they set, on `host`, `port`, `database`, `user` and `password` (pg lets the URL win, so put them in `databaseUrl`), and on `ssl` when the URL carries an `ssl*` parameter. It is outside the contract: nothing in it is tested here, and moving off pg deletes it.
|
|
12
12
|
- `PostgresPool`, `PostgresClient`: the pool and a checked-out connection. `await using` calls the pool's `end()` when its block ends. Name these in your own signatures, not `pg`'s types. They are pg's own pool and client, because Drizzle's `drizzle({ client })` needs the real ones.
|
|
13
13
|
- `postgresTransactionRun({ pgPool, work, timeoutMs? })`: runs `work(pgClient)` in one transaction; COMMIT on `ok`, ROLLBACK on `err` or a throw. It ends after `timeoutMs` (default 5 s).
|
|
14
|
-
- `postgresMigrate({ databaseUrl, migrationsFolder, vendor? })`: applies pending Drizzle migrations under an advisory lock, so concurrent starts apply each one once. `vendor` takes pg's settings for its one connection, under the same rules as the pool's.
|
|
14
|
+
- `postgresMigrate({ databaseUrl, migrationsFolder, lockWaitMs?, vendor? })`: applies pending Drizzle migrations under an advisory lock, so concurrent starts apply each one once. A run waits up to `lockWaitMs` (default 60 s) for the lock, then fails with an error that names it. The lock belongs to the session, so `databaseUrl` must reach Postgres itself: through a connection pooler such as PgBouncer the run refuses before it changes anything. Queries here have no time limit. `vendor` takes pg's settings for its one connection, under the same rules as the pool's.
|
|
15
|
+
- `@n8io/baseline-postgres/testing`: `postgresTestInterleave` for race tests (below). Import it from tests only.
|
|
15
16
|
- `postgresIdColumnCreate(prefix)`: a Drizzle `uuid` column that reads and writes `Id<prefix>`.
|
|
16
17
|
- `postgresUniqueViolationIs(error, constraint)`: true when a unique violation on that constraint is anywhere in the `cause` chain.
|
|
18
|
+
- `postgresUniqueViolationRead(error)`: the first unique violation in the `cause` chain as `{ constraint, columns, values }`, or `undefined`. `columns` and `values` come from Postgres's detail (`Key (league_id, name)=(7, Eagles)`), so a value is text as Postgres printed it. Postgres quotes nothing there, so when a key of two or more columns holds a value with `", "` in it, both lists are empty rather than misaligned; an expression key such as `lower(name)` comes back as Postgres printed it, and one with `", "` inside may not split cleanly. Read `constraint` when you need certainty.
|
|
19
|
+
- `postgresKeysetCreate({ after?, descending, idColumn, kind, sortColumn })`: keyset paging on `(sortColumn, idColumn)`. Returns `{ where, orderBy }` for a Drizzle `select`; `where` is `undefined` on the first page. `after` is the last row of the previous page, `{ id, value }`, with `id` written through `idColumn` (a prefixed id works). `kind` says how the sort compares: `text` by byte order (`COLLATE "C"`, the same in every locale, so give the column a `COLLATE "C"` index); `number` as the column's own type, so its index is used (for `integer`, `real` and `double precision` columns; a `numeric` or `bigint` column reads as a string or a bigint and is not supported); `date` as an instant, with `value` an ISO string with an offset (`toISOString()`). The sort column must be `NOT NULL`, because a NULL never compares, and a `date` column must hold no more than milliseconds (`timestamp({ precision: 3, withTimezone: true })`, or written only from a JS `Date`): a cursor made from a `Date` drops the microseconds a `now()` default stores, and pages would repeat or skip rows. A `value` of the wrong type for `kind`, or a date that is not an ISO instant, throws a `TypeError`: decode and check a cursor before you pass it.
|
|
20
|
+
- `postgresAdvisoryLockTake(db, key)`: waits for and takes a transaction-scoped advisory lock on the string `key`; Postgres drops it at COMMIT or ROLLBACK. Use it to serialize work that has no row to lock, such as counting a League's Teams before inserting one. `db` is anything with Drizzle's `execute` (`PostgresExecutor`). Call it inside `postgresTransactionRun`: outside a transaction the lock ends with the statement.
|
|
17
21
|
|
|
18
22
|
## Example
|
|
19
23
|
|
|
@@ -46,11 +50,91 @@ const result = await postgresTransactionRun({
|
|
|
46
50
|
});
|
|
47
51
|
```
|
|
48
52
|
|
|
53
|
+
## Keyset paging
|
|
54
|
+
|
|
55
|
+
```ts
|
|
56
|
+
import { postgresKeysetCreate } from "@n8io/baseline-postgres";
|
|
57
|
+
|
|
58
|
+
const keyset = postgresKeysetCreate({
|
|
59
|
+
after: cursor ? { id: cursor.id, value: cursor.value } : null,
|
|
60
|
+
descending: true,
|
|
61
|
+
idColumn: fooTable.id,
|
|
62
|
+
kind: "date",
|
|
63
|
+
sortColumn: fooTable.createdAt,
|
|
64
|
+
});
|
|
65
|
+
const rows = await db
|
|
66
|
+
.select()
|
|
67
|
+
.from(fooTable)
|
|
68
|
+
.where(and(isNull(fooTable.archivedAt), keyset.where))
|
|
69
|
+
.orderBy(...keyset.orderBy)
|
|
70
|
+
.limit(limit + 1);
|
|
71
|
+
const page = rows.slice(0, limit);
|
|
72
|
+
const last = page.at(-1);
|
|
73
|
+
const nextCursor =
|
|
74
|
+
rows.length > limit && last
|
|
75
|
+
? { id: last.id, value: last.createdAt.toISOString() }
|
|
76
|
+
: null;
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
## Advisory locks
|
|
80
|
+
|
|
81
|
+
```ts
|
|
82
|
+
import { postgresAdvisoryLockTake } from "@n8io/baseline-postgres";
|
|
83
|
+
|
|
84
|
+
// In a transaction: two Team creates in one League count one after the other.
|
|
85
|
+
await postgresAdvisoryLockTake(db, `team-count:${leagueId}`);
|
|
86
|
+
const [counted] = await db
|
|
87
|
+
.select({ total: count() })
|
|
88
|
+
.from(teamTable)
|
|
89
|
+
.where(eq(teamTable.leagueId, leagueId));
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
## Race tests
|
|
93
|
+
|
|
94
|
+
`@n8io/baseline-postgres/testing` exports `postgresTestInterleave({ first, second, pgPool, timeoutMs? })`, for tests only. `first(pause)` runs until it awaits `pause()`, typically right after it read and locked what it will write. Then `second()` starts, and `first` resumes once `second` waits on a lock (`pg_stat_activity`) or has finished. It returns `{ first, second, secondWaited }`; `secondWaited` is false when no lock made `second` wait. It fails when `first` never pauses, or when `second` neither waits nor finishes within `timeoutMs` (default 5 s). A connection that already waited on a lock before `second` started does not count; any other one in the same database does, so keep other work off that database while it runs. It needs three connections from `pgPool` at once: `first`'s, `second`'s, and one to watch `pg_stat_activity`.
|
|
95
|
+
|
|
96
|
+
```ts
|
|
97
|
+
import { postgresTestInterleave } from "@n8io/baseline-postgres/testing";
|
|
98
|
+
|
|
99
|
+
// Reads a count, then writes it back plus one: a read-modify-write Use case.
|
|
100
|
+
const increment = (pause?: () => Promise<void>) =>
|
|
101
|
+
postgresTransactionRun({
|
|
102
|
+
pgPool,
|
|
103
|
+
work: async (pgClient) => {
|
|
104
|
+
const db = drizzle({ client: pgClient });
|
|
105
|
+
const [row] = await db
|
|
106
|
+
.select()
|
|
107
|
+
.from(counterTable)
|
|
108
|
+
.where(eq(counterTable.id, id))
|
|
109
|
+
.for("update"); // remove it and the test below must fail
|
|
110
|
+
await pause?.();
|
|
111
|
+
await db
|
|
112
|
+
.update(counterTable)
|
|
113
|
+
.set({ n: (row?.n ?? 0) + 1 })
|
|
114
|
+
.where(eq(counterTable.id, id));
|
|
115
|
+
return ok(undefined);
|
|
116
|
+
},
|
|
117
|
+
});
|
|
118
|
+
|
|
119
|
+
const result = await postgresTestInterleave({
|
|
120
|
+
first: (pause) => increment(pause),
|
|
121
|
+
pgPool,
|
|
122
|
+
second: () => increment(),
|
|
123
|
+
});
|
|
124
|
+
expect(result.secondWaited).toBe(true);
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
A race test must fail with the lock removed. Before you trust one, take the lock out (`FOR UPDATE`, the advisory lock), see the test go red for the right reason, and put it back. A plain `Promise.all` of two writes usually passes either way, because one finishes before the other starts.
|
|
128
|
+
|
|
49
129
|
## Needs
|
|
50
130
|
|
|
51
131
|
- Peers: `pg`, `drizzle-orm` (`1.0.0-rc.5`, pinned until 1.0 is out) and, for types, `@types/pg`.
|
|
52
|
-
- Env, by the template's convention: `DATABASE_URL` (`postgres://` or `postgresql://`). Transactions use `transaction_timeout`, so Postgres 17 or newer.
|
|
132
|
+
- Env, by the template's convention: `DATABASE_URL` (`postgres://` or `postgresql://`), `DATABASE_POOL_MAX` for `max`, and `DATABASE_MIGRATE_URL` for `postgresMigrate` where `DATABASE_URL` goes through a pooler. Transactions use `transaction_timeout`, so Postgres 17 or newer.
|
|
133
|
+
|
|
134
|
+
## Sizing the pool
|
|
135
|
+
|
|
136
|
+
Every process opens up to `max` connections, and Postgres refuses new ones past its `max_connections`. Keep the sum over every process (`max` times the number of processes, counting the extra ones a rollout starts) under that limit, with a few left for migrate and for people. Behind a pooler, keep it under the pooler's limit instead.
|
|
53
137
|
|
|
54
138
|
## Tests
|
|
55
139
|
|
|
56
|
-
`pnpm test` needs Docker: it starts a Postgres container.
|
|
140
|
+
`pnpm test` needs Docker: it starts a Postgres container and, for the migrate tests, PgBouncer.
|
package/dist/index.d.ts
CHANGED
|
@@ -1,13 +1,37 @@
|
|
|
1
|
+
import { AnyColumn, SQL } from "drizzle-orm";
|
|
1
2
|
import { Id } from "@n8io/baseline-id";
|
|
2
3
|
import { PgCustomColumnBuilder } from "drizzle-orm/pg-core";
|
|
3
4
|
import pg from "pg";
|
|
4
5
|
import { Result } from "@n8io/baseline-result";
|
|
6
|
+
/** Anything that runs a Drizzle query: a Drizzle database or transaction. */
|
|
7
|
+
type PostgresExecutor = {
|
|
8
|
+
execute(query: SQL): Promise<unknown>;
|
|
9
|
+
};
|
|
10
|
+
export declare const postgresAdvisoryLockTake: (db: PostgresExecutor, key: string) => Promise<void>;
|
|
5
11
|
type PostgresIdColumnBuilder<P extends string> = PgCustomColumnBuilder<{
|
|
6
12
|
data: Id<P>;
|
|
7
13
|
dataType: "custom";
|
|
8
14
|
driverParam: string;
|
|
9
15
|
}>;
|
|
10
16
|
export declare const postgresIdColumnCreate: <P extends string>(prefix: P) => ((name?: string) => PostgresIdColumnBuilder<P>);
|
|
17
|
+
/** How the sort column compares: `text` by byte order (`COLLATE "C"`), `date` as an instant. */
|
|
18
|
+
type PostgresKeysetKind = "date" | "number" | "text";
|
|
19
|
+
type PostgresKeysetOptions = {
|
|
20
|
+
/** The last row of the previous page; none for the first page. */
|
|
21
|
+
after?: {
|
|
22
|
+
id: string;
|
|
23
|
+
value: string | number;
|
|
24
|
+
} | null;
|
|
25
|
+
descending: boolean;
|
|
26
|
+
/** The tie-breaker, unique per row. The cursor's id goes through its encoder. */
|
|
27
|
+
idColumn: AnyColumn;
|
|
28
|
+
kind: PostgresKeysetKind;
|
|
29
|
+
sortColumn: AnyColumn;
|
|
30
|
+
};
|
|
31
|
+
export declare const postgresKeysetCreate: ({ after, descending, idColumn, kind, sortColumn }: PostgresKeysetOptions) => {
|
|
32
|
+
orderBy: SQL[];
|
|
33
|
+
where: SQL | undefined;
|
|
34
|
+
};
|
|
11
35
|
declare const postgresVendorOwnedKeys: {
|
|
12
36
|
readonly connectionString: "databaseUrl";
|
|
13
37
|
readonly connectionTimeoutMillis: "connectTimeoutMs";
|
|
@@ -25,6 +49,8 @@ type PostgresVendorOwnedKey = keyof typeof postgresVendorOwnedKeys;
|
|
|
25
49
|
type PostgresMigrateVendorOptions = Omit<pg.ClientConfig, PostgresVendorOwnedKey>;
|
|
26
50
|
export declare const postgresMigrate: (options: {
|
|
27
51
|
databaseUrl: string;
|
|
52
|
+
/** How long to wait for another run to release the lock. Default 60 s. */
|
|
53
|
+
lockWaitMs?: number | null;
|
|
28
54
|
migrationsFolder: string;
|
|
29
55
|
/** Settings only pg has, such as `ssl`: the same ones the pool takes. */
|
|
30
56
|
vendor?: PostgresMigrateVendorOptions;
|
|
@@ -41,21 +67,34 @@ type PostgresClient = pg.PoolClient;
|
|
|
41
67
|
* here is tested, and moving off pg deletes it. The keys the named options and the URL set are
|
|
42
68
|
* refused.
|
|
43
69
|
*/
|
|
44
|
-
type PostgresPoolVendorOptions = Omit<pg.PoolConfig, PostgresVendorOwnedKey | "max">;
|
|
70
|
+
type PostgresPoolVendorOptions = Omit<pg.PoolConfig, PostgresVendorOwnedKey | "max" | "query_timeout">;
|
|
45
71
|
type PostgresPoolOptions = {
|
|
46
72
|
databaseUrl: string;
|
|
47
73
|
/** How long a connect may take. Default 3 s. */
|
|
48
74
|
connectTimeoutMs?: number;
|
|
49
75
|
/** The most connections the pool opens. Default 10. */
|
|
50
76
|
max?: number;
|
|
77
|
+
/**
|
|
78
|
+
* How long the caller waits for one query before it rejects. Default 15 s; `null` turns it off.
|
|
79
|
+
* The pool closes that connection; the server may still finish the query.
|
|
80
|
+
*/
|
|
81
|
+
queryTimeoutMs?: number | null;
|
|
51
82
|
/** Settings only pg has, such as `ssl` or `idleTimeoutMillis`. */
|
|
52
83
|
vendor?: PostgresPoolVendorOptions;
|
|
53
84
|
};
|
|
54
|
-
export declare const postgresPoolCreate: ({ connectTimeoutMs, databaseUrl, max, vendor }: PostgresPoolOptions) => PostgresPool;
|
|
85
|
+
export declare const postgresPoolCreate: ({ connectTimeoutMs, databaseUrl, max, queryTimeoutMs, vendor }: PostgresPoolOptions) => PostgresPool;
|
|
55
86
|
export declare const postgresTransactionRun: <T, E>({ pgPool, timeoutMs: timeoutMsGiven, work }: {
|
|
56
87
|
pgPool: PostgresPool;
|
|
57
88
|
timeoutMs?: number | null;
|
|
58
89
|
work: (pgClient: PostgresClient) => Promise<Result<T, E>>;
|
|
59
90
|
}) => Promise<Result<T, E>>;
|
|
91
|
+
type PostgresUniqueViolation = {
|
|
92
|
+
/** The key's columns, or expressions for an expression index. Empty when Postgres sent no detail. */
|
|
93
|
+
columns: string[];
|
|
94
|
+
constraint: string;
|
|
95
|
+
/** The clashing values as Postgres printed them, in column order. Empty when unreadable. */
|
|
96
|
+
values: string[];
|
|
97
|
+
};
|
|
60
98
|
export declare const postgresUniqueViolationIs: (error: unknown, constraint: string) => boolean;
|
|
61
|
-
export
|
|
99
|
+
export declare const postgresUniqueViolationRead: (error: unknown) => PostgresUniqueViolation | undefined;
|
|
100
|
+
export type { PostgresClient, PostgresExecutor, PostgresKeysetKind, PostgresKeysetOptions, PostgresMigrateVendorOptions, PostgresPool, PostgresPoolOptions, PostgresPoolVendorOptions, PostgresUniqueViolation };
|
package/dist/index.js
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { sql } from "drizzle-orm";
|
|
1
2
|
import { idFromUuid, idToUuid } from "@n8io/baseline-id";
|
|
2
3
|
import { customType } from "drizzle-orm/pg-core";
|
|
3
4
|
import { createHash } from "node:crypto";
|
|
@@ -5,6 +6,10 @@ import { drizzle } from "drizzle-orm/node-postgres";
|
|
|
5
6
|
import { migrate } from "drizzle-orm/node-postgres/migrator";
|
|
6
7
|
import pg from "pg";
|
|
7
8
|
import { logger } from "@n8io/baseline-logger";
|
|
9
|
+
// src/postgres-advisory-lock-take.ts
|
|
10
|
+
const postgresAdvisoryLockTake = async (db, key) => {
|
|
11
|
+
await db.execute(sql`SELECT pg_advisory_xact_lock(hashtextextended(${key}, 0))`);
|
|
12
|
+
};
|
|
8
13
|
// src/postgres-id-column.ts
|
|
9
14
|
const postgresIdColumnCreate = (prefix) => {
|
|
10
15
|
const column = customType({
|
|
@@ -17,6 +22,31 @@ const postgresIdColumnCreate = (prefix) => {
|
|
|
17
22
|
});
|
|
18
23
|
return (name) => name === void 0 ? column() : column(name);
|
|
19
24
|
};
|
|
25
|
+
// src/postgres-keyset.ts
|
|
26
|
+
const isoInstantPattern = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d{1,6})?(Z|[+-]\d{2}:\d{2})$/;
|
|
27
|
+
const isoInstantIs = (value) => {
|
|
28
|
+
if (!isoInstantPattern.test(value) || Number.isNaN(Date.parse(value))) return false;
|
|
29
|
+
const [year = 0, month = 0, day = 0] = value.slice(0, 10).split("-").map(Number);
|
|
30
|
+
return new Date(Date.UTC(year, month - 1, day)).getUTCDate() === day;
|
|
31
|
+
};
|
|
32
|
+
const afterValueAssert = (kind, value) => {
|
|
33
|
+
if (!(kind === "number" ? typeof value === "number" && Number.isFinite(value) : typeof value === "string" && (kind === "text" || isoInstantIs(value)))) throw new TypeError(`A ${kind} keyset needs ${kind === "number" ? "a finite number" : kind === "date" ? "an ISO instant such as 2026-01-01T00:00:00.000Z" : "a string"}, got ${JSON.stringify(value)}`);
|
|
34
|
+
};
|
|
35
|
+
const postgresKeysetCreate = ({ after, descending, idColumn, kind, sortColumn }) => {
|
|
36
|
+
const sortBy = kind === "text" ? sql`${sortColumn} COLLATE "C"` : sql`${sortColumn}`;
|
|
37
|
+
const direction = descending ? sql`DESC` : sql`ASC`;
|
|
38
|
+
const orderBy = [sql`${sortBy} ${direction}`, sql`${idColumn} ${direction}`];
|
|
39
|
+
if (!after) return {
|
|
40
|
+
orderBy,
|
|
41
|
+
where: void 0
|
|
42
|
+
};
|
|
43
|
+
afterValueAssert(kind, after.value);
|
|
44
|
+
const value = kind === "date" ? sql`${after.value}::timestamptz` : sql`${after.value}`;
|
|
45
|
+
return {
|
|
46
|
+
orderBy,
|
|
47
|
+
where: sql`(${sortBy}, ${idColumn}) ${descending ? sql`<` : sql`>`} (${value}, ${sql.param(after.id, idColumn)})`
|
|
48
|
+
};
|
|
49
|
+
};
|
|
20
50
|
// src/postgres-url-assert.ts
|
|
21
51
|
const postgresUrlAssert = (databaseUrl) => {
|
|
22
52
|
if (/^postgres(ql)?:\/\//.test(databaseUrl)) return;
|
|
@@ -52,12 +82,15 @@ const postgresVendorAssert = ({ databaseUrl, owned = {}, vendor }) => {
|
|
|
52
82
|
};
|
|
53
83
|
// src/postgres-migrate.ts
|
|
54
84
|
const postgresMigrateLockKey = createHash("sha256").update("baseline:migrate").digest().readBigInt64BE(0).toString();
|
|
85
|
+
const lockNotAvailable = "55P03";
|
|
55
86
|
const postgresMigrate = async (options) => {
|
|
56
87
|
postgresUrlAssert(options.databaseUrl);
|
|
57
88
|
postgresVendorAssert({
|
|
58
89
|
vendor: options.vendor ?? {},
|
|
59
90
|
databaseUrl: options.databaseUrl
|
|
60
91
|
});
|
|
92
|
+
const lockWaitMs = options.lockWaitMs ?? 6e4;
|
|
93
|
+
if (!Number.isSafeInteger(lockWaitMs) || lockWaitMs <= 0) throw new TypeError("lockWaitMs must be a positive integer");
|
|
61
94
|
const pgClient = new pg.Client({
|
|
62
95
|
...options.vendor,
|
|
63
96
|
connectionString: options.databaseUrl,
|
|
@@ -65,8 +98,16 @@ const postgresMigrate = async (options) => {
|
|
|
65
98
|
});
|
|
66
99
|
await pgClient.connect();
|
|
67
100
|
try {
|
|
101
|
+
const { rows } = await pgClient.query("SELECT pg_backend_pid() AS pid");
|
|
102
|
+
if (rows[0]?.pid !== pgClient.processID) throw new Error("postgresMigrate needs a direct connection to Postgres: databaseUrl goes through a connection pooler, which can hand the migrate lock to another session and keep it after this run ends. Use a URL that reaches Postgres itself");
|
|
68
103
|
await pgClient.query("SET statement_timeout = 0");
|
|
69
|
-
await pgClient.query(
|
|
104
|
+
await pgClient.query(`SET lock_timeout = '${lockWaitMs}ms'`);
|
|
105
|
+
try {
|
|
106
|
+
await pgClient.query("SELECT pg_advisory_lock($1::bigint)", [postgresMigrateLockKey]);
|
|
107
|
+
} catch (error) {
|
|
108
|
+
if (error.code !== lockNotAvailable) throw error;
|
|
109
|
+
throw new Error(`postgresMigrate waited ${lockWaitMs} ms for the migrate lock (advisory lock ${postgresMigrateLockKey}) and gave up: another migrate is still running, or one that ran through a connection pooler left the lock on a pooled connection. Find the holder in pg_locks (locktype 'advisory') and end that session`, { cause: error });
|
|
110
|
+
}
|
|
70
111
|
await pgClient.query("SET lock_timeout = '5s'");
|
|
71
112
|
await migrate(drizzle({ client: pgClient }), { migrationsFolder: options.migrationsFolder });
|
|
72
113
|
} finally {
|
|
@@ -74,19 +115,25 @@ const postgresMigrate = async (options) => {
|
|
|
74
115
|
}
|
|
75
116
|
};
|
|
76
117
|
// src/postgres-pool.ts
|
|
77
|
-
const postgresPoolCreate = ({ connectTimeoutMs = 3e3, databaseUrl, max = 10, vendor = {} }) => {
|
|
118
|
+
const postgresPoolCreate = ({ connectTimeoutMs = 3e3, databaseUrl, max = 10, queryTimeoutMs = 15e3, vendor = {} }) => {
|
|
78
119
|
postgresUrlAssert(databaseUrl);
|
|
79
120
|
postgresVendorAssert({
|
|
80
121
|
vendor,
|
|
81
122
|
databaseUrl,
|
|
82
|
-
owned: {
|
|
123
|
+
owned: {
|
|
124
|
+
max: "max",
|
|
125
|
+
query_timeout: "queryTimeoutMs"
|
|
126
|
+
}
|
|
83
127
|
});
|
|
128
|
+
if (queryTimeoutMs !== null && (!Number.isSafeInteger(queryTimeoutMs) || queryTimeoutMs <= 0)) throw new TypeError("queryTimeoutMs must be a positive integer or null");
|
|
84
129
|
const pgPool = new pg.Pool({
|
|
85
130
|
keepAlive: true,
|
|
131
|
+
keepAliveInitialDelayMillis: 1e4,
|
|
86
132
|
...vendor,
|
|
87
133
|
connectionString: databaseUrl,
|
|
88
134
|
connectionTimeoutMillis: connectTimeoutMs,
|
|
89
|
-
max
|
|
135
|
+
max,
|
|
136
|
+
...queryTimeoutMs === null ? {} : { query_timeout: queryTimeoutMs }
|
|
90
137
|
});
|
|
91
138
|
pgPool.on("error", (error) => {
|
|
92
139
|
logger.error({ err: error }, "Postgres idle client error");
|
|
@@ -134,15 +181,35 @@ const postgresTransactionRun = async ({ pgPool, timeoutMs: timeoutMsGiven, work
|
|
|
134
181
|
pgClient.release(broken || void 0);
|
|
135
182
|
}
|
|
136
183
|
};
|
|
137
|
-
// src/postgres-unique-violation
|
|
138
|
-
const
|
|
184
|
+
// src/postgres-unique-violation.ts
|
|
185
|
+
const violationFind = (error) => {
|
|
139
186
|
const seen = /* @__PURE__ */ new Set();
|
|
140
187
|
let current = error;
|
|
141
188
|
while (typeof current === "object" && current !== null && !seen.has(current)) {
|
|
142
189
|
seen.add(current);
|
|
143
|
-
if ("code" in current && current.code === "23505" && "constraint" in current && current.constraint ===
|
|
190
|
+
if ("code" in current && current.code === "23505" && "constraint" in current && typeof current.constraint === "string") return {
|
|
191
|
+
constraint: current.constraint,
|
|
192
|
+
detail: "detail" in current ? current.detail : void 0
|
|
193
|
+
};
|
|
144
194
|
current = "cause" in current ? current.cause : void 0;
|
|
145
195
|
}
|
|
146
|
-
return false;
|
|
147
196
|
};
|
|
148
|
-
|
|
197
|
+
const postgresUniqueViolationIs = (error, constraint) => violationFind(error)?.constraint === constraint;
|
|
198
|
+
const postgresUniqueViolationRead = (error) => {
|
|
199
|
+
const found = violationFind(error);
|
|
200
|
+
if (!found) return void 0;
|
|
201
|
+
const match = typeof found.detail === "string" ? /^Key \((.*?)\)=\((.*)\) already exists\.$/s.exec(found.detail) : null;
|
|
202
|
+
const columns = match?.[1]?.split(", ") ?? [];
|
|
203
|
+
const printed = match?.[2];
|
|
204
|
+
const values = printed === void 0 ? [] : columns.length === 1 ? [printed] : printed.split(", ");
|
|
205
|
+
return columns.length === values.length ? {
|
|
206
|
+
columns,
|
|
207
|
+
constraint: found.constraint,
|
|
208
|
+
values
|
|
209
|
+
} : {
|
|
210
|
+
columns: [],
|
|
211
|
+
constraint: found.constraint,
|
|
212
|
+
values: []
|
|
213
|
+
};
|
|
214
|
+
};
|
|
215
|
+
export { postgresAdvisoryLockTake, postgresIdColumnCreate, postgresKeysetCreate, postgresMigrate, postgresPoolCreate, postgresTransactionRun, postgresUniqueViolationIs, postgresUniqueViolationRead };
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
/** The pool the helper watches `pg_stat_activity` through: a `PostgresPool` fits. */
|
|
2
|
+
type PostgresTestInterleavePool = {
|
|
3
|
+
query(text: string): Promise<{
|
|
4
|
+
rows: unknown[];
|
|
5
|
+
}>;
|
|
6
|
+
};
|
|
7
|
+
type PostgresTestInterleaveOptions<A, B> = {
|
|
8
|
+
/** Runs until it awaits `pause()`, typically right after it took a lock. */
|
|
9
|
+
first: (pause: () => Promise<void>) => Promise<A>;
|
|
10
|
+
pgPool: PostgresTestInterleavePool;
|
|
11
|
+
/** Starts once `first` paused. */
|
|
12
|
+
second: () => Promise<B>;
|
|
13
|
+
/** How long `second` may run without waiting on a lock or finishing. Default 5 s. */
|
|
14
|
+
timeoutMs?: number | null;
|
|
15
|
+
};
|
|
16
|
+
export declare const postgresTestInterleave: <A, B>({ first, pgPool, second, timeoutMs: timeoutMsGiven }: PostgresTestInterleaveOptions<A, B>) => Promise<{
|
|
17
|
+
first: A;
|
|
18
|
+
second: B;
|
|
19
|
+
secondWaited: boolean;
|
|
20
|
+
}>;
|
|
21
|
+
export type { PostgresTestInterleaveOptions, PostgresTestInterleavePool };
|
package/dist/testing.js
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
import { setTimeout } from "node:timers/promises";
|
|
2
|
+
// src/postgres-test-interleave.ts
|
|
3
|
+
const lockWaitersRead = async (pgPool) => {
|
|
4
|
+
const { rows } = await pgPool.query(`SELECT pid FROM pg_stat_activity
|
|
5
|
+
WHERE datname = current_database() AND wait_event_type = 'Lock' AND pid <> pg_backend_pid()`);
|
|
6
|
+
return new Set(rows.map((row) => typeof row === "object" && row !== null && "pid" in row ? row.pid : void 0));
|
|
7
|
+
};
|
|
8
|
+
const postgresTestInterleave = async ({ first, pgPool, second, timeoutMs: timeoutMsGiven }) => {
|
|
9
|
+
const timeoutMs = timeoutMsGiven ?? 5e3;
|
|
10
|
+
const { promise: paused, resolve: pausedResolve } = Promise.withResolvers();
|
|
11
|
+
const { promise: resumed, resolve: resume } = Promise.withResolvers();
|
|
12
|
+
let pauseCalled = false;
|
|
13
|
+
let resumeCalled = false;
|
|
14
|
+
const firstRun = Promise.try(() => first(() => {
|
|
15
|
+
if (pauseCalled) throw new Error("first called pause() twice");
|
|
16
|
+
pauseCalled = true;
|
|
17
|
+
pausedResolve();
|
|
18
|
+
return resumed;
|
|
19
|
+
})).then((value) => {
|
|
20
|
+
if (!resumeCalled) throw new Error("first finished before it was resumed; await pause()");
|
|
21
|
+
return value;
|
|
22
|
+
});
|
|
23
|
+
if (await Promise.race([paused.then(() => false), firstRun.then(() => true, () => true)]) && !pauseCalled) await firstRun.catch((error) => {
|
|
24
|
+
throw error instanceof Error && error.message.startsWith("first finished before") ? /* @__PURE__ */ new Error("first finished without calling pause()") : error;
|
|
25
|
+
});
|
|
26
|
+
let secondWaited = false;
|
|
27
|
+
try {
|
|
28
|
+
const waitingBefore = await lockWaitersRead(pgPool);
|
|
29
|
+
let secondDone = false;
|
|
30
|
+
const secondRun = Promise.try(second).finally(() => {
|
|
31
|
+
secondDone = true;
|
|
32
|
+
});
|
|
33
|
+
secondRun.catch(() => void 0);
|
|
34
|
+
const deadline = Date.now() + timeoutMs;
|
|
35
|
+
for (;;) {
|
|
36
|
+
if ([...await lockWaitersRead(pgPool)].some((pid) => !waitingBefore.has(pid))) {
|
|
37
|
+
secondWaited = true;
|
|
38
|
+
break;
|
|
39
|
+
}
|
|
40
|
+
if (secondDone) break;
|
|
41
|
+
if (Date.now() > deadline) throw new Error(`second neither waited on a lock nor finished within ${timeoutMs} ms`);
|
|
42
|
+
await setTimeout(10);
|
|
43
|
+
}
|
|
44
|
+
resumeCalled = true;
|
|
45
|
+
resume();
|
|
46
|
+
const [firstResult, secondResult] = await Promise.all([firstRun, secondRun]);
|
|
47
|
+
return {
|
|
48
|
+
first: firstResult,
|
|
49
|
+
second: secondResult,
|
|
50
|
+
secondWaited
|
|
51
|
+
};
|
|
52
|
+
} finally {
|
|
53
|
+
resumeCalled = true;
|
|
54
|
+
resume();
|
|
55
|
+
}
|
|
56
|
+
};
|
|
57
|
+
export { postgresTestInterleave };
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@n8io/baseline-postgres",
|
|
3
|
-
"version": "5.
|
|
3
|
+
"version": "5.2.0",
|
|
4
4
|
"description": "Postgres pool, Result-aware transactions, migrations and prefixed-id columns on pg and Drizzle.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -20,6 +20,10 @@
|
|
|
20
20
|
".": {
|
|
21
21
|
"types": "./dist/index.d.ts",
|
|
22
22
|
"default": "./dist/index.js"
|
|
23
|
+
},
|
|
24
|
+
"./testing": {
|
|
25
|
+
"types": "./dist/testing.d.ts",
|
|
26
|
+
"default": "./dist/testing.js"
|
|
23
27
|
}
|
|
24
28
|
},
|
|
25
29
|
"publishConfig": {
|