@ultimat3/db 16.0.0 → 17.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +66 -0
- package/README.md +3 -2
- package/package.json +2 -2
- package/src/bun-sql.ts +32 -0
- package/src/client.ts +14 -232
- package/src/connection-url.ts +60 -0
- package/src/db-health.ts +31 -0
- package/src/default-client.ts +2 -1
- package/src/drift-errors.ts +36 -0
- package/src/drift-findings.ts +63 -6
- package/src/drift-fixtures.ts +23 -0
- package/src/errors.ts +0 -9
- package/src/index.ts +15 -15
- package/src/migrate.ts +19 -11
- package/src/pool-profile.ts +125 -0
- package/src/pool-reserve.ts +50 -0
- package/src/readonly-query.ts +21 -8
- package/src/replica-client.ts +16 -3
- package/src/sql.ts +36 -0
package/CLAUDE.md
CHANGED
|
@@ -9,6 +9,7 @@ reaches down to this package for it. **Never** import `entity`, `jobs`, `http` o
|
|
|
9
9
|
|---|---|
|
|
10
10
|
| Deps | none. `@electric-sql/pglite` is an **optional peer**, imported by variable specifier inside `loadPgliteDriver()` so no consumer's `tsc` or bundler resolves it. **No ORM** — `entity`'s hand-written `postgresDriver()` is the production backing |
|
|
11
11
|
| SQL | `sql` binds `$n`; anything non-scalar and non-fragment throws `X_SQL_UNSAFE` |
|
|
12
|
+
| A name reaching a `fix:` | `shellInertIdentifier()` (`sql.ts`), the tree's ONE screen for it, `As of 2026-08-26`. `identifier()` alone does not close it: it refuses `"`, `\` and whitespace and **accepts** a backtick and a `$` — `SAFE_IDENTIFIER` allows `$` on its fast path — which are the two characters a shell substitutes inside DOUBLE quotes. A refused name is left OUT of the command, never escaped into it |
|
|
12
13
|
| Escape hatches | `raw()`, `identifier()`, `literal()` — each call is an audit point. `literal()` is the tree's ONE SQL-string-literal escape (`scripts/sql-literal-copies.ts`, pinned at zero) and it emits `E'…'` when the value carries a backslash |
|
|
13
14
|
| SQLSTATE | one reader, `sqlState()` (`sqlstate.ts`). Never read `error.code` for a SQLSTATE |
|
|
14
15
|
| Reading a caught value | `renderThrowable()` from core; never `error instanceof Error ? error.message : String(error)` — both halves RUN app code (a `Proxy` trap, `Symbol.toPrimitive`) and `checkDb` backs `/readyz`, where a render that throws is an exception in place of the report the kubelet asked for |
|
|
@@ -1276,12 +1277,77 @@ takes, and left unsaid an `alter database … set statement_timeout` on the serv
|
|
|
1276
1277
|
that must outlive it. The splitter honours libpq's backslash escape, so a `search_path=two\ words`
|
|
1277
1278
|
survives the round trip whole.
|
|
1278
1279
|
|
|
1280
|
+
- **Every numeric option this package bounds anything with is screened, `As of 2026-08-26`** —
|
|
1281
|
+
through core's `finiteCount`, which borrows `X_INVARIANT` as this package already does.
|
|
1282
|
+
`replicaClient`'s `breakerFailures` and `breakerCooldownMs` (a breaker is two comparisons and
|
|
1283
|
+
nothing else: `failures >= NaN` never opens it, `monotonic() < NaN` never parks it, so every read
|
|
1284
|
+
keeps going to the replica that is failing), `migrate`'s `lockWaitMs` (`NaN - elapsed <= 0` is
|
|
1285
|
+
false and `Bun.sleep(NaN)` does not sleep — a tight spin re-taking `pg_try_advisory_lock`, not an
|
|
1286
|
+
unbounded wait) and `readonlyQuery`'s `timeoutMs`, plus `client.ts`'s pool profile. The last one
|
|
1287
|
+
is a **behaviour change**: it used to normalise `NaN` to the default silently, so an agent read
|
|
1288
|
+
ran under a ceiling nobody wrote. Only an explicit `0` disables that layer, which is why its floor
|
|
1289
|
+
is 0 and not 1.
|
|
1290
|
+
|
|
1291
|
+
- **`client.ts` reached the 500-line ceiling on 2026-08-26, and shed the five jobs that were not
|
|
1292
|
+
"open a connection and send a statement".** `pool-profile.ts` owns the five numbers a pool runs
|
|
1293
|
+
on — the per-role table, `DATABASE_POOL_MAX` and the screen every merged profile passes;
|
|
1294
|
+
`connection-url.ts` builds the connection string (the libpq `options` merge and the
|
|
1295
|
+
`application_name` label); `bun-sql.ts` declares the slice of `Bun.SQL` this package uses and
|
|
1296
|
+
looks the global up lazily;
|
|
1297
|
+
`pool-reserve.ts` is `reserve()` under the acquire deadline; `db-health.ts` is `checkDb`, the
|
|
1298
|
+
`/readyz` report. `client.ts` keeps connecting, the statement funnel and the ambient `db()` —
|
|
1299
|
+
and it still opens no socket at import, because `bunSqlFactory()` is reached from inside
|
|
1300
|
+
`connect()`. **The public surface did not move**: `src/index.ts` exports every one of those
|
|
1301
|
+
names from its new module, so `@ultimat3/db` is byte-identical to what it was. The same day,
|
|
1302
|
+
`drift.test.ts` split three ways along the three questions it was asking — `drift.test.ts`
|
|
1303
|
+
(tables and columns), `drift-index.test.ts` and `drift-ledger.test.ts` (what the migrations
|
|
1304
|
+
declare, and the post-migrate check) — over one shared `drift-fixtures.ts`, which
|
|
1305
|
+
`drift-foreign-key.test.ts` now imports instead of carrying its own byte-identical copy.
|
|
1306
|
+
|
|
1307
|
+
- **`unexpectedTable`'s `fix:` no longer names `x db gen`, `As of 2026-08-26`** (issue #345). That
|
|
1308
|
+
command diffs the ENTITY REGISTRY against the newest snapshot, and a table nothing declares is on
|
|
1309
|
+
neither side of it — so the diff came back empty, the generator's empty-diff branch writes NO
|
|
1310
|
+
file, and the reader had nothing to run and the same finding on the next deploy. The two edits
|
|
1311
|
+
that do resolve it are named instead: a `create table if not exists` in a migration (which
|
|
1312
|
+
`x db migrate` then accepts, through `@ultimat3/cli`'s `acceptCreatedTables`), or `psql … drop
|
|
1313
|
+
table` for a table nothing owns. No migration PATH is named — where an app keeps its migrations is
|
|
1314
|
+
the CLI's fact. `X_DB_DRIFT` is a shipped code and is unchanged; only this `fix:` text moved.
|
|
1315
|
+
|
|
1316
|
+
- **A name a `fix:` puts in a command is screened ONCE, by `shellInertIdentifier()` (`sql.ts`),
|
|
1317
|
+
`As of 2026-08-26`.** `identifier()` answers about SQL and cannot close this: it refuses `"`,
|
|
1318
|
+
`\` and whitespace and **accepts** a backtick and a `$` — `SAFE_IDENTIFIER` allows `$` on its
|
|
1319
|
+
fast path — which are exactly the two characters a shell substitutes inside DOUBLE quotes. A
|
|
1320
|
+
`fix:` is pasted into a shell at least as often as into a psql session, so a column named
|
|
1321
|
+
`$(id)` inside `x db gen "add $(id)"` RUNS `id` the moment its reader pastes the line, and a
|
|
1322
|
+
screen reusing `identifier()` unchanged ships a green suite over a live command-execution hole.
|
|
1323
|
+
It began as a private `writableName` in `drift-findings.ts` and was promoted rather than copied:
|
|
1324
|
+
three copies of a string-literal escape shipped here once and two were wrong the same way
|
|
1325
|
+
(`scripts/sql-literal-copies.ts`). Callers **degrade to prose** — the argument to `x db gen` is a
|
|
1326
|
+
migration DESCRIPTION, not an identifier, so no quoted form makes a hostile name safe to pass,
|
|
1327
|
+
and the name is read off `cause`/`meta` instead. Every benign rendering is byte-identical: the
|
|
1328
|
+
screen sits on the refusal branch alone, because roughly ten pages across `packages/cli`,
|
|
1329
|
+
`packages/core`, `wiki/` and `docs/` quote `x db gen "add <name>"` verbatim.
|
|
1330
|
+
|
|
1331
|
+
- **`dbDrift()` lives in `drift-errors.ts` and not in `errors.ts`, for exactly the reason
|
|
1332
|
+
`dependent-view.ts` states.** Its `fix:` needs `shellInertIdentifier` and `sql.ts` imports
|
|
1333
|
+
`errors.ts`, so keeping the constructor there is an import cycle around the module whose
|
|
1334
|
+
evaluation REGISTERS every code. `dependent-view.ts` avoided the same cycle by handing
|
|
1335
|
+
`errors.ts` a finished string; that is not available here, because `dbDrift(table, column)` is
|
|
1336
|
+
public API shipped since 1.0 and its signature cannot change. So the constructor moved instead,
|
|
1337
|
+
the way `migration-errors.ts` and `invariant-errors.ts` did — `X_DB_DRIFT` is still declared,
|
|
1338
|
+
titled and registered in `errors.ts`, and `src/index.ts` still exports the same name, so the
|
|
1339
|
+
public surface is byte-identical. `@ultimat3/entity`'s mirror screens through the **same**
|
|
1340
|
+
export across the tier seam (tier 2 → tier 1), which is what keeps the "keep in sync" comment on
|
|
1341
|
+
both declarations true; `packages/entity/src/errors.test.ts` asserts the two texts are equal,
|
|
1342
|
+
so a one-sided edit is a failing test rather than a comment nobody read.
|
|
1343
|
+
|
|
1279
1344
|
```bash
|
|
1280
1345
|
bun test # from packages/db
|
|
1281
1346
|
bun run typecheck
|
|
1282
1347
|
```
|
|
1283
1348
|
|
|
1284
1349
|
Gotchas:
|
|
1350
|
+
|
|
1285
1351
|
- `exactOptionalPropertyTypes` — declare optional fields as `x?: T | undefined`.
|
|
1286
1352
|
- `noUncheckedIndexedAccess` — array reads are `T | undefined`; `chunks[i] ?? ''` everywhere.
|
|
1287
1353
|
- Tests use `createRecordingClient()` + `setDbClient()`; no test may need a live database.
|
package/README.md
CHANGED
|
@@ -26,6 +26,7 @@ await withTransaction(async (tx) => {
|
|
|
26
26
|
| Export | |
|
|
27
27
|
|---|---|
|
|
28
28
|
| `sql` / `raw` / `identifier` / `literal` / `join` | fragment builders |
|
|
29
|
+
| `shellInertIdentifier()` | `As of 2026-08-26`: a quoted identifier that is also inert wherever a human PASTES it — or `null`. The one screen a catalog name goes through before it reaches a `fix:`. `identifier()` answers about SQL and **accepts** a backtick and a `$`, which are exactly what a shell substitutes inside double quotes, so a column called `$(id)` inside `x db gen "add $(id)"` runs `id` on paste |
|
|
29
30
|
| `db()` / `baseClient()` / `setDbClient()` | the ambient client; `db()` returns the open tx if any |
|
|
30
31
|
| `DbTx.origin` | `As of 2026-08`: the client the transaction was **opened on** — `options.client` or `baseClient()`, never the reservation it runs statements through. `@ultimat3/entity` compares a pinned repository's client against it, so a pinned repo joins its own shard's transaction instead of being refused |
|
|
31
32
|
| `withTransaction()` / `currentTx()` | transaction scope; `currentTx()` is the outbox seam. `{ retry: n }` (`As of 2026-08`) re-runs `fn` from the top on a `40001`/`40P01` and on nothing else — default 0, so `fn` must be idempotent before you ask for it. Each re-run **waits first**, `As of 2026-08-23`: exponential from 10ms, capped at 500ms, full jitter (`@ultimat3/core`'s `backoffDelay`). A budget of 0 waits not at all |
|
|
@@ -216,9 +217,9 @@ X_DB_DRIFT: schema differs from migrations
|
|
|
216
217
|
|
|
217
218
|
| Difference | cause | fix |
|
|
218
219
|
|---|---|---|
|
|
219
|
-
| live column, no migration | `table "T" has column "C" not present in any migration` | `x db gen "add C"` |
|
|
220
|
+
| live column, no migration | `table "T" has column "C" not present in any migration` | `x db gen "add C"` — the name goes through `shellInertIdentifier()`, and one it refuses is left OUT of the command rather than escaped into it (`x db gen "add the undeclared column"`, the name in the cause) |
|
|
220
221
|
| migrated column, not live | `table "T" is missing column "C" that migrations declare` | `x db migrate` |
|
|
221
|
-
| live table, no migration | `table "T" is not present in any migration` | `x db gen
|
|
222
|
+
| live table, no migration | `table "T" is not present in any migration` | a `create table if not exists` in a migration, then `x db migrate` — or `drop table` in `psql` where nothing owns it. Never `x db gen`, which diffs a table nothing declares against nothing and writes no file (issue #345). The name goes through `shellInertIdentifier()`, and one it refuses leaves the fix as prose |
|
|
222
223
|
| migrated table, not live | `table "T" is declared by migrations but does not exist` | `x db migrate` |
|
|
223
224
|
| index rebuilt differently | `index "I" on "T" covers (…)` / `is unique` / `is descending` / `is partial`, `not what migrations declare` | `x db migrate` |
|
|
224
225
|
| foreign key, rule moved | `foreign key on "T" (C) to "R" is on delete cascade, not what migrations declare` | the `drop constraint` + `add constraint` pair, in a new migration |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ultimat3/db",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "17.0.0",
|
|
4
4
|
"description": "Postgres access, transactions, migrations and drift detection",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -31,7 +31,7 @@
|
|
|
31
31
|
"test": "bun test"
|
|
32
32
|
},
|
|
33
33
|
"dependencies": {
|
|
34
|
-
"@ultimat3/core": "
|
|
34
|
+
"@ultimat3/core": "17.0.0"
|
|
35
35
|
},
|
|
36
36
|
"peerDependencies": {
|
|
37
37
|
"@electric-sql/pglite": ">=0.5.0"
|
package/src/bun-sql.ts
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
// Single responsibility: the slice of `Bun.SQL` this package uses, declared structurally, and the
|
|
2
|
+
// lazy lookup of the global that provides it. Reached through a function so importing the client
|
|
3
|
+
// never touches `Bun` at module evaluation — the CLI imports it to print help.
|
|
4
|
+
|
|
5
|
+
import { dbUnavailable } from './errors';
|
|
6
|
+
|
|
7
|
+
/** One connection pinned out of `Bun.SQL`'s pool, released back by hand. */
|
|
8
|
+
export interface BunSqlReserved {
|
|
9
|
+
unsafe(text: string, values?: readonly unknown[]): Promise<unknown>;
|
|
10
|
+
release(): void;
|
|
11
|
+
}
|
|
12
|
+
|
|
13
|
+
/** The slice of `Bun.SQL` we use. Declared structurally so this package has no dependency. */
|
|
14
|
+
export interface BunSqlDriver {
|
|
15
|
+
unsafe(text: string, values?: readonly unknown[]): Promise<unknown>;
|
|
16
|
+
reserve(): Promise<BunSqlReserved>;
|
|
17
|
+
close(options?: { readonly timeout?: number }): Promise<void>;
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
export type BunSqlFactory = new (
|
|
21
|
+
url: string,
|
|
22
|
+
options?: Readonly<Record<string, unknown>>,
|
|
23
|
+
) => BunSqlDriver;
|
|
24
|
+
|
|
25
|
+
export function bunSqlFactory(): BunSqlFactory {
|
|
26
|
+
const host = globalThis as unknown as { readonly Bun?: { readonly SQL?: unknown } };
|
|
27
|
+
const factory = host.Bun?.SQL;
|
|
28
|
+
if (typeof factory !== 'function') {
|
|
29
|
+
throw dbUnavailable('Bun.SQL is unavailable — this package requires Bun >= 1.3');
|
|
30
|
+
}
|
|
31
|
+
return factory as BunSqlFactory;
|
|
32
|
+
}
|
package/src/client.ts
CHANGED
|
@@ -1,17 +1,20 @@
|
|
|
1
|
-
// Single responsibility: the Postgres
|
|
2
|
-
//
|
|
3
|
-
//
|
|
4
|
-
//
|
|
1
|
+
// Single responsibility: the pooled Postgres client and the ambient `db()` handle — one statement
|
|
2
|
+
// funnel, the reserved-connection pin, and the process-wide client every repository reaches
|
|
3
|
+
// through. Sizing lives in `pool-profile.ts`, the connection string in `connection-url.ts`, the
|
|
4
|
+
// `Bun.SQL` slice in `bun-sql.ts`, so importing this module never opens a socket.
|
|
5
5
|
|
|
6
|
-
import { type Role,
|
|
6
|
+
import { type Role, resolveRole } from '@ultimat3/core';
|
|
7
7
|
import { statementAttribution } from './attribution';
|
|
8
|
+
import { type BunSqlDriver, type BunSqlReserved, bunSqlFactory } from './bun-sql';
|
|
9
|
+
import { connectionUrl } from './connection-url';
|
|
8
10
|
// Deliberate cycle, the same shape as `client.ts ⇄ transaction.ts`: nothing here is referenced at
|
|
9
11
|
// module evaluation, and both sides are `function` declarations, so hoisting covers the TDZ.
|
|
10
12
|
import { defaultClient } from './default-client';
|
|
11
|
-
import { DbError,
|
|
13
|
+
import { DbError, driverError } from './errors';
|
|
12
14
|
import { expectedQueryLoopReason } from './expected-loop';
|
|
13
|
-
import { declaresLibpqOption, mergeLibpqOptions } from './libpq-options';
|
|
14
15
|
import { statementObserver } from './observe';
|
|
16
|
+
import { assertPoolProfile, type PoolProfile, poolProfileFor } from './pool-profile';
|
|
17
|
+
import { reserveWithin } from './pool-reserve';
|
|
15
18
|
import { type SqlFragment, sql } from './sql';
|
|
16
19
|
import { withStatementSpan } from './statement-span';
|
|
17
20
|
import { currentTx } from './transaction';
|
|
@@ -41,123 +44,6 @@ export function isReservable(client: DbClient): client is ReservableClient {
|
|
|
41
44
|
return typeof (client as Partial<ReservableClient>).reserve === 'function';
|
|
42
45
|
}
|
|
43
46
|
|
|
44
|
-
export interface PoolProfile {
|
|
45
|
-
readonly max: number;
|
|
46
|
-
/** 0 disables the timeout — only `migrate`, which is allowed to take as long as it takes. */
|
|
47
|
-
readonly statementTimeoutMs: number;
|
|
48
|
-
readonly idleTimeoutMs: number;
|
|
49
|
-
/**
|
|
50
|
-
* How long a statement may **wait for a lock** before `55P03`, distinct from how long it may run.
|
|
51
|
-
* 0 everywhere but `migrate`, which is the only role that takes `ACCESS EXCLUSIVE`: an `alter
|
|
52
|
-
* table` queued behind a long `SELECT` puts every later query on that table behind it too,
|
|
53
|
-
* because Postgres' lock queue is FIFO — and `migrate` runs `statement_timeout = 0`, so nothing
|
|
54
|
-
* else would ever end the wait. Read by `migrate()` as a `SET LOCAL`, never by the pool.
|
|
55
|
-
*/
|
|
56
|
-
readonly lockTimeoutMs: number;
|
|
57
|
-
/**
|
|
58
|
-
* How long `reserve()` may wait for a free connection before `X_DB_POOL_EXHAUSTED`. 0 waits
|
|
59
|
-
* forever, which is what a run-once role wants and what a request-serving one must never do:
|
|
60
|
-
* queueing turns exhaustion into a hang, `/readyz`'s `select 1` joins the same queue, the kubelet
|
|
61
|
-
* kills the pod, and the replacement inherits the same saturated database.
|
|
62
|
-
*/
|
|
63
|
-
readonly acquireTimeoutMs: number;
|
|
64
|
-
}
|
|
65
|
-
|
|
66
|
-
/** Sized per role because the failure modes differ: RPS bursts vs. queue depth vs. run-once. */
|
|
67
|
-
export const POOL_PROFILES = Object.freeze<Record<Role, PoolProfile>>({
|
|
68
|
-
web: {
|
|
69
|
-
max: 20,
|
|
70
|
-
statementTimeoutMs: 10_000,
|
|
71
|
-
idleTimeoutMs: 30_000,
|
|
72
|
-
lockTimeoutMs: 0,
|
|
73
|
-
acquireTimeoutMs: 5_000,
|
|
74
|
-
},
|
|
75
|
-
sync: {
|
|
76
|
-
max: 10,
|
|
77
|
-
statementTimeoutMs: 10_000,
|
|
78
|
-
idleTimeoutMs: 60_000,
|
|
79
|
-
lockTimeoutMs: 0,
|
|
80
|
-
acquireTimeoutMs: 5_000,
|
|
81
|
-
},
|
|
82
|
-
worker: {
|
|
83
|
-
max: 8,
|
|
84
|
-
statementTimeoutMs: 120_000,
|
|
85
|
-
idleTimeoutMs: 30_000,
|
|
86
|
-
lockTimeoutMs: 0,
|
|
87
|
-
acquireTimeoutMs: 10_000,
|
|
88
|
-
},
|
|
89
|
-
scheduler: {
|
|
90
|
-
max: 2,
|
|
91
|
-
statementTimeoutMs: 15_000,
|
|
92
|
-
idleTimeoutMs: 60_000,
|
|
93
|
-
lockTimeoutMs: 0,
|
|
94
|
-
acquireTimeoutMs: 10_000,
|
|
95
|
-
},
|
|
96
|
-
// `migrate` waits: its pool is `max: 1` and the advisory-lock pin holds it for the whole run, so
|
|
97
|
-
// a deadline here would refuse the migration's own session. The wait that needed bounding is the
|
|
98
|
-
// advisory lock's, and `MIGRATION_LOCK_WAIT_MS` bounds it.
|
|
99
|
-
migrate: {
|
|
100
|
-
max: 1,
|
|
101
|
-
statementTimeoutMs: 0,
|
|
102
|
-
idleTimeoutMs: 10_000,
|
|
103
|
-
lockTimeoutMs: 3_000,
|
|
104
|
-
acquireTimeoutMs: 0,
|
|
105
|
-
},
|
|
106
|
-
replicator: {
|
|
107
|
-
max: 4,
|
|
108
|
-
statementTimeoutMs: 0,
|
|
109
|
-
idleTimeoutMs: 60_000,
|
|
110
|
-
lockTimeoutMs: 0,
|
|
111
|
-
acquireTimeoutMs: 0,
|
|
112
|
-
},
|
|
113
|
-
});
|
|
114
|
-
|
|
115
|
-
export function poolProfileFor(role: Role = resolveRole()): PoolProfile {
|
|
116
|
-
return POOL_PROFILES[role];
|
|
117
|
-
}
|
|
118
|
-
|
|
119
|
-
/** The one pool knob an operator can turn without a rebuild. Layered over the role default. */
|
|
120
|
-
export const POOL_MAX_ENV = 'DATABASE_POOL_MAX';
|
|
121
|
-
|
|
122
|
-
/**
|
|
123
|
-
* `DATABASE_POOL_MAX`, or nothing. `POOL_PROFILES` is frozen into the build, so before this the
|
|
124
|
-
* only way to change a fleet's connection count was to ship a new image — and 400 `web` pods at
|
|
125
|
-
* `max: 20` is 8,000 backends against a `max_connections` of 450. An unparseable value **refuses**
|
|
126
|
-
* rather than falling back: a fleet that ignored the number it was given is the failure the
|
|
127
|
-
* variable exists to prevent, and it would only be found in `pg_stat_activity` at 3am.
|
|
128
|
-
*/
|
|
129
|
-
export function poolMaxFromEnv(): Partial<PoolProfile> {
|
|
130
|
-
const raw = process.env[POOL_MAX_ENV];
|
|
131
|
-
if (raw === undefined || raw.trim() === '') return {};
|
|
132
|
-
const max = Number(raw);
|
|
133
|
-
if (!Number.isSafeInteger(max) || max < 1) throw poolMaxInvalid(raw);
|
|
134
|
-
return { max };
|
|
135
|
-
}
|
|
136
|
-
|
|
137
|
-
/** One connection pinned out of `Bun.SQL`'s pool, released back by hand. */
|
|
138
|
-
interface BunSqlReserved {
|
|
139
|
-
unsafe(text: string, values?: readonly unknown[]): Promise<unknown>;
|
|
140
|
-
release(): void;
|
|
141
|
-
}
|
|
142
|
-
|
|
143
|
-
/** The slice of `Bun.SQL` we use. Declared structurally so this package has no dependency. */
|
|
144
|
-
interface BunSqlDriver {
|
|
145
|
-
unsafe(text: string, values?: readonly unknown[]): Promise<unknown>;
|
|
146
|
-
reserve(): Promise<BunSqlReserved>;
|
|
147
|
-
close(options?: { readonly timeout?: number }): Promise<void>;
|
|
148
|
-
}
|
|
149
|
-
|
|
150
|
-
type BunSqlFactory = new (url: string, options?: Readonly<Record<string, unknown>>) => BunSqlDriver;
|
|
151
|
-
|
|
152
|
-
function bunSqlFactory(): BunSqlFactory {
|
|
153
|
-
const host = globalThis as unknown as { readonly Bun?: { readonly SQL?: unknown } };
|
|
154
|
-
const factory = host.Bun?.SQL;
|
|
155
|
-
if (typeof factory !== 'function') {
|
|
156
|
-
throw dbUnavailable('Bun.SQL is unavailable — this package requires Bun >= 1.3');
|
|
157
|
-
}
|
|
158
|
-
return factory as BunSqlFactory;
|
|
159
|
-
}
|
|
160
|
-
|
|
161
47
|
export interface PostgresClientOptions {
|
|
162
48
|
readonly url?: string | undefined;
|
|
163
49
|
readonly role?: Role | undefined;
|
|
@@ -165,46 +51,6 @@ export interface PostgresClientOptions {
|
|
|
165
51
|
readonly applicationName?: string | undefined;
|
|
166
52
|
}
|
|
167
53
|
|
|
168
|
-
function connectionUrl(options: PostgresClientOptions, profile: PoolProfile): string {
|
|
169
|
-
const raw = options.url ?? process.env['DATABASE_URL'];
|
|
170
|
-
if (raw === undefined || raw === '') {
|
|
171
|
-
throw dbUnavailable('DATABASE_URL is not set, so there is no database to connect to');
|
|
172
|
-
}
|
|
173
|
-
let url: URL;
|
|
174
|
-
try {
|
|
175
|
-
url = new URL(raw);
|
|
176
|
-
} catch (error) {
|
|
177
|
-
throw dbUnavailable(`DATABASE_URL is not a valid url: ${raw}`, error);
|
|
178
|
-
}
|
|
179
|
-
// libpq `options` is the portable way to pin a statement timeout for every pooled connection —
|
|
180
|
-
// MERGED into the operator's own, never assigned over it, and emitted for every role including
|
|
181
|
-
// the two whose bound is 0. `set` here dropped a `?options=-c search_path=app` on `web`, `sync`,
|
|
182
|
-
// `worker` and `scheduler` and kept it on `migrate` and `replicator`, so the role that runs the
|
|
183
|
-
// migrations and the role that serves the traffic read different schemas. 0 is a value, not a
|
|
184
|
-
// silence: it is `migrate` saying it may take as long as it takes, and left unsaid a server-side
|
|
185
|
-
// `alter database ... set statement_timeout` kills the one role that must outlive it.
|
|
186
|
-
// `application_name` is a LABEL, not a bound: 'ultimate' is a DEFAULT, and a default may not
|
|
187
|
-
// overwrite what the operator wrote — `?application_name=billing-api` is the filter their
|
|
188
|
-
// `pg_stat_activity` query, their pooler rule and their audit rule all match on, and losing it
|
|
189
|
-
// is silent. Both spellings count, or the URL parameter and a `-c application_name=` in
|
|
190
|
-
// `options` disagree and which one the backend honours is argument order nobody here measured.
|
|
191
|
-
const named = options.applicationName;
|
|
192
|
-
const settings: Record<string, string> = {
|
|
193
|
-
statement_timeout: String(profile.statementTimeoutMs),
|
|
194
|
-
};
|
|
195
|
-
const inOptions = declaresLibpqOption(url.searchParams.get('options'), 'application_name');
|
|
196
|
-
// An explicit `applicationName` is a deliberate call by the role that opened the pool, so it
|
|
197
|
-
// wins. Only then is the setting named to the merge, and only when the operator wrote the other
|
|
198
|
-
// spelling: `mergeLibpqOptions` drops their assignment before appending, so the two cannot
|
|
199
|
-
// disagree — and a URL with no assignment in it keeps the exact `options` it always had.
|
|
200
|
-
if (named !== undefined && inOptions) settings['application_name'] = named;
|
|
201
|
-
const declared = url.searchParams.has('application_name') || inOptions;
|
|
202
|
-
url.searchParams.set('options', mergeLibpqOptions(url.searchParams.get('options'), settings));
|
|
203
|
-
if (named !== undefined) url.searchParams.set('application_name', named);
|
|
204
|
-
else if (!declared) url.searchParams.set('application_name', 'ultimate');
|
|
205
|
-
return url.toString();
|
|
206
|
-
}
|
|
207
|
-
|
|
208
54
|
function rowsOf<T>(result: unknown): readonly T[] {
|
|
209
55
|
return Array.isArray(result) ? (result as readonly T[]) : [];
|
|
210
56
|
}
|
|
@@ -219,49 +65,6 @@ function affectedBy(result: unknown): number {
|
|
|
219
65
|
return typeof count === 'number' && count > 0 ? count : result.length;
|
|
220
66
|
}
|
|
221
67
|
|
|
222
|
-
/**
|
|
223
|
-
* `pool.reserve()` under a deadline. Without one an exhausted pool does not fail, it **queues** —
|
|
224
|
-
* so a slow endpoint filling all 20 slots turns every later request, `/readyz`'s `select 1`
|
|
225
|
-
* included, into a wait with no end and no error, and the pod is killed for being unready rather
|
|
226
|
-
* than answering 503 for the requests it cannot serve.
|
|
227
|
-
*
|
|
228
|
-
* The losing reservation is released, never dropped: the pool hands out a connection whenever one
|
|
229
|
-
* frees, deadline or no deadline, and a pin nobody holds is a connection nobody gets back. That is
|
|
230
|
-
* the whole reason this is not a bare `Promise.race`.
|
|
231
|
-
*/
|
|
232
|
-
async function reserveWithin(
|
|
233
|
-
pool: Pick<BunSqlDriver, 'reserve'>,
|
|
234
|
-
profile: PoolProfile,
|
|
235
|
-
): Promise<BunSqlReserved> {
|
|
236
|
-
const budget = profile.acquireTimeoutMs;
|
|
237
|
-
if (budget <= 0) return pool.reserve();
|
|
238
|
-
let timer: ReturnType<typeof setTimeout> | undefined;
|
|
239
|
-
let expired = false;
|
|
240
|
-
const pending = pool.reserve();
|
|
241
|
-
try {
|
|
242
|
-
return await Promise.race([
|
|
243
|
-
pending,
|
|
244
|
-
new Promise<never>((_resolve, reject) => {
|
|
245
|
-
timer = setTimeout(() => {
|
|
246
|
-
expired = true;
|
|
247
|
-
reject(poolAcquireTimeout(budget, profile.max));
|
|
248
|
-
}, budget);
|
|
249
|
-
// The deadline must not be what keeps a finished process alive.
|
|
250
|
-
timer.unref?.();
|
|
251
|
-
}),
|
|
252
|
-
]);
|
|
253
|
-
} finally {
|
|
254
|
-
if (timer !== undefined) clearTimeout(timer);
|
|
255
|
-
// Attached unconditionally so a rejection arriving after we gave up is handled, not unhandled.
|
|
256
|
-
void pending.then(
|
|
257
|
-
(late) => {
|
|
258
|
-
if (expired) late.release();
|
|
259
|
-
},
|
|
260
|
-
() => undefined,
|
|
261
|
-
);
|
|
262
|
-
}
|
|
263
|
-
}
|
|
264
|
-
|
|
265
68
|
export interface PostgresClient extends ReservableClient {
|
|
266
69
|
readonly profile: PoolProfile;
|
|
267
70
|
ping(): Promise<void>;
|
|
@@ -271,7 +74,10 @@ export interface PostgresClient extends ReservableClient {
|
|
|
271
74
|
/** Lazily connects: the pool opens on the first statement, never at import. */
|
|
272
75
|
export function createPostgresClient(options: PostgresClientOptions = {}): PostgresClient {
|
|
273
76
|
const role = options.role ?? resolveRole();
|
|
274
|
-
const profile: PoolProfile =
|
|
77
|
+
const profile: PoolProfile = assertPoolProfile({
|
|
78
|
+
...poolProfileFor(role),
|
|
79
|
+
...(options.profile ?? {}),
|
|
80
|
+
});
|
|
275
81
|
let driver: BunSqlDriver | undefined;
|
|
276
82
|
|
|
277
83
|
function connect(): BunSqlDriver {
|
|
@@ -455,27 +261,3 @@ export function baseClient(): DbClient {
|
|
|
455
261
|
export function db(): DbClient {
|
|
456
262
|
return currentTx() ?? baseClient();
|
|
457
263
|
}
|
|
458
|
-
|
|
459
|
-
/** Named `Db*` because `@ultimat3/core` already exports a `HealthReport` for the lifecycle. */
|
|
460
|
-
export interface DbHealthReport {
|
|
461
|
-
readonly ok: boolean;
|
|
462
|
-
readonly latencyMs: number;
|
|
463
|
-
readonly error?: string | undefined;
|
|
464
|
-
}
|
|
465
|
-
|
|
466
|
-
/** Backs `/readyz` for every role. Never throws — the probe wants a report, not an exception. */
|
|
467
|
-
export async function checkDb(client: DbClient = baseClient()): Promise<DbHealthReport> {
|
|
468
|
-
const started = performance.now();
|
|
469
|
-
try {
|
|
470
|
-
await client.query(sql`select 1`);
|
|
471
|
-
return { ok: true, latencyMs: Math.round(performance.now() - started) };
|
|
472
|
-
} catch (error) {
|
|
473
|
-
return {
|
|
474
|
-
ok: false,
|
|
475
|
-
latencyMs: Math.round(performance.now() - started),
|
|
476
|
-
// `renderThrowable`, never `error.message`: the probe wants a report, and a render that
|
|
477
|
-
// throws is an exception out of `/readyz` — the one caller that cannot catch it.
|
|
478
|
-
error: renderThrowable(error),
|
|
479
|
-
};
|
|
480
|
-
}
|
|
481
|
-
}
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
// Single responsibility: turning `DATABASE_URL` plus a resolved pool profile into the connection
|
|
2
|
+
// string the driver opens — the libpq `options` merge and the `application_name` label. Split from
|
|
3
|
+
// `client.ts` because which settings reach a connection is a rule, not a step of connecting.
|
|
4
|
+
|
|
5
|
+
import { describeValue } from '@ultimat3/core';
|
|
6
|
+
import { dbUnavailable } from './errors';
|
|
7
|
+
import { declaresLibpqOption, mergeLibpqOptions } from './libpq-options';
|
|
8
|
+
import type { PoolProfile } from './pool-profile';
|
|
9
|
+
|
|
10
|
+
export interface ConnectionUrlOptions {
|
|
11
|
+
readonly url?: string | undefined;
|
|
12
|
+
readonly applicationName?: string | undefined;
|
|
13
|
+
}
|
|
14
|
+
|
|
15
|
+
export function connectionUrl(options: ConnectionUrlOptions, profile: PoolProfile): string {
|
|
16
|
+
const raw = options.url ?? process.env['DATABASE_URL'];
|
|
17
|
+
if (raw === undefined || raw === '') {
|
|
18
|
+
throw dbUnavailable('DATABASE_URL is not set, so there is no database to connect to');
|
|
19
|
+
}
|
|
20
|
+
let url: URL;
|
|
21
|
+
try {
|
|
22
|
+
url = new URL(raw);
|
|
23
|
+
} catch (error) {
|
|
24
|
+
// The SHAPE of the rejected value, never the value. A connection string is
|
|
25
|
+
// `user:password@host` by construction, and this `cause` is the boot log line AND the `--json`
|
|
26
|
+
// payload — the logger redacts `fields` by key, so a password baked into a message has no key
|
|
27
|
+
// left to redact it by. `@ultimat3/core`'s `defineEnv` reached the identical conclusion about
|
|
28
|
+
// the identical variable (`packages/core/src/env.ts`); this is the same rule at the second
|
|
29
|
+
// reader, not a second rule. The value still has a printer: `x env check`, through
|
|
30
|
+
// `maskedEnvValues`.
|
|
31
|
+
throw dbUnavailable(`DATABASE_URL is not a valid url: received ${describeValue(raw)}`, error);
|
|
32
|
+
}
|
|
33
|
+
// libpq `options` is the portable way to pin a statement timeout for every pooled connection —
|
|
34
|
+
// MERGED into the operator's own, never assigned over it, and emitted for every role including
|
|
35
|
+
// the two whose bound is 0. `set` here dropped a `?options=-c search_path=app` on `web`, `sync`,
|
|
36
|
+
// `worker` and `scheduler` and kept it on `migrate` and `replicator`, so the role that runs the
|
|
37
|
+
// migrations and the role that serves the traffic read different schemas. 0 is a value, not a
|
|
38
|
+
// silence: it is `migrate` saying it may take as long as it takes, and left unsaid a server-side
|
|
39
|
+
// `alter database ... set statement_timeout` kills the one role that must outlive it.
|
|
40
|
+
// `application_name` is a LABEL, not a bound: 'ultimate' is a DEFAULT, and a default may not
|
|
41
|
+
// overwrite what the operator wrote — `?application_name=billing-api` is the filter their
|
|
42
|
+
// `pg_stat_activity` query, their pooler rule and their audit rule all match on, and losing it
|
|
43
|
+
// is silent. Both spellings count, or the URL parameter and a `-c application_name=` in
|
|
44
|
+
// `options` disagree and which one the backend honours is argument order nobody here measured.
|
|
45
|
+
const named = options.applicationName;
|
|
46
|
+
const settings: Record<string, string> = {
|
|
47
|
+
statement_timeout: String(profile.statementTimeoutMs),
|
|
48
|
+
};
|
|
49
|
+
const inOptions = declaresLibpqOption(url.searchParams.get('options'), 'application_name');
|
|
50
|
+
// An explicit `applicationName` is a deliberate call by the role that opened the pool, so it
|
|
51
|
+
// wins. Only then is the setting named to the merge, and only when the operator wrote the other
|
|
52
|
+
// spelling: `mergeLibpqOptions` drops their assignment before appending, so the two cannot
|
|
53
|
+
// disagree — and a URL with no assignment in it keeps the exact `options` it always had.
|
|
54
|
+
if (named !== undefined && inOptions) settings['application_name'] = named;
|
|
55
|
+
const declared = url.searchParams.has('application_name') || inOptions;
|
|
56
|
+
url.searchParams.set('options', mergeLibpqOptions(url.searchParams.get('options'), settings));
|
|
57
|
+
if (named !== undefined) url.searchParams.set('application_name', named);
|
|
58
|
+
else if (!declared) url.searchParams.set('application_name', 'ultimate');
|
|
59
|
+
return url.toString();
|
|
60
|
+
}
|
package/src/db-health.ts
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
// Single responsibility: the database's readiness answer — one `select 1` timed and reported, never
|
|
2
|
+
// thrown. Split from `client.ts` because a probe's report is a different job from opening a pool,
|
|
3
|
+
// and every role's `/readyz` reads this one and nothing else of the client.
|
|
4
|
+
|
|
5
|
+
import { renderThrowable } from '@ultimat3/core';
|
|
6
|
+
import { baseClient, type DbClient } from './client';
|
|
7
|
+
import { sql } from './sql';
|
|
8
|
+
|
|
9
|
+
/** Named `Db*` because `@ultimat3/core` already exports a `HealthReport` for the lifecycle. */
|
|
10
|
+
export interface DbHealthReport {
|
|
11
|
+
readonly ok: boolean;
|
|
12
|
+
readonly latencyMs: number;
|
|
13
|
+
readonly error?: string | undefined;
|
|
14
|
+
}
|
|
15
|
+
|
|
16
|
+
/** Backs `/readyz` for every role. Never throws — the probe wants a report, not an exception. */
|
|
17
|
+
export async function checkDb(client: DbClient = baseClient()): Promise<DbHealthReport> {
|
|
18
|
+
const started = performance.now();
|
|
19
|
+
try {
|
|
20
|
+
await client.query(sql`select 1`);
|
|
21
|
+
return { ok: true, latencyMs: Math.round(performance.now() - started) };
|
|
22
|
+
} catch (error) {
|
|
23
|
+
return {
|
|
24
|
+
ok: false,
|
|
25
|
+
latencyMs: Math.round(performance.now() - started),
|
|
26
|
+
// `renderThrowable`, never `error.message`: the probe wants a report, and a render that
|
|
27
|
+
// throws is an exception out of `/readyz` — the one caller that cannot catch it.
|
|
28
|
+
error: renderThrowable(error),
|
|
29
|
+
};
|
|
30
|
+
}
|
|
31
|
+
}
|
package/src/default-client.ts
CHANGED
|
@@ -3,7 +3,8 @@
|
|
|
3
3
|
// framework decides a process's database topology from the environment, and `client.ts` is at the
|
|
4
4
|
// line ceiling.
|
|
5
5
|
|
|
6
|
-
import { createPostgresClient, type DbClient
|
|
6
|
+
import { createPostgresClient, type DbClient } from './client';
|
|
7
|
+
import { poolMaxFromEnv } from './pool-profile';
|
|
7
8
|
import { replicatedClient } from './replica-client';
|
|
8
9
|
|
|
9
10
|
/**
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
// Single responsibility: the `X_DB_DRIFT` constructor — the error form of the finding
|
|
2
|
+
// `drift-findings.ts`'s `unexpectedColumn` reports, thrown where a caller has no report to hand
|
|
3
|
+
// back. Split out of `errors.ts` for the import it needs and nothing else: the fix line puts a
|
|
4
|
+
// column name into a shell command, so it screens through `shellInertIdentifier` (`sql.ts`) —
|
|
5
|
+
// and `sql.ts` imports `errors.ts`, so this cannot live there without a cycle around the module
|
|
6
|
+
// whose evaluation registers every code. The code is still declared, titled and registered in
|
|
7
|
+
// `errors.ts`, exactly as `migration-errors.ts` and `invariant-errors.ts` are.
|
|
8
|
+
|
|
9
|
+
import { DbError } from './errors';
|
|
10
|
+
import { shellInertIdentifier } from './sql';
|
|
11
|
+
|
|
12
|
+
/**
|
|
13
|
+
* The contract's pinned wording. Mirror of `@ultimat3/entity`'s `dbDrift()` — keep in sync; that
|
|
14
|
+
* one screens the column through the same `@ultimat3/db` export, so the two lines are the same
|
|
15
|
+
* text on both sides of the tier seam.
|
|
16
|
+
*
|
|
17
|
+
* The column name is the CATALOG's, so it is data: whoever can add a column picks the text that
|
|
18
|
+
* lands here, and `x db gen "add C"` puts it inside SHELL DOUBLE QUOTES, where `$(…)` and a
|
|
19
|
+
* backtick substitute before `x` is reached at all. The argument is a migration DESCRIPTION and
|
|
20
|
+
* not an identifier, so no quoted form makes a hostile name safe to pass — a name the screen
|
|
21
|
+
* refuses is left OUT of the command rather than escaped into it. The command still runs and
|
|
22
|
+
* still generates the migration; the name is read off `cause` and `meta`, which are prose nobody
|
|
23
|
+
* pastes.
|
|
24
|
+
*/
|
|
25
|
+
export const dbDrift = (tableName: string, columnName: string): DbError =>
|
|
26
|
+
new DbError({
|
|
27
|
+
code: 'X_DB_DRIFT',
|
|
28
|
+
cause: `table "${tableName}" has column "${columnName}" not present in any migration`,
|
|
29
|
+
fix:
|
|
30
|
+
shellInertIdentifier(columnName) === null
|
|
31
|
+
? 'x db gen "add the column named in this error" # its name carries a backtick, a ' +
|
|
32
|
+
'dollar sign, a quote, a backslash or whitespace, so it is in the cause and not in ' +
|
|
33
|
+
'this command'
|
|
34
|
+
: `x db gen "add ${columnName}"`,
|
|
35
|
+
meta: { table: tableName, column: columnName },
|
|
36
|
+
});
|
package/src/drift-findings.ts
CHANGED
|
@@ -14,6 +14,7 @@
|
|
|
14
14
|
import { onDeleteRule, rebuildForeignKey } from './foreign-key';
|
|
15
15
|
import type { CheckDescription, ForeignKeyDescription } from './introspect';
|
|
16
16
|
import type { Migration } from './migrate';
|
|
17
|
+
import { shellInertIdentifier } from './sql';
|
|
17
18
|
|
|
18
19
|
export type DriftKind =
|
|
19
20
|
| 'unexpected-column'
|
|
@@ -41,6 +42,15 @@ export interface DriftReport {
|
|
|
41
42
|
readonly differences: readonly DriftDifference[];
|
|
42
43
|
}
|
|
43
44
|
|
|
45
|
+
/**
|
|
46
|
+
* The one `fix:` here whose second layer no quoting closes. `x db gen "add C"` puts the column
|
|
47
|
+
* inside SHELL DOUBLE QUOTES, where `$(…)` and a backtick substitute before `x` is reached at all
|
|
48
|
+
* — and the argument is a migration DESCRIPTION, not an identifier, so there is no quoted form
|
|
49
|
+
* that would make a hostile name safe to pass. A name `shellInertIdentifier` (`sql.ts`) refuses
|
|
50
|
+
* is therefore left out of the command rather than escaped into it: the command still runs and
|
|
51
|
+
* still generates the migration, and the name is read off `cause` and `column`, which are prose
|
|
52
|
+
* nobody pastes.
|
|
53
|
+
*/
|
|
44
54
|
export function unexpectedColumn(table: string, column: string): DriftDifference {
|
|
45
55
|
return {
|
|
46
56
|
kind: 'unexpected-column',
|
|
@@ -48,7 +58,12 @@ export function unexpectedColumn(table: string, column: string): DriftDifference
|
|
|
48
58
|
column,
|
|
49
59
|
// Pinned by the contract. Do not reword without changing docs/errors/X_DB_DRIFT.
|
|
50
60
|
cause: `table "${table}" has column "${column}" not present in any migration`,
|
|
51
|
-
fix:
|
|
61
|
+
fix:
|
|
62
|
+
shellInertIdentifier(column) === null
|
|
63
|
+
? 'x db gen "add the undeclared column" # the live column name carries a backtick, a ' +
|
|
64
|
+
'dollar sign, a quote, a backslash or whitespace, so it is in the cause and not in ' +
|
|
65
|
+
'this command'
|
|
66
|
+
: `x db gen "add ${column}"`,
|
|
52
67
|
};
|
|
53
68
|
}
|
|
54
69
|
|
|
@@ -82,6 +97,8 @@ export function changedColumn(
|
|
|
82
97
|
liveNullable: boolean,
|
|
83
98
|
): DriftDifference {
|
|
84
99
|
const clause = liveNullable ? 'set not null' : 'drop not null';
|
|
100
|
+
const relation = shellInertIdentifier(table);
|
|
101
|
+
const attribute = shellInertIdentifier(column);
|
|
85
102
|
return {
|
|
86
103
|
kind: 'changed-column',
|
|
87
104
|
table,
|
|
@@ -89,19 +106,47 @@ export function changedColumn(
|
|
|
89
106
|
cause: liveNullable
|
|
90
107
|
? `table "${table}" allows NULL in column "${column}" that migrations declare not null`
|
|
91
108
|
: `table "${table}" forbids NULL in column "${column}" that migrations declare nullable`,
|
|
109
|
+
// Both identifiers are the catalog's, so both go through the one screen. A refusal names the
|
|
110
|
+
// column as the thing it could not spell, which is what tells this line apart from
|
|
111
|
+
// `missingCheck`'s refusal in a report that carries both.
|
|
92
112
|
fix:
|
|
93
|
-
|
|
94
|
-
|
|
113
|
+
relation === null || attribute === null
|
|
114
|
+
? `${clause} on the column named in this difference, in a new migration, then ` +
|
|
115
|
+
'x db migrate — its table or column name carries a backtick, a dollar sign, a quote, ' +
|
|
116
|
+
'a backslash or whitespace, so no statement here can spell it'
|
|
117
|
+
: `alter table ${relation} alter column ${attribute} ${clause}; # in a new migration` +
|
|
118
|
+
(liveNullable ? ' — backfill the existing NULLs first' : ''),
|
|
95
119
|
};
|
|
96
120
|
}
|
|
97
121
|
|
|
122
|
+
/**
|
|
123
|
+
* The `fix:` names the two edits that actually resolve this, and neither is `x db gen` (issue
|
|
124
|
+
* #345). That command diffs the ENTITY REGISTRY against the newest snapshot, and a table nothing
|
|
125
|
+
* declares is absent from both sides of that diff — so it wrote an EMPTY migration, and the
|
|
126
|
+
* generator's own empty-diff branch writes no file at all, leaving the reader with nothing to run
|
|
127
|
+
* and the same finding on the next deploy.
|
|
128
|
+
*
|
|
129
|
+
* What is left once `@ultimat3/cli`'s `acceptCreatedTables` has run is a table no migration's SQL
|
|
130
|
+
* creates and no entity declares — so either a migration should claim it (`if not exists`, because
|
|
131
|
+
* the relation is already there, and `x db migrate` then accepts a table its own SQL creates), or
|
|
132
|
+
* nothing owns it and it should not be in this schema. No migration PATH is named: where an app
|
|
133
|
+
* keeps its migrations is the CLI's fact, not this package's.
|
|
134
|
+
*/
|
|
98
135
|
export function unexpectedTable(table: string): DriftDifference {
|
|
136
|
+
const name = shellInertIdentifier(table);
|
|
99
137
|
return {
|
|
100
138
|
kind: 'unexpected-table',
|
|
101
139
|
table,
|
|
102
140
|
column: null,
|
|
103
141
|
cause: `table "${table}" is not present in any migration`,
|
|
104
|
-
fix:
|
|
142
|
+
fix:
|
|
143
|
+
name === null
|
|
144
|
+
? 'claim it in a migration with create table if not exists, or drop it by hand — its ' +
|
|
145
|
+
'table name carries a backtick, a dollar sign, a quote, a backslash or whitespace, so ' +
|
|
146
|
+
'no statement here can spell it'
|
|
147
|
+
: `put a create table if not exists ${name} (…) statement in a migration — x db migrate ` +
|
|
148
|
+
'then accepts a table its own SQL creates — or, if nothing owns it, run ' +
|
|
149
|
+
`drop table ${name}; inside psql "$DATABASE_URL"`,
|
|
105
150
|
};
|
|
106
151
|
}
|
|
107
152
|
|
|
@@ -174,6 +219,8 @@ export function changedIndex(table: string, index: string, detail: string): Drif
|
|
|
174
219
|
* the predicate, which is what makes an executable fix possible at all.
|
|
175
220
|
*/
|
|
176
221
|
export function missingCheck(table: string, check: CheckDescription): DriftDifference {
|
|
222
|
+
const relation = shellInertIdentifier(table);
|
|
223
|
+
const constraint = shellInertIdentifier(check.name);
|
|
177
224
|
return {
|
|
178
225
|
kind: 'missing-check',
|
|
179
226
|
table,
|
|
@@ -183,9 +230,19 @@ export function missingCheck(table: string, check: CheckDescription): DriftDiffe
|
|
|
183
230
|
// banned advice word the `errors` gate demands a command beside: writing the migration is half
|
|
184
231
|
// the repair and applying it is the other half, and `changedColumn`'s bare `# in a new
|
|
185
232
|
// migration` leaves the second half to be guessed.
|
|
233
|
+
//
|
|
234
|
+
// Both NAMES go through the one screen; the EXPRESSION deliberately does not, and cannot. It
|
|
235
|
+
// is a predicate, so no screen could accept `status in ('draft', 'published')` and reject a
|
|
236
|
+
// second statement — and it is the DECLARED side's own text, out of the author's migration,
|
|
237
|
+
// where both names are the catalog's and a sidecar's. Narrower than "this line is safe", and
|
|
238
|
+
// it is the honest claim.
|
|
186
239
|
fix:
|
|
187
|
-
|
|
188
|
-
|
|
240
|
+
relation === null || constraint === null
|
|
241
|
+
? 'add the constraint named in this difference back in a new migration, then ' +
|
|
242
|
+
'x db migrate — its table or constraint name carries a backtick, a dollar sign, a ' +
|
|
243
|
+
'quote, a backslash or whitespace, so no statement here can spell it'
|
|
244
|
+
: `alter table ${relation} add constraint ${constraint} ` +
|
|
245
|
+
`check (${check.expression}); # in a new migration, then x db migrate`,
|
|
189
246
|
};
|
|
190
247
|
}
|
|
191
248
|
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
// TEST-ONLY. The two builders every drift suite compares with — a `TableDescription` of text
|
|
2
|
+
// columns and the `SchemaDescription` around it. One copy, because three suites arguing about
|
|
3
|
+
// schemas built differently would each be judging a different fixture. Never exported from
|
|
4
|
+
// `index.ts`.
|
|
5
|
+
|
|
6
|
+
import type { SchemaDescription, TableDescription } from './introspect';
|
|
7
|
+
|
|
8
|
+
export const table = (name: string, columns: readonly string[]): TableDescription => ({
|
|
9
|
+
schema: 'public',
|
|
10
|
+
name,
|
|
11
|
+
columns: columns.map((column, index) => ({
|
|
12
|
+
name: column,
|
|
13
|
+
dataType: 'text',
|
|
14
|
+
nullable: true,
|
|
15
|
+
default: null,
|
|
16
|
+
position: index + 1,
|
|
17
|
+
})),
|
|
18
|
+
primaryKey: ['id'],
|
|
19
|
+
indexes: [],
|
|
20
|
+
foreignKeys: [],
|
|
21
|
+
});
|
|
22
|
+
|
|
23
|
+
export const schema = (...tables: readonly TableDescription[]): SchemaDescription => ({ tables });
|
package/src/errors.ts
CHANGED
|
@@ -285,15 +285,6 @@ export const serializationExhausted = (attempts: number, sourceError: unknown):
|
|
|
285
285
|
sourceError,
|
|
286
286
|
});
|
|
287
287
|
|
|
288
|
-
/** The contract's pinned wording. Mirror of `@ultimat3/entity`'s `dbDrift()` — keep in sync. */
|
|
289
|
-
export const dbDrift = (tableName: string, columnName: string): DbError =>
|
|
290
|
-
new DbError({
|
|
291
|
-
code: 'X_DB_DRIFT',
|
|
292
|
-
cause: `table "${tableName}" has column "${columnName}" not present in any migration`,
|
|
293
|
-
fix: `x db gen "add ${columnName}"`,
|
|
294
|
-
meta: { table: tableName, column: columnName },
|
|
295
|
-
});
|
|
296
|
-
|
|
297
288
|
export const sqlUnsafe = (received: string, position: number): DbError =>
|
|
298
289
|
new DbError({
|
|
299
290
|
code: 'X_SQL_UNSAFE',
|
package/src/index.ts
CHANGED
|
@@ -23,25 +23,15 @@ export {
|
|
|
23
23
|
export type {
|
|
24
24
|
DbClient,
|
|
25
25
|
DbConnection,
|
|
26
|
-
DbHealthReport,
|
|
27
|
-
PoolProfile,
|
|
28
26
|
PostgresClient,
|
|
29
27
|
PostgresClientOptions,
|
|
30
28
|
ReservableClient,
|
|
31
29
|
} from './client';
|
|
32
|
-
export {
|
|
33
|
-
baseClient,
|
|
34
|
-
checkDb,
|
|
35
|
-
createPostgresClient,
|
|
36
|
-
db,
|
|
37
|
-
isReservable,
|
|
38
|
-
POOL_MAX_ENV,
|
|
39
|
-
POOL_PROFILES,
|
|
40
|
-
poolProfileFor,
|
|
41
|
-
setDbClient,
|
|
42
|
-
} from './client';
|
|
30
|
+
export { baseClient, createPostgresClient, db, isReservable, setDbClient } from './client';
|
|
43
31
|
export type { ColumnDefaultLike } from './column-default';
|
|
44
32
|
export { defaultExpression } from './column-default';
|
|
33
|
+
export type { DbHealthReport } from './db-health';
|
|
34
|
+
export { checkDb } from './db-health';
|
|
45
35
|
export { defaultClient, REPLICA_URL_ENV } from './default-client';
|
|
46
36
|
export type { DestructiveKind, DestructiveStatement } from './destructive';
|
|
47
37
|
export {
|
|
@@ -62,6 +52,7 @@ export {
|
|
|
62
52
|
expectedSchema,
|
|
63
53
|
FRAMEWORK_TABLE_PREFIX,
|
|
64
54
|
} from './drift';
|
|
55
|
+
export { dbDrift } from './drift-errors';
|
|
65
56
|
export type {
|
|
66
57
|
ColumnDescriptionLike,
|
|
67
58
|
EntityDescriptionLike,
|
|
@@ -76,7 +67,6 @@ export {
|
|
|
76
67
|
DB_ERROR_RETRY,
|
|
77
68
|
DB_ERROR_TITLES,
|
|
78
69
|
DbError,
|
|
79
|
-
dbDrift,
|
|
80
70
|
dbNotImplemented,
|
|
81
71
|
dbUnavailable,
|
|
82
72
|
driverError,
|
|
@@ -169,6 +159,8 @@ export {
|
|
|
169
159
|
} from './pglite';
|
|
170
160
|
export type { PgliteBranchInfo, PgliteBranchOptions } from './pglite-branch';
|
|
171
161
|
export { branchPglite, pgliteBranchDir } from './pglite-branch';
|
|
162
|
+
export type { PoolProfile } from './pool-profile';
|
|
163
|
+
export { POOL_MAX_ENV, POOL_PROFILES, poolProfileFor } from './pool-profile';
|
|
172
164
|
export type { ReadOnlyQueryOptions, ReadOnlyQueryResult } from './readonly-query';
|
|
173
165
|
export { READONLY_TIMEOUT_MS, readOnlyQuery } from './readonly-query';
|
|
174
166
|
export type { ReadOnlyRoleOptions } from './readonly-role';
|
|
@@ -185,7 +177,15 @@ export { markScopeWrote, replicaScope, withReplicaReads } from './replica-scope'
|
|
|
185
177
|
export { snapshotJson } from './snapshot-json';
|
|
186
178
|
export { parseSnapshot } from './snapshot-parse';
|
|
187
179
|
export type { SqlFragment } from './sql';
|
|
188
|
-
export {
|
|
180
|
+
export {
|
|
181
|
+
identifier,
|
|
182
|
+
isSqlFragment,
|
|
183
|
+
join,
|
|
184
|
+
literal,
|
|
185
|
+
raw,
|
|
186
|
+
shellInertIdentifier,
|
|
187
|
+
sql,
|
|
188
|
+
} from './sql';
|
|
189
189
|
export { stripSqlNoise } from './sql-noise';
|
|
190
190
|
export type { DbSqlStateCode } from './sqlstate';
|
|
191
191
|
export { DB_SQLSTATE_CODES, isRetryableState, SQLSTATE, sqlState, sqlStateCode } from './sqlstate';
|
package/src/migrate.ts
CHANGED
|
@@ -3,18 +3,13 @@
|
|
|
3
3
|
// app-version fence is the `migrate` role's contract — a pod must refuse to migrate a database
|
|
4
4
|
// another build already owns, because the alternative is two schemas racing during a rollout.
|
|
5
5
|
|
|
6
|
-
import { appVersion } from '@ultimat3/core';
|
|
7
|
-
import {
|
|
8
|
-
baseClient,
|
|
9
|
-
type DbClient,
|
|
10
|
-
type DbConnection,
|
|
11
|
-
isReservable,
|
|
12
|
-
poolProfileFor,
|
|
13
|
-
} from './client';
|
|
6
|
+
import { appVersion, finiteCount } from '@ultimat3/core';
|
|
7
|
+
import { baseClient, type DbClient, type DbConnection, isReservable } from './client';
|
|
14
8
|
import { refuseDependentViews } from './dependent-view';
|
|
15
9
|
import { expectedQueryLoop } from './expected-loop';
|
|
16
10
|
import type { SchemaDescription } from './introspect';
|
|
17
11
|
import { migrateConcurrent, migrationConflict, rollbackStepsInvalid } from './migration-errors';
|
|
12
|
+
import { poolProfileFor } from './pool-profile';
|
|
18
13
|
import { raw, sql } from './sql';
|
|
19
14
|
import { SQLSTATE, sqlState } from './sqlstate';
|
|
20
15
|
import { statementsOf } from './statement-split';
|
|
@@ -237,6 +232,9 @@ async function acquireLock(session: DbClient, waitMs: number): Promise<void> {
|
|
|
237
232
|
sql`select pg_try_advisory_lock(${MIGRATION_LOCK_KEY}) as locked`,
|
|
238
233
|
);
|
|
239
234
|
if (row?.locked === true) return;
|
|
235
|
+
// `waitMs` is screened at both call sites: `NaN - elapsed` is `NaN`, `NaN <= 0` is false
|
|
236
|
+
// and `Bun.sleep(Math.min(POLL, NaN))` does not sleep — so an unbounded wait is not what a
|
|
237
|
+
// non-finite one produced, a tight spin re-taking `pg_try_advisory_lock` was.
|
|
240
238
|
const remaining = waitMs - (performance.now() - started);
|
|
241
239
|
if (remaining <= 0) {
|
|
242
240
|
throw migrateConcurrent(MIGRATION_LOCK_KEY, Math.round(performance.now() - started));
|
|
@@ -317,7 +315,17 @@ async function setLockTimeout(tx: DbTx, lockTimeoutMs: number): Promise<void> {
|
|
|
317
315
|
* same `ACCESS EXCLUSIVE` locks against the same tables.
|
|
318
316
|
*/
|
|
319
317
|
function migrationLockTimeoutMs(explicit: number | undefined): number {
|
|
320
|
-
|
|
318
|
+
// Screened here, where BOTH `migrate()` and `rollback()` resolve it, and before either takes the
|
|
319
|
+
// advisory lock. `lockWaitMs` beside it was screened from the start and this one was not, so
|
|
320
|
+
// `Number(process.env.…)` on an unset variable travelled all the way to `SET LOCAL lock_timeout
|
|
321
|
+
// = NaN`, which the server rejects inside the migration's own transaction — the option that was
|
|
322
|
+
// wrong appears nowhere in what the deploy prints. `finiteCount`, never a second bound checker.
|
|
323
|
+
return finiteCount(
|
|
324
|
+
'migrate',
|
|
325
|
+
'lockTimeoutMs',
|
|
326
|
+
explicit ?? poolProfileFor('migrate').lockTimeoutMs,
|
|
327
|
+
0,
|
|
328
|
+
);
|
|
321
329
|
}
|
|
322
330
|
|
|
323
331
|
export async function migrate(options: MigrateOptions): Promise<MigrationReport> {
|
|
@@ -329,7 +337,7 @@ export async function migrate(options: MigrateOptions): Promise<MigrationReport>
|
|
|
329
337
|
return withAdvisoryLock(
|
|
330
338
|
client,
|
|
331
339
|
options.lock !== false,
|
|
332
|
-
options.lockWaitMs ?? MIGRATION_LOCK_WAIT_MS,
|
|
340
|
+
finiteCount('migrate', 'lockWaitMs', options.lockWaitMs ?? MIGRATION_LOCK_WAIT_MS, 0),
|
|
333
341
|
async (session) => {
|
|
334
342
|
await ensureLedger(session);
|
|
335
343
|
const ledger = await readLedger(session);
|
|
@@ -419,7 +427,7 @@ export async function rollback(options: RollbackOptions): Promise<readonly strin
|
|
|
419
427
|
return withAdvisoryLock(
|
|
420
428
|
client,
|
|
421
429
|
options.lock !== false,
|
|
422
|
-
options.lockWaitMs ?? MIGRATION_LOCK_WAIT_MS,
|
|
430
|
+
finiteCount('migrate', 'lockWaitMs', options.lockWaitMs ?? MIGRATION_LOCK_WAIT_MS, 0),
|
|
423
431
|
async (session) => {
|
|
424
432
|
const ledger = await readLedger(session);
|
|
425
433
|
const targets = [...ledger].reverse().slice(0, steps);
|
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
// Single responsibility: the five numbers a Postgres pool runs on — the per-role defaults, the one
|
|
2
|
+
// environment override an operator may layer over them, and the screen every resolved profile
|
|
3
|
+
// passes. Split from `client.ts`, which now owns connecting and nothing about sizing.
|
|
4
|
+
|
|
5
|
+
import { assert, type Role, resolveRole } from '@ultimat3/core';
|
|
6
|
+
import { poolMaxInvalid } from './errors';
|
|
7
|
+
|
|
8
|
+
export interface PoolProfile {
|
|
9
|
+
readonly max: number;
|
|
10
|
+
/** 0 disables the timeout — only `migrate`, which is allowed to take as long as it takes. */
|
|
11
|
+
readonly statementTimeoutMs: number;
|
|
12
|
+
readonly idleTimeoutMs: number;
|
|
13
|
+
/**
|
|
14
|
+
* How long a statement may **wait for a lock** before `55P03`, distinct from how long it may run.
|
|
15
|
+
* 0 everywhere but `migrate`, which is the only role that takes `ACCESS EXCLUSIVE`: an `alter
|
|
16
|
+
* table` queued behind a long `SELECT` puts every later query on that table behind it too,
|
|
17
|
+
* because Postgres' lock queue is FIFO — and `migrate` runs `statement_timeout = 0`, so nothing
|
|
18
|
+
* else would ever end the wait. Read by `migrate()` as a `SET LOCAL`, never by the pool.
|
|
19
|
+
*/
|
|
20
|
+
readonly lockTimeoutMs: number;
|
|
21
|
+
/**
|
|
22
|
+
* How long `reserve()` may wait for a free connection before `X_DB_POOL_EXHAUSTED`. 0 waits
|
|
23
|
+
* forever, which is what a run-once role wants and what a request-serving one must never do:
|
|
24
|
+
* queueing turns exhaustion into a hang, `/readyz`'s `select 1` joins the same queue, the kubelet
|
|
25
|
+
* kills the pod, and the replacement inherits the same saturated database.
|
|
26
|
+
*/
|
|
27
|
+
readonly acquireTimeoutMs: number;
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
/** Sized per role because the failure modes differ: RPS bursts vs. queue depth vs. run-once. */
|
|
31
|
+
export const POOL_PROFILES = Object.freeze<Record<Role, PoolProfile>>({
|
|
32
|
+
web: {
|
|
33
|
+
max: 20,
|
|
34
|
+
statementTimeoutMs: 10_000,
|
|
35
|
+
idleTimeoutMs: 30_000,
|
|
36
|
+
lockTimeoutMs: 0,
|
|
37
|
+
acquireTimeoutMs: 5_000,
|
|
38
|
+
},
|
|
39
|
+
sync: {
|
|
40
|
+
max: 10,
|
|
41
|
+
statementTimeoutMs: 10_000,
|
|
42
|
+
idleTimeoutMs: 60_000,
|
|
43
|
+
lockTimeoutMs: 0,
|
|
44
|
+
acquireTimeoutMs: 5_000,
|
|
45
|
+
},
|
|
46
|
+
worker: {
|
|
47
|
+
max: 8,
|
|
48
|
+
statementTimeoutMs: 120_000,
|
|
49
|
+
idleTimeoutMs: 30_000,
|
|
50
|
+
lockTimeoutMs: 0,
|
|
51
|
+
acquireTimeoutMs: 10_000,
|
|
52
|
+
},
|
|
53
|
+
scheduler: {
|
|
54
|
+
max: 2,
|
|
55
|
+
statementTimeoutMs: 15_000,
|
|
56
|
+
idleTimeoutMs: 60_000,
|
|
57
|
+
lockTimeoutMs: 0,
|
|
58
|
+
acquireTimeoutMs: 10_000,
|
|
59
|
+
},
|
|
60
|
+
// `migrate` waits: its pool is `max: 1` and the advisory-lock pin holds it for the whole run, so
|
|
61
|
+
// a deadline here would refuse the migration's own session. The wait that needed bounding is the
|
|
62
|
+
// advisory lock's, and `MIGRATION_LOCK_WAIT_MS` bounds it.
|
|
63
|
+
migrate: {
|
|
64
|
+
max: 1,
|
|
65
|
+
statementTimeoutMs: 0,
|
|
66
|
+
idleTimeoutMs: 10_000,
|
|
67
|
+
lockTimeoutMs: 3_000,
|
|
68
|
+
acquireTimeoutMs: 0,
|
|
69
|
+
},
|
|
70
|
+
replicator: {
|
|
71
|
+
max: 4,
|
|
72
|
+
statementTimeoutMs: 0,
|
|
73
|
+
idleTimeoutMs: 60_000,
|
|
74
|
+
lockTimeoutMs: 0,
|
|
75
|
+
acquireTimeoutMs: 0,
|
|
76
|
+
},
|
|
77
|
+
});
|
|
78
|
+
|
|
79
|
+
export function poolProfileFor(role: Role = resolveRole()): PoolProfile {
|
|
80
|
+
return POOL_PROFILES[role];
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
/** The one pool knob an operator can turn without a rebuild. Layered over the role default. */
|
|
84
|
+
export const POOL_MAX_ENV = 'DATABASE_POOL_MAX';
|
|
85
|
+
|
|
86
|
+
/**
|
|
87
|
+
* `DATABASE_POOL_MAX`, or nothing. `POOL_PROFILES` is frozen into the build, so before this the
|
|
88
|
+
* only way to change a fleet's connection count was to ship a new image — and 400 `web` pods at
|
|
89
|
+
* `max: 20` is 8,000 backends against a `max_connections` of 450. An unparseable value **refuses**
|
|
90
|
+
* rather than falling back: a fleet that ignored the number it was given is the failure the
|
|
91
|
+
* variable exists to prevent, and it would only be found in `pg_stat_activity` at 3am.
|
|
92
|
+
*/
|
|
93
|
+
export function poolMaxFromEnv(): Partial<PoolProfile> {
|
|
94
|
+
const raw = process.env[POOL_MAX_ENV];
|
|
95
|
+
if (raw === undefined || raw.trim() === '') return {};
|
|
96
|
+
const max = Number(raw);
|
|
97
|
+
if (!Number.isSafeInteger(max) || max < 1) throw poolMaxInvalid(raw);
|
|
98
|
+
return { max };
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
/**
|
|
102
|
+
* The five numbers a pool runs on, screened on the MERGED profile — an override is spread over a
|
|
103
|
+
* role default the caller never restated, so the resolved object is the only one that can be
|
|
104
|
+
* judged. Every one of them is a plausible `Number(process.env.…)`, which is `NaN` for an unset
|
|
105
|
+
* variable and not nullish, so `??` and the spread both keep it. None of the five then fails
|
|
106
|
+
* loudly: `idleTimeout: NaN` goes to `Bun.SQL`, `statement_timeout=NaN` goes into the libpq
|
|
107
|
+
* options string for the SERVER to reject on connect, and a timer given `NaN` fires at 1ms in this
|
|
108
|
+
* Bun — so a pool with free connections reports itself exhausted. `0` stays legal for the three
|
|
109
|
+
* budgets that document it as "no bound"; `max` is at least one connection, or nothing can run.
|
|
110
|
+
*/
|
|
111
|
+
export function assertPoolProfile(profile: PoolProfile): PoolProfile {
|
|
112
|
+
const whole = (option: string, value: number, min: 0 | 1): void => {
|
|
113
|
+
assert(
|
|
114
|
+
Number.isSafeInteger(value) && value >= min,
|
|
115
|
+
`pool profile ${option} is ${String(value)}; it must be a whole number of ${min === 1 ? 'at least 1' : '0 or more, where 0 is the documented "no bound"'}`,
|
|
116
|
+
`pass a whole number for ${option} in createPostgresClient({ profile }), and parse an environment value first — Number(process.env.DATABASE_${option.toUpperCase()} ?? '') is NaN when the variable is unset`,
|
|
117
|
+
);
|
|
118
|
+
};
|
|
119
|
+
whole('max', profile.max, 1);
|
|
120
|
+
whole('statementTimeoutMs', profile.statementTimeoutMs, 0);
|
|
121
|
+
whole('idleTimeoutMs', profile.idleTimeoutMs, 0);
|
|
122
|
+
whole('lockTimeoutMs', profile.lockTimeoutMs, 0);
|
|
123
|
+
whole('acquireTimeoutMs', profile.acquireTimeoutMs, 0);
|
|
124
|
+
return profile;
|
|
125
|
+
}
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
// Single responsibility: pinning a connection out of the pool under the profile's acquire deadline,
|
|
2
|
+
// and giving back a reservation that arrives after the deadline has passed. Split from `client.ts`,
|
|
3
|
+
// which now asks for a pin rather than owning what "waited too long" means.
|
|
4
|
+
|
|
5
|
+
import type { BunSqlDriver, BunSqlReserved } from './bun-sql';
|
|
6
|
+
import { poolAcquireTimeout } from './errors';
|
|
7
|
+
import type { PoolProfile } from './pool-profile';
|
|
8
|
+
|
|
9
|
+
/**
|
|
10
|
+
* `pool.reserve()` under a deadline. Without one an exhausted pool does not fail, it **queues** —
|
|
11
|
+
* so a slow endpoint filling all 20 slots turns every later request, `/readyz`'s `select 1`
|
|
12
|
+
* included, into a wait with no end and no error, and the pod is killed for being unready rather
|
|
13
|
+
* than answering 503 for the requests it cannot serve.
|
|
14
|
+
*
|
|
15
|
+
* The losing reservation is released, never dropped: the pool hands out a connection whenever one
|
|
16
|
+
* frees, deadline or no deadline, and a pin nobody holds is a connection nobody gets back. That is
|
|
17
|
+
* the whole reason this is not a bare `Promise.race`.
|
|
18
|
+
*/
|
|
19
|
+
export async function reserveWithin(
|
|
20
|
+
pool: Pick<BunSqlDriver, 'reserve'>,
|
|
21
|
+
profile: PoolProfile,
|
|
22
|
+
): Promise<BunSqlReserved> {
|
|
23
|
+
const budget = profile.acquireTimeoutMs;
|
|
24
|
+
if (budget <= 0) return pool.reserve();
|
|
25
|
+
let timer: ReturnType<typeof setTimeout> | undefined;
|
|
26
|
+
let expired = false;
|
|
27
|
+
const pending = pool.reserve();
|
|
28
|
+
try {
|
|
29
|
+
return await Promise.race([
|
|
30
|
+
pending,
|
|
31
|
+
new Promise<never>((_resolve, reject) => {
|
|
32
|
+
timer = setTimeout(() => {
|
|
33
|
+
expired = true;
|
|
34
|
+
reject(poolAcquireTimeout(budget, profile.max));
|
|
35
|
+
}, budget);
|
|
36
|
+
// The deadline must not be what keeps a finished process alive.
|
|
37
|
+
timer.unref?.();
|
|
38
|
+
}),
|
|
39
|
+
]);
|
|
40
|
+
} finally {
|
|
41
|
+
if (timer !== undefined) clearTimeout(timer);
|
|
42
|
+
// Attached unconditionally so a rejection arriving after we gave up is handled, not unhandled.
|
|
43
|
+
void pending.then(
|
|
44
|
+
(late) => {
|
|
45
|
+
if (expired) late.release();
|
|
46
|
+
},
|
|
47
|
+
() => undefined,
|
|
48
|
+
);
|
|
49
|
+
}
|
|
50
|
+
}
|
package/src/readonly-query.ts
CHANGED
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
// enforcing read-only beats a regex, and the timeout bounds a runaway scan the agent never
|
|
4
4
|
// meant to ask for.
|
|
5
5
|
|
|
6
|
+
import { finiteCount } from '@ultimat3/core';
|
|
6
7
|
import { baseClient, type DbClient, type DbConnection, isReservable } from './client';
|
|
7
8
|
import { multipleStatements } from './errors';
|
|
8
9
|
import { identifier, raw, sql } from './sql';
|
|
@@ -89,6 +90,26 @@ export async function readOnlyQuery<T>(
|
|
|
89
90
|
const statements = statementsOf(statement);
|
|
90
91
|
if (statements.length > 1) throw multipleStatements(statement, statements.length);
|
|
91
92
|
|
|
93
|
+
// Decided before anything is opened, for the reason `rollback({ steps })` screens before it
|
|
94
|
+
// takes the advisory lock: a value this build cannot honour is not a fact about the pool. It
|
|
95
|
+
// was computed after `reserve()` and after `BEGIN READ ONLY`, so an unbounded `timeoutMs`
|
|
96
|
+
// against an exhausted pool answered with the pool's error — or waited for a connection it was
|
|
97
|
+
// never going to use — in place of the `X_INVARIANT` naming the option that was wrong.
|
|
98
|
+
//
|
|
99
|
+
// Clamped and truncated to an integer: the result is a JS number the caller never touches
|
|
100
|
+
// as text, so there is nothing here for `raw()` to inject — `SET LOCAL` can't bind `$n`
|
|
101
|
+
// parameters, which is why this can't go through `sql` the normal, parameterised way.
|
|
102
|
+
// REFUSED rather than normalised: `NaN` used to take the default silently, so a config typo
|
|
103
|
+
// ran under a timeout nobody wrote. Only an explicit 0 disables the layer, which is why the
|
|
104
|
+
// floor is 0 and not 1; the hour ceiling is a clamp on a number that IS one.
|
|
105
|
+
const asked = finiteCount(
|
|
106
|
+
'readonlyQuery',
|
|
107
|
+
'timeoutMs',
|
|
108
|
+
options.timeoutMs ?? READONLY_TIMEOUT_MS,
|
|
109
|
+
0,
|
|
110
|
+
);
|
|
111
|
+
const ms = Math.min(3_600_000, asked);
|
|
112
|
+
|
|
92
113
|
const client = options.client ?? baseClient();
|
|
93
114
|
// A pooled BEGIN that lands on a different physical connection than the query that follows is
|
|
94
115
|
// not a transaction at all, so a reservable client must pin one connection for the sequence.
|
|
@@ -104,14 +125,6 @@ export async function readOnlyQuery<T>(
|
|
|
104
125
|
await connection.execute(raw('BEGIN READ ONLY'));
|
|
105
126
|
guards.push('txn:read-only');
|
|
106
127
|
|
|
107
|
-
// Clamped and truncated to an integer: the result is a JS number the caller never touches
|
|
108
|
-
// as text, so there is nothing here for `raw()` to inject — `SET LOCAL` can't bind `$n`
|
|
109
|
-
// parameters, which is why this can't go through `sql` the normal, parameterised way.
|
|
110
|
-
// NaN normalises to the default first: only an explicit 0 disables the timeout, and
|
|
111
|
-
// `Math.min(3_600_000, NaN)` is NaN, which would fail `ms > 0` and silently skip the layer.
|
|
112
|
-
const asked = options.timeoutMs ?? READONLY_TIMEOUT_MS;
|
|
113
|
-
const requested = Number.isNaN(asked) ? READONLY_TIMEOUT_MS : asked;
|
|
114
|
-
const ms = Math.max(0, Math.min(3_600_000, Math.trunc(requested)));
|
|
115
128
|
if (ms > 0) {
|
|
116
129
|
// `LOCAL`, so the setting dies with the transaction — one agent read must not re-time
|
|
117
130
|
// every request the pool serves afterwards. Caveat: embedded PGlite applies the setting
|
package/src/replica-client.ts
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
// handle a statement is sent on, what happens when the replica will not answer, and the counters
|
|
4
4
|
// that make both visible to a test that cannot scrape a metrics endpoint.
|
|
5
5
|
|
|
6
|
-
import { type Clock, logger, renderThrowable, systemClock } from '@ultimat3/core';
|
|
6
|
+
import { type Clock, finiteCount, logger, renderThrowable, systemClock } from '@ultimat3/core';
|
|
7
7
|
import { type DbClient, type DbConnection, isReservable, type ReservableClient } from './client';
|
|
8
8
|
import { isPlainRead } from './replica-route';
|
|
9
9
|
import { markScopeWrote, replicaScope } from './replica-scope';
|
|
@@ -53,8 +53,21 @@ export function replicatedClient(
|
|
|
53
53
|
options: ReplicatedClientOptions = {},
|
|
54
54
|
): ReplicatedClient {
|
|
55
55
|
const clock = options.clock ?? systemClock;
|
|
56
|
-
|
|
57
|
-
|
|
56
|
+
// Both are comparisons and nothing else — `consecutiveFailures >= limit` opens the breaker,
|
|
57
|
+
// `monotonic() < parkedUntil` holds it open — so a `NaN` in either is a breaker that never trips
|
|
58
|
+
// and never parks, with every read still going to the replica that is failing.
|
|
59
|
+
const limit = finiteCount(
|
|
60
|
+
'replicatedClient',
|
|
61
|
+
'breakerFailures',
|
|
62
|
+
options.breakerFailures ?? BREAKER_FAILURES,
|
|
63
|
+
1,
|
|
64
|
+
);
|
|
65
|
+
const cooldown = finiteCount(
|
|
66
|
+
'replicatedClient',
|
|
67
|
+
'breakerCooldownMs',
|
|
68
|
+
options.breakerCooldownMs ?? BREAKER_COOLDOWN_MS,
|
|
69
|
+
1,
|
|
70
|
+
);
|
|
58
71
|
let replicaCount = 0;
|
|
59
72
|
let primaryCount = 0;
|
|
60
73
|
let fallbackCount = 0;
|
package/src/sql.ts
CHANGED
|
@@ -131,6 +131,42 @@ export function identifier(name: string): SqlFragment {
|
|
|
131
131
|
return raw(`"${name}"`);
|
|
132
132
|
}
|
|
133
133
|
|
|
134
|
+
/**
|
|
135
|
+
* The two characters a shell substitutes INSIDE double quotes. Rejected before `identifier` is
|
|
136
|
+
* consulted at all, because `identifier` accepts both.
|
|
137
|
+
*/
|
|
138
|
+
const SHELL_ACTIVE = /[`$]/;
|
|
139
|
+
|
|
140
|
+
/**
|
|
141
|
+
* A quoted identifier that is ALSO inert wherever a human pastes it — or `null` for a name no line
|
|
142
|
+
* built here may spell. The one screen a catalog name goes through before it reaches a `fix:`.
|
|
143
|
+
*
|
|
144
|
+
* TWO layers, and `identifier` closes only the first. It answers about SQL: it refuses `"`, `\`
|
|
145
|
+
* and whitespace, and ACCEPTS a backtick and a `$` — `SAFE_IDENTIFIER` above allows `$` on its
|
|
146
|
+
* fast path — which are exactly the two characters `$(…)` and a command substitution are built
|
|
147
|
+
* from. A column called `$(id)` inside `x db gen "add $(id)"` therefore RUNS `id` the moment its
|
|
148
|
+
* reader pastes the line, and a screen reusing `identifier` unchanged would ship a green suite
|
|
149
|
+
* over that.
|
|
150
|
+
*
|
|
151
|
+
* The name is DATA at every caller: `create table "x""; drop table users; --" ("id" int)` is legal
|
|
152
|
+
* DDL, so whoever can create a table or a column picks the text that lands in a `fix:`. A refusal
|
|
153
|
+
* is degraded to prose by its caller and never escaped — the argument to `x db gen` is a migration
|
|
154
|
+
* DESCRIPTION, not an identifier, so there is no quoted form that makes a hostile name safe to
|
|
155
|
+
* pass, and a fix naming no command beats one running a second command the reader never read.
|
|
156
|
+
*
|
|
157
|
+
* `'` is deliberately NOT refused: it is legal in an identifier and inert both in a psql session
|
|
158
|
+
* and inside shell double quotes. The price of the two that ARE refused is a legal `a$b` losing
|
|
159
|
+
* its executable fix line, which is prose in place of something nobody read running.
|
|
160
|
+
*/
|
|
161
|
+
export function shellInertIdentifier(name: string): string | null {
|
|
162
|
+
if (SHELL_ACTIVE.test(name)) return null;
|
|
163
|
+
try {
|
|
164
|
+
return identifier(name).text;
|
|
165
|
+
} catch {
|
|
166
|
+
return null;
|
|
167
|
+
}
|
|
168
|
+
}
|
|
169
|
+
|
|
134
170
|
/**
|
|
135
171
|
* A quoted string literal Postgres reads IDENTICALLY under both settings of
|
|
136
172
|
* `standard_conforming_strings`. Utility and DDL statements (`CREATE DATABASE`, `COMMENT ON`,
|