@ultimat3/db 17.0.0 → 19.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +131 -15
- package/README.md +1 -1
- package/package.json +3 -3
- package/src/array-parameter.ts +91 -0
- package/src/bun-sql.ts +46 -4
- package/src/client.ts +43 -95
- package/src/connection-url.ts +43 -1
- package/src/errors.ts +20 -0
- package/src/generate.ts +40 -2
- package/src/introspect.ts +16 -0
- package/src/pool-profile.ts +29 -4
- package/src/pool-reserve.ts +2 -2
- package/src/replica-identity.ts +84 -0
- package/src/snapshot-parse.ts +11 -2
- package/src/statement-funnel.ts +101 -0
- package/src/ungeneratable.ts +11 -0
package/CLAUDE.md
CHANGED
|
@@ -233,6 +233,30 @@ means the nested scope never opened, and a release that failed means its work is
|
|
|
233
233
|
the outer one. Swallowing either would keep running against a transaction that is not the one the
|
|
234
234
|
caller thinks it is in.
|
|
235
235
|
|
|
236
|
+
**`close()` is BOUNDED, `As of 2026-08-27`, and by the driver's OWN option rather than a race here**
|
|
237
|
+
(#394). `BunSqlDriver.close` has declared `{ timeout }` since this package's `Bun.SQL` slice was
|
|
238
|
+
written and **nothing ever passed it** — a capability sitting unused in the seam, the same shape as
|
|
239
|
+
`setOfflineMode` on the CDP port. Measured against a real server, three runs per case: `end()` waits
|
|
240
|
+
on an outstanding RESERVED connection and never returns, on Bun 1.3.14 **and** on 1.4.0, with the
|
|
241
|
+
database perfectly healthy; once that connection's backend has been terminated it becomes a race
|
|
242
|
+
1.3.14 loses 3 of 3 and 1.4.0 loses 1 of 3. So the runtime was never the variable — an unbounded
|
|
243
|
+
await was, and `@ultimat3/cli`'s `releaseQueue` awaits this method. A container that will not drain
|
|
244
|
+
is drained by SIGKILL, and the operator's only signal is a pod that took its full grace period.
|
|
245
|
+
|
|
246
|
+
Three rules ride with it. **The unit is SECONDS** — `close({ timeout: profile.drainTimeoutMs /
|
|
247
|
+
1000 })`, and `timeout: 5000` would be an eighty-three minute budget, which is the same hang with
|
|
248
|
+
extra steps. **`drainTimeoutMs: 0` sends no option at all**, rather than `{ timeout: 0 }`: `migrate`
|
|
249
|
+
and `replicator` mean "wait", for `acquireTimeoutMs`' reason, and a zero handed to the driver is an
|
|
250
|
+
instruction whose reading is the driver's. **The verdict is the elapsed time**, because the driver
|
|
251
|
+
RESOLVES when it gives up rather than rejecting — a drain that abandoned in-flight work looks exactly
|
|
252
|
+
like a clean one, so `X_DB_DRAIN_TIMEOUT` is raised on the clock or nothing is said at all. That
|
|
253
|
+
clock is `performance.now()` and never `Date.now()`, and the reason is this repo rather than NTP:
|
|
254
|
+
the framework preload freezes `Date` for every test in the tree (`installDeterminism`), so a duration
|
|
255
|
+
subtracted from `Date.now()` is 0 in all of them and the branch could not fire — a test asserting it
|
|
256
|
+
would have been one that cannot fail. `pool-drain.test.ts` pins what is ASKED for, against a fake
|
|
257
|
+
pool; `pool-drain.live.test.ts` pins that a real server's driver honours it, because a fake's
|
|
258
|
+
`close()` is whatever the fake decided and the finding is about the real one.
|
|
259
|
+
|
|
236
260
|
`close()` reads its cached driver into a local, clears the field, **then** awaits the teardown —
|
|
237
261
|
`client.ts` and `pglite.ts` both. A teardown that rejects has still torn the pool down, so clearing
|
|
238
262
|
after the await left the corpse cached for the next `connect()`, and a second `close()` threw in
|
|
@@ -240,7 +264,7 @@ the same place rather than clearing it. The rejection still reaches the caller o
|
|
|
240
264
|
(`pglite.ts` swallows a failed *boot*, which is a different thing: there is nothing to close).
|
|
241
265
|
|
|
242
266
|
`execute()` trusts the command tag only when it is `> 0`, in **both** drivers — `rowsOf`
|
|
243
|
-
(`pglite.ts`) and `affectedBy` (`
|
|
267
|
+
(`pglite.ts`) and `affectedBy` (`statement-funnel.ts`) are one rule written twice, not two rules. PGlite
|
|
244
268
|
counts MODIFIED rows, so a SELECT that returned rows is tagged `0` and `??` would report 0 for
|
|
245
269
|
every read; a driver that tags a read `0` on the pooled side would have diverged from PGlite the
|
|
246
270
|
same way, and the same guard closes both. A write that modified nothing returned no rows either, so
|
|
@@ -257,7 +281,7 @@ the seam swallows nothing** — a throw from `onStatement` is how strict test mo
|
|
|
257
281
|
N+1 happened in, so a guarding facade here would silently delete that mode. `onStatement` is
|
|
258
282
|
synchronous, runs on the caller's stack after the statement settled, and must not issue SQL: a
|
|
259
283
|
statement from inside it re-enters the funnel and observes itself. Only two places may invoke it —
|
|
260
|
-
`runOn` (`
|
|
284
|
+
`runOn` (`statement-funnel.ts`) and `statement()` (`pglite.ts`), the funnels every statement already passes
|
|
261
285
|
through. Reserving a connection, booting PGlite and closing a pool are not statements and stay out.
|
|
262
286
|
|
|
263
287
|
Both funnels are now split in two, and the split is the whole design: `sendOn`/`send` is the raw
|
|
@@ -271,7 +295,7 @@ timeouts are still fifty statements; **notify outside the statement's own `try`*
|
|
|
271
295
|
that succeeded as `X_DB_UNAVAILABLE` and delete strict test mode's failure. On the failing path
|
|
272
296
|
the observer's throw replaces the DB error instead, which is the price of never swallowing — an
|
|
273
297
|
observer that only reports must not throw. `rows` comes from the same helper `execute()` uses
|
|
274
|
-
(`affectedBy` in `
|
|
298
|
+
(`affectedBy` in `statement-funnel.ts`, `rowsOf` in `pglite.ts`, hoisted to module scope for it), so the
|
|
275
299
|
report and the return value cannot disagree about one statement.
|
|
276
300
|
|
|
277
301
|
`attribution.ts` is `StatementEvent.attribution`'s producer: `withStatementAttribution(entity, op,
|
|
@@ -289,8 +313,8 @@ is the same fact written five times, with every path an author forgot it emittin
|
|
|
289
313
|
**Nesting keeps the innermost pair**, exactly as `expectedQueryLoop` keeps the innermost reason: a
|
|
290
314
|
relation preloaded during `findMany` reads through the *related* repository, so its statement is
|
|
291
315
|
attributed to that entity and its own operation, not to the read that triggered the preload.
|
|
292
|
-
**The funnels stamp, on both settle paths** — `runOn` (`
|
|
293
|
-
read `statementAttribution()` inside the branch that already found an observer, next to
|
|
316
|
+
**The funnels stamp, on both settle paths** — `runOn` (`statement-funnel.ts`) and
|
|
317
|
+
`statement()` (`pglite.ts`) read `statementAttribution()` inside the branch that already found an observer, next to
|
|
294
318
|
`expectedQueryLoopReason()`, and put it on the event whether the statement succeeded or failed, the
|
|
295
319
|
same argument as `expected`: a diagnostic that judges a whole request runs long after every scope
|
|
296
320
|
in it closed. `@ultimat3/entity`'s `postgresRepo` is the one producer — the last caller that still
|
|
@@ -1007,13 +1031,36 @@ generate repairs nothing. The empty-diff exclusion is `@ultimat3/cli`'s
|
|
|
1007
1031
|
makes every `x db gen` write a file holding no statement — a ledger row, a checksum and a place in
|
|
1008
1032
|
the apply order for nothing.
|
|
1009
1033
|
|
|
1010
|
-
**`REPLICA IDENTITY FULL` is
|
|
1011
|
-
|
|
1012
|
-
|
|
1013
|
-
|
|
1014
|
-
|
|
1015
|
-
|
|
1016
|
-
|
|
1034
|
+
**`REPLICA IDENTITY FULL` is emitted, `As of 2026-08-26` — by a PARAMETER, never by an entity
|
|
1035
|
+
field.** `@ultimat3/realtime` refuses a live query on a table without it and nothing in the
|
|
1036
|
+
framework wrote it, so a scaffolded app generated a schema its own preflight rejected (issue #357).
|
|
1037
|
+
Which tables need it is **declared** by each `live: true` query's `subscribes:` and read out of the
|
|
1038
|
+
manifest — **not derived**, and this file said "derived" until 2026-08-26. It cannot be derived: the
|
|
1039
|
+
relation name is a string inside the query's `sql:` callback, which no generator can invoke without
|
|
1040
|
+
valid input, so a live-query-to-table set does not exist anywhere to be read. `liveFeed` in the
|
|
1041
|
+
reference app requires `{ orgId: t.uuid, limit }` and its table is the `'posts'` literal inside
|
|
1042
|
+
`from<PostSummary>('posts', …)`; `packages/query/src/sql.ts` says the same thing about itself —
|
|
1043
|
+
"`null` when no sample input was supplied". `X_QUERY_SUBSCRIBES_DRIFT` is what keeps the declaration
|
|
1044
|
+
honest, checked against the resolved shape at first subscribe. It is still a PARAMETER and never an
|
|
1045
|
+
`EntityDescriptionLike` field — this package is tier 1 and can see neither the manifest nor
|
|
1046
|
+
`@ultimat3/query` — so such a field would have been a declared-and-never-wired key, the defect class
|
|
1047
|
+
this release exists to eliminate. `GenerateOptions.replicaIdentityFull: readonly string[] |
|
|
1048
|
+
undefined` is the shape, passed by `@ultimat3/cli`'s `db-generate.ts` from `describeQueries()` —
|
|
1049
|
+
the descriptor, one hop BEFORE the manifest. `x.manifest.json` projects the same declaration
|
|
1050
|
+
(`QueryFact.subscribes`) and is what any other reader should use, but `appManifest(root)` re-loads
|
|
1051
|
+
the app and calls `appIdentity(root)`, which throws `X_APP_PACKAGE_INVALID` where there is no
|
|
1052
|
+
`package.json` — and `x db gen` has never needed one. `replica-identity.ts` owns every rule that
|
|
1053
|
+
rides with it.
|
|
1054
|
+
|
|
1055
|
+
| Rule | Why |
|
|
1056
|
+
|---|---|
|
|
1057
|
+
| recorded on the snapshot as `TableDescription.replicaIdentityFull` | `true` or **absent**, never `false` — the literal type is the enforcement. Absent is "nothing recorded", the reading `checks` and `using` already have, so a sidecar written before the field emits the ALTER once more and Postgres accepts it on a table that has it. Without the record the statement lands in **every** migration forever, which is a generator an author learns to ignore |
|
|
1058
|
+
| the snapshot records the **union** with what was already recorded | a caller passing no set must not erase the fact. `snapshotOf(entities)` alone answers `NONE`, so the one place the union is computed is `generateMigration` |
|
|
1059
|
+
| dead **last** in `up` | the table has to exist and a `create table` in this same migration is why it might not. It is ordered against nothing else — replica identity constrains no column, index or constraint — so the end is the only placement that cannot read as depending on a statement above it |
|
|
1060
|
+
| never `-- destructive: true` | it drops no row, rewrites no column and matches none of `destructive.ts`'s four rules. `generate-replica-identity.test.ts` asserts both the verdict and `destructiveStatements()` |
|
|
1061
|
+
| a name **no entity declares** is skipped, silently | the list comes from the manifest, and a live query whose entity was deleted is an app fault this generator cannot repair. Emitting it anyway is `42P01` at `ROLE=migrate`, which is the one place this package refuses to put a fault |
|
|
1062
|
+
| nothing is ever **reverted** | the option is optional, so "absent" and "no live query subscribes any more" are the same value. Reading them alike would let a caller that never passes it turn off replication for every subscribed table in the app |
|
|
1063
|
+
| `down` is `replica identity default`, except on a table this migration **creates** | that table's whole `down` is already `drop table`; a second statement ahead of it is a line an author reads and nothing performs |
|
|
1017
1064
|
|
|
1018
1065
|
**A column the DATABASE computes is a different thing at every step, and `generated-column.ts` is
|
|
1019
1066
|
all of them** — `As of 2026-08-24`. `ColumnDescriptionLike.generated` carries the
|
|
@@ -1289,21 +1336,50 @@ survives the round trip whole.
|
|
|
1289
1336
|
is 0 and not 1.
|
|
1290
1337
|
|
|
1291
1338
|
- **`client.ts` reached the 500-line ceiling on 2026-08-26, and shed the five jobs that were not
|
|
1292
|
-
"open a connection and send a statement".** `pool-profile.ts` owns the
|
|
1339
|
+
"open a connection and send a statement".** `pool-profile.ts` owns the six numbers a pool runs
|
|
1293
1340
|
on — the per-role table, `DATABASE_POOL_MAX` and the screen every merged profile passes;
|
|
1294
1341
|
`connection-url.ts` builds the connection string (the libpq `options` merge and the
|
|
1295
1342
|
`application_name` label); `bun-sql.ts` declares the slice of `Bun.SQL` this package uses and
|
|
1296
1343
|
looks the global up lazily;
|
|
1297
1344
|
`pool-reserve.ts` is `reserve()` under the acquire deadline; `db-health.ts` is `checkDb`, the
|
|
1298
|
-
`/readyz` report. `client.ts` keeps connecting, the
|
|
1345
|
+
`/readyz` report. `client.ts` keeps connecting, the client object and the ambient `db()` —
|
|
1299
1346
|
and it still opens no socket at import, because `bunSqlFactory()` is reached from inside
|
|
1300
|
-
`connect()`.
|
|
1347
|
+
`connect()`. The **statement funnel** left with it on the same day, once the 500-line ceiling
|
|
1348
|
+
turned out not to be the bound this package is held to: `packages/db/src/**/*.ts` carries a
|
|
1349
|
+
path instruction of 200, and 263 lines is over it. `statement-funnel.ts` is `sendOn`/`runOn`
|
|
1350
|
+
plus the two shape helpers (`rowsOf`, `affectedBy`) — the seam this file already documents, and
|
|
1351
|
+
the one piece of `createPostgresClient` that closed over none of its state, so the move is a
|
|
1352
|
+
cut and a paste with no signature invented for it. Nothing outside this package imported any of
|
|
1353
|
+
the four, so no test's imports moved and no assertion changed. **The public surface did not move**: `src/index.ts` exports every one of those
|
|
1301
1354
|
names from its new module, so `@ultimat3/db` is byte-identical to what it was. The same day,
|
|
1302
1355
|
`drift.test.ts` split three ways along the three questions it was asking — `drift.test.ts`
|
|
1303
1356
|
(tables and columns), `drift-index.test.ts` and `drift-ledger.test.ts` (what the migrations
|
|
1304
1357
|
declare, and the post-migrate check) — over one shared `drift-fixtures.ts`, which
|
|
1305
1358
|
`drift-foreign-key.test.ts` now imports instead of carrying its own byte-identical copy.
|
|
1306
1359
|
|
|
1360
|
+
- **`DATABASE_URL`'s SCHEME is screened at boot, `As of 2026-08-26`** (issue #367). `new URL()`
|
|
1361
|
+
accepts a scheme-less connection string — `db.internal:5432/app` parses with `db.internal:` as
|
|
1362
|
+
the SCHEME and `5432/app` as the path — so `connectionUrl` saw a well-formed url and handed it
|
|
1363
|
+
on. Measured on bun 1.4.0, `Bun.SQL` then reads it as host `db.internal`, port 5432, database
|
|
1364
|
+
`app` and opens a Postgres pool on it, so the first symptom is a connect failure at the first
|
|
1365
|
+
QUERY, in another process phase, worded by the driver and naming neither the variable nor the
|
|
1366
|
+
missing `postgres://`. `POSTGRES_SCHEMES` is closed at **`postgres:` and `postgresql:`** —
|
|
1367
|
+
measured, not assumed: those two answer `adapter: 'postgres'`, while `pg:`, `tcp:` and
|
|
1368
|
+
`postgresql+ssl:` are refused by the driver itself (`Unsupported protocol: … Supported adapters:
|
|
1369
|
+
"postgres", "sqlite", "mysql", "mariadb"`), so excluding them costs a capability nobody has. The
|
|
1370
|
+
direction that matters is the one the driver ACCEPTS: `mysql:`, `mariadb:`, `sqlite:` and
|
|
1371
|
+
`file:` open a **different engine** and every statement generated here is Postgres. A
|
|
1372
|
+
**behaviour change**, not a defect repair — it narrows what the framework accepts, which is why
|
|
1373
|
+
it was deferred out of #364.
|
|
1374
|
+
**The received scheme is deliberately never echoed**, and this is the one refusal in the package
|
|
1375
|
+
that withholds the actionable token. `URL` reads the first token as the scheme, and for the value
|
|
1376
|
+
this exists for that token is the HOST (`db.internal:`); one dashboard field over
|
|
1377
|
+
(`app:hunter2@db.internal/app`) it is the USERNAME. Naming "the scheme" therefore puts a host or
|
|
1378
|
+
a credential in the boot log and the `--json` payload, where the logger has no key left to redact
|
|
1379
|
+
it by. The REQUIRED scheme is a constant and carries the whole instruction, and `describeValue`
|
|
1380
|
+
still keeps the shape, so an empty variable is told apart from a truncated one.
|
|
1381
|
+
`connection-url.test.ts` asserts the absence, so echoing it back is a failing test.
|
|
1382
|
+
|
|
1307
1383
|
- **`unexpectedTable`'s `fix:` no longer names `x db gen`, `As of 2026-08-26`** (issue #345). That
|
|
1308
1384
|
command diffs the ENTITY REGISTRY against the newest snapshot, and a table nothing declares is on
|
|
1309
1385
|
neither side of it — so the diff came back empty, the generator's empty-diff branch writes NO
|
|
@@ -1341,6 +1417,46 @@ survives the round trip whole.
|
|
|
1341
1417
|
both declarations true; `packages/entity/src/errors.test.ts` asserts the two texts are equal,
|
|
1342
1418
|
so a one-sided edit is a failing test rather than a comment nobody read.
|
|
1343
1419
|
|
|
1420
|
+
- **A JS array bound as a parameter is rendered here, because `Bun.SQL` does not render it,
|
|
1421
|
+
`As of 2026-08-26`** (issue #384). `Bun.SQL`'s positional form serialises an array by JOINING ITS
|
|
1422
|
+
ELEMENTS WITH COMMAS, so `unsafe('select $1::text[]', [['x', 'y']])` sends the string `x,y` and
|
|
1423
|
+
Postgres answers `22P02 malformed array literal: "x,y"` — measured on bun 1.4.0 against Postgres
|
|
1424
|
+
17. **Three shipped statements bind an array and all three failed**: `@ultimat3/jobs`' `SQL_CLAIM`
|
|
1425
|
+
(the whole loop of every `ROLE=worker` container the framework produces, so a real deployment
|
|
1426
|
+
claimed nothing and every job sat in its queue), `SQL_OUTBOX_RELEASE` (the relay giving an
|
|
1427
|
+
unpublished batch back) and `@ultimat3/notify`'s `SQL_NOTIFY_INBOX_MARK_READ`.
|
|
1428
|
+
`array-parameter.ts` is the encoder and `sendOn` (`statement-funnel.ts`) is the one caller — this
|
|
1429
|
+
driver's only `unsafe` call, so one encoder is every caller fixed and a helper each site imports
|
|
1430
|
+
is three chances to forget and a fourth site tomorrow that does (axiom 1).
|
|
1431
|
+
|
|
1432
|
+
**Why nothing caught it, and why the repair test is in `@ultimat3/cli`.** `pglite.ts` is a
|
|
1433
|
+
separate driver that encodes an array correctly, and `x dev` runs the embedded default — so the
|
|
1434
|
+
framework's own dev loop is blind by construction and only a container with `DATABASE_URL` ever
|
|
1435
|
+
meets the failure. Every other test of those three statements runs against a recording executor
|
|
1436
|
+
and asserts their SQL as TEXT, which cannot see whether a parameter PARSES;
|
|
1437
|
+
`grep -rln '\.claim(' --include=*.live.test.ts packages/` answered ONE file before this landed.
|
|
1438
|
+
`packages/db/src/array-parameter.live.test.ts` pins the grammar against a real server — and
|
|
1439
|
+
asserts the RAW array is still refused, so deleting the encoder fails rather than passing on any
|
|
1440
|
+
driver that happens to encode. `packages/cli/src/pg-array.live.test.ts` is the composition test:
|
|
1441
|
+
it is in `cli` because nothing else can see all three — this package is tier 1 and may not import
|
|
1442
|
+
`jobs` (3) or `notify` (4), and neither of those can build a db-backed `PgExecutor` — so
|
|
1443
|
+
`pgExecutorFor(createPostgresClient(...))`, the executor every booted role actually gets, is the
|
|
1444
|
+
only place the three real statements meet the real driver.
|
|
1445
|
+
|
|
1446
|
+
Three grammar rules earn their line. **`NULL` bare is the array null and `"NULL"` is the
|
|
1447
|
+
four-character string**, so a JS `null` renders bare and a queue really spelled `NULL` must not
|
|
1448
|
+
become one. **Quoting is by content, not by type** — a comma, a brace, a quote, a backslash,
|
|
1449
|
+
surrounding whitespace or the empty string, which unquoted is not an element at all. **A
|
|
1450
|
+
`Uint8Array` is BYTEA and is deliberately not an array**: `Array.isArray` answers `false` for a
|
|
1451
|
+
typed array, which is behaviour this relies on rather than a case it writes. **A RAGGED nest is
|
|
1452
|
+
REFUSED**, never rendered — Postgres has no jagged array and `{{a,b},{c}}` is the same `22P02`,
|
|
1453
|
+
measured on 17 beside the rectangular `{{a,b},{c,d}}` that parses, so a literal this module is
|
|
1454
|
+
willing to emit is one the server is willing to read. `X_INVARIANT` through core's `assert`, the
|
|
1455
|
+
code this package already borrows for a value this build cannot honour; mixed depth (`{a,{b,c}}`)
|
|
1456
|
+
is caught by the same guard, which a rule comparing row LENGTHS alone would let through. And the
|
|
1457
|
+
common path allocates nothing — one `some` over a short list, then the caller's own array by identity, because
|
|
1458
|
+
every statement the framework runs passes through here and almost none binds an array (axiom 6).
|
|
1459
|
+
|
|
1344
1460
|
```bash
|
|
1345
1461
|
bun test # from packages/db
|
|
1346
1462
|
bun run typecheck
|
package/README.md
CHANGED
|
@@ -43,7 +43,7 @@ await withTransaction(async (tx) => {
|
|
|
43
43
|
| `snapshotJson()` | `As of 2026-08`: the sidecar's **bytes** — the JSON Biome would have printed, trailing newline included. The one writer of a `<id>.snapshot.json`, because `JSON.stringify(…, null, 2)` is not formatter-clean and an app's `lint` step rejected the file `x db gen` had just written |
|
|
44
44
|
| `isLedgerMissing()` | `As of 2026-08`: whether an error is Postgres' `undefined_table` for `x_migrations` — the one condition a caller may read as "nothing applied" |
|
|
45
45
|
| `appTables()` / `FRAMEWORK_TABLE_PREFIX` | `As of 2026-08`: the live schema minus the `x_` namespace — no migration declares the ledger, the queue, the outbox or an auth table, so none of them is drift |
|
|
46
|
-
| `generateMigration()` | `x db gen "<name>"` — reversible up/down SQL, and `destructive` for the marker the file must carry. `As of 2026-08` a foreign key is its own `alter table … add constraint`, emitted after every table statement: inline, a `references()` had to point at a table entity registration order happened to create first, and `down` had to drop them in an order it did not control. `As of 2026-08-19` a **removed** `references()` emits its `drop constraint` (it emitted nothing, and the snapshot then denied a constraint the database still held), a changed `onDelete` is a drop-and-add rebuild, and a declared `on delete` rule reaches the clause at all. `As of 2026-08-25` a **retype** drops the partial indexes and CHECK constraints written against that column first and restores them in `down`: Postgres compiles both predicates against the old type and cannot recompile either, so `alter column … type text using …::text` was `42883 operator does not exist: text = post_status` and the migration aborted mid-run. A plain btree over the column is left alone — measured, Postgres rebuilds that one itself |
|
|
46
|
+
| `generateMigration()` | `x db gen "<name>"` — reversible up/down SQL, and `destructive` for the marker the file must carry. `As of 2026-08` a foreign key is its own `alter table … add constraint`, emitted after every table statement: inline, a `references()` had to point at a table entity registration order happened to create first, and `down` had to drop them in an order it did not control. `As of 2026-08-19` a **removed** `references()` emits its `drop constraint` (it emitted nothing, and the snapshot then denied a constraint the database still held), a changed `onDelete` is a drop-and-add rebuild, and a declared `on delete` rule reaches the clause at all. `As of 2026-08-25` a **retype** drops the partial indexes and CHECK constraints written against that column first and restores them in `down`: Postgres compiles both predicates against the old type and cannot recompile either, so `alter column … type text using …::text` was `42883 operator does not exist: text = post_status` and the migration aborted mid-run. A plain btree over the column is left alone — measured, Postgres rebuilds that one itself. `As of 2026-08-26` `replicaIdentityFull` names the tables a live query subscribes to and emits one `alter table … replica identity full` each, last in `up`, recorded on the snapshot so the next generation emits none — a parameter and never an entity field, because the live-query set is a tier-3 fact (`replica-identity.ts`) |
|
|
47
47
|
| `declaredIndexes()` / `invariantChecks()` / `constraintNameFor()` | `As of 2026-08-25`: the DDL an entity **invariant** becomes — a `check` as a named `CONSTRAINT`, a `unique` as a partial-capable unique INDEX, an `assert` as nothing. `EntityDescriptionLike` had no `invariants` field for three majors, so a regenerated migration silently held **none** of them, including the composite UNIQUE `upsertAll`'s `on conflict` is inferred against |
|
|
48
48
|
| `declaredChecks()` / `checkClauses()` / `checkPlan()` / `columnChecks()` / `columnCheckName()` / `columnNamesConstraint()` | `As of 2026-08-25`: **every** CHECK a table declares — a column's own (`enumerated()`'s value set, `tz()`'s IANA whitelist, `locale()`'s tags, money's currency pattern and scale bound) and an invariant's — on ONE list, so `createTable`, `diffTable` and `snapshotOf` agree about what exists. A column's check reached `create table` **inline and anonymous** and nothing else: the snapshot recorded none and the diff had no arm, so a value added to `enumerated()` generated no migration and a regenerated ENUM column came back as bare `text`. The name is `<table>_<column>_check` because that is the name **Postgres itself mints** for the old anonymous form — measured — so the repair lands on the constraint an already-generated database is holding; `checkPlan` emits `drop constraint if exists` before the `add` for exactly that column, because a bare add is `42710` there and a no-op everywhere else |
|
|
49
49
|
| `defaultExpression()` / `ColumnDefaultLike` | `As of 2026-08-25`: a column's `default` as SQL. A DECLARED default (`{ kind: 'value', value }`) wins; `gen_random_uuid()` and `now()` stay as the inference for a description that carries only `hasDefault` |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ultimat3/db",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "19.0.0",
|
|
4
4
|
"description": "Postgres access, transactions, migrations and drift detection",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -24,14 +24,14 @@
|
|
|
24
24
|
"LICENSE"
|
|
25
25
|
],
|
|
26
26
|
"engines": {
|
|
27
|
-
"bun": ">=1.
|
|
27
|
+
"bun": ">=1.4.0"
|
|
28
28
|
},
|
|
29
29
|
"scripts": {
|
|
30
30
|
"typecheck": "tsc --noEmit -p tsconfig.json",
|
|
31
31
|
"test": "bun test"
|
|
32
32
|
},
|
|
33
33
|
"dependencies": {
|
|
34
|
-
"@ultimat3/core": "
|
|
34
|
+
"@ultimat3/core": "19.0.0"
|
|
35
35
|
},
|
|
36
36
|
"peerDependencies": {
|
|
37
37
|
"@electric-sql/pglite": ">=0.5.0"
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
// Single responsibility: a JS array bound as a statement parameter, rendered as the Postgres array
|
|
2
|
+
// literal `Bun.SQL` does not render.
|
|
3
|
+
//
|
|
4
|
+
// WHAT IS BROKEN WITHOUT IT. `Bun.SQL`'s positional form serialises an array by JOINING ITS
|
|
5
|
+
// ELEMENTS WITH COMMAS, so `unsafe('select $1::text[]', [['x', 'y']])` sends the string `x,y` and
|
|
6
|
+
// Postgres answers `malformed array literal: "x,y"` (SQLSTATE 22P02). Measured on Bun 1.4.0
|
|
7
|
+
// against Postgres 17. Three shipped statements bound an array through that path and every one
|
|
8
|
+
// of them failed — `SQL_CLAIM` (the worker's claim loop), `SQL_OUTBOX_RELEASE` and
|
|
9
|
+
// `SQL_NOTIFY_INBOX_MARK_READ`. Issue #384.
|
|
10
|
+
//
|
|
11
|
+
// WHY HERE AND NOT AT THE THREE CALL SITES. `sendOn` is the one place this driver's `unsafe` is
|
|
12
|
+
// called, so one encoder here is every caller fixed and none of them has to remember — a helper
|
|
13
|
+
// each site imports is three chances to forget and a fourth site tomorrow that does. Axiom 1.
|
|
14
|
+
//
|
|
15
|
+
// WHY PGLITE IS UNTOUCHED. `pglite.ts` is a separate driver with its own `send`, and it encodes an
|
|
16
|
+
// array parameter correctly already — which is exactly why nothing caught this: `x dev` runs the
|
|
17
|
+
// embedded default, so the framework's own dev loop is systematically blind to a defect that only
|
|
18
|
+
// appears once `DATABASE_URL` selects `Bun.SQL`. A container is where it bites.
|
|
19
|
+
|
|
20
|
+
import { assert } from '@ultimat3/core';
|
|
21
|
+
|
|
22
|
+
/**
|
|
23
|
+
* One element, quoted only when it has to be.
|
|
24
|
+
*
|
|
25
|
+
* `NULL` unquoted is the array NULL and `"NULL"` is the four-character string, so a JS `null`
|
|
26
|
+
* MUST render bare and a string that happens to spell it must not. Everything else is quoted when
|
|
27
|
+
* it holds a character the literal grammar reads as structure — a comma, a brace, a quote, a
|
|
28
|
+
* backslash, or leading/trailing whitespace the parser would strip — plus the empty string, which
|
|
29
|
+
* unquoted is not an element at all.
|
|
30
|
+
*/
|
|
31
|
+
function element(value: unknown): string {
|
|
32
|
+
if (value === null || value === undefined) return 'NULL';
|
|
33
|
+
// A Date is ALWAYS quoted, even though an ISO-8601 instant carries no character the grammar
|
|
34
|
+
// reads as structure. A timestamp element is conventionally quoted, and the alternative is a
|
|
35
|
+
// rule that holds only while nothing ever renders a timestamp with a space in it.
|
|
36
|
+
if (value instanceof Date) return `"${value.toISOString()}"`;
|
|
37
|
+
const text = String(value);
|
|
38
|
+
const structural = /[{},"\\\s]/.test(text) || text.length === 0 || text.toUpperCase() === 'NULL';
|
|
39
|
+
if (!structural) return text;
|
|
40
|
+
return `"${text.replaceAll('\\', '\\\\').replaceAll('"', '\\"')}"`;
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
/**
|
|
44
|
+
* The array literal for one parameter: `{a,b,c}`, elements escaped.
|
|
45
|
+
*
|
|
46
|
+
* NESTED ARRAYS ARE RENDERED, not refused: Postgres reads `{{a,b},{c,d}}` as a 2-dimensional
|
|
47
|
+
* array, and rendering one is strictly closer to right than sending `a,b,c,d`. Nothing in this
|
|
48
|
+
* tree binds one today.
|
|
49
|
+
*
|
|
50
|
+
* A RAGGED nest is REFUSED, never rendered. Postgres has no jagged array — every extent of a
|
|
51
|
+
* dimension must match — and `{{a,b},{c}}` is `22P02 malformed array literal`, measured on 17
|
|
52
|
+
* beside the rectangular `{{a,b},{c,d}}` that parses (`array-parameter.live.test.ts`). So a
|
|
53
|
+
* literal this function is willing to emit is one the server is willing to read: rendering the
|
|
54
|
+
* jagged one puts the fault two layers away, in the driver's words, naming neither the parameter
|
|
55
|
+
* nor which row is short. `X_INVARIANT` through core's `assert`, the code this package already
|
|
56
|
+
* borrows for a value this build cannot honour (`createIndex`'s unique GIN, `generatedClause`'s
|
|
57
|
+
* generated-and-defaulted column).
|
|
58
|
+
*/
|
|
59
|
+
export function pgArrayLiteral(values: readonly unknown[]): string {
|
|
60
|
+
const nested = values.filter((value): value is readonly unknown[] => Array.isArray(value));
|
|
61
|
+
// Mixed depth is ragged too — `{a,{b,c}}` is a scalar beside a dimension, which Postgres reads
|
|
62
|
+
// as the same malformed literal. Comparing counts alone would let it through.
|
|
63
|
+
assert(
|
|
64
|
+
nested.length === 0 || nested.length === values.length,
|
|
65
|
+
'a nested array parameter mixes scalars and arrays at one level, and Postgres has no such array',
|
|
66
|
+
'bind one array of scalars, or one array whose every element is an array of equal length',
|
|
67
|
+
);
|
|
68
|
+
const width = nested[0]?.length;
|
|
69
|
+
assert(
|
|
70
|
+
nested.every((row) => row.length === width),
|
|
71
|
+
`a nested array parameter is ragged — its rows are ${nested.map((row) => row.length).join(', ')} long, and Postgres has no jagged array`,
|
|
72
|
+
'give every row the same length, or bind one array per row',
|
|
73
|
+
);
|
|
74
|
+
return `{${values.map((value) => (Array.isArray(value) ? pgArrayLiteral(value) : element(value))).join(',')}}`;
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
/**
|
|
78
|
+
* Every parameter of one statement, arrays rendered and everything else passed through untouched.
|
|
79
|
+
*
|
|
80
|
+
* A NEW ARRAY ONLY WHEN SOMETHING CHANGED. Every statement the framework runs goes through this
|
|
81
|
+
* function, and almost none of them binds an array — so the common path is one `some` over a short
|
|
82
|
+
* list and the caller's own array object, byte for byte, which is what `sendOn` had before this
|
|
83
|
+
* existed (axiom 6).
|
|
84
|
+
*
|
|
85
|
+
* A `Uint8Array` is BYTEA and is deliberately not an array here: `Array.isArray` answers `false`
|
|
86
|
+
* for a typed array, which is the behaviour this relies on rather than a special case it writes.
|
|
87
|
+
*/
|
|
88
|
+
export function encodeArrayParameters(values: readonly unknown[]): readonly unknown[] {
|
|
89
|
+
if (!values.some(Array.isArray)) return values;
|
|
90
|
+
return values.map((value) => (Array.isArray(value) ? pgArrayLiteral(value) : value));
|
|
91
|
+
}
|
package/src/bun-sql.ts
CHANGED
|
@@ -1,13 +1,55 @@
|
|
|
1
|
-
// Single responsibility: the slice of `Bun.SQL` this package uses, declared structurally,
|
|
2
|
-
//
|
|
3
|
-
// never touches `Bun` at module evaluation —
|
|
1
|
+
// Single responsibility: the slice of `Bun.SQL` this package uses, declared structurally, the lazy
|
|
2
|
+
// lookup of the global that provides it, and the one safe way to hand a pinned connection back.
|
|
3
|
+
// Reached through a function so importing the client never touches `Bun` at module evaluation —
|
|
4
|
+
// the CLI imports it to print help.
|
|
4
5
|
|
|
6
|
+
import { logger, renderThrowable } from '@ultimat3/core';
|
|
5
7
|
import { dbUnavailable } from './errors';
|
|
6
8
|
|
|
7
9
|
/** One connection pinned out of `Bun.SQL`'s pool, released back by hand. */
|
|
8
10
|
export interface BunSqlReserved {
|
|
9
11
|
unsafe(text: string, values?: readonly unknown[]): Promise<unknown>;
|
|
10
|
-
|
|
12
|
+
/**
|
|
13
|
+
* **Answers a PROMISE, and typing it `void` is what made both callers float it.** Measured on
|
|
14
|
+
* Bun 1.3.14 and 1.4.0 against a real server: `release()` returns a promise on both, and on
|
|
15
|
+
* 1.3.14 that promise REJECTS with `ERR_POSTGRES_CONNECTION_CLOSED` when the pool has already
|
|
16
|
+
* been closed. Nothing was attached to it, so it surfaced as an UNHANDLED REJECTION — which Bun
|
|
17
|
+
* takes the process down for. `unknown` rather than `Promise<void>` because a fake reserved
|
|
18
|
+
* connection returns nothing at all, and the caller has to handle both anyway.
|
|
19
|
+
*/
|
|
20
|
+
release(): unknown;
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
/**
|
|
24
|
+
* Hand a pin back, totally. The one place that knows `release()` answers a promise, so neither
|
|
25
|
+
* caller can forget it (axiom 1) — `client.ts`'s `DbConnection.release` and `pool-reserve.ts`'s
|
|
26
|
+
* late arrival both route here.
|
|
27
|
+
*
|
|
28
|
+
* A failed release is **best-effort, exactly where a throw would mask the error that caused it** —
|
|
29
|
+
* the rule this package already applies to `ROLLBACK`. `[Symbol.dispose]` is `DbConnection.release`
|
|
30
|
+
* itself, so a throw there replaces whatever error reached the `using` block, or invents one where
|
|
31
|
+
* the body succeeded. And the news is unactionable: the connection this would hand back is gone
|
|
32
|
+
* either way.
|
|
33
|
+
*
|
|
34
|
+
* Reachable, and reachable BECAUSE `close()` is bounded: an abandoned drain leaves every
|
|
35
|
+
* still-pinned connection to be released against a pool that no longer exists.
|
|
36
|
+
*/
|
|
37
|
+
export function releaseReserved(reserved: BunSqlReserved): void {
|
|
38
|
+
const report = (error: unknown): void => {
|
|
39
|
+
logger.debug('db.release_failed', { error: renderThrowable(error) });
|
|
40
|
+
};
|
|
41
|
+
let settled: unknown;
|
|
42
|
+
try {
|
|
43
|
+
settled = reserved.release();
|
|
44
|
+
} catch (error) {
|
|
45
|
+
report(error);
|
|
46
|
+
return;
|
|
47
|
+
}
|
|
48
|
+
// `then` and not `instanceof Promise`: the value comes from the driver, and a thenable is the
|
|
49
|
+
// contract every await in this package already relies on.
|
|
50
|
+
if (typeof (settled as PromiseLike<unknown> | undefined)?.then === 'function') {
|
|
51
|
+
void (settled as PromiseLike<unknown>).then(undefined, report);
|
|
52
|
+
}
|
|
11
53
|
}
|
|
12
54
|
|
|
13
55
|
/** The slice of `Bun.SQL` we use. Declared structurally so this package has no dependency. */
|
package/src/client.ts
CHANGED
|
@@ -1,22 +1,20 @@
|
|
|
1
|
-
// Single responsibility: the pooled Postgres client and the ambient `db()` handle —
|
|
2
|
-
//
|
|
1
|
+
// Single responsibility: the pooled Postgres client and the ambient `db()` handle — the lazy
|
|
2
|
+
// connect, the reserved-connection pin, and the process-wide client every repository reaches
|
|
3
3
|
// through. Sizing lives in `pool-profile.ts`, the connection string in `connection-url.ts`, the
|
|
4
|
-
// `Bun.SQL` slice in `bun-sql.ts
|
|
4
|
+
// `Bun.SQL` slice in `bun-sql.ts` and the observed statement funnel in `statement-funnel.ts`, so
|
|
5
|
+
// importing this module never opens a socket.
|
|
5
6
|
|
|
6
7
|
import { type Role, resolveRole } from '@ultimat3/core';
|
|
7
|
-
import {
|
|
8
|
-
import { type BunSqlDriver, type BunSqlReserved, bunSqlFactory } from './bun-sql';
|
|
8
|
+
import { type BunSqlDriver, type BunSqlReserved, bunSqlFactory, releaseReserved } from './bun-sql';
|
|
9
9
|
import { connectionUrl } from './connection-url';
|
|
10
10
|
// Deliberate cycle, the same shape as `client.ts ⇄ transaction.ts`: nothing here is referenced at
|
|
11
11
|
// module evaluation, and both sides are `function` declarations, so hoisting covers the TDZ.
|
|
12
12
|
import { defaultClient } from './default-client';
|
|
13
|
-
import { DbError, driverError } from './errors';
|
|
14
|
-
import { expectedQueryLoopReason } from './expected-loop';
|
|
15
|
-
import { statementObserver } from './observe';
|
|
13
|
+
import { DbError, drainTimeout, driverError } from './errors';
|
|
16
14
|
import { assertPoolProfile, type PoolProfile, poolProfileFor } from './pool-profile';
|
|
17
15
|
import { reserveWithin } from './pool-reserve';
|
|
18
16
|
import { type SqlFragment, sql } from './sql';
|
|
19
|
-
import {
|
|
17
|
+
import { affectedBy, rowsOf, runOn } from './statement-funnel';
|
|
20
18
|
import { currentTx } from './transaction';
|
|
21
19
|
|
|
22
20
|
export interface DbClient {
|
|
@@ -51,20 +49,6 @@ export interface PostgresClientOptions {
|
|
|
51
49
|
readonly applicationName?: string | undefined;
|
|
52
50
|
}
|
|
53
51
|
|
|
54
|
-
function rowsOf<T>(result: unknown): readonly T[] {
|
|
55
|
-
return Array.isArray(result) ? (result as readonly T[]) : [];
|
|
56
|
-
}
|
|
57
|
-
|
|
58
|
-
// The command tag only when it counted something, exactly like `rowsOf` in `pglite.ts` — one rule
|
|
59
|
-
// across both drivers, so `execute()` and the observer's event cannot answer differently for the
|
|
60
|
-
// same statement depending on which database is behind them. A driver that tags a read `0` while
|
|
61
|
-
// returning rows would otherwise report 0 here and the row count there.
|
|
62
|
-
function affectedBy(result: unknown): number {
|
|
63
|
-
if (!Array.isArray(result)) return 0;
|
|
64
|
-
const count = (result as { count?: unknown }).count;
|
|
65
|
-
return typeof count === 'number' && count > 0 ? count : result.length;
|
|
66
|
-
}
|
|
67
|
-
|
|
68
52
|
export interface PostgresClient extends ReservableClient {
|
|
69
53
|
readonly profile: PoolProfile;
|
|
70
54
|
ping(): Promise<void>;
|
|
@@ -88,76 +72,6 @@ export function createPostgresClient(options: PostgresClientOptions = {}): Postg
|
|
|
88
72
|
return driver;
|
|
89
73
|
}
|
|
90
74
|
|
|
91
|
-
/** The send itself: one statement on one handle, every driver failure typed on the way out. */
|
|
92
|
-
async function sendOn(
|
|
93
|
-
driver: Pick<BunSqlDriver, 'unsafe'>,
|
|
94
|
-
fragment: SqlFragment,
|
|
95
|
-
): Promise<unknown> {
|
|
96
|
-
try {
|
|
97
|
-
return await driver.unsafe(fragment.text, fragment.values);
|
|
98
|
-
} catch (error) {
|
|
99
|
-
// `driverError`, not `dbUnavailable`: the SQLSTATE has always been on this error and nothing
|
|
100
|
-
// read it, so a `23505` from two clicks racing a signup told the operator the database was
|
|
101
|
-
// unreachable and paged on-call for an outage that never happened. Everything the table does
|
|
102
|
-
// not classify is still `X_DB_UNAVAILABLE`, byte for byte.
|
|
103
|
-
throw driverError(`statement failed: ${fragment.text.slice(0, 120)}`, error);
|
|
104
|
-
}
|
|
105
|
-
}
|
|
106
|
-
|
|
107
|
-
/**
|
|
108
|
-
* The funnel — pooled and pinned statements both arrive here, which is why the observer hangs
|
|
109
|
-
* off this one function and nowhere else. Uninstalled it costs one property read and one
|
|
110
|
-
* branch: no clock read, no span, no event object, and `sendOn` receives exactly the call `runOn`
|
|
111
|
-
* made before the seam existed (axiom 6).
|
|
112
|
-
*/
|
|
113
|
-
async function runOn(
|
|
114
|
-
driver: Pick<BunSqlDriver, 'unsafe'>,
|
|
115
|
-
fragment: SqlFragment,
|
|
116
|
-
): Promise<unknown> {
|
|
117
|
-
const observer = statementObserver();
|
|
118
|
-
if (observer === undefined) return sendOn(driver, fragment);
|
|
119
|
-
// Read here, not by the consumer: the scope is gone by the time a per-request detector judges
|
|
120
|
-
// what it collected, so the reason has to be captured with the statement it defends.
|
|
121
|
-
const expected = expectedQueryLoopReason();
|
|
122
|
-
// Same moment, same argument: `postgresRepo` is several frames and a microtask above this one,
|
|
123
|
-
// and what it knows — the entity and the operation — is what turns fifty identical `select`s
|
|
124
|
-
// into "50× findById on members". Absent for hand-written SQL, a migration, a health probe.
|
|
125
|
-
const attribution = statementAttribution();
|
|
126
|
-
const started = performance.now();
|
|
127
|
-
let result: unknown;
|
|
128
|
-
try {
|
|
129
|
-
// The span wraps the send and nothing else, so its duration is the statement's and the
|
|
130
|
-
// observer's own work is not charged to the database.
|
|
131
|
-
result = await withStatementSpan(fragment.text, () => sendOn(driver, fragment));
|
|
132
|
-
} catch (error) {
|
|
133
|
-
// A statement that failed is still a statement: fifty identical timeouts are an N+1 of
|
|
134
|
-
// timeouts. The error is already `X_DB_UNAVAILABLE`, so the event carries what the caller
|
|
135
|
-
// is about to be thrown — and an observer that throws here replaces it, which is why
|
|
136
|
-
// `observe.ts` says a reporting-only observer must not throw.
|
|
137
|
-
observer.onStatement({
|
|
138
|
-
text: fragment.text,
|
|
139
|
-
values: fragment.values,
|
|
140
|
-
durationMs: performance.now() - started,
|
|
141
|
-
rows: 0,
|
|
142
|
-
error,
|
|
143
|
-
attribution,
|
|
144
|
-
expected,
|
|
145
|
-
});
|
|
146
|
-
throw error;
|
|
147
|
-
}
|
|
148
|
-
// Outside the `try` deliberately: a throw from `onStatement` is the observer's, not the
|
|
149
|
-
// database's, and catching it above would report a statement that succeeded as failed.
|
|
150
|
-
observer.onStatement({
|
|
151
|
-
text: fragment.text,
|
|
152
|
-
values: fragment.values,
|
|
153
|
-
durationMs: performance.now() - started,
|
|
154
|
-
rows: affectedBy(result),
|
|
155
|
-
attribution,
|
|
156
|
-
expected,
|
|
157
|
-
});
|
|
158
|
-
return result;
|
|
159
|
-
}
|
|
160
|
-
|
|
161
75
|
async function run(fragment: SqlFragment): Promise<unknown> {
|
|
162
76
|
return runOn(connect(), fragment);
|
|
163
77
|
}
|
|
@@ -206,7 +120,8 @@ export function createPostgresClient(options: PostgresClientOptions = {}): Postg
|
|
|
206
120
|
const release = (): void => {
|
|
207
121
|
if (!held) return;
|
|
208
122
|
held = false;
|
|
209
|
-
|
|
123
|
+
// Total by construction — `releaseReserved` owns the reason (`bun-sql.ts`).
|
|
124
|
+
releaseReserved(reserved);
|
|
210
125
|
};
|
|
211
126
|
return {
|
|
212
127
|
query: async <T>(fragment: SqlFragment) => rowsOf<T>(await on(fragment)),
|
|
@@ -227,7 +142,40 @@ export function createPostgresClient(options: PostgresClientOptions = {}): Postg
|
|
|
227
142
|
// rejection still reaches the caller — a shutdown that could not drain wants to know.
|
|
228
143
|
const pool = driver;
|
|
229
144
|
driver = undefined;
|
|
230
|
-
|
|
145
|
+
if (pool === undefined) return;
|
|
146
|
+
// BOUNDED, `As of 2026-08-27`, and through the driver's OWN option rather than a race here.
|
|
147
|
+
// This was a bare `await pool.close()`, and `Bun.SQL`'s `end()` waits on an outstanding
|
|
148
|
+
// reserved connection without ever giving up — measured three runs per case on Bun 1.3.14
|
|
149
|
+
// AND 1.4.0, no database outage involved (#394). So a role whose database went away
|
|
150
|
+
// mid-shutdown never finished shutting down, and the operator's only signal was a container
|
|
151
|
+
// that burned its whole termination grace period before SIGKILL.
|
|
152
|
+
//
|
|
153
|
+
// `BunSqlDriver.close` has declared `{ timeout }` since this port was written and NOTHING
|
|
154
|
+
// ever passed it — the capability was in the seam, unused, exactly like `setOfflineMode` on
|
|
155
|
+
// the CDP port. Measured with a reserve outstanding: `close({ timeout: 1 })` returns in
|
|
156
|
+
// ~1002ms on 1.3.14, 1.4.0 and 1.4.1-canary alike, where a bare `close()` never returns.
|
|
157
|
+
//
|
|
158
|
+
// **The unit is SECONDS**, not milliseconds. `timeout: 5000` would be an eighty-three minute
|
|
159
|
+
// shutdown budget, which is the same hang with extra steps.
|
|
160
|
+
if (profile.drainTimeoutMs === 0) {
|
|
161
|
+
// `migrate` and `replicator`, for `acquireTimeoutMs`' reason: a run-once role cutting off
|
|
162
|
+
// its own session mid-statement is worse than a slow exit.
|
|
163
|
+
await pool.close();
|
|
164
|
+
return;
|
|
165
|
+
}
|
|
166
|
+
// `performance.now()`, never `Date.now()`, and the reason is this repo rather than NTP: the
|
|
167
|
+
// framework preload freezes `Date` for every test in the tree (`installDeterminism`), so a
|
|
168
|
+
// duration subtracted from `Date.now()` is 0 in all of them — the branch below could not
|
|
169
|
+
// fire, and the test asserting it would have been one that cannot fail.
|
|
170
|
+
const started = performance.now();
|
|
171
|
+
await pool.close({ timeout: profile.drainTimeoutMs / 1000 });
|
|
172
|
+
// The driver RESOLVES when it gives up — it does not reject — so the elapsed time is the only
|
|
173
|
+
// thing that separates "drained" from "abandoned". Reporting it is the point: a drain that
|
|
174
|
+
// silently gave up looks exactly like a clean one, and the work still in flight is lost with
|
|
175
|
+
// no line anywhere saying so. The pool is gone either way, which is why this is terminal.
|
|
176
|
+
if (performance.now() - started >= profile.drainTimeoutMs) {
|
|
177
|
+
throw drainTimeout(profile.drainTimeoutMs, role);
|
|
178
|
+
}
|
|
231
179
|
},
|
|
232
180
|
};
|
|
233
181
|
return client;
|
package/src/connection-url.ts
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
// `client.ts` because which settings reach a connection is a rule, not a step of connecting.
|
|
4
4
|
|
|
5
5
|
import { describeValue } from '@ultimat3/core';
|
|
6
|
-
import { dbUnavailable } from './errors';
|
|
6
|
+
import { DbError, dbUnavailable } from './errors';
|
|
7
7
|
import { declaresLibpqOption, mergeLibpqOptions } from './libpq-options';
|
|
8
8
|
import type { PoolProfile } from './pool-profile';
|
|
9
9
|
|
|
@@ -12,6 +12,41 @@ export interface ConnectionUrlOptions {
|
|
|
12
12
|
readonly applicationName?: string | undefined;
|
|
13
13
|
}
|
|
14
14
|
|
|
15
|
+
/**
|
|
16
|
+
* The schemes on which `Bun.SQL` opens a POSTGRES connection — measured against bun 1.4.0, never
|
|
17
|
+
* assumed. `postgres:` and `postgresql:` both answer `adapter: 'postgres'`; `pg:`, `tcp:` and
|
|
18
|
+
* `postgresql+ssl:` are refused by the driver itself (`Unsupported protocol: … Supported adapters:
|
|
19
|
+
* "postgres", "sqlite", "mysql", "mariadb"`), so excluding them costs a capability nobody has.
|
|
20
|
+
* The two that matter are the ones the driver ACCEPTS and this package cannot speak: `mysql:`,
|
|
21
|
+
* `mariadb:`, `sqlite:` and `file:` open a different engine, and every statement generated here is
|
|
22
|
+
* Postgres — a pool that connects and then answers a syntax error to the migration is strictly
|
|
23
|
+
* worse than one that refuses at boot.
|
|
24
|
+
*/
|
|
25
|
+
const POSTGRES_SCHEMES: ReadonlySet<string> = new Set(['postgres:', 'postgresql:']);
|
|
26
|
+
|
|
27
|
+
/**
|
|
28
|
+
* The received scheme is deliberately NOT named, which is the one place this refusal differs from
|
|
29
|
+
* every other "errors are instructions" case in the package. `new URL()` accepts a scheme-less
|
|
30
|
+
* string by reading the first token as the scheme, and that token is exactly the value this must
|
|
31
|
+
* not echo: `db.internal:5432/app` parses with `db.internal:` — the HOST — as its protocol, and
|
|
32
|
+
* `app:hunter2@db.internal/app`, the same typo copied one dashboard field over, parses with the
|
|
33
|
+
* USERNAME as its protocol. Naming "the scheme" therefore puts a host or a credential in the boot
|
|
34
|
+
* log AND the `--json` payload, where the logger has no key left to redact it by (see the block in
|
|
35
|
+
* the parse `catch` below). The REQUIRED scheme is a constant and carries the whole instruction, so
|
|
36
|
+
* nothing actionable is lost by withholding the received one; `describeValue` keeps the shape, so
|
|
37
|
+
* an empty variable is still told apart from a truncated one.
|
|
38
|
+
*/
|
|
39
|
+
const schemeUnsupported = (raw: string): DbError =>
|
|
40
|
+
new DbError({
|
|
41
|
+
code: 'X_DB_UNAVAILABLE',
|
|
42
|
+
cause:
|
|
43
|
+
'DATABASE_URL does not name a postgres:// or postgresql:// url: ' +
|
|
44
|
+
`received ${describeValue(raw)}`,
|
|
45
|
+
fix:
|
|
46
|
+
'set DATABASE_URL to postgres://user@host:5432/database — a value with no scheme parses ' +
|
|
47
|
+
'as a url whose scheme is its own first token — or run `x dev` to use the embedded PGlite',
|
|
48
|
+
});
|
|
49
|
+
|
|
15
50
|
export function connectionUrl(options: ConnectionUrlOptions, profile: PoolProfile): string {
|
|
16
51
|
const raw = options.url ?? process.env['DATABASE_URL'];
|
|
17
52
|
if (raw === undefined || raw === '') {
|
|
@@ -30,6 +65,13 @@ export function connectionUrl(options: ConnectionUrlOptions, profile: PoolProfil
|
|
|
30
65
|
// `maskedEnvValues`.
|
|
31
66
|
throw dbUnavailable(`DATABASE_URL is not a valid url: received ${describeValue(raw)}`, error);
|
|
32
67
|
}
|
|
68
|
+
// Screened here because `Bun.SQL` will not screen it: measured on bun 1.4.0, it reads
|
|
69
|
+
// `db.internal:5432/app` as host `db.internal`, port 5432, database `app` and opens a Postgres
|
|
70
|
+
// pool on it, so the first symptom is a connect failure at the first QUERY, in another process
|
|
71
|
+
// phase, worded by the driver and naming neither `DATABASE_URL` nor the missing scheme. Worse,
|
|
72
|
+
// `sqlite://./dev.db` succeeds outright on a different engine. A boot-time refusal is the only
|
|
73
|
+
// point at which the value is still nameable.
|
|
74
|
+
if (!POSTGRES_SCHEMES.has(url.protocol)) throw schemeUnsupported(raw);
|
|
33
75
|
// libpq `options` is the portable way to pin a statement timeout for every pooled connection —
|
|
34
76
|
// MERGED into the operator's own, never assigned over it, and emitted for every role including
|
|
35
77
|
// the two whose bound is 0. `set` here dropped a `?options=-c search_path=app` on `web`, `sync`,
|
package/src/errors.ts
CHANGED
|
@@ -25,6 +25,7 @@ export const DB_OWNED_ERROR_CODES = [
|
|
|
25
25
|
'X_DB_STATEMENT_TIMEOUT',
|
|
26
26
|
'X_DB_LOCK_TIMEOUT',
|
|
27
27
|
'X_DB_POOL_EXHAUSTED',
|
|
28
|
+
'X_DB_DRAIN_TIMEOUT',
|
|
28
29
|
'X_DB_DRIFT',
|
|
29
30
|
'X_MIGRATION_CONFLICT',
|
|
30
31
|
'X_MIGRATION_IRREVERSIBLE',
|
|
@@ -66,6 +67,7 @@ export const DB_ERROR_TITLES: Readonly<Record<DbOwnedErrorCode, string>> = {
|
|
|
66
67
|
X_DB_STATEMENT_TIMEOUT: 'the statement ran past its statement_timeout',
|
|
67
68
|
X_DB_LOCK_TIMEOUT: 'the statement waited past its lock_timeout',
|
|
68
69
|
X_DB_POOL_EXHAUSTED: 'no connection was available',
|
|
70
|
+
X_DB_DRAIN_TIMEOUT: 'the pool did not drain inside its shutdown budget',
|
|
69
71
|
X_DB_DRIFT: 'schema differs from migrations',
|
|
70
72
|
X_MIGRATION_CONFLICT: 'the migration ledger disagrees with this build',
|
|
71
73
|
X_MIGRATE_CONCURRENT: 'another migrator holds the migration lock',
|
|
@@ -238,6 +240,24 @@ export const driverError = (detail: string, sourceError: unknown): DbError => {
|
|
|
238
240
|
});
|
|
239
241
|
};
|
|
240
242
|
|
|
243
|
+
/**
|
|
244
|
+
* `close()` gave up waiting for the pool. TERMINAL, and deliberately not retryable: the pool is
|
|
245
|
+
* gone either way — `close()` clears the handle before it awaits — so a caller that retried would
|
|
246
|
+
* be closing a pool that no longer exists. What this reports is that connections were still held
|
|
247
|
+
* when the process stopped waiting, which is a fact about the shutdown an operator has to see.
|
|
248
|
+
*
|
|
249
|
+
* The alternative was to resolve quietly on the deadline, and that is the version that hides the
|
|
250
|
+
* bug: a drain that silently gave up looks exactly like a clean one, and the rows still in flight
|
|
251
|
+
* are lost with no line anywhere saying so.
|
|
252
|
+
*/
|
|
253
|
+
export const drainTimeout = (ms: number, role: string): DbError =>
|
|
254
|
+
new DbError({
|
|
255
|
+
code: 'X_DB_DRAIN_TIMEOUT',
|
|
256
|
+
cause: `the ${role} pool still held connections after ${String(ms)}ms, so close() stopped waiting`,
|
|
257
|
+
fix: `find the statement that will not finish — psql "$DATABASE_URL" -c "select pid, state, query from pg_stat_activity where state <> 'idle'" — or raise drainTimeoutMs in createPostgresClient({ profile }) for the ${role} role`,
|
|
258
|
+
meta: { drainTimeoutMs: ms, role },
|
|
259
|
+
});
|
|
260
|
+
|
|
241
261
|
/**
|
|
242
262
|
* The pool answered nothing inside `acquireTimeoutMs`. Distinct from the server's own `53300` and
|
|
243
263
|
* deliberately the same code: to a caller both mean "there was no connection for this unit of
|
package/src/generate.ts
CHANGED
|
@@ -22,6 +22,8 @@ import {
|
|
|
22
22
|
} from './introspect';
|
|
23
23
|
import { declaredIndexes } from './invariant-ddl';
|
|
24
24
|
import { migrationIrreversible } from './migration-errors';
|
|
25
|
+
import type { ReplicaIdentityInput } from './replica-identity';
|
|
26
|
+
import { replicaIdentityFullAfter, replicaIdentityPlan } from './replica-identity';
|
|
25
27
|
import type { MovedAside } from './retype-dependents';
|
|
26
28
|
import { moveDependentsAside } from './retype-dependents';
|
|
27
29
|
import { moveKeysAside, retypedColumns, retypedIn } from './retype-keys';
|
|
@@ -54,7 +56,13 @@ function columnClause(column: ColumnDescriptionLike): string {
|
|
|
54
56
|
return parts.join(' ');
|
|
55
57
|
}
|
|
56
58
|
|
|
57
|
-
|
|
59
|
+
/** No table subscribed to, which is what every caller outside `generateMigration` describes. */
|
|
60
|
+
const NONE: ReadonlySet<string> = new Set();
|
|
61
|
+
|
|
62
|
+
export function snapshotOf(
|
|
63
|
+
entities: readonly EntityDescriptionLike[],
|
|
64
|
+
replicaIdentityFull: ReadonlySet<string> = NONE,
|
|
65
|
+
): SchemaDescription {
|
|
58
66
|
const tables = [...entities]
|
|
59
67
|
.sort((a, b) => (a.table < b.table ? -1 : 1))
|
|
60
68
|
.map((entity): TableDescription => {
|
|
@@ -99,6 +107,9 @@ export function snapshotOf(entities: readonly EntityDescriptionLike[]): SchemaDe
|
|
|
99
107
|
// read as "nothing recorded" so the next generation adds the constraints the database is
|
|
100
108
|
// genuinely missing — the rule `using` and `generated` already state one field up.
|
|
101
109
|
...(checks.length === 0 ? {} : { checks }),
|
|
110
|
+
// The same rule once more: `true` or absent, never `false`. This is what makes the ALTER
|
|
111
|
+
// beside it a one-time statement rather than a line every `x db gen` writes again.
|
|
112
|
+
...(replicaIdentityFull.has(entity.table) ? { replicaIdentityFull: true as const } : {}),
|
|
102
113
|
};
|
|
103
114
|
});
|
|
104
115
|
return { tables };
|
|
@@ -227,6 +238,17 @@ export interface GenerateOptions {
|
|
|
227
238
|
readonly now?: Date | undefined;
|
|
228
239
|
/** Allow a DROP COLUMN whose down cannot restore the data. `x db gen --allow-destructive`. */
|
|
229
240
|
readonly allowDestructive?: boolean | undefined;
|
|
241
|
+
/**
|
|
242
|
+
* Tables a live query subscribes to, so logical replication carries the old row —
|
|
243
|
+
* `@ultimat3/realtime` refuses a subscription without it. Absent or empty emits nothing.
|
|
244
|
+
*
|
|
245
|
+
* A parameter and never an `EntityDescriptionLike` field: which tables need it is DECLARED by
|
|
246
|
+
* each `live: true` query's `subscribes:`, never derived — the relation is a literal inside the
|
|
247
|
+
* query's `sql:` callback, which nothing can invoke without valid input. This package is tier 1
|
|
248
|
+
* and can see neither the manifest nor `@ultimat3/query`, so `@ultimat3/cli`'s `db-generate.ts`
|
|
249
|
+
* is the one caller that can answer it.
|
|
250
|
+
*/
|
|
251
|
+
readonly replicaIdentityFull?: readonly string[] | undefined;
|
|
230
252
|
}
|
|
231
253
|
|
|
232
254
|
export interface GeneratedMigration {
|
|
@@ -275,6 +297,9 @@ export function generateMigration(options: GenerateOptions): GeneratedMigration
|
|
|
275
297
|
// of one key can move in two different entities' diffs (`retype-keys.ts`).
|
|
276
298
|
const preAlters: Plan = { up: [], down: [] };
|
|
277
299
|
const wanted = new Set(options.entities.map((entity) => entity.table));
|
|
300
|
+
// The tables this run brings into being, read by the replica-identity arm below: their `down` is
|
|
301
|
+
// already `drop table`, so reverting the identity ahead of it performs nothing.
|
|
302
|
+
const created = new Set<string>();
|
|
278
303
|
|
|
279
304
|
const doomed = new Set(
|
|
280
305
|
current.tables.filter((table) => !wanted.has(table.name)).map((table) => table.name),
|
|
@@ -291,6 +316,7 @@ export function generateMigration(options: GenerateOptions): GeneratedMigration
|
|
|
291
316
|
if (live === undefined) {
|
|
292
317
|
plan.up.push(...createTable(entity));
|
|
293
318
|
plan.down.push(`drop table ${identifier(entity.table).text};`);
|
|
319
|
+
created.add(entity.table);
|
|
294
320
|
continue;
|
|
295
321
|
}
|
|
296
322
|
diffTable(entity, live, plan, retypedIn(retyped, entity.table));
|
|
@@ -337,6 +363,18 @@ export function generateMigration(options: GenerateOptions): GeneratedMigration
|
|
|
337
363
|
plan.up.push(...constraints.up);
|
|
338
364
|
plan.down.push(...constraints.down);
|
|
339
365
|
|
|
366
|
+
// Dead last in `up`, and it is the only placement that is right for every arm: the table has to
|
|
367
|
+
// exist, and a `create table` in this same migration is the reason it might not. It is ordered
|
|
368
|
+
// against nothing else — replica identity constrains no column, no index and no constraint — so
|
|
369
|
+
// the end is where it can never be read as depending on a statement above it.
|
|
370
|
+
const replicaIdentity: ReplicaIdentityInput = {
|
|
371
|
+
wanted: options.replicaIdentityFull,
|
|
372
|
+
declared: wanted,
|
|
373
|
+
created,
|
|
374
|
+
current,
|
|
375
|
+
};
|
|
376
|
+
replicaIdentityPlan(plan, replicaIdentity);
|
|
377
|
+
|
|
340
378
|
const id = `${migrationStamp(options.now ?? systemClock.now())}_${slugify(options.name)}`;
|
|
341
379
|
// At the TOP of `up`, so what is MISSING is the first thing read — and a line comment, so it is
|
|
342
380
|
// noise to every reader that matters: `statementsOf` drops a chunk of comments alone,
|
|
@@ -362,7 +400,7 @@ export function generateMigration(options: GenerateOptions): GeneratedMigration
|
|
|
362
400
|
// FRONT here precisely so reversal puts it last — a key is added back only once both of its
|
|
363
401
|
// ends have been retyped back, which is every other statement in the script.
|
|
364
402
|
down: [...preAlters.down, ...plan.down].reverse().join('\n'),
|
|
365
|
-
snapshot: snapshotOf(options.entities),
|
|
403
|
+
snapshot: snapshotOf(options.entities, replicaIdentityFullAfter(replicaIdentity)),
|
|
366
404
|
destructive: isDestructive(up),
|
|
367
405
|
unrendered,
|
|
368
406
|
};
|
package/src/introspect.ts
CHANGED
|
@@ -105,6 +105,22 @@ export interface TableDescription {
|
|
|
105
105
|
* read one instead of reporting every declared constraint as missing.
|
|
106
106
|
*/
|
|
107
107
|
readonly checkNames?: readonly string[] | undefined;
|
|
108
|
+
/**
|
|
109
|
+
* That a migration set `replica identity full` on this table — what `@ultimat3/realtime` requires
|
|
110
|
+
* of every table a live query subscribes to, and the only reason this generator emits it.
|
|
111
|
+
*
|
|
112
|
+
* `true` or absent, never `false`, and the literal type is what enforces it. Absent means
|
|
113
|
+
* *nothing recorded*: a sidecar written before this field existed, exactly as `checks` absent
|
|
114
|
+
* means "no constraint was recorded" rather than "none is declared". Writing `false` onto every
|
|
115
|
+
* table an app has would rewrite every sidecar in the tree on the next `x db gen` for a fact that
|
|
116
|
+
* was already true — the argument `IndexDescription.using` makes about `btree`.
|
|
117
|
+
*
|
|
118
|
+
* `introspect()` never answers it. The catalog's half is `pg_class.relreplident`, which
|
|
119
|
+
* `@ultimat3/realtime`'s preflight already reads at the only moment it matters; a second reader
|
|
120
|
+
* here would let a `diffSchema` compare a declaration against a catalog value, which is the
|
|
121
|
+
* mistake `checks` and `checkNames` exist as two fields to prevent.
|
|
122
|
+
*/
|
|
123
|
+
readonly replicaIdentityFull?: true | undefined;
|
|
108
124
|
}
|
|
109
125
|
|
|
110
126
|
export interface SchemaDescription {
|
package/src/pool-profile.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
// Single responsibility: the
|
|
1
|
+
// Single responsibility: the six numbers a Postgres pool runs on — the per-role defaults, the one
|
|
2
2
|
// environment override an operator may layer over them, and the screen every resolved profile
|
|
3
3
|
// passes. Split from `client.ts`, which now owns connecting and nothing about sizing.
|
|
4
4
|
|
|
@@ -25,6 +25,24 @@ export interface PoolProfile {
|
|
|
25
25
|
* kills the pod, and the replacement inherits the same saturated database.
|
|
26
26
|
*/
|
|
27
27
|
readonly acquireTimeoutMs: number;
|
|
28
|
+
/**
|
|
29
|
+
* How long `close()` may wait for the pool to drain before `X_DB_DRAIN_TIMEOUT`. 0 waits forever.
|
|
30
|
+
*
|
|
31
|
+
* **A drain that cannot finish is the failure this bounds, and it is not hypothetical.** Measured
|
|
32
|
+
* against a real Postgres, three runs per case: `Bun.SQL`'s `end()` waits on an outstanding
|
|
33
|
+
* RESERVED connection and never stops waiting — 3 of 3 on Bun 1.3.14 *and* 3 of 3 on 1.4.0, with
|
|
34
|
+
* no database outage involved at all. Once that connection's backend has been terminated it
|
|
35
|
+
* becomes a race, which 1.3.14 loses 3 of 3 and 1.4.0 loses 1 of 3. So the runtime is not the
|
|
36
|
+
* variable; an unbounded await is (#394).
|
|
37
|
+
*
|
|
38
|
+
* What that cost, before this: `releaseQueue` awaits `db.close()`, so a role whose database went
|
|
39
|
+
* away mid-shutdown never finished shutting down. A container that will not drain is drained by
|
|
40
|
+
* SIGKILL, and the operator's only signal is a pod that took its full termination grace period.
|
|
41
|
+
*
|
|
42
|
+
* `migrate` and `replicator` wait forever, deliberately, for `acquireTimeoutMs`' reason: a
|
|
43
|
+
* run-once role cutting off its own session mid-statement is worse than a slow exit.
|
|
44
|
+
*/
|
|
45
|
+
readonly drainTimeoutMs: number;
|
|
28
46
|
}
|
|
29
47
|
|
|
30
48
|
/** Sized per role because the failure modes differ: RPS bursts vs. queue depth vs. run-once. */
|
|
@@ -35,6 +53,7 @@ export const POOL_PROFILES = Object.freeze<Record<Role, PoolProfile>>({
|
|
|
35
53
|
idleTimeoutMs: 30_000,
|
|
36
54
|
lockTimeoutMs: 0,
|
|
37
55
|
acquireTimeoutMs: 5_000,
|
|
56
|
+
drainTimeoutMs: 5_000,
|
|
38
57
|
},
|
|
39
58
|
sync: {
|
|
40
59
|
max: 10,
|
|
@@ -42,6 +61,7 @@ export const POOL_PROFILES = Object.freeze<Record<Role, PoolProfile>>({
|
|
|
42
61
|
idleTimeoutMs: 60_000,
|
|
43
62
|
lockTimeoutMs: 0,
|
|
44
63
|
acquireTimeoutMs: 5_000,
|
|
64
|
+
drainTimeoutMs: 5_000,
|
|
45
65
|
},
|
|
46
66
|
worker: {
|
|
47
67
|
max: 8,
|
|
@@ -49,6 +69,7 @@ export const POOL_PROFILES = Object.freeze<Record<Role, PoolProfile>>({
|
|
|
49
69
|
idleTimeoutMs: 30_000,
|
|
50
70
|
lockTimeoutMs: 0,
|
|
51
71
|
acquireTimeoutMs: 10_000,
|
|
72
|
+
drainTimeoutMs: 15_000,
|
|
52
73
|
},
|
|
53
74
|
scheduler: {
|
|
54
75
|
max: 2,
|
|
@@ -56,6 +77,7 @@ export const POOL_PROFILES = Object.freeze<Record<Role, PoolProfile>>({
|
|
|
56
77
|
idleTimeoutMs: 60_000,
|
|
57
78
|
lockTimeoutMs: 0,
|
|
58
79
|
acquireTimeoutMs: 10_000,
|
|
80
|
+
drainTimeoutMs: 5_000,
|
|
59
81
|
},
|
|
60
82
|
// `migrate` waits: its pool is `max: 1` and the advisory-lock pin holds it for the whole run, so
|
|
61
83
|
// a deadline here would refuse the migration's own session. The wait that needed bounding is the
|
|
@@ -66,6 +88,7 @@ export const POOL_PROFILES = Object.freeze<Record<Role, PoolProfile>>({
|
|
|
66
88
|
idleTimeoutMs: 10_000,
|
|
67
89
|
lockTimeoutMs: 3_000,
|
|
68
90
|
acquireTimeoutMs: 0,
|
|
91
|
+
drainTimeoutMs: 0,
|
|
69
92
|
},
|
|
70
93
|
replicator: {
|
|
71
94
|
max: 4,
|
|
@@ -73,6 +96,7 @@ export const POOL_PROFILES = Object.freeze<Record<Role, PoolProfile>>({
|
|
|
73
96
|
idleTimeoutMs: 60_000,
|
|
74
97
|
lockTimeoutMs: 0,
|
|
75
98
|
acquireTimeoutMs: 0,
|
|
99
|
+
drainTimeoutMs: 0,
|
|
76
100
|
},
|
|
77
101
|
});
|
|
78
102
|
|
|
@@ -99,13 +123,13 @@ export function poolMaxFromEnv(): Partial<PoolProfile> {
|
|
|
99
123
|
}
|
|
100
124
|
|
|
101
125
|
/**
|
|
102
|
-
* The
|
|
126
|
+
* The six numbers a pool runs on, screened on the MERGED profile — an override is spread over a
|
|
103
127
|
* role default the caller never restated, so the resolved object is the only one that can be
|
|
104
128
|
* judged. Every one of them is a plausible `Number(process.env.…)`, which is `NaN` for an unset
|
|
105
|
-
* variable and not nullish, so `??` and the spread both keep it. None of the
|
|
129
|
+
* variable and not nullish, so `??` and the spread both keep it. None of the six then fails
|
|
106
130
|
* loudly: `idleTimeout: NaN` goes to `Bun.SQL`, `statement_timeout=NaN` goes into the libpq
|
|
107
131
|
* options string for the SERVER to reject on connect, and a timer given `NaN` fires at 1ms in this
|
|
108
|
-
* Bun — so a pool with free connections reports itself exhausted. `0` stays legal for the
|
|
132
|
+
* Bun — so a pool with free connections reports itself exhausted. `0` stays legal for the five
|
|
109
133
|
* budgets that document it as "no bound"; `max` is at least one connection, or nothing can run.
|
|
110
134
|
*/
|
|
111
135
|
export function assertPoolProfile(profile: PoolProfile): PoolProfile {
|
|
@@ -121,5 +145,6 @@ export function assertPoolProfile(profile: PoolProfile): PoolProfile {
|
|
|
121
145
|
whole('idleTimeoutMs', profile.idleTimeoutMs, 0);
|
|
122
146
|
whole('lockTimeoutMs', profile.lockTimeoutMs, 0);
|
|
123
147
|
whole('acquireTimeoutMs', profile.acquireTimeoutMs, 0);
|
|
148
|
+
whole('drainTimeoutMs', profile.drainTimeoutMs, 0);
|
|
124
149
|
return profile;
|
|
125
150
|
}
|
package/src/pool-reserve.ts
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
// and giving back a reservation that arrives after the deadline has passed. Split from `client.ts`,
|
|
3
3
|
// which now asks for a pin rather than owning what "waited too long" means.
|
|
4
4
|
|
|
5
|
-
import type
|
|
5
|
+
import { type BunSqlDriver, type BunSqlReserved, releaseReserved } from './bun-sql';
|
|
6
6
|
import { poolAcquireTimeout } from './errors';
|
|
7
7
|
import type { PoolProfile } from './pool-profile';
|
|
8
8
|
|
|
@@ -42,7 +42,7 @@ export async function reserveWithin(
|
|
|
42
42
|
// Attached unconditionally so a rejection arriving after we gave up is handled, not unhandled.
|
|
43
43
|
void pending.then(
|
|
44
44
|
(late) => {
|
|
45
|
-
if (expired) late
|
|
45
|
+
if (expired) releaseReserved(late);
|
|
46
46
|
},
|
|
47
47
|
() => undefined,
|
|
48
48
|
);
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
// Single responsibility: the `alter table … replica identity full` a live query needs, emitted once.
|
|
2
|
+
// `@ultimat3/realtime` refuses a subscription to a table without it — logical replication carries no
|
|
3
|
+
// old row on an UPDATE, so no patch can be computed — and which tables need it is a tier-3 fact no
|
|
4
|
+
// entity carries, so it arrives as `GenerateOptions.replicaIdentityFull` and is recorded on the
|
|
5
|
+
// snapshot: the record is what stops the statement being re-emitted on every `x db gen`.
|
|
6
|
+
|
|
7
|
+
import type { Plan } from './foreign-key-plan';
|
|
8
|
+
import { findTable, type SchemaDescription } from './introspect';
|
|
9
|
+
import { identifier } from './sql';
|
|
10
|
+
|
|
11
|
+
export interface ReplicaIdentityInput {
|
|
12
|
+
/** Tables a live query subscribes to. Absent or empty emits nothing and reverts nothing. */
|
|
13
|
+
readonly wanted: readonly string[] | undefined;
|
|
14
|
+
/** The tables this migration leaves standing — the ALTER may name no other. */
|
|
15
|
+
readonly declared: ReadonlySet<string>;
|
|
16
|
+
/** Of those, the ones it CREATES: their whole `down` is already `drop table`. */
|
|
17
|
+
readonly created: ReadonlySet<string>;
|
|
18
|
+
/** What migrations already recorded — `expectedSchema(migrations, ledger)`. */
|
|
19
|
+
readonly current: SchemaDescription;
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
/**
|
|
23
|
+
* Whether the recorded schema already says this table carries it.
|
|
24
|
+
*
|
|
25
|
+
* Absent — never `false` — is a sidecar written before the field existed: it declares nothing, so
|
|
26
|
+
* the ALTER is emitted once more against a table that may already have it, which Postgres accepts.
|
|
27
|
+
* The opposite reading would be "recorded as not full", and a snapshot that predates the field
|
|
28
|
+
* cannot mean that. `TableDescription.checks` states the same rule for the same reason.
|
|
29
|
+
*/
|
|
30
|
+
export function recordsReplicaIdentityFull(current: SchemaDescription, table: string): boolean {
|
|
31
|
+
return findTable(current, table)?.replicaIdentityFull === true;
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
/** Which tables to record on the snapshot — sorted only by the caller's own table order. */
|
|
35
|
+
function pending(input: ReplicaIdentityInput): readonly string[] {
|
|
36
|
+
// Deduplicated and sorted, so the same live-query set generates the same bytes whatever order the
|
|
37
|
+
// manifest walked its queries in: a diff that moves a line is a diff an author has to read.
|
|
38
|
+
return [...new Set(input.wanted ?? [])]
|
|
39
|
+
.filter((table) => input.declared.has(table))
|
|
40
|
+
.filter((table) => !recordsReplicaIdentityFull(input.current, table))
|
|
41
|
+
.sort();
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
/**
|
|
45
|
+
* The tables carrying it once this migration has applied — what `snapshotOf` records.
|
|
46
|
+
*
|
|
47
|
+
* The UNION with what is already recorded, never this run's set alone. A caller that passes no
|
|
48
|
+
* live-query set (`x db gen` from a command that never learned about one) must not silently erase
|
|
49
|
+
* the fact from the sidecar, or the very next run emits the ALTER again on a table that has it.
|
|
50
|
+
*/
|
|
51
|
+
export function replicaIdentityFullAfter(input: ReplicaIdentityInput): ReadonlySet<string> {
|
|
52
|
+
const after = new Set<string>();
|
|
53
|
+
for (const table of input.declared) {
|
|
54
|
+
if (recordsReplicaIdentityFull(input.current, table)) after.add(table);
|
|
55
|
+
}
|
|
56
|
+
for (const table of pending(input)) after.add(table);
|
|
57
|
+
return after;
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
/**
|
|
61
|
+
* Additive only, and never destructive. `alter table … replica identity full` widens what logical
|
|
62
|
+
* replication carries; it drops no row, rewrites no column and matches none of `destructive.ts`'s
|
|
63
|
+
* four rules, so the migration it lands in needs no `-- destructive: true` marker.
|
|
64
|
+
*
|
|
65
|
+
* A name matching no entity is SKIPPED rather than refused: the list is derived from the manifest's
|
|
66
|
+
* live queries, and a query whose entity has been deleted is an app fault the generator cannot
|
|
67
|
+
* repair — emitting the statement anyway would be `42P01` at `ROLE=migrate`, which is the one place
|
|
68
|
+
* this package refuses to put a fault.
|
|
69
|
+
*
|
|
70
|
+
* Nothing is ever reverted. A table dropping out of the live-query set keeps the identity it has:
|
|
71
|
+
* the option is optional, so "absent" and "no live query subscribes any more" are the same value,
|
|
72
|
+
* and reading them alike would let a caller that never passes the option turn off replication for
|
|
73
|
+
* every subscribed table in the app.
|
|
74
|
+
*/
|
|
75
|
+
export function replicaIdentityPlan(plan: Plan, input: ReplicaIdentityInput): void {
|
|
76
|
+
for (const table of pending(input)) {
|
|
77
|
+
const name = identifier(table).text;
|
|
78
|
+
plan.up.push(`alter table ${name} replica identity full;`);
|
|
79
|
+
// A table this migration created is dropped by its own `down`; a second statement ahead of that
|
|
80
|
+
// is a line an author reads and nothing performs.
|
|
81
|
+
if (input.created.has(table)) continue;
|
|
82
|
+
plan.down.push(`alter table ${name} replica identity default;`);
|
|
83
|
+
}
|
|
84
|
+
}
|
package/src/snapshot-parse.ts
CHANGED
|
@@ -112,11 +112,20 @@ function tableOf(value: unknown): TableDescription | undefined {
|
|
|
112
112
|
// Absent, never `[]`. A sidecar written before constraints were recorded says nothing about
|
|
113
113
|
// them, and reading that as "this table declares none" would drop every invariant an app has
|
|
114
114
|
// already generated instead of adding the ones its database is missing.
|
|
115
|
+
// `true` or nothing. `false` is accepted and NORMALISED away rather than rejecting the file: it
|
|
116
|
+
// is a hand-edit meaning exactly what absence means, and discarding the whole snapshot over it
|
|
117
|
+
// makes `x db gen` refuse with `X_MIGRATION_SNAPSHOT_MISSING` on a sidecar that says nothing
|
|
118
|
+
// wrong. Any other value is garbage and takes the file with it, like every field above.
|
|
119
|
+
const identity = value['replicaIdentityFull'];
|
|
120
|
+
if (!(identity === undefined || bool(identity))) return undefined;
|
|
121
|
+
const replica = identity === true ? { replicaIdentityFull: true as const } : {};
|
|
115
122
|
const raw = value['checks'];
|
|
116
|
-
if (raw === undefined)
|
|
123
|
+
if (raw === undefined) {
|
|
124
|
+
return { schema, name, columns, primaryKey, indexes, foreignKeys, ...replica };
|
|
125
|
+
}
|
|
117
126
|
const checks = all(raw, check);
|
|
118
127
|
if (checks === undefined) return undefined;
|
|
119
|
-
return { schema, name, columns, primaryKey, indexes, foreignKeys, checks };
|
|
128
|
+
return { schema, name, columns, primaryKey, indexes, foreignKeys, checks, ...replica };
|
|
120
129
|
}
|
|
121
130
|
|
|
122
131
|
/**
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
// Single responsibility: the pooled driver's statement funnel — one statement on one handle, every
|
|
2
|
+
// driver failure typed on the way out, and the installed `StatementObserver` notified on both
|
|
3
|
+
// settle paths. Split from `client.ts` at the file-size rule; `pglite.ts` holds the mirror pair
|
|
4
|
+
// (`send`/`statement`) for the embedded driver, and the two must keep answering the same way.
|
|
5
|
+
|
|
6
|
+
import { encodeArrayParameters } from './array-parameter';
|
|
7
|
+
import { statementAttribution } from './attribution';
|
|
8
|
+
import type { BunSqlDriver } from './bun-sql';
|
|
9
|
+
import { driverError } from './errors';
|
|
10
|
+
import { expectedQueryLoopReason } from './expected-loop';
|
|
11
|
+
import { statementObserver } from './observe';
|
|
12
|
+
import type { SqlFragment } from './sql';
|
|
13
|
+
import { withStatementSpan } from './statement-span';
|
|
14
|
+
|
|
15
|
+
export function rowsOf<T>(result: unknown): readonly T[] {
|
|
16
|
+
return Array.isArray(result) ? (result as readonly T[]) : [];
|
|
17
|
+
}
|
|
18
|
+
|
|
19
|
+
// The command tag only when it counted something, exactly like `rowsOf` in `pglite.ts` — one rule
|
|
20
|
+
// across both drivers, so `execute()` and the observer's event cannot answer differently for the
|
|
21
|
+
// same statement depending on which database is behind them. A driver that tags a read `0` while
|
|
22
|
+
// returning rows would otherwise report 0 here and the row count there.
|
|
23
|
+
export function affectedBy(result: unknown): number {
|
|
24
|
+
if (!Array.isArray(result)) return 0;
|
|
25
|
+
const count = (result as { count?: unknown }).count;
|
|
26
|
+
return typeof count === 'number' && count > 0 ? count : result.length;
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
/** The send itself: one statement on one handle, every driver failure typed on the way out. */
|
|
30
|
+
async function sendOn(
|
|
31
|
+
driver: Pick<BunSqlDriver, 'unsafe'>,
|
|
32
|
+
fragment: SqlFragment,
|
|
33
|
+
): Promise<unknown> {
|
|
34
|
+
try {
|
|
35
|
+
// `encodeArrayParameters`, never `fragment.values` raw: `Bun.SQL` joins a JS array's elements
|
|
36
|
+
// with commas, so every `any($n::T[])` in this framework sent Postgres a malformed literal and
|
|
37
|
+
// failed — the worker's claim loop among them (#384). One encoder here rather than one import
|
|
38
|
+
// per call site, because this is the only place this driver's `unsafe` is called.
|
|
39
|
+
return await driver.unsafe(fragment.text, encodeArrayParameters(fragment.values));
|
|
40
|
+
} catch (error) {
|
|
41
|
+
// `driverError`, not `dbUnavailable`: the SQLSTATE has always been on this error and nothing
|
|
42
|
+
// read it, so a `23505` from two clicks racing a signup told the operator the database was
|
|
43
|
+
// unreachable and paged on-call for an outage that never happened. Everything the table does
|
|
44
|
+
// not classify is still `X_DB_UNAVAILABLE`, byte for byte.
|
|
45
|
+
throw driverError(`statement failed: ${fragment.text.slice(0, 120)}`, error);
|
|
46
|
+
}
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
/**
|
|
50
|
+
* The funnel — pooled and pinned statements both arrive here, which is why the observer hangs
|
|
51
|
+
* off this one function and nowhere else. Uninstalled it costs one property read and one
|
|
52
|
+
* branch: no clock read, no span, no event object, and `sendOn` receives exactly the call `runOn`
|
|
53
|
+
* made before the seam existed (axiom 6).
|
|
54
|
+
*/
|
|
55
|
+
export async function runOn(
|
|
56
|
+
driver: Pick<BunSqlDriver, 'unsafe'>,
|
|
57
|
+
fragment: SqlFragment,
|
|
58
|
+
): Promise<unknown> {
|
|
59
|
+
const observer = statementObserver();
|
|
60
|
+
if (observer === undefined) return sendOn(driver, fragment);
|
|
61
|
+
// Read here, not by the consumer: the scope is gone by the time a per-request detector judges
|
|
62
|
+
// what it collected, so the reason has to be captured with the statement it defends.
|
|
63
|
+
const expected = expectedQueryLoopReason();
|
|
64
|
+
// Same moment, same argument: `postgresRepo` is several frames and a microtask above this one,
|
|
65
|
+
// and what it knows — the entity and the operation — is what turns fifty identical `select`s
|
|
66
|
+
// into "50× findById on members". Absent for hand-written SQL, a migration, a health probe.
|
|
67
|
+
const attribution = statementAttribution();
|
|
68
|
+
const started = performance.now();
|
|
69
|
+
let result: unknown;
|
|
70
|
+
try {
|
|
71
|
+
// The span wraps the send and nothing else, so its duration is the statement's and the
|
|
72
|
+
// observer's own work is not charged to the database.
|
|
73
|
+
result = await withStatementSpan(fragment.text, () => sendOn(driver, fragment));
|
|
74
|
+
} catch (error) {
|
|
75
|
+
// A statement that failed is still a statement: fifty identical timeouts are an N+1 of
|
|
76
|
+
// timeouts. The error is already `X_DB_UNAVAILABLE`, so the event carries what the caller
|
|
77
|
+
// is about to be thrown — and an observer that throws here replaces it, which is why
|
|
78
|
+
// `observe.ts` says a reporting-only observer must not throw.
|
|
79
|
+
observer.onStatement({
|
|
80
|
+
text: fragment.text,
|
|
81
|
+
values: fragment.values,
|
|
82
|
+
durationMs: performance.now() - started,
|
|
83
|
+
rows: 0,
|
|
84
|
+
error,
|
|
85
|
+
attribution,
|
|
86
|
+
expected,
|
|
87
|
+
});
|
|
88
|
+
throw error;
|
|
89
|
+
}
|
|
90
|
+
// Outside the `try` deliberately: a throw from `onStatement` is the observer's, not the
|
|
91
|
+
// database's, and catching it above would report a statement that succeeded as failed.
|
|
92
|
+
observer.onStatement({
|
|
93
|
+
text: fragment.text,
|
|
94
|
+
values: fragment.values,
|
|
95
|
+
durationMs: performance.now() - started,
|
|
96
|
+
rows: affectedBy(result),
|
|
97
|
+
attribution,
|
|
98
|
+
expected,
|
|
99
|
+
});
|
|
100
|
+
return result;
|
|
101
|
+
}
|
package/src/ungeneratable.ts
CHANGED
|
@@ -59,6 +59,17 @@ export const GENERATABLE_FORMS: readonly GeneratableForm[] = [
|
|
|
59
59
|
name: 'alter column drop expression',
|
|
60
60
|
pattern: /^alter\s+table\s[\s\S]*?\balter\s+column\s[\s\S]*?\bdrop\s+expression\b/,
|
|
61
61
|
},
|
|
62
|
+
// The form the doc block above calls "the statement that started this", finally on the list:
|
|
63
|
+
// `GenerateOptions.replicaIdentityFull` emits it `As of 2026-08-26`, so without this entry the
|
|
64
|
+
// rail reports SQL the generator itself just wrote. Covers `full` and `default` in one phrase —
|
|
65
|
+
// the down side is as generated as the up side.
|
|
66
|
+
{
|
|
67
|
+
name: 'alter table … replica identity full/default',
|
|
68
|
+
// `full` and `default` ONLY — the two modes `replicaIdentityPlan` emits. `using index` and
|
|
69
|
+
// `nothing` are hand-written configuration no entity declares, and admitting them here would
|
|
70
|
+
// read them as generatable and let a squash discard the replication setup in silence.
|
|
71
|
+
pattern: /^alter\s+table\s[\s\S]*?\breplica\s+identity\s+(?:full|default)\b/,
|
|
72
|
+
},
|
|
62
73
|
];
|
|
63
74
|
|
|
64
75
|
/**
|