@ultimat3/db 22.15.0 → 23.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +39 -1
- package/README.md +81 -0
- package/package.json +5 -3
- package/src/bun-sql.ts +12 -0
- package/src/catalog-fold.ts +113 -0
- package/src/catalog-objects.ts +200 -0
- package/src/catalog-relations.ts +184 -0
- package/src/catalog.ts +166 -0
- package/src/client.ts +57 -9
- package/src/drift-findings.ts +4 -1
- package/src/dump-drift.ts +142 -0
- package/src/errors.ts +2 -0
- package/src/index.ts +4 -0
- package/src/introspect-catalog.ts +154 -0
- package/src/listen.ts +62 -0
- package/src/object-drift.ts +105 -0
- package/src/pglite-extensions.ts +112 -0
- package/src/pglite-package.ts +11 -0
- package/src/pglite-snapshot.ts +121 -0
- package/src/pglite.ts +133 -10
- package/src/pool-gauge.ts +70 -0
- package/src/schema-dump-entry.ts +39 -0
- package/src/schema-dump-table.ts +72 -0
- package/src/schema-dump.ts +192 -0
- package/src/schema-load.ts +119 -0
- package/src/sqlstate.ts +3 -0
package/CLAUDE.md
CHANGED
|
@@ -14,7 +14,7 @@ Tier 1 — it imports `@ultimat3/core` and nothing else. That placement is load-
|
|
|
14
14
|
| SQLSTATE | one reader, `sqlState()` (`sqlstate.ts`). Never read `error.code` for a SQLSTATE |
|
|
15
15
|
| Reading a caught value | `renderThrowable()` from core (`checkDb` backs `/readyz`) |
|
|
16
16
|
| Errors | subclass `DbError`; never `throw new Error` in source. A test simulating a database failure throws `dbUnavailable()`; one simulating the caller's body failing throws a bare `Error` on purpose |
|
|
17
|
-
| New code |
|
|
17
|
+
| New code | `bun run new-error-code <CODE> --package db --title '…' --fix '…'` writes `DB_OWNED_ERROR_CODES`, `DB_ERROR_TITLES` and the wiki row together; then add it to `errors.test.ts`'s pinned list. The constructor lives where its imports allow (`migration-errors.ts`, `invariant-errors.ts`, `drift-errors.ts`, `dump-drift.ts`); `src/index.ts` re-exports all |
|
|
18
18
|
| A value ambient across an `await` | `asyncContext<T>(subject)` from core — never `new AsyncLocalStorage`. Three scopes: `transaction.ts`, `attribution.ts`, `expected-loop.ts` |
|
|
19
19
|
| Exports | explicit in `src/index.ts`; no `export *` |
|
|
20
20
|
| Files | < 200 LOC (the `packages/db/src/**/*.ts` path instruction), one responsibility, `kebab-case.ts`, test beside source |
|
|
@@ -58,7 +58,14 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
|
|
|
58
58
|
`drainTimeoutMs: 0` sends no option; the verdict is elapsed time on `performance.now()`
|
|
59
59
|
(`X_DB_DRAIN_TIMEOUT`). `pool-drain.test.ts`, `pool-drain.live.test.ts`. `close()` clears the cached
|
|
60
60
|
driver before awaiting the teardown.
|
|
61
|
+
- **`client.listen` is ONE session beside the pool** (`listen.ts`; `Bun.SQL.listen`, PGlite's
|
|
62
|
+
`listen` under a turn), never a reserved pin. `onListening` fires on every re-dial; a channel is
|
|
63
|
+
refused unless it is a plain identifier. `listen.test.ts`, `listen.live.test.ts`.
|
|
61
64
|
- `execute()` trusts the command tag only when `> 0`, in both drivers (`rowsOf`, `affectedBy`).
|
|
65
|
+
- **`pool-gauge.ts` derives `db_pool_max` / `db_pool_in_use` / `db_pool_waiting` from DEMAND** —
|
|
66
|
+
`Bun.SQL` publishes no occupancy. `client.ts` is the one counter: `run()` from send to settle, a
|
|
67
|
+
pin from the ask to its (idempotent) release; a statement ON a pin is not counted again. Declared
|
|
68
|
+
on the first tracked pool, never at import. `pool-gauge.test.ts`.
|
|
62
69
|
- **`client.ts` connects, holds the client and the ambient `db()`**, and opens no socket at import;
|
|
63
70
|
`pool-profile.ts`, `connection-url.ts`, `bun-sql.ts`, `pool-reserve.ts`, `db-health.ts` (`checkDb`)
|
|
64
71
|
and `statement-funnel.ts` (`sendOn`/`runOn`) hold the rest.
|
|
@@ -220,6 +227,37 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
|
|
|
220
227
|
- `drift-findings.ts` holds every `DriftDifference` constructor and `DriftKind`; `drift.ts` keeps the
|
|
221
228
|
comparisons.
|
|
222
229
|
|
|
230
|
+
## The schema dump
|
|
231
|
+
|
|
232
|
+
- **Its own entry: `@ultimat3/db/schema-dump`** (`schema-dump-entry.ts`). The barrel is in every
|
|
233
|
+
role's boot graph and must evaluate none of this family — `schema-dump-entry.test.ts`.
|
|
234
|
+
- **Two readings of one catalog, never mixed.** `introspect()` → `SchemaDescription`, the entity
|
|
235
|
+
vocabulary a snapshot is diffed in. `introspectCatalog()` (`catalog.ts`; queries in `catalog-relations.ts` and
|
|
236
|
+
`catalog-objects.ts`; the pure fold in `catalog-fold.ts`) → `CatalogDescription`, Postgres' own `pg_get_*def` text, compared only to
|
|
237
|
+
itself. A catalog spelling on a `SchemaDescription` field is the `checks`/`checkNames` mistake.
|
|
238
|
+
- **Sorted in JS** (`byCodeUnit`), never `order by` and never `localeCompare`: collations differ.
|
|
239
|
+
- **`renderSchemaDump()` is pure**; `schema-dump-table.ts` spells one table. `quoted()` escapes any
|
|
240
|
+
catalog name — `identifier()` refuses whitespace and `"`, which a migration may have created.
|
|
241
|
+
- **`x_` owner → `framework/` twin** (`FRAMEWORK_TABLE_PREFIX`). No list is handed in.
|
|
242
|
+
- **What cannot be rendered is named** (`unrenderedRows`, a closed list → `unrendered.sql`). A kind
|
|
243
|
+
rendered later leaves that list in the same diff. Never render a partition as a plain table.
|
|
244
|
+
- **`loadSchemaDump()`**: one transaction, `check_function_bodies` off, a savepoint per file, and
|
|
245
|
+
only `42P01`/`42883`/`42704` are retried. Its refusal is `X_SCHEMA_DUMP_DRIFT` (`dump-drift.ts`).
|
|
246
|
+
- **`object-drift.ts`**: `unexpectedObjects(live, expected)` compares IDENTITY (kind, table, name,
|
|
247
|
+
arguments). The live side is the operator's Postgres, the expected side a PGlite replay.
|
|
248
|
+
- **No app path here.** Where the dump lives, which engine replays, and when it is written are
|
|
249
|
+
`@ultimat3/cli`'s (`db-schema-dump.ts`). `schema-load.contract.test.ts` reads the reference app's
|
|
250
|
+
migrations as text and runs on Postgres when `TEST_DATABASE_URL` is set.
|
|
251
|
+
- **`pglite-extensions.ts` is the one linker** (`linkPgliteExtensions` → `{ linked, missing }`):
|
|
252
|
+
`contrib/<name>` then the package root; the name is data (read from migration text), screened
|
|
253
|
+
by `pgliteExtensionExport` before it reaches a specifier; `plpgsql` is built in, never missing.
|
|
254
|
+
Only a not-found import is `missing`; a bundle that throws is `X_DB_UNAVAILABLE`.
|
|
255
|
+
- **`pglite-snapshot.ts`**: `snapshotDir` makes a `memory://` boot a restore. Key = PGlite version
|
|
256
|
+
(read off its `package.json`; unreadable = no cache), never the extension set. One file, checksum in its
|
|
257
|
+
header; unsound or unopenable → deleted and rebuilt; unreadable → a miss. Temp name + `rename`. Uncompressed by
|
|
258
|
+
measurement: gzip taxes the boot that writes, and a CI checkout always writes.
|
|
259
|
+
- **One embedded boot** serves every database-backed dump test: `schema-dump.test.ts`.
|
|
260
|
+
|
|
223
261
|
## Branches, replicas, read-only
|
|
224
262
|
|
|
225
263
|
- **`reapBranches` sweeps branches of THIS database**: the marker is `ultimate:branch:<base>:<iso>`
|
package/README.md
CHANGED
|
@@ -51,6 +51,13 @@ await withTransaction(async (tx) => {
|
|
|
51
51
|
| `destructiveStatements()` / `hasDestructiveMarker()` / `isDestructive()` / `DESTRUCTIVE_MARKER` | `As of 2026-08`: the destructive-SQL rail — does this `up` drop, truncate or retype, and does the file declare it with `-- destructive: true`? One classifier, read by `x db gen` when it writes the marker and by `x verify` when it demands one |
|
|
52
52
|
| `stripSqlNoise()` | comments, literals, dollar-quoted bodies and quoted identifiers blanked **in source order**, so a reader sees the operation and not the prose. Shared by `readOnlyQuery()` and the destructive rail |
|
|
53
53
|
| `introspect()` | live schema → `SchemaDescription`. **App tables only**, `As of 2026-08-24`: a relation an extension owns (`pg_depend`, `deptype = 'e'`) and anything that is not an ordinary or partitioned table are excluded before the fold, and an explicit `exclude` cannot bring them back |
|
|
54
|
+
| `introspectCatalog()` / `CatalogDescription` / `emptyCatalog()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: the WHOLE schema in the catalog's own spelling — extensions, enum and domain types, sequences, tables, indexes, foreign keys, views, functions, triggers — sorted in code-unit order, plus `unrendered`: what exists and the dump cannot spell. Comparable only to another reading of itself; `introspect()` stays the entity-vocabulary reading a snapshot is diffed in |
|
|
55
|
+
| `renderSchemaDump()` / `SchemaDumpFile` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: a catalog → the schema dump's files. Pure and byte-deterministic. [The schema dump](#the-schema-dump) |
|
|
56
|
+
| `loadSchemaDump()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: build a schema from those files, in one transaction, retrying a file that names something not created yet |
|
|
57
|
+
| `compareSchemaDump()` / `reloadDifferences()` / `schemaDumpDrift()` / `schemaDumpDifferenceOf()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: `X_SCHEMA_DUMP_DRIFT` — committed files against rendered ones in both directions, and load-equals-replay as a comparison |
|
|
58
|
+
| `unexpectedObjects()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: the triggers, functions, views, types and sequences a live catalog holds and an expected one does not, as `unexpected-object` drift. Identity, never definition text |
|
|
59
|
+
| `PgliteOptions.extensions` / `linkPgliteExtensions()` | `As of 2026-10`: Postgres extensions to link at boot, by name — a list, or a function for a caller whose list is read from disk. `linkPgliteExtensions(names)` answers `{ linked, missing }` without booting anything: `missing` is what the installed PGlite ships no bundle for, which is how `@ultimat3/cli` decides a replay needs a real Postgres. A missing name is skipped at boot and refused by `create extension` itself |
|
|
60
|
+
| `PgliteOptions.snapshotDir` | `As of 2026-10`: a directory for the post-`initdb` snapshot, so an in-memory boot is a restore (~0.4 s against ~2.7 s). Keyed on the PGlite version alone — `initdb` never sees a linked extension, so one snapshot serves every set; checksummed, never trusted, written by temp-name-then-rename. Ignored for a data directory on disk |
|
|
54
61
|
| `createBranch()` / `dropBranch()` / `reapBranches()` | copy-on-write branch databases. `As of 2026-08-19` the marker comment records the **base** as well as the instant (`ultimate:branch:<base>:<iso>`, on `BranchInfo.base`), and `reapBranches()` sweeps only branches of the database it is connected to — one Postgres hosting two Ultimate apps used to mean one app's nightly reap dropped the other's branches. A pre-3.x marker records no base and is skipped, never dropped |
|
|
55
62
|
| `createPgliteClient()` / `branchPglite()` | the embedded database — Postgres in this process |
|
|
56
63
|
| `ensureReadOnlyRole()` / `grantReadOnlySql()` / `READONLY_ROLE` | a `NOLOGIN`, SELECT-only Postgres role — layer 1 of `db.query`'s defence |
|
|
@@ -231,6 +238,52 @@ the exit code — and a deploy that rolled on past a schema nobody can reconstru
|
|
|
231
238
|
drift exists to catch. There is no `x db drift`, and `x verify`'s `drift` step is the *source*
|
|
232
239
|
detector (`checkSourceDrift`), which needs no database and never calls this.
|
|
233
240
|
|
|
241
|
+
## The schema dump
|
|
242
|
+
|
|
243
|
+
**Imported from `@ultimat3/db/schema-dump`, never from the barrel** (23.0.0): `introspectCatalog`,
|
|
244
|
+
`emptyCatalog`, the `Catalog*` types, `renderSchemaDump`, `loadSchemaDump`, `compareSchemaDump`,
|
|
245
|
+
`reloadDifferences`, `schemaDumpDrift`, `schemaDumpDifferenceOf` and `unexpectedObjects`. Three
|
|
246
|
+
callers run them — `x db gen`, `x db migrate`, the gate's `drift` step — and `@ultimat3/db` is in
|
|
247
|
+
every role's boot graph, where these ten modules served nothing. `schema-dump-entry.test.ts` holds
|
|
248
|
+
both halves: the barrel evaluates none of them, and no name has two homes.
|
|
249
|
+
|
|
250
|
+
`renderSchemaDump(await introspectCatalog({ client }))` is the schema as files: one directory per
|
|
251
|
+
object kind, numbered in the order a database is built in, one file per named object.
|
|
252
|
+
|
|
253
|
+
| Directory | Holds |
|
|
254
|
+
|---|---|
|
|
255
|
+
| `01_extensions/` | `create extension if not exists` — never `plpgsql` |
|
|
256
|
+
| `02_types/` | enums, domains |
|
|
257
|
+
| `03_sequences/` | sequences no column owns |
|
|
258
|
+
| `04_tables/` | the table; a `serial` column's sequence before it and its ownership after; `replica identity full` |
|
|
259
|
+
| `05_indexes/` | per table, indexes no constraint backs; `replica identity using index` |
|
|
260
|
+
| `06_foreign_keys/` | per table, `alter table … add constraint` |
|
|
261
|
+
| `07_views/` | views; a materialized view with its indexes |
|
|
262
|
+
| `08_functions/` | per name, overloads together |
|
|
263
|
+
| `09_triggers/` | per table |
|
|
264
|
+
|
|
265
|
+
- An object whose owner — its table, else itself — starts with `x_` goes to a `framework/` twin of
|
|
266
|
+
the same layout. The rule is `FRAMEWORK_TABLE_PREFIX`, the one `appTables()` already holds.
|
|
267
|
+
- Postgres' own spellings (`format_type`, `pg_get_expr`, `pg_get_*def`), so a loaded dump renders
|
|
268
|
+
the same bytes. Sorted in JS, never by `order by`: a name's order follows the server's collation.
|
|
269
|
+
- No timestamp, version, owner or grant. Every sequence option is written, defaults included.
|
|
270
|
+
- A name becomes a file name with everything outside `[A-Za-z0-9_-]` percent-encoded.
|
|
271
|
+
- A file over 500 lines (`SCHEMA_DUMP_MAX_LINES`, the `filesize` ceiling) is split `<name>.1.sql`, `<name>.2.sql`;
|
|
272
|
+
`loadSchemaDump()` joins the parts in numeric order.
|
|
273
|
+
- What cannot be spelled — partitions, inheritance, foreign tables, composite and range types,
|
|
274
|
+
aggregates, row-security policies, rules — is named in `unrendered.sql` as comments.
|
|
275
|
+
|
|
276
|
+
`loadSchemaDump()` runs kind by kind, the framework twin first, each file in a savepoint. A file
|
|
277
|
+
refused with `42P01`, `42883` or `42704` — "not created yet" — is retried after the rest; a pass
|
|
278
|
+
that loads nothing ends with `X_SCHEMA_DUMP_DRIFT` naming the file.
|
|
279
|
+
|
|
280
|
+
**Not the snapshot.** `<id>.snapshot.json` is `x db gen`'s diff base, in the entity's vocabulary:
|
|
281
|
+
both sides of that diff are generator spellings. The dump is what the database holds, in SQL:
|
|
282
|
+
both sides of ITS comparison are catalog spellings. Neither can be compared with the other.
|
|
283
|
+
|
|
284
|
+
Which engine, where the files live, when they are written and what holds them is `@ultimat3/cli`'s
|
|
285
|
+
(`x db gen`, `x db migrate`, the `drift` step). This package renders, loads and compares.
|
|
286
|
+
|
|
234
287
|
## The embedded database
|
|
235
288
|
|
|
236
289
|
No `DATABASE_URL` means no Docker: `createPgliteClient()` runs Postgres as WASM inside this
|
|
@@ -312,6 +365,33 @@ so nothing else would ever end that wait. `migrate()` emits it as `SET LOCAL loc
|
|
|
312
365
|
each migration's own transaction — it reverts at COMMIT, so a DDL value never leaks onto the session
|
|
313
366
|
the ledger insert runs on.
|
|
314
367
|
|
|
368
|
+
## `LISTEN` on a session of its own
|
|
369
|
+
|
|
370
|
+
A pooled statement cannot hold a subscription: the next one runs on another connection. Both
|
|
371
|
+
clients hold ONE session beside the pool for it.
|
|
372
|
+
|
|
373
|
+
```ts
|
|
374
|
+
import { createPostgresClient } from '@ultimat3/db';
|
|
375
|
+
|
|
376
|
+
const client = createPostgresClient({ url: 'postgres://localhost:5432/app_test' });
|
|
377
|
+
const subscription = await client.listen(
|
|
378
|
+
'x_jobs_wake',
|
|
379
|
+
(payload) => console.log('notified', payload),
|
|
380
|
+
() => console.log('listening'), // again after every re-dial
|
|
381
|
+
);
|
|
382
|
+
await subscription.unlisten();
|
|
383
|
+
```
|
|
384
|
+
|
|
385
|
+
| Fact | Detail |
|
|
386
|
+
|---|---|
|
|
387
|
+
| the session | `Bun.SQL.listen` on the pooled client — outside `max`, untouched by `idleTimeout`, ended by `unlisten()` or `close()`; PGlite's own `listen` on the embedded one. Every channel a client listens on shares it |
|
|
388
|
+
| `onListening` | fires each time the subscription is (re-)established. The driver re-dials a session that died, and what was notified in between is LOST — re-read the source of truth there |
|
|
389
|
+
| the channel | a lower-case identifier of at most 63 characters, refused otherwise before a driver sees it |
|
|
390
|
+
| a transaction-pooling proxy | `listen()` resolves and nothing is ever delivered. Only a notification that arrives proves the path |
|
|
391
|
+
| `canListen(client)` | `true` for both shipped clients. `false` for a replicated pair — listen on the primary it was built from, as the boot does |
|
|
392
|
+
|
|
393
|
+
`@ultimat3/jobs`' `startQueueWake` is the framework's one caller.
|
|
394
|
+
|
|
315
395
|
## Migrations
|
|
316
396
|
|
|
317
397
|
`migrate()` takes the advisory lock by **polling** `pg_try_advisory_lock(4919202607)` every 500ms
|
|
@@ -424,6 +504,7 @@ job the moment Postgres fails over.
|
|
|
424
504
|
| `X_MIGRATION_IRREVERSIBLE` | generated `down` would lose data |
|
|
425
505
|
| `X_SQL_UNSAFE` | non-bindable interpolation, or an unsafe identifier/branch name |
|
|
426
506
|
| `X_BRANCH_EXISTS` | branch database already exists (or is the connected one) |
|
|
507
|
+
| `X_SCHEMA_DUMP_DRIFT` | the committed schema dump is not what the migrations produce, or does not load back |
|
|
427
508
|
| `X_NOT_IMPLEMENTED` | branching an in-memory PGlite — a copy needs a directory |
|
|
428
509
|
| `X_ENV_MISSING` | core's — `DATABASE_POOL_MAX` is set to something that is not a positive integer |
|
|
429
510
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ultimat3/db",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "23.0.0",
|
|
4
4
|
"description": "Postgres access, transactions, migrations and drift detection",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -14,11 +14,13 @@
|
|
|
14
14
|
"provenance": true
|
|
15
15
|
},
|
|
16
16
|
"exports": {
|
|
17
|
-
".": "./src/index.ts"
|
|
17
|
+
".": "./src/index.ts",
|
|
18
|
+
"./schema-dump": "./src/schema-dump-entry.ts"
|
|
18
19
|
},
|
|
19
20
|
"files": [
|
|
20
21
|
"src",
|
|
21
22
|
"!src/**/*.test.ts",
|
|
23
|
+
"!src/**/*-fixture.ts",
|
|
22
24
|
"CLAUDE.md",
|
|
23
25
|
"README.md",
|
|
24
26
|
"LICENSE"
|
|
@@ -31,7 +33,7 @@
|
|
|
31
33
|
"test": "bun test"
|
|
32
34
|
},
|
|
33
35
|
"dependencies": {
|
|
34
|
-
"@ultimat3/core": "
|
|
36
|
+
"@ultimat3/core": "23.0.0"
|
|
35
37
|
},
|
|
36
38
|
"peerDependencies": {
|
|
37
39
|
"@electric-sql/pglite": ">=0.5.0"
|
package/src/bun-sql.ts
CHANGED
|
@@ -58,6 +58,18 @@ export interface BunSqlDriver {
|
|
|
58
58
|
unsafe(text: string, values?: readonly unknown[]): Promise<unknown>;
|
|
59
59
|
reserve(): Promise<BunSqlReserved>;
|
|
60
60
|
close(options?: { readonly timeout?: number }): Promise<void>;
|
|
61
|
+
/**
|
|
62
|
+
* `LISTEN` on a connection the driver opens for it, OUTSIDE the pool. Measured on Bun 1.4.0
|
|
63
|
+
* against Postgres 17: a `max: 1` pool still answers statements while it is held;
|
|
64
|
+
* `idleTimeout` does not close it; a backend killed under it is re-dialled by the driver, with
|
|
65
|
+
* its own growing gap between attempts, and `onlisten` fires again once it is back; `close()`
|
|
66
|
+
* and `unlisten()` both end the session. Optional because a fake driver has none.
|
|
67
|
+
*/
|
|
68
|
+
listen?(
|
|
69
|
+
channel: string,
|
|
70
|
+
onnotify: (payload: string) => void,
|
|
71
|
+
onlisten?: () => void,
|
|
72
|
+
): Promise<{ unlisten(): Promise<void> }>;
|
|
61
73
|
}
|
|
62
74
|
|
|
63
75
|
export type BunSqlFactory = new (
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
// Single responsibility: catalog ROWS into one table's description — its columns in ordinal order,
|
|
2
|
+
// its constraints ranked, the sequences its `serial` and identity columns own. Pure; the queries
|
|
3
|
+
// are `catalog-relations.ts`'s and the orchestration is `introspect-catalog.ts`'s.
|
|
4
|
+
|
|
5
|
+
import type {
|
|
6
|
+
CatalogColumn,
|
|
7
|
+
CatalogOwnedSequence,
|
|
8
|
+
CatalogReplicaIdentity,
|
|
9
|
+
CatalogSequence,
|
|
10
|
+
CatalogTable,
|
|
11
|
+
} from './catalog';
|
|
12
|
+
import type { ColumnRow, ConstraintRow, SequenceRow, TableRow } from './catalog-relations';
|
|
13
|
+
|
|
14
|
+
/**
|
|
15
|
+
* By UTF-16 code unit, never `localeCompare` and never SQL's `order by`: both follow a locale, and
|
|
16
|
+
* the same database must produce the same bytes on a laptop and in CI.
|
|
17
|
+
*/
|
|
18
|
+
export const byCodeUnit = (a: string, b: string): number => (a < b ? -1 : a > b ? 1 : 0);
|
|
19
|
+
|
|
20
|
+
export const by =
|
|
21
|
+
<T>(...keys: readonly ((value: T) => string)[]) =>
|
|
22
|
+
(a: T, b: T): number => {
|
|
23
|
+
for (const key of keys) {
|
|
24
|
+
const order = byCodeUnit(key(a), key(b));
|
|
25
|
+
if (order !== 0) return order;
|
|
26
|
+
}
|
|
27
|
+
return 0;
|
|
28
|
+
};
|
|
29
|
+
|
|
30
|
+
export const sequenceOf = (row: SequenceRow): CatalogSequence => ({
|
|
31
|
+
name: row.name,
|
|
32
|
+
dataType: row.data_type,
|
|
33
|
+
start: row.seq_start,
|
|
34
|
+
increment: row.seq_increment,
|
|
35
|
+
min: row.seq_min,
|
|
36
|
+
max: row.seq_max,
|
|
37
|
+
cache: row.seq_cache,
|
|
38
|
+
cycle: row.seq_cycle,
|
|
39
|
+
});
|
|
40
|
+
|
|
41
|
+
function columnOf(row: ColumnRow, sequences: readonly SequenceRow[]): CatalogColumn {
|
|
42
|
+
const identity = sequences.find(
|
|
43
|
+
(sequence) =>
|
|
44
|
+
sequence.ownership === 'i' &&
|
|
45
|
+
sequence.owner_table === row.table_name &&
|
|
46
|
+
sequence.owner_column === row.name,
|
|
47
|
+
);
|
|
48
|
+
const generated = row.generated !== '' && row.generated !== null;
|
|
49
|
+
return {
|
|
50
|
+
name: row.name,
|
|
51
|
+
type: row.type,
|
|
52
|
+
notNull: row.not_null,
|
|
53
|
+
default: generated ? null : row.expression,
|
|
54
|
+
generated: generated ? row.expression : null,
|
|
55
|
+
identity:
|
|
56
|
+
identity === undefined
|
|
57
|
+
? null
|
|
58
|
+
: { mode: row.identity === 'a' ? 'always' : 'by default', sequence: sequenceOf(identity) },
|
|
59
|
+
collation: row.collation,
|
|
60
|
+
};
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
function replicaIdentityOf(row: TableRow): CatalogReplicaIdentity {
|
|
64
|
+
if (row.replica_identity === 'f') return { kind: 'full' };
|
|
65
|
+
if (row.replica_identity === 'n') return { kind: 'nothing' };
|
|
66
|
+
if (row.replica_identity === 'i' && row.replica_index !== null)
|
|
67
|
+
return { kind: 'index', index: row.replica_index };
|
|
68
|
+
return { kind: 'default' };
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
/** Primary key first, then unique, check, exclusion — each group by name. */
|
|
72
|
+
const CONSTRAINT_RANK: ReadonlyMap<string, string> = new Map([
|
|
73
|
+
['p', '0'],
|
|
74
|
+
['u', '1'],
|
|
75
|
+
['c', '2'],
|
|
76
|
+
['x', '3'],
|
|
77
|
+
]);
|
|
78
|
+
|
|
79
|
+
export function tableOf(
|
|
80
|
+
row: TableRow,
|
|
81
|
+
columns: readonly ColumnRow[],
|
|
82
|
+
constraints: readonly ConstraintRow[],
|
|
83
|
+
sequences: readonly SequenceRow[],
|
|
84
|
+
): CatalogTable {
|
|
85
|
+
return {
|
|
86
|
+
name: row.name,
|
|
87
|
+
unlogged: row.persistence === 'u',
|
|
88
|
+
options: row.options,
|
|
89
|
+
columns: columns
|
|
90
|
+
.filter((column) => column.table_name === row.name)
|
|
91
|
+
.sort((a, b) => a.position - b.position)
|
|
92
|
+
.map((column) => columnOf(column, sequences)),
|
|
93
|
+
constraints: constraints
|
|
94
|
+
.filter((constraint) => constraint.table_name === row.name && constraint.type !== 'f')
|
|
95
|
+
.sort(
|
|
96
|
+
by(
|
|
97
|
+
(c) => CONSTRAINT_RANK.get(c.type) ?? '9',
|
|
98
|
+
(c) => c.name,
|
|
99
|
+
),
|
|
100
|
+
)
|
|
101
|
+
.map((constraint) => ({ name: constraint.name, definition: constraint.definition })),
|
|
102
|
+
sequences: sequences
|
|
103
|
+
.filter((sequence) => sequence.ownership === 'a' && sequence.owner_table === row.name)
|
|
104
|
+
.sort(by((sequence) => sequence.name))
|
|
105
|
+
.map(
|
|
106
|
+
(sequence): CatalogOwnedSequence => ({
|
|
107
|
+
...sequenceOf(sequence),
|
|
108
|
+
column: sequence.owner_column ?? '',
|
|
109
|
+
}),
|
|
110
|
+
),
|
|
111
|
+
replicaIdentity: replicaIdentityOf(row),
|
|
112
|
+
};
|
|
113
|
+
}
|
|
@@ -0,0 +1,200 @@
|
|
|
1
|
+
// Single responsibility: the catalog queries for every object that is NOT a table — extensions,
|
|
2
|
+
// enum and domain types, views, functions, triggers — plus the list of objects the schema dump has
|
|
3
|
+
// no statement for. Flat rows; `introspect-catalog.ts` folds and sorts them.
|
|
4
|
+
|
|
5
|
+
import { notExtensionOwned } from './catalog-relations';
|
|
6
|
+
import type { DbClient } from './client';
|
|
7
|
+
import { sql } from './sql';
|
|
8
|
+
|
|
9
|
+
export interface NameRow {
|
|
10
|
+
readonly name: string;
|
|
11
|
+
}
|
|
12
|
+
|
|
13
|
+
export interface EnumRow {
|
|
14
|
+
readonly name: string;
|
|
15
|
+
readonly label: string;
|
|
16
|
+
readonly position: number;
|
|
17
|
+
}
|
|
18
|
+
|
|
19
|
+
export interface DomainRow {
|
|
20
|
+
readonly name: string;
|
|
21
|
+
readonly base_type: string;
|
|
22
|
+
readonly not_null: boolean;
|
|
23
|
+
readonly expression: string | null;
|
|
24
|
+
}
|
|
25
|
+
|
|
26
|
+
export interface DomainCheckRow {
|
|
27
|
+
readonly domain_name: string;
|
|
28
|
+
readonly name: string;
|
|
29
|
+
readonly definition: string;
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
export interface ViewRow {
|
|
33
|
+
readonly name: string;
|
|
34
|
+
readonly kind: string;
|
|
35
|
+
readonly options: string | null;
|
|
36
|
+
readonly definition: string;
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
export interface FunctionRow {
|
|
40
|
+
readonly name: string;
|
|
41
|
+
readonly arguments: string;
|
|
42
|
+
readonly definition: string;
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
export interface TriggerRow {
|
|
46
|
+
readonly table_name: string;
|
|
47
|
+
readonly name: string;
|
|
48
|
+
readonly definition: string;
|
|
49
|
+
readonly enabled: string;
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
export interface UnrenderedRow {
|
|
53
|
+
readonly kind: string;
|
|
54
|
+
readonly name: string;
|
|
55
|
+
readonly table_name: string | null;
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
/**
|
|
59
|
+
* Database-wide, so no schema filter. `plpgsql` is left out: every database has it, `create
|
|
60
|
+
* extension` on it is a no-op, and a dump line that can never differ is noise in every app.
|
|
61
|
+
*/
|
|
62
|
+
export const extensionRows = (client: DbClient): Promise<readonly NameRow[]> =>
|
|
63
|
+
client.query<NameRow>(sql`select extname as name from pg_extension where extname <> 'plpgsql'`);
|
|
64
|
+
|
|
65
|
+
export const enumRows = (client: DbClient, schema: string): Promise<readonly EnumRow[]> =>
|
|
66
|
+
client.query<EnumRow>(sql`
|
|
67
|
+
select t.typname as name, e.enumlabel as label, e.enumsortorder::float8 as position
|
|
68
|
+
from pg_enum e
|
|
69
|
+
join pg_type t on t.oid = e.enumtypid
|
|
70
|
+
join pg_namespace n on n.oid = t.typnamespace
|
|
71
|
+
where n.nspname = ${schema} and ${notExtensionOwned('pg_type', 't.oid')}
|
|
72
|
+
`);
|
|
73
|
+
|
|
74
|
+
export const domainRows = (client: DbClient, schema: string): Promise<readonly DomainRow[]> =>
|
|
75
|
+
client.query<DomainRow>(sql`
|
|
76
|
+
select
|
|
77
|
+
t.typname as name,
|
|
78
|
+
format_type(t.typbasetype, t.typtypmod) as base_type,
|
|
79
|
+
t.typnotnull as not_null,
|
|
80
|
+
t.typdefault as expression
|
|
81
|
+
from pg_type t
|
|
82
|
+
join pg_namespace n on n.oid = t.typnamespace
|
|
83
|
+
where n.nspname = ${schema} and t.typtype = 'd' and ${notExtensionOwned('pg_type', 't.oid')}
|
|
84
|
+
`);
|
|
85
|
+
|
|
86
|
+
export const domainCheckRows = (
|
|
87
|
+
client: DbClient,
|
|
88
|
+
schema: string,
|
|
89
|
+
): Promise<readonly DomainCheckRow[]> =>
|
|
90
|
+
client.query<DomainCheckRow>(sql`
|
|
91
|
+
select t.typname as domain_name, k.conname as name, pg_get_constraintdef(k.oid) as definition
|
|
92
|
+
from pg_constraint k
|
|
93
|
+
join pg_type t on t.oid = k.contypid
|
|
94
|
+
join pg_namespace n on n.oid = t.typnamespace
|
|
95
|
+
where n.nspname = ${schema} and k.contype = 'c'
|
|
96
|
+
`);
|
|
97
|
+
|
|
98
|
+
/** The non-pretty `pg_get_viewdef`: the pretty form's line breaks follow a width, not the query. */
|
|
99
|
+
export const viewRows = (client: DbClient, schema: string): Promise<readonly ViewRow[]> =>
|
|
100
|
+
client.query<ViewRow>(sql`
|
|
101
|
+
select
|
|
102
|
+
c.relname as name,
|
|
103
|
+
c.relkind as kind,
|
|
104
|
+
array_to_string(c.reloptions, ', ') as options,
|
|
105
|
+
pg_get_viewdef(c.oid) as definition
|
|
106
|
+
from pg_class c
|
|
107
|
+
join pg_namespace n on n.oid = c.relnamespace
|
|
108
|
+
where n.nspname = ${schema}
|
|
109
|
+
and c.relkind in ('v', 'm')
|
|
110
|
+
and ${notExtensionOwned('pg_class', 'c.oid')}
|
|
111
|
+
`);
|
|
112
|
+
|
|
113
|
+
/** Functions, procedures and window functions. An aggregate is refused by `pg_get_functiondef`. */
|
|
114
|
+
export const functionRows = (client: DbClient, schema: string): Promise<readonly FunctionRow[]> =>
|
|
115
|
+
client.query<FunctionRow>(sql`
|
|
116
|
+
select
|
|
117
|
+
p.proname as name,
|
|
118
|
+
pg_get_function_identity_arguments(p.oid) as arguments,
|
|
119
|
+
pg_get_functiondef(p.oid) as definition
|
|
120
|
+
from pg_proc p
|
|
121
|
+
join pg_namespace n on n.oid = p.pronamespace
|
|
122
|
+
where n.nspname = ${schema}
|
|
123
|
+
and p.prokind in ('f', 'p', 'w')
|
|
124
|
+
and ${notExtensionOwned('pg_proc', 'p.oid')}
|
|
125
|
+
`);
|
|
126
|
+
|
|
127
|
+
/**
|
|
128
|
+
* `not tgisinternal`: a foreign key's own enforcement triggers are the constraint's, not the app's.
|
|
129
|
+
* A trigger on an extension-owned table is skipped with its table: the dump creates no such table,
|
|
130
|
+
* so its `09_triggers/` file could never load.
|
|
131
|
+
*/
|
|
132
|
+
export const triggerRows = (client: DbClient, schema: string): Promise<readonly TriggerRow[]> =>
|
|
133
|
+
client.query<TriggerRow>(sql`
|
|
134
|
+
select
|
|
135
|
+
c.relname as table_name,
|
|
136
|
+
t.tgname as name,
|
|
137
|
+
pg_get_triggerdef(t.oid) as definition,
|
|
138
|
+
t.tgenabled as enabled
|
|
139
|
+
from pg_trigger t
|
|
140
|
+
join pg_class c on c.oid = t.tgrelid
|
|
141
|
+
join pg_namespace n on n.oid = c.relnamespace
|
|
142
|
+
where n.nspname = ${schema}
|
|
143
|
+
and not t.tgisinternal
|
|
144
|
+
and ${notExtensionOwned('pg_class', 'c.oid')}
|
|
145
|
+
`);
|
|
146
|
+
|
|
147
|
+
/**
|
|
148
|
+
* What exists in the schema and the dump cannot spell. One query, one closed list — a kind added
|
|
149
|
+
* here is a kind the dump admits it does not carry, and a kind rendered later leaves this list in
|
|
150
|
+
* the same diff.
|
|
151
|
+
*/
|
|
152
|
+
export const unrenderedRows = (
|
|
153
|
+
client: DbClient,
|
|
154
|
+
schema: string,
|
|
155
|
+
): Promise<readonly UnrenderedRow[]> =>
|
|
156
|
+
client.query<UnrenderedRow>(sql`
|
|
157
|
+
select kind, name, table_name from (
|
|
158
|
+
select
|
|
159
|
+
case
|
|
160
|
+
when c.relkind = 'p' then 'partitioned table'
|
|
161
|
+
when c.relkind = 'f' then 'foreign table'
|
|
162
|
+
when c.relkind = 'c' then 'composite type'
|
|
163
|
+
else 'partition or inheritance child'
|
|
164
|
+
end as kind,
|
|
165
|
+
c.relname as name,
|
|
166
|
+
null::text as table_name,
|
|
167
|
+
c.relnamespace as namespace
|
|
168
|
+
from pg_class c
|
|
169
|
+
where (
|
|
170
|
+
c.relkind in ('p', 'f', 'c')
|
|
171
|
+
or (c.relkind = 'r' and (
|
|
172
|
+
c.relispartition or exists (select 1 from pg_inherits h where h.inhrelid = c.oid)
|
|
173
|
+
))
|
|
174
|
+
)
|
|
175
|
+
and ${notExtensionOwned('pg_class', 'c.oid')}
|
|
176
|
+
union all
|
|
177
|
+
select 'range type', t.typname, null::text, t.typnamespace
|
|
178
|
+
from pg_type t
|
|
179
|
+
where t.typtype = 'r' and ${notExtensionOwned('pg_type', 't.oid')}
|
|
180
|
+
union all
|
|
181
|
+
select 'aggregate', p.proname, null::text, p.pronamespace
|
|
182
|
+
from pg_proc p
|
|
183
|
+
where p.prokind = 'a' and ${notExtensionOwned('pg_proc', 'p.oid')}
|
|
184
|
+
union all
|
|
185
|
+
select 'row security policy', pol.polname, c.relname, c.relnamespace
|
|
186
|
+
from pg_policy pol
|
|
187
|
+
join pg_class c on c.oid = pol.polrelid
|
|
188
|
+
union all
|
|
189
|
+
select 'row security', c.relname, c.relname, c.relnamespace
|
|
190
|
+
from pg_class c
|
|
191
|
+
where c.relrowsecurity
|
|
192
|
+
union all
|
|
193
|
+
select 'rule', r.rulename, c.relname, c.relnamespace
|
|
194
|
+
from pg_rewrite r
|
|
195
|
+
join pg_class c on c.oid = r.ev_class
|
|
196
|
+
where r.rulename <> '_RETURN'
|
|
197
|
+
) objects
|
|
198
|
+
join pg_namespace n on n.oid = objects.namespace
|
|
199
|
+
where n.nspname = ${schema}
|
|
200
|
+
`);
|