@ultimat3/db 23.0.0 → 25.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +83 -85
- package/README.md +88 -27
- package/package.json +3 -3
- package/src/array-parameter.ts +38 -1
- package/src/bound-parameters.ts +22 -3
- package/src/catalog-fold.ts +4 -1
- package/src/catalog-objects.ts +29 -0
- package/src/catalog.ts +10 -2
- package/src/client.ts +2 -2
- package/src/column-alter.ts +9 -3
- package/src/commit-tag.ts +21 -0
- package/src/default-client.ts +4 -4
- package/src/dependent-view.ts +7 -5
- package/src/destructive.ts +1 -1
- package/src/drift-append-only.ts +53 -0
- package/src/drift-errors.ts +3 -3
- package/src/drift-findings.ts +211 -59
- package/src/drift.ts +33 -19
- package/src/entity-shape.ts +5 -0
- package/src/errors.ts +12 -8
- package/src/fake.ts +1 -1
- package/src/foreign-key.ts +0 -34
- package/src/generate-append-only.ts +146 -0
- package/src/generate.ts +20 -6
- package/src/index.ts +11 -9
- package/src/introspect-catalog.ts +19 -2
- package/src/introspect.ts +92 -12
- package/src/migrate-rollback.ts +44 -0
- package/src/migrate.ts +48 -158
- package/src/migration-ledger.ts +167 -0
- package/src/object-drift.ts +77 -20
- package/src/pglite-branch.ts +7 -7
- package/src/pglite.ts +16 -4
- package/src/pool-profile.ts +1 -1
- package/src/primary-key.ts +210 -0
- package/src/schema-dump-table.ts +4 -1
- package/src/sibling-turn.ts +49 -0
- package/src/snapshot-parse.ts +9 -3
- package/src/sqlstate.ts +30 -10
- package/src/statement-funnel.ts +16 -5
- package/src/transaction-errors.ts +66 -0
- package/src/transaction-options.ts +122 -0
- package/src/transaction.ts +131 -121
- package/src/drift-fixtures.ts +0 -23
- package/src/fake-pglite.ts +0 -32
- package/src/fake-reservable.ts +0 -50
package/CLAUDE.md
CHANGED
|
@@ -35,47 +35,48 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
|
|
|
35
35
|
only while its turn is held. `pglite-embedded.test.ts`, `pglite.test.ts`,
|
|
36
36
|
`pglite-two-clients.test.ts`, `pglite-observer.test.ts`.
|
|
37
37
|
- **The third rule is both drivers'**: `client.ts`'s pinned handle also runs direct only while held.
|
|
38
|
-
`release()` is idempotent on both; `DbConnection` and `Turn` are `Disposable
|
|
39
|
-
`release()`).
|
|
38
|
+
`release()` is idempotent on both; `DbConnection` and `Turn` are `Disposable`.
|
|
40
39
|
- **A pin is held by `using`, never a hand-rolled `try/finally`** (`withTransaction`,
|
|
41
40
|
`readOnlyQuery`); `BEGIN` lives inside the guarded scope.
|
|
42
|
-
- **`sqlstate.ts`**: `errno` first, `code` second,
|
|
43
|
-
`DB_SQLSTATE_CODES` is closed; `driverError()` is its one consumer, `sendOn` its one caller.
|
|
41
|
+
- **`sqlstate.ts`**: `errno` first, `code` second, shape AND provenance (`isState`: `severity` =
|
|
42
|
+
server; `syscall`/numeric `errno` = socket; else needs a digit). `DB_SQLSTATE_CODES` is closed; `driverError()` is its one consumer, `sendOn` its one caller.
|
|
44
43
|
- **`DbTx.origin` is the client the scope was opened on**, never the pin (entity's pinned-repository
|
|
45
44
|
check reads it); a nested scope reports the root's.
|
|
46
45
|
- **`withTransaction(fn, { retry })` re-runs `fn` only on `40001`/`40P01`**, default 0; each attempt
|
|
47
46
|
its own pin, `BEGIN` and undo list (`runRoot`); a nested `retry` is `X_INVARIANT`. A re-run waits
|
|
48
|
-
(`transaction-backoff.ts`:
|
|
49
|
-
|
|
50
|
-
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
`
|
|
57
|
-
-
|
|
58
|
-
`
|
|
59
|
-
|
|
60
|
-
|
|
47
|
+
(`transaction-backoff.ts`: 10 ms → 500 ms, full jitter; `{ sleep, random }` are seams).
|
|
48
|
+
- **Four codes are `retryable`** (`DB_ERROR_RETRY`); terminal ones stay unclassified
|
|
49
|
+
(`errors-retry.test.ts`). Core's `retry()` executor is NOT adopted.
|
|
50
|
+
- **`BEGIN` re-derives its isolation level from the closed set** (`isolationMode`; else `X_SQL_UNSAFE`).
|
|
51
|
+
- **An aborted transaction is reported** (`X_DB_TRANSACTION_ABORTED`; README "Transactions that end
|
|
52
|
+
badly"): a failure carrying a SQLSTATE marks the root's `abort`, only a successful `ROLLBACK TO`
|
|
53
|
+
clears it (a failed one sets it), and `COMMIT` is then refused unsent. `commit-tag.ts` reads the
|
|
54
|
+
COMMIT tag in BOTH funnels. `COMMIT` rejecting with no SQLSTATE is `X_DB_COMMIT_UNKNOWN`: neither
|
|
55
|
+
list runs. `transaction-errors.ts`; `transaction-options.ts` holds `DbTx`, options, `BEGIN` text.
|
|
56
|
+
- **Sibling nested scopes take turns** (`TxState.children`; savepoints are a stack) under a deadline
|
|
57
|
+
(`sibling-turn.ts`, `siblingWaitMs`, `X_DB_SIBLING_SCOPE_TIMEOUT`). A nested
|
|
58
|
+
`isolation`/`readOnly`/`deferrable`/foreign `client` is `X_INVARIANT`.
|
|
59
|
+
- **`close()` is BOUNDED by the driver's own `{ timeout }` in SECONDS** (`drainTimeoutMs / 1000`; `0`
|
|
60
|
+
sends none); the verdict is elapsed `performance.now()` (`X_DB_DRAIN_TIMEOUT`). It clears the
|
|
61
|
+
cached driver before awaiting the teardown. `pool-drain{,.live}.test.ts`.
|
|
61
62
|
- **`client.listen` is ONE session beside the pool** (`listen.ts`; `Bun.SQL.listen`, PGlite's
|
|
62
63
|
`listen` under a turn), never a reserved pin. `onListening` fires on every re-dial; a channel is
|
|
63
64
|
refused unless it is a plain identifier. `listen.test.ts`, `listen.live.test.ts`.
|
|
64
65
|
- `execute()` trusts the command tag only when `> 0`, in both drivers (`rowsOf`, `affectedBy`).
|
|
65
|
-
- **`pool-gauge.ts` derives `db_pool_max` / `db_pool_in_use` / `db_pool_waiting` from DEMAND**
|
|
66
|
-
`Bun.SQL` publishes no occupancy. `client.ts` is the one counter: `run()` from send to settle, a
|
|
67
|
-
pin from
|
|
68
|
-
on the first tracked pool, never at import. `pool-gauge.test.ts`.
|
|
66
|
+
- **`pool-gauge.ts` derives `db_pool_max` / `db_pool_in_use` / `db_pool_waiting` from DEMAND**
|
|
67
|
+
(`Bun.SQL` publishes no occupancy). `client.ts` is the one counter: `run()` from send to settle, a
|
|
68
|
+
pin from ask to release; a statement ON a pin is not counted again. Declared on the first pool.
|
|
69
69
|
- **`client.ts` connects, holds the client and the ambient `db()`**, and opens no socket at import;
|
|
70
70
|
`pool-profile.ts`, `connection-url.ts`, `bun-sql.ts`, `pool-reserve.ts`, `db-health.ts` (`checkDb`)
|
|
71
71
|
and `statement-funnel.ts` (`sendOn`/`runOn`) hold the rest.
|
|
72
|
-
- **`libpq-options.ts` merges the framework's `options` into the operator's**: the framework wins
|
|
73
|
-
the names it sets, the operator keeps
|
|
72
|
+
- **`libpq-options.ts` merges the framework's `options` into the operator's**: the framework wins
|
|
73
|
+
on the names it sets, the operator keeps the rest; emitted for all six roles.
|
|
74
74
|
- **`DATABASE_URL`'s scheme is screened at boot** (`POSTGRES_SCHEMES`: `postgres:`, `postgresql:`); the
|
|
75
|
-
received scheme is never echoed (it may be a
|
|
75
|
+
received scheme is never echoed (it may be a credential).
|
|
76
76
|
- **A JS array bound as a parameter is rendered here** (`array-parameter.ts`, `bound-parameters.ts`,
|
|
77
77
|
called only by `sendOn`) — `Bun.SQL` joins elements with commas. `NULL` bare vs `"NULL"`; quoting by
|
|
78
|
-
content; a `Uint8Array` is BYTEA; a ragged nest
|
|
78
|
+
content; a `Uint8Array` is BYTEA, in an array too; a ragged nest and an Invalid Date are refused
|
|
79
|
+
(`X_INVARIANT`), ABOVE the driver's `try` in both funnels. `array-parameter.live.test.ts`;
|
|
79
80
|
`packages/cli/src/pg-array.live.test.ts` is the composition test.
|
|
80
81
|
- **Every numeric option is screened** through core's `finiteCount` (`replicaClient`'s breaker,
|
|
81
82
|
`migrate`'s `lockWaitMs`, `readonlyQuery`'s `timeoutMs` — only an explicit `0` disables it — the pool
|
|
@@ -83,11 +84,10 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
|
|
|
83
84
|
|
|
84
85
|
## Observation
|
|
85
86
|
|
|
86
|
-
- **`observe.ts`: one process-wide `StatementObserver`** (`setStatementObserver()`
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
statement's own `try`.
|
|
87
|
+
- **`observe.ts`: one process-wide `StatementObserver`** (`setStatementObserver()`). Guard at the
|
|
88
|
+
call site; one observer, not a list; the seam swallows nothing; `onStatement` is synchronous and
|
|
89
|
+
must not issue SQL. Only `runOn` (`statement-funnel.ts`) and `statement()` (`pglite.ts`) invoke
|
|
90
|
+
it; both observe success and failure, and notify outside the statement's own `try`.
|
|
91
91
|
- **`attribution.ts`**: `withStatementAttribution(entity, op, fn)` — guard first (two strings, no
|
|
92
92
|
allocation), a scope not a parameter, innermost pair wins; the funnels stamp it on both settle paths.
|
|
93
93
|
`@ultimat3/entity`'s `postgresRepo` is the one producer.
|
|
@@ -95,20 +95,19 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
|
|
|
95
95
|
text) and `statementKind(text)` off `statementVerb(text)`. Read by `x dev`'s ledger and
|
|
96
96
|
`@ultimat3/testing`'s `statements` fixture. It counts nothing.
|
|
97
97
|
- **`statement-span.ts`**: `withStatementSpan` wraps the send alone — `db.<verb>`, attribute
|
|
98
|
-
`STATEMENT_ATTRIBUTE` (
|
|
99
|
-
|
|
98
|
+
`STATEMENT_ATTRIBUTE` (`@ultimat3/cli`'s `dev-traces.ts` imports it), OTel kind `client`, opened
|
|
99
|
+
only when an observer is installed.
|
|
100
100
|
- **`expected-loop.ts` is the ONLY suppression**: `expectedQueryLoop(reason, fn)`, innermost reason,
|
|
101
|
-
blank is `X_INVARIANT`; the funnel stamps `expected`; it suppresses a verdict, never a statement.
|
|
102
|
-
framework's own loops declare themselves (`migrate()`, `rollback()`,
|
|
103
|
-
|
|
104
|
-
-
|
|
105
|
-
`packages/cli/src/dev-queue.ts` wraps a real client for its `PgExecutor`, unattributed.
|
|
101
|
+
blank is `X_INVARIANT`; the funnel stamps `expected`; it suppresses a verdict, never a statement.
|
|
102
|
+
The framework's own loops declare themselves (`migrate()`, `rollback()`, admin's `search.ts`).
|
|
103
|
+
- `@ultimat3/jobs` never imports this package; its statements pass the observer only as
|
|
104
|
+
`packages/cli/src/runtime-queue.ts` wraps a real client for its `PgExecutor`, unattributed.
|
|
106
105
|
|
|
107
106
|
## Migrations
|
|
108
107
|
|
|
109
108
|
- **The migration lock is polled** (`pg_try_advisory_lock` every `MIGRATION_LOCK_POLL_MS` until
|
|
110
109
|
`MIGRATION_LOCK_WAIT_MS`, then `X_MIGRATE_CONCURRENT`), declared with `expectedQueryLoop`;
|
|
111
|
-
`
|
|
110
|
+
`recordingClient` stubs the lock as `locked: true`.
|
|
112
111
|
- **`lock_timeout` is the migration's** (`SET LOCAL` inside each migration's transaction, from the
|
|
113
112
|
`migrate` profile's 3 s).
|
|
114
113
|
- **The advisory lock is held by one pinned session, and `migrate()`/`rollback()` run every statement
|
|
@@ -130,8 +129,8 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
|
|
|
130
129
|
(`rollbackStepsInvalid`, `X_INVARIANT`).
|
|
131
130
|
- **`refuseDependentViews(tx, script)`** (`dependent-view.ts`) runs before each migration's first
|
|
132
131
|
statement: a word scan over `sql-scan.ts` finds retyped columns, one catalog round trip, the pair
|
|
133
|
-
filtered in JS, and `X_MIGRATION_VIEW_DEPENDS` carries the `drop view` /
|
|
134
|
-
`pg_get_viewdef` (built through `identifier()` inside a `try`).
|
|
132
|
+
filtered in JS (visible tables only), and `X_MIGRATION_VIEW_DEPENDS` carries the `drop view` /
|
|
133
|
+
`create view` from `pg_get_viewdef` (built through `identifier()` inside a `try`).
|
|
135
134
|
- `runningAppVersion()` delegates to core's `appVersion()`.
|
|
136
135
|
|
|
137
136
|
## Generation (`x db gen`)
|
|
@@ -142,7 +141,7 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
|
|
|
142
141
|
declared is CLOSED, live is OPEN (`indexMethodOf` passes the catalog through; `declaredMethod`
|
|
143
142
|
refuses); absent is `btree` through one function; `snapshotOf` records `using` only when declared;
|
|
144
143
|
`indexMethodSql` re-derives the literal (`X_SQL_UNSAFE` default); a unique or ordered GIN is
|
|
145
|
-
`X_INVARIANT`. `
|
|
144
|
+
`X_INVARIANT`. `introspectSchema()` reads `pg_am` (`introspect-embedded.test.ts`).
|
|
146
145
|
- **`index-plan.ts` walks both directions** (declared first, removed last). `dropRecordedIndex` emits
|
|
147
146
|
`alter table … drop constraint if exists` then `drop index` for a shape a constraint could back
|
|
148
147
|
(`mayBeConstraintBacked`); four names are skipped (primary, moved aside, rebuilt, over a dropped
|
|
@@ -171,59 +170,58 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
|
|
|
171
170
|
in `preAlters` at the top of `up`, re-adding is `foreignKeyPlan`'s, both `breaksOn` ends are needed
|
|
172
171
|
(`generate-retype-key.live.test.ts`). `sql-type.ts` reads `SQL_TYPES` with `Object.hasOwn`.
|
|
173
172
|
- **A generated column** (`generated-column.ts`): the clause right after the type; generated-and-
|
|
174
|
-
defaulted refused; an expression change is `set expression as (…)`; a retype
|
|
175
|
-
NULL add is one statement; generated → plain is `drop expression`; plain → generated rebuilds
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
`generate-generated-column
|
|
179
|
-
- **`REPLICA IDENTITY FULL` is
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
`
|
|
173
|
+
defaulted refused; an expression change is `set expression as (…)`; a retype has no `using`; a NOT
|
|
174
|
+
NULL add is one statement; generated → plain is `drop expression`; plain → generated rebuilds
|
|
175
|
+
(`rebuilt`) and moves dependents aside, its own type change does not. `introspectSchema` never reads
|
|
176
|
+
`generation_expression`.
|
|
177
|
+
`generate-generated-{column,rebuild}.live.test.ts`.
|
|
178
|
+
- **`REPLICA IDENTITY FULL` is a PARAMETER** (`GenerateOptions.replicaIdentityFull`, from the CLI's
|
|
179
|
+
`db-generate.ts`; `replica-identity.ts`): recorded `true` or absent, as a union; last in `up`; an
|
|
180
|
+
undeclared name skipped; `down` reverts it unless this migration created the table.
|
|
181
|
+
- **`appendOnly: true` is a trigger** (`generate-append-only.ts`): one `create or replace` function
|
|
182
|
+
(SQLSTATE `23001`, message leads `X_ENTITY_APPEND_ONLY`) and a fixed-name `ultimate_append_only`
|
|
183
|
+
`before update or delete … for each row`; recorded `true` or absent; no `truncate`. Drift:
|
|
184
|
+
enabled `triggerNames` (`drift-append-only.ts`, `X_APPEND_ONLY_TRIGGER_MISSING`).
|
|
184
185
|
- **A foreign key is `alter table … add constraint`**, collected into a bucket merged after every table
|
|
185
186
|
statement (`foreign-key-plan.ts`); **dropping a table has its own bucket emitted BEFORE the table
|
|
186
187
|
statements** (`preDrops`), ordered children-first by `drop-order.ts`, which breaks a two-table cycle
|
|
187
188
|
by dropping one key first. `foreignKeyPlan` walks both directions, drops the name the previous
|
|
188
189
|
snapshot recorded, and rebuilds a key whose `onDelete` moved. **`on delete` reaches the SQL**
|
|
189
190
|
(`onDeleteRule`, `foreign-key.ts`; an unknown rule is `X_INVARIANT`).
|
|
190
|
-
- **`entity-shape.ts` holds the
|
|
191
|
-
- **`snapshot-json.ts` writes
|
|
192
|
-
|
|
193
|
-
- **`declaredSchema()` answers the NEWEST migration's snapshot or `undefined
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
migration's files FIRST and only then run `x db gen`. `snapshotSiblings` / `migrationNameOf` build
|
|
197
|
-
the second command from the caller's path; both commands are screened (`unknownSchema` through
|
|
198
|
-
`shellInertIdentifier`, `migrationSnapshotMissing` through `renderFixShellArg`), degrading the whole
|
|
199
|
-
line to prose.
|
|
191
|
+
- **`entity-shape.ts` holds the `*Like` interfaces** (every later field optional).
|
|
192
|
+
- **`snapshot-json.ts` writes a fixed point of Biome** (an array collapses when it fits at `<= 100`
|
|
193
|
+
with its trailing comma); `snapshot-json.test.ts` runs `biome format`.
|
|
194
|
+
- **`declaredSchema()` answers the NEWEST migration's snapshot or `undefined`** (`checkDrift`:
|
|
195
|
+
`unknown-schema`; `x db gen`: `X_MIGRATION_SNAPSHOT_MISSING`). Both fixes: restore the sidecar
|
|
196
|
+
(`git checkout --`), else delete the migration's files FIRST, then `x db gen`; both screened.
|
|
200
197
|
|
|
201
198
|
## Drift and introspection
|
|
202
199
|
|
|
203
200
|
- **`checkDrift()` is the post-migrate verification** (live catalog vs the ledger just written, asked
|
|
204
201
|
by `@ultimat3/cli`'s `runMigrations`), returned never thrown. The OTHER `X_DB_DRIFT` is the CLI's
|
|
205
202
|
`checkSourceDrift`. Neither grows the other's half.
|
|
206
|
-
- `compareTable` compares existence
|
|
207
|
-
|
|
203
|
+
- `compareTable` compares existence, **nullability** (the DECLARED key's columns excluded) and the
|
|
204
|
+
**primary key** in column order (`changed-primary-key`, fix = one `psql -c` of the pair; `primary-key.ts`
|
|
205
|
+
holds `x db gen`'s arm, its `drop not null`s, its two refusals). The type is not compared.
|
|
208
206
|
- **A missing CHECK is drift, compared by NAME**: `TableDescription.checks` (declared: name and
|
|
209
207
|
expression) vs `TableDescription.checkNames` (catalog: `conname` for `contype = 'c'`, always written
|
|
210
|
-
by `
|
|
208
|
+
by `introspectSchema()`, `[]` included). Only the declared side is judged; never a `changed-check`.
|
|
211
209
|
`drift-check.live.test.ts`.
|
|
212
210
|
- `compareTable` judges declared indexes (`missing-index`, `changed-index` over method, column list,
|
|
213
211
|
uniqueness, predicate presence and direction; `asc` normalises to `null`); never the predicate text.
|
|
214
212
|
- `compareForeignKeys` matches on where a key points (`foreignKeyTarget`, the one copy) and compares
|
|
215
|
-
`onDelete` through `onDeleteRule` (`changed-foreign-key
|
|
216
|
-
-
|
|
213
|
+
`onDelete` through `onDeleteRule` (`changed-foreign-key`; its fix, `changed-column`'s and
|
|
214
|
+
`missing-check`'s are one `psql -c` too — `repair()`, schema-scoped off `public`).
|
|
215
|
+
- `introspectSchema()` reads index columns in key order (`indkey`) and a foreign key's two column lists
|
|
217
216
|
together (`unnest(a, b) with ordinality`), pinned by `introspect-embedded.test.ts`.
|
|
218
|
-
- **`appTables()`** excludes
|
|
217
|
+
- **`appTables()`** excludes all of `x_` for drift; `introspectSchema()` alone excludes
|
|
219
218
|
`x_migrations` by default. **`app-relation.ts`**: `nonAppRelations(client, schema)` — extension
|
|
220
219
|
ownership from `pg_depend` (`deptype = 'e'`) plus views, materialised views and foreign tables —
|
|
221
220
|
merged into `excluded` unconditionally.
|
|
222
|
-
- **`unexpectedTable`'s `fix:` never names `x db gen`**:
|
|
223
|
-
(
|
|
224
|
-
- **`dbDrift()` lives in `drift-errors.ts`** (it needs `shellInertIdentifier`)
|
|
225
|
-
|
|
226
|
-
`packages/entity/src/errors.test.ts`. `errors.ts` registers `DB_ERROR_TITLES` unconditionally.
|
|
221
|
+
- **`unexpectedTable`'s `fix:` never names `x db gen`**: `psql -c '\d "T"'`, commented with the two
|
|
222
|
+
repairs (claim it with `create table if not exists`, or drop it).
|
|
223
|
+
- **`dbDrift()` lives in `drift-errors.ts`** (it needs `shellInertIdentifier`) and is the only one —
|
|
224
|
+
`@ultimat3/entity`'s copy is deleted. `errors.ts` registers `DB_ERROR_TITLES` unconditionally.
|
|
227
225
|
- `drift-findings.ts` holds every `DriftDifference` constructor and `DriftKind`; `drift.ts` keeps the
|
|
228
226
|
comparisons.
|
|
229
227
|
|
|
@@ -231,16 +229,17 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
|
|
|
231
229
|
|
|
232
230
|
- **Its own entry: `@ultimat3/db/schema-dump`** (`schema-dump-entry.ts`). The barrel is in every
|
|
233
231
|
role's boot graph and must evaluate none of this family — `schema-dump-entry.test.ts`.
|
|
234
|
-
- **Two readings of one catalog, never mixed.** `
|
|
235
|
-
vocabulary a snapshot is diffed in. `introspectCatalog()` (`catalog.ts`; queries
|
|
236
|
-
`catalog-objects.ts`;
|
|
232
|
+
- **Two readings of one catalog, never mixed.** `introspectSchema()` → `SchemaDescription`, the entity
|
|
233
|
+
vocabulary a snapshot is diffed in. `introspectCatalog()` (`catalog.ts`; queries: `catalog-relations.ts`,
|
|
234
|
+
`catalog-objects.ts`; pure fold: `catalog-fold.ts`) → `CatalogDescription`, Postgres' own `pg_get_*def` text, compared only to
|
|
237
235
|
itself. A catalog spelling on a `SchemaDescription` field is the `checks`/`checkNames` mistake.
|
|
238
236
|
- **Sorted in JS** (`byCodeUnit`), never `order by` and never `localeCompare`: collations differ.
|
|
239
237
|
- **`renderSchemaDump()` is pure**; `schema-dump-table.ts` spells one table. `quoted()` escapes any
|
|
240
238
|
catalog name — `identifier()` refuses whitespace and `"`, which a migration may have created.
|
|
241
239
|
- **`x_` owner → `framework/` twin** (`FRAMEWORK_TABLE_PREFIX`). No list is handed in.
|
|
242
|
-
- **What cannot be rendered is named** (`unrenderedRows`, a closed list → `unrendered.sql
|
|
243
|
-
|
|
240
|
+
- **What cannot be rendered is named** (`unrenderedRows`, a closed list → `unrendered.sql`; a trigger
|
|
241
|
+
on an unrendered relation is named by the fold). A kind rendered later leaves that list in the
|
|
242
|
+
same diff. Never render a partition as a plain table. `generated` carries `stored`/`virtual`.
|
|
244
243
|
- **`loadSchemaDump()`**: one transaction, `check_function_bodies` off, a savepoint per file, and
|
|
245
244
|
only `42P01`/`42883`/`42704` are retried. Its refusal is `X_SCHEMA_DUMP_DRIFT` (`dump-drift.ts`).
|
|
246
245
|
- **`object-drift.ts`**: `unexpectedObjects(live, expected)` compares IDENTITY (kind, table, name,
|
|
@@ -253,16 +252,15 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
|
|
|
253
252
|
by `pgliteExtensionExport` before it reaches a specifier; `plpgsql` is built in, never missing.
|
|
254
253
|
Only a not-found import is `missing`; a bundle that throws is `X_DB_UNAVAILABLE`.
|
|
255
254
|
- **`pglite-snapshot.ts`**: `snapshotDir` makes a `memory://` boot a restore. Key = PGlite version
|
|
256
|
-
(
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
- **One embedded boot** serves every database-backed dump test: `schema-dump.test.ts`.
|
|
255
|
+
(unreadable = no cache), never the extension set. One file, checksum in its header; unsound or
|
|
256
|
+
unopenable → deleted and rebuilt; unreadable → a miss. Temp name + `rename`. Uncompressed.
|
|
257
|
+
- **Embedded boots for dump tests**: `schema-dump.test.ts` and `schema-dump-fidelity.test.ts`.
|
|
260
258
|
|
|
261
259
|
## Branches, replicas, read-only
|
|
262
260
|
|
|
263
|
-
- **`reapBranches` sweeps branches of THIS database**: the marker is `ultimate:branch:<base>:<iso
|
|
264
|
-
|
|
265
|
-
|
|
261
|
+
- **`reapBranches` sweeps branches of THIS database**: the marker is `ultimate:branch:<base>:<iso>`,
|
|
262
|
+
split on the ISO tail; an older one-segment marker or an unparseable `createdAt` is skipped, never
|
|
263
|
+
dropped. `@ultimat3/cli`'s `ls`/`drop` scope by name prefix.
|
|
266
264
|
- **Read replicas are opt-in twice**: a pool when `DATABASE_REPLICA_URL` names one
|
|
267
265
|
(`default-client.ts`), and a read offered only inside `withReplicaReads(fn)` (`replica-scope.ts`);
|
|
268
266
|
read-your-writes is `ReplicaScope.wrote`, never a request-id map. **`withTransaction` is on the
|
|
@@ -287,7 +285,7 @@ Gotchas:
|
|
|
287
285
|
|
|
288
286
|
- `exactOptionalPropertyTypes` — declare optional fields as `x?: T | undefined`.
|
|
289
287
|
- `noUncheckedIndexedAccess` — array reads are `T | undefined`; `chunks[i] ?? ''` everywhere.
|
|
290
|
-
- Tests use `
|
|
288
|
+
- Tests use `recordingClient()` + `setDbClient()`; no test needs a live database.
|
|
291
289
|
- A test that must prove a pin came back uses `reservableOver()` (`fake-reservable.ts`), never a copy.
|
|
292
290
|
- `ALTER DEFAULT PRIVILEGES` is scoped to an object's creator, so layer 1 covers future tables only for
|
|
293
291
|
the roles in `creators` (default: the connected user).
|
package/README.md
CHANGED
|
@@ -29,7 +29,8 @@ await withTransaction(async (tx) => {
|
|
|
29
29
|
| `shellInertIdentifier()` | `As of 2026-08-26`: a quoted identifier that is also inert wherever a human PASTES it — or `null`. The one screen a catalog name goes through before it reaches a `fix:`. `identifier()` answers about SQL and **accepts** a backtick and a `$`, which are exactly what a shell substitutes inside double quotes, so a column called `$(id)` inside `x db gen "add $(id)"` runs `id` on paste |
|
|
30
30
|
| `db()` / `baseClient()` / `setDbClient()` | the ambient client; `db()` returns the open tx if any |
|
|
31
31
|
| `DbTx.origin` | `As of 2026-08`: the client the transaction was **opened on** — `options.client` or `baseClient()`, never the reservation it runs statements through. `@ultimat3/entity` compares a pinned repository's client against it, so a pinned repo joins its own shard's transaction instead of being refused |
|
|
32
|
-
| `withTransaction()` / `currentTx()` | transaction scope; `currentTx()` is the outbox seam. `{ retry: n }` (`As of 2026-08`) re-runs `fn` from the top on a `40001`/`40P01` and on nothing else — default 0, so `fn` must be idempotent before you ask for it. Each re-run **waits first**, `As of 2026-08-23`: exponential from 10ms, capped at 500ms, full jitter (`@ultimat3/core`'s `backoffDelay`). A budget of 0 waits not at all |
|
|
32
|
+
| `withTransaction()` / `currentTx()` | transaction scope; `currentTx()` is the outbox seam. `{ retry: n }` (`As of 2026-08`) re-runs `fn` from the top on a `40001`/`40P01` and on nothing else — default 0, so `fn` must be idempotent before you ask for it. Each re-run **waits first**, `As of 2026-08-23`: exponential from 10ms, capped at 500ms, full jitter (`@ultimat3/core`'s `backoffDelay`). A budget of 0 waits not at all. **A transaction the server aborted is reported as one**, `As of 2026-10-02` → [Transactions that end badly](#transactions-that-end-badly) |
|
|
33
|
+
| `liveTxConnection()` | the connection of the transaction still **open** on this async context, or `undefined`. `currentTx()` keeps answering a finished scope's handle to a promise chain the body forgot to await; this is the one that says whether a statement sent now is really inside a transaction. `@ultimat3/action` reads it before binding an idempotency settlement to a commit |
|
|
33
34
|
| `sqlState()` / `sqlStateCode()` / `isRetryableState()` / `SQLSTATE` | `As of 2026-08`: the SQLSTATE a driver error carries, and the closed table from it to a code. `Bun.SQL` puts it on `errno`; PGlite puts it on `code`; **one** reader answers for both |
|
|
34
35
|
| `migrate()` / `rollback()` / `readLedger()` | the `x_migrations` ledger |
|
|
35
36
|
| `statementsOf()` | `As of 2026-08`: a SQL script → the statements a driver sends one at a time. One send is one statement, so `migrate()` splits with this — a `;` inside a literal, an identifier, a dollar-quoted body or a comment is data |
|
|
@@ -37,7 +38,7 @@ await withTransaction(async (tx) => {
|
|
|
37
38
|
| `replicatedClient()` / `ReplicaStats` / `REPLICA_URL_ENV` | `As of 2026-08-24`: one `DbClient` over a primary and a standby. `baseClient()` builds one when `DATABASE_REPLICA_URL` is set and the single-pool client when it is not |
|
|
38
39
|
| `INDEX_METHODS` / `IndexMethod` / `indexMethodOf()` / `indexMethodSql()` / `declaredMethod()` / `isIndexMethod()` | `As of 2026-08-24`: an index's access method — `btree` or `gin`, closed. Absent is `btree`, the live side is read open (whatever `pg_am` said), and the DDL literal is re-derived from the set rather than spliced from the input |
|
|
39
40
|
| `isPlainRead()` | `As of 2026-08-24`: whether a statement may leave the primary. An allow-list — everything it cannot vouch for is the primary's |
|
|
40
|
-
| `checkDrift()` / `diffSchema()` / `
|
|
41
|
+
| `checkDrift()` / `diffSchema()` / `assertNoSchemaDrift()` | drift, with a `--json` report. `checkDrift()` is the **post-migrate verification** — the live database against the ledger: columns, declared indexes (access method `As of 2026-08-24`, columns, uniqueness, direction, and whether a predicate is there at all — never its text), declared CHECK constraints by NAME (`missing-check`, `As of 2026-08-25` — never a predicate, which the catalog answers rewritten) and declared foreign keys, matched on where the key points and not on its constraint name, with the `on delete` rule compared through one normalisation `As of 2026-08-19` |
|
|
41
42
|
| `declaredSchema()` / `expectedSchema()` | `As of 2026-08`: the schema the migrations write down, or `undefined` when the newest one carries no snapshot — never an older snapshot standing in for it |
|
|
42
43
|
| `parseSnapshot()` | `As of 2026-08`: a `<id>.snapshot.json` sidecar validated to the last nested field, or `undefined`. `{"tables":[null]}` is valid JSON and is not a schema |
|
|
43
44
|
| `snapshotJson()` | `As of 2026-08`: the sidecar's **bytes** — the JSON Biome would have printed, trailing newline included. The one writer of a `<id>.snapshot.json`, because `JSON.stringify(…, null, 2)` is not formatter-clean and an app's `lint` step rejected the file `x db gen` had just written |
|
|
@@ -48,18 +49,18 @@ await withTransaction(async (tx) => {
|
|
|
48
49
|
| `declaredChecks()` / `checkClauses()` / `checkPlan()` / `columnChecks()` / `columnCheckName()` / `columnNamesConstraint()` | `As of 2026-08-25`: **every** CHECK a table declares — a column's own (`enumerated()`'s value set, `tz()`'s IANA whitelist, `locale()`'s tags, money's currency pattern and scale bound) and an invariant's — on ONE list, so `createTable`, `diffTable` and `snapshotOf` agree about what exists. A column's check reached `create table` **inline and anonymous** and nothing else: the snapshot recorded none and the diff had no arm, so a value added to `enumerated()` generated no migration and a regenerated ENUM column came back as bare `text`. The name is `<table>_<column>_check` because that is the name **Postgres itself mints** for the old anonymous form — measured — so the repair lands on the constraint an already-generated database is holding; `checkPlan` emits `drop constraint if exists` before the `add` for exactly that column, because a bare add is `42710` there and a no-op everywhere else |
|
|
49
50
|
| `defaultExpression()` / `ColumnDefaultLike` | `As of 2026-08-25`: a column's `default` as SQL. A DECLARED default (`{ kind: 'value', value }`) wins; `gen_random_uuid()` and `now()` stay as the inference for a description that carries only `hasDefault` |
|
|
50
51
|
| `unrenderedOf()` / `unrenderedComment()` / `UnrenderedDeclaration` | `As of 2026-08-25`: what the generator could **not** write, on `GeneratedMigration.unrendered` and as a `-- UNRENDERED` block at the top of a non-empty `up`. A generator that emits less than the declaration in silence is the defect the whole file exists against, and `x verify`'s `drift` step reads a source hash — it never reads the SQL, so the loss was green. **`unrenderedOf(entities, current)` takes the recorded schema**, required and nullable: a rule declared as an `assert` reaches no SQL by design and is no loss on its own, but one whose CHECK a previous migration RECORDED is dropped by this run and reported by nothing — five in `examples/dummy`, and `@ultimat3/cli`'s `repairFix` then offered `x db gen "drop <name>"` as the repair for the loss that command performs |
|
|
51
|
-
| `destructiveStatements()` / `hasDestructiveMarker()` / `
|
|
52
|
+
| `destructiveStatements()` / `hasDestructiveMarker()` / `isDestructiveMigration()` / `DESTRUCTIVE_MARKER` | `As of 2026-08`: the destructive-SQL rail — does this `up` drop, truncate or retype, and does the file declare it with `-- destructive: true`? One classifier, read by `x db gen` when it writes the marker and by `x verify` when it demands one |
|
|
52
53
|
| `stripSqlNoise()` | comments, literals, dollar-quoted bodies and quoted identifiers blanked **in source order**, so a reader sees the operation and not the prose. Shared by `readOnlyQuery()` and the destructive rail |
|
|
53
|
-
| `
|
|
54
|
-
| `introspectCatalog()` / `CatalogDescription` / `emptyCatalog()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: the WHOLE schema in the catalog's own spelling — extensions, enum and domain types, sequences, tables, indexes, foreign keys, views, functions, triggers — sorted in code-unit order, plus `unrendered`: what exists and the dump cannot spell. Comparable only to another reading of itself; `
|
|
54
|
+
| `introspectSchema()` | live schema → `SchemaDescription`. **App tables only**, `As of 2026-08-24`: a relation an extension owns (`pg_depend`, `deptype = 'e'`) and anything that is not an ordinary or partitioned table are excluded before the fold, and an explicit `exclude` cannot bring them back |
|
|
55
|
+
| `introspectCatalog()` / `CatalogDescription` / `emptyCatalog()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: the WHOLE schema in the catalog's own spelling — extensions, enum and domain types, sequences, tables, indexes, foreign keys, views, functions, triggers — sorted in code-unit order, plus `unrendered`: what exists and the dump cannot spell. Comparable only to another reading of itself; `introspectSchema()` stays the entity-vocabulary reading a snapshot is diffed in |
|
|
55
56
|
| `renderSchemaDump()` / `SchemaDumpFile` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: a catalog → the schema dump's files. Pure and byte-deterministic. [The schema dump](#the-schema-dump) |
|
|
56
57
|
| `loadSchemaDump()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: build a schema from those files, in one transaction, retrying a file that names something not created yet |
|
|
57
58
|
| `compareSchemaDump()` / `reloadDifferences()` / `schemaDumpDrift()` / `schemaDumpDifferenceOf()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: `X_SCHEMA_DUMP_DRIFT` — committed files against rendered ones in both directions, and load-equals-replay as a comparison |
|
|
58
|
-
| `unexpectedObjects()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: the triggers, functions, views, types and sequences a live catalog holds and an expected one does not, as `unexpected-object` drift. Identity, never definition text |
|
|
59
|
+
| `unexpectedObjects()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: the triggers, functions, views, types and sequences a live catalog holds and an expected one does not, as `unexpected-object` drift. Identity, never definition text. Its `fix:` is one `psql -c` command that prints the object's definition, qualified by the catalog's schema — `\d+` a view, `\dT+` an enum, `\dD+` a domain, `\d` a sequence or a trigger's table, `pg_get_functiondef` by namespace for a function — with the repair as its comment |
|
|
59
60
|
| `PgliteOptions.extensions` / `linkPgliteExtensions()` | `As of 2026-10`: Postgres extensions to link at boot, by name — a list, or a function for a caller whose list is read from disk. `linkPgliteExtensions(names)` answers `{ linked, missing }` without booting anything: `missing` is what the installed PGlite ships no bundle for, which is how `@ultimat3/cli` decides a replay needs a real Postgres. A missing name is skipped at boot and refused by `create extension` itself |
|
|
60
61
|
| `PgliteOptions.snapshotDir` | `As of 2026-10`: a directory for the post-`initdb` snapshot, so an in-memory boot is a restore (~0.4 s against ~2.7 s). Keyed on the PGlite version alone — `initdb` never sees a linked extension, so one snapshot serves every set; checksummed, never trusted, written by temp-name-then-rename. Ignored for a data directory on disk |
|
|
61
62
|
| `createBranch()` / `dropBranch()` / `reapBranches()` | copy-on-write branch databases. `As of 2026-08-19` the marker comment records the **base** as well as the instant (`ultimate:branch:<base>:<iso>`, on `BranchInfo.base`), and `reapBranches()` sweeps only branches of the database it is connected to — one Postgres hosting two Ultimate apps used to mean one app's nightly reap dropped the other's branches. A pre-3.x marker records no base and is skipped, never dropped |
|
|
62
|
-
| `
|
|
63
|
+
| `pgliteClient()` / `branchPglite()` | the embedded database — Postgres in this process |
|
|
63
64
|
| `ensureReadOnlyRole()` / `grantReadOnlySql()` / `READONLY_ROLE` | a `NOLOGIN`, SELECT-only Postgres role — layer 1 of `db.query`'s defence |
|
|
64
65
|
| `readOnlyQuery()` / `READONLY_TIMEOUT_MS` | one statement inside `BEGIN READ ONLY` with a statement timeout — layer 2 |
|
|
65
66
|
| `setStatementObserver()` / `statementObserver()` | `As of 2026-08`: one event **and one `db.<verb>` span** per settled statement, both drivers; uninstalled is one branch |
|
|
@@ -67,7 +68,7 @@ await withTransaction(async (tx) => {
|
|
|
67
68
|
| `withStatementAttribution()` / `statementAttribution()` | `As of 2026-08`: the `{ entity, op }` pair on `StatementEvent.attribution`, scoped exactly like `expectedQueryLoop()` — `@ultimat3/entity`'s `postgresRepo` is the one producer |
|
|
68
69
|
| `STATEMENT_ATTRIBUTE` | `As of 2026-08`: `db.statement`, the OTel attribute each span carries its text under — declared here, read by `x dev`'s timeline |
|
|
69
70
|
| `statementFingerprint()` / `statementKind()` / `statementVerb()` | `As of 2026-08`: what shape a statement is — `entity.op` when attributed else its own collapsed text, read or write from the leading verb. One rule, so two detectors group identically |
|
|
70
|
-
| `
|
|
71
|
+
| `recordingClient()` | in-memory `DbClient` that records SQL, for tests |
|
|
71
72
|
|
|
72
73
|
## `sql` is parameters-only
|
|
73
74
|
|
|
@@ -75,13 +76,37 @@ String interpolation is how every SQL injection ships, and an agent writing SQL
|
|
|
75
76
|
trusted to remember the difference between a value and a fragment. So:
|
|
76
77
|
|
|
77
78
|
- scalars (`string`, `number`, `boolean`, `bigint`, `Date`, `Uint8Array`, arrays, `null`) become
|
|
78
|
-
`$1..$n` and never touch `.text
|
|
79
|
+
`$1..$n` and never touch `.text`. A value that cannot be SENT — an Invalid Date, a ragged array —
|
|
80
|
+
is `X_INVARIANT` before the driver is called, never `X_DB_UNAVAILABLE`; a `Uint8Array` inside an
|
|
81
|
+
array is one `bytea` element. Both drivers refuse alike;
|
|
79
82
|
- a nested fragment is spliced and its parameters are renumbered;
|
|
80
83
|
- **anything else throws `X_SQL_UNSAFE`** — including an object shaped like a `SqlFragment` that
|
|
81
84
|
`sql`/`raw` did not produce;
|
|
82
85
|
- `raw(trusted)` is the one audited escape hatch, `identifier(name)` the safe way to interpolate
|
|
83
86
|
a table or column, `literal(text)` for utility statements that reject bound parameters.
|
|
84
87
|
|
|
88
|
+
## Transactions that end badly
|
|
89
|
+
|
|
90
|
+
`As of 2026-10-02`. Postgres aborts the WHOLE transaction on any statement error and answers the
|
|
91
|
+
`COMMIT` that follows with the tag `ROLLBACK` and no error. `withTransaction` used to resolve on
|
|
92
|
+
that, fire `onCommit`, and store nothing.
|
|
93
|
+
|
|
94
|
+
| Situation | What happens |
|
|
95
|
+
|---|---|
|
|
96
|
+
| the body catches a failed statement and returns | `X_DB_TRANSACTION_ABORTED`, cause = the first failing statement; `ROLLBACK`, `onRollback` undos run, `onCommit` never fires |
|
|
97
|
+
| the fallible statement sits in a nested `withTransaction` and THAT is caught | only the savepoint rolls back; the outer scope commits. This is the fix the error names |
|
|
98
|
+
| a nested body swallows a failed statement | the nested call rejects with `X_DB_TRANSACTION_ABORTED`; its savepoint is rolled back and the outer scope is usable again |
|
|
99
|
+
| `ROLLBACK TO SAVEPOINT` itself fails | the nested call still rejects with the body's error; the root is marked aborted and its `COMMIT` is refused, never sent |
|
|
100
|
+
| two nested scopes under `Promise.all` | run one after the other, per parent scope — savepoints are a stack, and interleaved ones destroyed each other |
|
|
101
|
+
| a nested scope waits past `siblingWaitMs` (default `SIBLING_SCOPE_WAIT_MS`, 30 s; `0` = no deadline) for its sibling | `X_DB_SIBLING_SCOPE_TIMEOUT`, naming the parent and the savepoint holding the turn. The shape it exists for: a body awaiting a sibling started after it, which waits for itself — a permanent hang before the deadline. The waiter never opened and its place is handed on |
|
|
102
|
+
| any `COMMIT` answered `ROLLBACK`, on either driver | `X_DB_TRANSACTION_ABORTED` from the statement funnel (`commit-tag.ts`) — covers an abort the scope never saw and a hand-written `COMMIT` |
|
|
103
|
+
| `COMMIT` rejects with a SQLSTATE (a deferred constraint, `40001`) | rolled back: the server's error surfaces, undos run |
|
|
104
|
+
| `COMMIT` rejects with NO SQLSTATE | `X_DB_COMMIT_UNKNOWN`: neither `onCommit` nor `onRollback` runs, and it is never retried |
|
|
105
|
+
| a nested scope passes `isolation`, `readOnly: true`, `deferrable: true`, or a `client` other than the root's | `X_INVARIANT` — a savepoint can honour none of them; `retry` was already refused |
|
|
106
|
+
|
|
107
|
+
Only a failure carrying a SQLSTATE marks the scope aborted: a refusal raised before the send left
|
|
108
|
+
the transaction untouched.
|
|
109
|
+
|
|
85
110
|
## Read-only access for anything an LLM drives
|
|
86
111
|
|
|
87
112
|
`As of 2026-07`: a bug in one defence must not become a write, so `db.query` on the MCP dev
|
|
@@ -172,7 +197,7 @@ production traffic is routed.
|
|
|
172
197
|
|
|
173
198
|
## The drift contract
|
|
174
199
|
|
|
175
|
-
`checkDrift()` compares `
|
|
200
|
+
`checkDrift()` compares `introspectSchema()` against the snapshot the newest applied migration carries —
|
|
176
201
|
`expectedSchema(migrations, ledger)`. `declaredSchema(migrations)` is the same read with the ledger
|
|
177
202
|
left out: the schema the files *declare*, applied or not, which is what `x db gen` diffs the app's
|
|
178
203
|
entities against so generation needs no database at all. One implementation, two callers — a
|
|
@@ -190,7 +215,7 @@ A migration the ledger has not recorded is **not** drift: `expectedSchema` reads
|
|
|
190
215
|
subset, so a database that simply has not migrated yet is pending, not divergent. Neither is a
|
|
191
216
|
table in the `x_` namespace — `x_migrations`, the queue's tables, the outbox and every
|
|
192
217
|
`@ultimat3/auth` table are created by `create table if not exists` at boot and appear in no
|
|
193
|
-
snapshot, so `appTables()` drops them before the diff. `
|
|
218
|
+
snapshot, so `appTables()` drops them before the diff. `introspectSchema()` keeps its own narrower
|
|
194
219
|
exclusion (the ledger alone), reserving `x_users` for a schema view that wants it.
|
|
195
220
|
|
|
196
221
|
**A CHECK the catalog no longer holds is drift, `As of 2026-08-25` — by NAME.**
|
|
@@ -199,15 +224,16 @@ back as `CHECK ((status = ANY (ARRAY['draft'::text, 'published'::text])))`), so
|
|
|
199
224
|
read as `conname` alone and lands on `TableDescription.checkNames`, a **separate field** from the
|
|
200
225
|
declaration's `checks`. Only the declared side is judged, so a NOT NULL, an `enumerated()` column's
|
|
201
226
|
old anonymous form and an extension's own constraint are all silent; a declared one the catalog
|
|
202
|
-
does not hold is `missing-check`, whose `fix:` is the `add constraint` statement
|
|
203
|
-
|
|
227
|
+
does not hold is `missing-check`, whose `fix:` is the `add constraint` statement as one
|
|
228
|
+
`psql "$DATABASE_URL" -c '…'` command against that database, because the migration that declares
|
|
229
|
+
it is already in the ledger and `x db migrate` would apply nothing. There is
|
|
204
230
|
no `changed-check`: presence is a boolean, a predicate is text, and normalising the text is an
|
|
205
231
|
expression parser competing with the server's.
|
|
206
232
|
|
|
207
233
|
**Nor is a relation an extension owns, `As of 2026-08-24`.** `create extension pg_stat_statements`
|
|
208
234
|
in `public` is the CNPG, RDS, Supabase and Neon default, and its view read as `unexpected-table`
|
|
209
235
|
with `x db gen "add pg_stat_statements"` as the fix — so every deploy failed terminally and the fix
|
|
210
|
-
would have written an extension's internal view into the app's migration set. `
|
|
236
|
+
would have written an extension's internal view into the app's migration set. `introspectSchema()` now
|
|
211
237
|
excludes every relation Postgres records as extension-owned (`pg_depend`, `deptype = 'e'`), which is
|
|
212
238
|
ownership rather than a name: a `pg_*` prefix rule covers that view and misses `postgis`'
|
|
213
239
|
`spatial_ref_sys`. Views, materialised views and foreign tables go with them — no snapshot records
|
|
@@ -226,18 +252,41 @@ X_DB_DRIFT: schema differs from migrations
|
|
|
226
252
|
|---|---|---|
|
|
227
253
|
| live column, no migration | `table "T" has column "C" not present in any migration` | `x db gen "add C"` — the name goes through `shellInertIdentifier()`, and one it refuses is left OUT of the command rather than escaped into it (`x db gen "add the undeclared column"`, the name in the cause) |
|
|
228
254
|
| migrated column, not live | `table "T" is missing column "C" that migrations declare` | `x db migrate` |
|
|
229
|
-
| live table, no migration | `table "T" is not present in any migration` | a `create table if not exists` in a migration, then `x db migrate
|
|
255
|
+
| live table, no migration | `table "T" is not present in any migration` | `psql "$DATABASE_URL" -c '\d "T"'` — one harmless command that shows the table — with the two repairs as its comment: a `create table if not exists` in a migration, then `x db migrate`, or `drop table` where nothing owns it. Never `x db gen`, which diffs a table nothing declares against nothing and writes no file (issue #345). The name goes through `shellInertIdentifier()`, and one it refuses leaves `psql "$DATABASE_URL" # <the steps>` |
|
|
230
256
|
| migrated table, not live | `table "T" is declared by migrations but does not exist` | `x db migrate` |
|
|
231
257
|
| index rebuilt differently | `index "I" on "T" covers (…)` / `is unique` / `is descending` / `is partial`, `not what migrations declare` | `x db migrate` |
|
|
232
|
-
| foreign key, rule moved | `foreign key on "T" (C) to "R" is on delete cascade, not what migrations declare` | the
|
|
258
|
+
| foreign key, rule moved | `foreign key "K" on "T" (C) to "R" is on delete cascade, not what migrations declare` — `K` is the constraint the database holds | `psql "$DATABASE_URL" -c '<the drop constraint + add constraint pair>'` — one command, against the drifted database — then `x db migrate` |
|
|
259
|
+
| column nullability differs (`changed-column`) | `table "T" allows NULL in column "C" that migrations declare not null` / `forbids NULL in column "C" that migrations declare nullable` | `psql "$DATABASE_URL" -c 'alter table "T" alter column "C" set not null;'` (or `drop not null`) — one command, then `x db migrate`. A table read from a schema other than `public` gets `set search_path = "<schema>";` ahead of the statement, in the same `psql -c` word, so it and every table it references resolve there. A name `shellInertIdentifier()` refuses degrades every one of these three to `psql "$DATABASE_URL" # <the steps>`: still a command that runs, with no name in it |
|
|
260
|
+
| append-only trigger gone (`missing-append-only-trigger`, `As of 2026-10-06`) — raised as `X_APPEND_ONLY_TRIGGER_MISSING`, the one kind with its own code | `table "T" is declared appendOnly, and its trigger ultimate_append_only is missing or disabled` — a disabled (`D`) or replica-only (`R`) trigger counts as missing, because an ordinary session fires neither | `psql "$DATABASE_URL" -c '<create or replace function …; drop trigger if exists "ultimate_append_only" on "T"; create trigger "ultimate_append_only" …>'` against the drifted database, then `x db migrate`. Drop-if-exists first: a disabled trigger still exists, and a bare `create trigger` would fail on it (`42710`) |
|
|
261
|
+
| primary key differs (`changed-primary-key`, `As of 2026-10-02`) | `table "T" has primary key (id) as constraint "T_pkey", and migrations declare (slug)` — the constraint named is the one the DATABASE holds — compared in column ORDER; `has no primary key` when the database holds none | `psql "$DATABASE_URL" -c '<drop constraint "<the live key>"; add constraint "T_pkey" primary key (…)>'` — one command, against the drifted database — then `x db migrate`. Nullability is skipped for the DECLARED key's columns only |
|
|
233
262
|
|
|
234
|
-
`checkDrift()` returns every difference; `
|
|
263
|
+
`checkDrift()` returns every difference; `assertNoSchemaDrift()` throws the first. `x db migrate` renders
|
|
235
264
|
them all as findings and exits non-zero; a `ROLE=migrate` container throws the first one
|
|
236
|
-
(`
|
|
265
|
+
(`assertNoSchemaDrift`, in `runRole`) and exits non-zero too, because the release phase has one channel —
|
|
237
266
|
the exit code — and a deploy that rolled on past a schema nobody can reconstruct is the failure
|
|
238
267
|
drift exists to catch. There is no `x db drift`, and `x verify`'s `drift` step is the *source*
|
|
239
268
|
detector (`checkSourceDrift`), which needs no database and never calls this.
|
|
240
269
|
|
|
270
|
+
## Append-only tables
|
|
271
|
+
|
|
272
|
+
`EntityDescriptionLike.appendOnly` (from `entity({ appendOnly: true })`) is a trigger, written by
|
|
273
|
+
`x db gen` (`generate-append-only.ts`) and recorded on the snapshot as `appendOnly: true`:
|
|
274
|
+
|
|
275
|
+
```sql
|
|
276
|
+
create or replace function "ultimate_refuse_append_only"() returns trigger language plpgsql as $append_only$ begin raise exception 'X_ENTITY_APPEND_ONLY: % on %.% is refused, the table is append-only', tg_op, tg_table_schema, tg_table_name using errcode = '23001', hint = '…'; end; $append_only$;
|
|
277
|
+
create trigger "ultimate_append_only" before update or delete on "ledger" for each row execute function "ultimate_refuse_append_only"();
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
| | |
|
|
281
|
+
|---|---|
|
|
282
|
+
| Names | fixed: the trigger is `ultimate_append_only` on every table (a trigger name is per table, so no 63-byte truncation), the function one shared `ultimate_refuse_append_only()` |
|
|
283
|
+
| Refuses | UPDATE, DELETE, and the update arm of `insert … on conflict do update`. Not `truncate` — no row write, and `destructive.ts` already gates it |
|
|
284
|
+
| The error | message leads with `X_ENTITY_APPEND_ONLY`, SQLSTATE `23001` (`restrict_violation`, class 23: an integrity rule refused it, so never retried) |
|
|
285
|
+
| On / off | adding `appendOnly` emits the function (once per migration) and the trigger (drop-if-exists first on an existing table), `down` drops the trigger; removing it drops the trigger, `down` restores both. A dropped table takes its trigger with it; the function is never dropped |
|
|
286
|
+
| NOT NULL with no backfill | a NEW NOT NULL column needs a default (or `.nullable()`), and an EXISTING column cannot be turned NOT NULL: `x db gen` refuses both (`X_MIGRATION_APPEND_ONLY_BACKFILL`) rather than emit the usual `-- backfill …, then: set not null` note — that backfill is an UPDATE the trigger refuses, and a default fills no NULL already stored (`generate-append-only-column.test.ts`) |
|
|
287
|
+
| Engines | Postgres and PGlite alike (`generate-append-only.live.test.ts`, `generate-append-only-embedded.test.ts`) |
|
|
288
|
+
| Drift | `introspectSchema()` writes `triggerNames` (enabled, non-internal) on every table — the catalog half; `appendOnly` is the snapshot half, never read from the catalog |
|
|
289
|
+
|
|
241
290
|
## The schema dump
|
|
242
291
|
|
|
243
292
|
**Imported from `@ultimat3/db/schema-dump`, never from the barrel** (23.0.0): `introspectCatalog`,
|
|
@@ -271,7 +320,10 @@ object kind, numbered in the order a database is built in, one file per named ob
|
|
|
271
320
|
- A file over 500 lines (`SCHEMA_DUMP_MAX_LINES`, the `filesize` ceiling) is split `<name>.1.sql`, `<name>.2.sql`;
|
|
272
321
|
`loadSchemaDump()` joins the parts in numeric order.
|
|
273
322
|
- What cannot be spelled — partitions, inheritance, foreign tables, composite and range types,
|
|
274
|
-
aggregates, row-security policies
|
|
323
|
+
aggregates, row-security policies (and `force row level security`), rules, extended statistics,
|
|
324
|
+
a column's non-default `storage`, a materialized view created `with no data`, and a trigger on
|
|
325
|
+
a relation the dump does not create — is named in `unrendered.sql` as comments.
|
|
326
|
+
- A generated column is spelled `stored` or `virtual` (Postgres 18), as `attgenerated` says.
|
|
275
327
|
|
|
276
328
|
`loadSchemaDump()` runs kind by kind, the framework twin first, each file in a savepoint. A file
|
|
277
329
|
refused with `42P01`, `42883` or `42704` — "not created yet" — is retried after the rest; a pass
|
|
@@ -286,17 +338,17 @@ Which engine, where the files live, when they are written and what holds them is
|
|
|
286
338
|
|
|
287
339
|
## The embedded database
|
|
288
340
|
|
|
289
|
-
No `DATABASE_URL` means no Docker: `
|
|
341
|
+
No `DATABASE_URL` means no Docker: `pgliteClient()` runs Postgres as WASM inside this
|
|
290
342
|
process. The module is resolved on the first statement, never at import, so an image that only
|
|
291
343
|
ever talks to a managed Postgres never loads it.
|
|
292
344
|
|
|
293
345
|
```ts
|
|
294
|
-
const dev =
|
|
346
|
+
const dev = pgliteClient({ dataDir: pgliteDataDir(services.db.url) }); // or memory://
|
|
295
347
|
await dev.ping(); // pay the ~3s boot before serving
|
|
296
348
|
setDbClient(dev);
|
|
297
349
|
|
|
298
350
|
const branch = await branchPglite('feature_x', { from: '.x/pgdata' });
|
|
299
|
-
setDbClient(
|
|
351
|
+
setDbClient(pgliteClient({ dataDir: branch.dataDir }));
|
|
300
352
|
```
|
|
301
353
|
|
|
302
354
|
PGlite has no `CREATE DATABASE ... TEMPLATE`, so `branchPglite()` copies the data directory —
|
|
@@ -306,7 +358,7 @@ lands in a filesystem path, so an unvalidated one is traversal rather than a typ
|
|
|
306
358
|
|
|
307
359
|
### One session, so callers take turns
|
|
308
360
|
|
|
309
|
-
Embedded Postgres is a single session, not a pool. `
|
|
361
|
+
Embedded Postgres is a single session, not a pool. `pgliteClient()` is therefore
|
|
310
362
|
`ReservableClient`: `withTransaction()` and `readOnlyQuery()` pin it, and every other statement
|
|
311
363
|
waits for its turn. Without that pin two concurrent units of work each run `BEGIN` on the same
|
|
312
364
|
connection — the second `COMMIT` commits the first's rows and the first `ROLLBACK` finds no
|
|
@@ -371,9 +423,9 @@ A pooled statement cannot hold a subscription: the next one runs on another conn
|
|
|
371
423
|
clients hold ONE session beside the pool for it.
|
|
372
424
|
|
|
373
425
|
```ts
|
|
374
|
-
import {
|
|
426
|
+
import { postgresClient } from '@ultimat3/db';
|
|
375
427
|
|
|
376
|
-
const client =
|
|
428
|
+
const client = postgresClient({ url: 'postgres://localhost:5432/app_test' });
|
|
377
429
|
const subscription = await client.listen(
|
|
378
430
|
'x_jobs_wake',
|
|
379
431
|
(payload) => console.log('notified', payload),
|
|
@@ -407,7 +459,13 @@ migration inside its own transaction. It refuses **before applying anything** wh
|
|
|
407
459
|
the running one — another version owns the database; or
|
|
408
460
|
- an applied migration's `up` SQL no longer matches its recorded checksum.
|
|
409
461
|
|
|
410
|
-
|
|
462
|
+
**Except a rollback** (`As of 2026-10-05`, `migrate-rollback.ts`): when every unknown ledger row
|
|
463
|
+
sorts after the newest migration this build ships, and this build has nothing left to apply, the
|
|
464
|
+
ledger is an older image rolled back onto a newer build's schema. Nothing is applied, the rows are
|
|
465
|
+
reported in `ahead` and logged as `ultimate migrate ledger ahead of build`; any other unknown row is
|
|
466
|
+
still `X_MIGRATION_CONFLICT`.
|
|
467
|
+
|
|
468
|
+
Report (`--json`): `{ applied: [{ id, name, durationMs }], skipped: [id], durationMs, appVersion, ahead: [id] }`.
|
|
411
469
|
|
|
412
470
|
## A loop of queries that is deliberate says so
|
|
413
471
|
|
|
@@ -495,11 +553,14 @@ job the moment Postgres fails over.
|
|
|
495
553
|
| `X_DB_UNIQUE_VIOLATION` | `23505` — `fix:` names `upsertAll(rows, { onConflict: [...] })` and the constraint the server named |
|
|
496
554
|
| `X_DB_FOREIGN_KEY_VIOLATION` | `23503` |
|
|
497
555
|
| `X_DB_SERIALIZATION_FAILURE` | `40001` / `40P01`, and an exhausted `withTransaction(fn, { retry: n })` budget |
|
|
556
|
+
| `X_DB_TRANSACTION_ABORTED` | a statement failed inside a transaction and its error was caught; the server rolled the unit of work back. Also any `COMMIT` answered with the tag `ROLLBACK` |
|
|
557
|
+
| `X_DB_COMMIT_UNKNOWN` | `COMMIT` was sent and the connection failed before the answer: durable or rolled back, unknowable from here |
|
|
558
|
+
| `X_DB_SIBLING_SCOPE_TIMEOUT` | a nested `withTransaction` waited past `siblingWaitMs` for a sibling scope that never finished |
|
|
498
559
|
| `X_DB_STATEMENT_TIMEOUT` | `57014` — the statement ran past `statement_timeout` |
|
|
499
560
|
| `X_DB_LOCK_TIMEOUT` | `55P03` — it waited past `lock_timeout` for a lock it never got |
|
|
500
561
|
| `X_DB_POOL_EXHAUSTED` | `53300` / `53200`, or `reserve()` past `acquireTimeoutMs` |
|
|
501
562
|
| `X_DB_DRIFT` | live schema differs from migrations |
|
|
502
|
-
| `X_MIGRATION_CONFLICT` | ledger app-version fence or checksum mismatch |
|
|
563
|
+
| `X_MIGRATION_CONFLICT` | ledger app-version fence or checksum mismatch — never for rows that are all newer than the build (a rollback) |
|
|
503
564
|
| `X_MIGRATE_CONCURRENT` | another migrator still held the lock when the wait ran out |
|
|
504
565
|
| `X_MIGRATION_IRREVERSIBLE` | generated `down` would lose data |
|
|
505
566
|
| `X_SQL_UNSAFE` | non-bindable interpolation, or an unsafe identifier/branch name |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ultimat3/db",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "25.0.0",
|
|
4
4
|
"description": "Postgres access, transactions, migrations and drift detection",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -26,14 +26,14 @@
|
|
|
26
26
|
"LICENSE"
|
|
27
27
|
],
|
|
28
28
|
"engines": {
|
|
29
|
-
"bun": ">=1.4.
|
|
29
|
+
"bun": ">=1.4.2"
|
|
30
30
|
},
|
|
31
31
|
"scripts": {
|
|
32
32
|
"typecheck": "tsc --noEmit -p tsconfig.json",
|
|
33
33
|
"test": "bun test"
|
|
34
34
|
},
|
|
35
35
|
"dependencies": {
|
|
36
|
-
"@ultimat3/core": "
|
|
36
|
+
"@ultimat3/core": "25.0.0"
|
|
37
37
|
},
|
|
38
38
|
"peerDependencies": {
|
|
39
39
|
"@electric-sql/pglite": ">=0.5.0"
|