@ultimat3/db 23.0.0 → 25.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/CLAUDE.md +83 -85
  2. package/README.md +88 -27
  3. package/package.json +3 -3
  4. package/src/array-parameter.ts +38 -1
  5. package/src/bound-parameters.ts +22 -3
  6. package/src/catalog-fold.ts +4 -1
  7. package/src/catalog-objects.ts +29 -0
  8. package/src/catalog.ts +10 -2
  9. package/src/client.ts +2 -2
  10. package/src/column-alter.ts +9 -3
  11. package/src/commit-tag.ts +21 -0
  12. package/src/default-client.ts +4 -4
  13. package/src/dependent-view.ts +7 -5
  14. package/src/destructive.ts +1 -1
  15. package/src/drift-append-only.ts +53 -0
  16. package/src/drift-errors.ts +3 -3
  17. package/src/drift-findings.ts +211 -59
  18. package/src/drift.ts +33 -19
  19. package/src/entity-shape.ts +5 -0
  20. package/src/errors.ts +12 -8
  21. package/src/fake.ts +1 -1
  22. package/src/foreign-key.ts +0 -34
  23. package/src/generate-append-only.ts +146 -0
  24. package/src/generate.ts +20 -6
  25. package/src/index.ts +11 -9
  26. package/src/introspect-catalog.ts +19 -2
  27. package/src/introspect.ts +92 -12
  28. package/src/migrate-rollback.ts +44 -0
  29. package/src/migrate.ts +48 -158
  30. package/src/migration-ledger.ts +167 -0
  31. package/src/object-drift.ts +77 -20
  32. package/src/pglite-branch.ts +7 -7
  33. package/src/pglite.ts +16 -4
  34. package/src/pool-profile.ts +1 -1
  35. package/src/primary-key.ts +210 -0
  36. package/src/schema-dump-table.ts +4 -1
  37. package/src/sibling-turn.ts +49 -0
  38. package/src/snapshot-parse.ts +9 -3
  39. package/src/sqlstate.ts +30 -10
  40. package/src/statement-funnel.ts +16 -5
  41. package/src/transaction-errors.ts +66 -0
  42. package/src/transaction-options.ts +122 -0
  43. package/src/transaction.ts +131 -121
  44. package/src/drift-fixtures.ts +0 -23
  45. package/src/fake-pglite.ts +0 -32
  46. package/src/fake-reservable.ts +0 -50
package/CLAUDE.md CHANGED
@@ -35,47 +35,48 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
35
35
  only while its turn is held. `pglite-embedded.test.ts`, `pglite.test.ts`,
36
36
  `pglite-two-clients.test.ts`, `pglite-observer.test.ts`.
37
37
  - **The third rule is both drivers'**: `client.ts`'s pinned handle also runs direct only while held.
38
- `release()` is idempotent on both; `DbConnection` and `Turn` are `Disposable` (`[Symbol.dispose]` is
39
- `release()`).
38
+ `release()` is idempotent on both; `DbConnection` and `Turn` are `Disposable`.
40
39
  - **A pin is held by `using`, never a hand-rolled `try/finally`** (`withTransaction`,
41
40
  `readOnlyQuery`); `BEGIN` lives inside the guarded scope.
42
- - **`sqlstate.ts`**: `errno` first, `code` second, both shape-tested (`^[0-9A-Z]{5}$`).
43
- `DB_SQLSTATE_CODES` is closed; `driverError()` is its one consumer, `sendOn` its one caller.
41
+ - **`sqlstate.ts`**: `errno` first, `code` second, shape AND provenance (`isState`: `severity` =
42
+ server; `syscall`/numeric `errno` = socket; else needs a digit). `DB_SQLSTATE_CODES` is closed; `driverError()` is its one consumer, `sendOn` its one caller.
44
43
  - **`DbTx.origin` is the client the scope was opened on**, never the pin (entity's pinned-repository
45
44
  check reads it); a nested scope reports the root's.
46
45
  - **`withTransaction(fn, { retry })` re-runs `fn` only on `40001`/`40P01`**, default 0; each attempt
47
46
  its own pin, `BEGIN` and undo list (`runRoot`); a nested `retry` is `X_INVARIANT`. A re-run waits
48
- (`transaction-backoff.ts`: core's `backoffDelay`, 10 ms → 500 ms, full jitter; `{ sleep, random }`
49
- are injection seams); nothing waits at retry 0 or after the last attempt.
50
- - **Four codes are classified `retryable`** (`DB_ERROR_RETRY`: `X_DB_SERIALIZATION_FAILURE`,
51
- `X_DB_LOCK_TIMEOUT`, `X_DB_POOL_EXHAUSTED`, `X_MIGRATE_CONCURRENT`); terminal ones are deliberately
52
- unclassified (`errors-retry.test.ts` asserts the absence). Core's `retry()` executor is NOT adopted.
53
- - **`BEGIN` re-derives its isolation level from the closed set** (`isolationMode` switch with a `never`
54
- default; anything else `X_SQL_UNSAFE`).
55
- - Transaction control: `ROLLBACK` / `ROLLBACK TO SAVEPOINT` are best-effort; `SAVEPOINT` and
56
- `RELEASE SAVEPOINT` are deliberately uncaught.
57
- - **`close()` is BOUNDED by the driver's own `{ timeout }` in SECONDS** (`drainTimeoutMs / 1000`);
58
- `drainTimeoutMs: 0` sends no option; the verdict is elapsed time on `performance.now()`
59
- (`X_DB_DRAIN_TIMEOUT`). `pool-drain.test.ts`, `pool-drain.live.test.ts`. `close()` clears the cached
60
- driver before awaiting the teardown.
47
+ (`transaction-backoff.ts`: 10 ms → 500 ms, full jitter; `{ sleep, random }` are seams).
48
+ - **Four codes are `retryable`** (`DB_ERROR_RETRY`); terminal ones stay unclassified
49
+ (`errors-retry.test.ts`). Core's `retry()` executor is NOT adopted.
50
+ - **`BEGIN` re-derives its isolation level from the closed set** (`isolationMode`; else `X_SQL_UNSAFE`).
51
+ - **An aborted transaction is reported** (`X_DB_TRANSACTION_ABORTED`; README "Transactions that end
52
+ badly"): a failure carrying a SQLSTATE marks the root's `abort`, only a successful `ROLLBACK TO`
53
+ clears it (a failed one sets it), and `COMMIT` is then refused unsent. `commit-tag.ts` reads the
54
+ COMMIT tag in BOTH funnels. `COMMIT` rejecting with no SQLSTATE is `X_DB_COMMIT_UNKNOWN`: neither
55
+ list runs. `transaction-errors.ts`; `transaction-options.ts` holds `DbTx`, options, `BEGIN` text.
56
+ - **Sibling nested scopes take turns** (`TxState.children`; savepoints are a stack) under a deadline
57
+ (`sibling-turn.ts`, `siblingWaitMs`, `X_DB_SIBLING_SCOPE_TIMEOUT`). A nested
58
+ `isolation`/`readOnly`/`deferrable`/foreign `client` is `X_INVARIANT`.
59
+ - **`close()` is BOUNDED by the driver's own `{ timeout }` in SECONDS** (`drainTimeoutMs / 1000`; `0`
60
+ sends none); the verdict is elapsed `performance.now()` (`X_DB_DRAIN_TIMEOUT`). It clears the
61
+ cached driver before awaiting the teardown. `pool-drain{,.live}.test.ts`.
61
62
  - **`client.listen` is ONE session beside the pool** (`listen.ts`; `Bun.SQL.listen`, PGlite's
62
63
  `listen` under a turn), never a reserved pin. `onListening` fires on every re-dial; a channel is
63
64
  refused unless it is a plain identifier. `listen.test.ts`, `listen.live.test.ts`.
64
65
  - `execute()` trusts the command tag only when `> 0`, in both drivers (`rowsOf`, `affectedBy`).
65
- - **`pool-gauge.ts` derives `db_pool_max` / `db_pool_in_use` / `db_pool_waiting` from DEMAND** —
66
- `Bun.SQL` publishes no occupancy. `client.ts` is the one counter: `run()` from send to settle, a
67
- pin from the ask to its (idempotent) release; a statement ON a pin is not counted again. Declared
68
- on the first tracked pool, never at import. `pool-gauge.test.ts`.
66
+ - **`pool-gauge.ts` derives `db_pool_max` / `db_pool_in_use` / `db_pool_waiting` from DEMAND**
67
+ (`Bun.SQL` publishes no occupancy). `client.ts` is the one counter: `run()` from send to settle, a
68
+ pin from ask to release; a statement ON a pin is not counted again. Declared on the first pool.
69
69
  - **`client.ts` connects, holds the client and the ambient `db()`**, and opens no socket at import;
70
70
  `pool-profile.ts`, `connection-url.ts`, `bun-sql.ts`, `pool-reserve.ts`, `db-health.ts` (`checkDb`)
71
71
  and `statement-funnel.ts` (`sendOn`/`runOn`) hold the rest.
72
- - **`libpq-options.ts` merges the framework's `options` into the operator's**: the framework wins on
73
- the names it sets, the operator keeps every other flag; the bound is emitted for all six roles.
72
+ - **`libpq-options.ts` merges the framework's `options` into the operator's**: the framework wins
73
+ on the names it sets, the operator keeps the rest; emitted for all six roles.
74
74
  - **`DATABASE_URL`'s scheme is screened at boot** (`POSTGRES_SCHEMES`: `postgres:`, `postgresql:`); the
75
- received scheme is never echoed (it may be a host or a credential). `connection-url.test.ts`.
75
+ received scheme is never echoed (it may be a credential).
76
76
  - **A JS array bound as a parameter is rendered here** (`array-parameter.ts`, `bound-parameters.ts`,
77
77
  called only by `sendOn`) — `Bun.SQL` joins elements with commas. `NULL` bare vs `"NULL"`; quoting by
78
- content; a `Uint8Array` is BYTEA; a ragged nest is refused. `array-parameter.live.test.ts`;
78
+ content; a `Uint8Array` is BYTEA, in an array too; a ragged nest and an Invalid Date are refused
79
+ (`X_INVARIANT`), ABOVE the driver's `try` in both funnels. `array-parameter.live.test.ts`;
79
80
  `packages/cli/src/pg-array.live.test.ts` is the composition test.
80
81
  - **Every numeric option is screened** through core's `finiteCount` (`replicaClient`'s breaker,
81
82
  `migrate`'s `lockWaitMs`, `readonlyQuery`'s `timeoutMs` — only an explicit `0` disables it — the pool
@@ -83,11 +84,10 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
83
84
 
84
85
  ## Observation
85
86
 
86
- - **`observe.ts`: one process-wide `StatementObserver`** (`setStatementObserver()` /
87
- `statementObserver()`). Guard at the call site; one observer, not a list; the seam swallows nothing;
88
- `onStatement` is synchronous and must not issue SQL. Only `runOn` (`statement-funnel.ts`) and
89
- `statement()` (`pglite.ts`) invoke it; both observe success and failure, and notify outside the
90
- statement's own `try`.
87
+ - **`observe.ts`: one process-wide `StatementObserver`** (`setStatementObserver()`). Guard at the
88
+ call site; one observer, not a list; the seam swallows nothing; `onStatement` is synchronous and
89
+ must not issue SQL. Only `runOn` (`statement-funnel.ts`) and `statement()` (`pglite.ts`) invoke
90
+ it; both observe success and failure, and notify outside the statement's own `try`.
91
91
  - **`attribution.ts`**: `withStatementAttribution(entity, op, fn)` — guard first (two strings, no
92
92
  allocation), a scope not a parameter, innermost pair wins; the funnels stamp it on both settle paths.
93
93
  `@ultimat3/entity`'s `postgresRepo` is the one producer.
@@ -95,20 +95,19 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
95
95
  text) and `statementKind(text)` off `statementVerb(text)`. Read by `x dev`'s ledger and
96
96
  `@ultimat3/testing`'s `statements` fixture. It counts nothing.
97
97
  - **`statement-span.ts`**: `withStatementSpan` wraps the send alone — `db.<verb>`, attribute
98
- `STATEMENT_ATTRIBUTE` (exported; `@ultimat3/cli`'s `dev-traces.ts` imports it), OTel kind `client`,
99
- opened only when an observer is installed.
98
+ `STATEMENT_ATTRIBUTE` (`@ultimat3/cli`'s `dev-traces.ts` imports it), OTel kind `client`, opened
99
+ only when an observer is installed.
100
100
  - **`expected-loop.ts` is the ONLY suppression**: `expectedQueryLoop(reason, fn)`, innermost reason,
101
- blank is `X_INVARIANT`; the funnel stamps `expected`; it suppresses a verdict, never a statement. The
102
- framework's own loops declare themselves (`migrate()`, `rollback()`, `@ultimat3/admin`'s
103
- `search.ts`).
104
- - `@ultimat3/jobs` never imports this package; its statements pass the observer only because
105
- `packages/cli/src/dev-queue.ts` wraps a real client for its `PgExecutor`, unattributed.
101
+ blank is `X_INVARIANT`; the funnel stamps `expected`; it suppresses a verdict, never a statement.
102
+ The framework's own loops declare themselves (`migrate()`, `rollback()`, admin's `search.ts`).
103
+ - `@ultimat3/jobs` never imports this package; its statements pass the observer only as
104
+ `packages/cli/src/runtime-queue.ts` wraps a real client for its `PgExecutor`, unattributed.
106
105
 
107
106
  ## Migrations
108
107
 
109
108
  - **The migration lock is polled** (`pg_try_advisory_lock` every `MIGRATION_LOCK_POLL_MS` until
110
109
  `MIGRATION_LOCK_WAIT_MS`, then `X_MIGRATE_CONCURRENT`), declared with `expectedQueryLoop`;
111
- `createRecordingClient` stubs the lock as `locked: true`.
110
+ `recordingClient` stubs the lock as `locked: true`.
112
111
  - **`lock_timeout` is the migration's** (`SET LOCAL` inside each migration's transaction, from the
113
112
  `migrate` profile's 3 s).
114
113
  - **The advisory lock is held by one pinned session, and `migrate()`/`rollback()` run every statement
@@ -130,8 +129,8 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
130
129
  (`rollbackStepsInvalid`, `X_INVARIANT`).
131
130
  - **`refuseDependentViews(tx, script)`** (`dependent-view.ts`) runs before each migration's first
132
131
  statement: a word scan over `sql-scan.ts` finds retyped columns, one catalog round trip, the pair
133
- filtered in JS, and `X_MIGRATION_VIEW_DEPENDS` carries the `drop view` / `create view` from
134
- `pg_get_viewdef` (built through `identifier()` inside a `try`).
132
+ filtered in JS (visible tables only), and `X_MIGRATION_VIEW_DEPENDS` carries the `drop view` /
133
+ `create view` from `pg_get_viewdef` (built through `identifier()` inside a `try`).
135
134
  - `runningAppVersion()` delegates to core's `appVersion()`.
136
135
 
137
136
  ## Generation (`x db gen`)
@@ -142,7 +141,7 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
142
141
  declared is CLOSED, live is OPEN (`indexMethodOf` passes the catalog through; `declaredMethod`
143
142
  refuses); absent is `btree` through one function; `snapshotOf` records `using` only when declared;
144
143
  `indexMethodSql` re-derives the literal (`X_SQL_UNSAFE` default); a unique or ordered GIN is
145
- `X_INVARIANT`. `introspect()` reads `pg_am` (`introspect-embedded.test.ts`).
144
+ `X_INVARIANT`. `introspectSchema()` reads `pg_am` (`introspect-embedded.test.ts`).
146
145
  - **`index-plan.ts` walks both directions** (declared first, removed last). `dropRecordedIndex` emits
147
146
  `alter table … drop constraint if exists` then `drop index` for a shape a constraint could back
148
147
  (`mayBeConstraintBacked`); four names are skipped (primary, moved aside, rebuilt, over a dropped
@@ -171,59 +170,58 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
171
170
  in `preAlters` at the top of `up`, re-adding is `foreignKeyPlan`'s, both `breaksOn` ends are needed
172
171
  (`generate-retype-key.live.test.ts`). `sql-type.ts` reads `SQL_TYPES` with `Object.hasOwn`.
173
172
  - **A generated column** (`generated-column.ts`): the clause right after the type; generated-and-
174
- defaulted refused; an expression change is `set expression as (…)`; a retype carries no `using`; a NOT
175
- NULL add is one statement; generated → plain is `drop expression`; plain → generated rebuilds the
176
- column (`regenerate` answers `rebuilt`) and moves its dependents aside; a generated column's own type
177
- change deliberately does not. `introspect` never reads `generation_expression` back.
178
- `generate-generated-column.live.test.ts`, `generate-generated-rebuild.live.test.ts`.
179
- - **`REPLICA IDENTITY FULL` is emitted by a PARAMETER** (`GenerateOptions.replicaIdentityFull`, passed
180
- by `@ultimat3/cli`'s `db-generate.ts` from `describeQueries()`' `subscribes:`), in
181
- `replica-identity.ts`: recorded as `replicaIdentityFull: true` or absent; the snapshot records the
182
- union; dead last in `up`; never destructive; a name no entity declares is skipped; never reverted;
183
- `down` is `replica identity default` except on a table this migration creates.
173
+ defaulted refused; an expression change is `set expression as (…)`; a retype has no `using`; a NOT
174
+ NULL add is one statement; generated → plain is `drop expression`; plain → generated rebuilds
175
+ (`rebuilt`) and moves dependents aside, its own type change does not. `introspectSchema` never reads
176
+ `generation_expression`.
177
+ `generate-generated-{column,rebuild}.live.test.ts`.
178
+ - **`REPLICA IDENTITY FULL` is a PARAMETER** (`GenerateOptions.replicaIdentityFull`, from the CLI's
179
+ `db-generate.ts`; `replica-identity.ts`): recorded `true` or absent, as a union; last in `up`; an
180
+ undeclared name skipped; `down` reverts it unless this migration created the table.
181
+ - **`appendOnly: true` is a trigger** (`generate-append-only.ts`): one `create or replace` function
182
+ (SQLSTATE `23001`, message leads `X_ENTITY_APPEND_ONLY`) and a fixed-name `ultimate_append_only`
183
+ `before update or delete … for each row`; recorded `true` or absent; no `truncate`. Drift:
184
+ enabled `triggerNames` (`drift-append-only.ts`, `X_APPEND_ONLY_TRIGGER_MISSING`).
184
185
  - **A foreign key is `alter table … add constraint`**, collected into a bucket merged after every table
185
186
  statement (`foreign-key-plan.ts`); **dropping a table has its own bucket emitted BEFORE the table
186
187
  statements** (`preDrops`), ordered children-first by `drop-order.ts`, which breaks a two-table cycle
187
188
  by dropping one key first. `foreignKeyPlan` walks both directions, drops the name the previous
188
189
  snapshot recorded, and rebuilds a key whose `onDelete` moved. **`on delete` reaches the SQL**
189
190
  (`onDeleteRule`, `foreign-key.ts`; an unknown rule is `X_INVARIANT`).
190
- - **`entity-shape.ts` holds the three `*Like` interfaces** (optional `onDelete` / `generated`).
191
- - **`snapshot-json.ts` writes bytes that are a fixed point of Biome** (arrays collapse when they fit at
192
- `<= 100` counting the trailing comma); `snapshot-json.test.ts` runs the repo's own `biome format`.
193
- - **`declaredSchema()` answers the NEWEST migration's snapshot or `undefined`**; `checkDrift` turns
194
- that into `unknown-schema`, and `x db gen` refuses with `X_MIGRATION_SNAPSHOT_MISSING`. Both lead with
195
- the same two remedies in the same order: restore the sidecar (`git checkout --`), or delete the
196
- migration's files FIRST and only then run `x db gen`. `snapshotSiblings` / `migrationNameOf` build
197
- the second command from the caller's path; both commands are screened (`unknownSchema` through
198
- `shellInertIdentifier`, `migrationSnapshotMissing` through `renderFixShellArg`), degrading the whole
199
- line to prose.
191
+ - **`entity-shape.ts` holds the `*Like` interfaces** (every later field optional).
192
+ - **`snapshot-json.ts` writes a fixed point of Biome** (an array collapses when it fits at `<= 100`
193
+ with its trailing comma); `snapshot-json.test.ts` runs `biome format`.
194
+ - **`declaredSchema()` answers the NEWEST migration's snapshot or `undefined`** (`checkDrift`:
195
+ `unknown-schema`; `x db gen`: `X_MIGRATION_SNAPSHOT_MISSING`). Both fixes: restore the sidecar
196
+ (`git checkout --`), else delete the migration's files FIRST, then `x db gen`; both screened.
200
197
 
201
198
  ## Drift and introspection
202
199
 
203
200
  - **`checkDrift()` is the post-migrate verification** (live catalog vs the ledger just written, asked
204
201
  by `@ultimat3/cli`'s `runMigrations`), returned never thrown. The OTHER `X_DB_DRIFT` is the CLI's
205
202
  `checkSourceDrift`. Neither grows the other's half.
206
- - `compareTable` compares existence and **nullability** (primary-key columns excluded by the union of
207
- both sides' keys); the type is not compared. The `fix:` is the `alter table … set not null` itself.
203
+ - `compareTable` compares existence, **nullability** (the DECLARED key's columns excluded) and the
204
+ **primary key** in column order (`changed-primary-key`, fix = one `psql -c` of the pair; `primary-key.ts`
205
+ holds `x db gen`'s arm, its `drop not null`s, its two refusals). The type is not compared.
208
206
  - **A missing CHECK is drift, compared by NAME**: `TableDescription.checks` (declared: name and
209
207
  expression) vs `TableDescription.checkNames` (catalog: `conname` for `contype = 'c'`, always written
210
- by `introspect()`, `[]` included). Only the declared side is judged; no `changed-check`, ever.
208
+ by `introspectSchema()`, `[]` included). Only the declared side is judged; never a `changed-check`.
211
209
  `drift-check.live.test.ts`.
212
210
  - `compareTable` judges declared indexes (`missing-index`, `changed-index` over method, column list,
213
211
  uniqueness, predicate presence and direction; `asc` normalises to `null`); never the predicate text.
214
212
  - `compareForeignKeys` matches on where a key points (`foreignKeyTarget`, the one copy) and compares
215
- `onDelete` through `onDeleteRule` (`changed-foreign-key`, fix = drop/add pair).
216
- - `introspect()` reads index columns in key order (`indkey`) and a foreign key's two column lists
213
+ `onDelete` through `onDeleteRule` (`changed-foreign-key`; its fix, `changed-column`'s and
214
+ `missing-check`'s are one `psql -c` too — `repair()`, schema-scoped off `public`).
215
+ - `introspectSchema()` reads index columns in key order (`indkey`) and a foreign key's two column lists
217
216
  together (`unnest(a, b) with ordinality`), pinned by `introspect-embedded.test.ts`.
218
- - **`appTables()`** excludes the whole `x_` namespace for drift; `introspect()` alone excludes
217
+ - **`appTables()`** excludes all of `x_` for drift; `introspectSchema()` alone excludes
219
218
  `x_migrations` by default. **`app-relation.ts`**: `nonAppRelations(client, schema)` — extension
220
219
  ownership from `pg_depend` (`deptype = 'e'`) plus views, materialised views and foreign tables —
221
220
  merged into `excluded` unconditionally.
222
- - **`unexpectedTable`'s `fix:` never names `x db gen`**: a `create table if not exists` in a migration
223
- (accepted by `@ultimat3/cli`'s `acceptCreatedTables`) or dropping a table nothing owns.
224
- - **`dbDrift()` lives in `drift-errors.ts`** (it needs `shellInertIdentifier`); the `X_DB_DRIFT`
225
- rendering and title are duplicated in `@ultimat3/entity`, held equal by
226
- `packages/entity/src/errors.test.ts`. `errors.ts` registers `DB_ERROR_TITLES` unconditionally.
221
+ - **`unexpectedTable`'s `fix:` never names `x db gen`**: `psql -c '\d "T"'`, commented with the two
222
+ repairs (claim it with `create table if not exists`, or drop it).
223
+ - **`dbDrift()` lives in `drift-errors.ts`** (it needs `shellInertIdentifier`) and is the only one —
224
+ `@ultimat3/entity`'s copy is deleted. `errors.ts` registers `DB_ERROR_TITLES` unconditionally.
227
225
  - `drift-findings.ts` holds every `DriftDifference` constructor and `DriftKind`; `drift.ts` keeps the
228
226
  comparisons.
229
227
 
@@ -231,16 +229,17 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
231
229
 
232
230
  - **Its own entry: `@ultimat3/db/schema-dump`** (`schema-dump-entry.ts`). The barrel is in every
233
231
  role's boot graph and must evaluate none of this family — `schema-dump-entry.test.ts`.
234
- - **Two readings of one catalog, never mixed.** `introspect()` → `SchemaDescription`, the entity
235
- vocabulary a snapshot is diffed in. `introspectCatalog()` (`catalog.ts`; queries in `catalog-relations.ts` and
236
- `catalog-objects.ts`; the pure fold in `catalog-fold.ts`) → `CatalogDescription`, Postgres' own `pg_get_*def` text, compared only to
232
+ - **Two readings of one catalog, never mixed.** `introspectSchema()` → `SchemaDescription`, the entity
233
+ vocabulary a snapshot is diffed in. `introspectCatalog()` (`catalog.ts`; queries: `catalog-relations.ts`,
234
+ `catalog-objects.ts`; pure fold: `catalog-fold.ts`) → `CatalogDescription`, Postgres' own `pg_get_*def` text, compared only to
237
235
  itself. A catalog spelling on a `SchemaDescription` field is the `checks`/`checkNames` mistake.
238
236
  - **Sorted in JS** (`byCodeUnit`), never `order by` and never `localeCompare`: collations differ.
239
237
  - **`renderSchemaDump()` is pure**; `schema-dump-table.ts` spells one table. `quoted()` escapes any
240
238
  catalog name — `identifier()` refuses whitespace and `"`, which a migration may have created.
241
239
  - **`x_` owner → `framework/` twin** (`FRAMEWORK_TABLE_PREFIX`). No list is handed in.
242
- - **What cannot be rendered is named** (`unrenderedRows`, a closed list → `unrendered.sql`). A kind
243
- rendered later leaves that list in the same diff. Never render a partition as a plain table.
240
+ - **What cannot be rendered is named** (`unrenderedRows`, a closed list → `unrendered.sql`; a trigger
241
+ on an unrendered relation is named by the fold). A kind rendered later leaves that list in the
242
+ same diff. Never render a partition as a plain table. `generated` carries `stored`/`virtual`.
244
243
  - **`loadSchemaDump()`**: one transaction, `check_function_bodies` off, a savepoint per file, and
245
244
  only `42P01`/`42883`/`42704` are retried. Its refusal is `X_SCHEMA_DUMP_DRIFT` (`dump-drift.ts`).
246
245
  - **`object-drift.ts`**: `unexpectedObjects(live, expected)` compares IDENTITY (kind, table, name,
@@ -253,16 +252,15 @@ consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Kee
253
252
  by `pgliteExtensionExport` before it reaches a specifier; `plpgsql` is built in, never missing.
254
253
  Only a not-found import is `missing`; a bundle that throws is `X_DB_UNAVAILABLE`.
255
254
  - **`pglite-snapshot.ts`**: `snapshotDir` makes a `memory://` boot a restore. Key = PGlite version
256
- (read off its `package.json`; unreadable = no cache), never the extension set. One file, checksum in its
257
- header; unsound or unopenable → deleted and rebuilt; unreadable → a miss. Temp name + `rename`. Uncompressed by
258
- measurement: gzip taxes the boot that writes, and a CI checkout always writes.
259
- - **One embedded boot** serves every database-backed dump test: `schema-dump.test.ts`.
255
+ (unreadable = no cache), never the extension set. One file, checksum in its header; unsound or
256
+ unopenable → deleted and rebuilt; unreadable → a miss. Temp name + `rename`. Uncompressed.
257
+ - **Embedded boots for dump tests**: `schema-dump.test.ts` and `schema-dump-fidelity.test.ts`.
260
258
 
261
259
  ## Branches, replicas, read-only
262
260
 
263
- - **`reapBranches` sweeps branches of THIS database**: the marker is `ultimate:branch:<base>:<iso>`
264
- (`BranchInfo.base`), split on the ISO tail; an older one-segment marker is skipped, never dropped; an
265
- unparseable `createdAt` is skipped. `@ultimat3/cli`'s `ls`/`drop` scope by name prefix.
261
+ - **`reapBranches` sweeps branches of THIS database**: the marker is `ultimate:branch:<base>:<iso>`,
262
+ split on the ISO tail; an older one-segment marker or an unparseable `createdAt` is skipped, never
263
+ dropped. `@ultimat3/cli`'s `ls`/`drop` scope by name prefix.
266
264
  - **Read replicas are opt-in twice**: a pool when `DATABASE_REPLICA_URL` names one
267
265
  (`default-client.ts`), and a read offered only inside `withReplicaReads(fn)` (`replica-scope.ts`);
268
266
  read-your-writes is `ReplicaScope.wrote`, never a request-id map. **`withTransaction` is on the
@@ -287,7 +285,7 @@ Gotchas:
287
285
 
288
286
  - `exactOptionalPropertyTypes` — declare optional fields as `x?: T | undefined`.
289
287
  - `noUncheckedIndexedAccess` — array reads are `T | undefined`; `chunks[i] ?? ''` everywhere.
290
- - Tests use `createRecordingClient()` + `setDbClient()`; no test may need a live database.
288
+ - Tests use `recordingClient()` + `setDbClient()`; no test needs a live database.
291
289
  - A test that must prove a pin came back uses `reservableOver()` (`fake-reservable.ts`), never a copy.
292
290
  - `ALTER DEFAULT PRIVILEGES` is scoped to an object's creator, so layer 1 covers future tables only for
293
291
  the roles in `creators` (default: the connected user).
package/README.md CHANGED
@@ -29,7 +29,8 @@ await withTransaction(async (tx) => {
29
29
  | `shellInertIdentifier()` | `As of 2026-08-26`: a quoted identifier that is also inert wherever a human PASTES it — or `null`. The one screen a catalog name goes through before it reaches a `fix:`. `identifier()` answers about SQL and **accepts** a backtick and a `$`, which are exactly what a shell substitutes inside double quotes, so a column called `$(id)` inside `x db gen "add $(id)"` runs `id` on paste |
30
30
  | `db()` / `baseClient()` / `setDbClient()` | the ambient client; `db()` returns the open tx if any |
31
31
  | `DbTx.origin` | `As of 2026-08`: the client the transaction was **opened on** — `options.client` or `baseClient()`, never the reservation it runs statements through. `@ultimat3/entity` compares a pinned repository's client against it, so a pinned repo joins its own shard's transaction instead of being refused |
32
- | `withTransaction()` / `currentTx()` | transaction scope; `currentTx()` is the outbox seam. `{ retry: n }` (`As of 2026-08`) re-runs `fn` from the top on a `40001`/`40P01` and on nothing else — default 0, so `fn` must be idempotent before you ask for it. Each re-run **waits first**, `As of 2026-08-23`: exponential from 10ms, capped at 500ms, full jitter (`@ultimat3/core`'s `backoffDelay`). A budget of 0 waits not at all |
32
+ | `withTransaction()` / `currentTx()` | transaction scope; `currentTx()` is the outbox seam. `{ retry: n }` (`As of 2026-08`) re-runs `fn` from the top on a `40001`/`40P01` and on nothing else — default 0, so `fn` must be idempotent before you ask for it. Each re-run **waits first**, `As of 2026-08-23`: exponential from 10ms, capped at 500ms, full jitter (`@ultimat3/core`'s `backoffDelay`). A budget of 0 waits not at all. **A transaction the server aborted is reported as one**, `As of 2026-10-02` → [Transactions that end badly](#transactions-that-end-badly) |
33
+ | `liveTxConnection()` | the connection of the transaction still **open** on this async context, or `undefined`. `currentTx()` keeps answering a finished scope's handle to a promise chain the body forgot to await; this is the one that says whether a statement sent now is really inside a transaction. `@ultimat3/action` reads it before binding an idempotency settlement to a commit |
33
34
  | `sqlState()` / `sqlStateCode()` / `isRetryableState()` / `SQLSTATE` | `As of 2026-08`: the SQLSTATE a driver error carries, and the closed table from it to a code. `Bun.SQL` puts it on `errno`; PGlite puts it on `code`; **one** reader answers for both |
34
35
  | `migrate()` / `rollback()` / `readLedger()` | the `x_migrations` ledger |
35
36
  | `statementsOf()` | `As of 2026-08`: a SQL script → the statements a driver sends one at a time. One send is one statement, so `migrate()` splits with this — a `;` inside a literal, an identifier, a dollar-quoted body or a comment is data |
@@ -37,7 +38,7 @@ await withTransaction(async (tx) => {
37
38
  | `replicatedClient()` / `ReplicaStats` / `REPLICA_URL_ENV` | `As of 2026-08-24`: one `DbClient` over a primary and a standby. `baseClient()` builds one when `DATABASE_REPLICA_URL` is set and the single-pool client when it is not |
38
39
  | `INDEX_METHODS` / `IndexMethod` / `indexMethodOf()` / `indexMethodSql()` / `declaredMethod()` / `isIndexMethod()` | `As of 2026-08-24`: an index's access method — `btree` or `gin`, closed. Absent is `btree`, the live side is read open (whatever `pg_am` said), and the DDL literal is re-derived from the set rather than spliced from the input |
39
40
  | `isPlainRead()` | `As of 2026-08-24`: whether a statement may leave the primary. An allow-list — everything it cannot vouch for is the primary's |
40
- | `checkDrift()` / `diffSchema()` / `assertNoDrift()` | drift, with a `--json` report. `checkDrift()` is the **post-migrate verification** — the live database against the ledger: columns, declared indexes (access method `As of 2026-08-24`, columns, uniqueness, direction, and whether a predicate is there at all — never its text), declared CHECK constraints by NAME (`missing-check`, `As of 2026-08-25` — never a predicate, which the catalog answers rewritten) and declared foreign keys, matched on where the key points and not on its constraint name, with the `on delete` rule compared through one normalisation `As of 2026-08-19` |
41
+ | `checkDrift()` / `diffSchema()` / `assertNoSchemaDrift()` | drift, with a `--json` report. `checkDrift()` is the **post-migrate verification** — the live database against the ledger: columns, declared indexes (access method `As of 2026-08-24`, columns, uniqueness, direction, and whether a predicate is there at all — never its text), declared CHECK constraints by NAME (`missing-check`, `As of 2026-08-25` — never a predicate, which the catalog answers rewritten) and declared foreign keys, matched on where the key points and not on its constraint name, with the `on delete` rule compared through one normalisation `As of 2026-08-19` |
41
42
  | `declaredSchema()` / `expectedSchema()` | `As of 2026-08`: the schema the migrations write down, or `undefined` when the newest one carries no snapshot — never an older snapshot standing in for it |
42
43
  | `parseSnapshot()` | `As of 2026-08`: a `<id>.snapshot.json` sidecar validated to the last nested field, or `undefined`. `{"tables":[null]}` is valid JSON and is not a schema |
43
44
  | `snapshotJson()` | `As of 2026-08`: the sidecar's **bytes** — the JSON Biome would have printed, trailing newline included. The one writer of a `<id>.snapshot.json`, because `JSON.stringify(…, null, 2)` is not formatter-clean and an app's `lint` step rejected the file `x db gen` had just written |
@@ -48,18 +49,18 @@ await withTransaction(async (tx) => {
48
49
  | `declaredChecks()` / `checkClauses()` / `checkPlan()` / `columnChecks()` / `columnCheckName()` / `columnNamesConstraint()` | `As of 2026-08-25`: **every** CHECK a table declares — a column's own (`enumerated()`'s value set, `tz()`'s IANA whitelist, `locale()`'s tags, money's currency pattern and scale bound) and an invariant's — on ONE list, so `createTable`, `diffTable` and `snapshotOf` agree about what exists. A column's check reached `create table` **inline and anonymous** and nothing else: the snapshot recorded none and the diff had no arm, so a value added to `enumerated()` generated no migration and a regenerated ENUM column came back as bare `text`. The name is `<table>_<column>_check` because that is the name **Postgres itself mints** for the old anonymous form — measured — so the repair lands on the constraint an already-generated database is holding; `checkPlan` emits `drop constraint if exists` before the `add` for exactly that column, because a bare add is `42710` there and a no-op everywhere else |
49
50
  | `defaultExpression()` / `ColumnDefaultLike` | `As of 2026-08-25`: a column's `default` as SQL. A DECLARED default (`{ kind: 'value', value }`) wins; `gen_random_uuid()` and `now()` stay as the inference for a description that carries only `hasDefault` |
50
51
  | `unrenderedOf()` / `unrenderedComment()` / `UnrenderedDeclaration` | `As of 2026-08-25`: what the generator could **not** write, on `GeneratedMigration.unrendered` and as a `-- UNRENDERED` block at the top of a non-empty `up`. A generator that emits less than the declaration in silence is the defect the whole file exists against, and `x verify`'s `drift` step reads a source hash — it never reads the SQL, so the loss was green. **`unrenderedOf(entities, current)` takes the recorded schema**, required and nullable: a rule declared as an `assert` reaches no SQL by design and is no loss on its own, but one whose CHECK a previous migration RECORDED is dropped by this run and reported by nothing — five in `examples/dummy`, and `@ultimat3/cli`'s `repairFix` then offered `x db gen "drop <name>"` as the repair for the loss that command performs |
51
- | `destructiveStatements()` / `hasDestructiveMarker()` / `isDestructive()` / `DESTRUCTIVE_MARKER` | `As of 2026-08`: the destructive-SQL rail — does this `up` drop, truncate or retype, and does the file declare it with `-- destructive: true`? One classifier, read by `x db gen` when it writes the marker and by `x verify` when it demands one |
52
+ | `destructiveStatements()` / `hasDestructiveMarker()` / `isDestructiveMigration()` / `DESTRUCTIVE_MARKER` | `As of 2026-08`: the destructive-SQL rail — does this `up` drop, truncate or retype, and does the file declare it with `-- destructive: true`? One classifier, read by `x db gen` when it writes the marker and by `x verify` when it demands one |
52
53
  | `stripSqlNoise()` | comments, literals, dollar-quoted bodies and quoted identifiers blanked **in source order**, so a reader sees the operation and not the prose. Shared by `readOnlyQuery()` and the destructive rail |
53
- | `introspect()` | live schema → `SchemaDescription`. **App tables only**, `As of 2026-08-24`: a relation an extension owns (`pg_depend`, `deptype = 'e'`) and anything that is not an ordinary or partitioned table are excluded before the fold, and an explicit `exclude` cannot bring them back |
54
- | `introspectCatalog()` / `CatalogDescription` / `emptyCatalog()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: the WHOLE schema in the catalog's own spelling — extensions, enum and domain types, sequences, tables, indexes, foreign keys, views, functions, triggers — sorted in code-unit order, plus `unrendered`: what exists and the dump cannot spell. Comparable only to another reading of itself; `introspect()` stays the entity-vocabulary reading a snapshot is diffed in |
54
+ | `introspectSchema()` | live schema → `SchemaDescription`. **App tables only**, `As of 2026-08-24`: a relation an extension owns (`pg_depend`, `deptype = 'e'`) and anything that is not an ordinary or partitioned table are excluded before the fold, and an explicit `exclude` cannot bring them back |
55
+ | `introspectCatalog()` / `CatalogDescription` / `emptyCatalog()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: the WHOLE schema in the catalog's own spelling — extensions, enum and domain types, sequences, tables, indexes, foreign keys, views, functions, triggers — sorted in code-unit order, plus `unrendered`: what exists and the dump cannot spell. Comparable only to another reading of itself; `introspectSchema()` stays the entity-vocabulary reading a snapshot is diffed in |
55
56
  | `renderSchemaDump()` / `SchemaDumpFile` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: a catalog → the schema dump's files. Pure and byte-deterministic. [The schema dump](#the-schema-dump) |
56
57
  | `loadSchemaDump()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: build a schema from those files, in one transaction, retrying a file that names something not created yet |
57
58
  | `compareSchemaDump()` / `reloadDifferences()` / `schemaDumpDrift()` / `schemaDumpDifferenceOf()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: `X_SCHEMA_DUMP_DRIFT` — committed files against rendered ones in both directions, and load-equals-replay as a comparison |
58
- | `unexpectedObjects()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: the triggers, functions, views, types and sequences a live catalog holds and an expected one does not, as `unexpected-object` drift. Identity, never definition text |
59
+ | `unexpectedObjects()` | **`@ultimat3/db/schema-dump`.** `As of 2026-10`: the triggers, functions, views, types and sequences a live catalog holds and an expected one does not, as `unexpected-object` drift. Identity, never definition text. Its `fix:` is one `psql -c` command that prints the object's definition, qualified by the catalog's schema — `\d+` a view, `\dT+` an enum, `\dD+` a domain, `\d` a sequence or a trigger's table, `pg_get_functiondef` by namespace for a function — with the repair as its comment |
59
60
  | `PgliteOptions.extensions` / `linkPgliteExtensions()` | `As of 2026-10`: Postgres extensions to link at boot, by name — a list, or a function for a caller whose list is read from disk. `linkPgliteExtensions(names)` answers `{ linked, missing }` without booting anything: `missing` is what the installed PGlite ships no bundle for, which is how `@ultimat3/cli` decides a replay needs a real Postgres. A missing name is skipped at boot and refused by `create extension` itself |
60
61
  | `PgliteOptions.snapshotDir` | `As of 2026-10`: a directory for the post-`initdb` snapshot, so an in-memory boot is a restore (~0.4 s against ~2.7 s). Keyed on the PGlite version alone — `initdb` never sees a linked extension, so one snapshot serves every set; checksummed, never trusted, written by temp-name-then-rename. Ignored for a data directory on disk |
61
62
  | `createBranch()` / `dropBranch()` / `reapBranches()` | copy-on-write branch databases. `As of 2026-08-19` the marker comment records the **base** as well as the instant (`ultimate:branch:<base>:<iso>`, on `BranchInfo.base`), and `reapBranches()` sweeps only branches of the database it is connected to — one Postgres hosting two Ultimate apps used to mean one app's nightly reap dropped the other's branches. A pre-3.x marker records no base and is skipped, never dropped |
62
- | `createPgliteClient()` / `branchPglite()` | the embedded database — Postgres in this process |
63
+ | `pgliteClient()` / `branchPglite()` | the embedded database — Postgres in this process |
63
64
  | `ensureReadOnlyRole()` / `grantReadOnlySql()` / `READONLY_ROLE` | a `NOLOGIN`, SELECT-only Postgres role — layer 1 of `db.query`'s defence |
64
65
  | `readOnlyQuery()` / `READONLY_TIMEOUT_MS` | one statement inside `BEGIN READ ONLY` with a statement timeout — layer 2 |
65
66
  | `setStatementObserver()` / `statementObserver()` | `As of 2026-08`: one event **and one `db.<verb>` span** per settled statement, both drivers; uninstalled is one branch |
@@ -67,7 +68,7 @@ await withTransaction(async (tx) => {
67
68
  | `withStatementAttribution()` / `statementAttribution()` | `As of 2026-08`: the `{ entity, op }` pair on `StatementEvent.attribution`, scoped exactly like `expectedQueryLoop()` — `@ultimat3/entity`'s `postgresRepo` is the one producer |
68
69
  | `STATEMENT_ATTRIBUTE` | `As of 2026-08`: `db.statement`, the OTel attribute each span carries its text under — declared here, read by `x dev`'s timeline |
69
70
  | `statementFingerprint()` / `statementKind()` / `statementVerb()` | `As of 2026-08`: what shape a statement is — `entity.op` when attributed else its own collapsed text, read or write from the leading verb. One rule, so two detectors group identically |
70
- | `createRecordingClient()` | in-memory `DbClient` that records SQL, for tests |
71
+ | `recordingClient()` | in-memory `DbClient` that records SQL, for tests |
71
72
 
72
73
  ## `sql` is parameters-only
73
74
 
@@ -75,13 +76,37 @@ String interpolation is how every SQL injection ships, and an agent writing SQL
75
76
  trusted to remember the difference between a value and a fragment. So:
76
77
 
77
78
  - scalars (`string`, `number`, `boolean`, `bigint`, `Date`, `Uint8Array`, arrays, `null`) become
78
- `$1..$n` and never touch `.text`;
79
+ `$1..$n` and never touch `.text`. A value that cannot be SENT — an Invalid Date, a ragged array —
80
+ is `X_INVARIANT` before the driver is called, never `X_DB_UNAVAILABLE`; a `Uint8Array` inside an
81
+ array is one `bytea` element. Both drivers refuse alike;
79
82
  - a nested fragment is spliced and its parameters are renumbered;
80
83
  - **anything else throws `X_SQL_UNSAFE`** — including an object shaped like a `SqlFragment` that
81
84
  `sql`/`raw` did not produce;
82
85
  - `raw(trusted)` is the one audited escape hatch, `identifier(name)` the safe way to interpolate
83
86
  a table or column, `literal(text)` for utility statements that reject bound parameters.
84
87
 
88
+ ## Transactions that end badly
89
+
90
+ `As of 2026-10-02`. Postgres aborts the WHOLE transaction on any statement error and answers the
91
+ `COMMIT` that follows with the tag `ROLLBACK` and no error. `withTransaction` used to resolve on
92
+ that, fire `onCommit`, and store nothing.
93
+
94
+ | Situation | What happens |
95
+ |---|---|
96
+ | the body catches a failed statement and returns | `X_DB_TRANSACTION_ABORTED`, cause = the first failing statement; `ROLLBACK`, `onRollback` undos run, `onCommit` never fires |
97
+ | the fallible statement sits in a nested `withTransaction` and THAT is caught | only the savepoint rolls back; the outer scope commits. This is the fix the error names |
98
+ | a nested body swallows a failed statement | the nested call rejects with `X_DB_TRANSACTION_ABORTED`; its savepoint is rolled back and the outer scope is usable again |
99
+ | `ROLLBACK TO SAVEPOINT` itself fails | the nested call still rejects with the body's error; the root is marked aborted and its `COMMIT` is refused, never sent |
100
+ | two nested scopes under `Promise.all` | run one after the other, per parent scope — savepoints are a stack, and interleaved ones destroyed each other |
101
+ | a nested scope waits past `siblingWaitMs` (default `SIBLING_SCOPE_WAIT_MS`, 30 s; `0` = no deadline) for its sibling | `X_DB_SIBLING_SCOPE_TIMEOUT`, naming the parent and the savepoint holding the turn. The shape it exists for: a body awaiting a sibling started after it, which waits for itself — a permanent hang before the deadline. The waiter never opened and its place is handed on |
102
+ | any `COMMIT` answered `ROLLBACK`, on either driver | `X_DB_TRANSACTION_ABORTED` from the statement funnel (`commit-tag.ts`) — covers an abort the scope never saw and a hand-written `COMMIT` |
103
+ | `COMMIT` rejects with a SQLSTATE (a deferred constraint, `40001`) | rolled back: the server's error surfaces, undos run |
104
+ | `COMMIT` rejects with NO SQLSTATE | `X_DB_COMMIT_UNKNOWN`: neither `onCommit` nor `onRollback` runs, and it is never retried |
105
+ | a nested scope passes `isolation`, `readOnly: true`, `deferrable: true`, or a `client` other than the root's | `X_INVARIANT` — a savepoint can honour none of them; `retry` was already refused |
106
+
107
+ Only a failure carrying a SQLSTATE marks the scope aborted: a refusal raised before the send left
108
+ the transaction untouched.
109
+
85
110
  ## Read-only access for anything an LLM drives
86
111
 
87
112
  `As of 2026-07`: a bug in one defence must not become a write, so `db.query` on the MCP dev
@@ -172,7 +197,7 @@ production traffic is routed.
172
197
 
173
198
  ## The drift contract
174
199
 
175
- `checkDrift()` compares `introspect()` against the snapshot the newest applied migration carries —
200
+ `checkDrift()` compares `introspectSchema()` against the snapshot the newest applied migration carries —
176
201
  `expectedSchema(migrations, ledger)`. `declaredSchema(migrations)` is the same read with the ledger
177
202
  left out: the schema the files *declare*, applied or not, which is what `x db gen` diffs the app's
178
203
  entities against so generation needs no database at all. One implementation, two callers — a
@@ -190,7 +215,7 @@ A migration the ledger has not recorded is **not** drift: `expectedSchema` reads
190
215
  subset, so a database that simply has not migrated yet is pending, not divergent. Neither is a
191
216
  table in the `x_` namespace — `x_migrations`, the queue's tables, the outbox and every
192
217
  `@ultimat3/auth` table are created by `create table if not exists` at boot and appear in no
193
- snapshot, so `appTables()` drops them before the diff. `introspect()` keeps its own narrower
218
+ snapshot, so `appTables()` drops them before the diff. `introspectSchema()` keeps its own narrower
194
219
  exclusion (the ledger alone), reserving `x_users` for a schema view that wants it.
195
220
 
196
221
  **A CHECK the catalog no longer holds is drift, `As of 2026-08-25` — by NAME.**
@@ -199,15 +224,16 @@ back as `CHECK ((status = ANY (ARRAY['draft'::text, 'published'::text])))`), so
199
224
  read as `conname` alone and lands on `TableDescription.checkNames`, a **separate field** from the
200
225
  declaration's `checks`. Only the declared side is judged, so a NOT NULL, an `enumerated()` column's
201
226
  old anonymous form and an extension's own constraint are all silent; a declared one the catalog
202
- does not hold is `missing-check`, whose `fix:` is the `add constraint` statement itself, because the
203
- migration that declares it is already in the ledger and `x db migrate` would apply nothing. There is
227
+ does not hold is `missing-check`, whose `fix:` is the `add constraint` statement as one
228
+ `psql "$DATABASE_URL" -c '…'` command against that database, because the migration that declares
229
+ it is already in the ledger and `x db migrate` would apply nothing. There is
204
230
  no `changed-check`: presence is a boolean, a predicate is text, and normalising the text is an
205
231
  expression parser competing with the server's.
206
232
 
207
233
  **Nor is a relation an extension owns, `As of 2026-08-24`.** `create extension pg_stat_statements`
208
234
  in `public` is the CNPG, RDS, Supabase and Neon default, and its view read as `unexpected-table`
209
235
  with `x db gen "add pg_stat_statements"` as the fix — so every deploy failed terminally and the fix
210
- would have written an extension's internal view into the app's migration set. `introspect()` now
236
+ would have written an extension's internal view into the app's migration set. `introspectSchema()` now
211
237
  excludes every relation Postgres records as extension-owned (`pg_depend`, `deptype = 'e'`), which is
212
238
  ownership rather than a name: a `pg_*` prefix rule covers that view and misses `postgis`'
213
239
  `spatial_ref_sys`. Views, materialised views and foreign tables go with them — no snapshot records
@@ -226,18 +252,41 @@ X_DB_DRIFT: schema differs from migrations
226
252
  |---|---|---|
227
253
  | live column, no migration | `table "T" has column "C" not present in any migration` | `x db gen "add C"` — the name goes through `shellInertIdentifier()`, and one it refuses is left OUT of the command rather than escaped into it (`x db gen "add the undeclared column"`, the name in the cause) |
228
254
  | migrated column, not live | `table "T" is missing column "C" that migrations declare` | `x db migrate` |
229
- | live table, no migration | `table "T" is not present in any migration` | a `create table if not exists` in a migration, then `x db migrate` — or `drop table` in `psql` where nothing owns it. Never `x db gen`, which diffs a table nothing declares against nothing and writes no file (issue #345). The name goes through `shellInertIdentifier()`, and one it refuses leaves the fix as prose |
255
+ | live table, no migration | `table "T" is not present in any migration` | `psql "$DATABASE_URL" -c '\d "T"'` — one harmless command that shows the table — with the two repairs as its comment: a `create table if not exists` in a migration, then `x db migrate`, or `drop table` where nothing owns it. Never `x db gen`, which diffs a table nothing declares against nothing and writes no file (issue #345). The name goes through `shellInertIdentifier()`, and one it refuses leaves `psql "$DATABASE_URL" # <the steps>` |
230
256
  | migrated table, not live | `table "T" is declared by migrations but does not exist` | `x db migrate` |
231
257
  | index rebuilt differently | `index "I" on "T" covers (…)` / `is unique` / `is descending` / `is partial`, `not what migrations declare` | `x db migrate` |
232
- | foreign key, rule moved | `foreign key on "T" (C) to "R" is on delete cascade, not what migrations declare` | the `drop constraint` + `add constraint` pair, in a new migration |
258
+ | foreign key, rule moved | `foreign key "K" on "T" (C) to "R" is on delete cascade, not what migrations declare` — `K` is the constraint the database holds | `psql "$DATABASE_URL" -c '<the drop constraint + add constraint pair>'` — one command, against the drifted database — then `x db migrate` |
259
+ | column nullability differs (`changed-column`) | `table "T" allows NULL in column "C" that migrations declare not null` / `forbids NULL in column "C" that migrations declare nullable` | `psql "$DATABASE_URL" -c 'alter table "T" alter column "C" set not null;'` (or `drop not null`) — one command, then `x db migrate`. A table read from a schema other than `public` gets `set search_path = "<schema>";` ahead of the statement, in the same `psql -c` word, so it and every table it references resolve there. A name `shellInertIdentifier()` refuses degrades every one of these three to `psql "$DATABASE_URL" # <the steps>`: still a command that runs, with no name in it |
260
+ | append-only trigger gone (`missing-append-only-trigger`, `As of 2026-10-06`) — raised as `X_APPEND_ONLY_TRIGGER_MISSING`, the one kind with its own code | `table "T" is declared appendOnly, and its trigger ultimate_append_only is missing or disabled` — a disabled (`D`) or replica-only (`R`) trigger counts as missing, because an ordinary session fires neither | `psql "$DATABASE_URL" -c '<create or replace function …; drop trigger if exists "ultimate_append_only" on "T"; create trigger "ultimate_append_only" …>'` against the drifted database, then `x db migrate`. Drop-if-exists first: a disabled trigger still exists, and a bare `create trigger` would fail on it (`42710`) |
261
+ | primary key differs (`changed-primary-key`, `As of 2026-10-02`) | `table "T" has primary key (id) as constraint "T_pkey", and migrations declare (slug)` — the constraint named is the one the DATABASE holds — compared in column ORDER; `has no primary key` when the database holds none | `psql "$DATABASE_URL" -c '<drop constraint "<the live key>"; add constraint "T_pkey" primary key (…)>'` — one command, against the drifted database — then `x db migrate`. Nullability is skipped for the DECLARED key's columns only |
233
262
 
234
- `checkDrift()` returns every difference; `assertNoDrift()` throws the first. `x db migrate` renders
263
+ `checkDrift()` returns every difference; `assertNoSchemaDrift()` throws the first. `x db migrate` renders
235
264
  them all as findings and exits non-zero; a `ROLE=migrate` container throws the first one
236
- (`assertNoDrift`, in `runRole`) and exits non-zero too, because the release phase has one channel —
265
+ (`assertNoSchemaDrift`, in `runRole`) and exits non-zero too, because the release phase has one channel —
237
266
  the exit code — and a deploy that rolled on past a schema nobody can reconstruct is the failure
238
267
  drift exists to catch. There is no `x db drift`, and `x verify`'s `drift` step is the *source*
239
268
  detector (`checkSourceDrift`), which needs no database and never calls this.
240
269
 
270
+ ## Append-only tables
271
+
272
+ `EntityDescriptionLike.appendOnly` (from `entity({ appendOnly: true })`) is a trigger, written by
273
+ `x db gen` (`generate-append-only.ts`) and recorded on the snapshot as `appendOnly: true`:
274
+
275
+ ```sql
276
+ create or replace function "ultimate_refuse_append_only"() returns trigger language plpgsql as $append_only$ begin raise exception 'X_ENTITY_APPEND_ONLY: % on %.% is refused, the table is append-only', tg_op, tg_table_schema, tg_table_name using errcode = '23001', hint = '…'; end; $append_only$;
277
+ create trigger "ultimate_append_only" before update or delete on "ledger" for each row execute function "ultimate_refuse_append_only"();
278
+ ```
279
+
280
+ | | |
281
+ |---|---|
282
+ | Names | fixed: the trigger is `ultimate_append_only` on every table (a trigger name is per table, so no 63-byte truncation), the function one shared `ultimate_refuse_append_only()` |
283
+ | Refuses | UPDATE, DELETE, and the update arm of `insert … on conflict do update`. Not `truncate` — no row write, and `destructive.ts` already gates it |
284
+ | The error | message leads with `X_ENTITY_APPEND_ONLY`, SQLSTATE `23001` (`restrict_violation`, class 23: an integrity rule refused it, so never retried) |
285
+ | On / off | adding `appendOnly` emits the function (once per migration) and the trigger (drop-if-exists first on an existing table), `down` drops the trigger; removing it drops the trigger, `down` restores both. A dropped table takes its trigger with it; the function is never dropped |
286
+ | NOT NULL with no backfill | a NEW NOT NULL column needs a default (or `.nullable()`), and an EXISTING column cannot be turned NOT NULL: `x db gen` refuses both (`X_MIGRATION_APPEND_ONLY_BACKFILL`) rather than emit the usual `-- backfill …, then: set not null` note — that backfill is an UPDATE the trigger refuses, and a default fills no NULL already stored (`generate-append-only-column.test.ts`) |
287
+ | Engines | Postgres and PGlite alike (`generate-append-only.live.test.ts`, `generate-append-only-embedded.test.ts`) |
288
+ | Drift | `introspectSchema()` writes `triggerNames` (enabled, non-internal) on every table — the catalog half; `appendOnly` is the snapshot half, never read from the catalog |
289
+
241
290
  ## The schema dump
242
291
 
243
292
  **Imported from `@ultimat3/db/schema-dump`, never from the barrel** (23.0.0): `introspectCatalog`,
@@ -271,7 +320,10 @@ object kind, numbered in the order a database is built in, one file per named ob
271
320
  - A file over 500 lines (`SCHEMA_DUMP_MAX_LINES`, the `filesize` ceiling) is split `<name>.1.sql`, `<name>.2.sql`;
272
321
  `loadSchemaDump()` joins the parts in numeric order.
273
322
  - What cannot be spelled — partitions, inheritance, foreign tables, composite and range types,
274
- aggregates, row-security policies, rules — is named in `unrendered.sql` as comments.
323
+ aggregates, row-security policies (and `force row level security`), rules, extended statistics,
324
+ a column's non-default `storage`, a materialized view created `with no data`, and a trigger on
325
+ a relation the dump does not create — is named in `unrendered.sql` as comments.
326
+ - A generated column is spelled `stored` or `virtual` (Postgres 18), as `attgenerated` says.
275
327
 
276
328
  `loadSchemaDump()` runs kind by kind, the framework twin first, each file in a savepoint. A file
277
329
  refused with `42P01`, `42883` or `42704` — "not created yet" — is retried after the rest; a pass
@@ -286,17 +338,17 @@ Which engine, where the files live, when they are written and what holds them is
286
338
 
287
339
  ## The embedded database
288
340
 
289
- No `DATABASE_URL` means no Docker: `createPgliteClient()` runs Postgres as WASM inside this
341
+ No `DATABASE_URL` means no Docker: `pgliteClient()` runs Postgres as WASM inside this
290
342
  process. The module is resolved on the first statement, never at import, so an image that only
291
343
  ever talks to a managed Postgres never loads it.
292
344
 
293
345
  ```ts
294
- const dev = createPgliteClient({ dataDir: pgliteDataDir(services.db.url) }); // or memory://
346
+ const dev = pgliteClient({ dataDir: pgliteDataDir(services.db.url) }); // or memory://
295
347
  await dev.ping(); // pay the ~3s boot before serving
296
348
  setDbClient(dev);
297
349
 
298
350
  const branch = await branchPglite('feature_x', { from: '.x/pgdata' });
299
- setDbClient(createPgliteClient({ dataDir: branch.dataDir }));
351
+ setDbClient(pgliteClient({ dataDir: branch.dataDir }));
300
352
  ```
301
353
 
302
354
  PGlite has no `CREATE DATABASE ... TEMPLATE`, so `branchPglite()` copies the data directory —
@@ -306,7 +358,7 @@ lands in a filesystem path, so an unvalidated one is traversal rather than a typ
306
358
 
307
359
  ### One session, so callers take turns
308
360
 
309
- Embedded Postgres is a single session, not a pool. `createPgliteClient()` is therefore
361
+ Embedded Postgres is a single session, not a pool. `pgliteClient()` is therefore
310
362
  `ReservableClient`: `withTransaction()` and `readOnlyQuery()` pin it, and every other statement
311
363
  waits for its turn. Without that pin two concurrent units of work each run `BEGIN` on the same
312
364
  connection — the second `COMMIT` commits the first's rows and the first `ROLLBACK` finds no
@@ -371,9 +423,9 @@ A pooled statement cannot hold a subscription: the next one runs on another conn
371
423
  clients hold ONE session beside the pool for it.
372
424
 
373
425
  ```ts
374
- import { createPostgresClient } from '@ultimat3/db';
426
+ import { postgresClient } from '@ultimat3/db';
375
427
 
376
- const client = createPostgresClient({ url: 'postgres://localhost:5432/app_test' });
428
+ const client = postgresClient({ url: 'postgres://localhost:5432/app_test' });
377
429
  const subscription = await client.listen(
378
430
  'x_jobs_wake',
379
431
  (payload) => console.log('notified', payload),
@@ -407,7 +459,13 @@ migration inside its own transaction. It refuses **before applying anything** wh
407
459
  the running one — another version owns the database; or
408
460
  - an applied migration's `up` SQL no longer matches its recorded checksum.
409
461
 
410
- Report (`--json`): `{ applied: [{ id, name, durationMs }], skipped: [id], durationMs, appVersion }`.
462
+ **Except a rollback** (`As of 2026-10-05`, `migrate-rollback.ts`): when every unknown ledger row
463
+ sorts after the newest migration this build ships, and this build has nothing left to apply, the
464
+ ledger is an older image rolled back onto a newer build's schema. Nothing is applied, the rows are
465
+ reported in `ahead` and logged as `ultimate migrate ledger ahead of build`; any other unknown row is
466
+ still `X_MIGRATION_CONFLICT`.
467
+
468
+ Report (`--json`): `{ applied: [{ id, name, durationMs }], skipped: [id], durationMs, appVersion, ahead: [id] }`.
411
469
 
412
470
  ## A loop of queries that is deliberate says so
413
471
 
@@ -495,11 +553,14 @@ job the moment Postgres fails over.
495
553
  | `X_DB_UNIQUE_VIOLATION` | `23505` — `fix:` names `upsertAll(rows, { onConflict: [...] })` and the constraint the server named |
496
554
  | `X_DB_FOREIGN_KEY_VIOLATION` | `23503` |
497
555
  | `X_DB_SERIALIZATION_FAILURE` | `40001` / `40P01`, and an exhausted `withTransaction(fn, { retry: n })` budget |
556
+ | `X_DB_TRANSACTION_ABORTED` | a statement failed inside a transaction and its error was caught; the server rolled the unit of work back. Also any `COMMIT` answered with the tag `ROLLBACK` |
557
+ | `X_DB_COMMIT_UNKNOWN` | `COMMIT` was sent and the connection failed before the answer: durable or rolled back, unknowable from here |
558
+ | `X_DB_SIBLING_SCOPE_TIMEOUT` | a nested `withTransaction` waited past `siblingWaitMs` for a sibling scope that never finished |
498
559
  | `X_DB_STATEMENT_TIMEOUT` | `57014` — the statement ran past `statement_timeout` |
499
560
  | `X_DB_LOCK_TIMEOUT` | `55P03` — it waited past `lock_timeout` for a lock it never got |
500
561
  | `X_DB_POOL_EXHAUSTED` | `53300` / `53200`, or `reserve()` past `acquireTimeoutMs` |
501
562
  | `X_DB_DRIFT` | live schema differs from migrations |
502
- | `X_MIGRATION_CONFLICT` | ledger app-version fence or checksum mismatch |
563
+ | `X_MIGRATION_CONFLICT` | ledger app-version fence or checksum mismatch — never for rows that are all newer than the build (a rollback) |
503
564
  | `X_MIGRATE_CONCURRENT` | another migrator still held the lock when the wait ran out |
504
565
  | `X_MIGRATION_IRREVERSIBLE` | generated `down` would lose data |
505
566
  | `X_SQL_UNSAFE` | non-bindable interpolation, or an unsafe identifier/branch name |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@ultimat3/db",
3
- "version": "23.0.0",
3
+ "version": "25.0.0",
4
4
  "description": "Postgres access, transactions, migrations and drift detection",
5
5
  "license": "MIT",
6
6
  "type": "module",
@@ -26,14 +26,14 @@
26
26
  "LICENSE"
27
27
  ],
28
28
  "engines": {
29
- "bun": ">=1.4.0"
29
+ "bun": ">=1.4.2"
30
30
  },
31
31
  "scripts": {
32
32
  "typecheck": "tsc --noEmit -p tsconfig.json",
33
33
  "test": "bun test"
34
34
  },
35
35
  "dependencies": {
36
- "@ultimat3/core": "23.0.0"
36
+ "@ultimat3/core": "25.0.0"
37
37
  },
38
38
  "peerDependencies": {
39
39
  "@electric-sql/pglite": ">=0.5.0"