@ultimat3/db 14.0.0 → 15.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +95 -0
- package/README.md +13 -2
- package/package.json +2 -2
- package/src/check-ddl.ts +17 -5
- package/src/drift-findings.ts +231 -0
- package/src/drift.ts +47 -182
- package/src/generate.ts +26 -133
- package/src/index-ddl.ts +137 -0
- package/src/introspect.ts +46 -1
- package/src/retype-dependents.ts +135 -0
package/CLAUDE.md
CHANGED
|
@@ -653,6 +653,101 @@ right database, and `x db gen`'s `retypeColumn` owns that question where both si
|
|
|
653
653
|
The `fix:` is the `alter table … set not null` itself and deliberately not `x db gen`, which has
|
|
654
654
|
never emitted one and would answer with an empty migration.
|
|
655
655
|
|
|
656
|
+
**A CHECK that went missing is drift, `As of 2026-08-25`, and it is compared by NAME because it
|
|
657
|
+
cannot be compared any other way.** `pg_get_constraintdef` answers Postgres' own rewriting —
|
|
658
|
+
`status in ('draft', 'published')` reads back as
|
|
659
|
+
`CHECK ((status = ANY (ARRAY['draft'::text, 'published'::text])))`, measured on 18.4
|
|
660
|
+
(`drift-check.live.test.ts`) — so a catalog value could never equal a generated one and a text
|
|
661
|
+
comparison reports a correct database as wrong forever. That is why nothing here read
|
|
662
|
+
`pg_constraint` for CHECKs at all, and why `alter table … drop constraint` in a psql session was
|
|
663
|
+
`ok: true` on every check that followed it.
|
|
664
|
+
|
|
665
|
+
**The two readings do not share a field, and that split is the whole design.**
|
|
666
|
+
`TableDescription.checks` is the DECLARED side — name **and** expression, `snapshotOf`'s own
|
|
667
|
+
spelling, the value `checkPlan` diffs. `TableDescription.checkNames` is the CATALOG side — `conname`
|
|
668
|
+
for `contype = 'c'`, names and nothing else, written only by `introspect()`. Filling `checks` from
|
|
669
|
+
the catalog instead would put a rewritten expression where `checkPlan` expects a generated one, and
|
|
670
|
+
every `x db gen` in every app would then drop and re-add every constraint it has, forever, because
|
|
671
|
+
the two strings can never be equal. Split, the TYPE says which reading a value came from and
|
|
672
|
+
`checkPlan` cannot be handed a catalog value by accident.
|
|
673
|
+
|
|
674
|
+
Three rules ride with it. **Absent and `[]` are different on both sides** — an absent `checks` is a
|
|
675
|
+
sidecar written before the field existed (declares nothing, so nothing can be missing), and an
|
|
676
|
+
absent `checkNames` is a description that never asked the catalog, which reading as "the database
|
|
677
|
+
holds none" is one finding per declared constraint against a database nobody looked at.
|
|
678
|
+
`introspect()` therefore always writes `checkNames`, `[]` included. **Only the declared side is
|
|
679
|
+
judged**, the rule `compareIndexes` and `compareForeignKeys` already state: a NOT NULL (`contype =
|
|
680
|
+
'n'` from Postgres 17 on), an `enumerated()` column's old anonymous form and every constraint an
|
|
681
|
+
extension brought would each be a finding against a database that is exactly right. **There is no
|
|
682
|
+
`changed-check` and there never will be** — presence is a boolean, the predicate is text, and
|
|
683
|
+
normalising the text is an expression parser competing with the server's. `missing-check`'s `fix:`
|
|
684
|
+
is the `add constraint` statement itself, not `x db migrate`: the migration declaring it is already
|
|
685
|
+
in the ledger, so the migrator applies nothing, and the declared side carries the predicate that
|
|
686
|
+
makes an executable fix possible at all.
|
|
687
|
+
|
|
688
|
+
**A retype takes the objects written against the column out of its way first, `As of 2026-08-25`,
|
|
689
|
+
and `retype-dependents.ts` decides which those are.** Postgres compiles a partial index's predicate
|
|
690
|
+
and a CHECK's expression against the column's type at creation and cannot recompile either:
|
|
691
|
+
`alter table "posts" alter column "status" type text using "status"::text` answered
|
|
692
|
+
`42883 operator does not exist: text = post_status` and the migration aborted mid-run — inside
|
|
693
|
+
`ROLE=migrate`, with the ledger recording nothing. It is what blocked `examples/dummy` from
|
|
694
|
+
regenerating at all.
|
|
695
|
+
|
|
696
|
+
**Which objects are dependent is measured, never assumed** (`generate-retype.live.test.ts`, one
|
|
697
|
+
shape at a time on 18.4):
|
|
698
|
+
|
|
699
|
+
| recorded object | survives the ALTER |
|
|
700
|
+
|---|---|
|
|
701
|
+
| btree over the column — plain, unique or composite | **yes**, Postgres rebuilds it itself |
|
|
702
|
+
| partial index whose predicate names the column | **no — 42883** |
|
|
703
|
+
| partial index naming another column | yes |
|
|
704
|
+
| CHECK whose expression names the column | **no — 42883** |
|
|
705
|
+
| a view over the column | no, `0A000`, and no snapshot records a view |
|
|
706
|
+
|
|
707
|
+
So only an expression that MENTIONS the column is moved, and a plain btree is left alone — dropping
|
|
708
|
+
it is a table scan to rebuild for nothing.
|
|
709
|
+
|
|
710
|
+
**The reference test over-approximates on purpose, and it cannot be narrowed by type name.**
|
|
711
|
+
Measured: `char(1)` → `char(3)`, `varchar(80)` → `text` and `numeric` → `integer` all re-derive
|
|
712
|
+
their predicates cleanly, while `integer` → `text` under `check (c >= 0)` is `42883` — both sides
|
|
713
|
+
built-ins. Whether an expression re-resolves depends on operator resolution, which is exactly the
|
|
714
|
+
knowledge a generator with no database cannot have, so every ambiguous case answers "dependent":
|
|
715
|
+
a miss is `42883` in the release phase, a false positive is a rebuild on a statement that is
|
|
716
|
+
already rewriting the whole table under ACCESS EXCLUSIVE. `referencesColumn` walks `sql-scan.ts`
|
|
717
|
+
rather than matching a substring — a name inside a literal or a comment is not a reference,
|
|
718
|
+
`status_code` is not `status`, and a **quoted** identifier IS one, which is the one span the lexer
|
|
719
|
+
calls noise and this reader must not skip.
|
|
720
|
+
|
|
721
|
+
**What is moved aside is put back by the ORDINARY diff, never twice.** `up` drops the dependents
|
|
722
|
+
before the ALTER and `MovedAside` carries their names to the two arms that would otherwise act on a
|
|
723
|
+
thing that is no longer there: the index loop CREATES a declared name instead of comparing it
|
|
724
|
+
(`redefineIndex` is silent on a definition that never moved, which here means the table comes out
|
|
725
|
+
with no index at all), and `checkPlan` neither drops nor re-adds a predropped name — a declared one
|
|
726
|
+
takes the bare `add constraint` because the name is provably free, and a recorded one the entity no
|
|
727
|
+
longer declares is simply gone, which is what `checkPlan` would have done to it anyway. `down`
|
|
728
|
+
pushes the restores forwards and is reversed as a whole, so it reads: drop the new objects, retype
|
|
729
|
+
back, then recreate the ones compiled against the old type — restoring first is `42883` in the
|
|
730
|
+
other direction. What it restores is what the snapshot RECORDED, never what the entity declares.
|
|
731
|
+
|
|
732
|
+
**Three things it does not move, and each is measured rather than assumed.** A **foreign key** over
|
|
733
|
+
the retyped column is `ERROR: foreign key constraint "c_k_fkey" cannot be implemented` — the
|
|
734
|
+
dependents are in `live.foreignKeys` *and* in every OTHER table's, which `diffTable` cannot see from
|
|
735
|
+
one entity, and the statements belong to `foreign-key-plan.ts`'s own buckets, so half a fix here
|
|
736
|
+
would collide with it. Neither tracked app can reach it (every key is `uuid` on both sides) and it
|
|
737
|
+
is the next thing to close. A **generated column's** own retype (`regenerate`) does not move
|
|
738
|
+
dependents either: it emits `set expression` and `alter … type` with no `using`, and nothing covers
|
|
739
|
+
a partial index over one. And **what no migration wrote down** is invisible by construction —
|
|
740
|
+
`x db gen` runs with no database open, so a hand-added expression index over the column is still
|
|
741
|
+
`42883` and a VIEW over it is `0A000` whatever this does, since `SchemaDescription` has a field for
|
|
742
|
+
neither.
|
|
743
|
+
|
|
744
|
+
**`index-ddl.ts` holds `createIndex`, `redefineIndex`, `indexShape`, `dropIndex` and
|
|
745
|
+
`asDeclared`**, split out of `generate.ts` at the 500-line ceiling along the seam `check-ddl.ts` and
|
|
746
|
+
`generated-column.ts` already drew — `generate.ts` assembles a plan, `index-ddl.ts` writes the index
|
|
747
|
+
statements in it. `drift-findings.ts` is the same split on the other file: every `DriftDifference`
|
|
748
|
+
constructor and the `DriftKind` union, with `drift.ts` keeping the comparisons and re-exporting both
|
|
749
|
+
types explicitly so the public surface does not move.
|
|
750
|
+
|
|
656
751
|
**An entity's INVARIANTS reach the DDL, `As of 2026-08-25`, and `invariant-ddl.ts` is what they
|
|
657
752
|
become.** `EntityDescriptionLike` had no `invariants` field at all — the same seam gap
|
|
658
753
|
`onDelete` carried until 3.0 — so a regenerated migration held **none** of them: measured on
|
package/README.md
CHANGED
|
@@ -36,13 +36,13 @@ await withTransaction(async (tx) => {
|
|
|
36
36
|
| `replicatedClient()` / `ReplicaStats` / `REPLICA_URL_ENV` | `As of 2026-08-24`: one `DbClient` over a primary and a standby. `baseClient()` builds one when `DATABASE_REPLICA_URL` is set and the single-pool client when it is not |
|
|
37
37
|
| `INDEX_METHODS` / `IndexMethod` / `indexMethodOf()` / `indexMethodSql()` / `declaredMethod()` / `isIndexMethod()` | `As of 2026-08-24`: an index's access method — `btree` or `gin`, closed. Absent is `btree`, the live side is read open (whatever `pg_am` said), and the DDL literal is re-derived from the set rather than spliced from the input |
|
|
38
38
|
| `isPlainRead()` | `As of 2026-08-24`: whether a statement may leave the primary. An allow-list — everything it cannot vouch for is the primary's |
|
|
39
|
-
| `checkDrift()` / `diffSchema()` / `assertNoDrift()` | drift, with a `--json` report. `checkDrift()` is the **post-migrate verification** — the live database against the ledger: columns, declared indexes (access method `As of 2026-08-24`, columns, uniqueness, direction, and whether a predicate is there at all — never its text) and declared foreign keys, matched on where the key points and not on its constraint name, with the `on delete` rule compared through one normalisation `As of 2026-08-19` |
|
|
39
|
+
| `checkDrift()` / `diffSchema()` / `assertNoDrift()` | drift, with a `--json` report. `checkDrift()` is the **post-migrate verification** — the live database against the ledger: columns, declared indexes (access method `As of 2026-08-24`, columns, uniqueness, direction, and whether a predicate is there at all — never its text), declared CHECK constraints by NAME (`missing-check`, `As of 2026-08-25` — never a predicate, which the catalog answers rewritten) and declared foreign keys, matched on where the key points and not on its constraint name, with the `on delete` rule compared through one normalisation `As of 2026-08-19` |
|
|
40
40
|
| `declaredSchema()` / `expectedSchema()` | `As of 2026-08`: the schema the migrations write down, or `undefined` when the newest one carries no snapshot — never an older snapshot standing in for it |
|
|
41
41
|
| `parseSnapshot()` | `As of 2026-08`: a `<id>.snapshot.json` sidecar validated to the last nested field, or `undefined`. `{"tables":[null]}` is valid JSON and is not a schema |
|
|
42
42
|
| `snapshotJson()` | `As of 2026-08`: the sidecar's **bytes** — the JSON Biome would have printed, trailing newline included. The one writer of a `<id>.snapshot.json`, because `JSON.stringify(…, null, 2)` is not formatter-clean and an app's `lint` step rejected the file `x db gen` had just written |
|
|
43
43
|
| `isLedgerMissing()` | `As of 2026-08`: whether an error is Postgres' `undefined_table` for `x_migrations` — the one condition a caller may read as "nothing applied" |
|
|
44
44
|
| `appTables()` / `FRAMEWORK_TABLE_PREFIX` | `As of 2026-08`: the live schema minus the `x_` namespace — no migration declares the ledger, the queue, the outbox or an auth table, so none of them is drift |
|
|
45
|
-
| `generateMigration()` | `x db gen "<name>"` — reversible up/down SQL, and `destructive` for the marker the file must carry. `As of 2026-08` a foreign key is its own `alter table … add constraint`, emitted after every table statement: inline, a `references()` had to point at a table entity registration order happened to create first, and `down` had to drop them in an order it did not control. `As of 2026-08-19` a **removed** `references()` emits its `drop constraint` (it emitted nothing, and the snapshot then denied a constraint the database still held), a changed `onDelete` is a drop-and-add rebuild, and a declared `on delete` rule reaches the clause at all |
|
|
45
|
+
| `generateMigration()` | `x db gen "<name>"` — reversible up/down SQL, and `destructive` for the marker the file must carry. `As of 2026-08` a foreign key is its own `alter table … add constraint`, emitted after every table statement: inline, a `references()` had to point at a table entity registration order happened to create first, and `down` had to drop them in an order it did not control. `As of 2026-08-19` a **removed** `references()` emits its `drop constraint` (it emitted nothing, and the snapshot then denied a constraint the database still held), a changed `onDelete` is a drop-and-add rebuild, and a declared `on delete` rule reaches the clause at all. `As of 2026-08-25` a **retype** drops the partial indexes and CHECK constraints written against that column first and restores them in `down`: Postgres compiles both predicates against the old type and cannot recompile either, so `alter column … type text using …::text` was `42883 operator does not exist: text = post_status` and the migration aborted mid-run. A plain btree over the column is left alone — measured, Postgres rebuilds that one itself |
|
|
46
46
|
| `declaredIndexes()` / `invariantChecks()` / `constraintNameFor()` | `As of 2026-08-25`: the DDL an entity **invariant** becomes — a `check` as a named `CONSTRAINT`, a `unique` as a partial-capable unique INDEX, an `assert` as nothing. `EntityDescriptionLike` had no `invariants` field for three majors, so a regenerated migration silently held **none** of them, including the composite UNIQUE `upsertAll`'s `on conflict` is inferred against |
|
|
47
47
|
| `declaredChecks()` / `checkClauses()` / `checkPlan()` / `columnChecks()` / `columnCheckName()` / `columnNamesConstraint()` | `As of 2026-08-25`: **every** CHECK a table declares — a column's own (`enumerated()`'s value set, `tz()`'s IANA whitelist, `locale()`'s tags, money's currency pattern and scale bound) and an invariant's — on ONE list, so `createTable`, `diffTable` and `snapshotOf` agree about what exists. A column's check reached `create table` **inline and anonymous** and nothing else: the snapshot recorded none and the diff had no arm, so a value added to `enumerated()` generated no migration and a regenerated ENUM column came back as bare `text`. The name is `<table>_<column>_check` because that is the name **Postgres itself mints** for the old anonymous form — measured — so the repair lands on the constraint an already-generated database is holding; `checkPlan` emits `drop constraint if exists` before the `add` for exactly that column, because a bare add is `42710` there and a no-op everywhere else |
|
|
48
48
|
| `defaultExpression()` / `ColumnDefaultLike` | `As of 2026-08-25`: a column's `default` as SQL. A DECLARED default (`{ kind: 'value', value }`) wins; `gen_random_uuid()` and `now()` stay as the inference for a description that carries only `hasDefault` |
|
|
@@ -185,6 +185,17 @@ table in the `x_` namespace — `x_migrations`, the queue's tables, the outbox a
|
|
|
185
185
|
snapshot, so `appTables()` drops them before the diff. `introspect()` keeps its own narrower
|
|
186
186
|
exclusion (the ledger alone), reserving `x_users` for a schema view that wants it.
|
|
187
187
|
|
|
188
|
+
**A CHECK the catalog no longer holds is drift, `As of 2026-08-25` — by NAME.**
|
|
189
|
+
`pg_get_constraintdef` answers Postgres' own rewriting (`status in ('draft', 'published')` reads
|
|
190
|
+
back as `CHECK ((status = ANY (ARRAY['draft'::text, 'published'::text])))`), so the catalog side is
|
|
191
|
+
read as `conname` alone and lands on `TableDescription.checkNames`, a **separate field** from the
|
|
192
|
+
declaration's `checks`. Only the declared side is judged, so a NOT NULL, an `enumerated()` column's
|
|
193
|
+
old anonymous form and an extension's own constraint are all silent; a declared one the catalog
|
|
194
|
+
does not hold is `missing-check`, whose `fix:` is the `add constraint` statement itself, because the
|
|
195
|
+
migration that declares it is already in the ledger and `x db migrate` would apply nothing. There is
|
|
196
|
+
no `changed-check`: presence is a boolean, a predicate is text, and normalising the text is an
|
|
197
|
+
expression parser competing with the server's.
|
|
198
|
+
|
|
188
199
|
**Nor is a relation an extension owns, `As of 2026-08-24`.** `create extension pg_stat_statements`
|
|
189
200
|
in `public` is the CNPG, RDS, Supabase and Neon default, and its view read as `unexpected-table`
|
|
190
201
|
with `x db gen "add pg_stat_statements"` as the fix — so every deploy failed terminally and the fix
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ultimat3/db",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "15.0.0",
|
|
4
4
|
"description": "Postgres access, transactions, migrations and drift detection",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -31,7 +31,7 @@
|
|
|
31
31
|
"test": "bun test"
|
|
32
32
|
},
|
|
33
33
|
"dependencies": {
|
|
34
|
-
"@ultimat3/core": "
|
|
34
|
+
"@ultimat3/core": "15.0.0"
|
|
35
35
|
},
|
|
36
36
|
"peerDependencies": {
|
|
37
37
|
"@electric-sql/pglite": ">=0.5.0"
|
package/src/check-ddl.ts
CHANGED
|
@@ -125,11 +125,13 @@ export function checkClauses(entity: EntityDescriptionLike): readonly string[] {
|
|
|
125
125
|
);
|
|
126
126
|
}
|
|
127
127
|
|
|
128
|
-
|
|
128
|
+
/** Exported for `retype-dependents.ts`: a constraint moved out of a retype's way is put back by
|
|
129
|
+
* the same statement that would have added it, never by a second spelling of `add constraint`. */
|
|
130
|
+
export const addCheck = (table: string, check: CheckDescription): string =>
|
|
129
131
|
`alter table ${identifier(table).text} add constraint ${identifier(check.name).text} ` +
|
|
130
132
|
`check (${check.expression});`;
|
|
131
133
|
|
|
132
|
-
const dropCheck = (table: string, name: string): string =>
|
|
134
|
+
export const dropCheck = (table: string, name: string): string =>
|
|
133
135
|
`alter table ${identifier(table).text} drop constraint ${identifier(name).text};`;
|
|
134
136
|
|
|
135
137
|
/**
|
|
@@ -161,12 +163,21 @@ const rebuildCheck = (table: string, check: CheckDescription): readonly string[]
|
|
|
161
163
|
* plain -> generated path). The constraint went with the column and the snapshot still records it,
|
|
162
164
|
* so without this the check would be silently gone — the defect class this file exists against,
|
|
163
165
|
* one level in.
|
|
166
|
+
*
|
|
167
|
+
* `predropped` names the CONSTRAINTS this plan already dropped, ahead of a retype whose predicate
|
|
168
|
+
* they were compiled against (`retype-dependents.ts`). Two arms read it and both are about a name
|
|
169
|
+
* that is provably free: a declared one takes the bare `add constraint` rather than the
|
|
170
|
+
* drop-if-exists pair, and a recorded one the entity no longer declares is left alone entirely —
|
|
171
|
+
* `drop constraint` on it a second time is `42704`, and its `down` belongs to the retype that
|
|
172
|
+
* moved it. Keyed by name and not by column because an INVARIANT's check reads a column without
|
|
173
|
+
* being derived from one, which is exactly the constraint `examples/dummy` retypes under.
|
|
164
174
|
*/
|
|
165
175
|
export function checkPlan(
|
|
166
176
|
entity: EntityDescriptionLike,
|
|
167
177
|
live: TableDescription,
|
|
168
178
|
plan: { up: string[]; down: string[] },
|
|
169
179
|
rebuilt: ReadonlySet<string> = new Set(),
|
|
180
|
+
predropped: ReadonlySet<string> = new Set(),
|
|
170
181
|
): void {
|
|
171
182
|
const recorded = new Map((live.checks ?? []).map((check) => [check.name, check]));
|
|
172
183
|
const present = new Set(live.columns.map((column) => column.name));
|
|
@@ -189,10 +200,11 @@ export function checkPlan(
|
|
|
189
200
|
);
|
|
190
201
|
const wanted = declaredChecks(entity);
|
|
191
202
|
for (const check of wanted) {
|
|
192
|
-
const
|
|
203
|
+
const gone = dropped.has(check.name) || predropped.has(check.name);
|
|
204
|
+
const held = gone ? undefined : recorded.get(check.name);
|
|
193
205
|
if (held === undefined) {
|
|
194
206
|
plan.up.push(
|
|
195
|
-
...(exposed.has(check.name)
|
|
207
|
+
...(exposed.has(check.name) && !predropped.has(check.name)
|
|
196
208
|
? rebuildCheck(entity.table, check)
|
|
197
209
|
: [addCheck(entity.table, check)]),
|
|
198
210
|
);
|
|
@@ -205,7 +217,7 @@ export function checkPlan(
|
|
|
205
217
|
}
|
|
206
218
|
const declared = new Set(wanted.map((check) => check.name));
|
|
207
219
|
for (const check of live.checks ?? []) {
|
|
208
|
-
if (declared.has(check.name)) continue;
|
|
220
|
+
if (declared.has(check.name) || predropped.has(check.name)) continue;
|
|
209
221
|
plan.up.push(dropCheck(entity.table, check.name));
|
|
210
222
|
plan.down.push(addCheck(entity.table, check));
|
|
211
223
|
}
|
|
@@ -0,0 +1,231 @@
|
|
|
1
|
+
// Single responsibility: what a schema difference is CALLED and what its `fix:` line says — one
|
|
2
|
+
// constructor per `DriftKind`, and nothing that compares anything. Split out of `drift.ts` at the
|
|
3
|
+
// 500-line ceiling, along the seam that file already drew: comparison decides *whether* two
|
|
4
|
+
// schemas disagree, and this decides how the disagreement reads.
|
|
5
|
+
//
|
|
6
|
+
// The rendered `X_DB_DRIFT` output is byte-for-byte pinned by the framework contract and
|
|
7
|
+
// duplicated in `@ultimat3/entity` — do not reword a `cause` without changing both.
|
|
8
|
+
//
|
|
9
|
+
// Two rules run through every one of them. A `fix:` is a command the reader can RUN: `x db
|
|
10
|
+
// migrate` where the migration has not been applied, and the statement itself where it has, since
|
|
11
|
+
// re-running the migrator applies nothing a ledger row already claims. And a difference names the
|
|
12
|
+
// declared side's own spelling, never the catalog's, because the catalog's is Postgres' rewriting.
|
|
13
|
+
|
|
14
|
+
import { onDeleteRule, rebuildForeignKey } from './foreign-key';
|
|
15
|
+
import type { CheckDescription, ForeignKeyDescription } from './introspect';
|
|
16
|
+
import type { Migration } from './migrate';
|
|
17
|
+
|
|
18
|
+
export type DriftKind =
|
|
19
|
+
| 'unexpected-column'
|
|
20
|
+
| 'missing-column'
|
|
21
|
+
| 'changed-column'
|
|
22
|
+
| 'unexpected-table'
|
|
23
|
+
| 'missing-table'
|
|
24
|
+
| 'unknown-schema'
|
|
25
|
+
| 'missing-index'
|
|
26
|
+
| 'changed-index'
|
|
27
|
+
| 'missing-check'
|
|
28
|
+
| 'missing-foreign-key'
|
|
29
|
+
| 'changed-foreign-key';
|
|
30
|
+
|
|
31
|
+
export interface DriftDifference {
|
|
32
|
+
readonly kind: DriftKind;
|
|
33
|
+
readonly table: string;
|
|
34
|
+
readonly column: string | null;
|
|
35
|
+
readonly cause: string;
|
|
36
|
+
readonly fix: string;
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
export interface DriftReport {
|
|
40
|
+
readonly ok: boolean;
|
|
41
|
+
readonly differences: readonly DriftDifference[];
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
export function unexpectedColumn(table: string, column: string): DriftDifference {
|
|
45
|
+
return {
|
|
46
|
+
kind: 'unexpected-column',
|
|
47
|
+
table,
|
|
48
|
+
column,
|
|
49
|
+
// Pinned by the contract. Do not reword without changing docs/errors/X_DB_DRIFT.
|
|
50
|
+
cause: `table "${table}" has column "${column}" not present in any migration`,
|
|
51
|
+
fix: `x db gen "add ${column}"`,
|
|
52
|
+
};
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
export function missingColumn(table: string, column: string): DriftDifference {
|
|
56
|
+
return {
|
|
57
|
+
kind: 'missing-column',
|
|
58
|
+
table,
|
|
59
|
+
column,
|
|
60
|
+
cause: `table "${table}" is missing column "${column}" that migrations declare`,
|
|
61
|
+
fix: 'x db migrate',
|
|
62
|
+
};
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
/**
|
|
66
|
+
* The column exists on both sides and one of them lets it be `NULL`.
|
|
67
|
+
*
|
|
68
|
+
* This is the finding the expand/contract flow needs and never had. `generate.ts` emits a `NOT
|
|
69
|
+
* NULL` add as nullable plus a `-- backfill "c", then: … set not null;` comment, because the
|
|
70
|
+
* strict version cannot succeed on a populated table — and phase 2 is a comment, so it is a thing
|
|
71
|
+
* a human has to remember. Nobody did, and `compareTable` compared columns by name and by type
|
|
72
|
+
* while `snapshotOf` had recorded `nullable` all along, so the column stayed nullable forever
|
|
73
|
+
* against an entity schema that said otherwise, with `ok: true` on every check. The first
|
|
74
|
+
* `undefined` write then lands as `NULL` and crashes three services away from the migration.
|
|
75
|
+
*
|
|
76
|
+
* `x db gen` is deliberately not the fix: it diffs types and indexes and has never emitted a
|
|
77
|
+
* `set not null`, so naming it would send a reader to a command that generates an empty migration.
|
|
78
|
+
*/
|
|
79
|
+
export function changedColumn(
|
|
80
|
+
table: string,
|
|
81
|
+
column: string,
|
|
82
|
+
liveNullable: boolean,
|
|
83
|
+
): DriftDifference {
|
|
84
|
+
const clause = liveNullable ? 'set not null' : 'drop not null';
|
|
85
|
+
return {
|
|
86
|
+
kind: 'changed-column',
|
|
87
|
+
table,
|
|
88
|
+
column,
|
|
89
|
+
cause: liveNullable
|
|
90
|
+
? `table "${table}" allows NULL in column "${column}" that migrations declare not null`
|
|
91
|
+
: `table "${table}" forbids NULL in column "${column}" that migrations declare nullable`,
|
|
92
|
+
fix:
|
|
93
|
+
`alter table "${table}" alter column "${column}" ${clause}; # in a new migration` +
|
|
94
|
+
(liveNullable ? ' — backfill the existing NULLs first' : ''),
|
|
95
|
+
};
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
export function unexpectedTable(table: string): DriftDifference {
|
|
99
|
+
return {
|
|
100
|
+
kind: 'unexpected-table',
|
|
101
|
+
table,
|
|
102
|
+
column: null,
|
|
103
|
+
cause: `table "${table}" is not present in any migration`,
|
|
104
|
+
fix: `x db gen "add ${table}"`,
|
|
105
|
+
};
|
|
106
|
+
}
|
|
107
|
+
|
|
108
|
+
export function missingTable(table: string): DriftDifference {
|
|
109
|
+
return {
|
|
110
|
+
kind: 'missing-table',
|
|
111
|
+
table,
|
|
112
|
+
column: null,
|
|
113
|
+
cause: `table "${table}" is declared by migrations but does not exist`,
|
|
114
|
+
fix: 'x db migrate',
|
|
115
|
+
};
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
/**
|
|
119
|
+
* Not a difference between two schemas but the absence of one to compare against — reported
|
|
120
|
+
* through the same channel so it reaches an operator, since a check that quietly answered "clean"
|
|
121
|
+
* because it had nothing to check is the one failure mode drift detection cannot have.
|
|
122
|
+
*/
|
|
123
|
+
export function unknownSchema(migrations: readonly Migration[]): DriftDifference {
|
|
124
|
+
const newest = [...migrations].sort((a, b) => (a.id < b.id ? -1 : 1)).at(-1);
|
|
125
|
+
return {
|
|
126
|
+
kind: 'unknown-schema',
|
|
127
|
+
table: '',
|
|
128
|
+
column: null,
|
|
129
|
+
cause:
|
|
130
|
+
`migration "${newest?.id ?? ''}" records no schema snapshot, so what this database owes ` +
|
|
131
|
+
'cannot be established',
|
|
132
|
+
// The same two remedies `X_MIGRATION_SNAPSHOT_MISSING` names, in the same order, because it is
|
|
133
|
+
// the same condition. It used to lead with `x db gen`, which raises that error and whose own
|
|
134
|
+
// fix pointed back here — a cycle a scaffolded app hit on its first `x db migrate`. The
|
|
135
|
+
// pathspec is a glob because this package is tier 1: only `@ultimat3/cli` knows the directory.
|
|
136
|
+
fix:
|
|
137
|
+
`git checkout -- "*${newest?.id ?? ''}.snapshot.json" # or, if it was never written: ` +
|
|
138
|
+
`delete migration "${newest?.id ?? ''}" and rerun x db gen "${newest?.name ?? 'initial'}"`,
|
|
139
|
+
};
|
|
140
|
+
}
|
|
141
|
+
|
|
142
|
+
export function missingIndex(table: string, index: string): DriftDifference {
|
|
143
|
+
return {
|
|
144
|
+
kind: 'missing-index',
|
|
145
|
+
table,
|
|
146
|
+
column: null,
|
|
147
|
+
cause: `table "${table}" is missing index "${index}" that migrations declare`,
|
|
148
|
+
fix: 'x db migrate',
|
|
149
|
+
};
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
export function changedIndex(table: string, index: string, detail: string): DriftDifference {
|
|
153
|
+
return {
|
|
154
|
+
kind: 'changed-index',
|
|
155
|
+
table,
|
|
156
|
+
column: null,
|
|
157
|
+
cause: `index "${index}" on "${table}" ${detail}, not what migrations declare`,
|
|
158
|
+
fix: 'x db migrate',
|
|
159
|
+
};
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
/**
|
|
163
|
+
* A CHECK a migration declares and the catalog does not hold.
|
|
164
|
+
*
|
|
165
|
+
* There is no `changed-check` beside it and there never will be, for the reason
|
|
166
|
+
* `IndexDescription.where` gives: `pg_get_constraintdef` answers Postgres' own rewriting —
|
|
167
|
+
* `status in ('draft','published')` reads back as `CHECK ((status = ANY (ARRAY['draft'::text,
|
|
168
|
+
* 'published'::text])))` — so a text comparison reports drift on a correct database forever, and
|
|
169
|
+
* normalising it is an expression parser competing with the server's. Presence is not text.
|
|
170
|
+
*
|
|
171
|
+
* The `fix` is the statement, not `x db migrate`: the migration that declares this constraint is
|
|
172
|
+
* already in the ledger, so re-running the migrator applies nothing. Same reasoning as
|
|
173
|
+
* `changedColumn` and `changedForeignKey` — the declared side holds the author's own spelling of
|
|
174
|
+
* the predicate, which is what makes an executable fix possible at all.
|
|
175
|
+
*/
|
|
176
|
+
export function missingCheck(table: string, check: CheckDescription): DriftDifference {
|
|
177
|
+
return {
|
|
178
|
+
kind: 'missing-check',
|
|
179
|
+
table,
|
|
180
|
+
column: null,
|
|
181
|
+
cause: `table "${table}" is missing check constraint "${check.name}" that migrations declare`,
|
|
182
|
+
// The command rides on the same line as the statement, and not only because `check` is a
|
|
183
|
+
// banned advice word the `errors` gate demands a command beside: writing the migration is half
|
|
184
|
+
// the repair and applying it is the other half, and `changedColumn`'s bare `# in a new
|
|
185
|
+
// migration` leaves the second half to be guessed.
|
|
186
|
+
fix:
|
|
187
|
+
`alter table "${table}" add constraint "${check.name}" ` +
|
|
188
|
+
`check (${check.expression}); # in a new migration, then x db migrate`,
|
|
189
|
+
};
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
export function missingForeignKey(table: string, key: ForeignKeyDescription): DriftDifference {
|
|
193
|
+
return {
|
|
194
|
+
kind: 'missing-foreign-key',
|
|
195
|
+
table,
|
|
196
|
+
column: null,
|
|
197
|
+
cause:
|
|
198
|
+
`table "${table}" has no foreign key on (${key.columns.join(', ')}) to ` +
|
|
199
|
+
`"${key.referencedTable}" (${key.referencedColumns.join(', ')}) that migrations declare`,
|
|
200
|
+
fix: 'x db migrate',
|
|
201
|
+
};
|
|
202
|
+
}
|
|
203
|
+
|
|
204
|
+
/**
|
|
205
|
+
* The key points where it was declared to point and one side's `on delete` rule is not the other's
|
|
206
|
+
* — reported apart from `missing-foreign-key` because it is a different repair: the constraint is
|
|
207
|
+
* there, and what changed is what happens to the child rows.
|
|
208
|
+
*
|
|
209
|
+
* The `fix` is the pair, not `x db migrate`: a rule cannot be altered in place, `add constraint`
|
|
210
|
+
* alone is `42710` on a name already taken, and no `x db gen` diff emits either statement, so
|
|
211
|
+
* naming a command would send a reader to one that generates an empty migration. Same reasoning
|
|
212
|
+
* as `changedColumn`.
|
|
213
|
+
*/
|
|
214
|
+
export function changedForeignKey(
|
|
215
|
+
table: string,
|
|
216
|
+
declared: ForeignKeyDescription,
|
|
217
|
+
held: ForeignKeyDescription,
|
|
218
|
+
): DriftDifference {
|
|
219
|
+
const rule = onDeleteRule(held.onDelete);
|
|
220
|
+
return {
|
|
221
|
+
kind: 'changed-foreign-key',
|
|
222
|
+
table,
|
|
223
|
+
column: null,
|
|
224
|
+
cause:
|
|
225
|
+
`foreign key on "${table}" (${declared.columns.join(', ')}) to ` +
|
|
226
|
+
`"${declared.referencedTable}" ` +
|
|
227
|
+
`${rule === null ? 'declares no on delete rule' : `is on delete ${rule}`}, not what ` +
|
|
228
|
+
'migrations declare',
|
|
229
|
+
fix: `${rebuildForeignKey(table, declared, held)} # in a new migration`,
|
|
230
|
+
};
|
|
231
|
+
}
|
package/src/drift.ts
CHANGED
|
@@ -4,198 +4,35 @@
|
|
|
4
4
|
// by the framework contract; `x verify` fails on it and `--json` carries every difference.
|
|
5
5
|
|
|
6
6
|
import { baseClient, type DbClient } from './client';
|
|
7
|
+
import type { DriftDifference } from './drift-findings';
|
|
8
|
+
import {
|
|
9
|
+
changedColumn,
|
|
10
|
+
changedForeignKey,
|
|
11
|
+
changedIndex,
|
|
12
|
+
missingCheck,
|
|
13
|
+
missingColumn,
|
|
14
|
+
missingForeignKey,
|
|
15
|
+
missingIndex,
|
|
16
|
+
missingTable,
|
|
17
|
+
unexpectedColumn,
|
|
18
|
+
unexpectedTable,
|
|
19
|
+
unknownSchema,
|
|
20
|
+
} from './drift-findings';
|
|
7
21
|
import { DbError } from './errors';
|
|
8
|
-
import { foreignKeyTarget, onDeleteRule
|
|
22
|
+
import { foreignKeyTarget, onDeleteRule } from './foreign-key';
|
|
9
23
|
import { indexMethodOf } from './index-method';
|
|
10
|
-
import {
|
|
11
|
-
type ForeignKeyDescription,
|
|
12
|
-
findTable,
|
|
13
|
-
introspect,
|
|
14
|
-
type SchemaDescription,
|
|
15
|
-
type TableDescription,
|
|
16
|
-
} from './introspect';
|
|
24
|
+
import { findTable, introspect, type SchemaDescription, type TableDescription } from './introspect';
|
|
17
25
|
import { type LedgerRow, type Migration, readLedger } from './migrate';
|
|
18
26
|
|
|
19
|
-
export
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
| 'changed-column'
|
|
23
|
-
| 'unexpected-table'
|
|
24
|
-
| 'missing-table'
|
|
25
|
-
| 'unknown-schema'
|
|
26
|
-
| 'missing-index'
|
|
27
|
-
| 'changed-index'
|
|
28
|
-
| 'missing-foreign-key'
|
|
29
|
-
| 'changed-foreign-key';
|
|
30
|
-
|
|
31
|
-
export interface DriftDifference {
|
|
32
|
-
readonly kind: DriftKind;
|
|
33
|
-
readonly table: string;
|
|
34
|
-
readonly column: string | null;
|
|
35
|
-
readonly cause: string;
|
|
36
|
-
readonly fix: string;
|
|
37
|
-
}
|
|
27
|
+
// Re-exported explicitly, never `export *`: `src/index.ts` publishes both from `'./drift'`, so the
|
|
28
|
+
// split is invisible to `@ultimat3/db`'s public surface and no consumer moves with it.
|
|
29
|
+
export type { DriftDifference, DriftKind } from './drift-findings';
|
|
38
30
|
|
|
39
31
|
export interface DriftReport {
|
|
40
32
|
readonly ok: boolean;
|
|
41
33
|
readonly differences: readonly DriftDifference[];
|
|
42
34
|
}
|
|
43
35
|
|
|
44
|
-
function unexpectedColumn(table: string, column: string): DriftDifference {
|
|
45
|
-
return {
|
|
46
|
-
kind: 'unexpected-column',
|
|
47
|
-
table,
|
|
48
|
-
column,
|
|
49
|
-
// Pinned by the contract. Do not reword without changing docs/errors/X_DB_DRIFT.
|
|
50
|
-
cause: `table "${table}" has column "${column}" not present in any migration`,
|
|
51
|
-
fix: `x db gen "add ${column}"`,
|
|
52
|
-
};
|
|
53
|
-
}
|
|
54
|
-
|
|
55
|
-
function missingColumn(table: string, column: string): DriftDifference {
|
|
56
|
-
return {
|
|
57
|
-
kind: 'missing-column',
|
|
58
|
-
table,
|
|
59
|
-
column,
|
|
60
|
-
cause: `table "${table}" is missing column "${column}" that migrations declare`,
|
|
61
|
-
fix: 'x db migrate',
|
|
62
|
-
};
|
|
63
|
-
}
|
|
64
|
-
|
|
65
|
-
/**
|
|
66
|
-
* The column exists on both sides and one of them lets it be `NULL`.
|
|
67
|
-
*
|
|
68
|
-
* This is the finding the expand/contract flow needs and never had. `generate.ts` emits a `NOT
|
|
69
|
-
* NULL` add as nullable plus a `-- backfill "c", then: … set not null;` comment, because the
|
|
70
|
-
* strict version cannot succeed on a populated table — and phase 2 is a comment, so it is a thing
|
|
71
|
-
* a human has to remember. Nobody did, and `compareTable` compared columns by name and by type
|
|
72
|
-
* while `snapshotOf` had recorded `nullable` all along, so the column stayed nullable forever
|
|
73
|
-
* against an entity schema that said otherwise, with `ok: true` on every check. The first
|
|
74
|
-
* `undefined` write then lands as `NULL` and crashes three services away from the migration.
|
|
75
|
-
*
|
|
76
|
-
* `x db gen` is deliberately not the fix: it diffs types and indexes and has never emitted a
|
|
77
|
-
* `set not null`, so naming it would send a reader to a command that generates an empty migration.
|
|
78
|
-
*/
|
|
79
|
-
function changedColumn(table: string, column: string, liveNullable: boolean): DriftDifference {
|
|
80
|
-
const clause = liveNullable ? 'set not null' : 'drop not null';
|
|
81
|
-
return {
|
|
82
|
-
kind: 'changed-column',
|
|
83
|
-
table,
|
|
84
|
-
column,
|
|
85
|
-
cause: liveNullable
|
|
86
|
-
? `table "${table}" allows NULL in column "${column}" that migrations declare not null`
|
|
87
|
-
: `table "${table}" forbids NULL in column "${column}" that migrations declare nullable`,
|
|
88
|
-
fix:
|
|
89
|
-
`alter table "${table}" alter column "${column}" ${clause}; # in a new migration` +
|
|
90
|
-
(liveNullable ? ' — backfill the existing NULLs first' : ''),
|
|
91
|
-
};
|
|
92
|
-
}
|
|
93
|
-
|
|
94
|
-
function unexpectedTable(table: string): DriftDifference {
|
|
95
|
-
return {
|
|
96
|
-
kind: 'unexpected-table',
|
|
97
|
-
table,
|
|
98
|
-
column: null,
|
|
99
|
-
cause: `table "${table}" is not present in any migration`,
|
|
100
|
-
fix: `x db gen "add ${table}"`,
|
|
101
|
-
};
|
|
102
|
-
}
|
|
103
|
-
|
|
104
|
-
function missingTable(table: string): DriftDifference {
|
|
105
|
-
return {
|
|
106
|
-
kind: 'missing-table',
|
|
107
|
-
table,
|
|
108
|
-
column: null,
|
|
109
|
-
cause: `table "${table}" is declared by migrations but does not exist`,
|
|
110
|
-
fix: 'x db migrate',
|
|
111
|
-
};
|
|
112
|
-
}
|
|
113
|
-
|
|
114
|
-
/**
|
|
115
|
-
* Not a difference between two schemas but the absence of one to compare against — reported
|
|
116
|
-
* through the same channel so it reaches an operator, since a check that quietly answered "clean"
|
|
117
|
-
* because it had nothing to check is the one failure mode drift detection cannot have.
|
|
118
|
-
*/
|
|
119
|
-
function unknownSchema(migrations: readonly Migration[]): DriftDifference {
|
|
120
|
-
const newest = [...migrations].sort((a, b) => (a.id < b.id ? -1 : 1)).at(-1);
|
|
121
|
-
return {
|
|
122
|
-
kind: 'unknown-schema',
|
|
123
|
-
table: '',
|
|
124
|
-
column: null,
|
|
125
|
-
cause:
|
|
126
|
-
`migration "${newest?.id ?? ''}" records no schema snapshot, so what this database owes ` +
|
|
127
|
-
'cannot be established',
|
|
128
|
-
// The same two remedies `X_MIGRATION_SNAPSHOT_MISSING` names, in the same order, because it is
|
|
129
|
-
// the same condition. It used to lead with `x db gen`, which raises that error and whose own
|
|
130
|
-
// fix pointed back here — a cycle a scaffolded app hit on its first `x db migrate`. The
|
|
131
|
-
// pathspec is a glob because this package is tier 1: only `@ultimat3/cli` knows the directory.
|
|
132
|
-
fix:
|
|
133
|
-
`git checkout -- "*${newest?.id ?? ''}.snapshot.json" # or, if it was never written: ` +
|
|
134
|
-
`delete migration "${newest?.id ?? ''}" and rerun x db gen "${newest?.name ?? 'initial'}"`,
|
|
135
|
-
};
|
|
136
|
-
}
|
|
137
|
-
|
|
138
|
-
function missingIndex(table: string, index: string): DriftDifference {
|
|
139
|
-
return {
|
|
140
|
-
kind: 'missing-index',
|
|
141
|
-
table,
|
|
142
|
-
column: null,
|
|
143
|
-
cause: `table "${table}" is missing index "${index}" that migrations declare`,
|
|
144
|
-
fix: 'x db migrate',
|
|
145
|
-
};
|
|
146
|
-
}
|
|
147
|
-
|
|
148
|
-
function changedIndex(table: string, index: string, detail: string): DriftDifference {
|
|
149
|
-
return {
|
|
150
|
-
kind: 'changed-index',
|
|
151
|
-
table,
|
|
152
|
-
column: null,
|
|
153
|
-
cause: `index "${index}" on "${table}" ${detail}, not what migrations declare`,
|
|
154
|
-
fix: 'x db migrate',
|
|
155
|
-
};
|
|
156
|
-
}
|
|
157
|
-
|
|
158
|
-
function missingForeignKey(table: string, key: ForeignKeyDescription): DriftDifference {
|
|
159
|
-
return {
|
|
160
|
-
kind: 'missing-foreign-key',
|
|
161
|
-
table,
|
|
162
|
-
column: null,
|
|
163
|
-
cause:
|
|
164
|
-
`table "${table}" has no foreign key on (${key.columns.join(', ')}) to ` +
|
|
165
|
-
`"${key.referencedTable}" (${key.referencedColumns.join(', ')}) that migrations declare`,
|
|
166
|
-
fix: 'x db migrate',
|
|
167
|
-
};
|
|
168
|
-
}
|
|
169
|
-
|
|
170
|
-
/**
|
|
171
|
-
* The key points where it was declared to point and one side's `on delete` rule is not the other's
|
|
172
|
-
* — reported apart from `missing-foreign-key` because it is a different repair: the constraint is
|
|
173
|
-
* there, and what changed is what happens to the child rows.
|
|
174
|
-
*
|
|
175
|
-
* The `fix` is the pair, not `x db migrate`: a rule cannot be altered in place, `add constraint`
|
|
176
|
-
* alone is `42710` on a name already taken, and no `x db gen` diff emits either statement, so
|
|
177
|
-
* naming a command would send a reader to one that generates an empty migration. Same reasoning
|
|
178
|
-
* as `changedColumn`.
|
|
179
|
-
*/
|
|
180
|
-
function changedForeignKey(
|
|
181
|
-
table: string,
|
|
182
|
-
declared: ForeignKeyDescription,
|
|
183
|
-
held: ForeignKeyDescription,
|
|
184
|
-
): DriftDifference {
|
|
185
|
-
const rule = onDeleteRule(held.onDelete);
|
|
186
|
-
return {
|
|
187
|
-
kind: 'changed-foreign-key',
|
|
188
|
-
table,
|
|
189
|
-
column: null,
|
|
190
|
-
cause:
|
|
191
|
-
`foreign key on "${table}" (${declared.columns.join(', ')}) to ` +
|
|
192
|
-
`"${declared.referencedTable}" ` +
|
|
193
|
-
`${rule === null ? 'declares no on delete rule' : `is on delete ${rule}`}, not what ` +
|
|
194
|
-
'migrations declare',
|
|
195
|
-
fix: `${rebuildForeignKey(table, declared, held)} # in a new migration`,
|
|
196
|
-
};
|
|
197
|
-
}
|
|
198
|
-
|
|
199
36
|
/**
|
|
200
37
|
* Indexes migrations declare, against the ones the catalog holds — by column list and by
|
|
201
38
|
* uniqueness, which is what caught a composite index rebuilt with its columns the other way round
|
|
@@ -314,6 +151,33 @@ function compareForeignKeys(live: TableDescription, expected: TableDescription):
|
|
|
314
151
|
return differences;
|
|
315
152
|
}
|
|
316
153
|
|
|
154
|
+
/**
|
|
155
|
+
* CHECK constraints migrations declare, against the NAMES the catalog holds — `checkNames`, which
|
|
156
|
+
* is a separate field from `checks` precisely so this comparison cannot reach a definition it must
|
|
157
|
+
* not read (`introspect.ts`).
|
|
158
|
+
*
|
|
159
|
+
* Two absences, and they mean opposite things. `expected.checks` absent is a sidecar written
|
|
160
|
+
* before constraints were recorded: it declares nothing, so nothing can be missing. `live.checkNames`
|
|
161
|
+
* absent is a description that never asked the catalog — a stub, a fake client's rows, a
|
|
162
|
+
* `TableDescription` built by hand — and reading that as "the database holds none" is one finding
|
|
163
|
+
* per declared constraint against a database nobody looked at. `introspect()` always answers with
|
|
164
|
+
* the field, `[]` included, so a real read is never mistaken for an unread one.
|
|
165
|
+
*
|
|
166
|
+
* Only the declared side is judged, the rule `compareIndexes` and `compareForeignKeys` both state:
|
|
167
|
+
* a NOT NULL, an `enumerated()` column's old anonymous form, a constraint an extension brought and
|
|
168
|
+
* every hand-written CHECK an app has ever added would each be a finding against a database that
|
|
169
|
+
* is exactly right.
|
|
170
|
+
*/
|
|
171
|
+
function compareChecks(live: TableDescription, expected: TableDescription): DriftDifference[] {
|
|
172
|
+
const declared = expected.checks;
|
|
173
|
+
const held = live.checkNames;
|
|
174
|
+
if (declared === undefined || held === undefined) return [];
|
|
175
|
+
const present = new Set(held);
|
|
176
|
+
return declared
|
|
177
|
+
.filter((check) => !present.has(check.name))
|
|
178
|
+
.map((check) => missingCheck(live.name, check));
|
|
179
|
+
}
|
|
180
|
+
|
|
317
181
|
/**
|
|
318
182
|
* A primary key column is `NOT NULL` in the catalog whether or not anything declared it — Postgres
|
|
319
183
|
* adds the constraint with the key. Both sides are therefore read through the union of the two
|
|
@@ -350,6 +214,7 @@ function compareTable(live: TableDescription, expected: TableDescription): Drift
|
|
|
350
214
|
}
|
|
351
215
|
}
|
|
352
216
|
differences.push(...compareIndexes(live, expected));
|
|
217
|
+
differences.push(...compareChecks(live, expected));
|
|
353
218
|
differences.push(...compareForeignKeys(live, expected));
|
|
354
219
|
return differences;
|
|
355
220
|
}
|
package/src/generate.ts
CHANGED
|
@@ -3,29 +3,26 @@
|
|
|
3
3
|
// the CLI passes `describeEntities()` and the types below mirror `EntityDescription` field for
|
|
4
4
|
// field. Every generated migration must be reversible; a drop that loses data refuses instead.
|
|
5
5
|
|
|
6
|
-
import {
|
|
6
|
+
import { systemClock } from '@ultimat3/core';
|
|
7
7
|
import { checkClauses, checkPlan, declaredChecks } from './check-ddl';
|
|
8
8
|
import { defaultExpression } from './column-default';
|
|
9
9
|
import { isDestructive } from './destructive';
|
|
10
10
|
import { dropOrder } from './drop-order';
|
|
11
|
-
import type {
|
|
12
|
-
ColumnDescriptionLike,
|
|
13
|
-
EntityDescriptionLike,
|
|
14
|
-
IndexDescriptionLike,
|
|
15
|
-
} from './entity-shape';
|
|
11
|
+
import type { ColumnDescriptionLike, EntityDescriptionLike } from './entity-shape';
|
|
16
12
|
import { migrationIrreversible } from './errors';
|
|
17
13
|
import { type ConstraintPlans, foreignKeyPlan, foreignKeysOf, type Plan } from './foreign-key-plan';
|
|
18
14
|
import type { Regeneration } from './generated-column';
|
|
19
15
|
import { generatedClause, isGenerated, regenerate } from './generated-column';
|
|
20
|
-
import {
|
|
16
|
+
import { createIndex, dropIndex, impliedByColumnClause, redefineIndex } from './index-ddl';
|
|
21
17
|
import {
|
|
22
18
|
type ColumnDescription,
|
|
23
19
|
findTable,
|
|
24
|
-
type IndexDescription,
|
|
25
20
|
type SchemaDescription,
|
|
26
21
|
type TableDescription,
|
|
27
22
|
} from './introspect';
|
|
28
23
|
import { declaredIndexes } from './invariant-ddl';
|
|
24
|
+
import type { MovedAside } from './retype-dependents';
|
|
25
|
+
import { moveDependentsAside } from './retype-dependents';
|
|
29
26
|
import { identifier } from './sql';
|
|
30
27
|
import { type UnrenderedDeclaration, unrenderedComment, unrenderedOf } from './unrendered';
|
|
31
28
|
|
|
@@ -74,34 +71,6 @@ function columnClause(column: ColumnDescriptionLike): string {
|
|
|
74
71
|
return parts.join(' ');
|
|
75
72
|
}
|
|
76
73
|
|
|
77
|
-
/**
|
|
78
|
-
* A `unique` column clause already creates an index, and Postgres names it exactly what the
|
|
79
|
-
* entity's own convention names it — `<table>_<column>_key`. Emitting `create unique index` for
|
|
80
|
-
* it too is the same index twice: `42P07`, and a migration that cannot be applied at all.
|
|
81
|
-
* Mirrors the rule `entity()` already applies to a foreign key indexing its own column.
|
|
82
|
-
*
|
|
83
|
-
* A **partial** unique index is not that index: the column clause constrains every row, so
|
|
84
|
-
* skipping the partial one would silently widen the constraint the entity declared.
|
|
85
|
-
*/
|
|
86
|
-
function impliedByColumnClause(
|
|
87
|
-
entity: EntityDescriptionLike,
|
|
88
|
-
index: IndexDescriptionLike,
|
|
89
|
-
added: ReadonlySet<string>,
|
|
90
|
-
): boolean {
|
|
91
|
-
const [only] = index.columns;
|
|
92
|
-
if (!index.unique || index.where !== null || index.columns.length !== 1 || only === undefined) {
|
|
93
|
-
return false;
|
|
94
|
-
}
|
|
95
|
-
const column = entity.columns.find((each) => each.column === only);
|
|
96
|
-
// `columnClause` writes `unique` under exactly this condition — keep the two in step.
|
|
97
|
-
//
|
|
98
|
-
// NOT an optional chain, despite what biome's useOptionalChain suggests: `column?.unique` is
|
|
99
|
-
// `boolean | undefined`, and this function returns `boolean`. The lint rule marks its own fix
|
|
100
|
-
// unsafe for exactly this reason — applying it turned a green typecheck red.
|
|
101
|
-
// biome-ignore lint/complexity/useOptionalChain: an optional chain widens the return to include undefined
|
|
102
|
-
return column !== undefined && column.unique && !column.primaryKey && added.has(only);
|
|
103
|
-
}
|
|
104
|
-
|
|
105
74
|
export function snapshotOf(entities: readonly EntityDescriptionLike[]): SchemaDescription {
|
|
106
75
|
const tables = [...entities]
|
|
107
76
|
.sort((a, b) => (a.table < b.table ? -1 : 1))
|
|
@@ -172,68 +141,35 @@ function createTable(entity: EntityDescriptionLike): readonly string[] {
|
|
|
172
141
|
return statements;
|
|
173
142
|
}
|
|
174
143
|
|
|
175
|
-
/**
|
|
176
|
-
* Every part of the declaration reaches the statement: the whole column list in its declared
|
|
177
|
-
* order, the direction when one was asked for, and the predicate that makes it partial. A part
|
|
178
|
-
* dropped here is a constraint the database does not hold or an index the planner cannot use.
|
|
179
|
-
*/
|
|
180
|
-
function createIndex(table: string, index: IndexDescriptionLike): string {
|
|
181
|
-
assert(
|
|
182
|
-
index.columns.length > 0,
|
|
183
|
-
`index "${index.name}" on "${table}" names no columns`,
|
|
184
|
-
`indexes: [{ on: ['<column>'] }] # name the columns in the entity(), then x db gen`,
|
|
185
|
-
);
|
|
186
|
-
const method = index.using ?? 'btree';
|
|
187
|
-
// Two rules Postgres has and a declaration can break, refused here rather than at migrate time:
|
|
188
|
-
// GIN supports neither a unique index nor an ASC/DESC option, and either one reaches the server
|
|
189
|
-
// as a syntax error inside `ROLE=migrate` — a release phase that fails with the server's words
|
|
190
|
-
// and none of the entity's. `X_INVARIANT` for the reason `createIndex` already uses it on an
|
|
191
|
-
// index naming no columns: a declaration this build cannot honour is refused, never reinterpreted.
|
|
192
|
-
assert(
|
|
193
|
-
method === 'btree' || !index.unique,
|
|
194
|
-
`index "${index.name}" on "${table}" is unique and ${method}; Postgres has no unique ${method} index`,
|
|
195
|
-
`indexes: [{ on: ['<column>'], using: '${method}' }] # drop unique, or drop using`,
|
|
196
|
-
);
|
|
197
|
-
assert(
|
|
198
|
-
method === 'btree' || index.order === null,
|
|
199
|
-
`index "${index.name}" on "${table}" is ${method} and ${index.order}; only a btree orders its keys`,
|
|
200
|
-
`indexes: [{ on: ['<column>'], using: '${method}' }] # drop order, or drop using`,
|
|
201
|
-
);
|
|
202
|
-
const kind = index.unique ? 'create unique index' : 'create index';
|
|
203
|
-
const direction = index.order === null ? '' : ` ${index.order}`;
|
|
204
|
-
const columns = index.columns
|
|
205
|
-
.map((column) => `${identifier(column).text}${direction}`)
|
|
206
|
-
.join(', ');
|
|
207
|
-
const predicate = index.where === null ? '' : ` where (${index.where})`;
|
|
208
|
-
// Re-derived from the closed set, never spliced: `indexMethodSql` answers `''` for a btree, so
|
|
209
|
-
// an index that declared no method emits the statement this generator always emitted, byte for
|
|
210
|
-
// byte, and one that declared a method Postgres does not have is refused instead of built.
|
|
211
|
-
return (
|
|
212
|
-
`${kind} ${identifier(index.name).text} on ${identifier(table).text}` +
|
|
213
|
-
`${indexMethodSql(method)} (${columns})${predicate};`
|
|
214
|
-
);
|
|
215
|
-
}
|
|
216
|
-
|
|
217
144
|
/**
|
|
218
145
|
* Skipping an existing column by name alone missed the type moving under it: a table created
|
|
219
146
|
* while money's currency was bare `char` keeps `char(1)` and rejects every ISO 4217 code, yet the
|
|
220
147
|
* snapshot this run records claims `char(3)` — two claims with no statement between them. Both
|
|
221
148
|
* sides are generated spellings (`current` is a previous migration's own snapshot), so any
|
|
222
149
|
* difference is a real kind change, not a catalog alias.
|
|
150
|
+
*
|
|
151
|
+
* The ALTER is not the whole statement list: Postgres compiled every predicate written against
|
|
152
|
+
* this column with its OLD type and cannot recompile one, so a partial index or a CHECK that reads
|
|
153
|
+
* it is dropped FIRST and `moved` carries the names on to the arms that would otherwise act on
|
|
154
|
+
* them. Without that the retype is `42883` and the migration aborts mid-run
|
|
155
|
+
* (`retype-dependents.ts`).
|
|
223
156
|
*/
|
|
224
157
|
function retypeColumn(
|
|
225
|
-
|
|
158
|
+
live: TableDescription,
|
|
226
159
|
column: ColumnDescriptionLike,
|
|
227
160
|
recorded: ColumnDescription,
|
|
228
161
|
plan: Plan,
|
|
162
|
+
moved: MovedAside,
|
|
229
163
|
): Regeneration {
|
|
230
164
|
const wanted = sqlType(column.kind);
|
|
165
|
+
const table = live.name;
|
|
231
166
|
// A generated column moves by its own rules — see `generated-column.ts`. Asked whenever EITHER
|
|
232
167
|
// side is one, because becoming generated and ceasing to be are both changes with a statement.
|
|
233
168
|
if (isGenerated(column) || recorded.generated !== undefined) {
|
|
234
169
|
return regenerate(table, column, wanted, recorded, plan);
|
|
235
170
|
}
|
|
236
171
|
if (recorded.dataType === wanted) return 'unchanged';
|
|
172
|
+
moveDependentsAside(live, column.column, plan, moved);
|
|
237
173
|
const alter = (type: string): string =>
|
|
238
174
|
`alter table ${identifier(table).text} alter column ${identifier(column.column).text} ` +
|
|
239
175
|
`type ${type} using ${identifier(column.column).text}::${type};`;
|
|
@@ -242,64 +178,18 @@ function retypeColumn(
|
|
|
242
178
|
return 'altered';
|
|
243
179
|
}
|
|
244
180
|
|
|
245
|
-
/** The parts of an index Postgres cannot alter in place — every one of them is a rebuild. */
|
|
246
|
-
function indexShape(index: IndexDescriptionLike | IndexDescription): string {
|
|
247
|
-
return JSON.stringify([
|
|
248
|
-
[...index.columns],
|
|
249
|
-
index.unique,
|
|
250
|
-
index.where,
|
|
251
|
-
index.order ?? null,
|
|
252
|
-
indexMethodOf(index),
|
|
253
|
-
]);
|
|
254
|
-
}
|
|
255
|
-
|
|
256
|
-
/**
|
|
257
|
-
* A same-named index whose definition moved is dropped and recreated, because Postgres has no
|
|
258
|
-
* `alter index` for any of it — the column list, the uniqueness, the predicate and the direction
|
|
259
|
-
* are all fixed at creation.
|
|
260
|
-
*
|
|
261
|
-
* Matching on the name alone was the gap: `where` and `order` were not even recorded, so an
|
|
262
|
-
* entity narrowing an index to a predicate, or reversing it to `desc`, generated an empty
|
|
263
|
-
* migration and the database kept serving the old one. Both sides here are *generated* spellings
|
|
264
|
-
* — `recorded` is a previous migration's own snapshot, never the catalog's rewriting of it — so a
|
|
265
|
-
* text difference in `where` is a real change and not a formatting one.
|
|
266
|
-
*/
|
|
267
|
-
function redefineIndex(
|
|
268
|
-
table: string,
|
|
269
|
-
index: IndexDescriptionLike,
|
|
270
|
-
recorded: IndexDescription,
|
|
271
|
-
plan: Plan,
|
|
272
|
-
): void {
|
|
273
|
-
if (indexShape(index) === indexShape(recorded)) return;
|
|
274
|
-
plan.up.push(`drop index ${identifier(index.name).text};`, createIndex(table, index));
|
|
275
|
-
// `down` is reversed at assembly, so the pair is pushed forwards and read backwards: recreating
|
|
276
|
-
// the recorded definition is what must land last, after the new one is dropped.
|
|
277
|
-
plan.down.push(
|
|
278
|
-
createIndex(table, {
|
|
279
|
-
name: recorded.name,
|
|
280
|
-
columns: recorded.columns,
|
|
281
|
-
unique: recorded.unique,
|
|
282
|
-
where: recorded.where,
|
|
283
|
-
order: recorded.order,
|
|
284
|
-
// `declaredMethod`, never a cast: `recorded` is a snapshot's, typed open because the catalog
|
|
285
|
-
// shares the shape, and a method this generator cannot emit must refuse rather than be
|
|
286
|
-
// rebuilt as a btree — a `down` that recreates the wrong structure is worse than none.
|
|
287
|
-
...(recorded.using === undefined ? {} : { using: declaredMethod(recorded.using) }),
|
|
288
|
-
}),
|
|
289
|
-
`drop index ${identifier(index.name).text};`,
|
|
290
|
-
);
|
|
291
|
-
}
|
|
292
|
-
|
|
293
181
|
function diffTable(entity: EntityDescriptionLike, live: TableDescription, plan: Plan): void {
|
|
294
182
|
const existing = new Map(live.columns.map((column) => [column.name, column]));
|
|
295
183
|
const added = new Set<string>();
|
|
296
184
|
// A column `regenerate` had to replace outright: `add column` implies no index, so every index
|
|
297
185
|
// over it has to be stated again even though its own definition never moved.
|
|
298
186
|
const rebuilt = new Set<string>();
|
|
187
|
+
// What a retype dropped ahead of itself, read by the two arms below.
|
|
188
|
+
const moved: MovedAside = { indexes: new Set(), checks: new Set() };
|
|
299
189
|
for (const column of entity.columns) {
|
|
300
190
|
const recorded = existing.get(column.column);
|
|
301
191
|
if (recorded !== undefined) {
|
|
302
|
-
if (retypeColumn(
|
|
192
|
+
if (retypeColumn(live, column, recorded, plan, moved) === 'rebuilt') {
|
|
303
193
|
rebuilt.add(column.column);
|
|
304
194
|
}
|
|
305
195
|
continue;
|
|
@@ -329,9 +219,11 @@ function diffTable(entity: EntityDescriptionLike, live: TableDescription, plan:
|
|
|
329
219
|
const indexed = new Map(live.indexes.map((index) => [index.name, index]));
|
|
330
220
|
for (const index of declaredIndexes(entity)) {
|
|
331
221
|
const recorded = indexed.get(index.name);
|
|
332
|
-
// A rebuilt column took its indexes down with it,
|
|
222
|
+
// A rebuilt column took its indexes down with it, and a retype dropped the ones whose
|
|
223
|
+
// predicate it could not survive — either way this one is CREATED rather than compared:
|
|
333
224
|
// `redefineIndex` sees a definition that never moved and would emit nothing at all.
|
|
334
|
-
|
|
225
|
+
const gone = moved.indexes.has(index.name) || index.columns.some((each) => rebuilt.has(each));
|
|
226
|
+
if (recorded !== undefined && !gone) {
|
|
335
227
|
redefineIndex(entity.table, index, recorded, plan);
|
|
336
228
|
continue;
|
|
337
229
|
}
|
|
@@ -339,13 +231,14 @@ function diffTable(entity: EntityDescriptionLike, live: TableDescription, plan:
|
|
|
339
231
|
// migration emits, so it still needs a statement of its own.
|
|
340
232
|
if (impliedByColumnClause(entity, index, added)) continue;
|
|
341
233
|
plan.up.push(createIndex(entity.table, index));
|
|
342
|
-
plan.down.push(
|
|
234
|
+
plan.down.push(dropIndex(index.name));
|
|
343
235
|
}
|
|
344
236
|
|
|
345
237
|
// Last: a CHECK may read a column this migration just added, and `add constraint` on a column
|
|
346
238
|
// that does not exist yet is `42703`. `check-ddl.ts` owns which of them move; `rebuilt` because a
|
|
347
|
-
// column dropped and re-added lost its constraint while the snapshot still records it
|
|
348
|
-
|
|
239
|
+
// column dropped and re-added lost its constraint while the snapshot still records it, and
|
|
240
|
+
// `moved.checks` because a retype already dropped the ones written against the old type.
|
|
241
|
+
checkPlan(entity, live, plan, rebuilt, moved.checks);
|
|
349
242
|
}
|
|
350
243
|
|
|
351
244
|
export interface GenerateOptions {
|
package/src/index-ddl.ts
ADDED
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
// Single responsibility: the DDL an entity's INDEX declaration becomes — the `create index` a
|
|
2
|
+
// declaration writes out, what makes two definitions the same index, and what a moved definition
|
|
3
|
+
// rebuilds. Split out of `generate.ts` at the 500-line ceiling, along the seam `check-ddl.ts` and
|
|
4
|
+
// `generated-column.ts` already drew: `generate.ts` assembles a plan, this file writes the index
|
|
5
|
+
// statements it puts in it, and `invariant-ddl.ts` decides which indexes a table declares.
|
|
6
|
+
|
|
7
|
+
import { assert } from '@ultimat3/core';
|
|
8
|
+
import type { EntityDescriptionLike, IndexDescriptionLike } from './entity-shape';
|
|
9
|
+
import type { Plan } from './foreign-key-plan';
|
|
10
|
+
import { declaredMethod, indexMethodOf, indexMethodSql } from './index-method';
|
|
11
|
+
import type { IndexDescription } from './introspect';
|
|
12
|
+
import { identifier } from './sql';
|
|
13
|
+
|
|
14
|
+
/**
|
|
15
|
+
* A `unique` column clause already creates an index, and Postgres names it exactly what the
|
|
16
|
+
* entity's own convention names it — `<table>_<column>_key`. Emitting `create unique index` for
|
|
17
|
+
* it too is the same index twice: `42P07`, and a migration that cannot be applied at all.
|
|
18
|
+
* Mirrors the rule `entity()` already applies to a foreign key indexing its own column.
|
|
19
|
+
*
|
|
20
|
+
* A **partial** unique index is not that index: the column clause constrains every row, so
|
|
21
|
+
* skipping the partial one would silently widen the constraint the entity declared.
|
|
22
|
+
*/
|
|
23
|
+
export function impliedByColumnClause(
|
|
24
|
+
entity: EntityDescriptionLike,
|
|
25
|
+
index: IndexDescriptionLike,
|
|
26
|
+
added: ReadonlySet<string>,
|
|
27
|
+
): boolean {
|
|
28
|
+
const [only] = index.columns;
|
|
29
|
+
if (!index.unique || index.where !== null || index.columns.length !== 1 || only === undefined) {
|
|
30
|
+
return false;
|
|
31
|
+
}
|
|
32
|
+
const column = entity.columns.find((each) => each.column === only);
|
|
33
|
+
// `columnClause` writes `unique` under exactly this condition — keep the two in step.
|
|
34
|
+
//
|
|
35
|
+
// NOT an optional chain, despite what biome's useOptionalChain suggests: `column?.unique` is
|
|
36
|
+
// `boolean | undefined`, and this function returns `boolean`. The lint rule marks its own fix
|
|
37
|
+
// unsafe for exactly this reason — applying it turned a green typecheck red.
|
|
38
|
+
// biome-ignore lint/complexity/useOptionalChain: an optional chain widens the return to include undefined
|
|
39
|
+
return column !== undefined && column.unique && !column.primaryKey && added.has(only);
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
/**
|
|
43
|
+
* Every part of the declaration reaches the statement: the whole column list in its declared
|
|
44
|
+
* order, the direction when one was asked for, and the predicate that makes it partial. A part
|
|
45
|
+
* dropped here is a constraint the database does not hold or an index the planner cannot use.
|
|
46
|
+
*/
|
|
47
|
+
export function createIndex(table: string, index: IndexDescriptionLike): string {
|
|
48
|
+
assert(
|
|
49
|
+
index.columns.length > 0,
|
|
50
|
+
`index "${index.name}" on "${table}" names no columns`,
|
|
51
|
+
`indexes: [{ on: ['<column>'] }] # name the columns in the entity(), then x db gen`,
|
|
52
|
+
);
|
|
53
|
+
const method = index.using ?? 'btree';
|
|
54
|
+
// Two rules Postgres has and a declaration can break, refused here rather than at migrate time:
|
|
55
|
+
// GIN supports neither a unique index nor an ASC/DESC option, and either one reaches the server
|
|
56
|
+
// as a syntax error inside `ROLE=migrate` — a release phase that fails with the server's words
|
|
57
|
+
// and none of the entity's. `X_INVARIANT` for the reason `createIndex` already uses it on an
|
|
58
|
+
// index naming no columns: a declaration this build cannot honour is refused, never reinterpreted.
|
|
59
|
+
assert(
|
|
60
|
+
method === 'btree' || !index.unique,
|
|
61
|
+
`index "${index.name}" on "${table}" is unique and ${method}; Postgres has no unique ${method} index`,
|
|
62
|
+
`indexes: [{ on: ['<column>'], using: '${method}' }] # drop unique, or drop using`,
|
|
63
|
+
);
|
|
64
|
+
assert(
|
|
65
|
+
method === 'btree' || index.order === null,
|
|
66
|
+
`index "${index.name}" on "${table}" is ${method} and ${index.order}; only a btree orders its keys`,
|
|
67
|
+
`indexes: [{ on: ['<column>'], using: '${method}' }] # drop order, or drop using`,
|
|
68
|
+
);
|
|
69
|
+
const kind = index.unique ? 'create unique index' : 'create index';
|
|
70
|
+
const direction = index.order === null ? '' : ` ${index.order}`;
|
|
71
|
+
const columns = index.columns
|
|
72
|
+
.map((column) => `${identifier(column).text}${direction}`)
|
|
73
|
+
.join(', ');
|
|
74
|
+
const predicate = index.where === null ? '' : ` where (${index.where})`;
|
|
75
|
+
// Re-derived from the closed set, never spliced: `indexMethodSql` answers `''` for a btree, so
|
|
76
|
+
// an index that declared no method emits the statement this generator always emitted, byte for
|
|
77
|
+
// byte, and one that declared a method Postgres does not have is refused instead of built.
|
|
78
|
+
return (
|
|
79
|
+
`${kind} ${identifier(index.name).text} on ${identifier(table).text}` +
|
|
80
|
+
`${indexMethodSql(method)} (${columns})${predicate};`
|
|
81
|
+
);
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
/** `drop index "n";` — the one spelling, so a drop and its recreate cannot name it differently. */
|
|
85
|
+
export const dropIndex = (name: string): string => `drop index ${identifier(name).text};`;
|
|
86
|
+
|
|
87
|
+
/**
|
|
88
|
+
* A RECORDED index as a declaration this generator can emit again — `declaredMethod`, never a
|
|
89
|
+
* cast: the recorded side is typed open because the catalog shares the shape, and a method this
|
|
90
|
+
* generator cannot write must refuse rather than be rebuilt as a btree. One copy, because
|
|
91
|
+
* `redefineIndex`'s `down` and `retype-dependents.ts`'s restore ask the same question.
|
|
92
|
+
*/
|
|
93
|
+
export function asDeclared(index: IndexDescription): IndexDescriptionLike {
|
|
94
|
+
return {
|
|
95
|
+
name: index.name,
|
|
96
|
+
columns: index.columns,
|
|
97
|
+
unique: index.unique,
|
|
98
|
+
where: index.where,
|
|
99
|
+
order: index.order,
|
|
100
|
+
...(index.using === undefined ? {} : { using: declaredMethod(index.using) }),
|
|
101
|
+
};
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
/** The parts of an index Postgres cannot alter in place — every one of them is a rebuild. */
|
|
105
|
+
export function indexShape(index: IndexDescriptionLike | IndexDescription): string {
|
|
106
|
+
return JSON.stringify([
|
|
107
|
+
[...index.columns],
|
|
108
|
+
index.unique,
|
|
109
|
+
index.where,
|
|
110
|
+
index.order ?? null,
|
|
111
|
+
indexMethodOf(index),
|
|
112
|
+
]);
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
/**
|
|
116
|
+
* A same-named index whose definition moved is dropped and recreated, because Postgres has no
|
|
117
|
+
* `alter index` for any of it — the column list, the uniqueness, the predicate and the direction
|
|
118
|
+
* are all fixed at creation.
|
|
119
|
+
*
|
|
120
|
+
* Matching on the name alone was the gap: `where` and `order` were not even recorded, so an
|
|
121
|
+
* entity narrowing an index to a predicate, or reversing it to `desc`, generated an empty
|
|
122
|
+
* migration and the database kept serving the old one. Both sides here are *generated* spellings
|
|
123
|
+
* — `recorded` is a previous migration's own snapshot, never the catalog's rewriting of it — so a
|
|
124
|
+
* text difference in `where` is a real change and not a formatting one.
|
|
125
|
+
*/
|
|
126
|
+
export function redefineIndex(
|
|
127
|
+
table: string,
|
|
128
|
+
index: IndexDescriptionLike,
|
|
129
|
+
recorded: IndexDescription,
|
|
130
|
+
plan: Plan,
|
|
131
|
+
): void {
|
|
132
|
+
if (indexShape(index) === indexShape(recorded)) return;
|
|
133
|
+
plan.up.push(dropIndex(index.name), createIndex(table, index));
|
|
134
|
+
// `down` is reversed at assembly, so the pair is pushed forwards and read backwards: recreating
|
|
135
|
+
// the recorded definition is what must land last, after the new one is dropped.
|
|
136
|
+
plan.down.push(createIndex(table, asDeclared(recorded)), dropIndex(index.name));
|
|
137
|
+
}
|
package/src/introspect.ts
CHANGED
|
@@ -87,6 +87,24 @@ export interface TableDescription {
|
|
|
87
87
|
* would leave every already-generated app's invariants unenforced forever.
|
|
88
88
|
*/
|
|
89
89
|
readonly checks?: readonly CheckDescription[] | undefined;
|
|
90
|
+
/**
|
|
91
|
+
* The catalog's half of `checks`, and a **separate field rather than the same one** — names
|
|
92
|
+
* only, `conname` for `contype = 'c'`, never a definition.
|
|
93
|
+
*
|
|
94
|
+
* The two readings cannot share `checks` because they are not the same value. A catalog read
|
|
95
|
+
* carries Postgres' own rewriting of the predicate and a declaration carries this generator's
|
|
96
|
+
* spelling, so a `checks` filled from `pg_constraint` would put a rewritten expression on the
|
|
97
|
+
* field `checkPlan` diffs against a generated one — every regenerated migration would then drop
|
|
98
|
+
* and re-add every constraint in the app, forever, because the two strings can never be equal.
|
|
99
|
+
* Splitting them means the type says which reading a value came from, and `checkPlan` cannot be
|
|
100
|
+
* handed a catalog value by accident.
|
|
101
|
+
*
|
|
102
|
+
* `snapshotOf` never writes it and `parseSnapshot` never reads it, so a sidecar carries `checks`
|
|
103
|
+
* alone. `introspect()` always answers with it, `[]` included: absent therefore means "nobody
|
|
104
|
+
* asked the catalog", which is what keeps `compareChecks` silent on a description that never
|
|
105
|
+
* read one instead of reporting every declared constraint as missing.
|
|
106
|
+
*/
|
|
107
|
+
readonly checkNames?: readonly string[] | undefined;
|
|
90
108
|
}
|
|
91
109
|
|
|
92
110
|
export interface SchemaDescription {
|
|
@@ -135,6 +153,12 @@ interface ForeignKeyRow {
|
|
|
135
153
|
readonly on_delete: string | null;
|
|
136
154
|
}
|
|
137
155
|
|
|
156
|
+
/** A CHECK constraint's NAME. There is deliberately no column for its definition — see `checkNames`. */
|
|
157
|
+
interface CheckRow {
|
|
158
|
+
readonly table_name: string;
|
|
159
|
+
readonly constraint_name: string;
|
|
160
|
+
}
|
|
161
|
+
|
|
138
162
|
const byName = (a: { name: string }, b: { name: string }): number => (a.name < b.name ? -1 : 1);
|
|
139
163
|
|
|
140
164
|
export async function introspect(options: IntrospectOptions = {}): Promise<SchemaDescription> {
|
|
@@ -206,7 +230,21 @@ export async function introspect(options: IntrospectOptions = {}): Promise<Schem
|
|
|
206
230
|
order by src.relname, c.conname
|
|
207
231
|
`);
|
|
208
232
|
|
|
209
|
-
|
|
233
|
+
// `conname` and nothing else. `pg_get_constraintdef(c.oid)` is one word further along this line
|
|
234
|
+
// and is the reason this query did not exist: it answers Postgres' rewriting of the predicate,
|
|
235
|
+
// which no generated spelling can ever equal, so reading it would make every drift check report
|
|
236
|
+
// a correct database as wrong. `contype = 'c'` is the CHECK constraints alone — Postgres 17
|
|
237
|
+
// onwards records a NOT NULL as `'n'`, and a domain's as `'c'` on the domain rather than here.
|
|
238
|
+
const checks = await client.query<CheckRow>(sql`
|
|
239
|
+
select src.relname as table_name, c.conname as constraint_name
|
|
240
|
+
from pg_constraint c
|
|
241
|
+
join pg_class src on src.oid = c.conrelid
|
|
242
|
+
join pg_namespace n on n.oid = src.relnamespace
|
|
243
|
+
where c.contype = 'c' and n.nspname = ${schema} and src.relkind = 'r'
|
|
244
|
+
order by src.relname, c.conname
|
|
245
|
+
`);
|
|
246
|
+
|
|
247
|
+
return buildSchema(schema, excluded, columns, indexes, foreignKeys, checks);
|
|
210
248
|
}
|
|
211
249
|
|
|
212
250
|
/** Pure, so the row -> description mapping is testable without a database. */
|
|
@@ -216,6 +254,7 @@ export function buildSchema(
|
|
|
216
254
|
columns: readonly ColumnRow[],
|
|
217
255
|
indexes: readonly IndexRow[],
|
|
218
256
|
foreignKeys: readonly ForeignKeyRow[],
|
|
257
|
+
checks: readonly CheckRow[] = [],
|
|
219
258
|
): SchemaDescription {
|
|
220
259
|
const names = [...new Set(columns.map((row) => row.table_name))]
|
|
221
260
|
.filter((name) => !excluded.includes(name))
|
|
@@ -261,6 +300,12 @@ export function buildSchema(
|
|
|
261
300
|
onDelete: row.on_delete,
|
|
262
301
|
}))
|
|
263
302
|
.sort(byName),
|
|
303
|
+
// Always written, `[]` included: this reading of a table HAS asked the catalog, and absence
|
|
304
|
+
// is reserved for a description that has not (`compareChecks` is silent on that one).
|
|
305
|
+
checkNames: checks
|
|
306
|
+
.filter((row) => row.table_name === name)
|
|
307
|
+
.map((row) => row.constraint_name)
|
|
308
|
+
.sort(),
|
|
264
309
|
};
|
|
265
310
|
});
|
|
266
311
|
|
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
// Single responsibility: which RECORDED objects a retype of one column breaks, and the statements
|
|
2
|
+
// that take them out of the way before the ALTER and put them back in the `down`.
|
|
3
|
+
//
|
|
4
|
+
// Postgres compiles a partial index's predicate and a CHECK's expression against the column's type
|
|
5
|
+
// at creation and cannot recompile either: `alter table "posts" alter column "status" type text
|
|
6
|
+
// using "status"::text` answers `42883 operator does not exist: text = post_status` and the
|
|
7
|
+
// migration aborts mid-run — inside `ROLE=migrate`, with the ledger recording nothing. Measured on
|
|
8
|
+
// Postgres 18.4 (`generate-retype.live.test.ts`), one dependent shape at a time:
|
|
9
|
+
//
|
|
10
|
+
// | recorded object | survives the ALTER |
|
|
11
|
+
// |------------------------------------------|--------------------|
|
|
12
|
+
// | btree on the column, plain or unique | yes — Postgres rebuilds it itself |
|
|
13
|
+
// | composite btree including the column | yes |
|
|
14
|
+
// | partial index whose predicate names it | **no — 42883** |
|
|
15
|
+
// | partial index naming another column | yes |
|
|
16
|
+
// | CHECK whose expression names it | **no — 42883** |
|
|
17
|
+
//
|
|
18
|
+
// So only an expression that MENTIONS the column is dependent, and dropping the rest would be a
|
|
19
|
+
// table scan per index for nothing.
|
|
20
|
+
|
|
21
|
+
import { addCheck, dropCheck } from './check-ddl';
|
|
22
|
+
import type { Plan } from './foreign-key-plan';
|
|
23
|
+
import { asDeclared, createIndex, dropIndex } from './index-ddl';
|
|
24
|
+
import type { CheckDescription, IndexDescription, TableDescription } from './introspect';
|
|
25
|
+
import { IDENTIFIER_PART, noiseAt } from './sql-scan';
|
|
26
|
+
|
|
27
|
+
/**
|
|
28
|
+
* Whether `expression` reads `column`, over-approximating on purpose.
|
|
29
|
+
*
|
|
30
|
+
* The two errors are not symmetrical. A dependent object missed is `42883` in the release phase;
|
|
31
|
+
* one reported that was not is a rebuild nobody asked for — so every ambiguous case answers `true`,
|
|
32
|
+
* and the folding is case-insensitive because Postgres folds an unquoted identifier to lower case
|
|
33
|
+
* and `"Status"` naming a different column is a rarity beside a predicate this must not miss.
|
|
34
|
+
*
|
|
35
|
+
* What it does NOT count is noise, through this package's one lexer (`sql-scan.ts`): the `status`
|
|
36
|
+
* in `where kind = 'status'` is data, not a reference, and the one in `-- status` is prose. A
|
|
37
|
+
* QUOTED identifier is counted — `"status"` is the reference the catalog stores for an author who
|
|
38
|
+
* quoted it, and skipping it as noise is exactly the miss that ends in `42883`.
|
|
39
|
+
*/
|
|
40
|
+
export function referencesColumn(expression: string, column: string): boolean {
|
|
41
|
+
const wanted = column.toLowerCase();
|
|
42
|
+
let at = 0;
|
|
43
|
+
while (at < expression.length) {
|
|
44
|
+
const noise = noiseAt(expression, at);
|
|
45
|
+
if (noise !== null) {
|
|
46
|
+
if (
|
|
47
|
+
noise.kind === 'identifier' &&
|
|
48
|
+
expression.slice(at + 1, noise.end - 1).toLowerCase() === wanted
|
|
49
|
+
) {
|
|
50
|
+
return true;
|
|
51
|
+
}
|
|
52
|
+
at = noise.end;
|
|
53
|
+
continue;
|
|
54
|
+
}
|
|
55
|
+
if (!IDENTIFIER_PART.test(expression[at] ?? '')) {
|
|
56
|
+
at += 1;
|
|
57
|
+
continue;
|
|
58
|
+
}
|
|
59
|
+
let end = at;
|
|
60
|
+
while (end < expression.length && IDENTIFIER_PART.test(expression[end] ?? '')) end += 1;
|
|
61
|
+
if (expression.slice(at, end).toLowerCase() === wanted) return true;
|
|
62
|
+
at = end;
|
|
63
|
+
}
|
|
64
|
+
return false;
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
/** The recorded objects a retype of `column` cannot leave in place. */
|
|
68
|
+
export interface RetypeDependents {
|
|
69
|
+
/**
|
|
70
|
+
* Partial indexes whose predicate reads the column. A `primary` one is structurally impossible —
|
|
71
|
+
* a primary key index has no predicate — which is what keeps `drop index` off the two indexes
|
|
72
|
+
* Postgres refuses it on: a primary key's and a unique constraint's.
|
|
73
|
+
*/
|
|
74
|
+
readonly indexes: readonly IndexDescription[];
|
|
75
|
+
/** CHECK constraints whose expression reads the column. */
|
|
76
|
+
readonly checks: readonly CheckDescription[];
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
/**
|
|
80
|
+
* What a retype of `table`.`column` breaks, read off the RECORDED schema — never the catalog.
|
|
81
|
+
* `x db gen` runs with no database open, so a hand-added expression index over the same column is
|
|
82
|
+
* invisible here and still `42883`; what this can see is every object a migration wrote down.
|
|
83
|
+
*/
|
|
84
|
+
export function retypeDependents(column: string, live: TableDescription): RetypeDependents {
|
|
85
|
+
return {
|
|
86
|
+
indexes: live.indexes.filter(
|
|
87
|
+
(index) => !index.primary && index.where !== null && referencesColumn(index.where, column),
|
|
88
|
+
),
|
|
89
|
+
checks: (live.checks ?? []).filter((check) => referencesColumn(check.expression, column)),
|
|
90
|
+
};
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
/**
|
|
94
|
+
* What this plan has already dropped ahead of a retype — names only, because that is all the two
|
|
95
|
+
* readers need. The ordinary diff runs AFTER the ALTER and must not act on an object that is no
|
|
96
|
+
* longer there: the index loop CREATES a name in `indexes` instead of comparing it (a `drop index`
|
|
97
|
+
* on a name already dropped is `42704`, and a definition that never moved would emit nothing at
|
|
98
|
+
* all, leaving the table with no index), and `checkPlan` neither drops nor re-adds a name in
|
|
99
|
+
* `checks` — the declared side is added back by its own arm, and a recorded constraint the entity
|
|
100
|
+
* no longer declares is simply gone, which is what `checkPlan` would have done to it anyway.
|
|
101
|
+
*/
|
|
102
|
+
export interface MovedAside {
|
|
103
|
+
readonly indexes: Set<string>;
|
|
104
|
+
readonly checks: Set<string>;
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
/**
|
|
108
|
+
* Drop every dependent in `up`, restore it in `down`, and record what was moved.
|
|
109
|
+
*
|
|
110
|
+
* `down` is reversed at assembly, so the restores are pushed FORWARDS here and the retype's own
|
|
111
|
+
* reversal is pushed after them — the reversed script therefore reads: retype back to the old
|
|
112
|
+
* type, then recreate the objects that were compiled against it. Restoring first would recreate a
|
|
113
|
+
* predicate against a type the column no longer has, which is `42883` in the other direction.
|
|
114
|
+
*
|
|
115
|
+
* What is restored is what the snapshot RECORDED, never what the entity declares: an object still
|
|
116
|
+
* declared is re-created by the ordinary diff, one statement later, in its current shape.
|
|
117
|
+
*/
|
|
118
|
+
export function moveDependentsAside(
|
|
119
|
+
live: TableDescription,
|
|
120
|
+
column: string,
|
|
121
|
+
plan: Plan,
|
|
122
|
+
moved: MovedAside,
|
|
123
|
+
): void {
|
|
124
|
+
const dependents = retypeDependents(column, live);
|
|
125
|
+
for (const index of dependents.indexes) {
|
|
126
|
+
plan.up.push(dropIndex(index.name));
|
|
127
|
+
plan.down.push(createIndex(live.name, asDeclared(index)));
|
|
128
|
+
moved.indexes.add(index.name);
|
|
129
|
+
}
|
|
130
|
+
for (const check of dependents.checks) {
|
|
131
|
+
plan.up.push(dropCheck(live.name, check.name));
|
|
132
|
+
plan.down.push(addCheck(live.name, check));
|
|
133
|
+
moved.checks.add(check.name);
|
|
134
|
+
}
|
|
135
|
+
}
|