@ultimat3/cli 12.0.0 → 14.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +121 -1
- package/package.json +29 -28
- package/src/cmd-db.ts +11 -3
- package/src/cmd-doctor.ts +2 -2
- package/src/db-generate.ts +53 -3
- package/src/db-ungeneratable.ts +108 -0
- package/src/dev-queue.ts +10 -36
- package/src/e2e-dom-fixture.ts +117 -0
- package/src/e2e-driver.ts +78 -0
- package/src/e2e-errors.ts +103 -0
- package/src/e2e-evaluate.ts +156 -0
- package/src/e2e-locator.ts +86 -0
- package/src/e2e-page.ts +124 -0
- package/src/e2e-selection.ts +182 -0
- package/src/error-catalog.ts +1 -0
- package/src/error-codes.ts +40 -0
- package/src/framework-schema.ts +152 -0
- package/src/index.ts +42 -0
- package/src/mcp-errors.ts +29 -0
- package/src/schema-diff.ts +275 -0
- package/src/schema-drift.ts +135 -0
- package/src/schema-errors.ts +28 -0
- package/src/verify-checks.ts +10 -3
package/CLAUDE.md
CHANGED
|
@@ -293,6 +293,45 @@ is a **required** field, so shelling out to GitHub without stating a remedy is a
|
|
|
293
293
|
than a review comment. A GraphQL response is untrusted input and is parsed against a schema, never
|
|
294
294
|
cast: a `null` where an id was expected would otherwise become a mutation against `undefined`.
|
|
295
295
|
|
|
296
|
+
## The browser-backed e2e driver lives here, because the adapter has nowhere else to be
|
|
297
|
+
|
|
298
|
+
`@ultimat3/testing` declares `PageLike` and has never had a driver for it; `@ultimat3/scraping` owns
|
|
299
|
+
the only real browser in the tree and speaks `ScrapePage`. Both are tier 5, so neither may import
|
|
300
|
+
the other, and `testing -> scraping` would be a NEW sideways edge. This package already holds
|
|
301
|
+
declared edges to **both** (`SIDEWAYS_ALLOW`, `scripts/lib/tiers.ts`) and is the one package allowed
|
|
302
|
+
to know about everything — so the join is here, and it is the same rule
|
|
303
|
+
`docs/architecture/01-package-map.md` states for wiring a route table into `pwa`.
|
|
304
|
+
|
|
305
|
+
| File | Job |
|
|
306
|
+
|---|---|
|
|
307
|
+
| `e2e-driver.ts` | `installE2eDriver({ page, baseUrl })` — the ONE call an app's test preload makes. Registers `page` over its declaration and installs the `e2eTest` seam; returns the undo |
|
|
308
|
+
| `e2e-page.ts` | `PageLike` over four members of `ScrapePage`, declared structurally so a test stands one up in six lines |
|
|
309
|
+
| `e2e-locator.ts` | `LocatorLike` — a handle that resolves nothing until asked, one round trip per question |
|
|
310
|
+
| `e2e-selection.ts` | what a locator SELECTS, as data, and the one in-page expression that resolves it |
|
|
311
|
+
| `e2e-evaluate.ts` | the closure→string crossing, which is the only lossy edge in the adapter |
|
|
312
|
+
| `e2e-errors.ts` | one constructor per refusal |
|
|
313
|
+
| `e2e-dom-fixture.ts` | a document small enough to hold in a test and real enough to RUN the expressions above |
|
|
314
|
+
|
|
315
|
+
**Absent by default, and that is a requirement rather than a state.** CI has no Chrome. Nothing here
|
|
316
|
+
runs until `installE2eDriver` is called, so `hasE2eDriver()` still answers `false` and the gate's
|
|
317
|
+
`e2e` step still refuses instead of passing over a browser it does not have.
|
|
318
|
+
|
|
319
|
+
**`evaluate` is the edge that cannot be lossless.** `PageLike.evaluate` takes a closure and every
|
|
320
|
+
browser port in this framework takes a string, so what crosses is `Function.prototype.toString()`
|
|
321
|
+
and nothing else. A zero-parameter closure naming only page globals is supported; a native or bound
|
|
322
|
+
function, a declared parameter and a method shorthand are refused STATICALLY, before a byte leaves;
|
|
323
|
+
a binding the page does not have comes back named, from the page's own `ReferenceError`. Measured on
|
|
324
|
+
Bun 1.4.0 and load-bearing: **Bun's transpiler folds `wanted === 3` to `!0` before `toString()` ever
|
|
325
|
+
runs**, so a captured PRIMITIVE can vanish from the source and never fail at all, while a captured
|
|
326
|
+
reference always survives as its name. No static rule in this process can see the difference — which
|
|
327
|
+
is why the refusal is raised from the page's answer rather than from a scan of the source.
|
|
328
|
+
|
|
329
|
+
**Three of `E2eFixtures`' four members refuse, deliberately.** `offline()` and `online()` need a CDP
|
|
330
|
+
method for the browser's own network state and `CdpPageLike` declares none; `update()` needs a second
|
|
331
|
+
build served under a new id, which is a fact about the server. A fixture that silently no-opped would
|
|
332
|
+
make the assertion after it read as proof — `offline()` followed by "the fallback rendered" is the
|
|
333
|
+
app's ONLINE page passing an offline test.
|
|
334
|
+
|
|
296
335
|
## The `errors` step enforces the error contract
|
|
297
336
|
|
|
298
337
|
| File | Job |
|
|
@@ -534,8 +573,11 @@ regex and `+` is a quantifier — `n1` is what actually selects these tests.
|
|
|
534
573
|
| `db-branch.ts` | what a branch IS: the closed verb set, the name it takes on disk and in `pg_database`, and list/create/drop per mode |
|
|
535
574
|
| `cmd-db-branch.ts` | `x db branch`'s wiring alone — which verb, which refusal, and the one connection an external clone runs on |
|
|
536
575
|
| `db-finding.ts` | one thrown value → one `Finding`, shared by `cmd-db.ts` and `cmd-db-branch.ts` |
|
|
537
|
-
| `drift.ts` | `checkSourceDrift`: the `.hash` sidecar
|
|
576
|
+
| `drift.ts` | `checkSourceDrift`: the `.hash` sidecar the `drift` step compares, no database needed |
|
|
577
|
+
| `schema-diff.ts` | what two GENERATED snapshots disagree about, as data — the pure half |
|
|
578
|
+
| `schema-drift.ts` | `checkMigrationDrift`: entity declarations against the newest `.snapshot.json`, and the composition the `drift` step and `x doctor` both read |
|
|
538
579
|
| `db-destructive.ts` | `checkDestructiveMigrations`: the same step's second half — every committed `up` that drops, truncates or retypes must carry `-- destructive: true` |
|
|
580
|
+
| `db-ungeneratable.ts` | `checkUngeneratableMigrations`: the same step's fourth rail — every committed `up` holding SQL `x db gen` could not have written must say how many, as `-- ungeneratable: <n>` |
|
|
539
581
|
| `db-backfill.ts` | `x db backfill --list`: the flag parsing, the ledger read and the table |
|
|
540
582
|
| `db-seed.ts` | `x db seed`, everything except the argv: `SEED_GLOBS` (where a seed is declared), `discoverSeeds`, `parseSeedTierFlag`, `selectSeeds` (which seeds this invocation runs, and its two refusals — `X_DECLARATION_UNKNOWN` and `X_SEED_ENVIRONMENT`), `runSeeds` (one transaction **per seed**, never one around the run) and the two renderers. The `db-backfill.ts` split repeated: a driver plus plain strings in, plain rows out, so every rule is testable with no `ParsedArgs` and no boot. Which tiers an environment takes is `@ultimat3/entity`'s `seedTiersFor` — two copies of "may this seed run" would be two answers |
|
|
541
583
|
|
|
@@ -643,6 +685,84 @@ file, never one per statement — the marker declares the whole migration. It ri
|
|
|
643
685
|
than becoming an eighteenth step because it is this step's own question over this step's own files;
|
|
644
686
|
a new step is for a genuinely new question.
|
|
645
687
|
|
|
688
|
+
**The `drift` step reads the SNAPSHOT, not only the hash, `As of 2026-08-25`.** `checkSourceDrift`
|
|
689
|
+
compares a schema-source hash to a `.hash` sidecar and never reads what the migration RECORDED, so
|
|
690
|
+
`dummy/social-media-clone` sat green while **nine declared CHECK constraints had never reached any
|
|
691
|
+
database** — a comment body could be whitespace, a like count could go negative, an email needed no
|
|
692
|
+
`@` — and a squash that dropped ten invariants and nine defaults would have been green too. The
|
|
693
|
+
source had not moved, so nothing that hashes source could see it. `checkSnapshotDrift`
|
|
694
|
+
(`schema-drift.ts`) diffs `snapshotOf(describeEntities())` against `declaredSchema(readMigrations())`
|
|
695
|
+
— both sides are `snapshotOf`'s own spelling, which is what makes a check, a default and a column
|
|
696
|
+
type comparable at all. Measured against that app rolled back to the state its gate was green in:
|
|
697
|
+
**20 findings**, 9 checks and 11 defaults.
|
|
698
|
+
|
|
699
|
+
**Two directions, two codes, because they are two repairs.** `X_DB_SCHEMA_UNMIGRATED` is a
|
|
700
|
+
declaration the migrations do not carry — the database will never get it. `X_DB_SCHEMA_UNDECLARED`
|
|
701
|
+
is a migration carrying what nothing declares any more, whose fix names BOTH branches, because the
|
|
702
|
+
declaration may have been lost rather than removed and `x db gen` would emit the DROP. One "drift"
|
|
703
|
+
verdict over both teaches a reader neither.
|
|
704
|
+
|
|
705
|
+
**Absent is not empty, and reading it as "recorded none" would fail every existing app on its first
|
|
706
|
+
run.** `TableDescription.checks` is absent — never `[]` — on a table declaring none, exactly like
|
|
707
|
+
`IndexDescription.using` and `ColumnDescription.generated`. Both sides normalise to empty
|
|
708
|
+
(`schema-diff.ts`), `using` reads through `indexMethodOf` and `order` through `?? 'asc'`, or an app
|
|
709
|
+
whose sidecar predates any of the three reports a difference on every index it has.
|
|
710
|
+
|
|
711
|
+
**The hash half stays, and runs second.** It catches what the snapshot comparison cannot see at all
|
|
712
|
+
— a seed, a helper or a TS-only invariant moving under `packages/db/src` with no statement behind
|
|
713
|
+
it, which is what `reconcileSchemaHash` exists to re-record. It is SUPPRESSED when the snapshot half
|
|
714
|
+
found something: both then answer one condition and only one of them is an instruction, since
|
|
715
|
+
`schema hashes to 3f2a, newest migration recorded 91bc` names no constraint, no column and no table.
|
|
716
|
+
|
|
717
|
+
**An app whose modules will not import is not judged here.** `appEntities` answers `undefined`
|
|
718
|
+
rather than a short registry, which would read as "every table was dropped" and hand out a DROP per
|
|
719
|
+
table for one file's syntax error — the same stance `generateAppMigration` takes with its `blocked`
|
|
720
|
+
outcome. The cost is that a schema check can be silently skipped under an already-red gate; the
|
|
721
|
+
alternative is a false red whose fix destroys data.
|
|
722
|
+
|
|
723
|
+
**The FOURTH thing the `drift` step asks is what neither hash nor snapshot can see: SQL no
|
|
724
|
+
declaration carries at all.** `ALTER TABLE posts REPLICA IDENTITY FULL;` sits in
|
|
725
|
+
`examples/dummy/packages/db/migrations/0001_init.sql`, no generator emits it, no snapshot records
|
|
726
|
+
it, and a squash drops it in silence — and no declaration-based check can ever see it, because a
|
|
727
|
+
regenerated sidecar equals the declaration by construction. `db-ungeneratable.ts` reports it,
|
|
728
|
+
`As of 2026-08-25`. It classifies nothing itself: `@ultimat3/db`'s `ungeneratableStatements`
|
|
729
|
+
matches each statement's leading verb phrase against `GENERATABLE_FORMS` — everything
|
|
730
|
+
`generateMigration` emits, held honest in both directions by that package's own test — because
|
|
731
|
+
every SQL classifier in the tree is db's (`sql-scan.ts`, `statement-split.ts`, `sql-noise.ts`,
|
|
732
|
+
`destructive.ts`) and a second one here is the reimplementation this file's own rule forbids.
|
|
733
|
+
Measured: **7 statements in `examples/dummy`** (five `CREATE TYPE … AS ENUM`, two
|
|
734
|
+
`REPLICA IDENTITY FULL`), **0 in all four `dummy/social-media-clone` migrations** and 0 in
|
|
735
|
+
`examples/dummy`'s own generated `0002_money_scale.sql` — real generator output reports nothing,
|
|
736
|
+
which is the half a rail like this lives or dies on.
|
|
737
|
+
|
|
738
|
+
**The declaration is a header line, and it carries a COUNT: `-- ungeneratable: 7`.** Not
|
|
739
|
+
`@ultimat3/db`'s code but this package's (`X_MIGRATION_UNGENERATABLE`), because the only remedy
|
|
740
|
+
available for every statement it reports is a line in the migration file, and where an app keeps
|
|
741
|
+
its migrations is this package's fact — db classifies and deliberately declares no code. Three
|
|
742
|
+
decisions behind that shape:
|
|
743
|
+
|
|
744
|
+
| Decision | Why |
|
|
745
|
+
|---|---|
|
|
746
|
+
| in the migration file, not a pin table here | the gate runs in every generated app, and a table in `packages/cli/src` can hold no row for an app it has never seen. `-- destructive: true` in the same directory, read by the same reader, is the precedent |
|
|
747
|
+
| a count, not a boolean | the first hand-written statement would otherwise buy the file an unlimited allowance, and statements a reader never sees again are this rail's whole subject. A ratchet in the `README_FENCE_BACKLOG` sense: `found > declared` reports, a declared count that is too high is a pin nobody lowered |
|
|
748
|
+
| the **header** — before the first statement | `hasDestructiveMarker` had to become a lexical scan because a regex over the raw file matched its marker inside a block comment and inside a dollar-quoted body, and `noiseAt` is db's and unexported. A run anchored at index 0 needs no scanner to be exact: before the first statement there is no string, no quoted identifier and no dollar body for a marker to hide in |
|
|
749
|
+
|
|
750
|
+
**The `fix:` names the marker first and `x db gen` second, and that order is the point.**
|
|
751
|
+
Regenerating is exactly what *discards* these statements, so the command every other db code
|
|
752
|
+
answers with is the one this one must not lead with — `X_MIGRATION_UNGENERATABLE`'s `CLI_FIXES` row
|
|
753
|
+
is `x verify --only drift`, and the re-declare branch (an enum is a text column plus a check
|
|
754
|
+
invariant) rides behind an em-dash because it is available for some of the statements and never
|
|
755
|
+
for `REPLICA IDENTITY FULL`, which nothing in the framework emits.
|
|
756
|
+
|
|
757
|
+
**`x db gen` reports what it could not write, and exits 0.** `GeneratedMigration.unrendered` reached
|
|
758
|
+
the committed `.sql` as a `-- UNRENDERED` comment and nothing else read it; `db-generate.ts` now
|
|
759
|
+
carries it on every branch (a REQUIRED field, so forgetting to project it is a type error) and
|
|
760
|
+
`cmd-db.ts` prints the count plus each entry's own remedy and carries the list under
|
|
761
|
+
`data.unrendered`. Not a non-zero exit: `x db gen` is the `fix:` on `X_DB_DRIFT` and four other
|
|
762
|
+
shipped errors, and a fix that always exits 1 is not an instruction — the `x i18n add fr` failure,
|
|
763
|
+
repeated. The red belongs at the gate, and the `drift` step reads the SAME list to decide that
|
|
764
|
+
`x db gen` is not the fix it should be handing out.
|
|
765
|
+
|
|
646
766
|
The *source* half is a different question with a different answer: `checkSourceDrift` hashes the
|
|
647
767
|
entity source against what `x db gen` recorded, answers the same before and after a migration, and
|
|
648
768
|
opens nothing — which is what lets the gate run it in a CI with no database. It stays on `x verify`
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ultimat3/cli",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "14.0.0",
|
|
4
4
|
"description": "The `x` binary: new, dev, build, verify, generate, db, mcp, doctor, deploy",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -37,33 +37,34 @@
|
|
|
37
37
|
},
|
|
38
38
|
"dependencies": {
|
|
39
39
|
"@babel/core": "^7.28.4",
|
|
40
|
-
"@ultimat3/action": "
|
|
41
|
-
"@ultimat3/admin": "
|
|
42
|
-
"@ultimat3/ai": "
|
|
43
|
-
"@ultimat3/auth": "
|
|
44
|
-
"@ultimat3/cache": "
|
|
45
|
-
"@ultimat3/core": "
|
|
46
|
-
"@ultimat3/db": "
|
|
47
|
-
"@ultimat3/entity": "
|
|
48
|
-
"@ultimat3/flags": "
|
|
49
|
-
"@ultimat3/http": "
|
|
50
|
-
"@ultimat3/i18n": "
|
|
51
|
-
"@ultimat3/jobs": "
|
|
52
|
-
"@ultimat3/mail": "
|
|
53
|
-
"@ultimat3/manifest": "
|
|
54
|
-
"@ultimat3/mcp": "
|
|
55
|
-
"@ultimat3/money": "
|
|
56
|
-
"@ultimat3/
|
|
57
|
-
"@ultimat3/
|
|
58
|
-
"@ultimat3/
|
|
59
|
-
"@ultimat3/
|
|
60
|
-
"@ultimat3/
|
|
61
|
-
"@ultimat3/
|
|
62
|
-
"@ultimat3/
|
|
63
|
-
"@ultimat3/
|
|
64
|
-
"@ultimat3/
|
|
65
|
-
"@ultimat3/
|
|
66
|
-
"@ultimat3/
|
|
40
|
+
"@ultimat3/action": "14.0.0",
|
|
41
|
+
"@ultimat3/admin": "14.0.0",
|
|
42
|
+
"@ultimat3/ai": "14.0.0",
|
|
43
|
+
"@ultimat3/auth": "14.0.0",
|
|
44
|
+
"@ultimat3/cache": "14.0.0",
|
|
45
|
+
"@ultimat3/core": "14.0.0",
|
|
46
|
+
"@ultimat3/db": "14.0.0",
|
|
47
|
+
"@ultimat3/entity": "14.0.0",
|
|
48
|
+
"@ultimat3/flags": "14.0.0",
|
|
49
|
+
"@ultimat3/http": "14.0.0",
|
|
50
|
+
"@ultimat3/i18n": "14.0.0",
|
|
51
|
+
"@ultimat3/jobs": "14.0.0",
|
|
52
|
+
"@ultimat3/mail": "14.0.0",
|
|
53
|
+
"@ultimat3/manifest": "14.0.0",
|
|
54
|
+
"@ultimat3/mcp": "14.0.0",
|
|
55
|
+
"@ultimat3/money": "14.0.0",
|
|
56
|
+
"@ultimat3/notify": "14.0.0",
|
|
57
|
+
"@ultimat3/policy": "14.0.0",
|
|
58
|
+
"@ultimat3/pwa": "14.0.0",
|
|
59
|
+
"@ultimat3/query": "14.0.0",
|
|
60
|
+
"@ultimat3/realtime": "14.0.0",
|
|
61
|
+
"@ultimat3/render": "14.0.0",
|
|
62
|
+
"@ultimat3/schema": "14.0.0",
|
|
63
|
+
"@ultimat3/scraping": "14.0.0",
|
|
64
|
+
"@ultimat3/seo": "14.0.0",
|
|
65
|
+
"@ultimat3/storage": "14.0.0",
|
|
66
|
+
"@ultimat3/testing": "14.0.0",
|
|
67
|
+
"@ultimat3/time": "14.0.0",
|
|
67
68
|
"babel-preset-solid": "^1.9.15"
|
|
68
69
|
}
|
|
69
70
|
}
|
package/src/cmd-db.ts
CHANGED
|
@@ -21,7 +21,7 @@ import { plannedSubcommand } from './cmd-planned';
|
|
|
21
21
|
import type { CliCommand, CommandContext } from './command';
|
|
22
22
|
import { BRANCH_SUBCOMMANDS } from './db-branch';
|
|
23
23
|
import { stepFinding } from './db-finding';
|
|
24
|
-
import { generateAppMigration } from './db-generate';
|
|
24
|
+
import { generateAppMigration, unrenderedJson, unrenderedLines } from './db-generate';
|
|
25
25
|
import type { SeedPassRow } from './db-seed';
|
|
26
26
|
import {
|
|
27
27
|
discoverSeeds,
|
|
@@ -192,6 +192,12 @@ async function runGen(ctx: CommandContext, root: string, name: string): Promise<
|
|
|
192
192
|
findings: [stepFinding(error, 'X_DB_GEN_FAILED')],
|
|
193
193
|
};
|
|
194
194
|
}
|
|
195
|
+
// Loud on BOTH branches, and never an exit code. `x db gen` is the `fix:` on `X_DB_DRIFT` and on
|
|
196
|
+
// four other shipped errors, so a run that exits 1 whenever the description carries a default
|
|
197
|
+
// this build cannot project would make every one of those instructions unfollowable — the
|
|
198
|
+
// `x i18n add fr` failure, repeated. The count is the verdict; the gate's own `drift` step is
|
|
199
|
+
// where a red belongs, and it reads the same list to decide that `x db gen` is not the fix.
|
|
200
|
+
const lost = unrenderedLines(generated.unrendered);
|
|
195
201
|
const migration = generated.migration;
|
|
196
202
|
if (migration === undefined) {
|
|
197
203
|
return {
|
|
@@ -205,12 +211,13 @@ async function runGen(ctx: CommandContext, root: string, name: string): Promise<
|
|
|
205
211
|
: msg('cli.db.gen.unchanged'),
|
|
206
212
|
findings: generated.findings,
|
|
207
213
|
// `files` is what this command WROTE, so the empty array here was a false claim.
|
|
208
|
-
lines: generated.files.map((file) => ` ${file}`),
|
|
214
|
+
lines: [...generated.files.map((file) => ` ${file}`), ...lost],
|
|
209
215
|
data: {
|
|
210
216
|
outcome: generated.outcome,
|
|
211
217
|
migration: null,
|
|
212
218
|
files: [...generated.files],
|
|
213
219
|
schemaHash: generated.schemaHash ?? null,
|
|
220
|
+
unrendered: unrenderedJson(generated.unrendered),
|
|
214
221
|
},
|
|
215
222
|
};
|
|
216
223
|
}
|
|
@@ -218,13 +225,14 @@ async function runGen(ctx: CommandContext, root: string, name: string): Promise<
|
|
|
218
225
|
ok: true,
|
|
219
226
|
command: 'db',
|
|
220
227
|
summary: msg('cli.db.gen.written', { id: migration.id }),
|
|
221
|
-
lines: generated.files.map((file) => ` ${file}`),
|
|
228
|
+
lines: [...generated.files.map((file) => ` ${file}`), ...lost],
|
|
222
229
|
data: {
|
|
223
230
|
outcome: generated.outcome,
|
|
224
231
|
migration: migration.id,
|
|
225
232
|
name: migration.name,
|
|
226
233
|
files: [...generated.files],
|
|
227
234
|
schemaHash: generated.schemaHash ?? null,
|
|
235
|
+
unrendered: unrenderedJson(generated.unrendered),
|
|
228
236
|
},
|
|
229
237
|
};
|
|
230
238
|
}
|
package/src/cmd-doctor.ts
CHANGED
|
@@ -11,12 +11,12 @@ import { findAppRoot, REQUIRED_BUN, versionAtLeast } from './app-root';
|
|
|
11
11
|
import type { CliCommand, CommandContext } from './command';
|
|
12
12
|
import { checkMigrationSnapshots } from './db-snapshot';
|
|
13
13
|
import { ICON_SOURCE } from './dev-assets';
|
|
14
|
-
import { checkSourceDrift } from './drift';
|
|
15
14
|
import { intFlagOr, neighbouringPort, PORT_RANGE } from './flag-number';
|
|
16
15
|
import { msg } from './messages';
|
|
17
16
|
import type { CommandResult, Finding } from './output';
|
|
18
17
|
import type { ParsedArgs } from './parse';
|
|
19
18
|
import { portFree } from './port-probe';
|
|
19
|
+
import { checkMigrationDrift } from './schema-drift';
|
|
20
20
|
|
|
21
21
|
/**
|
|
22
22
|
* The injection seam `runDoctor` reads instead of the environment. Not a semver surface —
|
|
@@ -262,7 +262,7 @@ export function probeFor(cwd: string, bunVersion: string, port: number): DoctorP
|
|
|
262
262
|
exists: (relativePath) => (root === undefined ? false : existsSync(join(root, relativePath))),
|
|
263
263
|
portFree,
|
|
264
264
|
database: () => probeDatabase(process.env['DATABASE_URL']),
|
|
265
|
-
drift: async () => (root === undefined ? [] :
|
|
265
|
+
drift: async () => (root === undefined ? [] : checkMigrationDrift(root)),
|
|
266
266
|
snapshots: async () => (root === undefined ? [] : checkMigrationSnapshots(root)),
|
|
267
267
|
};
|
|
268
268
|
}
|
package/src/db-generate.ts
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
// `node:path` — Bun ships no path joiner of its own, and these paths are built, not opened.
|
|
6
6
|
import { join } from 'node:path';
|
|
7
|
-
import type { GeneratedMigration } from '@ultimat3/db';
|
|
7
|
+
import type { GeneratedMigration, UnrenderedDeclaration } from '@ultimat3/db';
|
|
8
8
|
import {
|
|
9
9
|
DESTRUCTIVE_MARKER,
|
|
10
10
|
declaredSchema,
|
|
@@ -39,6 +39,14 @@ export interface GeneratedFiles {
|
|
|
39
39
|
readonly files: readonly string[];
|
|
40
40
|
/** What the source hashes to, whenever this run was in a position to record it. */
|
|
41
41
|
readonly schemaHash?: string | undefined;
|
|
42
|
+
/**
|
|
43
|
+
* Every declaration the generator could not write down, on EVERY branch that got as far as
|
|
44
|
+
* running it — a `.default('draft')` the description carries no expression for is unrendered
|
|
45
|
+
* whether or not this run had a diff to write. Required, not optional: a caller that forgets to
|
|
46
|
+
* project it is a type error, which is the only reason the field was silent for as long as it
|
|
47
|
+
* was.
|
|
48
|
+
*/
|
|
49
|
+
readonly unrendered: readonly UnrenderedDeclaration[];
|
|
42
50
|
/** Modules that would not load. Non-empty means nothing was generated. */
|
|
43
51
|
readonly findings: readonly Finding[];
|
|
44
52
|
}
|
|
@@ -81,7 +89,9 @@ export async function generateAppMigration(
|
|
|
81
89
|
options: GenerateMigrationOptions,
|
|
82
90
|
): Promise<GeneratedFiles> {
|
|
83
91
|
const app = await loadApp(root);
|
|
84
|
-
if (app.findings.length > 0)
|
|
92
|
+
if (app.findings.length > 0) {
|
|
93
|
+
return { outcome: 'blocked', files: [], findings: app.findings, unrendered: [] };
|
|
94
|
+
}
|
|
85
95
|
|
|
86
96
|
const migrations = await readMigrations(root);
|
|
87
97
|
const current = declaredSchema(migrations);
|
|
@@ -115,13 +125,16 @@ export async function generateAppMigration(
|
|
|
115
125
|
const newest = migrations[migrations.length - 1];
|
|
116
126
|
// No migration to record against, which in this branch means no entity is declared either —
|
|
117
127
|
// a registry against zero migrations is `create table` for all of it, never an empty diff.
|
|
118
|
-
if (newest === undefined)
|
|
128
|
+
if (newest === undefined) {
|
|
129
|
+
return { outcome: 'unchanged', files: [], findings: [], unrendered: migration.unrendered };
|
|
130
|
+
}
|
|
119
131
|
const reconciled = await reconcileSchemaHash(root, newest.id);
|
|
120
132
|
return {
|
|
121
133
|
outcome: reconciled.written ? 'hash-recorded' : 'unchanged',
|
|
122
134
|
schemaHash: reconciled.hash,
|
|
123
135
|
files: reconciled.written ? [`${MIGRATIONS_DIR}/${hashFileName(newest.id)}`] : [],
|
|
124
136
|
findings: [],
|
|
137
|
+
unrendered: migration.unrendered,
|
|
125
138
|
};
|
|
126
139
|
}
|
|
127
140
|
|
|
@@ -140,5 +153,42 @@ export async function generateAppMigration(
|
|
|
140
153
|
schemaHash,
|
|
141
154
|
files: [sql, snapshot, hashFileName(migration.id)].map((file) => `${MIGRATIONS_DIR}/${file}`),
|
|
142
155
|
findings: [],
|
|
156
|
+
unrendered: migration.unrendered,
|
|
143
157
|
};
|
|
144
158
|
}
|
|
159
|
+
|
|
160
|
+
/**
|
|
161
|
+
* The loss, on the human path. `-- UNRENDERED` in the emitted SQL is a comment, and a comment in a
|
|
162
|
+
* file nobody opens is not a verdict — this is what `x db gen` PRINTS, so the count is read the
|
|
163
|
+
* moment the migration is written rather than at the review that may not happen.
|
|
164
|
+
*
|
|
165
|
+
* Empty in, empty out: a header on every run marks none of them, the rule `unrenderedComment` and
|
|
166
|
+
* `destructive.ts` both already state for their own markers.
|
|
167
|
+
*/
|
|
168
|
+
export function unrenderedLines(entries: readonly UnrenderedDeclaration[]): readonly string[] {
|
|
169
|
+
if (entries.length === 0) return [];
|
|
170
|
+
return [
|
|
171
|
+
` ${entries.length} declaration${entries.length === 1 ? '' : 's'} reached no SQL — this migration is smaller than the entities declare`,
|
|
172
|
+
...entries.flatMap((entry) => [
|
|
173
|
+
` ${entry.kind} on "${entry.table}"."${entry.name}": ${entry.cause}`,
|
|
174
|
+
` fix: ${entry.fix}`,
|
|
175
|
+
]),
|
|
176
|
+
];
|
|
177
|
+
}
|
|
178
|
+
|
|
179
|
+
/**
|
|
180
|
+
* The same list under `--json`, field for field. Spread into plain objects because `data` is a
|
|
181
|
+
* `JsonValue` and the payload has to carry the same facts the human lines do — a `--json` reader
|
|
182
|
+
* that had to parse a rendered sentence back is the drift `renderHuman` exists to prevent.
|
|
183
|
+
*/
|
|
184
|
+
export function unrenderedJson(
|
|
185
|
+
entries: readonly UnrenderedDeclaration[],
|
|
186
|
+
): readonly Record<string, string>[] {
|
|
187
|
+
return entries.map((entry) => ({
|
|
188
|
+
kind: entry.kind,
|
|
189
|
+
table: entry.table,
|
|
190
|
+
name: entry.name,
|
|
191
|
+
cause: entry.cause,
|
|
192
|
+
fix: entry.fix,
|
|
193
|
+
}));
|
|
194
|
+
}
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
// The hand-written-SQL rail at the gate: a committed `up` holding statements `x db gen` could
|
|
2
|
+
// never have written must say how many, or a squash discards them in silence. Files, not the
|
|
3
|
+
// database, so the rail fires in CI. `@ultimat3/db` owns the classifier (`GENERATABLE_FORMS`);
|
|
4
|
+
// this file owns the declaration an app makes about its own migration, and the code for it.
|
|
5
|
+
|
|
6
|
+
import { UltimateError } from '@ultimat3/core';
|
|
7
|
+
import { ungeneratableStatements } from '@ultimat3/db';
|
|
8
|
+
import { MIGRATIONS_DIR, readMigrations } from './migrations';
|
|
9
|
+
import { type Finding, findingFrom } from './output';
|
|
10
|
+
|
|
11
|
+
/** The header line a migration carries to declare how many hand-written statements it holds. */
|
|
12
|
+
export const ungeneratableMarker = (count: number): string => `-- ungeneratable: ${count}`;
|
|
13
|
+
|
|
14
|
+
/**
|
|
15
|
+
* Everything before the first statement: whitespace and comments, and nothing else can be there.
|
|
16
|
+
*
|
|
17
|
+
* That restriction is the whole reason the marker is a HEADER line rather than any top-level
|
|
18
|
+
* comment. `hasDestructiveMarker` had to be rewritten as a lexical scan because a regex over the
|
|
19
|
+
* raw file matched its marker inside a block comment and inside a dollar-quoted body, where it
|
|
20
|
+
* declares nothing — and `noiseAt`, the scanner that answers that, is `@ultimat3/db`'s and is not
|
|
21
|
+
* exported. A run anchored at index 0 needs no scanner to be exact: before the first statement
|
|
22
|
+
* there is no string, no quoted identifier and no dollar body for a marker to hide in.
|
|
23
|
+
*
|
|
24
|
+
* A nested block comment (`/* /* *\/ *\/`, which Postgres allows) ends the run early, so a marker
|
|
25
|
+
* below one is not seen and the migration reports. Fail-closed, which is the safe direction: the
|
|
26
|
+
* repair is moving the line up, never a finding nobody can see.
|
|
27
|
+
*/
|
|
28
|
+
const HEADER = /^(?:\s*(?:--[^\n]*|\/\*[\s\S]*?\*\/))*/;
|
|
29
|
+
|
|
30
|
+
/** Anchored to the whole line, so `-- ungeneratable: 2 was wrong` declares nothing. */
|
|
31
|
+
const MARKER_LINE = /^[ \t]*--[ \t]*ungeneratable:[ \t]*(\d+)[ \t]*\r?$/im;
|
|
32
|
+
|
|
33
|
+
/**
|
|
34
|
+
* How many hand-written statements this `up` admits to. Absent is `0` — a migration that says
|
|
35
|
+
* nothing has declared nothing, which is what makes the rule apply to every app that never heard
|
|
36
|
+
* of it.
|
|
37
|
+
*
|
|
38
|
+
* The FIRST marker wins where a file carries two. Taking the largest would let a stale line raise
|
|
39
|
+
* the allowance of the file it sits in, and the direction that fires is `found > declared`.
|
|
40
|
+
*/
|
|
41
|
+
export function declaredUngeneratable(up: string): number {
|
|
42
|
+
const header = HEADER.exec(up)?.[0] ?? '';
|
|
43
|
+
// Block comments come out first: everything left in the run is then whitespace or a line
|
|
44
|
+
// comment, and a marker inside `/* … */` is prose about a marker rather than one.
|
|
45
|
+
const lines = header.replace(/\/\*[\s\S]*?\*\//g, ' ');
|
|
46
|
+
const declared = MARKER_LINE.exec(lines)?.[1];
|
|
47
|
+
return declared === undefined ? 0 : Number.parseInt(declared, 10);
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
/**
|
|
51
|
+
* A migration holding SQL no declaration carries and no snapshot records.
|
|
52
|
+
*
|
|
53
|
+
* One error per file, never one per statement: the header line declares the whole migration, so a
|
|
54
|
+
* second finding would repeat the instruction the first already gave — `migrationDestructive`'s
|
|
55
|
+
* rule, for the same reason.
|
|
56
|
+
*
|
|
57
|
+
* Two remedies, and the order is deliberate. The marker is always available and always correct:
|
|
58
|
+
* the statement stays, and the next author to squash these migrations is told by the file itself
|
|
59
|
+
* that regenerating it loses something. Re-declaring is available only for the statements an
|
|
60
|
+
* entity can express — an enum is a text column plus a check invariant, which `x db gen` writes —
|
|
61
|
+
* and never for `REPLICA IDENTITY FULL`, which nothing in the framework emits, so a `fix:` naming
|
|
62
|
+
* only the second branch would be an instruction half its readers cannot carry out.
|
|
63
|
+
*/
|
|
64
|
+
export class MigrationUngeneratableError extends UltimateError {
|
|
65
|
+
constructor(input: { file: string; declared: number; statements: readonly string[] }) {
|
|
66
|
+
const found = input.statements.length;
|
|
67
|
+
// The count is stated once. `migrationDestructive` says "(and 3 more)" because its subject is
|
|
68
|
+
// one statement plus an unstated remainder; here the number IS the declaration being asked for,
|
|
69
|
+
// so repeating it as a remainder would print the same fact twice in one line.
|
|
70
|
+
const first = `${found === 1 ? '' : `, the first of ${found}`}: ${input.statements[0] ?? ''}`;
|
|
71
|
+
super({
|
|
72
|
+
code: 'X_MIGRATION_UNGENERATABLE',
|
|
73
|
+
cause:
|
|
74
|
+
`${input.file} holds ${found} statement${found === 1 ? '' : 's'} x db gen could not have ` +
|
|
75
|
+
`written and declares ${input.declared}${first}`,
|
|
76
|
+
fix:
|
|
77
|
+
`add the header line "${ungeneratableMarker(found)}" to ${input.file}, so a squash carries ` +
|
|
78
|
+
'these statements by hand — or re-declare what an entity can express (an enum is a text ' +
|
|
79
|
+
'column plus a check invariant) and regenerate: x db gen "<name>"',
|
|
80
|
+
});
|
|
81
|
+
}
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
/**
|
|
85
|
+
* Every committed migration whose `up` holds more hand-written statements than its header admits.
|
|
86
|
+
*
|
|
87
|
+
* `readMigrations` is the reader `x db migrate` applies from, and only its `up` half is judged —
|
|
88
|
+
* a rail checking a list the migrator does not run, or SQL the migrator never sends, enforces
|
|
89
|
+
* nothing. An app with no migrations directory has nothing to declare and reports nothing.
|
|
90
|
+
*
|
|
91
|
+
* A declared count HIGHER than what is there is not a finding: the ratchet only refuses a rise,
|
|
92
|
+
* exactly as `README_FENCE_BACKLOG` and `TEST_TYPECHECK_PINS` do, and a count that fell is a pin
|
|
93
|
+
* nobody lowered rather than SQL nobody can see.
|
|
94
|
+
*/
|
|
95
|
+
export async function checkUngeneratableMigrations(root: string): Promise<readonly Finding[]> {
|
|
96
|
+
const findings: Finding[] = [];
|
|
97
|
+
for (const migration of await readMigrations(root)) {
|
|
98
|
+
const statements = ungeneratableStatements(migration.up);
|
|
99
|
+
const declared = declaredUngeneratable(migration.up);
|
|
100
|
+
if (statements.length <= declared) continue;
|
|
101
|
+
const file = `${MIGRATIONS_DIR}/${migration.id}.sql`;
|
|
102
|
+
findings.push({
|
|
103
|
+
...findingFrom(new MigrationUngeneratableError({ file, declared, statements })),
|
|
104
|
+
at: file,
|
|
105
|
+
});
|
|
106
|
+
}
|
|
107
|
+
return findings;
|
|
108
|
+
}
|
package/src/dev-queue.ts
CHANGED
|
@@ -8,11 +8,8 @@ import {
|
|
|
8
8
|
type PostgresIdempotencyStore,
|
|
9
9
|
postgresIdempotencyStore,
|
|
10
10
|
resetIdempotency,
|
|
11
|
-
SQL_AUDIT_TABLE,
|
|
12
|
-
SQL_IDEMPOTENCY_TABLE,
|
|
13
11
|
setIdempotencyStore,
|
|
14
12
|
} from '@ultimat3/action';
|
|
15
|
-
import { SQL_AUTH_LIMIT_TABLES } from '@ultimat3/auth';
|
|
16
13
|
import type { DbClient, PgliteClient, PostgresClient, SqlFragment } from '@ultimat3/db';
|
|
17
14
|
import {
|
|
18
15
|
createPgliteClient,
|
|
@@ -23,7 +20,6 @@ import {
|
|
|
23
20
|
setDbClient,
|
|
24
21
|
} from '@ultimat3/db';
|
|
25
22
|
import type { Tx } from '@ultimat3/entity';
|
|
26
|
-
import { SQL_RATE_LIMIT_TABLE } from '@ultimat3/http';
|
|
27
23
|
import type { EventBus, JobDriver, OutboxStore, PgExecutor } from '@ultimat3/jobs';
|
|
28
24
|
import {
|
|
29
25
|
createJobsFacade,
|
|
@@ -32,13 +28,13 @@ import {
|
|
|
32
28
|
createPgOutboxStore,
|
|
33
29
|
resetJobDriver,
|
|
34
30
|
resetJobsFacade,
|
|
35
|
-
SQL_JOBS_TABLE,
|
|
36
31
|
setEventBus,
|
|
37
32
|
setJobDriver,
|
|
38
33
|
setJobsFacade,
|
|
39
34
|
} from '@ultimat3/jobs';
|
|
40
35
|
import { attachReplica, type ReplicaEnv, replicaUrlFor } from './dev-replica';
|
|
41
36
|
import type { DevServices } from './dev-services';
|
|
37
|
+
import { applyFrameworkSchema } from './framework-schema';
|
|
42
38
|
import type { RuntimeOverrides } from './runtime-overrides';
|
|
43
39
|
|
|
44
40
|
/** Both embedded and external clients boot lazily and close explicitly. */
|
|
@@ -113,40 +109,18 @@ export function pgExecutorFor(client: DbClient): PgExecutor {
|
|
|
113
109
|
/**
|
|
114
110
|
* Every table this process's framework packages own, applied before anything reads one.
|
|
115
111
|
*
|
|
116
|
-
*
|
|
117
|
-
*
|
|
118
|
-
*
|
|
112
|
+
* The LIST is `FRAMEWORK_SCHEMA` and lives in `framework-schema.ts`, not here: this function is on
|
|
113
|
+
* every boot path the framework has — `x dev`, each served role, `x jobs`, `x db backfill`,
|
|
114
|
+
* `x mcp serve` and `ROLE=migrate` all reach it through `startQueue` — so the list it reads is the
|
|
115
|
+
* one place a framework table can be forgotten, and it is worth being a table somebody can read
|
|
116
|
+
* rather than an array literal inside a boot function.
|
|
119
117
|
*
|
|
120
|
-
*
|
|
121
|
-
* `@ultimat3/
|
|
122
|
-
*
|
|
123
|
-
* — the same reason `SQL_JOBS_TABLE` is applied here. Each one absent is the same failure at a
|
|
124
|
-
* different door: a retried `POST /api/payments/charge` charges the card twice, and the FIRST
|
|
125
|
-
* request a `rateLimitStore` deployment serves dies on a missing `x_rate_limit` relation. The
|
|
126
|
-
* table is installed whether or not this boot passes `runtime.rateLimitStore` — `create table if
|
|
127
|
-
* not exists` on an unused table costs one round trip at boot, and a store installed later must
|
|
128
|
-
* not be the thing that discovers the schema was never applied. The auth pair is the strongest
|
|
129
|
-
* case for that rule: `defineAuth` builds its limiter when the APP's modules import, which is
|
|
130
|
-
* after this, so the first failed sign-in would otherwise be what discovers the missing relation.
|
|
118
|
+
* Each package's DDL is here and not in `@ultimat3/action`, `@ultimat3/http`, `@ultimat3/auth` or
|
|
119
|
+
* `@ultimat3/notify` because a package that holds no database dependency cannot apply its own
|
|
120
|
+
* schema — the same reason `SQL_JOBS_TABLE` is applied by the boot.
|
|
131
121
|
*/
|
|
132
122
|
async function applySchema(client: DevDbClient): Promise<void> {
|
|
133
|
-
|
|
134
|
-
SQL_JOBS_TABLE,
|
|
135
|
-
SQL_IDEMPOTENCY_TABLE,
|
|
136
|
-
// The DDL only, and deliberately NO `setAuditSink` beside `setIdempotencyStore` below: there
|
|
137
|
-
// is no default audit sink on purpose, so `X_AUDIT_SINK_MISSING` keeps firing at boot for an
|
|
138
|
-
// app that declares `audit: true` and installs none. Applying the table without installing a
|
|
139
|
-
// sink is the same call `SQL_RATE_LIMIT_TABLE` already makes — one round trip at boot on a
|
|
140
|
-
// possibly-unused table, against `postgresAuditSink` failing its first write with
|
|
141
|
-
// `relation "x_audit" does not exist`.
|
|
142
|
-
SQL_AUDIT_TABLE,
|
|
143
|
-
SQL_RATE_LIMIT_TABLE,
|
|
144
|
-
SQL_AUTH_LIMIT_TABLES,
|
|
145
|
-
]) {
|
|
146
|
-
for (const statement of ddl.split(';')) {
|
|
147
|
-
if (statement.trim().length > 0) await client.execute(raw(statement));
|
|
148
|
-
}
|
|
149
|
-
}
|
|
123
|
+
await applyFrameworkSchema((statement) => client.execute(raw(statement)));
|
|
150
124
|
}
|
|
151
125
|
|
|
152
126
|
/**
|