@ultimat3/db 12.0.0 → 13.0.0

package/CLAUDE.md

@@ -653,6 +653,35 @@ right database, and `x db gen`'s `retypeColumn` owns that question where both si
 The `fix:` is the `alter table … set not null` itself and deliberately not `x db gen`, which has
 never emitted one and would answer with an empty migration.
 
+**A column the DATABASE computes is a different thing at every step, and `generated-column.ts` is
+all of them** — `As of 2026-08-24`. `ColumnDescriptionLike.generated` carries the
+`generated always as (<expr>) stored` body across the tier seam (this package cannot import
+`@ultimat3/entity`, so a field that is not on the projection reaches no DDL at all), and it reached
+none until this date: `@ultimat3/entity`'s `.searchable()` emitted a `tsvector not null` column that
+`columnClause` rendered plain, so nothing computed it and **the first insert was a `23502`**. Loud,
+which was deliberate — but a feature nobody can insert into is not shipped. Four rules ride with it,
+each one measured against a real server (`generate-generated-column.live.test.ts`):
+
+| Rule | Why it is not the ordinary column's rule |
+|---|---|
+| the clause sits directly after the type | `"c" tsvector generated always as (…) stored not null check (…)` is what Postgres accepts; a column constraint may follow it |
+| **generated and defaulted is refused** at `x db gen` | Postgres has no such column (`42601`) — a generated column's value IS its expression. `X_INVARIANT`, the same refusal `createIndex` gives a unique GIN, and for the same reason: the alternative is DDL whose first reader is `ROLE=migrate` |
+| an expression that moved is **`set expression as (…)`**, never a drop and recreate | Postgres 17's statement, and it rewrites the table, recomputes every row and **keeps the column's indexes** — measured. Dropping the column takes its GIN index with it and nothing in the diff puts one back, and `alter table … drop column` is what `destructive.ts` reads as data loss: every expression change would then carry `-- destructive: true` on a migration that loses nothing, and a marker on all is none |
+| a retype on it carries **no `using`** | Postgres refuses `using` on a generated column outright, which is exactly what `retypeColumn` emits for every other column — and there is nothing to convert, because the expression produces the new type itself |
+| the NOT NULL add is **one statement**, never nullable-then-backfill | the database computes it for every existing row inside the same `add column`. The ordinary path's `-- backfill "c", then: … set not null;` names a step nobody can perform: writing to a generated column is `428C9` |
+
+Two transitions have no `set expression`. **Generated → plain is `drop expression`**, which keeps
+every value the column already computed. **Plain → generated is the whole column again** — drop,
+add, and every index over it stated a second time, which is why `regenerate` answers `rebuilt` and
+`diffTable` carries that set into its index loop: `redefineIndex` sees a definition that never moved
+and would emit nothing, so the table would come back with no index at all.
+
+**`introspect` deliberately does not read `generation_expression` back.** Postgres stores its own
+rewriting (`COALESCE(title, ''::text)` for `coalesce("title", '')`), so a catalog value could never
+compare equal to a generated one and drift would report a correct database forever. The diff that
+DOES read it is `x db gen`'s, where both sides are this generator's own spellings — the rule
+`IndexDescription.where` already states.
+
 `compareTable` judges **declared** indexes: one the migrations name and the catalog does not hold is
 `missing-index`, and one whose access method, column list or uniqueness moved is `changed-index` — which is what
 catches a composite index rebuilt with its columns the other way round while the column diff said
@@ -750,7 +779,8 @@ rather than spliced DDL — the discipline `createIndex` already applies to an i
 ceiling and along the seam the tier already draws: they are the structural mirror of
 `@ultimat3/entity`'s description, which is how a snapshot crosses tier 2 → tier 1 with no import.
 `ColumnDescriptionLike.onDelete` is optional for exactly that reason — a description written before
-the field existed still satisfies the shape.
+the field existed still satisfies the shape — and `ColumnDescriptionLike.generated` is optional for
+the same one.
 
 **`snapshot-json.ts` writes the sidecar's bytes, and they must be a fixed point of Biome.** A
 scaffolded app's `lint` step is `biome check .` over `"includes": ["**"]`, and `.sql`/`.hash` are

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ultimat3/db",
-  "version": "12.0.0",
+  "version": "13.0.0",
   "description": "Postgres access, transactions, migrations and drift detection",
   "license": "MIT",
   "type": "module",
@@ -31,7 +31,7 @@
     "test": "bun test"
   },
   "dependencies": {
-    "@ultimat3/core": "12.0.0"
+    "@ultimat3/core": "13.0.0"
   },
   "peerDependencies": {
     "@electric-sql/pglite": ">=0.5.0"

package/src/entity-shape.ts

@@ -21,6 +21,17 @@ export interface ColumnDescriptionLike {
    * the field travelling structurally is the *only* way the rule crosses the tier boundary.
    */
   readonly onDelete?: string | null | undefined;
+  /**
+   * The `generated always as (<expr>) stored` body, when the DATABASE computes this column rather
+   * than a writer. Absent on every ordinary column, exactly like `IndexDescriptionLike.using`: a
+   * description written before this existed emits the statement it always emitted.
+   *
+   * `@ultimat3/entity` (tier 2) is the declarer and this package cannot import it, so the
+   * expression crosses the seam structurally or it reaches no DDL at all — which is where it was
+   * until `As of 2026-08-24`: the column landed as a plain `tsvector not null` and the first insert
+   * was a `23502`, because nothing computed it.
+   */
+  readonly generated?: string | undefined;
 }
 
 /**

package/src/generate.ts

@@ -13,6 +13,8 @@ import type {
 } from './entity-shape';
 import { migrationIrreversible } from './errors';
 import { type ConstraintPlans, foreignKeyPlan, foreignKeysOf, type Plan } from './foreign-key-plan';
+import type { Regeneration } from './generated-column';
+import { generatedClause, isGenerated, regenerate } from './generated-column';
 import { declaredMethod, indexMethodOf, indexMethodSql } from './index-method';
 import {
   type ColumnDescription,
@@ -54,7 +56,10 @@ function defaultExpression(column: ColumnDescriptionLike): string | null {
 }
 
 function columnClause(column: ColumnDescriptionLike): string {
-  const parts = [`"${column.column}"`, sqlType(column.kind)];
+  // The generation clause sits directly after the type, and `generatedClause` refuses the pairs
+  // Postgres has no column for. Every other part below is unchanged and unreachable for a
+  // generated column: it may carry no default, and `hasDefault` is what the refusal reads.
+  const parts = [`"${column.column}"`, `${sqlType(column.kind)}${generatedClause(column)}`];
   const expression = defaultExpression(column);
   if (expression !== null) parts.push(`default ${expression}`);
   if (column.notNull) parts.push('not null');
@@ -106,6 +111,9 @@ export function snapshotOf(entities: readonly EntityDescriptionLike[]): SchemaDe
           nullable: !column.notNull,
           default: defaultExpression(column),
           position: index + 1,
+          // Only when one was declared — absent stays absent, so no snapshot written before this
+          // field existed gains a key and no app's sidecar regenerates over a fact already true.
+          ...(column.generated === undefined ? {} : { generated: column.generated }),
         }));
       return {
         schema: 'public',
@@ -197,14 +205,20 @@ function retypeColumn(
   column: ColumnDescriptionLike,
   recorded: ColumnDescription,
   plan: Plan,
-): void {
+): Regeneration {
   const wanted = sqlType(column.kind);
-  if (recorded.dataType === wanted) return;
+  // A generated column moves by its own rules — see `generated-column.ts`. Asked whenever EITHER
+  // side is one, because becoming generated and ceasing to be are both changes with a statement.
+  if (isGenerated(column) || recorded.generated !== undefined) {
+    return regenerate(table, column, wanted, recorded, plan);
+  }
+  if (recorded.dataType === wanted) return 'unchanged';
   const alter = (type: string): string =>
     `alter table "${table}" alter column "${column.column}" type ${type} ` +
     `using "${column.column}"::${type};`;
   plan.up.push(alter(wanted));
   plan.down.push(alter(recorded.dataType));
+  return 'altered';
 }
 
 /** The parts of an index Postgres cannot alter in place — every one of them is a rebuild. */
@@ -258,16 +272,26 @@ function redefineIndex(
 function diffTable(entity: EntityDescriptionLike, live: TableDescription, plan: Plan): void {
   const existing = new Map(live.columns.map((column) => [column.name, column]));
   const added = new Set<string>();
+  // A column `regenerate` had to replace outright: `add column` implies no index, so every index
+  // over it has to be stated again even though its own definition never moved.
+  const rebuilt = new Set<string>();
   for (const column of entity.columns) {
     const recorded = existing.get(column.column);
     if (recorded !== undefined) {
-      retypeColumn(entity.table, column, recorded, plan);
+      if (retypeColumn(entity.table, column, recorded, plan) === 'rebuilt') {
+        rebuilt.add(column.column);
+      }
       continue;
     }
     added.add(column.column);
     // A NOT NULL add with no default cannot succeed on a populated table; emit it nullable and
     // leave the agent the exact follow-up rather than a migration that fails at 3am.
-    const nullable = column.notNull && defaultExpression(column) === null;
+    //
+    // A GENERATED column is the exception and not a special case of it: the database computes it
+    // for every existing row inside the same `add column`, so it lands NOT NULL and populated in
+    // one statement — measured. Emitting it nullable would leave a `-- backfill` comment naming a
+    // step nobody can perform, since a generated column cannot be written to.
+    const nullable = column.notNull && !isGenerated(column) && defaultExpression(column) === null;
     const clause = nullable ? columnClause({ ...column, notNull: false }) : columnClause(column);
     plan.up.push(`alter table "${entity.table}" add column ${clause};`);
     if (nullable) {
@@ -282,7 +306,9 @@ function diffTable(entity: EntityDescriptionLike, live: TableDescription, plan:
   const indexed = new Map(live.indexes.map((index) => [index.name, index]));
   for (const index of entity.indexes) {
     const recorded = indexed.get(index.name);
-    if (recorded !== undefined) {
+    // A rebuilt column took its indexes down with it, so this one is CREATED rather than compared:
+    // `redefineIndex` sees a definition that never moved and would emit nothing at all.
+    if (recorded !== undefined && !index.columns.some((column) => rebuilt.has(column))) {
       redefineIndex(entity.table, index, recorded, plan);
       continue;
     }

package/src/generated-column.ts

@@ -0,0 +1,110 @@
+// Single responsibility: what a column the DATABASE computes contributes to DDL, and what changes
+// to one a migration may emit. Split from `generate.ts` because Postgres treats a generated column
+// as a different thing at every step — its clause, its retype, its NOT NULL add and the way its
+// expression moves are each a rule of their own, and none of them is the ordinary column's.
+
+import { assert } from '@ultimat3/core';
+import type { ColumnDescriptionLike } from './entity-shape';
+import type { Plan } from './foreign-key-plan';
+import type { ColumnDescription } from './introspect';
+
+/** How a column that moved was brought back into line — what the caller has to do next, if anything. */
+export type Regeneration = 'unchanged' | 'altered' | 'rebuilt';
+
+const alterColumn = (table: string, column: string): string =>
+  `alter table "${table}" alter column "${column}"`;
+
+/**
+ * The `generated always as (…) stored` clause, or `''` for every ordinary column — so a description
+ * written before this field existed emits the statement it always emitted, byte for byte.
+ *
+ * The two rules Postgres has about the pair are refused HERE, where the entity is still named, for
+ * the reason `createIndex` refuses a unique GIN in the same file: an unguarded generator writes DDL
+ * whose first reader is `ROLE=migrate`, and the server's message carries none of the declaration's
+ * words. A column may not be both DEFAULTED and GENERATED (`42601` — a generated column's value IS
+ * its expression), and an empty expression is not one.
+ */
+export function generatedClause(column: ColumnDescriptionLike): string {
+  const expression = column.generated;
+  if (expression === undefined || expression === null) return '';
+  assert(
+    !column.hasDefault,
+    `column "${column.column}" is declared both generated and defaulted, and Postgres has neither`,
+    `drop the default from "${column.property}" — a generated column's value is its expression, computed on every write`,
+  );
+  assert(
+    expression.trim().length > 0,
+    `column "${column.column}" is generated by an empty expression`,
+    `give "${column.property}" an expression, or drop the generated declaration`,
+  );
+  return ` generated always as (${expression}) stored`;
+}
+
+export const isGenerated = (column: ColumnDescriptionLike): boolean =>
+  typeof column.generated === 'string';
+
+/**
+ * A generated column whose TYPE or whose EXPRESSION moved, brought into line without rebuilding it.
+ *
+ * `set expression as (…)` (Postgres 17) rewrites the table and recomputes every row, and the
+ * column's indexes survive — measured. Drop-and-recreate was the alternative and is worse in two
+ * ways that matter: dropping the column takes its indexes with it and nothing in this diff puts
+ * them back, and `alter table … drop column` is what `destructive.ts` reads as a data loss, so
+ * every expression change would have carried `-- destructive: true` on a migration that loses
+ * nothing. A marker on a migration that destroys nothing is a marker reviewers learn to ignore.
+ *
+ * The retype carries no `using`: Postgres refuses one on a generated column outright ("column … is
+ * a generated column"), which is exactly the statement `retypeColumn` emits for every other column
+ * — and there is nothing to convert, because the expression produces the new type itself.
+ *
+ * Two transitions this cannot express, and both are refused rather than half-emitted:
+ * plain → generated (there is no `set expression` for a column that has none) and a column whose
+ * recorded expression is unknown. `drop expression` is the one that HAS a statement, and it is the
+ * generated → plain direction, which keeps the values it computed.
+ */
+export function regenerate(
+  table: string,
+  column: ColumnDescriptionLike,
+  wantedType: string,
+  recorded: ColumnDescription,
+  plan: Plan,
+): Regeneration {
+  const wanted = column.generated ?? null;
+  const held = recorded.generated ?? null;
+  if (wanted === null && held === null) return 'unchanged';
+  // Generated -> plain: the column keeps every value it computed and simply stops being derived.
+  if (wanted === null) {
+    plan.up.push(`${alterColumn(table, column.column)} drop expression;`);
+    plan.down.push(`${alterColumn(table, column.column)} set expression as (${held ?? ''});`);
+    return 'altered';
+  }
+  // Plain -> generated: `set expression` needs a column that already has one, so this is the whole
+  // column again. Reported as `rebuilt` so the caller can put the indexes back — an `add column`
+  // implies none of them.
+  if (held === null) {
+    plan.up.push(
+      `alter table "${table}" drop column "${column.column}";`,
+      `alter table "${table}" add column "${column.column}" ${wantedType}` +
+        `${generatedClause(column)}${column.notNull ? ' not null' : ''};`,
+    );
+    // Pushed forwards and read backwards — `down` is reversed at assembly.
+    plan.down.push(
+      `alter table "${table}" add column "${column.column}" ${recorded.dataType};` +
+        ' -- was not a generated column',
+      `alter table "${table}" drop column "${column.column}";`,
+    );
+    return 'rebuilt';
+  }
+  let moved = false;
+  if (recorded.dataType !== wantedType) {
+    plan.up.push(`${alterColumn(table, column.column)} type ${wantedType};`);
+    plan.down.push(`${alterColumn(table, column.column)} type ${recorded.dataType};`);
+    moved = true;
+  }
+  if (held !== wanted) {
+    plan.up.push(`${alterColumn(table, column.column)} set expression as (${wanted});`);
+    plan.down.push(`${alterColumn(table, column.column)} set expression as (${held});`);
+    moved = true;
+  }
+  return moved ? 'altered' : 'unchanged';
+}

package/src/introspect.ts

@@ -14,6 +14,15 @@ export interface ColumnDescription {
   readonly nullable: boolean;
   readonly default: string | null;
   readonly position: number;
+  /**
+   * The generation expression, as the SNAPSHOT spells it. Absent for an ordinary column and absent
+   * for every row this module reads out of the live catalog — deliberately: Postgres stores its own
+   * rewriting of the expression (`COALESCE(title, ''::text)` for `coalesce("title", '')`), so a
+   * catalog value could never compare equal to a generated one, and drift would report a correct
+   * database forever. Both sides of the diff that DOES read it — `x db gen`'s — are generated
+   * spellings, which is the same rule `IndexDescription.where` states one field down.
+   */
+  readonly generated?: string | undefined;
 }
 
 export interface IndexDescription {