@happyvertical/smrt-cli 0.47.1 → 0.48.0

package/AGENTS.md

@@ -10,16 +10,18 @@ smrt doctor                  # Umbrella diagnostics; can verify a generation sna
 smrt doctor --db             # Add the live-schema parity section (see below)
 smrt db:status               # Pending schema changes + failed migration classification
 smrt db:status --parity      # Same, plus live-schema parity (see below)
-smrt db:migrate              # Apply migrations
+smrt db:orphans              # agents/db-orphans.md
+smrt db:migrate              # agents/type-drift.md
 smrt db:migrate --postgres-safe # PostgreSQL concurrent-index mode (see below)
 smrt db:migrate --force-migration <exact-id> [--force-migration <exact-id>...] # Force exact generated migrations in one atomic batch
+smrt db:migrate --apply-unblocked / --null-orphans # agents/db-migrate-partial-apply.md
 smrt db:migrate-uuid         # Convert schema-declared UUID text columns after data remap
 smrt db:migrate-int8         # Explicitly widen pre-#2373 int4 columns after preflight
 smrt db:drop-framework-base-tables # One-time drop of the five #2644-orphaned framework-base tables
 smrt db:drop-framework-base-tables --dry-run # Print the drop plan without executing
 smrt db:diff                 # Show schema differences without generating migration files
 smrt db:rollback             # Roll back migrations by executing their recorded DOWN
-smrt db:rollback --mark-only # Record-only flip; schema deliberately untouched
+smrt db:rollback --mark-only # Record-only flip; schema untouched
 smrt db:prune                # Prune framework system tables to their retention windows
 smrt db:prune --dry-run      # Same predicates, counted rather than deleted
 smrt docs:agents             # Generate .agents/smrt-framework.md
@@ -50,7 +52,8 @@ cycles are created without the cyclic clauses first, then receive named
 constraints after both tables exist. When a same-package constraint is missing
 on an existing table, `db:migrate` probes the exact child/parent columns for
 orphans before `ADD ... NOT VALID` and `VALIDATE CONSTRAINT`; orphaned data or a
-failed probe stays manual with detector/repair SQL.
+failed probe stays manual with detector/repair SQL by default
+(agents/db-migrate-partial-apply.md).
 
 `db:migrate --dry-run` and deprecated `db:setup --dry-run` print the same
 engine-specific dependency plan used for execution, including exact table DDL
@@ -326,33 +329,33 @@ drop `includeDroppedTables` (default `false`, both call sites explicit —
 ## Architecture
 
 - **Lazy command loading**: commands loaded on-demand via dynamic import (~100ms overhead on first use)
-- **Manifest discovery**: auto-finds `.smrt/manifest.json` + scans `node_modules/@happyvertical/smrt-*`
-- **Aggregate manifest identity**: preload and schema discovery resolve package ownership per entry (`definition.packageName` before the container), so dependency objects retain their qualified registrations
+- **Manifest discovery**: finds `.smrt/manifest.json` + scans `node_modules/@happyvertical/smrt-*`
+- **Aggregate manifest identity**: preload/schema discovery resolve package ownership per entry (`definition.packageName` before the container), keeping dependency objects' qualified registrations
 - **Class loading order**: config.entryPoint → package.json exports['.'] → package.json main → `./dist/index.js`
-- **Object method exposure**: custom methods on SMRT objects auto-become CLI commands
+- **Object method exposure**: custom SMRT object methods auto-become CLI commands
 
 ## Key Files
 
-- `src/cli-generator.ts` — core dispatcher, lazy command loading, class loading
+- `src/cli-generator.ts` — core dispatcher, lazy command/class loading
 - `src/commands/` — individual command implementations
 - `src/loaders/` — class-loader, local-loader, npm-loader, git-loader, template-loader
 - `src/discovery/manifest-discovery.ts` — manifest auto-discovery
-- `src/commands/docs-claude.ts` — downstream AGENTS.md generation plus Claude compatibility alias; uses core knowledge discovery without object scanning
+- `src/commands/docs-claude.ts` — downstream AGENTS.md generation + Claude alias; no object scanning
 
 ## Gotchas
 
-- **Test mode detection**: checks `NODE_ENV=test`, `VITEST=true`, `global.it`/`describe` — could conflict with other test runners
-- **External package load failures silenced**: one package failing doesn't prevent others from loading
+- **Test mode detection**: checks `NODE_ENV=test`, `VITEST=true`, `global.it`/`describe` — may conflict with other test runners
+- **External package load failures silenced**: one package failing doesn't block others
 - **Generation command names are hyphenated**: `generate-mcp` and `generate-types` have
-  no colon alias. Only `generate-routes` and `generate-register` declare one, so
-  `smrt generate:mcp` is an unknown command (#2279).
+  no colon alias. Only `generate-routes`/`generate-register` declare one, so
+  `smrt generate:mcp` is unknown (#2279).
 - **Subcommand `--version` beats the global flag**: `parseCliArgs` in
   `@happyvertical/utils` dispatches to the built-in `version` command for a
   `--version` token anywhere in argv. `parseCliCommandArgs` rescopes that token to
   the named subcommand (`smrt generate-mcp --version 0.1.0`) and only leaves it
   global when it appears before or without a subcommand (#2279). `--help` is
   deliberately untouched.
-- **Schema history nuance**: `db:status` / `db:history` should distinguish active live drift from superseded failed generated schema repairs instead of treating all failed rows as current blockers
+- **Schema history nuance**: `db:status`/`db:history` should distinguish active live drift from superseded failed repairs, not treat every failed row as a current blocker
 - **Decorator check follows the Vite major**: doctor requires `oxc.decorator` in
   vite.config on Vite 8+ and accepts tsconfig `experimentalDecorators` only
   below it. The old unconditional tsconfig check flagged correct Vite 8 projects

package/agents/db-migrate-partial-apply.md

@@ -0,0 +1,189 @@
+# `db:migrate --apply-unblocked` and `--null-orphans` (#2748)
+
+Both are opt-in and off by default; default `db:migrate` behavior is
+unchanged either way. Implementation: `packages/cli/src/commands/
+db-migrate-actions.ts` (pure partition/planning functions) and
+`packages/cli/src/commands/utilities.ts`'s `db:migrate` handler (wiring,
+I/O). Core side: `SchemaChange.orphanBlocked` / `.orphanNullable` /
+`.engineUnsupported` (`packages/core/src/schema/types.ts`), set by the
+FK advisory branches in `compareForeignKeys()`
+(`packages/core/src/migrations/differ.ts`), and
+`renderForeignKeyAddStatements()` (`packages/core/src/schema/
+foreign-key-ddl.ts`).
+
+## `--apply-unblocked`: partial apply
+
+Today's unflagged batch already excludes every individually blocked change
+(a FK-with-orphans, a manual `type_upgrade`/`alter_column`) — those never
+leave `SchemaDiff.changes` as executable SQL, so they never enter the atomic
+tracker batch on their own. `--apply-unblocked` adds a second, opt-in
+partition on top of that: it withholds not just the blocked changes
+themselves but anything that *depends* on one, and applies everything else.
+
+**Dependency rule.** `computeBlockedColumns()` reads every manual
+intervention's blocked `table.column` identity, plus every report-only
+`type_upgrade` **advisory** (the #2608 refused uuid convergence — the only
+producer of that shape) — an advisory-only finding never enters
+`manualInterventions` (it carries no SQL) but names a live column just as
+concretely, and a dependent index/alter/drop on that column is exactly as
+unsafe as one on a manual-intervention column (review finding, #2748: the
+advisory bucket was previously excluded, so `--apply-unblocked` did not
+honor its own stated guarantee for that class of blocked type upgrade).
+`partitionUnblockedMigrations()` then withholds any `add_index` /
+`add_foreign_key` / `alter_column` / `drop_column` that reads or writes one
+of those columns:
+
+- an index on a column whose type upgrade is blocked (manual intervention
+  or report-only advisory);
+- a foreign key whose child *or* parent column is blocked (a parent column
+  mid-type-upgrade is just as unsafe to reference as the child);
+- an `alter_column`/`drop_column` on the blocked column itself.
+
+**A report-only `alter_column` advisory is deliberately excluded from that
+second loop**, even though `SchemaAdvisory` allows the type. Unlike
+`type_upgrade`, the only producer of an advisory-only `alter_column` is an
+un-opted-into relaxation (`drop_default`/`drop_not_null` when
+`--relax-columns` was not passed) — it says the live column is *stricter*
+than the manifest, not that anything about the column blocks convergence,
+and it reappears on every run regardless of `--apply-unblocked`. Blocking on
+it reproduced the exact `engineUnsupported` defect for a different shape:
+permanently withholding unrelated executable DDL on that column, worse than
+omitting the flag (review finding, #2748, second pass). A genuinely blocked
+`alter_column` — NOT NULL required with live NULLs and no default to
+backfill — carries its refusal as comment-only SQL and already reaches
+`manualInterventions` (keyed by `columnName`), so it was never affected.
+
+`add_column` and an executable `type_upgrade` never depend on another
+column's state — a new column add is self-contained, and a type upgrade is
+the fix itself — so they always apply. Everything independent goes through
+the exact same transaction/tracker path an unflagged run uses.
+
+**An `add_foreign_key` blocked only because this engine cannot express the
+constraint at all is not a column-state block.** SQLite (table rebuild
+required) and DuckDB (no `ALTER TABLE ADD CONSTRAINT`) each carry
+`engineUnsupported: true` on that advisory — distinct from a conflicting
+constraint or an incompatible-type block, which genuinely do implicate the
+column and stay blocking. `blockedColumnForAction()` excludes only the
+`engineUnsupported` case: every real differ `add_foreign_key` advisory sets
+`foreignKey`, so gating on its mere presence blocked this case too (review
+finding, #2748) and, since no rerun on that engine ever resolves it, made
+`--apply-unblocked` permanently withhold unrelated dependent DDL on that
+column — strictly worse than omitting the flag.
+
+**An orphan-blocked `add_foreign_key`'s block is a row-data condition, not
+a column-state one — an `alter_column` relaxation on that same column is
+its remediation, not something unsafe against it.** With `--relax-columns
+--apply-unblocked`, a FK child column that is physically `NOT NULL` while
+the manifest declares it nullable (with live orphan rows) stays
+orphan-blocked — `--null-orphans` can't resolve a NOT NULL child. The
+executable `DROP NOT NULL` relaxation for that same column is exactly what
+makes the block eventually resolvable (the column becomes nullable, then a
+later `--null-orphans` run can null the references). Gating that
+relaxation on the orphan block (review finding, #2748) made the run
+withhold it on every pass, so the column never became nullable, the
+orphan block never cleared, and the batch failed forever — worse than
+omitting the flag. `computeOrphanOnlyBlockedColumns()` identifies a
+`table.column` whose *only* manual-intervention block is an orphan-blocked
+FK on that column, and `partitionUnblockedMigrations()` excludes that key
+from gating an `alter_column` specifically — an index, a different foreign
+key, or a `drop_column` on that same orphan-blocked column still correctly
+stays withheld.
+
+Withheld items print under `🔒 Withheld — depends on a blocked change`, each
+naming the `table.column` it depends on and the same reason text the manual
+intervention above it carries. `--dry-run` shows the identical
+applied/withheld partition (the partition is pure and runs before the
+dry-run/apply branch splits). A rerun is a no-op for what already applied
+and repeats the same withheld list until the underlying blocker is resolved
+— nothing here is remembered between runs; the partition is recomputed from
+the live diff every time.
+
+## `--null-orphans`: opt-in orphan-FK disposition
+
+Targets one specific blocked-item shape: a foreign key `db:migrate` would
+otherwise refuse only because live child rows don't match any parent row
+(`SchemaChange.orphanBlocked`). `planOrphanDispositions()` splits manual
+interventions into nullable (a disposition is possible) and not-nullable
+(unconditional refusal) using `SchemaChange.orphanNullable` — never by
+parsing `advisory.message` text.
+
+For a nullable child column: the *exact* `UPDATE ... SET <column> = NULL
+WHERE ...` the differ already generated for that relationship
+(`advisory.suggestedSql[1]` — never re-derived, so the disposition can never
+drift from what the differ decided the safe repair is) and the `ADD
+CONSTRAINT ... NOT VALID` + `VALIDATE CONSTRAINT` pair from
+`renderForeignKeyAddStatements()` (the same helper the differ's own safe-add
+branch uses) are combined into **one** `add_foreign_key` migration action —
+`sqlStatements: [repairSql, ...addStatements]` — and applied atomically
+through the normal tracker path, not run as a separate direct query before
+it (review finding, #2748: running the UPDATE outside the batch let a later
+failure elsewhere in the same batch roll back the FK add while leaving the
+just-nulled rows committed, silently splitting one promised disposition
+into two). The resolved FK moves from `manualInterventions` into the
+executable `migrations` bucket before the `--apply-unblocked` partition
+runs, so a relationship this flag resolves is never a blocked column for
+that dependency rule. Post-apply before/after counts are only printed once
+the batch that carries this disposition has actually committed; on a full
+rollback the generic atomic-failure message already covers it — nothing in
+that batch, including this disposition, applied.
+
+**The combined migration can still be withheld by `--apply-unblocked`
+itself**, if the FK's *parent* column is separately blocked (e.g. a
+report-only `type_upgrade` advisory) — resolving out of
+`manualInterventions` only means this specific relationship's *child*-side
+orphan block is gone, not that the dependency partition can't withhold it
+for an unrelated reason. `filterUnresolvedOrphanDispositions()` drops a
+pending post-apply report entry whose exact migration action ended up
+withheld, matched by object identity (the same reference is pushed into
+both `migrations` and the pending entry) rather than by table/column, so
+the report never prints a `✓ ... resolved` line for a migration that never
+executed (review finding, #2748) — that relationship is already covered by
+the `🔒 Withheld` listing.
+
+**"Committed" is checked per disposition, not for the whole batch.** In
+PostgreSQL concurrent-index mode (`--postgres-safe` with
+`migrations.postgres.useConcurrently`), the non-index batch — including
+this disposition's combined migration — commits in its own transaction
+before any deferred `CREATE INDEX CONCURRENTLY` migrations run separately
+and non-transactionally; a later index build failure still increments
+`errorCount` even though the non-index changes already committed. Gating
+the report on `errorCount === 0` alone silently suppressed the resolution
+line for a mutation that did commit (review finding, #2748). A batch-wide
+proxy for "did the non-index phase commit" is not enough either: it can't
+tell that apart from the non-index transaction itself rolling back (recall
+finding, #2748) — both raise `errorCount`, but only one means this
+disposition's own migration applied. The report instead re-derives each
+pending disposition's synthetic migration name
+(`getSyntheticMigrationNameForAction()`, the same helper that named it when
+building the tracker batch) and checks that exact name's own `success` in
+`tracker.applyAll()`'s per-migration results — the ground truth regardless
+of what else in the batch failed — printing a caveat line only when some
+other change in the batch failed but this disposition's own migration
+still succeeded.
+
+For a `NOT NULL` child column: refuse with the same "Manual repair required"
+text `db:migrate` always prints for that case. Nulling is not a legal repair
+there, and **this flag never deletes rows** either way — the only mutating
+statement is the UPDATE that nulls references, never a DELETE.
+
+`--dry-run` prints the exact UPDATE statement plus the live orphan count
+(`orphanCountSql()` wraps the differ's read-only detector SELECT as a
+`COUNT(*)`) before applying anything; a real run reports the count before
+and after. Composes with `smrt db:orphans` (#2753, `packages/cli/agents/
+db-orphans.md`) — same probe, same repair SQL, no duplicated logic.
+
+**Fails closed on the count probe too** (review finding, #2748):
+`countOrphanRows()` throws rather than swallowing a query failure into `0`.
+An orphan-count probe that cannot run at all is exactly the "unexpected
+shape" case this flag must report and withhold on — proceeding to null data
+on an unverified count would violate the fail-closed requirement. A probe
+failure withholds that one relationship's disposition (printed, left in
+`manualInterventions`) without aborting the rest of the batch or the other
+dispositions.
+
+## Fails closed on an unexpected shape
+
+`planOrphanDispositions()` routes an orphan-blocked, nullable change to the
+refusal bucket (never guesses SQL) if it is missing its `foreignKey`
+definition or its `advisory.suggestedSql` pair — a differ-side contract
+violation, not something this disposition should paper over.

package/agents/db-orphans.md

@@ -0,0 +1,46 @@
+# `db:orphans` — per-foreign-key orphan count report (#2753)
+
+`db:migrate`'s orphan probe already gates `ADD CONSTRAINT` with a `LIMIT 1`
+existence check and prints a suggested repair (see the PostgreSQL section of
+`packages/cli/AGENTS.md`); it cannot say *how many* orphans exist or across
+which relationships. `db:orphans` runs core's `collectForeignKeyOrphanCounts()`,
+which probes every manifest-declared foreign key from
+`ObjectRegistry.getAllSchemasAsDefinitions()` as a `COUNT(*)` aggregate —
+reusing `renderForeignKeyOrphanDetector()`'s exact FROM/JOIN/WHERE predicate
+via its `countOnly` option, never a second hand-written copy — and reports
+child/parent table and column, the live orphan count, and whether the child
+column is `NOT NULL` (so the two dispositions the repair-SQL builder already
+distinguishes — manual repair vs. an executable null-out — are visible up
+front, without hand-writing the same SQL against the database). Results print
+sorted by count descending. A relationship whose child or parent table does
+not exist live is skipped and listed separately rather than failing the whole
+report.
+
+Strictly read-only and diagnostic: it never repairs anything, and the orphan
+counts themselves never affect the exit code — a database full of orphans
+still exits 0, matching the existing "no foreign-key orphan repair, by
+design" boundary. It exits non-zero only on a configuration or runtime error
+(no database configured, or the report itself could not run). `--json` emits
+the full report for scripting; `--verbose` also lists foreign keys with zero
+orphans.
+
+`ForeignKeyOrphanSkipped.kind` distinguishes a benign `missing_table` skip
+(a manifest table not yet migrated) from `probe_failed` (the `COUNT(*)`
+query itself errored — permissions, a malformed live column, a genuine SQL
+failure). `smrt db:orphans` marks a `probe_failed` skip `[PROBE FAILED]` in
+its Skipped section and never reports the "No orphan rows found" clean
+summary while any probe has failed, since an incomplete scan is not the same
+claim as a clean one.
+
+`db:status` runs the same probe unconditionally (not gated behind `--parity`,
+and independent of whether the configured adapter supports
+`getTableSchema` — the probe only needs `db.query()`) and, when any manifest
+foreign key has live orphans, prints one compact line per affected foreign
+key plus a pointer to `smrt db:orphans` for the full report. A `probe_failed`
+skip surfaces as its own distinct warning (`status.orphanProbeFailures`)
+rather than being read as "no orphans found"; `missing_table` skips stay
+silent in `db:status` (benign, and already visible in full via
+`smrt db:orphans`). Never a new `SchemaDiff` finding kind, and never
+touching `db:status`'s exit code. A total probe failure (the collector call
+itself throws, e.g. it cannot list live tables at all) is reported as
+"Foreign-key orphan check unavailable" rather than failing `db:status`.

package/agents/type-drift.md

@@ -0,0 +1,53 @@
+# Float-width, timestamptz, and jsonb convergence (#2770, #2771, #2772)
+
+## Float-width drift (#2770)
+
+`db:status`/`db:diff` report float-width drift (`real` vs `double precision`)
+as `column_type_drift`, the same severity class as `legacy_integer_width`; a
+live single-precision column backed by a double-precision declaration widens
+automatically (lossless), but a double-precision live column backed by a
+single-precision declaration stays advisory-only (narrowing loses precision).
+
+## text -> timestamptz convergence (#2771)
+
+A `text` column on a table smrt itself creates, backed by a manifest
+`TIMESTAMP` (-> `TIMESTAMPTZ` on PostgreSQL) declaration, converges instead of
+staying blocked forever: `db:migrate` runs a server-side shape probe over
+every non-null value first (`packages/core/src/schema/text-cast-probe.ts`)
+and only plans the executable `ALTER COLUMN … TYPE timestamptz USING
+col::timestamptz` when every value is confirmed to parse unambiguously. A
+probe that finds an unparsable value fails closed with a masked sample
+instead of running the cast — repair the value, then rerun; the migration is
+a no-op once converged. This is independent of
+`postgresTimestampMigration.legacyTimezone`
+(`--postgres-timestamp-legacy-timezone=UTC`), which
+stays reserved for ambiguous naive wall-clock strings the probe does not
+accept. SQLite has no distinct storage for `text` vs `timestamptz`, so this
+stays advisory-only there.
+
+## text -> jsonb convergence and uuid visibility (#2772)
+
+A `text` column on a table smrt itself creates, backed by a manifest `JSON`
+(-> `jsonb` on PostgreSQL) declaration, was previously invisible to
+`db:status`/`db:diff` and had no `db:migrate` repair path (the #1335
+text/json tolerance is directional: it stays silent for the reverse pairing —
+a native `json`/`jsonb` live column backed by a text-convention manifest
+field — but this direction now runs the same shared shape probe as #2771's
+timestamptz path and converges the same way: a clean probe plans `ALTER
+COLUMN … TYPE jsonb USING col::jsonb`; a dirty probe fails closed with a
+masked sample. `db:status` also now reports an `info`-level finding for a
+declared-`uuid` structural column still backed by live `text` (previously
+silent by design), pointing at `db:migrate-uuid` — the tolerance itself is
+unchanged, only its visibility. SQLite stays advisory-only for both.
+
+## Live-only column defaults on either conversion
+
+PostgreSQL rejects `ALTER COLUMN … TYPE` outright whenever the column has
+any existing default that can't auto-cast to the target type, so both
+conversions above always `DROP DEFAULT` first when the *live* column has
+one — restoring it afterward (`SET DEFAULT`) only when the *manifest* also
+declares one. A live-only default (the manifest declares none) is the same
+"relaxation" `db:status --parity`'s default-drift finding already gates
+behind `relaxColumns`/`--relax-columns` elsewhere: without that opt-in, the
+conversion stays a fail-closed advisory naming the blocking default instead
+of executing.