@jarenjs/db 0.87.0 → 0.89.0

package/ARCHITECTURE.md

@@ -951,16 +951,32 @@ back so the retry plans the same statements again.
 
 ## The migration engine, relationally (`src/migrate.js`, `src/cli.js`)
 
-The strategy-table diff renders SELF-CONTAINED steps — additive
-columns, data steps, and one rebuild implementation following
-SQLite's documented twelve-step procedure with `foreign_key_check`
-inside the transaction. Shape EQUALITY (schemaShapeOf versus a fresh
-createModelShape build) is the acceptance criterion, asserted on the
-shadow before the real database is touched and again after. Probed
-and designed around: node:sqlite enables foreign keys BY DEFAULT (the
-pragma bracket is load-bearing), `jsonb()` PARSES its argument
-(column folds pass plain SQL values), and a table rename does not
-rename columns (join tables rename their endpoint keys explicitly).
+The strategy-table diff renders managed hybrid-model steps. Physical
+declaration changes remain an explicit planning-policy refusal; reviewed
+`table` steps delegate to `table-migration.js` with their entire source,
+checksum, storage, identity and object guards. `migrate.js` owns the one
+history/receipt transaction for both structural and data steps.
+
+`physical-transform.js` reuses entity codecs and the document-step kernel
+for bounded column reads and changed-only writes. Each step can select a
+historical model. Keys remain fixed; generated/default ownership and
+unchanged raw storage survive. Text cursor bytes are checked against the
+native binding before exposing their batch; unsupported round-trips refuse.
+
+`migration-target.js` owns connection acquisition/cleanup and complete
+physical target acceptance for apply, repeated startup and status.
+Application-owned inventories include retained programs while unrelated
+tables stay outside the selected scope. `schema-sql.js` is the single
+literal-aware comparison owner: physical order is strict, and safe managed
+named-column comparison opts into a narrow relaxation. Exact source
+snapshots and checksums never use relaxed comparison.
+
+`foreign-key-scope.js` brackets FK/legacy settings around the driver's
+IMMEDIATE transaction. Nested work shares savepoints. Owned handles close
+even after initialization failure; borrowed synchronous work retains its
+value boundary. `shadowFixture` initializes a disposable historical schema
+before replay calls the same `migrate` and receipt executor, with recursive
+replay disabled. Every selected target is accepted before its receipt.
 
 ## One cross-runtime seam worth remembering
 
@@ -1176,8 +1192,9 @@ flattening join this native subset; strict binding failures stop before a scan.
 read classifier's tokenizer and the driver's scope owner. Writes invalidate all
 clean tracked entities; pending edits and incomplete capture populations refuse.
 `invariants.js` uses the shared Query compiler for store rules; the dialect lowers
-a bounded database subset into ordered trigger bodies. `migrate.js` reuses its
-existing rebuild/receipt transaction and verifies preservation before publication.
+a bounded database subset into ordered trigger bodies. `migrate.js` composes
+guarded table plans and codec-aware transforms in its existing receipt
+transaction, checking preservation and complete selected targets before publication.
 The backup publisher uses Node online backup and Bun disk-backed VACUUM INTO.
 The standalone snapshot helper shares destination reservation and cleanup across
 both hosts, including transaction failures.

package/README.md

@@ -1432,10 +1432,18 @@ and read-only views have explicit contracts in [MODEL-FORMAT](docs/MODEL-FORMAT.
 Inside `store.transaction`, `tx.sql.prepare(text, { access: 'read' | 'write' })`
 shares the entity/outbox connection and savepoint owner. Statements expire with
 the scope. [The client recipe](../linq/docs/DB-CLIENT.md#trusted-sql-during-adoption)
-documents trust, invalidation and synchronous execution. Schema changes use
-`planPhysicalMigration` and `migrate`; [preservation and forward recovery](docs/MIGRATION-FORMAT.md#existing-physical-files-and-forward-recovery)
-require explicit dispositions and assertions. `planInvariants` supplies declared
-SQLite constraint/audit triggers for installation through that migration boundary.
+documents trust, invalidation and synchronous execution. Schema changes compose
+a complete saved `planTableMigration` artifact as a guarded `table` step through
+`planPhysicalMigration` and `migrate`. Optional transforms/assertions carry their
+historical `model`; a reviewed `physicalTarget` checks complete owned objects,
+including on repeated startup. A borrowed `{ connection }` stays open and can
+finish synchronously with `shadow: false`; an owned `{ driver, path? }` returns
+a Promise and closes acquired resources on every path. Populated `shadowFixture`
+replay uses the same guarded executor and history owner. Start with the
+[runnable physical lifecycle](docs/MIGRATION-FORMAT.md#runnable-physical-lifecycle).
+Changed physical model diffs remain a specific `JD0021` policy refusal;
+structural changes are explicitly reviewed. `planInvariants` supplies declared
+SQLite constraint/audit triggers for installation through that boundary.
 
 Native column reads and bounded mutation documents are specified in [NATIVE-PLANS](docs/NATIVE-PLANS.md), including SQL census coverage, resource accounting and refusals.
 

package/docs/MIGRATION-FORMAT.md

@@ -46,11 +46,14 @@ shape change is a **transformation of values**, not a table rebuild.
   compiled JSLT stylesheet, in batches, inside the migration's
   transaction. The empty stylesheet (`[]`) is the identity transform.
   Over an ENTITY table the stylesheet sees the whole row — the mapped
-  columns merged into the document under the TARGET model's mapping —
-  and what it returns is split back into columns and document by that
-  mapping; the key member is kept from the row (a stylesheet that
-  omits it loses nothing) and a stylesheet that changes it is
-  `JD0023`. A step carrying `"draft": true` is a planner placeholder
+  columns merged into the document under `step.model`, when supplied,
+  or the target model otherwise. A hybrid row is split back into columns
+  and document by that mapping. A column-only physical entity is read
+  and written through its declared codecs, without a `doc` column.
+  Key members omitted by the stylesheet are retained; a changed key is
+  `JD0023`. Physical transforms assign only changed writable columns,
+  preserve omitted database-default/generated values, and refuse changed
+  generated fields, read-only views and unknown output fields. A step carrying `"draft": true` is a planner placeholder
   and MUST refuse to run (`JD0021`) until the author fills it in.
 - `kind: "query"` is an assertion: the query runs over the
   collection's documents and MUST answer an empty sequence (`expect:
@@ -68,6 +71,15 @@ shape change is a **transformation of values**, not a table rebuild.
   dry run always prints it with its note.
 - `kind: "rebuild"` is the entity restructure of §10, self-contained:
   the `CREATE` of the new shape, the copy and the index DDL.
+- `kind: "table"` carries `{ plan }`, the COMPLETE saved artifact from
+  `planTableMigration`. It delegates to the guarded table executor,
+  retaining source/checksum, row/storage, identity and object checks.
+  `statements` and `finish` are review output; flattening them into DDL
+  steps loses those guards and is not an equivalent migration.
+- `jslt` and `query` may carry an immutable `$model` 0.1 `model` describing
+  the layout at that step, including names absent from the final model.
+  An assertion before structural DDL can use the old model, and one after
+  it can use the new model. The selected mapping is verified before reads.
 - Steps are ordered, and the order is the contract.
 
 ### 2.1 Derived spatial columns and the backfill
@@ -140,6 +152,12 @@ recorded repair. The stored-column mapping does not need this repair.
 
 ### Model differences
 
+`planModelMigration` returns an empty migration for identical physical
+models. A changed physical declaration remains a specific `JD0021` policy
+refusal: use `planTableMigration` against the live schema and compose its
+saved artifact through `planPhysicalMigration`. The model diff does not
+infer application-owned DDL, key rewrites or business backfills.
+
 `planMigration(fromModel, toModel, { dialect, id, derived })` produces
 `{ migration, report }` by diffing the two models' PHYSICAL plans. The
 from-model is the previous model — the previous model FILE, or, under
@@ -189,25 +207,43 @@ so the previous shape lives beside the code, where a diff can read it.
 
 ## 4. The shadow database
 
-Before the real store is touched, the WHOLE chain — the baseline
-shape, every applied migration, every pending migration — replays on a
-shadow database (`:memory:` by default; free with SQLite, no server).
-The shadow proves **structure**: every DDL statement runs, every
-stylesheet and assertion compiles and executes, and the end shape is
-verified against the target model. A failure there leaves the real
-store untouched.
-
-The shadow runs over an empty data set; the real-data facts (the
-widening check, key consistency, the assertions over real rows) run on
-the real store inside its transaction. Batched transforms and validation
-include every row identity, including negative integer primary keys;
-target-schema validation visits every declared entity, even after an
-empty entity table.
-The shadow registers the same functions as the real run:
-`migrate(…, { registerFunctions })` runs on
-the shadow, the real and the reference connections before any DDL
-(§10), so a hand-created index over a registered deterministic
-function neither fails the shadow nor is silently dropped by it.
+When pending work exists and `shadow` is not `false`, the whole chain
+replays on a disposable shadow before real migration writes. The default
+initializer builds the baseline model into an empty database; a supplied
+`shadowFixture(connection)` instead creates the historical schema and
+fixture rows. Physical preservation plans require that initializer or an
+explicit `shadow: false`, because a query mapping cannot recreate all
+application-owned programs.
+
+Replay calls the SAME `migrate` executor with the shadow connection borrowed
+and recursive replay disabled. Guarded table steps, per-step mappings,
+preservation checks, target acceptance and history receipts all run there.
+`shadowDriver` defaults to the owned target's driver; borrowed targets need
+an explicit independent driver. `shadowPath` defaults to `':memory:'` and
+must name a disposable database independent of the primary. The acquired
+shadow closes after success, initialization failure or replay failure.
+
+Before shadow registration, fixture callbacks or replay, Node and Bun compare
+the opened files by device and inode, rejecting relative-path, dot-path, symlink
+and hardlink aliases of the primary, including borrowed primary connections.
+Separate private `:memory:` connections are allowed. An injected SQLite driver's
+optional `databaseIdentity(connection)` hook can provide the same guarantee;
+without it, the runner compares SQLite's canonical main filenames. Such a custom
+driver owns alias detection beyond that filename comparison. An opener returning
+the actual primary handle is refused without closing it.
+
+Fixture initialization is a callback outside the saved migration artifact.
+Use synthetic or appropriately isolated fixtures; do not put application
+row snapshots into a shared plan. Empty default replay proves structure;
+populated replay also exercises its fixture facts. Real-data validation
+and assertions still run inside the primary migration transaction.
+Batched work includes negative integer keys and visits every declared
+entity, even after an empty entity table.
+
+`registerFunctions` runs on the primary, shadow and independent reference
+connections before their DDL. No replay or fixture initialization is needed
+when the full chain is already applied; a supplied complete target is still
+checked at repeated startup.
 
 ## 5. History and checksums
 
@@ -227,7 +263,23 @@ hash of the `baseline` model when no migration has run.
 
 ## 6. Running and batching
 
-`migrate({ driver, path }, migrations, options)`:
+`migrate(target, migrations, options)` accepts two distinct ownership forms:
+
+| Target | Ownership and return boundary |
+|---|---|
+| `{ driver, path?, busyTimeout? }` | Opens and closes its own connection; returns a Promise. Acquired resources close even if function registration or initialization fails. |
+| `{ connection }` | Borrows an existing driver connection and leaves it open on success or failure; returns a value or Promise. Synchronous work with `shadow: false` stays synchronous when its connection and hooks do. |
+
+A borrowed target cannot also carry `driver`, `path` or `busyTimeout`.
+`migrationStatus` uses the same ownership forms. Borrowed model-only status
+comparison needs `shadowDriver`; a complete `physicalTarget` needs no
+fresh reference database. For a nested rebuild, enter
+`withForeignKeysSuspended(connection, callback)` before the migration so
+the driver can bracket FK settings outside the transaction and use nested
+savepoints inside it. Each acquired connection has one cleanup owner;
+if the operation and cleanup both fail, both errors are retained.
+
+The run options include:
 
 - `options.baseline` (REQUIRED) — the model the store was FIRST
   created with: the chain's anchor and the shadow's starting shape.
@@ -235,8 +287,18 @@ hash of the `baseline` model when no migration has run.
   last pending migration's `to` MUST equal its shape hash (`JD0020`
   otherwise), the physical end shape is verified, and the real-data
   validation of §3 runs.
-- `dryRun: true` prints every statement and the affected document
-  counts, validates the chain on the shadow, and writes NOTHING — not
+- `physicalTarget: { objects, tables? }` supplies the complete reviewed
+  application schema, using `readSchema(reference, { tables }).objects`.
+  It overrides the last plan's saved `physical.target` for final acceptance
+  and is also checked when there are no pending migrations. Each pending
+  plan's selected target is checked before its receipt is inserted.
+- `shadowFixture`, `shadowDriver` and `shadowPath` configure the disposable
+  replay described in §4.
+- `dryRun: true` prints every statement and current row counts by transform
+  collection. A count is `null` when that step's mapped table/view does not
+  exist yet. Counts describe the current database, not predicted affected
+  rows after pending SQL runs. It validates the chain on the shadow unless `shadow: false`, and
+  reports whether shadow validation ran. It writes NOTHING — not
   even the history table: it PROBES for one and reads an absent one as
   an empty history, so a dry run may be pointed at a production
   database and leave its file byte-identical. The API default is to
@@ -454,16 +516,16 @@ literally, OUTSIDE the transaction (inside one the pragma is a no-op):
 table could not even be dropped, so a migration holding a rebuild
 step turns enforcement off before `BEGIN IMMEDIATE` and back on after
 it settles, and `foreign_key_check` inside the transaction provides
-the guarantee the bracket suspended; (2) triggers and views are not re-created because this
-store creates none — a hand-added trigger is outside the model and
-outside the diff, which drift (§12) will name.
+the guarantee the bracket suspended; (2) this hybrid-model rebuild does
+not recreate arbitrary application triggers or views. Existing physical
+programs use reviewed `table` steps and preservation dispositions instead.
 
-**Shape equality is the acceptance criterion.** After a rebuild —
-after ANY relational migration — the database's declared schema
-(`schemaShapeOf`) must equal what a fresh `createModelShape(toModel)`
-produces, indexes, foreign keys and constraints included. The shadow
-asserts it before the real database is touched, and the real run
-asserts it again after the last migration.
+**Shape equality is the acceptance criterion.** A managed hybrid-model
+migration compares the database with a fresh `createModelShape(toModel)`,
+including indexes, foreign keys and constraints. Physical mappings verify
+their declared columns and invariants; complete application schema acceptance
+requires a reviewed `physicalTarget`, including retained programs. The
+shadow and primary use the same selected acceptance rule before publication.
 
 **UDF-expression indexes.** An index over a registered deterministic
 function is invisible to any connection that has not registered the
@@ -587,13 +649,31 @@ jaren-db documents --migrations <dir> --in <file|-> (--out <file|-> | --in-place
 
 ## 12. Drift
 
-Drift is the database not matching what its history says it should
-be: someone changed it by hand. `status`/`check` detect it by
-verifying the current model's physical shape against the actual
-database (`schemaShapeOf` against `createModelShape`, when the chain
-is fully applied) — a hand-added index, a dropped column or a foreign
-key edited outside a migration is named early, which is the
-difference between a puzzled afternoon and a five-minute fix.
+Drift is a database that disagrees with its expected schema. Once the
+chain is fully applied, `migrationStatus` compares the explicit
+`options.physicalTarget`, or the last migration's saved `physical.target`.
+Without either target, `options.model` enables comparison with a fresh
+model build. With neither a target nor a model it reports history only.
+An already-applied `migrate` call also checks a supplied complete target.
+
+`comparableDeclaredSql` is the single conservative declaration comparator;
+`normalizeDeclaredSql` and `schemaShapeOf` delegate to it. Ordinary SQLite
+token whitespace, comments and a CREATE-prefix `IF NOT EXISTS` are
+formatting. String contents, quoted identifiers and escapes retain their
+bytes. Quote style and SQL/identifier case are not silently canonicalized.
+Physical column order, constraint order, index term order/collations/
+predicates and trigger programs remain significant. Malformed or unsupported
+lexical input refuses with `TypeError`; this is not a general SQL-equivalence
+proof.
+
+Managed named-column comparison explicitly selects
+`{ columnOrder: 'ignore' }`. It may reorder ordinary column declarations,
+but retains table constraints and falls back to strict order for unfamiliar
+forms or inline `DEFAULT`, `CHECK`, `UNIQUE`, `REFERENCES`, `COLLATE` and
+`CONFLICT` clauses. Even static defaults retain order; expression purity is
+not inferred. Direct public comparison and
+complete physical targets default to `{ columnOrder: 'preserve' }`.
+Exact saved source snapshots and checksums never use this relaxed policy.
 
 Down migrations REMAIN a non-goal (§7's reasoning is unchanged): a
 down migration is a data-loss generator wearing a seatbelt; recovery
@@ -601,34 +681,143 @@ is a backup restored plus the forward chain.
 
 ## Existing physical files and forward recovery
 
-`planPhysicalMigration(connection, fromModel, toModel, options)` records the
-source schema and explicit DDL/SQL/rebuild steps as an ordinary migration document.
-`options` supplies an `id`, `steps`, a disposition for every source `type:name`
-(`preserve`, `replace`, or `drop`), and optional `{ sql, params, expected }`
-preservation assertions. Assertions are SELECTs evaluated before and after the
-steps. They should cover committed identities, exact BLOB hex and application
-history facts. Unknown objects cannot disappear without a declared disposition.
-Automatic hybrid model diffing refuses column layouts; an explicit plan is required.
-
-Apply through `migrate(target, [plan], { baseline, model, shadow: false })`.
-A physical plan must be qualified on an explicit backup and fresh-target fixture;
-an empty model-generated shadow cannot recreate the original file's application
-programs. The runner uses its existing immediate transaction, ordered steps,
-checksummed receipt and FK checks. It checks the source schema before destructive
-steps and preserved objects and assertions before publication. Target mapped
-columns and declared invariant triggers are verified. A changed source is
-`JD0020`, a lost object/fact is `JD0023`, and an edited applied receipt is `JD0022`.
-
-Migration history is created inside the applying transaction only when needed.
-An identical second run executes no DDL or DML. Failed steps and failed commits
-roll back; after a process kill SQLite recovery leaves the source or the committed
-target. Re-running resumes from committed receipts. Forward repair plans start
-from the newest file, including later application edits; restoring an older
-backup does not qualify as forward repair.
+`planPhysicalMigration(connection, fromModel, toModel, options)` returns
+a migration document or Promise, preserving the supplied step types in its
+`PhysicalMigrationDocument<Steps>` declaration. Its `options` contain `id`,
+ordered `steps`, and a disposition for EVERY observed application source
+`type:name`: `preserve`, `replace` or `drop`. Steps may be `ddl`, `sql`,
+`rebuild`, guarded `table`, `jslt` or `query`. Engine-owned metadata is
+excluded from source snapshots; arbitrary user objects are not.
+
+Optional `{ sql, params, expected }` preservation assertions are SELECTs
+checked before and after the steps. Use them for portable committed facts
+that must survive. A changed exact source is `JD0020`, a lost preserved
+object/fact or mismatched target is `JD0023`, and an edited applied
+migration is `JD0022`. SQL steps cannot take transaction or connection
+ownership; each step carries one statement or one complete trigger program.
+
+`options.physicalTarget` is copied into `physical.target`. Its `objects`
+must be the complete declarations for its owned scope: tables/views and
+their indexes/triggers, each with `{ type, name, owner, sql }` and SQL text.
+`tables` defaults to object owners. Include an absent table name explicitly
+when the plan requires a dropped table to remain absent. Extra objects
+attached to owned tables are drift; unrelated tables outside the scope
+are allowed. Keep the inventory scoped with `readSchema(reference,
+{ tables: [...] })`; never silently discard unknown objects within it.
+A physical model's column mapping alone is not this complete target.
+
+### Runnable physical lifecycle
+
+Run this ES module in a project with `@jarenjs/db` and `@jarenjs/linq`
+installed, using Node 24 or Bun. All databases are disposable. The example
+uses aliased physical columns, a historical transform before a guarded
+rebuild, a target fixture, populated replay and checked repeat startup.
+The saved document contains schema and instructions; the synthetic rows
+remain in the initializer. In an application, persist and review that
+document before applying it to the intended connection.
+
+```js
+import assert from 'node:assert/strict';
+import { defineModel, object, integer, string } from '@jarenjs/linq/model';
+import { defineMigration, fromPlanned } from '@jarenjs/linq/migration';
+import { planTable, planTableMigration, applyTableMigration, planPhysicalMigration,
+  readSchema, migrate, migrationStatus } from '@jarenjs/db';
+
+const driver = process.versions.bun
+  ? (await import('@jarenjs/db/bun')).bunDriver()
+  : (await import('@jarenjs/db/node')).nodeDriver();
+const columns = {
+  id: { name: 'item_id', codec: 'integer', null: 'reject' },
+  value: { name: 'label', codec: 'text', null: 'reject' },
+};
+const before = defineModel({ entities: {
+  Item: object({ id: integer().key(), value: string() })
+    .physical({ table: 'items', columns }),
+} });
+const after = defineModel({ entities: {
+  Item: object({ id: integer().key(), value: string(), revision: integer() })
+    .physical({ table: 'items', columns: { ...columns,
+      revision: { name: 'revision', codec: 'integer', null: 'reject', default: 'database' },
+    } }),
+} });
+const oldTable = { name: 'items', primaryKey: ['item_id'], columns: [
+  { name: 'item_id', type: 'INTEGER', identity: 'autoincrement', nullable: false },
+  { name: 'label', type: 'TEXT', nullable: false },
+] };
+const newTable = { ...oldTable, columns: [...oldTable.columns,
+  { name: 'revision', type: 'INTEGER', default: 1, nullable: false },
+] };
+
+// Disposable synthetic data belongs to the fixture, outside the saved plan.
+function initializeHistorical(connection) {
+  for (const sql of planTable(oldTable).createSql) connection.exec(sql);
+  connection.exec("INSERT INTO items VALUES(1,'example'); INSERT INTO items VALUES(99,'retired'); DELETE FROM items WHERE item_id=99");
+  connection.exec('CREATE TABLE unrelated(note TEXT)');
+}
+
+const connection = await driver.open(':memory:');
+try {
+  initializeHistorical(connection);
+  const tablePlan = planTableMigration(connection, newTable, {
+    id: 'items-revision', allowRebuild: true,
+  });
+  // Review and save the complete artifact, including its guards and checksum.
+  const savedTablePlan = JSON.parse(JSON.stringify(tablePlan));
+  const reference = await driver.open(':memory:');
+  let physicalTarget;
+  try {
+    initializeHistorical(reference);
+    applyTableMigration(reference, savedTablePlan);
+    physicalTarget = {
+      objects: (await readSchema(reference, { tables: ['items'] })).objects,
+      tables: ['items', 'retired_items'], // retired_items must remain absent
+    };
+  }
+  finally { await reference.close(); }
+
+  const steps = defineMigration({ id: 'items-revision', from: before, to: after })
+    .assert('Item', (row) => row.value.isEmpty(), { model: before })
+    .transform('Item', (row) => ({ id: row.id, value: row.value.upper() }), { model: before })
+    .step({ kind: 'table', plan: savedTablePlan })
+    .assert('Item', (row) => row.revision.lt(1), { model: after }).document.steps;
+  const inventory = await readSchema(connection);
+  const planned = await planPhysicalMigration(connection, before, after, {
+    id: 'items-revision', steps, physicalTarget,
+    dispositions: Object.fromEntries(inventory.objects.map((item) =>
+      [`${item.type}:${item.name}`, item.name === 'items' ? 'replace' : 'preserve'])),
+  });
+  const migration = fromPlanned(planned, { from: before, to: after }).document;
+  const savedMigration = JSON.parse(JSON.stringify(migration));
+  const options = { baseline: before, model: after,
+    shadowDriver: driver, shadowFixture: initializeHistorical };
+  assert.deepEqual((await migrate({ connection }, [savedMigration], options)).applied, ['items-revision']);
+  assert.deepEqual({ ...connection.prepare('SELECT * FROM items').get([]) },
+    { item_id: 1, label: 'EXAMPLE', revision: 1 });
+  connection.exec('CREATE TABLE later_unrelated(note TEXT)');
+  assert.deepEqual((await migrate({ connection }, [savedMigration], options)).applied, []);
+  assert.equal((await migrationStatus({ connection }, [savedMigration])).upToDate, true);
+  assert.equal(Number(connection.prepare("INSERT INTO items(label) VALUES('later')").run([]).lastInsertRowid), 100);
+}
+finally { await connection.close(); }
+```
+
+The table executor preserves row counts, unchanged values/storage classes,
+keys/rowids, retained programs and AUTOINCREMENT high-water marks. The
+migration executor owns its existing IMMEDIATE transaction and receipt;
+it checks the selected target before recording that receipt. A successful
+repeat executes no migration DDL/DML and returns `upToDate: true`; an
+explicit target still detects drift. The borrowed handle stays open.
+
+Failures in steps, target checks, receipt writes or COMMIT roll back the
+active migration. SQLite process-recovery tests establish source-or-committed-
+target recovery after interruption, not power-loss durability. Earlier
+committed migrations remain committed. Re-running resumes from receipts;
+forward repair plans start from the newest file, including later application
+edits, rather than replacing it with an older backup.
 
 `backupTo()` publishes a sibling temporary only after a complete snapshot.
-Node uses online backup. Bun uses its native serialized SQLite snapshot under
+Node uses online backup. Bun writes a disk-backed `VACUUM INTO` snapshot under
 the store gate, then flushes and atomically renames through the same publisher.
-Bun holds a full database image in memory and cannot offer page-granular copy
-cancellation. Both snapshots include committed WAL; interruption before rename
+Bun does not build a full database image in JavaScript memory; it cannot offer
+page-granular copy cancellation. Both snapshots include committed WAL; interruption before rename
 leaves the previous destination valid, while a leftover temporary is not published.

package/docs/MODEL-FORMAT.md

@@ -353,8 +353,8 @@ a control character, on SQLite) is `JD0004`.
 **Opening an existing database verifies, never alters.** If a declared
 collection's table already exists it MUST match what the model would
 create; any disagreement is `JD0002` naming the first difference.
-Reshaping a live database is the migration story — a later capability —
-and `openStore` MUST NOT attempt it.
+Reshaping a live database requires an explicit migration;
+`openStore` MUST NOT attempt it.
 
 "Match" means every physical property that decides behaviour, not just
 the names and types:
@@ -376,10 +376,14 @@ the names and types:
 - an index the database has and the model does not declare is also
   drift: it changes deletion semantics and the plans the optimizer picks.
 
-Physical column ORDER is deliberately **not** drift. SQLite's
-`ALTER TABLE … ADD COLUMN` can only append, so a migrated table and a
-freshly built one legitimately disagree there, and this store never reads
-a column positionally.
+Managed named-column comparison explicitly permits safe column reordering:
+SQLite's `ALTER TABLE … ADD COLUMN` appends, and these consumers address
+columns by name. The comparison retains constraint order and stays strict
+for inline defaults, other order-sensitive clauses or unfamiliar forms. Complete reviewed
+physical targets and direct `schemaShapeOf`/`comparableDeclaredSql` calls
+preserve physical column order by default. Literal and quoted-identifier
+bytes always remain significant; whitespace inside them is never formatting.
+See [the comparison policy](MIGRATION-FORMAT.md#12-drift).
 
 ### 3.1 Derived columns, and the capability branch
 
@@ -2374,6 +2378,34 @@ Capture/live/replication for adopted application triggers is not qualified and
 is refused, rather than advertised as a complete change stream. PostgreSQL
 column adoption is not qualified; physical inventory remains available.
 
+### Physical migration rows and acceptance
+
+Physical mappings can participate in migration `jslt` and `query` steps.
+The step's optional `model` selects its current layout; absent it, the
+final model is used. The runner verifies that layout before reading mapped
+columns. Transforms keep scalar/composite keys fixed and assign only changed
+writable columns, preserving unchanged raw text/JSON/BLOB storage and avoiding
+unrelated `UPDATE OF` triggers. Omitted database-default/generated values
+remain owned by the database. Changed generated fields, read-only writes and
+unknown output fields refuse.
+
+Migration table scans use bounded pages with text, integer or bigint key
+codecs; view assertions use bounded offset pages, and views remain read-only.
+This internal migration cursor does not enable the public physical `page`/
+`after` APIs. Text keys must round-trip through the database encoding and
+native binding. Malformed text refuses `JD0021` before its batch is exposed,
+rolling back the active migration. Node supports leading-BOM text keys.
+Bun 1.4 strips a leading BOM when binding a cursor parameter; the migration
+reader detects that limitation and refuses those keys with `JD0021`.
+That is a binding limitation, not a claim that ordinary reads corrupt them.
+
+A mapping covers declared fields and invariants, not every schema object
+an application owns. For complete startup acceptance, supply a reviewed
+`physicalTarget` inventory of the intended owned tables and their programs.
+Changed physical declarations are a `JD0021` policy refusal in
+`planModelMigration`; use an explicit guarded table plan and the existing
+migration lifecycle. See [the runnable lifecycle](MIGRATION-FORMAT.md#runnable-physical-lifecycle).
+
 ## 13. Persistence invariants
 
 An entity may declare `invariants`: each has `name`, `on` (insert/update/delete),

package/docs/SQLITE-RELATIONAL.md

@@ -135,8 +135,10 @@ Entity `.physical(...)` metadata can also supply column `type`, `defaultValue`,
 `planEntity(...).createSql` now emits explicit table DDL for complete writable
 declarations; views and incomplete generated/default definitions remain adoption
 metadata. `openStore` still verifies physical tables without creating them.
-Identical physical models produce an empty `planModelMigration`; changed
-physical tables use the live-schema `planTableMigration` API.
+Identical physical models produce an empty `planModelMigration`. Changed
+physical declarations remain a specific `JD0021` policy refusal; use the
+live-schema `planTableMigration` API and review the resulting structural plan.
+The model diff does not infer application-owned DDL or business backfills.
 
 ## Guarded upgrades
 
@@ -155,6 +157,8 @@ unchanged values/storage classes, drops/renames atomically, restores indexes and
 triggers, and checks foreign keys and the target schema. Primary-key columns,
 unshadowed hidden rowids, raw text, bytes and AUTOINCREMENT high-water marks are
 preserved. A key change, rowid-ownership change or ambiguous rowid alias refuses.
+This includes converting an `INTEGER PRIMARY KEY DESC` with an independent
+hidden rowid into an ordinary INTEGER primary key that owns the rowid.
 No migration history table or model adoption is needed.
 
 Rebuilds require SQLite's foreign-key transition outside a transaction. For a
@@ -167,6 +171,31 @@ refuses before DDL. Node/Bun regressions cover populated history and references,
 failed copies, repeat reopening, nested rollback, and process death after DROP
 with WAL recovery. These tests do not establish power-loss durability.
 
+### Composing a guarded plan with migration history
+
+Pass the complete saved table artifact as `{ kind: 'table', plan }` in
+`planPhysicalMigration` or `.step({ kind: 'table', plan })` in the migration
+pen. It retains `version`, `id`, `table`, `source`, `after`, `rebuild`,
+`temporary`, `unchanged`, `statements`, `finish` and `checksum`; incomplete
+plans refuse. Do not flatten its SQL arrays into separate DDL steps: the
+guarded executor also preserves storage classes and allocated identities.
+
+`migrate` supplies the existing history/receipt transaction around that
+executor. It admits the writer with IMMEDIATE semantics before source reads,
+uses savepoints for nested work and restores FK/legacy settings exactly.
+`{ connection }` borrows the same driver handle; `{ driver, path? }` owns its
+resource. Both compose with optional historical transforms/assertions and a
+complete `physicalTarget` copied from a disposable target fixture. With
+`shadowFixture`, replay uses the same migration and history executor.
+The [runnable example](MIGRATION-FORMAT.md#runnable-physical-lifecycle) proves
+this composition and a checked second run.
+
+Complete physical targets retain physical column and constraint order,
+quoted text, index predicates/collations and trigger programs. Only token
+whitespace/comments and CREATE-prefix `IF NOT EXISTS` are formatting;
+quote styles and case are not inferred equivalent. The separate exact
+source snapshot/checksum remains the stale-plan guard.
+
 ## Additive and object operations
 
 `planSchemaChange(connection, operation)` returns a reviewable

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@jarenjs/db",
   "private": false,
-  "version": "0.87.0",
+  "version": "0.89.0",
   "type": "module",
   "main": "./src/index.js",
   "types": "./types/index.d.ts",
@@ -108,9 +108,9 @@
     "prepack": "npm run build:types"
   },
   "dependencies": {
-    "@jarenjs/core": "^0.87.0",
-    "@jarenjs/json": "^0.87.0",
-    "@jarenjs/validate": "^0.87.0"
+    "@jarenjs/core": "^0.89.0",
+    "@jarenjs/json": "^0.89.0",
+    "@jarenjs/validate": "^0.89.0"
   },
   "bin": {
     "jaren-db": "./src/cli.js"