@voltro/plugin-auth 0.45.0 → 0.46.0

package/CHANGELOG.md

@@ -39,6 +39,182 @@ _Changes staged for the next release accumulate here (rolled up from
 
 ---
 
+## [0.46.0] — 2026-08-22
+
+### ⚠ BREAKING
+
+- **@voltro/runtime, @voltro/cli** — An aggregate's `incremental.source` is a table name, and is now typed and audited like one.
+
+  0.45.0 narrowed a query's and a stream's `source:` for a specific failure: a name matching no table does not error, it produces a subscription that serves once and goes quiet. An aggregate's CDC source misses the same way and is quieter still — the runner subscribes to a table nothing writes, no delta arrives, and the aggregate stops tracking its input while every read of it succeeds and returns a number.
+
+  It stayed `string` in that change, and not because anyone weighed it. It reads differently — `incremental.source`, a single name on a definition, rather than a list on a descriptor — so it did not fit the loop, and a shape that does not fit reads as a type mismatch instead of a gap.
+
+  Both halves close now. The field is `TableName`, so a name no table carries is a compile error from the next `voltro dev`. And the boot audit takes aggregates alongside queries and streams, so a stale one is named in the same warning — the runtime half matters because a source can be correct at the type level and still be a table the deployment does not have.
+
+  The audit's aggregate half runs on both boot paths and needed its OWN call on each, since aggregates are discovered several hundred lines after the existing one — which is precisely the shape that ends up wired on one path only, so it is asserted by name over both files.
+
+  A recompute-only aggregate declares no `incremental` at all and is not audited: reporting an absence as an unresolved source would report a choice as a defect.
+
+  **`voltro update` carries you across this** — codemod `0.46.0/01_typed-aggregate-source`.
+
+### Added
+
+- **@voltro/data-transfer, @voltro/cli** — A foreign-key cycle is caught BEFORE a staged `replace` loads, and leftover staging tables have a command.
+
+  Two things the staged swap left open, closed.
+
+  **The cycle.** The swap inserts parents first, so two tables referencing each other cannot both be satisfied by a bulk copy on postgres, sqlite or SQL Server — and `SET CONSTRAINTS ALL DEFERRED` does not rescue it, because postgres only defers a constraint declared `DEFERRABLE` and the framework declares none. Until now that surfaced as a FAILED SWAP after the whole bundle had loaded: minutes of work, then a refusal. `topo.ts` already orders parents-first and breaks a cycle at its closing edge, so a cycle is exactly a reference pointing FORWARD in that order — cheap to see before anything is loaded. Such a run says so and takes the row-by-row path, whose deferred-FK pass exists for that shape. A table referencing ITSELF is deliberately not a cycle: one statement carries the whole table, measured on all five engines, and treating it as one would cost every app with an `audit()` mixin the staged path.
+
+  **The leftovers.** A staged run drops-then-creates, so it collects its own; what survives is staging for a table set a later run does not touch. `voltro data clear-staging --yes` lists them and drops them.
+
+  It is a COMMAND and not a boot sweep, which is the decision worth recording: a booting process cannot tell a leftover from a staging table another replica is loading into right now, and with several replicas that is not a rare race — one booting pod would delete an import in flight. The refusal without `--yes` says so, and names the tables so the operator can check before answering.
+
+  **And `--no-atomic` stages too now, which is where the change is largest.** The flag exists for resumability on a large bundle, and it used to be the mode with the WORST failure: the target emptied and partially refilled, in neither state — the kill test measured 889 rows of 8 000. Staged, the ledger keeps its exact meaning (a recorded table is one fully loaded; it just lands in staging) while the target stays untouched until the swap. Resumable AND all-or-nothing, which the two flags could not be at once before.
+
+  Three things came out of wiring it, and none was visible from the design. Dropping staging in the `finally` destroyed exactly what a resume needs, so a second run read from tables that no longer existed — found by the resume test, not by reasoning; staging is dropped only after a successful swap now, and a run that kept its rows says so. The SUCCESS case needed its own guard: after a completed run the ledger still says every table is done while staging is gone, so a re-run would have copied nothing over the target — `ledger.truncated` has always meant "the destructive step already happened" and the old path is guarded by exactly that flag, so it guards this one too. And the test lever was wrong at first: withholding the snapshot disables staging but also fail-closes the rollback capture, so the run refuses before loading — the unstaged case is now driven by the real reason, a registered write recorder.
+
+  Two existing assertions inverted, and BOTH truths are kept rather than one replaced: `--no-atomic` now keeps the target when it can stage, and still costs exactly what it always did when it cannot. A killed staged run leaves the target's rows intact and writes no interrupted-replace marker — that state cannot arise on the staged path, so there is nothing for a boot to refuse over.
+- **@voltro/runtime, @voltro/cli** — `voltro dev` says when a query reads a table it did not declare in `source:`.
+
+  The other half of the `source:` problem, and the one no static tool can see. A stale name is a compile error now, and the boot warns about one that resolves to nothing. A name that is simply ABSENT has never had an observer: the write lands, the row is in the database, a reload shows it, and the open panel does not move. The type is satisfied, the audit is satisfied, the write path is correct and its tests are green.
+
+  So while `voltro dev` runs, every read is attributed to the query that made it and compared against that query's own `source:`. The finding names the table and what will not happen, once per query per boot.
+
+  **The design question was never the recording, it was compose-vs-restrict.** Only a read that CONTRIBUTES rows belongs in `source:` — a restricting read re-running on every unrelated write puts every list back on the wire. An app that scanned its own source for this needed two hand-written exceptions to get from a thousand findings to thirty, and a rule needing an exception list on a correct codebase has already spent its attention.
+
+  Neither exception is a list here. A table reached only through a predicate subquery is narrowing BY CONSTRUCTION — it returns no column to anybody — so it classifies itself off the descriptor, on every app, with nothing to maintain. And the framework's own restricting reads are ours: they are issued below the wrapper, or, where the framework runs APP code to decide access (a row filter's loader), marked at the call site we control instead of at the ones we do not.
+
+  Deliberately narrow, and each edge is a decision rather than a limitation: it reports what it SAW and never claims a declaration is otherwise complete; a query with no `source:` at all is left alone, because the finding is about an incomplete list and not a missing one; and it says which tables it did not count, so the classification can be checked rather than trusted.
+
+  Dev only. `voltro serve` installs no sink, which makes every part of it inert — no wrapper, no async-local write, no comparison. `VOLTRO_SOURCE_RECORDER=off` turns it off in dev.
+
+  `restrictingReads` (`@voltro/runtime`) is the escape for an app helper that resolves access somewhere the framework does not call it. It is a no-op outside a recording session, so it can be left in place.
+
+  Measured against a running `voltro dev`, not only in tests: a query declaring one table while reading two is reported once across eleven requests, and the correct queries beside it produce nothing.
+
+  **Eager-loaded relations count, and they were the hole.** `.with({ subTasks: true })` issues no second read — `compileEagerJson` folds the whole spec into ONE round trip — so the loaded table is never a read's own table and never a join. It is a relation NAME on the descriptor, and the recorder resolves it through the relation registry: the target, and for a many-to-many the JUNCTION as well, since a write there changes membership, which is precisely the change a user makes. Nested `with:` recurses against the target's relations, the same walk the compiler does; an unresolvable name yields nothing rather than an invented table, and a throwing target thunk cannot take the request down with it.
+
+  This was the reported failure's own shape, so the first version of the recorder could not see the case it was built for.
+
+  Measured against a running `voltro dev` and real `POST /rpc` calls: twelve requests across four queries produced exactly two findings — the two deliberately-incomplete ones. The SAME eager load declared correctly beside them is silent, and so is a query with no `source:` at all. A check that only ever fires is not evidence that it fires for a reason.
+- **@voltro/cli** — `--rollback-key <key>` — a `replace` over `--target api` has the INSTANCE store the target's current rows in its own object storage before deleting them.
+
+  The direct path already captured beside the bundle. That is useless on this transport, and the reason is the whole point: the process that would roll a transaction back IS the instance, so a capture in the pod's filesystem goes away with exactly the failure it exists for — a deployment lost 240 172 rows to a run whose api pod disappeared nine minutes in. Object storage is durable, is already configured wherever the storage-push export works, and is reachable afterwards from anywhere.
+
+  Pushed BEFORE the first delete, through the same archive sink the export uses — one sink, not two, because two is how an instance comes to push a bundle and fail to keep one. A failure to store it stops the import with the target untouched.
+
+  Fail-closed in both directions, deliberately:
+
+  - asked for and impossible (no storage configured) → **409**, naming the fix. The request is the operator saying "I cannot afford to lose this", and serving them anyway is the one answer that removes their precaution while looking like agreement. - not asked for → the run proceeds and says what it did not keep. A `replace` into a scratch environment is legitimate, and refusing it would push people to the flag that turns the safety off everywhere.
+
+  The decision is made from the headers alone, before a byte of the bundle is read: a run that cannot take the capture it was asked for must not cost an upload first.
+- **@voltro/database, @voltro/data-transfer, @voltro/cli** — An interrupted `replace` cannot be silent any more.
+
+  The capture only helps if somebody knows to reach for it, and a half-replaced database is indistinguishable from an empty one FROM THE INSIDE — every table exists, every constraint holds, every query returns nothing without erroring. A deployment served over one for ninety minutes and only found out through an unrelated fingerprint mismatch.
+
+  So a `replace` writes one row (`_voltro_replace_in_progress`) before the first delete and removes it after the last insert, and finding it at boot is a REFUSAL — on both boot paths, out of one function. The message names how many tables, how long ago, over which transport, and the capture to restore from, with the command spelled out.
+
+  The row lives in the SAME transaction as the emptying, so it is present exactly when the emptying is: a run that rolls back cleanly takes the marker with it, and a boot over a database nothing happened to is not refused. A completed `replace` clears its own marker and every older one, so the recovery import restores the data and silences the alarm in one command.
+
+  Nothing expires — a half-replaced database does not become whole with time, so `voltro data clear-replace-marker --yes` is a decision somebody makes.
+
+  Measured against a live mariadb by killing a run mid-load: the marker is there, the refusal names the capture, and a completing run clears it.
+- **@voltro/data-transfer** — The staged-swap primitive for `replace` — load somewhere else, then swap the content in one short transaction.
+
+  `--mode replace` empties the target and loads into it inside ONE transaction, held open for the whole network-bound load. A deployment measured nine minutes for 242 950 rows, and the promise that the target is left as it was found rests entirely on a live process being there to roll it back. A promise that rests on the process surviving is a promise about the weather.
+
+  **The design the plan carried was wrong, and measurably so.** Shadow tables plus a final `RENAME` moves every inbound foreign key WITH the renamed table — postgres 17 by OID, MariaDB 11 and MySQL 8.4 by tracking the rename, including inside MySQL's atomic multi-pair `RENAME TABLE`. After the swap every key points at the table the design then DROPS. The atomicity of the rename, which that design reasoned about carefully, was never the hard part.
+
+  Keeping the table OBJECTS and swapping the CONTENT has none of that: every constraint stays pointed at the same object, and the long client-driven load moves OUT of the destructive transaction, which then holds only server-side bulk SQL.
+
+  **Four of five engines need no integrity switch, which inverts what the design assumed.** Three self-referencing rows — `actors.createdBy → actors`, the framework's own pattern — inserted by one `INSERT … SELECT`: postgres, SQLite and SQL Server take all three (they check at STATEMENT end); MariaDB and MySQL answer `ERROR 1452` (they check per ROW) and need the switch their own `emptyTables` already uses. Measured on each, and measured again after: a genuine violation attempted following the swap is still refused on all five, so the suspension does not leak past it.
+
+  The statement builders refuse a table name outside the framework's identifier class. This module concatenates SQL and its names arrive from a bundle MANIFEST — a file an operator can edit — so the rule applied at declaration is re-asserted where the concatenation happens rather than assumed to have survived the round trip. Staging tables are `_voltro_staging_<t>`, so the boot differ's framework-table asymmetry treats them as ours instead of planning them as user tables somebody forgot to declare.
+
+  **The importer does not use it yet**, and attempting that integration is what surfaced two blockers worth stating: a staged write would fire a change event (waking reactivity, CDC and the analytics mirror for tables nobody declared), and it would miss the target's column metadata (`encodeRowForSchema` looks the table up by NAME, so a `json()` column would be written unencoded — a wrong value, not an error). Both are tractable; neither is a line-level change. Until then the two protections already shipped — the capture written before the first delete, and the marker that refuses the next boot after an interrupted run — remain what covers the reported outcome.
+- **@voltro/data-transfer, @voltro/cli** — `--mode replace` writes down what it is about to destroy.
+
+  Before the first delete it exports the target's CURRENT rows — exactly the tables it will empty — as an ordinary bundle beside yours, and says where:
+
+  rollback capture: 240172 row(s) across 75 table(s) → ./out.rollback-2026-… If this run does not finish, restore with: voltro data import ./out.rollback-… --mode replace
+
+  It is on disk BEFORE anything is destroyed, so it depends on no transaction and on no process being alive to roll one back. That is the whole point: a deployment lost 240 172 rows to a `replace` whose api pod disappeared nine minutes in, and recovered from an export they had taken twenty minutes earlier out of HABIT. This is that habit as behaviour. It is NOT the fix for the class — the emptying must not become visible until the load stands, which is a rebuild — it is the small half that covers the reported outcome today.
+
+  **Fail-closed.** A capture that cannot be taken stops the import before it starts, target untouched. A net you believe in and do not have is worse than none: the belief is what stops you taking your own export.
+
+  The capture is a COMPLETE bundle over a snapshot narrowed to the emptied tables, not a `tables`-scoped one over the whole schema. Same files, different manifest — and the manifest decides whether it can be restored at all, since `replace` refuses a partial bundle for a reason that is exactly false here.
+
+  `--no-rollback` opts out, `--rollback-dir <path>` relocates it. Only `replace` takes one: `upsert` and `append` destroy nothing.
+
+  **Not on `--target api`, and it says so.** The capture would live inside the instance — the thing that can go away, which is the failure it exists for. A replace over that transport warns and names the export to take first.
+- **@voltro/database, @voltro/data-transfer** — Staging clones — the load-side half of the staged swap for `replace`.
+
+  The swap primitive shipped without the importer using it, and two things stood in the way. Both are solved by ONE answer.
+
+  A typed write resolves its table by NAME: `encodeRowForSchema` and `stampGeneratedId` both look it up in the registry. Writing to `_voltro_staging_notes` therefore found nothing — and the failure mode is not an error, it is a `json()` column written UNENCODED. Separately, every dialect store's insert ends in `routeEvent`, whose reactive guard reads `isTableReactive`, which is `isReactive !== false` — so an UNREGISTERED name counts as reactive, and loading a large bundle into staging would emit an event per row for tables nobody declared.
+
+  A staging table registered as a CLONE of its target, marked `isReactive: false`, answers both: the columns resolve, and the guard the framework already has returns before the emit. `.nonReactive()` is the documented way to say exactly that, so nothing at the store needed a special case — and it is the clone rather than the absence that makes the load quiet, which is the part worth remembering if this is ever simplified.
+
+  `registerStagingClones` returns a REQUIRED undo instead of trusting a caller to remember one. `allRegisteredTables()` feeds the declared set, the boot differ and `voltro doctor`, so a clone left in the registry reads as a table the app declares and nobody created. A failure part-way through registers nothing at all.
+
+  `unregisterTable` is new in `@voltro/database` for this: narrow on purpose. `clearTableRegistry` wipes everything and exists for tests; this removes ONE name a bounded operation owns for its duration.
+
+  **The precision is recovered, and it is strictly better than what it replaces.** Staging carries no foreign keys — required, since a staged row whose parent has not been staged yet must not be refused — so a violation moves from load time to swap time, where the database answers with one message naming a constraint. `stagedReferences` reads the edges INSIDE the replaced set off the snapshot the importer already holds (deliberately not `incomingForeignKeys`, which answers the already-answered question of keys pointing in from OUTSIDE), and `danglingProbeSql` asks STAGING the question the database was asking — against the STAGED parent set, because the swap inserts parents from staging and what matters is whether the reference resolves AFTER it.
+
+  Measured against a live postgres with four staged children — one good, one NULL, two dangling — driven through a real failing swap: the database named `ghost`; the probe named `ghost` AND `phantom`, skipping the NULL and the good row. The row-by-row path stops at the FIRST failure, so a bundle with four bad references costs four round trips; this reports all of them in one pass. The target was verified unchanged afterwards, which is the first thing the message says.
+
+  **The importer uses it now.** `--mode replace` stages when it can: create a staging table per table, load into those OUTSIDE any transaction, then swap the content across in one short transaction of server-side SQL. A process that dies during the load leaves the target exactly as it was, because nothing has been deleted yet — the destructive window shrinks from the length of the load to the length of a copy.
+
+  Wiring it surfaced two more things, and neither was visible from the design.
+
+  `INSERT … SELECT *` fails the moment the target has a STORED generated column — `CREATE TABLE … (LIKE t)` copies such a column as a PLAIN one (measured: `is_generated: NEVER`), so the select hands the target a value for a column it computes itself: `cannot insert a non-DEFAULT value into column "slug"`. The swap names its columns now, minus the generated ones, and `swapStatements` REFUSES an empty column list rather than falling back to `SELECT *`, so the trap cannot return by omission.
+
+  And a WRITE RECORDER on any table in the set rules staging out. A recorder is keyed by table NAME, so a staged write looks up `_voltro_staging_notes`, finds none, and never runs — `versioningPlugin({ timing: 'in-transaction' })` promises "if the change committed, the entry is there", and the swap's bulk SQL has no per-row hook to keep that with. Measured, not guessed: the recorder went from more than one call to zero. Such a run keeps the path that honours it and SAYS why, as does a `--no-atomic` run or a store the framework cannot send DDL to.
+
+  Every staged run says it staged. The two paths are indistinguishable from outside — both end with the target holding the bundle — and an operator deciding whether they can afford to interrupt needs to know which one is running.
+
+  Verified through the real importer, and falsified before being kept: with staging off, the assertion that the target still holds its own rows WHILE the bundle loads goes red. On real postgres the staged path runs the existing foreign-key replace suite unchanged.
+
+### Fixed
+
+- **@voltro/protocol, @voltro/voltro** — A `_voltro_*` name in `source:` is no longer narrowed against the generated table declaration.
+
+  Which framework tables an app declares is DEPLOYMENT-dependent. The measurement is already in the maintainer notes: at one `NODE_ENV`, on one dialect, flipping a single flag adds or removes `_voltro_traces`, `_voltro_undo_log` or `_voltro_cdc_offsets` from the declared set. The generated `voltro-tables.generated.d.ts` is written by ONE `voltro dev` run, on one machine, with one set of those inputs.
+
+  Narrowing framework names against it therefore made `source: '_voltro_traces'` compile for whoever generated the file and fail for a colleague — a type error decided by an environment variable, which is the exact class the declared-schema rule forbids one layer up. An app's own tables are unaffected: `_voltro_` is a reserved prefix, so every name a user writes for their own data narrows exactly as before.
+
+  **How it surfaced is the part worth recording.** Until now `keyof VoltroTableNames` was always `never` inside this repo, so `TableName` was always `string`, so the narrow and wide types were the same type and every rule about them held vacuously. The first time an augmentation was ever present — a fixture that boots a real server writing the declaration beside its generated rpc group — four framework source files stopped compiling. The split had shipped without once being exercised in the direction that matters.
+
+  Two things now stop that from going quiet again. A type-test program compiles the framework's own sources under an augmentation that deliberately declares NONE of its tables, and it lives in its OWN tsconfig: `declare module` merging is program-global, so a sibling type test's augmentation had silently rescued the very assertion this one exists to make. And `scripts/check-type-tests.mjs` (CI + `pnpm gate`) DISCOVERS type-test programs and runs them — because nothing did. The existing narrowing assertions had never been compiled once: excluded from their package's tsconfig for a good reason, and picked up by nothing else. It refuses a zero-program run, and a program that compiles zero `*.test-d.ts` files, for the same reason every other check here carries a floor.
+
+  **Why `apiSurface: compatible`, and how to check it rather than take it.** The gate flagged the golden line as CHANGED and asked the right question — can this turn code that compiled into code that does not? Here it cannot, because the edit WIDENS a union, and every public position the type appears in is an INPUT: `source?:` on a descriptor, and `normalizeSource`'s parameter. Nothing in the published surface RETURNS `TableName` or `ReactivitySource`, which is the only direction in which widening breaks a consumer — an assignment FROM the type into something narrower. `@voltro/voltro` is listed beside `@voltro/protocol` because it re-exports the type, so its golden moved too; the gate matches per package, and one package's classification must not vouch for another's.
+- **@voltro/sql-mssql** — A `json()` column could not take a value on mssql. At all.
+
+  store.insert(t, { payload: { a: 1 } }) -> TypeError: Invalid string. [EPARAM] store.insert(t, { payload: null }) -> OK
+
+  So the column worked only while it held nothing. Reached first through `@voltro/plugin-versioning`, whose history row carries a full-row snapshot in exactly such a column: the versioning recorder could not write on that dialect, and therefore neither could any write to a table it covers.
+
+  The store handed the caller's row straight to `sql.insert(row)`. The mysql store runs `encodeRowForSchema` first — the schema-driven step that turns a json value into text a driver can bind — and mssql had no equivalent anywhere on its write path, so tedious received a JS object for an NVARCHAR parameter and refused it.
+
+  **Encoding alone would have been worse than the bug.** A value written as text and handed back as text means a WRITE returns a string where a READ of the same row returns an object, and nothing errors — the caller gets a different type depending on how it got there. So every `OUTPUT INSERTED.*` path decodes too (insert, insertMany, update, updateMany, patchJson, the delete old-image, and the MERGE upsert), and the test asserts the ROUND TRIP rather than the absence of an error, against a live SQL Server.
+
+  With it, the mssql case is back in the versioning key-length suite — the bound on `id()` exists on that dialect too, and it was absent for one release only because nothing could write there.
+- **@voltro/data-transfer, @voltro/cli** — A `scope: all` bundle carried the exporting deployment's own bookkeeping, and `replace` wrote it into the target. The target's next boot refused to start:
+
+  auto-migrate: SCHEMA FINGERPRINT MISMATCH — declared=6e2c61081a9ed80c live=28af9a54414f22f1
+
+  The refusal was correct and the row was the defect. A migration-ledger row is not DATA — it states which schema THIS deployment applied — and the fingerprint is computed over the declared table set, which legitimately differs per environment (`NODE_ENV=production` declares `_voltro_traces` and `_voltro_undo_log`; a `development` run does not). So the imported row was not stale, it was FOREIGN. The environment was down for ninety minutes, and the ledger row also HID the incident it travelled with: the api would not start, and the reason looked like the failed import rather than a row in a bookkeeping table.
+
+  Ten framework tables are now classified as environment-local — the migration ledger, the file-migration and seed records, CDC offsets, schedule claims, wakeups, workflow watermarks / pending starts / admissions / pauses. They are dropped from an export's `all` scope, skipped on import, and never emptied by a `replace`, each with the reason a foreign row would be wrong stated beside it. Two of them would have made the target ACT: a pending start runs a workflow somebody queued elsewhere, a pause silently stops one here.
+
+  `all` is the only scope filtered. A caller who NAMES one of these tables gets it — an explicit name is an expectation, and this module already refuses to drop those silently.
+
+  The classification refuses to be incomplete: a guard scans every framework table and fails until a new one is decided either way. A hand-list rots by omission, and the omission cost ninety minutes.
+
+  Also fixed, found while reading that path: the admin import endpoint spread `atomic` only when truthy, so an explicit `atomic: false` was dropped and the importer applied its own default — which for `replace` is `true`. The one value a caller can only express by asking for it was the one the wire discarded.
+
+---
+
 ## [0.45.0] — 2026-08-21
 
 ### ⚠ BREAKING

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@voltro/plugin-auth",
-  "version": "0.45.0",
+  "version": "0.46.0",
   "description": "Authentication primitives: password hashing, session creation, schema mixin + tables. Pairs with the `auth` app template for the UI; both independently usable. Server-side only (uses node:crypto).",
   "keywords": [
     "voltro",
@@ -79,8 +79,8 @@
   },
   "dependencies": {
     "@effect/sql": "^0.52.0",
-    "@voltro/database": "0.45.0",
-    "@voltro/protocol": "0.45.0"
+    "@voltro/database": "0.46.0",
+    "@voltro/protocol": "0.46.0"
   },
   "peerDependencies": {
     "effect": "^3.22.0",