@voltro/sql-sqlite 0.43.1 → 0.44.0

package/CHANGELOG.md

@@ -39,6 +39,71 @@ _Changes staged for the next release accumulate here (rolled up from
 
 ---
 
+## [0.44.0] — 2026-08-19
+
+### ⚠ BREAKING
+
+- **@voltro/data-transfer, @voltro/cli** — `runImport` returns an `ImportOutcome` instead of the bundle's `Manifest`, and `--mode replace` runs as ONE transaction by default.
+
+  **Why the return type changed.** The summary line counted the rows the BUNDLE carries, not the rows the run wrote. Those differ most exactly where it matters: a bundle directory carries its own resume ledger, so a copied directory imports nothing — correctly, with a warning naming the file to delete — and the run then printed `import complete … rows: 242950` over a target it had not touched. The warning was one line above, which is one line too far for anyone piping the output through `tail -1`. The outcome carries `rowsWritten`, `rowsSkipped` and `fullyResumed`, the CLI reports written-vs-carried, and a fully-skipped run says so on its LAST line.
+
+  Migration: `runImport(...)` now resolves to `{ manifest, tablesWritten, rowsWritten, tablesSkipped, rowsSkipped, fullyResumed }`. Read `.manifest` where you read the manifest before.
+
+  **Why replace is atomic now.** The all-or-nothing guarantee was written for the emptying step, and read — reasonably — as covering the run. A replace that died partway through the LOAD left the target emptied of its old rows and holding part of the new ones, measured on a live instance over sixteen minutes. There is no useful state for a replace to stop in, so it is a default rather than a flag you have to know about. It also closes the window that produced that failure: with the tables emptied and the load uncommitted, a concurrent writer in the application waits instead of inserting a row the bundle is about to insert too.
+
+  `--no-atomic` (CLI) / `atomic: false` (API) opts out. The trade is stated where it bites: every write to those tables waits for the load, and on postgres the bulk `COPY` loader cannot join a transaction it does not own — an atomic run now says that once rather than being quietly slower.
+
+  Two smaller things from the same report: `--mode replace` against a running instance warns about the empty-target window when it is NOT atomic, and the `--target api` upload reports progress per chunk plus a line explaining that the final request stays open for the whole import — sixteen minutes of silence is indistinguishable from a hang, and one operator killed a run that had finished.
+
+  **`voltro update` carries you across this** — codemod `0.44.0/01_import-outcome`.
+
+### Fixed
+
+- **@voltro/sql-mysql, @voltro/sql-postgres, @voltro/sql-sqlite, @voltro/sql-mssql, @voltro/data-transfer** — A write and its write-recorders now succeed or fail together on every SQL dialect, and the data importer's retry is idempotent.
+
+  A recorder (a versioning trail, an audit log) runs on the caller's connection and is ALLOWED to fail — a recorder that throws must take the write down with it, that is its contract. Outside a caller transaction the two were not one unit: the row's statement committed on its own, and the recorder's INSERT ran afterwards as a second autocommit statement. So a recorder that threw left a COMMITTED row behind a write that reported failure. Measured directly: `insert` throws, the row is in the table, and a second attempt at the same row is `ER_DUP_ENTRY` on PRIMARY.
+
+  Anything that retries a failed write then meets its own row. The data importer retries by design — it holds a row whose write failed and tries again once the remaining tables have streamed — so a `--mode replace` that had just emptied a table failed on a duplicate key IN that table. From outside, that is the impossible-looking thing: an import that emptied a table and then failed because something was already in it. Reproduced verbatim, including the table name and `[ER_DUP_ENTRY/1062]`, by putting the old retry back.
+
+  Both ends are closed, and they are independent on purpose:
+
+  - **The cause.** A table that HAS recorders writes inside a transaction now, so the row and the trail commit together or not at all — on mysql, mariadb, postgres, sqlite and mssql, verified by one suite that asks all five the same question. A table with no recorders — the default — takes the direct path unchanged; `recordsTable` is a Map-size check first, so it costs one comparison. - **The defence.** The importer's retry upserts instead of inserting in `replace` mode. That covers every OTHER way a write can land while reporting failure: a driver timeout on a write the server applied, a connection lost after the commit, a concurrent writer inserting the same key. It is sound precisely because the table was emptied by this same run — there is nothing in it that is not ours.
+
+  The test that pinned the divergence went red when the fix landed, exactly as its own note said it would, and is inverted with that note kept.
+
+---
+
+## [0.43.2] — 2026-08-18
+
+### Fixed
+
+- **@voltro/sql-mysql, @voltro/database, @voltro/cli** — The mysql-family binlog reader no longer keeps a table excluded after the very migration that fixed it.
+
+  A UNIQUE on an unbounded text column is a MariaDB hash long-unique, whose hidden `DB_ROW_HASH_n` column the reader can never account for — so the table is held out of binlog capture and the exclusion is reported. `voltro dev` builds its store BEFORE it migrates (kv, cross-replica broadcast and the analytics mirror all need one), so on a boot whose own auto-migration bounds the column, that finding was drawn from a schema that stopped existing about a second later. The exclusion then outlived its cause for the life of the process, and the message — correct when written, and typically the only error line in the boot log — went on describing the pre-migration database.
+
+  Three changes:
+
+  - `DataStore.refreshChangeCaptureExclusions()` (optional; implemented by the mysql-family store) re-runs the probe and re-points the LIVE reader, in both directions — schema work that CREATES the condition now excludes the table immediately instead of after three failed writes. `voltro dev` calls it once, after all its schema work. `voltro serve` needs no equivalent: it builds its store after every schema step and never applies DDL itself. - The definitive message now states its own durability — that it is the schema as read at reader attach, and that applying the remedy does not by itself lift the exclusion. - On a boot that will re-check, the finding is reported as a provisional note rather than as a verdict, so a boot that fixes the condition leaves no error line about it. The note escalates to the full verdict on its own if the re-check never runs. - Two reader fixes the re-check depended on: applying a new exclusion set now WAITS for a reconnect already in flight (it is what applies the set, so resolving before it landed meant the caller's next write hit the old filter), and the reconnect loop no longer keeps resuming from an offset it has just jumped away from — that turned one purged offset into a reconnect every watchdog interval, forever, delivering nothing.
+- **@voltro/data-transfer, @voltro/database, @voltro/sql-mysql, @voltro/sql-postgres, @voltro/sql-sqlite, @voltro/sql-mssql, @voltro/cli** — `voltro data import --mode replace` no longer leaves a target in neither state, and now works against schemas that have foreign keys.
+
+  The delete step ran table by table and stopped at the first refusal, so a run that could not finish left dozens of tables emptied and nothing loaded — and a second attempt destroyed more than the first, because it got further before hitting the same wall. The wall itself was not exotic: MySQL, MariaDB and SQL Server check a foreign key as each ROW is deleted, so a table that references ITSELF cannot be emptied by any ordering of the tables. `createdBy → actors` on the `actors` table is exactly that shape, and it is what an audit mixin on an actor table produces.
+
+  - `DataStore.emptyTables()` (per dialect) empties the whole set as one unit, in one transaction, with referential integrity suspended for the duration — `FOREIGN_KEY_CHECKS` on the mysql family, a multi-table `TRUNCATE` on postgres, `defer_foreign_keys` on sqlite, per-table `NOCHECK`/`WITH CHECK CHECK` on mssql. All-or-nothing on every engine, including under `--atomic`, where it runs on the transaction the import already holds. - A **pre-flight refusal**: if a table OUTSIDE the bundle holds rows referencing one inside it, the import refuses before deleting anything and names the tables, the columns and the row counts. Those rows cannot be restored from the bundle, so forcing it is not an option. An EMPTY outside table blocks nothing. - Table-level failures carry the driver's own reason and code, the way row-level failures already did. `truncate <table> failed: Failed to execute statement` fits every plausible cause equally; the classification that produced `foreign key <name>: … [ER_NO_REFERENCED_ROW_2/1452]` one level down now applies one level up. The word "truncate" is gone from the message too — the step issues DELETE, and naming a statement it does not run sends whoever reads it to reproduce the wrong thing. - A typed refusal reaching the `--target api` transport keeps its text: the admin import endpoint answers `409` with the reason instead of flattening it to `import failed`, on the one transport where the operator has no other way to see it.
+
+  The bulk empty emits no change events, where the per-table loop emitted one per row. An import through `--target api` now asks every live subscription to re-read once it lands — the coarse refresh the framework already uses after a broadcast gap — so neither the missing deletes nor a table the bundle carries EMPTY leaves a subscriber holding rows that are gone. Wired where the route is mounted, which is the one place both boot paths share.
+
+  On postgres the emptying is a `DELETE` per table, not a `TRUNCATE`, and the difference is not performance: postgres refuses `TRUNCATE` on a table with ANY incoming foreign key, rows or not, while the mysql family refuses a DELETE only when rows actually reference the doomed ones. A `TRUNCATE` version made an EMPTY table outside the bundle block a replace on postgres and not on mariadb — one import, refused on one engine and accepted on the other, over a table holding nothing.
+
+  `--target api` also no longer times out on a full bundle. Both api-target calls went through `fetch`, whose undici default gives up after 300 s — a bound on the caller's database size, on a call whose response arrives only when the import does. They wait as long as the instance needs now, stream the body instead of buffering the whole bundle, and take `--timeout <seconds>` when a deadline is wanted. If one is hit, the message says the instance is probably still importing — and gives different advice for `replace` than for the idempotent modes, since re-running the first while it is mid-flight would empty the target under it.
+
+  Two more, found by measuring rather than by reading:
+
+  - **`--atomic` on postgres could not import a bundle that needed the deferred-FK repair at all.** A failed statement aborts the transaction there, and that repair depends on a row whose parent has not loaded yet failing, being held, and being retried — so the first such row poisoned every write after it. Every tolerated write now runs inside a savepoint. Per-row savepoints measured 2.40x the time of none on 5 000 rows, so they are amortised: one savepoint per batch of 200, and a batch that fails rolls back whole and replays row by row. The mysql family and sqlite leave a transaction usable after a failed statement and pay nothing for any of this. - **The replace pre-flight asked the caller's snapshot.** Over `--target api` that is the app's DECLARED schema, which cannot show a table the app stopped declaring but the database still has — and rows in a table nobody declares are exactly the rows nobody is watching. `DataStore.incomingForeignKeys()` reads the live catalog per dialect; the snapshot remains the fallback for stores without one.
+
+  A bundle bigger than one chunk is now uploaded as a series of short requests, so a proxy body cap or an ingress read timeout has nothing large to choke on, and the switch is automatic — the packer's stream is buffered one chunk ahead, so a small bundle is sent exactly as before and nobody has to know in advance which table is the big one. The import still runs ONCE, at the end, over the whole bundle. Resume is byte-exact (`packBundle` is deterministic over a directory, which this package now asserts), guarded by a bundle key so a different bundle under the same upload id is refused rather than spliced into the partial one, and by a contiguity check so a mis-ordered append cannot produce an archive that only fails later during decode. `--chunk-size <mb>` overrides the 16 MiB default.
+
+---
+
 ## [0.43.1] — 2026-08-18
 
 ### Fixed

package/dist/index.d.ts

@@ -136,6 +136,16 @@ export declare class SqliteStore implements DataStore {
      * ask the question at all.
      */
     private localWrite;
+    /**
+     * A write and its write-recorders, as ONE unit — see `MysqlStore.
+     * writeWithRecorders` for the failure this closes.
+     *
+     * The same divergence exists here even though sqlite is one process: the row's
+     * statement commits on its own and the recorder's INSERT is a second one, so a
+     * recorder that throws leaves a committed row behind a write that reported
+     * failure — and a caller that retries meets its own row.
+     */
+    private writeWithRecorders;
     insert(table: string, row: Row): Promise<Row>;
     insertMany(table: string, rows: ReadonlyArray<Row>): Promise<ReadonlyArray<Row>>;
     patchJson(table: string, primaryKey: string, path: string, value: unknown): Promise<Row | null>;
@@ -179,6 +189,30 @@ export declare class SqliteStore implements DataStore {
      * defect, preserving crash semantics). Without this the Turso MVCC
      * retry — the whole point of `BEGIN CONCURRENT` — would never fire.
      */
+    /**
+     * Empty `tables` as one unit — see `DataStore.emptyTables`.
+     *
+     * SQLite ships the exact switch for this: `defer_foreign_keys` holds every
+     * foreign-key check until COMMIT, and unlike `PRAGMA foreign_keys` it is
+     * settable INSIDE a transaction (that one is a silent no-op there, which is
+     * the trap to avoid). So the whole set empties in any order and integrity is
+     * verified once, at the end — which is what makes a self-referencing table
+     * possible to clear at all.
+     *
+     * The pragma resets itself at COMMIT, so there is nothing to restore; the
+     * explicit reset below covers the ROLLBACK path, where it does not.
+     */
+    emptyTables(tables: ReadonlyArray<string>): Promise<void>;
+    /** See `DataStore.incomingForeignKeys`. SQLite has no catalog VIEW of foreign
+     *  keys — `PRAGMA foreign_key_list(t)` answers per table — so this walks the
+     *  table list once. That is fine at the one call site (a pre-flight, once per
+     *  import) and is why it is not a general-purpose helper. */
+    incomingForeignKeys(tables: ReadonlyArray<string>): Promise<ReadonlyArray<{
+        from: string;
+        column: string;
+        to: string;
+    }>>;
+    /* Excluded from this release type: emptyTablesOn */
     transactional<T>(work: (tx: DataStore) => Promise<T>): Promise<T>;
     /**
      * The DIALECT half of the shared transaction bracket

package/dist/index.js

@@ -1,14 +1,14 @@
 import { SqliteClient as e } from "@effect/sql-sqlite-node";
-import { Config as t, Context as n, Effect as r, Layer as i, ManagedRuntime as a } from "effect";
-import { EventEmitter as o } from "node:events";
-import { createLogger as s } from "@voltro/logger";
-import { SqlClient as c, TransactionConnection as l } from "@effect/sql/SqlClient";
-import { EagerCardinalityError as u, attachEagerLoads as d, attributionFields as f, bulkInsertLimitsFor as p, chunkRowsForInsert as m, compileEagerJson as h, compilePredicate as g, compileRawFragment as _, compileSelect as v, decodeRowsFromSchema as y, externalChangeEvent as b, hasEagerLoads as x, isTableReactive as S, makeEagerFallbackReporter as C, observeDbOp as w, qualifyTable as T, raiseChangeListenerCeiling as E, recordsTable as D, requireTable as O, runStoreTransaction as k, runWriteRecorders as A, stampGeneratedId as j, stampGeneratedIds as M, withCapturedAttribution as N } from "@voltro/database";
+import { Config as t, Context as n, Effect as r, Layer as i, ManagedRuntime as a, Option as o } from "effect";
+import { EventEmitter as s } from "node:events";
+import { createLogger as c } from "@voltro/logger";
+import { SqlClient as l, TransactionConnection as u } from "@effect/sql/SqlClient";
+import { EagerCardinalityError as d, attachEagerLoads as f, attributionFields as p, bulkInsertLimitsFor as m, chunkRowsForInsert as h, compileEagerJson as g, compilePredicate as _, compileRawFragment as v, compileSelect as y, decodeRowsFromSchema as b, externalChangeEvent as x, hasEagerLoads as S, isTableReactive as C, makeEagerFallbackReporter as w, observeDbOp as T, qualifyTable as E, quoteIdent as D, raiseChangeListenerCeiling as O, recordsTable as k, requireTable as A, runStoreTransaction as j, runWriteRecorders as M, stampGeneratedId as N, stampGeneratedIds as P, withCapturedAttribution as F } from "@voltro/database";
 //#region src/sqlLayer.ts
-var P = (r) => e.layerConfig({
+var I = (r) => e.layerConfig({
 	filename: t.succeed(r.filename),
 	...r.readonly === void 0 ? {} : { readonly: t.succeed(r.readonly) }
-}).pipe(i.tap((t) => n.get(t, e.SqliteClient)`PRAGMA foreign_keys = ON`)), F = (e) => {
+}).pipe(i.tap((t) => n.get(t, e.SqliteClient)`PRAGMA foreign_keys = ON`)), L = (e) => {
 	if (e.url) {
 		if (e.url === ":memory:") return { filename: ":memory:" };
 		if (e.url.startsWith("file:")) return { filename: e.url.slice(5) };
@@ -16,12 +16,12 @@ var P = (r) => e.layerConfig({
 	}
 	if (e.database) return { filename: e.database };
 	throw Error("@voltro/sql-sqlite: no filename supplied. Set DB_URL=file:./db.sqlite or DB_DATABASE=path/to/db.sqlite.");
-}, I = (e) => P(F(e)), L = /* @__PURE__ */ new Set([
+}, R = (e) => I(L(e)), z = /* @__PURE__ */ new Set([
 	"SQLITE_BUSY",
 	"SQLITE_LOCKED",
 	"5",
 	"6"
-]), R = (e) => {
+]), B = (e) => {
 	let t = e;
 	for (let e = 0; e < 5 && typeof t == "object" && t; e++) {
 		let e = t.code;
@@ -29,18 +29,18 @@ var P = (r) => e.layerConfig({
 		if (typeof e == "number") return String(e);
 		t = t.cause;
 	}
-}, z = (e) => {
-	let t = R(e);
-	return t !== void 0 && L.has(t);
-}, B = (e) => z(e) ? "retry" : "noRetry", V = s({ scope: "voltro:sqlite" }), H = async (e) => {
+}, V = (e) => {
+	let t = B(e);
+	return t !== void 0 && z.has(t);
+}, H = (e) => V(e) ? "retry" : "noRetry", U = c({ scope: "voltro:sqlite" }), W = async (e) => {
 	let t = e.tracerLayer ? i.mergeAll(e.sqlLayer, e.tracerLayer) : e.sqlLayer, n = a.make(t);
-	return new G(await n.runPromise(c), n, null, void 0, e.isRetryable ?? z, e.systemName ?? "sqlite", e.wrapTransaction);
-}, U = (e) => e, W = (e) => {
+	return new q(await n.runPromise(l), n, null, void 0, e.isRetryable ?? V, e.systemName ?? "sqlite", e.wrapTransaction);
+}, G = (e) => e, K = (e) => {
 	if (typeof e != "object" || !e) return e;
 	let t = {};
 	for (let [n, r] of Object.entries(e)) t[n] = r instanceof Date ? r.toISOString() : typeof r == "boolean" ? +!!r : r;
 	return t;
-}, G = class e {
+}, q = class e {
 	sql;
 	runtime;
 	namespace;
@@ -50,36 +50,36 @@ var P = (r) => e.layerConfig({
 	emitter;
 	inflightTxns = 0;
 	reportEagerFallback;
-	constructor(e, t, n = null, r, i = z, a = "sqlite", s = U) {
-		this.sql = e, this.runtime = t, this.namespace = n, this.isRetryable = i, this.systemName = a, this.wrapTransaction = s, this.emitter = r ?? new o(), E(this.emitter), this.reportEagerFallback = C(V);
+	constructor(e, t, n = null, r, i = V, a = "sqlite", o = G) {
+		this.sql = e, this.runtime = t, this.namespace = n, this.isRetryable = i, this.systemName = a, this.wrapTransaction = o, this.emitter = r ?? new s(), O(this.emitter), this.reportEagerFallback = w(U);
 	}
 	withNamespace(t) {
 		return t === this.namespace ? this : new e(this.sql, this.runtime, t, this.emitter, this.isRetryable, this.systemName, this.wrapTransaction);
 	}
 	nsT(e) {
-		return T(this.namespace, e);
+		return E(this.namespace, e);
 	}
 	async attachNamespace(e, t) {
 		let n = this.sql, r = t ?? `${e}.db`;
 		await this.runtime.runPromise(n`ATTACH DATABASE ${r} AS ${n(e)}`);
 	}
 	async executeQuery(e, t, n) {
-		let i = v(e, this.sql, this.namespace), a = t ? r.provideService(i, l, t) : i, o = await w(this.systemName, "select", () => this.runtime.runPromise(a));
-		return y(o, e.table, "sqlite");
+		let i = y(e, this.sql, this.namespace), a = t ? r.provideService(i, u, t) : i, o = await T(this.systemName, "select", () => this.runtime.runPromise(a));
+		return b(o, e.table, "sqlite");
 	}
 	async executeInsert(e, t, n, i, a) {
-		t = j(e, t);
-		let o = this.sql, s = o`INSERT INTO ${o(this.nsT(e))} ${o.insert(W(t))} RETURNING *`, c = n ? r.provideService(s, l, n) : s, u = (await this.runtime.runPromise(c))[0];
-		if (!u) throw Error(`SqliteStore.insert: no row returned for table '${e}'`);
+		t = N(e, t);
+		let o = this.sql, s = o`INSERT INTO ${o(this.nsT(e))} ${o.insert(K(t))} RETURNING *`, c = n ? r.provideService(s, u, n) : s, l = (await this.runtime.runPromise(c))[0];
+		if (!l) throw Error(`SqliteStore.insert: no row returned for table '${e}'`);
 		return await this.routeEvent({
 			table: e,
 			op: "insert",
 			old: null,
-			new: u
-		}, i, n, a), u;
+			new: l
+		}, i, n, a), l;
 	}
 	async executeUpdate(e, t, n, i, a, o) {
-		let s = this.sql, c = s`UPDATE ${s(this.nsT(e))} SET ${s.update(W(n))} WHERE ${s("id")} = ${t} RETURNING *`, u = i ? r.provideService(c, l, i) : c, d = (await this.runtime.runPromise(u))[0];
+		let s = this.sql, c = s`UPDATE ${s(this.nsT(e))} SET ${s.update(K(n))} WHERE ${s("id")} = ${t} RETURNING *`, l = i ? r.provideService(c, u, i) : c, d = (await this.runtime.runPromise(l))[0];
 		return d ? (await this.routeEvent({
 			table: e,
 			op: "update",
@@ -88,16 +88,16 @@ var P = (r) => e.layerConfig({
 		}, a, i, o), d) : null;
 	}
 	async executeInsertMany(e, t, n, i, a) {
-		if (t = M(e, t), t.length === 0) return [];
-		let o = this.sql, s = t.map((e) => W(e)), c = m(s, p("sqlite")), u = (t) => o`INSERT INTO ${o(this.nsT(e))} ${o.insert(t)} RETURNING *`, d;
+		if (t = P(e, t), t.length === 0) return [];
+		let o = this.sql, s = t.map((e) => K(e)), c = h(s, m("sqlite")), l = (t) => o`INSERT INTO ${o(this.nsT(e))} ${o.insert(t)} RETURNING *`, d;
 		if (c.length === 1) {
-			let e = u(c[0]), t = n ? r.provideService(e, l, n) : e;
+			let e = l(c[0]), t = n ? r.provideService(e, u, n) : e;
 			d = await this.runtime.runPromise(t);
 		} else if (n) {
 			let e = [];
-			for (let t of c) e.push(...await this.runtime.runPromise(r.provideService(u(t), l, n)));
+			for (let t of c) e.push(...await this.runtime.runPromise(r.provideService(l(t), u, n)));
 			d = e;
-		} else d = await this.runtime.runPromise(o.withTransaction(r.map(r.forEach(c, u, { concurrency: 1 }), (e) => e.flat())));
+		} else d = await this.runtime.runPromise(o.withTransaction(r.map(r.forEach(c, l, { concurrency: 1 }), (e) => e.flat())));
 		for (let t of d) await this.routeEvent({
 			table: e,
 			op: "insert",
@@ -107,7 +107,7 @@ var P = (r) => e.layerConfig({
 		return d;
 	}
 	async executePatchJson(e, t, n, i, a, o, s) {
-		let c = this.sql, u = n.split("."), d = u[0], f = u.slice(1), p = JSON.stringify(i ?? null), m = f.length === 0 ? c`${c(d)} = json_patch(COALESCE(${c(d)}, '{}'), json(${p}))` : c`${c(d)} = json_set(COALESCE(${c(d)}, '{}'), ${`$.${f.join(".")}`}, json(${p}))`, h = c`UPDATE ${c(this.nsT(e))} SET ${m} WHERE ${c("id")} = ${t} RETURNING *`, g = a ? r.provideService(h, l, a) : h, _ = (await this.runtime.runPromise(g))[0];
+		let c = this.sql, l = n.split("."), d = l[0], f = l.slice(1), p = JSON.stringify(i ?? null), m = f.length === 0 ? c`${c(d)} = json_patch(COALESCE(${c(d)}, '{}'), json(${p}))` : c`${c(d)} = json_set(COALESCE(${c(d)}, '{}'), ${`$.${f.join(".")}`}, json(${p}))`, h = c`UPDATE ${c(this.nsT(e))} SET ${m} WHERE ${c("id")} = ${t} RETURNING *`, g = a ? r.provideService(h, u, a) : h, _ = (await this.runtime.runPromise(g))[0];
 		return _ ? (await this.routeEvent({
 			table: e,
 			op: "update",
@@ -116,27 +116,27 @@ var P = (r) => e.layerConfig({
 		}, o, a, s), _) : null;
 	}
 	async executeDelete(e, t, n, i, a) {
-		let o = this.sql, s = o`DELETE FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t} RETURNING *`, c = n ? r.provideService(s, l, n) : s, u = (await this.runtime.runPromise(c))[0];
-		return u ? (await this.routeEvent({
+		let o = this.sql, s = o`DELETE FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t} RETURNING *`, c = n ? r.provideService(s, u, n) : s, l = (await this.runtime.runPromise(c))[0];
+		return l ? (await this.routeEvent({
 			table: e,
 			op: "delete",
-			old: u,
+			old: l,
 			new: null
 		}, i, n, a), !0) : !1;
 	}
 	async appendInTxn(e, t, n) {
-		let i = this.sql, a = i`INSERT INTO ${i(this.nsT(e))} ${i.insert(W(t))}`;
-		await this.runtime.runPromise(n ? r.provideService(a, l, n) : a);
+		let i = this.sql, a = i`INSERT INTO ${i(this.nsT(e))} ${i.insert(K(t))}`;
+		await this.runtime.runPromise(n ? r.provideService(a, u, n) : a);
 	}
 	async maxInTxn(e, t, n, i) {
-		let a = this.sql, o = Object.entries(n).map(([e, t]) => a`${a(e)} = ${t}`), s = a`SELECT MAX(${a(t)}) AS ${a("m")} FROM ${a(this.nsT(e))} WHERE ${a.and(o)}`, c = (await this.runtime.runPromise(i ? r.provideService(s, l, i) : s))[0]?.m;
+		let a = this.sql, o = Object.entries(n).map(([e, t]) => a`${a(e)} = ${t}`), s = a`SELECT MAX(${a(t)}) AS ${a("m")} FROM ${a(this.nsT(e))} WHERE ${a.and(o)}`, c = (await this.runtime.runPromise(i ? r.provideService(s, u, i) : s))[0]?.m;
 		return c == null ? null : Number(c);
 	}
 	async routeEvent(e, t, n = null, r) {
 		e = {
-			...f(r),
+			...p(r),
 			...e
-		}, D(e.table) && await A({
+		}, k(e.table) && await M({
 			append: (e, t) => this.appendInTxn(e, t, n),
 			maxOf: (e, t, r) => this.maxInTxn(e, t, r, n)
 		}, {
@@ -146,14 +146,14 @@ var P = (r) => e.layerConfig({
 			prev: e.old,
 			traceId: e.traceId,
 			subjectId: e.subjectId
-		}), S(e.table) && (t === null ? this.emitter.emit("change", e) : t.push(e));
+		}), C(e.table) && (t === null ? this.emitter.emit("change", e) : t.push(e));
 	}
 	async executeUpsert(e, t, n, i, a, o) {
 		if (typeof n.update == "function") {
 			let r = await this.findByConflict(e, t, n.conflictColumns, i);
 			return r ? await this.executeUpdate(e, r.id, n.update(r), i, a) ?? r : this.executeInsert(e, t, i, a);
 		}
-		let s = this.sql, c = n.conflictColumns.map((e) => s`${s(e)}`), u = Object.keys(t).filter((e) => t[e] !== void 0), d = n.update === void 0 ? u.filter((e) => e !== "id" && !n.conflictColumns.includes(e)) : n.update, f = d.length > 0 ? s.csv(d.map((e) => s`${s(e)} = excluded.${s(e)}`)) : s`${s(n.conflictColumns[0])} = excluded.${s(n.conflictColumns[0])}`, p = s`INSERT INTO ${s(this.nsT(e))} ${s.insert(W(t))} ON CONFLICT (${s.csv(c)}) DO UPDATE SET ${f} RETURNING *`, m = i ? r.provideService(p, l, i) : p, h = (await this.runtime.runPromise(m))[0];
+		let s = this.sql, c = n.conflictColumns.map((e) => s`${s(e)}`), l = Object.keys(t).filter((e) => t[e] !== void 0), d = n.update === void 0 ? l.filter((e) => e !== "id" && !n.conflictColumns.includes(e)) : n.update, f = d.length > 0 ? s.csv(d.map((e) => s`${s(e)} = excluded.${s(e)}`)) : s`${s(n.conflictColumns[0])} = excluded.${s(n.conflictColumns[0])}`, p = s`INSERT INTO ${s(this.nsT(e))} ${s.insert(K(t))} ON CONFLICT (${s.csv(c)}) DO UPDATE SET ${f} RETURNING *`, m = i ? r.provideService(p, u, i) : p, h = (await this.runtime.runPromise(m))[0];
 		if (!h) throw Error(`SqliteStore.upsert: no row returned for table '${e}'`);
 		let g = t.id !== void 0 && t.id === h.id ? "insert" : "update";
 		return await this.routeEvent({
@@ -164,8 +164,8 @@ var P = (r) => e.layerConfig({
 		}, a, i, o), h;
 	}
 	async executeInsertIgnore(e, t, n, i, a, o) {
-		t = j(e, t);
-		let s = this.sql, c = n.conflictColumns.map((e) => s`${s(e)}`), u = s`INSERT INTO ${s(this.nsT(e))} ${s.insert(W(t))} ON CONFLICT (${s.csv(c)}) DO NOTHING RETURNING *`, d = i ? r.provideService(u, l, i) : u, f = (await this.runtime.runPromise(d))[0];
+		t = N(e, t);
+		let s = this.sql, c = n.conflictColumns.map((e) => s`${s(e)}`), l = s`INSERT INTO ${s(this.nsT(e))} ${s.insert(K(t))} ON CONFLICT (${s.csv(c)}) DO NOTHING RETURNING *`, d = i ? r.provideService(l, u, i) : l, f = (await this.runtime.runPromise(d))[0];
 		if (f) return await this.routeEvent({
 			table: e,
 			op: "insert",
@@ -178,24 +178,24 @@ var P = (r) => e.layerConfig({
 	}
 	async findByConflict(e, t, n, i) {
 		if (n.length === 0) return;
-		let a = this.sql, o = n.map((e) => a`${a(e)} = ${t[e]}`), s = a`SELECT * FROM ${a(this.nsT(e))} WHERE ${a.and(o)} LIMIT 1`, c = i ? r.provideService(s, l, i) : s;
+		let a = this.sql, o = n.map((e) => a`${a(e)} = ${t[e]}`), s = a`SELECT * FROM ${a(this.nsT(e))} WHERE ${a.and(o)} LIMIT 1`, c = i ? r.provideService(s, u, i) : s;
 		return (await this.runtime.runPromise(c))[0];
 	}
 	query(e) {
 		return this.runWithEager(e, null);
 	}
 	raw(e, t) {
-		let n = _(e, this.sql);
-		return w(this.systemName, "raw", () => this.runtime.runPromise(n));
+		let n = v(e, this.sql);
+		return T(this.systemName, "raw", () => this.runtime.runPromise(n));
 	}
 	async runWithEager(e, t) {
-		if (!x(e)) return this.executeQuery(e, t);
-		let n = this.namespace === null ? h(e, this.sql, "sqlite") : null;
+		if (!S(e)) return this.executeQuery(e, t);
+		let n = this.namespace === null ? g(e, this.sql, "sqlite") : null;
 		if (n !== null) try {
-			let e = t ? r.provideService(n.fragment, l, t) : n.fragment, i = await w(this.systemName, "select", () => this.runtime.runPromise(e));
+			let e = t ? r.provideService(n.fragment, u, t) : n.fragment, i = await T(this.systemName, "select", () => this.runtime.runPromise(e));
 			return n.decode(i);
 		} catch (t) {
-			if (t instanceof u) throw t;
+			if (t instanceof d) throw t;
 			this.reportEagerFallback({
 				dialect: this.systemName,
 				table: e.table,
@@ -209,31 +209,40 @@ var P = (r) => e.layerConfig({
 			reason: "not-compilable"
 		});
 		let i = await this.executeQuery(e, t);
-		return d(i, e.eager, e.sourceTable ?? O(e.table), (e) => this.executeQuery(e, t));
+		return f(i, e.eager, e.sourceTable ?? A(e.table), (e) => this.executeQuery(e, t));
 	}
 	getInternalRunWithEager() {
 		return (e, t) => this.runWithEager(e, t);
 	}
-	localWrite(e, t) {
-		return w(this.systemName, e, () => N(t));
+	localWrite(e, t, n) {
+		let r = (e) => k(t) ? this.writeWithRecorders(n, e) : n(e, null, null);
+		return T(this.systemName, e, () => F(r));
+	}
+	async writeWithRecorders(e, t) {
+		let n = [], i = this.sql, a = await this.runtime.runPromise(i.withTransaction(r.flatMap(r.serviceOption(u), (i) => o.isNone(i) ? r.fail(/* @__PURE__ */ Error("SqliteStore.write: TransactionConnection missing.")) : r.tryPromise({
+			try: () => e(t, i.value, n),
+			catch: (e) => e
+		}))));
+		for (let e of n) this.emitChange(e);
+		return a;
 	}
 	insert(e, t) {
-		return this.localWrite("insert", (n) => this.executeInsert(e, t, null, null, n));
+		return this.localWrite("insert", e, (n, r, i) => this.executeInsert(e, t, r, i, n));
 	}
 	insertMany(e, t) {
-		return this.localWrite("insert", (n) => this.executeInsertMany(e, t, null, null, n));
+		return this.localWrite("insert", e, (n, r, i) => this.executeInsertMany(e, t, r, i, n));
 	}
 	patchJson(e, t, n, r) {
-		return this.localWrite("update", (i) => this.executePatchJson(e, t, n, r, null, null, i));
+		return this.localWrite("update", e, (i, a, o) => this.executePatchJson(e, t, n, r, a, o, i));
 	}
 	update(e, t, n) {
-		return this.localWrite("update", (r) => this.executeUpdate(e, t, n, null, null, r));
+		return this.localWrite("update", e, (r, i, a) => this.executeUpdate(e, t, n, i, a, r));
 	}
 	delete(e, t) {
-		return this.localWrite("delete", (n) => this.executeDelete(e, t, null, null, n));
+		return this.localWrite("delete", e, (n, r, i) => this.executeDelete(e, t, r, i, n));
 	}
 	async updateMany(e, t, n) {
-		let r = this.sql, i = g(n.where, r, this.namespace), a = r`UPDATE ${r(this.nsT(e))} SET ${r.update(W(t))} WHERE ${i} RETURNING *`, o = await w(this.systemName, "update", () => this.runtime.runPromise(a));
+		let r = this.sql, i = _(n.where, r, this.namespace), a = r`UPDATE ${r(this.nsT(e))} SET ${r.update(K(t))} WHERE ${i} RETURNING *`, o = await T(this.systemName, "update", () => this.runtime.runPromise(a));
 		for (let t of o) await this.routeEvent({
 			table: e,
 			op: "update",
@@ -243,7 +252,7 @@ var P = (r) => e.layerConfig({
 		return o.length;
 	}
 	async deleteMany(e, t) {
-		let n = this.sql, r = g(t.where, n, this.namespace), i = n`DELETE FROM ${n(this.nsT(e))} WHERE ${r} RETURNING *`, a = await w(this.systemName, "delete", () => this.runtime.runPromise(i));
+		let n = this.sql, r = _(t.where, n, this.namespace), i = n`DELETE FROM ${n(this.nsT(e))} WHERE ${r} RETURNING *`, a = await T(this.systemName, "delete", () => this.runtime.runPromise(i));
 		for (let t of a) await this.routeEvent({
 			table: e,
 			op: "delete",
@@ -253,10 +262,10 @@ var P = (r) => e.layerConfig({
 		return a.length;
 	}
 	upsert(e, t, n) {
-		return this.localWrite("upsert", (r) => this.executeUpsert(e, t, n, null, null, r));
+		return this.localWrite("upsert", e, (r, i, a) => this.executeUpsert(e, t, n, i, a, r));
 	}
 	insertIgnore(e, t, n) {
-		return this.localWrite("upsert", (r) => this.executeInsertIgnore(e, t, n, null, null, r));
+		return this.localWrite("upsert", e, (r, i, a) => this.executeInsertIgnore(e, t, n, i, a, r));
 	}
 	emitChange(e) {
 		this.emitter.emit("change", e);
@@ -285,13 +294,35 @@ var P = (r) => e.layerConfig({
 	getInternalExecuteInsertIgnore() {
 		return this.executeInsertIgnore.bind(this);
 	}
+	async emptyTables(e) {
+		await this.emptyTablesOn(e, null);
+	}
+	async incomingForeignKeys(e) {
+		if (e.length === 0) return [];
+		let t = this.sql, n = new Set(e), r = await this.runtime.runPromise(t`SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'`), i = [];
+		for (let e of r) {
+			if (n.has(e.name)) continue;
+			let r = await this.runtime.runPromise(t`${t.unsafe(`PRAGMA foreign_key_list(${D(e.name, "sqlite")})`)}`).catch(() => []);
+			for (let t of r) n.has(t.table) && i.push({
+				from: e.name,
+				column: t.from,
+				to: t.table
+			});
+		}
+		return i;
+	}
+	async emptyTablesOn(e, t) {
+		if (e.length === 0) return;
+		let n = this.sql, i = (e) => this.nsT(e), a = r.acquireUseRelease(n.unsafe("PRAGMA defer_foreign_keys = ON"), () => r.forEach(e, (e) => n`DELETE FROM ${n(i(e))}`, { discard: !0 }), () => r.orDie(r.ignore(n.unsafe("PRAGMA defer_foreign_keys = OFF")))), o = t === null ? n.withTransaction(a) : r.provideService(a, u, t);
+		await this.runtime.runPromise(o);
+	}
 	async transactional(e) {
 		this.inflightTxns++;
 		try {
-			return await k({
+			return await j({
 				...this.txnSpec("SqliteStore.transactional"),
 				work: e,
-				makeView: (e, t) => new K(this, e, t)
+				makeView: (e, t) => new J(this, e, t)
 			});
 		} finally {
 			this.inflightTxns--;
@@ -320,7 +351,7 @@ var P = (r) => e.layerConfig({
 		};
 	}
 	injectExternalChange(e) {
-		S(e.table) && this.emitter.emit("change", b(e));
+		C(e.table) && this.emitter.emit("change", x(e));
 	}
 	run(e) {
 		return this.runtime.runPromise(e);
@@ -329,7 +360,7 @@ var P = (r) => e.layerConfig({
 		if (this.inflightTxns > 0) {
 			let t = Date.now() + e;
 			for (; this.inflightTxns > 0 && Date.now() < t;) await new Promise((e) => setTimeout(e, 25));
-			this.inflightTxns > 0 && V.warn("close: grace period expired with in-flight transactions", {
+			this.inflightTxns > 0 && U.warn("close: grace period expired with in-flight transactions", {
 				gracePeriodMs: e,
 				inflight: this.inflightTxns
 			});
@@ -339,7 +370,7 @@ var P = (r) => e.layerConfig({
 	async ping() {
 		await this.runtime.runPromise(this.sql`SELECT 1`);
 	}
-}, K = class {
+}, J = class {
 	parent;
 	txn;
 	attr;
@@ -351,6 +382,9 @@ var P = (r) => e.layerConfig({
 	query(e) {
 		return this.parent.getInternalRunWithEager()(e, this.txn);
 	}
+	emptyTables(e) {
+		return this.parent.emptyTablesOn(e, this.txn);
+	}
 	insert(e, t) {
 		return this.parent.getInternalExecuteInsert()(e, t, this.txn, this.events, this.attr);
 	}
@@ -412,12 +446,12 @@ var P = (r) => e.layerConfig({
 			this.events.length = 0;
 		}
 	}
-}, q = {
+}, Y = {
 	id: "sqlite",
-	makeSqlLayer: (e) => I(e),
-	makeStore: (e) => H(e),
+	makeSqlLayer: (e) => R(e),
+	makeStore: (e) => W(e),
 	compileContains: (e, t, n) => n`${e} LIKE ${`%${t.replace(/[\\%_]/g, (e) => `\\${e}`)}%`} ESCAPE '\\'`,
-	retryFilter: B
+	retryFilter: H
 };
 //#endregion
-export { e as SqliteClient, F as connectionFromConfig, P as makeSqliteSqlLayer, I as makeSqliteSqlLayerFromConfig, H as makeSqliteStore, q as sqliteDialect, B as sqliteRetryFilter };
+export { e as SqliteClient, L as connectionFromConfig, I as makeSqliteSqlLayer, R as makeSqliteSqlLayerFromConfig, W as makeSqliteStore, Y as sqliteDialect, H as sqliteRetryFilter };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@voltro/sql-sqlite",
-  "version": "0.43.1",
+  "version": "0.44.0",
   "description": "SQLite dialect adapter for Voltro's cross-dialect DataStore (single-process; in-process change events).",
   "keywords": [
     "voltro",
@@ -35,8 +35,8 @@
   "dependencies": {
     "@effect/sql": "^0.52.0",
     "@effect/sql-sqlite-node": "^0.53.0",
-    "@voltro/database": "0.43.1",
-    "@voltro/logger": "0.43.1"
+    "@voltro/database": "0.44.0",
+    "@voltro/logger": "0.44.0"
   },
   "peerDependencies": {
     "effect": "^3.22.0"