@voltro/sql-mysql 0.43.0 → 0.43.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -39,6 +39,106 @@ _Changes staged for the next release accumulate here (rolled up from
39
39
 
40
40
  ---
41
41
 
42
+ ## [0.43.2] — 2026-08-18
43
+
44
+ ### Fixed
45
+
46
+ - **@voltro/sql-mysql, @voltro/database, @voltro/cli** — The mysql-family binlog reader no longer keeps a table excluded after the very migration that fixed it.
47
+
48
+ A UNIQUE on an unbounded text column is a MariaDB hash long-unique, whose hidden `DB_ROW_HASH_n` column the reader can never account for — so the table is held out of binlog capture and the exclusion is reported. `voltro dev` builds its store BEFORE it migrates (kv, cross-replica broadcast and the analytics mirror all need one), so on a boot whose own auto-migration bounds the column, that finding was drawn from a schema that stopped existing about a second later. The exclusion then outlived its cause for the life of the process, and the message — correct when written, and typically the only error line in the boot log — went on describing the pre-migration database.
49
+
50
+ Three changes:
51
+
52
+ - `DataStore.refreshChangeCaptureExclusions()` (optional; implemented by the mysql-family store) re-runs the probe and re-points the LIVE reader, in both directions — schema work that CREATES the condition now excludes the table immediately instead of after three failed writes. `voltro dev` calls it once, after all its schema work. `voltro serve` needs no equivalent: it builds its store after every schema step and never applies DDL itself. - The definitive message now states its own durability — that it is the schema as read at reader attach, and that applying the remedy does not by itself lift the exclusion. - On a boot that will re-check, the finding is reported as a provisional note rather than as a verdict, so a boot that fixes the condition leaves no error line about it. The note escalates to the full verdict on its own if the re-check never runs. - Two reader fixes the re-check depended on: applying a new exclusion set now WAITS for a reconnect already in flight (it is what applies the set, so resolving before it landed meant the caller's next write hit the old filter), and the reconnect loop no longer keeps resuming from an offset it has just jumped away from — that turned one purged offset into a reconnect every watchdog interval, forever, delivering nothing.
53
+ - **@voltro/data-transfer, @voltro/database, @voltro/sql-mysql, @voltro/sql-postgres, @voltro/sql-sqlite, @voltro/sql-mssql, @voltro/cli** — `voltro data import --mode replace` no longer leaves a target in neither state, and now works against schemas that have foreign keys.
54
+
55
+ The delete step ran table by table and stopped at the first refusal, so a run that could not finish left dozens of tables emptied and nothing loaded — and a second attempt destroyed more than the first, because it got further before hitting the same wall. The wall itself was not exotic: MySQL, MariaDB and SQL Server check a foreign key as each ROW is deleted, so a table that references ITSELF cannot be emptied by any ordering of the tables. `createdBy → actors` on the `actors` table is exactly that shape, and it is what an audit mixin on an actor table produces.
56
+
57
+ - `DataStore.emptyTables()` (per dialect) empties the whole set as one unit, in one transaction, with referential integrity suspended for the duration — `FOREIGN_KEY_CHECKS` on the mysql family, a multi-table `TRUNCATE` on postgres, `defer_foreign_keys` on sqlite, per-table `NOCHECK`/`WITH CHECK CHECK` on mssql. All-or-nothing on every engine, including under `--atomic`, where it runs on the transaction the import already holds. - A **pre-flight refusal**: if a table OUTSIDE the bundle holds rows referencing one inside it, the import refuses before deleting anything and names the tables, the columns and the row counts. Those rows cannot be restored from the bundle, so forcing it is not an option. An EMPTY outside table blocks nothing. - Table-level failures carry the driver's own reason and code, the way row-level failures already did. `truncate <table> failed: Failed to execute statement` fits every plausible cause equally; the classification that produced `foreign key <name>: … [ER_NO_REFERENCED_ROW_2/1452]` one level down now applies one level up. The word "truncate" is gone from the message too — the step issues DELETE, and naming a statement it does not run sends whoever reads it to reproduce the wrong thing. - A typed refusal reaching the `--target api` transport keeps its text: the admin import endpoint answers `409` with the reason instead of flattening it to `import failed`, on the one transport where the operator has no other way to see it.
58
+
59
+ The bulk empty emits no change events, where the per-table loop emitted one per row. An import through `--target api` now asks every live subscription to re-read once it lands — the coarse refresh the framework already uses after a broadcast gap — so neither the missing deletes nor a table the bundle carries EMPTY leaves a subscriber holding rows that are gone. Wired where the route is mounted, which is the one place both boot paths share.
60
+
61
+ On postgres the emptying is a `DELETE` per table, not a `TRUNCATE`, and the difference is not performance: postgres refuses `TRUNCATE` on a table with ANY incoming foreign key, rows or not, while the mysql family refuses a DELETE only when rows actually reference the doomed ones. A `TRUNCATE` version made an EMPTY table outside the bundle block a replace on postgres and not on mariadb — one import, refused on one engine and accepted on the other, over a table holding nothing.
62
+
63
+ `--target api` also no longer times out on a full bundle. Both api-target calls went through `fetch`, whose undici default gives up after 300 s — a bound on the caller's database size, on a call whose response arrives only when the import does. They wait as long as the instance needs now, stream the body instead of buffering the whole bundle, and take `--timeout <seconds>` when a deadline is wanted. If one is hit, the message says the instance is probably still importing — and gives different advice for `replace` than for the idempotent modes, since re-running the first while it is mid-flight would empty the target under it.
64
+
65
+ Two more, found by measuring rather than by reading:
66
+
67
+ - **`--atomic` on postgres could not import a bundle that needed the deferred-FK repair at all.** A failed statement aborts the transaction there, and that repair depends on a row whose parent has not loaded yet failing, being held, and being retried — so the first such row poisoned every write after it. Every tolerated write now runs inside a savepoint. Per-row savepoints measured 2.40x the time of none on 5 000 rows, so they are amortised: one savepoint per batch of 200, and a batch that fails rolls back whole and replays row by row. The mysql family and sqlite leave a transaction usable after a failed statement and pay nothing for any of this. - **The replace pre-flight asked the caller's snapshot.** Over `--target api` that is the app's DECLARED schema, which cannot show a table the app stopped declaring but the database still has — and rows in a table nobody declares are exactly the rows nobody is watching. `DataStore.incomingForeignKeys()` reads the live catalog per dialect; the snapshot remains the fallback for stores without one.
68
+
69
+ A bundle bigger than one chunk is now uploaded as a series of short requests, so a proxy body cap or an ingress read timeout has nothing large to choke on, and the switch is automatic — the packer's stream is buffered one chunk ahead, so a small bundle is sent exactly as before and nobody has to know in advance which table is the big one. The import still runs ONCE, at the end, over the whole bundle. Resume is byte-exact (`packBundle` is deterministic over a directory, which this package now asserts), guarded by a bundle key so a different bundle under the same upload id is refused rather than spliced into the partial one, and by a contiguity check so a mis-ordered append cannot produce an archive that only fails later during decode. `--chunk-size <mb>` overrides the 16 MiB default.
70
+
71
+ ---
72
+
73
+ ## [0.43.1] — 2026-08-18
74
+
75
+ ### Fixed
76
+
77
+ - **@voltro/database** — A write recorder that fails inside someone's transaction now says which recorder, on which write, and what the database actually said.
78
+
79
+ `@effect/sql` renders every driver failure as `SqlError: Failed to execute statement` — one sentence that fits a missing column, a dangling foreign key, an over-long value and a duplicate key equally. The driver's own words hang off a SYMBOL on a `FiberFailure`, so a caller who reaches for `.cause` gets `undefined` and concludes there is nothing there.
80
+
81
+ That is expensive precisely where recorders run: the caller's write was ordinary, and what failed was framework machinery one table over. The message now reads
82
+
83
+ ```
84
+ write recorder '_voltro_row_history' failed while recording an update on 'users':
85
+ Duplicate entry 'rowver_…' for key 'PRIMARY' [code=ER_DUP_ENTRY errno=1062 …]
86
+ ```
87
+
88
+ and the original error is kept as `cause` for anyone who does walk the chain. The failure still takes the transaction down — that is the guarantee and it is unchanged.
89
+
90
+ `describeDriverError` (`@voltro/database`) is the shared summariser, built on the existing cause extractor rather than a second walker. It returns nothing when the chain carries nothing driver-shaped, so an ordinary programming error from a recorder arrives as itself instead of wrapped in prose about a database.
91
+ - **@voltro/database** — On MariaDB, a `varchar` column could introspect as `json` because a DIFFERENT table had a json column with the same column name.
92
+
93
+ MariaDB names a column-level CHECK after the COLUMN, and those names are unique per table, not per schema. `information_schema.check_constraints` on MySQL has no `TABLE_NAME`, so the introspector recovered it by joining `table_constraints` on `(schema, constraint_name)` — which on MariaDB cross-products every same-named check across every table. Measured on 11.8:
94
+
95
+ ```
96
+ a.payload LONGTEXT CHECK (json_valid(`payload`))
97
+ b.payload VARCHAR(255) CHECK (`payload` in ('x','y'))
98
+
99
+ join result: a → json_valid, a → in(…), b → json_valid, b → in(…)
100
+ ```
101
+
102
+ So `b.payload` reads as `json`, and `a.payload` picks up an enum it does not have. Downstream that is not a cosmetic label: the planner emits a blocked `alter-column-type` with `from: 'json'` that no `.narrowedFrom()` can honestly acknowledge, because the premise is false — and since the data-transfer manifest records introspected types, the same misreading travels into the bundle and reappears as schema drift on import.
103
+
104
+ MariaDB's own `check_constraints` HAS `TABLE_NAME`. The introspector asks for it directly now and keeps the join as the MySQL path, where check-constraint names are schema-unique and the join is sound. That asymmetry is why a single-engine test could not see this: the wrong query passes on MySQL.
105
+ - **@voltro/database** — `voltro db apply` could not drop a CHECK constraint on the mysql family at all, and each of the three reasons hid the next.
106
+
107
+ **1. `DROP CHECK` is MySQL-8 syntax.** MariaDB has never had it — measured on 11.8, `ALTER TABLE t DROP CHECK c` is `ERROR 1064`, while `DROP CONSTRAINT c` works on both engines. A plan containing a `drop-check` therefore died on the first one, on a family whose migrations are NOT atomic: the run stopped with the earlier statements committed and no rollback.
108
+
109
+ **2. The name was assumed, not read.** The applier dropped `<table>_<column>_check` — which is only what its own `add-check` would have named it. A CHECK created at table bring-up is INLINE and UNNAMED, so the server names it (`CONSTRAINT_1` / `<table>_chk_1` / the column name). Dropping a name that does not exist reports "does not exist", which is indistinguishable from the "already gone" a resume legitimately produces — so the statement succeeded, the constraint stayed, and the plan re-proposed the identical `drop-check` forever. The name comes from the catalog now.
110
+
111
+ **3. A column-level CHECK cannot be dropped by name on MariaDB at all.** Measured: the catalog lists it under the column's name, `DROP CONSTRAINT` on that name answers 1091, and only redefining the column removes it. The applier now drops by catalog name, ASKS whether the constraint survived, and redefines the column when it did — so a rebuild happens only in the case that needs one.
112
+
113
+ **And the same investigation closed the MySQL `.oneOf()` round-trip gap.** `parseEnumCheck` was documented as handling MySQL's rendering and did not: MySQL backslash-escapes the string DELIMITERS (`_utf8mb4\'draft\'`), and the parser rewrote those to the SQL doubling `''` — which is how a quote INSIDE a value is written. Every delimiter became escaped content, every value came back empty, and the filter dropped them. `.oneOf()` now round-trips on MySQL, and the suite that asserted the gap as a known one asserts the round-trip instead.
114
+
115
+ Plus, on a failed apply: the error now states how many operations were already applied and whether this dialect rolls back. The ledger held that number; it never reached the operator, who had to re-plan and diff the counts to learn how far the run got.
116
+ - **@voltro/plugin-versioning** — `versioningPlugin({ timing: 'in-transaction' })` built its history row's primary key from `(rowId, version)` while the version counter three lines above was scoped to `(tableName, rowId)`. Two versioned tables carrying the same row id therefore collided — permanently.
117
+
118
+ The shape is not exotic: `actors.id === users.id` is what the framework's own audit trail asks for, an `actors` row whose id is the user's so an audited write satisfies `createdBy → actors`. In an app that follows it, every user row has a twin.
119
+
120
+ The collision does not heal, and that is what turns a duplicate into an outage. The second table's insert fails, so its history row is never written, so `maxOf` for that table stays `null`, so the next attempt computes the same version and the same id. Every write to that row is dead from then on — surfacing as `ER_DUP_ENTRY` on an ordinary `store.update`, naming a row id in a table the caller never wrote to.
121
+
122
+ The key is `(table, rowId, version)` now — the same shape the post-commit path always built. It stays deterministic (no clock, no process-local counter), which is what lets it survive a replica restart; it just carries every part of the key it claims to be unique over.
123
+
124
+ **No cleanup is needed for rows already written.** They keep their old ids and belong to whichever table wrote them; the new keys cannot collide with them, and `byRow` is not unique. An app blocked by this is unblocked by the upgrade alone.
125
+
126
+ Covered twice: the recorder against a port that refuses duplicates (the mechanism, including that a repeat does not settle), and two versioned tables sharing an id against live postgres (the real primary key, inside the caller's transaction). The suite that existed exercised ONE table, which cannot produce a collision at all — and read exactly like a suite that covered this.
127
+
128
+ ### Internal (no consumer-facing effect)
129
+
130
+ - **@voltro/sql-mysql** — Two test-only defects in `sql-mysql`, both found by a release gate, both of the same family: a check that could not fail, and a failure reported in the wrong place. No product code changed.
131
+
132
+ **An assertion that could not fail.** `dropCheckSyntax.integration.test.ts` fell back to a HAND-BUILT plan when the planner produced no operations — and the fabricated operation was a `drop-check`, which is exactly what the next line asserts the plan contains. So an engine whose planner stopped emitting it would have been handed one and reported green. The fallback is deleted; both engines produce the operation now, which is what this release fixed, and the assertion is load-bearing again (4/4 on mysql AND mariadb without it).
133
+
134
+ It surfaced as a TYPE error rather than a false pass, because the fallback's object widened `plan` into a union `applyPlan` does not accept. Worth noting which check caught it: `vitest` transpiles without type-checking, so the suite was green and only `tsc` objected — the gate's `typecheck` and `lint` steps are what went red.
135
+
136
+ **A wait that gave up in silence.** `waitFor` in both binlog CDC suites looped to a deadline and then RETURNED, so a "prove the reader is live" wait that expired let the test carry on, kill the binlog dump thread, and fail twenty lines later on `expect(ids).toContain('todo_wd_before')` — an assertion about a different claim, in a different place. It throws now, naming the wait and the window, and all twelve call sites carry a label.
137
+
138
+ The window is named too: `FIRST_ATTACH_MS = 30_000`, up from 12 s. The reasoning is the 40 s window already in the same file, whose comment says a re-attach plus binlog catch-up takes longer under a loaded full-suite run — a FIRST attach does both and only skips the backoff, so 12 s beside 40 s was an asymmetry the file's own reasoning did not support. That is an argument from the neighbouring comment, not a measurement; if it expires again, `waitFor` now says which wait and for how long, and that number is the one to argue with rather than raising this one twice.
139
+
140
+ ---
141
+
42
142
  ## [0.43.0] — 2026-08-18
43
143
 
44
144
  ### ⚠ BREAKING
package/dist/index.d.ts CHANGED
@@ -1,5 +1,6 @@
1
1
  import { CDC_OFFSETS_TABLE } from '@voltro/database';
2
2
  import { CdcConfig } from '@voltro/database';
3
+ import { ChangeCaptureExclusionRefresh } from '@voltro/database';
3
4
  import { ChangeEvent } from '@voltro/database';
4
5
  import { ChangeStrategy } from '@voltro/database';
5
6
  import { ConfigError } from 'effect';
@@ -31,6 +32,22 @@ export declare interface BinlogCdcHandle {
31
32
  readonly stop: () => Promise<void>;
32
33
  /** Last position the reader has advanced past (for a final checkpoint). */
33
34
  readonly currentPosition: () => BinlogPosition | null;
35
+ /**
36
+ * Replace the undecodable-table set on a LIVE reader.
37
+ *
38
+ * Exists because the set is a property of the SCHEMA, and the schema can move
39
+ * after the reader attached — a boot that migrates in the same process is the
40
+ * ordinary case, not an exotic one. Without this the verdict taken at attach
41
+ * outlives its own cause: a table whose column got bounded seconds later stays
42
+ * excluded for the life of the process.
43
+ *
44
+ * Re-attaches only when the effective set CHANGED, because a re-attach costs a
45
+ * dropped replication connection and a resume — pointless when the answer is
46
+ * the same. Resumes from `currentPosition()`, so no event window is skipped.
47
+ *
48
+ * Returns whether it re-attached.
49
+ */
50
+ readonly setUndecodableTables: (tables: ReadonlyArray<string>) => Promise<boolean>;
34
51
  }
35
52
 
36
53
  export declare interface BinlogCdcOptions {
@@ -254,7 +271,24 @@ export declare class MysqlStore implements DataStore {
254
271
  private inflightTxns;
255
272
  private readonly log;
256
273
  private cdcHandle;
274
+ /** The reader's declared filter + the exclusions currently in force, so a
275
+ * refresh can diff against the decision that is actually running. */
276
+ private cdcIncludeTables;
277
+ private cdcExcluded;
278
+ /** Owns what gets SAID about those exclusions, and when — including the
279
+ * escalation that stands behind `exclusionRefreshFollows`. */
280
+ private readonly cdcReporter;
257
281
  private cdcCheckpointTimer;
282
+ /**
283
+ * A one-connection SQL runtime on the CDC credentials.
284
+ *
285
+ * The store's OWN pool runs as the app user, who does not have `REPLICATION
286
+ * CLIENT` — so `SHOW MASTER STATUS` fails there and every position question
287
+ * degraded to "start at the end". The replication connection has those grants
288
+ * by definition (it could not read a binlog otherwise), so asking over IT is
289
+ * the difference between knowing where the primary is and guessing.
290
+ */
291
+ private cdcAdminRuntime;
258
292
  private cdcPendingPosition;
259
293
  private cdcReplicaId;
260
294
  private cdcStreamName;
@@ -483,7 +517,27 @@ export declare class MysqlStore implements DataStore {
483
517
  /* Excluded from this release type: emitChange */
484
518
  /* Excluded from this release type: startCdcConsumer */
485
519
  /* Excluded from this release type: assertBinlogConfig */
486
- /* Excluded from this release type: resolveBinlogEnd */
520
+ /**
521
+ * Resolve the current binlog (file, position). The statement DIVERGES
522
+ * by engine: MySQL 8.4 REMOVED `SHOW MASTER STATUS` in favour of
523
+ * `SHOW BINARY LOG STATUS`; MariaDB (and mysql < 8.4) only know
524
+ * `SHOW MASTER STATUS`. Pick by variant, then fall back to the other
525
+ * spelling so a mysql < 8.4 or a future rename still resolves. Both
526
+ * return the same `File`/`Position` columns. @internal
527
+ */
528
+ /**
529
+ * Does the server still have this binlog file?
530
+ *
531
+ * `SHOW BINARY LOGS` needs `REPLICATION CLIENT` (or `SUPER`), which is why it
532
+ * runs on the CDC credentials rather than the store's. A probe that cannot run
533
+ * answers `true`: refusing to resume on a question we could not ask would
534
+ * throw away a perfectly good offset, and the reader's existing purge recovery
535
+ * still covers the case we then get wrong.
536
+ */
537
+ private binlogFileExists;
538
+ private resolveBinlogEnd;
539
+ /** The primary's current binlog end, asked over the replication credentials. */
540
+ private resolveBinlogEndOverCdc;
487
541
  /* Excluded from this release type: readCdcOffset */
488
542
  /* Excluded from this release type: flushCdcOffset */
489
543
  /* Excluded from this release type: getInternalExecuteQuery */
@@ -494,6 +548,41 @@ export declare class MysqlStore implements DataStore {
494
548
  /* Excluded from this release type: getInternalExecuteDelete */
495
549
  /* Excluded from this release type: getInternalExecuteUpsert */
496
550
  /* Excluded from this release type: getInternalExecuteInsertIgnore */
551
+ /**
552
+ * Empty `tables` as one unit — see `DataStore.emptyTables`.
553
+ *
554
+ * The engine-specific half is why this cannot live in the importer: MySQL and
555
+ * MariaDB check a foreign key as each ROW is deleted, so no ordering of the
556
+ * TABLES can empty a table that references itself. `actors.createdBy →
557
+ * actors` is exactly that shape, it is the framework's own actor pattern, and
558
+ * it made `--mode replace` fail on the first parent table of any real schema.
559
+ * `FOREIGN_KEY_CHECKS` is the session switch that lets the whole set go at
560
+ * once; the final state satisfies every constraint because the caller reloads
561
+ * all of these tables next, and refuses beforehand if anything OUTSIDE the set
562
+ * points into it.
563
+ *
564
+ * Two details are load-bearing:
565
+ *
566
+ * - The switch is SESSION-scoped, so it must be set on the connection the
567
+ * deletes actually run on — hence one transaction, and hence `sql`
568
+ * directly rather than the transactional VIEW (whose `deleteMany` walks
569
+ * row by row, which is both the wrong shape here and unaffordable at
570
+ * bundle scale).
571
+ * - Restoring it is not optional and not conditional. A connection handed
572
+ * back to the pool with checks off would silently accept dangling
573
+ * references for every later caller, so the restore is a release step
574
+ * that runs on success, on failure and on interrupt alike.
575
+ */
576
+ emptyTables(tables: ReadonlyArray<string>): Promise<void>;
577
+ /** See `DataStore.incomingForeignKeys`. `information_schema` is the catalog on
578
+ * both engines of this family, and `REFERENCED_TABLE_NAME` is only set on the
579
+ * rows that ARE foreign keys — so no join is needed to exclude the rest. */
580
+ incomingForeignKeys(tables: ReadonlyArray<string>): Promise<ReadonlyArray<{
581
+ from: string;
582
+ column: string;
583
+ to: string;
584
+ }>>;
585
+ /* Excluded from this release type: emptyTablesOn */
497
586
  transactional<T>(work: (tx: DataStore) => Promise<T>): Promise<T>;
498
587
  /**
499
588
  * The DIALECT half of the shared transaction bracket
@@ -517,6 +606,18 @@ export declare class MysqlStore implements DataStore {
517
606
  get changeScope(): 'local' | 'fleet';
518
607
  injectExternalChange(event: ChangeEvent): void;
519
608
  run<A, E>(effect: Effect.Effect<A, E, SqlClient.SqlClient>): Promise<A>;
609
+ /**
610
+ * Re-run the undecodable probe against the CURRENT schema and hand the answer
611
+ * to the live reader. See `DataStore.refreshChangeCaptureExclusions`.
612
+ *
613
+ * Diffs in both directions on purpose. Re-admission is the direction a boot
614
+ * that migrates in-process needs; the opposite — schema work that CREATES an
615
+ * unbounded unique on a captured table — is the same staleness with the signs
616
+ * flipped, and it is the one that loses events quietly, since the reader keeps
617
+ * tailing a table it can no longer decode until the reconnect loop diagnoses
618
+ * it three failures later.
619
+ */
620
+ refreshChangeCaptureExclusions(): Promise<ChangeCaptureExclusionRefresh>;
520
621
  close(gracePeriodMs?: number): Promise<void>;
521
622
  /** Liveness probe — `SELECT 1`. Rejects if the pool can't answer. */
522
623
  ping(): Promise<void>;
package/dist/index.js CHANGED
@@ -1,18 +1,18 @@
1
1
  import { MysqlClient as e } from "@effect/sql-mysql2";
2
2
  import { Config as t, Effect as n, Layer as r, ManagedRuntime as i, Option as a, Redacted as o, Schedule as s } from "effect";
3
- import { CDC_OFFSETS_TABLE as c, DEFAULT_ACQUIRE_TIMEOUT_MS as l, EagerCardinalityError as u, _voltroCdcOffsetsTable as d, attachEagerLoads as f, attributionFields as p, attributionKey as m, beginLocalWrite as h, bulkInsertLimitsFor as g, chunkRowsForInsert as _, compileEagerJson as v, compilePredicate as y, compileRawFragment as b, compileSelect as x, decodeRowsFromSchema as S, encodeRowForSchema as C, endLocalWrite as w, externalChangeEvent as T, getTable as E, hasEagerLoads as ee, isTableReactive as D, makeEagerFallbackReporter as te, observeDbOp as O, qualifyTable as ne, raiseChangeListenerCeiling as re, recordsTable as ie, registerPendingAttribution as k, requireTable as A, resolveEchoAttribution as j, runStoreTransaction as ae, runWriteRecorders as oe, stampGeneratedId as M, stampGeneratedIds as se, withCapturedAttribution as N } from "@voltro/database";
4
- import { EventEmitter as ce } from "node:events";
5
- import { createLogger as P } from "@voltro/logger";
6
- import { SqlClient as F, TransactionConnection as I } from "@effect/sql/SqlClient";
3
+ import { CDC_OFFSETS_TABLE as c, DEFAULT_ACQUIRE_TIMEOUT_MS as l, EagerCardinalityError as u, _voltroCdcOffsetsTable as d, attachEagerLoads as f, attributionFields as p, attributionKey as m, beginLocalWrite as h, bulkInsertLimitsFor as g, chunkRowsForInsert as _, compileEagerJson as v, compilePredicate as y, compileRawFragment as b, compileSelect as x, decodeRowsFromSchema as S, encodeRowForSchema as C, endLocalWrite as w, externalChangeEvent as T, getTable as E, hasEagerLoads as D, isTableReactive as O, makeEagerFallbackReporter as k, observeDbOp as A, qualifyTable as ee, raiseChangeListenerCeiling as j, recordsTable as M, registerPendingAttribution as te, requireTable as ne, resolveEchoAttribution as re, runStoreTransaction as ie, runWriteRecorders as ae, stampGeneratedId as N, stampGeneratedIds as oe, withCapturedAttribution as P } from "@voltro/database";
4
+ import { EventEmitter as se } from "node:events";
5
+ import { createLogger as F } from "@voltro/logger";
6
+ import { SqlClient as I, TransactionConnection as L } from "@effect/sql/SqlClient";
7
7
  //#region src/sqlLayer.ts
8
- var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
8
+ var ce = (e) => e ? { rejectUnauthorized: !1 } : void 0, le = (e) => {
9
9
  let t = e.acquireTimeoutMs ?? l;
10
10
  return t > 0 ? t : void 0;
11
- }, L = (e) => {
11
+ }, ue = (e) => {
12
12
  let t = e.acquireQueueLimit;
13
13
  return t !== void 0 && t > 0 ? t : void 0;
14
14
  }, R = (e) => {
15
- let t = e.ssl === void 0 ? void 0 : le(e.ssl), n = ue(e), r = L(e);
15
+ let t = e.ssl === void 0 ? void 0 : ce(e.ssl), n = le(e), r = ue(e);
16
16
  return {
17
17
  ...t === void 0 ? {} : { ssl: t },
18
18
  ...n === void 0 ? {} : { connectTimeout: n },
@@ -70,69 +70,103 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
70
70
  }, H = (e) => z(V(e)), U = (e, t) => e === null ? !1 : t === null ? !0 : e.filename === t.filename ? e.position > t.position : e.filename > t.filename, de = (e) => e.msSinceProgress < e.stallThresholdMs ? "healthy" : e.primary !== null && !U(e.primary, e.reader) ? "idle-caught-up" : "reconnect", fe = (e) => (e instanceof Error ? e.message : String(e ?? "")).includes("schema changed between binlog event and metadata fetch"), pe = (e) => {
71
71
  let t = e instanceof Error ? e.message : String(e ?? "");
72
72
  return /Table\s+[^\s.]+\.(\S+)\s+schema changed between binlog event and metadata fetch/.exec(t)?.[1] ?? null;
73
- }, me = "\n SELECT DISTINCT s.TABLE_NAME AS tableName\n FROM information_schema.STATISTICS s\n JOIN information_schema.COLUMNS c\n ON c.TABLE_SCHEMA = s.TABLE_SCHEMA\n AND c.TABLE_NAME = s.TABLE_NAME\n AND c.COLUMN_NAME = s.COLUMN_NAME\n WHERE s.TABLE_SCHEMA = DATABASE()\n AND s.NON_UNIQUE = 0\n AND s.SUB_PART IS NULL\n AND c.DATA_TYPE IN ('text','tinytext','mediumtext','longtext','blob','tinyblob','mediumblob','longblob')\n", W = (e) => `cdc: table '${e}' is EXCLUDED from binlog capture — its row image carries a hidden column the reader cannot account for. Cause: a UNIQUE constraint on an UNBOUNDED text column, which MariaDB backs with a HASH long-unique index; that index adds a hidden DB_ROW_HASH_n column to the row, present in the binlog and absent from information_schema.COLUMNS. Remedy: bound the column — text().maxLength(n) — so the constraint becomes an ordinary B-tree index with no hidden column. ALTER TABLE FORCE does NOT help: the rebuild recreates the index and the hidden column. Until then, cross-instance change events for this table are lost; own-node reactivity is unaffected (writes still emit inline).`, he = 3e5, ge = 3, _e = (e, t) => {
74
- let n = [...e.filter((e) => t - e < he), t];
73
+ }, me = "\n SELECT DISTINCT s.TABLE_NAME AS tableName\n FROM information_schema.STATISTICS s\n JOIN information_schema.COLUMNS c\n ON c.TABLE_SCHEMA = s.TABLE_SCHEMA\n AND c.TABLE_NAME = s.TABLE_NAME\n AND c.COLUMN_NAME = s.COLUMN_NAME\n WHERE s.TABLE_SCHEMA = DATABASE()\n AND s.NON_UNIQUE = 0\n AND s.SUB_PART IS NULL\n AND c.DATA_TYPE IN ('text','tinytext','mediumtext','longtext','blob','tinyblob','mediumblob','longblob')\n", he = "This verdict is the schema as read when the reader attached; applying the remedy does not lift it by itself. A boot that migrates in-process re-checks it once its schema work is done and re-admits the table in the same run; otherwise it holds until the process restarts.", W = (e) => `cdc: table '${e}' is EXCLUDED from binlog capture — its row image carries a hidden column the reader cannot account for. Cause: a UNIQUE constraint on an UNBOUNDED text column, which MariaDB backs with a HASH long-unique index; that index adds a hidden DB_ROW_HASH_n column to the row, present in the binlog and absent from information_schema.COLUMNS. Remedy: bound the column — text().maxLength(n) — so the constraint becomes an ordinary B-tree index with no hidden column. ALTER TABLE FORCE does NOT help: the rebuild recreates the index and the hidden column. Until then, cross-instance change events for this table are lost; own-node reactivity is unaffected (writes still emit inline). ${he}`, ge = (e) => `cdc: table '${e}' looks undecodable (a UNIQUE on an UNBOUNDED text column → MariaDB hash long-unique → a hidden DB_ROW_HASH_n column in the row image). Held OUT of binlog capture for now and re-checked after this boot's schema work — a migration that bounds the column re-admits it in the same run. Not a verdict yet; the definitive line follows.`, _e = (e) => `cdc: table '${e}' re-admitted to binlog capture — the hidden-column condition is gone (the schema moved since the reader attached). Cross-instance change events for it flow again.`, ve = 12e4, ye = (e, t = ve) => {
74
+ let n = null, r = [], i = () => {
75
+ n &&= (clearTimeout(n), null), r = [];
76
+ };
77
+ return {
78
+ announce: (a, o) => {
79
+ if (i(), a.length !== 0) {
80
+ if (!o) {
81
+ for (let t of a) e("error", W(t));
82
+ return;
83
+ }
84
+ for (let t of a) e("warn", ge(t));
85
+ r = a, n = setTimeout(() => {
86
+ let i = r;
87
+ n = null, r = [];
88
+ for (let n of i) e("error", `${W(n)} (Held provisionally for ${Math.round(t / 1e3)}s awaiting a post-migration re-check that never ran; standing by the reading taken at attach.)`);
89
+ }, t), n.unref && n.unref();
90
+ }
91
+ },
92
+ settle: (t) => {
93
+ i();
94
+ for (let n of t.stillExcluded) e("error", W(n));
95
+ for (let n of t.newlyExcluded) e("error", W(n));
96
+ for (let n of t.readmitted) e("info", _e(n));
97
+ },
98
+ dispose: i
99
+ };
100
+ }, be = (e, t) => {
101
+ let n = new Set(e), r = new Set(t);
75
102
  return {
76
- verdict: n.length >= ge ? "persistent" : "backlog",
103
+ stillExcluded: t.filter((e) => n.has(e)),
104
+ readmitted: e.filter((e) => !r.has(e)),
105
+ newlyExcluded: t.filter((e) => !n.has(e))
106
+ };
107
+ }, xe = 3e5, Se = 3, Ce = (e, t) => {
108
+ let n = [...e.filter((e) => t - e < xe), t];
109
+ return {
110
+ verdict: n.length >= Se ? "persistent" : "backlog",
77
111
  hits: n
78
112
  };
79
- }, ve = /* @__PURE__ */ new Set([
113
+ }, we = /* @__PURE__ */ new Set([
80
114
  "writerows",
81
115
  "updaterows",
82
116
  "deleterows"
83
- ]), ye = /\b(alter|rename|drop|create)\s+(table|column)?/i, G = (e) => new Promise((t) => setTimeout(t, e)), K = async (e) => {
84
- let t = P({ scope: `voltro:${e.variant}:cdc` }), n;
117
+ ]), Te = /\b(alter|rename|drop|create)\s+(table|column)?/i, G = (e) => new Promise((t) => setTimeout(t, e)), K = async (e) => {
118
+ let t = F({ scope: `voltro:${e.variant}:cdc` }), n;
85
119
  try {
86
120
  n = (await import("@vlasky/zongji")).default;
87
121
  } catch (t) {
88
122
  throw Error(`changeStrategy='cdc' on ${e.variant} requires the '@vlasky/zongji' optional dependency. Install it (it ships as an optionalDependency of @voltro/sql-mysql) to enable binlog CDC.`, { cause: t });
89
123
  }
90
- let r = new Set(e.undecodableTables ?? []), i = e.includeTables ? new Set(e.includeTables.filter((e) => !r.has(e))) : null, a = e.connection.database, o = e.startPosition?.filename ?? null, s = e.startPosition ?? null, c = (t, n) => S([n], t, e.variant)[0], l = null, u = !1, d = 0, f = Date.now(), p = !1, m = /* @__PURE__ */ new Map(), h = /* @__PURE__ */ new Set(), g = null, _ = e.stallThresholdMs ?? 25e3, v = e.watchdogIntervalMs ?? 15e3, y = () => {
124
+ let r = new Set(e.undecodableTables ?? []), i = e.includeTables ? new Set(e.includeTables) : null, a = e.connection.database, o = e.startPosition?.filename ?? null, s = e.startPosition ?? null, c = (t, n) => S([n], t, e.variant)[0], l = null, u = !1, d = 0, f = Date.now(), p = !1, m = null, h = /* @__PURE__ */ new Map(), g = /* @__PURE__ */ new Set(), _ = null, v = e.stallThresholdMs ?? 25e3, y = e.watchdogIntervalMs ?? 15e3, b = () => {
91
125
  f = Date.now();
92
- }, b = (n) => {
93
- y();
94
- let r = n.getEventName();
95
- if (r === "rotate" && n.binlogName) {
126
+ }, x = (n) => {
127
+ b();
128
+ let u = n.getEventName();
129
+ if (u === "rotate" && n.binlogName) {
96
130
  o = n.binlogName;
97
131
  return;
98
132
  }
99
- if (r === "query" && n.query && ye.test(n.query)) {
133
+ if (u === "query" && n.query && Te.test(n.query)) {
100
134
  l && (l.tableMap = {});
101
135
  return;
102
136
  }
103
137
  if (n.nextPosition && o && (s = {
104
138
  filename: o,
105
139
  position: n.nextPosition
106
- }, e.onPosition?.(s)), !ve.has(r)) return;
107
- let u = n.tableMap[n.tableId];
108
- if (!u || u.parentSchema !== a) return;
109
- let d = u.tableName;
110
- if (!d.startsWith("_voltro_") && !(i && !i.has(d))) try {
111
- if (r === "writerows") for (let t of n.rows) e.onChange({
112
- table: d,
140
+ }, e.onPosition?.(s)), !we.has(u)) return;
141
+ let d = n.tableMap[n.tableId];
142
+ if (!d || d.parentSchema !== a) return;
143
+ let f = d.tableName;
144
+ if (!f.startsWith("_voltro_") && !(i && !i.has(f)) && !r.has(f)) try {
145
+ if (u === "writerows") for (let t of n.rows) e.onChange({
146
+ table: f,
113
147
  op: "insert",
114
148
  old: null,
115
- new: c(d, t)
149
+ new: c(f, t)
116
150
  });
117
- else if (r === "deleterows") for (let t of n.rows) e.onChange({
118
- table: d,
151
+ else if (u === "deleterows") for (let t of n.rows) e.onChange({
152
+ table: f,
119
153
  op: "delete",
120
- old: c(d, t),
154
+ old: c(f, t),
121
155
  new: null
122
156
  });
123
157
  else for (let t of n.rows) e.onChange({
124
- table: d,
158
+ table: f,
125
159
  op: "update",
126
- old: c(d, t.before),
127
- new: c(d, t.after)
160
+ old: c(f, t.before),
161
+ new: c(f, t.after)
128
162
  });
129
163
  } catch (n) {
130
164
  t.warn("cdc: failed to map a row event", {
131
- table: d,
132
- name: r
165
+ table: f,
166
+ name: u
133
167
  }, n), e.onError?.(n);
134
168
  }
135
- }, x = (t) => {
169
+ }, C = (t) => {
136
170
  let n = {
137
171
  serverId: e.serverId,
138
172
  includeEvents: [
@@ -146,10 +180,10 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
146
180
  ]
147
181
  };
148
182
  return e.includeTables && e.includeTables.length > 0 && (n.includeSchema = { [a]: e.includeTables.filter((e) => !r.has(e)) }), t ? (n.filename = t.filename, n.position = t.position) : n.startAtEnd = !0, n;
149
- }, C = (e) => {
183
+ }, w = (e) => {
150
184
  let t = e;
151
185
  return t?.errno === 1236 || String(t?.code ?? "").includes("ER_MASTER_FATAL_ERROR_READING_BINLOG");
152
- }, w = (r) => new Promise((i, a) => {
186
+ }, T = (r) => new Promise((i, a) => {
153
187
  let o = new n({
154
188
  host: e.connection.host,
155
189
  port: e.connection.port,
@@ -160,8 +194,8 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
160
194
  });
161
195
  l = o;
162
196
  let s = !1;
163
- o.on("binlog", b), o.on("ready", () => {
164
- y(), !s && (s = !0, d = 0, t.info("cdc: binlog reader attached", {
197
+ o.on("binlog", x), o.on("ready", () => {
198
+ b(), !s && (s = !0, d = 0, t.info("cdc: binlog reader attached", {
165
199
  serverId: e.serverId,
166
200
  from: r ?? "current-end"
167
201
  }), i());
@@ -170,82 +204,101 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
170
204
  s = !0, a(e instanceof Error ? e : Error(String(e)));
171
205
  return;
172
206
  }
173
- T("binlog reader error", e);
174
- }), o.start(x(r));
175
- }), T = async (n, i) => {
176
- if (!(u || p)) {
177
- p = !0, t.warn(`cdc: reconnecting — ${n}`, {}, i instanceof Error ? i : void 0), e.onError?.(i);
178
- try {
179
- let n = i;
180
- for (; !u;) {
181
- try {
182
- l?.stop();
183
- } catch {}
184
- if (d++, await G(Math.min(3e4, 500 * 2 ** Math.min(d, 6))), u) return;
185
- let i = s, a = C(n), c = !a && fe(n), f = !1;
186
- if (c) {
187
- let e = pe(n), i = e ?? "<unknown>", { verdict: a, hits: o } = _e(m.get(i) ?? [], Date.now());
188
- m.set(i, o), f = a === "persistent", f && !h.has(i) && (h.add(i), e !== null && r.add(e), t.error(W(i)));
189
- }
190
- (a || c) && (f || t.warn(c ? "cdc: un-replayable backlog event (schema moved past it) — jumping to current end + self-heal" : "cdc: binlog gap (purged/failover) — jumping to current end + self-heal"), i = e.resolveStartPosition ? await e.resolveStartPosition().catch(() => null) : null, o = i?.filename ?? null, f || e.onResync?.());
191
- try {
192
- await w(i), d = 0, y();
193
- return;
194
- } catch (e) {
195
- n = e;
196
- }
207
+ D("binlog reader error", e);
208
+ }), o.start(C(r));
209
+ }), E = async (n, i) => {
210
+ p = !0, t.warn(`cdc: reconnecting — ${n}`, {}, i instanceof Error ? i : void 0), e.onError?.(i);
211
+ try {
212
+ let n = i;
213
+ for (; !u;) {
214
+ try {
215
+ l?.stop();
216
+ } catch {}
217
+ if (d++, await G(Math.min(3e4, 500 * 2 ** Math.min(d, 6))), u) return;
218
+ let i = s, a = w(n), c = !a && fe(n), f = !1;
219
+ if (c) {
220
+ let e = pe(n), i = e ?? "<unknown>", { verdict: a, hits: o } = Ce(h.get(i) ?? [], Date.now());
221
+ h.set(i, o), f = a === "persistent", f && !g.has(i) && (g.add(i), e !== null && r.add(e), t.error(W(i)));
222
+ }
223
+ (a || c) && (f || t.warn(c ? "cdc: un-replayable backlog event (schema moved past it) — jumping to current end + self-heal" : "cdc: binlog gap (purged/failover) — jumping to current end + self-heal"), i = e.resolveStartPosition ? await e.resolveStartPosition().catch(() => null) : null, o = i?.filename ?? null, s = i, f || e.onResync?.());
224
+ try {
225
+ await T(i), d = 0, b();
226
+ return;
227
+ } catch (e) {
228
+ n = e;
197
229
  }
198
- } finally {
199
- p = !1;
200
230
  }
231
+ } finally {
232
+ p = !1;
201
233
  }
202
- };
234
+ }, D = (e, t) => u ? Promise.resolve() : p ? m ?? Promise.resolve() : (m = E(e, t).finally(() => {
235
+ m = null;
236
+ }), m);
203
237
  try {
204
- await w(e.startPosition ?? null);
238
+ await T(e.startPosition ?? null);
205
239
  } catch (n) {
206
- if (C(n) && e.startPosition) {
240
+ if (w(n) && e.startPosition) {
207
241
  t.warn("cdc: persisted offset purged — starting at current end + self-heal");
208
242
  try {
209
243
  l?.stop();
210
244
  } catch {}
211
245
  let n = e.resolveStartPosition ? await e.resolveStartPosition().catch(() => null) : null;
212
- o = n?.filename ?? null, e.onResync?.(), await G(500), await w(n);
246
+ o = n?.filename ?? null, s = n, e.onResync?.(), await G(500), await T(n);
213
247
  } else throw n;
214
248
  }
215
- let E = async () => {
216
- if (u || p || Date.now() - f < _) return;
249
+ let O = async () => {
250
+ if (u || p || Date.now() - f < v) return;
217
251
  let n = null;
218
252
  if (e.resolveStartPosition && (n = await e.resolveStartPosition().catch(() => null)), u || p) return;
219
253
  let r = de({
220
254
  msSinceProgress: Date.now() - f,
221
- stallThresholdMs: _,
255
+ stallThresholdMs: v,
222
256
  primary: n,
223
257
  reader: s
224
258
  });
225
259
  if (r !== "reconnect") {
226
- r === "idle-caught-up" && y();
260
+ r === "idle-caught-up" && b();
227
261
  return;
228
262
  }
229
263
  t.warn("cdc: watchdog detected a stalled reader — forcing reconnect", {
230
264
  reader: s,
231
265
  primary: n,
232
266
  stalledForMs: Date.now() - f
233
- }), T("watchdog: stream stalled (no events while primary advanced)", /* @__PURE__ */ Error("cdc watchdog stall"));
267
+ }), D("watchdog: stream stalled (no events while primary advanced)", /* @__PURE__ */ Error("cdc watchdog stall"));
234
268
  };
235
- return g = setInterval(() => {
236
- E();
237
- }, v), g.unref && g.unref(), {
269
+ return _ = setInterval(() => {
270
+ O();
271
+ }, y), _.unref && _.unref(), {
238
272
  stop: async () => {
239
- u = !0, g &&= (clearInterval(g), null);
273
+ u = !0, _ &&= (clearInterval(_), null);
240
274
  try {
241
275
  l?.stop();
242
276
  } catch (e) {
243
277
  t.warn("cdc: error stopping binlog reader", {}, e);
244
278
  }
245
279
  },
246
- currentPosition: () => s
280
+ currentPosition: () => s,
281
+ setUndecodableTables: async (t) => {
282
+ let n = new Set(t);
283
+ if (n.size === r.size && [...n].every((e) => r.has(e))) return !1;
284
+ r.clear();
285
+ for (let e of n) r.add(e);
286
+ if (u) return !1;
287
+ if (p) return await (m ?? Promise.resolve()), !0;
288
+ p = !0;
289
+ try {
290
+ try {
291
+ l?.stop();
292
+ } catch {}
293
+ return await G(500), !u && (s === null && e.onResync?.(), await T(s), b(), !0);
294
+ } catch (e) {
295
+ return p = !1, await D("re-attach after an exclusion refresh failed", e), !1;
296
+ } finally {
297
+ p = !1;
298
+ }
299
+ }
247
300
  };
248
- }, be = /* @__PURE__ */ new Set(["1213", "1205"]), xe = (e) => {
301
+ }, Ee = /* @__PURE__ */ new Set(["1213", "1205"]), De = (e) => {
249
302
  let t = e;
250
303
  for (let e = 0; e < 5 && typeof t == "object" && t; e++) {
251
304
  let e = t.errno;
@@ -255,8 +308,8 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
255
308
  t = t.cause;
256
309
  }
257
310
  }, q = (e) => {
258
- let t = xe(e);
259
- return t !== void 0 && be.has(t);
311
+ let t = De(e);
312
+ return t !== void 0 && Ee.has(t);
260
313
  }, J = (e) => q(e) ? "retry" : "noRetry", Y = (e) => {
261
314
  if (e == null) return "null";
262
315
  let t = typeof e;
@@ -266,19 +319,19 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
266
319
  if (Array.isArray(e)) return `[${e.map(Y).join(",")}]`;
267
320
  let n = e;
268
321
  return `{${Object.keys(n).sort().map((e) => `${JSON.stringify(e)}:${Y(n[e])}`).join(",")}}`;
269
- }, Se = (e) => {
322
+ }, Oe = (e) => {
270
323
  let t = Y(e), n = 2166136261;
271
324
  for (let e = 0; e < t.length; e++) n ^= t.charCodeAt(e), n = Math.imul(n, 16777619);
272
325
  return (n >>> 0).toString(36);
273
- }, Ce = (e, t) => {
326
+ }, ke = (e, t) => {
274
327
  let n = setTimeout(e, t);
275
328
  typeof n.unref == "function" && n.unref();
276
- }, we = class {
329
+ }, Ae = class {
277
330
  variant;
278
331
  ttlMs;
279
332
  schedule;
280
333
  seen = /* @__PURE__ */ new Map();
281
- constructor(e, t = 6e4, n = Ce) {
334
+ constructor(e, t = 6e4, n = ke) {
282
335
  this.variant = e, this.ttlMs = t, this.schedule = n;
283
336
  }
284
337
  key(e) {
@@ -292,7 +345,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
292
345
  } catch {
293
346
  r = t;
294
347
  }
295
- return `${e.table} ${e.op} ${String(n)} ${Se(r)}`;
348
+ return `${e.table} ${e.op} ${String(n)} ${Oe(r)}`;
296
349
  }
297
350
  admit(e) {
298
351
  let t = this.key(e);
@@ -312,9 +365,9 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
312
365
  1062,
313
366
  1586
314
367
  ]), Z = async (e) => {
315
- let t = e.variant ?? "mysql", n = P({ scope: `voltro:${t}` }), a = e.changeStrategy ?? "inline", o = a;
368
+ let t = e.variant ?? "mysql", n = F({ scope: `voltro:${t}` }), a = e.changeStrategy ?? "inline", o = a;
316
369
  a === "cdc" && !e.cdcConfig && (n.warn("changeStrategy='cdc' requires cdcConfig (serverId + connection); falling back to 'inline'."), o = "inline");
317
- let s = e.tracerLayer ? r.mergeAll(e.sqlLayer, e.tracerLayer) : e.sqlLayer, c = i.make(s), l = new Q(await c.runPromise(F), c, t, o);
370
+ let s = e.tracerLayer ? r.mergeAll(e.sqlLayer, e.tracerLayer) : e.sqlLayer, c = i.make(s), l = new Q(await c.runPromise(I), c, t, o);
318
371
  return o === "cdc" && e.cdcConfig && await l.startCdcConsumer(e.cdcConfig), l;
319
372
  }, Q = class e {
320
373
  sql;
@@ -326,20 +379,26 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
326
379
  inflightTxns = 0;
327
380
  log;
328
381
  cdcHandle = null;
382
+ cdcIncludeTables = void 0;
383
+ cdcExcluded = [];
384
+ cdcReporter;
329
385
  cdcCheckpointTimer = null;
386
+ cdcAdminRuntime = null;
330
387
  cdcPendingPosition = null;
331
388
  cdcReplicaId = "";
332
389
  cdcStreamName = "default";
333
390
  cdcGate;
334
391
  reportEagerFallback;
335
392
  constructor(e, t, n, r = "inline", i = null, a, o) {
336
- this.sql = e, this.runtime = t, this.variant = n, this.changeStrategy = r, this.namespace = i, this.log = P({ scope: `voltro:${n}` }), this.reportEagerFallback = te(this.log), this.emitter = a ?? new ce(), re(this.emitter), this.cdcGate = o ?? new we(n);
393
+ this.sql = e, this.runtime = t, this.variant = n, this.changeStrategy = r, this.namespace = i, this.log = F({ scope: `voltro:${n}` }), this.cdcReporter = ye((e, t) => {
394
+ e === "error" ? this.log.error(t) : e === "warn" ? this.log.warn(t) : this.log.info(t);
395
+ }), this.reportEagerFallback = k(this.log), this.emitter = a ?? new se(), j(this.emitter), this.cdcGate = o ?? new Ae(n);
337
396
  }
338
397
  withNamespace(t) {
339
398
  return t === this.namespace ? this : new e(this.sql, this.runtime, this.variant, this.changeStrategy, t, this.emitter, this.cdcGate);
340
399
  }
341
400
  nsT(e) {
342
- return ne(this.namespace, e);
401
+ return ee(this.namespace, e);
343
402
  }
344
403
  get dialectId() {
345
404
  return this.variant;
@@ -351,7 +410,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
351
410
  };
352
411
  }
353
412
  async executeQuery(e, t, r) {
354
- let i = x(e, this.sql, this.namespace), a = t ? n.provideService(i, I, t) : i, o = await O(this.variant, "select", () => this.runtime.runPromise(a));
413
+ let i = x(e, this.sql, this.namespace), a = t ? n.provideService(i, L, t) : i, o = await A(this.variant, "select", () => this.runtime.runPromise(a));
355
414
  return S(o, e.table, this.variant);
356
415
  }
357
416
  get supportsInsertReturning() {
@@ -364,10 +423,10 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
364
423
  return !1;
365
424
  }
366
425
  async executeInsert(e, t, r, i, a) {
367
- t = M(e, t);
426
+ t = N(e, t);
368
427
  let o = this.sql;
369
428
  if (this.supportsInsertReturning) {
370
- let s = o`INSERT INTO ${o(this.nsT(e))} ${o.insert(C(t, e))} RETURNING *`, c = r ? n.provideService(s, I, r) : s, l = (await this.runtime.runPromise(c))[0];
429
+ let s = o`INSERT INTO ${o(this.nsT(e))} ${o.insert(C(t, e))} RETURNING *`, c = r ? n.provideService(s, L, r) : s, l = (await this.runtime.runPromise(c))[0];
371
430
  if (!l) throw Error(`MysqlStore.insert: no row returned for table '${e}'`);
372
431
  return await this.routeEvent({
373
432
  table: e,
@@ -386,9 +445,9 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
386
445
  new: t
387
446
  }, i, r, a), t;
388
447
  }
389
- let l = o`INSERT INTO ${o(this.nsT(e))} ${o.insert(s)}`, u = r ? n.provideService(l, I, r) : l;
448
+ let l = o`INSERT INTO ${o(this.nsT(e))} ${o.insert(s)}`, u = r ? n.provideService(l, L, r) : l;
390
449
  await this.runtime.runPromise(u);
391
- let d = o`SELECT * FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${c}`, f = r ? n.provideService(d, I, r) : d, p = (await this.runtime.runPromise(f))[0];
450
+ let d = o`SELECT * FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${c}`, f = r ? n.provideService(d, L, r) : d, p = (await this.runtime.runPromise(f))[0];
392
451
  if (!p) throw Error(`MysqlStore.insert: row not found post-insert in '${e}'`);
393
452
  return await this.routeEvent({
394
453
  table: e,
@@ -401,16 +460,16 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
401
460
  let i = this.sql;
402
461
  return this.runPinned(r, (r) => n.gen(this, function* () {
403
462
  let a = i`INSERT INTO ${i(this.nsT(e))} ${i.insert(t)}`;
404
- yield* n.provideService(a, I, r);
405
- let o = (yield* n.provideService(i`SELECT LAST_INSERT_ID() AS ${i("lastId")}`, I, r))[0]?.lastId;
406
- return o === void 0 || Number(o) === 0 ? yield* n.fail(/* @__PURE__ */ Error(`MysqlStore.insert: LAST_INSERT_ID() returned no id for '${e}' — is the primary key AUTO_INCREMENT?`)) : (yield* n.provideService(i`SELECT * FROM ${i(this.nsT(e))} WHERE ${i("id")} = ${o}`, I, r))[0] || (yield* n.fail(/* @__PURE__ */ Error(`MysqlStore.insert: row not found post-insert in '${e}'`)));
463
+ yield* n.provideService(a, L, r);
464
+ let o = (yield* n.provideService(i`SELECT LAST_INSERT_ID() AS ${i("lastId")}`, L, r))[0]?.lastId;
465
+ return o === void 0 || Number(o) === 0 ? yield* n.fail(/* @__PURE__ */ Error(`MysqlStore.insert: LAST_INSERT_ID() returned no id for '${e}' — is the primary key AUTO_INCREMENT?`)) : (yield* n.provideService(i`SELECT * FROM ${i(this.nsT(e))} WHERE ${i("id")} = ${o}`, L, r))[0] || (yield* n.fail(/* @__PURE__ */ Error(`MysqlStore.insert: row not found post-insert in '${e}'`)));
407
466
  }), "insert");
408
467
  }
409
468
  async runPinned(e, t, r) {
410
469
  if (e) return this.runtime.runPromise(t(e));
411
470
  this.inflightTxns++;
412
471
  try {
413
- let e = n.suspend(() => this.sql.withTransaction(n.flatMap(n.serviceOption(I), (e) => a.isNone(e) ? n.fail(/* @__PURE__ */ Error(`MysqlStore.${r}: TransactionConnection missing.`)) : t(e.value)))), i = s.exponential("10 millis").pipe(s.compose(s.recurs(3)), s.whileInput(q)), o = e.pipe(n.retry(i), n.withSpan(`store.${r}`, { attributes: {
472
+ let e = n.suspend(() => this.sql.withTransaction(n.flatMap(n.serviceOption(L), (e) => a.isNone(e) ? n.fail(/* @__PURE__ */ Error(`MysqlStore.${r}: TransactionConnection missing.`)) : t(e.value)))), i = s.exponential("10 millis").pipe(s.compose(s.recurs(3)), s.whileInput(q)), o = e.pipe(n.retry(i), n.withSpan(`store.${r}`, { attributes: {
414
473
  "db.system": this.variant,
415
474
  "db.operation": r
416
475
  } }));
@@ -420,16 +479,16 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
420
479
  }
421
480
  }
422
481
  async executeInsertMany(e, t, r, i, a) {
423
- if (t = se(e, t), t.length === 0) return [];
482
+ if (t = oe(e, t), t.length === 0) return [];
424
483
  let o = this.sql, s = t.map((t) => C(t, e)), c = _(s, g(this.variant));
425
484
  if (this.supportsInsertReturning) {
426
485
  let t = (t) => o`INSERT INTO ${o(this.nsT(e))} ${o.insert(t)} RETURNING *`, s;
427
486
  if (c.length === 1) {
428
- let e = t(c[0]), i = r ? n.provideService(e, I, r) : e;
487
+ let e = t(c[0]), i = r ? n.provideService(e, L, r) : e;
429
488
  s = await this.runtime.runPromise(i);
430
489
  } else if (r) {
431
490
  let e = [];
432
- for (let i of c) e.push(...await this.runtime.runPromise(n.provideService(t(i), I, r)));
491
+ for (let i of c) e.push(...await this.runtime.runPromise(n.provideService(t(i), L, r)));
433
492
  s = e;
434
493
  } else s = await this.runtime.runPromise(o.withTransaction(n.map(n.forEach(c, t, { concurrency: 1 }), (e) => e.flat())));
435
494
  for (let t of s) await this.routeEvent({
@@ -454,16 +513,16 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
454
513
  if (l.some((e) => e === void 0)) throw Error("MysqlStore.insertMany: rows mix client-supplied and missing 'id's — supply an id for every row or none (AUTO_INCREMENT recovery).");
455
514
  let u = (t) => o`INSERT INTO ${o(this.nsT(e))} ${o.insert(t)}`;
456
515
  if (c.length === 1) {
457
- let e = r ? n.provideService(u(c[0]), I, r) : u(c[0]);
516
+ let e = r ? n.provideService(u(c[0]), L, r) : u(c[0]);
458
517
  await this.runtime.runPromise(e);
459
- } else if (r) for (let e of c) await this.runtime.runPromise(n.provideService(u(e), I, r));
518
+ } else if (r) for (let e of c) await this.runtime.runPromise(n.provideService(u(e), L, r));
460
519
  else await this.runtime.runPromise(o.withTransaction(n.forEach(c, u, {
461
520
  concurrency: 1,
462
521
  discard: !0
463
522
  })));
464
523
  let d = _(l.map((e) => ({ id: e })), g(this.variant)), f = [];
465
524
  for (let t of d) {
466
- let i = o`SELECT * FROM ${o(this.nsT(e))} WHERE ${o("id")} IN ${o.in(t.map((e) => e.id))}`, a = r ? n.provideService(i, I, r) : i;
525
+ let i = o`SELECT * FROM ${o(this.nsT(e))} WHERE ${o("id")} IN ${o.in(t.map((e) => e.id))}`, a = r ? n.provideService(i, L, r) : i;
467
526
  f.push(...await this.runtime.runPromise(a));
468
527
  }
469
528
  let p = new Map(f.map((e) => [e.id, e])), m = l.map((e) => p.get(e)).filter((e) => e !== void 0);
@@ -481,19 +540,19 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
481
540
  let a = [];
482
541
  for (let o of t) {
483
542
  let t = i`INSERT INTO ${i(this.nsT(e))} ${i.insert(o)}`;
484
- yield* n.provideService(t, I, r);
485
- let s = (yield* n.provideService(i`SELECT LAST_INSERT_ID() AS ${i("firstId")}`, I, r))[0]?.firstId;
543
+ yield* n.provideService(t, L, r);
544
+ let s = (yield* n.provideService(i`SELECT LAST_INSERT_ID() AS ${i("firstId")}`, L, r))[0]?.firstId;
486
545
  if (s === void 0 || Number(s) === 0) return yield* n.fail(/* @__PURE__ */ Error(`MysqlStore.insertMany: LAST_INSERT_ID() returned no id for '${e}' — is the primary key AUTO_INCREMENT?`));
487
- let c = Number(s), l = yield* n.provideService(i`SELECT * FROM ${i(this.nsT(e))} WHERE ${i("id")} BETWEEN ${c} AND ${c + o.length - 1} ORDER BY ${i("id")} ASC`, I, r);
546
+ let c = Number(s), l = yield* n.provideService(i`SELECT * FROM ${i(this.nsT(e))} WHERE ${i("id")} BETWEEN ${c} AND ${c + o.length - 1} ORDER BY ${i("id")} ASC`, L, r);
488
547
  a.push(...l);
489
548
  }
490
549
  return a;
491
550
  }), "insert");
492
551
  }
493
552
  async executePatchJson(e, t, r, i, a, o, s) {
494
- let c = this.sql, l = r.split("."), u = l[0], d = l.slice(1), f = JSON.stringify(i ?? null), p = d.length === 0 ? "$" : `$.${d.join(".")}`, m = c`UPDATE ${c(this.nsT(e))} SET ${c(u)} = JSON_SET(COALESCE(${c(u)}, '{}'), ${p}, CAST(${f} AS JSON)) WHERE ${c("id")} = ${t}`, h = a ? n.provideService(m, I, a) : m, g = await this.runtime.runPromise(h);
553
+ let c = this.sql, l = r.split("."), u = l[0], d = l.slice(1), f = JSON.stringify(i ?? null), p = d.length === 0 ? "$" : `$.${d.join(".")}`, m = c`UPDATE ${c(this.nsT(e))} SET ${c(u)} = JSON_SET(COALESCE(${c(u)}, '{}'), ${p}, CAST(${f} AS JSON)) WHERE ${c("id")} = ${t}`, h = a ? n.provideService(m, L, a) : m, g = await this.runtime.runPromise(h);
495
554
  if (g && typeof g.affectedRows == "number" && g.affectedRows === 0) return null;
496
- let _ = c`SELECT * FROM ${c(this.nsT(e))} WHERE ${c("id")} = ${t}`, v = a ? n.provideService(_, I, a) : _, y = (await this.runtime.runPromise(v))[0];
555
+ let _ = c`SELECT * FROM ${c(this.nsT(e))} WHERE ${c("id")} = ${t}`, v = a ? n.provideService(_, L, a) : _, y = (await this.runtime.runPromise(v))[0];
497
556
  return y ? (await this.routeEvent({
498
557
  table: e,
499
558
  op: "update",
@@ -504,7 +563,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
504
563
  async executeUpdate(e, t, r, i, a, o) {
505
564
  let s = this.sql;
506
565
  if (this.supportsUpdateReturning) {
507
- let c = s`UPDATE ${s(this.nsT(e))} SET ${s.update(C(r, e))} WHERE ${s("id")} = ${t} RETURNING *`, l = i ? n.provideService(c, I, i) : c, u = (await this.runtime.runPromise(l))[0];
566
+ let c = s`UPDATE ${s(this.nsT(e))} SET ${s.update(C(r, e))} WHERE ${s("id")} = ${t} RETURNING *`, l = i ? n.provideService(c, L, i) : c, u = (await this.runtime.runPromise(l))[0];
508
567
  return u ? (await this.routeEvent({
509
568
  table: e,
510
569
  op: "update",
@@ -512,9 +571,9 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
512
571
  new: u
513
572
  }, a, i, o), u) : null;
514
573
  }
515
- let c = s`UPDATE ${s(this.nsT(e))} SET ${s.update(C(r, e))} WHERE ${s("id")} = ${t}`, l = i ? n.provideService(c, I, i) : c, u = await this.runtime.runPromise(l);
574
+ let c = s`UPDATE ${s(this.nsT(e))} SET ${s.update(C(r, e))} WHERE ${s("id")} = ${t}`, l = i ? n.provideService(c, L, i) : c, u = await this.runtime.runPromise(l);
516
575
  if (u && typeof u.affectedRows == "number" && u.affectedRows === 0) return null;
517
- let d = s`SELECT * FROM ${s(this.nsT(e))} WHERE ${s("id")} = ${t}`, f = i ? n.provideService(d, I, i) : d, p = (await this.runtime.runPromise(f))[0];
576
+ let d = s`SELECT * FROM ${s(this.nsT(e))} WHERE ${s("id")} = ${t}`, f = i ? n.provideService(d, L, i) : d, p = (await this.runtime.runPromise(f))[0];
518
577
  return p ? (await this.routeEvent({
519
578
  table: e,
520
579
  op: "update",
@@ -525,7 +584,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
525
584
  async executeDelete(e, t, r, i, a) {
526
585
  let o = this.sql;
527
586
  if (this.supportsDeleteReturning) {
528
- let s = o`DELETE FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t} RETURNING *`, c = r ? n.provideService(s, I, r) : s, l = (await this.runtime.runPromise(c))[0];
587
+ let s = o`DELETE FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t} RETURNING *`, c = r ? n.provideService(s, L, r) : s, l = (await this.runtime.runPromise(c))[0];
529
588
  return l ? (await this.routeEvent({
530
589
  table: e,
531
590
  op: "delete",
@@ -533,9 +592,9 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
533
592
  new: null
534
593
  }, i, r, a), !0) : !1;
535
594
  }
536
- let s = o`SELECT * FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t}`, c = r ? n.provideService(s, I, r) : s, l = (await this.runtime.runPromise(c))[0];
595
+ let s = o`SELECT * FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t}`, c = r ? n.provideService(s, L, r) : s, l = (await this.runtime.runPromise(c))[0];
537
596
  if (!l) return !1;
538
- let u = o`DELETE FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t}`, d = r ? n.provideService(u, I, r) : u;
597
+ let u = o`DELETE FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t}`, d = r ? n.provideService(u, L, r) : u;
539
598
  return await this.runtime.runPromise(d), await this.routeEvent({
540
599
  table: e,
541
600
  op: "delete",
@@ -545,17 +604,17 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
545
604
  }
546
605
  async appendInTxn(e, t, r) {
547
606
  let i = this.sql, a = i`INSERT INTO ${i(this.nsT(e))} ${i.insert(C(t, e))}`;
548
- await this.runtime.runPromise(r ? n.provideService(a, I, r) : a);
607
+ await this.runtime.runPromise(r ? n.provideService(a, L, r) : a);
549
608
  }
550
609
  async maxInTxn(e, t, r, i) {
551
- let a = this.sql, o = Object.entries(r).map(([e, t]) => a`${a(e)} = ${t}`), s = a`SELECT MAX(${a(t)}) AS ${a("m")} FROM ${a(this.nsT(e))} WHERE ${a.and(o)}`, c = (await this.runtime.runPromise(i ? n.provideService(s, I, i) : s))[0]?.m;
610
+ let a = this.sql, o = Object.entries(r).map(([e, t]) => a`${a(e)} = ${t}`), s = a`SELECT MAX(${a(t)}) AS ${a("m")} FROM ${a(this.nsT(e))} WHERE ${a.and(o)}`, c = (await this.runtime.runPromise(i ? n.provideService(s, L, i) : s))[0]?.m;
552
611
  return c == null ? null : Number(c);
553
612
  }
554
613
  async routeEvent(e, t, n = null, r) {
555
614
  if (e = {
556
615
  ...p(r),
557
616
  ...e
558
- }, ie(e.table) && await oe({
617
+ }, M(e.table) && await ae({
559
618
  append: (e, t) => this.appendInTxn(e, t, n),
560
619
  maxOf: (e, t, r) => this.maxInTxn(e, t, r, n)
561
620
  }, {
@@ -567,7 +626,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
567
626
  subjectId: e.subjectId
568
627
  }), this.changeStrategy === "cdc") {
569
628
  let t = (e.op === "delete" ? e.old : e.new)?.id;
570
- t != null && k(m(e.table, e.op, t), {
629
+ t != null && te(m(e.table, e.op, t), {
571
630
  ...e.traceId === void 0 ? {} : { traceId: e.traceId },
572
631
  ...e.subjectId === void 0 ? {} : { subjectId: e.subjectId }
573
632
  });
@@ -605,12 +664,12 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
605
664
  }
606
665
  return await this.executeUpdate(e, o.id, a, r, i) ?? o;
607
666
  }
608
- return N((n) => this.executeInsert(e, t, r, i, n));
667
+ return P((n) => this.executeInsert(e, t, r, i, n));
609
668
  }
610
669
  async executeMariadbUpsert(e, t, r, i, a, o) {
611
670
  let s = this.sql, c = C(t, e), l = Object.keys(c).filter((e) => c[e] !== void 0), u = r.update === void 0 ? l.filter((e) => e !== "id" && !r.conflictColumns.includes(e)) : r.update, d = u.length > 0 ? s.csv(u.map((e) => s`${s(e)} = VALUES(${s(e)})`)) : s`${s(r.conflictColumns[0])} = VALUES(${s(r.conflictColumns[0])})`, f = await this.runPinned(i, (i) => n.tryPromise({
612
671
  try: async () => {
613
- let a = s`INSERT INTO ${s(this.nsT(e))} ${s.insert(c)} ON DUPLICATE KEY UPDATE ${d} RETURNING *`, o = (await this.runtime.runPromise(n.provideService(a, I, i)))[0];
672
+ let a = s`INSERT INTO ${s(this.nsT(e))} ${s.insert(c)} ON DUPLICATE KEY UPDATE ${d} RETURNING *`, o = (await this.runtime.runPromise(n.provideService(a, L, i)))[0];
614
673
  if (!o) throw Error(`MysqlStore.upsert: no row returned for table '${e}'`);
615
674
  let l = t.id;
616
675
  if (l != null && o.id !== l) throw Error(`MysqlStore.upsert: the row written to '${e}' is not the row that was passed in. Upserting id '${String(l)}' matched an existing row with id '${String(o.id)}' on a DIFFERENT unique constraint than the conflictColumns [${r.conflictColumns.join(", ")}] you named, so that row would have been updated and yours never written. Nothing was changed. Name the constraint that actually collides, or resolve the duplicate first.`);
@@ -626,7 +685,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
626
685
  }, a, i, o), f;
627
686
  }
628
687
  async executeInsertIgnore(e, t, r, i, a, o) {
629
- if (t = M(e, t), t.id === void 0) return await this.findByConflict(e, t, r.conflictColumns, i) || N((n) => this.executeInsert(e, t, i, a, n));
688
+ if (t = N(e, t), t.id === void 0) return await this.findByConflict(e, t, r.conflictColumns, i) || P((n) => this.executeInsert(e, t, i, a, n));
630
689
  let s = await this.runPinned(i, (i) => n.tryPromise({
631
690
  try: () => this.decideInsertIgnore(e, t, r, i),
632
691
  catch: (e) => e
@@ -639,7 +698,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
639
698
  }, a, i, o), s.row;
640
699
  }
641
700
  async decideInsertIgnore(e, t, r, i) {
642
- let a = this.sql, o = C(t, e), s = (e) => this.runtime.runPromise(n.provideService(e, I, i));
701
+ let a = this.sql, o = C(t, e), s = (e) => this.runtime.runPromise(n.provideService(e, L, i));
643
702
  if (this.supportsInsertReturning) {
644
703
  let n = (await s(a`INSERT IGNORE INTO ${a(this.nsT(e))} ${a.insert(o)} RETURNING *`))[0];
645
704
  if (n) return {
@@ -690,7 +749,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
690
749
  if (e === null) return [];
691
750
  try {
692
751
  let t = this.sql`SHOW WARNINGS`.unprepared;
693
- return (await this.runtime.runPromise(n.provideService(t, I, e))).map((e) => ({
752
+ return (await this.runtime.runPromise(n.provideService(t, L, e))).map((e) => ({
694
753
  code: Number(e.Code ?? e.code ?? 0),
695
754
  message: String(e.Message ?? e.message ?? "")
696
755
  }));
@@ -700,7 +759,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
700
759
  }
701
760
  async findByConflict(e, t, r, i) {
702
761
  if (r.length === 0) return;
703
- let a = this.sql, o = r.map((e) => a`${a(e)} = ${t[e]}`), s = a`SELECT * FROM ${a(this.nsT(e))} WHERE ${a.and(o)} LIMIT 1`, c = i ? n.provideService(s, I, i) : s;
762
+ let a = this.sql, o = r.map((e) => a`${a(e)} = ${t[e]}`), s = a`SELECT * FROM ${a(this.nsT(e))} WHERE ${a.and(o)} LIMIT 1`, c = i ? n.provideService(s, L, i) : s;
704
763
  return (await this.runtime.runPromise(c))[0];
705
764
  }
706
765
  query(e) {
@@ -708,13 +767,13 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
708
767
  }
709
768
  raw(e, t) {
710
769
  let n = b(e, this.sql);
711
- return O(this.variant, "raw", () => this.runtime.runPromise(n));
770
+ return A(this.variant, "raw", () => this.runtime.runPromise(n));
712
771
  }
713
772
  async runWithEager(e, t) {
714
- if (!ee(e)) return this.executeQuery(e, t);
773
+ if (!D(e)) return this.executeQuery(e, t);
715
774
  let r = this.variant === "mariadb" ? "mariadb" : "mysql", i = this.namespace === null ? v(e, this.sql, r) : null;
716
775
  if (i !== null) try {
717
- let e = t ? n.provideService(i.fragment, I, t) : i.fragment, r = await O(this.variant, "select", () => this.runtime.runPromise(e));
776
+ let e = t ? n.provideService(i.fragment, L, t) : i.fragment, r = await A(this.variant, "select", () => this.runtime.runPromise(e));
718
777
  return i.decode(r);
719
778
  } catch (t) {
720
779
  if (t instanceof u) throw t;
@@ -731,17 +790,17 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
731
790
  reason: "not-compilable"
732
791
  });
733
792
  let a = await this.executeQuery(e, t);
734
- return f(a, e.eager, e.sourceTable ?? A(e.table), (e) => this.executeQuery(e, t));
793
+ return f(a, e.eager, e.sourceTable ?? ne(e.table), (e) => this.executeQuery(e, t));
735
794
  }
736
795
  getInternalRunWithEager() {
737
796
  return this.runWithEager.bind(this);
738
797
  }
739
798
  async localWrite(e, t, n) {
740
- return O(this.variant, e, async () => {
741
- if (this.changeStrategy !== "cdc") return N(n);
799
+ return A(this.variant, e, async () => {
800
+ if (this.changeStrategy !== "cdc") return P(n);
742
801
  h(t);
743
802
  try {
744
- return await N(n);
803
+ return await P(n);
745
804
  } finally {
746
805
  w(t);
747
806
  }
@@ -782,13 +841,13 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
782
841
  let i = this.sql, o = y(r.where, i, this.namespace);
783
842
  this.changeStrategy === "cdc" && h(e), this.inflightTxns++;
784
843
  try {
785
- let r = n.suspend(() => this.sql.withTransaction(n.flatMap(n.serviceOption(I), (r) => {
844
+ let r = n.suspend(() => this.sql.withTransaction(n.flatMap(n.serviceOption(L), (r) => {
786
845
  if (a.isNone(r)) return n.fail(/* @__PURE__ */ Error("MysqlStore.updateMany: TransactionConnection missing."));
787
846
  let s = r.value, c = i`SELECT id FROM ${i(this.nsT(e))} WHERE ${o} FOR UPDATE`, l = i`UPDATE ${i(this.nsT(e))} SET ${i.update(C(t, e))} WHERE ${o}`;
788
- return n.flatMap(n.provideService(c, I, s), (t) => {
847
+ return n.flatMap(n.provideService(c, L, s), (t) => {
789
848
  if (t.length === 0) return n.succeed([]);
790
849
  let r = t.map((e) => e.id), a = i`SELECT * FROM ${i(this.nsT(e))} WHERE ${i("id")} IN ${i.in(r)}`;
791
- return n.flatMap(n.provideService(l, I, s), () => n.provideService(a, I, s));
850
+ return n.flatMap(n.provideService(l, L, s), () => n.provideService(a, L, s));
792
851
  });
793
852
  }))), c = s.exponential("10 millis").pipe(s.compose(s.recurs(3)), s.whileInput(q)), l = r.pipe(n.retry(c), n.withSpan("store.updateMany", { attributes: {
794
853
  "db.system": this.variant,
@@ -821,10 +880,10 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
821
880
  }
822
881
  this.changeStrategy === "cdc" && h(e), this.inflightTxns++;
823
882
  try {
824
- let t = n.suspend(() => this.sql.withTransaction(n.flatMap(n.serviceOption(I), (t) => {
883
+ let t = n.suspend(() => this.sql.withTransaction(n.flatMap(n.serviceOption(L), (t) => {
825
884
  if (a.isNone(t)) return n.fail(/* @__PURE__ */ Error("MysqlStore.deleteMany: TransactionConnection missing."));
826
885
  let o = t.value, s = r`SELECT * FROM ${r(this.nsT(e))} WHERE ${i} FOR UPDATE`, c = r`DELETE FROM ${r(this.nsT(e))} WHERE ${i}`;
827
- return n.flatMap(n.provideService(s, I, o), (e) => e.length === 0 ? n.succeed(e) : n.as(n.provideService(c, I, o), e));
886
+ return n.flatMap(n.provideService(s, L, o), (e) => e.length === 0 ? n.succeed(e) : n.as(n.provideService(c, L, o), e));
828
887
  }))), o = s.exponential("10 millis").pipe(s.compose(s.recurs(3)), s.whileInput(q)), c = t.pipe(n.retry(o), n.withSpan("store.deleteMany", { attributes: {
829
888
  "db.system": this.variant,
830
889
  "db.operation": "delete"
@@ -843,23 +902,27 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
843
902
  }
844
903
  }
845
904
  emitChange(e) {
846
- D(e.table) && (this.changeStrategy === "cdc" && !this.cdcGate.admit(e) || this.emitter.emit("change", e));
905
+ O(e.table) && (this.changeStrategy === "cdc" && !this.cdcGate.admit(e) || this.emitter.emit("change", e));
847
906
  }
848
907
  async startCdcConsumer(e) {
849
908
  if (this.cdcHandle) return;
850
- await this.assertBinlogConfig(), this.cdcReplicaId = e.replicaId;
851
- let t = await this.readCdcOffset(e.replicaId) ?? await this.resolveBinlogEnd(), n = await this.findUndecodableCdcTables(e.includeTables);
852
- for (let e of n) this.log.error(W(e));
853
- this.cdcHandle = await K({
909
+ await this.assertBinlogConfig(), this.cdcReplicaId = e.replicaId, this.cdcAdminRuntime = i.make(z({
910
+ ...e.connection,
911
+ maxConnections: 1
912
+ }));
913
+ let t = await this.readCdcOffset(e.replicaId), n = t !== null && await this.binlogFileExists(t.filename);
914
+ t !== null && !n && this.log.warn(`cdc: the persisted offset points at ${t.filename}, which the server no longer has (the binlog was purged or rotated away while this replica was down). Starting at the current end instead — changes written in the gap are not replayed, and subscriptions self-heal on the next change.`);
915
+ let r = (n ? t : null) ?? await this.resolveBinlogEnd(), a = await this.findUndecodableCdcTables(e.includeTables);
916
+ this.cdcIncludeTables = e.includeTables, this.cdcExcluded = a, this.cdcReporter.announce(a, e.exclusionRefreshFollows === !0), this.cdcHandle = await K({
854
917
  connection: e.connection,
855
918
  serverId: e.serverId,
856
919
  variant: this.variant,
857
920
  ...e.includeTables ? { includeTables: e.includeTables } : {},
858
- ...n.length > 0 ? { undecodableTables: n } : {},
921
+ ...a.length > 0 ? { undecodableTables: a } : {},
859
922
  ...e.watchdogIntervalMs === void 0 ? {} : { watchdogIntervalMs: e.watchdogIntervalMs },
860
923
  ...e.stallThresholdMs === void 0 ? {} : { stallThresholdMs: e.stallThresholdMs },
861
924
  ...e.keepAliveInitialDelayMs === void 0 ? {} : { keepAliveInitialDelayMs: e.keepAliveInitialDelayMs },
862
- startPosition: t,
925
+ startPosition: r,
863
926
  onChange: (e) => this.injectExternalChange(e),
864
927
  onPosition: (e) => {
865
928
  this.cdcPendingPosition = e;
@@ -880,6 +943,16 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
880
943
  if (t.format !== "ROW") throw Error(`${this.variant} binlog CDC requires binlog_format=ROW (got '${t.format}'). Set --binlog-format=ROW, or CDC=0 for inline emit.`);
881
944
  t.rowImage !== "FULL" && this.log.warn(`cdc: binlog_row_image='${t.rowImage}' (expected FULL) — UPDATE/DELETE before-images may be incomplete. Set --binlog-row-image=FULL.`);
882
945
  }
946
+ async binlogFileExists(e) {
947
+ let t = this.cdcAdminRuntime;
948
+ if (t === null) return !0;
949
+ try {
950
+ let r = await t.runPromise(n.flatMap(I, (e) => e`SHOW BINARY LOGS`.unprepared));
951
+ return r.length === 0 || r.some((t) => (t.Log_name ?? t.log_name) === e);
952
+ } catch (e) {
953
+ return this.log.debug(`cdc: could not list binary logs (${e?.message ?? String(e)}) — trusting the persisted offset`), !0;
954
+ }
955
+ }
883
956
  async resolveBinlogEnd() {
884
957
  let e = this.sql, t = this.variant === "mysql" ? e`SHOW BINARY LOG STATUS`.unprepared : e`SHOW MASTER STATUS`.unprepared, n = this.variant === "mysql" ? e`SHOW MASTER STATUS`.unprepared : e`SHOW BINARY LOG STATUS`.unprepared, r = async (e) => {
885
958
  let t = (await this.runtime.runPromise(e))[0];
@@ -887,7 +960,8 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
887
960
  filename: t.File,
888
961
  position: Number(t.Position)
889
962
  } : null;
890
- };
963
+ }, i = await this.resolveBinlogEndOverCdc();
964
+ if (i !== null) return i;
891
965
  try {
892
966
  return await r(t);
893
967
  } catch {
@@ -898,6 +972,19 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
898
972
  }
899
973
  }
900
974
  }
975
+ async resolveBinlogEndOverCdc() {
976
+ let e = this.cdcAdminRuntime;
977
+ if (e === null) return null;
978
+ let t = (e) => n.flatMap(I, (t) => t`${t.unsafe(e)}`.unprepared);
979
+ for (let n of this.variant === "mysql" ? ["SHOW BINARY LOG STATUS", "SHOW MASTER STATUS"] : ["SHOW MASTER STATUS", "SHOW BINARY LOG STATUS"]) try {
980
+ let r = (await e.runPromise(t(n)))[0];
981
+ if (r?.File) return {
982
+ filename: r.File,
983
+ position: Number(r.Position)
984
+ };
985
+ } catch {}
986
+ return null;
987
+ }
901
988
  async readCdcOffset(e) {
902
989
  try {
903
990
  let t = this.sql, n = (await this.runtime.runPromise(t`
@@ -965,13 +1052,35 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
965
1052
  getInternalExecuteInsertIgnore() {
966
1053
  return this.executeInsertIgnore.bind(this);
967
1054
  }
1055
+ async emptyTables(e) {
1056
+ await this.emptyTablesOn(e, null);
1057
+ }
1058
+ async incomingForeignKeys(e) {
1059
+ if (e.length === 0) return [];
1060
+ let t = this.sql;
1061
+ return (await this.runtime.runPromise(t`
1062
+ SELECT TABLE_NAME, COLUMN_NAME, REFERENCED_TABLE_NAME
1063
+ FROM information_schema.KEY_COLUMN_USAGE
1064
+ WHERE TABLE_SCHEMA = DATABASE()
1065
+ AND REFERENCED_TABLE_NAME IN ${t.in(e)}
1066
+ AND TABLE_NAME NOT IN ${t.in(e)}`)).map((e) => ({
1067
+ from: e.TABLE_NAME,
1068
+ column: e.COLUMN_NAME,
1069
+ to: e.REFERENCED_TABLE_NAME
1070
+ }));
1071
+ }
1072
+ async emptyTablesOn(e, t) {
1073
+ if (e.length === 0) return;
1074
+ let r = this.sql, i = (e) => this.nsT(e), a = (e) => r.unsafe(`SET FOREIGN_KEY_CHECKS = ${+!!e}`).unprepared, o = n.acquireUseRelease(a(!1), () => n.forEach(e, (e) => r`DELETE FROM ${r(i(e))}`, { discard: !0 }), () => n.orDie(n.ignore(a(!0)))), s = t === null ? r.withTransaction(o) : n.provideService(o, L, t);
1075
+ await this.runtime.runPromise(s);
1076
+ }
968
1077
  async transactional(e) {
969
1078
  this.inflightTxns++;
970
1079
  try {
971
- return await ae({
1080
+ return await ie({
972
1081
  ...this.txnSpec("MysqlStore.transactional"),
973
1082
  work: e,
974
- makeView: (e, t) => new Te(this, e, t)
1083
+ makeView: (e, t) => new je(this, e, t)
975
1084
  });
976
1085
  } finally {
977
1086
  this.inflightTxns--;
@@ -1002,16 +1111,29 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
1002
1111
  return this.changeStrategy === "cdc" ? "fleet" : "local";
1003
1112
  }
1004
1113
  injectExternalChange(e) {
1005
- if (this.changeStrategy === "cdc" && !this.cdcGate.admit(e) || !D(e.table)) return;
1114
+ if (this.changeStrategy === "cdc" && !this.cdcGate.admit(e) || !O(e.table)) return;
1006
1115
  let t = (e.op === "delete" ? e.old : e.new)?.id;
1007
- j(e.table, e.op, t, (t) => {
1116
+ re(e.table, e.op, t, (t) => {
1008
1117
  this.emitter.emit("change", T(e, t));
1009
1118
  });
1010
1119
  }
1011
1120
  run(e) {
1012
1121
  return this.runtime.runPromise(e);
1013
1122
  }
1123
+ async refreshChangeCaptureExclusions() {
1124
+ if (!this.cdcHandle) return {
1125
+ stillExcluded: [],
1126
+ readmitted: [],
1127
+ newlyExcluded: []
1128
+ };
1129
+ let e = await this.findUndecodableCdcTables(this.cdcIncludeTables), t = be(this.cdcExcluded, e);
1130
+ return this.cdcReporter.settle(t), this.cdcExcluded = e, (t.readmitted.length > 0 || t.newlyExcluded.length > 0) && await this.cdcHandle.setUndecodableTables(e), t;
1131
+ }
1014
1132
  async close(e = 5e3) {
1133
+ if (this.cdcReporter.dispose(), this.cdcAdminRuntime) {
1134
+ let e = this.cdcAdminRuntime;
1135
+ this.cdcAdminRuntime = null, await e.dispose().catch(() => void 0);
1136
+ }
1015
1137
  if (this.cdcCheckpointTimer &&= (clearInterval(this.cdcCheckpointTimer), null), this.cdcHandle) {
1016
1138
  let e = this.cdcHandle.currentPosition();
1017
1139
  e && (this.cdcPendingPosition = e), await this.flushCdcOffset(), await this.cdcHandle.stop(), this.cdcHandle = null;
@@ -1029,7 +1151,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
1029
1151
  async ping() {
1030
1152
  await this.runtime.runPromise(this.sql`SELECT 1`);
1031
1153
  }
1032
- }, Te = class {
1154
+ }, je = class {
1033
1155
  parent;
1034
1156
  txn;
1035
1157
  attr;
@@ -1041,6 +1163,9 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
1041
1163
  query(e) {
1042
1164
  return this.parent.getInternalRunWithEager()(e, this.txn);
1043
1165
  }
1166
+ emptyTables(e) {
1167
+ return this.parent.emptyTablesOn(e, this.txn);
1168
+ }
1044
1169
  insert(e, t) {
1045
1170
  return this.parent.getInternalExecuteInsert()(e, t, this.txn, this.events, this.attr);
1046
1171
  }
@@ -1102,12 +1227,12 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
1102
1227
  this.events.length = 0;
1103
1228
  }
1104
1229
  }
1105
- }, $ = (e) => e.__mysqlReplicationFriend ?? null, Ee = () => ({
1230
+ }, $ = (e) => e.__mysqlReplicationFriend ?? null, Me = () => ({
1106
1231
  async capturePrimaryPosition(e) {
1107
1232
  let t = $(e);
1108
1233
  if (t === null) throw Error("mysqlReplicationAdapter: primary is not a MysqlStore (missing __mysqlReplicationFriend).");
1109
1234
  return t.runEffect(n.gen(function* () {
1110
- let e = yield* F, r = (t.variant === "mariadb" ? yield* e`SELECT @@global.gtid_current_pos AS gtid` : yield* e`SELECT @@global.gtid_executed AS gtid`)[0]?.gtid;
1235
+ let e = yield* I, r = (t.variant === "mariadb" ? yield* e`SELECT @@global.gtid_current_pos AS gtid` : yield* e`SELECT @@global.gtid_executed AS gtid`)[0]?.gtid;
1111
1236
  return typeof r == "string" ? r : yield* n.die("mysql/mariadb did not return a GTID set");
1112
1237
  }));
1113
1238
  },
@@ -1115,14 +1240,14 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
1115
1240
  let t = $(e);
1116
1241
  if (t === null) throw Error("mysqlReplicationAdapter: replica is not a MysqlStore.");
1117
1242
  return t.runEffect(n.gen(function* () {
1118
- let e = yield* F, r = (t.variant === "mariadb" ? yield* e`SELECT @@global.gtid_current_pos AS gtid` : yield* e`SELECT @@global.gtid_executed AS gtid`)[0]?.gtid;
1243
+ let e = yield* I, r = (t.variant === "mariadb" ? yield* e`SELECT @@global.gtid_current_pos AS gtid` : yield* e`SELECT @@global.gtid_executed AS gtid`)[0]?.gtid;
1119
1244
  return typeof r == "string" ? r : yield* n.die("mysql/mariadb replica did not return a GTID set");
1120
1245
  }));
1121
1246
  },
1122
1247
  compare(e, t) {
1123
1248
  return "behind";
1124
1249
  }
1125
- }), De = {
1250
+ }), Ne = {
1126
1251
  id: "mysql",
1127
1252
  makeSqlLayer: (e) => H(e),
1128
1253
  makeStore: (e) => Z({
@@ -1131,7 +1256,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
1131
1256
  }),
1132
1257
  compileContains: (e, t, n) => n`${e} LIKE ${`%${t.replace(/[\\%_]/g, (e) => `\\${e}`)}%`} ESCAPE '\\'`,
1133
1258
  retryFilter: J
1134
- }, Oe = {
1259
+ }, Pe = {
1135
1260
  id: "mariadb",
1136
1261
  makeSqlLayer: (e) => H(e),
1137
1262
  makeStore: (e) => Z({
@@ -1142,4 +1267,4 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
1142
1267
  retryFilter: J
1143
1268
  };
1144
1269
  //#endregion
1145
- export { c as CDC_OFFSETS_TABLE, e as MysqlClient, d as _voltroCdcOffsetsTable, V as connectionFromConfig, z as makeMysqlSqlLayer, H as makeMysqlSqlLayerFromConfig, Z as makeMysqlStore, Oe as mariadbDialect, De as mysqlDialect, Ee as mysqlReplicationAdapter, J as mysqlRetryFilter, K as startBinlogCdc };
1270
+ export { c as CDC_OFFSETS_TABLE, e as MysqlClient, d as _voltroCdcOffsetsTable, V as connectionFromConfig, z as makeMysqlSqlLayer, H as makeMysqlSqlLayerFromConfig, Z as makeMysqlStore, Pe as mariadbDialect, Ne as mysqlDialect, Me as mysqlReplicationAdapter, J as mysqlRetryFilter, K as startBinlogCdc };
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@voltro/sql-mysql",
3
- "version": "0.43.0",
3
+ "version": "0.43.2",
4
4
  "description": "MySQL/MariaDB dialect adapter for Voltro's cross-dialect DataStore (mariadb binlog CDC; mysql inline reactivity).",
5
5
  "keywords": [
6
6
  "voltro",
@@ -35,8 +35,8 @@
35
35
  "dependencies": {
36
36
  "@effect/sql": "^0.52.0",
37
37
  "@effect/sql-mysql2": "^0.53.0",
38
- "@voltro/database": "0.43.0",
39
- "@voltro/logger": "0.43.0"
38
+ "@voltro/database": "0.43.2",
39
+ "@voltro/logger": "0.43.2"
40
40
  },
41
41
  "optionalDependencies": {
42
42
  "@vlasky/zongji": "^0.9.0"