@voltro/sql-mysql 0.43.1 → 0.44.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +65 -0
- package/dist/index.d.ts +126 -1
- package/dist/index.js +302 -168
- package/package.json +3 -3
package/CHANGELOG.md
CHANGED
|
@@ -39,6 +39,71 @@ _Changes staged for the next release accumulate here (rolled up from
|
|
|
39
39
|
|
|
40
40
|
---
|
|
41
41
|
|
|
42
|
+
## [0.44.0] — 2026-08-19
|
|
43
|
+
|
|
44
|
+
### ⚠ BREAKING
|
|
45
|
+
|
|
46
|
+
- **@voltro/data-transfer, @voltro/cli** — `runImport` returns an `ImportOutcome` instead of the bundle's `Manifest`, and `--mode replace` runs as ONE transaction by default.
|
|
47
|
+
|
|
48
|
+
**Why the return type changed.** The summary line counted the rows the BUNDLE carries, not the rows the run wrote. Those differ most exactly where it matters: a bundle directory carries its own resume ledger, so a copied directory imports nothing — correctly, with a warning naming the file to delete — and the run then printed `import complete … rows: 242950` over a target it had not touched. The warning was one line above, which is one line too far for anyone piping the output through `tail -1`. The outcome carries `rowsWritten`, `rowsSkipped` and `fullyResumed`, the CLI reports written-vs-carried, and a fully-skipped run says so on its LAST line.
|
|
49
|
+
|
|
50
|
+
Migration: `runImport(...)` now resolves to `{ manifest, tablesWritten, rowsWritten, tablesSkipped, rowsSkipped, fullyResumed }`. Read `.manifest` where you read the manifest before.
|
|
51
|
+
|
|
52
|
+
**Why replace is atomic now.** The all-or-nothing guarantee was written for the emptying step, and read — reasonably — as covering the run. A replace that died partway through the LOAD left the target emptied of its old rows and holding part of the new ones, measured on a live instance over sixteen minutes. There is no useful state for a replace to stop in, so it is a default rather than a flag you have to know about. It also closes the window that produced that failure: with the tables emptied and the load uncommitted, a concurrent writer in the application waits instead of inserting a row the bundle is about to insert too.
|
|
53
|
+
|
|
54
|
+
`--no-atomic` (CLI) / `atomic: false` (API) opts out. The trade is stated where it bites: every write to those tables waits for the load, and on postgres the bulk `COPY` loader cannot join a transaction it does not own — an atomic run now says that once rather than being quietly slower.
|
|
55
|
+
|
|
56
|
+
Two smaller things from the same report: `--mode replace` against a running instance warns about the empty-target window when it is NOT atomic, and the `--target api` upload reports progress per chunk plus a line explaining that the final request stays open for the whole import — sixteen minutes of silence is indistinguishable from a hang, and one operator killed a run that had finished.
|
|
57
|
+
|
|
58
|
+
**`voltro update` carries you across this** — codemod `0.44.0/01_import-outcome`.
|
|
59
|
+
|
|
60
|
+
### Fixed
|
|
61
|
+
|
|
62
|
+
- **@voltro/sql-mysql, @voltro/sql-postgres, @voltro/sql-sqlite, @voltro/sql-mssql, @voltro/data-transfer** — A write and its write-recorders now succeed or fail together on every SQL dialect, and the data importer's retry is idempotent.
|
|
63
|
+
|
|
64
|
+
A recorder (a versioning trail, an audit log) runs on the caller's connection and is ALLOWED to fail — a recorder that throws must take the write down with it, that is its contract. Outside a caller transaction the two were not one unit: the row's statement committed on its own, and the recorder's INSERT ran afterwards as a second autocommit statement. So a recorder that threw left a COMMITTED row behind a write that reported failure. Measured directly: `insert` throws, the row is in the table, and a second attempt at the same row is `ER_DUP_ENTRY` on PRIMARY.
|
|
65
|
+
|
|
66
|
+
Anything that retries a failed write then meets its own row. The data importer retries by design — it holds a row whose write failed and tries again once the remaining tables have streamed — so a `--mode replace` that had just emptied a table failed on a duplicate key IN that table. From outside, that is the impossible-looking thing: an import that emptied a table and then failed because something was already in it. Reproduced verbatim, including the table name and `[ER_DUP_ENTRY/1062]`, by putting the old retry back.
|
|
67
|
+
|
|
68
|
+
Both ends are closed, and they are independent on purpose:
|
|
69
|
+
|
|
70
|
+
- **The cause.** A table that HAS recorders writes inside a transaction now, so the row and the trail commit together or not at all — on mysql, mariadb, postgres, sqlite and mssql, verified by one suite that asks all five the same question. A table with no recorders — the default — takes the direct path unchanged; `recordsTable` is a Map-size check first, so it costs one comparison. - **The defence.** The importer's retry upserts instead of inserting in `replace` mode. That covers every OTHER way a write can land while reporting failure: a driver timeout on a write the server applied, a connection lost after the commit, a concurrent writer inserting the same key. It is sound precisely because the table was emptied by this same run — there is nothing in it that is not ours.
|
|
71
|
+
|
|
72
|
+
The test that pinned the divergence went red when the fix landed, exactly as its own note said it would, and is inverted with that note kept.
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## [0.43.2] — 2026-08-18
|
|
77
|
+
|
|
78
|
+
### Fixed
|
|
79
|
+
|
|
80
|
+
- **@voltro/sql-mysql, @voltro/database, @voltro/cli** — The mysql-family binlog reader no longer keeps a table excluded after the very migration that fixed it.
|
|
81
|
+
|
|
82
|
+
A UNIQUE on an unbounded text column is a MariaDB hash long-unique, whose hidden `DB_ROW_HASH_n` column the reader can never account for — so the table is held out of binlog capture and the exclusion is reported. `voltro dev` builds its store BEFORE it migrates (kv, cross-replica broadcast and the analytics mirror all need one), so on a boot whose own auto-migration bounds the column, that finding was drawn from a schema that stopped existing about a second later. The exclusion then outlived its cause for the life of the process, and the message — correct when written, and typically the only error line in the boot log — went on describing the pre-migration database.
|
|
83
|
+
|
|
84
|
+
Three changes:
|
|
85
|
+
|
|
86
|
+
- `DataStore.refreshChangeCaptureExclusions()` (optional; implemented by the mysql-family store) re-runs the probe and re-points the LIVE reader, in both directions — schema work that CREATES the condition now excludes the table immediately instead of after three failed writes. `voltro dev` calls it once, after all its schema work. `voltro serve` needs no equivalent: it builds its store after every schema step and never applies DDL itself. - The definitive message now states its own durability — that it is the schema as read at reader attach, and that applying the remedy does not by itself lift the exclusion. - On a boot that will re-check, the finding is reported as a provisional note rather than as a verdict, so a boot that fixes the condition leaves no error line about it. The note escalates to the full verdict on its own if the re-check never runs. - Two reader fixes the re-check depended on: applying a new exclusion set now WAITS for a reconnect already in flight (it is what applies the set, so resolving before it landed meant the caller's next write hit the old filter), and the reconnect loop no longer keeps resuming from an offset it has just jumped away from — that turned one purged offset into a reconnect every watchdog interval, forever, delivering nothing.
|
|
87
|
+
- **@voltro/data-transfer, @voltro/database, @voltro/sql-mysql, @voltro/sql-postgres, @voltro/sql-sqlite, @voltro/sql-mssql, @voltro/cli** — `voltro data import --mode replace` no longer leaves a target in neither state, and now works against schemas that have foreign keys.
|
|
88
|
+
|
|
89
|
+
The delete step ran table by table and stopped at the first refusal, so a run that could not finish left dozens of tables emptied and nothing loaded — and a second attempt destroyed more than the first, because it got further before hitting the same wall. The wall itself was not exotic: MySQL, MariaDB and SQL Server check a foreign key as each ROW is deleted, so a table that references ITSELF cannot be emptied by any ordering of the tables. `createdBy → actors` on the `actors` table is exactly that shape, and it is what an audit mixin on an actor table produces.
|
|
90
|
+
|
|
91
|
+
- `DataStore.emptyTables()` (per dialect) empties the whole set as one unit, in one transaction, with referential integrity suspended for the duration — `FOREIGN_KEY_CHECKS` on the mysql family, a multi-table `TRUNCATE` on postgres, `defer_foreign_keys` on sqlite, per-table `NOCHECK`/`WITH CHECK CHECK` on mssql. All-or-nothing on every engine, including under `--atomic`, where it runs on the transaction the import already holds. - A **pre-flight refusal**: if a table OUTSIDE the bundle holds rows referencing one inside it, the import refuses before deleting anything and names the tables, the columns and the row counts. Those rows cannot be restored from the bundle, so forcing it is not an option. An EMPTY outside table blocks nothing. - Table-level failures carry the driver's own reason and code, the way row-level failures already did. `truncate <table> failed: Failed to execute statement` fits every plausible cause equally; the classification that produced `foreign key <name>: … [ER_NO_REFERENCED_ROW_2/1452]` one level down now applies one level up. The word "truncate" is gone from the message too — the step issues DELETE, and naming a statement it does not run sends whoever reads it to reproduce the wrong thing. - A typed refusal reaching the `--target api` transport keeps its text: the admin import endpoint answers `409` with the reason instead of flattening it to `import failed`, on the one transport where the operator has no other way to see it.
|
|
92
|
+
|
|
93
|
+
The bulk empty emits no change events, where the per-table loop emitted one per row. An import through `--target api` now asks every live subscription to re-read once it lands — the coarse refresh the framework already uses after a broadcast gap — so neither the missing deletes nor a table the bundle carries EMPTY leaves a subscriber holding rows that are gone. Wired where the route is mounted, which is the one place both boot paths share.
|
|
94
|
+
|
|
95
|
+
On postgres the emptying is a `DELETE` per table, not a `TRUNCATE`, and the difference is not performance: postgres refuses `TRUNCATE` on a table with ANY incoming foreign key, rows or not, while the mysql family refuses a DELETE only when rows actually reference the doomed ones. A `TRUNCATE` version made an EMPTY table outside the bundle block a replace on postgres and not on mariadb — one import, refused on one engine and accepted on the other, over a table holding nothing.
|
|
96
|
+
|
|
97
|
+
`--target api` also no longer times out on a full bundle. Both api-target calls went through `fetch`, whose undici default gives up after 300 s — a bound on the caller's database size, on a call whose response arrives only when the import does. They wait as long as the instance needs now, stream the body instead of buffering the whole bundle, and take `--timeout <seconds>` when a deadline is wanted. If one is hit, the message says the instance is probably still importing — and gives different advice for `replace` than for the idempotent modes, since re-running the first while it is mid-flight would empty the target under it.
|
|
98
|
+
|
|
99
|
+
Two more, found by measuring rather than by reading:
|
|
100
|
+
|
|
101
|
+
- **`--atomic` on postgres could not import a bundle that needed the deferred-FK repair at all.** A failed statement aborts the transaction there, and that repair depends on a row whose parent has not loaded yet failing, being held, and being retried — so the first such row poisoned every write after it. Every tolerated write now runs inside a savepoint. Per-row savepoints measured 2.40x the time of none on 5 000 rows, so they are amortised: one savepoint per batch of 200, and a batch that fails rolls back whole and replays row by row. The mysql family and sqlite leave a transaction usable after a failed statement and pay nothing for any of this. - **The replace pre-flight asked the caller's snapshot.** Over `--target api` that is the app's DECLARED schema, which cannot show a table the app stopped declaring but the database still has — and rows in a table nobody declares are exactly the rows nobody is watching. `DataStore.incomingForeignKeys()` reads the live catalog per dialect; the snapshot remains the fallback for stores without one.
|
|
102
|
+
|
|
103
|
+
A bundle bigger than one chunk is now uploaded as a series of short requests, so a proxy body cap or an ingress read timeout has nothing large to choke on, and the switch is automatic — the packer's stream is buffered one chunk ahead, so a small bundle is sent exactly as before and nobody has to know in advance which table is the big one. The import still runs ONCE, at the end, over the whole bundle. Resume is byte-exact (`packBundle` is deterministic over a directory, which this package now asserts), guarded by a bundle key so a different bundle under the same upload id is refused rather than spliced into the partial one, and by a contiguity check so a mis-ordered append cannot produce an archive that only fails later during decode. `--chunk-size <mb>` overrides the 16 MiB default.
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
42
107
|
## [0.43.1] — 2026-08-18
|
|
43
108
|
|
|
44
109
|
### Fixed
|
package/dist/index.d.ts
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import { CDC_OFFSETS_TABLE } from '@voltro/database';
|
|
2
2
|
import { CdcConfig } from '@voltro/database';
|
|
3
|
+
import { ChangeCaptureExclusionRefresh } from '@voltro/database';
|
|
3
4
|
import { ChangeEvent } from '@voltro/database';
|
|
4
5
|
import { ChangeStrategy } from '@voltro/database';
|
|
5
6
|
import { ConfigError } from 'effect';
|
|
@@ -31,6 +32,22 @@ export declare interface BinlogCdcHandle {
|
|
|
31
32
|
readonly stop: () => Promise<void>;
|
|
32
33
|
/** Last position the reader has advanced past (for a final checkpoint). */
|
|
33
34
|
readonly currentPosition: () => BinlogPosition | null;
|
|
35
|
+
/**
|
|
36
|
+
* Replace the undecodable-table set on a LIVE reader.
|
|
37
|
+
*
|
|
38
|
+
* Exists because the set is a property of the SCHEMA, and the schema can move
|
|
39
|
+
* after the reader attached — a boot that migrates in the same process is the
|
|
40
|
+
* ordinary case, not an exotic one. Without this the verdict taken at attach
|
|
41
|
+
* outlives its own cause: a table whose column got bounded seconds later stays
|
|
42
|
+
* excluded for the life of the process.
|
|
43
|
+
*
|
|
44
|
+
* Re-attaches only when the effective set CHANGED, because a re-attach costs a
|
|
45
|
+
* dropped replication connection and a resume — pointless when the answer is
|
|
46
|
+
* the same. Resumes from `currentPosition()`, so no event window is skipped.
|
|
47
|
+
*
|
|
48
|
+
* Returns whether it re-attached.
|
|
49
|
+
*/
|
|
50
|
+
readonly setUndecodableTables: (tables: ReadonlyArray<string>) => Promise<boolean>;
|
|
34
51
|
}
|
|
35
52
|
|
|
36
53
|
export declare interface BinlogCdcOptions {
|
|
@@ -254,7 +271,24 @@ export declare class MysqlStore implements DataStore {
|
|
|
254
271
|
private inflightTxns;
|
|
255
272
|
private readonly log;
|
|
256
273
|
private cdcHandle;
|
|
274
|
+
/** The reader's declared filter + the exclusions currently in force, so a
|
|
275
|
+
* refresh can diff against the decision that is actually running. */
|
|
276
|
+
private cdcIncludeTables;
|
|
277
|
+
private cdcExcluded;
|
|
278
|
+
/** Owns what gets SAID about those exclusions, and when — including the
|
|
279
|
+
* escalation that stands behind `exclusionRefreshFollows`. */
|
|
280
|
+
private readonly cdcReporter;
|
|
257
281
|
private cdcCheckpointTimer;
|
|
282
|
+
/**
|
|
283
|
+
* A one-connection SQL runtime on the CDC credentials.
|
|
284
|
+
*
|
|
285
|
+
* The store's OWN pool runs as the app user, who does not have `REPLICATION
|
|
286
|
+
* CLIENT` — so `SHOW MASTER STATUS` fails there and every position question
|
|
287
|
+
* degraded to "start at the end". The replication connection has those grants
|
|
288
|
+
* by definition (it could not read a binlog otherwise), so asking over IT is
|
|
289
|
+
* the difference between knowing where the primary is and guessing.
|
|
290
|
+
*/
|
|
291
|
+
private cdcAdminRuntime;
|
|
258
292
|
private cdcPendingPosition;
|
|
259
293
|
private cdcReplicaId;
|
|
260
294
|
private cdcStreamName;
|
|
@@ -461,6 +495,30 @@ export declare class MysqlStore implements DataStore {
|
|
|
461
495
|
* The gap being closed is specific: a plain write's statement commits ITSELF,
|
|
462
496
|
* so the transport fires while this is still awaiting the driver.
|
|
463
497
|
*/
|
|
498
|
+
/**
|
|
499
|
+
* A write and its write-recorders, as ONE unit.
|
|
500
|
+
*
|
|
501
|
+
* Without this they are two: the row's statement commits on its own, and the
|
|
502
|
+
* recorder's INSERT is a second autocommit statement afterwards. A recorder
|
|
503
|
+
* that throws — which it is SUPPOSED to be able to do, that is the whole
|
|
504
|
+
* contract — then leaves a committed row behind a write that reported failure.
|
|
505
|
+
*
|
|
506
|
+
* Measured against MariaDB: `insert` throws, the row is in the table, and a
|
|
507
|
+
* second attempt at the same row is `ER_DUP_ENTRY` on PRIMARY. Any caller that
|
|
508
|
+
* retries a failed write — the data importer holds and retries by design —
|
|
509
|
+
* meets its own row and reports a duplicate-key error for a row nobody wrote
|
|
510
|
+
* twice. That reads like the impossible thing it looks like: "we replaced
|
|
511
|
+
* everything and then failed because something was already there."
|
|
512
|
+
*
|
|
513
|
+
* Only tables that HAVE a recorder pay for the transaction. `recordsTable` is
|
|
514
|
+
* a Map-size check first, so the default — no recorders — costs one comparison
|
|
515
|
+
* and takes the direct path exactly as before.
|
|
516
|
+
*
|
|
517
|
+
* Events are buffered and emitted after COMMIT, the same order the
|
|
518
|
+
* transactional view uses: a subscriber must not see a change that then rolls
|
|
519
|
+
* back.
|
|
520
|
+
*/
|
|
521
|
+
private writeWithRecorders;
|
|
464
522
|
private localWrite;
|
|
465
523
|
insert(t: string, r: Row): Promise<Readonly<Record<string, unknown>>>;
|
|
466
524
|
insertMany(t: string, rows: ReadonlyArray<Row>): Promise<readonly Readonly<Record<string, unknown>>[]>;
|
|
@@ -483,7 +541,27 @@ export declare class MysqlStore implements DataStore {
|
|
|
483
541
|
/* Excluded from this release type: emitChange */
|
|
484
542
|
/* Excluded from this release type: startCdcConsumer */
|
|
485
543
|
/* Excluded from this release type: assertBinlogConfig */
|
|
486
|
-
|
|
544
|
+
/**
|
|
545
|
+
* Resolve the current binlog (file, position). The statement DIVERGES
|
|
546
|
+
* by engine: MySQL 8.4 REMOVED `SHOW MASTER STATUS` in favour of
|
|
547
|
+
* `SHOW BINARY LOG STATUS`; MariaDB (and mysql < 8.4) only know
|
|
548
|
+
* `SHOW MASTER STATUS`. Pick by variant, then fall back to the other
|
|
549
|
+
* spelling so a mysql < 8.4 or a future rename still resolves. Both
|
|
550
|
+
* return the same `File`/`Position` columns. @internal
|
|
551
|
+
*/
|
|
552
|
+
/**
|
|
553
|
+
* Does the server still have this binlog file?
|
|
554
|
+
*
|
|
555
|
+
* `SHOW BINARY LOGS` needs `REPLICATION CLIENT` (or `SUPER`), which is why it
|
|
556
|
+
* runs on the CDC credentials rather than the store's. A probe that cannot run
|
|
557
|
+
* answers `true`: refusing to resume on a question we could not ask would
|
|
558
|
+
* throw away a perfectly good offset, and the reader's existing purge recovery
|
|
559
|
+
* still covers the case we then get wrong.
|
|
560
|
+
*/
|
|
561
|
+
private binlogFileExists;
|
|
562
|
+
private resolveBinlogEnd;
|
|
563
|
+
/** The primary's current binlog end, asked over the replication credentials. */
|
|
564
|
+
private resolveBinlogEndOverCdc;
|
|
487
565
|
/* Excluded from this release type: readCdcOffset */
|
|
488
566
|
/* Excluded from this release type: flushCdcOffset */
|
|
489
567
|
/* Excluded from this release type: getInternalExecuteQuery */
|
|
@@ -494,6 +572,41 @@ export declare class MysqlStore implements DataStore {
|
|
|
494
572
|
/* Excluded from this release type: getInternalExecuteDelete */
|
|
495
573
|
/* Excluded from this release type: getInternalExecuteUpsert */
|
|
496
574
|
/* Excluded from this release type: getInternalExecuteInsertIgnore */
|
|
575
|
+
/**
|
|
576
|
+
* Empty `tables` as one unit — see `DataStore.emptyTables`.
|
|
577
|
+
*
|
|
578
|
+
* The engine-specific half is why this cannot live in the importer: MySQL and
|
|
579
|
+
* MariaDB check a foreign key as each ROW is deleted, so no ordering of the
|
|
580
|
+
* TABLES can empty a table that references itself. `actors.createdBy →
|
|
581
|
+
* actors` is exactly that shape, it is the framework's own actor pattern, and
|
|
582
|
+
* it made `--mode replace` fail on the first parent table of any real schema.
|
|
583
|
+
* `FOREIGN_KEY_CHECKS` is the session switch that lets the whole set go at
|
|
584
|
+
* once; the final state satisfies every constraint because the caller reloads
|
|
585
|
+
* all of these tables next, and refuses beforehand if anything OUTSIDE the set
|
|
586
|
+
* points into it.
|
|
587
|
+
*
|
|
588
|
+
* Two details are load-bearing:
|
|
589
|
+
*
|
|
590
|
+
* - The switch is SESSION-scoped, so it must be set on the connection the
|
|
591
|
+
* deletes actually run on — hence one transaction, and hence `sql`
|
|
592
|
+
* directly rather than the transactional VIEW (whose `deleteMany` walks
|
|
593
|
+
* row by row, which is both the wrong shape here and unaffordable at
|
|
594
|
+
* bundle scale).
|
|
595
|
+
* - Restoring it is not optional and not conditional. A connection handed
|
|
596
|
+
* back to the pool with checks off would silently accept dangling
|
|
597
|
+
* references for every later caller, so the restore is a release step
|
|
598
|
+
* that runs on success, on failure and on interrupt alike.
|
|
599
|
+
*/
|
|
600
|
+
emptyTables(tables: ReadonlyArray<string>): Promise<void>;
|
|
601
|
+
/** See `DataStore.incomingForeignKeys`. `information_schema` is the catalog on
|
|
602
|
+
* both engines of this family, and `REFERENCED_TABLE_NAME` is only set on the
|
|
603
|
+
* rows that ARE foreign keys — so no join is needed to exclude the rest. */
|
|
604
|
+
incomingForeignKeys(tables: ReadonlyArray<string>): Promise<ReadonlyArray<{
|
|
605
|
+
from: string;
|
|
606
|
+
column: string;
|
|
607
|
+
to: string;
|
|
608
|
+
}>>;
|
|
609
|
+
/* Excluded from this release type: emptyTablesOn */
|
|
497
610
|
transactional<T>(work: (tx: DataStore) => Promise<T>): Promise<T>;
|
|
498
611
|
/**
|
|
499
612
|
* The DIALECT half of the shared transaction bracket
|
|
@@ -517,6 +630,18 @@ export declare class MysqlStore implements DataStore {
|
|
|
517
630
|
get changeScope(): 'local' | 'fleet';
|
|
518
631
|
injectExternalChange(event: ChangeEvent): void;
|
|
519
632
|
run<A, E>(effect: Effect.Effect<A, E, SqlClient.SqlClient>): Promise<A>;
|
|
633
|
+
/**
|
|
634
|
+
* Re-run the undecodable probe against the CURRENT schema and hand the answer
|
|
635
|
+
* to the live reader. See `DataStore.refreshChangeCaptureExclusions`.
|
|
636
|
+
*
|
|
637
|
+
* Diffs in both directions on purpose. Re-admission is the direction a boot
|
|
638
|
+
* that migrates in-process needs; the opposite — schema work that CREATES an
|
|
639
|
+
* unbounded unique on a captured table — is the same staleness with the signs
|
|
640
|
+
* flipped, and it is the one that loses events quietly, since the reader keeps
|
|
641
|
+
* tailing a table it can no longer decode until the reconnect loop diagnoses
|
|
642
|
+
* it three failures later.
|
|
643
|
+
*/
|
|
644
|
+
refreshChangeCaptureExclusions(): Promise<ChangeCaptureExclusionRefresh>;
|
|
520
645
|
close(gracePeriodMs?: number): Promise<void>;
|
|
521
646
|
/** Liveness probe — `SELECT 1`. Rejects if the pool can't answer. */
|
|
522
647
|
ping(): Promise<void>;
|
package/dist/index.js
CHANGED
|
@@ -1,18 +1,18 @@
|
|
|
1
1
|
import { MysqlClient as e } from "@effect/sql-mysql2";
|
|
2
2
|
import { Config as t, Effect as n, Layer as r, ManagedRuntime as i, Option as a, Redacted as o, Schedule as s } from "effect";
|
|
3
|
-
import { CDC_OFFSETS_TABLE as c, DEFAULT_ACQUIRE_TIMEOUT_MS as l, EagerCardinalityError as u, _voltroCdcOffsetsTable as d, attachEagerLoads as f, attributionFields as p, attributionKey as m, beginLocalWrite as h, bulkInsertLimitsFor as g, chunkRowsForInsert as _, compileEagerJson as v, compilePredicate as y, compileRawFragment as b, compileSelect as x, decodeRowsFromSchema as S, encodeRowForSchema as C, endLocalWrite as w, externalChangeEvent as T, getTable as E, hasEagerLoads as
|
|
4
|
-
import { EventEmitter as
|
|
5
|
-
import { createLogger as
|
|
6
|
-
import { SqlClient as
|
|
3
|
+
import { CDC_OFFSETS_TABLE as c, DEFAULT_ACQUIRE_TIMEOUT_MS as l, EagerCardinalityError as u, _voltroCdcOffsetsTable as d, attachEagerLoads as f, attributionFields as p, attributionKey as m, beginLocalWrite as h, bulkInsertLimitsFor as g, chunkRowsForInsert as _, compileEagerJson as v, compilePredicate as y, compileRawFragment as b, compileSelect as x, decodeRowsFromSchema as S, encodeRowForSchema as C, endLocalWrite as w, externalChangeEvent as T, getTable as E, hasEagerLoads as D, isTableReactive as O, makeEagerFallbackReporter as k, observeDbOp as A, qualifyTable as j, raiseChangeListenerCeiling as M, recordsTable as N, registerPendingAttribution as ee, requireTable as te, resolveEchoAttribution as ne, runStoreTransaction as re, runWriteRecorders as ie, stampGeneratedId as P, stampGeneratedIds as ae, withCapturedAttribution as F } from "@voltro/database";
|
|
4
|
+
import { EventEmitter as oe } from "node:events";
|
|
5
|
+
import { createLogger as I } from "@voltro/logger";
|
|
6
|
+
import { SqlClient as L, TransactionConnection as R } from "@effect/sql/SqlClient";
|
|
7
7
|
//#region src/sqlLayer.ts
|
|
8
|
-
var
|
|
8
|
+
var se = (e) => e ? { rejectUnauthorized: !1 } : void 0, ce = (e) => {
|
|
9
9
|
let t = e.acquireTimeoutMs ?? l;
|
|
10
10
|
return t > 0 ? t : void 0;
|
|
11
|
-
},
|
|
11
|
+
}, le = (e) => {
|
|
12
12
|
let t = e.acquireQueueLimit;
|
|
13
13
|
return t !== void 0 && t > 0 ? t : void 0;
|
|
14
|
-
},
|
|
15
|
-
let t = e.ssl === void 0 ? void 0 :
|
|
14
|
+
}, ue = (e) => {
|
|
15
|
+
let t = e.ssl === void 0 ? void 0 : se(e.ssl), n = ce(e), r = le(e);
|
|
16
16
|
return {
|
|
17
17
|
...t === void 0 ? {} : { ssl: t },
|
|
18
18
|
...n === void 0 ? {} : { connectTimeout: n },
|
|
@@ -24,7 +24,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
24
24
|
username: t.succeed(n.username),
|
|
25
25
|
password: t.succeed(o.make(n.password)),
|
|
26
26
|
database: t.succeed(n.database),
|
|
27
|
-
poolConfig: t.succeed(
|
|
27
|
+
poolConfig: t.succeed(ue(n)),
|
|
28
28
|
...n.maxConnections === void 0 ? {} : { maxConnections: t.succeed(n.maxConnections) }
|
|
29
29
|
}), B = (e) => {
|
|
30
30
|
let t = e.get("sslmode");
|
|
@@ -70,69 +70,103 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
70
70
|
}, H = (e) => z(V(e)), U = (e, t) => e === null ? !1 : t === null ? !0 : e.filename === t.filename ? e.position > t.position : e.filename > t.filename, de = (e) => e.msSinceProgress < e.stallThresholdMs ? "healthy" : e.primary !== null && !U(e.primary, e.reader) ? "idle-caught-up" : "reconnect", fe = (e) => (e instanceof Error ? e.message : String(e ?? "")).includes("schema changed between binlog event and metadata fetch"), pe = (e) => {
|
|
71
71
|
let t = e instanceof Error ? e.message : String(e ?? "");
|
|
72
72
|
return /Table\s+[^\s.]+\.(\S+)\s+schema changed between binlog event and metadata fetch/.exec(t)?.[1] ?? null;
|
|
73
|
-
}, me = "\n SELECT DISTINCT s.TABLE_NAME AS tableName\n FROM information_schema.STATISTICS s\n JOIN information_schema.COLUMNS c\n ON c.TABLE_SCHEMA = s.TABLE_SCHEMA\n AND c.TABLE_NAME = s.TABLE_NAME\n AND c.COLUMN_NAME = s.COLUMN_NAME\n WHERE s.TABLE_SCHEMA = DATABASE()\n AND s.NON_UNIQUE = 0\n AND s.SUB_PART IS NULL\n AND c.DATA_TYPE IN ('text','tinytext','mediumtext','longtext','blob','tinyblob','mediumblob','longblob')\n", W = (e) => `cdc: table '${e}' is EXCLUDED from binlog capture — its row image carries a hidden column the reader cannot account for. Cause: a UNIQUE constraint on an UNBOUNDED text column, which MariaDB backs with a HASH long-unique index; that index adds a hidden DB_ROW_HASH_n column to the row, present in the binlog and absent from information_schema.COLUMNS. Remedy: bound the column — text().maxLength(n) — so the constraint becomes an ordinary B-tree index with no hidden column. ALTER TABLE FORCE does NOT help: the rebuild recreates the index and the hidden column. Until then, cross-instance change events for this table are lost; own-node reactivity is unaffected (writes still emit inline)
|
|
74
|
-
let n =
|
|
73
|
+
}, me = "\n SELECT DISTINCT s.TABLE_NAME AS tableName\n FROM information_schema.STATISTICS s\n JOIN information_schema.COLUMNS c\n ON c.TABLE_SCHEMA = s.TABLE_SCHEMA\n AND c.TABLE_NAME = s.TABLE_NAME\n AND c.COLUMN_NAME = s.COLUMN_NAME\n WHERE s.TABLE_SCHEMA = DATABASE()\n AND s.NON_UNIQUE = 0\n AND s.SUB_PART IS NULL\n AND c.DATA_TYPE IN ('text','tinytext','mediumtext','longtext','blob','tinyblob','mediumblob','longblob')\n", he = "This verdict is the schema as read when the reader attached; applying the remedy does not lift it by itself. A boot that migrates in-process re-checks it once its schema work is done and re-admits the table in the same run; otherwise it holds until the process restarts.", W = (e) => `cdc: table '${e}' is EXCLUDED from binlog capture — its row image carries a hidden column the reader cannot account for. Cause: a UNIQUE constraint on an UNBOUNDED text column, which MariaDB backs with a HASH long-unique index; that index adds a hidden DB_ROW_HASH_n column to the row, present in the binlog and absent from information_schema.COLUMNS. Remedy: bound the column — text().maxLength(n) — so the constraint becomes an ordinary B-tree index with no hidden column. ALTER TABLE FORCE does NOT help: the rebuild recreates the index and the hidden column. Until then, cross-instance change events for this table are lost; own-node reactivity is unaffected (writes still emit inline). ${he}`, ge = (e) => `cdc: table '${e}' looks undecodable (a UNIQUE on an UNBOUNDED text column → MariaDB hash long-unique → a hidden DB_ROW_HASH_n column in the row image). Held OUT of binlog capture for now and re-checked after this boot's schema work — a migration that bounds the column re-admits it in the same run. Not a verdict yet; the definitive line follows.`, _e = (e) => `cdc: table '${e}' re-admitted to binlog capture — the hidden-column condition is gone (the schema moved since the reader attached). Cross-instance change events for it flow again.`, ve = 12e4, ye = (e, t = ve) => {
|
|
74
|
+
let n = null, r = [], i = () => {
|
|
75
|
+
n &&= (clearTimeout(n), null), r = [];
|
|
76
|
+
};
|
|
75
77
|
return {
|
|
76
|
-
|
|
78
|
+
announce: (a, o) => {
|
|
79
|
+
if (i(), a.length !== 0) {
|
|
80
|
+
if (!o) {
|
|
81
|
+
for (let t of a) e("error", W(t));
|
|
82
|
+
return;
|
|
83
|
+
}
|
|
84
|
+
for (let t of a) e("warn", ge(t));
|
|
85
|
+
r = a, n = setTimeout(() => {
|
|
86
|
+
let i = r;
|
|
87
|
+
n = null, r = [];
|
|
88
|
+
for (let n of i) e("error", `${W(n)} (Held provisionally for ${Math.round(t / 1e3)}s awaiting a post-migration re-check that never ran; standing by the reading taken at attach.)`);
|
|
89
|
+
}, t), n.unref && n.unref();
|
|
90
|
+
}
|
|
91
|
+
},
|
|
92
|
+
settle: (t) => {
|
|
93
|
+
i();
|
|
94
|
+
for (let n of t.stillExcluded) e("error", W(n));
|
|
95
|
+
for (let n of t.newlyExcluded) e("error", W(n));
|
|
96
|
+
for (let n of t.readmitted) e("info", _e(n));
|
|
97
|
+
},
|
|
98
|
+
dispose: i
|
|
99
|
+
};
|
|
100
|
+
}, be = (e, t) => {
|
|
101
|
+
let n = new Set(e), r = new Set(t);
|
|
102
|
+
return {
|
|
103
|
+
stillExcluded: t.filter((e) => n.has(e)),
|
|
104
|
+
readmitted: e.filter((e) => !r.has(e)),
|
|
105
|
+
newlyExcluded: t.filter((e) => !n.has(e))
|
|
106
|
+
};
|
|
107
|
+
}, xe = 3e5, Se = 3, Ce = (e, t) => {
|
|
108
|
+
let n = [...e.filter((e) => t - e < xe), t];
|
|
109
|
+
return {
|
|
110
|
+
verdict: n.length >= Se ? "persistent" : "backlog",
|
|
77
111
|
hits: n
|
|
78
112
|
};
|
|
79
|
-
},
|
|
113
|
+
}, we = /* @__PURE__ */ new Set([
|
|
80
114
|
"writerows",
|
|
81
115
|
"updaterows",
|
|
82
116
|
"deleterows"
|
|
83
|
-
]),
|
|
84
|
-
let t =
|
|
117
|
+
]), Te = /\b(alter|rename|drop|create)\s+(table|column)?/i, G = (e) => new Promise((t) => setTimeout(t, e)), K = async (e) => {
|
|
118
|
+
let t = I({ scope: `voltro:${e.variant}:cdc` }), n;
|
|
85
119
|
try {
|
|
86
120
|
n = (await import("@vlasky/zongji")).default;
|
|
87
121
|
} catch (t) {
|
|
88
122
|
throw Error(`changeStrategy='cdc' on ${e.variant} requires the '@vlasky/zongji' optional dependency. Install it (it ships as an optionalDependency of @voltro/sql-mysql) to enable binlog CDC.`, { cause: t });
|
|
89
123
|
}
|
|
90
|
-
let r = new Set(e.undecodableTables ?? []), i = e.includeTables ? new Set(e.includeTables
|
|
124
|
+
let r = new Set(e.undecodableTables ?? []), i = e.includeTables ? new Set(e.includeTables) : null, a = e.connection.database, o = e.startPosition?.filename ?? null, s = e.startPosition ?? null, c = (t, n) => S([n], t, e.variant)[0], l = null, u = !1, d = 0, f = Date.now(), p = !1, m = null, h = /* @__PURE__ */ new Map(), g = /* @__PURE__ */ new Set(), _ = null, v = e.stallThresholdMs ?? 25e3, y = e.watchdogIntervalMs ?? 15e3, b = () => {
|
|
91
125
|
f = Date.now();
|
|
92
|
-
},
|
|
93
|
-
|
|
94
|
-
let
|
|
95
|
-
if (
|
|
126
|
+
}, x = (n) => {
|
|
127
|
+
b();
|
|
128
|
+
let u = n.getEventName();
|
|
129
|
+
if (u === "rotate" && n.binlogName) {
|
|
96
130
|
o = n.binlogName;
|
|
97
131
|
return;
|
|
98
132
|
}
|
|
99
|
-
if (
|
|
133
|
+
if (u === "query" && n.query && Te.test(n.query)) {
|
|
100
134
|
l && (l.tableMap = {});
|
|
101
135
|
return;
|
|
102
136
|
}
|
|
103
137
|
if (n.nextPosition && o && (s = {
|
|
104
138
|
filename: o,
|
|
105
139
|
position: n.nextPosition
|
|
106
|
-
}, e.onPosition?.(s)), !
|
|
107
|
-
let
|
|
108
|
-
if (!
|
|
109
|
-
let
|
|
110
|
-
if (!
|
|
111
|
-
if (
|
|
112
|
-
table:
|
|
140
|
+
}, e.onPosition?.(s)), !we.has(u)) return;
|
|
141
|
+
let d = n.tableMap[n.tableId];
|
|
142
|
+
if (!d || d.parentSchema !== a) return;
|
|
143
|
+
let f = d.tableName;
|
|
144
|
+
if (!f.startsWith("_voltro_") && !(i && !i.has(f)) && !r.has(f)) try {
|
|
145
|
+
if (u === "writerows") for (let t of n.rows) e.onChange({
|
|
146
|
+
table: f,
|
|
113
147
|
op: "insert",
|
|
114
148
|
old: null,
|
|
115
|
-
new: c(
|
|
149
|
+
new: c(f, t)
|
|
116
150
|
});
|
|
117
|
-
else if (
|
|
118
|
-
table:
|
|
151
|
+
else if (u === "deleterows") for (let t of n.rows) e.onChange({
|
|
152
|
+
table: f,
|
|
119
153
|
op: "delete",
|
|
120
|
-
old: c(
|
|
154
|
+
old: c(f, t),
|
|
121
155
|
new: null
|
|
122
156
|
});
|
|
123
157
|
else for (let t of n.rows) e.onChange({
|
|
124
|
-
table:
|
|
158
|
+
table: f,
|
|
125
159
|
op: "update",
|
|
126
|
-
old: c(
|
|
127
|
-
new: c(
|
|
160
|
+
old: c(f, t.before),
|
|
161
|
+
new: c(f, t.after)
|
|
128
162
|
});
|
|
129
163
|
} catch (n) {
|
|
130
164
|
t.warn("cdc: failed to map a row event", {
|
|
131
|
-
table:
|
|
132
|
-
name:
|
|
165
|
+
table: f,
|
|
166
|
+
name: u
|
|
133
167
|
}, n), e.onError?.(n);
|
|
134
168
|
}
|
|
135
|
-
},
|
|
169
|
+
}, C = (t) => {
|
|
136
170
|
let n = {
|
|
137
171
|
serverId: e.serverId,
|
|
138
172
|
includeEvents: [
|
|
@@ -146,10 +180,10 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
146
180
|
]
|
|
147
181
|
};
|
|
148
182
|
return e.includeTables && e.includeTables.length > 0 && (n.includeSchema = { [a]: e.includeTables.filter((e) => !r.has(e)) }), t ? (n.filename = t.filename, n.position = t.position) : n.startAtEnd = !0, n;
|
|
149
|
-
},
|
|
183
|
+
}, w = (e) => {
|
|
150
184
|
let t = e;
|
|
151
185
|
return t?.errno === 1236 || String(t?.code ?? "").includes("ER_MASTER_FATAL_ERROR_READING_BINLOG");
|
|
152
|
-
},
|
|
186
|
+
}, T = (r) => new Promise((i, a) => {
|
|
153
187
|
let o = new n({
|
|
154
188
|
host: e.connection.host,
|
|
155
189
|
port: e.connection.port,
|
|
@@ -160,8 +194,8 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
160
194
|
});
|
|
161
195
|
l = o;
|
|
162
196
|
let s = !1;
|
|
163
|
-
o.on("binlog",
|
|
164
|
-
|
|
197
|
+
o.on("binlog", x), o.on("ready", () => {
|
|
198
|
+
b(), !s && (s = !0, d = 0, t.info("cdc: binlog reader attached", {
|
|
165
199
|
serverId: e.serverId,
|
|
166
200
|
from: r ?? "current-end"
|
|
167
201
|
}), i());
|
|
@@ -170,82 +204,101 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
170
204
|
s = !0, a(e instanceof Error ? e : Error(String(e)));
|
|
171
205
|
return;
|
|
172
206
|
}
|
|
173
|
-
|
|
174
|
-
}), o.start(
|
|
175
|
-
}),
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
n = e;
|
|
196
|
-
}
|
|
207
|
+
D("binlog reader error", e);
|
|
208
|
+
}), o.start(C(r));
|
|
209
|
+
}), E = async (n, i) => {
|
|
210
|
+
p = !0, t.warn(`cdc: reconnecting — ${n}`, {}, i instanceof Error ? i : void 0), e.onError?.(i);
|
|
211
|
+
try {
|
|
212
|
+
let n = i;
|
|
213
|
+
for (; !u;) {
|
|
214
|
+
try {
|
|
215
|
+
l?.stop();
|
|
216
|
+
} catch {}
|
|
217
|
+
if (d++, await G(Math.min(3e4, 500 * 2 ** Math.min(d, 6))), u) return;
|
|
218
|
+
let i = s, a = w(n), c = !a && fe(n), f = !1;
|
|
219
|
+
if (c) {
|
|
220
|
+
let e = pe(n), i = e ?? "<unknown>", { verdict: a, hits: o } = Ce(h.get(i) ?? [], Date.now());
|
|
221
|
+
h.set(i, o), f = a === "persistent", f && !g.has(i) && (g.add(i), e !== null && r.add(e), t.error(W(i)));
|
|
222
|
+
}
|
|
223
|
+
(a || c) && (f || t.warn(c ? "cdc: un-replayable backlog event (schema moved past it) — jumping to current end + self-heal" : "cdc: binlog gap (purged/failover) — jumping to current end + self-heal"), i = e.resolveStartPosition ? await e.resolveStartPosition().catch(() => null) : null, o = i?.filename ?? null, s = i, f || e.onResync?.());
|
|
224
|
+
try {
|
|
225
|
+
await T(i), d = 0, b();
|
|
226
|
+
return;
|
|
227
|
+
} catch (e) {
|
|
228
|
+
n = e;
|
|
197
229
|
}
|
|
198
|
-
} finally {
|
|
199
|
-
p = !1;
|
|
200
230
|
}
|
|
231
|
+
} finally {
|
|
232
|
+
p = !1;
|
|
201
233
|
}
|
|
202
|
-
}
|
|
234
|
+
}, D = (e, t) => u ? Promise.resolve() : p ? m ?? Promise.resolve() : (m = E(e, t).finally(() => {
|
|
235
|
+
m = null;
|
|
236
|
+
}), m);
|
|
203
237
|
try {
|
|
204
|
-
await
|
|
238
|
+
await T(e.startPosition ?? null);
|
|
205
239
|
} catch (n) {
|
|
206
|
-
if (
|
|
240
|
+
if (w(n) && e.startPosition) {
|
|
207
241
|
t.warn("cdc: persisted offset purged — starting at current end + self-heal");
|
|
208
242
|
try {
|
|
209
243
|
l?.stop();
|
|
210
244
|
} catch {}
|
|
211
245
|
let n = e.resolveStartPosition ? await e.resolveStartPosition().catch(() => null) : null;
|
|
212
|
-
o = n?.filename ?? null, e.onResync?.(), await G(500), await
|
|
246
|
+
o = n?.filename ?? null, s = n, e.onResync?.(), await G(500), await T(n);
|
|
213
247
|
} else throw n;
|
|
214
248
|
}
|
|
215
|
-
let
|
|
216
|
-
if (u || p || Date.now() - f <
|
|
249
|
+
let O = async () => {
|
|
250
|
+
if (u || p || Date.now() - f < v) return;
|
|
217
251
|
let n = null;
|
|
218
252
|
if (e.resolveStartPosition && (n = await e.resolveStartPosition().catch(() => null)), u || p) return;
|
|
219
253
|
let r = de({
|
|
220
254
|
msSinceProgress: Date.now() - f,
|
|
221
|
-
stallThresholdMs:
|
|
255
|
+
stallThresholdMs: v,
|
|
222
256
|
primary: n,
|
|
223
257
|
reader: s
|
|
224
258
|
});
|
|
225
259
|
if (r !== "reconnect") {
|
|
226
|
-
r === "idle-caught-up" &&
|
|
260
|
+
r === "idle-caught-up" && b();
|
|
227
261
|
return;
|
|
228
262
|
}
|
|
229
263
|
t.warn("cdc: watchdog detected a stalled reader — forcing reconnect", {
|
|
230
264
|
reader: s,
|
|
231
265
|
primary: n,
|
|
232
266
|
stalledForMs: Date.now() - f
|
|
233
|
-
}),
|
|
267
|
+
}), D("watchdog: stream stalled (no events while primary advanced)", /* @__PURE__ */ Error("cdc watchdog stall"));
|
|
234
268
|
};
|
|
235
|
-
return
|
|
236
|
-
|
|
237
|
-
},
|
|
269
|
+
return _ = setInterval(() => {
|
|
270
|
+
O();
|
|
271
|
+
}, y), _.unref && _.unref(), {
|
|
238
272
|
stop: async () => {
|
|
239
|
-
u = !0,
|
|
273
|
+
u = !0, _ &&= (clearInterval(_), null);
|
|
240
274
|
try {
|
|
241
275
|
l?.stop();
|
|
242
276
|
} catch (e) {
|
|
243
277
|
t.warn("cdc: error stopping binlog reader", {}, e);
|
|
244
278
|
}
|
|
245
279
|
},
|
|
246
|
-
currentPosition: () => s
|
|
280
|
+
currentPosition: () => s,
|
|
281
|
+
setUndecodableTables: async (t) => {
|
|
282
|
+
let n = new Set(t);
|
|
283
|
+
if (n.size === r.size && [...n].every((e) => r.has(e))) return !1;
|
|
284
|
+
r.clear();
|
|
285
|
+
for (let e of n) r.add(e);
|
|
286
|
+
if (u) return !1;
|
|
287
|
+
if (p) return await (m ?? Promise.resolve()), !0;
|
|
288
|
+
p = !0;
|
|
289
|
+
try {
|
|
290
|
+
try {
|
|
291
|
+
l?.stop();
|
|
292
|
+
} catch {}
|
|
293
|
+
return await G(500), !u && (s === null && e.onResync?.(), await T(s), b(), !0);
|
|
294
|
+
} catch (e) {
|
|
295
|
+
return p = !1, await D("re-attach after an exclusion refresh failed", e), !1;
|
|
296
|
+
} finally {
|
|
297
|
+
p = !1;
|
|
298
|
+
}
|
|
299
|
+
}
|
|
247
300
|
};
|
|
248
|
-
},
|
|
301
|
+
}, Ee = /* @__PURE__ */ new Set(["1213", "1205"]), De = (e) => {
|
|
249
302
|
let t = e;
|
|
250
303
|
for (let e = 0; e < 5 && typeof t == "object" && t; e++) {
|
|
251
304
|
let e = t.errno;
|
|
@@ -255,8 +308,8 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
255
308
|
t = t.cause;
|
|
256
309
|
}
|
|
257
310
|
}, q = (e) => {
|
|
258
|
-
let t =
|
|
259
|
-
return t !== void 0 &&
|
|
311
|
+
let t = De(e);
|
|
312
|
+
return t !== void 0 && Ee.has(t);
|
|
260
313
|
}, J = (e) => q(e) ? "retry" : "noRetry", Y = (e) => {
|
|
261
314
|
if (e == null) return "null";
|
|
262
315
|
let t = typeof e;
|
|
@@ -266,19 +319,19 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
266
319
|
if (Array.isArray(e)) return `[${e.map(Y).join(",")}]`;
|
|
267
320
|
let n = e;
|
|
268
321
|
return `{${Object.keys(n).sort().map((e) => `${JSON.stringify(e)}:${Y(n[e])}`).join(",")}}`;
|
|
269
|
-
},
|
|
322
|
+
}, Oe = (e) => {
|
|
270
323
|
let t = Y(e), n = 2166136261;
|
|
271
324
|
for (let e = 0; e < t.length; e++) n ^= t.charCodeAt(e), n = Math.imul(n, 16777619);
|
|
272
325
|
return (n >>> 0).toString(36);
|
|
273
|
-
},
|
|
326
|
+
}, ke = (e, t) => {
|
|
274
327
|
let n = setTimeout(e, t);
|
|
275
328
|
typeof n.unref == "function" && n.unref();
|
|
276
|
-
},
|
|
329
|
+
}, Ae = class {
|
|
277
330
|
variant;
|
|
278
331
|
ttlMs;
|
|
279
332
|
schedule;
|
|
280
333
|
seen = /* @__PURE__ */ new Map();
|
|
281
|
-
constructor(e, t = 6e4, n =
|
|
334
|
+
constructor(e, t = 6e4, n = ke) {
|
|
282
335
|
this.variant = e, this.ttlMs = t, this.schedule = n;
|
|
283
336
|
}
|
|
284
337
|
key(e) {
|
|
@@ -292,7 +345,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
292
345
|
} catch {
|
|
293
346
|
r = t;
|
|
294
347
|
}
|
|
295
|
-
return `${e.table} ${e.op} ${String(n)} ${
|
|
348
|
+
return `${e.table} ${e.op} ${String(n)} ${Oe(r)}`;
|
|
296
349
|
}
|
|
297
350
|
admit(e) {
|
|
298
351
|
let t = this.key(e);
|
|
@@ -312,9 +365,9 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
312
365
|
1062,
|
|
313
366
|
1586
|
|
314
367
|
]), Z = async (e) => {
|
|
315
|
-
let t = e.variant ?? "mysql", n =
|
|
368
|
+
let t = e.variant ?? "mysql", n = I({ scope: `voltro:${t}` }), a = e.changeStrategy ?? "inline", o = a;
|
|
316
369
|
a === "cdc" && !e.cdcConfig && (n.warn("changeStrategy='cdc' requires cdcConfig (serverId + connection); falling back to 'inline'."), o = "inline");
|
|
317
|
-
let s = e.tracerLayer ? r.mergeAll(e.sqlLayer, e.tracerLayer) : e.sqlLayer, c = i.make(s), l = new Q(await c.runPromise(
|
|
370
|
+
let s = e.tracerLayer ? r.mergeAll(e.sqlLayer, e.tracerLayer) : e.sqlLayer, c = i.make(s), l = new Q(await c.runPromise(L), c, t, o);
|
|
318
371
|
return o === "cdc" && e.cdcConfig && await l.startCdcConsumer(e.cdcConfig), l;
|
|
319
372
|
}, Q = class e {
|
|
320
373
|
sql;
|
|
@@ -326,20 +379,26 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
326
379
|
inflightTxns = 0;
|
|
327
380
|
log;
|
|
328
381
|
cdcHandle = null;
|
|
382
|
+
cdcIncludeTables = void 0;
|
|
383
|
+
cdcExcluded = [];
|
|
384
|
+
cdcReporter;
|
|
329
385
|
cdcCheckpointTimer = null;
|
|
386
|
+
cdcAdminRuntime = null;
|
|
330
387
|
cdcPendingPosition = null;
|
|
331
388
|
cdcReplicaId = "";
|
|
332
389
|
cdcStreamName = "default";
|
|
333
390
|
cdcGate;
|
|
334
391
|
reportEagerFallback;
|
|
335
392
|
constructor(e, t, n, r = "inline", i = null, a, o) {
|
|
336
|
-
this.sql = e, this.runtime = t, this.variant = n, this.changeStrategy = r, this.namespace = i, this.log =
|
|
393
|
+
this.sql = e, this.runtime = t, this.variant = n, this.changeStrategy = r, this.namespace = i, this.log = I({ scope: `voltro:${n}` }), this.cdcReporter = ye((e, t) => {
|
|
394
|
+
e === "error" ? this.log.error(t) : e === "warn" ? this.log.warn(t) : this.log.info(t);
|
|
395
|
+
}), this.reportEagerFallback = k(this.log), this.emitter = a ?? new oe(), M(this.emitter), this.cdcGate = o ?? new Ae(n);
|
|
337
396
|
}
|
|
338
397
|
withNamespace(t) {
|
|
339
398
|
return t === this.namespace ? this : new e(this.sql, this.runtime, this.variant, this.changeStrategy, t, this.emitter, this.cdcGate);
|
|
340
399
|
}
|
|
341
400
|
nsT(e) {
|
|
342
|
-
return
|
|
401
|
+
return j(this.namespace, e);
|
|
343
402
|
}
|
|
344
403
|
get dialectId() {
|
|
345
404
|
return this.variant;
|
|
@@ -351,7 +410,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
351
410
|
};
|
|
352
411
|
}
|
|
353
412
|
async executeQuery(e, t, r) {
|
|
354
|
-
let i = x(e, this.sql, this.namespace), a = t ? n.provideService(i,
|
|
413
|
+
let i = x(e, this.sql, this.namespace), a = t ? n.provideService(i, R, t) : i, o = await A(this.variant, "select", () => this.runtime.runPromise(a));
|
|
355
414
|
return S(o, e.table, this.variant);
|
|
356
415
|
}
|
|
357
416
|
get supportsInsertReturning() {
|
|
@@ -364,10 +423,10 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
364
423
|
return !1;
|
|
365
424
|
}
|
|
366
425
|
async executeInsert(e, t, r, i, a) {
|
|
367
|
-
t =
|
|
426
|
+
t = P(e, t);
|
|
368
427
|
let o = this.sql;
|
|
369
428
|
if (this.supportsInsertReturning) {
|
|
370
|
-
let s = o`INSERT INTO ${o(this.nsT(e))} ${o.insert(C(t, e))} RETURNING *`, c = r ? n.provideService(s,
|
|
429
|
+
let s = o`INSERT INTO ${o(this.nsT(e))} ${o.insert(C(t, e))} RETURNING *`, c = r ? n.provideService(s, R, r) : s, l = (await this.runtime.runPromise(c))[0];
|
|
371
430
|
if (!l) throw Error(`MysqlStore.insert: no row returned for table '${e}'`);
|
|
372
431
|
return await this.routeEvent({
|
|
373
432
|
table: e,
|
|
@@ -386,9 +445,9 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
386
445
|
new: t
|
|
387
446
|
}, i, r, a), t;
|
|
388
447
|
}
|
|
389
|
-
let l = o`INSERT INTO ${o(this.nsT(e))} ${o.insert(s)}`, u = r ? n.provideService(l,
|
|
448
|
+
let l = o`INSERT INTO ${o(this.nsT(e))} ${o.insert(s)}`, u = r ? n.provideService(l, R, r) : l;
|
|
390
449
|
await this.runtime.runPromise(u);
|
|
391
|
-
let d = o`SELECT * FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${c}`, f = r ? n.provideService(d,
|
|
450
|
+
let d = o`SELECT * FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${c}`, f = r ? n.provideService(d, R, r) : d, p = (await this.runtime.runPromise(f))[0];
|
|
392
451
|
if (!p) throw Error(`MysqlStore.insert: row not found post-insert in '${e}'`);
|
|
393
452
|
return await this.routeEvent({
|
|
394
453
|
table: e,
|
|
@@ -401,16 +460,16 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
401
460
|
let i = this.sql;
|
|
402
461
|
return this.runPinned(r, (r) => n.gen(this, function* () {
|
|
403
462
|
let a = i`INSERT INTO ${i(this.nsT(e))} ${i.insert(t)}`;
|
|
404
|
-
yield* n.provideService(a,
|
|
405
|
-
let o = (yield* n.provideService(i`SELECT LAST_INSERT_ID() AS ${i("lastId")}`,
|
|
406
|
-
return o === void 0 || Number(o) === 0 ? yield* n.fail(/* @__PURE__ */ Error(`MysqlStore.insert: LAST_INSERT_ID() returned no id for '${e}' — is the primary key AUTO_INCREMENT?`)) : (yield* n.provideService(i`SELECT * FROM ${i(this.nsT(e))} WHERE ${i("id")} = ${o}`,
|
|
463
|
+
yield* n.provideService(a, R, r);
|
|
464
|
+
let o = (yield* n.provideService(i`SELECT LAST_INSERT_ID() AS ${i("lastId")}`, R, r))[0]?.lastId;
|
|
465
|
+
return o === void 0 || Number(o) === 0 ? yield* n.fail(/* @__PURE__ */ Error(`MysqlStore.insert: LAST_INSERT_ID() returned no id for '${e}' — is the primary key AUTO_INCREMENT?`)) : (yield* n.provideService(i`SELECT * FROM ${i(this.nsT(e))} WHERE ${i("id")} = ${o}`, R, r))[0] || (yield* n.fail(/* @__PURE__ */ Error(`MysqlStore.insert: row not found post-insert in '${e}'`)));
|
|
407
466
|
}), "insert");
|
|
408
467
|
}
|
|
409
468
|
async runPinned(e, t, r) {
|
|
410
469
|
if (e) return this.runtime.runPromise(t(e));
|
|
411
470
|
this.inflightTxns++;
|
|
412
471
|
try {
|
|
413
|
-
let e = n.suspend(() => this.sql.withTransaction(n.flatMap(n.serviceOption(
|
|
472
|
+
let e = n.suspend(() => this.sql.withTransaction(n.flatMap(n.serviceOption(R), (e) => a.isNone(e) ? n.fail(/* @__PURE__ */ Error(`MysqlStore.${r}: TransactionConnection missing.`)) : t(e.value)))), i = s.exponential("10 millis").pipe(s.compose(s.recurs(3)), s.whileInput(q)), o = e.pipe(n.retry(i), n.withSpan(`store.${r}`, { attributes: {
|
|
414
473
|
"db.system": this.variant,
|
|
415
474
|
"db.operation": r
|
|
416
475
|
} }));
|
|
@@ -420,16 +479,16 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
420
479
|
}
|
|
421
480
|
}
|
|
422
481
|
async executeInsertMany(e, t, r, i, a) {
|
|
423
|
-
if (t =
|
|
482
|
+
if (t = ae(e, t), t.length === 0) return [];
|
|
424
483
|
let o = this.sql, s = t.map((t) => C(t, e)), c = _(s, g(this.variant));
|
|
425
484
|
if (this.supportsInsertReturning) {
|
|
426
485
|
let t = (t) => o`INSERT INTO ${o(this.nsT(e))} ${o.insert(t)} RETURNING *`, s;
|
|
427
486
|
if (c.length === 1) {
|
|
428
|
-
let e = t(c[0]), i = r ? n.provideService(e,
|
|
487
|
+
let e = t(c[0]), i = r ? n.provideService(e, R, r) : e;
|
|
429
488
|
s = await this.runtime.runPromise(i);
|
|
430
489
|
} else if (r) {
|
|
431
490
|
let e = [];
|
|
432
|
-
for (let i of c) e.push(...await this.runtime.runPromise(n.provideService(t(i),
|
|
491
|
+
for (let i of c) e.push(...await this.runtime.runPromise(n.provideService(t(i), R, r)));
|
|
433
492
|
s = e;
|
|
434
493
|
} else s = await this.runtime.runPromise(o.withTransaction(n.map(n.forEach(c, t, { concurrency: 1 }), (e) => e.flat())));
|
|
435
494
|
for (let t of s) await this.routeEvent({
|
|
@@ -454,16 +513,16 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
454
513
|
if (l.some((e) => e === void 0)) throw Error("MysqlStore.insertMany: rows mix client-supplied and missing 'id's — supply an id for every row or none (AUTO_INCREMENT recovery).");
|
|
455
514
|
let u = (t) => o`INSERT INTO ${o(this.nsT(e))} ${o.insert(t)}`;
|
|
456
515
|
if (c.length === 1) {
|
|
457
|
-
let e = r ? n.provideService(u(c[0]),
|
|
516
|
+
let e = r ? n.provideService(u(c[0]), R, r) : u(c[0]);
|
|
458
517
|
await this.runtime.runPromise(e);
|
|
459
|
-
} else if (r) for (let e of c) await this.runtime.runPromise(n.provideService(u(e),
|
|
518
|
+
} else if (r) for (let e of c) await this.runtime.runPromise(n.provideService(u(e), R, r));
|
|
460
519
|
else await this.runtime.runPromise(o.withTransaction(n.forEach(c, u, {
|
|
461
520
|
concurrency: 1,
|
|
462
521
|
discard: !0
|
|
463
522
|
})));
|
|
464
523
|
let d = _(l.map((e) => ({ id: e })), g(this.variant)), f = [];
|
|
465
524
|
for (let t of d) {
|
|
466
|
-
let i = o`SELECT * FROM ${o(this.nsT(e))} WHERE ${o("id")} IN ${o.in(t.map((e) => e.id))}`, a = r ? n.provideService(i,
|
|
525
|
+
let i = o`SELECT * FROM ${o(this.nsT(e))} WHERE ${o("id")} IN ${o.in(t.map((e) => e.id))}`, a = r ? n.provideService(i, R, r) : i;
|
|
467
526
|
f.push(...await this.runtime.runPromise(a));
|
|
468
527
|
}
|
|
469
528
|
let p = new Map(f.map((e) => [e.id, e])), m = l.map((e) => p.get(e)).filter((e) => e !== void 0);
|
|
@@ -481,19 +540,19 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
481
540
|
let a = [];
|
|
482
541
|
for (let o of t) {
|
|
483
542
|
let t = i`INSERT INTO ${i(this.nsT(e))} ${i.insert(o)}`;
|
|
484
|
-
yield* n.provideService(t,
|
|
485
|
-
let s = (yield* n.provideService(i`SELECT LAST_INSERT_ID() AS ${i("firstId")}`,
|
|
543
|
+
yield* n.provideService(t, R, r);
|
|
544
|
+
let s = (yield* n.provideService(i`SELECT LAST_INSERT_ID() AS ${i("firstId")}`, R, r))[0]?.firstId;
|
|
486
545
|
if (s === void 0 || Number(s) === 0) return yield* n.fail(/* @__PURE__ */ Error(`MysqlStore.insertMany: LAST_INSERT_ID() returned no id for '${e}' — is the primary key AUTO_INCREMENT?`));
|
|
487
|
-
let c = Number(s), l = yield* n.provideService(i`SELECT * FROM ${i(this.nsT(e))} WHERE ${i("id")} BETWEEN ${c} AND ${c + o.length - 1} ORDER BY ${i("id")} ASC`,
|
|
546
|
+
let c = Number(s), l = yield* n.provideService(i`SELECT * FROM ${i(this.nsT(e))} WHERE ${i("id")} BETWEEN ${c} AND ${c + o.length - 1} ORDER BY ${i("id")} ASC`, R, r);
|
|
488
547
|
a.push(...l);
|
|
489
548
|
}
|
|
490
549
|
return a;
|
|
491
550
|
}), "insert");
|
|
492
551
|
}
|
|
493
552
|
async executePatchJson(e, t, r, i, a, o, s) {
|
|
494
|
-
let c = this.sql, l = r.split("."), u = l[0], d = l.slice(1), f = JSON.stringify(i ?? null), p = d.length === 0 ? "$" : `$.${d.join(".")}`, m = c`UPDATE ${c(this.nsT(e))} SET ${c(u)} = JSON_SET(COALESCE(${c(u)}, '{}'), ${p}, CAST(${f} AS JSON)) WHERE ${c("id")} = ${t}`, h = a ? n.provideService(m,
|
|
553
|
+
let c = this.sql, l = r.split("."), u = l[0], d = l.slice(1), f = JSON.stringify(i ?? null), p = d.length === 0 ? "$" : `$.${d.join(".")}`, m = c`UPDATE ${c(this.nsT(e))} SET ${c(u)} = JSON_SET(COALESCE(${c(u)}, '{}'), ${p}, CAST(${f} AS JSON)) WHERE ${c("id")} = ${t}`, h = a ? n.provideService(m, R, a) : m, g = await this.runtime.runPromise(h);
|
|
495
554
|
if (g && typeof g.affectedRows == "number" && g.affectedRows === 0) return null;
|
|
496
|
-
let _ = c`SELECT * FROM ${c(this.nsT(e))} WHERE ${c("id")} = ${t}`, v = a ? n.provideService(_,
|
|
555
|
+
let _ = c`SELECT * FROM ${c(this.nsT(e))} WHERE ${c("id")} = ${t}`, v = a ? n.provideService(_, R, a) : _, y = (await this.runtime.runPromise(v))[0];
|
|
497
556
|
return y ? (await this.routeEvent({
|
|
498
557
|
table: e,
|
|
499
558
|
op: "update",
|
|
@@ -504,7 +563,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
504
563
|
async executeUpdate(e, t, r, i, a, o) {
|
|
505
564
|
let s = this.sql;
|
|
506
565
|
if (this.supportsUpdateReturning) {
|
|
507
|
-
let c = s`UPDATE ${s(this.nsT(e))} SET ${s.update(C(r, e))} WHERE ${s("id")} = ${t} RETURNING *`, l = i ? n.provideService(c,
|
|
566
|
+
let c = s`UPDATE ${s(this.nsT(e))} SET ${s.update(C(r, e))} WHERE ${s("id")} = ${t} RETURNING *`, l = i ? n.provideService(c, R, i) : c, u = (await this.runtime.runPromise(l))[0];
|
|
508
567
|
return u ? (await this.routeEvent({
|
|
509
568
|
table: e,
|
|
510
569
|
op: "update",
|
|
@@ -512,9 +571,9 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
512
571
|
new: u
|
|
513
572
|
}, a, i, o), u) : null;
|
|
514
573
|
}
|
|
515
|
-
let c = s`UPDATE ${s(this.nsT(e))} SET ${s.update(C(r, e))} WHERE ${s("id")} = ${t}`, l = i ? n.provideService(c,
|
|
574
|
+
let c = s`UPDATE ${s(this.nsT(e))} SET ${s.update(C(r, e))} WHERE ${s("id")} = ${t}`, l = i ? n.provideService(c, R, i) : c, u = await this.runtime.runPromise(l);
|
|
516
575
|
if (u && typeof u.affectedRows == "number" && u.affectedRows === 0) return null;
|
|
517
|
-
let d = s`SELECT * FROM ${s(this.nsT(e))} WHERE ${s("id")} = ${t}`, f = i ? n.provideService(d,
|
|
576
|
+
let d = s`SELECT * FROM ${s(this.nsT(e))} WHERE ${s("id")} = ${t}`, f = i ? n.provideService(d, R, i) : d, p = (await this.runtime.runPromise(f))[0];
|
|
518
577
|
return p ? (await this.routeEvent({
|
|
519
578
|
table: e,
|
|
520
579
|
op: "update",
|
|
@@ -525,7 +584,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
525
584
|
async executeDelete(e, t, r, i, a) {
|
|
526
585
|
let o = this.sql;
|
|
527
586
|
if (this.supportsDeleteReturning) {
|
|
528
|
-
let s = o`DELETE FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t} RETURNING *`, c = r ? n.provideService(s,
|
|
587
|
+
let s = o`DELETE FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t} RETURNING *`, c = r ? n.provideService(s, R, r) : s, l = (await this.runtime.runPromise(c))[0];
|
|
529
588
|
return l ? (await this.routeEvent({
|
|
530
589
|
table: e,
|
|
531
590
|
op: "delete",
|
|
@@ -533,9 +592,9 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
533
592
|
new: null
|
|
534
593
|
}, i, r, a), !0) : !1;
|
|
535
594
|
}
|
|
536
|
-
let s = o`SELECT * FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t}`, c = r ? n.provideService(s,
|
|
595
|
+
let s = o`SELECT * FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t}`, c = r ? n.provideService(s, R, r) : s, l = (await this.runtime.runPromise(c))[0];
|
|
537
596
|
if (!l) return !1;
|
|
538
|
-
let u = o`DELETE FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t}`, d = r ? n.provideService(u,
|
|
597
|
+
let u = o`DELETE FROM ${o(this.nsT(e))} WHERE ${o("id")} = ${t}`, d = r ? n.provideService(u, R, r) : u;
|
|
539
598
|
return await this.runtime.runPromise(d), await this.routeEvent({
|
|
540
599
|
table: e,
|
|
541
600
|
op: "delete",
|
|
@@ -545,17 +604,17 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
545
604
|
}
|
|
546
605
|
async appendInTxn(e, t, r) {
|
|
547
606
|
let i = this.sql, a = i`INSERT INTO ${i(this.nsT(e))} ${i.insert(C(t, e))}`;
|
|
548
|
-
await this.runtime.runPromise(r ? n.provideService(a,
|
|
607
|
+
await this.runtime.runPromise(r ? n.provideService(a, R, r) : a);
|
|
549
608
|
}
|
|
550
609
|
async maxInTxn(e, t, r, i) {
|
|
551
|
-
let a = this.sql, o = Object.entries(r).map(([e, t]) => a`${a(e)} = ${t}`), s = a`SELECT MAX(${a(t)}) AS ${a("m")} FROM ${a(this.nsT(e))} WHERE ${a.and(o)}`, c = (await this.runtime.runPromise(i ? n.provideService(s,
|
|
610
|
+
let a = this.sql, o = Object.entries(r).map(([e, t]) => a`${a(e)} = ${t}`), s = a`SELECT MAX(${a(t)}) AS ${a("m")} FROM ${a(this.nsT(e))} WHERE ${a.and(o)}`, c = (await this.runtime.runPromise(i ? n.provideService(s, R, i) : s))[0]?.m;
|
|
552
611
|
return c == null ? null : Number(c);
|
|
553
612
|
}
|
|
554
613
|
async routeEvent(e, t, n = null, r) {
|
|
555
614
|
if (e = {
|
|
556
615
|
...p(r),
|
|
557
616
|
...e
|
|
558
|
-
},
|
|
617
|
+
}, N(e.table) && await ie({
|
|
559
618
|
append: (e, t) => this.appendInTxn(e, t, n),
|
|
560
619
|
maxOf: (e, t, r) => this.maxInTxn(e, t, r, n)
|
|
561
620
|
}, {
|
|
@@ -567,7 +626,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
567
626
|
subjectId: e.subjectId
|
|
568
627
|
}), this.changeStrategy === "cdc") {
|
|
569
628
|
let t = (e.op === "delete" ? e.old : e.new)?.id;
|
|
570
|
-
t != null &&
|
|
629
|
+
t != null && ee(m(e.table, e.op, t), {
|
|
571
630
|
...e.traceId === void 0 ? {} : { traceId: e.traceId },
|
|
572
631
|
...e.subjectId === void 0 ? {} : { subjectId: e.subjectId }
|
|
573
632
|
});
|
|
@@ -605,12 +664,12 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
605
664
|
}
|
|
606
665
|
return await this.executeUpdate(e, o.id, a, r, i) ?? o;
|
|
607
666
|
}
|
|
608
|
-
return
|
|
667
|
+
return F((n) => this.executeInsert(e, t, r, i, n));
|
|
609
668
|
}
|
|
610
669
|
async executeMariadbUpsert(e, t, r, i, a, o) {
|
|
611
670
|
let s = this.sql, c = C(t, e), l = Object.keys(c).filter((e) => c[e] !== void 0), u = r.update === void 0 ? l.filter((e) => e !== "id" && !r.conflictColumns.includes(e)) : r.update, d = u.length > 0 ? s.csv(u.map((e) => s`${s(e)} = VALUES(${s(e)})`)) : s`${s(r.conflictColumns[0])} = VALUES(${s(r.conflictColumns[0])})`, f = await this.runPinned(i, (i) => n.tryPromise({
|
|
612
671
|
try: async () => {
|
|
613
|
-
let a = s`INSERT INTO ${s(this.nsT(e))} ${s.insert(c)} ON DUPLICATE KEY UPDATE ${d} RETURNING *`, o = (await this.runtime.runPromise(n.provideService(a,
|
|
672
|
+
let a = s`INSERT INTO ${s(this.nsT(e))} ${s.insert(c)} ON DUPLICATE KEY UPDATE ${d} RETURNING *`, o = (await this.runtime.runPromise(n.provideService(a, R, i)))[0];
|
|
614
673
|
if (!o) throw Error(`MysqlStore.upsert: no row returned for table '${e}'`);
|
|
615
674
|
let l = t.id;
|
|
616
675
|
if (l != null && o.id !== l) throw Error(`MysqlStore.upsert: the row written to '${e}' is not the row that was passed in. Upserting id '${String(l)}' matched an existing row with id '${String(o.id)}' on a DIFFERENT unique constraint than the conflictColumns [${r.conflictColumns.join(", ")}] you named, so that row would have been updated and yours never written. Nothing was changed. Name the constraint that actually collides, or resolve the duplicate first.`);
|
|
@@ -626,7 +685,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
626
685
|
}, a, i, o), f;
|
|
627
686
|
}
|
|
628
687
|
async executeInsertIgnore(e, t, r, i, a, o) {
|
|
629
|
-
if (t =
|
|
688
|
+
if (t = P(e, t), t.id === void 0) return await this.findByConflict(e, t, r.conflictColumns, i) || F((n) => this.executeInsert(e, t, i, a, n));
|
|
630
689
|
let s = await this.runPinned(i, (i) => n.tryPromise({
|
|
631
690
|
try: () => this.decideInsertIgnore(e, t, r, i),
|
|
632
691
|
catch: (e) => e
|
|
@@ -639,7 +698,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
639
698
|
}, a, i, o), s.row;
|
|
640
699
|
}
|
|
641
700
|
async decideInsertIgnore(e, t, r, i) {
|
|
642
|
-
let a = this.sql, o = C(t, e), s = (e) => this.runtime.runPromise(n.provideService(e,
|
|
701
|
+
let a = this.sql, o = C(t, e), s = (e) => this.runtime.runPromise(n.provideService(e, R, i));
|
|
643
702
|
if (this.supportsInsertReturning) {
|
|
644
703
|
let n = (await s(a`INSERT IGNORE INTO ${a(this.nsT(e))} ${a.insert(o)} RETURNING *`))[0];
|
|
645
704
|
if (n) return {
|
|
@@ -690,7 +749,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
690
749
|
if (e === null) return [];
|
|
691
750
|
try {
|
|
692
751
|
let t = this.sql`SHOW WARNINGS`.unprepared;
|
|
693
|
-
return (await this.runtime.runPromise(n.provideService(t,
|
|
752
|
+
return (await this.runtime.runPromise(n.provideService(t, R, e))).map((e) => ({
|
|
694
753
|
code: Number(e.Code ?? e.code ?? 0),
|
|
695
754
|
message: String(e.Message ?? e.message ?? "")
|
|
696
755
|
}));
|
|
@@ -700,7 +759,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
700
759
|
}
|
|
701
760
|
async findByConflict(e, t, r, i) {
|
|
702
761
|
if (r.length === 0) return;
|
|
703
|
-
let a = this.sql, o = r.map((e) => a`${a(e)} = ${t[e]}`), s = a`SELECT * FROM ${a(this.nsT(e))} WHERE ${a.and(o)} LIMIT 1`, c = i ? n.provideService(s,
|
|
762
|
+
let a = this.sql, o = r.map((e) => a`${a(e)} = ${t[e]}`), s = a`SELECT * FROM ${a(this.nsT(e))} WHERE ${a.and(o)} LIMIT 1`, c = i ? n.provideService(s, R, i) : s;
|
|
704
763
|
return (await this.runtime.runPromise(c))[0];
|
|
705
764
|
}
|
|
706
765
|
query(e) {
|
|
@@ -708,13 +767,13 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
708
767
|
}
|
|
709
768
|
raw(e, t) {
|
|
710
769
|
let n = b(e, this.sql);
|
|
711
|
-
return
|
|
770
|
+
return A(this.variant, "raw", () => this.runtime.runPromise(n));
|
|
712
771
|
}
|
|
713
772
|
async runWithEager(e, t) {
|
|
714
|
-
if (!
|
|
773
|
+
if (!D(e)) return this.executeQuery(e, t);
|
|
715
774
|
let r = this.variant === "mariadb" ? "mariadb" : "mysql", i = this.namespace === null ? v(e, this.sql, r) : null;
|
|
716
775
|
if (i !== null) try {
|
|
717
|
-
let e = t ? n.provideService(i.fragment,
|
|
776
|
+
let e = t ? n.provideService(i.fragment, R, t) : i.fragment, r = await A(this.variant, "select", () => this.runtime.runPromise(e));
|
|
718
777
|
return i.decode(r);
|
|
719
778
|
} catch (t) {
|
|
720
779
|
if (t instanceof u) throw t;
|
|
@@ -731,42 +790,51 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
731
790
|
reason: "not-compilable"
|
|
732
791
|
});
|
|
733
792
|
let a = await this.executeQuery(e, t);
|
|
734
|
-
return f(a, e.eager, e.sourceTable ??
|
|
793
|
+
return f(a, e.eager, e.sourceTable ?? te(e.table), (e) => this.executeQuery(e, t));
|
|
735
794
|
}
|
|
736
795
|
getInternalRunWithEager() {
|
|
737
796
|
return this.runWithEager.bind(this);
|
|
738
797
|
}
|
|
798
|
+
async writeWithRecorders(e, t) {
|
|
799
|
+
let r = [], i = await this.runPinned(null, (i) => n.tryPromise({
|
|
800
|
+
try: () => e(t, i, r),
|
|
801
|
+
catch: (e) => e
|
|
802
|
+
}), "insert");
|
|
803
|
+
for (let e of r) this.emitChange(e);
|
|
804
|
+
return i;
|
|
805
|
+
}
|
|
739
806
|
async localWrite(e, t, n) {
|
|
740
|
-
|
|
741
|
-
|
|
807
|
+
let r = (e) => N(t) ? this.writeWithRecorders(n, e) : n(e, null, null);
|
|
808
|
+
return A(this.variant, e, async () => {
|
|
809
|
+
if (this.changeStrategy !== "cdc") return F(r);
|
|
742
810
|
h(t);
|
|
743
811
|
try {
|
|
744
|
-
return await
|
|
812
|
+
return await F(r);
|
|
745
813
|
} finally {
|
|
746
814
|
w(t);
|
|
747
815
|
}
|
|
748
816
|
});
|
|
749
817
|
}
|
|
750
818
|
insert(e, t) {
|
|
751
|
-
return this.localWrite("insert", e, (n) => this.executeInsert(e, t,
|
|
819
|
+
return this.localWrite("insert", e, (n, r, i) => this.executeInsert(e, t, r, i, n));
|
|
752
820
|
}
|
|
753
821
|
insertMany(e, t) {
|
|
754
|
-
return this.localWrite("insert", e, (n) => this.executeInsertMany(e, t,
|
|
822
|
+
return this.localWrite("insert", e, (n, r, i) => this.executeInsertMany(e, t, r, i, n));
|
|
755
823
|
}
|
|
756
824
|
patchJson(e, t, n, r) {
|
|
757
|
-
return this.localWrite("update", e, (i) => this.executePatchJson(e, t, n, r,
|
|
825
|
+
return this.localWrite("update", e, (i, a, o) => this.executePatchJson(e, t, n, r, a, o, i));
|
|
758
826
|
}
|
|
759
827
|
upsert(e, t, n) {
|
|
760
|
-
return this.localWrite("upsert", e, (r) => this.executeUpsert(e, t, n,
|
|
828
|
+
return this.localWrite("upsert", e, (r, i, a) => this.executeUpsert(e, t, n, i, a, r));
|
|
761
829
|
}
|
|
762
830
|
insertIgnore(e, t, n) {
|
|
763
|
-
return this.localWrite("upsert", e, (r) => this.executeInsertIgnore(e, t, n,
|
|
831
|
+
return this.localWrite("upsert", e, (r, i, a) => this.executeInsertIgnore(e, t, n, i, a, r));
|
|
764
832
|
}
|
|
765
833
|
update(e, t, n) {
|
|
766
|
-
return this.localWrite("update", e, (r) => this.executeUpdate(e, t, n,
|
|
834
|
+
return this.localWrite("update", e, (r, i, a) => this.executeUpdate(e, t, n, i, a, r));
|
|
767
835
|
}
|
|
768
836
|
delete(e, t) {
|
|
769
|
-
return this.localWrite("delete", e, (n) => this.executeDelete(e, t,
|
|
837
|
+
return this.localWrite("delete", e, (n, r, i) => this.executeDelete(e, t, r, i, n));
|
|
770
838
|
}
|
|
771
839
|
async updateMany(e, t, r) {
|
|
772
840
|
if (this.supportsUpdateReturning) {
|
|
@@ -782,13 +850,13 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
782
850
|
let i = this.sql, o = y(r.where, i, this.namespace);
|
|
783
851
|
this.changeStrategy === "cdc" && h(e), this.inflightTxns++;
|
|
784
852
|
try {
|
|
785
|
-
let r = n.suspend(() => this.sql.withTransaction(n.flatMap(n.serviceOption(
|
|
853
|
+
let r = n.suspend(() => this.sql.withTransaction(n.flatMap(n.serviceOption(R), (r) => {
|
|
786
854
|
if (a.isNone(r)) return n.fail(/* @__PURE__ */ Error("MysqlStore.updateMany: TransactionConnection missing."));
|
|
787
855
|
let s = r.value, c = i`SELECT id FROM ${i(this.nsT(e))} WHERE ${o} FOR UPDATE`, l = i`UPDATE ${i(this.nsT(e))} SET ${i.update(C(t, e))} WHERE ${o}`;
|
|
788
|
-
return n.flatMap(n.provideService(c,
|
|
856
|
+
return n.flatMap(n.provideService(c, R, s), (t) => {
|
|
789
857
|
if (t.length === 0) return n.succeed([]);
|
|
790
858
|
let r = t.map((e) => e.id), a = i`SELECT * FROM ${i(this.nsT(e))} WHERE ${i("id")} IN ${i.in(r)}`;
|
|
791
|
-
return n.flatMap(n.provideService(l,
|
|
859
|
+
return n.flatMap(n.provideService(l, R, s), () => n.provideService(a, R, s));
|
|
792
860
|
});
|
|
793
861
|
}))), c = s.exponential("10 millis").pipe(s.compose(s.recurs(3)), s.whileInput(q)), l = r.pipe(n.retry(c), n.withSpan("store.updateMany", { attributes: {
|
|
794
862
|
"db.system": this.variant,
|
|
@@ -821,10 +889,10 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
821
889
|
}
|
|
822
890
|
this.changeStrategy === "cdc" && h(e), this.inflightTxns++;
|
|
823
891
|
try {
|
|
824
|
-
let t = n.suspend(() => this.sql.withTransaction(n.flatMap(n.serviceOption(
|
|
892
|
+
let t = n.suspend(() => this.sql.withTransaction(n.flatMap(n.serviceOption(R), (t) => {
|
|
825
893
|
if (a.isNone(t)) return n.fail(/* @__PURE__ */ Error("MysqlStore.deleteMany: TransactionConnection missing."));
|
|
826
894
|
let o = t.value, s = r`SELECT * FROM ${r(this.nsT(e))} WHERE ${i} FOR UPDATE`, c = r`DELETE FROM ${r(this.nsT(e))} WHERE ${i}`;
|
|
827
|
-
return n.flatMap(n.provideService(s,
|
|
895
|
+
return n.flatMap(n.provideService(s, R, o), (e) => e.length === 0 ? n.succeed(e) : n.as(n.provideService(c, R, o), e));
|
|
828
896
|
}))), o = s.exponential("10 millis").pipe(s.compose(s.recurs(3)), s.whileInput(q)), c = t.pipe(n.retry(o), n.withSpan("store.deleteMany", { attributes: {
|
|
829
897
|
"db.system": this.variant,
|
|
830
898
|
"db.operation": "delete"
|
|
@@ -843,23 +911,27 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
843
911
|
}
|
|
844
912
|
}
|
|
845
913
|
emitChange(e) {
|
|
846
|
-
|
|
914
|
+
O(e.table) && (this.changeStrategy === "cdc" && !this.cdcGate.admit(e) || this.emitter.emit("change", e));
|
|
847
915
|
}
|
|
848
916
|
async startCdcConsumer(e) {
|
|
849
917
|
if (this.cdcHandle) return;
|
|
850
|
-
await this.assertBinlogConfig(), this.cdcReplicaId = e.replicaId
|
|
851
|
-
|
|
852
|
-
|
|
853
|
-
|
|
918
|
+
await this.assertBinlogConfig(), this.cdcReplicaId = e.replicaId, this.cdcAdminRuntime = i.make(z({
|
|
919
|
+
...e.connection,
|
|
920
|
+
maxConnections: 1
|
|
921
|
+
}));
|
|
922
|
+
let t = await this.readCdcOffset(e.replicaId), n = t !== null && await this.binlogFileExists(t.filename);
|
|
923
|
+
t !== null && !n && this.log.warn(`cdc: the persisted offset points at ${t.filename}, which the server no longer has (the binlog was purged or rotated away while this replica was down). Starting at the current end instead — changes written in the gap are not replayed, and subscriptions self-heal on the next change.`);
|
|
924
|
+
let r = (n ? t : null) ?? await this.resolveBinlogEnd(), a = await this.findUndecodableCdcTables(e.includeTables);
|
|
925
|
+
this.cdcIncludeTables = e.includeTables, this.cdcExcluded = a, this.cdcReporter.announce(a, e.exclusionRefreshFollows === !0), this.cdcHandle = await K({
|
|
854
926
|
connection: e.connection,
|
|
855
927
|
serverId: e.serverId,
|
|
856
928
|
variant: this.variant,
|
|
857
929
|
...e.includeTables ? { includeTables: e.includeTables } : {},
|
|
858
|
-
...
|
|
930
|
+
...a.length > 0 ? { undecodableTables: a } : {},
|
|
859
931
|
...e.watchdogIntervalMs === void 0 ? {} : { watchdogIntervalMs: e.watchdogIntervalMs },
|
|
860
932
|
...e.stallThresholdMs === void 0 ? {} : { stallThresholdMs: e.stallThresholdMs },
|
|
861
933
|
...e.keepAliveInitialDelayMs === void 0 ? {} : { keepAliveInitialDelayMs: e.keepAliveInitialDelayMs },
|
|
862
|
-
startPosition:
|
|
934
|
+
startPosition: r,
|
|
863
935
|
onChange: (e) => this.injectExternalChange(e),
|
|
864
936
|
onPosition: (e) => {
|
|
865
937
|
this.cdcPendingPosition = e;
|
|
@@ -880,6 +952,16 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
880
952
|
if (t.format !== "ROW") throw Error(`${this.variant} binlog CDC requires binlog_format=ROW (got '${t.format}'). Set --binlog-format=ROW, or CDC=0 for inline emit.`);
|
|
881
953
|
t.rowImage !== "FULL" && this.log.warn(`cdc: binlog_row_image='${t.rowImage}' (expected FULL) — UPDATE/DELETE before-images may be incomplete. Set --binlog-row-image=FULL.`);
|
|
882
954
|
}
|
|
955
|
+
async binlogFileExists(e) {
|
|
956
|
+
let t = this.cdcAdminRuntime;
|
|
957
|
+
if (t === null) return !0;
|
|
958
|
+
try {
|
|
959
|
+
let r = await t.runPromise(n.flatMap(L, (e) => e`SHOW BINARY LOGS`.unprepared));
|
|
960
|
+
return r.length === 0 || r.some((t) => (t.Log_name ?? t.log_name) === e);
|
|
961
|
+
} catch (e) {
|
|
962
|
+
return this.log.debug(`cdc: could not list binary logs (${e?.message ?? String(e)}) — trusting the persisted offset`), !0;
|
|
963
|
+
}
|
|
964
|
+
}
|
|
883
965
|
async resolveBinlogEnd() {
|
|
884
966
|
let e = this.sql, t = this.variant === "mysql" ? e`SHOW BINARY LOG STATUS`.unprepared : e`SHOW MASTER STATUS`.unprepared, n = this.variant === "mysql" ? e`SHOW MASTER STATUS`.unprepared : e`SHOW BINARY LOG STATUS`.unprepared, r = async (e) => {
|
|
885
967
|
let t = (await this.runtime.runPromise(e))[0];
|
|
@@ -887,7 +969,8 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
887
969
|
filename: t.File,
|
|
888
970
|
position: Number(t.Position)
|
|
889
971
|
} : null;
|
|
890
|
-
};
|
|
972
|
+
}, i = await this.resolveBinlogEndOverCdc();
|
|
973
|
+
if (i !== null) return i;
|
|
891
974
|
try {
|
|
892
975
|
return await r(t);
|
|
893
976
|
} catch {
|
|
@@ -898,6 +981,19 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
898
981
|
}
|
|
899
982
|
}
|
|
900
983
|
}
|
|
984
|
+
async resolveBinlogEndOverCdc() {
|
|
985
|
+
let e = this.cdcAdminRuntime;
|
|
986
|
+
if (e === null) return null;
|
|
987
|
+
let t = (e) => n.flatMap(L, (t) => t`${t.unsafe(e)}`.unprepared);
|
|
988
|
+
for (let n of this.variant === "mysql" ? ["SHOW BINARY LOG STATUS", "SHOW MASTER STATUS"] : ["SHOW MASTER STATUS", "SHOW BINARY LOG STATUS"]) try {
|
|
989
|
+
let r = (await e.runPromise(t(n)))[0];
|
|
990
|
+
if (r?.File) return {
|
|
991
|
+
filename: r.File,
|
|
992
|
+
position: Number(r.Position)
|
|
993
|
+
};
|
|
994
|
+
} catch {}
|
|
995
|
+
return null;
|
|
996
|
+
}
|
|
901
997
|
async readCdcOffset(e) {
|
|
902
998
|
try {
|
|
903
999
|
let t = this.sql, n = (await this.runtime.runPromise(t`
|
|
@@ -965,13 +1061,35 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
965
1061
|
getInternalExecuteInsertIgnore() {
|
|
966
1062
|
return this.executeInsertIgnore.bind(this);
|
|
967
1063
|
}
|
|
1064
|
+
async emptyTables(e) {
|
|
1065
|
+
await this.emptyTablesOn(e, null);
|
|
1066
|
+
}
|
|
1067
|
+
async incomingForeignKeys(e) {
|
|
1068
|
+
if (e.length === 0) return [];
|
|
1069
|
+
let t = this.sql;
|
|
1070
|
+
return (await this.runtime.runPromise(t`
|
|
1071
|
+
SELECT TABLE_NAME, COLUMN_NAME, REFERENCED_TABLE_NAME
|
|
1072
|
+
FROM information_schema.KEY_COLUMN_USAGE
|
|
1073
|
+
WHERE TABLE_SCHEMA = DATABASE()
|
|
1074
|
+
AND REFERENCED_TABLE_NAME IN ${t.in(e)}
|
|
1075
|
+
AND TABLE_NAME NOT IN ${t.in(e)}`)).map((e) => ({
|
|
1076
|
+
from: e.TABLE_NAME,
|
|
1077
|
+
column: e.COLUMN_NAME,
|
|
1078
|
+
to: e.REFERENCED_TABLE_NAME
|
|
1079
|
+
}));
|
|
1080
|
+
}
|
|
1081
|
+
async emptyTablesOn(e, t) {
|
|
1082
|
+
if (e.length === 0) return;
|
|
1083
|
+
let r = this.sql, i = (e) => this.nsT(e), a = (e) => r.unsafe(`SET FOREIGN_KEY_CHECKS = ${+!!e}`).unprepared, o = n.acquireUseRelease(a(!1), () => n.forEach(e, (e) => r`DELETE FROM ${r(i(e))}`, { discard: !0 }), () => n.orDie(n.ignore(a(!0)))), s = t === null ? r.withTransaction(o) : n.provideService(o, R, t);
|
|
1084
|
+
await this.runtime.runPromise(s);
|
|
1085
|
+
}
|
|
968
1086
|
async transactional(e) {
|
|
969
1087
|
this.inflightTxns++;
|
|
970
1088
|
try {
|
|
971
|
-
return await
|
|
1089
|
+
return await re({
|
|
972
1090
|
...this.txnSpec("MysqlStore.transactional"),
|
|
973
1091
|
work: e,
|
|
974
|
-
makeView: (e, t) => new
|
|
1092
|
+
makeView: (e, t) => new je(this, e, t)
|
|
975
1093
|
});
|
|
976
1094
|
} finally {
|
|
977
1095
|
this.inflightTxns--;
|
|
@@ -1002,16 +1120,29 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
1002
1120
|
return this.changeStrategy === "cdc" ? "fleet" : "local";
|
|
1003
1121
|
}
|
|
1004
1122
|
injectExternalChange(e) {
|
|
1005
|
-
if (this.changeStrategy === "cdc" && !this.cdcGate.admit(e) || !
|
|
1123
|
+
if (this.changeStrategy === "cdc" && !this.cdcGate.admit(e) || !O(e.table)) return;
|
|
1006
1124
|
let t = (e.op === "delete" ? e.old : e.new)?.id;
|
|
1007
|
-
|
|
1125
|
+
ne(e.table, e.op, t, (t) => {
|
|
1008
1126
|
this.emitter.emit("change", T(e, t));
|
|
1009
1127
|
});
|
|
1010
1128
|
}
|
|
1011
1129
|
run(e) {
|
|
1012
1130
|
return this.runtime.runPromise(e);
|
|
1013
1131
|
}
|
|
1132
|
+
async refreshChangeCaptureExclusions() {
|
|
1133
|
+
if (!this.cdcHandle) return {
|
|
1134
|
+
stillExcluded: [],
|
|
1135
|
+
readmitted: [],
|
|
1136
|
+
newlyExcluded: []
|
|
1137
|
+
};
|
|
1138
|
+
let e = await this.findUndecodableCdcTables(this.cdcIncludeTables), t = be(this.cdcExcluded, e);
|
|
1139
|
+
return this.cdcReporter.settle(t), this.cdcExcluded = e, (t.readmitted.length > 0 || t.newlyExcluded.length > 0) && await this.cdcHandle.setUndecodableTables(e), t;
|
|
1140
|
+
}
|
|
1014
1141
|
async close(e = 5e3) {
|
|
1142
|
+
if (this.cdcReporter.dispose(), this.cdcAdminRuntime) {
|
|
1143
|
+
let e = this.cdcAdminRuntime;
|
|
1144
|
+
this.cdcAdminRuntime = null, await e.dispose().catch(() => void 0);
|
|
1145
|
+
}
|
|
1015
1146
|
if (this.cdcCheckpointTimer &&= (clearInterval(this.cdcCheckpointTimer), null), this.cdcHandle) {
|
|
1016
1147
|
let e = this.cdcHandle.currentPosition();
|
|
1017
1148
|
e && (this.cdcPendingPosition = e), await this.flushCdcOffset(), await this.cdcHandle.stop(), this.cdcHandle = null;
|
|
@@ -1029,7 +1160,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
1029
1160
|
async ping() {
|
|
1030
1161
|
await this.runtime.runPromise(this.sql`SELECT 1`);
|
|
1031
1162
|
}
|
|
1032
|
-
},
|
|
1163
|
+
}, je = class {
|
|
1033
1164
|
parent;
|
|
1034
1165
|
txn;
|
|
1035
1166
|
attr;
|
|
@@ -1041,6 +1172,9 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
1041
1172
|
query(e) {
|
|
1042
1173
|
return this.parent.getInternalRunWithEager()(e, this.txn);
|
|
1043
1174
|
}
|
|
1175
|
+
emptyTables(e) {
|
|
1176
|
+
return this.parent.emptyTablesOn(e, this.txn);
|
|
1177
|
+
}
|
|
1044
1178
|
insert(e, t) {
|
|
1045
1179
|
return this.parent.getInternalExecuteInsert()(e, t, this.txn, this.events, this.attr);
|
|
1046
1180
|
}
|
|
@@ -1102,12 +1236,12 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
1102
1236
|
this.events.length = 0;
|
|
1103
1237
|
}
|
|
1104
1238
|
}
|
|
1105
|
-
}, $ = (e) => e.__mysqlReplicationFriend ?? null,
|
|
1239
|
+
}, $ = (e) => e.__mysqlReplicationFriend ?? null, Me = () => ({
|
|
1106
1240
|
async capturePrimaryPosition(e) {
|
|
1107
1241
|
let t = $(e);
|
|
1108
1242
|
if (t === null) throw Error("mysqlReplicationAdapter: primary is not a MysqlStore (missing __mysqlReplicationFriend).");
|
|
1109
1243
|
return t.runEffect(n.gen(function* () {
|
|
1110
|
-
let e = yield*
|
|
1244
|
+
let e = yield* L, r = (t.variant === "mariadb" ? yield* e`SELECT @@global.gtid_current_pos AS gtid` : yield* e`SELECT @@global.gtid_executed AS gtid`)[0]?.gtid;
|
|
1111
1245
|
return typeof r == "string" ? r : yield* n.die("mysql/mariadb did not return a GTID set");
|
|
1112
1246
|
}));
|
|
1113
1247
|
},
|
|
@@ -1115,14 +1249,14 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
1115
1249
|
let t = $(e);
|
|
1116
1250
|
if (t === null) throw Error("mysqlReplicationAdapter: replica is not a MysqlStore.");
|
|
1117
1251
|
return t.runEffect(n.gen(function* () {
|
|
1118
|
-
let e = yield*
|
|
1252
|
+
let e = yield* L, r = (t.variant === "mariadb" ? yield* e`SELECT @@global.gtid_current_pos AS gtid` : yield* e`SELECT @@global.gtid_executed AS gtid`)[0]?.gtid;
|
|
1119
1253
|
return typeof r == "string" ? r : yield* n.die("mysql/mariadb replica did not return a GTID set");
|
|
1120
1254
|
}));
|
|
1121
1255
|
},
|
|
1122
1256
|
compare(e, t) {
|
|
1123
1257
|
return "behind";
|
|
1124
1258
|
}
|
|
1125
|
-
}),
|
|
1259
|
+
}), Ne = {
|
|
1126
1260
|
id: "mysql",
|
|
1127
1261
|
makeSqlLayer: (e) => H(e),
|
|
1128
1262
|
makeStore: (e) => Z({
|
|
@@ -1131,7 +1265,7 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
1131
1265
|
}),
|
|
1132
1266
|
compileContains: (e, t, n) => n`${e} LIKE ${`%${t.replace(/[\\%_]/g, (e) => `\\${e}`)}%`} ESCAPE '\\'`,
|
|
1133
1267
|
retryFilter: J
|
|
1134
|
-
},
|
|
1268
|
+
}, Pe = {
|
|
1135
1269
|
id: "mariadb",
|
|
1136
1270
|
makeSqlLayer: (e) => H(e),
|
|
1137
1271
|
makeStore: (e) => Z({
|
|
@@ -1142,4 +1276,4 @@ var le = (e) => e ? { rejectUnauthorized: !1 } : void 0, ue = (e) => {
|
|
|
1142
1276
|
retryFilter: J
|
|
1143
1277
|
};
|
|
1144
1278
|
//#endregion
|
|
1145
|
-
export { c as CDC_OFFSETS_TABLE, e as MysqlClient, d as _voltroCdcOffsetsTable, V as connectionFromConfig, z as makeMysqlSqlLayer, H as makeMysqlSqlLayerFromConfig, Z as makeMysqlStore,
|
|
1279
|
+
export { c as CDC_OFFSETS_TABLE, e as MysqlClient, d as _voltroCdcOffsetsTable, V as connectionFromConfig, z as makeMysqlSqlLayer, H as makeMysqlSqlLayerFromConfig, Z as makeMysqlStore, Pe as mariadbDialect, Ne as mysqlDialect, Me as mysqlReplicationAdapter, J as mysqlRetryFilter, K as startBinlogCdc };
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@voltro/sql-mysql",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.44.0",
|
|
4
4
|
"description": "MySQL/MariaDB dialect adapter for Voltro's cross-dialect DataStore (mariadb binlog CDC; mysql inline reactivity).",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"voltro",
|
|
@@ -35,8 +35,8 @@
|
|
|
35
35
|
"dependencies": {
|
|
36
36
|
"@effect/sql": "^0.52.0",
|
|
37
37
|
"@effect/sql-mysql2": "^0.53.0",
|
|
38
|
-
"@voltro/database": "0.
|
|
39
|
-
"@voltro/logger": "0.
|
|
38
|
+
"@voltro/database": "0.44.0",
|
|
39
|
+
"@voltro/logger": "0.44.0"
|
|
40
40
|
},
|
|
41
41
|
"optionalDependencies": {
|
|
42
42
|
"@vlasky/zongji": "^0.9.0"
|