@voltro/sql-postgres 0.43.1 → 0.44.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +65 -0
- package/dist/index.d.ts +43 -0
- package/dist/index.js +152 -100
- package/package.json +3 -3
package/CHANGELOG.md
CHANGED
|
@@ -39,6 +39,71 @@ _Changes staged for the next release accumulate here (rolled up from
|
|
|
39
39
|
|
|
40
40
|
---
|
|
41
41
|
|
|
42
|
+
## [0.44.0] — 2026-08-19
|
|
43
|
+
|
|
44
|
+
### ⚠ BREAKING
|
|
45
|
+
|
|
46
|
+
- **@voltro/data-transfer, @voltro/cli** — `runImport` returns an `ImportOutcome` instead of the bundle's `Manifest`, and `--mode replace` runs as ONE transaction by default.
|
|
47
|
+
|
|
48
|
+
**Why the return type changed.** The summary line counted the rows the BUNDLE carries, not the rows the run wrote. Those differ most exactly where it matters: a bundle directory carries its own resume ledger, so a copied directory imports nothing — correctly, with a warning naming the file to delete — and the run then printed `import complete … rows: 242950` over a target it had not touched. The warning was one line above, which is one line too far for anyone piping the output through `tail -1`. The outcome carries `rowsWritten`, `rowsSkipped` and `fullyResumed`, the CLI reports written-vs-carried, and a fully-skipped run says so on its LAST line.
|
|
49
|
+
|
|
50
|
+
Migration: `runImport(...)` now resolves to `{ manifest, tablesWritten, rowsWritten, tablesSkipped, rowsSkipped, fullyResumed }`. Read `.manifest` where you read the manifest before.
|
|
51
|
+
|
|
52
|
+
**Why replace is atomic now.** The all-or-nothing guarantee was written for the emptying step, and read — reasonably — as covering the run. A replace that died partway through the LOAD left the target emptied of its old rows and holding part of the new ones, measured on a live instance over sixteen minutes. There is no useful state for a replace to stop in, so it is a default rather than a flag you have to know about. It also closes the window that produced that failure: with the tables emptied and the load uncommitted, a concurrent writer in the application waits instead of inserting a row the bundle is about to insert too.
|
|
53
|
+
|
|
54
|
+
`--no-atomic` (CLI) / `atomic: false` (API) opts out. The trade is stated where it bites: every write to those tables waits for the load, and on postgres the bulk `COPY` loader cannot join a transaction it does not own — an atomic run now says that once rather than being quietly slower.
|
|
55
|
+
|
|
56
|
+
Two smaller things from the same report: `--mode replace` against a running instance warns about the empty-target window when it is NOT atomic, and the `--target api` upload reports progress per chunk plus a line explaining that the final request stays open for the whole import — sixteen minutes of silence is indistinguishable from a hang, and one operator killed a run that had finished.
|
|
57
|
+
|
|
58
|
+
**`voltro update` carries you across this** — codemod `0.44.0/01_import-outcome`.
|
|
59
|
+
|
|
60
|
+
### Fixed
|
|
61
|
+
|
|
62
|
+
- **@voltro/sql-mysql, @voltro/sql-postgres, @voltro/sql-sqlite, @voltro/sql-mssql, @voltro/data-transfer** — A write and its write-recorders now succeed or fail together on every SQL dialect, and the data importer's retry is idempotent.
|
|
63
|
+
|
|
64
|
+
A recorder (a versioning trail, an audit log) runs on the caller's connection and is ALLOWED to fail — a recorder that throws must take the write down with it, that is its contract. Outside a caller transaction the two were not one unit: the row's statement committed on its own, and the recorder's INSERT ran afterwards as a second autocommit statement. So a recorder that threw left a COMMITTED row behind a write that reported failure. Measured directly: `insert` throws, the row is in the table, and a second attempt at the same row is `ER_DUP_ENTRY` on PRIMARY.
|
|
65
|
+
|
|
66
|
+
Anything that retries a failed write then meets its own row. The data importer retries by design — it holds a row whose write failed and tries again once the remaining tables have streamed — so a `--mode replace` that had just emptied a table failed on a duplicate key IN that table. From outside, that is the impossible-looking thing: an import that emptied a table and then failed because something was already in it. Reproduced verbatim, including the table name and `[ER_DUP_ENTRY/1062]`, by putting the old retry back.
|
|
67
|
+
|
|
68
|
+
Both ends are closed, and they are independent on purpose:
|
|
69
|
+
|
|
70
|
+
- **The cause.** A table that HAS recorders writes inside a transaction now, so the row and the trail commit together or not at all — on mysql, mariadb, postgres, sqlite and mssql, verified by one suite that asks all five the same question. A table with no recorders — the default — takes the direct path unchanged; `recordsTable` is a Map-size check first, so it costs one comparison. - **The defence.** The importer's retry upserts instead of inserting in `replace` mode. That covers every OTHER way a write can land while reporting failure: a driver timeout on a write the server applied, a connection lost after the commit, a concurrent writer inserting the same key. It is sound precisely because the table was emptied by this same run — there is nothing in it that is not ours.
|
|
71
|
+
|
|
72
|
+
The test that pinned the divergence went red when the fix landed, exactly as its own note said it would, and is inverted with that note kept.
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## [0.43.2] — 2026-08-18
|
|
77
|
+
|
|
78
|
+
### Fixed
|
|
79
|
+
|
|
80
|
+
- **@voltro/sql-mysql, @voltro/database, @voltro/cli** — The mysql-family binlog reader no longer keeps a table excluded after the very migration that fixed it.
|
|
81
|
+
|
|
82
|
+
A UNIQUE on an unbounded text column is a MariaDB hash long-unique, whose hidden `DB_ROW_HASH_n` column the reader can never account for — so the table is held out of binlog capture and the exclusion is reported. `voltro dev` builds its store BEFORE it migrates (kv, cross-replica broadcast and the analytics mirror all need one), so on a boot whose own auto-migration bounds the column, that finding was drawn from a schema that stopped existing about a second later. The exclusion then outlived its cause for the life of the process, and the message — correct when written, and typically the only error line in the boot log — went on describing the pre-migration database.
|
|
83
|
+
|
|
84
|
+
Three changes:
|
|
85
|
+
|
|
86
|
+
- `DataStore.refreshChangeCaptureExclusions()` (optional; implemented by the mysql-family store) re-runs the probe and re-points the LIVE reader, in both directions — schema work that CREATES the condition now excludes the table immediately instead of after three failed writes. `voltro dev` calls it once, after all its schema work. `voltro serve` needs no equivalent: it builds its store after every schema step and never applies DDL itself. - The definitive message now states its own durability — that it is the schema as read at reader attach, and that applying the remedy does not by itself lift the exclusion. - On a boot that will re-check, the finding is reported as a provisional note rather than as a verdict, so a boot that fixes the condition leaves no error line about it. The note escalates to the full verdict on its own if the re-check never runs. - Two reader fixes the re-check depended on: applying a new exclusion set now WAITS for a reconnect already in flight (it is what applies the set, so resolving before it landed meant the caller's next write hit the old filter), and the reconnect loop no longer keeps resuming from an offset it has just jumped away from — that turned one purged offset into a reconnect every watchdog interval, forever, delivering nothing.
|
|
87
|
+
- **@voltro/data-transfer, @voltro/database, @voltro/sql-mysql, @voltro/sql-postgres, @voltro/sql-sqlite, @voltro/sql-mssql, @voltro/cli** — `voltro data import --mode replace` no longer leaves a target in neither state, and now works against schemas that have foreign keys.
|
|
88
|
+
|
|
89
|
+
The delete step ran table by table and stopped at the first refusal, so a run that could not finish left dozens of tables emptied and nothing loaded — and a second attempt destroyed more than the first, because it got further before hitting the same wall. The wall itself was not exotic: MySQL, MariaDB and SQL Server check a foreign key as each ROW is deleted, so a table that references ITSELF cannot be emptied by any ordering of the tables. `createdBy → actors` on the `actors` table is exactly that shape, and it is what an audit mixin on an actor table produces.
|
|
90
|
+
|
|
91
|
+
- `DataStore.emptyTables()` (per dialect) empties the whole set as one unit, in one transaction, with referential integrity suspended for the duration — `FOREIGN_KEY_CHECKS` on the mysql family, a multi-table `TRUNCATE` on postgres, `defer_foreign_keys` on sqlite, per-table `NOCHECK`/`WITH CHECK CHECK` on mssql. All-or-nothing on every engine, including under `--atomic`, where it runs on the transaction the import already holds. - A **pre-flight refusal**: if a table OUTSIDE the bundle holds rows referencing one inside it, the import refuses before deleting anything and names the tables, the columns and the row counts. Those rows cannot be restored from the bundle, so forcing it is not an option. An EMPTY outside table blocks nothing. - Table-level failures carry the driver's own reason and code, the way row-level failures already did. `truncate <table> failed: Failed to execute statement` fits every plausible cause equally; the classification that produced `foreign key <name>: … [ER_NO_REFERENCED_ROW_2/1452]` one level down now applies one level up. The word "truncate" is gone from the message too — the step issues DELETE, and naming a statement it does not run sends whoever reads it to reproduce the wrong thing. - A typed refusal reaching the `--target api` transport keeps its text: the admin import endpoint answers `409` with the reason instead of flattening it to `import failed`, on the one transport where the operator has no other way to see it.
|
|
92
|
+
|
|
93
|
+
The bulk empty emits no change events, where the per-table loop emitted one per row. An import through `--target api` now asks every live subscription to re-read once it lands — the coarse refresh the framework already uses after a broadcast gap — so neither the missing deletes nor a table the bundle carries EMPTY leaves a subscriber holding rows that are gone. Wired where the route is mounted, which is the one place both boot paths share.
|
|
94
|
+
|
|
95
|
+
On postgres the emptying is a `DELETE` per table, not a `TRUNCATE`, and the difference is not performance: postgres refuses `TRUNCATE` on a table with ANY incoming foreign key, rows or not, while the mysql family refuses a DELETE only when rows actually reference the doomed ones. A `TRUNCATE` version made an EMPTY table outside the bundle block a replace on postgres and not on mariadb — one import, refused on one engine and accepted on the other, over a table holding nothing.
|
|
96
|
+
|
|
97
|
+
`--target api` also no longer times out on a full bundle. Both api-target calls went through `fetch`, whose undici default gives up after 300 s — a bound on the caller's database size, on a call whose response arrives only when the import does. They wait as long as the instance needs now, stream the body instead of buffering the whole bundle, and take `--timeout <seconds>` when a deadline is wanted. If one is hit, the message says the instance is probably still importing — and gives different advice for `replace` than for the idempotent modes, since re-running the first while it is mid-flight would empty the target under it.
|
|
98
|
+
|
|
99
|
+
Two more, found by measuring rather than by reading:
|
|
100
|
+
|
|
101
|
+
- **`--atomic` on postgres could not import a bundle that needed the deferred-FK repair at all.** A failed statement aborts the transaction there, and that repair depends on a row whose parent has not loaded yet failing, being held, and being retried — so the first such row poisoned every write after it. Every tolerated write now runs inside a savepoint. Per-row savepoints measured 2.40x the time of none on 5 000 rows, so they are amortised: one savepoint per batch of 200, and a batch that fails rolls back whole and replays row by row. The mysql family and sqlite leave a transaction usable after a failed statement and pay nothing for any of this. - **The replace pre-flight asked the caller's snapshot.** Over `--target api` that is the app's DECLARED schema, which cannot show a table the app stopped declaring but the database still has — and rows in a table nobody declares are exactly the rows nobody is watching. `DataStore.incomingForeignKeys()` reads the live catalog per dialect; the snapshot remains the fallback for stores without one.
|
|
102
|
+
|
|
103
|
+
A bundle bigger than one chunk is now uploaded as a series of short requests, so a proxy body cap or an ingress read timeout has nothing large to choke on, and the switch is automatic — the packer's stream is buffered one chunk ahead, so a small bundle is sent exactly as before and nobody has to know in advance which table is the big one. The import still runs ONCE, at the end, over the whole bundle. Resume is byte-exact (`packBundle` is deterministic over a directory, which this package now asserts), guarded by a bundle key so a different bundle under the same upload id is refused rather than spliced into the partial one, and by a contiguity check so a mis-ordered append cannot produce an archive that only fails later during decode. `--chunk-size <mb>` overrides the 16 MiB default.
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
42
107
|
## [0.43.1] — 2026-08-18
|
|
43
108
|
|
|
44
109
|
### Fixed
|
package/dist/index.d.ts
CHANGED
|
@@ -379,6 +379,19 @@ export declare class PostgresDataStore implements DataStore {
|
|
|
379
379
|
* The gap being closed is specific: a plain write's statement commits ITSELF,
|
|
380
380
|
* so the transport fires while this is still awaiting the driver.
|
|
381
381
|
*/
|
|
382
|
+
/**
|
|
383
|
+
* A write and its write-recorders, as ONE unit — see `MysqlStore.
|
|
384
|
+
* writeWithRecorders` for the failure this closes.
|
|
385
|
+
*
|
|
386
|
+
* Same divergence on this dialect: the row's statement commits on its own and
|
|
387
|
+
* the recorder's INSERT is a second autocommit statement, so a recorder that
|
|
388
|
+
* throws leaves a committed row behind a write that reported failure. A caller
|
|
389
|
+
* that retries then meets its own row.
|
|
390
|
+
*
|
|
391
|
+
* Only tables that HAVE a recorder pay for the transaction; `recordsTable` is
|
|
392
|
+
* a Map-size check first, so the default costs one comparison.
|
|
393
|
+
*/
|
|
394
|
+
private writeWithRecorders;
|
|
382
395
|
private localWrite;
|
|
383
396
|
insert(table: string, row: Row): Promise<Row>;
|
|
384
397
|
insertMany(table: string, rows: ReadonlyArray<Row>): Promise<ReadonlyArray<Row>>;
|
|
@@ -467,6 +480,36 @@ export declare class PostgresDataStore implements DataStore {
|
|
|
467
480
|
* - The Exit settles through `settleTransactionExit`, so a typed error
|
|
468
481
|
* reaches the caller with its `_tag` rather than as a `FiberFailure`.
|
|
469
482
|
*/
|
|
483
|
+
/**
|
|
484
|
+
* Empty `tables` as one unit — see `DataStore.emptyTables`.
|
|
485
|
+
*
|
|
486
|
+
* Postgres needs no integrity switch, because it does not check a foreign key
|
|
487
|
+
* per deleted ROW: the constraint triggers are queued and run at the END of
|
|
488
|
+
* the statement. So a single `DELETE FROM t` clears a self-referencing table —
|
|
489
|
+
* the case that is impossible to order around on the mysql family — and the
|
|
490
|
+
* caller's children-first order covers the rest.
|
|
491
|
+
*
|
|
492
|
+
* `TRUNCATE` would be faster and was the first version of this, and it is
|
|
493
|
+
* WRONG here for a reason worth recording: postgres refuses `TRUNCATE` on a
|
|
494
|
+
* table with ANY incoming foreign key, whether or not that key has rows behind
|
|
495
|
+
* it. The mysql family refuses a DELETE only when rows actually reference the
|
|
496
|
+
* doomed ones. Keeping `TRUNCATE` would have made an EMPTY table outside the
|
|
497
|
+
* bundle block a replace on postgres and not on mariadb — the same import
|
|
498
|
+
* refused on one engine and accepted on the other, for a table holding
|
|
499
|
+
* nothing. Speed is not worth a divergence an operator cannot predict.
|
|
500
|
+
*/
|
|
501
|
+
emptyTables(tables: ReadonlyArray<string>): Promise<void>;
|
|
502
|
+
/** See `DataStore.incomingForeignKeys`. Read from `pg_catalog` rather than
|
|
503
|
+
* `information_schema`: the OID joins are the same ones the introspector was
|
|
504
|
+
* moved to, and the view's row-per-column-per-privilege shape is what makes
|
|
505
|
+
* the information_schema version slow on FK-dense schemas. */
|
|
506
|
+
incomingForeignKeys(tables: ReadonlyArray<string>): Promise<ReadonlyArray<{
|
|
507
|
+
from: string;
|
|
508
|
+
column: string;
|
|
509
|
+
to: string;
|
|
510
|
+
}>>;
|
|
511
|
+
/* Excluded from this release type: runStatementOnTransaction */
|
|
512
|
+
/* Excluded from this release type: emptyTablesOn */
|
|
470
513
|
transactional<T>(work: (tx: DataStore) => Promise<T>): Promise<T>;
|
|
471
514
|
onChange(listener: (event: ChangeEvent) => void): () => void;
|
|
472
515
|
/** Cross-instance reactivity seam — emit an externally-sourced event to
|
package/dist/index.js
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
import { PgClient as e, PgClient as t } from "@effect/sql-pg";
|
|
2
|
-
import { Config as n, Duration as r, Effect as i, Fiber as a, Layer as o, ManagedRuntime as s, Metric as c,
|
|
3
|
-
import
|
|
4
|
-
import { DEFAULT_ACQUIRE_TIMEOUT_MS as
|
|
5
|
-
import { EventEmitter as
|
|
6
|
-
import { createLogger as
|
|
2
|
+
import { Config as n, Duration as r, Effect as i, Fiber as a, Layer as o, ManagedRuntime as s, Metric as c, Option as l, Redacted as u, Schedule as d, Stream as f } from "effect";
|
|
3
|
+
import p from "pg";
|
|
4
|
+
import { DEFAULT_ACQUIRE_TIMEOUT_MS as m, EagerCardinalityError as h, attachEagerLoads as g, attributionFields as ee, attributionKey as te, beginLocalWrite as ne, bulkInsertLimitsFor as re, chunkRowsForInsert as ie, compileEagerJson as ae, compilePredicate as _, compileRawFragment as oe, compileSelect as se, encodeRowForSchema as v, endLocalWrite as ce, externalChangeEvent as le, hasEagerLoads as ue, isTableReactive as y, makeEagerFallbackReporter as de, observeDbOp as b, raiseChangeListenerCeiling as fe, recordsTable as x, registerPendingAttribution as S, requireTable as C, resolveEchoAttribution as w, runRetryingTransaction as pe, runStoreTransaction as T, runWriteRecorders as me, stampGeneratedId as E, stampGeneratedIds as he, withCapturedAttribution as D } from "@voltro/database";
|
|
5
|
+
import { EventEmitter as ge } from "node:events";
|
|
6
|
+
import { createLogger as _e } from "@voltro/logger";
|
|
7
7
|
import { SqlClient as O, TransactionConnection as k } from "@effect/sql/SqlClient";
|
|
8
8
|
//#region src/sqlLayer.ts
|
|
9
9
|
var A = (e) => {
|
|
10
|
-
let t = e.acquireTimeoutMs ??
|
|
10
|
+
let t = e.acquireTimeoutMs ?? m;
|
|
11
11
|
return t > 0 ? t : void 0;
|
|
12
12
|
}, j = (e) => e ? { rejectUnauthorized: !1 } : !1, M = /^[A-Za-z_][A-Za-z0-9_]*$/, N = (e) => {
|
|
13
13
|
let t = A(e);
|
|
@@ -15,7 +15,7 @@ var A = (e) => {
|
|
|
15
15
|
host: n.succeed(e.host),
|
|
16
16
|
port: n.succeed(e.port),
|
|
17
17
|
username: n.succeed(e.username),
|
|
18
|
-
password: n.succeed(
|
|
18
|
+
password: n.succeed(u.make(e.password)),
|
|
19
19
|
database: n.succeed(e.database),
|
|
20
20
|
...e.maxConnections === void 0 ? {} : { maxConnections: n.succeed(e.maxConnections) },
|
|
21
21
|
...e.ssl === void 0 ? {} : { ssl: n.succeed(j(e.ssl)) },
|
|
@@ -28,7 +28,7 @@ var A = (e) => {
|
|
|
28
28
|
if (!M.test(t.schema)) throw Error(`DB_SCHEMA '${t.schema}' is not a valid postgres identifier (expected ${M}).`);
|
|
29
29
|
n = { options: `-c search_path="${t.schema}"` };
|
|
30
30
|
}
|
|
31
|
-
let r = A(t), a = i.acquireRelease(i.sync(() => new
|
|
31
|
+
let r = A(t), a = i.acquireRelease(i.sync(() => new p.Pool({
|
|
32
32
|
host: t.host,
|
|
33
33
|
port: t.port,
|
|
34
34
|
user: t.username,
|
|
@@ -83,37 +83,37 @@ var A = (e) => {
|
|
|
83
83
|
...n,
|
|
84
84
|
...t
|
|
85
85
|
};
|
|
86
|
-
}, L = (e) => P(I(e)),
|
|
86
|
+
}, L = (e) => P(I(e)), ve = /* @__PURE__ */ new Set(["40001", "40P01"]), ye = (e) => {
|
|
87
87
|
let t = e;
|
|
88
88
|
for (let e = 0; e < 5 && typeof t == "object" && t; e++) {
|
|
89
89
|
let e = t.code;
|
|
90
90
|
if (typeof e == "string") return e;
|
|
91
91
|
t = t.cause;
|
|
92
92
|
}
|
|
93
|
-
},
|
|
94
|
-
let t =
|
|
95
|
-
return t !== void 0 &&
|
|
96
|
-
},
|
|
97
|
-
rehydrated: c.tagged(
|
|
98
|
-
tombstone: c.tagged(
|
|
99
|
-
unrecovered: c.tagged(
|
|
100
|
-
},
|
|
101
|
-
i.runSync(c.increment(
|
|
102
|
-
},
|
|
93
|
+
}, R = (e) => {
|
|
94
|
+
let t = ye(e);
|
|
95
|
+
return t !== void 0 && ve.has(t);
|
|
96
|
+
}, z = (e) => R(e) ? "retry" : "noRetry", B = c.counter("voltro_cdc_oversized_total", { description: "Change events whose row images exceeded the postgres NOTIFY payload cap, by what re-hydration recovered (rehydrated = row re-read · tombstone = delete, primary key only · unrecovered = content lost to taps)." }), be = {
|
|
97
|
+
rehydrated: c.tagged(B, "outcome", "rehydrated"),
|
|
98
|
+
tombstone: c.tagged(B, "outcome", "tombstone"),
|
|
99
|
+
unrecovered: c.tagged(B, "outcome", "unrecovered")
|
|
100
|
+
}, V = (e) => {
|
|
101
|
+
i.runSync(c.increment(be[e]));
|
|
102
|
+
}, xe = {
|
|
103
103
|
rehydrateTimeoutMs: 5e3,
|
|
104
104
|
rehydrateRetries: 2
|
|
105
|
-
},
|
|
105
|
+
}, H = (e) => {
|
|
106
106
|
let t = process.env[e];
|
|
107
107
|
if (t === void 0 || t.trim() === "") return;
|
|
108
108
|
let n = Number(t);
|
|
109
109
|
return Number.isFinite(n) && n >= 0 ? n : void 0;
|
|
110
|
-
},
|
|
111
|
-
let t =
|
|
110
|
+
}, U = (e) => {
|
|
111
|
+
let t = xe, n = e ?? {};
|
|
112
112
|
return {
|
|
113
|
-
rehydrateTimeoutMs: Math.max(1, n.rehydrateTimeoutMs ??
|
|
114
|
-
rehydrateRetries: Math.max(0, n.rehydrateRetries ??
|
|
113
|
+
rehydrateTimeoutMs: Math.max(1, n.rehydrateTimeoutMs ?? H("VOLTRO_CDC_REHYDRATE_TIMEOUT_MS") ?? t.rehydrateTimeoutMs),
|
|
114
|
+
rehydrateRetries: Math.max(0, n.rehydrateRetries ?? H("VOLTRO_CDC_REHYDRATE_RETRIES") ?? t.rehydrateRetries)
|
|
115
115
|
};
|
|
116
|
-
},
|
|
116
|
+
}, Se = (e) => i.suspend(() => {
|
|
117
117
|
let t = e.notification, n = t.id, a = {
|
|
118
118
|
table: t.table,
|
|
119
119
|
op: t.op
|
|
@@ -127,9 +127,9 @@ var A = (e) => {
|
|
|
127
127
|
outcome: "unrecovered",
|
|
128
128
|
reason: e
|
|
129
129
|
});
|
|
130
|
-
if (typeof n != "string" && typeof n != "number") return
|
|
130
|
+
if (typeof n != "string" && typeof n != "number") return V("unrecovered"), i.succeed(o("no-key"));
|
|
131
131
|
if (t.op === "delete") {
|
|
132
|
-
|
|
132
|
+
V("tombstone");
|
|
133
133
|
let e = {
|
|
134
134
|
event: {
|
|
135
135
|
...a,
|
|
@@ -141,12 +141,12 @@ var A = (e) => {
|
|
|
141
141
|
};
|
|
142
142
|
return i.succeed(e);
|
|
143
143
|
}
|
|
144
|
-
let s =
|
|
144
|
+
let s = d.exponential(r.millis(50), 2).pipe(d.intersect(d.recurs(e.tunables.rehydrateRetries)));
|
|
145
145
|
return e.fetchRow({
|
|
146
146
|
schema: t.schema,
|
|
147
147
|
table: t.table,
|
|
148
148
|
key: n
|
|
149
|
-
}).pipe(i.retry(s), i.timeout(r.millis(e.tunables.rehydrateTimeoutMs)), i.map((e) => e === null ? (
|
|
149
|
+
}).pipe(i.retry(s), i.timeout(r.millis(e.tunables.rehydrateTimeoutMs)), i.map((e) => e === null ? (V("unrecovered"), o("row-gone")) : (V("rehydrated"), {
|
|
150
150
|
event: {
|
|
151
151
|
...a,
|
|
152
152
|
old: null,
|
|
@@ -154,8 +154,8 @@ var A = (e) => {
|
|
|
154
154
|
oversized: "rehydrated"
|
|
155
155
|
},
|
|
156
156
|
outcome: "rehydrated"
|
|
157
|
-
})), i.catchAllCause(() => (
|
|
158
|
-
}),
|
|
157
|
+
})), i.catchAllCause(() => (V("unrecovered"), i.succeed(o("read-failed")))));
|
|
158
|
+
}), Ce = (e) => {
|
|
159
159
|
let t = /* @__PURE__ */ new Set();
|
|
160
160
|
return (n, r) => {
|
|
161
161
|
if (r.outcome === "unrecovered") {
|
|
@@ -173,25 +173,25 @@ var A = (e) => {
|
|
|
173
173
|
note: r.outcome === "tombstone" ? "a delete of an oversized row delivers the primary key only — the pre-image is unrecoverable" : "the re-read returns the row as it is NOW, not the image at commit"
|
|
174
174
|
}));
|
|
175
175
|
};
|
|
176
|
-
},
|
|
177
|
-
let t = e.tracerLayer ? o.mergeAll(e.sqlLayer, e.tracerLayer) : e.sqlLayer, n = s.make(t), r = await n.runPromise(O), i = e.changeStrategy ?? "inline", a = new
|
|
176
|
+
}, W = ["json"], G = _e({ scope: "voltro:postgres" }), K = async (e) => {
|
|
177
|
+
let t = e.tracerLayer ? o.mergeAll(e.sqlLayer, e.tracerLayer) : e.sqlLayer, n = s.make(t), r = await n.runPromise(O), i = e.changeStrategy ?? "inline", a = new we(r, n, i, e.cdcChannel ?? "framework_changes", U({
|
|
178
178
|
...e.cdcRehydrateTimeoutMs === void 0 ? {} : { rehydrateTimeoutMs: e.cdcRehydrateTimeoutMs },
|
|
179
179
|
...e.cdcRehydrateRetries === void 0 ? {} : { rehydrateRetries: e.cdcRehydrateRetries }
|
|
180
180
|
}));
|
|
181
181
|
return i === "cdc" && await a.startCdcConsumer(), a;
|
|
182
|
-
},
|
|
182
|
+
}, we = class {
|
|
183
183
|
sql;
|
|
184
184
|
runtime;
|
|
185
185
|
changeStrategy;
|
|
186
186
|
cdcChannel;
|
|
187
187
|
cdcRehydrate;
|
|
188
|
-
emitter = new
|
|
188
|
+
emitter = new ge();
|
|
189
189
|
cdcFiber = null;
|
|
190
190
|
inflightTxns = 0;
|
|
191
|
-
reportOversized =
|
|
192
|
-
reportEagerFallback =
|
|
193
|
-
constructor(e, t, n, r, i =
|
|
194
|
-
this.sql = e, this.runtime = t, this.changeStrategy = n, this.cdcChannel = r, this.cdcRehydrate = i,
|
|
191
|
+
reportOversized = Ce(G);
|
|
192
|
+
reportEagerFallback = de(G);
|
|
193
|
+
constructor(e, t, n, r, i = U()) {
|
|
194
|
+
this.sql = e, this.runtime = t, this.changeStrategy = n, this.cdcChannel = r, this.cdcRehydrate = i, fe(this.emitter);
|
|
195
195
|
}
|
|
196
196
|
txnSpec(e, t, n) {
|
|
197
197
|
return {
|
|
@@ -199,7 +199,7 @@ var A = (e) => {
|
|
|
199
199
|
dialect: "postgres",
|
|
200
200
|
withTransaction: (e) => this.sql.withTransaction(e),
|
|
201
201
|
runPromiseExit: (e) => this.runtime.runPromiseExit(e),
|
|
202
|
-
isRetryable:
|
|
202
|
+
isRetryable: R,
|
|
203
203
|
span: {
|
|
204
204
|
name: t,
|
|
205
205
|
attributes: {
|
|
@@ -210,7 +210,7 @@ var A = (e) => {
|
|
|
210
210
|
};
|
|
211
211
|
}
|
|
212
212
|
withNamespace(e) {
|
|
213
|
-
return e === null ? this : new
|
|
213
|
+
return e === null ? this : new Te(this, e);
|
|
214
214
|
}
|
|
215
215
|
async runInNamespace(e, t) {
|
|
216
216
|
let n = this.sql;
|
|
@@ -220,7 +220,7 @@ var A = (e) => {
|
|
|
220
220
|
...this.txnSpec("PostgresDataStore.runInNamespace", "store.namespace", "namespace.transaction"),
|
|
221
221
|
prepare: (t) => i.provideService(n`SET LOCAL search_path TO ${n(e)}`, k, t),
|
|
222
222
|
work: t,
|
|
223
|
-
makeView: (e, t) => new
|
|
223
|
+
makeView: (e, t) => new q(this, e, t)
|
|
224
224
|
});
|
|
225
225
|
} finally {
|
|
226
226
|
this.inflightTxns--;
|
|
@@ -228,12 +228,12 @@ var A = (e) => {
|
|
|
228
228
|
}
|
|
229
229
|
__postgresReplicationFriend = { runEffect: (e) => this.runtime.runPromise(e) };
|
|
230
230
|
async executeQuery(e, t, n = null) {
|
|
231
|
-
let r =
|
|
231
|
+
let r = se(e, this.sql, n), a = t ? i.provideService(r, k, t) : r;
|
|
232
232
|
return b("postgres", "select", () => this.runtime.runPromise(a));
|
|
233
233
|
}
|
|
234
234
|
async executeInsert(e, t, n, r, a) {
|
|
235
235
|
t = E(e, t);
|
|
236
|
-
let o = this.sql, s = o`INSERT INTO ${o(e)} ${o.insert(v(t, e,
|
|
236
|
+
let o = this.sql, s = o`INSERT INTO ${o(e)} ${o.insert(v(t, e, W))} RETURNING *`, c = n ? i.provideService(s, k, n) : s, l = (await this.runtime.runPromise(c))[0];
|
|
237
237
|
if (!l) throw Error(`PostgresDataStore.insert: no row returned for table '${e}'`);
|
|
238
238
|
return await this.routeEvent({
|
|
239
239
|
table: e,
|
|
@@ -243,7 +243,7 @@ var A = (e) => {
|
|
|
243
243
|
}, r, n, a), l;
|
|
244
244
|
}
|
|
245
245
|
async executeUpdate(e, t, n, r, a, o) {
|
|
246
|
-
let s = this.sql, c = s`UPDATE ${s(e)} SET ${s.update(v(n, e,
|
|
246
|
+
let s = this.sql, c = s`UPDATE ${s(e)} SET ${s.update(v(n, e, W))} WHERE ${s("id")} = ${t} RETURNING *`, l = r ? i.provideService(c, k, r) : c, u = (await this.runtime.runPromise(l))[0];
|
|
247
247
|
return u ? (await this.routeEvent({
|
|
248
248
|
table: e,
|
|
249
249
|
op: "update",
|
|
@@ -261,7 +261,7 @@ var A = (e) => {
|
|
|
261
261
|
if (r) return l(r);
|
|
262
262
|
this.inflightTxns++;
|
|
263
263
|
try {
|
|
264
|
-
return await
|
|
264
|
+
return await pe({
|
|
265
265
|
...this.txnSpec("PostgresDataStore.upsert", "store.upsert", "upsert.transaction"),
|
|
266
266
|
body: (e) => l(e)
|
|
267
267
|
});
|
|
@@ -269,7 +269,7 @@ var A = (e) => {
|
|
|
269
269
|
this.inflightTxns--;
|
|
270
270
|
}
|
|
271
271
|
}
|
|
272
|
-
let c = n.conflictColumns.map((e) => s`${s(e)}`), l = Object.keys(t).filter((e) => t[e] !== void 0), u = n.update === void 0 ? l.filter((e) => e !== "id" && !n.conflictColumns.includes(e)) : n.update, d = u.length > 0 ? s.csv(u.map((e) => s`${s(e)} = EXCLUDED.${s(e)}`)) : s`${s(n.conflictColumns[0])} = EXCLUDED.${s(n.conflictColumns[0])}`, f = s`INSERT INTO ${s(e)} ${s.insert(v(t, e,
|
|
272
|
+
let c = n.conflictColumns.map((e) => s`${s(e)}`), l = Object.keys(t).filter((e) => t[e] !== void 0), u = n.update === void 0 ? l.filter((e) => e !== "id" && !n.conflictColumns.includes(e)) : n.update, d = u.length > 0 ? s.csv(u.map((e) => s`${s(e)} = EXCLUDED.${s(e)}`)) : s`${s(n.conflictColumns[0])} = EXCLUDED.${s(n.conflictColumns[0])}`, f = s`INSERT INTO ${s(e)} ${s.insert(v(t, e, W))} ON CONFLICT (${s.csv(c)}) DO UPDATE SET ${d} RETURNING *`, p = r ? i.provideService(f, k, r) : f, m = (await this.runtime.runPromise(p))[0];
|
|
273
273
|
if (!m) throw Error(`PostgresDataStore.upsert: no row returned for table '${e}'`);
|
|
274
274
|
{
|
|
275
275
|
let n = t.id !== void 0 && t.id === m.id ? "insert" : "update";
|
|
@@ -284,7 +284,7 @@ var A = (e) => {
|
|
|
284
284
|
}
|
|
285
285
|
async executeInsertIgnore(e, t, n, r, a, o) {
|
|
286
286
|
t = E(e, t);
|
|
287
|
-
let s = this.sql, c = n.conflictColumns.map((e) => s`${s(e)}`), l = s`INSERT INTO ${s(e)} ${s.insert(v(t, e,
|
|
287
|
+
let s = this.sql, c = n.conflictColumns.map((e) => s`${s(e)}`), l = s`INSERT INTO ${s(e)} ${s.insert(v(t, e, W))} ON CONFLICT (${s.csv(c)}) DO NOTHING RETURNING *`, u = r ? i.provideService(l, k, r) : l, d = (await this.runtime.runPromise(u))[0];
|
|
288
288
|
if (d) return await this.routeEvent({
|
|
289
289
|
table: e,
|
|
290
290
|
op: "insert",
|
|
@@ -296,8 +296,8 @@ var A = (e) => {
|
|
|
296
296
|
return h[0];
|
|
297
297
|
}
|
|
298
298
|
async executeInsertMany(e, t, n, r, a) {
|
|
299
|
-
if (t =
|
|
300
|
-
let o = this.sql, s = t.map((t) => v(t, e,
|
|
299
|
+
if (t = he(e, t), t.length === 0) return [];
|
|
300
|
+
let o = this.sql, s = t.map((t) => v(t, e, W)), c = ie(s, re("postgres")), l = (t) => o`INSERT INTO ${o(e)} ${o.insert(t)} RETURNING *`, u;
|
|
301
301
|
if (c.length === 1) {
|
|
302
302
|
let e = l(c[0]), t = n ? i.provideService(e, k, n) : e;
|
|
303
303
|
u = await this.runtime.runPromise(t);
|
|
@@ -333,7 +333,7 @@ var A = (e) => {
|
|
|
333
333
|
}, r, n, a), !0) : !1;
|
|
334
334
|
}
|
|
335
335
|
async appendInTxn(e, t, n) {
|
|
336
|
-
let r = this.sql, a = r`INSERT INTO ${r(e)} ${r.insert(v(t, e,
|
|
336
|
+
let r = this.sql, a = r`INSERT INTO ${r(e)} ${r.insert(v(t, e, W))}`;
|
|
337
337
|
await this.runtime.runPromise(n ? i.provideService(a, k, n) : a);
|
|
338
338
|
}
|
|
339
339
|
async maxInTxn(e, t, n, r) {
|
|
@@ -342,9 +342,9 @@ var A = (e) => {
|
|
|
342
342
|
}
|
|
343
343
|
async routeEvent(e, t, n = null, r) {
|
|
344
344
|
if (e = {
|
|
345
|
-
...
|
|
345
|
+
...ee(r),
|
|
346
346
|
...e
|
|
347
|
-
},
|
|
347
|
+
}, x(e.table) && await me({
|
|
348
348
|
append: (e, t) => this.appendInTxn(e, t, n),
|
|
349
349
|
maxOf: (e, t, r) => this.maxInTxn(e, t, r, n)
|
|
350
350
|
}, {
|
|
@@ -357,7 +357,7 @@ var A = (e) => {
|
|
|
357
357
|
}), y(e.table)) {
|
|
358
358
|
if (this.changeStrategy === "cdc") {
|
|
359
359
|
let t = (e.op === "delete" ? e.old : e.new)?.id;
|
|
360
|
-
t != null &&
|
|
360
|
+
t != null && S(te(e.table, e.op, t), {
|
|
361
361
|
...e.traceId === void 0 ? {} : { traceId: e.traceId },
|
|
362
362
|
...e.subjectId === void 0 ? {} : { subjectId: e.subjectId }
|
|
363
363
|
});
|
|
@@ -370,17 +370,17 @@ var A = (e) => {
|
|
|
370
370
|
return this.runWithEager(e, null);
|
|
371
371
|
}
|
|
372
372
|
raw(e, t) {
|
|
373
|
-
let n =
|
|
373
|
+
let n = oe(e, this.sql);
|
|
374
374
|
return b("postgres", "raw", () => this.runtime.runPromise(n));
|
|
375
375
|
}
|
|
376
376
|
async runWithEager(e, t, n = null) {
|
|
377
|
-
if (!
|
|
378
|
-
let r = n === null ?
|
|
377
|
+
if (!ue(e)) return this.executeQuery(e, t, n);
|
|
378
|
+
let r = n === null ? ae(e, this.sql, "postgres") : null;
|
|
379
379
|
if (r !== null) try {
|
|
380
380
|
let e = t ? i.provideService(r.fragment, k, t) : r.fragment, n = await b("postgres", "select", () => this.runtime.runPromise(e));
|
|
381
381
|
return r.decode(n);
|
|
382
382
|
} catch (t) {
|
|
383
|
-
if (t instanceof
|
|
383
|
+
if (t instanceof h) throw t;
|
|
384
384
|
this.reportEagerFallback({
|
|
385
385
|
dialect: "postgres",
|
|
386
386
|
table: e.table,
|
|
@@ -394,7 +394,7 @@ var A = (e) => {
|
|
|
394
394
|
reason: "not-compilable"
|
|
395
395
|
});
|
|
396
396
|
let a = await this.executeQuery(e, t, n);
|
|
397
|
-
return
|
|
397
|
+
return g(a, e.eager, e.sourceTable ?? C(e.table), (e) => this.executeQuery(e, t, n));
|
|
398
398
|
}
|
|
399
399
|
getInternalRunWithEager() {
|
|
400
400
|
return (e, t) => this.runWithEager(e, t);
|
|
@@ -402,40 +402,49 @@ var A = (e) => {
|
|
|
402
402
|
async queryInNamespace(e, t) {
|
|
403
403
|
return this.runWithEager(t, null, e);
|
|
404
404
|
}
|
|
405
|
+
async writeWithRecorders(e, t) {
|
|
406
|
+
let n = [], r = this.sql, a = await this.runtime.runPromise(r.withTransaction(i.flatMap(i.serviceOption(k), (r) => l.isNone(r) ? i.fail(/* @__PURE__ */ Error("PostgresDataStore.write: TransactionConnection missing.")) : i.tryPromise({
|
|
407
|
+
try: () => e(t, r.value, n),
|
|
408
|
+
catch: (e) => e
|
|
409
|
+
}))));
|
|
410
|
+
for (let e of n) this.emitChange(e);
|
|
411
|
+
return a;
|
|
412
|
+
}
|
|
405
413
|
async localWrite(e, t, n) {
|
|
414
|
+
let r = (e) => x(t) ? this.writeWithRecorders(n, e) : n(e, null, null);
|
|
406
415
|
return b("postgres", e, async () => {
|
|
407
|
-
if (this.changeStrategy !== "cdc") return D(
|
|
408
|
-
|
|
416
|
+
if (this.changeStrategy !== "cdc") return D(r);
|
|
417
|
+
ne(t);
|
|
409
418
|
try {
|
|
410
|
-
return await D(
|
|
419
|
+
return await D(r);
|
|
411
420
|
} finally {
|
|
412
|
-
|
|
421
|
+
ce(t);
|
|
413
422
|
}
|
|
414
423
|
});
|
|
415
424
|
}
|
|
416
425
|
insert(e, t) {
|
|
417
|
-
return this.localWrite("insert", e, (n) => this.executeInsert(e, t,
|
|
426
|
+
return this.localWrite("insert", e, (n, r, i) => this.executeInsert(e, t, r, i, n));
|
|
418
427
|
}
|
|
419
428
|
insertMany(e, t) {
|
|
420
|
-
return this.localWrite("insert", e, (n) => this.executeInsertMany(e, t,
|
|
429
|
+
return this.localWrite("insert", e, (n, r, i) => this.executeInsertMany(e, t, r, i, n));
|
|
421
430
|
}
|
|
422
431
|
patchJson(e, t, n, r) {
|
|
423
|
-
return this.localWrite("update", e, (i) => this.executePatchJson(e, t, n, r,
|
|
432
|
+
return this.localWrite("update", e, (i, a, o) => this.executePatchJson(e, t, n, r, a, o, i));
|
|
424
433
|
}
|
|
425
434
|
update(e, t, n) {
|
|
426
|
-
return this.localWrite("update", e, (r) => this.executeUpdate(e, t, n,
|
|
435
|
+
return this.localWrite("update", e, (r, i, a) => this.executeUpdate(e, t, n, i, a, r));
|
|
427
436
|
}
|
|
428
437
|
delete(e, t) {
|
|
429
|
-
return this.localWrite("delete", e, (n) => this.executeDelete(e, t,
|
|
438
|
+
return this.localWrite("delete", e, (n, r, i) => this.executeDelete(e, t, r, i, n));
|
|
430
439
|
}
|
|
431
440
|
async updateMany(e, t, n) {
|
|
432
|
-
return this.localWrite("update", e, (r) => this.executeUpdateMany(e, t, n,
|
|
441
|
+
return this.localWrite("update", e, (r, i, a) => this.executeUpdateMany(e, t, n, i, a, r));
|
|
433
442
|
}
|
|
434
443
|
async deleteMany(e, t) {
|
|
435
|
-
return this.localWrite("delete", e, (n) => this.executeDeleteMany(e, t,
|
|
444
|
+
return this.localWrite("delete", e, (n, r, i) => this.executeDeleteMany(e, t, r, i, n));
|
|
436
445
|
}
|
|
437
446
|
async executeUpdateMany(e, t, n, r, a, o) {
|
|
438
|
-
let s = this.sql, c = _(n.where, s), l = s`UPDATE ${s(e)} SET ${s.update(v(t, e,
|
|
447
|
+
let s = this.sql, c = _(n.where, s), l = s`UPDATE ${s(e)} SET ${s.update(v(t, e, W))} WHERE ${c} RETURNING *`, u = r ? i.provideService(l, k, r) : l, d = await this.runtime.runPromise(u);
|
|
439
448
|
for (let t of d) await this.routeEvent({
|
|
440
449
|
table: e,
|
|
441
450
|
op: "update",
|
|
@@ -455,10 +464,10 @@ var A = (e) => {
|
|
|
455
464
|
return u.length;
|
|
456
465
|
}
|
|
457
466
|
upsert(e, t, n) {
|
|
458
|
-
return this.localWrite("upsert", e, (r) => this.executeUpsert(e, t, n,
|
|
467
|
+
return this.localWrite("upsert", e, (r, i, a) => this.executeUpsert(e, t, n, i, a, r));
|
|
459
468
|
}
|
|
460
469
|
insertIgnore(e, t, n) {
|
|
461
|
-
return this.localWrite("upsert", e, (r) => this.executeInsertIgnore(e, t, n,
|
|
470
|
+
return this.localWrite("upsert", e, (r, i, a) => this.executeInsertIgnore(e, t, n, i, a, r));
|
|
462
471
|
}
|
|
463
472
|
emitChange(e) {
|
|
464
473
|
this.emitter.emit("change", e);
|
|
@@ -493,13 +502,42 @@ var A = (e) => {
|
|
|
493
502
|
getInternalExecuteInsertIgnore() {
|
|
494
503
|
return this.executeInsertIgnore.bind(this);
|
|
495
504
|
}
|
|
505
|
+
async emptyTables(e) {
|
|
506
|
+
await this.emptyTablesOn(e, null);
|
|
507
|
+
}
|
|
508
|
+
async incomingForeignKeys(e) {
|
|
509
|
+
if (e.length === 0) return [];
|
|
510
|
+
let t = this.sql;
|
|
511
|
+
return (await this.runtime.runPromise(t`
|
|
512
|
+
SELECT child.relname AS from_table, att.attname AS column_name, parent.relname AS to_table
|
|
513
|
+
FROM pg_constraint c
|
|
514
|
+
JOIN pg_class child ON child.oid = c.conrelid
|
|
515
|
+
JOIN pg_class parent ON parent.oid = c.confrelid
|
|
516
|
+
JOIN LATERAL unnest(c.conkey) AS k(attnum) ON true
|
|
517
|
+
JOIN pg_attribute att ON att.attrelid = c.conrelid AND att.attnum = k.attnum
|
|
518
|
+
WHERE c.contype = 'f'
|
|
519
|
+
AND parent.relname IN ${t.in(e)}
|
|
520
|
+
AND child.relname NOT IN ${t.in(e)}`)).map((e) => ({
|
|
521
|
+
from: e.from_table,
|
|
522
|
+
column: e.column_name,
|
|
523
|
+
to: e.to_table
|
|
524
|
+
}));
|
|
525
|
+
}
|
|
526
|
+
runStatementOnTransaction(e, t) {
|
|
527
|
+
return this.runtime.runPromise(i.provideService(this.sql.unsafe(e), k, t));
|
|
528
|
+
}
|
|
529
|
+
async emptyTablesOn(e, t) {
|
|
530
|
+
if (e.length === 0) return;
|
|
531
|
+
let n = this.sql, r = i.forEach(e, (e) => n`DELETE FROM ${n(e)}`, { discard: !0 }), a = t === null ? n.withTransaction(r) : i.provideService(r, k, t);
|
|
532
|
+
await this.runtime.runPromise(a);
|
|
533
|
+
}
|
|
496
534
|
async transactional(e) {
|
|
497
535
|
this.inflightTxns++;
|
|
498
536
|
try {
|
|
499
537
|
return await T({
|
|
500
538
|
...this.txnSpec("PostgresDataStore.transactional", "store.transactional", "transaction"),
|
|
501
539
|
work: e,
|
|
502
|
-
makeView: (e, t) => new
|
|
540
|
+
makeView: (e, t) => new q(this, e, t)
|
|
503
541
|
});
|
|
504
542
|
} finally {
|
|
505
543
|
this.inflightTxns--;
|
|
@@ -516,8 +554,8 @@ var A = (e) => {
|
|
|
516
554
|
injectExternalChange(e) {
|
|
517
555
|
if (!y(e.table)) return;
|
|
518
556
|
let t = (e.op === "delete" ? e.old : e.new)?.id;
|
|
519
|
-
|
|
520
|
-
this.emitter.emit("change",
|
|
557
|
+
w(e.table, e.op, t, (t) => {
|
|
558
|
+
this.emitter.emit("change", le(e, t));
|
|
521
559
|
});
|
|
522
560
|
}
|
|
523
561
|
run(e) {
|
|
@@ -527,7 +565,7 @@ var A = (e) => {
|
|
|
527
565
|
if (this.inflightTxns > 0) {
|
|
528
566
|
let t = Date.now() + e;
|
|
529
567
|
for (; this.inflightTxns > 0 && Date.now() < t;) await new Promise((e) => setTimeout(e, 25));
|
|
530
|
-
this.inflightTxns > 0 &&
|
|
568
|
+
this.inflightTxns > 0 && G.warn("close: grace period expired with in-flight transactions — forcing dispose", {
|
|
531
569
|
gracePeriodMs: e,
|
|
532
570
|
inflight: this.inflightTxns
|
|
533
571
|
});
|
|
@@ -552,7 +590,7 @@ var A = (e) => {
|
|
|
552
590
|
try {
|
|
553
591
|
t = JSON.parse(e);
|
|
554
592
|
} catch (e) {
|
|
555
|
-
return
|
|
593
|
+
return G.warn("cdc: bad payload", { channel: this.cdcChannel }, e), i.void;
|
|
556
594
|
}
|
|
557
595
|
if (t.oversized !== !0) {
|
|
558
596
|
let e = {
|
|
@@ -565,7 +603,7 @@ var A = (e) => {
|
|
|
565
603
|
this.injectExternalChange(e);
|
|
566
604
|
});
|
|
567
605
|
}
|
|
568
|
-
return y(t.table) ?
|
|
606
|
+
return y(t.table) ? Se({
|
|
569
607
|
notification: t,
|
|
570
608
|
fetchRow: ({ schema: e, table: t, key: n }) => this.readRowAsCdcJson(e, t, n),
|
|
571
609
|
tunables: this.cdcRehydrate
|
|
@@ -573,9 +611,9 @@ var A = (e) => {
|
|
|
573
611
|
this.reportOversized(t, e), this.injectExternalChange(e.event);
|
|
574
612
|
})) : i.void;
|
|
575
613
|
});
|
|
576
|
-
this.cdcFiber = this.runtime.runFork(e.pipe(
|
|
614
|
+
this.cdcFiber = this.runtime.runFork(e.pipe(f.runForEach(n)));
|
|
577
615
|
}
|
|
578
|
-
},
|
|
616
|
+
}, q = class {
|
|
579
617
|
parent;
|
|
580
618
|
txn;
|
|
581
619
|
attr;
|
|
@@ -587,6 +625,20 @@ var A = (e) => {
|
|
|
587
625
|
query(e) {
|
|
588
626
|
return this.parent.getInternalRunWithEager()(e, this.txn);
|
|
589
627
|
}
|
|
628
|
+
emptyTables(e) {
|
|
629
|
+
return this.parent.emptyTablesOn(e, this.txn);
|
|
630
|
+
}
|
|
631
|
+
async savepoint(e) {
|
|
632
|
+
let t = `voltro_sp_${++this.savepointCounter}`, n = (e) => this.parent.runStatementOnTransaction(e, this.txn);
|
|
633
|
+
await n(`SAVEPOINT ${t}`);
|
|
634
|
+
try {
|
|
635
|
+
let r = await e();
|
|
636
|
+
return await n(`RELEASE SAVEPOINT ${t}`), r;
|
|
637
|
+
} catch (e) {
|
|
638
|
+
throw await n(`ROLLBACK TO SAVEPOINT ${t}`).catch(() => void 0), await n(`RELEASE SAVEPOINT ${t}`).catch(() => void 0), e;
|
|
639
|
+
}
|
|
640
|
+
}
|
|
641
|
+
savepointCounter = 0;
|
|
590
642
|
insert(e, t) {
|
|
591
643
|
return this.parent.getInternalExecuteInsert()(e, t, this.txn, this.events, this.attr);
|
|
592
644
|
}
|
|
@@ -630,7 +682,7 @@ var A = (e) => {
|
|
|
630
682
|
this.events.length = 0;
|
|
631
683
|
}
|
|
632
684
|
}
|
|
633
|
-
},
|
|
685
|
+
}, Te = class {
|
|
634
686
|
parent;
|
|
635
687
|
namespace;
|
|
636
688
|
constructor(e, t) {
|
|
@@ -684,14 +736,14 @@ var A = (e) => {
|
|
|
684
736
|
return r === void 0 ? Promise.reject(/* @__PURE__ */ Error("PostgresNamespaceView.raw: underlying store has no raw()")) : r(e, t);
|
|
685
737
|
});
|
|
686
738
|
}
|
|
687
|
-
},
|
|
739
|
+
}, J = (e) => e.__postgresReplicationFriend ?? null, Ee = (e, t) => {
|
|
688
740
|
let [n, r] = e.split("/"), [i, a] = t.split("/");
|
|
689
741
|
if (!n || !r || !i || !a) throw Error(`postgres LSN compare: invalid format (${e} vs ${t})`);
|
|
690
742
|
let o = parseInt(n, 16), s = parseInt(r, 16), c = parseInt(i, 16);
|
|
691
743
|
return o === c ? s - parseInt(a, 16) : o - c;
|
|
692
|
-
},
|
|
744
|
+
}, Y = () => ({
|
|
693
745
|
async capturePrimaryPosition(e) {
|
|
694
|
-
let t =
|
|
746
|
+
let t = J(e);
|
|
695
747
|
if (t === null) throw Error("postgresReplicationAdapter: primary is not a PostgresDataStore (missing __postgresReplicationFriend). Pass the postgres store directly.");
|
|
696
748
|
return t.runEffect(i.gen(function* () {
|
|
697
749
|
let e = (yield* (yield* O)`SELECT pg_current_wal_lsn()::text AS lsn`)[0]?.lsn;
|
|
@@ -699,7 +751,7 @@ var A = (e) => {
|
|
|
699
751
|
}));
|
|
700
752
|
},
|
|
701
753
|
async probeReplicaPosition(e) {
|
|
702
|
-
let t =
|
|
754
|
+
let t = J(e);
|
|
703
755
|
if (t === null) throw Error("postgresReplicationAdapter: replica is not a PostgresDataStore.");
|
|
704
756
|
return t.runEffect(i.gen(function* () {
|
|
705
757
|
let e = (yield* (yield* O)`
|
|
@@ -709,16 +761,16 @@ var A = (e) => {
|
|
|
709
761
|
}));
|
|
710
762
|
},
|
|
711
763
|
compare(e, t) {
|
|
712
|
-
return
|
|
764
|
+
return Ee(t, e) >= 0 ? "caught-up" : "behind";
|
|
713
765
|
}
|
|
714
|
-
}), X = (e) => `"${e.replace(/"/g, "\"\"")}"`,
|
|
766
|
+
}), X = (e) => `"${e.replace(/"/g, "\"\"")}"`, De = (e) => e.replace(/\\/g, "\\\\").replace(/\t/g, "\\t").replace(/\n/g, "\\n").replace(/\r/g, "\\r").replace(/\v/g, "\\v").replace(/\f/g, "\\f").replace(/[\b]/g, "\\b"), Oe = (e) => {
|
|
715
767
|
let t = "";
|
|
716
768
|
for (let n of e) t += n.toString(16).padStart(2, "0");
|
|
717
769
|
return t;
|
|
718
|
-
}, Z = (e) => `{${e.map((e) => e == null ? "NULL" : Array.isArray(e) ? Z(e) : typeof e == "number" || typeof e == "bigint" ? String(e) : typeof e == "boolean" ? e ? "t" : "f" : `"${(e instanceof Date ? e.toISOString() : String(e)).replace(/\\/g, "\\\\").replace(/"/g, "\\\"")}"`).join(",")}}`, Q = (e, t) => e == null ? null : typeof e == "string" ? e : typeof e == "number" || typeof e == "bigint" ? String(e) : typeof e == "boolean" ? e ? "t" : "f" : e instanceof Date ? e.toISOString() : e instanceof Uint8Array ? `\\x${
|
|
770
|
+
}, Z = (e) => `{${e.map((e) => e == null ? "NULL" : Array.isArray(e) ? Z(e) : typeof e == "number" || typeof e == "bigint" ? String(e) : typeof e == "boolean" ? e ? "t" : "f" : `"${(e instanceof Date ? e.toISOString() : String(e)).replace(/\\/g, "\\\\").replace(/"/g, "\\\"")}"`).join(",")}}`, Q = (e, t) => e == null ? null : typeof e == "string" ? e : typeof e == "number" || typeof e == "bigint" ? String(e) : typeof e == "boolean" ? e ? "t" : "f" : e instanceof Date ? e.toISOString() : e instanceof Uint8Array ? `\\x${Oe(e)}` : Array.isArray(e) && t === "array" ? Z(e) : JSON.stringify(e), $ = (e, t, n = {}) => t.map((t) => {
|
|
719
771
|
let r = Q(e[t], n[t]);
|
|
720
|
-
return r === null ? "\\N" :
|
|
721
|
-
}).join(" "),
|
|
772
|
+
return r === null ? "\\N" : De(r);
|
|
773
|
+
}).join(" "), ke = class {
|
|
722
774
|
text;
|
|
723
775
|
chunks;
|
|
724
776
|
done;
|
|
@@ -749,7 +801,7 @@ var A = (e) => {
|
|
|
749
801
|
handleReadyForQuery() {
|
|
750
802
|
this.settle(void 0);
|
|
751
803
|
}
|
|
752
|
-
},
|
|
804
|
+
}, Ae = (e) => ({
|
|
753
805
|
...e.url ? { connectionString: e.url } : {
|
|
754
806
|
host: e.host ?? "localhost",
|
|
755
807
|
port: e.port ?? 5432,
|
|
@@ -758,9 +810,9 @@ var A = (e) => {
|
|
|
758
810
|
database: e.database ?? "postgres"
|
|
759
811
|
},
|
|
760
812
|
...e.ssl === !0 ? { ssl: { rejectUnauthorized: !1 } } : {}
|
|
761
|
-
}),
|
|
813
|
+
}), je = 65536, Me = (e) => {
|
|
762
814
|
let t, n, r = () => t ? Promise.resolve(t) : (n ??= (async () => {
|
|
763
|
-
let n = new
|
|
815
|
+
let n = new p.Client(Ae(e));
|
|
764
816
|
return await n.connect(), t = n, n;
|
|
765
817
|
})(), n);
|
|
766
818
|
return {
|
|
@@ -768,9 +820,9 @@ var A = (e) => {
|
|
|
768
820
|
copyInto: async ({ table: e, columns: t, columnTypes: n, rows: i }) => {
|
|
769
821
|
if (i.length === 0) return 0;
|
|
770
822
|
let a = await r(), o = `COPY ${X(e)} (${t.map(X).join(", ")}) FROM STDIN`, s = [], c = "";
|
|
771
|
-
for (let e of i) c += $(e, t, n ?? {}), c += "\n", c.length >=
|
|
823
|
+
for (let e of i) c += $(e, t, n ?? {}), c += "\n", c.length >= je && (s.push(Buffer.from(c, "utf8")), c = "");
|
|
772
824
|
return c.length > 0 && s.push(Buffer.from(c, "utf8")), await new Promise((e, t) => {
|
|
773
|
-
let n = new
|
|
825
|
+
let n = new ke(o, s, (n, r) => {
|
|
774
826
|
n ? t(n) : e(r);
|
|
775
827
|
});
|
|
776
828
|
a.query(n);
|
|
@@ -781,12 +833,12 @@ var A = (e) => {
|
|
|
781
833
|
t = void 0, n = void 0, e && await e.end().catch(() => void 0);
|
|
782
834
|
}
|
|
783
835
|
};
|
|
784
|
-
},
|
|
836
|
+
}, Ne = {
|
|
785
837
|
id: "postgres",
|
|
786
838
|
makeSqlLayer: (e) => L(e),
|
|
787
|
-
makeStore: (e) =>
|
|
839
|
+
makeStore: (e) => K(e),
|
|
788
840
|
compileContains: (e, t, n) => n`${e} ILIKE ${`%${t.replace(/[\\%_]/g, (e) => `\\${e}`)}%`} ESCAPE '\\'`,
|
|
789
|
-
retryFilter:
|
|
841
|
+
retryFilter: z
|
|
790
842
|
};
|
|
791
843
|
//#endregion
|
|
792
|
-
export { e as PgClient, I as connectionFromConfig, $ as encodeCopyRow, Q as encodeCopyValue,
|
|
844
|
+
export { e as PgClient, I as connectionFromConfig, $ as encodeCopyRow, Q as encodeCopyValue, Me as makePgCopySession, K as makePostgresDataStore, P as makePostgresSqlLayer, L as makePostgresSqlLayerFromConfig, Ne as postgresDialect, Y as postgresReplicationAdapter, z as postgresRetryFilter };
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@voltro/sql-postgres",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.44.0",
|
|
4
4
|
"description": "PostgreSQL dialect adapter for Voltro's cross-dialect DataStore (LISTEN/NOTIFY reactivity, logical-replication CDC).",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"voltro",
|
|
@@ -35,8 +35,8 @@
|
|
|
35
35
|
"dependencies": {
|
|
36
36
|
"@effect/sql": "^0.52.0",
|
|
37
37
|
"@effect/sql-pg": "^0.53.0",
|
|
38
|
-
"@voltro/database": "0.
|
|
39
|
-
"@voltro/logger": "0.
|
|
38
|
+
"@voltro/database": "0.44.0",
|
|
39
|
+
"@voltro/logger": "0.44.0",
|
|
40
40
|
"pg": "^8.23.0"
|
|
41
41
|
},
|
|
42
42
|
"peerDependencies": {
|