@vlasky/zongji 0.8.0 → 0.9.0

package/CHANGELOG.md

@@ -2,6 +2,13 @@
 
 All notable changes to @vlasky/zongji since forking from nevill/zongji.
 
+## [0.9.0] - 2026-07-06
+
+- Export the `MariadbGtidPosition` class (`import { MariadbGtidPosition } from '@vlasky/zongji'`), the MariaDB counterpart of `GtidSet` and the model behind `zongji.gtidSet` on MariaDB servers: parse a persisted position (a `zongji.gtidSet` checkpoint or `@@gtid_binlog_pos`), build one from a `mariadbgtidlist` event with `fromGtidList()`, extend it with `add()`, and test transaction coverage with `covers(event.gtid, { inclusive })` instead of hand-rolling sequence-number comparators (which silently round above `Number.MAX_SAFE_INTEGER` when done with Number). The inclusive form (default) answers "was this transaction's commit at or before the position?" for snapshot barriers; redelivery watermarks must pass `{ inclusive: false }`, because all events of a transaction share one GTID and an inclusive check would drop every replayed event of the watermark transaction after the first. A server id differing from the position's entry for that domain reports not covered even for a lower sequence number (post-failover sequence numbers can regress under the new server id; for at-least-once consumers an idempotent redelivery is recoverable where a skip is not), as does a domain the position does not list.
+- `GtidSet` and `MariadbGtidPosition` now use BigInt internally, making them exact over MySQL's full GNO range (1 to 2^63-1) and MariaDB's full u64 sequence range. Previously `GtidSet.parse`/`add` threw on GNOs above 2^53, so a snapshot barrier or checkpoint parse failed outright against a server carrying such GTIDs. Public APIs are unchanged: strings in, strings and booleans out.
+- Fix Previous_gtids decoding silently rounding interval bounds above 2^53: the u64 bounds were combined with Number arithmetic, corrupting `event.gtidSet` - the exact text that seeds `zongji.gtidSet` when a dump starts at a binlog file boundary. Bounds are now decoded as BigInt, and the `event.sids` intervals follow the Number-or-exact-string convention used for other 64-bit values (typed `number | string`; previously typed `number` while silently wrong beyond 2^53).
+- Document on the `seqNo` fields of `mariadbgtid` and `mariadbgtidlist` events that values above `Number.MAX_SAFE_INTEGER` arrive as exact strings and must never be compared numerically - use `event.gtid` with `MariadbGtidPosition`/`GtidSet` instead.
+
 ## [0.8.0] - 2026-07-06
 
 - Export the `GtidSet` class (`import ZongJi, { GtidSet } from '@vlasky/zongji'`): parse a persisted set (e.g. a `zongji.gtidSet` checkpoint or `@@gtid_executed`), `add()` transactions, and test single-transaction membership with `contains(event.gtid)` - handy for snapshot drain barriers ("was this event already covered when I took my snapshot?") without maintaining a separate GTID parser. Fully tagged-GTID aware.

package/README.md

@@ -224,6 +224,18 @@ if (!snapshot.contains(event.gtid)) {
 }
 ```
 
+Its MariaDB counterpart `MariadbGtidPosition` is exported too, with a `covers()` method so consumers do not have to compare sequence numbers themselves (`seqNo` values above `Number.MAX_SAFE_INTEGER` arrive as exact strings, and comparing them as Numbers silently rounds; internally `GtidSet` is exact over MySQL's full GNO range of 1 to 2^63-1 and `MariadbGtidPosition` over MariaDB's full u64 sequence range):
+
+```javascript
+import { MariadbGtidPosition } from '@vlasky/zongji';
+
+const snapshot = MariadbGtidPosition.parse(savedGtidPos); // or @@gtid_binlog_pos
+snapshot.covers(event.gtid);                      // at or before the position?
+snapshot.covers(event.gtid, { inclusive: false }); // strictly before it?
+```
+
+The default (inclusive) form answers "was this transaction's commit at or before the position?", which is what snapshot-barrier checks want: a recorded position covers the whole of its final transaction. Redelivery watermarks must pass `{ inclusive: false }`: all events of a transaction share one GTID, so an inclusive check would treat the watermark transaction's own replayed events as already seen and drop every one after the first. A server id different from the position's entry for that domain reports **not covered**, even for a lower sequence number: after a failover, sequence numbers can regress under the new server id, so ordering against another server's watermark is not meaningful, and for at-least-once consumers an idempotent redelivery is recoverable where a skip is not. A domain the position does not list is likewise not covered.
+
 `zongji.gtidSet` is available when `start()` was given a `gtidSet`, with `startAtEnd` (seeded from the server's `gtid_executed` on MySQL, `gtid_current_pos` on MariaDB), or when reading from the start of a binlog file (seeded from its Previous_gtids or MariaDB GTID list event). It is `undefined` when resuming from an arbitrary mid-file file+position checkpoint, where no exact value can be known. Tagged GTIDs (MySQL 8.3+) are fully supported: tagged transactions appear in the set as `uuid:…:tag:intervals` sections, and a checkpoint containing tags resumes correctly (its wire encoding is only understood by MySQL 8.3+ servers).
 
 On MySQL, GTID resume requires `gtid_mode=ON` and a set is a per-source-UUID collection of intervals. On MariaDB, GTIDs are always on and a position is a single watermark per replication domain (`domain-server-sequence`): resuming replays each listed domain after its watermark, and domains not listed replay from the beginning. If the server has purged binlogs your checkpoint still needs, `start()` emits an error (code 1236) naming the problem; MariaDB additionally reports a checkpoint ahead of its binlog as error 1945/1955. A fresh snapshot is then required.

package/index.d.ts

@@ -151,8 +151,16 @@ export interface AnonymousGtidEvent extends BinlogEvent {
 
 // Previous GTIDs event
 export interface GtidInterval {
-  start: number;
-  end: number;
+  /**
+   * Interval start GNO, inclusive (exact string above
+   * Number.MAX_SAFE_INTEGER)
+   */
+  start: number | string;
+  /**
+   * Interval end GNO, exclusive as on the wire (exact string above
+   * Number.MAX_SAFE_INTEGER)
+   */
+  end: number | string;
 }
 
 export interface GtidSidEntry {
@@ -360,7 +368,12 @@ export interface HeartbeatEvent extends BinlogEvent {
 export interface MariadbGtidEvent extends BinlogEvent {
   getTypeName(): 'MariadbGtid';
   getEventName(): 'mariadbgtid';
-  /** Sequence number within the domain (exact string beyond 2^53) */
+  /**
+   * Sequence number within the domain. Arrives as an exact string above
+   * Number.MAX_SAFE_INTEGER, so never compare it numerically: use
+   * event.gtid with MariadbGtidPosition.covers() (or GtidSet on MySQL)
+   * instead.
+   */
   seqNo: number | string;
   /** Replication domain id */
   domainId: number;
@@ -401,6 +414,11 @@ export interface MariadbGtidEvent extends BinlogEvent {
 export interface MariadbGtidListEntry {
   domainId: number;
   serverId: number;
+  /**
+   * Sequence number within the domain. Arrives as an exact string above
+   * Number.MAX_SAFE_INTEGER, so never compare it numerically: use the
+   * gtid string with MariadbGtidPosition.covers() instead.
+   */
   seqNo: number | string;
   /** Full GTID string (domain-server-sequence) */
   gtid: string;
@@ -671,3 +689,58 @@ export class GtidSet {
   /** Canonical text form (sorted; tagged intervals after untagged) */
   toString(): string;
 }
+
+/**
+ * A MariaDB GTID position (the model behind zongji.gtidSet on MariaDB):
+ * the last transaction processed per replication domain. String form
+ * matches @@gtid_binlog_pos / @@gtid_current_pos, e.g. '0-1-1234,1-3-45'.
+ * Exported so consumers can parse persisted positions and test coverage
+ * via covers() instead of hand-rolling seqNo comparators (which silently
+ * round above Number.MAX_SAFE_INTEGER when done with Number; internally
+ * this class is exact over the full u64 sequence range).
+ */
+export class MariadbGtidPosition {
+  /**
+   * Parses a GTID position string ('' gives an empty position). Rejects
+   * two entries for one domain, as the server would. Throws on malformed
+   * input.
+   */
+  static parse(text: string): MariadbGtidPosition;
+  /**
+   * Builds the position implied by a MariadbGtidList event's gtids
+   * array: the last entry per domain wins
+   */
+  static fromGtidList(entries: Array<{
+    domainId: number;
+    serverId: number;
+    seqNo: number | string;
+  }>): MariadbGtidPosition;
+  /**
+   * Records a transaction as the last processed in its domain
+   * (an overwrite, not a max: "last processed" survives failovers where
+   * sequence numbers regress under a new server id)
+   */
+  add(gtid: {
+    domainId: number;
+    serverId: number;
+    seqNo: number | string;
+  }): void;
+  /**
+   * Whether a transaction ('domain-server-sequence', e.g. an event.gtid)
+   * is covered by this position. inclusive (default true) counts the
+   * position's own last transaction as covered (snapshot-barrier
+   * semantics); redelivery watermarks must pass { inclusive: false },
+   * because all events of a transaction share one GTID and an inclusive
+   * check would drop every replayed event of the watermark transaction
+   * after the first. A server-id mismatch within a domain and an unknown
+   * domain both report not covered; null/undefined is never covered; a
+   * malformed string throws.
+   */
+  covers(
+    gtid: string | null | undefined,
+    options?: { inclusive?: boolean },
+  ): boolean;
+  isEmpty(): boolean;
+  /** Text form, sorted by domain ('D-S-N[,D-S-N...]') */
+  toString(): string;
+}

package/index.js

@@ -989,3 +989,8 @@ export default ZongJi;
 // already covered by my snapshot?") without their own parser. Handles
 // tagged GTIDs (MySQL 8.3+).
 export { GtidSet };
+// Its MariaDB counterpart: the position model behind zongji.gtidSet on
+// MariaDB servers, exported so consumers can parse persisted positions
+// and test coverage via covers() instead of hand-rolling seqNo
+// comparators (which silently round beyond 2^53 when done with Number).
+export { MariadbGtidPosition };

package/lib/binlog_event.js

@@ -230,8 +230,7 @@ class Gtid extends BinlogEvent {
     parser.parseUnsignedNumber(1);
     const gno =
       Common.parseSerializationVarintSigned(parser, messageEnd, what);
-    this.gno = gno >= BigInt(Number.MIN_SAFE_INTEGER) &&
-      gno <= BigInt(Number.MAX_SAFE_INTEGER) ? Number(gno) : String(gno);
+    this.gno = Common.bigintToNumberOrString(gno);
     this.tag = '';
     if (nextFieldIs(3)) {
       parser.parseUnsignedNumber(1);
@@ -307,13 +306,24 @@ class PreviousGtids extends BinlogEvent {
         'PreviousGtids');
       const intervals = [];
       for (let j = 0; j < intervalCount; j++) {
-        const start = parser.parseUnsignedNumber(8);
-        const end = parser.parseUnsignedNumber(8);
+        // u64 bounds as BigInt: parseUnsignedNumber(8) would silently
+        // round beyond 2^53, corrupting the gtidSet text this event seeds
+        // the executed-set tracker with
+        const start = parser.parseBuffer(8).readBigUInt64LE(0);
+        const end = parser.parseBuffer(8).readBigUInt64LE(0);
         intervals.push({ start, end });
       }
       sids.push(tagged ? { sid, tag, intervals } : { sid, intervals });
     }
-    this.sids = sids;
+    // The public sids follow the Number-or-exact-string convention for
+    // 64-bit values (events must stay JSON-serialisable, so no BigInt)
+    this.sids = sids.map(entry => ({
+      ...entry,
+      intervals: entry.intervals.map(({ start, end }) => ({
+        start: Common.bigintToNumberOrString(start),
+        end: Common.bigintToNumberOrString(end),
+      })),
+    }));
     // Entries sharing a uuid print as one block with the untagged
     // intervals first: 'uuid:1-5:tag_a:1:tag_b:1', matching
     // @@gtid_executed. The server writes entries in that order already;
@@ -329,9 +339,9 @@ class PreviousGtids extends BinlogEvent {
       group.sort((a, b) => (a.tag || '') < (b.tag || '') ? -1 : 1);
       return sid + group.map(entry => {
         const ranges = entry.intervals.map(interval =>
-          interval.start === interval.end - 1 ?
+          interval.start === interval.end - 1n ?
             `${interval.start}` :
-            `${interval.start}-${interval.end - 1}`);
+            `${interval.start}-${interval.end - 1n}`);
         return (entry.tag ? `:${entry.tag}` : '') + `:${ranges.join(':')}`;
       }).join('');
     }).join(',');

package/lib/common.js

@@ -58,6 +58,14 @@ export function unsignedInt64ToNumberOrString(high, low) {
   return value <= MAX_SAFE_BIGINT ? Number(value) : value.toString();
 }
 
+// An exact BigInt to the Number-or-exact-string convention used for
+// 64-bit values: a Number within the safe integer range, otherwise an
+// exact decimal string.
+export function bigintToNumberOrString(value) {
+  return (value >= MIN_SAFE_BIGINT && value <= MAX_SAFE_BIGINT) ?
+    Number(value) : value.toString();
+}
+
 // Combine two unsigned 32-bit halves into an exact signed (two's
 // complement) 64-bit value. Returns a Number when within the safe integer
 // range, otherwise an exact string.

package/lib/gtid_set.js

@@ -5,9 +5,10 @@
 // MySQL 8.3+ GTIDs may carry a tag; a (uuid, tag) pair is its own
 // source, printed merged into the uuid's block after the untagged
 // intervals: 'uuid:1-5:tag_a:1:tag_b:1' (tags lowercase, sorted).
-// Intervals are stored internally as [start, end] inclusive, kept sorted
-// and coalesced. GNOs are JavaScript Numbers: MySQL allows up to 2^63-1,
-// but real-world transaction counts sit comfortably inside 2^53.
+// Intervals are stored internally as [start, end] inclusive BigInts, kept
+// sorted and coalesced, so the full GNO range MySQL allows (1 to 2^63-1)
+// is handled exactly; the public API stays strings in, strings and
+// booleans out.
 class GtidSet {
   constructor() {
     // uuid (lowercase, dashed) -> Map of tag ('' = untagged) ->
@@ -97,13 +98,17 @@ class GtidSet {
     }
     const existing = tags.get(tag) || [];
     existing.push([start, end]);
-    existing.sort((a, b) => a[0] - b[0]);
+    // BigInt bounds: a subtraction comparator would hand sort() a BigInt,
+    // which ToNumber rejects
+    existing.sort((a, b) => a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0);
     const merged = [existing[0]];
     for (let i = 1; i < existing.length; i++) {
       const last = merged[merged.length - 1];
       const next = existing[i];
-      if (next[0] <= last[1] + 1) {
-        last[1] = Math.max(last[1], next[1]);
+      if (next[0] <= last[1] + 1n) {
+        if (next[1] > last[1]) {
+          last[1] = next[1];
+        }
       } else {
         merged.push(next);
       }
@@ -178,8 +183,8 @@ class GtidSet {
       }
       offset = buffer.writeBigUInt64LE(BigInt(intervals.length), offset);
       for (const [start, end] of intervals) {
-        offset = buffer.writeBigUInt64LE(BigInt(start), offset);
-        offset = buffer.writeBigUInt64LE(BigInt(end) + 1n, offset);
+        offset = buffer.writeBigUInt64LE(start, offset);
+        offset = buffer.writeBigUInt64LE(end + 1n, offset);
       }
     }
     return buffer;
@@ -222,13 +227,15 @@ function normaliseUuid(text) {
 }
 
 function parsePositiveInt(text, context) {
-  // Decimal digits only, per the MySQL GTID grammar: Number() alone would
-  // also accept '1e3', '0x10' or '1.0' and silently reinterpret them
+  // Decimal digits only, per the MySQL GTID grammar: BigInt() alone would
+  // also accept '0x10' or '0b1' and silently reinterpret them
   if (!/^\d+$/.test(text)) {
     throw new Error(`Invalid GTID number in '${context}': '${text}'`);
   }
-  const value = Number(text);
-  if (!Number.isSafeInteger(value) || value < 1) {
+  const value = BigInt(text);
+  // GNOs are signed 64-bit on the server and start at 1
+  // (GNO_END = 2^63 - 1)
+  if (value < 1n || value > 0x7fffffffffffffffn) {
     throw new Error(`Invalid GTID number in '${context}': '${text}'`);
   }
   return value;

package/lib/mariadb_gtid.js

@@ -9,7 +9,8 @@
 // are streamed from the beginning.
 class MariadbGtidPosition {
   constructor() {
-    // domainId (number) -> { domainId, serverId, seqNo }
+    // domainId (number) -> { domainId, serverId, seqNo }; seqNo is a
+    // BigInt so the full u64 sequence range stays exact
     this._domains = new Map();
   }
 
@@ -21,13 +22,7 @@ class MariadbGtidPosition {
       return position;
     }
     for (const entry of String(text).split(',')) {
-      const parts = entry.trim().split('-');
-      if (parts.length !== 3) {
-        throw new Error(`Invalid MariaDB GTID: '${entry.trim()}'`);
-      }
-      const domainId = parseUInt32(parts[0], entry);
-      const serverId = parseUInt32(parts[1], entry);
-      const seqNo = parseSeqNo(parts[2], entry);
+      const { domainId, serverId, seqNo } = parseSingleGtid(entry);
       if (position._domains.has(domainId)) {
         throw new Error(
           `Duplicate replication domain ${domainId} in GTID position ` +
@@ -59,10 +54,46 @@ class MariadbGtidPosition {
     this._domains.set(gtid.domainId, {
       domainId: gtid.domainId,
       serverId: gtid.serverId,
-      seqNo: gtid.seqNo,
+      // event.seqNo follows the Number-or-exact-string convention for
+      // 64-bit values; BigInt accepts both and stays exact beyond 2^53
+      seqNo: BigInt(gtid.seqNo),
     });
   }
 
+  // Whether a transaction ('domain-server-sequence', e.g. an event.gtid)
+  // is covered by this position. A position names the last transaction
+  // processed per domain, and within a domain the binlog order is total,
+  // so any earlier sequence number from the same server is covered.
+  //
+  // inclusive (default true) decides whether the position's own last
+  // transaction counts as covered. A recorded position means that
+  // transaction completed, so snapshot-barrier checks ("did my snapshot
+  // already include this commit?") want the default. Redelivery
+  // watermarks must pass { inclusive: false }: all events of a
+  // transaction share one GTID, so an inclusive check would treat the
+  // watermark transaction's own replayed events as already seen and drop
+  // every one after the first.
+  //
+  // A server-id mismatch within a domain reports NOT covered even for a
+  // lower sequence number: after a failover, sequence numbers can regress
+  // under the new server id, so ordering against another server's
+  // watermark is not meaningful; for at-least-once consumers an
+  // idempotent redelivery is recoverable where a skip is not. An unknown
+  // domain is likewise not covered. null/undefined (an event.gtid seen
+  // before the stream's first GTID event) is never covered; a malformed
+  // string throws, as it would corrupt rather than answer.
+  covers(gtid, { inclusive = true } = {}) {
+    if (gtid === undefined || gtid === null) {
+      return false;
+    }
+    const { domainId, serverId, seqNo } = parseSingleGtid(gtid);
+    const last = this._domains.get(domainId);
+    if (!last || last.serverId !== serverId) {
+      return false;
+    }
+    return inclusive ? seqNo <= last.seqNo : seqNo < last.seqNo;
+  }
+
   isEmpty() {
     return this._domains.size === 0;
   }
@@ -76,6 +107,19 @@ class MariadbGtidPosition {
   }
 }
 
+function parseSingleGtid(gtid) {
+  const entry = String(gtid).trim();
+  const parts = entry.split('-');
+  if (parts.length !== 3) {
+    throw new Error(`Invalid MariaDB GTID: '${entry}'`);
+  }
+  return {
+    domainId: parseUInt32(parts[0], entry),
+    serverId: parseUInt32(parts[1], entry),
+    seqNo: parseSeqNo(parts[2], entry),
+  };
+}
+
 function parseUInt32(text, context) {
   // Decimal digits only: Number() alone would also accept '1e3' or '0x10'
   if (!/^\d+$/.test(text)) {
@@ -89,19 +133,16 @@ function parseUInt32(text, context) {
 }
 
 function parseSeqNo(text, context) {
+  // Decimal digits only: BigInt() alone would also accept '0x10' or '0b1'
   if (!/^\d+$/.test(text)) {
     throw new Error(`Invalid MariaDB GTID in '${context}': '${text}'`);
   }
-  // seq_no is u64; keep the Number-or-exact-string convention used for
-  // other 64-bit values
-  const value = Number(text);
-  if (Number.isSafeInteger(value)) {
-    return value;
-  }
-  if (BigInt(text) > 0xffffffffffffffffn) {
+  // seq_no is u64; BigInt keeps it exact beyond 2^53
+  const value = BigInt(text);
+  if (value > 0xffffffffffffffffn) {
     throw new Error(`Invalid MariaDB GTID in '${context}': '${text}'`);
   }
-  return text;
+  return value;
 }
 
 export { MariadbGtidPosition };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@vlasky/zongji",
-  "version": "0.8.0",
+  "version": "0.9.0",
   "description": "MySQL and MariaDB binlog-based change data capture (CDC) for Node.js",
   "type": "module",
   "main": "index.js",