@vlasky/zongji 0.7.0 → 0.7.1

package/CHANGELOG.md

@@ -2,6 +2,14 @@
 
 All notable changes to @vlasky/zongji since forking from nevill/zongji.
 
+## [0.7.1] - 2026-07-04
+
+- Fix `start()` calls made while a previous `start()` was still initialising silently discarding their filters. Since 0.7.0 filters are snapshotted and re-calling `start()` is the documented way to update them, but updates made between `start()` and the `ready` event were lost; consumers registering tables during boot (e.g. @vlasky/mysql-live-select) missed events for tables added in that window. Filters passed during initialisation now apply, exactly as when already running; stream options (filename/position/serverId) still come from the first call.
+- Fix a resume-position gap that could silently drop row events. `options.position` was advanced past TableMap events on the cached-metadata path, so a consumer persisting `filename`/`position` for reconnect could resume between a TableMap and its row events; the resumed instance had no metadata for the table id and dropped those rows with no error. TableMap events no longer advance the resume position, closing the gap for single-table statements (the common case). A narrower window remains for multi-table statements (multi-table UPDATE, foreign-key cascades), where the server writes all TableMap events before any row events: emitting the first table's rows still advances the position past the later TableMaps. Rows already processed before a crash may be re-delivered after resume (at-least-once), which is recoverable where dropping is not.
+- Fix rotate events corrupting the `filename`/`position` resume pair. A rotate's header position refers to the old binlog file (0 for the artificial rotate at the start of every dump), yet it was written into `options.position` alongside the new file's name; a consumer resuming from that pair after a real rotation could get "position > file size" or a mid-event read, and the artificial rotate silently reset the start position to 0. The rotate's payload position (the start of the new file) is now used, and the filename update is unconditional. Present in every zongji release since the original upstream project.
+- Fix a corrupt GTID event's parse error being swallowed when `gtid` is excluded by `includeEvents`; the whole following transaction was then silently mislabelled as anonymous. The error now reaches the `error` event regardless of filtering.
+- Schema drift between a binlog event being written and the metadata fetch (e.g. a column dropped in between) now emits a descriptive error naming the table and column counts, instead of throwing a bare TypeError from inside event parsing; the affected table's rows are skipped until its next TableMap event refreshes the metadata.
+
 ## [0.7.0] - 2026-07-04
 
 ### Breaking changes

package/README.md

@@ -169,7 +169,7 @@ Event name  | Description
 `transactionpayload` | Compressed transaction from MySQL 8.0.20+ servers with `binlog_transaction_compression=ON`. ZongJi cannot decode the row events inside it, so it emits an `error` (once per instance) naming the server setting responsible.
 `partialupdaterows` | Partial JSON update from MySQL 8.0+ servers with `binlog_row_value_options=PARTIAL_JSON`. ZongJi cannot decode the JSON diff format, so it emits an `error` (once per instance) naming the server setting responsible.
 
-When the server runs with `gtid_mode=ON`, every emitted event carries a `gtid` property (`'uuid:sequence'`) identifying the transaction it belongs to, tracked internally even if `gtid` events are excluded by `includeEvents`. It is `undefined` for anonymous transactions and for events seen before the stream's first GTID event. Use it for checkpointing and deduplication.
+When the server runs with `gtid_mode=ON`, every emitted event carries a `gtid` property (`'uuid:sequence'`) identifying the transaction it belongs to, tracked internally even if `gtid` events are excluded by `includeEvents`. It is `undefined` for anonymous transactions and for events seen before the stream's first GTID event. Note that non-row events arriving between a transaction's commit and the next transaction's GTID event (e.g. a rotate) still carry the previous transaction's `gtid`. Use it for checkpointing and deduplication.
 
 **Event Methods**
 

package/index.js

@@ -343,12 +343,17 @@ class ZongJi extends EventEmitter {
       return;
     }
 
-    // A duplicate start() while one is already initialising is ignored;
-    // the first call completes. But if stop() intervened, this start()
-    // must proceed as a restart: the epoch below makes the stale
-    // initialisation chain abort, so exactly one binlog dump command is
-    // ever enqueued.
+    // A duplicate start() while one is already initialising is ignored -
+    // the first call completes - but its filters are applied, exactly as
+    // in the already-running branch above: filters are snapshotted, and
+    // re-calling start() is the documented way to update them, so updates
+    // made during the initialisation window must not be lost. Stream
+    // options (filename/position/serverId) still come from the first
+    // call. If stop() intervened, this start() instead proceeds as a
+    // restart: the epoch below makes the stale initialisation chain
+    // abort, so exactly one binlog dump command is ever enqueued.
     if (this._starting && !this.stopped) {
+      this._filters(options);
       return;
     }
     this._starting = true;
@@ -429,8 +434,18 @@ class ZongJi extends EventEmitter {
             this.connection.pause();
             attachGtid(event);
             this._fetchTableInfo(event, () => {
-              // merge the column info with metadata
-              event.updateColumnInfo();
+              try {
+                // merge the column info with metadata
+                event.updateColumnInfo();
+              } catch (err) {
+                // Schema drift between binlog write and metadata fetch:
+                // drop the cached entry (subsequent row events for this
+                // table id are skipped rather than misdecoded) and report
+                delete this.tableMap[event.tableId];
+                this.emit('error', err);
+                if (this.connection) this.connection.resume();
+                return;
+              }
               this.emit('binlog', event);
               if (this.connection) this.connection.resume();
             });
@@ -439,12 +454,31 @@ class ZongJi extends EventEmitter {
           break;
         }
         case 'Rotate':
-          if (this.options.filename !== event.binlogName) {
-            this.options.filename = event.binlogName;
-          }
+          // The payload position is the first event of the NEW file: the
+          // only value coherent with binlogName. The header nextPosition
+          // refers to the OLD file (0 for the artificial rotate at dump
+          // start), so rotates are excluded from the generic update below;
+          // persisting (new filename, old-file offset) would corrupt the
+          // resume point until the next non-rotate event repaired it.
+          this.options.filename = event.binlogName;
+          this.options.position = event.position;
           break;
       }
-      this.options.position = event.nextPosition;
+      // Never advance the resume position past a TableMap: a consumer
+      // persisting options.position could otherwise resume in the gap
+      // between a TableMap and its row events, and with no cached table
+      // metadata those rows would be silently dropped. Holding position
+      // back means the TableMap is replayed before its rows on resume;
+      // rows already seen may be re-emitted (at-least-once), which is
+      // recoverable where dropping is not. A narrower window remains for
+      // multi-table statements (all TableMaps precede all row events, so
+      // emitting the first table's rows advances past the later
+      // TableMaps); the complete fix is advancing only at transaction
+      // boundaries, tracked pre-filter.
+      const typeName = event.getTypeName();
+      if (typeName !== 'TableMap' && typeName !== 'Rotate') {
+        this.options.position = event.nextPosition;
+      }
       this.emit('binlog', attachGtid(event));
     };
 

package/lib/binlog_event.js

@@ -261,6 +261,16 @@ class TableMap extends BinlogEvent {
     const tableMap = this.tableMap[this.tableId];
 
     const columnSchemas = tableMap.columnSchemas;
+    // Schema drift: the table was altered between this binlog event being
+    // written and the metadata fetch. Fail diagnosably rather than with a
+    // bare TypeError from indexing missing columns.
+    if (!columnSchemas || columnSchemas.length < this.columnCount) {
+      throw new Error(
+        `Table ${this.schemaName}.${this.tableName} schema changed between ` +
+        `binlog event and metadata fetch: the event has ${this.columnCount} ` +
+        'columns, fetched metadata has ' +
+        `${columnSchemas ? columnSchemas.length : 0}`);
+    }
     const columns = [];
     for (let j = 0; j < this.columnCount; j++) {
       columns.push({

package/lib/sequence/binlog.js

@@ -159,7 +159,10 @@ export default function initBinlogClass(zongji) {
           }
         }
 
-        if (zongji._skipEvent(eventName)) {
+        // A GTID parse failure corrupts transaction attribution for the
+        // whole following transaction, so it must surface even when the
+        // event type itself is filtered out
+        if (zongji._skipEvent(eventName) && !error) {
           return Binlog.prototype.binlogData;
         }
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@vlasky/zongji",
-  "version": "0.7.0",
+  "version": "0.7.1",
   "description": "MySQL binlog-based change data capture (CDC) for Node.js",
   "type": "module",
   "main": "index.js",