@vlasky/zongji 0.7.0 → 0.8.0

package/CHANGELOG.md

@@ -2,6 +2,28 @@
 
 All notable changes to @vlasky/zongji since forking from nevill/zongji.
 
+## [0.8.0] - 2026-07-06
+
+- Export the `GtidSet` class (`import ZongJi, { GtidSet } from '@vlasky/zongji'`): parse a persisted set (e.g. a `zongji.gtidSet` checkpoint or `@@gtid_executed`), `add()` transactions, and test single-transaction membership with `contains(event.gtid)` - handy for snapshot drain barriers ("was this event already covered when I took my snapshot?") without maintaining a separate GTID parser. Fully tagged-GTID aware.
+- Close the remaining resume-position gap for multi-table statements (multi-table `UPDATE`, foreign-key cascades). The server writes all of a statement's TableMap events before any row event, so a `filename`/`position` pair persisted while the first table's rows were being processed pointed past the later TableMaps; a consumer resuming there dropped the remaining tables' rows with no error. The resume position now freezes at a transaction's first TableMap and advances to the commit marker's end position, so a persisted position always replays whole transactions. Behaviour change: for row-logged transactions the position advances per transaction rather than per event, so resuming after a mid-transaction crash re-delivers that transaction's earlier row events (at-least-once, as already documented for the single-table case fixed in 0.7.1). Statement-logged content (`binlog_format=STATEMENT`/`MIXED`) still advances per event; its replay has no table-metadata dependency, so it never had the dropped-rows hazard.
+- Tagged GTID support (MySQL 8.3+, `GTID_NEXT='AUTOMATIC:tag'`). Tagged transactions arrive as GTID_TAGGED_LOG_EVENT (code 42, a self-describing serialization format unrelated to the classic layout, previously `unknown`) and now decode as ordinary `gtid` events with `event.tag` set and `event.gtid` in `uuid:tag:gno` form. GTID sets accept and print the tagged grammar (`uuid:1-5:tag_a:1`), `zongji.gtidSet` tracks tagged transactions, `start({ gtidSet })` resumes from a tagged checkpoint (COM_BINLOG_DUMP_GTID's tagged payload encoding, understood by MySQL 8.3+ only), and `startAtEnd` seeds from a `gtid_executed` containing tags. This is also a compatibility fix: once a MySQL 8.3+ server has ever executed a tagged GTID it writes every Previous_gtids event in the tagged set encoding forever (surviving `RESET BINARY LOGS AND GTIDS`), which previously crashed parsing at the start of every dump, and any tag in `gtid_executed` broke `startAtEnd` seeding.
+- New `rowsquery` event (MySQL ROWS_QUERY_LOG_EVENT, previously `unknown`): the SQL statement text behind the row events that follow, MySQL's analogue of MariaDB's `annotaterows`. The server sends it to every consumer while `binlog_rows_query_log_events=ON` (default `OFF`); no start option is involved, unlike MariaDB's opt-in `requestAnnotateRows`.
+- `event.gtid` no longer leaks onto events that belong to no transaction: it is now cleared once a commit marker (Xid, XA Prepare, or a COMMIT/ROLLBACK/XA COMMIT/XA ROLLBACK query) has been delivered, so e.g. a rotate arriving between two transactions reports `gtid: undefined` instead of the previous transaction's GTID. The commit event itself still carries its transaction's GTID. A DDL statement has no commit marker of its own, so its GTID persists until the next transaction begins. New `zongji.lastGtid` getter exposes the same tracked value (the GTID of the transaction currently being delivered); it is only coherent read synchronously inside a `'binlog'` handler.
+- Fix text CHAR and VARCHAR columns always decoding as utf8: they now decode via the column's own character set, as TEXT columns already did, so e.g. `latin1` and `greek` columns no longer produce mojibake. In the same pass, charsets whose stored byte order differs from the iconv-lite encoding of the same name are now mapped explicitly: MySQL's `latin1` is decoded as cp1252 (its real definition; `€`, `Ÿ` and friends in 0x80-0x9F previously came back as control characters from TEXT columns), and `ucs2`/`utf16`/`utf32` are decoded big-endian (previously byte-swapped garbage from TEXT columns). Behaviour change: consumers that stored the old garbled output will see correct strings after upgrading.
+- MariaDB support (tested against MariaDB 11.8): ZongJi now detects the server flavour and follows MariaDB binlogs natively, announcing `@mariadb_slave_capability=4` so the server sends its real GTID events instead of rewriting them for legacy clients. MariaDB's GTID model (a `domain-server-sequence` watermark per replication domain) is fully supported: events carry `event.gtid` in MariaDB format, `zongji.gtidSet` exposes a MariaDB GTID position (seeded from `start({ gtidSet })`, from `@@gtid_current_pos` under `startAtEnd`, or from the GTID list event at the start of a binlog file), and `start({ gtidSet: '0-1-1234' })` resumes via `@slave_connect_state` with server-side skipping, including across failover. New event types: `mariadbgtid`, `mariadbgtidlist`, `binlogcheckpoint`, `startencryption`, plus `xaprepare` (also emitted by MySQL 5.7+, previously `unknown`). MariaDB data types decode to match mysql2 query results: `UUID`, `INET4` and `INET6` as canonical text, `VECTOR` as a raw Buffer, `JSON` as the LONGTEXT alias text it is, `COMPRESSED` columns transparently decompressed (with correct charsets, under all `binlog_row_metadata` settings), MariaDB 5.3-era "hires" temporals (`mysql56_temporal_format=OFF`) decoded correctly instead of desynchronising the row, and compressed binlog events (`log_bin_compress=ON`) decoded transparently as ordinary query/row events. `binlog_row_metadata=FULL` works on MariaDB 10.5+ with the same self-describing decode as MySQL; tables whose classic temporal columns could hide hires encodings automatically use `INFORMATION_SCHEMA` instead, and `UUID`/`INET*`/`VECTOR` values arrive as raw Buffers in FULL mode (the binlog cannot distinguish them from `BINARY`). An opt-in `requestAnnotateRows` start option asks the server for `annotaterows` events carrying the SQL statement text behind each row operation. The test suite runs against MariaDB 11.8 in CI alongside MySQL 5.7/8.0/8.4, and an offline MariaDB fixture pins the captured wire bytes of all of the above in the parser tests.
+- Fix the resume position (`options.position`) going stale when the events carrying real positions are excluded by `includeEvents`: filtered events now advance it at the packet layer under the same safety rules as delivered ones (never past a TableMap, never a zero position). On MySQL the position merely lagged; on MariaDB, where events inside a transaction carry `end_log_pos=0`, common filter sets froze it entirely.
+- Fix a filtered `rotate` event leaving an incoherent resume pair: the filename never updated while later events advanced the position into the new file. Rotates now update the `filename`/`position` pair before event filtering, whether delivered or not.
+- GTID-based resume: `start({ gtidSet })` issues COM_BINLOG_DUMP_GTID, letting the server locate the correct binlog file and skip already-processed transactions itself; a persisted checkpoint therefore survives failover to another server in the same replication topology (requires `gtid_mode=ON`). `zongji.gtidSet` exposes the executed set for persisting: seeded exactly from the start set, from the server's `gtid_executed` under `startAtEnd`, or from the stream's Previous_gtids event when reading from the start of a binlog file, and extended only as observed transactions commit. Purged-GTID and GTID-mode errors surface through the `error` event. Heartbeat events (sent in place of server-side-skipped transactions and while idle) are now decoded instead of falling through to `unknown`, and Previous_gtids string formatting is canonical (single transactions print as `8`, not `8-8`)
+- Support `binlog_row_metadata=FULL` (MySQL 8.0+): TableMap events now parse the optional metadata block (column names, signedness, character sets, enum/set value lists, primary key, column visibility), and when it is complete ZongJi decodes rows entirely from the binlog stream with no `INFORMATION_SCHEMA` queries and no connection pauses. Each TableMap event rebuilds the table's metadata, so `ALTER TABLE` can no longer leave stale column definitions behind. Enum/set values containing commas or quotes decode correctly in this mode (the `INFORMATION_SCHEMA` path cannot represent them). Under the default `binlog_row_metadata=MINIMAL`, integer signedness now comes from the binlog instead of being inferred from the `COLUMN_TYPE` string. TableMap events expose `columnNames`, `signedness`, `primaryKey` and `columnVisibility` where available, and column schemas gain `UNSIGNED`, `ENUM_VALUES` and `SET_VALUES`.
+
+## [0.7.1] - 2026-07-04
+
+- Fix `start()` calls made while a previous `start()` was still initialising silently discarding their filters. Since 0.7.0 filters are snapshotted and re-calling `start()` is the documented way to update them, but updates made between `start()` and the `ready` event were lost; consumers registering tables during boot (e.g. @vlasky/mysql-live-select) missed events for tables added in that window. Filters passed during initialisation now apply, exactly as when already running; stream options (filename/position/serverId) still come from the first call.
+- Fix a resume-position gap that could silently drop row events. `options.position` was advanced past TableMap events on the cached-metadata path, so a consumer persisting `filename`/`position` for reconnect could resume between a TableMap and its row events; the resumed instance had no metadata for the table id and dropped those rows with no error. TableMap events no longer advance the resume position, closing the gap for single-table statements (the common case). A narrower window remains for multi-table statements (multi-table UPDATE, foreign-key cascades), where the server writes all TableMap events before any row events: emitting the first table's rows still advances the position past the later TableMaps. Rows already processed before a crash may be re-delivered after resume (at-least-once), which is recoverable where dropping is not.
+- Fix rotate events corrupting the `filename`/`position` resume pair. A rotate's header position refers to the old binlog file (0 for the artificial rotate at the start of every dump), yet it was written into `options.position` alongside the new file's name; a consumer resuming from that pair after a real rotation could get "position > file size" or a mid-event read, and the artificial rotate silently reset the start position to 0. The rotate's payload position (the start of the new file) is now used, and the filename update is unconditional. Present in every zongji release since the original upstream project.
+- Fix a corrupt GTID event's parse error being swallowed when `gtid` is excluded by `includeEvents`; the whole following transaction was then silently mislabelled as anonymous. The error now reaches the `error` event regardless of filtering.
+- Schema drift between a binlog event being written and the metadata fetch (e.g. a column dropped in between) now emits a descriptive error naming the table and column counts, instead of throwing a bare TypeError from inside event parsing; the affected table's rows are skipped until its next TableMap event refreshes the metadata.
+
 ## [0.7.0] - 2026-07-04
 
 ### Breaking changes

package/README.md

@@ -3,9 +3,9 @@
 [![node version](https://img.shields.io/node/v/@vlasky/zongji.svg)](https://www.npmjs.com/package/@vlasky/zongji)
 [![license](https://img.shields.io/npm/l/@vlasky/zongji.svg)](https://github.com/vlasky/zongji/blob/master/LICENSE)
 
-MySQL binlog-based change data capture (CDC) for Node.js, [originally created by Nevill Dutt](https://github.com/nevill/zongji).
+MySQL and MariaDB binlog-based change data capture (CDC) for Node.js, [originally created by Nevill Dutt](https://github.com/nevill/zongji).
 
-[@vlasky/zongji](https://github.com/vlasky/zongji) has been tested working with MySQL 5.7, 8.0 and 8.4.
+[@vlasky/zongji](https://github.com/vlasky/zongji) has been tested working with MySQL 5.7, 8.0 and 8.4, and MariaDB 11.8.
 
 It leverages [`mysql2`](https://github.com/sidorares/node-mysql2) for connections and authentication, while using zongji's binlog parsing and event pipeline.
 
@@ -72,8 +72,9 @@ const { default: ZongJi } = await import('@vlasky/zongji');
   $ npm install @vlasky/zongji
   ```
 
-* Enable MySQL binlog in `my.cnf`, restart MySQL server after making the changes.
-  > Binlog checksum is enabled by default in all supported MySQL versions. Zongji can work with it, but it doesn't verify the checksum.
+* Enable the binlog in `my.cnf`, restart the server after making the changes.
+  > Binlog checksum is enabled by default in all supported MySQL and MariaDB versions. Zongji can work with it, but it doesn't verify the checksum.
+  > On MariaDB, `binlog_format` defaults to `MIXED`, so setting it to `row` explicitly matters there in particular.
 
   ```
   # Must be unique integer from 1-2^32
@@ -87,6 +88,12 @@ const { default: ZongJi } = await import('@vlasky/zongji');
   binlog_do_db     = employees   # Optional, limit which databases to log
   expire_logs_days = 10          # Optional, purge old logs
   max_binlog_size  = 100M        # Optional, limit log size
+
+  # Recommended on MySQL 8.0+ / MariaDB 10.5+: makes binlog events carry
+  #   complete column metadata (names, signedness, charsets, enum/set
+  #   values), so ZongJi decodes rows without querying INFORMATION_SCHEMA
+  #   (see below)
+  binlog_row_metadata = FULL
   ```
 * Create an account with replication privileges, e.g. given privileges to account `zongji` (or any account that you use to read binary logs)
 
@@ -112,6 +119,17 @@ Option | Effect on row values
 `decimalNumbers` | `DECIMAL` columns are returned as exact strings by default (e.g. `'-123.4500'`); set `decimalNumbers: true` to receive Numbers (may lose precision beyond 15 significant digits).
 `jsonStrings` | `JSON` columns are returned as parsed JavaScript values by default; set `jsonStrings: true` to receive JSON strings.
 
+### Table metadata source
+
+To decode row events, ZongJi needs each table's column metadata. It obtains this in one of two ways:
+
+* **From the binlog itself** (MySQL 8.0+ or MariaDB 10.5+ with `binlog_row_metadata=FULL`): every `TableMap` event carries column names, signedness, character sets and enum/set value lists. ZongJi decodes rows entirely from the stream: no `INFORMATION_SCHEMA` queries, no pausing the binlog connection, and no risk of the metadata being stale after an `ALTER TABLE` (each event describes the schema in force when it was written). Enum/set values containing commas or quotes also decode correctly only in this mode.
+* **From `INFORMATION_SCHEMA`** (all other cases, including MySQL 5.7, the MySQL default `binlog_row_metadata=MINIMAL` and the MariaDB default `NO_LOG`): on the first event for each table, ZongJi briefly pauses the stream and fetches column metadata over its control connection. If the table is altered between the binlog write and the fetch, decoded rows may be misinterpreted until the next `TableMap` refresh.
+
+Setting `binlog_row_metadata=FULL` on the server is therefore recommended. It is a dynamic global variable (`SET GLOBAL binlog_row_metadata=FULL`) with negligible binlog size overhead for typical schemas. Even under `MINIMAL`, MySQL 8.0+ binlogs carry exact integer signedness, which ZongJi uses in preference to parsing the column definition.
+
+MariaDB caveats under `FULL`: the binlog cannot distinguish `UUID`, `INET4`, `INET6` and `VECTOR` columns from `BINARY`/`VARBINARY`, so in this mode their values arrive as raw `Buffer`s rather than the formatted values the `INFORMATION_SCHEMA` path produces; and tables containing classic temporal type codes (which may hide MariaDB 5.3-era "hires" columns the binlog cannot describe) automatically fall back to the `INFORMATION_SCHEMA` path.
+
 Each instance includes the following methods:
 
 Method Name | Arguments | Description
@@ -139,6 +157,8 @@ Option Name | Type | Description
 `startAtEnd` | `boolean` | Pass `true` to only emit binlog events that occur after ZongJi's instantiation. Must be used in `start()` method for effect.<br>**Default:** `false`
 `filename` | `string` | Begin reading events from this binlog file. If specified together with `position`, will take precedence over `startAtEnd`.
 `position` | `integer` | Begin reading events from this position. Must be included with `filename`.
+`gtidSet` | `string` | Resume from executed GTIDs (e.g. a persisted `zongji.gtidSet`): a MySQL GTID set (`'uuid:1-27'`; tagged transactions print as `'uuid:1-27:tag_a:1'` on MySQL 8.3+) or a MariaDB GTID position (`'0-1-1234'`), matching the server flavour. The server locates the correct binlog file itself and skips transactions already processed, so a checkpoint remains valid across failover to another server in the same replication topology. Requires `gtid_mode=ON` on MySQL; always available on MariaDB. Pass `''` to stream the entire available history. Takes precedence over `filename`/`position` and `startAtEnd`.
+`requestAnnotateRows` | `boolean` | MariaDB only: ask the server to send `annotaterows` events (the SQL statement text preceding each row operation's events). Off by default; note the statement text may contain sensitive literals that the row images alone would not expose. Ignored on MySQL.<br>**Default:** `false`
 `includeEvents` | `[string]` | Array of event names to include<br>**Example:** `['writerows', 'updaterows', 'deleterows']`
 `excludeEvents` | `[string]` | Array of event names to exclude<br>**Example:** `['rotate', 'tablemap']`
 `includeSchema` | `object` | Object describing which databases and tables to include (Only for row events). Use database names as the key and pass an array of table names or `true` (for the entire database).<br>**Example:** ```{ 'my_database': ['allow_table', 'another_table'], 'another_db': true }```
@@ -156,20 +176,57 @@ Event name  | Description
 `unknown`   | Catch any other events
 `query`     | [Insert/Update/Delete Query](https://dev.mysql.com/doc/internals/en/query-event.html)
 `intvar`    | [Autoincrement and LAST_INSERT_ID](https://dev.mysql.com/doc/internals/en/intvar-event.html)
-`rotate`    | [New Binlog file](https://dev.mysql.com/doc/internals/en/rotate-event.html) Not required to be included to rotate to new files, but it is required to be included in order to keep the `filename` and `position` properties updated with current values for [graceful restarting on errors](https://gist.github.com/numtel/5b37b2a7f47b380c1a099596c6f3db2f).
+`rotate`    | [New Binlog file](https://dev.mysql.com/doc/internals/en/rotate-event.html). The `filename` and `position` resume properties stay updated across rotations whether or not this event is included; include it only if you want to observe rotations yourself.
 `format`    | [Format Description](https://dev.mysql.com/doc/internals/en/format-description-event.html)
 `xid`       | [Transaction ID](https://dev.mysql.com/doc/internals/en/xid-event.html)
-`gtid`      | GTID event with `gtid`, `sid`, `gno` properties
-`anonymousgtid` | Anonymous GTID event (same shape as `gtid`)
-`previousgtids` | Previous GTIDs event with `gtidSet` and `sids`
+`gtid`      | MySQL GTID event with `gtid`, `sid`, `gno` properties. Tagged GTIDs (MySQL 8.3+, `GTID_NEXT='AUTOMATIC:tag'`) arrive as the same event with `tag` set and `gtid` in `uuid:tag:gno` form
+`anonymousgtid` | MySQL anonymous GTID event (same shape as `gtid`)
+`previousgtids` | MySQL Previous GTIDs event with `gtidSet` and `sids`
+`mariadbgtid` | MariaDB GTID event with `gtid` (`'domain-server-sequence'`), `domainId`, `serverId`, `seqNo` and flags (`standalone`, `isDdl`, XA states). Replaces the `BEGIN` query event of a transaction.
+`mariadbgtidlist` | MariaDB GTID list event (start of every binlog file): the last GTID per domain and server seen so far, MariaDB's analogue of `previousgtids`
+`binlogcheckpoint` | MariaDB binlog checkpoint event naming the oldest binlog file still needed for crash recovery (informational)
+`annotaterows` | MariaDB: the SQL `statement` that produced the row events that follow (the analogue of MySQL's rows-query event). Only sent when `start()` was given `requestAnnotateRows: true`; useful for audit trails and statement attribution in CDC records
+`rowsquery` | MySQL: the SQL `statement` that produced the row events that follow (the analogue of MariaDB's `annotaterows`). Sent to every consumer while the server runs with `binlog_rows_query_log_events=ON` (default `OFF`); no `start()` option is involved. The same sensitivity note applies: statement text may contain literals
+`startencryption` | MariaDB encrypted-binlog marker (informational: the server decrypts before sending, the stream itself is cleartext)
+`xaprepare` | XA PREPARE event (MySQL 5.7+ and MariaDB) terminating an XA-prepared event group, with `xidFormatId`, `xidGtrid`, `xidBqual` and `onePhase`
+`heartbeat` | Sent by the server instead of real events: while the connection idles, and in place of transactions skipped server-side during a GTID resume. Carries `binlogName`; `nextPosition` holds the advanced position.
 `tablemap`  | Before any row event (must be included for any other row events)
 `writerows` | Rows inserted, row data array available as `rows` property on event object
 `updaterows` | Rows changed, row data array available as `rows` property on event object
 `deleterows` | Rows deleted, row data array available as `rows` property on event object
-`transactionpayload` | Compressed transaction from MySQL 8.0.20+ servers with `binlog_transaction_compression=ON`. ZongJi cannot decode the row events inside it, so it emits an `error` (once per instance) naming the server setting responsible.
+`transactionpayload` | Compressed transaction from MySQL 8.0.20+ servers with `binlog_transaction_compression=ON`. ZongJi cannot decode the row events inside it, so it emits an `error` (once per instance) naming the server setting responsible. (MariaDB's equivalent, `log_bin_compress=ON`, IS supported: its compressed events decode transparently as ordinary `query` and row events.)
 `partialupdaterows` | Partial JSON update from MySQL 8.0+ servers with `binlog_row_value_options=PARTIAL_JSON`. ZongJi cannot decode the JSON diff format, so it emits an `error` (once per instance) naming the server setting responsible.
 
-When the server runs with `gtid_mode=ON`, every emitted event carries a `gtid` property (`'uuid:sequence'`) identifying the transaction it belongs to, tracked internally even if `gtid` events are excluded by `includeEvents`. It is `undefined` for anonymous transactions and for events seen before the stream's first GTID event. Use it for checkpointing and deduplication.
+When the server runs with `gtid_mode=ON` (MySQL), or always on MariaDB, every emitted event carries a `gtid` property (`'uuid:sequence'` on MySQL, `'domain-server-sequence'` on MariaDB) identifying the transaction it belongs to, tracked internally even if GTID events are excluded by `includeEvents`. It is `undefined` for anonymous transactions, for events seen before the stream's first GTID event, and for events that belong to no transaction (e.g. a rotate arriving between one transaction's commit and the next transaction's GTID event; the commit event itself still carries its transaction's `gtid`). One exception: a DDL statement has no commit marker of its own, so its `gtid` persists on such trailing events until the next transaction begins. Use it for checkpointing and deduplication. The same value is available as `zongji.lastGtid` while an event is being delivered; it is only coherent when read synchronously inside a `'binlog'` handler, so checkpointing code should prefer `event.gtid`.
+
+### File and position resume
+
+`zongji.options.filename` and `zongji.options.position` always hold a safe resume point for the connected server. The position advances only at transaction boundaries: it freezes when a transaction's first `TableMap` arrives and jumps to the commit marker's end position, so a persisted pair never points inside a transaction (where row events could be separated from the table metadata they need and silently dropped). Resuming from a pair persisted mid-transaction therefore replays that whole transaction: consumers should treat delivery as at-least-once and make their processing idempotent. Read the pair synchronously inside a `'binlog'` handler when checkpointing per event.
+
+### GTID-based resume
+
+`zongji.gtidSet` exposes the executed GTIDs as a string: a MySQL GTID set (e.g. `'3e11fa47-…:1-27'`) or a MariaDB GTID position (e.g. `'0-1-1234'`), depending on the connected server. It is the value ZongJi started from, extended as each observed transaction *commits*, so a persisted value never claims a transaction whose row events were still in flight. Persist it alongside (or instead of) `filename`/`position` and resume with:
+
+```javascript
+zongji.start({ gtidSet: savedGtidSet, serverId: 5 });
+```
+
+Unlike file+position checkpoints, a GTID checkpoint is valid on any server in the same replication topology, which makes resuming across a failover possible. The server performs the skipping itself, so already-processed transactions are not resent.
+
+The `GtidSet` class behind it is exported for consumers that need to work with GTID sets themselves, e.g. testing whether an incoming `event.gtid` was already covered by a snapshot:
+
+```javascript
+import { GtidSet } from '@vlasky/zongji';
+
+const snapshot = GtidSet.parse(savedGtidSet); // or @@gtid_executed
+if (!snapshot.contains(event.gtid)) {
+  applyChange(event);
+}
+```
+
+`zongji.gtidSet` is available when `start()` was given a `gtidSet`, with `startAtEnd` (seeded from the server's `gtid_executed` on MySQL, `gtid_current_pos` on MariaDB), or when reading from the start of a binlog file (seeded from its Previous_gtids or MariaDB GTID list event). It is `undefined` when resuming from an arbitrary mid-file file+position checkpoint, where no exact value can be known. Tagged GTIDs (MySQL 8.3+) are fully supported: tagged transactions appear in the set as `uuid:…:tag:intervals` sections, and a checkpoint containing tags resumes correctly (its wire encoding is only understood by MySQL 8.3+ servers).
+
+On MySQL, GTID resume requires `gtid_mode=ON` and a set is a per-source-UUID collection of intervals. On MariaDB, GTIDs are always on and a position is a single watermark per replication domain (`domain-server-sequence`): resuming replays each listed domain after its watermark, and domains not listed replay from the beginning. If the server has purged binlogs your checkpoint still needs, `start()` emits an error (code 1236) naming the problem; MariaDB additionally reports a checkpoint ahead of its binlog as error 1945/1955. A fresh snapshot is then required.
 
 **Event Methods**
 
@@ -183,6 +240,7 @@ Name   | Description
 ## Important Notes
 
 * :star2: All MySQL column types are supported, with type casting similar to [mysql2](https://github.com/sidorares/node-mysql2).
+* MariaDB-specific types and features are supported: `UUID`, `INET4` and `INET6` decode to their canonical text forms and `VECTOR` to a raw `Buffer` of little-endian float32s (matching mysql2 query results); MariaDB `JSON` is a `LONGTEXT` alias and yields strings; `COMPRESSED` columns are decompressed transparently; compressed binlog events (`log_bin_compress=ON`) decode as ordinary query/row events; and MariaDB 5.3-era "hires" temporal columns (tables written under `mysql56_temporal_format=OFF`) decode correctly.
 * :speak_no_evil: 64-bit integers are decoded exactly using native BigInt (see #108). If an integer is within the safe range of JS numbers (-2^53, 2^53), a Number is returned, otherwise an exact String. This also applies to 64-bit integers inside JSON columns.
 * :point_right: `TRUNCATE` statement does not cause corresponding `DeleteRows` event. Use unqualified `DELETE FROM` for same effect.
 * When using fractional seconds with `DATETIME` and `TIMESTAMP` data types, only millisecond precision is available due to the limit of Javascript's `Date` object.
@@ -191,8 +249,9 @@ Name   | Description
 ## Run Tests
 
 * Install [Docker](https://www.docker.com/community-edition#download)
-* Run `docker-compose up -d` to start MySQL containers
-* Run `npm test` to execute the test suite
+* Run `docker-compose up -d` to start the MySQL and MariaDB containers
+* Run `npm test` to execute the test suite against MySQL 8.4 (port 3306)
+* Run `TEST_MYSQL_PORT=3308 npm test` to execute it against MariaDB 11.8; MySQL-only tests skip automatically, and the MariaDB-specific suite in `test/mariadb.js` runs (it no-ops against MySQL)
 
 ## References
 

package/index.d.ts

@@ -11,8 +11,27 @@ export interface ZongJiOptions {
   filename?: string;
   /** Binlog position to start from */
   position?: number;
+  /**
+   * Executed GTIDs to resume from (e.g. a persisted zongji.gtidSet): a
+   * MySQL GTID set ('uuid:1-5,...', tagged intervals as
+   * 'uuid:1-5:tag_a:1,...' on MySQL 8.3+) or a MariaDB GTID position
+   * ('0-1-1234,...'), matching the server flavour. The server locates
+   * the right binlog file and skips transactions already processed
+   * (MySQL: COM_BINLOG_DUMP_GTID, requires gtid_mode=ON; MariaDB:
+   * @slave_connect_state, always available), so this works across
+   * failover to another server in the same topology. '' streams the
+   * server's entire available history. Takes precedence over
+   * filename/position. A set containing tags uses a wire encoding only
+   * MySQL 8.3+ understands; older servers reject it.
+   */
+  gtidSet?: string;
   /** If true, use non-blocking mode */
   nonBlock?: boolean;
+  /**
+   * MariaDB only: request ANNOTATE_ROWS events (the SQL statement text
+   * preceding each row operation's TableMap events). Ignored on MySQL.
+   */
+  requestAnnotateRows?: boolean;
   /** List of event types to include (e.g., ['tablemap', 'writerows']) */
   includeEvents?: string[];
   /** List of event types to exclude */
@@ -23,13 +42,22 @@ export interface ZongJiOptions {
   excludeSchema?: Record<string, string[] | true>;
 }
 
-// Column metadata from INFORMATION_SCHEMA
+// Column metadata, either fetched from INFORMATION_SCHEMA or synthesised
+// from the binlog's own TableMap metadata (binlog_row_metadata=FULL).
+// In the synthesised form COLUMN_TYPE is approximate for character columns
+// (no display width) and COLLATION_NAME/COLUMN_COMMENT are not available.
 export interface ColumnSchema {
   COLUMN_NAME: string;
   COLLATION_NAME: string | null;
   CHARACTER_SET_NAME: string | null;
   COLUMN_COMMENT: string;
   COLUMN_TYPE: string;
+  /** Exact signedness from binlog metadata (MySQL 8.0+); numeric columns only */
+  UNSIGNED?: boolean;
+  /** ENUM value list from binlog metadata (binlog_row_metadata=FULL) */
+  ENUM_VALUES?: string[];
+  /** SET value list from binlog metadata (binlog_row_metadata=FULL) */
+  SET_VALUES?: string[];
 }
 
 // Column information from TableMap event
@@ -95,9 +123,15 @@ export interface GtidEvent extends BinlogEvent {
   flags: number;
   /** Source UUID */
   sid: string;
-  /** Group number */
-  gno: number;
-  /** Full GTID string (sid:gno) */
+  /** Group number (exact string beyond Number.MAX_SAFE_INTEGER) */
+  gno: number | string;
+  /**
+   * GTID tag (MySQL 8.3+, set via GTID_NEXT='AUTOMATIC:tag'); '' for
+   * untagged transactions. Only present on events that arrived as
+   * GTID_TAGGED_LOG_EVENT; classic GTID events leave it undefined.
+   */
+  tag?: string;
+  /** Full GTID string (sid:gno, or sid:tag:gno when tagged) */
   gtid: string;
 }
 
@@ -109,8 +143,8 @@ export interface AnonymousGtidEvent extends BinlogEvent {
   flags: number;
   /** Source UUID */
   sid: string;
-  /** Group number */
-  gno: number;
+  /** Group number (exact string beyond Number.MAX_SAFE_INTEGER) */
+  gno: number | string;
   /** Full GTID string (sid:gno) */
   gtid: string;
 }
@@ -123,6 +157,12 @@ export interface GtidInterval {
 
 export interface GtidSidEntry {
   sid: string;
+  /**
+   * GTID tag ('' for the untagged entry). Only present when the server
+   * wrote the tagged set encoding (MySQL 8.3+ after any tagged GTID);
+   * each (sid, tag) pair is its own entry, repeating the sid.
+   */
+  tag?: string;
   intervals: GtidInterval[];
 }
 
@@ -197,7 +237,45 @@ export interface TableMapEvent extends BinlogEvent {
   columnsMetadata: Record<string, unknown>[];
   /** Reference to the tableMap cache */
   tableMap: Record<number, TableMapEntry>;
-  /** Update column info after fetching from INFORMATION_SCHEMA */
+  /**
+   * Column names carried by the event itself (MySQL 8.0+ with
+   * binlog_row_metadata=FULL)
+   */
+  columnNames?: string[];
+  /**
+   * Per-column signedness from binlog metadata (MySQL 8.0+, MINIMAL and
+   * FULL); true = unsigned. Undefined entries are non-numeric columns.
+   */
+  signedness?: (boolean | undefined)[];
+  /**
+   * Primary key column indexes (0-based, all-columns numbering) from
+   * binlog_row_metadata=FULL
+   */
+  primaryKey?: number[];
+  /**
+   * Per-column visibility from binlog_row_metadata=FULL (MySQL 8.0.23+);
+   * false = INVISIBLE column
+   */
+  columnVisibility?: boolean[];
+  /**
+   * Per-column geometry subtype from binlog metadata (0 = geometry,
+   * 1 = point, ...); undefined entries are non-spatial columns
+   */
+  geometryTypes?: (number | undefined)[];
+  /**
+   * True when the event carries binlog_row_metadata=FULL metadata, in
+   * which case zongji decodes rows without querying INFORMATION_SCHEMA
+   */
+  hasSelfDescribingMetadata(): boolean;
+  /**
+   * True when the table contains classic temporal type codes, which on
+   * MariaDB may hide 5.3 "hires" columns that binlog metadata cannot
+   * reveal; such tables use the INFORMATION_SCHEMA path even under FULL
+   */
+  hasAmbiguousTemporalColumns(): boolean;
+  /** Build ColumnSchema entries from the event's own metadata (FULL only) */
+  buildColumnSchemas(): ColumnSchema[];
+  /** Update column info after table metadata is available */
   updateColumnInfo(): void;
 }
 
@@ -262,6 +340,141 @@ export interface PartialUpdateRowsEvent extends BinlogEvent {
   getEventName(): 'partialupdaterows';
 }
 
+// Heartbeat event - sent while the connection idles (when a heartbeat
+// period is configured) and after transactions were skipped server-side
+// during a GTID dump; nextPosition carries the advanced position
+export interface HeartbeatEvent extends BinlogEvent {
+  getTypeName(): 'Heartbeat';
+  getEventName(): 'heartbeat';
+  /** Current binlog filename */
+  binlogName: string;
+  /**
+   * Heartbeat position beyond 4 GiB (MariaDB only: sent in a sub-header
+   * when the u32 nextPosition cannot represent it, which is then 0)
+   */
+  position?: number | string;
+}
+
+// MariaDB GTID event (code 162) - replaces the BEGIN Query event of a
+// transaction (standalone=false) or marks a standalone group such as DDL
+export interface MariadbGtidEvent extends BinlogEvent {
+  getTypeName(): 'MariadbGtid';
+  getEventName(): 'mariadbgtid';
+  /** Sequence number within the domain (exact string beyond 2^53) */
+  seqNo: number | string;
+  /** Replication domain id */
+  domainId: number;
+  /** Originating server id (from the event header) */
+  serverId: number;
+  /** Raw flags2 byte */
+  flags2: number;
+  /** True for standalone groups (DDL, ...) with no terminating commit */
+  standalone: boolean;
+  /** True when the group can be safely rolled back */
+  transactional: boolean;
+  /** True when the group contains DDL */
+  isDdl: boolean;
+  /** XA PREPARE group; XID fields present */
+  preparedXa: boolean;
+  /** XA COMMIT/ROLLBACK group; XID fields present */
+  completedXa: boolean;
+  /** Group-commit id (same for all members of one group commit) */
+  commitId?: number | string;
+  /** XA format id, when preparedXa/completedXa */
+  xidFormatId?: number;
+  /** XA gtrid, when preparedXa/completedXa */
+  xidGtrid?: Buffer;
+  /** XA bqual, when preparedXa/completedXa */
+  xidBqual?: Buffer;
+  /** Raw flags_extra byte, when present */
+  flagsExtra?: number;
+  /** Number of extra engines, when FL_EXTRA_MULTI_ENGINE is set */
+  extraEngines?: number;
+  /** Start-alter sequence number, for COMMIT/ROLLBACK ALTER groups */
+  saSeqNo?: number | string;
+  /** Thread id of the originating connection (MariaDB 11.5+) */
+  threadId?: number;
+  /** Full GTID string (domain-server-sequence) */
+  gtid: string;
+}
+
+export interface MariadbGtidListEntry {
+  domainId: number;
+  serverId: number;
+  seqNo: number | string;
+  /** Full GTID string (domain-server-sequence) */
+  gtid: string;
+}
+
+// MariaDB GTID list event (code 163) - at the start of every binlog file,
+// the last GTID per (domain, server) seen so far; MariaDB's analogue of
+// MySQL's Previous_gtids
+export interface MariadbGtidListEvent extends BinlogEvent {
+  getTypeName(): 'MariadbGtidList';
+  getEventName(): 'mariadbgtidlist';
+  count: number;
+  /** High 4 bits of the count word (FLAG_UNTIL_REACHED, FLAG_IGN_GTIDS) */
+  flags: number;
+  gtids: MariadbGtidListEntry[];
+}
+
+// MariaDB binlog checkpoint event (code 161) - XA-recovery marker naming
+// the oldest binlog file that may still be needed for crash recovery
+export interface BinlogCheckpointEvent extends BinlogEvent {
+  getTypeName(): 'BinlogCheckpoint';
+  getEventName(): 'binlogcheckpoint';
+  binlogName: string;
+}
+
+// MariaDB annotate rows event (code 160) - the SQL statement text for the
+// row events that follow
+export interface AnnotateRowsEvent extends BinlogEvent {
+  getTypeName(): 'AnnotateRows';
+  getEventName(): 'annotaterows';
+  /**
+   * The event carries the statement's original bytes with no charset
+   * metadata; decoded as utf8 (exact for utf8/utf8mb4 clients, the
+   * overwhelmingly common case).
+   */
+  statement: string;
+}
+
+// MySQL rows query event (code 29) - the SQL statement text for the row
+// events that follow, sent while the server runs with
+// binlog_rows_query_log_events=ON
+export interface RowsQueryEvent extends BinlogEvent {
+  getTypeName(): 'RowsQuery';
+  getEventName(): 'rowsquery';
+  /**
+   * The event carries the statement's original bytes with no charset
+   * metadata; decoded as utf8 (exact for utf8/utf8mb4 clients, the
+   * overwhelmingly common case).
+   */
+  statement: string;
+}
+
+// MariaDB start encryption event (code 164) - informational; the dump
+// thread decrypts server-side and the stream itself is cleartext
+export interface StartEncryptionEvent extends BinlogEvent {
+  getTypeName(): 'StartEncryption';
+  getEventName(): 'startencryption';
+  scheme: number;
+  keyVersion: number;
+  nonce: Buffer;
+}
+
+// XA prepare event (code 38, MySQL 5.7+ and MariaDB) - terminates an
+// XA-prepared event group
+export interface XaPrepareEvent extends BinlogEvent {
+  getTypeName(): 'XaPrepare';
+  getEventName(): 'xaprepare';
+  /** XA COMMIT ... ONE PHASE (commits immediately) */
+  onePhase: boolean;
+  xidFormatId: number;
+  xidGtrid: Buffer;
+  xidBqual: Buffer;
+}
+
 // Unknown event type
 export interface UnknownEvent extends BinlogEvent {
   getTypeName(): 'Unknown';
@@ -284,6 +497,14 @@ export type AnyBinlogEvent =
   | UpdateRowsEvent
   | TransactionPayloadEvent
   | PartialUpdateRowsEvent
+  | HeartbeatEvent
+  | MariadbGtidEvent
+  | MariadbGtidListEvent
+  | BinlogCheckpointEvent
+  | AnnotateRowsEvent
+  | RowsQueryEvent
+  | StartEncryptionEvent
+  | XaPrepareEvent
   | UnknownEvent;
 
 /** Union of all possible getTypeName() return values (e.g. 'Rotate', 'WriteRows') */
@@ -329,14 +550,42 @@ declare class ZongJi extends EventEmitter {
   stopped: boolean;
   /** Whether binlog checksum is enabled */
   useChecksum: boolean;
+  /**
+   * Server version string as reported by SELECT VERSION(), e.g.
+   * '8.4.5' or '11.8.8-MariaDB-ubu2404-log'. Set during start()
+   * initialisation, before the 'ready' event.
+   */
+  serverVersion: string | undefined;
+  /** True when the connected server is MariaDB (set during start()) */
+  isMariaDb: boolean;
   /** Current binlog options */
   options: {
     serverId?: number;
     filename?: string;
     position?: number;
     startAtEnd?: boolean;
+    gtidSet?: string;
     nonBlock?: boolean;
+    requestAnnotateRows?: boolean;
   };
+  /**
+   * The transactions this instance knows to be processed: the start()
+   * seed plus every transaction whose commit has been observed. A MySQL
+   * GTID set ('uuid:1-5,...') or a MariaDB GTID position ('0-1-1234,...')
+   * depending on the connected server. Persist it and pass to
+   * start({ gtidSet }) to resume, including on a different server.
+   * Undefined when no exact seed was available (a mid-file file+position
+   * start).
+   */
+  readonly gtidSet: string | undefined;
+  /**
+   * The GTID of the transaction whose events are currently being
+   * delivered (equal to their event.gtid); undefined between
+   * transactions and for anonymous transactions. Only coherent when
+   * read synchronously inside a 'binlog' handler: checkpointing code
+   * should use event.gtid.
+   */
+  readonly lastGtid: string | undefined;
   /** Current filter settings */
   filters: {
     includeEvents?: string[];
@@ -394,3 +643,31 @@ declare class ZongJi extends EventEmitter {
 }
 
 export default ZongJi;
+
+/**
+ * A MySQL GTID set (the model behind zongji.gtidSet): which transactions
+ * per source are already executed. String form matches @@gtid_executed,
+ * including tagged GTIDs (MySQL 8.3+): 'uuid:1-5:11,uuid2:1-27',
+ * 'uuid:1-5:tag_a:1'. Exported so consumers can parse persisted sets and
+ * test event.gtid membership without their own parser.
+ */
+export class GtidSet {
+  /**
+   * Parses a GTID set string ('' gives an empty set). Within a uuid
+   * block a non-numeric item is a tag naming the source of the interval
+   * items that follow it. Throws on malformed input.
+   */
+  static parse(text: string): GtidSet;
+  /** Adds a single transaction ('uuid:gno' or 'uuid:tag:gno') */
+  add(gtid: string): void;
+  /**
+   * Whether a single transaction ('uuid:gno' or 'uuid:tag:gno', e.g. an
+   * event.gtid) is contained in the set. null/undefined (an anonymous
+   * transaction's event.gtid) is never contained; a malformed string
+   * throws.
+   */
+  contains(gtid: string | null | undefined): boolean;
+  isEmpty(): boolean;
+  /** Canonical text form (sorted; tagged intervals after untagged) */
+  toString(): string;
+}