mysql_replicator 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (5) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +281 -37
  3. data/lib/mysql_replicator/binlogs/rows_event_parser.rb +5 -1
  4. data/lib/mysql_replicator/version.rb +1 -1
  5. metadata +2 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 1fc2f85268cd0a2187b97a8a62761ddaec7730c8d019fcb02b154600fbe00ddb
4
- data.tar.gz: 30977b5f6c98d581c73733b9d9c683802935079b1396b3bb01cfecc8ed140486
3
+ metadata.gz: b8caf562beab557ef33cf471becf5af562c46d13b0613c1dff120b8f6b22f9a3
4
+ data.tar.gz: 511544da5cda5f94331f08fdead13f100eb233e8810320903ed0343a5be50554
5
5
  SHA512:
6
- metadata.gz: b8ec42a9aa2f9fd13b765662777d9f3a68ea6a2551cf56bbeb52402a49a3b29e7b97a4f10971b97547133cd6093091c18271ac0a1ad05cb277db9deb3682d914
7
- data.tar.gz: 94c2da74bc3eff4045590a6e30d9de8b2b38947d191fb779de5eaeb911819eeb89a9b3f0087cd1cb3e18af3239b24eb8092470feb3c6ee4f49d2e4f6195113b9
6
+ metadata.gz: 9ca4f3dcba764981ce012d11699eaf740f303e8ecfc649b498eadd59a72cf5c9aaccc470c7db436adb0ec7df39b79ff8312aca6f511f8580722e48066b92b8cf
7
+ data.tar.gz: ccee5ba75a950ff020314e488b2368460ed16d90aabdba21c0e3023f400c40e4ba8990e9fb7d8077aa0cd1dd42029b79a49c9b03e0a72a8320b7a20971ab7295
data/README.md CHANGED
@@ -1,9 +1,14 @@
1
1
  # MySQL Replicator
2
2
 
3
- `MysqlReplicator` gem is a library for processing MySQL Binlog events using the MySQL replication protocol.
3
+ `MysqlReplicator` gem is a library for processing MySQL Binlog events with [MySQL Replication Protocol](https://dev.mysql.com/doc/dev/mysql-server/9.5.0/page_protocol_replication.html).
4
4
 
5
5
  And also, this is lightweight because only depend on the Ruby standard library.
6
6
 
7
+ ## Support version
8
+
9
+ - MySQL >= 8.0
10
+ - Ruby >= 3.3
11
+
7
12
  ## Installation
8
13
 
9
14
  Install the gem and add to the application's Gemfile by executing:
@@ -26,11 +31,10 @@ gem install mysql_replicator
26
31
 
27
32
  ## Usage
28
33
 
29
- ```rb
30
- # Custom logger available
31
- MysqlReplicator.logger = your_custom_logger
34
+ Run with convenience method.
32
35
 
33
- # Connect to MySQL server, and start to handle replication event
36
+ ```rb
37
+ # `run` method: Connect to MySQL server, and start to handle replication event
34
38
  MysqlReplicator.run(
35
39
  host: 'your_mysql_host', # default localhost
36
40
  port: 3307, # default 3306
@@ -38,40 +42,280 @@ MysqlReplicator.run(
38
42
  password: 'password', # default empty string
39
43
  database: 'test' # default empty string
40
44
  ) do |binlog_event|
41
- # write code to process binlog event
42
-
43
- puts binlog_event[:timestamp]
44
- # => 2025-12-01 12:00:00
45
-
46
- puts binlog_event[:event_type]
47
- # => :WRITE_ROWS
48
-
49
- puts binlog_event[:execution]
50
- # => {
51
- # table_id: 10,
52
- # flags: 0,
53
- # extra_data_length: 0,
54
- # column_count: 2,
55
- # rows: [
56
- # {
57
- # ordinal_position: 1,
58
- # data_type: 'int',
59
- # column_name: 'id',
60
- # value: 1,
61
- # primary_key: true
62
- # },
63
- # {
64
- # ordinal_position: 2,
65
- # data_type: 'varchar',
66
- # column_name: 'name',
67
- # value: 'alice',
68
- # primary_key: false
69
- # }
70
- # ]
71
- # }
72
- end
45
+ # write code to process binlog event
46
+ end
47
+ ```
48
+
49
+ Or run with interface manually.
50
+
51
+ ```rb
52
+ # create connection to MySQL server
53
+ conn = MysqlReplicator::Connection.new(
54
+ host: 'your_mysql_host', # default localhost
55
+ port: 3307, # default 3306
56
+ user: 'username', # default root
57
+ password: 'password', # default empty string
58
+ database: 'test' # default empty string
59
+ )
60
+ # create binlog client
61
+ client = MysqlReplicator::BinlogClient.new(conn)
62
+ # set binlog event handler
63
+ client.on do |binlog_event|
64
+ # write code to process binlog event
65
+ end
66
+ # start replication
67
+ client.start_replication
68
+ ```
69
+
70
+ If you run a create table and insert query.
71
+ ```
72
+ CREATE TABLE users (
73
+ id INT NOT NULL AUTO_INCREMENT,
74
+ name VARCHAR(255),
75
+ PRIMARY KEY (id)
76
+ );
77
+
78
+ INSERT INTO users VALUES ('alice');
79
+ ```
80
+
81
+ MysqlReplicator received `binlog_event` with `:WRITE_ROWS` event.
82
+ When INSERT query, you can get inserted row data.
83
+ ```rb
84
+ puts binlog_event
85
+ # =>
86
+ {
87
+ timestamp: "2025-12-01 12:00:00",
88
+ event_type: :WRITE_ROWS,
89
+ server_id: 1,
90
+ execution: {
91
+ database: 'example',
92
+ table: 'users',
93
+ table_id: 10,
94
+ flags: 0,
95
+ extra_data_length: 0,
96
+ column_count: 2,
97
+ rows: [
98
+ {
99
+ ordinal_position: 1,
100
+ data_type: 'int',
101
+ column_name: 'id',
102
+ value: 1,
103
+ primary_key: true
104
+ },
105
+ {
106
+ ordinal_position: 2,
107
+ data_type: 'varchar',
108
+ column_name: 'name',
109
+ value: 'alice',
110
+ primary_key: false
111
+ }
112
+ ]
113
+ }
114
+ }
115
+ ```
116
+
117
+ If you run the update query.
118
+ ```
119
+ UPDATE users SET name = 'bob' WHERE id = 1;
120
+ ```
121
+
122
+ MysqlReplicator received `binlog_event` with `:UPDATE_ROWS` event.
123
+ When UPDATE query, you can get row data before and after the change.
124
+ ```rb
125
+ puts binlog_event
126
+ # =>
127
+ {
128
+ timestamp: "2025-12-01 12:01:00",
129
+ event_type: :UPDATE_ROWS,
130
+ server_id: 1,
131
+ execution: {
132
+ database: 'example',
133
+ table: 'users',
134
+ table_id: 10,
135
+ flags: 0,
136
+ extra_data_length: 0,
137
+ column_count: 2,
138
+ rows: [
139
+ {
140
+ before: [
141
+ {
142
+ ordinal_position: 1,
143
+ data_type: 'int',
144
+ column_name: 'id',
145
+ value: 1,
146
+ primary_key: true
147
+ },
148
+ {
149
+ ordinal_position: 2,
150
+ data_type: 'varchar',
151
+ column_name: 'name',
152
+ value: 'alice', # before name
153
+ primary_key: false
154
+ }
155
+ ],
156
+ after: [
157
+ {
158
+ ordinal_position: 1,
159
+ data_type: 'int',
160
+ column_name: 'id',
161
+ value: 1,
162
+ primary_key: true
163
+ },
164
+ {
165
+ ordinal_position: 2,
166
+ data_type: 'varchar',
167
+ column_name: 'name',
168
+ value: 'bob', # after name
169
+ primary_key: false
170
+ }
171
+ ]
172
+ }
173
+ ]
174
+ }
175
+ }
73
176
  ```
74
177
 
178
+ If you run the delete query.
179
+ ```
180
+ DELETE FROM users WHERE id = 1;
181
+ ```
182
+
183
+ MysqlReplicator received `binlog_event` with `:DELETE_ROWS` event.
184
+ When DELETE query, you can get row data before and after the change.
185
+ ```rb
186
+ puts binlog_event
187
+ # =>
188
+ {
189
+ timestamp: "2025-12-01 12:02:00",
190
+ event_type: :DELETE_ROWS,
191
+ server_id: 1,
192
+ execution: {
193
+ database: 'example',
194
+ table: 'users',
195
+ table_id: 10,
196
+ flags: 0,
197
+ extra_data_length: 0,
198
+ column_count: 2,
199
+ rows: [
200
+ {
201
+ ordinal_position: 1,
202
+ data_type: 'int',
203
+ column_name: 'id',
204
+ value: 1,
205
+ primary_key: true
206
+ },
207
+ {
208
+ ordinal_position: 2,
209
+ data_type: 'varchar',
210
+ column_name: 'name',
211
+ value: 'bob',
212
+ primary_key: false
213
+ }
214
+ ]
215
+ }
216
+ }
217
+ ```
218
+
219
+ If you want to use your custom logger.
220
+
221
+ ```rb
222
+ # Custom logger available
223
+ MysqlReplicator.logger = your_custom_logger
224
+ ```
225
+
226
+ ## Supported event
227
+
228
+ | Event type | |
229
+ | :--- | :---: |
230
+ | ROTATE_EVENT | ✅ |
231
+ | FORMAT_DESCRIPTION_EVENT | ✅ |
232
+ | QUERY_EVENT | ✅ |
233
+ | TABLE_MAP_EVENT | ✅ |
234
+ | WRITE_ROWS_EVENT (V2) | ✅ |
235
+ | UPDATE_ROWS_EVENT (V2) | ✅ |
236
+ | DELETE_ROWS_EVENT (V2) | ✅ |
237
+ | XID_EVENT | ✅ |
238
+ | GTID_LOG_EVENT | ❌ |
239
+ | PREVIOUS_GTIDS_LOG_EVENT | ❌ |
240
+ | HEARTBEAT_LOG_EVENT | ❌ |
241
+ | STOP_EVENT | ❌ |
242
+
243
+ ## Supported MySQL data type
244
+
245
+ ### Numeric type
246
+
247
+ | MySQL type | |
248
+ | :--- | :---: |
249
+ | TINYINT | ✅ |
250
+ | SMALLINT | ✅ |
251
+ | MEDIUMINT | ✅ |
252
+ | INT | ✅ |
253
+ | BIGINT | ✅ |
254
+ | FLOAT | ✅ |
255
+ | DOUBLE | ✅ |
256
+ | DECIMAL | ✅ |
257
+ | BIT | ❌ |
258
+
259
+ ### String type
260
+
261
+ | MySQL type | |
262
+ | :--- | :---: |
263
+ | CHAR | ✅ |
264
+ | VARCHAR | ✅ |
265
+ | TINYTEXT | ✅ |
266
+ | TEXT | ✅ |
267
+ | MEDIUMTEXT | ✅ |
268
+ | LONGTEXT | ✅ |
269
+ | ENUM | ✅ |
270
+ | SET | ❌ |
271
+
272
+ ### Binary type
273
+
274
+ | MySQL type | |
275
+ | :--- | :---: |
276
+ | BINARY | ✅ |
277
+ | VARBINARY | ✅ |
278
+ | TINYBLOB | ✅ |
279
+ | BLOB | ✅ |
280
+ | MEDIUMBLOB | ✅ |
281
+ | LONGBLOB | ✅ |
282
+
283
+ ### Date and Time type
284
+
285
+ | MySQL type | |
286
+ | :--- | :---: |
287
+ | DATE | ✅ |
288
+ | TIME | ✅ |
289
+ | DATETIME | ✅ |
290
+ | TIMESTAMP | ✅ |
291
+ | YEAR | ❌ |
292
+
293
+ ### Special type
294
+
295
+ | MySQL type | |
296
+ | :--- | :---: |
297
+ | JSON | ✅ |
298
+ | GEOMETRY | ❌ |
299
+
300
+ ## Supported Binary JSON value type
301
+
302
+ | JSON value type | |
303
+ | :--- | :---: |
304
+ | SMALL_OBJECT | ✅ |
305
+ | LARGE_OBJECT | ✅ |
306
+ | SMALL_ARRAY | ✅ |
307
+ | LARGE_ARRAY | ✅ |
308
+ | LITERAL (null / true / false) | ✅ |
309
+ | INT16 | ✅ |
310
+ | UINT16 | ✅ |
311
+ | INT32 | ✅ |
312
+ | UINT32 | ✅ |
313
+ | INT64 | ✅ |
314
+ | UINT64 | ✅ |
315
+ | DOUBLE | ✅ |
316
+ | STRING | ✅ |
317
+ | OPAQUE | ❌ |
318
+
75
319
  ## Development
76
320
 
77
321
  Start MySQL server.
data/lib/mysql_replicator/binlogs/rows_event_parser.rb CHANGED
@@ -17,6 +17,8 @@ module MysqlReplicator
17
17
 
18
18
  # @rbs!
19
19
  # type execution = {
20
+ # database: String | nil,
21
+ # table: String | nil,
20
22
  # table_id: Integer,
21
23
  # flags: Integer,
22
24
  # extra_data_length: Integer,
@@ -26,7 +28,7 @@ module MysqlReplicator
26
28
 
27
29
  # @rbs event_type: :WRITE_ROWS | :UPDATE_ROWS | :DELETE_ROWS
28
30
  # @rbs payload: String
29
- # @rbs checksum_enabled bool
31
+ # @rbs checksum_enabled: bool
30
32
  # @rbs table_map: Hash[Integer, MysqlReplicator::Binlogs::TableMapEventParser::execution]
31
33
  # @rbs return: execution
32
34
  def self.parse(event_type, payload, checksum_enabled, table_map)
@@ -70,6 +72,8 @@ module MysqlReplicator
70
72
  end
71
73
 
72
74
  {
75
+ database: table_def[:database],
76
+ table: table_def[:table],
73
77
  table_id: table_id,
74
78
  flags: flags,
75
79
  extra_data_length: extra_data_length,
data/lib/mysql_replicator/version.rb CHANGED
@@ -2,5 +2,5 @@
2
2
  # rbs_inline: enabled
3
3
 
4
4
  module MysqlReplicator
5
- VERSION = '0.1.0' #: String
5
+ VERSION = '0.2.0' #: String
6
6
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: mysql_replicator
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - yo_waka
@@ -118,7 +118,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
118
118
  - !ruby/object:Gem::Version
119
119
  version: '0'
120
120
  requirements: []
121
- rubygems_version: 3.6.8
121
+ rubygems_version: 4.0.3
122
122
  specification_version: 4
123
123
  summary: The MySQL Binlog event handler using MySQL Replication Protocol
124
124
  test_files: []