pgoutput-source-adapter 0.1.1 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: ab9b7c87b7ede5dd583ae9457f30fe30fdfcf1dd55b2ca33e08162d23f36601b
4
- data.tar.gz: 5c7a7b1a081f8a67a07397ecd8034a462e679e636dd2a51d71b9512e90333c85
3
+ metadata.gz: 96fa814d19258cf12c3f2d6398f72f21f1a2dfab59e89562b880f26fbb7bf1cd
4
+ data.tar.gz: 300f6a751391ee90ba84dfdbe743839e605fb6655aafc5057f9b87a1e25c3faf
5
5
  SHA512:
6
- metadata.gz: 3a19f060ba7f5552dec37471a114fa4ee30f0c960d20a27c82676f888753c53fe76c4473038170185806dda76988217ab882cf90f5186fdfa28d5dfef438c12a
7
- data.tar.gz: 00f2d815d0ff8f2d8b9865ecdbee82c528fba3341ecf0125c8f32ddc930365c669bb28ad0cc3c1274c7208089a4eca9ff05bf6d47a9061b234563c92b9c3fd18
6
+ metadata.gz: 314c92d8ec4d3853056f4c69d99eefb6794aaefbc58a1b5f9e0b2de25c372cce1e8c4864210ade04400d23feb60fc47d68d535c1df901d7ef60a373bc4c735b8
7
+ data.tar.gz: 88c5f21bc01667b196a1b347e176cca6aa7caa655a058fa1550bdee3bcb3b88c511fd9371225f5a95fe585dac572a219b46a14a7135e956c97ac47fd8466e986
data/CHANGELOG.md CHANGED
@@ -1,5 +1,47 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ## [0.3.0] - 2026-07-17
4
+
5
+ ### Added
6
+
7
+ - Added `ReplicaIdentityResolver` for schema-aware non-`id` and composite keys,
8
+ keyed by relation id, schema/table pair, or qualified relation name.
9
+ - Added default extraction from decoded relation key-column metadata when
10
+ events expose replica-identity columns or flagged relation columns.
11
+
12
+ ### Fixed
13
+
14
+ - Preserved complete composite and non-`id` old-key tuples for normalized
15
+ UPDATE and DELETE events.
16
+ - Fail closed when configured replica-key columns are absent instead of
17
+ emitting an incomplete `ChangeEvent#primary_key`.
18
+
19
+ ### Documentation
20
+
21
+ - Documented relation metadata, schema-aware resolver configuration, and the
22
+ legacy `id` compatibility fallback.
23
+
24
+ ## [0.2.0] - 2026-07-17
25
+
26
+ ### Added
27
+
28
+ - Added incremental `Cdc#each_normalized` transaction normalization for
29
+ unbounded replication streams.
30
+ - Added `Cdc#stream_event` so source integrations can preserve transport WAL
31
+ positions without owning transaction buffering.
32
+
33
+ ### Documentation
34
+
35
+ - Raised and enforced generated YARD API documentation coverage at 100%.
36
+
37
+ ### Quality
38
+
39
+ - Raised and enforced SimpleCov line and branch coverage at 100%.
40
+
41
+ ## [0.1.1] - 2026-06-16
42
+
43
+ - Improved gemspec summary and description
44
+
3
45
  ## [0.1.0] - 2026-06-16
4
46
 
5
47
  ### Added
@@ -31,4 +73,4 @@
31
73
  ### Coverage
32
74
 
33
75
  - Line Coverage: 97.62%
34
- - Branch Coverage: 91.18%
76
+ - Branch Coverage: 91.18%
data/README.md CHANGED
@@ -85,20 +85,57 @@ envelope = results.first
85
85
  # => CDC::Core::TransactionEnvelope
86
86
  ```
87
87
 
88
+ Normalize an unbounded replication stream incrementally:
89
+
90
+ ```ruby
91
+ inputs = Enumerator.new do |stream|
92
+ runner.start do |payload, metadata|
93
+ decoded = decoder.decode(parser.process(payload))
94
+ stream << adapter.stream_event(decoded, source_position: metadata.wal_end_lsn)
95
+ end
96
+ end
97
+
98
+ adapter.each_normalized(inputs) do |work|
99
+ processor.process(work)
100
+ end
101
+ ```
102
+
103
+ Transaction buffering belongs to the source adapter. Changes outside a
104
+ transaction are yielded immediately; changes between `Begin` and `Commit` are
105
+ yielded as a `CDC::Core::TransactionEnvelope`. `normalize_many` uses the same
106
+ normalization path for finite batches.
107
+
88
108
  ## Primary keys
89
109
 
90
- For update and delete events, pgoutput may provide an old-key tuple. When it does, that tuple is used as the `CDC::Core::ChangeEvent#primary_key`.
110
+ For update and delete events, pgoutput may provide an old-key tuple. When it
111
+ does, the complete tuple is used as `CDC::Core::ChangeEvent#primary_key`,
112
+ including composite and non-`id` replica keys.
91
113
 
92
- For insert events, or for sources without old-key tuples, the adapter defaults to `id` / `"id"` when present.
114
+ For inserts and events without an old-key tuple, the default resolver uses
115
+ relation key-column metadata when the decoded event exposes
116
+ `replica_identity_columns`, `key_columns`, or a relation whose column flags mark
117
+ replica-key columns. The adapter extracts every declared column and raises
118
+ `Pgoutput::SourceAdapter::Error` rather than returning a partial key.
93
119
 
94
- You can provide your own resolver:
120
+ When decoded events do not carry relation key metadata, configure the reusable
121
+ schema-aware resolver. Relations may be keyed by relation id,
122
+ `[schema, table]`, or a qualified name:
95
123
 
96
124
  ```ruby
125
+ resolver = Pgoutput::SourceAdapter::ReplicaIdentityResolver.new(
126
+ ["public", "memberships"] => ["tenant_id", "member_uuid"],
127
+ "public.accounts" => ["account_uuid"]
128
+ )
129
+
97
130
  adapter = Pgoutput::SourceAdapter::Cdc.new(
98
- primary_key_resolver: ->(_event, values) { { "uuid" => values.fetch("uuid") } }
131
+ primary_key_resolver: resolver
99
132
  )
100
133
  ```
101
134
 
135
+ The legacy `id` / `"id"` inference remains as a compatibility fallback only
136
+ when neither relation metadata nor an explicit resolver is available. Custom
137
+ callables remain supported through `primary_key_resolver:`.
138
+
102
139
  ## Metadata
103
140
 
104
141
  Each normalized event includes pgoutput metadata:
@@ -124,6 +161,7 @@ adapter = Pgoutput::SourceAdapter::Cdc.new(
124
161
  ```ruby
125
162
  Pgoutput::SourceAdapter
126
163
  Pgoutput::SourceAdapter::Cdc
164
+ Pgoutput::SourceAdapter::ReplicaIdentityResolver
127
165
  ```
128
166
 
129
167
  A compatibility alias is also provided for the generated gem path:
@@ -151,6 +189,10 @@ Those responsibilities belong to `pgoutput-client`, `pgoutput-parser`, `pgoutput
151
189
  bundle exec rake
152
190
  ```
153
191
 
192
+ The default quality task enforces 100% line and branch coverage and 100% YARD
193
+ API documentation coverage. Run `bundle exec yard stats --list-undoc` to inspect
194
+ the documented public API objects.
195
+
154
196
  ## License
155
197
 
156
198
  [MIT](./LICENSE.txt).
data/Rakefile CHANGED
@@ -1,6 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require 'bundler/gem_tasks'
4
+ require 'open3'
4
5
  require 'rake/testtask'
5
6
  require 'rubocop/rake_task'
6
7
  require 'yard'
@@ -17,6 +18,16 @@ end
17
18
 
18
19
  YARD::Rake::YardocTask.new(:yard)
19
20
 
21
+ desc 'Require complete public API documentation'
22
+ task yard_coverage: :yard do
23
+ output, status = Open3.capture2e('bundle', 'exec', 'yard', 'stats', '--list-undoc')
24
+ puts output
25
+
26
+ next if status.success? && output.include?('100.00% documented')
27
+
28
+ abort 'YARD documentation coverage must remain at 100%'
29
+ end
30
+
20
31
  desc 'Validate rbs sig files'
21
32
  task :steep do
22
33
  sh 'bundle exec steep check'
@@ -27,4 +38,4 @@ task :coverage do
27
38
  sh 'xdg-open coverage/index.html'
28
39
  end
29
40
 
30
- task default: %i[test rubocop steep yard]
41
+ task default: %i[test rubocop steep yard_coverage]
@@ -2,7 +2,15 @@
2
2
 
3
3
  require_relative '../source_adapter'
4
4
 
5
+ # Root namespace for pgoutput protocol libraries.
6
+ #
7
+ # @api public
5
8
  module Pgoutput
9
+ # Backward-compatible namespace for the original generated gem layout.
10
+ #
11
+ # New integrations should use {Pgoutput::SourceAdapter} directly.
12
+ #
13
+ # @api public
6
14
  module Source
7
15
  # Compatibility alias for {Pgoutput::SourceAdapter}.
8
16
  #
@@ -2,12 +2,20 @@
2
2
 
3
3
  require 'cdc/core'
4
4
 
5
- unless CDC::Core.const_defined?(:SourceAdapter)
6
- raise LoadError, 'pgoutput-source-adapter requires a cdc-core version that defines CDC::Core::SourceAdapter'
7
- end
8
-
9
5
  module Pgoutput
10
6
  module SourceAdapter
7
+ # Resolves the required cdc-core adapter contract.
8
+ #
9
+ # @api private
10
+ def self.ensure_core_source_adapter!(core)
11
+ return if core.const_defined?(:SourceAdapter)
12
+
13
+ raise LoadError, 'pgoutput-source-adapter requires a cdc-core version that defines CDC::Core::SourceAdapter'
14
+ end
15
+ private_class_method :ensure_core_source_adapter!
16
+
17
+ ensure_core_source_adapter!(CDC::Core)
18
+
11
19
  # Normalizes Pgoutput::Decoder::Events into CDC::Core primitives.
12
20
  #
13
21
  # This adapter is intentionally located under the Pgoutput namespace because
@@ -22,10 +30,40 @@ module Pgoutput
22
30
  # @example Normalize a decoded transaction event batch
23
31
  # envelope = adapter.normalize_many([begin_event, insert_event, commit_event]).first
24
32
  #
33
+ # @example Normalize an unbounded decoded event stream
34
+ # adapter.each_normalized(decoded_events) { |work| process(work) }
35
+ #
25
36
  # @api public
37
+ # rubocop:disable Metrics/ClassLength
26
38
  class Cdc < CDC::Core::SourceAdapter
39
+ # Canonical source name included in normalized event metadata.
40
+ #
41
+ # @return [String]
27
42
  SOURCE_NAME = 'pgoutput'
28
43
 
44
+ # A decoded event paired with its transport-level source position.
45
+ #
46
+ # Source integrations use this value object to preserve WAL position
47
+ # metadata without teaching downstream applications about pgoutput
48
+ # transaction boundaries.
49
+ #
50
+ # @api public
51
+ class StreamEvent
52
+ # @return [Object] decoded pgoutput event
53
+ attr_reader :event
54
+ # @return [String, nil] transport WAL position
55
+ attr_reader :source_position
56
+
57
+ # @param event [Object] decoded pgoutput event
58
+ # @param source_position [String, nil] transport WAL position
59
+ # @return [void]
60
+ def initialize(event, source_position)
61
+ @event = event
62
+ @source_position = source_position
63
+ freeze
64
+ end
65
+ end
66
+
29
67
  # @param primary_key_resolver [#call, nil] optional callable used to infer
30
68
  # primary keys from decoded row values when pgoutput does not provide an
31
69
  # old-key tuple. The callable receives the decoded event and value hash.
@@ -50,7 +88,7 @@ module Pgoutput
50
88
  # @raise [Pgoutput::SourceAdapter::Error] when the decoded event type is
51
89
  # unsupported.
52
90
  # rubocop:disable Metrics/MethodLength
53
- def normalize(event)
91
+ def normalize(event, source_position: nil)
54
92
  case event_name(event)
55
93
  when 'Insert'
56
94
  change_event(
@@ -58,7 +96,8 @@ module Pgoutput
58
96
  operation: :insert,
59
97
  old_values: nil,
60
98
  new_values: event.values,
61
- primary_key: primary_key_for(event, event.values)
99
+ primary_key: primary_key_for(event, event.values),
100
+ source_position: source_position
62
101
  )
63
102
  when 'Update'
64
103
  old_values = event.old_values || event.old_key
@@ -68,7 +107,8 @@ module Pgoutput
68
107
  operation: :update,
69
108
  old_values: old_values,
70
109
  new_values: event.new_values,
71
- primary_key: primary_key_for(event, event.new_values)
110
+ primary_key: primary_key_for(event, event.new_values),
111
+ source_position: source_position
72
112
  )
73
113
  when 'Delete'
74
114
  old_values = event.old_values || event.old_key
@@ -78,7 +118,8 @@ module Pgoutput
78
118
  operation: :delete,
79
119
  old_values: old_values,
80
120
  new_values: nil,
81
- primary_key: primary_key_for(event, old_values)
121
+ primary_key: primary_key_for(event, old_values),
122
+ source_position: source_position
82
123
  )
83
124
  when 'Begin', 'Commit'
84
125
  nil
@@ -88,6 +129,31 @@ module Pgoutput
88
129
  end
89
130
  # rubocop:enable Metrics/MethodLength
90
131
 
132
+ # Pair a decoded event with the source position supplied by its transport.
133
+ #
134
+ # @param event [Object] decoded pgoutput event
135
+ # @param source_position [Object, nil] WAL position associated with the event
136
+ # @return [StreamEvent] immutable streaming input
137
+ def stream_event(event, source_position: nil)
138
+ StreamEvent.new(event, lsn_string(source_position))
139
+ end
140
+
141
+ # Incrementally normalize a decoded pgoutput event stream.
142
+ #
143
+ # Transaction buffering belongs to this source adapter: row changes are
144
+ # retained after Begin and emitted as one CDC::Core::TransactionEnvelope
145
+ # at Commit. Changes outside a transaction are yielded immediately.
146
+ #
147
+ # @param events [Enumerable<Object, StreamEvent>] decoded stream
148
+ # @yieldparam work [CDC::Core::ChangeEvent, CDC::Core::TransactionEnvelope]
149
+ # @return [Enumerator, nil] enumerator without a block, otherwise nil
150
+ def each_normalized(events, &block)
151
+ return enum_for(:each_normalized, events) unless block
152
+
153
+ normalize_stream(events, flush_incomplete: false, &block)
154
+ nil
155
+ end
156
+
91
157
  # Normalize a sequence of decoded pgoutput events.
92
158
  #
93
159
  # If the sequence contains transaction boundaries, row changes between a
@@ -99,17 +165,28 @@ module Pgoutput
99
165
  # normalized row changes and transaction envelopes in input order.
100
166
  # @raise [Pgoutput::SourceAdapter::Error] when any decoded event type is
101
167
  # unsupported.
168
+ def normalize_many(events)
169
+ results = [] # : Array[CDC::Core::ChangeEvent | CDC::Core::TransactionEnvelope]
170
+ normalize_stream(events, flush_incomplete: true) { |work| results << work }
171
+ share(results.freeze)
172
+ end
173
+
174
+ private
175
+
176
+ attr_reader :primary_key_resolver, :metadata_builder
177
+
102
178
  # rubocop:disable Metrics/AbcSize
103
179
  # rubocop:disable Metrics/PerceivedComplexity
104
180
  # rubocop:disable Metrics/CyclomaticComplexity
105
181
  # rubocop:disable Metrics/MethodLength
106
- def normalize_many(events)
107
- results = [] # : Array[CDC::Core::ChangeEvent | CDC::Core::TransactionEnvelope]
182
+ def normalize_stream(events, flush_incomplete:, &block)
108
183
  transaction_id = nil
109
184
  transaction_events = [] # : Array[CDC::Core::ChangeEvent]
110
185
  transaction_metadata = {} # : Hash[String, untyped]
111
186
 
112
- events.each do |event|
187
+ # rubocop:disable Metrics/BlockLength
188
+ events.each do |input|
189
+ event, source_position = stream_input(input)
113
190
  case event_name(event)
114
191
  when 'Begin'
115
192
  transaction_id = event.transaction_id
@@ -120,11 +197,14 @@ module Pgoutput
120
197
  )
121
198
  when 'Commit'
122
199
  if transaction_id || !transaction_events.empty?
123
- results << transaction_envelope(
124
- event,
125
- transaction_id: transaction_id || event.transaction_id,
126
- events: transaction_events,
127
- metadata: transaction_metadata
200
+ block.call(
201
+ transaction_envelope(
202
+ event,
203
+ transaction_id: transaction_id || event.transaction_id,
204
+ events: transaction_events,
205
+ metadata: transaction_metadata,
206
+ source_position: source_position
207
+ )
128
208
  )
129
209
  end
130
210
 
@@ -132,30 +212,26 @@ module Pgoutput
132
212
  transaction_events = []
133
213
  transaction_metadata = {}
134
214
  else
135
- normalized = normalize(event)
215
+ normalized = normalize(event, source_position: source_position)
136
216
  next if normalized.nil?
137
217
 
138
218
  if transaction_id
139
219
  transaction_events << normalized
140
220
  else
141
- results << normalized
221
+ block.call(normalized)
142
222
  end
143
223
  end
144
224
  end
225
+ # rubocop:enable Metrics/BlockLength
145
226
 
146
- results.concat(transaction_events) if transaction_id && !transaction_events.empty?
147
- share(results.freeze)
227
+ transaction_events.each(&block) if flush_incomplete && transaction_id && !transaction_events.empty?
148
228
  end
149
229
  # rubocop:enable Metrics/AbcSize
150
230
  # rubocop:enable Metrics/PerceivedComplexity
151
231
  # rubocop:enable Metrics/CyclomaticComplexity
152
232
  # rubocop:enable Metrics/MethodLength
153
233
 
154
- private
155
-
156
- attr_reader :primary_key_resolver, :metadata_builder
157
-
158
- def change_event(event, operation:, old_values:, new_values:, primary_key:)
234
+ def change_event(event, operation:, old_values:, new_values:, primary_key:, source_position:)
159
235
  share(
160
236
  CDC::Core::ChangeEvent.new(
161
237
  operation: operation,
@@ -165,17 +241,18 @@ module Pgoutput
165
241
  new_values: new_values,
166
242
  primary_key: primary_key,
167
243
  transaction_id: event.transaction_id,
244
+ commit_lsn: lsn_string(source_position),
168
245
  metadata: metadata_for(event)
169
246
  )
170
247
  )
171
248
  end
172
249
 
173
- def transaction_envelope(event, transaction_id:, events:, metadata:)
250
+ def transaction_envelope(event, transaction_id:, events:, metadata:, source_position: nil)
174
251
  share(
175
252
  CDC::Core::TransactionEnvelope.new(
176
253
  transaction_id: transaction_id,
177
254
  events: events.freeze,
178
- commit_lsn: lsn_string(event.commit_lsn),
255
+ commit_lsn: lsn_string(event.commit_lsn || source_position),
179
256
  committed_at: event.commit_timestamp,
180
257
  metadata: metadata.merge(metadata_for(event)).merge(
181
258
  'commit_flags' => event.flags,
@@ -191,7 +268,15 @@ module Pgoutput
191
268
  primary_key_resolver.call(event, values)
192
269
  end
193
270
 
194
- def default_primary_key(_event, values)
271
+ def stream_input(input)
272
+ return [input.event, input.source_position] if input.is_a?(StreamEvent)
273
+
274
+ [input, nil]
275
+ end
276
+
277
+ def default_primary_key(event, values)
278
+ key_columns = replica_identity_columns_for(event)
279
+ return key_from_columns(event, values, key_columns) unless key_columns.nil?
195
280
  return nil unless values.respond_to?(:key?)
196
281
 
197
282
  if values.key?('id')
@@ -201,6 +286,27 @@ module Pgoutput
201
286
  end
202
287
  end
203
288
 
289
+ def replica_identity_columns_for(event)
290
+ return normalize_key_columns(event.replica_identity_columns) if event.respond_to?(:replica_identity_columns)
291
+ return normalize_key_columns(event.key_columns) if event.respond_to?(:key_columns)
292
+ return unless event.respond_to?(:relation) && event.relation.respond_to?(:columns)
293
+
294
+ normalize_key_columns(
295
+ event.relation.columns.select { |column| column.flags.allbits?(1) }.map(&:name)
296
+ )
297
+ end
298
+
299
+ def normalize_key_columns(columns)
300
+ column_list = Array(columns) # : Array[untyped]
301
+ column_list.map(&:to_s)
302
+ end
303
+
304
+ def key_from_columns(event, values, columns)
305
+ return nil if columns.empty?
306
+
307
+ ReplicaIdentityResolver.extract(event, values, columns)
308
+ end
309
+
204
310
  def metadata_for(event)
205
311
  metadata = {
206
312
  'source' => SOURCE_NAME,
@@ -232,11 +338,12 @@ module Pgoutput
232
338
  hash.compact
233
339
  end
234
340
 
235
- def share(object)
236
- Ractor.make_shareable(object)
341
+ def share(object, ractor: Ractor)
342
+ ractor.make_shareable(object)
237
343
  rescue Ractor::Error
238
344
  object
239
345
  end
240
346
  end
347
+ # rubocop:enable Metrics/ClassLength
241
348
  end
242
349
  end
@@ -0,0 +1,99 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Pgoutput
4
+ module SourceAdapter
5
+ # Resolves schema-aware replica keys from decoded row values.
6
+ #
7
+ # Relations may be keyed by pgoutput relation id, `[schema, table]`, or a
8
+ # `"schema.table"` string. Each value is the ordered list of replica-key
9
+ # columns. Relation-id mappings take precedence because they identify the
10
+ # exact relation metadata seen by the replication stream.
11
+ #
12
+ # @example Resolve a composite key
13
+ # resolver = ReplicaIdentityResolver.new(
14
+ # ["public", "memberships"] => ["tenant_id", "member_uuid"]
15
+ # )
16
+ #
17
+ # @api public
18
+ class ReplicaIdentityResolver
19
+ # Normalized relation keys mapped to ordered replica-identity columns.
20
+ #
21
+ # @return [Hash]
22
+ attr_reader :relations
23
+
24
+ # Extract a complete replica key from an ordered column list.
25
+ #
26
+ # @param event [Object] decoded row event with schema/table metadata
27
+ # @param values [Hash, nil] decoded row values
28
+ # @param columns [Array<String>] ordered replica-key column names
29
+ # @return [Hash{String=>Object}] complete replica key
30
+ # @raise [Error] when values or any configured column are unavailable
31
+ def self.extract(event, values, columns)
32
+ unless values.respond_to?(:key?)
33
+ raise Error, "replica identity values are unavailable for #{qualified_name(event)}"
34
+ end
35
+
36
+ columns.to_h do |column|
37
+ [column, fetch_column(event, values, column)]
38
+ end
39
+ end
40
+
41
+ # @param relations [Hash{Integer, Array<String>, Array<String>=>Array<String>}]
42
+ # relation identity to ordered replica-key column names
43
+ # @return [void]
44
+ def initialize(relations)
45
+ @relations = relations.to_h.transform_values { |columns| normalize_columns(columns) }.freeze
46
+ freeze
47
+ end
48
+
49
+ # Extract the configured replica key for a decoded row event.
50
+ #
51
+ # @param event [Object] decoded row event with relation/schema/table metadata
52
+ # @param values [Hash, nil] decoded row values
53
+ # @return [Hash{String=>Object}, nil] complete replica key, or `nil` when
54
+ # the relation is not configured
55
+ # @raise [Error] when a configured key column is absent from row values
56
+ def call(event, values)
57
+ columns = columns_for(event)
58
+ return nil unless columns
59
+
60
+ self.class.extract(event, values, columns)
61
+ end
62
+
63
+ def self.fetch_column(event, values, column)
64
+ return values[column] if values.key?(column)
65
+
66
+ symbol = column.to_sym
67
+ return values[symbol] if values.key?(symbol)
68
+
69
+ raise Error, "replica identity column #{column.inspect} is missing from #{qualified_name(event)} values"
70
+ end
71
+ private_class_method :fetch_column
72
+
73
+ def self.qualified_name(event)
74
+ [event.schema, event.table].compact.join('.')
75
+ end
76
+ private_class_method :qualified_name
77
+
78
+ private
79
+
80
+ def normalize_columns(columns)
81
+ column_list = Array(columns) # : Array[untyped]
82
+ names = column_list.map(&:to_s)
83
+ raise ArgumentError, 'replica identity columns must not be empty' if names.empty?
84
+ raise ArgumentError, 'replica identity columns must be unique' unless names.uniq.length == names.length
85
+
86
+ names.freeze
87
+ end
88
+
89
+ def columns_for(event)
90
+ relation_id = event.relation_id if event.respond_to?(:relation_id)
91
+ return relations[relation_id] if relation_id && relations.key?(relation_id)
92
+
93
+ schema = event.schema
94
+ table = event.table
95
+ relations[[schema.to_s, table.to_s]] || relations["#{schema}.#{table}"]
96
+ end
97
+ end
98
+ end
99
+ end
@@ -5,6 +5,6 @@ module Pgoutput
5
5
  # Current pgoutput-source-adapter gem version.
6
6
  #
7
7
  # @return [String] semantic version published to RubyGems.
8
- VERSION = '0.1.1'
8
+ VERSION = '0.3.0'
9
9
  end
10
10
  end
@@ -24,4 +24,5 @@ module Pgoutput
24
24
  end
25
25
  end
26
26
 
27
+ require_relative 'source_adapter/replica_identity_resolver'
27
28
  require_relative 'source_adapter/cdc'
@@ -1,5 +1,7 @@
1
1
  module Pgoutput
2
2
  module SourceAdapter
3
+ def self.ensure_core_source_adapter!: (untyped core) -> nil
4
+
3
5
  # Normalizes Pgoutput::Decoder::Events into CDC::Core primitives.
4
6
  #
5
7
  # This adapter is intentionally located under the Pgoutput namespace because
@@ -22,6 +24,13 @@ module Pgoutput
22
24
 
23
25
  SOURCE_NAME: "pgoutput"
24
26
 
27
+ class StreamEvent
28
+ attr_reader event: untyped
29
+ attr_reader source_position: untyped
30
+
31
+ def initialize: (untyped event, untyped source_position) -> void
32
+ end
33
+
25
34
  # @param primary_key_resolver [#call, nil] optional callable used to infer
26
35
  # primary keys from decoded row values when pgoutput does not provide an
27
36
  # old-key tuple.
@@ -37,7 +46,13 @@ module Pgoutput
37
46
  #
38
47
  # @param event [Object] a Pgoutput::Decoder::Events object.
39
48
  # @return [CDC::Core::ChangeEvent, nil]
40
- def normalize: (untyped event) -> untyped
49
+ def normalize: (untyped event, ?source_position: untyped) -> untyped
50
+
51
+ # Pair a decoded event with its transport-level source position.
52
+ def stream_event: (untyped event, ?source_position: untyped) -> StreamEvent
53
+
54
+ # Incrementally normalize an unbounded decoded event stream.
55
+ def each_normalized: (untyped events) ?{ (untyped work) -> untyped } -> (Enumerator[untyped, nil] | nil)
41
56
 
42
57
  # Normalize a sequence of decoded pgoutput events.
43
58
  #
@@ -55,13 +70,23 @@ module Pgoutput
55
70
 
56
71
  attr_reader metadata_builder: untyped
57
72
 
58
- def change_event: (untyped event, operation: untyped, old_values: untyped, new_values: untyped, primary_key: untyped) -> untyped
73
+ def normalize_stream: (untyped events, flush_incomplete: bool) { (untyped work) -> untyped } -> void
59
74
 
60
- def transaction_envelope: (untyped event, transaction_id: untyped, events: untyped, metadata: untyped) -> untyped
75
+ def change_event: (untyped event, operation: untyped, old_values: untyped, new_values: untyped, primary_key: untyped, source_position: untyped) -> untyped
76
+
77
+ def transaction_envelope: (untyped event, transaction_id: untyped, events: untyped, metadata: untyped, ?source_position: untyped) -> untyped
61
78
 
62
79
  def primary_key_for: (untyped event, untyped values) -> untyped
63
80
 
64
- def default_primary_key: (untyped _event, untyped values) -> (nil | untyped)
81
+ def stream_input: (untyped input) -> [untyped, untyped]
82
+
83
+ def default_primary_key: (untyped event, untyped values) -> (nil | untyped)
84
+
85
+ def replica_identity_columns_for: (untyped event) -> untyped
86
+
87
+ def normalize_key_columns: (untyped columns) -> Array[String]
88
+
89
+ def key_from_columns: (untyped event, untyped values, Array[String] columns) -> untyped
65
90
 
66
91
  def metadata_for: (untyped event) -> untyped
67
92
 
@@ -0,0 +1,20 @@
1
+ module Pgoutput
2
+ module SourceAdapter
3
+ class ReplicaIdentityResolver
4
+ @relations: untyped
5
+
6
+ attr_reader relations: untyped
7
+
8
+ def self.extract: (untyped event, untyped values, Array[String] columns) -> Hash[String, untyped]
9
+ def initialize: (untyped relations) -> void
10
+ def call: (untyped event, untyped values) -> untyped
11
+
12
+ private
13
+
14
+ def self.fetch_column: (untyped event, untyped values, String column) -> untyped
15
+ def self.qualified_name: (untyped event) -> String
16
+ def normalize_columns: (untyped columns) -> Array[String]
17
+ def columns_for: (untyped event) -> untyped
18
+ end
19
+ end
20
+ end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pgoutput-source-adapter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ken C. Demanawa
@@ -64,11 +64,13 @@ files:
64
64
  - lib/pgoutput/source/adapter.rb
65
65
  - lib/pgoutput/source_adapter.rb
66
66
  - lib/pgoutput/source_adapter/cdc.rb
67
+ - lib/pgoutput/source_adapter/replica_identity_resolver.rb
67
68
  - lib/pgoutput/source_adapter/version.rb
68
69
  - mise.toml
69
70
  - sig/pgoutput/source/adapter.rbs
70
71
  - sig/pgoutput/source_adapter.rbs
71
72
  - sig/pgoutput/source_adapter/cdc.rbs
73
+ - sig/pgoutput/source_adapter/replica_identity_resolver.rbs
72
74
  - sig/pgoutput/source_adapter/version.rbs
73
75
  homepage: https://github.com/kanutocd/pgoutput-source-adapter
74
76
  licenses: