cdc-core 0.0.0 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5b050ff23c85039e0169e83b14b7afc1f3c034cd5eedd372982c0f9f82844a2f
-  data.tar.gz: 5dfb41f2cd37724277c33b6932d75cfd57f4cd6b3564aff5317944aa9d97114e
+  metadata.gz: 5ace8b27c4df8285c5354599650cb450cddaad1402c913689e638fb9ec57d8b9
+  data.tar.gz: 381f77ee16278ea147b083b677d5e9248a2fb6bf1955c40f9d2eab82055a1e49
 SHA512:
-  metadata.gz: e34666e6300fbdf9ad44c3a442cc3cee75b0edaad9cde14cbdede71ff68a666752838b2b2aafb0db54254857f704727f26465961bdf0c6e876a80c44acb86174
-  data.tar.gz: ac796f126b00581d4e6d44294953dbe16c0082e0f4b7cdfd85ed1699e813dd82cd0ff2d0e663b0f4ac8c3c7de50d02e2e12801a86c3b1879a0f29f4288f4f8c9
+  metadata.gz: c0447c01564b04f20170e8a7f9e60cb95c12f77ef87ce54ab8a090083b7054a041224247349a9424e803f22255bbe097b33706a16c267ed1710e67fab7e6fff0
+  data.tar.gz: 9aef7f1a3ffd777c6031e700c49f17a3aa460fa46e898d648d55b81c1a2afaf6df4e19fd700bbadad57be382d0809990909f9a97ac721ea7121fc3e327c868b7

data/CHANGELOG.md

@@ -1,5 +1,33 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
 ## [Unreleased]
 
+### Added
+
+- Placeholder for future development.
+
+---
+
 ## [0.1.0] - 2026-05-31
 
-- Initial release
+### Added
+
+- Added immutable `CDC::Core::ChangeEvent`.
+- Added immutable `CDC::Core::ColumnChange`.
+- Added immutable `CDC::Core::TransactionEnvelope`.
+- Added `CDC::Core::EventMetadata`.
+- Added operation validation.
+- Added `CDC::Core::Processor` base contract.
+- Added `CDC::Core::ProcessorResult`.
+- Added `CDC::Core::CompositeProcessor`.
+- Added `CDC::Core::Filter`.
+- Added `CDC::Core::Pipeline`.
+- Added Ractor-safe processor intent declaration via `ractor_safe!`.
+- Added RBS signatures.
+- Added Minitest coverage.
+- Added README and examples.

data/LICENSE.txt

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2026 Ken C. Demanawa
+Copyright (c) 2026 Kenneth C. Demanawa
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
@@ -19,3 +19,4 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 THE SOFTWARE.
+

data/README.md

@@ -1,43 +1,166 @@
-# Cdc::Core
-
-TODO: Delete this and the text below, and describe your gem
+# cdc-core
+
+Database-agnostic Change Data Capture domain primitives for Ruby.
+
+`cdc-core` provides immutable, Ractor-safe event objects and processor contracts for building CDC systems. It intentionally does not connect to databases, parse wire protocols, decode PostgreSQL OIDs, or integrate with Rails.
+
+## Requirements
+
+- Ruby 3.4+
+
+## Features
+
+- Immutable `ChangeEvent` objects
+- Transaction grouping via `TransactionEnvelope`
+- Column-level change objects
+- Processor and composite processor contracts
+- Event filters
+- Small pipeline orchestration object
+- Ractor-safe event and transaction objects
+- RBS signatures
+- YARD-compatible documentation
+- No runtime dependencies
+
+## Ecosystem Position
+
+```text
+pgoutput-client
+      │
+      ▼
+pgoutput-parser
+      │
+      ▼
+pgoutput-decoder
+      │
+      ▼
+cdc-core
+      │
+      ▼
+whodunit-chronicles
+```
 
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/cdc/core`. To experiment with that code, run `bin/console` for an interactive prompt.
+`cdc-core` is the shared vocabulary layer. It defines what a change event, transaction, processor, and pipeline result means without caring where the event came from.
 
 ## Installation
 
-TODO: Replace `UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG` with your gem name right after releasing it to RubyGems.org. Please do not do it earlier due to security reasons. Alternatively, replace this section with instructions to install your gem from git if you don't plan to release to RubyGems.org.
+```ruby
+gem "cdc-core"
+```
+
+```ruby
+require "cdc/core"
+```
 
-Install the gem and add to the application's Gemfile by executing:
+## Change Events
 
-```bash
-bundle add UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG
+```ruby
+event = CDC::Core::ChangeEvent.new(
+  operation: :update,
+  schema: "public",
+  table: "users",
+  old_values: { "email" => "old@example.com" },
+  new_values: { "email" => "new@example.com" },
+  primary_key: { "id" => 7 },
+  transaction_id: 789,
+  commit_lsn: "0/16B6C50"
+)
+
+event.update?
+# => true
+
+event.qualified_table_name
+# => "public.users"
+
+event.changes.map(&:name)
+# => ["email"]
 ```
 
-If bundler is not being used to manage dependencies, install the gem by executing:
+## Transactions
 
-```bash
-gem install UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG
+```ruby
+transaction = CDC::Core::TransactionEnvelope.new(
+  transaction_id: 789,
+  events: [event],
+  commit_lsn: "0/16B6C50",
+  committed_at: Time.now.utc
+)
 ```
 
-## Usage
+A transaction envelope is the natural unit for future parallel processing because it preserves database transaction boundaries.
 
-TODO: Write usage instructions here
+## Processors
 
-## Development
+```ruby
+class AuditProcessor < CDC::Core::Processor
+  def process(event)
+    puts event.to_h
+    CDC::Core::ProcessorResult.success(event)
+  end
+end
+```
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+## Ractor-safe processor intent
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+```ruby
+class AnalyticsProcessor < CDC::Core::Processor
+  ractor_safe!
 
-## Contributing
+  def process(event)
+    CDC::Core::ProcessorResult.success(event)
+  end
+end
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/cdc-core. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/[USERNAME]/cdc-core/blob/main/CODE_OF_CONDUCT.md).
+AnalyticsProcessor.new.ractor_safe?
+# => true
+```
 
-## License
+This declares intent only. `cdc-core` does not execute processors in Ractors. A future runtime gem can use this signal.
+
+## Composite Processor
 
-The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+```ruby
+processor = CDC::Core::CompositeProcessor.new([
+  AuditProcessor.new,
+  AnalyticsProcessor.new
+])
 
-## Code of Conduct
+results = processor.process(event)
+```
+
+## Filters and Pipeline
+
+```ruby
+pipeline = CDC::Core::Pipeline.new(
+  processor: AuditProcessor.new,
+  filters: [
+    CDC::Core::Filter.schema("public"),
+    CDC::Core::Filter.table("users")
+  ]
+)
+
+result = pipeline.process(event)
+```
+
+## Non-goals
+
+`cdc-core` does not:
+
+- Connect to PostgreSQL
+- Parse `pgoutput`
+- Decode PostgreSQL values
+- Manage replication slots
+- Run Ractor pools
+- Persist audit records
+- Integrate with ActiveRecord
+- Publish to Kafka, Redis, or HTTP sinks
+
+## Development
+
+```bash
+bundle exec rake
+bundle exec steep check
+```
+
+## License
 
-Everyone interacting in the Cdc::Core project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/[USERNAME]/cdc-core/blob/main/CODE_OF_CONDUCT.md).
+MIT.

data/lib/cdc/core/change_event.rb

@@ -0,0 +1,116 @@
+# frozen_string_literal: true
+
+module CDC
+  module Core
+    # Immutable representation of one logical database change.
+    #
+    # ChangeEvent is the core data structure passed through filters, pipelines,
+    # and processors. It is database-agnostic but carries common CDC fields such
+    # as operation, schema, table, before/after values, primary key, LSN, and
+    # metadata.
+    class ChangeEvent
+      # @return [Symbol] normalized CDC operation
+      # @return [String] database schema name
+      # @return [String] database table name
+      # @return [Hash, nil] values before the change
+      # @return [Hash, nil] values after the change
+      # @return [Hash, nil] primary key values for the changed row
+      # @return [Object, nil] transaction identifier from the upstream source
+      # @return [String, nil] commit log sequence number
+      # @return [Integer, nil] event sequence within a transaction or stream
+      # @return [Time, nil] timestamp associated with the event
+      # @return [EventMetadata] additional normalized metadata
+      attr_reader :operation, :schema, :table, :old_values, :new_values, :primary_key,
+                  :transaction_id, :commit_lsn, :sequence_number, :occurred_at, :metadata
+
+      # Build a change event.
+      #
+      # @param operation [#to_sym] CDC operation
+      # @param schema [#to_s] schema name
+      # @param table [#to_s] table name
+      # @param old_values [Hash, nil] values before the change
+      # @param new_values [Hash, nil] values after the change
+      # @param primary_key [Hash, nil] primary key values
+      # @param transaction_id [Object, nil] source transaction identifier
+      # @param commit_lsn [#to_s, nil] commit log sequence number
+      # @param sequence_number [Integer, nil] event sequence number
+      # @param occurred_at [Time, nil] event timestamp
+      # @param metadata [Hash, EventMetadata] additional event metadata
+      def initialize(operation:, schema:, table:, old_values: nil, new_values: nil, primary_key: nil,
+                     transaction_id: nil, commit_lsn: nil, sequence_number: nil, occurred_at: nil,
+                     metadata: {})
+        @operation = Operation.normalize(operation)
+        @schema = String(schema).freeze
+        @table = String(table).freeze
+        @old_values = freeze_hash_or_nil(old_values)
+        @new_values = freeze_hash_or_nil(new_values)
+        @primary_key = freeze_hash_or_nil(primary_key)
+        @transaction_id = transaction_id
+        @commit_lsn = commit_lsn&.to_s&.freeze
+        @sequence_number = sequence_number
+        @occurred_at = occurred_at
+        @metadata = metadata.is_a?(EventMetadata) ? metadata : EventMetadata.new(metadata)
+        Ractor.make_shareable(self)
+      end
+
+      # @return [Boolean] true for insert events
+      def insert? = operation == Operation::INSERT
+
+      # @return [Boolean] true for update events
+      def update? = operation == Operation::UPDATE
+
+      # @return [Boolean] true for delete events
+      def delete? = operation == Operation::DELETE
+
+      # Fully qualified table name in schema.table form.
+      #
+      # @return [String]
+      def qualified_table_name = "#{schema}.#{table}".freeze
+
+      # Compute changed columns by comparing old and new values.
+      #
+      # Columns with equal old and new values are omitted. Insert and delete
+      # events can pass nil for one side; missing values are represented as nil.
+      #
+      # @return [Array<ColumnChange>] Ractor-shareable changed columns
+      def changes
+        keys = ((old_values || {}).keys | (new_values || {}).keys)
+        keys.filter_map do |key|
+          change = ColumnChange.new(name: key, old_value: old_values&.[](key), new_value: new_values&.[](key))
+          change if change.changed?
+        end.then { |items| Ractor.make_shareable(items.freeze) }
+      end
+
+      # Convert the event into a Ractor-shareable hash.
+      #
+      # @return [Hash{String=>Object,nil}]
+      def to_h
+        Ractor.make_shareable({
+          'operation' => operation,
+          'schema' => schema,
+          'table' => table,
+          'old_values' => old_values,
+          'new_values' => new_values,
+          'primary_key' => primary_key,
+          'transaction_id' => transaction_id,
+          'commit_lsn' => commit_lsn,
+          'sequence_number' => sequence_number,
+          'occurred_at' => occurred_at,
+          'metadata' => metadata.to_h
+        }.freeze)
+      end
+
+      private
+
+      # Convert a hash into immutable EventMetadata storage, preserving nil.
+      #
+      # @param hash [Hash, nil]
+      # @return [Hash, nil]
+      def freeze_hash_or_nil(hash)
+        return nil if hash.nil?
+
+        EventMetadata.new(hash).to_h
+      end
+    end
+  end
+end

data/lib/cdc/core/column_change.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module CDC
+  module Core
+    # Represents a single column-level value change.
+    #
+    # ColumnChange is immutable and Ractor-shareable. Values that cannot be made
+    # shareable by Ruby are represented by their frozen #inspect string so the
+    # enclosing event can still cross Ractor boundaries safely.
+    class ColumnChange
+      # @return [String] column name
+      # @return [Object, nil] value before the change
+      # @return [Object, nil] value after the change
+      attr_reader :name, :old_value, :new_value
+
+      # Build a column-level change object.
+      #
+      # @param name [#to_s] column name
+      # @param old_value [Object, nil] previous value
+      # @param new_value [Object, nil] new value
+      def initialize(name:, old_value:, new_value:)
+        @name = String(name).freeze
+        @old_value = make_value_shareable(old_value)
+        @new_value = make_value_shareable(new_value)
+        Ractor.make_shareable(self)
+      end
+
+      # Whether the old and new values differ.
+      #
+      # @return [Boolean]
+      def changed?
+        old_value != new_value
+      end
+
+      # Convert the change into a Ractor-shareable hash.
+      #
+      # @return [Hash{String=>Object,nil}]
+      def to_h
+        Ractor.make_shareable({ 'name' => name, 'old_value' => old_value, 'new_value' => new_value }.freeze)
+      end
+
+      private
+
+      # Convert a value into a Ractor-shareable representation.
+      #
+      # @param value [Object, nil]
+      # @return [Object, String, nil]
+      def make_value_shareable(value)
+        Ractor.make_shareable(value)
+      rescue Ractor::Error
+        Ractor.make_shareable(value.inspect.freeze)
+      end
+    end
+  end
+end

data/lib/cdc/core/composite_processor.rb

@@ -0,0 +1,71 @@
+# frozen_string_literal: true
+
+module CDC
+  module Core
+    # Processor that delegates each event to multiple processors.
+    #
+    # CompositeProcessor enables fan-out processing while preserving a simple
+    # sequential execution model. It normalizes truthy/falsey processor returns
+    # into ProcessorResult objects and can stop at the first failure.
+    class CompositeProcessor < Processor
+      # @return [Array<Processor>] processors executed for each event
+      # @return [Boolean] whether processing stops on the first failure
+      attr_reader :processors, :fail_fast
+
+      # Build a composite processor.
+      #
+      # @param processors [Array<#process>] processors to execute
+      # @param fail_fast [Boolean] whether to stop after the first failure
+      def initialize(processors, fail_fast: true) # rubocop:disable Lint/MissingSuper
+        @processors = processors.freeze
+        @fail_fast = fail_fast
+      end
+
+      # Process an event through each configured processor.
+      #
+      # @param event [ChangeEvent] event to process
+      # @return [Array<ProcessorResult>] result from each attempted processor
+      def process(event)
+        results = [] # : Array[ProcessorResult]
+        processors.each do |processor|
+          result = normalize_result(processor.process(event), event)
+          results << result
+          break if fail_fast && result.failure?
+        rescue StandardError => e
+          result = ProcessorResult.failure(e, event:)
+          results << result
+          break if fail_fast
+        end
+        Ractor.make_shareable(results.freeze) if results.none?(&:failure?)
+        results.freeze
+      end
+
+      # Processors that declared Ractor safety.
+      #
+      # @return [Array<Processor>]
+      def ractor_safe_processors
+        processors.select(&:ractor_safe?).freeze
+      end
+
+      # Processors that should remain sequential in the core runtime.
+      #
+      # @return [Array<Processor>]
+      def sequential_processors
+        processors.reject(&:ractor_safe?).freeze
+      end
+
+      private
+
+      # Normalize processor return values into ProcessorResult objects.
+      #
+      # @param result [Object] raw processor result
+      # @param event [ChangeEvent] processed event
+      # @return [ProcessorResult]
+      def normalize_result(result, event)
+        return result if result.is_a?(ProcessorResult)
+
+        result ? ProcessorResult.success(event) : ProcessorResult.skipped(event)
+      end
+    end
+  end
+end

data/lib/cdc/core/errors.rb

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+module CDC
+  module Core
+    # Base error class for all cdc-core specific failures.
+    class Error < StandardError; end
+
+    # Raised when an operation cannot be normalized to a supported CDC action.
+    class InvalidOperationError < Error; end
+
+    # Raised by processors when a processor-specific failure needs wrapping.
+    class ProcessorError < Error; end
+  end
+end

data/lib/cdc/core/event_metadata.rb

@@ -0,0 +1,78 @@
+# frozen_string_literal: true
+
+module CDC
+  module Core
+    # Immutable metadata container for CDC domain objects.
+    #
+    # Metadata keys are normalized to frozen strings. Nested hashes and arrays
+    # are recursively converted into Ractor-shareable objects. Values that Ruby
+    # cannot make shareable are stored as frozen #inspect strings.
+    class EventMetadata
+      # @return [Hash{String=>Object}] normalized metadata
+      attr_reader :data
+
+      # Build metadata from a hash-like structure.
+      #
+      # @param data [Hash] metadata values
+      def initialize(data = {})
+        @data = deep_shareable_hash(data)
+        Ractor.make_shareable(self)
+      end
+
+      # Fetch a metadata value by string or symbol key.
+      #
+      # @param key [String, Symbol] metadata key
+      # @return [Object, nil]
+      def [](key)
+        data[key] || data[key.to_s] || data[key.to_sym]
+      end
+
+      # Return the normalized Ractor-shareable hash.
+      #
+      # @return [Hash{String=>Object}]
+      def to_h
+        data
+      end
+
+      private
+
+      # Recursively normalize and freeze a hash.
+      #
+      # @param hash [Hash]
+      # @return [Hash{String=>Object}]
+      def deep_shareable_hash(hash)
+        converted = hash.each_with_object(
+          {} # : Hash[String, untyped]
+        ) do |(key, value), memo|
+          memo[normalize_key(key)] = normalize_value(value)
+        end
+        Ractor.make_shareable(converted.freeze)
+      end
+
+      # Normalize metadata keys to frozen strings.
+      #
+      # @param key [Object]
+      # @return [String]
+      def normalize_key(key)
+        key.to_s.freeze
+      end
+
+      # Normalize a metadata value into a shareable representation.
+      #
+      # @param value [Object]
+      # @return [Object]
+      def normalize_value(value)
+        case value
+        when Hash
+          deep_shareable_hash(value)
+        when Array
+          Ractor.make_shareable(value.map { |item| normalize_value(item) }.freeze)
+        else
+          Ractor.make_shareable(value)
+        end
+      rescue Ractor::Error
+        Ractor.make_shareable(value.inspect.freeze)
+      end
+    end
+  end
+end

data/lib/cdc/core/filter.rb

@@ -0,0 +1,77 @@
+# frozen_string_literal: true
+
+module CDC
+  module Core
+    # Predicate object used to decide whether a pipeline should process an event.
+    #
+    # Filters are composable with #& and #|. A filter only matches when its
+    # predicate returns true exactly, keeping accidental truthy values from
+    # silently passing events through a pipeline.
+    class Filter
+      # Match every event.
+      #
+      # @return [Filter]
+      def self.all = new { |_event| true }
+
+      # Match events from a schema.
+      #
+      # @param name [#to_s] schema name
+      # @return [Filter]
+      def self.schema(name) = new { |event| event.schema == name.to_s }
+
+      # Match events from a table regardless of schema.
+      #
+      # @param name [#to_s] table name
+      # @return [Filter]
+      def self.table(name) = new { |event| event.table == name.to_s }
+
+      # Match events from a fully qualified schema.table name.
+      #
+      # @param name [#to_s] qualified table name
+      # @return [Filter]
+      def self.qualified_table(name) = new { |event| event.qualified_table_name == name.to_s }
+
+      # Match events by operation.
+      #
+      # @param operation [#to_sym] CDC operation
+      # @return [Filter]
+      def self.operation(operation) = new { |event| event.operation == Operation.normalize(operation) }
+
+      # Build a custom filter.
+      #
+      # @yieldparam event [ChangeEvent] event being tested
+      # @yieldreturn [Boolean] true to match the event
+      # @raise [ArgumentError] when no predicate block is provided
+      def initialize(&predicate)
+        raise ArgumentError, 'predicate block required' unless predicate
+
+        @predicate = predicate
+      end
+
+      # Whether this filter matches an event.
+      #
+      # @param event [ChangeEvent] event to test
+      # @return [Boolean]
+      def match?(event)
+        @predicate.call(event) == true
+      end
+      alias =~ match?
+
+      # Compose this filter with another filter using logical AND.
+      #
+      # @param other [Filter] other filter
+      # @return [Filter]
+      def &(other)
+        self.class.new { |event| match?(event) && other.match?(event) }
+      end
+
+      # Compose this filter with another filter using logical OR.
+      #
+      # @param other [Filter] other filter
+      # @return [Filter]
+      def |(other)
+        self.class.new { |event| match?(event) || other.match?(event) }
+      end
+    end
+  end
+end