pgoutput-parser 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: af80f7a3e1fadcd7bac5878116c55b4c21c8b9b4ccb5583153c13703abe2211d
+  data.tar.gz: eb474803e09cde2cc8aaae6daefb8f214461781e93d469d8684698338b230d97
+SHA512:
+  metadata.gz: cf622ea5b01013ed1005cae3958cc758126e42936e46c9e8f6af79c25daaa8d0db71a9fd51f698a8182e7870a5b208dcd4d83af4d6b6d52c4c6a230457fffa00
+  data.tar.gz: 9866a5d67ddd3463566e46cc81bff3ef433117254484bbddd962207aa2af9a1d2af5a409d578c571b9459dbcc6db37e7d3090c366f2a54e70e0da1e41b59a464

data/CHANGELOG.md

@@ -0,0 +1,117 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+### Added
+
+* Placeholder for future development.
+
+---
+
+## [0.1.0] - 2026-05-31
+
+### Added
+
+#### Protocol Support
+
+* Added support for PostgreSQL `pgoutput` logical replication protocol parsing.
+* Added support for Begin (`B`) messages.
+* Added support for Relation (`R`) messages.
+* Added support for Insert (`I`) messages.
+* Added support for Update (`U`) messages.
+* Added support for Delete (`D`) messages.
+* Added support for Commit (`C`) messages.
+* Added support for TupleData value markers:
+
+  * `n` (NULL)
+  * `u` (Unchanged TOAST)
+  * `t` (Text)
+  * `b` (Binary)
+
+#### Message Models
+
+* Added immutable protocol message classes:
+
+  * `Pgoutput::Messages::Begin`
+  * `Pgoutput::Messages::Relation`
+  * `Pgoutput::Messages::Column`
+  * `Pgoutput::Messages::TupleValue`
+  * `Pgoutput::Messages::Insert`
+  * `Pgoutput::Messages::Update`
+  * `Pgoutput::Messages::Delete`
+  * `Pgoutput::Messages::Commit`
+
+#### Parsing Infrastructure
+
+* Added `Pgoutput::BinaryParser`.
+* Added offset-based binary parsing implementation.
+* Added support for parsing PostgreSQL null-terminated strings.
+* Added support for parsing PostgreSQL integer wire types.
+* Added protocol validation and truncation detection.
+* Added parser error hierarchy.
+
+#### Relation Tracking
+
+* Added `Pgoutput::RelationTracker`.
+* Added relation metadata cache.
+* Added relation ID to column OID mapping.
+* Added tuple annotation with PostgreSQL type OIDs.
+* Added validation for unknown relation references.
+
+#### Concurrency
+
+* Added Ractor-safe message generation.
+* Added deep-shareable protocol message objects.
+* Added immutable arrays and strings throughout parsed outputs.
+
+#### Type System
+
+* Added RBS type signatures.
+* Added Steep compatibility.
+
+#### Documentation
+
+* Added YARD documentation coverage for public API.
+* Added project README.
+* Added architecture documentation.
+* Added protocol examples.
+* Added Ractor usage examples.
+
+#### Testing
+
+* Added Minitest test suite.
+* Added protocol message unit tests.
+* Added end-to-end stream integration tests.
+* Added Ractor compatibility tests.
+* Added binary payload builders for test fixtures.
+
+#### Tooling
+
+* Added Bundler project setup.
+* Added GitHub Actions CI workflow.
+* Added SimpleCov coverage support.
+* Added Rake tasks for development workflows.
+
+### Notes
+
+This release intentionally focuses on protocol parsing only.
+
+The library does not:
+
+* Manage replication connections
+* Manage replication slots
+* Track WAL positions
+* Decode PostgreSQL values into Ruby objects
+
+A future companion project (`pgoutput-decoder`) may provide PostgreSQL type decoding and higher-level row representations.
+
+---
+
+[Unreleased]: https://github.com/your-github-username/pgoutput-parser/compare/v0.1.0...HEAD
+[0.1.0]: https://github.com/your-github-username/pgoutput-parser/releases/tag/v0.1.0
+

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Kenneth C. Demanawa
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,471 @@
+# pgoutput-parser
+
+A high-performance, Ractor-safe PostgreSQL `pgoutput` logical replication protocol parser written in pure Ruby.
+
+`pgoutput-parser` parses PostgreSQL logical replication `CopyData` payloads into immutable protocol message objects. It focuses on the `pgoutput` wire format: transaction boundaries, relation metadata, DML message structure, tuple payload markers, and raw tuple bytes.
+
+It intentionally does **not** convert PostgreSQL values into application-specific Ruby objects. That belongs to a higher-level decoder layer, such as a future `pgoutput-decoder` gem.
+
+---
+
+## Requirements
+
+- Ruby 3.4+
+- PostgreSQL 10+
+
+---
+
+## Features
+
+- Pure Ruby implementation
+- Ruby 3.4+
+- Ractor-safe parsed messages
+- Immutable protocol message objects
+- PostgreSQL logical replication protocol support
+- Relation metadata tracking
+- Binary-safe tuple parsing
+- RBS type signatures included
+- YARD documentation included
+- No runtime dependencies
+
+---
+
+## Why Another pgoutput Library?
+
+This gem focuses exclusively on protocol parsing.
+
+It intentionally separates:
+
+- Protocol parsing (`pgoutput-parser`)
+- Type decoding (`pgoutput-decoder`)
+- Replication transport/client management
+
+This keeps the parser small, predictable, dependency-free, and faithful to PostgreSQL's wire format.
+
+---
+
+## Supported MVP Scope
+
+Supports the core pgoutput row-change replication messages:
+
+- Begin (`B`)
+- Relation (`R`)
+- Insert (`I`)
+- Update (`U`)
+- Delete (`D`)
+- Commit (`C`)
+
+The currently supported message formats are stable across PostgreSQL 10 through PostgreSQL 18.
+
+TupleData supports all base column markers:
+
+| Tuple Value Tag | Meaning                    |
+| --------------- | -------------------------- |
+| `n`             | NULL                       |
+| `u`             | Unchanged TOAST value      |
+| `t`             | Text-formatted raw value   |
+| `b`             | Binary-formatted raw value |
+
+### Planned Support
+
+Future releases may add support for:
+
+- Message (`M`)
+- Truncate (`T`)
+- Origin (`O`)
+- Type (`Y`)
+- Stream Start (`S`)
+- Stream Stop (`E`)
+- Stream Commit (`c`)
+- Stream Abort (`A`)
+- Two-Phase Commit messages
+
+---
+
+## What This Gem Does
+
+```text
+PostgreSQL CopyData payload
+           │
+           ▼
+    pgoutput-parser
+           │
+           ▼
+Immutable protocol messages
+```
+
+The parser understands:
+
+- Message tags and binary field sizes
+- Transaction begin metadata
+- Transaction commit metadata
+- Relation metadata
+- Column flags
+- Column names
+- PostgreSQL type OIDs
+- PostgreSQL type modifiers
+- Insert tuples
+- Update old-key tuples
+- Update old full tuples
+- Update new tuples
+- Delete old-key tuples
+- Delete old full tuples
+- Tuple value markers (`n`, `u`, `t`, `b`)
+
+---
+
+## What This Gem Does Not Do
+
+The parser does not perform application-level type decoding.
+
+It does not convert:
+
+- UUID
+- JSONB
+- Timestamp
+- Numeric
+- Array
+- Range
+- PostGIS
+- Custom PostgreSQL types
+
+Example:
+
+```ruby
+value.raw
+# => "2026-05-31 12:34:56+00"
+```
+
+The raw value is preserved exactly as received.
+
+A higher-level decoder may later interpret it.
+
+---
+
+## Non-goals
+
+This project intentionally does not:
+
+- Manage replication slots
+- Open replication connections
+- Maintain WAL positions
+- Reconnect to PostgreSQL
+- Decode PostgreSQL types
+- Integrate with ActiveRecord
+- Publish events
+- Build CDC pipelines
+
+Its sole responsibility is parsing pgoutput protocol messages.
+
+---
+
+## Installation
+
+Add this line to your Gemfile:
+
+```ruby
+gem "pgoutput-parser"
+```
+
+Then run:
+
+```bash
+bundle install
+```
+
+Require the library:
+
+```ruby
+require "pgoutput"
+```
+
+---
+
+## Quick Start
+
+```ruby
+require "pgoutput"
+
+stream = Pgoutput::RelationTracker.new
+
+stream.process(relation_payload)
+
+insert = stream.process(insert_payload)
+
+insert.relation_id
+# => 42
+
+insert.tuple.first.raw
+# => "7"
+
+insert.tuple.first.oid
+# => 23
+```
+
+---
+
+## Binary Tuple Values
+
+When PostgreSQL publishes tuple values using binary format (`b`), the parser preserves the raw bytes exactly as received.
+
+```ruby
+value.raw
+# => "\x00\x00\x00\x07".b
+```
+
+The parser does not interpret binary values.
+
+---
+
+## Update Messages
+
+PostgreSQL `Update` messages may contain:
+
+- No old tuple
+- An old key tuple (`K`)
+- An old full tuple (`O`)
+
+They always contain a new tuple (`N`).
+
+```ruby
+update = stream.process(update_payload)
+
+update.old_key_tuple
+# => [Pgoutput::Messages::TupleValue, ...] or nil
+
+update.old_tuple
+# => [Pgoutput::Messages::TupleValue, ...] or nil
+
+update.new_tuple
+# => [Pgoutput::Messages::TupleValue, ...]
+```
+
+### Update Tuple Example
+
+```ruby
+update = stream.process(update_payload)
+
+update.old_key_tuple
+update.old_tuple
+update.new_tuple
+```
+
+---
+
+## Delete Messages
+
+PostgreSQL `Delete` messages contain either:
+
+- An old key tuple (`K`)
+- An old full tuple (`O`)
+
+```ruby
+delete = stream.process(delete_payload)
+
+delete.old_key_tuple
+# => [Pgoutput::Messages::TupleValue, ...] or nil
+
+delete.old_tuple
+# => [Pgoutput::Messages::TupleValue, ...] or nil
+```
+
+---
+
+## Relation Metadata Tracking
+
+`RelationTracker` keeps a local relation cache so tuple values can be associated with PostgreSQL column OIDs defined by preceding Relation (`R`) messages.
+
+No type conversion is performed.
+
+Only protocol metadata is attached.
+
+```ruby
+stream.process(relation_payload)
+
+message = stream.process(update_payload)
+
+message.new_tuple.map(&:oid)
+# => [23, 25, 16]
+```
+
+The relation tracker itself is stateful and maintains relation metadata encountered in the replication stream.
+
+---
+
+## Ractor Safety
+
+```ruby
+message = stream.process(update_payload)
+
+Ractor.shareable?(message)
+# => true
+```
+
+Passing parsed messages to a Ractor:
+
+```ruby
+message = stream.process(update_payload)
+
+result = Ractor.new(message) do |update|
+  update.new_tuple.map(&:raw)
+end.take
+```
+
+---
+
+## Architecture
+
+```text
+PostgreSQL
+      │
+      ▼
+CopyData payload
+      │
+      ▼
+Pgoutput::BinaryParser
+      │
+      ▼
+Parsed protocol message
+      │
+      ▼
+Pgoutput::RelationTracker
+      │
+      ▼
+Protocol message with relation metadata
+      │
+      ▼
+Ractor-safe protocol message
+```
+
+---
+
+## Public API
+
+### Pgoutput::BinaryParser
+
+Parses a single pgoutput payload without stream state.
+
+```ruby
+message = Pgoutput::BinaryParser.new(payload).parse
+```
+
+### Pgoutput::RelationTracker
+
+Parses messages in stream order and remembers relation metadata.
+
+```ruby
+stream = Pgoutput::RelationTracker.new
+
+stream.process(relation_payload)
+
+message = stream.process(insert_payload)
+```
+
+### Optional Usage
+
+`RelationTracker` is optional.
+
+If relation metadata tracking is not required, payloads can be parsed directly:
+
+```ruby
+message =
+  Pgoutput::BinaryParser
+    .new(payload)
+    .parse
+```
+
+---
+
+## RelationTracker Lifecycle
+
+A `RelationTracker` should be created per logical replication stream.
+
+```ruby
+stream = Pgoutput::RelationTracker.new
+```
+
+The tracker maintains relation metadata discovered during the stream and therefore should not be reused across unrelated replication sessions.
+
+---
+
+## Message Classes
+
+```ruby
+Pgoutput::Messages::Begin
+Pgoutput::Messages::Relation
+Pgoutput::Messages::Column
+Pgoutput::Messages::TupleValue
+Pgoutput::Messages::Insert
+Pgoutput::Messages::Update
+Pgoutput::Messages::Delete
+Pgoutput::Messages::Commit
+```
+
+---
+
+## Type Signatures
+
+RBS signatures are included:
+
+```text
+sig/pgoutput.rbs
+```
+
+Run Steep:
+
+```bash
+bundle exec steep check
+```
+
+---
+
+## Testing
+
+Run all tests:
+
+```bash
+bundle exec rake test
+```
+
+Run with coverage:
+
+```bash
+COVERAGE=true bundle exec rake test
+```
+
+---
+
+## Development
+
+Generate YARD documentation:
+
+```bash
+bundle exec yard doc
+```
+
+---
+
+## Ecosystem Direction
+
+This gem is the protocol layer.
+
+```text
+pgoutput-parser
+      │
+      ▼
+Protocol messages
+      │
+      ▼
+pgoutput-decoder
+      │
+      ▼
+Application objects
+```
+
+`pgoutput-parser` should remain small, dependency-free, binary-safe, and faithful to PostgreSQL's wire format.
+
+---
+
+## License
+
+MIT.

data/lib/pgoutput/binary_parser.rb

@@ -0,0 +1,171 @@
+# frozen_string_literal: true
+
+module Pgoutput
+  # Offset-based binary parser for one PostgreSQL pgoutput logical replication
+  # message payload.
+  #
+  # A parser instance is intentionally short-lived and mutable while reading a
+  # single payload. Its returned message object is deeply frozen/shareable and may
+  # cross Ractor boundaries safely.
+  #
+  # @api public
+  class BinaryParser
+    # @param payload [String] one pgoutput message payload from a CopyData frame.
+    # @return [void]
+    def initialize(payload)
+      @payload = payload.b
+      @offset = 0
+    end
+
+    # Parse one supported pgoutput message.
+    #
+    # Supported MVP tags are `B`, `R`, `I`, `U`, `D`, and `C`.
+    #
+    # @return [Pgoutput::Messages::Begin, Pgoutput::Messages::Relation,
+    #   Pgoutput::Messages::Insert, Pgoutput::Messages::Update,
+    #   Pgoutput::Messages::Delete, Pgoutput::Messages::Commit]
+    # @raise [UnsupportedMessageError] if the message tag is unsupported.
+    # @raise [TruncatedMessageError] if the payload is incomplete.
+    def parse
+      case read_byte_chr
+      when "B" then parse_begin
+      when "R" then parse_relation
+      when "I" then parse_insert
+      when "U" then parse_update
+      when "D" then parse_delete
+      when "C" then parse_commit
+      else
+        raise UnsupportedMessageError, "unsupported pgoutput message tag"
+      end
+    end
+
+    private
+
+    def parse_begin
+      share(Messages::Begin.new(read_uint64, read_uint64, read_uint32))
+    end
+
+    def parse_relation
+      relation_id = read_uint32
+      schema = read_cstring
+      table = read_cstring
+      replica_identity = read_uint8
+      column_count = read_uint16
+
+      columns = Array.new(column_count) do
+        Messages::Column.new(read_uint8, read_cstring, read_uint32, read_int32)
+      end.freeze
+
+      share(Messages::Relation.new(relation_id, schema, table, replica_identity, columns))
+    end
+
+    def parse_insert
+      relation_id = read_uint32
+      tuple_tag = read_byte_chr
+      raise UnsupportedMessageError, "expected insert tuple tag N, got #{tuple_tag.inspect}" unless tuple_tag == "N"
+
+      share(Messages::Insert.new(relation_id, parse_tuple_data))
+    end
+
+    def parse_update
+      relation_id = read_uint32
+      old_key_tuple = nil
+      old_tuple = nil
+
+      first_tag = read_byte_chr
+      case first_tag
+      when "K"
+        old_key_tuple = parse_tuple_data
+        new_tag = read_byte_chr
+      when "O"
+        old_tuple = parse_tuple_data
+        new_tag = read_byte_chr
+      when "N"
+        new_tag = first_tag
+      else
+        raise UnsupportedMessageError, "expected update tuple tag K, O, or N, got #{first_tag.inspect}"
+      end
+
+      raise UnsupportedMessageError, "expected update new tuple tag N, got #{new_tag.inspect}" unless new_tag == "N"
+
+      share(Messages::Update.new(relation_id, old_key_tuple, old_tuple, parse_tuple_data))
+    end
+
+    def parse_delete
+      relation_id = read_uint32
+      tuple_tag = read_byte_chr
+
+      case tuple_tag
+      when "K"
+        share(Messages::Delete.new(relation_id, parse_tuple_data, nil))
+      when "O"
+        share(Messages::Delete.new(relation_id, nil, parse_tuple_data))
+      else
+        raise UnsupportedMessageError, "expected delete tuple tag K or O, got #{tuple_tag.inspect}"
+      end
+    end
+
+    def parse_commit
+      share(Messages::Commit.new(read_uint8, read_uint64, read_uint64, read_uint64))
+    end
+
+    def parse_tuple_data
+      column_count = read_uint16
+
+      Array.new(column_count) do
+        tag = read_byte_chr
+
+        case tag
+        when "n"
+          Messages::TupleValue.new(:null, nil, nil)
+        when "u"
+          Messages::TupleValue.new(:unchanged_toast, nil, nil)
+        when "t", "b"
+          raw = read_bytes(read_int32).freeze
+          Messages::TupleValue.new(tag == "t" ? :text : :binary, raw, nil)
+        else
+          raise UnsupportedMessageError, "unsupported tuple data tag: #{tag.inspect}"
+        end
+      end.freeze
+    end
+
+    def read_uint8 = read_bytes(1).unpack1("C")
+
+    def read_uint16 = read_bytes(2).unpack1("n")
+
+    def read_uint32 = read_bytes(4).unpack1("N")
+
+    def read_int32
+      value = read_uint32
+      value >= 0x8000_0000 ? value - 0x1_0000_0000 : value
+    end
+
+    def read_uint64 = read_bytes(8).unpack1("Q>")
+
+    def read_byte_chr = read_bytes(1)
+
+    def read_cstring
+      zero = @payload.index("\0", @offset)
+      raise TruncatedMessageError, "unterminated cstring at offset #{@offset}" unless zero
+
+      value = @payload.byteslice(@offset, zero - @offset).freeze
+      @offset = zero + 1
+      value
+    end
+
+    def read_bytes(length)
+      raise TruncatedMessageError, "negative byte length #{length}" if length.negative?
+      if @offset + length > @payload.bytesize
+        raise TruncatedMessageError, "need #{length} bytes at offset #{@offset}, payload has #{@payload.bytesize} bytes"
+      end
+
+      value = @payload.byteslice(@offset, length)
+      @offset += length
+      value
+    end
+
+    def share(message)
+      Ractor.make_shareable(message)
+    end
+  end
+end

data/lib/pgoutput/errors.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+module Pgoutput
+  # Base error for all parser failures.
+  #
+  # @api public
+  class Error < StandardError; end
+
+  # Raised when a payload ends before the requested protocol field can be read.
+  #
+  # @api public
+  class TruncatedMessageError < Error; end
+
+  # Raised when the parser sees a message or tuple tag outside this MVP scope.
+  #
+  # @api public
+  class UnsupportedMessageError < Error; end
+
+  # Raised when row data references a relation id that has not been observed via
+  # a preceding Relation (`R`) message in the current stream decoder.
+  #
+  # @api public
+  class UnknownRelationError < Error; end
+end

data/lib/pgoutput/messages.rb

@@ -0,0 +1,106 @@
+# frozen_string_literal: true
+
+module Pgoutput
+  # Immutable message model classes for the PostgreSQL pgoutput protocol.
+  #
+  # Every value returned by the parser is deeply shareable via
+  # {Ractor.make_shareable}. These classes are protocol-level structures only;
+  # they preserve tuple bytes and metadata but do not convert PostgreSQL values
+  # into application-specific Ruby types.
+  #
+  # @api public
+  module Messages
+    # Transaction begin message.
+    #
+    # @!attribute [r] final_lsn
+    #   @return [Integer] final transaction LSN.
+    # @!attribute [r] commit_timestamp
+    #   @return [Integer] microseconds since PostgreSQL epoch.
+    # @!attribute [r] xid
+    #   @return [Integer] transaction id.
+    Begin = Data.define(:final_lsn, :commit_timestamp, :xid)
+
+    # Relation column metadata.
+    #
+    # @!attribute [r] flags
+    #   @return [Integer] column flags; key columns use flag 1.
+    # @!attribute [r] name
+    #   @return [String] column name.
+    # @!attribute [r] oid
+    #   @return [Integer] PostgreSQL type OID.
+    # @!attribute [r] type_modifier
+    #   @return [Integer] PostgreSQL type modifier.
+    Column = Data.define(:flags, :name, :oid, :type_modifier)
+
+    # Relation metadata message.
+    #
+    # @!attribute [r] relation_id
+    #   @return [Integer] relation OID used by later DML messages.
+    # @!attribute [r] schema
+    #   @return [String] namespace name.
+    # @!attribute [r] table
+    #   @return [String] relation name.
+    # @!attribute [r] replica_identity
+    #   @return [Integer] relation replica identity setting.
+    # @!attribute [r] columns
+    #   @return [Array<Column>] immutable column metadata.
+    Relation = Data.define(:relation_id, :schema, :table, :replica_identity, :columns)
+
+    # One tuple column value.
+    #
+    # @!attribute [r] format
+    #   @return [:null, :unchanged_toast, :text, :binary] protocol value format.
+    # @!attribute [r] raw
+    #   @return [String, nil] immutable raw payload, or nil for NULL/TOAST markers.
+    # @!attribute [r] oid
+    #   @return [Integer, nil] PostgreSQL type OID when relation metadata is known.
+    TupleValue = Data.define(:format, :raw, :oid)
+
+    # Insert DML message.
+    #
+    # @!attribute [r] relation_id
+    #   @return [Integer] relation OID.
+    # @!attribute [r] tuple
+    #   @return [Array<TupleValue>] new tuple data.
+    Insert = Data.define(:relation_id, :tuple)
+
+    # Update DML message.
+    #
+    # The message may contain either an old key tuple, an old full tuple, or
+    # neither; it always contains a new tuple.
+    #
+    # @!attribute [r] relation_id
+    #   @return [Integer] relation OID.
+    # @!attribute [r] old_key_tuple
+    #   @return [Array<TupleValue>, nil] replica identity key tuple.
+    # @!attribute [r] old_tuple
+    #   @return [Array<TupleValue>, nil] full old tuple when replica identity is FULL.
+    # @!attribute [r] new_tuple
+    #   @return [Array<TupleValue>] new tuple data.
+    Update = Data.define(:relation_id, :old_key_tuple, :old_tuple, :new_tuple)
+
+    # Delete DML message.
+    #
+    # The message contains either an old key tuple or an old full tuple.
+    #
+    # @!attribute [r] relation_id
+    #   @return [Integer] relation OID.
+    # @!attribute [r] old_key_tuple
+    #   @return [Array<TupleValue>, nil] replica identity key tuple.
+    # @!attribute [r] old_tuple
+    #   @return [Array<TupleValue>, nil] full old tuple when replica identity is FULL.
+    Delete = Data.define(:relation_id, :old_key_tuple, :old_tuple)
+
+    # Transaction commit message.
+    #
+    # @!attribute [r] flags
+    #   @return [Integer] commit flags; currently unused by PostgreSQL.
+    # @!attribute [r] commit_lsn
+    #   @return [Integer] commit LSN.
+    # @!attribute [r] transaction_end_lsn
+    #   @return [Integer] transaction end LSN.
+    # @!attribute [r] commit_timestamp
+    #   @return [Integer] microseconds since PostgreSQL epoch.
+    Commit = Data.define(:flags, :commit_lsn, :transaction_end_lsn, :commit_timestamp)
+  end
+end

data/lib/pgoutput/relation_tracker.rb

@@ -0,0 +1,109 @@
+# frozen_string_literal: true
+
+module Pgoutput
+  # Stateful relation tracker for pgoutput message sequences.
+  #
+  # The relation tracker remembers Relation (`R`) messages so DML tuple values can
+  # be annotated with PostgreSQL type OIDs. It does not decode or convert values.
+  # It only adds protocol metadata to tuple values while keeping returned objects
+  # deeply shareable.
+  #
+  # The instance contains mutable relation-cache state and should not be shared
+  # across Ractors. Returned message objects are Ractor-safe.
+  #
+  # @api public
+  class RelationTracker
+    # @return [void]
+    def initialize
+      @relations = {}
+    end
+
+    # Process one pgoutput payload in stream order.
+    #
+    # @param payload [String] one pgoutput logical replication message payload.
+    # @return [Pgoutput::Messages::Begin, Pgoutput::Messages::Relation,
+    #   Pgoutput::Messages::Insert, Pgoutput::Messages::Update,
+    #   Pgoutput::Messages::Delete, Pgoutput::Messages::Commit]
+    # @raise [UnknownRelationError] if DML references an unseen relation id.
+    def process(payload)
+      message = BinaryParser.new(payload).parse
+
+      case message
+      when Messages::Relation
+        @relations[message.relation_id] = message
+        message
+      when Messages::Insert
+        annotate_insert(message)
+      when Messages::Update
+        annotate_update(message)
+      when Messages::Delete
+        annotate_delete(message)
+      else
+        message
+      end
+    end
+
+    # Backwards-compatible alias for callers migrating from RelationTracker.
+    #
+    # @param payload [String] one pgoutput logical replication message payload.
+    # @return [Pgoutput::Messages::Begin, Pgoutput::Messages::Relation,
+    #   Pgoutput::Messages::Insert, Pgoutput::Messages::Update,
+    #   Pgoutput::Messages::Delete, Pgoutput::Messages::Commit]
+    def decode(payload)
+      process(payload)
+    end
+
+    private
+
+    def annotate_insert(message)
+      relation = relation_for(message.relation_id)
+
+      Ractor.make_shareable(
+        Messages::Insert.new(
+          message.relation_id,
+          annotate_tuple(message.tuple, relation)
+        )
+      )
+    end
+
+    def annotate_update(message)
+      relation = relation_for(message.relation_id)
+
+      Ractor.make_shareable(
+        Messages::Update.new(
+          message.relation_id,
+          annotate_tuple(message.old_key_tuple, relation),
+          annotate_tuple(message.old_tuple, relation),
+          annotate_tuple(message.new_tuple, relation)
+        )
+      )
+    end
+
+    def annotate_delete(message)
+      relation = relation_for(message.relation_id)
+
+      Ractor.make_shareable(
+        Messages::Delete.new(
+          message.relation_id,
+          annotate_tuple(message.old_key_tuple, relation),
+          annotate_tuple(message.old_tuple, relation)
+        )
+      )
+    end
+
+    def annotate_tuple(tuple, relation)
+      return nil if tuple.nil?
+
+      tuple.each_with_index.map do |value, index|
+        column = relation.columns[index]
+        Messages::TupleValue.new(value.format, value.raw, column&.oid)
+      end.freeze
+    end
+
+    def relation_for(relation_id)
+      @relations.fetch(relation_id) do
+        raise UnknownRelationError, "unknown relation id #{relation_id}; parse Relation message first"
+      end
+    end
+  end
+end

data/lib/pgoutput/version.rb

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+
+module Pgoutput
+  # Current gem version.
+  #
+  # @return [String] semantic version string.
+  VERSION = "0.1.0"
+end

data/lib/pgoutput.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+require_relative "pgoutput/version"
+require_relative "pgoutput/errors"
+require_relative "pgoutput/messages"
+require_relative "pgoutput/binary_parser"
+require_relative "pgoutput/relation_tracker"
+
+# Top-level namespace for pgoutput-parser.
+#
+# pgoutput-parser parses PostgreSQL `pgoutput` logical replication protocol
+# payloads into immutable Ruby protocol message objects. The namespace is kept
+# short as `Pgoutput`, while the RubyGems package name is `pgoutput-parser`.
+#
+# @api public
+module Pgoutput
+end

data/sig/pgoutput.rbs

@@ -0,0 +1,129 @@
+module Pgoutput
+  VERSION: String
+
+  type message =
+    Messages::Begin |
+    Messages::Relation |
+    Messages::Insert |
+    Messages::Update |
+    Messages::Delete |
+    Messages::Commit
+
+  class Error < StandardError
+  end
+
+  class TruncatedMessageError < Error
+  end
+
+  class UnsupportedMessageError < Error
+  end
+
+  class UnknownRelationError < Error
+  end
+
+  module Messages
+    class Begin < Data
+      attr_reader final_lsn: Integer
+      attr_reader commit_timestamp: Integer
+      attr_reader xid: Integer
+
+      def self.new: (Integer final_lsn, Integer commit_timestamp, Integer xid) -> Begin
+    end
+
+    class Column < Data
+      attr_reader flags: Integer
+      attr_reader name: String
+      attr_reader oid: Integer
+      attr_reader type_modifier: Integer
+
+      def self.new: (Integer flags, String name, Integer oid, Integer type_modifier) -> Column
+    end
+
+    class Relation < Data
+      attr_reader relation_id: Integer
+      attr_reader schema: String
+      attr_reader table: String
+      attr_reader replica_identity: Integer
+      attr_reader columns: Array[Column]
+
+      def self.new: (
+        Integer relation_id,
+        String schema,
+        String table,
+        Integer replica_identity,
+        Array[Column] columns
+      ) -> Relation
+    end
+
+    class TupleValue < Data
+      attr_reader format: :null | :unchanged_toast | :text | :binary
+      attr_reader raw: String?
+      attr_reader oid: Integer?
+
+      def self.new: (
+        :null | :unchanged_toast | :text | :binary format,
+        String? raw,
+        Integer? oid
+      ) -> TupleValue
+    end
+
+    class Insert < Data
+      attr_reader relation_id: Integer
+      attr_reader tuple: Array[TupleValue]
+
+      def self.new: (Integer relation_id, Array[TupleValue] tuple) -> Insert
+    end
+
+    class Update < Data
+      attr_reader relation_id: Integer
+      attr_reader old_key_tuple: Array[TupleValue]?
+      attr_reader old_tuple: Array[TupleValue]?
+      attr_reader new_tuple: Array[TupleValue]
+
+      def self.new: (
+        Integer relation_id,
+        Array[TupleValue]? old_key_tuple,
+        Array[TupleValue]? old_tuple,
+        Array[TupleValue] new_tuple
+      ) -> Update
+    end
+
+    class Delete < Data
+      attr_reader relation_id: Integer
+      attr_reader old_key_tuple: Array[TupleValue]?
+      attr_reader old_tuple: Array[TupleValue]?
+
+      def self.new: (
+        Integer relation_id,
+        Array[TupleValue]? old_key_tuple,
+        Array[TupleValue]? old_tuple
+      ) -> Delete
+    end
+
+    class Commit < Data
+      attr_reader flags: Integer
+      attr_reader commit_lsn: Integer
+      attr_reader transaction_end_lsn: Integer
+      attr_reader commit_timestamp: Integer
+
+      def self.new: (
+        Integer flags,
+        Integer commit_lsn,
+        Integer transaction_end_lsn,
+        Integer commit_timestamp
+      ) -> Commit
+    end
+  end
+
+  class BinaryParser
+    def initialize: (String payload) -> void
+
+    def parse: () -> message
+  end
+
+  class RelationTracker
+    def initialize: () -> void
+
+    def process: (String payload) -> message
+  end
+end

metadata

@@ -0,0 +1,153 @@
+--- !ruby/object:Gem::Specification
+name: pgoutput-parser
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Ken C. Demanawa
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.27'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.27'
+- !ruby/object:Gem::Dependency
+  name: pry
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.16.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.16.0
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.4'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.4'
+- !ruby/object:Gem::Dependency
+  name: rubocop
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.87'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.87'
+- !ruby/object:Gem::Dependency
+  name: simplecov
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.22'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.22'
+- !ruby/object:Gem::Dependency
+  name: steep
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.10'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.10'
+- !ruby/object:Gem::Dependency
+  name: yard
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.9.44
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.9.44
+description: A pure Ruby parser for PostgreSQL pgoutput logical replication CopyData
+  payloads.
+email:
+- kenneth.c.demanawa@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE.txt
+- README.md
+- lib/pgoutput.rb
+- lib/pgoutput/binary_parser.rb
+- lib/pgoutput/errors.rb
+- lib/pgoutput/messages.rb
+- lib/pgoutput/relation_tracker.rb
+- lib/pgoutput/version.rb
+- sig/pgoutput.rbs
+homepage: https://github.com/kanutocd/pgoutput-parser
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/kanutocd/pgoutput-parser
+  source_code_uri: https://github.com/kanutocd/pgoutput-parser
+  changelog_uri: https://github.com/kanutocd/pgoutput-parser/CHANGELOG.md
+  rubygems_mfa_required: 'true'
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.4'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.6.9
+specification_version: 4
+summary: Ractor-safe PostgreSQL pgoutput logical replication protocol parser.
+test_files: []