pgoutput-client 0.2.3 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a375cce8bbb2d330974bfd22bdc0a7542c1c1ed5fe73f448e41bba717532aac2
-  data.tar.gz: 3c20c8e182548d5556d4a8a503a92d2d8c89d5c926a59eaaf537812d4bb5cdca
+  metadata.gz: 9a0c63a95d0df00f5a4b8b5a0e85b01bbec1ed71688a41f853c065e2f2b01a03
+  data.tar.gz: 3e86b0d6e62c54c03edd73a86be9dce33997a60f2d5bd097459b3a2a0d75795a
 SHA512:
-  metadata.gz: 78fc7ede75ae5fcd4b27adddef61d1f6b59ba8a26943bb44bf93d0dc5f2d97a477f36d12ec935fdf083792cbc8d37418e0b2a9f84fb7d14d15aa7e5847f8d2fd
-  data.tar.gz: 58ea9e8475789846b3caad462fcfef693df07ad8ef9cf724e877f38ba2ca53a73b3e1476971a7b5b1c9ad47bb0b8c20fae2e31f47674a377bf03c7d09cb1533c
+  metadata.gz: 0577c07284f05bb0daa59ae055a39cdd456e2104f936c33b8a921473fb5554cf65dfae2f9d9d69ce82900e27dfb1a0f632c960654bae7526b73b62ab664f6007
+  data.tar.gz: e2ea5d853aee2079d280139f58f287063fc411e8ef846782c14b8f41c286bd1975027022ecfa395619f5494ce5622d640fc0a9b6bd1a0ca9a192ebbab5701d20

data/CHANGELOG.md

@@ -2,6 +2,44 @@
 
 ## Unreleased
 
+## 0.3.0 - 2026-07-17
+
+### Added
+
+- Added `Runner#slot_status` and `SlotInspector` for version-tolerant
+  `pg_replication_slots` inspection.
+- Added immutable `SlotStatus` snapshots exposing transport and slot-health
+  fields without imposing downstream recovery policy.
+
+## 0.2.4 - 2026-06-17
+
+### Added
+
+- Added socket-aware replication stream polling.
+- Added E2E coverage for PostgreSQL restart and replication stream recovery.
+- Added connection readiness checks to E2E infrastructure.
+
+### Changed
+
+- Improved replication stream handling during idle periods.
+- Improved reconnect behavior after PostgreSQL restarts.
+- Improved standby feedback reliability during long-running streams.
+- Improved E2E test stability across PostgreSQL startup and restart scenarios.
+- Normalized `PG#get_copy_data` idle responses to simplify stream processing.
+
+### Fixed
+
+- Fixed replication stream recovery after PostgreSQL restart.
+- Fixed handling of idle COPY stream reads.
+- Fixed reconnect loops triggered by PostgreSQL replication timeouts.
+- Fixed E2E race conditions during PostgreSQL initialization.
+- Fixed replication slot creation behavior when a slot already exists.
+- Fixed several edge cases uncovered by Mammoth integration testing.
+
+### Documentation
+
+- Expanded YARD documentation coverage to 98.95%.
+
 ## 0.2.3 - 2026-06-17
 
 ### Fixed

data/README.md

@@ -58,6 +58,7 @@ Decoded row events
 - Supports `CREATE_REPLICATION_SLOT`
 - Supports `DROP_REPLICATION_SLOT`
 - Supports `START_REPLICATION SLOT ... LOGICAL ...`
+- Inspects version-dependent `pg_replication_slots` state
 - Parses XLogData envelopes
 - Parses primary keepalive messages
 - Builds standby feedback messages
@@ -175,6 +176,7 @@ It owns:
 - Keepalive handling
 - Standby status feedback
 - LSN conversion
+- Replication-slot catalog inspection
 
 ---
 
@@ -241,6 +243,45 @@ client = Pgoutput::Client::Runner.new(...)
 client.start { |payload, metadata| ... }
 ```
 
+Inspect the configured replication slot before applying a downstream recovery
+policy:
+
+```ruby
+status = client.slot_status
+
+if status.nil?
+  warn "replication slot is missing"
+else
+  puts status.restart_lsn
+  puts status.confirmed_flush_lsn
+  puts status.wal_status
+  puts status.invalidation_reason
+end
+```
+
+`SlotStatus` reflects the fields available on the connected PostgreSQL version.
+Optional fields are `nil` on versions that do not expose them. The client
+reports catalog state only; deciding whether a durable checkpoint is safe to
+resume remains the downstream runtime's responsibility.
+
+### Pgoutput::Client::SlotInspector
+
+Queries `pg_replication_slots` through a short-lived ordinary PostgreSQL
+connection. `#fetch(slot_name)` returns `nil` when the slot is missing and a
+`SlotStatus` snapshot otherwise. `Runner#slot_status` is the convenient entry
+point for inspecting the runner's configured slot.
+
+### Pgoutput::Client::SlotStatus
+
+Immutable replication-slot catalog snapshot. It exposes identity, activity,
+WAL position, retention, and invalidation fields including:
+
+- `slot_name`, `plugin`, `slot_type`, and `database`
+- `active`, `active_pid`, and `inactive_since`
+- `restart_lsn` and `confirmed_flush_lsn`
+- `wal_status`, `safe_wal_size`, and `catalog_xmin`
+- `conflicting` and `invalidation_reason`
+
 ### Pgoutput::Client::Configuration
 
 Immutable configuration object.
@@ -363,6 +404,62 @@ Equivalent Rake task:
 bundle exec rake e2e:run
 ```
 
+## Transport lifecycle behavior
+
+`pgoutput-client` owns PostgreSQL logical replication transport and lifecycle
+management. It opens the replication connection, optionally creates the logical
+replication slot, starts streaming, sends standby status feedback, and retries
+reconnectable failures.
+
+### Idle standby feedback
+
+Long-running replication streams can be quiet for long periods when no WAL
+changes are produced. During those idle periods the client wakes periodically
+and sends standby status feedback so PostgreSQL does not terminate the walsender
+for replication timeout.
+
+Control the feedback cadence with `feedback_interval`:
+
+```ruby
+runner = Pgoutput::Client::Runner.new(
+  database_url: ENV.fetch("DATABASE_URL"),
+  slot_name: "mammoth_live",
+  publication_names: ["mammoth_publication"],
+  feedback_interval: 10.0
+)
+```
+
+### Idempotent automatic slot creation
+
+When `auto_create_slot` is enabled, the client treats slot creation as
+"ensure this slot exists". Missing slots are created before streaming; existing
+slots are reused and do not cause startup failure.
+
+```ruby
+runner = Pgoutput::Client::Runner.new(
+  database_url: ENV.fetch("DATABASE_URL"),
+  slot_name: "mammoth_live",
+  publication_names: ["mammoth_publication"],
+  auto_create_slot: true,
+  temporary_slot: false
+)
+```
+
+Existence alone does not prove that an existing or newly created slot can serve
+a downstream durable checkpoint. Use `Runner#slot_status` to obtain catalog
+state and apply continuity or recovery policy in the downstream runtime before
+streaming.
+
+Publication creation remains outside this gem. Create publications through
+application migrations, database bootstrap SQL, or infrastructure tooling.
+
+### Restart recovery
+
+After a stream has connected successfully, transient PostgreSQL outages are
+retried through the reconnect lifecycle. This includes ordinary container or
+process restart windows where PostgreSQL temporarily refuses connections or
+reports that the database system is starting up.
+
 ---
 
 ## License

data/lib/pgoutput/client/connection.rb

@@ -76,14 +76,19 @@ module Pgoutput
 
       # Receive one CopyData payload from the server.
       #
-      # The call is non-blocking because the underlying `pg` call receives
-      # `false` for its blocking argument. `nil` means no complete CopyData
-      # payload is currently available.
+      # The stream must not block forever while PostgreSQL is idle, because the
+      # caller needs opportunities to send periodic standby feedback. Wait
+      # briefly for socket readability, then use the pg driver's blocking
+      # CopyData read only when data is available. `nil` means the stream is
+      # currently idle.
       #
       # @return [String, nil] raw CopyData payload or `nil`
       # @raise [ConnectionError] if receiving fails
       def get_copy_data # rubocop:disable Naming/AccessorMethodName
-        @pg_connection.get_copy_data(false)
+        return nil unless copy_data_readable?
+
+        copy_data = @pg_connection.get_copy_data(false)
+        copy_data == false ? nil : copy_data
       rescue PG::Error => e
         raise ConnectionError, e.message
       end
@@ -110,6 +115,15 @@ module Pgoutput
 
       private
 
+      def copy_data_readable?
+        return true unless @pg_connection.respond_to?(:socket_io)
+
+        socket = @pg_connection.socket_io
+        return true unless socket
+
+        !!socket.wait_readable(0.1)
+      end
+
       def exec(sql)
         @pg_connection.exec(sql)
       rescue PG::Error => e

data/lib/pgoutput/client/feedback.rb

@@ -2,6 +2,9 @@
 
 module Pgoutput
   module Client
+    # Internal immutable base class generated by `Data.define` for {Feedback}.
+    #
+    # @api private
     FeedbackData = Data.define(:received_lsn, :flushed_lsn, :applied_lsn, :client_clock, :reply_requested)
 
     # Standby status feedback message builder.

data/lib/pgoutput/client/keepalive.rb

@@ -2,6 +2,9 @@
 
 module Pgoutput
   module Client
+    # Internal immutable base class generated by `Data.define` for {Keepalive}.
+    #
+    # @api private
     KeepaliveData = Data.define(:wal_end, :server_clock, :reply_requested)
 
     # Immutable primary keepalive replication message.

data/lib/pgoutput/client/runner.rb

@@ -37,7 +37,17 @@ module Pgoutput
     # @see Stream
     # @api public
     class Runner
+      # Default number of reconnect attempts after a previously healthy stream
+      # fails. The default is intentionally large enough to survive ordinary
+      # PostgreSQL restart windows.
+      #
+      # @return [Integer]
       DEFAULT_RECONNECT_ATTEMPTS = 30
+
+      # Base reconnect backoff, in seconds. Attempt `n` sleeps for
+      # `n * DEFAULT_RECONNECT_BACKOFF`.
+      #
+      # @return [Float]
       DEFAULT_RECONNECT_BACKOFF = 0.5
 
       # Configuration used by this runner.
@@ -60,6 +70,7 @@ module Pgoutput
       #   {Configuration#initialize}
       # @return [void]
       # @raise [ConfigurationError] if the supplied configuration is invalid
+      # rubocop:disable Style/ArgumentsForwarding -- YARD needs a named parameter
       def initialize(**options)
         @configuration = Configuration.new(
           **options # : untyped
@@ -74,6 +85,7 @@ module Pgoutput
         @last_error = nil
         @reconnect_attempts = 0
       end
+      # rubocop:enable Style/ArgumentsForwarding
 
       # Start streaming raw pgoutput payloads.
       #
@@ -134,9 +146,9 @@ module Pgoutput
       #
       # @yield [payload, metadata] called once for each XLogData payload
       # @return [void]
-      def restart(&block)
+      def restart(&)
         stop
-        start(&block)
+        start(&)
       end
 
       # Whether the runner is currently inside its streaming loop.
@@ -191,6 +203,17 @@ module Pgoutput
         )
       end
 
+      # Inspect the configured replication slot through PostgreSQL's catalog.
+      #
+      # This reports transport state without applying downstream checkpoint or
+      # recovery policy.
+      #
+      # @return [SlotStatus, nil] current slot state, or `nil` when missing
+      # @raise [ConnectionError] when PostgreSQL cannot be queried
+      def slot_status
+        SlotInspector.new(database_url: configuration.database_url).fetch(configuration.slot_name)
+      end
+
       private
 
       def setup_connection(connection)
@@ -212,12 +235,12 @@ module Pgoutput
         error.message.match?(/replication slot .* already exists/i)
       end
 
-      def run_stream_cycle(configuration, &block)
+      def run_stream_cycle(configuration, &)
         connection = Connection.open(configuration)
         setup_connection(connection)
         @connected_once = true
         @stream = Stream.new(connection:, configuration:, acked_lsn: @acked_lsn)
-        @stream.start(&block)
+        @stream.start(&)
         :done
       rescue ConnectionError => e
         @last_error = e

data/lib/pgoutput/client/slot_inspector.rb

@@ -0,0 +1,74 @@
+# frozen_string_literal: true
+
+require "json"
+
+module Pgoutput
+  module Client
+    # Reads PostgreSQL replication-slot catalog state.
+    #
+    # Inspection uses a short-lived ordinary database connection because
+    # catalog queries are separate from the long-lived replication protocol
+    # connection. Querying the whole catalog row through `to_jsonb` keeps the
+    # result compatible with PostgreSQL versions that expose different optional
+    # slot-health columns.
+    #
+    # @api public
+    class SlotInspector
+      # Version-tolerant catalog query for one replication slot.
+      #
+      # @return [String]
+      QUERY = <<~SQL
+        SELECT to_jsonb(slot) AS slot
+        FROM pg_catalog.pg_replication_slots AS slot
+        WHERE slot.slot_name = $1
+      SQL
+
+      # @return [String] PostgreSQL connection URL
+      attr_reader :database_url
+
+      # @param database_url [String] PostgreSQL connection URL
+      # @param connection_factory [#call, nil] optional connection factory for tests
+      def initialize(database_url:, connection_factory: nil)
+        @database_url = database_url
+        @connection_factory = connection_factory
+      end
+
+      # Fetch the configured slot's current catalog state.
+      #
+      # @param slot_name [String] replication slot name
+      # @return [SlotStatus, nil] snapshot, or `nil` when the slot is missing
+      # @raise [ConnectionError] when PostgreSQL cannot be queried
+      def fetch(slot_name)
+        connection = open_connection
+        row = connection.exec_params(QUERY, [String(slot_name)]).first
+        return nil unless row
+
+        SlotStatus.from_catalog(parse_catalog(row.fetch("slot")))
+      rescue pg_error_class => e
+        raise ConnectionError, e.message
+      ensure
+        connection&.close unless connection&.finished?
+      end
+
+      private
+
+      def open_connection
+        return @connection_factory.call(database_url) if @connection_factory
+
+        require "pg"
+        PG.connect(database_url)
+      end
+
+      def parse_catalog(value)
+        return value if value.is_a?(Hash)
+
+        JSON.parse(value)
+      end
+
+      def pg_error_class
+        require "pg"
+        PG::Error
+      end
+    end
+  end
+end

data/lib/pgoutput/client/slot_status.rb

@@ -0,0 +1,64 @@
+# frozen_string_literal: true
+
+module Pgoutput
+  module Client
+    # Immutable snapshot of one PostgreSQL logical replication slot.
+    #
+    # Fields introduced by newer PostgreSQL versions are `nil` when the server
+    # does not expose them. The client reports transport/catalog state only;
+    # downstream runtimes remain responsible for deciding whether a checkpoint
+    # can safely resume from this slot.
+    #
+    # @api public
+    SlotStatusData = Data.define(
+      :slot_name,
+      :plugin,
+      :slot_type,
+      :database,
+      :active,
+      :active_pid,
+      :catalog_xmin,
+      :restart_lsn,
+      :confirmed_flush_lsn,
+      :wal_status,
+      :safe_wal_size,
+      :inactive_since,
+      :conflicting,
+      :invalidation_reason
+    )
+
+    # Typed replication-slot catalog snapshot.
+    #
+    # @api public
+    class SlotStatus < SlotStatusData
+      # Build a snapshot from a `pg_replication_slots` JSON object.
+      #
+      # @param attributes [Hash{String, Symbol=>Object}] catalog attributes
+      # @return [SlotStatus]
+      def self.from_catalog(attributes)
+        fetch = lambda do |key|
+          next attributes[key] if attributes.key?(key)
+
+          attributes[key.to_sym]
+        end
+
+        new(
+          slot_name: fetch.call("slot_name"),
+          plugin: fetch.call("plugin"),
+          slot_type: fetch.call("slot_type"),
+          database: fetch.call("database"),
+          active: fetch.call("active") == true,
+          active_pid: fetch.call("active_pid"),
+          catalog_xmin: fetch.call("catalog_xmin"),
+          restart_lsn: fetch.call("restart_lsn"),
+          confirmed_flush_lsn: fetch.call("confirmed_flush_lsn"),
+          wal_status: fetch.call("wal_status"),
+          safe_wal_size: fetch.call("safe_wal_size"),
+          inactive_since: fetch.call("inactive_since"),
+          conflicting: fetch.call("conflicting"),
+          invalidation_reason: fetch.call("invalidation_reason")
+        )
+      end
+    end
+  end
+end

data/lib/pgoutput/client/state.rb

@@ -2,6 +2,9 @@
 
 module Pgoutput
   module Client
+    # Internal immutable base class generated by `Data.define` for {RunnerState}.
+    #
+    # @api private
     RunnerStateData = Data.define(
       :running,
       :stop_requested,

data/lib/pgoutput/client/stream.rb

@@ -71,7 +71,7 @@ module Pgoutput
       # @raise [ProtocolError] if an unknown or malformed replication message is
       #   received
       # @raise [ConnectionError] if standby feedback cannot be sent
-      def start(&block)
+      def start(&)
         raise ArgumentError, "block required" unless block_given?
 
         @running = true
@@ -83,7 +83,7 @@ module Pgoutput
             next
           end
 
-          process_copy_data(copy_data, &block)
+          process_copy_data(copy_data, &)
           send_periodic_feedback
         end
       ensure

data/lib/pgoutput/client/version.rb

@@ -5,6 +5,6 @@ module Pgoutput
     # Current pgoutput-client gem version.
     #
     # @return [String]
-    VERSION = "0.2.3"
+    VERSION = "0.3.0"
   end
 end

data/lib/pgoutput/client/xlog_data.rb

@@ -2,6 +2,9 @@
 
 module Pgoutput
   module Client
+    # Internal immutable base class generated by `Data.define` for {XLogData}.
+    #
+    # @api private
     ReplicationXLogData = Data.define(:wal_start, :wal_end, :server_clock, :payload)
 
     # Immutable XLogData replication envelope.

data/lib/pgoutput/client.rb

@@ -8,11 +8,20 @@ require_relative "client/xlog_data"
 require_relative "client/keepalive"
 require_relative "client/feedback"
 require_relative "client/state"
+require_relative "client/slot_status"
+require_relative "client/slot_inspector"
 require_relative "client/commands"
 require_relative "client/connection"
 require_relative "client/stream"
 require_relative "client/runner"
 
+# Namespace for PostgreSQL pgoutput logical replication components.
+#
+# The top-level namespace is shared by pgoutput ecosystem gems. This gem
+# defines only the `Pgoutput::Client` transport namespace and leaves protocol
+# parsing, value decoding, and CDC normalization to sibling libraries.
+#
+# @api public
 module Pgoutput
   # Namespace for PostgreSQL logical replication transport support.
   #

data/sig/json.rbs

@@ -0,0 +1,3 @@
+module JSON
+  def self.parse: (String source) -> Hash[(String | Symbol), untyped]
+end

data/sig/pgoutput/client/connection.rbs

@@ -61,9 +61,11 @@ module Pgoutput
 
       # Receive one CopyData payload from the server.
       #
-      # The call is non-blocking because the underlying `pg` call receives
-      # `false` for its blocking argument. `nil` means no complete CopyData
-      # payload is currently available.
+      # The stream must not block forever while PostgreSQL is idle, because the
+      # caller needs opportunities to send periodic standby feedback. Wait
+      # briefly for socket readability, then use the pg driver's blocking
+      # CopyData read only when data is available. `nil` means the stream is
+      # currently idle.
       #
       # @return [String, nil] raw CopyData payload or `nil`
       # @raise [ConnectionError] if receiving fails
@@ -85,6 +87,8 @@ module Pgoutput
 
       private
 
+      def copy_data_readable?: () -> bool
+
       def exec: (untyped sql) -> untyped
     end
   end

data/sig/pgoutput/client/runner.rbs

@@ -79,6 +79,8 @@ module Pgoutput
 
       def monitor: () -> Pgoutput::Client::RunnerState
 
+      def slot_status: () -> (Pgoutput::Client::SlotStatus | nil)
+
       private
 
       def setup_connection: (untyped connection) -> untyped

data/sig/pgoutput/client/slot_inspector.rbs

@@ -0,0 +1,18 @@
+module Pgoutput
+  module Client
+    class SlotInspector
+      QUERY: String
+
+      attr_reader database_url: String
+
+      def initialize: (database_url: String, ?connection_factory: untyped) -> void
+      def fetch: (String slot_name) -> (SlotStatus | nil)
+
+      private
+
+      def open_connection: () -> untyped
+      def parse_catalog: (untyped value) -> Hash[(String | Symbol), untyped]
+      def pg_error_class: () -> singleton(::PG::Error)
+    end
+  end
+end

data/sig/pgoutput/client/slot_status.rbs

@@ -0,0 +1,41 @@
+module Pgoutput
+  module Client
+    class SlotStatusData < Data
+      attr_reader slot_name: String
+      attr_reader plugin: String?
+      attr_reader slot_type: String
+      attr_reader database: String?
+      attr_reader active: bool
+      attr_reader active_pid: Integer?
+      attr_reader catalog_xmin: String?
+      attr_reader restart_lsn: String?
+      attr_reader confirmed_flush_lsn: String?
+      attr_reader wal_status: String?
+      attr_reader safe_wal_size: Integer?
+      attr_reader inactive_since: String?
+      attr_reader conflicting: bool?
+      attr_reader invalidation_reason: String?
+
+      def self.new: (
+        slot_name: String,
+        plugin: String?,
+        slot_type: String,
+        database: String?,
+        active: bool,
+        active_pid: Integer?,
+        catalog_xmin: String?,
+        restart_lsn: String?,
+        confirmed_flush_lsn: String?,
+        wal_status: String?,
+        safe_wal_size: Integer?,
+        inactive_since: String?,
+        conflicting: bool?,
+        invalidation_reason: String?
+      ) -> instance
+    end
+
+    class SlotStatus < SlotStatusData
+      def self.from_catalog: (Hash[(String | Symbol), untyped] attributes) -> SlotStatus
+    end
+  end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: pgoutput-client
 version: !ruby/object:Gem::Version
-  version: 0.2.3
+  version: 0.3.0
 platform: ruby
 authors:
 - Ken C. Demanawa
@@ -53,11 +53,14 @@ files:
 - lib/pgoutput/client/keepalive.rb
 - lib/pgoutput/client/lsn.rb
 - lib/pgoutput/client/runner.rb
+- lib/pgoutput/client/slot_inspector.rb
+- lib/pgoutput/client/slot_status.rb
 - lib/pgoutput/client/state.rb
 - lib/pgoutput/client/stream.rb
 - lib/pgoutput/client/version.rb
 - lib/pgoutput/client/xlog_data.rb
 - lib/pgoutput_client.rb
+- sig/json.rbs
 - sig/pg.rbs
 - sig/pgoutput/client/commands.rbs
 - sig/pgoutput/client/configuration.rbs
@@ -67,6 +70,8 @@ files:
 - sig/pgoutput/client/keepalive.rbs
 - sig/pgoutput/client/lsn.rbs
 - sig/pgoutput/client/runner.rbs
+- sig/pgoutput/client/slot_inspector.rbs
+- sig/pgoutput/client/slot_status.rbs
 - sig/pgoutput/client/state.rbs
 - sig/pgoutput/client/stream.rbs
 - sig/pgoutput/client/version.rbs