pgoutput-client 0.1.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3dd08a56a5b98573babde8d5aa72e02c44668877803a5272006879c28bd69ebd
-  data.tar.gz: c385343185a60a6304ecc276b5f6fac207cb5791fb8035823d863620b238ebda
+  metadata.gz: 697aed85b1c507b7c0e6707bdff020075f31c35c1be6f74a3dff9a060987d69e
+  data.tar.gz: ae49847c1b076c79754149cf3b7db0f32ee14f487a1c1ab03531b4a37fc42ab8
 SHA512:
-  metadata.gz: 687bd8396cdf3b7a9e019c6cf2c7d584cfa2dcd273dcb5a4c662079d059c03aa2c55b487f9b1d8f58cc82ebfc36421d9d65c774f1fbb6ec6e65ba811418cb82e
-  data.tar.gz: 8d5e7c7a2b172084071006ca893e85fcff841539919bab78065a5a70d61fed89929dcd469b0837e959a164a7c3c882734bce24405c2660400c1f06b159e56401
+  metadata.gz: 326686eeecf787c33b5ed9daa8d0491fa441a17ac9936666b7e130fba67a1fdaa6b136a86ed903398fd6d2c48e4037199763e03d8390bd5671184e28311eb5f6
+  data.tar.gz: 68290690628bef4393ef04867fdfd63337159e85505943062960173a9e5c0eaffb2b00b18eec03372f8d4cba4f02cb2601166eb2d25d98fd0cf6e0ebac387ff9

data/CHANGELOG.md

@@ -1,13 +1,19 @@
 # Changelog
 
-All notable changes to this project will be documented in this file.
+## Unreleased
 
-The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
-and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [0.2.0] - 2026-06-16
 
-### Added
+### Fixed
 
-- Placeholder for future development.
+- Remove the configurable plugin surface; this transport layer is fixed to `pgoutput`.
+- Wire `Pgoutput::Client::Runner#stop` to the active stream for cooperative shutdown.
+- Back off briefly when the replication socket has no `CopyData` ready instead of busy polling.
+- Retry live stream connection loss with backoff and resume from the latest confirmed WAL position.
+- Avoid recreating an existing replication slot during reconnect attempts.
+- Document the Docker-backed E2E workflow in the README and test skip hint.
+- Document that replay, checkpointing, deduplication, and sink ordering belong to downstream layers.
+- Add a live E2E reconnect test that restarts PostgreSQL mid-stream and verifies resume behavior.
 
 ---
 

data/README.md

@@ -2,7 +2,6 @@
 
 [![Gem Version](https://badge.fury.io/rb/pgoutput-client.svg)](https://badge.fury.io/rb/pgoutput-client)
 [![CI](https://github.com/kanutocd/pgoutput-client/workflows/CI/badge.svg)](https://github.com/kanutocd/pgoutput-client/actions)
-[![Coverage Status](https://codecov.io/gh/kanutocd/pgoutput-client/branch/main/graph/badge.svg)](https://codecov.io/gh/kanutocd/pgoutput-client)
 [![Ruby Version](https://img.shields.io/badge/ruby-%3E%3D%203.4-ruby.svg)](https://www.ruby-lang.org/en/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
@@ -190,8 +189,15 @@ It does not:
 - Run processor pipelines
 - Manage Ractor worker pools
 - Store audit records
+- Own replay, checkpointing, deduplication, or sink ordering
 
-Those responsibilities belong to higher layers.
+Those responsibilities belong to higher layers, especially `cdc-core` and the sink that materializes downstream state.
+
+## Failure Semantics
+
+If the live replication stream loses its connection, `pgoutput-client` retries a small number of times with a backoff and resumes from the latest confirmed WAL position.
+
+It does not decide replay policy, deduplication strategy, checkpoint storage, or exactly-once delivery. Those concerns belong to the downstream CDC runtime and sink layer.
 
 ---
 
@@ -329,8 +335,36 @@ bundle exec rbs:validate
 bundle exec rake yard
 ```
 
+### End-to-End PostgreSQL
+
+Run the full Docker-backed E2E flow and clean up afterward:
+
+```bash
+script/test-e2e
+```
+
+Keep PostgreSQL running after the test for debugging:
+
+```bash
+KEEP_E2E_POSTGRES=1 script/test-e2e
+```
+
+You can also run the steps manually:
+
+```bash
+script/e2e-up
+PGOUTPUT_CLIENT_E2E=1 bundle exec rake test:e2e
+script/e2e-down
+```
+
+Equivalent Rake task:
+
+```bash
+bundle exec rake e2e:run
+```
+
 ---
 
 ## License
 
-MIT.
+[MIT](LICENSE.txt).

data/lib/pgoutput/client/commands.rb

@@ -26,7 +26,7 @@ module Pgoutput
       # @return [String] SQL command suitable for `PG::Connection#exec`
       def create_replication_slot(configuration)
         temporary = configuration.temporary_slot ? " TEMPORARY" : ""
-        "CREATE_REPLICATION_SLOT #{configuration.slot_name}#{temporary} LOGICAL #{configuration.plugin}"
+        "CREATE_REPLICATION_SLOT #{configuration.slot_name}#{temporary} LOGICAL #{Configuration::DEFAULT_PLUGIN}"
       end
 
       # Render a `DROP_REPLICATION_SLOT` command.

data/lib/pgoutput/client/configuration.rb

@@ -31,7 +31,7 @@ module Pgoutput
     #
     # @api public
     class Configuration
-      # Default logical decoding output plugin.
+      # Fixed logical decoding output plugin.
       #
       # @return [String]
       DEFAULT_PLUGIN = "pgoutput"
@@ -58,9 +58,6 @@ module Pgoutput
       # @!attribute [r] start_lsn
       #   Optional normalized starting LSN.
       #   @return [String, nil]
-      # @!attribute [r] plugin
-      #   Logical decoding output plugin name.
-      #   @return [String]
       # @!attribute [r] proto_version
       #   pgoutput protocol version.
       #   @return [Integer]
@@ -83,7 +80,6 @@ module Pgoutput
                   :slot_name,
                   :publication_names,
                   :start_lsn,
-                  :plugin,
                   :proto_version,
                   :binary,
                   :messages,
@@ -107,7 +103,6 @@ module Pgoutput
       #   names to pass to pgoutput
       # @param start_lsn [String, Integer, nil] starting LSN as a PostgreSQL LSN
       #   string, an integer WAL position, or `nil` for `0/0`
-      # @param plugin [#to_s] logical decoding plugin name
       # @param proto_version [#to_int, #to_s] pgoutput protocol version
       # @param binary [Object] truthy to request binary column values
       # @param messages [Object] truthy to request logical decoding messages
@@ -126,7 +121,6 @@ module Pgoutput
                      slot_name:,
                      publication_names:,
                      start_lsn: nil,
-                     plugin: DEFAULT_PLUGIN,
                      proto_version: DEFAULT_PROTO_VERSION,
                      binary: false,
                      messages: false,
@@ -139,7 +133,6 @@ module Pgoutput
           validate_identifier(name, "publication_name").freeze
         end.freeze
         @start_lsn = normalize_lsn(start_lsn).freeze
-        @plugin = String(plugin).freeze
         @proto_version = Integer(proto_version)
         @binary = boolean(binary, "binary")
         @messages = boolean(messages, "messages")

data/lib/pgoutput/client/runner.rb

@@ -0,0 +1,258 @@
+# frozen_string_literal: true
+
+module Pgoutput
+  module Client
+    # High-level logical replication client facade.
+    #
+    # `Runner` is the simplest public entry point for consumers that want to
+    # stream raw pgoutput payloads without manually managing a replication
+    # connection. It owns the connection lifecycle for one logical replication
+    # stream:
+    #
+    # 1. Build an immutable {Configuration} from keyword arguments.
+    # 2. Open a PostgreSQL replication connection.
+    # 3. Optionally create the configured replication slot.
+    # 4. Start logical replication.
+    # 5. Yield raw pgoutput payload bytes and {XLogData} metadata.
+    # 6. Close the connection when streaming exits.
+    #
+    # If a live stream loses its connection, the runner retries a small number
+    # of times with a backoff and resumes from the latest confirmed WAL
+    # position. Replay, checkpointing, and deduplication are not owned here;
+    # those concerns belong to the downstream CDC runtime and sink layer.
+    #
+    # @example Stream raw pgoutput messages
+    #   runner = Pgoutput::Client::Runner.new(
+    #     database_url: "postgres://localhost/app",
+    #     slot_name: "cdc_slot",
+    #     publication_names: ["app_publication"]
+    #   )
+    #
+    #   runner.start do |payload, metadata|
+    #     puts "received #{payload.bytesize} bytes at #{metadata.wal_end_lsn}"
+    #   end
+    #
+    # @see Configuration
+    # @see Connection
+    # @see Stream
+    # @api public
+    class Runner
+      DEFAULT_RECONNECT_ATTEMPTS = 3
+      DEFAULT_RECONNECT_BACKOFF = 0.5
+
+      # Configuration used by this runner.
+      #
+      # @return [Configuration]
+      attr_reader :configuration
+
+      # Last transport error seen by the runner.
+      #
+      # @return [Exception, nil]
+      attr_reader :last_error
+
+      # Build a runner from configuration keyword arguments.
+      #
+      # The accepted keywords are the same as {Configuration#initialize}. The
+      # resulting configuration object is immutable and reused for the lifetime
+      # of this runner.
+      #
+      # @param options [Hash{Symbol=>Object}] configuration options forwarded to
+      #   {Configuration#initialize}
+      # @return [void]
+      # @raise [ConfigurationError] if the supplied configuration is invalid
+      def initialize(**options)
+        @configuration = Configuration.new(
+          **options # : untyped
+        )
+        @stopped = true
+        @running = false
+        @stream = nil
+        @resume_lsn = configuration.start_lsn
+        @acked_lsn = configuration.start_lsn
+        @slot_created = false
+        @last_error = nil
+        @reconnect_attempts = 0
+      end
+
+      # Start streaming raw pgoutput payloads.
+      #
+      # This method blocks until the stream stops or the underlying connection
+      # raises an error. The yielded payload is the raw logical decoding plugin
+      # payload contained inside PostgreSQL's XLogData envelope; callers normally
+      # pass this payload to a pgoutput parser.
+      #
+      # @yield [payload, metadata] called once for each XLogData payload
+      # @yieldparam payload [String] frozen raw pgoutput payload bytes
+      # @yieldparam metadata [XLogData] WAL envelope metadata for the payload
+      # @return [void]
+      # @raise [ArgumentError] if no block is provided
+      # @raise [ConnectionError] if a PostgreSQL connection or command fails
+      # @raise [ProtocolError] if an invalid replication message is received
+      def start(&block)
+        raise ArgumentError, "block required" unless block
+
+        @stopped = false
+        @running = true
+        @last_error = nil
+        @reconnect_attempts = 0
+
+        loop do
+          current_configuration = configuration_for_resume
+          case run_stream_cycle(current_configuration, &block)
+          when :done
+            break
+          when :retry
+            @reconnect_attempts += 1
+            raise @last_error if @reconnect_attempts > DEFAULT_RECONNECT_ATTEMPTS
+
+            sleep(reconnect_backoff_for(@reconnect_attempts))
+          end
+        end
+      ensure
+        @running = false
+        @stopped = true
+      end
+
+      # Request graceful stop.
+      #
+      # This method records the caller's intent to stop and asks the active
+      # {Stream}, if any, to stop after its current iteration.
+      #
+      # @return [void]
+      def stop
+        @stopped = true
+        @stream&.stop
+        nil
+      end
+
+      # Stop the active stream and start again with the same block.
+      #
+      # The runner is synchronous, so this helper is primarily useful for
+      # supervisors that call it instead of manually calling {#stop} followed by
+      # {#start}.
+      #
+      # @yield [payload, metadata] called once for each XLogData payload
+      # @return [void]
+      def restart(&block)
+        stop
+        start(&block)
+      end
+
+      # Whether the runner is currently inside its streaming loop.
+      #
+      # @return [Boolean]
+      def running?
+        @running
+      end
+
+      # Whether the runner has stopped.
+      #
+      # @return [Boolean]
+      def stopped?
+        !running?
+      end
+
+      # Whether an active replication stream exists.
+      #
+      # @return [Boolean]
+      def connected?
+        !@stream.nil?
+      end
+
+      # Mark a WAL position as durably handled by downstream code.
+      #
+      # This does not checkpoint or persist anything. It only updates transport
+      # feedback state so future standby status updates can distinguish received
+      # WAL from downstream-acknowledged WAL.
+      #
+      # @param lsn [String, Integer] WAL position acknowledged by downstream code
+      # @return [Integer] normalized acknowledged WAL position
+      def ack(lsn)
+        parsed = normalize_lsn_value(lsn)
+        @acked_lsn = [@acked_lsn ? normalize_lsn_value(@acked_lsn) : 0, parsed].max
+        @stream&.ack(@acked_lsn)
+        @acked_lsn
+      end
+
+      # Return an immutable transport status snapshot.
+      #
+      # @return [RunnerState]
+      def monitor
+        RunnerState.new(
+          running: running?,
+          stop_requested: @stopped,
+          connected: connected?,
+          last_received_lsn: current_lsn_string(@stream&.latest_lsn || @resume_lsn),
+          last_feedback_lsn: current_lsn_string(@stream&.acked_lsn || @acked_lsn),
+          last_keepalive_at: @stream&.last_keepalive_at,
+          last_error: @last_error&.message,
+          reconnect_attempts: @reconnect_attempts
+        )
+      end
+
+      private
+
+      def setup_connection(connection)
+        if configuration.auto_create_slot && !@slot_created
+          connection.create_replication_slot
+          @slot_created = true
+        end
+
+        connection.start_replication
+      end
+
+      def run_stream_cycle(configuration, &block)
+        connection = Connection.open(configuration)
+        setup_connection(connection)
+        @stream = Stream.new(connection:, configuration:, acked_lsn: @acked_lsn)
+        @stream.start(&block)
+        :done
+      rescue ConnectionError => e
+        @last_error = e
+        raise if @stopped || @stream.nil?
+
+        @resume_lsn = @stream.latest_lsn || @resume_lsn
+        @acked_lsn = @stream.acked_lsn || @acked_lsn
+        :retry
+      ensure
+        @stream = nil
+        connection&.close
+      end
+
+      def configuration_for_resume
+        return configuration if @resume_lsn.nil?
+
+        Configuration.new(
+          database_url: configuration.database_url,
+          slot_name: configuration.slot_name,
+          publication_names: configuration.publication_names,
+          start_lsn: @resume_lsn,
+          proto_version: configuration.proto_version,
+          binary: configuration.binary,
+          messages: configuration.messages,
+          auto_create_slot: configuration.auto_create_slot,
+          temporary_slot: configuration.temporary_slot,
+          feedback_interval: configuration.feedback_interval
+        )
+      end
+
+      def normalize_lsn_value(value)
+        value.is_a?(String) ? LSN.parse(value) : LSN.parse(LSN.format(value))
+      end
+
+      def current_lsn_string(value)
+        return nil if value.nil?
+
+        value.is_a?(String) ? value : LSN.format(value)
+      end
+
+      def reconnect_backoff_for(attempts)
+        DEFAULT_RECONNECT_BACKOFF * attempts
+      end
+
+      def sleep(duration)
+        Kernel.sleep(duration)
+      end
+    end
+  end
+end

data/lib/pgoutput/client/state.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+module Pgoutput
+  module Client
+    RunnerStateData = Data.define(
+      :running,
+      :stop_requested,
+      :connected,
+      :last_received_lsn,
+      :last_feedback_lsn,
+      :last_keepalive_at,
+      :last_error,
+      :reconnect_attempts
+    )
+
+    # Immutable operational snapshot for a {Runner}.
+    #
+    # `RunnerState` is intentionally small and transport-focused. It exposes
+    # connection, feedback, keepalive, and retry state without claiming anything
+    # about downstream processing, sink delivery, or business-level consumption.
+    #
+    # @attr_reader running [Boolean] whether the runner is inside its streaming loop
+    # @attr_reader stop_requested [Boolean] whether graceful stop was requested
+    # @attr_reader connected [Boolean] whether an active replication stream exists
+    # @attr_reader last_received_lsn [String, nil] latest WAL position received from PostgreSQL
+    # @attr_reader last_feedback_lsn [String, nil] latest downstream-acknowledged WAL position
+    #   used for flushed/applied feedback
+    # @attr_reader last_keepalive_at [Time, nil] last time a primary keepalive was observed
+    # @attr_reader last_error [String, nil] last transport error message
+    # @attr_reader reconnect_attempts [Integer] reconnect attempts used by the current/last run
+    # @api public
+    class RunnerState < RunnerStateData
+      # Whether the runner currently has no active stream.
+      #
+      # @return [Boolean]
+      def stopped?
+        !running
+      end
+    end
+  end
+end

data/lib/pgoutput/client/stream.rb

@@ -15,19 +15,45 @@ module Pgoutput
     # handled internally by updating the latest known WAL position and sending
     # standby feedback when requested.
     #
+    # Feedback separates receipt from downstream acknowledgment. Received LSN
+    # follows the latest WAL position seen from PostgreSQL. Flushed/applied LSN
+    # follows {#ack}, allowing downstream consumers to decide when a WAL position
+    # is safe to report as durable.
+    #
     # @api private
     class Stream
+      # Latest WAL position observed from XLogData or keepalive messages.
+      #
+      # @return [Integer]
+      attr_reader :latest_lsn
+
+      # Latest downstream-acknowledged WAL position used as flushed/applied LSN
+      # in standby feedback.
+      #
+      # @return [Integer]
+      attr_reader :acked_lsn
+
+      # Last time a primary keepalive was observed.
+      #
+      # @return [Time, nil]
+      attr_reader :last_keepalive_at
+
       # Build a stream loop.
       #
       # @param connection [Connection] replication connection
       # @param configuration [Configuration] stream configuration
+      # @param acked_lsn [String, Integer, nil] initial downstream-acknowledged
+      #   WAL position
       # @return [void]
-      def initialize(connection:, configuration:)
+      def initialize(connection:, configuration:, acked_lsn: nil)
         @connection = connection
         @configuration = configuration
         @latest_lsn = LSN.parse(configuration.start_lsn_string)
+        @acked_lsn = acked_lsn ? normalize_lsn_value(acked_lsn) : @latest_lsn
         @last_feedback_at = monotonic_time
+        @last_keepalive_at = nil
         @running = false
+        @stop_requested = false
       end
 
       # Start the stream loop.
@@ -35,7 +61,7 @@ module Pgoutput
       # The method blocks while the stream is running. For every XLogData
       # envelope, it yields the raw pgoutput payload and the parsed envelope
       # metadata. When no CopyData payload is currently available, the loop
-      # continues and checks again.
+      # pauses briefly before checking again.
       #
       # @yield [payload, metadata] called for each XLogData payload
       # @yieldparam payload [String] frozen raw pgoutput payload bytes
@@ -51,18 +77,47 @@ module Pgoutput
         @running = true
         while @running
           copy_data = @connection.get_copy_data
-          next unless copy_data
+          if copy_data.nil?
+            sleep 0.01
+            next
+          end
 
           process_copy_data(copy_data, &block)
           send_periodic_feedback
         end
+      ensure
+        send_feedback(reply_requested: false) if @stop_requested
+        @running = false
       end
 
       # Stop the stream loop after the current iteration.
       #
       # @return [void]
       def stop
+        @stop_requested = true
         @running = false
+        nil
+      end
+
+      # Whether the stream loop is active.
+      #
+      # @return [Boolean]
+      def running?
+        @running
+      end
+
+      # Mark a WAL position as durably handled by downstream code.
+      #
+      # The stream never decides sink durability on its own. Downstream code may
+      # call this after checkpointing, enqueueing, or otherwise making progress
+      # durable. Feedback sent after this call reports the acknowledged LSN as
+      # both flushed and applied.
+      #
+      # @param lsn [String, Integer] WAL position acknowledged by downstream code
+      # @return [Integer] normalized acknowledged WAL position
+      def ack(lsn)
+        parsed = normalize_lsn_value(lsn)
+        @acked_lsn = [@acked_lsn, parsed].max
       end
 
       private
@@ -76,6 +131,7 @@ module Pgoutput
         when "k".ord
           keepalive = Keepalive.parse(copy_data)
           @latest_lsn = [@latest_lsn, keepalive.wal_end].max
+          @last_keepalive_at = Time.now.utc
           send_feedback(reply_requested: true) if keepalive.reply_requested
         else
           raise ProtocolError, "unknown CopyData replication message: #{copy_data.getbyte(0).inspect}"
@@ -89,11 +145,16 @@ module Pgoutput
       end
 
       def send_feedback(reply_requested:)
-        feedback = Feedback.now(received_lsn: @latest_lsn, reply_requested:)
+        feedback = Feedback.now(received_lsn: @latest_lsn, flushed_lsn: @acked_lsn, applied_lsn: @acked_lsn,
+                                reply_requested:)
         @connection.put_copy_data(feedback.to_copy_data)
         @last_feedback_at = monotonic_time
       end
 
+      def normalize_lsn_value(value)
+        value.is_a?(String) ? LSN.parse(value) : LSN.parse(LSN.format(value))
+      end
+
       def monotonic_time
         Process.clock_gettime(Process::CLOCK_MONOTONIC)
       end

data/lib/pgoutput/client/version.rb

@@ -5,6 +5,6 @@ module Pgoutput
     # Current pgoutput-client gem version.
     #
     # @return [String]
-    VERSION = "0.1.0"
+    VERSION = "0.2.1"
   end
 end

data/lib/pgoutput/client.rb

@@ -7,9 +7,11 @@ require_relative "client/lsn"
 require_relative "client/xlog_data"
 require_relative "client/keepalive"
 require_relative "client/feedback"
+require_relative "client/state"
 require_relative "client/commands"
 require_relative "client/connection"
 require_relative "client/stream"
+require_relative "client/runner"
 
 module Pgoutput
   # Namespace for PostgreSQL logical replication transport support.
@@ -25,101 +27,5 @@ module Pgoutput
   #
   # @api public
   module Client
-    # High-level logical replication client facade.
-    #
-    # `Runner` is the simplest public entry point for consumers that want to
-    # stream raw pgoutput payloads without manually managing a replication
-    # connection. It owns the connection lifecycle for one logical replication
-    # stream:
-    #
-    # 1. Build an immutable {Configuration} from keyword arguments.
-    # 2. Open a PostgreSQL replication connection.
-    # 3. Optionally create the configured replication slot.
-    # 4. Start logical replication.
-    # 5. Yield raw pgoutput payload bytes and {XLogData} metadata.
-    # 6. Close the connection when streaming exits.
-    #
-    # @example Stream raw pgoutput messages
-    #   runner = Pgoutput::Client::Runner.new(
-    #     database_url: "postgres://localhost/app",
-    #     slot_name: "cdc_slot",
-    #     publication_names: ["app_publication"]
-    #   )
-    #
-    #   runner.start do |payload, metadata|
-    #     puts "received #{payload.bytesize} bytes at #{metadata.wal_end_lsn}"
-    #   end
-    #
-    # @see Configuration
-    # @see Connection
-    # @see Stream
-    # @api public
-    class Runner
-      # Configuration used by this runner.
-      #
-      # @return [Configuration]
-      attr_reader :configuration
-
-      # Build a runner from configuration keyword arguments.
-      #
-      # The accepted keywords are the same as {Configuration#initialize}. The
-      # resulting configuration object is immutable and reused for the lifetime
-      # of this runner.
-      #
-      # @param options [Hash{Symbol=>Object}] configuration options forwarded to
-      #   {Configuration#initialize}
-      # @return [void]
-      # @raise [ConfigurationError] if the supplied configuration is invalid
-      def initialize(**options)
-        @configuration = Configuration.new(
-          **options # : untyped
-        )
-        @stopped = false
-      end
-
-      # Start streaming raw pgoutput payloads.
-      #
-      # This method blocks until the stream stops or the underlying connection
-      # raises an error. The yielded payload is the raw logical decoding plugin
-      # payload contained inside PostgreSQL's XLogData envelope; callers normally
-      # pass this payload to a pgoutput parser.
-      #
-      # @yield [payload, metadata] called once for each XLogData payload
-      # @yieldparam payload [String] frozen raw pgoutput payload bytes
-      # @yieldparam metadata [XLogData] WAL envelope metadata for the payload
-      # @return [void]
-      # @raise [ArgumentError] if no block is provided
-      # @raise [ConnectionError] if a PostgreSQL connection or command fails
-      # @raise [ProtocolError] if an invalid replication message is received
-      def start(&block)
-        raise ArgumentError, "block required" unless block
-
-        connection = Connection.open(configuration)
-        setup_connection(connection)
-        Stream.new(connection:, configuration:).start { |payload, metadata| block.call(payload, metadata) }
-      ensure
-        connection&.close
-      end
-
-      # Request graceful stop.
-      #
-      # This method records the caller's intent to stop. The current
-      # implementation does not yet interrupt an active {Stream}; it exists as
-      # part of the public lifecycle API and may be wired into cooperative stream
-      # shutdown in a future release.
-      #
-      # @return [void]
-      def stop
-        @stopped = true
-        nil
-      end
-
-      private
-
-      def setup_connection(connection)
-        connection.create_replication_slot if configuration.auto_create_slot
-        connection.start_replication
-      end
-    end
   end
 end

data/sig/pgoutput/client/configuration.rbs

@@ -37,8 +37,6 @@ module Pgoutput
 
       @start_lsn: untyped
 
-      @plugin: untyped
-
       @proto_version: untyped
 
       @binary: untyped
@@ -51,7 +49,7 @@ module Pgoutput
 
       @feedback_interval: untyped
 
-      # Default logical decoding output plugin.
+      # Fixed logical decoding output plugin.
       #
       # @return [String]
       DEFAULT_PLUGIN: "pgoutput"
@@ -78,9 +76,6 @@ module Pgoutput
       # @!attribute [r] start_lsn
       #   Optional normalized starting LSN.
       #   @return [String, nil]
-      # @!attribute [r] plugin
-      #   Logical decoding output plugin name.
-      #   @return [String]
       # @!attribute [r] proto_version
       #   pgoutput protocol version.
       #   @return [Integer]
@@ -113,9 +108,6 @@ module Pgoutput
       # @!attribute [r] start_lsn
       #   Optional normalized starting LSN.
       #   @return [String, nil]
-      # @!attribute [r] plugin
-      #   Logical decoding output plugin name.
-      #   @return [String]
       # @!attribute [r] proto_version
       #   pgoutput protocol version.
       #   @return [Integer]
@@ -148,9 +140,6 @@ module Pgoutput
       # @!attribute [r] start_lsn
       #   Optional normalized starting LSN.
       #   @return [String, nil]
-      # @!attribute [r] plugin
-      #   Logical decoding output plugin name.
-      #   @return [String]
       # @!attribute [r] proto_version
       #   pgoutput protocol version.
       #   @return [Integer]
@@ -183,9 +172,6 @@ module Pgoutput
       # @!attribute [r] start_lsn
       #   Optional normalized starting LSN.
       #   @return [String, nil]
-      # @!attribute [r] plugin
-      #   Logical decoding output plugin name.
-      #   @return [String]
       # @!attribute [r] proto_version
       #   pgoutput protocol version.
       #   @return [Integer]
@@ -218,44 +204,6 @@ module Pgoutput
       # @!attribute [r] start_lsn
       #   Optional normalized starting LSN.
       #   @return [String, nil]
-      # @!attribute [r] plugin
-      #   Logical decoding output plugin name.
-      #   @return [String]
-      # @!attribute [r] proto_version
-      #   pgoutput protocol version.
-      #   @return [Integer]
-      # @!attribute [r] binary
-      #   Whether to request binary column values from pgoutput.
-      #   @return [Boolean]
-      # @!attribute [r] messages
-      #   Whether to request logical decoding messages from pgoutput.
-      #   @return [Boolean]
-      # @!attribute [r] auto_create_slot
-      #   Whether the client should create the slot before streaming.
-      #   @return [Boolean]
-      # @!attribute [r] temporary_slot
-      #   Whether a newly created slot should be temporary.
-      #   @return [Boolean]
-      # @!attribute [r] feedback_interval
-      #   Standby feedback interval in seconds.
-      #   @return [Float]
-      attr_reader plugin: untyped
-
-      # @!attribute [r] database_url
-      #   PostgreSQL connection URL.
-      #   @return [String]
-      # @!attribute [r] slot_name
-      #   Logical replication slot name.
-      #   @return [String]
-      # @!attribute [r] publication_names
-      #   Publication names requested from pgoutput.
-      #   @return [Array<String>]
-      # @!attribute [r] start_lsn
-      #   Optional normalized starting LSN.
-      #   @return [String, nil]
-      # @!attribute [r] plugin
-      #   Logical decoding output plugin name.
-      #   @return [String]
       # @!attribute [r] proto_version
       #   pgoutput protocol version.
       #   @return [Integer]
@@ -288,9 +236,6 @@ module Pgoutput
       # @!attribute [r] start_lsn
       #   Optional normalized starting LSN.
       #   @return [String, nil]
-      # @!attribute [r] plugin
-      #   Logical decoding output plugin name.
-      #   @return [String]
       # @!attribute [r] proto_version
       #   pgoutput protocol version.
       #   @return [Integer]
@@ -323,9 +268,6 @@ module Pgoutput
       # @!attribute [r] start_lsn
       #   Optional normalized starting LSN.
       #   @return [String, nil]
-      # @!attribute [r] plugin
-      #   Logical decoding output plugin name.
-      #   @return [String]
       # @!attribute [r] proto_version
       #   pgoutput protocol version.
       #   @return [Integer]
@@ -358,9 +300,6 @@ module Pgoutput
       # @!attribute [r] start_lsn
       #   Optional normalized starting LSN.
       #   @return [String, nil]
-      # @!attribute [r] plugin
-      #   Logical decoding output plugin name.
-      #   @return [String]
       # @!attribute [r] proto_version
       #   pgoutput protocol version.
       #   @return [Integer]
@@ -393,9 +332,6 @@ module Pgoutput
       # @!attribute [r] start_lsn
       #   Optional normalized starting LSN.
       #   @return [String, nil]
-      # @!attribute [r] plugin
-      #   Logical decoding output plugin name.
-      #   @return [String]
       # @!attribute [r] proto_version
       #   pgoutput protocol version.
       #   @return [Integer]
@@ -428,9 +364,6 @@ module Pgoutput
       # @!attribute [r] start_lsn
       #   Optional normalized starting LSN.
       #   @return [String, nil]
-      # @!attribute [r] plugin
-      #   Logical decoding output plugin name.
-      #   @return [String]
       # @!attribute [r] proto_version
       #   pgoutput protocol version.
       #   @return [Integer]
@@ -467,7 +400,6 @@ module Pgoutput
       #   names to pass to pgoutput
       # @param start_lsn [String, Integer, nil] starting LSN as a PostgreSQL LSN
       #   string, an integer WAL position, or `nil` for `0/0`
-      # @param plugin [#to_s] logical decoding plugin name
       # @param proto_version [#to_int, #to_s] pgoutput protocol version
       # @param binary [Object] truthy to request binary column values
       # @param messages [Object] truthy to request logical decoding messages
@@ -482,7 +414,7 @@ module Pgoutput
       #   settings are invalid
       # @raise [ArgumentError] if `start_lsn`, `proto_version`, or
       #   `feedback_interval` cannot be coerced
-      def initialize: (database_url: untyped, slot_name: untyped, publication_names: untyped, ?start_lsn: untyped?, ?plugin: untyped, ?proto_version: untyped, ?binary: bool, ?messages: bool, ?auto_create_slot: bool, ?temporary_slot: bool, ?feedback_interval: untyped) -> void
+      def initialize: (database_url: untyped, slot_name: untyped, publication_names: untyped, ?start_lsn: untyped?, ?proto_version: untyped, ?binary: bool, ?messages: bool, ?auto_create_slot: bool, ?temporary_slot: bool, ?feedback_interval: untyped) -> void
 
       # Starting LSN to render in `START_REPLICATION`.
       #

data/sig/pgoutput/client/runner.rbs

@@ -0,0 +1,99 @@
+module Pgoutput
+  module Client
+    # High-level logical replication client facade.
+    #
+    # `Runner` is the simplest public entry point for consumers that want to
+    # stream raw pgoutput payloads without manually managing a replication
+    # connection. It owns the connection lifecycle for one logical replication
+    # stream:
+    #
+    # 1. Build an immutable {Configuration} from keyword arguments.
+    # 2. Open a PostgreSQL replication connection.
+    # 3. Optionally create the configured replication slot.
+    # 4. Start logical replication.
+    # 5. Yield raw pgoutput payload bytes and {XLogData} metadata.
+    # 6. Close the connection when streaming exits.
+    #
+    # If a live stream loses its connection, the runner retries a small number
+    # of times with a backoff and resumes from the latest confirmed WAL
+    # position. Replay, checkpointing, and deduplication are not owned here;
+    # those concerns belong to the downstream CDC runtime and sink layer.
+    #
+    # @example Stream raw pgoutput messages
+    #   runner = Pgoutput::Client::Runner.new(
+    #     database_url: "postgres://localhost/app",
+    #     slot_name: "cdc_slot",
+    #     publication_names: ["app_publication"]
+    #   )
+    #
+    #   runner.start do |payload, metadata|
+    #     puts "received #{payload.bytesize} bytes at #{metadata.wal_end_lsn}"
+    #   end
+    #
+    # @see Configuration
+    # @see Connection
+    # @see Stream
+    # @api public
+    class Runner
+      @configuration: untyped
+
+      @stopped: untyped
+
+      @running: untyped
+
+      @stream: untyped
+
+      @resume_lsn: untyped
+
+      @acked_lsn: untyped
+
+      @slot_created: untyped
+
+      @last_error: untyped
+
+      @reconnect_attempts: untyped
+
+      DEFAULT_RECONNECT_ATTEMPTS: 3
+
+      DEFAULT_RECONNECT_BACKOFF: ::Float
+
+      attr_reader configuration: untyped
+
+      attr_reader last_error: untyped
+
+      def initialize: (**untyped options) -> void
+
+      def start: () ?{ (?) -> untyped } -> untyped
+
+      def stop: () -> nil
+
+      def restart: () ?{ (?) -> untyped } -> untyped
+
+      def running?: () -> bool
+
+      def stopped?: () -> bool
+
+      def connected?: () -> bool
+
+      def ack: (untyped lsn) -> untyped
+
+      def monitor: () -> Pgoutput::Client::RunnerState
+
+      private
+
+      def setup_connection: (untyped connection) -> untyped
+
+      def run_stream_cycle: (untyped configuration) { (?) -> untyped } -> untyped
+
+      def configuration_for_resume: () -> untyped
+
+      def normalize_lsn_value: (untyped value) -> untyped
+
+      def current_lsn_string: (untyped value) -> untyped
+
+      def reconnect_backoff_for: (untyped attempts) -> untyped
+
+      def sleep: (untyped duration) -> untyped
+    end
+  end
+end

data/sig/pgoutput/client/state.rbs

@@ -0,0 +1,29 @@
+module Pgoutput
+  module Client
+    class RunnerStateData < Data
+      attr_reader running: bool
+      attr_reader stop_requested: bool
+      attr_reader connected: bool
+      attr_reader last_received_lsn: String?
+      attr_reader last_feedback_lsn: String?
+      attr_reader last_keepalive_at: Time?
+      attr_reader last_error: String?
+      attr_reader reconnect_attempts: Integer
+
+      def self.new: (
+        running: bool,
+        stop_requested: bool,
+        connected: bool,
+        last_received_lsn: String?,
+        last_feedback_lsn: String?,
+        last_keepalive_at: Time?,
+        last_error: String?,
+        reconnect_attempts: Integer
+      ) -> instance
+    end
+
+    class RunnerState < RunnerStateData
+      def stopped?: () -> bool
+    end
+  end
+end

data/sig/pgoutput/client/stream.rbs

@@ -21,23 +21,38 @@ module Pgoutput
 
       @latest_lsn: untyped
 
+      @acked_lsn: untyped
+
       @last_feedback_at: untyped
 
+      @last_keepalive_at: untyped
+
       @running: untyped
 
+      @stop_requested: untyped
+
+      # Latest confirmed WAL position observed by this stream.
+      #
+      # @return [Integer]
+      attr_reader latest_lsn: untyped
+
+      attr_reader acked_lsn: untyped
+
+      attr_reader last_keepalive_at: untyped
+
       # Build a stream loop.
       #
       # @param connection [Connection] replication connection
       # @param configuration [Configuration] stream configuration
       # @return [void]
-      def initialize: (connection: untyped, configuration: untyped) -> void
+      def initialize: (connection: untyped, configuration: untyped, ?acked_lsn: untyped) -> void
 
       # Start the stream loop.
       #
       # The method blocks while the stream is running. For every XLogData
       # envelope, it yields the raw pgoutput payload and the parsed envelope
       # metadata. When no CopyData payload is currently available, the loop
-      # continues and checks again.
+      # pauses briefly before checking again.
       #
       # @yield [payload, metadata] called for each XLogData payload
       # @yieldparam payload [String] frozen raw pgoutput payload bytes
@@ -52,7 +67,11 @@ module Pgoutput
       # Stop the stream loop after the current iteration.
       #
       # @return [void]
-      def stop: () -> untyped
+      def stop: () -> nil
+
+      def running?: () -> bool
+
+      def ack: (untyped lsn) -> untyped
 
       private
 
@@ -62,6 +81,8 @@ module Pgoutput
 
       def send_feedback: (reply_requested: untyped) -> untyped
 
+      def normalize_lsn_value: (untyped value) -> untyped
+
       def monotonic_time: () -> untyped
     end
   end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: pgoutput-client
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.1
 platform: ruby
 authors:
 - Ken C. Demanawa
@@ -23,8 +23,18 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1.6'
-description: Transport-only PostgreSQL logical replication client for receiving pgoutput
-  CopyData payloads.
+description: |
+  pgoutput-client provides a PostgreSQL logical replication transport for Ruby.
+
+  It manages replication connections, lifecycle, monitoring,
+  keepalive handling, graceful shutdown, and transport-level
+  acknowledgment boundaries while streaming pgoutput messages
+  to downstream consumers.
+
+  pgoutput-client owns transport.
+
+  Parsers, decoders, source adapters, runtimes, and sinks remain
+  separate concerns within the CDC ecosystem.
 email:
 - kenneth.c.demanawa@gmail.com
 executables: []
@@ -42,12 +52,13 @@ files:
 - lib/pgoutput/client/feedback.rb
 - lib/pgoutput/client/keepalive.rb
 - lib/pgoutput/client/lsn.rb
+- lib/pgoutput/client/runner.rb
+- lib/pgoutput/client/state.rb
 - lib/pgoutput/client/stream.rb
 - lib/pgoutput/client/version.rb
 - lib/pgoutput/client/xlog_data.rb
 - lib/pgoutput_client.rb
 - sig/pg.rbs
-- sig/pgoutput/client.rbs
 - sig/pgoutput/client/commands.rbs
 - sig/pgoutput/client/configuration.rbs
 - sig/pgoutput/client/connection.rbs
@@ -55,6 +66,8 @@ files:
 - sig/pgoutput/client/feedback.rbs
 - sig/pgoutput/client/keepalive.rbs
 - sig/pgoutput/client/lsn.rbs
+- sig/pgoutput/client/runner.rbs
+- sig/pgoutput/client/state.rbs
 - sig/pgoutput/client/stream.rbs
 - sig/pgoutput/client/version.rbs
 - sig/pgoutput/client/xlog_data.rbs
@@ -83,5 +96,6 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 3.6.9
 specification_version: 4
-summary: PostgreSQL pgoutput logical replication transport client.
+summary: PostgreSQL pgoutput logical replication transport and lifecycle management
+  for Ruby.
 test_files: []

data/sig/pgoutput/client.rbs

@@ -1,97 +0,0 @@
-module Pgoutput
-  # Namespace for PostgreSQL logical replication transport support.
-  #
-  # `Pgoutput::Client` is the replication transport layer of the CDC Ecosystem.
-  # It is responsible for connecting to PostgreSQL in replication mode, creating
-  # or consuming replication slots, issuing `START_REPLICATION`, reading CopyData
-  # messages, and sending standby feedback.
-  #
-  # This namespace intentionally does not parse pgoutput plugin payloads into
-  # table-level changes. Raw plugin bytes are yielded to downstream protocol and
-  # type layers such as `pgoutput-parser` and `pgoutput-decoder`.
-  #
-  # @api public
-  module Client
-    # High-level logical replication client facade.
-    #
-    # `Runner` is the simplest public entry point for consumers that want to
-    # stream raw pgoutput payloads without manually managing a replication
-    # connection. It owns the connection lifecycle for one logical replication
-    # stream:
-    #
-    # 1. Build an immutable {Configuration} from keyword arguments.
-    # 2. Open a PostgreSQL replication connection.
-    # 3. Optionally create the configured replication slot.
-    # 4. Start logical replication.
-    # 5. Yield raw pgoutput payload bytes and {XLogData} metadata.
-    # 6. Close the connection when streaming exits.
-    #
-    # @example Stream raw pgoutput messages
-    #   runner = Pgoutput::Client::Runner.new(
-    #     database_url: "postgres://localhost/app",
-    #     slot_name: "cdc_slot",
-    #     publication_names: ["app_publication"]
-    #   )
-    #
-    #   runner.start do |payload, metadata|
-    #     puts "received #{payload.bytesize} bytes at #{metadata.wal_end_lsn}"
-    #   end
-    #
-    # @see Configuration
-    # @see Connection
-    # @see Stream
-    # @api public
-    class Runner
-      @configuration: untyped
-
-      @stopped: untyped
-
-      # Configuration used by this runner.
-      #
-      # @return [Configuration]
-      attr_reader configuration: untyped
-
-      # Build a runner from configuration keyword arguments.
-      #
-      # The accepted keywords are the same as {Configuration#initialize}. The
-      # resulting configuration object is immutable and reused for the lifetime
-      # of this runner.
-      #
-      # @param options [Hash{Symbol=>Object}] configuration options forwarded to
-      #   {Configuration#initialize}
-      # @return [void]
-      # @raise [ConfigurationError] if the supplied configuration is invalid
-      def initialize: (**untyped options) -> void
-
-      # Start streaming raw pgoutput payloads.
-      #
-      # This method blocks until the stream stops or the underlying connection
-      # raises an error. The yielded payload is the raw logical decoding plugin
-      # payload contained inside PostgreSQL's XLogData envelope; callers normally
-      # pass this payload to a pgoutput parser.
-      #
-      # @yield [payload, metadata] called once for each XLogData payload
-      # @yieldparam payload [String] frozen raw pgoutput payload bytes
-      # @yieldparam metadata [XLogData] WAL envelope metadata for the payload
-      # @return [void]
-      # @raise [ArgumentError] if no block is provided
-      # @raise [ConnectionError] if a PostgreSQL connection or command fails
-      # @raise [ProtocolError] if an invalid replication message is received
-      def start: () ?{ (?) -> untyped } -> untyped
-
-      # Request graceful stop.
-      #
-      # This method records the caller's intent to stop. The current
-      # implementation does not yet interrupt an active {Stream}; it exists as
-      # part of the public lifecycle API and may be wired into cooperative stream
-      # shutdown in a future release.
-      #
-      # @return [void]
-      def stop: () -> nil
-
-      private
-
-      def setup_connection: (untyped connection) -> untyped
-    end
-  end
-end