pgbus 0.13.0 → 0.13.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '0589d2d6e69eb8b2ba5863cb8c8e2779bca29528030410508e1930fcfbd0d774'
-  data.tar.gz: c491442cd7b1d2bdccc5ff95153dc31abb424111906e17872b6b112830a1cde7
+  metadata.gz: 8c955f8fb2ac4e66fb2db388505ee83341aa6dec7c00f606162f7fba7c29cb75
+  data.tar.gz: a31e43c6f48cb10609d73dc507e9d177d25a299ae25c6f5d5a39a045018c32e8
 SHA512:
-  metadata.gz: 63e8480dcd7efb70bd7933cba734c23d7baa6f9e50ff658b575720da5ccc53e43fde1e4aca0fc7c30b6715448d040c1bb2fff472f5ac1686ae3e62853beb0bdd
-  data.tar.gz: 186da9576934f14264e9d9989c1a035d1c725e8283d1a735692da90f1610f4c538776b558a526d0c536466f92e405ffcbf99fd5bf229f9bf9b92dd1e0d7ec876
+  metadata.gz: 7cd3fba720a5ab724fe4f0b5b89168cb2df4ba68e0a5a35cca725093724f8f2b2109c379a2263442111367001a52d2eda440eef8e1c2469653d2022d2a92d148
+  data.tar.gz: 4a4e5e1c2c229c599541e61ac6ad5602dfbfcf057b346b22b0271c10a48752c41075c7166b43dd55ece7ef4a30d79ec3fd6f5cac5b898b2578ecb90c73bcc2a8

data/CHANGELOG.md

@@ -2,6 +2,8 @@
 
 ### Added
 
+- **Streams: one LISTEN connection per web host — `streams_listen_scope` (issue #382).** ⚠️ **Default behavior change.** Previously every Puma worker lazily opened its own dedicated streams LISTEN connection on first SSE use, so a web host pinned one direct connection per worker. Under the new default (`streams_listen_scope = :master`) the `pgbus_streams` Puma plugin runs a **MasterHub** in the preforking master: ONE `Web::Streamer::Listener` on the refcounted union of every worker's stream channels, fanning wakes — **including ephemeral payloads** — out to workers over a Unix domain socket with length-prefixed frames (`Streamer::HubProtocol`). Workers connect lazily (nothing is inherited across fork) and the synchronous `ensure_listening` ack contract is preserved cross-process: a sub is registered before LISTEN executes and acked only after, so the no-lost-broadcast guarantee holds. Backpressure follows the streams rules: durable wakes are droppable at a per-worker cap (they self-heal via `read_after`), **ephemeral wakes are never dropped** — a worker that stops draining is evicted, which triggers its own fallback. **Fallback is per-worker listeners, not loss**: whenever the hub is absent or dies (no `preload_app!`, single-mode Puma, crash, eviction) each worker's `FailoverListener` swaps in a real per-worker `Listener` and re-LISTENs its recorded subscriptions — connection footprint balloons back to pre-#382 levels (census-visible) but no broadcast semantics change; the worker stays local until it recycles. Measured (local PG, n=50): the master→worker hop is noise-level free — single-broadcast SSE roundtrip p50 16.00ms via the hub vs 16.93ms per-worker. **`:master` effectively requires `preload_app!`** (the hub waits for the app's pgbus initializer; without it the deadline expires quietly and workers stay per-worker). **Rollback:** `config.streams_listen_scope = :process`. Refs #382, builds on the #381 patterns.
+
 - **Host-level shared LISTEN: `worker_notify_scope` — the supervisor now owns ONE direct LISTEN connection for the whole host (issue #381).** ⚠️ **Default behavior change.** Previously every worker fork and every consumer fork opened its own dedicated LISTEN connection (`NotifyListener`), so a host's direct-connection footprint scaled with fork count — on transaction-pool PgBouncer platforms those connections come out of the scarcest slice of `max_connections`, and a 5-capsule + 2-consumer host pinned 7. Under the new default (`config.worker_notify_scope = :supervisor`) the supervisor runs a single `NotifyHub`: one `NotifyListener` on the union of every capsule's and consumer's queue channels (wildcards via the shared resolver, consumer sets via the registry), fanning wakes out to forks over per-fork pipes (`W` wake / `H` healthy / `P` degraded bytes; a fork whose pipe reports degraded or reaches EOF falls back to fast polling exactly like a failed local listener). Footprint drops to **1 direct LISTEN connection per job host**, verified by integration test: routing is per-fork (an insert wakes only the forks reading that queue, wildcard capsules unconditionally), and `pg_terminate_backend` on the shared connection is survived — reconnect, re-LISTEN, wakes flow again. **Rollback:** `config.worker_notify_scope = :fork` restores the previous per-fork listeners byte-for-byte. Dedicated LISTEN connections are now census-tagged `application_name=pgbus-listen` so `pg_stat_activity` can count them. Refs #381.
 - **`pgbus doctor`: new "Connection budget" check (issue #381).** Prints how many direct LISTEN connections the current config pins — 1 per host under `:supervisor` scope, capsules + consumers under `:fork` (honoring `config.roles`), plus a "+1 per web-server process (streams)" clause — so operators can do pooler capacity math from the doctor output alone. Informational, always `:ok`. Refs #381.
 - **Benchmarks: `rake bench:notify_wake` and `rake bench:notify_chaos` (issue #381).** Wake-path latency (send → wake, p50/p95/p99, direct vs hub-mediated), empty-read cost, LISTEN connection census, and failure-mode measurements (killed LISTEN backend, wedged fork, FD churn, fan-out cost). Refs #381.

data/README.md

@@ -1876,7 +1876,7 @@ A single preflight command that answers "is this environment healthy enough to r
 | Broadcast queue | — | Turbo broadcasts share the default queue in production, or `streams_broadcast_queue` is set but no worker capsule drains it |
 | Primary affinity | — | Job connection is on a read-only replica (`pg_is_in_recovery`) — a read/write-splitting pooler may be stalling jobs |
 | Dedicated connections | Streamer LISTEN and/or worker notify dedicated path cannot connect | — |
-| Connection budget | — (informational: prints how many direct LISTEN connections the current config pins — 1 per host under `worker_notify_scope: :supervisor`, one per fork under `:fork`, plus 1 per web process when streams are enabled) | — |
+| Connection budget | — (informational: prints how many direct LISTEN connections the current config pins — 1 per host under `worker_notify_scope: :supervisor`, one per fork under `:fork`; streams add 1 per web host under `streams_listen_scope: :master` or 1 per web process under `:process`) | — |
 
 ```bash
 pgbus doctor       # prints the report; exit 1 unless every check passed

data/Rakefile

@@ -25,7 +25,7 @@ namespace :bench do
   # no-DB unit suite that bench:all runs in CI.
   db_benches = %w[connection_pool_bench integration_bench streams_bench streams_read_pool_bench
                   execution_modes_bench pool_swap_bench pool_autoscale_bench job_burst_bench
-                  notify_wake_bench notify_chaos_bench].freeze
+                  notify_wake_bench notify_chaos_bench streams_hub_bench].freeze
   # The unit suite is every *_bench.rb that doesn't need a database, derived
   # from the directory so a new unit bench is picked up automatically (kept in
   # sync with bench:one, which globs the same files).
@@ -95,6 +95,11 @@ namespace :bench do
     ruby "benchmarks/notify_chaos_bench.rb"
   end
 
+  desc "Run streams master-hub latency benchmark (#382 hop cost + census; requires PGBUS_DATABASE_URL)"
+  task :streams_hub do
+    ruby "benchmarks/streams_hub_bench.rb"
+  end
+
   desc "Run a single benchmark: rake bench:one[client_bench]"
   task :one, [:name] do |_t, args|
     name = args[:name] or abort "Usage: rake bench:one[serialization_bench|client_bench|...]"

data/lib/pgbus/configuration.rb

@@ -270,6 +270,7 @@ module Pgbus
 
       @worker_notify_wakeup = nil
       @worker_notify_scope = :supervisor
+      @streams_listen_scope = :master
       @worker_notify_host = nil
       @worker_notify_port = nil
       @worker_notify_database_url = nil
@@ -632,6 +633,34 @@ module Pgbus
       @doctor_on_boot = coerced
     end
 
+    # Where the streams LISTEN connection lives (issue #382):
+    #   :master (default) — ONE shared listener in the preforking web master
+    #     (MasterHub); workers connect lazily over a Unix socket and fall back
+    #     to a per-worker listener whenever the hub is absent or dies.
+    #   :process — one listener per web process: the pre-0.13 behavior, and
+    #     the automatic behavior on single-mode / non-preforking servers.
+    attr_reader :streams_listen_scope
+
+    VALID_STREAMS_LISTEN_SCOPES = %i[master process].freeze
+
+    def streams_listen_scope=(scope)
+      coerced = case scope
+                when Symbol then scope
+                when String then scope.to_sym
+                else
+                  raise Pgbus::ConfigurationError,
+                        "Invalid streams_listen_scope type: #{scope.class}. " \
+                        "Must be :master (one shared LISTEN connection per web host) or :process (one per worker)"
+                end
+      unless VALID_STREAMS_LISTEN_SCOPES.include?(coerced)
+        raise Pgbus::ConfigurationError,
+              "Invalid streams_listen_scope: #{coerced.inspect}. " \
+              "Must be :master (one shared LISTEN connection per web host) or :process (one per worker)"
+      end
+
+      @streams_listen_scope = coerced
+    end
+
     VALID_WORKER_NOTIFY_SCOPES = %i[supervisor fork].freeze
 
     # Validated at assignment time like the other enum options. A String is

data/lib/pgbus/doctor.rb

@@ -394,12 +394,23 @@ module Pgbus
         consumers: consumers, con_plural: consumers == 1 ? "" : "s",
         share: count == 1 && @config.worker_notify_scope == :supervisor ? " share it" : ""
       )
-      detail += " + 1 per web-server process (streams)" if @config.streams_enabled
+      detail += streams_budget_clause if @config.streams_enabled
       Check.new(name: "Connection budget", status: :ok, detail: detail)
     rescue StandardError => e
       Check.new(name: "Connection budget", status: :warn, detail: "#{e.class}: #{e.message}")
     end
 
+    # Streams add their own LISTEN footprint on web hosts: one per host with
+    # the master hub (#382, the default — workers fall back per-worker only
+    # during a hub outage), one per web process under :process scope.
+    def streams_budget_clause
+      if @config.streams_listen_scope == :master
+        " + 1 per web host (streams master hub; per-worker fallback during a hub outage costs 1 per web process)"
+      else
+        " + 1 per web-server process (streams)"
+      end
+    end
+
     # Open one dedicated connection the way the runtime does, verify it
     # answers, close it. Returns nil on success, "label: error" on failure.
     def probe_dedicated_connection(label, opts)

data/lib/pgbus/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Pgbus
-  VERSION = "0.13.0"
+  VERSION = "0.13.1"
 end

data/lib/pgbus/web/streamer/failover_listener.rb

@@ -0,0 +1,130 @@
+# frozen_string_literal: true
+
+module Pgbus
+  module Web
+    module Streamer
+      # The worker-side seam between the two listening modes (issue #382):
+      # starts on the master hub (HubClient) and fails over — once, one-way —
+      # to a per-worker Listener when the hub transport dies (master gone,
+      # ack deadline, eviction). The Dispatcher/Instance consume the same
+      # ensure_listening/remove_listening/stop surface either way and never
+      # learn which mode is active.
+      #
+      # Fallback direction is settled on #382: per-worker listener, not
+      # poll-only — ephemeral broadcasts have no polling equivalent (their
+      # payload exists only in the NOTIFY), so an outage trades connections
+      # for unchanged semantics. Once fallen back, the worker stays local
+      # until it recycles; no flap-back.
+      #
+      # The subscription set is recorded here so failover can rebuild the
+      # exact LISTEN set on the fresh local connection before anything else
+      # relies on it. ensure_listening NEVER raises to the dispatcher: on a
+      # double failure (hub dead AND local build failing — e.g. DB down) it
+      # logs and returns nil, matching the Listener's own ack-timeout
+      # contract, which the dispatcher already tolerates.
+      class FailoverListener
+        def initialize(hub_client:, local_listener_factory:, logger: Pgbus.logger)
+          @hub_client = hub_client
+          @local_listener_factory = local_listener_factory
+          @logger = logger
+          # @state_mutex guards the cheap shared state (@subscriptions, @impl,
+          # @failed_over) and is only ever held for constant-time work — the
+          # dispatcher's ensure/remove path must never wait behind a failover
+          # build. @failover_mutex serializes the (blocking) build + replay:
+          # a fresh PG connect + N re-LISTEN acks can stall for seconds when
+          # the trigger IS a database problem (review on #384).
+          @state_mutex = Mutex.new
+          @failover_mutex = Mutex.new
+          @subscriptions = Set.new
+          @impl = hub_client
+          @failed_over = false
+        end
+
+        # Interface parity with Listener for Instance#start: the hub client
+        # connected at construction and the fallback starts itself on swap.
+        def start
+          self
+        end
+
+        def ensure_listening(queue)
+          @state_mutex.synchronize { @subscriptions.add(queue) }
+          current_impl.ensure_listening(queue)
+        rescue HubClient::HubUnavailableError
+          fail_over!
+          begin
+            current_impl.ensure_listening(queue)
+          rescue HubClient::HubUnavailableError
+            # fail_over! itself failed (factory raised) and @impl is still the
+            # dead client — reported there; honor the nil-on-timeout contract.
+            nil
+          end
+        end
+
+        def remove_listening(queue)
+          @state_mutex.synchronize { @subscriptions.delete(queue) }
+          current_impl.remove_listening(queue)
+        rescue HubClient::HubUnavailableError => e
+          @logger.debug do
+            "[Pgbus::Streamer::FailoverListener] remove_listening on a dead hub client " \
+              "(#{e.message}) — ignoring, unlisten GC is best-effort"
+          end
+          nil
+        end
+
+        # Idempotent, callable from the client's on_failure (reader thread)
+        # and from a synchronous ensure failure (dispatcher thread).
+        # @failover_mutex serializes concurrent callers — the second blocks
+        # until the first finishes and then no-ops, so a synchronous retry
+        # after fail_over! always lands on the swapped-in local listener.
+        # The blocking build + replay runs OUTSIDE @state_mutex so concurrent
+        # ensure/remove/stop calls never stall behind it.
+        def fail_over!
+          local = nil
+          @failover_mutex.synchronize do
+            return if @state_mutex.synchronize { @failed_over }
+
+            local = @local_listener_factory.call
+            @state_mutex.synchronize { @subscriptions.dup }.each { |q| local.ensure_listening(q) }
+            # Subscriptions recorded between the snapshot and this swap arrive
+            # via their own retried ensure_listening call on the new impl.
+            @state_mutex.synchronize do
+              @impl = local
+              @failed_over = true
+            end
+            # Ownership transferred to @impl — the rescue must not stop it.
+            local = nil
+          end
+        rescue StandardError => e
+          # A listener the factory STARTED but that never swapped in (the
+          # replay raised) would otherwise leak its thread and LISTEN
+          # connection alongside the dead hub client.
+          begin
+            local&.stop
+          rescue StandardError
+            nil
+          end
+          # Hub dead AND the local listener can't be built (DB down, config
+          # broken). Mark failed-over so callers stop rebuilding; @impl stays
+          # on the dead client — every ensure_listening resolves nil and the
+          # dispatcher rides its existing timeout tolerance until the worker
+          # recycles.
+          @state_mutex.synchronize { @failed_over = true }
+          @logger.error do
+            "[Pgbus::Streamer::FailoverListener] fallback listener failed to build " \
+              "(#{e.class}: #{e.message}) — streams degraded until this worker recycles"
+          end
+        end
+
+        def stop
+          current_impl.stop
+        end
+
+        private
+
+        def current_impl
+          @state_mutex.synchronize { @impl }
+        end
+      end
+    end
+  end
+end

data/lib/pgbus/web/streamer/hub_client.rb

@@ -0,0 +1,199 @@
+# frozen_string_literal: true
+
+require "socket"
+
+module Pgbus
+  module Web
+    module Streamer
+      # Worker-side client for the MasterHub (issue #382). Presents the same
+      # surface the Dispatcher consumes from a Listener — synchronous
+      # `ensure_listening` (the no-lost-broadcast ack contract, now crossing
+      # the process boundary), async `remove_listening` — while wakes arrive
+      # as HubProtocol frames and are re-materialized into the worker's
+      # dispatch queue as WakeMessages.
+      #
+      # Failure model: this class never retries. Connect refusal, an ack
+      # deadline, or transport EOF (master died / eviction) marks the client
+      # dead, fails every pending sub, and fires +on_failure+ exactly once —
+      # the FailoverListener's cue to swap in a per-worker Listener. One-way:
+      # once a worker has fallen back it stays local until it recycles
+      # (settled on #382 — no flap-back complexity).
+      class HubClient
+        class HubUnavailableError < StandardError; end
+
+        # Optimistic before the first status broadcast, mirroring WakePipe /
+        # NotifyListener: a just-connected worker isn't treated as degraded
+        # before the hub has said anything.
+        def initialize(socket_path:, dispatch_queue:, ack_timeout: 2.0,
+                       on_failure: nil, logger: Pgbus.logger)
+          @socket_path = socket_path
+          @dispatch_queue = dispatch_queue
+          @ack_timeout = ack_timeout
+          @on_failure = on_failure
+          @logger = logger
+          @write_mutex = Mutex.new
+          @ack_mutex = Mutex.new
+          @pending_acks = Hash.new { |h, k| h[k] = [] }
+          @hub_healthy = true
+          @dead = false
+          @stopping = false
+          @sock = nil
+          @reader = nil
+        end
+
+        def connect
+          @sock = UNIXSocket.new(@socket_path)
+          @reader = Thread.new { reader_loop }
+          self
+        rescue SystemCallError, IOError, ArgumentError, ThreadError => e
+          # ArgumentError: a socket path over the platform sun_path limit;
+          # IOError: a path that exists but is not a socket; ThreadError: the
+          # reader thread could not spawn. All must fall back exactly like a
+          # refused connect, never abort worker boot — and never leak the
+          # half-opened socket.
+          close_quietly(@sock)
+          @sock = nil
+          raise HubUnavailableError, "cannot reach master hub at #{@socket_path}: #{e.class}: #{e.message}"
+        end
+
+        def hub_healthy?
+          @hub_healthy
+        end
+
+        def dead?
+          @dead
+        end
+
+        # Synchronous, bounded: returns :done once the master has confirmed
+        # LISTEN is active for +queue+. Raises HubUnavailableError on a dead
+        # transport or an expired ack deadline (which also kills the
+        # transport — a hub that can't ack in time can't be trusted with the
+        # no-lost-broadcast contract either).
+        def ensure_listening(queue)
+          raise HubUnavailableError, "master hub transport is dead" if @dead
+
+          waiter = Queue.new
+          @ack_mutex.synchronize { @pending_acks[queue] << waiter }
+          write_frame({ "t" => "sub", "q" => queue })
+
+          result = waiter.pop(timeout: @ack_timeout)
+          if result.nil?
+            discard_waiter(queue, waiter)
+            mark_dead("sub ack for #{queue} not received within #{@ack_timeout}s")
+            raise HubUnavailableError, "master hub ack timeout for #{queue}"
+          end
+          raise HubUnavailableError, "master hub died while awaiting ack for #{queue}" if result == :dead
+
+          :done
+        end
+
+        # Lazy GC, fire-and-forget — no correctness path waits on UNLISTEN
+        # (mirrors Listener#remove_listening). A dead transport is a no-op:
+        # the master's EOF cleanup already released this worker's refs.
+        def remove_listening(queue)
+          return if @dead
+
+          write_frame({ "t" => "unsub", "q" => queue })
+        rescue HubUnavailableError
+          nil
+        end
+
+        def stop
+          @stopping = true
+          close_quietly(@sock)
+          @reader&.join(2)
+          @reader = nil
+          self
+        end
+
+        private
+
+        def reader_loop
+          loop do
+            frame = HubProtocol.read_frame(@sock)
+            break if frame.nil?
+
+            handle_frame(frame)
+          end
+          mark_dead("master hub closed the transport") unless @stopping
+        rescue HubProtocol::ProtocolError => e
+          mark_dead("master hub protocol error: #{e.message}") unless @stopping
+        rescue IOError, Errno::EBADF, Errno::ECONNRESET
+          mark_dead("master hub transport error") unless @stopping
+        rescue StandardError => e
+          # The reader thread is the ONLY detector of hub death — an
+          # unexpected error must not let it exit with the client still
+          # reporting healthy, or the worker goes silently deaf.
+          mark_dead("master hub reader crashed: #{e.class}: #{e.message}") unless @stopping
+        end
+
+        def handle_frame(frame)
+          case frame["t"]
+          when "wake"
+            @dispatch_queue << Listener::WakeMessage.new(queue_name: frame["q"], payload: frame["p"])
+          when "ack"
+            @ack_mutex.synchronize { @pending_acks[frame["q"]].shift }&.push(:ack)
+          when "status"
+            @hub_healthy = frame["healthy"]
+          else
+            @logger.warn { "[Pgbus::Streamer::HubClient] unknown frame from master: #{frame["t"].inspect}" }
+          end
+        end
+
+        # Frames must never interleave — all writes go through one mutex
+        # (writers: dispatcher thread via ensure/remove; no writer thread
+        # needed client-side, sub/unsub frames are tiny). Bounded: a master
+        # that stopped draining its input would otherwise block this write
+        # forever, and the ack deadline only starts ticking AFTER the write
+        # returns — so a stalled write is itself a failover trigger.
+        def write_frame(message)
+          data = HubProtocol.encode(message)
+          deadline = ::Process.clock_gettime(::Process::CLOCK_MONOTONIC) + @ack_timeout
+          @write_mutex.synchronize do
+            until data.empty?
+              begin
+                written = @sock.write_nonblock(data)
+                data = data.byteslice(written..)
+              rescue IO::WaitWritable
+                remaining = deadline - ::Process.clock_gettime(::Process::CLOCK_MONOTONIC)
+                raise Errno::ETIMEDOUT, "write stalled" if remaining <= 0 || !@sock.wait_writable(remaining)
+              end
+            end
+          end
+        rescue IOError, Errno::EPIPE, Errno::EBADF, Errno::ECONNRESET, Errno::ETIMEDOUT => e
+          mark_dead("write to master hub failed: #{e.class}")
+          raise HubUnavailableError, "master hub transport is dead"
+        end
+
+        # Idempotent: first caller flips @dead, fails every waiter, fires
+        # on_failure once. Reachable from the reader (EOF/protocol error) and
+        # from ack timeouts / failed writes on caller threads.
+        def mark_dead(reason)
+          waiters = @ack_mutex.synchronize do
+            return if @dead
+
+            @dead = true
+            drained = @pending_acks.values.flatten
+            @pending_acks.clear
+            drained
+          end
+          @hub_healthy = false
+          waiters.each { |w| w << :dead }
+          close_quietly(@sock)
+          @logger.warn { "[Pgbus::Streamer::HubClient] #{reason} — falling back to a per-worker listener" }
+          @on_failure&.call
+        end
+
+        def discard_waiter(queue, waiter)
+          @ack_mutex.synchronize { @pending_acks[queue].delete(waiter) }
+        end
+
+        def close_quietly(io)
+          io.close if io && !io.closed?
+        rescue IOError, Errno::EBADF
+          nil
+        end
+      end
+    end
+  end
+end

data/lib/pgbus/web/streamer/hub_protocol.rb

@@ -0,0 +1,85 @@
+# frozen_string_literal: true
+
+require "json"
+
+module Pgbus
+  module Web
+    module Streamer
+      # Framing for the master-hub Unix socket (issue #382): 4-byte big-endian
+      # payload length + UTF-8 JSON. Unlike the job-side wake pipes (1-byte,
+      # lossy-by-design — Process::WakePipe), stream frames can carry an
+      # ephemeral broadcast's ONLY copy of its HTML, so the transport is
+      # length-prefixed and lossless; drop decisions are made per-message by
+      # the MasterHub, never by the wire format.
+      #
+      # Message shapes (JSON objects; "t" is the discriminator):
+      #   worker → master: {t:"sub", q:}   subscribe, synchronous — master acks
+      #                    {t:"unsub", q:} unsubscribe, fire-and-forget
+      #   master → worker: {t:"ack", q:}                 sub acknowledged (LISTEN active)
+      #                    {t:"wake", q:, p: <String|nil>} durable (p:nil) or ephemeral wake
+      #                    {t:"status", healthy: <bool>}   listener health broadcast
+      #
+      # Reads are blocking (each side owns a dedicated reader thread); a short
+      # read means the peer died mid-frame and is reported as EOF (nil), never
+      # as a truncated message.
+      module HubProtocol
+        class ProtocolError < StandardError; end
+
+        HEADER_BYTES = 4
+        # Generous ceiling for ephemeral HTML payloads; a frame announcing
+        # more than this is a corrupt stream or a runaway producer — sever
+        # rather than allocate.
+        MAX_FRAME_BYTES = 4 * 1024 * 1024
+
+        module_function
+
+        def encode(message)
+          json = JSON.generate(message)
+          bytes = json.b
+          raise ProtocolError, "frame too large: #{bytes.bytesize} bytes (max #{MAX_FRAME_BYTES})" if
+            bytes.bytesize > MAX_FRAME_BYTES
+
+          [bytes.bytesize].pack("N") + bytes
+        end
+
+        # Returns the decoded Hash, or nil on EOF — clean close, peer death
+        # mid-frame, OR a connection reset: an abrupt close can surface as
+        # ECONNRESET instead of orderly EOF depending on unread data and
+        # platform (Ruby 4.0 reports it deterministically where 3.x saw EOF),
+        # and both mean the same thing here: the peer is gone. Raises
+        # ProtocolError on an oversized announcement or malformed JSON.
+        def read_frame(io)
+          header = read_exactly(io, HEADER_BYTES)
+          return nil unless header
+
+          length = header.unpack1("N")
+          raise ProtocolError, "frame too large: #{length} bytes (max #{MAX_FRAME_BYTES})" if length > MAX_FRAME_BYTES
+
+          body = read_exactly(io, length)
+          return nil unless body
+
+          body = body.force_encoding(Encoding::UTF_8)
+          raise ProtocolError, "malformed frame: invalid UTF-8" unless body.valid_encoding?
+
+          decoded = JSON.parse(body)
+          raise ProtocolError, "malformed frame: expected a JSON object, got #{decoded.class}" unless decoded.is_a?(Hash)
+
+          decoded
+        rescue JSON::ParserError => e
+          raise ProtocolError, "malformed frame: #{e.message}"
+        rescue Errno::ECONNRESET
+          nil
+        end
+
+        # Blocking read of exactly +count+ bytes; nil on EOF (including EOF
+        # partway through — IO#read returns the short tail once, then nil).
+        def read_exactly(io, count)
+          data = io.read(count)
+          return nil if data.nil? || data.bytesize < count
+
+          data
+        end
+      end
+    end
+  end
+end

data/lib/pgbus/web/streamer/instance.rb

@@ -39,7 +39,6 @@ module Pgbus
           @dispatch_queue = dispatch_queue || Queue.new
 
           @stream_counter = StreamCounter.new
-          @pg_connection = pg_connection || build_pg_connection
           # Self-tuning streams-pool autoscaler (issue #323). Opt-in; nil unless
           # enabled AND on the dedicated connection path (the shared-AR streams
           # pool aliases the non-thread-safe job pool and resize is a no-op there).
@@ -50,25 +49,7 @@ module Pgbus
             if @config.streams_pool_autoscale && !@client.shared_connection?
               Pgbus::Streams::PoolAutoscaler.new(client: @client, config: @config, logger: @logger)
             end
-          @listener = Listener.new(
-            pg_connection: @pg_connection,
-            dispatch_queue: @dispatch_queue,
-            health_check_ms: @config.streams_listen_health_check_ms,
-            # Opt-in dispatch-queue backpressure (issue #315 item 3). 0 =
-            # unbounded (default). The queue itself stays an unbounded
-            # Queue.new so the request-thread Connect push and the dispatcher's
-            # own prune_dead self-post never block.
-            dispatch_queue_limit: @config.streams_dispatch_queue_limit,
-            maintenance: build_autoscale_maintenance,
-            logger: @logger,
-            # On reconnect the Listener rebuilds its OWN connection via this
-            # factory (fresh connect re-resolves DNS, converges on the promoted
-            # primary after a failover) instead of resetting a possibly-dead
-            # socket. Always provided — even when an initial pg_connection: is
-            # injected, the reconnect path builds a fresh raw connection. A test
-            # can inject its own factory to avoid touching real configuration.
-            connection_factory: connection_factory || -> { build_raw_pg_connection }
-          )
+          @listener = build_listener(pg_connection, connection_factory)
           # Off-thread durable fanout writer (issue #321). Built only when
           # streams_writer_threads > 0; nil means fanout writes stay inline on
           # the dispatcher thread (the default, pre-#321 behavior). The pump
@@ -169,6 +150,74 @@ module Pgbus
 
         private
 
+        # Selects the wake source by streams_listen_scope (issue #382).
+        # :master with a reachable hub socket → FailoverListener over a
+        # HubClient (NO per-worker LISTEN connection is opened). Anything
+        # else — scope :process, no socket exported (single mode,
+        # non-preforking server, hub failed to start), or a refused connect —
+        # keeps today's per-worker Listener.
+        def build_listener(pg_connection, connection_factory)
+          hub = build_hub_listener(connection_factory)
+          return hub if hub
+
+          build_local_listener(pg_connection || build_pg_connection, connection_factory)
+        end
+
+        def build_hub_listener(connection_factory)
+          return nil unless @config.streams_listen_scope == :master
+
+          socket_path = ENV.fetch("PGBUS_STREAMS_HUB_SOCKET", nil)
+          return nil if socket_path.nil? || socket_path.empty?
+
+          # The worker's ack deadline must exceed the master's own internal
+          # ensure_listening budget (its listener's health-check cycle + 1s).
+          failover = nil
+          client = HubClient.new(
+            socket_path: socket_path,
+            dispatch_queue: @dispatch_queue,
+            ack_timeout: (@config.streams_listen_health_check_ms / 1000.0) + 2.0,
+            # failover is assigned right below; a transport death in the gap
+            # is caught by the FailoverListener's synchronous ensure path.
+            on_failure: -> { failover&.fail_over! },
+            logger: @logger
+          )
+          client.connect
+          failover = FailoverListener.new(
+            hub_client: client,
+            local_listener_factory: lambda do
+              build_local_listener(build_pg_connection, connection_factory).tap(&:start)
+            end,
+            logger: @logger
+          )
+        rescue HubClient::HubUnavailableError => e
+          @logger.info do
+            "[Pgbus::Streamer] master hub not reachable (#{e.message}) — using a per-worker listener"
+          end
+          nil
+        end
+
+        def build_local_listener(pg_connection, connection_factory)
+          Listener.new(
+            pg_connection: pg_connection,
+            dispatch_queue: @dispatch_queue,
+            health_check_ms: @config.streams_listen_health_check_ms,
+            # Opt-in dispatch-queue backpressure (issue #315 item 3). 0 =
+            # unbounded (default). The queue itself stays an unbounded
+            # Queue.new so the request-thread Connect push and the dispatcher's
+            # own prune_dead self-post never block.
+            dispatch_queue_limit: @config.streams_dispatch_queue_limit,
+            maintenance: build_autoscale_maintenance,
+            logger: @logger,
+            # On reconnect the Listener rebuilds its OWN connection via this
+            # factory (fresh connect re-resolves DNS, converges on the promoted
+            # primary after a failover) instead of resetting a possibly-dead
+            # socket. Always provided — even when an initial pg_connection: is
+            # injected, the reconnect path builds a fresh raw connection. A test
+            # can inject its own factory to avoid touching real configuration.
+            connection_factory: connection_factory || -> { build_raw_pg_connection }
+          )
+        end
+
         def safely
           yield
         rescue StandardError => e