pgbus 0.11.3 → 0.12.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: df6f1cbda597599aaad006242fbb0b964edaa8fa599c413dfde949fc3b7e19a2
-  data.tar.gz: 4a57b939f12238bf4aac8fc601ee5cd529fd1311e6c6994dacfb55f925545e4b
+  metadata.gz: f2e4a0eacc67a1d3edf2fa7aae4ffd835cf03ae275f016aa92c2f490923897cd
+  data.tar.gz: defeb677222dc6ac50c755225e3da9b57752667e03569af747f1f8a26974b95d
 SHA512:
-  metadata.gz: 336b503f39a11252b461b98c124617d16f252a20e8ef343ea48d637d8552a6048ff8c0d9465d95fcb7c737616010da63669a8db3eacb9971d79b6ae9286ed432
-  data.tar.gz: 33d5b40c21ef5213f4014f27577e19c617ee0e71420dc3c3432c4e3d35f941cafbe0e6a41d33bac42cc9ec2eb65a159d3685f1cce28ff44e97a14ca9749134ab
+  metadata.gz: 2a9b8e1d6efa1106a5f56325d48bae1bcc650916154e54ccffa675fa9a5fde887f412536ca6d40d56e907ec804de66b3b1ca2000c5c66142889d9f8ff07dce3d
+  data.tar.gz: 0f5da2baedef6ad70d304e1d91339a6f7613687c642d40bb7b52fd508d198fde5840a22b2f42f2a0bbf3429aea7b9f8f1c71d20d63e35dbaa29b787b43fb4601

data/CHANGELOG.md

@@ -10,6 +10,9 @@
 
 ### Added
 
+- **`Pgbus::Client#reload` — operator escape hatch that drops every pooled PGMQ connection (job pool AND the live streams pool) and lets the pools rebuild lazily on next checkout (issue #354).** Built on pgmq-ruby 0.7.1's `PGMQ::Client#reload`; use it to recover connections libpq still reports as `CONNECTION_OK` but that are in fact wedged (e.g. after a wall-clock interrupt cut a query mid-flight), which pgmq-ruby's checkout health check cannot detect. Unlike `#close` the pools stay usable, and connections checked out by other threads mid-reload are unaffected. The streams half goes through the `ResizablePool` wrapper, so it always targets the live (possibly hot-swapped, #323) client. No-op with a warning on the shared-AR Proc path — those pool slots wrap ActiveRecord's own raw connection, and pgbus won't close a socket it doesn't own. Refs #354.
+- **PGMQ: vendored schema v1.12.0.** Adds `pgmq.read_grouped_head_with_poll` — a polling wrapper over `read_grouped_head` (reads the heads of N FIFO groups, waiting up to `max_poll_seconds` for messages); the 1.11.1 → 1.12.0 upstream delta is otherwise comments/whitespace only, so nothing pgbus calls through `Pgbus::Client` changes. Fresh embedded installs get 1.12.0; existing installs: `rake pgbus:pgmq:status`, then `rails generate pgbus:upgrade_pgmq` + `rails db:migrate` (`rails db:migrate:pgbus` on a separate-database install). No `Pgbus::Client` API is added for the new function (the locked pgmq-ruby does not expose it; pgbus has no consumer). Refs #351.
+- **Doctor check 10: "Dedicated connections" — the preflight now opens the streamer/notify-listener connections exactly the way the runtime does (issue #352).** The streamer's LISTEN connection and the worker `NotifyListener` bypass the Client's pgmq pools entirely (they own a single raw `PG` connection each), so every existing doctor check — all of which probe through `Pgbus::Client` — passed while both dedicated paths were broken. The new check builds each configured dedicated connection via `Pgbus::DedicatedConnection` (streams when `streams_enabled`, notify when `worker_notify_wakeup?`), runs `SELECT 1`, and closes it, failing with the underlying PG error and the affected path label. Not strict-fatal (same reasoning as the Database check: a transient DB blip at boot must not lockstep-abort a fleet), but with `doctor_on_boot` it turns "every SSE request 500s after deploy" into a named failing check in the boot report. Refs #352.
 - **`pgbus dashboard` — print the vendored AppSignal dashboards as import-ready JSON.** The gem ships four AppSignal dashboard definitions (the main one submitted to appsignal/public_config#80 plus the health/streams/throughput extras), but they're stored in the automated-dashboard format — a `metric_keys` trigger wrapping the `dashboard` object — which AppSignal's "Import dashboard" dialog doesn't accept, so trying one on a real app meant hand-unwrapping JSON. `pgbus dashboard [main|health|streams|throughput]` prints the inner dashboard object ready to paste into the import dialog (`pgbus dashboard health | pbcopy`); `pgbus dashboard --list` enumerates the available names and titles.
 - **`config.doctor_on_boot` (+ `pgbus start --doctor` / `--doctor-strict`) — run the doctor preflight inside the booting supervisor, one Rails boot instead of two (issue #347).** The recommended deploy preflight was `bin/pgbus doctor || true` in the container entrypoint before `bin/pgbus start` — but both commands `require config/environment`, so the job container boots the full Rails app **twice** on every deploy (cost scaling with app boot time), and running doctor pre-supervisor makes the `Process liveness` check false-fail (no workers exist yet), which is exactly why the entrypoint has to swallow the exit code with `|| true` — losing any chance to gate on a genuinely-fatal finding. Now `config.doctor_on_boot = :report` (or `pgbus start --doctor`) runs the checks in the already-booted supervisor, after the DB is verified reachable and queues are bootstrapped but **before any worker is forked**: one boot, and the worker-dependent `Process liveness` check is skipped (it has no workers to observe yet and would false-fail on stale prior-generation rows during a redeploy). `config.doctor_on_boot = :strict` (or `--doctor-strict`) additionally **refuses to boot** — raising before forking anything, so the `ensure shutdown` path tears down the heartbeat/health server and no child starts — but *only* on a genuinely-fatal, non-transient check: a `Configuration` failure (a real config bug) or an **absent** PGMQ schema. It deliberately does **not** abort on `Queues`/`Database` failures — those are the transients the lenient queue bootstrap is built to ride out (children crash-and-backoff until the DB recovers), and `verify_connection!` already gated a hard-down DB moments earlier; making them strict-fatal would take down a whole fleet's cold boot in lockstep on a momentarily-saturated primary. Default `nil`/`false` = off (byte-identical to today; the supervisor constructs no `Doctor` and does zero extra work). The standalone `pgbus doctor` command and its exit-code semantics are unchanged. Refs #347.
 - **`Pgbus::Streams::PhlexHelpers` — a Phlex-includable `pgbus_stream_from`, and `Pgbus.stream_name_budget` (issue #334).** Two small helpers that stop consuming apps reimplementing pgbus glue. (1) `pgbus_stream_from` is a Rails view helper, so a Phlex component couldn't call it without hand-registering an output-helper macro (phlex-rails ships no `Phlex::Rails::Helpers::PgbusStreamFrom`). `include Pgbus::Streams::PhlexHelpers` now bridges it exactly like phlex-rails' own `TurboStreamFrom`. phlex-rails stays an **optional** dependency — the module references its `HelperMacros` and is loaded on demand (`require "pgbus/streams/phlex_helpers"`), never eagerly. (2) `Pgbus.stream_name_budget` exposes the maximum stream-name length (the pgmq queue-name cap minus the queue prefix) so apps size/truncate stream identifiers up front instead of hand-computing `MAX_QUEUE_NAME_LENGTH - queue_prefix.length - 1` at every call site. Refs #334.
@@ -28,12 +31,15 @@
 
 ### Changed
 
+- **Streaming/notify failures now reach `config.error_reporters`, not just the log (issue #352).** Four error-level rescues were logger-only, so an error tracker (AppSignal/Sentry/…) never saw them: a `StreamApp` request rescued into a 500 (which, for a broken streamer config, is *every* SSE request on the worker), a `NotifyListener` fatal thread exit, a `NotifyListener` reconnect failure, and a streamer `Listener` reconnect failure. All four now go through `Pgbus::ErrorReporter.report` with an `action:` context tag (`stream_connect`, `notify_listener_fatal`, `notify_listener_reconnect`, `streamer_reconnect`); ErrorReporter logs as before, so operators without reporters lose nothing. Refs #352.
 - **Fanout socket writes now use a 250ms deadline (was the full 5s `streams_write_deadline_ms`), so a transiently-slow SSE client is marked dead ~20× sooner during broadcast fanout.** This is the behavior change behind the head-of-line-blocking fix (see Added, #315): a client that can't absorb a broadcast frame within `streams_fanout_write_deadline_ms` (default 250ms) is dropped and reconnects, replaying the missed frames from the durable archive (durable streams) or getting a fresh re-render (ephemeral). Connect-time replay is unaffected (keeps the full deadline). To restore the previous timing, set `config.streams_fanout_write_deadline_ms = config.streams_write_deadline_ms`. Refs #315.
 - **The generated `config/initializers/pgbus.rb` documents every config group and bakes in realtime broadcast isolation.** The initializer that `rails generate pgbus:install` now writes (see Breaking Changes for the YAML removal) documents each config group inline — queues, pool, retries, recycling, dispatcher, event consumers — with the gem defaults shown as commented reference, and it sets the #311 realtime broadcast isolation by default: `c.streams_broadcast_queue = "realtime"` plus a dedicated `c.capsule :realtime` that drains it (so `pgbus doctor` stays quiet — no undrained-queue footgun). The **code default in `configuration.rb` stays `nil`**, so a programmatic `Pgbus.configure` and existing installs are unchanged; the recommended setup lives in the generated Ruby. Refs #317.
 - **1.0 API freeze: config renames as deprecated aliases, and removal of provably-dead surface.** Renaming now is cheap; renaming after 1.0 is a major-version break, so the semver-audit renames land as **deprecated aliases** (old name works, warns once, removed in 2.0): `skip_recurring` → `recurring_enabled` (positive polarity, the only negative toggle among the `*_enabled` switches — the alias inverts the boolean); `dashboard_filter_parameters`/`dashboard_filter_sensitive` → `web_filter_parameters`/`web_filter_sensitive` (unify on the incumbent `web_` prefix); and `recurring_tasks_file` (singular), which now warns once when set alongside the plural `recurring_tasks_files` instead of being silently ignored. Both spellings of each alias resolve to the same setter. **Removed** (each verified zero-caller in the audit): the `lock_ttl:` keyword on `ensures_uniqueness` (validated and stored in metadata but read by nothing — passing it now raises `ArgumentError` naming the removal and the upgrade guide); the `pgbus:add_job_locks` generator and the `Pgbus::JobLock` model (new installs use `pgbus:add_uniqueness_keys`; `pgbus:migrate_job_locks` still retires the legacy `pgbus_job_locks` table); the internal `with_pgbus_durable` streams helper (use `with_pgbus_broadcast_opts(durable:)`); and the streamer's `reconnect_via_reset` fallback (`connection_factory` is now required and always injected, so reconnect always rebuilds a fresh connection). `log_format=` no longer clobbers a custom logger's formatter, and `streams_presence_patterns`/`streams_presence_member`, `group_mode`, and `streams_falcon_streaming_body` are annotated **experimental** (exempt from the 1.0 stability promise). The `workers=` Array form is documented as permanent (not "legacy") — it is the only way to express N identical anonymous capsules. Refs #283.
 
 ### Fixed
 
+- **The Ruby-Timeout read fallback no longer leaves a wedged `CONNECTION_OK` connection in the pgmq pool to re-hang the next read (issue #354).** On the one path where libpq can't bound a hung socket (dedicated connection on non-Linux hosts or libpq < 12), the last-resort `Timeout.timeout` interrupts the read via `Thread#raise` — and libpq may leave the pooled `PG::Connection` reporting `CONNECTION_OK` while it will in fact re-hang on reuse, invisible to pgmq-ruby's checkout health check (it isn't `CONNECTION_BAD`). This was a documented KNOWN LIMITATION waiting on a public pool-reload upstream (cf. mensfeld/pgmq-ruby#94); pgmq-ruby 0.7.1 shipped it, so the fallback now raises an internal `ReadTimeoutError` subclass and reloads the job pool before re-raising — the poisoned connection (already checked back in by `connection_pool`'s ensure) is dropped and the pool rebuilds lazily. A clean server-side `statement_timeout` cancel still never triggers a reload (the connection is healthy), and a reload failure is logged without masking the timeout. Requires pgmq-ruby >= 0.7.1 (dependency bumped to `~> 0.7.1`). Refs #354.
+- **`connection_guc_mode = :session` no longer kills every SSE stream and NOTIFY wakeup with `PG::Error: invalid connection option "variables"` (issue #352).** In `:session` mode, `forward_connection_variables` deliberately leaves the database.yml `variables:` hash on the connection options for the *caller* to strip and apply via post-connect `SET` (a transaction-mode pooler rejects the libpq `options` startup param — the reason `:session` mode exists). `Pgbus::Client#wrap_session_gucs` honors that contract on both pgmq pools, but the two dedicated raw-connect paths — the streamer's `build_raw_pg_connection` and the worker `NotifyListener` — passed the hash straight to `PG.connect`, which rejects the non-libpq `:variables` key. Result: the first stream request per process raised, `StreamApp` rescued it into a 500 (so **every** `/pgbus/streams/…` request failed from then on), and NOTIFY-gated wakeups died with workers silently falling back to polling. Both paths now connect through the new `Pgbus::DedicatedConnection.connect`, which strips `:variables` and applies each GUC via post-connect `SET` — dedicated connections *keep* the operator's GUCs (`statement_timeout`, `timezone`, …), matching the pooled paths, and the mechanism works on both a direct port and a session-mode pooler. A guard spec confines raw `PG.connect` call sites to `Client` and `DedicatedConnection`, so the next dedicated-connection path can't silently reintroduce the bypass. Refs #352.
 - **AppSignal main dashboard: job and event lines now split by `job_class` and `routing_key`.** Applied the upstream review suggestions committed on appsignal/public_config#80 to the vendored `dashboard.json`: the "Job status per queue" panel gains a `job_class` wildcard tag filter and label (`%job_class% - %queue% - %status%`), and "Event handler status" gains `routing_key` (`%handler% - %status% - %routing_key%`). Without the wildcard tag entry a `%placeholder%` in the line label never resolves, so lines from different job classes / routing keys were collapsed together. The subscriber already emitted both tags; only the dashboard definition lagged.
 - **Migration generators now route to the pgbus database when `connects_to` is configured, even without an explicit `--database` (issue #344).** `rails g pgbus:add_stream_queues` (and every sibling generator via `Generators::MigrationPath`) keyed the separate-database decision purely on the `--database` flag. An app that had already set `config.connects_to = { database: { writing: :pgbus } }` (so its pgbus migrations live in `db/pgbus_migrate/` per database.yml's `migrations_paths:`) still got the migration written to the **primary** DB's `db/migrate/` on a bare invocation — where it then ran against the wrong database — and the "Next steps" output told the operator to run `rails db:migrate` instead of `rails db:migrate:pgbus`. `MigrationPath` now auto-detects the target database from `connects_to` (via the existing `DatabaseTargetDetector`: runtime config → initializer scan → application.rb scan) when `--database` is absent; an explicit `--database` still wins. The detected name drives both the migration path (resolving `migrations_paths` for that database, falling back to the `db/pgbus_migrate` convention) and the `db:migrate:<name>` suffix in every generator's post-install output. An app with no separate database is unaffected. Refs #344.
 - **`extract_ar_connection_hash` no longer forces `host: "localhost"` / `port: 5432` on a socket-based (host-less) database.yml — pgmq's connections now match ActiveRecord's on Unix-socket dev setups (issue #343).** A local database.yml with no `host:` is a Unix-socket connection: ActiveRecord connects via libpq's default socket (`PGHOST` / the default socket dir), but pgbus's AR-config extraction defaulted the absent `host`/`port` to TCP `localhost:5432`, silently diverging from AR and pointing pgmq's dedicated raw connections at a *different* server (or nothing) on any machine where the socket dir isn't `localhost`. Apps on socket-based dev DBs therefore couldn't use the AR-extraction path at all and had to pin an explicit `connection_params` Hash whose only real job was *not* defaulting `host` (cosmos carried an `after_initialize` block for exactly this). The two fallbacks are dropped: `host`/`port` now pass through as-is and `.compact` removes them when absent, so libpq applies its own socket defaults and matches AR. A config that *does* set `host`/`port` is byte-identical to before. Refs #343.

data/lib/pgbus/client/resizable_pool.rb

@@ -101,6 +101,15 @@ module Pgbus
         end
       end
 
+      # Drop every pooled connection on the CURRENT streams client and let it
+      # rebuild lazily on next checkout (pgmq-ruby >= 0.7.1) — see
+      # Client#reload (issue #354). Takes the swap mutex so a reload cannot
+      # race a concurrent swap/close into reloading a pool that is being
+      # retired; reads stay lock-free.
+      def reload
+        @swap_mutex.synchronize { @ref.get.pgmq.reload }
+      end
+
       def stats_snapshot
         @last || SwapStats.new(swap_count: 0, last_drain_seconds: 0.0, last_conns_closed: 0,
                                last_from_size: nil, last_to_size: nil, last_drained: nil)

data/lib/pgbus/client.rb

@@ -794,6 +794,32 @@ module Pgbus
       @streams_pool.stats_snapshot
     end
 
+    # Operator escape hatch (issue #354): drop every pooled PGMQ connection —
+    # job pool AND the live streams pool — and let the pools rebuild lazily on
+    # next checkout (pgmq-ruby >= 0.7.1). Use to recover connections libpq
+    # still reports as CONNECTION_OK but that are in fact wedged (e.g. after a
+    # wall-clock interrupt cut a query mid-flight), which pgmq-ruby's checkout
+    # health check cannot detect. Unlike #close, the pools stay usable.
+    # Connections checked out by other threads mid-reload are unaffected.
+    #
+    # No-op (returns false) on the shared-AR Proc path: those pool slots wrap
+    # ActiveRecord's own raw connection — reloading would close AR's socket
+    # out from under the application. Returns true after a reload.
+    def reload # rubocop:disable Naming/PredicateMethod -- command that reports whether it acted, like #ping
+      if @shared_connection
+        Pgbus.logger.warn do
+          "[Pgbus::Client] reload skipped: pgbus is sharing ActiveRecord's connection " \
+            "(Proc connection_options) and won't close a socket it doesn't own. " \
+            "Manage that connection through ActiveRecord instead."
+        end
+        return false
+      end
+
+      @pgmq.reload
+      @streams_pool.reload
+      true
+    end
+
     private
 
     # Human-readable label for which config knob supplied the connection
@@ -1146,6 +1172,18 @@ module Pgbus
     READ_TIMEOUT_SLACK = 5
     private_constant :READ_TIMEOUT_SLACK
 
+    # Raised (instead of plain ReadTimeoutError) when the Ruby Timeout last
+    # resort fires — i.e. Thread#raise interrupted a read mid-flight on a
+    # socket libpq couldn't bound (issue #354). IS-A Pgbus::ReadTimeoutError,
+    # so the public contract is unchanged; the distinct class lets
+    # with_read_timeout tell "wedged socket — reload the pool" (this) apart
+    # from "clean server-side statement_timeout cancel — connection healthy"
+    # (plain ReadTimeoutError), where reloading would churn a healthy pool on
+    # every slow query. Internal signal carried on the unwind path itself (no
+    # thread-local/ivar state around a Thread#raise interrupt); application
+    # code should rescue Pgbus::ReadTimeoutError.
+    class WedgedReadTimeout < Pgbus::ReadTimeoutError; end
+
     # Bound a read and surface a timeout as Pgbus::ReadTimeoutError. Prefer
     # libpq-native bounds baked into the connection; the Ruby Timeout is a
     # narrow, last-resort fallback used only where libpq cannot bound a hung
@@ -1179,12 +1217,19 @@ module Pgbus
     #      which AR passes straight through to the connection. #initialize logs a
     #      one-time hint when read_timeout is set on a Proc connection.
     #
-    #      KNOWN LIMITATION: when (3) fires on a genuinely hung socket, libpq may
-    #      leave the pooled PG::Connection reporting CONNECTION_OK while it will
-    #      in fact re-hang on reuse, and pgmq-ruby's health check won't discard
-    #      it (it isn't CONNECTION_BAD). The proper fix is a public pool-reload on
-    #      pgmq-ruby (follow-up, cf. mensfeld/pgmq-ruby#94); until then it's
-    #      documented and confined to the non-Linux dedicated path.
+    #      WEDGED-SOCKET RECOVERY (issue #354): when (3) fires on a genuinely
+    #      hung socket, libpq may leave the pooled PG::Connection reporting
+    #      CONNECTION_OK while it will in fact re-hang on reuse, and pgmq-ruby's
+    #      checkout health check won't discard it (it isn't CONNECTION_BAD). So
+    #      the Timeout raises WedgedReadTimeout (a ReadTimeoutError subclass)
+    #      and the rescue below drops every pooled connection via @pgmq.reload
+    #      (pgmq-ruby >= 0.7.1) — the pool rebuilds lazily on next checkout.
+    #      By the time the rescue runs, Thread#raise has unwound the read and
+    #      connection_pool's ensure has checked the poisoned connection back in
+    #      as idle, so reload does discard it. A small window remains where
+    #      another thread checks it out first; that thread's own read bound /
+    #      stale-retry covers it — reload narrows the window, it doesn't need
+    #      to close it atomically.
     #
     # MUST wrap only the bare `@pgmq.read*` call, inside both `synchronized` and
     # `with_stale_connection_retry`, so the Timeout clock starts only after the
@@ -1203,10 +1248,26 @@ module Pgbus
       return mapping_statement_timeout(&block) unless timeout&.positive?
 
       # rubocop:disable Pgbus/NoRubyTimeout -- deliberate last-resort bound; see above
-      Timeout.timeout(timeout + READ_TIMEOUT_SLACK, Pgbus::ReadTimeoutError) do
+      Timeout.timeout(timeout + READ_TIMEOUT_SLACK, WedgedReadTimeout) do
         mapping_statement_timeout(&block)
       end
       # rubocop:enable Pgbus/NoRubyTimeout
+    rescue WedgedReadTimeout
+      reload_pool_after_wedged_timeout
+      raise
+    end
+
+    # Best-effort job-pool reload after the Ruby Timeout fallback interrupted a
+    # read (see with_read_timeout). A reload failure is logged, never raised —
+    # the caller is already unwinding with ReadTimeoutError, which is the
+    # actionable error; masking it with a secondary pool failure would hide
+    # which read timed out.
+    def reload_pool_after_wedged_timeout
+      @pgmq.reload
+    rescue StandardError => e
+      Pgbus.logger.warn do
+        "[Pgbus::Client] pool reload after wedged read timeout failed: #{e.class}: #{e.message}"
+      end
     end
 
     # True when libpq's connection-baked read bounds (statement_timeout +

data/lib/pgbus/dedicated_connection.rb

@@ -0,0 +1,53 @@
+# frozen_string_literal: true
+
+module Pgbus
+  # Single choke point for opening a DEDICATED PG connection outside the
+  # pgmq pools — the streamer's LISTEN connection and the worker
+  # NotifyListener. Every such path MUST route through here (enforced by
+  # spec/pgbus/pg_connect_guard_spec.rb): in :session GUC mode,
+  # Configuration#forward_connection_variables leaves the database.yml
+  # `variables:` hash on the connection options for the caller to apply
+  # post-connect, and :variables is not a libpq keyword — a raw
+  # PG.connect(**opts) fails with `invalid connection option "variables"`
+  # (issue #352). This mirrors Client#wrap_session_gucs, which does the
+  # same for pool connections: the GUCs are applied via post-connect SET
+  # because a transaction-mode pooler rejects the libpq `options` startup
+  # param (the reason :session mode exists).
+  module DedicatedConnection
+    module_function
+
+    def connect(opts)
+      require "pg" unless defined?(::PG::Connection)
+      case opts
+      when String then ::PG.connect(opts)
+      when Hash then connect_from_hash(opts)
+      else
+        raise Pgbus::ConfigurationError,
+              "Cannot build a dedicated PG connection from #{opts.class}. " \
+              "Set database_url or connection_params so pgbus can open its own connection."
+      end
+    end
+
+    def connect_from_hash(opts)
+      variables = opts[:variables]
+      conn = ::PG.connect(**opts.except(:variables))
+      begin
+        variables&.each { |name, value| conn.exec("SET #{name} = '#{value}'") }
+      rescue StandardError
+        # A failing SET (e.g. a bogus GUC name in database.yml variables:)
+        # must not orphan the freshly opened socket — the reconnect loops
+        # retry on a tight backoff and would leak one server connection per
+        # attempt until PostgreSQL exhausts max_connections.
+        close_quietly(conn)
+        raise
+      end
+      conn
+    end
+
+    def close_quietly(conn)
+      conn.close
+    rescue StandardError
+      nil
+    end
+  end
+end

data/lib/pgbus/doctor.rb

@@ -5,16 +5,19 @@ require "pgbus/mcp/health_analyzer"
 
 module Pgbus
   # Preflight diagnostics for a pgbus deployment — the single command that
-  # answers "is this environment healthy enough to run?". Runs nine checks and
+  # answers "is this environment healthy enough to run?". Runs ten checks and
   # returns a machine-readable result plus a human report, so `pgbus doctor`
   # and `rake pgbus:doctor` can gate a deploy or CI run (exit 0 on success,
   # 1 on any failure).
   #
-  # Doctor never touches PGMQ or PostgreSQL directly: every probe goes through
-  # Pgbus::Client (DB, PGMQ schema, queues, NOTIFY) or Pgbus::Web::DataSource
-  # (process liveness, via Pgbus::MCP::HealthAnalyzer). It also never raises —
-  # a broken environment turns into :fail results, never a crash — so it is
-  # safe to run against a database that is down or a half-installed schema.
+  # Doctor probes through Pgbus::Client (DB, PGMQ schema, queues, NOTIFY),
+  # Pgbus::Web::DataSource (process liveness, via Pgbus::MCP::HealthAnalyzer),
+  # or Pgbus::DedicatedConnection (the streamer/notify-listener check — a
+  # deliberate exception to "everything via Client": those runtime paths
+  # bypass the Client's pools, so probing via the Client cannot catch a
+  # broken dedicated path; see issue #352). It never raises — a broken
+  # environment turns into :fail results, never a crash — so it is safe to
+  # run against a database that is down or a half-installed schema.
   class Doctor
     # A single check result. status is one of :ok, :warn, :fail.
     Check = Struct.new(:name, :status, :detail) do
@@ -36,7 +39,8 @@ module Pgbus
       "Process liveness" => :check_processes,
       "GlobalID allowlist" => :check_allowed_global_id_models,
       "Broadcast queue" => :check_broadcast_queue,
-      "Primary affinity" => :check_primary
+      "Primary affinity" => :check_primary,
+      "Dedicated connections" => :check_dedicated_connections
     }.freeze
 
     # Process liveness reads the pgbus_processes table (via HealthAnalyzer), so
@@ -331,6 +335,51 @@ module Pgbus
       Check.new(name: "Primary affinity", status: :warn, detail: "could not determine (#{e.class}: #{e.message})")
     end
 
+    # 10. Dedicated connections — the streamer's LISTEN connection and the
+    # worker NotifyListener bypass the Client's pgmq pools and connect via
+    # DedicatedConnection, so a broken dedicated path is invisible to every
+    # other check (issue #352: :session-mode `:variables` broke ONLY these
+    # connections while every Client path worked). Open each configured
+    # dedicated connection exactly the way the runtime does, probe it, close
+    # it. NOT strict-fatal, same reasoning as the Database check: a transient
+    # DB blip at boot must not lockstep-abort a fleet.
+    def check_dedicated_connections
+      targets = []
+      targets << ["streams", @config.streams_connection_options] if @config.streams_enabled
+      targets << ["worker notify", @config.worker_notify_connection_options] if @config.worker_notify_wakeup?
+
+      if targets.empty?
+        return Check.new(name: "Dedicated connections", status: :ok,
+                         detail: "disabled in config (streams + notify wakeup off)")
+      end
+
+      failures = targets.filter_map { |label, opts| probe_dedicated_connection(label, opts) }
+      if failures.empty?
+        Check.new(name: "Dedicated connections", status: :ok,
+                  detail: "#{targets.map(&:first).join(" + ")} connect OK")
+      else
+        Check.new(name: "Dedicated connections", status: :fail, detail: failures.join("; "))
+      end
+    rescue StandardError => e
+      Check.new(name: "Dedicated connections", status: :fail, detail: "#{e.class}: #{e.message}")
+    end
+
+    # Open one dedicated connection the way the runtime does, verify it
+    # answers, close it. Returns nil on success, "label: error" on failure.
+    def probe_dedicated_connection(label, opts)
+      conn = Pgbus::DedicatedConnection.connect(opts)
+      conn.exec("SELECT 1")
+      nil
+    rescue StandardError => e
+      "#{label}: #{e.class}: #{e.message}"
+    ensure
+      begin
+        conn&.close if conn.respond_to?(:close)
+      rescue StandardError
+        nil
+      end
+    end
+
     # True when some configured worker capsule drains the given queue — either
     # by naming it explicitly or via a "*" wildcard (which drains every queue).
     def worker_drains?(queue)