pgbus 0.12.2 → 0.12.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3393ad40ec489306eff74a6504c6694aa27747e5e0d2ce533ba4b7d7e989fe93
-  data.tar.gz: b1e7244a064bfa8fe1776a5eaa9dd7317682e098bf98133759eadc0c6402e4c8
+  metadata.gz: b79edc460be04508034f7c4b34ba889f33493ab88878b3a272cf23d7f75d43ee
+  data.tar.gz: 9a000c729dfa32c5e3e6bb8b8df751fe75e159ff0a2160afee104b201e4f78e1
 SHA512:
-  metadata.gz: 440cfbc362912153a1cfb694033e667b32559a965b5b2b8c99214d11bc06a09d717127e8616a49dc59858f6a8048619a0d4a0267398f5cccf78019ca2f0dedf3
-  data.tar.gz: 45163ef47567b3ff8be5e4c415311f6619d11a4712ecbc970aafe7e949cc09ebd75632727ebf6b7817c124cfbb9dec0cf3c9d29c41061d37704048c9522d6851
+  metadata.gz: 3ca04a2bb7436c7fc08f953b5fa19cc5130bb36de9d678287ffc49768bd6e3f142266a547e4ca852377b0fd66ce15cf18f1eac34857f75365dd62f9c7c10a754
+  data.tar.gz: 101a052d947241a6e35b80093846c20bfaec42454e40767073ad6cde8e6f64e09f3ae1ef879bd4686482c8b5e355d4770ddcba7ed61672465e7867b9382925f4

data/CHANGELOG.md

@@ -1,7 +1,14 @@
 ## [Unreleased]
 
+### Changed
+
+- **README: doctor table and configuration reference catch up to the shipped surface (issue #369).** The doctor section still said "six checks" and listed only the original six; `Pgbus::Doctor::CHECKS` has 10 (GlobalID allowlist, Broadcast queue, Primary affinity, Dedicated connections were missing — the first two are the ones most likely to warn after upgrade on nil defaults). The configuration reference table was missing ~20 shipped options operators actually hit: `allowed_global_id_models` (incl. `[]` = deny-all), streams GC (`streams_orphan_threshold` / `streams_orphan_sweep_interval` — 24h / hourly defaults that auto-drop dormant durable stream queues), retention/durable/presence/broadcast-queue knobs, `streams_pool_*` / LISTEN connection overrides, `doctor_on_boot`, `require_primary`, `connects_to`, `zombie_detection`, `connection_guc_mode`, `worker_notify_*`, `group_mode`, and neighbors. A unit drift guard pins the README doctor table to every `Doctor::CHECKS` name. Full typed reference remains on the docs site. Refs #369.
+
 ### Fixed
 
+- **The health verdict no longer emits false STALLED reports — `max_read_ct` was never populated, and the wedge signal counted queues no capsule drains (issue #367).** Two correctness defects in `Pgbus::MCP::HealthAnalyzer` (surfaced through `pgbus doctor` / `pgbus_health` / the MCP health tool). **(1)** The `all_unread?` wedge check read `:max_read_ct`, but the metrics query never selected it — so the guard always degenerated to "never claimed" and a busy-but-healthy queue caught mid-burst produced a STALLED verdict with a factually wrong "read_ct=0 (never claimed)" reason. The metrics query now exposes a per-queue **`visible_unread_length`** (`count(*) WHERE vt <= NOW() AND read_ct = 0`) and the analyzer keys the wedge off *visible, never-claimed* messages. Counting per visible message (not `max(read_ct)` over the whole table) means one retried message left in-queue after its backoff, or an in-flight message claimed by a peer, can no longer veto the signal for a pile of genuinely-unclaimed jobs. **(2)** The verdict reasoned about **every** non-DLQ/non-stream queue against the global worker fleet — but a worker can only claim from queues its capsule subscribes to, so "M workers alive but never claimed" was vacuous for a queue nobody drains (ad-hoc queues, unregistered stream queues — see #366). The analyzer now intersects the STALLED backlog with `Web::DataSource#drained_queue_names` (each configured capsule's queues, priority `_pN` sub-tables expanded via the client's queue strategy, unioned with EventBus handler queues; `nil` for a `*` wildcard = drains everything, fail-open on error). Queues nobody drains get their own DEGRADED signal — *"N queue(s) hold messages but no capsule is configured to drain them"* — instead of being folded into the worker-wedge verdict. A heart-beating-but-`:stalled` worker is still reported STALLED regardless of which queue holds the backlog. Reason strings now truncate to the first 10 queue names with `(+N more)` so a flagged fleet of hundreds of queues doesn't produce a multi-KB log line. Refs #367, #366.
+- **`allowed_global_id_models` now actually guards ActiveJob arguments, not only EventBus payloads (issue #368).** The doctor warned in production that `nil` means "allow-all GlobalID arguments", but the allowlist was only enforced in `Serializer.locate_global_id` — reached from EventBus `_global_id` payloads — while the ordinary job path (`Executor` → `ActiveJob::Base.deserialize` → Rails' unrestricted `GlobalID::Locator`) never checked it. Operators who set an allowlist after following the doctor had a false sense of security; the common `SomeJob.perform_later(record)` pattern was unguarded. Job deserialization now goes through `Serializer.deserialize_job_data`, which walks `_aj_globalid` keys (including nested arrays/hashes) and reuses the same gate as EventBus when the allowlist is set; `nil` remains zero-cost allow-all. Rejected models raise `Pgbus::SerializationError` and are treated as a normal job failure. Apps with ActiveStorage attachments should include `ActiveStorage::Blob` (and related models) on the allowlist. Docs + doctor copy updated. Refs #368.
+- **Dormant pre-registry stream queues no longer false-STALL the doctor or escape the orphan sweep (issue #366).** Stream queues created before the `pgbus_stream_queues` migration only register on their next broadcast — completed checkout-style flows, ended chats, and one-shot progress streams never broadcast again, so they stay unregistered forever. Health/doctor then treated their permanent `read_ct=0` visible backlog as a worker wedge (STALLED on ~hundreds of healthy stream queues), and the orphan sweep skipped them because it only iterated the registry. Detection is fingerprint-based (the archive `a_<queue>_msg_id_idx` index that only `ensure_stream_queue` creates — no name heuristics). `StreamQueue.known_names` = registry ∪ fingerprints, used by health exclusion, orphan sweep / stream-archive prune, and wildcard workers; `rake pgbus:streams:backfill_registry` persists the missing rows; doctor surfaces a DEGRADED hint pointing at the backfill when unregistered fingerprints remain; `add_stream_queues` generator post-install mentions the task. Refs #366.
 - **`ensure_stream_queue` recovers when a process-local `@queues_created` memo is stale after another process drops the physical queue.** Stream queues are created lazily and memoized per process. The dispatcher orphan stream sweep (and dashboard / manual `drop_queue`) only clears the *local* process memo when it drops an empty or aged stream queue. Peer processes — typically long-lived web workers that already published or subscribed once — keep the memo, skip `pgmq.create` on the next durable broadcast, and fail in `pgmq.enable_notify_insert` with `Queue "…" does not exist. Create it first using pgmq.create()`. That exception often surfaces from an `after_commit` Turbo/stream broadcast and turns a successful write into a 500. On that specific missing-queue `PGMQ::Errors::ConnectionError`, `ensure_stream_queue` now forgets the local queue + archive-index memo and retries create/notify once. Unrelated connection errors still raise immediately. Happy path: no extra round-trips (recovery is error-path only).
 - **`streams_pool_database_url` / `streams_pool_host` / `streams_pool_port` — route the streams PGMQ pool independently of the streamer's LISTEN connection (issue #358).** 0.12.0 started building the dedicated streams pool from `streams_connection_options` (correct for a separate streams database), but that method is also how pooler-bypass installs pin the LISTEN connection to the **direct** Postgres port (`streams_port = 5432`, the documented "workers go through PgBouncer, streamer goes direct" pattern) — so upgrading silently moved up to `streams_pool_size` connections **per process** onto the direct port, whose `max_connections` ceiling on managed Postgres is typically low. A modest fleet exhausts it (`FATAL: remaining connection slots are reserved for roles with the SUPERUSER attribute`): `StreamApp` then 500s every SSE connect and durable broadcasts fail at publish, intermittently, because the pools are lazy. Only LISTEN actually needs the direct port (it dies at transaction-pool COMMIT boundaries); the pool's broadcast INSERTs and replay reads are plain pooler-safe SQL. The new `streams_pool_*` triple routes the POOL independently — applied to the base options exactly like the `streams_*` and `worker_notify_*` groups: set `streams_pool_port` back to the pooled port and only the LISTEN pins remain direct. Default `nil` = follows `streams_connection_options`, byte-identical to 0.12.0, so separate-streams-DB installs are unchanged. Refs #358.
 - **The health verdict no longer reads durable stream queues as a wedged fleet (issue #359).** The Process-liveness signal (`pgbus doctor`, `pgbus_health`, the MCP health tool) treats *visible messages with `read_ct=0` while workers are alive* as the silent-worker-wedge signature — but durable stream delivery is a non-consuming peek, so **every** stream queue matches it permanently by design. On a streams-heavy install the verdict screamed STALLED listing hundreds of healthy stream queues, burying real wedges in noise. `HealthAnalyzer` now excludes queues registered in the `pgbus_stream_queues` registry from the operational set (exactly like DLQs) — out of the STALLED/DEGRADED reasons and the backlog totals — via a new `Web::DataSource#stream_queue_names` (loaded fresh per verdict; degrades to an empty set on pre-registry installs). Same bug class as #308/#309, same registry cure. Refs #359.

data/README.md

@@ -1862,7 +1862,7 @@ pgbus help      # Show help
 
 #### pgbus doctor
 
-A single preflight command that answers "is this environment healthy enough to run?" — useful as a deploy or CI gate. It runs six checks and never raises; a broken environment turns every probe into a failed/warned check instead of a crash:
+A single preflight command that answers "is this environment healthy enough to run?" — useful as a deploy or CI gate. It runs 10 checks and never raises; a broken environment turns every probe into a failed/warned check instead of a crash:
 
 | Check | Fails (`:fail`) when | Warns (`:warn`) when |
 |---|---|---|
@@ -1872,6 +1872,10 @@ A single preflight command that answers "is this environment healthy enough to r
 | Queues | A configured queue has no PGMQ table | — |
 | LISTEN/NOTIFY | — | A configured queue is missing its insert trigger (falls back to polling) |
 | Process liveness | Verdict is `STALLED` | Verdict is `DEGRADED` |
+| GlobalID allowlist | — | `allowed_global_id_models` is `nil` (allow-all) in production |
+| Broadcast queue | — | Turbo broadcasts share the default queue in production, or `streams_broadcast_queue` is set but no worker capsule drains it |
+| Primary affinity | — | Job connection is on a read-only replica (`pg_is_in_recovery`) — a read/write-splitting pooler may be stalling jobs |
+| Dedicated connections | Streamer LISTEN and/or worker notify dedicated path cannot connect | — |
 
 ```bash
 pgbus doctor       # prints the report; exit 1 unless every check passed
@@ -2058,12 +2062,19 @@ PostgreSQL + PGMQ
 
 ### Configuration reference
 
+Curated headline options for the README. The full operator reference (with types, groups, and drift-checked accessors) lives on the [Configuration](https://pgbus.zoolutions.llc/docs/configuration) docs page.
+
 | Option | Default | Description |
 |--------|---------|-------------|
 | `database_url` | `nil` | PostgreSQL connection URL (auto-detected in Rails) |
+| `connection_params` | `nil` | Extra connection parameters merged into the pool |
+| `connects_to` | `nil` | Rails multi-database config for a dedicated pgbus database (`{ database: { writing: :pgbus } }`) |
+| `require_primary` | `false` | Reject a job connection that lands on a read-only replica (`pg_is_in_recovery`) at boot — pooler safety against a read/write splitter |
+| `connection_guc_mode` | `:options` | How database.yml GUCs (`variables:`) reach pgbus connections — `:options` (libpq startup) or `:session` (post-connect `SET`, for transaction-mode PgBouncer) |
 | `queue_prefix` | `"pgbus"` | Prefix for all PGMQ queue names |
 | `default_queue` | `"default"` | Default queue for jobs without explicit queue |
 | `pool_size` | `nil` (auto) | Connection pool size. Auto-tuned from worker thread counts: `sum(workers.threads) + sum(event_consumers.threads) + 2`. Set explicitly to override. |
+| `pool_timeout` | `5` | Seconds to wait for a pooled connection |
 | `workers` | `[{queues: ["default"], threads: 5}]` | Worker capsule definitions. String DSL (`"default: 5; critical: 10"`), Array, or `nil`. |
 | `event_consumers` | `nil` | Event consumer process definitions (same format as workers) |
 | `roles` | `nil` (all) | Supervisor role filter — usually set via CLI flags (`--workers-only` etc.) |
@@ -2077,18 +2088,25 @@ PostgreSQL + PGMQ
 | `max_memory_mb` | `nil` | Recycle worker when memory exceeds N MB |
 | `max_worker_lifetime` | `nil` | Recycle worker after N seconds. Accepts seconds or Duration. |
 | `listen_notify` | `true` | Use PGMQ's LISTEN/NOTIFY for instant wake-up |
+| `worker_notify_wakeup` | `nil` (follows `listen_notify`) | Dedicated LISTEN connection for worker wake-up. `nil` follows `listen_notify`; set `false` to force polling. |
+| `worker_notify_host` / `worker_notify_port` / `worker_notify_database_url` | `nil` | Override host/port/URL for the worker notify LISTEN connection (e.g. direct primary port when jobs go through PgBouncer) |
 | `prefetch_limit` | `nil` | Max in-flight messages per worker (nil = unlimited) |
 | `dispatch_interval` | `1.0` | Seconds between dispatcher maintenance ticks |
 | `circuit_breaker_enabled` | `true` | Enable auto-pause on consecutive failures (threshold and backoff are tuned via `Pgbus::CircuitBreaker` constants) |
+| `zombie_detection` | `true` | Detect and reclaim work from crashed workers |
 | `read_timeout` | `30` | Seconds before a single PGMQ read is bounded (libpq `statement_timeout` + `tcp_user_timeout` on a dedicated connection; nil disables) |
+| `drain_timeout` | `30` | Seconds to wait for in-flight jobs during graceful shutdown before abandoning them |
+| `stall_threshold` | `300` | Seconds without progress before a worker is considered stalled |
 | `priority_levels` | `nil` | Number of priority sub-queues (nil = disabled, 2-10) |
 | `default_priority` | `1` | Default priority for jobs without explicit priority |
+| `group_mode` | `nil` | Grouped-read ordering mode for a queue. Experimental — exempt from the 1.0 stability promise. |
 | `archive_retention` | `7.days` | How long to keep archived messages. Accepts seconds, Duration, or `nil` to disable cleanup |
 | `outbox_enabled` | `false` | Enable transactional outbox poller process |
 | `outbox_poll_interval` | `1.0` | Seconds between outbox poll cycles |
 | `outbox_batch_size` | `100` | Max entries per outbox poll cycle |
 | `outbox_retention` | `1.day` | How long to keep published outbox entries. Accepts seconds, Duration, or `nil` to disable cleanup |
 | `idempotency_ttl` | `7.days` | How long to keep processed event records. Accepts seconds, Duration, or `nil` to disable cleanup |
+| `allowed_global_id_models` | `nil` | Allowlist of Class/Module models permitted as GlobalID **job arguments** and EventBus payloads. `nil` = allow-all (default). `[]` = deny-all. Apps with ActiveStorage should include `ActiveStorage::Blob` (and related models). |
 | `base_controller_class` | `"::ActionController::Base"` | Base class for dashboard controllers (string, constantized at load time) |
 | `return_to_app_url` | `nil` | URL for "back to app" button in dashboard nav (nil hides the button) |
 | `web_auth` | `nil` | Lambda for dashboard authentication |
@@ -2098,6 +2116,22 @@ PostgreSQL + PGMQ
 | `stats_retention` | `30.days` | How long to keep job stats. Accepts seconds, Duration, or `nil` to disable cleanup |
 | `stats_flush_size` | `100` | Buffered stat entries per worker before a bulk insert flush. Positive integer. Lower = tighter SIGKILL loss window. |
 | `stats_flush_interval` | `5` | Seconds between periodic stat buffer flushes. Positive number. |
+| `streams_enabled` | `true` | Enable the SSE streams transport |
+| `streams_default_retention` | `300` | Default stream archive retention in **seconds** (5 minutes). Chat-style apps raise this. |
+| `streams_retention` | `{}` | Per-stream retention overrides (exact string or `Regexp` keys → seconds/Duration) |
+| `streams_orphan_threshold` | `86400` | Age (seconds) after which a durable stream queue is eligible for the orphan sweep (default **24h**). `nil` disables age-based drop. Empty queues are dropped regardless. |
+| `streams_orphan_sweep_interval` | `3600` | Seconds between orphan stream sweeps (default **hourly**). `nil` disables the sweep. |
+| `streams_durable_patterns` | `[]` | Streams (exact string or `Regexp`) that default to durable broadcast mode |
+| `streams_default_broadcast_mode` | `:ephemeral` | Default broadcast mode when no pattern matches (`:ephemeral` or `:durable`) |
+| `streams_presence_patterns` | `[]` | Streams (exact string or `Regexp`) that get connection-driven presence. Experimental. |
+| `streams_presence_member` | `nil` | Custom `->(context) { { id:, metadata: } }` extractor for presence. Experimental. |
+| `streams_broadcast_queue` | `nil` | Dedicated queue for turbo-rails async broadcast jobs. `nil` leaves them on the default queue (can wait behind long jobs). Set a name and back it with a worker capsule. |
+| `streams_pool_size` | `5` | Dedicated DB pool size for durable-stream publish + replay (isolated from the job pool) |
+| `streams_pool_timeout` | `5` | Seconds to wait for a streams-pool connection |
+| `streams_pool_autoscale` | `false` | Grow/shrink the streams pool from live Postgres headroom under saturation (opt-in) |
+| `streams_pool_max` | `nil` | Optional hard ceiling for streams-pool autoscaling |
+| `streams_database_url` / `streams_host` / `streams_port` | `nil` | Override the streamer's LISTEN connection target (e.g. direct primary port when workers use a pooler) |
+| `streams_pool_database_url` / `streams_pool_host` / `streams_pool_port` | `nil` | Override the streams **pool** target independently of LISTEN (so pool traffic can stay on the pooler while LISTEN stays direct) |
 | `streams_test_mode` | `false` | Return a stub SSE response without hijack or background threads. Auto-enabled by `Pgbus::Testing.fake!`/`.inline!`. See [SSE streams in tests](#sse-streams-in-tests). |
 | `streams_stats_enabled` | `false` | Record stream broadcast/connect/disconnect stats (opt-in, can be high volume) |
 | `streams_path` | `nil` | Custom URL path for the SSE endpoint (nil = auto-detected from engine mount) |
@@ -2110,6 +2144,8 @@ PostgreSQL + PGMQ
 | `statsd_port` | `8125` | StatsD UDP port (used when `metrics_backend = :statsd`) |
 | `health_port` | `nil` | Port for standalone HTTP liveness/readiness probes served by the supervisor; nil disables |
 | `health_bind` | `"127.0.0.1"` | Bind address for the standalone health server |
+| `pgmq_schema_mode` | `:auto` | PGMQ schema install mode (`:auto`, `:extension`, `:embedded`) |
+| `doctor_on_boot` | `nil` | Run doctor inside the booting supervisor: `nil`/`false` = off, `:report` = log and boot, `:strict` = refuse to boot on a fatal check. Also via `--doctor` / `--doctor-strict`. |
 | `eager_validation` | `true` | Run `Configuration#validate!` automatically after the `Pgbus.configure` block yields; an invalid value raises `Pgbus::ConfigurationError` at boot. Set `false` to suppress and validate manually. |
 
 ## Development

data/app/models/pgbus/stream_queue.rb

@@ -14,6 +14,14 @@ module Pgbus
   # is absent — `record!` no-ops and `all_names` returns an empty set, so the
   # pre-registry behavior (streams treated as job queues by maintenance) is
   # preserved rather than raising.
+  #
+  # Pre-registry dormant streams (issue #366): queues created before this
+  # table existed never re-broadcast, so they never call `record!`. Those
+  # queues still carry the archive `a_<name>_msg_id_idx` index that only
+  # `ensure_stream_queue` creates — `fingerprint_matched_names` discovers
+  # them, `known_names` unions them with the registry for classification,
+  # and `backfill!` (via `rake pgbus:streams:backfill_registry`) writes
+  # the missing rows so the registry is complete.
   class StreamQueue < BusRecord
     self.table_name = "pgbus_stream_queues"
 
@@ -22,8 +30,10 @@ module Pgbus
       # every broadcast; the caller (`ensure_stream_queue`) also memoizes
       # per-process, so the DB write happens once per stream per process.
       # Errors are swallowed — a registry hiccup must never abort a broadcast.
+      # Returns true on a successful write, false when the table is absent or
+      # the upsert failed (so callers like `backfill!` can report accurately).
       def record!(queue_name)
-        return unless table_exists?
+        return false unless table_exists?
 
         upsert({ queue_name: queue_name }, unique_by: :queue_name)
         # Keep the in-process cache consistent with the write so a subsequent
@@ -35,9 +45,10 @@ module Pgbus
         # nil lets the next all_names call do a real load, which already
         # includes this row since the upsert above has committed.
         @all_names&.add(queue_name)
-        nil
+        true
       rescue StandardError => e
         Pgbus.logger.debug { "[Pgbus] Failed to record stream queue #{queue_name}: #{e.message}" }
+        false
       end
 
       # Set of all registered physical stream queue names. Memoized so a
@@ -49,6 +60,45 @@ module Pgbus
         @all_names ||= load_names
       end
 
+      # Registry ∪ fingerprint-matched names. Use this for classification
+      # (health verdict exclusion, orphan sweep, wildcard worker exclusion)
+      # so dormant pre-registry streams are not treated as job queues.
+      # Does not write — call `backfill!` to persist the fingerprint matches.
+      def known_names
+        all_names | fingerprint_matched_names
+      end
+
+      # Physical queue names that carry the stream-only archive msg_id index
+      # (`a_<queue>_msg_id_idx`). Job queues never get this index. Safe to call
+      # when pgmq is absent — returns an empty set rather than raising.
+      def fingerprint_matched_names
+        Set.new(connection.select_values(<<~SQL.squish))
+          SELECT m.queue_name
+          FROM pgmq.meta m
+          INNER JOIN pg_indexes i
+            ON i.schemaname = 'pgmq'
+           AND i.indexname = 'a_' || m.queue_name || '_msg_id_idx'
+        SQL
+      rescue StandardError => e
+        Pgbus.logger.debug { "[Pgbus] Failed to discover stream queue fingerprints: #{e.message}" }
+        Set.new
+      end
+
+      # Registers fingerprint-matched queues missing from the registry.
+      # Idempotent. Returns the number of names successfully persisted
+      # (failed upserts are not counted — see `record!`). No-ops (returns 0)
+      # when the registry table is absent.
+      def backfill!
+        return 0 unless table_exists?
+
+        missing = fingerprint_matched_names - all_names
+        return 0 if missing.empty?
+
+        registered = missing.count { |name| record!(name) }
+        reset_cache! if registered.positive?
+        registered
+      end
+
       def stream?(queue_name)
         all_names.include?(queue_name)
       end

data/lib/generators/pgbus/add_stream_queues_generator.rb

@@ -31,12 +31,18 @@ module Pgbus
         say ""
         say "Without this table, stream queues are indistinguishable from job"
         say "queues: per-stream archive retention and the orphan sweep skip them,"
-        say "and wildcard ('*') workers can claim durable broadcasts. Existing"
+        say "and wildcard ('*') workers can claim durable broadcasts. Active"
         say "streams register themselves on their next broadcast after migrating."
+        say "Dormant durable streams (completed checkout flows, ended chats) never"
+        say "broadcast again, but doctor / orphan sweep / wildcard workers already"
+        say "see them via the archive-index fingerprint (known_names). Backfill"
+        say "persists those names in the registry and clears the doctor DEGRADED"
+        say "hint (issue #366)."
         say ""
         say "Next steps:"
         say "  1. Run: rails db:migrate#{migrate_command_suffix}"
-        say "  2. Restart pgbus: bin/pgbus start"
+        say "  2. Backfill dormant streams: rake pgbus:streams:backfill_registry"
+        say "  3. Restart pgbus: bin/pgbus start"
         say ""
       end
 

data/lib/pgbus/active_job/executor.rb

@@ -80,7 +80,11 @@ module Pgbus
           msg_id: msg_id
         }
         Instrumentation.instrument("pgbus.executor.execute", instrument_payload) do
-          job = ::ActiveJob::Base.deserialize(payload)
+          # Route through Serializer so allowed_global_id_models gates job
+          # arguments the same way it gates EventBus `_global_id` payloads
+          # (issue #368). Pass this executor's config so an injected allowlist
+          # is not silently ignored in favour of Pgbus.configuration.
+          job = Serializer.deserialize_job_data(payload, configuration: config)
           Pgbus.logger.debug { "[Pgbus::Executor] running #{tag} job_class=#{job_class}" }
           execute_job(job)
           Pgbus.logger.debug { "[Pgbus::Executor] perform_returned #{tag} job_class=#{job_class}" }

data/lib/pgbus/configuration.rb

@@ -69,9 +69,13 @@ module Pgbus
     attr_accessor :outbox_enabled, :outbox_poll_interval, :outbox_batch_size
     attr_reader :outbox_retention # rubocop:disable Style/AccessorGrouping
 
-    # Event bus
+    # GlobalID allowlist for job arguments (`_aj_globalid`) and EventBus
+    # payloads (`_global_id`). nil = allow-all; [] = deny-all; Array of
+    # Class/Module = only those models may resolve. Enforced in
+    # Serializer (issue #368 for the job path).
     attr_accessor :allowed_global_id_models
-    attr_reader :idempotency_ttl # rubocop:disable Style/AccessorGrouping
+    # Event bus
+    attr_reader :idempotency_ttl
 
     # Logging
     attr_accessor :logger

data/lib/pgbus/doctor.rb

@@ -258,15 +258,17 @@ module Pgbus
     end
 
     # 7. GlobalID allowlist — security. allowed_global_id_models = nil means
-    # "allow ANY model as a GlobalID event/job argument", which lets a crafted
-    # payload deserialize arbitrary AR models. It's the default for upgrade
-    # continuity, so this is a warning (never a failure), and only in production
-    # where the blast radius is real.
+    # "allow ANY model as a GlobalID EventBus payload or ActiveJob argument",
+    # which lets a crafted queue message deserialize arbitrary AR models
+    # (issue #368 closed the job-path hole; both paths share the gate). It's
+    # the default for upgrade continuity, so this is a warning (never a
+    # failure), and only in production where the blast radius is real.
     def check_allowed_global_id_models
       if @config.allowed_global_id_models.nil? && production?
         return Check.new(name: "GlobalID allowlist", status: :warn,
                          detail: "allowed_global_id_models is nil (allow-all) in production — " \
-                                 "set an explicit allowlist of models permitted as GlobalID arguments")
+                                 "set an explicit allowlist of models permitted as GlobalID " \
+                                 "job arguments and EventBus payloads")
       end
 
       Check.new(name: "GlobalID allowlist", status: :ok,

data/lib/pgbus/mcp/health_analyzer.rb

@@ -23,6 +23,18 @@ module Pgbus
       # heartbeats while doing no work — exactly the case we must catch.
       WORKER_KIND = "worker"
 
+      # Process kinds that claim from queues: workers drain job queues, consumers
+      # drain EventBus handler queues. `drained_queue_names` unions both kinds'
+      # queues, so the wedge signal must count both kinds as liveness evidence —
+      # otherwise a consumer-only fleet with a wedged handler queue is missed
+      # (the worker set is empty and the branch never runs).
+      DRAINING_KINDS = %w[worker consumer].freeze
+
+      # Cap on how many queue names a reason string lists before truncating to
+      # "(+N more)" — keeps a flagged fleet of hundreds of queues from producing
+      # a multi-KB log line (issue #367 minor).
+      NAME_LIMIT = 10
+
       def initialize(data_source)
         @data_source = data_source
       end
@@ -48,8 +60,14 @@ module Pgbus
         backlog = non_dlq.select { |q| q[:queue_visible_length].to_i.positive? }
         active_backlog = backlog.reject { |q| q[:paused] }
 
-        stalled  = stalled_reasons(active_backlog, processes)
-        degraded = degraded_reasons(queues, backlog, processes, health, health_error)
+        # Split the active backlog by whether some configured capsule or handler
+        # drains the queue (issue #367 defect 2). Live workers say nothing about
+        # a queue they can never claim from, so only the drained side can be the
+        # silent-worker-wedge; the undrained side is its own DEGRADED hazard.
+        drained, undrained = partition_by_drain(active_backlog)
+
+        stalled  = stalled_reasons(active_backlog, drained, processes)
+        degraded = degraded_reasons(queues, backlog, undrained, processes, health, health_error)
 
         status = if stalled.any?
                    "STALLED"
@@ -73,6 +91,18 @@ module Pgbus
         queue[:name].to_s.end_with?(Pgbus::DEAD_LETTER_SUFFIX)
       end
 
+      # Partition a backlog into [drained, undrained] using the set of queues
+      # some configured capsule or EventBus handler drains. A nil drained set
+      # means a wildcard capsule drains everything, so nothing is undrained.
+      def partition_by_drain(backlog)
+        return [backlog, []] if backlog.empty?
+
+        drained_names = @data_source.drained_queue_names
+        return [backlog, []] if drained_names.nil?
+
+        backlog.partition { |q| drained_names.include?(q[:name].to_s) }
+      end
+
       # Returns [health_hash, error_or_nil]. We never raise — pgbus_health
       # must always produce a verdict — but we DO surface the failure so the
       # caller knows the verdict was built from partial data, and we log it
@@ -84,31 +114,42 @@ module Pgbus
         [{}, e]
       end
 
-      # STALLED detection: an active (non-paused) backed-up queue while a
-      # worker is in the :stalled state, or backed up with live-but-idle
-      # workers present.
-      def stalled_reasons(backlog, processes)
-        return [] if backlog.empty?
-
-        workers = processes.select { |p| p[:kind] == WORKER_KIND }
-        return [] if workers.empty?
-
-        stalled_workers = workers.select { |w| w[:status].to_s == "stalled" }
-        live_workers    = workers.select { |w| %w[healthy stalled].include?(w[:status].to_s) }
+      # STALLED detection. Two distinct wedge signatures, with different scopes:
+      #
+      #   * A worker self-reporting :stalled (heart-beating but claim loop frozen)
+      #     is a wedge regardless of WHICH queue holds the backlog — the worker is
+      #     broken. So this branch reasons against the FULL active backlog. This
+      #     is the specific #179 signature and stays worker-scoped.
+      #   * Live-but-idle draining processes with a never-claimed backlog is only
+      #     a wedge for queues those processes can actually claim from — so this
+      #     branch reasons against the DRAINED backlog only (issue #367 defect 2).
+      #     Liveness is the evidence, and `drained_queue_names` covers both
+      #     worker-drained job queues and consumer-drained handler queues, so a
+      #     live process of either draining kind counts (a consumer-only fleet
+      #     must still catch a wedged handler queue).
+      def stalled_reasons(active_backlog, drained_backlog, processes)
+        stalled_workers = processes.select { |p| p[:kind] == WORKER_KIND && p[:status].to_s == "stalled" }
+        live_drainers   = processes.select do |p|
+          DRAINING_KINDS.include?(p[:kind].to_s) && %w[healthy stalled].include?(p[:status].to_s)
+        end
 
         reasons = []
-        if stalled_workers.any?
+        if stalled_workers.any? && active_backlog.any?
           reasons << "#{stalled_workers.size} worker(s) stalled (heart-beating but claim loop not advancing) " \
-                     "while #{backlog.size} queue(s) have visible backlog: #{backlog_names(backlog)}"
-        elsif live_workers.any? && all_unread?(backlog)
-          reasons << "#{backlog.size} queue(s) have visible messages with read_ct=0 (never claimed) " \
-                     "while #{live_workers.size} worker(s) are alive: #{backlog_names(backlog)}"
+                     "while #{active_backlog.size} queue(s) have visible backlog: #{backlog_names(active_backlog)}"
+        elsif live_drainers.any? && drained_backlog.any? && any_unread?(drained_backlog)
+          unread = drained_backlog.select { |q| unread?(q) }
+          reasons << "#{unread.size} queue(s) have visible messages with read_ct=0 (never claimed) " \
+                     "while #{live_drainers.size} draining process(es) are alive: #{backlog_names(unread)}"
         end
         reasons
       end
 
       # DEGRADED detection: conditions worth surfacing that are not the wedge.
-      def degraded_reasons(queues, backlog, processes, health, health_error)
+      # +undrained+ is the active backlog on queues no configured capsule/handler
+      # drains (issue #367 defect 2) — a real hazard (nothing will empty them),
+      # but not the worker wedge, so it is DEGRADED rather than STALLED.
+      def degraded_reasons(queues, backlog, undrained, processes, health, health_error)
         reasons = []
 
         reasons << "queue health stats unavailable: #{health_error.class}: #{health_error.message}" if health_error
@@ -119,24 +160,71 @@ module Pgbus
         paused_backlog = backlog.select { |q| q[:paused] }
         reasons << "#{paused_backlog.size} paused queue(s) holding a backlog: #{backlog_names(paused_backlog)}" if paused_backlog.any?
 
+        # Every active undrained queue is the "nobody drains this" hazard: it
+        # holds a visible backlog and no configured capsule/handler will empty
+        # it. Past reads don't make it safe — if nothing drains it now, whatever
+        # read those messages is gone. (`undrained` is already the visible,
+        # non-paused backlog on queues no capsule drains.)
+        if undrained.any?
+          reasons << "#{undrained.size} queue(s) hold messages but no capsule is " \
+                     "configured to drain them: #{backlog_names(undrained)}"
+        end
+
         dlq = queues.select { |q| dlq?(q) && q[:queue_length].to_i.positive? }
         reasons << "#{dlq.size} dead-letter queue(s) hold messages" if dlq.any?
 
         age = health[:oldest_transaction_age_sec]
         reasons << "oldest open transaction is #{age}s old (MVCC horizon pinning risk)" if age && age > 300
 
+        unregistered_hint = unregistered_stream_hint
+        reasons << unregistered_hint if unregistered_hint
+
         reasons
       end
 
-      # The strongest wedge signal: visible messages that have never been read.
-      # When queue metrics don't expose read_ct we conservatively treat the
-      # backlog as unread (the wedge default) so we don't miss the condition.
-      def all_unread?(backlog)
-        backlog.all? { |q| !q.key?(:max_read_ct) || q[:max_read_ct].to_i.zero? }
+      # Issue #366: fingerprint-matched streams excluded from the wedge signal
+      # still need the registry for orphan sweep bookkeeping and durable
+      # wildcard safety. Surface a DEGRADED hint so operators run the backfill.
+      def unregistered_stream_hint
+        return unless @data_source.respond_to?(:unregistered_stream_queue_count)
+
+        count = @data_source.unregistered_stream_queue_count
+        return if count.to_i <= 0
+
+        "#{count} unregistered stream queue(s) detected (dormant pre-registry durable " \
+          "streams). Run: rake pgbus:streams:backfill_registry"
+      rescue StandardError => e
+        Pgbus.logger.debug { "[Pgbus::MCP] unregistered stream hint failed: #{e.class}: #{e.message}" }
+        nil
+      end
+
+      # True when any queue in the backlog holds a visible, never-claimed
+      # message — the wedge signal.
+      def any_unread?(backlog)
+        backlog.any? { |q| unread?(q) }
+      end
+
+      # True when a queue has at least one visible (claimable) message with
+      # read_ct=0. This counts per visible message (issue #367 review), so one
+      # retried or in-flight message (read_ct>0) can no longer veto the signal
+      # for a pile of genuinely-unclaimed jobs the way max(read_ct) did.
+      #
+      # When metrics don't expose the count (an old hash / a stubbed test) we
+      # conservatively treat the queue as unread so we never miss the wedge.
+      def unread?(queue)
+        return true unless queue.key?(:visible_unread_length)
+
+        queue[:visible_unread_length].to_i.positive?
       end
 
+      # Join backlog queue names for a reason string, truncated so a flagged
+      # fleet of hundreds of queues doesn't produce a multi-KB log line
+      # (issue #367 minor). Lists the first NAME_LIMIT, then "(+N more)".
       def backlog_names(queues)
-        queues.map { |q| q[:name] }.join(", ")
+        names = queues.map { |q| q[:name] }
+        shown = names.first(NAME_LIMIT).join(", ")
+        overflow = names.size - NAME_LIMIT
+        overflow.positive? ? "#{shown} (+#{overflow} more)" : shown
       end
 
       def build_summary(queues, non_dlq, processes, health)

data/lib/pgbus/process/dispatcher.rb

@@ -637,12 +637,14 @@ module Pgbus
 
       # Fresh set of physical stream-queue names for this maintenance pass.
       # Reset first so a stream created since the last hourly pass is picked
-      # up without a process restart. Empty on unmigrated installs (registry
-      # table absent) — callers then treat every queue as a job queue, which
-      # is the pre-registry behavior.
+      # up without a process restart. Includes fingerprint-matched dormant
+      # streams that never re-registered after the registry migration
+      # (issue #366) so orphan sweep / stream-archive prune can still see
+      # them. Empty only when neither the registry nor any fingerprint
+      # matches exist — callers then treat every queue as a job queue.
       def current_stream_queue_names
         Pgbus::StreamQueue.reset_cache!
-        Pgbus::StreamQueue.all_names
+        Pgbus::StreamQueue.known_names
       end
 
       def cleanup_recurring_executions

data/lib/pgbus/process/worker.rb

@@ -410,10 +410,12 @@ module Pgbus
         # Stream queues share the job namespace (pgbus_<name>) but must never
         # be adopted by a wildcard worker: a worker would claim durable
         # broadcasts, fail to deserialize them, and DLQ-move them out of the
-        # stream's replay history. The registry is what tells them apart.
-        # Reset first so a stream created since the last resolve is excluded.
+        # stream's replay history. known_names includes fingerprint-matched
+        # dormant pre-registry streams (issue #366) so they are excluded even
+        # before backfill. Reset first so a stream created since the last
+        # resolve is excluded.
         Pgbus::StreamQueue.reset_cache!
-        stream_names = Pgbus::StreamQueue.all_names
+        stream_names = Pgbus::StreamQueue.known_names
 
         # Event-bus subscriber queues also share the job namespace (pgbus_<handler>)
         # but carry event payloads, not ActiveJob jobs. A wildcard worker that

data/lib/pgbus/serializer.rb

@@ -6,6 +6,11 @@ module Pgbus
   module Serializer
     module_function
 
+    # ActiveJob encodes GlobalID job arguments with this key (private constant
+    # on ActiveJob::Arguments). Walked by the job-path allowlist gate (#368).
+    AJ_GLOBALID_KEY = "_aj_globalid"
+    private_constant :AJ_GLOBALID_KEY
+
     def serialize_job(active_job)
       Instrumentation.instrument("pgbus.serializer.serialize", kind: :job) do
         data = active_job.serialize
@@ -21,13 +26,25 @@ module Pgbus
       end
     end
 
-    def deserialize_job(json_string)
+    def deserialize_job(json_string, configuration: Pgbus.configuration)
       Instrumentation.instrument("pgbus.serializer.deserialize", kind: :job) do
-        data = JSON.parse(json_string)
-        ActiveJob::Base.deserialize(data)
+        deserialize_job_data(JSON.parse(json_string), configuration: configuration)
       end
     end
 
+    # Job-hash entry point used by the executor (payload already parsed) and by
+    # `deserialize_job`. When `allowed_global_id_models` is configured, every
+    # `_aj_globalid` in the tree is checked before Rails' unrestricted
+    # GlobalID::Locator runs (issue #368). Nil allowlist = zero-cost allow-all.
+    # Prefer the caller's `configuration` (e.g. Executor's injected config) so a
+    # non-global allowlist is not silently ignored.
+    def deserialize_job_data(data, configuration: Pgbus.configuration)
+      assert_job_global_ids_allowed!(data, configuration: configuration)
+      # Top-level constant: bare ActiveJob::Base can resolve to
+      # Pgbus::ActiveJob::Base under Zeitwerk's Pgbus::ActiveJob namespace.
+      ::ActiveJob::Base.deserialize(data)
+    end
+
     def serialize_event(event)
       payload = event.respond_to?(:to_global_id) ? { "_global_id" => event.to_global_id.to_s } : event
       JSON.generate({
@@ -37,11 +54,13 @@ module Pgbus
                     })
     end
 
-    def deserialize_event(json_string)
+    def deserialize_event(json_string, configuration: Pgbus.configuration)
       data = JSON.parse(json_string)
       payload = data["payload"]
 
-      data["payload"] = locate_global_id(payload["_global_id"]) if payload.is_a?(Hash) && payload["_global_id"]
+      if payload.is_a?(Hash) && payload["_global_id"]
+        data["payload"] = locate_global_id(payload["_global_id"], configuration: configuration)
+      end
 
       Event.new(
         event_id: data["event_id"],
@@ -53,11 +72,19 @@ module Pgbus
     # Locate a GlobalID with optional type restriction.
     # When allowed_global_id_models is configured, only those model classes
     # can be resolved — prevents loading arbitrary objects from crafted payloads.
-    def locate_global_id(gid_string)
+    # Shared by EventBus payloads (`_global_id`) and job arguments (`_aj_globalid`).
+    def locate_global_id(gid_string, configuration: Pgbus.configuration)
+      gid = assert_allowed_global_id!(gid_string, configuration: configuration)
+      GlobalID::Locator.locate(gid)
+    end
+
+    # Raises SerializationError unless the GlobalID's model is permitted.
+    # Returns the parsed GlobalID on success (so locate can skip re-parse).
+    def assert_allowed_global_id!(gid_string, configuration: Pgbus.configuration)
       gid = GlobalID.parse(gid_string)
       raise Pgbus::SerializationError, "Invalid GlobalID: #{gid_string.inspect}" unless gid
 
-      allowed = Pgbus.configuration.allowed_global_id_models
+      allowed = configuration.allowed_global_id_models
       if allowed && allowed.empty?
         raise Pgbus::SerializationError,
               "GlobalID deserialization is disabled (allowed_global_id_models is empty). " \
@@ -74,7 +101,26 @@ module Pgbus
               "Add it to Pgbus.configuration.allowed_global_id_models to permit deserialization."
       end
 
-      GlobalID::Locator.locate(gid)
+      gid
+    end
+
+    # Walk a job payload (or any nested structure) and enforce the allowlist on
+    # every ActiveJob `_aj_globalid` value. No-op when allowlist is nil.
+    def assert_job_global_ids_allowed!(data, configuration: Pgbus.configuration)
+      return if configuration.allowed_global_id_models.nil?
+
+      walk_job_global_ids(data, configuration)
+    end
+
+    def walk_job_global_ids(value, configuration)
+      case value
+      when Hash
+        assert_allowed_global_id!(value[AJ_GLOBALID_KEY], configuration: configuration) if value.key?(AJ_GLOBALID_KEY)
+        value.each_value { |child| walk_job_global_ids(child, configuration) }
+      when Array
+        value.each { |child| walk_job_global_ids(child, configuration) }
+      end
     end
+    private_class_method :walk_job_global_ids
   end
 end

data/lib/pgbus/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Pgbus
-  VERSION = "0.12.2"
+  VERSION = "0.12.3"
 end

data/lib/pgbus/web/data_source.rb

@@ -65,17 +65,60 @@ module Pgbus
       end
       private :fetch_queues_with_metrics
 
-      # Physical queue names registered as stream queues (the
-      # pgbus_stream_queues registry). Stream delivery is a non-consuming
-      # peek, so stream messages sit visible with read_ct=0 forever — health
-      # verdicts must not read that as a wedge (issue #359). Loaded fresh per
-      # call (reset + one query): verdicts run on coarse intervals and a
-      # long-lived process must see streams registered since the last check.
-      # Degrades to an empty Set when the registry table is absent
-      # (pre-migration installs) or unreadable — StreamQueue swallows both.
+      # Physical queue names known to back streams: the pgbus_stream_queues
+      # registry plus any dormant pre-registry queues discovered by the
+      # archive msg_id index fingerprint (issue #366). Stream delivery is a
+      # non-consuming peek, so stream messages sit visible with read_ct=0
+      # forever — health verdicts must not read that as a wedge (issue #359).
+      # Loaded fresh per call (reset + registry + fingerprint queries):
+      # verdicts run on coarse intervals and a long-lived process must see
+      # streams registered since the last check. Degrades to an empty Set
+      # when the registry/pgmq schema is absent or unreadable.
       def stream_queue_names
         StreamQueue.reset_cache!
-        StreamQueue.all_names
+        StreamQueue.known_names
+      end
+
+      # Fingerprint-matched stream queues missing from the registry (issue #366).
+      # Used by HealthAnalyzer to surface a DEGRADED hint pointing at
+      # `rake pgbus:streams:backfill_registry`. Zero when fully backfilled or
+      # when the registry table is absent.
+      def unregistered_stream_queue_count
+        return 0 unless StreamQueue.table_exists?
+
+        (StreamQueue.fingerprint_matched_names - StreamQueue.all_names).size
+      rescue StandardError => e
+        Pgbus.logger.debug { "[Pgbus::Web] unregistered_stream_queue_count failed: #{e.message}" }
+        0
+      end
+
+      # The set of physical queue names some configured worker capsule or
+      # EventBus handler is set up to drain. Used by the HealthAnalyzer to keep
+      # the silent-worker-wedge verdict from flagging queues nobody drains
+      # (ad-hoc queues, unregistered stream queues — issue #367, #366): live
+      # workers prove nothing about a queue they can never claim from.
+      #
+      # Returns nil when a capsule uses the "*" wildcard — that capsule drains
+      # every job queue (Worker#resolve_wildcard_queues), so there is nothing to
+      # intersect against. Otherwise returns a Set of physical names: each
+      # capsule's explicit logical queues expanded to the physical tables that
+      # actually exist (via the client's queue strategy, so a priority queue's
+      # _p0.._pN sub-tables are matched, not the bare prefixed name priority mode
+      # never creates), unioned with the EventBus handler queues consumers drain.
+      def drained_queue_names
+        capsules = Array(Pgbus.configuration.workers)
+        capsule_queues = capsules.flat_map { |c| c[:queues] || c["queues"] || [] }
+        return nil if capsule_queues.include?("*")
+
+        physical = capsule_queues.flat_map { |q| @client.physical_queue_names(q) }
+        (physical + handler_queue_physical_names).to_set
+      rescue StandardError => e
+        # Fail open: nil means "a wildcard drains everything", which restores the
+        # pre-#367 behavior of intersecting against the whole backlog. A raise
+        # here (e.g. a malformed capsule queue name) must never break the
+        # HealthAnalyzer's "always produces a verdict" contract.
+        Pgbus.logger.debug { "[Pgbus::Web] Error computing drained queues: #{e.class}: #{e.message}" }
+        nil
       end
 
       # name is the full PGMQ queue name (e.g. "pgbus_default") as returned
@@ -1132,7 +1175,9 @@ module Pgbus
               (SELECT count(*) FROM pgmq.#{qtable} WHERE vt <= NOW()) AS queue_visible_length,
               (SELECT EXTRACT(epoch FROM (NOW() - max(enqueued_at)))::int FROM pgmq.#{qtable}) AS newest_msg_age_sec,
               (SELECT EXTRACT(epoch FROM (NOW() - min(enqueued_at)))::int FROM pgmq.#{qtable}) AS oldest_msg_age_sec,
-              (SELECT CASE WHEN is_called THEN last_value ELSE 0 END FROM pgmq.#{seq_name}) AS total_messages
+              (SELECT CASE WHEN is_called THEN last_value ELSE 0 END FROM pgmq.#{seq_name}) AS total_messages,
+              (SELECT max(read_ct) FROM pgmq.#{qtable}) AS max_read_ct,
+              (SELECT count(*) FROM pgmq.#{qtable} WHERE vt <= NOW() AND read_ct = 0) AS visible_unread_length
           SQL
         rescue StandardError => e
           Pgbus.logger.debug { "[Pgbus::Web] Skipping queue metrics for #{name}: #{e.message}" }
@@ -1150,7 +1195,9 @@ module Pgbus
             queue_visible_length: row["queue_visible_length"].to_i,
             oldest_msg_age_sec: row["oldest_msg_age_sec"]&.to_i,
             newest_msg_age_sec: row["newest_msg_age_sec"]&.to_i,
-            total_messages: row["total_messages"].to_i
+            total_messages: row["total_messages"].to_i,
+            max_read_ct: row["max_read_ct"]&.to_i,
+            visible_unread_length: row["visible_unread_length"].to_i
           }
         end
       rescue StandardError => e
@@ -1168,7 +1215,9 @@ module Pgbus
               count(*) AS queue_length,
               count(CASE WHEN vt <= NOW() THEN 1 END) AS queue_visible_length,
               EXTRACT(epoch FROM (NOW() - max(enqueued_at)))::int AS newest_msg_age_sec,
-              EXTRACT(epoch FROM (NOW() - min(enqueued_at)))::int AS oldest_msg_age_sec
+              EXTRACT(epoch FROM (NOW() - min(enqueued_at)))::int AS oldest_msg_age_sec,
+              max(read_ct) AS max_read_ct,
+              count(CASE WHEN vt <= NOW() AND read_ct = 0 THEN 1 END) AS visible_unread_length
             FROM pgmq.#{qtable}
           ),
           all_metrics AS (
@@ -1180,6 +1229,8 @@ module Pgbus
             q_summary.queue_visible_length,
             q_summary.newest_msg_age_sec,
             q_summary.oldest_msg_age_sec,
+            q_summary.max_read_ct,
+            q_summary.visible_unread_length,
             all_metrics.total_messages
           FROM q_summary, all_metrics
         SQL
@@ -1192,7 +1243,9 @@ module Pgbus
           queue_visible_length: row["queue_visible_length"].to_i,
           oldest_msg_age_sec: row["oldest_msg_age_sec"]&.to_i,
           newest_msg_age_sec: row["newest_msg_age_sec"]&.to_i,
-          total_messages: row["total_messages"].to_i
+          total_messages: row["total_messages"].to_i,
+          max_read_ct: row["max_read_ct"]&.to_i,
+          visible_unread_length: row["visible_unread_length"].to_i
         }
       rescue StandardError => e
         Pgbus.logger.error { "[Pgbus::Web] Error fetching metrics for #{queue_name}: #{e.class}: #{e.message}" }

data/lib/tasks/pgbus_streams.rake

@@ -2,6 +2,36 @@
 
 namespace :pgbus do
   namespace :streams do
+    desc "Register pre-registry durable stream queues into pgbus_stream_queues " \
+         "(issue #366). Detects queues by the archive msg_id index fingerprint " \
+         "that only ensure_stream_queue creates. Idempotent."
+    task backfill_registry: :environment do
+      unless Pgbus::StreamQueue.table_exists?
+        abort "pgbus_stream_queues table is missing. " \
+              "Run: rails generate pgbus:add_stream_queues && rails db:migrate " \
+              "(rails db:migrate:pgbus when config.connects_to uses a separate database)"
+      end
+
+      matched = Pgbus::StreamQueue.fingerprint_matched_names
+      already = Pgbus::StreamQueue.all_names
+      already_matched = matched & already
+      missing = matched - already
+
+      puts "Fingerprint-matched stream queues: #{matched.size}"
+      puts "Already registered (of matched):   #{already_matched.size}"
+      puts "Missing from registry:             #{missing.size}"
+
+      if missing.empty?
+        puts "Nothing to backfill."
+        next
+      end
+
+      count = Pgbus::StreamQueue.backfill!
+      puts "Registered #{count} stream queue(s)."
+      puts "Warning: #{missing.size - count} write(s) failed — re-run after fixing DB errors." if count < missing.size
+      missing.sort.each { |name| puts "  + #{name}" }
+    end
+
     desc "Fail if any pgbus controller or web component includes ActionController::Live"
     task :lint_no_live do
       # ActionController::Live has well-documented interactions with Puma

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: pgbus
 version: !ruby/object:Gem::Version
-  version: 0.12.2
+  version: 0.12.3
 platform: ruby
 authors:
 - Mikael Henriksson