pgbus 0.12.0 → 0.12.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: f2e4a0eacc67a1d3edf2fa7aae4ffd835cf03ae275f016aa92c2f490923897cd
4
- data.tar.gz: defeb677222dc6ac50c755225e3da9b57752667e03569af747f1f8a26974b95d
3
+ metadata.gz: a966ed340b2cd097f82b926f90647a0d481df35da76bb4ef06f4e3c13e3d5692
4
+ data.tar.gz: 779124d08216918b02a162f2a1263c4115cd7a7fb9404dc0055dc940da28793f
5
5
  SHA512:
6
- metadata.gz: 2a9b8e1d6efa1106a5f56325d48bae1bcc650916154e54ccffa675fa9a5fde887f412536ca6d40d56e907ec804de66b3b1ca2000c5c66142889d9f8ff07dce3d
7
- data.tar.gz: 0f5da2baedef6ad70d304e1d91339a6f7613687c642d40bb7b52fd508d198fde5840a22b2f42f2a0bbf3429aea7b9f8f1c71d20d63e35dbaa29b787b43fb4601
6
+ metadata.gz: b25519a1d9ea9e1e2700b959a5ebcf03adf8efbcb130001f1e962067e0def2546b1644e199d24d2b849b29ba67f15bc83eef5f41506ffde3d6ea5d79da752af7
7
+ data.tar.gz: f0ec03ccc2b075b90f6b87cc5eb1315d5d3834b2cdd7c9c19fc851f350196f6860b941ec0566aac0315f531bf29356f9a1f94b68e852aa497eb642833ddc3800
data/CHANGELOG.md CHANGED
@@ -1,5 +1,11 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ### Fixed
4
+
5
+ - **`streams_pool_database_url` / `streams_pool_host` / `streams_pool_port` — route the streams PGMQ pool independently of the streamer's LISTEN connection (issue #358).** 0.12.0 started building the dedicated streams pool from `streams_connection_options` (correct for a separate streams database), but that method is also how pooler-bypass installs pin the LISTEN connection to the **direct** Postgres port (`streams_port = 5432`, the documented "workers go through PgBouncer, streamer goes direct" pattern) — so upgrading silently moved up to `streams_pool_size` connections **per process** onto the direct port, whose `max_connections` ceiling on managed Postgres is typically low. A modest fleet exhausts it (`FATAL: remaining connection slots are reserved for roles with the SUPERUSER attribute`): `StreamApp` then 500s every SSE connect and durable broadcasts fail at publish, intermittently, because the pools are lazy. Only LISTEN actually needs the direct port (it dies at transaction-pool COMMIT boundaries); the pool's broadcast INSERTs and replay reads are plain pooler-safe SQL. The new `streams_pool_*` triple routes the POOL independently — applied to the base options exactly like the `streams_*` and `worker_notify_*` groups: set `streams_pool_port` back to the pooled port and only the LISTEN pins remain direct. Default `nil` = follows `streams_connection_options`, byte-identical to 0.12.0, so separate-streams-DB installs are unchanged. Refs #358.
6
+ - **The health verdict no longer reads durable stream queues as a wedged fleet (issue #359).** The Process-liveness signal (`pgbus doctor`, `pgbus_health`, the MCP health tool) treats *visible messages with `read_ct=0` while workers are alive* as the silent-worker-wedge signature — but durable stream delivery is a non-consuming peek, so **every** stream queue matches it permanently by design. On a streams-heavy install the verdict screamed STALLED listing hundreds of healthy stream queues, burying real wedges in noise. `HealthAnalyzer` now excludes queues registered in the `pgbus_stream_queues` registry from the operational set (exactly like DLQs) — out of the STALLED/DEGRADED reasons and the backlog totals — via a new `Web::DataSource#stream_queue_names` (loaded fresh per verdict; degrades to an empty set on pre-registry installs). Same bug class as #308/#309, same registry cure. Refs #359.
7
+ - **The `upgrade_pgmq` migration no longer silently kills NOTIFY-gated wakeups fleet-wide (issue #360).** The generated migration drops every pgmq function **with CASCADE** — which also drops the `trigger_notify_queue_insert_listeners` trigger from every existing queue table (it depends on the dropped `pgmq.notify_queue_listeners()`), and `install_sql` re-created the functions but never the per-queue triggers. Result: after every PGMQ schema upgrade, NOTIFY wakeups (#174) silently died and workers fell back to `polling_interval` polling — nothing errored; latency degraded and DB poll load rose. Trigger re-install only happened in `ensure_single_queue` (per-process memoized), so on the common deploy ordering where the job supervisor boots *before* migrations run, the supervisor installed triggers at boot and the migration wiped them minutes later — with nothing left to re-install them until a full process restart. New `PgmqSchema.reinstall_notify_triggers_sql` replays `pgmq.enable_notify_insert` for every row in `pgmq.notify_insert_throttle` (a table, preserved by the upgrade, recording exactly which queues had notify and at what throttle; the function is idempotent), and the migration template runs it after `install_sql` — restoring every trigger at its original interval. No-op on vendored versions without the notify feature. **Already ran an upgrade migration?** Run `Pgbus::PgmqSchema.reinstall_notify_triggers_sql` once (e.g. from a console: `ActiveRecord::Base.connection.execute(...)`) or restart the fleet; `pgbus doctor`'s LISTEN/NOTIFY check confirms the repair. Refs #360.
8
+
3
9
  ### Breaking Changes
4
10
 
5
11
  - **`config.validate!` now rejects malformed values for 12 core config keys at boot instead of failing deep in a worker or silently misbehaving (issue #335).** Before the 1.0 surface freeze, these headline job-path keys had **zero** validation while streams keys had 14 checks — so a bad value surfaced as a crash inside a worker/dispatcher/poller/scheduler thread, per-enqueue, or by silently corrupting every derived queue name / leaving the dashboard open. `Pgbus.configure` (eager-validating by default) now raises `Pgbus::ConfigurationError` naming the key for: the worker-recycling trio `max_jobs_per_worker`/`max_memory_mb`/`max_worker_lifetime` (positive number or `nil` to disable); the intervals `dispatch_interval`/`outbox_poll_interval`/`recurring_schedule_interval` (positive number); `outbox_batch_size` (positive integer); `default_priority` (non-negative integer — `0` is a valid level); `queue_prefix`/`default_queue` (non-empty String — an empty prefix silently produced malformed queue names); `web_auth` (a callable or `nil` — a non-callable silently left the dashboard open); `error_reporters` (an Array); and `connects_to` (a Hash or `nil` — previously a malformed value raised a raw `TypeError` deep in engine boot). This only rejects configuration that was already broken; if a `Pgbus.configure` block set one of these to an invalid value it will now fail loudly at boot (the intended pre-freeze tightening). Set `config.eager_validation = false` to defer. Refs #335, #282.
@@ -38,6 +44,7 @@
38
44
 
39
45
  ### Fixed
40
46
 
47
+ - **`ensures_uniqueness` / `limits_concurrency` declared on a base class now reach every subclass — previously the declaration was silently inert for them (issue #357).** Both macros stored their config in a class-level ivar (`@pgbus_uniqueness` / `@pgbus_concurrency`) and every lookup read `active_job.class.pgbus_uniqueness` without walking ancestors — class-level ivars are not inherited in Ruby, so `class RecurringBase < ApplicationJob; ensures_uniqueness strategy: :until_executed; end` gave subclasses NO uniqueness key, no scheduler overlap protection, and no #333 guard, with no warning that the natural reading of the code was false. The readers now fall back to the nearest ancestor's declaration (a class's own declaration still beats an inherited one). Coupled fix, required to make inheritance safe rather than harmful: no default key proc is stored anymore — the old `key || ->(*) { name }` captured the DECLARING class, so an inheritance fix alone would have collapsed every subclass into ONE shared key (the base class's name), and with `on_conflict: :discard` sibling jobs would silently discard each other's enqueues — strictly worse than inert. The class-name default is now resolved from the ENQUEUED job's class at all three call sites (`Uniqueness.resolve_key`, `Concurrency.resolve_key`, and the recurring scheduler's `resolve_uniqueness_key`), and `config[:key]` is `nil` when no explicit `key:` was given. Consumer note: concrete-class declarations resolve to exactly the same keys as before, but a base-class declaration that was silently inert now ACTIVATES per-subclass uniqueness/concurrency on upgrade — audit any base-class declarations (the macro finally does what it says). In the scheduler path the no-key default is additionally qualified with the recurring task's `args:` (`SyncJob:["site_a"]`) so two tasks pointing at the same job class with different arguments don't share one lock — the scheduler's analogue of the #333 raise-at-enqueue guard, which a scheduler loop can't use — and a failure to resolve ANY uniqueness key — default or user-supplied key proc — now fails closed (skips the tick, logs, and reaches `error_reporters`) instead of degrading to nil and enqueuing WITHOUT a lock, which silently disabled the very protection the proc configured. Both macros also now reject a non-nil, non-callable `key:` (e.g. `key: false`) at definition time with the documented ArgumentError — previously `false` slipped the truthiness guard and surfaced as a `NoMethodError` at enqueue (uniqueness) or silently acted as the class-name default (concurrency). Measured (benchmark-ips, enqueue-path budget per docs/performance.md): the shared `Support.call_key_proc` dispatch is ~17% faster than the old inline dispatch (136ns vs 160ns/op), the class-name default is 3.7× faster than the old stored-proc call (23ns vs 86ns/op), and the inherited-config ancestor walk adds ~15ns/op. Refs #357, #333.
41
48
  - **The Ruby-Timeout read fallback no longer leaves a wedged `CONNECTION_OK` connection in the pgmq pool to re-hang the next read (issue #354).** On the one path where libpq can't bound a hung socket (dedicated connection on non-Linux hosts or libpq < 12), the last-resort `Timeout.timeout` interrupts the read via `Thread#raise` — and libpq may leave the pooled `PG::Connection` reporting `CONNECTION_OK` while it will in fact re-hang on reuse, invisible to pgmq-ruby's checkout health check (it isn't `CONNECTION_BAD`). This was a documented KNOWN LIMITATION waiting on a public pool-reload upstream (cf. mensfeld/pgmq-ruby#94); pgmq-ruby 0.7.1 shipped it, so the fallback now raises an internal `ReadTimeoutError` subclass and reloads the job pool before re-raising — the poisoned connection (already checked back in by `connection_pool`'s ensure) is dropped and the pool rebuilds lazily. A clean server-side `statement_timeout` cancel still never triggers a reload (the connection is healthy), and a reload failure is logged without masking the timeout. Requires pgmq-ruby >= 0.7.1 (dependency bumped to `~> 0.7.1`). Refs #354.
42
49
  - **`connection_guc_mode = :session` no longer kills every SSE stream and NOTIFY wakeup with `PG::Error: invalid connection option "variables"` (issue #352).** In `:session` mode, `forward_connection_variables` deliberately leaves the database.yml `variables:` hash on the connection options for the *caller* to strip and apply via post-connect `SET` (a transaction-mode pooler rejects the libpq `options` startup param — the reason `:session` mode exists). `Pgbus::Client#wrap_session_gucs` honors that contract on both pgmq pools, but the two dedicated raw-connect paths — the streamer's `build_raw_pg_connection` and the worker `NotifyListener` — passed the hash straight to `PG.connect`, which rejects the non-libpq `:variables` key. Result: the first stream request per process raised, `StreamApp` rescued it into a 500 (so **every** `/pgbus/streams/…` request failed from then on), and NOTIFY-gated wakeups died with workers silently falling back to polling. Both paths now connect through the new `Pgbus::DedicatedConnection.connect`, which strips `:variables` and applies each GUC via post-connect `SET` — dedicated connections *keep* the operator's GUCs (`statement_timeout`, `timezone`, …), matching the pooled paths, and the mechanism works on both a direct port and a session-mode pooler. A guard spec confines raw `PG.connect` call sites to `Client` and `DedicatedConnection`, so the next dedicated-connection path can't silently reintroduce the bypass. Refs #352.
43
50
  - **AppSignal main dashboard: job and event lines now split by `job_class` and `routing_key`.** Applied the upstream review suggestions committed on appsignal/public_config#80 to the vendored `dashboard.json`: the "Job status per queue" panel gains a `job_class` wildcard tag filter and label (`%job_class% - %queue% - %status%`), and "Event handler status" gains `routing_key` (`%handler% - %status% - %routing_key%`). Without the wildcard tag entry a `%placeholder%` in the line label never resolves, so lines from different job classes / routing keys were collapsed together. The subscriber already emitted both tags; only the dashboard definition lagged.
@@ -8,6 +8,13 @@ class UpgradePgmqToV<%= target_version_slug.camelize %> < ActiveRecord::Migratio
8
8
  # This uses the vendored SQL which doesn't require the pgmq extension.
9
9
  execute Pgbus::PgmqSchema.install_sql("<%= target_version %>")
10
10
 
11
+ # Re-install the per-queue NOTIFY insert triggers the function drop
12
+ # cascaded away (pgmq.notify_insert_throttle records which queues had
13
+ # them and at what throttle; enable_notify_insert is idempotent).
14
+ # Without this, NOTIFY-gated wakeups silently die and workers fall back
15
+ # to polling until each queue is re-ensured by a process restart.
16
+ execute Pgbus::PgmqSchema.reinstall_notify_triggers_sql
17
+
11
18
  # Record the upgrade
12
19
  execute <<~SQL
13
20
  CREATE TABLE IF NOT EXISTS pgbus_pgmq_schema_versions (
data/lib/pgbus/client.rb CHANGED
@@ -89,15 +89,17 @@ module Pgbus
89
89
  # persistent connection instead of a fresh PG.connect per call. Its own
90
90
  # PGMQ::Client → its own connection_pool, sized independently of worker
91
91
  # thread counts.
92
- # Build the streams pool from streams_connection_options (which defaults
93
- # to connection_options but honors streams_database_url/host/port for a
94
- # separate/direct streams DB — issue #315), bounds-applied, and tagged
95
- # with a per-process application_name so the autoscaler can count peer
96
- # processes from pg_stat_activity (issue #323 P1/P2). Snapshot it so a
97
- # hot-swap rebuilds a byte-identical pool at a new size.
92
+ # Build the streams pool from streams_pool_connection_options (defaults
93
+ # to streams_connection_options so a separate streams DB carries the
94
+ # pool with it — issue #315 but overridable via streams_pool_* so a
95
+ # pooler-bypass install keeps the pool off the direct port issue
96
+ # #358), bounds-applied, and tagged with a per-process application_name
97
+ # so the autoscaler can count peer processes from pg_stat_activity
98
+ # (issue #323 P1/P2). Snapshot it so a hot-swap rebuilds a
99
+ # byte-identical pool at a new size.
98
100
  @streams_conn_opts = wrap_session_gucs(
99
101
  tag_application_name(
100
- apply_connection_bounds(config.streams_connection_options)
102
+ apply_connection_bounds(config.streams_pool_connection_options)
101
103
  )
102
104
  )
103
105
  @streams_pgmq = PGMQ::Client.new(@streams_conn_opts, pool_size: config.streams_pool_size,
@@ -16,25 +16,30 @@ module Pgbus
16
16
  #
17
17
  # Options:
18
18
  # to: Maximum concurrent jobs for the same key (required)
19
- # key: Proc receiving job arguments, returns a string key. Default: job class name.
19
+ # key: Proc receiving job arguments, returns a string key.
20
+ # Default: the ENQUEUED job's class name (resolved at
21
+ # resolve time, so an inherited declaration keys each
22
+ # subclass separately — issue #357).
20
23
  # duration: Safety expiry for semaphore (default: 15 minutes)
21
24
  # on_conflict: What to do when limit is reached — :block, :discard, or :raise (default: :block)
22
25
  def limits_concurrency(to:, key: nil, duration: 15 * 60, on_conflict: :block) # rubocop:disable Naming/MethodParameterName
23
26
  raise ArgumentError, "to: must be a positive integer" unless to.is_a?(Integer) && to.positive?
24
27
  raise ArgumentError, "on_conflict must be :block, :discard, or :raise" unless %i[block discard raise].include?(on_conflict)
25
28
  raise ArgumentError, "duration must be a positive number" unless duration.is_a?(Numeric) && duration.positive?
26
- raise ArgumentError, "key must be callable (Proc or lambda)" if key && !key.respond_to?(:call)
29
+ raise ArgumentError, "key must be callable (Proc or lambda)" if !key.nil? && !key.respond_to?(:call)
27
30
 
28
31
  @pgbus_concurrency = {
29
32
  limit: to,
30
- key: key || ->(*) { name },
33
+ key: key,
31
34
  duration: duration,
32
35
  on_conflict: on_conflict
33
36
  }.freeze
34
37
  end
35
38
 
39
+ # The nearest declaration in the ancestor chain wins — same inheritance
40
+ # contract as Uniqueness#pgbus_uniqueness (issue #357).
36
41
  def pgbus_concurrency
37
- @pgbus_concurrency
42
+ @pgbus_concurrency || (superclass.pgbus_concurrency if superclass.respond_to?(:pgbus_concurrency))
38
43
  end
39
44
  end
40
45
 
@@ -47,13 +52,11 @@ module Pgbus
47
52
  config = active_job.class.pgbus_concurrency
48
53
  return nil unless config
49
54
 
50
- args = active_job.arguments
51
- last = args.last
52
- if last.is_a?(Hash) && last.each_key.all?(Symbol)
53
- config[:key].call(*args[...-1], **last)
54
- else
55
- config[:key].call(*args)
56
- end
55
+ # Class-name default, resolved from the ENQUEUED job's class so an
56
+ # inherited declaration keys each subclass separately (#357).
57
+ return active_job.class.name unless config[:key]
58
+
59
+ Support.call_key_proc(config[:key], active_job.arguments)
57
60
  end
58
61
 
59
62
  # Inject the resolved concurrency key into the job's serialized payload.
@@ -167,6 +167,7 @@ module Pgbus
167
167
  :streams_durable_patterns, :streams_broadcast_queue,
168
168
  :streams_presence_patterns, :streams_presence_member,
169
169
  :streams_host, :streams_port, :streams_database_url,
170
+ :streams_pool_host, :streams_pool_port, :streams_pool_database_url,
170
171
  :streams_pool_size, :streams_pool_timeout,
171
172
  :streams_fanout_write_deadline_ms, :streams_dispatch_queue_limit,
172
173
  :streams_writer_threads, :streams_writer_buffer_limit,
@@ -338,6 +339,22 @@ module Pgbus
338
339
  # thread-safe and streams keep using the single serialized connection.
339
340
  @streams_pool_size = 5
340
341
  @streams_pool_timeout = 5
342
+ # Streams-POOL-only connection overrides (issue #358). The pool above
343
+ # defaults to wherever the streamer connects (streams_host/port/
344
+ # database_url), which is right for a separate streams database — but
345
+ # wrong for the pooler-bypass pattern, where streams_port pins the
346
+ # LISTEN connection to the direct Postgres port purely because LISTEN
347
+ # dies in transaction pooling. The pool's traffic (broadcast INSERTs,
348
+ # replay reads) is plain pooler-safe SQL, and on a direct port with a
349
+ # low max_connections ceiling every process's streams_pool_size
350
+ # connections eat scarce direct slots. Set any of these to route the
351
+ # POOL independently of the LISTEN connection:
352
+ #
353
+ # c.streams_port = 5432 # LISTEN bypasses the pooler
354
+ # c.streams_pool_port = 6432 # the pool stays on the pooler
355
+ @streams_pool_host = nil
356
+ @streams_pool_port = nil
357
+ @streams_pool_database_url = nil
341
358
  # Self-tuning streams-pool autoscaling (issue #323). Opt-in, default off.
342
359
  # When true, a per-web-process control loop grows the dedicated streams
343
360
  # pool into a FAIR SHARE of live Postgres connection headroom under
@@ -1214,6 +1231,33 @@ module Pgbus
1214
1231
  override_connection_options(url: streams_database_url, host: streams_host, port: streams_port)
1215
1232
  end
1216
1233
 
1234
+ # Connection options for the dedicated streams PGMQ pool — durable-broadcast
1235
+ # publish INSERTs and the dispatcher's replay reads (issue #315). Defaults
1236
+ # to `streams_connection_options`, so a genuinely separate streams database
1237
+ # carries the pool with it. When any of `streams_pool_database_url` /
1238
+ # `streams_pool_host` / `streams_pool_port` is set, the pool is routed
1239
+ # independently — applied to the BASE options, exactly like the other two
1240
+ # override groups.
1241
+ #
1242
+ # Why this exists (issue #358): only the streamer's LISTEN connection needs
1243
+ # to bypass a transaction-mode pooler (LISTEN dies at COMMIT boundaries);
1244
+ # the pool's INSERT/SELECT traffic is pooler-safe. Without this split, a
1245
+ # `streams_port` direct-port pin drags streams_pool_size connections per
1246
+ # process onto the direct port's low max_connections ceiling:
1247
+ #
1248
+ # c.streams_port = 5432 # LISTEN bypasses the pooler
1249
+ # c.streams_pool_port = 6432 # the pool stays on the pooler
1250
+ #
1251
+ # Precedence: streams_pool_database_url > streams_pool_host/port over the
1252
+ # base > streams_connection_options.
1253
+ def streams_pool_connection_options
1254
+ return streams_connection_options unless streams_pool_database_url || streams_pool_host || streams_pool_port
1255
+
1256
+ override_connection_options(
1257
+ url: streams_pool_database_url, host: streams_pool_host, port: streams_pool_port
1258
+ )
1259
+ end
1260
+
1217
1261
  # Connection options for the Worker's dedicated NotifyListener connection.
1218
1262
  # Mirrors streams_connection_options: defaults to the base connection_options,
1219
1263
  # overridable via worker_notify_database_url / worker_notify_host /
@@ -32,13 +32,19 @@ module Pgbus
32
32
  def verdict
33
33
  queues = @data_source.queues_with_metrics
34
34
  processes = @data_source.processes
35
+ stream_names = @data_source.stream_queue_names
35
36
  health, health_error = safe_queue_health
36
37
 
37
- # Partition queues once: non-DLQ (the operational set) and the subset
38
- # of those with visible, claimable backlog. Paused queues are removed
39
- # from the STALLED backlog (an intentional pause is not the wedge —
40
- # it's reported under DEGRADED), but kept in `non_dlq` for the summary.
41
- non_dlq = queues.reject { |q| dlq?(q) }
38
+ # Partition queues once: the operational set excludes DLQs and
39
+ # registered stream queues, and the backlog is the subset with
40
+ # visible, claimable messages. Stream queues are excluded like DLQs
41
+ # (issue #359): stream delivery is a non-consuming peek, so their
42
+ # messages are permanently visible with read_ct=0 exactly the wedge
43
+ # signature — and counting them makes the verdict scream STALLED on
44
+ # every streams-heavy install. Paused queues are removed from the
45
+ # STALLED backlog (an intentional pause is not the wedge — it's
46
+ # reported under DEGRADED), but kept in `non_dlq` for the summary.
47
+ non_dlq = queues.reject { |q| dlq?(q) || stream_names.include?(q[:name].to_s) }
42
48
  backlog = non_dlq.select { |q| q[:queue_visible_length].to_i.positive? }
43
49
  active_backlog = backlog.reject { |q| q[:paused] }
44
50
 
@@ -135,6 +135,46 @@ module Pgbus
135
135
  SQL
136
136
  end
137
137
 
138
+ # SQL to re-install the per-queue NOTIFY insert triggers that
139
+ # drop_pgmq_functions_sql cascades away (issue #360). Dropping
140
+ # pgmq.notify_queue_listeners() with CASCADE also drops the
141
+ # trigger_notify_queue_insert_listeners trigger from every queue table;
142
+ # install_sql re-creates the function but nothing re-creates the
143
+ # triggers, so NOTIFY-gated wakeups silently die fleet-wide until each
144
+ # queue happens to be re-ensured by a process restart.
145
+ #
146
+ # The dropped state is fully recoverable: pgmq.notify_insert_throttle is
147
+ # a TABLE (preserved by the upgrade — its rows record exactly which
148
+ # queues had notify enabled and at what throttle, FK-bound to pgmq.meta
149
+ # so it can't reference a dropped queue), and pgmq.enable_notify_insert
150
+ # is idempotent. Replaying it per recorded row restores every trigger at
151
+ # its original interval. No-ops when the throttle table or the enable
152
+ # function is absent (a vendored version without the notify feature).
153
+ def reinstall_notify_triggers_sql
154
+ <<~SQL
155
+ DO $$
156
+ DECLARE
157
+ r RECORD;
158
+ BEGIN
159
+ IF to_regclass('pgmq.notify_insert_throttle') IS NULL THEN
160
+ RETURN;
161
+ END IF;
162
+ IF to_regprocedure('pgmq.enable_notify_insert(text, integer)') IS NULL THEN
163
+ RETURN;
164
+ END IF;
165
+
166
+ -- The FOR loop iterates a snapshot, so enable_notify_insert's
167
+ -- internal DELETE + re-INSERT of the same throttle row is safe.
168
+ FOR r IN
169
+ SELECT queue_name, throttle_interval_ms
170
+ FROM pgmq.notify_insert_throttle
171
+ LOOP
172
+ PERFORM pgmq.enable_notify_insert(r.queue_name, r.throttle_interval_ms);
173
+ END LOOP;
174
+ END $$;
175
+ SQL
176
+ end
177
+
138
178
  private
139
179
 
140
180
  # Strips extension-specific blocks (pg_extension_config_dump, pg_depend checks)
@@ -139,7 +139,7 @@ module Pgbus
139
139
  return nil unless config
140
140
  return nil unless config[:strategy] == :until_executed
141
141
 
142
- key = resolve_uniqueness_key(config, task)
142
+ key = resolve_uniqueness_key(job_class, config, task)
143
143
  return nil unless key
144
144
 
145
145
  acquired = UniquenessKey.acquire!(key, queue_name: resolve_queue(task), msg_id: 0)
@@ -155,6 +155,7 @@ module Pgbus
155
155
  "[Pgbus] Uniqueness lock check failed for #{task.key}: #{e.class}: #{e.message} — " \
156
156
  "skipping enqueue to prevent duplicates"
157
157
  end
158
+ ErrorReporter.report(e, { action: "recurring_uniqueness_lock", task: task.key })
158
159
  :already_locked # Fail closed — skip enqueue when lock check errors
159
160
  end
160
161
 
@@ -168,19 +169,39 @@ module Pgbus
168
169
  end
169
170
 
170
171
  # Resolve the uniqueness key for a recurring task.
171
- # For no-argument recurring jobs, the key defaults to the class name.
172
- def resolve_uniqueness_key(config, task)
172
+ # With no explicit key: the key is the SCHEDULED job class's name
173
+ # resolved here, not via a stored proc, so a declaration inherited from
174
+ # a base class keys each subclass separately (issue #357) — qualified
175
+ # with the task's arguments so two recurring tasks pointing at the same
176
+ # job class with different args: don't share one lock (the scheduler
177
+ # can't use #333's raise-at-enqueue guard, so it disambiguates instead).
178
+ #
179
+ # Every failure here — default path or user-supplied key proc — must
180
+ # propagate to acquire_uniqueness_lock's fail-closed rescue (skip this
181
+ # tick, log, report). Returning nil instead would mean "no key
182
+ # configured", and the task would be enqueued WITHOUT a lock: a broken
183
+ # key proc silently disabling the very protection it configures. The
184
+ # warn adds the failing-proc context before re-raising.
185
+ def resolve_uniqueness_key(job_class, config, task)
173
186
  key_proc = config[:key]
174
187
  args = task.arguments || []
188
+ return default_uniqueness_key(job_class, args) unless key_proc
175
189
 
176
- if args.empty?
177
- key_proc.call
178
- else
179
- key_proc.call(*args)
190
+ begin
191
+ args.empty? ? key_proc.call : key_proc.call(*args)
192
+ rescue StandardError => e
193
+ Pgbus.logger.warn { "[Pgbus] Could not resolve uniqueness key for #{task.key}: #{e.message}" }
194
+ raise
180
195
  end
181
- rescue StandardError => e
182
- Pgbus.logger.warn { "[Pgbus] Could not resolve uniqueness key for #{task.key}: #{e.message}" }
183
- nil
196
+ end
197
+
198
+ # Class-name default for the scheduler path, args-qualified so distinct
199
+ # argument sets get distinct locks. JSON keeps the key deterministic
200
+ # and readable in the dashboard/locks table.
201
+ def default_uniqueness_key(job_class, args)
202
+ return job_class.name if args.empty?
203
+
204
+ "#{job_class.name}:#{JSON.generate(args)}"
184
205
  end
185
206
 
186
207
  # Inject uniqueness metadata into the payload so the executor releases
@@ -197,7 +218,7 @@ module Pgbus
197
218
  return payload unless config
198
219
  return payload unless config[:strategy] == :until_executed
199
220
 
200
- key = resolve_uniqueness_key(config, task)
221
+ key = resolve_uniqueness_key(job_class, config, task)
201
222
  return payload unless key
202
223
 
203
224
  payload.merge(
@@ -0,0 +1,22 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Pgbus
4
+ # Shared internal helpers.
5
+ module Support
6
+ module_function
7
+
8
+ # Call a user-supplied key proc with a job's arguments, preserving
9
+ # ActiveJob's keyword-argument convention: a trailing Hash whose keys are
10
+ # all Symbols is splatted as keywords; everything else is positional.
11
+ # Single implementation for Uniqueness.resolve_key and
12
+ # Concurrency.resolve_key so the dispatch semantics cannot drift apart.
13
+ def call_key_proc(key_proc, args)
14
+ last = args.last
15
+ if last.is_a?(Hash) && last.each_key.all?(Symbol)
16
+ key_proc.call(*args[...-1], **last)
17
+ else
18
+ key_proc.call(*args)
19
+ end
20
+ end
21
+ end
22
+ end
@@ -60,7 +60,7 @@ module Pgbus
60
60
  raise ArgumentError, "unknown keyword: #{opts.keys.first.inspect}" if opts.any?
61
61
  raise ArgumentError, "strategy must be one of: #{VALID_STRATEGIES.join(", ")}" unless VALID_STRATEGIES.include?(strategy)
62
62
  raise ArgumentError, "on_conflict must be one of: #{VALID_CONFLICTS.join(", ")}" unless VALID_CONFLICTS.include?(on_conflict)
63
- raise ArgumentError, "key must be callable (Proc or lambda)" if key && !key.respond_to?(:call)
63
+ raise ArgumentError, "key must be callable (Proc or lambda)" if !key.nil? && !key.respond_to?(:call)
64
64
 
65
65
  # Record whether an explicit key was given. With NO explicit key the key
66
66
  # defaults to the class name; that is safe for a no-argument job (one
@@ -70,16 +70,26 @@ module Pgbus
70
70
  # order into ONE per-class singleton. resolve_key raises at resolve time
71
71
  # when the class-name default meets non-empty arguments (see #333); the
72
72
  # no-arg case keeps working. explicit_key marks which is which.
73
+ #
74
+ # No proc is stored for the default — resolve_key derives it from the
75
+ # ENQUEUED job's class. A stored `->(*) { name }` would capture the
76
+ # DECLARING class, so a base-class declaration would collapse every
77
+ # subclass into one shared key once configs became inheritable (#357).
73
78
  @pgbus_uniqueness = {
74
79
  strategy: strategy,
75
- key: key || ->(*) { name },
80
+ key: key,
76
81
  explicit_key: !key.nil?,
77
82
  on_conflict: on_conflict
78
83
  }.freeze
79
84
  end
80
85
 
86
+ # The nearest declaration in the ancestor chain wins: a class's own
87
+ # `ensures_uniqueness` beats an inherited one, and a base-class
88
+ # declaration reaches every subclass (issue #357 — class-level ivars
89
+ # don't inherit, so without the superclass walk a base declaration was
90
+ # silently inert for subclasses).
81
91
  def pgbus_uniqueness
82
- @pgbus_uniqueness
92
+ @pgbus_uniqueness || (superclass.pgbus_uniqueness if superclass.respond_to?(:pgbus_uniqueness))
83
93
  end
84
94
  end
85
95
 
@@ -90,11 +100,12 @@ module Pgbus
90
100
 
91
101
  args = active_job.arguments
92
102
  guard_class_name_default!(active_job, config, args)
93
- last = args.last
94
- key = if last.is_a?(Hash) && last.each_key.all?(Symbol)
95
- config[:key].call(*args[...-1], **last)
103
+ key = if config[:explicit_key]
104
+ Support.call_key_proc(config[:key], args)
96
105
  else
97
- config[:key].call(*args)
106
+ # Class-name default, resolved from the ENQUEUED job's class so
107
+ # an inherited declaration keys each subclass separately (#357).
108
+ active_job.class.name
98
109
  end
99
110
 
100
111
  # Automatically serialize GlobalID-compatible objects (e.g. ActiveRecord models)
data/lib/pgbus/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Pgbus
4
- VERSION = "0.12.0"
4
+ VERSION = "0.12.1"
5
5
  end
@@ -65,6 +65,19 @@ module Pgbus
65
65
  end
66
66
  private :fetch_queues_with_metrics
67
67
 
68
+ # Physical queue names registered as stream queues (the
69
+ # pgbus_stream_queues registry). Stream delivery is a non-consuming
70
+ # peek, so stream messages sit visible with read_ct=0 forever — health
71
+ # verdicts must not read that as a wedge (issue #359). Loaded fresh per
72
+ # call (reset + one query): verdicts run on coarse intervals and a
73
+ # long-lived process must see streams registered since the last check.
74
+ # Degrades to an empty Set when the registry table is absent
75
+ # (pre-migration installs) or unreadable — StreamQueue swallows both.
76
+ def stream_queue_names
77
+ StreamQueue.reset_cache!
78
+ StreamQueue.all_names
79
+ end
80
+
68
81
  # name is the full PGMQ queue name (e.g. "pgbus_default") as returned
69
82
  # by queues_with_metrics. No prefix is added.
70
83
  def queue_detail(name)
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pgbus
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.12.0
4
+ version: 0.12.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mikael Henriksson
@@ -365,6 +365,7 @@ files:
365
365
  - lib/pgbus/streams/turbo_broadcastable.rb
366
366
  - lib/pgbus/streams/turbo_stream_override.rb
367
367
  - lib/pgbus/streams/watermark_cache_middleware.rb
368
+ - lib/pgbus/support.rb
368
369
  - lib/pgbus/table_maintenance.rb
369
370
  - lib/pgbus/testing.rb
370
371
  - lib/pgbus/testing/assertions.rb