pgbus 0.16.5 → 0.16.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 4382e9b750a7e3538c2ba640aa476bb914cd1c1deba00fda429510ea1dcdc851
4
- data.tar.gz: bc8bff2b3b08ccc6de7d67c677a8797be4d0a1b9f65019a9728d0531ad5a830a
3
+ metadata.gz: 82a6db71dd099a9b4fa24cb0a278184a3aed8bbee7acfb3f3cc6ef6b55064038
4
+ data.tar.gz: ad7034ee934aee91afdb4a6278200f8178caf0da609d49e57bc6bbcf27d3b7b5
5
5
  SHA512:
6
- metadata.gz: 9630731c5e8b4ef6bb9db80e1e950ad6abee7748e49868fb613645b0d16469f477a6b0fe819427135a6bdf85589fb518e6204f80a3a20810939ed1c2038238a7
7
- data.tar.gz: b4c5b23b4e4513b5447985ab96b1578a3a1931fda6b3b4700338b71d7d822a989cda6498670a31e7ef30848f69f4075b23a2f018764072f51b6e91ae258d8445
6
+ metadata.gz: 86977d3f705b8e3732211956582334f6e49a60e426af1082fdf585a175c9a189c9dc9ec83d5ceeaac2f0be7dcee40c1414e55a99225fcc90ac955537d8b4f01c
7
+ data.tar.gz: 3af2ae55db88774c89f3082b70e5456201c9a85ffd994d43beb197fe0f52d282209fc5de57db66c44de10afe34b37d37fa5404522eabb97d4eb2e91cbfc80da7
data/CHANGELOG.md CHANGED
@@ -2,12 +2,21 @@
2
2
 
3
3
  ### Added
4
4
 
5
+ - **`coalesce:` now works on the `Turbo::StreamsChannel` broadcast path.** `Pgbus::Streams::Stream#broadcast` has accepted `coalesce:` since #171, but the Turbo patch forwarded only four options (`durable:`, `exclude:`, `visible_to:`, `event:`), so nothing that broadcasts through `Turbo::StreamsChannel.broadcast_*_to` — every `Turbo::Broadcastable` model, and phlex-reactive's `Streamable.broadcast_to` — could debounce. A background job settling a 177-row collection therefore pushed 177 idempotent replaces of the same count badge at every peer. Any targeted helper now takes `coalesce:` (`@order.broadcast_replace_to :account, coalesce: 100`), as does a direct channel call (`Turbo::StreamsChannel.broadcast_replace_to(@account, target: "order-count", coalesce: true)`), with the same semantics as the direct API: opt-in, per `(stream, target)`, last-write-wins, `true` = the 50ms default window, an integer = the window in ms. Two things the option's plumbing needed beyond a fifth thread-local: the coalescing key, which dedupes on `(stream, target)` while `broadcast_stream_to` only ever sees the rendered `content:` — it is now resolved from the frame's own `target:`/`targets:` through the same `ActionView::RecordIdentifier.dom_id` conversion Turbo applies, so a record target keys on `order_7` rather than on a per-instance `#to_s`; and extraction of the pgbus options at the channel itself, since a direct `Turbo::StreamsChannel` call never passes through `Turbo::Broadcastable` and so previously leaked `durable: true` (and now `coalesce:`) into Turbo's renderer. `broadcast_refresh_to` and `broadcast_render_to` carry no target and so cannot coalesce — they raise the existing actionable "requires target:" error rather than guessing a key — and the `_later_to` variants remain out of scope, as the thread-local cannot follow into a job. The wire is byte-identical when `coalesce:` is absent. Closes #465.
6
+
5
7
  - **Concurrency is now visible: a section on the Locks page, three gauges, a home stat card and an MCP tool.** `limits_concurrency … on_conflict: :block` became durable in #460 — a parked job is never dropped and a key never runs past `to:` — but nothing showed it, so an operator whose per-record pipeline looked "stuck" had no way to see that the key's slot was held, how many jobs were parked behind it, or how long the oldest had waited, short of SQL. `pgbus_semaphores` and `pgbus_blocked_executions` appeared on no page, in no gauge, in no tool. One new read, `Web::DataSource#concurrency_stats`, now feeds four surfaces: the Locks page's **Concurrency** section (parked jobs, oldest wait, slots held, keys at limit, then a row per key with value / limit, lease state, parked count and oldest wait, auto-refreshing like the other dashboard panels); a **Parked jobs** card on the dashboard home; three unlabelled gauges on both `/pgbus/api/metrics` and the AppSignal probe (`pgbus_concurrency_blocked_executions`, `pgbus_concurrency_blocked_oldest_age_seconds`, `pgbus_concurrency_slots_held` — keys are per-record, so a per-key label would be an unbounded series); and a read-only `pgbus_concurrency` MCP tool returning the same data. The key rows come from a FULL OUTER JOIN, not a LEFT one: a key can have parked jobs and no semaphore (its holder died and the sweep removed the row) or a semaphore and nothing parked, and both shapes matter to an operator. Each row carries two guarded escape hatches. **Release** drops the semaphore and promotes through `Concurrency::BlockedExecution.promote_next`, so it takes each slot through the same guarded upsert an enqueue uses and can never reintroduce the over-admission #460 closed; when the lease is still fresh the confirm says so, because a job is then probably still running and releasing lets another start beside it. **Discard parked** drops a single key's parked jobs, naming the exact count in the confirm, and resolves the bookkeeping those jobs will never resolve themselves — a parked batch child is marked failed so its batch stops waiting and `on_failure` fires (#413), and an `:until_executed` uniqueness lock is released rather than orphaned (#423); each discard logs a warning naming the job and emits `pgbus.blocked_execution_discarded`. There is deliberately no cross-key "discard all parked": not losing work is the whole point of `:block`. Nothing is added to the enqueue or execute path. Refs #461.
6
8
 
7
9
  - **The batches list and a batch's progress panel now refresh themselves.** Batches was the only dashboard list whose `turbo-frame` was missing `data-auto-refresh`, and the batch detail page had no frame at all, so watching a batch drain meant reloading by hand. Both now poll on the existing `web_refresh_interval` (5s, pauses while you interact) like every other list — the batch progress panel is served by `GET /batches/:id?frame=progress`. This reuses the dashboard's turbo-frame polling rather than the SSE stack in `lib/pgbus/web/stream_app.rb`: that is the host-app-facing stream feature, and pointing it at the dashboard would open a PG `LISTEN` connection per viewer to move four integers.
8
10
 
9
11
  ### Fixed
10
12
 
13
+ - **A still-running event handler is no longer re-run as if it had crashed, and a slow one is no longer redelivered mid-run.** Two halves of one hole, follow-up to #469. First: `Process::Consumer` had no equivalent of `ActiveJob::Executor#with_visibility_heartbeat`, so an event message's visibility timeout was never re-armed — a handler slower than `visibility_timeout` (30s by default) was redelivered *while it was still running*, `read_ct` climbed on every redelivery, and after `max_retries` the event was dead-lettered without the handler ever raising. Workers have been protected from this since the heartbeat landed; event consumers never were. The consumer now tracks each message through `VisibilityHeartbeat` for exactly as long as its handlers run, releasing the entry before the archive so a beat can never re-arm a message that is already gone. Second: `Handler#claim_idempotency?` read a `pgbus_processed_events` row with `completed_at IS NULL` as proof that the previous holder had been killed mid-handler, and re-ran. That state equally describes a handler that is simply still running — on another thread, another fork or another host — so a second delivery (a genuine redelivery, or a duplicate envelope, whose independent visibility timeouts no heartbeat can serialize) executed the handler *concurrently with* the live one, which is precisely the double-execution `idempotent!` exists to prevent. The claim now carries liveness rather than only a claim instant: `EventBus::ClaimBeat` refreshes every in-flight claim's `processed_at` from the same beat that re-arms the message's visibility timeout, so message visibility and claim liveness go quiet together when a process dies. A pending claim silent for longer than twice the heartbeat interval is abandoned and re-runs, exactly as before; a fresher one is owned and the delivery skips, deferring to the holder — which either completes (nothing is lost) or fails, leaving its own message for visibility-timeout redelivery to recover. A skip is also no longer silent: `pgbus.event_skipped` carries the reason (`:completed`, `:cached` or `:owned`), the claim's age in seconds and the delivery's `read_ct`, and the metrics subscriber counts it as `pgbus_event_count` with `status: "skipped"` and the reason as a tag. No migration, no new column, and no change on a table that has not run `pgbus:add_processed_event_completion` — a single-phase claim has no pending state and therefore no ownership question. Closes #470.
14
+
15
+ - **An event is no longer handed to every handler whose pattern matches it, so a wildcard handler ran once per subscriber on the topic.** Each subscriber gets its own queue (`Subscriber#setup!` creates and binds one), so a topic with N matching subscribers puts N copies of every event on the bus — but `Consumer#handle_message` resolved handlers with `Registry#handlers_for(routing_key)`, a pattern match that ignored which queue the message had just been read from, and ran every match. Each of the N copies therefore fanned out to all N handlers: **every handler ran N times per event**, with four `"#"` subscribers meaning four invocations of each handler for every event on the bus. `idempotent!` was the only thing hiding it, and it hides it imperfectly by design: the two-phase claim (#385) deliberately *re-runs* when the existing claim is still pending, on the theory that the prior holder crashed — but a pending claim is also exactly what a handler still running on another host looks like. So one event could have host A win the claim and start a 700ms handler while host B, reading a different subscriber's copy of the same event a few hundred milliseconds later, dispatched to that same handler, lost the claim insert, read `completed_at IS NULL` as "crashed, re-run", and executed it concurrently. In one host app that meant a "task completed" record written twice and the user emailed twice, with a single row in `pgbus_processed_events`, `read_ct = 1` on every archived copy, and the second execution's side effects timestamped inside the first's window — dispatch, not redelivery. A non-idempotent handler simply ran N times with no guard at all. `handlers_for` now takes a required `queue_name:` and selects on ownership *and* pattern, so a message read from queue Q goes to Q's owner(s) alone and each handler runs exactly once per event; duplicate execution again requires a real redelivery, which is the at-least-once contract the docs describe. The keyword is required rather than defaulted precisely so a keyword-less call cannot silently restore the fan-out — the two callers that genuinely want the pattern view (the `Testing.inline!` publish path and `Testing::EventStore#drain!`, neither of which has a queue, since the event never reaches PGMQ) moved to an explicitly named `Registry#subscribers_matching`, where per-subscriber delivery counts already match what owner-only dispatch now produces in production. Two handlers registered against the same explicit `queue_name:` still both run on that queue's delivery. The pattern check is kept alongside the ownership check because a routing key the owner's pattern does not match means a stale `pgmq.topic_bindings` row, and a stale binding must not run the handler. A message on a queue no subscriber in this process owns is still archived rather than looped through visibility-timeout redelivery into the DLQ — that was already the behavior when `handlers_for` returned `[]` — but it is no longer silent: it logs a warning naming the queue and routing key, rate-limited to once per queue per process so a permanently stale binding cannot flood the log, and emits `pgbus.event_unrouted` on every occurrence so the real rate stays visible in metrics. The pending-claim re-run in `Handler#claim_idempotency?` is deliberately untouched here; it is a separate question now that dispatch no longer manufactures the concurrency it was misreading. Closes #469.
16
+
17
+ - **A dropped ActiveRecord socket no longer pages the host app for an event that was handled successfully.** EventBus consumers are long-lived threads holding a leased AR connection, and a pooler restart, an admin disconnect or a brief failover can kill that socket at any point. Rails reconnects most statements transparently, but `Relation#update_all` is marked `allow_retry: false` — and that is exactly the phase-2 claim stamp in `Handler#complete_claim!`, the one AR write that happens *after* `handle` has already returned. A drop there surfaced as `ActiveRecord::ConnectionFailed` ("PQconsumeInput() SSL error: unexpected eof while reading") on a message whose work was done, so the host app's exception tracker paged for a false failure and PGMQ redelivered the event for a re-run. `EventBus::StaleConnectionRetry` now reconnects this thread's lease (via `ConnectionPool#active_connection?`, the accessor that exists across the supported Rails range) and repeats the stamp once; a second drop still raises, leaving the message to VT redelivery. Only the leased connections are reconnected — `clear_all_connections!` would yank sockets out from under sibling consumers in the same process, turning one recoverable drop into many. The retryable patterns are deliberately *broader* than `Client::STALE_CONNECTION_PATTERNS` and for the opposite reason: that list excludes mid-flight drops because a half-committed enqueue would duplicate a message, whereas this only ever repeats an idempotent `SET completed_at = <now>`. Phase 1 (`claim_idempotency?`) is deliberately not wrapped — its INSERT may have committed before the socket died, and on a legacy schema the retry's empty `result.rows` would read as "another consumer owns this claim", turning a recoverable drop into a silently skipped event.
18
+
19
+
11
20
  - **The AppSignal probe's gauges and every MCP tool response were frozen at their first read.** `Web::DataSource` memoizes `queues_with_metrics` for the life of the instance — correct for the dashboard, which builds one instance per request, but the AppSignal `Probe::Runner` holds a single DataSource for the life of the process and `MCP::Server.build` injects one into the server context, so both replayed the first read forever: queue depth in AppSignal never moved after the first minute, and `pgbus_queues` returned the same snapshot for every call in a session. `DataSource#reset_cache!` drops the memos, and both long-lived consumers call it — the probe once per iteration, `MCP::BaseTool.data_source_from` once per tool call. Found while reviewing the concurrency aggregates added in the same release, which memoize the same way.
12
21
 
13
22
  - **`limits_concurrency … on_conflict: :block` now constrains a job without ever losing it, and never runs more than `to:` at once.** The same bug class solid_queue closed in rails/solid_queue#712 and #761, checked against pgbus's implementation. Four defects, each with a regression spec:
data/README.md CHANGED
@@ -221,6 +221,14 @@ Pgbus::EventBus::Registry.instance.subscribe(
221
221
  )
222
222
  ```
223
223
 
224
+ Each subscriber gets its own queue, and a queue is consumed only by the
225
+ handler(s) registered against it: one published event matching N subscribers
226
+ becomes N deliveries, one per handler, so every handler runs exactly once per
227
+ event. (Two handlers sharing an explicit `queue_name:` both run on that queue's
228
+ delivery.) A message on a queue no subscriber in this process owns — a stale
229
+ topic binding left by a renamed or removed handler — is archived, logged once
230
+ per queue, and reported as `pgbus.event_unrouted`.
231
+
224
232
  `idempotent!` uses a **two-phase claim**: a *pending* row in
225
233
  `pgbus_processed_events` is inserted before `handle` runs, and only stamped
226
234
  `completed_at` after `handle` returns. Deduplication applies to **completed**
@@ -0,0 +1,80 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Pgbus
4
+ module EventBus
5
+ # Liveness for the idempotency claims in flight for one PGMQ message.
6
+ #
7
+ # A two-phase claim (issue #385) is a `pgbus_processed_events` row with
8
+ # `completed_at IS NULL`. That state says "claimed, not finished" — it does
9
+ # NOT say whether the claimer is dead or simply still running (issue #470).
10
+ # Handler resolves the ambiguity by age: a claim whose `processed_at` has
11
+ # gone quiet for longer than the ownership window is abandoned, anything
12
+ # fresher is owned. For that age to mean "silence" rather than
13
+ # "time since the claim was taken", something has to keep stamping it while
14
+ # the handler runs. This is that something.
15
+ #
16
+ # One beat per message, created by Process::Consumer and handed to every
17
+ # handler dispatched for it. A handler registers its claim for exactly the
18
+ # duration of `handle` and the consumer's VisibilityHeartbeat `on_beat` hook
19
+ # drives #touch! on the same cadence that re-arms the message's visibility
20
+ # timeout — so the claim and the message go quiet together when the process
21
+ # dies, and both stay fresh while it lives.
22
+ #
23
+ # #touch! runs on the heartbeat ticker thread while #register / #release run
24
+ # on a pool thread, hence the mutex.
25
+ class ClaimBeat
26
+ def initialize
27
+ @mutex = Mutex.new
28
+ @claims = []
29
+ end
30
+
31
+ def register(event_id, handler_class)
32
+ claim = [event_id, handler_class]
33
+ @mutex.synchronize { @claims << claim unless @claims.include?(claim) }
34
+ self
35
+ end
36
+
37
+ def release(event_id, handler_class)
38
+ @mutex.synchronize { @claims.delete([event_id, handler_class]) }
39
+ self
40
+ end
41
+
42
+ def size
43
+ @mutex.synchronize { @claims.size }
44
+ end
45
+
46
+ def empty?
47
+ size.zero?
48
+ end
49
+
50
+ # Refresh every registered claim's liveness stamp. Returns the number of
51
+ # claims touched. No-op on a legacy schema: without `completed_at` there
52
+ # are no pending claims to keep alive, and Handler's single-phase
53
+ # fallback never consults the age.
54
+ #
55
+ # A claim that fails to update is logged and skipped rather than raised:
56
+ # this runs inside the visibility heartbeat's beat, and one unwritable
57
+ # row must not cost every other in-flight message its VT extension.
58
+ def touch!
59
+ return 0 unless ProcessedEvent.completion_column?
60
+
61
+ now = Time.now.utc
62
+ @mutex.synchronize { @claims.dup }.count { |event_id, handler_class| touch(event_id, handler_class, now) }
63
+ end
64
+
65
+ private
66
+
67
+ def touch(event_id, handler_class, now)
68
+ ProcessedEvent
69
+ .where(event_id: event_id, handler_class: handler_class, completed_at: nil)
70
+ .update_all(processed_at: now)
71
+ true
72
+ rescue StandardError => e
73
+ Pgbus.logger.warn do
74
+ "[Pgbus] Could not refresh idempotency claim #{handler_class}/#{event_id}: #{e.class}: #{e.message}"
75
+ end
76
+ false
77
+ end
78
+ end
79
+ end
80
+ end
@@ -17,8 +17,20 @@ module Pgbus
17
17
  end
18
18
  end
19
19
 
20
- def process(message)
21
- with_rails_executor { process!(message) }
20
+ # Outcome of the two-phase claim. `age` is how long the losing delivery
21
+ # found the existing claim to have been silent, in seconds (nil unless
22
+ # the claim was pending).
23
+ ClaimResult = Data.define(:status, :age) do
24
+ def granted?
25
+ status == :claimed
26
+ end
27
+ end
28
+
29
+ # @param claim_beat [ClaimBeat, nil] the message's claim-liveness beat,
30
+ # supplied by Process::Consumer. Absent for a hand-rolled caller: the
31
+ # handler still runs, its claim simply ages from the claim instant.
32
+ def process(message, claim_beat: nil)
33
+ with_rails_executor { process!(message, claim_beat) }
22
34
  end
23
35
 
24
36
  def handle(event)
@@ -27,12 +39,18 @@ module Pgbus
27
39
 
28
40
  private
29
41
 
30
- def process!(message)
42
+ def process!(message, claim_beat = nil)
31
43
  raw = JSON.parse(message.message)
32
44
  event = build_event(raw)
33
45
  routing_key = raw.dig("headers", "routing_key") || raw["routing_key"]
34
46
 
35
- return :skipped if self.class.idempotent? && !claim_idempotency?(event.event_id)
47
+ if self.class.idempotent?
48
+ claim = claim_idempotency(event.event_id)
49
+ unless claim.granted?
50
+ instrument_skip(claim, event, message, routing_key)
51
+ return :skipped
52
+ end
53
+ end
36
54
 
37
55
  instrument_payload = {
38
56
  event_id: event.event_id,
@@ -42,11 +60,13 @@ module Pgbus
42
60
  read_ct: message.read_ct.to_i,
43
61
  msg_id: message.msg_id.to_i
44
62
  }
45
- Instrumentation.instrument("pgbus.event_processed", instrument_payload) do
46
- # Publisher's Current attributes (issue #431) are set for the handler
47
- # and reverted after (CurrentAttributes#set semantics); the Rails
48
- # executor wrap above additionally resets at completion.
49
- Pgbus::CurrentAttributes.restore(event.context) { handle(event) }
63
+ with_claim_beat(claim_beat, event.event_id) do
64
+ Instrumentation.instrument("pgbus.event_processed", instrument_payload) do
65
+ # Publisher's Current attributes (issue #431) are set for the handler
66
+ # and reverted after (CurrentAttributes#set semantics); the Rails
67
+ # executor wrap above additionally resets at completion.
68
+ Pgbus::CurrentAttributes.restore(event.context) { handle(event) }
69
+ end
50
70
  end
51
71
  complete_claim!(event.event_id) if self.class.idempotent?
52
72
  :handled
@@ -64,7 +84,7 @@ module Pgbus
64
84
 
65
85
  # Mirrors Pgbus::ActiveJob::Executor#execute_job: wrap the handler
66
86
  # invocation in Rails.application.executor (or the reloader in dev)
67
- # so AR connections leased by `claim_idempotency?` and `handle` are
87
+ # so AR connections leased by `claim_idempotency` and `handle` are
68
88
  # released back to the pool when this method returns. Without the
69
89
  # wrap, every consumed event leaks one AR connection on the consumer
70
90
  # thread — in dev that wedges `clear_reloadable_connections!`,
@@ -113,24 +133,32 @@ module Pgbus
113
133
 
114
134
  # Two-phase idempotency claim (issue #385). Phase 1: atomically claim
115
135
  # via INSERT ... ON CONFLICT DO NOTHING with completed_at NULL — a
116
- # *pending* claim. Returns true when this delivery should run handle:
136
+ # *pending* claim. Returns a ClaimResult whose status is one of:
117
137
  #
118
- # - insert won → fresh claim
119
- # - insert lost, completed_at NULL → a prior attempt claimed but was
120
- # killed before finishing (SIGKILL mid-handler); re-run so the crash
121
- # doesn't silently drop the execution. Safe: PGMQ's VT means the
122
- # prior holder is dead or wedged past its timeout — the same
123
- # at-least-once window every non-idempotent handler has.
138
+ # :claimed — insert won (fresh claim), or the row was purged between
139
+ # the losing insert and the read, or an existing pending
140
+ # claim has gone silent for longer than the ownership
141
+ # window: the holder is dead by the heartbeat's own
142
+ # definition, so re-run rather than silently drop the
143
+ # execution a SIGKILL interrupted.
144
+ # :completed — the execution already finished. Skip.
145
+ # :owned — pending, and its liveness stamp is fresh: the holder is
146
+ # still running (issue #470). Skip — running `handle`
147
+ # concurrently with the holder is exactly the
148
+ # double-execution `idempotent!` promises not to do. The
149
+ # holder either completes (nothing lost) or fails, leaving
150
+ # its own message for VT redelivery to recover.
151
+ # :cached — a completed execution already in this process's memory.
124
152
  #
125
- # Returns false (skip) only for a *completed* execution. Phase 2 is
126
- # complete_claim! after handle returns; only completed executions enter
127
- # the in-memory dedup cache.
153
+ # Phase 2 is complete_claim! after handle returns; only completed
154
+ # executions enter the in-memory dedup cache.
128
155
  #
129
156
  # Legacy fallback: without the completed_at column (upgraded gem,
130
- # not-yet-migrated table) this degrades to the old single-phase claim.
131
- def claim_idempotency?(event_id)
157
+ # not-yet-migrated table) this degrades to the old single-phase claim,
158
+ # which has no pending state and therefore no ownership question.
159
+ def claim_idempotency(event_id)
132
160
  cache_key = dedup_key(event_id)
133
- return false if self.class.dedup_cache.seen?(cache_key)
161
+ return ClaimResult.new(status: :cached, age: nil) if self.class.dedup_cache.seen?(cache_key)
134
162
 
135
163
  result = ProcessedEvent.insert(
136
164
  { event_id: event_id, handler_class: self.class.name, processed_at: Time.now.utc },
@@ -139,18 +167,79 @@ module Pgbus
139
167
 
140
168
  unless ProcessedEvent.completion_column?
141
169
  self.class.dedup_cache.mark!(cache_key)
142
- return result.rows.any?
170
+ return ClaimResult.new(status: result.rows.any? ? :claimed : :completed, age: nil)
143
171
  end
144
172
 
145
- return true if result.rows.any?
173
+ return ClaimResult.new(status: :claimed, age: nil) if result.rows.any?
146
174
 
147
- completed_at = ProcessedEvent
148
- .where(event_id: event_id, handler_class: self.class.name)
149
- .pick(:completed_at)
150
- return true if completed_at.nil? # pending claim (or purged row) → re-run
175
+ inspect_existing_claim(event_id, cache_key)
176
+ end
177
+
178
+ # The insert lost, so a row exists (or existed). `pick` returns nil for
179
+ # the whole row when it has since been purged — not a pending claim,
180
+ # nothing is running, so claim it.
181
+ def inspect_existing_claim(event_id, cache_key)
182
+ completed_at, processed_at = ProcessedEvent
183
+ .where(event_id: event_id, handler_class: self.class.name)
184
+ .pick(:completed_at, :processed_at)
151
185
 
152
- self.class.dedup_cache.mark!(cache_key)
153
- false
186
+ if completed_at
187
+ self.class.dedup_cache.mark!(cache_key)
188
+ return ClaimResult.new(status: :completed, age: nil)
189
+ end
190
+
191
+ return ClaimResult.new(status: :claimed, age: nil) if processed_at.nil?
192
+
193
+ age = Time.now.utc - processed_at.to_time.utc
194
+ return ClaimResult.new(status: :owned, age: age) if age < claim_ownership_window
195
+
196
+ ClaimResult.new(status: :claimed, age: age)
197
+ end
198
+
199
+ # How long a pending claim may stay silent before its holder counts as
200
+ # dead. ClaimBeat refreshes a live claim from the visibility heartbeat,
201
+ # which lands every extension inside [interval, 1.5 * interval] of the
202
+ # previous one — two intervals leaves margin for a late beat without
203
+ # stretching the window past the visibility timeout it rides on.
204
+ #
205
+ # With the heartbeat disabled a claim is never refreshed, so the window
206
+ # degrades to "roughly the first two thirds of one visibility timeout
207
+ # after the claim" — a redelivery, which cannot arrive before the VT has
208
+ # lapsed, still re-runs exactly as it did before issue #470.
209
+ def claim_ownership_window
210
+ Pgbus.configuration.effective_visibility_heartbeat_interval * 2
211
+ end
212
+
213
+ # Register this claim with the message's beat for exactly the duration of
214
+ # handle: before it, there is nothing to keep alive; after it,
215
+ # complete_claim! owns the row and a beat touching processed_at would
216
+ # race the completion stamp.
217
+ def with_claim_beat(claim_beat, event_id)
218
+ return yield unless claim_beat && self.class.idempotent? && ProcessedEvent.completion_column?
219
+
220
+ claim_beat.register(event_id, self.class.name)
221
+ begin
222
+ yield
223
+ ensure
224
+ claim_beat.release(event_id, self.class.name)
225
+ end
226
+ end
227
+
228
+ # A skip used to be silent, which made an over-eager re-run (issue #470)
229
+ # invisible in production: nothing distinguished "deduplicated" from
230
+ # "deferred to a live holder". The claim age and read_ct are what tell
231
+ # an operator which one happened.
232
+ def instrument_skip(claim, event, message, routing_key)
233
+ Instrumentation.instrument(
234
+ "pgbus.event_skipped",
235
+ event_id: event.event_id,
236
+ handler: self.class.name,
237
+ routing_key: routing_key,
238
+ reason: claim.status,
239
+ claim_age: claim.age,
240
+ read_ct: message.read_ct.to_i,
241
+ msg_id: message.msg_id.to_i
242
+ )
154
243
  end
155
244
 
156
245
  # Phase 2: stamp the claim completed and only then admit it to the
@@ -158,12 +247,26 @@ module Pgbus
158
247
  # already cached at claim time). If this write fails, process!'s rescue
159
248
  # re-raises, the consumer leaves the message for VT redelivery, and the
160
249
  # still-pending claim re-runs — at-least-once, never a silent drop.
250
+ #
251
+ # Wrapped in StaleConnectionRetry because this is the one AR write that
252
+ # happens AFTER handle() has already succeeded: a socket dropped here
253
+ # costs the host app a paging exception for work that was in fact done.
254
+ # The stamp is an idempotent `SET completed_at = <now>`, so repeating a
255
+ # statement that may already have committed is safe.
256
+ #
257
+ # Phase 1 (claim_idempotency) is deliberately NOT wrapped. Its INSERT
258
+ # may have committed before the socket died, and on a legacy schema the
259
+ # retry's empty `result.rows` would read as "someone else owns this
260
+ # claim" and report :completed — turning a recoverable drop into a
261
+ # silently skipped event. VT redelivery is the correct recovery there.
161
262
  def complete_claim!(event_id)
162
263
  return unless ProcessedEvent.completion_column?
163
264
 
164
- ProcessedEvent
165
- .where(event_id: event_id, handler_class: self.class.name)
166
- .update_all(completed_at: Time.now.utc)
265
+ StaleConnectionRetry.call(context: self.class.name) do
266
+ ProcessedEvent
267
+ .where(event_id: event_id, handler_class: self.class.name)
268
+ .update_all(completed_at: Time.now.utc)
269
+ end
167
270
  self.class.dedup_cache.mark!(dedup_key(event_id))
168
271
  end
169
272
 
@@ -25,7 +25,7 @@ module Pgbus
25
25
  Pgbus::Testing.store.push_event(event)
26
26
 
27
27
  if Pgbus::Testing.inline? && delay.to_i <= 0
28
- Pgbus::EventBus::Registry.instance.handlers_for(routing_key).each do |subscriber|
28
+ Pgbus::EventBus::Registry.instance.subscribers_matching(routing_key).each do |subscriber|
29
29
  Pgbus::CurrentAttributes.restore(event.context) { subscriber.handler_class.new.handle(event) }
30
30
  end
31
31
  end
@@ -62,7 +62,30 @@ module Pgbus
62
62
  end
63
63
  end
64
64
 
65
- def handlers_for(routing_key)
65
+ # Subscribers a message read from +queue_name+ must be dispatched to
66
+ # (issue #469). Every subscriber gets its own queue, so a topic with N
67
+ # matching subscribers produces N queue copies of each event; selecting by
68
+ # pattern alone fanned every copy out to every match, running each handler
69
+ # N times per event — and, across hosts, concurrently. Ownership is the
70
+ # primary filter; the pattern check still applies because a routing key
71
+ # the owner's pattern does not match means a stale pgmq.topic_bindings
72
+ # row, and a stale binding must not run the handler.
73
+ #
74
+ # +queue_name+ is required on purpose: a keyword-less call would silently
75
+ # restore the fan-out this closed. Callers that genuinely want the
76
+ # pattern view (the Testing inline/drain paths, which never touch a
77
+ # queue) use #subscribers_matching.
78
+ def handlers_for(routing_key, queue_name:)
79
+ @subscribers.select do |s|
80
+ s.queue_name == queue_name && matches?(s.pattern, routing_key)
81
+ end
82
+ end
83
+
84
+ # Pattern-only selection, with no queue in play. Used by the Testing
85
+ # inline/drain paths, where the event never reaches PGMQ and each matching
86
+ # subscriber is invoked exactly once — the same per-subscriber delivery
87
+ # count owner-only dispatch produces in production.
88
+ def subscribers_matching(routing_key)
66
89
  @subscribers.select { |s| matches?(s.pattern, routing_key) }
67
90
  end
68
91
 
@@ -0,0 +1,85 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Pgbus
4
+ module EventBus
5
+ # Reconnect + one retry for an ActiveRecord socket that died underneath a
6
+ # long-lived consumer thread.
7
+ #
8
+ # A consumer holds its leased connection for the life of the message, and a
9
+ # pooler restart (PgBouncer `server_idle_timeout`), an admin disconnect or a
10
+ # brief failover can kill that socket at any point. Rails reconnects most
11
+ # statements transparently, but `Relation#update_all` is marked
12
+ # `allow_retry: false`, so the drop propagates — and the only call that
13
+ # matters here, the phase-2 claim stamp, is exactly an `update_all`. The
14
+ # host app saw a paging exception on a message whose handler had in fact
15
+ # succeeded.
16
+ #
17
+ # Deliberately narrower than `Client::STALE_CONNECTION_PATTERNS`, and for
18
+ # the opposite reason. That list excludes mid-flight drops because a
19
+ # half-committed *enqueue* would duplicate a message on retry. This one
20
+ # only ever wraps an idempotent `UPDATE ... SET completed_at = <now>`, so a
21
+ # statement that may already have committed is safe to repeat — which is
22
+ # what makes the mid-flight shapes retryable here and not there.
23
+ module StaleConnectionRetry
24
+ # The socket dying under a statement, not a server refusing us. A refused
25
+ # or timed-out connection is an outage: retrying it in-process buys
26
+ # nothing and hides the outage behind a doubled statement timeout.
27
+ #
28
+ # "ssl syscall error" is libpq's wording when the peer vanished without a
29
+ # TLS close_notify — the same drop as "unexpected eof while reading",
30
+ # reported from the syscall layer instead. It is in
31
+ # Client::STALE_CONNECTION_PATTERNS for the same reason.
32
+ TRANSIENT_DROP = /
33
+ PQconsumeInput|
34
+ server\ closed\ the\ connection\ unexpectedly|
35
+ unexpected\ eof\ while\ reading|
36
+ ssl\ syscall\ error
37
+ /ix
38
+
39
+ module_function
40
+
41
+ # Runs the block, and on a transient drop reconnects this thread's lease
42
+ # and runs it exactly once more. A second drop raises: the socket is not
43
+ # coming back inside this attempt, and PGMQ's visibility timeout is the
44
+ # right recovery — it redelivers to a healthy consumer.
45
+ #
46
+ # context — an identifier for the log line (the handler class name).
47
+ def call(context: nil, &block)
48
+ block.call
49
+ rescue ActiveRecord::ConnectionFailed => e
50
+ raise unless transient_drop?(e)
51
+
52
+ reconnect_leased!
53
+ Pgbus.logger.warn do
54
+ "[Pgbus::EventBus] Retrying after stale ActiveRecord connection drop" \
55
+ "#{" (#{context})" if context}: #{e.message}"
56
+ end
57
+ block.call
58
+ end
59
+
60
+ def transient_drop?(error)
61
+ return false unless error.is_a?(ActiveRecord::ConnectionFailed)
62
+
63
+ [error.message, error.cause&.message].compact.any? { |message| TRANSIENT_DROP.match?(message) }
64
+ end
65
+
66
+ # Only the connections this thread has actually leased.
67
+ # `clear_all_connections!` would yank sockets out from under sibling
68
+ # consumers sharing the process, turning one recoverable drop into many.
69
+ # `active_connection?` returns the lease or nil, and is the accessor this
70
+ # gem uses everywhere (Streams#current_open_transaction documents why we
71
+ # never reach for `ActiveRecord::Base.connection`). Its `active_connection`
72
+ # alias is `:nodoc:` and does not exist at all before Rails 7.2 — below
73
+ # our floor, so calling it would raise NoMethodError on the one path that
74
+ # exists to recover from an error.
75
+ def reconnect_leased!
76
+ ActiveRecord::Base.connection_handler.each_connection_pool(:all) do |pool|
77
+ connection = pool.active_connection?
78
+ next unless connection
79
+
80
+ connection.reconnect!
81
+ end
82
+ end
83
+ end
84
+ end
85
+ end
@@ -20,6 +20,10 @@ module Pgbus
20
20
  # payload: queue, job_class, msg_id, vt, extensions
21
21
  # pgbus.event_processed — event handler succeeded
22
22
  # pgbus.event_failed — event handler raised; carries :exception_object
23
+ # pgbus.event_unrouted — a consumer read an event from a queue no
24
+ # subscriber in this process owns (stale topic
25
+ # binding); the message is archived
26
+ # payload: queue_name, routing_key
23
27
  # pgbus.stream.broadcast — stream broadcast (sync or deferred)
24
28
  # pgbus.outbox.publish — outbox row created
25
29
  # pgbus.recurring.enqueue — scheduler enqueued a due recurring task
@@ -33,6 +33,7 @@ module Pgbus
33
33
  subscribe("pgbus.job_visibility_extended") { |event| on_job_visibility_extended(event) },
34
34
  subscribe("pgbus.event_processed") { |event| on_event_processed(event) },
35
35
  subscribe("pgbus.event_failed") { |event| on_event_failed(event) },
36
+ subscribe("pgbus.event_skipped") { |event| on_event_skipped(event) },
36
37
  subscribe("pgbus.client.send_message") { |event| on_send_message(event) },
37
38
  subscribe("pgbus.client.send_batch") { |event| on_send_batch(event) },
38
39
  subscribe("pgbus.client.read_batch") { |event| on_read_batch(event) },
@@ -139,6 +140,19 @@ module Pgbus
139
140
  )
140
141
  end
141
142
 
143
+ # An idempotent handler that did not run this delivery. `reason` is what
144
+ # makes the skip readable: :completed/:cached is deduplication working,
145
+ # :owned means a second delivery arrived while the holder was still
146
+ # running and was deferred to it (issue #470).
147
+ def on_event_skipped(event)
148
+ payload = event.payload
149
+ backend.increment(
150
+ "#{METRIC_PREFIX}event_count", 1,
151
+ compact(handler: payload[:handler], routing_key: payload[:routing_key],
152
+ status: "skipped", reason: payload[:reason])
153
+ )
154
+ end
155
+
142
156
  # ── Client (PGMQ wrapper) ─────────────────────────────────────────
143
157
 
144
158
  def on_send_message(event)
@@ -78,6 +78,11 @@ module Pgbus
78
78
  )
79
79
  @registry = EventBus::Registry.instance
80
80
  @circuit_breaker = Pgbus::CircuitBreaker.new(config: config)
81
+ # Queues already warned about for an unroutable message (issue #469).
82
+ # handle_message runs on the execution pool, so this is touched from
83
+ # several threads — Concurrent::Set makes add? the atomic
84
+ # test-and-set the once-per-queue guarantee needs.
85
+ @unrouted_queues = Concurrent::Set.new
81
86
  # stat_buffer: :default means "build one iff config.stats_enabled";
82
87
  # passing an explicit value (including nil) overrides that for tests.
83
88
  @stat_buffer =
@@ -222,10 +227,12 @@ module Pgbus
222
227
  raw = JSON.parse(message.message)
223
228
  routing_key = raw.dig("headers", "routing_key") || raw["routing_key"]
224
229
 
225
- handlers = @registry.handlers_for(routing_key || "")
226
- handlers.each do |subscriber|
227
- handler = subscriber.handler_class.new
228
- handler.process(message)
230
+ handlers = @registry.handlers_for(routing_key || "", queue_name: queue_name)
231
+
232
+ if handlers.empty?
233
+ report_unrouted(queue_name, routing_key)
234
+ else
235
+ dispatch(handlers, message, queue_name)
229
236
  end
230
237
 
231
238
  Pgbus.client.archive_message(queue_name, message.msg_id.to_i)
@@ -247,6 +254,65 @@ module Pgbus
247
254
  @jobs_processed.increment
248
255
  end
249
256
 
257
+ # Run every handler that owns this message, with the message's visibility
258
+ # timeout held open for as long as they take (issue #470).
259
+ #
260
+ # Without this the consumer had no equivalent of
261
+ # ActiveJob::Executor#with_visibility_heartbeat: an event handler slower
262
+ # than config.visibility_timeout (30s by default) was redelivered *while
263
+ # still running*, a second consumer read the same envelope, and the
264
+ # holder's pending idempotency claim was indistinguishable from a claim
265
+ # left by a crash — so the handler ran twice, concurrently.
266
+ #
267
+ # The beat also refreshes the claims the handlers register, which is what
268
+ # lets Handler tell "holder still running" from "holder died": message
269
+ # visibility and claim liveness go quiet together when this process does.
270
+ #
271
+ # Tracking ends before the caller archives — a beat must never re-arm the
272
+ # VT of a message that is already gone (same rule as the executor's).
273
+ def dispatch(handlers, message, queue_name)
274
+ claim_beat = EventBus::ClaimBeat.new
275
+
276
+ VisibilityHeartbeat.track(
277
+ client: Pgbus.client,
278
+ queue_name: queue_name,
279
+ msg_id: message.msg_id.to_i,
280
+ job_class: "EventConsumer",
281
+ config: config,
282
+ on_beat: -> { claim_beat.touch! }
283
+ ) do
284
+ handlers.each { |subscriber| subscriber.handler_class.new.process(message, claim_beat: claim_beat) }
285
+ end
286
+ end
287
+
288
+ # No subscriber in this process owns +queue_name+ (a stale
289
+ # pgmq.topic_bindings row left by a renamed or removed handler, another
290
+ # app bound to the same bus), or the owner's pattern no longer matches the
291
+ # routing key. The message is archived by the caller either way — looping
292
+ # it through VT redelivery would only walk it into the DLQ — but that used
293
+ # to happen in total silence. The warning is rate-limited to once per
294
+ # queue per process so a permanently stale binding cannot flood the log;
295
+ # the instrumentation fires on every message, so the real rate stays
296
+ # visible in metrics (issue #469).
297
+ def report_unrouted(queue_name, routing_key)
298
+ first_for_queue = @unrouted_queues.add?(queue_name)
299
+
300
+ if first_for_queue
301
+ Pgbus.logger.warn do
302
+ "[Pgbus] Consumer read an unroutable event from queue #{queue_name} " \
303
+ "(routing_key=#{routing_key.inspect}): no subscriber in this process owns that queue. " \
304
+ "Archiving. This usually means a stale topic binding — a handler was renamed or removed " \
305
+ "without unbinding its queue. Further unrouted messages on this queue are not logged."
306
+ end
307
+ end
308
+
309
+ Pgbus::Instrumentation.instrument(
310
+ "pgbus.event_unrouted",
311
+ queue_name: queue_name,
312
+ routing_key: routing_key
313
+ )
314
+ end
315
+
250
316
  # Record a job stat for the handled message, mirroring the shape the
251
317
  # executor pushes (Executor#record_stat) so consumer and worker throughput
252
318
  # land in the same pgbus_job_stats table. No-op unless stats are enabled.
@@ -521,6 +587,7 @@ module Pgbus
521
587
  # wait IS its drain window — bound it by the same knob workers use
522
588
  # instead of a hardcoded 30s (issue #386).
523
589
  @pool.wait_for_termination(config.drain_timeout)
590
+ VisibilityHeartbeat.stop
524
591
  @stat_buffer&.stop
525
592
  @heartbeat&.stop
526
593
  restore_signals
@@ -0,0 +1,70 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Pgbus
4
+ module Streams
5
+ # The thread-local channel that carries pgbus-specific broadcast options
6
+ # from a `Turbo::Broadcastable` / `Turbo::StreamsChannel` call site down to
7
+ # `TurboBroadcastable#broadcast_stream_to`, which is the only place with a
8
+ # `Pgbus::Streams::Stream` to hand them to.
9
+ #
10
+ # turbo-rails' broadcast helpers funnel everything through
11
+ # `broadcast_stream_to(*streamables, content:)` — a signature with nowhere
12
+ # to put `durable:`, `exclude:`, `visible_to:`, `event:` or `coalesce:`.
13
+ # Rather than fork turbo's whole helper surface, the patches pull those
14
+ # kwargs out before calling `super` and stash them here for the duration of
15
+ # the broadcast.
16
+ #
17
+ # Both patches (`BroadcastableOverride` on the model concern and
18
+ # `TurboBroadcastable` on the channel) share this module so there is one
19
+ # definition of which keys exist and how they are saved/restored.
20
+ module BroadcastOpts
21
+ # Options a caller may pass to any Turbo broadcast helper. They are
22
+ # deleted from the kwargs so they never reach turbo-rails' renderer.
23
+ KEYS = %i[durable exclude visible_to event coalesce].freeze
24
+
25
+ # `coalesce_target` is not a caller-facing kwarg — it is derived from the
26
+ # broadcast's own `target:`/`targets:` by the channel patch, because
27
+ # coalescing keys on `(stream, target)` and `broadcast_stream_to` never
28
+ # sees the target otherwise.
29
+ THREAD_LOCALS = {
30
+ durable: :pgbus_broadcast_durable,
31
+ exclude: :pgbus_broadcast_exclude,
32
+ visible_to: :pgbus_broadcast_visible_to,
33
+ event: :pgbus_broadcast_event,
34
+ coalesce: :pgbus_broadcast_coalesce,
35
+ coalesce_target: :pgbus_broadcast_coalesce_target
36
+ }.freeze
37
+
38
+ # Removes the pgbus options from `kwargs` (mutating it) and returns them.
39
+ # Keys the caller did not pass are absent from the result, so an outer
40
+ # `BroadcastOpts.with` block's values survive an inner broadcast that
41
+ # doesn't set them.
42
+ def self.extract!(kwargs)
43
+ KEYS.each_with_object({}) do |key, opts|
44
+ opts[key] = kwargs.delete(key) if kwargs.key?(key)
45
+ end
46
+ end
47
+
48
+ # Sets the thread-locals for the given options for the duration of the
49
+ # block, restoring the previous values afterwards (nesting-safe), even on
50
+ # error. Only the keys passed are touched.
51
+ def self.with(**opts)
52
+ previous = {}
53
+
54
+ opts.each do |key, value|
55
+ tl_key = THREAD_LOCALS.fetch(key)
56
+ previous[tl_key] = Thread.current[tl_key]
57
+ Thread.current[tl_key] = value
58
+ end
59
+
60
+ yield
61
+ ensure
62
+ previous.each { |tl_key, value| Thread.current[tl_key] = value }
63
+ end
64
+
65
+ def self.[](key)
66
+ Thread.current[THREAD_LOCALS.fetch(key)]
67
+ end
68
+ end
69
+ end
70
+ end
@@ -40,7 +40,7 @@ module Pgbus
40
40
  # don't understand. We pull them out of kwargs (so they never reach
41
41
  # turbo-rails' renderer) and thread them to broadcast_stream_to via
42
42
  # thread-locals, mirroring the original durable: shim.
43
- PGBUS_BROADCAST_OPTS = %i[durable exclude visible_to event].freeze
43
+ PGBUS_BROADCAST_OPTS = BroadcastOpts::KEYS
44
44
 
45
45
  BROADCAST_METHODS.each do |method_name|
46
46
  define_method(method_name) do |*streamables, **kwargs|
@@ -128,32 +128,19 @@ module Pgbus
128
128
  private
129
129
 
130
130
  def extract_pgbus_broadcast_opts(kwargs)
131
- PGBUS_BROADCAST_OPTS.each_with_object({}) do |key, opts|
132
- opts[key] = kwargs.delete(key) if kwargs.key?(key)
133
- end
131
+ BroadcastOpts.extract!(kwargs)
134
132
  end
135
133
 
136
134
  # Set the pgbus broadcast thread-locals for the duration of the block,
137
135
  # restoring previous values afterwards (nested/concurrent-safe). Only
138
136
  # keys actually passed are touched, so unrelated outer broadcasts keep
139
137
  # their values.
140
- def with_pgbus_broadcast_opts(durable: :__unset__, exclude: :__unset__, visible_to: :__unset__, event: :__unset__)
141
- previous = {}
142
- set = lambda do |tl_key, value|
143
- next if value == :__unset__
144
-
145
- previous[tl_key] = Thread.current[tl_key]
146
- Thread.current[tl_key] = value
147
- end
148
-
149
- set.call(:pgbus_broadcast_durable, durable)
150
- set.call(:pgbus_broadcast_exclude, exclude)
151
- set.call(:pgbus_broadcast_visible_to, visible_to)
152
- set.call(:pgbus_broadcast_event, event)
138
+ def with_pgbus_broadcast_opts(durable: :__unset__, exclude: :__unset__, visible_to: :__unset__,
139
+ event: :__unset__, coalesce: :__unset__, &)
140
+ opts = { durable: durable, exclude: exclude, visible_to: visible_to, event: event, coalesce: coalesce }
141
+ opts.reject! { |_key, value| value == :__unset__ }
153
142
 
154
- yield
155
- ensure
156
- previous.each { |tl_key, value| Thread.current[tl_key] = value }
143
+ BroadcastOpts.with(**opts, &)
157
144
  end
158
145
  end
159
146
  end
@@ -34,9 +34,57 @@ module Pgbus
34
34
  # long as `Turbo.signed_stream_verifier_key` is set, which the Rails
35
35
  # app is already responsible for).
36
36
  module TurboBroadcastable
37
+ # Every targeted broadcast helper (`broadcast_replace_to`,
38
+ # `broadcast_append_to`, `broadcast_remove_to`, …) funnels through
39
+ # `broadcast_action_to`, so this is the one place that sees both the
40
+ # caller's kwargs and the `target:`/`targets:` the frame will carry.
41
+ #
42
+ # It does two things the model-level `BroadcastableOverride` can't:
43
+ #
44
+ # 1. Extracts the pgbus options from a *direct* channel call —
45
+ # `Turbo::StreamsChannel.broadcast_replace_to(..., coalesce: true)` —
46
+ # which never passes through `Turbo::Broadcastable` at all (this is
47
+ # the path phlex-reactive's `Streamable.broadcast_to` takes).
48
+ # 2. Records the coalescing key. `coalesce:` dedupes on
49
+ # `(stream, target)`, but `broadcast_stream_to` only receives the
50
+ # rendered `content:` — the target is gone by then. We resolve it the
51
+ # same way turbo will (`convert_to_turbo_stream_dom_id`), so a record
52
+ # target keys on its stable dom_id rather than its object id.
53
+ #
54
+ # Resolving the key is skipped entirely unless coalescing was actually
55
+ # requested, so the uncoalesced path stays byte-identical and free.
56
+ def broadcast_action_to(*streamables, action:, target: nil, targets: nil, **rendering)
57
+ opts = BroadcastOpts.extract!(rendering)
58
+
59
+ coalesce = opts.key?(:coalesce) ? opts[:coalesce] : BroadcastOpts[:coalesce]
60
+ opts[:coalesce_target] = pgbus_coalesce_target(target, targets) if coalesce
61
+
62
+ BroadcastOpts.with(**opts) do
63
+ super(*streamables, action: action, target: target, targets: targets, **rendering)
64
+ end
65
+ end
66
+
67
+ # The two helpers that don't pass through `broadcast_action_to`. They
68
+ # carry no target, so they can't coalesce (`Stream#broadcast` raises an
69
+ # actionable error, same as `Pgbus.stream(x).broadcast(coalesce:)`
70
+ # without one) — but they still need their pgbus options pulled out of
71
+ # the kwargs, or a direct channel call leaks `durable: true` into
72
+ # turbo's renderer and it ends up as an HTML attribute.
73
+ def broadcast_refresh_to(*streamables, **attributes)
74
+ BroadcastOpts.with(**BroadcastOpts.extract!(attributes)) do
75
+ super(*streamables, **attributes)
76
+ end
77
+ end
78
+
79
+ def broadcast_render_to(*streamables, **rendering)
80
+ BroadcastOpts.with(**BroadcastOpts.extract!(rendering)) do
81
+ super(*streamables, **rendering)
82
+ end
83
+ end
84
+
37
85
  def broadcast_stream_to(*streamables, content:)
38
86
  name = stream_name_from(streamables)
39
- override = Thread.current[:pgbus_broadcast_durable]
87
+ override = BroadcastOpts[:durable]
40
88
  # When no explicit thread-local override is present, let the config
41
89
  # resolver decide: it checks `streams_durable_patterns` first (exact
42
90
  # string or regex match), then falls back to
@@ -46,11 +94,24 @@ module Pgbus
46
94
  durable = override.nil? ? Pgbus.configuration.stream_durable?(name) : override
47
95
  Pgbus.stream(name, durable: durable).broadcast(
48
96
  content,
49
- exclude: Thread.current[:pgbus_broadcast_exclude],
50
- visible_to: Thread.current[:pgbus_broadcast_visible_to],
51
- event: Thread.current[:pgbus_broadcast_event]
97
+ exclude: BroadcastOpts[:exclude],
98
+ visible_to: BroadcastOpts[:visible_to],
99
+ event: BroadcastOpts[:event],
100
+ coalesce: BroadcastOpts[:coalesce],
101
+ target: BroadcastOpts[:coalesce_target]
52
102
  )
53
103
  end
104
+
105
+ private
106
+
107
+ # The coalescing key, resolved exactly as turbo resolves the rendered
108
+ # `target=`/`targets=` attribute. `targets:` (a CSS selector) is the
109
+ # fallback turbo itself uses when `target:` is absent; we drop the `#`
110
+ # selector prefix because the key is never rendered — it only has to be
111
+ # stable and distinct.
112
+ def pgbus_coalesce_target(target, targets)
113
+ convert_to_turbo_stream_dom_id(target) || convert_to_turbo_stream_dom_id(targets)
114
+ end
54
115
  end
55
116
 
56
117
  # Apply the patch to Turbo::StreamsChannel's singleton class. Idempotent:
data/lib/pgbus/testing.rb CHANGED
@@ -58,7 +58,7 @@ module Pgbus
58
58
  event = @mutex.synchronize { @events.first }
59
59
  break unless event
60
60
 
61
- Pgbus::EventBus::Registry.instance.handlers_for(event.routing_key).each do |subscriber|
61
+ Pgbus::EventBus::Registry.instance.subscribers_matching(event.routing_key).each do |subscriber|
62
62
  # Restore the publisher's Current (issue #431) like the consumer does.
63
63
  Pgbus::CurrentAttributes.restore(event.context) { subscriber.handler_class.new.handle(event) }
64
64
  end
data/lib/pgbus/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Pgbus
4
- VERSION = "0.16.5"
4
+ VERSION = "0.16.7"
5
5
  end
@@ -25,9 +25,15 @@ module Pgbus
25
25
  # out with `pgbus_visibility_heartbeat false`.
26
26
  module VisibilityHeartbeat
27
27
  # `concurrency` is `[key, duration]` for a concurrency-limited job, else
28
- # nil — one member, so the struct stays inside an 80-byte slot.
28
+ # nil. `on_beat` is an optional callable run on every extension — the event
29
+ # consumer uses it to refresh its handlers' idempotency claims (issue #470).
30
+ #
31
+ # The ninth member takes the struct out of the 80-byte slot it used to fit
32
+ # (measured: 80 → 160). That is paid at most once per in-flight message, so
33
+ # the whole table is bounded by the execution pool's capacity — a handful of
34
+ # entries per process, not one per enqueued job.
29
35
  Entry = Struct.new(:client, :queue_name, :prefixed, :msg_id, :job_class, :extended_at, :extensions,
30
- :concurrency, keyword_init: true)
36
+ :concurrency, :on_beat, keyword_init: true)
31
37
 
32
38
  # Per-job opt-out, included on ActiveJob::Base by the engine:
33
39
  #
@@ -59,13 +65,14 @@ module Pgbus
59
65
  # @param config [Pgbus::Configuration]
60
66
  # @param concurrency [Array(String, Numeric), nil] semaphore key to keep alive alongside
61
67
  # the message and how far to push its expiry on each beat
68
+ # @param on_beat [#call, nil] run after each extension; its failures are contained
62
69
  def track(client:, queue_name:, msg_id:, prefixed: true, job_class: nil, config: Pgbus.configuration,
63
- concurrency: nil)
70
+ concurrency: nil, on_beat: nil)
64
71
  return yield unless config.visibility_heartbeat
65
72
 
66
73
  entry = Entry.new(client: client, queue_name: queue_name, prefixed: prefixed, msg_id: msg_id.to_i,
67
74
  job_class: job_class, extended_at: monotonic_now, extensions: 0,
68
- concurrency: concurrency)
75
+ concurrency: concurrency, on_beat: on_beat)
69
76
  register(entry, config)
70
77
  begin
71
78
  yield
@@ -130,6 +137,7 @@ module Pgbus
130
137
  entry.extended_at = now
131
138
  entry.extensions += 1
132
139
  touch_semaphore(entry)
140
+ run_on_beat(entry)
133
141
  Instrumentation.instrument(
134
142
  "pgbus.job_visibility_extended",
135
143
  queue: entry.queue_name, job_class: entry.job_class, msg_id: entry.msg_id, vt: vt,
@@ -186,6 +194,17 @@ module Pgbus
186
194
  end
187
195
  end
188
196
 
197
+ # Same containment as touch_semaphore: a lease the beat keeps alive
198
+ # alongside the message must never cost the message its extension.
199
+ def run_on_beat(entry)
200
+ entry.on_beat&.call
201
+ rescue StandardError => e
202
+ Pgbus.logger.warn do
203
+ "[Pgbus::VisibilityHeartbeat] on_beat hook failed for msg_id=#{entry.msg_id} " \
204
+ "queue=#{entry.queue_name}: #{e.class}: #{e.message}"
205
+ end
206
+ end
207
+
189
208
  def forget_parent_entries!
190
209
  return if @pid == ::Process.pid
191
210
 
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pgbus
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.16.5
4
+ version: 0.16.7
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mikael Henriksson
@@ -291,9 +291,11 @@ files:
291
291
  - lib/pgbus/engine.rb
292
292
  - lib/pgbus/error_reporter.rb
293
293
  - lib/pgbus/event.rb
294
+ - lib/pgbus/event_bus/claim_beat.rb
294
295
  - lib/pgbus/event_bus/handler.rb
295
296
  - lib/pgbus/event_bus/publisher.rb
296
297
  - lib/pgbus/event_bus/registry.rb
298
+ - lib/pgbus/event_bus/stale_connection_retry.rb
297
299
  - lib/pgbus/event_bus/subscriber.rb
298
300
  - lib/pgbus/execution_pools.rb
299
301
  - lib/pgbus/execution_pools/async_pool.rb
@@ -375,6 +377,7 @@ files:
375
377
  - lib/pgbus/serializer.rb
376
378
  - lib/pgbus/stat_buffer.rb
377
379
  - lib/pgbus/streams.rb
380
+ - lib/pgbus/streams/broadcast_opts.rb
378
381
  - lib/pgbus/streams/broadcastable_override.rb
379
382
  - lib/pgbus/streams/coalescer.rb
380
383
  - lib/pgbus/streams/cursor.rb