pgbus 0.27.1 → 0.27.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 6b6f949cf25759a3498971f5c00a1ba4c3beee0bdcadac41c6bd978af96ba392
4
- data.tar.gz: 0cd2f5abd43c6f5b380f2439b40553dd707ec0283dc42fbe696d6dd2f0a0172e
3
+ metadata.gz: c71d931a9535fbd5de255faea22fec90d0b39b156e91bf43dcbce1b5741947d9
4
+ data.tar.gz: 3d5603ca4af7d362fc79a4e9da056deb727beea997102c5ab5ad4bca68790891
5
5
  SHA512:
6
- metadata.gz: 2d9fa8de5bcae929a3546fba35c7734e83ca9794aa582adc69cb9fee0aadbd397061324cea4e00fa5fcb77bc79e11e39909ff96fd4c27e83d8f94c7acca46a42
7
- data.tar.gz: 7fba6d23df3a93fa218b2f23cb59b6a942a45ebdd8aadd57cf00f01d3f110adb7eb406086aed7922b0b54931bb937cfe366fef5230bb01fc517dd50d3bb64c0a
6
+ metadata.gz: 85600c2edcff67e30302f53cf06fa0e31acc411700213f5324a3254c85b06c0d8694ed4d29cb0b24a3f49a15e1f488a2af4176dd7f0a6e444c27370edec53db7
7
+ data.tar.gz: c055469e1e4d110b5f64d84d4591c618c5e057d2c2ca901683fc514823d8ff21ac7d0dd9160c17d0be0792f7e806bfb73230a9480342cf3735416f00a81052a0
data/CHANGELOG.md CHANGED
@@ -1,5 +1,26 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ## [0.27.2] - 2026-11-21
4
+
5
+ ### Fixed
6
+
7
+ - **`ActiveJob.perform_all_later` now reads each job's outcome from the job: the adapter's `enqueue_all` sets `successfully_enqueued` (issue #679).** Rails documents that an adapter reports per-job status on the instances, and Rails' own fallback and Solid Queue do; pgbus set only `provider_job_id`, so `successfully_enqueued?` stayed falsy for every job, sent or not. `enqueue_all` now resets `successfully_enqueued = false` and `enqueue_error = nil` on every job first, then marks a job `true` when it was sent (bulk `send_batch` or the individual path) or parked behind its concurrency limit as a blocked execution; a job a uniqueness or concurrency `on_conflict` dropped stays `false`. Errors still raise, as before. A job whose send or park was not confirmed stays `false` (`JobNotUnique` under `:reject`, `ConcurrencyLimitExceeded`, a send error, or a short `send_batch`, which still raises `Pgbus::EnqueueError` and leaves that bulk without ids), and so does every job not yet reached; after a short `send_batch` or a connection error during the send, its message may still be live. A job already sent or parked stays `true`, including one whose bookkeeping afterwards (batch backfill, retry notes) raised. The return value, which ActiveJob reports as the `enqueue_all.active_job` notification's `enqueued_count`, is now the number of jobs marked enqueued, not the number passed in, so a dropped job no longer counts. Rails' `enqueue_all` log line therefore now reports a job dropped on purpose (`on_conflict: :discard` or `:log`) under "Failed enqueuing N jobs". When `enqueue_all` raises, ActiveJob still publishes no `enqueued_count`, so its failed count is the whole batch: after a raise partway through a batch of 3, the log line now reads "Enqueued 1 job … Failed enqueuing 3 jobs". The jobs it lists as enqueued are the ones sent or parked. `perform_later` is unchanged: `enqueue` and `enqueue_at` never set the flag, ActiveJob does. Refs #679.
8
+
9
+ ## [0.27.1] - 2026-11-20
10
+
11
+ ### Fixed
12
+
13
+ - **`Web::DataSource#queues_with_metrics` reads 1 000 queues in a tenth of the time, and a metrics query that fails or times out no longer reads as "no queues" (issue #677).** The read was one statement with nine scalar subqueries per queue (eight scans of each queue table plus one sequence read), so its cost grew with the queue count until `statement_timeout` cancelled it, and the rescue returned `[]`: the dashboard, the dead-letter views, `pgbus_health`, the Prometheus scrape and the AppSignal probe all read a timeout as an empty, healthy system. Inside a caller's transaction the cancelled statement also aborted it.
14
+
15
+ - **One scan per queue, in chunks.** Each queue's metrics are one aggregate over its table, and the queues are read `web_queue_metrics_chunk_size` (new, default 100) per statement, each statement in its own savepoint. Same machine, one message per queue: 100 queues 57.7 ms → 37.4 ms, 1 000 queues 3 711 ms → 373 ms, 5 000 queues 126 s → 1.9 s (`rake bench:queue_metrics`, new). A healthy queue's hash is unchanged.
16
+ - **A failed chunk is visible.** Its queues stay in the list with `metrics_unavailable: true` and `nil` counts; the other chunks still report, and an enclosing transaction stays usable. After 3 chunks that timed out (a cancelled statement, a lock wait, a deadlock, a connection that cannot be established) the rest are not read and come back unavailable at once, so a database where every chunk times out costs a few `statement_timeout`s, not one per chunk; a chunk that fails at once (a queue whose table is gone) never stops the others. Totals (`summary_stats` `total_depth` / `dlq_depth`, the `pgbus_dlq_depth` and AppSignal summary gauges) leave an unmeasured queue out rather than guess, so they read low while the verdict is DEGRADED. New `DataSource#queue_metrics_errors` returns `[]` or `[{ queues: [...], error: "Class: message" }]` (`queues: []` when `pgmq.meta` itself could not be read). `pgbus_health` / `/readyz` report `DEGRADED` with a reason naming the queues; the dashboard queue tables say "Metrics unavailable" in place of the counts; the AppSignal probe skips the queue's gauges and increments `pgbus_queue_metrics_unavailable`; the Prometheus scrape writes no count for it and adds the `pgbus_queue_metrics_unavailable` gauge; the MCP `pgbus_queues` tool carries the flag per row; the dead-letter total counts the rows of a DLQ whose metrics are unavailable (in a savepoint) rather than reading it as 0. An unreadable queue list increments the AppSignal counter untagged, writes one unlabelled Prometheus sample and leaves `pgbus_dlq_depth` out of the scrape; a failure after the queue list was read keeps every queue, flagged.
17
+ - **Names without metrics.** New `DataSource#queue_names` reads `pgmq.meta` only (memoized, cleared by `reset_cache!`). The job list, `dlq_messages`, `dlq_message_detail`, discard-all, priority siblings and the MCP `pgbus_dlq_detail` scan use it and run no metrics statement (the dead-letter page's counts, `dlq_total_count` and `dlq_counts_by_queue`, still read the metrics).
18
+ - **Throughput.** `summary_stats[:throughput_rate]` was always 0.0: `compute_throughput` called `Process.clock_gettime` inside `Pgbus::Web`, where `Process` is `Pgbus::Process`, and rescued the `NoMethodError`. It now reads the clock, never reports a negative rate (a dropped queue lowers the total), and keeps its last complete snapshot while any queue's metrics are unavailable.
19
+
20
+ Refs #677.
21
+
22
+ ## [0.27.0] - 2026-11-18
23
+
3
24
  ### Added
4
25
 
5
26
  - **Process signals: `Pgbus.on_signal(name) { |payload| ... }` runs a block in every web process when any process calls `Pgbus.signal(name, payload = nil)` (issue #674).** This covers a per-process in-memory cache that every Puma worker must drop when its data changes, which had no supported route: the event bus delivers an event to one competing consumer in a worker process, and the streamer only woke SSE connections.
@@ -16,24 +37,19 @@
16
37
 
17
38
  Refs #674.
18
39
 
19
- ### Changed
20
-
21
- - **A member-keyed presence `touch` no longer probes `information_schema` on a table without the seat columns, and a forced re-check no longer restarts the backoff (issue #666).** On a `pgbus_presence_members` table where `rails g pgbus:add_presence_seats` has not run, every member-keyed `touch`, `leave` and `sweep!` re-probed a cached "no seats" answer before it ran and restarted the #665 backoff at 30 s. The streamer touches once per connection per heartbeat, so a process with N member-keyed connections sent N catalog queries every `streams_heartbeat_interval` (15 s by default), and its backoff never took effect. A member-keyed `touch` now uses the cached answer and re-checks only on the backoff's schedule: three touches after a reset send one probe instead of four, and none while a "no" is cached. Assumed: on a table migrated under a running process, such a `touch` can also refresh that member's seat rows until the next re-check (at most 10 minutes, `SCHEMA_RECHECK_MAX_SECONDS`); it is an `UPDATE` of `last_seen_at`, so nothing is deleted. A member-keyed `leave` and `sweep!` still re-check a cached "no" before they delete, so they never delete seat rows on a migrated table (#515), now with one probe instead of up to two, answered from that probe. That re-check leaves the backoff alone: over a fresh "no" it changes neither the step nor the schedule however many disconnects force one, and over a "no" already due it is the timed re-check, one step. `reset_schema_cache!` still re-probes at once and starts again from 30 s. Refs #666.
22
-
23
- - **pgmq-ruby 0.9.0 (was 0.8.2); the gemspec now requires `~> 0.9.0` (issue #656).** What reaches pgbus users is the connection pool: pgmq-ruby's checkout health check now resets a pooled connection in any state other than `CONNECTION_OK`, so a connection a `Thread#raise` left half-way through a reset (`CONNECTION_STARTED`) is recovered on the next checkout. Before, the checkout handed it out again, libpq refused every send (`no connection to the server`), and pgbus's stale-connection retry only recovered it on its last attempt by dropping both pools (#645): now the call succeeds on the first attempt, with no retry, no pool reload and no warning. pgmq-ruby also resets a connection whose call raised `PG::ConnectionBad` or `PG::UnableToSend` before it goes back to the pool (not a connection pgbus hands it through a callable, which it neither resets nor closes: the shared ActiveRecord connection, and dedicated connections in `connection_guc_mode = :session` with database.yml `variables`); native statements re-prepare on the new backend as on any reconnect. pgbus keeps its reload before the last stale retry for a connection that still fails after the health check passed it. A checkout reset of a connection in `connection_guc_mode = :session` with `variables` applies its `SET`s again (#671). `produce_topic` and `produce_batch_topic` now raise `PGMQ::Errors::MessageInvalidError` for a message that is not a String or headers that are neither a String nor nil, as `produce` has since 0.8.0. `Client#publish_to_topic` (what `Pgbus.publish` and the outbox poller call; `Pgbus::Testing`'s fake and inline modes never reach PGMQ) serializes the payload and headers to JSON Strings first, so it is unaffected; `headers: false`, which pgmq-ruby dropped before with an integer delay (and stored as JSON `false` with a fractional one) and now rejects, is passed as nil, so it is dropped at any delay. `spec/integration/connection_failure_spec.rb` now asserts the mid-reset connection is recovered at checkout (it failed on 0.8.2), in place of the two examples that pinned the old last-retry recovery. Refs #656.
40
+ ## [0.26.1] - 2026-11-15
24
41
 
25
42
  ### Fixed
26
43
 
27
- - **`Web::DataSource#queues_with_metrics` reads 1 000 queues in a tenth of the time, and a metrics query that fails or times out no longer reads as "no queues" (issue #677).** The read was one statement with nine scalar subqueries per queue (eight scans of each queue table plus one sequence read), so its cost grew with the queue count until `statement_timeout` cancelled it, and the rescue returned `[]`: the dashboard, the dead-letter views, `pgbus_health`, the Prometheus scrape and the AppSignal probe all read a timeout as an empty, healthy system. Inside a caller's transaction the cancelled statement also aborted it.
44
+ - **A pooled connection in `connection_guc_mode = :session` keeps its database.yml `variables` when it is reset (issue #671).** In `:session` mode the `variables` (`statement_timeout`, `lock_timeout` and the like) are applied with `SET` after connect, and `PG::Connection#reset` opens a new backend without them. pgmq-ruby's checkout health check resets an unhealthy pooled connection (`finished?` or `CONNECTION_BAD` since #332, and an intermediate state such as `CONNECTION_STARTED` since pgmq-ruby 0.9.0), so that connection went on serving the job or streams pool with the server's defaults. Each connection the `:session` connect lambda opens now re-applies its `SET`s when it is reset, whoever resets it. A connection whose `SET`s do not all complete, after a connect or a reset, is closed before the error reaches the caller, so it is never pooled without them; with pg 1.5.9 or later the next checkout resets it and applies the `SET`s again (an older pg cannot reset a closed connection, so it is replaced when the pool is reloaded before the last stale-connection retry; the call itself is retried only for a stale-connection error, so a `SET` that always fails, such as an invalid value, fails each time). The completion pipeline's own reset (#619) no longer applies the `SET`s itself (they ran twice), and a `SET` that fails there is now logged as `Resetting the connection or re-applying its session variables failed` and leaves the connection closed, where before the session carried on without its `variables`. `:options` mode, a String URL and the shared ActiveRecord connection are unaffected. Refs #671.
28
45
 
29
- - **One scan per queue, in chunks.** Each queue's metrics are one aggregate over its table, and the queues are read `web_queue_metrics_chunk_size` (new, default 100) per statement, each statement in its own savepoint. Same machine, one message per queue: 100 queues 57.7 ms → 37.4 ms, 1 000 queues 3 711 ms → 373 ms, 5 000 queues 126 s → 1.9 s (`rake bench:queue_metrics`, new). A healthy queue's hash is unchanged.
30
- - **A failed chunk is visible.** Its queues stay in the list with `metrics_unavailable: true` and `nil` counts; the other chunks still report, and an enclosing transaction stays usable. After 3 chunks that timed out (a cancelled statement, a lock wait, a deadlock, a connection that cannot be established) the rest are not read and come back unavailable at once, so a database where every chunk times out costs a few `statement_timeout`s, not one per chunk; a chunk that fails at once (a queue whose table is gone) never stops the others. Totals (`summary_stats` `total_depth` / `dlq_depth`, the `pgbus_dlq_depth` and AppSignal summary gauges) leave an unmeasured queue out rather than guess, so they read low while the verdict is DEGRADED. New `DataSource#queue_metrics_errors` returns `[]` or `[{ queues: [...], error: "Class: message" }]` (`queues: []` when `pgmq.meta` itself could not be read). `pgbus_health` / `/readyz` report `DEGRADED` with a reason naming the queues; the dashboard queue tables say "Metrics unavailable" in place of the counts; the AppSignal probe skips the queue's gauges and increments `pgbus_queue_metrics_unavailable`; the Prometheus scrape writes no count for it and adds the `pgbus_queue_metrics_unavailable` gauge; the MCP `pgbus_queues` tool carries the flag per row; the dead-letter total counts the rows of a DLQ whose metrics are unavailable (in a savepoint) rather than reading it as 0. An unreadable queue list increments the AppSignal counter untagged, writes one unlabelled Prometheus sample and leaves `pgbus_dlq_depth` out of the scrape; a failure after the queue list was read keeps every queue, flagged.
31
- - **Names without metrics.** New `DataSource#queue_names` reads `pgmq.meta` only (memoized, cleared by `reset_cache!`). The job list, `dlq_messages`, `dlq_message_detail`, discard-all, priority siblings and the MCP `pgbus_dlq_detail` scan use it and run no metrics statement (the dead-letter page's counts, `dlq_total_count` and `dlq_counts_by_queue`, still read the metrics).
32
- - **Throughput.** `summary_stats[:throughput_rate]` was always 0.0: `compute_throughput` called `Process.clock_gettime` inside `Pgbus::Web`, where `Process` is `Pgbus::Process`, and rescued the `NoMethodError`. It now reads the clock, never reports a negative rate (a dropped queue lowers the total), and keeps its last complete snapshot while any queue's metrics are unavailable.
46
+ ## [0.26.0] - 2026-11-15
33
47
 
34
- Refs #677.
48
+ ### Changed
35
49
 
36
- - **A pooled connection in `connection_guc_mode = :session` keeps its database.yml `variables` when it is reset (issue #671).** In `:session` mode the `variables` (`statement_timeout`, `lock_timeout` and the like) are applied with `SET` after connect, and `PG::Connection#reset` opens a new backend without them. pgmq-ruby's checkout health check resets an unhealthy pooled connection (`finished?` or `CONNECTION_BAD` since #332, and an intermediate state such as `CONNECTION_STARTED` since pgmq-ruby 0.9.0), so that connection went on serving the job or streams pool with the server's defaults. Each connection the `:session` connect lambda opens now re-applies its `SET`s when it is reset, whoever resets it. A connection whose `SET`s do not all complete, after a connect or a reset, is closed before the error reaches the caller, so it is never pooled without them; with pg 1.5.9 or later the next checkout resets it and applies the `SET`s again (an older pg cannot reset a closed connection, so it is replaced when the pool is reloaded before the last stale-connection retry; the call itself is retried only for a stale-connection error, so a `SET` that always fails, such as an invalid value, fails each time). The completion pipeline's own reset (#619) no longer applies the `SET`s itself (they ran twice), and a `SET` that fails there is now logged as `Resetting the connection or re-applying its session variables failed` and leaves the connection closed, where before the session carried on without its `variables`. `:options` mode, a String URL and the shared ActiveRecord connection are unaffected. Refs #671.
50
+ - **A member-keyed presence `touch` no longer probes `information_schema` on a table without the seat columns, and a forced re-check no longer restarts the backoff (issue #666).** On a `pgbus_presence_members` table where `rails g pgbus:add_presence_seats` has not run, every member-keyed `touch`, `leave` and `sweep!` re-probed a cached "no seats" answer before it ran and restarted the #665 backoff at 30 s. The streamer touches once per connection per heartbeat, so a process with N member-keyed connections sent N catalog queries every `streams_heartbeat_interval` (15 s by default), and its backoff never took effect. A member-keyed `touch` now uses the cached answer and re-checks only on the backoff's schedule: three touches after a reset send one probe instead of four, and none while a "no" is cached. Assumed: on a table migrated under a running process, such a `touch` can also refresh that member's seat rows until the next re-check (at most 10 minutes, `SCHEMA_RECHECK_MAX_SECONDS`); it is an `UPDATE` of `last_seen_at`, so nothing is deleted. A member-keyed `leave` and `sweep!` still re-check a cached "no" before they delete, so they never delete seat rows on a migrated table (#515), now with one probe instead of up to two, answered from that probe. That re-check leaves the backoff alone: over a fresh "no" it changes neither the step nor the schedule however many disconnects force one, and over a "no" already due it is the timed re-check, one step. `reset_schema_cache!` still re-probes at once and starts again from 30 s. Refs #666.
51
+
52
+ - **pgmq-ruby 0.9.0 (was 0.8.2); the gemspec now requires `~> 0.9.0` (issue #656).** What reaches pgbus users is the connection pool: pgmq-ruby's checkout health check now resets a pooled connection in any state other than `CONNECTION_OK`, so a connection a `Thread#raise` left half-way through a reset (`CONNECTION_STARTED`) is recovered on the next checkout. Before, the checkout handed it out again, libpq refused every send (`no connection to the server`), and pgbus's stale-connection retry only recovered it on its last attempt by dropping both pools (#645): now the call succeeds on the first attempt, with no retry, no pool reload and no warning. pgmq-ruby also resets a connection whose call raised `PG::ConnectionBad` or `PG::UnableToSend` before it goes back to the pool (not a connection pgbus hands it through a callable, which it neither resets nor closes: the shared ActiveRecord connection, and dedicated connections in `connection_guc_mode = :session` with database.yml `variables`); native statements re-prepare on the new backend as on any reconnect. pgbus keeps its reload before the last stale retry for a connection that still fails after the health check passed it. A checkout reset of a connection in `connection_guc_mode = :session` with `variables` applies its `SET`s again (#671). `produce_topic` and `produce_batch_topic` now raise `PGMQ::Errors::MessageInvalidError` for a message that is not a String or headers that are neither a String nor nil, as `produce` has since 0.8.0. `Client#publish_to_topic` (what `Pgbus.publish` and the outbox poller call; `Pgbus::Testing`'s fake and inline modes never reach PGMQ) serializes the payload and headers to JSON Strings first, so it is unaffected; `headers: false`, which pgmq-ruby dropped before with an integer delay (and stored as JSON `false` with a fractional one) and now rejects, is passed as nil, so it is dropped at any delay. `spec/integration/connection_failure_spec.rb` now asserts the mid-reset connection is recovered at checkout (it failed on 0.8.2), in place of the two examples that pinned the old last-retry recovery. Refs #656.
37
53
 
38
54
  ## [0.25.1] - 2026-11-15
39
55
 
@@ -6,55 +6,28 @@ module Pgbus
6
6
  module ActiveJob
7
7
  class Adapter
8
8
  def enqueue(active_job)
9
- queue = active_job.queue_name || Pgbus.configuration.default_queue
10
- payload_hash = Serializer.serialize_job_hash(active_job)
11
- payload_hash = Concurrency.inject_metadata(active_job, payload_hash)
12
- payload_hash = Uniqueness.inject_metadata(active_job, payload_hash)
13
- payload_hash = FairShare.inject_metadata(active_job, payload_hash)
14
- payload_hash = inject_batch_metadata(payload_hash, active_job)
15
- note_deploy_task_run_enqueued(active_job, false)
16
-
17
- if uniqueness_rejected?(active_job, payload_hash, queue: queue)
18
- uncount_batch_job(payload_hash, active_job)
19
- return active_job
20
- end
21
-
22
- enqueue_with_concurrency(active_job, queue, payload_hash)
23
- ensure
24
- forget_enqueue_batch_id(active_job)
9
+ enqueue_job(active_job)
25
10
  end
26
11
 
27
12
  def enqueue_at(active_job, timestamp)
28
- queue = active_job.queue_name || Pgbus.configuration.default_queue
29
- payload_hash = Serializer.serialize_job_hash(active_job)
30
- payload_hash = Concurrency.inject_metadata(active_job, payload_hash)
31
- payload_hash = Uniqueness.inject_metadata(active_job, payload_hash)
32
- payload_hash = FairShare.inject_metadata(active_job, payload_hash)
33
- payload_hash = inject_batch_metadata(payload_hash, active_job)
34
- delay = [(timestamp - Time.current.to_f).ceil, 0].max
35
- note_deploy_task_run_enqueued(active_job, false)
36
-
37
- if uniqueness_rejected?(active_job, payload_hash, queue: queue)
38
- uncount_batch_job(payload_hash, active_job)
39
- return active_job
40
- end
41
-
42
- enqueue_with_concurrency(active_job, queue, payload_hash, delay: delay)
43
- ensure
44
- forget_enqueue_batch_id(active_job)
13
+ enqueue_job(active_job, delay: delay_until(timestamp))
45
14
  end
46
15
 
16
+ # Sets successfully_enqueued on every job, as ActiveJob.perform_all_later
17
+ # reads it (issue #679): true for a job sent or parked behind its
18
+ # concurrency limit, false for one dropped or never reached because an
19
+ # error raised. Returns the number marked, so the enqueue_all.active_job
20
+ # notification's enqueued_count agrees with the jobs.
47
21
  def enqueue_all(active_jobs)
22
+ active_jobs.each do |j|
23
+ j.successfully_enqueued = false
24
+ j.enqueue_error = nil
25
+ end
26
+
48
27
  # Jobs with uniqueness or concurrency must go through individual enqueue
49
28
  # to acquire locks/semaphores — the bulk path cannot (issue #413)
50
29
  individual, bulk = active_jobs.partition { |j| Uniqueness.uniqueness_config(j) || concurrency_config(j) }
51
- individual.each do |j|
52
- if scheduled_in_future?(j)
53
- enqueue_at(j, j.scheduled_at.to_f)
54
- else
55
- enqueue(j)
56
- end
57
- end
30
+ individual.each { |j| enqueue_one_of_all(j) }
58
31
 
59
32
  # Group by priority too: send_batch routes through the queue strategy,
60
33
  # so a mixed-priority bulk send needs one produce_batch per level.
@@ -62,15 +35,50 @@ module Pgbus
62
35
  .each do |(queue, priority), jobs|
63
36
  immediate, scheduled = jobs.partition { |j| !scheduled_in_future?(j) }
64
37
  enqueue_immediate(queue, immediate, priority: priority)
65
- scheduled.each { |j| enqueue_at(j, j.scheduled_at.to_f) }
38
+ scheduled.each { |j| enqueue_one_of_all(j) }
66
39
  end
67
40
 
68
- active_jobs.count
41
+ active_jobs.count(&:successfully_enqueued?)
69
42
  end
70
43
 
71
44
  private
72
45
 
73
- def enqueue_with_concurrency(active_job, queue, payload_hash, delay: 0)
46
+ # One job of an enqueue_all that cannot take the bulk path.
47
+ # enqueue_with_concurrency marks it once it is live, as enqueue_immediate
48
+ # marks the bulk; on perform_later the adapter never sets
49
+ # successfully_enqueued, ActiveJob sets it around enqueue/enqueue_at.
50
+ def enqueue_one_of_all(active_job)
51
+ delay = scheduled_in_future?(active_job) ? delay_until(active_job.scheduled_at.to_f) : 0
52
+ enqueue_job(active_job, delay: delay, mark_enqueued: true)
53
+ end
54
+
55
+ def delay_until(timestamp)
56
+ [(timestamp - Time.current.to_f).ceil, 0].max
57
+ end
58
+
59
+ # Returns the job. With mark_enqueued (enqueue_all), sets
60
+ # successfully_enqueued once the job is live: sent, or parked behind its
61
+ # concurrency limit to run later.
62
+ def enqueue_job(active_job, delay: 0, mark_enqueued: false)
63
+ queue = active_job.queue_name || Pgbus.configuration.default_queue
64
+ payload_hash = Serializer.serialize_job_hash(active_job)
65
+ payload_hash = Concurrency.inject_metadata(active_job, payload_hash)
66
+ payload_hash = Uniqueness.inject_metadata(active_job, payload_hash)
67
+ payload_hash = FairShare.inject_metadata(active_job, payload_hash)
68
+ payload_hash = inject_batch_metadata(payload_hash, active_job)
69
+ note_deploy_task_run_enqueued(active_job, false)
70
+
71
+ if uniqueness_rejected?(active_job, payload_hash, queue: queue)
72
+ uncount_batch_job(payload_hash, active_job)
73
+ return active_job
74
+ end
75
+
76
+ enqueue_with_concurrency(active_job, queue, payload_hash, delay: delay, mark_enqueued: mark_enqueued)
77
+ ensure
78
+ forget_enqueue_batch_id(active_job)
79
+ end
80
+
81
+ def enqueue_with_concurrency(active_job, queue, payload_hash, delay: 0, mark_enqueued: false)
74
82
  key = Concurrency.extract_key(payload_hash)
75
83
  concurrency = concurrency_config(active_job)
76
84
  priority = active_job.try(:priority)
@@ -117,6 +125,10 @@ module Pgbus
117
125
  active_job.provider_job_id = msg_id
118
126
  end
119
127
 
128
+ # Marked before any bookkeeping below can raise: the job is live
129
+ # whatever happens next (issue #679).
130
+ active_job.successfully_enqueued = true if mark_enqueued && (msg_id || blocked)
131
+
120
132
  # Bind before backfill so a live message is never left with an unbound
121
133
  # uniqueness row if execution-row bookkeeping raises.
122
134
  bind_acquired_uniqueness_lock(queue, msg_id) if msg_id
@@ -468,7 +480,10 @@ module Pgbus
468
480
  raise Pgbus::EnqueueError, "Pgbus batch enqueue failed: expected #{jobs.size} ids, got #{msg_ids&.size || 0}"
469
481
  end
470
482
 
471
- jobs.zip(msg_ids).each { |job, id| job.provider_job_id = id }
483
+ jobs.zip(msg_ids).each do |job, id|
484
+ job.provider_job_id = id
485
+ job.successfully_enqueued = true
486
+ end
472
487
  payloads.zip(msg_ids).each { |payload, id| Batch.backfill_execution(payload, id, physical) }
473
488
  rescue Pgbus::EnqueueError
474
489
  Array(payloads).each_with_index do |payload, index|
data/lib/pgbus/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Pgbus
4
- VERSION = "0.27.1"
4
+ VERSION = "0.27.2"
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pgbus
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.27.1
4
+ version: 0.27.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mikael Henriksson