pgbus 0.21.4 → 0.22.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 34cf3117ba1bd7fdc9355f8593496af02af9e7e878efc5247856ba2afd7533e8
-  data.tar.gz: b301f2fcf44e83d9438ece6df8f7e94bb627ea91b83070e8e56ce6e3a10f66ae
+  metadata.gz: e8f6e5116d9a4abb573d35bb3b24275f52791adb51d0ae1ad40ee09995779cb8
+  data.tar.gz: d30361a159009532f0d2cb155572b35bbf3ebcc677be32feb01315006199fff8
 SHA512:
-  metadata.gz: 95ccb84dcc956d3d4e483976ee7e62f8fb4525622761e9f767c3cf9237b9de7069c36ab5aba05820a0acbe3da564ca105add4512b010e085dcb11a51ffb97fd8
-  data.tar.gz: cf6fe7554b273ad00988f449a18410ba456520f808337a316671302fcef507e16037e8a64764bb08130521337be065fdc6b9939a7ea788afa872db0423f9775a
+  metadata.gz: df02b7572bd24042bc70b39b0b59373eb84a42b051fc5a4e4e97b3da3df8d2f22bd0fddbbaf503f5d2d2ed581c84a8ce61054e76ff8c4194e888c9e705a6b78e
+  data.tar.gz: e5fc1246616d9633cd447334bd70a495bb18e60ea4f6420e8d3c7f8d481e824cd28e32c0a790f984b1df2ceb3593ac958f4ef8fe0b6046cb046b4a220b54c215

data/CHANGELOG.md

@@ -1,5 +1,26 @@
 ## [Unreleased]
 
+### Added
+
+- **Docs: a "Throughput checklist" on the docs site's Performance & tuning page (issue #608).** One ordered list of every knob that moves jobs/s — `processes:`, `native_statements`, `completion_batching`, `completed_jobs = :delete`, `read_ahead` with `pool_size`, `listen_notify`, YJIT, ActiveJob's per-job log lines, `stats_enabled` and the Postgres side — each with its key and default, the number `docs/performance.md` measured for it (or "not measured") and what it costs, plus one line each on why Ractors, a C or Rust extension and Spinel / Roundhouse are not levers. It gives ActiveJob its own `warn` logger through `config.active_job.logger`, because `ActiveJob::Base.logger` is `Rails.logger` and lowering its level silences the app. The Running workers page's `processes:` and `read_ahead` sections link to it, and `docs/performance.md` links to it from the top and summarises its native-compilation verdict in one line. No new measurements. Refs #601.
+
+- **Opt-in `config.completed_jobs = :delete`: a successful job's message is deleted with `pgmq.delete` instead of archived (issue #607).** The archive is a `DELETE … RETURNING` from `q_<queue>` plus an `INSERT` into `a_<queue>`, its `archived_at` index upkeep and the hourly compaction that trims it; `:delete` is the DELETE alone, and with `completion_batching` the worker's completer flushes with `delete_batch` instead of `archive_batch`. Equally reliable: either way the message is gone once the statement commits, and the executor's contract is unchanged (a message already gone is the `:duplicate` path, a connection error is retried once, a `false` after its own retry is `:ambiguous` and still sends the completion signals). What it costs is the completed-job history in the archive; with job stats on (`stats_enabled`, the default, and the `pgbus_job_stats` table migrated), every success is still recorded there. The dashboard shows no job history from the archive, so no page changes. `Batch::Sweep` resolves a stale batch execution whose message is in neither queue nor DLQ as completed without probing an archive `:delete` never writes, and without the "no queue, archive, or DLQ message" warning; with no archive to consult it cannot tell a deleted batch job from one whose message vanished another way (a purged queue), and counts both as completed. ActiveJob only: EventBus handler queues keep archiving (event replay reads their archive); failed jobs still back off in their queue and dead letters still move to the DLQ. Default `:archive`; anything but `:archive` / `:delete` (Symbol or String) raises `ConfigurationError`. The fleet bench takes `WP_BENCH_COMPLETED_JOBS=archive|delete` (new). On one local Postgres (three interleaved rounds, medians): statements per job unchanged, the completion statement about half the archive's Postgres time unbatched and a third to a half batched; PG ms/job lower in five of six unbatched rows (0.058 → 0.022 at one YJIT process), jobs/s up 10–35 % in four of six unbatched rows with overlapping ranges, and batched delete at four processes +11–13 % jobs/s over batched archive (19 830 → 22 000 with YJIT). `docs/performance.md` → "Delete mode (issue #607)". Refs #601.
+
+- **Opt-in batched completion: `archive_batch` calls of up to `completion_batch_size` ids instead of one `pgmq.archive` per job (issue #606).** `config.completion_batching = true` gives each worker a `Process::Completer` (thread `pgbus-completer`) that archives the jobs finishing close together with one `archive_batch` per physical queue per flush (more when a queue has over `completion_batch_size` ids pending). There is no timer: an idle completer archives a lone job at once, and whatever finishes while a flush is in flight goes in the next; `config.completion_batch_size` (default 100) caps the ids per statement. The job thread waits for its own result, so `:duplicate` (archived elsewhere), the retry-once-on-connection-error and `:ambiguous` handling, and every post-archive signal stay as they were; a message held by two threads of one worker is reported archived to only one of them. Threads only: `validate!` and `Worker.new` raise `ConfigurationError` next to a global or capsule `execution_mode: :async`. `Worker#immediate_shutdown` stops the completer before it kills the pool, so a flush that answers in time resolves every waiter before any job thread is killed; a job finishing after the stop archives on its own thread. Shutdown waits for a flush at most `immediate_shutdown_grace` (QUIT) or what the pool wait left of its 5 s residual (TERM); a flush wedged on a hung Postgres past that is abandoned, and its jobs fail with `Completer::Stopped` and re-run after `visibility_timeout` (left completed but unsignalled if Postgres had committed it and only the reply stalled). The crash window widens from one job to one flush: on SIGKILL or OOM the jobs waiting in the current flush re-run after `visibility_timeout`, and a flush that committed just before the crash leaves its jobs unsignalled until the reapers and sweeps run. Default off. Measured with the fleet bench (`WP_BENCH_COMPLETION_BATCHING=1`, new): statements per job 1.21–1.26 → 0.37–0.45 in every row (~5 ids per `archive_batch`); at four local processes Postgres ms per job 0.26–0.34 → 0.06–0.07, peak connections ~60 → ~19, jobs/s +17–35 % (two processes gained nothing); one local process +2 % to −33 % and `read_ahead` 12 across +1 ms latency −10 to −30 %, so leave it off there. `docs/performance.md` → "Batched completion (issue #606)". Refs #601.
+
+- **Fleet bench cell: jobs/s at N worker processes on one queue and one Postgres, with the Postgres cost per job (issue #602).** `WP_BENCH_ROLE=fleet bundle exec rake bench:worker_profile` (or `rake bench:fleet`) drains `WP_BENCH_JOBS × N` jobs with `WP_BENCH_PROCESSES` (default `1,2,4`) worker subprocesses that warm up on their own queue and start together from a barrier, and reports jobs/s, jobs/s per process, PG ms/job, statements/job and average busy backends from `pg_stat_statements` (scoped to the bench database; `n/a` when the extension is not in `shared_preload_libraries`), plus the top statements and a sampled split of what the database's active backends were waiting on. Each child's pool is `threads + 2`; each child caps its ActiveRecord pool at 2, and a row whose `N × (pool_size + 2) + 10` connections do not fit the free connections is skipped with the reason, so `8` is opt-in on a stock `max_connections = 100`, and every row prints the peak connections it held. `WP_BENCH_ROLE=all` runs the worker, consumer and fleet cells. `docs/performance.md` → "Fleet scaling (issue #602)" records this machine's baseline and the Ractor / C-extension verdict. Refs #601.
+
+### Changed
+
+- **`Client#read_batch`, `#archive_message`, `#archive_batch`, `#delete_message`, `#delete_batch`, `#send_message` and `#send_batch` run pgbus's own SQL in named prepared statements, not pgmq's plpgsql functions (issue #605). On by default; `config.native_statements = false` restores the function calls.** pgmq's `read`, `archive`, `delete`, `send` and `send_batch` build their SQL with `FORMAT` and run it with `EXECUTE`, which Postgres parses and plans on every call. pgbus now runs the same statements, prepared once per pooled connection: return values, `PGMQ::Message`, `read_ct`, `vt`, `last_read_at`, headers and the queue's NOTIFY insert trigger are unchanged. Reads of up to 64 messages carry their LIMIT in the statement, because with `LIMIT $1` Postgres never settles on a cached plan. A queue whose `q_`/`a_` table lacks a column the statements use (a pgmq extension older than `headers`) keeps the functions, with one info log per queue per client. Inside a transaction the server has begun (an enqueue inside the app's transaction on the shared-connection path, once Rails has sent its `BEGIN`) the SQL is sent unprepared, so a missing statement cannot abort the transaction. Connections built from a database.yml that sets `prepared_statements: false` send it unprepared from the start. Behind a transaction-mode PgBouncer, prepared statements need PgBouncer 1.21+ with `max_prepared_statements`; a process that sees its statements disappear from their connection sends the same SQL unprepared from then on and logs a warning once. Connection errors still surface as `PGMQ::Errors::ConnectionError` with pgmq-ruby's one reconnect retry. Grouped, fair and multi-queue reads, priority-routed reads (`priority_levels` set), `set_vt`, streams and the dead-letter move keep the functions. In the fleet bench the median Postgres time per job fell about 70 % at one worker process and 34–42 % at four; median jobs/s rose 11 % (one process) and 20 % (four) with YJIT and did not move without a JIT, on a shared machine (`docs/performance.md` → "Native statements"). The fleet cell's PG ms/job now adds `pg_stat_statements`' `total_plan_time` (0 unless `track_planning` is on), where top-level statements record their planning. Refs #605.
+- **The executor reads a message's `enqueued_at` with `Time.new`, not `Time.parse` (issue #604).** pgmq-ruby hands `enqueued_at` over as PostgreSQL's TIMESTAMPTZ text (`2026-10-29 10:11:12.123456+00`); with stats on (the default) every job parsed it for `enqueue_latency_ms` through `Time.parse`, ~7 µs, the largest single pgbus frame in the worker profile (12–14 % of on-CPU time). Core Ruby's `Time.new(String)` reads it in C at ~0.3 µs; `pg`'s `TimestampWithTimeZone` decoder, the first candidate, took ~3 µs and still showed at ~5 % in the profile. Every offset shape PostgreSQL sends (`+00`, `-03`, `+05:30`) reads as before, so do ISO 8601 strings with a `T` or `Z`; a string without an offset is still UTC, and text `Time.new` cannot read still records the stat without latency (that now includes shapes no sender emits but `Time.parse` accepted: a trailing space, a `UTC` word, a lowercase `z`). The latency is measured against `Process.clock_gettime(CLOCK_REALTIME)`, not `Time.now`, so a test that freezes time (`travel_to`, Timecop) no longer changes `enqueue_latency_ms`. `VisibilityHeartbeat` builds each in-flight message's key once instead of on register, unregister and every beat check. The executor allocation budget spec now runs with a PostgreSQL-shaped `enqueued_at`: 9 pgbus-owned objects per job against 10 before, under the same `< 10` gate. The executor micro-bench with stats on went ~40k → ~60k jobs/s; the real-worker numbers are in `docs/performance.md`. Refs #604.
+
+### Fixed
+
+- **An archive or delete whose reply was lost is no longer mistaken for a duplicate (issue #607).** pgmq-ruby's `with_connection` re-runs its block after a lost connection ("server closed the connection"). That usually means the socket died idle in the pool, but it can also follow a commit whose reply was lost. Re-run, the native `archive`, `archive_batch`, `delete` or `delete_batch` found the ids gone, so the executor took its own completed job for one another worker had finished and skipped its signals (uniqueness lock, batch counter, concurrency slot), with or without `completion_batching`. A re-run of these four that comes back short now raises `PGMQ::Errors::ConnectionError` ("…the re-run found ids gone: it may have committed: <the original error>"), and the executor's retry-once claims the message as `:ambiguous`. A re-run that finds every id still returns normally, because then the first run never committed. The client's stale-connection retry never re-runs that error, even when the original text matches one of its patterns. The EventBus consumer counts such an archive as handled, because its handlers ran and the message is gone. Reads and sends are unchanged. The function path (`native_statements = false`, or a queue without the columns the statements need) still takes pgmq-ruby's plain retry. Refs #607.
+
+- **A job in no batch no longer pays an extra database round trip at enqueue (issue #603).** After every `perform_later`, and once per job of every `perform_all_later` group, the adapter ran `UPDATE pgbus_batch_executions SET msg_id = …, queue_name = … WHERE job_id = $1` on the ActiveRecord connection, a no-op for a job outside any batch. `Batch.backfill_execution` now returns early for a payload without `pgbus_batch_id`; batch members, including a `retry_on` re-enqueue and a concurrency-parked job promoted later, still get their execution row pointed at the new message. `rake bench:integration` "adapter: plain job" went from about 4 570 to 7 400–8 000 enqueues/s on one local machine (219 → 125–135 µs). The bench now creates `pgbus_batch_executions` so it measures the path installs take. Refs #603.
+
 ## [0.21.4] - 2026-10-28
 
 ### Added

data/README.md

@@ -206,7 +206,7 @@ class OrderCreatedHandler < Pgbus::EventBus::Handler
   idempotent!  # Deduplicate by (event_id, handler_class)
 
   def handle(event)
-    order_id = event.payload["order_id"]
+    order_id = event["order_id"]
     Analytics.track_order(order_id)
     InventoryService.reserve(order_id)
   end

data/Rakefile

@@ -61,11 +61,18 @@ namespace :bench do
     ruby "benchmarks/integration_bench.rb"
   end
 
-  desc "Run worker profiling bench: jobs/s and where a job's time goes (requires PGBUS_DATABASE_URL)"
+  desc "Run worker profiling bench: jobs/s and where a job's time goes; WP_BENCH_ROLE=worker|consumer|both|fleet|all " \
+       "(requires PGBUS_DATABASE_URL)"
   task :worker_profile do
     ruby "benchmarks/worker_profile_bench.rb"
   end
 
+  desc "Run the fleet cell: jobs/s and Postgres ms/job at WP_BENCH_PROCESSES (default 1,2,4) worker processes " \
+       "on one queue (requires PGBUS_DATABASE_URL)"
+  task :fleet do
+    sh({ "WP_BENCH_ROLE" => "fleet" }, FileUtils::RUBY, "benchmarks/worker_profile_bench.rb")
+  end
+
   desc "Run the workflow insights query budget: ~1M step rows (requires PGBUS_DATABASE_URL)"
   task :workflow_insights do
     ruby "benchmarks/workflow_insights_bench.rb"

data/lib/pgbus/active_job/executor.rb

@@ -1,16 +1,21 @@
 # frozen_string_literal: true
 
-require "time"
-
 module Pgbus
   module ActiveJob
     class Executor
       attr_reader :client, :config
 
-      def initialize(client: Pgbus.client, config: Pgbus.configuration, stat_buffer: nil)
+      # completer: the worker's Process::Completer when config.completion_batching
+      # is on (issue #606): complete_from goes through it instead of one
+      # archive_message per job. config.completed_jobs (issue #607) is read
+      # once here: :delete finishes a successful job with delete_message.
+      def initialize(client: Pgbus.client, config: Pgbus.configuration, stat_buffer: nil, completer: nil)
         @client = client
         @config = config
         @stat_buffer = stat_buffer
+        @completer = completer
+        @delete_completed = config.completed_jobs == :delete
+        @completed_word = @delete_completed ? "deleted" : "archived"
       end
 
       # Exceptions we never want to swallow — let the process die/signal propagate.
@@ -117,14 +122,15 @@ module Pgbus
           # heartbeat lapsed and it was redelivered): that worker owns the
           # completion signals, and signalling again here would release the
           # concurrency slot a second time (rails/solid_queue#761).
-          if archive_from(queue_name, msg_id, source_queue: source_queue) == :already_archived
+          if complete_from(queue_name, msg_id, source_queue: source_queue) == :already_archived
             Pgbus.logger.warn do
-              "[Pgbus::Executor] already archived elsewhere, skipping signals #{log_tag(message, queue_name)} job_class=#{job_class}"
+              "[Pgbus::Executor] already #{@completed_word} elsewhere, skipping signals " \
+                "#{log_tag(message, queue_name)} job_class=#{job_class}"
             end
             release_duplicate_execution_lock(uniqueness_key, uniqueness_strategy, queue_name, msg_id)
             return :duplicate
           end
-          Pgbus.logger.debug { "[Pgbus::Executor] archived #{log_tag(message, queue_name)} job_class=#{job_class}" }
+          Pgbus.logger.debug { "[Pgbus::Executor] #{@completed_word} #{log_tag(message, queue_name)} job_class=#{job_class}" }
           job_succeeded = true
           release_uniqueness_lock(uniqueness_key, queue_name, message)
           # Only FailedEventRecorder.record! writes this table, and only after a
@@ -147,7 +153,7 @@ module Pgbus
       rescue Exception => e # rubocop:disable Lint/RescueException
         # Widened from StandardError to catch Async::Stop / Async::Cancel
         # (both inherit from Exception, not StandardError) under execution_mode: :async.
-        # Before this, a fiber interruption between perform_now and archive_from
+        # Before this, a fiber interruption between perform_now and complete_from
         # silently lost control flow — no failed event row, no job_failed
         # notification, uniqueness lock held until VT expired. See issue #126.
         handle_failure(message, queue_name, e, payload: payload, source_queue: source_queue)
@@ -196,7 +202,8 @@ module Pgbus
       # The table the message was read from: the _pN sub-queue under priority
       # routing, else the logical queue's own. Each table numbers its messages
       # from its own sequence, so a failed-event row is keyed on it (issue
-      # #583). Built only on the failure and redelivery paths.
+      # #583). Built only on the failure and redelivery paths, and for the
+      # completer (issue #606), which groups by physical queue.
       def physical_queue(queue_name, source_queue)
         source_queue || config.queue_name(queue_name)
       end
@@ -282,33 +289,43 @@ module Pgbus
         Pgbus.logger.debug { "[Pgbus] Stat recording failed: #{e.message}" }
       end
 
+      # pgmq-ruby sets no result type map, so enqueued_at is the TIMESTAMPTZ
+      # text PostgreSQL sends ("2026-10-29 10:11:12.123456+00", any offset).
+      # Time.new reads it in C at ~0.3 µs; Time.parse took ~7 µs a job and
+      # pg's TimestampWithTimeZone decoder ~3 µs (issue #604). Unreadable text
+      # raises ArgumentError: the stat is recorded without latency.
       def compute_enqueue_latency(message)
         return unless message
 
         enqueued_at = message.enqueued_at
         return unless enqueued_at
 
-        # Fast path: numeric epoch (float seconds) avoids Time.parse entirely.
-        # PGMQ returns enqueued_at as a Time or string depending on the driver.
-        case enqueued_at
-        when Numeric
-          [((Time.now.to_f - enqueued_at) * 1000).round, 0].max
-        when Time
-          [((Time.now.utc - enqueued_at.utc) * 1000).round, 0].max
-        else
-          parse_enqueue_latency_from_string(enqueued_at.to_s)
-        end
+        epoch = case enqueued_at
+                when Numeric then enqueued_at
+                when Time then enqueued_at.to_f
+                else enqueued_at_epoch(enqueued_at.to_s)
+                end
+        return unless epoch
+
+        [((::Process.clock_gettime(::Process::CLOCK_REALTIME) - epoch) * 1000).round, 0].max
       rescue ArgumentError, TypeError
         nil
       end
 
-      def parse_enqueue_latency_from_string(str)
-        # PGMQ enqueued_at is TIMESTAMPTZ (always UTC internally).
-        # If the string lacks an explicit offset, assume UTC to avoid
-        # misinterpretation when the system timezone is non-UTC.
-        str = "#{str} UTC" unless str.match?(/[+-]\d{2}:?\d{2}\s*$|Z\s*$/i)
-        enqueued_at = Time.parse(str)
-        [((Time.now.utc - enqueued_at) * 1000).round, 0].max
+      # A trailing "+00", "-03", "+05:30", "+0530" or "Z".
+      UTC_OFFSET_SUFFIX = /(?:[+-]\d\d(?::?\d\d)?|Z)\z/
+      private_constant :UTC_OFFSET_SUFFIX
+
+      # Time.new reads a string without an offset in the host's zone, but
+      # TIMESTAMPTZ is UTC inside, so such a string gets a "Z". Checked here
+      # rather than with `in: "UTC"`: that keyword costs a Hash per job.
+      # Time.new also takes a bare year ("2026") as a time; a string without a
+      # time of day is not a timestamp, so it has no latency.
+      def enqueued_at_epoch(str)
+        return unless str.include?(":")
+
+        str = "#{str}Z" unless str.match?(UTC_OFFSET_SUFFIX)
+        Time.new(str).to_f
       end
 
       def handle_failure(message, queue_name, error, payload: nil, source_queue: nil)
@@ -412,7 +429,11 @@ module Pgbus
       # that succeeded but failed to archive redelivers after VT expiry and
       # runs twice. If the retry also fails, fall through to the normal
       # failure path (recorded failure + VT-based redelivery).
-      # Returns :archived, :already_archived, or :ambiguous.
+      # Returns :archived, :already_archived, or :ambiguous. Under
+      # config.completed_jobs = :delete it deletes the message instead
+      # (delete_message, or the completer's delete_batch): pgmq.delete is as
+      # idempotent and returns false the same way, so the symbols keep their
+      # names and "archived" means "completed".
       #
       # A `false` reported after our OWN retry is ambiguous, not a duplicate:
       # the first archive may well have committed with only its reply lost to
@@ -421,10 +442,18 @@ module Pgbus
       # batch and uniqueness state on every such blip, whereas a genuine
       # duplicate would additionally require another worker to have taken and
       # finished the same message inside the retry window.
-      def archive_from(queue_name, msg_id, source_queue: nil)
+      def complete_from(queue_name, msg_id, source_queue: nil)
         attempts = 0
         begin
-          archived = if source_queue
+          archived = if @completer
+                       @completer.complete(physical_queue(queue_name, source_queue), msg_id) == :archived
+                     # Spelled out, no `**` forward: on Ruby 3.3 that
+                     # allocates per job even without keywords (#484 budget).
+                     elsif @delete_completed && source_queue
+                       client.delete_message(source_queue, msg_id, prefixed: false)
+                     elsif @delete_completed
+                       client.delete_message(queue_name, msg_id)
+                     elsif source_queue
                        client.archive_message(source_queue, msg_id, prefixed: false)
                      else
                        client.archive_message(queue_name, msg_id)
@@ -437,7 +466,7 @@ module Pgbus
           raise unless attempts == 1 && connection_error?(e)
 
           Pgbus.logger.warn do
-            "[Pgbus::Executor] Archive failed on connection error, retrying once: #{e.message}"
+            "[Pgbus::Executor] #{@delete_completed ? "Delete" : "Archive"} failed on connection error, retrying once: #{e.message}"
           end
           retry
         end

data/lib/pgbus/batch/sweep.rb

@@ -47,6 +47,8 @@ module Pgbus
 
           dlq = client.dead_letter_physical_name(row.queue_name, prefixed: false)
           return :failed if client.message_with_job_id?(dlq, job_id: row.job_id, prefixed: false) == true
+          # completed_jobs :delete (issue #607) leaves no archive row to find.
+          return :completed if Pgbus.configuration.completed_jobs == :delete
 
           return :completed if client.message_archived?(row.queue_name, msg_id: row.msg_id, prefixed: false) == true
 

data/lib/pgbus/batch.rb

@@ -425,8 +425,11 @@ module Pgbus
     end
 
     def self.backfill_execution(payload, msg_id, queue_name)
-      return unless executions_migrated?
       return unless payload && msg_id
+      # Only batch members have an execution row: without this, every plain
+      # enqueue paid a no-op UPDATE round trip after the send (issue #603).
+      return unless payload[METADATA_KEY]
+      return unless executions_migrated?
 
       job_id = payload["job_id"]
       return unless job_id