pgbus 0.21.3 → 0.22.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: afa142ad44c901764a77528d7e387f708321353b5323c6dafcf23577cd27556e
-  data.tar.gz: 3bdcb708e9547c70115ccc9babd430d038a8b0ffb483faf615842a5a96cbbdff
+  metadata.gz: e8f6e5116d9a4abb573d35bb3b24275f52791adb51d0ae1ad40ee09995779cb8
+  data.tar.gz: d30361a159009532f0d2cb155572b35bbf3ebcc677be32feb01315006199fff8
 SHA512:
-  metadata.gz: 3204bb1c094c7765ca32942b76857774e6696070b3b1787057b06da5f185d55de3069397bb1deb1dbf7221daf22eb064dddcb7a7906471e40fddafd5cf14cb38
-  data.tar.gz: 28c7d2ff21d9c8f97a2db27694c1e21184c9549d5af9cbebbef7244e3690d14b13aa546f79d006eb14104189ca3246d1416a6bb1679d59cfe80353d59511e6e5
+  metadata.gz: df02b7572bd24042bc70b39b0b59373eb84a42b051fc5a4e4e97b3da3df8d2f22bd0fddbbaf503f5d2d2ed581c84a8ce61054e76ff8c4194e888c9e705a6b78e
+  data.tar.gz: e5fc1246616d9633cd447334bd70a495bb18e60ea4f6420e8d3c7f8d481e824cd28e32c0a790f984b1df2ceb3593ac958f4ef8fe0b6046cb046b4a220b54c215

data/CHANGELOG.md

@@ -2,10 +2,56 @@
 
 ### Added
 
+- **Docs: a "Throughput checklist" on the docs site's Performance & tuning page (issue #608).** One ordered list of every knob that moves jobs/s — `processes:`, `native_statements`, `completion_batching`, `completed_jobs = :delete`, `read_ahead` with `pool_size`, `listen_notify`, YJIT, ActiveJob's per-job log lines, `stats_enabled` and the Postgres side — each with its key and default, the number `docs/performance.md` measured for it (or "not measured") and what it costs, plus one line each on why Ractors, a C or Rust extension and Spinel / Roundhouse are not levers. It gives ActiveJob its own `warn` logger through `config.active_job.logger`, because `ActiveJob::Base.logger` is `Rails.logger` and lowering its level silences the app. The Running workers page's `processes:` and `read_ahead` sections link to it, and `docs/performance.md` links to it from the top and summarises its native-compilation verdict in one line. No new measurements. Refs #601.
+
+- **Opt-in `config.completed_jobs = :delete`: a successful job's message is deleted with `pgmq.delete` instead of archived (issue #607).** The archive is a `DELETE … RETURNING` from `q_<queue>` plus an `INSERT` into `a_<queue>`, its `archived_at` index upkeep and the hourly compaction that trims it; `:delete` is the DELETE alone, and with `completion_batching` the worker's completer flushes with `delete_batch` instead of `archive_batch`. Equally reliable: either way the message is gone once the statement commits, and the executor's contract is unchanged (a message already gone is the `:duplicate` path, a connection error is retried once, a `false` after its own retry is `:ambiguous` and still sends the completion signals). What it costs is the completed-job history in the archive; with job stats on (`stats_enabled`, the default, and the `pgbus_job_stats` table migrated), every success is still recorded there. The dashboard shows no job history from the archive, so no page changes. `Batch::Sweep` resolves a stale batch execution whose message is in neither queue nor DLQ as completed without probing an archive `:delete` never writes, and without the "no queue, archive, or DLQ message" warning; with no archive to consult it cannot tell a deleted batch job from one whose message vanished another way (a purged queue), and counts both as completed. ActiveJob only: EventBus handler queues keep archiving (event replay reads their archive); failed jobs still back off in their queue and dead letters still move to the DLQ. Default `:archive`; anything but `:archive` / `:delete` (Symbol or String) raises `ConfigurationError`. The fleet bench takes `WP_BENCH_COMPLETED_JOBS=archive|delete` (new). On one local Postgres (three interleaved rounds, medians): statements per job unchanged, the completion statement about half the archive's Postgres time unbatched and a third to a half batched; PG ms/job lower in five of six unbatched rows (0.058 → 0.022 at one YJIT process), jobs/s up 10–35 % in four of six unbatched rows with overlapping ranges, and batched delete at four processes +11–13 % jobs/s over batched archive (19 830 → 22 000 with YJIT). `docs/performance.md` → "Delete mode (issue #607)". Refs #601.
+
+- **Opt-in batched completion: `archive_batch` calls of up to `completion_batch_size` ids instead of one `pgmq.archive` per job (issue #606).** `config.completion_batching = true` gives each worker a `Process::Completer` (thread `pgbus-completer`) that archives the jobs finishing close together with one `archive_batch` per physical queue per flush (more when a queue has over `completion_batch_size` ids pending). There is no timer: an idle completer archives a lone job at once, and whatever finishes while a flush is in flight goes in the next; `config.completion_batch_size` (default 100) caps the ids per statement. The job thread waits for its own result, so `:duplicate` (archived elsewhere), the retry-once-on-connection-error and `:ambiguous` handling, and every post-archive signal stay as they were; a message held by two threads of one worker is reported archived to only one of them. Threads only: `validate!` and `Worker.new` raise `ConfigurationError` next to a global or capsule `execution_mode: :async`. `Worker#immediate_shutdown` stops the completer before it kills the pool, so a flush that answers in time resolves every waiter before any job thread is killed; a job finishing after the stop archives on its own thread. Shutdown waits for a flush at most `immediate_shutdown_grace` (QUIT) or what the pool wait left of its 5 s residual (TERM); a flush wedged on a hung Postgres past that is abandoned, and its jobs fail with `Completer::Stopped` and re-run after `visibility_timeout` (left completed but unsignalled if Postgres had committed it and only the reply stalled). The crash window widens from one job to one flush: on SIGKILL or OOM the jobs waiting in the current flush re-run after `visibility_timeout`, and a flush that committed just before the crash leaves its jobs unsignalled until the reapers and sweeps run. Default off. Measured with the fleet bench (`WP_BENCH_COMPLETION_BATCHING=1`, new): statements per job 1.21–1.26 → 0.37–0.45 in every row (~5 ids per `archive_batch`); at four local processes Postgres ms per job 0.26–0.34 → 0.06–0.07, peak connections ~60 → ~19, jobs/s +17–35 % (two processes gained nothing); one local process +2 % to −33 % and `read_ahead` 12 across +1 ms latency −10 to −30 %, so leave it off there. `docs/performance.md` → "Batched completion (issue #606)". Refs #601.
+
+- **Fleet bench cell: jobs/s at N worker processes on one queue and one Postgres, with the Postgres cost per job (issue #602).** `WP_BENCH_ROLE=fleet bundle exec rake bench:worker_profile` (or `rake bench:fleet`) drains `WP_BENCH_JOBS × N` jobs with `WP_BENCH_PROCESSES` (default `1,2,4`) worker subprocesses that warm up on their own queue and start together from a barrier, and reports jobs/s, jobs/s per process, PG ms/job, statements/job and average busy backends from `pg_stat_statements` (scoped to the bench database; `n/a` when the extension is not in `shared_preload_libraries`), plus the top statements and a sampled split of what the database's active backends were waiting on. Each child's pool is `threads + 2`; each child caps its ActiveRecord pool at 2, and a row whose `N × (pool_size + 2) + 10` connections do not fit the free connections is skipped with the reason, so `8` is opt-in on a stock `max_connections = 100`, and every row prints the peak connections it held. `WP_BENCH_ROLE=all` runs the worker, consumer and fleet cells. `docs/performance.md` → "Fleet scaling (issue #602)" records this machine's baseline and the Ractor / C-extension verdict. Refs #601.
+
+### Changed
+
+- **`Client#read_batch`, `#archive_message`, `#archive_batch`, `#delete_message`, `#delete_batch`, `#send_message` and `#send_batch` run pgbus's own SQL in named prepared statements, not pgmq's plpgsql functions (issue #605). On by default; `config.native_statements = false` restores the function calls.** pgmq's `read`, `archive`, `delete`, `send` and `send_batch` build their SQL with `FORMAT` and run it with `EXECUTE`, which Postgres parses and plans on every call. pgbus now runs the same statements, prepared once per pooled connection: return values, `PGMQ::Message`, `read_ct`, `vt`, `last_read_at`, headers and the queue's NOTIFY insert trigger are unchanged. Reads of up to 64 messages carry their LIMIT in the statement, because with `LIMIT $1` Postgres never settles on a cached plan. A queue whose `q_`/`a_` table lacks a column the statements use (a pgmq extension older than `headers`) keeps the functions, with one info log per queue per client. Inside a transaction the server has begun (an enqueue inside the app's transaction on the shared-connection path, once Rails has sent its `BEGIN`) the SQL is sent unprepared, so a missing statement cannot abort the transaction. Connections built from a database.yml that sets `prepared_statements: false` send it unprepared from the start. Behind a transaction-mode PgBouncer, prepared statements need PgBouncer 1.21+ with `max_prepared_statements`; a process that sees its statements disappear from their connection sends the same SQL unprepared from then on and logs a warning once. Connection errors still surface as `PGMQ::Errors::ConnectionError` with pgmq-ruby's one reconnect retry. Grouped, fair and multi-queue reads, priority-routed reads (`priority_levels` set), `set_vt`, streams and the dead-letter move keep the functions. In the fleet bench the median Postgres time per job fell about 70 % at one worker process and 34–42 % at four; median jobs/s rose 11 % (one process) and 20 % (four) with YJIT and did not move without a JIT, on a shared machine (`docs/performance.md` → "Native statements"). The fleet cell's PG ms/job now adds `pg_stat_statements`' `total_plan_time` (0 unless `track_planning` is on), where top-level statements record their planning. Refs #605.
+- **The executor reads a message's `enqueued_at` with `Time.new`, not `Time.parse` (issue #604).** pgmq-ruby hands `enqueued_at` over as PostgreSQL's TIMESTAMPTZ text (`2026-10-29 10:11:12.123456+00`); with stats on (the default) every job parsed it for `enqueue_latency_ms` through `Time.parse`, ~7 µs, the largest single pgbus frame in the worker profile (12–14 % of on-CPU time). Core Ruby's `Time.new(String)` reads it in C at ~0.3 µs; `pg`'s `TimestampWithTimeZone` decoder, the first candidate, took ~3 µs and still showed at ~5 % in the profile. Every offset shape PostgreSQL sends (`+00`, `-03`, `+05:30`) reads as before, so do ISO 8601 strings with a `T` or `Z`; a string without an offset is still UTC, and text `Time.new` cannot read still records the stat without latency (that now includes shapes no sender emits but `Time.parse` accepted: a trailing space, a `UTC` word, a lowercase `z`). The latency is measured against `Process.clock_gettime(CLOCK_REALTIME)`, not `Time.now`, so a test that freezes time (`travel_to`, Timecop) no longer changes `enqueue_latency_ms`. `VisibilityHeartbeat` builds each in-flight message's key once instead of on register, unregister and every beat check. The executor allocation budget spec now runs with a PostgreSQL-shaped `enqueued_at`: 9 pgbus-owned objects per job against 10 before, under the same `< 10` gate. The executor micro-bench with stats on went ~40k → ~60k jobs/s; the real-worker numbers are in `docs/performance.md`. Refs #604.
+
+### Fixed
+
+- **An archive or delete whose reply was lost is no longer mistaken for a duplicate (issue #607).** pgmq-ruby's `with_connection` re-runs its block after a lost connection ("server closed the connection"). That usually means the socket died idle in the pool, but it can also follow a commit whose reply was lost. Re-run, the native `archive`, `archive_batch`, `delete` or `delete_batch` found the ids gone, so the executor took its own completed job for one another worker had finished and skipped its signals (uniqueness lock, batch counter, concurrency slot), with or without `completion_batching`. A re-run of these four that comes back short now raises `PGMQ::Errors::ConnectionError` ("…the re-run found ids gone: it may have committed: <the original error>"), and the executor's retry-once claims the message as `:ambiguous`. A re-run that finds every id still returns normally, because then the first run never committed. The client's stale-connection retry never re-runs that error, even when the original text matches one of its patterns. The EventBus consumer counts such an archive as handled, because its handlers ran and the message is gone. Reads and sends are unchanged. The function path (`native_statements = false`, or a queue without the columns the statements need) still takes pgmq-ruby's plain retry. Refs #607.
+
+- **A job in no batch no longer pays an extra database round trip at enqueue (issue #603).** After every `perform_later`, and once per job of every `perform_all_later` group, the adapter ran `UPDATE pgbus_batch_executions SET msg_id = …, queue_name = … WHERE job_id = $1` on the ActiveRecord connection, a no-op for a job outside any batch. `Batch.backfill_execution` now returns early for a payload without `pgbus_batch_id`; batch members, including a `retry_on` re-enqueue and a concurrency-parked job promoted later, still get their execution row pointed at the new message. `rake bench:integration` "adapter: plain job" went from about 4 570 to 7 400–8 000 enqueues/s on one local machine (219 → 125–135 µs). The bench now creates `pgbus_batch_executions` so it measures the path installs take. Refs #603.
+
+## [0.21.4] - 2026-10-28
+
+### Added
+
+- **`Pgbus::Event#dig` reads a nested payload value, the nested counterpart of `Event#[]` (issue #594).** `event.dig("meta", "type")` reads Symbol keys as Strings at every level (payloads are JSON-decoded) and passes Integer indexes through (`event.dig("data", "items", 0, "id")`); a missing key or a payload that is not a Hash returns `nil`, as `#[]` does, and an intermediate value without `#dig` raises `TypeError`, as `Hash#dig` does. Refs #594.
+
+### Changed
+
+- **PGMQ: vendored schema v1.13.1, a security release upstream asks everyone on 1.13.0 or earlier to take (issue #586).** Every PGMQ function pins `search_path = pg_catalog, pg_temp` and schema-qualifies its built-ins, so a role that can create objects on another role's search path can no longer shadow a built-in and run code with that role's privileges. `pg_monitor` loses `SELECT` on every table and sequence in the `pgmq` schema: it keeps `SELECT` on `pgmq.meta`, `SELECT (vt, enqueued_at)` on each queue table and `SELECT` on each queue's msg_id sequence, what `pgmq.metrics()`, `pgmq.metrics_all()` and `pgmq.list_queues()` need; a monitoring role that read message payloads through `pg_monitor` no longer can. Fresh embedded installs get 1.13.1. Existing *embedded* installs: `rake pgbus:pgmq:status`, then `rails generate pgbus:upgrade_pgmq` + `rails db:migrate` (`rails db:migrate:pgbus` on a separate-database install). Its new fixup (`lib/pgbus/pgmq_schema/fixups/pgmq_v1.13.1.sql`, upstream's migration blocks) revokes `pg_monitor`'s default privileges and whole-table grants in `pgmq`, grants the metrics columns and sequence on every existing queue, and rebinds a PGMQ column default that resolved to `now()` or `to_timestamp()` outside `pg_catalog`; a grant the migrating role cannot revoke raises a `WARNING` starting `pgmq:` instead of failing (Rails logs it only with `config.active_record.db_warnings_action` set; otherwise it goes to stderr), and a default it cannot alter stops the migration. Upstream's default rebinding selects extension member tables only, which would skip every embedded install, so the fixup selects pgmq's tables by name. `Pgbus::PgmqSchema.install_sql` now restores the caller's `search_path` after the vendored SQL, because 1.13.1 pins it with a transaction-local setting that would otherwise outlive the install inside a migration and send the next unqualified `CREATE TABLE` to `pg_catalog`. An `:extension` install gets 1.13.1 from `ALTER EXTENSION pgmq UPDATE`. Refs #586.
+
+### Fixed
+
+- **NOTIFY wakeups fire for a queue whose name has an uppercase letter (issue #586).** PGMQ's insert trigger rebuilt the queue name from its lowercased table name, which matched no `pgmq.notify_insert_throttle` row, so it never sent the NOTIFY and that queue's workers polled. PGMQ 1.13.1 passes the queue name to the trigger, and the upgrade migration's NOTIFY repair re-creates every queue's trigger that way; the channel, `pgmq.q_<queue name>.INSERT`, keeps the queue name's case, which is what pgbus LISTENs on. Refs #586.
+
+## [0.21.3] - 2026-10-22
+
+### Added
+
+- **`config.health_unclaimed_grace` (default 60 seconds) and the per-queue metric `oldest_unread_claimable_age_sec` (issue #589).** The grace is how long a visible, never-read (`read_ct = 0`) message must have been claimable before the health verdict counts it as "never claimed while draining processes are alive". It takes a number of seconds or an `ActiveSupport::Duration`; `0` restores the old rule (any such message counts), and `validate!` raises `ConfigurationError` on nil, a negative or a non-finite value. `Web::DataSource#queues_with_metrics` and `#queue_detail` report `oldest_unread_claimable_age_sec`, the seconds since the oldest visible `read_ct = 0` message became claimable (`min(vt)`, so a delayed job's age starts when its wait expires and a retried message is left out), nil when there is none. The verdict's `summary` carries it as the oldest over the drained queues, and the STALLED reason names it (`never claimed, oldest claimable for 74s`). `Pgbus::MCP::HealthAnalyzer.new` takes an optional `grace:`. Refs #589.
+
+### Fixed
+
+- **`/readyz` and `pgbus_health` no longer flap to `STALLED` on a healthy fleet (issue #589).** The verdict reported `STALLED` (and the mounted `/readyz` 503) whenever a drained queue held one visible, never-read message while a worker or consumer was alive, however briefly it had been claimable: every enqueue and every delayed job whose wait just expired is such a message until a worker's next poll, so a busy install answered 503 on a large share of checks. That branch now counts a queue only once its oldest such message has been claimable for `health_unclaimed_grace` seconds; a wedged worker still reads `STALLED` after the grace. A queue hash without the age (an older or stubbed data source) still counts at once. The worker-reported stalled branch (heart-beating, claim loop not advancing) is unchanged. Refs #589.
+
+## [0.21.2] - 2026-10-21
+
+### Added
+
 - **`config.boot_timeout` (default 120 seconds): the supervisor kills a child that never checks in (issue #587).** A fork checks in when it registers its `pgbus_processes` row on this host or writes its first liveness byte. One that has done neither `boot_timeout` seconds after the fork is logged at ERROR, sent `TTIN`, given 2 seconds, then SIGKILLed, and the normal restart path forks it again. A boot-deadline kill always counts as a crash, whatever the child's uptime, so a boot that hangs every time is re-forked with the crash-loop backoff (1 s doubling to 60 s) instead of every `boot_timeout`. Applies to workers, consumers, the dispatcher, the scheduler and the outbox poller. `nil` turns it off; anything else must be a positive number, or `validate!` raises `ConfigurationError`. When the watchdog's read of the process table fails, the boot deadline waits for a pass that can read it. Refs #587.
 - **A process whose registration in `pgbus_processes` failed at start retries it on each heartbeat (issue #587).** It used to stay without a row for good after one failed `create!` (a database blip at boot); a dispatcher, scheduler or outbox poller in that state would now be killed by the boot deadline even though it runs. Refs #587.
 - **`kill -TTIN <child pid>` logs every thread of a pgbus child with its backtrace at ERROR; the child keeps running (issue #587).** The watchdog sends it before every SIGKILL (boot timeout or stalled loop), so a hang is diagnosable from the log. Best effort: a child blocked in native code holding the GVL does not answer. Refs #587.
-- **`config.health_unclaimed_grace` (default 60 seconds) and the per-queue metric `oldest_unread_claimable_age_sec` (issue #589).** The grace is how long a visible, never-read (`read_ct = 0`) message must have been claimable before the health verdict counts it as "never claimed while draining processes are alive". It takes a number of seconds or an `ActiveSupport::Duration`; `0` restores the old rule (any such message counts), and `validate!` raises `ConfigurationError` on nil, a negative or a non-finite value. `Web::DataSource#queues_with_metrics` and `#queue_detail` report `oldest_unread_claimable_age_sec`, the seconds since the oldest visible `read_ct = 0` message became claimable (`min(vt)`, so a delayed job's age starts when its wait expires and a retried message is left out), nil when there is none. The verdict's `summary` carries it as the oldest over the drained queues, and the STALLED reason names it (`never claimed, oldest claimable for 74s`). `Pgbus::MCP::HealthAnalyzer.new` takes an optional `grace:`. Refs #589.
 - **The supervisor's standalone `/readyz` body carries `forked` (issue #587).** `forked` is the fork-table size (what `live` used to be); `live` now counts the children that have checked in. `Pgbus::Process::ReadinessSnapshot` gains `forked:`, defaulting to `live`. Refs #587.
 
 ### Changed
@@ -14,11 +60,14 @@
 
 ### Fixed
 
-- **`/readyz` and `pgbus_health` no longer flap to `STALLED` on a healthy fleet (issue #589).** The verdict reported `STALLED` (and the mounted `/readyz` 503) whenever a drained queue held one visible, never-read message while a worker or consumer was alive, however briefly it had been claimable: every enqueue and every delayed job whose wait just expired is such a message until a worker's next poll, so a busy install answered 503 on a large share of checks. That branch now counts a queue only once its oldest such message has been claimable for `health_unclaimed_grace` seconds; a wedged worker still reads `STALLED` after the grace. A queue hash without the age (an older or stubbed data source) still counts at once. The worker-reported stalled branch (heart-beating, claim loop not advancing) is unchanged. Refs #589.
 - **A child that hangs before its first liveness signal is killed and forked again, and `/readyz` no longer counts it as live (issue #587).** A fork stuck between `fork` and its first check-in (in an initializer, the eager load or a connection) never exited, so nothing reaped it, and the watchdog skipped it because no liveness channel had spoken; the container-local `/readyz` counted it as live because it was in the fork table, so a rolling deploy passed its health gate with a dead capsule. The boot deadline above now kills it, and `/readyz` stays at 503 `DEGRADED` (`live < expected`) until every child has checked in. A dispatcher, scheduler or outbox poller checks in through its process row, which the watchdog reads every 10 seconds, so `/readyz` turns 200 up to 10 seconds after the last one registers, and stays at 503 while the database is unreachable at boot (those children cannot be seen to be alive). The watchdog reads only this host's process rows last beaten since the fork, so a row another container wrote under the same pid, or one a SIGKILLed predecessor left behind (a container restarted in place reuses its hostname and pids), no longer vouches for a local child or hides a stalled loop. Refs #587.
 - **A failed job's retry backoff applies under priority routing (issue #582).** With `priority_levels > 1` the executor set the backoff's visibility timeout on the logical queue's base table, which priority routing never creates; the call raised, was logged at debug, and the job came back after the read's visibility timeout instead of its backoff delay. The backoff now goes to the `_pN` sub-queue the worker read the message from, as the visibility heartbeat and the archive already did. Refs #582.
 - **Two priority sub-queues' failures with the same msg_id keep a row each (issue #583).** Each `_pN` sub-queue numbers its messages from its own sequence, and `pgbus_failed_events` was unique on the logical queue and the msg_id, so a second job failing with the same msg_id in another sub-queue overwrote the first one's error and kept its payload: the dashboard showed one failed job where two had failed, Retry and Discard acted on the first job's message and deleted the only row, and the first job's success cleared the second one's failure. Rows now record the physical table (`source_queue`) and are unique on `(queue_name, msg_id, source_queue)`. The executor, the event consumer and the worker's zombie check read, write and clear only the row of the table the message came from. On the Jobs page a failed row joins only its own table's message and shows as an orphan once that message is gone, Retry, Discard and Discard all look the message up in the table the row names, and Discard releases the job's uniqueness lock bounded by when that table's message was sent, not by a sibling sub-queue's. A row written before the migration has no `source_queue` and is treated as before: it can belong to any table of its queue. Refs #583.
 
+## [0.21.1] - 2026-10-19
+
+### Fixed
+
 - **The dashboard's Retry, Discard and Discard all reach a failed job's message under priority routing (issue #577).** A failed-job row records the logical queue only, and with `priority_levels > 1` the message sits in a `_pN` sub-queue, never in the base table. Retry reset the visibility timeout on `pgmq.q_pgbus_<queue>`, which priority routing never creates, and answered "Could not retry job."; Discard and Discard all archived from that same table, so they reported the job discarded while its message stayed queued and ran again. Each sub-queue also numbers its messages from its own sequence, so the msg_id alone could name another job's message in a sibling sub-queue. The dashboard now finds the sub-queue whose message has the row's msg_id and carries the job's `job_id` (`Pgbus::Client#locate_message`, which returns that physical table or nil, and `#locate_messages` for many at once, one query per table), resets or archives the message there, and leaves a same-numbered message elsewhere alone. Discard all locates a queue's messages together and archives them in one batch per table. When no sub-queue holds it, Retry enqueues a fresh copy as before. Without priority routing there is one table and nothing changes, except that a message reusing the msg_id under another `job_id` is no longer reset or archived in the job's place. Refs #577.
 
 ## [0.21.0] - 2026-10-19

data/README.md

@@ -206,7 +206,7 @@ class OrderCreatedHandler < Pgbus::EventBus::Handler
   idempotent!  # Deduplicate by (event_id, handler_class)
 
   def handle(event)
-    order_id = event.payload["order_id"]
+    order_id = event["order_id"]
     Analytics.track_order(order_id)
     InventoryService.reserve(order_id)
   end

data/Rakefile

@@ -61,11 +61,18 @@ namespace :bench do
     ruby "benchmarks/integration_bench.rb"
   end
 
-  desc "Run worker profiling bench: jobs/s and where a job's time goes (requires PGBUS_DATABASE_URL)"
+  desc "Run worker profiling bench: jobs/s and where a job's time goes; WP_BENCH_ROLE=worker|consumer|both|fleet|all " \
+       "(requires PGBUS_DATABASE_URL)"
   task :worker_profile do
     ruby "benchmarks/worker_profile_bench.rb"
   end
 
+  desc "Run the fleet cell: jobs/s and Postgres ms/job at WP_BENCH_PROCESSES (default 1,2,4) worker processes " \
+       "on one queue (requires PGBUS_DATABASE_URL)"
+  task :fleet do
+    sh({ "WP_BENCH_ROLE" => "fleet" }, FileUtils::RUBY, "benchmarks/worker_profile_bench.rb")
+  end
+
   desc "Run the workflow insights query budget: ~1M step rows (requires PGBUS_DATABASE_URL)"
   task :workflow_insights do
     ruby "benchmarks/workflow_insights_bench.rb"

data/lib/pgbus/active_job/executor.rb

@@ -1,16 +1,21 @@
 # frozen_string_literal: true
 
-require "time"
-
 module Pgbus
   module ActiveJob
     class Executor
       attr_reader :client, :config
 
-      def initialize(client: Pgbus.client, config: Pgbus.configuration, stat_buffer: nil)
+      # completer: the worker's Process::Completer when config.completion_batching
+      # is on (issue #606): complete_from goes through it instead of one
+      # archive_message per job. config.completed_jobs (issue #607) is read
+      # once here: :delete finishes a successful job with delete_message.
+      def initialize(client: Pgbus.client, config: Pgbus.configuration, stat_buffer: nil, completer: nil)
         @client = client
         @config = config
         @stat_buffer = stat_buffer
+        @completer = completer
+        @delete_completed = config.completed_jobs == :delete
+        @completed_word = @delete_completed ? "deleted" : "archived"
       end
 
       # Exceptions we never want to swallow — let the process die/signal propagate.
@@ -117,14 +122,15 @@ module Pgbus
           # heartbeat lapsed and it was redelivered): that worker owns the
           # completion signals, and signalling again here would release the
           # concurrency slot a second time (rails/solid_queue#761).
-          if archive_from(queue_name, msg_id, source_queue: source_queue) == :already_archived
+          if complete_from(queue_name, msg_id, source_queue: source_queue) == :already_archived
             Pgbus.logger.warn do
-              "[Pgbus::Executor] already archived elsewhere, skipping signals #{log_tag(message, queue_name)} job_class=#{job_class}"
+              "[Pgbus::Executor] already #{@completed_word} elsewhere, skipping signals " \
+                "#{log_tag(message, queue_name)} job_class=#{job_class}"
             end
             release_duplicate_execution_lock(uniqueness_key, uniqueness_strategy, queue_name, msg_id)
             return :duplicate
           end
-          Pgbus.logger.debug { "[Pgbus::Executor] archived #{log_tag(message, queue_name)} job_class=#{job_class}" }
+          Pgbus.logger.debug { "[Pgbus::Executor] #{@completed_word} #{log_tag(message, queue_name)} job_class=#{job_class}" }
           job_succeeded = true
           release_uniqueness_lock(uniqueness_key, queue_name, message)
           # Only FailedEventRecorder.record! writes this table, and only after a
@@ -147,7 +153,7 @@ module Pgbus
       rescue Exception => e # rubocop:disable Lint/RescueException
         # Widened from StandardError to catch Async::Stop / Async::Cancel
         # (both inherit from Exception, not StandardError) under execution_mode: :async.
-        # Before this, a fiber interruption between perform_now and archive_from
+        # Before this, a fiber interruption between perform_now and complete_from
         # silently lost control flow — no failed event row, no job_failed
         # notification, uniqueness lock held until VT expired. See issue #126.
         handle_failure(message, queue_name, e, payload: payload, source_queue: source_queue)
@@ -196,7 +202,8 @@ module Pgbus
       # The table the message was read from: the _pN sub-queue under priority
       # routing, else the logical queue's own. Each table numbers its messages
       # from its own sequence, so a failed-event row is keyed on it (issue
-      # #583). Built only on the failure and redelivery paths.
+      # #583). Built only on the failure and redelivery paths, and for the
+      # completer (issue #606), which groups by physical queue.
       def physical_queue(queue_name, source_queue)
         source_queue || config.queue_name(queue_name)
       end
@@ -282,33 +289,43 @@ module Pgbus
         Pgbus.logger.debug { "[Pgbus] Stat recording failed: #{e.message}" }
       end
 
+      # pgmq-ruby sets no result type map, so enqueued_at is the TIMESTAMPTZ
+      # text PostgreSQL sends ("2026-10-29 10:11:12.123456+00", any offset).
+      # Time.new reads it in C at ~0.3 µs; Time.parse took ~7 µs a job and
+      # pg's TimestampWithTimeZone decoder ~3 µs (issue #604). Unreadable text
+      # raises ArgumentError: the stat is recorded without latency.
       def compute_enqueue_latency(message)
         return unless message
 
         enqueued_at = message.enqueued_at
         return unless enqueued_at
 
-        # Fast path: numeric epoch (float seconds) avoids Time.parse entirely.
-        # PGMQ returns enqueued_at as a Time or string depending on the driver.
-        case enqueued_at
-        when Numeric
-          [((Time.now.to_f - enqueued_at) * 1000).round, 0].max
-        when Time
-          [((Time.now.utc - enqueued_at.utc) * 1000).round, 0].max
-        else
-          parse_enqueue_latency_from_string(enqueued_at.to_s)
-        end
+        epoch = case enqueued_at
+                when Numeric then enqueued_at
+                when Time then enqueued_at.to_f
+                else enqueued_at_epoch(enqueued_at.to_s)
+                end
+        return unless epoch
+
+        [((::Process.clock_gettime(::Process::CLOCK_REALTIME) - epoch) * 1000).round, 0].max
       rescue ArgumentError, TypeError
         nil
       end
 
-      def parse_enqueue_latency_from_string(str)
-        # PGMQ enqueued_at is TIMESTAMPTZ (always UTC internally).
-        # If the string lacks an explicit offset, assume UTC to avoid
-        # misinterpretation when the system timezone is non-UTC.
-        str = "#{str} UTC" unless str.match?(/[+-]\d{2}:?\d{2}\s*$|Z\s*$/i)
-        enqueued_at = Time.parse(str)
-        [((Time.now.utc - enqueued_at) * 1000).round, 0].max
+      # A trailing "+00", "-03", "+05:30", "+0530" or "Z".
+      UTC_OFFSET_SUFFIX = /(?:[+-]\d\d(?::?\d\d)?|Z)\z/
+      private_constant :UTC_OFFSET_SUFFIX
+
+      # Time.new reads a string without an offset in the host's zone, but
+      # TIMESTAMPTZ is UTC inside, so such a string gets a "Z". Checked here
+      # rather than with `in: "UTC"`: that keyword costs a Hash per job.
+      # Time.new also takes a bare year ("2026") as a time; a string without a
+      # time of day is not a timestamp, so it has no latency.
+      def enqueued_at_epoch(str)
+        return unless str.include?(":")
+
+        str = "#{str}Z" unless str.match?(UTC_OFFSET_SUFFIX)
+        Time.new(str).to_f
       end
 
       def handle_failure(message, queue_name, error, payload: nil, source_queue: nil)
@@ -412,7 +429,11 @@ module Pgbus
       # that succeeded but failed to archive redelivers after VT expiry and
       # runs twice. If the retry also fails, fall through to the normal
       # failure path (recorded failure + VT-based redelivery).
-      # Returns :archived, :already_archived, or :ambiguous.
+      # Returns :archived, :already_archived, or :ambiguous. Under
+      # config.completed_jobs = :delete it deletes the message instead
+      # (delete_message, or the completer's delete_batch): pgmq.delete is as
+      # idempotent and returns false the same way, so the symbols keep their
+      # names and "archived" means "completed".
       #
       # A `false` reported after our OWN retry is ambiguous, not a duplicate:
       # the first archive may well have committed with only its reply lost to
@@ -421,10 +442,18 @@ module Pgbus
       # batch and uniqueness state on every such blip, whereas a genuine
       # duplicate would additionally require another worker to have taken and
       # finished the same message inside the retry window.
-      def archive_from(queue_name, msg_id, source_queue: nil)
+      def complete_from(queue_name, msg_id, source_queue: nil)
         attempts = 0
         begin
-          archived = if source_queue
+          archived = if @completer
+                       @completer.complete(physical_queue(queue_name, source_queue), msg_id) == :archived
+                     # Spelled out, no `**` forward: on Ruby 3.3 that
+                     # allocates per job even without keywords (#484 budget).
+                     elsif @delete_completed && source_queue
+                       client.delete_message(source_queue, msg_id, prefixed: false)
+                     elsif @delete_completed
+                       client.delete_message(queue_name, msg_id)
+                     elsif source_queue
                        client.archive_message(source_queue, msg_id, prefixed: false)
                      else
                        client.archive_message(queue_name, msg_id)
@@ -437,7 +466,7 @@ module Pgbus
           raise unless attempts == 1 && connection_error?(e)
 
           Pgbus.logger.warn do
-            "[Pgbus::Executor] Archive failed on connection error, retrying once: #{e.message}"
+            "[Pgbus::Executor] #{@delete_completed ? "Delete" : "Archive"} failed on connection error, retrying once: #{e.message}"
           end
           retry
         end

data/lib/pgbus/batch/sweep.rb

@@ -47,6 +47,8 @@ module Pgbus
 
           dlq = client.dead_letter_physical_name(row.queue_name, prefixed: false)
           return :failed if client.message_with_job_id?(dlq, job_id: row.job_id, prefixed: false) == true
+          # completed_jobs :delete (issue #607) leaves no archive row to find.
+          return :completed if Pgbus.configuration.completed_jobs == :delete
 
           return :completed if client.message_archived?(row.queue_name, msg_id: row.msg_id, prefixed: false) == true
 

data/lib/pgbus/batch.rb

@@ -425,8 +425,11 @@ module Pgbus
     end
 
     def self.backfill_execution(payload, msg_id, queue_name)
-      return unless executions_migrated?
       return unless payload && msg_id
+      # Only batch members have an execution row: without this, every plain
+      # enqueue paid a no-op UPDATE round trip after the send (issue #603).
+      return unless payload[METADATA_KEY]
+      return unless executions_migrated?
 
       job_id = payload["job_id"]
       return unless job_id