pgbus 0.15.1 → 0.15.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c419d130102e92e1bd355df6e98211e44eeead76298f9ce58bd6c285b1de24a9
-  data.tar.gz: dab0eefe9c7a23aee676b928521841adffb6d38f084993c6bb3ab56f9ade207f
+  metadata.gz: 13ec61547a6821edab1a7aa96e794d522cc6fdcd38dd354bf601c9c760a2fded
+  data.tar.gz: b79ef51aa82c336ba98e3df071d40e9de5467bec196fb53edaf8a4fa5507327d
 SHA512:
-  metadata.gz: d8cedff08ffed91f3b9a9c1b9df1b2147790ec9bf7e5117b22a4af19b865b239df4883d6708153ec2efc9de5207e007cb4d2cfd06440ac3b4bd25e63d1eea369
-  data.tar.gz: fcb2ec48f7daf53dc46d5c388cb3b4e1acd99a568e74e92253736efb8ba4bc52fd9704d287fbaafcea893d063676b3f68ceac3d82b79fc3906ef4d7bbc43dcd3
+  metadata.gz: 8647f5cff936fae72bc95ea49c793fc68e514c29c2955d88b668dd6e5905a95418f7efdb8082e77048a584b9a0c86aa2cea42bbcd1f0777df349606ddbd4db08
+  data.tar.gz: ee10d3f15f0e03a8a46d359c5aaf19f3329ed628dd4ae72d98fe00aec4b9689ee43005b73884b62e997fb3305c0b2c5ec2ce9a747646e558a1ade1cbeca27f0e

data/CHANGELOG.md

@@ -2,6 +2,9 @@
 
 ### Fixed
 
+- **Worker forks no longer kill the supervisor's shared LISTEN connection (issue #437).** Under `worker_notify_scope = :supervisor` every fork produced one `[Pgbus::NotifyListener] connection error (PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly …) — reconnecting` in the supervisor ~2 s after `Worker started`, and a LISTEN gap (polling fallback) until `reconnect!` completed. The child's fork hygiene closed the Ruby `IO` wrapper of the inherited LISTEN socket, but pg builds `socket_io` with `autoclose=false`, so the fd stayed open — and when the child's GC freed the inherited `PG::Connection`, its `PQfinish` sent a libpq Terminate down that fd, i.e. down the **parent's** connection. `NotifyListener#close_inherited_socket!` now repoints the fd at `/dev/null` (`socket_io.reopen(IO::NULL)`, the ActiveRecord `PostgreSQLAdapter#discard!` idiom) so the eventual `PQfinish` is harmless. Regression-covered by a real-fork integration spec that asserts the parent's `pgbus-listen` backend pid is unchanged and no reconnect is logged. Refs #437.
+- **A process whose `pgbus_processes` row is deleted underneath it now re-registers instead of staying invisible forever (issue #438).** `Heartbeat#beat` updated its row by id; when the row was gone — the dispatcher's stale-process reaper after a ≥ 5-minute heartbeat gap, a manual cleanup, another host's clock skew — `update_all` matched 0 rows, raised nothing, and the process (healthy, logging, serving `/readyz`) was absent from `pgbus_processes` until it restarted, so the supervisor's DB loop-tick check silently fell back to the pipe and `ConsumerPriority` could not see it. The beat now treats an affected-row count of 0 as "row is gone": it logs one WARN naming the old id, kind and pid (so whatever deleted the row can be hunted), re-registers through the existing boot path, and lands that beat's `last_heartbeat_at` / metadata on the new row. A mutex plus a stopped flag keep a beat that races `stop` from resurrecting a row `deregister_process` just deleted. Zero extra queries on the happy path. Refs #438.
+- **Supervisor logs a worker recycle as a clean exit, not a crash (issue #438).** A worker or consumer hitting `max_jobs` / `max_memory` / `max_lifetime` exits 0 by design, but the supervisor logged `Child worker pid=N exited unexpectedly (status=0)`, burying real crashes. A clean exit outside shutdown is now INFO `exited cleanly (status=0) — restarting (worker recycle)`; non-zero exits keep the WARN, and a signaled exit reports `signal=N` instead of an empty status so an OOM SIGKILL is distinguishable. Restart policy is unchanged. Refs #438.
 - **`current_attributes` capture skips an unpersisted record instead of raising at enqueue (issue #435).** An attribute holding an Active Record instance with no id — a dev-mode fallback record assigned to `Current`, a form-built model captured before `save`, a destroyed record whose locate is guaranteed to fail — made every `perform_later` in that context raise `Pgbus::CurrentAttributesError`, even though such a record can never round-trip (no id → no GlobalID) and capture is ambient: the enqueuer never opted into persisting that attribute per-call, so its momentary state must not abort the enqueue. `capture` now skips any attribute value that answers `persisted?` falsey (so destroyed-but-id-bearing records are skipped too, not just `new_record?`) with a debug log naming the class, attribute and why; the rest of the class's attributes still persist. The `except:` guidance and the loud `CurrentAttributesError` remain for genuinely unserializable values — objects without the Active Record duck-type (`respond_to?(:persisted?)`) are untouched. Applies to jobs and event-bus publish alike (same capture path). Refs #435.
 
 ### Added

data/lib/pgbus/client.rb

@@ -1753,11 +1753,10 @@ module Pgbus
       timeout = config.read_timeout
       return mapping_statement_timeout(&block) unless timeout&.positive?
 
-      # rubocop:disable Pgbus/NoRubyTimeout -- deliberate last-resort bound; see above
+      # rubocop:disable-next Pgbus/NoRubyTimeout -- deliberate last-resort bound; see above
       Timeout.timeout(timeout + READ_TIMEOUT_SLACK, WedgedReadTimeout) do
         mapping_statement_timeout(&block)
       end
-      # rubocop:enable Pgbus/NoRubyTimeout
     rescue WedgedReadTimeout
       reload_pool_after_wedged_timeout
       raise

data/lib/pgbus/execution_pools/async_pool.rb

@@ -95,7 +95,7 @@ module Pgbus
               "Add `gem \"async\"` to your Gemfile. Original error: #{e.message}"
       end
 
-      # rubocop:disable Lint/RescueException
+      # rubocop:disable-next Lint/RescueException
       def start_reactor
         Thread.new do
           Thread.current.name = "pgbus-async-reactor-#{object_id}"
@@ -122,7 +122,6 @@ module Pgbus
           raise
         end
       end
-      # rubocop:enable Lint/RescueException
 
       def wait_for_executions(semaphore)
         loop do

data/lib/pgbus/process/heartbeat.rb

@@ -18,6 +18,12 @@ module Pgbus
         @loop_tick_supplier = loop_tick_supplier
         @metadata_supplier = metadata_supplier
         @timer = nil
+        @stopped = false
+        # Guards @process_id between the timer thread (beat) and the main
+        # thread (stop): TimerTask#shutdown does not wait for an in-flight
+        # beat, and a beat that re-registers after deregister_process ran
+        # would leave a zombie row (issue #438).
+        @mutex = Mutex.new
       end
 
       def start
@@ -27,18 +33,19 @@ module Pgbus
       end
 
       def stop
+        @stopped = true
         @timer&.shutdown
-        deregister_process
+        @mutex.synchronize { deregister_process }
       end
 
       def beat
-        return unless @process_id
+        return unless @process_id && !@stopped
 
         @on_beat&.call
         updates = { last_heartbeat_at: Time.current }
         metadata = beat_metadata
         updates[:metadata] = metadata unless metadata.nil?
-        ProcessEntry.where(id: @process_id).update_all(updates)
+        @mutex.synchronize { write_beat(updates) }
       rescue StandardError => e
         Pgbus.logger.warn { "[Pgbus] Heartbeat failed: #{e.message}" }
       end
@@ -59,6 +66,25 @@ module Pgbus
         metadata
       end
 
+      # update_all by id matches 0 rows when the row was deleted underneath a
+      # live process (stale-process reaper after a heartbeat gap, manual
+      # cleanup, another host's clock skew). Nothing raises, so treat the
+      # count as the signal: re-register and log once so the cause of the
+      # deletion is discoverable, then land this beat's updates on the new
+      # row (issue #438). Skipped after stop so a beat racing deregistration
+      # cannot resurrect the row.
+      def write_beat(updates)
+        return if ProcessEntry.where(id: @process_id).update_all(updates).positive? || @stopped
+
+        old_id = @process_id
+        Pgbus.logger.warn do
+          "[Pgbus] Process row id=#{old_id} kind=#{@kind} pid=#{::Process.pid} is gone " \
+            "(stale-process reaper, manual cleanup, or clock skew?) — re-registering"
+        end
+        register_process
+        ProcessEntry.where(id: @process_id).update_all(updates) if @process_id != old_id
+      end
+
       def register_process
         record = ProcessEntry.create!(
           kind: @kind,

data/lib/pgbus/process/notify_listener.rb

@@ -143,13 +143,21 @@ module Pgbus
         @state_mutex.synchronize { @running }
       end
 
-      # Called ONLY inside a just-forked child (issue #381 hub hygiene): drop
-      # this process's copy of the LISTEN socket fd WITHOUT PQfinish — #close
+      # Called ONLY inside a just-forked child (issues #381 / #437): release
+      # this process's copy of the LISTEN socket WITHOUT PQfinish — #close
       # would send a libpq Terminate over the socket shared with the parent,
-      # killing the parent's LISTEN session. Closing the IO wrapper just
-      # closes the child's fd. The listener thread does not exist in the
-      # child (fork copies only the calling thread), so there is no
-      # concurrent owner and the single-owner rule (#375) does not apply.
+      # killing the parent's LISTEN session.
+      #
+      # Closing the IO wrapper is not enough: pg builds socket_io with
+      # autoclose=false, so IO#close leaves the fd open, and when GC frees the
+      # inherited PG::Connection its PQfinish still writes Terminate on that
+      # fd — i.e. on the PARENT's connection (issue #437, one reconnect per
+      # fork). Repointing the fd at /dev/null (IO#reopen, the ActiveRecord
+      # PostgreSQLAdapter#discard! idiom) makes the eventual PQfinish harmless.
+      #
+      # The listener thread does not exist in the child (fork copies only the
+      # calling thread), so there is no concurrent owner and the single-owner
+      # rule (#375) does not apply.
       def close_inherited_socket!
         conn = @state_mutex.synchronize do
           c = @conn
@@ -157,10 +165,11 @@ module Pgbus
           @running = false
           c
         end
-        conn&.socket_io&.close
+        conn&.socket_io&.reopen(IO::NULL)
       rescue StandardError => e
-        # Best-effort (a lingering fd copy is benign until the parent dies),
-        # but never silent: the child keeps booting either way.
+        # Best-effort but never silent: the child keeps booting either way.
+        # If this fails the child's GC-time PQfinish will hit the parent's
+        # session, which the parent's reconnect! survives.
         @logger.warn do
           "[Pgbus::NotifyListener] inherited socket cleanup failed: #{e.class}: #{e.message}"
         end

data/lib/pgbus/process/supervisor.rb

@@ -613,9 +613,7 @@ module Pgbus
           if @shutting_down
             Pgbus.logger.info { "[Pgbus] Child #{info[:type]} pid=#{pid} exited (status=#{status.exitstatus})" }
           else
-            Pgbus.logger.warn do
-              "[Pgbus] Child #{info[:type]} pid=#{pid} exited unexpectedly (status=#{status&.exitstatus})"
-            end
+            log_child_exit(info, pid, status)
             schedule_restart(info, status)
           end
         rescue Errno::ECHILD
@@ -623,6 +621,27 @@ module Pgbus
         end
       end
 
+      # A clean exit outside shutdown is a worker/consumer recycle (max_jobs,
+      # max_memory, max_lifetime) — expected, so INFO. Anything else is a
+      # crash: WARN, naming the signal when there is one so an OOM SIGKILL
+      # reads differently from an exit code (issue #438).
+      def log_child_exit(info, pid, status)
+        if status&.success?
+          Pgbus.logger.info do
+            "[Pgbus] Child #{info[:type]} pid=#{pid} exited cleanly (status=0) — restarting (worker recycle)"
+          end
+        else
+          Pgbus.logger.warn do
+            detail = if status && status.exitstatus.nil? && status.signaled?
+                       "signal=#{status.termsig}"
+                     else
+                       "status=#{status&.exitstatus}"
+                     end
+            "[Pgbus] Child #{info[:type]} pid=#{pid} exited unexpectedly (#{detail})"
+          end
+        end
+      end
+
       # Restart policy: a clean exit (worker recycling) or a crash after a
       # stable run restarts immediately with a fresh crash streak. A crash
       # within RESTART_STABLE_UPTIME of forking is a crash loop — the child

data/lib/pgbus/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Pgbus
-  VERSION = "0.15.1"
+  VERSION = "0.15.2"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: pgbus
 version: !ruby/object:Gem::Version
-  version: 0.15.1
+  version: 0.15.2
 platform: ruby
 authors:
 - Mikael Henriksson
@@ -424,13 +424,13 @@ files:
 - lib/tasks/pgbus_pgmq.rake
 - lib/tasks/pgbus_queues.rake
 - lib/tasks/pgbus_streams.rake
-homepage: https://github.com/mhenrixon/pgbus
+homepage: https://github.com/zoolutions/pgbus
 licenses:
 - MIT
 metadata:
-  homepage_uri: https://github.com/mhenrixon/pgbus
-  source_code_uri: https://github.com/mhenrixon/pgbus/tree/main
-  changelog_uri: https://github.com/mhenrixon/pgbus/blob/main/CHANGELOG.md
+  homepage_uri: https://github.com/zoolutions/pgbus
+  source_code_uri: https://github.com/zoolutions/pgbus/tree/main
+  changelog_uri: https://github.com/zoolutions/pgbus/blob/main/CHANGELOG.md
   documentation_uri: https://pgbus.zoolutions.llc
   rubygems_mfa_required: 'true'
 rdoc_options: []