pgmq-ruby 0.8.2 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 943d0ebf4cffa43c1163f651575cbfb0e9785fc675f388ef7b208538e9dab8a3
-  data.tar.gz: fd9ab2693a0bf87c6230330ac8c4e046c585c420ecbdce0b77b06a6b652fd266
+  metadata.gz: 79570c7c6e4591584c43e16f99064d7d249148601a74d8ff853791511d9d87ee
+  data.tar.gz: 235af4a2895ffb1dd3a3b29b2e2f7448b8c0e32899af3a62d7f30fc9c7277d50
 SHA512:
-  metadata.gz: 6cc612c51d2b89e7ecfd3ea6879eb3e731a8ce96cba961863119a34b9dd094b0e5c139045d0c0cdf0df964130ec0e65a9df4961defe6843603767e9369b60800
-  data.tar.gz: ff1caf1d8012504204f5d793b577c0c3603ba2d156257786a32fab73618d52971d0c49edfba0860f9a5ad9dd56e34dca272891e594b55521761f514fd8fdc053
+  metadata.gz: 9483f8855343ad8a5a9292d76544726624380bef0d14dcfe8321ade758ebb32d71543a28a87176ffe323a4577bc0d1c2a9407510d1bf6bd5e41d360ec416aedd
+  data.tar.gz: 8a09f3994b147dece5c30e449362331337e89e0fbe0315b18b727552b234a5de2f7c74a48fd0329ff9b650d9c0ddc27080a89b4910d807d61b83afbe000421d9

data/CHANGELOG.md

@@ -1,12 +1,45 @@
 # Changelog
 
+## 0.9.0 (2026-11-10)
+
+### Producer
+- **[Breaking]** `produce_topic` and `produce_batch_topic` raise `PGMQ::Errors::MessageInvalidError` when a message is
+  not a String or its headers are neither a String nor `nil`, like `produce` and `produce_batch`. Most such calls
+  succeeded before and now raise: non-String messages and headers were stored as JSON (Integer, Float, `true`, `false`
+  and Array values) or as NULL (a `nil` message; Hash headers in `produce_topic` with an integer delay), and
+  `produce_topic` with an integer delay dropped `false` headers. The other `nil` and Hash cases raised
+  `PGMQ::Errors::ConnectionError`.
+
+### Validation
+- **[Change]** A contract check allocates its errors Array on the first error only, so valid input allocates one
+  object less per validated message in `produce`, `produce_batch`, `produce_topic` and `produce_batch_topic`.
+  Until then a `PGMQ::Contractable::Contract` rule gets a frozen empty `errors` Array: a custom rule that appends to it
+  raises `FrozenError`.
+
+### Connection Management
+- **[Fix]** `PGMQ::Connection#verify_connection!` resets a pooled connection whose status is anything other than
+  `PG::CONNECTION_OK` (#217). It reset only `CONNECTION_BAD` before, so a connection left in an intermediate state
+  (`CONNECTION_STARTED`, `CONNECTION_MADE`, ...) by a reset or connect interrupted before it completed (a
+  `Thread#raise` during `PG::Connection#reset`, a `reset_start` never polled to completion) passed the health check,
+  and every call on it, the `auto_reconnect` retry included, raised `PGMQ::Errors::ConnectionError` (`PG::UnableToSend:
+  no connection to the server`) until `reload` or a process restart. The next checkout now resets it.
+- **[Fix]** `PGMQ::Connection#with_connection` resets a pooled connection when the block raises `PG::ConnectionBad`
+  or `PG::UnableToSend`, before the connection goes back to the pool. A connection can pass the checkout health check
+  and still fail on the next send; it went back to the pool broken, and with `auto_reconnect: false` (or when it still
+  reported `PG::CONNECTION_OK`) every later checkout failed the same way until `reload`. The caller still gets
+  `PGMQ::Errors::ConnectionError`. When the reset fails too, the connection is closed: with `auto_reconnect` the next
+  checkout resets it; without it, the next operation on it fails once and that failure resets it. Connections returned
+  by a callable (e.g. ActiveRecord's raw connection) are not reset or closed.
+
 ## 0.8.2 (2026-11-02)
 
 ### Queue Naming
 - **[Fix]** The `InvalidQueueNameError` message for a too-long queue name states the real limit: "exceeds maximum
   length of 47 characters" (was 48; a 48-character name was already rejected). The README now gives 47 as the maximum
   and no longer calls queue names case-sensitive: names differing only in case share one table (PGMQ lower-cases
-  table names) but get separate `pgmq.meta` rows, so use one spelling per queue.### Documentation
+  table names) but get separate `pgmq.meta` rows, so use one spelling per queue.
+
+### Documentation
 - **[Fix]** The README `wait_for_notify` example said the call returns the notification payload; it returns the
   channel name (`nil` on timeout).
 

data/README.md

@@ -93,7 +93,6 @@ This gem provides complete support for all core PGMQ SQL functions. Based on the
 | | Consumer | Minimal single-threaded `PGMQ::Consumer` loop over one or more queues with ack on success | ✅ |
 | | Queue Validation | 47-character limit and name validation | ✅ |
 | | Connection Pooling | Thread-safe connection pool for concurrency | ✅ |
-| | Pluggable Serializers | JSON (default) with custom serializer support | ✅ |
 
 ## Requirements
 
@@ -308,6 +307,9 @@ Every PGMQ operation checks out a pooled connection through `client.with_connect
 The method is public, so you can run PostgreSQL statements PGMQ does not wrap
 without standing up a second connection pool. The connection is health-checked
 (when `auto_reconnect` is enabled) and returned to the pool when the block exits.
+If the block raises `PG::ConnectionBad` or `PG::UnableToSend`, the connection is
+reset before it goes back to the pool (connections from a callable are left as
+they are); you still get `PGMQ::Errors::ConnectionError`.
 
 ```ruby
 # Fire a custom NOTIFY alongside your queues
@@ -1230,8 +1232,8 @@ end
 
 Invalid arguments are rejected by a contract before they reach PostgreSQL, with a validation error:
 
-- `PGMQ::Errors::MessageInvalidError` from `produce`, `produce_batch` and `produce_batch_topic`, for example when a
-  message is not a String or its headers are neither a String nor `nil`
+- `PGMQ::Errors::MessageInvalidError` from `produce`, `produce_batch`, `produce_topic` and `produce_batch_topic`, for
+  example when a message is not a String or its headers are neither a String nor `nil`
 - `PGMQ::Errors::MultiQueueInvalidError` from the multi-queue operations (`read_multi`, `read_multi_with_poll`,
   `pop_multi`, `delete_multi`, `archive_multi`, `set_vt_multi`)
 - `PGMQ::Errors::AutovacuumSettingsInvalidError` from `tune_autovacuum` for an unknown storage parameter or a
@@ -1268,9 +1270,6 @@ bin/integrations
 
 # Run a specific integration spec
 bin/integrations spec/integration/basic_produce_consume_spec.rb
-
-# Run console
-bundle exec bin/console
 ```
 
 ## Author

data/lib/pgmq/client/producer.rb

@@ -103,17 +103,9 @@ module PGMQ
         delay: 0
       )
         validate_queue_name!(queue_name)
-        BATCH_CONTRACT.validate!({ messages: messages, headers: headers }, Errors::MessageInvalidError)
+        validate_messages!(messages, headers)
         return [] if messages.empty?
 
-        messages.each_with_index do |message, index|
-          MESSAGE_CONTRACT.validate!(
-            { message: message, headers: headers && headers[index] },
-            Errors::MessageInvalidError,
-            scope: ["messages", index]
-          )
-        end
-
         result = with_connection do |conn|
           encoder = PG::TextEncoder::Array.new
           encoded_messages = encoder.encode(messages)

data/lib/pgmq/client/topics.rb

@@ -73,6 +73,8 @@ module PGMQ
       # @param delay [Numeric, ActiveSupport::Duration] delay in seconds before message becomes visible (integer,
       #   float or Duration; a float such as 1.5 needs PGMQ v1.10.0+)
       # @return [Integer] count of queues the message was delivered to
+      # @raise [PGMQ::Errors::MessageInvalidError] if message is not a String, or headers is neither a String nor
+      #   nil
       #
       # @example Basic topic send
       #   count = client.produce_topic("orders.new", '{"order_id":123}')
@@ -83,6 +85,8 @@ module PGMQ
       #     headers: '{"trace_id":"abc"}',
       #     delay: 30)
       def produce_topic(routing_key, message, headers: nil, delay: 0)
+        MESSAGE_CONTRACT.validate!({ message: message, headers: headers }, Errors::MessageInvalidError)
+
         # send_topic takes whole seconds only, so a fractional delay goes through send_batch_topic
         if delay.is_a?(Numeric) && !delay.integer?
           return produce_batch_topic(routing_key, [message], headers: headers && [headers], delay: delay).size
@@ -122,7 +126,8 @@ module PGMQ
       #   become visible, including ActiveSupport::TimeWithZone (PGMQ v1.10.0+)
       # @return [Array<Hash>] array of hashes with :queue_name and :msg_id
       # @raise [PGMQ::Errors::MessageInvalidError] if messages is not an Array, headers is neither an Array nor nil,
-      #   or headers length doesn't match messages length
+      #   headers length doesn't match messages length, a message is not a String, or a header is neither a String
+      #   nor nil
       #
       # @example Batch topic send
       #   results = client.produce_batch_topic("orders.new", [
@@ -131,7 +136,7 @@ module PGMQ
       #   ])
       #   # => [{ queue_name: "new_orders", msg_id: "1" }, ...]
       def produce_batch_topic(routing_key, messages, headers: nil, delay: 0)
-        BATCH_CONTRACT.validate!({ messages: messages, headers: headers }, Errors::MessageInvalidError)
+        validate_messages!(messages, headers)
         return [] if messages.empty?
 
         result = with_connection do |conn|

data/lib/pgmq/client.rb

@@ -173,6 +173,28 @@ module PGMQ
       nil
     end
 
+    # Validates a batch of messages and their headers before they are bound
+    #
+    # @param messages [Array<String>] message payloads as JSON strings
+    # @param headers [Array<String, nil>, nil] headers as JSON strings, one per message
+    # @return [void]
+    # @raise [PGMQ::Errors::MessageInvalidError] if messages is not an Array, headers is neither an Array nor nil,
+    #   headers length doesn't match messages length, a message is not a String, or a header is neither a String nor
+    #   nil
+    def validate_messages!(messages, headers)
+      BATCH_CONTRACT.validate!({ messages: messages, headers: headers }, Errors::MessageInvalidError)
+
+      messages.each_with_index do |message, index|
+        MESSAGE_CONTRACT.validate!(
+          { message: message, headers: headers && headers[index] },
+          Errors::MessageInvalidError,
+          scope: ["messages", index]
+        )
+      end
+
+      nil
+    end
+
     # Builds the SQL argument and bind value for a +delay:+ given in seconds or as an absolute Time
     #
     # PostgreSQL rejects a Float such as +1.5+ for an +integer+ bind, so a non-Integer Numeric is passed to the

data/lib/pgmq/connection.rb

@@ -116,11 +116,17 @@ module PGMQ
       @pool_size = pool_size
       @pool_timeout = pool_timeout
       @auto_reconnect = auto_reconnect
+      # A callable hands us connections it may share with its owner (e.g. ActiveRecord's raw connection), so we never
+      # reset or close them after a failure; only connections we open ourselves are reset.
+      @owns_connections = !conn_params.respond_to?(:call)
       @pool = create_pool
     end
 
     # Executes a block with a connection from the pool
     #
+    # When the block raises `PG::ConnectionBad` or `PG::UnableToSend`, the connection is reset before it goes back to
+    # the pool, so the next checkout gets a working one. Connections from a callable are left as they are.
+    #
     # @yield [PG::Connection] database connection
     # @return [Object] result of the block
     # @raise [PGMQ::Errors::ConnectionError] if the block raises any `PG::Error` (the original is in `#cause`), or a
@@ -134,7 +140,12 @@ module PGMQ
           # Health check: verify connection is alive
           verify_connection!(conn) if @auto_reconnect
 
-          yield conn
+          begin
+            yield conn
+          rescue *LOST_CONNECTION_ERRORS
+            reset_lost_connection(conn) if @owns_connections
+            raise
+          end
         end
       rescue PG::Error => e
         attempts += 1
@@ -161,8 +172,8 @@ module PGMQ
     #
     # Use this to recover from a connection that libpq still reports as +CONNECTION_OK+ but which is in fact wedged -
     # e.g. after a wall-clock timeout (+Timeout.timeout+) interrupted a query mid-flight, leaving the socket poisoned so
-    # it re-hangs on reuse. {#verify_connection!} cannot catch that case (the connection does not report
-    # +CONNECTION_BAD+), so the caller must discard it explicitly; +reload+ is the safe, pool-wide way to do so.
+    # it re-hangs on reuse. {#verify_connection!} cannot catch that case (the connection still reports
+    # +CONNECTION_OK+), so the caller must discard it explicitly; +reload+ is the safe, pool-wide way to do so.
     #
     # @return [void]
     # @example Discard the pool after a query had to be force-interrupted
@@ -191,6 +202,10 @@ module PGMQ
 
     private
 
+    # libpq's dedicated connection-failure classes. A pooled connection that raises one of them mid-operation is reset.
+    LOST_CONNECTION_ERRORS = [PG::ConnectionBad, PG::UnableToSend].freeze
+    private_constant :LOST_CONNECTION_ERRORS
+
     # Messages libpq raises when the server/pooler has already torn down the socket. The list has grown organically with
     # each pooler/TLS variant we see in the wild; the class check below catches future variants that libpq raises as
     # `PG::ConnectionBad` or `PG::UnableToSend` without waiting for a new message to hit production.
@@ -220,7 +235,7 @@ module PGMQ
     # @return [Boolean] true if the connection was lost and a retry on a
     #   fresh connection is appropriate
     def connection_lost_error?(error)
-      return true if error.is_a?(PG::ConnectionBad) || error.is_a?(PG::UnableToSend)
+      return true if LOST_CONNECTION_ERRORS.any? { |klass| error.is_a?(klass) }
 
       extra_classes = self.class.reconnectable_error_classes
       return true if extra_classes.any? { |klass| error.is_a?(klass) }
@@ -240,19 +255,36 @@ module PGMQ
 
     # Verifies a connection is alive and working.
     #
-    # Also resets when the connection reports `PG::CONNECTION_BAD`, which happens when the server (or an intermediate
-    # pooler such as PgBouncer) has closed the socket while the client-side `PG::Connection` object still exists.
-    # `#finished?` alone only catches connections closed explicitly from the client side.
+    # Also resets when the connection reports any status other than `PG::CONNECTION_OK`: `CONNECTION_BAD` when the
+    # server (or an intermediate pooler such as PgBouncer) closed the socket while the client-side `PG::Connection`
+    # object still exists, and an intermediate state (`CONNECTION_STARTED`, `CONNECTION_MADE`, ...) when a previous
+    # reset or connect was interrupted before it completed. libpq refuses to send on either. `#finished?` alone only
+    # catches connections closed explicitly from the client side.
     #
     # @param conn [PG::Connection] connection to verify
     # @raise [PG::Error] if the reset itself fails
     def verify_connection!(conn)
       return conn.reset if conn.finished?
-      return conn.reset if conn.status == PG::CONNECTION_BAD
+      return conn.reset unless conn.status == PG::CONNECTION_OK
 
       nil
     end
 
+    # Resets a connection that failed mid-operation so it does not go back to the pool broken. A connection can pass
+    # {#verify_connection!} at checkout (`CONNECTION_OK`) and still fail on the next send.
+    #
+    # A failing reset must not replace the error the caller sees, so it is swallowed and the connection is closed
+    # instead. With `auto_reconnect`, the next checkout sees it as `finished?` and resets it. Without it, the next
+    # operation on it fails with `PG::ConnectionBad` ("connection is closed"), and that failure resets it.
+    #
+    # @param conn [PG::Connection] connection the failed operation used
+    # @return [void]
+    def reset_lost_connection(conn)
+      conn.reset
+    rescue
+      conn.close unless conn.finished?
+    end
+
     # Normalizes various connection parameter formats
     # @param params [String, Hash, Proc]
     # @return [Hash, Proc]

data/lib/pgmq/contractable/contract.rb

@@ -21,7 +21,8 @@ module PGMQ
       DIG_MISS = Object.new
 
       # Empty array for scope default to avoid allocating a new Array on each `#call` / `#validate!` invocation. Safe
-      # because scope is never mutated - it is only used in `scope + rule.path` which creates a new Array.
+      # because scope is never mutated - it is only used in `scope + rule.path` which creates a new Array. Also passed
+      # to validators as `errors` until the first error.
       EMPTY_ARRAY = [].freeze
 
       private_constant :DIG_MISS, :EMPTY_ARRAY
@@ -77,7 +78,7 @@ module PGMQ
         #
         # @param block [Proc] validation rule, called with `(data, errors, contract)`. It must return either a
         #   non-Array (`true`/`nil`/`false`) for "no errors", or an `Array` of `[path, message]` error pairs (where
-        #   `path` is itself an `Array` of symbols).
+        #   `path` is itself an `Array` of symbols). `errors` holds the errors found so far and must not be mutated.
         #
         # @note The returned error `Array` is owned by the contract: `#call` prepends the current scope onto each pair
         #   in place and collects them, so a rule must return a freshly built `Array` on every call.
@@ -102,39 +103,41 @@ module PGMQ
       #   needed if contract starts from root
       # @return [Result] validation result
       def call(data, scope: EMPTY_ARRAY)
-        errors = []
+        # Allocated on the first error, so valid data allocates no Array
+        errors = nil
 
         # A non-Hash root has no keys to dig, so every required rule reports missing
         data_is_hash = data.is_a?(Hash)
 
         self.class.rules.each do |rule|
           if rule.type == :virtual
-            result = rule.validator.call(data, errors, self)
+            result = rule.validator.call(data, errors || EMPTY_ARRAY, self)
 
             # Any non-Array result (true, but also nil/false) means "no errors"
             next unless result.is_a?(Array)
+            next if result.empty?
 
             result.each do |sub_result|
               sub_result[0] = scope + sub_result[0]
             end
 
-            errors.push(*result)
+            (errors ||= []).push(*result)
           else
             for_checking = data_is_hash ? dig(data, rule.path) : DIG_MISS
 
             if DIG_MISS.equal?(for_checking)
-              errors << [scope + rule.path, :missing] if rule.type == :required
+              (errors ||= []) << [scope + rule.path, :missing] if rule.type == :required
             else
-              result = rule.validator.call(for_checking, data, errors, self)
+              result = rule.validator.call(for_checking, data, errors || EMPTY_ARRAY, self)
 
               next if result == true
 
-              errors << [scope + rule.path, result || :format]
+              (errors ||= []) << [scope + rule.path, result || :format]
             end
           end
         end
 
-        return Result.success if errors.empty?
+        return Result.success unless errors
 
         Result.new(errors, self)
       end

data/lib/pgmq/version.rb

@@ -2,5 +2,5 @@
 
 module PGMQ
   # Current version of the pgmq-ruby gem
-  VERSION = "0.8.2"
+  VERSION = "0.9.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: pgmq-ruby
 version: !ruby/object:Gem::Version
-  version: 0.8.2
+  version: 0.9.0
 platform: ruby
 authors:
 - Maciej Mensfeld