async-rabbitmq 0.5.0 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 63e32dac7aed43a3627da9fec4b8b2df543592f940f5301eddc4b388869f5728
-  data.tar.gz: '08d691a8d45ec1d1cb4162b6b8e874a3bd1bf52813c9ca9004d2439a0eaa2d67'
+  metadata.gz: 77df2cb6ea955db0392941563a8e6c43cc55ad06d429c1f8d45ea05bd79ecb50
+  data.tar.gz: 1ea2fbdc554c5901638002e7d4c16cbaf5c9af7189434b06fb1003eef2894d14
 SHA512:
-  metadata.gz: 0c3b6732b01c0a6014bf6bd420ffd59e8f4093be2c253194d74accf67b57545d5655e5ac70856c18c4dd4c5697aed8e79e30282af00954fd9e482bc04d65d19f
-  data.tar.gz: 64dcd5ac758a50d502fc86d99809a7df1ddb64a19c34251282fc12ccf33ea657011cf86d51c35c1b86e59efd699205ef93e7bad97bf5fc01d1523609559dd50e
+  metadata.gz: '0197dc20bc43a0863883f0a723a4120844c05bbc1542a42509fa7cb98473414f860d034a65cdf8e645a8a7aec414c6bfa347887b803d191c14745ec4dc4b2681'
+  data.tar.gz: aaa2b3835cfefed5612f5b0e93f68dd3f9985d0cd4436c1201e5d7907b2fe25c194a05249de67680b7531ce3c8aaa22bc8d95a8fa15317539755da9c6eb8f0ba

data/CHANGELOG.md

@@ -2,6 +2,29 @@
 
 All notable changes to this project will be documented in this file.
 
+## [0.6.0] - 2026-10-11
+
+### Fixed
+
+- A consumer handler raising outside `StandardError` no longer stops the channel dispatching.
+  `SystemStackError` and `ScriptError` now fail that delivery through the configured error
+  disposition. `Async::Stop`, `SystemExit`, `SignalException` and `NoMemoryError` still propagate.
+  Applies equally to the `on_handler_error` hook and to settling after an error.
+- Handler workers are created on demand instead of one per prefetch slot. `basic_qos(prefetch_count: 500)`
+  previously created 500 fibers and retired all but `pool_size` of them unused: 12.6 MB at prefetch
+  500 across two channels. Concurrency is unchanged.
+
+### Added
+
+- `on_error:` on a consumer, and `Channel#on_error_disposition` for the channel default, choose what
+  happens to a delivery whose handler raised:
+  - `:dead_letter` (default) — `reject(requeue: false)`: one attempt, then the dead-letter route.
+  - `:retry` — `reject(requeue: true)`: counts towards `x-delivery-limit`.
+  - `:release` — `nack(requeue: true)`: requeued without counting towards the limit.
+  - `:leave` — settles nothing; the handler is responsible.
+- `Channel#on_handler_error` receives `(error, delivery_info, queue_name)` and may return a
+  disposition to override the default for that delivery.
+
 ## [0.5.0] - 2026-10-09
 
 ### Fixed

data/README.md

@@ -3,6 +3,23 @@
 A fiber-native RabbitMQ (AMQP 0-9-1) client for Ruby, built on the amazing work
 of [Bunny](https://github.com/ruby-amqp/bunny).
 
+## Installing
+
+```ruby
+# Gemfile
+gem "async-rabbitmq", "~> 0.5"
+```
+
+or
+
+```bash
+gem install async-rabbitmq
+```
+
+Requires Ruby 3.4+ and RabbitMQ 3.13+ (tested against 4.x). The gem also
+installs an `async-rabbitmq` command for publishing, consuming, inspecting and
+purging from a terminal.
+
 ## Bunny is the gold standard
 
 Bunny is the client Ruby learned RabbitMQ on. Its API is the one Ruby developers
@@ -221,17 +238,98 @@ sent and `false` when the tag belonged to an earlier generation and was
 dropped. A dropped ack is not a lost message: the broker requeued it when the
 channel went, and it is redelivered on the new connection.
 
-When a handler raises, the delivery is nacked with `requeue: false` — so a
-message that always fails dead-letters instead of looping — and
-`on_handler_error` is called. A delivery the handler already settled itself is
-left alone, so acking and then raising in whatever follows is safe:
+### When a handler raises
+
+The delivery is settled for you. What happens to it is the *disposition*, and
+there are three:
+
+| | what it does | when |
+|---|---|---|
+| `:dead_letter` (default) | `reject(requeue: false)` | one attempt, then the queue's dead-letter route. Safe on any queue. |
+| `:retry` | `reject(requeue: true)` | the message failed — hand it back and let the broker count it. See below. |
+| `:release` | `nack(requeue: true)` | *not* the message's fault — hand it back without counting it. |
+| `:leave` | nothing | you settle it yourself. This is Bunny's behaviour. |
+
+`:retry` and `:release` both return the message; the difference is blame.
+RabbitMQ counts a delivery as failed only on `basic.reject` (AMQP 1.0
+`modified` with `delivery-failed`), not on `basic.nack` (AMQP 1.0 `released`).
+So use `:retry` for a message that cannot be processed, and `:release` when the
+message is innocent and you simply cannot handle it now — shutting down,
+shedding load, a downstream outage. Rejecting during a gateway outage would
+walk your whole backlog to the dead-letter queue in `x-delivery-limit`
+attempts; releasing it costs the messages nothing.
+
+The trade is that `:release` has no backstop: nothing advances towards
+`x-delivery-limit`, so a genuinely poison message is redelivered forever. Use it
+when you know the failure is yours, not the message's.
+
+Set it per consumer, or channel-wide for all of them:
+
+```ruby
+ch.on_error_disposition = :retry                       # channel default
+ch.basic_consume("q", manual_ack: true, on_error: :leave) { |d, h, b| ... }
+```
+
+A delivery the handler already settled itself is left alone, so acking and then
+raising in whatever follows is safe.
+
+`on_handler_error` runs **before** the delivery is settled, so what it returns
+can decide this one message's fate. Return a disposition to override; return
+anything else, or nothing, to take the configured default:
 
 ```ruby
 ch.on_handler_error do |error, delivery, queue_name|
   Sentry.capture_exception(error, extra: { queue: queue_name })
+  case error
+  when GatewayDown           then :release      # innocent, do not count it
+  when TransientUpstreamError then :retry       # failed, count towards the limit
+  else                            :dead_letter
+  end
 end
 ```
 
+A hook that raises is logged and the configured default applies, so a broken
+hook cannot strand a delivery.
+
+### Letting the broker do the retrying
+
+`:retry` is built for quorum queues, which count failed deliveries themselves:
+
+```ruby
+ch.quorum_queue("orders.commands", arguments: {
+  "x-delivery-limit"          => 5,          # retire after 5 failures
+  "x-delayed-retry-type"      => "failed",   # back off between attempts
+  "x-delayed-retry-min"       => 1_000,
+  "x-delayed-retry-max"       => 30_000,
+  "x-dead-letter-exchange"    => "",
+  "x-dead-letter-routing-key" => "orders.commands.parked",
+})
+```
+
+The broker spaces the attempts (`min(min_delay × delivery_count, max_delay)`),
+counts them in the `x-delivery-count` header, and dead-letters the message once
+`x-delivery-limit` is reached. There is no second delay queue to declare and no
+attempt counting to write.
+
+Two things that are easy to get wrong:
+
+**It has to be `reject`, not `nack`.** RabbitMQ only treats a delivery as
+*failed* on `basic.reject`; after `basic.nack` the `x-delivery-count` does not
+move, so a nacked message is redelivered forever and never reaches the limit.
+This client always rejects, which is why `:retry` works at all. If you settle
+deliveries yourself, use `basic_reject`.
+
+**Reconnects spend the same budget.** A channel closing with the delivery
+unacknowledged — a disconnect, a recovery, a broker-side channel error — counts
+as a failed delivery too. A message can therefore be dead-lettered by three
+reconnects without your handler ever failing. Size `x-delivery-limit` for
+handler failures *plus* the reconnects you expect, and read the dead-letter
+queue as "failed or unlucky".
+
+Without `x-delivery-limit`, `:retry` is an infinite loop for a message that
+always fails. That is why the default is `:dead_letter`: it is safe on any
+queue, and a default should not depend on configuration the client cannot see.
+
 ### RabbitMQ 4.2+ and transient queues
 
 RabbitMQ 4.2 and later refuse to declare a queue that is both non-durable

data/lib/async_rabbitmq/channel.rb

@@ -9,6 +9,51 @@ module AsyncRabbitMQ
   # Duck-typed stream interface: #each (yields deliveries) and #write (publishes).
   # Does NOT inherit Async::IO::Stream — a channel is logical, not physical IO.
   class Channel
+    # What to do with a delivery whose handler raised. Chosen per consumer, or
+    # returned by the on_handler_error hook to decide per message.
+    #
+    # :dead_letter  reject(requeue: false) -- one attempt, then the queue's
+    #               dead-letter route. Safe on any queue, so it is the default:
+    #               :retry on a queue with no delivery limit loops forever, and
+    #               a default must not depend on configuration the client
+    #               cannot see.
+    # :retry        reject(requeue: true) -- hand it back and let the broker
+    #               count. A quorum queue with x-delivery-limit retires it (and
+    #               dead-letters it) once the limit is reached, with
+    #               x-delayed-retry-* spacing the attempts.
+    # :release      nack(requeue: true) -- hand it back WITHOUT counting the
+    #               failure against the message. For a failure that is not the
+    #               message's fault: shutting down, shedding load, a downstream
+    #               outage. On a poison message this loops forever, since
+    #               nothing advances towards x-delivery-limit.
+    # :leave        settle nothing. Bunny's behaviour: the delivery stays
+    #               unacknowledged and it is yours to resolve.
+    #
+    # :retry rejects and :release nacks, and the difference is the whole point.
+    # The broker counts a delivery as failed only on basic.reject (AMQP 1.0
+    # modified/delivery-failed); after basic.nack the x-delivery-count does not
+    # move (AMQP 1.0 released). So reject means "this message failed, hold it
+    # against it" and nack means "have it back, no blame". For a single
+    # delivery the verbs are otherwise equivalent.
+    ERROR_DISPOSITIONS = %i[dead_letter retry release leave].freeze
+
+    # A handler is application code and can raise anything. Catching only
+    # StandardError let a handler raising outside that hierarchy escape and take
+    # the channel's frame dispatch with it, so the catch below is deliberately
+    # as wide as Ruby allows - and these are what it has to put back.
+    #
+    # Async stops a task by raising Async::Stop into it, and Async::Stop is not
+    # a StandardError precisely so that ordinary rescues leave it alone.
+    # Swallowing it would leave handler workers unstoppable, and a channel close
+    # or a reactor shutdown would hang on a parked handler. The signals beside
+    # it are not this delivery's problem either: the process is exiting, was
+    # interrupted, or has run out of memory.
+    #
+    # SystemStackError and ScriptError are deliberately absent. An accidental
+    # infinite recursion in a handler is an ordinary bug and should fail that
+    # one delivery, not silence the channel.
+    PROPAGATE_FROM_HANDLER = [Async::Stop, SystemExit, SignalException, NoMemoryError].freeze
+
     # A message published under confirms whose fate is unknown: the broker
     # neither acked nor nacked it before the connection went. +payload+ and its
     # routing are kept so it can be published again on another connection.
@@ -77,6 +122,7 @@ module AsyncRabbitMQ
       @on_cancel         = nil
       @on_error          = nil
       @on_handler_error  = nil
+      @on_error_disposition = :dead_letter
       @resyncing         = false
       @resync_condition  = nil
       # Incremented on every reopen (connection recovery or a single-channel
@@ -426,13 +472,19 @@ module AsyncRabbitMQ
     # gated by the channel's pool_size semaphore so at most +pool_size+
     # handlers run concurrently across all consumers on this channel.
     # Returns the consumer tag.
-    def basic_consume(queue_name, consumer_tag: "", manual_ack: false, exclusive: false, arguments: {}, &block)
+    # +on_error+ decides what happens to a delivery whose handler raises; see
+    # ERROR_DISPOSITIONS. nil takes the channel's default, set with
+    # #on_error_disposition=.
+    def basic_consume(queue_name, consumer_tag: "", manual_ack: false, exclusive: false,
+                      arguments: {}, on_error: nil, &block)
+      validate_disposition!(on_error) if on_error
       warn_unbounded_prefetch(queue_name, manual_ack)
       resp = rpc(
         AMQ::Protocol::Basic::Consume.encode(@channel_id, queue_name, consumer_tag, false, !manual_ack, exclusive, false, arguments),
         AMQ::Protocol::Basic::ConsumeOk
       )
-      @consumers[resp.consumer_tag] = { queue_name: queue_name, block: block, manual_ack: manual_ack }
+      @consumers[resp.consumer_tag] = { queue_name: queue_name, block: block,
+                                        manual_ack: manual_ack, on_error: on_error }
       topology&.record_consumer(resp.consumer_tag, queue_name)
       instrument("consumer.registered") do
         { channel: @channel_id, queue: queue_name, consumer_tag: resp.consumer_tag, manual_ack: manual_ack }
@@ -605,10 +657,23 @@ module AsyncRabbitMQ
     # receives the exception, the Basic::Deliver method frame and the queue
     # name. The delivery has already been nacked (requeue: false) if the
     # consumer was registered with manual_ack: true.
+    # The block receives the exception, the Basic::Deliver frame and the queue
+    # name, and runs *before* the delivery is settled. Return one of
+    # ERROR_DISPOSITIONS to decide this message's fate; return anything else
+    # (or nothing) to take the consumer's or channel's configured default.
     def on_handler_error(&block)
       @on_handler_error = block
     end
 
+    # The channel-wide default disposition for a handler that raises, used by
+    # any consumer that did not set its own. See ERROR_DISPOSITIONS.
+    attr_reader :on_error_disposition
+
+    def on_error_disposition=(value)
+      validate_disposition!(value)
+      @on_error_disposition = value
+    end
+
     # -------------------------------------------------------------------------
     # Flow control
     # -------------------------------------------------------------------------
@@ -905,33 +970,67 @@ module AsyncRabbitMQ
     # the consumer stops receiving anything at all.
     def handle_consumer_error(error, method, entry)
       @logger.error(
-        "Channel #{@channel_id}: consumer #{method.consumer_tag} raised " \
-        "#{error.class}: #{error.message}"
+        "Channel #{@channel_id}: consumer #{method.consumer_tag} raised "         "#{error.class}: #{error.message}"
       )
       instrument("consumer.error") do
         { channel: @channel_id, consumer_tag: method.consumer_tag, queue: entry[:queue_name],
           error: error.class.name, message: error.message }
       end
 
+      # The hook runs before the delivery is settled, so what it returns can
+      # decide the disposition. Anything outside ERROR_DISPOSITIONS (including
+      # nil, and whatever a hook that was not written to choose happens to
+      # return) falls through to the configured default.
+      chosen = call_handler_error_hook(error, method, entry)
+      disposition = ERROR_DISPOSITIONS.include?(chosen) ? chosen : consumer_disposition(entry)
+      return if disposition == :leave
+
       # Only a manual-ack delivery the handler has not already settled. A
       # handler that acks and then raises in whatever follows would otherwise
-      # be nacking a tag the broker has already resolved: that is a 406, which
-      # closes the channel and takes its consumers with it.
-      if entry[:manual_ack] && unsettled?(method.delivery_tag)
-        begin
-          basic_nack(method.delivery_tag, requeue: false)
-        rescue => e
-          @logger.error("Channel #{@channel_id}: could not nack after consumer error: #{e.class}: #{e.message}")
+      # be rejecting a tag the broker has already resolved: that is a 406,
+      # which closes the channel and takes its consumers with it.
+      return unless entry[:manual_ack] && unsettled?(method.delivery_tag)
+
+      begin
+        if disposition == :release
+          # nack, not reject: released rather than failed, so the broker does
+          # not count it against the message.
+          basic_nack(method.delivery_tag, requeue: true)
+        else
+          basic_reject(method.delivery_tag, requeue: disposition == :retry)
         end
+      rescue *PROPAGATE_FROM_HANDLER
+        raise
+      rescue Exception => e # rubocop:disable Lint/RescueException
+        @logger.error("Channel #{@channel_id}: could not settle after consumer error: #{e.class}: #{e.message}")
       end
+    end
 
-      return unless @on_handler_error
+    # Returns whatever the hook returned, or nil if there is no hook or it
+    # raised. A hook that blows up must not also lose the delivery.
+    def call_handler_error_hook(error, method, entry)
+      return nil unless @on_handler_error
 
-      begin
-        @on_handler_error.call(error, method, entry[:queue_name])
-      rescue => e
-        @logger.error("Channel #{@channel_id}: on_handler_error hook raised #{e.class}: #{e.message}")
-      end
+      @on_handler_error.call(error, method, entry[:queue_name])
+    rescue *PROPAGATE_FROM_HANDLER
+      raise
+    rescue Exception => e # rubocop:disable Lint/RescueException
+      # The hook is the caller's code as much as the handler is, and it runs
+      # while a delivery is still unsettled: letting it escape would strand the
+      # message and kill the worker.
+      @logger.error("Channel #{@channel_id}: on_handler_error hook raised #{e.class}: #{e.message}")
+      nil
+    end
+
+    def consumer_disposition(entry)
+      entry[:on_error] || @on_error_disposition
+    end
+
+    def validate_disposition!(value)
+      return if ERROR_DISPOSITIONS.include?(value)
+
+      raise ArgumentError,
+            "on_error must be one of #{ERROR_DISPOSITIONS.map(&:inspect).join(', ')} (got #{value.inspect})"
     end
 
     # -------------------------------------------------------------------------
@@ -943,16 +1042,38 @@ module AsyncRabbitMQ
       @session.spawn_background { dispatch_loop }
     end
 
-    # One long-lived fiber per pool_size, each draining the work queue for the
-    # life of this channel generation. A backlogged delivery costs a 4-element
-    # array (~48 bytes) rather than a parked Async::Task and its fiber stack
-    # (~15.7 KB measured), so memory tracks the messages held, not the fibers
-    # holding them. Bunny's ConsumerWorkPool is the same shape: fixed workers
-    # draining an unbounded ::Queue.
+    # Up to pool_size long-lived fibers drain the work queue for the life of
+    # this channel generation. A backlogged delivery costs a 4-element array
+    # (~48 bytes) rather than a parked Async::Task and its fiber stack (~15.7 KB
+    # measured), so memory tracks the messages held, not the fibers holding
+    # them. Bunny's ConsumerWorkPool is the same shape: workers draining an
+    # unbounded ::Queue.
+    #
+    # The queue is made here; the workers are not. See #ensure_worker.
     def start_workers
       stop_workers(discard: true)
-      queue    = @work_queue = Async::Queue.new
-      @workers = Array.new(@pool_size) { @session.spawn_background { worker_loop(queue) } }
+      @work_queue = Async::Queue.new
+      @workers    = []
+    end
+
+    # Create a worker on demand, never up front.
+    #
+    # basic_qos couples pool_size to prefetch_count, so creating them eagerly
+    # meant a caller asking for a prefetch of 500 got 500 fibers, and a caller
+    # who then set pool_size back down - the documented escape hatch from that
+    # coupling - retired all but pool_size of them without a single delivery
+    # between them. A fiber's touched stack pages stay in the process RSS after
+    # it exits, measured at ~12 KB apiece: 12.6 MB at prefetch 500 across two
+    # channels, which is the whole of the consumer memory difference the 24-hour
+    # soaks showed.
+    #
+    # Demand never exceeds pool_size, so the concurrency bound is unchanged: a
+    # channel that genuinely runs pool_size handlers at once still gets there,
+    # one delivery at a time.
+    def ensure_worker(queue)
+      return if queue.nil? || @workers.size >= @pool_size
+
+      @workers << @session.spawn_background { worker_loop(queue) }
     end
 
     # Retire this generation's workers. The sentinel goes on the back of the
@@ -972,21 +1093,27 @@ module AsyncRabbitMQ
       @work_queue = nil
     end
 
-    # Workers are bound to the queue they were started on, never to @work_queue,
-    # so a worker outliving its generation cannot start draining the next one.
+    # Only ever shrinks. Growing is left to #ensure_worker, so raising pool_size
+    # costs nothing until there are deliveries that need the extra concurrency.
     def resize_workers
-      queue = @work_queue
-      return unless queue
-
-      delta = @pool_size - @workers.size
-      if delta.positive?
-        delta.times { @workers << @session.spawn_background { worker_loop(queue) } }
-      elsif delta.negative?
-        (-delta).times { queue.push(WORKER_STOP) }
-        @workers.pop(-delta)
-      end
+      queue  = @work_queue
+      excess = @workers.size - @pool_size
+      return if queue.nil? || !excess.positive?
+
+      excess.times { queue.push(WORKER_STOP) }
+      @workers.pop(excess)
     end
 
+    # Workers are bound to the queue they were started on, never to @work_queue,
+    # so a worker outliving its generation cannot start draining the next one.
+    #
+    # A worker spawned from dispatch_loop takes its first delivery inside
+    # dispatch_loop's own turn, because Async runs a new task immediately rather
+    # than scheduling it. That is safe: a handler that waits - on a confirm, on
+    # an RPC reply - suspends, and suspending hands control back to
+    # dispatch_loop, so the reply still arrives. The two reentrancy examples in
+    # 57_consumer_backlog_spec.rb pin that. An earlier revision yielded here
+    # first; it was removed because no test could show it preventing anything.
     def worker_loop(queue)
       loop do
         job = queue.dequeue
@@ -1000,7 +1127,9 @@ module AsyncRabbitMQ
       started = instrument_clock
       begin
         entry[:block].call(method, header, body)
-      rescue => e
+      rescue *PROPAGATE_FROM_HANDLER
+        raise
+      rescue Exception => e # rubocop:disable Lint/RescueException
         handle_consumer_error(e, method, entry)
       ensure
         if started
@@ -1019,8 +1148,20 @@ module AsyncRabbitMQ
         break if msg.nil?
         handle_message(msg)
       end
+    rescue *PROPAGATE_FROM_HANDLER
+      raise
     rescue => e
       @logger.error("Channel #{@channel_id} dispatch error: #{e.class}: #{e.message}")
+    rescue Exception => e # rubocop:disable Lint/RescueException
+      # Handlers are contained now, so nothing should reach here. If something
+      # does, this channel is about to stop delivering anything at all - no
+      # confirms, no RPC replies, no close-ok - and the one thing that must not
+      # happen is for it to go quiet without saying so.
+      @logger.error(
+        "Channel #{@channel_id} dispatch died on #{e.class}: #{e.message}. " \
+        "The channel will not deliver anything further."
+      )
+      raise
     end
 
     def handle_message(msg)
@@ -1093,7 +1234,10 @@ module AsyncRabbitMQ
           # is not hypothetical: it is rabbitmq-amqp-java-client#328, where a
           # bounded work pool parked the I/O thread in LinkedBlockingQueue#put.
           # A nil queue means the channel is going away and the delivery is moot.
-          @work_queue&.push([method, header, body, entry])
+          if (queue = @work_queue)
+            queue.push([method, header, body, entry])
+            ensure_worker(queue)
+          end
         else
           @logger.warn("Delivery on channel #{@channel_id} for unknown consumer #{method.consumer_tag}")
         end

data/lib/async_rabbitmq/version.rb

@@ -1,3 +1,3 @@
 module AsyncRabbitMQ
-  VERSION = "0.5.0"
+  VERSION = "0.6.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: async-rabbitmq
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.6.0
 platform: ruby
 authors:
 - Russell Penney