flu-rails 8.0.8 → 8.0.10

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bbe2d214138e198e66d6291a86e2614950e53f14c5556ee424cc98bd1a993981
-  data.tar.gz: c4884ce66ae50205a1a5033a3264da67dbd105d5f4e6210a6afbd9619161a1c8
+  metadata.gz: 9245f314def815225067e1198b65862370df7b9c5a23ed9aaa4650325123a2d1
+  data.tar.gz: 11ec07a03357de79a783674d6782d1034b3c879d32a4f7f2321a07d9b2b88420
 SHA512:
-  metadata.gz: fbd2777d9c2e08e2a46b2af5ed1baa357fa0bd16d75bf38b69352ef20318d4aa2b6ef75892a7e5438a2f31659e9fda808b0caef7897909e1c13928923e10c395
-  data.tar.gz: bb2e43abb4c22309616ca7c1bdca175922a89732cbd259978f634e9064e1f61a39bfc8b8a7007c0ba96f96b87dd573b1f5c71253ac1784921b0a97fa423d6eba
+  metadata.gz: '08c9fa68a5162a63d1b2c3949f21675a64d59a513661e98672d1b4f35211e9b5295ef6086c13168bb18ddbe13e4e21b822800e303ad6672af9f0aa4b1c2e57d9'
+  data.tar.gz: 1707f100cf25d4d77d7462e11c6d9484cb6e5f0089f06d56e42b9a54a5b24f90e3cf2e75061090dc4ef5e5203c96eb51fb8993846bcd4a4d9e859e6f78e3c0bb

data/CHANGELOG.md

@@ -5,6 +5,22 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+### [8.0.10] - 2026-09-15
+
+**Fixed**
+
+* Keep a pending publication waiting while Bunny reopens the channels, rather than spend its attempts. Since 8.0.9, publishing while Bunny reopens the channels raises `Flu::ConnectionLostError`, and the event is kept for another attempt. But the publisher still said it was `connected?` in that window -- the connection is `open?` from its handshake on -- so every drain of the pending publications retried the event at once and charged it an attempt: one right after the failing commit, another at the next commit on the thread or at the end of the job. An event failing there had one attempt left out of three, and a thread committing twice within the few milliseconds Bunny takes to reopen the channels lost it for a transaction that had committed. `connected?` is now false while Bunny reopens the channels, as it is while the connection is down, and a pending publication waits for Bunny to be done.
+
+### [8.0.9] - 2026-09-14
+
+* Enforce gem `json < 3`
+
+**Fixed**
+
+* Close the channel of a thread that has ended. Since 8.0.5, each thread publishes on its own Bunny channel, and that channel was never closed when the thread ended. RabbitMQ allows 2047 channels per connection, so a server that keeps creating and ending threads (Puma does, when `min_threads` is lower than `max_threads`) reached that limit after about a day. From then on, every new thread failed to publish with `RuntimeError: Cannot open a channel: max number of channels on connection reached`, while the older threads kept publishing normally, which made the problem look random. The channels of the threads that have ended are now closed each time a new channel is opened.
+* Publish from a fiber on the channel of its thread. The channel was stored in `Thread.current[]`, which is per fiber, not per thread: a publication made from an `Enumerator` or a streaming response opened one more channel, and never closed it either.
+* Publish on the channel Bunny reopens rather than on a new one. After a lost connection, Bunny reopens the connection first, then the channels it had on it. A thread that published in between opened one more channel, or hung for 15 seconds when Bunny was still in the handshake, and the one Bunny was reopening stayed open on the broker with no thread to use it. Publishing in between now raises `Flu::ConnectionLostError`, as it does while the connection is down, and goes on with the reopened channel once Bunny is done.
+
 ### [8.0.8] - 2026-08-26
 
 **Fixed**

data/README.md

@@ -16,7 +16,7 @@ For now, events are generated from:
 Add the gem to your project's Gemfile:
 
   ```ruby
-  gem "flu-rails", "8.0.8"
+  gem "flu-rails", "8.0.10"
   ```
 
 Then, create an initializer into your Rails app (`config/initializers/flu-rails.rb`)
@@ -53,6 +53,9 @@ Each configuration is detailed below.
 Its startup waits for its RabbitMQ exchange to be connected, for at most `max_connect_wait` seconds. A broker that is still not there by then does not keep the application from booting:
 publishing reopens the connection itself once the broker answers again.
 
+It opens one connection per process, and one channel on it per thread that publishes. A fiber publishes on the channel of its thread.
+The channels of the threads that have ended are closed the next time a thread opens one, so a server that keeps creating and ending threads (Puma does, when `min_threads` is lower than `max_threads`) does not accumulate them.
+
 ### Track changes on an ActiveRecord model
 
 All subclasses of `ActiveRecord::Base` that call `track_entity_changes` are "tracked". _E.g._:
@@ -317,8 +320,8 @@ scoped RubyGems credential.
 3. Tag the commit and push the tag:
 
 ```
-  $ git tag -a v8.0.8 -m "Version 8.0.8"
-  $ git push origin v8.0.8
+  $ git tag -a v8.0.10 -m "Version 8.0.10"
+  $ git push origin v8.0.10
 ```
 
 The workflow then checks that the tag matches `Flu::VERSION`, runs the tests, builds the gem

data/flu-rails.gemspec

@@ -28,6 +28,9 @@ Gem::Specification.new do |spec|
   spec.add_dependency "activerecord",             "~> 8.0"
   spec.add_dependency "activesupport",            "~> 8.0"
   spec.add_dependency "bunny",                    "~> 3.1"
+  # json 3 refuses the options Faraday 2.14.3 still passes to 'JSON.parse' as a hash.
+  # Fixed on Faraday's main branch, not released yet.
+  spec.add_dependency "json",                     "< 3"
   spec.add_dependency "logger",                   "~> 1.7"
   spec.add_dependency "rabbitmq_http_api_client", "~> 3.2"
 

data/lib/flu-rails/event_publisher.rb

@@ -22,6 +22,8 @@ module Flu
       @configuration   = configuration
       @mutex           = Mutex.new
       @next_attempt_at = 0
+      @exchanges       = {}
+      @exchanges_mutex = Mutex.new
     end
 
     def publish(event, persistent=true)
@@ -51,8 +53,15 @@ module Flu
       end
     end
 
+    # Not connected while Bunny is reopening the channels either, although the connection is 'open?'
+    # from its handshake on: 'exchange' refuses to publish in that window, and 'PendingPublications'
+    # asks here before it retries. A publisher that said it was connected and then refused cost the
+    # event one of its attempts on every drain -- the one right after the failing commit, then the
+    # next commit or the end of the job -- and a thread committing twice within the few milliseconds
+    # Bunny takes to reopen the channels lost the event for a committed transaction. Saying it is not
+    # connected makes the event wait for Bunny to be done, as it does while the connection is down.
     def connected?
-      !forked? && !@connection.nil? && @connection.open?
+      !forked? && !@connection.nil? && @connection.open? && !being_reopened?
     end
 
     # Closing the connection closes every channel opened on it, and stops the heartbeat and
@@ -65,7 +74,7 @@ module Flu
         @connection.close if connected?
         @connection = nil
         @pid        = nil
-        Thread.current[exchange_key] = nil
+        forget_exchanges
       end
     end
 
@@ -115,26 +124,80 @@ module Flu
       !@pid.nil? && @pid != Process.pid
     end
 
-    # Per process too: a channel cached before the fork still reports itself open in the child.
-    def exchange_key
-      :"flu_exchange_#{object_id}_#{Process.pid}"
-    end
-
-    # One channel per thread rather than one for the whole publisher. 
-    # Bunny serialises every publication on the channel's own mutex.
+    # One channel per thread, kept in '@exchanges' (thread => exchange).
+    #
+    # Why not 'Thread.current[]': it is per fiber, not per thread, and a channel stored there is
+    # never closed when the thread ends. RabbitMQ allows 2047 channels per connection, so a server
+    # that keeps creating and ending threads hit that limit after a day and could not publish
+    # from any new thread. Now, each time a channel is opened, the channels of the threads that
+    # have ended are closed.
     #
-    # No bookkeeping of the channels handed out is needed. 
-    # Closing the connection closes all of them, so a thread holding a closed channel simply opens a new one on its next publication:
-    # 'disconnect' and a reconnection are both covered without reaching into other threads.
+    # Until then, '@exchanges' still references the ended threads and what
+    # their thread-local variables hold: a weak reference would lose the channel before it is closed.
+    #
+    # A thread whose channel is closed opens a new one on its next publication.
+    # That is what makes 'disconnect' and a reconnection safe.
     # A connection that is down is reported as such rather than left to 'create_channel', which
     # raises a bare 'RuntimeError' the caller has no way to tell from any other.
+    # The connection is checked before the cached channel: Bunny marks the channels open before it
+    # announces it is done, so a thread that read its channel closed, then the announcement, would
+    # open one more.
     def exchange
       reconnect if forked? || (abandoned? && due_for_another_attempt?)
-      cached = Thread.current[exchange_key]
-      return cached if cached && cached.channel.open?
       raise NotConnectedError, NOT_CONNECTED_MESSAGE if @connection.nil?
       raise ConnectionLostError, CONNECTION_LOST_MESSAGE unless @connection.open?
-      Thread.current[exchange_key] = declare_exchange
+      raise ConnectionLostError, CONNECTION_LOST_MESSAGE if being_reopened?
+      cached = @exchanges_mutex.synchronize { @exchanges[Thread.current] }
+      return cached if cached && cached.channel.open?
+      remember_exchange(declare_exchange)
+    end
+
+    # Bunny reopens the connection first, then the channels it had on it. A channel opened in between
+    # stays open on the broker with no thread to use it, and the session does not show that window:
+    # 'open?' is true as soon as the socket is. Bunny announces each recovery attempt and its
+    # completion instead. The session is kept rather than a flag: a child process inherits the flag
+    # of a connection its parent was reopening, and opens one of its own.
+    # Bunny holds one callback of each: the ones the application gave in 'bunny_options' are called
+    # from here.
+    def hold_publishing_while_bunny_reopens(session, options)
+      started   = options[:recovery_attempt_started]
+      completed = options[:recovery_completed]
+      session.before_recovery_attempt_starts { @being_reopened = session; started&.call }
+      session.after_recovery_completed { @being_reopened = nil if @being_reopened.equal?(session); completed&.call }
+    end
+
+    def being_reopened?
+      @being_reopened&.equal?(@connection)
+    end
+
+    def remember_exchange(exchange)
+      ended = @exchanges_mutex.synchronize do
+        @exchanges[Thread.current] = exchange
+        @exchanges.keys.reject(&:alive?).map { |thread| @exchanges.delete(thread) }
+      end
+
+      unless ended.empty?
+        @logger.debug { "Closing the channels of #{ended.size} threads that have ended." }
+      end
+
+      ended.each { |orphan| close_channel(orphan.channel) }
+      exchange
+    end
+
+    # Forgets the channels without closing them.
+    # Their connection is gone: closed by 'disconnect',
+    # lost, or inherited from a parent process
+    # (closing that one would close the parent's socket).
+    def forget_exchanges
+      @exchanges_mutex.synchronize { @exchanges.clear }
+    end
+
+    # Closing may fail (the broker may have closed the channel already). That is fine: the thread
+    # is gone, and the publication in progress must not fail because of it.
+    def close_channel(channel)
+      channel.close if channel.open?
+    rescue StandardError => error
+      @logger.debug { "Could not close the channel of an ended thread: #{error.class}: #{error.message}" }
     end
 
     def declare_exchange
@@ -153,10 +216,14 @@ module Flu
         automatically_recover: true
       }.merge(@configuration.bunny_options || {})
 
+      # Before 'start': when it fails, the channels of the previous connection must be gone already,
+      # or the forking thread would publish on the parent's channel in a child that has no broker.
+      forget_exchanges
       @connection = Bunny.new(options)
+      hold_publishing_while_bunny_reopens(@connection, options)
       @connection.start
       @pid = Process.pid
-      Thread.current[exchange_key] = declare_exchange
+      remember_exchange(declare_exchange)
     end
   end
 end

data/lib/flu-rails/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Flu
-  VERSION = "8.0.8"
+  VERSION = "8.0.10"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: flu-rails
 version: !ruby/object:Gem::Version
-  version: 8.0.8
+  version: 8.0.10
 platform: ruby
 authors:
 - Loïc Vigneron
@@ -68,6 +68,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '3.1'
+- !ruby/object:Gem::Dependency
+  name: json
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3'
 - !ruby/object:Gem::Dependency
   name: logger
   requirement: !ruby/object:Gem::Requirement