flu-rails 8.0.4 → 8.0.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5fadf301a677ec0ec47d67f9eb8159aac7dfab535c268030331ce4d54c391e58
-  data.tar.gz: 90c8b9dc6c71fa9f01630b1e183114132ec1d63735cdc2cb62c5974750f3ff49
+  metadata.gz: 80a0fc1ea3123f0811eb936297c9aacc868e2b3f8e6d771a53dfd768901c6f83
+  data.tar.gz: 4764f353735e58b7e8a9183190fd104836caa15c7f99ed8df03a26bed5b21565
 SHA512:
-  metadata.gz: 2a8f50d7eb2796ce8408f3a1ca36b68d55020d8858134fc136cef7be6fcb7978c316e99298a2d394ef0f0555602c1a955b3ff497b0a72e9b1f49ddf23ec4e1b8
-  data.tar.gz: dbc5b57979a4ebe601b8df0c26822c7dfa127c44f7292b97b654aff0363d04266552a60c2bda0d0ea00b2865421e47677d4f66592bf8c9e36bf6820f1ea290e8
+  metadata.gz: cfdf59bb4782cad462e48110c90084506141bb4d5b16ee39db055feafc5f925eb3cf705bd772038d1fb67bc4c5f9e40841d5a82959362c3c999adad3951d7175
+  data.tar.gz: 0f888571495893ff2e503b10133364bb5fe556aaf0f48105ab6a4c106b11234cbc5fbb5c59002224542674f73976da593b535d5e90a3485a3558e3fc95e9f9d1

data/CHANGELOG.md

@@ -5,6 +5,72 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+### [8.0.6] - 2026-08-19
+
+**Fixed**
+
+* Publish the events of a transaction from its commit rather than from each record's `after_commit`, which used to lose events two separate ways: Rails runs the transactional callbacks of a row on one single instance of it and skips the others (which one it picks is what `run_commit_callbacks_on_first_saved_instances_in_transaction` decides, and neither value was safe), and it skips every callback still queued as soon as one of them raises. Events now go out from the commit itself, which nothing can skip, once per recorded change and in the order the records were saved. A non-joinable transaction, such as the one `use_transactional_tests` wraps an example in, is never waited on, so a test suite sees what production does.
+* Keep an event whose publication failed for another attempt rather than giving up on it there and then. It waits in memory, per thread, up to `max_pending_events`, and is published by the next transaction to commit on that thread or at the end of the request or the job, whichever comes first. Nothing is attempted while the publisher reports itself unreachable, and an event still refused after three attempts, or pushed out of a full buffer, is handed to `on_publication_failure`.
+* Build and publish every change on its own, so an event that cannot be built or cannot reach the broker no longer costs the events queued behind it. A failed publication does not fail the transaction it belongs to either -- it has already committed by then.
+* Reconnect to RabbitMQ after a fork. A forked child inherited the parent's `Bunny` connection and published on it, which the broker then ended for both. `EventPublisher` now remembers the pid it connected under, caches channels per process as well as per thread, and drops an inherited connection instead of closing it.
+* Serialize `connect` and `disconnect` on a mutex. Two threads reaching `connect` together both opened a connection, the second overwriting the first, which stayed open and unreachable.
+
+**Added**
+
+* `Flu::ConnectionLostError`, raised by `publish` while the connection to RabbitMQ is down.
+* `on_publication_failure`, called with the event and the error when an event cannot be published, so that an application can keep it rather than read about it in the logs. The event is `nil` when it could not even be built.
+* `max_pending_events` (default `1000`), how many events a thread keeps waiting for the broker to be reachable again before the oldest are handed to `on_publication_failure`.
+
+**Changed**
+
+* Publishing while the connection is down now raises `Flu::ConnectionLostError` instead of the bare `RuntimeError` `Bunny::Session#create_channel` raises ("this connection is not open"), which nothing could tell apart from any other `RuntimeError`. It still fails immediately rather than waiting: Bunny reopens the connection in the background, and publishing works again once it has.
+
+### [8.0.5] - 2026-08-03
+
+**Fixed**
+
+* `QueueRepository#find_all` no longer lists queues across every vhost on the broker. It called `list_queues` without a vhost, which lists every vhost's queues -- on a broker shared with other applications, that meant every other application's queues too, not just this one's.
+* Sanitize the event name before building the routing key.
+* [Breaking change?] Publish only Flu's own events from `flu_publish_events!`. It called `run_callbacks(:commit)`, which runs every `after_commit` callback registered on the record (including the host application's own (mailers, jobs, cache invalidation)) rather than just the one `track_entity_changes` installs. It now calls `flu_commit_changes` directly, the same way the real `after_commit` callback does.
+* Guard the railtie require on `Rails::Railtie` rather than on `Rails`. Gems such as `rails-html-sanitizer` define an empty `Rails` namespace, which `is_testing_environment?` already accounted for a few lines below -- the railtie require itself did not, and raised `NameError: uninitialized constant Rails::Railtie` in that case.
+* Stop assuming `ActionDispatch` is loaded when serializing an event. `Event#to_json` referenced `ActionDispatch::Http::UploadedFile` unconditionally to special-case uploaded files, on every value it serialized. A plain script that builds and serializes an event without the rest of Rails loaded raised `NameError: uninitialized constant Flu::Event::ActionDispatch` on the very first field.
+* Track Single Table Inheritance subclasses. ActiveRecord hands the callbacks registered by `track_entity_changes` down to subclasses, but the class-level instance variables holding their settings were not inherited, so every save of an STI subclass of a tracked model raised `NoMethodError` on `nil`. `flu_is_tracked`, `flu_user_metadata_lambdas`, `flu_ignored_model_changes` and `flu_overriden_emitter_lambda` are now declared with `class_attribute`, which a subclass inherits until it sets its own value.
+* Always clear the per-request tracking state, even when an action raises. The request id and the entity metadata are stored in thread-local storage and were cleared from an `after_action`, which does not run when the action raises. Since application servers reuse their threads, a failed request left its id behind for the next request served by that same thread, whose entity changes were then attributed to a request that had already died. `track_requests` now installs a single `around_action` that clears both from an `ensure` block.
+
+**Added**
+
+* `rabbitmq_vhost` configuration option (default `"/"`), used by both `EventPublisher`'s AMQP connection and `QueueRepository`'s management API calls, to operate on a vhost other than the default.
+* `Flu::Error`, the base class of every error this gem raises on its own, and `Flu::NotConnectedError`.
+
+**Changed**
+
+* `EventFactory` no longer serializes an event to JSON before logging it, unless debug logging is actually enabled.
+* Publishing through a publisher that holds no connection now raises `Flu::NotConnectedError`, naming what to call and the option that turns the automatic connection off, instead of a `NoMethodError` on `nil`.
+* A `request` event is now built once every other `after_action` has run, so its `duration` covers them too. An action that raises still publishes nothing.
+* `EventPublisher` opens one Bunny channel per thread instead of sharing a single one.
+* `EventPublisher#connected?` now reports on the connection alone. The exchange is declared per thread, so it is no longer a single object whose absence says anything about the publisher as a whole.
+
+**Performance**
+
+* Optimize `deep_camelize` to use `each_with_object` and `ActiveSupport::Inflector#camelize` to improve performance and reduce memory consumption.
+* Add `# frozen_string_literal: true` to all ruby files to reduce string allocations.
+
+**Tests**
+
+* Cover `rabbitmq_vhost` against a real broker: `EventPublisher#connect` fails rather than silently falling back to the default vhost when it points at one that does not exist, and `QueueRepository#find_all` scoped to a dedicated vhost does not see a queue declared on another one -- proving the cross-vhost leak above was real.
+* Cover `duration`, which had no test at all: it is the elapsed wall-clock time since `request_start_time`, and a real controller request emits one that is a small, non-negative `Float`.
+* Cover `Event#to_routing_key`: dots are stripped from the name, spaces are left untouched, a name that would push the routing key past 255 characters is truncated with a warning through `Flu.logger`, and none of this raises when `Flu.logger` was never set. Cover the same against a real broker: a name with dots is delivered under a single routing-key segment, and a 300-character name that used to make Bunny raise is published and delivered instead.
+* Cover that `flu_publish_events!` does not run a host application's own `after_commit` callback, registered on the record's singleton class, exactly the way `run_callbacks(:commit)` used to run it too.
+* Cover loading `flu-rails` with an empty `Rails` namespace defined, through the same kind of subprocess as the `ActionDispatch` example above.
+* Cover that building an entity change, request or manual event does not serialize it when the logger's level is above `DEBUG`.
+* Cover `Event#to_json` without `ActionDispatch` loaded, through a subprocess that never requires actionpack -- the main suite always has it loaded, so this is the only honest way to exercise the guard.
+* Cover which channel a thread publishes on, against a real broker: each thread gets its own, reuses it across publications, and picks up a fresh one after the connection was closed under it.
+* Cover `entity_metadata`, which had no test at all: its lambda is evaluated before the action runs, and its result is cleared afterwards.
+* Cover what a raising action leaves behind in the thread-local tracking state.
+* Add `simplecov` as a development dependency to track and report code coverage.
+* Add support for the `RABBITMQ_VERSION` environment variable in `docker-compose.yml` to allow testing locally against different versions of RabbitMQ.
+* Configure GitHub Actions CI to run the test suite against a matrix of RabbitMQ versions (3.12, 3.13, and 4.0).
+
 ### [8.0.4] - 2026-08-01
 
 **Security**

data/README.md

@@ -16,7 +16,7 @@ For now, events are generated from:
 Add the gem to your project's Gemfile:
 
   ```ruby
-  gem "flu-rails"
+  gem "flu-rails", "8.0.6"
   ```
 
 Then, create an initializer into your Rails app (`config/initializers/flu-rails.rb`)
@@ -43,6 +43,7 @@ Each configuration is detailed below.
 * RabbitMQ: no specific version is required by `flu-rails` itself. The broker is reached through
   `bunny` 3.x and `rabbitmq_http_api_client` 3.x, which target the
   [currently supported RabbitMQ release series](https://www.rabbitmq.com/release-information).
+  It is actively tested against RabbitMQ 3.12, 3.13, and 4.x.
 
 ## Usage
 
@@ -167,6 +168,8 @@ All options have a default value. However, all of them can be changed in your in
 | `default_ignored_request_params` | `[:password, :password_confirmation, :controller, :action]` | Boolean | Optional | By default, all these parameters will be ignored from controller request's `params` when creating an event. Independently of this option, any parameter your Rails application already masks through `config.filter_parameters` (passwords, tokens,...) is replaced with `"[FILTERED]"` in the event too, including inside nested params. | `false` |
 | `application_name` | `Rails.application.class.module_parent_name`, resolved on startup | String | Required | Is used as `emitter` for each event created by `flu-rails`, if not overriden by the `track_met`. | `my_app` |
 | `bunny_options` | `{}` | Hash of symbols | Optional | Additional options to add when connecting the RabbitMQ broker. This overrides the existing options with the same name. | `{ verify_peer: true }` |
+| `max_pending_events` | `1000` | Integer | Optional | An event the broker refused waits for the connection to be back, and is published by the next commit of the thread or at the end of the request. This is how many a thread keeps waiting before handing the oldest to `on_publication_failure`. They are held in memory: a process that dies takes them with it. | `5000` |
+| `on_publication_failure` | `nil` | Lambda | Optional | Called with the event and the error when an event cannot be published, instead of logging it. The transaction the event belongs to has already committed by then, so this is the last chance to keep it: store it and publish it again later. The event is `nil` when it could not even be built. | `lambda { |event, error| OutboxEvent.create!(payload: event&.to_json, error: error.message) }` |
 
 ## How to execute tests
 
@@ -187,6 +190,14 @@ your machine, start a broker first:
   $ bundle exec rspec
 ```
 
+You can optionally specify the RabbitMQ version to use by providing the `RABBITMQ_VERSION` environment variable (defaults to `4-management-alpine`):
+
+```
+  $ RABBITMQ_VERSION=3.13-management-alpine docker compose up -d rabbitmq
+```
+
+Our continuous integration pipeline (via GitHub Actions) systematically runs the integration test suite against a matrix of RabbitMQ versions (3.12, 3.13, and 4.x) to ensure seamless backward and forward compatibility.
+
 Without one, those examples are reported as skipped and the rest of the suite still runs. Set
 `FLU_REQUIRE_RABBITMQ` to turn that skip into a failure — this is what the CI workflow does, so that
 a build cannot report success without having reached RabbitMQ:
@@ -304,8 +315,8 @@ scoped RubyGems credential.
 3. Tag the commit and push the tag:
 
 ```
-  $ git tag -a v8.0.4 -m "Version 8.0.4"
-  $ git push origin v8.0.4
+  $ git tag -a v8.0.6 -m "Version 8.0.6"
+  $ git push origin v8.0.6
 ```
 
 The workflow then checks that the tag matches `Flu::VERSION`, runs the tests, builds the gem

data/flu-rails.gemspec

@@ -35,5 +35,6 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "ostruct",      "~> 0.6"
   spec.add_development_dependency "rake",         ">= 13.0"
   spec.add_development_dependency "rspec",        "~> 3.13"
+  spec.add_development_dependency "simplecov"
   spec.add_development_dependency "sqlite3",      "~> 2.9"
 end

data/lib/flu-rails/action_controller_extender.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 require "active_support/core_ext/string/inflections"
 require "active_support/core_ext/time/zones"
 require "random/formatter"
@@ -69,19 +71,17 @@ module Flu
               end
             end
 
-            before_action do
+            prepend_around_action do |_controller, action|
               flu_define_request_id
-              @request_start_time = Time.zone.now
-            end
-            prepend_before_action do
               flu_define_request_entity_metadata
-            end
-            prepend_after_action do
-              flu_track_request
-              flu_remove_request_entity_metadata
-            end
-            after_action do
-              flu_remove_request_id
+              @request_start_time = Time.zone.now
+              begin
+                action.call
+                flu_track_request
+              ensure
+                flu_remove_request_entity_metadata
+                flu_remove_request_id
+              end
             end
           end
         end

data/lib/flu-rails/active_record_extender.rb

@@ -1,14 +1,32 @@
+# frozen_string_literal: true
+
 require "active_support/core_ext/object/try"
+require "active_support/notifications"
 
 module Flu
   class ActiveRecordExtender
     def self.extend_models(event_factory, event_publisher)
+      publish_on_transaction_commit
+
       ActiveRecord::Base.class_eval do
+        unless singleton_class.method_defined?(:flu_is_tracked)
+          class_attribute :flu_is_tracked,               instance_accessor: false, default: false
+          class_attribute :flu_user_metadata_lambdas,    instance_accessor: false, default: {}.freeze
+          class_attribute :flu_ignored_model_changes,    instance_accessor: false, default: [].freeze
+          class_attribute :flu_overriden_emitter_lambda, instance_accessor: false, default: nil
+          # Held per model rather than looked up on 'Flu', so that the publication driven by the
+          # transaction reaches the very publisher the model was tracked with.
+          class_attribute :flu_event_factory,            instance_accessor: false, default: nil
+          class_attribute :flu_event_publisher,          instance_accessor: false, default: nil
+        end
+
         define_singleton_method(:track_entity_changes) do |options = {}|
           self.flu_is_tracked               = true
           self.flu_user_metadata_lambdas    = options.fetch(:user_metadata, {})
           self.flu_ignored_model_changes    = options.fetch(:ignored_model_changes, []).map(&:to_s)
           self.flu_overriden_emitter_lambda = options.fetch(:emitter, nil)
+          self.flu_event_factory            = event_factory
+          self.flu_event_publisher          = event_publisher
         
           after_create   { flu_track_entity_change(:create, saved_changes, event_factory) }
           after_update   { flu_track_entity_change(:update, saved_changes, event_factory) }
@@ -17,38 +35,6 @@ module Flu
           after_rollback { flu_rollback_changes }
         end
 
-        def self.flu_ignored_model_changes=(ignored_model_changes)
-          @flu_ignored_model_changes = ignored_model_changes
-        end
-
-        def self.flu_ignored_model_changes
-          @flu_ignored_model_changes
-        end
-
-        def self.flu_overriden_emitter_lambda=(overriden_emitter_lambda)
-          @flu_overriden_emitter_lambda = overriden_emitter_lambda
-        end
-
-        def self.flu_overriden_emitter_lambda
-          @flu_overriden_emitter_lambda
-        end
-
-        def self.flu_user_metadata_lambdas=(user_metadata_lambdas)
-          @flu_user_metadata_lambdas = user_metadata_lambdas
-        end
-
-        def self.flu_user_metadata_lambdas
-          @flu_user_metadata_lambdas
-        end
-
-        def self.flu_is_tracked=(is_tracked)
-          @flu_is_tracked = is_tracked
-        end
-
-        def self.flu_is_tracked
-          @flu_is_tracked || false
-        end
-
         def self.flu_association_columns
           @flu_association_columns ||= reflect_on_all_associations(:belongs_to).flat_map do |association|
             column_names = [association.foreign_key]
@@ -61,8 +47,16 @@ module Flu
           @flu_changes ||= []
         end
 
-        def flu_publish_events!
-          run_callbacks(:commit)
+        # Defined with 'define_method' rather than 'def', for the same reason as the tracking
+        # helpers in 'ActionControllerExtender#extend_controllers': a bare 'def' opens a new scope
+        # and would not close over 'event_factory' and 'event_publisher'. 
+        #
+        # Without them, publishing manually had no way to call 'flu_commit_changes' directly,
+        # and used 'run_callbacks(:commit)' instead, which runs *every* 'after_commit' callback on the record, 
+        # including the hoste application's own (mailers, jobs, cache invalidation), not just Flu's.
+        define_method(:flu_publish_events!) do
+          flu_commit_changes(self.class.flu_event_factory   || event_factory,
+                             self.class.flu_event_publisher || event_publisher)
         end
 
         def flu_add_manual_event(name, data)
@@ -72,24 +66,37 @@ module Flu
             data: data,
             flu_is_a_manual_event: true
           })
+          Flu::TransactionBuffer.current.record(self) if self.class.flu_is_tracked
         end
 
         def flu_changes_as_events(event_factory)
-          flu_changes.select do |data|
-            !data[:changes].try(:empty?) || data[:flu_is_a_manual_event]
-          end.map do |data|
-            if data[:flu_is_a_manual_event]
-              event_factory.build_manual_event(data[:name], data[:data])
-            else
-              event_factory.build_entity_change_event(data)
-            end
+          flu_publishable_changes.map { |change| flu_change_as_event(change, event_factory) }
+        end
+
+        def flu_publishable_changes
+          flu_changes.select do |change|
+            !change[:changes].try(:empty?) || change[:flu_is_a_manual_event]
+          end
+        end
+
+        def flu_change_as_event(change, event_factory)
+          if change[:flu_is_a_manual_event]
+            event_factory.build_manual_event(change[:name], change[:data])
+          else
+            event_factory.build_entity_change_event(change)
           end
         end
 
+        # Every change is built and published on its own: one that cannot be is reported and the next
+        # one goes out all the same, where a raise would take the whole rest of the batch with it.
         def flu_commit_changes(event_factory, event_publisher)
-          flu_changes_as_events(event_factory).each do |event|
+          flu_publishable_changes.each do |change|
+            event = flu_change_as_event(change, event_factory)
             event_publisher.publish(event)
+          rescue StandardError => error
+            Flu.publication_failed(event, error, event_publisher)
           end
+        ensure
           flu_flush_changes
         end
 
@@ -114,10 +121,49 @@ module Flu
                                                                                            self.class.flu_association_columns,
                                                                                            self.class.flu_ignored_model_changes,
                                                                                            self.class.flu_overriden_emitter_lambda)
-            flu_changes.push(data) unless data.nil?
+            return if data.nil?
+            flu_changes.push(data)
+            Flu::TransactionBuffer.current.record(self)
           end
         end
       end
     end
+
+    # The commit of the transaction, unlike the 'after_commit' callbacks that follow it, is a moment
+    # Rails cannot skip: the notification is emitted once the COMMIT is through and before the first
+    # callback runs, so no callback raising afterwards can cost anybody their events.
+    def self.publish_on_transaction_commit
+      unless @subscribed
+        @subscribed = true
+
+        ActiveSupport::Notifications.subscribe("start_transaction.active_record") do |*, payload|
+          TransactionBuffer.current.transaction_started if joinable?(payload)
+        end
+
+        ActiveSupport::Notifications.subscribe("transaction.active_record") do |*, payload|
+          next unless joinable?(payload)
+
+          buffer = TransactionBuffer.current
+          if payload[:outcome] == :commit
+            buffer.transaction_committed.each do |entity|
+              # Nothing may travel from here into the commit that is calling us.
+              entity.flu_commit_changes(entity.class.flu_event_factory, entity.class.flu_event_publisher)
+            rescue StandardError => error
+              Flu.config.logger&.error("Flu could not build the events of #{entity.class}: " \
+                                      "#{error.class}: #{error.message}")
+            end
+            Flu.retry_pending_publications
+          else
+            buffer.transaction_rolled_back.each(&:flu_rollback_changes)
+          end
+        end
+      end
+    end
+
+    # A transaction opened as non-joinable, such as the one 'use_transactional_tests' wraps an
+    # example in, is rolled back rather than committed and only its savepoints publish anything.
+    def self.joinable?(payload)
+      !payload[:transaction].equal?(ActiveRecord::Transaction::NULL_TRANSACTION)
+    end
   end
 end

data/lib/flu-rails/configuration.rb

@@ -1,9 +1,12 @@
+# frozen_string_literal: true
+
 module Flu
   class Configuration
     attr_accessor :development_environments,
                   :rejected_user_agents,
                   :logger,
                   :rabbitmq_host,
+                  :rabbitmq_vhost,
                   :rabbitmq_management_scheme,
                   :rabbitmq_management_port,
                   :rabbitmq_port,
@@ -15,6 +18,8 @@ module Flu
                   :default_ignored_model_changes,
                   :default_ignored_request_params,
                   :application_name,
-                  :bunny_options
+                  :bunny_options,
+                  :on_publication_failure,
+                  :max_pending_events
   end
 end

data/lib/flu-rails/core_ext.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 require_relative "active_record_extender"
 require_relative "action_controller_extender"
 

data/lib/flu-rails/dummy/in_memory_event_publisher.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Flu
   module Dummy
     class InMemoryEventPublisher < Flu::EventPublisher

data/lib/flu-rails/errors.rb

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+module Flu
+  class Error < StandardError
+  end
+
+  class NotConnectedError < Error
+  end
+
+  class ConnectionLostError < Error
+  end
+end

data/lib/flu-rails/event.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 require "json"
 
 module Flu
@@ -22,8 +24,11 @@ module Flu
       @data = data || {}
     end
 
+    ROUTING_KEY_MAX_LENGTH = 255
+
     def to_routing_key
-      "#{@meta[:status]}.#{@meta[:emitter]}.#{@meta[:kind]}.#{@meta[:name]}"
+      prefix = "#{@meta[:status]}.#{@meta[:emitter]}.#{@meta[:kind]}."
+      "#{prefix}#{routing_key_name(prefix.length)}"
     end
 
     def to_json(options=nil)
@@ -67,12 +72,22 @@ module Flu
 
     private
 
+    def routing_key_name(prefix_length)
+      name   = @meta[:name].to_s.delete(".")
+      budget = [ROUTING_KEY_MAX_LENGTH - prefix_length, 0].max
+      return name if name.length <= budget
+
+      Flu.logger&.warn("flu-rails: event name '#{@meta[:name]}' was truncated to #{budget} " \
+                        "characters to keep its routing key within Bunny's #{ROUTING_KEY_MAX_LENGTH}-character limit.")
+      name[0, budget]
+    end
+
     def map_complex_object(object)
       if object.is_a?(Array)
         map_array(object)
       elsif object.is_a?(Hash)
         map_hash(object)
-      elsif object.is_a?(ActionDispatch::Http::UploadedFile)
+      elsif defined?(ActionDispatch::Http::UploadedFile) && object.is_a?(ActionDispatch::Http::UploadedFile)
         map_file(object)
       else
         object

data/lib/flu-rails/event_factory.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 require "securerandom"
 require "active_support/core_ext/object/blank"
 require "active_support/core_ext/string/inflections"
@@ -20,7 +22,7 @@ module Flu
       raise ArgumentError, "data must have an controller_name" if !data.has_key?(:controller_name) || data[:controller_name].empty?
       name  = "request to #{data[:action_name]} #{data[:controller_name]}"
       event = build_event(name, :request, data)
-      @logger.debug("Track action: #{JSON.pretty_generate(event)}")
+      @logger.debug { "Track action: #{event.to_json}" }
       event
     end
 
@@ -31,7 +33,7 @@ module Flu
       raise ArgumentError, "data must have an entity_name" if !data.has_key?(:entity_name) || data[:entity_name].empty?
       name  = "#{data[:action_name]} #{data[:entity_name]}"
       event = build_event(name, :entity_change, data)
-      @logger.debug("Track change: " + JSON.pretty_generate(event))
+      @logger.debug { "Track change: #{event.to_json}" }
       event
     end
 
@@ -40,7 +42,7 @@ module Flu
       raise ArgumentError, "data must be a hash"           if !data.is_a?(Hash)
       raise ArgumentError, "name must not be nil or empty" if name.nil? || name.empty?
       event = build_event(name.to_s, :manual, data)
-      @logger.debug("Track manual: " + JSON.pretty_generate(event))
+      @logger.debug { "Track manual: #{event.to_json}" }
       event
     end
 
@@ -91,25 +93,16 @@ module Flu
     def deep_camelize(value)
       case value
       when Array
-        value.map { |v| deep_camelize v }
+        value.map { |v| deep_camelize(v) }
       when Hash
-        value.reduce({}) do |camelized_hash, (k,v)|
-          camelized_hash[camelize(sanitize(k.to_s), false)] = deep_camelize v
-          camelized_hash
+        value.each_with_object({}) do |(k, v), camelized_hash|
+          camelized_hash[sanitize(k.to_s).camelize(:lower)] = deep_camelize(v)
         end
       else
         sanitize(value)
       end
     end
 
-    def camelize(lower_case_and_underscored_word, first_letter_in_uppercase = true)
-      if first_letter_in_uppercase
-        lower_case_and_underscored_word.to_s.gsub(/\/(.?)/) { "::" + $1.upcase }.gsub(/(^|_)(.)/) { $2.upcase }
-      else
-        (lower_case_and_underscored_word[0] || "") + camelize(lower_case_and_underscored_word)[1..-1]
-      end
-    end
-
     def extract_associations_from(entity, association_columns)
       association_columns.reduce({}) do |associations, column_name|
         associations[column_name] = entity[column_name]

data/lib/flu-rails/event_publisher.rb

@@ -1,74 +1,118 @@
+# frozen_string_literal: true
+
 require "bunny"
+require_relative "errors"
 require_relative "event"
 
 
 module Flu
   class EventPublisher
+    NOT_CONNECTED_MESSAGE = "no connection to RabbitMQ: 'connect' was never called, or " \
+                            "'disconnect' was. The railtie calls it at boot unless " \
+                            "'auto_connect_to_exchange' is false."
+    CONNECTION_LOST_MESSAGE = "the connection to RabbitMQ is down. Bunny reopens it in the " \
+                              "background when 'automatically_recover' is on, and publishing " \
+                              "works again once it has."
+
     def initialize(configuration)
       @logger        = configuration.logger
       @configuration = configuration
+      @mutex         = Mutex.new
     end
 
     def publish(event, persistent=true)
       routing_key = event.to_routing_key
-      @logger.debug("Publishing event with id '#{event.id}' with routing key: #{routing_key}")
-      @exchange.publish(event.to_json, routing_key: routing_key, persistent: persistent)
-      @logger.debug("Event published.")
+      @logger.debug { "Publishing event with id '#{event.id}' with routing key: #{routing_key}" }
+      exchange.publish(event.to_json, routing_key: routing_key, persistent: persistent)
+      @logger.debug { "Event published." }
+    rescue Bunny::ConnectionClosedError
+      raise ConnectionLostError, CONNECTION_LOST_MESSAGE
     end
 
     def connect
-      unless connected?
-        connected = false
-        while !connected
-          begin
-            connect_to_exchange
-            connected = true
-          rescue Bunny::TCPConnectionFailedForAllHosts
-            @logger.warn("RabbitMQ connection failed, try again in 1 second.")
-            sleep 1
+      @mutex.synchronize do
+        unless connected?
+          connected = false
+          while !connected
+            begin
+              connect_to_exchange
+              connected = true
+            rescue Bunny::TCPConnectionFailedForAllHosts
+              @logger.warn("RabbitMQ connection failed, try again in 1 second.")
+              sleep 1
+            end
           end
         end
       end
     end
 
     def connected?
-      if @connection.nil? || @exchange.nil?
-        false
-      else
-        @connection.open?
-      end
+      !forked? && !@connection.nil? && @connection.open?
     end
 
-    # Closing the connection closes the channel opened on it, and stops the heartbeat and recovery
-    # threads Bunny runs alongside it.
-    # The guard is on the connection alone, not on 'connected?': a connection that was opened before
-    # the exchange could be declared still has to be closed.
+    # Closing the connection closes every channel opened on it, and stops the heartbeat and
+    # recovery threads Bunny runs alongside it.
+    # The guard is on the connection alone: a connection that was opened before the exchange could
+    # be declared still has to be closed.
+    # An inherited connection is dropped rather than closed: its socket is the parent's.
     def disconnect
-      if !@connection.nil? && @connection.open?
-        @connection.close 
+      @mutex.synchronize do
+        @connection.close if connected?
+        @connection = nil
+        @pid        = nil
+        Thread.current[exchange_key] = nil
       end
-      @connection = nil
-      @channel    = nil
-      @exchange   = nil
     end
 
     private
 
+    # A child inherits the parent's socket but none of the threads Bunny runs on it.
+    def forked?
+      !@pid.nil? && @pid != Process.pid
+    end
+
+    # Per process too: a channel cached before the fork still reports itself open in the child.
+    def exchange_key
+      :"flu_exchange_#{object_id}_#{Process.pid}"
+    end
+
+    # One channel per thread rather than one for the whole publisher. 
+    # Bunny serialises every publication on the channel's own mutex.
+    #
+    # No bookkeeping of the channels handed out is needed. 
+    # Closing the connection closes all of them, so a thread holding a closed channel simply opens a new one on its next publication:
+    # 'disconnect' and a reconnection are both covered without reaching into other threads.
+    # A connection that is down is reported as such rather than left to 'create_channel', which
+    # raises a bare 'RuntimeError' the caller has no way to tell from any other.
+    def exchange
+      connect if forked?
+      cached = Thread.current[exchange_key]
+      return cached if cached && cached.channel.open?
+      raise NotConnectedError, NOT_CONNECTED_MESSAGE if @connection.nil?
+      raise ConnectionLostError, CONNECTION_LOST_MESSAGE unless @connection.open?
+      Thread.current[exchange_key] = declare_exchange
+    end
+
+    def declare_exchange
+      channel = @connection.create_channel
+      channel.topic(@configuration.rabbitmq_exchange_name,
+                    durable: @configuration.rabbitmq_exchange_durable)
+    end
+
     def connect_to_exchange
       options = {
         host:     @configuration.rabbitmq_host,
+        vhost:    @configuration.rabbitmq_vhost,
         port:     @configuration.rabbitmq_port&.to_i,
         user:     @configuration.rabbitmq_user,
         password: @configuration.rabbitmq_password,
         automatically_recover: true
       }.merge(@configuration.bunny_options || {})
-      
+
       @connection = Bunny.new(options)
       @connection.start
-      @channel  = @connection.create_channel
-      @exchange = @channel.send(:topic,
-                                @configuration.rabbitmq_exchange_name,
-                                durable: @configuration.rabbitmq_exchange_durable)
+      @pid = Process.pid
+      Thread.current[exchange_key] = declare_exchange
     end
   end
 end

data/lib/flu-rails/pending_publications.rb

@@ -0,0 +1,67 @@
+# frozen_string_literal: true
+
+module Flu
+  # The events whose publication failed, kept for another attempt.
+  #
+  # A broker that closes a connection takes a few seconds to be usable again, since Bunny reopens it
+  # in the background, and an event published in that window has nowhere to go. Rather than being
+  # given up on there, it waits here for the connection to be back: the next transaction to commit on
+  # the thread comes back for it, and so does the end of the request or the job it belongs to.
+  #
+  # In memory, and per thread: what is waiting here dies with the process. An application that cannot
+  # afford to lose an event keeps it itself, from 'on_publication_failure', which is called for every
+  # event this gives up on.
+  class PendingPublications
+    MAX_ATTEMPTS = 3
+
+    def self.current
+      Thread.current[:flu_pending_publications] ||= new
+    end
+
+    def initialize
+      @entries = []
+    end
+
+    def size
+      @entries.size
+    end
+
+    def push(event, publisher, error)
+      give_up(@entries.shift) while @entries.size >= Flu.config.max_pending_events
+      @entries.push({ event: event, publisher: publisher, error: error, attempts: 1 })
+    end
+
+    # Publishes again what its publisher can reach again, keeps what it cannot, and gives up on what
+    # has been refused MAX_ATTEMPTS times, an event the broker itself rejects being no more publishable
+    # on the tenth attempt than on the first.
+    def drain
+      return if @entries.empty?
+
+      kept = []
+      @entries.each do |entry|
+        next kept.push(entry) unless reachable?(entry[:publisher])
+
+        begin
+          entry[:publisher].publish(entry[:event])
+        rescue StandardError => error
+          entry[:error]     = error
+          entry[:attempts] += 1
+          entry[:attempts] < MAX_ATTEMPTS ? kept.push(entry) : give_up(entry)
+        end
+      end
+      @entries = kept
+    end
+
+    private
+
+    def reachable?(publisher)
+      publisher.connected?
+    rescue StandardError
+      false
+    end
+
+    def give_up(entry)
+      Flu.report_publication_failure(entry[:event], entry[:error])
+    end
+  end
+end

data/lib/flu-rails/queue_repository.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 require "rabbitmq/http/client"
 
 module Flu
@@ -9,23 +11,23 @@ module Flu
     end
 
     def find_all
-      @management_client.list_queues
+      @management_client.list_queues(@configuration.rabbitmq_vhost)
     end
 
     def find_queue(name)
-      @management_client.queue_info("/", name)
+      @management_client.queue_info(@configuration.rabbitmq_vhost, name)
     end
 
     def find_bindings_for_queue(name)
-      @management_client.list_queue_bindings("/", name)
+      @management_client.list_queue_bindings(@configuration.rabbitmq_vhost, name)
     end
 
     def purge_queue(name)
-      @management_client.purge_queue("/", name)
+      @management_client.purge_queue(@configuration.rabbitmq_vhost, name)
     end
 
     def delete_queue(name)
-      @management_client.delete_queue("/", name)
+      @management_client.delete_queue(@configuration.rabbitmq_vhost, name)
     end
 
     private

data/lib/flu-rails/railtie.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Flu
   class Railtie < Rails::Railtie
     railtie_name :flu
@@ -13,5 +15,11 @@ module Flu
       Flu.init
       Flu.start
     end
+
+    # The end of a request or of a job, which runs even when it is an exception that ended it, is the
+    # last chance to publish what the broker could not take a moment earlier.
+    initializer "flu.retry_pending_publications" do |application|
+      application.executor.to_complete { Flu.retry_pending_publications }
+    end
   end
 end

data/lib/flu-rails/transaction_buffer.rb

@@ -0,0 +1,61 @@
+# frozen_string_literal: true
+
+module Flu
+  # The entities that recorded events in the transaction currently open, so that its commit can
+  # publish them.
+  #
+  # A record hands its events over from 'after_commit', and Rails skips every transactional callback
+  # still to run as soon as one of them raises: it re-commits the rest of the batch with
+  # 'should_run_callbacks: false' (ActiveRecord::ConnectionAdapters::Transaction#commit_records). One
+  # raising callback, in this gem or in the application, therefore drops the events of every record
+  # left in the queue. The commit of the transaction itself is the one moment nothing can skip, and
+  # it is where these are published instead.
+  #
+  # One buffer per thread, one mark per open transaction: what a transaction recorded is what was
+  # pushed past its mark, which is what its rollback discards and what the outermost commit hands
+  # back. A thread holding transactions open on several databases at once shares that one stack, so
+  # the events of the first to commit wait for the last.
+  class TransactionBuffer
+    def self.current
+      Thread.current[:flu_transaction_buffer] ||= new
+    end
+
+    def initialize
+      @entities = []
+      @marks    = []
+    end
+
+    # @return [Boolean] false when no transaction is open, in which case nothing will ever drain the
+    #   buffer and the entity is left to publish its own changes.
+    def record(entity)
+      if @marks.empty?
+        false
+      else
+        @entities.push(entity)
+        true
+      end
+    end
+
+    def transaction_started
+      @marks.push(@entities.size)
+    end
+
+    # @return [Array] the entities to publish: those of the outermost transaction, none otherwise,
+    #   a nested transaction leaving what it recorded to the one it is nested in.
+    def transaction_committed
+      @marks.pop
+      if @marks.empty?
+        committed = @entities
+        @entities = []
+        committed
+      else
+        []
+      end
+    end
+
+    # @return [Array] the entities whose events the rollback discards.
+    def transaction_rolled_back
+      @entities.slice!((@marks.pop || 0)..) || []
+    end
+  end
+end

data/lib/flu-rails/util.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Flu
   module Util
     class ExportService

data/lib/flu-rails/version.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Flu
-  VERSION = "8.0.4"
+  VERSION = "8.0.6"
 end

data/lib/flu-rails.rb

@@ -1,16 +1,21 @@
+# frozen_string_literal: true
+
 require "logger"
 require "json"
 require "active_support/core_ext/module/introspection"
 require_relative "flu-rails/version"
+require_relative "flu-rails/errors"
 require_relative "flu-rails/event"
 require_relative "flu-rails/event_factory"
 require_relative "flu-rails/queue_repository"
 require_relative "flu-rails/configuration"
 require_relative "flu-rails/core_ext"
+require_relative "flu-rails/transaction_buffer"
+require_relative "flu-rails/pending_publications"
 require_relative "flu-rails/event_publisher"
 require_relative "flu-rails/util"
 require_relative "flu-rails/dummy/in_memory_event_publisher"
-require_relative "flu-rails/railtie" if defined?(Rails)
+require_relative "flu-rails/railtie" if defined?(Rails::Railtie)
 
 module Flu
   def self.configure
@@ -33,6 +38,36 @@ module Flu
     @event_publisher
   end
 
+  # Keeps an event whose publication failed for another attempt, a broker being reachable again
+  # within seconds. One that could not even be built is reported at once instead.
+  def self.publication_failed(event, error, event_publisher)
+    if event.nil?
+      report_publication_failure(event, error) 
+    else
+      PendingPublications.current.push(event, event_publisher, error)
+    end
+  end
+
+  # Publishes again what the last attempt could not. Never raises: its callers are a transaction that
+  # has committed and the end of a request, neither of which is a place to fail.
+  def self.retry_pending_publications
+    PendingPublications.current.drain
+  rescue StandardError => error
+    config.logger&.error("Flu could not retry the publications it had kept: #{error.class}: #{error.message}")
+  end
+
+  # @param event [Flu::Event, nil] nil when the event could not even be built.
+  def self.report_publication_failure(event, error)
+    handler = config.on_publication_failure
+    if handler.nil?
+      subject = event.nil? ? "an event it could not build" : "the event '#{event.id}' ('#{event.name}')"
+      config.logger&.error("Flu could not publish #{subject}: #{error.class}: #{error.message}. " \
+                           "The event is lost unless 'on_publication_failure' is configured to keep it.")
+    else
+      handler.call(event, error) 
+    end
+  end
+
   def self.init
     @configuration.application_name ||= default_application_name
     raise "configuration.application_name must not be nil" if @configuration.application_name.nil?
@@ -87,6 +122,7 @@ module Flu
       config.rejected_user_agents           = []
       config.logger                         = ::Logger.new(STDOUT)
       config.rabbitmq_host                  = "localhost"
+      config.rabbitmq_vhost                 = "/"
       config.rabbitmq_port                  = 5672
       config.rabbitmq_management_scheme     = "http"
       config.rabbitmq_management_port       = 15672
@@ -99,6 +135,8 @@ module Flu
       config.default_ignored_request_params = [:password, :password_confirmation, :controller, :action]
       config.application_name               = nil
       config.bunny_options                  = {}
+      config.on_publication_failure         = nil
+      config.max_pending_events             = 1000
     end
   end
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: flu-rails
 version: !ruby/object:Gem::Version
-  version: 8.0.4
+  version: 8.0.6
 platform: ruby
 authors:
 - Loïc Vigneron
@@ -152,6 +152,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '3.13'
+- !ruby/object:Gem::Dependency
+  name: simplecov
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: sqlite3
   requirement: !ruby/object:Gem::Requirement
@@ -185,11 +199,14 @@ files:
 - lib/flu-rails/configuration.rb
 - lib/flu-rails/core_ext.rb
 - lib/flu-rails/dummy/in_memory_event_publisher.rb
+- lib/flu-rails/errors.rb
 - lib/flu-rails/event.rb
 - lib/flu-rails/event_factory.rb
 - lib/flu-rails/event_publisher.rb
+- lib/flu-rails/pending_publications.rb
 - lib/flu-rails/queue_repository.rb
 - lib/flu-rails/railtie.rb
+- lib/flu-rails/transaction_buffer.rb
 - lib/flu-rails/util.rb
 - lib/flu-rails/version.rb
 homepage: https://github.com/crepesourcing/flu-rails