listenable 0.2.0 → 0.3.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 39c0faefb1a4b56eb6e359bc950a75bad47be727181b3fa7e751722c737a9fb3
-  data.tar.gz: 271cb3a4ccf5b357943967e9c503e8c6344be7e10e0439b7f6b0838b205fa228
+  metadata.gz: a8ebfdc90196162ce2a340f960b6c7f0f1de287c4806cf230bded917eee5bd3f
+  data.tar.gz: 8435bcb681df67ae94a66225e5f95765145cb81dfbed4b4f02362682fceadff4
 SHA512:
-  metadata.gz: 4012acf44c48b3ebbc64060fe4488f81efd61eac703f476d20447ca6270003df1ddb5b27a997c43a59474827a2c2ab0bb0e92b18226379a7e19a3325540d8830
-  data.tar.gz: a8046f1f29c00b3b2db7fa90603c5b6acf69ccf868fdd84ce7064f1964a6e112e44cd9749a67c5bd538064b72bc4112e203cfd47674918945a97d6e792c57c9e
+  metadata.gz: 80f93245b2d340672b0c9178f637f3a127230a4348e08b3099f27d2ff43cdd9b8baa77cdf6bf8492ef446c5e55666c3f2b4a16e1b073b1ac33bdd7439f5f1491
+  data.tar.gz: 69681153c86034c8a4c8563478cf6c580b97409566bc50d87537c477106d040fa6837e29b578a0b39ae37f23dbf90a9f4fdab23d7a743a4153afb65f59ee46e4

data/README.md

@@ -136,65 +136,6 @@ class UserListener
 end
 ```
 
-## Supported hooks
-| Listener hook         | Model callback        | Execution Mode |
-|-----------------------|-----------------------|----------------|
-| `on_created`          | `after_create`       | Synchronous (default) or Asynchronous with `async: true` |
-| `on_updated`          | `after_update`       | Synchronous (default) or Asynchronous with `async: true` |
-| `on_deleted`          | `after_destroy`      | Synchronous (default) or Asynchronous with `async: true` |
-
-All hooks support both synchronous and asynchronous execution modes via the `async: true` option.
-
-## Runtime Toggle
-By default, listeners are always active in development and production.
-
-You can enable/disable them dynamically at runtime using:
-
-```ruby
-Listenable.enabled = false  # disable all listeners
-Listenable.enabled = true   # re-enable listeners
-```
-
-This does not require restarting your Rails server or test suite.
-
-## RSpec/Test Integration
-You usually don’t want listeners firing in tests (e.g. sending jobs or emails).
-
-Disable them globally in your test suite:
-
-```ruby
-# spec/rails_helper.rb
-RSpec.configure do |config|
-  config.before(:suite) do
-    Listenable.enabled = false
-  end
-
-  # Enable listeners selectively
-  config.around(:each, listenable: true) do |example|
-    prev = Listenable.enabled
-    Listenable.enabled = true
-    example.run
-    Listenable.enabled = prev
-  end
-end
-```
-
-Now:
-
-```ruby
-RSpec.describe User do
-  it 'does not fire listeners by default' do
-    expect(UserListener).not_to receive(:on_created)
-    User.create!(name: 'Pedro')
-  end
-
-  it 'fires synchronous listeners when enabled', listenable: true do
-    expect(UserListener).to receive(:on_created)
-    User.create!(name: 'Pedro')
-  end
-end
-```
-
 ## ⚠️ Important: Execution Modes and Performance
 
 ### Synchronous Listeners (Default Behavior)
@@ -253,17 +194,126 @@ end
 
 **Note**: Asynchronous listeners use `Concurrent::Promises` for thread-safe execution. Errors in async listeners are logged but won't affect the main request flow.
 
-### Choosing the Right Mode
+#### Database Connection Management
+
+Asynchronous listeners are automatically wrapped in ActiveRecord's connection pool management (`connection_pool.with_connection`) to prevent connection pool exhaustion. The record is reloaded in the async thread to ensure thread-safe database access:
+
+```ruby
+class UserListener
+  include Listenable
+
+  listen :on_updated, async: true
+
+  def self.on_updated(user)
+    # The record is automatically reloaded in this thread
+    # Safe to access attributes and associations
+    user.name  # ✅ Safe
+    user.orders.count  # ✅ Safe - proper connection management
+
+    # Heavy operations that query the database
+    UserAnalyticsService.new(user).calculate_metrics  # ✅ Safe
+  end
+end
+```
+
+**Important**: The record passed to async listeners is reloaded from the database in the async thread. If the record is deleted before the async listener executes, the listener will receive `nil` and a warning will be logged.
+
+#### Thread Pool & Bulk Operations
 
-- **Use synchronous listeners** for:
-  - Critical operations that must complete before the request finishes
-  - Simple, fast operations (logging, simple updates)
-  - Operations that need to participate in the same database transaction
+Async listeners use a **bounded thread pool** that automatically scales based on your database connection pool size:
 
-- **Use asynchronous listeners** for:
-  - Heavy computations or external API calls
-  - Non-critical operations that can fail independently
-  - Operations that don't need to complete before the response is sent
+```ruby
+# Automatically configured based on your connection pool
+Listenable.async_executor  # Auto-scales intelligently
+```
+
+**Auto-Scaling Formula** (Very Conservative):
+- `max_threads`: **25% of connection pool size** (minimum 1, maximum 3)
+- `max_queue`: 10,000 tasks can be queued
+- `fallback_policy`: Falls back to synchronous execution if queue is full
+
+**Examples**:
+- Connection pool of 5 → **1 thread** for async listeners
+- Connection pool of 10 → **2 threads** for async listeners
+- Connection pool of 20 → **3 threads** for async listeners (capped at 3)
+
+**Why 25%?** Your main application needs most connections for handling requests. By using only 25% of the pool for async work, we ensure:
+- Bulk operations never exhaust the connection pool
+- Your main app always has connections available
+- Async work is processed reliably through queuing
+
+This ensures that bulk operations (updating thousands of records) don't exhaust your database connection pool:
+
+```ruby
+# This works safely even with thousands of records
+CSV.foreach('users.csv') do |row|
+  user = User.find(row['id'])
+  user.update!(name: row['name'])  # Async listeners execute without exhausting pool
+end
+```
+
+**Manual Override (if needed)**:
+
+If you want to override the auto-scaling, you can reset the executor with a custom size:
+
+```ruby
+# config/initializers/listenable.rb
+
+# Increase for larger connection pools
+Listenable.reset_async_executor!  # Reset existing executor
+# Then create a custom executor if needed
+# (Note: Auto-scaling is recommended for most cases)
+```
+
+**⚠️ Warning**: The auto-scaling is conservative by design. Only override if you have a very large connection pool (20+) and understand the implications.
+By default, listeners are always active in development and production.
+
+You can enable/disable them dynamically at runtime using:
+
+```ruby
+Listenable.enabled = false  # disable all listeners
+Listenable.enabled = true   # re-enable listeners
+```
+
+This does not require restarting your Rails server or test suite.
+
+## RSpec/Test Integration
+You usually don’t want listeners firing in tests (e.g. sending jobs or emails).
+
+Disable them globally in your test suite:
+
+```ruby
+# spec/rails_helper.rb
+RSpec.configure do |config|
+  config.before(:suite) do
+    Listenable.enabled = false
+  end
+
+  # Enable listeners selectively
+  config.around(:each, listenable: true) do |example|
+    prev = Listenable.enabled
+    Listenable.enabled = true
+    example.run
+    Listenable.enabled = prev
+  end
+end
+```
+
+Now:
+
+```ruby
+RSpec.describe User do
+  it 'does not fire listeners by default' do
+    expect(UserListener).not_to receive(:on_created)
+    User.create!(name: 'Pedro')
+  end
+
+  it 'fires synchronous listeners when enabled', listenable: true do
+    expect(UserListener).to receive(:on_created)
+    User.create!(name: 'Pedro')
+  end
+end
+```
 
 ## Development
 

data/lib/listenable/concern.rb

@@ -1,17 +1,20 @@
 # frozen_string_literal: true
 
 module Listenable
+  extend ActiveSupport::Concern
+
   CALLBACK_MAP = {
     'created' => :after_create,
     'updated' => :after_update,
     'deleted' => :after_destroy
   }.freeze
 
-  def self.included(base)
-    base.extend(ClassMethods)
+  included do
+    # Register this class when Listenable is included
+    Listenable.register_listener(self)
   end
 
-  module ClassMethods
+  class_methods do
     def listen(*hooks, async: false)
       @pending_hooks ||= []
 

data/lib/listenable/railtie.rb

@@ -2,62 +2,142 @@
 
 module Listenable
   class Railtie < Rails::Railtie
+    AFTER_COMMIT_MAP = {
+      'created'  => :create,
+      'updated'  => :update,
+      'destroyed' => :destroy
+    }.freeze
+
+    # Cleanup on Rails reload to prevent memory leaks
+    config.to_prepare do
+      Listenable.cleanup!
+    end
+
+    # Graceful shutdown on Rails exit
+    config.after_initialize do
+      at_exit do
+        Listenable.shutdown_async_executor!
+      end
+    end
+
     initializer 'listenable.load' do
       Rails.application.config.to_prepare do
         # Load all listeners (recursive, supports namespaced)
-        Dir[Rails.root.join('app/listeners/**/*.rb')].each { |f| require_dependency f }
+        Dir[Rails.root.join('app/listeners/**/*.rb')].each do |file|
+          require_dependency file
+        end
 
-        # Find all listener classes
-        ObjectSpace.each_object(Class).select { |klass| klass < Listenable }.each do |listener_class|
+        Listenable.listener_classes.each do |listener_class|
           model_class_name = listener_class.name.sub('Listener', '')
           model_class      = model_class_name.safe_constantize
           next unless model_class
 
+          injected_events =
+            model_class.instance_variable_get(:@_listenable_injected_events) || []
+
           listener_class.pending_hooks.each do |hook_info|
             hook     = hook_info[:name]
             async    = hook_info[:async]
             action   = hook.sub('on_', '')
-            callback = Listenable::CALLBACK_MAP[action] or next
             method   = "on_#{action}"
             event    = "#{model_class_name.underscore}.#{action}"
 
-            # unsubscribe old subscribers
-            ActiveSupport::Notifications.notifier.listeners_for(event).each do |subscriber|
-              ActiveSupport::Notifications.unsubscribe(subscriber)
-            end
+            next unless listener_class.respond_to?(method)
 
-            injected_events = model_class.instance_variable_get(:@_listenable_injected_events) || []
+            # Inject callback once per model
             unless injected_events.include?(event)
-              model_class.send(callback) do
+              commit_action = AFTER_COMMIT_MAP[action]
+              next unless commit_action
+
+              model_class.after_commit(on: commit_action) do
                 next unless Listenable.enabled
 
-                ActiveSupport::Notifications.instrument(event, record: self)
+                ActiveSupport::Notifications.instrument(
+                  event,
+                  record_class: self.class,
+                  record_id: id
+                )
               end
+
               injected_events << event
-              model_class.instance_variable_set(:@_listenable_injected_events, injected_events)
+              model_class.instance_variable_set(
+                :@_listenable_injected_events,
+                injected_events
+              )
             end
 
-            next unless listener_class.respond_to?(method)
-
-            ActiveSupport::Notifications.subscribe(event) do |*args|
+            # Subscribe (only once per reload)
+            subscriber = ActiveSupport::Notifications.subscribe(event) do |*args|
               next unless Listenable.enabled
 
               _name, _start, _finish, _id, payload = args
-              record = payload[:record]
 
               if async
-                Concurrent::Promises.future do
-                  listener_class.public_send(method, record)
-                end.rescue do |e|
-                  Rails.logger.error("[Listenable] #{listener_class}##{method} failed: #{e.message}")
-                end
+                Railtie.handle_async_listener(
+                  listener_class,
+                  method,
+                  payload[:record_class],
+                  payload[:record_id]
+                )
               else
-                listener_class.public_send(method, record)
+                Railtie.handle_sync_listener(
+                  listener_class,
+                  method,
+                  payload[:record_class],
+                  payload[:record_id]
+                )
               end
             end
+
+            Listenable.subscribers << subscriber
           end
         end
       end
     end
+
+    class << self
+      def handle_async_listener(listener_class, method, record_class, record_id)
+        Concurrent::Promises.future_on(Listenable.async_executor) do
+          ActiveRecord::Base.connection_pool.with_connection do
+            execute_listener(listener_class, method, record_class, record_id)
+          end
+        rescue ActiveRecord::ConnectionTimeoutError => e
+          Rails.logger&.error(
+            "[Listenable] DB pool exhausted for #{listener_class}##{method}: #{e.message}"
+          )
+        rescue StandardError => e
+          log_error(listener_class, method, e)
+        end
+      end
+
+      def handle_sync_listener(listener_class, method, record_class, record_id)
+        execute_listener(listener_class, method, record_class, record_id)
+      rescue StandardError => e
+        log_error(listener_class, method, e)
+        raise
+      end
+
+      private
+
+      def execute_listener(listener_class, method, record_class, record_id)
+        record = record_class.find_by(id: record_id)
+
+        unless record
+          Rails.logger&.warn(
+            "[Listenable] #{record_class}##{record_id} not found for #{listener_class}##{method}"
+          )
+          return
+        end
+
+        listener_class.public_send(method, record)
+      end
+
+      def log_error(listener_class, method, error)
+        Rails.logger&.error(
+          "[Listenable] #{listener_class}##{method} failed: " \
+          "#{error.message}\n#{error.backtrace.first(5).join("\n")}"
+        )
+      end
+    end
   end
 end

data/lib/listenable/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Listenable
-  VERSION = '0.2.0'
+  VERSION = '0.3.1'
 end

data/lib/listenable.rb

@@ -3,6 +3,8 @@
 require 'active_support'
 require 'active_support/concern'
 require 'active_support/notifications'
+require 'concurrent'
+require 'set'
 
 require_relative 'listenable/version'
 require_relative 'listenable/concern'
@@ -11,6 +13,96 @@ require_relative 'listenable/railtie' if defined?(Rails)
 module Listenable
   mattr_accessor :enabled, default: true
 
+  # Connection pool safety configuration
+  mattr_accessor :connection_checkout_timeout, default: 5 # seconds
+  mattr_accessor :max_thread_pool_ratio, default: 0.25 # 25% of connection pool
+  mattr_accessor :max_thread_pool_size, default: 3 # absolute max threads
+
   class Error < StandardError; end
-  # Your code goes here...
+  class ConnectionPoolExhausted < Error; end
+
+  class << self
+    attr_writer :async_executor
+
+    # Registry of all listener classes (better than ObjectSpace scan)
+    def listener_classes
+      @listener_classes ||= Set.new
+    end
+
+    # Register a listener class when Listenable is included
+    def register_listener(klass)
+      listener_classes.add(klass) if klass.name && !klass.name.empty?
+    end
+
+    # Clear registry on reload
+    def clear_listeners!
+      @listener_classes = Set.new
+    end
+
+    # Track active subscribers to prevent memory leaks on reload
+    def subscribers
+      @subscribers ||= []
+    end
+
+    # Calculate a safe thread pool size based on connection pool
+    # Very conservative: use configurable ratio of pool (default 25%)
+    def default_thread_pool_size
+      return 2 unless defined?(ActiveRecord::Base)
+
+      pool_size = ActiveRecord::Base.connection_pool.size
+      # Use configured ratio of pool, but at least 1 thread, max configured limit
+      [[pool_size * max_thread_pool_ratio, 1].max, max_thread_pool_size].min.to_i
+    end
+
+    # Thread pool executor for async listeners
+    # Auto-scales to 25% of connection pool (very conservative)
+    def async_executor
+      @async_executor ||= Concurrent::ThreadPoolExecutor.new(
+        min_threads: 0,
+        max_threads: default_thread_pool_size,
+        max_queue: 10_000,
+        fallback_policy: :caller_runs,
+        idletime: 60 # Threads idle for 60s are cleaned up
+      )
+    end
+
+    # Cleanup all subscribers and shutdown thread pool
+    # Called on Rails reload to prevent memory leaks
+    def cleanup!
+      # Unsubscribe all tracked subscribers
+      subscribers.each do |subscriber|
+        ActiveSupport::Notifications.unsubscribe(subscriber)
+      rescue StandardError => e
+        Rails.logger&.warn("[Listenable] Failed to unsubscribe: #{e.message}")
+      end
+      @subscribers = []
+
+      # Clear listener registry for fresh reload
+      clear_listeners!
+
+      # Shutdown thread pool gracefully
+      shutdown_async_executor!
+    end
+
+    # Graceful shutdown of thread pool
+    def shutdown_async_executor!
+      return unless @async_executor
+
+      @async_executor.shutdown
+      unless @async_executor.wait_for_termination(10)
+        Rails.logger&.warn('[Listenable] Thread pool shutdown timeout, forcing kill')
+        @async_executor.kill
+      end
+      @async_executor = nil
+    end
+
+    # For testing - immediate shutdown
+    def reset_async_executor!
+      return unless @async_executor
+
+      @async_executor.shutdown
+      @async_executor.kill unless @async_executor.wait_for_termination(5)
+      @async_executor = nil
+    end
+  end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: listenable
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.1
 platform: ruby
 authors:
 - Den Meralpis
@@ -51,6 +51,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '6.0'
+- !ruby/object:Gem::Dependency
+  name: ostruct
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: rails
   requirement: !ruby/object:Gem::Requirement
@@ -118,7 +132,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.7.1
+rubygems_version: 4.0.3
 specification_version: 4
 summary: A Rails DSL for model event listeners using ActiveSupport::Notifications.
 test_files: []