listenable 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 39c0faefb1a4b56eb6e359bc950a75bad47be727181b3fa7e751722c737a9fb3
-  data.tar.gz: 271cb3a4ccf5b357943967e9c503e8c6344be7e10e0439b7f6b0838b205fa228
+  metadata.gz: a40496b36e34ee9be039eafd3ca5ce235d7fff2c2ccd93dfb1041bd75fa5b87f
+  data.tar.gz: 89505e376d1f337dd5e9d7172383a07501846613fcf556fa94682383b2a42d85
 SHA512:
-  metadata.gz: 4012acf44c48b3ebbc64060fe4488f81efd61eac703f476d20447ca6270003df1ddb5b27a997c43a59474827a2c2ab0bb0e92b18226379a7e19a3325540d8830
-  data.tar.gz: a8046f1f29c00b3b2db7fa90603c5b6acf69ccf868fdd84ce7064f1964a6e112e44cd9749a67c5bd538064b72bc4112e203cfd47674918945a97d6e792c57c9e
+  metadata.gz: 7b2a55fee58e33e6f9f3fe1354d67a86668c93294e9e94274fba575ebe415805952f5e53179bf0fd041d00ab361e4c01b07390df3ac4a1f1eebd82eddf50505b
+  data.tar.gz: ef07a946e9bfc66eb038d09b765977cf44e766e2209295d2427bb7f3ad24e35d9827a3c473905d2baecb10f28814e344600ba857650e453c976622604f65074b

data/README.md

@@ -136,65 +136,6 @@ class UserListener
 end
 ```
 
-## Supported hooks
-| Listener hook         | Model callback        | Execution Mode |
-|-----------------------|-----------------------|----------------|
-| `on_created`          | `after_create`       | Synchronous (default) or Asynchronous with `async: true` |
-| `on_updated`          | `after_update`       | Synchronous (default) or Asynchronous with `async: true` |
-| `on_deleted`          | `after_destroy`      | Synchronous (default) or Asynchronous with `async: true` |
-
-All hooks support both synchronous and asynchronous execution modes via the `async: true` option.
-
-## Runtime Toggle
-By default, listeners are always active in development and production.
-
-You can enable/disable them dynamically at runtime using:
-
-```ruby
-Listenable.enabled = false  # disable all listeners
-Listenable.enabled = true   # re-enable listeners
-```
-
-This does not require restarting your Rails server or test suite.
-
-## RSpec/Test Integration
-You usually don’t want listeners firing in tests (e.g. sending jobs or emails).
-
-Disable them globally in your test suite:
-
-```ruby
-# spec/rails_helper.rb
-RSpec.configure do |config|
-  config.before(:suite) do
-    Listenable.enabled = false
-  end
-
-  # Enable listeners selectively
-  config.around(:each, listenable: true) do |example|
-    prev = Listenable.enabled
-    Listenable.enabled = true
-    example.run
-    Listenable.enabled = prev
-  end
-end
-```
-
-Now:
-
-```ruby
-RSpec.describe User do
-  it 'does not fire listeners by default' do
-    expect(UserListener).not_to receive(:on_created)
-    User.create!(name: 'Pedro')
-  end
-
-  it 'fires synchronous listeners when enabled', listenable: true do
-    expect(UserListener).to receive(:on_created)
-    User.create!(name: 'Pedro')
-  end
-end
-```
-
 ## ⚠️ Important: Execution Modes and Performance
 
 ### Synchronous Listeners (Default Behavior)
@@ -253,17 +194,126 @@ end
 
 **Note**: Asynchronous listeners use `Concurrent::Promises` for thread-safe execution. Errors in async listeners are logged but won't affect the main request flow.
 
-### Choosing the Right Mode
+#### Database Connection Management
+
+Asynchronous listeners are automatically wrapped in ActiveRecord's connection pool management (`connection_pool.with_connection`) to prevent connection pool exhaustion. The record is reloaded in the async thread to ensure thread-safe database access:
+
+```ruby
+class UserListener
+  include Listenable
+
+  listen :on_updated, async: true
+
+  def self.on_updated(user)
+    # The record is automatically reloaded in this thread
+    # Safe to access attributes and associations
+    user.name  # ✅ Safe
+    user.orders.count  # ✅ Safe - proper connection management
+
+    # Heavy operations that query the database
+    UserAnalyticsService.new(user).calculate_metrics  # ✅ Safe
+  end
+end
+```
+
+**Important**: The record passed to async listeners is reloaded from the database in the async thread. If the record is deleted before the async listener executes, the listener will receive `nil` and a warning will be logged.
+
+#### Thread Pool & Bulk Operations
 
-- **Use synchronous listeners** for:
-  - Critical operations that must complete before the request finishes
-  - Simple, fast operations (logging, simple updates)
-  - Operations that need to participate in the same database transaction
+Async listeners use a **bounded thread pool** that automatically scales based on your database connection pool size:
 
-- **Use asynchronous listeners** for:
-  - Heavy computations or external API calls
-  - Non-critical operations that can fail independently
-  - Operations that don't need to complete before the response is sent
+```ruby
+# Automatically configured based on your connection pool
+Listenable.async_executor  # Auto-scales intelligently
+```
+
+**Auto-Scaling Formula** (Very Conservative):
+- `max_threads`: **25% of connection pool size** (minimum 1, maximum 3)
+- `max_queue`: 10,000 tasks can be queued
+- `fallback_policy`: Falls back to synchronous execution if queue is full
+
+**Examples**:
+- Connection pool of 5 → **1 thread** for async listeners
+- Connection pool of 10 → **2 threads** for async listeners
+- Connection pool of 20 → **3 threads** for async listeners (capped at 3)
+
+**Why 25%?** Your main application needs most connections for handling requests. By using only 25% of the pool for async work, we ensure:
+- Bulk operations never exhaust the connection pool
+- Your main app always has connections available
+- Async work is processed reliably through queuing
+
+This ensures that bulk operations (updating thousands of records) don't exhaust your database connection pool:
+
+```ruby
+# This works safely even with thousands of records
+CSV.foreach('users.csv') do |row|
+  user = User.find(row['id'])
+  user.update!(name: row['name'])  # Async listeners execute without exhausting pool
+end
+```
+
+**Manual Override (if needed)**:
+
+If you want to override the auto-scaling, you can reset the executor with a custom size:
+
+```ruby
+# config/initializers/listenable.rb
+
+# Increase for larger connection pools
+Listenable.reset_async_executor!  # Reset existing executor
+# Then create a custom executor if needed
+# (Note: Auto-scaling is recommended for most cases)
+```
+
+**⚠️ Warning**: The auto-scaling is conservative by design. Only override if you have a very large connection pool (20+) and understand the implications.
+By default, listeners are always active in development and production.
+
+You can enable/disable them dynamically at runtime using:
+
+```ruby
+Listenable.enabled = false  # disable all listeners
+Listenable.enabled = true   # re-enable listeners
+```
+
+This does not require restarting your Rails server or test suite.
+
+## RSpec/Test Integration
+You usually don’t want listeners firing in tests (e.g. sending jobs or emails).
+
+Disable them globally in your test suite:
+
+```ruby
+# spec/rails_helper.rb
+RSpec.configure do |config|
+  config.before(:suite) do
+    Listenable.enabled = false
+  end
+
+  # Enable listeners selectively
+  config.around(:each, listenable: true) do |example|
+    prev = Listenable.enabled
+    Listenable.enabled = true
+    example.run
+    Listenable.enabled = prev
+  end
+end
+```
+
+Now:
+
+```ruby
+RSpec.describe User do
+  it 'does not fire listeners by default' do
+    expect(UserListener).not_to receive(:on_created)
+    User.create!(name: 'Pedro')
+  end
+
+  it 'fires synchronous listeners when enabled', listenable: true do
+    expect(UserListener).to receive(:on_created)
+    User.create!(name: 'Pedro')
+  end
+end
+```
 
 ## Development
 

data/lib/listenable/railtie.rb

@@ -2,13 +2,27 @@
 
 module Listenable
   class Railtie < Rails::Railtie
+    # Cleanup on Rails reload to prevent memory leaks
+    config.to_prepare do
+      Listenable.cleanup!
+    end
+
+    # Graceful shutdown on Rails exit
+    config.after_initialize do
+      at_exit do
+        Listenable.shutdown_async_executor!
+      end
+    end
+
     initializer 'listenable.load' do
       Rails.application.config.to_prepare do
         # Load all listeners (recursive, supports namespaced)
-        Dir[Rails.root.join('app/listeners/**/*.rb')].each { |f| require_dependency f }
+        listener_files = Dir[Rails.root.join('app/listeners/**/*.rb')]
+        listener_files.each { |f| require_dependency f }
 
         # Find all listener classes
-        ObjectSpace.each_object(Class).select { |klass| klass < Listenable }.each do |listener_class|
+        listener_classes = ObjectSpace.each_object(Class).select { |klass| klass < Listenable }
+        listener_classes.each do |listener_class|
           model_class_name = listener_class.name.sub('Listener', '')
           model_class      = model_class_name.safe_constantize
           next unless model_class
@@ -39,25 +53,76 @@ module Listenable
 
             next unless listener_class.respond_to?(method)
 
-            ActiveSupport::Notifications.subscribe(event) do |*args|
+            # Subscribe and track subscriber for cleanup
+            subscriber = ActiveSupport::Notifications.subscribe(event) do |*args|
               next unless Listenable.enabled
 
               _name, _start, _finish, _id, payload = args
               record = payload[:record]
 
               if async
-                Concurrent::Promises.future do
-                  listener_class.public_send(method, record)
-                end.rescue do |e|
-                  Rails.logger.error("[Listenable] #{listener_class}##{method} failed: #{e.message}")
-                end
+                Railtie.handle_async_listener(listener_class, method, record)
               else
-                listener_class.public_send(method, record)
+                Railtie.handle_sync_listener(listener_class, method, record)
               end
             end
+
+            # Track subscriber for cleanup on reload
+            Listenable.subscribers << subscriber
           end
         end
       end
     end
+
+    class << self
+      # Handle async listener with proper error handling and connection management
+      def handle_async_listener(listener_class, method, record)
+        # Extract minimal data to pass to thread
+        record_id = record.id
+        record_class = record.class
+
+        # Use bounded thread pool to prevent spawning unlimited threads
+        Concurrent::Promises.future_on(Listenable.async_executor) do
+          # Wrap in connection pool management to prevent connection exhaustion
+          ActiveRecord::Base.connection_pool.with_connection do
+            execute_listener(listener_class, method, record_class, record_id)
+          rescue StandardError => e
+            log_error(listener_class, method, e)
+          end
+        end.rescue do |e|
+          Rails.logger&.error(
+            "[Listenable] Promise failed for #{listener_class}##{method}: #{e.message}"
+          )
+        end
+      end
+
+      # Handle sync listener with proper error handling
+      def handle_sync_listener(listener_class, method, record)
+        listener_class.public_send(method, record)
+      rescue StandardError => e
+        log_error(listener_class, method, e)
+        raise # Re-raise for sync listeners to maintain transaction integrity
+      end
+
+      private
+
+      def execute_listener(listener_class, method, record_class, record_id)
+        reloaded_record = record_class.find_by(id: record_id)
+
+        if reloaded_record
+          listener_class.public_send(method, reloaded_record)
+        else
+          Rails.logger&.warn(
+            "[Listenable] Record #{record_class}##{record_id} not found for #{listener_class}##{method}"
+          )
+        end
+      end
+
+      def log_error(listener_class, method, error)
+        Rails.logger&.error(
+          "[Listenable] #{listener_class}##{method} failed: #{error.message}\n#{error.backtrace.first(5).join("\n")}"
+        )
+      end
+    end
   end
 end

data/lib/listenable/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Listenable
-  VERSION = '0.2.0'
+  VERSION = '0.3.0'
 end

data/lib/listenable.rb

@@ -3,6 +3,7 @@
 require 'active_support'
 require 'active_support/concern'
 require 'active_support/notifications'
+require 'concurrent'
 
 require_relative 'listenable/version'
 require_relative 'listenable/concern'
@@ -12,5 +13,71 @@ module Listenable
   mattr_accessor :enabled, default: true
 
   class Error < StandardError; end
-  # Your code goes here...
+
+  class << self
+    attr_writer :async_executor
+
+    # Track active subscribers to prevent memory leaks on reload
+    def subscribers
+      @subscribers ||= []
+    end
+
+    # Calculate a safe thread pool size based on connection pool
+    # Very conservative: use only 1/4 of pool (min 1, max 3)
+    def default_thread_pool_size
+      return 2 unless defined?(ActiveRecord::Base)
+
+      pool_size = ActiveRecord::Base.connection_pool.size
+      # Use 25% of pool, but at least 1 thread, max 3 threads
+      [[pool_size / 4, 1].max, 3].min
+    end
+
+    # Thread pool executor for async listeners
+    # Auto-scales to 25% of connection pool (very conservative)
+    def async_executor
+      @async_executor ||= Concurrent::ThreadPoolExecutor.new(
+        min_threads: 0,
+        max_threads: default_thread_pool_size,
+        max_queue: 10_000,
+        fallback_policy: :caller_runs,
+        idletime: 60 # Threads idle for 60s are cleaned up
+      )
+    end
+
+    # Cleanup all subscribers and shutdown thread pool
+    # Called on Rails reload to prevent memory leaks
+    def cleanup!
+      # Unsubscribe all tracked subscribers
+      subscribers.each do |subscriber|
+        ActiveSupport::Notifications.unsubscribe(subscriber)
+      rescue StandardError => e
+        Rails.logger&.warn("[Listenable] Failed to unsubscribe: #{e.message}")
+      end
+      @subscribers = []
+
+      # Shutdown thread pool gracefully
+      shutdown_async_executor!
+    end
+
+    # Graceful shutdown of thread pool
+    def shutdown_async_executor!
+      return unless @async_executor
+
+      @async_executor.shutdown
+      unless @async_executor.wait_for_termination(10)
+        Rails.logger&.warn('[Listenable] Thread pool shutdown timeout, forcing kill')
+        @async_executor.kill
+      end
+      @async_executor = nil
+    end
+
+    # For testing - immediate shutdown
+    def reset_async_executor!
+      return unless @async_executor
+
+      @async_executor.shutdown
+      @async_executor.kill unless @async_executor.wait_for_termination(5)
+      @async_executor = nil
+    end
+  end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: listenable
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Den Meralpis
@@ -118,7 +118,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.7.1
+rubygems_version: 3.6.9
 specification_version: 4
 summary: A Rails DSL for model event listeners using ActiveSupport::Notifications.
 test_files: []