listenable 0.1.1 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ebe46a89c412b54c70f0af2f7851f46176dfd08c3dc5a611ef61c35d0bea1717
-  data.tar.gz: c19ed7b192adc50833b432058cfcdf652add0401bbbca28e869c282186ef7e88
+  metadata.gz: a40496b36e34ee9be039eafd3ca5ce235d7fff2c2ccd93dfb1041bd75fa5b87f
+  data.tar.gz: 89505e376d1f337dd5e9d7172383a07501846613fcf556fa94682383b2a42d85
 SHA512:
-  metadata.gz: 6083b9a19eb74cceaec96942846f3a7b7754bf0ed27f42686f67351e8b6bd7246392c485a7b0ad4e35d909d85eb2b0f4f2cf33ebf0529ff475c9ed11b9857e68
-  data.tar.gz: 2b4c5d0b93694b35f3431f9cef48794182a621e8f937fc01bb745b895d3edbff5b38ce27c42a78c69622fecd2d2e5e1d35d5b2e171f5080dd2fd4344619c6ebd
+  metadata.gz: 7b2a55fee58e33e6f9f3fe1354d67a86668c93294e9e94274fba575ebe415805952f5e53179bf0fd041d00ab361e4c01b07390df3ac4a1f1eebd82eddf50505b
+  data.tar.gz: ef07a946e9bfc66eb038d09b765977cf44e766e2209295d2427bb7f3ad24e35d9827a3c473905d2baecb10f28814e344600ba857650e453c976622604f65074b

data/README.md

@@ -2,7 +2,7 @@
 
 Listenable is a Rails DSL that connects your ActiveRecord models to dedicated listener classes using `ActiveSupport::Notifications`.
 
-Instead of cluttering your models with callbacks, you declare listeners in `app/listeners`. Listenable automatically wires up the callbacks, instruments events, and runs your listener methods.
+Instead of cluttering your models with callbacks, you declare listeners in `app/listeners`. Listenable automatically wires up the callbacks, instruments events, and runs your listener methods. It supports both synchronous (blocking) and asynchronous (non-blocking) execution modes.
 
 ## Installation
 
@@ -18,6 +18,8 @@ If bundler is not being used to manage dependencies, install the gem by executin
 gem install listenable
 ```
 
+**Note**: For asynchronous listener support, make sure you have the `concurrent-ruby` gem installed (usually included with Rails by default).
+
 ## Usage
 
 #### 1. Define a model
@@ -36,19 +38,19 @@ class UserListener
   listen :on_created, :on_updated, :on_deleted
 
   # Handle user creation
-  def on_created(record)
+  def self.on_created(record)
     Rails.logger.info "User created: #{user.id}"
     SendWelcomeEmailJob.perform_later(user)
   end
 
   # Handle user update
-  def on_updated(record)
+  def self.on_updated(record)
     Rails.logger.info "User updated: #{user.id}"
     SendProfileUpdateNotificationJob.perform_later(user)
   end
 
   # Handle user deletion
-  def on_deleted(record)
+  def self.on_deleted(record)
     Rails.logger.info "User deleted: #{user.id}"
     ArchiveUserDataJob.perform_later(user)
   end
@@ -65,14 +67,205 @@ Under the hood:
 * `ActiveSupport::Notifications.instrument` fires events like `user.created`.
 * The Railtie subscribes your listener methods to those events.
 
-## Supported hooks
-| Listener hook         | Model callback        |
-|-----------------------|-----------------------|
-| `on_created`          | `after_create`       |
-| `on_updated`          | `after_update`       |
-| `on_deleted`          | `after_destroy`      |
+## Synchronous vs Asynchronous Execution
+
+Listenable supports both synchronous (blocking) and asynchronous (non-blocking) listener execution:
+
+### Synchronous Listeners (Default)
+By default, listeners execute synchronously in the same thread as your model operations:
+
+```ruby
+class UserListener
+  include Listenable
+
+  # Synchronous execution (default)
+  listen :on_created, :on_updated, :on_deleted
+
+  def self.on_created(user)
+    Rails.logger.info "User created: #{user.id}"
+    # This runs in the same request thread
+  end
+end
+```
+
+### Asynchronous Listeners
+For non-blocking execution, use the `async: true` option:
+
+```ruby
+class UserListener
+  include Listenable
+
+  # Asynchronous execution - runs in background thread
+  listen :on_created, :on_updated, :on_deleted, async: true
+
+  def self.on_created(user)
+    Rails.logger.info "User created: #{user.id}"
+    # This runs in a separate thread, doesn't block the request
+    SendWelcomeEmailService.call(user)  # Safe for heavier operations
+  end
+end
+```
+
+### Mixed Execution Modes
+You can mix synchronous and asynchronous listeners by calling `listen` multiple times:
+
+```ruby
+class UserListener
+  include Listenable
+
+  # Some listeners run synchronously
+  listen :on_created
+
+  # Others run asynchronously
+  listen :on_updated, :on_deleted, async: true
+
+  def self.on_created(user)
+    # Runs synchronously - blocks request
+    user.update!(status: 'active')
+  end
+
+  def self.on_updated(user)
+    # Runs asynchronously - doesn't block request
+    UserAnalyticsService.new(user).calculate_metrics
+  end
+
+  def self.on_deleted(user)
+    # Also runs asynchronously
+    CleanupUserDataService.call(user)
+  end
+end
+```
+
+## ⚠️ Important: Execution Modes and Performance
+
+### Synchronous Listeners (Default Behavior)
+
+**Synchronous listeners execute in the same thread and will block the current request.** This means that all synchronous listener methods run in the same request/transaction as your model operations, which can impact performance and response times.
+
+**For synchronous listeners**: Always queue heavy operations in background jobs to maintain application performance:
+
+```ruby
+class UserListener
+  include Listenable
+
+  # Synchronous listeners (default)
+  listen :on_created, :on_updated
+
+  def self.on_created(user)
+    # ✅ Good - Lightweight operations or queue background jobs
+    SendWelcomeEmailJob.perform_later(user)
+    NotifyAdminsJob.perform_later(user)
+  end
+
+  def self.on_updated(user)
+    # ❌ Avoid - Heavy synchronous operations that block requests
+    # UserAnalyticsService.new(user).calculate_metrics  # This blocks!
+
+    # ✅ Better - Queue in background
+    CalculateUserMetricsJob.perform_later(user)
+  end
+end
+```
+
+### Asynchronous Listeners (Non-blocking)
+
+**Asynchronous listeners execute in separate threads and don't block requests.** This allows for heavier operations without impacting response times:
+
+```ruby
+class UserListener
+  include Listenable
+
+  # Asynchronous listeners - safe for heavier operations
+  listen :on_created, :on_updated, async: true
+
+  def self.on_created(user)
+    # ✅ Safe - Runs in background thread
+    UserAnalyticsService.new(user).calculate_metrics
+    SendWelcomeEmailService.call(user)
+  end
+
+  def self.on_updated(user)
+    # ✅ Safe - Heavy operations won't block requests
+    ExternalApiService.notify_user_update(user)
+    GenerateUserReportService.call(user)
+  end
+end
+```
+
+**Note**: Asynchronous listeners use `Concurrent::Promises` for thread-safe execution. Errors in async listeners are logged but won't affect the main request flow.
+
+#### Database Connection Management
+
+Asynchronous listeners are automatically wrapped in ActiveRecord's connection pool management (`connection_pool.with_connection`) to prevent connection pool exhaustion. The record is reloaded in the async thread to ensure thread-safe database access:
+
+```ruby
+class UserListener
+  include Listenable
+
+  listen :on_updated, async: true
 
-## Runtime Toggle
+  def self.on_updated(user)
+    # The record is automatically reloaded in this thread
+    # Safe to access attributes and associations
+    user.name  # ✅ Safe
+    user.orders.count  # ✅ Safe - proper connection management
+
+    # Heavy operations that query the database
+    UserAnalyticsService.new(user).calculate_metrics  # ✅ Safe
+  end
+end
+```
+
+**Important**: The record passed to async listeners is reloaded from the database in the async thread. If the record is deleted before the async listener executes, the listener will receive `nil` and a warning will be logged.
+
+#### Thread Pool & Bulk Operations
+
+Async listeners use a **bounded thread pool** that automatically scales based on your database connection pool size:
+
+```ruby
+# Automatically configured based on your connection pool
+Listenable.async_executor  # Auto-scales intelligently
+```
+
+**Auto-Scaling Formula** (Very Conservative):
+- `max_threads`: **25% of connection pool size** (minimum 1, maximum 3)
+- `max_queue`: 10,000 tasks can be queued
+- `fallback_policy`: Falls back to synchronous execution if queue is full
+
+**Examples**:
+- Connection pool of 5 → **1 thread** for async listeners
+- Connection pool of 10 → **2 threads** for async listeners
+- Connection pool of 20 → **3 threads** for async listeners (capped at 3)
+
+**Why 25%?** Your main application needs most connections for handling requests. By using only 25% of the pool for async work, we ensure:
+- Bulk operations never exhaust the connection pool
+- Your main app always has connections available
+- Async work is processed reliably through queuing
+
+This ensures that bulk operations (updating thousands of records) don't exhaust your database connection pool:
+
+```ruby
+# This works safely even with thousands of records
+CSV.foreach('users.csv') do |row|
+  user = User.find(row['id'])
+  user.update!(name: row['name'])  # Async listeners execute without exhausting pool
+end
+```
+
+**Manual Override (if needed)**:
+
+If you want to override the auto-scaling, you can reset the executor with a custom size:
+
+```ruby
+# config/initializers/listenable.rb
+
+# Increase for larger connection pools
+Listenable.reset_async_executor!  # Reset existing executor
+# Then create a custom executor if needed
+# (Note: Auto-scaling is recommended for most cases)
+```
+
+**⚠️ Warning**: The auto-scaling is conservative by design. Only override if you have a very large connection pool (20+) and understand the implications.
 By default, listeners are always active in development and production.
 
 You can enable/disable them dynamically at runtime using:
@@ -115,14 +308,13 @@ RSpec.describe User do
     User.create!(name: 'Pedro')
   end
 
-  it 'fires listeners when enabled', listenable: true do
+  it 'fires synchronous listeners when enabled', listenable: true do
     expect(UserListener).to receive(:on_created)
     User.create!(name: 'Pedro')
   end
 end
 ```
 
-
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
@@ -131,7 +323,6 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
 
 ## Todo:
 * Create rake tasks to generate listener files.
-* RSpec tests for Railtie and integration tests.
 
 ## Contributing
 

data/Rakefile

@@ -1,11 +1,11 @@
 # frozen_string_literal: true
 
-require "bundler/gem_tasks"
-require "rspec/core/rake_task"
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
 
 RSpec::Core::RakeTask.new(:spec)
 
-require "rubocop/rake_task"
+require 'rubocop/rake_task'
 
 RuboCop::RakeTask.new
 

data/lib/listenable/concern.rb

@@ -2,9 +2,9 @@
 
 module Listenable
   CALLBACK_MAP = {
-    "created" => :after_create,
-    "updated" => :after_update,
-    "deleted" => :after_destroy
+    'created' => :after_create,
+    'updated' => :after_update,
+    'deleted' => :after_destroy
   }.freeze
 
   def self.included(base)
@@ -12,9 +12,12 @@ module Listenable
   end
 
   module ClassMethods
-    def listen(*hooks)
+    def listen(*hooks, async: false)
       @pending_hooks ||= []
-      @pending_hooks.concat(hooks.map(&:to_s))
+
+      hooks.each do |hook|
+        @pending_hooks << { name: hook.to_s, async: async }
+      end
     end
 
     def pending_hooks

data/lib/listenable/railtie.rb

@@ -2,50 +2,127 @@
 
 module Listenable
   class Railtie < Rails::Railtie
-    initializer "listenable.load" do
+    # Cleanup on Rails reload to prevent memory leaks
+    config.to_prepare do
+      Listenable.cleanup!
+    end
+
+    # Graceful shutdown on Rails exit
+    config.after_initialize do
+      at_exit do
+        Listenable.shutdown_async_executor!
+      end
+    end
+
+    initializer 'listenable.load' do
       Rails.application.config.to_prepare do
         # Load all listeners (recursive, supports namespaced)
-        Dir[Rails.root.join("app/listeners/**/*.rb")].each { |f| require_dependency f }
+        listener_files = Dir[Rails.root.join('app/listeners/**/*.rb')]
+        listener_files.each { |f| require_dependency f }
 
         # Find all listener classes
-        ObjectSpace.each_object(Class).select { |klass| klass < Listenable }.each do |listener_class|
-          model_class_name = listener_class.name.sub("Listener", "")
+        listener_classes = ObjectSpace.each_object(Class).select { |klass| klass < Listenable }
+        listener_classes.each do |listener_class|
+          model_class_name = listener_class.name.sub('Listener', '')
           model_class      = model_class_name.safe_constantize
           next unless model_class
 
-          listener_class.pending_hooks.each do |hook|
-            action   = hook.sub("on_", "")
+          listener_class.pending_hooks.each do |hook_info|
+            hook     = hook_info[:name]
+            async    = hook_info[:async]
+            action   = hook.sub('on_', '')
             callback = Listenable::CALLBACK_MAP[action] or next
             method   = "on_#{action}"
             event    = "#{model_class_name.underscore}.#{action}"
 
-            # Unsubscribe duplicates
+            # unsubscribe old subscribers
             ActiveSupport::Notifications.notifier.listeners_for(event).each do |subscriber|
               ActiveSupport::Notifications.unsubscribe(subscriber)
             end
 
-            # Inject AR callback once per model/event
             injected_events = model_class.instance_variable_get(:@_listenable_injected_events) || []
             unless injected_events.include?(event)
               model_class.send(callback) do
                 next unless Listenable.enabled
+
                 ActiveSupport::Notifications.instrument(event, record: self)
               end
               injected_events << event
               model_class.instance_variable_set(:@_listenable_injected_events, injected_events)
             end
 
-            # Subscribe listener (runtime-guarded)
-            if listener_class.respond_to?(method)
-              ActiveSupport::Notifications.subscribe(event) do |*args|
-                next unless Listenable.enabled
-                _name, _start, _finish, _id, payload = args
-                listener_class.public_send(method, payload[:record])
+            next unless listener_class.respond_to?(method)
+
+            # Subscribe and track subscriber for cleanup
+            subscriber = ActiveSupport::Notifications.subscribe(event) do |*args|
+              next unless Listenable.enabled
+
+              _name, _start, _finish, _id, payload = args
+              record = payload[:record]
+
+              if async
+                Railtie.handle_async_listener(listener_class, method, record)
+              else
+                Railtie.handle_sync_listener(listener_class, method, record)
               end
             end
+
+            # Track subscriber for cleanup on reload
+            Listenable.subscribers << subscriber
+          end
+        end
+      end
+    end
+
+    class << self
+      # Handle async listener with proper error handling and connection management
+      def handle_async_listener(listener_class, method, record)
+        # Extract minimal data to pass to thread
+        record_id = record.id
+        record_class = record.class
+
+        # Use bounded thread pool to prevent spawning unlimited threads
+        Concurrent::Promises.future_on(Listenable.async_executor) do
+          # Wrap in connection pool management to prevent connection exhaustion
+          ActiveRecord::Base.connection_pool.with_connection do
+            execute_listener(listener_class, method, record_class, record_id)
+          rescue StandardError => e
+            log_error(listener_class, method, e)
           end
+        end.rescue do |e|
+          Rails.logger&.error(
+            "[Listenable] Promise failed for #{listener_class}##{method}: #{e.message}"
+          )
         end
       end
+
+      # Handle sync listener with proper error handling
+      def handle_sync_listener(listener_class, method, record)
+        listener_class.public_send(method, record)
+      rescue StandardError => e
+        log_error(listener_class, method, e)
+        raise # Re-raise for sync listeners to maintain transaction integrity
+      end
+
+      private
+
+      def execute_listener(listener_class, method, record_class, record_id)
+        reloaded_record = record_class.find_by(id: record_id)
+
+        if reloaded_record
+          listener_class.public_send(method, reloaded_record)
+        else
+          Rails.logger&.warn(
+            "[Listenable] Record #{record_class}##{record_id} not found for #{listener_class}##{method}"
+          )
+        end
+      end
+
+      def log_error(listener_class, method, error)
+        Rails.logger&.error(
+          "[Listenable] #{listener_class}##{method} failed: #{error.message}\n#{error.backtrace.first(5).join("\n")}"
+        )
+      end
     end
   end
 end

data/lib/listenable/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Listenable
-  VERSION = "0.1.1"
+  VERSION = '0.3.0'
 end

data/lib/listenable.rb

@@ -1,16 +1,83 @@
 # frozen_string_literal: true
 
-require "active_support"
-require "active_support/concern"
-require "active_support/notifications"
+require 'active_support'
+require 'active_support/concern'
+require 'active_support/notifications'
+require 'concurrent'
 
-require_relative "listenable/version"
-require_relative "listenable/concern"
-require_relative "listenable/railtie" if defined?(Rails)
+require_relative 'listenable/version'
+require_relative 'listenable/concern'
+require_relative 'listenable/railtie' if defined?(Rails)
 
 module Listenable
   mattr_accessor :enabled, default: true
 
   class Error < StandardError; end
-  # Your code goes here...
+
+  class << self
+    attr_writer :async_executor
+
+    # Track active subscribers to prevent memory leaks on reload
+    def subscribers
+      @subscribers ||= []
+    end
+
+    # Calculate a safe thread pool size based on connection pool
+    # Very conservative: use only 1/4 of pool (min 1, max 3)
+    def default_thread_pool_size
+      return 2 unless defined?(ActiveRecord::Base)
+
+      pool_size = ActiveRecord::Base.connection_pool.size
+      # Use 25% of pool, but at least 1 thread, max 3 threads
+      [[pool_size / 4, 1].max, 3].min
+    end
+
+    # Thread pool executor for async listeners
+    # Auto-scales to 25% of connection pool (very conservative)
+    def async_executor
+      @async_executor ||= Concurrent::ThreadPoolExecutor.new(
+        min_threads: 0,
+        max_threads: default_thread_pool_size,
+        max_queue: 10_000,
+        fallback_policy: :caller_runs,
+        idletime: 60 # Threads idle for 60s are cleaned up
+      )
+    end
+
+    # Cleanup all subscribers and shutdown thread pool
+    # Called on Rails reload to prevent memory leaks
+    def cleanup!
+      # Unsubscribe all tracked subscribers
+      subscribers.each do |subscriber|
+        ActiveSupport::Notifications.unsubscribe(subscriber)
+      rescue StandardError => e
+        Rails.logger&.warn("[Listenable] Failed to unsubscribe: #{e.message}")
+      end
+      @subscribers = []
+
+      # Shutdown thread pool gracefully
+      shutdown_async_executor!
+    end
+
+    # Graceful shutdown of thread pool
+    def shutdown_async_executor!
+      return unless @async_executor
+
+      @async_executor.shutdown
+      unless @async_executor.wait_for_termination(10)
+        Rails.logger&.warn('[Listenable] Thread pool shutdown timeout, forcing kill')
+        @async_executor.kill
+      end
+      @async_executor = nil
+    end
+
+    # For testing - immediate shutdown
+    def reset_async_executor!
+      return unless @async_executor
+
+      @async_executor.shutdown
+      @async_executor.kill unless @async_executor.wait_for_termination(5)
+      @async_executor = nil
+    end
+  end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: listenable
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.3.0
 platform: ruby
 authors:
 - Den Meralpis
@@ -23,6 +23,62 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '6.0'
+- !ruby/object:Gem::Dependency
+  name: concurrent-ruby
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.1'
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+- !ruby/object:Gem::Dependency
+  name: rails
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.1'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.1'
 description: Listenable makes it easy to wire ActiveRecord models to listener classes.
   Define <Model>Listener classes in app/listeners, declare listen :on_created, :on_updated,
   etc., and Listenable automatically injects callbacks and subscribes to ActiveSupport::Notifications.
@@ -62,7 +118,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.7.1
+rubygems_version: 3.6.9
 specification_version: 4
 summary: A Rails DSL for model event listeners using ActiveSupport::Notifications.
 test_files: []