good_job 1.1.0 → 1.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7c6d495a4455453f6f4af736d3fd31685fc2026c98e0cd564ea0f47b188cc473
-  data.tar.gz: 794137e732ed3fcebec859daa8e58a2c2c8fd283cc43c5cfc7824263a1ca93fd
+  metadata.gz: 5fbc2c6b6b2d464188d5c89aa0d1a270c753bea661cda246ac41a4a86fa9cd77
+  data.tar.gz: 5fa873aa2732881e5b42cc63bbf2bdfe8fd823f224380fdc5c7e92e665b96584
 SHA512:
-  metadata.gz: 258757117262f25f5507ceb47c20f0f43a1f40427c7a0f9c3d0dc7803f25019062a160de08e96fd5ba74ceeeec5666d1084f6dff7ca9d381b2bbb198fabb10fd
-  data.tar.gz: e2a5d09cfb57a7f2a08d59f84c3b39e3582cb5b0b76a7e6ea2da421542d5e0ebee2e25a79298c1f5d3351d58e156e43ac0770e1d9e26e0f26169620f0bfcc493
+  metadata.gz: d7278790575965381edc0c5b9ee478e4156bbfe4dc97e9304dfed5a60f6d89043bf1ff567249680ff24aa85dd6a3b1539fda519e98a19762b02d657b1d943630
+  data.tar.gz: ce1bc4d1c2d9fa480f1a7544010f58ee7d1b656fe31d77869999e265cf3d0481a9da1e95ee855a351e36f11d992fb40ac85113cf0d0082c22e4988ca32ee149c

data/CHANGELOG.md

@@ -1,6 +1,67 @@
 # Changelog
 
-## [v1.1.0](https://github.com/bensheldon/good_job/tree/v1.1.0) (2020-08-09)
+## [v1.2.0](https://github.com/bensheldon/good_job/tree/v1.2.0) (2020-08-19)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.1.4...v1.2.0)
+
+**Merged pull requests:**
+
+- Document GoodJob module [\#83](https://github.com/bensheldon/good_job/pull/83) ([bensheldon](https://github.com/bensheldon))
+
+## [v1.1.4](https://github.com/bensheldon/good_job/tree/v1.1.4) (2020-08-19)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.1.3...v1.1.4)
+
+**Implemented enhancements:**
+
+- Explicitly name threads for easier debugging [\#64](https://github.com/bensheldon/good_job/issues/64)
+- Investigate Listen/Notify as alternative to polling [\#54](https://github.com/bensheldon/good_job/issues/54)
+
+**Merged pull requests:**
+
+- Add Postgres LISTEN/NOTIFY support [\#82](https://github.com/bensheldon/good_job/pull/82) ([bensheldon](https://github.com/bensheldon))
+- Allow Schedulers to filter \#create\_thread to avoid flood of queries when running async with multiple schedulers [\#81](https://github.com/bensheldon/good_job/pull/81) ([bensheldon](https://github.com/bensheldon))
+- Fully name scheduler threadpools and thread names; refactor CLI STDOUT [\#80](https://github.com/bensheldon/good_job/pull/80) ([bensheldon](https://github.com/bensheldon))
+
+## [v1.1.3](https://github.com/bensheldon/good_job/tree/v1.1.3) (2020-08-14)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.1.2...v1.1.3)
+
+**Fixed bugs:**
+
+- Job exceptions not properly attached to good\_jobs record  [\#72](https://github.com/bensheldon/good_job/issues/72)
+
+**Merged pull requests:**
+
+- Capture errors via instrumentation from retry\_on and discard\_on [\#79](https://github.com/bensheldon/good_job/pull/79) ([bensheldon](https://github.com/bensheldon))
+- Document GoodJob::Scheduler with Yard [\#78](https://github.com/bensheldon/good_job/pull/78) ([bensheldon](https://github.com/bensheldon))
+
+## [v1.1.2](https://github.com/bensheldon/good_job/tree/v1.1.2) (2020-08-13)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.1.1...v1.1.2)
+
+**Implemented enhancements:**
+
+- Allow the omission of queue names within a scheduler [\#73](https://github.com/bensheldon/good_job/issues/73)
+
+**Merged pull requests:**
+
+- Allow named queues to be excluded with a minus [\#77](https://github.com/bensheldon/good_job/pull/77) ([bensheldon](https://github.com/bensheldon))
+
+## [v1.1.1](https://github.com/bensheldon/good_job/tree/v1.1.1) (2020-08-12)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.1.0...v1.1.1)
+
+**Implemented enhancements:**
+
+- Allow multiple schedulers within the same process. e.g. `queues=mice:2,elephants:4` [\#45](https://github.com/bensheldon/good_job/issues/45)
+
+**Merged pull requests:**
+
+- Allow instantiation of multiple schedulers via --queues [\#76](https://github.com/bensheldon/good_job/pull/76) ([bensheldon](https://github.com/bensheldon))
+- Extract options parsing to Configuration object [\#74](https://github.com/bensheldon/good_job/pull/74) ([bensheldon](https://github.com/bensheldon))
+
+## [v1.1.0](https://github.com/bensheldon/good_job/tree/v1.1.0) (2020-08-10)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.0.3...v1.1.0)
 
@@ -44,10 +105,6 @@
 
 - Fix counting of available execution threads [\#58](https://github.com/bensheldon/good_job/pull/58) ([bensheldon](https://github.com/bensheldon))
 
-**Closed issues:**
-
-- repeating/recurring jobs [\#53](https://github.com/bensheldon/good_job/issues/53)
-
 **Merged pull requests:**
 
 - Add migration generator [\#56](https://github.com/bensheldon/good_job/pull/56) ([thedanbob](https://github.com/thedanbob))

data/README.md

@@ -6,7 +6,7 @@ GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails.
 
 - **Designed for ActiveJob.** Complete support for [async, queues, delays, priorities, timeouts, and retries](https://edgeguides.rubyonrails.org/active_job_basics.html) with near-zero configuration. 
 - **Built for Rails.** Fully adopts Ruby on Rails [threading and code execution guidelines](https://guides.rubyonrails.org/threading_and_code_execution.html) with [Concurrent::Ruby](https://github.com/ruby-concurrency/concurrent-ruby). 
-- **Backed by Postgres.** Relies upon Postgres integrity and session-level Advisory Locks to provide run-once safety and stay within the limits of `schema.rb`.
+- **Backed by Postgres.** Relies upon Postgres integrity, session-level Advisory Locks to provide run-once safety and stay within the limits of `schema.rb`, and LISTEN/NOTIFY to reduce queuing latency.
 - **For most workloads.** Targets full-stack teams, economy-minded solo developers, and applications that enqueue less than 1-million jobs/day.
 
 For more of the story of GoodJob, read the [introductory blog post](https://island94.org/2020/07/introducing-goodjob-1-0).
@@ -72,17 +72,62 @@ $ bundle install
    
    Configuration options available with `help`:
 
-   ```bash
-   $ bundle exec good_job help start
+    ```bash
+    $ bundle exec good_job help start
    
-   # Usage:
-   #  good_job start
-   #
-   # Options:
-   #   [--max-threads=N]         # Maximum number of threads to use for working jobs (default: ActiveRecord::Base.connection_pool.size)
-   #   [--queues=queue1,queue2]  # Queues to work from. Separate multiple queues with commas (default: *)
-   #   [--poll-interval=N]       # Interval between polls for available jobs in seconds (default: 1)
-   ```
+    Usage:
+      good_job start
+    
+    Options:
+      [--max-threads=N]                                          # Maximum number of threads to use for working jobs (default: ActiveRecord::Base.connection_pool.size)
+      [--queues=queue1,queue2(;queue3,queue4:5;-queue1,queue2)]  # Queues to work from. Separate multiple queues with commas; exclude queues with a leading minus; separate isolated execution pools with semicolons and threads with colons (default: *)
+      [--poll-interval=N]                                        # Interval between polls for available jobs in seconds (default: 1)    
+    
+    Start job worker
+    ```
+
+1. Optimize execution to reduce congestion and execution latency. 
+
+    By default, GoodJob creates a single thread execution pool that will execute jobs from any queue. Depending on your application's workload, job types, and service level objectives, you may wish to optimize execution resources; for example, providing dedicated execution resources for transactional emails so they are not delayed by long-running batch jobs. Some options:
+
+    - Multiple execution pools within a single process:
+        
+        ```bash
+        $ bundle exec good_job --queues=transactional_messages:2;batch_processing:1;-transactional_messages,batch_processing:2;* --max-threads=5
+        ```
+      
+        This configuration will result in a single process with 4 isolated thread execution pools. Isolated execution pools are separated with a semicolon (`;`) and queue names and thread counts with a colon (`:`)
+        
+        - `transactional_messages:2`: execute jobs enqueued on `transactional_messages` with up to 2 threads.
+        - `batch_processing:1` execute jobs enqueued on `batch_processing` with a single thread.
+        - `-transactional_messages,batch_processing`: execute jobs enqueued on _any_ queue _excluding_ `transactional_messages` or `batch_processing` with up to 2 threads.
+        - `*`: execute jobs on any queue on up to 5 threads, as configured by `--max-threads=5`
+        
+        For moderate workloads, multiple isolated thread execution pools offers a good balance between congestion management and economy. 
+        
+        Configuration can be injected by environment variables too:
+        
+        ```bash
+        $ GOOD_JOB_QUEUES="transactional_messages:2;batch_processing:1;-transactional_messages,batch_processing:2;*" GOOD_JOB_MAX_THREADS=5 bundle exec good_job
+        ```
+        
+    - Multiple processes; for example, on Heroku:
+    
+        ```procfile
+        # Procfile
+        
+        # Separate dyno types
+        worker: bundle exec good_job --max-threads=5
+        transactional_worker: bundle exec good_job --queues=transactional_messages --max-threads=2
+        batch_worker: bundle exec good_job --queues=batch_processing --max-threads=1
+      
+        # Combined multi-process dyno
+        combined_worker: bundle exec good_job --max-threads=5 & bundle exec good_job --queues=transactional_messages --max-threads=2 & bundle exec good_job --queues=batch_processing --max-threads=1 & wait -n
+        ```
+      
+      Running multiple processes can optimize for CPU performance at the expense of greater memory and system resource usage.
+
+    _Keep in mind, queue operations and management is an advanced discipline. This stuff is complex, especially for heavy workloads and unique processing requirements. Good job 👍_
 
 ### Error handling, retries, and reliability
 
@@ -239,15 +284,20 @@ Depending on your application configuration, you may need to take additional ste
     # config/puma.rb
   
     before_fork do
-      GoodJob::Scheduler.instances.each { |s| s.shutdown }
+      GoodJob.shutdown
     end
     
     on_worker_boot do
-      GoodJob::Scheduler.instances.each { |s| s.restart }
+      GoodJob.restart
     end
     
     on_worker_shutdown do
-      GoodJob::Scheduler.instances.each { |s| s.shutdown }
+      GoodJob.shutdown
+    end
+  
+    MAIN_PID = Process.pid
+    at_exit do
+      GoodJob.shutdown if Process.pid == MAIN_PID
     end
     ```
   

data/exe/good_job

@@ -1,3 +1,4 @@
 #!/usr/bin/env ruby
 require 'good_job/cli'
+GOOD_JOB_LOG_TO_STDOUT = true
 GoodJob::CLI.start(ARGV)

data/lib/active_job/queue_adapters/good_job_adapter.rb

@@ -1,35 +1,9 @@
 module ActiveJob
   module QueueAdapters
     class GoodJobAdapter < GoodJob::Adapter
-      def initialize(execution_mode: nil, max_threads: nil, poll_interval: nil, scheduler: nil)
-        execution_mode = if execution_mode
-                           execution_mode
-                         elsif ENV['GOOD_JOB_EXECUTION_MODE'].present?
-                           ENV['GOOD_JOB_EXECUTION_MODE'].to_sym
-                         elsif Rails.env.development?
-                           :inline
-                         elsif Rails.env.test?
-                           :inline
-                         else
-                           :external
-                         end
-
-        if execution_mode == :async && scheduler.blank?
-          max_threads = (
-            max_threads.presence ||
-            ENV['GOOD_JOB_MAX_THREADS'] ||
-            ENV['RAILS_MAX_THREADS'] ||
-            ActiveRecord::Base.connection_pool.size
-          ).to_i
-
-          poll_interval = (
-            poll_interval.presence ||
-            ENV['GOOD_JOB_POLL_INTERVAL'] ||
-            1
-          ).to_i
-        end
-
-        super(execution_mode: execution_mode, max_threads: max_threads, poll_interval: poll_interval, scheduler: scheduler)
+      def initialize(execution_mode: nil, max_threads: nil, poll_interval: nil, scheduler: nil, inline: false)
+        configuration = GoodJob::Configuration.new({ execution_mode: execution_mode }, env: ENV)
+        super(execution_mode: configuration.rails_execution_mode, max_threads: max_threads, poll_interval: poll_interval, scheduler: scheduler, inline: inline)
       end
     end
   end

data/lib/good_job.rb

@@ -1,21 +1,73 @@
 require "rails"
 require 'good_job/railtie'
 
+require 'good_job/configuration'
 require 'good_job/log_subscriber'
 require 'good_job/lockable'
 require 'good_job/job'
 require 'good_job/scheduler'
+require 'good_job/multi_scheduler'
 require 'good_job/adapter'
 require 'good_job/pg_locks'
 require 'good_job/performer'
+require 'good_job/current_execution'
+require 'good_job/notifier'
 
 require 'active_job/queue_adapters/good_job_adapter'
 
+# GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails.
+#
+# +GoodJob+ is the top-level namespace and exposes configuration attributes.
 module GoodJob
-  cattr_accessor :logger, default: ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDOUT))
+  # @!attribute [rw] logger
+  #   @!scope class
+  #   The logger used by GoodJob
+  #   @return [Logger]
+  mattr_accessor :logger, default: ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDOUT))
+
+  # @!attribute [rw] preserve_job_records
+  #   @!scope class
+  #   Whether to preserve job records in the database after they have finished for inspection
+  #   @return [Boolean]
   mattr_accessor :preserve_job_records, default: false
+
+  # @!attribute [rw] reperform_jobs_on_standard_error
+  #   @!scope class
+  #   Whether to re-perform a job when a type of +StandardError+ is raised and bubbles up to the GoodJob backend
+  #   @return [Boolean]
   mattr_accessor :reperform_jobs_on_standard_error, default: true
+
+  # @!attribute [rw] on_thread_error
+  #   @!scope class
+  #   Called when a thread raises an error
+  #   @example Send errors to Sentry
+  #     # config/initializers/good_job.rb
+  #
+  #     # With Sentry (or Bugsnag, Airbrake, Honeybadger, etc.)
+  #     GoodJob.on_thread_error = -> (exception) { Raven.capture_exception(exception) }
+  #   @return [#call, nil]
   mattr_accessor :on_thread_error, default: nil
 
+  # Shuts down all execution pools
+  # @param wait [Boolean] whether to wait for shutdown
+  # @return [void]
+  def self.shutdown(wait: true)
+    Notifier.instances.each { |adapter| adapter.shutdown(wait: wait) }
+    Scheduler.instances.each { |scheduler| scheduler.shutdown(wait: wait) }
+  end
+
+  # Tests if execution pools are shut down
+  # @return [Boolean] whether execution pools are shut down
+  def self.shutdown?
+    Notifier.instances.all?(&:shutdown?) && Scheduler.instances.all?(&:shutdown?)
+  end
+
+  # Restarts all execution pools
+  # @return [void]
+  def self.restart
+    Notifier.instances.each(&:restart)
+    Scheduler.instances.each(&:restart)
+  end
+
   ActiveSupport.run_load_hooks(:good_job, self)
 end

data/lib/good_job/adapter.rb

@@ -2,28 +2,26 @@ module GoodJob
   class Adapter
     EXECUTION_MODES = [:async, :external, :inline].freeze
 
-    def initialize(execution_mode: nil, max_threads: nil, poll_interval: nil, scheduler: nil, inline: false)
-      if inline
+    def initialize(execution_mode: nil, queues: nil, max_threads: nil, poll_interval: nil, scheduler: nil, notifier: nil, inline: false)
+      if inline && execution_mode.nil?
         ActiveSupport::Deprecation.warn('GoodJob::Adapter#new(inline: true) is deprecated; use GoodJob::Adapter.new(execution_mode: :inline) instead')
-        @execution_mode = :inline
-      elsif execution_mode
-        raise ArgumentError, "execution_mode: must be one of #{EXECUTION_MODES.join(', ')}." unless EXECUTION_MODES.include?(execution_mode)
-
-        @execution_mode = execution_mode
-      else
-        @execution_mode = :external
+        execution_mode = :inline
       end
 
-      @scheduler = scheduler
-      if @execution_mode == :async && @scheduler.blank? # rubocop:disable Style/GuardClause
-        timer_options = {}
-        timer_options[:execution_interval] = poll_interval if poll_interval.present?
+      configuration = GoodJob::Configuration.new(
+        execution_mode: execution_mode,
+        queues: queues,
+        max_threads: max_threads,
+        poll_interval: poll_interval
+      )
 
-        pool_options = {}
-        pool_options[:max_threads] = max_threads if max_threads.present?
+      @execution_mode = configuration.execution_mode
+      raise ArgumentError, "execution_mode: must be one of #{EXECUTION_MODES.join(', ')}." unless EXECUTION_MODES.include?(@execution_mode)
 
-        job_performer = GoodJob::Performer.new(GoodJob::Job, :perform_with_advisory_lock, name: '*')
-        @scheduler = GoodJob::Scheduler.new(job_performer, timer_options: timer_options, pool_options: pool_options)
+      if @execution_mode == :async # rubocop:disable Style/GuardClause
+        @notifier = notifier || GoodJob::Notifier.new
+        @scheduler = scheduler || GoodJob::Scheduler.from_configuration(configuration)
+        @notifier.recipients << [@scheduler, :create_thread]
       end
     end
 
@@ -46,12 +44,14 @@ module GoodJob
         end
       end
 
-      @scheduler.create_thread if execute_async?
+      executed_locally = execute_async? && @scheduler.create_thread(queue_name: good_job.queue_name)
+      Notifier.notify(queue_name: good_job.queue_name) unless executed_locally
 
       good_job
     end
 
     def shutdown(wait: true)
+      @notifier&.shutdown(wait: wait)
       @scheduler&.shutdown(wait: wait)
     end
 

data/lib/good_job/cli.rb

@@ -10,43 +10,18 @@ module GoodJob
                   desc: "Maximum number of threads to use for working jobs (default: ActiveRecord::Base.connection_pool.size)"
     method_option :queues,
                   type: :string,
-                  banner: "queue1,queue2",
-                  desc: "Queues to work from. Separate multiple queues with commas (default: *)"
+                  banner: "queue1,queue2(;queue3,queue4:5;-queue1,queue2)",
+                  desc: "Queues to work from. Separate multiple queues with commas; exclude queues with a leading minus; separate isolated execution pools with semicolons and threads with colons (default: *)"
     method_option :poll_interval,
                   type: :numeric,
                   desc: "Interval between polls for available jobs in seconds (default: 1)"
     def start
       set_up_application!
 
-      max_threads = (
-        options[:max_threads] ||
-        ENV['GOOD_JOB_MAX_THREADS'] ||
-        ENV['RAILS_MAX_THREADS'] ||
-        ActiveRecord::Base.connection_pool.size
-      ).to_i
-
-      queue_string = (
-        options[:queues] ||
-        ENV['GOOD_JOB_QUEUES'] ||
-        '*'
-      )
-
-      poll_interval = (
-        options[:poll_interval] ||
-        ENV['GOOD_JOB_POLL_INTERVAL']
-      ).to_i
-
-      job_query = GoodJob::Job.queue_string(queue_string)
-      job_performer = GoodJob::Performer.new(job_query, :perform_with_advisory_lock, name: queue_string)
-
-      timer_options = {}
-      timer_options[:execution_interval] = poll_interval if poll_interval.positive?
-
-      pool_options = {
-        max_threads: max_threads,
-      }
-
-      scheduler = GoodJob::Scheduler.new(job_performer, timer_options: timer_options, pool_options: pool_options)
+      notifier = GoodJob::Notifier.new
+      configuration = GoodJob::Configuration.new(options)
+      scheduler = GoodJob::Scheduler.from_configuration(configuration)
+      notifier.recipients << [scheduler, :create_thread]
 
       @stop_good_job_executable = false
       %w[INT TERM].each do |signal|
@@ -55,9 +30,10 @@ module GoodJob
 
       Kernel.loop do
         sleep 0.1
-        break if @stop_good_job_executable || scheduler.shutdown?
+        break if @stop_good_job_executable || scheduler.shutdown? || notifier.shutdown?
       end
 
+      notifier.shutdown
       scheduler.shutdown
     end
 
@@ -68,6 +44,7 @@ module GoodJob
                   type: :numeric,
                   default: 24 * 60 * 60,
                   desc: "Delete records finished more than this many seconds ago"
+
     def cleanup_preserved_jobs
       set_up_application!
 
@@ -82,6 +59,10 @@ module GoodJob
     no_commands do
       def set_up_application!
         require RAILS_ENVIRONMENT_RB
+        return unless defined?(GOOD_JOB_LOG_TO_STDOUT) && GOOD_JOB_LOG_TO_STDOUT && !ActiveSupport::Logger.logger_outputs_to?(GoodJob.logger, STDOUT)
+
+        GoodJob::LogSubscriber.loggers << ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDOUT))
+        GoodJob::LogSubscriber.reset_logger
       end
     end
   end

data/lib/good_job/configuration.rb

@@ -0,0 +1,55 @@
+module GoodJob
+  class Configuration
+    attr_reader :options, :env
+
+    def initialize(options, env: ENV)
+      @options = options
+      @env = env
+    end
+
+    def execution_mode(default: :external)
+      if options[:execution_mode]
+        options[:execution_mode]
+      elsif env['GOOD_JOB_EXECUTION_MODE'].present?
+        env['GOOD_JOB_EXECUTION_MODE'].to_sym
+      else
+        default
+      end
+    end
+
+    def rails_execution_mode
+      if execution_mode(default: nil)
+        execution_mode
+      elsif Rails.env.development?
+        :inline
+      elsif Rails.env.test?
+        :inline
+      else
+        :external
+      end
+    end
+
+    def max_threads
+      (
+        options[:max_threads] ||
+        env['GOOD_JOB_MAX_THREADS'] ||
+        env['RAILS_MAX_THREADS'] ||
+        ActiveRecord::Base.connection_pool.size
+      ).to_i
+    end
+
+    def queue_string
+      options[:queues] ||
+        env['GOOD_JOB_QUEUES'] ||
+        '*'
+    end
+
+    def poll_interval
+      (
+        options[:poll_interval] ||
+        env['GOOD_JOB_POLL_INTERVAL'] ||
+        1
+      ).to_i
+    end
+  end
+end

data/lib/good_job/current_execution.rb

@@ -0,0 +1,35 @@
+module GoodJob
+  # Thread-local attributes for passing values from Instrumentation.
+  # (Cannot use ActiveSupport::CurrentAttributes because ActiveJob resets it)
+
+  module CurrentExecution
+    # @!attribute [rw] error_on_retry
+    #   @!scope class
+    #   Error captured by retry_on
+    #   @return [Exception, nil]
+    thread_mattr_accessor :error_on_retry
+
+    # @!attribute [rw] error_on_discard
+    #   @!scope class
+    #   Error captured by discard_on
+    #   @return [Exception, nil]
+    thread_mattr_accessor :error_on_discard
+
+    # Resets attributes
+    # @return [void]
+    def self.reset
+      self.error_on_retry = nil
+      self.error_on_discard = nil
+    end
+
+    # @return [Integer] Current process ID
+    def self.process_id
+      Process.pid
+    end
+
+    # @return [String] Current thread name
+    def self.thread_name
+      (Thread.current.name || Thread.current.object_id).to_s
+    end
+  end
+end

data/lib/good_job/job.rb

@@ -9,6 +9,25 @@ module GoodJob
 
     self.table_name = 'good_jobs'.freeze
 
+    def self.queue_parser(string)
+      string = string.presence || '*'
+
+      if string.first == '-'
+        exclude_queues = true
+        string = string[1..-1]
+      end
+
+      queues = string.split(',').map(&:strip)
+
+      if queues.include?('*')
+        { all: true }
+      elsif exclude_queues
+        { exclude: queues }
+      else
+        { include: queues }
+      end
+    end
+
     scope :unfinished, (lambda do
       if column_names.include?('finished_at')
         where(finished_at: nil)
@@ -21,8 +40,15 @@ module GoodJob
     scope :priority_ordered, -> { order('priority DESC NULLS LAST') }
     scope :finished, ->(timestamp = nil) { timestamp ? where(arel_table['finished_at'].lteq(timestamp)) : where.not(finished_at: nil) }
     scope :queue_string, (lambda do |string|
-      queue_names_without_all = (string.presence || '*').split(',').map(&:strip).reject { |q| q == '*' }
-      where(queue_name: queue_names_without_all) unless queue_names_without_all.size.zero?
+      parsed = queue_parser(string)
+
+      if parsed[:all]
+        all
+      elsif parsed[:exclude]
+        where.not(queue_name: parsed[:exclude]).or where(queue_name: nil)
+      elsif parsed[:include]
+        where(queue_name: parsed[:include])
+      end
     end)
 
     def self.perform_with_advisory_lock
@@ -47,7 +73,7 @@ module GoodJob
           queue_name: active_job.queue_name.presence || DEFAULT_QUEUE_NAME,
           priority: active_job.priority || DEFAULT_PRIORITY,
           serialized_params: active_job.serialize,
-          scheduled_at: scheduled_at || Time.current,
+          scheduled_at: scheduled_at,
           create_with_advisory_lock: create_with_advisory_lock
         )
 
@@ -63,11 +89,11 @@ module GoodJob
     def perform(destroy_after: !GoodJob.preserve_job_records, reperform_on_standard_error: GoodJob.reperform_jobs_on_standard_error)
       raise PreviouslyPerformedError, 'Cannot perform a job that has already been performed' if finished_at
 
+      GoodJob::CurrentExecution.reset
       result = nil
       rescued_error = nil
       error = nil
 
-      ActiveSupport::Notifications.instrument("before_perform_job.good_job", { good_job: self })
       self.performed_at = Time.current
       save! unless destroy_after
 
@@ -76,18 +102,23 @@ module GoodJob
       )
 
       begin
-        ActiveSupport::Notifications.instrument("perform_job.good_job", { good_job: self }) do
+        ActiveSupport::Notifications.instrument("perform_job.good_job", { good_job: self, process_id: GoodJob::CurrentExecution.process_id, thread_name: GoodJob::CurrentExecution.thread_name }) do
           result = ActiveJob::Base.execute(params)
         end
       rescue StandardError => e
         rescued_error = e
       end
 
+      retry_or_discard_error = GoodJob::CurrentExecution.error_on_retry ||
+                               GoodJob::CurrentExecution.error_on_discard
+
       if rescued_error
         error = rescued_error
       elsif result.is_a?(Exception)
         error = result
         result = nil
+      elsif retry_or_discard_error
+        error = retry_or_discard_error
       end
 
       error_message = "#{error.class}: #{error.message}" if error

data/lib/good_job/log_subscriber.rb

@@ -32,7 +32,7 @@ module GoodJob
       performer_name = event.payload[:performer_name]
       process_id = event.payload[:process_id]
 
-      info_and_stdout(tags: [process_id]) do
+      info(tags: [process_id]) do
         "GoodJob started scheduler with queues=#{performer_name} max_threads=#{max_threads} poll_interval=#{poll_interval}."
       end
     end
@@ -40,7 +40,7 @@ module GoodJob
     def scheduler_shutdown_start(event)
       process_id = event.payload[:process_id]
 
-      info_and_stdout(tags: [process_id]) do
+      info(tags: [process_id]) do
         "GoodJob shutting down scheduler..."
       end
     end
@@ -48,7 +48,7 @@ module GoodJob
     def scheduler_shutdown(event)
       process_id = event.payload[:process_id]
 
-      info_and_stdout(tags: [process_id]) do
+      info(tags: [process_id]) do
         "GoodJob scheduler is shut down."
       end
     end
@@ -56,24 +56,100 @@ module GoodJob
     def scheduler_restart_pools(event)
       process_id = event.payload[:process_id]
 
-      info_and_stdout(tags: [process_id]) do
+      info(tags: [process_id]) do
         "GoodJob scheduler has restarted."
       end
     end
 
+    def perform_job(event)
+      good_job = event.payload[:good_job]
+      process_id = event.payload[:process_id]
+      thread_name = event.payload[:thread_name]
+
+      info(tags: [process_id, thread_name]) do
+        "Executed GoodJob #{good_job.id}"
+      end
+    end
+
+    def notifier_listen(_event)
+      info do
+        "Notifier subscribed with LISTEN"
+      end
+    end
+
+    def notifier_notified(event)
+      payload = event.payload[:payload]
+
+      debug do
+        "Notifier received payload: #{payload}"
+      end
+    end
+
+    def notifier_notify_error(event)
+      error = event.payload[:error]
+
+      error do
+        "Notifier errored: #{error}"
+      end
+    end
+
+    def notifier_unlisten(_event)
+      info do
+        "Notifier unsubscribed with UNLISTEN"
+      end
+    end
+
     def cleanup_preserved_jobs(event)
       timestamp = event.payload[:timestamp]
       deleted_records_count = event.payload[:deleted_records_count]
 
-      info_and_stdout do
+      info do
         "GoodJob deleted #{deleted_records_count} preserved #{'job'.pluralize(deleted_records_count)} finished before #{timestamp}."
       end
     end
 
-    private
+    class << self
+      def loggers
+        @_loggers ||= [GoodJob.logger]
+      end
+
+      def logger
+        @_logger ||= begin
+                      logger = Logger.new(StringIO.new)
+                      loggers.each do |each_logger|
+                        logger.extend(ActiveSupport::Logger.broadcast(each_logger))
+                      end
+                      logger
+                    end
+      end
+
+      def reset_logger
+        @_logger = nil
+      end
+    end
 
     def logger
-      GoodJob.logger
+      GoodJob::LogSubscriber.logger
+    end
+
+    private
+
+    def tag_logger(*tags, &block)
+      tags = tags.dup.unshift("GoodJob").compact
+
+      self.class.loggers.inject(block) do |inner, each_logger|
+        if each_logger.respond_to?(:tagged)
+          tags_for_logger = if each_logger.formatter.current_tags.include?("ActiveJob")
+                              ["ActiveJob"] + tags
+                            else
+                              tags
+                            end
+
+          proc { each_logger.tagged(*tags_for_logger, &inner) }
+        else
+          inner
+        end
+      end.call
     end
 
     %w(info debug warn error fatal unknown).each do |level|
@@ -81,30 +157,11 @@ module GoodJob
         def #{level}(progname = nil, tags: [], &block)
           return unless logger
 
-          if logger.respond_to?(:tagged)
-            tags.unshift "GoodJob" unless logger.formatter.current_tags.include?("GoodJob")
-            logger.tagged(*tags.compact) do
-              logger.#{level}(progname, &block)
-            end
-          else
+          tag_logger(*tags) do
             logger.#{level}(progname, &block)
           end
         end
       METHOD
     end
-
-    def info_and_stdout(progname = nil, tags: [], &block)
-      unless ActiveSupport::Logger.logger_outputs_to?(logger, STDOUT)
-        tags_string = (['GoodJob'] + tags).map { |t| "[#{t}]" }.join(' ')
-        stdout_message = "#{tags_string}#{yield}"
-        $stdout.puts stdout_message
-      end
-
-      info(progname, tags: [], &block)
-    end
-
-    def thread_name
-      Thread.current.name || Thread.current.object_id
-    end
   end
 end

data/lib/good_job/multi_scheduler.rb

@@ -0,0 +1,34 @@
+module GoodJob
+  class MultiScheduler
+    attr_reader :schedulers
+
+    def initialize(schedulers)
+      @schedulers = schedulers
+    end
+
+    def shutdown(wait: true)
+      schedulers.each { |s| s.shutdown(wait: wait) }
+    end
+
+    def shutdown?
+      schedulers.all?(&:shutdown?)
+    end
+
+    def restart(wait: true)
+      schedulers.each { |s| s.restart(wait: wait) }
+    end
+
+    def create_thread(state = nil)
+      results = []
+      any_true = schedulers.any? do |scheduler|
+        scheduler.create_thread(state).tap { |result| results << result }
+      end
+
+      if any_true
+        true
+      else
+        results.any? { |result| result == false } ? false : nil
+      end
+    end
+  end
+end

data/lib/good_job/notifier.rb

@@ -0,0 +1,116 @@
+require 'concurrent/atomic/atomic_boolean'
+
+module GoodJob # :nodoc:
+  #
+  # Wrapper for Postgres LISTEN/NOTIFY
+  #
+  class Notifier
+    CHANNEL = 'good_job'.freeze
+    POOL_OPTIONS = {
+      min_threads: 0,
+      max_threads: 1,
+      auto_terminate: true,
+      idletime: 60,
+      max_queue: 1,
+      fallback_policy: :discard,
+    }.freeze
+    WAIT_INTERVAL = 1
+
+    # @!attribute [r] instances
+    #   @!scope class
+    #   @return [array<GoodJob:Adapter>] the instances of +GoodJob::Notifier+
+    cattr_reader :instances, default: [], instance_reader: false
+
+    def self.notify(message)
+      connection = ActiveRecord::Base.connection
+      connection.exec_query <<~SQL
+        NOTIFY #{CHANNEL}, #{connection.quote(message.to_json)}
+      SQL
+    end
+
+    attr_reader :recipients
+
+    def initialize(*recipients)
+      @recipients = Concurrent::Array.new(recipients)
+      @listening = Concurrent::AtomicBoolean.new(false)
+
+      self.class.instances << self
+
+      create_pool
+      listen
+    end
+
+    def listening?
+      @listening.true?
+    end
+
+    def restart(wait: true)
+      shutdown(wait: wait)
+      create_pool
+      listen
+    end
+
+    def shutdown(wait: true)
+      return unless @pool.running?
+
+      @pool.shutdown
+      @pool.wait_for_termination if wait
+    end
+
+    def shutdown?
+      !@pool.running?
+    end
+
+    private
+
+    def create_pool
+      @pool = Concurrent::ThreadPoolExecutor.new(POOL_OPTIONS)
+    end
+
+    def listen
+      future = Concurrent::Future.new(args: [@recipients, @pool, @listening], executor: @pool) do |recipients, pool, listening|
+        Rails.application.reloader.wrap do
+          conn = ActiveRecord::Base.connection.raw_connection
+          ActiveSupport::Notifications.instrument("notifier_listen.good_job") do
+            conn.async_exec "LISTEN #{CHANNEL}"
+          end
+
+          begin
+            ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
+              while pool.running?
+                listening.make_true
+                conn.wait_for_notify(WAIT_INTERVAL) do |channel, _pid, payload|
+                  listening.make_false
+                  next unless channel == CHANNEL
+
+                  ActiveSupport::Notifications.instrument("notifier_notified.good_job", { payload: payload })
+                  parsed_payload = JSON.parse(payload, symbolize_names: true)
+                  recipients.each do |recipient|
+                    target, method_name = recipient.is_a?(Array) ? recipient : [recipient, :call]
+                    target.send(method_name, parsed_payload)
+                  end
+                end
+                listening.make_false
+              end
+            end
+          rescue StandardError => e
+            ActiveSupport::Notifications.instrument("notifier_notify_error.good_job", { error: e })
+            raise
+          ensure
+            @listening.make_false
+            ActiveSupport::Notifications.instrument("notifier_unlisten.good_job") do
+              conn.async_exec "UNLISTEN *"
+            end
+          end
+        end
+      end
+
+      future.add_observer(self, :listen_observer)
+      future.execute
+    end
+
+    def listen_observer(_time, _result, _thread_error)
+      listen unless shutdown?
+    end
+  end
+end

data/lib/good_job/performer.rb

@@ -2,14 +2,21 @@ module GoodJob
   class Performer
     attr_reader :name
 
-    def initialize(target, method_name, name: nil)
+    def initialize(target, method_name, name: nil, filter: nil)
       @target = target
       @method_name = method_name
       @name = name
+      @filter = filter
     end
 
     def next
       @target.public_send(@method_name)
     end
+
+    def next?(state = {})
+      return true unless @filter.respond_to?(:call)
+
+      @filter.call(state)
+    end
   end
 end

data/lib/good_job/railtie.rb

@@ -4,5 +4,15 @@ module GoodJob
       ActiveSupport.on_load(:good_job) { self.logger = ::Rails.logger }
       GoodJob::LogSubscriber.attach_to :good_job
     end
+
+    initializer "good_job.active_job_notifications" do
+      ActiveSupport::Notifications.subscribe "enqueue_retry.active_job" do |event|
+        GoodJob::CurrentExecution.error_on_retry = event.payload[:error]
+      end
+
+      ActiveSupport::Notifications.subscribe "discard.active_job" do |event|
+        GoodJob::CurrentExecution.error_on_discard = event.payload[:error]
+      end
+    end
   end
 end

data/lib/good_job/scheduler.rb

@@ -2,16 +2,25 @@ require "concurrent/executor/thread_pool_executor"
 require "concurrent/timer_task"
 require "concurrent/utility/processor_counter"
 
-module GoodJob
+module GoodJob # :nodoc:
+  #
+  # Schedulers are generic thread execution pools that are responsible for
+  # periodically checking for available execution tasks, executing tasks in a
+  # bounded thread-pool, and efficiently scaling execution threads.
+  #
+  # Schedulers are "generic" in the sense that they delegate task execution
+  # details to a "Performer" object that responds to #next.
+  #
   class Scheduler
+    # Defaults for instance of Concurrent::TimerTask
     DEFAULT_TIMER_OPTIONS = {
       execution_interval: 1,
       timeout_interval: 1,
       run_now: true,
     }.freeze
 
+    # Defaults for instance of Concurrent::ThreadPoolExecutor
     DEFAULT_POOL_OPTIONS = {
-      name: 'good_job',
       min_threads: 0,
       max_threads: Concurrent.processor_count,
       auto_terminate: true,
@@ -20,8 +29,53 @@ module GoodJob
       fallback_policy: :discard,
     }.freeze
 
+    # @!attribute [r] instances
+    #   @!scope class
+    #   All instantiated Schedulers in the current process.
+    #   @return [array<GoodJob:Scheduler>]
     cattr_reader :instances, default: [], instance_reader: false
 
+    # Creates GoodJob::Scheduler(s) and Performers from a GoodJob::Configuration instance.
+    # @param configuration [GoodJob::Configuration]
+    # @return [GoodJob::Scheduler, GoodJob::MultiScheduler]
+    def self.from_configuration(configuration)
+      schedulers = configuration.queue_string.split(';').map do |queue_string_and_max_threads|
+        queue_string, max_threads = queue_string_and_max_threads.split(':')
+        max_threads = (max_threads || configuration.max_threads).to_i
+
+        job_query = GoodJob::Job.queue_string(queue_string)
+        parsed = GoodJob::Job.queue_parser(queue_string)
+        job_filter = proc do |state|
+          if parsed[:exclude]
+            !parsed[:exclude].include? state[:queue_name]
+          elsif parsed[:include]
+            parsed[:include].include? state[:queue_name]
+          else
+            true
+          end
+        end
+        job_performer = GoodJob::Performer.new(job_query, :perform_with_advisory_lock, name: queue_string, filter: job_filter)
+
+        timer_options = {}
+        timer_options[:execution_interval] = configuration.poll_interval
+
+        pool_options = {
+          max_threads: max_threads,
+        }
+
+        GoodJob::Scheduler.new(job_performer, timer_options: timer_options, pool_options: pool_options)
+      end
+
+      if schedulers.size > 1
+        GoodJob::MultiScheduler.new(schedulers)
+      else
+        schedulers.first
+      end
+    end
+
+    # @param performer [GoodJob::Performer]
+    # @param timer_options [Hash] Options to instantiate a Concurrent::TimerTask
+    # @param pool_options [Hash] Options to instantiate a Concurrent::ThreadPoolExecutor
     def initialize(performer, timer_options: {}, pool_options: {})
       raise ArgumentError, "Performer argument must implement #next" unless performer.respond_to?(:next)
 
@@ -31,14 +85,19 @@ module GoodJob
       @pool_options = DEFAULT_POOL_OPTIONS.merge(pool_options)
       @timer_options = DEFAULT_TIMER_OPTIONS.merge(timer_options)
 
+      @pool_options[:name] = "GoodJob::Scheduler(queues=#{@performer.name} max_threads=#{@pool_options[:max_threads]} poll_interval=#{@timer_options[:execution_interval]})"
+
       create_pools
     end
 
+    # Shut down the Scheduler.
+    # @param wait [Boolean] Wait for actively executing jobs to finish
+    # @return [void]
     def shutdown(wait: true)
       @_shutdown = true
 
-      ActiveSupport::Notifications.instrument("scheduler_shutdown_start.good_job", { wait: wait, process_id: process_id })
-      ActiveSupport::Notifications.instrument("scheduler_shutdown.good_job", { wait: wait, process_id: process_id }) do
+      instrument("scheduler_shutdown_start", { wait: wait })
+      instrument("scheduler_shutdown", { wait: wait }) do
         if @timer&.running?
           @timer.shutdown
           @timer.wait_for_termination if wait
@@ -51,19 +110,32 @@ module GoodJob
       end
     end
 
+    # True when the Scheduler is shutdown.
+    # @return [true, false, nil]
     def shutdown?
       @_shutdown
     end
 
+    # Restart the Scheduler. When shutdown, start; or shutdown and start.
+    # @param wait [Boolean] Wait for actively executing jobs to finish
+    # @return [void]
     def restart(wait: true)
-      ActiveSupport::Notifications.instrument("scheduler_restart_pools.good_job", { process_id: process_id }) do
+      instrument("scheduler_restart_pools") do
         shutdown(wait: wait) unless shutdown?
         create_pools
+        @_shutdown = false
       end
     end
 
-    def create_thread
-      return false unless @pool.ready_worker_count.positive?
+    # Triggers a Performer execution, if an execution thread is available.
+    # @param state [nil, Object] Allows Performer#next? to accept or reject the execution
+    # @return [nil, Boolean] if the thread was created
+    def create_thread(state = nil)
+      return nil unless @pool.running? && @pool.ready_worker_count.positive?
+
+      if state
+        return false unless @performer.next?(state)
+      end
 
       future = Concurrent::Future.new(args: [@performer], executor: @pool) do |performer|
         output = nil
@@ -72,35 +144,32 @@ module GoodJob
       end
       future.add_observer(self, :task_observer)
       future.execute
+
+      true
     end
 
+    # Invoked on completion of TimerTask task.
+    # @!visibility private
+    # @return [void]
     def timer_observer(time, executed_task, thread_error)
       GoodJob.on_thread_error.call(thread_error) if thread_error && GoodJob.on_thread_error.respond_to?(:call)
-      ActiveSupport::Notifications.instrument("finished_timer_task.good_job", { result: executed_task, error: thread_error, time: time })
+      instrument("finished_timer_task", { result: executed_task, error: thread_error, time: time })
     end
 
+    # Invoked on completion of ThreadPoolExecutor task
+    # @!visibility private
+    # @return [void]
     def task_observer(time, output, thread_error)
       GoodJob.on_thread_error.call(thread_error) if thread_error && GoodJob.on_thread_error.respond_to?(:call)
-      ActiveSupport::Notifications.instrument("finished_job_task.good_job", { result: output, error: thread_error, time: time })
+      instrument("finished_job_task", { result: output, error: thread_error, time: time })
       create_thread if output
     end
 
-    class ThreadPoolExecutor < Concurrent::ThreadPoolExecutor
-      # https://github.com/ruby-concurrency/concurrent-ruby/issues/684#issuecomment-427594437
-      def ready_worker_count
-        synchronize do
-          workers_still_to_be_created = @max_length - @pool.length
-          workers_created_but_waiting = @ready.length
-
-          workers_still_to_be_created + workers_created_but_waiting
-        end
-      end
-    end
-
     private
 
+    # @return [void]
     def create_pools
-      ActiveSupport::Notifications.instrument("scheduler_create_pools.good_job", { performer_name: @performer.name, max_threads: @pool_options[:max_threads], poll_interval: @timer_options[:execution_interval], process_id: process_id }) do
+      instrument("scheduler_create_pools", { performer_name: @performer.name, max_threads: @pool_options[:max_threads], poll_interval: @timer_options[:execution_interval] }) do
         @pool = ThreadPoolExecutor.new(@pool_options)
         next unless @timer_options[:execution_interval].positive?
 
@@ -110,12 +179,29 @@ module GoodJob
       end
     end
 
-    def process_id
-      Process.pid
+    def instrument(name, payload = {}, &block)
+      payload = payload.reverse_merge({
+                                        scheduler: self,
+                                        process_id: GoodJob::CurrentExecution.process_id,
+                                        thread_name: GoodJob::CurrentExecution.thread_name,
+                                      })
+
+      ActiveSupport::Notifications.instrument("#{name}.good_job", payload, &block)
     end
+  end
 
-    def thread_name
-      Thread.current.name || Thread.current.object_id
+  # Slightly customized sub-class of Concurrent::ThreadPoolExecutor
+  class ThreadPoolExecutor < Concurrent::ThreadPoolExecutor
+    # Number of idle or potential threads available to execute tasks
+    # https://github.com/ruby-concurrency/concurrent-ruby/issues/684#issuecomment-427594437
+    # @return [Integer]
+    def ready_worker_count
+      synchronize do
+        workers_still_to_be_created = @max_length - @pool.length
+        workers_created_but_waiting = @ready.length
+
+        workers_still_to_be_created + workers_created_but_waiting
+      end
     end
   end
 end

data/lib/good_job/version.rb

@@ -1,3 +1,3 @@
 module GoodJob
-  VERSION = '1.1.0'.freeze
+  VERSION = '1.2.0'.freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 1.1.0
+  version: 1.2.0
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2020-08-10 00:00:00.000000000 Z
+date: 2020-08-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: concurrent-ruby
@@ -151,7 +151,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: pry
+  name: pry-rails
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -178,6 +178,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: rbtrace
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: rspec-rails
   requirement: !ruby/object:Gem::Requirement
@@ -248,6 +262,34 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: sigdump
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: yard
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: A multithreaded, Postgres-based ActiveJob backend for Ruby on Rails
 email:
 - bensheldon@gmail.com
@@ -269,9 +311,13 @@ files:
 - lib/good_job.rb
 - lib/good_job/adapter.rb
 - lib/good_job/cli.rb
+- lib/good_job/configuration.rb
+- lib/good_job/current_execution.rb
 - lib/good_job/job.rb
 - lib/good_job/lockable.rb
 - lib/good_job/log_subscriber.rb
+- lib/good_job/multi_scheduler.rb
+- lib/good_job/notifier.rb
 - lib/good_job/performer.rb
 - lib/good_job/pg_locks.rb
 - lib/good_job/railtie.rb