good_job 3.16.2 → 3.16.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: df700f1e4e911894d7f47d8b46c6159200e9f68bc6c7db9c9d8592999320851f
-  data.tar.gz: 0f8994e16848891c3a2a41064b8ee12555c9880035735ed1635a6ffd888ef276
+  metadata.gz: c4853b6c3598f46587b44f56dc6c59f4694753184da2d75d84cc87b8e901a3c3
+  data.tar.gz: 8086b714bc3a69a9260c36ef2b8295fc86ed6797eace2d8d8a4724ba509a7e65
 SHA512:
-  metadata.gz: f73d66cf94b957e59aaf4352d0bd0f0e4d5477144875a5923f846f5dadaf938b7a3f6c1f0cf21ae353258145e5ad2f4cc4c98a5c433d77c6b45b74c315358cdc
-  data.tar.gz: 99a1ba23209daa2890fee61443a91ff61a5ce2c8b7a5c5eeb55f54d85cc8166b97c3551d132a765ef6907924c78291e52d1b5cec705a9829f833f688506ecd76
+  metadata.gz: 5962d1992f8e283a6ca8349c1a9cf7d986bce54a2dc5f0abc09c1940ec97c7bc24af022eaa8e5979bcde395a4876741f34065cc24dfd1c88cb08a2a97baae77d
+  data.tar.gz: a1407ceaa0930ecf591fbf330e310124f6aaf9fd5ffc05aa579ae6176b8346dc004ba2ce2dc8d15b57efe9c1b07e68a248408cf0bec501bf5f84e586903d5c73

data/CHANGELOG.md

@@ -1,5 +1,46 @@
 # Changelog
 
+## [v3.16.4](https://github.com/bensheldon/good_job/tree/v3.16.4) (2023-07-30)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.16.3...v3.16.4)
+
+**Implemented enhancements:**
+
+- Add database\_connection\_pool stats to `GoodJob::Process.current_state` [\#1019](https://github.com/bensheldon/good_job/pull/1019) ([dixpac](https://github.com/dixpac))
+
+**Fixed bugs:**
+
+- Move `Execution#active_job` to BaseExecution to share with Job; refactor `Batch#active_jobs` to use job directly [\#1022](https://github.com/bensheldon/good_job/pull/1022) ([bensheldon](https://github.com/bensheldon))
+
+**Closed issues:**
+
+- Notifier errored: ArgumentError: wrong number of arguments \(given 1, expected 0\) [\#1016](https://github.com/bensheldon/good_job/issues/1016)
+- Understanding Database Connections and Cron [\#1013](https://github.com/bensheldon/good_job/issues/1013)
+- Experiencing various database exceptions with Rails 7.1  [\#796](https://github.com/bensheldon/good_job/issues/796)
+
+**Merged pull requests:**
+
+- Refactor Notifier to ensure \#restart is threadsafe [\#1021](https://github.com/bensheldon/good_job/pull/1021) ([bensheldon](https://github.com/bensheldon))
+- Notifier and CronManager share a 2-thread executor within the capsule [\#1018](https://github.com/bensheldon/good_job/pull/1018) ([bensheldon](https://github.com/bensheldon))
+- Clarify database connections and recurring processes in README.md [\#1015](https://github.com/bensheldon/good_job/pull/1015) ([blumhardts](https://github.com/blumhardts))
+- Deprecate `GoodJob::Lockable` and rename to `GoodJob::AdvisoryLockable` [\#1012](https://github.com/bensheldon/good_job/pull/1012) ([bensheldon](https://github.com/bensheldon))
+
+## [v3.16.3](https://github.com/bensheldon/good_job/tree/v3.16.3) (2023-07-18)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.16.2...v3.16.3)
+
+**Fixed bugs:**
+
+- Fix bulk enqueue for unmigrated 'error\_event'; add `GoodJob.migrated?` check method; use custom enum implementation [\#1011](https://github.com/bensheldon/good_job/pull/1011) ([bensheldon](https://github.com/bensheldon))
+
+**Closed issues:**
+
+- GoodJob::Bulk.enqueue not handling missing migrations [\#1010](https://github.com/bensheldon/good_job/issues/1010)
+
+**Merged pull requests:**
+
+- Move shared `BaseExecution` concerns into the base class. [\#1009](https://github.com/bensheldon/good_job/pull/1009) ([dixpac](https://github.com/dixpac))
+
 ## [v3.16.2](https://github.com/bensheldon/good_job/tree/v3.16.2) (2023-07-13)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.16.1...v3.16.2)

data/README.md

@@ -59,8 +59,8 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
     - [Timeouts](#timeouts)
     - [Optimize queues, threads, and processes](#optimize-queues-threads-and-processes)
     - [Database connections](#database-connections)
-        - [Production setup](#production-setup)
-        - [Queue performance with Queue Select Limit](#queue-performance-with-queue-select-limit)
+    - [Production setup](#production-setup)
+    - [Queue performance with Queue Select Limit](#queue-performance-with-queue-select-limit)
     - [Execute jobs async / in-process](#execute-jobs-async--in-process)
     - [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
     - [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
@@ -475,7 +475,7 @@ GoodJob's concurrency control strategy for `perform_limit` is "optimistic retry
 
 GoodJob can enqueue jobs on a recurring basis that can be used as a replacement for cron.
 
-Cron-style jobs are run on every GoodJob process (e.g. CLI or `async` execution mode) when `config.good_job.enable_cron = true`.
+Cron-style jobs can be performed by any GoodJob process (e.g., CLI or `:async` execution mode) that has `config.good_job.enable_cron` set to `true`. That is, one or more job executor processes can be configured to perform recurring jobs.
 
 GoodJob's cron uses unique indexes to ensure that only a single job is enqueued at the given time interval. In order for this to work, GoodJob must preserve cron-created job records; these records will be automatically deleted like any other preserved record.
 
@@ -484,7 +484,7 @@ Cron-format is parsed by the [`fugit`](https://github.com/floraison/fugit) gem,
 ```ruby
 # config/environments/application.rb or a specific environment e.g. production.rb
 
-# Enable cron in this process; e.g. only run on the first Heroku worker process
+# Enable cron in this process, e.g., only run on the first Heroku worker process
 config.good_job.enable_cron = ENV['DYNO'] == 'worker.1' # or `true` or via $GOOD_JOB_ENABLE_CRON
 
 # Configure cron with a hash that has a unique key for each recurring job
@@ -703,7 +703,9 @@ GoodJob follows semantic versioning, though updates may be encouraged through de
 
 #### Upgrading minor versions
 
-Upgrading between minor versions (e.g. v1.4 to v1.5) should not introduce breaking changes, but can introduce new deprecation warnings and database migration notices.
+Upgrading between minor versions (e.g. v1.4 to v1.5) should not introduce breaking changes, but can introduce new deprecation warnings and database migration warnings.
+
+Database migrations introduced in minor releases are _not required_ to be applied until the next major release. If you would like apply newly introduced migrations immediately, assert `GoodJob.migrated?` in your application's test suite.
 
 To perform upgrades to the GoodJob database tables:
 
@@ -952,24 +954,50 @@ Keep in mind, queue operations and management is an advanced discipline. This st
 
 ### Database connections
 
-Each GoodJob execution thread requires its own database connection that is automatically checked out from Rails’ connection pool. For example:
+GoodJob job executor processes require the following database connections:
+
+- 1 connection per execution pool thread. E.g., `--queues=mice:2;elephants:1` is 3 threads and thus 3 connections. Pool size defaults to `--max-threads`.
+- 2 additional connections that GoodJob uses for utility functionality (e.g. LISTEN/NOTIFY, cron, etc.)
+- 1 connection per subthread, if your application makes multithreaded database queries (e.g. `load_async`) within a job.
+
+The executor process will not crash if the connections pool is exhausted, instead it will report an exception (eg. `ActiveRecord::ConnectionTimeoutError`).
+
+When GoodJob runs in `:inline` mode (in Rails' test environment, by default), the default database pool configuration works.
+
+```yml
+# config/database.yml
+
+pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
+```
+
+When GoodJob runs in `:async` mode (in Rails's development environment, by default), the following database pool configuration works, where:
+
+- `ENV.fetch("RAILS_MAX_THREADS", 5)` is the number of threads used by the web server
+- `1` is the number of connections used by the job listener
+- `2` is the number of connections used by the cron scheduler and executor
+- `ENV.fetch("GOOD_JOB_MAX_THREADS", 5)` is the number of threads used to perform jobs
 
 ```yaml
 # config/database.yml
-pool: <%= ENV.fetch("RAILS_MAX_THREADS", 5).to_i + 3 + ENV.fetch("GOOD_JOB_MAX_THREADS", 5).to_i %>
+
+pool: <%= ENV.fetch("RAILS_MAX_THREADS", 5).to_i + 1 + 2 + ENV.fetch("GOOD_JOB_MAX_THREADS", 5).to_i %>
 ```
 
-To calculate the total number of the database connections you'll need:
+When GoodJob runs in `:external` mode (in Rails' production environment, by default), the following database pool configurations work for web servers and worker processes, respectively.
 
-- 1 connection dedicated to the scheduler aka `LISTEN/NOTIFY`
-- 1 connection per query pool thread e.g. `--queues=mice:2;elephants:1` is 3 threads. Pool thread size defaults to `--max-threads`
-- (optional) 2 connections for Cron scheduler if you're running it
-- (optional) 1 connection per subthread, if your application makes multithreaded database queries within a job
-- When running `:async`, you must also add the number of threads by the webserver
+```yaml
+# config/database.yml
+
+pool: <%= ENV.fetch("RAILS_MAX_THREADS", 5) %>
+```
 
-The queue process will not crash if the connections pool is exhausted, instead it will report an exception (eg. `ActiveRecord::ConnectionTimeoutError`).
+```yaml
+# config/database.yml
+
+pool: <%= 1 + 2 + ENV.fetch("GOOD_JOB_MAX_THREADS", 5).to_i %>
+```
 
-#### Production setup
+### Production setup
 
 When running GoodJob in a production environment, you should be mindful of:
 
@@ -984,7 +1012,7 @@ The recommended way to monitor the queue in production is:
 - keep an eye on the number of jobs in the queue (abnormal high number of unscheduled jobs means the queue could be underperforming)
 - consider performance monitoring services which support the built-in Rails instrumentation (eg. Sentry, Skylight, etc.)
 
-#### Queue performance with Queue Select Limit
+### Queue performance with Queue Select Limit
 
 GoodJob’s advisory locking strategy uses a materialized CTE (Common Table Expression). This strategy can be non-performant when querying a very large queue of executable jobs (100,000+) because the database query must materialize all executable jobs before acquiring an advisory lock.
 

data/app/models/concerns/good_job/{lockable.rb → advisory_lockable.rb}

@@ -16,7 +16,7 @@ module GoodJob
   #     end
   #   end
   #
-  module Lockable
+  module AdvisoryLockable
     extend ActiveSupport::Concern
 
     # Indicates an advisory lock is already held on a record by another

data/app/models/concerns/good_job/error_events.rb

@@ -14,15 +14,32 @@ module GoodJob
       ERROR_EVENT_DISCARDED = 'discarded',
     ].freeze
 
-    included do
-      enum error_event: {
-        ERROR_EVENT_INTERRUPTED => 0,
-        ERROR_EVENT_UNHANDLED => 1,
-        ERROR_EVENT_HANDLED => 2,
-        ERROR_EVENT_RETRIED => 3,
-        ERROR_EVENT_RETRY_STOPPED => 4,
-        ERROR_EVENT_DISCARDED => 5,
-      }.freeze, _prefix: :error_event
+    ERROR_EVENT_ENUMS = {
+      ERROR_EVENT_INTERRUPTED => 0,
+      ERROR_EVENT_UNHANDLED => 1,
+      ERROR_EVENT_HANDLED => 2,
+      ERROR_EVENT_RETRIED => 3,
+      ERROR_EVENT_RETRY_STOPPED => 4,
+      ERROR_EVENT_DISCARDED => 5,
+    }.freeze
+
+    # TODO: GoodJob v4 can make this an `enum` once migrations are guaranteed.
+    def error_event
+      return unless self.class.columns_hash['error_event']
+
+      enum = super
+      return unless enum
+
+      ERROR_EVENT_ENUMS.key(enum)
+    end
+
+    def error_event=(event)
+      return unless self.class.columns_hash['error_event']
+
+      enum = ERROR_EVENT_ENUMS[event]
+      raise(ArgumentError, "Invalid error_event: #{event}") if event && !enum
+
+      super(enum)
     end
   end
 end

data/app/models/good_job/base_execution.rb

@@ -4,7 +4,10 @@ module GoodJob
   # ActiveRecord model to share behavior between {Job} and {Execution} models
   # which both read out of the same table.
   class BaseExecution < BaseRecord
+    include AdvisoryLockable
     include ErrorEvents
+    include Filterable
+    include Reportable
 
     self.table_name = 'good_jobs'
 
@@ -57,5 +60,28 @@ module GoodJob
     def discrete?
       self.class.discrete_support? && is_discrete?
     end
+
+    # Build an ActiveJob instance and deserialize the arguments, using `#active_job_data`.
+    #
+    # @param ignore_deserialization_errors [Boolean]
+    #   Whether to ignore ActiveJob::DeserializationError when deserializing the arguments.
+    #   This is most useful if you aren't planning to use the arguments directly.
+    def active_job(ignore_deserialization_errors: false)
+      ActiveJob::Base.deserialize(active_job_data).tap do |aj|
+        aj.send(:deserialize_arguments_if_needed)
+      rescue ActiveJob::DeserializationError
+        raise unless ignore_deserialization_errors
+      end
+    end
+
+    private
+
+    def active_job_data
+      serialized_params.deep_dup
+                       .tap do |job_data|
+        job_data["provider_job_id"] = id
+        job_data["good_job_concurrency_key"] = concurrency_key if concurrency_key
+      end
+    end
   end
 end

data/app/models/good_job/batch.rb

@@ -124,11 +124,11 @@ module GoodJob
     end
 
     def active_jobs
-      record.jobs.map(&:head_execution).map(&:active_job)
+      record.jobs.map(&:active_job)
     end
 
     def callback_active_jobs
-      record.callback_jobs.map(&:head_execution).map(&:active_job)
+      record.callback_jobs.map(&:active_job)
     end
 
     def assign_properties(properties)

data/app/models/good_job/batch_record.rb

@@ -2,7 +2,7 @@
 
 module GoodJob
   class BatchRecord < BaseRecord
-    include Lockable
+    include AdvisoryLockable
 
     self.table_name = 'good_job_batches'
 

data/app/models/good_job/execution.rb

@@ -3,10 +3,6 @@
 module GoodJob
   # ActiveRecord model that represents an +ActiveJob+ job.
   class Execution < BaseExecution
-    include Lockable
-    include Filterable
-    include Reportable
-
     # Raised if something attempts to execute a previously completed Execution again.
     PreviouslyPerformedError = Class.new(StandardError)
 
@@ -384,7 +380,7 @@ module GoodJob
                 error: interrupt_error_string,
                 finished_at: Time.current,
               }
-              discrete_execution_attrs[:error_event] = GoodJob::DiscreteExecution.error_events[GoodJob::DiscreteExecution::ERROR_EVENT_INTERRUPTED] if self.class.error_event_migrated?
+              discrete_execution_attrs[:error_event] = GoodJob::ErrorEvents::ERROR_EVENT_ENUMS[GoodJob::ErrorEvents::ERROR_EVENT_INTERRUPTED] if self.class.error_event_migrated?
               discrete_executions.where(finished_at: nil).where.not(performed_at: nil).update_all(discrete_execution_attrs) # rubocop:disable Rails/SkipsModelValidations
             end
           end
@@ -512,19 +508,6 @@ module GoodJob
       self.scheduled_at ||= current_time
     end
 
-    # Build an ActiveJob instance and deserialize the arguments, using `#active_job_data`.
-    #
-    # @param ignore_deserialization_errors [Boolean]
-    #   Whether to ignore ActiveJob::DeserializationError when deserializing the arguments.
-    #   This is most useful if you aren't planning to use the arguments directly.
-    def active_job(ignore_deserialization_errors: false)
-      ActiveJob::Base.deserialize(active_job_data).tap do |aj|
-        aj.send(:deserialize_arguments_if_needed)
-      rescue ActiveJob::DeserializationError
-        raise unless ignore_deserialization_errors
-      end
-    end
-
     # Return formatted serialized_params for display in the dashboard
     # @return [Hash]
     def display_serialized_params
@@ -569,14 +552,6 @@ module GoodJob
 
     private
 
-    def active_job_data
-      serialized_params.deep_dup
-                       .tap do |job_data|
-        job_data["provider_job_id"] = id
-        job_data["good_job_concurrency_key"] = concurrency_key if concurrency_key
-      end
-    end
-
     def reset_batch_values(&block)
       GoodJob::Batch.within_thread(batch_id: nil, batch_callback_id: nil, &block)
     end

data/app/models/good_job/job.rb

@@ -7,10 +7,6 @@ module GoodJob
   # A single row from the +good_jobs+ table of executions is fetched to represent a Job.
   #
   class Job < BaseExecution
-    include Filterable
-    include Lockable
-    include Reportable
-
     # Raised when an inappropriate action is applied to a Job based on its state.
     ActionForStateMismatchError = Class.new(StandardError)
     # Raised when an action requires GoodJob to be the ActiveJob Queue Adapter but GoodJob is not.
@@ -228,9 +224,12 @@ module GoodJob
 
         update_execution = proc do
           execution.update(
-            finished_at: Time.current,
-            error: GoodJob::Execution.format_error(job_error),
-            error_event: :discarded
+            {
+              finished_at: Time.current,
+              error: GoodJob::Execution.format_error(job_error),
+            }.tap do |attrs|
+              attrs[:error_event] = ERROR_EVENT_DISCARDED if self.class.error_event_migrated?
+            end
           )
         end
 

data/app/models/good_job/process.rb

@@ -5,8 +5,8 @@ require 'socket'
 module GoodJob # :nodoc:
   # ActiveRecord model that represents an GoodJob process (either async or CLI).
   class Process < BaseRecord
+    include AdvisoryLockable
     include AssignableConnection
-    include Lockable
 
     # Interval until the process record being updated
     STALE_INTERVAL = 30.seconds
@@ -63,6 +63,10 @@ module GoodJob # :nodoc:
         cron_enabled: GoodJob.configuration.enable_cron?,
         total_succeeded_executions_count: GoodJob::Scheduler.instances.sum { |scheduler| scheduler.stats.fetch(:succeeded_executions_count) },
         total_errored_executions_count: GoodJob::Scheduler.instances.sum { |scheduler| scheduler.stats.fetch(:errored_executions_count) },
+        database_connection_pool: {
+          size: connection_pool.size,
+          active: connection_pool.connections.count(&:in_use?),
+        },
       }
     end
 

data/lib/good_job/adapter.rb

@@ -64,7 +64,15 @@ module GoodJob
 
       inline_executions = []
       GoodJob::Execution.transaction(requires_new: true, joinable: false) do
-        results = GoodJob::Execution.insert_all(executions.map(&:attributes), returning: %w[id active_job_id]) # rubocop:disable Rails/SkipsModelValidations
+        execution_attributes = executions.map do |execution|
+          if GoodJob::Execution.error_event_migrated?
+            execution.attributes
+          else
+            execution.attributes.except('error_event')
+          end
+        end
+
+        results = GoodJob::Execution.insert_all(execution_attributes, returning: %w[id active_job_id]) # rubocop:disable Rails/SkipsModelValidations
 
         job_id_to_provider_job_id = results.each_with_object({}) { |result, hash| hash[result['active_job_id']] = result['id'] }
         active_jobs.each do |active_job|

data/lib/good_job/capsule.rb

@@ -29,13 +29,14 @@ module GoodJob
       @mutex.synchronize do
         return unless startable?(force: force)
 
-        @notifier = GoodJob::Notifier.new(enable_listening: @configuration.enable_listen_notify)
+        @shared_executor = GoodJob::SharedExecutor.new
+        @notifier = GoodJob::Notifier.new(enable_listening: @configuration.enable_listen_notify, executor: @shared_executor.executor)
         @poller = GoodJob::Poller.new(poll_interval: @configuration.poll_interval)
         @scheduler = GoodJob::Scheduler.from_configuration(@configuration, warm_cache_on_initialize: true)
         @notifier.recipients << [@scheduler, :create_thread]
         @poller.recipients << [@scheduler, :create_thread]
 
-        @cron_manager = GoodJob::CronManager.new(@configuration.cron_entries, start_on_initialize: true) if @configuration.enable_cron?
+        @cron_manager = GoodJob::CronManager.new(@configuration.cron_entries, start_on_initialize: true, executor: @shared_executor.executor) if @configuration.enable_cron?
 
         @startable = false
         @running = true
@@ -51,7 +52,7 @@ module GoodJob
     # @return [void]
     def shutdown(timeout: :default)
       timeout = @configuration.shutdown_timeout if timeout == :default
-      GoodJob._shutdown_all([@notifier, @poller, @scheduler, @cron_manager].compact, timeout: timeout)
+      GoodJob._shutdown_all([@shared_executor, @notifier, @poller, @scheduler, @cron_manager].compact, timeout: timeout)
       @startable = false
       @running = false
     end
@@ -73,7 +74,7 @@ module GoodJob
 
     # @return [Boolean] Whether the capsule has been shutdown.
     def shutdown?
-      [@notifier, @poller, @scheduler, @cron_manager].compact.all?(&:shutdown?)
+      [@shared_executor, @notifier, @poller, @scheduler, @cron_manager].compact.all?(&:shutdown?)
     end
 
     # Creates an execution thread(s) with the given attributes.

data/lib/good_job/configuration.rb

@@ -51,13 +51,10 @@ module GoodJob
     # @param warn [Boolean] whether to print a warning when over the limit
     # @return [Integer]
     def self.total_estimated_threads(warn: false)
-      configuration = new({})
-
-      cron_threads = configuration.enable_cron? ? 2 : 0
-      notifier_threads = 1
+      utility_threads = GoodJob::SharedExecutor::MAX_THREADS
       scheduler_threads = GoodJob::Scheduler.instances.sum { |scheduler| scheduler.stats[:max_threads] }
 
-      good_job_threads = cron_threads + notifier_threads + scheduler_threads
+      good_job_threads = utility_threads + scheduler_threads
       puma_threads = (Puma::Server.current&.max_threads if defined?(Puma::Server)) || 0
 
       total_threads = good_job_threads + puma_threads

data/lib/good_job/cron_manager.rb

@@ -31,7 +31,8 @@ module GoodJob # :nodoc:
 
     # @param cron_entries [Array<CronEntry>]
     # @param start_on_initialize [Boolean]
-    def initialize(cron_entries = [], start_on_initialize: false)
+    def initialize(cron_entries = [], start_on_initialize: false, executor: Concurrent.global_io_executor)
+      @executor = executor
       @running = false
       @cron_entries = cron_entries
       @tasks = Concurrent::Hash.new
@@ -84,7 +85,7 @@ module GoodJob # :nodoc:
     def create_task(cron_entry)
       cron_at = cron_entry.next_at
       delay = [(cron_at - Time.current).to_f, 0].max
-      future = Concurrent::ScheduledTask.new(delay, args: [self, cron_entry, cron_at]) do |thr_scheduler, thr_cron_entry, thr_cron_at|
+      future = Concurrent::ScheduledTask.new(delay, args: [self, cron_entry, cron_at], executor: @executor) do |thr_scheduler, thr_cron_entry, thr_cron_at|
         # Re-schedule the next cron task before executing the current task
         thr_scheduler.create_task(thr_cron_entry)
 

data/lib/good_job/notifier.rb

@@ -20,16 +20,6 @@ module GoodJob # :nodoc:
 
     # Default Postgres channel for LISTEN/NOTIFY
     CHANNEL = 'good_job'
-    # Defaults for instance of Concurrent::ThreadPoolExecutor
-    EXECUTOR_OPTIONS = {
-      name: name,
-      min_threads: 0,
-      max_threads: 1,
-      auto_terminate: true,
-      idletime: 60,
-      max_queue: 1,
-      fallback_policy: :discard,
-    }.freeze
     # Seconds to block while LISTENing for a message
     WAIT_INTERVAL = 1
     # Seconds to wait if database cannot be connected to
@@ -70,19 +60,30 @@ module GoodJob # :nodoc:
 
     # @param recipients [Array<#call, Array(Object, Symbol)>]
     # @param enable_listening [true, false]
-    def initialize(*recipients, enable_listening: true)
+    # @param executor [Concurrent::ExecutorService]
+    def initialize(*recipients, enable_listening: true, executor: Concurrent.global_io_executor)
       @recipients = Concurrent::Array.new(recipients)
+      @enable_listening = enable_listening
+      @executor = executor
+
+      @mutex = Mutex.new
+      @shutdown_event = Concurrent::Event.new.tap(&:set)
+      @running = Concurrent::AtomicBoolean.new(false)
       @connected = Concurrent::AtomicBoolean.new(false)
       @listening = Concurrent::AtomicBoolean.new(false)
       @connection_errors_count = Concurrent::AtomicFixnum.new(0)
       @connection_errors_reported = Concurrent::AtomicBoolean.new(false)
       @enable_listening = enable_listening
+      @task = nil
 
-      create_executor
-      listen
+      start
       self.class.instances << self
     end
 
+    def running?
+      @executor.running? && @running.true?
+    end
+
     # Tests whether the notifier is active and has acquired a dedicated database connection.
     # @return [true, false, nil]
     def connected?
@@ -95,15 +96,9 @@ module GoodJob # :nodoc:
       @listening.true?
     end
 
-    # Tests whether the notifier is running.
-    # @!method running?
-    # @return [true, false, nil]
-    delegate :running?, to: :executor, allow_nil: true
-
-    # Tests whether the scheduler is shutdown.
-    # @!method shutdown?
-    # @return [true, false, nil]
-    delegate :shutdown?, to: :executor, allow_nil: true
+    def shutdown?
+      @shutdown_event.set?
+    end
 
     # Shut down the notifier.
     # This stops the background LISTENing thread.
@@ -111,17 +106,21 @@ module GoodJob # :nodoc:
     # @param timeout [Numeric, nil] Seconds to wait for active threads.
     #   * +nil+, the scheduler will trigger a shutdown but not wait for it to complete.
     #   * +-1+, the scheduler will wait until the shutdown is complete.
-    #   * +0+, the scheduler will immediately shutdown and stop any threads.
     #   * A positive number will wait that many seconds before stopping any remaining active threads.
     # @return [void]
     def shutdown(timeout: -1)
-      return if executor.nil? || executor.shutdown?
-
-      executor.shutdown if executor.running?
-
-      if executor.shuttingdown? && timeout # rubocop:disable Style/GuardClause
-        executor_wait = timeout.negative? ? nil : timeout
-        executor.kill unless executor.wait_for_termination(executor_wait)
+      synchronize do
+        @running.make_false
+
+        if @executor.shutdown? || @task&.complete?
+          # clean up in the even the executor is killed
+          @connected.make_false
+          @listening.make_false
+          @shutdown_event.set
+        else
+          @shutdown_event.wait(timeout == -1 ? nil : timeout) unless timeout.nil?
+        end
+        @shutdown_event.set?
       end
     end
 
@@ -130,9 +129,10 @@ module GoodJob # :nodoc:
     # @param timeout [nil, Numeric] Seconds to wait; shares same values as {#shutdown}.
     # @return [void]
     def restart(timeout: -1)
-      shutdown(timeout: timeout) if running?
-      create_executor
-      listen
+      synchronize do
+        shutdown(timeout: timeout) unless @shutdown_event.set?
+        start
+      end
     end
 
     # Invoked on completion of ThreadPoolExecutor task
@@ -160,21 +160,27 @@ module GoodJob # :nodoc:
         end
       end
 
-      return if shutdown?
-
-      listen(delay: connection_error ? RECONNECT_INTERVAL : 0)
+      if @running.true?
+        create_listen_task(delay: connection_error ? RECONNECT_INTERVAL : 0)
+      else
+        @shutdown_event.set
+      end
     end
 
     private
 
-    attr_reader :executor
+    def start
+      synchronize do
+        return if @running.true?
 
-    def create_executor
-      @executor = Concurrent::ThreadPoolExecutor.new(EXECUTOR_OPTIONS)
+        @running.make_true
+        @shutdown_event.reset
+        create_listen_task(delay: 0)
+      end
     end
 
-    def listen(delay: 0)
-      future = Concurrent::ScheduledTask.new(delay, args: [@recipients, executor, @enable_listening, @listening], executor: @executor) do |thr_recipients, thr_executor, thr_enable_listening, thr_listening|
+    def create_listen_task(delay: 0)
+      @task = Concurrent::ScheduledTask.new(delay, args: [@recipients, @running, @executor, @enable_listening, @listening], executor: @executor) do |thr_recipients, thr_running, thr_executor, thr_enable_listening, thr_listening|
         with_connection do
           begin
             Rails.application.executor.wrap do
@@ -188,7 +194,7 @@ module GoodJob # :nodoc:
               end
             end
 
-            while thr_executor.running?
+            while thr_executor.running? && thr_running.true?
               run_callbacks :tick do
                 wait_for_notify do |channel, payload|
                   next unless channel == CHANNEL
@@ -219,8 +225,9 @@ module GoodJob # :nodoc:
         end
       end
 
-      future.add_observer(self, :listen_observer)
-      future.execute
+      @task.add_observer(self, :listen_observer)
+      @task.execute
+      @task
     end
 
     def with_connection
@@ -263,5 +270,13 @@ module GoodJob # :nodoc:
       @connection_errors_count.value = 0
       @connection_errors_reported.make_false
     end
+
+    def synchronize(&block)
+      if @mutex.owned?
+        yield
+      else
+        @mutex.synchronize(&block)
+      end
+    end
   end
 end

data/lib/good_job/shared_executor.rb

@@ -0,0 +1,69 @@
+# frozen_string_literal: true
+
+module GoodJob
+  class SharedExecutor
+    MAX_THREADS = 2
+
+    # @!attribute [r] instances
+    #   @!scope class
+    #   List of all instantiated SharedExecutor in the current process.
+    #   @return [Array<GoodJob::SharedExecutor>, nil]
+    cattr_reader :instances, default: Concurrent::Array.new, instance_reader: false
+
+    attr_reader :executor
+
+    def initialize
+      self.class.instances << self
+      create_executor
+    end
+
+    def running?
+      @executor&.running?
+    end
+
+    def shutdown?
+      if @executor
+        @executor.shutdown?
+      else
+        true
+      end
+    end
+
+    # Shut down the SharedExecutor.
+    # Use {#shutdown?} to determine whether threads have stopped.
+    # @param timeout [Numeric, nil] Seconds to wait for active threads.
+    #   * +nil+, the scheduler will trigger a shutdown but not wait for it to complete.
+    #   * +-1+, the scheduler will wait until the shutdown is complete.
+    #   * +0+, the scheduler will immediately shutdown and stop any threads.
+    #   * A positive number will wait that many seconds before stopping any remaining active threads.
+    # @return [void]
+    def shutdown(timeout: -1)
+      return if @executor.nil? || @executor.shutdown?
+
+      @executor.shutdown if @executor.running?
+
+      if @executor.shuttingdown? && timeout # rubocop:disable Style/GuardClause
+        executor_wait = timeout.negative? ? nil : timeout
+        @executor.kill unless @executor.wait_for_termination(executor_wait)
+      end
+    end
+
+    def restart(timeout: -1)
+      shutdown(timeout: timeout) if running?
+      create_executor
+    end
+
+    private
+
+    def create_executor
+      @executor = Concurrent::ThreadPoolExecutor.new(
+        min_threads: 0,
+        max_threads: MAX_THREADS,
+        auto_terminate: true,
+        idletime: 60,
+        max_queue: 0,
+        fallback_policy: :discard
+      )
+    end
+  end
+end

data/lib/good_job/version.rb

@@ -2,7 +2,7 @@
 
 module GoodJob
   # GoodJob gem version.
-  VERSION = '3.16.2'
+  VERSION = '3.16.4'
 
   # GoodJob version as Gem::Version object
   GEM_VERSION = Gem::Version.new(VERSION)

data/lib/good_job.rb

@@ -31,6 +31,7 @@ require "good_job/notifier"
 require "good_job/poller"
 require "good_job/probe_server"
 require "good_job/scheduler"
+require "good_job/shared_executor"
 
 # GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails.
 #
@@ -246,5 +247,17 @@ module GoodJob
     end
   end
 
+  include ActiveSupport::Deprecation::DeprecatedConstantAccessor
+  deprecate_constant :Lockable, 'GoodJob::AdvisoryLockable', deprecator: deprecator
+
+  # Whether all GoodJob migrations have been applied.
+  # For use in tests/CI to validate GoodJob is up-to-date.
+  # @return [Boolean]
+  def self.migrated?
+    # Always update with the most recent migration check
+    GoodJob::Execution.reset_column_information
+    GoodJob::Execution.error_event_migrated?
+  end
+
   ActiveSupport.run_load_hooks(:good_job, self)
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 3.16.2
+  version: 3.16.4
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2023-07-13 00:00:00.000000000 Z
+date: 2023-07-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -343,9 +343,9 @@ files:
 - app/frontend/good_job/vendor/rails_ujs.js
 - app/frontend/good_job/vendor/stimulus.js
 - app/helpers/good_job/application_helper.rb
+- app/models/concerns/good_job/advisory_lockable.rb
 - app/models/concerns/good_job/error_events.rb
 - app/models/concerns/good_job/filterable.rb
-- app/models/concerns/good_job/lockable.rb
 - app/models/concerns/good_job/reportable.rb
 - app/models/good_job/active_record_parent_class.rb
 - app/models/good_job/base_execution.rb
@@ -437,6 +437,7 @@ files:
 - lib/good_job/poller.rb
 - lib/good_job/probe_server.rb
 - lib/good_job/scheduler.rb
+- lib/good_job/shared_executor.rb
 - lib/good_job/version.rb
 homepage: https://github.com/bensheldon/good_job
 licenses: