good_job 3.8.0 → 3.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d5d0c455a8d5d8f18fda38f1f9f126cc406250d188ee313d0352717c829cd126
-  data.tar.gz: 208b61ec3b48a7973b160093100b216eafbb96eee604e5e7edc2e7fb67bedfba
+  metadata.gz: 69560b4af2d1cab2a3783fb80573794066c3704aacafa7bd315a9dc6e1b75441
+  data.tar.gz: 55dff98e3c8ebbcebd5f09664d02f4ddd23b298c0ee2bc7cb163e43f6963de96
 SHA512:
-  metadata.gz: cac4f2c135e6c01cbdd6cd58c591d726ee8fbb66e623e3d1ed8bfb26e85e6b058e45bd60e3d2ba67b3133666df07d972db8eca75aa05df671b8284c01f45398f
-  data.tar.gz: 189ba8af30dc9e5a84c064090d5ede8a8f1da4bb56d2249f6ac4130ce3105c9c498f957ad05cf054f2697f83885570013c3c41ed5d197354c992b08518110656
+  metadata.gz: dd4763e6d473a22ea7b0f6f7b501249c7e5e29d06c1fdd9b3d028c923ba407fb485eea748fd37857a5eb09f5b26659d67f4792f4a66848542bfc1f47f7384577
+  data.tar.gz: 89e165d003d156fa16450762b8a8a269172509557ab59be60e7d54794d079c9963f8c11f45047ecfdea8e0963d901c8d9e1c297743392a05817d21d417c78dab

data/CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## [v3.9.0](https://github.com/bensheldon/good_job/tree/v3.9.0) (2023-01-31)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.8.0...v3.9.0)
+
+**Implemented enhancements:**
+
+- Abort enqueue when the concurrency limit is reached [\#820](https://github.com/bensheldon/good_job/pull/820) ([TAGraves](https://github.com/TAGraves))
+- Add bulk enqueue functionality [\#790](https://github.com/bensheldon/good_job/pull/790) ([julik](https://github.com/julik))
+
+**Merged pull requests:**
+
+- Bump alex-page/github-project-automation-plus from 0.8.2 to 0.8.3 [\#819](https://github.com/bensheldon/good_job/pull/819) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump concurrent-ruby from 1.1.10 to 1.2.0 [\#818](https://github.com/bensheldon/good_job/pull/818) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump rails from 6.1.7 to 6.1.7.2 [\#817](https://github.com/bensheldon/good_job/pull/817) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump selenium-webdriver from 4.7.1 to 4.8.0 [\#816](https://github.com/bensheldon/good_job/pull/816) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump rubocop from 1.43.0 to 1.44.1 [\#815](https://github.com/bensheldon/good_job/pull/815) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Ensure that anytime the Notifier uses autoloaded constants \(ActiveRecord\), they are wrapped with a Rails Executor [\#797](https://github.com/bensheldon/good_job/pull/797) ([bensheldon](https://github.com/bensheldon))
+- Remove support for Ruby 2.5 and JRuby 9.2; reactivate appraisal tests for Rails HEAD [\#756](https://github.com/bensheldon/good_job/pull/756) ([bensheldon](https://github.com/bensheldon))
+
 ## [v3.8.0](https://github.com/bensheldon/good_job/tree/v3.8.0) (2023-01-27)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.7.4...v3.8.0)
@@ -68,13 +87,16 @@
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.7.1...v3.7.2)
 
+**Fixed bugs:**
+
+- Ignore ActiveJob::DeserializationError when discarding jobs [\#771](https://github.com/bensheldon/good_job/pull/771) ([nickcampbell18](https://github.com/nickcampbell18))
+
 **Closed issues:**
 
 - Unable to discard failed jobs which crashed with `ActiveJob::DeserializationError` [\#770](https://github.com/bensheldon/good_job/issues/770)
 
 **Merged pull requests:**
 
-- Ignore ActiveJob::DeserializationError when discarding jobs [\#771](https://github.com/bensheldon/good_job/pull/771) ([nickcampbell18](https://github.com/nickcampbell18))
 - Bump rubocop from 1.39.0 to 1.40.0 [\#769](https://github.com/bensheldon/good_job/pull/769) ([dependabot[bot]](https://github.com/apps/dependabot))
 
 ## [v3.7.1](https://github.com/bensheldon/good_job/tree/v3.7.1) (2022-12-12)

data/README.md

@@ -58,6 +58,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
     - [Database connections](#database-connections)
         - [Production setup](#production-setup)
         - [Queue performance with Queue Select Limit](#queue-performance-with-queue-select-limit)
+    - [Bulk enqueue](#bulk-enqueue)
     - [Execute jobs async / in-process](#execute-jobs-async--in-process)
     - [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
     - [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
@@ -146,7 +147,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
 ## Compatibility
 
 - **Ruby on Rails:** 6.0+
-- **Ruby:** Ruby 2.5+. JRuby 9.2.13+
+- **Ruby:** Ruby 2.6+. JRuby 9.3+
 - **Postgres:** 10.0+
 
 ## Configuration
@@ -793,6 +794,26 @@ To explain where this value is used, here is the pseudo-query that GoodJob uses
   )
 ```
 
+### Bulk enqueue
+
+GoodJob's Bulk-enqueue functionality can buffer and enqueue multiple jobs at once, using a single INSERT statement. This can more performant when enqueuing a large number of jobs.
+
+```ruby
+# Capture jobs using `.perform_later`:
+active_jobs = GoodJob::Bulk.enqueue do
+  MyJob.perform_later
+  AnotherJob.perform_later
+  # If an exception is raised within this block, no jobs will be inserted.
+end
+
+# All ActiveJob instances are returned from GoodJob::Bulk.enqueue.
+# Jobs that have been successfully enqueued have a `provider_job_id` set.
+active_jobs.all?(&:provider_job_id)
+
+# Bulk enqueue ActiveJob instances directly without using `.perform_later`:
+GoodJob::Bulk.enqueue(MyJob.new, AnotherJob.new)
+```
+
 ### Execute jobs async / in-process
 
 GoodJob can execute jobs "async" in the same process as the web server (e.g. `bin/rails s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:

data/app/models/good_job/execution.rb

@@ -43,10 +43,10 @@ module GoodJob
       case string.first
       when '-'
         exclude_queues = true
-        string = string[1..-1]
+        string = string[1..]
       when '+'
         ordered_queues = true
-        string = string[1..-1]
+        string = string[1..]
       end
 
       queues = string.split(',').map(&:strip)
@@ -197,6 +197,28 @@ module GoodJob
       end
     end)
 
+    # Construct a GoodJob::Execution from an ActiveJob instance.
+    def self.build_for_enqueue(active_job, overrides = {})
+      execution_args = {
+        active_job_id: active_job.job_id,
+        queue_name: active_job.queue_name.presence || DEFAULT_QUEUE_NAME,
+        priority: active_job.priority || DEFAULT_PRIORITY,
+        serialized_params: active_job.serialize,
+        scheduled_at: active_job.scheduled_at,
+      }
+
+      execution_args[:concurrency_key] = active_job.good_job_concurrency_key if active_job.respond_to?(:good_job_concurrency_key)
+
+      if CurrentThread.cron_key
+        execution_args[:cron_key] = CurrentThread.cron_key
+        execution_args[:cron_at] = CurrentThread.cron_at
+      elsif CurrentThread.active_job_id && CurrentThread.active_job_id == active_job.job_id
+        execution_args[:cron_key] = CurrentThread.execution.cron_key
+      end
+
+      new(**execution_args.merge(overrides))
+    end
+
     # Finds the next eligible Execution, acquire an advisory lock related to it, and
     # executes the job.
     # @return [ExecutionResult, nil]
@@ -244,33 +266,19 @@ module GoodJob
     # @param active_job [ActiveJob::Base]
     #   The job to enqueue.
     # @param scheduled_at [Float]
-    #   Epoch timestamp when the job should be executed.
+    #   Epoch timestamp when the job should be executed, if blank will delegate to the ActiveJob instance
     # @param create_with_advisory_lock [Boolean]
     #   Whether to establish a lock on the {Execution} record after it is created.
+    # @param persist_immediately [Boolean]
+    #   Whether to save the record immediately or just initialize it with values. When bulk-inserting
+    #   jobs the caller takes care of the persistence and sets this parameter to `false`
     # @return [Execution]
     #   The new {Execution} instance representing the queued ActiveJob job.
     def self.enqueue(active_job, scheduled_at: nil, create_with_advisory_lock: false)
       ActiveSupport::Notifications.instrument("enqueue_job.good_job", { active_job: active_job, scheduled_at: scheduled_at, create_with_advisory_lock: create_with_advisory_lock }) do |instrument_payload|
-        execution_args = {
-          active_job_id: active_job.job_id,
-          queue_name: active_job.queue_name.presence || DEFAULT_QUEUE_NAME,
-          priority: active_job.priority || DEFAULT_PRIORITY,
-          serialized_params: active_job.serialize,
-          scheduled_at: scheduled_at,
-          create_with_advisory_lock: create_with_advisory_lock,
-        }
-
-        execution_args[:concurrency_key] = active_job.good_job_concurrency_key if active_job.respond_to?(:good_job_concurrency_key)
-
-        if CurrentThread.cron_key
-          execution_args[:cron_key] = CurrentThread.cron_key
-          execution_args[:cron_at] = CurrentThread.cron_at
-        elsif CurrentThread.active_job_id && CurrentThread.active_job_id == active_job.job_id
-          execution_args[:cron_key] = CurrentThread.execution.cron_key
-        end
-
-        execution = GoodJob::Execution.new(**execution_args)
+        execution = build_for_enqueue(active_job, { scheduled_at: scheduled_at })
 
+        execution.create_with_advisory_lock = create_with_advisory_lock
         instrument_payload[:execution] = execution
 
         execution.save!

data/lib/good_job/active_job_extensions/concurrency.rb

@@ -25,40 +25,13 @@ module GoodJob
         class_attribute :good_job_concurrency_config, instance_accessor: false, default: {}
         attr_writer :good_job_concurrency_key
 
-        around_enqueue do |job, block|
-          # Don't attempt to enforce concurrency limits with other queue adapters.
-          next(block.call) unless job.class.queue_adapter.is_a?(GoodJob::Adapter)
-
-          # Always allow jobs to be retried because the current job's execution will complete momentarily
-          next(block.call) if CurrentThread.active_job_id == job.job_id
-
-          # Only generate the concurrency key on the initial enqueue in case it is dynamic
-          job.good_job_concurrency_key ||= job._good_job_concurrency_key
-          key = job.good_job_concurrency_key
-          next(block.call) if key.blank?
-
-          enqueue_limit = job.class.good_job_concurrency_config[:enqueue_limit]
-          enqueue_limit = instance_exec(&enqueue_limit) if enqueue_limit.respond_to?(:call)
-          enqueue_limit = nil unless enqueue_limit.present? && (0...Float::INFINITY).cover?(enqueue_limit)
-
-          unless enqueue_limit
-            total_limit = job.class.good_job_concurrency_config[:total_limit]
-            total_limit = instance_exec(&total_limit) if total_limit.respond_to?(:call)
-            total_limit = nil unless total_limit.present? && (0...Float::INFINITY).cover?(total_limit)
+        if ActiveJob.gem_version >= Gem::Version.new("6.1.0")
+          before_enqueue do |job|
+            good_job_enqueue_concurrency_check(job, on_abort: -> { throw(:abort) }, on_enqueue: nil)
           end
-
-          limit = enqueue_limit || total_limit
-          next(block.call) unless limit
-
-          GoodJob::Execution.advisory_lock_key(key, function: "pg_advisory_lock") do
-            enqueue_concurrency = if enqueue_limit
-                                    GoodJob::Execution.where(concurrency_key: key).unfinished.advisory_unlocked.count
-                                  else
-                                    GoodJob::Execution.where(concurrency_key: key).unfinished.count
-                                  end
-
-            # The job has not yet been enqueued, so check if adding it will go over the limit
-            block.call unless (enqueue_concurrency + 1) > limit
+        else
+          around_enqueue do |job, block|
+            good_job_enqueue_concurrency_check(job, on_abort: nil, on_enqueue: block)
           end
         end
 
@@ -113,6 +86,50 @@ module GoodJob
         @good_job_concurrency_key || _good_job_concurrency_key
       end
 
+      private
+
+      def good_job_enqueue_concurrency_check(job, on_abort:, on_enqueue:)
+        # Don't attempt to enforce concurrency limits with other queue adapters.
+        return on_enqueue&.call unless job.class.queue_adapter.is_a?(GoodJob::Adapter)
+
+        # Always allow jobs to be retried because the current job's execution will complete momentarily
+        return on_enqueue&.call if CurrentThread.active_job_id == job.job_id
+
+        # Only generate the concurrency key on the initial enqueue in case it is dynamic
+        job.good_job_concurrency_key ||= job._good_job_concurrency_key
+        key = job.good_job_concurrency_key
+        return on_enqueue&.call if key.blank?
+
+        enqueue_limit = job.class.good_job_concurrency_config[:enqueue_limit]
+        enqueue_limit = instance_exec(&enqueue_limit) if enqueue_limit.respond_to?(:call)
+        enqueue_limit = nil unless enqueue_limit.present? && (0...Float::INFINITY).cover?(enqueue_limit)
+
+        unless enqueue_limit
+          total_limit = job.class.good_job_concurrency_config[:total_limit]
+          total_limit = instance_exec(&total_limit) if total_limit.respond_to?(:call)
+          total_limit = nil unless total_limit.present? && (0...Float::INFINITY).cover?(total_limit)
+        end
+
+        limit = enqueue_limit || total_limit
+        return on_enqueue&.call unless limit
+
+        GoodJob::Execution.advisory_lock_key(key, function: "pg_advisory_lock") do
+          enqueue_concurrency = if enqueue_limit
+                                  GoodJob::Execution.where(concurrency_key: key).unfinished.advisory_unlocked.count
+                                else
+                                  GoodJob::Execution.where(concurrency_key: key).unfinished.count
+                                end
+
+          # The job has not yet been enqueued, so check if adding it will go over the limit
+          if (enqueue_concurrency + 1) > limit
+            logger.info "Aborted enqueue of #{job.class.name} (Job ID: #{job.job_id}) because the concurrency key '#{key}' has reached its limit of #{limit} #{'job'.pluralize(limit)}"
+            on_abort&.call
+          else
+            on_enqueue&.call
+          end
+        end
+      end
+
       # Generates the concurrency key from the configuration
       # @return [Object] concurrency key
       def _good_job_concurrency_key

data/lib/good_job/adapter.rb

@@ -40,6 +40,70 @@ module GoodJob
       enqueue_at(active_job, nil)
     end
 
+    # Enqueues multiple ActiveJob instances at once
+    # @param active_jobs [Array<ActiveJob::Base>] jobs to be enqueued
+    # @return [Integer] number of jobs that were successfully enqueued
+    def enqueue_all(active_jobs)
+      active_jobs = Array(active_jobs)
+      return 0 if active_jobs.empty?
+
+      current_time = Time.current
+      executions = active_jobs.map do |active_job|
+        GoodJob::Execution.build_for_enqueue(active_job, {
+                                               id: SecureRandom.uuid,
+                                               created_at: current_time,
+                                               updated_at: current_time,
+                                             })
+      end
+
+      inline_executions = []
+      GoodJob::Execution.transaction(requires_new: true, joinable: false) do
+        results = GoodJob::Execution.insert_all(executions.map(&:attributes), returning: %w[id active_job_id]) # rubocop:disable Rails/SkipsModelValidations
+
+        job_id_to_provider_job_id = results.each_with_object({}) { |result, hash| hash[result['active_job_id']] = result['id'] }
+        active_jobs.each do |active_job|
+          active_job.provider_job_id = job_id_to_provider_job_id[active_job.job_id]
+        end
+        executions.each do |execution|
+          execution.instance_variable_set(:@new_record, false) if job_id_to_provider_job_id[execution.active_job_id]
+        end
+        executions = executions.select(&:persisted?) # prune unpersisted executions
+
+        if execute_inline?
+          inline_executions = executions.select { |execution| (execution.scheduled_at.nil? || execution.scheduled_at <= Time.current) }
+          inline_executions.each(&:advisory_lock!)
+        end
+      end
+
+      begin
+        until inline_executions.empty?
+          begin
+            inline_execution = inline_executions.shift
+            inline_result = inline_execution.perform
+          ensure
+            inline_execution.advisory_unlock
+            inline_execution.run_callbacks(:perform_unlocked)
+          end
+          raise inline_result.unhandled_error if inline_result.unhandled_error
+        end
+      ensure
+        inline_executions.each(&:advisory_unlock)
+      end
+
+      executions.reject(&:finished_at).group_by(&:queue_name).each do |queue_name, executions_by_queue|
+        executions_by_queue.group_by(&:scheduled_at).each do |scheduled_at, executions_by_queue_and_scheduled_at|
+          # TODO: have Adapter#create_thread handle state[:count] values
+          state = { queue_name: queue_name, count: executions_by_queue_and_scheduled_at.size }
+          state[:scheduled_at] = scheduled_at if scheduled_at
+
+          executed_locally = execute_async? && @scheduler&.create_thread(state)
+          Notifier.notify(state) unless executed_locally
+        end
+      end
+
+      active_jobs.count(&:provider_job_id)
+    end
+
     # Enqueues an ActiveJob job to be run at a specific time.
     # For use by Rails; you should generally not call this directly.
     # @param active_job [ActiveJob::Base] the job to be enqueued from +#perform_later+
@@ -47,8 +111,12 @@ module GoodJob
     # @return [GoodJob::Execution]
     def enqueue_at(active_job, timestamp)
       scheduled_at = timestamp ? Time.zone.at(timestamp) : nil
-      will_execute_inline = execute_inline? && (scheduled_at.nil? || scheduled_at <= Time.current)
 
+      # If there is a currently open Bulk in the current thread, direct the
+      # job there to be enqueued using enqueue_all
+      return if GoodJob::Bulk.capture(active_job, queue_adapter: self)
+
+      will_execute_inline = execute_inline? && (scheduled_at.nil? || scheduled_at <= Time.current)
       execution = GoodJob::Execution.enqueue(
         active_job,
         scheduled_at: scheduled_at,

data/lib/good_job/bulk.rb

@@ -0,0 +1,120 @@
+# frozen_string_literal: true
+require 'active_support/core_ext/module/attribute_accessors_per_thread'
+
+module GoodJob
+  module Bulk
+    Error = Class.new(StandardError)
+
+    # @!attribute [rw] current_buffer
+    #   @!scope class
+    #   Current buffer of jobs to be enqueued.
+    #   @return [GoodJob::Bulk::Buffer, nil]
+    thread_mattr_accessor :current_buffer
+
+    # Capture jobs to a buffer. Pass either a block, or specific Active Jobs to be buffered.
+    # @param active_jobs [Array<ActiveJob::Base>] Active Jobs to be buffered.
+    # @param queue_adapter Override the jobs implict queue adapter with an explicit one.
+    # @return [nil, Array<ActiveJob::Base>] The ActiveJob instances that have been buffered; nil if no active buffer
+    def self.capture(active_jobs = nil, queue_adapter: nil)
+      raise(ArgumentError, "Use either the block form or the argument form, not both") if block_given? && active_jobs
+
+      if block_given?
+        begin
+          original_buffer = current_buffer
+          self.current_buffer = Buffer.new
+          yield
+          current_buffer.active_jobs
+        ensure
+          self.current_buffer = original_buffer
+        end
+      else
+        current_buffer&.add(active_jobs, queue_adapter: queue_adapter)
+      end
+    end
+
+    # Capture jobs to a buffer and enqueue them all at once; or enqueue the current buffer.
+    # @param active_jobs [Array<ActiveJob::Base>] Active Jobs to be enqueued.
+    # @return [Array<ActiveJob::Base>] The ActiveJob instances that have been captured; check provider_job_id to confirm enqueued.
+    def self.enqueue(active_jobs = nil)
+      raise(ArgumentError, "Use either the block form or the argument form, not both") if block_given? && active_jobs
+
+      if block_given?
+        capture do
+          yield
+          current_buffer&.enqueue
+        end
+      elsif active_jobs.present?
+        buffer = Buffer.new
+        buffer.add(active_jobs)
+        buffer.enqueue
+        buffer.active_jobs
+      elsif current_buffer.present?
+        current_buffer.enqueue
+        current_buffer.active_jobs
+      end
+    end
+
+    # Temporarily unset the current buffer; used to enqueue buffered jobs.
+    # @return [void]
+    def self.unbuffer
+      original_buffer = current_buffer
+      self.current_buffer = nil
+      yield
+    ensure
+      self.current_buffer = original_buffer
+    end
+
+    class Buffer
+      def initialize
+        @values = []
+      end
+
+      def add(active_jobs, queue_adapter: nil)
+        new_pairs = Array(active_jobs).map do |active_job|
+          adapter = queue_adapter || active_job.class.queue_adapter
+          raise Error, "Jobs must have a Queue Adapter" unless adapter
+
+          [active_job, adapter]
+        end
+        @values.append(*new_pairs)
+
+        true
+      end
+
+      def enqueue
+        Bulk.unbuffer do
+          active_jobs_by_queue_adapter.each do |adapter, jobs|
+            jobs = jobs.reject(&:provider_job_id) # Do not re-enqueue already enqueued jobs
+
+            if adapter.respond_to?(:enqueue_all)
+              unbulkable_jobs, bulkable_jobs = jobs.partition do |job|
+                job.respond_to?(:good_job_concurrency_key) && job.good_job_concurrency_key &&
+                  (job.class.good_job_concurrency_config[:enqueue_limit] || job.class.good_job_concurrency_config[:total_limit])
+              end
+              adapter.enqueue_all(bulkable_jobs) if bulkable_jobs.any?
+            else
+              unbulkable_jobs = jobs
+            end
+
+            unbulkable_jobs.each do |job|
+              job.enqueue
+            rescue GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError
+              # ignore
+            end
+          end
+        end
+      end
+
+      def active_jobs_by_queue_adapter
+        @values.each_with_object({}) do |(job, adapter), memo|
+          memo[adapter] ||= []
+          memo[adapter] << job
+        end
+      end
+
+      def active_jobs
+        @values.map(&:first)
+      end
+    end
+  end
+end

data/lib/good_job/notifier.rb

@@ -168,35 +168,37 @@ module GoodJob # :nodoc:
       future = Concurrent::ScheduledTask.new(delay, args: [@recipients, executor, @listening], executor: @executor) do |thr_recipients, thr_executor, thr_listening|
         with_connection do
           begin
-            run_callbacks :listen do
-              ActiveSupport::Notifications.instrument("notifier_listen.good_job") do
-                connection.execute("LISTEN #{CHANNEL}")
+            Rails.application.executor.wrap do
+              run_callbacks :listen do
+                ActiveSupport::Notifications.instrument("notifier_listen.good_job") do
+                  connection.execute("LISTEN #{CHANNEL}")
+                end
+                thr_listening.make_true
               end
-              thr_listening.make_true
             end
 
-            ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
-              while thr_executor.running?
-                wait_for_notify do |channel, payload|
-                  next unless channel == CHANNEL
-
-                  ActiveSupport::Notifications.instrument("notifier_notified.good_job", { payload: payload })
-                  parsed_payload = JSON.parse(payload, symbolize_names: true)
-                  thr_recipients.each do |recipient|
-                    target, method_name = recipient.is_a?(Array) ? recipient : [recipient, :call]
-                    target.send(method_name, parsed_payload)
-                  end
-                end
+            while thr_executor.running?
+              wait_for_notify do |channel, payload|
+                next unless channel == CHANNEL
 
-                reset_connection_errors
+                ActiveSupport::Notifications.instrument("notifier_notified.good_job", { payload: payload })
+                parsed_payload = JSON.parse(payload, symbolize_names: true)
+                thr_recipients.each do |recipient|
+                  target, method_name = recipient.is_a?(Array) ? recipient : [recipient, :call]
+                  target.send(method_name, parsed_payload)
+                end
               end
+
+              reset_connection_errors
             end
           end
         ensure
-          run_callbacks :unlisten do
-            thr_listening.make_false
-            ActiveSupport::Notifications.instrument("notifier_unlisten.good_job") do
-              connection.execute("UNLISTEN *")
+          Rails.application.executor.wrap do
+            run_callbacks :unlisten do
+              thr_listening.make_false
+              ActiveSupport::Notifications.instrument("notifier_unlisten.good_job") do
+                connection.execute("UNLISTEN *")
+              end
             end
           end
         end
@@ -207,8 +209,10 @@ module GoodJob # :nodoc:
     end
 
     def with_connection
-      self.connection = Execution.connection_pool.checkout.tap do |conn|
-        Execution.connection_pool.remove(conn)
+      Rails.application.executor.wrap do
+        self.connection = Execution.connection_pool.checkout.tap do |conn|
+          Execution.connection_pool.remove(conn)
+        end
       end
       connection.execute("SET application_name = #{connection.quote(self.class.name)}")
 

data/lib/good_job/scheduler.rb

@@ -155,6 +155,19 @@ module GoodJob # :nodoc:
       if state
         return false unless performer.next?(state)
 
+        if state[:count]
+          # When given state for multiple jobs, try to create a thread for each one.
+          # Return true if a thread can be created for all of them, nil if partial or none.
+
+          state_without_count = state.without(:count)
+          result = state[:count].times do
+            value = create_thread(state_without_count)
+            break(value) unless value
+          end
+
+          return result.nil? ? nil : true
+        end
+
         if state[:scheduled_at]
           scheduled_at = if state[:scheduled_at].is_a? String
                            Time.zone.parse state[:scheduled_at]

data/lib/good_job/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 module GoodJob
   # GoodJob gem version.
-  VERSION = '3.8.0'
+  VERSION = '3.9.0'
 end

data/lib/good_job.rb

@@ -10,6 +10,7 @@ require "active_job/queue_adapters/good_job_adapter"
 require "good_job/active_job_extensions/concurrency"
 
 require "good_job/assignable_connection"
+require "good_job/bulk"
 require "good_job/cleanup_tracker"
 require "good_job/cli"
 require "good_job/configuration"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 3.8.0
+  version: 3.9.0
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2023-01-27 00:00:00.000000000 Z
+date: 2023-01-31 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -136,20 +136,6 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-- !ruby/object:Gem::Dependency
-  name: database_cleaner
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
 - !ruby/object:Gem::Dependency
   name: dotenv
   requirement: !ruby/object:Gem::Requirement
@@ -409,6 +395,7 @@ files:
 - lib/good_job/active_job_extensions/concurrency.rb
 - lib/good_job/adapter.rb
 - lib/good_job/assignable_connection.rb
+- lib/good_job/bulk.rb
 - lib/good_job/cleanup_tracker.rb
 - lib/good_job/cli.rb
 - lib/good_job/configuration.rb
@@ -451,14 +438,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: 2.5.0
+      version: 2.6.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.1.6
+rubygems_version: 3.4.5
 signing_key: 
 specification_version: 4
 summary: A multithreaded, Postgres-based ActiveJob backend for Ruby on Rails