good_job 3.7.4 → 3.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fda5fd7d15b1e93616e9dfa44dad4ed89abbad549c5c83d85d3d0fadcb5a3082
-  data.tar.gz: f114b7657cdd5d1845e9314688c0befaa3c1615df6b203fc6d9cd44e894e7071
+  metadata.gz: 69560b4af2d1cab2a3783fb80573794066c3704aacafa7bd315a9dc6e1b75441
+  data.tar.gz: 55dff98e3c8ebbcebd5f09664d02f4ddd23b298c0ee2bc7cb163e43f6963de96
 SHA512:
-  metadata.gz: d1b5bbbc7f8e407a74d65976996f29df1bea1271a327733a5c685804984c7d5e95d696e3ec823995e856e17e7c22f24edbf7c8425d7caf7ec4795f94f5038357
-  data.tar.gz: fad2cc50096beae6594326189defab7eec6527e2b2e297518e5c42f220eea3a6201ce7375864b95085ed0462d019da6829e8d34ee02e6ac042968a8f0096a059
+  metadata.gz: dd4763e6d473a22ea7b0f6f7b501249c7e5e29d06c1fdd9b3d028c923ba407fb485eea748fd37857a5eb09f5b26659d67f4792f4a66848542bfc1f47f7384577
+  data.tar.gz: 89e165d003d156fa16450762b8a8a269172509557ab59be60e7d54794d079c9963f8c11f45047ecfdea8e0963d901c8d9e1c297743392a05817d21d417c78dab

data/CHANGELOG.md

@@ -1,10 +1,60 @@
 # Changelog
 
+## [v3.9.0](https://github.com/bensheldon/good_job/tree/v3.9.0) (2023-01-31)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.8.0...v3.9.0)
+
+**Implemented enhancements:**
+
+- Abort enqueue when the concurrency limit is reached [\#820](https://github.com/bensheldon/good_job/pull/820) ([TAGraves](https://github.com/TAGraves))
+- Add bulk enqueue functionality [\#790](https://github.com/bensheldon/good_job/pull/790) ([julik](https://github.com/julik))
+
+**Merged pull requests:**
+
+- Bump alex-page/github-project-automation-plus from 0.8.2 to 0.8.3 [\#819](https://github.com/bensheldon/good_job/pull/819) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump concurrent-ruby from 1.1.10 to 1.2.0 [\#818](https://github.com/bensheldon/good_job/pull/818) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump rails from 6.1.7 to 6.1.7.2 [\#817](https://github.com/bensheldon/good_job/pull/817) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump selenium-webdriver from 4.7.1 to 4.8.0 [\#816](https://github.com/bensheldon/good_job/pull/816) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump rubocop from 1.43.0 to 1.44.1 [\#815](https://github.com/bensheldon/good_job/pull/815) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Ensure that anytime the Notifier uses autoloaded constants \(ActiveRecord\), they are wrapped with a Rails Executor [\#797](https://github.com/bensheldon/good_job/pull/797) ([bensheldon](https://github.com/bensheldon))
+- Remove support for Ruby 2.5 and JRuby 9.2; reactivate appraisal tests for Rails HEAD [\#756](https://github.com/bensheldon/good_job/pull/756) ([bensheldon](https://github.com/bensheldon))
+
+## [v3.8.0](https://github.com/bensheldon/good_job/tree/v3.8.0) (2023-01-27)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.7.4...v3.8.0)
+
+**Implemented enhancements:**
+
+- Capture and log ActiveJob IDs that are interrupted when Scheduler is forced to shutdown [\#794](https://github.com/bensheldon/good_job/pull/794) ([bensheldon](https://github.com/bensheldon))
+
+**Fixed bugs:**
+
+- Ensure Concurrency Keys are string-like and return a better error when they cannot be cast to a string [\#791](https://github.com/bensheldon/good_job/pull/791) ([Earlopain](https://github.com/Earlopain))
+
+**Closed issues:**
+
+- Work is not being picked up at the expected rate [\#802](https://github.com/bensheldon/good_job/issues/802)
+- Cleaning up preserved jobs only removes a subset of the jobs [\#801](https://github.com/bensheldon/good_job/issues/801)
+- Dashboard fails to execute JS on latest Firefox 108 [\#792](https://github.com/bensheldon/good_job/issues/792)
+- Concurrency key doesn't handle Hash: TypeError \(can't cast Hash\) [\#784](https://github.com/bensheldon/good_job/issues/784)
+
+**Merged pull requests:**
+
+- Bump fugit from 1.8.0 to 1.8.1 [\#808](https://github.com/bensheldon/good_job/pull/808) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump rubocop-rspec from 2.17.1 to 2.18.1 [\#807](https://github.com/bensheldon/good_job/pull/807) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump globalid from 1.0.0 to 1.0.1 [\#804](https://github.com/bensheldon/good_job/pull/804) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump rack from 2.2.4 to 2.2.6.2 [\#803](https://github.com/bensheldon/good_job/pull/803) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump nokogiri from 1.13.10 to 1.14.0 [\#800](https://github.com/bensheldon/good_job/pull/800) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump rubocop from 1.42.0 to 1.43.0 [\#799](https://github.com/bensheldon/good_job/pull/799) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump rubocop-rspec from 2.16.0 to 2.17.1 [\#798](https://github.com/bensheldon/good_job/pull/798) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Add French translation [\#795](https://github.com/bensheldon/good_job/pull/795) ([francois-ferrandis](https://github.com/francois-ferrandis))
+- Bump rubocop-rails from 2.17.3 to 2.17.4 [\#780](https://github.com/bensheldon/good_job/pull/780) ([dependabot[bot]](https://github.com/apps/dependabot))
+
 ## [v3.7.4](https://github.com/bensheldon/good_job/tree/v3.7.4) (2023-01-10)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.7.3...v3.7.4)
 
-**Merged pull requests:**
+**Fixed bugs:**
 
 - Update to es-module-shims v1.6.3 and use an inline script entry-point; remove script.js entrypoint; remove sourcemap references [\#793](https://github.com/bensheldon/good_job/pull/793) ([bensheldon](https://github.com/bensheldon))
 
@@ -37,13 +87,16 @@
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.7.1...v3.7.2)
 
+**Fixed bugs:**
+
+- Ignore ActiveJob::DeserializationError when discarding jobs [\#771](https://github.com/bensheldon/good_job/pull/771) ([nickcampbell18](https://github.com/nickcampbell18))
+
 **Closed issues:**
 
 - Unable to discard failed jobs which crashed with `ActiveJob::DeserializationError` [\#770](https://github.com/bensheldon/good_job/issues/770)
 
 **Merged pull requests:**
 
-- Ignore ActiveJob::DeserializationError when discarding jobs [\#771](https://github.com/bensheldon/good_job/pull/771) ([nickcampbell18](https://github.com/nickcampbell18))
 - Bump rubocop from 1.39.0 to 1.40.0 [\#769](https://github.com/bensheldon/good_job/pull/769) ([dependabot[bot]](https://github.com/apps/dependabot))
 
 ## [v3.7.1](https://github.com/bensheldon/good_job/tree/v3.7.1) (2022-12-12)

data/README.md

@@ -58,6 +58,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
     - [Database connections](#database-connections)
         - [Production setup](#production-setup)
         - [Queue performance with Queue Select Limit](#queue-performance-with-queue-select-limit)
+    - [Bulk enqueue](#bulk-enqueue)
     - [Execute jobs async / in-process](#execute-jobs-async--in-process)
     - [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
     - [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
@@ -146,7 +147,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
 ## Compatibility
 
 - **Ruby on Rails:** 6.0+
-- **Ruby:** Ruby 2.5+. JRuby 9.2.13+
+- **Ruby:** Ruby 2.6+. JRuby 9.3+
 - **Postgres:** 10.0+
 
 ## Configuration
@@ -793,6 +794,26 @@ To explain where this value is used, here is the pseudo-query that GoodJob uses
   )
 ```
 
+### Bulk enqueue
+
+GoodJob's Bulk-enqueue functionality can buffer and enqueue multiple jobs at once, using a single INSERT statement. This can more performant when enqueuing a large number of jobs.
+
+```ruby
+# Capture jobs using `.perform_later`:
+active_jobs = GoodJob::Bulk.enqueue do
+  MyJob.perform_later
+  AnotherJob.perform_later
+  # If an exception is raised within this block, no jobs will be inserted.
+end
+
+# All ActiveJob instances are returned from GoodJob::Bulk.enqueue.
+# Jobs that have been successfully enqueued have a `provider_job_id` set.
+active_jobs.all?(&:provider_job_id)
+
+# Bulk enqueue ActiveJob instances directly without using `.perform_later`:
+GoodJob::Bulk.enqueue(MyJob.new, AnotherJob.new)
+```
+
 ### Execute jobs async / in-process
 
 GoodJob can execute jobs "async" in the same process as the web server (e.g. `bin/rails s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:

data/app/controllers/good_job/jobs_controller.rb

@@ -80,7 +80,7 @@ module GoodJob
     def destroy
       @job = Job.find(params[:id])
       @job.destroy_job
-      redirect_to jobs_path, notice: "Job has been destroyed" # rubocop:disable Rails/I18nLocaleTexts
+      redirect_to jobs_path, notice: "Job has been destroyed"
     end
 
     private

data/app/models/good_job/execution.rb

@@ -43,10 +43,10 @@ module GoodJob
       case string.first
       when '-'
         exclude_queues = true
-        string = string[1..-1]
+        string = string[1..]
       when '+'
         ordered_queues = true
-        string = string[1..-1]
+        string = string[1..]
       end
 
       queues = string.split(',').map(&:strip)
@@ -197,6 +197,28 @@ module GoodJob
       end
     end)
 
+    # Construct a GoodJob::Execution from an ActiveJob instance.
+    def self.build_for_enqueue(active_job, overrides = {})
+      execution_args = {
+        active_job_id: active_job.job_id,
+        queue_name: active_job.queue_name.presence || DEFAULT_QUEUE_NAME,
+        priority: active_job.priority || DEFAULT_PRIORITY,
+        serialized_params: active_job.serialize,
+        scheduled_at: active_job.scheduled_at,
+      }
+
+      execution_args[:concurrency_key] = active_job.good_job_concurrency_key if active_job.respond_to?(:good_job_concurrency_key)
+
+      if CurrentThread.cron_key
+        execution_args[:cron_key] = CurrentThread.cron_key
+        execution_args[:cron_at] = CurrentThread.cron_at
+      elsif CurrentThread.active_job_id && CurrentThread.active_job_id == active_job.job_id
+        execution_args[:cron_key] = CurrentThread.execution.cron_key
+      end
+
+      new(**execution_args.merge(overrides))
+    end
+
     # Finds the next eligible Execution, acquire an advisory lock related to it, and
     # executes the job.
     # @return [ExecutionResult, nil]
@@ -212,6 +234,7 @@ module GoodJob
         break if execution.blank?
         break :unlocked unless execution&.executable?
 
+        yield(execution) if block_given?
         result = execution.perform
       end
       execution&.run_callbacks(:perform_unlocked)
@@ -243,33 +266,19 @@ module GoodJob
     # @param active_job [ActiveJob::Base]
     #   The job to enqueue.
     # @param scheduled_at [Float]
-    #   Epoch timestamp when the job should be executed.
+    #   Epoch timestamp when the job should be executed, if blank will delegate to the ActiveJob instance
     # @param create_with_advisory_lock [Boolean]
     #   Whether to establish a lock on the {Execution} record after it is created.
+    # @param persist_immediately [Boolean]
+    #   Whether to save the record immediately or just initialize it with values. When bulk-inserting
+    #   jobs the caller takes care of the persistence and sets this parameter to `false`
     # @return [Execution]
     #   The new {Execution} instance representing the queued ActiveJob job.
     def self.enqueue(active_job, scheduled_at: nil, create_with_advisory_lock: false)
       ActiveSupport::Notifications.instrument("enqueue_job.good_job", { active_job: active_job, scheduled_at: scheduled_at, create_with_advisory_lock: create_with_advisory_lock }) do |instrument_payload|
-        execution_args = {
-          active_job_id: active_job.job_id,
-          queue_name: active_job.queue_name.presence || DEFAULT_QUEUE_NAME,
-          priority: active_job.priority || DEFAULT_PRIORITY,
-          serialized_params: active_job.serialize,
-          scheduled_at: scheduled_at,
-          create_with_advisory_lock: create_with_advisory_lock,
-        }
-
-        execution_args[:concurrency_key] = active_job.good_job_concurrency_key if active_job.respond_to?(:good_job_concurrency_key)
-
-        if CurrentThread.cron_key
-          execution_args[:cron_key] = CurrentThread.cron_key
-          execution_args[:cron_at] = CurrentThread.cron_at
-        elsif CurrentThread.active_job_id && CurrentThread.active_job_id == active_job.job_id
-          execution_args[:cron_key] = CurrentThread.execution.cron_key
-        end
-
-        execution = GoodJob::Execution.new(**execution_args)
+        execution = build_for_enqueue(active_job, { scheduled_at: scheduled_at })
 
+        execution.create_with_advisory_lock = create_with_advisory_lock
         instrument_payload[:execution] = execution
 
         execution.save!

data/app/models/good_job/lockable.rb

@@ -193,6 +193,7 @@ module GoodJob
         binds = [
           ActiveRecord::Relation::QueryAttribute.new('key', key, ActiveRecord::Type::String.new),
         ]
+
         locked = connection.exec_query(pg_or_jdbc_query(query), 'GoodJob::Lockable Advisory Lock', binds).first['locked']
         return locked unless block_given?
         return nil unless locked

data/config/locales/fr.yml

@@ -0,0 +1,83 @@
+---
+fr:
+  datetime:
+    distance_in_words:
+      about_x_hours:
+        one: environ 1 heure
+        other: environ %{count} heures
+      about_x_months:
+        one: environ 1 heure
+        other: environ %{count} heures
+      about_x_years:
+        one: environ 1 an
+        other: environ %{count} ans
+      almost_x_years:
+        one: presque 1 an
+        other: presque %{count} ans
+      half_a_minute: une demi-minute
+      less_than_x_minutes:
+        one: moins d'une minute
+        other: moins de %{count} minutes
+      less_than_x_seconds:
+        one: moins d'une seconde
+        other: moins de %{count} secondes
+      over_x_years:
+        one: plus d'un an
+        other: plus de %{count} ans
+      x_days:
+        one: 1 jour
+        other: "%{count} jours"
+      x_minutes:
+        one: 1 minute
+        other: "%{count} minutes"
+      x_months:
+        one: 1 mois
+        other: "%{count} mois"
+      x_seconds:
+        one: 1 seconde
+        other: "%{count} secondes"
+      x_years:
+        one: 1 an
+        other: "%{count} ans"
+  duration:
+    hours: "%{hour}h %{min}m"
+    less_than_10_seconds: "%{sec}s"
+    milliseconds: "%{ms}ms"
+    minutes: "%{min}m %{sec}s"
+    seconds: "%{sec}s"
+  good_job:
+    shared:
+      footer:
+        last_update_html: Dernière mise à jour <time id="page-updated-at" datetime="%{time}">%{time}</time>
+        wording: N'oublie pas, toi aussi tu fais du bon boulot !
+      navbar:
+        cron_schedules: Cron
+        jobs: Jobs
+        live_poll: En direct
+        name: "GoodJob 👍"
+        processes: Processus
+    status:
+      discarded: Abandonnés
+      queued: À la file
+      retried: Réessayés
+      running: En cours
+      scheduled: Programmés
+      succeeded: Réussis
+  number:
+    format:
+      delimiter: " "
+      separator: ","
+    human:
+      decimal_units:
+        format: "%n%u"
+        units:
+          billion: B
+          million: M
+          quadrillion: q
+          thousand: k
+          trillion: T
+          unit: ''
+      format:
+        delimiter: " "
+        precision: 3
+        separator: ","

data/lib/good_job/active_job_extensions/concurrency.rb

@@ -4,6 +4,8 @@ module GoodJob
     module Concurrency
       extend ActiveSupport::Concern
 
+      VALID_TYPES = [String, Symbol, Numeric, Date, Time, TrueClass, FalseClass, NilClass].freeze
+
       class ConcurrencyExceededError < StandardError
         def backtrace
           [] # suppress backtrace
@@ -23,40 +25,13 @@ module GoodJob
         class_attribute :good_job_concurrency_config, instance_accessor: false, default: {}
         attr_writer :good_job_concurrency_key
 
-        around_enqueue do |job, block|
-          # Don't attempt to enforce concurrency limits with other queue adapters.
-          next(block.call) unless job.class.queue_adapter.is_a?(GoodJob::Adapter)
-
-          # Always allow jobs to be retried because the current job's execution will complete momentarily
-          next(block.call) if CurrentThread.active_job_id == job.job_id
-
-          enqueue_limit = job.class.good_job_concurrency_config[:enqueue_limit]
-          enqueue_limit = instance_exec(&enqueue_limit) if enqueue_limit.respond_to?(:call)
-          enqueue_limit = nil unless enqueue_limit.present? && (0...Float::INFINITY).cover?(enqueue_limit)
-
-          unless enqueue_limit
-            total_limit = job.class.good_job_concurrency_config[:total_limit]
-            total_limit = instance_exec(&total_limit) if total_limit.respond_to?(:call)
-            total_limit = nil unless total_limit.present? && (0...Float::INFINITY).cover?(total_limit)
+        if ActiveJob.gem_version >= Gem::Version.new("6.1.0")
+          before_enqueue do |job|
+            good_job_enqueue_concurrency_check(job, on_abort: -> { throw(:abort) }, on_enqueue: nil)
           end
-
-          limit = enqueue_limit || total_limit
-          next(block.call) unless limit
-
-          # Only generate the concurrency key on the initial enqueue in case it is dynamic
-          job.good_job_concurrency_key ||= job._good_job_concurrency_key
-          key = job.good_job_concurrency_key
-          next(block.call) if key.blank?
-
-          GoodJob::Execution.advisory_lock_key(key, function: "pg_advisory_lock") do
-            enqueue_concurrency = if enqueue_limit
-                                    GoodJob::Execution.where(concurrency_key: key).unfinished.advisory_unlocked.count
-                                  else
-                                    GoodJob::Execution.where(concurrency_key: key).unfinished.count
-                                  end
-
-            # The job has not yet been enqueued, so check if adding it will go over the limit
-            block.call unless (enqueue_concurrency + 1) > limit
+        else
+          around_enqueue do |job, block|
+            good_job_enqueue_concurrency_check(job, on_abort: nil, on_enqueue: block)
           end
         end
 
@@ -111,17 +86,60 @@ module GoodJob
         @good_job_concurrency_key || _good_job_concurrency_key
       end
 
+      private
+
+      def good_job_enqueue_concurrency_check(job, on_abort:, on_enqueue:)
+        # Don't attempt to enforce concurrency limits with other queue adapters.
+        return on_enqueue&.call unless job.class.queue_adapter.is_a?(GoodJob::Adapter)
+
+        # Always allow jobs to be retried because the current job's execution will complete momentarily
+        return on_enqueue&.call if CurrentThread.active_job_id == job.job_id
+
+        # Only generate the concurrency key on the initial enqueue in case it is dynamic
+        job.good_job_concurrency_key ||= job._good_job_concurrency_key
+        key = job.good_job_concurrency_key
+        return on_enqueue&.call if key.blank?
+
+        enqueue_limit = job.class.good_job_concurrency_config[:enqueue_limit]
+        enqueue_limit = instance_exec(&enqueue_limit) if enqueue_limit.respond_to?(:call)
+        enqueue_limit = nil unless enqueue_limit.present? && (0...Float::INFINITY).cover?(enqueue_limit)
+
+        unless enqueue_limit
+          total_limit = job.class.good_job_concurrency_config[:total_limit]
+          total_limit = instance_exec(&total_limit) if total_limit.respond_to?(:call)
+          total_limit = nil unless total_limit.present? && (0...Float::INFINITY).cover?(total_limit)
+        end
+
+        limit = enqueue_limit || total_limit
+        return on_enqueue&.call unless limit
+
+        GoodJob::Execution.advisory_lock_key(key, function: "pg_advisory_lock") do
+          enqueue_concurrency = if enqueue_limit
+                                  GoodJob::Execution.where(concurrency_key: key).unfinished.advisory_unlocked.count
+                                else
+                                  GoodJob::Execution.where(concurrency_key: key).unfinished.count
+                                end
+
+          # The job has not yet been enqueued, so check if adding it will go over the limit
+          if (enqueue_concurrency + 1) > limit
+            logger.info "Aborted enqueue of #{job.class.name} (Job ID: #{job.job_id}) because the concurrency key '#{key}' has reached its limit of #{limit} #{'job'.pluralize(limit)}"
+            on_abort&.call
+          else
+            on_enqueue&.call
+          end
+        end
+      end
+
       # Generates the concurrency key from the configuration
       # @return [Object] concurrency key
       def _good_job_concurrency_key
         key = self.class.good_job_concurrency_config[:key]
         return if key.blank?
 
-        if key.respond_to? :call
-          instance_exec(&key)
-        else
-          key
-        end
+        key = key.respond_to?(:call) ? instance_exec(&key) : key
+        raise TypeError, "Concurrency key must be a String; was a #{key.class}" unless VALID_TYPES.any? { |type| key.is_a?(type) }
+
+        key
       end
     end
   end

data/lib/good_job/adapter.rb

@@ -40,6 +40,70 @@ module GoodJob
       enqueue_at(active_job, nil)
     end
 
+    # Enqueues multiple ActiveJob instances at once
+    # @param active_jobs [Array<ActiveJob::Base>] jobs to be enqueued
+    # @return [Integer] number of jobs that were successfully enqueued
+    def enqueue_all(active_jobs)
+      active_jobs = Array(active_jobs)
+      return 0 if active_jobs.empty?
+
+      current_time = Time.current
+      executions = active_jobs.map do |active_job|
+        GoodJob::Execution.build_for_enqueue(active_job, {
+                                               id: SecureRandom.uuid,
+                                               created_at: current_time,
+                                               updated_at: current_time,
+                                             })
+      end
+
+      inline_executions = []
+      GoodJob::Execution.transaction(requires_new: true, joinable: false) do
+        results = GoodJob::Execution.insert_all(executions.map(&:attributes), returning: %w[id active_job_id]) # rubocop:disable Rails/SkipsModelValidations
+
+        job_id_to_provider_job_id = results.each_with_object({}) { |result, hash| hash[result['active_job_id']] = result['id'] }
+        active_jobs.each do |active_job|
+          active_job.provider_job_id = job_id_to_provider_job_id[active_job.job_id]
+        end
+        executions.each do |execution|
+          execution.instance_variable_set(:@new_record, false) if job_id_to_provider_job_id[execution.active_job_id]
+        end
+        executions = executions.select(&:persisted?) # prune unpersisted executions
+
+        if execute_inline?
+          inline_executions = executions.select { |execution| (execution.scheduled_at.nil? || execution.scheduled_at <= Time.current) }
+          inline_executions.each(&:advisory_lock!)
+        end
+      end
+
+      begin
+        until inline_executions.empty?
+          begin
+            inline_execution = inline_executions.shift
+            inline_result = inline_execution.perform
+          ensure
+            inline_execution.advisory_unlock
+            inline_execution.run_callbacks(:perform_unlocked)
+          end
+          raise inline_result.unhandled_error if inline_result.unhandled_error
+        end
+      ensure
+        inline_executions.each(&:advisory_unlock)
+      end
+
+      executions.reject(&:finished_at).group_by(&:queue_name).each do |queue_name, executions_by_queue|
+        executions_by_queue.group_by(&:scheduled_at).each do |scheduled_at, executions_by_queue_and_scheduled_at|
+          # TODO: have Adapter#create_thread handle state[:count] values
+          state = { queue_name: queue_name, count: executions_by_queue_and_scheduled_at.size }
+          state[:scheduled_at] = scheduled_at if scheduled_at
+
+          executed_locally = execute_async? && @scheduler&.create_thread(state)
+          Notifier.notify(state) unless executed_locally
+        end
+      end
+
+      active_jobs.count(&:provider_job_id)
+    end
+
     # Enqueues an ActiveJob job to be run at a specific time.
     # For use by Rails; you should generally not call this directly.
     # @param active_job [ActiveJob::Base] the job to be enqueued from +#perform_later+
@@ -47,8 +111,12 @@ module GoodJob
     # @return [GoodJob::Execution]
     def enqueue_at(active_job, timestamp)
       scheduled_at = timestamp ? Time.zone.at(timestamp) : nil
-      will_execute_inline = execute_inline? && (scheduled_at.nil? || scheduled_at <= Time.current)
 
+      # If there is a currently open Bulk in the current thread, direct the
+      # job there to be enqueued using enqueue_all
+      return if GoodJob::Bulk.capture(active_job, queue_adapter: self)
+
+      will_execute_inline = execute_inline? && (scheduled_at.nil? || scheduled_at <= Time.current)
       execution = GoodJob::Execution.enqueue(
         active_job,
         scheduled_at: scheduled_at,

data/lib/good_job/bulk.rb

@@ -0,0 +1,120 @@
+# frozen_string_literal: true
+require 'active_support/core_ext/module/attribute_accessors_per_thread'
+
+module GoodJob
+  module Bulk
+    Error = Class.new(StandardError)
+
+    # @!attribute [rw] current_buffer
+    #   @!scope class
+    #   Current buffer of jobs to be enqueued.
+    #   @return [GoodJob::Bulk::Buffer, nil]
+    thread_mattr_accessor :current_buffer
+
+    # Capture jobs to a buffer. Pass either a block, or specific Active Jobs to be buffered.
+    # @param active_jobs [Array<ActiveJob::Base>] Active Jobs to be buffered.
+    # @param queue_adapter Override the jobs implict queue adapter with an explicit one.
+    # @return [nil, Array<ActiveJob::Base>] The ActiveJob instances that have been buffered; nil if no active buffer
+    def self.capture(active_jobs = nil, queue_adapter: nil)
+      raise(ArgumentError, "Use either the block form or the argument form, not both") if block_given? && active_jobs
+
+      if block_given?
+        begin
+          original_buffer = current_buffer
+          self.current_buffer = Buffer.new
+          yield
+          current_buffer.active_jobs
+        ensure
+          self.current_buffer = original_buffer
+        end
+      else
+        current_buffer&.add(active_jobs, queue_adapter: queue_adapter)
+      end
+    end
+
+    # Capture jobs to a buffer and enqueue them all at once; or enqueue the current buffer.
+    # @param active_jobs [Array<ActiveJob::Base>] Active Jobs to be enqueued.
+    # @return [Array<ActiveJob::Base>] The ActiveJob instances that have been captured; check provider_job_id to confirm enqueued.
+    def self.enqueue(active_jobs = nil)
+      raise(ArgumentError, "Use either the block form or the argument form, not both") if block_given? && active_jobs
+
+      if block_given?
+        capture do
+          yield
+          current_buffer&.enqueue
+        end
+      elsif active_jobs.present?
+        buffer = Buffer.new
+        buffer.add(active_jobs)
+        buffer.enqueue
+        buffer.active_jobs
+      elsif current_buffer.present?
+        current_buffer.enqueue
+        current_buffer.active_jobs
+      end
+    end
+
+    # Temporarily unset the current buffer; used to enqueue buffered jobs.
+    # @return [void]
+    def self.unbuffer
+      original_buffer = current_buffer
+      self.current_buffer = nil
+      yield
+    ensure
+      self.current_buffer = original_buffer
+    end
+
+    class Buffer
+      def initialize
+        @values = []
+      end
+
+      def add(active_jobs, queue_adapter: nil)
+        new_pairs = Array(active_jobs).map do |active_job|
+          adapter = queue_adapter || active_job.class.queue_adapter
+          raise Error, "Jobs must have a Queue Adapter" unless adapter
+
+          [active_job, adapter]
+        end
+        @values.append(*new_pairs)
+
+        true
+      end
+
+      def enqueue
+        Bulk.unbuffer do
+          active_jobs_by_queue_adapter.each do |adapter, jobs|
+            jobs = jobs.reject(&:provider_job_id) # Do not re-enqueue already enqueued jobs
+
+            if adapter.respond_to?(:enqueue_all)
+              unbulkable_jobs, bulkable_jobs = jobs.partition do |job|
+                job.respond_to?(:good_job_concurrency_key) && job.good_job_concurrency_key &&
+                  (job.class.good_job_concurrency_config[:enqueue_limit] || job.class.good_job_concurrency_config[:total_limit])
+              end
+              adapter.enqueue_all(bulkable_jobs) if bulkable_jobs.any?
+            else
+              unbulkable_jobs = jobs
+            end
+
+            unbulkable_jobs.each do |job|
+              job.enqueue
+            rescue GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError
+              # ignore
+            end
+          end
+        end
+      end
+
+      def active_jobs_by_queue_adapter
+        @values.each_with_object({}) do |(job, adapter), memo|
+          memo[adapter] ||= []
+          memo[adapter] << job
+        end
+      end
+
+      def active_jobs
+        @values.map(&:first)
+      end
+    end
+  end
+end

data/lib/good_job/job_performer.rb

@@ -10,6 +10,8 @@ module GoodJob
   # The JobPerformer must be safe to execute across multiple threads.
   #
   class JobPerformer
+    cattr_accessor :performing_active_job_ids, default: Concurrent::Set.new
+
     # @param queue_string [String] Queues to execute jobs from
     def initialize(queue_string)
       @queue_string = queue_string
@@ -24,7 +26,13 @@ module GoodJob
     # Perform the next eligible job
     # @return [Object, nil] Returns job result or +nil+ if no job was found
     def next
-      job_query.perform_with_advisory_lock(parsed_queues: parsed_queues, queue_select_limit: GoodJob.configuration.queue_select_limit)
+      active_job_id = nil
+      job_query.perform_with_advisory_lock(parsed_queues: parsed_queues, queue_select_limit: GoodJob.configuration.queue_select_limit) do |execution|
+        active_job_id = execution.active_job_id
+        performing_active_job_ids << active_job_id
+      end
+    ensure
+      performing_active_job_ids.delete(active_job_id)
     end
 
     # Tests whether this performer should be used in GoodJob's current state.

data/lib/good_job/log_subscriber.rb

@@ -85,6 +85,19 @@ module GoodJob
       end
     end
 
+    def scheduler_shutdown_kill(event)
+      process_id = event.payload[:process_id]
+
+      warn(tags: [process_id]) do
+        active_job_ids = event.payload.fetch(:active_job_ids, [])
+        if active_job_ids.any?
+          "GoodJob scheduler has been killed. The following Active Jobs were interrupted: #{active_job_ids.join(' ')}"
+        else
+          "GoodJob scheduler has been killed."
+        end
+      end
+    end
+
     # @!macro notification_responder
     def scheduler_restart_pools(event)
       process_id = event.payload[:process_id]

data/lib/good_job/notifier.rb

@@ -168,35 +168,37 @@ module GoodJob # :nodoc:
       future = Concurrent::ScheduledTask.new(delay, args: [@recipients, executor, @listening], executor: @executor) do |thr_recipients, thr_executor, thr_listening|
         with_connection do
           begin
-            run_callbacks :listen do
-              ActiveSupport::Notifications.instrument("notifier_listen.good_job") do
-                connection.execute("LISTEN #{CHANNEL}")
+            Rails.application.executor.wrap do
+              run_callbacks :listen do
+                ActiveSupport::Notifications.instrument("notifier_listen.good_job") do
+                  connection.execute("LISTEN #{CHANNEL}")
+                end
+                thr_listening.make_true
               end
-              thr_listening.make_true
             end
 
-            ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
-              while thr_executor.running?
-                wait_for_notify do |channel, payload|
-                  next unless channel == CHANNEL
-
-                  ActiveSupport::Notifications.instrument("notifier_notified.good_job", { payload: payload })
-                  parsed_payload = JSON.parse(payload, symbolize_names: true)
-                  thr_recipients.each do |recipient|
-                    target, method_name = recipient.is_a?(Array) ? recipient : [recipient, :call]
-                    target.send(method_name, parsed_payload)
-                  end
-                end
+            while thr_executor.running?
+              wait_for_notify do |channel, payload|
+                next unless channel == CHANNEL
 
-                reset_connection_errors
+                ActiveSupport::Notifications.instrument("notifier_notified.good_job", { payload: payload })
+                parsed_payload = JSON.parse(payload, symbolize_names: true)
+                thr_recipients.each do |recipient|
+                  target, method_name = recipient.is_a?(Array) ? recipient : [recipient, :call]
+                  target.send(method_name, parsed_payload)
+                end
               end
+
+              reset_connection_errors
             end
           end
         ensure
-          run_callbacks :unlisten do
-            thr_listening.make_false
-            ActiveSupport::Notifications.instrument("notifier_unlisten.good_job") do
-              connection.execute("UNLISTEN *")
+          Rails.application.executor.wrap do
+            run_callbacks :unlisten do
+              thr_listening.make_false
+              ActiveSupport::Notifications.instrument("notifier_unlisten.good_job") do
+                connection.execute("UNLISTEN *")
+              end
             end
           end
         end
@@ -207,8 +209,10 @@ module GoodJob # :nodoc:
     end
 
     def with_connection
-      self.connection = Execution.connection_pool.checkout.tap do |conn|
-        Execution.connection_pool.remove(conn)
+      Rails.application.executor.wrap do
+        self.connection = Execution.connection_pool.checkout.tap do |conn|
+          Execution.connection_pool.remove(conn)
+        end
       end
       connection.execute("SET application_name = #{connection.quote(self.class.name)}")
 

data/lib/good_job/scheduler.rb

@@ -121,7 +121,11 @@ module GoodJob # :nodoc:
 
         if executor.shuttingdown? && timeout
           executor_wait = timeout.negative? ? nil : timeout
-          executor.kill unless executor.wait_for_termination(executor_wait)
+
+          unless executor.wait_for_termination(executor_wait)
+            instrument("scheduler_shutdown_kill", { active_job_ids: @performer.performing_active_job_ids.to_a })
+            executor.kill
+          end
         end
       end
     end
@@ -151,6 +155,19 @@ module GoodJob # :nodoc:
       if state
         return false unless performer.next?(state)
 
+        if state[:count]
+          # When given state for multiple jobs, try to create a thread for each one.
+          # Return true if a thread can be created for all of them, nil if partial or none.
+
+          state_without_count = state.without(:count)
+          result = state[:count].times do
+            value = create_thread(state_without_count)
+            break(value) unless value
+          end
+
+          return result.nil? ? nil : true
+        end
+
         if state[:scheduled_at]
           scheduled_at = if state[:scheduled_at].is_a? String
                            Time.zone.parse state[:scheduled_at]

data/lib/good_job/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 module GoodJob
   # GoodJob gem version.
-  VERSION = '3.7.4'
+  VERSION = '3.9.0'
 end

data/lib/good_job.rb

@@ -10,6 +10,7 @@ require "active_job/queue_adapters/good_job_adapter"
 require "good_job/active_job_extensions/concurrency"
 
 require "good_job/assignable_connection"
+require "good_job/bulk"
 require "good_job/cleanup_tracker"
 require "good_job/cli"
 require "good_job/configuration"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 3.7.4
+  version: 3.9.0
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2023-01-10 00:00:00.000000000 Z
+date: 2023-01-31 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -136,20 +136,6 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-- !ruby/object:Gem::Dependency
-  name: database_cleaner
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
 - !ruby/object:Gem::Dependency
   name: dotenv
   requirement: !ruby/object:Gem::Requirement
@@ -392,6 +378,7 @@ files:
 - app/views/layouts/good_job/application.html.erb
 - config/locales/en.yml
 - config/locales/es.yml
+- config/locales/fr.yml
 - config/locales/nl.yml
 - config/locales/ru.yml
 - config/locales/ua.yml
@@ -408,6 +395,7 @@ files:
 - lib/good_job/active_job_extensions/concurrency.rb
 - lib/good_job/adapter.rb
 - lib/good_job/assignable_connection.rb
+- lib/good_job/bulk.rb
 - lib/good_job/cleanup_tracker.rb
 - lib/good_job/cli.rb
 - lib/good_job/configuration.rb
@@ -450,14 +438,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: 2.5.0
+      version: 2.6.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.1.6
+rubygems_version: 3.4.5
 signing_key: 
 specification_version: 4
 summary: A multithreaded, Postgres-based ActiveJob backend for Ruby on Rails