good_job 1.12.0 → 1.13.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bdf4b1eebb1224acd53f07753d5cda9cb9fde7a44baf300824d7816da94980f2
-  data.tar.gz: 4b90c9cd8133f179f87094683218ed1a1aed3582512169dd914a00dd62a81628
+  metadata.gz: 945c5a01f5e0e936bbd863ce9cdcd8ca3956d1b0e8462932821ca3d502fc255f
+  data.tar.gz: e0503d0b3c89d43fd7a03be68f34f578100baf9b0ad01f1a7e40a6833c35a929
 SHA512:
-  metadata.gz: e95350cb8bd69198231d8030cc07adb9c87649499d30b6aa467ad4a06244920f2e950f46eae891e6963532fa879000959fcaa2eddf6093b8cd9d960eefef9abb
-  data.tar.gz: 1e09cf2851ca64fc34407e29f5afbac5bd7a72f072cef7c22798458b9ebc5678e287abbb03ee0d070cea8aa0665d85b017b94b68737e3e26ce7c0f67905c66e0
+  metadata.gz: 16326be1563033a3f94d64fa926a85d58cf9ec5026f1aed1fc1471d0b7e301c0e7f69101129964d91ca7cfafd51146fba71c5b7f2ce8643d152590ce8e826a05
+  data.tar.gz: ffdd7c4274c04a14d44af48d0f590fa10b984787e88e00359584911521c406451209ccd0525875ab80646bb7a9ea16fc062159b3a1513231106c7d0eb1a2e23a

data/CHANGELOG.md

@@ -1,5 +1,64 @@
 # Changelog
 
+## [v1.13.1](https://github.com/bensheldon/good_job/tree/v1.13.1) (2021-08-18)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.13.0...v1.13.1)
+
+**Fixed bugs:**
+
+- Don’t attempt to enforce concurrency limits with other queue adapters [\#333](https://github.com/bensheldon/good_job/pull/333) ([codyrobbins](https://github.com/codyrobbins))
+
+## [v1.13.0](https://github.com/bensheldon/good_job/tree/v1.13.0) (2021-08-18)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.12.2...v1.13.0)
+
+**Implemented enhancements:**
+
+- Track if a GoodJob::Job has been subsequently retried [\#331](https://github.com/bensheldon/good_job/pull/331) ([bensheldon](https://github.com/bensheldon))
+- Wrap and truncate error message, which can be a huge text [\#294](https://github.com/bensheldon/good_job/pull/294) ([morgoth](https://github.com/morgoth))
+
+**Closed issues:**
+
+- Add hyphen to lock string. e.g. "table\_name-column" instead of "table\_namecolumn [\#334](https://github.com/bensheldon/good_job/issues/334)
+- Optimize db indexes in advance of v2.0.0 [\#332](https://github.com/bensheldon/good_job/issues/332)
+- wait\_until in development? [\#330](https://github.com/bensheldon/good_job/issues/330)
+- Race conditions in ActiveJob concurrency extension [\#325](https://github.com/bensheldon/good_job/issues/325)
+- Store in database if a job has been ActiveJob retried [\#321](https://github.com/bensheldon/good_job/issues/321)
+- Revisit and embrace concurrency control, scheduled jobs, and other extensions of ActiveJob [\#255](https://github.com/bensheldon/good_job/issues/255)
+- Why 1 million jobs per day? [\#222](https://github.com/bensheldon/good_job/issues/222)
+
+## [v1.12.2](https://github.com/bensheldon/good_job/tree/v1.12.2) (2021-08-13)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.12.1...v1.12.2)
+
+**Fixed bugs:**
+
+- Fixes for race conditions in ActiveJob concurrency extension [\#326](https://github.com/bensheldon/good_job/pull/326) ([codyrobbins](https://github.com/codyrobbins))
+
+**Merged pull requests:**
+
+- On gem release, add instructions to author a Github Release [\#324](https://github.com/bensheldon/good_job/pull/324) ([bensheldon](https://github.com/bensheldon))
+
+## [v1.12.1](https://github.com/bensheldon/good_job/tree/v1.12.1) (2021-08-05)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.12.0...v1.12.1)
+
+**Fixed bugs:**
+
+- Ensure CLI can shutdown cleanly with multiple queues and timeout [\#319](https://github.com/bensheldon/good_job/pull/319) ([bensheldon](https://github.com/bensheldon))
+
+**Closed issues:**
+
+- Setting a shutdown timeout causes the CLI executor to throw an exception on shutdown. [\#318](https://github.com/bensheldon/good_job/issues/318)
+- PgBouncer and prepared statements [\#269](https://github.com/bensheldon/good_job/issues/269)
+- Question about locking internals [\#212](https://github.com/bensheldon/good_job/issues/212)
+- Encoding::UndefinedConversionError \("\xE2" from ASCII-8BIT to UTF-8\) [\#198](https://github.com/bensheldon/good_job/issues/198)
+- tools for managing a 'fleet' of processes [\#150](https://github.com/bensheldon/good_job/issues/150)
+
+**Merged pull requests:**
+
+- Fix Readme lint warnings [\#320](https://github.com/bensheldon/good_job/pull/320) ([bensheldon](https://github.com/bensheldon))
+
 ## [v1.12.0](https://github.com/bensheldon/good_job/tree/v1.12.0) (2021-07-27)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.11.3...v1.12.0)
@@ -9,7 +68,7 @@
 - Add the ability to schedule repeating / recurring / cron-like jobs [\#53](https://github.com/bensheldon/good_job/issues/53)
 - Add cron-like support for recurring/repeating jobs [\#297](https://github.com/bensheldon/good_job/pull/297) ([bensheldon](https://github.com/bensheldon))
 
-**Merged pull requests:**
+**Fixed bugs:**
 
 - Place Dashboard shared view partials under `good_job` namespace [\#310](https://github.com/bensheldon/good_job/pull/310) ([bensheldon](https://github.com/bensheldon))
 - Ensure Dashboard inline javascript has CSP nonce for strict Content-Security Policy [\#309](https://github.com/bensheldon/good_job/pull/309) ([bensheldon](https://github.com/bensheldon))

data/README.md

@@ -10,7 +10,7 @@ GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails.
 - **Designed for ActiveJob.** Complete support for [async, queues, delays, priorities, timeouts, and retries](https://edgeguides.rubyonrails.org/active_job_basics.html) with near-zero configuration.
 - **Built for Rails.** Fully adopts Ruby on Rails [threading and code execution guidelines](https://guides.rubyonrails.org/threading_and_code_execution.html) with [Concurrent::Ruby](https://github.com/ruby-concurrency/concurrent-ruby).
 - **Backed by Postgres.** Relies upon Postgres integrity, session-level Advisory Locks to provide run-once safety and stay within the limits of `schema.rb`, and LISTEN/NOTIFY to reduce queuing latency.
-- **For most workloads.** Targets full-stack teams, economy-minded solo developers, and applications that enqueue less than 1-million jobs/day.
+- **For most workloads.** Targets full-stack teams, economy-minded solo developers, and applications that enqueue 1-million jobs/day and more.
 
 For more of the story of GoodJob, read the [introductory blog post](https://island94.org/2020/07/introducing-goodjob-1-0).
 
@@ -52,6 +52,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
     - [Execute jobs async / in-process](#execute-jobs-async--in-process)
     - [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
     - [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
+    - [PgBouncer compatibility](#pgbouncer-compatibility)
 - [Contribute](#contribute)
     - [Gem development](#gem-development)
     - [Release](#release)
@@ -334,7 +335,7 @@ GoodJob includes a Dashboard as a mountable `Rails::Engine`.
 
 ### ActiveJob concurrency
 
-GoodJob can extend ActiveJob to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_.
+GoodJob can extend ActiveJob to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_. Limiting concurrency can help prevent duplicate, double or unecessary jobs from being enqueued, or race conditions when performing, for example when interacting with 3rd-party APIs.
 
 **Note:** Limiting concurrency at _enqueue_ requires Rails 6.0+ because Rails 5.2 does not support `throw :abort` in ActiveJob callbacks.
 
@@ -361,6 +362,13 @@ class MyJob < ApplicationJob
 end
 ```
 
+When testing, the resulting concurrency key value can be inspected:
+
+```ruby
+job = MyJob.perform_later("Alice")
+job.good_job_concurrency_key #=> "Unique-Alice"
+```
+
 ### Cron-style repeating/recurring jobs
 
 GoodJob can enqueue jobs on a recurring basis that can be used as a replacement for cron.
@@ -680,6 +688,43 @@ It is also necessary to delete these preserved jobs from the database after a ce
     $ bundle exec good_job cleanup_preserved_jobs --before-seconds-ago=86400
     ```
 
+### PgBouncer compatibility
+
+GoodJob is not compatible with PgBouncer in _transaction_ mode, but is compatible with PgBouncer's _connection_ mode. GoodJob uses connection-based advisory locks and LISTEN/NOTIFY, both of which require full database connections.
+
+A workaround to this limitation is to make a direct database connection available to GoodJob. With Rails 6.0's support for [multiple databases](https://guides.rubyonrails.org/active_record_multiple_databases.html), a direct connection to the database can be configured:
+
+1. Define a direct connection to your database that is not proxied through PgBouncer, for example:
+
+    ```yml
+    # config/database.yml
+
+    production:
+      primary:
+        url: postgres://pgbouncer_host/my_database
+      primary_direct:
+        url: postgres://database_host/my_database
+    ```
+
+1. Create a new ActiveRecord base class that uses the direct database connection
+
+    ```ruby
+    # app/models/application_direct_record.rb
+
+    class ApplicationDirectRecord < ActiveRecord::Base
+      self.abstract_class = true
+      connects_to database: :primary_direct
+    end
+    ```
+
+1. Configure GoodJob to use the newly created ActiveRecord base class:
+
+    ```ruby
+    # config/initializers/good_job.rb
+
+    GoodJob.active_record_parent_class = "ApplicationDirectRecord"
+    ```
+
 ## Contribute
 
 Contributions are welcomed and appreciated 🙏

data/engine/app/views/good_job/shared/_jobs_table.erb

@@ -19,7 +19,7 @@
             <td><%= job.serialized_params['job_class'] %></td>
             <td><%= job.queue_name %></td>
             <td><%= job.scheduled_at || job.created_at %></td>
-            <td><%= job.error %></td>
+            <td class="text-break"><%= truncate(job.error, length: 1_000) %></td>
             <td>
               <%= tag.button "Preview", type: "button", class: "btn btn-sm btn-outline-primary", role: "button",
                 data: {bs_toggle: "collapse", bs_target: "##{dom_id(job, 'params')}"},

data/lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb

@@ -16,6 +16,7 @@ class CreateGoodJobs < ActiveRecord::Migration[5.2]
       t.uuid :active_job_id
       t.text :concurrency_key
       t.text :cron_key
+      t.uuid :retried_good_job_id
     end
 
     add_index :good_jobs, :scheduled_at, where: "(finished_at IS NULL)", name: "index_good_jobs_on_scheduled_at"

data/lib/generators/good_job/templates/update/migrations/04_add_retried_good_job_id_to_good_jobs.rb

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+class AddRetriedGoodJobIdToGoodJobs < ActiveRecord::Migration[5.2]
+  def change
+    reversible do |dir|
+      dir.up do
+        # Ensure this incremental update migration is idempotent
+        # with monolithic install migration.
+        return if connection.column_exists?(:good_jobs, :retried_good_job_id)
+      end
+    end
+
+    add_column :good_jobs, :retried_good_job_id, :uuid
+  end
+end

data/lib/good_job.rb

@@ -126,13 +126,13 @@ module GoodJob
     _shutdown_all(_executables, :restart, timeout: timeout)
   end
 
-  # Sends +#shutdown+ or +#restart+ to executable objects ({GoodJob::Notifier}, {GoodJob::Poller}, {GoodJob::Scheduler})
-  # @param executables [Array<Notifier, Poller, Scheduler, MultiScheduler>] Objects to shut down.
+  # Sends +#shutdown+ or +#restart+ to executable objects ({GoodJob::Notifier}, {GoodJob::Poller}, {GoodJob::Scheduler}, {GoodJob::MultiScheduler}, {GoodJob::CronManager})
+  # @param executables [Array<Notifier, Poller, Scheduler, MultiScheduler, CronManager>] Objects to shut down.
   # @param method_name [:symbol] Method to call, e.g. +:shutdown+ or +:restart+.
   # @param timeout [nil,Numeric]
   # @return [void]
   def self._shutdown_all(executables, method_name = :shutdown, timeout: -1)
-    if timeout.positive?
+    if timeout.is_a?(Numeric) && timeout.positive?
       executables.each { |executable| executable.send(method_name, timeout: nil) }
 
       stop_at = Time.current + timeout

data/lib/good_job/active_job_extensions/concurrency.rb

@@ -9,21 +9,24 @@ module GoodJob
       included do
         class_attribute :good_job_concurrency_config, instance_accessor: false, default: {}
 
-        before_enqueue do |job|
+        around_enqueue do |job, block|
+          # Don't attempt to enforce concurrency limits with other queue adapters.
+          next(block.call) unless job.class.queue_adapter.is_a?(GoodJob::Adapter)
+
           # Always allow jobs to be retried because the current job's execution will complete momentarily
-          next if CurrentExecution.active_job_id == job.job_id
+          next(block.call) if CurrentExecution.active_job_id == job.job_id
 
           limit = job.class.good_job_concurrency_config.fetch(:enqueue_limit, Float::INFINITY)
-          next if limit.blank? || (0...Float::INFINITY).exclude?(limit)
+          next(block.call) if limit.blank? || (0...Float::INFINITY).exclude?(limit)
 
           key = job.good_job_concurrency_key
-          next if key.blank?
+          next(block.call) if key.blank?
 
           GoodJob::Job.new.with_advisory_lock(key: key, function: "pg_advisory_lock") do
             # TODO: Why is `unscoped` necessary? Nested scope is bleeding into subsequent query?
             enqueue_concurrency = GoodJob::Job.unscoped.where(concurrency_key: key).unfinished.count
             # The job has not yet been enqueued, so check if adding it will go over the limit
-            throw :abort if enqueue_concurrency + 1 > limit
+            block.call unless enqueue_concurrency + 1 > limit
           end
         end
 
@@ -34,6 +37,9 @@ module GoodJob
         )
 
         before_perform do |job|
+          # Don't attempt to enforce concurrency limits with other queue adapters.
+          next unless job.class.queue_adapter.is_a?(GoodJob::Adapter)
+
           limit = job.class.good_job_concurrency_config.fetch(:perform_limit, Float::INFINITY)
           next if limit.blank? || (0...Float::INFINITY).exclude?(limit)
 
@@ -41,9 +47,9 @@ module GoodJob
           next if key.blank?
 
           GoodJob::Job.new.with_advisory_lock(key: key, function: "pg_advisory_lock") do
-            perform_concurrency = GoodJob::Job.unscoped.where(concurrency_key: key).advisory_locked.count
+            allowed_active_job_ids = GoodJob::Job.unscoped.where(concurrency_key: key).advisory_locked.order(Arel.sql("COALESCE(performed_at, scheduled_at, created_at) ASC")).limit(limit).pluck(:active_job_id)
             # The current job has already been locked and will appear in the previous query
-            raise GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError if perform_concurrency > limit
+            raise GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError unless allowed_active_job_ids.include? job.job_id
           end
         end
       end

data/lib/good_job/current_execution.rb

@@ -5,12 +5,6 @@ module GoodJob
   # Thread-local attributes for passing values from Instrumentation.
   # (Cannot use ActiveSupport::CurrentAttributes because ActiveJob resets it)
   module CurrentExecution
-    # @!attribute [rw] active_job_id
-    #   @!scope class
-    #   ActiveJob ID
-    #   @return [String, nil]
-    thread_mattr_accessor :active_job_id
-
     # @!attribute [rw] cron_key
     #   @!scope class
     #   Cron Key
@@ -29,14 +23,26 @@ module GoodJob
     #   @return [Exception, nil]
     thread_mattr_accessor :error_on_retry
 
+    # @!attribute [rw] good_job
+    #   @!scope class
+    #   Cron Key
+    #   @return [GoodJob::Job, nil]
+    thread_mattr_accessor :good_job
+
     # Resets attributes
     # @return [void]
     def self.reset
-      self.active_job_id = nil
+      self.cron_key = nil
+      self.good_job = nil
       self.error_on_discard = nil
       self.error_on_retry = nil
     end
 
+    # @return [String] UUID of the currently executing GoodJob::Job
+    def self.active_job_id
+      good_job&.active_job_id
+    end
+
     # @return [Integer] Current process ID
     def self.process_id
       Process.pid

data/lib/good_job/job.rb

@@ -52,6 +52,20 @@ module GoodJob
       end
     end
 
+    def self._migration_pending_warning
+      ActiveSupport::Deprecation.warn(<<~DEPRECATION)
+        GoodJob has pending database migrations. To create the migration files, run:
+
+            rails generate good_job:update
+
+        To apply the migration files, run:
+
+            rails db:migrate
+
+      DEPRECATION
+      nil
+    end
+
     # Get Jobs with given class name
     # @!method with_job_class
     # @!scope class
@@ -110,6 +124,18 @@ module GoodJob
     # @return [ActiveRecord::Relation]
     scope :running, -> { where.not(performed_at: nil).where(finished_at: nil) }
 
+    # Get Jobs that do not have subsequent retries
+    # @!method running
+    # @!scope class
+    # @return [ActiveRecord::Relation]
+    scope :head, -> { where(retried_good_job_id: nil) }
+
+    # Get Jobs have errored that will not be retried further
+    # @!method running
+    # @!scope class
+    # @return [ActiveRecord::Relation]
+    scope :dead, -> { head.where.not(error: nil) }
+
     # Get Jobs on queues that match the given queue string.
     # @!method queue_string(string)
     # @!scope class
@@ -199,7 +225,6 @@ module GoodJob
     def self.enqueue(active_job, scheduled_at: nil, create_with_advisory_lock: false)
       ActiveSupport::Notifications.instrument("enqueue_job.good_job", { active_job: active_job, scheduled_at: scheduled_at, create_with_advisory_lock: create_with_advisory_lock }) do |instrument_payload|
         good_job_args = {
-          cron_key: CurrentExecution.cron_key,
           queue_name: active_job.queue_name.presence || DEFAULT_QUEUE_NAME,
           priority: active_job.priority || DEFAULT_PRIORITY,
           serialized_params: active_job.serialize,
@@ -210,31 +235,23 @@ module GoodJob
         if column_names.include?('active_job_id')
           good_job_args[:active_job_id] = active_job.job_id
         else
-          ActiveSupport::Deprecation.warn(<<~DEPRECATION)
-            GoodJob has pending database migrations. To create the migration files, run:
-
-                rails generate good_job:update
-
-            To apply the migration files, run:
-
-                rails db:migrate
-
-          DEPRECATION
+          _migration_pending_warning
         end
 
         if column_names.include?('concurrency_key')
           good_job_args[:concurrency_key] = active_job.good_job_concurrency_key if active_job.respond_to?(:good_job_concurrency_key)
         else
-          ActiveSupport::Deprecation.warn(<<~DEPRECATION)
-            GoodJob has pending database migrations. To create the migration files, run:
-
-                rails generate good_job:update
-
-            To apply the migration files, run:
-
-                rails db:migrate
+          _migration_pending_warning
+        end
 
-          DEPRECATION
+        if column_names.include?('cron_key')
+          if CurrentExecution.cron_key
+            good_job_args[:cron_key] = CurrentExecution.cron_key
+          elsif CurrentExecution.active_job_id == active_job.job_id
+            good_job_args[:cron_key] = CurrentExecution.good_job.cron_key
+          end
+        else
+          _migration_pending_warning
         end
 
         good_job = GoodJob::Job.new(**good_job_args)
@@ -244,6 +261,12 @@ module GoodJob
         good_job.save!
         active_job.provider_job_id = good_job.id
 
+        if column_names.include?('retried_good_job_id')
+          CurrentExecution.good_job.retried_good_job_id = good_job.id if CurrentExecution.good_job && CurrentExecution.good_job.active_job_id == active_job.job_id
+        else
+          _migration_pending_warning
+        end
+
         good_job
       end
     end
@@ -283,24 +306,19 @@ module GoodJob
     end
 
     def active_job_id
-      super || serialized_params['job_id']
+      if self.class.column_names.include?('active_job_id')
+        super
+      else
+        self.class._migration_pending_warning
+        serialized_params['job_id']
+      end
     end
 
     def cron_key
       if self.class.column_names.include?('cron_key')
         super
       else
-        ActiveSupport::Deprecation.warn(<<~DEPRECATION)
-          GoodJob has pending database migrations. To create the migration files, run:
-
-              rails generate good_job:update
-
-          To apply the migration files, run:
-
-              rails db:migrate
-
-        DEPRECATION
-
+        self.class._migration_pending_warning
         nil
       end
     end
@@ -314,8 +332,7 @@ module GoodJob
       )
 
       GoodJob::CurrentExecution.reset
-      GoodJob::CurrentExecution.active_job_id = active_job_id
-      GoodJob::CurrentExecution.cron_key = cron_key
+      GoodJob::CurrentExecution.good_job = self
       ActiveSupport::Notifications.instrument("perform_job.good_job", { good_job: self, process_id: GoodJob::CurrentExecution.process_id, thread_name: GoodJob::CurrentExecution.thread_name }) do
         value = ActiveJob::Base.execute(params)
 

data/lib/good_job/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 module GoodJob
   # GoodJob gem version.
-  VERSION = '1.12.0'
+  VERSION = '1.13.1'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 1.12.0
+  version: 1.13.1
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2021-07-27 00:00:00.000000000 Z
+date: 2021-08-18 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -374,6 +374,7 @@ files:
 - lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb
 - lib/generators/good_job/templates/update/migrations/02_add_active_job_id_concurrency_key_cron_key_to_good_jobs.rb
 - lib/generators/good_job/templates/update/migrations/03_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb
+- lib/generators/good_job/templates/update/migrations/04_add_retried_good_job_id_to_good_jobs.rb
 - lib/generators/good_job/update_generator.rb
 - lib/good_job.rb
 - lib/good_job/active_job_extensions.rb