good_job 2.8.0 → 2.9.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ae78ac0322802107488f4a8b55963665cf9b3f173eb05174382bf05afdcef5a6
-  data.tar.gz: 2f41dd00281bcf2d0a2c29f6124de36964dcc730436f1cbd2f4c740c873263ed
+  metadata.gz: 43a641edf5b6ff33dc8752874c6b317ad96bfd74ed4cbcb3a9bbeee7ff9c14cf
+  data.tar.gz: bfcf81a0d6c2d7bef01c3abdad9222fcc92cc8a5fd0ead3059fe5b9afd96aefa
 SHA512:
-  metadata.gz: 0ec2d40fdd87f293f8372e27040c7b14f8668645f50bc968ce91ec758202a400bb7c4a9ecb2e31a5827d59717dc3069840d5402981e1dc9f98c113120e259258
-  data.tar.gz: 0dd784cee33ab996ad332d1bfc371c357ffe3fb9727629096cff7189cc391195c109400240569444221a2ebf3340e2ab0d2d1c7bbd2830ee1c5b2ada9117d995
+  metadata.gz: ea801613572d4ccdce2af524303220eed9f50532ce80160c8fd9063bb994f0be725cf806d84aad8ea2bf5028f593a79fa13b5d1f7e796ece29eee70d42df9fa8
+  data.tar.gz: 9344d31ec8ffacdaa531a7e8bb7292270b151ad440d9dd95cd7ebfbe4106ca6b276ec3cff671df2ad97c9aee50892d453b584f948a7fc5e1bf7710fd935b2d40

data/CHANGELOG.md

@@ -1,5 +1,62 @@
 # Changelog
 
+## [v2.9.2](https://github.com/bensheldon/good_job/tree/v2.9.2) (2022-01-19)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.9.1...v2.9.2)
+
+**Fixed bugs:**
+
+- Error on GJ admin UI search form [\#487](https://github.com/bensheldon/good_job/issues/487)
+- Use `websearch_to_tsquery` or \(`plainto_tsquery` for Postgres \< v11\) for Dashboard search filter [\#488](https://github.com/bensheldon/good_job/pull/488) ([bensheldon](https://github.com/bensheldon))
+
+**Merged pull requests:**
+
+- Update README to illustrate using named arguments for the unique key. [\#486](https://github.com/bensheldon/good_job/pull/486) ([phallstrom](https://github.com/phallstrom))
+- Add details about exactly where to require the engine. [\#485](https://github.com/bensheldon/good_job/pull/485) ([phallstrom](https://github.com/phallstrom))
+- $ symbol gets copied when clicking on the copy button [\#484](https://github.com/bensheldon/good_job/pull/484) ([zeevy](https://github.com/zeevy))
+
+## [v2.9.1](https://github.com/bensheldon/good_job/tree/v2.9.1) (2022-01-13)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.9.0...v2.9.1)
+
+**Fixed bugs:**
+
+- Start async adapters once `ActiveRecord` and `ActiveJob` have loaded, potentially before `Rails.application.initialized?` [\#483](https://github.com/bensheldon/good_job/pull/483) ([bensheldon](https://github.com/bensheldon))
+
+**Closed issues:**
+
+- Graceful fallback to polling when LISTEN/NOTIFY isn't available [\#482](https://github.com/bensheldon/good_job/issues/482)
+- Long running locks on latest good job [\#480](https://github.com/bensheldon/good_job/issues/480)
+
+## [v2.9.0](https://github.com/bensheldon/good_job/tree/v2.9.0) (2022-01-09)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.8.1...v2.9.0)
+
+**Implemented enhancements:**
+
+- Add JRuby / JDBC support for LISTEN  [\#479](https://github.com/bensheldon/good_job/pull/479) ([bensheldon](https://github.com/bensheldon))
+
+**Merged pull requests:**
+
+- Remove demo CleanupJob in favor of using built-in cleanup intervals [\#478](https://github.com/bensheldon/good_job/pull/478) ([bensheldon](https://github.com/bensheldon))
+
+## [v2.8.1](https://github.com/bensheldon/good_job/tree/v2.8.1) (2022-01-03)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.8.0...v2.8.1)
+
+**Implemented enhancements:**
+
+- Add indexes to `good_jobs.finished_at` and have `GoodJob.cleanup_preserved_jobs` delete all executions for a given job [\#477](https://github.com/bensheldon/good_job/pull/477) ([bensheldon](https://github.com/bensheldon))
+
+**Closed issues:**
+
+- finished\_at should be indexed and clean up should clean up all of a job's executions [\#476](https://github.com/bensheldon/good_job/issues/476)
+
+**Merged pull requests:**
+
+- Update development Ruby \(2.7.5\) and Rails \(6.1.4.4\) versions [\#475](https://github.com/bensheldon/good_job/pull/475) ([bensheldon](https://github.com/bensheldon))
+- Clean up server integration tests [\#474](https://github.com/bensheldon/good_job/pull/474) ([bensheldon](https://github.com/bensheldon))
+
 ## [v2.8.0](https://github.com/bensheldon/good_job/tree/v2.8.0) (2021-12-31)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v2.7.4...v2.8.0)

data/README.md

@@ -73,19 +73,19 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
 1. Install the gem:
 
     ```bash
-    $ bundle install
+    bundle install
     ```
 
 1. Run the GoodJob install generator. This will generate a database migration to create a table for GoodJob's job records:
 
     ```bash
-    $ bin/rails g good_job:install
+    bin/rails g good_job:install
     ```
 
     Run the migration:
 
     ```bash
-    $ bin/rails db:migrate
+    bin/rails db:migrate
     ```
 
    Optional: If using Rails' multiple databases with the `migrations_paths` configuration option, use the `--database` option:
@@ -142,7 +142,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
 ## Compatibility
 
 - **Ruby on Rails:** 5.2+
-- **Ruby:** MRI 2.5+. JRuby 9.2.13+ (_JRuby's `activerecord-jdbcpostgresql-adapter` gem does not support Postgres LISTEN/NOTIFY)._
+- **Ruby:** MRI 2.5+. JRuby 9.2.13+
 - **Postgres:** 9.6+
 
 ## Configuration
@@ -328,7 +328,7 @@ _🚧 GoodJob's dashboard is a work in progress. Please contribute ideas and cod
 
 GoodJob includes a Dashboard as a mountable `Rails::Engine`.
 
-1. Explicitly require the Engine code at the top of your `config/application.rb` file, immediately after Rails is required. This is necessary because the mountable engine is an optional feature of GoodJob.
+1. Explicitly require the Engine code at the top of your `config/application.rb` file, immediately after Rails is required and before Bundler requires the Rails' groups. This is necessary because the mountable engine is an optional feature of GoodJob.
 
     ```ruby
     # config/application.rb
@@ -339,6 +339,8 @@ GoodJob includes a Dashboard as a mountable `Rails::Engine`.
     # ...
     ```
 
+   Note: If you find the dashboard fails to reload due to a routing error and uninitialized constant `GoodJob::ExecutionsController`, this is likely because you are not requiring the engine early enough.
+
 1. Mount the engine in your `config/routes.rb` file. The following will mount it at `http://example.com/good_job`.
 
     ```ruby
@@ -399,6 +401,9 @@ class MyJob < ApplicationJob
     # Can be String or Lambda/Proc that is invoked in the context of the job.
     # Note: Arguments passed to #perform_later must be accessed through `arguments` method.
     key: -> { "Unique-#{arguments.first}" } #  MyJob.perform_later("Alice") => "Unique-Alice"
+
+    # If the method uses named parameters, they can be accessed like so:
+    # key: -> { "Unique-#{arguments.first['name']}" } #  MyJob.perform_later(name: "Alice")
   )
 
   def perform(first_name)

data/lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb

@@ -32,5 +32,7 @@ class CreateGoodJobs < ActiveRecord::Migration<%= migration_version %>
     add_index :good_jobs, :concurrency_key, where: "(finished_at IS NULL)", name: :index_good_jobs_on_concurrency_key_when_unfinished
     add_index :good_jobs, [:cron_key, :created_at], name: :index_good_jobs_on_cron_key_and_created_at
     add_index :good_jobs, [:cron_key, :cron_at], name: :index_good_jobs_on_cron_key_and_cron_at, unique: true
+    add_index :good_jobs, [:active_job_id], name: :index_good_jobs_on_active_job_id
+    add_index :good_jobs, [:finished_at], where: "retried_good_job_id IS NULL AND finished_at IS NOT NULL", name: :index_good_jobs_jobs_on_finished_at
   end
 end

data/lib/generators/good_job/templates/update/migrations/04_index_good_job_jobs_on_finished_at.rb.erb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+class IndexGoodJobJobsOnFinishedAt < ActiveRecord::Migration<%= migration_version %>
+  disable_ddl_transaction!
+
+  def change
+    reversible do |dir|
+      dir.up do
+        # Ensure this incremental update migration is idempotent
+        # with monolithic install migration.
+        return if connection.index_name_exists?(:good_jobs, :index_good_jobs_on_active_job_id)
+      end
+    end
+
+    add_index :good_jobs,
+              [:active_job_id],
+              name: :index_good_jobs_on_active_job_id,
+              algorithm: :concurrently
+
+    add_index :good_jobs,
+              [:finished_at],
+              where: "retried_good_job_id IS NULL AND finished_at IS NOT NULL",
+              name: :index_good_jobs_jobs_on_finished_at,
+              algorithm: :concurrently
+  end
+end

data/lib/good_job/active_job_job.rb

@@ -19,7 +19,7 @@ module GoodJob
     self.primary_key = 'active_job_id'
     self.advisory_lockable_column = 'active_job_id'
 
-    has_many :executions, -> { order(created_at: :asc) }, class_name: 'GoodJob::Execution', foreign_key: 'active_job_id'
+    has_many :executions, -> { order(created_at: :asc) }, class_name: 'GoodJob::Execution', foreign_key: 'active_job_id', inverse_of: :job
 
     # Only the most-recent unretried execution represents a "Job"
     default_scope { where(retried_good_job_id: nil) }
@@ -27,11 +27,17 @@ module GoodJob
     # Get Jobs with given class name
     # @!method job_class
     # @!scope class
-    # @param string [String]
-    #   Execution class name
+    # @param string [String] Execution class name
     # @return [ActiveRecord::Relation]
     scope :job_class, ->(job_class) { where("serialized_params->>'job_class' = ?", job_class) }
 
+    # Get Jobs finished before the given timestamp.
+    # @!method finished_before(timestamp)
+    # @!scope class
+    # @param timestamp (DateTime, Time)
+    # @return [ActiveRecord::Relation]
+    scope :finished_before, ->(timestamp) { where(arel_table['finished_at'].lteq(timestamp)) }
+
     # First execution will run in the future
     scope :scheduled, -> { where(finished_at: nil).where('COALESCE(scheduled_at, created_at) > ?', DateTime.current).where("(serialized_params->>'executions')::integer < 2") }
     # Execution errored, will run in the future

data/lib/good_job/adapter.rb

@@ -27,7 +27,7 @@ module GoodJob
     # @param queues [String, nil] determines which queues to execute jobs from when +execution_mode+ is set to +:async+. See {file:README.md#optimize-queues-threads-and-processes} for more details on the format of this string. You can also set this with the environment variable +GOOD_JOB_QUEUES+. Defaults to +"*"+.
     # @param poll_interval [Integer, nil] sets the number of seconds between polls for jobs when +execution_mode+ is set to +:async+. You can also set this with the environment variable +GOOD_JOB_POLL_INTERVAL+. Defaults to +1+.
     # @param start_async_on_initialize [Boolean] whether to start the async scheduler when the adapter is initialized.
-    def initialize(execution_mode: nil, queues: nil, max_threads: nil, poll_interval: nil, start_async_on_initialize: Rails.application.initialized?)
+    def initialize(execution_mode: nil, queues: nil, max_threads: nil, poll_interval: nil, start_async_on_initialize: GoodJob.async_ready?)
       @configuration = GoodJob::Configuration.new(
         {
           execution_mode: execution_mode,
@@ -149,7 +149,7 @@ module GoodJob
     def in_server_process?
       return @_in_server_process if defined? @_in_server_process
 
-      @_in_server_process = Rails.const_defined?('Server') ||
+      @_in_server_process = Rails.const_defined?(:Server) ||
                             caller.grep(%r{config.ru}).any? || # EXAMPLE: config.ru:3:in `block in <main>' OR config.ru:3:in `new_from_string'
                             caller.grep(%{/rack/handler/}).any? || # EXAMPLE: iodine-0.7.44/lib/rack/handler/iodine.rb:13:in `start'
                             (Concurrent.on_jruby? && caller.grep(%r{jruby/rack/rails_booter}).any?) # EXAMPLE: uri:classloader:/jruby/rack/rails_booter.rb:83:in `load_environment'

data/lib/good_job/dependencies.rb

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+module GoodJob # :nodoc:
+  # Extends GoodJob module to track Rails boot dependencies.
+  module Dependencies
+    extend ActiveSupport::Concern
+
+    included do
+      # @!attribute [rw] _rails_after_initialize_hook_called
+      #   @!scope class
+      #   Whether Railtie.after_initialize has been called yet (default: +false+).
+      #   This will be set on  but before +Rails.application.initialize?+ is +true+.
+      #   @return [Boolean]
+      mattr_accessor :_rails_after_initialize_hook_called, default: false
+
+      # @!attribute [rw] _active_job_loaded
+      #   @!scope class
+      #   Whether ActiveJob has loaded (default: +false+).
+      #   @return [Boolean]
+      mattr_accessor :_active_job_loaded, default: false
+
+      # @!attribute [rw] _active_record_loaded
+      #   @!scope class
+      #   Whether ActiveRecord has loaded (default: +false+).
+      #   @return [Boolean]
+      mattr_accessor :_active_record_loaded, default: false
+    end
+
+    class_methods do
+      # Whether GoodJob's  has been initialized as of the calling of +Railtie.after_initialize+.
+      # @return [Boolean]
+      def async_ready?
+        Rails.application.initialized? || (
+           _rails_after_initialize_hook_called &&
+           _active_job_loaded &&
+           _active_record_loaded
+         )
+      end
+
+      def start_async_adapters
+        return unless async_ready?
+
+        GoodJob::Adapter.instances
+                        .select(&:execute_async?)
+                        .reject(&:async_started?)
+                        .each(&:start_async)
+      end
+    end
+  end
+end

data/lib/good_job/execution.rb

@@ -51,6 +51,8 @@ module GoodJob
       end
     end
 
+    belongs_to :job, class_name: 'GoodJob::ActiveJobJob', foreign_key: 'active_job_id', primary_key: 'active_job_id', optional: true, inverse_of: :executions
+
     # Get Jobs with given ActiveJob ID
     # @!method active_job_id
     # @!scope class

data/lib/good_job/filterable.rb

@@ -34,9 +34,18 @@ module GoodJob
         next if query.blank?
 
         tsvector = "(to_tsvector('english', serialized_params) || to_tsvector('english', id::text) || to_tsvector('english', COALESCE(error, '')::text))"
-        where("#{tsvector} @@ to_tsquery(?)", query)
-          .order(sanitize_sql_for_order([Arel.sql("ts_rank(#{tsvector}, to_tsquery(?))"), query]) => 'DESC')
+        to_tsquery_function = database_supports_websearch_to_tsquery? ? 'websearch_to_tsquery' : 'plainto_tsquery'
+        where("#{tsvector} @@ #{to_tsquery_function}(?)", query)
+          .order(sanitize_sql_for_order([Arel.sql("ts_rank(#{tsvector}, #{to_tsquery_function}(?))"), query]) => 'DESC')
       end)
     end
+
+    class_methods do
+      def database_supports_websearch_to_tsquery?
+        return @_database_supports_websearch_to_tsquery if defined?(@_database_supports_websearch_to_tsquery)
+
+        @_database_supports_websearch_to_tsquery = connection.postgresql_version >= 110000
+      end
+    end
   end
 end

data/lib/good_job/notifier.rb

@@ -16,9 +16,6 @@ module GoodJob # :nodoc:
 
     include Notifier::ProcessRegistration
 
-    # Raised if the Database adapter does not implement LISTEN.
-    AdapterCannotListenError = Class.new(StandardError)
-
     # Default Postgres channel for LISTEN/NOTIFY
     CHANNEL = 'good_job'
     # Defaults for instance of Concurrent::ThreadPoolExecutor
@@ -129,8 +126,6 @@ module GoodJob # :nodoc:
     # @!visibility private
     # @return [void]
     def listen_observer(_time, _result, thread_error)
-      return if thread_error.is_a? AdapterCannotListenError
-
       if thread_error
         GoodJob._on_thread_error(thread_error)
         ActiveSupport::Notifications.instrument("notifier_notify_error.good_job", { error: thread_error })
@@ -214,6 +209,15 @@ module GoodJob # :nodoc:
         raw_connection.wait_for_notify(WAIT_INTERVAL) do |channel, _pid, payload|
           yield(channel, payload)
         end
+      elsif raw_connection.respond_to?(:jdbc_connection)
+        raw_connection.execute_query("SELECT 1")
+        notifications = raw_connection.jdbc_connection.getNotifications
+        Array(notifications).each do |notification|
+          channel = notification.getName
+          payload = notification.getParameter
+          yield(channel, payload)
+        end
+        sleep WAIT_INTERVAL
       else
         sleep WAIT_INTERVAL
       end

data/lib/good_job/railtie.rb

@@ -34,8 +34,27 @@ module GoodJob
     end
 
     initializer "good_job.start_async" do
+      # This hooks into the hookable places during Rails boot, which is unfortunately not Rails.application.initialized?
+      # If an Adapter is initialized during boot, we want to want to start its async executors once the framework dependencies have loaded.
+      # When exactly that happens is out of our control because gems or application code may touch things earlier than expected.
+      # For example, as of Rails 6.1, if an ActiveRecord model is touched during boot, that triggers ActiveRecord to load,
+      # which touches DestroyAssociationAsyncJob, which loads ActiveJob, which may initialize a GoodJob::Adapter, all of which
+      # happens _before_ ActiveRecord finishes loading. GoodJob will deadlock if an async executor is started in the middle of
+      # ActiveRecord loading.
+
       config.after_initialize do
-        GoodJob::Adapter.instances.each(&:start_async)
+        ActiveSupport.on_load(:active_record) do
+          GoodJob._active_record_loaded = true
+          GoodJob.start_async_adapters
+        end
+
+        ActiveSupport.on_load(:active_job) do
+          GoodJob._active_job_loaded = true
+          GoodJob.start_async_adapters
+        end
+
+        GoodJob._rails_after_initialize_hook_called = true
+        GoodJob.start_async_adapters
       end
     end
   end

data/lib/good_job/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 module GoodJob
   # GoodJob gem version.
-  VERSION = '2.8.0'
+  VERSION = '2.9.2'
 end

data/lib/good_job.rb

@@ -18,6 +18,8 @@ require "good_job/railtie"
 #
 # +GoodJob+ is the top-level namespace and exposes configuration attributes.
 module GoodJob
+  include GoodJob::Dependencies
+
   DEFAULT_LOGGER = ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new($stdout))
 
   # @!attribute [rw] active_record_parent_class
@@ -138,9 +140,11 @@ module GoodJob
     timestamp = Time.current - older_than
 
     ActiveSupport::Notifications.instrument("cleanup_preserved_jobs.good_job", { older_than: older_than, timestamp: timestamp }) do |payload|
-      deleted_records_count = GoodJob::Execution.finished(timestamp).delete_all
+      old_jobs = GoodJob::ActiveJobJob.where('finished_at <= ?', timestamp)
+      old_jobs_count = old_jobs.count
 
-      payload[:deleted_records_count] = deleted_records_count
+      GoodJob::Execution.where(job: old_jobs).delete_all
+      payload[:deleted_records_count] = old_jobs_count
     end
   end
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 2.8.0
+  version: 2.9.2
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2021-12-31 00:00:00.000000000 Z
+date: 2022-01-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -405,6 +405,7 @@ files:
 - lib/generators/good_job/templates/update/migrations/02_add_cron_at_to_good_jobs.rb.erb
 - lib/generators/good_job/templates/update/migrations/03_add_cron_key_cron_at_index_to_good_jobs.rb.erb
 - lib/generators/good_job/templates/update/migrations/04_create_good_job_processes.rb.erb
+- lib/generators/good_job/templates/update/migrations/04_index_good_job_jobs_on_finished_at.rb.erb
 - lib/generators/good_job/update_generator.rb
 - lib/good_job.rb
 - lib/good_job/active_job_extensions.rb
@@ -420,6 +421,7 @@ files:
 - lib/good_job/cron_manager.rb
 - lib/good_job/current_thread.rb
 - lib/good_job/daemon.rb
+- lib/good_job/dependencies.rb
 - lib/good_job/execution.rb
 - lib/good_job/execution_result.rb
 - lib/good_job/filterable.rb
@@ -445,6 +447,7 @@ metadata:
   documentation_uri: https://rdoc.info/github/bensheldon/good_job
   homepage_uri: https://github.com/bensheldon/good_job
   source_code_uri: https://github.com/bensheldon/good_job
+  rubygems_mfa_required: 'true'
 post_install_message: 
 rdoc_options:
 - "--title"
@@ -468,7 +471,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.30
+rubygems_version: 3.1.6
 signing_key: 
 specification_version: 4
 summary: A multithreaded, Postgres-based ActiveJob backend for Ruby on Rails