good_job 2.7.3 → 2.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f6f10b3664d49f86aef9b010af1ca673fc34235966ae98c55fed5ba271bcaae2
-  data.tar.gz: bd7f84c9c0b75342ea10d72ad40d7fba4cd51aa8105348e362587de16452a19b
+  metadata.gz: ba4a26a2047c0e7dd8b78264fee7c6d2c05c2dd59e38f1df402b698826b56288
+  data.tar.gz: 3bd4c49139941c2d37751abb9d32808b0efd7a33370aa7370afeed21cc845ab0
 SHA512:
-  metadata.gz: 9a1de45c7584ec2d55b54187ee818fdb0c58a954463425a5c5394e097fb7a7236c02219f6cdbbf26168103bd359ed1b72f5cbb096b8d3ae5a524ca3c3277be23
-  data.tar.gz: 1d2ab4cb0c942e9818bbefaac4fa518afde793f7648cc64350d2c4c45201745e2e00be47ecc1942d093ce8a1a0459fa30060f46680bc77c789044b0f56a4e89b
+  metadata.gz: d2ade6a5c9b26e1d9d1bc1522dc10159f78943b5da249effa1405dccf5c3de5078f74a08c27657eef63d51967bb04eb8486d933e9d817664a25c33f3740f2280
+  data.tar.gz: 40dc9f0b3b1d210ddcec9d7ce5dd7014e2409d5602bb4b3c55314b5bf799ef29fab9fd0764931c2026242c91383a335b64c4f3243ea16d50997e59ef9c7820e3

data/CHANGELOG.md

@@ -1,15 +1,79 @@
 # Changelog
 
-## [v2.7.3](https://github.com/bensheldon/good_job/tree/v2.7.3) (2021-11-30)
+## [v2.9.0](https://github.com/bensheldon/good_job/tree/v2.9.0) (2022-01-09)
 
-[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.7.2...v2.7.3)
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.8.1...v2.9.0)
+
+**Implemented enhancements:**
+
+- Add JRuby / JDBC support for LISTEN  [\#479](https://github.com/bensheldon/good_job/pull/479) ([bensheldon](https://github.com/bensheldon))
+
+**Merged pull requests:**
+
+- Remove demo CleanupJob in favor of using built-in cleanup intervals [\#478](https://github.com/bensheldon/good_job/pull/478) ([bensheldon](https://github.com/bensheldon))
+
+## [v2.8.1](https://github.com/bensheldon/good_job/tree/v2.8.1) (2022-01-03)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.8.0...v2.8.1)
+
+**Implemented enhancements:**
+
+- Add indexes to `good_jobs.finished_at` and have `GoodJob.cleanup_preserved_jobs` delete all executions for a given job [\#477](https://github.com/bensheldon/good_job/pull/477) ([bensheldon](https://github.com/bensheldon))
+
+**Closed issues:**
+
+- finished\_at should be indexed and clean up should clean up all of a job's executions [\#476](https://github.com/bensheldon/good_job/issues/476)
+
+**Merged pull requests:**
+
+- Update development Ruby \(2.7.5\) and Rails \(6.1.4.4\) versions [\#475](https://github.com/bensheldon/good_job/pull/475) ([bensheldon](https://github.com/bensheldon))
+- Clean up server integration tests [\#474](https://github.com/bensheldon/good_job/pull/474) ([bensheldon](https://github.com/bensheldon))
+
+## [v2.8.0](https://github.com/bensheldon/good_job/tree/v2.8.0) (2021-12-31)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.7.4...v2.8.0)
+
+**Implemented enhancements:**
+
+- GoodJob should automatically clean up after itself and delete old job records [\#412](https://github.com/bensheldon/good_job/issues/412)
+- Track processes in the database and on the Dashboard [\#472](https://github.com/bensheldon/good_job/pull/472) ([bensheldon](https://github.com/bensheldon))
+- Allow Scheduler to automatically clean up preserved jobs every N jobs or seconds [\#465](https://github.com/bensheldon/good_job/pull/465) ([bensheldon](https://github.com/bensheldon))
+
+**Closed issues:**
+
+- Is there a way to show how many worker/process is running currently [\#471](https://github.com/bensheldon/good_job/issues/471)
+- Jobs stuck in the unfinished state [\#448](https://github.com/bensheldon/good_job/issues/448)
+
+**Merged pull requests:**
+
+- Doublequote Ruby 3.0 in testing matrix [\#473](https://github.com/bensheldon/good_job/pull/473) ([bensheldon](https://github.com/bensheldon))
+- Have demo CleanupJob use GoodJob.cleanup\_preserved\_jobs [\#470](https://github.com/bensheldon/good_job/pull/470) ([bensheldon](https://github.com/bensheldon))
+- Test with Rails 7.0.0 [\#469](https://github.com/bensheldon/good_job/pull/469) ([aried3r](https://github.com/aried3r))
+
+## [v2.7.4](https://github.com/bensheldon/good_job/tree/v2.7.4) (2021-12-16)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.7.3...v2.7.4)
 
 **Fixed bugs:**
 
-- Logger error on 2.7.2 [\#463](https://github.com/bensheldon/good_job/issues/463)
+- Add nonce: true to javascript\_include\_tag in dashboard [\#468](https://github.com/bensheldon/good_job/pull/468) ([bouk](https://github.com/bouk))
+
+**Closed issues:**
+
+- Add nonce: true to engine views  [\#467](https://github.com/bensheldon/good_job/issues/467)
+- Updating good\_job breaks my Rails 7 alpha 2 local development [\#462](https://github.com/bensheldon/good_job/issues/462)
 
 **Merged pull requests:**
 
+- Update appraisal for Rails 7.0.0.rc1 [\#466](https://github.com/bensheldon/good_job/pull/466) ([bensheldon](https://github.com/bensheldon))
+
+## [v2.7.3](https://github.com/bensheldon/good_job/tree/v2.7.3) (2021-11-30)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.7.2...v2.7.3)
+
+**Fixed bugs:**
+
+- Logger error on 2.7.2 [\#463](https://github.com/bensheldon/good_job/issues/463)
 - Fix Railtie configuration assignment when Rails configuration is a Hash, not an OrderedOptions [\#464](https://github.com/bensheldon/good_job/pull/464) ([bensheldon](https://github.com/bensheldon))
 
 ## [v2.7.2](https://github.com/bensheldon/good_job/tree/v2.7.2) (2021-11-29)

data/README.md

@@ -2,6 +2,7 @@
 
 [![Gem Version](https://badge.fury.io/rb/good_job.svg)](https://rubygems.org/gems/good_job)
 [![Test Status](https://github.com/bensheldon/good_job/workflows/Test/badge.svg)](https://github.com/bensheldon/good_job/actions)
+[![Ruby Toolbox](https://img.shields.io/badge/dynamic/json?color=blue&label=Ruby%20Toolbox&query=%24.projects%5B0%5D.score&url=https%3A%2F%2Fwww.ruby-toolbox.com%2Fapi%2Fprojects%2Fcompare%2Fgood_job&logo=data:image/svg+xml;base64,PHN2ZyBhcmlhLWhpZGRlbj0idHJ1ZSIgZm9jdXNhYmxlPSJmYWxzZSIgZGF0YS1wcmVmaXg9ImZhcyIgZGF0YS1pY29uPSJmbGFzayIgY2xhc3M9InN2Zy1pbmxpbmUtLWZhIGZhLWZsYXNrIGZhLXctMTQiIHJvbGU9ImltZyIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB2aWV3Qm94PSIwIDAgNDQ4IDUxMiI+PHBhdGggZmlsbD0id2hpdGUiIGQ9Ik00MzcuMiA0MDMuNUwzMjAgMjE1VjY0aDhjMTMuMyAwIDI0LTEwLjcgMjQtMjRWMjRjMC0xMy4zLTEwLjctMjQtMjQtMjRIMTIwYy0xMy4zIDAtMjQgMTAuNy0yNCAyNHYxNmMwIDEzLjMgMTAuNyAyNCAyNCAyNGg4djE1MUwxMC44IDQwMy41Qy0xOC41IDQ1MC42IDE1LjMgNTEyIDcwLjkgNTEyaDMwNi4yYzU1LjcgMCA4OS40LTYxLjUgNjAuMS0xMDguNXpNMTM3LjkgMzIwbDQ4LjItNzcuNmMzLjctNS4yIDUuOC0xMS42IDUuOC0xOC40VjY0aDY0djE2MGMwIDYuOSAyLjIgMTMuMiA1LjggMTguNGw0OC4yIDc3LjZoLTE3MnoiPjwvcGF0aD48L3N2Zz4=)](https://www.ruby-toolbox.com/projects/good_job)
 
 GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails.
 
@@ -270,6 +271,9 @@ Available configuration options are:
 - `shutdown_timeout` (float) number of seconds to wait for jobs to finish when shutting down before stopping the thread. Defaults to forever: `-1`. You can also set this with the environment variable `GOOD_JOB_SHUTDOWN_TIMEOUT`.
 - `enable_cron` (boolean) whether to run cron process. Defaults to `false`. You can also set this with the environment variable `GOOD_JOB_ENABLE_CRON`.
 - `cron` (hash) cron configuration. Defaults to `{}`. You can also set this as a JSON string with the environment variable `GOOD_JOB_CRON`
+- `cleanup_preserved_jobs_before_seconds_ago` (integer) number of seconds to preserve jobs when using the `$ good_job cleanup_preserved_jobs` CLI command or calling `GoodJob.cleanup_preserved_jobs`. Defaults to `86400` (1 day). Can also be set with  the environment variable `GOOD_JOB_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO`.  _This configuration is only used when {GoodJob.preserve_job_records} is `true`._
+- `cleanup_interval_jobs` (integer) Number of jobs a Scheduler will execute before cleaning up preserved jobs. Defaults to `nil`. Can also be set with  the environment variable `GOOD_JOB_CLEANUP_INTERVAL_JOBS`.
+- `cleanup_interval_seconds` (integer) Number of seconds a Scheduler will wait before cleaning up preserved jobs. Defaults to `nil`. Can also be set with  the environment variable `GOOD_JOB_CLEANUP_INTERVAL_SECONDS`.
 - `logger` ([Rails Logger](https://api.rubyonrails.org/classes/ActiveSupport/Logger.html)) lets you set a custom logger for GoodJob. It should be an instance of a Rails `Logger` (Default: `Rails.logger`).
 - `preserve_job_records` (boolean) keeps job records in your database even after jobs are completed. (Default: `false`)
 - `retry_on_unhandled_error` (boolean) causes jobs to be re-queued and retried if they raise an instance of `StandardError`. Instances of `Exception`, like SIGINT, will *always* be retried, regardless of this attribute’s value. (Default: `true`)

data/engine/app/controllers/good_job/processes_controller.rb

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+module GoodJob
+  class ProcessesController < GoodJob::BaseController
+    def index
+      @processes = GoodJob::Process.active.order(created_at: :desc) if GoodJob::Process.migrated?
+    end
+  end
+end

data/engine/app/views/good_job/processes/index.html.erb

@@ -0,0 +1,40 @@
+<% if !GoodJob::Process.migrated? %>
+  <div class="card my-3">
+    <div class="card-body">
+      <p class="card-text">
+        <em>Feature unavailable because of pending database migration.</em>
+      </p>
+    </div>
+  </div>
+<% elsif @processes.present? %>
+  <div class="card my-3">
+    <div class="table-responsive">
+      <table class="table card-table table-bordered table-hover table-sm mb-0">
+        <thead>
+        <tr>
+          <th>Process UUID</th>
+          <th>Created At</th></th>
+          <th>State</th>
+        </tr>
+        </thead>
+        <tbody>
+        <% @processes.each do |process| %>
+          <tr class="<%= dom_class(process) %>" id="<%= dom_id(process) %>">
+            <td><%= process.id %></td>
+            <td><%= relative_time(process.created_at) %></td>
+            <td><%= tag.pre JSON.pretty_generate(process.state) %></td>
+          </tr>
+        <% end %>
+        </tbody>
+      </table>
+    </div>
+  </div>
+<% else %>
+  <div class="card my-3">
+    <div class="card-body">
+      <p class="card-text">
+        <em>No GoodJob processes found.</em>
+      </p>
+    </div>
+  </div>
+<% end %>

data/engine/app/views/layouts/good_job/base.html.erb

@@ -8,11 +8,11 @@
   <%= stylesheet_link_tag bootstrap_path(format: :css, v: GoodJob::VERSION) %>
   <%= stylesheet_link_tag style_path(format: :css, v: GoodJob::VERSION) %>
 
-  <%= javascript_include_tag bootstrap_path(format: :js, v: GoodJob::VERSION) %>
-  <%= javascript_include_tag chartjs_path(format: :js, v: GoodJob::VERSION) %>
-  <%= javascript_include_tag scripts_path(format: :js, v: GoodJob::VERSION) %>
+  <%= javascript_include_tag bootstrap_path(format: :js, v: GoodJob::VERSION), nonce: true %>
+  <%= javascript_include_tag chartjs_path(format: :js, v: GoodJob::VERSION), nonce: true %>
+  <%= javascript_include_tag scripts_path(format: :js, v: GoodJob::VERSION), nonce: true %>
 
-  <%= javascript_include_tag rails_ujs_path(format: :js, v: GoodJob::VERSION) %>
+  <%= javascript_include_tag rails_ujs_path(format: :js, v: GoodJob::VERSION), nonce: true %>
 </head>
 <body>
   <nav class="navbar navbar-expand-lg navbar-light bg-light">
@@ -33,23 +33,14 @@
           <li class="nav-item">
             <%= link_to "Cron Schedules", cron_entries_path, class: ["nav-link", ("active" if current_page?(cron_entries_path))] %>
           </li>
+          <li class="nav-item">
+            <%= link_to "Processes", processes_path, class: ["nav-link", ("active" if current_page?(processes_path))] %>
+          </li>
           <li class="nav-item">
             <div class="nav-link">
               <span class="badge bg-secondary">More views coming soon</span>
             </div>
           </li>
-
-          <!-- Coming Soon
-            <li class="nav-item">
-              <%= link_to "Upcoming Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
-            </li>
-            <li class="nav-item">
-              <%= link_to "Finished Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
-            </li>
-            <li class="nav-item">
-              <%= link_to "Errored Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
-            </li>
-          -->
         </ul>
         <div class="text-muted" title="Now is <%= Time.current %>">Times are displayed in <%= Time.current.zone %> timezone</div>
       </div>

data/engine/config/routes.rb

@@ -2,11 +2,7 @@
 GoodJob::Engine.routes.draw do
   root to: 'executions#index'
 
-  resources :cron_entries, only: %i[index show] do
-    member do
-      post :enqueue
-    end
-  end
+  resources :executions, only: %i[destroy]
 
   resources :jobs, only: %i[index show] do
     member do
@@ -15,7 +11,14 @@ GoodJob::Engine.routes.draw do
       put :retry
     end
   end
-  resources :executions, only: %i[destroy]
+
+  resources :cron_entries, only: %i[index show] do
+    member do
+      post :enqueue
+    end
+  end
+
+  resources :processes, only: %i[index]
 
   scope controller: :assets do
     constraints(format: :css) do

data/lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb

@@ -21,11 +21,18 @@ class CreateGoodJobs < ActiveRecord::Migration<%= migration_version %>
       t.timestamp :cron_at
     end
 
+    create_table :good_job_processes, id: :uuid do |t|
+      t.timestamps
+      t.jsonb :state
+    end
+
     add_index :good_jobs, :scheduled_at, where: "(finished_at IS NULL)", name: "index_good_jobs_on_scheduled_at"
     add_index :good_jobs, [:queue_name, :scheduled_at], where: "(finished_at IS NULL)", name: :index_good_jobs_on_queue_name_and_scheduled_at
     add_index :good_jobs, [:active_job_id, :created_at], name: :index_good_jobs_on_active_job_id_and_created_at
     add_index :good_jobs, :concurrency_key, where: "(finished_at IS NULL)", name: :index_good_jobs_on_concurrency_key_when_unfinished
     add_index :good_jobs, [:cron_key, :created_at], name: :index_good_jobs_on_cron_key_and_created_at
     add_index :good_jobs, [:cron_key, :cron_at], name: :index_good_jobs_on_cron_key_and_cron_at, unique: true
+    add_index :good_jobs, [:active_job_id], name: :index_good_jobs_on_active_job_id
+    add_index :good_jobs, [:finished_at], where: "retried_good_job_id IS NULL AND finished_at IS NOT NULL", name: :index_good_jobs_jobs_on_finished_at
   end
 end

data/lib/generators/good_job/templates/update/migrations/04_create_good_job_processes.rb.erb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+class CreateGoodJobProcesses < ActiveRecord::Migration<%= migration_version %>
+  def change
+    reversible do |dir|
+      dir.up do
+        # Ensure this incremental update migration is idempotent
+        # with monolithic install migration.
+        return if connection.table_exists?(:good_job_processes)
+      end
+    end
+
+    create_table :good_job_processes, id: :uuid do |t|
+      t.timestamps
+      t.jsonb :state
+    end
+  end
+end

data/lib/generators/good_job/templates/update/migrations/04_index_good_job_jobs_on_finished_at.rb.erb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+class IndexGoodJobJobsOnFinishedAt < ActiveRecord::Migration<%= migration_version %>
+  disable_ddl_transaction!
+
+  def change
+    reversible do |dir|
+      dir.up do
+        # Ensure this incremental update migration is idempotent
+        # with monolithic install migration.
+        return if connection.index_name_exists?(:good_jobs, :index_good_jobs_on_active_job_id)
+      end
+    end
+
+    add_index :good_jobs,
+              [:active_job_id],
+              name: :index_good_jobs_on_active_job_id,
+              algorithm: :concurrently
+
+    add_index :good_jobs,
+              [:finished_at],
+              where: "retried_good_job_id IS NULL AND finished_at IS NOT NULL",
+              name: :index_good_jobs_jobs_on_finished_at,
+              algorithm: :concurrently
+  end
+end

data/lib/good_job/active_job_job.rb

@@ -4,10 +4,7 @@ module GoodJob
   # There is not a table in the database whose discrete rows represents "Jobs".
   # The +good_jobs+ table is a table of individual {GoodJob::Execution}s that share the same +active_job_id+.
   # A single row from the +good_jobs+ table of executions is fetched to represent an ActiveJobJob
-  # Parent class can be configured with +GoodJob.active_record_parent_class+.
-  # @!parse
-  #   class ActiveJob < ActiveRecord::Base; end
-  class ActiveJobJob < Object.const_get(GoodJob.active_record_parent_class)
+  class ActiveJobJob < BaseRecord
     include Filterable
     include Lockable
 
@@ -22,7 +19,7 @@ module GoodJob
     self.primary_key = 'active_job_id'
     self.advisory_lockable_column = 'active_job_id'
 
-    has_many :executions, -> { order(created_at: :asc) }, class_name: 'GoodJob::Execution', foreign_key: 'active_job_id'
+    has_many :executions, -> { order(created_at: :asc) }, class_name: 'GoodJob::Execution', foreign_key: 'active_job_id', inverse_of: :job
 
     # Only the most-recent unretried execution represents a "Job"
     default_scope { where(retried_good_job_id: nil) }
@@ -30,11 +27,17 @@ module GoodJob
     # Get Jobs with given class name
     # @!method job_class
     # @!scope class
-    # @param string [String]
-    #   Execution class name
+    # @param string [String] Execution class name
     # @return [ActiveRecord::Relation]
     scope :job_class, ->(job_class) { where("serialized_params->>'job_class' = ?", job_class) }
 
+    # Get Jobs finished before the given timestamp.
+    # @!method finished_before(timestamp)
+    # @!scope class
+    # @param timestamp (DateTime, Time)
+    # @return [ActiveRecord::Relation]
+    scope :finished_before, ->(timestamp) { where(arel_table['finished_at'].lteq(timestamp)) }
+
     # First execution will run in the future
     scope :scheduled, -> { where(finished_at: nil).where('COALESCE(scheduled_at, created_at) > ?', DateTime.current).where("(serialized_params->>'executions')::integer < 2") }
     # Execution errored, will run in the future

data/lib/good_job/adapter.rb

@@ -121,7 +121,7 @@ module GoodJob
     end
 
     # Start async executors
-    # @return void
+    # @return [void]
     def start_async
       return unless execute_async?
 
@@ -149,7 +149,7 @@ module GoodJob
     def in_server_process?
       return @_in_server_process if defined? @_in_server_process
 
-      @_in_server_process = Rails.const_defined?('Server') ||
+      @_in_server_process = Rails.const_defined?(:Server) ||
                             caller.grep(%r{config.ru}).any? || # EXAMPLE: config.ru:3:in `block in <main>' OR config.ru:3:in `new_from_string'
                             caller.grep(%{/rack/handler/}).any? || # EXAMPLE: iodine-0.7.44/lib/rack/handler/iodine.rb:13:in `start'
                             (Concurrent.on_jruby? && caller.grep(%r{jruby/rack/rails_booter}).any?) # EXAMPLE: uri:classloader:/jruby/rack/rails_booter.rb:83:in `load_environment'

data/lib/good_job/assignable_connection.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+module GoodJob # :nodoc:
+  # Extends an ActiveRecord odel to override the connection and use
+  # an explicit connection that has been removed from the pool.
+  module AssignableConnection
+    extend ActiveSupport::Concern
+
+    included do
+      thread_cattr_accessor :_connection
+    end
+
+    class_methods do
+      # Assigns a connection to the model.
+      # @param conn [ActiveRecord::ConnectionAdapters::AbstractAdapter]
+      # @return [void]
+      def connection=(conn)
+        self._connection = conn
+      end
+
+      # Overrides the existing connection method to use the assigned connection
+      # @return [ActiveRecord::ConnectionAdapters::AbstractAdapter]
+      def connection
+        _connection || super
+      end
+
+      # Block interface to assign the connection, yield, then unassign the connection.
+      # @param conn [ActiveRecord::ConnectionAdapters::AbstractAdapter]
+      # @return [void]
+      def with_connection(conn)
+        original_conn = _connection
+        self.connection = conn
+        yield
+      ensure
+        self._connection = original_conn
+      end
+    end
+  end
+end

data/lib/good_job/base_record.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+module GoodJob
+  # Base ActiveRecord class that all GoodJob models inherit from.
+  # Parent class can be configured with +GoodJob.active_record_parent_class+.
+  # @!parse
+  #   class BaseRecord < ActiveRecord::Base; end
+  class BaseRecord < Object.const_get(GoodJob.active_record_parent_class)
+    self.abstract_class = true
+
+    def self.migration_pending_warning!
+      ActiveSupport::Deprecation.warn(<<~DEPRECATION)
+        GoodJob has pending database migrations. To create the migration files, run:
+            rails generate good_job:update
+        To apply the migration files, run:
+            rails db:migrate
+      DEPRECATION
+      nil
+    end
+  end
+end

data/lib/good_job/cleanup_tracker.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+module GoodJob # :nodoc:
+  # Tracks thresholds for cleaning up old jobs.
+  class CleanupTracker
+    attr_accessor :cleanup_interval_seconds,
+                  :cleanup_interval_jobs,
+                  :job_count,
+                  :last_at
+
+    def initialize(cleanup_interval_seconds: nil, cleanup_interval_jobs: nil)
+      self.cleanup_interval_seconds = cleanup_interval_seconds
+      self.cleanup_interval_jobs = cleanup_interval_jobs
+
+      reset
+    end
+
+    # Increments job count.
+    # @return [void]
+    def increment
+      self.job_count += 1
+    end
+
+    # Whether a cleanup should be run.
+    # @return [Boolean]
+    def cleanup?
+      (cleanup_interval_jobs && job_count > cleanup_interval_jobs) ||
+        (cleanup_interval_seconds && last_at < Time.current - cleanup_interval_seconds) ||
+        false
+    end
+
+    # Resets the counters.
+    # @return [void]
+    def reset
+      self.job_count = 0
+      self.last_at = Time.current
+    end
+  end
+end

data/lib/good_job/configuration.rb

@@ -18,6 +18,10 @@ module GoodJob
     DEFAULT_MAX_CACHE = 10000
     # Default number of seconds to preserve jobs for {CLI#cleanup_preserved_jobs} and {GoodJob.cleanup_preserved_jobs}
     DEFAULT_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO = 24 * 60 * 60
+    # Default number of jobs to execute between preserved job cleanup runs
+    DEFAULT_CLEANUP_INTERVAL_JOBS = nil
+    # Default number of seconds to wait between preserved job cleanup runs
+    DEFAULT_CLEANUP_INTERVAL_SECONDS = nil
     # Default to always wait for jobs to finish for {Adapter#shutdown}
     DEFAULT_SHUTDOWN_TIMEOUT = -1
     # Default to not running cron
@@ -179,6 +183,28 @@ module GoodJob
       ).to_i
     end
 
+    # Number of jobs a {Scheduler} will execute before cleaning up preserved jobs.
+    # @return [Integer, nil]
+    def cleanup_interval_jobs
+      value = (
+        rails_config[:cleanup_interval_jobs] ||
+          env['GOOD_JOB_CLEANUP_INTERVAL_JOBS'] ||
+          DEFAULT_CLEANUP_INTERVAL_JOBS
+      )
+      value.present? ? value.to_i : nil
+    end
+
+    # Number of seconds a {Scheduler} will wait before cleaning up preserved jobs.
+    # @return [Integer, nil]
+    def cleanup_interval_seconds
+      value = (
+        rails_config[:cleanup_interval_seconds] ||
+          env['GOOD_JOB_CLEANUP_INTERVAL_SECONDS'] ||
+          DEFAULT_CLEANUP_INTERVAL_SECONDS
+      )
+      value.present? ? value.to_i : nil
+    end
+
     # Tests whether to daemonize the process.
     # @return [Boolean]
     def daemonize?

data/lib/good_job/current_thread.rb

@@ -68,7 +68,7 @@ module GoodJob
 
     # @return [Integer] Current process ID
     def self.process_id
-      Process.pid
+      ::Process.pid
     end
 
     # @return [String] Current thread name

data/lib/good_job/daemon.rb

@@ -17,7 +17,7 @@ module GoodJob
     # @return [void]
     def daemonize
       check_pid
-      Process.daemon
+      ::Process.daemon
       write_pid
     end
 
@@ -25,7 +25,7 @@ module GoodJob
 
     # @return [void]
     def write_pid
-      File.open(pidfile, ::File::CREAT | ::File::EXCL | ::File::WRONLY) { |f| f.write(Process.pid.to_s) }
+      File.open(pidfile, ::File::CREAT | ::File::EXCL | ::File::WRONLY) { |f| f.write(::Process.pid.to_s) }
       at_exit { File.delete(pidfile) if File.exist?(pidfile) }
     rescue Errno::EEXIST
       check_pid
@@ -55,7 +55,7 @@ module GoodJob
       pid = ::File.read(pidfile).to_i
       return :dead if pid.zero?
 
-      Process.kill(0, pid) # check process status
+      ::Process.kill(0, pid) # check process status
       :running
     rescue Errno::ESRCH
       :dead

data/lib/good_job/execution.rb

@@ -1,10 +1,7 @@
 # frozen_string_literal: true
 module GoodJob
   # ActiveRecord model that represents an +ActiveJob+ job.
-  # Parent class can be configured with +GoodJob.active_record_parent_class+.
-  # @!parse
-  #   class Execution < ActiveRecord::Base; end
-  class Execution < Object.const_get(GoodJob.active_record_parent_class)
+  class Execution < BaseRecord
     include Lockable
     include Filterable
 
@@ -54,15 +51,7 @@ module GoodJob
       end
     end
 
-    def self._migration_pending_warning
-      ActiveSupport::Deprecation.warn(<<~DEPRECATION)
-        GoodJob has pending database migrations. To create the migration files, run:
-            rails generate good_job:update
-        To apply the migration files, run:
-            rails db:migrate
-      DEPRECATION
-      nil
-    end
+    belongs_to :job, class_name: 'GoodJob::ActiveJobJob', foreign_key: 'active_job_id', primary_key: 'active_job_id', optional: true, inverse_of: :executions
 
     # Get Jobs with given ActiveJob ID
     # @!method active_job_id
@@ -224,7 +213,7 @@ module GoodJob
           if @cron_at_index
             execution_args[:cron_at] = CurrentThread.cron_at
           else
-            _migration_pending_warning
+            migration_pending_warning!
           end
         elsif CurrentThread.active_job_id && CurrentThread.active_job_id == active_job.job_id
           execution_args[:cron_key] = CurrentThread.execution.cron_key

data/lib/good_job/job_performer.rb

@@ -57,6 +57,12 @@ module GoodJob
       job_query.next_scheduled_at(after: after, limit: limit, now_limit: now_limit)
     end
 
+    # Delete expired preserved jobs
+    # @return [void]
+    def cleanup
+      GoodJob.cleanup_preserved_jobs
+    end
+
     private
 
     attr_reader :queue_string

data/lib/good_job/notifier/process_registration.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module GoodJob # :nodoc:
+  class Notifier # :nodoc:
+    # Extends the Notifier to register the process in the database.
+    module ProcessRegistration
+      extend ActiveSupport::Concern
+
+      included do
+        set_callback :listen, :after, :register_process
+        set_callback :unlisten, :after, :deregister_process
+      end
+
+      # Registers the current process.
+      def register_process
+        GoodJob::Process.with_connection(connection) do
+          next unless Process.migrated?
+
+          GoodJob::Process.cleanup
+          @process = GoodJob::Process.register
+        end
+      end
+
+      # Deregisters the current process.
+      def deregister_process
+        GoodJob::Process.with_connection(connection) do
+          next unless Process.migrated?
+
+          @process&.deregister
+        end
+      end
+    end
+  end
+end

data/lib/good_job/notifier.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+require 'active_support/core_ext/module/attribute_accessors_per_thread'
 require 'concurrent/atomic/atomic_boolean'
 
 module GoodJob # :nodoc:
@@ -10,8 +11,10 @@ module GoodJob # :nodoc:
   # When a message is received, the notifier passes the message to each of its recipients.
   #
   class Notifier
-    # Raised if the Database adapter does not implement LISTEN.
-    AdapterCannotListenError = Class.new(StandardError)
+    include ActiveSupport::Callbacks
+    define_callbacks :listen, :unlisten
+
+    include Notifier::ProcessRegistration
 
     # Default Postgres channel for LISTEN/NOTIFY
     CHANNEL = 'good_job'
@@ -43,6 +46,12 @@ module GoodJob # :nodoc:
     #   @return [Array<GoodJob::Notifier>, nil]
     cattr_reader :instances, default: [], instance_reader: false
 
+    # @!attribute [rw] connection
+    #   @!scope class
+    #   ActiveRecord Connection that has been established for the Notifier.
+    #   @return [ActiveRecord::ConnectionAdapters::AbstractAdapter, nil]
+    thread_cattr_accessor :connection
+
     # Send a message via Postgres NOTIFY
     # @param message [#to_json]
     def self.notify(message)
@@ -117,8 +126,6 @@ module GoodJob # :nodoc:
     # @!visibility private
     # @return [void]
     def listen_observer(_time, _result, thread_error)
-      return if thread_error.is_a? AdapterCannotListenError
-
       if thread_error
         GoodJob._on_thread_error(thread_error)
         ActiveSupport::Notifications.instrument("notifier_notify_error.good_job", { error: thread_error })
@@ -146,30 +153,36 @@ module GoodJob # :nodoc:
 
     def listen(delay: 0)
       future = Concurrent::ScheduledTask.new(delay, args: [@recipients, executor, @listening], executor: @executor) do |thr_recipients, thr_executor, thr_listening|
-        with_listen_connection do |conn|
-          ActiveSupport::Notifications.instrument("notifier_listen.good_job") do
-            conn.async_exec("LISTEN #{CHANNEL}").clear
-          end
+        with_connection do
+          begin
+            run_callbacks :listen do
+              ActiveSupport::Notifications.instrument("notifier_listen.good_job") do
+                connection.execute("LISTEN #{CHANNEL}")
+              end
+              thr_listening.make_true
+            end
 
-          ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
-            thr_listening.make_true
-            while thr_executor.running?
-              conn.wait_for_notify(WAIT_INTERVAL) do |channel, _pid, payload|
-                next unless channel == CHANNEL
-
-                ActiveSupport::Notifications.instrument("notifier_notified.good_job", { payload: payload })
-                parsed_payload = JSON.parse(payload, symbolize_names: true)
-                thr_recipients.each do |recipient|
-                  target, method_name = recipient.is_a?(Array) ? recipient : [recipient, :call]
-                  target.send(method_name, parsed_payload)
+            ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
+              while thr_executor.running?
+                wait_for_notify do |channel, payload|
+                  next unless channel == CHANNEL
+
+                  ActiveSupport::Notifications.instrument("notifier_notified.good_job", { payload: payload })
+                  parsed_payload = JSON.parse(payload, symbolize_names: true)
+                  thr_recipients.each do |recipient|
+                    target, method_name = recipient.is_a?(Array) ? recipient : [recipient, :call]
+                    target.send(method_name, parsed_payload)
+                  end
                 end
               end
             end
           end
         ensure
-          thr_listening.make_false
-          ActiveSupport::Notifications.instrument("notifier_unlisten.good_job") do
-            conn.async_exec("UNLISTEN *").clear
+          run_callbacks :unlisten do
+            thr_listening.make_false
+            ActiveSupport::Notifications.instrument("notifier_unlisten.good_job") do
+              connection.execute("UNLISTEN *")
+            end
           end
         end
       end
@@ -178,17 +191,36 @@ module GoodJob # :nodoc:
       future.execute
     end
 
-    def with_listen_connection
-      ar_conn = Execution.connection_pool.checkout.tap do |conn|
+    def with_connection
+      self.connection = Execution.connection_pool.checkout.tap do |conn|
         Execution.connection_pool.remove(conn)
       end
-      pg_conn = ar_conn.raw_connection
-      raise AdapterCannotListenError unless pg_conn.respond_to? :wait_for_notify
+      connection.execute("SET application_name = #{connection.quote(self.class.name)}")
 
-      pg_conn.async_exec("SET application_name = #{pg_conn.escape_identifier(self.class.name)}").clear
-      yield pg_conn
+      yield
     ensure
-      ar_conn&.disconnect!
+      connection&.disconnect!
+      self.connection = nil
+    end
+
+    def wait_for_notify
+      raw_connection = connection.raw_connection
+      if raw_connection.respond_to?(:wait_for_notify)
+        raw_connection.wait_for_notify(WAIT_INTERVAL) do |channel, _pid, payload|
+          yield(channel, payload)
+        end
+      elsif raw_connection.respond_to?(:jdbc_connection)
+        raw_connection.execute_query("SELECT 1")
+        notifications = raw_connection.jdbc_connection.getNotifications
+        Array(notifications).each do |notification|
+          channel = notification.getName
+          payload = notification.getParameter
+          yield(channel, payload)
+        end
+        sleep WAIT_INTERVAL
+      else
+        sleep WAIT_INTERVAL
+      end
     end
   end
 end

data/lib/good_job/process.rb

@@ -0,0 +1,82 @@
+# frozen_string_literal: true
+require 'socket'
+
+module GoodJob # :nodoc:
+  # ActiveRecord model that represents an GoodJob process (either async or CLI).
+  class Process < BaseRecord
+    include AssignableConnection
+    include Lockable
+
+    self.table_name = 'good_job_processes'
+
+    cattr_reader :mutex, default: Mutex.new
+    cattr_accessor :_current_id, default: nil
+    cattr_accessor :_pid, default: nil
+
+    # Processes that are active and locked.
+    # @!method active
+    # @!scope class
+    # @return [ActiveRecord::Relation]
+    scope :active, -> { advisory_locked }
+
+    # Processes that are inactive and unlocked (e.g. SIGKILLed)
+    # @!method active
+    # @!scope class
+    # @return [ActiveRecord::Relation]
+    scope :inactive, -> { advisory_unlocked }
+
+    # Whether the +good_job_processes+ table exsists.
+    # @return [Boolean]
+    def self.migrated?
+      return true if connection.table_exists?(table_name)
+
+      migration_pending_warning!
+      false
+    end
+
+    # UUID that is unique to the current process and changes when forked.
+    # @return [String]
+    def self.current_id
+      mutex.synchronize do
+        if _current_id.nil? || _pid != ::Process.pid
+          self._current_id = SecureRandom.uuid
+          self._pid = ::Process.pid
+        end
+        _current_id
+      end
+    end
+
+    # Hash representing metadata about the current process.
+    # @return [Hash]
+    def self.current_state
+      {
+        id: current_id,
+        hostname: Socket.gethostname,
+        pid: ::Process.pid,
+        proctitle: $PROGRAM_NAME,
+        schedulers: GoodJob::Scheduler.instances.map(&:name),
+      }
+    end
+
+    # Deletes all inactive process records.
+    def self.cleanup
+      inactive.delete_all
+    end
+
+    # Registers the current process in the database
+    # @return [GoodJob::Process]
+    def self.register
+      create(id: current_id, state: current_state, create_with_advisory_lock: true)
+    rescue ActiveRecord::RecordNotUnique
+      nil
+    end
+
+    # Unregisters the instance.
+    def deregister
+      return unless owns_advisory_lock?
+
+      destroy!
+      advisory_unlock
+    end
+  end
+end

data/lib/good_job/scheduler.rb

@@ -48,7 +48,9 @@ module GoodJob # :nodoc:
           job_performer,
           max_threads: max_threads,
           max_cache: configuration.max_cache,
-          warm_cache_on_initialize: warm_cache_on_initialize
+          warm_cache_on_initialize: warm_cache_on_initialize,
+          cleanup_interval_seconds: configuration.cleanup_interval_seconds,
+          cleanup_interval_jobs: configuration.cleanup_interval_jobs
         )
       end
 
@@ -59,11 +61,17 @@ module GoodJob # :nodoc:
       end
     end
 
+    # Human readable name of the scheduler that includes configuration values.
+    # @return [String]
+    attr_reader :name
+
     # @param performer [GoodJob::JobPerformer]
     # @param max_threads [Numeric, nil] number of seconds between polls for jobs
     # @param max_cache [Numeric, nil] maximum number of scheduled jobs to cache in memory
     # @param warm_cache_on_initialize [Boolean] whether to warm the cache immediately, or manually by calling +warm_cache+
-    def initialize(performer, max_threads: nil, max_cache: nil, warm_cache_on_initialize: false)
+    # @param cleanup_interval_seconds [Numeric, nil] number of seconds between cleaning up job records
+    # @param cleanup_interval_jobs [Numeric, nil] number of executed jobs between cleaning up job records
+    def initialize(performer, max_threads: nil, max_cache: nil, warm_cache_on_initialize: false, cleanup_interval_seconds: nil, cleanup_interval_jobs: nil)
       raise ArgumentError, "Performer argument must implement #next" unless performer.respond_to?(:next)
 
       self.class.instances << self
@@ -76,8 +84,10 @@ module GoodJob # :nodoc:
         @executor_options[:max_threads] = max_threads
         @executor_options[:max_queue] = max_threads
       end
-      @executor_options[:name] = "GoodJob::Scheduler(queues=#{@performer.name} max_threads=#{@executor_options[:max_threads]})"
+      @name = "GoodJob::Scheduler(queues=#{@performer.name} max_threads=#{@executor_options[:max_threads]})"
+      @executor_options[:name] = name
 
+      @cleanup_tracker = CleanupTracker.new(cleanup_interval_seconds: cleanup_interval_seconds, cleanup_interval_jobs: cleanup_interval_jobs)
       create_executor
       warm_cache if warm_cache_on_initialize
     end
@@ -172,7 +182,14 @@ module GoodJob # :nodoc:
       GoodJob._on_thread_error(error) if error
 
       instrument("finished_job_task", { result: output, error: thread_error, time: time })
-      create_task if output
+      return unless output
+
+      @cleanup_tracker.increment
+      if @cleanup_tracker.cleanup?
+        cleanup
+      else
+        create_task
+      end
     end
 
     # Information about the Scheduler
@@ -210,7 +227,25 @@ module GoodJob # :nodoc:
         create_task # If cache-warming exhausts the threads, ensure there isn't an executable task remaining
       end
       future.add_observer(observer, :call)
+      future.execute
+    end
+
+    # Preload existing runnable and future-scheduled jobs
+    # @return [void]
+    def cleanup
+      @cleanup_tracker.reset
 
+      future = Concurrent::Future.new(args: [self, @performer], executor: executor) do |_thr_scheduler, thr_performer|
+        Rails.application.executor.wrap do
+          thr_performer.cleanup
+        end
+      end
+
+      observer = lambda do |_time, _output, thread_error|
+        GoodJob._on_thread_error(thread_error) if thread_error
+        create_task
+      end
+      future.add_observer(observer, :call)
       future.execute
     end
 

data/lib/good_job/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 module GoodJob
   # GoodJob gem version.
-  VERSION = '2.7.3'
+  VERSION = '2.9.0'
 end

data/lib/good_job.rb

@@ -131,16 +131,18 @@ module GoodJob
   # analyze or inspect job performance.
   # If you are preserving job records this way, use this method regularly to
   # delete old records and preserve space in your database.
-  # @params older_than [nil,Numeric,ActiveSupport::Duration] Jobs olders than this will be deleted (default: +86400+).
+  # @params older_than [nil,Numeric,ActiveSupport::Duration] Jobs older than this will be deleted (default: +86400+).
   # @return [Integer] Number of jobs that were deleted.
   def self.cleanup_preserved_jobs(older_than: nil)
     older_than ||= GoodJob::Configuration.new({}).cleanup_preserved_jobs_before_seconds_ago
     timestamp = Time.current - older_than
 
     ActiveSupport::Notifications.instrument("cleanup_preserved_jobs.good_job", { older_than: older_than, timestamp: timestamp }) do |payload|
-      deleted_records_count = GoodJob::Execution.finished(timestamp).delete_all
+      old_jobs = GoodJob::ActiveJobJob.where('finished_at <= ?', timestamp)
+      old_jobs_count = old_jobs.count
 
-      payload[:deleted_records_count] = deleted_records_count
+      GoodJob::Execution.where(job: old_jobs).delete_all
+      payload[:deleted_records_count] = old_jobs_count
     end
   end
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 2.7.3
+  version: 2.9.0
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2021-11-30 00:00:00.000000000 Z
+date: 2022-01-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -372,6 +372,7 @@ files:
 - engine/app/controllers/good_job/cron_entries_controller.rb
 - engine/app/controllers/good_job/executions_controller.rb
 - engine/app/controllers/good_job/jobs_controller.rb
+- engine/app/controllers/good_job/processes_controller.rb
 - engine/app/filters/good_job/base_filter.rb
 - engine/app/filters/good_job/executions_filter.rb
 - engine/app/filters/good_job/jobs_filter.rb
@@ -383,6 +384,7 @@ files:
 - engine/app/views/good_job/jobs/_table.erb
 - engine/app/views/good_job/jobs/index.html.erb
 - engine/app/views/good_job/jobs/show.html.erb
+- engine/app/views/good_job/processes/index.html.erb
 - engine/app/views/good_job/shared/_chart.erb
 - engine/app/views/good_job/shared/_filter.erb
 - engine/app/views/good_job/shared/icons/_arrow_clockwise.html.erb
@@ -402,12 +404,17 @@ files:
 - lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb.erb
 - lib/generators/good_job/templates/update/migrations/02_add_cron_at_to_good_jobs.rb.erb
 - lib/generators/good_job/templates/update/migrations/03_add_cron_key_cron_at_index_to_good_jobs.rb.erb
+- lib/generators/good_job/templates/update/migrations/04_create_good_job_processes.rb.erb
+- lib/generators/good_job/templates/update/migrations/04_index_good_job_jobs_on_finished_at.rb.erb
 - lib/generators/good_job/update_generator.rb
 - lib/good_job.rb
 - lib/good_job/active_job_extensions.rb
 - lib/good_job/active_job_extensions/concurrency.rb
 - lib/good_job/active_job_job.rb
 - lib/good_job/adapter.rb
+- lib/good_job/assignable_connection.rb
+- lib/good_job/base_record.rb
+- lib/good_job/cleanup_tracker.rb
 - lib/good_job/cli.rb
 - lib/good_job/configuration.rb
 - lib/good_job/cron_entry.rb
@@ -423,8 +430,10 @@ files:
 - lib/good_job/log_subscriber.rb
 - lib/good_job/multi_scheduler.rb
 - lib/good_job/notifier.rb
+- lib/good_job/notifier/process_registration.rb
 - lib/good_job/poller.rb
 - lib/good_job/probe_server.rb
+- lib/good_job/process.rb
 - lib/good_job/railtie.rb
 - lib/good_job/scheduler.rb
 - lib/good_job/version.rb
@@ -437,6 +446,7 @@ metadata:
   documentation_uri: https://rdoc.info/github/bensheldon/good_job
   homepage_uri: https://github.com/bensheldon/good_job
   source_code_uri: https://github.com/bensheldon/good_job
+  rubygems_mfa_required: 'true'
 post_install_message: 
 rdoc_options:
 - "--title"
@@ -460,7 +470,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.30
+rubygems_version: 3.1.6
 signing_key: 
 specification_version: 4
 summary: A multithreaded, Postgres-based ActiveJob backend for Ruby on Rails