good_job 1.11.3 → 1.13.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5e0f0ca8d7f8b024f5e84b515b8603d362b6c755d54680221e517369b6877c07
-  data.tar.gz: e69518c9d71d7b08e3707c9df8c6429ca4d9823fd1adea32adc6c0bf41ad33f4
+  metadata.gz: b9c8c8a0842bb8b25ee147d92186ca5596d4dc0bdc1a9a7638a094dc80c41806
+  data.tar.gz: 30b21a20dbe0b6d81854e29448b1218494359dfeaf8ebc4e0a51d4aaa87ca101
 SHA512:
-  metadata.gz: 50d2542bfad082dfba516c08bd994dad3fcd6e8311a8380dc012988c3e2745378271adc7d41cb91897d087a6b2b100cfb69362d3a11cd55c92c2610ceabeb80c
-  data.tar.gz: 69ffc79dc9367fc8522bdcf669cede6791d4e7903847edbc99887404b29ec43361de54a960841ee6cdeaaf236cc77d8c9c60b2c61793b5256bf2455667e0c387
+  metadata.gz: c2a0663d5af40fd294a9409c249677f73b683f5d50a3d2af58f120749429789de6f650ad58d61cc3704d0fa8a5ccc6681982ed106df37cf2d64856895e63892a
+  data.tar.gz: 0c8f717f2d763ecec47acc6dae85cadf153847d51cce0daf9c5b07f4aa953f4b57b612a79f0ae38335e9ed44192017567286448e761f1bfa688a5a6f04401820

data/CHANGELOG.md

@@ -1,5 +1,70 @@
 # Changelog
 
+## [v1.13.0](https://github.com/bensheldon/good_job/tree/v1.13.0) (2021-08-18)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.12.2...v1.13.0)
+
+**Implemented enhancements:**
+
+- Track if a GoodJob::Job has been subsequently retried [\#331](https://github.com/bensheldon/good_job/pull/331) ([bensheldon](https://github.com/bensheldon))
+- Wrap and truncate error message, which can be a huge text [\#294](https://github.com/bensheldon/good_job/pull/294) ([morgoth](https://github.com/morgoth))
+
+**Closed issues:**
+
+- Add hyphen to lock string. e.g. "table\_name-column" instead of "table\_namecolumn [\#334](https://github.com/bensheldon/good_job/issues/334)
+- Optimize db indexes in advance of v2.0.0 [\#332](https://github.com/bensheldon/good_job/issues/332)
+- wait\_until in development? [\#330](https://github.com/bensheldon/good_job/issues/330)
+- Race conditions in ActiveJob concurrency extension [\#325](https://github.com/bensheldon/good_job/issues/325)
+- Store in database if a job has been ActiveJob retried [\#321](https://github.com/bensheldon/good_job/issues/321)
+- Revisit and embrace concurrency control, scheduled jobs, and other extensions of ActiveJob [\#255](https://github.com/bensheldon/good_job/issues/255)
+- Why 1 million jobs per day? [\#222](https://github.com/bensheldon/good_job/issues/222)
+
+## [v1.12.2](https://github.com/bensheldon/good_job/tree/v1.12.2) (2021-08-13)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.12.1...v1.12.2)
+
+**Fixed bugs:**
+
+- Fixes for race conditions in ActiveJob concurrency extension [\#326](https://github.com/bensheldon/good_job/pull/326) ([codyrobbins](https://github.com/codyrobbins))
+
+**Merged pull requests:**
+
+- On gem release, add instructions to author a Github Release [\#324](https://github.com/bensheldon/good_job/pull/324) ([bensheldon](https://github.com/bensheldon))
+
+## [v1.12.1](https://github.com/bensheldon/good_job/tree/v1.12.1) (2021-08-05)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.12.0...v1.12.1)
+
+**Fixed bugs:**
+
+- Ensure CLI can shutdown cleanly with multiple queues and timeout [\#319](https://github.com/bensheldon/good_job/pull/319) ([bensheldon](https://github.com/bensheldon))
+
+**Closed issues:**
+
+- Setting a shutdown timeout causes the CLI executor to throw an exception on shutdown. [\#318](https://github.com/bensheldon/good_job/issues/318)
+- PgBouncer and prepared statements [\#269](https://github.com/bensheldon/good_job/issues/269)
+- Question about locking internals [\#212](https://github.com/bensheldon/good_job/issues/212)
+- Encoding::UndefinedConversionError \("\xE2" from ASCII-8BIT to UTF-8\) [\#198](https://github.com/bensheldon/good_job/issues/198)
+- tools for managing a 'fleet' of processes [\#150](https://github.com/bensheldon/good_job/issues/150)
+
+**Merged pull requests:**
+
+- Fix Readme lint warnings [\#320](https://github.com/bensheldon/good_job/pull/320) ([bensheldon](https://github.com/bensheldon))
+
+## [v1.12.0](https://github.com/bensheldon/good_job/tree/v1.12.0) (2021-07-27)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.11.3...v1.12.0)
+
+**Implemented enhancements:**
+
+- Add the ability to schedule repeating / recurring / cron-like jobs [\#53](https://github.com/bensheldon/good_job/issues/53)
+- Add cron-like support for recurring/repeating jobs [\#297](https://github.com/bensheldon/good_job/pull/297) ([bensheldon](https://github.com/bensheldon))
+
+**Fixed bugs:**
+
+- Place Dashboard shared view partials under `good_job` namespace [\#310](https://github.com/bensheldon/good_job/pull/310) ([bensheldon](https://github.com/bensheldon))
+- Ensure Dashboard inline javascript has CSP nonce for strict Content-Security Policy [\#309](https://github.com/bensheldon/good_job/pull/309) ([bensheldon](https://github.com/bensheldon))
+
 ## [v1.11.3](https://github.com/bensheldon/good_job/tree/v1.11.3) (2021-07-25)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.11.2...v1.11.3)

data/README.md

@@ -10,7 +10,7 @@ GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails.
 - **Designed for ActiveJob.** Complete support for [async, queues, delays, priorities, timeouts, and retries](https://edgeguides.rubyonrails.org/active_job_basics.html) with near-zero configuration.
 - **Built for Rails.** Fully adopts Ruby on Rails [threading and code execution guidelines](https://guides.rubyonrails.org/threading_and_code_execution.html) with [Concurrent::Ruby](https://github.com/ruby-concurrency/concurrent-ruby).
 - **Backed by Postgres.** Relies upon Postgres integrity, session-level Advisory Locks to provide run-once safety and stay within the limits of `schema.rb`, and LISTEN/NOTIFY to reduce queuing latency.
-- **For most workloads.** Targets full-stack teams, economy-minded solo developers, and applications that enqueue less than 1-million jobs/day.
+- **For most workloads.** Targets full-stack teams, economy-minded solo developers, and applications that enqueue 1-million jobs/day and more.
 
 For more of the story of GoodJob, read the [introductory blog post](https://island94.org/2020/07/introducing-goodjob-1-0).
 
@@ -38,7 +38,8 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
     - [Configuration options](#configuration-options)
     - [Global options](#global-options)
     - [Dashboard](#dashboard)
-    - [ActiveJob Concurrency](#activejob-concurrency)
+    - [ActiveJob concurrency](#activejob-concurrency)
+    - [Cron-style repeating/recurring jobs](#cron-style-repeatingrecurring-jobs)
     - [Updating](#updating)
 - [Go deeper](#go-deeper)
     - [Exceptions, retries, and reliability](#exceptions-retries-and-reliability)
@@ -51,6 +52,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
     - [Execute jobs async / in-process](#execute-jobs-async--in-process)
     - [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
     - [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
+    - [PgBouncer compatibility](#pgbouncer-compatibility)
 - [Contribute](#contribute)
     - [Gem development](#gem-development)
     - [Release](#release)
@@ -156,6 +158,7 @@ Options:
   [--poll-interval=SECONDS]    # Interval between polls for available jobs in seconds (env var: GOOD_JOB_POLL_INTERVAL, default: 1)
   [--max-cache=COUNT]          # Maximum number of scheduled jobs to cache in memory (env var: GOOD_JOB_MAX_CACHE, default: 10000)
   [--shutdown-timeout=SECONDS] # Number of seconds to wait for jobs to finish when shutting down before stopping the thread. (env var: GOOD_JOB_SHUTDOWN_TIMEOUT, default: -1 (forever))
+  [--enable-cron]              # Whether to run cron process (default: false)
   [--daemonize]                # Run as a background daemon (default: false)
   [--pidfile=PIDFILE]          # Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)
 
@@ -212,7 +215,8 @@ config.good_job.execution_mode = :async_server
 config.good_job.max_threads = 5
 config.good_job.poll_interval = 30 # seconds
 config.good_job.shutdown_timeout = 25 # seconds
-
+config.good_job.enable_cron = true
+config.good_job.cron = { example: { cron: '0 * * * *', class: 'ExampleJob'  } }
 
 # ...or all at once.
 config.good_job = {
@@ -220,6 +224,13 @@ config.good_job = {
   max_threads: 5,
   poll_interval: 30,
   shutdown_timeout: 25,
+  enable_cron: true,
+  cron: {
+    example: {
+      cron: '0 * * * *',
+      class: 'ExampleJob'
+    },
+  },
 }
 ```
 
@@ -235,6 +246,8 @@ Available configuration options are:
 - `poll_interval` (integer) sets the number of seconds between polls for jobs when `execution_mode` is set to `:async` or `:async_server`. You can also set this with the environment variable `GOOD_JOB_POLL_INTERVAL`.
 - `max_cache` (integer) sets the maximum number of scheduled jobs that will be stored in memory to reduce execution latency when also polling for scheduled jobs. Caching 10,000 scheduled jobs uses approximately 20MB of memory. You can also set this with the environment variable `GOOD_JOB_MAX_CACHE`.
 - `shutdown_timeout` (float) number of seconds to wait for jobs to finish when shutting down before stopping the thread. Defaults to forever: `-1`. You can also set this with the environment variable `GOOD_JOB_SHUTDOWN_TIMEOUT`.
+- `enable_cron` (boolean) whether to run cron process. Defaults to `false`. You can also set this with the environment variable `GOOD_JOB_ENABLE_CRON`.
+- `cron` (hash) cron configuration. Defaults to `{}`. You can also set this as a JSON string with the environment variable `GOOD_JOB_CRON`
 
 By default, GoodJob configures the following execution modes per environment:
 
@@ -320,9 +333,9 @@ GoodJob includes a Dashboard as a mountable `Rails::Engine`.
     end
     ```
 
-### ActiveJob Concurrency
+### ActiveJob concurrency
 
-GoodJob can extend ActiveJob to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_.
+GoodJob can extend ActiveJob to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_. Limiting concurrency can help prevent duplicate, double or unecessary jobs from being enqueued, or race conditions when performing, for example when interacting with 3rd-party APIs.
 
 **Note:** Limiting concurrency at _enqueue_ requires Rails 6.0+ because Rails 5.2 does not support `throw :abort` in ActiveJob callbacks.
 
@@ -349,6 +362,45 @@ class MyJob < ApplicationJob
 end
 ```
 
+When testing, the resulting concurrency key value can be inspected:
+
+```ruby
+job = MyJob.perform_later("Alice")
+job.good_job_concurrency_key #=> "Unique-Alice"
+```
+
+### Cron-style repeating/recurring jobs
+
+GoodJob can enqueue jobs on a recurring basis that can be used as a replacement for cron.
+
+Cron-style jobs are run on every GoodJob process (e.g. CLI or `async` execution mode) when `config.good_job.enable_cron = true`; use GoodJob's [ActiveJob concurrency](#activejob-concurrency) extension to limit the number of jobs that are enqueued.
+
+Cron-format is parsed by the [`fugit`](https://github.com/floraison/fugit) gem, which has support for seconds-level resolution (e.g. `* * * * * *`).
+
+```ruby
+# config/environments/application.rb or a specific environment e.g. production.rb
+
+# Enable cron in this process; e.g. only run on the first Heroku worker process
+config.good_job.enable_cron = ENV['DYNO'] == 'worker.1' # or `true` or via $GOOD_JOB_ENABLE_CRON
+
+# Configure cron with a hash that has a unique key for each recurring job
+config.good_job.cron = {
+  # Every 15 minutes, enqueue `ExampleJob.set(priority: -10).perform_later(52, name: "Alice")`
+  frequent_task: { # each recurring job must have a unique key
+    cron: "*/15 * * * *", # cron-style scheduling format by fugit gem
+    class: "ExampleJob", # reference the Job class with a string
+    args: [42, { name: "Alice" }], # arguments to pass; can also be a proc e.g. `-> { { when: Time.now } }`
+    set: { priority: -10 }, # additional ActiveJob properties; can also be a lambda/proc e.g. `-> { { priority: [1,2].sample } }`
+    description: "Something helpful", # optional description that appears in Dashboard (coming soon!)
+  },
+  another_task: {
+    cron: "0 0,12 * * *",
+    class: "AnotherJob",
+  },
+  # etc.
+}
+```
+
 ### Updating
 
 GoodJob follows semantic versioning, though updates may be encouraged through deprecation warnings in minor versions.
@@ -636,6 +688,43 @@ It is also necessary to delete these preserved jobs from the database after a ce
     $ bundle exec good_job cleanup_preserved_jobs --before-seconds-ago=86400
     ```
 
+### PgBouncer compatibility
+
+GoodJob is not compatible with PgBouncer in _transaction_ mode, but is compatible with PgBouncer's _connection_ mode. GoodJob uses connection-based advisory locks and LISTEN/NOTIFY, both of which require full database connections.
+
+A workaround to this limitation is to make a direct database connection available to GoodJob. With Rails 6.0's support for [multiple databases](https://guides.rubyonrails.org/active_record_multiple_databases.html), a direct connection to the database can be configured:
+
+1. Define a direct connection to your database that is not proxied through PgBouncer, for example:
+
+    ```yml
+    # config/database.yml
+
+    production:
+      primary:
+        url: postgres://pgbouncer_host/my_database
+      primary_direct:
+        url: postgres://database_host/my_database
+    ```
+
+1. Create a new ActiveRecord base class that uses the direct database connection
+
+    ```ruby
+    # app/models/application_direct_record.rb
+
+    class ApplicationDirectRecord < ActiveRecord::Base
+      self.abstract_class = true
+      connects_to database: :primary_direct
+    end
+    ```
+
+1. Configure GoodJob to use the newly created ActiveRecord base class:
+
+    ```ruby
+    # config/initializers/good_job.rb
+
+    GoodJob.active_record_parent_class = "ApplicationDirectRecord"
+    ```
+
 ## Contribute
 
 Contributions are welcomed and appreciated 🙏

data/engine/app/views/good_job/active_jobs/show.html.erb

@@ -1 +1 @@
-<%= render 'shared/jobs_table', jobs: @jobs %>
+<%= render 'good_job/shared/jobs_table', jobs: @jobs %>

data/engine/app/views/good_job/dashboards/index.html.erb

@@ -1,5 +1,5 @@
 <div class="card my-3 p-6">
-  <%= render 'shared/chart', chart_data: @chart %>
+  <%= render 'good_job/shared/chart', chart_data: @chart %>
 </div>
 
 <div class='card mb-2'>
@@ -38,7 +38,7 @@
 </div>
 
 <% if @filter.jobs.present? %>
-  <%= render 'shared/jobs_table', jobs: @filter.jobs %>
+  <%= render 'good_job/shared/jobs_table', jobs: @filter.jobs %>
 
   <nav aria-label="Job pagination" class="mt-3">
     <ul class="pagination">

data/engine/app/views/{shared → good_job/shared}/_chart.erb

@@ -1,6 +1,6 @@
 <div id="chart"></div>
 
-<script>
+<%= javascript_tag nonce: true do %>
   new Chartist.Line('#chart', <%== chart_data.to_json %>, {
     height: '300px',
     fullWidth: true,
@@ -49,4 +49,4 @@
     tooltipEl.style.left = (event.offsetX || event.originalEvent.layerX) + tooltipEl.offsetWidth + 10 + 'px';
     tooltipEl.style.top = (event.offsetY || event.originalEvent.layerY) + tooltipEl.offsetHeight - 20 + 'px';
   }, true);
-</script>
+<% end %>

data/engine/app/views/{shared → good_job/shared}/_jobs_table.erb

@@ -19,7 +19,7 @@
             <td><%= job.serialized_params['job_class'] %></td>
             <td><%= job.queue_name %></td>
             <td><%= job.scheduled_at || job.created_at %></td>
-            <td><%= job.error %></td>
+            <td class="text-break"><%= truncate(job.error, length: 1_000) %></td>
             <td>
               <%= tag.button "Preview", type: "button", class: "btn btn-sm btn-outline-primary", role: "button",
                 data: {bs_toggle: "collapse", bs_target: "##{dom_id(job, 'params')}"},
@@ -29,7 +29,7 @@
             </td>
             <td>
               <%= button_to job_path(job.id), method: :delete, class: "btn btn-sm btn-outline-danger", title: "Delete job" do %>
-                <%= render "shared/icons/trash" %>
+                <%= render "good_job/shared/icons/trash" %>
               <% end %>
             </td>
           </tr>

data/engine/app/views/layouts/good_job/base.html.erb

@@ -53,13 +53,13 @@
 
     <% if notice %>
       <div class="alert alert-success alert-dismissible fade show d-flex align-items-center offset-md-3 col-6" role="alert">
-        <%= render "shared/icons/check", class: "flex-shrink-0 me-2" %>
+        <%= render "good_job/shared/icons/check", class: "flex-shrink-0 me-2" %>
         <div><%= notice %></div>
         <button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>
       </div>
     <% elsif alert %>
       <div class="alert alert-warning alert-dismissible fade show d-flex align-items-center offset-md-3 col-6" role="alert">
-        <%= render "shared/icons/check", class: "flex-shrink-0 me-2" %>
+        <%= render "good_job/shared/icons/check", class: "flex-shrink-0 me-2" %>
         <div><%= alert %></div>
         <button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>
       </div>

data/lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb

@@ -16,6 +16,7 @@ class CreateGoodJobs < ActiveRecord::Migration[5.2]
       t.uuid :active_job_id
       t.text :concurrency_key
       t.text :cron_key
+      t.uuid :retried_good_job_id
     end
 
     add_index :good_jobs, :scheduled_at, where: "(finished_at IS NULL)", name: "index_good_jobs_on_scheduled_at"

data/lib/generators/good_job/templates/update/migrations/04_add_retried_good_job_id_to_good_jobs.rb

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+class AddRetriedGoodJobIdToGoodJobs < ActiveRecord::Migration[5.2]
+  def change
+    reversible do |dir|
+      dir.up do
+        # Ensure this incremental update migration is idempotent
+        # with monolithic install migration.
+        return if connection.column_exists?(:good_jobs, :retried_good_job_id)
+      end
+    end
+
+    add_column :good_jobs, :retried_good_job_id, :uuid
+  end
+end

data/lib/good_job.rb

@@ -106,16 +106,13 @@ module GoodJob
                 wait ? -1 : nil
               end
 
-    executables = Array(Notifier.instances) + Array(Poller.instances) + Array(Scheduler.instances)
-    _shutdown_all(executables, timeout: timeout)
+    _shutdown_all(_executables, timeout: timeout)
   end
 
   # Tests whether jobs have stopped executing.
   # @return [Boolean] whether background threads are shut down
   def self.shutdown?
-    Notifier.instances.all?(&:shutdown?) &&
-      Poller.instances.all?(&:shutdown?) &&
-      Scheduler.instances.all?(&:shutdown?)
+    _executables.all?(&:shutdown?)
   end
 
   # Stops and restarts executing jobs.
@@ -126,17 +123,16 @@ module GoodJob
   # @param timeout [Numeric, nil] Seconds to wait for active threads to finish.
   # @return [void]
   def self.restart(timeout: -1)
-    executables = Array(Notifier.instances) + Array(Poller.instances) + Array(Scheduler.instances)
-    _shutdown_all(executables, :restart, timeout: timeout)
+    _shutdown_all(_executables, :restart, timeout: timeout)
   end
 
-  # Sends +#shutdown+ or +#restart+ to executable objects ({GoodJob::Notifier}, {GoodJob::Poller}, {GoodJob::Scheduler})
-  # @param executables [Array<Notifier, Poller, Scheduler, MultiScheduler>] Objects to shut down.
+  # Sends +#shutdown+ or +#restart+ to executable objects ({GoodJob::Notifier}, {GoodJob::Poller}, {GoodJob::Scheduler}, {GoodJob::MultiScheduler}, {GoodJob::CronManager})
+  # @param executables [Array<Notifier, Poller, Scheduler, MultiScheduler, CronManager>] Objects to shut down.
   # @param method_name [:symbol] Method to call, e.g. +:shutdown+ or +:restart+.
   # @param timeout [nil,Numeric]
   # @return [void]
   def self._shutdown_all(executables, method_name = :shutdown, timeout: -1)
-    if timeout.positive?
+    if timeout.is_a?(Numeric) && timeout.positive?
       executables.each { |executable| executable.send(method_name, timeout: nil) }
 
       stop_at = Time.current + timeout
@@ -146,5 +142,14 @@ module GoodJob
     end
   end
 
+  def self._executables
+    [].concat(
+      CronManager.instances,
+      Notifier.instances,
+      Poller.instances,
+      Scheduler.instances
+    )
+  end
+
   ActiveSupport.run_load_hooks(:good_job, self)
 end

data/lib/good_job/active_job_extensions/concurrency.rb

@@ -9,21 +9,21 @@ module GoodJob
       included do
         class_attribute :good_job_concurrency_config, instance_accessor: false, default: {}
 
-        before_enqueue do |job|
+        around_enqueue do |job, block|
           # Always allow jobs to be retried because the current job's execution will complete momentarily
-          next if CurrentExecution.active_job_id == job.job_id
+          next(block.call) if CurrentExecution.active_job_id == job.job_id
 
           limit = job.class.good_job_concurrency_config.fetch(:enqueue_limit, Float::INFINITY)
-          next if limit.blank? || (0...Float::INFINITY).exclude?(limit)
+          next(block.call) if limit.blank? || (0...Float::INFINITY).exclude?(limit)
 
           key = job.good_job_concurrency_key
-          next if key.blank?
+          next(block.call) if key.blank?
 
           GoodJob::Job.new.with_advisory_lock(key: key, function: "pg_advisory_lock") do
             # TODO: Why is `unscoped` necessary? Nested scope is bleeding into subsequent query?
             enqueue_concurrency = GoodJob::Job.unscoped.where(concurrency_key: key).unfinished.count
             # The job has not yet been enqueued, so check if adding it will go over the limit
-            throw :abort if enqueue_concurrency + 1 > limit
+            block.call unless enqueue_concurrency + 1 > limit
           end
         end
 
@@ -41,9 +41,9 @@ module GoodJob
           next if key.blank?
 
           GoodJob::Job.new.with_advisory_lock(key: key, function: "pg_advisory_lock") do
-            perform_concurrency = GoodJob::Job.unscoped.where(concurrency_key: key).advisory_locked.count
+            allowed_active_job_ids = GoodJob::Job.unscoped.where(concurrency_key: key).advisory_locked.order(Arel.sql("COALESCE(performed_at, scheduled_at, created_at) ASC")).limit(limit).pluck(:active_job_id)
             # The current job has already been locked and will appear in the previous query
-            raise GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError if perform_concurrency > limit
+            raise GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError unless allowed_active_job_ids.include? job.job_id
           end
         end
       end

data/lib/good_job/adapter.rb

@@ -57,6 +57,8 @@ module GoodJob
         @scheduler = GoodJob::Scheduler.from_configuration(@configuration, warm_cache_on_initialize: Rails.application.initialized?)
         @notifier.recipients << [@scheduler, :create_thread]
         @poller.recipients << [@scheduler, :create_thread]
+
+        @cron_manager = GoodJob::CronManager.new(@configuration.cron, start_on_initialize: Rails.application.initialized?) if @configuration.enable_cron?
       end
     end
 

data/lib/good_job/cli.rb

@@ -70,12 +70,16 @@ module GoodJob
                   type: :numeric,
                   banner: 'SECONDS',
                   desc: "Number of seconds to wait for jobs to finish when shutting down before stopping the thread. (env var: GOOD_JOB_SHUTDOWN_TIMEOUT, default: -1 (forever))"
+    method_option :enable_cron,
+                  type: :boolean,
+                  desc: "Whether to run cron process (default: false)"
     method_option :daemonize,
                   type: :boolean,
                   desc: "Run as a background daemon (default: false)"
     method_option :pidfile,
                   type: :string,
                   desc: "Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)"
+
     def start
       set_up_application!
       configuration = GoodJob::Configuration.new(options)
@@ -87,7 +91,7 @@ module GoodJob
       scheduler = GoodJob::Scheduler.from_configuration(configuration, warm_cache_on_initialize: true)
       notifier.recipients << [scheduler, :create_thread]
       poller.recipients << [scheduler, :create_thread]
-
+      cron_manager = GoodJob::CronManager.new(configuration.cron, start_on_initialize: true) if configuration.enable_cron?
       @stop_good_job_executable = false
       %w[INT TERM].each do |signal|
         trap(signal) { @stop_good_job_executable = true }
@@ -98,7 +102,7 @@ module GoodJob
         break if @stop_good_job_executable || scheduler.shutdown? || notifier.shutdown?
       end
 
-      executors = [notifier, poller, scheduler]
+      executors = [notifier, poller, cron_manager, scheduler].compact
       GoodJob._shutdown_all(executors, timeout: configuration.shutdown_timeout)
     end
 
@@ -124,6 +128,7 @@ module GoodJob
                   type: :numeric,
                   banner: 'SECONDS',
                   desc: "Delete records finished more than this many seconds ago (env var: GOOD_JOB_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO, default: 86400)"
+
     def cleanup_preserved_jobs
       set_up_application!
 

data/lib/good_job/configuration.rb

@@ -18,6 +18,8 @@ module GoodJob
     DEFAULT_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO = 24 * 60 * 60
     # Default to always wait for jobs to finish for {Adapter#shutdown}
     DEFAULT_SHUTDOWN_TIMEOUT = -1
+    # Default to not running cron
+    DEFAULT_ENABLE_CRON = false
 
     # The options that were explicitly set when initializing +Configuration+.
     # @return [Hash]
@@ -129,6 +131,28 @@ module GoodJob
       ).to_f
     end
 
+    # Whether to run cron
+    # @return [Boolean]
+    def enable_cron
+      value = ActiveModel::Type::Boolean.new.cast(
+        options[:enable_cron] ||
+        rails_config[:enable_cron] ||
+        env['GOOD_JOB_ENABLE_CRON'] ||
+        false
+      )
+      value && cron.size.positive?
+    end
+    alias enable_cron? enable_cron
+
+    def cron
+      env_cron = JSON.parse(ENV['GOOD_JOB_CRON']) if ENV['GOOD_JOB_CRON'].present?
+
+      options[:cron] ||
+        rails_config[:cron] ||
+        env_cron ||
+        {}
+    end
+
     # Number of seconds to preserve jobs when using the +good_job cleanup_preserved_jobs+ CLI command.
     # This configuration is only used when {GoodJob.preserve_job_records} is +true+.
     # @return [Integer]

data/lib/good_job/cron_manager.rb

@@ -0,0 +1,115 @@
+# frozen_string_literal: true
+require "concurrent/hash"
+require "concurrent/scheduled_task"
+require "fugit"
+
+module GoodJob # :nodoc:
+  #
+  # CronManagers enqueue jobs on a repeating schedule.
+  #
+  class CronManager
+    # @!attribute [r] instances
+    #   @!scope class
+    #   List of all instantiated CronManagers in the current process.
+    #   @return [Array<GoodJob::CronManagers>, nil]
+    cattr_reader :instances, default: [], instance_reader: false
+
+    # Task observer for cron task
+    # @param time [Time]
+    # @param output [Object]
+    # @param thread_error [Exception]
+    def self.task_observer(time, output, thread_error) # rubocop:disable Lint/UnusedMethodArgument
+      return if thread_error.is_a? Concurrent::CancelledOperationError
+
+      GoodJob.on_thread_error.call(thread_error) if thread_error && GoodJob.on_thread_error.respond_to?(:call)
+    end
+
+    # Job configuration to be scheduled
+    # @return [Hash]
+    attr_reader :schedules
+
+    # @param schedules [Hash]
+    # @param start_on_initialize [Boolean]
+    def initialize(schedules = {}, start_on_initialize: false)
+      @running = false
+      @schedules = schedules
+      @tasks = Concurrent::Hash.new
+
+      self.class.instances << self
+
+      start if start_on_initialize
+    end
+
+    # Schedule tasks that will enqueue jobs based on their schedule
+    def start
+      ActiveSupport::Notifications.instrument("cron_manager_start.good_job", cron_jobs: @schedules) do
+        @running = true
+        schedules.each_key { |cron_key| create_task(cron_key) }
+      end
+    end
+
+    # Stop/cancel any scheduled tasks
+    # @param timeout [Numeric, nil] Unused but retained for compatibility
+    def shutdown(timeout: nil) # rubocop:disable Lint/UnusedMethodArgument
+      @running = false
+      @tasks.each do |_cron_key, task|
+        task.cancel
+      end
+      @tasks.clear
+    end
+
+    # Stop and restart
+    # @param timeout [Numeric, nil] Unused but retained for compatibility
+    def restart(timeout: nil) # rubocop:disable Lint/UnusedMethodArgument
+      shutdown
+      start
+    end
+
+    # Tests whether the manager is running.
+    # @return [Boolean, nil]
+    def running?
+      @running
+    end
+
+    # Tests whether the manager is shutdown.
+    # @return [Boolean, nil]
+    def shutdown?
+      !running?
+    end
+
+    # Enqueues a scheduled task
+    # @param cron_key [Symbol, String] the key within the schedule to use
+    def create_task(cron_key)
+      schedule = @schedules[cron_key]
+      return false if schedule.blank?
+
+      fugit = Fugit::Cron.parse(schedule.fetch(:cron))
+      delay = [(fugit.next_time - Time.current).to_f, 0].max
+
+      future = Concurrent::ScheduledTask.new(delay, args: [self, cron_key]) do |thr_scheduler, thr_cron_key|
+        # Re-schedule the next cron task before executing the current task
+        thr_scheduler.create_task(thr_cron_key)
+
+        CurrentExecution.reset
+        CurrentExecution.cron_key = thr_cron_key
+
+        Rails.application.executor.wrap do
+          schedule = thr_scheduler.schedules.fetch(thr_cron_key).with_indifferent_access
+          job_class = schedule.fetch(:class).constantize
+
+          job_set_value = schedule.fetch(:set, {})
+          job_set = job_set_value.respond_to?(:call) ? job_set_value.call : job_set_value
+
+          job_args_value = schedule.fetch(:args, [])
+          job_args = job_args_value.respond_to?(:call) ? job_args_value.call : job_args_value
+
+          job_class.set(job_set).perform_later(*job_args)
+        end
+      end
+
+      @tasks[cron_key] = future
+      future.add_observer(self.class, :task_observer)
+      future.execute
+    end
+  end
+end

data/lib/good_job/current_execution.rb

@@ -5,11 +5,11 @@ module GoodJob
   # Thread-local attributes for passing values from Instrumentation.
   # (Cannot use ActiveSupport::CurrentAttributes because ActiveJob resets it)
   module CurrentExecution
-    # @!attribute [rw] active_job_id
+    # @!attribute [rw] cron_key
     #   @!scope class
-    #   ActiveJob ID
+    #   Cron Key
     #   @return [String, nil]
-    thread_mattr_accessor :active_job_id
+    thread_mattr_accessor :cron_key
 
     # @!attribute [rw] error_on_discard
     #   @!scope class
@@ -23,14 +23,26 @@ module GoodJob
     #   @return [Exception, nil]
     thread_mattr_accessor :error_on_retry
 
+    # @!attribute [rw] good_job
+    #   @!scope class
+    #   Cron Key
+    #   @return [GoodJob::Job, nil]
+    thread_mattr_accessor :good_job
+
     # Resets attributes
     # @return [void]
     def self.reset
-      self.active_job_id = nil
+      self.cron_key = nil
+      self.good_job = nil
       self.error_on_discard = nil
       self.error_on_retry = nil
     end
 
+    # @return [String] UUID of the currently executing GoodJob::Job
+    def self.active_job_id
+      good_job&.active_job_id
+    end
+
     # @return [Integer] Current process ID
     def self.process_id
       Process.pid

data/lib/good_job/job.rb

@@ -52,6 +52,20 @@ module GoodJob
       end
     end
 
+    def self._migration_pending_warning
+      ActiveSupport::Deprecation.warn(<<~DEPRECATION)
+        GoodJob has pending database migrations. To create the migration files, run:
+
+            rails generate good_job:update
+
+        To apply the migration files, run:
+
+            rails db:migrate
+
+      DEPRECATION
+      nil
+    end
+
     # Get Jobs with given class name
     # @!method with_job_class
     # @!scope class
@@ -110,6 +124,18 @@ module GoodJob
     # @return [ActiveRecord::Relation]
     scope :running, -> { where.not(performed_at: nil).where(finished_at: nil) }
 
+    # Get Jobs that do not have subsequent retries
+    # @!method running
+    # @!scope class
+    # @return [ActiveRecord::Relation]
+    scope :head, -> { where(retried_good_job_id: nil) }
+
+    # Get Jobs have errored that will not be retried further
+    # @!method running
+    # @!scope class
+    # @return [ActiveRecord::Relation]
+    scope :dead, -> { head.where.not(error: nil) }
+
     # Get Jobs on queues that match the given queue string.
     # @!method queue_string(string)
     # @!scope class
@@ -209,31 +235,23 @@ module GoodJob
         if column_names.include?('active_job_id')
           good_job_args[:active_job_id] = active_job.job_id
         else
-          ActiveSupport::Deprecation.warn(<<~DEPRECATION)
-            GoodJob has pending database migrations. To create the migration files, run:
-
-                rails generate good_job:update
-
-            To apply the migration files, run:
-
-                rails db:migrate
-
-          DEPRECATION
+          _migration_pending_warning
         end
 
         if column_names.include?('concurrency_key')
           good_job_args[:concurrency_key] = active_job.good_job_concurrency_key if active_job.respond_to?(:good_job_concurrency_key)
         else
-          ActiveSupport::Deprecation.warn(<<~DEPRECATION)
-            GoodJob has pending database migrations. To create the migration files, run:
-
-                rails generate good_job:update
-
-            To apply the migration files, run:
-
-                rails db:migrate
+          _migration_pending_warning
+        end
 
-          DEPRECATION
+        if column_names.include?('cron_key')
+          if CurrentExecution.cron_key
+            good_job_args[:cron_key] = CurrentExecution.cron_key
+          elsif CurrentExecution.active_job_id == active_job.job_id
+            good_job_args[:cron_key] = CurrentExecution.good_job.cron_key
+          end
+        else
+          _migration_pending_warning
         end
 
         good_job = GoodJob::Job.new(**good_job_args)
@@ -243,6 +261,12 @@ module GoodJob
         good_job.save!
         active_job.provider_job_id = good_job.id
 
+        if column_names.include?('retried_good_job_id')
+          CurrentExecution.good_job.retried_good_job_id = good_job.id if CurrentExecution.good_job && CurrentExecution.good_job.active_job_id == active_job.job_id
+        else
+          _migration_pending_warning
+        end
+
         good_job
       end
     end
@@ -282,7 +306,21 @@ module GoodJob
     end
 
     def active_job_id
-      super || serialized_params['job_id']
+      if self.class.column_names.include?('active_job_id')
+        super
+      else
+        self.class._migration_pending_warning
+        serialized_params['job_id']
+      end
+    end
+
+    def cron_key
+      if self.class.column_names.include?('cron_key')
+        super
+      else
+        self.class._migration_pending_warning
+        nil
+      end
     end
 
     private
@@ -294,7 +332,7 @@ module GoodJob
       )
 
       GoodJob::CurrentExecution.reset
-      GoodJob::CurrentExecution.active_job_id = active_job_id
+      GoodJob::CurrentExecution.good_job = self
       ActiveSupport::Notifications.instrument("perform_job.good_job", { good_job: self, process_id: GoodJob::CurrentExecution.process_id, thread_name: GoodJob::CurrentExecution.thread_name }) do
         value = ActiveJob::Base.execute(params)
 

data/lib/good_job/log_subscriber.rb

@@ -57,6 +57,16 @@ module GoodJob
       end
     end
 
+    # @!macro notification_responder
+    def cron_manager_start(event)
+      cron_jobs = event.payload[:cron_jobs]
+      cron_jobs_count = cron_jobs.size
+
+      info do
+        "GoodJob started cron with #{cron_jobs_count} #{'jobs'.pluralize(cron_jobs_count)}."
+      end
+    end
+
     # @!macro notification_responder
     def scheduler_shutdown_start(event)
       process_id = event.payload[:process_id]

data/lib/good_job/railtie.rb

@@ -3,6 +3,7 @@ module GoodJob
   # Ruby on Rails integration.
   class Railtie < ::Rails::Railtie
     config.good_job = ActiveSupport::OrderedOptions.new
+    config.good_job.cron = {}
 
     initializer "good_job.logger" do |_app|
       ActiveSupport.on_load(:good_job) do
@@ -23,6 +24,7 @@ module GoodJob
 
     config.after_initialize do
       GoodJob::Scheduler.instances.each(&:warm_cache)
+      GoodJob::CronManager.instances.each(&:start)
     end
   end
 end

data/lib/good_job/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 module GoodJob
   # GoodJob gem version.
-  VERSION = '1.11.3'
+  VERSION = '1.13.0'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 1.11.3
+  version: 1.13.0
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2021-07-25 00:00:00.000000000 Z
+date: 2021-08-18 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -52,6 +52,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 1.0.2
+- !ruby/object:Gem::Dependency
+  name: fugit
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.1'
 - !ruby/object:Gem::Dependency
   name: railties
   requirement: !ruby/object:Gem::Requirement
@@ -345,12 +359,12 @@ files:
 - engine/app/helpers/good_job/application_helper.rb
 - engine/app/views/good_job/active_jobs/show.html.erb
 - engine/app/views/good_job/dashboards/index.html.erb
+- engine/app/views/good_job/shared/_chart.erb
+- engine/app/views/good_job/shared/_jobs_table.erb
+- engine/app/views/good_job/shared/icons/_check.html.erb
+- engine/app/views/good_job/shared/icons/_exclamation.html.erb
+- engine/app/views/good_job/shared/icons/_trash.html.erb
 - engine/app/views/layouts/good_job/base.html.erb
-- engine/app/views/shared/_chart.erb
-- engine/app/views/shared/_jobs_table.erb
-- engine/app/views/shared/icons/_check.html.erb
-- engine/app/views/shared/icons/_exclamation.html.erb
-- engine/app/views/shared/icons/_trash.html.erb
 - engine/config/routes.rb
 - engine/lib/good_job/engine.rb
 - exe/good_job
@@ -360,6 +374,7 @@ files:
 - lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb
 - lib/generators/good_job/templates/update/migrations/02_add_active_job_id_concurrency_key_cron_key_to_good_jobs.rb
 - lib/generators/good_job/templates/update/migrations/03_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb
+- lib/generators/good_job/templates/update/migrations/04_add_retried_good_job_id_to_good_jobs.rb
 - lib/generators/good_job/update_generator.rb
 - lib/good_job.rb
 - lib/good_job/active_job_extensions.rb
@@ -367,6 +382,7 @@ files:
 - lib/good_job/adapter.rb
 - lib/good_job/cli.rb
 - lib/good_job/configuration.rb
+- lib/good_job/cron_manager.rb
 - lib/good_job/current_execution.rb
 - lib/good_job/daemon.rb
 - lib/good_job/execution_result.rb