good_job 1.11.1 → 1.12.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 91a09f77d3cb46eb91c4673732e1c52ea752f8d94f95c23c81d3457dc63aa051
-  data.tar.gz: ef21168d4aa96f3dfbf50ebed2339132bea52542209d5732d84ac23028227537
+  metadata.gz: 5b4629010561604c4f1d77812c3ca7301e5600fa8e0637fd584a769898d43aa5
+  data.tar.gz: 1216b9d590c40c047fa25c13afec94021891752bab8ec36ff7f6c968af8e285e
 SHA512:
-  metadata.gz: 8140a558ee43eeb2e3673462b7279f25cbed256bac385b131aa48c2f0ff59106d2038a39c65f3f1fd11f65cbcf7d3fb232bbfd20c790e7de629047a8eaed742b
-  data.tar.gz: 9daf8d432f5a013fbe5af10176b788c6c2e8c89e0f9b5e12568cbef7a7ecddda2b9296968e1076f71e5f26ed35a2f37d987227e7ad260030807f95404c7dbdeb
+  metadata.gz: ef51328facf01deeec6a2e67c38c526c944d1c7381fa45cf97f7f05aa4897ac711c324f616e8d948c3c5e6e4e5fae636ea2c7c3c9d48d79de0ead78ed438580f
+  data.tar.gz: e4efa15858ac7a6c0849ec084a00de8fbb8f6751523e78c9234b118bee2c84f3709c549cfc1f161af5166b2d54ff1f358b2dbd18e54adca06d6c62ad10757827

data/CHANGELOG.md

@@ -1,17 +1,85 @@
 # Changelog
 
-## [v1.11.1](https://github.com/bensheldon/good_job/tree/v1.11.1) (2021-07-07)
+## [v1.12.1](https://github.com/bensheldon/good_job/tree/v1.12.1) (2021-08-05)
 
-[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.11.0...v1.11.1)
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.12.0...v1.12.1)
+
+**Fixed bugs:**
+
+- Ensure CLI can shutdown cleanly with multiple queues and timeout [\#319](https://github.com/bensheldon/good_job/pull/319) ([bensheldon](https://github.com/bensheldon))
 
 **Closed issues:**
 
-- Database connection required while loading the code on 1.10.x [\#291](https://github.com/bensheldon/good_job/issues/291)
+- Setting a shutdown timeout causes the CLI executor to throw an exception on shutdown. [\#318](https://github.com/bensheldon/good_job/issues/318)
+- PgBouncer and prepared statements [\#269](https://github.com/bensheldon/good_job/issues/269)
+- Question about locking internals [\#212](https://github.com/bensheldon/good_job/issues/212)
+- Encoding::UndefinedConversionError \("\xE2" from ASCII-8BIT to UTF-8\) [\#198](https://github.com/bensheldon/good_job/issues/198)
+- tools for managing a 'fleet' of processes [\#150](https://github.com/bensheldon/good_job/issues/150)
+
+**Merged pull requests:**
+
+- Fix Readme lint warnings [\#320](https://github.com/bensheldon/good_job/pull/320) ([bensheldon](https://github.com/bensheldon))
+
+## [v1.12.0](https://github.com/bensheldon/good_job/tree/v1.12.0) (2021-07-27)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.11.3...v1.12.0)
+
+**Implemented enhancements:**
+
+- Add the ability to schedule repeating / recurring / cron-like jobs [\#53](https://github.com/bensheldon/good_job/issues/53)
+- Add cron-like support for recurring/repeating jobs [\#297](https://github.com/bensheldon/good_job/pull/297) ([bensheldon](https://github.com/bensheldon))
 
 **Merged pull requests:**
 
+- Place Dashboard shared view partials under `good_job` namespace [\#310](https://github.com/bensheldon/good_job/pull/310) ([bensheldon](https://github.com/bensheldon))
+- Ensure Dashboard inline javascript has CSP nonce for strict Content-Security Policy [\#309](https://github.com/bensheldon/good_job/pull/309) ([bensheldon](https://github.com/bensheldon))
+
+## [v1.11.3](https://github.com/bensheldon/good_job/tree/v1.11.3) (2021-07-25)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.11.2...v1.11.3)
+
+**Closed issues:**
+
+- ERROR:  relation "good\_jobs" does not exist at character 454 [\#308](https://github.com/bensheldon/good_job/issues/308)
+- Add Frozen String Literal to all files [\#298](https://github.com/bensheldon/good_job/issues/298)
+- Support for good\_job without Rails? [\#295](https://github.com/bensheldon/good_job/issues/295)
+
+**Merged pull requests:**
+
+- Have prettier Dashboard asset urls e.g. `bootstrap.css` instead of `bootstrap_css.css` [\#306](https://github.com/bensheldon/good_job/pull/306) ([bensheldon](https://github.com/bensheldon))
+- Create dashboard demo app on Heroku [\#305](https://github.com/bensheldon/good_job/pull/305) ([bensheldon](https://github.com/bensheldon))
+- Add Frozen String Literal to all files [\#302](https://github.com/bensheldon/good_job/pull/302) ([tedhexaflow](https://github.com/tedhexaflow))
+
+## [v1.11.2](https://github.com/bensheldon/good_job/tree/v1.11.2) (2021-07-20)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.11.1...v1.11.2)
+
+**Fixed bugs:**
+
+- Notifier waits to retry listening when database is unavailable [\#301](https://github.com/bensheldon/good_job/pull/301) ([bensheldon](https://github.com/bensheldon))
+
+**Closed issues:**
+
+- Handle database connection drops [\#296](https://github.com/bensheldon/good_job/issues/296)
+- Using the `async` worker results in `ActiveModel::UnknownAttributeError  unknown attribute 'create_with_advisory_lock' for GoodJob::Job`. [\#290](https://github.com/bensheldon/good_job/issues/290)
+
+**Merged pull requests:**
+
+- Rename development and test databases to be `good_job` [\#300](https://github.com/bensheldon/good_job/pull/300) ([bensheldon](https://github.com/bensheldon))
+- Move generators spec into top-level spec directory; update dependencies [\#299](https://github.com/bensheldon/good_job/pull/299) ([bensheldon](https://github.com/bensheldon))
+
+## [v1.11.1](https://github.com/bensheldon/good_job/tree/v1.11.1) (2021-07-07)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.11.0...v1.11.1)
+
+**Fixed bugs:**
+
 - Defer accessing ActiveRecord `primary_key` in Lockable [\#293](https://github.com/bensheldon/good_job/pull/293) ([bensheldon](https://github.com/bensheldon))
 
+**Closed issues:**
+
+- Database connection required while loading the code on 1.10.x [\#291](https://github.com/bensheldon/good_job/issues/291)
+
 ## [v1.11.0](https://github.com/bensheldon/good_job/tree/v1.11.0) (2021-07-07)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.10.1...v1.11.0)
@@ -32,9 +100,12 @@
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.10.0...v1.10.1)
 
-**Merged pull requests:**
+**Fixed bugs:**
 
 - Remove `FOR UPDATE SKIP LOCKED` from job locking sql statement [\#288](https://github.com/bensheldon/good_job/pull/288) ([bensheldon](https://github.com/bensheldon))
+
+**Merged pull requests:**
+
 - Update GH Test Matrix with latest JRuby 9.2.19.0 [\#283](https://github.com/bensheldon/good_job/pull/283) ([tedhexaflow](https://github.com/tedhexaflow))
 
 ## [v1.10.0](https://github.com/bensheldon/good_job/tree/v1.10.0) (2021-06-29)
@@ -43,11 +114,11 @@
 
 **Implemented enhancements:**
 
+- Use `pg_advisory_unlock_all` after each thread's job execution; fix Lockable return values; improve test stability [\#285](https://github.com/bensheldon/good_job/pull/285) ([bensheldon](https://github.com/bensheldon))
 - Add `rails g good_job:update` command to add idempotent migration files, including `active_job_id`, `concurrency_key`, `cron_key` columns [\#266](https://github.com/bensheldon/good_job/pull/266) ([bensheldon](https://github.com/bensheldon))
 
 **Fixed bugs:**
 
-- Use `pg_advisory_unlock_all` after each thread's job execution; fix Lockable return values; improve test stability [\#285](https://github.com/bensheldon/good_job/pull/285) ([bensheldon](https://github.com/bensheldon))
 - Dashboard AssetsController does not raise if verify\_authenticity\_token is not in the callback chain [\#284](https://github.com/bensheldon/good_job/pull/284) ([bensheldon](https://github.com/bensheldon))
 
 **Closed issues:**

data/README.md

@@ -38,7 +38,8 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
     - [Configuration options](#configuration-options)
     - [Global options](#global-options)
     - [Dashboard](#dashboard)
-    - [ActiveJob Concurrency](#activejob-concurrency)
+    - [ActiveJob concurrency](#activejob-concurrency)
+    - [Cron-style repeating/recurring jobs](#cron-style-repeatingrecurring-jobs)
     - [Updating](#updating)
 - [Go deeper](#go-deeper)
     - [Exceptions, retries, and reliability](#exceptions-retries-and-reliability)
@@ -51,6 +52,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
     - [Execute jobs async / in-process](#execute-jobs-async--in-process)
     - [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
     - [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
+    - [PgBouncer compatibility](#pgbouncer-compatibility)
 - [Contribute](#contribute)
     - [Gem development](#gem-development)
     - [Release](#release)
@@ -156,6 +158,7 @@ Options:
   [--poll-interval=SECONDS]    # Interval between polls for available jobs in seconds (env var: GOOD_JOB_POLL_INTERVAL, default: 1)
   [--max-cache=COUNT]          # Maximum number of scheduled jobs to cache in memory (env var: GOOD_JOB_MAX_CACHE, default: 10000)
   [--shutdown-timeout=SECONDS] # Number of seconds to wait for jobs to finish when shutting down before stopping the thread. (env var: GOOD_JOB_SHUTDOWN_TIMEOUT, default: -1 (forever))
+  [--enable-cron]              # Whether to run cron process (default: false)
   [--daemonize]                # Run as a background daemon (default: false)
   [--pidfile=PIDFILE]          # Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)
 
@@ -212,7 +215,8 @@ config.good_job.execution_mode = :async_server
 config.good_job.max_threads = 5
 config.good_job.poll_interval = 30 # seconds
 config.good_job.shutdown_timeout = 25 # seconds
-
+config.good_job.enable_cron = true
+config.good_job.cron = { example: { cron: '0 * * * *', class: 'ExampleJob'  } }
 
 # ...or all at once.
 config.good_job = {
@@ -220,6 +224,13 @@ config.good_job = {
   max_threads: 5,
   poll_interval: 30,
   shutdown_timeout: 25,
+  enable_cron: true,
+  cron: {
+    example: {
+      cron: '0 * * * *',
+      class: 'ExampleJob'
+    },
+  },
 }
 ```
 
@@ -235,6 +246,8 @@ Available configuration options are:
 - `poll_interval` (integer) sets the number of seconds between polls for jobs when `execution_mode` is set to `:async` or `:async_server`. You can also set this with the environment variable `GOOD_JOB_POLL_INTERVAL`.
 - `max_cache` (integer) sets the maximum number of scheduled jobs that will be stored in memory to reduce execution latency when also polling for scheduled jobs. Caching 10,000 scheduled jobs uses approximately 20MB of memory. You can also set this with the environment variable `GOOD_JOB_MAX_CACHE`.
 - `shutdown_timeout` (float) number of seconds to wait for jobs to finish when shutting down before stopping the thread. Defaults to forever: `-1`. You can also set this with the environment variable `GOOD_JOB_SHUTDOWN_TIMEOUT`.
+- `enable_cron` (boolean) whether to run cron process. Defaults to `false`. You can also set this with the environment variable `GOOD_JOB_ENABLE_CRON`.
+- `cron` (hash) cron configuration. Defaults to `{}`. You can also set this as a JSON string with the environment variable `GOOD_JOB_CRON`
 
 By default, GoodJob configures the following execution modes per environment:
 
@@ -320,9 +333,9 @@ GoodJob includes a Dashboard as a mountable `Rails::Engine`.
     end
     ```
 
-### ActiveJob Concurrency
+### ActiveJob concurrency
 
-GoodJob can extend ActiveJob to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_.
+GoodJob can extend ActiveJob to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_. Limiting concurrency can help prevent duplicate, double or unecessary jobs from being enqueued, or race conditions when performing, for example when interacting with 3rd-party APIs.
 
 **Note:** Limiting concurrency at _enqueue_ requires Rails 6.0+ because Rails 5.2 does not support `throw :abort` in ActiveJob callbacks.
 
@@ -349,6 +362,45 @@ class MyJob < ApplicationJob
 end
 ```
 
+When testing, the resulting concurrency key value can be inspected:
+
+```ruby
+job = MyJob.perform_later("Alice")
+job.good_job_concurrency_key #=> "Unique-Alice"
+```
+
+### Cron-style repeating/recurring jobs
+
+GoodJob can enqueue jobs on a recurring basis that can be used as a replacement for cron.
+
+Cron-style jobs are run on every GoodJob process (e.g. CLI or `async` execution mode) when `config.good_job.enable_cron = true`; use GoodJob's [ActiveJob concurrency](#activejob-concurrency) extension to limit the number of jobs that are enqueued.
+
+Cron-format is parsed by the [`fugit`](https://github.com/floraison/fugit) gem, which has support for seconds-level resolution (e.g. `* * * * * *`).
+
+```ruby
+# config/environments/application.rb or a specific environment e.g. production.rb
+
+# Enable cron in this process; e.g. only run on the first Heroku worker process
+config.good_job.enable_cron = ENV['DYNO'] == 'worker.1' # or `true` or via $GOOD_JOB_ENABLE_CRON
+
+# Configure cron with a hash that has a unique key for each recurring job
+config.good_job.cron = {
+  # Every 15 minutes, enqueue `ExampleJob.set(priority: -10).perform_later(52, name: "Alice")`
+  frequent_task: { # each recurring job must have a unique key
+    cron: "*/15 * * * *", # cron-style scheduling format by fugit gem
+    class: "ExampleJob", # reference the Job class with a string
+    args: [42, { name: "Alice" }], # arguments to pass; can also be a proc e.g. `-> { { when: Time.now } }`
+    set: { priority: -10 }, # additional ActiveJob properties; can also be a lambda/proc e.g. `-> { { priority: [1,2].sample } }`
+    description: "Something helpful", # optional description that appears in Dashboard (coming soon!)
+  },
+  another_task: {
+    cron: "0 0,12 * * *",
+    class: "AnotherJob",
+  },
+  # etc.
+}
+```
+
 ### Updating
 
 GoodJob follows semantic versioning, though updates may be encouraged through deprecation warnings in minor versions.
@@ -636,6 +688,43 @@ It is also necessary to delete these preserved jobs from the database after a ce
     $ bundle exec good_job cleanup_preserved_jobs --before-seconds-ago=86400
     ```
 
+### PgBouncer compatibility
+
+GoodJob is not compatible with PgBouncer in _transaction_ mode, but is compatible with PgBouncer's _connection_ mode. GoodJob uses connection-based advisory locks and LISTEN/NOTIFY, both of which require full database connections.
+
+A workaround to this limitation is to make a direct database connection available to GoodJob. With Rails 6.0's support for [multiple databases](https://guides.rubyonrails.org/active_record_multiple_databases.html), a direct connection to the database can be configured:
+
+1. Define a direct connection to your database that is not proxied through PgBouncer, for example:
+
+    ```yml
+    # config/database.yml
+
+    production:
+      primary:
+        url: postgres://pgbouncer_host/my_database
+      primary_direct:
+        url: postgres://database_host/my_database
+    ```
+
+1. Create a new ActiveRecord base class that uses the direct database connection
+
+    ```ruby
+    # app/models/application_direct_record.rb
+
+    class ApplicationDirectRecord < ActiveRecord::Base
+      self.abstract_class = true
+      connects_to database: :primary_direct
+    end
+    ```
+
+1. Configure GoodJob to use the newly created ActiveRecord base class:
+
+    ```ruby
+    # config/initializers/good_job.rb
+
+    GoodJob.active_record_parent_class = "ApplicationDirectRecord"
+    ```
+
 ## Contribute
 
 Contributions are welcomed and appreciated 🙏

data/engine/app/controllers/good_job/active_jobs_controller.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   class ActiveJobsController < GoodJob::BaseController
     def show

data/engine/app/controllers/good_job/assets_controller.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   class AssetsController < ActionController::Base # rubocop:disable Rails/ApplicationController
     skip_before_action :verify_authenticity_token, raise: false

data/engine/app/controllers/good_job/base_controller.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   class BaseController < ActionController::Base # rubocop:disable Rails/ApplicationController
     protect_from_forgery with: :exception

data/engine/app/controllers/good_job/dashboards_controller.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   class DashboardsController < GoodJob::BaseController
     class JobFilter

data/engine/app/controllers/good_job/jobs_controller.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   class JobsController < GoodJob::BaseController
     def destroy

data/engine/app/helpers/good_job/application_helper.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   module ApplicationHelper
   end

data/engine/app/views/good_job/active_jobs/show.html.erb

@@ -1 +1 @@
-<%= render 'shared/jobs_table', jobs: @jobs %>
+<%= render 'good_job/shared/jobs_table', jobs: @jobs %>

data/engine/app/views/good_job/dashboards/index.html.erb

@@ -1,5 +1,5 @@
 <div class="card my-3 p-6">
-  <%= render 'shared/chart', chart_data: @chart %>
+  <%= render 'good_job/shared/chart', chart_data: @chart %>
 </div>
 
 <div class='card mb-2'>
@@ -38,7 +38,7 @@
 </div>
 
 <% if @filter.jobs.present? %>
-  <%= render 'shared/jobs_table', jobs: @filter.jobs %>
+  <%= render 'good_job/shared/jobs_table', jobs: @filter.jobs %>
 
   <nav aria-label="Job pagination" class="mt-3">
     <ul class="pagination">

data/engine/app/views/{shared → good_job/shared}/_chart.erb

@@ -1,6 +1,6 @@
 <div id="chart"></div>
 
-<script>
+<%= javascript_tag nonce: true do %>
   new Chartist.Line('#chart', <%== chart_data.to_json %>, {
     height: '300px',
     fullWidth: true,
@@ -49,4 +49,4 @@
     tooltipEl.style.left = (event.offsetX || event.originalEvent.layerX) + tooltipEl.offsetWidth + 10 + 'px';
     tooltipEl.style.top = (event.offsetY || event.originalEvent.layerY) + tooltipEl.offsetHeight - 20 + 'px';
   }, true);
-</script>
+<% end %>

data/engine/app/views/{shared → good_job/shared}/_jobs_table.erb

@@ -29,7 +29,7 @@
             </td>
             <td>
               <%= button_to job_path(job.id), method: :delete, class: "btn btn-sm btn-outline-danger", title: "Delete job" do %>
-                <%= render "shared/icons/trash" %>
+                <%= render "good_job/shared/icons/trash" %>
               <% end %>
             </td>
           </tr>

data/engine/app/views/layouts/good_job/base.html.erb

@@ -5,12 +5,12 @@
   <%= csrf_meta_tags %>
   <%= csp_meta_tag %>
 
-  <%= stylesheet_link_tag bootstrap_css_path(v: GoodJob::VERSION) %>
-  <%= stylesheet_link_tag chartist_css_path(v: GoodJob::VERSION) %>
-  <%= stylesheet_link_tag style_css_path(v: GoodJob::VERSION) %>
+  <%= stylesheet_link_tag bootstrap_path(format: :css, v: GoodJob::VERSION) %>
+  <%= stylesheet_link_tag chartist_path(format: :css, v: GoodJob::VERSION) %>
+  <%= stylesheet_link_tag style_path(format: :css, v: GoodJob::VERSION) %>
 
-  <%= javascript_include_tag bootstrap_js_path(v: GoodJob::VERSION) %>
-  <%= javascript_include_tag chartist_js_path(v: GoodJob::VERSION) %>
+  <%= javascript_include_tag bootstrap_path(format: :js, v: GoodJob::VERSION) %>
+  <%= javascript_include_tag chartist_path(format: :js, v: GoodJob::VERSION) %>
 </head>
 <body>
   <nav class="navbar navbar-expand-lg navbar-light bg-light">
@@ -53,13 +53,13 @@
 
     <% if notice %>
       <div class="alert alert-success alert-dismissible fade show d-flex align-items-center offset-md-3 col-6" role="alert">
-        <%= render "shared/icons/check", class: "flex-shrink-0 me-2" %>
+        <%= render "good_job/shared/icons/check", class: "flex-shrink-0 me-2" %>
         <div><%= notice %></div>
         <button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>
       </div>
     <% elsif alert %>
       <div class="alert alert-warning alert-dismissible fade show d-flex align-items-center offset-md-3 col-6" role="alert">
-        <%= render "shared/icons/check", class: "flex-shrink-0 me-2" %>
+        <%= render "good_job/shared/icons/check", class: "flex-shrink-0 me-2" %>
         <div><%= alert %></div>
         <button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>
       </div>

data/engine/config/routes.rb

@@ -1,13 +1,19 @@
+# frozen_string_literal: true
 GoodJob::Engine.routes.draw do
   root to: 'dashboards#index'
   resources :active_jobs, only: %i[show]
   resources :jobs, only: %i[destroy]
 
   scope controller: :assets do
-    get :bootstrap_css
-    get :bootstrap_js
-    get :chartist_css
-    get :chartist_js
-    get :style_css
+    constraints(format: :css) do
+      get :bootstrap, action: :bootstrap_css
+      get :chartist, action: :chartist_css
+      get :style, action: :style_css
+    end
+
+    constraints(format: :js) do
+      get :bootstrap, action: :bootstrap_js
+      get :chartist, action: :chartist_js
+    end
   end
 end

data/engine/lib/good_job/engine.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   class Engine < ::Rails::Engine
     isolate_namespace GoodJob

data/exe/good_job

@@ -1,4 +1,5 @@
 #!/usr/bin/env ruby
+# frozen_string_literal: true
 require 'good_job/cli'
 
 GoodJob::CLI.within_exe = true

data/lib/active_job/queue_adapters/good_job_adapter.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module ActiveJob # :nodoc:
   module QueueAdapters # :nodoc:
     # See {GoodJob::Adapter} for details.

data/lib/generators/good_job/install_generator.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'rails/generators'
 require 'rails/generators/active_record'
 module GoodJob

data/lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 class CreateGoodJobs < ActiveRecord::Migration[5.2]
   def change
     enable_extension 'pgcrypto'

data/lib/generators/good_job/templates/update/migrations/02_add_active_job_id_concurrency_key_cron_key_to_good_jobs.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 class AddActiveJobIdConcurrencyKeyCronKeyToGoodJobs < ActiveRecord::Migration[5.2]
   def change
     reversible do |dir|

data/lib/generators/good_job/templates/update/migrations/03_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 class AddActiveJobIdIndexAndConcurrencyKeyIndexToGoodJobs < ActiveRecord::Migration[5.2]
   disable_ddl_transaction!
 

data/lib/generators/good_job/update_generator.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'rails/generators'
 require 'rails/generators/active_record'
 

data/lib/good_job.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require "rails"
 require "active_job"
 require "active_job/queue_adapters"
@@ -105,16 +106,13 @@ module GoodJob
                 wait ? -1 : nil
               end
 
-    executables = Array(Notifier.instances) + Array(Poller.instances) + Array(Scheduler.instances)
-    _shutdown_all(executables, timeout: timeout)
+    _shutdown_all(_executables, timeout: timeout)
   end
 
   # Tests whether jobs have stopped executing.
   # @return [Boolean] whether background threads are shut down
   def self.shutdown?
-    Notifier.instances.all?(&:shutdown?) &&
-      Poller.instances.all?(&:shutdown?) &&
-      Scheduler.instances.all?(&:shutdown?)
+    _executables.all?(&:shutdown?)
   end
 
   # Stops and restarts executing jobs.
@@ -125,17 +123,16 @@ module GoodJob
   # @param timeout [Numeric, nil] Seconds to wait for active threads to finish.
   # @return [void]
   def self.restart(timeout: -1)
-    executables = Array(Notifier.instances) + Array(Poller.instances) + Array(Scheduler.instances)
-    _shutdown_all(executables, :restart, timeout: timeout)
+    _shutdown_all(_executables, :restart, timeout: timeout)
   end
 
-  # Sends +#shutdown+ or +#restart+ to executable objects ({GoodJob::Notifier}, {GoodJob::Poller}, {GoodJob::Scheduler})
-  # @param executables [Array<Notifier, Poller, Scheduler, MultiScheduler>] Objects to shut down.
+  # Sends +#shutdown+ or +#restart+ to executable objects ({GoodJob::Notifier}, {GoodJob::Poller}, {GoodJob::Scheduler}, {GoodJob::MultiScheduler}, {GoodJob::CronManager})
+  # @param executables [Array<Notifier, Poller, Scheduler, MultiScheduler, CronManager>] Objects to shut down.
   # @param method_name [:symbol] Method to call, e.g. +:shutdown+ or +:restart+.
   # @param timeout [nil,Numeric]
   # @return [void]
   def self._shutdown_all(executables, method_name = :shutdown, timeout: -1)
-    if timeout.positive?
+    if timeout.is_a?(Numeric) && timeout.positive?
       executables.each { |executable| executable.send(method_name, timeout: nil) }
 
       stop_at = Time.current + timeout
@@ -145,5 +142,14 @@ module GoodJob
     end
   end
 
+  def self._executables
+    [].concat(
+      CronManager.instances,
+      Notifier.instances,
+      Poller.instances,
+      Scheduler.instances
+    )
+  end
+
   ActiveSupport.run_load_hooks(:good_job, self)
 end

data/lib/good_job/active_job_extensions.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   module ActiveJobExtensions
   end

data/lib/good_job/active_job_extensions/concurrency.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   module ActiveJobExtensions
     module Concurrency

data/lib/good_job/adapter.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   #
   # ActiveJob Adapter.
@@ -56,6 +57,8 @@ module GoodJob
         @scheduler = GoodJob::Scheduler.from_configuration(@configuration, warm_cache_on_initialize: Rails.application.initialized?)
         @notifier.recipients << [@scheduler, :create_thread]
         @poller.recipients << [@scheduler, :create_thread]
+
+        @cron_manager = GoodJob::CronManager.new(@configuration.cron, start_on_initialize: Rails.application.initialized?) if @configuration.enable_cron?
       end
     end
 

data/lib/good_job/cli.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'thor'
 
 module GoodJob
@@ -69,12 +70,16 @@ module GoodJob
                   type: :numeric,
                   banner: 'SECONDS',
                   desc: "Number of seconds to wait for jobs to finish when shutting down before stopping the thread. (env var: GOOD_JOB_SHUTDOWN_TIMEOUT, default: -1 (forever))"
+    method_option :enable_cron,
+                  type: :boolean,
+                  desc: "Whether to run cron process (default: false)"
     method_option :daemonize,
                   type: :boolean,
                   desc: "Run as a background daemon (default: false)"
     method_option :pidfile,
                   type: :string,
                   desc: "Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)"
+
     def start
       set_up_application!
       configuration = GoodJob::Configuration.new(options)
@@ -86,7 +91,7 @@ module GoodJob
       scheduler = GoodJob::Scheduler.from_configuration(configuration, warm_cache_on_initialize: true)
       notifier.recipients << [scheduler, :create_thread]
       poller.recipients << [scheduler, :create_thread]
-
+      cron_manager = GoodJob::CronManager.new(configuration.cron, start_on_initialize: true) if configuration.enable_cron?
       @stop_good_job_executable = false
       %w[INT TERM].each do |signal|
         trap(signal) { @stop_good_job_executable = true }
@@ -97,7 +102,7 @@ module GoodJob
         break if @stop_good_job_executable || scheduler.shutdown? || notifier.shutdown?
       end
 
-      executors = [notifier, poller, scheduler]
+      executors = [notifier, poller, cron_manager, scheduler].compact
       GoodJob._shutdown_all(executors, timeout: configuration.shutdown_timeout)
     end
 
@@ -123,6 +128,7 @@ module GoodJob
                   type: :numeric,
                   banner: 'SECONDS',
                   desc: "Delete records finished more than this many seconds ago (env var: GOOD_JOB_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO, default: 86400)"
+
     def cleanup_preserved_jobs
       set_up_application!
 

data/lib/good_job/configuration.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   #
   # +GoodJob::Configuration+ provides normalized configuration information to
@@ -17,6 +18,8 @@ module GoodJob
     DEFAULT_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO = 24 * 60 * 60
     # Default to always wait for jobs to finish for {Adapter#shutdown}
     DEFAULT_SHUTDOWN_TIMEOUT = -1
+    # Default to not running cron
+    DEFAULT_ENABLE_CRON = false
 
     # The options that were explicitly set when initializing +Configuration+.
     # @return [Hash]
@@ -128,6 +131,28 @@ module GoodJob
       ).to_f
     end
 
+    # Whether to run cron
+    # @return [Boolean]
+    def enable_cron
+      value = ActiveModel::Type::Boolean.new.cast(
+        options[:enable_cron] ||
+        rails_config[:enable_cron] ||
+        env['GOOD_JOB_ENABLE_CRON'] ||
+        false
+      )
+      value && cron.size.positive?
+    end
+    alias enable_cron? enable_cron
+
+    def cron
+      env_cron = JSON.parse(ENV['GOOD_JOB_CRON']) if ENV['GOOD_JOB_CRON'].present?
+
+      options[:cron] ||
+        rails_config[:cron] ||
+        env_cron ||
+        {}
+    end
+
     # Number of seconds to preserve jobs when using the +good_job cleanup_preserved_jobs+ CLI command.
     # This configuration is only used when {GoodJob.preserve_job_records} is +true+.
     # @return [Integer]

data/lib/good_job/cron_manager.rb

@@ -0,0 +1,115 @@
+# frozen_string_literal: true
+require "concurrent/hash"
+require "concurrent/scheduled_task"
+require "fugit"
+
+module GoodJob # :nodoc:
+  #
+  # CronManagers enqueue jobs on a repeating schedule.
+  #
+  class CronManager
+    # @!attribute [r] instances
+    #   @!scope class
+    #   List of all instantiated CronManagers in the current process.
+    #   @return [Array<GoodJob::CronManagers>, nil]
+    cattr_reader :instances, default: [], instance_reader: false
+
+    # Task observer for cron task
+    # @param time [Time]
+    # @param output [Object]
+    # @param thread_error [Exception]
+    def self.task_observer(time, output, thread_error) # rubocop:disable Lint/UnusedMethodArgument
+      return if thread_error.is_a? Concurrent::CancelledOperationError
+
+      GoodJob.on_thread_error.call(thread_error) if thread_error && GoodJob.on_thread_error.respond_to?(:call)
+    end
+
+    # Job configuration to be scheduled
+    # @return [Hash]
+    attr_reader :schedules
+
+    # @param schedules [Hash]
+    # @param start_on_initialize [Boolean]
+    def initialize(schedules = {}, start_on_initialize: false)
+      @running = false
+      @schedules = schedules
+      @tasks = Concurrent::Hash.new
+
+      self.class.instances << self
+
+      start if start_on_initialize
+    end
+
+    # Schedule tasks that will enqueue jobs based on their schedule
+    def start
+      ActiveSupport::Notifications.instrument("cron_manager_start.good_job", cron_jobs: @schedules) do
+        @running = true
+        schedules.each_key { |cron_key| create_task(cron_key) }
+      end
+    end
+
+    # Stop/cancel any scheduled tasks
+    # @param timeout [Numeric, nil] Unused but retained for compatibility
+    def shutdown(timeout: nil) # rubocop:disable Lint/UnusedMethodArgument
+      @running = false
+      @tasks.each do |_cron_key, task|
+        task.cancel
+      end
+      @tasks.clear
+    end
+
+    # Stop and restart
+    # @param timeout [Numeric, nil] Unused but retained for compatibility
+    def restart(timeout: nil) # rubocop:disable Lint/UnusedMethodArgument
+      shutdown
+      start
+    end
+
+    # Tests whether the manager is running.
+    # @return [Boolean, nil]
+    def running?
+      @running
+    end
+
+    # Tests whether the manager is shutdown.
+    # @return [Boolean, nil]
+    def shutdown?
+      !running?
+    end
+
+    # Enqueues a scheduled task
+    # @param cron_key [Symbol, String] the key within the schedule to use
+    def create_task(cron_key)
+      schedule = @schedules[cron_key]
+      return false if schedule.blank?
+
+      fugit = Fugit::Cron.parse(schedule.fetch(:cron))
+      delay = [(fugit.next_time - Time.current).to_f, 0].max
+
+      future = Concurrent::ScheduledTask.new(delay, args: [self, cron_key]) do |thr_scheduler, thr_cron_key|
+        # Re-schedule the next cron task before executing the current task
+        thr_scheduler.create_task(thr_cron_key)
+
+        CurrentExecution.reset
+        CurrentExecution.cron_key = thr_cron_key
+
+        Rails.application.executor.wrap do
+          schedule = thr_scheduler.schedules.fetch(thr_cron_key).with_indifferent_access
+          job_class = schedule.fetch(:class).constantize
+
+          job_set_value = schedule.fetch(:set, {})
+          job_set = job_set_value.respond_to?(:call) ? job_set_value.call : job_set_value
+
+          job_args_value = schedule.fetch(:args, [])
+          job_args = job_args_value.respond_to?(:call) ? job_args_value.call : job_args_value
+
+          job_class.set(job_set).perform_later(*job_args)
+        end
+      end
+
+      @tasks[cron_key] = future
+      future.add_observer(self.class, :task_observer)
+      future.execute
+    end
+  end
+end

data/lib/good_job/current_execution.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'active_support/core_ext/module/attribute_accessors_per_thread'
 
 module GoodJob
@@ -10,6 +11,12 @@ module GoodJob
     #   @return [String, nil]
     thread_mattr_accessor :active_job_id
 
+    # @!attribute [rw] cron_key
+    #   @!scope class
+    #   Cron Key
+    #   @return [String, nil]
+    thread_mattr_accessor :cron_key
+
     # @!attribute [rw] error_on_discard
     #   @!scope class
     #   Error captured by discard_on

data/lib/good_job/daemon.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   #
   # Manages daemonization of the current process.

data/lib/good_job/execution_result.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   # Stores the results of job execution
   class ExecutionResult

data/lib/good_job/job.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   # ActiveRecord model that represents an +ActiveJob+ job.
   # Parent class can be configured with +GoodJob.active_record_parent_class+.
@@ -10,11 +11,12 @@ module GoodJob
     PreviouslyPerformedError = Class.new(StandardError)
 
     # ActiveJob jobs without a +queue_name+ attribute are placed on this queue.
-    DEFAULT_QUEUE_NAME = 'default'.freeze
+    DEFAULT_QUEUE_NAME = 'default'
     # ActiveJob jobs without a +priority+ attribute are given this priority.
     DEFAULT_PRIORITY = 0
 
-    self.table_name = 'good_jobs'.freeze
+    self.table_name = 'good_jobs'
+    self.advisory_lockable_column = 'id'
 
     attr_readonly :serialized_params
 
@@ -197,6 +199,7 @@ module GoodJob
     def self.enqueue(active_job, scheduled_at: nil, create_with_advisory_lock: false)
       ActiveSupport::Notifications.instrument("enqueue_job.good_job", { active_job: active_job, scheduled_at: scheduled_at, create_with_advisory_lock: create_with_advisory_lock }) do |instrument_payload|
         good_job_args = {
+          cron_key: CurrentExecution.cron_key,
           queue_name: active_job.queue_name.presence || DEFAULT_QUEUE_NAME,
           priority: active_job.priority || DEFAULT_PRIORITY,
           serialized_params: active_job.serialize,
@@ -283,6 +286,25 @@ module GoodJob
       super || serialized_params['job_id']
     end
 
+    def cron_key
+      if self.class.column_names.include?('cron_key')
+        super
+      else
+        ActiveSupport::Deprecation.warn(<<~DEPRECATION)
+          GoodJob has pending database migrations. To create the migration files, run:
+
+              rails generate good_job:update
+
+          To apply the migration files, run:
+
+              rails db:migrate
+
+        DEPRECATION
+
+        nil
+      end
+    end
+
     private
 
     # @return [ExecutionResult]
@@ -293,6 +315,7 @@ module GoodJob
 
       GoodJob::CurrentExecution.reset
       GoodJob::CurrentExecution.active_job_id = active_job_id
+      GoodJob::CurrentExecution.cron_key = cron_key
       ActiveSupport::Notifications.instrument("perform_job.good_job", { good_job: self, process_id: GoodJob::CurrentExecution.process_id, thread_name: GoodJob::CurrentExecution.thread_name }) do
         value = ActiveJob::Base.execute(params)
 

data/lib/good_job/job_performer.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'concurrent/delay'
 
 module GoodJob

data/lib/good_job/lockable.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   #
   # Adds Postgres advisory locking capabilities to an ActiveRecord record.

data/lib/good_job/log_subscriber.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   #
   # Listens to GoodJob notifications and logs them.
@@ -56,6 +57,16 @@ module GoodJob
       end
     end
 
+    # @!macro notification_responder
+    def cron_manager_start(event)
+      cron_jobs = event.payload[:cron_jobs]
+      cron_jobs_count = cron_jobs.size
+
+      info do
+        "GoodJob started cron with #{cron_jobs_count} #{'jobs'.pluralize(cron_jobs_count)}."
+      end
+    end
+
     # @!macro notification_responder
     def scheduler_shutdown_start(event)
       process_id = event.payload[:process_id]

data/lib/good_job/multi_scheduler.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module GoodJob
   # Delegates the interface of a single {Scheduler} to multiple Schedulers.
   class MultiScheduler

data/lib/good_job/notifier.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'concurrent/atomic/atomic_boolean'
 
 module GoodJob # :nodoc:
@@ -13,7 +14,7 @@ module GoodJob # :nodoc:
     AdapterCannotListenError = Class.new(StandardError)
 
     # Default Postgres channel for LISTEN/NOTIFY
-    CHANNEL = 'good_job'.freeze
+    CHANNEL = 'good_job'
     # Defaults for instance of Concurrent::ThreadPoolExecutor
     EXECUTOR_OPTIONS = {
       name: name,
@@ -24,6 +25,8 @@ module GoodJob # :nodoc:
       max_queue: 1,
       fallback_policy: :discard,
     }.freeze
+    # Seconds to wait if database cannot be connected to
+    RECONNECT_INTERVAL = 5
     # Seconds to block while LISTENing for a message
     WAIT_INTERVAL = 1
 
@@ -114,7 +117,13 @@ module GoodJob # :nodoc:
         ActiveSupport::Notifications.instrument("notifier_notify_error.good_job", { error: thread_error })
       end
 
-      listen unless shutdown?
+      return if shutdown?
+
+      if thread_error.is_a?(ActiveRecord::ConnectionNotEstablished) || thread_error.is_a?(ActiveRecord::StatementInvalid)
+        listen(delay: RECONNECT_INTERVAL)
+      else
+        listen
+      end
     end
 
     private
@@ -125,8 +134,8 @@ module GoodJob # :nodoc:
       @executor = Concurrent::ThreadPoolExecutor.new(EXECUTOR_OPTIONS)
     end
 
-    def listen
-      future = Concurrent::Future.new(args: [@recipients, executor, @listening], executor: @executor) do |thr_recipients, thr_executor, thr_listening|
+    def listen(delay: 0)
+      future = Concurrent::ScheduledTask.new(delay, args: [@recipients, executor, @listening], executor: @executor) do |thr_recipients, thr_executor, thr_listening|
         with_listen_connection do |conn|
           ActiveSupport::Notifications.instrument("notifier_listen.good_job") do
             conn.async_exec("LISTEN #{CHANNEL}").clear

data/lib/good_job/poller.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'concurrent/atomic/atomic_boolean'
 
 module GoodJob # :nodoc:

data/lib/good_job/railtie.rb

@@ -1,7 +1,9 @@
+# frozen_string_literal: true
 module GoodJob
   # Ruby on Rails integration.
   class Railtie < ::Rails::Railtie
     config.good_job = ActiveSupport::OrderedOptions.new
+    config.good_job.cron = {}
 
     initializer "good_job.logger" do |_app|
       ActiveSupport.on_load(:good_job) do
@@ -22,6 +24,7 @@ module GoodJob
 
     config.after_initialize do
       GoodJob::Scheduler.instances.each(&:warm_cache)
+      GoodJob::CronManager.instances.each(&:start)
     end
   end
 end

data/lib/good_job/scheduler.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require "concurrent/executor/thread_pool_executor"
 require "concurrent/executor/timer_set"
 require "concurrent/scheduled_task"

data/lib/good_job/version.rb

@@ -1,4 +1,5 @@
+# frozen_string_literal: true
 module GoodJob
   # GoodJob gem version.
-  VERSION = '1.11.1'.freeze
+  VERSION = '1.12.1'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 1.11.1
+  version: 1.12.1
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2021-07-07 00:00:00.000000000 Z
+date: 2021-08-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -52,6 +52,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 1.0.2
+- !ruby/object:Gem::Dependency
+  name: fugit
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.1'
 - !ruby/object:Gem::Dependency
   name: railties
   requirement: !ruby/object:Gem::Requirement
@@ -345,12 +359,12 @@ files:
 - engine/app/helpers/good_job/application_helper.rb
 - engine/app/views/good_job/active_jobs/show.html.erb
 - engine/app/views/good_job/dashboards/index.html.erb
+- engine/app/views/good_job/shared/_chart.erb
+- engine/app/views/good_job/shared/_jobs_table.erb
+- engine/app/views/good_job/shared/icons/_check.html.erb
+- engine/app/views/good_job/shared/icons/_exclamation.html.erb
+- engine/app/views/good_job/shared/icons/_trash.html.erb
 - engine/app/views/layouts/good_job/base.html.erb
-- engine/app/views/shared/_chart.erb
-- engine/app/views/shared/_jobs_table.erb
-- engine/app/views/shared/icons/_check.html.erb
-- engine/app/views/shared/icons/_exclamation.html.erb
-- engine/app/views/shared/icons/_trash.html.erb
 - engine/config/routes.rb
 - engine/lib/good_job/engine.rb
 - exe/good_job
@@ -367,6 +381,7 @@ files:
 - lib/good_job/adapter.rb
 - lib/good_job/cli.rb
 - lib/good_job/configuration.rb
+- lib/good_job/cron_manager.rb
 - lib/good_job/current_execution.rb
 - lib/good_job/daemon.rb
 - lib/good_job/execution_result.rb