good_job 3.5.1 → 3.6.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1d98ed0a03595d8d2a70f2a811296cf01d1b0a4d9ff480d0a480fb91591ecafd
-  data.tar.gz: 90307d30a72de7f7af5d33a7d64ec9eaa67bb42fc8e8179e48c96f22a0440b5e
+  metadata.gz: 3cdc97553bd567b0286d79c9a41187cf6a91478855f3469af986453a6c311f72
+  data.tar.gz: ef18533f656f9d1b2a4f131e5e745760ea8976a94f1c7f46a195c0042c9a35ff
 SHA512:
-  metadata.gz: 710500b2704938b319863efe46d462296afa19f11177fb7e03f64e3cfa32872b1d1c8e793a58e6561e0f15a36125b4c3485be99edf6d1e2f8d9953da408ecb3b
-  data.tar.gz: 6d0ae65d53a40340e846cd393a3936fe6ece5b7f5e2ccfdf17ee99f628f528135af11bb2714ee223f1e800b4ced333acc7616a153774d85d9889a0e17753a162
+  metadata.gz: d411c12b7b7b16c7787942d4a5998f319b44fea819a2caf756513d8b7e9f2c6ae44b8afe41576299d3c841aabda50c0669e087cfb13c885e86225fc25192d927
+  data.tar.gz: 0df610fb8995d52e24b55984399bbe59b645eb570891fdc262faf6482d324e42c647bdd33c18d7025323f99d57f5a6bd5455be75b3ccf60b31bfdcf65b0a4137

data/CHANGELOG.md

@@ -1,17 +1,55 @@
 # Changelog
 
-## [v3.5.1](https://github.com/bensheldon/good_job/tree/v3.5.1) (2022-10-20)
+## [v3.6.1](https://github.com/bensheldon/good_job/tree/v3.6.1) (2022-11-30)
 
-[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.5.0...v3.5.1)
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.6.0...v3.6.1)
+
+**Implemented enhancements:**
+
+- add UA to i18n [\#747](https://github.com/bensheldon/good_job/pull/747) ([yshmarov](https://github.com/yshmarov))
 
 **Closed issues:**
 
-- Assert cancelled jobs [\#724](https://github.com/bensheldon/good_job/issues/724)
+- Minor Cron Concurrency Clarification [\#743](https://github.com/bensheldon/good_job/issues/743)
+- Materialized CTE performance bottleneck [\#720](https://github.com/bensheldon/good_job/issues/720)
+- Calculating database connections [\#669](https://github.com/bensheldon/good_job/issues/669)
 
 **Merged pull requests:**
 
+- Update gems, rubocop, and development ruby \(v2.7.7\) [\#752](https://github.com/bensheldon/good_job/pull/752) ([bensheldon](https://github.com/bensheldon))
+- Add chrome/chromdriver to Codespace configuration [\#746](https://github.com/bensheldon/good_job/pull/746) ([bensheldon](https://github.com/bensheldon))
+- Add GitHub Codespaces / devcontainer configuration [\#745](https://github.com/bensheldon/good_job/pull/745) ([bensheldon](https://github.com/bensheldon))
+- Bump fugit from 1.7.1 to 1.7.2 [\#742](https://github.com/bensheldon/good_job/pull/742) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bundle update and Rubocop lints [\#740](https://github.com/bensheldon/good_job/pull/740) ([bensheldon](https://github.com/bensheldon))
+- Add dependabot to update development dependencies [\#734](https://github.com/bensheldon/good_job/pull/734) ([bensheldon](https://github.com/bensheldon))
+- Fix misspellings and grammar in README [\#732](https://github.com/bensheldon/good_job/pull/732) ([aisayo](https://github.com/aisayo))
+
+## [v3.6.0](https://github.com/bensheldon/good_job/tree/v3.6.0) (2022-10-22)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.5.1...v3.6.0)
+
+**Implemented enhancements:**
+
+- Add configurable limit \(`queue_select_limit`\) when querying candidate jobs [\#727](https://github.com/bensheldon/good_job/pull/727) ([mitchellhenke](https://github.com/mitchellhenke))
+- Add index to `good_jobs` to improve querying candidate jobs [\#726](https://github.com/bensheldon/good_job/pull/726) ([mitchellhenke](https://github.com/mitchellhenke))
+
+**Fixed bugs:**
+
+- 3.4.8 release breaks job retrying [\#728](https://github.com/bensheldon/good_job/issues/728)
+- Redo: When not preserving job records, ensure all prior executions are deleted after successful retry [\#730](https://github.com/bensheldon/good_job/pull/730) ([bensheldon](https://github.com/bensheldon))
+
+## [v3.5.1](https://github.com/bensheldon/good_job/tree/v3.5.1) (2022-10-20)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.5.0...v3.5.1)
+
+**Fixed bugs:**
+
 - Revert "When not preserving job records, ensure all prior executions are deleted after successful retry" because some retry patterns stopped working [\#729](https://github.com/bensheldon/good_job/pull/729) ([bensheldon](https://github.com/bensheldon))
 
+**Closed issues:**
+
+- Assert cancelled jobs [\#724](https://github.com/bensheldon/good_job/issues/724)
+
 ## [v3.5.0](https://github.com/bensheldon/good_job/tree/v3.5.0) (2022-10-18)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.4.8...v3.5.0)
@@ -26,7 +64,7 @@
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.4.7...v3.4.8)
 
-**Merged pull requests:**
+**Fixed bugs:**
 
 - When not preserving job records, ensure all prior executions are deleted after successful retry [\#719](https://github.com/bensheldon/good_job/pull/719) ([ylansegal](https://github.com/ylansegal))
 
@@ -110,6 +148,7 @@
 **Fixed bugs:**
 
 - Jobs enqueued via dashboard ignores app default\_locale [\#697](https://github.com/bensheldon/good_job/issues/697)
+- Enqueues Cron jobs with I18n default locale [\#698](https://github.com/bensheldon/good_job/pull/698) ([esasse](https://github.com/esasse))
 - Include better exception log messages, including class and backtrace [\#693](https://github.com/bensheldon/good_job/pull/693) ([bensheldon](https://github.com/bensheldon))
 
 **Closed issues:**
@@ -119,10 +158,6 @@
 - ActiveRecord::StatementInvalid: PG::ConnectionBad: PQsocket\(\) can't get socket descriptor every 30 minutes aprox. [\#579](https://github.com/bensheldon/good_job/issues/579)
 - Handle assets in dashboard when rails app is behind proxy path [\#424](https://github.com/bensheldon/good_job/issues/424)
 
-**Merged pull requests:**
-
-- Enqueues Cron jobs with I18n default locale [\#698](https://github.com/bensheldon/good_job/pull/698) ([esasse](https://github.com/esasse))
-
 ## [v3.4.1](https://github.com/bensheldon/good_job/tree/v3.4.1) (2022-08-06)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.4.0...v3.4.1)
@@ -1008,7 +1043,7 @@
 - Warn in Readme that configuration should not go into `config/initializers/*.rb` [\#418](https://github.com/bensheldon/good_job/pull/418) ([bensheldon](https://github.com/bensheldon))
 - Replace worker wording [\#409](https://github.com/bensheldon/good_job/pull/409) ([Hugo-Hache](https://github.com/Hugo-Hache))
 - Improve Readme's "Optimize queues, threads, processes" section [\#405](https://github.com/bensheldon/good_job/pull/405) ([Hugo-Hache](https://github.com/Hugo-Hache))
-- Update GH Test Matrix with more PG versions [\#401](https://github.com/bensheldon/good_job/pull/401) ([tedhexaflow](https://github.com/tedhexaflow))
+- Update GH Test Matrix with more PG versions [\#401](https://github.com/bensheldon/good_job/pull/401) ([tedthetnaungsoe](https://github.com/tedthetnaungsoe))
 - Extract cron configuration hash into CronEntry ActiveModel objects [\#400](https://github.com/bensheldon/good_job/pull/400) ([bensheldon](https://github.com/bensheldon))
 - Remove errant copy-paste from app.json [\#397](https://github.com/bensheldon/good_job/pull/397) ([morgoth](https://github.com/morgoth))
 
@@ -1048,7 +1083,7 @@
 
 **Merged pull requests:**
 
-- Update GH Test Matrix with latest JRuby 9.3.0.0 [\#387](https://github.com/bensheldon/good_job/pull/387) ([tedhexaflow](https://github.com/tedhexaflow))
+- Update GH Test Matrix with latest JRuby 9.3.0.0 [\#387](https://github.com/bensheldon/good_job/pull/387) ([tedthetnaungsoe](https://github.com/tedthetnaungsoe))
 - Improve test support's ShellOut command's process termination and add test logs [\#385](https://github.com/bensheldon/good_job/pull/385) ([bensheldon](https://github.com/bensheldon))
 - @bensheldon Add Rails 7 alpha to Appraisal; update development dependencies [\#384](https://github.com/bensheldon/good_job/pull/384) ([bensheldon](https://github.com/bensheldon))
 
@@ -1307,7 +1342,7 @@
 
 - Have prettier Dashboard asset urls e.g. `bootstrap.css` instead of `bootstrap_css.css` [\#306](https://github.com/bensheldon/good_job/pull/306) ([bensheldon](https://github.com/bensheldon))
 - Create dashboard demo app on Heroku [\#305](https://github.com/bensheldon/good_job/pull/305) ([bensheldon](https://github.com/bensheldon))
-- Add Frozen String Literal to all files [\#302](https://github.com/bensheldon/good_job/pull/302) ([tedhexaflow](https://github.com/tedhexaflow))
+- Add Frozen String Literal to all files [\#302](https://github.com/bensheldon/good_job/pull/302) ([tedthetnaungsoe](https://github.com/tedthetnaungsoe))
 
 ## [v1.11.2](https://github.com/bensheldon/good_job/tree/v1.11.2) (2021-07-20)
 
@@ -1365,7 +1400,7 @@
 
 **Merged pull requests:**
 
-- Update GH Test Matrix with latest JRuby 9.2.19.0 [\#283](https://github.com/bensheldon/good_job/pull/283) ([tedhexaflow](https://github.com/tedhexaflow))
+- Update GH Test Matrix with latest JRuby 9.2.19.0 [\#283](https://github.com/bensheldon/good_job/pull/283) ([tedthetnaungsoe](https://github.com/tedthetnaungsoe))
 
 ## [v1.10.0](https://github.com/bensheldon/good_job/tree/v1.10.0) (2021-06-29)
 
@@ -1515,11 +1550,11 @@
 - Move executable flags from constants to accessors on GoodJob::CLI [\#234](https://github.com/bensheldon/good_job/pull/234) ([bensheldon](https://github.com/bensheldon))
 - Add custom Scheduler::TimerSet [\#232](https://github.com/bensheldon/good_job/pull/232) ([bensheldon](https://github.com/bensheldon))
 - Fix assorted constant references in YARD documentation [\#231](https://github.com/bensheldon/good_job/pull/231) ([bensheldon](https://github.com/bensheldon))
-- Update GH Test Matrix with latest JRuby 9.2.17.0 [\#228](https://github.com/bensheldon/good_job/pull/228) ([tedhexaflow](https://github.com/tedhexaflow))
+- Update GH Test Matrix with latest JRuby 9.2.17.0 [\#228](https://github.com/bensheldon/good_job/pull/228) ([tedthetnaungsoe](https://github.com/tedthetnaungsoe))
 - Update gem dependencies [\#227](https://github.com/bensheldon/good_job/pull/227) ([bensheldon](https://github.com/bensheldon))
 - Remove leftover text from Readme [\#226](https://github.com/bensheldon/good_job/pull/226) ([weh](https://github.com/weh))
 - Fix appraisal and bundler version CI conflicts [\#224](https://github.com/bensheldon/good_job/pull/224) ([bensheldon](https://github.com/bensheldon))
-- Update GH Test Matrix with latest JRuby [\#223](https://github.com/bensheldon/good_job/pull/223) ([tedhexaflow](https://github.com/tedhexaflow))
+- Update GH Test Matrix with latest JRuby [\#223](https://github.com/bensheldon/good_job/pull/223) ([tedthetnaungsoe](https://github.com/tedthetnaungsoe))
 
 ## [v1.8.0](https://github.com/bensheldon/good_job/tree/v1.8.0) (2021-03-04)
 
@@ -1602,8 +1637,8 @@
 **Merged pull requests:**
 
 - Update bundler version to 2.2.5 [\#200](https://github.com/bensheldon/good_job/pull/200) ([bensheldon](https://github.com/bensheldon))
-- Update GH Test Matrix with minimum & latest JRuby version [\#197](https://github.com/bensheldon/good_job/pull/197) ([tedhexaflow](https://github.com/tedhexaflow))
-- Fix JRuby version number [\#193](https://github.com/bensheldon/good_job/pull/193) ([tedhexaflow](https://github.com/tedhexaflow))
+- Update GH Test Matrix with minimum & latest JRuby version [\#197](https://github.com/bensheldon/good_job/pull/197) ([tedthetnaungsoe](https://github.com/tedthetnaungsoe))
+- Fix JRuby version number [\#193](https://github.com/bensheldon/good_job/pull/193) ([tedthetnaungsoe](https://github.com/tedthetnaungsoe))
 
 ## [v1.4.1](https://github.com/bensheldon/good_job/tree/v1.4.1) (2021-01-09)
 

data/README.md

@@ -57,6 +57,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
     - [Optimize queues, threads, and processes](#optimize-queues-threads-and-processes)
     - [Database connections](#database-connections)
         - [Production setup](#production-setup)
+        - [Queue performance with Queue Select Limit](#queue-performance-with-queue-select-limit)
     - [Execute jobs async / in-process](#execute-jobs-async--in-process)
     - [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
     - [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
@@ -152,7 +153,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
 
 ### Command-line options
 
-There several top-level commands available through the `good_job` command-line tool.
+There are several top-level commands available through the `good_job` command-line tool.
 
 Configuration options are available with `help`.
 
@@ -176,6 +177,7 @@ Options:
   [--daemonize]                # Run as a background daemon (default: false)
   [--pidfile=PIDFILE]          # Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)
   [--probe-port=PORT]          # Port for http health check (env var: GOOD_JOB_PROBE_PORT, default: nil)
+  [--queue-select-limit=COUNT] # The number of queued jobs to select when polling for a job to run. (env var: GOOD_JOB_QUEUE_SELECT_LIMIT, default: nil)"
 
 Executes queued jobs.
 
@@ -273,7 +275,7 @@ Available configuration options are:
 - `max_threads` (integer) sets the default number of threads per pool to use for working jobs. You can also set this with the environment variable `GOOD_JOB_MAX_THREADS`.
 - `poll_interval` (integer) sets the number of seconds between polls for jobs when `execution_mode` is set to `:async`. You can also set this with the environment variable `GOOD_JOB_POLL_INTERVAL`. A poll interval of `-1` disables polling completely.
 - `max_cache` (integer) sets the maximum number of scheduled jobs that will be stored in memory to reduce execution latency when also polling for scheduled jobs. Caching 10,000 scheduled jobs uses approximately 20MB of memory. You can also set this with the environment variable `GOOD_JOB_MAX_CACHE`.
-- `shutdown_timeout` (float) number of seconds to wait for jobs to finish when shutting down before stopping the thread. Defaults to forever: `-1`. You can also set this with the environment variable `GOOD_JOB_SHUTDOWN_TIMEOUT`.
+- `shutdown_timeout` (integer) number of seconds to wait for jobs to finish when shutting down before stopping the thread. Defaults to forever: `-1`. You can also set this with the environment variable `GOOD_JOB_SHUTDOWN_TIMEOUT`.
 - `enable_cron` (boolean) whether to run cron process. Defaults to `false`. You can also set this with the environment variable `GOOD_JOB_ENABLE_CRON`.
 - `cron` (hash) cron configuration. Defaults to `{}`. You can also set this as a JSON string with the environment variable `GOOD_JOB_CRON`
 - `cleanup_discarded_jobs` (boolean) whether to destroy discarded jobs when cleaning up preserved jobs using the `$ good_job cleanup_preserved_jobs` CLI command or calling `GoodJob.cleanup_preserved_jobs`. Defaults to `true`. Can also be set with  the environment variable `GOOD_JOB_CLEANUP_DISCARDED_JOBS`. _This configuration is only used when {GoodJob.preserve_job_records} is `true`._
@@ -284,7 +286,7 @@ Available configuration options are:
 - `logger` ([Rails Logger](https://api.rubyonrails.org/classes/ActiveSupport/Logger.html)) lets you set a custom logger for GoodJob. It should be an instance of a Rails `Logger` (Default: `Rails.logger`).
 - `preserve_job_records` (boolean) keeps job records in your database even after jobs are completed. (Default: `true`)
 - `retry_on_unhandled_error` (boolean) causes jobs to be re-queued and retried if they raise an instance of `StandardError`. Be advised this may lead to jobs being repeated infinitely ([see below for more on retries](#retries)). Instances of `Exception`, like SIGINT, will *always* be retried, regardless of this attribute’s value. (Default: `false`)
-- `on_thread_error` (proc, lambda, or callable) will be called when an Exception. It can be useful for logging errors to bug tracking services, like Sentry or Airbrake. Example:
+- `on_thread_error` (proc, lambda, or callable) will be called when there is an Exception. It can be useful for logging errors to bug tracking services, like Sentry or Airbrake. Example:
 
     ```ruby
     config.good_job.on_thread_error = -> (exception) { Sentry.capture_exception(exception) }
@@ -325,7 +327,7 @@ The following options are also configurable via accessors, but you are encourage
 - **`GoodJob.logger`** ([Rails Logger](https://api.rubyonrails.org/classes/ActiveSupport/Logger.html)) lets you set a custom logger for GoodJob. It should be an instance of a Rails `Logger`.
 - **`GoodJob.preserve_job_records`** (boolean) keeps job records in your database even after jobs are completed. (Default: `true`)
 - **`GoodJob.retry_on_unhandled_error`** (boolean) causes jobs to be re-queued and retried if they raise an instance of `StandardError`. Be advised this may lead to jobs being repeated infinitely ([see below for more on retries](#retries)). Instances of `Exception`, like SIGINT, will *always* be retried, regardless of this attribute’s value. (Default: `false`)
-- **`GoodJob.on_thread_error`** (proc, lambda, or callable) will be called when an Exception. It can be useful for logging errors to bug tracking services, like Sentry or Airbrake.
+- **`GoodJob.on_thread_error`** (proc, lambda, or callable) will be called when there is an Exception. It can be useful for logging errors to bug tracking services, like Sentry or Airbrake.
 
 ### Dashboard
 
@@ -377,7 +379,7 @@ require 'good_job/engine' # <= Add this line
 
 #### API-only Rails applications
 
-API-only Rails applications may not have all of the required Rack middleware for the GoodJob Dashboard to function. To re-add the middlware:
+API-only Rails applications may not have all of the required Rack middleware for the GoodJob Dashboard to function. To re-add the middleware:
 
 ```ruby
 # config/application.rb
@@ -398,7 +400,7 @@ The Dashboard can be set to automatically refresh by checking "Live Poll" in the
 
 ### ActiveJob concurrency
 
-GoodJob can extend ActiveJob to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_. Limiting concurrency can help prevent duplicate, double or unecessary jobs from being enqueued, or race conditions when performing, for example when interacting with 3rd-party APIs.
+GoodJob can extend ActiveJob to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_. Limiting concurrency can help prevent duplicate, double or unnecessary jobs from being enqueued, or race conditions when performing, for example when interacting with 3rd-party APIs.
 
 ```ruby
 class MyJob < ApplicationJob
@@ -458,7 +460,7 @@ GoodJob's concurrency control strategy for `perform_limit` is "optimistic retry
 
 GoodJob can enqueue jobs on a recurring basis that can be used as a replacement for cron.
 
-Cron-style jobs are run on every GoodJob process (e.g. CLI or `async` execution mode) when `config.good_job.enable_cron = true`, but GoodJob's cron uses unique indexes to ensure that only a single job is enqeued at the given time interval.
+Cron-style jobs are run on every GoodJob process (e.g. CLI or `async` execution mode) when `config.good_job.enable_cron = true`, but GoodJob's cron uses unique indexes to ensure that only a single job is enqueued at the given time interval.
 
 Cron-format is parsed by the [`fugit`](https://github.com/floraison/fugit) gem, which has support for seconds-level resolution (e.g. `* * * * * *`) and natural language parsing (e.g. `every second`).
 
@@ -540,7 +542,7 @@ GoodJob v2 introduces a new Advisory Lock key format that is operationally diffe
 
 1. Upgrade your production environment to `v1.99.x` following the minor version upgrade process, including database migrations. `v1.99` is a transitional release that is safely compatible with both `v1.x` and `v2.0.0` because it uses both `v1`- and `v2`-formatted advisory locks.
 1. Address any deprecation warnings generated by `v1.99`.
-1. Upgrade your production environment to `v1.99.x` to `v2.0.x` again following the _minor_ upgrade process.
+1. Upgrade your production environment from `v1.99.x` to `v2.0.x` again following the _minor_ upgrade process.
 
 Notable changes:
 
@@ -746,17 +748,56 @@ The queue process will not crash if the connections pool is exhausted, instead i
 
 When running GoodJob in a production environment, you should be mindful of:
 
-- [Execution mode](execute-jobs-async--in-process)
+- [Execution mode](#execute-jobs-async--in-process)
 - [Database connection pool size](#database-connections)
 - [Health check probes](#cli-http-health-check-probes) and potentially the [instrumentation support](#monitor-and-preserve-worked-jobs)
 
 The recommended way to monitor the queue in production is:
 
-- have an exception notifier callback (see `on_thread_error`)
-- if possible, run the queue as a dedicated instance and use available HTTP health check probes instead of pid-based monitoring
+- have an exception notifier callback (see [`on_thread_error`](#configuration-options))
+- if possible, run the queue as a dedicated instance and use available HTTP health check probes instead of PID-based monitoring
 - keep an eye on the number of jobs in the queue (abnormal high number of unscheduled jobs means the queue could be underperforming)
 - consider performance monitoring services which support the built-in Rails instrumentation (eg. Sentry, Skylight, etc.)
 
+#### Queue performance with Queue Select Limit
+
+GoodJob’s advisory locking strategy uses a materialized CTE (Common Table Expression). This strategy can be non-performant when querying a very large queue of executable jobs (100,000+) because the database query must materialize all executable jobs before acquiring an advisory lock.
+
+GoodJob offers an optional optimization to limit the number of jobs that are queried: Queue Select Limit.
+
+```none
+# CLI option
+--queue-select-limit=1000
+
+# Rails configuration
+config.good_job.queue_select_limit = 1000
+
+# Environment Variable
+GOOD_JOB_QUEUE_SELECT_LIMIT=1000
+```
+
+The Queue Select Limit value should be set to a rough upper-bound that exceeds all GoodJob execution threads / database connections. `1000` is a number that likely exceeds the available database connections on most PaaS offerings, but still offers a performance boost for GoodJob when executing very large queues.
+
+To explain where this value is used, here is the pseudo-query that GoodJob uses to find executable jobs:
+
+```sql
+  SELECT *
+  FROM good_jobs
+  WHERE id IN (
+    WITH rows AS MATERIALIZED (
+      SELECT id, active_job_id
+      FROM good_jobs
+      WHERE (scheduled_at <= NOW() OR scheduled_at IS NULL) AND finished_at IS NULL
+      ORDER BY priority DESC NULLS LAST, created_at ASC
+      [LIMIT 1000] -- <= introduced when queue_select_limit is set
+    )
+    SELECT id
+    FROM rows
+    WHERE pg_try_advisory_lock(('x' || substr(md5('good_jobs' || '-' || active_job_id::text), 1, 16))::bit(64)::bigint)
+    LIMIT 1
+  )
+```
+
 ### Execute jobs async / in-process
 
 GoodJob can execute jobs "async" in the same process as the web server (e.g. `bin/rails s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:
@@ -912,7 +953,7 @@ It is also possible to manually trigger a cleanup of preserved job records:
 
 ### Write tests
 
-By default, GoodJob uses its inline adapter in the test environment; the inline adapter is designed for the test environment. When enquing a job with GoodJob's inline adapter, the job will be executed immediately on the current thread; unhandled exceptions will be raised.
+By default, GoodJob uses its inline adapter in the test environment; the inline adapter is designed for the test environment. When enqueuing a job with GoodJob's inline adapter, the job will be executed immediately on the current thread; unhandled exceptions will be raised.
 
 In GoodJob 2.0, the inline adapter will execute future scheduled jobs immediately. In the next major release, GoodJob 3.0, the inline adapter will not execute future scheduled jobs and instead enqueue them in the database.
 
@@ -1026,7 +1067,7 @@ spec:
 All contributions, from feedback to code and beyond, are welcomed and appreciated 🙏
 
 - Review the [Prioritized Project Backlog](https://github.com/bensheldon/good_job/projects/1).
-- Open a new Issue or contribute to an [existing Issue](https://github.com/bensheldon/good_job/issues). Questions or suggestions are fantastic.
+- Open a new issue or contribute to an [existing Issue](https://github.com/bensheldon/good_job/issues). Questions or suggestions are fantastic.
 - Participate according to our [Code of Conduct](/CODE_OF_CONDUCT.md).
 - Financially support the project via [Sponsorship](https://github.com/sponsors/bensheldon).
 

data/app/models/good_job/execution.rb

@@ -66,6 +66,7 @@ module GoodJob
     end
 
     belongs_to :job, class_name: 'GoodJob::Job', foreign_key: 'active_job_id', primary_key: 'active_job_id', optional: true, inverse_of: :executions
+    after_destroy -> { self.class.active_job_id(active_job_id).delete_all }, if: -> { @_destroy_job }
 
     # Get executions with given ActiveJob ID
     # @!method active_job_id
@@ -203,10 +204,10 @@ module GoodJob
     #   return value for the job's +#perform+ method, and the exception the job
     #   raised, if any (if the job raised, then the second array entry will be
     #   +nil+). If there were no jobs to execute, returns +nil+.
-    def self.perform_with_advisory_lock(parsed_queues: nil)
+    def self.perform_with_advisory_lock(parsed_queues: nil, queue_select_limit: nil)
       execution = nil
       result = nil
-      unfinished.dequeueing_ordered(parsed_queues).only_scheduled.limit(1).with_advisory_lock(unlock_session: true) do |executions|
+      unfinished.dequeueing_ordered(parsed_queues).only_scheduled.limit(1).with_advisory_lock(unlock_session: true, select_limit: queue_select_limit) do |executions|
         execution = executions.first
         break if execution.blank?
         break :unlocked unless execution&.executable?
@@ -296,13 +297,14 @@ module GoodJob
       job_error = result.handled_error || result.unhandled_error
       self.error = [job_error.class, ERROR_MESSAGE_SEPARATOR, job_error.message].join if job_error
 
+      reenqueued = result.retried? || retried_good_job_id.present?
       if result.unhandled_error && GoodJob.retry_on_unhandled_error
         save!
-      elsif GoodJob.preserve_job_records == true || (result.unhandled_error && GoodJob.preserve_job_records == :on_unhandled_error)
+      elsif GoodJob.preserve_job_records == true || reenqueued || (result.unhandled_error && GoodJob.preserve_job_records == :on_unhandled_error)
         self.finished_at = Time.current
         save!
       else
-        destroy!
+        destroy_job
       end
 
       result
@@ -354,6 +356,14 @@ module GoodJob
       (finished_at || Time.zone.now) - performed_at if performed_at
     end
 
+    # Destroys this execution and all executions within the same job
+    def destroy_job
+      @_destroy_job = true
+      destroy!
+    ensure
+      @_destroy_job = false
+    end
+
     private
 
     def active_job_data
@@ -379,7 +389,7 @@ module GoodJob
           end
           handled_error ||= current_thread.error_on_retry || current_thread.error_on_discard
 
-          ExecutionResult.new(value: value, handled_error: handled_error)
+          ExecutionResult.new(value: value, handled_error: handled_error, retried: current_thread.error_on_retry.present?)
         rescue StandardError => e
           ExecutionResult.new(value: nil, unhandled_error: e)
         end

data/app/models/good_job/execution_result.rb

@@ -8,14 +8,18 @@ module GoodJob
     attr_reader :handled_error
     # @return [Exception, nil]
     attr_reader :unhandled_error
+    # @return [Exception, nil]
+    attr_reader :retried
+    alias retried? retried
 
     # @param value [Object, nil]
     # @param handled_error [Exception, nil]
     # @param unhandled_error [Exception, nil]
-    def initialize(value:, handled_error: nil, unhandled_error: nil)
+    def initialize(value:, handled_error: nil, unhandled_error: nil, retried: false)
       @value = value
       @handled_error = handled_error
       @unhandled_error = unhandled_error
+      @retried = retried
     end
   end
 end

data/app/models/good_job/lockable.rb

@@ -37,11 +37,12 @@ module GoodJob
       # @param function [String, Symbol]  Postgres Advisory Lock function name to use
       # @return [ActiveRecord::Relation]
       #   A relation selecting only the records that were locked.
-      scope :advisory_lock, (lambda do |column: _advisory_lockable_column, function: advisory_lockable_function|
+      scope :advisory_lock, (lambda do |column: _advisory_lockable_column, function: advisory_lockable_function, select_limit: nil|
         original_query = self
 
         cte_table = Arel::Table.new(:rows)
         cte_query = original_query.select(primary_key, column).except(:limit)
+        cte_query = cte_query.limit(select_limit) if select_limit
         cte_type = if supports_cte_materialization_specifiers?
                      'MATERIALIZED'
                    else
@@ -154,10 +155,10 @@ module GoodJob
       #   MyLockableRecord.order(created_at: :asc).limit(2).with_advisory_lock do |record|
       #     do_something_with record
       #   end
-      def with_advisory_lock(column: _advisory_lockable_column, function: advisory_lockable_function, unlock_session: false)
+      def with_advisory_lock(column: _advisory_lockable_column, function: advisory_lockable_function, unlock_session: false, select_limit: nil)
         raise ArgumentError, "Must provide a block" unless block_given?
 
-        records = advisory_lock(column: column, function: function).to_a
+        records = advisory_lock(column: column, function: function, select_limit: select_limit).to_a
 
         begin
           unscoped { yield(records) }

data/config/locales/ua.yml

@@ -0,0 +1,107 @@
+---
+ua:
+  datetime:
+    distance_in_words:
+      about_x_hours:
+        few: близько %{count} години
+        many: близько %{count} годин
+        one: близько 1 години
+        other: близько %{count} години
+      about_x_months:
+        few: близько %{count} місяців
+        many: близько %{count} місяців
+        one: близько 1 місяцю
+        other: близько %{count} місяцю
+      about_x_years:
+        few: близько %{count} років
+        many: близько %{count} років
+        one: близько 1 року
+        other: близько %{count} років
+      almost_x_years:
+        few: майже %{count} роки
+        many: майже %{count} років
+        one: майже 1 рік
+        other: майже %{count} років
+      half_a_minute: пів хвилини
+      less_than_x_minutes:
+        few: менше %{count} хвилин
+        many: менше %{count} хвилин
+        one: менше 1 хвилини
+        other: менше %{count} хвилини
+      less_than_x_seconds:
+        few: менше %{count} секунд
+        many: менше %{count} секунд
+        one: менше 1 секунди
+        other: менше %{count} секунди
+      over_x_years:
+        few: більше %{count} років
+        many: більше %{count} років
+        one: більше 1 року
+        other: більше %{count} років
+      x_days:
+        few: "%{count} дня"
+        many: "%{count} днів"
+        one: 1 день
+        other: "%{count} дня"
+      x_minutes:
+        few: "%{count} хвилини"
+        many: "%{count} хвилин"
+        one: 1 хвилину
+        other: "%{count} хвилини"
+      x_months:
+        few: "%{count} місяцю"
+        many: "%{count} місяців"
+        one: 1 місяць
+        other: "%{count} місяцю"
+      x_seconds:
+        few: "%{count} секунди"
+        many: "%{count} секунд"
+        one: 1 секунду
+        other: "%{count} секунди"
+      x_years:
+        few: "%{count} року"
+        many: "%{count} років"
+        one: 1 рік
+        other: "%{count} року"
+  duration:
+    hours: "%{hour}h %{min}m"
+    less_than_10_seconds: "%{sec}s"
+    milliseconds: "%{ms}мс"
+    minutes: "%{min}м %{sec}с"
+    seconds: "%{sec}s"
+  good_job:
+    shared:
+      footer:
+        last_update_html: Оснаннє оновлення <time id="page-updated-at" datetime="%{time}">%{time}</time>
+        wording: Запам'ятай, ти теж робиш Good Job!
+      navbar:
+        cron_schedules: Cron
+        jobs: Задачі
+        live_poll: Живе Опитування
+        name: "GoodJob 👍"
+        processes: Процеси
+    status:
+      discarded: Відхилено
+      queued: В черзі
+      retried: Повторна спроба
+      running: Біг
+      scheduled: за розкладом
+      succeeded: вдалося
+  number:
+    format:
+      delimiter: " "
+      separator: ","
+    human:
+      decimal_units:
+        format: "%n%u"
+        units:
+          billion: Блн
+          million: Млн
+          quadrillion: Q
+          thousand: Тис
+          trillion: Трил
+          unit: ''
+      format:
+        delimiter: " "
+        precision: 3
+        separator: ","

data/lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb

@@ -41,5 +41,7 @@ class CreateGoodJobs < ActiveRecord::Migration<%= migration_version %>
     add_index :good_jobs, [:cron_key, :cron_at], name: :index_good_jobs_on_cron_key_and_cron_at, unique: true
     add_index :good_jobs, [:active_job_id], name: :index_good_jobs_on_active_job_id
     add_index :good_jobs, [:finished_at], where: "retried_good_job_id IS NULL AND finished_at IS NOT NULL", name: :index_good_jobs_jobs_on_finished_at
+    add_index :good_jobs, [:priority, :created_at], order: { priority: "DESC NULLS LAST", created_at: :asc },
+      where: "finished_at IS NULL", name: :index_good_jobs_jobs_on_priority_created_at_when_unfinished
   end
 end

data/lib/generators/good_job/templates/update/migrations/03_create_index_good_jobs_jobs_on_priority_created_at_when_unfinished.rb.erb

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+class CreateIndexGoodJobsJobsOnPriorityCreatedAtWhenUnfinished < ActiveRecord::Migration<%= migration_version %>
+  disable_ddl_transaction!
+
+   def change
+    reversible do |dir|
+      dir.up do
+        # Ensure this incremental update migration is idempotent
+        # with monolithic install migration.
+        return if connection.index_name_exists?(:good_jobs, :index_good_jobs_jobs_on_priority_created_at_when_unfinished)
+      end
+    end
+
+    add_index :good_jobs, [:priority, :created_at], order: { priority: "DESC NULLS LAST", created_at: :asc },
+      where: "finished_at IS NULL", name: :index_good_jobs_jobs_on_priority_created_at_when_unfinished,
+      algorithm: :concurrently
+  end
+end

data/lib/good_job/cli.rb

@@ -81,7 +81,12 @@ module GoodJob
                   desc: "Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)"
     method_option :probe_port,
                   type: :numeric,
+                  banner: 'PORT',
                   desc: "Port for http health check (env var: GOOD_JOB_PROBE_PORT, default: nil)"
+    method_option :queue_select_limit,
+                  type: :numeric,
+                  banner: 'COUNT',
+                  desc: "The number of queued jobs to select when polling for a job to run. (env var: GOOD_JOB_QUEUE_SELECT_LIMIT, default: nil)"
 
     def start
       set_up_application!

data/lib/good_job/configuration.rb

@@ -208,6 +208,19 @@ module GoodJob
       cron.map { |cron_key, params| GoodJob::CronEntry.new(params.merge(key: cron_key)) }
     end
 
+    # The number of queued jobs to select when polling for a job to run.
+    # This limit is intended to avoid locking a large number of rows when selecting eligible jobs
+    # from the queue. This value should be higher than the total number of threads across all good_job
+    # processes to ensure a thread can retrieve an eligible and unlocked job.
+    # @return [Integer, nil]
+    def queue_select_limit
+      (
+        options[:queue_select_limit] ||
+        rails_config[:queue_select_limit] ||
+        env['GOOD_JOB_QUEUE_SELECT_LIMIT']
+      )&.to_i
+    end
+
     # Whether to destroy discarded jobs when cleaning up preserved jobs.
     # This configuration is only used when {GoodJob.preserve_job_records} is +true+.
     # @return [Boolean]

data/lib/good_job/job_performer.rb

@@ -24,7 +24,7 @@ module GoodJob
     # Perform the next eligible job
     # @return [Object, nil] Returns job result or +nil+ if no job was found
     def next
-      job_query.perform_with_advisory_lock(parsed_queues: parsed_queues)
+      job_query.perform_with_advisory_lock(parsed_queues: parsed_queues, queue_select_limit: GoodJob.configuration.queue_select_limit)
     end
 
     # Tests whether this performer should be used in GoodJob's current state.

data/lib/good_job/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 module GoodJob
   # GoodJob gem version.
-  VERSION = '3.5.1'
+  VERSION = '3.6.1'
 end

data/lib/good_job.rb

@@ -57,7 +57,7 @@ module GoodJob
   #   By default, GoodJob deletes job records after the job is completed successfully.
   #   If you want to preserve jobs for latter inspection, set this to +true+.
   #   If you want to preserve only jobs that finished with error for latter inspection, set this to +:on_unhandled_error+.
-  #   @return [Boolean, nil]
+  #   @return [Boolean, Symbol, nil]
   mattr_accessor :preserve_job_records, default: true
 
   # @!attribute [rw] retry_on_unhandled_error

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 3.5.1
+  version: 3.6.1
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2022-10-20 00:00:00.000000000 Z
+date: 2022-11-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -395,6 +395,7 @@ files:
 - config/locales/es.yml
 - config/locales/nl.yml
 - config/locales/ru.yml
+- config/locales/ua.yml
 - config/routes.rb
 - exe/good_job
 - lib/active_job/queue_adapters/good_job_adapter.rb
@@ -402,6 +403,7 @@ files:
 - lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb
 - lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb.erb
 - lib/generators/good_job/templates/update/migrations/02_create_good_job_settings.rb.erb
+- lib/generators/good_job/templates/update/migrations/03_create_index_good_jobs_jobs_on_priority_created_at_when_unfinished.rb.erb
 - lib/generators/good_job/update_generator.rb
 - lib/good_job.rb
 - lib/good_job/active_job_extensions/concurrency.rb