good_job 3.5.0 → 3.6.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +26 -0
- data/README.md +41 -0
- data/app/models/good_job/execution.rb +4 -3
- data/app/models/good_job/lockable.rb +4 -3
- data/lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb +2 -0
- data/lib/generators/good_job/templates/update/migrations/03_create_index_good_jobs_jobs_on_priority_created_at_when_unfinished.rb.erb +18 -0
- data/lib/good_job/cli.rb +5 -0
- data/lib/good_job/configuration.rb +13 -0
- data/lib/good_job/job_performer.rb +1 -1
- data/lib/good_job/version.rb +1 -1
- metadata +3 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: a3932cd8e0d33ab7f8a56a43c9c458f69db206291b092d9fcac16dfef11849ad
|
4
|
+
data.tar.gz: 695e46da59becc3450c0a0e33c1557784cef56794db979a648dce5d792c57651
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 0ce689ec771af8a472d964cf14739ecc1fd2bf5c903692fbb2d73aeb19870560919c42b9aff5a0ca1671068ad6d914ceaa503aab84649e281ac53c5a17f3a6b5
|
7
|
+
data.tar.gz: 8c85d21ef4dbe48400c24dffa86f7ea78a5ae827ce8057cd8ab2efc439a980c24b61ae8224e14153a0a23b5933d49d8bf7c6e6d0e269b7cdeb7f28ced476b509
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,31 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v3.6.0](https://github.com/bensheldon/good_job/tree/v3.6.0) (2022-10-22)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.5.1...v3.6.0)
|
6
|
+
|
7
|
+
**Fixed bugs:**
|
8
|
+
|
9
|
+
- 3.4.8 release breaks job retrying [\#728](https://github.com/bensheldon/good_job/issues/728)
|
10
|
+
|
11
|
+
**Merged pull requests:**
|
12
|
+
|
13
|
+
- Redo: When not preserving job records, ensure all prior executions are deleted after successful retry [\#730](https://github.com/bensheldon/good_job/pull/730) ([bensheldon](https://github.com/bensheldon))
|
14
|
+
- Add configurable limit \(`queue_select_limit`\) when querying candidate jobs [\#727](https://github.com/bensheldon/good_job/pull/727) ([mitchellhenke](https://github.com/mitchellhenke))
|
15
|
+
- Add index to `good_jobs` to improve querying candidate jobs [\#726](https://github.com/bensheldon/good_job/pull/726) ([mitchellhenke](https://github.com/mitchellhenke))
|
16
|
+
|
17
|
+
## [v3.5.1](https://github.com/bensheldon/good_job/tree/v3.5.1) (2022-10-20)
|
18
|
+
|
19
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.5.0...v3.5.1)
|
20
|
+
|
21
|
+
**Closed issues:**
|
22
|
+
|
23
|
+
- Assert cancelled jobs [\#724](https://github.com/bensheldon/good_job/issues/724)
|
24
|
+
|
25
|
+
**Merged pull requests:**
|
26
|
+
|
27
|
+
- Revert "When not preserving job records, ensure all prior executions are deleted after successful retry" because some retry patterns stopped working [\#729](https://github.com/bensheldon/good_job/pull/729) ([bensheldon](https://github.com/bensheldon))
|
28
|
+
|
3
29
|
## [v3.5.0](https://github.com/bensheldon/good_job/tree/v3.5.0) (2022-10-18)
|
4
30
|
|
5
31
|
[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.4.8...v3.5.0)
|
data/README.md
CHANGED
@@ -57,6 +57,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
|
|
57
57
|
- [Optimize queues, threads, and processes](#optimize-queues-threads-and-processes)
|
58
58
|
- [Database connections](#database-connections)
|
59
59
|
- [Production setup](#production-setup)
|
60
|
+
- [Queue performance with Queue Select Limit](#queue-performance-with-queue-select-limit)
|
60
61
|
- [Execute jobs async / in-process](#execute-jobs-async--in-process)
|
61
62
|
- [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
|
62
63
|
- [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
|
@@ -176,6 +177,7 @@ Options:
|
|
176
177
|
[--daemonize] # Run as a background daemon (default: false)
|
177
178
|
[--pidfile=PIDFILE] # Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)
|
178
179
|
[--probe-port=PORT] # Port for http health check (env var: GOOD_JOB_PROBE_PORT, default: nil)
|
180
|
+
[--queue-select-limit=COUNT] # The number of queued jobs to select when polling for a job to run. (env var: GOOD_JOB_QUEUE_SELECT_LIMIT, default: nil)"
|
179
181
|
|
180
182
|
Executes queued jobs.
|
181
183
|
|
@@ -757,6 +759,45 @@ The recommended way to monitor the queue in production is:
|
|
757
759
|
- keep an eye on the number of jobs in the queue (abnormal high number of unscheduled jobs means the queue could be underperforming)
|
758
760
|
- consider performance monitoring services which support the built-in Rails instrumentation (eg. Sentry, Skylight, etc.)
|
759
761
|
|
762
|
+
#### Queue performance with Queue Select Limit
|
763
|
+
|
764
|
+
GoodJob’s advisory locking strategy uses a materialized CTE (Common Table Expression). This strategy can be non-performant when querying a very large queue of executable jobs (100,000+) because the database query must materialize all executable jobs before acquiring an advisory lock.
|
765
|
+
|
766
|
+
GoodJob offers an optional optimization to limit the number of jobs that are queried: Queue Select Limit.
|
767
|
+
|
768
|
+
```none
|
769
|
+
# CLI option
|
770
|
+
--queue-select-limit=1000
|
771
|
+
|
772
|
+
# Rails configuration
|
773
|
+
config.good_job.queue_select_limit = 1000
|
774
|
+
|
775
|
+
# Environment Variable
|
776
|
+
GOOD_JOB_QUEUE_SELECT_LIMIT=1000
|
777
|
+
```
|
778
|
+
|
779
|
+
The Queue Select Limit value should be set to a rough upper-bound that exceeds all GoodJob execution threads / database connections. `1000` is a number that likely exceeds the available database connections on most PaaS offerings, but still offers a performance boost for GoodJob when executing very large queues.
|
780
|
+
|
781
|
+
To explain where this value is used, here is the pseudo-query that GoodJob uses to find executable jobs:
|
782
|
+
|
783
|
+
```sql
|
784
|
+
SELECT *
|
785
|
+
FROM good_jobs
|
786
|
+
WHERE id IN (
|
787
|
+
WITH rows AS MATERIALIZED (
|
788
|
+
SELECT id, active_job_id
|
789
|
+
FROM good_jobs
|
790
|
+
WHERE (scheduled_at <= NOW() OR scheduled_at IS NULL) AND finished_at IS NULL
|
791
|
+
ORDER BY priority DESC NULLS LAST, created_at ASC
|
792
|
+
[LIMIT 1000] -- <= introduced when queue_select_limit is set
|
793
|
+
)
|
794
|
+
SELECT id
|
795
|
+
FROM rows
|
796
|
+
WHERE pg_try_advisory_lock(('x' || substr(md5('good_jobs' || '-' || active_job_id::text), 1, 16))::bit(64)::bigint)
|
797
|
+
LIMIT 1
|
798
|
+
)
|
799
|
+
```
|
800
|
+
|
760
801
|
### Execute jobs async / in-process
|
761
802
|
|
762
803
|
GoodJob can execute jobs "async" in the same process as the web server (e.g. `bin/rails s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:
|
@@ -204,10 +204,10 @@ module GoodJob
|
|
204
204
|
# return value for the job's +#perform+ method, and the exception the job
|
205
205
|
# raised, if any (if the job raised, then the second array entry will be
|
206
206
|
# +nil+). If there were no jobs to execute, returns +nil+.
|
207
|
-
def self.perform_with_advisory_lock(parsed_queues: nil)
|
207
|
+
def self.perform_with_advisory_lock(parsed_queues: nil, queue_select_limit: nil)
|
208
208
|
execution = nil
|
209
209
|
result = nil
|
210
|
-
unfinished.dequeueing_ordered(parsed_queues).only_scheduled.limit(1).with_advisory_lock(unlock_session: true) do |executions|
|
210
|
+
unfinished.dequeueing_ordered(parsed_queues).only_scheduled.limit(1).with_advisory_lock(unlock_session: true, select_limit: queue_select_limit) do |executions|
|
211
211
|
execution = executions.first
|
212
212
|
break if execution.blank?
|
213
213
|
break :unlocked unless execution&.executable?
|
@@ -297,9 +297,10 @@ module GoodJob
|
|
297
297
|
job_error = result.handled_error || result.unhandled_error
|
298
298
|
self.error = [job_error.class, ERROR_MESSAGE_SEPARATOR, job_error.message].join if job_error
|
299
299
|
|
300
|
+
reenqueued = result.retried? || retried_good_job_id.present?
|
300
301
|
if result.unhandled_error && GoodJob.retry_on_unhandled_error
|
301
302
|
save!
|
302
|
-
elsif GoodJob.preserve_job_records == true ||
|
303
|
+
elsif GoodJob.preserve_job_records == true || reenqueued || (result.unhandled_error && GoodJob.preserve_job_records == :on_unhandled_error)
|
303
304
|
self.finished_at = Time.current
|
304
305
|
save!
|
305
306
|
else
|
@@ -37,11 +37,12 @@ module GoodJob
|
|
37
37
|
# @param function [String, Symbol] Postgres Advisory Lock function name to use
|
38
38
|
# @return [ActiveRecord::Relation]
|
39
39
|
# A relation selecting only the records that were locked.
|
40
|
-
scope :advisory_lock, (lambda do |column: _advisory_lockable_column, function: advisory_lockable_function|
|
40
|
+
scope :advisory_lock, (lambda do |column: _advisory_lockable_column, function: advisory_lockable_function, select_limit: nil|
|
41
41
|
original_query = self
|
42
42
|
|
43
43
|
cte_table = Arel::Table.new(:rows)
|
44
44
|
cte_query = original_query.select(primary_key, column).except(:limit)
|
45
|
+
cte_query = cte_query.limit(select_limit) if select_limit
|
45
46
|
cte_type = if supports_cte_materialization_specifiers?
|
46
47
|
'MATERIALIZED'
|
47
48
|
else
|
@@ -154,10 +155,10 @@ module GoodJob
|
|
154
155
|
# MyLockableRecord.order(created_at: :asc).limit(2).with_advisory_lock do |record|
|
155
156
|
# do_something_with record
|
156
157
|
# end
|
157
|
-
def with_advisory_lock(column: _advisory_lockable_column, function: advisory_lockable_function, unlock_session: false)
|
158
|
+
def with_advisory_lock(column: _advisory_lockable_column, function: advisory_lockable_function, unlock_session: false, select_limit: nil)
|
158
159
|
raise ArgumentError, "Must provide a block" unless block_given?
|
159
160
|
|
160
|
-
records = advisory_lock(column: column, function: function).to_a
|
161
|
+
records = advisory_lock(column: column, function: function, select_limit: select_limit).to_a
|
161
162
|
|
162
163
|
begin
|
163
164
|
unscoped { yield(records) }
|
@@ -41,5 +41,7 @@ class CreateGoodJobs < ActiveRecord::Migration<%= migration_version %>
|
|
41
41
|
add_index :good_jobs, [:cron_key, :cron_at], name: :index_good_jobs_on_cron_key_and_cron_at, unique: true
|
42
42
|
add_index :good_jobs, [:active_job_id], name: :index_good_jobs_on_active_job_id
|
43
43
|
add_index :good_jobs, [:finished_at], where: "retried_good_job_id IS NULL AND finished_at IS NOT NULL", name: :index_good_jobs_jobs_on_finished_at
|
44
|
+
add_index :good_jobs, [:priority, :created_at], order: { priority: "DESC NULLS LAST", created_at: :asc },
|
45
|
+
where: "finished_at IS NULL", name: :index_good_jobs_jobs_on_priority_created_at_when_unfinished
|
44
46
|
end
|
45
47
|
end
|
@@ -0,0 +1,18 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
class CreateIndexGoodJobsJobsOnPriorityCreatedAtWhenUnfinished < ActiveRecord::Migration<%= migration_version %>
|
3
|
+
disable_ddl_transaction!
|
4
|
+
|
5
|
+
def change
|
6
|
+
reversible do |dir|
|
7
|
+
dir.up do
|
8
|
+
# Ensure this incremental update migration is idempotent
|
9
|
+
# with monolithic install migration.
|
10
|
+
return if connection.index_name_exists?(:good_jobs, :index_good_jobs_jobs_on_priority_created_at_when_unfinished)
|
11
|
+
end
|
12
|
+
end
|
13
|
+
|
14
|
+
add_index :good_jobs, [:priority, :created_at], order: { priority: "DESC NULLS LAST", created_at: :asc },
|
15
|
+
where: "finished_at IS NULL", name: :index_good_jobs_jobs_on_priority_created_at_when_unfinished,
|
16
|
+
algorithm: :concurrently
|
17
|
+
end
|
18
|
+
end
|
data/lib/good_job/cli.rb
CHANGED
@@ -81,7 +81,12 @@ module GoodJob
|
|
81
81
|
desc: "Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)"
|
82
82
|
method_option :probe_port,
|
83
83
|
type: :numeric,
|
84
|
+
banner: 'PORT',
|
84
85
|
desc: "Port for http health check (env var: GOOD_JOB_PROBE_PORT, default: nil)"
|
86
|
+
method_option :queue_select_limit,
|
87
|
+
type: :numeric,
|
88
|
+
banner: 'COUNT',
|
89
|
+
desc: "The number of queued jobs to select when polling for a job to run. (env var: GOOD_JOB_QUEUE_SELECT_LIMIT, default: nil)"
|
85
90
|
|
86
91
|
def start
|
87
92
|
set_up_application!
|
@@ -208,6 +208,19 @@ module GoodJob
|
|
208
208
|
cron.map { |cron_key, params| GoodJob::CronEntry.new(params.merge(key: cron_key)) }
|
209
209
|
end
|
210
210
|
|
211
|
+
# The number of queued jobs to select when polling for a job to run.
|
212
|
+
# This limit is intended to avoid locking a large number of rows when selecting eligible jobs
|
213
|
+
# from the queue. This value should be higher than the total number of threads across all good_job
|
214
|
+
# processes to ensure a thread can retrieve an eligible and unlocked job.
|
215
|
+
# @return [Integer, nil]
|
216
|
+
def queue_select_limit
|
217
|
+
(
|
218
|
+
options[:queue_select_limit] ||
|
219
|
+
rails_config[:queue_select_limit] ||
|
220
|
+
env['GOOD_JOB_QUEUE_SELECT_LIMIT']
|
221
|
+
)&.to_i
|
222
|
+
end
|
223
|
+
|
211
224
|
# Whether to destroy discarded jobs when cleaning up preserved jobs.
|
212
225
|
# This configuration is only used when {GoodJob.preserve_job_records} is +true+.
|
213
226
|
# @return [Boolean]
|
@@ -24,7 +24,7 @@ module GoodJob
|
|
24
24
|
# Perform the next eligible job
|
25
25
|
# @return [Object, nil] Returns job result or +nil+ if no job was found
|
26
26
|
def next
|
27
|
-
job_query.perform_with_advisory_lock(parsed_queues: parsed_queues)
|
27
|
+
job_query.perform_with_advisory_lock(parsed_queues: parsed_queues, queue_select_limit: GoodJob.configuration.queue_select_limit)
|
28
28
|
end
|
29
29
|
|
30
30
|
# Tests whether this performer should be used in GoodJob's current state.
|
data/lib/good_job/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: good_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 3.
|
4
|
+
version: 3.6.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ben Sheldon
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2022-10-
|
11
|
+
date: 2022-10-22 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activejob
|
@@ -402,6 +402,7 @@ files:
|
|
402
402
|
- lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb
|
403
403
|
- lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb.erb
|
404
404
|
- lib/generators/good_job/templates/update/migrations/02_create_good_job_settings.rb.erb
|
405
|
+
- lib/generators/good_job/templates/update/migrations/03_create_index_good_jobs_jobs_on_priority_created_at_when_unfinished.rb.erb
|
405
406
|
- lib/generators/good_job/update_generator.rb
|
406
407
|
- lib/good_job.rb
|
407
408
|
- lib/good_job/active_job_extensions/concurrency.rb
|