good_job 3.19.3 → 3.20.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +38 -0
- data/README.md +1 -1
- data/app/models/concerns/good_job/advisory_lockable.rb +8 -6
- data/app/models/good_job/batch.rb +7 -3
- data/app/models/good_job/execution.rb +5 -2
- data/lib/good_job/active_job_extensions/concurrency.rb +7 -1
- data/lib/good_job/adapter.rb +80 -76
- data/lib/good_job/version.rb +1 -1
- data/lib/good_job.rb +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 62d33742278a734dd514fc3147abd1393ee62c3fd644c4c065b10c4fc6a24755
|
4
|
+
data.tar.gz: 83d56214ec454cbc3d910d4cbefb0413877ca3b18577168c0d9b9b456385a7a3
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 90a3730d4989f26837b0ac9cd8f15ea3029f0c1c474159a4798a58d695153cb2067c336d382f971ba42a6faf371f677f7b295741c8ab1557add7c6b403a1fa94
|
7
|
+
data.tar.gz: 684e5d8473b4f460707394fd670067b0aaf7a2749ed525be2893967c0953ed1c5afe4f35e8acfce4acaa7f7fa39ae00d16394b38d186a11f7eb8812839d65ff3
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,43 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v3.20.0](https://github.com/bensheldon/good_job/tree/v3.20.0) (2023-10-23)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.19.4...v3.20.0)
|
6
|
+
|
7
|
+
**Closed issues:**
|
8
|
+
|
9
|
+
- Dashboard blocked in iframe tag [\#1111](https://github.com/bensheldon/good_job/issues/1111)
|
10
|
+
- PG::ConnectionBad: PQsocket\(\) can't get socket descriptor [\#1100](https://github.com/bensheldon/good_job/issues/1100)
|
11
|
+
|
12
|
+
**Merged pull requests:**
|
13
|
+
|
14
|
+
- Wrap Adapter enqueue methods and Batch callbacks with Rails Reloader; verify in tests that no Advisory locks remain at database connection check-in [\#1124](https://github.com/bensheldon/good_job/pull/1124) ([bensheldon](https://github.com/bensheldon))
|
15
|
+
- Run all RSpec examples within a Rails Executor [\#1122](https://github.com/bensheldon/good_job/pull/1122) ([bensheldon](https://github.com/bensheldon))
|
16
|
+
- Print better debugging for retained advisory locks in test [\#1121](https://github.com/bensheldon/good_job/pull/1121) ([bensheldon](https://github.com/bensheldon))
|
17
|
+
- Replace Heroku-specific Rake tasks with `db:prepare` now that Demo is upgraded to Rails 7.1 [\#1120](https://github.com/bensheldon/good_job/pull/1120) ([bensheldon](https://github.com/bensheldon))
|
18
|
+
- Do not error debug logs if `pg_stat_activity` join is empty [\#1119](https://github.com/bensheldon/good_job/pull/1119) ([bensheldon](https://github.com/bensheldon))
|
19
|
+
- Remove pinned psych version [\#1114](https://github.com/bensheldon/good_job/pull/1114) ([bensheldon](https://github.com/bensheldon))
|
20
|
+
- Remove `pg_advisory_unlock_all()` after job is run; only verify blank `finished_at` \(and not lock presence\) before performing job [\#1113](https://github.com/bensheldon/good_job/pull/1113) ([bensheldon](https://github.com/bensheldon))
|
21
|
+
- Update docs url [\#1112](https://github.com/bensheldon/good_job/pull/1112) ([ur5us](https://github.com/ur5us))
|
22
|
+
- Pin psych gem 5.1.0 [\#1108](https://github.com/bensheldon/good_job/pull/1108) ([bensheldon](https://github.com/bensheldon))
|
23
|
+
- Add sampling for Skylight traces on Demo [\#1107](https://github.com/bensheldon/good_job/pull/1107) ([bensheldon](https://github.com/bensheldon))
|
24
|
+
- Add Rails 7.1 to test matrix [\#1105](https://github.com/bensheldon/good_job/pull/1105) ([bensheldon](https://github.com/bensheldon))
|
25
|
+
- Add spec to verify unhandled thread errors are reported [\#1104](https://github.com/bensheldon/good_job/pull/1104) ([bensheldon](https://github.com/bensheldon))
|
26
|
+
- Update Codespace configuration [\#1101](https://github.com/bensheldon/good_job/pull/1101) ([bensheldon](https://github.com/bensheldon))
|
27
|
+
|
28
|
+
## [v3.19.4](https://github.com/bensheldon/good_job/tree/v3.19.4) (2023-10-04)
|
29
|
+
|
30
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.19.3...v3.19.4)
|
31
|
+
|
32
|
+
**Closed issues:**
|
33
|
+
|
34
|
+
- Including GoodJob::ActiveJobExtensions::Concurrency triggers rails deprecation notice [\#1096](https://github.com/bensheldon/good_job/issues/1096)
|
35
|
+
- Add retries exhaused callback [\#1080](https://github.com/bensheldon/good_job/issues/1080)
|
36
|
+
|
37
|
+
**Merged pull requests:**
|
38
|
+
|
39
|
+
- Address rails 7.1 deprecation when using `ActiveJobExtensions::Concurrency` [\#1097](https://github.com/bensheldon/good_job/pull/1097) ([Earlopain](https://github.com/Earlopain))
|
40
|
+
|
3
41
|
## [v3.19.3](https://github.com/bensheldon/good_job/tree/v3.19.3) (2023-09-28)
|
4
42
|
|
5
43
|
[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.19.2...v3.19.3)
|
data/README.md
CHANGED
@@ -816,7 +816,7 @@ GoodJob.on_thread_error = -> (exception) { Rails.error.report(exception) }
|
|
816
816
|
|
817
817
|
By default, GoodJob relies on ActiveJob's retry functionality.
|
818
818
|
|
819
|
-
ActiveJob can be configured to retry an infinite number of times, with
|
819
|
+
ActiveJob can be configured to retry an infinite number of times, with a polynomial backoff. Using ActiveJob's `retry_on` prevents exceptions from reaching GoodJob:
|
820
820
|
|
821
821
|
```ruby
|
822
822
|
class ApplicationJob < ActiveJob::Base
|
@@ -53,7 +53,7 @@ module GoodJob
|
|
53
53
|
composed_cte = Arel::Nodes::As.new(cte_table, Arel::Nodes::SqlLiteral.new([cte_type, "(", cte_query.to_sql, ")"].join(' ')))
|
54
54
|
query = cte_table.project(cte_table[:id])
|
55
55
|
.with(composed_cte)
|
56
|
-
.where(Arel.sql(
|
56
|
+
.where(Arel.sql("#{function}(('x' || substr(md5(#{connection.quote(table_name)} || '-' || #{connection.quote_table_name(cte_table.name)}.#{connection.quote_column_name(column)}::text), 1, 16))::bit(64)::bigint)"))
|
57
57
|
|
58
58
|
limit = original_query.arel.ast.limit
|
59
59
|
query.limit = limit.value if limit.present?
|
@@ -74,14 +74,12 @@ module GoodJob
|
|
74
74
|
# @example Get the records that have a session awaiting a lock:
|
75
75
|
# MyLockableRecord.joins_advisory_locks.where("pg_locks.granted = ?", false)
|
76
76
|
scope :joins_advisory_locks, (lambda do |column: _advisory_lockable_column|
|
77
|
-
|
77
|
+
joins(<<~SQL.squish)
|
78
78
|
LEFT JOIN pg_locks ON pg_locks.locktype = 'advisory'
|
79
79
|
AND pg_locks.objsubid = 1
|
80
|
-
AND pg_locks.classid = ('x' || substr(md5(
|
81
|
-
AND pg_locks.objid = (('x' || substr(md5(
|
80
|
+
AND pg_locks.classid = ('x' || substr(md5(#{connection.quote(table_name)} || '-' || #{quoted_table_name}.#{connection.quote_column_name(column)}::text), 1, 16))::bit(32)::int
|
81
|
+
AND pg_locks.objid = (('x' || substr(md5(#{connection.quote(table_name)} || '-' || #{quoted_table_name}.#{connection.quote_column_name(column)}::text), 1, 16))::bit(64) << 32)::bit(32)::int
|
82
82
|
SQL
|
83
|
-
|
84
|
-
joins(sanitize_sql_for_conditions([join_sql, { table_name: table_name }]))
|
85
83
|
end)
|
86
84
|
|
87
85
|
# Joins the current query with Postgres's +pg_locks+ table AND SELECTs the resulting columns
|
@@ -151,6 +149,10 @@ module GoodJob
|
|
151
149
|
# can (as in {Lockable.advisory_lock}) and only pass those that could be
|
152
150
|
# locked to the block.
|
153
151
|
#
|
152
|
+
# If the Active Record Relation has WHERE conditions that have the potential
|
153
|
+
# to be updated/changed elsewhere, be sure to verify the conditions are still
|
154
|
+
# satisfied, or check the lock status, as an unlocked and out-of-date record could be returned.
|
155
|
+
#
|
154
156
|
# @param column [String, Symbol] name of advisory lock or unlock function
|
155
157
|
# @param function [String, Symbol] Postgres Advisory Lock function name to use
|
156
158
|
# @param unlock_session [Boolean] Whether to unlock all advisory locks in the session afterwards
|
@@ -101,9 +101,13 @@ module GoodJob
|
|
101
101
|
|
102
102
|
active_jobs = add(active_jobs, &block)
|
103
103
|
|
104
|
-
|
105
|
-
record.
|
106
|
-
|
104
|
+
Rails.application.reloader.wrap do
|
105
|
+
record.with_advisory_lock(function: "pg_advisory_lock") do
|
106
|
+
record.update!(enqueued_at: Time.current)
|
107
|
+
|
108
|
+
# During inline execution, this could enqueue and execute further jobs
|
109
|
+
record._continue_discard_or_finish(lock: false)
|
110
|
+
end
|
107
111
|
end
|
108
112
|
|
109
113
|
active_jobs
|
@@ -256,12 +256,13 @@ module GoodJob
|
|
256
256
|
def self.perform_with_advisory_lock(parsed_queues: nil, queue_select_limit: nil)
|
257
257
|
execution = nil
|
258
258
|
result = nil
|
259
|
-
unfinished.dequeueing_ordered(parsed_queues).only_scheduled.limit(1).with_advisory_lock(
|
259
|
+
unfinished.dequeueing_ordered(parsed_queues).only_scheduled.limit(1).with_advisory_lock(select_limit: queue_select_limit) do |executions|
|
260
260
|
execution = executions.first
|
261
261
|
break if execution.blank?
|
262
262
|
|
263
263
|
unless execution.executable?
|
264
264
|
result = ExecutionResult.new(value: nil, unexecutable: true)
|
265
|
+
execution = nil
|
265
266
|
break
|
266
267
|
end
|
267
268
|
|
@@ -491,7 +492,9 @@ module GoodJob
|
|
491
492
|
# Tests whether this job is safe to be executed by this thread.
|
492
493
|
# @return [Boolean]
|
493
494
|
def executable?
|
494
|
-
|
495
|
+
reload.finished_at.blank?
|
496
|
+
rescue ActiveRecord::RecordNotFound
|
497
|
+
false
|
495
498
|
end
|
496
499
|
|
497
500
|
def make_discrete
|
@@ -36,10 +36,16 @@ module GoodJob
|
|
36
36
|
end
|
37
37
|
end
|
38
38
|
|
39
|
+
wait_key = if ActiveJob.gem_version >= Gem::Version.new("7.1.0.a")
|
40
|
+
:polynomially_longer
|
41
|
+
else
|
42
|
+
:exponentially_longer
|
43
|
+
end
|
44
|
+
|
39
45
|
retry_on(
|
40
46
|
GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError,
|
41
47
|
attempts: Float::INFINITY,
|
42
|
-
wait:
|
48
|
+
wait: wait_key
|
43
49
|
)
|
44
50
|
|
45
51
|
before_perform do |job|
|
data/lib/good_job/adapter.rb
CHANGED
@@ -49,74 +49,76 @@ module GoodJob
|
|
49
49
|
active_jobs = Array(active_jobs)
|
50
50
|
return 0 if active_jobs.empty?
|
51
51
|
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
52
|
+
Rails.application.reloader.wrap do
|
53
|
+
current_time = Time.current
|
54
|
+
executions = active_jobs.map do |active_job|
|
55
|
+
GoodJob::Execution.build_for_enqueue(active_job).tap do |execution|
|
56
|
+
if GoodJob::Execution.discrete_support?
|
57
|
+
execution.make_discrete
|
58
|
+
execution.scheduled_at = current_time if execution.scheduled_at == execution.created_at
|
59
|
+
end
|
59
60
|
|
60
|
-
|
61
|
-
|
61
|
+
execution.created_at = current_time
|
62
|
+
execution.updated_at = current_time
|
63
|
+
end
|
62
64
|
end
|
63
|
-
end
|
64
65
|
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
70
|
-
|
71
|
-
|
66
|
+
inline_executions = []
|
67
|
+
GoodJob::Execution.transaction(requires_new: true, joinable: false) do
|
68
|
+
execution_attributes = executions.map do |execution|
|
69
|
+
if GoodJob::Execution.error_event_migrated?
|
70
|
+
execution.attributes
|
71
|
+
else
|
72
|
+
execution.attributes.except('error_event')
|
73
|
+
end
|
72
74
|
end
|
73
|
-
end
|
74
75
|
|
75
|
-
|
76
|
+
results = GoodJob::Execution.insert_all(execution_attributes, returning: %w[id active_job_id]) # rubocop:disable Rails/SkipsModelValidations
|
76
77
|
|
77
|
-
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
84
|
-
|
85
|
-
|
78
|
+
job_id_to_provider_job_id = results.each_with_object({}) { |result, hash| hash[result['active_job_id']] = result['id'] }
|
79
|
+
active_jobs.each do |active_job|
|
80
|
+
active_job.provider_job_id = job_id_to_provider_job_id[active_job.job_id]
|
81
|
+
active_job.successfully_enqueued = active_job.provider_job_id.present? if active_job.respond_to?(:successfully_enqueued=)
|
82
|
+
end
|
83
|
+
executions.each do |execution|
|
84
|
+
execution.instance_variable_set(:@new_record, false) if job_id_to_provider_job_id[execution.active_job_id]
|
85
|
+
end
|
86
|
+
executions = executions.select(&:persisted?) # prune unpersisted executions
|
86
87
|
|
87
|
-
|
88
|
-
|
89
|
-
|
88
|
+
if execute_inline?
|
89
|
+
inline_executions = executions.select { |execution| (execution.scheduled_at.nil? || execution.scheduled_at <= Time.current) }
|
90
|
+
inline_executions.each(&:advisory_lock!)
|
91
|
+
end
|
90
92
|
end
|
91
|
-
end
|
92
93
|
|
93
|
-
|
94
|
-
|
95
|
-
|
96
|
-
|
97
|
-
|
98
|
-
|
99
|
-
|
100
|
-
|
94
|
+
begin
|
95
|
+
until inline_executions.empty?
|
96
|
+
begin
|
97
|
+
inline_execution = inline_executions.shift
|
98
|
+
inline_result = inline_execution.perform
|
99
|
+
ensure
|
100
|
+
inline_execution.advisory_unlock
|
101
|
+
inline_execution.run_callbacks(:perform_unlocked)
|
102
|
+
end
|
103
|
+
raise inline_result.unhandled_error if inline_result.unhandled_error
|
101
104
|
end
|
102
|
-
|
105
|
+
ensure
|
106
|
+
inline_executions.each(&:advisory_unlock)
|
103
107
|
end
|
104
|
-
ensure
|
105
|
-
inline_executions.each(&:advisory_unlock)
|
106
|
-
end
|
107
108
|
|
108
|
-
|
109
|
-
|
110
|
-
|
111
|
-
|
112
|
-
|
113
|
-
|
114
|
-
|
115
|
-
|
116
|
-
|
117
|
-
|
118
|
-
|
119
|
-
|
109
|
+
non_inline_executions = executions.reject(&:finished_at)
|
110
|
+
if non_inline_executions.any?
|
111
|
+
job_id_to_active_jobs = active_jobs.index_by(&:job_id)
|
112
|
+
non_inline_executions.group_by(&:queue_name).each do |queue_name, executions_by_queue|
|
113
|
+
executions_by_queue.group_by(&:scheduled_at).each do |scheduled_at, executions_by_queue_and_scheduled_at|
|
114
|
+
state = { queue_name: queue_name, count: executions_by_queue_and_scheduled_at.size }
|
115
|
+
state[:scheduled_at] = scheduled_at if scheduled_at
|
116
|
+
|
117
|
+
executed_locally = execute_async? && @capsule&.create_thread(state)
|
118
|
+
unless executed_locally
|
119
|
+
state[:count] = job_id_to_active_jobs.values_at(*executions_by_queue_and_scheduled_at.map(&:active_job_id)).count { |active_job| send_notify?(active_job) }
|
120
|
+
Notifier.notify(state) unless state[:count].zero?
|
121
|
+
end
|
120
122
|
end
|
121
123
|
end
|
122
124
|
end
|
@@ -137,30 +139,32 @@ module GoodJob
|
|
137
139
|
# job there to be enqueued using enqueue_all
|
138
140
|
return if GoodJob::Bulk.capture(active_job, queue_adapter: self)
|
139
141
|
|
140
|
-
|
141
|
-
|
142
|
-
|
143
|
-
|
144
|
-
|
145
|
-
|
142
|
+
Rails.application.reloader.wrap do
|
143
|
+
will_execute_inline = execute_inline? && (scheduled_at.nil? || scheduled_at <= Time.current)
|
144
|
+
execution = GoodJob::Execution.enqueue(
|
145
|
+
active_job,
|
146
|
+
scheduled_at: scheduled_at,
|
147
|
+
create_with_advisory_lock: will_execute_inline
|
148
|
+
)
|
146
149
|
|
147
|
-
|
148
|
-
|
149
|
-
|
150
|
-
|
151
|
-
|
152
|
-
|
150
|
+
if will_execute_inline
|
151
|
+
begin
|
152
|
+
result = execution.perform
|
153
|
+
ensure
|
154
|
+
execution.advisory_unlock
|
155
|
+
execution.run_callbacks(:perform_unlocked)
|
156
|
+
end
|
157
|
+
raise result.unhandled_error if result.unhandled_error
|
158
|
+
else
|
159
|
+
job_state = { queue_name: execution.queue_name }
|
160
|
+
job_state[:scheduled_at] = execution.scheduled_at if execution.scheduled_at
|
161
|
+
|
162
|
+
executed_locally = execute_async? && @capsule&.create_thread(job_state)
|
163
|
+
Notifier.notify(job_state) if !executed_locally && send_notify?(active_job)
|
153
164
|
end
|
154
|
-
raise result.unhandled_error if result.unhandled_error
|
155
|
-
else
|
156
|
-
job_state = { queue_name: execution.queue_name }
|
157
|
-
job_state[:scheduled_at] = execution.scheduled_at if execution.scheduled_at
|
158
165
|
|
159
|
-
|
160
|
-
Notifier.notify(job_state) if !executed_locally && send_notify?(active_job)
|
166
|
+
execution
|
161
167
|
end
|
162
|
-
|
163
|
-
execution
|
164
168
|
end
|
165
169
|
|
166
170
|
# Shut down the thread pool executors.
|
data/lib/good_job/version.rb
CHANGED
data/lib/good_job.rb
CHANGED
@@ -237,7 +237,7 @@ module GoodJob
|
|
237
237
|
def self.perform_inline(queue_string = "*")
|
238
238
|
job_performer = JobPerformer.new(queue_string)
|
239
239
|
loop do
|
240
|
-
result = job_performer.next
|
240
|
+
result = Rails.application.reloader.wrap { job_performer.next }
|
241
241
|
break unless result
|
242
242
|
raise result.unhandled_error if result.unhandled_error
|
243
243
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: good_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 3.
|
4
|
+
version: 3.20.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ben Sheldon
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2023-
|
11
|
+
date: 2023-10-23 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activejob
|
@@ -384,7 +384,7 @@ licenses:
|
|
384
384
|
metadata:
|
385
385
|
bug_tracker_uri: https://github.com/bensheldon/good_job/issues
|
386
386
|
changelog_uri: https://github.com/bensheldon/good_job/blob/master/CHANGELOG.md
|
387
|
-
documentation_uri: https://
|
387
|
+
documentation_uri: https://rubydoc.info/gems/good_job
|
388
388
|
homepage_uri: https://github.com/bensheldon/good_job
|
389
389
|
source_code_uri: https://github.com/bensheldon/good_job
|
390
390
|
rubygems_mfa_required: 'true'
|