sidekiq-unique-jobs 7.1.19 → 7.1.22
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of sidekiq-unique-jobs might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +33 -0
- data/README.md +3 -1
- data/lib/sidekiq_unique_jobs/exceptions.rb +3 -3
- data/lib/sidekiq_unique_jobs/lock/until_executing.rb +3 -0
- data/lib/sidekiq_unique_jobs/lock_args.rb +18 -14
- data/lib/sidekiq_unique_jobs/lock_config.rb +4 -4
- data/lib/sidekiq_unique_jobs/lock_digest.rb +6 -6
- data/lib/sidekiq_unique_jobs/lock_timeout.rb +4 -4
- data/lib/sidekiq_unique_jobs/lock_ttl.rb +4 -4
- data/lib/sidekiq_unique_jobs/locksmith.rb +4 -4
- data/lib/sidekiq_unique_jobs/middleware.rb +4 -4
- data/lib/sidekiq_unique_jobs/on_conflict/reschedule.rb +3 -3
- data/lib/sidekiq_unique_jobs/options_with_fallback.rb +3 -3
- data/lib/sidekiq_unique_jobs/orphans/manager.rb +1 -14
- data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +28 -4
- data/lib/sidekiq_unique_jobs/sidekiq_worker_methods.rb +33 -24
- data/lib/sidekiq_unique_jobs/timer_task.rb +19 -77
- data/lib/sidekiq_unique_jobs/version.rb +1 -1
- data/lib/tasks/changelog.rake +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 51aa02bd059a91e83c5f65865db8a098346bace6ead052e63a4298c3521cdcb0
|
4
|
+
data.tar.gz: 7dc30256282491c0f1d8818f28266054c526f488a8ffccd4048dfda149e04644
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 9dac46329bb7bb7a6ede794b114554a618e4e577a024e592fb70970d0081d17f1ad651b2fd57cd2ecd2809d86d39518de4d2c97151663e4e1e03d7a700df6afa
|
7
|
+
data.tar.gz: f422fa2de3f44d900e2060e7301d7a9ceed8610d6f9416ef320a9a99a4589be1af7dd6c8f564f4ec372b11a37a325db4aba3aba548cb99abb2240ac83a4cfea8
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,38 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v7.1.21](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.21) (2022-04-23)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.20...v7.1.21)
|
6
|
+
|
7
|
+
**Implemented enhancements:**
|
8
|
+
|
9
|
+
- Prepare for Sidekiq v7 [\#707](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/707) ([mhenrixon](https://github.com/mhenrixon))
|
10
|
+
|
11
|
+
**Closed issues:**
|
12
|
+
|
13
|
+
- DEPRECATION WARNING: default\_worker\_options is deprecated and will be removed from Sidekiq 7.0 \(use default\_job\_options instead\) \(called from notify\_agents at /Users/hackeron/Development/Tether/timeline/app/models/user.rb:303\) [\#705](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/705)
|
14
|
+
|
15
|
+
## [v7.1.20](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.20) (2022-04-22)
|
16
|
+
|
17
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.19...v7.1.20)
|
18
|
+
|
19
|
+
**Implemented enhancements:**
|
20
|
+
|
21
|
+
- Manually handle timeouts [\#706](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/706) ([mhenrixon](https://github.com/mhenrixon))
|
22
|
+
|
23
|
+
**Merged pull requests:**
|
24
|
+
|
25
|
+
- improve README wrt. middleware config [\#704](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/704) ([slhck](https://github.com/slhck))
|
26
|
+
|
27
|
+
## [v7.1.19](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.19) (2022-04-09)
|
28
|
+
|
29
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.18...v7.1.19)
|
30
|
+
|
31
|
+
**Fixed bugs:**
|
32
|
+
|
33
|
+
- concurrent-ruby 1.1.10 spikes volume of jobs [\#701](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/701)
|
34
|
+
- Reimplement the entire TimerTask as it was [\#702](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/702) ([mhenrixon](https://github.com/mhenrixon))
|
35
|
+
|
3
36
|
## [v7.1.18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.18) (2022-04-05)
|
4
37
|
|
5
38
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.17...v7.1.18)
|
data/README.md
CHANGED
@@ -123,9 +123,11 @@ Before v7, the middleware was configured automatically. Since some people report
|
|
123
123
|
|
124
124
|
*NOTE* if you want to use the reaper you also need to configure the server middleware.
|
125
125
|
|
126
|
-
[
|
126
|
+
The following shows how to modify your `config/initializers/sidekiq.rb` file to use the middleware. [Here is a full example.](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/initializers/sidekiq.rb#L12)
|
127
127
|
|
128
128
|
```ruby
|
129
|
+
require "sidekiq-unique-jobs"
|
130
|
+
|
129
131
|
Sidekiq.configure_server do |config|
|
130
132
|
config.redis = { url: ENV["REDIS_URL"], driver: :hiredis }
|
131
133
|
|
@@ -72,14 +72,14 @@ module SidekiqUniqueJobs
|
|
72
72
|
class InvalidUniqueArguments < UniqueJobsError
|
73
73
|
def initialize(options)
|
74
74
|
given = options[:given]
|
75
|
-
|
75
|
+
job_class = options[:job_class]
|
76
76
|
lock_args_method = options[:lock_args_method]
|
77
|
-
lock_args_meth =
|
77
|
+
lock_args_meth = job_class.method(lock_args_method)
|
78
78
|
num_args = lock_args_meth.arity
|
79
79
|
source_location = lock_args_meth.source_location
|
80
80
|
|
81
81
|
super(
|
82
|
-
"#{
|
82
|
+
"#{job_class}##{lock_args_method} takes #{num_args} arguments, received #{given.inspect}" \
|
83
83
|
"\n\n" \
|
84
84
|
" #{source_location.join(':')}"
|
85
85
|
)
|
@@ -26,9 +26,9 @@ module SidekiqUniqueJobs
|
|
26
26
|
|
27
27
|
# @param [Hash] item a Sidekiq job hash
|
28
28
|
def initialize(item)
|
29
|
-
@item
|
30
|
-
@
|
31
|
-
|
29
|
+
@item = item
|
30
|
+
@args = item[ARGS]
|
31
|
+
self.job_class = item[CLASS]
|
32
32
|
end
|
33
33
|
|
34
34
|
# The unique arguments to use for creating a lock
|
@@ -83,31 +83,31 @@ module SidekiqUniqueJobs
|
|
83
83
|
|
84
84
|
# Filters unique arguments by method configured in the sidekiq worker
|
85
85
|
# @param [Array] args the arguments passed to the sidekiq worker
|
86
|
-
# @return [Array] unfiltered unless {#
|
86
|
+
# @return [Array] unfiltered unless {#job_method_defined?}
|
87
87
|
# @return [Array] with the filtered arguments
|
88
88
|
def filter_by_symbol(args)
|
89
|
-
return args unless
|
89
|
+
return args unless job_method_defined?(lock_args_method)
|
90
90
|
|
91
|
-
|
91
|
+
job_class.send(lock_args_method, args)
|
92
92
|
rescue ArgumentError
|
93
93
|
raise SidekiqUniqueJobs::InvalidUniqueArguments,
|
94
94
|
given: args,
|
95
|
-
|
95
|
+
job_class: job_class,
|
96
96
|
lock_args_method: lock_args_method
|
97
97
|
end
|
98
98
|
|
99
99
|
# The method to use for filtering unique arguments
|
100
100
|
def lock_args_method
|
101
|
-
@lock_args_method ||=
|
102
|
-
@lock_args_method ||= :lock_args if
|
103
|
-
@lock_args_method ||= :unique_args if
|
101
|
+
@lock_args_method ||= job_options.slice(LOCK_ARGS_METHOD, UNIQUE_ARGS_METHOD).values.first
|
102
|
+
@lock_args_method ||= :lock_args if job_method_defined?(:lock_args)
|
103
|
+
@lock_args_method ||= :unique_args if job_method_defined?(:unique_args)
|
104
104
|
@lock_args_method ||= default_lock_args_method
|
105
105
|
end
|
106
106
|
|
107
107
|
# The global worker options defined in Sidekiq directly
|
108
108
|
def default_lock_args_method
|
109
|
-
|
110
|
-
|
109
|
+
default_job_options[LOCK_ARGS_METHOD] ||
|
110
|
+
default_job_options[UNIQUE_ARGS_METHOD]
|
111
111
|
end
|
112
112
|
|
113
113
|
#
|
@@ -116,8 +116,12 @@ module SidekiqUniqueJobs
|
|
116
116
|
#
|
117
117
|
# @return [Hash<String, Object>]
|
118
118
|
#
|
119
|
-
def
|
120
|
-
@
|
119
|
+
def default_job_options
|
120
|
+
@default_job_options ||= if Sidekiq.respond_to?(:default_job_options)
|
121
|
+
Sidekiq.default_job_options.stringify_keys
|
122
|
+
else
|
123
|
+
Sidekiq.default_worker_options.stringify_keys
|
124
|
+
end
|
121
125
|
end
|
122
126
|
end
|
123
127
|
end
|
@@ -13,9 +13,9 @@ module SidekiqUniqueJobs
|
|
13
13
|
# @return [Symbol] the type of lock
|
14
14
|
attr_reader :type
|
15
15
|
#
|
16
|
-
# @!attribute [r]
|
17
|
-
# @return [Symbol] the
|
18
|
-
attr_reader :
|
16
|
+
# @!attribute [r] job
|
17
|
+
# @return [Symbol] the job class
|
18
|
+
attr_reader :job
|
19
19
|
#
|
20
20
|
# @!attribute [r] limit
|
21
21
|
# @return [Integer] the number of simultaneous locks
|
@@ -58,7 +58,7 @@ module SidekiqUniqueJobs
|
|
58
58
|
|
59
59
|
def initialize(job_hash = {})
|
60
60
|
@type = job_hash[LOCK]&.to_sym
|
61
|
-
@
|
61
|
+
@job = SidekiqUniqueJobs.safe_constantize(job_hash[CLASS])
|
62
62
|
@limit = job_hash.fetch(LOCK_LIMIT, 1)&.to_i
|
63
63
|
@timeout = job_hash.fetch(LOCK_TIMEOUT, 0)&.to_i
|
64
64
|
@ttl = job_hash.fetch(LOCK_TTL) { job_hash.fetch(LOCK_EXPIRATION, nil) }.to_i
|
@@ -36,10 +36,10 @@ module SidekiqUniqueJobs
|
|
36
36
|
|
37
37
|
# @param [Hash] item a Sidekiq job hash
|
38
38
|
def initialize(item)
|
39
|
-
@item
|
40
|
-
@
|
41
|
-
@
|
42
|
-
|
39
|
+
@item = item
|
40
|
+
@lock_args = item[LOCK_ARGS] || item[UNIQUE_ARGS] # TODO: Deprecate UNIQUE_ARGS
|
41
|
+
@lock_prefix = item[LOCK_PREFIX] || item[UNIQUE_PREFIX] # TODO: Deprecate UNIQUE_PREFIX
|
42
|
+
self.job_class = item[CLASS]
|
43
43
|
end
|
44
44
|
|
45
45
|
# Memoized lock_digest
|
@@ -67,13 +67,13 @@ module SidekiqUniqueJobs
|
|
67
67
|
# Checks if we should disregard the queue when creating the unique digest
|
68
68
|
# @return [true, false]
|
69
69
|
def unique_across_queues?
|
70
|
-
item[UNIQUE_ACROSS_QUEUES] ||
|
70
|
+
item[UNIQUE_ACROSS_QUEUES] || job_options[UNIQUE_ACROSS_QUEUES]
|
71
71
|
end
|
72
72
|
|
73
73
|
# Checks if we should disregard the worker when creating the unique digest
|
74
74
|
# @return [true, false]
|
75
75
|
def unique_across_workers?
|
76
|
-
item[UNIQUE_ACROSS_WORKERS] ||
|
76
|
+
item[UNIQUE_ACROSS_WORKERS] || job_options[UNIQUE_ACROSS_WORKERS]
|
77
77
|
end
|
78
78
|
end
|
79
79
|
end
|
@@ -30,8 +30,8 @@ module SidekiqUniqueJobs
|
|
30
30
|
# @option item [String] :class the class of the sidekiq worker
|
31
31
|
# @option item [Float] :at the unix time the job is scheduled at
|
32
32
|
def initialize(item)
|
33
|
-
@item
|
34
|
-
|
33
|
+
@item = item
|
34
|
+
self.job_class = item[CLASS]
|
35
35
|
end
|
36
36
|
|
37
37
|
#
|
@@ -42,9 +42,9 @@ module SidekiqUniqueJobs
|
|
42
42
|
# @return [Integer, nil]
|
43
43
|
#
|
44
44
|
def calculate
|
45
|
-
timeout =
|
45
|
+
timeout = default_job_options[LOCK_TIMEOUT]
|
46
46
|
timeout = default_lock_timeout if default_lock_timeout
|
47
|
-
timeout =
|
47
|
+
timeout = job_options[LOCK_TIMEOUT] if job_options.key?(LOCK_TIMEOUT)
|
48
48
|
timeout
|
49
49
|
end
|
50
50
|
|
@@ -33,8 +33,8 @@ module SidekiqUniqueJobs
|
|
33
33
|
# @option item [String] :class the class of the sidekiq worker
|
34
34
|
# @option item [Float] :at the unix time the job is scheduled at
|
35
35
|
def initialize(item)
|
36
|
-
@item
|
37
|
-
|
36
|
+
@item = item
|
37
|
+
self.job_class = item[CLASS]
|
38
38
|
end
|
39
39
|
|
40
40
|
#
|
@@ -67,9 +67,9 @@ module SidekiqUniqueJobs
|
|
67
67
|
#
|
68
68
|
def calculate
|
69
69
|
ttl = item[LOCK_TTL]
|
70
|
-
ttl ||=
|
70
|
+
ttl ||= job_options[LOCK_TTL]
|
71
71
|
ttl ||= item[LOCK_EXPIRATION] # TODO: Deprecate at some point
|
72
|
-
ttl ||=
|
72
|
+
ttl ||= job_options[LOCK_EXPIRATION] # TODO: Deprecate at some point
|
73
73
|
ttl ||= SidekiqUniqueJobs.config.lock_ttl
|
74
74
|
ttl && (ttl.to_i + time_until_scheduled)
|
75
75
|
end
|
@@ -188,18 +188,18 @@ module SidekiqUniqueJobs
|
|
188
188
|
# @yieldparam [string] job_id the sidekiq JID
|
189
189
|
# @yieldreturn [void] whatever the calling block returns
|
190
190
|
def lock!(conn, primed_method, wait = nil)
|
191
|
-
return yield
|
191
|
+
return yield if locked?(conn)
|
192
192
|
|
193
193
|
enqueue(conn) do |queued_jid|
|
194
|
-
reflect(:debug, item, queued_jid)
|
194
|
+
reflect(:debug, :queued, item, queued_jid)
|
195
195
|
|
196
196
|
primed_method.call(conn, wait) do |primed_jid|
|
197
197
|
reflect(:debug, :primed, item, primed_jid)
|
198
|
-
|
199
198
|
locked_jid = call_script(:lock, key.to_a, argv, conn)
|
199
|
+
|
200
200
|
if locked_jid
|
201
201
|
reflect(:debug, :locked, item, locked_jid)
|
202
|
-
return yield
|
202
|
+
return yield
|
203
203
|
end
|
204
204
|
end
|
205
205
|
end
|
@@ -29,10 +29,10 @@ module SidekiqUniqueJobs
|
|
29
29
|
# @yieldparam [<type>] if <description>
|
30
30
|
# @yieldreturn [<type>] <describe what yield should return>
|
31
31
|
def call(worker_class, item, queue, redis_pool = nil)
|
32
|
-
@
|
33
|
-
@
|
34
|
-
@
|
35
|
-
|
32
|
+
@item = item
|
33
|
+
@queue = queue
|
34
|
+
@redis_pool = redis_pool
|
35
|
+
self.job_class = item[CLASS]
|
36
36
|
return yield if unique_disabled?
|
37
37
|
|
38
38
|
SidekiqUniqueJobs::Job.prepare(item) unless item[LOCK_DIGEST]
|
@@ -14,14 +14,14 @@ module SidekiqUniqueJobs
|
|
14
14
|
# @param [Hash] item sidekiq job hash
|
15
15
|
def initialize(item, redis_pool = nil)
|
16
16
|
super(item, redis_pool)
|
17
|
-
|
17
|
+
self.job_class = item[CLASS]
|
18
18
|
end
|
19
19
|
|
20
20
|
# Create a new job from the current one.
|
21
21
|
# This will mess up sidekiq stats because a new job is created
|
22
22
|
def call
|
23
|
-
if
|
24
|
-
if
|
23
|
+
if sidekiq_job_class?
|
24
|
+
if job_class.set(queue: item["queue"].to_sym).perform_in(5, *item[ARGS])
|
25
25
|
reflect(:rescheduled, item)
|
26
26
|
else
|
27
27
|
reflect(:reschedule_failed, item)
|
@@ -6,7 +6,7 @@ module SidekiqUniqueJobs
|
|
6
6
|
# Requires the following methods to be defined in the including class
|
7
7
|
# 1. item (required)
|
8
8
|
# 2. options (can be nil)
|
9
|
-
# 3.
|
9
|
+
# 3. job_class (required, can be anything)
|
10
10
|
# @author Mikael Henriksson <mikael@mhenrixon.com>
|
11
11
|
module OptionsWithFallback
|
12
12
|
def self.included(base)
|
@@ -69,8 +69,8 @@ module SidekiqUniqueJobs
|
|
69
69
|
#
|
70
70
|
def options
|
71
71
|
@options ||= begin
|
72
|
-
opts =
|
73
|
-
opts.merge!(
|
72
|
+
opts = default_job_options.dup
|
73
|
+
opts.merge!(job_options) if sidekiq_job_class?
|
74
74
|
(opts || {}).stringify_keys
|
75
75
|
end
|
76
76
|
end
|
@@ -110,13 +110,7 @@ module SidekiqUniqueJobs
|
|
110
110
|
# @return [Hash]
|
111
111
|
#
|
112
112
|
def timer_task_options
|
113
|
-
|
114
|
-
|
115
|
-
if VersionCheck.satisfied?(::Concurrent::VERSION, "< 1.1.10")
|
116
|
-
timer_task_options[:timeout_interval] = reaper_timeout
|
117
|
-
end
|
118
|
-
|
119
|
-
timer_task_options
|
113
|
+
{ run_now: true, execution_interval: reaper_interval }
|
120
114
|
end
|
121
115
|
|
122
116
|
#
|
@@ -133,13 +127,6 @@ module SidekiqUniqueJobs
|
|
133
127
|
SidekiqUniqueJobs.config.reaper_interval
|
134
128
|
end
|
135
129
|
|
136
|
-
#
|
137
|
-
# @see SidekiqUniqueJobs::Config#reaper_timeout
|
138
|
-
#
|
139
|
-
def reaper_timeout
|
140
|
-
SidekiqUniqueJobs.config.reaper_timeout
|
141
|
-
end
|
142
|
-
|
143
130
|
#
|
144
131
|
# A context to use for all log entries
|
145
132
|
#
|
@@ -11,6 +11,8 @@ module SidekiqUniqueJobs
|
|
11
11
|
#
|
12
12
|
# rubocop:disable Metrics/ClassLength
|
13
13
|
class RubyReaper < Reaper
|
14
|
+
include SidekiqUniqueJobs::Timing
|
15
|
+
|
14
16
|
#
|
15
17
|
# @return [String] the suffix for :RUN locks
|
16
18
|
RUN_SUFFIX = ":RUN"
|
@@ -30,6 +32,16 @@ module SidekiqUniqueJobs
|
|
30
32
|
# @return [Redis::SortedSet] the Sidekiq RetrySet
|
31
33
|
attr_reader :retried
|
32
34
|
|
35
|
+
#
|
36
|
+
# @!attribute [r] start_time
|
37
|
+
# @return [Integer] The timestamp this execution started represented as integer
|
38
|
+
attr_reader :start_time
|
39
|
+
|
40
|
+
#
|
41
|
+
# @!attribute [r] timeout_ms
|
42
|
+
# @return [Integer] The allowed ms before timeout
|
43
|
+
attr_reader :timeout_ms
|
44
|
+
|
33
45
|
#
|
34
46
|
# Initialize a new instance of DeleteOrphans
|
35
47
|
#
|
@@ -37,9 +49,11 @@ module SidekiqUniqueJobs
|
|
37
49
|
#
|
38
50
|
def initialize(conn)
|
39
51
|
super(conn)
|
40
|
-
@digests
|
41
|
-
@scheduled
|
42
|
-
@retried
|
52
|
+
@digests = SidekiqUniqueJobs::Digests.new
|
53
|
+
@scheduled = Redis::SortedSet.new(SCHEDULE)
|
54
|
+
@retried = Redis::SortedSet.new(RETRY)
|
55
|
+
@start_time = time_source.call
|
56
|
+
@timeout_ms = SidekiqUniqueJobs.config.reaper_timeout * 1000
|
43
57
|
end
|
44
58
|
|
45
59
|
#
|
@@ -60,7 +74,7 @@ module SidekiqUniqueJobs
|
|
60
74
|
#
|
61
75
|
# @return [Array<String>] an array of orphaned digests
|
62
76
|
#
|
63
|
-
def orphans # rubocop:disable Metrics/MethodLength
|
77
|
+
def orphans # rubocop:disable Metrics/MethodLength, Metrics/CyclomaticComplexity
|
64
78
|
page = 0
|
65
79
|
per = reaper_count * 2
|
66
80
|
orphans = []
|
@@ -68,12 +82,14 @@ module SidekiqUniqueJobs
|
|
68
82
|
|
69
83
|
while results.size.positive?
|
70
84
|
results.each do |digest|
|
85
|
+
break if timeout?
|
71
86
|
next if belongs_to_job?(digest)
|
72
87
|
|
73
88
|
orphans << digest
|
74
89
|
break if orphans.size >= reaper_count
|
75
90
|
end
|
76
91
|
|
92
|
+
break if timeout?
|
77
93
|
break if orphans.size >= reaper_count
|
78
94
|
|
79
95
|
page += 1
|
@@ -83,6 +99,14 @@ module SidekiqUniqueJobs
|
|
83
99
|
orphans
|
84
100
|
end
|
85
101
|
|
102
|
+
def timeout?
|
103
|
+
elapsed_ms >= timeout_ms
|
104
|
+
end
|
105
|
+
|
106
|
+
def elapsed_ms
|
107
|
+
time_source.call - start_time
|
108
|
+
end
|
109
|
+
|
86
110
|
#
|
87
111
|
# Checks if the digest has a matching job.
|
88
112
|
# 1. It checks the scheduled set
|
@@ -5,50 +5,55 @@ module SidekiqUniqueJobs
|
|
5
5
|
#
|
6
6
|
# @author Mikael Henriksson <mikael@mhenrixon.com>
|
7
7
|
module SidekiqWorkerMethods
|
8
|
+
#
|
9
|
+
# @!attribute [r] job_class
|
10
|
+
# @return [Sidekiq::Worker] The Sidekiq::Worker implementation
|
11
|
+
attr_reader :job_class
|
12
|
+
|
8
13
|
# Avoids duplicating worker_class.respond_to? in multiple places
|
9
14
|
# @return [true, false]
|
10
|
-
def
|
11
|
-
|
15
|
+
def job_method_defined?(method_sym)
|
16
|
+
job_class.respond_to?(method_sym)
|
12
17
|
end
|
13
18
|
|
14
19
|
# Wraps #get_sidekiq_options to always work with a hash
|
15
20
|
# @return [Hash] of the worker class sidekiq options
|
16
|
-
def
|
17
|
-
return {} unless
|
21
|
+
def job_options
|
22
|
+
return {} unless sidekiq_job_class?
|
18
23
|
|
19
|
-
|
24
|
+
job_class.get_sidekiq_options.deep_stringify_keys
|
20
25
|
end
|
21
26
|
|
22
27
|
# Tests that the
|
23
|
-
# @return [true] if
|
24
|
-
# @return [false] if
|
25
|
-
def
|
26
|
-
|
28
|
+
# @return [true] if job_class responds to get_sidekiq_options
|
29
|
+
# @return [false] if job_class does not respond to get_sidekiq_options
|
30
|
+
def sidekiq_job_class?
|
31
|
+
job_method_defined?(:get_sidekiq_options)
|
27
32
|
end
|
28
33
|
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
@
|
34
|
+
def job_class=(obj)
|
35
|
+
# this is what was originally passed in, it can be an instance or a class depending on sidekiq version
|
36
|
+
@original_job_class = obj
|
37
|
+
@job_class = job_class_constantize(obj)
|
33
38
|
end
|
34
39
|
|
35
40
|
# The hook to call after a successful unlock
|
36
41
|
# @return [Proc]
|
37
42
|
def after_unlock_hook # rubocop:disable Metrics/MethodLength
|
38
43
|
lambda do
|
39
|
-
if @
|
44
|
+
if @original_job_class.respond_to?(:after_unlock)
|
40
45
|
# instance method in sidekiq v6
|
41
|
-
if @
|
42
|
-
@
|
46
|
+
if @original_job_class.method(:after_unlock).arity.positive? # arity check to maintain backwards compatibility
|
47
|
+
@original_job_class.after_unlock(item)
|
43
48
|
else
|
44
|
-
@
|
49
|
+
@original_job_class.after_unlock
|
45
50
|
end
|
46
|
-
elsif
|
51
|
+
elsif job_class.respond_to?(:after_unlock)
|
47
52
|
# class method regardless of sidekiq version
|
48
|
-
if
|
49
|
-
|
53
|
+
if job_class.method(:after_unlock).arity.positive? # arity check to maintain backwards compatibility
|
54
|
+
job_class.after_unlock(item)
|
50
55
|
else
|
51
|
-
|
56
|
+
job_class.after_unlock
|
52
57
|
end
|
53
58
|
end
|
54
59
|
end
|
@@ -58,7 +63,7 @@ module SidekiqUniqueJobs
|
|
58
63
|
# failing back to the original argument when the constant can't be found
|
59
64
|
#
|
60
65
|
# @return [Sidekiq::Worker]
|
61
|
-
def
|
66
|
+
def job_class_constantize(klazz = @job_class)
|
62
67
|
SidekiqUniqueJobs.safe_constantize(klazz)
|
63
68
|
end
|
64
69
|
|
@@ -68,8 +73,12 @@ module SidekiqUniqueJobs
|
|
68
73
|
#
|
69
74
|
# @return [Hash<Symbol, Object>]
|
70
75
|
#
|
71
|
-
def
|
72
|
-
Sidekiq.
|
76
|
+
def default_job_options
|
77
|
+
if Sidekiq.respond_to?(:default_job_options)
|
78
|
+
Sidekiq.default_job_options
|
79
|
+
else
|
80
|
+
Sidekiq.default_worker_options
|
81
|
+
end
|
73
82
|
end
|
74
83
|
end
|
75
84
|
end
|
@@ -26,9 +26,7 @@ module SidekiqUniqueJobs
|
|
26
26
|
# Should the task experience an unrecoverable crash only the task thread will
|
27
27
|
# crash. This makes the `TimerTask` very fault tolerant. Additionally, the
|
28
28
|
# `TimerTask` thread can respond to the success or failure of the task,
|
29
|
-
# performing logging or ancillary operations.
|
30
|
-
# configured with a timeout value allowing it to kill a task that runs too
|
31
|
-
# long.
|
29
|
+
# performing logging or ancillary operations.
|
32
30
|
#
|
33
31
|
# One other advantage of `TimerTask` is that it forces the business logic to
|
34
32
|
# be completely decoupled from the concurrency logic. The business logic can
|
@@ -49,9 +47,7 @@ module SidekiqUniqueJobs
|
|
49
47
|
# {http://ruby-doc.org/stdlib-2.0/libdoc/observer/rdoc/Observable.html
|
50
48
|
# Observable} module. On execution the `TimerTask` will notify the observers
|
51
49
|
# with three arguments: time of execution, the result of the block (or nil on
|
52
|
-
# failure), and any raised exceptions (or nil on success).
|
53
|
-
# interval is exceeded the observer will receive a `Concurrent::TimeoutError`
|
54
|
-
# object as the third argument.
|
50
|
+
# failure), and any raised exceptions (or nil on success).
|
55
51
|
#
|
56
52
|
# @!macro copy_options
|
57
53
|
#
|
@@ -60,20 +56,18 @@ module SidekiqUniqueJobs
|
|
60
56
|
# task.execute
|
61
57
|
#
|
62
58
|
# task.execution_interval #=> 60 (default)
|
63
|
-
# task.timeout_interval #=> 30 (default)
|
64
59
|
#
|
65
60
|
# # wait 60 seconds...
|
66
61
|
# #=> 'Boom!'
|
67
62
|
#
|
68
63
|
# task.shutdown #=> true
|
69
64
|
#
|
70
|
-
# @example Configuring `:execution_interval`
|
71
|
-
# task = Concurrent::TimerTask.new(execution_interval: 5
|
65
|
+
# @example Configuring `:execution_interval`
|
66
|
+
# task = Concurrent::TimerTask.new(execution_interval: 5) do
|
72
67
|
# puts 'Boom!'
|
73
68
|
# end
|
74
69
|
#
|
75
70
|
# task.execution_interval #=> 5
|
76
|
-
# task.timeout_interval #=> 5
|
77
71
|
#
|
78
72
|
# @example Immediate execution with `:run_now`
|
79
73
|
# task = Concurrent::TimerTask.new(run_now: true){ puts 'Boom!' }
|
@@ -116,15 +110,13 @@ module SidekiqUniqueJobs
|
|
116
110
|
# def update(time, result, ex)
|
117
111
|
# if result
|
118
112
|
# print "(#{time}) Execution successfully returned #{result}\n"
|
119
|
-
# elsif ex.is_a?(Concurrent::TimeoutError)
|
120
|
-
# print "(#{time}) Execution timed out\n"
|
121
113
|
# else
|
122
114
|
# print "(#{time}) Execution failed with error #{ex}\n"
|
123
115
|
# end
|
124
116
|
# end
|
125
117
|
# end
|
126
118
|
#
|
127
|
-
# task = Concurrent::TimerTask.new(execution_interval: 1
|
119
|
+
# task = Concurrent::TimerTask.new(execution_interval: 1){ 42 }
|
128
120
|
# task.add_observer(TaskObserver.new)
|
129
121
|
# task.execute
|
130
122
|
# sleep 4
|
@@ -134,7 +126,7 @@ module SidekiqUniqueJobs
|
|
134
126
|
# #=> (2013-10-13 19:09:00 -0400) Execution successfully returned 42
|
135
127
|
# task.shutdown
|
136
128
|
#
|
137
|
-
# task = Concurrent::TimerTask.new(execution_interval: 1
|
129
|
+
# task = Concurrent::TimerTask.new(execution_interval: 1){ sleep }
|
138
130
|
# task.add_observer(TaskObserver.new)
|
139
131
|
# task.execute
|
140
132
|
#
|
@@ -154,7 +146,7 @@ module SidekiqUniqueJobs
|
|
154
146
|
#
|
155
147
|
# @see http://ruby-doc.org/stdlib-2.0/libdoc/observer/rdoc/Observable.html
|
156
148
|
# @see http://docs.oracle.com/javase/7/docs/api/java/util/TimerTask.html
|
157
|
-
class TimerTask < Concurrent::RubyExecutorService
|
149
|
+
class TimerTask < Concurrent::RubyExecutorService
|
158
150
|
include Concurrent::Concern::Dereferenceable
|
159
151
|
include Concurrent::Concern::Observable
|
160
152
|
|
@@ -170,8 +162,6 @@ module SidekiqUniqueJobs
|
|
170
162
|
# @param [Hash] opts the options defining task execution.
|
171
163
|
# @option opts [Integer] :execution_interval number of seconds between
|
172
164
|
# task executions (default: EXECUTION_INTERVAL)
|
173
|
-
# @option opts [Integer] :timeout_interval number of seconds a task can
|
174
|
-
# run before it is considered to have failed (default: TIMEOUT_INTERVAL)
|
175
165
|
# @option opts [Boolean] :run_now Whether to run the task immediately
|
176
166
|
# upon instantiation or to wait until the first # execution_interval
|
177
167
|
# has passed (default: false)
|
@@ -233,7 +223,7 @@ module SidekiqUniqueJobs
|
|
233
223
|
# task = Concurrent::TimerTask.execute(execution_interval: 10){ print "Hello World\n" }
|
234
224
|
# task.running? #=> true
|
235
225
|
def self.execute(opts = {}, &task)
|
236
|
-
|
226
|
+
TimerTask.new(opts, &task).execute
|
237
227
|
end
|
238
228
|
|
239
229
|
# @!attribute [rw] execution_interval
|
@@ -252,22 +242,6 @@ module SidekiqUniqueJobs
|
|
252
242
|
synchronize { @execution_interval = value }
|
253
243
|
end
|
254
244
|
|
255
|
-
# @!attribute [rw] timeout_interval
|
256
|
-
# @return [Fixnum] Number of seconds the task can run before it is
|
257
|
-
# considered to have failed.
|
258
|
-
def timeout_interval
|
259
|
-
synchronize { @timeout_interval }
|
260
|
-
end
|
261
|
-
|
262
|
-
# @!attribute [rw] timeout_interval
|
263
|
-
# @return [Fixnum] Number of seconds the task can run before it is
|
264
|
-
# considered to have failed.
|
265
|
-
def timeout_interval=(value)
|
266
|
-
raise ArgumentError, "must be greater than zero" if (value = value.to_f) <= 0.0
|
267
|
-
|
268
|
-
synchronize { @timeout_interval = value }
|
269
|
-
end
|
270
|
-
|
271
245
|
private :post, :<<
|
272
246
|
|
273
247
|
private
|
@@ -276,12 +250,13 @@ module SidekiqUniqueJobs
|
|
276
250
|
set_deref_options(opts)
|
277
251
|
|
278
252
|
self.execution_interval = opts[:execution] || opts[:execution_interval] || EXECUTION_INTERVAL
|
279
|
-
|
280
|
-
|
281
|
-
|
282
|
-
@
|
283
|
-
@
|
284
|
-
@
|
253
|
+
if opts[:timeout] || opts[:timeout_interval]
|
254
|
+
warn "TimeTask timeouts are now ignored as these were not able to be implemented correctly"
|
255
|
+
end
|
256
|
+
@run_now = opts[:now] || opts[:run_now]
|
257
|
+
@executor = Concurrent::SafeTaskExecutor.new(task)
|
258
|
+
@running = Concurrent::AtomicBoolean.new(false)
|
259
|
+
@value = nil
|
285
260
|
|
286
261
|
self.observers = Concurrent::Collection::CopyOnNotifyObserverSet.new
|
287
262
|
end
|
@@ -306,52 +281,19 @@ module SidekiqUniqueJobs
|
|
306
281
|
end
|
307
282
|
|
308
283
|
# @!visibility private
|
309
|
-
def execute_task(completion)
|
284
|
+
def execute_task(completion)
|
310
285
|
return nil unless @running.true?
|
311
286
|
|
312
|
-
|
313
|
-
|
314
|
-
Concurrent::ScheduledTask.execute(
|
315
|
-
timeout_interval,
|
316
|
-
args: [completion],
|
317
|
-
&timeout_task
|
318
|
-
)
|
319
|
-
@thread_completed = Concurrent::Event.new
|
320
|
-
|
321
|
-
@value = @reason = nil
|
322
|
-
@executor.post do
|
323
|
-
@value = @task.call(self)
|
324
|
-
rescue Exception => ex # rubocop:disable Lint/RescueException
|
325
|
-
@reason = ex
|
326
|
-
ensure
|
327
|
-
@thread_completed.set
|
328
|
-
end
|
329
|
-
|
330
|
-
@thread_completed.wait
|
331
|
-
|
287
|
+
_success, value, reason = @executor.execute(self)
|
332
288
|
if completion.try?
|
289
|
+
self.value = value
|
333
290
|
schedule_next_task
|
334
291
|
time = Time.now
|
335
292
|
observers.notify_observers do
|
336
|
-
[time, value,
|
293
|
+
[time, self.value, reason]
|
337
294
|
end
|
338
295
|
end
|
339
296
|
nil
|
340
297
|
end
|
341
|
-
|
342
|
-
# @!visibility private
|
343
|
-
def timeout_task(completion)
|
344
|
-
return unless @running.true?
|
345
|
-
return unless completion.try?
|
346
|
-
|
347
|
-
@executor.kill
|
348
|
-
@executor.wait_for_termination
|
349
|
-
@executor = Concurrent::RubySingleThreadExecutor.new
|
350
|
-
|
351
|
-
@thread_completed.set
|
352
|
-
|
353
|
-
schedule_next_task
|
354
|
-
observers.notify_observers(Time.now, nil, Concurrent::TimeoutError.new)
|
355
|
-
end
|
356
298
|
end
|
357
299
|
end
|
data/lib/tasks/changelog.rake
CHANGED
@@ -17,7 +17,7 @@ COMMIT_CHANGELOG_CMD = "git commit -a -m 'Update changelog'"
|
|
17
17
|
desc "Generate a Changelog"
|
18
18
|
task :changelog do
|
19
19
|
sh("git checkout main")
|
20
|
-
sh(*CHANGELOG_CMD.push(ENV
|
20
|
+
sh(*CHANGELOG_CMD.push(ENV.fetch("CHANGELOG_GITHUB_TOKEN", nil)))
|
21
21
|
sh(ADD_CHANGELOG_CMD)
|
22
22
|
sh(COMMIT_CHANGELOG_CMD)
|
23
23
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: sidekiq-unique-jobs
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 7.1.
|
4
|
+
version: 7.1.22
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mikael Henriksson
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2022-04
|
11
|
+
date: 2022-05-04 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: brpoplpush-redis_script
|