sidekiq-unique-jobs 7.1.15 → 7.1.19
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of sidekiq-unique-jobs might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +50 -0
- data/README.md +8 -9
- data/lib/sidekiq_unique_jobs/lock/until_executed.rb +5 -1
- data/lib/sidekiq_unique_jobs/lock/until_expired.rb +3 -1
- data/lib/sidekiq_unique_jobs/lock/while_executing.rb +4 -1
- data/lib/sidekiq_unique_jobs/lock_args.rb +1 -1
- data/lib/sidekiq_unique_jobs/lock_config.rb +2 -2
- data/lib/sidekiq_unique_jobs/lock_digest.rb +2 -2
- data/lib/sidekiq_unique_jobs/locksmith.rb +1 -0
- data/lib/sidekiq_unique_jobs/lua/shared/_common.lua +1 -1
- data/lib/sidekiq_unique_jobs/orphans/manager.rb +9 -3
- data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +21 -0
- data/lib/sidekiq_unique_jobs/testing.rb +22 -8
- data/lib/sidekiq_unique_jobs/timer_task.rb +281 -2
- data/lib/sidekiq_unique_jobs/version.rb +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5432e71679519bcfbc3605de7c156e0820cf2c463ee9a5d458c1b59bd7c63fec
|
4
|
+
data.tar.gz: ecd548ff2add944949acc25341d92e2f74aee5256978d32ee74180ba1eacbd66
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 926bde9f4894aba3806723c1d75faffdf8bde86514b37f69b4156ca12b3dd64228d6eb768a03f521d119ebb42f50bcfbe87392d788f54a6e49b233565743dcb4
|
7
|
+
data.tar.gz: 28d14de13123d83b6d948d8453a901efe0ae1667ebc0d6a444e702371579528eccb4ac42c298aa2eb425c23b402c155cc58f553def21cba9c7a6d46757a167e2
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,55 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v7.1.18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.18) (2022-04-05)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.17...v7.1.18)
|
6
|
+
|
7
|
+
**Implemented enhancements:**
|
8
|
+
|
9
|
+
- Make sure we reflect on execution failure [\#700](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/700) ([mhenrixon](https://github.com/mhenrixon))
|
10
|
+
|
11
|
+
## [v7.1.17](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.17) (2022-04-05)
|
12
|
+
|
13
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.16...v7.1.17)
|
14
|
+
|
15
|
+
## [v7.1.16](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.16) (2022-04-02)
|
16
|
+
|
17
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.15...v7.1.16)
|
18
|
+
|
19
|
+
**Implemented enhancements:**
|
20
|
+
|
21
|
+
- Abort Ruby Reaper when sidekiq queues are full [\#690](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/690) ([francesmcmullin](https://github.com/francesmcmullin))
|
22
|
+
- Quote '3.0' to ensure CI uses Ruby 3.0.x for the 3.0 entry [\#689](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/689) ([petergoldstein](https://github.com/petergoldstein))
|
23
|
+
|
24
|
+
**Fixed bugs:**
|
25
|
+
|
26
|
+
- Hotfix: Ensure consistent lock args [\#699](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/699) ([mhenrixon](https://github.com/mhenrixon))
|
27
|
+
- Expire older changelog entries first [\#698](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/698) ([mhenrixon](https://github.com/mhenrixon))
|
28
|
+
- Fix drift [\#688](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/688) ([mhenrixon](https://github.com/mhenrixon))
|
29
|
+
|
30
|
+
**Closed issues:**
|
31
|
+
|
32
|
+
- concurrent-ruby has dropped support for TimerTask timeouts [\#697](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/697)
|
33
|
+
- Most recent changelogs are removed first [\#696](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/696)
|
34
|
+
- Improve README slightly [\#694](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/694)
|
35
|
+
- locksmith.rb:327: NoMethodError: undefined method `+' for nil:NilClass [\#686](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/686)
|
36
|
+
- lock\_timeout cannot be nil [\#675](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/675)
|
37
|
+
- Skip reaping when queues are too large [\#670](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/670)
|
38
|
+
|
39
|
+
**Merged pull requests:**
|
40
|
+
|
41
|
+
- Improve readme [\#695](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/695) ([fwolfst](https://github.com/fwolfst))
|
42
|
+
- Add funding\_uri to gemspec [\#693](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/693) ([fwolfst](https://github.com/fwolfst))
|
43
|
+
- Fix worker validator [\#685](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/685) ([victorfgs](https://github.com/victorfgs))
|
44
|
+
|
45
|
+
## [v7.1.15](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.15) (2022-02-10)
|
46
|
+
|
47
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.14...v7.1.15)
|
48
|
+
|
49
|
+
**Merged pull requests:**
|
50
|
+
|
51
|
+
- Fixing reschedule when using a non default queue [\#679](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/679) ([bigzed](https://github.com/bigzed))
|
52
|
+
|
3
53
|
## [v7.1.14](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.14) (2022-02-04)
|
4
54
|
|
5
55
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.13...v7.1.14)
|
data/README.md
CHANGED
@@ -90,7 +90,7 @@ Want to show me some ❤️ for the hard work I do on this gem? You can use the
|
|
90
90
|
|
91
91
|
This gem adds unique constraints to sidekiq jobs. The uniqueness is achieved by creating a set of keys in redis based off of `queue`, `class`, `args` (in the sidekiq job hash).
|
92
92
|
|
93
|
-
By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen [Conflict Strategy](#conflict-strategy) strategy. Unless a conflict strategy is chosen
|
93
|
+
By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen [Conflict Strategy](#conflict-strategy) strategy. Unless a conflict strategy is chosen (?)
|
94
94
|
|
95
95
|
This is the documentation for the `main` branch. You can find the documentation for each release by navigating to its tag.
|
96
96
|
|
@@ -151,7 +151,7 @@ end
|
|
151
151
|
|
152
152
|
### Your first worker
|
153
153
|
|
154
|
-
The most likely to be
|
154
|
+
The lock type most likely to be is `:until_executed`. This type of lock creates a lock from when `UntilExecutedWorker.perform_async` is called until right after `UntilExecutedWorker.new.perform` has been called.
|
155
155
|
|
156
156
|
```ruby
|
157
157
|
# frozen_string_literal: true
|
@@ -159,8 +159,7 @@ The most likely to be used worker is `:until_executed`. This type of lock create
|
|
159
159
|
class UntilExecutedWorker
|
160
160
|
include Sidekiq::Worker
|
161
161
|
|
162
|
-
sidekiq_options
|
163
|
-
lock: :until_executed
|
162
|
+
sidekiq_options lock: :until_executed
|
164
163
|
|
165
164
|
def perform
|
166
165
|
logger.info("cowboy")
|
@@ -346,9 +345,9 @@ Please not that if you try to override a default lock, an `ArgumentError` will b
|
|
346
345
|
|
347
346
|
## Conflict Strategy
|
348
347
|
|
349
|
-
Decides how we handle conflict. We can either reject the job to the dead queue or reschedule it. Both are useful for jobs that absolutely need to run and have been configured to use the lock `WhileExecuting` that is used only by the sidekiq server process.
|
348
|
+
Decides how we handle conflict. We can either `reject` the job to the dead queue or `reschedule` it. Both are useful for jobs that absolutely need to run and have been configured to use the lock `WhileExecuting` that is used only by the sidekiq server process.
|
350
349
|
|
351
|
-
|
350
|
+
Furthermore, `log` can be be used with the lock `UntilExecuted` and `UntilExpired`. Now we write a log entry saying the job could not be pushed because it is a duplicate of another job with the same arguments.
|
352
351
|
|
353
352
|
It is possible for locks to have different conflict strategy for the client and server. This is useful for `:until_and_while_executing`.
|
354
353
|
|
@@ -394,7 +393,7 @@ queue and retry the lock again.
|
|
394
393
|
This is slightly dangerous and should probably only be used for jobs that are
|
395
394
|
always scheduled in the future. Currently only attempting to retry one time.
|
396
395
|
|
397
|
-
###
|
396
|
+
### reschedule
|
398
397
|
|
399
398
|
```ruby
|
400
399
|
sidekiq_options on_conflict: :reschedule
|
@@ -568,7 +567,7 @@ The reason this happens is that the server couldn't find a valid sidekiq worker
|
|
568
567
|
|
569
568
|
### Validating Worker Configuration
|
570
569
|
|
571
|
-
Since v7 it is possible to perform some simple validation against your workers sidekiq_options
|
570
|
+
Since v7 it is possible to perform some simple validation against your workers `sidekiq_options`. What it does is scan for some issues that are known to cause problems in production.
|
572
571
|
|
573
572
|
Let's take a _bad_ worker:
|
574
573
|
|
@@ -604,7 +603,7 @@ assert_raise(InvalidWorker){ SidekiqUniqueJobs.validate_worker!(BadWorker.get_si
|
|
604
603
|
|
605
604
|
### Uniqueness
|
606
605
|
|
607
|
-
This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I there for decided to do what Mike is doing for sidekiq enterprise. Read the section about unique jobs: [Enterprise unique jobs][]
|
606
|
+
This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I there for decided to do what Mike is doing for sidekiq enterprise. Read the section about unique jobs: [Enterprise unique jobs][](?)
|
608
607
|
|
609
608
|
```ruby
|
610
609
|
SidekiqUniqueJobs.configure do |config|
|
@@ -33,10 +33,14 @@ module SidekiqUniqueJobs
|
|
33
33
|
# Executes in the Sidekiq server process
|
34
34
|
# @yield to the worker class perform method
|
35
35
|
def execute
|
36
|
-
locksmith.execute do
|
36
|
+
executed = locksmith.execute do
|
37
37
|
yield
|
38
38
|
unlock_and_callback
|
39
39
|
end
|
40
|
+
|
41
|
+
reflect(:execution_failed, item) unless executed
|
42
|
+
|
43
|
+
nil
|
40
44
|
end
|
41
45
|
end
|
42
46
|
end
|
@@ -33,7 +33,9 @@ module SidekiqUniqueJobs
|
|
33
33
|
# Executes in the Sidekiq server process
|
34
34
|
# @yield to the worker class perform method
|
35
35
|
def execute(&block)
|
36
|
-
locksmith.execute(&block)
|
36
|
+
executed = locksmith.execute(&block)
|
37
|
+
|
38
|
+
reflect(:execution_failed, item) unless executed
|
37
39
|
end
|
38
40
|
end
|
39
41
|
end
|
@@ -34,7 +34,7 @@ module SidekiqUniqueJobs
|
|
34
34
|
# The unique arguments to use for creating a lock
|
35
35
|
# @return [Array] the arguments filters by the {#filtered_args} method if {#lock_args_enabled?}
|
36
36
|
def lock_args
|
37
|
-
@lock_args ||= filtered_args
|
37
|
+
@lock_args ||= filtered_args || []
|
38
38
|
end
|
39
39
|
|
40
40
|
# Checks if the worker class has enabled lock_args
|
@@ -113,13 +113,13 @@ module SidekiqUniqueJobs
|
|
113
113
|
|
114
114
|
# the strategy to use as conflict resolution from sidekiq client
|
115
115
|
def on_client_conflict
|
116
|
-
@on_client_conflict ||= on_conflict["client"] if on_conflict.is_a?(Hash)
|
116
|
+
@on_client_conflict ||= on_conflict["client"] || on_conflict[:client] if on_conflict.is_a?(Hash)
|
117
117
|
@on_client_conflict ||= on_conflict
|
118
118
|
end
|
119
119
|
|
120
120
|
# the strategy to use as conflict resolution from sidekiq server
|
121
121
|
def on_server_conflict
|
122
|
-
@on_server_conflict ||= on_conflict["server"] if on_conflict.is_a?(Hash)
|
122
|
+
@on_server_conflict ||= on_conflict["server"] || on_conflict[:server] if on_conflict.is_a?(Hash)
|
123
123
|
@on_server_conflict ||= on_conflict
|
124
124
|
end
|
125
125
|
end
|
@@ -38,8 +38,8 @@ module SidekiqUniqueJobs
|
|
38
38
|
def initialize(item)
|
39
39
|
@item = item
|
40
40
|
@worker_class = item[CLASS]
|
41
|
-
@lock_args = item
|
42
|
-
@lock_prefix = item
|
41
|
+
@lock_args = item[LOCK_ARGS] || item[UNIQUE_ARGS] # TODO: Deprecate UNIQUE_ARGS
|
42
|
+
@lock_prefix = item[LOCK_PREFIX] || item[UNIQUE_PREFIX] # TODO: Deprecate UNIQUE_PREFIX
|
43
43
|
end
|
44
44
|
|
45
45
|
# Memoized lock_digest
|
@@ -31,7 +31,7 @@ local function log(message, prev_jid)
|
|
31
31
|
log_debug("ZADD", changelog, current_time, entry);
|
32
32
|
redis.call("ZADD", changelog, current_time, entry);
|
33
33
|
local total_entries = redis.call("ZCARD", changelog)
|
34
|
-
local removed_entries = redis.call("ZREMRANGEBYRANK", changelog,
|
34
|
+
local removed_entries = redis.call("ZREMRANGEBYRANK", changelog, 0, -1 * max_history)
|
35
35
|
if removed_entries > 0 then
|
36
36
|
log_debug("Removing", removed_entries , "entries from changelog (total entries", total_entries, "exceeds max_history:", max_history ..")");
|
37
37
|
end
|
@@ -1,5 +1,7 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
+
require "concurrent/version"
|
4
|
+
|
3
5
|
module SidekiqUniqueJobs
|
4
6
|
module Orphans
|
5
7
|
#
|
@@ -108,9 +110,13 @@ module SidekiqUniqueJobs
|
|
108
110
|
# @return [Hash]
|
109
111
|
#
|
110
112
|
def timer_task_options
|
111
|
-
{ run_now: true,
|
112
|
-
|
113
|
-
|
113
|
+
timer_task_options = { run_now: true, execution_interval: reaper_interval }
|
114
|
+
|
115
|
+
if VersionCheck.satisfied?(::Concurrent::VERSION, "< 1.1.10")
|
116
|
+
timer_task_options[:timeout_interval] = reaper_timeout
|
117
|
+
end
|
118
|
+
|
119
|
+
timer_task_options
|
114
120
|
end
|
115
121
|
|
116
122
|
#
|
@@ -15,6 +15,9 @@ module SidekiqUniqueJobs
|
|
15
15
|
# @return [String] the suffix for :RUN locks
|
16
16
|
RUN_SUFFIX = ":RUN"
|
17
17
|
#
|
18
|
+
# @return [Integer] the maximum combined length of sidekiq queues for running the reaper
|
19
|
+
MAX_QUEUE_LENGTH = 1000
|
20
|
+
#
|
18
21
|
# @!attribute [r] digests
|
19
22
|
# @return [SidekiqUniqueJobs::Digests] digest collection
|
20
23
|
attr_reader :digests
|
@@ -46,6 +49,8 @@ module SidekiqUniqueJobs
|
|
46
49
|
# @return [Integer] the number of reaped locks
|
47
50
|
#
|
48
51
|
def call
|
52
|
+
return if queues_very_full?
|
53
|
+
|
49
54
|
BatchDelete.call(orphans, conn)
|
50
55
|
end
|
51
56
|
|
@@ -212,6 +217,22 @@ module SidekiqUniqueJobs
|
|
212
217
|
end
|
213
218
|
end
|
214
219
|
|
220
|
+
# If sidekiq queues are very full, it becomes highly inefficient for the reaper
|
221
|
+
# because it must check every queued job to verify a digest is safe to delete
|
222
|
+
# The reaper checks queued jobs in batches of 50, adding 2 reads per digest
|
223
|
+
# With a queue length of 1,000 jobs, that's over 20 extra reads per digest.
|
224
|
+
def queues_very_full?
|
225
|
+
total_queue_size = 0
|
226
|
+
Sidekiq.redis do |conn|
|
227
|
+
queues(conn) do |queue|
|
228
|
+
total_queue_size += conn.llen("queue:#{queue}")
|
229
|
+
|
230
|
+
return true if total_queue_size > MAX_QUEUE_LENGTH
|
231
|
+
end
|
232
|
+
end
|
233
|
+
false
|
234
|
+
end
|
235
|
+
|
215
236
|
#
|
216
237
|
# Checks a sorted set for the existance of this digest
|
217
238
|
#
|
@@ -21,16 +21,24 @@ module Sidekiq
|
|
21
21
|
#
|
22
22
|
# @param [Hash<Symbol, Object>] tmp_config the temporary config to use
|
23
23
|
#
|
24
|
-
def self.use_options(tmp_config = {})
|
25
|
-
|
24
|
+
def self.use_options(tmp_config = {}) # rubocop:disable Metrics/MethodLength
|
25
|
+
if respond_to?(:default_job_options)
|
26
|
+
default_job_options.clear
|
27
|
+
self.default_job_options = tmp_config
|
28
|
+
else
|
29
|
+
default_worker_options.clear
|
30
|
+
self.default_worker_options = tmp_config
|
31
|
+
end
|
26
32
|
|
27
|
-
default_worker_options.clear
|
28
|
-
self.default_worker_options = tmp_config
|
29
33
|
yield
|
30
34
|
ensure
|
31
|
-
|
32
|
-
|
33
|
-
|
35
|
+
if respond_to?(:default_job_options)
|
36
|
+
default_job_options.clear
|
37
|
+
self.default_job_options = default_job_options
|
38
|
+
else
|
39
|
+
default_worker_options.clear
|
40
|
+
self.default_worker_options = DEFAULT_WORKER_OPTIONS
|
41
|
+
end
|
34
42
|
end
|
35
43
|
|
36
44
|
#
|
@@ -54,7 +62,13 @@ module Sidekiq
|
|
54
62
|
|
55
63
|
yield
|
56
64
|
ensure
|
57
|
-
self.sidekiq_options_hash =
|
65
|
+
self.sidekiq_options_hash =
|
66
|
+
if Sidekiq.respond_to?(:default_job_options)
|
67
|
+
Sidekiq.default_job_options
|
68
|
+
else
|
69
|
+
DEFAULT_WORKER_OPTIONS
|
70
|
+
end
|
71
|
+
|
58
72
|
sidekiq_options(old_options)
|
59
73
|
end
|
60
74
|
|
@@ -1,9 +1,275 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
+
require "concurrent/collection/copy_on_notify_observer_set"
|
4
|
+
require "concurrent/concern/dereferenceable"
|
5
|
+
require "concurrent/concern/observable"
|
6
|
+
require "concurrent/atomic/atomic_boolean"
|
7
|
+
require "concurrent/executor/executor_service"
|
8
|
+
require "concurrent/executor/ruby_executor_service"
|
9
|
+
require "concurrent/executor/safe_task_executor"
|
10
|
+
require "concurrent/scheduled_task"
|
11
|
+
|
3
12
|
module SidekiqUniqueJobs
|
4
|
-
#
|
13
|
+
# A very common concurrency pattern is to run a thread that performs a task at
|
14
|
+
# regular intervals. The thread that performs the task sleeps for the given
|
15
|
+
# interval then wakes up and performs the task. Lather, rinse, repeat... This
|
16
|
+
# pattern causes two problems. First, it is difficult to test the business
|
17
|
+
# logic of the task because the task itself is tightly coupled with the
|
18
|
+
# concurrency logic. Second, an exception raised while performing the task can
|
19
|
+
# cause the entire thread to abend. In a long-running application where the
|
20
|
+
# task thread is intended to run for days/weeks/years a crashed task thread
|
21
|
+
# can pose a significant problem. `TimerTask` alleviates both problems.
|
22
|
+
#
|
23
|
+
# When a `TimerTask` is launched it starts a thread for monitoring the
|
24
|
+
# execution interval. The `TimerTask` thread does not perform the task,
|
25
|
+
# however. Instead, the TimerTask launches the task on a separate thread.
|
26
|
+
# Should the task experience an unrecoverable crash only the task thread will
|
27
|
+
# crash. This makes the `TimerTask` very fault tolerant. Additionally, the
|
28
|
+
# `TimerTask` thread can respond to the success or failure of the task,
|
29
|
+
# performing logging or ancillary operations. `TimerTask` can also be
|
30
|
+
# configured with a timeout value allowing it to kill a task that runs too
|
31
|
+
# long.
|
32
|
+
#
|
33
|
+
# One other advantage of `TimerTask` is that it forces the business logic to
|
34
|
+
# be completely decoupled from the concurrency logic. The business logic can
|
35
|
+
# be tested separately then passed to the `TimerTask` for scheduling and
|
36
|
+
# running.
|
37
|
+
#
|
38
|
+
# In some cases it may be necessary for a `TimerTask` to affect its own
|
39
|
+
# execution cycle. To facilitate this, a reference to the TimerTask instance
|
40
|
+
# is passed as an argument to the provided block every time the task is
|
41
|
+
# executed.
|
42
|
+
#
|
43
|
+
# The `TimerTask` class includes the `Dereferenceable` mixin module so the
|
44
|
+
# result of the last execution is always available via the `#value` method.
|
45
|
+
# Dereferencing options can be passed to the `TimerTask` during construction or
|
46
|
+
# at any later time using the `#set_deref_options` method.
|
47
|
+
#
|
48
|
+
# `TimerTask` supports notification through the Ruby standard library
|
49
|
+
# {http://ruby-doc.org/stdlib-2.0/libdoc/observer/rdoc/Observable.html
|
50
|
+
# Observable} module. On execution the `TimerTask` will notify the observers
|
51
|
+
# with three arguments: time of execution, the result of the block (or nil on
|
52
|
+
# failure), and any raised exceptions (or nil on success). If the timeout
|
53
|
+
# interval is exceeded the observer will receive a `Concurrent::TimeoutError`
|
54
|
+
# object as the third argument.
|
55
|
+
#
|
56
|
+
# @!macro copy_options
|
57
|
+
#
|
58
|
+
# @example Basic usage
|
59
|
+
# task = Concurrent::TimerTask.new{ puts 'Boom!' }
|
60
|
+
# task.execute
|
61
|
+
#
|
62
|
+
# task.execution_interval #=> 60 (default)
|
63
|
+
# task.timeout_interval #=> 30 (default)
|
64
|
+
#
|
65
|
+
# # wait 60 seconds...
|
66
|
+
# #=> 'Boom!'
|
67
|
+
#
|
68
|
+
# task.shutdown #=> true
|
69
|
+
#
|
70
|
+
# @example Configuring `:execution_interval` and `:timeout_interval`
|
71
|
+
# task = Concurrent::TimerTask.new(execution_interval: 5, timeout_interval: 5) do
|
72
|
+
# puts 'Boom!'
|
73
|
+
# end
|
74
|
+
#
|
75
|
+
# task.execution_interval #=> 5
|
76
|
+
# task.timeout_interval #=> 5
|
77
|
+
#
|
78
|
+
# @example Immediate execution with `:run_now`
|
79
|
+
# task = Concurrent::TimerTask.new(run_now: true){ puts 'Boom!' }
|
80
|
+
# task.execute
|
81
|
+
#
|
82
|
+
# #=> 'Boom!'
|
5
83
|
#
|
6
|
-
|
84
|
+
# @example Last `#value` and `Dereferenceable` mixin
|
85
|
+
# task = Concurrent::TimerTask.new(
|
86
|
+
# dup_on_deref: true,
|
87
|
+
# execution_interval: 5
|
88
|
+
# ){ Time.now }
|
89
|
+
#
|
90
|
+
# task.execute
|
91
|
+
# Time.now #=> 2013-11-07 18:06:50 -0500
|
92
|
+
# sleep(10)
|
93
|
+
# task.value #=> 2013-11-07 18:06:55 -0500
|
94
|
+
#
|
95
|
+
# @example Controlling execution from within the block
|
96
|
+
# timer_task = Concurrent::TimerTask.new(execution_interval: 1) do |task|
|
97
|
+
# task.execution_interval.times{ print 'Boom! ' }
|
98
|
+
# print "\n"
|
99
|
+
# task.execution_interval += 1
|
100
|
+
# if task.execution_interval > 5
|
101
|
+
# puts 'Stopping...'
|
102
|
+
# task.shutdown
|
103
|
+
# end
|
104
|
+
# end
|
105
|
+
#
|
106
|
+
# timer_task.execute # blocking call - this task will stop itself
|
107
|
+
# #=> Boom!
|
108
|
+
# #=> Boom! Boom!
|
109
|
+
# #=> Boom! Boom! Boom!
|
110
|
+
# #=> Boom! Boom! Boom! Boom!
|
111
|
+
# #=> Boom! Boom! Boom! Boom! Boom!
|
112
|
+
# #=> Stopping...
|
113
|
+
#
|
114
|
+
# @example Observation
|
115
|
+
# class TaskObserver
|
116
|
+
# def update(time, result, ex)
|
117
|
+
# if result
|
118
|
+
# print "(#{time}) Execution successfully returned #{result}\n"
|
119
|
+
# elsif ex.is_a?(Concurrent::TimeoutError)
|
120
|
+
# print "(#{time}) Execution timed out\n"
|
121
|
+
# else
|
122
|
+
# print "(#{time}) Execution failed with error #{ex}\n"
|
123
|
+
# end
|
124
|
+
# end
|
125
|
+
# end
|
126
|
+
#
|
127
|
+
# task = Concurrent::TimerTask.new(execution_interval: 1, timeout_interval: 1){ 42 }
|
128
|
+
# task.add_observer(TaskObserver.new)
|
129
|
+
# task.execute
|
130
|
+
# sleep 4
|
131
|
+
#
|
132
|
+
# #=> (2013-10-13 19:08:58 -0400) Execution successfully returned 42
|
133
|
+
# #=> (2013-10-13 19:08:59 -0400) Execution successfully returned 42
|
134
|
+
# #=> (2013-10-13 19:09:00 -0400) Execution successfully returned 42
|
135
|
+
# task.shutdown
|
136
|
+
#
|
137
|
+
# task = Concurrent::TimerTask.new(execution_interval: 1, timeout_interval: 1){ sleep }
|
138
|
+
# task.add_observer(TaskObserver.new)
|
139
|
+
# task.execute
|
140
|
+
#
|
141
|
+
# #=> (2013-10-13 19:07:25 -0400) Execution timed out
|
142
|
+
# #=> (2013-10-13 19:07:27 -0400) Execution timed out
|
143
|
+
# #=> (2013-10-13 19:07:29 -0400) Execution timed out
|
144
|
+
# task.shutdown
|
145
|
+
#
|
146
|
+
# task = Concurrent::TimerTask.new(execution_interval: 1){ raise StandardError }
|
147
|
+
# task.add_observer(TaskObserver.new)
|
148
|
+
# task.execute
|
149
|
+
#
|
150
|
+
# #=> (2013-10-13 19:09:37 -0400) Execution failed with error StandardError
|
151
|
+
# #=> (2013-10-13 19:09:38 -0400) Execution failed with error StandardError
|
152
|
+
# #=> (2013-10-13 19:09:39 -0400) Execution failed with error StandardError
|
153
|
+
# task.shutdown
|
154
|
+
#
|
155
|
+
# @see http://ruby-doc.org/stdlib-2.0/libdoc/observer/rdoc/Observable.html
|
156
|
+
# @see http://docs.oracle.com/javase/7/docs/api/java/util/TimerTask.html
|
157
|
+
class TimerTask < Concurrent::RubyExecutorService # rubocop:disable Metrics/ClassLength
|
158
|
+
include Concurrent::Concern::Dereferenceable
|
159
|
+
include Concurrent::Concern::Observable
|
160
|
+
|
161
|
+
# Default `:execution_interval` in seconds.
|
162
|
+
EXECUTION_INTERVAL = 60
|
163
|
+
|
164
|
+
# Default `:timeout_interval` in seconds.
|
165
|
+
TIMEOUT_INTERVAL = 30
|
166
|
+
|
167
|
+
# Create a new TimerTask with the given task and configuration.
|
168
|
+
#
|
169
|
+
# @!macro timer_task_initialize
|
170
|
+
# @param [Hash] opts the options defining task execution.
|
171
|
+
# @option opts [Integer] :execution_interval number of seconds between
|
172
|
+
# task executions (default: EXECUTION_INTERVAL)
|
173
|
+
# @option opts [Integer] :timeout_interval number of seconds a task can
|
174
|
+
# run before it is considered to have failed (default: TIMEOUT_INTERVAL)
|
175
|
+
# @option opts [Boolean] :run_now Whether to run the task immediately
|
176
|
+
# upon instantiation or to wait until the first # execution_interval
|
177
|
+
# has passed (default: false)
|
178
|
+
#
|
179
|
+
# @!macro deref_options
|
180
|
+
#
|
181
|
+
# @raise ArgumentError when no block is given.
|
182
|
+
#
|
183
|
+
# @yield to the block after :execution_interval seconds have passed since
|
184
|
+
# the last yield
|
185
|
+
# @yieldparam task a reference to the `TimerTask` instance so that the
|
186
|
+
# block can control its own lifecycle. Necessary since `self` will
|
187
|
+
# refer to the execution context of the block rather than the running
|
188
|
+
# `TimerTask`.
|
189
|
+
#
|
190
|
+
# @return [TimerTask] the new `TimerTask`
|
191
|
+
def initialize(opts = {}, &task)
|
192
|
+
raise ArgumentError, "no block given" unless task
|
193
|
+
|
194
|
+
super
|
195
|
+
set_deref_options opts
|
196
|
+
end
|
197
|
+
|
198
|
+
# Is the executor running?
|
199
|
+
#
|
200
|
+
# @return [Boolean] `true` when running, `false` when shutting down or shutdown
|
201
|
+
def running?
|
202
|
+
@running.true?
|
203
|
+
end
|
204
|
+
|
205
|
+
# Execute a previously created `TimerTask`.
|
206
|
+
#
|
207
|
+
# @return [TimerTask] a reference to `self`
|
208
|
+
#
|
209
|
+
# @example Instance and execute in separate steps
|
210
|
+
# task = Concurrent::TimerTask.new(execution_interval: 10){ print "Hello World\n" }
|
211
|
+
# task.running? #=> false
|
212
|
+
# task.execute
|
213
|
+
# task.running? #=> true
|
214
|
+
#
|
215
|
+
# @example Instance and execute in one line
|
216
|
+
# task = Concurrent::TimerTask.new(execution_interval: 10){ print "Hello World\n" }.execute
|
217
|
+
# task.running? #=> true
|
218
|
+
def execute
|
219
|
+
synchronize do
|
220
|
+
if @running.false?
|
221
|
+
@running.make_true
|
222
|
+
schedule_next_task(@run_now ? 0 : @execution_interval)
|
223
|
+
end
|
224
|
+
end
|
225
|
+
self
|
226
|
+
end
|
227
|
+
|
228
|
+
# Create and execute a new `TimerTask`.
|
229
|
+
#
|
230
|
+
# @!macro timer_task_initialize
|
231
|
+
#
|
232
|
+
# @example
|
233
|
+
# task = Concurrent::TimerTask.execute(execution_interval: 10){ print "Hello World\n" }
|
234
|
+
# task.running? #=> true
|
235
|
+
def self.execute(opts = {}, &task)
|
236
|
+
SidekiqUniqueJobs::TimerTask.new(opts, &task).execute
|
237
|
+
end
|
238
|
+
|
239
|
+
# @!attribute [rw] execution_interval
|
240
|
+
# @return [Fixnum] Number of seconds after the task completes before the
|
241
|
+
# task is performed again.
|
242
|
+
def execution_interval
|
243
|
+
synchronize { @execution_interval }
|
244
|
+
end
|
245
|
+
|
246
|
+
# @!attribute [rw] execution_interval
|
247
|
+
# @return [Fixnum] Number of seconds after the task completes before the
|
248
|
+
# task is performed again.
|
249
|
+
def execution_interval=(value)
|
250
|
+
raise ArgumentError, "must be greater than zero" if (value = value.to_f) <= 0.0
|
251
|
+
|
252
|
+
synchronize { @execution_interval = value }
|
253
|
+
end
|
254
|
+
|
255
|
+
# @!attribute [rw] timeout_interval
|
256
|
+
# @return [Fixnum] Number of seconds the task can run before it is
|
257
|
+
# considered to have failed.
|
258
|
+
def timeout_interval
|
259
|
+
synchronize { @timeout_interval }
|
260
|
+
end
|
261
|
+
|
262
|
+
# @!attribute [rw] timeout_interval
|
263
|
+
# @return [Fixnum] Number of seconds the task can run before it is
|
264
|
+
# considered to have failed.
|
265
|
+
def timeout_interval=(value)
|
266
|
+
raise ArgumentError, "must be greater than zero" if (value = value.to_f) <= 0.0
|
267
|
+
|
268
|
+
synchronize { @timeout_interval = value }
|
269
|
+
end
|
270
|
+
|
271
|
+
private :post, :<<
|
272
|
+
|
7
273
|
private
|
8
274
|
|
9
275
|
def ns_initialize(opts, &task)
|
@@ -20,6 +286,19 @@ module SidekiqUniqueJobs
|
|
20
286
|
self.observers = Concurrent::Collection::CopyOnNotifyObserverSet.new
|
21
287
|
end
|
22
288
|
|
289
|
+
# @!visibility private
|
290
|
+
def ns_shutdown_execution
|
291
|
+
@running.make_false
|
292
|
+
super
|
293
|
+
end
|
294
|
+
|
295
|
+
# @!visibility private
|
296
|
+
def ns_kill_execution
|
297
|
+
@running.make_false
|
298
|
+
super
|
299
|
+
end
|
300
|
+
|
301
|
+
# @!visibility private
|
23
302
|
def schedule_next_task(interval = execution_interval)
|
24
303
|
exec_task = ->(completion) { execute_task(completion) }
|
25
304
|
Concurrent::ScheduledTask.execute(interval, args: [Concurrent::Event.new], &exec_task)
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: sidekiq-unique-jobs
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 7.1.
|
4
|
+
version: 7.1.19
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mikael Henriksson
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2022-
|
11
|
+
date: 2022-04-09 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: brpoplpush-redis_script
|
@@ -257,7 +257,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
257
257
|
- !ruby/object:Gem::Version
|
258
258
|
version: '0'
|
259
259
|
requirements: []
|
260
|
-
rubygems_version: 3.3.
|
260
|
+
rubygems_version: 3.3.7
|
261
261
|
signing_key:
|
262
262
|
specification_version: 4
|
263
263
|
summary: Sidekiq middleware that prevents duplicates jobs
|