sidekiq-unique-jobs 7.0.13 → 7.1.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq-unique-jobs might be problematic. Click here for more details.

Files changed (34) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +13 -0
  3. data/README.md +92 -25
  4. data/lib/sidekiq_unique_jobs/config.rb +16 -8
  5. data/lib/sidekiq_unique_jobs/constants.rb +44 -45
  6. data/lib/sidekiq_unique_jobs/deprecation.rb +35 -0
  7. data/lib/sidekiq_unique_jobs/exceptions.rb +9 -0
  8. data/lib/sidekiq_unique_jobs/lock/base_lock.rb +56 -51
  9. data/lib/sidekiq_unique_jobs/lock/until_and_while_executing.rb +31 -9
  10. data/lib/sidekiq_unique_jobs/lock/until_executed.rb +17 -5
  11. data/lib/sidekiq_unique_jobs/lock/until_executing.rb +15 -1
  12. data/lib/sidekiq_unique_jobs/lock/until_expired.rb +21 -0
  13. data/lib/sidekiq_unique_jobs/lock/while_executing.rb +12 -7
  14. data/lib/sidekiq_unique_jobs/lock_config.rb +1 -1
  15. data/lib/sidekiq_unique_jobs/lock_ttl.rb +1 -1
  16. data/lib/sidekiq_unique_jobs/locksmith.rb +80 -81
  17. data/lib/sidekiq_unique_jobs/middleware/client.rb +8 -10
  18. data/lib/sidekiq_unique_jobs/middleware/server.rb +2 -0
  19. data/lib/sidekiq_unique_jobs/on_conflict/reschedule.rb +7 -3
  20. data/lib/sidekiq_unique_jobs/options_with_fallback.rb +4 -11
  21. data/lib/sidekiq_unique_jobs/orphans/manager.rb +1 -0
  22. data/lib/sidekiq_unique_jobs/orphans/reaper_resurrector.rb +170 -0
  23. data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +1 -1
  24. data/lib/sidekiq_unique_jobs/redis/sorted_set.rb +1 -1
  25. data/lib/sidekiq_unique_jobs/reflectable.rb +17 -0
  26. data/lib/sidekiq_unique_jobs/reflections.rb +68 -0
  27. data/lib/sidekiq_unique_jobs/script/caller.rb +3 -1
  28. data/lib/sidekiq_unique_jobs/server.rb +2 -1
  29. data/lib/sidekiq_unique_jobs/sidekiq_unique_ext.rb +13 -35
  30. data/lib/sidekiq_unique_jobs/sidekiq_unique_jobs.rb +21 -0
  31. data/lib/sidekiq_unique_jobs/version.rb +1 -1
  32. data/lib/sidekiq_unique_jobs.rb +4 -0
  33. data/lib/tasks/changelog.rake +14 -14
  34. metadata +12 -8
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 31de42f9427b913ae2c311fe70bd63c1ebed4dba96de6837d5e1f489644f7222
4
- data.tar.gz: 46a500dc20edc1cbaa562a27ad66522819359deead834a97cf7bd08d1c50cc08
3
+ metadata.gz: e47af152a018a49730a81e9f77ad68debea6a82d1750c6f41e8375be664da2bc
4
+ data.tar.gz: 3bbb5a4fdb354b6c49e774e26c525b8a5678e83d4a1deb156d544522e76ecf3e
5
5
  SHA512:
6
- metadata.gz: b8fd43403d772a57add9fa5a6d1fef4e728b53e14a72df440e68eb345d6e3a867132de697ad483fd197ebfe3bc77bc2d80ab17af5727eac96c69ef31e29d4447
7
- data.tar.gz: fa2ecad50220f6cf6cb2b72666094f1761cb1c2671a505d4d804374b50b08a4b78b80ba82561f4a45f3108eb6098b8d2dd2ebfef4683aac226849412cb0eb1f9
6
+ metadata.gz: 0c4d4c979b44940d230d3ad0f3df7487f223b2545aae90eca00eddf0b05fb450fe21beb924929aa40c53bb012fa01cd82e0e611136048370c33baea0375bb184
7
+ data.tar.gz: 289cbba203cde12c4c070057fd642843d036f12f4623c70ecd3e9c5a777f12f416ea567e971512a86fe8556df4e03316b726f1320096655d0c54ada469758f94
data/CHANGELOG.md CHANGED
@@ -1,5 +1,18 @@
1
1
  # Changelog
2
2
 
3
+ ## [v7.0.12](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.12) (2021-06-04)
4
+
5
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.11...v7.0.12)
6
+
7
+ **Implemented enhancements:**
8
+
9
+ - Reduce noise of perfectly valid scenario [\#610](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/610) ([mhenrixon](https://github.com/mhenrixon))
10
+
11
+ **Merged pull requests:**
12
+
13
+ - Set correct namespace for custom strategy example [\#609](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/609) ([Wolfer](https://github.com/Wolfer))
14
+ - Clarify the documentation related to lock\_ttl [\#607](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/607) ([donaldpiret](https://github.com/donaldpiret))
15
+
3
16
  ## [v7.0.11](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.11) (2021-05-16)
4
17
 
5
18
  [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.10...v7.0.11)
data/README.md CHANGED
@@ -12,6 +12,18 @@
12
12
  - [Support Me](#support-me)
13
13
  - [Requirements](#requirements)
14
14
  - [General Information](#general-information)
15
+ - [Reflections \(metrics, logging, etc.\)](#reflections-metrics-logging-etc)
16
+ - [after_unlock_callback_failed](#after_unlock_callback_failed)
17
+ - [error](#error)
18
+ - [execution_failed](#execution_failed)
19
+ - [lock_failed](#lock_failed)
20
+ - [locked](#locked)
21
+ - [reschedule_failed](#reschedule_failed)
22
+ - [rescheduled](#rescheduled)
23
+ - [timeout](#timeout)
24
+ - [unlock_failed](#unlock_failed)
25
+ - [unlocked](#unlocked)
26
+ - [unknown_sidekiq_worker](#unknown_sidekiq_worker)
15
27
  - [Global Configuration](#global-configuration)
16
28
  - [debug_lua](#debug_lua)
17
29
  - [lock_timeout](#lock_timeout)
@@ -49,7 +61,6 @@
49
61
  - [Usage](#usage-1)
50
62
  - [Finer Control over Uniqueness](#finer-control-over-uniqueness)
51
63
  - [After Unlock Callback](#after-unlock-callback)
52
- - [Logging](#logging)
53
64
  - [Cleanup Dead Locks](#cleanup-dead-locks)
54
65
  - [Other Sidekiq gems](#other-sidekiq-gems)
55
66
  - [apartment-sidekiq](#apartment-sidekiq)
@@ -130,20 +141,16 @@ end
130
141
 
131
142
  ### Your first worker
132
143
 
144
+ The most likely to be used worker is `:until_executed`. This type of lock creates a lock from when `UntilExecutedWorker.perform_async` is called until the the sidekiq processor has processed the job.
145
+
133
146
  ```ruby
134
147
  # frozen_string_literal: true
135
148
 
136
149
  class UntilExecutedWorker
137
150
  include Sidekiq::Worker
138
151
 
139
- sidekiq_options queue: :special,
140
- retry: false,
141
- lock: :until_executed,
142
- lock_info: true,
143
- lock_timeout: 0,
144
- lock_prefix: "special",
145
- lock_ttl: 0,
146
- lock_limit: 5
152
+ sidekiq_options queue: :until_executed,
153
+ lock: :until_executed
147
154
 
148
155
  def perform
149
156
  logger.info("cowboy")
@@ -151,7 +158,6 @@ class UntilExecutedWorker
151
158
  logger.info("beebop")
152
159
  end
153
160
  end
154
-
155
161
  ```
156
162
 
157
163
  You can read more about the worker configuration in [Worker Configuration](#worker-configuration) below.
@@ -179,6 +185,82 @@ See [Interaction w/ Sidekiq](https://github.com/mhenrixon/sidekiq-unique-jobs/wi
179
185
 
180
186
  See [Locking & Unlocking](https://github.com/mhenrixon/sidekiq-unique-jobs/wiki/Locking-&-Unlocking) for an overview of the differences on when the various lock types are locked and unlocked.
181
187
 
188
+ ## Reflections (metrics, logging, etc.)
189
+
190
+ To be able to gather some insights on what is going on inside this gem. I provide a reflection API that can be used.
191
+
192
+ To setup reflections for logging or metrics, use the following API:
193
+
194
+ ```ruby
195
+
196
+ def extract_log_from_job(message, job_hash)
197
+ worker = job_hash['class']
198
+ args = job_hash['args']
199
+ lock_args = job_hash['lock_args']
200
+ queue = job_hash['queue']
201
+ {
202
+ message: message,
203
+ worker: worker,
204
+ args: args,
205
+ lock_args: lock_args,
206
+ queue: queue
207
+ }
208
+ end
209
+
210
+ SidekiqUniqueJobs.reflect do |on|
211
+ on.lock_failed do |job_hash|
212
+ message = extract_log_from_job('Lock Failed', job_hash)
213
+ Sidekiq.logger.warn(message)
214
+ end
215
+ end
216
+ ```
217
+
218
+ ### after_unlock_callback_failed
219
+
220
+ This is called when you have configured a custom callback for when a lock has been released.
221
+
222
+ ### error
223
+
224
+ Not in use yet but will be used deep into the stack to provide a means to catch and report errors inside the gem.
225
+
226
+ ### execution_failed
227
+
228
+ When the sidekiq processor picks the job of the queue for certain jobs but your job raised an error to the middleware. This will be the reflection. It is probably nothing to worry about. When your worker raises an error, we need to handle some edge cases for until and while executing.
229
+
230
+ ### lock_failed
231
+
232
+ If we can't achieve a lock, this will be the reflection. It most likely is nothing to worry about. We just couldn't retrieve a lock in a timely fashion.
233
+
234
+ The biggest reason for this reflection would be to gather metrics on which workers fail the most at the locking step for example.
235
+
236
+ ### locked
237
+
238
+ For when a lock has been successful. Again, mostly useful for metrics I suppose.
239
+
240
+ ### reschedule_failed
241
+
242
+ For when the reschedule strategy failed to reschedule the job.
243
+
244
+ ### rescheduled
245
+
246
+ For when a job was successfully rescheduled
247
+
248
+ ### timeout
249
+
250
+ This is also mostly useful for reporting/metrics purposes. What this reflection does is signal that the job was configured to wait (`lock_timeout` was configured), but we couldn't retrieve a lock even though we waited for some time.
251
+
252
+ ### unlock_failed
253
+
254
+ This is not got, this is worth
255
+
256
+ ### unlocked
257
+
258
+ Also mostly useful for reporting purposes. The job was successfully unlocked.
259
+
260
+ ### unknown_sidekiq_worker
261
+
262
+ The reason this happens is that the server couldn't find a valid sidekiq worker class. Most likely, that worker isn't intended to be processed by this sidekiq server instance.
263
+
182
264
  ## Global Configuration
183
265
 
184
266
  The gem supports a few different configuration options that might be of interest if you run into some weird issues.
@@ -683,21 +765,6 @@ class UniqueJobWithFilterMethod
683
765
  end.
684
766
  ```
685
767
 
686
- ### Logging
687
-
688
- To see logging in sidekiq when duplicate payload has been filtered out you can enable on a per worker basis using the sidekiq options. The default value is false
689
-
690
- ```ruby
691
- class UniqueJobWithFilterMethod
692
- include Sidekiq::Worker
693
- sidekiq_options lock: :while_executing,
694
- log_duplicate: true
695
-
696
- ...
697
-
698
- end
699
- ```
700
-
701
768
  ### Cleanup Dead Locks
702
769
 
703
770
  For sidekiq versions before 5.1 a `sidekiq_retries_exhausted` block is required per worker class. This is deprecated in Sidekiq 6.0
@@ -16,6 +16,8 @@ module SidekiqUniqueJobs
16
16
  :reaper_count,
17
17
  :reaper_interval,
18
18
  :reaper_timeout,
19
+ :reaper_resurrector_interval,
20
+ :reaper_resurrector_enabled,
19
21
  :lock_info,
20
22
  :raise_on_config_error,
21
23
  :current_redis_version)
@@ -109,6 +111,14 @@ module SidekiqUniqueJobs
109
111
  #
110
112
  # @return [10] stop reaper after 10 seconds
111
113
  REAPER_TIMEOUT = 10
114
+ #
115
+ # @return [3600] check if reaper is dead each 3600 seconds
116
+ REAPER_RESURRECTOR_INTERVAL = 3600
117
+
118
+ #
119
+ # @return [true] enable reaper resurrector
120
+ REAPER_RESURRECTOR_ENABLED = true
121
+
112
122
  #
113
123
  # @return [false] while useful it also adds overhead so disable lock_info by default
114
124
  USE_LOCK_INFO = false
@@ -178,6 +188,8 @@ module SidekiqUniqueJobs
178
188
  REAPER_COUNT,
179
189
  REAPER_INTERVAL,
180
190
  REAPER_TIMEOUT,
191
+ REAPER_RESURRECTOR_INTERVAL,
192
+ REAPER_RESURRECTOR_ENABLED,
181
193
  USE_LOCK_INFO,
182
194
  RAISE_ON_CONFIG_ERROR,
183
195
  REDIS_VERSION,
@@ -185,26 +197,22 @@ module SidekiqUniqueJobs
185
197
  end
186
198
 
187
199
  def default_lock_ttl=(obj)
188
- warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
189
- " Please use `#{class_name}#lock_ttl=` instead."
200
+ warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. Please use `#{class_name}#lock_ttl=` instead."
190
201
  self.lock_ttl = obj
191
202
  end
192
203
 
193
204
  def default_lock_timeout=(obj)
194
- warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
195
- " Please use `#{class_name}#lock_timeout=` instead."
205
+ warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. Please use `#{class_name}#lock_timeout=` instead."
196
206
  self.lock_timeout = obj
197
207
  end
198
208
 
199
209
  def default_lock_ttl
200
- warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
201
- " Please use `#{class_name}#lock_ttl` instead."
210
+ warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. Please use `#{class_name}#lock_ttl` instead."
202
211
  lock_ttl
203
212
  end
204
213
 
205
214
  def default_lock_timeout
206
- warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
207
- " Please use `#{class_name}#lock_timeout` instead."
215
+ warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. Please use `#{class_name}#lock_timeout` instead."
208
216
  lock_timeout
209
217
  end
210
218
 
@@ -6,49 +6,48 @@
6
6
  # @author Mikael Henriksson <mikael@mhenrixon.com>
7
7
  #
8
8
  module SidekiqUniqueJobs
9
- ARGS = "args"
10
- APARTMENT = "apartment"
11
- AT = "at"
12
- CHANGELOGS = "uniquejobs:changelog"
13
- CLASS = "class"
14
- CREATED_AT = "created_at"
15
- DEAD_VERSION = "uniquejobs:dead"
16
- DIGESTS = "uniquejobs:digests"
17
- ERRORS = "errors"
18
- JID = "jid"
19
- LIMIT = "limit"
20
- LIVE_VERSION = "uniquejobs:live"
21
- LOCK = "lock"
22
- LOCK_ARGS = "lock_args"
23
- LOCK_ARGS_METHOD = "lock_args_method"
24
- LOCK_DIGEST = "lock_digest"
25
- LOCK_EXPIRATION = "lock_expiration"
26
- LOCK_INFO = "lock_info"
27
- LOCK_LIMIT = "lock_limit"
28
- LOCK_PREFIX = "lock_prefix"
29
- LOCK_TIMEOUT = "lock_timeout"
30
- LOCK_TTL = "lock_ttl"
31
- LOCK_TYPE = "lock_type"
32
- LOG_DUPLICATE = "log_duplicate"
33
- ON_CLIENT_CONFLICT = "on_client_conflict"
34
- ON_CONFLICT = "on_conflict"
35
- ON_SERVER_CONFLICT = "on_server_conflict"
36
- PAYLOAD = "payload"
37
- PROCESSES = "processes"
38
- QUEUE = "queue"
39
- RETRY = "retry"
40
- SCHEDULE = "schedule"
41
- TIME = "time"
42
- TIMEOUT = "timeout"
43
- TTL = "ttl"
44
- TYPE = "type"
45
- UNIQUE = "unique"
46
- UNIQUE_ACROSS_QUEUES = "unique_across_queues"
47
- UNIQUE_ACROSS_WORKERS = "unique_across_workers"
48
- UNIQUE_ARGS = "unique_args"
49
- UNIQUE_ARGS_METHOD = "unique_args_method"
50
- UNIQUE_DIGEST = "unique_digest"
51
- UNIQUE_PREFIX = "unique_prefix"
52
- UNIQUE_REAPER = "uniquejobs:reaper"
53
- WORKER = "worker"
9
+ ARGS ||= "args"
10
+ APARTMENT ||= "apartment"
11
+ AT ||= "at"
12
+ CHANGELOGS ||= "uniquejobs:changelog"
13
+ CLASS ||= "class"
14
+ CREATED_AT ||= "created_at"
15
+ DEAD_VERSION ||= "uniquejobs:dead"
16
+ DIGESTS ||= "uniquejobs:digests"
17
+ ERRORS ||= "errors"
18
+ JID ||= "jid"
19
+ LIMIT ||= "limit"
20
+ LIVE_VERSION ||= "uniquejobs:live"
21
+ LOCK ||= "lock"
22
+ LOCK_ARGS ||= "lock_args"
23
+ LOCK_ARGS_METHOD ||= "lock_args_method"
24
+ LOCK_DIGEST ||= "lock_digest"
25
+ LOCK_EXPIRATION ||= "lock_expiration"
26
+ LOCK_INFO ||= "lock_info"
27
+ LOCK_LIMIT ||= "lock_limit"
28
+ LOCK_PREFIX ||= "lock_prefix"
29
+ LOCK_TIMEOUT ||= "lock_timeout"
30
+ LOCK_TTL ||= "lock_ttl"
31
+ LOCK_TYPE ||= "lock_type"
32
+ ON_CLIENT_CONFLICT ||= "on_client_conflict"
33
+ ON_CONFLICT ||= "on_conflict"
34
+ ON_SERVER_CONFLICT ||= "on_server_conflict"
35
+ PAYLOAD ||= "payload"
36
+ PROCESSES ||= "processes"
37
+ QUEUE ||= "queue"
38
+ RETRY ||= "retry"
39
+ SCHEDULE ||= "schedule"
40
+ TIME ||= "time"
41
+ TIMEOUT ||= "timeout"
42
+ TTL ||= "ttl"
43
+ TYPE ||= "type"
44
+ UNIQUE ||= "unique"
45
+ UNIQUE_ACROSS_QUEUES ||= "unique_across_queues"
46
+ UNIQUE_ACROSS_WORKERS ||= "unique_across_workers"
47
+ UNIQUE_ARGS ||= "unique_args"
48
+ UNIQUE_ARGS_METHOD ||= "unique_args_method"
49
+ UNIQUE_DIGEST ||= "unique_digest"
50
+ UNIQUE_PREFIX ||= "unique_prefix"
51
+ UNIQUE_REAPER ||= "uniquejobs:reaper"
52
+ WORKER ||= "worker"
54
53
  end
@@ -0,0 +1,35 @@
1
+ # frozen_string_literal: true
2
+
3
+ module SidekiqUniqueJobs
4
+ #
5
+ # Class Deprecation provides logging of deprecations
6
+ #
7
+ # @author Mikael Henriksson <mikael@mhenrixon.com>
8
+ #
9
+ class Deprecation
10
+ def self.muted
11
+ orig_val = Thread.current[:uniquejobs_mute_deprecations]
12
+ Thread.current[:uniquejobs_mute_deprecations] = true
13
+ yield
14
+ ensure
15
+ Thread.current[:uniquejobs_mute_deprecations] = orig_val
16
+ end
17
+
18
+ def self.muted?
19
+ Thread.current[:uniquejobs_mute_deprecations] == true
20
+ end
21
+
22
+ def self.warn(msg)
23
+ return if SidekiqUniqueJobs::Deprecation.muted?
24
+
25
+ warn "DEPRECATION WARNING: #{msg}"
26
+ end
27
+
28
+ def self.warn_with_backtrace(msg)
29
+ return if SidekiqUniqueJobs::Deprecation.muted?
30
+
31
+ trace = "\n\nCALLED FROM:\n#{caller.join("\n")}"
32
+ warn(msg + trace)
33
+ end
34
+ end
35
+ end
@@ -18,6 +18,15 @@ module SidekiqUniqueJobs
18
18
  end
19
19
  end
20
20
 
21
+ #
22
+ # Raised when no block was given
23
+ #
24
+ class NoBlockGiven < SidekiqUniqueJobs::UniqueJobsError; end
25
+ #
26
+ # Raised when a notification has been mistyped
27
+ #
28
+ class NoSuchNotificationError < UniqueJobsError; end
29
+
21
30
  #
22
31
  # Error raised when trying to add a duplicate lock
23
32
  #
@@ -7,8 +7,16 @@ module SidekiqUniqueJobs
7
7
  # @abstract
8
8
  # @author Mikael Henriksson <mikael@mhenrixon.com>
9
9
  class BaseLock
10
+ extend Forwardable
11
+
12
+ # includes "SidekiqUniqueJobs::Logging"
13
+ # @!parse include SidekiqUniqueJobs::Logging
10
14
  include SidekiqUniqueJobs::Logging
11
15
 
16
+ # includes "SidekiqUniqueJobs::Reflectable"
17
+ # @!parse include SidekiqUniqueJobs::Reflectable
18
+ include SidekiqUniqueJobs::Reflectable
19
+
12
20
  #
13
21
  # Validates that the sidekiq_options for the worker is valid
14
22
  #
@@ -20,6 +28,10 @@ module SidekiqUniqueJobs
20
28
  Validator.validate(options)
21
29
  end
22
30
 
31
+ # NOTE: Mainly used for a clean testing API
32
+ #
33
+ def_delegators :locksmith, :locked?
34
+
23
35
  # @param [Hash] item the Sidekiq job hash
24
36
  # @param [Proc] callback the callback to use after unlock
25
37
  # @param [Sidekiq::RedisConnection, ConnectionPool] redis_pool the redis connection
@@ -41,10 +53,8 @@ module SidekiqUniqueJobs
41
53
  #
42
54
  # @yield to the caller when given a block
43
55
  #
44
- def lock(&block)
45
- return call_strategy unless (locked_token = locksmith.lock(&block))
46
-
47
- locked_token
56
+ def lock
57
+ raise NotImplementedError, "##{__method__} needs to be implemented in #{self.class}"
48
58
  end
49
59
 
50
60
  # Execute the job in the Sidekiq server processor
@@ -53,31 +63,6 @@ module SidekiqUniqueJobs
53
63
  raise NotImplementedError, "##{__method__} needs to be implemented in #{self.class}"
54
64
  end
55
65
 
56
- # Unlocks the job from redis
57
- # @return [String] sidekiq job id when successful
58
- # @return [false] when unsuccessful
59
- def unlock
60
- locksmith.unlock # Only signal to release the lock
61
- end
62
-
63
- # Deletes the job from redis if it is locked.
64
- def delete
65
- locksmith.delete # Soft delete (don't forcefully remove when expiration is set)
66
- end
67
-
68
- # Forcefully deletes the job from redis.
69
- # This is good for jobs when a previous lock was not unlocked
70
- def delete!
71
- locksmith.delete! # Force delete the lock
72
- end
73
-
74
- # Checks if the item has achieved a lock
75
- # @return [true] when this jid has locked the job
76
- # @return [false] when this jid has not locked the job
77
- def locked?
78
- locksmith.locked?
79
- end
80
-
81
66
  #
82
67
  # The lock manager/client
83
68
  #
@@ -90,23 +75,6 @@ module SidekiqUniqueJobs
90
75
 
91
76
  private
92
77
 
93
- def prepare_item
94
- return if item.key?(LOCK_DIGEST)
95
-
96
- # The below should only be done to ease testing
97
- # in production this will be done by the middleware
98
- SidekiqUniqueJobs::Job.prepare(item)
99
- end
100
-
101
- def call_strategy
102
- @attempt += 1
103
- client_strategy.call { lock if replace? }
104
- end
105
-
106
- def replace?
107
- client_strategy.replace? && attempt < 2
108
- end
109
-
110
78
  # @!attribute [r] item
111
79
  # @return [Hash<String, Object>] the Sidekiq job hash
112
80
  attr_reader :item
@@ -123,18 +91,55 @@ module SidekiqUniqueJobs
123
91
  # @return [Integer] the current locking attempt
124
92
  attr_reader :attempt
125
93
 
126
- def unlock_with_callback
127
- return log_warn("Might need to be unlocked manually", item) unless unlock
94
+ def prepare_item
95
+ return if item.key?(LOCK_DIGEST)
128
96
 
129
- callback_safely
130
- item[JID]
97
+ # The below should only be done to ease testing
98
+ # in production this will be done by the middleware
99
+ SidekiqUniqueJobs::Job.prepare(item)
100
+ end
101
+
102
+ #
103
+ # Handle when lock failed
104
+ #
105
+ # @param [Symbol] location: :client or :server
106
+ #
107
+ # @return [void]
108
+ #
109
+ def lock_failed(origin: :client)
110
+ reflect(:lock_failed, item)
111
+ call_strategy(origin: origin)
112
+ end
113
+
114
+ def call_strategy(origin:)
115
+ @attempt += 1
116
+
117
+ case origin
118
+ when :client
119
+ client_strategy.call { lock if replace? }
120
+ when :server
121
+ server_strategy.call { lock if replace? }
122
+ else
123
+ raise SidekiqUniqueJobs::InvalidArgument,
124
+ "either `for: :server` or `for: :client` needs to be specified"
125
+ end
126
+ end
127
+
128
+ def replace?
129
+ client_strategy.replace? && attempt < 2
130
+ end
131
+
132
+ def unlock_and_callback
133
+ return callback_safely if locksmith.unlock
134
+
135
+ reflect(:unlock_failed, item)
131
136
  end
132
137
 
133
138
  def callback_safely
134
139
  callback&.call
135
140
  item[JID]
136
141
  rescue StandardError
137
- log_warn("Unlocked successfully but the #after_unlock callback failed!", item)
142
+ reflect(:after_unlock_callback_failed, item)
138
143
  raise
139
144
  end
140
145
 
@@ -13,30 +13,52 @@ module SidekiqUniqueJobs
13
13
  #
14
14
  # @author Mikael Henriksson <mikael@mhenrixon.com>
15
15
  class UntilAndWhileExecuting < BaseLock
16
+ #
17
+ # Locks a sidekiq job
18
+ #
19
+ # @note Will call a conflict strategy if lock can't be achieved.
20
+ #
21
+ # @return [String, nil] the locked jid when properly locked, else nil.
22
+ #
23
+ # @yield to the caller when given a block
24
+ #
25
+ def lock(origin: :client)
26
+ return lock_failed(origin: origin) unless (token = locksmith.lock)
27
+ return yield token if block_given?
28
+
29
+ token
30
+ end
31
+
16
32
  # Executes in the Sidekiq server process
17
33
  # @yield to the worker class perform method
18
34
  def execute
19
- if unlock
20
- lock_on_failure do
21
- runtime_lock.execute { return yield }
22
- end
35
+ if locksmith.unlock
36
+ # ensure_relocked do
37
+ runtime_lock.execute { return yield }
38
+ # end
23
39
  else
24
- log_warn("Couldn't unlock digest: #{item[LOCK_DIGEST]}, jid: #{item[JID]}")
40
+ reflect(:unlock_failed, item)
25
41
  end
42
+ rescue Exception # rubocop:disable Lint/RescueException
43
+ reflect(:execution_failed, item)
44
+ locksmith.lock(wait: 2)
45
+
46
+ raise
26
47
  end
27
48
 
28
49
  private
29
50
 
30
- def lock_on_failure
51
+ def ensure_relocked
31
52
  yield
32
53
  rescue Exception # rubocop:disable Lint/RescueException
33
- log_error("Runtime lock failed to execute job, restoring server lock", item)
34
- lock
54
+ reflect(:execution_failed, item)
55
+ locksmith.lock
56
+
35
57
  raise
36
58
  end
37
59
 
38
60
  def runtime_lock
39
- @runtime_lock ||= SidekiqUniqueJobs::Lock::WhileExecuting.new(item, callback, redis_pool)
61
+ @runtime_lock ||= SidekiqUniqueJobs::Lock::WhileExecuting.new(item.dup, callback, redis_pool)
40
62
  end
41
63
  end
42
64
  end
@@ -8,16 +8,28 @@ module SidekiqUniqueJobs
8
8
  #
9
9
  # @author Mikael Henriksson <mikael@mhenrixon.com>
10
10
  class UntilExecuted < BaseLock
11
- OK = "OK"
11
+ #
12
+ # Locks a sidekiq job
13
+ #
14
+ # @note Will call a conflict strategy if lock can't be achieved.
15
+ #
16
+ # @return [String, nil] the locked jid when properly locked, else nil.
17
+ #
18
+ # @yield to the caller when given a block
19
+ #
20
+ def lock
21
+ return lock_failed(origin: :client) unless (token = locksmith.lock)
22
+ return yield token if block_given?
23
+
24
+ token
25
+ end
12
26
 
13
27
  # Executes in the Sidekiq server process
14
28
  # @yield to the worker class perform method
15
29
  def execute
16
- lock do
30
+ locksmith.execute do
17
31
  yield
18
- unlock_with_callback
19
- callback_safely
20
- item[JID]
32
+ unlock_and_callback
21
33
  end
22
34
  end
23
35
  end