sidekiq-unique-jobs 7.1.3 → 7.1.8

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq-unique-jobs might be problematic. Click here for more details.

Files changed (32) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +53 -3
  3. data/README.md +3 -2
  4. data/lib/sidekiq_unique_jobs/config.rb +43 -4
  5. data/lib/sidekiq_unique_jobs/constants.rb +44 -44
  6. data/lib/sidekiq_unique_jobs/deprecation.rb +30 -0
  7. data/lib/sidekiq_unique_jobs/digests.rb +3 -6
  8. data/lib/sidekiq_unique_jobs/exceptions.rb +1 -0
  9. data/lib/sidekiq_unique_jobs/json.rb +7 -0
  10. data/lib/sidekiq_unique_jobs/lock/base_lock.rb +30 -22
  11. data/lib/sidekiq_unique_jobs/lock/until_and_while_executing.rb +9 -3
  12. data/lib/sidekiq_unique_jobs/lock/until_executed.rb +9 -3
  13. data/lib/sidekiq_unique_jobs/lock/until_executing.rb +10 -4
  14. data/lib/sidekiq_unique_jobs/lock/until_expired.rb +10 -4
  15. data/lib/sidekiq_unique_jobs/lock/while_executing.rb +6 -4
  16. data/lib/sidekiq_unique_jobs/lock_ttl.rb +1 -1
  17. data/lib/sidekiq_unique_jobs/locksmith.rb +4 -1
  18. data/lib/sidekiq_unique_jobs/logging.rb +9 -0
  19. data/lib/sidekiq_unique_jobs/lua/lock.lua +3 -3
  20. data/lib/sidekiq_unique_jobs/lua/unlock.lua +8 -0
  21. data/lib/sidekiq_unique_jobs/middleware/client.rb +1 -1
  22. data/lib/sidekiq_unique_jobs/options_with_fallback.rb +2 -4
  23. data/lib/sidekiq_unique_jobs/orphans/manager.rb +35 -0
  24. data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +3 -1
  25. data/lib/sidekiq_unique_jobs/reflectable.rb +11 -2
  26. data/lib/sidekiq_unique_jobs/reflections.rb +12 -1
  27. data/lib/sidekiq_unique_jobs/server.rb +13 -1
  28. data/lib/sidekiq_unique_jobs/sidekiq_unique_ext.rb +35 -13
  29. data/lib/sidekiq_unique_jobs/timing.rb +1 -1
  30. data/lib/sidekiq_unique_jobs/version.rb +1 -1
  31. data/lib/tasks/changelog.rake +15 -15
  32. metadata +9 -9
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 57accefc169e11923a88ad34751e2e7c5f2a2c25a8f672084d946d1c536a7a46
4
- data.tar.gz: e019c6288d6b87fb0b306b18e0435095a81a56ab04046ca25485b531c418f692
3
+ metadata.gz: f0525a9235a69f86ef177c23a54afea5977a4a99fe4c09867d88c88d29eeb280
4
+ data.tar.gz: f105453b5082cb2a37ff2484e8c92213ec578a488b5f97f894b7f06e901aca23
5
5
  SHA512:
6
- metadata.gz: 1989a4f0d6a335be4f7b35e97028a32fabd39212dd3f914471fa78c6654cefacc9e2725654299a10b7df5c55f0778566283f7e5e0ba01542de7900995dcfb734
7
- data.tar.gz: d9be5add9dbaac2c523dbb6ed8dfe2879026087132c4812a2a03e4e58945cf481047aff00ed49e78942d8450749fc2864b09b44e6646dd4125452568cbc902ec
6
+ metadata.gz: 24eeed23d326f74ebf0c06d34913c8429d1ce7848bdb384606176bf8362ae74f8e1a3dbd0cffad10fbdac1344fe7a1170804e332f5974713c890f6fb330fae91
7
+ data.tar.gz: 61ce06ebeb3b49f25dbb3ffb12cfe90f164e9c668756e47b81d527181f66e27a07dc836f5230bb1949ca94429c59bdfe2fa7bc13c3dbac3a53ebe1feb2257080
data/CHANGELOG.md CHANGED
@@ -1,13 +1,64 @@
1
1
  # Changelog
2
2
 
3
- ## [Unreleased](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/HEAD)
3
+ ## [v7.0.13](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.13) (2021-09-27)
4
4
 
5
- [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.2...HEAD)
5
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.7...v7.0.13)
6
+
7
+ ## [v7.1.7](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.7) (2021-09-27)
8
+
9
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.6...v7.1.7)
10
+
11
+ **Implemented enhancements:**
12
+
13
+ - Styles [\#642](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/642) ([mhenrixon](https://github.com/mhenrixon))
6
14
 
7
15
  **Fixed bugs:**
8
16
 
17
+ - OnConflict::Replace: yield when lock was achieved [\#640](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/640) ([mhenrixon](https://github.com/mhenrixon))
18
+
19
+ ## [v7.1.6](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.6) (2021-09-21)
20
+
21
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.5...v7.1.6)
22
+
23
+ **Closed issues:**
24
+
25
+ - until\_and\_while\_executing is not running the job at all in sidekiq-unique-jobs 7.1.4 [\#626](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/626)
26
+
27
+ **Merged pull requests:**
28
+
29
+ - Necessary upgrades for Sidekiq v6.2.2 [\#639](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/639) ([mhenrixon](https://github.com/mhenrixon))
30
+ - Tese to these in README.md [\#633](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/633) ([carrickr](https://github.com/carrickr))
31
+
32
+ ## [v7.1.5](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.5) (2021-07-28)
33
+
34
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.4...v7.1.5)
35
+
36
+ **Fixed bugs:**
37
+
38
+ - Fix: UntilAndWhileExecuting [\#627](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/627) ([mhenrixon](https://github.com/mhenrixon))
39
+
40
+ ## [v7.1.4](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.4) (2021-07-21)
41
+
42
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.3...v7.1.4)
43
+
44
+ **Fixed bugs:**
45
+
46
+ - Pass lock timeout to primed\_async [\#624](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/624) ([millerjs](https://github.com/millerjs))
47
+
48
+ ## [v7.1.3](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.3) (2021-07-20)
49
+
50
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.2...v7.1.3)
51
+
52
+ **Fixed bugs:**
53
+
54
+ - Locks are not released: seeing 'Might need to be unlocked manually" warnings [\#594](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/594)
55
+ - Disable resurrector by default [\#623](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/623) ([mhenrixon](https://github.com/mhenrixon))
9
56
  - Documentation fixes [\#622](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/622) ([ursm](https://github.com/ursm))
10
57
 
58
+ **Closed issues:**
59
+
60
+ - Lock until\_and\_while\_executing not working as expected [\#613](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/613)
61
+
11
62
  **Merged pull requests:**
12
63
 
13
64
  - Improve readme [\#621](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/621) ([mhenrixon](https://github.com/mhenrixon))
@@ -90,7 +141,6 @@
90
141
 
91
142
  **Fixed bugs:**
92
143
 
93
- - lock\_info set to true but no lock info showing up in web ui [\#589](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/589)
94
144
  - Fix recording lock\_info [\#599](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/599) ([mhenrixon](https://github.com/mhenrixon))
95
145
 
96
146
  ## [v7.0.8](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.8) (2021-04-14)
data/README.md CHANGED
@@ -92,10 +92,11 @@ This gem adds unique constraints to sidekiq jobs. The uniqueness is achieved by
92
92
 
93
93
  By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen [Conflict Strategy](#conflict-strategy) strategy. Unless a conflict strategy is chosen
94
94
 
95
- This is the documentation for the master branch. You can find the documentation for each release by navigating to its tag.
95
+ This is the documentation for the `main` branch. You can find the documentation for each release by navigating to its tag.
96
96
 
97
97
  Here are links to some of the old versions
98
98
 
99
+ - [v7.0.12](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.12)
99
100
  - [v6.0.25](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.25)
100
101
  - [v5.0.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10)
101
102
  - [v4.0.18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18)
@@ -268,7 +269,7 @@ end
268
269
 
269
270
  ### While Executing
270
271
 
271
- Tese locks are put on a queue without any type of locking mechanism, the locking doesn't happen until Sidekiq pops the job from the queue and starts processing it.
272
+ These locks are put on a queue without any type of locking mechanism, the locking doesn't happen until Sidekiq pops the job from the queue and starts processing it.
272
273
 
273
274
  #### Example worker
274
275
 
@@ -196,26 +196,65 @@ module SidekiqUniqueJobs
196
196
  )
197
197
  end
198
198
 
199
+ #
200
+ # Set the default_lock_ttl
201
+ # @deprecated
202
+ #
203
+ # @param [Integer] obj value to set (seconds)
204
+ #
205
+ # @return [<type>] <description>
206
+ #
199
207
  def default_lock_ttl=(obj)
200
- warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. Please use `#{class_name}#lock_ttl=` instead."
208
+ warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
209
+ " Please use `#{class_name}#lock_ttl=` instead."
201
210
  self.lock_ttl = obj
202
211
  end
203
212
 
213
+ #
214
+ # Set new value for default_lock_timeout
215
+ # @deprecated
216
+ #
217
+ # @param [Integer] obj value to set (seconds)
218
+ #
219
+ # @return [Integer]
220
+ #
204
221
  def default_lock_timeout=(obj)
205
- warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. Please use `#{class_name}#lock_timeout=` instead."
222
+ warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
223
+ " Please use `#{class_name}#lock_timeout=` instead."
206
224
  self.lock_timeout = obj
207
225
  end
208
226
 
227
+ #
228
+ # Default lock TTL (Time To Live)
229
+ # @deprecated
230
+ #
231
+ # @return [nil, Integer] configured value or nil
232
+ #
209
233
  def default_lock_ttl
210
- warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. Please use `#{class_name}#lock_ttl` instead."
234
+ warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
235
+ " Please use `#{class_name}#lock_ttl` instead."
211
236
  lock_ttl
212
237
  end
213
238
 
239
+ #
240
+ # Default Lock Timeout
241
+ # @deprecated
242
+ #
243
+ #
244
+ # @return [nil, Integer] configured value or nil
245
+ #
214
246
  def default_lock_timeout
215
- warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. Please use `#{class_name}#lock_timeout` instead."
247
+ warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
248
+ " Please use `#{class_name}#lock_timeout` instead."
216
249
  lock_timeout
217
250
  end
218
251
 
252
+ #
253
+ # Memoized variable to get the class name
254
+ #
255
+ #
256
+ # @return [String] name of the class
257
+ #
219
258
  def class_name
220
259
  @class_name ||= self.class.name
221
260
  end
@@ -6,48 +6,48 @@
6
6
  # @author Mikael Henriksson <mikael@mhenrixon.com>
7
7
  #
8
8
  module SidekiqUniqueJobs
9
- ARGS ||= "args"
10
- APARTMENT ||= "apartment"
11
- AT ||= "at"
12
- CHANGELOGS ||= "uniquejobs:changelog"
13
- CLASS ||= "class"
14
- CREATED_AT ||= "created_at"
15
- DEAD_VERSION ||= "uniquejobs:dead"
16
- DIGESTS ||= "uniquejobs:digests"
17
- ERRORS ||= "errors"
18
- JID ||= "jid"
19
- LIMIT ||= "limit"
20
- LIVE_VERSION ||= "uniquejobs:live"
21
- LOCK ||= "lock"
22
- LOCK_ARGS ||= "lock_args"
23
- LOCK_ARGS_METHOD ||= "lock_args_method"
24
- LOCK_DIGEST ||= "lock_digest"
25
- LOCK_EXPIRATION ||= "lock_expiration"
26
- LOCK_INFO ||= "lock_info"
27
- LOCK_LIMIT ||= "lock_limit"
28
- LOCK_PREFIX ||= "lock_prefix"
29
- LOCK_TIMEOUT ||= "lock_timeout"
30
- LOCK_TTL ||= "lock_ttl"
31
- LOCK_TYPE ||= "lock_type"
32
- ON_CLIENT_CONFLICT ||= "on_client_conflict"
33
- ON_CONFLICT ||= "on_conflict"
34
- ON_SERVER_CONFLICT ||= "on_server_conflict"
35
- PAYLOAD ||= "payload"
36
- PROCESSES ||= "processes"
37
- QUEUE ||= "queue"
38
- RETRY ||= "retry"
39
- SCHEDULE ||= "schedule"
40
- TIME ||= "time"
41
- TIMEOUT ||= "timeout"
42
- TTL ||= "ttl"
43
- TYPE ||= "type"
44
- UNIQUE ||= "unique"
45
- UNIQUE_ACROSS_QUEUES ||= "unique_across_queues"
46
- UNIQUE_ACROSS_WORKERS ||= "unique_across_workers"
47
- UNIQUE_ARGS ||= "unique_args"
48
- UNIQUE_ARGS_METHOD ||= "unique_args_method"
49
- UNIQUE_DIGEST ||= "unique_digest"
50
- UNIQUE_PREFIX ||= "unique_prefix"
51
- UNIQUE_REAPER ||= "uniquejobs:reaper"
52
- WORKER ||= "worker"
9
+ ARGS = "args"
10
+ APARTMENT = "apartment"
11
+ AT = "at"
12
+ CHANGELOGS = "uniquejobs:changelog"
13
+ CLASS = "class"
14
+ CREATED_AT = "created_at"
15
+ DEAD_VERSION = "uniquejobs:dead"
16
+ DIGESTS = "uniquejobs:digests"
17
+ ERRORS = "errors"
18
+ JID = "jid"
19
+ LIMIT = "limit"
20
+ LIVE_VERSION = "uniquejobs:live"
21
+ LOCK = "lock"
22
+ LOCK_ARGS = "lock_args"
23
+ LOCK_ARGS_METHOD = "lock_args_method"
24
+ LOCK_DIGEST = "lock_digest"
25
+ LOCK_EXPIRATION = "lock_expiration"
26
+ LOCK_INFO = "lock_info"
27
+ LOCK_LIMIT = "lock_limit"
28
+ LOCK_PREFIX = "lock_prefix"
29
+ LOCK_TIMEOUT = "lock_timeout"
30
+ LOCK_TTL = "lock_ttl"
31
+ LOCK_TYPE = "lock_type"
32
+ ON_CLIENT_CONFLICT = "on_client_conflict"
33
+ ON_CONFLICT = "on_conflict"
34
+ ON_SERVER_CONFLICT = "on_server_conflict"
35
+ PAYLOAD = "payload"
36
+ PROCESSES = "processes"
37
+ QUEUE = "queue"
38
+ RETRY = "retry"
39
+ SCHEDULE = "schedule"
40
+ TIME = "time"
41
+ TIMEOUT = "timeout"
42
+ TTL = "ttl"
43
+ TYPE = "type"
44
+ UNIQUE = "unique"
45
+ UNIQUE_ACROSS_QUEUES = "unique_across_queues"
46
+ UNIQUE_ACROSS_WORKERS = "unique_across_workers"
47
+ UNIQUE_ARGS = "unique_args"
48
+ UNIQUE_ARGS_METHOD = "unique_args_method"
49
+ UNIQUE_DIGEST = "unique_digest"
50
+ UNIQUE_PREFIX = "unique_prefix"
51
+ UNIQUE_REAPER = "uniquejobs:reaper"
52
+ WORKER = "worker"
53
53
  end
@@ -7,6 +7,13 @@ module SidekiqUniqueJobs
7
7
  # @author Mikael Henriksson <mikael@mhenrixon.com>
8
8
  #
9
9
  class Deprecation
10
+ #
11
+ # Mute warnings from this gem in a threaded context
12
+ #
13
+ #
14
+ # @return [void] <description>
15
+ #
16
+ # @yieldreturn [void]
10
17
  def self.muted
11
18
  orig_val = Thread.current[:uniquejobs_mute_deprecations]
12
19
  Thread.current[:uniquejobs_mute_deprecations] = true
@@ -15,21 +22,44 @@ module SidekiqUniqueJobs
15
22
  Thread.current[:uniquejobs_mute_deprecations] = orig_val
16
23
  end
17
24
 
25
+ #
26
+ # Check if deprecation warnings have been muted
27
+ #
28
+ #
29
+ # @return [true,false]
30
+ #
18
31
  def self.muted?
19
32
  Thread.current[:uniquejobs_mute_deprecations] == true
20
33
  end
21
34
 
35
+ #
36
+ # Warn about deprecation
37
+ #
38
+ # @param [String] msg a descriptive reason for why the deprecation
39
+ #
40
+ # @return [void]
41
+ #
22
42
  def self.warn(msg)
23
43
  return if SidekiqUniqueJobs::Deprecation.muted?
24
44
 
25
45
  warn "DEPRECATION WARNING: #{msg}"
46
+ nil
26
47
  end
27
48
 
49
+ #
50
+ # Warn about deprecation and provide a context
51
+ #
52
+ # @param [String] msg a descriptive reason for why the deprecation
53
+ #
54
+ # @return [void]
55
+ #
28
56
  def self.warn_with_backtrace(msg)
29
57
  return if SidekiqUniqueJobs::Deprecation.muted?
30
58
 
31
59
  trace = "\n\nCALLED FROM:\n#{caller.join("\n")}"
32
60
  warn(msg + trace)
61
+
62
+ nil
33
63
  end
34
64
  end
35
65
  end
@@ -31,7 +31,8 @@ module SidekiqUniqueJobs
31
31
  #
32
32
  # @param [String] pattern a key pattern to match with
33
33
  # @param [Integer] count the maximum number
34
- # @return [Array<String>] with unique digests
34
+ # @return [Hash<String,Float>] Hash mapping of digest matching the given pattern and score
35
+
35
36
  def delete_by_pattern(pattern, count: DEFAULT_COUNT)
36
37
  result, elapsed = timed do
37
38
  digests = entries(pattern: pattern, count: count).keys
@@ -80,11 +81,7 @@ module SidekiqUniqueJobs
80
81
  options[:match] = pattern
81
82
  options[:count] = count
82
83
 
83
- result = redis { |conn| conn.zscan_each(key, **options).to_a }
84
-
85
- result.each_with_object({}) do |entry, hash|
86
- hash[entry[0]] = entry[1]
87
- end
84
+ redis { |conn| conn.zscan_each(key, **options).to_a }.to_h
88
85
  end
89
86
 
90
87
  #
@@ -22,6 +22,7 @@ module SidekiqUniqueJobs
22
22
  # Raised when no block was given
23
23
  #
24
24
  class NoBlockGiven < SidekiqUniqueJobs::UniqueJobsError; end
25
+
25
26
  #
26
27
  # Raised when a notification has been mistyped
27
28
  #
@@ -20,6 +20,13 @@ module SidekiqUniqueJobs
20
20
  ::JSON.parse(string)
21
21
  end
22
22
 
23
+ #
24
+ # Prevents trying JSON.load from raising errors given argument is a hash
25
+ #
26
+ # @param [String, Hash] string the JSON string to parse
27
+ #
28
+ # @return [Hash,Array]
29
+ #
23
30
  def safe_load_json(string)
24
31
  return string if string.is_a?(Hash)
25
32
 
@@ -91,6 +91,13 @@ module SidekiqUniqueJobs
91
91
  # @return [Integer] the current locking attempt
92
92
  attr_reader :attempt
93
93
 
94
+ #
95
+ # Eases testing by allowing the lock implementation to add the missing
96
+ # keys to the job hash.
97
+ #
98
+ #
99
+ # @return [void] the return value should be irrelevant
100
+ #
94
101
  def prepare_item
95
102
  return if item.key?(LOCK_DIGEST)
96
103
 
@@ -100,33 +107,22 @@ module SidekiqUniqueJobs
100
107
  end
101
108
 
102
109
  #
103
- # Handle when lock failed
110
+ # Call whatever strategry that has been configured
104
111
  #
105
- # @param [Symbol] location: :client or :server
112
+ # @param [Symbol] origin: the origin `:client` or `:server`
106
113
  #
107
- # @return [void]
114
+ # @return [void] the return value is irrelevant
108
115
  #
109
- def lock_failed(origin: :client)
110
- reflect(:lock_failed, item)
111
- call_strategy(origin: origin)
112
- end
113
-
116
+ # @yieldparam [void] if a new job id was set and a block is given
117
+ # @yieldreturn [void] the yield is irrelevant, it only provides a mechanism in
118
+ # one specific situation to yield back to the middleware.
114
119
  def call_strategy(origin:)
115
- @attempt += 1
120
+ new_job_id = nil
121
+ strategy = strategy_for(origin)
122
+ @attempt += 1
116
123
 
117
- case origin
118
- when :client
119
- client_strategy.call { lock if replace? }
120
- when :server
121
- server_strategy.call { lock if replace? }
122
- else
123
- raise SidekiqUniqueJobs::InvalidArgument,
124
- "either `for: :server` or `for: :client` needs to be specified"
125
- end
126
- end
127
-
128
- def replace?
129
- client_strategy.replace? && attempt < 2
124
+ strategy.call { new_job_id = lock if strategy.replace? && @attempt < 2 }
125
+ yield if new_job_id && block_given?
130
126
  end
131
127
 
132
128
  def unlock_and_callback
@@ -143,6 +139,18 @@ module SidekiqUniqueJobs
143
139
  raise
144
140
  end
145
141
 
142
+ def strategy_for(origin)
143
+ case origin
144
+ when :client
145
+ client_strategy
146
+ when :server
147
+ server_strategy
148
+ else
149
+ raise SidekiqUniqueJobs::InvalidArgument,
150
+ "#origin needs to be either `:server` or `:client`"
151
+ end
152
+ end
153
+
146
154
  def client_strategy
147
155
  @client_strategy ||=
148
156
  OnConflict.find_strategy(lock_config.on_client_conflict).new(item, redis_pool)
@@ -22,9 +22,15 @@ module SidekiqUniqueJobs
22
22
  #
23
23
  # @yield to the caller when given a block
24
24
  #
25
- def lock(origin: :client)
26
- return lock_failed(origin: origin) unless (token = locksmith.lock)
27
- return yield token if block_given?
25
+ def lock(origin: :client, &block)
26
+ unless (token = locksmith.lock)
27
+ reflect(:lock_failed, item)
28
+ call_strategy(origin: origin, &block)
29
+
30
+ return
31
+ end
32
+
33
+ yield if block
28
34
 
29
35
  token
30
36
  end
@@ -17,9 +17,15 @@ module SidekiqUniqueJobs
17
17
  #
18
18
  # @yield to the caller when given a block
19
19
  #
20
- def lock
21
- return lock_failed(origin: :client) unless (token = locksmith.lock)
22
- return yield token if block_given?
20
+ def lock(&block)
21
+ unless (token = locksmith.lock)
22
+ reflect(:lock_failed, item)
23
+ call_strategy(origin: :client, &block)
24
+
25
+ return
26
+ end
27
+
28
+ yield if block
23
29
 
24
30
  token
25
31
  end
@@ -15,11 +15,17 @@ module SidekiqUniqueJobs
15
15
  #
16
16
  # @return [String, nil] the locked jid when properly locked, else nil.
17
17
  #
18
- def lock
19
- return lock_failed unless (job_id = locksmith.lock)
20
- return yield job_id if block_given?
18
+ def lock(&block)
19
+ unless (token = locksmith.lock)
20
+ reflect(:lock_failed, item)
21
+ call_strategy(origin: :client, &block)
21
22
 
22
- job_id
23
+ return
24
+ end
25
+
26
+ yield if block
27
+
28
+ token
23
29
  end
24
30
 
25
31
  # Executes in the Sidekiq server process
@@ -17,11 +17,17 @@ module SidekiqUniqueJobs
17
17
  #
18
18
  # @yield to the caller when given a block
19
19
  #
20
- def lock
21
- return lock_failed unless (job_id = locksmith.lock)
22
- return yield job_id if block_given?
20
+ def lock(&block)
21
+ unless (token = locksmith.lock)
22
+ reflect(:lock_failed, item)
23
+ call_strategy(origin: :client, &block)
23
24
 
24
- job_id
25
+ return
26
+ end
27
+
28
+ yield if block
29
+
30
+ token
25
31
  end
26
32
 
27
33
  # Executes in the Sidekiq server process
@@ -11,7 +11,7 @@ module SidekiqUniqueJobs
11
11
  #
12
12
  # @author Mikael Henriksson <mikael@mhenrixon.com>
13
13
  class WhileExecuting < BaseLock
14
- RUN_SUFFIX ||= ":RUN"
14
+ RUN_SUFFIX = ":RUN"
15
15
 
16
16
  include SidekiqUniqueJobs::OptionsWithFallback
17
17
  include SidekiqUniqueJobs::Logging::Middleware
@@ -30,7 +30,7 @@ module SidekiqUniqueJobs
30
30
  # @return [true] always returns true
31
31
  def lock
32
32
  job_id = item[JID]
33
- yield job_id if block_given?
33
+ yield if block_given?
34
34
 
35
35
  job_id
36
36
  end
@@ -38,14 +38,16 @@ module SidekiqUniqueJobs
38
38
  # Executes in the Sidekiq server process.
39
39
  # These jobs are locked in the server process not from the client
40
40
  # @yield to the worker class perform method
41
- def execute
41
+ def execute(&block)
42
42
  with_logging_context do
43
- call_strategy(origin: :server) unless locksmith.execute do
43
+ executed = locksmith.execute do
44
44
  yield
45
45
  callback_safely if locksmith.unlock
46
46
  ensure
47
47
  locksmith.unlock
48
48
  end
49
+
50
+ call_strategy(origin: :server, &block) unless executed
49
51
  end
50
52
  end
51
53
 
@@ -71,7 +71,7 @@ module SidekiqUniqueJobs
71
71
  ttl ||= item[LOCK_EXPIRATION] # TODO: Deprecate at some point
72
72
  ttl ||= worker_options[LOCK_EXPIRATION] # TODO: Deprecate at some point
73
73
  ttl ||= SidekiqUniqueJobs.config.lock_ttl
74
- ttl && ttl.to_i + time_until_scheduled
74
+ ttl && (ttl.to_i + time_until_scheduled)
75
75
  end
76
76
  end
77
77
  end
@@ -179,6 +179,9 @@ module SidekiqUniqueJobs
179
179
  #
180
180
  # Used to reduce some duplication from the two methods
181
181
  #
182
+ # @see lock
183
+ # @see execute
184
+ #
182
185
  # @param [Sidekiq::RedisConnection, ConnectionPool] conn the redis connection
183
186
  # @param [Method] primed_method reference to the method to use for getting a primed token
184
187
  #
@@ -238,7 +241,7 @@ module SidekiqUniqueJobs
238
241
  def primed_async(conn, wait = nil, &block)
239
242
  primed_jid = Concurrent::Promises
240
243
  .future(conn) { |red_con| pop_queued(red_con, wait) }
241
- .value(add_drift(wait || config.ttl))
244
+ .value(add_drift(wait || config.timeout))
242
245
 
243
246
  handle_primed(primed_jid, &block)
244
247
  end
@@ -92,9 +92,18 @@ module SidekiqUniqueJobs
92
92
  def log_fatal(message_or_exception = nil, item = nil, &block)
93
93
  message = build_message(message_or_exception, item)
94
94
  logger.fatal(message, &block)
95
+
95
96
  nil
96
97
  end
97
98
 
99
+ #
100
+ # Build a log message
101
+ #
102
+ # @param [String, Exception] message_or_exception an entry to log
103
+ # @param [Hash] item the sidekiq job hash
104
+ #
105
+ # @return [String] a complete log entry
106
+ #
98
107
  def build_message(message_or_exception, item = nil)
99
108
  return nil if message_or_exception.nil?
100
109
  return message_or_exception if item.nil?
@@ -76,6 +76,9 @@ if pttl and pttl > 0 then
76
76
 
77
77
  log_debug("PEXPIRE", locked, pttl)
78
78
  redis.call("PEXPIRE", locked, pttl)
79
+
80
+ log_debug("PEXPIRE", info, pttl)
81
+ redis.call("PEXPIRE", info, pttl)
79
82
  end
80
83
 
81
84
  log_debug("PEXPIRE", queued, 1000)
@@ -84,9 +87,6 @@ redis.call("PEXPIRE", queued, 1000)
84
87
  log_debug("PEXPIRE", primed, 1000)
85
88
  redis.call("PEXPIRE", primed, 1000)
86
89
 
87
- log_debug("PEXPIRE", info, 1000)
88
- redis.call("PEXPIRE", info, 1000)
89
-
90
90
  log("Locked")
91
91
  log_debug("END lock digest:", digest, "job_id:", job_id)
92
92
  return job_id
@@ -76,6 +76,7 @@ local del_cmd = "DEL"
76
76
  if tonumber(redis_version["major"]) >= 4 then del_cmd = "UNLINK"; end
77
77
 
78
78
  if lock_type ~= "until_expired" then
79
+
79
80
  log_debug(del_cmd, digest, info)
80
81
  redis.call(del_cmd, digest, info)
81
82
 
@@ -83,6 +84,13 @@ if lock_type ~= "until_expired" then
83
84
  redis.call("HDEL", locked, job_id)
84
85
  end
85
86
 
87
+ local locked_count = redis.call("HLEN", locked)
88
+
89
+ if tonumber(locked_count) < 1 then
90
+ log_debug(del_cmd, locked)
91
+ redis.call(del_cmd, locked)
92
+ end
93
+
86
94
  log_debug("LPUSH", queued, "1")
87
95
  redis.call("LPUSH", queued, "1")
88
96
 
@@ -30,7 +30,7 @@ module SidekiqUniqueJobs
30
30
  private
31
31
 
32
32
  def lock
33
- lock_instance.lock do |_locked_jid|
33
+ lock_instance.lock do
34
34
  reflect(:locked, item)
35
35
  return yield
36
36
  end
@@ -46,10 +46,8 @@ module SidekiqUniqueJobs
46
46
  # @return [Class]
47
47
  #
48
48
  def lock_class
49
- @lock_class ||= begin
50
- locks.fetch(lock_type.to_sym) do
51
- raise UnknownLock, "No implementation for `lock: :#{lock_type}`"
52
- end
49
+ @lock_class ||= locks.fetch(lock_type.to_sym) do
50
+ raise UnknownLock, "No implementation for `lock: :#{lock_type}`"
53
51
  end
54
52
  end
55
53
 
@@ -10,10 +10,18 @@ module SidekiqUniqueJobs
10
10
  module Manager
11
11
  module_function
12
12
 
13
+ #
14
+ # @return [Float] the amount to add to the reaper interval
13
15
  DRIFT_FACTOR = 0.02
16
+ #
17
+ # @return [Symbol] allowed reapers (:ruby or :lua)
14
18
  REAPERS = [:ruby, :lua].freeze
15
19
 
20
+ # includes "SidekiqUniqueJobs::Connection"
21
+ # @!parse include SidekiqUniqueJobs::Connection
16
22
  include SidekiqUniqueJobs::Connection
23
+ # includes "SidekiqUniqueJobs::Logging"
24
+ # @!parse include SidekiqUniqueJobs::Logging
17
25
  include SidekiqUniqueJobs::Logging
18
26
 
19
27
  #
@@ -65,6 +73,12 @@ module SidekiqUniqueJobs
65
73
  @task ||= default_task
66
74
  end
67
75
 
76
+ #
77
+ # A properly configured timer task
78
+ #
79
+ #
80
+ # @return [SidekiqUniqueJobs::TimerTask]
81
+ #
68
82
  def default_task
69
83
  SidekiqUniqueJobs::TimerTask.new(timer_task_options) do
70
84
  with_logging_context do
@@ -76,6 +90,13 @@ module SidekiqUniqueJobs
76
90
  end
77
91
  end
78
92
 
93
+ #
94
+ # Store a task to use for scheduled execution
95
+ #
96
+ # @param [SidekiqUniqueJobs::TimerTask] task the task to use
97
+ #
98
+ # @return [void]
99
+ #
79
100
  def task=(task)
80
101
  @task = task
81
102
  end
@@ -201,10 +222,24 @@ module SidekiqUniqueJobs
201
222
  redis { |conn| conn.del(UNIQUE_REAPER) }
202
223
  end
203
224
 
225
+ #
226
+ # Reaper interval with a little drift
227
+ # Redis isn't exact enough so to give a little bufffer,
228
+ # we add a tiny value to the reaper interval.
229
+ #
230
+ #
231
+ # @return [Integer] <description>
232
+ #
204
233
  def drift_reaper_interval
205
234
  reaper_interval + (reaper_interval * DRIFT_FACTOR).to_i
206
235
  end
207
236
 
237
+ #
238
+ # Current time (as integer value)
239
+ #
240
+ #
241
+ # @return [Integer]
242
+ #
208
243
  def current_timestamp
209
244
  Time.now.to_i
210
245
  end
@@ -10,6 +10,8 @@ module SidekiqUniqueJobs
10
10
  # @author Mikael Henriksson <mikael@mhenrixon.com>
11
11
  #
12
12
  class RubyReaper < Reaper
13
+ #
14
+ # @return [String] the suffix for :RUN locks
13
15
  RUN_SUFFIX = ":RUN"
14
16
  #
15
17
  # @!attribute [r] digests
@@ -182,7 +184,7 @@ module SidekiqUniqueJobs
182
184
  page_size = 50
183
185
 
184
186
  loop do
185
- range_start = page * page_size - deleted_size
187
+ range_start = (page * page_size) - deleted_size
186
188
  range_end = range_start + page_size - 1
187
189
  entries = conn.lrange(queue_key, range_start, range_end)
188
190
  page += 1
@@ -7,11 +7,20 @@ module SidekiqUniqueJobs
7
7
  # @author Mikael Henriksson <mikael@mhenrixon.com>
8
8
  #
9
9
  module Reflectable
10
- def reflect(name, *args)
11
- SidekiqUniqueJobs.reflections.dispatch(name, *args)
10
+ #
11
+ # Reflects on specific event
12
+ #
13
+ # @param [Symbol] reflection the reflected event
14
+ # @param [Array] args arguments to provide to reflector
15
+ #
16
+ # @return [void]
17
+ #
18
+ def reflect(reflection, *args)
19
+ SidekiqUniqueJobs.reflections.dispatch(reflection, *args)
12
20
  nil
13
21
  rescue UniqueJobsError => ex
14
22
  SidekiqUniqueJobs.logger.error(ex)
23
+ nil
15
24
  end
16
25
  end
17
26
  end
@@ -42,6 +42,14 @@ module SidekiqUniqueJobs
42
42
  @reflections = {}
43
43
  end
44
44
 
45
+ #
46
+ # Dispatch a reflected event
47
+ #
48
+ # @param [reflection] reflection the reflected event
49
+ # @param [Array] args the arguments to provide to the block
50
+ #
51
+ # @return [void] <description>
52
+ #
45
53
  def dispatch(reflection, *args)
46
54
  if (block = @reflections[reflection])
47
55
  block.call(*args)
@@ -49,12 +57,15 @@ module SidekiqUniqueJobs
49
57
  if DEPRECATIONS.key?(reflection)
50
58
  replacement, removal_version = DEPRECATIONS[reflection]
51
59
  SidekiqUniqueJobs::Deprecation.warn(
52
- "#{reflection} is deprecated and will be removed in version #{removal_version}. Use #{replacement} instead.",
60
+ "#{reflection} is deprecated and will be removed in version #{removal_version}." \
61
+ " Use #{replacement} instead.",
53
62
  )
54
63
  end
55
64
  elsif misconfigured?(reflection)
56
65
  raise NoSuchNotificationError, reflection
57
66
  end
67
+
68
+ nil
58
69
  end
59
70
 
60
71
  def configured?(reflection)
@@ -5,7 +5,7 @@ module SidekiqUniqueJobs
5
5
  #
6
6
  # @author Mikael Henriksson <mikael@mhenrixon.com>
7
7
  class Server
8
- DEATH_HANDLER ||= (lambda do |job, _ex|
8
+ DEATH_HANDLER = (lambda do |job, _ex|
9
9
  return unless (digest = job["lock_digest"])
10
10
 
11
11
  SidekiqUniqueJobs::Digests.new.delete_by_digest(digest)
@@ -25,6 +25,12 @@ module SidekiqUniqueJobs
25
25
  config.death_handlers << death_handler
26
26
  end
27
27
 
28
+ #
29
+ # Start the sidekiq unique jobs server process
30
+ #
31
+ #
32
+ # @return [void]
33
+ #
28
34
  def self.start
29
35
  SidekiqUniqueJobs::UpdateVersion.call
30
36
  SidekiqUniqueJobs::UpgradeLocks.call
@@ -32,6 +38,12 @@ module SidekiqUniqueJobs
32
38
  SidekiqUniqueJobs::Orphans::ReaperResurrector.start
33
39
  end
34
40
 
41
+ #
42
+ # Stop the sidekiq unique jobs server process
43
+ #
44
+ #
45
+ # @return [void]
46
+ #
35
47
  def self.stop
36
48
  SidekiqUniqueJobs::Orphans::Manager.stop
37
49
  end
@@ -68,24 +68,46 @@ module Sidekiq
68
68
  prepend UniqueExtension
69
69
  end
70
70
 
71
- # See Sidekiq::Api
72
- class Job
73
- #
74
- # Provides extensions for unlocking jobs that are removed and deleted
75
- #
76
- # @author Mikael Henriksson <mikael@mhenrixon.com>
77
- #
78
- module UniqueExtension
71
+ if Sidekiq.const_defined?("JobRecord")
72
+ # See Sidekiq::Api
73
+ class JobRecord
79
74
  #
80
- # Wraps the original method to ensure locks for the job are deleted
75
+ # Provides extensions for unlocking jobs that are removed and deleted
81
76
  #
82
- def delete
83
- SidekiqUniqueJobs::Unlockable.delete!(item)
84
- super
77
+ # @author Mikael Henriksson <mikael@mhenrixon.com>
78
+ #
79
+ module UniqueExtension
80
+ #
81
+ # Wraps the original method to ensure locks for the job are deleted
82
+ #
83
+ def delete
84
+ SidekiqUniqueJobs::Unlockable.delete!(item)
85
+ super
86
+ end
85
87
  end
88
+
89
+ prepend UniqueExtension
86
90
  end
91
+ else
92
+ # See Sidekiq::Api
93
+ class Job
94
+ #
95
+ # Provides extensions for unlocking jobs that are removed and deleted
96
+ #
97
+ # @author Mikael Henriksson <mikael@mhenrixon.com>
98
+ #
99
+ module UniqueExtension
100
+ #
101
+ # Wraps the original method to ensure locks for the job are deleted
102
+ #
103
+ def delete
104
+ SidekiqUniqueJobs::Unlockable.delete!(item)
105
+ super
106
+ end
107
+ end
87
108
 
88
- prepend UniqueExtension
109
+ prepend UniqueExtension
110
+ end
89
111
  end
90
112
 
91
113
  # See Sidekiq::Api
@@ -51,7 +51,7 @@ module SidekiqUniqueJobs
51
51
  if Process.const_defined?("CLOCK_MONOTONIC")
52
52
  Process.clock_gettime(Process::CLOCK_MONOTONIC)
53
53
  else
54
- Time.now.to_f
54
+ now_f
55
55
  end
56
56
  end
57
57
  end
@@ -3,5 +3,5 @@
3
3
  module SidekiqUniqueJobs
4
4
  #
5
5
  # @return [String] the current SidekiqUniqueJobs version
6
- VERSION = "7.1.3"
6
+ VERSION = "7.1.8"
7
7
  end
@@ -1,22 +1,22 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ # rubocop:disable Style/MutableConstant
4
+ CHANGELOG_CMD = %w[
5
+ github_changelog_generator
6
+ --no-verbose
7
+ --user
8
+ mhenrixon
9
+ --project
10
+ sidekiq-unique-jobs
11
+ --token
12
+ ]
13
+ ADD_CHANGELOG_CMD = "git add --all"
14
+ COMMIT_CHANGELOG_CMD = "git commit -a -m 'Update changelog'"
15
+ # rubocop:enable Style/MutableConstant
16
+
3
17
  desc "Generate a Changelog"
4
18
  task :changelog do
5
- # rubocop:disable Style/MutableConstant
6
- CHANGELOG_CMD ||= %w[
7
- github_changelog_generator
8
- --no-verbose
9
- --user
10
- mhenrixon
11
- --project
12
- sidekiq-unique-jobs
13
- --token
14
- ]
15
- ADD_CHANGELOG_CMD ||= "git add --all"
16
- COMMIT_CHANGELOG_CMD ||= "git commit -a -m 'Update changelog'"
17
- # rubocop:enable Style/MutableConstant
18
-
19
- sh("git checkout master")
19
+ sh("git checkout main")
20
20
  sh(*CHANGELOG_CMD.push(ENV["CHANGELOG_GITHUB_TOKEN"]))
21
21
  sh(ADD_CHANGELOG_CMD)
22
22
  sh(COMMIT_CHANGELOG_CMD)
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: sidekiq-unique-jobs
3
3
  version: !ruby/object:Gem::Version
4
- version: 7.1.3
4
+ version: 7.1.8
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mikael Henriksson
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2021-07-20 00:00:00.000000000 Z
11
+ date: 2021-10-08 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: brpoplpush-redis_script
@@ -59,7 +59,7 @@ dependencies:
59
59
  version: '5.0'
60
60
  - - "<"
61
61
  - !ruby/object:Gem::Version
62
- version: '7.0'
62
+ version: '8.0'
63
63
  type: :runtime
64
64
  prerelease: false
65
65
  version_requirements: !ruby/object:Gem::Requirement
@@ -69,7 +69,7 @@ dependencies:
69
69
  version: '5.0'
70
70
  - - "<"
71
71
  - !ruby/object:Gem::Version
72
- version: '7.0'
72
+ version: '8.0'
73
73
  - !ruby/object:Gem::Dependency
74
74
  name: thor
75
75
  requirement: !ruby/object:Gem::Requirement
@@ -79,7 +79,7 @@ dependencies:
79
79
  version: '0.20'
80
80
  - - "<"
81
81
  - !ruby/object:Gem::Version
82
- version: '2.0'
82
+ version: '3.0'
83
83
  type: :runtime
84
84
  prerelease: false
85
85
  version_requirements: !ruby/object:Gem::Requirement
@@ -89,7 +89,7 @@ dependencies:
89
89
  version: '0.20'
90
90
  - - "<"
91
91
  - !ruby/object:Gem::Version
92
- version: '2.0'
92
+ version: '3.0'
93
93
  description: |
94
94
  Prevents simultaneous Sidekiq jobs with the same unique arguments to run.
95
95
  Highly configurable to suite your specific needs.
@@ -223,7 +223,7 @@ metadata:
223
223
  post_install_message: |
224
224
  IMPORTANT!
225
225
 
226
- Automatic configuration of the sidekiq middelware is no longer done.
226
+ Automatic configuration of the sidekiq middleware is no longer done.
227
227
  Please see: https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/README.md#add-the-middleware
228
228
 
229
229
  This version deprecated the following sidekiq_options
@@ -254,14 +254,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
254
254
  requirements:
255
255
  - - ">="
256
256
  - !ruby/object:Gem::Version
257
- version: 2.5.0
257
+ version: '2.5'
258
258
  required_rubygems_version: !ruby/object:Gem::Requirement
259
259
  requirements:
260
260
  - - ">="
261
261
  - !ruby/object:Gem::Version
262
262
  version: '0'
263
263
  requirements: []
264
- rubygems_version: 3.0.3.1
264
+ rubygems_version: 3.2.28
265
265
  signing_key:
266
266
  specification_version: 4
267
267
  summary: Sidekiq middleware that prevents duplicates jobs