sidekiq-unique-jobs 7.1.1 → 7.1.6

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq-unique-jobs might be problematic. Click here for more details.

data/README.md CHANGED
@@ -2,6 +2,10 @@
2
2
 
3
3
  [![Join the chat at https://gitter.im/mhenrixon/sidekiq-unique-jobs](https://badges.gitter.im/mhenrixon/sidekiq-unique-jobs.svg)](https://gitter.im/mhenrixon/sidekiq-unique-jobs?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) ![Build Status](https://github.com/mhenrixon/sidekiq-unique-jobs/actions/workflows/rspec.yml/badge.svg?branch=master) [![Code Climate](https://codeclimate.com/github/mhenrixon/sidekiq-unique-jobs.svg)](https://codeclimate.com/github/mhenrixon/sidekiq-unique-jobs) [![Test Coverage](https://codeclimate.com/github/mhenrixon/sidekiq-unique-jobs/badges/coverage.svg)](https://codeclimate.com/github/mhenrixon/sidekiq-unique-jobs/coverage)
4
4
 
5
+ ## Support Me
6
+
7
+ Want to show me some ❤️ for the hard work I do on this gem? You can use the following PayPal link: [https://paypal.me/mhenrixon1](https://paypal.me/mhenrixon1). Any amount is welcome and let me tell you it feels good to be appreciated. Even a dollar makes me super excited about all of this.
8
+
5
9
  <!-- MarkdownTOC -->
6
10
 
7
11
  - [Introduction](#introduction)
@@ -9,47 +13,18 @@
9
13
  - [Installation](#installation)
10
14
  - [Add the middleware](#add-the-middleware)
11
15
  - [Your first worker](#your-first-worker)
12
- - [Support Me](#support-me)
13
16
  - [Requirements](#requirements)
14
- - [General Information](#general-information)
15
- - [Reflections \(metrics, logging, etc.\)](#reflections-metrics-logging-etc)
16
- - [after_unlock_callback_failed](#after_unlock_callback_failed)
17
- - [error](#error)
18
- - [execution_failed](#execution_failed)
19
- - [lock_failed](#lock_failed)
20
- - [locked](#locked)
21
- - [reschedule_failed](#reschedule_failed)
22
- - [rescheduled](#rescheduled)
23
- - [timeout](#timeout)
24
- - [unlock_failed](#unlock_failed)
25
- - [unlocked](#unlocked)
26
- - [unknown_sidekiq_worker](#unknown_sidekiq_worker)
27
- - [Global Configuration](#global-configuration)
28
- - [debug_lua](#debug_lua)
29
- - [lock_timeout](#lock_timeout)
30
- - [lock_ttl](#lock_ttl)
31
- - [enabled](#enabled)
32
- - [logger](#logger)
33
- - [max_history](#max_history)
34
- - [reaper](#reaper)
35
- - [reaper_count](#reaper_count)
36
- - [reaper_interval](#reaper_interval)
37
- - [reaper_timeout](#reaper_timeout)
38
- - [lock_prefix](#lock_prefix)
39
- - [lock_info](#lock_info)
40
- - [Worker Configuration](#worker-configuration)
41
- - [lock_info](#lock_info-1)
42
- - [lock_prefix](#lock_prefix-1)
43
- - [lock_ttl](#lock_ttl-1)
44
- - [lock_timeout](#lock_timeout-1)
45
- - [unique_across_queues](#unique_across_queues)
46
- - [unique_across_workers](#unique_across_workers)
47
17
  - [Locks](#locks)
48
18
  - [Until Executing](#until-executing)
19
+ - [Example worker](#example-worker)
49
20
  - [Until Executed](#until-executed)
21
+ - [Example worker](#example-worker-1)
50
22
  - [Until Expired](#until-expired)
23
+ - [Example worker](#example-worker-2)
51
24
  - [Until And While Executing](#until-and-while-executing)
25
+ - [Example worker](#example-worker-3)
52
26
  - [While Executing](#while-executing)
27
+ - [Example worker](#example-worker-4)
53
28
  - [Custom Locks](#custom-locks)
54
29
  - [Conflict Strategy](#conflict-strategy)
55
30
  - [log](#log)
@@ -58,22 +33,54 @@
58
33
  - [replace](#replace)
59
34
  - [Reschedule](#reschedule)
60
35
  - [Custom Strategies](#custom-strategies)
61
- - [Usage](#usage-1)
62
- - [Finer Control over Uniqueness](#finer-control-over-uniqueness)
63
- - [After Unlock Callback](#after-unlock-callback)
64
- - [Cleanup Dead Locks](#cleanup-dead-locks)
65
- - [Other Sidekiq gems](#other-sidekiq-gems)
66
- - [apartment-sidekiq](#apartment-sidekiq)
67
- - [sidekiq-global_id](#sidekiq-global_id)
68
- - [sidekiq-status](#sidekiq-status)
36
+ - [3 Cleanup Dead Locks](#3-cleanup-dead-locks)
69
37
  - [Debugging](#debugging)
70
38
  - [Sidekiq Web](#sidekiq-web)
39
+ - [Reflections \(metrics, logging, etc.\)](#reflections-metrics-logging-etc)
40
+ - [after_unlock_callback_failed](#after_unlock_callback_failed)
41
+ - [error](#error)
42
+ - [execution_failed](#execution_failed)
43
+ - [lock_failed](#lock_failed)
44
+ - [locked](#locked)
45
+ - [reschedule_failed](#reschedule_failed)
46
+ - [rescheduled](#rescheduled)
47
+ - [timeout](#timeout)
48
+ - [unlock_failed](#unlock_failed)
49
+ - [unlocked](#unlocked)
50
+ - [unknown_sidekiq_worker](#unknown_sidekiq_worker)
71
51
  - [Show Locks](#show-locks)
72
52
  - [Show Lock](#show-lock)
73
- - [Communication](#communication)
74
53
  - [Testing](#testing)
75
- - [Unique Sidekiq Configuration](#unique-sidekiq-configuration)
54
+ - [Validating Worker Configuration](#validating-worker-configuration)
76
55
  - [Uniqueness](#uniqueness)
56
+ - [Configuration](#configuration)
57
+ - [Other Sidekiq gems](#other-sidekiq-gems)
58
+ - [apartment-sidekiq](#apartment-sidekiq)
59
+ - [sidekiq-global_id](#sidekiq-global_id)
60
+ - [sidekiq-status](#sidekiq-status)
61
+ - [Global Configuration](#global-configuration)
62
+ - [debug_lua](#debug_lua)
63
+ - [lock_timeout](#lock_timeout)
64
+ - [lock_ttl](#lock_ttl)
65
+ - [enabled](#enabled)
66
+ - [logger](#logger)
67
+ - [max_history](#max_history)
68
+ - [reaper](#reaper)
69
+ - [reaper_count](#reaper_count)
70
+ - [reaper_interval](#reaper_interval)
71
+ - [reaper_timeout](#reaper_timeout)
72
+ - [lock_prefix](#lock_prefix)
73
+ - [lock_info](#lock_info)
74
+ - [Worker Configuration](#worker-configuration)
75
+ - [lock_info](#lock_info-1)
76
+ - [lock_prefix](#lock_prefix-1)
77
+ - [lock_ttl](#lock_ttl-1)
78
+ - [lock_timeout](#lock_timeout-1)
79
+ - [unique_across_queues](#unique_across_queues)
80
+ - [unique_across_workers](#unique_across_workers)
81
+ - [Finer Control over Uniqueness](#finer-control-over-uniqueness)
82
+ - [After Unlock Callback](#after-unlock-callback)
83
+ - [Communication](#communication)
77
84
  - [Contributing](#contributing)
78
85
  - [Contributors](#contributors)
79
86
 
@@ -81,7 +88,9 @@
81
88
 
82
89
  ## Introduction
83
90
 
84
- This gem adds unique constraints to the sidekiq queues. The uniqueness is achieved by acquiring locks for a hash of a queue name, a worker class, and job's arguments. By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen `on_conflict`strategy.
91
+ This gem adds unique constraints to sidekiq jobs. The uniqueness is achieved by creating a set of keys in redis based off of `queue`, `class`, `args` (in the sidekiq job hash).
92
+
93
+ By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen [Conflict Strategy](#conflict-strategy) strategy. Unless a conflict strategy is chosen
85
94
 
86
95
  This is the documentation for the master branch. You can find the documentation for each release by navigating to its tag.
87
96
 
@@ -141,7 +150,7 @@ end
141
150
 
142
151
  ### Your first worker
143
152
 
144
- The most likely to be used worker is `:until_executed`. This type of lock creates a lock from when `UntilExecutedWorker.perform_async` is called until the the sidekiq processor has processed the job.
153
+ The most likely to be used worker is `:until_executed`. This type of lock creates a lock from when `UntilExecutedWorker.perform_async` is called until right after `UntilExecutedWorker.new.perform` has been called.
145
154
 
146
155
  ```ruby
147
156
  # frozen_string_literal: true
@@ -162,10 +171,6 @@ end
162
171
 
163
172
  You can read more about the worker configuration in [Worker Configuration](#worker-configuration) below.
164
173
 
165
- ## Support Me
166
-
167
- Want to show me some ❤️ for the hard work I do on this gem? You can use the following PayPal link: [https://paypal.me/mhenrixon1](https://paypal.me/mhenrixon1). Any amount is welcome and let me tell you it feels good to be appreciated. Even a dollar makes me super excited about all of this.
168
-
169
174
  ## Requirements
170
175
 
171
176
  - Sidekiq `>= 5.0` (`>= 5.2` recommended)
@@ -179,514 +184,770 @@ Want to show me some ❤️ for the hard work I do on this gem? You can use the
179
184
 
180
185
  See [Sidekiq requirements][24] for detailed requirements of Sidekiq itself (be sure to check the right sidekiq version).
181
186
 
182
- ## General Information
183
-
184
- See [Interaction w/ Sidekiq](https://github.com/mhenrixon/sidekiq-unique-jobs/wiki/How-this-gem-interacts-with-Sidekiq) on how the gem interacts with Sidekiq.
185
-
186
- See [Locking & Unlocking](https://github.com/mhenrixon/sidekiq-unique-jobs/wiki/Locking-&-Unlocking) for an overview of the differences on when the various lock types are locked and unlocked.
187
+ ## Locks
187
188
 
188
- ## Reflections (metrics, logging, etc.)
189
+ ### Until Executing
189
190
 
190
- To be able to gather some insights on what is going on inside this gem. I provide a reflection API that can be used.
191
+ A lock is created when `UntilExecuting.perform_async` is called. Then it is either unlocked when `lock_ttl` is hit or before Sidekiq calls the `perform` method on your worker.
191
192
 
192
- To setup reflections for logging or metrics, use the following API:
193
+ #### Example worker
193
194
 
194
195
  ```ruby
196
+ class UntilExecuting
197
+ include Sidekiq::Workers
195
198
 
196
- def extract_log_from_job(message, job_hash)
197
- worker = job_hash['class']
198
- args = job_hash['args']
199
- lock_args = job_hash['lock_args']
200
- queue = job_hash['queue']
201
- {
202
- message: message,
203
- worker: worker,
204
- args: args,
205
- lock_args: lock_args,
206
- queue: queue
207
- }
208
- end
199
+ sidekiq_options lock: :until_executing
209
200
 
210
- SidekiqUniqueJobs.reflect do |on|
211
- on.lock_failed do |job_hash|
212
- message = extract_log_from_job('Lock Failed', job_hash)
213
- Sidekiq.logger.warn(message)
201
+ def perform(id)
202
+ # Do work
214
203
  end
215
204
  end
216
205
  ```
217
206
 
218
- ### after_unlock_callback_failed
219
-
220
- This is called when you have configured a custom callback for when a lock has been released.
221
-
222
- ### error
223
-
224
- Not in use yet but will be used deep into the stack to provide a means to catch and report errors inside the gem.
225
-
226
- ### execution_failed
207
+ **NOTE** this is probably not so good for jobs that shouldn't be running simultaneously (aka slow jobs).
227
208
 
228
- When the sidekiq processor picks the job of the queue for certain jobs but your job raised an error to the middleware. This will be the reflection. It is probably nothing to worry about. When your worker raises an error, we need to handle some edge cases for until and while executing.
209
+ The reason this type of lock exists is to fix the following problem: [sidekiq/issues/3471](https://github.com/mperham/sidekiq/issues/3471#issuecomment-300866335)
229
210
 
230
- ### lock_failed
211
+ ### Until Executed
231
212
 
232
- If we can't achieve a lock, this will be the reflection. It most likely is nothing to worry about. We just couldn't retrieve a lock in a timely fashion.
213
+ A lock is created when `UntilExecuted.perform_async` is called. Then it is either unlocked when `lock_ttl` is hit or when Sidekiq has called the `perform` method on your worker.
233
214
 
234
- The biggest reason for this reflection would be to gather metrics on which workers fail the most at the locking step for example.
215
+ #### Example worker
235
216
 
236
- ### locked
217
+ ```ruby
218
+ class UntilExecuted
219
+ include Sidekiq::Workers
237
220
 
238
- For when a lock has been successful. Again, mostly useful for metrics I suppose.
221
+ sidekiq_options lock: :until_executed
239
222
 
240
- ### reschedule_failed
223
+ def perform(id)
224
+ # Do work
225
+ end
226
+ end
227
+ ```
241
228
 
242
- For when the reschedule strategy failed to reschedule the job.
229
+ ### Until Expired
243
230
 
244
- ### rescheduled
231
+ This lock behaves identically to the [Until Executed](#until-executed) except for one thing. This job won't be unlocked until the expiration is hit. For jobs that need to run only once per day, this would be the perfect lock. This way, we can't create more jobs until one day after this job was first pushed.
245
232
 
246
- For when a job was successfully rescheduled
233
+ #### Example worker
247
234
 
248
- ### timeout
235
+ ```ruby
236
+ class UntilExpired
237
+ include Sidekiq::Workers
249
238
 
250
- This is also mostly useful for reporting/metrics purposes. What this reflection does is signal that the job was configured to wait (`lock_timeout` was configured), but we couldn't retrieve a lock even though we waited for some time.
239
+ sidekiq_options lock: :until_expired, lock_ttl: 1.day
251
240
 
252
- ### unlock_failed
241
+ def perform
242
+ # Do work
243
+ end
244
+ end
245
+ ```
253
246
 
254
- This is not got, this is worth
247
+ ### Until And While Executing
255
248
 
256
- ### unlocked
249
+ This lock is a combination of two locks (`:until_executing` and `:while_executing`). Please see the configuration for [Until Executing](#until-executing) and [While Executing](#while-executing)
257
250
 
258
- Also mostly useful for reporting purposes. The job was successfully unlocked.
251
+ #### Example worker
259
252
 
260
- ### unknown_sidekiq_worker
253
+ ```ruby
254
+ class UntilAndWhileExecutingWorker
255
+ include Sidekiq::Workers
261
256
 
262
- The reason this happens is that the server couldn't find a valid sidekiq worker class. Most likely, that worker isn't intended to be processed by this sidekiq server instance.
257
+ sidekiq_options lock: :until_and_while_executing,
258
+ lock_timeout: 2,
259
+ on_conflict: {
260
+ client: :log,
261
+ server: :raise
262
+ }
263
+ def perform(id)
264
+ # Do work
265
+ end
266
+ end
267
+ ```
263
268
 
264
- ## Global Configuration
269
+ ### While Executing
265
270
 
266
- The gem supports a few different configuration options that might be of interest if you run into some weird issues.
271
+ These locks are put on a queue without any type of locking mechanism, the locking doesn't happen until Sidekiq pops the job from the queue and starts processing it.
267
272
 
268
- Configure SidekiqUniqueJobs in an initializer or the sidekiq initializer on application startup.
273
+ #### Example worker
269
274
 
270
275
  ```ruby
271
- SidekiqUniqueJobs.configure do |config|
272
- config.logger = Sidekiq.logger # default, change at your own discretion
273
- config.debug_lua = false # Turn on when debugging
274
- config.lock_info = false # Turn on when debugging
275
- config.lock_ttl = 600 # Expire locks after 10 minutes
276
- config.lock_timeout = nil # turn off lock timeout
277
- config.max_history = 0 # Turn on when debugging
278
- config.reaper = :ruby # :ruby, :lua or :none/nil
279
- config.reaper_count = 1000 # Stop reaping after this many keys
280
- config.reaper_interval = 600 # Reap orphans every 10 minutes
281
- config.reaper_timeout = 150 # Timeout reaper after 2.5 minutes
276
+ class WhileExecutingWorker
277
+ include Sidekiq::Workers
278
+
279
+ sidekiq_options lock: :while_executing,
280
+ lock_timeout: 2,
281
+ on_conflict: {
282
+ server: :raise
283
+ }
284
+ def perform(id)
285
+ # Do work
286
+ end
282
287
  end
283
288
  ```
284
289
 
285
- ### debug_lua
290
+ **NOTE** Unless a conflict strategy of `:raise` is specified, if lock fails, the job will be dropped without notice. When told to raise, the job will be put back and retried. It would also be possible to use `:reschedule` with this lock.
286
291
 
287
- ```ruby
288
- SidekiqUniqueJobs.config.debug_lua #=> false
289
- ```
292
+ **NOTE** Unless this job is configured with a `lock_timeout: nil` or `lock_timeout: > 0` then all jobs that are attempted to be executed will just be dropped without waiting.
290
293
 
291
- Turning on debug_lua will allow the lua scripts to output debug information about what the lua scripts do. It will log all redis commands that are executed and also some helpful messages about what is going on inside the lua script.
294
+ There is an example of this to try it out in the `myapp` application. Run `foreman start` in the root of the directory and open the url: `localhost:5000/work/duplicate_while_executing`.
292
295
 
293
- ### lock_timeout
296
+ In the console you should see something like:
294
297
 
295
- ```ruby
296
- SidekiqUniqueJobs.config.lock_timeout #=> 0
298
+ ```bash
299
+ 0:32:24 worker.1 | 2017-04-23T08:32:24.955Z 84404 TID-ougq4thko WhileExecutingWorker JID-400ec51c9523f41cd4a35058 INFO: start
300
+ 10:32:24 worker.1 | 2017-04-23T08:32:24.956Z 84404 TID-ougq8csew WhileExecutingWorker JID-8d6d9168368eedaed7f75763 INFO: start
301
+ 10:32:24 worker.1 | 2017-04-23T08:32:24.957Z 84404 TID-ougq8crt8 WhileExecutingWorker JID-affcd079094c9b26e8b9ba60 INFO: start
302
+ 10:32:24 worker.1 | 2017-04-23T08:32:24.959Z 84404 TID-ougq8cs8s WhileExecutingWorker JID-9e197460c067b22eb1b5d07f INFO: start
303
+ 10:32:24 worker.1 | 2017-04-23T08:32:24.959Z 84404 TID-ougq4thko WhileExecutingWorker JID-400ec51c9523f41cd4a35058 WhileExecutingWorker INFO: perform(1, 2)
304
+ 10:32:34 worker.1 | 2017-04-23T08:32:34.964Z 84404 TID-ougq4thko WhileExecutingWorker JID-400ec51c9523f41cd4a35058 INFO: done: 10.009 sec
305
+ 10:32:34 worker.1 | 2017-04-23T08:32:34.965Z 84404 TID-ougq8csew WhileExecutingWorker JID-8d6d9168368eedaed7f75763 WhileExecutingWorker INFO: perform(1, 2)
306
+ 10:32:44 worker.1 | 2017-04-23T08:32:44.965Z 84404 TID-ougq8crt8 WhileExecutingWorker JID-affcd079094c9b26e8b9ba60 WhileExecutingWorker INFO: perform(1, 2)
307
+ 10:32:44 worker.1 | 2017-04-23T08:32:44.965Z 84404 TID-ougq8csew WhileExecutingWorker JID-8d6d9168368eedaed7f75763 INFO: done: 20.009 sec
308
+ 10:32:54 worker.1 | 2017-04-23T08:32:54.970Z 84404 TID-ougq8cs8s WhileExecutingWorker JID-9e197460c067b22eb1b5d07f WhileExecutingWorker INFO: perform(1, 2)
309
+ 10:32:54 worker.1 | 2017-04-23T08:32:54.969Z 84404 TID-ougq8crt8 WhileExecutingWorker JID-affcd079094c9b26e8b9ba60 INFO: done: 30.012 sec
310
+ 10:33:04 worker.1 | 2017-04-23T08:33:04.973Z 84404 TID-ougq8cs8s WhileExecutingWorker JID-9e197460c067b22eb1b5d07f INFO: done: 40.014 sec
297
311
  ```
298
312
 
299
- Set a global lock_timeout to use for all jobs that don't otherwise specify a lock_timeout.
300
-
301
- Lock timeout decides how long to wait for acquiring the lock. A value of nil means to wait indefinitely for a lock resource to become available.
313
+ ### Custom Locks
302
314
 
303
- ### lock_ttl
315
+ You may need to define some custom lock. You can define it in one project folder:
304
316
 
305
317
  ```ruby
306
- SidekiqUniqueJobs.config.lock_ttl #=> nil
318
+ # lib/locks/my_custom_lock.rb
319
+ module Locks
320
+ class MyCustomLock < SidekiqUniqueJobs::Lock::BaseLock
321
+ def execute
322
+ # Do something ...
323
+ end
324
+ end
325
+ end
307
326
  ```
308
327
 
309
- Set a global lock_ttl to use for all jobs that don't otherwise specify a lock_ttl.
328
+ You can refer on all the locks defined in `lib/sidekiq_unique_jobs/lock/*.rb`.
310
329
 
311
- Lock TTL decides how long to wait at most before considering a lock to be expired and making it possible to reuse that lock.
330
+ In order to make it available, you should call in your project startup:
312
331
 
313
- ### enabled
332
+ (For rails application config/initializers/sidekiq_unique_jobs.rb or other projects, wherever you prefer)
314
333
 
315
334
  ```ruby
316
- SidekiqUniqueJobs.config.enabled #=> true
335
+ SidekiqUniqueJobs.configure do |config|
336
+ config.add_lock :my_custom_lock, Locks::MyCustomLock
337
+ end
317
338
  ```
318
339
 
319
- Globally turn the locking mechanism on or off.
320
-
321
- ### logger
322
-
323
- ```ruby
324
- SidekiqUniqueJobs.config.logger #=> #<Sidekiq::Logger:0x00007fdc1f96d180>
325
- ```
340
+ And then you can use it in the jobs definition:
326
341
 
327
- By default this gem piggybacks on the Sidekiq logger. It is not recommended to change this as the gem uses some features in the Sidekiq logger and you might run into problems. If you need a different logger and you do run into problems then get in touch and we'll see what we can do about it.
342
+ `sidekiq_options lock: :my_custom_lock, on_conflict: :log`
328
343
 
329
- ### max_history
344
+ Please not that if you try to override a default lock, an `ArgumentError` will be raised.
330
345
 
331
- ```ruby
332
- SidekiqUniqueJobs.config.max_history #=> 1_000
333
- ```
346
+ ## Conflict Strategy
334
347
 
335
- The max_history setting can be used to tweak the number of changelogs generated. It can also be completely turned off if performance suffers or if you are just not interested in using the changelog.
348
+ Decides how we handle conflict. We can either reject the job to the dead queue or reschedule it. Both are useful for jobs that absolutely need to run and have been configured to use the lock `WhileExecuting` that is used only by the sidekiq server process.
336
349
 
337
- This is a log that can be accessed by a lock to see what happened for that lock. Any items after the configured `max_history` will be automatically deleted as new items are added.
350
+ The last one is log which can be be used with the lock `UntilExecuted` and `UntilExpired`. Now we write a log entry saying the job could not be pushed because it is a duplicate of another job with the same arguments.
338
351
 
339
- ### reaper
352
+ It is possible for locks to have different conflict strategy for the client and server. This is useful for `:until_and_while_executing`.
340
353
 
341
354
  ```ruby
342
- SidekiqUniqueJobs.config.reaper #=> :ruby
355
+ sidekiq_options lock: :until_and_while_executing,
356
+ on_conflict: { client: :log, server: :reject }
343
357
  ```
344
358
 
345
- If using the orphans cleanup process it is critical to be aware of the following. The `:ruby` job is much slower but the `:lua` job locks redis while executing. While doing intense processing it is best to avoid locking redis with a lua script. There for the batch size (controlled by the `reaper_count` setting) needs to be reduced.
346
-
347
- In my benchmarks deleting 1000 orphaned locks with lua performs around 65% faster than deleting 1000 keys in ruby.
348
-
349
- On the other hand if I increase it to 10 000 orphaned locks per cleanup (`reaper_count: 10_0000`) then redis starts throwing:
350
-
351
- > BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE. (Redis::CommandError)
352
-
353
- If you want to disable the reaper set it to `:none`, `nil` or `false`. Actually, any value that isn't `:ruby` or `:lua` will disable the reaping.
359
+ ### log
354
360
 
355
361
  ```ruby
356
- SidekiqUniqueJobs.config.reaper = :none
357
- SidekiqUniqueJobs.config.reaper = nil
358
- SidekiqUniqueJobs.config.reaper = false
362
+ sidekiq_options on_conflict: :log
359
363
  ```
360
364
 
361
- ### reaper_count
365
+ This strategy is intended to be used with `UntilExecuted` and `UntilExpired`. It will log a line about that this is job is a duplicate of another.
366
+
367
+ ### raise
362
368
 
363
369
  ```ruby
364
- SidekiqUniqueJobs.config.reaper_count #=> 1_000
370
+ sidekiq_options on_conflict: :raise
365
371
  ```
366
372
 
367
- The reaper_count setting configures how many orphans at a time will be cleaned up by the orphan cleanup job. This might have to be tweaked depending on which orphan job is running.
373
+ This strategy is intended to be used with `WhileExecuting`. Basically it will allow us to let the server process crash with a specific error message and be retried without messing up the Sidekiq stats.
368
374
 
369
- ### reaper_interval
375
+ ### reject
370
376
 
371
377
  ```ruby
372
- SidekiqUniqueJobs.config.reaper_interval #=> 600
378
+ sidekiq_options on_conflict: :reject
373
379
  ```
374
380
 
375
- The number of seconds between reaping.
381
+ This strategy is intended to be used with `WhileExecuting` and will push the job to the dead queue on conflict.
376
382
 
377
- ### reaper_timeout
383
+ ### replace
378
384
 
379
385
  ```ruby
380
- SidekiqUniqueJobs.config.reaper_timeout #=> 10
386
+ sidekiq_options on_conflict: :replace
381
387
  ```
382
388
 
383
- The number of seconds to wait for the reaper to finish before raising a TimeoutError. This is done to ensure that the next time we reap isn't getting stuck due to the previous process already running.
389
+ This strategy is intended to be used with client locks like `UntilExecuted`.
390
+ It will delete any existing job for these arguments from retry, schedule and
391
+ queue and retry the lock again.
392
+
393
+ This is slightly dangerous and should probably only be used for jobs that are
394
+ always scheduled in the future. Currently only attempting to retry one time.
384
395
 
385
- ### lock_prefix
396
+ ### Reschedule
386
397
 
387
398
  ```ruby
388
- SidekiqUniqueJobs.config.lock_prefix #=> "uniquejobs"
399
+ sidekiq_options on_conflict: :reschedule
389
400
  ```
390
401
 
391
- Use if you want a different key prefix for the keys in redis.
402
+ This strategy is intended to be used with `WhileExecuting` and will delay the job to be tried again in 5 seconds. This will mess up the sidekiq stats but will prevent exceptions from being logged and confuse your sysadmins.
392
403
 
393
- ### lock_info
404
+ ### Custom Strategies
405
+
406
+ You may need to define some custom strategy. You can define it in one project folder:
394
407
 
395
408
  ```ruby
396
- SidekiqUniqueJobs.config.lock_info #=> false
409
+ # lib/strategies/my_custom_strategy.rb
410
+ module Strategies
411
+ class MyCustomStrategy < SidekiqUniqueJobs::OnConflict::Strategy
412
+ def call
413
+ # Do something ...
414
+ end
415
+ end
416
+ end
397
417
  ```
398
418
 
399
- Using lock info will create an additional key for the lock with a json object containing information about the lock. This will be presented in the web interface and might help track down why some jobs are getting stuck.
400
-
401
- ## Worker Configuration
419
+ You can refer to all the strategies defined in `lib/sidekiq_unique_jobs/on_conflict`.
402
420
 
403
- ### lock_info
421
+ In order to make it available, you should call in your project startup:
404
422
 
405
- Lock info gathers information about a specific lock. It collects things like which `lock_args` where used to compute the `lock_digest` that is used for maintaining uniqueness.
423
+ (For rails application config/initializers/sidekiq_unique_jobs.rb for other projects, wherever you prefer)
406
424
 
407
425
  ```ruby
408
- sidekiq_options lock_info: false # this is the default, set to true to turn on
426
+ SidekiqUniqueJobs.configure do |config|
427
+ config.add_strategy :my_custom_strategy, Strategies::MyCustomStrategy
428
+ end
409
429
  ```
410
430
 
411
- ### lock_prefix
412
-
413
- Use if you want a different key prefix for the keys in redis.
431
+ And then you can use it in the jobs definition:
414
432
 
415
433
  ```ruby
416
- sidekiq_options lock_prefix: "uniquejobs" # this is the default value
434
+ sidekiq_options lock: :while_executing, on_conflict: :my_custom_strategy
417
435
  ```
418
436
 
419
- ### lock_ttl
437
+ Please not that if you try to override a default lock, an `ArgumentError` will be raised.
420
438
 
421
- Lock TTL decides how long to wait at most before considering a lock to be expired and making it possible to reuse that lock.
439
+ ### 3 Cleanup Dead Locks
422
440
 
423
- Starting from `v7` the expiration will take place when the job is pushed to the queue.
441
+ For sidekiq versions < 5.1 a `sidekiq_retries_exhausted` block is required per worker class. This is deprecated in Sidekiq 6.0
424
442
 
425
443
  ```ruby
426
- sidekiq_options lock_ttl: nil # default - don't expire keys
427
- sidekiq_options lock_ttl: 20.days.to_i # expire this lock in 20 days
428
- ```
429
-
430
- ### lock_timeout
431
-
432
- This is the timeout (how long to wait) when creating the lock. By default we don't use a timeout so we won't wait for the lock to be created. If you want it is possible to set this like below.
433
-
444
+ class MyWorker
445
+ sidekiq_retries_exhausted do |msg, _ex|
446
+ digest = msg['lock_digest']
447
+ SidekiqUniqueJobs::Digests.new.delete_by_digest(digest) if digest
448
+ end
449
+ end
450
+ ```
451
+
452
+ Starting in v5.1, Sidekiq can also fire a global callback when a job dies: In version 7, this is handled automatically for you. You don't need to add a death handler, if you configure v7 like in [Add the middleware](#add-the-middleware) you don't have to worry about the below.
453
+
434
454
  ```ruby
435
- sidekiq_options lock_timeout: 0 # default - don't wait at all
436
- sidekiq_options lock_timeout: 5 # wait 5 seconds
437
- sidekiq_options lock_timeout: nil # lock indefinitely, this process won't continue until it gets a lock. VERY DANGEROUS!!
455
+ Sidekiq.configure_server do |config|
456
+ config.death_handlers << ->(job, _ex) do
457
+ digest = job['lock_digest']
458
+ SidekiqUniqueJobs::Digests.new.delete_by_digest(digest) if digest
459
+ end
460
+ end
438
461
  ```
439
462
 
440
- ### unique_across_queues
463
+ ## Debugging
441
464
 
442
- This configuration option is slightly misleading. It doesn't disregard the queue on other jobs. Just on itself, this means that a worker that might schedule jobs into multiple queues will be able to have uniqueness enforced on all queues it is pushed to.
465
+ There are several ways of removing keys that are stuck. The prefered way is by using the unique extension to `Sidekiq::Web`. The old console and command line versions still work but might be deprecated in the future. It is better to search for the digest itself and delete the keys matching that digest.
443
466
 
444
- This is mainly intended for `Worker.set(queue: :another).perform_async`.
467
+ ### Sidekiq Web
468
+
469
+ To use the web extension you need to require it in your routes.
445
470
 
446
471
  ```ruby
447
- class Worker
448
- include Sidekiq::Worker
472
+ #app/config/routes.rb
473
+ require 'sidekiq_unique_jobs/web'
474
+ mount Sidekiq::Web, at: '/sidekiq'
475
+ ```
449
476
 
450
- sidekiq_options unique_across_queues: true, queue: 'default'
477
+ There is no need to `require 'sidekiq/web'` since `sidekiq_unique_jobs/web`
478
+ already does this.
451
479
 
452
- def perform(args); end
480
+ To filter/search for keys we can use the wildcard `*`. If we have a unique digest `'uniquejobs:9e9b5ce5d423d3ea470977004b50ff84` we can search for it by enter `*ff84` and it should return all digests that end with `ff84`.
481
+
482
+ ### Reflections (metrics, logging, etc.)
483
+
484
+ To be able to gather some insights on what is going on inside this gem. I provide a reflection API that can be used.
485
+
486
+ To setup reflections for logging or metrics, use the following API:
487
+
488
+ ```ruby
489
+
490
+ def extract_log_from_job(message, job_hash)
491
+ worker = job_hash['class']
492
+ args = job_hash['args']
493
+ lock_args = job_hash['lock_args']
494
+ queue = job_hash['queue']
495
+ {
496
+ message: message,
497
+ worker: worker,
498
+ args: args,
499
+ lock_args: lock_args,
500
+ queue: queue
501
+ }
502
+ end
503
+
504
+ SidekiqUniqueJobs.reflect do |on|
505
+ on.lock_failed do |job_hash|
506
+ message = extract_log_from_job('Lock Failed', job_hash)
507
+ Sidekiq.logger.warn(message)
508
+ end
453
509
  end
454
510
  ```
455
511
 
456
- Now if you push override the queue with `Worker.set(queue: 'another').perform_async(1)` it will still be considered unique when compared to `Worker.perform_async(1)` (that was actually pushed to the queue `default`).
512
+ #### after_unlock_callback_failed
457
513
 
458
- ### unique_across_workers
514
+ This is called when you have configured a custom callback for when a lock has been released.
459
515
 
460
- This configuration option is slightly misleading. It doesn't disregard the worker class on other jobs. Just on itself, this means that the worker class won't be used for generating the unique digest. The only way this option really makes sense is when you want to have uniqueness between two different worker classes.
516
+ #### error
517
+
518
+ Not in use yet but will be used deep into the stack to provide a means to catch and report errors inside the gem.
519
+
520
+ #### execution_failed
521
+
522
+ When the sidekiq processor picks the job of the queue for certain jobs but your job raised an error to the middleware. This will be the reflection. It is probably nothing to worry about. When your worker raises an error, we need to handle some edge cases for until and while executing.
523
+
524
+ #### lock_failed
525
+
526
+ If we can't achieve a lock, this will be the reflection. It most likely is nothing to worry about. We just couldn't retrieve a lock in a timely fashion.
527
+
528
+ The biggest reason for this reflection would be to gather metrics on which workers fail the most at the locking step for example.
529
+
530
+ #### locked
531
+
532
+ For when a lock has been successful. Again, mostly useful for metrics I suppose.
533
+
534
+ #### reschedule_failed
535
+
536
+ For when the reschedule strategy failed to reschedule the job.
537
+
538
+ #### rescheduled
539
+
540
+ For when a job was successfully rescheduled
541
+
542
+ #### timeout
543
+
544
+ This is also mostly useful for reporting/metrics purposes. What this reflection does is signal that the job was configured to wait (`lock_timeout` was configured), but we couldn't retrieve a lock even though we waited for some time.
545
+
546
+ ### unlock_failed
547
+
548
+ This is not got, this is worth
549
+
550
+ ### unlocked
551
+
552
+ Also mostly useful for reporting purposes. The job was successfully unlocked.
553
+
554
+ ### unknown_sidekiq_worker
555
+
556
+ The reason this happens is that the server couldn't find a valid sidekiq worker class. Most likely, that worker isn't intended to be processed by this sidekiq server instance.
557
+
558
+ #### Show Locks
559
+
560
+ ![Locks](assets/unique_digests_1.png)
561
+
562
+ #### Show Lock
563
+
564
+ ![Lock](assets/unique_digests_2.png)
565
+
566
+ ## Testing
567
+
568
+ ### Validating Worker Configuration
569
+
570
+ Since v7 it is possible to perform some simple validation against your workers sidekiq_options. What it does is scan for some issues that are known to cause problems in production.
571
+
572
+ Let's take a _bad_ worker:
461
573
 
462
574
  ```ruby
463
- class WorkerOne
464
- include Sidekiq::Worker
575
+ #app/workers/bad_worker.rb
576
+ class BadWorker
577
+ sidekiq_options lock: :while_executing, on_conflict: :replace
578
+ end
465
579
 
466
- sidekiq_options unique_across_workers: true, queue: 'default'
580
+ #spec/workers/bad_worker_spec.rb
467
581
 
468
- def perform(args); end
582
+ require "sidekiq_unique_jobs/testing"
583
+ #OR
584
+ require "sidekiq_unique_jobs/rspec/matchers"
585
+
586
+ RSpec.describe BadWorker do
587
+ specify { expect(described_class).to have_valid_sidekiq_options }
469
588
  end
589
+ ```
470
590
 
471
- class WorkerTwo
472
- include Sidekiq::Worker
591
+ This gives us a helpful error message for a wrongly configured worker:
473
592
 
474
- sidekiq_options unique_across_workers: true, queue: 'default'
593
+ ```bash
594
+ Expected BadWorker to have valid sidekiq options but found the following problems:
595
+ on_server_conflict: :replace is incompatible with the server process
596
+ ```
475
597
 
476
- def perform(args); end
598
+ If you are not using RSpec (a lot of people prefer minitest or test unit) you can do something like:
599
+
600
+ ```ruby
601
+ assert SidekiqUniqueJobs.validate_worker!(BadWorker.get_sidekiq_options)
602
+ ```
603
+
604
+ ### Uniqueness
605
+
606
+ This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I there for decided to do what Mike is doing for sidekiq enterprise. Read the section about unique jobs: [Enterprise unique jobs][]
607
+
608
+ ```ruby
609
+ SidekiqUniqueJobs.configure do |config|
610
+ config.enabled = !Rails.env.test?
477
611
  end
612
+ ```
478
613
 
614
+ If you truly wanted to test the sidekiq client push you could do something like below. Note that it will only work for the jobs that lock when the client pushes the job to redis (UntilExecuted, UntilAndWhileExecuting and UntilExpired).
479
615
 
480
- WorkerOne.perform_async(1)
481
- # => 'the jobs unique id'
616
+ ```ruby
617
+ require "sidekiq_unique_jobs/testing"
482
618
 
483
- WorkerTwo.perform_async(1)
484
- # => nil because WorkerOne just stole the lock
619
+ RSpec.describe Workers::CoolOne do
620
+ before do
621
+ SidekiqUniqueJobs.config.enabled = false
622
+ end
623
+
624
+ # ... your tests that don't test uniqueness
625
+
626
+ context 'when Sidekiq::Testing.disabled?' do
627
+ before do
628
+ Sidekiq::Testing.disable!
629
+ Sidekiq.redis(&:flushdb)
630
+ end
631
+
632
+ after do
633
+ Sidekiq.redis(&:flushdb)
634
+ end
635
+
636
+ it 'prevents duplicate jobs from being scheduled' do
637
+ SidekiqUniqueJobs.use_config(enabled: true) do
638
+ expect(described_class.perform_in(3600, 1)).not_to eq(nil)
639
+ expect(described_class.perform_async(1)).to eq(nil)
640
+ end
641
+ end
642
+ end
643
+ end
485
644
  ```
486
645
 
487
- ## Locks
646
+ It is recommended to leave the uniqueness testing to the gem maintainers. If you care about how the gem is integration tested have a look at the following specs:
488
647
 
489
- ### Until Executing
648
+ - [spec/sidekiq_unique_jobs/lock/until_and_while_executing_spec.rb](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/spec/sidekiq_unique_jobs/lock/until_and_while_executing_spec.rb)
649
+ - [spec/sidekiq_unique_jobs/lock/until_executed_spec.rb](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/spec/sidekiq_unique_jobs/lock/until_executed_spec.rb)
650
+ - [spec/sidekiq_unique_jobs/lock/until_expired_spec.rb](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/spec/sidekiq_unique_jobs/lock/until_expired_spec.rb)
651
+ - [spec/sidekiq_unique_jobs/lock/while_executing_reject_spec.rb](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/spec/sidekiq_unique_jobs/lock/while_executing_reject_spec.rb)
652
+ - [spec/sidekiq_unique_jobs/lock/while_executing_spec.rb](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/spec/sidekiq_unique_jobs/lock/while_executing_spec.rb)
653
+
654
+ ## Configuration
655
+
656
+ ### Other Sidekiq gems
657
+
658
+ #### apartment-sidekiq
659
+
660
+ It was reported in [#536](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/536) that the order of the Sidekiq middleware needs to be as follows.
661
+
662
+ ```ruby
663
+ Sidekiq.client_middleware do |chain|
664
+ chain.add Apartment::Sidekiq::Middleware::Client
665
+ chain.add SidekiqUniqueJobs::Middleware::Client
666
+ end
667
+
668
+ Sidekiq.server_middleware do |chain|
669
+ chain.add Apartment::Sidekiq::Middleware::Server
670
+ chain.add SidekiqUniqueJobs::Middleware::Server
671
+ end
672
+ ```
673
+
674
+ The reason being that this gem needs to be configured AFTER the apartment gem or the apartment will not be able to be considered for uniqueness
675
+
676
+ #### sidekiq-global_id
677
+
678
+ It was reported in [#235](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/235) that the order of the Sidekiq middleware needs to be as follows.
679
+
680
+ For a working setup check the following [file](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/sidekiq.rb#L12).
490
681
 
491
682
  ```ruby
492
- sidekiq_options lock: :until_executing
683
+ Sidekiq.client_middleware do |chain|
684
+ chain.add Sidekiq::GlobalId::ClientMiddleware
685
+ chain.add SidekiqUniqueJobs::Middleware::Client
686
+ end
687
+
688
+ Sidekiq.server_middleware do |chain|
689
+ chain.add Sidekiq::GlobalId::ServerMiddleware
690
+ chain.add SidekiqUniqueJobs::Middleware::Server
691
+ end
493
692
  ```
494
693
 
495
- Locks from when the client pushes the job to the queue. Will be unlocked before the server starts processing the job.
694
+ The reason for this is that the global id needs to be set before the unique jobs middleware runs. Otherwise that won't be available for uniqueness.
496
695
 
497
- **NOTE** this is probably not so good for jobs that shouldn't be running simultaneously (aka slow jobs).
696
+ #### sidekiq-status
498
697
 
499
- The reason this type of lock exists is to fix the following problem: [sidekiq/issues/3471](https://github.com/mperham/sidekiq/issues/3471#issuecomment-300866335)
698
+ It was reported in [#564](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/564) that the order of the middleware needs to be as follows.
500
699
 
501
- ### Until Executed
700
+ ```ruby
701
+ # Thanks to @ArturT for the correction
702
+
703
+ Sidekiq.configure_server do |config|
704
+ config.client_middleware do |chain|
705
+ chain.add SidekiqUniqueJobs::Middleware::Client
706
+ chain.add Sidekiq::Status::ClientMiddleware, expiration: 30.minutes
707
+ end
708
+
709
+ config.server_middleware do |chain|
710
+ chain.add Sidekiq::Status::ServerMiddleware, expiration: 30.minutes
711
+ chain.add SidekiqUniqueJobs::Middleware::Server
712
+ end
713
+
714
+ SidekiqUniqueJobs::Server.configure(config)
715
+ end
716
+
717
+
718
+ Sidekiq.configure_client do |config|
719
+ config.client_middleware do |chain|
720
+ chain.add SidekiqUniqueJobs::Middleware::Client
721
+ chain.add Sidekiq::Status::ClientMiddleware, expiration: 30.minutes
722
+ end
723
+ end
724
+ ```
725
+
726
+ The reason for this is that if a job is duplicated it shouldn't end up with the status middleware at all. Status is just a monitor so to prevent clashes, leftovers and ensure cleanup. The status middleware should run after uniqueness on client and before on server. This will lead to less surprises.
727
+
728
+ ### Global Configuration
729
+
730
+ The gem supports a few different configuration options that might be of interest if you run into some weird issues.
731
+
732
+ Configure SidekiqUniqueJobs in an initializer or the sidekiq initializer on application startup.
502
733
 
503
734
  ```ruby
504
- sidekiq_options lock: :until_executed
735
+ SidekiqUniqueJobs.configure do |config|
736
+ config.logger = Sidekiq.logger # default, change at your own discretion
737
+ config.debug_lua = false # Turn on when debugging
738
+ config.lock_info = false # Turn on when debugging
739
+ config.lock_ttl = 600 # Expire locks after 10 minutes
740
+ config.lock_timeout = nil # turn off lock timeout
741
+ config.max_history = 0 # Turn on when debugging
742
+ config.reaper = :ruby # :ruby, :lua or :none/nil
743
+ config.reaper_count = 1000 # Stop reaping after this many keys
744
+ config.reaper_interval = 600 # Reap orphans every 10 minutes
745
+ config.reaper_timeout = 150 # Timeout reaper after 2.5 minutes
746
+ end
505
747
  ```
506
748
 
507
- Locks from when the client pushes the job to the queue. Will be unlocked when the server has successfully processed the job.
749
+ #### debug_lua
508
750
 
509
- ### Until Expired
751
+ ```ruby
752
+ SidekiqUniqueJobs.config.debug_lua #=> false
753
+ ```
754
+
755
+ Turning on debug_lua will allow the lua scripts to output debug information about what the lua scripts do. It will log all redis commands that are executed and also some helpful messages about what is going on inside the lua script.
756
+
757
+ #### lock_timeout
510
758
 
511
759
  ```ruby
512
- sidekiq_options lock: :until_expired
760
+ SidekiqUniqueJobs.config.lock_timeout #=> 0
513
761
  ```
514
762
 
515
- Locks from when the client pushes the job to the queue. Will be unlocked when the specified timeout has been reached.
763
+ Set a global lock_timeout to use for all jobs that don't otherwise specify a lock_timeout.
516
764
 
517
- ### Until And While Executing
765
+ Lock timeout decides how long to wait for acquiring the lock. A value of nil means to wait indefinitely for a lock resource to become available.
766
+
767
+ #### lock_ttl
518
768
 
519
769
  ```ruby
520
- sidekiq_options lock: :until_and_while_executing
770
+ SidekiqUniqueJobs.config.lock_ttl #=> nil
521
771
  ```
522
772
 
523
- Locks when the client pushes the job to the queue. The queue will be unlocked when the server starts processing the job. The server then goes on to creating a runtime lock for the job to prevent simultaneous jobs from being executed. As soon as the server starts processing a job, the client can push the same job to the queue.
773
+ Set a global lock_ttl to use for all jobs that don't otherwise specify a lock_ttl.
524
774
 
525
- ### While Executing
775
+ Lock TTL decides how long to wait at most before considering a lock to be expired and making it possible to reuse that lock.
776
+
777
+ #### enabled
526
778
 
527
779
  ```ruby
528
- sidekiq_options lock: :while_executing, lock_timeout: 10
780
+ SidekiqUniqueJobs.config.enabled #=> true
529
781
  ```
530
782
 
531
- With this lock type it is possible to put any number of these jobs on the queue, but as the server pops the job from the queue it will create a lock and then wait until other locks are done processing. It _looks_ like multiple jobs are running at the same time but in fact the second job will only be waiting for the first job to finish.
783
+ Globally turn the locking mechanism on or off.
532
784
 
533
- **NOTE** Unless this job is configured with a `lock_timeout: nil` or `lock_timeout: > 0` then all jobs that are attempted to be executed will just be dropped without waiting.
785
+ #### logger
534
786
 
535
- There is an example of this to try it out in the `myapp` application. Run `foreman start` in the root of the directory and open the url: `localhost:5000/work/duplicate_while_executing`.
787
+ ```ruby
788
+ SidekiqUniqueJobs.config.logger #=> #<Sidekiq::Logger:0x00007fdc1f96d180>
789
+ ```
536
790
 
537
- In the console you should see something like:
791
+ By default this gem piggybacks on the Sidekiq logger. It is not recommended to change this as the gem uses some features in the Sidekiq logger and you might run into problems. If you need a different logger and you do run into problems then get in touch and we'll see what we can do about it.
538
792
 
539
- ```bash
540
- 0:32:24 worker.1 | 2017-04-23T08:32:24.955Z 84404 TID-ougq4thko WhileExecutingWorker JID-400ec51c9523f41cd4a35058 INFO: start
541
- 10:32:24 worker.1 | 2017-04-23T08:32:24.956Z 84404 TID-ougq8csew WhileExecutingWorker JID-8d6d9168368eedaed7f75763 INFO: start
542
- 10:32:24 worker.1 | 2017-04-23T08:32:24.957Z 84404 TID-ougq8crt8 WhileExecutingWorker JID-affcd079094c9b26e8b9ba60 INFO: start
543
- 10:32:24 worker.1 | 2017-04-23T08:32:24.959Z 84404 TID-ougq8cs8s WhileExecutingWorker JID-9e197460c067b22eb1b5d07f INFO: start
544
- 10:32:24 worker.1 | 2017-04-23T08:32:24.959Z 84404 TID-ougq4thko WhileExecutingWorker JID-400ec51c9523f41cd4a35058 WhileExecutingWorker INFO: perform(1, 2)
545
- 10:32:34 worker.1 | 2017-04-23T08:32:34.964Z 84404 TID-ougq4thko WhileExecutingWorker JID-400ec51c9523f41cd4a35058 INFO: done: 10.009 sec
546
- 10:32:34 worker.1 | 2017-04-23T08:32:34.965Z 84404 TID-ougq8csew WhileExecutingWorker JID-8d6d9168368eedaed7f75763 WhileExecutingWorker INFO: perform(1, 2)
547
- 10:32:44 worker.1 | 2017-04-23T08:32:44.965Z 84404 TID-ougq8crt8 WhileExecutingWorker JID-affcd079094c9b26e8b9ba60 WhileExecutingWorker INFO: perform(1, 2)
548
- 10:32:44 worker.1 | 2017-04-23T08:32:44.965Z 84404 TID-ougq8csew WhileExecutingWorker JID-8d6d9168368eedaed7f75763 INFO: done: 20.009 sec
549
- 10:32:54 worker.1 | 2017-04-23T08:32:54.970Z 84404 TID-ougq8cs8s WhileExecutingWorker JID-9e197460c067b22eb1b5d07f WhileExecutingWorker INFO: perform(1, 2)
550
- 10:32:54 worker.1 | 2017-04-23T08:32:54.969Z 84404 TID-ougq8crt8 WhileExecutingWorker JID-affcd079094c9b26e8b9ba60 INFO: done: 30.012 sec
551
- 10:33:04 worker.1 | 2017-04-23T08:33:04.973Z 84404 TID-ougq8cs8s WhileExecutingWorker JID-9e197460c067b22eb1b5d07f INFO: done: 40.014 sec
793
+ #### max_history
794
+
795
+ ```ruby
796
+ SidekiqUniqueJobs.config.max_history #=> 1_000
552
797
  ```
553
798
 
554
- ### Custom Locks
799
+ The max_history setting can be used to tweak the number of changelogs generated. It can also be completely turned off if performance suffers or if you are just not interested in using the changelog.
555
800
 
556
- You may need to define some custom lock. You can define it in one project folder:
801
+ This is a log that can be accessed by a lock to see what happened for that lock. Any items after the configured `max_history` will be automatically deleted as new items are added.
802
+
803
+ #### reaper
557
804
 
558
805
  ```ruby
559
- # lib/locks/my_custom_lock.rb
560
- module Locks
561
- class MyCustomLock < SidekiqUniqueJobs::Lock::BaseLock
562
- def execute
563
- # Do something ...
564
- end
565
- end
566
- end
806
+ SidekiqUniqueJobs.config.reaper #=> :ruby
567
807
  ```
568
808
 
569
- You can refer on all the locks defined in `lib/sidekiq_unique_jobs/lock/*.rb`.
809
+ If using the orphans cleanup process it is critical to be aware of the following. The `:ruby` job is much slower but the `:lua` job locks redis while executing. While doing intense processing it is best to avoid locking redis with a lua script. There for the batch size (controlled by the `reaper_count` setting) needs to be reduced.
570
810
 
571
- In order to make it available, you should call in your project startup:
811
+ In my benchmarks deleting 1000 orphaned locks with lua performs around 65% faster than deleting 1000 keys in ruby.
572
812
 
573
- (For rails application config/initializers/sidekiq_unique_jobs.rb or other projects, wherever you prefer)
813
+ On the other hand if I increase it to 10 000 orphaned locks per cleanup (`reaper_count: 10_0000`) then redis starts throwing:
814
+
815
+ > BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE. (Redis::CommandError)
816
+
817
+ If you want to disable the reaper set it to `:none`, `nil` or `false`. Actually, any value that isn't `:ruby` or `:lua` will disable the reaping.
574
818
 
575
819
  ```ruby
576
- SidekiqUniqueJobs.configure do |config|
577
- config.add_lock :my_custom_lock, Locks::MyCustomLock
578
- end
820
+ SidekiqUniqueJobs.config.reaper = :none
821
+ SidekiqUniqueJobs.config.reaper = nil
822
+ SidekiqUniqueJobs.config.reaper = false
579
823
  ```
580
824
 
581
- And then you can use it in the jobs definition:
825
+ #### reaper_count
582
826
 
583
- `sidekiq_options lock: :my_custom_lock, on_conflict: :log`
827
+ ```ruby
828
+ SidekiqUniqueJobs.config.reaper_count #=> 1_000
829
+ ```
584
830
 
585
- Please not that if you try to override a default lock, an `ArgumentError` will be raised.
831
+ The reaper_count setting configures how many orphans at a time will be cleaned up by the orphan cleanup job. This might have to be tweaked depending on which orphan job is running.
586
832
 
587
- ## Conflict Strategy
833
+ #### reaper_interval
588
834
 
589
- Decides how we handle conflict. We can either reject the job to the dead queue or reschedule it. Both are useful for jobs that absolutely need to run and have been configured to use the lock `WhileExecuting` that is used only by the sidekiq server process.
835
+ ```ruby
836
+ SidekiqUniqueJobs.config.reaper_interval #=> 600
837
+ ```
590
838
 
591
- The last one is log which can be be used with the lock `UntilExecuted` and `UntilExpired`. Now we write a log entry saying the job could not be pushed because it is a duplicate of another job with the same arguments.
839
+ The number of seconds between reaping.
592
840
 
593
- It is possible for locks to have different conflict strategy for the client and server. This is useful for `:until_and_while_executing`.
841
+ #### reaper_timeout
594
842
 
595
843
  ```ruby
596
- sidekiq_options lock: :until_and_while_executing,
597
- on_conflict: { client: :log, server: :reject }
844
+ SidekiqUniqueJobs.config.reaper_timeout #=> 10
598
845
  ```
599
846
 
600
- ### log
847
+ The number of seconds to wait for the reaper to finish before raising a TimeoutError. This is done to ensure that the next time we reap isn't getting stuck due to the previous process already running.
848
+
849
+ #### lock_prefix
601
850
 
602
851
  ```ruby
603
- sidekiq_options on_conflict: :log
852
+ SidekiqUniqueJobs.config.lock_prefix #=> "uniquejobs"
604
853
  ```
605
854
 
606
- This strategy is intended to be used with `UntilExecuted` and `UntilExpired`. It will log a line about that this is job is a duplicate of another.
855
+ Use if you want a different key prefix for the keys in redis.
607
856
 
608
- ### raise
857
+ ### lock_info
609
858
 
610
859
  ```ruby
611
- sidekiq_options on_conflict: :raise
860
+ SidekiqUniqueJobs.config.lock_info #=> false
612
861
  ```
613
862
 
614
- This strategy is intended to be used with `WhileExecuting`. Basically it will allow us to let the server process crash with a specific error message and be retried without messing up the Sidekiq stats.
863
+ Using lock info will create an additional key for the lock with a json object containing information about the lock. This will be presented in the web interface and might help track down why some jobs are getting stuck.
615
864
 
616
- ### reject
865
+ ### Worker Configuration
866
+
867
+ #### lock_info
868
+
869
+ Lock info gathers information about a specific lock. It collects things like which `lock_args` where used to compute the `lock_digest` that is used for maintaining uniqueness.
617
870
 
618
871
  ```ruby
619
- sidekiq_options on_conflict: :reject
872
+ sidekiq_options lock_info: false # this is the default, set to true to turn on
620
873
  ```
621
874
 
622
- This strategy is intended to be used with `WhileExecuting` and will push the job to the dead queue on conflict.
875
+ #### lock_prefix
623
876
 
624
- ### replace
877
+ Use if you want a different key prefix for the keys in redis.
625
878
 
626
879
  ```ruby
627
- sidekiq_options on_conflict: :replace
880
+ sidekiq_options lock_prefix: "uniquejobs" # this is the default value
628
881
  ```
629
882
 
630
- This strategy is intended to be used with client locks like `UntilExecuted`.
631
- It will delete any existing job for these arguments from retry, schedule and
632
- queue and retry the lock again.
883
+ #### lock_ttl
633
884
 
634
- This is slightly dangerous and should probably only be used for jobs that are
635
- always scheduled in the future. Currently only attempting to retry one time.
885
+ Lock TTL decides how long to wait at most before considering a lock to be expired and making it possible to reuse that lock.
636
886
 
637
- ### Reschedule
887
+ Starting from `v7` the expiration will take place when the job is pushed to the queue.
638
888
 
639
889
  ```ruby
640
- sidekiq_options on_conflict: :reschedule
890
+ sidekiq_options lock_ttl: nil # default - don't expire keys
891
+ sidekiq_options lock_ttl: 20.days.to_i # expire this lock in 20 days
641
892
  ```
642
893
 
643
- This strategy is intended to be used with `WhileExecuting` and will delay the job to be tried again in 5 seconds. This will mess up the sidekiq stats but will prevent exceptions from being logged and confuse your sysadmins.
894
+ #### lock_timeout
644
895
 
645
- ### Custom Strategies
896
+ This is the timeout (how long to wait) when creating the lock. By default we don't use a timeout so we won't wait for the lock to be created. If you want it is possible to set this like below.
897
+
898
+ ```ruby
899
+ sidekiq_options lock_timeout: 0 # default - don't wait at all
900
+ sidekiq_options lock_timeout: 5 # wait 5 seconds
901
+ sidekiq_options lock_timeout: nil # lock indefinitely, this process won't continue until it gets a lock. VERY DANGEROUS!!
902
+ ```
903
+
904
+ #### unique_across_queues
905
+
906
+ This configuration option is slightly misleading. It doesn't disregard the queue on other jobs. Just on itself, this means that a worker that might schedule jobs into multiple queues will be able to have uniqueness enforced on all queues it is pushed to.
907
+
908
+ This is mainly intended for `Worker.set(queue: :another).perform_async`.
909
+
910
+ ```ruby
911
+ class Worker
912
+ include Sidekiq::Worker
646
913
 
647
- You may need to define some custom strategy. You can define it in one project folder:
914
+ sidekiq_options unique_across_queues: true, queue: 'default'
648
915
 
649
- ```ruby
650
- # lib/strategies/my_custom_strategy.rb
651
- module Strategies
652
- class MyCustomStrategy < SidekiqUniqueJobs::OnConflict::Strategy
653
- def call
654
- # Do something ...
655
- end
656
- end
916
+ def perform(args); end
657
917
  end
658
918
  ```
659
919
 
660
- You can refer to all the strategies defined in `lib/sidekiq_unique_jobs/on_conflict`.
920
+ Now if you push override the queue with `Worker.set(queue: 'another').perform_async(1)` it will still be considered unique when compared to `Worker.perform_async(1)` (that was actually pushed to the queue `default`).
661
921
 
662
- In order to make it available, you should call in your project startup:
922
+ #### unique_across_workers
663
923
 
664
- (For rails application config/initializers/sidekiq_unique_jobs.rb for other projects, wherever you prefer)
924
+ This configuration option is slightly misleading. It doesn't disregard the worker class on other jobs. Just on itself, this means that the worker class won't be used for generating the unique digest. The only way this option really makes sense is when you want to have uniqueness between two different worker classes.
665
925
 
666
926
  ```ruby
667
- SidekiqUniqueJobs.configure do |config|
668
- config.add_strategy :my_custom_strategy, Strategies::MyCustomStrategy
927
+ class WorkerOne
928
+ include Sidekiq::Worker
929
+
930
+ sidekiq_options unique_across_workers: true, queue: 'default'
931
+
932
+ def perform(args); end
669
933
  end
670
- ```
671
934
 
672
- And then you can use it in the jobs definition:
935
+ class WorkerTwo
936
+ include Sidekiq::Worker
673
937
 
674
- ```ruby
675
- sidekiq_options lock: :while_executing, on_conflict: :my_custom_strategy
676
- ```
938
+ sidekiq_options unique_across_workers: true, queue: 'default'
677
939
 
678
- Please not that if you try to override a default lock, an `ArgumentError` will be raised.
940
+ def perform(args); end
941
+ end
679
942
 
680
- ## Usage
681
943
 
682
- All that is required is that you specifically set the sidekiq option for _unique_ to a valid value like below:
944
+ WorkerOne.perform_async(1)
945
+ # => 'the jobs unique id'
683
946
 
684
- ```ruby
685
- sidekiq_options lock: :while_executing
947
+ WorkerTwo.perform_async(1)
948
+ # => nil because WorkerOne just stole the lock
686
949
  ```
687
950
 
688
- Requiring the gem in your gemfile should be sufficient to enable unique jobs.
689
-
690
951
  ### Finer Control over Uniqueness
691
952
 
692
953
  Sometimes it is desired to have a finer control over which arguments are used in determining uniqueness of the job, and others may be _transient_. For this use-case, you need to define either a `lock_args` method, or a ruby proc.
@@ -765,221 +1026,10 @@ class UniqueJobWithFilterMethod
765
1026
  end.
766
1027
  ```
767
1028
 
768
- ### Cleanup Dead Locks
769
-
770
- For sidekiq versions before 5.1 a `sidekiq_retries_exhausted` block is required per worker class. This is deprecated in Sidekiq 6.0
771
-
772
- ```ruby
773
- class MyWorker
774
- sidekiq_retries_exhausted do |msg, _ex|
775
- digest = msg['lock_digest']
776
- SidekiqUniqueJobs::Digests.new.delete_by_digest(digest) if digest
777
- end
778
- end
779
- ```
780
-
781
- Starting in v5.1, Sidekiq can also fire a global callback when a job dies: In version 7, this is handled automatically for you. You don't need to add a death handler, if you configure v7 like in [Add the middleware](#add-the-middleware) you don't have to worry about the below.
782
-
783
- ```ruby
784
- Sidekiq.configure_server do |config|
785
- config.death_handlers << ->(job, _ex) do
786
- digest = job['lock_digest']
787
- SidekiqUniqueJobs::Digests.new.delete_by_digest(digest) if digest
788
- end
789
- end
790
- ```
791
-
792
- ### Other Sidekiq gems
793
-
794
- #### apartment-sidekiq
795
-
796
- It was reported in [#536](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/536) that the order of the Sidekiq middleware needs to be as follows.
797
-
798
- ```ruby
799
- Sidekiq.client_middleware do |chain|
800
- chain.add Apartment::Sidekiq::Middleware::Client
801
- chain.add SidekiqUniqueJobs::Middleware::Client
802
- end
803
-
804
- Sidekiq.server_middleware do |chain|
805
- chain.add Apartment::Sidekiq::Middleware::Server
806
- chain.add SidekiqUniqueJobs::Middleware::Server
807
- end
808
- ```
809
-
810
- The reason being that this gem needs to be configured AFTER the apartment gem or the apartment will not be able to be considered for uniqueness
811
-
812
- #### sidekiq-global_id
813
-
814
- It was reported in [#235](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/235) that the order of the Sidekiq middleware needs to be as follows.
815
-
816
- For a working setup check the following [file](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/sidekiq.rb#L12).
817
-
818
- ```ruby
819
- Sidekiq.client_middleware do |chain|
820
- chain.add Sidekiq::GlobalId::ClientMiddleware
821
- chain.add SidekiqUniqueJobs::Middleware::Client
822
- end
823
-
824
- Sidekiq.server_middleware do |chain|
825
- chain.add Sidekiq::GlobalId::ServerMiddleware
826
- chain.add SidekiqUniqueJobs::Middleware::Server
827
- end
828
- ```
829
-
830
- The reason for this is that the global id needs to be set before the unique jobs middleware runs. Otherwise that won't be available for uniqueness.
831
-
832
- #### sidekiq-status
833
-
834
- It was reported in [#564](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/564) that the order of the middleware needs to be as follows.
835
-
836
- ```ruby
837
- # Thanks to @ArturT for the correction
838
-
839
- Sidekiq.configure_server do |config|
840
- config.client_middleware do |chain|
841
- chain.add SidekiqUniqueJobs::Middleware::Client
842
- chain.add Sidekiq::Status::ClientMiddleware, expiration: 30.minutes
843
- end
844
-
845
- config.server_middleware do |chain|
846
- chain.add Sidekiq::Status::ServerMiddleware, expiration: 30.minutes
847
- chain.add SidekiqUniqueJobs::Middleware::Server
848
- end
849
-
850
- SidekiqUniqueJobs::Server.configure(config)
851
- end
852
-
853
-
854
- Sidekiq.configure_client do |config|
855
- config.client_middleware do |chain|
856
- chain.add SidekiqUniqueJobs::Middleware::Client
857
- chain.add Sidekiq::Status::ClientMiddleware, expiration: 30.minutes
858
- end
859
- end
860
- ```
861
-
862
- The reason for this is that if a job is duplicated it shouldn't end up with the status middleware at all. Status is just a monitor so to prevent clashes, leftovers and ensure cleanup. The status middleware should run after uniqueness on client and before on server. This will lead to less surprises.
863
-
864
- ## Debugging
865
-
866
- There are several ways of removing keys that are stuck. The prefered way is by using the unique extension to `Sidekiq::Web`. The old console and command line versions still work but might be deprecated in the future. It is better to search for the digest itself and delete the keys matching that digest.
867
-
868
- ### Sidekiq Web
869
-
870
- To use the web extension you need to require it in your routes.
871
-
872
- ```ruby
873
- #app/config/routes.rb
874
- require 'sidekiq_unique_jobs/web'
875
- mount Sidekiq::Web, at: '/sidekiq'
876
- ```
877
-
878
- There is no need to `require 'sidekiq/web'` since `sidekiq_unique_jobs/web`
879
- already does this.
880
-
881
- To filter/search for keys we can use the wildcard `*`. If we have a unique digest `'uniquejobs:9e9b5ce5d423d3ea470977004b50ff84` we can search for it by enter `*ff84` and it should return all digests that end with `ff84`.
882
-
883
- #### Show Locks
884
-
885
- ![Locks](assets/unique_digests_1.png)
886
-
887
- #### Show Lock
888
-
889
- ![Lock](assets/unique_digests_2.png)
890
-
891
1029
  ## Communication
892
1030
 
893
1031
  There is a [![Join the chat at https://gitter.im/mhenrixon/sidekiq-unique-jobs](https://badges.gitter.im/mhenrixon/sidekiq-unique-jobs.svg)](https://gitter.im/mhenrixon/sidekiq-unique-jobs?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) for praise or scorn. This would be a good place to have lengthy discuss or brilliant suggestions or simply just nudge me if I forget about anything.
894
1032
 
895
- ## Testing
896
-
897
- ### Unique Sidekiq Configuration
898
-
899
- Since v7 it is possible to perform some simple validation against your workers sidekiq_options. What it does is scan for some issues that are known to cause problems in production.
900
-
901
- Let's take a _bad_ worker:
902
-
903
- ```ruby
904
- #app/workers/bad_worker.rb
905
- class BadWorker
906
- sidekiq_options lock: :while_executing, on_conflict: :replace
907
- end
908
-
909
- #spec/workers/bad_worker_spec.rb
910
-
911
- require "sidekiq_unique_jobs/testing"
912
- #OR
913
- require "sidekiq_unique_jobs/rspec/matchers"
914
-
915
- RSpec.describe BadWorker do
916
- specify { expect(described_class).to have_valid_sidekiq_options }
917
- end
918
- ```
919
-
920
- This gives us a helpful error message for a wrongly configured worker:
921
-
922
- ```bash
923
- Expected BadWorker to have valid sidekiq options but found the following problems:
924
- on_server_conflict: :replace is incompatible with the server process
925
- ```
926
-
927
- If you are not using RSpec (a lot of people prefer minitest or test unit) you can do something like:
928
-
929
- ```ruby
930
- assert SidekiqUniqueJobs.validate_worker!(BadWorker.get_sidekiq_options)
931
- ```
932
-
933
- ### Uniqueness
934
-
935
- This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I there for decided to do what Mike is doing for sidekiq enterprise. Read the section about unique jobs: [Enterprise unique jobs][]
936
-
937
- ```ruby
938
- SidekiqUniqueJobs.configure do |config|
939
- config.enabled = !Rails.env.test?
940
- end
941
- ```
942
-
943
- If you truly wanted to test the sidekiq client push you could do something like below. Note that it will only work for the jobs that lock when the client pushes the job to redis (UntilExecuted, UntilAndWhileExecuting and UntilExpired).
944
-
945
- ```ruby
946
- require "sidekiq_unique_jobs/testing"
947
-
948
- RSpec.describe Workers::CoolOne do
949
- before do
950
- SidekiqUniqueJobs.config.enabled = false
951
- end
952
-
953
- # ... your tests that don't test uniqueness
954
-
955
- context 'when Sidekiq::Testing.disabled?' do
956
- before do
957
- Sidekiq::Testing.disable!
958
- Sidekiq.redis(&:flushdb)
959
- end
960
-
961
- after do
962
- Sidekiq.redis(&:flushdb)
963
- end
964
-
965
- it 'prevents duplicate jobs from being scheduled' do
966
- SidekiqUniqueJobs.use_config(enabled: true) do
967
- expect(described_class.perform_in(3600, 1)).not_to eq(nil)
968
- expect(described_class.perform_async(1)).to eq(nil)
969
- end
970
- end
971
- end
972
- end
973
- ```
974
-
975
- It is recommended to leave the uniqueness testing to the gem maintainers. If you care about how the gem is integration tested have a look at the following specs:
976
-
977
- - [spec/sidekiq_unique_jobs/lock/until_and_while_executing_spec.rb](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/spec/sidekiq_unique_jobs/lock/until_and_while_executing_spec.rb)
978
- - [spec/sidekiq_unique_jobs/lock/until_executed_spec.rb](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/spec/sidekiq_unique_jobs/lock/until_executed_spec.rb)
979
- - [spec/sidekiq_unique_jobs/lock/until_expired_spec.rb](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/spec/sidekiq_unique_jobs/lock/until_expired_spec.rb)
980
- - [spec/sidekiq_unique_jobs/lock/while_executing_reject_spec.rb](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/spec/sidekiq_unique_jobs/lock/while_executing_reject_spec.rb)
981
- - [spec/sidekiq_unique_jobs/lock/while_executing_spec.rb](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/spec/sidekiq_unique_jobs/lock/while_executing_spec.rb)
982
-
983
1033
  ## Contributing
984
1034
 
985
1035
  1. Fork it
@@ -992,8 +1042,8 @@ It is recommended to leave the uniqueness testing to the gem maintainers. If you
992
1042
 
993
1043
  You can find a list of contributors over on [Contributors][]
994
1044
 
995
- [v5.0.10]: https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10.
996
- [v4.0.18]: https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18
997
- [Sidekiq requirements]: https://github.com/mperham/sidekiq#requirements
998
1045
  [Enterprise unique jobs]: https://www.dailydrip.com/topics/sidekiq/drips/sidekiq-enterprise-unique-jobs
999
1046
  [Contributors]: https://github.com/mhenrixon/sidekiq-unique-jobs/graphs/contributors
1047
+ [v4.0.18]: https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18
1048
+ [v5.0.10]: https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10.
1049
+ [Sidekiq requirements]: https://github.com/mperham/sidekiq#requirements