sidekiq-unique-jobs 7.0.0.beta27 → 7.0.2
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of sidekiq-unique-jobs might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +90 -14
- data/README.md +162 -30
- data/lib/sidekiq-unique-jobs.rb +0 -2
- data/lib/sidekiq_unique_jobs.rb +1 -0
- data/lib/sidekiq_unique_jobs/batch_delete.rb +1 -1
- data/lib/sidekiq_unique_jobs/changelog.rb +11 -4
- data/lib/sidekiq_unique_jobs/config.rb +2 -2
- data/lib/sidekiq_unique_jobs/constants.rb +2 -0
- data/lib/sidekiq_unique_jobs/digests.rb +1 -1
- data/lib/sidekiq_unique_jobs/job.rb +1 -1
- data/lib/sidekiq_unique_jobs/json.rb +7 -1
- data/lib/sidekiq_unique_jobs/lock.rb +31 -1
- data/lib/sidekiq_unique_jobs/lock/while_executing.rb +1 -1
- data/lib/sidekiq_unique_jobs/lock_config.rb +2 -0
- data/lib/sidekiq_unique_jobs/locksmith.rb +1 -1
- data/lib/sidekiq_unique_jobs/lua/lock.lua +10 -11
- data/lib/sidekiq_unique_jobs/lua/reap_orphans.lua +8 -7
- data/lib/sidekiq_unique_jobs/lua/shared/_find_digest_in_process_set.lua +9 -2
- data/lib/sidekiq_unique_jobs/middleware.rb +0 -57
- data/lib/sidekiq_unique_jobs/on_conflict/replace.rb +9 -8
- data/lib/sidekiq_unique_jobs/on_conflict/reschedule.rb +1 -1
- data/lib/sidekiq_unique_jobs/orphans/lua_reaper.rb +1 -1
- data/lib/sidekiq_unique_jobs/orphans/reaper.rb +10 -0
- data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +14 -5
- data/lib/sidekiq_unique_jobs/redis/entity.rb +9 -3
- data/lib/sidekiq_unique_jobs/redis/sorted_set.rb +27 -0
- data/lib/sidekiq_unique_jobs/server.rb +48 -0
- data/lib/sidekiq_unique_jobs/sidekiq_unique_jobs.rb +1 -1
- data/lib/sidekiq_unique_jobs/version.rb +1 -1
- data/lib/sidekiq_unique_jobs/web.rb +26 -9
- data/lib/sidekiq_unique_jobs/web/helpers.rb +24 -3
- data/lib/sidekiq_unique_jobs/web/views/changelogs.erb +54 -0
- data/lib/sidekiq_unique_jobs/web/views/locks.erb +1 -1
- metadata +12 -8
- data/lib/sidekiq_unique_jobs/profiler.rb +0 -55
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b34494d8b6a8a456d1bae7f88d19d0effa3772a14ec6e835fdb1818684c6429c
|
4
|
+
data.tar.gz: 05abaa45fb5ae3ff65f0e921da86c9b7df8fea97ba11fbe1f2314f239ccd99e0
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: f715b5e6afd217ab5339a295bb43c234ef50e723b683021da9f7136e91d6cc6cf8d966ea50ea8acbe8b700c2f3e6d4503caab89cd39c570d5a906e332d11c873
|
7
|
+
data.tar.gz: d2dd8182d2e59029a6c440f2a9260a2ea1b8c59be03218c4df616b17e27610fe44f5dae77c492fd156510cd9fa89cd5fcd5212a69f9050069ce6579561f95381
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,85 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v7.0.1](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.1) (2021-01-22)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0...v7.0.1)
|
6
|
+
|
7
|
+
**Implemented enhancements:**
|
8
|
+
|
9
|
+
- Any way to manually clear/reset the changelog history? [\#568](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/568)
|
10
|
+
- Present the entire changelog in its own view [\#569](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/569) ([mhenrixon](https://github.com/mhenrixon))
|
11
|
+
|
12
|
+
**Fixed bugs:**
|
13
|
+
|
14
|
+
- Fix configuration [\#570](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/570) ([mhenrixon](https://github.com/mhenrixon))
|
15
|
+
|
16
|
+
**Closed issues:**
|
17
|
+
|
18
|
+
- undefined method 'delete\_by\_digest' for SidekiqUniqueJobs::Digests:Class [\#567](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/567)
|
19
|
+
- Rejected jobs are still displayed as 'Queued' with gem 'sidekiq-status' on /sidekiq/statuses [\#564](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/564)
|
20
|
+
|
21
|
+
## [v7.0.0](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0) (2021-01-20)
|
22
|
+
|
23
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta29...v7.0.0)
|
24
|
+
|
25
|
+
**Implemented enhancements:**
|
26
|
+
|
27
|
+
- Give user full control over adding middleware [\#566](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/566) ([mhenrixon](https://github.com/mhenrixon))
|
28
|
+
- Fix coverage reporting and add coverage [\#565](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/565) ([mhenrixon](https://github.com/mhenrixon))
|
29
|
+
|
30
|
+
**Fixed bugs:**
|
31
|
+
|
32
|
+
- Race condition in ruby reaper [\#559](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/559)
|
33
|
+
- Fix until and while executed and improve documentation [\#397](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/397)
|
34
|
+
- Fix race condition to avoid reaping active jobs [\#563](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/563) ([mhenrixon](https://github.com/mhenrixon))
|
35
|
+
|
36
|
+
**Closed issues:**
|
37
|
+
|
38
|
+
- Is it possible to have a :until\_executed lock with an expiration time? [\#524](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/524)
|
39
|
+
|
40
|
+
## [v7.0.0.beta29](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta29) (2021-01-16)
|
41
|
+
|
42
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta28...v7.0.0.beta29)
|
43
|
+
|
44
|
+
**Fixed bugs:**
|
45
|
+
|
46
|
+
- Ruby Reaper active check incorrect [\#557](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/557)
|
47
|
+
- Routes with authentication should work with web [\#562](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/562) ([mhenrixon](https://github.com/mhenrixon))
|
48
|
+
|
49
|
+
**Closed issues:**
|
50
|
+
|
51
|
+
- Can't add the lock tab o nthe website when there is authentication through devise [\#561](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/561)
|
52
|
+
|
53
|
+
## [v7.0.0.beta28](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta28) (2021-01-07)
|
54
|
+
|
55
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta27...v7.0.0.beta28)
|
56
|
+
|
57
|
+
**Fixed bugs:**
|
58
|
+
|
59
|
+
- lock\_args does not work when you define the lock\_args argument and default lock\_args function at the same time. [\#548](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/548)
|
60
|
+
- incorrect `:while\_executing` behavior [\#432](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/432)
|
61
|
+
- Fix active check for the worker hash [\#558](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/558) ([mhenrixon](https://github.com/mhenrixon))
|
62
|
+
- Prefer lock\_prefix not unique\_prefix [\#554](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/554) ([mhenrixon](https://github.com/mhenrixon))
|
63
|
+
- Fix issue 432 [\#552](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/552) ([mhenrixon](https://github.com/mhenrixon))
|
64
|
+
|
65
|
+
## [v7.0.0.beta27](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta27) (2020-11-03)
|
66
|
+
|
67
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta26...v7.0.0.beta27)
|
68
|
+
|
69
|
+
**Implemented enhancements:**
|
70
|
+
|
71
|
+
- Adds coverage for regression purposes [\#550](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/550) ([mhenrixon](https://github.com/mhenrixon))
|
72
|
+
|
73
|
+
**Fixed bugs:**
|
74
|
+
|
75
|
+
- Rename lock\_args to lock\_args\_method [\#551](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/551) ([mhenrixon](https://github.com/mhenrixon))
|
76
|
+
- Fix RubyReaper active? [\#538](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/538) ([tanner-rutgers](https://github.com/tanner-rutgers))
|
77
|
+
|
78
|
+
**Closed issues:**
|
79
|
+
|
80
|
+
- Documentation incorrect for `delete\_by\_digest` [\#547](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/547)
|
81
|
+
- Locked jobs after kill -9 with while\_executing lock [\#546](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/546)
|
82
|
+
|
3
83
|
## [v7.0.0.beta26](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta26) (2020-10-28)
|
4
84
|
|
5
85
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.25...v7.0.0.beta26)
|
@@ -33,7 +113,6 @@
|
|
33
113
|
**Fixed bugs:**
|
34
114
|
|
35
115
|
- Ruby reaper not working, active jobs queried incorrectly [\#537](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/537)
|
36
|
-
- Fix RubyReaper active? [\#538](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/538) ([tanner-rutgers](https://github.com/tanner-rutgers))
|
37
116
|
|
38
117
|
**Closed issues:**
|
39
118
|
|
@@ -125,6 +204,7 @@
|
|
125
204
|
**Fixed bugs:**
|
126
205
|
|
127
206
|
- Stringify on\_conflict hash in Job prepare method [\#507](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/507) ([jasonbekolay](https://github.com/jasonbekolay))
|
207
|
+
- Fix access to both server and client conflict [\#503](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/503) ([mhenrixon](https://github.com/mhenrixon))
|
128
208
|
|
129
209
|
## [v7.0.0.beta17](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta17) (2020-05-20)
|
130
210
|
|
@@ -134,10 +214,6 @@
|
|
134
214
|
|
135
215
|
- Try GitHub actions [\#505](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/505) ([mhenrixon](https://github.com/mhenrixon))
|
136
216
|
|
137
|
-
**Fixed bugs:**
|
138
|
-
|
139
|
-
- Deep stringify worker options to account for hash in on\_conflict [\#506](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/506) ([jasonbekolay](https://github.com/jasonbekolay))
|
140
|
-
|
141
217
|
## [v7.0.0.beta16](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta16) (2020-05-19)
|
142
218
|
|
143
219
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.22...v7.0.0.beta16)
|
@@ -145,7 +221,6 @@
|
|
145
221
|
**Fixed bugs:**
|
146
222
|
|
147
223
|
- Deprecate configuration options with `default\_` [\#504](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/504) ([mhenrixon](https://github.com/mhenrixon))
|
148
|
-
- Fix access to both server and client conflict [\#503](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/503) ([mhenrixon](https://github.com/mhenrixon))
|
149
224
|
|
150
225
|
**Closed issues:**
|
151
226
|
|
@@ -157,6 +232,7 @@
|
|
157
232
|
|
158
233
|
**Fixed bugs:**
|
159
234
|
|
235
|
+
- Deep stringify worker options to account for hash in on\_conflict [\#506](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/506) ([jasonbekolay](https://github.com/jasonbekolay))
|
160
236
|
- \[LockType\] Allow overwriting default lock type at item level [\#497](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/497) ([EmmN](https://github.com/EmmN))
|
161
237
|
|
162
238
|
## [v7.0.0.beta15](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta15) (2020-04-10)
|
@@ -193,12 +269,15 @@
|
|
193
269
|
**Fixed bugs:**
|
194
270
|
|
195
271
|
- Use thread-safe digest creation mechanism [\#484](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/484) ([mhenrixon](https://github.com/mhenrixon))
|
196
|
-
- Remove digest deletion for concurrent locks [\#482](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/482) ([mhenrixon](https://github.com/mhenrixon))
|
197
272
|
|
198
273
|
## [v7.0.0.beta13](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta13) (2020-03-26)
|
199
274
|
|
200
275
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta12...v7.0.0.beta13)
|
201
276
|
|
277
|
+
**Fixed bugs:**
|
278
|
+
|
279
|
+
- Remove digest deletion for concurrent locks [\#482](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/482) ([mhenrixon](https://github.com/mhenrixon))
|
280
|
+
|
202
281
|
## [v7.0.0.beta12](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta12) (2020-03-25)
|
203
282
|
|
204
283
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.20...v7.0.0.beta12)
|
@@ -207,6 +286,7 @@
|
|
207
286
|
|
208
287
|
- until\_expired is not setting TTL [\#468](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/468)
|
209
288
|
- Fix bug where expiration wasn't set until unlock [\#481](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/481) ([mhenrixon](https://github.com/mhenrixon))
|
289
|
+
- Fix access to lock [\#476](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/476) ([mhenrixon](https://github.com/mhenrixon))
|
210
290
|
|
211
291
|
## [v6.0.20](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.20) (2020-03-22)
|
212
292
|
|
@@ -260,7 +340,6 @@
|
|
260
340
|
|
261
341
|
**Fixed bugs:**
|
262
342
|
|
263
|
-
- Fix access to lock [\#476](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/476) ([mhenrixon](https://github.com/mhenrixon))
|
264
343
|
- Backport v7 fix for conflicts [\#461](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/461) ([mhenrixon](https://github.com/mhenrixon))
|
265
344
|
|
266
345
|
## [v7.0.0.beta9](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta9) (2019-12-04)
|
@@ -270,7 +349,6 @@
|
|
270
349
|
**Implemented enhancements:**
|
271
350
|
|
272
351
|
- Keys without TTL [\#417](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/417)
|
273
|
-
- Various changes to test and verify reliability [\#463](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/463) ([mhenrixon](https://github.com/mhenrixon))
|
274
352
|
|
275
353
|
**Closed issues:**
|
276
354
|
|
@@ -283,6 +361,7 @@
|
|
283
361
|
**Implemented enhancements:**
|
284
362
|
|
285
363
|
- Allow worker to configure client and server strategies separately [\#402](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/402)
|
364
|
+
- Various changes to test and verify reliability [\#463](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/463) ([mhenrixon](https://github.com/mhenrixon))
|
286
365
|
- Separate client and server on\_conflict [\#462](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/462) ([mhenrixon](https://github.com/mhenrixon))
|
287
366
|
|
288
367
|
**Fixed bugs:**
|
@@ -507,6 +586,7 @@
|
|
507
586
|
- Version 6 Ignores Jobs Enqueued in Version 5 [\#345](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/345)
|
508
587
|
- Job will not enqueue even with no existing match [\#342](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/342)
|
509
588
|
- Convert v5 locks when needed [\#375](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/375) ([mhenrixon](https://github.com/mhenrixon))
|
589
|
+
- Always Remove Job ID from uniquejobs Hash [\#347](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/347) ([chadrschroeder](https://github.com/chadrschroeder))
|
510
590
|
|
511
591
|
**Closed issues:**
|
512
592
|
|
@@ -590,10 +670,6 @@
|
|
590
670
|
|
591
671
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.6...v5.0.11)
|
592
672
|
|
593
|
-
**Fixed bugs:**
|
594
|
-
|
595
|
-
- Always Remove Job ID from uniquejobs Hash [\#347](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/347) ([chadrschroeder](https://github.com/chadrschroeder))
|
596
|
-
|
597
673
|
**Closed issues:**
|
598
674
|
|
599
675
|
- concurrent-ruby 1.1.1 is causing this gem to break [\#340](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/340)
|
@@ -802,7 +878,6 @@
|
|
802
878
|
|
803
879
|
- Legacy support [\#280](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/280)
|
804
880
|
- Adds legacy support [\#281](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/281) ([mhenrixon](https://github.com/mhenrixon))
|
805
|
-
- Adds guard-reek [\#279](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/279) ([mhenrixon](https://github.com/mhenrixon))
|
806
881
|
- Fix UntilExpired [\#278](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/278) ([mhenrixon](https://github.com/mhenrixon))
|
807
882
|
|
808
883
|
## [v6.0.0.beta2](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0.beta2) (2018-06-25)
|
@@ -820,6 +895,7 @@
|
|
820
895
|
|
821
896
|
**Implemented enhancements:**
|
822
897
|
|
898
|
+
- Adds guard-reek [\#279](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/279) ([mhenrixon](https://github.com/mhenrixon))
|
823
899
|
- Code smells [\#275](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/275) ([mhenrixon](https://github.com/mhenrixon))
|
824
900
|
- Reject while scheduling [\#273](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/273) ([mhenrixon](https://github.com/mhenrixon))
|
825
901
|
- Improve testing [\#272](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/272) ([mhenrixon](https://github.com/mhenrixon))
|
data/README.md
CHANGED
@@ -3,9 +3,12 @@
|
|
3
3
|
<!-- MarkdownTOC -->
|
4
4
|
|
5
5
|
- [Introduction](#introduction)
|
6
|
-
- [
|
7
|
-
- [Installation](#installation)
|
6
|
+
- [Usage](#usage)
|
7
|
+
- [Installation](#installation)
|
8
|
+
- [Add the middleware](#add-the-middleware)
|
9
|
+
- [Your first worker](#your-first-worker)
|
8
10
|
- [Support Me](#support-me)
|
11
|
+
- [Requirements](#requirements)
|
9
12
|
- [General Information](#general-information)
|
10
13
|
- [Global Configuration](#global-configuration)
|
11
14
|
- [debug_lua](#debug_lua)
|
@@ -18,9 +21,11 @@
|
|
18
21
|
- [reaper_count](#reaper_count)
|
19
22
|
- [reaper_interval](#reaper_interval)
|
20
23
|
- [reaper_timeout](#reaper_timeout)
|
21
|
-
- [
|
24
|
+
- [lock_prefix](#lock_prefix)
|
22
25
|
- [lock_info](#lock_info)
|
23
26
|
- [Worker Configuration](#worker-configuration)
|
27
|
+
- [lock_info](#lock_info-1)
|
28
|
+
- [lock_prefix](#lock_prefix-1)
|
24
29
|
- [lock_ttl](#lock_ttl-1)
|
25
30
|
- [lock_timeout](#lock_timeout-1)
|
26
31
|
- [unique_across_queues](#unique_across_queues)
|
@@ -39,13 +44,15 @@
|
|
39
44
|
- [replace](#replace)
|
40
45
|
- [Reschedule](#reschedule)
|
41
46
|
- [Custom Strategies](#custom-strategies)
|
42
|
-
- [Usage](#usage)
|
47
|
+
- [Usage](#usage-1)
|
43
48
|
- [Finer Control over Uniqueness](#finer-control-over-uniqueness)
|
44
49
|
- [After Unlock Callback](#after-unlock-callback)
|
45
50
|
- [Logging](#logging)
|
46
51
|
- [Cleanup Dead Locks](#cleanup-dead-locks)
|
47
52
|
- [Other Sidekiq gems](#other-sidekiq-gems)
|
53
|
+
- [apartment-sidekiq](#apartment-sidekiq)
|
48
54
|
- [sidekiq-global_id](#sidekiq-global_id)
|
55
|
+
- [sidekiq-status](#sidekiq-status)
|
49
56
|
- [Debugging](#debugging)
|
50
57
|
- [Sidekiq Web](#sidekiq-web)
|
51
58
|
- [Show Locks](#show-locks)
|
@@ -61,30 +68,19 @@
|
|
61
68
|
|
62
69
|
## Introduction
|
63
70
|
|
64
|
-
The
|
71
|
+
This gem adds unique constraints to the sidekiq queues. The uniqueness is achieved by acquiring locks for a hash of a queue name, a worker class, and job's arguments. By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen `on_conflict`strategy.
|
65
72
|
|
66
73
|
This is the documentation for the master branch. You can find the documentation for each release by navigating to its tag.
|
67
74
|
|
68
75
|
Here are links to some of the old versions
|
69
76
|
|
70
|
-
- [v6.0.
|
77
|
+
- [v6.0.25](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.25)
|
71
78
|
- [v5.0.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10)
|
72
79
|
- [v4.0.18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18)
|
73
80
|
|
74
|
-
##
|
75
|
-
|
76
|
-
- Sidekiq `>= 4.0` (`>= 5.2` recommended)
|
77
|
-
- Ruby:
|
78
|
-
- MRI `>= 2.3` (`>= 2.5` recommended)
|
79
|
-
- JRuby `>= 9.0` (`>= 9.2` recommended)
|
80
|
-
- Truffleruby
|
81
|
-
- Redis Server `>= 3.0.2` (`>= 3.2` recommended)
|
82
|
-
- [ActiveJob officially not supported][48]
|
83
|
-
- [redis-namespace officially not supported][49]
|
84
|
-
|
85
|
-
See [Sidekiq requirements][24] for detailed requirements of Sidekiq itself (be sure to check the right sidekiq version).
|
81
|
+
## Usage
|
86
82
|
|
87
|
-
|
83
|
+
### Installation
|
88
84
|
|
89
85
|
Add this line to your application's Gemfile:
|
90
86
|
|
@@ -98,16 +94,83 @@ And then execute:
|
|
98
94
|
bundle
|
99
95
|
```
|
100
96
|
|
101
|
-
|
97
|
+
### Add the middleware
|
98
|
+
|
99
|
+
Before v7, the middleware was configured automatically. Since some people reported issues with other gems (see [Other Sidekiq Gems](#other-sidekiq-gems)) it was decided to give full control over to the user.
|
100
|
+
|
101
|
+
*NOTE* if you want to use the reaper you also need to configure the server middleware.
|
102
|
+
|
103
|
+
[A full example](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/initializers/sidekiq.rb#L12)
|
104
|
+
|
105
|
+
```ruby
|
106
|
+
Sidekiq.configure_server do |config|
|
107
|
+
config.redis = { url: ENV["REDIS_URL"], driver: :hiredis }
|
108
|
+
|
109
|
+
config.client_middleware do |chain|
|
110
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
111
|
+
end
|
112
|
+
|
113
|
+
config.server_middleware do |chain|
|
114
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
115
|
+
end
|
116
|
+
|
117
|
+
SidekiqUniqueJobs::Server.configure(config)
|
118
|
+
end
|
119
|
+
|
120
|
+
Sidekiq.configure_client do |config|
|
121
|
+
config.redis = { url: ENV["REDIS_URL"], driver: :hiredis }
|
122
|
+
|
123
|
+
config.client_middleware do |chain|
|
124
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
125
|
+
end
|
126
|
+
end
|
127
|
+
```
|
128
|
+
|
129
|
+
### Your first worker
|
130
|
+
|
131
|
+
```ruby
|
132
|
+
# frozen_string_literal: true
|
133
|
+
|
134
|
+
class UntilExecutedWorker
|
135
|
+
include Sidekiq::Worker
|
136
|
+
|
137
|
+
sidekiq_options queue: :special,
|
138
|
+
retry: false,
|
139
|
+
lock: :until_executed,
|
140
|
+
lock_info: true,
|
141
|
+
lock_timeout: 0,
|
142
|
+
lock_prefix: "special",
|
143
|
+
lock_ttl: 0,
|
144
|
+
lock_limit: 5
|
145
|
+
|
146
|
+
def perform
|
147
|
+
logger.info("cowboy")
|
148
|
+
sleep(1) # hardcore processing
|
149
|
+
logger.info("beebop")
|
150
|
+
end
|
151
|
+
end
|
102
152
|
|
103
|
-
```bash
|
104
|
-
gem install sidekiq-unique-jobs
|
105
153
|
```
|
106
154
|
|
155
|
+
You can read more about the worker configuration in [Worker Configuration](#worker-configuration) below.
|
156
|
+
|
107
157
|
## Support Me
|
108
158
|
|
109
159
|
Want to show me some ❤️ for the hard work I do on this gem? You can use the following PayPal link: [https://paypal.me/mhenrixon1](https://paypal.me/mhenrixon1). Any amount is welcome and let me tell you it feels good to be appreciated. Even a dollar makes me super excited about all of this.
|
110
160
|
|
161
|
+
## Requirements
|
162
|
+
|
163
|
+
- Sidekiq `>= 5.0` (`>= 5.2` recommended)
|
164
|
+
- Ruby:
|
165
|
+
- MRI `>= 2.5` (`>= 2.6` recommended)
|
166
|
+
- JRuby `>= 9.0` (`>= 9.2` recommended)
|
167
|
+
- Truffleruby
|
168
|
+
- Redis Server `>= 3.2` (`>= 5.0` recommended)
|
169
|
+
- [ActiveJob officially not supported][48]
|
170
|
+
- [redis-namespace officially not supported][49]
|
171
|
+
|
172
|
+
See [Sidekiq requirements][24] for detailed requirements of Sidekiq itself (be sure to check the right sidekiq version).
|
173
|
+
|
111
174
|
## General Information
|
112
175
|
|
113
176
|
See [Interaction w/ Sidekiq](https://github.com/mhenrixon/sidekiq-unique-jobs/wiki/How-this-gem-interacts-with-Sidekiq) on how the gem interacts with Sidekiq.
|
@@ -235,10 +298,10 @@ SidekiqUniqueJobs.config.reaper_timeout #=> 10
|
|
235
298
|
|
236
299
|
The number of seconds to wait for the reaper to finish before raising a TimeoutError. This is done to ensure that the next time we reap isn't getting stuck due to the previous process already running.
|
237
300
|
|
238
|
-
###
|
301
|
+
### lock_prefix
|
239
302
|
|
240
303
|
```ruby
|
241
|
-
SidekiqUniqueJobs.config.
|
304
|
+
SidekiqUniqueJobs.config.lock_prefix #=> "uniquejobs"
|
242
305
|
```
|
243
306
|
|
244
307
|
Use if you want a different key prefix for the keys in redis.
|
@@ -253,11 +316,27 @@ Using lock info will create an additional key for the lock with a json object co
|
|
253
316
|
|
254
317
|
## Worker Configuration
|
255
318
|
|
319
|
+
### lock_info
|
320
|
+
|
321
|
+
Lock info gathers information about a specific lock. It collects things like which `lock_args` where used to compute the `lock_digest` that is used for maintaining uniqueness.
|
322
|
+
|
323
|
+
```ruby
|
324
|
+
sidekiq_options lock_info: false # this is the default, set to true to turn on
|
325
|
+
```
|
326
|
+
|
327
|
+
### lock_prefix
|
328
|
+
|
329
|
+
Use if you want a different key prefix for the keys in redis.
|
330
|
+
|
331
|
+
```ruby
|
332
|
+
sidekiq_options lock_prefix: "uniquejobs" # this is the default value
|
333
|
+
```
|
334
|
+
|
256
335
|
### lock_ttl
|
257
336
|
|
258
337
|
Lock TTL decides how long to wait after the job has been successfully processed before making it possible to reuse that lock.
|
259
338
|
|
260
|
-
|
339
|
+
Starting from `v7` the expiration will take place when the job is pushed to the queue.
|
261
340
|
|
262
341
|
```ruby
|
263
342
|
sidekiq_options lock_ttl: nil # default - don't expire keys
|
@@ -278,6 +357,8 @@ sidekiq_options lock_timeout: nil # lock indefinitely, this process won't contin
|
|
278
357
|
|
279
358
|
This configuration option is slightly misleading. It doesn't disregard the queue on other jobs. Just on itself, this means that a worker that might schedule jobs into multiple queues will be able to have uniqueness enforced on all queues it is pushed to.
|
280
359
|
|
360
|
+
This is mainly intended for `Worker.set(queue: :another).perform_async`.
|
361
|
+
|
281
362
|
```ruby
|
282
363
|
class Worker
|
283
364
|
include Sidekiq::Worker
|
@@ -428,7 +509,7 @@ The last one is log which can be be used with the lock `UntilExecuted` and `Unti
|
|
428
509
|
It is possible for locks to have different conflict strategy for the client and server. This is useful for `:until_and_while_executing`.
|
429
510
|
|
430
511
|
```ruby
|
431
|
-
sidekiq_options lock: :until_and_while_executing,
|
512
|
+
sidekiq_options lock: :until_and_while_executing,
|
432
513
|
on_conflict: { client: :log, server: :reject }
|
433
514
|
```
|
434
515
|
|
@@ -628,7 +709,7 @@ class MyWorker
|
|
628
709
|
end
|
629
710
|
```
|
630
711
|
|
631
|
-
Starting in v5.1, Sidekiq can also fire a global callback when a job dies:
|
712
|
+
Starting in v5.1, Sidekiq can also fire a global callback when a job dies: In version 7, this is handled automatically for you. You don't need to add a death handler, if you configure v7 like in [Add the middleware](#add-the-middleware) you don't have to worry about the below.
|
632
713
|
|
633
714
|
```ruby
|
634
715
|
Sidekiq.configure_server do |config|
|
@@ -641,10 +722,30 @@ end
|
|
641
722
|
|
642
723
|
### Other Sidekiq gems
|
643
724
|
|
725
|
+
#### apartment-sidekiq
|
726
|
+
|
727
|
+
It was reported in [#536](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/536) that the order of the Sidekiq middleware needs to be as follows.
|
728
|
+
|
729
|
+
```ruby
|
730
|
+
Sidekiq.client_middleware do |chain|
|
731
|
+
chain.add Apartment::Sidekiq::Middleware::Client
|
732
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
733
|
+
end
|
734
|
+
|
735
|
+
Sidekiq.server_middleware do |chain|
|
736
|
+
chain.add Apartment::Sidekiq::Middleware::Server
|
737
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
738
|
+
end
|
739
|
+
```
|
740
|
+
|
741
|
+
The reason being that this gem needs to be configured AFTER the apartment gem or the apartment will not be able to be considered for uniqueness
|
742
|
+
|
644
743
|
#### sidekiq-global_id
|
645
744
|
|
646
745
|
It was reported in [#235](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/235) that the order of the Sidekiq middleware needs to be as follows.
|
647
746
|
|
747
|
+
For a working setup check the following [file](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/sidekiq.rb#L12).
|
748
|
+
|
648
749
|
```ruby
|
649
750
|
Sidekiq.client_middleware do |chain|
|
650
751
|
chain.add Sidekiq::GlobalId::ClientMiddleware
|
@@ -652,12 +753,44 @@ Sidekiq.client_middleware do |chain|
|
|
652
753
|
end
|
653
754
|
|
654
755
|
Sidekiq.server_middleware do |chain|
|
655
|
-
chain.add SidekiqUniqueJobs::Middleware::Server
|
656
756
|
chain.add Sidekiq::GlobalId::ServerMiddleware
|
757
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
758
|
+
end
|
759
|
+
```
|
760
|
+
|
761
|
+
The reason for this is that the global id needs to be set before the unique jobs middleware runs. Otherwise that won't be available for uniqueness.
|
762
|
+
|
763
|
+
#### sidekiq-status
|
764
|
+
|
765
|
+
It was reported in [#564](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/564) that the order of the middleware needs to be as follows.
|
766
|
+
|
767
|
+
```ruby
|
768
|
+
# Thanks to @ArturT for the correction
|
769
|
+
|
770
|
+
Sidekiq.configure_server do |config|
|
771
|
+
config.client_middleware do |chain|
|
772
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
773
|
+
chain.add Sidekiq::Status::ClientMiddleware, expiration: 30.minutes
|
774
|
+
end
|
775
|
+
|
776
|
+
config.server_middleware do |chain|
|
777
|
+
chain.add Sidekiq::Status::ServerMiddleware, expiration: 30.minutes
|
778
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
779
|
+
end
|
780
|
+
|
781
|
+
SidekiqUniqueJobs::Server.configure(config)
|
782
|
+
end
|
783
|
+
|
784
|
+
|
785
|
+
Sidekiq.configure_client do |config|
|
786
|
+
config.client_middleware do |chain|
|
787
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
788
|
+
chain.add Sidekiq::Status::ClientMiddleware, expiration: 30.minutes
|
789
|
+
end
|
657
790
|
end
|
658
791
|
```
|
659
792
|
|
660
|
-
|
793
|
+
The reason for this is that if a job is duplicated it shouldn't end up with the status middleware at all. Status is just a monitor so to prevent clashes, leftovers and ensure cleanup. The status middleware should run after uniqueness on client and before on server. This will lead to less surprises.
|
661
794
|
|
662
795
|
## Debugging
|
663
796
|
|
@@ -728,7 +861,6 @@ If you are not using RSpec (a lot of people prefer minitest or test unit) you ca
|
|
728
861
|
assert SidekiqUniqueJobs.validate_worker!(BadWorker.get_sidekiq_options)
|
729
862
|
```
|
730
863
|
|
731
|
-
|
732
864
|
### Uniqueness
|
733
865
|
|
734
866
|
This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I there for decided to do what Mike is doing for sidekiq enterprise. Read the section about unique jobs: [Enterprise unique jobs][]
|