sidekiq-unique-jobs 7.0.0.beta27 → 7.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of sidekiq-unique-jobs might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +90 -14
- data/README.md +162 -30
- data/lib/sidekiq-unique-jobs.rb +0 -2
- data/lib/sidekiq_unique_jobs.rb +1 -0
- data/lib/sidekiq_unique_jobs/batch_delete.rb +1 -1
- data/lib/sidekiq_unique_jobs/changelog.rb +11 -4
- data/lib/sidekiq_unique_jobs/config.rb +2 -2
- data/lib/sidekiq_unique_jobs/constants.rb +2 -0
- data/lib/sidekiq_unique_jobs/digests.rb +1 -1
- data/lib/sidekiq_unique_jobs/job.rb +1 -1
- data/lib/sidekiq_unique_jobs/json.rb +7 -1
- data/lib/sidekiq_unique_jobs/lock.rb +31 -1
- data/lib/sidekiq_unique_jobs/lock/while_executing.rb +1 -1
- data/lib/sidekiq_unique_jobs/lock_config.rb +2 -0
- data/lib/sidekiq_unique_jobs/locksmith.rb +1 -1
- data/lib/sidekiq_unique_jobs/lua/lock.lua +10 -11
- data/lib/sidekiq_unique_jobs/lua/reap_orphans.lua +8 -7
- data/lib/sidekiq_unique_jobs/lua/shared/_find_digest_in_process_set.lua +9 -2
- data/lib/sidekiq_unique_jobs/middleware.rb +0 -57
- data/lib/sidekiq_unique_jobs/on_conflict/replace.rb +9 -8
- data/lib/sidekiq_unique_jobs/on_conflict/reschedule.rb +1 -1
- data/lib/sidekiq_unique_jobs/orphans/lua_reaper.rb +1 -1
- data/lib/sidekiq_unique_jobs/orphans/reaper.rb +10 -0
- data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +14 -5
- data/lib/sidekiq_unique_jobs/redis/entity.rb +9 -3
- data/lib/sidekiq_unique_jobs/redis/sorted_set.rb +27 -0
- data/lib/sidekiq_unique_jobs/server.rb +48 -0
- data/lib/sidekiq_unique_jobs/sidekiq_unique_jobs.rb +1 -1
- data/lib/sidekiq_unique_jobs/version.rb +1 -1
- data/lib/sidekiq_unique_jobs/web.rb +26 -9
- data/lib/sidekiq_unique_jobs/web/helpers.rb +24 -3
- data/lib/sidekiq_unique_jobs/web/views/changelogs.erb +54 -0
- data/lib/sidekiq_unique_jobs/web/views/locks.erb +1 -1
- metadata +12 -8
- data/lib/sidekiq_unique_jobs/profiler.rb +0 -55
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b34494d8b6a8a456d1bae7f88d19d0effa3772a14ec6e835fdb1818684c6429c
|
4
|
+
data.tar.gz: 05abaa45fb5ae3ff65f0e921da86c9b7df8fea97ba11fbe1f2314f239ccd99e0
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: f715b5e6afd217ab5339a295bb43c234ef50e723b683021da9f7136e91d6cc6cf8d966ea50ea8acbe8b700c2f3e6d4503caab89cd39c570d5a906e332d11c873
|
7
|
+
data.tar.gz: d2dd8182d2e59029a6c440f2a9260a2ea1b8c59be03218c4df616b17e27610fe44f5dae77c492fd156510cd9fa89cd5fcd5212a69f9050069ce6579561f95381
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,85 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v7.0.1](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.1) (2021-01-22)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0...v7.0.1)
|
6
|
+
|
7
|
+
**Implemented enhancements:**
|
8
|
+
|
9
|
+
- Any way to manually clear/reset the changelog history? [\#568](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/568)
|
10
|
+
- Present the entire changelog in its own view [\#569](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/569) ([mhenrixon](https://github.com/mhenrixon))
|
11
|
+
|
12
|
+
**Fixed bugs:**
|
13
|
+
|
14
|
+
- Fix configuration [\#570](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/570) ([mhenrixon](https://github.com/mhenrixon))
|
15
|
+
|
16
|
+
**Closed issues:**
|
17
|
+
|
18
|
+
- undefined method 'delete\_by\_digest' for SidekiqUniqueJobs::Digests:Class [\#567](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/567)
|
19
|
+
- Rejected jobs are still displayed as 'Queued' with gem 'sidekiq-status' on /sidekiq/statuses [\#564](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/564)
|
20
|
+
|
21
|
+
## [v7.0.0](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0) (2021-01-20)
|
22
|
+
|
23
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta29...v7.0.0)
|
24
|
+
|
25
|
+
**Implemented enhancements:**
|
26
|
+
|
27
|
+
- Give user full control over adding middleware [\#566](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/566) ([mhenrixon](https://github.com/mhenrixon))
|
28
|
+
- Fix coverage reporting and add coverage [\#565](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/565) ([mhenrixon](https://github.com/mhenrixon))
|
29
|
+
|
30
|
+
**Fixed bugs:**
|
31
|
+
|
32
|
+
- Race condition in ruby reaper [\#559](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/559)
|
33
|
+
- Fix until and while executed and improve documentation [\#397](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/397)
|
34
|
+
- Fix race condition to avoid reaping active jobs [\#563](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/563) ([mhenrixon](https://github.com/mhenrixon))
|
35
|
+
|
36
|
+
**Closed issues:**
|
37
|
+
|
38
|
+
- Is it possible to have a :until\_executed lock with an expiration time? [\#524](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/524)
|
39
|
+
|
40
|
+
## [v7.0.0.beta29](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta29) (2021-01-16)
|
41
|
+
|
42
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta28...v7.0.0.beta29)
|
43
|
+
|
44
|
+
**Fixed bugs:**
|
45
|
+
|
46
|
+
- Ruby Reaper active check incorrect [\#557](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/557)
|
47
|
+
- Routes with authentication should work with web [\#562](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/562) ([mhenrixon](https://github.com/mhenrixon))
|
48
|
+
|
49
|
+
**Closed issues:**
|
50
|
+
|
51
|
+
- Can't add the lock tab o nthe website when there is authentication through devise [\#561](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/561)
|
52
|
+
|
53
|
+
## [v7.0.0.beta28](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta28) (2021-01-07)
|
54
|
+
|
55
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta27...v7.0.0.beta28)
|
56
|
+
|
57
|
+
**Fixed bugs:**
|
58
|
+
|
59
|
+
- lock\_args does not work when you define the lock\_args argument and default lock\_args function at the same time. [\#548](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/548)
|
60
|
+
- incorrect `:while\_executing` behavior [\#432](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/432)
|
61
|
+
- Fix active check for the worker hash [\#558](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/558) ([mhenrixon](https://github.com/mhenrixon))
|
62
|
+
- Prefer lock\_prefix not unique\_prefix [\#554](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/554) ([mhenrixon](https://github.com/mhenrixon))
|
63
|
+
- Fix issue 432 [\#552](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/552) ([mhenrixon](https://github.com/mhenrixon))
|
64
|
+
|
65
|
+
## [v7.0.0.beta27](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta27) (2020-11-03)
|
66
|
+
|
67
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta26...v7.0.0.beta27)
|
68
|
+
|
69
|
+
**Implemented enhancements:**
|
70
|
+
|
71
|
+
- Adds coverage for regression purposes [\#550](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/550) ([mhenrixon](https://github.com/mhenrixon))
|
72
|
+
|
73
|
+
**Fixed bugs:**
|
74
|
+
|
75
|
+
- Rename lock\_args to lock\_args\_method [\#551](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/551) ([mhenrixon](https://github.com/mhenrixon))
|
76
|
+
- Fix RubyReaper active? [\#538](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/538) ([tanner-rutgers](https://github.com/tanner-rutgers))
|
77
|
+
|
78
|
+
**Closed issues:**
|
79
|
+
|
80
|
+
- Documentation incorrect for `delete\_by\_digest` [\#547](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/547)
|
81
|
+
- Locked jobs after kill -9 with while\_executing lock [\#546](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/546)
|
82
|
+
|
3
83
|
## [v7.0.0.beta26](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta26) (2020-10-28)
|
4
84
|
|
5
85
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.25...v7.0.0.beta26)
|
@@ -33,7 +113,6 @@
|
|
33
113
|
**Fixed bugs:**
|
34
114
|
|
35
115
|
- Ruby reaper not working, active jobs queried incorrectly [\#537](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/537)
|
36
|
-
- Fix RubyReaper active? [\#538](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/538) ([tanner-rutgers](https://github.com/tanner-rutgers))
|
37
116
|
|
38
117
|
**Closed issues:**
|
39
118
|
|
@@ -125,6 +204,7 @@
|
|
125
204
|
**Fixed bugs:**
|
126
205
|
|
127
206
|
- Stringify on\_conflict hash in Job prepare method [\#507](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/507) ([jasonbekolay](https://github.com/jasonbekolay))
|
207
|
+
- Fix access to both server and client conflict [\#503](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/503) ([mhenrixon](https://github.com/mhenrixon))
|
128
208
|
|
129
209
|
## [v7.0.0.beta17](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta17) (2020-05-20)
|
130
210
|
|
@@ -134,10 +214,6 @@
|
|
134
214
|
|
135
215
|
- Try GitHub actions [\#505](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/505) ([mhenrixon](https://github.com/mhenrixon))
|
136
216
|
|
137
|
-
**Fixed bugs:**
|
138
|
-
|
139
|
-
- Deep stringify worker options to account for hash in on\_conflict [\#506](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/506) ([jasonbekolay](https://github.com/jasonbekolay))
|
140
|
-
|
141
217
|
## [v7.0.0.beta16](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta16) (2020-05-19)
|
142
218
|
|
143
219
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.22...v7.0.0.beta16)
|
@@ -145,7 +221,6 @@
|
|
145
221
|
**Fixed bugs:**
|
146
222
|
|
147
223
|
- Deprecate configuration options with `default\_` [\#504](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/504) ([mhenrixon](https://github.com/mhenrixon))
|
148
|
-
- Fix access to both server and client conflict [\#503](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/503) ([mhenrixon](https://github.com/mhenrixon))
|
149
224
|
|
150
225
|
**Closed issues:**
|
151
226
|
|
@@ -157,6 +232,7 @@
|
|
157
232
|
|
158
233
|
**Fixed bugs:**
|
159
234
|
|
235
|
+
- Deep stringify worker options to account for hash in on\_conflict [\#506](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/506) ([jasonbekolay](https://github.com/jasonbekolay))
|
160
236
|
- \[LockType\] Allow overwriting default lock type at item level [\#497](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/497) ([EmmN](https://github.com/EmmN))
|
161
237
|
|
162
238
|
## [v7.0.0.beta15](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta15) (2020-04-10)
|
@@ -193,12 +269,15 @@
|
|
193
269
|
**Fixed bugs:**
|
194
270
|
|
195
271
|
- Use thread-safe digest creation mechanism [\#484](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/484) ([mhenrixon](https://github.com/mhenrixon))
|
196
|
-
- Remove digest deletion for concurrent locks [\#482](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/482) ([mhenrixon](https://github.com/mhenrixon))
|
197
272
|
|
198
273
|
## [v7.0.0.beta13](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta13) (2020-03-26)
|
199
274
|
|
200
275
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta12...v7.0.0.beta13)
|
201
276
|
|
277
|
+
**Fixed bugs:**
|
278
|
+
|
279
|
+
- Remove digest deletion for concurrent locks [\#482](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/482) ([mhenrixon](https://github.com/mhenrixon))
|
280
|
+
|
202
281
|
## [v7.0.0.beta12](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta12) (2020-03-25)
|
203
282
|
|
204
283
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.20...v7.0.0.beta12)
|
@@ -207,6 +286,7 @@
|
|
207
286
|
|
208
287
|
- until\_expired is not setting TTL [\#468](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/468)
|
209
288
|
- Fix bug where expiration wasn't set until unlock [\#481](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/481) ([mhenrixon](https://github.com/mhenrixon))
|
289
|
+
- Fix access to lock [\#476](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/476) ([mhenrixon](https://github.com/mhenrixon))
|
210
290
|
|
211
291
|
## [v6.0.20](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.20) (2020-03-22)
|
212
292
|
|
@@ -260,7 +340,6 @@
|
|
260
340
|
|
261
341
|
**Fixed bugs:**
|
262
342
|
|
263
|
-
- Fix access to lock [\#476](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/476) ([mhenrixon](https://github.com/mhenrixon))
|
264
343
|
- Backport v7 fix for conflicts [\#461](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/461) ([mhenrixon](https://github.com/mhenrixon))
|
265
344
|
|
266
345
|
## [v7.0.0.beta9](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta9) (2019-12-04)
|
@@ -270,7 +349,6 @@
|
|
270
349
|
**Implemented enhancements:**
|
271
350
|
|
272
351
|
- Keys without TTL [\#417](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/417)
|
273
|
-
- Various changes to test and verify reliability [\#463](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/463) ([mhenrixon](https://github.com/mhenrixon))
|
274
352
|
|
275
353
|
**Closed issues:**
|
276
354
|
|
@@ -283,6 +361,7 @@
|
|
283
361
|
**Implemented enhancements:**
|
284
362
|
|
285
363
|
- Allow worker to configure client and server strategies separately [\#402](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/402)
|
364
|
+
- Various changes to test and verify reliability [\#463](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/463) ([mhenrixon](https://github.com/mhenrixon))
|
286
365
|
- Separate client and server on\_conflict [\#462](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/462) ([mhenrixon](https://github.com/mhenrixon))
|
287
366
|
|
288
367
|
**Fixed bugs:**
|
@@ -507,6 +586,7 @@
|
|
507
586
|
- Version 6 Ignores Jobs Enqueued in Version 5 [\#345](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/345)
|
508
587
|
- Job will not enqueue even with no existing match [\#342](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/342)
|
509
588
|
- Convert v5 locks when needed [\#375](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/375) ([mhenrixon](https://github.com/mhenrixon))
|
589
|
+
- Always Remove Job ID from uniquejobs Hash [\#347](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/347) ([chadrschroeder](https://github.com/chadrschroeder))
|
510
590
|
|
511
591
|
**Closed issues:**
|
512
592
|
|
@@ -590,10 +670,6 @@
|
|
590
670
|
|
591
671
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.6...v5.0.11)
|
592
672
|
|
593
|
-
**Fixed bugs:**
|
594
|
-
|
595
|
-
- Always Remove Job ID from uniquejobs Hash [\#347](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/347) ([chadrschroeder](https://github.com/chadrschroeder))
|
596
|
-
|
597
673
|
**Closed issues:**
|
598
674
|
|
599
675
|
- concurrent-ruby 1.1.1 is causing this gem to break [\#340](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/340)
|
@@ -802,7 +878,6 @@
|
|
802
878
|
|
803
879
|
- Legacy support [\#280](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/280)
|
804
880
|
- Adds legacy support [\#281](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/281) ([mhenrixon](https://github.com/mhenrixon))
|
805
|
-
- Adds guard-reek [\#279](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/279) ([mhenrixon](https://github.com/mhenrixon))
|
806
881
|
- Fix UntilExpired [\#278](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/278) ([mhenrixon](https://github.com/mhenrixon))
|
807
882
|
|
808
883
|
## [v6.0.0.beta2](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0.beta2) (2018-06-25)
|
@@ -820,6 +895,7 @@
|
|
820
895
|
|
821
896
|
**Implemented enhancements:**
|
822
897
|
|
898
|
+
- Adds guard-reek [\#279](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/279) ([mhenrixon](https://github.com/mhenrixon))
|
823
899
|
- Code smells [\#275](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/275) ([mhenrixon](https://github.com/mhenrixon))
|
824
900
|
- Reject while scheduling [\#273](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/273) ([mhenrixon](https://github.com/mhenrixon))
|
825
901
|
- Improve testing [\#272](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/272) ([mhenrixon](https://github.com/mhenrixon))
|
data/README.md
CHANGED
@@ -3,9 +3,12 @@
|
|
3
3
|
<!-- MarkdownTOC -->
|
4
4
|
|
5
5
|
- [Introduction](#introduction)
|
6
|
-
- [
|
7
|
-
- [Installation](#installation)
|
6
|
+
- [Usage](#usage)
|
7
|
+
- [Installation](#installation)
|
8
|
+
- [Add the middleware](#add-the-middleware)
|
9
|
+
- [Your first worker](#your-first-worker)
|
8
10
|
- [Support Me](#support-me)
|
11
|
+
- [Requirements](#requirements)
|
9
12
|
- [General Information](#general-information)
|
10
13
|
- [Global Configuration](#global-configuration)
|
11
14
|
- [debug_lua](#debug_lua)
|
@@ -18,9 +21,11 @@
|
|
18
21
|
- [reaper_count](#reaper_count)
|
19
22
|
- [reaper_interval](#reaper_interval)
|
20
23
|
- [reaper_timeout](#reaper_timeout)
|
21
|
-
- [
|
24
|
+
- [lock_prefix](#lock_prefix)
|
22
25
|
- [lock_info](#lock_info)
|
23
26
|
- [Worker Configuration](#worker-configuration)
|
27
|
+
- [lock_info](#lock_info-1)
|
28
|
+
- [lock_prefix](#lock_prefix-1)
|
24
29
|
- [lock_ttl](#lock_ttl-1)
|
25
30
|
- [lock_timeout](#lock_timeout-1)
|
26
31
|
- [unique_across_queues](#unique_across_queues)
|
@@ -39,13 +44,15 @@
|
|
39
44
|
- [replace](#replace)
|
40
45
|
- [Reschedule](#reschedule)
|
41
46
|
- [Custom Strategies](#custom-strategies)
|
42
|
-
- [Usage](#usage)
|
47
|
+
- [Usage](#usage-1)
|
43
48
|
- [Finer Control over Uniqueness](#finer-control-over-uniqueness)
|
44
49
|
- [After Unlock Callback](#after-unlock-callback)
|
45
50
|
- [Logging](#logging)
|
46
51
|
- [Cleanup Dead Locks](#cleanup-dead-locks)
|
47
52
|
- [Other Sidekiq gems](#other-sidekiq-gems)
|
53
|
+
- [apartment-sidekiq](#apartment-sidekiq)
|
48
54
|
- [sidekiq-global_id](#sidekiq-global_id)
|
55
|
+
- [sidekiq-status](#sidekiq-status)
|
49
56
|
- [Debugging](#debugging)
|
50
57
|
- [Sidekiq Web](#sidekiq-web)
|
51
58
|
- [Show Locks](#show-locks)
|
@@ -61,30 +68,19 @@
|
|
61
68
|
|
62
69
|
## Introduction
|
63
70
|
|
64
|
-
The
|
71
|
+
This gem adds unique constraints to the sidekiq queues. The uniqueness is achieved by acquiring locks for a hash of a queue name, a worker class, and job's arguments. By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen `on_conflict`strategy.
|
65
72
|
|
66
73
|
This is the documentation for the master branch. You can find the documentation for each release by navigating to its tag.
|
67
74
|
|
68
75
|
Here are links to some of the old versions
|
69
76
|
|
70
|
-
- [v6.0.
|
77
|
+
- [v6.0.25](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.25)
|
71
78
|
- [v5.0.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10)
|
72
79
|
- [v4.0.18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18)
|
73
80
|
|
74
|
-
##
|
75
|
-
|
76
|
-
- Sidekiq `>= 4.0` (`>= 5.2` recommended)
|
77
|
-
- Ruby:
|
78
|
-
- MRI `>= 2.3` (`>= 2.5` recommended)
|
79
|
-
- JRuby `>= 9.0` (`>= 9.2` recommended)
|
80
|
-
- Truffleruby
|
81
|
-
- Redis Server `>= 3.0.2` (`>= 3.2` recommended)
|
82
|
-
- [ActiveJob officially not supported][48]
|
83
|
-
- [redis-namespace officially not supported][49]
|
84
|
-
|
85
|
-
See [Sidekiq requirements][24] for detailed requirements of Sidekiq itself (be sure to check the right sidekiq version).
|
81
|
+
## Usage
|
86
82
|
|
87
|
-
|
83
|
+
### Installation
|
88
84
|
|
89
85
|
Add this line to your application's Gemfile:
|
90
86
|
|
@@ -98,16 +94,83 @@ And then execute:
|
|
98
94
|
bundle
|
99
95
|
```
|
100
96
|
|
101
|
-
|
97
|
+
### Add the middleware
|
98
|
+
|
99
|
+
Before v7, the middleware was configured automatically. Since some people reported issues with other gems (see [Other Sidekiq Gems](#other-sidekiq-gems)) it was decided to give full control over to the user.
|
100
|
+
|
101
|
+
*NOTE* if you want to use the reaper you also need to configure the server middleware.
|
102
|
+
|
103
|
+
[A full example](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/initializers/sidekiq.rb#L12)
|
104
|
+
|
105
|
+
```ruby
|
106
|
+
Sidekiq.configure_server do |config|
|
107
|
+
config.redis = { url: ENV["REDIS_URL"], driver: :hiredis }
|
108
|
+
|
109
|
+
config.client_middleware do |chain|
|
110
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
111
|
+
end
|
112
|
+
|
113
|
+
config.server_middleware do |chain|
|
114
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
115
|
+
end
|
116
|
+
|
117
|
+
SidekiqUniqueJobs::Server.configure(config)
|
118
|
+
end
|
119
|
+
|
120
|
+
Sidekiq.configure_client do |config|
|
121
|
+
config.redis = { url: ENV["REDIS_URL"], driver: :hiredis }
|
122
|
+
|
123
|
+
config.client_middleware do |chain|
|
124
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
125
|
+
end
|
126
|
+
end
|
127
|
+
```
|
128
|
+
|
129
|
+
### Your first worker
|
130
|
+
|
131
|
+
```ruby
|
132
|
+
# frozen_string_literal: true
|
133
|
+
|
134
|
+
class UntilExecutedWorker
|
135
|
+
include Sidekiq::Worker
|
136
|
+
|
137
|
+
sidekiq_options queue: :special,
|
138
|
+
retry: false,
|
139
|
+
lock: :until_executed,
|
140
|
+
lock_info: true,
|
141
|
+
lock_timeout: 0,
|
142
|
+
lock_prefix: "special",
|
143
|
+
lock_ttl: 0,
|
144
|
+
lock_limit: 5
|
145
|
+
|
146
|
+
def perform
|
147
|
+
logger.info("cowboy")
|
148
|
+
sleep(1) # hardcore processing
|
149
|
+
logger.info("beebop")
|
150
|
+
end
|
151
|
+
end
|
102
152
|
|
103
|
-
```bash
|
104
|
-
gem install sidekiq-unique-jobs
|
105
153
|
```
|
106
154
|
|
155
|
+
You can read more about the worker configuration in [Worker Configuration](#worker-configuration) below.
|
156
|
+
|
107
157
|
## Support Me
|
108
158
|
|
109
159
|
Want to show me some ❤️ for the hard work I do on this gem? You can use the following PayPal link: [https://paypal.me/mhenrixon1](https://paypal.me/mhenrixon1). Any amount is welcome and let me tell you it feels good to be appreciated. Even a dollar makes me super excited about all of this.
|
110
160
|
|
161
|
+
## Requirements
|
162
|
+
|
163
|
+
- Sidekiq `>= 5.0` (`>= 5.2` recommended)
|
164
|
+
- Ruby:
|
165
|
+
- MRI `>= 2.5` (`>= 2.6` recommended)
|
166
|
+
- JRuby `>= 9.0` (`>= 9.2` recommended)
|
167
|
+
- Truffleruby
|
168
|
+
- Redis Server `>= 3.2` (`>= 5.0` recommended)
|
169
|
+
- [ActiveJob officially not supported][48]
|
170
|
+
- [redis-namespace officially not supported][49]
|
171
|
+
|
172
|
+
See [Sidekiq requirements][24] for detailed requirements of Sidekiq itself (be sure to check the right sidekiq version).
|
173
|
+
|
111
174
|
## General Information
|
112
175
|
|
113
176
|
See [Interaction w/ Sidekiq](https://github.com/mhenrixon/sidekiq-unique-jobs/wiki/How-this-gem-interacts-with-Sidekiq) on how the gem interacts with Sidekiq.
|
@@ -235,10 +298,10 @@ SidekiqUniqueJobs.config.reaper_timeout #=> 10
|
|
235
298
|
|
236
299
|
The number of seconds to wait for the reaper to finish before raising a TimeoutError. This is done to ensure that the next time we reap isn't getting stuck due to the previous process already running.
|
237
300
|
|
238
|
-
###
|
301
|
+
### lock_prefix
|
239
302
|
|
240
303
|
```ruby
|
241
|
-
SidekiqUniqueJobs.config.
|
304
|
+
SidekiqUniqueJobs.config.lock_prefix #=> "uniquejobs"
|
242
305
|
```
|
243
306
|
|
244
307
|
Use if you want a different key prefix for the keys in redis.
|
@@ -253,11 +316,27 @@ Using lock info will create an additional key for the lock with a json object co
|
|
253
316
|
|
254
317
|
## Worker Configuration
|
255
318
|
|
319
|
+
### lock_info
|
320
|
+
|
321
|
+
Lock info gathers information about a specific lock. It collects things like which `lock_args` where used to compute the `lock_digest` that is used for maintaining uniqueness.
|
322
|
+
|
323
|
+
```ruby
|
324
|
+
sidekiq_options lock_info: false # this is the default, set to true to turn on
|
325
|
+
```
|
326
|
+
|
327
|
+
### lock_prefix
|
328
|
+
|
329
|
+
Use if you want a different key prefix for the keys in redis.
|
330
|
+
|
331
|
+
```ruby
|
332
|
+
sidekiq_options lock_prefix: "uniquejobs" # this is the default value
|
333
|
+
```
|
334
|
+
|
256
335
|
### lock_ttl
|
257
336
|
|
258
337
|
Lock TTL decides how long to wait after the job has been successfully processed before making it possible to reuse that lock.
|
259
338
|
|
260
|
-
|
339
|
+
Starting from `v7` the expiration will take place when the job is pushed to the queue.
|
261
340
|
|
262
341
|
```ruby
|
263
342
|
sidekiq_options lock_ttl: nil # default - don't expire keys
|
@@ -278,6 +357,8 @@ sidekiq_options lock_timeout: nil # lock indefinitely, this process won't contin
|
|
278
357
|
|
279
358
|
This configuration option is slightly misleading. It doesn't disregard the queue on other jobs. Just on itself, this means that a worker that might schedule jobs into multiple queues will be able to have uniqueness enforced on all queues it is pushed to.
|
280
359
|
|
360
|
+
This is mainly intended for `Worker.set(queue: :another).perform_async`.
|
361
|
+
|
281
362
|
```ruby
|
282
363
|
class Worker
|
283
364
|
include Sidekiq::Worker
|
@@ -428,7 +509,7 @@ The last one is log which can be be used with the lock `UntilExecuted` and `Unti
|
|
428
509
|
It is possible for locks to have different conflict strategy for the client and server. This is useful for `:until_and_while_executing`.
|
429
510
|
|
430
511
|
```ruby
|
431
|
-
sidekiq_options lock: :until_and_while_executing,
|
512
|
+
sidekiq_options lock: :until_and_while_executing,
|
432
513
|
on_conflict: { client: :log, server: :reject }
|
433
514
|
```
|
434
515
|
|
@@ -628,7 +709,7 @@ class MyWorker
|
|
628
709
|
end
|
629
710
|
```
|
630
711
|
|
631
|
-
Starting in v5.1, Sidekiq can also fire a global callback when a job dies:
|
712
|
+
Starting in v5.1, Sidekiq can also fire a global callback when a job dies: In version 7, this is handled automatically for you. You don't need to add a death handler, if you configure v7 like in [Add the middleware](#add-the-middleware) you don't have to worry about the below.
|
632
713
|
|
633
714
|
```ruby
|
634
715
|
Sidekiq.configure_server do |config|
|
@@ -641,10 +722,30 @@ end
|
|
641
722
|
|
642
723
|
### Other Sidekiq gems
|
643
724
|
|
725
|
+
#### apartment-sidekiq
|
726
|
+
|
727
|
+
It was reported in [#536](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/536) that the order of the Sidekiq middleware needs to be as follows.
|
728
|
+
|
729
|
+
```ruby
|
730
|
+
Sidekiq.client_middleware do |chain|
|
731
|
+
chain.add Apartment::Sidekiq::Middleware::Client
|
732
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
733
|
+
end
|
734
|
+
|
735
|
+
Sidekiq.server_middleware do |chain|
|
736
|
+
chain.add Apartment::Sidekiq::Middleware::Server
|
737
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
738
|
+
end
|
739
|
+
```
|
740
|
+
|
741
|
+
The reason being that this gem needs to be configured AFTER the apartment gem or the apartment will not be able to be considered for uniqueness
|
742
|
+
|
644
743
|
#### sidekiq-global_id
|
645
744
|
|
646
745
|
It was reported in [#235](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/235) that the order of the Sidekiq middleware needs to be as follows.
|
647
746
|
|
747
|
+
For a working setup check the following [file](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/sidekiq.rb#L12).
|
748
|
+
|
648
749
|
```ruby
|
649
750
|
Sidekiq.client_middleware do |chain|
|
650
751
|
chain.add Sidekiq::GlobalId::ClientMiddleware
|
@@ -652,12 +753,44 @@ Sidekiq.client_middleware do |chain|
|
|
652
753
|
end
|
653
754
|
|
654
755
|
Sidekiq.server_middleware do |chain|
|
655
|
-
chain.add SidekiqUniqueJobs::Middleware::Server
|
656
756
|
chain.add Sidekiq::GlobalId::ServerMiddleware
|
757
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
758
|
+
end
|
759
|
+
```
|
760
|
+
|
761
|
+
The reason for this is that the global id needs to be set before the unique jobs middleware runs. Otherwise that won't be available for uniqueness.
|
762
|
+
|
763
|
+
#### sidekiq-status
|
764
|
+
|
765
|
+
It was reported in [#564](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/564) that the order of the middleware needs to be as follows.
|
766
|
+
|
767
|
+
```ruby
|
768
|
+
# Thanks to @ArturT for the correction
|
769
|
+
|
770
|
+
Sidekiq.configure_server do |config|
|
771
|
+
config.client_middleware do |chain|
|
772
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
773
|
+
chain.add Sidekiq::Status::ClientMiddleware, expiration: 30.minutes
|
774
|
+
end
|
775
|
+
|
776
|
+
config.server_middleware do |chain|
|
777
|
+
chain.add Sidekiq::Status::ServerMiddleware, expiration: 30.minutes
|
778
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
779
|
+
end
|
780
|
+
|
781
|
+
SidekiqUniqueJobs::Server.configure(config)
|
782
|
+
end
|
783
|
+
|
784
|
+
|
785
|
+
Sidekiq.configure_client do |config|
|
786
|
+
config.client_middleware do |chain|
|
787
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
788
|
+
chain.add Sidekiq::Status::ClientMiddleware, expiration: 30.minutes
|
789
|
+
end
|
657
790
|
end
|
658
791
|
```
|
659
792
|
|
660
|
-
|
793
|
+
The reason for this is that if a job is duplicated it shouldn't end up with the status middleware at all. Status is just a monitor so to prevent clashes, leftovers and ensure cleanup. The status middleware should run after uniqueness on client and before on server. This will lead to less surprises.
|
661
794
|
|
662
795
|
## Debugging
|
663
796
|
|
@@ -728,7 +861,6 @@ If you are not using RSpec (a lot of people prefer minitest or test unit) you ca
|
|
728
861
|
assert SidekiqUniqueJobs.validate_worker!(BadWorker.get_sidekiq_options)
|
729
862
|
```
|
730
863
|
|
731
|
-
|
732
864
|
### Uniqueness
|
733
865
|
|
734
866
|
This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I there for decided to do what Mike is doing for sidekiq enterprise. Read the section about unique jobs: [Enterprise unique jobs][]
|