sidekiq-unique-jobs 7.0.0.beta29 → 7.0.0
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of sidekiq-unique-jobs might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +44 -36
- data/README.md +139 -22
- data/lib/sidekiq-unique-jobs.rb +0 -2
- data/lib/sidekiq_unique_jobs/changelog.rb +1 -1
- data/lib/sidekiq_unique_jobs/constants.rb +1 -0
- data/lib/sidekiq_unique_jobs/digests.rb +1 -1
- data/lib/sidekiq_unique_jobs/lock_config.rb +2 -0
- data/lib/sidekiq_unique_jobs/lua/lock.lua +1 -2
- data/lib/sidekiq_unique_jobs/lua/reap_orphans.lua +8 -7
- data/lib/sidekiq_unique_jobs/lua/shared/_find_digest_in_process_set.lua +9 -2
- data/lib/sidekiq_unique_jobs/middleware.rb +0 -57
- data/lib/sidekiq_unique_jobs/on_conflict/replace.rb +9 -8
- data/lib/sidekiq_unique_jobs/on_conflict/reschedule.rb +1 -1
- data/lib/sidekiq_unique_jobs/orphans/lua_reaper.rb +1 -1
- data/lib/sidekiq_unique_jobs/orphans/manager.rb +6 -1
- data/lib/sidekiq_unique_jobs/orphans/reaper.rb +10 -0
- data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +9 -2
- data/lib/sidekiq_unique_jobs/redis/entity.rb +9 -3
- data/lib/sidekiq_unique_jobs/version.rb +1 -1
- data/lib/sidekiq_unique_jobs/web.rb +10 -4
- metadata +9 -7
- data/lib/sidekiq_unique_jobs/profiler.rb +0 -55
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 3a0ef0c8257f0c79b137bab502492428dcbd9060b44d1b1fa754e5c591ef9185
|
4
|
+
data.tar.gz: '03380b590a1e077f429e2b9bce284716b6f6d891cf35a1eeb7c01e82b2b23782'
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: b7c5881985fd2b2d96a2af3ad29e39e56a1df353b888e0a10720d4bee03cebeafee05e327b5fe54f809b0a1e567f0466e5754bee4e5e0179d14ce232a39a1e8b
|
7
|
+
data.tar.gz: 4fa3667cb51c5b44791b10f3672dbebd5f3733adae80b512775629a29d3feed4e67c5df8ce20993fa89d4810a6ca9aef7c0922e1f03bd305e0421679316ba3f9
|
data/CHANGELOG.md
CHANGED
@@ -1,20 +1,21 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
-
## [
|
3
|
+
## [v7.0.0.beta29](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta29) (2021-01-16)
|
4
4
|
|
5
|
-
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta28...
|
5
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta28...v7.0.0.beta29)
|
6
6
|
|
7
|
-
**
|
7
|
+
**Fixed bugs:**
|
8
8
|
|
9
9
|
- Ruby Reaper active check incorrect [\#557](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/557)
|
10
|
+
- Routes with authentication should work with web [\#562](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/562) ([mhenrixon](https://github.com/mhenrixon))
|
10
11
|
|
11
|
-
|
12
|
+
**Closed issues:**
|
12
13
|
|
13
|
-
[
|
14
|
+
- Can't add the lock tab o nthe website when there is authentication through devise [\#561](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/561)
|
14
15
|
|
15
|
-
|
16
|
+
## [v7.0.0.beta28](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta28) (2021-01-07)
|
16
17
|
|
17
|
-
|
18
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta27...v7.0.0.beta28)
|
18
19
|
|
19
20
|
**Fixed bugs:**
|
20
21
|
|
@@ -28,6 +29,10 @@
|
|
28
29
|
|
29
30
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta26...v7.0.0.beta27)
|
30
31
|
|
32
|
+
**Implemented enhancements:**
|
33
|
+
|
34
|
+
- Adds coverage for regression purposes [\#550](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/550) ([mhenrixon](https://github.com/mhenrixon))
|
35
|
+
|
31
36
|
**Fixed bugs:**
|
32
37
|
|
33
38
|
- Rename lock\_args to lock\_args\_method [\#551](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/551) ([mhenrixon](https://github.com/mhenrixon))
|
@@ -66,7 +71,6 @@
|
|
66
71
|
**Implemented enhancements:**
|
67
72
|
|
68
73
|
- Bump rubocop [\#539](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/539) ([mhenrixon](https://github.com/mhenrixon))
|
69
|
-
- Support both instance method and class method [\#527](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/527) ([mhenrixon](https://github.com/mhenrixon))
|
70
74
|
|
71
75
|
**Fixed bugs:**
|
72
76
|
|
@@ -85,6 +89,10 @@
|
|
85
89
|
|
86
90
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta23...v7.0.0.beta24)
|
87
91
|
|
92
|
+
**Implemented enhancements:**
|
93
|
+
|
94
|
+
- Support both instance method and class method [\#527](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/527) ([mhenrixon](https://github.com/mhenrixon))
|
95
|
+
|
88
96
|
**Closed issues:**
|
89
97
|
|
90
98
|
- Leaked keys in version 5.0.10 [\#519](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/519)
|
@@ -148,10 +156,6 @@
|
|
148
156
|
|
149
157
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta18...v7.0.0.beta19)
|
150
158
|
|
151
|
-
**Implemented enhancements:**
|
152
|
-
|
153
|
-
- Try GitHub actions [\#505](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/505) ([mhenrixon](https://github.com/mhenrixon))
|
154
|
-
|
155
159
|
**Fixed bugs:**
|
156
160
|
|
157
161
|
- Expire reaper when not checking in [\#508](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/508) ([mhenrixon](https://github.com/mhenrixon))
|
@@ -168,6 +172,10 @@
|
|
168
172
|
|
169
173
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta16...v7.0.0.beta17)
|
170
174
|
|
175
|
+
**Implemented enhancements:**
|
176
|
+
|
177
|
+
- Try GitHub actions [\#505](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/505) ([mhenrixon](https://github.com/mhenrixon))
|
178
|
+
|
171
179
|
**Fixed bugs:**
|
172
180
|
|
173
181
|
- Deep stringify worker options to account for hash in on\_conflict [\#506](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/506) ([jasonbekolay](https://github.com/jasonbekolay))
|
@@ -178,6 +186,7 @@
|
|
178
186
|
|
179
187
|
**Fixed bugs:**
|
180
188
|
|
189
|
+
- Deprecate configuration options with `default\_` [\#504](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/504) ([mhenrixon](https://github.com/mhenrixon))
|
181
190
|
- Fix access to both server and client conflict [\#503](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/503) ([mhenrixon](https://github.com/mhenrixon))
|
182
191
|
|
183
192
|
**Closed issues:**
|
@@ -207,7 +216,6 @@
|
|
207
216
|
- Brpoplpush::RedisScript::LuaError: WRONGTYPE Operation against a key holding the wrong kind of value [\#491](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/491)
|
208
217
|
- Lua script bug [\#489](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/489)
|
209
218
|
- Reaper will delete locks for running jobs [\#488](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/488)
|
210
|
-
- Deprecate configuration options with `default\_` [\#504](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/504) ([mhenrixon](https://github.com/mhenrixon))
|
211
219
|
- Fix access to hash members [\#496](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/496) ([mhenrixon](https://github.com/mhenrixon))
|
212
220
|
- Fix cursor assignment [\#494](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/494) ([mhenrixon](https://github.com/mhenrixon))
|
213
221
|
- Prevent reaping of active jobs [\#493](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/493) ([mhenrixon](https://github.com/mhenrixon))
|
@@ -258,13 +266,13 @@
|
|
258
266
|
- Please keep some recent versions on rubygems.org [\#478](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/478)
|
259
267
|
- validate\_worker! throws error [\#466](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/466)
|
260
268
|
|
261
|
-
|
269
|
+
**Merged pull requests:**
|
262
270
|
|
263
|
-
[
|
271
|
+
- Allow v7 configuration to work [\#479](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/479) ([mhenrixon](https://github.com/mhenrixon))
|
264
272
|
|
265
|
-
|
273
|
+
## [v7.0.0.beta11](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta11) (2020-03-21)
|
266
274
|
|
267
|
-
|
275
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta10...v7.0.0.beta11)
|
268
276
|
|
269
277
|
## [v7.0.0.beta10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta10) (2020-03-21)
|
270
278
|
|
@@ -280,11 +288,11 @@
|
|
280
288
|
**Fixed bugs:**
|
281
289
|
|
282
290
|
- With v6.0.18, Sidekiq doesn't run at all [\#471](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/471)
|
291
|
+
- Only configure RSpec when constant is defined [\#477](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/477) ([mhenrixon](https://github.com/mhenrixon))
|
283
292
|
- Fix errors\_as\_string on lock\_config.rb [\#469](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/469) ([donaldpiret](https://github.com/donaldpiret))
|
284
293
|
|
285
294
|
**Merged pull requests:**
|
286
295
|
|
287
|
-
- Allow v7 configuration to work [\#479](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/479) ([mhenrixon](https://github.com/mhenrixon))
|
288
296
|
- README: Use SVG badges ✨ [\#470](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/470) ([olleolleolle](https://github.com/olleolleolle))
|
289
297
|
- remove deprecated/broken OptionsWithFallback\#unique\_type [\#435](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/435) ([zvkemp](https://github.com/zvkemp))
|
290
298
|
|
@@ -295,6 +303,7 @@
|
|
295
303
|
**Fixed bugs:**
|
296
304
|
|
297
305
|
- Fix access to lock [\#476](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/476) ([mhenrixon](https://github.com/mhenrixon))
|
306
|
+
- Backport v7 fix for conflicts [\#461](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/461) ([mhenrixon](https://github.com/mhenrixon))
|
298
307
|
|
299
308
|
## [v7.0.0.beta9](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta9) (2019-12-04)
|
300
309
|
|
@@ -305,10 +314,6 @@
|
|
305
314
|
- Keys without TTL [\#417](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/417)
|
306
315
|
- Various changes to test and verify reliability [\#463](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/463) ([mhenrixon](https://github.com/mhenrixon))
|
307
316
|
|
308
|
-
**Fixed bugs:**
|
309
|
-
|
310
|
-
- Make deletion compatible with redis-namespace [\#452](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/452) ([mhenrixon](https://github.com/mhenrixon))
|
311
|
-
|
312
317
|
**Closed issues:**
|
313
318
|
|
314
319
|
- until\_and\_while\_executing with sidekiq pro `reliable\_scheduler!` [\#411](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/411)
|
@@ -339,7 +344,6 @@
|
|
339
344
|
**Fixed bugs:**
|
340
345
|
|
341
346
|
- A worker with "While Executing" lock and "Reschedule" strategy is rescheduled forever [\#457](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/457)
|
342
|
-
- Backport v7 fix for conflicts [\#461](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/461) ([mhenrixon](https://github.com/mhenrixon))
|
343
347
|
- Prevent callbacks from preventing locks [\#460](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/460) ([mhenrixon](https://github.com/mhenrixon))
|
344
348
|
|
345
349
|
## [v7.0.0.beta6](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta6) (2019-11-28)
|
@@ -381,8 +385,8 @@
|
|
381
385
|
**Fixed bugs:**
|
382
386
|
|
383
387
|
- Prevent multiple reapers [\#453](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/453) ([mhenrixon](https://github.com/mhenrixon))
|
388
|
+
- Make deletion compatible with redis-namespace [\#452](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/452) ([mhenrixon](https://github.com/mhenrixon))
|
384
389
|
- Make sure server process stays locked [\#448](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/448) ([mhenrixon](https://github.com/mhenrixon))
|
385
|
-
- Make the ruby reaper plain ruby [\#443](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/443) ([mhenrixon](https://github.com/mhenrixon))
|
386
390
|
|
387
391
|
**Merged pull requests:**
|
388
392
|
|
@@ -420,7 +424,6 @@
|
|
420
424
|
|
421
425
|
- Brpoplpush redis script [\#434](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/434) ([mhenrixon](https://github.com/mhenrixon))
|
422
426
|
- Drop support for almost EOL ruby 2.4 [\#433](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/433) ([mhenrixon](https://github.com/mhenrixon))
|
423
|
-
- Bump ruby versions in Travis CI [\#425](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/425) ([giraffate](https://github.com/giraffate))
|
424
427
|
|
425
428
|
**Fixed bugs:**
|
426
429
|
|
@@ -440,6 +443,7 @@
|
|
440
443
|
|
441
444
|
**Fixed bugs:**
|
442
445
|
|
446
|
+
- Make the ruby reaper plain ruby [\#443](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/443) ([mhenrixon](https://github.com/mhenrixon))
|
443
447
|
- Pass redis\_version into scripts [\#431](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/431) ([mhenrixon](https://github.com/mhenrixon))
|
444
448
|
|
445
449
|
**Closed issues:**
|
@@ -452,11 +456,13 @@
|
|
452
456
|
|
453
457
|
**Implemented enhancements:**
|
454
458
|
|
459
|
+
- Bump ruby versions in Travis CI [\#425](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/425) ([giraffate](https://github.com/giraffate))
|
455
460
|
- Allow lock info to be configured from worker [\#407](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/407) ([mhenrixon](https://github.com/mhenrixon))
|
456
461
|
- Validate worker configuration [\#406](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/406) ([mhenrixon](https://github.com/mhenrixon))
|
457
462
|
- Codeclimate configuration [\#405](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/405) ([mhenrixon](https://github.com/mhenrixon))
|
458
463
|
- Ensure uniquejobs namespace [\#400](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/400) ([mhenrixon](https://github.com/mhenrixon))
|
459
464
|
- Prepare for version 7 [\#387](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/387) ([mhenrixon](https://github.com/mhenrixon))
|
465
|
+
- Provide some configuration DSL for custom Strategies and Locks [\#383](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/383) ([mberlanda](https://github.com/mberlanda))
|
460
466
|
|
461
467
|
**Fixed bugs:**
|
462
468
|
|
@@ -532,7 +538,6 @@
|
|
532
538
|
|
533
539
|
**Implemented enhancements:**
|
534
540
|
|
535
|
-
- Provide some configuration DSL for custom Strategies and Locks [\#383](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/383) ([mberlanda](https://github.com/mberlanda))
|
536
541
|
- Reduce leftover keys [\#374](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/374) ([mhenrixon](https://github.com/mhenrixon))
|
537
542
|
- Prepare for sidekiq 6 [\#373](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/373) ([mhenrixon](https://github.com/mhenrixon))
|
538
543
|
|
@@ -581,10 +586,6 @@
|
|
581
586
|
|
582
587
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.7...v6.0.8)
|
583
588
|
|
584
|
-
**Fixed bugs:**
|
585
|
-
|
586
|
-
- Close \#359 [\#364](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/364) ([mhenrixon](https://github.com/mhenrixon))
|
587
|
-
|
588
589
|
**Closed issues:**
|
589
590
|
|
590
591
|
- Automatic unlock of jobs [\#362](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/362)
|
@@ -607,6 +608,7 @@
|
|
607
608
|
**Fixed bugs:**
|
608
609
|
|
609
610
|
- Version 5: Job ID Hash Entries Not Removed if Unique Key Expires [\#346](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/346)
|
611
|
+
- Close \#359 [\#364](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/364) ([mhenrixon](https://github.com/mhenrixon))
|
610
612
|
- Move the lpush last [\#354](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/354) ([mhenrixon](https://github.com/mhenrixon))
|
611
613
|
- Convert expiration time to integer [\#330](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/330) ([dareddov](https://github.com/dareddov))
|
612
614
|
|
@@ -621,7 +623,6 @@
|
|
621
623
|
- fix CHANGELOG syntax [\#344](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/344) ([timoschilling](https://github.com/timoschilling))
|
622
624
|
- Define Config class inside SidekiqUniqueJobs module [\#343](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/343) ([Slike9](https://github.com/Slike9))
|
623
625
|
- fix readme testing section [\#333](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/333) ([edmartins](https://github.com/edmartins))
|
624
|
-
- Fix typo in documentation \[ci-skip\] [\#327](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/327) ([mhenrixon](https://github.com/mhenrixon))
|
625
626
|
|
626
627
|
## [v5.0.11](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.11) (2018-11-19)
|
627
628
|
|
@@ -666,6 +667,10 @@
|
|
666
667
|
|
667
668
|
- Why is lock\_timeout: nil VERY DANGEROUS? [\#313](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/313)
|
668
669
|
|
670
|
+
**Merged pull requests:**
|
671
|
+
|
672
|
+
- Fix typo in documentation \[ci-skip\] [\#327](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/327) ([mhenrixon](https://github.com/mhenrixon))
|
673
|
+
|
669
674
|
## [v6.0.4](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.4) (2018-08-02)
|
670
675
|
|
671
676
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.3...v6.0.4)
|
@@ -792,13 +797,16 @@
|
|
792
797
|
|
793
798
|
**Implemented enhancements:**
|
794
799
|
|
795
|
-
- Prepare for v6 [\#286](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/286) ([mhenrixon](https://github.com/mhenrixon))
|
796
800
|
- Only unlock not delete [\#285](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/285) ([mhenrixon](https://github.com/mhenrixon))
|
797
801
|
|
798
802
|
## [v6.0.0.rc3](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0.rc3) (2018-06-29)
|
799
803
|
|
800
804
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.0.rc2...v6.0.0.rc3)
|
801
805
|
|
806
|
+
**Implemented enhancements:**
|
807
|
+
|
808
|
+
- Prepare for v6 [\#286](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/286) ([mhenrixon](https://github.com/mhenrixon))
|
809
|
+
|
802
810
|
**Fixed bugs:**
|
803
811
|
|
804
812
|
- Fix waiting for locks [\#284](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/284) ([mhenrixon](https://github.com/mhenrixon))
|
@@ -811,6 +819,7 @@
|
|
811
819
|
|
812
820
|
- Within tests: workers enqueued in the future don't clear their unique locks after being drained/executed [\#254](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/254)
|
813
821
|
- Unexpected behavior with until\_executed [\#250](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/250)
|
822
|
+
- Fix UntilExpired [\#278](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/278) ([mhenrixon](https://github.com/mhenrixon))
|
814
823
|
|
815
824
|
**Fixed bugs:**
|
816
825
|
|
@@ -866,7 +875,6 @@
|
|
866
875
|
|
867
876
|
**Implemented enhancements:**
|
868
877
|
|
869
|
-
- Fix UntilExpired [\#278](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/278) ([mhenrixon](https://github.com/mhenrixon))
|
870
878
|
- Until and while executing [\#271](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/271) ([mhenrixon](https://github.com/mhenrixon))
|
871
879
|
- Solidify master [\#270](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/270) ([mhenrixon](https://github.com/mhenrixon))
|
872
880
|
- Minor adjustments [\#268](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/268) ([mhenrixon](https://github.com/mhenrixon))
|
@@ -953,6 +961,7 @@
|
|
953
961
|
|
954
962
|
- Different unique arguments depending on lock type [\#203](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/203)
|
955
963
|
- Strategy until\_and\_while\_executing not working properly [\#199](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/199)
|
964
|
+
- while\_executing working wrong [\#193](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/193)
|
956
965
|
|
957
966
|
## [v5.0.5](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.5) (2017-04-23)
|
958
967
|
|
@@ -1014,7 +1023,6 @@
|
|
1014
1023
|
|
1015
1024
|
- Could not find a valid gem 'sidekiq-unique-jobs' \(= 3.0.15\) in any repository [\#197](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/197)
|
1016
1025
|
- `uniquejobs` hash doesn't get cleaned up [\#195](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/195)
|
1017
|
-
- while\_executing working wrong [\#193](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/193)
|
1018
1026
|
- Code block under "Finer Control over Uniqueness" in your documentation might have the wrong option specified [\#191](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/191)
|
1019
1027
|
- not able to run test without live Redis [\#186](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/186)
|
1020
1028
|
- unique while not sucessfully completed? [\#185](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/185)
|
@@ -1184,6 +1192,7 @@
|
|
1184
1192
|
|
1185
1193
|
**Merged pull requests:**
|
1186
1194
|
|
1195
|
+
- Forces to look for testing namespace in Sidekiq and not his ancestors [\#129](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/129) ([antek-drzewiecki](https://github.com/antek-drzewiecki))
|
1187
1196
|
- Fix outdated phrasing and add test coverage to readme [\#123](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/123) ([albertyw](https://github.com/albertyw))
|
1188
1197
|
|
1189
1198
|
## [v4.0.4](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.4) (2015-10-09)
|
@@ -1274,13 +1283,11 @@
|
|
1274
1283
|
|
1275
1284
|
**Merged pull requests:**
|
1276
1285
|
|
1277
|
-
- Forces to look for testing namespace in Sidekiq and not his ancestors [\#129](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/129) ([antek-drzewiecki](https://github.com/antek-drzewiecki))
|
1278
1286
|
- Allow job with jid matching unique lock to pass unique check [\#105](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/105) ([deltaroe](https://github.com/deltaroe))
|
1279
1287
|
- Prevent Jobs from deleting mutexes they don't own [\#96](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/96) ([pik](https://github.com/pik))
|
1280
1288
|
- Add after unlock hook [\#92](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/92) ([HParker](https://github.com/HParker))
|
1281
1289
|
- Do not unlock on sidekiq shutdown [\#87](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/87) ([deltaroe](https://github.com/deltaroe))
|
1282
1290
|
- Remove no-op code, protect global space from test code [\#86](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/86) ([stevenjonescgm](https://github.com/stevenjonescgm))
|
1283
|
-
- Remove unique lock when executing and clearing jobs in sidekiq fake mode [\#83](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/83) ([crberube](https://github.com/crberube))
|
1284
1291
|
- Fix tests. Tests with latest sidekiq versions and ruby versions [\#82](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/82) ([simonoff](https://github.com/simonoff))
|
1285
1292
|
- Duplicate Payload logging configuration [\#81](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/81) ([jprincipe](https://github.com/jprincipe))
|
1286
1293
|
- output log if not unique [\#79](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/79) ([sonots](https://github.com/sonots))
|
@@ -1327,6 +1334,7 @@
|
|
1327
1334
|
|
1328
1335
|
**Merged pull requests:**
|
1329
1336
|
|
1337
|
+
- Remove unique lock when executing and clearing jobs in sidekiq fake mode [\#83](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/83) ([crberube](https://github.com/crberube))
|
1330
1338
|
- Refactoring connectors to use them in client and server [\#56](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/56) ([salrepe](https://github.com/salrepe))
|
1331
1339
|
- Fix dependency error in inline testing connector [\#54](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/54) ([salrepe](https://github.com/salrepe))
|
1332
1340
|
- Add extension to Sidekiq API that is uniqueness-aware [\#52](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/52) ([rickenharp](https://github.com/rickenharp))
|
data/README.md
CHANGED
@@ -3,9 +3,12 @@
|
|
3
3
|
<!-- MarkdownTOC -->
|
4
4
|
|
5
5
|
- [Introduction](#introduction)
|
6
|
-
- [
|
7
|
-
- [Installation](#installation)
|
6
|
+
- [Usage](#usage)
|
7
|
+
- [Installation](#installation)
|
8
|
+
- [Add the middleware](#add-the-middleware)
|
9
|
+
- [Your first worker](#your-first-worker)
|
8
10
|
- [Support Me](#support-me)
|
11
|
+
- [Requirements](#requirements)
|
9
12
|
- [General Information](#general-information)
|
10
13
|
- [Global Configuration](#global-configuration)
|
11
14
|
- [debug_lua](#debug_lua)
|
@@ -21,6 +24,8 @@
|
|
21
24
|
- [lock_prefix](#lock_prefix)
|
22
25
|
- [lock_info](#lock_info)
|
23
26
|
- [Worker Configuration](#worker-configuration)
|
27
|
+
- [lock_info](#lock_info-1)
|
28
|
+
- [lock_prefix](#lock_prefix-1)
|
24
29
|
- [lock_ttl](#lock_ttl-1)
|
25
30
|
- [lock_timeout](#lock_timeout-1)
|
26
31
|
- [unique_across_queues](#unique_across_queues)
|
@@ -39,13 +44,15 @@
|
|
39
44
|
- [replace](#replace)
|
40
45
|
- [Reschedule](#reschedule)
|
41
46
|
- [Custom Strategies](#custom-strategies)
|
42
|
-
- [Usage](#usage)
|
47
|
+
- [Usage](#usage-1)
|
43
48
|
- [Finer Control over Uniqueness](#finer-control-over-uniqueness)
|
44
49
|
- [After Unlock Callback](#after-unlock-callback)
|
45
50
|
- [Logging](#logging)
|
46
51
|
- [Cleanup Dead Locks](#cleanup-dead-locks)
|
47
52
|
- [Other Sidekiq gems](#other-sidekiq-gems)
|
53
|
+
- [apartment-sidekiq](#apartment-sidekiq)
|
48
54
|
- [sidekiq-global_id](#sidekiq-global_id)
|
55
|
+
- [sidekiq-status](#sidekiq-status)
|
49
56
|
- [Debugging](#debugging)
|
50
57
|
- [Sidekiq Web](#sidekiq-web)
|
51
58
|
- [Show Locks](#show-locks)
|
@@ -61,7 +68,7 @@
|
|
61
68
|
|
62
69
|
## Introduction
|
63
70
|
|
64
|
-
This gem adds unique constraints to the sidekiq queues. The uniqueness is achieved by acquiring locks for a hash of a queue name, a worker class, and job's arguments.
|
71
|
+
This gem adds unique constraints to the sidekiq queues. The uniqueness is achieved by acquiring locks for a hash of a queue name, a worker class, and job's arguments. By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen `on_conflict`strategy.
|
65
72
|
|
66
73
|
This is the documentation for the master branch. You can find the documentation for each release by navigating to its tag.
|
67
74
|
|
@@ -71,20 +78,9 @@ Here are links to some of the old versions
|
|
71
78
|
- [v5.0.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10)
|
72
79
|
- [v4.0.18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18)
|
73
80
|
|
74
|
-
##
|
75
|
-
|
76
|
-
- Sidekiq `>= 4.0` (`>= 5.2` recommended)
|
77
|
-
- Ruby:
|
78
|
-
- MRI `>= 2.3` (`>= 2.5` recommended)
|
79
|
-
- JRuby `>= 9.0` (`>= 9.2` recommended)
|
80
|
-
- Truffleruby
|
81
|
-
- Redis Server `>= 3.0.2` (`>= 3.2` recommended)
|
82
|
-
- [ActiveJob officially not supported][48]
|
83
|
-
- [redis-namespace officially not supported][49]
|
84
|
-
|
85
|
-
See [Sidekiq requirements][24] for detailed requirements of Sidekiq itself (be sure to check the right sidekiq version).
|
81
|
+
## Usage
|
86
82
|
|
87
|
-
|
83
|
+
### Installation
|
88
84
|
|
89
85
|
Add this line to your application's Gemfile:
|
90
86
|
|
@@ -98,16 +94,81 @@ And then execute:
|
|
98
94
|
bundle
|
99
95
|
```
|
100
96
|
|
101
|
-
|
97
|
+
### Add the middleware
|
102
98
|
|
103
|
-
|
104
|
-
|
99
|
+
Before v7, the middleware was configured automatically. Since some people reported issues with other gems (see [Other Sidekiq Gems](#other-sidekiq-gems)) it was decided to give full control over to the user.
|
100
|
+
|
101
|
+
[A full and hopefully working example](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/sidekiq.rb#L12)
|
102
|
+
|
103
|
+
```ruby
|
104
|
+
Sidekiq.configure_server do |config|
|
105
|
+
config.redis = { url: ENV["REDIS_URL"], driver: :hiredis }
|
106
|
+
|
107
|
+
config.server_middleware do |chain|
|
108
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
109
|
+
end
|
110
|
+
|
111
|
+
config.error_handlers << ->(ex, ctx_hash) { p ex, ctx_hash }
|
112
|
+
config.death_handlers << lambda do |job, _ex|
|
113
|
+
digest = job["lock_digest"]
|
114
|
+
SidekiqUniqueJobs::Digests.delete_by_digest(digest) if digest
|
115
|
+
end
|
116
|
+
end
|
117
|
+
|
118
|
+
Sidekiq.configure_client do |config|
|
119
|
+
config.redis = { url: ENV["REDIS_URL"], driver: :hiredis }
|
120
|
+
|
121
|
+
config.client_middleware do |chain|
|
122
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
123
|
+
end
|
124
|
+
end
|
105
125
|
```
|
106
126
|
|
127
|
+
### Your first worker
|
128
|
+
|
129
|
+
```ruby
|
130
|
+
# frozen_string_literal: true
|
131
|
+
|
132
|
+
class UntilExecutedWorker
|
133
|
+
include Sidekiq::Worker
|
134
|
+
|
135
|
+
sidekiq_options queue: :special,
|
136
|
+
retry: false,
|
137
|
+
lock: :until_executed,
|
138
|
+
lock_info: true,
|
139
|
+
lock_timeout: 0,
|
140
|
+
lock_prefix: "special",
|
141
|
+
lock_ttl: 0,
|
142
|
+
lock_limit: 5
|
143
|
+
|
144
|
+
def perform
|
145
|
+
logger.info("cowboy")
|
146
|
+
sleep(1) # hardcore processing
|
147
|
+
logger.info("beebop")
|
148
|
+
end
|
149
|
+
end
|
150
|
+
|
151
|
+
```
|
152
|
+
|
153
|
+
You can read more about the worker configuration in [Worker Configuration](#worker-configuration) below.
|
154
|
+
|
107
155
|
## Support Me
|
108
156
|
|
109
157
|
Want to show me some ❤️ for the hard work I do on this gem? You can use the following PayPal link: [https://paypal.me/mhenrixon1](https://paypal.me/mhenrixon1). Any amount is welcome and let me tell you it feels good to be appreciated. Even a dollar makes me super excited about all of this.
|
110
158
|
|
159
|
+
## Requirements
|
160
|
+
|
161
|
+
- Sidekiq `>= 4.0` (`>= 5.2` recommended)
|
162
|
+
- Ruby:
|
163
|
+
- MRI `>= 2.5` (`>= 2.6` recommended)
|
164
|
+
- JRuby `>= 9.0` (`>= 9.2` recommended)
|
165
|
+
- Truffleruby
|
166
|
+
- Redis Server `>= 3.0.2` (`>= 3.2` recommended)
|
167
|
+
- [ActiveJob officially not supported][48]
|
168
|
+
- [redis-namespace officially not supported][49]
|
169
|
+
|
170
|
+
See [Sidekiq requirements][24] for detailed requirements of Sidekiq itself (be sure to check the right sidekiq version).
|
171
|
+
|
111
172
|
## General Information
|
112
173
|
|
113
174
|
See [Interaction w/ Sidekiq](https://github.com/mhenrixon/sidekiq-unique-jobs/wiki/How-this-gem-interacts-with-Sidekiq) on how the gem interacts with Sidekiq.
|
@@ -253,11 +314,27 @@ Using lock info will create an additional key for the lock with a json object co
|
|
253
314
|
|
254
315
|
## Worker Configuration
|
255
316
|
|
317
|
+
### lock_info
|
318
|
+
|
319
|
+
Lock info gathers information about a specific lock. It collects things like which `lock_args` where used to compute the `lock_digest` that is used for maintaining uniqueness.
|
320
|
+
|
321
|
+
```ruby
|
322
|
+
sidekiq_options lock_info: false # this is the default, set to true to turn on
|
323
|
+
```
|
324
|
+
|
325
|
+
### lock_prefix
|
326
|
+
|
327
|
+
Use if you want a different key prefix for the keys in redis.
|
328
|
+
|
329
|
+
```ruby
|
330
|
+
sidekiq_options lock_prefix: "uniquejobs" # this is the default value
|
331
|
+
```
|
332
|
+
|
256
333
|
### lock_ttl
|
257
334
|
|
258
335
|
Lock TTL decides how long to wait after the job has been successfully processed before making it possible to reuse that lock.
|
259
336
|
|
260
|
-
|
337
|
+
Starting from `v7` the expiration will take place when the job is pushed to the queue.
|
261
338
|
|
262
339
|
```ruby
|
263
340
|
sidekiq_options lock_ttl: nil # default - don't expire keys
|
@@ -278,6 +355,8 @@ sidekiq_options lock_timeout: nil # lock indefinitely, this process won't contin
|
|
278
355
|
|
279
356
|
This configuration option is slightly misleading. It doesn't disregard the queue on other jobs. Just on itself, this means that a worker that might schedule jobs into multiple queues will be able to have uniqueness enforced on all queues it is pushed to.
|
280
357
|
|
358
|
+
This is mainly intended for `Worker.set(queue: :another).perform_async`.
|
359
|
+
|
281
360
|
```ruby
|
282
361
|
class Worker
|
283
362
|
include Sidekiq::Worker
|
@@ -641,10 +720,30 @@ end
|
|
641
720
|
|
642
721
|
### Other Sidekiq gems
|
643
722
|
|
723
|
+
#### apartment-sidekiq
|
724
|
+
|
725
|
+
It was reported in [#536](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/536) that the order of the Sidekiq middleware needs to be as follows.
|
726
|
+
|
727
|
+
```ruby
|
728
|
+
Sidekiq.client_middleware do |chain|
|
729
|
+
chain.add Apartment::Sidekiq::Middleware::Client
|
730
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
731
|
+
end
|
732
|
+
|
733
|
+
Sidekiq.server_middleware do |chain|
|
734
|
+
chain.add Apartment::Sidekiq::Middleware::Server
|
735
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
736
|
+
end
|
737
|
+
```
|
738
|
+
|
739
|
+
The reason being that this gem needs to be configured AFTER the apartment gem or the apartment will not be able to be considered for uniqueness
|
740
|
+
|
644
741
|
#### sidekiq-global_id
|
645
742
|
|
646
743
|
It was reported in [#235](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/235) that the order of the Sidekiq middleware needs to be as follows.
|
647
744
|
|
745
|
+
For a working setup check the following [file](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/sidekiq.rb#L12).
|
746
|
+
|
648
747
|
```ruby
|
649
748
|
Sidekiq.client_middleware do |chain|
|
650
749
|
chain.add Sidekiq::GlobalId::ClientMiddleware
|
@@ -657,7 +756,25 @@ Sidekiq.server_middleware do |chain|
|
|
657
756
|
end
|
658
757
|
```
|
659
758
|
|
660
|
-
|
759
|
+
The reason for this is that the global id needs to be set before the unique jobs middleware runs. Otherwise that won't be available for uniqueness.
|
760
|
+
|
761
|
+
#### sidekiq-status
|
762
|
+
|
763
|
+
It was reported in [#564](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/564) that the order of the middleware needs to be as follows.
|
764
|
+
|
765
|
+
```ruby
|
766
|
+
Sidekiq.client_middleware do |chain|
|
767
|
+
chain.add Sidekiq::Status::ClientMiddleware, expiration: 10.minutes
|
768
|
+
chain.add Sidekiq::Status::ServerMiddleware
|
769
|
+
end
|
770
|
+
|
771
|
+
Sidekiq.server_middleware do |chain|
|
772
|
+
chain.add Sidekiq::Status::ServerMiddleware, expiration: 10.minutes
|
773
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
774
|
+
end
|
775
|
+
```
|
776
|
+
|
777
|
+
The reason for this is that if a job is duplicated it shouldn't end up with the status middleware at all. Status is just a monitor so to prevent clashes, leftovers and ensure cleanup. The status middleware should run after uniqueness on client and before on server. This will lead to less surprises.
|
661
778
|
|
662
779
|
## Debugging
|
663
780
|
|
data/lib/sidekiq-unique-jobs.rb
CHANGED
@@ -78,7 +78,7 @@ module SidekiqUniqueJobs
|
|
78
78
|
def entries(pattern: SCAN_PATTERN, count: DEFAULT_COUNT)
|
79
79
|
options = {}
|
80
80
|
options[:match] = pattern
|
81
|
-
options[:count] = count
|
81
|
+
options[:count] = count
|
82
82
|
|
83
83
|
result = redis { |conn| conn.zscan_each(key, **options).to_a }
|
84
84
|
|
@@ -69,8 +69,7 @@ redis.call("LREM", queued, -1, job_id)
|
|
69
69
|
log_debug("LREM", primed, 1, job_id)
|
70
70
|
redis.call("LREM", primed, 1, job_id)
|
71
71
|
|
72
|
-
-- The Sidekiq client
|
73
|
-
-- The Sidekiq server should set pttl for all other jobs
|
72
|
+
-- The Sidekiq client sets pttl
|
74
73
|
if pttl and pttl > 0 then
|
75
74
|
log_debug("PEXPIRE", digest, pttl)
|
76
75
|
redis.call("PEXPIRE", digest, pttl)
|
@@ -7,15 +7,16 @@ local retry_set = KEYS[3]
|
|
7
7
|
-------- END keys ---------
|
8
8
|
|
9
9
|
-------- BEGIN argv ---------
|
10
|
-
local reaper_count
|
10
|
+
local reaper_count = tonumber(ARGV[1])
|
11
|
+
local threshold = tonumber(ARGV[2])
|
11
12
|
-------- END argv ---------
|
12
13
|
|
13
14
|
-------- BEGIN injected arguments --------
|
14
|
-
local current_time = tonumber(ARGV[
|
15
|
-
local debug_lua = ARGV[
|
16
|
-
local max_history = tonumber(ARGV[
|
17
|
-
local script_name = ARGV[
|
18
|
-
local redisversion = ARGV[
|
15
|
+
local current_time = tonumber(ARGV[3])
|
16
|
+
local debug_lua = ARGV[4] == "true"
|
17
|
+
local max_history = tonumber(ARGV[5])
|
18
|
+
local script_name = ARGV[6] .. ".lua"
|
19
|
+
local redisversion = ARGV[7]
|
19
20
|
--------- END injected arguments ---------
|
20
21
|
|
21
22
|
|
@@ -65,7 +66,7 @@ repeat
|
|
65
66
|
-- TODO: Add check for jobs checked out by process
|
66
67
|
if found ~= true then
|
67
68
|
log_debug("Searching for digest:", digest, "in process sets")
|
68
|
-
found = find_digest_in_process_set(digest)
|
69
|
+
found = find_digest_in_process_set(digest, threshold)
|
69
70
|
end
|
70
71
|
|
71
72
|
if found ~= true then
|
@@ -1,4 +1,4 @@
|
|
1
|
-
local function find_digest_in_process_set(digest)
|
1
|
+
local function find_digest_in_process_set(digest, threshold)
|
2
2
|
local process_cursor = 0
|
3
3
|
local job_cursor = 0
|
4
4
|
local pattern = "*" .. digest .. "*"
|
@@ -26,11 +26,18 @@ local function find_digest_in_process_set(digest)
|
|
26
26
|
log_debug("No entries in:", workers_key)
|
27
27
|
else
|
28
28
|
for i = 1, #jobs, 2 do
|
29
|
-
|
29
|
+
local jobstr = jobs[i +1]
|
30
|
+
if string.find(jobstr, digest) then
|
30
31
|
log_debug("Found digest", digest, "in:", workers_key)
|
31
32
|
found = true
|
32
33
|
break
|
33
34
|
end
|
35
|
+
|
36
|
+
local job = cjson.decode(jobstr)
|
37
|
+
if job.created_at > threshold then
|
38
|
+
found = true
|
39
|
+
break
|
40
|
+
end
|
34
41
|
end
|
35
42
|
end
|
36
43
|
|
@@ -11,63 +11,6 @@ module SidekiqUniqueJobs
|
|
11
11
|
include SidekiqUniqueJobs::OptionsWithFallback
|
12
12
|
include SidekiqUniqueJobs::JSON
|
13
13
|
|
14
|
-
#
|
15
|
-
# Configure both server and client
|
16
|
-
#
|
17
|
-
def self.configure
|
18
|
-
configure_server
|
19
|
-
configure_client
|
20
|
-
end
|
21
|
-
|
22
|
-
#
|
23
|
-
# Configures the Sidekiq server
|
24
|
-
#
|
25
|
-
def self.configure_server # rubocop:disable Metrics/MethodLength
|
26
|
-
Sidekiq.configure_server do |config|
|
27
|
-
config.client_middleware do |chain|
|
28
|
-
if defined?(Apartment::Sidekiq::Middleware::Client)
|
29
|
-
chain.insert_after Apartment::Sidekiq::Middleware::Client, SidekiqUniqueJobs::Middleware::Client
|
30
|
-
else
|
31
|
-
chain.add SidekiqUniqueJobs::Middleware::Client
|
32
|
-
end
|
33
|
-
end
|
34
|
-
|
35
|
-
config.server_middleware do |chain|
|
36
|
-
if defined?(Apartment::Sidekiq::Middleware::Server)
|
37
|
-
chain.insert_after Apartment::Sidekiq::Middleware::Server, SidekiqUniqueJobs::Middleware::Server
|
38
|
-
else
|
39
|
-
chain.add SidekiqUniqueJobs::Middleware::Server
|
40
|
-
end
|
41
|
-
end
|
42
|
-
|
43
|
-
config.on(:startup) do
|
44
|
-
SidekiqUniqueJobs::UpdateVersion.call
|
45
|
-
SidekiqUniqueJobs::UpgradeLocks.call
|
46
|
-
|
47
|
-
SidekiqUniqueJobs::Orphans::Manager.start
|
48
|
-
end
|
49
|
-
|
50
|
-
config.on(:shutdown) do
|
51
|
-
SidekiqUniqueJobs::Orphans::Manager.stop
|
52
|
-
end
|
53
|
-
end
|
54
|
-
end
|
55
|
-
|
56
|
-
#
|
57
|
-
# Configures the Sidekiq client
|
58
|
-
#
|
59
|
-
def self.configure_client
|
60
|
-
Sidekiq.configure_client do |config|
|
61
|
-
config.client_middleware do |chain|
|
62
|
-
if defined?(Apartment::Sidekiq::Middleware::Client)
|
63
|
-
chain.insert_after Apartment::Sidekiq::Middleware::Client, SidekiqUniqueJobs::Middleware::Client
|
64
|
-
else
|
65
|
-
chain.add SidekiqUniqueJobs::Middleware::Client
|
66
|
-
end
|
67
|
-
end
|
68
|
-
end
|
69
|
-
end
|
70
|
-
|
71
14
|
# The sidekiq job hash
|
72
15
|
# @return [Hash] the Sidekiq job hash
|
73
16
|
attr_reader :item
|
@@ -11,9 +11,9 @@ module SidekiqUniqueJobs
|
|
11
11
|
# @return [String] rthe sidekiq queue this job belongs to
|
12
12
|
attr_reader :queue
|
13
13
|
#
|
14
|
-
# @!attribute [r]
|
14
|
+
# @!attribute [r] lock_digest
|
15
15
|
# @return [String] the unique digest to use for locking
|
16
|
-
attr_reader :
|
16
|
+
attr_reader :lock_digest
|
17
17
|
|
18
18
|
#
|
19
19
|
# Initialize a new Replace strategy
|
@@ -22,8 +22,8 @@ module SidekiqUniqueJobs
|
|
22
22
|
#
|
23
23
|
def initialize(item, redis_pool = nil)
|
24
24
|
super(item, redis_pool)
|
25
|
-
@queue
|
26
|
-
@
|
25
|
+
@queue = item[QUEUE]
|
26
|
+
@lock_digest = item[LOCK_DIGEST]
|
27
27
|
end
|
28
28
|
|
29
29
|
#
|
@@ -37,10 +37,11 @@ module SidekiqUniqueJobs
|
|
37
37
|
def call(&block)
|
38
38
|
return unless (deleted_job = delete_job_by_digest)
|
39
39
|
|
40
|
-
log_info("
|
40
|
+
log_info("Deleted job: #{deleted_job}")
|
41
41
|
if (del_count = delete_lock)
|
42
|
-
log_info("Deleted `#{del_count}` keys for #{
|
42
|
+
log_info("Deleted `#{del_count}` keys for #{lock_digest}")
|
43
43
|
end
|
44
|
+
|
44
45
|
block&.call
|
45
46
|
end
|
46
47
|
|
@@ -54,7 +55,7 @@ module SidekiqUniqueJobs
|
|
54
55
|
def delete_job_by_digest
|
55
56
|
call_script(:delete_job_by_digest,
|
56
57
|
keys: ["#{QUEUE}:#{queue}", SCHEDULE, RETRY],
|
57
|
-
argv: [
|
58
|
+
argv: [lock_digest])
|
58
59
|
end
|
59
60
|
|
60
61
|
#
|
@@ -64,7 +65,7 @@ module SidekiqUniqueJobs
|
|
64
65
|
# @return [Integer] the number of keys deleted
|
65
66
|
#
|
66
67
|
def delete_lock
|
67
|
-
digests.delete_by_digest(
|
68
|
+
digests.delete_by_digest(lock_digest)
|
68
69
|
end
|
69
70
|
|
70
71
|
#
|
@@ -21,7 +21,7 @@ module SidekiqUniqueJobs
|
|
21
21
|
def call
|
22
22
|
if sidekiq_worker_class?
|
23
23
|
log_info("Rescheduling #{item[LOCK_DIGEST]}")
|
24
|
-
worker_class
|
24
|
+
worker_class.perform_in(5, *item[ARGS])
|
25
25
|
else
|
26
26
|
log_warn("Skip rescheduling of #{item[LOCK_DIGEST]} because #{worker_class} is not a Sidekiq::Worker")
|
27
27
|
end
|
@@ -59,7 +59,12 @@ module SidekiqUniqueJobs
|
|
59
59
|
# @return [<type>] <description>
|
60
60
|
#
|
61
61
|
def task
|
62
|
-
@task ||= Concurrent::TimerTask.new(timer_task_options)
|
62
|
+
@task ||= Concurrent::TimerTask.new(timer_task_options, &task_body)
|
63
|
+
end
|
64
|
+
|
65
|
+
# @private
|
66
|
+
def task_body
|
67
|
+
@task_body ||= lambda do
|
63
68
|
with_logging_context do
|
64
69
|
redis do |conn|
|
65
70
|
refresh_reaper_mutex
|
@@ -76,6 +76,16 @@ module SidekiqUniqueJobs
|
|
76
76
|
config.reaper
|
77
77
|
end
|
78
78
|
|
79
|
+
#
|
80
|
+
# The configured timeout for the reaper
|
81
|
+
#
|
82
|
+
#
|
83
|
+
# @return [Integer] timeout in seconds
|
84
|
+
#
|
85
|
+
def reaper_timeout
|
86
|
+
config.reaper_timeout
|
87
|
+
end
|
88
|
+
|
79
89
|
#
|
80
90
|
# The number of locks to reap at a time
|
81
91
|
#
|
@@ -117,7 +117,7 @@ module SidekiqUniqueJobs
|
|
117
117
|
end
|
118
118
|
end
|
119
119
|
|
120
|
-
def active?(digest) # rubocop:disable Metrics/MethodLength
|
120
|
+
def active?(digest) # rubocop:disable Metrics/MethodLength, Metrics/CyclomaticComplexity
|
121
121
|
Sidekiq.redis do |conn|
|
122
122
|
procs = conn.sscan_each("processes").to_a
|
123
123
|
return false if procs.empty?
|
@@ -132,7 +132,10 @@ module SidekiqUniqueJobs
|
|
132
132
|
next unless workers.any?
|
133
133
|
|
134
134
|
workers.each_pair do |_tid, job|
|
135
|
-
|
135
|
+
item = load_json(job)
|
136
|
+
|
137
|
+
return true if item.dig(PAYLOAD, LOCK_DIGEST) == digest
|
138
|
+
return true if considered_active?(item[CREATED_AT])
|
136
139
|
end
|
137
140
|
end
|
138
141
|
|
@@ -140,6 +143,10 @@ module SidekiqUniqueJobs
|
|
140
143
|
end
|
141
144
|
end
|
142
145
|
|
146
|
+
def considered_active?(time_f)
|
147
|
+
(Time.now - reaper_timeout).to_f < time_f
|
148
|
+
end
|
149
|
+
|
143
150
|
#
|
144
151
|
# Loops through all the redis queues and yields them one by one
|
145
152
|
#
|
@@ -48,10 +48,10 @@ module SidekiqUniqueJobs
|
|
48
48
|
def exist?
|
49
49
|
redis do |conn|
|
50
50
|
value = conn.exists(key)
|
51
|
-
return true if value.is_a?(TrueClass)
|
52
|
-
return false if value.is_a?(FalseClass)
|
53
51
|
|
54
|
-
value
|
52
|
+
return value if boolean?(value)
|
53
|
+
|
54
|
+
value.to_i.positive?
|
55
55
|
end
|
56
56
|
end
|
57
57
|
|
@@ -95,6 +95,12 @@ module SidekiqUniqueJobs
|
|
95
95
|
def count
|
96
96
|
0
|
97
97
|
end
|
98
|
+
|
99
|
+
private
|
100
|
+
|
101
|
+
def boolean?(value)
|
102
|
+
[TrueClass, FalseClass].any? { |klazz| value.is_a?(klazz) }
|
103
|
+
end
|
98
104
|
end
|
99
105
|
end
|
100
106
|
end
|
@@ -53,7 +53,13 @@ module SidekiqUniqueJobs
|
|
53
53
|
end
|
54
54
|
end
|
55
55
|
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
56
|
+
begin
|
57
|
+
require "delegate" unless defined?(DelegateClass)
|
58
|
+
require "sidekiq/web" unless defined?(Sidekiq::Web)
|
59
|
+
|
60
|
+
Sidekiq::Web.register(SidekiqUniqueJobs::Web)
|
61
|
+
Sidekiq::Web.tabs["Locks"] = "locks"
|
62
|
+
Sidekiq::Web.settings.locales << File.join(File.dirname(__FILE__), "locales")
|
63
|
+
rescue NameError, LoadError => ex
|
64
|
+
SidekiqUniqueJobs.logger.error(ex)
|
65
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: sidekiq-unique-jobs
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 7.0.0
|
4
|
+
version: 7.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mikael Henriksson
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-01-
|
11
|
+
date: 2021-01-20 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: brpoplpush-redis_script
|
@@ -56,7 +56,7 @@ dependencies:
|
|
56
56
|
requirements:
|
57
57
|
- - ">="
|
58
58
|
- !ruby/object:Gem::Version
|
59
|
-
version: '
|
59
|
+
version: '5.0'
|
60
60
|
- - "<"
|
61
61
|
- !ruby/object:Gem::Version
|
62
62
|
version: '7.0'
|
@@ -66,7 +66,7 @@ dependencies:
|
|
66
66
|
requirements:
|
67
67
|
- - ">="
|
68
68
|
- !ruby/object:Gem::Version
|
69
|
-
version: '
|
69
|
+
version: '5.0'
|
70
70
|
- - "<"
|
71
71
|
- !ruby/object:Gem::Version
|
72
72
|
version: '7.0'
|
@@ -177,7 +177,6 @@ files:
|
|
177
177
|
- lib/sidekiq_unique_jobs/orphans/observer.rb
|
178
178
|
- lib/sidekiq_unique_jobs/orphans/reaper.rb
|
179
179
|
- lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb
|
180
|
-
- lib/sidekiq_unique_jobs/profiler.rb
|
181
180
|
- lib/sidekiq_unique_jobs/redis.rb
|
182
181
|
- lib/sidekiq_unique_jobs/redis/entity.rb
|
183
182
|
- lib/sidekiq_unique_jobs/redis/hash.rb
|
@@ -217,6 +216,9 @@ metadata:
|
|
217
216
|
post_install_message: |
|
218
217
|
IMPORTANT!
|
219
218
|
|
219
|
+
Automatic configuration of the sidekiq middelware is no longer done.
|
220
|
+
Please see: https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/README.md#add-the-middleware
|
221
|
+
|
220
222
|
This version deprecated the following sidekiq_options
|
221
223
|
|
222
224
|
- sidekiq_options lock_args: :method_name
|
@@ -248,9 +250,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
248
250
|
version: 2.5.0
|
249
251
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
250
252
|
requirements:
|
251
|
-
- - "
|
253
|
+
- - ">="
|
252
254
|
- !ruby/object:Gem::Version
|
253
|
-
version:
|
255
|
+
version: '0'
|
254
256
|
requirements: []
|
255
257
|
rubygems_version: 3.2.4
|
256
258
|
signing_key:
|
@@ -1,55 +0,0 @@
|
|
1
|
-
# frozen_string_literal: true
|
2
|
-
|
3
|
-
module SidekiqUniqueJobs
|
4
|
-
#
|
5
|
-
# Class MethodProfiler provides method level profiling
|
6
|
-
#
|
7
|
-
# @author Mikael Henriksson <mikael@mhenrixon.com>
|
8
|
-
#
|
9
|
-
class Profiler
|
10
|
-
def self.patch(klass, methods, name) # rubocop:disable Metrics/MethodLength
|
11
|
-
patches = methods.map do |method_name|
|
12
|
-
<<~RUBY
|
13
|
-
unless defined?(#{method_name}__mp_unpatched)
|
14
|
-
alias_method :#{method_name}__mp_unpatched, :#{method_name}
|
15
|
-
def #{method_name}(*args, &blk)
|
16
|
-
unless prof = Thread.current[:_method_profiler]
|
17
|
-
return #{method_name}__mp_unpatched(*args, &blk)
|
18
|
-
end
|
19
|
-
begin
|
20
|
-
start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
21
|
-
#{method_name}__mp_unpatched(*args, &blk)
|
22
|
-
ensure
|
23
|
-
data = (prof[:#{name}] ||= {duration: 0.0, calls: 0})
|
24
|
-
data[:duration] += Process.clock_gettime(Process::CLOCK_MONOTONIC) - start
|
25
|
-
data[:calls] += 1
|
26
|
-
end
|
27
|
-
end
|
28
|
-
end
|
29
|
-
RUBY
|
30
|
-
end.join("\n")
|
31
|
-
|
32
|
-
klass.class_eval patches
|
33
|
-
end
|
34
|
-
|
35
|
-
def self.start
|
36
|
-
Thread.current[:_method_profiler] = {
|
37
|
-
__start: current_timestamp,
|
38
|
-
}
|
39
|
-
end
|
40
|
-
|
41
|
-
def self.stop
|
42
|
-
finish = current_timestamp
|
43
|
-
return unless (data = Thread.current[:_method_profiler])
|
44
|
-
|
45
|
-
Thread.current[:_method_profiler] = nil
|
46
|
-
start = data.delete(:__start)
|
47
|
-
data[:total_duration] = finish - start
|
48
|
-
data
|
49
|
-
end
|
50
|
-
|
51
|
-
def self.current_timestamp
|
52
|
-
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
53
|
-
end
|
54
|
-
end
|
55
|
-
end
|