sidekiq-unique-jobs 7.0.0.beta26 → 7.0.1
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of sidekiq-unique-jobs might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +89 -11
- data/README.md +154 -36
- data/lib/sidekiq-unique-jobs.rb +0 -2
- data/lib/sidekiq_unique_jobs.rb +1 -0
- data/lib/sidekiq_unique_jobs/changelog.rb +11 -4
- data/lib/sidekiq_unique_jobs/config.rb +2 -2
- data/lib/sidekiq_unique_jobs/constants.rb +4 -0
- data/lib/sidekiq_unique_jobs/digests.rb +1 -1
- data/lib/sidekiq_unique_jobs/job.rb +1 -1
- data/lib/sidekiq_unique_jobs/lock/validator.rb +2 -1
- data/lib/sidekiq_unique_jobs/lock/while_executing.rb +1 -1
- data/lib/sidekiq_unique_jobs/lock_args.rb +4 -4
- data/lib/sidekiq_unique_jobs/lock_config.rb +2 -0
- data/lib/sidekiq_unique_jobs/locksmith.rb +1 -1
- data/lib/sidekiq_unique_jobs/lua/delete_by_digest.lua +1 -1
- data/lib/sidekiq_unique_jobs/lua/lock.lua +1 -2
- data/lib/sidekiq_unique_jobs/lua/reap_orphans.lua +8 -7
- data/lib/sidekiq_unique_jobs/lua/shared/_find_digest_in_process_set.lua +9 -2
- data/lib/sidekiq_unique_jobs/middleware.rb +0 -57
- data/lib/sidekiq_unique_jobs/on_conflict/replace.rb +9 -8
- data/lib/sidekiq_unique_jobs/on_conflict/reschedule.rb +1 -1
- data/lib/sidekiq_unique_jobs/orphans/lua_reaper.rb +1 -1
- data/lib/sidekiq_unique_jobs/orphans/reaper.rb +10 -0
- data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +9 -2
- data/lib/sidekiq_unique_jobs/redis/entity.rb +9 -3
- data/lib/sidekiq_unique_jobs/redis/sorted_set.rb +27 -0
- data/lib/sidekiq_unique_jobs/server.rb +48 -0
- data/lib/sidekiq_unique_jobs/sidekiq_unique_jobs.rb +1 -1
- data/lib/sidekiq_unique_jobs/sidekiq_worker_methods.rb +1 -1
- data/lib/sidekiq_unique_jobs/testing.rb +2 -1
- data/lib/sidekiq_unique_jobs/version.rb +1 -1
- data/lib/sidekiq_unique_jobs/web.rb +26 -9
- data/lib/sidekiq_unique_jobs/web/helpers.rb +24 -3
- data/lib/sidekiq_unique_jobs/web/views/changelogs.erb +54 -0
- data/lib/sidekiq_unique_jobs/web/views/locks.erb +1 -1
- metadata +25 -9
- data/lib/sidekiq_unique_jobs/profiler.rb +0 -55
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 791ef31e7978f706fa00198280417ebb4f230d8bf5225f5e914cc8092b347b1e
|
4
|
+
data.tar.gz: b085a1ac1974d74a73248cd931eff8a387f5017e06e66a863844616e6ac29184
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 71cb338220b1d3bb77b9834f5c4fe42c48532b95bf794bbad286382f1a12297b8411276b1e802c921203a345fd25151feb22b63dd284d111eea2018f29ec4ea1
|
7
|
+
data.tar.gz: 7cca4aa29025359b33cd914b402c446a4e9b03cb98f3b0839732d2bae03b2503f5d3425d57cd2efbd4705a438c334b36f8dd2b60a279cf8e505b188c4a3003bc
|
data/CHANGELOG.md
CHANGED
@@ -2,7 +2,84 @@
|
|
2
2
|
|
3
3
|
## [Unreleased](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/HEAD)
|
4
4
|
|
5
|
-
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/
|
5
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0...HEAD)
|
6
|
+
|
7
|
+
|
8
|
+
|
9
|
+
## [v7.0.0](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0) (2021-01-20)
|
10
|
+
|
11
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.25...v7.0.0)
|
12
|
+
|
13
|
+
**Implemented enhancements:**
|
14
|
+
|
15
|
+
- Give user full control over adding middleware [\#566](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/566) ([mhenrixon](https://github.com/mhenrixon))
|
16
|
+
|
17
|
+
**Fixed bugs:**
|
18
|
+
|
19
|
+
- Race condition in ruby reaper [\#559](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/559)
|
20
|
+
- Fix until and while executed and improve documentation [\#397](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/397)
|
21
|
+
- Fix 532 [\#549](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/549) ([mhenrixon](https://github.com/mhenrixon))
|
22
|
+
|
23
|
+
**Closed issues:**
|
24
|
+
|
25
|
+
- Rejected jobs are still displayed as 'Queued' with gem 'sidekiq-status' on /sidekiq/statuses [\#564](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/564)
|
26
|
+
- Is it possible to have a :until\_executed lock with an expiration time? [\#524](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/524)
|
27
|
+
|
28
|
+
## [v7.0.0.beta29](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta29) (2021-01-16)
|
29
|
+
|
30
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta28...v7.0.0.beta29)
|
31
|
+
|
32
|
+
**Implemented enhancements:**
|
33
|
+
|
34
|
+
- Fix coverage reporting and add coverage [\#565](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/565) ([mhenrixon](https://github.com/mhenrixon))
|
35
|
+
|
36
|
+
**Fixed bugs:**
|
37
|
+
|
38
|
+
- Ruby Reaper active check incorrect [\#557](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/557)
|
39
|
+
- Routes with authentication should work with web [\#562](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/562) ([mhenrixon](https://github.com/mhenrixon))
|
40
|
+
|
41
|
+
**Closed issues:**
|
42
|
+
|
43
|
+
- Can't add the lock tab o nthe website when there is authentication through devise [\#561](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/561)
|
44
|
+
|
45
|
+
## [v7.0.0.beta28](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta28) (2021-01-07)
|
46
|
+
|
47
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta27...v7.0.0.beta28)
|
48
|
+
|
49
|
+
**Fixed bugs:**
|
50
|
+
|
51
|
+
- lock\_args does not work when you define the lock\_args argument and default lock\_args function at the same time. [\#548](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/548)
|
52
|
+
- incorrect `:while\_executing` behavior [\#432](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/432)
|
53
|
+
- Fix race condition to avoid reaping active jobs [\#563](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/563) ([mhenrixon](https://github.com/mhenrixon))
|
54
|
+
- Fix active check for the worker hash [\#558](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/558) ([mhenrixon](https://github.com/mhenrixon))
|
55
|
+
- Prefer lock\_prefix not unique\_prefix [\#554](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/554) ([mhenrixon](https://github.com/mhenrixon))
|
56
|
+
- Fix issue 432 [\#552](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/552) ([mhenrixon](https://github.com/mhenrixon))
|
57
|
+
|
58
|
+
## [v7.0.0.beta27](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta27) (2020-11-03)
|
59
|
+
|
60
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta26...v7.0.0.beta27)
|
61
|
+
|
62
|
+
**Implemented enhancements:**
|
63
|
+
|
64
|
+
- Adds coverage for regression purposes [\#550](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/550) ([mhenrixon](https://github.com/mhenrixon))
|
65
|
+
|
66
|
+
**Fixed bugs:**
|
67
|
+
|
68
|
+
- Rename lock\_args to lock\_args\_method [\#551](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/551) ([mhenrixon](https://github.com/mhenrixon))
|
69
|
+
|
70
|
+
**Closed issues:**
|
71
|
+
|
72
|
+
- Documentation incorrect for `delete\_by\_digest` [\#547](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/547)
|
73
|
+
- Locked jobs after kill -9 with while\_executing lock [\#546](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/546)
|
74
|
+
|
75
|
+
## [v7.0.0.beta26](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta26) (2020-10-28)
|
76
|
+
|
77
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.25...v7.0.0.beta26)
|
78
|
+
|
79
|
+
**Implemented enhancements:**
|
80
|
+
|
81
|
+
- How to disable Reaper [\#543](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/543)
|
82
|
+
- Allow disabling of reaper [\#544](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/544) ([mhenrixon](https://github.com/mhenrixon))
|
6
83
|
|
7
84
|
**Merged pull requests:**
|
8
85
|
|
@@ -88,6 +165,7 @@
|
|
88
165
|
**Fixed bugs:**
|
89
166
|
|
90
167
|
- Prevent indefinitely looping entries [\#516](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/516) ([mhenrixon](https://github.com/mhenrixon))
|
168
|
+
- Don't bypass lock\_args if arguments are empty [\#511](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/511) ([nhippenmeyer](https://github.com/nhippenmeyer))
|
91
169
|
|
92
170
|
**Closed issues:**
|
93
171
|
|
@@ -169,7 +247,6 @@
|
|
169
247
|
- Brpoplpush::RedisScript::LuaError: WRONGTYPE Operation against a key holding the wrong kind of value [\#491](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/491)
|
170
248
|
- Lua script bug [\#489](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/489)
|
171
249
|
- Reaper will delete locks for running jobs [\#488](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/488)
|
172
|
-
- Fix access to hash members [\#496](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/496) ([mhenrixon](https://github.com/mhenrixon))
|
173
250
|
- Fix cursor assignment [\#494](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/494) ([mhenrixon](https://github.com/mhenrixon))
|
174
251
|
- Prevent reaping of active jobs [\#493](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/493) ([mhenrixon](https://github.com/mhenrixon))
|
175
252
|
|
@@ -187,6 +264,7 @@
|
|
187
264
|
|
188
265
|
**Fixed bugs:**
|
189
266
|
|
267
|
+
- Fix access to hash members [\#496](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/496) ([mhenrixon](https://github.com/mhenrixon))
|
190
268
|
- Use thread-safe digest creation mechanism [\#484](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/484) ([mhenrixon](https://github.com/mhenrixon))
|
191
269
|
|
192
270
|
## [v7.0.0.beta13](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta13) (2020-03-26)
|
@@ -422,6 +500,7 @@
|
|
422
500
|
|
423
501
|
**Fixed bugs:**
|
424
502
|
|
503
|
+
- Include redis directory [\#430](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/430) ([mhenrixon](https://github.com/mhenrixon))
|
425
504
|
- Allow Sidekiq::Context to be used for logging [\#429](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/429) ([mhenrixon](https://github.com/mhenrixon))
|
426
505
|
- Fix sidekiq develop [\#426](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/426) ([mhenrixon](https://github.com/mhenrixon))
|
427
506
|
- Reap as many orphans as advertised [\#403](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/403) ([mhenrixon](https://github.com/mhenrixon))
|
@@ -448,7 +527,6 @@
|
|
448
527
|
|
449
528
|
- Duplicate job was pushed \( v6.0.13 \) [\#414](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/414)
|
450
529
|
- Constant SidekiqUniqueJobs::Web::Digests not found [\#396](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/396)
|
451
|
-
- Include redis directory [\#430](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/430) ([mhenrixon](https://github.com/mhenrixon))
|
452
530
|
- Fix v6 [\#428](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/428) ([mhenrixon](https://github.com/mhenrixon))
|
453
531
|
|
454
532
|
**Closed issues:**
|
@@ -569,7 +647,7 @@
|
|
569
647
|
|
570
648
|
- Version 5: Job ID Hash Entries Not Removed if Unique Key Expires [\#346](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/346)
|
571
649
|
- Move the lpush last [\#354](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/354) ([mhenrixon](https://github.com/mhenrixon))
|
572
|
-
-
|
650
|
+
- Fix the broken expiration [\#316](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/316) ([mhenrixon](https://github.com/mhenrixon))
|
573
651
|
|
574
652
|
**Closed issues:**
|
575
653
|
|
@@ -591,6 +669,7 @@
|
|
591
669
|
**Fixed bugs:**
|
592
670
|
|
593
671
|
- Always Remove Job ID from uniquejobs Hash [\#347](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/347) ([chadrschroeder](https://github.com/chadrschroeder))
|
672
|
+
- Convert expiration time to integer [\#330](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/330) ([dareddov](https://github.com/dareddov))
|
594
673
|
|
595
674
|
**Closed issues:**
|
596
675
|
|
@@ -631,10 +710,6 @@
|
|
631
710
|
|
632
711
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.3...v6.0.4)
|
633
712
|
|
634
|
-
**Fixed bugs:**
|
635
|
-
|
636
|
-
- Fix the broken expiration [\#316](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/316) ([mhenrixon](https://github.com/mhenrixon))
|
637
|
-
|
638
713
|
**Closed issues:**
|
639
714
|
|
640
715
|
- Question about until\_timeout with 6.0.0 [\#303](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/303)
|
@@ -682,6 +757,7 @@
|
|
682
757
|
|
683
758
|
- Dead jobs [\#308](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/308) ([mhenrixon](https://github.com/mhenrixon))
|
684
759
|
- Fix require path for sidekiq\_unique\_jobs/web [\#305](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/305) ([soundasleep](https://github.com/soundasleep))
|
760
|
+
- Update README.md [\#300](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/300) ([pirj](https://github.com/pirj))
|
685
761
|
|
686
762
|
## [v6.0.0](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0) (2018-07-27)
|
687
763
|
|
@@ -700,7 +776,6 @@
|
|
700
776
|
**Merged pull requests:**
|
701
777
|
|
702
778
|
- Add more details about testing uniqueness [\#301](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/301) ([mhenrixon](https://github.com/mhenrixon))
|
703
|
-
- Update README.md [\#300](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/300) ([pirj](https://github.com/pirj))
|
704
779
|
|
705
780
|
## [v6.0.0.rc7](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0.rc7) (2018-07-23)
|
706
781
|
|
@@ -745,7 +820,6 @@
|
|
745
820
|
**Fixed bugs:**
|
746
821
|
|
747
822
|
- bundle exec jobs console does not work [\#253](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/253)
|
748
|
-
- Rename command line binary [\#287](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/287) ([mhenrixon](https://github.com/mhenrixon))
|
749
823
|
|
750
824
|
## [v6.0.0.rc4](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0.rc4) (2018-06-30)
|
751
825
|
|
@@ -756,6 +830,10 @@
|
|
756
830
|
- Prepare for v6 [\#286](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/286) ([mhenrixon](https://github.com/mhenrixon))
|
757
831
|
- Only unlock not delete [\#285](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/285) ([mhenrixon](https://github.com/mhenrixon))
|
758
832
|
|
833
|
+
**Fixed bugs:**
|
834
|
+
|
835
|
+
- Rename command line binary [\#287](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/287) ([mhenrixon](https://github.com/mhenrixon))
|
836
|
+
|
759
837
|
## [v6.0.0.rc3](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0.rc3) (2018-06-29)
|
760
838
|
|
761
839
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.0.rc2...v6.0.0.rc3)
|
@@ -819,7 +897,6 @@
|
|
819
897
|
**Implemented enhancements:**
|
820
898
|
|
821
899
|
- Code smells [\#275](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/275) ([mhenrixon](https://github.com/mhenrixon))
|
822
|
-
- Reject while scheduling [\#273](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/273) ([mhenrixon](https://github.com/mhenrixon))
|
823
900
|
- Improve testing [\#272](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/272) ([mhenrixon](https://github.com/mhenrixon))
|
824
901
|
|
825
902
|
## [v6.0.0.beta](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0.beta) (2018-06-17)
|
@@ -828,6 +905,7 @@
|
|
828
905
|
|
829
906
|
**Implemented enhancements:**
|
830
907
|
|
908
|
+
- Reject while scheduling [\#273](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/273) ([mhenrixon](https://github.com/mhenrixon))
|
831
909
|
- Until and while executing [\#271](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/271) ([mhenrixon](https://github.com/mhenrixon))
|
832
910
|
- Solidify master [\#270](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/270) ([mhenrixon](https://github.com/mhenrixon))
|
833
911
|
- Minor adjustments [\#268](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/268) ([mhenrixon](https://github.com/mhenrixon))
|
data/README.md
CHANGED
@@ -3,9 +3,12 @@
|
|
3
3
|
<!-- MarkdownTOC -->
|
4
4
|
|
5
5
|
- [Introduction](#introduction)
|
6
|
-
- [
|
7
|
-
- [Installation](#installation)
|
6
|
+
- [Usage](#usage)
|
7
|
+
- [Installation](#installation)
|
8
|
+
- [Add the middleware](#add-the-middleware)
|
9
|
+
- [Your first worker](#your-first-worker)
|
8
10
|
- [Support Me](#support-me)
|
11
|
+
- [Requirements](#requirements)
|
9
12
|
- [General Information](#general-information)
|
10
13
|
- [Global Configuration](#global-configuration)
|
11
14
|
- [debug_lua](#debug_lua)
|
@@ -18,9 +21,11 @@
|
|
18
21
|
- [reaper_count](#reaper_count)
|
19
22
|
- [reaper_interval](#reaper_interval)
|
20
23
|
- [reaper_timeout](#reaper_timeout)
|
21
|
-
- [
|
24
|
+
- [lock_prefix](#lock_prefix)
|
22
25
|
- [lock_info](#lock_info)
|
23
26
|
- [Worker Configuration](#worker-configuration)
|
27
|
+
- [lock_info](#lock_info-1)
|
28
|
+
- [lock_prefix](#lock_prefix-1)
|
24
29
|
- [lock_ttl](#lock_ttl-1)
|
25
30
|
- [lock_timeout](#lock_timeout-1)
|
26
31
|
- [unique_across_queues](#unique_across_queues)
|
@@ -39,13 +44,15 @@
|
|
39
44
|
- [replace](#replace)
|
40
45
|
- [Reschedule](#reschedule)
|
41
46
|
- [Custom Strategies](#custom-strategies)
|
42
|
-
- [Usage](#usage)
|
47
|
+
- [Usage](#usage-1)
|
43
48
|
- [Finer Control over Uniqueness](#finer-control-over-uniqueness)
|
44
49
|
- [After Unlock Callback](#after-unlock-callback)
|
45
50
|
- [Logging](#logging)
|
46
51
|
- [Cleanup Dead Locks](#cleanup-dead-locks)
|
47
52
|
- [Other Sidekiq gems](#other-sidekiq-gems)
|
53
|
+
- [apartment-sidekiq](#apartment-sidekiq)
|
48
54
|
- [sidekiq-global_id](#sidekiq-global_id)
|
55
|
+
- [sidekiq-status](#sidekiq-status)
|
49
56
|
- [Debugging](#debugging)
|
50
57
|
- [Sidekiq Web](#sidekiq-web)
|
51
58
|
- [Show Locks](#show-locks)
|
@@ -61,30 +68,19 @@
|
|
61
68
|
|
62
69
|
## Introduction
|
63
70
|
|
64
|
-
The
|
71
|
+
This gem adds unique constraints to the sidekiq queues. The uniqueness is achieved by acquiring locks for a hash of a queue name, a worker class, and job's arguments. By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen `on_conflict`strategy.
|
65
72
|
|
66
73
|
This is the documentation for the master branch. You can find the documentation for each release by navigating to its tag.
|
67
74
|
|
68
75
|
Here are links to some of the old versions
|
69
76
|
|
70
|
-
- [v6.0.
|
77
|
+
- [v6.0.25](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.25)
|
71
78
|
- [v5.0.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10)
|
72
79
|
- [v4.0.18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18)
|
73
80
|
|
74
|
-
##
|
75
|
-
|
76
|
-
- Sidekiq `>= 4.0` (`>= 5.2` recommended)
|
77
|
-
- Ruby:
|
78
|
-
- MRI `>= 2.3` (`>= 2.5` recommended)
|
79
|
-
- JRuby `>= 9.0` (`>= 9.2` recommended)
|
80
|
-
- Truffleruby
|
81
|
-
- Redis Server `>= 3.0.2` (`>= 3.2` recommended)
|
82
|
-
- [ActiveJob officially not supported][48]
|
83
|
-
- [redis-namespace officially not supported][49]
|
84
|
-
|
85
|
-
See [Sidekiq requirements][24] for detailed requirements of Sidekiq itself (be sure to check the right sidekiq version).
|
81
|
+
## Usage
|
86
82
|
|
87
|
-
|
83
|
+
### Installation
|
88
84
|
|
89
85
|
Add this line to your application's Gemfile:
|
90
86
|
|
@@ -98,16 +94,83 @@ And then execute:
|
|
98
94
|
bundle
|
99
95
|
```
|
100
96
|
|
101
|
-
|
97
|
+
### Add the middleware
|
102
98
|
|
103
|
-
|
104
|
-
|
99
|
+
Before v7, the middleware was configured automatically. Since some people reported issues with other gems (see [Other Sidekiq Gems](#other-sidekiq-gems)) it was decided to give full control over to the user.
|
100
|
+
|
101
|
+
*NOTE* if you want to use the reaper you also need to configure the server middleware.
|
102
|
+
|
103
|
+
[A full and hopefully working example](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/sidekiq.rb#L12)
|
104
|
+
|
105
|
+
```ruby
|
106
|
+
Sidekiq.configure_server do |config|
|
107
|
+
config.redis = { url: ENV["REDIS_URL"], driver: :hiredis }
|
108
|
+
|
109
|
+
config.client_middleware do |chain|
|
110
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
111
|
+
end
|
112
|
+
|
113
|
+
config.server_middleware do |chain|
|
114
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
115
|
+
end
|
116
|
+
|
117
|
+
SidekiqUniqueJobs::Server.configure(config)
|
118
|
+
end
|
119
|
+
|
120
|
+
Sidekiq.configure_client do |config|
|
121
|
+
config.redis = { url: ENV["REDIS_URL"], driver: :hiredis }
|
122
|
+
|
123
|
+
config.client_middleware do |chain|
|
124
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
125
|
+
end
|
126
|
+
end
|
105
127
|
```
|
106
128
|
|
129
|
+
### Your first worker
|
130
|
+
|
131
|
+
```ruby
|
132
|
+
# frozen_string_literal: true
|
133
|
+
|
134
|
+
class UntilExecutedWorker
|
135
|
+
include Sidekiq::Worker
|
136
|
+
|
137
|
+
sidekiq_options queue: :special,
|
138
|
+
retry: false,
|
139
|
+
lock: :until_executed,
|
140
|
+
lock_info: true,
|
141
|
+
lock_timeout: 0,
|
142
|
+
lock_prefix: "special",
|
143
|
+
lock_ttl: 0,
|
144
|
+
lock_limit: 5
|
145
|
+
|
146
|
+
def perform
|
147
|
+
logger.info("cowboy")
|
148
|
+
sleep(1) # hardcore processing
|
149
|
+
logger.info("beebop")
|
150
|
+
end
|
151
|
+
end
|
152
|
+
|
153
|
+
```
|
154
|
+
|
155
|
+
You can read more about the worker configuration in [Worker Configuration](#worker-configuration) below.
|
156
|
+
|
107
157
|
## Support Me
|
108
158
|
|
109
159
|
Want to show me some ❤️ for the hard work I do on this gem? You can use the following PayPal link: [https://paypal.me/mhenrixon1](https://paypal.me/mhenrixon1). Any amount is welcome and let me tell you it feels good to be appreciated. Even a dollar makes me super excited about all of this.
|
110
160
|
|
161
|
+
## Requirements
|
162
|
+
|
163
|
+
- Sidekiq `>= 5.0` (`>= 5.2` recommended)
|
164
|
+
- Ruby:
|
165
|
+
- MRI `>= 2.5` (`>= 2.6` recommended)
|
166
|
+
- JRuby `>= 9.0` (`>= 9.2` recommended)
|
167
|
+
- Truffleruby
|
168
|
+
- Redis Server `>= 3.2` (`>= 5.0` recommended)
|
169
|
+
- [ActiveJob officially not supported][48]
|
170
|
+
- [redis-namespace officially not supported][49]
|
171
|
+
|
172
|
+
See [Sidekiq requirements][24] for detailed requirements of Sidekiq itself (be sure to check the right sidekiq version).
|
173
|
+
|
111
174
|
## General Information
|
112
175
|
|
113
176
|
See [Interaction w/ Sidekiq](https://github.com/mhenrixon/sidekiq-unique-jobs/wiki/How-this-gem-interacts-with-Sidekiq) on how the gem interacts with Sidekiq.
|
@@ -235,10 +298,10 @@ SidekiqUniqueJobs.config.reaper_timeout #=> 10
|
|
235
298
|
|
236
299
|
The number of seconds to wait for the reaper to finish before raising a TimeoutError. This is done to ensure that the next time we reap isn't getting stuck due to the previous process already running.
|
237
300
|
|
238
|
-
###
|
301
|
+
### lock_prefix
|
239
302
|
|
240
303
|
```ruby
|
241
|
-
SidekiqUniqueJobs.config.
|
304
|
+
SidekiqUniqueJobs.config.lock_prefix #=> "uniquejobs"
|
242
305
|
```
|
243
306
|
|
244
307
|
Use if you want a different key prefix for the keys in redis.
|
@@ -253,11 +316,27 @@ Using lock info will create an additional key for the lock with a json object co
|
|
253
316
|
|
254
317
|
## Worker Configuration
|
255
318
|
|
319
|
+
### lock_info
|
320
|
+
|
321
|
+
Lock info gathers information about a specific lock. It collects things like which `lock_args` where used to compute the `lock_digest` that is used for maintaining uniqueness.
|
322
|
+
|
323
|
+
```ruby
|
324
|
+
sidekiq_options lock_info: false # this is the default, set to true to turn on
|
325
|
+
```
|
326
|
+
|
327
|
+
### lock_prefix
|
328
|
+
|
329
|
+
Use if you want a different key prefix for the keys in redis.
|
330
|
+
|
331
|
+
```ruby
|
332
|
+
sidekiq_options lock_prefix: "uniquejobs" # this is the default value
|
333
|
+
```
|
334
|
+
|
256
335
|
### lock_ttl
|
257
336
|
|
258
337
|
Lock TTL decides how long to wait after the job has been successfully processed before making it possible to reuse that lock.
|
259
338
|
|
260
|
-
|
339
|
+
Starting from `v7` the expiration will take place when the job is pushed to the queue.
|
261
340
|
|
262
341
|
```ruby
|
263
342
|
sidekiq_options lock_ttl: nil # default - don't expire keys
|
@@ -278,6 +357,8 @@ sidekiq_options lock_timeout: nil # lock indefinitely, this process won't contin
|
|
278
357
|
|
279
358
|
This configuration option is slightly misleading. It doesn't disregard the queue on other jobs. Just on itself, this means that a worker that might schedule jobs into multiple queues will be able to have uniqueness enforced on all queues it is pushed to.
|
280
359
|
|
360
|
+
This is mainly intended for `Worker.set(queue: :another).perform_async`.
|
361
|
+
|
281
362
|
```ruby
|
282
363
|
class Worker
|
283
364
|
include Sidekiq::Worker
|
@@ -428,7 +509,7 @@ The last one is log which can be be used with the lock `UntilExecuted` and `Unti
|
|
428
509
|
It is possible for locks to have different conflict strategy for the client and server. This is useful for `:until_and_while_executing`.
|
429
510
|
|
430
511
|
```ruby
|
431
|
-
sidekiq_options lock: :until_and_while_executing,
|
512
|
+
sidekiq_options lock: :until_and_while_executing,
|
432
513
|
on_conflict: { client: :log, server: :reject }
|
433
514
|
```
|
434
515
|
|
@@ -536,7 +617,7 @@ The method or the proc can return a modified version of args without the transie
|
|
536
617
|
class UniqueJobWithFilterMethod
|
537
618
|
include Sidekiq::Worker
|
538
619
|
sidekiq_options lock: :until_and_while_executing,
|
539
|
-
|
620
|
+
lock_args_method: :lock_args # this is default and will be used if such a method is defined
|
540
621
|
|
541
622
|
def self.lock_args(args)
|
542
623
|
[ args[0], args[2][:type] ]
|
@@ -549,7 +630,7 @@ end
|
|
549
630
|
class UniqueJobWithFilterProc
|
550
631
|
include Sidekiq::Worker
|
551
632
|
sidekiq_options lock: :until_executed,
|
552
|
-
|
633
|
+
lock_args_method: ->(args) { [ args.first ] }
|
553
634
|
|
554
635
|
...
|
555
636
|
|
@@ -561,7 +642,7 @@ It is possible to ensure different types of unique args based on context. I can'
|
|
561
642
|
```ruby
|
562
643
|
class UniqueJobWithFilterMethod
|
563
644
|
include Sidekiq::Worker
|
564
|
-
sidekiq_options lock: :until_and_while_executing,
|
645
|
+
sidekiq_options lock: :until_and_while_executing, lock_args_method: :lock_args
|
565
646
|
|
566
647
|
def self.lock_args(args)
|
567
648
|
if Sidekiq::ProcessSet.new.size > 1
|
@@ -622,8 +703,8 @@ For sidekiq versions before 5.1 a `sidekiq_retries_exhausted` block is required
|
|
622
703
|
```ruby
|
623
704
|
class MyWorker
|
624
705
|
sidekiq_retries_exhausted do |msg, _ex|
|
625
|
-
digest = msg['
|
626
|
-
SidekiqUniqueJobs::Digests.delete_by_digest(digest) if digest
|
706
|
+
digest = msg['lock_digest']
|
707
|
+
SidekiqUniqueJobs::Digests.new.delete_by_digest(digest) if digest
|
627
708
|
end
|
628
709
|
end
|
629
710
|
```
|
@@ -633,18 +714,38 @@ Starting in v5.1, Sidekiq can also fire a global callback when a job dies:
|
|
633
714
|
```ruby
|
634
715
|
Sidekiq.configure_server do |config|
|
635
716
|
config.death_handlers << ->(job, _ex) do
|
636
|
-
digest = job['
|
637
|
-
SidekiqUniqueJobs::Digests.delete_by_digest(digest) if digest
|
717
|
+
digest = job['lock_digest']
|
718
|
+
SidekiqUniqueJobs::Digests.new.delete_by_digest(digest) if digest
|
638
719
|
end
|
639
720
|
end
|
640
721
|
```
|
641
722
|
|
642
723
|
### Other Sidekiq gems
|
643
724
|
|
725
|
+
#### apartment-sidekiq
|
726
|
+
|
727
|
+
It was reported in [#536](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/536) that the order of the Sidekiq middleware needs to be as follows.
|
728
|
+
|
729
|
+
```ruby
|
730
|
+
Sidekiq.client_middleware do |chain|
|
731
|
+
chain.add Apartment::Sidekiq::Middleware::Client
|
732
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
733
|
+
end
|
734
|
+
|
735
|
+
Sidekiq.server_middleware do |chain|
|
736
|
+
chain.add Apartment::Sidekiq::Middleware::Server
|
737
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
738
|
+
end
|
739
|
+
```
|
740
|
+
|
741
|
+
The reason being that this gem needs to be configured AFTER the apartment gem or the apartment will not be able to be considered for uniqueness
|
742
|
+
|
644
743
|
#### sidekiq-global_id
|
645
744
|
|
646
745
|
It was reported in [#235](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/235) that the order of the Sidekiq middleware needs to be as follows.
|
647
746
|
|
747
|
+
For a working setup check the following [file](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/sidekiq.rb#L12).
|
748
|
+
|
648
749
|
```ruby
|
649
750
|
Sidekiq.client_middleware do |chain|
|
650
751
|
chain.add Sidekiq::GlobalId::ClientMiddleware
|
@@ -652,12 +753,30 @@ Sidekiq.client_middleware do |chain|
|
|
652
753
|
end
|
653
754
|
|
654
755
|
Sidekiq.server_middleware do |chain|
|
655
|
-
chain.add SidekiqUniqueJobs::Middleware::Server
|
656
756
|
chain.add Sidekiq::GlobalId::ServerMiddleware
|
757
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
657
758
|
end
|
658
759
|
```
|
659
760
|
|
660
|
-
|
761
|
+
The reason for this is that the global id needs to be set before the unique jobs middleware runs. Otherwise that won't be available for uniqueness.
|
762
|
+
|
763
|
+
#### sidekiq-status
|
764
|
+
|
765
|
+
It was reported in [#564](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/564) that the order of the middleware needs to be as follows.
|
766
|
+
|
767
|
+
```ruby
|
768
|
+
Sidekiq.client_middleware do |chain|
|
769
|
+
chain.add Sidekiq::Status::ClientMiddleware, expiration: 10.minutes
|
770
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
771
|
+
end
|
772
|
+
|
773
|
+
Sidekiq.server_middleware do |chain|
|
774
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
775
|
+
chain.add Sidekiq::Status::ServerMiddleware, expiration: 10.minutes
|
776
|
+
end
|
777
|
+
```
|
778
|
+
|
779
|
+
The reason for this is that if a job is duplicated it shouldn't end up with the status middleware at all. Status is just a monitor so to prevent clashes, leftovers and ensure cleanup. The status middleware should run after uniqueness on client and before on server. This will lead to less surprises.
|
661
780
|
|
662
781
|
## Debugging
|
663
782
|
|
@@ -728,7 +847,6 @@ If you are not using RSpec (a lot of people prefer minitest or test unit) you ca
|
|
728
847
|
assert SidekiqUniqueJobs.validate_worker!(BadWorker.get_sidekiq_options)
|
729
848
|
```
|
730
849
|
|
731
|
-
|
732
850
|
### Uniqueness
|
733
851
|
|
734
852
|
This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I there for decided to do what Mike is doing for sidekiq enterprise. Read the section about unique jobs: [Enterprise unique jobs][]
|