sidekiq-unique-jobs 8.0.10 → 8.0.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +31 -3
- data/README.md +56 -43
- data/lib/sidekiq_unique_jobs/cli.rb +2 -2
- data/lib/sidekiq_unique_jobs/config.rb +51 -32
- data/lib/sidekiq_unique_jobs/digests.rb +1 -1
- data/lib/sidekiq_unique_jobs/exceptions.rb +2 -2
- data/lib/sidekiq_unique_jobs/lock/base_lock.rb +1 -1
- data/lib/sidekiq_unique_jobs/lock/while_executing.rb +1 -1
- data/lib/sidekiq_unique_jobs/lock.rb +1 -1
- data/lib/sidekiq_unique_jobs/lock_args.rb +3 -3
- data/lib/sidekiq_unique_jobs/lock_digest.rb +6 -1
- data/lib/sidekiq_unique_jobs/lock_ttl.rb +28 -6
- data/lib/sidekiq_unique_jobs/locksmith.rb +6 -6
- data/lib/sidekiq_unique_jobs/logging.rb +2 -2
- data/lib/sidekiq_unique_jobs/on_conflict/reject.rb +10 -1
- data/lib/sidekiq_unique_jobs/on_conflict/replace.rb +3 -3
- data/lib/sidekiq_unique_jobs/on_conflict/reschedule.rb +1 -1
- data/lib/sidekiq_unique_jobs/on_conflict.rb +2 -2
- data/lib/sidekiq_unique_jobs/orphans/manager.rb +3 -3
- data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +4 -5
- data/lib/sidekiq_unique_jobs/reflections.rb +3 -3
- data/lib/sidekiq_unique_jobs/sidekiq_unique_ext.rb +2 -2
- data/lib/sidekiq_unique_jobs/sidekiq_unique_jobs.rb +4 -4
- data/lib/sidekiq_unique_jobs/sidekiq_worker_methods.rb +1 -1
- data/lib/sidekiq_unique_jobs/testing.rb +2 -2
- data/lib/sidekiq_unique_jobs/version.rb +1 -1
- data/lib/sidekiq_unique_jobs/web/helpers.rb +29 -1
- data/lib/sidekiq_unique_jobs/web.rb +38 -30
- metadata +5 -8
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 752654a036faee8511674b89a3a50ac370f497c29499e3f1fd4dd466b3c5d344
|
4
|
+
data.tar.gz: 40fbc25bd5639f248d6b34817e820d6b7f096a559ed3e0059d28570137b86274
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: cfe83dfbe84eaac52440e1178acf19abfe08bcf4e65c7dc8c9013c315f5e19c57a68c104ded980997c8b5f88400e90175a60858eaede00295327917062a26035
|
7
|
+
data.tar.gz: 128852982336a1d363eebf9f597fb30209369e4ce4376bfefe630fdd56c2bd426be724921b4d6493ccb1406ad65ba1fae37c8085efbdb4fb37e2c2cdd660f9a1
|
data/CHANGELOG.md
CHANGED
@@ -1,12 +1,33 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [Unreleased](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/HEAD)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.10...HEAD)
|
6
|
+
|
7
|
+
**Merged pull requests:**
|
8
|
+
|
9
|
+
- tweak changelog for 8.0.9 [\#836](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/836) ([Earlopain](https://github.com/Earlopain))
|
10
|
+
- Add digest scores for faster deletes in sorted sets [\#835](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/835) ([ezekg](https://github.com/ezekg))
|
11
|
+
|
12
|
+
## [v8.0.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.10) (2024-02-22)
|
13
|
+
|
14
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.33...v8.0.10)
|
15
|
+
|
16
|
+
**Closed issues:**
|
17
|
+
|
18
|
+
- until\_and\_while\_executing and lock\_ttl: jobs silently dropped [\#788](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/788)
|
19
|
+
- Slow evalsha causing timeouts [\#668](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/668)
|
20
|
+
|
21
|
+
## [v7.1.33](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.33) (2024-02-12)
|
22
|
+
|
23
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.9...v7.1.33)
|
24
|
+
|
3
25
|
## [v8.0.9](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.9) (2024-02-12)
|
4
26
|
|
5
27
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.8...v8.0.9)
|
6
28
|
|
7
29
|
**Fixed bugs:**
|
8
30
|
|
9
|
-
- note: The RCE vulnerability was a false alarm, `sidekiq-unique-jobs` was not vulnerable to RCE. You can find additional information in the PR linked below.
|
10
31
|
- fix\(rce\): prevent remot code execution [\#833](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/833) ([mhenrixon](https://github.com/mhenrixon))
|
11
32
|
|
12
33
|
## [v8.0.8](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.8) (2024-02-12)
|
@@ -50,7 +71,7 @@
|
|
50
71
|
|
51
72
|
## [v8.0.6](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.6) (2024-01-24)
|
52
73
|
|
53
|
-
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/
|
74
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.32...v8.0.6)
|
54
75
|
|
55
76
|
**Implemented enhancements:**
|
56
77
|
|
@@ -60,7 +81,6 @@
|
|
60
81
|
**Closed issues:**
|
61
82
|
|
62
83
|
- should respond to `has_valid_sidekiq_options?` [\#822](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/822)
|
63
|
-
- Reaper manager registration is subject to race conditions [\#801](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/801)
|
64
84
|
- `while_executing` with `on_conflict: :reschedule` Reschedule job after job execution [\#800](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/800)
|
65
85
|
- Large retry queue causes reaper to run too slow [\#759](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/759)
|
66
86
|
|
@@ -68,6 +88,14 @@
|
|
68
88
|
|
69
89
|
- fix: skip unless reaper was registered [\#820](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/820) ([mhenrixon](https://github.com/mhenrixon))
|
70
90
|
|
91
|
+
## [v7.1.32](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.32) (2023-11-11)
|
92
|
+
|
93
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.5...v7.1.32)
|
94
|
+
|
95
|
+
**Closed issues:**
|
96
|
+
|
97
|
+
- Reaper manager registration is subject to race conditions [\#801](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/801)
|
98
|
+
|
71
99
|
## [v8.0.5](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.5) (2023-11-11)
|
72
100
|
|
73
101
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.4...v8.0.5)
|
data/README.md
CHANGED
@@ -4,7 +4,7 @@
|
|
4
4
|
|
5
5
|
## Support Me
|
6
6
|
|
7
|
-
Want to show me some ❤️ for the hard work I do on this gem? You can use the following PayPal link: [https://paypal.me/
|
7
|
+
Want to show me some ❤️ for the hard work I do on this gem? You can use the following PayPal link: [https://paypal.me/mhenrixon2](https://paypal.me/mhenrixon2). Any amount is welcome and let me tell you it feels good to be appreciated. Even a dollar makes me super excited about all of this.
|
8
8
|
|
9
9
|
<!-- MarkdownTOC -->
|
10
10
|
|
@@ -59,6 +59,7 @@ Want to show me some ❤️ for the hard work I do on this gem? You can use the
|
|
59
59
|
- [sidekiq-global_id](#sidekiq-global_id)
|
60
60
|
- [sidekiq-status](#sidekiq-status)
|
61
61
|
- [Global Configuration](#global-configuration)
|
62
|
+
- [digest_algorithm](#digest_algorithm)
|
62
63
|
- [debug_lua](#debug_lua)
|
63
64
|
- [lock_timeout](#lock_timeout)
|
64
65
|
- [lock_ttl](#lock_ttl)
|
@@ -88,7 +89,7 @@ Want to show me some ❤️ for the hard work I do on this gem? You can use the
|
|
88
89
|
|
89
90
|
## Introduction
|
90
91
|
|
91
|
-
This gem adds unique constraints to
|
92
|
+
This gem adds unique constraints to Sidekiq jobs. The uniqueness is achieved by creating a set of keys in redis based off of `queue`, `class`, `args` (in the Sidekiq job hash).
|
92
93
|
|
93
94
|
By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen [Conflict Strategy](#conflict-strategy) strategy. Unless a conflict strategy is chosen (?)
|
94
95
|
|
@@ -108,7 +109,7 @@ Here are links to some of the old versions
|
|
108
109
|
Add this line to your application's Gemfile:
|
109
110
|
|
110
111
|
```ruby
|
111
|
-
gem
|
112
|
+
gem "sidekiq-unique-jobs"
|
112
113
|
```
|
113
114
|
|
114
115
|
And then execute:
|
@@ -153,7 +154,7 @@ end
|
|
153
154
|
|
154
155
|
### Your first worker
|
155
156
|
|
156
|
-
The lock type most likely to be is `:until_executed`. This type of lock creates a lock from when `UntilExecutedWorker.perform_async` is called until right after `UntilExecutedWorker.new.perform` has been called.
|
157
|
+
The lock type most likely to be used is `:until_executed`. This type of lock creates a lock from when `UntilExecutedWorker.perform_async` is called until right after `UntilExecutedWorker.new.perform` has been called.
|
157
158
|
|
158
159
|
```ruby
|
159
160
|
# frozen_string_literal: true
|
@@ -184,7 +185,7 @@ You can read more about the worker configuration in [Worker Configuration](#work
|
|
184
185
|
- [ActiveJob officially not supported][48]
|
185
186
|
- [redis-namespace officially not supported][49]
|
186
187
|
|
187
|
-
See [Sidekiq requirements][24] for detailed requirements of Sidekiq itself (be sure to check the right sidekiq version).
|
188
|
+
See [Sidekiq requirements][24] for detailed requirements of Sidekiq itself (be sure to check the right `sidekiq` version).
|
188
189
|
|
189
190
|
## Locks
|
190
191
|
|
@@ -347,7 +348,7 @@ Please not that if you try to override a default lock, an `ArgumentError` will b
|
|
347
348
|
|
348
349
|
## Conflict Strategy
|
349
350
|
|
350
|
-
Decides how we handle conflict. We can either `reject` the job to the dead queue or `reschedule` it. Both are useful for jobs that absolutely need to run and have been configured to use the lock `WhileExecuting` that is used only by the
|
351
|
+
Decides how we handle conflict. We can either `reject` the job to the dead queue or `reschedule` it. Both are useful for jobs that absolutely need to run and have been configured to use the lock `WhileExecuting` that is used only by the Sidekiq server process.
|
351
352
|
|
352
353
|
Furthermore, `log` can be be used with the lock `UntilExecuted` and `UntilExpired`. Now we write a log entry saying the job could not be pushed because it is a duplicate of another job with the same arguments.
|
353
354
|
|
@@ -401,7 +402,7 @@ always scheduled in the future. Currently only attempting to retry one time.
|
|
401
402
|
sidekiq_options on_conflict: :reschedule
|
402
403
|
```
|
403
404
|
|
404
|
-
This strategy is intended to be used with `WhileExecuting` and will delay the job to be tried again in 5 seconds. This will mess up the
|
405
|
+
This strategy is intended to be used with `WhileExecuting` and will delay the job to be tried again in 5 seconds (this delay can be configured via `sidekiq_options schedule_in: {seconds}`). This will mess up the Sidekiq stats but will prevent exceptions from being logged and confuse your sysadmins.
|
405
406
|
|
406
407
|
### Custom Strategies
|
407
408
|
|
@@ -440,12 +441,12 @@ Please not that if you try to override a default lock, an `ArgumentError` will b
|
|
440
441
|
|
441
442
|
### 3 Cleanup Dead Locks
|
442
443
|
|
443
|
-
For sidekiq versions < 5.1 a `sidekiq_retries_exhausted` block is required per worker class. This is deprecated in Sidekiq 6.0
|
444
|
+
For `sidekiq` versions < 5.1 a `sidekiq_retries_exhausted` block is required per worker class. This is deprecated in Sidekiq 6.0
|
444
445
|
|
445
446
|
```ruby
|
446
447
|
class MyWorker
|
447
448
|
sidekiq_retries_exhausted do |msg, _ex|
|
448
|
-
digest = msg[
|
449
|
+
digest = msg["lock_digest"]
|
449
450
|
SidekiqUniqueJobs::Digests.new.delete_by_digest(digest) if digest
|
450
451
|
end
|
451
452
|
end
|
@@ -456,7 +457,7 @@ Starting in v5.1, Sidekiq can also fire a global callback when a job dies: In ve
|
|
456
457
|
```ruby
|
457
458
|
Sidekiq.configure_server do |config|
|
458
459
|
config.death_handlers << ->(job, _ex) do
|
459
|
-
digest = job[
|
460
|
+
digest = job["lock_digest"]
|
460
461
|
SidekiqUniqueJobs::Digests.new.delete_by_digest(digest) if digest
|
461
462
|
end
|
462
463
|
end
|
@@ -472,11 +473,11 @@ To use the web extension you need to require it in your routes.
|
|
472
473
|
|
473
474
|
```ruby
|
474
475
|
#app/config/routes.rb
|
475
|
-
require
|
476
|
-
mount Sidekiq::Web, at:
|
476
|
+
require "sidekiq_unique_jobs/web"
|
477
|
+
mount Sidekiq::Web, at: "/sidekiq"
|
477
478
|
```
|
478
479
|
|
479
|
-
There is no need to `require
|
480
|
+
There is no need to `require "sidekiq/web"` since `sidekiq_unique_jobs/web`
|
480
481
|
already does this.
|
481
482
|
|
482
483
|
To filter/search for keys we can use the wildcard `*`. If we have a unique digest `'uniquejobs:9e9b5ce5d423d3ea470977004b50ff84` we can search for it by enter `*ff84` and it should return all digests that end with `ff84`.
|
@@ -490,10 +491,10 @@ To setup reflections for logging or metrics, use the following API:
|
|
490
491
|
```ruby
|
491
492
|
|
492
493
|
def extract_log_from_job(message, job_hash)
|
493
|
-
worker = job_hash[
|
494
|
-
args = job_hash[
|
495
|
-
lock_args = job_hash[
|
496
|
-
queue = job_hash[
|
494
|
+
worker = job_hash["class"]
|
495
|
+
args = job_hash["args"]
|
496
|
+
lock_args = job_hash["lock_args"]
|
497
|
+
queue = job_hash["queue"]
|
497
498
|
{
|
498
499
|
message: message,
|
499
500
|
worker: worker,
|
@@ -505,7 +506,7 @@ end
|
|
505
506
|
|
506
507
|
SidekiqUniqueJobs.reflect do |on|
|
507
508
|
on.lock_failed do |job_hash|
|
508
|
-
message = extract_log_from_job(
|
509
|
+
message = extract_log_from_job("Lock Failed", job_hash)
|
509
510
|
Sidekiq.logger.warn(message)
|
510
511
|
end
|
511
512
|
end
|
@@ -555,7 +556,7 @@ Also mostly useful for reporting purposes. The job was successfully unlocked.
|
|
555
556
|
|
556
557
|
#### unknown_sidekiq_worker
|
557
558
|
|
558
|
-
The reason this happens is that the server couldn't find a valid
|
559
|
+
The reason this happens is that the server couldn't find a valid Sidekiq worker class. Most likely, that worker isn't intended to be processed by this Sidekiq server instance.
|
559
560
|
|
560
561
|
### Show Locks
|
561
562
|
|
@@ -603,7 +604,7 @@ assert_raise(InvalidWorker){ SidekiqUniqueJobs.validate_worker!(BadWorker.get_si
|
|
603
604
|
|
604
605
|
### Uniqueness
|
605
606
|
|
606
|
-
This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I
|
607
|
+
This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I therefore decided to do what Mike is doing for Sidekiq Enterprise. Read the section about unique jobs: [Enterprise unique jobs][](?)
|
607
608
|
|
608
609
|
```ruby
|
609
610
|
SidekiqUniqueJobs.configure do |config|
|
@@ -612,7 +613,7 @@ SidekiqUniqueJobs.configure do |config|
|
|
612
613
|
end
|
613
614
|
```
|
614
615
|
|
615
|
-
If you truly wanted to test the sidekiq client push you could do something like below. Note that it will only work for the jobs that lock when the client pushes the job to redis (UntilExecuted, UntilAndWhileExecuting and UntilExpired).
|
616
|
+
If you truly wanted to test the `sidekiq` client push you could do something like below. Note that it will only work for the jobs that lock when the client pushes the job to redis (UntilExecuted, UntilAndWhileExecuting and UntilExpired).
|
616
617
|
|
617
618
|
```ruby
|
618
619
|
require "sidekiq_unique_jobs/testing"
|
@@ -624,7 +625,7 @@ RSpec.describe Workers::CoolOne do
|
|
624
625
|
|
625
626
|
# ... your tests that don't test uniqueness
|
626
627
|
|
627
|
-
context
|
628
|
+
context "when Sidekiq::Testing.disabled?" do
|
628
629
|
before do
|
629
630
|
Sidekiq::Testing.disable!
|
630
631
|
Sidekiq.redis(&:flushdb)
|
@@ -634,7 +635,7 @@ RSpec.describe Workers::CoolOne do
|
|
634
635
|
Sidekiq.redis(&:flushdb)
|
635
636
|
end
|
636
637
|
|
637
|
-
it
|
638
|
+
it "prevents duplicate jobs from being scheduled" do
|
638
639
|
SidekiqUniqueJobs.use_config(enabled: true) do
|
639
640
|
expect(described_class.perform_in(3600, 1)).not_to eq(nil)
|
640
641
|
expect(described_class.perform_async(1)).to eq(nil)
|
@@ -734,17 +735,29 @@ Configure SidekiqUniqueJobs in an initializer or the sidekiq initializer on appl
|
|
734
735
|
|
735
736
|
```ruby
|
736
737
|
SidekiqUniqueJobs.configure do |config|
|
737
|
-
config.logger = Sidekiq.logger
|
738
|
-
config.logger_enabled
|
739
|
-
config.debug_lua
|
740
|
-
config.lock_info
|
741
|
-
config.lock_ttl
|
742
|
-
config.lock_timeout
|
743
|
-
config.max_history
|
744
|
-
config.reaper
|
745
|
-
config.reaper_count
|
746
|
-
config.reaper_interval
|
747
|
-
config.reaper_timeout
|
738
|
+
config.logger = Sidekiq.logger # default, change at your own discretion
|
739
|
+
config.logger_enabled = true # default, disable for test environments
|
740
|
+
config.debug_lua = false # Turn on when debugging
|
741
|
+
config.lock_info = false # Turn on when debugging
|
742
|
+
config.lock_ttl = 600 # Expire locks after 10 minutes
|
743
|
+
config.lock_timeout = nil # turn off lock timeout
|
744
|
+
config.max_history = 0 # Turn on when debugging
|
745
|
+
config.reaper = :ruby # :ruby, :lua or :none/nil
|
746
|
+
config.reaper_count = 1000 # Stop reaping after this many keys
|
747
|
+
config.reaper_interval = 600 # Reap orphans every 10 minutes
|
748
|
+
config.reaper_timeout = 150 # Timeout reaper after 2.5 minutes
|
749
|
+
config.digest_algorithm = :modern # Timeout reaper after 2.5 minutes
|
750
|
+
end
|
751
|
+
```
|
752
|
+
#### digest_algorithm
|
753
|
+
|
754
|
+
For backwards compatibility this one is set to `:legacy` by the default. If you happen to run into issues with FIPS being enabled on your redis server you might want to set this to `:modern`.
|
755
|
+
|
756
|
+
See: https://github.com/mhenrixon/sidekiq-unique-jobs/issues/848 for explanation
|
757
|
+
|
758
|
+
```ruby
|
759
|
+
SidekiqUniqueJobs.configure do |config|
|
760
|
+
config.digest_algorithm = :modern # Timeout reaper after 2.5 minutes
|
748
761
|
end
|
749
762
|
```
|
750
763
|
|
@@ -762,7 +775,7 @@ Turning on debug_lua will allow the lua scripts to output debug information abou
|
|
762
775
|
SidekiqUniqueJobs.config.lock_timeout #=> 0
|
763
776
|
```
|
764
777
|
|
765
|
-
Set a global lock_timeout to use for all jobs that don't otherwise specify a lock_timeout.
|
778
|
+
Set a global lock_timeout (in seconds) to use for all jobs that don't otherwise specify a lock_timeout.
|
766
779
|
|
767
780
|
Lock timeout decides how long to wait for acquiring the lock. A value of nil means to wait indefinitely for a lock resource to become available.
|
768
781
|
|
@@ -772,7 +785,7 @@ Lock timeout decides how long to wait for acquiring the lock. A value of nil mea
|
|
772
785
|
SidekiqUniqueJobs.config.lock_ttl #=> nil
|
773
786
|
```
|
774
787
|
|
775
|
-
Set a global lock_ttl to use for all jobs that don't otherwise specify a lock_ttl.
|
788
|
+
Set a global lock_ttl (in seconds) to use for all jobs that don't otherwise specify a lock_ttl.
|
776
789
|
|
777
790
|
Lock TTL decides how long to wait at most before considering a lock to be expired and making it possible to reuse that lock.
|
778
791
|
|
@@ -808,7 +821,7 @@ This is a log that can be accessed by a lock to see what happened for that lock.
|
|
808
821
|
SidekiqUniqueJobs.config.reaper #=> :ruby
|
809
822
|
```
|
810
823
|
|
811
|
-
If using the orphans cleanup process it is critical to be aware of the following. The `:ruby` job is much slower but the `:lua` job locks redis while executing. While doing intense processing it is best to avoid locking redis with a lua script.
|
824
|
+
If using the orphans cleanup process it is critical to be aware of the following. The `:ruby` job is much slower but the `:lua` job locks redis while executing. While doing intense processing it is best to avoid locking redis with a lua script. Therefore the batch size (controlled by the `reaper_count` setting) needs to be reduced.
|
812
825
|
|
813
826
|
In my benchmarks deleting 1000 orphaned locks with lua performs around 65% faster than deleting 1000 keys in ruby.
|
814
827
|
|
@@ -913,13 +926,13 @@ This is mainly intended for `Worker.set(queue: :another).perform_async`.
|
|
913
926
|
class Worker
|
914
927
|
include Sidekiq::Worker
|
915
928
|
|
916
|
-
sidekiq_options unique_across_queues: true, queue:
|
929
|
+
sidekiq_options unique_across_queues: true, queue: "default"
|
917
930
|
|
918
931
|
def perform(args); end
|
919
932
|
end
|
920
933
|
```
|
921
934
|
|
922
|
-
Now if you push override the queue with `Worker.set(queue:
|
935
|
+
Now if you push override the queue with `Worker.set(queue: "another").perform_async(1)` it will still be considered unique when compared to `Worker.perform_async(1)` (that was actually pushed to the queue `default`).
|
923
936
|
|
924
937
|
#### unique_across_workers
|
925
938
|
|
@@ -929,7 +942,7 @@ This configuration option is slightly misleading. It doesn't disregard the worke
|
|
929
942
|
class WorkerOne
|
930
943
|
include Sidekiq::Worker
|
931
944
|
|
932
|
-
sidekiq_options unique_across_workers: true, queue:
|
945
|
+
sidekiq_options unique_across_workers: true, queue: "default"
|
933
946
|
|
934
947
|
def perform(args); end
|
935
948
|
end
|
@@ -937,14 +950,14 @@ end
|
|
937
950
|
class WorkerTwo
|
938
951
|
include Sidekiq::Worker
|
939
952
|
|
940
|
-
sidekiq_options unique_across_workers: true, queue:
|
953
|
+
sidekiq_options unique_across_workers: true, queue: "default"
|
941
954
|
|
942
955
|
def perform(args); end
|
943
956
|
end
|
944
957
|
|
945
958
|
|
946
959
|
WorkerOne.perform_async(1)
|
947
|
-
# =>
|
960
|
+
# => "the jobs unique id"
|
948
961
|
|
949
962
|
WorkerTwo.perform_async(1)
|
950
963
|
# => nil because WorkerOne just stole the lock
|
@@ -1036,7 +1049,7 @@ There is a [
|
1039
|
-
1. Commit your changes (`git commit -am
|
1052
|
+
1. Commit your changes (`git commit -am "Add some feature"`)
|
1040
1053
|
1. Push to the branch (`git push origin my-new-feature`)
|
1041
1054
|
1. Create new Pull Request
|
1042
1055
|
|
@@ -12,7 +12,7 @@ module SidekiqUniqueJobs
|
|
12
12
|
# :nodoc:
|
13
13
|
# rubocop:disable Style/OptionalBooleanParameter
|
14
14
|
def self.banner(command, _namespace = nil, _subcommand = false) # rubocop:disable Style/OptionalBooleanParameter
|
15
|
-
"jobs #{@package_name} #{command.usage}" # rubocop:disable ThreadSafety/
|
15
|
+
"jobs #{@package_name} #{command.usage}" # rubocop:disable ThreadSafety/ClassInstanceVariable
|
16
16
|
end
|
17
17
|
# rubocop:enable Style/OptionalBooleanParameter
|
18
18
|
|
@@ -51,7 +51,7 @@ module SidekiqUniqueJobs
|
|
51
51
|
console_class.start
|
52
52
|
end
|
53
53
|
|
54
|
-
no_commands do
|
54
|
+
no_commands do
|
55
55
|
# :nodoc:
|
56
56
|
def digests
|
57
57
|
@digests ||= SidekiqUniqueJobs::Digests.new
|
@@ -2,32 +2,34 @@
|
|
2
2
|
|
3
3
|
module SidekiqUniqueJobs
|
4
4
|
# ThreadSafe config exists to be able to document the config class without errors
|
5
|
-
ThreadSafeConfig = Concurrent::MutableStruct.new(
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
5
|
+
ThreadSafeConfig = Concurrent::MutableStruct.new(
|
6
|
+
"ThreadSafeConfig",
|
7
|
+
:lock_timeout,
|
8
|
+
:lock_ttl,
|
9
|
+
:enabled,
|
10
|
+
:lock_prefix,
|
11
|
+
:logger,
|
12
|
+
:logger_enabled,
|
13
|
+
:locks,
|
14
|
+
:strategies,
|
15
|
+
:debug_lua,
|
16
|
+
:max_history,
|
17
|
+
:reaper,
|
18
|
+
:reaper_count,
|
19
|
+
:reaper_interval,
|
20
|
+
:reaper_timeout,
|
21
|
+
:reaper_resurrector_interval,
|
22
|
+
:reaper_resurrector_enabled,
|
23
|
+
:lock_info,
|
24
|
+
:raise_on_config_error,
|
25
|
+
:current_redis_version,
|
26
|
+
:digest_algorithm,
|
27
|
+
)
|
25
28
|
|
26
29
|
#
|
27
30
|
# Shared class for dealing with gem configuration
|
28
31
|
#
|
29
32
|
# @author Mauro Berlanda <mauro.berlanda@gmail.com>
|
30
|
-
# rubocop:disable Metrics/ClassLength
|
31
33
|
class Config < ThreadSafeConfig
|
32
34
|
#
|
33
35
|
# @return [Hash<Symbol, SidekiqUniqueJobs::Lock::BaseLock] all available queued locks
|
@@ -118,11 +120,9 @@ module SidekiqUniqueJobs
|
|
118
120
|
#
|
119
121
|
# @return [3600] check if reaper is dead each 3600 seconds
|
120
122
|
REAPER_RESURRECTOR_INTERVAL = 3600
|
121
|
-
|
122
123
|
#
|
123
124
|
# @return [false] enable reaper resurrector
|
124
125
|
REAPER_RESURRECTOR_ENABLED = false
|
125
|
-
|
126
126
|
#
|
127
127
|
# @return [false] while useful it also adds overhead so disable lock_info by default
|
128
128
|
USE_LOCK_INFO = false
|
@@ -132,6 +132,9 @@ module SidekiqUniqueJobs
|
|
132
132
|
#
|
133
133
|
# @return [0.0.0] default redis version is only to avoid NoMethodError on nil
|
134
134
|
REDIS_VERSION = "0.0.0"
|
135
|
+
#
|
136
|
+
# @return [:legacy] default digest algorithm :modern or :legacy
|
137
|
+
DIGEST_ALGORITHM = :legacy
|
135
138
|
|
136
139
|
#
|
137
140
|
# Returns a default configuration
|
@@ -177,7 +180,7 @@ module SidekiqUniqueJobs
|
|
177
180
|
#
|
178
181
|
# @return [SidekiqUniqueJobs::Config] a default configuration
|
179
182
|
#
|
180
|
-
def self.default
|
183
|
+
def self.default
|
181
184
|
new(
|
182
185
|
LOCK_TIMEOUT,
|
183
186
|
LOCK_TTL,
|
@@ -198,6 +201,7 @@ module SidekiqUniqueJobs
|
|
198
201
|
USE_LOCK_INFO,
|
199
202
|
RAISE_ON_CONFIG_ERROR,
|
200
203
|
REDIS_VERSION,
|
204
|
+
DIGEST_ALGORITHM,
|
201
205
|
)
|
202
206
|
end
|
203
207
|
|
@@ -210,8 +214,8 @@ module SidekiqUniqueJobs
|
|
210
214
|
# @return [<type>] <description>
|
211
215
|
#
|
212
216
|
def default_lock_ttl=(obj)
|
213
|
-
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
|
214
|
-
"
|
217
|
+
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. " \
|
218
|
+
"Please use `#{class_name}#lock_ttl=` instead."
|
215
219
|
self.lock_ttl = obj
|
216
220
|
end
|
217
221
|
|
@@ -224,8 +228,8 @@ module SidekiqUniqueJobs
|
|
224
228
|
# @return [Integer]
|
225
229
|
#
|
226
230
|
def default_lock_timeout=(obj)
|
227
|
-
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
|
228
|
-
"
|
231
|
+
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. " \
|
232
|
+
"Please use `#{class_name}#lock_timeout=` instead."
|
229
233
|
self.lock_timeout = obj
|
230
234
|
end
|
231
235
|
|
@@ -236,8 +240,8 @@ module SidekiqUniqueJobs
|
|
236
240
|
# @return [nil, Integer] configured value or nil
|
237
241
|
#
|
238
242
|
def default_lock_ttl
|
239
|
-
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
|
240
|
-
"
|
243
|
+
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. " \
|
244
|
+
"Please use `#{class_name}#lock_ttl` instead."
|
241
245
|
lock_ttl
|
242
246
|
end
|
243
247
|
|
@@ -249,8 +253,8 @@ module SidekiqUniqueJobs
|
|
249
253
|
# @return [nil, Integer] configured value or nil
|
250
254
|
#
|
251
255
|
def default_lock_timeout
|
252
|
-
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
|
253
|
-
"
|
256
|
+
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. " \
|
257
|
+
"Please use `#{class_name}#lock_timeout` instead."
|
254
258
|
lock_timeout
|
255
259
|
end
|
256
260
|
|
@@ -304,6 +308,21 @@ module SidekiqUniqueJobs
|
|
304
308
|
self.strategies = new_strategies
|
305
309
|
end
|
306
310
|
|
311
|
+
#
|
312
|
+
# Sets digest_algorithm to either :modern or :legacy
|
313
|
+
#
|
314
|
+
# @param [Symbol] value
|
315
|
+
#
|
316
|
+
# @return [Symbol] the new value
|
317
|
+
#
|
318
|
+
def digest_algorithm=(value)
|
319
|
+
unless [:modern, :legacy].include?(value)
|
320
|
+
raise ArgumentError, "Invalid digest algorithm: #{value} (should be :modern or :legacy)"
|
321
|
+
end
|
322
|
+
|
323
|
+
super
|
324
|
+
end
|
325
|
+
|
307
326
|
#
|
308
327
|
# The current version of redis
|
309
328
|
#
|
@@ -6,7 +6,7 @@ module SidekiqUniqueJobs
|
|
6
6
|
#
|
7
7
|
# @author Mikael Henriksson <mikael@mhenrixon.com>
|
8
8
|
#
|
9
|
-
class Lock
|
9
|
+
class Lock
|
10
10
|
# includes "SidekiqUniqueJobs::Connection"
|
11
11
|
# @!parse include SidekiqUniqueJobs::Connection
|
12
12
|
include SidekiqUniqueJobs::Connection
|
@@ -91,9 +91,9 @@ module SidekiqUniqueJobs
|
|
91
91
|
job_class.send(lock_args_method, args)
|
92
92
|
rescue ArgumentError
|
93
93
|
raise SidekiqUniqueJobs::InvalidUniqueArguments,
|
94
|
-
|
95
|
-
|
96
|
-
|
94
|
+
given: args,
|
95
|
+
job_class: job_class,
|
96
|
+
lock_args_method: lock_args_method
|
97
97
|
end
|
98
98
|
|
99
99
|
# The method to use for filtering unique arguments
|
@@ -51,7 +51,12 @@ module SidekiqUniqueJobs
|
|
51
51
|
# Creates a namespaced unique digest based on the {#digestable_hash} and the {#lock_prefix}
|
52
52
|
# @return [String] a unique digest
|
53
53
|
def create_digest
|
54
|
-
digest =
|
54
|
+
digest = if SidekiqUniqueJobs.config.digest_algorithm == :legacy
|
55
|
+
OpenSSL::Digest::MD5.hexdigest(dump_json(digestable_hash.sort))
|
56
|
+
else
|
57
|
+
OpenSSL::Digest.new("SHA3-256", dump_json(digestable_hash.sort)).hexdigest
|
58
|
+
end
|
59
|
+
|
55
60
|
"#{lock_prefix}:#{digest}"
|
56
61
|
end
|
57
62
|
|
@@ -66,12 +66,34 @@ module SidekiqUniqueJobs
|
|
66
66
|
# @return [Integer] the number of seconds to live
|
67
67
|
#
|
68
68
|
def calculate
|
69
|
-
ttl =
|
70
|
-
|
71
|
-
|
72
|
-
|
73
|
-
|
74
|
-
|
69
|
+
ttl = fetch_ttl
|
70
|
+
return unless ttl
|
71
|
+
|
72
|
+
timing = calculate_timing(ttl)
|
73
|
+
return unless timing
|
74
|
+
|
75
|
+
timing.to_i + time_until_scheduled
|
76
|
+
end
|
77
|
+
|
78
|
+
private
|
79
|
+
|
80
|
+
def fetch_ttl
|
81
|
+
item[LOCK_TTL] ||
|
82
|
+
job_options[LOCK_TTL] ||
|
83
|
+
item[LOCK_EXPIRATION] || # TODO: Deprecate at some point
|
84
|
+
job_options[LOCK_EXPIRATION] || # TODO: Deprecate at some point
|
85
|
+
SidekiqUniqueJobs.config.lock_ttl
|
86
|
+
end
|
87
|
+
|
88
|
+
def calculate_timing(ttl)
|
89
|
+
case ttl
|
90
|
+
when String, Numeric
|
91
|
+
ttl
|
92
|
+
when Proc
|
93
|
+
ttl.call(item[ARGS])
|
94
|
+
when Symbol
|
95
|
+
job_class.send(ttl, item[ARGS])
|
96
|
+
end
|
75
97
|
end
|
76
98
|
end
|
77
99
|
end
|
@@ -4,7 +4,7 @@ module SidekiqUniqueJobs
|
|
4
4
|
# Lock manager class that handles all the various locks
|
5
5
|
#
|
6
6
|
# @author Mikael Henriksson <mikael@mhenrixon.com>
|
7
|
-
class Locksmith
|
7
|
+
class Locksmith
|
8
8
|
# includes "SidekiqUniqueJobs::Connection"
|
9
9
|
# @!parse include SidekiqUniqueJobs::Connection
|
10
10
|
include SidekiqUniqueJobs::Connection
|
@@ -243,7 +243,7 @@ module SidekiqUniqueJobs
|
|
243
243
|
# @return [nil] when lock was not possible
|
244
244
|
# @return [Object] whatever the block returns when lock was acquired
|
245
245
|
#
|
246
|
-
def primed_async(conn, wait = nil, &block)
|
246
|
+
def primed_async(conn, wait = nil, &block)
|
247
247
|
timeout = (wait || config.timeout).to_i
|
248
248
|
timeout = 1 if timeout.zero?
|
249
249
|
|
@@ -251,10 +251,10 @@ module SidekiqUniqueJobs
|
|
251
251
|
concurrent_timeout = add_drift(timeout)
|
252
252
|
|
253
253
|
reflect(:debug, :timeouts, item,
|
254
|
-
|
255
|
-
|
256
|
-
|
257
|
-
|
254
|
+
timeouts: {
|
255
|
+
brpoplpush_timeout: brpoplpush_timeout,
|
256
|
+
concurrent_timeout: concurrent_timeout,
|
257
|
+
})
|
258
258
|
|
259
259
|
# NOTE: When debugging, change .value to .value!
|
260
260
|
primed_jid = Concurrent::Promises
|
@@ -223,8 +223,8 @@ module SidekiqUniqueJobs
|
|
223
223
|
end
|
224
224
|
|
225
225
|
def fake_logger_context(_context)
|
226
|
-
logger.warn "Don't know how to setup the logging context. Please open a feature request:" \
|
227
|
-
"
|
226
|
+
logger.warn "Don't know how to setup the logging context. Please open a feature request: " \
|
227
|
+
"https://github.com/mhenrixon/sidekiq-unique-jobs/issues/new?template=feature_request.md"
|
228
228
|
|
229
229
|
yield
|
230
230
|
end
|
@@ -28,7 +28,16 @@ module SidekiqUniqueJobs
|
|
28
28
|
# @return [false] when Sidekiq::Deadset#kill does not take multiple arguments
|
29
29
|
#
|
30
30
|
def kill_with_options?
|
31
|
-
Sidekiq::DeadSet.instance_method(:kill).arity
|
31
|
+
kill_arity = Sidekiq::DeadSet.instance_method(:kill).arity
|
32
|
+
# Method#arity returns:
|
33
|
+
# 1. a nonnegative number for methods that take a fixed number of arguments.
|
34
|
+
# 2. A negative number if it takes a variable number of arguments.
|
35
|
+
# Keyword arguments are considered a single argument, and are considered optional unless one of the kwargs is
|
36
|
+
# required.
|
37
|
+
# Therefore, to determine if `Sidekiq::DeadSet#kill` accepts options beyond the single positional payload
|
38
|
+
# argument, we need to check whether the absolute value of the arity is greater than 1.
|
39
|
+
# See: https://apidock.com/ruby/Method/arity
|
40
|
+
kill_arity > 1 || kill_arity < -1
|
32
41
|
end
|
33
42
|
|
34
43
|
#
|
@@ -21,7 +21,7 @@ module SidekiqUniqueJobs
|
|
21
21
|
# @param [Hash] item sidekiq job hash
|
22
22
|
#
|
23
23
|
def initialize(item, redis_pool = nil)
|
24
|
-
super
|
24
|
+
super
|
25
25
|
@queue = item[QUEUE]
|
26
26
|
@lock_digest = item[LOCK_DIGEST]
|
27
27
|
end
|
@@ -54,8 +54,8 @@ module SidekiqUniqueJobs
|
|
54
54
|
#
|
55
55
|
def delete_job_by_digest
|
56
56
|
call_script(:delete_job_by_digest,
|
57
|
-
|
58
|
-
|
57
|
+
keys: ["#{QUEUE}:#{queue}", SCHEDULE, RETRY],
|
58
|
+
argv: [lock_digest])
|
59
59
|
end
|
60
60
|
|
61
61
|
#
|
@@ -33,8 +33,8 @@ module SidekiqUniqueJobs
|
|
33
33
|
|
34
34
|
strategies.fetch(strategy.to_sym) do
|
35
35
|
SidekiqUniqueJobs.logger.warn(
|
36
|
-
"No matching implementation for strategy: #{strategy}, returning OnConflict::NullStrategy." \
|
37
|
-
"
|
36
|
+
"No matching implementation for strategy: #{strategy}, returning OnConflict::NullStrategy. " \
|
37
|
+
"Available strategies are (#{strategies.inspect})",
|
38
38
|
)
|
39
39
|
|
40
40
|
OnConflict::NullStrategy
|
@@ -32,7 +32,7 @@ module SidekiqUniqueJobs
|
|
32
32
|
#
|
33
33
|
# @return [SidekiqUniqueJobs::TimerTask] the task that was started
|
34
34
|
#
|
35
|
-
def start(test_task = nil)
|
35
|
+
def start(test_task = nil)
|
36
36
|
return if disabled?
|
37
37
|
return if registered?
|
38
38
|
|
@@ -73,7 +73,7 @@ module SidekiqUniqueJobs
|
|
73
73
|
# @return [<type>] <description>
|
74
74
|
#
|
75
75
|
def task
|
76
|
-
@task ||= default_task # rubocop:disable ThreadSafety/
|
76
|
+
@task ||= default_task # rubocop:disable ThreadSafety/ClassInstanceVariable
|
77
77
|
end
|
78
78
|
|
79
79
|
#
|
@@ -101,7 +101,7 @@ module SidekiqUniqueJobs
|
|
101
101
|
# @return [void]
|
102
102
|
#
|
103
103
|
def task=(task)
|
104
|
-
@task = task # rubocop:disable ThreadSafety/
|
104
|
+
@task = task # rubocop:disable ThreadSafety/ClassInstanceVariable
|
105
105
|
end
|
106
106
|
|
107
107
|
#
|
@@ -9,7 +9,6 @@ module SidekiqUniqueJobs
|
|
9
9
|
#
|
10
10
|
# @author Mikael Henriksson <mikael@mhenrixon.com>
|
11
11
|
#
|
12
|
-
# rubocop:disable Metrics/ClassLength
|
13
12
|
class RubyReaper < Reaper
|
14
13
|
include SidekiqUniqueJobs::Timing
|
15
14
|
|
@@ -57,7 +56,7 @@ module SidekiqUniqueJobs
|
|
57
56
|
# @param [Redis] conn a connection to redis
|
58
57
|
#
|
59
58
|
def initialize(conn)
|
60
|
-
super
|
59
|
+
super
|
61
60
|
@digests = SidekiqUniqueJobs::Digests.new
|
62
61
|
@scheduled = Redis::SortedSet.new(SCHEDULE)
|
63
62
|
@retried = Redis::SortedSet.new(RETRY)
|
@@ -107,7 +106,7 @@ module SidekiqUniqueJobs
|
|
107
106
|
#
|
108
107
|
# @return [Array<String>] an array of orphaned digests
|
109
108
|
#
|
110
|
-
def orphans
|
109
|
+
def orphans
|
111
110
|
orphans = []
|
112
111
|
page = 0
|
113
112
|
per = reaper_count * 2
|
@@ -197,7 +196,7 @@ module SidekiqUniqueJobs
|
|
197
196
|
end
|
198
197
|
end
|
199
198
|
|
200
|
-
def active?(digest)
|
199
|
+
def active?(digest)
|
201
200
|
Sidekiq.redis do |conn|
|
202
201
|
procs = conn.sscan("processes").to_a
|
203
202
|
return false if procs.empty?
|
@@ -253,7 +252,7 @@ module SidekiqUniqueJobs
|
|
253
252
|
conn.sscan("queues").each(&block)
|
254
253
|
end
|
255
254
|
|
256
|
-
def entries(conn, queue, &block)
|
255
|
+
def entries(conn, queue, &block)
|
257
256
|
queue_key = "queue:#{queue}"
|
258
257
|
initial_size = conn.llen(queue_key)
|
259
258
|
deleted_size = 0
|
@@ -50,15 +50,15 @@ module SidekiqUniqueJobs
|
|
50
50
|
#
|
51
51
|
# @return [void] <description>
|
52
52
|
#
|
53
|
-
def dispatch(reflection, *args)
|
53
|
+
def dispatch(reflection, *args)
|
54
54
|
if (block = @reflections[reflection])
|
55
55
|
block.call(*args)
|
56
56
|
|
57
57
|
if DEPRECATIONS.key?(reflection)
|
58
58
|
replacement, removal_version = DEPRECATIONS[reflection]
|
59
59
|
SidekiqUniqueJobs::Deprecation.warn(
|
60
|
-
"#{reflection} is deprecated and will be removed in version #{removal_version}." \
|
61
|
-
"
|
60
|
+
"#{reflection} is deprecated and will be removed in version #{removal_version}. " \
|
61
|
+
"Use #{replacement} instead.",
|
62
62
|
)
|
63
63
|
end
|
64
64
|
elsif misconfigured?(reflection)
|
@@ -60,7 +60,7 @@ module Sidekiq
|
|
60
60
|
#
|
61
61
|
def delete(score, job_id)
|
62
62
|
entry = find_job(job_id)
|
63
|
-
SidekiqUniqueJobs::Unlockable.delete!(entry.item) if super
|
63
|
+
SidekiqUniqueJobs::Unlockable.delete!(entry.item) if super
|
64
64
|
entry
|
65
65
|
end
|
66
66
|
end
|
@@ -132,7 +132,7 @@ module Sidekiq
|
|
132
132
|
# @param [String] value a sidekiq job hash
|
133
133
|
#
|
134
134
|
def delete_by_value(name, value)
|
135
|
-
SidekiqUniqueJobs::Unlockable.delete!(Sidekiq.load_json(value)) if super
|
135
|
+
SidekiqUniqueJobs::Unlockable.delete!(Sidekiq.load_json(value)) if super
|
136
136
|
end
|
137
137
|
end
|
138
138
|
|
@@ -4,7 +4,7 @@
|
|
4
4
|
# Contains configuration and utility methods that belongs top level
|
5
5
|
#
|
6
6
|
# @author Mikael Henriksson <mikael@mhenrixon.com>
|
7
|
-
module SidekiqUniqueJobs
|
7
|
+
module SidekiqUniqueJobs
|
8
8
|
include SidekiqUniqueJobs::Connection
|
9
9
|
extend SidekiqUniqueJobs::JSON
|
10
10
|
|
@@ -17,7 +17,7 @@ module SidekiqUniqueJobs # rubocop:disable Metrics/ModuleLength
|
|
17
17
|
# @return [SidekiqUniqueJobs::Config] the gem configuration
|
18
18
|
#
|
19
19
|
def config
|
20
|
-
@config ||= reset! # rubocop:disable ThreadSafety/
|
20
|
+
@config ||= reset! # rubocop:disable ThreadSafety/ClassInstanceVariable
|
21
21
|
end
|
22
22
|
|
23
23
|
#
|
@@ -108,7 +108,7 @@ module SidekiqUniqueJobs # rubocop:disable Metrics/ModuleLength
|
|
108
108
|
# @return [SidekiqUniqueJobs::Config] a default gem configuration
|
109
109
|
#
|
110
110
|
def reset!
|
111
|
-
@config = SidekiqUniqueJobs::Config.default # rubocop:disable ThreadSafety/
|
111
|
+
@config = SidekiqUniqueJobs::Config.default # rubocop:disable ThreadSafety/ClassInstanceVariable
|
112
112
|
end
|
113
113
|
|
114
114
|
#
|
@@ -288,7 +288,7 @@ module SidekiqUniqueJobs # rubocop:disable Metrics/ModuleLength
|
|
288
288
|
# @return [Reflections]
|
289
289
|
#
|
290
290
|
def reflections
|
291
|
-
@reflections ||= Reflections.new # rubocop:disable ThreadSafety/
|
291
|
+
@reflections ||= Reflections.new # rubocop:disable ThreadSafety/ClassInstanceVariable
|
292
292
|
end
|
293
293
|
|
294
294
|
#
|
@@ -39,7 +39,7 @@ module SidekiqUniqueJobs
|
|
39
39
|
|
40
40
|
# The hook to call after a successful unlock
|
41
41
|
# @return [Proc]
|
42
|
-
def after_unlock_hook
|
42
|
+
def after_unlock_hook
|
43
43
|
lambda do
|
44
44
|
if @original_job_class.respond_to?(:after_unlock)
|
45
45
|
# instance method in sidekiq v6
|
@@ -21,7 +21,7 @@ module Sidekiq
|
|
21
21
|
#
|
22
22
|
# @param [Hash<Symbol, Object>] tmp_config the temporary config to use
|
23
23
|
#
|
24
|
-
def self.use_options(tmp_config = {})
|
24
|
+
def self.use_options(tmp_config = {})
|
25
25
|
if respond_to?(:default_job_options)
|
26
26
|
default_job_options.clear
|
27
27
|
self.default_job_options = tmp_config
|
@@ -87,7 +87,7 @@ module Sidekiq
|
|
87
87
|
def sidekiq_options(options = {})
|
88
88
|
SidekiqUniqueJobs.validate_worker!(options) if SidekiqUniqueJobs.config.raise_on_config_error
|
89
89
|
|
90
|
-
super
|
90
|
+
super
|
91
91
|
end
|
92
92
|
|
93
93
|
#
|
@@ -116,7 +116,7 @@ module SidekiqUniqueJobs
|
|
116
116
|
#
|
117
117
|
# @return a redirect to the new subpath
|
118
118
|
#
|
119
|
-
def
|
119
|
+
def safe_redirect_to(subpath)
|
120
120
|
if respond_to?(:to)
|
121
121
|
# Sinatra-based web UI
|
122
122
|
redirect to(subpath)
|
@@ -170,6 +170,34 @@ module SidekiqUniqueJobs
|
|
170
170
|
Time.parse(time.to_s)
|
171
171
|
end
|
172
172
|
end
|
173
|
+
|
174
|
+
# Copied from sidekiq for compatibility with older versions
|
175
|
+
|
176
|
+
# stuff after ? or form input
|
177
|
+
# uses String keys, no Symbols!
|
178
|
+
def safe_url_params(key)
|
179
|
+
return url_params(key) if Sidekiq::MAJOR >= 8
|
180
|
+
|
181
|
+
if key.is_a?(Symbol)
|
182
|
+
warn do
|
183
|
+
"URL parameter `#{key}` should be accessed via String, not Symbol (at #{caller(3..3).first})"
|
184
|
+
end
|
185
|
+
end
|
186
|
+
request.params[key.to_s]
|
187
|
+
end
|
188
|
+
|
189
|
+
# variables embedded in path, `/metrics/:name`
|
190
|
+
# uses Symbol keys, no Strings!
|
191
|
+
def safe_route_params(key)
|
192
|
+
return route_params(key) if Sidekiq::MAJOR >= 8
|
193
|
+
|
194
|
+
if key.is_a?(String)
|
195
|
+
warn do
|
196
|
+
"Route parameter `#{key}` should be accessed via Symbol, not String (at #{caller(3..3).first})"
|
197
|
+
end
|
198
|
+
end
|
199
|
+
env["rack.route_params"][key.to_sym]
|
200
|
+
end
|
173
201
|
end
|
174
202
|
end
|
175
203
|
end
|
@@ -7,17 +7,15 @@ module SidekiqUniqueJobs
|
|
7
7
|
# Useful for deleting keys that for whatever reason wasn't deleted
|
8
8
|
# @author Mikael Henriksson <mikael@mhenrixon.com>
|
9
9
|
module Web
|
10
|
-
def self.registered(app)
|
11
|
-
app.helpers
|
12
|
-
include Web::Helpers
|
13
|
-
end
|
10
|
+
def self.registered(app)
|
11
|
+
app.helpers Web::Helpers
|
14
12
|
|
15
13
|
app.get "/changelogs" do
|
16
|
-
@filter = h(
|
14
|
+
@filter = h(safe_url_params("filter") || "*")
|
17
15
|
@filter = "*" if @filter == ""
|
18
|
-
@count = h(
|
19
|
-
@current_cursor = h(
|
20
|
-
@prev_cursor = h(
|
16
|
+
@count = h(safe_url_params("count") || 100).to_i
|
17
|
+
@current_cursor = h(safe_url_params("cursor")).to_i
|
18
|
+
@prev_cursor = h(safe_url_params("prev_cursor")).to_i
|
21
19
|
@total_size, @next_cursor, @changelogs = changelog.page(
|
22
20
|
cursor: @current_cursor,
|
23
21
|
pattern: @filter,
|
@@ -29,15 +27,15 @@ module SidekiqUniqueJobs
|
|
29
27
|
|
30
28
|
app.get "/changelogs/delete_all" do
|
31
29
|
changelog.clear
|
32
|
-
|
30
|
+
safe_redirect_to :changelogs
|
33
31
|
end
|
34
32
|
|
35
33
|
app.get "/locks" do
|
36
|
-
@filter = h(
|
34
|
+
@filter = h(safe_url_params("filter") || "*")
|
37
35
|
@filter = "*" if @filter == ""
|
38
|
-
@count = h(
|
39
|
-
@current_cursor = h(
|
40
|
-
@prev_cursor = h(
|
36
|
+
@count = h(safe_url_params("count") || 100).to_i
|
37
|
+
@current_cursor = h(safe_url_params("cursor")).to_i
|
38
|
+
@prev_cursor = h(safe_url_params("prev_cursor")).to_i
|
41
39
|
|
42
40
|
@total_size, @next_cursor, @locks = digests.page(
|
43
41
|
cursor: @current_cursor,
|
@@ -49,11 +47,11 @@ module SidekiqUniqueJobs
|
|
49
47
|
end
|
50
48
|
|
51
49
|
app.get "/expiring_locks" do
|
52
|
-
@filter = h(
|
50
|
+
@filter = h(safe_url_params("filter") || "*")
|
53
51
|
@filter = "*" if @filter == ""
|
54
|
-
@count = h(
|
55
|
-
@current_cursor = h(
|
56
|
-
@prev_cursor = h(
|
52
|
+
@count = h(safe_url_params("count") || 100).to_i
|
53
|
+
@current_cursor = h(safe_url_params("cursor")).to_i
|
54
|
+
@prev_cursor = h(safe_url_params("prev_cursor")).to_i
|
57
55
|
|
58
56
|
@total_size, @next_cursor, @locks = expiring_digests.page(
|
59
57
|
cursor: @current_cursor,
|
@@ -67,29 +65,29 @@ module SidekiqUniqueJobs
|
|
67
65
|
app.get "/locks/delete_all" do
|
68
66
|
digests.delete_by_pattern("*", count: digests.count)
|
69
67
|
expiring_digests.delete_by_pattern("*", count: digests.count)
|
70
|
-
|
68
|
+
safe_redirect_to :locks
|
71
69
|
end
|
72
70
|
|
73
71
|
app.get "/locks/:digest" do
|
74
|
-
@digest = h(
|
72
|
+
@digest = h(safe_route_params(:digest))
|
75
73
|
@lock = SidekiqUniqueJobs::Lock.new(@digest)
|
76
74
|
|
77
75
|
erb(unique_template(:lock))
|
78
76
|
end
|
79
77
|
|
80
78
|
app.get "/locks/:digest/delete" do
|
81
|
-
digests.delete_by_digest(h(
|
82
|
-
expiring_digests.delete_by_digest(h(
|
83
|
-
|
79
|
+
digests.delete_by_digest(h(safe_route_params(:digest)))
|
80
|
+
expiring_digests.delete_by_digest(h(safe_route_params(:digest)))
|
81
|
+
safe_redirect_to :locks
|
84
82
|
end
|
85
83
|
|
86
84
|
app.get "/locks/:digest/jobs/:job_id/delete" do
|
87
|
-
@digest = h(
|
88
|
-
@job_id = h(
|
85
|
+
@digest = h(safe_route_params(:digest))
|
86
|
+
@job_id = h(safe_route_params(:job_id))
|
89
87
|
@lock = SidekiqUniqueJobs::Lock.new(@digest)
|
90
88
|
@lock.unlock(@job_id)
|
91
89
|
|
92
|
-
|
90
|
+
safe_redirect_to "locks/#{@lock.key}"
|
93
91
|
end
|
94
92
|
end
|
95
93
|
end
|
@@ -99,11 +97,21 @@ begin
|
|
99
97
|
require "delegate" unless defined?(DelegateClass)
|
100
98
|
require "sidekiq/web" unless defined?(Sidekiq::Web)
|
101
99
|
|
102
|
-
Sidekiq::
|
103
|
-
|
104
|
-
|
105
|
-
|
106
|
-
|
100
|
+
if Sidekiq::MAJOR >= 8
|
101
|
+
Sidekiq::Web.configure do |config|
|
102
|
+
config.register_extension(
|
103
|
+
SidekiqUniqueJobs::Web,
|
104
|
+
name: "unique_jobs",
|
105
|
+
tab: ["Locks", "Expiring Locks", "Changelogs"],
|
106
|
+
index: %w[locks/ expiring_locks/ changelogs/],
|
107
|
+
)
|
108
|
+
end
|
109
|
+
else
|
110
|
+
Sidekiq::Web.register(SidekiqUniqueJobs::Web)
|
111
|
+
Sidekiq::Web.tabs["Locks"] = "locks"
|
112
|
+
Sidekiq::Web.tabs["Expiring Locks"] = "expiring_locks"
|
113
|
+
Sidekiq::Web.tabs["Changelogs"] = "changelogs"
|
114
|
+
end
|
107
115
|
rescue NameError, LoadError => ex
|
108
116
|
SidekiqUniqueJobs.logger.error(ex)
|
109
117
|
end
|
metadata
CHANGED
@@ -1,14 +1,13 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: sidekiq-unique-jobs
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 8.0.
|
4
|
+
version: 8.0.11
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mikael Henriksson
|
8
|
-
autorequire:
|
9
8
|
bindir: bin
|
10
9
|
cert_chain: []
|
11
|
-
date:
|
10
|
+
date: 2025-05-25 00:00:00.000000000 Z
|
12
11
|
dependencies:
|
13
12
|
- !ruby/object:Gem::Dependency
|
14
13
|
name: concurrent-ruby
|
@@ -39,7 +38,7 @@ dependencies:
|
|
39
38
|
version: 7.0.0
|
40
39
|
- - "<"
|
41
40
|
- !ruby/object:Gem::Version
|
42
|
-
version:
|
41
|
+
version: 9.0.0
|
43
42
|
type: :runtime
|
44
43
|
prerelease: false
|
45
44
|
version_requirements: !ruby/object:Gem::Requirement
|
@@ -49,7 +48,7 @@ dependencies:
|
|
49
48
|
version: 7.0.0
|
50
49
|
- - "<"
|
51
50
|
- !ruby/object:Gem::Version
|
52
|
-
version:
|
51
|
+
version: 9.0.0
|
53
52
|
- !ruby/object:Gem::Dependency
|
54
53
|
name: thor
|
55
54
|
requirement: !ruby/object:Gem::Requirement
|
@@ -208,7 +207,6 @@ licenses:
|
|
208
207
|
- MIT
|
209
208
|
metadata:
|
210
209
|
rubygems_mfa_required: 'true'
|
211
|
-
post_install_message:
|
212
210
|
rdoc_options: []
|
213
211
|
require_paths:
|
214
212
|
- lib
|
@@ -223,8 +221,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
223
221
|
- !ruby/object:Gem::Version
|
224
222
|
version: '0'
|
225
223
|
requirements: []
|
226
|
-
rubygems_version: 3.
|
227
|
-
signing_key:
|
224
|
+
rubygems_version: 3.6.2
|
228
225
|
specification_version: 4
|
229
226
|
summary: Sidekiq middleware that prevents duplicates jobs
|
230
227
|
test_files: []
|